2024-09-12T03:05:50.891 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-12T03:05:50.915 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-12T03:05:50.995 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901674 branch: squid description: fs/volumes/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/volumes/{overrides test/misc}} email: ceph-qa@ceph.com first_in_suite: false job_id: '7901674' kernel: client: sha1: distro kdb: 1 sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: squid ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug finisher: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 mon warn on pool no app: false osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - online, but wants - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy - OSD full dropping all updates - OSD near full - pausewr flag - failsafe engaged, dropping updates - failsafe disengaged, no longer dropping - is full \(reached quota - POOL_FULL - POOL_BACKFILLFULL sha1: b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc kclient: syntax: v2 selinux: allowlist: - scontext=system_u:system_r:getty_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: squid sha1: 086c737fa75c22d521a3932c59b31c5e1d1c9ef4 owner: scheduled_teuthology@teuthology priority: 100 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 481 sha1: b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc sleep_before_teardown: 0 subset: 57/512 suite: fs suite_branch: squid suite_path: /home/teuthworker/src/git.ceph.com_ceph_086c737fa75c22d521a3932c59b31c5e1d1c9ef4/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: 086c737fa75c22d521a3932c59b31c5e1d1c9ef4 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi027.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBpfvmMTLtMKDnONFCL9LTez8zipHWQdqL6ZFRgraFz4nAuripvlmJnLmyj4GSlvbakoBx7acTgaNLLHU0/qeVI= smithi045.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKWB5vVII7VIeh+UqY3X5wlyuY3D4MdoJx16ZtrOy1JdCcNzRittfueIME+xvpkZd56C61e8kSJFn8u/puLyKoE= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - sequential: - mgrmodules - kclient: null - cephfs_test_runner: fail_on_skip: false modules: - tasks.cephfs.test_volumes.TestMisc teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-11_21:24:17 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-12T03:05:50.995 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_086c737fa75c22d521a3932c59b31c5e1d1c9ef4/qa; will attempt to use it 2024-09-12T03:05:50.996 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_086c737fa75c22d521a3932c59b31c5e1d1c9ef4/qa/tasks 2024-09-12T03:05:50.996 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-12T03:05:50.997 INFO:teuthology.task.internal:Checking packages... 2024-09-12T03:05:51.022 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc' 2024-09-12T03:05:51.022 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-12T03:05:51.022 INFO:teuthology.packaging:ref: None 2024-09-12T03:05:51.022 INFO:teuthology.packaging:tag: None 2024-09-12T03:05:51.022 INFO:teuthology.packaging:branch: squid 2024-09-12T03:05:51.022 INFO:teuthology.packaging:sha1: b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-12T03:05:51.022 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=squid 2024-09-12T03:05:51.238 INFO:teuthology.task.internal:Found packages for ceph version 19.1.1-314.g086c737f 2024-09-12T03:05:51.240 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-12T03:05:51.246 INFO:teuthology.task.internal:no buildpackages task found 2024-09-12T03:05:51.246 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-12T03:05:51.288 INFO:teuthology.task.internal:Saving configuration 2024-09-12T03:05:51.306 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-12T03:05:51.322 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-12T03:05:51.348 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi027.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901674', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-12 02:59:57.902830', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBpfvmMTLtMKDnONFCL9LTez8zipHWQdqL6ZFRgraFz4nAuripvlmJnLmyj4GSlvbakoBx7acTgaNLLHU0/qeVI='} 2024-09-12T03:05:51.368 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi045.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901674', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-12 02:59:57.904342', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKWB5vVII7VIeh+UqY3X5wlyuY3D4MdoJx16ZtrOy1JdCcNzRittfueIME+xvpkZd56C61e8kSJFn8u/puLyKoE='} 2024-09-12T03:05:51.368 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-12T03:05:51.374 INFO:teuthology.task.internal:roles: ubuntu@smithi027.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2024-09-12T03:05:51.375 INFO:teuthology.task.internal:roles: ubuntu@smithi045.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2024-09-12T03:05:51.375 INFO:teuthology.run_tasks:Running task console_log... 2024-09-12T03:05:51.458 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f09f9defbe0>, signals=[15]) 2024-09-12T03:05:51.458 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-12T03:05:51.469 INFO:teuthology.task.internal:Opening connections... 2024-09-12T03:05:51.469 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi027.front.sepia.ceph.com 2024-09-12T03:05:51.471 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-12T03:05:51.558 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi045.front.sepia.ceph.com 2024-09-12T03:05:51.558 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi045.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-12T03:05:51.642 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-12T03:05:51.650 DEBUG:teuthology.orchestra.run.smithi027:> uname -m 2024-09-12T03:05:51.668 INFO:teuthology.orchestra.run.smithi027.stdout:x86_64 2024-09-12T03:05:51.669 DEBUG:teuthology.orchestra.run.smithi027:> cat /etc/os-release 2024-09-12T03:05:51.727 INFO:teuthology.orchestra.run.smithi027.stdout:NAME="CentOS Stream" 2024-09-12T03:05:51.727 INFO:teuthology.orchestra.run.smithi027.stdout:VERSION="9" 2024-09-12T03:05:51.727 INFO:teuthology.orchestra.run.smithi027.stdout:ID="centos" 2024-09-12T03:05:51.727 INFO:teuthology.orchestra.run.smithi027.stdout:ID_LIKE="rhel fedora" 2024-09-12T03:05:51.727 INFO:teuthology.orchestra.run.smithi027.stdout:VERSION_ID="9" 2024-09-12T03:05:51.727 INFO:teuthology.orchestra.run.smithi027.stdout:PLATFORM_ID="platform:el9" 2024-09-12T03:05:51.727 INFO:teuthology.orchestra.run.smithi027.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-12T03:05:51.727 INFO:teuthology.orchestra.run.smithi027.stdout:ANSI_COLOR="0;31" 2024-09-12T03:05:51.728 INFO:teuthology.orchestra.run.smithi027.stdout:LOGO="fedora-logo-icon" 2024-09-12T03:05:51.728 INFO:teuthology.orchestra.run.smithi027.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-12T03:05:51.728 INFO:teuthology.orchestra.run.smithi027.stdout:HOME_URL="https://centos.org/" 2024-09-12T03:05:51.728 INFO:teuthology.orchestra.run.smithi027.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-12T03:05:51.728 INFO:teuthology.orchestra.run.smithi027.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-12T03:05:51.728 INFO:teuthology.orchestra.run.smithi027.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-12T03:05:51.729 INFO:teuthology.lock.ops:Updating smithi027.front.sepia.ceph.com on lock server 2024-09-12T03:05:51.748 DEBUG:teuthology.orchestra.run.smithi045:> uname -m 2024-09-12T03:05:51.766 INFO:teuthology.orchestra.run.smithi045.stdout:x86_64 2024-09-12T03:05:51.766 DEBUG:teuthology.orchestra.run.smithi045:> cat /etc/os-release 2024-09-12T03:05:51.825 INFO:teuthology.orchestra.run.smithi045.stdout:NAME="CentOS Stream" 2024-09-12T03:05:51.825 INFO:teuthology.orchestra.run.smithi045.stdout:VERSION="9" 2024-09-12T03:05:51.825 INFO:teuthology.orchestra.run.smithi045.stdout:ID="centos" 2024-09-12T03:05:51.825 INFO:teuthology.orchestra.run.smithi045.stdout:ID_LIKE="rhel fedora" 2024-09-12T03:05:51.825 INFO:teuthology.orchestra.run.smithi045.stdout:VERSION_ID="9" 2024-09-12T03:05:51.825 INFO:teuthology.orchestra.run.smithi045.stdout:PLATFORM_ID="platform:el9" 2024-09-12T03:05:51.825 INFO:teuthology.orchestra.run.smithi045.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-12T03:05:51.825 INFO:teuthology.orchestra.run.smithi045.stdout:ANSI_COLOR="0;31" 2024-09-12T03:05:51.825 INFO:teuthology.orchestra.run.smithi045.stdout:LOGO="fedora-logo-icon" 2024-09-12T03:05:51.825 INFO:teuthology.orchestra.run.smithi045.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-12T03:05:51.825 INFO:teuthology.orchestra.run.smithi045.stdout:HOME_URL="https://centos.org/" 2024-09-12T03:05:51.825 INFO:teuthology.orchestra.run.smithi045.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-12T03:05:51.826 INFO:teuthology.orchestra.run.smithi045.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-12T03:05:51.826 INFO:teuthology.orchestra.run.smithi045.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-12T03:05:51.826 INFO:teuthology.lock.ops:Updating smithi045.front.sepia.ceph.com on lock server 2024-09-12T03:05:51.848 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-12T03:05:51.858 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-12T03:05:51.889 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-12T03:05:51.889 DEBUG:teuthology.orchestra.run.smithi027:> test '!' -e /home/ubuntu/cephtest 2024-09-12T03:05:51.891 DEBUG:teuthology.orchestra.run.smithi045:> test '!' -e /home/ubuntu/cephtest 2024-09-12T03:05:51.908 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-12T03:05:51.921 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-12T03:05:51.921 DEBUG:teuthology.orchestra.run.smithi027:> test -z $(ls -A /var/lib/ceph) 2024-09-12T03:05:51.952 DEBUG:teuthology.orchestra.run.smithi045:> test -z $(ls -A /var/lib/ceph) 2024-09-12T03:05:51.976 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-12T03:05:52.025 INFO:teuthology.run_tasks:Running task kernel... 2024-09-12T03:05:52.039 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': 1, 'sha1': 'distro'} 2024-09-12T03:05:52.040 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}, 'client.2': {'sha1': 'distro'}, 'client.1': {'sha1': 'distro'}, 'client.3': {'sha1': 'distro'}} 2024-09-12T03:05:52.040 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}, 'client.1': {'sha1': 'distro'}}, timeout 300 2024-09-12T03:05:52.040 DEBUG:teuthology.orchestra.run.smithi027:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-12T03:05:52.041 DEBUG:teuthology.orchestra.run.smithi045:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-12T03:05:52.057 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T03:05:52.058 DEBUG:teuthology.orchestra.run.smithi027:> uname -r 2024-09-12T03:05:52.058 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T03:05:52.059 DEBUG:teuthology.orchestra.run.smithi045:> uname -r 2024-09-12T03:05:52.116 INFO:teuthology.orchestra.run.smithi027.stdout:5.14.0-503.el9.x86_64 2024-09-12T03:05:52.117 INFO:teuthology.task.kernel:Running kernel on smithi027: 5.14.0-503.el9.x86_64 2024-09-12T03:05:52.117 DEBUG:teuthology.orchestra.run.smithi027:> sudo yum install -y kernel 2024-09-12T03:05:52.118 INFO:teuthology.orchestra.run.smithi045.stdout:5.14.0-503.el9.x86_64 2024-09-12T03:05:52.118 INFO:teuthology.task.kernel:Running kernel on smithi045: 5.14.0-503.el9.x86_64 2024-09-12T03:05:52.118 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum install -y kernel 2024-09-12T03:05:55.364 INFO:teuthology.orchestra.run.smithi027.stdout:CentOS Stream 9 - BaseOS 5.3 MB/s | 8.2 MB 00:01 2024-09-12T03:05:55.452 INFO:teuthology.orchestra.run.smithi045.stdout:CentOS Stream 9 - BaseOS 5.1 MB/s | 8.2 MB 00:01 2024-09-12T03:05:57.115 INFO:teuthology.orchestra.run.smithi027.stdout:CentOS Stream 9 - AppStream 34 MB/s | 20 MB 00:00 2024-09-12T03:05:57.286 INFO:teuthology.orchestra.run.smithi045.stdout:CentOS Stream 9 - AppStream 30 MB/s | 20 MB 00:00 2024-09-12T03:06:02.766 INFO:teuthology.orchestra.run.smithi027.stdout:CentOS Stream 9 - CRB 3.8 MB/s | 6.5 MB 00:01 2024-09-12T03:06:02.945 INFO:teuthology.orchestra.run.smithi045.stdout:CentOS Stream 9 - CRB 3.8 MB/s | 6.5 MB 00:01 2024-09-12T03:06:04.404 INFO:teuthology.orchestra.run.smithi045.stdout:CentOS Stream 9 - Extras packages 100 kB/s | 19 kB 00:00 2024-09-12T03:06:04.991 INFO:teuthology.orchestra.run.smithi027.stdout:CentOS Stream 9 - Extras packages 19 kB/s | 19 kB 00:00 2024-09-12T03:06:05.166 INFO:teuthology.orchestra.run.smithi045.stdout:Extra Packages for Enterprise Linux 36 MB/s | 23 MB 00:00 2024-09-12T03:06:06.071 INFO:teuthology.orchestra.run.smithi027.stdout:Extra Packages for Enterprise Linux 24 MB/s | 23 MB 00:00 2024-09-12T03:06:10.864 INFO:teuthology.orchestra.run.smithi045.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-09-12T03:06:11.701 INFO:teuthology.orchestra.run.smithi027.stdout:lab-extras 31 kB/s | 1.7 kB 00:00 2024-09-12T03:06:12.954 INFO:teuthology.orchestra.run.smithi045.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-12T03:06:12.954 INFO:teuthology.orchestra.run.smithi045.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-12T03:06:13.045 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2024-09-12T03:06:13.049 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-09-12T03:06:13.050 INFO:teuthology.orchestra.run.smithi045.stdout: Package Architecture Version Repository Size 2024-09-12T03:06:13.050 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-09-12T03:06:13.050 INFO:teuthology.orchestra.run.smithi045.stdout:Installing: 2024-09-12T03:06:13.050 INFO:teuthology.orchestra.run.smithi045.stdout: kernel x86_64 5.14.0-505.el9 baseos 23 k 2024-09-12T03:06:13.050 INFO:teuthology.orchestra.run.smithi045.stdout:Installing dependencies: 2024-09-12T03:06:13.050 INFO:teuthology.orchestra.run.smithi045.stdout: kernel-core x86_64 5.14.0-505.el9 baseos 16 M 2024-09-12T03:06:13.050 INFO:teuthology.orchestra.run.smithi045.stdout: kernel-modules x86_64 5.14.0-505.el9 baseos 35 M 2024-09-12T03:06:13.050 INFO:teuthology.orchestra.run.smithi045.stdout: kernel-modules-core x86_64 5.14.0-505.el9 baseos 29 M 2024-09-12T03:06:13.050 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-12T03:06:13.050 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction Summary 2024-09-12T03:06:13.050 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-09-12T03:06:13.050 INFO:teuthology.orchestra.run.smithi045.stdout:Install 4 Packages 2024-09-12T03:06:13.050 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-12T03:06:13.051 INFO:teuthology.orchestra.run.smithi045.stdout:Total download size: 79 M 2024-09-12T03:06:13.051 INFO:teuthology.orchestra.run.smithi045.stdout:Installed size: 126 M 2024-09-12T03:06:13.051 INFO:teuthology.orchestra.run.smithi045.stdout:Downloading Packages: 2024-09-12T03:06:13.257 INFO:teuthology.orchestra.run.smithi045.stdout:(1/4): kernel-5.14.0-505.el9.x86_64.rpm 264 kB/s | 23 kB 00:00 2024-09-12T03:06:13.674 INFO:teuthology.orchestra.run.smithi045.stdout:(2/4): kernel-core-5.14.0-505.el9.x86_64.rpm 31 MB/s | 16 MB 00:00 2024-09-12T03:06:13.800 INFO:teuthology.orchestra.run.smithi027.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-12T03:06:13.800 INFO:teuthology.orchestra.run.smithi027.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-12T03:06:13.889 INFO:teuthology.orchestra.run.smithi027.stdout:Dependencies resolved. 2024-09-12T03:06:13.895 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2024-09-12T03:06:13.896 INFO:teuthology.orchestra.run.smithi027.stdout: Package Architecture Version Repository Size 2024-09-12T03:06:13.896 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2024-09-12T03:06:13.896 INFO:teuthology.orchestra.run.smithi027.stdout:Installing: 2024-09-12T03:06:13.896 INFO:teuthology.orchestra.run.smithi027.stdout: kernel x86_64 5.14.0-505.el9 baseos 23 k 2024-09-12T03:06:13.896 INFO:teuthology.orchestra.run.smithi027.stdout:Installing dependencies: 2024-09-12T03:06:13.896 INFO:teuthology.orchestra.run.smithi027.stdout: kernel-core x86_64 5.14.0-505.el9 baseos 16 M 2024-09-12T03:06:13.896 INFO:teuthology.orchestra.run.smithi027.stdout: kernel-modules x86_64 5.14.0-505.el9 baseos 35 M 2024-09-12T03:06:13.896 INFO:teuthology.orchestra.run.smithi027.stdout: kernel-modules-core x86_64 5.14.0-505.el9 baseos 29 M 2024-09-12T03:06:13.896 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:06:13.897 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction Summary 2024-09-12T03:06:13.897 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2024-09-12T03:06:13.897 INFO:teuthology.orchestra.run.smithi027.stdout:Install 4 Packages 2024-09-12T03:06:13.897 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:06:13.898 INFO:teuthology.orchestra.run.smithi027.stdout:Total download size: 79 M 2024-09-12T03:06:13.898 INFO:teuthology.orchestra.run.smithi027.stdout:Installed size: 126 M 2024-09-12T03:06:13.898 INFO:teuthology.orchestra.run.smithi027.stdout:Downloading Packages: 2024-09-12T03:06:14.074 INFO:teuthology.orchestra.run.smithi045.stdout:(3/4): kernel-modules-5.14.0-505.el9.x86_64.rpm 38 MB/s | 35 MB 00:00 2024-09-12T03:06:14.103 INFO:teuthology.orchestra.run.smithi027.stdout:(1/4): kernel-5.14.0-505.el9.x86_64.rpm 123 kB/s | 23 kB 00:00 2024-09-12T03:06:14.416 INFO:teuthology.orchestra.run.smithi045.stdout:(4/4): kernel-modules-core-5.14.0-505.el9.x86_6 25 MB/s | 29 MB 00:01 2024-09-12T03:06:14.416 INFO:teuthology.orchestra.run.smithi045.stdout:-------------------------------------------------------------------------------- 2024-09-12T03:06:14.416 INFO:teuthology.orchestra.run.smithi045.stdout:Total 58 MB/s | 79 MB 00:01 2024-09-12T03:06:15.122 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction check 2024-09-12T03:06:15.518 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction check succeeded. 2024-09-12T03:06:15.518 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction test 2024-09-12T03:06:16.504 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction test succeeded. 2024-09-12T03:06:16.504 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction 2024-09-12T03:06:19.077 INFO:teuthology.orchestra.run.smithi045.stdout: Preparing : 1/1 2024-09-12T03:06:19.303 INFO:teuthology.orchestra.run.smithi027.stdout:(2/4): kernel-core-5.14.0-505.el9.x86_64.rpm 2.9 MB/s | 16 MB 00:05 2024-09-12T03:06:19.753 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : kernel-modules-core-5.14.0-505.el9.x86_64 1/4 2024-09-12T03:06:19.834 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-12T03:06:22.619 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-12T03:06:22.745 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-12T03:06:27.211 INFO:teuthology.orchestra.run.smithi027.stdout:(3/4): kernel-modules-core-5.14.0-505.el9.x86_6 2.2 MB/s | 29 MB 00:13 2024-09-12T03:06:27.875 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-12T03:06:27.963 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : kernel-5.14.0-505.el9.x86_64 4/4 2024-09-12T03:06:30.286 INFO:teuthology.orchestra.run.smithi027.stdout:(4/4): kernel-modules-5.14.0-505.el9.x86_64.rpm 2.1 MB/s | 35 MB 00:16 2024-09-12T03:06:30.286 INFO:teuthology.orchestra.run.smithi027.stdout:-------------------------------------------------------------------------------- 2024-09-12T03:06:30.286 INFO:teuthology.orchestra.run.smithi027.stdout:Total 4.8 MB/s | 79 MB 00:16 2024-09-12T03:06:30.993 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction check 2024-09-12T03:06:31.377 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction check succeeded. 2024-09-12T03:06:31.377 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction test 2024-09-12T03:06:32.389 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction test succeeded. 2024-09-12T03:06:32.389 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction 2024-09-12T03:06:33.138 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-12T03:06:34.986 INFO:teuthology.orchestra.run.smithi027.stdout: Preparing : 1/1 2024-09-12T03:06:35.681 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : kernel-modules-core-5.14.0-505.el9.x86_64 1/4 2024-09-12T03:06:35.764 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-12T03:06:38.684 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-12T03:06:38.807 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-12T03:06:44.028 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-12T03:06:44.065 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : kernel-5.14.0-505.el9.x86_64 4/4 2024-09-12T03:06:49.303 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-12T03:07:12.009 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 4/4 2024-09-12T03:07:12.050 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 4/4 2024-09-12T03:07:12.355 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: kernel-5.14.0-505.el9.x86_64 4/4 2024-09-12T03:07:12.355 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : kernel-5.14.0-505.el9.x86_64 1/4 2024-09-12T03:07:12.355 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-12T03:07:12.355 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-12T03:07:13.203 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-12T03:07:13.203 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-12T03:07:13.203 INFO:teuthology.orchestra.run.smithi045.stdout:Installed: 2024-09-12T03:07:13.203 INFO:teuthology.orchestra.run.smithi045.stdout: kernel-5.14.0-505.el9.x86_64 2024-09-12T03:07:13.203 INFO:teuthology.orchestra.run.smithi045.stdout: kernel-core-5.14.0-505.el9.x86_64 2024-09-12T03:07:13.203 INFO:teuthology.orchestra.run.smithi045.stdout: kernel-modules-5.14.0-505.el9.x86_64 2024-09-12T03:07:13.204 INFO:teuthology.orchestra.run.smithi045.stdout: kernel-modules-core-5.14.0-505.el9.x86_64 2024-09-12T03:07:13.204 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-12T03:07:13.204 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2024-09-12T03:07:13.563 DEBUG:teuthology.orchestra.run.smithi045:> rpm -q kernel | sort -rV | head -n 1 2024-09-12T03:07:13.651 INFO:teuthology.orchestra.run.smithi045.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-12T03:07:13.652 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-12T03:07:13.652 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-505.el9.x86_64 2024-09-12T03:07:13.652 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-12T03:07:13.652 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-12T03:07:13.652 INFO:teuthology.task.kernel:Installing distro kernel on client.1... 2024-09-12T03:07:13.652 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi045.front.sepia.ceph.com, path=None, version=distro) 2024-09-12T03:07:13.653 DEBUG:teuthology.orchestra.run.smithi045:> rpm -q kernel | sort -rV | head -n 1 2024-09-12T03:07:13.712 INFO:teuthology.orchestra.run.smithi045.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-12T03:07:13.712 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-12T03:07:13.712 DEBUG:teuthology.orchestra.run.smithi045:> sudo rpm -qi grub2-tools 2024-09-12T03:07:13.744 INFO:teuthology.orchestra.run.smithi045.stdout:Name : grub2-tools 2024-09-12T03:07:13.744 INFO:teuthology.orchestra.run.smithi045.stdout:Epoch : 1 2024-09-12T03:07:13.744 INFO:teuthology.orchestra.run.smithi045.stdout:Version : 2.06 2024-09-12T03:07:13.744 INFO:teuthology.orchestra.run.smithi045.stdout:Release : 82.el9 2024-09-12T03:07:13.744 INFO:teuthology.orchestra.run.smithi045.stdout:Architecture: x86_64 2024-09-12T03:07:13.744 INFO:teuthology.orchestra.run.smithi045.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-12T03:07:13.744 INFO:teuthology.orchestra.run.smithi045.stdout:Group : Unspecified 2024-09-12T03:07:13.744 INFO:teuthology.orchestra.run.smithi045.stdout:Size : 8274670 2024-09-12T03:07:13.744 INFO:teuthology.orchestra.run.smithi045.stdout:License : GPLv3+ 2024-09-12T03:07:13.744 INFO:teuthology.orchestra.run.smithi045.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-12T03:07:13.745 INFO:teuthology.orchestra.run.smithi045.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-12T03:07:13.745 INFO:teuthology.orchestra.run.smithi045.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-12T03:07:13.745 INFO:teuthology.orchestra.run.smithi045.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-12T03:07:13.745 INFO:teuthology.orchestra.run.smithi045.stdout:Packager : builder@centos.org 2024-09-12T03:07:13.745 INFO:teuthology.orchestra.run.smithi045.stdout:Vendor : CentOS 2024-09-12T03:07:13.745 INFO:teuthology.orchestra.run.smithi045.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-12T03:07:13.745 INFO:teuthology.orchestra.run.smithi045.stdout:Summary : Support tools for GRUB. 2024-09-12T03:07:13.745 INFO:teuthology.orchestra.run.smithi045.stdout:Description : 2024-09-12T03:07:13.745 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-12T03:07:13.745 INFO:teuthology.orchestra.run.smithi045.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-12T03:07:13.745 INFO:teuthology.orchestra.run.smithi045.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-12T03:07:13.745 INFO:teuthology.orchestra.run.smithi045.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-12T03:07:13.745 INFO:teuthology.orchestra.run.smithi045.stdout:hardware devices. 2024-09-12T03:07:13.745 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-12T03:07:13.746 INFO:teuthology.orchestra.run.smithi045.stdout:This subpackage provides tools for support of all platforms. 2024-09-12T03:07:13.746 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-12T03:07:13.747 INFO:teuthology.task.kernel:Updating grub on smithi045 to boot 5.14.0-505.el9.x86_64 2024-09-12T03:07:13.747 DEBUG:teuthology.orchestra.run.smithi045:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-12T03:07:14.342 INFO:teuthology.orchestra.run.smithi045.stderr:Generating grub configuration file ... 2024-09-12T03:07:15.695 INFO:teuthology.orchestra.run.smithi045.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-12T03:07:15.717 INFO:teuthology.orchestra.run.smithi045.stderr:done 2024-09-12T03:07:15.719 DEBUG:teuthology.orchestra.run.smithi045:> mktemp 2024-09-12T03:07:15.734 INFO:teuthology.orchestra.run.smithi045.stdout:/tmp/tmp.NSgExpb8Hl 2024-09-12T03:07:15.735 DEBUG:teuthology.orchestra.run.smithi045:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.NSgExpb8Hl 2024-09-12T03:07:15.809 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod 0666 /tmp/tmp.NSgExpb8Hl 2024-09-12T03:07:15.949 DEBUG:teuthology.orchestra.remote:smithi045:/tmp/tmp.NSgExpb8Hl is 6KB 2024-09-12T03:07:15.998 DEBUG:teuthology.orchestra.run.smithi045:> rm -fr /tmp/tmp.NSgExpb8Hl 2024-09-12T03:07:16.012 DEBUG:teuthology.orchestra.run.smithi045:> sudo /bin/ls /boot/loader/entries || true 2024-09-12T03:07:16.075 INFO:teuthology.orchestra.run.smithi045.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-12T03:07:16.075 INFO:teuthology.orchestra.run.smithi045.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-12T03:07:16.075 INFO:teuthology.orchestra.run.smithi045.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-12T03:07:16.075 INFO:teuthology.orchestra.run.smithi045.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64.conf 2024-09-12T03:07:16.076 DEBUG:teuthology.orchestra.run.smithi045:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64 2024-09-12T03:07:16.229 DEBUG:teuthology.orchestra.run.smithi045:> sudo shutdown -r now 2024-09-12T03:07:29.545 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 4/4 2024-09-12T03:07:29.586 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 4/4 2024-09-12T03:07:29.894 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: kernel-5.14.0-505.el9.x86_64 4/4 2024-09-12T03:07:29.894 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : kernel-5.14.0-505.el9.x86_64 1/4 2024-09-12T03:07:29.894 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-12T03:07:29.894 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-12T03:07:30.699 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-12T03:07:30.699 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:07:30.700 INFO:teuthology.orchestra.run.smithi027.stdout:Installed: 2024-09-12T03:07:30.700 INFO:teuthology.orchestra.run.smithi027.stdout: kernel-5.14.0-505.el9.x86_64 2024-09-12T03:07:30.700 INFO:teuthology.orchestra.run.smithi027.stdout: kernel-core-5.14.0-505.el9.x86_64 2024-09-12T03:07:30.700 INFO:teuthology.orchestra.run.smithi027.stdout: kernel-modules-5.14.0-505.el9.x86_64 2024-09-12T03:07:30.700 INFO:teuthology.orchestra.run.smithi027.stdout: kernel-modules-core-5.14.0-505.el9.x86_64 2024-09-12T03:07:30.700 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:07:30.700 INFO:teuthology.orchestra.run.smithi027.stdout:Complete! 2024-09-12T03:07:31.070 DEBUG:teuthology.orchestra.run.smithi027:> rpm -q kernel | sort -rV | head -n 1 2024-09-12T03:07:31.152 INFO:teuthology.orchestra.run.smithi027.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-12T03:07:31.153 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-12T03:07:31.153 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-505.el9.x86_64 2024-09-12T03:07:31.153 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-12T03:07:31.153 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-12T03:07:31.153 INFO:teuthology.task.kernel:Installing distro kernel on client.0... 2024-09-12T03:07:31.153 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi027.front.sepia.ceph.com, path=None, version=distro) 2024-09-12T03:07:31.153 DEBUG:teuthology.orchestra.run.smithi027:> rpm -q kernel | sort -rV | head -n 1 2024-09-12T03:07:31.213 INFO:teuthology.orchestra.run.smithi027.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-12T03:07:31.214 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-12T03:07:31.214 DEBUG:teuthology.orchestra.run.smithi027:> sudo rpm -qi grub2-tools 2024-09-12T03:07:31.256 INFO:teuthology.orchestra.run.smithi027.stdout:Name : grub2-tools 2024-09-12T03:07:31.256 INFO:teuthology.orchestra.run.smithi027.stdout:Epoch : 1 2024-09-12T03:07:31.256 INFO:teuthology.orchestra.run.smithi027.stdout:Version : 2.06 2024-09-12T03:07:31.256 INFO:teuthology.orchestra.run.smithi027.stdout:Release : 82.el9 2024-09-12T03:07:31.256 INFO:teuthology.orchestra.run.smithi027.stdout:Architecture: x86_64 2024-09-12T03:07:31.256 INFO:teuthology.orchestra.run.smithi027.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-12T03:07:31.256 INFO:teuthology.orchestra.run.smithi027.stdout:Group : Unspecified 2024-09-12T03:07:31.257 INFO:teuthology.orchestra.run.smithi027.stdout:Size : 8274670 2024-09-12T03:07:31.257 INFO:teuthology.orchestra.run.smithi027.stdout:License : GPLv3+ 2024-09-12T03:07:31.257 INFO:teuthology.orchestra.run.smithi027.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-12T03:07:31.257 INFO:teuthology.orchestra.run.smithi027.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-12T03:07:31.257 INFO:teuthology.orchestra.run.smithi027.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-12T03:07:31.257 INFO:teuthology.orchestra.run.smithi027.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-12T03:07:31.257 INFO:teuthology.orchestra.run.smithi027.stdout:Packager : builder@centos.org 2024-09-12T03:07:31.258 INFO:teuthology.orchestra.run.smithi027.stdout:Vendor : CentOS 2024-09-12T03:07:31.258 INFO:teuthology.orchestra.run.smithi027.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-12T03:07:31.258 INFO:teuthology.orchestra.run.smithi027.stdout:Summary : Support tools for GRUB. 2024-09-12T03:07:31.258 INFO:teuthology.orchestra.run.smithi027.stdout:Description : 2024-09-12T03:07:31.258 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:07:31.258 INFO:teuthology.orchestra.run.smithi027.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-12T03:07:31.258 INFO:teuthology.orchestra.run.smithi027.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-12T03:07:31.258 INFO:teuthology.orchestra.run.smithi027.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-12T03:07:31.259 INFO:teuthology.orchestra.run.smithi027.stdout:hardware devices. 2024-09-12T03:07:31.259 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:07:31.259 INFO:teuthology.orchestra.run.smithi027.stdout:This subpackage provides tools for support of all platforms. 2024-09-12T03:07:31.260 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-12T03:07:31.260 INFO:teuthology.task.kernel:Updating grub on smithi027 to boot 5.14.0-505.el9.x86_64 2024-09-12T03:07:31.260 DEBUG:teuthology.orchestra.run.smithi027:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-12T03:07:31.921 INFO:teuthology.orchestra.run.smithi027.stderr:Generating grub configuration file ... 2024-09-12T03:07:33.206 INFO:teuthology.orchestra.run.smithi027.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-12T03:07:33.228 INFO:teuthology.orchestra.run.smithi027.stderr:done 2024-09-12T03:07:33.230 DEBUG:teuthology.orchestra.run.smithi027:> mktemp 2024-09-12T03:07:33.246 INFO:teuthology.orchestra.run.smithi027.stdout:/tmp/tmp.5zSjA4V0LN 2024-09-12T03:07:33.247 DEBUG:teuthology.orchestra.run.smithi027:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.5zSjA4V0LN 2024-09-12T03:07:33.314 DEBUG:teuthology.orchestra.run.smithi027:> sudo chmod 0666 /tmp/tmp.5zSjA4V0LN 2024-09-12T03:07:33.461 DEBUG:teuthology.orchestra.remote:smithi027:/tmp/tmp.5zSjA4V0LN is 6KB 2024-09-12T03:07:33.509 DEBUG:teuthology.orchestra.run.smithi027:> rm -fr /tmp/tmp.5zSjA4V0LN 2024-09-12T03:07:33.524 DEBUG:teuthology.orchestra.run.smithi027:> sudo /bin/ls /boot/loader/entries || true 2024-09-12T03:07:33.594 INFO:teuthology.orchestra.run.smithi027.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-12T03:07:33.594 INFO:teuthology.orchestra.run.smithi027.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-12T03:07:33.594 INFO:teuthology.orchestra.run.smithi027.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-12T03:07:33.595 INFO:teuthology.orchestra.run.smithi027.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64.conf 2024-09-12T03:07:33.596 DEBUG:teuthology.orchestra.run.smithi027:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64 2024-09-12T03:07:33.742 DEBUG:teuthology.orchestra.run.smithi027:> sudo shutdown -r now 2024-09-12T03:07:46.243 INFO:teuthology.task.kernel:Checking client client.1 for new kernel version... 2024-09-12T03:07:46.243 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi045.front.sepia.ceph.com' 2024-09-12T03:07:46.244 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi045.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-12T03:08:03.761 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-09-12T03:08:03.761 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi027.front.sepia.ceph.com' 2024-09-12T03:08:03.762 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-12T03:08:04.801 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.45 2024-09-12T03:08:13.804 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi045.front.sepia.ceph.com' 2024-09-12T03:08:13.805 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi045.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-12T03:08:16.861 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.45 2024-09-12T03:08:22.205 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.27 2024-09-12T03:08:28.866 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi045.front.sepia.ceph.com' 2024-09-12T03:08:28.867 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi045.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-12T03:08:31.208 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi027.front.sepia.ceph.com' 2024-09-12T03:08:31.209 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-12T03:08:34.269 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.27 2024-09-12T03:08:46.281 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi027.front.sepia.ceph.com' 2024-09-12T03:08:46.283 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-12T03:08:49.341 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.27 2024-09-12T03:09:00.924 DEBUG:teuthology.orchestra.run.smithi045:> true 2024-09-12T03:09:01.510 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi045.front.sepia.ceph.com' 2024-09-12T03:09:01.511 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "5.14.0-505.el9.x86_64"... 2024-09-12T03:09:01.511 DEBUG:teuthology.orchestra.run.smithi045:> uname -r 2024-09-12T03:09:01.564 INFO:teuthology.orchestra.run.smithi045.stdout:5.14.0-505.el9.x86_64 2024-09-12T03:09:01.565 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-505.el9.x86_64 vs 5.14.0-505.el9.x86_64 2024-09-12T03:09:01.565 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-12T03:09:01.565 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-12T03:09:02.566 DEBUG:teuthology.parallel:result is None 2024-09-12T03:09:04.341 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi027.front.sepia.ceph.com' 2024-09-12T03:09:04.342 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-12T03:09:07.421 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.27 2024-09-12T03:09:25.422 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi027.front.sepia.ceph.com' 2024-09-12T03:09:25.423 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-12T03:09:57.708 DEBUG:teuthology.orchestra.run.smithi027:> true 2024-09-12T03:09:58.060 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi027.front.sepia.ceph.com' 2024-09-12T03:09:58.060 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "5.14.0-505.el9.x86_64"... 2024-09-12T03:09:58.060 DEBUG:teuthology.orchestra.run.smithi027:> uname -r 2024-09-12T03:09:58.075 INFO:teuthology.orchestra.run.smithi027.stdout:5.14.0-505.el9.x86_64 2024-09-12T03:09:58.076 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-505.el9.x86_64 vs 5.14.0-505.el9.x86_64 2024-09-12T03:09:58.076 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-12T03:09:58.076 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-12T03:09:59.077 DEBUG:teuthology.parallel:result is None 2024-09-12T03:09:59.077 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-12T03:09:59.084 INFO:teuthology.task.internal:Creating test directory... 2024-09-12T03:09:59.085 DEBUG:teuthology.orchestra.run.smithi027:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-12T03:09:59.087 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-12T03:09:59.108 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-12T03:09:59.131 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-12T03:09:59.181 INFO:teuthology.task.internal:Creating archive directory... 2024-09-12T03:09:59.181 DEBUG:teuthology.orchestra.run.smithi027:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-12T03:09:59.184 DEBUG:teuthology.orchestra.run.smithi045:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-12T03:09:59.220 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-12T03:09:59.265 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-12T03:09:59.265 DEBUG:teuthology.orchestra.run.smithi027:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-12T03:09:59.268 DEBUG:teuthology.orchestra.run.smithi045:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-12T03:09:59.333 INFO:teuthology.orchestra.run.smithi027.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-12T03:09:59.367 INFO:teuthology.orchestra.run.smithi027.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-12T03:09:59.378 INFO:teuthology.orchestra.run.smithi045.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-12T03:09:59.391 INFO:teuthology.orchestra.run.smithi045.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-12T03:09:59.393 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-12T03:09:59.401 INFO:teuthology.task.internal:Configuring sudo... 2024-09-12T03:09:59.401 DEBUG:teuthology.orchestra.run.smithi027:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-12T03:09:59.404 DEBUG:teuthology.orchestra.run.smithi045:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-12T03:09:59.434 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-12T03:09:59.454 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-12T03:09:59.454 DEBUG:teuthology.orchestra.run.smithi027:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-12T03:09:59.475 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-12T03:09:59.492 DEBUG:teuthology.orchestra.run.smithi027:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-12T03:09:59.580 DEBUG:teuthology.orchestra.run.smithi027:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-12T03:09:59.623 DEBUG:teuthology.orchestra.run.smithi027:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-12T03:09:59.708 DEBUG:teuthology.orchestra.run.smithi027:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-12T03:09:59.749 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-12T03:09:59.749 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-12T03:09:59.816 DEBUG:teuthology.orchestra.run.smithi045:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-12T03:09:59.863 DEBUG:teuthology.orchestra.run.smithi045:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-12T03:09:59.920 DEBUG:teuthology.orchestra.run.smithi045:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-12T03:10:00.007 DEBUG:teuthology.orchestra.run.smithi045:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-12T03:10:00.052 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-12T03:10:00.052 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-12T03:10:00.122 DEBUG:teuthology.orchestra.run.smithi027:> sudo service rsyslog restart 2024-09-12T03:10:00.125 DEBUG:teuthology.orchestra.run.smithi045:> sudo service rsyslog restart 2024-09-12T03:10:00.169 INFO:teuthology.orchestra.run.smithi027.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-12T03:10:00.218 INFO:teuthology.orchestra.run.smithi045.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-12T03:10:00.685 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-12T03:10:00.695 INFO:teuthology.task.internal:Starting timer... 2024-09-12T03:10:00.695 INFO:teuthology.run_tasks:Running task pcp... 2024-09-12T03:10:00.756 INFO:teuthology.run_tasks:Running task selinux... 2024-09-12T03:10:00.807 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:getty_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-09-12T03:10:00.808 DEBUG:teuthology.orchestra.run.smithi027:> sudo service auditd rotate 2024-09-12T03:10:00.855 INFO:teuthology.orchestra.run.smithi027.stdout:Rotating logs: 2024-09-12T03:10:00.857 DEBUG:teuthology.orchestra.run.smithi045:> sudo service auditd rotate 2024-09-12T03:10:00.921 INFO:teuthology.orchestra.run.smithi045.stdout:Rotating logs: 2024-09-12T03:10:00.923 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-12T03:10:00.923 DEBUG:teuthology.orchestra.run.smithi027:> /usr/sbin/getenforce 2024-09-12T03:10:00.949 INFO:teuthology.orchestra.run.smithi027.stdout:Permissive 2024-09-12T03:10:00.949 DEBUG:teuthology.orchestra.run.smithi045:> /usr/sbin/getenforce 2024-09-12T03:10:00.970 INFO:teuthology.orchestra.run.smithi045.stdout:Permissive 2024-09-12T03:10:00.971 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi027.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi045.front.sepia.ceph.com': 'permissive'} 2024-09-12T03:10:00.971 DEBUG:teuthology.orchestra.run.smithi027:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-12T03:10:01.018 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T03:10:01.019 DEBUG:teuthology.orchestra.run.smithi045:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-12T03:10:01.047 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T03:10:01.048 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-12T03:10:01.048 DEBUG:teuthology.orchestra.run.smithi027:> sudo /usr/sbin/setenforce permissive 2024-09-12T03:10:01.089 DEBUG:teuthology.orchestra.run.smithi045:> sudo /usr/sbin/setenforce permissive 2024-09-12T03:10:01.126 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-12T03:10:01.136 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-12T03:10:01.149 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-12T03:10:01.149 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi027.front.sepia.ceph.com,smithi045.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-12T03:13:51.528 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi027.front.sepia.ceph.com'), Remote(name='ubuntu@smithi045.front.sepia.ceph.com')] 2024-09-12T03:13:51.529 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi027.front.sepia.ceph.com' 2024-09-12T03:13:51.530 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-12T03:13:51.610 DEBUG:teuthology.orchestra.run.smithi027:> true 2024-09-12T03:13:51.693 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi027.front.sepia.ceph.com' 2024-09-12T03:13:51.693 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi045.front.sepia.ceph.com' 2024-09-12T03:13:51.694 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi045.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-12T03:13:51.771 DEBUG:teuthology.orchestra.run.smithi045:> true 2024-09-12T03:13:51.850 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi045.front.sepia.ceph.com' 2024-09-12T03:13:51.851 INFO:teuthology.run_tasks:Running task clock... 2024-09-12T03:13:51.861 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-12T03:13:51.862 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-12T03:13:51.862 DEBUG:teuthology.orchestra.run.smithi027:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-12T03:13:51.865 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-12T03:13:51.865 DEBUG:teuthology.orchestra.run.smithi045:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-12T03:13:51.894 INFO:teuthology.orchestra.run.smithi027.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-12T03:13:51.910 INFO:teuthology.orchestra.run.smithi027.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-12T03:13:51.923 INFO:teuthology.orchestra.run.smithi045.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-12T03:13:51.937 INFO:teuthology.orchestra.run.smithi027.stderr:sudo: ntpd: command not found 2024-09-12T03:13:51.938 INFO:teuthology.orchestra.run.smithi045.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-12T03:13:51.948 INFO:teuthology.orchestra.run.smithi027.stdout:506 Cannot talk to daemon 2024-09-12T03:13:51.964 INFO:teuthology.orchestra.run.smithi027.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-12T03:13:51.965 INFO:teuthology.orchestra.run.smithi045.stderr:sudo: ntpd: command not found 2024-09-12T03:13:51.977 INFO:teuthology.orchestra.run.smithi045.stdout:506 Cannot talk to daemon 2024-09-12T03:13:51.978 INFO:teuthology.orchestra.run.smithi027.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-12T03:13:51.992 INFO:teuthology.orchestra.run.smithi045.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-12T03:13:52.007 INFO:teuthology.orchestra.run.smithi045.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-12T03:13:52.035 INFO:teuthology.orchestra.run.smithi027.stderr:bash: line 1: ntpq: command not found 2024-09-12T03:13:52.039 INFO:teuthology.orchestra.run.smithi027.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-12T03:13:52.039 INFO:teuthology.orchestra.run.smithi027.stdout:=============================================================================== 2024-09-12T03:13:52.039 INFO:teuthology.orchestra.run.smithi027.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T03:13:52.039 INFO:teuthology.orchestra.run.smithi027.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T03:13:52.039 INFO:teuthology.orchestra.run.smithi027.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T03:13:52.039 INFO:teuthology.orchestra.run.smithi027.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T03:13:52.063 INFO:teuthology.orchestra.run.smithi045.stderr:bash: line 1: ntpq: command not found 2024-09-12T03:13:52.066 INFO:teuthology.orchestra.run.smithi045.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-12T03:13:52.067 INFO:teuthology.orchestra.run.smithi045.stdout:=============================================================================== 2024-09-12T03:13:52.067 INFO:teuthology.orchestra.run.smithi045.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T03:13:52.067 INFO:teuthology.orchestra.run.smithi045.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T03:13:52.067 INFO:teuthology.orchestra.run.smithi045.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T03:13:52.067 INFO:teuthology.orchestra.run.smithi045.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T03:13:52.068 INFO:teuthology.run_tasks:Running task install... 2024-09-12T03:13:52.076 DEBUG:teuthology.task.install:project ceph 2024-09-12T03:13:52.076 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc'}} 2024-09-12T03:13:52.076 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc'} 2024-09-12T03:13:52.076 INFO:teuthology.task.install:Using flavor: default 2024-09-12T03:13:52.082 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-12T03:13:52.082 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-09-12T03:13:52.082 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc', 'tag': None, 'wait_for_package': False} 2024-09-12T03:13:52.083 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-12T03:13:52.085 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc', 'tag': None, 'wait_for_package': False} 2024-09-12T03:13:52.085 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-12T03:13:52.235 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/ 2024-09-12T03:13:52.235 INFO:teuthology.task.install.rpm:Package version is 19.1.1-298.gb53b6d3e 2024-09-12T03:13:52.239 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/ 2024-09-12T03:13:52.239 INFO:teuthology.task.install.rpm:Package version is 19.1.1-298.gb53b6d3e 2024-09-12T03:13:52.347 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-12T03:13:52.347 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-12T03:13:52.347 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-12T03:13:52.353 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-12T03:13:52.354 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-12T03:13:52.354 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-12T03:13:52.375 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-12T03:13:52.375 DEBUG:teuthology.orchestra.run.smithi027:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-12T03:13:52.381 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-12T03:13:52.381 DEBUG:teuthology.orchestra.run.smithi045:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-12T03:13:52.442 DEBUG:teuthology.orchestra.run.smithi027:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-12T03:13:52.449 DEBUG:teuthology.orchestra.run.smithi045:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-12T03:13:52.519 DEBUG:teuthology.orchestra.run.smithi027:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-12T03:13:52.527 DEBUG:teuthology.orchestra.run.smithi045:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-12T03:13:52.584 INFO:teuthology.orchestra.run.smithi027.stdout:check_obsoletes = 1 2024-09-12T03:13:52.586 DEBUG:teuthology.orchestra.run.smithi027:> sudo yum clean all 2024-09-12T03:13:52.593 INFO:teuthology.orchestra.run.smithi045.stdout:check_obsoletes = 1 2024-09-12T03:13:52.595 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum clean all 2024-09-12T03:13:52.823 INFO:teuthology.orchestra.run.smithi027.stdout:45 files removed 2024-09-12T03:13:52.832 INFO:teuthology.orchestra.run.smithi045.stdout:45 files removed 2024-09-12T03:13:52.853 DEBUG:teuthology.orchestra.run.smithi027:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-12T03:13:52.859 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-12T03:13:53.539 INFO:teuthology.orchestra.run.smithi045.stdout:ceph packages for x86_64 220 kB/s | 84 kB 00:00 2024-09-12T03:13:53.549 INFO:teuthology.orchestra.run.smithi027.stdout:ceph packages for x86_64 211 kB/s | 84 kB 00:00 2024-09-12T03:13:53.981 INFO:teuthology.orchestra.run.smithi045.stdout:ceph noarch packages 29 kB/s | 12 kB 00:00 2024-09-12T03:13:53.992 INFO:teuthology.orchestra.run.smithi027.stdout:ceph noarch packages 29 kB/s | 12 kB 00:00 2024-09-12T03:13:54.257 INFO:teuthology.orchestra.run.smithi045.stdout:ceph source packages 8.0 kB/s | 1.9 kB 00:00 2024-09-12T03:13:54.292 INFO:teuthology.orchestra.run.smithi027.stdout:ceph source packages 7.3 kB/s | 1.9 kB 00:00 2024-09-12T03:13:54.876 INFO:teuthology.orchestra.run.smithi027.stdout:CentOS Stream 9 - BaseOS 15 MB/s | 8.2 MB 00:00 2024-09-12T03:13:54.932 INFO:teuthology.orchestra.run.smithi045.stdout:CentOS Stream 9 - BaseOS 13 MB/s | 8.2 MB 00:00 2024-09-12T03:13:56.802 INFO:teuthology.orchestra.run.smithi027.stdout:CentOS Stream 9 - AppStream 27 MB/s | 20 MB 00:00 2024-09-12T03:13:58.066 INFO:teuthology.orchestra.run.smithi045.stdout:CentOS Stream 9 - AppStream 10 MB/s | 20 MB 00:01 2024-09-12T03:14:01.703 INFO:teuthology.orchestra.run.smithi027.stdout:CentOS Stream 9 - CRB 7.3 MB/s | 6.5 MB 00:00 2024-09-12T03:14:04.012 INFO:teuthology.orchestra.run.smithi027.stdout:CentOS Stream 9 - Extras packages 18 kB/s | 19 kB 00:01 2024-09-12T03:14:04.542 INFO:teuthology.orchestra.run.smithi045.stdout:CentOS Stream 9 - CRB 2.6 MB/s | 6.5 MB 00:02 2024-09-12T03:14:05.133 INFO:teuthology.orchestra.run.smithi027.stdout:Extra Packages for Enterprise Linux 23 MB/s | 23 MB 00:00 2024-09-12T03:14:06.276 INFO:teuthology.orchestra.run.smithi045.stdout:CentOS Stream 9 - Extras packages 43 kB/s | 19 kB 00:00 2024-09-12T03:14:07.056 INFO:teuthology.orchestra.run.smithi045.stdout:Extra Packages for Enterprise Linux 35 MB/s | 23 MB 00:00 2024-09-12T03:14:10.798 INFO:teuthology.orchestra.run.smithi027.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-09-12T03:14:12.820 INFO:teuthology.orchestra.run.smithi045.stdout:lab-extras 31 kB/s | 1.7 kB 00:00 2024-09-12T03:14:13.185 INFO:teuthology.orchestra.run.smithi027.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-12T03:14:13.185 INFO:teuthology.orchestra.run.smithi027.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-12T03:14:13.199 INFO:teuthology.orchestra.run.smithi027.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-12T03:14:13.200 INFO:teuthology.orchestra.run.smithi027.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-12T03:14:13.201 INFO:teuthology.orchestra.run.smithi027.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-12T03:14:13.202 INFO:teuthology.orchestra.run.smithi027.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-12T03:14:13.204 INFO:teuthology.orchestra.run.smithi027.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-12T03:14:13.206 INFO:teuthology.orchestra.run.smithi027.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-12T03:14:13.207 INFO:teuthology.orchestra.run.smithi027.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-12T03:14:13.275 INFO:teuthology.orchestra.run.smithi027.stdout:Dependencies resolved. 2024-09-12T03:14:13.296 INFO:teuthology.orchestra.run.smithi027.stdout:====================================================================================== 2024-09-12T03:14:13.297 INFO:teuthology.orchestra.run.smithi027.stdout: Package Arch Version Repository Size 2024-09-12T03:14:13.297 INFO:teuthology.orchestra.run.smithi027.stdout:====================================================================================== 2024-09-12T03:14:13.297 INFO:teuthology.orchestra.run.smithi027.stdout:Installing: 2024-09-12T03:14:13.297 INFO:teuthology.orchestra.run.smithi027.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-12T03:14:13.297 INFO:teuthology.orchestra.run.smithi027.stdout: ceph x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 6.5 k 2024-09-12T03:14:13.299 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-base x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 5.4 M 2024-09-12T03:14:13.299 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-fuse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 897 k 2024-09-12T03:14:13.299 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 145 k 2024-09-12T03:14:13.300 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 1.5 M 2024-09-12T03:14:13.300 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-cephadm noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 146 k 2024-09-12T03:14:13.300 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-dashboard noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 3.6 M 2024-09-12T03:14:13.300 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 7.4 M 2024-09-12T03:14:13.300 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-rook noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 49 k 2024-09-12T03:14:13.300 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-radosgw x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 11 M 2024-09-12T03:14:13.300 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-test x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 49 M 2024-09-12T03:14:13.300 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-volume noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 285 k 2024-09-12T03:14:13.300 INFO:teuthology.orchestra.run.smithi027.stdout: cephadm noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 765 k 2024-09-12T03:14:13.300 INFO:teuthology.orchestra.run.smithi027.stdout: cephfs-mirror x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 228 k 2024-09-12T03:14:13.300 INFO:teuthology.orchestra.run.smithi027.stdout: cephfs-shell x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 27 k 2024-09-12T03:14:13.300 INFO:teuthology.orchestra.run.smithi027.stdout: cephfs-top noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 18 k 2024-09-12T03:14:13.301 INFO:teuthology.orchestra.run.smithi027.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-12T03:14:13.301 INFO:teuthology.orchestra.run.smithi027.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-12T03:14:13.301 INFO:teuthology.orchestra.run.smithi027.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-12T03:14:13.301 INFO:teuthology.orchestra.run.smithi027.stdout: libcephfs-devel x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 32 k 2024-09-12T03:14:13.301 INFO:teuthology.orchestra.run.smithi027.stdout: libcephfs2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 752 k 2024-09-12T03:14:13.301 INFO:teuthology.orchestra.run.smithi027.stdout: librados-devel x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 127 k 2024-09-12T03:14:13.301 INFO:teuthology.orchestra.run.smithi027.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-12T03:14:13.301 INFO:teuthology.orchestra.run.smithi027.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-12T03:14:13.301 INFO:teuthology.orchestra.run.smithi027.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-12T03:14:13.301 INFO:teuthology.orchestra.run.smithi027.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-12T03:14:13.301 INFO:teuthology.orchestra.run.smithi027.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-12T03:14:13.301 INFO:teuthology.orchestra.run.smithi027.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-12T03:14:13.302 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cephfs x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 162 k 2024-09-12T03:14:13.302 INFO:teuthology.orchestra.run.smithi027.stdout: python3-rados x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 321 k 2024-09-12T03:14:13.302 INFO:teuthology.orchestra.run.smithi027.stdout: python3-rbd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 301 k 2024-09-12T03:14:13.302 INFO:teuthology.orchestra.run.smithi027.stdout: python3-rgw x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 100 k 2024-09-12T03:14:13.302 INFO:teuthology.orchestra.run.smithi027.stdout: rbd-fuse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 85 k 2024-09-12T03:14:13.302 INFO:teuthology.orchestra.run.smithi027.stdout: rbd-mirror x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.1 M 2024-09-12T03:14:13.302 INFO:teuthology.orchestra.run.smithi027.stdout: rbd-nbd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 171 k 2024-09-12T03:14:13.302 INFO:teuthology.orchestra.run.smithi027.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-12T03:14:13.302 INFO:teuthology.orchestra.run.smithi027.stdout:Upgrading: 2024-09-12T03:14:13.302 INFO:teuthology.orchestra.run.smithi027.stdout: NetworkManager x86_64 1:1.49.90-1.el9 baseos 2.3 M 2024-09-12T03:14:13.302 INFO:teuthology.orchestra.run.smithi027.stdout: NetworkManager-libnm x86_64 1:1.49.90-1.el9 baseos 1.8 M 2024-09-12T03:14:13.302 INFO:teuthology.orchestra.run.smithi027.stdout: NetworkManager-team x86_64 1:1.49.90-1.el9 baseos 41 k 2024-09-12T03:14:13.303 INFO:teuthology.orchestra.run.smithi027.stdout: NetworkManager-tui x86_64 1:1.49.90-1.el9 baseos 249 k 2024-09-12T03:14:13.303 INFO:teuthology.orchestra.run.smithi027.stdout: librados2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.4 M 2024-09-12T03:14:13.303 INFO:teuthology.orchestra.run.smithi027.stdout: librbd1 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.2 M 2024-09-12T03:14:13.303 INFO:teuthology.orchestra.run.smithi027.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-12T03:14:13.303 INFO:teuthology.orchestra.run.smithi027.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-12T03:14:13.303 INFO:teuthology.orchestra.run.smithi027.stdout:Installing dependencies: 2024-09-12T03:14:13.303 INFO:teuthology.orchestra.run.smithi027.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-12T03:14:13.303 INFO:teuthology.orchestra.run.smithi027.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-12T03:14:13.303 INFO:teuthology.orchestra.run.smithi027.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-12T03:14:13.303 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-common x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 21 M 2024-09-12T03:14:13.303 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-grafana-dashboards noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 32 k 2024-09-12T03:14:13.303 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mds x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 2.4 M 2024-09-12T03:14:13.304 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-modules-core noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 249 k 2024-09-12T03:14:13.304 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mon x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 4.7 M 2024-09-12T03:14:13.304 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-osd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 17 M 2024-09-12T03:14:13.304 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-prometheus-alerts noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 16 k 2024-09-12T03:14:13.304 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-selinux x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 25 k 2024-09-12T03:14:13.304 INFO:teuthology.orchestra.run.smithi027.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-12T03:14:13.304 INFO:teuthology.orchestra.run.smithi027.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-12T03:14:13.304 INFO:teuthology.orchestra.run.smithi027.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-12T03:14:13.304 INFO:teuthology.orchestra.run.smithi027.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-12T03:14:13.304 INFO:teuthology.orchestra.run.smithi027.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-12T03:14:13.304 INFO:teuthology.orchestra.run.smithi027.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-12T03:14:13.305 INFO:teuthology.orchestra.run.smithi027.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-12T03:14:13.305 INFO:teuthology.orchestra.run.smithi027.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-12T03:14:13.305 INFO:teuthology.orchestra.run.smithi027.stdout: libcephsqlite x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 163 k 2024-09-12T03:14:13.305 INFO:teuthology.orchestra.run.smithi027.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-12T03:14:13.305 INFO:teuthology.orchestra.run.smithi027.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-12T03:14:13.305 INFO:teuthology.orchestra.run.smithi027.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-12T03:14:13.305 INFO:teuthology.orchestra.run.smithi027.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-12T03:14:13.305 INFO:teuthology.orchestra.run.smithi027.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-12T03:14:13.305 INFO:teuthology.orchestra.run.smithi027.stdout: libradosstriper1 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 506 k 2024-09-12T03:14:13.305 INFO:teuthology.orchestra.run.smithi027.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-12T03:14:13.305 INFO:teuthology.orchestra.run.smithi027.stdout: librgw2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 5.3 M 2024-09-12T03:14:13.305 INFO:teuthology.orchestra.run.smithi027.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-12T03:14:13.306 INFO:teuthology.orchestra.run.smithi027.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-12T03:14:13.306 INFO:teuthology.orchestra.run.smithi027.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-12T03:14:13.306 INFO:teuthology.orchestra.run.smithi027.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-12T03:14:13.306 INFO:teuthology.orchestra.run.smithi027.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-12T03:14:13.306 INFO:teuthology.orchestra.run.smithi027.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-12T03:14:13.306 INFO:teuthology.orchestra.run.smithi027.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-12T03:14:13.306 INFO:teuthology.orchestra.run.smithi027.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-12T03:14:13.306 INFO:teuthology.orchestra.run.smithi027.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-12T03:14:13.306 INFO:teuthology.orchestra.run.smithi027.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-12T03:14:13.306 INFO:teuthology.orchestra.run.smithi027.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-12T03:14:13.306 INFO:teuthology.orchestra.run.smithi027.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-12T03:14:13.306 INFO:teuthology.orchestra.run.smithi027.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-12T03:14:13.306 INFO:teuthology.orchestra.run.smithi027.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-12T03:14:13.307 INFO:teuthology.orchestra.run.smithi027.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-12T03:14:13.307 INFO:teuthology.orchestra.run.smithi027.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-12T03:14:13.307 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-12T03:14:13.307 INFO:teuthology.orchestra.run.smithi027.stdout: python3-ceph-argparse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 45 k 2024-09-12T03:14:13.307 INFO:teuthology.orchestra.run.smithi027.stdout: python3-ceph-common x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 136 k 2024-09-12T03:14:13.307 INFO:teuthology.orchestra.run.smithi027.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-12T03:14:13.307 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-12T03:14:13.307 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-12T03:14:13.307 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-12T03:14:13.307 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-12T03:14:13.307 INFO:teuthology.orchestra.run.smithi027.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-12T03:14:13.307 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-12T03:14:13.307 INFO:teuthology.orchestra.run.smithi027.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-12T03:14:13.307 INFO:teuthology.orchestra.run.smithi027.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-12T03:14:13.308 INFO:teuthology.orchestra.run.smithi027.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-12T03:14:13.308 INFO:teuthology.orchestra.run.smithi027.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-12T03:14:13.308 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-12T03:14:13.308 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-12T03:14:13.308 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-12T03:14:13.308 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-12T03:14:13.308 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-12T03:14:13.308 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-12T03:14:13.308 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-12T03:14:13.308 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-12T03:14:13.308 INFO:teuthology.orchestra.run.smithi027.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-12T03:14:13.308 INFO:teuthology.orchestra.run.smithi027.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-12T03:14:13.308 INFO:teuthology.orchestra.run.smithi027.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-12T03:14:13.309 INFO:teuthology.orchestra.run.smithi027.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-12T03:14:13.309 INFO:teuthology.orchestra.run.smithi027.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-12T03:14:13.309 INFO:teuthology.orchestra.run.smithi027.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-12T03:14:13.309 INFO:teuthology.orchestra.run.smithi027.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-12T03:14:13.309 INFO:teuthology.orchestra.run.smithi027.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-12T03:14:13.309 INFO:teuthology.orchestra.run.smithi027.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-12T03:14:13.309 INFO:teuthology.orchestra.run.smithi027.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-12T03:14:13.309 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-12T03:14:13.309 INFO:teuthology.orchestra.run.smithi027.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-12T03:14:13.309 INFO:teuthology.orchestra.run.smithi027.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-12T03:14:13.309 INFO:teuthology.orchestra.run.smithi027.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-12T03:14:13.309 INFO:teuthology.orchestra.run.smithi027.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-12T03:14:13.310 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-12T03:14:13.310 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-12T03:14:13.310 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-12T03:14:13.310 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-12T03:14:13.310 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-12T03:14:13.310 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-12T03:14:13.310 INFO:teuthology.orchestra.run.smithi027.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-12T03:14:13.310 INFO:teuthology.orchestra.run.smithi027.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-12T03:14:13.310 INFO:teuthology.orchestra.run.smithi027.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-12T03:14:13.310 INFO:teuthology.orchestra.run.smithi027.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-12T03:14:13.310 INFO:teuthology.orchestra.run.smithi027.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-12T03:14:13.310 INFO:teuthology.orchestra.run.smithi027.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-12T03:14:13.310 INFO:teuthology.orchestra.run.smithi027.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-12T03:14:13.311 INFO:teuthology.orchestra.run.smithi027.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-12T03:14:13.311 INFO:teuthology.orchestra.run.smithi027.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-12T03:14:13.311 INFO:teuthology.orchestra.run.smithi027.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-12T03:14:13.311 INFO:teuthology.orchestra.run.smithi027.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-12T03:14:13.311 INFO:teuthology.orchestra.run.smithi027.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-12T03:14:13.311 INFO:teuthology.orchestra.run.smithi027.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-12T03:14:13.311 INFO:teuthology.orchestra.run.smithi027.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-12T03:14:13.311 INFO:teuthology.orchestra.run.smithi027.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-12T03:14:13.311 INFO:teuthology.orchestra.run.smithi027.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-12T03:14:13.311 INFO:teuthology.orchestra.run.smithi027.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-12T03:14:13.311 INFO:teuthology.orchestra.run.smithi027.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-12T03:14:13.312 INFO:teuthology.orchestra.run.smithi027.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-12T03:14:13.312 INFO:teuthology.orchestra.run.smithi027.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-12T03:14:13.312 INFO:teuthology.orchestra.run.smithi027.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-12T03:14:13.312 INFO:teuthology.orchestra.run.smithi027.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-12T03:14:13.312 INFO:teuthology.orchestra.run.smithi027.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-12T03:14:13.312 INFO:teuthology.orchestra.run.smithi027.stdout:Installing weak dependencies: 2024-09-12T03:14:13.312 INFO:teuthology.orchestra.run.smithi027.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-12T03:14:13.312 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:14:13.312 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction Summary 2024-09-12T03:14:13.312 INFO:teuthology.orchestra.run.smithi027.stdout:====================================================================================== 2024-09-12T03:14:13.313 INFO:teuthology.orchestra.run.smithi027.stdout:Install 148 Packages 2024-09-12T03:14:13.313 INFO:teuthology.orchestra.run.smithi027.stdout:Upgrade 8 Packages 2024-09-12T03:14:13.313 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:14:13.313 INFO:teuthology.orchestra.run.smithi027.stdout:Total download size: 226 M 2024-09-12T03:14:13.313 INFO:teuthology.orchestra.run.smithi027.stdout:Downloading Packages: 2024-09-12T03:14:13.832 INFO:teuthology.orchestra.run.smithi027.stdout:(1/156): ceph-19.1.1-298.gb53b6d3e.el9.x86_64.r 47 kB/s | 6.5 kB 00:00 2024-09-12T03:14:14.124 INFO:teuthology.orchestra.run.smithi027.stdout:(2/156): ceph-fuse-19.1.1-298.gb53b6d3e.el9.x86 3.0 MB/s | 897 kB 00:00 2024-09-12T03:14:14.191 INFO:teuthology.orchestra.run.smithi027.stdout:(3/156): ceph-immutable-object-cache-19.1.1-298 2.1 MB/s | 145 kB 00:00 2024-09-12T03:14:14.283 INFO:teuthology.orchestra.run.smithi027.stdout:(4/156): ceph-base-19.1.1-298.gb53b6d3e.el9.x86 9.2 MB/s | 5.4 MB 00:00 2024-09-12T03:14:14.408 INFO:teuthology.orchestra.run.smithi027.stdout:(5/156): ceph-mds-19.1.1-298.gb53b6d3e.el9.x86_ 11 MB/s | 2.4 MB 00:00 2024-09-12T03:14:14.459 INFO:teuthology.orchestra.run.smithi027.stdout:(6/156): ceph-mgr-19.1.1-298.gb53b6d3e.el9.x86_ 8.7 MB/s | 1.5 MB 00:00 2024-09-12T03:14:14.792 INFO:teuthology.orchestra.run.smithi027.stdout:(7/156): ceph-mon-19.1.1-298.gb53b6d3e.el9.x86_ 12 MB/s | 4.7 MB 00:00 2024-09-12T03:14:15.076 INFO:teuthology.orchestra.run.smithi027.stdout:(8/156): ceph-common-19.1.1-298.gb53b6d3e.el9.x 15 MB/s | 21 MB 00:01 2024-09-12T03:14:15.118 INFO:teuthology.orchestra.run.smithi027.stdout:(9/156): ceph-selinux-19.1.1-298.gb53b6d3e.el9. 607 kB/s | 25 kB 00:00 2024-09-12T03:14:15.173 INFO:teuthology.orchestra.run.smithi045.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-12T03:14:15.174 INFO:teuthology.orchestra.run.smithi045.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-12T03:14:15.186 INFO:teuthology.orchestra.run.smithi045.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-12T03:14:15.187 INFO:teuthology.orchestra.run.smithi045.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-12T03:14:15.188 INFO:teuthology.orchestra.run.smithi045.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-12T03:14:15.189 INFO:teuthology.orchestra.run.smithi045.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-12T03:14:15.190 INFO:teuthology.orchestra.run.smithi045.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-12T03:14:15.193 INFO:teuthology.orchestra.run.smithi045.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-12T03:14:15.194 INFO:teuthology.orchestra.run.smithi045.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-12T03:14:15.264 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2024-09-12T03:14:15.280 INFO:teuthology.orchestra.run.smithi045.stdout:====================================================================================== 2024-09-12T03:14:15.280 INFO:teuthology.orchestra.run.smithi045.stdout: Package Arch Version Repository Size 2024-09-12T03:14:15.280 INFO:teuthology.orchestra.run.smithi045.stdout:====================================================================================== 2024-09-12T03:14:15.281 INFO:teuthology.orchestra.run.smithi045.stdout:Installing: 2024-09-12T03:14:15.281 INFO:teuthology.orchestra.run.smithi045.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-12T03:14:15.281 INFO:teuthology.orchestra.run.smithi045.stdout: ceph x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 6.5 k 2024-09-12T03:14:15.281 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-base x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 5.4 M 2024-09-12T03:14:15.281 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-fuse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 897 k 2024-09-12T03:14:15.281 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 145 k 2024-09-12T03:14:15.281 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 1.5 M 2024-09-12T03:14:15.282 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-cephadm noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 146 k 2024-09-12T03:14:15.282 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-dashboard noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 3.6 M 2024-09-12T03:14:15.282 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 7.4 M 2024-09-12T03:14:15.282 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-rook noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 49 k 2024-09-12T03:14:15.282 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-radosgw x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 11 M 2024-09-12T03:14:15.282 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-test x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 49 M 2024-09-12T03:14:15.282 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-volume noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 285 k 2024-09-12T03:14:15.282 INFO:teuthology.orchestra.run.smithi045.stdout: cephadm noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 765 k 2024-09-12T03:14:15.282 INFO:teuthology.orchestra.run.smithi045.stdout: cephfs-mirror x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 228 k 2024-09-12T03:14:15.282 INFO:teuthology.orchestra.run.smithi045.stdout: cephfs-shell x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 27 k 2024-09-12T03:14:15.283 INFO:teuthology.orchestra.run.smithi045.stdout: cephfs-top noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 18 k 2024-09-12T03:14:15.283 INFO:teuthology.orchestra.run.smithi045.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-12T03:14:15.283 INFO:teuthology.orchestra.run.smithi045.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-12T03:14:15.283 INFO:teuthology.orchestra.run.smithi045.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-12T03:14:15.283 INFO:teuthology.orchestra.run.smithi045.stdout: libcephfs-devel x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 32 k 2024-09-12T03:14:15.283 INFO:teuthology.orchestra.run.smithi045.stdout: libcephfs2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 752 k 2024-09-12T03:14:15.283 INFO:teuthology.orchestra.run.smithi045.stdout: librados-devel x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 127 k 2024-09-12T03:14:15.283 INFO:teuthology.orchestra.run.smithi045.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-12T03:14:15.283 INFO:teuthology.orchestra.run.smithi045.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-12T03:14:15.283 INFO:teuthology.orchestra.run.smithi045.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-12T03:14:15.283 INFO:teuthology.orchestra.run.smithi045.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-12T03:14:15.283 INFO:teuthology.orchestra.run.smithi045.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-12T03:14:15.284 INFO:teuthology.orchestra.run.smithi045.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-12T03:14:15.284 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cephfs x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 162 k 2024-09-12T03:14:15.284 INFO:teuthology.orchestra.run.smithi045.stdout: python3-rados x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 321 k 2024-09-12T03:14:15.284 INFO:teuthology.orchestra.run.smithi045.stdout: python3-rbd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 301 k 2024-09-12T03:14:15.284 INFO:teuthology.orchestra.run.smithi045.stdout: python3-rgw x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 100 k 2024-09-12T03:14:15.284 INFO:teuthology.orchestra.run.smithi045.stdout: rbd-fuse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 85 k 2024-09-12T03:14:15.284 INFO:teuthology.orchestra.run.smithi045.stdout: rbd-mirror x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.1 M 2024-09-12T03:14:15.284 INFO:teuthology.orchestra.run.smithi045.stdout: rbd-nbd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 171 k 2024-09-12T03:14:15.284 INFO:teuthology.orchestra.run.smithi045.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-12T03:14:15.284 INFO:teuthology.orchestra.run.smithi045.stdout:Upgrading: 2024-09-12T03:14:15.284 INFO:teuthology.orchestra.run.smithi045.stdout: NetworkManager x86_64 1:1.49.90-1.el9 baseos 2.3 M 2024-09-12T03:14:15.284 INFO:teuthology.orchestra.run.smithi045.stdout: NetworkManager-libnm x86_64 1:1.49.90-1.el9 baseos 1.8 M 2024-09-12T03:14:15.284 INFO:teuthology.orchestra.run.smithi045.stdout: NetworkManager-team x86_64 1:1.49.90-1.el9 baseos 41 k 2024-09-12T03:14:15.285 INFO:teuthology.orchestra.run.smithi045.stdout: NetworkManager-tui x86_64 1:1.49.90-1.el9 baseos 249 k 2024-09-12T03:14:15.285 INFO:teuthology.orchestra.run.smithi045.stdout: librados2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.4 M 2024-09-12T03:14:15.285 INFO:teuthology.orchestra.run.smithi045.stdout: librbd1 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.2 M 2024-09-12T03:14:15.285 INFO:teuthology.orchestra.run.smithi045.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-12T03:14:15.285 INFO:teuthology.orchestra.run.smithi045.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-12T03:14:15.285 INFO:teuthology.orchestra.run.smithi045.stdout:Installing dependencies: 2024-09-12T03:14:15.285 INFO:teuthology.orchestra.run.smithi045.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-12T03:14:15.285 INFO:teuthology.orchestra.run.smithi045.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-12T03:14:15.285 INFO:teuthology.orchestra.run.smithi045.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-12T03:14:15.285 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-common x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 21 M 2024-09-12T03:14:15.285 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-grafana-dashboards noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 32 k 2024-09-12T03:14:15.285 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mds x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 2.4 M 2024-09-12T03:14:15.286 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-modules-core noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 249 k 2024-09-12T03:14:15.286 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mon x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 4.7 M 2024-09-12T03:14:15.286 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-osd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 17 M 2024-09-12T03:14:15.286 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-prometheus-alerts noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 16 k 2024-09-12T03:14:15.286 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-selinux x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 25 k 2024-09-12T03:14:15.286 INFO:teuthology.orchestra.run.smithi045.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-12T03:14:15.287 INFO:teuthology.orchestra.run.smithi045.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-12T03:14:15.287 INFO:teuthology.orchestra.run.smithi045.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-12T03:14:15.287 INFO:teuthology.orchestra.run.smithi045.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-12T03:14:15.287 INFO:teuthology.orchestra.run.smithi045.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-12T03:14:15.287 INFO:teuthology.orchestra.run.smithi045.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-12T03:14:15.287 INFO:teuthology.orchestra.run.smithi045.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-12T03:14:15.287 INFO:teuthology.orchestra.run.smithi045.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-12T03:14:15.287 INFO:teuthology.orchestra.run.smithi045.stdout: libcephsqlite x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 163 k 2024-09-12T03:14:15.287 INFO:teuthology.orchestra.run.smithi045.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-12T03:14:15.287 INFO:teuthology.orchestra.run.smithi045.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-12T03:14:15.288 INFO:teuthology.orchestra.run.smithi045.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-12T03:14:15.288 INFO:teuthology.orchestra.run.smithi045.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-12T03:14:15.288 INFO:teuthology.orchestra.run.smithi045.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-12T03:14:15.288 INFO:teuthology.orchestra.run.smithi045.stdout: libradosstriper1 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 506 k 2024-09-12T03:14:15.288 INFO:teuthology.orchestra.run.smithi045.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-12T03:14:15.288 INFO:teuthology.orchestra.run.smithi045.stdout: librgw2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 5.3 M 2024-09-12T03:14:15.288 INFO:teuthology.orchestra.run.smithi045.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-12T03:14:15.288 INFO:teuthology.orchestra.run.smithi045.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-12T03:14:15.288 INFO:teuthology.orchestra.run.smithi045.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-12T03:14:15.288 INFO:teuthology.orchestra.run.smithi045.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-12T03:14:15.288 INFO:teuthology.orchestra.run.smithi045.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-12T03:14:15.289 INFO:teuthology.orchestra.run.smithi045.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-12T03:14:15.289 INFO:teuthology.orchestra.run.smithi045.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-12T03:14:15.289 INFO:teuthology.orchestra.run.smithi045.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-12T03:14:15.289 INFO:teuthology.orchestra.run.smithi045.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-12T03:14:15.289 INFO:teuthology.orchestra.run.smithi045.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-12T03:14:15.289 INFO:teuthology.orchestra.run.smithi045.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-12T03:14:15.290 INFO:teuthology.orchestra.run.smithi045.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-12T03:14:15.290 INFO:teuthology.orchestra.run.smithi045.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-12T03:14:15.290 INFO:teuthology.orchestra.run.smithi045.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-12T03:14:15.290 INFO:teuthology.orchestra.run.smithi045.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-12T03:14:15.290 INFO:teuthology.orchestra.run.smithi045.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-12T03:14:15.290 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-12T03:14:15.290 INFO:teuthology.orchestra.run.smithi045.stdout: python3-ceph-argparse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 45 k 2024-09-12T03:14:15.290 INFO:teuthology.orchestra.run.smithi045.stdout: python3-ceph-common x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 136 k 2024-09-12T03:14:15.290 INFO:teuthology.orchestra.run.smithi045.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-12T03:14:15.290 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-12T03:14:15.290 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-12T03:14:15.291 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-12T03:14:15.291 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-12T03:14:15.291 INFO:teuthology.orchestra.run.smithi045.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-12T03:14:15.291 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-12T03:14:15.291 INFO:teuthology.orchestra.run.smithi045.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-12T03:14:15.291 INFO:teuthology.orchestra.run.smithi045.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-12T03:14:15.291 INFO:teuthology.orchestra.run.smithi045.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-12T03:14:15.291 INFO:teuthology.orchestra.run.smithi045.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-12T03:14:15.291 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-12T03:14:15.291 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-12T03:14:15.292 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-12T03:14:15.292 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-12T03:14:15.292 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-12T03:14:15.292 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-12T03:14:15.292 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-12T03:14:15.292 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-12T03:14:15.292 INFO:teuthology.orchestra.run.smithi045.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-12T03:14:15.292 INFO:teuthology.orchestra.run.smithi045.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-12T03:14:15.292 INFO:teuthology.orchestra.run.smithi045.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-12T03:14:15.292 INFO:teuthology.orchestra.run.smithi045.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-12T03:14:15.292 INFO:teuthology.orchestra.run.smithi045.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-12T03:14:15.292 INFO:teuthology.orchestra.run.smithi045.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-12T03:14:15.292 INFO:teuthology.orchestra.run.smithi045.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-12T03:14:15.293 INFO:teuthology.orchestra.run.smithi045.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-12T03:14:15.293 INFO:teuthology.orchestra.run.smithi045.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-12T03:14:15.293 INFO:teuthology.orchestra.run.smithi045.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-12T03:14:15.293 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-12T03:14:15.293 INFO:teuthology.orchestra.run.smithi045.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-12T03:14:15.293 INFO:teuthology.orchestra.run.smithi045.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-12T03:14:15.293 INFO:teuthology.orchestra.run.smithi045.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-12T03:14:15.293 INFO:teuthology.orchestra.run.smithi045.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-12T03:14:15.293 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-12T03:14:15.293 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-12T03:14:15.293 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-12T03:14:15.293 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-12T03:14:15.293 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-12T03:14:15.294 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-12T03:14:15.294 INFO:teuthology.orchestra.run.smithi045.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-12T03:14:15.294 INFO:teuthology.orchestra.run.smithi045.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-12T03:14:15.294 INFO:teuthology.orchestra.run.smithi045.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-12T03:14:15.294 INFO:teuthology.orchestra.run.smithi045.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-12T03:14:15.294 INFO:teuthology.orchestra.run.smithi045.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-12T03:14:15.294 INFO:teuthology.orchestra.run.smithi045.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-12T03:14:15.294 INFO:teuthology.orchestra.run.smithi045.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-12T03:14:15.294 INFO:teuthology.orchestra.run.smithi045.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-12T03:14:15.294 INFO:teuthology.orchestra.run.smithi045.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-12T03:14:15.294 INFO:teuthology.orchestra.run.smithi045.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-12T03:14:15.294 INFO:teuthology.orchestra.run.smithi045.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-12T03:14:15.294 INFO:teuthology.orchestra.run.smithi045.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-12T03:14:15.294 INFO:teuthology.orchestra.run.smithi045.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-12T03:14:15.295 INFO:teuthology.orchestra.run.smithi045.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-12T03:14:15.295 INFO:teuthology.orchestra.run.smithi045.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-12T03:14:15.295 INFO:teuthology.orchestra.run.smithi045.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-12T03:14:15.295 INFO:teuthology.orchestra.run.smithi045.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-12T03:14:15.295 INFO:teuthology.orchestra.run.smithi045.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-12T03:14:15.295 INFO:teuthology.orchestra.run.smithi045.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-12T03:14:15.295 INFO:teuthology.orchestra.run.smithi045.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-12T03:14:15.295 INFO:teuthology.orchestra.run.smithi045.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-12T03:14:15.295 INFO:teuthology.orchestra.run.smithi045.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-12T03:14:15.295 INFO:teuthology.orchestra.run.smithi045.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-12T03:14:15.295 INFO:teuthology.orchestra.run.smithi045.stdout:Installing weak dependencies: 2024-09-12T03:14:15.295 INFO:teuthology.orchestra.run.smithi045.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-12T03:14:15.295 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-12T03:14:15.295 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction Summary 2024-09-12T03:14:15.296 INFO:teuthology.orchestra.run.smithi045.stdout:====================================================================================== 2024-09-12T03:14:15.296 INFO:teuthology.orchestra.run.smithi045.stdout:Install 148 Packages 2024-09-12T03:14:15.296 INFO:teuthology.orchestra.run.smithi045.stdout:Upgrade 8 Packages 2024-09-12T03:14:15.296 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-12T03:14:15.296 INFO:teuthology.orchestra.run.smithi045.stdout:Total download size: 226 M 2024-09-12T03:14:15.296 INFO:teuthology.orchestra.run.smithi045.stdout:Downloading Packages: 2024-09-12T03:14:15.493 INFO:teuthology.orchestra.run.smithi027.stdout:(10/156): ceph-osd-19.1.1-298.gb53b6d3e.el9.x86 16 MB/s | 17 MB 00:01 2024-09-12T03:14:15.610 INFO:teuthology.orchestra.run.smithi027.stdout:(11/156): ceph-radosgw-19.1.1-298.gb53b6d3e.el9 13 MB/s | 11 MB 00:00 2024-09-12T03:14:15.635 INFO:teuthology.orchestra.run.smithi027.stdout:(12/156): cephfs-mirror-19.1.1-298.gb53b6d3e.el 1.6 MB/s | 228 kB 00:00 2024-09-12T03:14:15.660 INFO:teuthology.orchestra.run.smithi027.stdout:(13/156): cephfs-shell-19.1.1-298.gb53b6d3e.el9 540 kB/s | 27 kB 00:00 2024-09-12T03:14:15.686 INFO:teuthology.orchestra.run.smithi027.stdout:(14/156): libcephfs-devel-19.1.1-298.gb53b6d3e. 630 kB/s | 32 kB 00:00 2024-09-12T03:14:15.745 INFO:teuthology.orchestra.run.smithi027.stdout:(15/156): libcephfs2-19.1.1-298.gb53b6d3e.el9.x 8.8 MB/s | 752 kB 00:00 2024-09-12T03:14:15.770 INFO:teuthology.orchestra.run.smithi027.stdout:(16/156): libcephsqlite-19.1.1-298.gb53b6d3e.el 1.9 MB/s | 163 kB 00:00 2024-09-12T03:14:15.812 INFO:teuthology.orchestra.run.smithi027.stdout:(17/156): librados-devel-19.1.1-298.gb53b6d3e.e 1.9 MB/s | 127 kB 00:00 2024-09-12T03:14:15.845 INFO:teuthology.orchestra.run.smithi027.stdout:(18/156): libradosstriper1-19.1.1-298.gb53b6d3e 6.6 MB/s | 506 kB 00:00 2024-09-12T03:14:15.920 INFO:teuthology.orchestra.run.smithi027.stdout:(19/156): python3-ceph-argparse-19.1.1-298.gb53 603 kB/s | 45 kB 00:00 2024-09-12T03:14:16.013 INFO:teuthology.orchestra.run.smithi027.stdout:(20/156): python3-ceph-common-19.1.1-298.gb53b6 1.4 MB/s | 136 kB 00:00 2024-09-12T03:14:16.088 INFO:teuthology.orchestra.run.smithi027.stdout:(21/156): python3-cephfs-19.1.1-298.gb53b6d3e.e 2.1 MB/s | 162 kB 00:00 2024-09-12T03:14:16.138 INFO:teuthology.orchestra.run.smithi045.stdout:(1/156): ceph-19.1.1-298.gb53b6d3e.el9.x86_64.r 29 kB/s | 6.5 kB 00:00 2024-09-12T03:14:16.172 INFO:teuthology.orchestra.run.smithi027.stdout:(22/156): librgw2-19.1.1-298.gb53b6d3e.el9.x86_ 15 MB/s | 5.3 MB 00:00 2024-09-12T03:14:16.197 INFO:teuthology.orchestra.run.smithi027.stdout:(23/156): python3-rados-19.1.1-298.gb53b6d3e.el 2.9 MB/s | 321 kB 00:00 2024-09-12T03:14:16.230 INFO:teuthology.orchestra.run.smithi027.stdout:(24/156): python3-rbd-19.1.1-298.gb53b6d3e.el9. 5.0 MB/s | 301 kB 00:00 2024-09-12T03:14:16.256 INFO:teuthology.orchestra.run.smithi027.stdout:(25/156): python3-rgw-19.1.1-298.gb53b6d3e.el9. 1.7 MB/s | 100 kB 00:00 2024-09-12T03:14:16.289 INFO:teuthology.orchestra.run.smithi027.stdout:(26/156): rbd-fuse-19.1.1-298.gb53b6d3e.el9.x86 1.4 MB/s | 85 kB 00:00 2024-09-12T03:14:16.390 INFO:teuthology.orchestra.run.smithi027.stdout:(27/156): rbd-nbd-19.1.1-298.gb53b6d3e.el9.x86_ 1.7 MB/s | 171 kB 00:00 2024-09-12T03:14:16.447 INFO:teuthology.orchestra.run.smithi045.stdout:(2/156): ceph-fuse-19.1.1-298.gb53b6d3e.el9.x86 2.8 MB/s | 897 kB 00:00 2024-09-12T03:14:16.465 INFO:teuthology.orchestra.run.smithi027.stdout:(28/156): ceph-grafana-dashboards-19.1.1-298.gb 426 kB/s | 32 kB 00:00 2024-09-12T03:14:16.522 INFO:teuthology.orchestra.run.smithi045.stdout:(3/156): ceph-immutable-object-cache-19.1.1-298 1.9 MB/s | 145 kB 00:00 2024-09-12T03:14:16.524 INFO:teuthology.orchestra.run.smithi027.stdout:(29/156): rbd-mirror-19.1.1-298.gb53b6d3e.el9.x 12 MB/s | 3.1 MB 00:00 2024-09-12T03:14:16.549 INFO:teuthology.orchestra.run.smithi027.stdout:(30/156): ceph-mgr-cephadm-19.1.1-298.gb53b6d3e 1.7 MB/s | 146 kB 00:00 2024-09-12T03:14:16.622 INFO:teuthology.orchestra.run.smithi045.stdout:(4/156): ceph-base-19.1.1-298.gb53b6d3e.el9.x86 7.6 MB/s | 5.4 MB 00:00 2024-09-12T03:14:16.939 INFO:teuthology.orchestra.run.smithi045.stdout:(5/156): ceph-mgr-19.1.1-298.gb53b6d3e.el9.x86_ 4.8 MB/s | 1.5 MB 00:00 2024-09-12T03:14:17.048 INFO:teuthology.orchestra.run.smithi045.stdout:(6/156): ceph-mds-19.1.1-298.gb53b6d3e.el9.x86_ 4.5 MB/s | 2.4 MB 00:00 2024-09-12T03:14:17.058 INFO:teuthology.orchestra.run.smithi027.stdout:(31/156): ceph-mgr-diskprediction-local-19.1.1- 15 MB/s | 7.4 MB 00:00 2024-09-12T03:14:17.133 INFO:teuthology.orchestra.run.smithi027.stdout:(32/156): ceph-mgr-modules-core-19.1.1-298.gb53 3.3 MB/s | 249 kB 00:00 2024-09-12T03:14:17.225 INFO:teuthology.orchestra.run.smithi027.stdout:(33/156): ceph-mgr-rook-19.1.1-298.gb53b6d3e.el 539 kB/s | 49 kB 00:00 2024-09-12T03:14:17.300 INFO:teuthology.orchestra.run.smithi027.stdout:(34/156): ceph-prometheus-alerts-19.1.1-298.gb5 220 kB/s | 16 kB 00:00 2024-09-12T03:14:17.384 INFO:teuthology.orchestra.run.smithi027.stdout:(35/156): ceph-volume-19.1.1-298.gb53b6d3e.el9. 3.3 MB/s | 285 kB 00:00 2024-09-12T03:14:17.406 INFO:teuthology.orchestra.run.smithi045.stdout:(7/156): ceph-mon-19.1.1-298.gb53b6d3e.el9.x86_ 10 MB/s | 4.7 MB 00:00 2024-09-12T03:14:17.542 INFO:teuthology.orchestra.run.smithi027.stdout:(36/156): ceph-mgr-dashboard-19.1.1-298.gb53b6d 3.5 MB/s | 3.6 MB 00:01 2024-09-12T03:14:17.568 INFO:teuthology.orchestra.run.smithi027.stdout:(37/156): cephadm-19.1.1-298.gb53b6d3e.el9.noar 4.1 MB/s | 765 kB 00:00 2024-09-12T03:14:17.874 INFO:teuthology.orchestra.run.smithi045.stdout:(8/156): ceph-common-19.1.1-298.gb53b6d3e.el9.x 11 MB/s | 21 MB 00:01 2024-09-12T03:14:17.924 INFO:teuthology.orchestra.run.smithi045.stdout:(9/156): ceph-selinux-19.1.1-298.gb53b6d3e.el9. 505 kB/s | 25 kB 00:00 2024-09-12T03:14:18.001 INFO:teuthology.orchestra.run.smithi027.stdout:(38/156): ceph-test-19.1.1-298.gb53b6d3e.el9.x8 17 MB/s | 49 MB 00:02 2024-09-12T03:14:18.027 INFO:teuthology.orchestra.run.smithi027.stdout:(39/156): cephfs-top-19.1.1-298.gb53b6d3e.el9.n 38 kB/s | 18 kB 00:00 2024-09-12T03:14:18.094 INFO:teuthology.orchestra.run.smithi027.stdout:(40/156): fuse-2.9.9-16.el9.x86_64.rpm 152 kB/s | 80 kB 00:00 2024-09-12T03:14:18.124 INFO:teuthology.orchestra.run.smithi045.stdout:(10/156): ceph-radosgw-19.1.1-298.gb53b6d3e.el9 15 MB/s | 11 MB 00:00 2024-09-12T03:14:18.152 INFO:teuthology.orchestra.run.smithi027.stdout:(41/156): libquadmath-11.5.0-2.el9.x86_64.rpm 1.5 MB/s | 191 kB 00:00 2024-09-12T03:14:18.174 INFO:teuthology.orchestra.run.smithi045.stdout:(11/156): cephfs-mirror-19.1.1-298.gb53b6d3e.el 4.5 MB/s | 228 kB 00:00 2024-09-12T03:14:18.186 INFO:teuthology.orchestra.run.smithi027.stdout:(42/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.7 MB/s | 253 kB 00:00 2024-09-12T03:14:18.225 INFO:teuthology.orchestra.run.smithi045.stdout:(12/156): cephfs-shell-19.1.1-298.gb53b6d3e.el9 540 kB/s | 27 kB 00:00 2024-09-12T03:14:18.236 INFO:teuthology.orchestra.run.smithi027.stdout:(43/156): libgfortran-11.5.0-2.el9.x86_64.rpm 3.3 MB/s | 801 kB 00:00 2024-09-12T03:14:18.261 INFO:teuthology.orchestra.run.smithi027.stdout:(44/156): python3-ply-3.11-14.el9.noarch.rpm 1.4 MB/s | 106 kB 00:00 2024-09-12T03:14:18.275 INFO:teuthology.orchestra.run.smithi045.stdout:(13/156): libcephfs-devel-19.1.1-298.gb53b6d3e. 635 kB/s | 32 kB 00:00 2024-09-12T03:14:18.286 INFO:teuthology.orchestra.run.smithi027.stdout:(45/156): python3-pycparser-2.20-6.el9.noarch.r 2.6 MB/s | 135 kB 00:00 2024-09-12T03:14:18.312 INFO:teuthology.orchestra.run.smithi027.stdout:(46/156): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-09-12T03:14:18.337 INFO:teuthology.orchestra.run.smithi027.stdout:(47/156): python3-urllib3-1.26.5-6.el9.noarch.r 4.2 MB/s | 215 kB 00:00 2024-09-12T03:14:18.367 INFO:teuthology.orchestra.run.smithi045.stdout:(14/156): libcephfs2-19.1.1-298.gb53b6d3e.el9.x 8.0 MB/s | 752 kB 00:00 2024-09-12T03:14:18.371 INFO:teuthology.orchestra.run.smithi027.stdout:(48/156): python3-cryptography-36.0.1-4.el9.x86 5.6 MB/s | 1.2 MB 00:00 2024-09-12T03:14:18.425 INFO:teuthology.orchestra.run.smithi027.stdout:(49/156): automake-1.16.2-8.el9.noarch.rpm 6.0 MB/s | 689 kB 00:00 2024-09-12T03:14:18.426 INFO:teuthology.orchestra.run.smithi045.stdout:(15/156): libcephsqlite-19.1.1-298.gb53b6d3e.el 2.7 MB/s | 163 kB 00:00 2024-09-12T03:14:18.446 INFO:teuthology.orchestra.run.smithi027.stdout:(50/156): boost-program-options-1.75.0-8.el9.x8 1.4 MB/s | 107 kB 00:00 2024-09-12T03:14:18.472 INFO:teuthology.orchestra.run.smithi027.stdout:(51/156): elfutils-libelf-devel-0.191-4.el9.x86 468 kB/s | 22 kB 00:00 2024-09-12T03:14:18.484 INFO:teuthology.orchestra.run.smithi045.stdout:(16/156): librados-devel-19.1.1-298.gb53b6d3e.e 2.1 MB/s | 127 kB 00:00 2024-09-12T03:14:18.514 INFO:teuthology.orchestra.run.smithi027.stdout:(52/156): bison-3.7.4-5.el9.x86_64.rpm 5.2 MB/s | 943 kB 00:00 2024-09-12T03:14:18.539 INFO:teuthology.orchestra.run.smithi027.stdout:(53/156): flex-2.6.4-9.el9.x86_64.rpm 3.3 MB/s | 313 kB 00:00 2024-09-12T03:14:18.551 INFO:teuthology.orchestra.run.smithi045.stdout:(17/156): libradosstriper1-19.1.1-298.gb53b6d3e 7.4 MB/s | 506 kB 00:00 2024-09-12T03:14:18.564 INFO:teuthology.orchestra.run.smithi027.stdout:(54/156): flexiblas-3.0.4-8.el9.x86_64.rpm 355 kB/s | 33 kB 00:00 2024-09-12T03:14:18.590 INFO:teuthology.orchestra.run.smithi027.stdout:(55/156): flexiblas-openblas-openmp-3.0.4-8.el9 362 kB/s | 18 kB 00:00 2024-09-12T03:14:18.640 INFO:teuthology.orchestra.run.smithi027.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 148 kB/s | 11 kB 00:00 2024-09-12T03:14:18.665 INFO:teuthology.orchestra.run.smithi027.stdout:(57/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.1 MB/s | 160 kB 00:00 2024-09-12T03:14:18.690 INFO:teuthology.orchestra.run.smithi027.stdout:(58/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 902 kB/s | 45 kB 00:00 2024-09-12T03:14:18.732 INFO:teuthology.orchestra.run.smithi027.stdout:(59/156): librdkafka-1.6.1-102.el9.x86_64.rpm 9.7 MB/s | 662 kB 00:00 2024-09-12T03:14:18.774 INFO:teuthology.orchestra.run.smithi027.stdout:(60/156): libtool-2.4.6-46.el9.x86_64.rpm 6.8 MB/s | 581 kB 00:00 2024-09-12T03:14:18.799 INFO:teuthology.orchestra.run.smithi027.stdout:(61/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 366 kB/s | 24 kB 00:00 2024-09-12T03:14:18.825 INFO:teuthology.orchestra.run.smithi027.stdout:(62/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 949 kB/s | 47 kB 00:00 2024-09-12T03:14:18.850 INFO:teuthology.orchestra.run.smithi027.stdout:(63/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.7 MB/s | 292 kB 00:00 2024-09-12T03:14:18.876 INFO:teuthology.orchestra.run.smithi027.stdout:(64/156): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 188 kB 00:00 2024-09-12T03:14:18.885 INFO:teuthology.orchestra.run.smithi045.stdout:(18/156): librgw2-19.1.1-298.gb53b6d3e.el9.x86_ 16 MB/s | 5.3 MB 00:00 2024-09-12T03:14:18.901 INFO:teuthology.orchestra.run.smithi027.stdout:(65/156): openblas-0.3.26-2.el9.x86_64.rpm 769 kB/s | 38 kB 00:00 2024-09-12T03:14:18.944 INFO:teuthology.orchestra.run.smithi045.stdout:(19/156): python3-ceph-argparse-19.1.1-298.gb53 775 kB/s | 45 kB 00:00 2024-09-12T03:14:18.960 INFO:teuthology.orchestra.run.smithi027.stdout:(66/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 6.7 MB/s | 3.0 MB 00:00 2024-09-12T03:14:19.060 INFO:teuthology.orchestra.run.smithi027.stdout:(67/156): openblas-openmp-0.3.26-2.el9.x86_64.r 27 MB/s | 4.9 MB 00:00 2024-09-12T03:14:19.102 INFO:teuthology.orchestra.run.smithi027.stdout:(68/156): postgresql-13.16-1.el9.x86_64.rpm 11 MB/s | 1.6 MB 00:00 2024-09-12T03:14:19.119 INFO:teuthology.orchestra.run.smithi045.stdout:(20/156): ceph-osd-19.1.1-298.gb53b6d3e.el9.x86 8.1 MB/s | 17 MB 00:02 2024-09-12T03:14:19.135 INFO:teuthology.orchestra.run.smithi027.stdout:(69/156): postgresql-contrib-13.16-1.el9.x86_64 12 MB/s | 910 kB 00:00 2024-09-12T03:14:19.144 INFO:teuthology.orchestra.run.smithi045.stdout:(21/156): python3-ceph-common-19.1.1-298.gb53b6 678 kB/s | 136 kB 00:00 2024-09-12T03:14:19.169 INFO:teuthology.orchestra.run.smithi045.stdout:(22/156): python3-cephfs-19.1.1-298.gb53b6d3e.e 3.2 MB/s | 162 kB 00:00 2024-09-12T03:14:19.194 INFO:teuthology.orchestra.run.smithi027.stdout:(70/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 15 MB/s | 4.4 MB 00:00 2024-09-12T03:14:19.194 INFO:teuthology.orchestra.run.smithi045.stdout:(23/156): python3-rados-19.1.1-298.gb53b6d3e.el 6.2 MB/s | 321 kB 00:00 2024-09-12T03:14:19.219 INFO:teuthology.orchestra.run.smithi027.stdout:(71/156): postgresql-private-libs-13.16-1.el9.x 1.1 MB/s | 137 kB 00:00 2024-09-12T03:14:19.220 INFO:teuthology.orchestra.run.smithi045.stdout:(24/156): python3-rbd-19.1.1-298.gb53b6d3e.el9. 5.9 MB/s | 301 kB 00:00 2024-09-12T03:14:19.246 INFO:teuthology.orchestra.run.smithi045.stdout:(25/156): python3-rgw-19.1.1-298.gb53b6d3e.el9. 1.9 MB/s | 100 kB 00:00 2024-09-12T03:14:19.261 INFO:teuthology.orchestra.run.smithi027.stdout:(72/156): protobuf-3.14.0-13.el9.x86_64.rpm 15 MB/s | 1.0 MB 00:00 2024-09-12T03:14:19.262 INFO:teuthology.orchestra.run.smithi045.stdout:(26/156): rbd-fuse-19.1.1-298.gb53b6d3e.el9.x86 2.0 MB/s | 85 kB 00:00 2024-09-12T03:14:19.287 INFO:teuthology.orchestra.run.smithi027.stdout:(73/156): python3-attrs-20.3.0-7.el9.noarch.rpm 1.3 MB/s | 87 kB 00:00 2024-09-12T03:14:19.313 INFO:teuthology.orchestra.run.smithi045.stdout:(27/156): rbd-nbd-19.1.1-298.gb53b6d3e.el9.x86_ 3.3 MB/s | 171 kB 00:00 2024-09-12T03:14:19.329 INFO:teuthology.orchestra.run.smithi027.stdout:(74/156): python3-colorama-0.4.6-3.el9.noarch.r 1.4 MB/s | 61 kB 00:00 2024-09-12T03:14:19.371 INFO:teuthology.orchestra.run.smithi045.stdout:(28/156): ceph-grafana-dashboards-19.1.1-298.gb 546 kB/s | 32 kB 00:00 2024-09-12T03:14:19.404 INFO:teuthology.orchestra.run.smithi027.stdout:(75/156): postgresql-server-13.16-1.el9.x86_64. 22 MB/s | 5.8 MB 00:00 2024-09-12T03:14:19.430 INFO:teuthology.orchestra.run.smithi027.stdout:(76/156): python3-devel-3.9.19-8.el9.x86_64.rpm 2.4 MB/s | 245 kB 00:00 2024-09-12T03:14:19.455 INFO:teuthology.orchestra.run.smithi027.stdout:(77/156): python3-jinja2-2.11.3-6.el9.noarch.rp 4.8 MB/s | 248 kB 00:00 2024-09-12T03:14:19.455 INFO:teuthology.orchestra.run.smithi045.stdout:(29/156): rbd-mirror-19.1.1-298.gb53b6d3e.el9.x 15 MB/s | 3.1 MB 00:00 2024-09-12T03:14:19.480 INFO:teuthology.orchestra.run.smithi027.stdout:(78/156): python3-jsonpatch-1.21-16.el9.noarch. 512 kB/s | 26 kB 00:00 2024-09-12T03:14:19.480 INFO:teuthology.orchestra.run.smithi045.stdout:(30/156): ceph-mgr-cephadm-19.1.1-298.gb53b6d3e 1.3 MB/s | 146 kB 00:00 2024-09-12T03:14:19.505 INFO:teuthology.orchestra.run.smithi027.stdout:(79/156): python3-jsonpointer-2.0-4.el9.noarch. 382 kB/s | 19 kB 00:00 2024-09-12T03:14:19.530 INFO:teuthology.orchestra.run.smithi027.stdout:(80/156): python3-mako-1.1.4-6.el9.noarch.rpm 3.3 MB/s | 172 kB 00:00 2024-09-12T03:14:19.555 INFO:teuthology.orchestra.run.smithi027.stdout:(81/156): python3-markupsafe-1.1.1-12.el9.x86_6 695 kB/s | 35 kB 00:00 2024-09-12T03:14:19.631 INFO:teuthology.orchestra.run.smithi027.stdout:(82/156): python3-babel-2.9.1-2.el9.noarch.rpm 16 MB/s | 6.0 MB 00:00 2024-09-12T03:14:19.656 INFO:teuthology.orchestra.run.smithi027.stdout:(83/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.4 MB/s | 250 kB 00:00 2024-09-12T03:14:19.682 INFO:teuthology.orchestra.run.smithi027.stdout:(84/156): python3-oauthlib-3.1.1-5.el9.noarch.r 4.3 MB/s | 222 kB 00:00 2024-09-12T03:14:19.689 INFO:teuthology.orchestra.run.smithi045.stdout:(31/156): ceph-mgr-dashboard-19.1.1-298.gb53b6d 15 MB/s | 3.6 MB 00:00 2024-09-12T03:14:19.707 INFO:teuthology.orchestra.run.smithi027.stdout:(85/156): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-09-12T03:14:19.732 INFO:teuthology.orchestra.run.smithi027.stdout:(86/156): python3-prettytable-0.7.2-27.el9.noar 842 kB/s | 42 kB 00:00 2024-09-12T03:14:19.739 INFO:teuthology.orchestra.run.smithi045.stdout:(32/156): ceph-mgr-modules-core-19.1.1-298.gb53 4.9 MB/s | 249 kB 00:00 2024-09-12T03:14:19.757 INFO:teuthology.orchestra.run.smithi027.stdout:(87/156): python3-protobuf-3.14.0-13.el9.noarch 5.2 MB/s | 269 kB 00:00 2024-09-12T03:14:19.783 INFO:teuthology.orchestra.run.smithi027.stdout:(88/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 159 kB 00:00 2024-09-12T03:14:19.798 INFO:teuthology.orchestra.run.smithi045.stdout:(33/156): ceph-mgr-rook-19.1.1-298.gb53b6d3e.el 845 kB/s | 49 kB 00:00 2024-09-12T03:14:19.808 INFO:teuthology.orchestra.run.smithi027.stdout:(89/156): python3-pyasn1-modules-0.4.8-6.el9.no 5.4 MB/s | 279 kB 00:00 2024-09-12T03:14:19.833 INFO:teuthology.orchestra.run.smithi027.stdout:(90/156): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-09-12T03:14:19.857 INFO:teuthology.orchestra.run.smithi045.stdout:(34/156): ceph-prometheus-alerts-19.1.1-298.gb5 282 kB/s | 16 kB 00:00 2024-09-12T03:14:19.858 INFO:teuthology.orchestra.run.smithi027.stdout:(91/156): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-09-12T03:14:19.901 INFO:teuthology.orchestra.run.smithi027.stdout:(92/156): python3-toml-0.10.2-6.el9.noarch.rpm 1.0 MB/s | 42 kB 00:00 2024-09-12T03:14:19.915 INFO:teuthology.orchestra.run.smithi045.stdout:(35/156): ceph-volume-19.1.1-298.gb53b6d3e.el9. 4.8 MB/s | 285 kB 00:00 2024-09-12T03:14:19.967 INFO:teuthology.orchestra.run.smithi027.stdout:(93/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 12 MB/s | 5.1 MB 00:00 2024-09-12T03:14:19.993 INFO:teuthology.orchestra.run.smithi027.stdout:(94/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 482 kB/s | 44 kB 00:00 2024-09-12T03:14:20.016 INFO:teuthology.orchestra.run.smithi045.stdout:(36/156): cephadm-19.1.1-298.gb53b6d3e.el9.noar 7.5 MB/s | 765 kB 00:00 2024-09-12T03:14:20.018 INFO:teuthology.orchestra.run.smithi027.stdout:(95/156): qatlib-24.02.0-1.el9.x86_64.rpm 4.3 MB/s | 221 kB 00:00 2024-09-12T03:14:20.043 INFO:teuthology.orchestra.run.smithi027.stdout:(96/156): qatlib-service-24.02.0-1.el9.x86_64.r 717 kB/s | 36 kB 00:00 2024-09-12T03:14:20.069 INFO:teuthology.orchestra.run.smithi027.stdout:(97/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 945 kB/s | 47 kB 00:00 2024-09-12T03:14:20.074 INFO:teuthology.orchestra.run.smithi045.stdout:(37/156): cephfs-top-19.1.1-298.gb53b6d3e.el9.n 313 kB/s | 18 kB 00:00 2024-09-12T03:14:20.094 INFO:teuthology.orchestra.run.smithi027.stdout:(98/156): socat-1.7.4.1-6.el9.x86_64.rpm 5.9 MB/s | 304 kB 00:00 2024-09-12T03:14:20.119 INFO:teuthology.orchestra.run.smithi027.stdout:(99/156): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-09-12T03:14:20.144 INFO:teuthology.orchestra.run.smithi027.stdout:(100/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 1.9 MB/s | 96 kB 00:00 2024-09-12T03:14:20.170 INFO:teuthology.orchestra.run.smithi027.stdout:(101/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-09-12T03:14:20.195 INFO:teuthology.orchestra.run.smithi027.stdout:(102/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 896 kB/s | 45 kB 00:00 2024-09-12T03:14:20.262 INFO:teuthology.orchestra.run.smithi027.stdout:(103/156): lua-devel-5.4.4-4.el9.x86_64.rpm 243 kB/s | 22 kB 00:00 2024-09-12T03:14:20.321 INFO:teuthology.orchestra.run.smithi027.stdout:(104/156): abseil-cpp-20211102.0-3.el9.x86_64.r 9.3 MB/s | 548 kB 00:00 2024-09-12T03:14:20.487 INFO:teuthology.orchestra.run.smithi027.stdout:(105/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 25 MB/s | 16 MB 00:00 2024-09-12T03:14:20.533 INFO:teuthology.orchestra.run.smithi045.stdout:(38/156): ceph-test-19.1.1-298.gb53b6d3e.el9.x8 19 MB/s | 49 MB 00:02 2024-09-12T03:14:20.546 INFO:teuthology.orchestra.run.smithi027.stdout:(106/156): protobuf-compiler-3.14.0-13.el9.x86_ 2.4 MB/s | 863 kB 00:00 2024-09-12T03:14:20.550 INFO:teuthology.orchestra.run.smithi045.stdout:(39/156): fuse-2.9.9-16.el9.x86_64.rpm 168 kB/s | 80 kB 00:00 2024-09-12T03:14:20.571 INFO:teuthology.orchestra.run.smithi027.stdout:(107/156): gperftools-libs-2.9.1-3.el9.x86_64.r 1.2 MB/s | 308 kB 00:00 2024-09-12T03:14:20.605 INFO:teuthology.orchestra.run.smithi027.stdout:(108/156): grpc-data-1.46.7-10.el9.noarch.rpm 166 kB/s | 19 kB 00:00 2024-09-12T03:14:20.630 INFO:teuthology.orchestra.run.smithi027.stdout:(109/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 516 kB/s | 26 kB 00:00 2024-09-12T03:14:20.650 INFO:teuthology.orchestra.run.smithi045.stdout:(40/156): ceph-mgr-diskprediction-local-19.1.1- 6.3 MB/s | 7.4 MB 00:01 2024-09-12T03:14:20.656 INFO:teuthology.orchestra.run.smithi027.stdout:(110/156): liboath-2.6.7-2.el9.x86_64.rpm 1.0 MB/s | 49 kB 00:00 2024-09-12T03:14:20.681 INFO:teuthology.orchestra.run.smithi027.stdout:(111/156): libunwind-1.6.2-1.el9.x86_64.rpm 1.3 MB/s | 67 kB 00:00 2024-09-12T03:14:20.706 INFO:teuthology.orchestra.run.smithi027.stdout:(112/156): luarocks-3.9.2-5.el9.noarch.rpm 2.9 MB/s | 151 kB 00:00 2024-09-12T03:14:20.717 INFO:teuthology.orchestra.run.smithi045.stdout:(41/156): libquadmath-11.5.0-2.el9.x86_64.rpm 1.1 MB/s | 191 kB 00:00 2024-09-12T03:14:20.757 INFO:teuthology.orchestra.run.smithi027.stdout:(113/156): python3-asyncssh-2.13.2-5.el9.noarch 11 MB/s | 548 kB 00:00 2024-09-12T03:14:20.759 INFO:teuthology.orchestra.run.smithi045.stdout:(42/156): libgfortran-11.5.0-2.el9.x86_64.rpm 3.5 MB/s | 801 kB 00:00 2024-09-12T03:14:20.782 INFO:teuthology.orchestra.run.smithi027.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-09-12T03:14:20.801 INFO:teuthology.orchestra.run.smithi045.stdout:(43/156): python3-ply-3.11-14.el9.noarch.rpm 2.5 MB/s | 106 kB 00:00 2024-09-12T03:14:20.815 INFO:teuthology.orchestra.run.smithi027.stdout:(115/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 6.1 MB/s | 839 kB 00:00 2024-09-12T03:14:20.826 INFO:teuthology.orchestra.run.smithi045.stdout:(44/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.4 MB/s | 253 kB 00:00 2024-09-12T03:14:20.840 INFO:teuthology.orchestra.run.smithi027.stdout:(116/156): python3-cachetools-4.2.4-1.el9.noarc 552 kB/s | 32 kB 00:00 2024-09-12T03:14:20.860 INFO:teuthology.orchestra.run.smithi045.stdout:(45/156): python3-cryptography-36.0.1-4.el9.x86 8.6 MB/s | 1.2 MB 00:00 2024-09-12T03:14:20.865 INFO:teuthology.orchestra.run.smithi027.stdout:(117/156): python3-certifi-2023.05.07-4.el9.noa 573 kB/s | 14 kB 00:00 2024-09-12T03:14:20.885 INFO:teuthology.orchestra.run.smithi045.stdout:(46/156): python3-pycparser-2.20-6.el9.noarch.r 1.6 MB/s | 135 kB 00:00 2024-09-12T03:14:20.891 INFO:teuthology.orchestra.run.smithi027.stdout:(118/156): python3-cheroot-8.6.0-4.el9.noarch.r 3.4 MB/s | 172 kB 00:00 2024-09-12T03:14:20.910 INFO:teuthology.orchestra.run.smithi045.stdout:(47/156): python3-requests-2.25.1-8.el9.noarch. 1.5 MB/s | 125 kB 00:00 2024-09-12T03:14:20.933 INFO:teuthology.orchestra.run.smithi027.stdout:(119/156): python3-cherrypy-18.6.1-2.el9.noarch 5.2 MB/s | 358 kB 00:00 2024-09-12T03:14:20.936 INFO:teuthology.orchestra.run.smithi045.stdout:(48/156): python3-urllib3-1.26.5-6.el9.noarch.r 2.8 MB/s | 215 kB 00:00 2024-09-12T03:14:20.958 INFO:teuthology.orchestra.run.smithi027.stdout:(120/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 4.4 MB/s | 303 kB 00:00 2024-09-12T03:14:20.992 INFO:teuthology.orchestra.run.smithi027.stdout:(121/156): python3-google-auth-2.34.0-1.el9.noa 6.5 MB/s | 220 kB 00:00 2024-09-12T03:14:21.034 INFO:teuthology.orchestra.run.smithi027.stdout:(122/156): python3-grpcio-tools-1.46.7-10.el9.x 3.4 MB/s | 144 kB 00:00 2024-09-12T03:14:21.051 INFO:teuthology.orchestra.run.smithi027.stdout:(123/156): python3-jaraco-8.2.1-3.el9.noarch.rp 648 kB/s | 11 kB 00:00 2024-09-12T03:14:21.053 INFO:teuthology.orchestra.run.smithi045.stdout:(49/156): boost-program-options-1.75.0-8.el9.x8 916 kB/s | 107 kB 00:00 2024-09-12T03:14:21.076 INFO:teuthology.orchestra.run.smithi027.stdout:(124/156): python3-jaraco-classes-3.2.1-5.el9.n 714 kB/s | 18 kB 00:00 2024-09-12T03:14:21.086 INFO:teuthology.orchestra.run.smithi045.stdout:(50/156): automake-1.16.2-8.el9.noarch.rpm 3.3 MB/s | 689 kB 00:00 2024-09-12T03:14:21.101 INFO:teuthology.orchestra.run.smithi027.stdout:(125/156): python3-jaraco-collections-3.0.0-8.e 928 kB/s | 23 kB 00:00 2024-09-12T03:14:21.103 INFO:teuthology.orchestra.run.smithi045.stdout:(51/156): elfutils-libelf-devel-0.191-4.el9.x86 431 kB/s | 22 kB 00:00 2024-09-12T03:14:21.127 INFO:teuthology.orchestra.run.smithi027.stdout:(126/156): python3-jaraco-functools-3.5.0-2.el9 782 kB/s | 19 kB 00:00 2024-09-12T03:14:21.138 INFO:teuthology.orchestra.run.smithi045.stdout:(52/156): bison-3.7.4-5.el9.x86_64.rpm 4.1 MB/s | 943 kB 00:00 2024-09-12T03:14:21.152 INFO:teuthology.orchestra.run.smithi027.stdout:(127/156): python3-jaraco-text-3.2.0-6.el9.noar 795 kB/s | 20 kB 00:00 2024-09-12T03:14:21.154 INFO:teuthology.orchestra.run.smithi045.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 650 kB/s | 33 kB 00:00 2024-09-12T03:14:21.180 INFO:teuthology.orchestra.run.smithi045.stdout:(54/156): flex-2.6.4-9.el9.x86_64.rpm 3.3 MB/s | 313 kB 00:00 2024-09-12T03:14:21.196 INFO:teuthology.orchestra.run.smithi045.stdout:(55/156): flexiblas-openblas-openmp-3.0.4-8.el9 435 kB/s | 18 kB 00:00 2024-09-12T03:14:21.219 INFO:teuthology.orchestra.run.smithi027.stdout:(128/156): python3-grpcio-1.46.7-10.el9.x86_64. 7.8 MB/s | 2.0 MB 00:00 2024-09-12T03:14:21.222 INFO:teuthology.orchestra.run.smithi045.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 266 kB/s | 11 kB 00:00 2024-09-12T03:14:21.252 INFO:teuthology.orchestra.run.smithi027.stdout:(129/156): python3-kubernetes-26.1.0-2.el9.noar 10 MB/s | 1.0 MB 00:00 2024-09-12T03:14:21.264 INFO:teuthology.orchestra.run.smithi045.stdout:(57/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.3 MB/s | 160 kB 00:00 2024-09-12T03:14:21.278 INFO:teuthology.orchestra.run.smithi027.stdout:(130/156): python3-logutils-0.3.5-21.el9.noarch 791 kB/s | 46 kB 00:00 2024-09-12T03:14:21.289 INFO:teuthology.orchestra.run.smithi045.stdout:(58/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 675 kB/s | 45 kB 00:00 2024-09-12T03:14:21.303 INFO:teuthology.orchestra.run.smithi027.stdout:(131/156): python3-more-itertools-8.12.0-2.el9. 1.5 MB/s | 79 kB 00:00 2024-09-12T03:14:21.323 INFO:teuthology.orchestra.run.smithi045.stdout:(59/156): librdkafka-1.6.1-102.el9.x86_64.rpm 11 MB/s | 662 kB 00:00 2024-09-12T03:14:21.328 INFO:teuthology.orchestra.run.smithi027.stdout:(132/156): python3-natsort-7.1.1-5.el9.noarch.r 1.1 MB/s | 58 kB 00:00 2024-09-12T03:14:21.354 INFO:teuthology.orchestra.run.smithi027.stdout:(133/156): python3-portend-3.1.0-2.el9.noarch.r 658 kB/s | 16 kB 00:00 2024-09-12T03:14:21.379 INFO:teuthology.orchestra.run.smithi027.stdout:(134/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 3.5 MB/s | 90 kB 00:00 2024-09-12T03:14:21.381 INFO:teuthology.orchestra.run.smithi045.stdout:(60/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 12 MB/s | 3.0 MB 00:00 2024-09-12T03:14:21.407 INFO:teuthology.orchestra.run.smithi045.stdout:(61/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 293 kB/s | 24 kB 00:00 2024-09-12T03:14:21.413 INFO:teuthology.orchestra.run.smithi027.stdout:(135/156): python3-pyperclip-1.8.2-2.el9.noarch 850 kB/s | 28 kB 00:00 2024-09-12T03:14:21.432 INFO:teuthology.orchestra.run.smithi045.stdout:(62/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 947 kB/s | 47 kB 00:00 2024-09-12T03:14:21.455 INFO:teuthology.orchestra.run.smithi027.stdout:(136/156): python3-pecan-1.4.2-3.el9.noarch.rpm 1.8 MB/s | 272 kB 00:00 2024-09-12T03:14:21.457 INFO:teuthology.orchestra.run.smithi045.stdout:(63/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.7 MB/s | 292 kB 00:00 2024-09-12T03:14:21.472 INFO:teuthology.orchestra.run.smithi027.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 523 kB/s | 31 kB 00:00 2024-09-12T03:14:21.488 INFO:teuthology.orchestra.run.smithi027.stdout:(138/156): python3-rsa-4.9-2.el9.noarch.rpm 3.5 MB/s | 59 kB 00:00 2024-09-12T03:14:21.491 INFO:teuthology.orchestra.run.smithi045.stdout:(64/156): libtool-2.4.6-46.el9.x86_64.rpm 2.8 MB/s | 581 kB 00:00 2024-09-12T03:14:21.505 INFO:teuthology.orchestra.run.smithi027.stdout:(139/156): python3-tempora-5.0.0-2.el9.noarch.r 2.1 MB/s | 36 kB 00:00 2024-09-12T03:14:21.516 INFO:teuthology.orchestra.run.smithi045.stdout:(65/156): openblas-0.3.26-2.el9.x86_64.rpm 657 kB/s | 38 kB 00:00 2024-09-12T03:14:21.531 INFO:teuthology.orchestra.run.smithi027.stdout:(140/156): python3-routes-2.5.1-5.el9.noarch.rp 2.4 MB/s | 188 kB 00:00 2024-09-12T03:14:21.541 INFO:teuthology.orchestra.run.smithi045.stdout:(66/156): lua-5.4.4-4.el9.x86_64.rpm 1.7 MB/s | 188 kB 00:00 2024-09-12T03:14:21.556 INFO:teuthology.orchestra.run.smithi027.stdout:(141/156): python3-typing-extensions-4.12.2-2.e 1.4 MB/s | 74 kB 00:00 2024-09-12T03:14:21.582 INFO:teuthology.orchestra.run.smithi027.stdout:(142/156): python3-websocket-client-1.2.3-2.el9 3.5 MB/s | 90 kB 00:00 2024-09-12T03:14:21.607 INFO:teuthology.orchestra.run.smithi027.stdout:(143/156): python3-webob-1.8.8-2.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-09-12T03:14:21.617 INFO:teuthology.orchestra.run.smithi045.stdout:(67/156): postgresql-13.16-1.el9.x86_64.rpm 21 MB/s | 1.6 MB 00:00 2024-09-12T03:14:21.632 INFO:teuthology.orchestra.run.smithi027.stdout:(144/156): python3-zc-lockfile-2.0-10.el9.noarc 798 kB/s | 20 kB 00:00 2024-09-12T03:14:21.692 INFO:teuthology.orchestra.run.smithi045.stdout:(68/156): openblas-openmp-0.3.26-2.el9.x86_64.r 25 MB/s | 4.9 MB 00:00 2024-09-12T03:14:21.700 INFO:teuthology.orchestra.run.smithi027.stdout:(145/156): libarrow-9.0.0-11.el9.x86_64.rpm 3.8 MB/s | 4.4 MB 00:01 2024-09-12T03:14:21.725 INFO:teuthology.orchestra.run.smithi027.stdout:(146/156): re2-20211101-20.el9.x86_64.rpm 2.0 MB/s | 191 kB 00:00 2024-09-12T03:14:21.751 INFO:teuthology.orchestra.run.smithi027.stdout:(147/156): python3-werkzeug-2.0.3-3.el9.1.noarc 2.5 MB/s | 427 kB 00:00 2024-09-12T03:14:21.759 INFO:teuthology.orchestra.run.smithi045.stdout:(69/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 18 MB/s | 4.4 MB 00:00 2024-09-12T03:14:21.784 INFO:teuthology.orchestra.run.smithi045.stdout:(70/156): postgresql-private-libs-13.16-1.el9.x 1.5 MB/s | 137 kB 00:00 2024-09-12T03:14:21.801 INFO:teuthology.orchestra.run.smithi027.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2024-09-12T03:14:21.818 INFO:teuthology.orchestra.run.smithi045.stdout:(71/156): postgresql-contrib-13.16-1.el9.x86_64 4.4 MB/s | 910 kB 00:00 2024-09-12T03:14:21.852 INFO:teuthology.orchestra.run.smithi045.stdout:(72/156): protobuf-3.14.0-13.el9.x86_64.rpm 15 MB/s | 1.0 MB 00:00 2024-09-12T03:14:21.868 INFO:teuthology.orchestra.run.smithi045.stdout:(73/156): python3-attrs-20.3.0-7.el9.noarch.rpm 1.7 MB/s | 87 kB 00:00 2024-09-12T03:14:21.911 INFO:teuthology.orchestra.run.smithi045.stdout:(74/156): python3-colorama-0.4.6-3.el9.noarch.r 1.4 MB/s | 61 kB 00:00 2024-09-12T03:14:21.960 INFO:teuthology.orchestra.run.smithi027.stdout:(149/156): NetworkManager-1.49.90-1.el9.x86_64. 15 MB/s | 2.3 MB 00:00 2024-09-12T03:14:21.994 INFO:teuthology.orchestra.run.smithi045.stdout:(75/156): postgresql-server-13.16-1.el9.x86_64. 25 MB/s | 5.8 MB 00:00 2024-09-12T03:14:22.020 INFO:teuthology.orchestra.run.smithi045.stdout:(76/156): python3-devel-3.9.19-8.el9.x86_64.rpm 2.2 MB/s | 245 kB 00:00 2024-09-12T03:14:22.035 INFO:teuthology.orchestra.run.smithi027.stdout:(150/156): librados2-19.1.1-298.gb53b6d3e.el9.x 11 MB/s | 3.4 MB 00:00 2024-09-12T03:14:22.045 INFO:teuthology.orchestra.run.smithi045.stdout:(77/156): python3-jinja2-2.11.3-6.el9.noarch.rp 4.8 MB/s | 248 kB 00:00 2024-09-12T03:14:22.070 INFO:teuthology.orchestra.run.smithi045.stdout:(78/156): python3-jsonpatch-1.21-16.el9.noarch. 512 kB/s | 26 kB 00:00 2024-09-12T03:14:22.085 INFO:teuthology.orchestra.run.smithi027.stdout:(151/156): NetworkManager-libnm-1.49.90-1.el9.x 14 MB/s | 1.8 MB 00:00 2024-09-12T03:14:22.104 INFO:teuthology.orchestra.run.smithi045.stdout:(79/156): python3-jsonpointer-2.0-4.el9.noarch. 325 kB/s | 19 kB 00:00 2024-09-12T03:14:22.121 INFO:teuthology.orchestra.run.smithi045.stdout:(80/156): python3-mako-1.1.4-6.el9.noarch.rpm 3.3 MB/s | 172 kB 00:00 2024-09-12T03:14:22.127 INFO:teuthology.orchestra.run.smithi027.stdout:(152/156): NetworkManager-tui-1.49.90-1.el9.x86 5.9 MB/s | 249 kB 00:00 2024-09-12T03:14:22.138 INFO:teuthology.orchestra.run.smithi045.stdout:(81/156): python3-markupsafe-1.1.1-12.el9.x86_6 1.0 MB/s | 35 kB 00:00 2024-09-12T03:14:22.152 INFO:teuthology.orchestra.run.smithi027.stdout:(153/156): NetworkManager-team-1.49.90-1.el9.x8 348 kB/s | 41 kB 00:00 2024-09-12T03:14:22.203 INFO:teuthology.orchestra.run.smithi027.stdout:(154/156): openssl-3.2.2-6.el9.x86_64.rpm 18 MB/s | 1.4 MB 00:00 2024-09-12T03:14:22.230 INFO:teuthology.orchestra.run.smithi045.stdout:(82/156): python3-babel-2.9.1-2.el9.noarch.rpm 16 MB/s | 6.0 MB 00:00 2024-09-12T03:14:22.255 INFO:teuthology.orchestra.run.smithi045.stdout:(83/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.1 MB/s | 250 kB 00:00 2024-09-12T03:14:22.280 INFO:teuthology.orchestra.run.smithi045.stdout:(84/156): python3-oauthlib-3.1.1-5.el9.noarch.r 4.3 MB/s | 222 kB 00:00 2024-09-12T03:14:22.305 INFO:teuthology.orchestra.run.smithi045.stdout:(85/156): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-09-12T03:14:22.322 INFO:teuthology.orchestra.run.smithi045.stdout:(86/156): python3-prettytable-0.7.2-27.el9.noar 1.0 MB/s | 42 kB 00:00 2024-09-12T03:14:22.328 INFO:teuthology.orchestra.run.smithi027.stdout:(155/156): librbd1-19.1.1-298.gb53b6d3e.el9.x86 5.5 MB/s | 3.2 MB 00:00 2024-09-12T03:14:22.348 INFO:teuthology.orchestra.run.smithi045.stdout:(87/156): python3-protobuf-3.14.0-13.el9.noarch 6.3 MB/s | 269 kB 00:00 2024-09-12T03:14:22.373 INFO:teuthology.orchestra.run.smithi045.stdout:(88/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 159 kB 00:00 2024-09-12T03:14:22.378 INFO:teuthology.orchestra.run.smithi027.stdout:(156/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 11 MB/s | 2.4 MB 00:00 2024-09-12T03:14:22.385 INFO:teuthology.orchestra.run.smithi027.stdout:-------------------------------------------------------------------------------- 2024-09-12T03:14:22.385 INFO:teuthology.orchestra.run.smithi027.stdout:Total 25 MB/s | 226 MB 00:09 2024-09-12T03:14:22.398 INFO:teuthology.orchestra.run.smithi045.stdout:(89/156): python3-pyasn1-modules-0.4.8-6.el9.no 5.4 MB/s | 279 kB 00:00 2024-09-12T03:14:22.424 INFO:teuthology.orchestra.run.smithi045.stdout:(90/156): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-09-12T03:14:22.449 INFO:teuthology.orchestra.run.smithi045.stdout:(91/156): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-09-12T03:14:22.533 INFO:teuthology.orchestra.run.smithi045.stdout:(92/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 12 MB/s | 5.1 MB 00:00 2024-09-12T03:14:22.558 INFO:teuthology.orchestra.run.smithi045.stdout:(93/156): python3-toml-0.10.2-6.el9.noarch.rpm 383 kB/s | 42 kB 00:00 2024-09-12T03:14:22.584 INFO:teuthology.orchestra.run.smithi045.stdout:(94/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 882 kB/s | 44 kB 00:00 2024-09-12T03:14:22.609 INFO:teuthology.orchestra.run.smithi045.stdout:(95/156): qatlib-24.02.0-1.el9.x86_64.rpm 4.3 MB/s | 221 kB 00:00 2024-09-12T03:14:22.634 INFO:teuthology.orchestra.run.smithi045.stdout:(96/156): qatlib-service-24.02.0-1.el9.x86_64.r 716 kB/s | 36 kB 00:00 2024-09-12T03:14:22.659 INFO:teuthology.orchestra.run.smithi045.stdout:(97/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 947 kB/s | 47 kB 00:00 2024-09-12T03:14:22.685 INFO:teuthology.orchestra.run.smithi045.stdout:(98/156): socat-1.7.4.1-6.el9.x86_64.rpm 5.9 MB/s | 304 kB 00:00 2024-09-12T03:14:22.710 INFO:teuthology.orchestra.run.smithi045.stdout:(99/156): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-09-12T03:14:22.735 INFO:teuthology.orchestra.run.smithi045.stdout:(100/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 1.9 MB/s | 96 kB 00:00 2024-09-12T03:14:22.760 INFO:teuthology.orchestra.run.smithi045.stdout:(101/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-09-12T03:14:22.786 INFO:teuthology.orchestra.run.smithi045.stdout:(102/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 891 kB/s | 45 kB 00:00 2024-09-12T03:14:23.019 INFO:teuthology.orchestra.run.smithi045.stdout:(103/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 27 MB/s | 16 MB 00:00 2024-09-12T03:14:23.044 INFO:teuthology.orchestra.run.smithi045.stdout:(104/156): lua-devel-5.4.4-4.el9.x86_64.rpm 79 kB/s | 22 kB 00:00 2024-09-12T03:14:23.103 INFO:teuthology.orchestra.run.smithi045.stdout:(105/156): abseil-cpp-20211102.0-3.el9.x86_64.r 6.4 MB/s | 548 kB 00:00 2024-09-12T03:14:23.128 INFO:teuthology.orchestra.run.smithi045.stdout:(106/156): grpc-data-1.46.7-10.el9.noarch.rpm 780 kB/s | 19 kB 00:00 2024-09-12T03:14:23.154 INFO:teuthology.orchestra.run.smithi045.stdout:(107/156): gperftools-libs-2.9.1-3.el9.x86_64.r 2.8 MB/s | 308 kB 00:00 2024-09-12T03:14:23.179 INFO:teuthology.orchestra.run.smithi045.stdout:(108/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.0 MB/s | 26 kB 00:00 2024-09-12T03:14:23.205 INFO:teuthology.orchestra.run.smithi045.stdout:(109/156): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-09-12T03:14:23.230 INFO:teuthology.orchestra.run.smithi045.stdout:(110/156): protobuf-compiler-3.14.0-13.el9.x86_ 1.9 MB/s | 863 kB 00:00 2024-09-12T03:14:23.263 INFO:teuthology.orchestra.run.smithi045.stdout:(111/156): libunwind-1.6.2-1.el9.x86_64.rpm 1.1 MB/s | 67 kB 00:00 2024-09-12T03:14:23.297 INFO:teuthology.orchestra.run.smithi045.stdout:(112/156): luarocks-3.9.2-5.el9.noarch.rpm 2.2 MB/s | 151 kB 00:00 2024-09-12T03:14:23.331 INFO:teuthology.orchestra.run.smithi045.stdout:(113/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 13 MB/s | 839 kB 00:00 2024-09-12T03:14:23.356 INFO:teuthology.orchestra.run.smithi045.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-09-12T03:14:23.381 INFO:teuthology.orchestra.run.smithi045.stdout:(115/156): python3-cachetools-4.2.4-1.el9.noarc 1.3 MB/s | 32 kB 00:00 2024-09-12T03:14:23.406 INFO:teuthology.orchestra.run.smithi045.stdout:(116/156): python3-certifi-2023.05.07-4.el9.noa 566 kB/s | 14 kB 00:00 2024-09-12T03:14:23.441 INFO:teuthology.orchestra.run.smithi045.stdout:(117/156): python3-asyncssh-2.13.2-5.el9.noarch 3.8 MB/s | 548 kB 00:00 2024-09-12T03:14:23.465 INFO:teuthology.orchestra.run.smithi045.stdout:(118/156): python3-cheroot-8.6.0-4.el9.noarch.r 2.9 MB/s | 172 kB 00:00 2024-09-12T03:14:23.499 INFO:teuthology.orchestra.run.smithi045.stdout:(119/156): python3-cherrypy-18.6.1-2.el9.noarch 6.0 MB/s | 358 kB 00:00 2024-09-12T03:14:23.533 INFO:teuthology.orchestra.run.smithi045.stdout:(120/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 4.4 MB/s | 303 kB 00:00 2024-09-12T03:14:23.558 INFO:teuthology.orchestra.run.smithi045.stdout:(121/156): python3-google-auth-2.34.0-1.el9.noa 3.7 MB/s | 220 kB 00:00 2024-09-12T03:14:23.600 INFO:teuthology.orchestra.run.smithi045.stdout:(122/156): python3-grpcio-tools-1.46.7-10.el9.x 3.4 MB/s | 144 kB 00:00 2024-09-12T03:14:23.625 INFO:teuthology.orchestra.run.smithi045.stdout:(123/156): python3-jaraco-8.2.1-3.el9.noarch.rp 428 kB/s | 11 kB 00:00 2024-09-12T03:14:23.651 INFO:teuthology.orchestra.run.smithi045.stdout:(124/156): python3-jaraco-classes-3.2.1-5.el9.n 708 kB/s | 18 kB 00:00 2024-09-12T03:14:23.654 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction check 2024-09-12T03:14:23.668 INFO:teuthology.orchestra.run.smithi045.stdout:(125/156): python3-jaraco-collections-3.0.0-8.e 1.4 MB/s | 23 kB 00:00 2024-09-12T03:14:23.685 INFO:teuthology.orchestra.run.smithi045.stdout:(126/156): python3-jaraco-functools-3.5.0-2.el9 1.1 MB/s | 19 kB 00:00 2024-09-12T03:14:23.701 INFO:teuthology.orchestra.run.smithi045.stdout:(127/156): python3-jaraco-text-3.2.0-6.el9.noar 1.2 MB/s | 20 kB 00:00 2024-09-12T03:14:23.760 INFO:teuthology.orchestra.run.smithi045.stdout:(128/156): python3-kubernetes-26.1.0-2.el9.noar 17 MB/s | 1.0 MB 00:00 2024-09-12T03:14:23.811 INFO:teuthology.orchestra.run.smithi045.stdout:(129/156): python3-grpcio-1.46.7-10.el9.x86_64. 7.3 MB/s | 2.0 MB 00:00 2024-09-12T03:14:23.826 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction check succeeded. 2024-09-12T03:14:23.826 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction test 2024-09-12T03:14:23.853 INFO:teuthology.orchestra.run.smithi045.stdout:(130/156): python3-logutils-0.3.5-21.el9.noarch 503 kB/s | 46 kB 00:00 2024-09-12T03:14:23.878 INFO:teuthology.orchestra.run.smithi045.stdout:(131/156): python3-more-itertools-8.12.0-2.el9. 1.2 MB/s | 79 kB 00:00 2024-09-12T03:14:23.903 INFO:teuthology.orchestra.run.smithi045.stdout:(132/156): python3-natsort-7.1.1-5.el9.noarch.r 1.1 MB/s | 58 kB 00:00 2024-09-12T03:14:23.929 INFO:teuthology.orchestra.run.smithi045.stdout:(133/156): python3-portend-3.1.0-2.el9.noarch.r 656 kB/s | 16 kB 00:00 2024-09-12T03:14:23.996 INFO:teuthology.orchestra.run.smithi045.stdout:(134/156): libarrow-9.0.0-11.el9.x86_64.rpm 5.1 MB/s | 4.4 MB 00:00 2024-09-12T03:14:24.021 INFO:teuthology.orchestra.run.smithi045.stdout:(135/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 977 kB/s | 90 kB 00:00 2024-09-12T03:14:24.047 INFO:teuthology.orchestra.run.smithi045.stdout:(136/156): python3-pyperclip-1.8.2-2.el9.noarch 561 kB/s | 28 kB 00:00 2024-09-12T03:14:24.072 INFO:teuthology.orchestra.run.smithi045.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 611 kB/s | 31 kB 00:00 2024-09-12T03:14:24.097 INFO:teuthology.orchestra.run.smithi045.stdout:(138/156): python3-pecan-1.4.2-3.el9.noarch.rpm 1.2 MB/s | 272 kB 00:00 2024-09-12T03:14:24.122 INFO:teuthology.orchestra.run.smithi045.stdout:(139/156): python3-rsa-4.9-2.el9.noarch.rpm 1.1 MB/s | 59 kB 00:00 2024-09-12T03:14:24.147 INFO:teuthology.orchestra.run.smithi045.stdout:(140/156): python3-tempora-5.0.0-2.el9.noarch.r 711 kB/s | 36 kB 00:00 2024-09-12T03:14:24.173 INFO:teuthology.orchestra.run.smithi045.stdout:(141/156): python3-typing-extensions-4.12.2-2.e 1.5 MB/s | 74 kB 00:00 2024-09-12T03:14:24.198 INFO:teuthology.orchestra.run.smithi045.stdout:(142/156): python3-routes-2.5.1-5.el9.noarch.rp 1.2 MB/s | 188 kB 00:00 2024-09-12T03:14:24.215 INFO:teuthology.orchestra.run.smithi045.stdout:(143/156): python3-websocket-client-1.2.3-2.el9 2.1 MB/s | 90 kB 00:00 2024-09-12T03:14:24.240 INFO:teuthology.orchestra.run.smithi045.stdout:(144/156): python3-zc-lockfile-2.0-10.el9.noarc 803 kB/s | 20 kB 00:00 2024-09-12T03:14:24.265 INFO:teuthology.orchestra.run.smithi045.stdout:(145/156): python3-webob-1.8.8-2.el9.noarch.rpm 1.9 MB/s | 230 kB 00:00 2024-09-12T03:14:24.291 INFO:teuthology.orchestra.run.smithi045.stdout:(146/156): re2-20211101-20.el9.x86_64.rpm 3.7 MB/s | 191 kB 00:00 2024-09-12T03:14:24.325 INFO:teuthology.orchestra.run.smithi045.stdout:(147/156): python3-werkzeug-2.0.3-3.el9.1.noarc 3.3 MB/s | 427 kB 00:00 2024-09-12T03:14:24.375 INFO:teuthology.orchestra.run.smithi045.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 15 MB/s | 1.6 MB 00:00 2024-09-12T03:14:24.559 INFO:teuthology.orchestra.run.smithi045.stdout:(149/156): NetworkManager-1.49.90-1.el9.x86_64. 13 MB/s | 2.3 MB 00:00 2024-09-12T03:14:24.634 INFO:teuthology.orchestra.run.smithi045.stdout:(150/156): NetworkManager-libnm-1.49.90-1.el9.x 24 MB/s | 1.8 MB 00:00 2024-09-12T03:14:24.793 INFO:teuthology.orchestra.run.smithi045.stdout:(151/156): NetworkManager-team-1.49.90-1.el9.x8 257 kB/s | 41 kB 00:00 2024-09-12T03:14:24.851 INFO:teuthology.orchestra.run.smithi045.stdout:(152/156): librados2-19.1.1-298.gb53b6d3e.el9.x 6.1 MB/s | 3.4 MB 00:00 2024-09-12T03:14:24.876 INFO:teuthology.orchestra.run.smithi045.stdout:(153/156): NetworkManager-tui-1.49.90-1.el9.x86 2.9 MB/s | 249 kB 00:00 2024-09-12T03:14:24.927 INFO:teuthology.orchestra.run.smithi045.stdout:(154/156): librbd1-19.1.1-298.gb53b6d3e.el9.x86 5.3 MB/s | 3.2 MB 00:00 2024-09-12T03:14:25.010 INFO:teuthology.orchestra.run.smithi045.stdout:(155/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 18 MB/s | 2.4 MB 00:00 2024-09-12T03:14:25.052 INFO:teuthology.orchestra.run.smithi045.stdout:(156/156): openssl-3.2.2-6.el9.x86_64.rpm 6.8 MB/s | 1.4 MB 00:00 2024-09-12T03:14:25.058 INFO:teuthology.orchestra.run.smithi045.stdout:-------------------------------------------------------------------------------- 2024-09-12T03:14:25.058 INFO:teuthology.orchestra.run.smithi045.stdout:Total 23 MB/s | 226 MB 00:09 2024-09-12T03:14:26.214 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction test succeeded. 2024-09-12T03:14:26.214 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction 2024-09-12T03:14:26.329 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction check 2024-09-12T03:14:26.498 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction check succeeded. 2024-09-12T03:14:26.498 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction test 2024-09-12T03:14:28.811 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction test succeeded. 2024-09-12T03:14:28.811 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction 2024-09-12T03:14:32.907 INFO:teuthology.orchestra.run.smithi027.stdout: Preparing : 1/1 2024-09-12T03:14:33.050 INFO:teuthology.orchestra.run.smithi027.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-12T03:14:33.089 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-12T03:14:33.151 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-12T03:14:33.200 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-12T03:14:33.225 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-12T03:14:33.281 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-12T03:14:33.635 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-12T03:14:33.734 INFO:teuthology.orchestra.run.smithi027.stdout: Upgrading : librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-12T03:14:33.860 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-12T03:14:33.885 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-12T03:14:33.979 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-12T03:14:34.007 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 10/164 2024-09-12T03:14:34.188 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-12T03:14:34.254 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-12T03:14:34.290 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-12T03:14:34.396 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-12T03:14:34.424 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 14/164 2024-09-12T03:14:34.490 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 14/164 2024-09-12T03:14:34.517 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-12T03:14:34.543 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-12T03:14:34.753 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-12T03:14:34.957 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-12T03:14:35.074 INFO:teuthology.orchestra.run.smithi027.stdout: Upgrading : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 17/164 2024-09-12T03:14:35.178 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-12T03:14:35.250 INFO:teuthology.orchestra.run.smithi027.stdout: Upgrading : NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-12T03:14:35.326 INFO:teuthology.orchestra.run.smithi045.stdout: Preparing : 1/1 2024-09-12T03:14:35.474 INFO:teuthology.orchestra.run.smithi045.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-12T03:14:35.491 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-12T03:14:35.515 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-12T03:14:35.572 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-12T03:14:35.617 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-12T03:14:35.663 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-12T03:14:35.714 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-12T03:14:35.747 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-12T03:14:35.782 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-12T03:14:35.852 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-12T03:14:35.857 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-12T03:14:35.909 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-12T03:14:35.911 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-12T03:14:35.957 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-12T03:14:36.009 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-12T03:14:36.042 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-12T03:14:36.069 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-12T03:14:36.140 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-12T03:14:36.201 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-12T03:14:36.230 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-12T03:14:36.260 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-12T03:14:36.274 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-12T03:14:36.352 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-12T03:14:36.364 INFO:teuthology.orchestra.run.smithi045.stdout: Upgrading : librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-12T03:14:36.393 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x 34/164 2024-09-12T03:14:36.438 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9 35/164 2024-09-12T03:14:36.465 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 36/164 2024-09-12T03:14:36.490 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-12T03:14:36.538 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-12T03:14:36.580 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-12T03:14:36.640 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-12T03:14:36.660 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-12T03:14:36.734 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-12T03:14:36.762 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 10/164 2024-09-12T03:14:36.804 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-12T03:14:36.877 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-12T03:14:36.943 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-12T03:14:37.001 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-12T03:14:37.028 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-12T03:14:37.117 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-12T03:14:37.138 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 14/164 2024-09-12T03:14:37.201 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 14/164 2024-09-12T03:14:37.231 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-12T03:14:37.252 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-12T03:14:37.430 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-12T03:14:37.451 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-12T03:14:37.523 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-12T03:14:37.565 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-12T03:14:37.608 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-12T03:14:37.624 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-12T03:14:37.646 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 46/164 2024-09-12T03:14:37.689 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-12T03:14:37.746 INFO:teuthology.orchestra.run.smithi045.stdout: Upgrading : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 17/164 2024-09-12T03:14:37.856 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-12T03:14:37.864 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-12T03:14:37.907 INFO:teuthology.orchestra.run.smithi027.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-12T03:14:37.931 INFO:teuthology.orchestra.run.smithi045.stdout: Upgrading : NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-12T03:14:37.964 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-12T03:14:38.038 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-12T03:14:38.083 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-12T03:14:38.128 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-12T03:14:38.169 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-12T03:14:38.175 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-12T03:14:38.210 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-12T03:14:38.254 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-12T03:14:38.274 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-12T03:14:38.296 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-12T03:14:38.370 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-12T03:14:38.411 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-12T03:14:38.439 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-12T03:14:38.447 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-12T03:14:38.523 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-12T03:14:38.582 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-12T03:14:38.621 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-12T03:14:38.665 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-12T03:14:38.697 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-12T03:14:38.725 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-12T03:14:38.757 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-12T03:14:38.806 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-12T03:14:38.817 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-12T03:14:38.858 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-12T03:14:38.895 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-12T03:14:38.927 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-12T03:14:38.939 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-12T03:14:38.973 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-12T03:14:39.013 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-12T03:14:39.017 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-12T03:14:39.046 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 64/164 2024-09-12T03:14:39.049 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x 34/164 2024-09-12T03:14:39.071 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-12T03:14:39.093 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9 35/164 2024-09-12T03:14:39.121 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 36/164 2024-09-12T03:14:39.174 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-12T03:14:39.197 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-12T03:14:39.314 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-12T03:14:39.356 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-12T03:14:39.385 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-12T03:14:39.393 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-12T03:14:39.423 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-12T03:14:39.467 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-12T03:14:39.473 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-12T03:14:39.507 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-12T03:14:39.544 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-12T03:14:39.587 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-12T03:14:39.621 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-12T03:14:39.629 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-12T03:14:39.671 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-12T03:14:39.682 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-12T03:14:39.705 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-12T03:14:39.795 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-12T03:14:39.845 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-12T03:14:39.875 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-12T03:14:39.991 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-12T03:14:40.052 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-12T03:14:40.249 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-12T03:14:40.347 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-12T03:14:40.380 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-12T03:14:40.423 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-12T03:14:40.462 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 46/164 2024-09-12T03:14:40.505 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-12T03:14:40.753 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-12T03:14:40.816 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-12T03:14:40.823 INFO:teuthology.orchestra.run.smithi045.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-12T03:14:40.857 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-12T03:14:40.880 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-12T03:14:40.900 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-12T03:14:40.963 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-12T03:14:40.999 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-12T03:14:41.003 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-12T03:14:41.036 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-12T03:14:41.064 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-12T03:14:41.118 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-12T03:14:41.207 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-12T03:14:41.244 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-12T03:14:41.252 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-12T03:14:41.286 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-12T03:14:41.332 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-12T03:14:41.372 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-12T03:14:41.435 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-12T03:14:41.467 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-12T03:14:41.518 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-12T03:14:41.554 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-12T03:14:41.564 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-12T03:14:41.606 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-12T03:14:41.715 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-12T03:14:41.840 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-12T03:14:41.880 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-12T03:14:41.913 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 64/164 2024-09-12T03:14:41.937 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-12T03:14:42.041 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-12T03:14:42.171 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-12T03:14:42.212 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-12T03:14:42.249 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-12T03:14:42.251 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-12T03:14:42.289 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-12T03:14:42.331 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-12T03:14:42.370 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-12T03:14:42.371 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-12T03:14:42.404 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-12T03:14:42.438 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-12T03:14:42.472 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-12T03:14:42.514 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-12T03:14:42.548 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-12T03:14:42.649 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-12T03:14:42.696 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-12T03:14:42.726 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-12T03:14:42.836 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-12T03:14:42.896 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-12T03:14:43.682 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-12T03:14:43.686 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-12T03:14:43.725 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-12T03:14:43.759 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-12T03:14:43.771 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-12T03:14:43.799 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-12T03:14:43.832 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-12T03:14:43.865 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-12T03:14:43.915 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-12T03:14:43.961 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-12T03:14:44.052 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-12T03:14:44.087 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-12T03:14:44.158 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-12T03:14:44.212 INFO:teuthology.orchestra.run.smithi027.stdout: Upgrading : librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 101/164 2024-09-12T03:14:44.279 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-12T03:14:44.309 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-12T03:14:44.311 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 101/164 2024-09-12T03:14:44.342 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 102/164 2024-09-12T03:14:44.378 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-12T03:14:44.390 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-12T03:14:44.883 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-12T03:14:44.915 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 105/164 2024-09-12T03:14:44.986 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 105/164 2024-09-12T03:14:45.016 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 106/164 2024-09-12T03:14:45.109 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-12T03:14:45.229 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-12T03:14:46.600 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-12T03:14:46.750 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-12T03:14:46.834 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-12T03:14:46.867 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-12T03:14:47.193 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-12T03:14:47.238 INFO:teuthology.orchestra.run.smithi045.stdout: Upgrading : librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 101/164 2024-09-12T03:14:47.338 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 101/164 2024-09-12T03:14:47.368 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 102/164 2024-09-12T03:14:47.404 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-12T03:14:47.478 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-12T03:14:47.515 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-12T03:14:47.910 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-12T03:14:47.949 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 105/164 2024-09-12T03:14:48.020 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 105/164 2024-09-12T03:14:48.050 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 106/164 2024-09-12T03:14:48.096 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-12T03:14:48.135 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 108/164 2024-09-12T03:14:48.232 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 108/164 2024-09-12T03:14:48.232 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-12T03:14:48.232 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-12T03:14:48.232 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:14:48.270 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 109/164 2024-09-12T03:14:50.488 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-12T03:14:50.532 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-12T03:14:51.132 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-12T03:14:51.170 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 108/164 2024-09-12T03:14:51.268 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 108/164 2024-09-12T03:14:51.269 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-12T03:14:51.269 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-12T03:14:51.269 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-12T03:14:51.312 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 109/164 2024-09-12T03:15:00.755 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 109/164 2024-09-12T03:15:00.755 INFO:teuthology.orchestra.run.smithi027.stdout:skipping the directory /sys 2024-09-12T03:15:00.755 INFO:teuthology.orchestra.run.smithi027.stdout:skipping the directory /proc 2024-09-12T03:15:00.755 INFO:teuthology.orchestra.run.smithi027.stdout:skipping the directory /mnt 2024-09-12T03:15:00.755 INFO:teuthology.orchestra.run.smithi027.stdout:skipping the directory /var/tmp 2024-09-12T03:15:00.755 INFO:teuthology.orchestra.run.smithi027.stdout:skipping the directory /home 2024-09-12T03:15:00.755 INFO:teuthology.orchestra.run.smithi027.stdout:skipping the directory /root 2024-09-12T03:15:00.756 INFO:teuthology.orchestra.run.smithi027.stdout:skipping the directory /tmp 2024-09-12T03:15:00.756 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:15:00.932 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 110/164 2024-09-12T03:15:00.949 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 110/164 2024-09-12T03:15:00.949 INFO:teuthology.orchestra.run.smithi027.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T03:15:00.949 INFO:teuthology.orchestra.run.smithi027.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-12T03:15:00.949 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:15:02.484 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 111/164 2024-09-12T03:15:02.513 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 111/164 2024-09-12T03:15:02.513 INFO:teuthology.orchestra.run.smithi027.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T03:15:02.513 INFO:teuthology.orchestra.run.smithi027.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-12T03:15:02.513 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-12T03:15:02.514 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-12T03:15:02.514 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:15:02.753 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 112/164 2024-09-12T03:15:02.777 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 112/164 2024-09-12T03:15:02.777 INFO:teuthology.orchestra.run.smithi027.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T03:15:02.777 INFO:teuthology.orchestra.run.smithi027.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-12T03:15:02.777 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-12T03:15:02.777 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-12T03:15:02.777 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:15:03.236 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 113/164 2024-09-12T03:15:03.261 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 113/164 2024-09-12T03:15:03.261 INFO:teuthology.orchestra.run.smithi027.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T03:15:03.261 INFO:teuthology.orchestra.run.smithi027.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-12T03:15:03.261 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-12T03:15:03.261 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-12T03:15:03.261 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:15:03.367 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-12T03:15:03.424 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-12T03:15:03.488 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-12T03:15:03.690 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-12T03:15:03.737 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-12T03:15:03.772 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 109/164 2024-09-12T03:15:03.772 INFO:teuthology.orchestra.run.smithi045.stdout:skipping the directory /sys 2024-09-12T03:15:03.772 INFO:teuthology.orchestra.run.smithi045.stdout:skipping the directory /proc 2024-09-12T03:15:03.772 INFO:teuthology.orchestra.run.smithi045.stdout:skipping the directory /mnt 2024-09-12T03:15:03.772 INFO:teuthology.orchestra.run.smithi045.stdout:skipping the directory /var/tmp 2024-09-12T03:15:03.772 INFO:teuthology.orchestra.run.smithi045.stdout:skipping the directory /home 2024-09-12T03:15:03.772 INFO:teuthology.orchestra.run.smithi045.stdout:skipping the directory /root 2024-09-12T03:15:03.772 INFO:teuthology.orchestra.run.smithi045.stdout:skipping the directory /tmp 2024-09-12T03:15:03.772 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-12T03:15:03.817 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-12T03:15:03.907 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-12T03:15:03.960 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 110/164 2024-09-12T03:15:03.981 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 110/164 2024-09-12T03:15:03.981 INFO:teuthology.orchestra.run.smithi045.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T03:15:03.981 INFO:teuthology.orchestra.run.smithi045.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-12T03:15:03.981 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-12T03:15:04.057 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-12T03:15:04.106 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-12T03:15:04.152 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-12T03:15:04.180 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-12T03:15:04.598 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 125/164 2024-09-12T03:15:04.629 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 125/164 2024-09-12T03:15:04.660 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el 126/164 2024-09-12T03:15:04.727 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.e 127/164 2024-09-12T03:15:04.760 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 128/164 2024-09-12T03:15:05.506 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 111/164 2024-09-12T03:15:05.534 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 111/164 2024-09-12T03:15:05.534 INFO:teuthology.orchestra.run.smithi045.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T03:15:05.535 INFO:teuthology.orchestra.run.smithi045.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-12T03:15:05.535 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-12T03:15:05.535 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-12T03:15:05.535 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-12T03:15:05.781 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 112/164 2024-09-12T03:15:05.807 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 112/164 2024-09-12T03:15:05.807 INFO:teuthology.orchestra.run.smithi045.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T03:15:05.807 INFO:teuthology.orchestra.run.smithi045.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-12T03:15:05.807 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-12T03:15:05.807 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-12T03:15:05.807 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-12T03:15:05.864 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 128/164 2024-09-12T03:15:05.904 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 129/164 2024-09-12T03:15:06.265 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 113/164 2024-09-12T03:15:06.290 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 113/164 2024-09-12T03:15:06.290 INFO:teuthology.orchestra.run.smithi045.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T03:15:06.291 INFO:teuthology.orchestra.run.smithi045.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-12T03:15:06.291 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-12T03:15:06.291 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-12T03:15:06.291 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-12T03:15:06.414 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-12T03:15:06.472 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-12T03:15:06.536 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-12T03:15:06.733 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-12T03:15:06.815 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-12T03:15:06.899 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-12T03:15:06.923 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 129/164 2024-09-12T03:15:06.954 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 130/164 2024-09-12T03:15:06.999 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-12T03:15:07.062 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 130/164 2024-09-12T03:15:07.143 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-12T03:15:07.197 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-12T03:15:07.232 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9 131/164 2024-09-12T03:15:07.250 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-12T03:15:07.264 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 132/164 2024-09-12T03:15:07.285 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-12T03:15:07.290 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 132/164 2024-09-12T03:15:07.290 INFO:teuthology.orchestra.run.smithi027.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T03:15:07.290 INFO:teuthology.orchestra.run.smithi027.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-12T03:15:07.290 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-12T03:15:07.290 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-12T03:15:07.290 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:15:07.340 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 133/164 2024-09-12T03:15:07.352 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 133/164 2024-09-12T03:15:07.453 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 134/164 2024-09-12T03:15:07.516 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 135/164 2024-09-12T03:15:07.747 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 125/164 2024-09-12T03:15:07.776 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 125/164 2024-09-12T03:15:07.808 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el 126/164 2024-09-12T03:15:07.874 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.e 127/164 2024-09-12T03:15:07.899 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 128/164 2024-09-12T03:15:08.499 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-12T03:15:08.534 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 137/164 2024-09-12T03:15:08.559 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 137/164 2024-09-12T03:15:08.560 INFO:teuthology.orchestra.run.smithi027.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T03:15:08.560 INFO:teuthology.orchestra.run.smithi027.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-12T03:15:08.560 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-12T03:15:08.560 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-12T03:15:08.560 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:15:08.601 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 138/164 2024-09-12T03:15:08.626 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 138/164 2024-09-12T03:15:08.626 INFO:teuthology.orchestra.run.smithi027.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T03:15:08.626 INFO:teuthology.orchestra.run.smithi027.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-12T03:15:08.626 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:15:08.677 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 139/164 2024-09-12T03:15:08.701 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 139/164 2024-09-12T03:15:08.701 INFO:teuthology.orchestra.run.smithi027.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T03:15:08.701 INFO:teuthology.orchestra.run.smithi027.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-12T03:15:08.702 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-12T03:15:08.702 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-12T03:15:08.702 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:15:09.013 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 128/164 2024-09-12T03:15:09.019 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 140/164 2024-09-12T03:15:09.044 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 140/164 2024-09-12T03:15:09.044 INFO:teuthology.orchestra.run.smithi027.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T03:15:09.044 INFO:teuthology.orchestra.run.smithi027.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-12T03:15:09.044 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-12T03:15:09.044 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-12T03:15:09.045 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:15:09.051 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 129/164 2024-09-12T03:15:10.069 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 129/164 2024-09-12T03:15:10.102 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 130/164 2024-09-12T03:15:10.213 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 130/164 2024-09-12T03:15:10.388 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9 131/164 2024-09-12T03:15:10.420 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 132/164 2024-09-12T03:15:10.446 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 132/164 2024-09-12T03:15:10.446 INFO:teuthology.orchestra.run.smithi045.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T03:15:10.446 INFO:teuthology.orchestra.run.smithi045.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-12T03:15:10.446 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-12T03:15:10.446 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-12T03:15:10.446 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-12T03:15:10.496 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 133/164 2024-09-12T03:15:10.508 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 133/164 2024-09-12T03:15:10.610 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 134/164 2024-09-12T03:15:10.666 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 135/164 2024-09-12T03:15:11.639 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-12T03:15:11.665 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 137/164 2024-09-12T03:15:11.691 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 137/164 2024-09-12T03:15:11.691 INFO:teuthology.orchestra.run.smithi045.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T03:15:11.691 INFO:teuthology.orchestra.run.smithi045.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-12T03:15:11.692 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-12T03:15:11.692 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-12T03:15:11.692 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-12T03:15:11.733 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 138/164 2024-09-12T03:15:11.757 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 138/164 2024-09-12T03:15:11.758 INFO:teuthology.orchestra.run.smithi045.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T03:15:11.758 INFO:teuthology.orchestra.run.smithi045.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-12T03:15:11.758 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-12T03:15:11.825 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 139/164 2024-09-12T03:15:11.850 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 139/164 2024-09-12T03:15:11.851 INFO:teuthology.orchestra.run.smithi045.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T03:15:11.851 INFO:teuthology.orchestra.run.smithi045.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-12T03:15:11.851 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-12T03:15:11.851 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-12T03:15:11.851 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-12T03:15:12.184 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 140/164 2024-09-12T03:15:12.210 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 140/164 2024-09-12T03:15:12.210 INFO:teuthology.orchestra.run.smithi045.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T03:15:12.210 INFO:teuthology.orchestra.run.smithi045.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-12T03:15:12.210 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-12T03:15:12.210 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-12T03:15:12.210 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-12T03:15:13.874 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 141/164 2024-09-12T03:15:13.915 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 142/164 2024-09-12T03:15:13.964 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 143/164 2024-09-12T03:15:14.004 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-12T03:15:14.040 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-12T03:15:14.249 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 146/164 2024-09-12T03:15:14.295 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-12T03:15:14.325 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_6 148/164 2024-09-12T03:15:14.355 INFO:teuthology.orchestra.run.smithi027.stdout: Upgrading : NetworkManager-team-1:1.49.90-1.el9.x86_64 149/164 2024-09-12T03:15:14.409 INFO:teuthology.orchestra.run.smithi027.stdout: Upgrading : NetworkManager-tui-1:1.49.90-1.el9.x86_64 150/164 2024-09-12T03:15:15.071 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-12T03:15:15.108 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-12T03:15:15.130 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-12T03:15:16.046 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 152/164 2024-09-12T03:15:16.137 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-12T03:15:16.185 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-12T03:15:16.276 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-12T03:15:16.307 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-12T03:15:16.333 INFO:teuthology.orchestra.run.smithi027.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-12T03:15:16.375 INFO:teuthology.orchestra.run.smithi027.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-12T03:15:16.375 INFO:teuthology.orchestra.run.smithi027.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-12T03:15:16.451 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-12T03:15:16.451 INFO:teuthology.orchestra.run.smithi027.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-12T03:15:16.527 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-12T03:15:16.552 INFO:teuthology.orchestra.run.smithi027.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-12T03:15:16.558 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-12T03:15:16.558 INFO:teuthology.orchestra.run.smithi027.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-12T03:15:16.746 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-12T03:15:16.746 INFO:teuthology.orchestra.run.smithi027.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-12T03:15:16.830 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-12T03:15:16.907 INFO:teuthology.orchestra.run.smithi027.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-12T03:15:17.056 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 141/164 2024-09-12T03:15:17.098 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 142/164 2024-09-12T03:15:17.139 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 143/164 2024-09-12T03:15:17.170 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-12T03:15:17.198 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-12T03:15:17.413 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 146/164 2024-09-12T03:15:17.461 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-12T03:15:17.491 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_6 148/164 2024-09-12T03:15:17.521 INFO:teuthology.orchestra.run.smithi045.stdout: Upgrading : NetworkManager-team-1:1.49.90-1.el9.x86_64 149/164 2024-09-12T03:15:17.575 INFO:teuthology.orchestra.run.smithi045.stdout: Upgrading : NetworkManager-tui-1:1.49.90-1.el9.x86_64 150/164 2024-09-12T03:15:18.250 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-12T03:15:18.312 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-12T03:15:18.334 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-12T03:15:19.263 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 152/164 2024-09-12T03:15:19.355 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-12T03:15:19.402 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-12T03:15:19.500 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-12T03:15:19.533 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-12T03:15:19.559 INFO:teuthology.orchestra.run.smithi045.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-12T03:15:19.593 INFO:teuthology.orchestra.run.smithi045.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-12T03:15:19.593 INFO:teuthology.orchestra.run.smithi045.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-12T03:15:19.686 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-12T03:15:19.686 INFO:teuthology.orchestra.run.smithi045.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-12T03:15:19.779 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-12T03:15:19.803 INFO:teuthology.orchestra.run.smithi045.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-12T03:15:19.810 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-12T03:15:19.810 INFO:teuthology.orchestra.run.smithi045.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-12T03:15:19.997 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-12T03:15:19.997 INFO:teuthology.orchestra.run.smithi045.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-12T03:15:20.073 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-12T03:15:20.141 INFO:teuthology.orchestra.run.smithi045.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-12T03:15:22.540 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-12T03:15:22.540 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/164 2024-09-12T03:15:22.540 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 2/164 2024-09-12T03:15:22.540 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 3/164 2024-09-12T03:15:22.540 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 4/164 2024-09-12T03:15:22.541 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 5/164 2024-09-12T03:15:22.541 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 6/164 2024-09-12T03:15:22.541 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 7/164 2024-09-12T03:15:22.541 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-12T03:15:22.541 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-12T03:15:22.541 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 10/164 2024-09-12T03:15:22.541 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 11/164 2024-09-12T03:15:22.541 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 12/164 2024-09-12T03:15:22.541 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-12T03:15:22.541 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 14/164 2024-09-12T03:15:22.541 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_6 15/164 2024-09-12T03:15:22.541 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 16/164 2024-09-12T03:15:22.541 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 17/164 2024-09-12T03:15:22.542 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 18/164 2024-09-12T03:15:22.542 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 19/164 2024-09-12T03:15:22.542 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 20/164 2024-09-12T03:15:22.542 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9 21/164 2024-09-12T03:15:22.542 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x 22/164 2024-09-12T03:15:22.542 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 23/164 2024-09-12T03:15:22.542 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 24/164 2024-09-12T03:15:22.542 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 25/164 2024-09-12T03:15:22.542 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 26/164 2024-09-12T03:15:22.542 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 27/164 2024-09-12T03:15:22.542 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 28/164 2024-09-12T03:15:22.542 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 29/164 2024-09-12T03:15:22.542 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.e 30/164 2024-09-12T03:15:22.543 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 31/164 2024-09-12T03:15:22.543 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 32/164 2024-09-12T03:15:22.543 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 33/164 2024-09-12T03:15:22.543 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9 34/164 2024-09-12T03:15:22.543 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 35/164 2024-09-12T03:15:22.543 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el 36/164 2024-09-12T03:15:22.543 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 37/164 2024-09-12T03:15:22.543 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 38/164 2024-09-12T03:15:22.543 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 39/164 2024-09-12T03:15:22.543 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-12T03:15:22.543 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-12T03:15:22.543 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-12T03:15:22.543 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-12T03:15:22.544 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-12T03:15:22.544 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-12T03:15:22.544 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-12T03:15:22.544 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-12T03:15:22.544 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-12T03:15:22.544 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-12T03:15:22.544 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-12T03:15:22.544 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-12T03:15:22.544 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-12T03:15:22.544 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-12T03:15:22.544 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-12T03:15:22.544 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-12T03:15:22.544 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-12T03:15:22.545 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-12T03:15:22.545 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-12T03:15:22.545 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-12T03:15:22.545 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-12T03:15:22.545 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-12T03:15:22.545 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-12T03:15:22.545 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-12T03:15:22.545 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-12T03:15:22.545 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-12T03:15:22.545 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-12T03:15:22.545 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-12T03:15:22.545 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-12T03:15:22.545 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-12T03:15:22.545 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-12T03:15:22.546 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-12T03:15:22.546 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-12T03:15:22.546 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-12T03:15:22.546 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-12T03:15:22.546 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-12T03:15:22.546 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-12T03:15:22.546 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-12T03:15:22.546 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-12T03:15:22.546 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-12T03:15:22.546 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-12T03:15:22.546 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-12T03:15:22.546 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-12T03:15:22.546 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-12T03:15:22.547 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-12T03:15:22.547 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-12T03:15:22.547 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-12T03:15:22.547 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-12T03:15:22.547 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-12T03:15:22.547 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-12T03:15:22.547 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-12T03:15:22.547 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-12T03:15:22.547 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-12T03:15:22.547 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-12T03:15:22.547 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-12T03:15:22.547 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-12T03:15:22.547 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-12T03:15:22.547 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-12T03:15:22.548 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-12T03:15:22.548 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-12T03:15:22.548 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-12T03:15:22.548 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-12T03:15:22.548 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-12T03:15:22.548 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-12T03:15:22.548 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-12T03:15:22.548 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-12T03:15:22.548 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-12T03:15:22.548 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 107/164 2024-09-12T03:15:22.548 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-12T03:15:22.548 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-12T03:15:22.548 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-12T03:15:22.549 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-12T03:15:22.549 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-12T03:15:22.549 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-12T03:15:22.549 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-12T03:15:22.549 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-12T03:15:22.549 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-12T03:15:22.549 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-12T03:15:22.549 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-12T03:15:22.549 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-12T03:15:22.549 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-12T03:15:22.549 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-12T03:15:22.549 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-12T03:15:22.549 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-12T03:15:22.550 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-12T03:15:22.550 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-12T03:15:22.550 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-12T03:15:22.550 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-12T03:15:22.550 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-12T03:15:22.550 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-12T03:15:22.550 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-12T03:15:22.550 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-12T03:15:22.550 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-12T03:15:22.550 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-12T03:15:22.550 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-12T03:15:22.550 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-12T03:15:22.550 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-12T03:15:22.550 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-12T03:15:22.551 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-12T03:15:22.551 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-12T03:15:22.551 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-12T03:15:22.551 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-12T03:15:22.551 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-12T03:15:22.551 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-12T03:15:22.551 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-12T03:15:22.551 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-12T03:15:22.551 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-12T03:15:22.551 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-12T03:15:22.551 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-12T03:15:22.551 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 149/164 2024-09-12T03:15:22.551 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-12T03:15:22.551 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 151/164 2024-09-12T03:15:22.552 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-12T03:15:22.552 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : NetworkManager-1:1.49.90-1.el9.x86_64 153/164 2024-09-12T03:15:22.552 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-12T03:15:22.552 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 155/164 2024-09-12T03:15:22.552 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-12T03:15:22.552 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : NetworkManager-team-1:1.49.90-1.el9.x86_64 157/164 2024-09-12T03:15:22.552 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-12T03:15:22.552 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : NetworkManager-tui-1:1.49.90-1.el9.x86_64 159/164 2024-09-12T03:15:22.552 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-12T03:15:22.552 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-12T03:15:22.552 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-12T03:15:22.552 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-12T03:15:25.539 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-12T03:15:25.540 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:15:25.540 INFO:teuthology.orchestra.run.smithi027.stdout:Upgraded: 2024-09-12T03:15:25.540 INFO:teuthology.orchestra.run.smithi027.stdout: NetworkManager-1:1.49.90-1.el9.x86_64 2024-09-12T03:15:25.540 INFO:teuthology.orchestra.run.smithi027.stdout: NetworkManager-libnm-1:1.49.90-1.el9.x86_64 2024-09-12T03:15:25.540 INFO:teuthology.orchestra.run.smithi027.stdout: NetworkManager-team-1:1.49.90-1.el9.x86_64 2024-09-12T03:15:25.540 INFO:teuthology.orchestra.run.smithi027.stdout: NetworkManager-tui-1:1.49.90-1.el9.x86_64 2024-09-12T03:15:25.540 INFO:teuthology.orchestra.run.smithi027.stdout: librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:15:25.540 INFO:teuthology.orchestra.run.smithi027.stdout: librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:15:25.540 INFO:teuthology.orchestra.run.smithi027.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-12T03:15:25.540 INFO:teuthology.orchestra.run.smithi027.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-12T03:15:25.541 INFO:teuthology.orchestra.run.smithi027.stdout:Installed: 2024-09-12T03:15:25.541 INFO:teuthology.orchestra.run.smithi027.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-12T03:15:25.541 INFO:teuthology.orchestra.run.smithi027.stdout: automake-1.16.2-8.el9.noarch 2024-09-12T03:15:25.541 INFO:teuthology.orchestra.run.smithi027.stdout: bison-3.7.4-5.el9.x86_64 2024-09-12T03:15:25.541 INFO:teuthology.orchestra.run.smithi027.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-12T03:15:25.541 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:15:25.541 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:15:25.541 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:15:25.541 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:15:25.541 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T03:15:25.541 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-immutable-object-cache-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:15:25.541 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:15:25.542 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:15:25.542 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T03:15:25.542 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T03:15:25.542 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T03:15:25.542 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T03:15:25.542 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T03:15:25.542 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:15:25.542 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:15:25.542 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T03:15:25.542 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:15:25.542 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:15:25.542 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:15:25.542 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T03:15:25.543 INFO:teuthology.orchestra.run.smithi027.stdout: cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T03:15:25.543 INFO:teuthology.orchestra.run.smithi027.stdout: cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:15:25.543 INFO:teuthology.orchestra.run.smithi027.stdout: cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:15:25.543 INFO:teuthology.orchestra.run.smithi027.stdout: cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T03:15:25.543 INFO:teuthology.orchestra.run.smithi027.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-12T03:15:25.543 INFO:teuthology.orchestra.run.smithi027.stdout: flex-2.6.4-9.el9.x86_64 2024-09-12T03:15:25.543 INFO:teuthology.orchestra.run.smithi027.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-12T03:15:25.543 INFO:teuthology.orchestra.run.smithi027.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-12T03:15:25.543 INFO:teuthology.orchestra.run.smithi027.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-12T03:15:25.543 INFO:teuthology.orchestra.run.smithi027.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-12T03:15:25.543 INFO:teuthology.orchestra.run.smithi027.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-12T03:15:25.543 INFO:teuthology.orchestra.run.smithi027.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-12T03:15:25.544 INFO:teuthology.orchestra.run.smithi027.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-12T03:15:25.544 INFO:teuthology.orchestra.run.smithi027.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-12T03:15:25.544 INFO:teuthology.orchestra.run.smithi027.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-12T03:15:25.544 INFO:teuthology.orchestra.run.smithi027.stdout: libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:15:25.544 INFO:teuthology.orchestra.run.smithi027.stdout: libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:15:25.544 INFO:teuthology.orchestra.run.smithi027.stdout: libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:15:25.544 INFO:teuthology.orchestra.run.smithi027.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-12T03:15:25.544 INFO:teuthology.orchestra.run.smithi027.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-12T03:15:25.544 INFO:teuthology.orchestra.run.smithi027.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-12T03:15:25.544 INFO:teuthology.orchestra.run.smithi027.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-12T03:15:25.544 INFO:teuthology.orchestra.run.smithi027.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-12T03:15:25.544 INFO:teuthology.orchestra.run.smithi027.stdout: librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:15:25.545 INFO:teuthology.orchestra.run.smithi027.stdout: libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:15:25.545 INFO:teuthology.orchestra.run.smithi027.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-12T03:15:25.545 INFO:teuthology.orchestra.run.smithi027.stdout: librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:15:25.545 INFO:teuthology.orchestra.run.smithi027.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-12T03:15:25.545 INFO:teuthology.orchestra.run.smithi027.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-12T03:15:25.545 INFO:teuthology.orchestra.run.smithi027.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-12T03:15:25.545 INFO:teuthology.orchestra.run.smithi027.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-12T03:15:25.545 INFO:teuthology.orchestra.run.smithi027.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-12T03:15:25.545 INFO:teuthology.orchestra.run.smithi027.stdout: lua-5.4.4-4.el9.x86_64 2024-09-12T03:15:25.545 INFO:teuthology.orchestra.run.smithi027.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-12T03:15:25.545 INFO:teuthology.orchestra.run.smithi027.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-12T03:15:25.545 INFO:teuthology.orchestra.run.smithi027.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-12T03:15:25.546 INFO:teuthology.orchestra.run.smithi027.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-12T03:15:25.546 INFO:teuthology.orchestra.run.smithi027.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-12T03:15:25.546 INFO:teuthology.orchestra.run.smithi027.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-12T03:15:25.546 INFO:teuthology.orchestra.run.smithi027.stdout: postgresql-13.16-1.el9.x86_64 2024-09-12T03:15:25.546 INFO:teuthology.orchestra.run.smithi027.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-12T03:15:25.546 INFO:teuthology.orchestra.run.smithi027.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-12T03:15:25.546 INFO:teuthology.orchestra.run.smithi027.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-12T03:15:25.546 INFO:teuthology.orchestra.run.smithi027.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-12T03:15:25.546 INFO:teuthology.orchestra.run.smithi027.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-12T03:15:25.546 INFO:teuthology.orchestra.run.smithi027.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-12T03:15:25.546 INFO:teuthology.orchestra.run.smithi027.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-12T03:15:25.547 INFO:teuthology.orchestra.run.smithi027.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-12T03:15:25.547 INFO:teuthology.orchestra.run.smithi027.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-12T03:15:25.547 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-12T03:15:25.547 INFO:teuthology.orchestra.run.smithi027.stdout: python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:15:25.547 INFO:teuthology.orchestra.run.smithi027.stdout: python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:15:25.547 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:15:25.547 INFO:teuthology.orchestra.run.smithi027.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-12T03:15:25.547 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-12T03:15:25.547 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-12T03:15:25.547 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-12T03:15:25.547 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-12T03:15:25.547 INFO:teuthology.orchestra.run.smithi027.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-12T03:15:25.547 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-12T03:15:25.548 INFO:teuthology.orchestra.run.smithi027.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-12T03:15:25.548 INFO:teuthology.orchestra.run.smithi027.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-12T03:15:25.548 INFO:teuthology.orchestra.run.smithi027.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-12T03:15:25.548 INFO:teuthology.orchestra.run.smithi027.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-12T03:15:25.548 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-12T03:15:25.548 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-12T03:15:25.548 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-12T03:15:25.548 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-12T03:15:25.548 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-12T03:15:25.548 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-12T03:15:25.548 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-12T03:15:25.548 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-12T03:15:25.549 INFO:teuthology.orchestra.run.smithi027.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-12T03:15:25.549 INFO:teuthology.orchestra.run.smithi027.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-12T03:15:25.549 INFO:teuthology.orchestra.run.smithi027.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-12T03:15:25.549 INFO:teuthology.orchestra.run.smithi027.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-12T03:15:25.549 INFO:teuthology.orchestra.run.smithi027.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-12T03:15:25.549 INFO:teuthology.orchestra.run.smithi027.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-12T03:15:25.549 INFO:teuthology.orchestra.run.smithi027.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-12T03:15:25.549 INFO:teuthology.orchestra.run.smithi027.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-12T03:15:25.549 INFO:teuthology.orchestra.run.smithi027.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-12T03:15:25.549 INFO:teuthology.orchestra.run.smithi027.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-12T03:15:25.549 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-12T03:15:25.549 INFO:teuthology.orchestra.run.smithi027.stdout: python3-ply-3.11-14.el9.noarch 2024-09-12T03:15:25.549 INFO:teuthology.orchestra.run.smithi027.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-12T03:15:25.549 INFO:teuthology.orchestra.run.smithi027.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-12T03:15:25.550 INFO:teuthology.orchestra.run.smithi027.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-12T03:15:25.550 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-12T03:15:25.550 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-12T03:15:25.550 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-12T03:15:25.550 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-12T03:15:25.550 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-12T03:15:25.550 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-12T03:15:25.550 INFO:teuthology.orchestra.run.smithi027.stdout: python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:15:25.550 INFO:teuthology.orchestra.run.smithi027.stdout: python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:15:25.550 INFO:teuthology.orchestra.run.smithi027.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-12T03:15:25.550 INFO:teuthology.orchestra.run.smithi027.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-12T03:15:25.550 INFO:teuthology.orchestra.run.smithi027.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-12T03:15:25.550 INFO:teuthology.orchestra.run.smithi027.stdout: python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:15:25.550 INFO:teuthology.orchestra.run.smithi027.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-12T03:15:25.551 INFO:teuthology.orchestra.run.smithi027.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-12T03:15:25.551 INFO:teuthology.orchestra.run.smithi027.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-12T03:15:25.551 INFO:teuthology.orchestra.run.smithi027.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-12T03:15:25.551 INFO:teuthology.orchestra.run.smithi027.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-12T03:15:25.551 INFO:teuthology.orchestra.run.smithi027.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-12T03:15:25.551 INFO:teuthology.orchestra.run.smithi027.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-12T03:15:25.551 INFO:teuthology.orchestra.run.smithi027.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-12T03:15:25.551 INFO:teuthology.orchestra.run.smithi027.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-12T03:15:25.551 INFO:teuthology.orchestra.run.smithi027.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-12T03:15:25.551 INFO:teuthology.orchestra.run.smithi027.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-12T03:15:25.551 INFO:teuthology.orchestra.run.smithi027.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-12T03:15:25.551 INFO:teuthology.orchestra.run.smithi027.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-12T03:15:25.551 INFO:teuthology.orchestra.run.smithi027.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-12T03:15:25.551 INFO:teuthology.orchestra.run.smithi027.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-12T03:15:25.552 INFO:teuthology.orchestra.run.smithi027.stdout: rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:15:25.552 INFO:teuthology.orchestra.run.smithi027.stdout: rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:15:25.552 INFO:teuthology.orchestra.run.smithi027.stdout: rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:15:25.552 INFO:teuthology.orchestra.run.smithi027.stdout: re2-1:20211101-20.el9.x86_64 2024-09-12T03:15:25.552 INFO:teuthology.orchestra.run.smithi027.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-12T03:15:25.552 INFO:teuthology.orchestra.run.smithi027.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-12T03:15:25.552 INFO:teuthology.orchestra.run.smithi027.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-12T03:15:25.552 INFO:teuthology.orchestra.run.smithi027.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-12T03:15:25.552 INFO:teuthology.orchestra.run.smithi027.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-12T03:15:25.552 INFO:teuthology.orchestra.run.smithi027.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-12T03:15:25.553 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:15:25.553 INFO:teuthology.orchestra.run.smithi027.stdout:Complete! 2024-09-12T03:15:25.743 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-12T03:15:25.743 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/164 2024-09-12T03:15:25.743 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 2/164 2024-09-12T03:15:25.743 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 3/164 2024-09-12T03:15:25.743 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 4/164 2024-09-12T03:15:25.743 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 5/164 2024-09-12T03:15:25.744 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 6/164 2024-09-12T03:15:25.744 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 7/164 2024-09-12T03:15:25.744 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-12T03:15:25.744 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-12T03:15:25.744 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 10/164 2024-09-12T03:15:25.744 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 11/164 2024-09-12T03:15:25.744 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 12/164 2024-09-12T03:15:25.744 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-12T03:15:25.744 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 14/164 2024-09-12T03:15:25.744 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_6 15/164 2024-09-12T03:15:25.744 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 16/164 2024-09-12T03:15:25.745 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 17/164 2024-09-12T03:15:25.745 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 18/164 2024-09-12T03:15:25.745 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 19/164 2024-09-12T03:15:25.745 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 20/164 2024-09-12T03:15:25.745 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9 21/164 2024-09-12T03:15:25.745 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x 22/164 2024-09-12T03:15:25.745 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 23/164 2024-09-12T03:15:25.745 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 24/164 2024-09-12T03:15:25.745 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 25/164 2024-09-12T03:15:25.745 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 26/164 2024-09-12T03:15:25.745 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 27/164 2024-09-12T03:15:25.745 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 28/164 2024-09-12T03:15:25.745 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 29/164 2024-09-12T03:15:25.746 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.e 30/164 2024-09-12T03:15:25.746 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 31/164 2024-09-12T03:15:25.746 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 32/164 2024-09-12T03:15:25.759 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 33/164 2024-09-12T03:15:25.759 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9 34/164 2024-09-12T03:15:25.759 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 35/164 2024-09-12T03:15:25.759 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el 36/164 2024-09-12T03:15:25.759 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 37/164 2024-09-12T03:15:25.759 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 38/164 2024-09-12T03:15:25.759 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 39/164 2024-09-12T03:15:25.759 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-12T03:15:25.759 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-12T03:15:25.760 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-12T03:15:25.760 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-12T03:15:25.760 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-12T03:15:25.760 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-12T03:15:25.760 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-12T03:15:25.760 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-12T03:15:25.760 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-12T03:15:25.760 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-12T03:15:25.760 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-12T03:15:25.760 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-12T03:15:25.760 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-12T03:15:25.760 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-12T03:15:25.760 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-12T03:15:25.760 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-12T03:15:25.761 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-12T03:15:25.761 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-12T03:15:25.761 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-12T03:15:25.761 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-12T03:15:25.761 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-12T03:15:25.761 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-12T03:15:25.761 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-12T03:15:25.761 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-12T03:15:25.761 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-12T03:15:25.761 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-12T03:15:25.761 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-12T03:15:25.761 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-12T03:15:25.761 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-12T03:15:25.762 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-12T03:15:25.762 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-12T03:15:25.762 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-12T03:15:25.762 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-12T03:15:25.762 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-12T03:15:25.762 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-12T03:15:25.762 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-12T03:15:25.762 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-12T03:15:25.762 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-12T03:15:25.762 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-12T03:15:25.762 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-12T03:15:25.762 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-12T03:15:25.762 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-12T03:15:25.763 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-12T03:15:25.763 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-12T03:15:25.763 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-12T03:15:25.763 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-12T03:15:25.763 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-12T03:15:25.763 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-12T03:15:25.763 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-12T03:15:25.763 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-12T03:15:25.763 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-12T03:15:25.763 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-12T03:15:25.763 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-12T03:15:25.763 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-12T03:15:25.763 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-12T03:15:25.763 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-12T03:15:25.764 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-12T03:15:25.764 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-12T03:15:25.764 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-12T03:15:25.764 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-12T03:15:25.764 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-12T03:15:25.764 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-12T03:15:25.764 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-12T03:15:25.764 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-12T03:15:25.764 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-12T03:15:25.764 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-12T03:15:25.764 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-12T03:15:25.764 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 107/164 2024-09-12T03:15:25.764 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-12T03:15:25.764 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-12T03:15:25.765 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-12T03:15:25.765 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-12T03:15:25.765 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-12T03:15:25.765 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-12T03:15:25.765 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-12T03:15:25.765 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-12T03:15:25.765 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-12T03:15:25.765 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-12T03:15:25.765 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-12T03:15:25.765 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-12T03:15:25.765 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-12T03:15:25.765 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-12T03:15:25.765 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-12T03:15:25.765 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-12T03:15:25.766 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-12T03:15:25.766 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-12T03:15:25.766 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-12T03:15:25.766 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-12T03:15:25.766 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-12T03:15:25.766 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-12T03:15:25.766 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-12T03:15:25.766 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-12T03:15:25.766 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-12T03:15:25.766 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-12T03:15:25.766 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-12T03:15:25.766 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-12T03:15:25.766 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-12T03:15:25.766 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-12T03:15:25.767 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-12T03:15:25.767 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-12T03:15:25.767 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-12T03:15:25.767 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-12T03:15:25.767 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-12T03:15:25.767 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-12T03:15:25.767 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-12T03:15:25.767 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-12T03:15:25.767 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-12T03:15:25.767 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-12T03:15:25.767 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-12T03:15:25.767 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 149/164 2024-09-12T03:15:25.767 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-12T03:15:25.768 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 151/164 2024-09-12T03:15:25.768 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-12T03:15:25.768 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : NetworkManager-1:1.49.90-1.el9.x86_64 153/164 2024-09-12T03:15:25.768 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-12T03:15:25.768 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 155/164 2024-09-12T03:15:25.768 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-12T03:15:25.768 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : NetworkManager-team-1:1.49.90-1.el9.x86_64 157/164 2024-09-12T03:15:25.768 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-12T03:15:25.768 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : NetworkManager-tui-1:1.49.90-1.el9.x86_64 159/164 2024-09-12T03:15:25.768 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-12T03:15:25.768 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-12T03:15:25.768 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-12T03:15:25.768 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-12T03:15:25.956 DEBUG:teuthology.parallel:result is None 2024-09-12T03:15:28.296 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-12T03:15:28.296 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-12T03:15:28.296 INFO:teuthology.orchestra.run.smithi045.stdout:Upgraded: 2024-09-12T03:15:28.296 INFO:teuthology.orchestra.run.smithi045.stdout: NetworkManager-1:1.49.90-1.el9.x86_64 2024-09-12T03:15:28.296 INFO:teuthology.orchestra.run.smithi045.stdout: NetworkManager-libnm-1:1.49.90-1.el9.x86_64 2024-09-12T03:15:28.296 INFO:teuthology.orchestra.run.smithi045.stdout: NetworkManager-team-1:1.49.90-1.el9.x86_64 2024-09-12T03:15:28.296 INFO:teuthology.orchestra.run.smithi045.stdout: NetworkManager-tui-1:1.49.90-1.el9.x86_64 2024-09-12T03:15:28.296 INFO:teuthology.orchestra.run.smithi045.stdout: librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:15:28.296 INFO:teuthology.orchestra.run.smithi045.stdout: librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:15:28.296 INFO:teuthology.orchestra.run.smithi045.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-12T03:15:28.297 INFO:teuthology.orchestra.run.smithi045.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-12T03:15:28.297 INFO:teuthology.orchestra.run.smithi045.stdout:Installed: 2024-09-12T03:15:28.297 INFO:teuthology.orchestra.run.smithi045.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-12T03:15:28.297 INFO:teuthology.orchestra.run.smithi045.stdout: automake-1.16.2-8.el9.noarch 2024-09-12T03:15:28.297 INFO:teuthology.orchestra.run.smithi045.stdout: bison-3.7.4-5.el9.x86_64 2024-09-12T03:15:28.297 INFO:teuthology.orchestra.run.smithi045.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-12T03:15:28.297 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:15:28.297 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:15:28.297 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:15:28.297 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:15:28.297 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T03:15:28.297 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-immutable-object-cache-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:15:28.297 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:15:28.298 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:15:28.298 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T03:15:28.298 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T03:15:28.298 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T03:15:28.298 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T03:15:28.298 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T03:15:28.298 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:15:28.298 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:15:28.298 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T03:15:28.298 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:15:28.298 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:15:28.298 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:15:28.298 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T03:15:28.299 INFO:teuthology.orchestra.run.smithi045.stdout: cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T03:15:28.299 INFO:teuthology.orchestra.run.smithi045.stdout: cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:15:28.299 INFO:teuthology.orchestra.run.smithi045.stdout: cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:15:28.299 INFO:teuthology.orchestra.run.smithi045.stdout: cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T03:15:28.299 INFO:teuthology.orchestra.run.smithi045.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-12T03:15:28.299 INFO:teuthology.orchestra.run.smithi045.stdout: flex-2.6.4-9.el9.x86_64 2024-09-12T03:15:28.299 INFO:teuthology.orchestra.run.smithi045.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-12T03:15:28.299 INFO:teuthology.orchestra.run.smithi045.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-12T03:15:28.299 INFO:teuthology.orchestra.run.smithi045.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-12T03:15:28.299 INFO:teuthology.orchestra.run.smithi045.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-12T03:15:28.299 INFO:teuthology.orchestra.run.smithi045.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-12T03:15:28.299 INFO:teuthology.orchestra.run.smithi045.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-12T03:15:28.299 INFO:teuthology.orchestra.run.smithi045.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-12T03:15:28.299 INFO:teuthology.orchestra.run.smithi045.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-12T03:15:28.300 INFO:teuthology.orchestra.run.smithi045.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-12T03:15:28.300 INFO:teuthology.orchestra.run.smithi045.stdout: libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:15:28.300 INFO:teuthology.orchestra.run.smithi045.stdout: libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:15:28.300 INFO:teuthology.orchestra.run.smithi045.stdout: libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:15:28.300 INFO:teuthology.orchestra.run.smithi045.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-12T03:15:28.300 INFO:teuthology.orchestra.run.smithi045.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-12T03:15:28.300 INFO:teuthology.orchestra.run.smithi045.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-12T03:15:28.300 INFO:teuthology.orchestra.run.smithi045.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-12T03:15:28.300 INFO:teuthology.orchestra.run.smithi045.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-12T03:15:28.300 INFO:teuthology.orchestra.run.smithi045.stdout: librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:15:28.301 INFO:teuthology.orchestra.run.smithi045.stdout: libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:15:28.301 INFO:teuthology.orchestra.run.smithi045.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-12T03:15:28.301 INFO:teuthology.orchestra.run.smithi045.stdout: librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:15:28.301 INFO:teuthology.orchestra.run.smithi045.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-12T03:15:28.301 INFO:teuthology.orchestra.run.smithi045.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-12T03:15:28.301 INFO:teuthology.orchestra.run.smithi045.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-12T03:15:28.301 INFO:teuthology.orchestra.run.smithi045.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-12T03:15:28.301 INFO:teuthology.orchestra.run.smithi045.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-12T03:15:28.301 INFO:teuthology.orchestra.run.smithi045.stdout: lua-5.4.4-4.el9.x86_64 2024-09-12T03:15:28.301 INFO:teuthology.orchestra.run.smithi045.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-12T03:15:28.301 INFO:teuthology.orchestra.run.smithi045.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-12T03:15:28.301 INFO:teuthology.orchestra.run.smithi045.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-12T03:15:28.301 INFO:teuthology.orchestra.run.smithi045.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-12T03:15:28.301 INFO:teuthology.orchestra.run.smithi045.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-12T03:15:28.302 INFO:teuthology.orchestra.run.smithi045.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-12T03:15:28.302 INFO:teuthology.orchestra.run.smithi045.stdout: postgresql-13.16-1.el9.x86_64 2024-09-12T03:15:28.302 INFO:teuthology.orchestra.run.smithi045.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-12T03:15:28.302 INFO:teuthology.orchestra.run.smithi045.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-12T03:15:28.302 INFO:teuthology.orchestra.run.smithi045.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-12T03:15:28.302 INFO:teuthology.orchestra.run.smithi045.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-12T03:15:28.302 INFO:teuthology.orchestra.run.smithi045.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-12T03:15:28.302 INFO:teuthology.orchestra.run.smithi045.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-12T03:15:28.302 INFO:teuthology.orchestra.run.smithi045.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-12T03:15:28.302 INFO:teuthology.orchestra.run.smithi045.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-12T03:15:28.302 INFO:teuthology.orchestra.run.smithi045.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-12T03:15:28.302 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-12T03:15:28.302 INFO:teuthology.orchestra.run.smithi045.stdout: python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:15:28.302 INFO:teuthology.orchestra.run.smithi045.stdout: python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:15:28.303 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:15:28.303 INFO:teuthology.orchestra.run.smithi045.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-12T03:15:28.303 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-12T03:15:28.303 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-12T03:15:28.303 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-12T03:15:28.303 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-12T03:15:28.303 INFO:teuthology.orchestra.run.smithi045.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-12T03:15:28.303 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-12T03:15:28.303 INFO:teuthology.orchestra.run.smithi045.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-12T03:15:28.303 INFO:teuthology.orchestra.run.smithi045.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-12T03:15:28.303 INFO:teuthology.orchestra.run.smithi045.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-12T03:15:28.303 INFO:teuthology.orchestra.run.smithi045.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-12T03:15:28.304 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-12T03:15:28.304 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-12T03:15:28.304 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-12T03:15:28.304 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-12T03:15:28.304 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-12T03:15:28.304 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-12T03:15:28.304 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-12T03:15:28.304 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-12T03:15:28.304 INFO:teuthology.orchestra.run.smithi045.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-12T03:15:28.304 INFO:teuthology.orchestra.run.smithi045.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-12T03:15:28.304 INFO:teuthology.orchestra.run.smithi045.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-12T03:15:28.304 INFO:teuthology.orchestra.run.smithi045.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-12T03:15:28.304 INFO:teuthology.orchestra.run.smithi045.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-12T03:15:28.304 INFO:teuthology.orchestra.run.smithi045.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-12T03:15:28.305 INFO:teuthology.orchestra.run.smithi045.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-12T03:15:28.305 INFO:teuthology.orchestra.run.smithi045.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-12T03:15:28.305 INFO:teuthology.orchestra.run.smithi045.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-12T03:15:28.305 INFO:teuthology.orchestra.run.smithi045.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-12T03:15:28.305 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-12T03:15:28.305 INFO:teuthology.orchestra.run.smithi045.stdout: python3-ply-3.11-14.el9.noarch 2024-09-12T03:15:28.305 INFO:teuthology.orchestra.run.smithi045.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-12T03:15:28.305 INFO:teuthology.orchestra.run.smithi045.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-12T03:15:28.305 INFO:teuthology.orchestra.run.smithi045.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-12T03:15:28.305 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-12T03:15:28.305 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-12T03:15:28.305 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-12T03:15:28.305 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-12T03:15:28.305 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-12T03:15:28.306 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-12T03:15:28.306 INFO:teuthology.orchestra.run.smithi045.stdout: python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:15:28.306 INFO:teuthology.orchestra.run.smithi045.stdout: python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:15:28.306 INFO:teuthology.orchestra.run.smithi045.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-12T03:15:28.306 INFO:teuthology.orchestra.run.smithi045.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-12T03:15:28.306 INFO:teuthology.orchestra.run.smithi045.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-12T03:15:28.306 INFO:teuthology.orchestra.run.smithi045.stdout: python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:15:28.306 INFO:teuthology.orchestra.run.smithi045.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-12T03:15:28.306 INFO:teuthology.orchestra.run.smithi045.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-12T03:15:28.306 INFO:teuthology.orchestra.run.smithi045.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-12T03:15:28.306 INFO:teuthology.orchestra.run.smithi045.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-12T03:15:28.306 INFO:teuthology.orchestra.run.smithi045.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-12T03:15:28.306 INFO:teuthology.orchestra.run.smithi045.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-12T03:15:28.307 INFO:teuthology.orchestra.run.smithi045.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-12T03:15:28.307 INFO:teuthology.orchestra.run.smithi045.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-12T03:15:28.307 INFO:teuthology.orchestra.run.smithi045.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-12T03:15:28.307 INFO:teuthology.orchestra.run.smithi045.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-12T03:15:28.307 INFO:teuthology.orchestra.run.smithi045.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-12T03:15:28.307 INFO:teuthology.orchestra.run.smithi045.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-12T03:15:28.307 INFO:teuthology.orchestra.run.smithi045.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-12T03:15:28.307 INFO:teuthology.orchestra.run.smithi045.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-12T03:15:28.307 INFO:teuthology.orchestra.run.smithi045.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-12T03:15:28.307 INFO:teuthology.orchestra.run.smithi045.stdout: rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:15:28.307 INFO:teuthology.orchestra.run.smithi045.stdout: rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:15:28.307 INFO:teuthology.orchestra.run.smithi045.stdout: rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:15:28.307 INFO:teuthology.orchestra.run.smithi045.stdout: re2-1:20211101-20.el9.x86_64 2024-09-12T03:15:28.307 INFO:teuthology.orchestra.run.smithi045.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-12T03:15:28.308 INFO:teuthology.orchestra.run.smithi045.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-12T03:15:28.308 INFO:teuthology.orchestra.run.smithi045.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-12T03:15:28.308 INFO:teuthology.orchestra.run.smithi045.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-12T03:15:28.308 INFO:teuthology.orchestra.run.smithi045.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-12T03:15:28.308 INFO:teuthology.orchestra.run.smithi045.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-12T03:15:28.308 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-12T03:15:28.308 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2024-09-12T03:15:28.726 DEBUG:teuthology.parallel:result is None 2024-09-12T03:15:28.726 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-12T03:15:28.863 DEBUG:teuthology.orchestra.run.smithi027:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-12T03:15:28.891 INFO:teuthology.orchestra.run.smithi027.stdout:19.1.1-298.gb53b6d3e.el9 2024-09-12T03:15:28.891 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-298.gb53b6d3e.el9 2024-09-12T03:15:28.891 INFO:teuthology.task.install:The correct ceph version 19.1.1-298.gb53b6d3e is installed. 2024-09-12T03:15:28.893 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-12T03:15:29.030 DEBUG:teuthology.orchestra.run.smithi045:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-12T03:15:29.056 INFO:teuthology.orchestra.run.smithi045.stdout:19.1.1-298.gb53b6d3e.el9 2024-09-12T03:15:29.056 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-298.gb53b6d3e.el9 2024-09-12T03:15:29.056 INFO:teuthology.task.install:The correct ceph version 19.1.1-298.gb53b6d3e is installed. 2024-09-12T03:15:29.058 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-12T03:15:29.058 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-12T03:15:29.058 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-12T03:15:29.092 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-12T03:15:29.092 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-12T03:15:29.126 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-12T03:15:29.126 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-12T03:15:29.126 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd of=/usr/bin/daemon-helper 2024-09-12T03:15:29.162 DEBUG:teuthology.orchestra.run.smithi027:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-12T03:15:29.231 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-12T03:15:29.231 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd of=/usr/bin/daemon-helper 2024-09-12T03:15:29.259 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-12T03:15:29.328 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-12T03:15:29.328 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-12T03:15:29.328 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-12T03:15:29.356 DEBUG:teuthology.orchestra.run.smithi027:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-12T03:15:29.425 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-12T03:15:29.425 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-12T03:15:29.454 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-12T03:15:29.523 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-12T03:15:29.523 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-12T03:15:29.524 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd of=/usr/bin/stdin-killer 2024-09-12T03:15:29.551 DEBUG:teuthology.orchestra.run.smithi027:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-12T03:15:29.618 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-12T03:15:29.618 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd of=/usr/bin/stdin-killer 2024-09-12T03:15:29.646 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-12T03:15:29.715 INFO:teuthology.run_tasks:Running task ceph... 2024-09-12T03:15:29.796 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-09-12T03:15:29.796 DEBUG:teuthology.orchestra.run.smithi027:> sudo chmod 777 /var/log/ceph 2024-09-12T03:15:29.799 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod 777 /var/log/ceph 2024-09-12T03:15:29.827 INFO:tasks.ceph:Disabling ceph logrotate... 2024-09-12T03:15:29.827 DEBUG:teuthology.orchestra.run.smithi027:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-12T03:15:29.866 DEBUG:teuthology.orchestra.run.smithi045:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-12T03:15:29.895 INFO:tasks.ceph:Creating extra log directories... 2024-09-12T03:15:29.896 DEBUG:teuthology.orchestra.run.smithi027:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-12T03:15:29.932 DEBUG:teuthology.orchestra.run.smithi045:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-12T03:15:29.970 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-09-12T03:15:29.970 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-09-12T03:15:29.970 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-09-12T03:15:29.971 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-12T03:15:29.971 DEBUG:teuthology.orchestra.run.smithi027:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-09-12T03:15:30.013 DEBUG:teuthology.orchestra.run.smithi027:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-09-12T03:15:30.124 DEBUG:teuthology.orchestra.run.smithi027:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-09-12T03:15:30.150 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-12T03:15:30.150 DEBUG:teuthology.orchestra.run.smithi045:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-09-12T03:15:30.168 DEBUG:teuthology.orchestra.run.smithi045:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-09-12T03:15:30.287 DEBUG:teuthology.orchestra.run.smithi045:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-09-12T03:15:30.321 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-09-12T03:15:30.321 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120, 'mon warn on pool no app': False}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-09-12T03:15:30.321 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901674', 'branch': 'squid', 'description': 'fs/volumes/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/volumes/{overrides test/misc}}', 'email': 'ceph-qa@ceph.com', 'first_in_suite': False, 'job_id': '7901674', 'kernel': {'client': {'sha1': 'distro'}, 'kdb': 1, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'mgrmodules': {'sequential': [{'print': 'Enabling mgr modules'}]}, 'name': 'teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi', 'no_nested_subset': False, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'squid'}, 'ceph': {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120, 'mon warn on pool no app': False}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc'}}, 'kclient': {'syntax': 'v2'}, 'selinux': {'allowlist': ['scontext=system_u:system_r:getty_t:s0', 'scontext=system_u:system_r:getty_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'squid', 'sha1': '086c737fa75c22d521a3932c59b31c5e1d1c9ef4'}}, 'owner': 'scheduled_teuthology@teuthology', 'priority': 100, 'repo': 'https://git.ceph.com/ceph.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 481, 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc', 'sleep_before_teardown': 0, 'subset': '57/512', 'suite': 'fs', 'suite_branch': 'squid', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph_086c737fa75c22d521a3932c59b31c5e1d1c9ef4/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph.git', 'suite_sha1': '086c737fa75c22d521a3932c59b31c5e1d1c9ef4', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi027.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBpfvmMTLtMKDnONFCL9LTez8zipHWQdqL6ZFRgraFz4nAuripvlmJnLmyj4GSlvbakoBx7acTgaNLLHU0/qeVI=', 'smithi045.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKWB5vVII7VIeh+UqY3X5wlyuY3D4MdoJx16ZtrOy1JdCcNzRittfueIME+xvpkZd56C61e8kSJFn8u/puLyKoE='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'sha1': 'distro'}, 'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc'}}, {'ceph': {'create_rbd_pool': False, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120, 'mon warn on pool no app': False}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc', 'cluster': 'ceph'}}, {'sequential': ['mgrmodules']}, {'kclient': None}, {'cephfs_test_runner': {'fail_on_skip': False, 'modules': ['tasks.cephfs.test_volumes.TestMisc']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ["local function is_kupstream() return yaml.ktype == 'upstream' end", "local function is_kdistro() return yaml.ktype == 'distro' end", 'local function is_fuse() return false end', 'local function syntax_version() return yaml.overrides.kclient.syntax end']}, 'teuthology_branch': 'main', 'teuthology_sha1': '3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a', 'timestamp': '2024-09-11_21:24:17', 'tube': 'smithi', 'user': 'teuthology', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856'} 2024-09-12T03:15:30.322 DEBUG:teuthology.orchestra.run.smithi027:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-12T03:15:30.324 DEBUG:teuthology.orchestra.run.smithi045:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-12T03:15:30.372 DEBUG:teuthology.orchestra.run.smithi027:> sudo install -d -m0777 -- /var/run/ceph 2024-09-12T03:15:30.380 DEBUG:teuthology.orchestra.run.smithi045:> sudo install -d -m0777 -- /var/run/ceph 2024-09-12T03:15:30.440 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-12T03:15:30.440 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/scratch_devs of=/dev/stdout 2024-09-12T03:15:30.459 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-12T03:15:30.460 DEBUG:teuthology.orchestra.run.smithi027:> stat /dev/vg_nvme/lv_1 2024-09-12T03:15:30.515 INFO:teuthology.orchestra.run.smithi027.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-12T03:15:30.515 INFO:teuthology.orchestra.run.smithi027.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T03:15:30.515 INFO:teuthology.orchestra.run.smithi027.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-09-12T03:15:30.515 INFO:teuthology.orchestra.run.smithi027.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T03:15:30.516 INFO:teuthology.orchestra.run.smithi027.stdout:Context: system_u:object_r:device_t:s0 2024-09-12T03:15:30.516 INFO:teuthology.orchestra.run.smithi027.stdout:Access: 2024-09-12 03:15:17.272160774 +0000 2024-09-12T03:15:30.516 INFO:teuthology.orchestra.run.smithi027.stdout:Modify: 2024-09-12 03:13:31.843385037 +0000 2024-09-12T03:15:30.516 INFO:teuthology.orchestra.run.smithi027.stdout:Change: 2024-09-12 03:13:31.843385037 +0000 2024-09-12T03:15:30.516 INFO:teuthology.orchestra.run.smithi027.stdout: Birth: 2024-09-12 03:13:31.843385037 +0000 2024-09-12T03:15:30.516 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-12T03:15:30.579 INFO:teuthology.orchestra.run.smithi027.stderr:1+0 records in 2024-09-12T03:15:30.579 INFO:teuthology.orchestra.run.smithi027.stderr:1+0 records out 2024-09-12T03:15:30.579 INFO:teuthology.orchestra.run.smithi027.stderr:512 bytes copied, 0.000198751 s, 2.6 MB/s 2024-09-12T03:15:30.581 DEBUG:teuthology.orchestra.run.smithi027:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-12T03:15:30.637 DEBUG:teuthology.orchestra.run.smithi027:> stat /dev/vg_nvme/lv_2 2024-09-12T03:15:30.693 INFO:teuthology.orchestra.run.smithi027.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-12T03:15:30.694 INFO:teuthology.orchestra.run.smithi027.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T03:15:30.694 INFO:teuthology.orchestra.run.smithi027.stdout:Device: 5h/5d Inode: 687 Links: 1 2024-09-12T03:15:30.694 INFO:teuthology.orchestra.run.smithi027.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T03:15:30.694 INFO:teuthology.orchestra.run.smithi027.stdout:Context: system_u:object_r:device_t:s0 2024-09-12T03:15:30.694 INFO:teuthology.orchestra.run.smithi027.stdout:Access: 2024-09-12 03:15:17.272160774 +0000 2024-09-12T03:15:30.694 INFO:teuthology.orchestra.run.smithi027.stdout:Modify: 2024-09-12 03:13:32.179376366 +0000 2024-09-12T03:15:30.694 INFO:teuthology.orchestra.run.smithi027.stdout:Change: 2024-09-12 03:13:32.179376366 +0000 2024-09-12T03:15:30.694 INFO:teuthology.orchestra.run.smithi027.stdout: Birth: 2024-09-12 03:13:32.179376366 +0000 2024-09-12T03:15:30.694 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-12T03:15:30.757 INFO:teuthology.orchestra.run.smithi027.stderr:1+0 records in 2024-09-12T03:15:30.757 INFO:teuthology.orchestra.run.smithi027.stderr:1+0 records out 2024-09-12T03:15:30.758 INFO:teuthology.orchestra.run.smithi027.stderr:512 bytes copied, 0.000173367 s, 3.0 MB/s 2024-09-12T03:15:30.759 DEBUG:teuthology.orchestra.run.smithi027:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-12T03:15:30.815 DEBUG:teuthology.orchestra.run.smithi027:> stat /dev/vg_nvme/lv_3 2024-09-12T03:15:30.871 INFO:teuthology.orchestra.run.smithi027.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-12T03:15:30.871 INFO:teuthology.orchestra.run.smithi027.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T03:15:30.872 INFO:teuthology.orchestra.run.smithi027.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-09-12T03:15:30.872 INFO:teuthology.orchestra.run.smithi027.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T03:15:30.872 INFO:teuthology.orchestra.run.smithi027.stdout:Context: system_u:object_r:device_t:s0 2024-09-12T03:15:30.872 INFO:teuthology.orchestra.run.smithi027.stdout:Access: 2024-09-12 03:15:17.272160774 +0000 2024-09-12T03:15:30.872 INFO:teuthology.orchestra.run.smithi027.stdout:Modify: 2024-09-12 03:13:32.506367928 +0000 2024-09-12T03:15:30.872 INFO:teuthology.orchestra.run.smithi027.stdout:Change: 2024-09-12 03:13:32.506367928 +0000 2024-09-12T03:15:30.872 INFO:teuthology.orchestra.run.smithi027.stdout: Birth: 2024-09-12 03:13:32.506367928 +0000 2024-09-12T03:15:30.872 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-12T03:15:30.936 INFO:teuthology.orchestra.run.smithi027.stderr:1+0 records in 2024-09-12T03:15:30.936 INFO:teuthology.orchestra.run.smithi027.stderr:1+0 records out 2024-09-12T03:15:30.936 INFO:teuthology.orchestra.run.smithi027.stderr:512 bytes copied, 0.000194241 s, 2.6 MB/s 2024-09-12T03:15:30.937 DEBUG:teuthology.orchestra.run.smithi027:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-12T03:15:30.994 DEBUG:teuthology.orchestra.run.smithi027:> stat /dev/vg_nvme/lv_4 2024-09-12T03:15:31.049 INFO:teuthology.orchestra.run.smithi027.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-12T03:15:31.049 INFO:teuthology.orchestra.run.smithi027.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T03:15:31.049 INFO:teuthology.orchestra.run.smithi027.stdout:Device: 5h/5d Inode: 725 Links: 1 2024-09-12T03:15:31.049 INFO:teuthology.orchestra.run.smithi027.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T03:15:31.049 INFO:teuthology.orchestra.run.smithi027.stdout:Context: system_u:object_r:device_t:s0 2024-09-12T03:15:31.049 INFO:teuthology.orchestra.run.smithi027.stdout:Access: 2024-09-12 03:15:17.272160774 +0000 2024-09-12T03:15:31.050 INFO:teuthology.orchestra.run.smithi027.stdout:Modify: 2024-09-12 03:13:32.851359025 +0000 2024-09-12T03:15:31.050 INFO:teuthology.orchestra.run.smithi027.stdout:Change: 2024-09-12 03:13:32.851359025 +0000 2024-09-12T03:15:31.050 INFO:teuthology.orchestra.run.smithi027.stdout: Birth: 2024-09-12 03:13:32.851359025 +0000 2024-09-12T03:15:31.050 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-12T03:15:31.113 INFO:teuthology.orchestra.run.smithi027.stderr:1+0 records in 2024-09-12T03:15:31.113 INFO:teuthology.orchestra.run.smithi027.stderr:1+0 records out 2024-09-12T03:15:31.114 INFO:teuthology.orchestra.run.smithi027.stderr:512 bytes copied, 0.000180873 s, 2.8 MB/s 2024-09-12T03:15:31.115 DEBUG:teuthology.orchestra.run.smithi027:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-12T03:15:31.171 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-12T03:15:31.171 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-12T03:15:31.171 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/scratch_devs of=/dev/stdout 2024-09-12T03:15:31.191 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-12T03:15:31.191 DEBUG:teuthology.orchestra.run.smithi045:> stat /dev/vg_nvme/lv_1 2024-09-12T03:15:31.251 INFO:teuthology.orchestra.run.smithi045.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-12T03:15:31.251 INFO:teuthology.orchestra.run.smithi045.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T03:15:31.251 INFO:teuthology.orchestra.run.smithi045.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-09-12T03:15:31.251 INFO:teuthology.orchestra.run.smithi045.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T03:15:31.251 INFO:teuthology.orchestra.run.smithi045.stdout:Context: system_u:object_r:device_t:s0 2024-09-12T03:15:31.251 INFO:teuthology.orchestra.run.smithi045.stdout:Access: 2024-09-12 03:15:20.509791612 +0000 2024-09-12T03:15:31.251 INFO:teuthology.orchestra.run.smithi045.stdout:Modify: 2024-09-12 03:13:31.990301038 +0000 2024-09-12T03:15:31.252 INFO:teuthology.orchestra.run.smithi045.stdout:Change: 2024-09-12 03:13:31.990301038 +0000 2024-09-12T03:15:31.252 INFO:teuthology.orchestra.run.smithi045.stdout: Birth: 2024-09-12 03:13:31.990301038 +0000 2024-09-12T03:15:31.252 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-12T03:15:31.321 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-09-12T03:15:31.321 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-09-12T03:15:31.321 INFO:teuthology.orchestra.run.smithi045.stderr:512 bytes copied, 0.000125545 s, 4.1 MB/s 2024-09-12T03:15:31.322 DEBUG:teuthology.orchestra.run.smithi045:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-12T03:15:31.382 DEBUG:teuthology.orchestra.run.smithi045:> stat /dev/vg_nvme/lv_2 2024-09-12T03:15:31.441 INFO:teuthology.orchestra.run.smithi045.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-12T03:15:31.441 INFO:teuthology.orchestra.run.smithi045.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T03:15:31.441 INFO:teuthology.orchestra.run.smithi045.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-09-12T03:15:31.442 INFO:teuthology.orchestra.run.smithi045.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T03:15:31.442 INFO:teuthology.orchestra.run.smithi045.stdout:Context: system_u:object_r:device_t:s0 2024-09-12T03:15:31.442 INFO:teuthology.orchestra.run.smithi045.stdout:Access: 2024-09-12 03:15:20.510791597 +0000 2024-09-12T03:15:31.442 INFO:teuthology.orchestra.run.smithi045.stdout:Modify: 2024-09-12 03:13:32.317296132 +0000 2024-09-12T03:15:31.442 INFO:teuthology.orchestra.run.smithi045.stdout:Change: 2024-09-12 03:13:32.317296132 +0000 2024-09-12T03:15:31.442 INFO:teuthology.orchestra.run.smithi045.stdout: Birth: 2024-09-12 03:13:32.317296132 +0000 2024-09-12T03:15:31.442 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-12T03:15:31.510 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-09-12T03:15:31.510 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-09-12T03:15:31.510 INFO:teuthology.orchestra.run.smithi045.stderr:512 bytes copied, 0.000127375 s, 4.0 MB/s 2024-09-12T03:15:31.512 DEBUG:teuthology.orchestra.run.smithi045:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-12T03:15:31.571 DEBUG:teuthology.orchestra.run.smithi045:> stat /dev/vg_nvme/lv_3 2024-09-12T03:15:31.630 INFO:teuthology.orchestra.run.smithi045.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-12T03:15:31.630 INFO:teuthology.orchestra.run.smithi045.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T03:15:31.630 INFO:teuthology.orchestra.run.smithi045.stdout:Device: 5h/5d Inode: 706 Links: 1 2024-09-12T03:15:31.630 INFO:teuthology.orchestra.run.smithi045.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T03:15:31.630 INFO:teuthology.orchestra.run.smithi045.stdout:Context: system_u:object_r:device_t:s0 2024-09-12T03:15:31.630 INFO:teuthology.orchestra.run.smithi045.stdout:Access: 2024-09-12 03:15:20.510791597 +0000 2024-09-12T03:15:31.631 INFO:teuthology.orchestra.run.smithi045.stdout:Modify: 2024-09-12 03:13:32.632291406 +0000 2024-09-12T03:15:31.631 INFO:teuthology.orchestra.run.smithi045.stdout:Change: 2024-09-12 03:13:32.632291406 +0000 2024-09-12T03:15:31.631 INFO:teuthology.orchestra.run.smithi045.stdout: Birth: 2024-09-12 03:13:32.632291406 +0000 2024-09-12T03:15:31.631 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-12T03:15:31.698 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-09-12T03:15:31.698 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-09-12T03:15:31.699 INFO:teuthology.orchestra.run.smithi045.stderr:512 bytes copied, 9.2638e-05 s, 5.5 MB/s 2024-09-12T03:15:31.700 DEBUG:teuthology.orchestra.run.smithi045:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-12T03:15:31.759 DEBUG:teuthology.orchestra.run.smithi045:> stat /dev/vg_nvme/lv_4 2024-09-12T03:15:31.817 INFO:teuthology.orchestra.run.smithi045.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-12T03:15:31.817 INFO:teuthology.orchestra.run.smithi045.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T03:15:31.817 INFO:teuthology.orchestra.run.smithi045.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-09-12T03:15:31.818 INFO:teuthology.orchestra.run.smithi045.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T03:15:31.818 INFO:teuthology.orchestra.run.smithi045.stdout:Context: system_u:object_r:device_t:s0 2024-09-12T03:15:31.818 INFO:teuthology.orchestra.run.smithi045.stdout:Access: 2024-09-12 03:15:20.510791597 +0000 2024-09-12T03:15:31.818 INFO:teuthology.orchestra.run.smithi045.stdout:Modify: 2024-09-12 03:13:32.958286515 +0000 2024-09-12T03:15:31.818 INFO:teuthology.orchestra.run.smithi045.stdout:Change: 2024-09-12 03:13:32.958286515 +0000 2024-09-12T03:15:31.818 INFO:teuthology.orchestra.run.smithi045.stdout: Birth: 2024-09-12 03:13:32.958286515 +0000 2024-09-12T03:15:31.818 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-12T03:15:31.885 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-09-12T03:15:31.885 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-09-12T03:15:31.885 INFO:teuthology.orchestra.run.smithi045.stderr:512 bytes copied, 0.000112864 s, 4.5 MB/s 2024-09-12T03:15:31.886 DEBUG:teuthology.orchestra.run.smithi045:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-12T03:15:31.945 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-12T03:15:31.945 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi027.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi045.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-09-12T03:15:31.945 INFO:tasks.ceph:Generating config... 2024-09-12T03:15:31.947 INFO:tasks.ceph:[client] client mount timeout = 600 2024-09-12T03:15:31.947 INFO:tasks.ceph:[client] debug client = 20 2024-09-12T03:15:31.947 INFO:tasks.ceph:[client] debug ms = 1 2024-09-12T03:15:31.947 INFO:tasks.ceph:[client] pid file = 2024-09-12T03:15:31.947 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-09-12T03:15:31.947 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-09-12T03:15:31.947 INFO:tasks.ceph:[global] ms die on skipped message = False 2024-09-12T03:15:31.947 INFO:tasks.ceph:[mds] debug mds = 20 2024-09-12T03:15:31.948 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-09-12T03:15:31.948 INFO:tasks.ceph:[mds] debug ms = 1 2024-09-12T03:15:31.948 INFO:tasks.ceph:[mds] mds debug frag = True 2024-09-12T03:15:31.948 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-09-12T03:15:31.948 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-09-12T03:15:31.948 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-09-12T03:15:31.948 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-09-12T03:15:31.948 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-09-12T03:15:31.948 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-09-12T03:15:31.948 INFO:tasks.ceph:[mgr] debug client = 20 2024-09-12T03:15:31.948 INFO:tasks.ceph:[mgr] debug finisher = 20 2024-09-12T03:15:31.948 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-09-12T03:15:31.949 INFO:tasks.ceph:[mgr] debug ms = 1 2024-09-12T03:15:31.949 INFO:tasks.ceph:[mon] debug mon = 20 2024-09-12T03:15:31.949 INFO:tasks.ceph:[mon] debug ms = 1 2024-09-12T03:15:31.949 INFO:tasks.ceph:[mon] debug paxos = 20 2024-09-12T03:15:31.949 INFO:tasks.ceph:[mon] mon down mkfs grace = 300 2024-09-12T03:15:31.949 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-09-12T03:15:31.949 INFO:tasks.ceph:[mon] mon warn on pool no app = False 2024-09-12T03:15:31.949 INFO:tasks.ceph:[osd] bdev async discard = True 2024-09-12T03:15:31.949 INFO:tasks.ceph:[osd] bdev enable discard = True 2024-09-12T03:15:31.949 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2024-09-12T03:15:31.949 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-09-12T03:15:31.949 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-09-12T03:15:31.950 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-09-12T03:15:31.950 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-09-12T03:15:31.950 INFO:tasks.ceph:[osd] debug ms = 1 2024-09-12T03:15:31.950 INFO:tasks.ceph:[osd] debug osd = 20 2024-09-12T03:15:31.950 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-09-12T03:15:31.950 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-09-12T03:15:31.950 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-09-12T03:15:31.950 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-09-12T03:15:31.950 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-09-12T03:15:31.950 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-09-12T03:15:31.950 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-09-12T03:15:31.950 INFO:tasks.ceph:Setting up mon.a... 2024-09-12T03:15:31.951 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-09-12T03:15:32.005 INFO:teuthology.orchestra.run.smithi027.stdout:creating /etc/ceph/ceph.keyring 2024-09-12T03:15:32.008 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-09-12T03:15:32.054 DEBUG:teuthology.orchestra.run.smithi027:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-12T03:15:32.079 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.27'), ('mon.b', '172.21.15.45'), ('mon.c', '[v2:172.21.15.45:3301,v1:172.21.15.45:6790]')] 2024-09-12T03:15:32.079 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.27,172.21.15.45,[v2:172.21.15.45:3301,v1:172.21.15.45:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20, 'debug finisher': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon down mkfs grace': 300, 'mon op complaint time': 120, 'mon warn on pool no app': False}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-09-12T03:15:32.080 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-12T03:15:32.080 DEBUG:teuthology.orchestra.run.smithi027:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-09-12T03:15:32.136 DEBUG:teuthology.orchestra.run.smithi027:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.27 --add b 172.21.15.45 --addv c '[v2:172.21.15.45:3301,v1:172.21.15.45:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-09-12T03:15:32.223 INFO:teuthology.orchestra.run.smithi027.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-09-12T03:15:32.224 INFO:teuthology.orchestra.run.smithi027.stdout:monmaptool: generated fsid d2df504e-2647-4385-a5db-9fdf69f487a0 2024-09-12T03:15:32.224 INFO:teuthology.orchestra.run.smithi027.stdout:setting min_mon_release = quincy 2024-09-12T03:15:32.224 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 0 2024-09-12T03:15:32.224 INFO:teuthology.orchestra.run.smithi027.stdout:fsid d2df504e-2647-4385-a5db-9fdf69f487a0 2024-09-12T03:15:32.224 INFO:teuthology.orchestra.run.smithi027.stdout:last_changed 2024-09-12T03:15:32.224591+0000 2024-09-12T03:15:32.224 INFO:teuthology.orchestra.run.smithi027.stdout:created 2024-09-12T03:15:32.224591+0000 2024-09-12T03:15:32.224 INFO:teuthology.orchestra.run.smithi027.stdout:min_mon_release 17 (quincy) 2024-09-12T03:15:32.224 INFO:teuthology.orchestra.run.smithi027.stdout:election_strategy: 1 2024-09-12T03:15:32.224 INFO:teuthology.orchestra.run.smithi027.stdout:0: [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] mon.a 2024-09-12T03:15:32.224 INFO:teuthology.orchestra.run.smithi027.stdout:1: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2024-09-12T03:15:32.224 INFO:teuthology.orchestra.run.smithi027.stdout:2: [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] mon.c 2024-09-12T03:15:32.225 INFO:teuthology.orchestra.run.smithi027.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-09-12T03:15:32.226 DEBUG:teuthology.orchestra.run.smithi027:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-09-12T03:15:32.241 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID d2df504e-2647-4385-a5db-9fdf69f487a0... 2024-09-12T03:15:32.242 DEBUG:teuthology.orchestra.run.smithi027:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-12T03:15:32.284 DEBUG:teuthology.orchestra.run.smithi045:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-12T03:15:32.326 INFO:teuthology.orchestra.run.smithi027.stdout:[global] 2024-09-12T03:15:32.326 INFO:teuthology.orchestra.run.smithi027.stdout: chdir = "" 2024-09-12T03:15:32.326 INFO:teuthology.orchestra.run.smithi027.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-12T03:15:32.326 INFO:teuthology.orchestra.run.smithi027.stdout: auth supported = cephx 2024-09-12T03:15:32.326 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:15:32.326 INFO:teuthology.orchestra.run.smithi027.stdout: filestore xattr use omap = true 2024-09-12T03:15:32.327 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:15:32.327 INFO:teuthology.orchestra.run.smithi027.stdout: mon clock drift allowed = 1.000 2024-09-12T03:15:32.327 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:15:32.327 INFO:teuthology.orchestra.run.smithi027.stdout: osd crush chooseleaf type = 0 2024-09-12T03:15:32.327 INFO:teuthology.orchestra.run.smithi027.stdout: auth debug = true 2024-09-12T03:15:32.327 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:15:32.327 INFO:teuthology.orchestra.run.smithi027.stdout: ms die on old message = true 2024-09-12T03:15:32.327 INFO:teuthology.orchestra.run.smithi027.stdout: ms die on bug = true 2024-09-12T03:15:32.327 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:15:32.327 INFO:teuthology.orchestra.run.smithi027.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-12T03:15:32.327 INFO:teuthology.orchestra.run.smithi027.stdout: mon pg warn max object skew = 0 2024-09-12T03:15:32.327 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:15:32.327 INFO:teuthology.orchestra.run.smithi027.stdout: # disable pg_autoscaler by default for new pools 2024-09-12T03:15:32.328 INFO:teuthology.orchestra.run.smithi027.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-12T03:15:32.328 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:15:32.328 INFO:teuthology.orchestra.run.smithi027.stdout: osd pool default size = 2 2024-09-12T03:15:32.328 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:15:32.328 INFO:teuthology.orchestra.run.smithi027.stdout: mon osd allow primary affinity = true 2024-09-12T03:15:32.328 INFO:teuthology.orchestra.run.smithi027.stdout: mon osd allow pg remap = true 2024-09-12T03:15:32.328 INFO:teuthology.orchestra.run.smithi027.stdout: mon warn on legacy crush tunables = false 2024-09-12T03:15:32.328 INFO:teuthology.orchestra.run.smithi027.stdout: mon warn on crush straw calc version zero = false 2024-09-12T03:15:32.328 INFO:teuthology.orchestra.run.smithi027.stdout: mon warn on no sortbitwise = false 2024-09-12T03:15:32.328 INFO:teuthology.orchestra.run.smithi027.stdout: mon warn on osd down out interval zero = false 2024-09-12T03:15:32.328 INFO:teuthology.orchestra.run.smithi027.stdout: mon warn on too few osds = false 2024-09-12T03:15:32.328 INFO:teuthology.orchestra.run.smithi027.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-12T03:15:32.329 INFO:teuthology.orchestra.run.smithi027.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-12T03:15:32.329 INFO:teuthology.orchestra.run.smithi027.stdout: mon_allow_pool_size_one = true 2024-09-12T03:15:32.329 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:15:32.329 INFO:teuthology.orchestra.run.smithi027.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-12T03:15:32.329 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:15:32.329 INFO:teuthology.orchestra.run.smithi027.stdout: osd default data pool replay window = 5 2024-09-12T03:15:32.329 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:15:32.329 INFO:teuthology.orchestra.run.smithi027.stdout: mon allow pool delete = true 2024-09-12T03:15:32.329 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:15:32.329 INFO:teuthology.orchestra.run.smithi027.stdout: mon cluster log file level = debug 2024-09-12T03:15:32.329 INFO:teuthology.orchestra.run.smithi027.stdout: debug asserts on shutdown = true 2024-09-12T03:15:32.329 INFO:teuthology.orchestra.run.smithi027.stdout: mon health detail to clog = false 2024-09-12T03:15:32.329 INFO:teuthology.orchestra.run.smithi027.stdout: mon host = "172.21.15.27,172.21.15.45,[v2:172.21.15.45:3301,v1:172.21.15.45:6790]" 2024-09-12T03:15:32.329 INFO:teuthology.orchestra.run.smithi027.stdout: ms die on skipped message = False 2024-09-12T03:15:32.330 INFO:teuthology.orchestra.run.smithi027.stdout: fsid = d2df504e-2647-4385-a5db-9fdf69f487a0 2024-09-12T03:15:32.330 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:15:32.330 INFO:teuthology.orchestra.run.smithi027.stdout:[osd] 2024-09-12T03:15:32.330 INFO:teuthology.orchestra.run.smithi027.stdout: osd journal size = 100 2024-09-12T03:15:32.330 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:15:32.330 INFO:teuthology.orchestra.run.smithi027.stdout: osd scrub load threshold = 5.0 2024-09-12T03:15:32.330 INFO:teuthology.orchestra.run.smithi027.stdout: osd scrub max interval = 600 2024-09-12T03:15:32.330 INFO:teuthology.orchestra.run.smithi027.stdout: osd mclock profile = high_recovery_ops 2024-09-12T03:15:32.330 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:15:32.330 INFO:teuthology.orchestra.run.smithi027.stdout: osd recover clone overlap = true 2024-09-12T03:15:32.330 INFO:teuthology.orchestra.run.smithi027.stdout: osd recovery max chunk = 1048576 2024-09-12T03:15:32.330 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:15:32.330 INFO:teuthology.orchestra.run.smithi027.stdout: osd debug shutdown = true 2024-09-12T03:15:32.331 INFO:teuthology.orchestra.run.smithi027.stdout: osd debug op order = true 2024-09-12T03:15:32.331 INFO:teuthology.orchestra.run.smithi027.stdout: osd debug verify stray on activate = true 2024-09-12T03:15:32.331 INFO:teuthology.orchestra.run.smithi027.stdout: osd debug trim objects = true 2024-09-12T03:15:32.331 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:15:32.331 INFO:teuthology.orchestra.run.smithi027.stdout: osd open classes on start = true 2024-09-12T03:15:32.331 INFO:teuthology.orchestra.run.smithi027.stdout: osd debug pg log writeout = true 2024-09-12T03:15:32.331 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:15:32.331 INFO:teuthology.orchestra.run.smithi027.stdout: osd deep scrub update digest min age = 30 2024-09-12T03:15:32.331 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:15:32.331 INFO:teuthology.orchestra.run.smithi027.stdout: osd map max advance = 10 2024-09-12T03:15:32.331 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:15:32.331 INFO:teuthology.orchestra.run.smithi027.stdout: journal zero on create = true 2024-09-12T03:15:32.331 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:15:32.331 INFO:teuthology.orchestra.run.smithi027.stdout: filestore ondisk finisher threads = 3 2024-09-12T03:15:32.332 INFO:teuthology.orchestra.run.smithi027.stdout: filestore apply finisher threads = 3 2024-09-12T03:15:32.332 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:15:32.332 INFO:teuthology.orchestra.run.smithi027.stdout: bdev debug aio = true 2024-09-12T03:15:32.332 INFO:teuthology.orchestra.run.smithi027.stdout: osd debug misdirected ops = true 2024-09-12T03:15:32.332 INFO:teuthology.orchestra.run.smithi027.stdout: bdev async discard = True 2024-09-12T03:15:32.332 INFO:teuthology.orchestra.run.smithi027.stdout: bdev enable discard = True 2024-09-12T03:15:32.332 INFO:teuthology.orchestra.run.smithi027.stdout: bluestore allocator = bitmap 2024-09-12T03:15:32.332 INFO:teuthology.orchestra.run.smithi027.stdout: bluestore block size = 96636764160 2024-09-12T03:15:32.332 INFO:teuthology.orchestra.run.smithi027.stdout: bluestore fsck on mount = True 2024-09-12T03:15:32.332 INFO:teuthology.orchestra.run.smithi027.stdout: debug bluefs = 1/20 2024-09-12T03:15:32.332 INFO:teuthology.orchestra.run.smithi027.stdout: debug bluestore = 1/20 2024-09-12T03:15:32.332 INFO:teuthology.orchestra.run.smithi027.stdout: debug ms = 1 2024-09-12T03:15:32.332 INFO:teuthology.orchestra.run.smithi027.stdout: debug osd = 20 2024-09-12T03:15:32.333 INFO:teuthology.orchestra.run.smithi027.stdout: debug rocksdb = 4/10 2024-09-12T03:15:32.333 INFO:teuthology.orchestra.run.smithi027.stdout: mon osd backfillfull_ratio = 0.85 2024-09-12T03:15:32.333 INFO:teuthology.orchestra.run.smithi027.stdout: mon osd full ratio = 0.9 2024-09-12T03:15:32.333 INFO:teuthology.orchestra.run.smithi027.stdout: mon osd nearfull ratio = 0.8 2024-09-12T03:15:32.333 INFO:teuthology.orchestra.run.smithi027.stdout: osd failsafe full ratio = 0.95 2024-09-12T03:15:32.333 INFO:teuthology.orchestra.run.smithi027.stdout: osd objectstore = bluestore 2024-09-12T03:15:32.333 INFO:teuthology.orchestra.run.smithi027.stdout: osd op complaint time = 180 2024-09-12T03:15:32.333 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:15:32.333 INFO:teuthology.orchestra.run.smithi027.stdout:[mgr] 2024-09-12T03:15:32.333 INFO:teuthology.orchestra.run.smithi027.stdout: debug ms = 1 2024-09-12T03:15:32.333 INFO:teuthology.orchestra.run.smithi027.stdout: debug mgr = 20 2024-09-12T03:15:32.333 INFO:teuthology.orchestra.run.smithi027.stdout: debug mon = 20 2024-09-12T03:15:32.333 INFO:teuthology.orchestra.run.smithi027.stdout: debug auth = 20 2024-09-12T03:15:32.333 INFO:teuthology.orchestra.run.smithi027.stdout: mon reweight min pgs per osd = 4 2024-09-12T03:15:32.334 INFO:teuthology.orchestra.run.smithi027.stdout: mon reweight min bytes per osd = 10 2024-09-12T03:15:32.334 INFO:teuthology.orchestra.run.smithi027.stdout: mgr/telemetry/nag = false 2024-09-12T03:15:32.334 INFO:teuthology.orchestra.run.smithi027.stdout: debug client = 20 2024-09-12T03:15:32.334 INFO:teuthology.orchestra.run.smithi027.stdout: debug finisher = 20 2024-09-12T03:15:32.334 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:15:32.334 INFO:teuthology.orchestra.run.smithi027.stdout:[mon] 2024-09-12T03:15:32.334 INFO:teuthology.orchestra.run.smithi027.stdout: debug ms = 1 2024-09-12T03:15:32.334 INFO:teuthology.orchestra.run.smithi027.stdout: debug mon = 20 2024-09-12T03:15:32.334 INFO:teuthology.orchestra.run.smithi027.stdout: debug paxos = 20 2024-09-12T03:15:32.334 INFO:teuthology.orchestra.run.smithi027.stdout: debug auth = 20 2024-09-12T03:15:32.334 INFO:teuthology.orchestra.run.smithi027.stdout: mon data avail warn = 5 2024-09-12T03:15:32.334 INFO:teuthology.orchestra.run.smithi027.stdout: mon mgr mkfs grace = 240 2024-09-12T03:15:32.334 INFO:teuthology.orchestra.run.smithi027.stdout: mon reweight min pgs per osd = 4 2024-09-12T03:15:32.335 INFO:teuthology.orchestra.run.smithi027.stdout: mon osd reporter subtree level = osd 2024-09-12T03:15:32.335 INFO:teuthology.orchestra.run.smithi027.stdout: mon osd prime pg temp = true 2024-09-12T03:15:32.335 INFO:teuthology.orchestra.run.smithi027.stdout: mon reweight min bytes per osd = 10 2024-09-12T03:15:32.335 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:15:32.335 INFO:teuthology.orchestra.run.smithi027.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-12T03:15:32.335 INFO:teuthology.orchestra.run.smithi027.stdout: auth mon ticket ttl = 660 # 11m 2024-09-12T03:15:32.335 INFO:teuthology.orchestra.run.smithi027.stdout: auth service ticket ttl = 240 # 4m 2024-09-12T03:15:32.335 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:15:32.335 INFO:teuthology.orchestra.run.smithi027.stdout: # don't complain about insecure global_id in the test suite 2024-09-12T03:15:32.335 INFO:teuthology.orchestra.run.smithi027.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-12T03:15:32.335 INFO:teuthology.orchestra.run.smithi027.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-12T03:15:32.335 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:15:32.335 INFO:teuthology.orchestra.run.smithi027.stdout: # 1m isn't quite enough 2024-09-12T03:15:32.336 INFO:teuthology.orchestra.run.smithi027.stdout: mon_down_mkfs_grace = 2m 2024-09-12T03:15:32.336 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:15:32.336 INFO:teuthology.orchestra.run.smithi027.stdout: mon_warn_on_filestore_osds = false 2024-09-12T03:15:32.336 INFO:teuthology.orchestra.run.smithi027.stdout: mon down mkfs grace = 300 2024-09-12T03:15:32.336 INFO:teuthology.orchestra.run.smithi027.stdout: mon op complaint time = 120 2024-09-12T03:15:32.336 INFO:teuthology.orchestra.run.smithi027.stdout: mon warn on pool no app = False 2024-09-12T03:15:32.336 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:15:32.336 INFO:teuthology.orchestra.run.smithi027.stdout:[client] 2024-09-12T03:15:32.336 INFO:teuthology.orchestra.run.smithi027.stdout: rgw cache enabled = true 2024-09-12T03:15:32.336 INFO:teuthology.orchestra.run.smithi027.stdout: rgw enable ops log = true 2024-09-12T03:15:32.336 INFO:teuthology.orchestra.run.smithi027.stdout: rgw enable usage log = true 2024-09-12T03:15:32.336 INFO:teuthology.orchestra.run.smithi027.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-12T03:15:32.336 INFO:teuthology.orchestra.run.smithi027.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-12T03:15:32.336 INFO:teuthology.orchestra.run.smithi027.stdout: client mount timeout = 600 2024-09-12T03:15:32.337 INFO:teuthology.orchestra.run.smithi027.stdout: debug client = 20 2024-09-12T03:15:32.337 INFO:teuthology.orchestra.run.smithi027.stdout: debug ms = 1 2024-09-12T03:15:32.337 INFO:teuthology.orchestra.run.smithi027.stdout: pid file = "" 2024-09-12T03:15:32.337 INFO:teuthology.orchestra.run.smithi027.stdout: rados mon op timeout = 900 2024-09-12T03:15:32.337 INFO:teuthology.orchestra.run.smithi027.stdout: rados osd op timeout = 900 2024-09-12T03:15:32.337 INFO:teuthology.orchestra.run.smithi027.stdout:[mon.a] 2024-09-12T03:15:32.337 INFO:teuthology.orchestra.run.smithi027.stdout:[mon.b] 2024-09-12T03:15:32.337 INFO:teuthology.orchestra.run.smithi027.stdout:[mon.c] 2024-09-12T03:15:32.337 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.a] 2024-09-12T03:15:32.337 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.c] 2024-09-12T03:15:32.337 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.b] 2024-09-12T03:15:32.337 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.d] 2024-09-12T03:15:32.337 INFO:teuthology.orchestra.run.smithi027.stdout:[mds] 2024-09-12T03:15:32.338 INFO:teuthology.orchestra.run.smithi027.stdout: debug mds = 20 2024-09-12T03:15:32.338 INFO:teuthology.orchestra.run.smithi027.stdout: debug mds balancer = 20 2024-09-12T03:15:32.338 INFO:teuthology.orchestra.run.smithi027.stdout: debug ms = 1 2024-09-12T03:15:32.338 INFO:teuthology.orchestra.run.smithi027.stdout: mds debug frag = True 2024-09-12T03:15:32.338 INFO:teuthology.orchestra.run.smithi027.stdout: mds debug scatterstat = True 2024-09-12T03:15:32.338 INFO:teuthology.orchestra.run.smithi027.stdout: mds op complaint time = 180 2024-09-12T03:15:32.338 INFO:teuthology.orchestra.run.smithi027.stdout: mds verify scatter = True 2024-09-12T03:15:32.338 INFO:teuthology.orchestra.run.smithi027.stdout: osd op complaint time = 180 2024-09-12T03:15:32.338 INFO:teuthology.orchestra.run.smithi027.stdout: rados mon op timeout = 900 2024-09-12T03:15:32.338 INFO:teuthology.orchestra.run.smithi027.stdout: rados osd op timeout = 900 2024-09-12T03:15:32.339 INFO:teuthology.orchestra.run.smithi045.stdout:[global] 2024-09-12T03:15:32.339 INFO:teuthology.orchestra.run.smithi045.stdout: chdir = "" 2024-09-12T03:15:32.339 INFO:teuthology.orchestra.run.smithi045.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-12T03:15:32.339 INFO:teuthology.orchestra.run.smithi045.stdout: auth supported = cephx 2024-09-12T03:15:32.339 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-12T03:15:32.339 INFO:teuthology.orchestra.run.smithi045.stdout: filestore xattr use omap = true 2024-09-12T03:15:32.339 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-12T03:15:32.339 INFO:teuthology.orchestra.run.smithi045.stdout: mon clock drift allowed = 1.000 2024-09-12T03:15:32.339 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-12T03:15:32.339 INFO:teuthology.orchestra.run.smithi045.stdout: osd crush chooseleaf type = 0 2024-09-12T03:15:32.340 INFO:teuthology.orchestra.run.smithi045.stdout: auth debug = true 2024-09-12T03:15:32.340 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-12T03:15:32.340 INFO:teuthology.orchestra.run.smithi045.stdout: ms die on old message = true 2024-09-12T03:15:32.340 INFO:teuthology.orchestra.run.smithi045.stdout: ms die on bug = true 2024-09-12T03:15:32.340 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-12T03:15:32.340 INFO:teuthology.orchestra.run.smithi045.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-12T03:15:32.340 INFO:teuthology.orchestra.run.smithi045.stdout: mon pg warn max object skew = 0 2024-09-12T03:15:32.340 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-12T03:15:32.340 INFO:teuthology.orchestra.run.smithi045.stdout: # disable pg_autoscaler by default for new pools 2024-09-12T03:15:32.340 INFO:teuthology.orchestra.run.smithi045.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-12T03:15:32.340 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-12T03:15:32.340 INFO:teuthology.orchestra.run.smithi045.stdout: osd pool default size = 2 2024-09-12T03:15:32.340 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-12T03:15:32.341 INFO:teuthology.orchestra.run.smithi045.stdout: mon osd allow primary affinity = true 2024-09-12T03:15:32.341 INFO:teuthology.orchestra.run.smithi045.stdout: mon osd allow pg remap = true 2024-09-12T03:15:32.341 INFO:teuthology.orchestra.run.smithi045.stdout: mon warn on legacy crush tunables = false 2024-09-12T03:15:32.341 INFO:teuthology.orchestra.run.smithi045.stdout: mon warn on crush straw calc version zero = false 2024-09-12T03:15:32.341 INFO:teuthology.orchestra.run.smithi045.stdout: mon warn on no sortbitwise = false 2024-09-12T03:15:32.341 INFO:teuthology.orchestra.run.smithi045.stdout: mon warn on osd down out interval zero = false 2024-09-12T03:15:32.341 INFO:teuthology.orchestra.run.smithi045.stdout: mon warn on too few osds = false 2024-09-12T03:15:32.341 INFO:teuthology.orchestra.run.smithi045.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-12T03:15:32.341 INFO:teuthology.orchestra.run.smithi045.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-12T03:15:32.341 INFO:teuthology.orchestra.run.smithi045.stdout: mon_allow_pool_size_one = true 2024-09-12T03:15:32.341 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-12T03:15:32.341 INFO:teuthology.orchestra.run.smithi045.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-12T03:15:32.341 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-12T03:15:32.341 INFO:teuthology.orchestra.run.smithi045.stdout: osd default data pool replay window = 5 2024-09-12T03:15:32.342 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-12T03:15:32.342 INFO:teuthology.orchestra.run.smithi045.stdout: mon allow pool delete = true 2024-09-12T03:15:32.342 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-12T03:15:32.342 INFO:teuthology.orchestra.run.smithi045.stdout: mon cluster log file level = debug 2024-09-12T03:15:32.342 INFO:teuthology.orchestra.run.smithi045.stdout: debug asserts on shutdown = true 2024-09-12T03:15:32.342 INFO:teuthology.orchestra.run.smithi045.stdout: mon health detail to clog = false 2024-09-12T03:15:32.342 INFO:teuthology.orchestra.run.smithi045.stdout: mon host = "172.21.15.27,172.21.15.45,[v2:172.21.15.45:3301,v1:172.21.15.45:6790]" 2024-09-12T03:15:32.342 INFO:teuthology.orchestra.run.smithi045.stdout: ms die on skipped message = False 2024-09-12T03:15:32.342 INFO:teuthology.orchestra.run.smithi045.stdout: fsid = d2df504e-2647-4385-a5db-9fdf69f487a0 2024-09-12T03:15:32.342 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-12T03:15:32.342 INFO:teuthology.orchestra.run.smithi045.stdout:[osd] 2024-09-12T03:15:32.342 INFO:teuthology.orchestra.run.smithi045.stdout: osd journal size = 100 2024-09-12T03:15:32.342 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-12T03:15:32.343 INFO:teuthology.orchestra.run.smithi045.stdout: osd scrub load threshold = 5.0 2024-09-12T03:15:32.343 INFO:teuthology.orchestra.run.smithi045.stdout: osd scrub max interval = 600 2024-09-12T03:15:32.343 INFO:teuthology.orchestra.run.smithi045.stdout: osd mclock profile = high_recovery_ops 2024-09-12T03:15:32.343 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-12T03:15:32.343 INFO:teuthology.orchestra.run.smithi045.stdout: osd recover clone overlap = true 2024-09-12T03:15:32.343 INFO:teuthology.orchestra.run.smithi045.stdout: osd recovery max chunk = 1048576 2024-09-12T03:15:32.343 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-12T03:15:32.343 INFO:teuthology.orchestra.run.smithi045.stdout: osd debug shutdown = true 2024-09-12T03:15:32.343 INFO:teuthology.orchestra.run.smithi045.stdout: osd debug op order = true 2024-09-12T03:15:32.343 INFO:teuthology.orchestra.run.smithi045.stdout: osd debug verify stray on activate = true 2024-09-12T03:15:32.343 INFO:teuthology.orchestra.run.smithi045.stdout: osd debug trim objects = true 2024-09-12T03:15:32.343 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-12T03:15:32.343 INFO:teuthology.orchestra.run.smithi045.stdout: osd open classes on start = true 2024-09-12T03:15:32.344 INFO:teuthology.orchestra.run.smithi045.stdout: osd debug pg log writeout = true 2024-09-12T03:15:32.344 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-12T03:15:32.344 INFO:teuthology.orchestra.run.smithi045.stdout: osd deep scrub update digest min age = 30 2024-09-12T03:15:32.344 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-12T03:15:32.344 INFO:teuthology.orchestra.run.smithi045.stdout: osd map max advance = 10 2024-09-12T03:15:32.344 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-12T03:15:32.344 INFO:teuthology.orchestra.run.smithi045.stdout: journal zero on create = true 2024-09-12T03:15:32.344 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-12T03:15:32.344 INFO:teuthology.orchestra.run.smithi045.stdout: filestore ondisk finisher threads = 3 2024-09-12T03:15:32.344 INFO:teuthology.orchestra.run.smithi045.stdout: filestore apply finisher threads = 3 2024-09-12T03:15:32.344 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-12T03:15:32.344 INFO:teuthology.orchestra.run.smithi045.stdout: bdev debug aio = true 2024-09-12T03:15:32.345 INFO:teuthology.orchestra.run.smithi045.stdout: osd debug misdirected ops = true 2024-09-12T03:15:32.345 INFO:teuthology.orchestra.run.smithi045.stdout: bdev async discard = True 2024-09-12T03:15:32.345 INFO:teuthology.orchestra.run.smithi045.stdout: bdev enable discard = True 2024-09-12T03:15:32.345 INFO:teuthology.orchestra.run.smithi045.stdout: bluestore allocator = bitmap 2024-09-12T03:15:32.345 INFO:teuthology.orchestra.run.smithi045.stdout: bluestore block size = 96636764160 2024-09-12T03:15:32.345 INFO:teuthology.orchestra.run.smithi045.stdout: bluestore fsck on mount = True 2024-09-12T03:15:32.345 INFO:teuthology.orchestra.run.smithi045.stdout: debug bluefs = 1/20 2024-09-12T03:15:32.345 INFO:teuthology.orchestra.run.smithi045.stdout: debug bluestore = 1/20 2024-09-12T03:15:32.345 INFO:teuthology.orchestra.run.smithi045.stdout: debug ms = 1 2024-09-12T03:15:32.345 INFO:teuthology.orchestra.run.smithi045.stdout: debug osd = 20 2024-09-12T03:15:32.345 INFO:teuthology.orchestra.run.smithi045.stdout: debug rocksdb = 4/10 2024-09-12T03:15:32.345 INFO:teuthology.orchestra.run.smithi045.stdout: mon osd backfillfull_ratio = 0.85 2024-09-12T03:15:32.345 INFO:teuthology.orchestra.run.smithi045.stdout: mon osd full ratio = 0.9 2024-09-12T03:15:32.346 INFO:teuthology.orchestra.run.smithi045.stdout: mon osd nearfull ratio = 0.8 2024-09-12T03:15:32.346 INFO:teuthology.orchestra.run.smithi045.stdout: osd failsafe full ratio = 0.95 2024-09-12T03:15:32.346 INFO:teuthology.orchestra.run.smithi045.stdout: osd objectstore = bluestore 2024-09-12T03:15:32.346 INFO:teuthology.orchestra.run.smithi045.stdout: osd op complaint time = 180 2024-09-12T03:15:32.346 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-12T03:15:32.346 INFO:teuthology.orchestra.run.smithi045.stdout:[mgr] 2024-09-12T03:15:32.346 INFO:teuthology.orchestra.run.smithi045.stdout: debug ms = 1 2024-09-12T03:15:32.346 INFO:teuthology.orchestra.run.smithi045.stdout: debug mgr = 20 2024-09-12T03:15:32.346 INFO:teuthology.orchestra.run.smithi045.stdout: debug mon = 20 2024-09-12T03:15:32.346 INFO:teuthology.orchestra.run.smithi045.stdout: debug auth = 20 2024-09-12T03:15:32.346 INFO:teuthology.orchestra.run.smithi045.stdout: mon reweight min pgs per osd = 4 2024-09-12T03:15:32.346 INFO:teuthology.orchestra.run.smithi045.stdout: mon reweight min bytes per osd = 10 2024-09-12T03:15:32.346 INFO:teuthology.orchestra.run.smithi045.stdout: mgr/telemetry/nag = false 2024-09-12T03:15:32.346 INFO:teuthology.orchestra.run.smithi045.stdout: debug client = 20 2024-09-12T03:15:32.347 INFO:teuthology.orchestra.run.smithi045.stdout: debug finisher = 20 2024-09-12T03:15:32.347 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-12T03:15:32.347 INFO:teuthology.orchestra.run.smithi045.stdout:[mon] 2024-09-12T03:15:32.347 INFO:teuthology.orchestra.run.smithi045.stdout: debug ms = 1 2024-09-12T03:15:32.347 INFO:teuthology.orchestra.run.smithi045.stdout: debug mon = 20 2024-09-12T03:15:32.347 INFO:teuthology.orchestra.run.smithi045.stdout: debug paxos = 20 2024-09-12T03:15:32.347 INFO:teuthology.orchestra.run.smithi045.stdout: debug auth = 20 2024-09-12T03:15:32.347 INFO:teuthology.orchestra.run.smithi045.stdout: mon data avail warn = 5 2024-09-12T03:15:32.347 INFO:teuthology.orchestra.run.smithi045.stdout: mon mgr mkfs grace = 240 2024-09-12T03:15:32.347 INFO:teuthology.orchestra.run.smithi045.stdout: mon reweight min pgs per osd = 4 2024-09-12T03:15:32.347 INFO:teuthology.orchestra.run.smithi045.stdout: mon osd reporter subtree level = osd 2024-09-12T03:15:32.347 INFO:teuthology.orchestra.run.smithi045.stdout: mon osd prime pg temp = true 2024-09-12T03:15:32.348 INFO:teuthology.orchestra.run.smithi045.stdout: mon reweight min bytes per osd = 10 2024-09-12T03:15:32.348 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-12T03:15:32.348 INFO:teuthology.orchestra.run.smithi045.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-12T03:15:32.348 INFO:teuthology.orchestra.run.smithi045.stdout: auth mon ticket ttl = 660 # 11m 2024-09-12T03:15:32.348 INFO:teuthology.orchestra.run.smithi045.stdout: auth service ticket ttl = 240 # 4m 2024-09-12T03:15:32.348 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-12T03:15:32.348 INFO:teuthology.orchestra.run.smithi045.stdout: # don't complain about insecure global_id in the test suite 2024-09-12T03:15:32.348 INFO:teuthology.orchestra.run.smithi045.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-12T03:15:32.348 INFO:teuthology.orchestra.run.smithi045.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-12T03:15:32.348 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-12T03:15:32.348 INFO:teuthology.orchestra.run.smithi045.stdout: # 1m isn't quite enough 2024-09-12T03:15:32.348 INFO:teuthology.orchestra.run.smithi045.stdout: mon_down_mkfs_grace = 2m 2024-09-12T03:15:32.348 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-12T03:15:32.349 INFO:teuthology.orchestra.run.smithi045.stdout: mon_warn_on_filestore_osds = false 2024-09-12T03:15:32.349 INFO:teuthology.orchestra.run.smithi045.stdout: mon down mkfs grace = 300 2024-09-12T03:15:32.349 INFO:teuthology.orchestra.run.smithi045.stdout: mon op complaint time = 120 2024-09-12T03:15:32.349 INFO:teuthology.orchestra.run.smithi045.stdout: mon warn on pool no app = False 2024-09-12T03:15:32.349 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-12T03:15:32.349 INFO:teuthology.orchestra.run.smithi045.stdout:[client] 2024-09-12T03:15:32.349 INFO:teuthology.orchestra.run.smithi045.stdout: rgw cache enabled = true 2024-09-12T03:15:32.349 INFO:teuthology.orchestra.run.smithi045.stdout: rgw enable ops log = true 2024-09-12T03:15:32.349 INFO:teuthology.orchestra.run.smithi045.stdout: rgw enable usage log = true 2024-09-12T03:15:32.349 INFO:teuthology.orchestra.run.smithi045.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-12T03:15:32.349 INFO:teuthology.orchestra.run.smithi045.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-12T03:15:32.349 INFO:teuthology.orchestra.run.smithi045.stdout: client mount timeout = 600 2024-09-12T03:15:32.349 INFO:teuthology.orchestra.run.smithi045.stdout: debug client = 20 2024-09-12T03:15:32.350 INFO:teuthology.orchestra.run.smithi045.stdout: debug ms = 1 2024-09-12T03:15:32.350 INFO:teuthology.orchestra.run.smithi045.stdout: pid file = "" 2024-09-12T03:15:32.350 INFO:teuthology.orchestra.run.smithi045.stdout: rados mon op timeout = 900 2024-09-12T03:15:32.350 INFO:teuthology.orchestra.run.smithi045.stdout: rados osd op timeout = 900 2024-09-12T03:15:32.350 INFO:teuthology.orchestra.run.smithi045.stdout:[mon.a] 2024-09-12T03:15:32.350 INFO:teuthology.orchestra.run.smithi045.stdout:[mon.b] 2024-09-12T03:15:32.350 INFO:teuthology.orchestra.run.smithi045.stdout:[mon.c] 2024-09-12T03:15:32.350 INFO:teuthology.orchestra.run.smithi045.stdout:[mds.a] 2024-09-12T03:15:32.350 INFO:teuthology.orchestra.run.smithi045.stdout:[mds.c] 2024-09-12T03:15:32.350 INFO:teuthology.orchestra.run.smithi045.stdout:[mds.b] 2024-09-12T03:15:32.350 INFO:teuthology.orchestra.run.smithi045.stdout:[mds.d] 2024-09-12T03:15:32.350 INFO:teuthology.orchestra.run.smithi045.stdout:[mds] 2024-09-12T03:15:32.350 INFO:teuthology.orchestra.run.smithi045.stdout: debug mds = 20 2024-09-12T03:15:32.350 INFO:teuthology.orchestra.run.smithi045.stdout: debug mds balancer = 20 2024-09-12T03:15:32.351 INFO:teuthology.orchestra.run.smithi045.stdout: debug ms = 1 2024-09-12T03:15:32.351 INFO:teuthology.orchestra.run.smithi045.stdout: mds debug frag = True 2024-09-12T03:15:32.351 INFO:teuthology.orchestra.run.smithi045.stdout: mds debug scatterstat = True 2024-09-12T03:15:32.351 INFO:teuthology.orchestra.run.smithi045.stdout: mds op complaint time = 180 2024-09-12T03:15:32.351 INFO:teuthology.orchestra.run.smithi045.stdout: mds verify scatter = True 2024-09-12T03:15:32.351 INFO:teuthology.orchestra.run.smithi045.stdout: osd op complaint time = 180 2024-09-12T03:15:32.351 INFO:teuthology.orchestra.run.smithi045.stdout: rados mon op timeout = 900 2024-09-12T03:15:32.351 INFO:teuthology.orchestra.run.smithi045.stdout: rados osd op timeout = 900 2024-09-12T03:15:32.352 INFO:tasks.ceph:Creating admin key on mon.a... 2024-09-12T03:15:32.353 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-09-12T03:15:32.448 INFO:tasks.ceph:Copying monmap to all nodes... 2024-09-12T03:15:32.448 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-12T03:15:32.448 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-09-12T03:15:32.464 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-12T03:15:32.464 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-09-12T03:15:32.520 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi027.front.sepia.ceph.com 2024-09-12T03:15:32.520 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-12T03:15:32.520 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-12T03:15:32.520 DEBUG:teuthology.orchestra.run.smithi027:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-12T03:15:32.595 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-12T03:15:32.595 DEBUG:teuthology.orchestra.run.smithi027:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-12T03:15:32.651 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi045.front.sepia.ceph.com 2024-09-12T03:15:32.652 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-12T03:15:32.652 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-12T03:15:32.652 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-12T03:15:32.694 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-12T03:15:32.694 DEBUG:teuthology.orchestra.run.smithi045:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-12T03:15:32.753 INFO:tasks.ceph:Setting up mon nodes... 2024-09-12T03:15:32.753 INFO:tasks.ceph:Setting up mgr nodes... 2024-09-12T03:15:32.753 DEBUG:teuthology.orchestra.run.smithi027:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-09-12T03:15:32.806 INFO:teuthology.orchestra.run.smithi027.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-09-12T03:15:32.808 DEBUG:teuthology.orchestra.run.smithi045:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-09-12T03:15:32.872 INFO:teuthology.orchestra.run.smithi045.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-09-12T03:15:32.875 INFO:tasks.ceph:Setting up mds nodes... 2024-09-12T03:15:32.875 DEBUG:teuthology.orchestra.run.smithi027:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-09-12T03:15:32.928 INFO:teuthology.orchestra.run.smithi027.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-09-12T03:15:32.931 DEBUG:teuthology.orchestra.run.smithi027:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-09-12T03:15:32.956 DEBUG:teuthology.orchestra.run.smithi027:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-09-12T03:15:33.049 INFO:teuthology.orchestra.run.smithi027.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-09-12T03:15:33.052 DEBUG:teuthology.orchestra.run.smithi027:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-09-12T03:15:33.077 DEBUG:teuthology.orchestra.run.smithi045:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-09-12T03:15:33.134 INFO:teuthology.orchestra.run.smithi045.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-09-12T03:15:33.137 DEBUG:teuthology.orchestra.run.smithi045:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-09-12T03:15:33.164 DEBUG:teuthology.orchestra.run.smithi045:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-09-12T03:15:33.259 INFO:teuthology.orchestra.run.smithi045.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-09-12T03:15:33.261 DEBUG:teuthology.orchestra.run.smithi045:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-09-12T03:15:33.287 INFO:tasks.ceph_client:Setting up client nodes... 2024-09-12T03:15:33.287 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-12T03:15:33.330 INFO:teuthology.orchestra.run.smithi027.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-09-12T03:15:33.343 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2024-09-12T03:15:33.425 INFO:teuthology.orchestra.run.smithi027.stdout:creating /etc/ceph/ceph.client.2.keyring 2024-09-12T03:15:33.438 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-09-12T03:15:33.481 INFO:teuthology.orchestra.run.smithi045.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-09-12T03:15:33.494 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2024-09-12T03:15:33.577 INFO:teuthology.orchestra.run.smithi045.stdout:creating /etc/ceph/ceph.client.3.keyring 2024-09-12T03:15:33.590 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-09-12T03:15:33.590 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi027.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi045.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-09-12T03:15:33.590 DEBUG:teuthology.orchestra.run.smithi027:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-09-12T03:15:33.616 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-12T03:15:33.616 INFO:tasks.ceph:role: osd.0 2024-09-12T03:15:33.616 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi027.front.sepia.ceph.com 2024-09-12T03:15:33.616 DEBUG:teuthology.orchestra.run.smithi027:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-12T03:15:33.689 INFO:teuthology.orchestra.run.smithi027.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-12T03:15:33.690 INFO:teuthology.orchestra.run.smithi027.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-12T03:15:33.690 INFO:teuthology.orchestra.run.smithi027.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-12T03:15:33.690 INFO:teuthology.orchestra.run.smithi027.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-12T03:15:33.690 INFO:teuthology.orchestra.run.smithi027.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-12T03:15:33.690 INFO:teuthology.orchestra.run.smithi027.stdout: = sunit=0 swidth=0 blks 2024-09-12T03:15:33.690 INFO:teuthology.orchestra.run.smithi027.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-12T03:15:33.690 INFO:teuthology.orchestra.run.smithi027.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-12T03:15:33.690 INFO:teuthology.orchestra.run.smithi027.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-12T03:15:33.690 INFO:teuthology.orchestra.run.smithi027.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-12T03:15:34.201 INFO:teuthology.orchestra.run.smithi027.stdout:Discarding blocks...Done. 2024-09-12T03:15:34.203 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi027.front.sepia.ceph.com -o noatime 2024-09-12T03:15:34.204 DEBUG:teuthology.orchestra.run.smithi027:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-09-12T03:15:34.323 DEBUG:teuthology.orchestra.run.smithi027:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-09-12T03:15:34.359 DEBUG:teuthology.orchestra.run.smithi027:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-09-12T03:15:34.427 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-12T03:15:34.427 INFO:tasks.ceph:role: osd.1 2024-09-12T03:15:34.427 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi027.front.sepia.ceph.com 2024-09-12T03:15:34.427 DEBUG:teuthology.orchestra.run.smithi027:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-12T03:15:34.502 INFO:teuthology.orchestra.run.smithi027.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-12T03:15:34.502 INFO:teuthology.orchestra.run.smithi027.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-12T03:15:34.502 INFO:teuthology.orchestra.run.smithi027.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-12T03:15:34.502 INFO:teuthology.orchestra.run.smithi027.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-12T03:15:34.502 INFO:teuthology.orchestra.run.smithi027.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-12T03:15:34.502 INFO:teuthology.orchestra.run.smithi027.stdout: = sunit=0 swidth=0 blks 2024-09-12T03:15:34.502 INFO:teuthology.orchestra.run.smithi027.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-12T03:15:34.502 INFO:teuthology.orchestra.run.smithi027.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-12T03:15:34.503 INFO:teuthology.orchestra.run.smithi027.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-12T03:15:34.503 INFO:teuthology.orchestra.run.smithi027.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-12T03:15:34.947 INFO:teuthology.orchestra.run.smithi027.stdout:Discarding blocks...Done. 2024-09-12T03:15:34.949 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi027.front.sepia.ceph.com -o noatime 2024-09-12T03:15:34.949 DEBUG:teuthology.orchestra.run.smithi027:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-09-12T03:15:35.042 DEBUG:teuthology.orchestra.run.smithi027:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-09-12T03:15:35.072 DEBUG:teuthology.orchestra.run.smithi027:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-09-12T03:15:35.137 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-12T03:15:35.137 INFO:tasks.ceph:role: osd.2 2024-09-12T03:15:35.137 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi027.front.sepia.ceph.com 2024-09-12T03:15:35.138 DEBUG:teuthology.orchestra.run.smithi027:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-12T03:15:35.211 INFO:teuthology.orchestra.run.smithi027.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-12T03:15:35.211 INFO:teuthology.orchestra.run.smithi027.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-12T03:15:35.211 INFO:teuthology.orchestra.run.smithi027.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-12T03:15:35.211 INFO:teuthology.orchestra.run.smithi027.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-12T03:15:35.211 INFO:teuthology.orchestra.run.smithi027.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-12T03:15:35.211 INFO:teuthology.orchestra.run.smithi027.stdout: = sunit=0 swidth=0 blks 2024-09-12T03:15:35.211 INFO:teuthology.orchestra.run.smithi027.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-12T03:15:35.211 INFO:teuthology.orchestra.run.smithi027.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-12T03:15:35.212 INFO:teuthology.orchestra.run.smithi027.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-12T03:15:35.212 INFO:teuthology.orchestra.run.smithi027.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-12T03:15:35.697 INFO:teuthology.orchestra.run.smithi027.stdout:Discarding blocks...Done. 2024-09-12T03:15:35.699 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi027.front.sepia.ceph.com -o noatime 2024-09-12T03:15:35.700 DEBUG:teuthology.orchestra.run.smithi027:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-09-12T03:15:35.836 DEBUG:teuthology.orchestra.run.smithi027:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-09-12T03:15:35.866 DEBUG:teuthology.orchestra.run.smithi027:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-09-12T03:15:35.932 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-12T03:15:35.932 INFO:tasks.ceph:role: osd.3 2024-09-12T03:15:35.932 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi027.front.sepia.ceph.com 2024-09-12T03:15:35.932 DEBUG:teuthology.orchestra.run.smithi027:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-12T03:15:36.004 INFO:teuthology.orchestra.run.smithi027.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-12T03:15:36.005 INFO:teuthology.orchestra.run.smithi027.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-12T03:15:36.005 INFO:teuthology.orchestra.run.smithi027.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-12T03:15:36.005 INFO:teuthology.orchestra.run.smithi027.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-12T03:15:36.005 INFO:teuthology.orchestra.run.smithi027.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-12T03:15:36.005 INFO:teuthology.orchestra.run.smithi027.stdout: = sunit=0 swidth=0 blks 2024-09-12T03:15:36.005 INFO:teuthology.orchestra.run.smithi027.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-12T03:15:36.005 INFO:teuthology.orchestra.run.smithi027.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-12T03:15:36.005 INFO:teuthology.orchestra.run.smithi027.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-12T03:15:36.005 INFO:teuthology.orchestra.run.smithi027.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-12T03:15:36.515 INFO:teuthology.orchestra.run.smithi027.stdout:Discarding blocks...Done. 2024-09-12T03:15:36.518 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi027.front.sepia.ceph.com -o noatime 2024-09-12T03:15:36.518 DEBUG:teuthology.orchestra.run.smithi027:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-09-12T03:15:36.653 DEBUG:teuthology.orchestra.run.smithi027:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-09-12T03:15:36.686 DEBUG:teuthology.orchestra.run.smithi027:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-12T03:15:36.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:15:36.792+0000 7f4daa141640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-09-12T03:15:36.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:15:36.793+0000 7f4daa141640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-09-12T03:15:36.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:15:36.793+0000 7f4daa141640 -1 bdev(0x56223f0dd800 /var/lib/ceph/osd/ceph-0/block) open stat got: (1) Operation not permitted 2024-09-12T03:15:36.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:15:36.793+0000 7f4daa141640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-09-12T03:15:40.212 DEBUG:teuthology.orchestra.run.smithi027:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-09-12T03:15:40.243 DEBUG:teuthology.orchestra.run.smithi027:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-12T03:15:40.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:15:40.335+0000 7f9d527dd640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-09-12T03:15:40.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:15:40.335+0000 7f9d527dd640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-09-12T03:15:40.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:15:40.335+0000 7f9d527dd640 -1 bdev(0x564a81c77800 /var/lib/ceph/osd/ceph-1/block) open stat got: (1) Operation not permitted 2024-09-12T03:15:40.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:15:40.335+0000 7f9d527dd640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-09-12T03:15:43.500 DEBUG:teuthology.orchestra.run.smithi027:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-09-12T03:15:43.531 DEBUG:teuthology.orchestra.run.smithi027:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-12T03:15:43.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:15:43.626+0000 7f4f284e2640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-09-12T03:15:43.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:15:43.626+0000 7f4f284e2640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-09-12T03:15:43.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:15:43.626+0000 7f4f284e2640 -1 bdev(0x557a035b9800 /var/lib/ceph/osd/ceph-2/block) open stat got: (1) Operation not permitted 2024-09-12T03:15:43.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:15:43.627+0000 7f4f284e2640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-09-12T03:15:46.551 DEBUG:teuthology.orchestra.run.smithi027:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-09-12T03:15:46.578 DEBUG:teuthology.orchestra.run.smithi027:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-12T03:15:46.667 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:15:46.667+0000 7fd06a345640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-09-12T03:15:46.667 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:15:46.667+0000 7fd06a345640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-09-12T03:15:46.667 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:15:46.667+0000 7fd06a345640 -1 bdev(0x55da02f19800 /var/lib/ceph/osd/ceph-3/block) open stat got: (1) Operation not permitted 2024-09-12T03:15:46.667 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:15:46.667+0000 7fd06a345640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-09-12T03:15:50.580 DEBUG:teuthology.orchestra.run.smithi027:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-09-12T03:15:50.611 DEBUG:teuthology.orchestra.run.smithi045:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-09-12T03:15:50.644 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-12T03:15:50.644 INFO:tasks.ceph:role: osd.4 2024-09-12T03:15:50.644 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi045.front.sepia.ceph.com 2024-09-12T03:15:50.644 DEBUG:teuthology.orchestra.run.smithi045:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-12T03:15:50.713 INFO:teuthology.orchestra.run.smithi045.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-12T03:15:50.713 INFO:teuthology.orchestra.run.smithi045.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-12T03:15:50.713 INFO:teuthology.orchestra.run.smithi045.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-12T03:15:50.713 INFO:teuthology.orchestra.run.smithi045.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-12T03:15:50.713 INFO:teuthology.orchestra.run.smithi045.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-12T03:15:50.714 INFO:teuthology.orchestra.run.smithi045.stdout: = sunit=0 swidth=0 blks 2024-09-12T03:15:50.714 INFO:teuthology.orchestra.run.smithi045.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-12T03:15:50.714 INFO:teuthology.orchestra.run.smithi045.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-12T03:15:50.714 INFO:teuthology.orchestra.run.smithi045.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-12T03:15:50.714 INFO:teuthology.orchestra.run.smithi045.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-12T03:15:50.746 INFO:teuthology.orchestra.run.smithi045.stdout:Discarding blocks...Done. 2024-09-12T03:15:50.749 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi045.front.sepia.ceph.com -o noatime 2024-09-12T03:15:50.749 DEBUG:teuthology.orchestra.run.smithi045:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-09-12T03:15:50.820 DEBUG:teuthology.orchestra.run.smithi045:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-09-12T03:15:50.891 DEBUG:teuthology.orchestra.run.smithi045:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-09-12T03:15:50.958 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-12T03:15:50.958 INFO:tasks.ceph:role: osd.5 2024-09-12T03:15:50.958 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi045.front.sepia.ceph.com 2024-09-12T03:15:50.959 DEBUG:teuthology.orchestra.run.smithi045:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-12T03:15:51.025 INFO:teuthology.orchestra.run.smithi045.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-12T03:15:51.025 INFO:teuthology.orchestra.run.smithi045.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-12T03:15:51.025 INFO:teuthology.orchestra.run.smithi045.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-12T03:15:51.025 INFO:teuthology.orchestra.run.smithi045.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-12T03:15:51.026 INFO:teuthology.orchestra.run.smithi045.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-12T03:15:51.026 INFO:teuthology.orchestra.run.smithi045.stdout: = sunit=0 swidth=0 blks 2024-09-12T03:15:51.026 INFO:teuthology.orchestra.run.smithi045.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-12T03:15:51.026 INFO:teuthology.orchestra.run.smithi045.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-12T03:15:51.026 INFO:teuthology.orchestra.run.smithi045.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-12T03:15:51.026 INFO:teuthology.orchestra.run.smithi045.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-12T03:15:51.058 INFO:teuthology.orchestra.run.smithi045.stdout:Discarding blocks...Done. 2024-09-12T03:15:51.059 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi045.front.sepia.ceph.com -o noatime 2024-09-12T03:15:51.060 DEBUG:teuthology.orchestra.run.smithi045:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-09-12T03:15:51.131 DEBUG:teuthology.orchestra.run.smithi045:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-09-12T03:15:51.202 DEBUG:teuthology.orchestra.run.smithi045:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-09-12T03:15:51.269 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-12T03:15:51.269 INFO:tasks.ceph:role: osd.6 2024-09-12T03:15:51.269 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi045.front.sepia.ceph.com 2024-09-12T03:15:51.269 DEBUG:teuthology.orchestra.run.smithi045:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-12T03:15:51.334 INFO:teuthology.orchestra.run.smithi045.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-12T03:15:51.334 INFO:teuthology.orchestra.run.smithi045.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-12T03:15:51.334 INFO:teuthology.orchestra.run.smithi045.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-12T03:15:51.334 INFO:teuthology.orchestra.run.smithi045.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-12T03:15:51.334 INFO:teuthology.orchestra.run.smithi045.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-12T03:15:51.334 INFO:teuthology.orchestra.run.smithi045.stdout: = sunit=0 swidth=0 blks 2024-09-12T03:15:51.335 INFO:teuthology.orchestra.run.smithi045.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-12T03:15:51.335 INFO:teuthology.orchestra.run.smithi045.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-12T03:15:51.335 INFO:teuthology.orchestra.run.smithi045.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-12T03:15:51.335 INFO:teuthology.orchestra.run.smithi045.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-12T03:15:51.372 INFO:teuthology.orchestra.run.smithi045.stdout:Discarding blocks...Done. 2024-09-12T03:15:51.374 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi045.front.sepia.ceph.com -o noatime 2024-09-12T03:15:51.374 DEBUG:teuthology.orchestra.run.smithi045:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-09-12T03:15:51.444 DEBUG:teuthology.orchestra.run.smithi045:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-09-12T03:15:51.516 DEBUG:teuthology.orchestra.run.smithi045:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-09-12T03:15:51.581 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-12T03:15:51.581 INFO:tasks.ceph:role: osd.7 2024-09-12T03:15:51.582 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi045.front.sepia.ceph.com 2024-09-12T03:15:51.582 DEBUG:teuthology.orchestra.run.smithi045:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-12T03:15:51.647 INFO:teuthology.orchestra.run.smithi045.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-12T03:15:51.647 INFO:teuthology.orchestra.run.smithi045.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-12T03:15:51.647 INFO:teuthology.orchestra.run.smithi045.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-12T03:15:51.647 INFO:teuthology.orchestra.run.smithi045.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-12T03:15:51.647 INFO:teuthology.orchestra.run.smithi045.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-12T03:15:51.647 INFO:teuthology.orchestra.run.smithi045.stdout: = sunit=0 swidth=0 blks 2024-09-12T03:15:51.647 INFO:teuthology.orchestra.run.smithi045.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-12T03:15:51.647 INFO:teuthology.orchestra.run.smithi045.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-12T03:15:51.647 INFO:teuthology.orchestra.run.smithi045.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-12T03:15:51.647 INFO:teuthology.orchestra.run.smithi045.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-12T03:15:51.680 INFO:teuthology.orchestra.run.smithi045.stdout:Discarding blocks...Done. 2024-09-12T03:15:51.682 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi045.front.sepia.ceph.com -o noatime 2024-09-12T03:15:51.682 DEBUG:teuthology.orchestra.run.smithi045:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-09-12T03:15:51.752 DEBUG:teuthology.orchestra.run.smithi045:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-09-12T03:15:51.821 DEBUG:teuthology.orchestra.run.smithi045:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-12T03:15:51.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:15:51.938+0000 7f8c2060f640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-09-12T03:15:51.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:15:51.938+0000 7f8c2060f640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-09-12T03:15:51.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:15:51.938+0000 7f8c2060f640 -1 bdev(0x56347eaf5800 /var/lib/ceph/osd/ceph-4/block) open stat got: (1) Operation not permitted 2024-09-12T03:15:51.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:15:51.938+0000 7f8c2060f640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-09-12T03:15:54.866 DEBUG:teuthology.orchestra.run.smithi045:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-09-12T03:15:54.896 DEBUG:teuthology.orchestra.run.smithi045:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-12T03:15:54.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:15:54.990+0000 7ff1c080c640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-09-12T03:15:54.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:15:54.990+0000 7ff1c080c640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-09-12T03:15:54.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:15:54.990+0000 7ff1c080c640 -1 bdev(0x556d9176f800 /var/lib/ceph/osd/ceph-5/block) open stat got: (1) Operation not permitted 2024-09-12T03:15:54.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:15:54.990+0000 7ff1c080c640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-09-12T03:15:58.661 DEBUG:teuthology.orchestra.run.smithi045:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-09-12T03:15:58.693 DEBUG:teuthology.orchestra.run.smithi045:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-12T03:15:58.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:15:58.792+0000 7fcc34b58640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-09-12T03:15:58.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:15:58.792+0000 7fcc34b58640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-09-12T03:15:58.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:15:58.792+0000 7fcc34b58640 -1 bdev(0x55b0eff87800 /var/lib/ceph/osd/ceph-6/block) open stat got: (1) Operation not permitted 2024-09-12T03:15:58.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:15:58.792+0000 7fcc34b58640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-09-12T03:16:00.323 DEBUG:teuthology.orchestra.run.smithi027:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-12T03:16:00.326 DEBUG:teuthology.orchestra.run.smithi045:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-12T03:16:01.709 DEBUG:teuthology.orchestra.run.smithi045:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-09-12T03:16:01.740 DEBUG:teuthology.orchestra.run.smithi045:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-12T03:16:01.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:01.834+0000 7feaf6dbd640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-09-12T03:16:01.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:01.834+0000 7feaf6dbd640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-09-12T03:16:01.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:01.834+0000 7feaf6dbd640 -1 bdev(0x559304aa3800 /var/lib/ceph/osd/ceph-7/block) open stat got: (1) Operation not permitted 2024-09-12T03:16:01.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:01.834+0000 7feaf6dbd640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-09-12T03:16:04.748 DEBUG:teuthology.orchestra.run.smithi045:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-09-12T03:16:04.780 INFO:tasks.ceph:Reading keys from all nodes... 2024-09-12T03:16:04.780 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-12T03:16:04.780 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-09-12T03:16:04.812 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-12T03:16:04.812 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-09-12T03:16:04.883 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-12T03:16:04.883 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-09-12T03:16:04.953 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-12T03:16:04.953 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-09-12T03:16:05.022 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-12T03:16:05.022 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-09-12T03:16:05.091 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-12T03:16:05.091 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-09-12T03:16:05.159 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-12T03:16:05.160 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-09-12T03:16:05.226 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-12T03:16:05.227 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-09-12T03:16:05.258 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-12T03:16:05.258 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-09-12T03:16:05.328 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-12T03:16:05.329 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-09-12T03:16:05.400 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-12T03:16:05.400 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-09-12T03:16:05.469 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-12T03:16:05.469 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-09-12T03:16:05.538 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-12T03:16:05.539 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-09-12T03:16:05.606 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-12T03:16:05.606 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-09-12T03:16:05.673 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-12T03:16:05.674 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-09-12T03:16:05.692 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-12T03:16:05.692 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2024-09-12T03:16:05.749 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-12T03:16:05.749 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-09-12T03:16:05.765 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-12T03:16:05.765 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2024-09-12T03:16:05.822 INFO:tasks.ceph:Adding keys to all mons... 2024-09-12T03:16:05.822 DEBUG:teuthology.orchestra.run.smithi027:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-12T03:16:05.824 DEBUG:teuthology.orchestra.run.smithi045:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-12T03:16:05.865 INFO:teuthology.orchestra.run.smithi027.stdout:[mgr.y] 2024-09-12T03:16:05.865 INFO:teuthology.orchestra.run.smithi027.stdout: key = AQDUXOJmJOgZMBAAQ/JTxfhox7HgxBMtsvUCBg== 2024-09-12T03:16:05.865 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.a] 2024-09-12T03:16:05.865 INFO:teuthology.orchestra.run.smithi027.stdout: key = AQDUXOJmIhxjNxAAaGg8SwtW6NRe/lIgNCV20w== 2024-09-12T03:16:05.865 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.c] 2024-09-12T03:16:05.865 INFO:teuthology.orchestra.run.smithi027.stdout: key = AQDVXOJm0sX7AhAAdynCcho2a/AgFF9Cw3+vAw== 2024-09-12T03:16:05.865 INFO:teuthology.orchestra.run.smithi027.stdout:[osd.0] 2024-09-12T03:16:05.865 INFO:teuthology.orchestra.run.smithi027.stdout: key = AQDYXOJmJTVYLxAAmo8kxRjaiwTT44dZLZlq4A== 2024-09-12T03:16:05.866 INFO:teuthology.orchestra.run.smithi027.stdout:[osd.1] 2024-09-12T03:16:05.866 INFO:teuthology.orchestra.run.smithi027.stdout: key = AQDcXOJmYV0BFBAAmoGX9gf9yLoE/qkWFnv9tg== 2024-09-12T03:16:05.866 INFO:teuthology.orchestra.run.smithi027.stdout:[osd.2] 2024-09-12T03:16:05.866 INFO:teuthology.orchestra.run.smithi027.stdout: key = AQDfXOJmEetiJRAAXk+AvfTfhjsB6T2UB4Z2/w== 2024-09-12T03:16:05.866 INFO:teuthology.orchestra.run.smithi027.stdout:[osd.3] 2024-09-12T03:16:05.866 INFO:teuthology.orchestra.run.smithi027.stdout: key = AQDiXOJmrQbPJxAAQsi2B5a1LnPTPGZVdo7j7w== 2024-09-12T03:16:05.866 INFO:teuthology.orchestra.run.smithi027.stdout:[mgr.x] 2024-09-12T03:16:05.866 INFO:teuthology.orchestra.run.smithi027.stdout: key = AQDUXOJmZcsONBAAMTY2KykuHaEkv1t6Vz5KjA== 2024-09-12T03:16:05.866 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.b] 2024-09-12T03:16:05.866 INFO:teuthology.orchestra.run.smithi027.stdout: key = AQDVXOJm63kWCBAAzjfOsBsWUI+IAsuZ49yEkw== 2024-09-12T03:16:05.866 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.d] 2024-09-12T03:16:05.866 INFO:teuthology.orchestra.run.smithi027.stdout: key = AQDVXOJm7Dd8DxAANHYmvKgpRZ6sVfXDSgPHWw== 2024-09-12T03:16:05.866 INFO:teuthology.orchestra.run.smithi027.stdout:[osd.4] 2024-09-12T03:16:05.866 INFO:teuthology.orchestra.run.smithi027.stdout: key = AQDnXOJmoMr1NxAAv3u2q8hYt9w//9gz+N7UAQ== 2024-09-12T03:16:05.866 INFO:teuthology.orchestra.run.smithi027.stdout:[osd.5] 2024-09-12T03:16:05.867 INFO:teuthology.orchestra.run.smithi027.stdout: key = AQDqXOJmxpUPOxAAxLH6RJsyLOZ2XMfzM0fQ1g== 2024-09-12T03:16:05.867 INFO:teuthology.orchestra.run.smithi027.stdout:[osd.6] 2024-09-12T03:16:05.867 INFO:teuthology.orchestra.run.smithi027.stdout: key = AQDuXOJmuP0+LxAAMHoz5eSO81hPtkdBIS2cig== 2024-09-12T03:16:05.867 INFO:teuthology.orchestra.run.smithi027.stdout:[osd.7] 2024-09-12T03:16:05.867 INFO:teuthology.orchestra.run.smithi027.stdout: key = AQDxXOJmuIC/MRAAPDM+lAHqdoQyGVODwjFp+Q== 2024-09-12T03:16:05.867 INFO:teuthology.orchestra.run.smithi027.stdout:[client.0] 2024-09-12T03:16:05.867 INFO:teuthology.orchestra.run.smithi027.stdout: key = AQDVXOJmLm+5ExAAECM4CKfuVElmp/n0b02mXg== 2024-09-12T03:16:05.867 INFO:teuthology.orchestra.run.smithi027.stdout:[client.2] 2024-09-12T03:16:05.867 INFO:teuthology.orchestra.run.smithi027.stdout: key = AQDVXOJmq/ZkGRAAOzmQzEIhb4b/YTxayc34Bw== 2024-09-12T03:16:05.867 INFO:teuthology.orchestra.run.smithi027.stdout:[client.1] 2024-09-12T03:16:05.867 INFO:teuthology.orchestra.run.smithi027.stdout: key = AQDVXOJm61/FHBAALCQouXySfQeZ5DESGmqkzA== 2024-09-12T03:16:05.867 INFO:teuthology.orchestra.run.smithi027.stdout:[client.3] 2024-09-12T03:16:05.867 INFO:teuthology.orchestra.run.smithi027.stdout: key = AQDVXOJm+DlzIhAATQKZe+Zyk0MM8efTa6g9PA== 2024-09-12T03:16:05.887 INFO:teuthology.orchestra.run.smithi045.stdout:[mgr.y] 2024-09-12T03:16:05.887 INFO:teuthology.orchestra.run.smithi045.stdout: key = AQDUXOJmJOgZMBAAQ/JTxfhox7HgxBMtsvUCBg== 2024-09-12T03:16:05.887 INFO:teuthology.orchestra.run.smithi045.stdout:[mds.a] 2024-09-12T03:16:05.887 INFO:teuthology.orchestra.run.smithi045.stdout: key = AQDUXOJmIhxjNxAAaGg8SwtW6NRe/lIgNCV20w== 2024-09-12T03:16:05.887 INFO:teuthology.orchestra.run.smithi045.stdout:[mds.c] 2024-09-12T03:16:05.887 INFO:teuthology.orchestra.run.smithi045.stdout: key = AQDVXOJm0sX7AhAAdynCcho2a/AgFF9Cw3+vAw== 2024-09-12T03:16:05.887 INFO:teuthology.orchestra.run.smithi045.stdout:[osd.0] 2024-09-12T03:16:05.887 INFO:teuthology.orchestra.run.smithi045.stdout: key = AQDYXOJmJTVYLxAAmo8kxRjaiwTT44dZLZlq4A== 2024-09-12T03:16:05.887 INFO:teuthology.orchestra.run.smithi045.stdout:[osd.1] 2024-09-12T03:16:05.887 INFO:teuthology.orchestra.run.smithi045.stdout: key = AQDcXOJmYV0BFBAAmoGX9gf9yLoE/qkWFnv9tg== 2024-09-12T03:16:05.888 INFO:teuthology.orchestra.run.smithi045.stdout:[osd.2] 2024-09-12T03:16:05.888 INFO:teuthology.orchestra.run.smithi045.stdout: key = AQDfXOJmEetiJRAAXk+AvfTfhjsB6T2UB4Z2/w== 2024-09-12T03:16:05.888 INFO:teuthology.orchestra.run.smithi045.stdout:[osd.3] 2024-09-12T03:16:05.888 INFO:teuthology.orchestra.run.smithi045.stdout: key = AQDiXOJmrQbPJxAAQsi2B5a1LnPTPGZVdo7j7w== 2024-09-12T03:16:05.888 INFO:teuthology.orchestra.run.smithi045.stdout:[mgr.x] 2024-09-12T03:16:05.888 INFO:teuthology.orchestra.run.smithi045.stdout: key = AQDUXOJmZcsONBAAMTY2KykuHaEkv1t6Vz5KjA== 2024-09-12T03:16:05.888 INFO:teuthology.orchestra.run.smithi045.stdout:[mds.b] 2024-09-12T03:16:05.888 INFO:teuthology.orchestra.run.smithi045.stdout: key = AQDVXOJm63kWCBAAzjfOsBsWUI+IAsuZ49yEkw== 2024-09-12T03:16:05.888 INFO:teuthology.orchestra.run.smithi045.stdout:[mds.d] 2024-09-12T03:16:05.888 INFO:teuthology.orchestra.run.smithi045.stdout: key = AQDVXOJm7Dd8DxAANHYmvKgpRZ6sVfXDSgPHWw== 2024-09-12T03:16:05.888 INFO:teuthology.orchestra.run.smithi045.stdout:[osd.4] 2024-09-12T03:16:05.888 INFO:teuthology.orchestra.run.smithi045.stdout: key = AQDnXOJmoMr1NxAAv3u2q8hYt9w//9gz+N7UAQ== 2024-09-12T03:16:05.888 INFO:teuthology.orchestra.run.smithi045.stdout:[osd.5] 2024-09-12T03:16:05.888 INFO:teuthology.orchestra.run.smithi045.stdout: key = AQDqXOJmxpUPOxAAxLH6RJsyLOZ2XMfzM0fQ1g== 2024-09-12T03:16:05.889 INFO:teuthology.orchestra.run.smithi045.stdout:[osd.6] 2024-09-12T03:16:05.889 INFO:teuthology.orchestra.run.smithi045.stdout: key = AQDuXOJmuP0+LxAAMHoz5eSO81hPtkdBIS2cig== 2024-09-12T03:16:05.889 INFO:teuthology.orchestra.run.smithi045.stdout:[osd.7] 2024-09-12T03:16:05.889 INFO:teuthology.orchestra.run.smithi045.stdout: key = AQDxXOJmuIC/MRAAPDM+lAHqdoQyGVODwjFp+Q== 2024-09-12T03:16:05.889 INFO:teuthology.orchestra.run.smithi045.stdout:[client.0] 2024-09-12T03:16:05.889 INFO:teuthology.orchestra.run.smithi045.stdout: key = AQDVXOJmLm+5ExAAECM4CKfuVElmp/n0b02mXg== 2024-09-12T03:16:05.889 INFO:teuthology.orchestra.run.smithi045.stdout:[client.2] 2024-09-12T03:16:05.889 INFO:teuthology.orchestra.run.smithi045.stdout: key = AQDVXOJmq/ZkGRAAOzmQzEIhb4b/YTxayc34Bw== 2024-09-12T03:16:05.889 INFO:teuthology.orchestra.run.smithi045.stdout:[client.1] 2024-09-12T03:16:05.889 INFO:teuthology.orchestra.run.smithi045.stdout: key = AQDVXOJm61/FHBAALCQouXySfQeZ5DESGmqkzA== 2024-09-12T03:16:05.889 INFO:teuthology.orchestra.run.smithi045.stdout:[client.3] 2024-09-12T03:16:05.889 INFO:teuthology.orchestra.run.smithi045.stdout: key = AQDVXOJm+DlzIhAATQKZe+Zyk0MM8efTa6g9PA== 2024-09-12T03:16:05.890 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-12T03:16:05.911 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-12T03:16:05.984 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-12T03:16:05.986 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-12T03:16:06.041 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-12T03:16:06.044 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-12T03:16:06.099 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-12T03:16:06.101 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-12T03:16:06.153 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-12T03:16:06.155 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-12T03:16:06.207 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-12T03:16:06.209 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-12T03:16:06.261 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-12T03:16:06.263 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-12T03:16:06.315 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-12T03:16:06.318 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-12T03:16:06.370 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-12T03:16:06.372 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-12T03:16:06.427 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-12T03:16:06.429 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-12T03:16:06.484 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-12T03:16:06.486 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-12T03:16:06.538 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-12T03:16:06.540 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-12T03:16:06.592 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-12T03:16:06.595 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-12T03:16:06.647 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-12T03:16:06.649 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-12T03:16:06.701 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-12T03:16:06.704 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-12T03:16:06.759 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-12T03:16:06.761 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-12T03:16:06.817 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-12T03:16:06.822 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-12T03:16:06.877 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-12T03:16:06.880 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-12T03:16:06.935 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-09-12T03:16:06.935 DEBUG:teuthology.orchestra.run.smithi027:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-09-12T03:16:06.960 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-12T03:16:07.058 DEBUG:teuthology.orchestra.run.smithi027:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-09-12T03:16:07.084 DEBUG:teuthology.orchestra.run.smithi045:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-09-12T03:16:07.109 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-12T03:16:07.207 DEBUG:teuthology.orchestra.run.smithi045:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-09-12T03:16:07.232 DEBUG:teuthology.orchestra.run.smithi045:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-09-12T03:16:07.298 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-12T03:16:07.395 DEBUG:teuthology.orchestra.run.smithi045:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-09-12T03:16:07.421 DEBUG:teuthology.orchestra.run.smithi027:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-12T03:16:07.424 DEBUG:teuthology.orchestra.run.smithi045:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-12T03:16:07.477 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-09-12T03:16:07.478 INFO:tasks.ceph.mon.a:Restarting daemon 2024-09-12T03:16:07.478 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-09-12T03:16:07.480 INFO:tasks.ceph.mon.a:Started 2024-09-12T03:16:07.480 INFO:tasks.ceph.mon.b:Restarting daemon 2024-09-12T03:16:07.480 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-09-12T03:16:07.520 INFO:tasks.ceph.mon.b:Started 2024-09-12T03:16:07.520 INFO:tasks.ceph.mon.c:Restarting daemon 2024-09-12T03:16:07.520 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-09-12T03:16:07.522 INFO:tasks.ceph.mon.c:Started 2024-09-12T03:16:07.522 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-09-12T03:16:07.523 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-09-12T03:16:07.523 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-09-12T03:16:07.525 INFO:tasks.ceph.mgr.y:Started 2024-09-12T03:16:07.525 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-09-12T03:16:07.525 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-09-12T03:16:07.527 INFO:tasks.ceph.mgr.x:Started 2024-09-12T03:16:07.527 INFO:tasks.ceph:Setting crush tunables to default 2024-09-12T03:16:07.527 DEBUG:teuthology.orchestra.run.smithi027:> sudo ceph --cluster ceph osd crush tunables default 2024-09-12T03:16:07.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:07.628+0000 7f98559a6640 1 Processor -- start 2024-09-12T03:16:07.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:07.628+0000 7f98559a6640 1 -- start start 2024-09-12T03:16:07.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:07.629+0000 7f98559a6640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f98500d1410 0x7f98500d0470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:07.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:07.629+0000 7f98559a6640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f98500b7af0 con 0x7f9850073280 2024-09-12T03:16:07.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:07.629+0000 7f98559a6640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f98500b7250 con 0x7f98500d09b0 2024-09-12T03:16:07.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:07.629+0000 7f98559a6640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f98500b7530 con 0x7f98500d1410 2024-09-12T03:16:07.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:07.629+0000 7f984f7fe640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f9850073280 0x7f985015d180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:57164/0 (socket says 172.21.15.27:57164) 2024-09-12T03:16:07.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:07.629+0000 7f984f7fe640 1 -- 172.21.15.27:0/2720048251 learned_addr learned my addr 172.21.15.27:0/2720048251 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:16:07.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:07.629+0000 7f984effd640 1 --2- 172.21.15.27:0/2720048251 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f98500d1410 0x7f98500d0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:07.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:07.630+0000 7f984effd640 1 -- 172.21.15.27:0/2720048251 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f98500d1410 msgr2=0x7f98500d0470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-12T03:16:07.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:07.630+0000 7f984effd640 1 -- 172.21.15.27:0/2720048251 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f98500d1410 msgr2=0x7f98500d0470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-12T03:16:07.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:07.630+0000 7f984effd640 1 --2- 172.21.15.27:0/2720048251 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f98500d1410 0x7f98500d0470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-12T03:16:07.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:07.630+0000 7f984effd640 1 --2- 172.21.15.27:0/2720048251 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f98500d1410 0x7f98500d0470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-12T03:16:07.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:07.785+0000 7f984dffb640 1 -- 172.21.15.27:0/2720048251 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2971454675 0 0) 0x7f98500b7af0 con 0x7f9850073280 2024-09-12T03:16:07.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:07.785+0000 7f984dffb640 1 -- 172.21.15.27:0/2720048251 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9838003520 con 0x7f9850073280 2024-09-12T03:16:07.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:07.785+0000 7f984dffb640 1 -- 172.21.15.27:0/2720048251 <== mon.1 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1028697376 0 0) 0x7f98500b7250 con 0x7f98500d09b0 2024-09-12T03:16:07.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:07.785+0000 7f984dffb640 1 -- 172.21.15.27:0/2720048251 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f98500b7af0 con 0x7f98500d09b0 2024-09-12T03:16:07.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:07.786+0000 7f984dffb640 1 -- 172.21.15.27:0/2720048251 <== mon.1 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1314137856 0 0) 0x7f98500b7af0 con 0x7f98500d09b0 2024-09-12T03:16:07.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:07.786+0000 7f984dffb640 1 -- 172.21.15.27:0/2720048251 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f98500d1410 msgr2=0x7f98500d0470 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-12T03:16:07.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:07.786+0000 7f984dffb640 1 --2- 172.21.15.27:0/2720048251 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f98500d1410 0x7f98500d0470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:07.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:07.786+0000 7f984dffb640 1 -- 172.21.15.27:0/2720048251 >> v1:172.21.15.27:6789/0 conn(0x7f9850073280 legacy=0x7f985015d180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:07.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:07.786+0000 7f984dffb640 1 -- 172.21.15.27:0/2720048251 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98500b7810 con 0x7f98500d09b0 2024-09-12T03:16:07.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:07.786+0000 7f984dffb640 1 -- 172.21.15.27:0/2720048251 <== mon.1 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 847755213 0 0) 0x7f98440019f0 con 0x7f98500d09b0 2024-09-12T03:16:07.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:07.786+0000 7f984dffb640 1 -- 172.21.15.27:0/2720048251 >> v1:172.21.15.45:6789/0 conn(0x7f98500d09b0 legacy=0x7f9850071480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:07.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:07.786+0000 7f984dffb640 1 --2- 172.21.15.27:0/2720048251 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9838003b40 0x7f9838003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:07.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:07.786+0000 7f984dffb640 1 --2- 172.21.15.27:0/2720048251 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f98500d1410 0x7f9838004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:07.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:07.786+0000 7f984dffb640 1 --2- 172.21.15.27:0/2720048251 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9838004d40 0x7f98380051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:07.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:07.786+0000 7f984dffb640 1 -- 172.21.15.27:0/2720048251 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f98500b7530 con 0x7f9838004d40 2024-09-12T03:16:07.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:07.786+0000 7f984dffb640 1 -- 172.21.15.27:0/2720048251 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f983800bb50 con 0x7f9838003b40 2024-09-12T03:16:07.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:07.786+0000 7f984dffb640 1 -- 172.21.15.27:0/2720048251 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f983800bd20 con 0x7f98500d1410 2024-09-12T03:16:07.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:07.786+0000 7f984f7fe640 1 --2- 172.21.15.27:0/2720048251 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9838004d40 0x7f98380051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:07.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:07.786+0000 7f984e7fc640 1 --2- 172.21.15.27:0/2720048251 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f98500d1410 0x7f9838004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:07.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:07.786+0000 7f984effd640 1 --2- 172.21.15.27:0/2720048251 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9838003b40 0x7f9838003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:07.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:07.786+0000 7f984f7fe640 1 -- 172.21.15.27:0/2720048251 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f98500d1410 msgr2=0x7f9838004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:07.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:07.786+0000 7f984f7fe640 1 --2- 172.21.15.27:0/2720048251 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f98500d1410 0x7f9838004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:07.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:07.786+0000 7f984f7fe640 1 -- 172.21.15.27:0/2720048251 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9838003b40 msgr2=0x7f9838003f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:07.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:07.786+0000 7f984f7fe640 1 --2- 172.21.15.27:0/2720048251 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9838003b40 0x7f9838003f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:07.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:07.786+0000 7f984f7fe640 1 -- 172.21.15.27:0/2720048251 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98500b7af0 con 0x7f9838004d40 2024-09-12T03:16:07.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:07.787+0000 7f984f7fe640 1 --2- 172.21.15.27:0/2720048251 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9838004d40 0x7f98380051c0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f98480019f0 tx=0x7f9838005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5ff7c97589f58c3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:07.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:07.788+0000 7f984dffb640 1 -- 172.21.15.27:0/2720048251 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9848001610 con 0x7f9838004d40 2024-09-12T03:16:07.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:07.788+0000 7f984dffb640 1 -- 172.21.15.27:0/2720048251 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f984802e070 con 0x7f9838004d40 2024-09-12T03:16:07.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:07.788+0000 7f984dffb640 1 -- 172.21.15.27:0/2720048251 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98480048a0 con 0x7f9838004d40 2024-09-12T03:16:07.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:07.789+0000 7f98559a6640 1 -- 172.21.15.27:0/2720048251 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9838004d40 msgr2=0x7f98380051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:07.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:07.789+0000 7f98559a6640 1 --2- 172.21.15.27:0/2720048251 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9838004d40 0x7f98380051c0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f98480019f0 tx=0x7f9838005830 comp rx=0 tx=0).stop 2024-09-12T03:16:07.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:07.789+0000 7f984f7fe640 1 -- 172.21.15.27:0/2720048251 reap_dead start 2024-09-12T03:16:07.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:07.789+0000 7f98559a6640 1 -- 172.21.15.27:0/2720048251 shutdown_connections 2024-09-12T03:16:07.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:07.789+0000 7f98559a6640 1 -- 172.21.15.27:0/2720048251 >> 172.21.15.27:0/2720048251 conn(0x7f98500b4320 msgr2=0x7f98500b4700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:07.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:07.789+0000 7f98559a6640 1 -- 172.21.15.27:0/2720048251 shutdown_connections 2024-09-12T03:16:07.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:07.789+0000 7f98559a6640 1 -- 172.21.15.27:0/2720048251 wait complete. 2024-09-12T03:16:07.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:07.790+0000 7f98559a6640 1 Processor -- start 2024-09-12T03:16:07.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:07.791+0000 7f98559a6640 1 -- start start 2024-09-12T03:16:07.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:07.791+0000 7f98559a6640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9838004d40 0x7f9850141e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:07.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:07.791+0000 7f98559a6640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f985006ce50 0x7f985015b620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:07.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:07.791+0000 7f98559a6640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f985015bb60 0x7f9850154760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:07.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:07.791+0000 7f98559a6640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f9850071b90 con 0x7f985015bb60 2024-09-12T03:16:07.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:07.791+0000 7f98559a6640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f98500d17f0 con 0x7f9838004d40 2024-09-12T03:16:07.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:07.791+0000 7f98559a6640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f98500b6e90 con 0x7f985006ce50 2024-09-12T03:16:07.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:07.792+0000 7f984f7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f985015bb60 0x7f9850154760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:07.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:07.792+0000 7f984f7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f985015bb60 0x7f9850154760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41656/0 (socket says 172.21.15.27:41656) 2024-09-12T03:16:07.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:07.792+0000 7f984f7fe640 1 -- 172.21.15.27:0/1656210405 learned_addr learned my addr 172.21.15.27:0/1656210405 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:16:07.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:07.792+0000 7f984effd640 1 --2- 172.21.15.27:0/1656210405 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9838004d40 0x7f9850141e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:07.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:07.792+0000 7f984e7fc640 1 --2- 172.21.15.27:0/1656210405 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f985006ce50 0x7f985015b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:07.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:07.792+0000 7f984f7fe640 1 -- 172.21.15.27:0/1656210405 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f985006ce50 msgr2=0x7f985015b620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:07.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:07.792+0000 7f984f7fe640 1 --2- 172.21.15.27:0/1656210405 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f985006ce50 0x7f985015b620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:07.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:07.792+0000 7f984f7fe640 1 -- 172.21.15.27:0/1656210405 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9838004d40 msgr2=0x7f9850141e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:07.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:07.792+0000 7f984f7fe640 1 --2- 172.21.15.27:0/1656210405 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9838004d40 0x7f9850141e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:07.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:07.792+0000 7f984f7fe640 1 -- 172.21.15.27:0/1656210405 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9850142370 con 0x7f985015bb60 2024-09-12T03:16:07.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:07.792+0000 7f984f7fe640 1 --2- 172.21.15.27:0/1656210405 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f985015bb60 0x7f9850154760 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f9848007fd0 tx=0x7f9848001340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:07.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:07.792+0000 7f9837fff640 1 -- 172.21.15.27:0/1656210405 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9848004bf0 con 0x7f985015bb60 2024-09-12T03:16:07.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:07.792+0000 7f9837fff640 1 -- 172.21.15.27:0/1656210405 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9848004d90 con 0x7f985015bb60 2024-09-12T03:16:07.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:07.792+0000 7f9837fff640 1 -- 172.21.15.27:0/1656210405 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9848007270 con 0x7f985015bb60 2024-09-12T03:16:07.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:07.792+0000 7f98559a6640 1 -- 172.21.15.27:0/1656210405 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98501f7960 con 0x7f985015bb60 2024-09-12T03:16:07.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:07.792+0000 7f98559a6640 1 -- 172.21.15.27:0/1656210405 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98501f7f20 con 0x7f985015bb60 2024-09-12T03:16:07.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:07.792+0000 7f9837fff640 1 -- 172.21.15.27:0/1656210405 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f9848030070 con 0x7f985015bb60 2024-09-12T03:16:07.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:07.793+0000 7f9837fff640 1 -- 172.21.15.27:0/1656210405 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f9848033760 con 0x7f985015bb60 2024-09-12T03:16:07.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:07.793+0000 7f98559a6640 1 -- 172.21.15.27:0/1656210405 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9850071520 con 0x7f985015bb60 2024-09-12T03:16:07.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:07.796+0000 7f9837fff640 1 -- 172.21.15.27:0/1656210405 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f98480016b0 con 0x7f985015bb60 2024-09-12T03:16:07.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:07.860+0000 7f98559a6640 1 -- 172.21.15.27:0/1656210405 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f98500d0470 con 0x7f985015bb60 2024-09-12T03:16:07.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:07.862+0000 7f9837fff640 1 -- 172.21.15.27:0/1656210405 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) ==== 124+0+0 (secure 0 0 0) 0x7f98480339b0 con 0x7f985015bb60 2024-09-12T03:16:07.863 INFO:teuthology.orchestra.run.smithi027.stderr:adjusted tunables profile to default 2024-09-12T03:16:07.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:07.863+0000 7f98559a6640 1 -- 172.21.15.27:0/1656210405 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f985015bb60 msgr2=0x7f9850154760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:07.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:07.863+0000 7f98559a6640 1 --2- 172.21.15.27:0/1656210405 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f985015bb60 0x7f9850154760 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f9848007fd0 tx=0x7f9848001340 comp rx=0 tx=0).stop 2024-09-12T03:16:07.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:07.863+0000 7f98559a6640 1 -- 172.21.15.27:0/1656210405 shutdown_connections 2024-09-12T03:16:07.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:07.863+0000 7f98559a6640 1 --2- 172.21.15.27:0/1656210405 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f985015bb60 0x7f9850154760 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:07.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:07.863+0000 7f98559a6640 1 --2- 172.21.15.27:0/1656210405 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f985006ce50 0x7f985015b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:07.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:07.863+0000 7f98559a6640 1 --2- 172.21.15.27:0/1656210405 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9838004d40 0x7f9850141e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:07.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:07.863+0000 7f98559a6640 1 -- 172.21.15.27:0/1656210405 >> 172.21.15.27:0/1656210405 conn(0x7f98500b4320 msgr2=0x7f98500b4700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:07.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:07.863+0000 7f98559a6640 1 -- 172.21.15.27:0/1656210405 shutdown_connections 2024-09-12T03:16:07.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:07.863+0000 7f98559a6640 1 -- 172.21.15.27:0/1656210405 wait complete. 2024-09-12T03:16:07.875 INFO:tasks.ceph:check_enable_crimson: False 2024-09-12T03:16:07.875 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-09-12T03:16:07.875 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-12T03:16:07.875 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-09-12T03:16:07.889 INFO:tasks.ceph.mgr.x.smithi045.stderr:2024-09-12T03:16:07.889+0000 7fc1f240d040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-12T03:16:07.901 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-12T03:16:07.902 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-09-12T03:16:07.910 INFO:tasks.ceph.mgr.y.smithi027.stderr:2024-09-12T03:16:07.909+0000 7f32df11f040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-12T03:16:07.931 INFO:tasks.ceph.mgr.x.smithi045.stderr:2024-09-12T03:16:07.932+0000 7fc1f240d040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-12T03:16:07.935 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-12T03:16:07.935 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-09-12T03:16:07.955 INFO:tasks.ceph.mgr.y.smithi027.stderr:2024-09-12T03:16:07.954+0000 7f32df11f040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-12T03:16:07.980 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-12T03:16:07.980 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-09-12T03:16:08.046 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-12T03:16:08.046 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-09-12T03:16:08.072 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-12T03:16:08.072 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-09-12T03:16:08.083 INFO:tasks.ceph.mgr.x.smithi045.stderr:2024-09-12T03:16:08.083+0000 7fc1f240d040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-12T03:16:08.105 INFO:tasks.ceph.mgr.y.smithi027.stderr:2024-09-12T03:16:08.104+0000 7f32df11f040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-12T03:16:08.108 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-12T03:16:08.109 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-09-12T03:16:08.137 INFO:tasks.ceph.mgr.x.smithi045.stderr:2024-09-12T03:16:08.137+0000 7fc1f240d040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-12T03:16:08.158 INFO:tasks.ceph.mgr.y.smithi027.stderr:2024-09-12T03:16:08.157+0000 7f32df11f040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-12T03:16:08.163 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-12T03:16:08.163 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-09-12T03:16:08.229 DEBUG:teuthology.orchestra.run.smithi045:> sudo ceph --cluster ceph osd new 1851116c-1fb3-4dc9-9772-f90e2ef4c355 0 2024-09-12T03:16:08.309 INFO:tasks.ceph.mgr.x.smithi045.stderr:2024-09-12T03:16:08.309+0000 7fc1f240d040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-12T03:16:08.321 INFO:tasks.ceph.mgr.y.smithi027.stderr:2024-09-12T03:16:08.320+0000 7f32df11f040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-12T03:16:08.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.371+0000 7f957d6e4640 1 Processor -- start 2024-09-12T03:16:08.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.371+0000 7f957d6e4640 1 -- start start 2024-09-12T03:16:08.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.372+0000 7f957d6e4640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f95780b6900 0x7f957806dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:08.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.372+0000 7f957d6e4640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f95780b2650 0x7f957806e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:08.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.372+0000 7f957d6e4640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f95780cda10 con 0x7f957815f7b0 2024-09-12T03:16:08.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.372+0000 7f957d6e4640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f95780cd7c0 con 0x7f95780b6900 2024-09-12T03:16:08.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.372+0000 7f957d6e4640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f95780cdcf0 con 0x7f95780b2650 2024-09-12T03:16:08.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.372+0000 7f95777fe640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f95780b2650 0x7f957806e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:08.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.373+0000 7f95777fe640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f95780b2650 0x7f957806e000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3301/0 says I am v2:172.21.15.45:45264/0 (socket says 172.21.15.45:45264) 2024-09-12T03:16:08.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.373+0000 7f95777fe640 1 -- 172.21.15.45:0/562369051 learned_addr learned my addr 172.21.15.45:0/562369051 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-12T03:16:08.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.373+0000 7f95767fc640 1 --2- 172.21.15.45:0/562369051 >> v2:172.21.15.27:3300/0 conn(0x7f95780b6900 0x7f957806dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:08.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.373+0000 7f95777fe640 1 -- 172.21.15.45:0/562369051 >> v2:172.21.15.27:3300/0 conn(0x7f95780b6900 msgr2=0x7f957806dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:08.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.373+0000 7f95777fe640 1 --2- 172.21.15.45:0/562369051 >> v2:172.21.15.27:3300/0 conn(0x7f95780b6900 0x7f957806dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:08.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.373+0000 7f95777fe640 1 -- 172.21.15.45:0/562369051 >> v1:172.21.15.27:6789/0 conn(0x7f957815f7b0 legacy=0x7f95780b61f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:08.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.373+0000 7f95777fe640 1 -- 172.21.15.45:0/562369051 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95780cdfd0 con 0x7f95780b2650 2024-09-12T03:16:08.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.374+0000 7f95777fe640 1 --2- 172.21.15.45:0/562369051 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f95780b2650 0x7f957806e000 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f9570009e50 tx=0x7f957002a680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b89b8e2cf6baea8f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:08.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.374+0000 7f9575ffb640 1 -- 172.21.15.45:0/562369051 <== mon.2 v2:172.21.15.45:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9570036070 con 0x7f95780b2650 2024-09-12T03:16:08.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.374+0000 7f9575ffb640 1 -- 172.21.15.45:0/562369051 <== mon.2 v2:172.21.15.45:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9570031600 con 0x7f95780b2650 2024-09-12T03:16:08.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.374+0000 7f9575ffb640 1 -- 172.21.15.45:0/562369051 <== mon.2 v2:172.21.15.45:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95700319d0 con 0x7f95780b2650 2024-09-12T03:16:08.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.375+0000 7f957d6e4640 1 -- 172.21.15.45:0/562369051 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f95780b2650 msgr2=0x7f957806e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:08.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.375+0000 7f957d6e4640 1 --2- 172.21.15.45:0/562369051 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f95780b2650 0x7f957806e000 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f9570009e50 tx=0x7f957002a680 comp rx=0 tx=0).stop 2024-09-12T03:16:08.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.375+0000 7f957d6e4640 1 -- 172.21.15.45:0/562369051 shutdown_connections 2024-09-12T03:16:08.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.375+0000 7f957d6e4640 1 --2- 172.21.15.45:0/562369051 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f95780b2650 0x7f957806e000 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:08.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.375+0000 7f957d6e4640 1 --2- 172.21.15.45:0/562369051 >> v2:172.21.15.27:3300/0 conn(0x7f95780b6900 0x7f957806dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:08.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.375+0000 7f957d6e4640 1 -- 172.21.15.45:0/562369051 >> 172.21.15.45:0/562369051 conn(0x7f95780b0b50 msgr2=0x7f95780b0f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:08.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.375+0000 7f957d6e4640 1 -- 172.21.15.45:0/562369051 shutdown_connections 2024-09-12T03:16:08.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.375+0000 7f957d6e4640 1 -- 172.21.15.45:0/562369051 wait complete. 2024-09-12T03:16:08.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.376+0000 7f957d6e4640 1 Processor -- start 2024-09-12T03:16:08.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.376+0000 7f957d6e4640 1 -- start start 2024-09-12T03:16:08.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.377+0000 7f957d6e4640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f95780b2650 0x7f95781e6bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:08.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.377+0000 7f957d6e4640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f95780b6900 0x7f95780aa730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:08.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.377+0000 7f957d6e4640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f957815f7b0 0x7f95781f3340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:08.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.377+0000 7f957d6e4640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f957815fb90 con 0x7f95780b6900 2024-09-12T03:16:08.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.377+0000 7f957d6e4640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f95780ce590 con 0x7f95780b2650 2024-09-12T03:16:08.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.377+0000 7f957d6e4640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f957815fd10 con 0x7f957815f7b0 2024-09-12T03:16:08.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.377+0000 7f95777fe640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f957815f7b0 0x7f95781f3340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:08.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.377+0000 7f95777fe640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f957815f7b0 0x7f95781f3340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3301/0 says I am v2:172.21.15.45:45266/0 (socket says 172.21.15.45:45266) 2024-09-12T03:16:08.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.377+0000 7f95777fe640 1 -- 172.21.15.45:0/3664797050 learned_addr learned my addr 172.21.15.45:0/3664797050 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-12T03:16:08.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.377+0000 7f95767fc640 1 --2- 172.21.15.45:0/3664797050 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f95780b6900 0x7f95780aa730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:08.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.377+0000 7f9576ffd640 1 --2- 172.21.15.45:0/3664797050 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f95780b2650 0x7f95781e6bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:08.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.377+0000 7f95767fc640 1 -- 172.21.15.45:0/3664797050 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f957815f7b0 msgr2=0x7f95781f3340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:08.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.377+0000 7f95767fc640 1 --2- 172.21.15.45:0/3664797050 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f957815f7b0 0x7f95781f3340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:08.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.377+0000 7f95767fc640 1 -- 172.21.15.45:0/3664797050 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f95780b2650 msgr2=0x7f95781e6bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:08.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.377+0000 7f95767fc640 1 --2- 172.21.15.45:0/3664797050 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f95780b2650 0x7f95781e6bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:08.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.377+0000 7f95767fc640 1 -- 172.21.15.45:0/3664797050 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9578149f90 con 0x7f95780b6900 2024-09-12T03:16:08.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.377+0000 7f9576ffd640 1 --2- 172.21.15.45:0/3664797050 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f95780b2650 0x7f95781e6bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:16:08.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.377+0000 7f95777fe640 1 --2- 172.21.15.45:0/3664797050 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f957815f7b0 0x7f95781f3340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:16:08.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.378+0000 7f95767fc640 1 --2- 172.21.15.45:0/3664797050 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f95780b6900 0x7f95780aa730 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f956c00e850 tx=0x7f956c00ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:08.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.378+0000 7f955f7fe640 1 -- 172.21.15.45:0/3664797050 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f956c017070 con 0x7f95780b6900 2024-09-12T03:16:08.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.378+0000 7f955f7fe640 1 -- 172.21.15.45:0/3664797050 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f956c00c940 con 0x7f95780b6900 2024-09-12T03:16:08.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.378+0000 7f957d6e4640 1 -- 172.21.15.45:0/3664797050 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95781e5840 con 0x7f95780b6900 2024-09-12T03:16:08.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.378+0000 7f955f7fe640 1 -- 172.21.15.45:0/3664797050 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f956c00cd10 con 0x7f95780b6900 2024-09-12T03:16:08.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.378+0000 7f957d6e4640 1 -- 172.21.15.45:0/3664797050 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95781e5e00 con 0x7f95780b6900 2024-09-12T03:16:08.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.378+0000 7f955f7fe640 1 -- 172.21.15.45:0/3664797050 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f956c01a070 con 0x7f95780b6900 2024-09-12T03:16:08.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.378+0000 7f955f7fe640 1 -- 172.21.15.45:0/3664797050 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 725+0+0 (secure 0 0 0) 0x7f956c013470 con 0x7f95780b6900 2024-09-12T03:16:08.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.379+0000 7f957d6e4640 1 -- 172.21.15.45:0/3664797050 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95781e68a0 con 0x7f95780b6900 2024-09-12T03:16:08.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.381+0000 7f955f7fe640 1 -- 172.21.15.45:0/3664797050 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f95781e68a0 con 0x7f95780b6900 2024-09-12T03:16:08.427 INFO:tasks.ceph.mgr.x.smithi045.stderr:2024-09-12T03:16:08.428+0000 7fc1f240d040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-12T03:16:08.437 INFO:tasks.ceph.mgr.y.smithi027.stderr:2024-09-12T03:16:08.437+0000 7f32df11f040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-12T03:16:08.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.452+0000 7f957d6e4640 1 -- 172.21.15.45:0/3664797050 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd new", "uuid": "1851116c-1fb3-4dc9-9772-f90e2ef4c355", "id": 0} v 0) -- 0x7f95780afb70 con 0x7f95780b6900 2024-09-12T03:16:08.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.453+0000 7f955f7fe640 1 -- 172.21.15.45:0/3664797050 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "1851116c-1fb3-4dc9-9772-f90e2ef4c355", "id": 0}]=0 v3) ==== 112+0+2 (secure 0 0 0) 0x7f956c0136c0 con 0x7f95780b6900 2024-09-12T03:16:08.453 INFO:teuthology.orchestra.run.smithi045.stdout:0 2024-09-12T03:16:08.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.454+0000 7f957d6e4640 1 -- 172.21.15.45:0/3664797050 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f95780b6900 msgr2=0x7f95780aa730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:08.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.454+0000 7f957d6e4640 1 --2- 172.21.15.45:0/3664797050 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f95780b6900 0x7f95780aa730 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f956c00e850 tx=0x7f956c00ed10 comp rx=0 tx=0).stop 2024-09-12T03:16:08.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.454+0000 7f957d6e4640 1 -- 172.21.15.45:0/3664797050 shutdown_connections 2024-09-12T03:16:08.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.454+0000 7f957d6e4640 1 --2- 172.21.15.45:0/3664797050 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f957815f7b0 0x7f95781f3340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:08.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.454+0000 7f957d6e4640 1 --2- 172.21.15.45:0/3664797050 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f95780b6900 0x7f95780aa730 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:08.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.454+0000 7f957d6e4640 1 --2- 172.21.15.45:0/3664797050 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f95780b2650 0x7f95781e6bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:08.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.454+0000 7f957d6e4640 1 -- 172.21.15.45:0/3664797050 >> 172.21.15.45:0/3664797050 conn(0x7f95780b0b50 msgr2=0x7f95780b0f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:08.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.454+0000 7f957d6e4640 1 -- 172.21.15.45:0/3664797050 shutdown_connections 2024-09-12T03:16:08.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.454+0000 7f957d6e4640 1 -- 172.21.15.45:0/3664797050 wait complete. 2024-09-12T03:16:08.465 DEBUG:teuthology.orchestra.run.smithi045:> sudo ceph --cluster ceph osd new f38ae17d-73eb-4204-b99d-975fa806fb87 1 2024-09-12T03:16:08.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.606+0000 7f1301313640 1 Processor -- start 2024-09-12T03:16:08.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.606+0000 7f1301313640 1 -- start start 2024-09-12T03:16:08.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.607+0000 7f1301313640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f12fc162040 0x7f12fc0d1da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:08.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.607+0000 7f1301313640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f12fc0d3790 0x7f12fc0b7a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:08.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.607+0000 7f1301313640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f12fc06d8c0 0x7f12fc06dd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:08.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.607+0000 7f1301313640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f12fc0b7f90 con 0x7f12fc162040 2024-09-12T03:16:08.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.607+0000 7f1301313640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f12fc0d2e30 con 0x7f12fc06d8c0 2024-09-12T03:16:08.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.607+0000 7f1301313640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f12fc162420 con 0x7f12fc0d3790 2024-09-12T03:16:08.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.607+0000 7f12fb7fe640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f12fc06d8c0 0x7f12fc06dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:08.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.607+0000 7f12fa7fc640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f12fc0d3790 0x7f12fc0b7a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:08.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.607+0000 7f12fb7fe640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f12fc06d8c0 0x7f12fc06dd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50882/0 (socket says 172.21.15.45:50882) 2024-09-12T03:16:08.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.607+0000 7f12fb7fe640 1 -- 172.21.15.45:0/2774090956 learned_addr learned my addr 172.21.15.45:0/2774090956 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-12T03:16:08.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.607+0000 7f12faffd640 1 --2- 172.21.15.45:0/2774090956 >> v2:172.21.15.27:3300/0 conn(0x7f12fc162040 0x7f12fc0d1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:08.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.607+0000 7f12fb7fe640 1 -- 172.21.15.45:0/2774090956 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f12fc0d3790 msgr2=0x7f12fc0b7a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:08.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.607+0000 7f12fb7fe640 1 --2- 172.21.15.45:0/2774090956 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f12fc0d3790 0x7f12fc0b7a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:08.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.607+0000 7f12fb7fe640 1 -- 172.21.15.45:0/2774090956 >> v2:172.21.15.27:3300/0 conn(0x7f12fc162040 msgr2=0x7f12fc0d1da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:08.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.607+0000 7f12fb7fe640 1 --2- 172.21.15.45:0/2774090956 >> v2:172.21.15.27:3300/0 conn(0x7f12fc162040 0x7f12fc0d1da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:08.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.607+0000 7f12fb7fe640 1 -- 172.21.15.45:0/2774090956 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12fc0d30d0 con 0x7f12fc06d8c0 2024-09-12T03:16:08.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.607+0000 7f12fa7fc640 1 --2- 172.21.15.45:0/2774090956 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f12fc0d3790 0x7f12fc0b7a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:16:08.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.607+0000 7f12faffd640 1 --2- 172.21.15.45:0/2774090956 >> v2:172.21.15.27:3300/0 conn(0x7f12fc162040 0x7f12fc0d1da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:16:08.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.608+0000 7f12fb7fe640 1 --2- 172.21.15.45:0/2774090956 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f12fc06d8c0 0x7f12fc06dd50 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f12fc0b7820 tx=0x7f12f402a6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d77e96b191b7d157 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:08.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.608+0000 7f12f9ffb640 1 -- 172.21.15.45:0/2774090956 <== mon.1 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12f4036070 con 0x7f12fc06d8c0 2024-09-12T03:16:08.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.608+0000 7f12f9ffb640 1 -- 172.21.15.45:0/2774090956 <== mon.1 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f12f4031600 con 0x7f12fc06d8c0 2024-09-12T03:16:08.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.608+0000 7f12f9ffb640 1 -- 172.21.15.45:0/2774090956 <== mon.1 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12f40319d0 con 0x7f12fc06d8c0 2024-09-12T03:16:08.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.609+0000 7f1301313640 1 -- 172.21.15.45:0/2774090956 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f12fc06d8c0 msgr2=0x7f12fc06dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:08.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.609+0000 7f1301313640 1 --2- 172.21.15.45:0/2774090956 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f12fc06d8c0 0x7f12fc06dd50 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f12fc0b7820 tx=0x7f12f402a6a0 comp rx=0 tx=0).stop 2024-09-12T03:16:08.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.609+0000 7f1301313640 1 -- 172.21.15.45:0/2774090956 shutdown_connections 2024-09-12T03:16:08.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.609+0000 7f1301313640 1 --2- 172.21.15.45:0/2774090956 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f12fc06d8c0 0x7f12fc06dd50 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:08.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.609+0000 7f1301313640 1 --2- 172.21.15.45:0/2774090956 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f12fc0d3790 0x7f12fc0b7a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:08.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.609+0000 7f1301313640 1 --2- 172.21.15.45:0/2774090956 >> v2:172.21.15.27:3300/0 conn(0x7f12fc162040 0x7f12fc0d1da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:08.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.609+0000 7f1301313640 1 -- 172.21.15.45:0/2774090956 >> 172.21.15.45:0/2774090956 conn(0x7f12fc0b6700 msgr2=0x7f12fc0b6b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:08.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.609+0000 7f1301313640 1 -- 172.21.15.45:0/2774090956 shutdown_connections 2024-09-12T03:16:08.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.609+0000 7f1301313640 1 -- 172.21.15.45:0/2774090956 wait complete. 2024-09-12T03:16:08.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.610+0000 7f1301313640 1 Processor -- start 2024-09-12T03:16:08.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.610+0000 7f1301313640 1 -- start start 2024-09-12T03:16:08.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.611+0000 7f1301313640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f12fc06d8c0 0x7f12fc157cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:08.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.611+0000 7f1301313640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f12fc0d3790 0x7f12fc0cc0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:08.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.611+0000 7f1301313640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f12fc162040 0x7f12fc0cc5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:08.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.611+0000 7f1301313640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f12fc0d4530 con 0x7f12fc06d8c0 2024-09-12T03:16:08.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.611+0000 7f1301313640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f12fc0cee00 con 0x7f12fc0d3790 2024-09-12T03:16:08.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.611+0000 7f1301313640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f12fc0d4150 con 0x7f12fc162040 2024-09-12T03:16:08.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.611+0000 7f12fa7fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f12fc0d3790 0x7f12fc0cc0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:08.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.611+0000 7f12fa7fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f12fc0d3790 0x7f12fc0cc0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50892/0 (socket says 172.21.15.45:50892) 2024-09-12T03:16:08.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.611+0000 7f12fb7fe640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f12fc162040 0x7f12fc0cc5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:08.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.611+0000 7f12fa7fc640 1 -- 172.21.15.45:0/2357429957 learned_addr learned my addr 172.21.15.45:0/2357429957 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-12T03:16:08.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.611+0000 7f12faffd640 1 --2- 172.21.15.45:0/2357429957 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f12fc06d8c0 0x7f12fc157cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:08.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.612+0000 7f12fa7fc640 1 -- 172.21.15.45:0/2357429957 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f12fc162040 msgr2=0x7f12fc0cc5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:08.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.612+0000 7f12fa7fc640 1 --2- 172.21.15.45:0/2357429957 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f12fc162040 0x7f12fc0cc5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:08.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.612+0000 7f12fa7fc640 1 -- 172.21.15.45:0/2357429957 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f12fc06d8c0 msgr2=0x7f12fc157cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:08.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.612+0000 7f12fa7fc640 1 --2- 172.21.15.45:0/2357429957 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f12fc06d8c0 0x7f12fc157cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:08.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.612+0000 7f12fa7fc640 1 -- 172.21.15.45:0/2357429957 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12fc158230 con 0x7f12fc0d3790 2024-09-12T03:16:08.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.612+0000 7f12fb7fe640 1 --2- 172.21.15.45:0/2357429957 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f12fc162040 0x7f12fc0cc5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:16:08.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.612+0000 7f12fa7fc640 1 --2- 172.21.15.45:0/2357429957 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f12fc0d3790 0x7f12fc0cc0a0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f12f000e9b0 tx=0x7f12f000ee70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:08.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.612+0000 7f12e37fe640 1 -- 172.21.15.45:0/2357429957 <== mon.1 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12f0017070 con 0x7f12fc0d3790 2024-09-12T03:16:08.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.612+0000 7f12e37fe640 1 -- 172.21.15.45:0/2357429957 <== mon.1 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f12f000c9a0 con 0x7f12fc0d3790 2024-09-12T03:16:08.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.612+0000 7f1301313640 1 -- 172.21.15.45:0/2357429957 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12fc1ea070 con 0x7f12fc0d3790 2024-09-12T03:16:08.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.612+0000 7f12e37fe640 1 -- 172.21.15.45:0/2357429957 <== mon.1 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12f000cd70 con 0x7f12fc0d3790 2024-09-12T03:16:08.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.612+0000 7f1301313640 1 -- 172.21.15.45:0/2357429957 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12fc15f1c0 con 0x7f12fc0d3790 2024-09-12T03:16:08.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.612+0000 7f12e37fe640 1 -- 172.21.15.45:0/2357429957 <== mon.1 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f12f001a070 con 0x7f12fc0d3790 2024-09-12T03:16:08.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.612+0000 7f12e37fe640 1 -- 172.21.15.45:0/2357429957 <== mon.1 v2:172.21.15.45:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 840+0+0 (secure 0 0 0) 0x7f12f00063e0 con 0x7f12fc0d3790 2024-09-12T03:16:08.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.613+0000 7f1301313640 1 -- 172.21.15.45:0/2357429957 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12fc0b7a50 con 0x7f12fc0d3790 2024-09-12T03:16:08.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.615+0000 7f12e37fe640 1 -- 172.21.15.45:0/2357429957 <== mon.1 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f12fc158230 con 0x7f12fc0d3790 2024-09-12T03:16:08.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.686+0000 7f1301313640 1 -- 172.21.15.45:0/2357429957 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd new", "uuid": "f38ae17d-73eb-4204-b99d-975fa806fb87", "id": 1} v 0) -- 0x7f12fc0d2bb0 con 0x7f12fc0d3790 2024-09-12T03:16:08.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.689+0000 7f12e37fe640 1 -- 172.21.15.45:0/2357429957 <== mon.1 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "f38ae17d-73eb-4204-b99d-975fa806fb87", "id": 1}]=0 v4) ==== 112+0+2 (secure 0 0 0) 0x7f12f0009380 con 0x7f12fc0d3790 2024-09-12T03:16:08.689 INFO:teuthology.orchestra.run.smithi045.stdout:1 2024-09-12T03:16:08.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.690+0000 7f1301313640 1 -- 172.21.15.45:0/2357429957 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f12fc0d3790 msgr2=0x7f12fc0cc0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:08.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.690+0000 7f1301313640 1 --2- 172.21.15.45:0/2357429957 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f12fc0d3790 0x7f12fc0cc0a0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f12f000e9b0 tx=0x7f12f000ee70 comp rx=0 tx=0).stop 2024-09-12T03:16:08.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.690+0000 7f1301313640 1 -- 172.21.15.45:0/2357429957 shutdown_connections 2024-09-12T03:16:08.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.690+0000 7f1301313640 1 --2- 172.21.15.45:0/2357429957 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f12fc162040 0x7f12fc0cc5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:08.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.690+0000 7f1301313640 1 --2- 172.21.15.45:0/2357429957 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f12fc0d3790 0x7f12fc0cc0a0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:08.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.690+0000 7f1301313640 1 --2- 172.21.15.45:0/2357429957 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f12fc06d8c0 0x7f12fc157cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:08.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.690+0000 7f1301313640 1 -- 172.21.15.45:0/2357429957 >> 172.21.15.45:0/2357429957 conn(0x7f12fc0b6700 msgr2=0x7f12fc0ba150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:08.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.690+0000 7f1301313640 1 -- 172.21.15.45:0/2357429957 shutdown_connections 2024-09-12T03:16:08.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.690+0000 7f1301313640 1 -- 172.21.15.45:0/2357429957 wait complete. 2024-09-12T03:16:08.701 DEBUG:teuthology.orchestra.run.smithi045:> sudo ceph --cluster ceph osd new fb12dcc8-bdd0-42b8-b46d-8b10106224bb 2 2024-09-12T03:16:08.778 INFO:tasks.ceph.mgr.y.smithi027.stderr:2024-09-12T03:16:08.777+0000 7f32df11f040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-12T03:16:08.788 INFO:tasks.ceph.mgr.x.smithi045.stderr:2024-09-12T03:16:08.789+0000 7fc1f240d040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-12T03:16:08.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.804+0000 7f377cba6640 1 Processor -- start 2024-09-12T03:16:08.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.804+0000 7f377cba6640 1 -- start start 2024-09-12T03:16:08.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.805+0000 7f377cba6640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f37780b1ae0 0x7f377806e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:08.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.805+0000 7f377cba6640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f377806fe70 0x7f37780702d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:08.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.805+0000 7f377cba6640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f377815ece0 con 0x7f377815e900 2024-09-12T03:16:08.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.805+0000 7f377cba6640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f3778070810 con 0x7f377806fe70 2024-09-12T03:16:08.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.805+0000 7f377cba6640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f377806d540 con 0x7f37780b1ae0 2024-09-12T03:16:08.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.805+0000 7f3775d9b640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f37780b1ae0 0x7f377806e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:08.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.805+0000 7f3775d9b640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f37780b1ae0 0x7f377806e5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50904/0 (socket says 172.21.15.45:50904) 2024-09-12T03:16:08.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.805+0000 7f3775d9b640 1 -- 172.21.15.45:0/503118569 learned_addr learned my addr 172.21.15.45:0/503118569 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-12T03:16:08.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.805+0000 7f3776d9d640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f377806fe70 0x7f37780702d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:08.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.805+0000 7f3775d9b640 1 -- 172.21.15.45:0/503118569 >> v2:172.21.15.27:3300/0 conn(0x7f377806fe70 msgr2=0x7f37780702d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:08.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.805+0000 7f3775d9b640 1 --2- 172.21.15.45:0/503118569 >> v2:172.21.15.27:3300/0 conn(0x7f377806fe70 0x7f37780702d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:08.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.805+0000 7f3775d9b640 1 -- 172.21.15.45:0/503118569 >> v1:172.21.15.27:6789/0 conn(0x7f377815e900 legacy=0x7f377806cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:08.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.805+0000 7f3775d9b640 1 -- 172.21.15.45:0/503118569 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f377806d820 con 0x7f37780b1ae0 2024-09-12T03:16:08.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.806+0000 7f3775d9b640 1 --2- 172.21.15.45:0/503118569 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f37780b1ae0 0x7f377806e5e0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f37780b52d0 tx=0x7f376c02a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ec64d7566f5b471f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:08.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.806+0000 7f377559a640 1 -- 172.21.15.45:0/503118569 <== mon.1 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f376c037070 con 0x7f37780b1ae0 2024-09-12T03:16:08.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.806+0000 7f377559a640 1 -- 172.21.15.45:0/503118569 <== mon.1 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f376c032440 con 0x7f37780b1ae0 2024-09-12T03:16:08.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.806+0000 7f377559a640 1 -- 172.21.15.45:0/503118569 <== mon.1 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f376c032810 con 0x7f37780b1ae0 2024-09-12T03:16:08.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.807+0000 7f377cba6640 1 -- 172.21.15.45:0/503118569 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f37780b1ae0 msgr2=0x7f377806e5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:08.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.807+0000 7f377cba6640 1 --2- 172.21.15.45:0/503118569 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f37780b1ae0 0x7f377806e5e0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f37780b52d0 tx=0x7f376c02a270 comp rx=0 tx=0).stop 2024-09-12T03:16:08.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.807+0000 7f377cba6640 1 -- 172.21.15.45:0/503118569 shutdown_connections 2024-09-12T03:16:08.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.807+0000 7f377cba6640 1 --2- 172.21.15.45:0/503118569 >> v2:172.21.15.27:3300/0 conn(0x7f377806fe70 0x7f37780702d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:08.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.807+0000 7f377cba6640 1 --2- 172.21.15.45:0/503118569 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f37780b1ae0 0x7f377806e5e0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:08.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.807+0000 7f377cba6640 1 -- 172.21.15.45:0/503118569 >> 172.21.15.45:0/503118569 conn(0x7f3778073800 msgr2=0x7f377809ba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:08.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.807+0000 7f377cba6640 1 -- 172.21.15.45:0/503118569 shutdown_connections 2024-09-12T03:16:08.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.807+0000 7f377cba6640 1 -- 172.21.15.45:0/503118569 wait complete. 2024-09-12T03:16:08.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.808+0000 7f377cba6640 1 Processor -- start 2024-09-12T03:16:08.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.808+0000 7f377cba6640 1 -- start start 2024-09-12T03:16:08.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.809+0000 7f377cba6640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f377806fe70 0x7f37781558d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:08.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.809+0000 7f377cba6640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f37780b1ae0 0x7f37780b0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:08.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.809+0000 7f377cba6640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f377815e900 0x7f3778147840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:08.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.809+0000 7f377cba6640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f377806eb20 con 0x7f37780b1ae0 2024-09-12T03:16:08.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.809+0000 7f377cba6640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f377806dde0 con 0x7f377815e900 2024-09-12T03:16:08.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.809+0000 7f377cba6640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f377806eca0 con 0x7f377806fe70 2024-09-12T03:16:08.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.809+0000 7f3776d9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f377815e900 0x7f3778147840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:08.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.809+0000 7f377659c640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f377806fe70 0x7f37781558d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:08.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.809+0000 7f3775d9b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f37780b1ae0 0x7f37780b0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:08.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.809+0000 7f377659c640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f377806fe70 0x7f37781558d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3301/0 says I am v2:172.21.15.45:45284/0 (socket says 172.21.15.45:45284) 2024-09-12T03:16:08.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.809+0000 7f377659c640 1 -- 172.21.15.45:0/1355279418 learned_addr learned my addr 172.21.15.45:0/1355279418 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-12T03:16:08.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.809+0000 7f3776d9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f377815e900 0x7f3778147840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50914/0 (socket says 172.21.15.45:50914) 2024-09-12T03:16:08.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.809+0000 7f377659c640 1 -- 172.21.15.45:0/1355279418 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f377815e900 msgr2=0x7f3778147840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:08.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.809+0000 7f377659c640 1 --2- 172.21.15.45:0/1355279418 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f377815e900 0x7f3778147840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:08.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.809+0000 7f377659c640 1 -- 172.21.15.45:0/1355279418 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f37780b1ae0 msgr2=0x7f37780b0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:08.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.809+0000 7f377659c640 1 --2- 172.21.15.45:0/1355279418 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f37780b1ae0 0x7f37780b0ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:08.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.809+0000 7f377659c640 1 -- 172.21.15.45:0/1355279418 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37781481b0 con 0x7f377806fe70 2024-09-12T03:16:08.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.810+0000 7f3775d9b640 1 --2- 172.21.15.45:0/1355279418 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f37780b1ae0 0x7f37780b0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:16:08.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.810+0000 7f377659c640 1 --2- 172.21.15.45:0/1355279418 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f377806fe70 0x7f37781558d0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f376800c3f0 tx=0x7f376800c8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:08.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.810+0000 7f3766ffd640 1 -- 172.21.15.45:0/1355279418 <== mon.2 v2:172.21.15.45:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37680052d0 con 0x7f377806fe70 2024-09-12T03:16:08.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.810+0000 7f3766ffd640 1 -- 172.21.15.45:0/1355279418 <== mon.2 v2:172.21.15.45:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3768005490 con 0x7f377806fe70 2024-09-12T03:16:08.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.810+0000 7f377cba6640 1 -- 172.21.15.45:0/1355279418 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3778148490 con 0x7f377806fe70 2024-09-12T03:16:08.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.810+0000 7f3766ffd640 1 -- 172.21.15.45:0/1355279418 <== mon.2 v2:172.21.15.45:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3768007260 con 0x7f377806fe70 2024-09-12T03:16:08.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.810+0000 7f377cba6640 1 -- 172.21.15.45:0/1355279418 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f3778155e10 con 0x7f377806fe70 2024-09-12T03:16:08.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.810+0000 7f3766ffd640 1 -- 172.21.15.45:0/1355279418 <== mon.2 v2:172.21.15.45:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f3768007400 con 0x7f377806fe70 2024-09-12T03:16:08.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.810+0000 7f3766ffd640 1 -- 172.21.15.45:0/1355279418 <== mon.2 v2:172.21.15.45:3301/0 5 ==== osd_map(4..4 src has 1..4) ==== 955+0+0 (secure 0 0 0) 0x7f3768011590 con 0x7f377806fe70 2024-09-12T03:16:08.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.811+0000 7f377cba6640 1 -- 172.21.15.45:0/1355279418 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f377809c240 con 0x7f377806fe70 2024-09-12T03:16:08.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.813+0000 7f3766ffd640 1 -- 172.21.15.45:0/1355279418 <== mon.2 v2:172.21.15.45:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f377809c240 con 0x7f377806fe70 2024-09-12T03:16:08.835 INFO:tasks.ceph.mgr.y.smithi027.stderr:2024-09-12T03:16:08.834+0000 7f32df11f040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-12T03:16:08.839 INFO:tasks.ceph.mgr.x.smithi045.stderr:2024-09-12T03:16:08.839+0000 7fc1f240d040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-12T03:16:08.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.885+0000 7f377cba6640 1 -- 172.21.15.45:0/1355279418 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_command({"prefix": "osd new", "uuid": "fb12dcc8-bdd0-42b8-b46d-8b10106224bb", "id": 2} v 0) -- 0x7f37780713c0 con 0x7f377806fe70 2024-09-12T03:16:08.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.888+0000 7f3766ffd640 1 -- 172.21.15.45:0/1355279418 <== mon.2 v2:172.21.15.45:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "fb12dcc8-bdd0-42b8-b46d-8b10106224bb", "id": 2}]=0 v5) ==== 112+0+2 (secure 0 0 0) 0x7f3768003630 con 0x7f377806fe70 2024-09-12T03:16:08.888 INFO:teuthology.orchestra.run.smithi045.stdout:2 2024-09-12T03:16:08.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.889+0000 7f377cba6640 1 -- 172.21.15.45:0/1355279418 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f377806fe70 msgr2=0x7f37781558d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:08.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.889+0000 7f377cba6640 1 --2- 172.21.15.45:0/1355279418 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f377806fe70 0x7f37781558d0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f376800c3f0 tx=0x7f376800c8b0 comp rx=0 tx=0).stop 2024-09-12T03:16:08.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.889+0000 7f377cba6640 1 -- 172.21.15.45:0/1355279418 shutdown_connections 2024-09-12T03:16:08.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.889+0000 7f377cba6640 1 --2- 172.21.15.45:0/1355279418 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f377815e900 0x7f3778147840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:08.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.889+0000 7f377cba6640 1 --2- 172.21.15.45:0/1355279418 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f37780b1ae0 0x7f37780b0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:08.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.889+0000 7f377cba6640 1 --2- 172.21.15.45:0/1355279418 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f377806fe70 0x7f37781558d0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:08.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.889+0000 7f377cba6640 1 -- 172.21.15.45:0/1355279418 >> 172.21.15.45:0/1355279418 conn(0x7f3778073800 msgr2=0x7f37780aff80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:08.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.889+0000 7f377cba6640 1 -- 172.21.15.45:0/1355279418 shutdown_connections 2024-09-12T03:16:08.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:08.889+0000 7f377cba6640 1 -- 172.21.15.45:0/1355279418 wait complete. 2024-09-12T03:16:08.899 DEBUG:teuthology.orchestra.run.smithi045:> sudo ceph --cluster ceph osd new d74d3f49-9c24-4b9f-98e2-3ecd7b5ef3b3 3 2024-09-12T03:16:08.994 INFO:tasks.ceph.mgr.y.smithi027.stderr:2024-09-12T03:16:08.993+0000 7f32df11f040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-12T03:16:09.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.000+0000 7f5368d18640 1 Processor -- start 2024-09-12T03:16:09.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.000+0000 7f5368d18640 1 -- start start 2024-09-12T03:16:09.000 INFO:tasks.ceph.mgr.x.smithi045.stderr:2024-09-12T03:16:09.000+0000 7fc1f240d040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-12T03:16:09.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.001+0000 7f5368d18640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f5364158d90 0x7f53640b49b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:09.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.001+0000 7f5368d18640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f536406f260 0x7f5364072900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:09.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.001+0000 7f5368d18640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f53640b5070 con 0x7f536415f830 2024-09-12T03:16:09.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.001+0000 7f5368d18640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f53640b5350 con 0x7f536406f260 2024-09-12T03:16:09.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.001+0000 7f5368d18640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f53640b5630 con 0x7f5364158d90 2024-09-12T03:16:09.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.001+0000 7f536259c640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f5364158d90 0x7f53640b49b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:09.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.001+0000 7f5361d9b640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f536415f830 0x7f536406eb50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:45052/0 (socket says 172.21.15.45:45052) 2024-09-12T03:16:09.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.001+0000 7f536259c640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f5364158d90 0x7f53640b49b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3301/0 says I am v2:172.21.15.45:45290/0 (socket says 172.21.15.45:45290) 2024-09-12T03:16:09.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.001+0000 7f5361d9b640 1 -- 172.21.15.45:0/314649002 learned_addr learned my addr 172.21.15.45:0/314649002 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-12T03:16:09.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.001+0000 7f5362d9d640 1 --2- 172.21.15.45:0/314649002 >> v2:172.21.15.27:3300/0 conn(0x7f536406f260 0x7f5364072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:09.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.001+0000 7f536259c640 1 -- 172.21.15.45:0/314649002 >> v2:172.21.15.27:3300/0 conn(0x7f536406f260 msgr2=0x7f5364072900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:09.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.001+0000 7f536259c640 1 --2- 172.21.15.45:0/314649002 >> v2:172.21.15.27:3300/0 conn(0x7f536406f260 0x7f5364072900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:09.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.001+0000 7f536259c640 1 -- 172.21.15.45:0/314649002 >> v1:172.21.15.45:6789/0 conn(0x7f536415f830 legacy=0x7f536406eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:09.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.001+0000 7f536259c640 1 -- 172.21.15.45:0/314649002 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53640b5910 con 0x7f5364158d90 2024-09-12T03:16:09.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.002+0000 7f536259c640 1 --2- 172.21.15.45:0/314649002 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f5364158d90 0x7f53640b49b0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f535c009d90 tx=0x7f535c02a5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2ecc63a99e153ecc server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:09.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.002+0000 7f536159a640 1 -- 172.21.15.45:0/314649002 <== mon.2 v2:172.21.15.45:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f535c031600 con 0x7f5364158d90 2024-09-12T03:16:09.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.002+0000 7f536159a640 1 -- 172.21.15.45:0/314649002 <== mon.2 v2:172.21.15.45:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f535c0317a0 con 0x7f5364158d90 2024-09-12T03:16:09.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.002+0000 7f536159a640 1 -- 172.21.15.45:0/314649002 <== mon.2 v2:172.21.15.45:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f535c031b70 con 0x7f5364158d90 2024-09-12T03:16:09.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.003+0000 7f5368d18640 1 -- 172.21.15.45:0/314649002 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f5364158d90 msgr2=0x7f53640b49b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:09.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.003+0000 7f5368d18640 1 --2- 172.21.15.45:0/314649002 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f5364158d90 0x7f53640b49b0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f535c009d90 tx=0x7f535c02a5d0 comp rx=0 tx=0).stop 2024-09-12T03:16:09.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.003+0000 7f5368d18640 1 -- 172.21.15.45:0/314649002 shutdown_connections 2024-09-12T03:16:09.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.003+0000 7f5368d18640 1 --2- 172.21.15.45:0/314649002 >> v2:172.21.15.27:3300/0 conn(0x7f536406f260 0x7f5364072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:09.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.003+0000 7f5368d18640 1 --2- 172.21.15.45:0/314649002 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f5364158d90 0x7f53640b49b0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:09.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.003+0000 7f5368d18640 1 -- 172.21.15.45:0/314649002 >> 172.21.15.45:0/314649002 conn(0x7f536406ca70 msgr2=0x7f536406ce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:09.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.003+0000 7f5368d18640 1 -- 172.21.15.45:0/314649002 shutdown_connections 2024-09-12T03:16:09.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.003+0000 7f5368d18640 1 -- 172.21.15.45:0/314649002 wait complete. 2024-09-12T03:16:09.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.005+0000 7f5368d18640 1 Processor -- start 2024-09-12T03:16:09.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.005+0000 7f5368d18640 1 -- start start 2024-09-12T03:16:09.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.005+0000 7f5368d18640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f536406f260 0x7f5364159d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:09.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.005+0000 7f5368d18640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5364158d90 0x7f53641f46d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:09.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.005+0000 7f5368d18640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f536415f830 0x7f5364153a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:09.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.005+0000 7f5368d18640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f5364159170 con 0x7f536406f260 2024-09-12T03:16:09.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.005+0000 7f5368d18640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f53640723b0 con 0x7f5364158d90 2024-09-12T03:16:09.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.005+0000 7f5368d18640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f5364072e40 con 0x7f536415f830 2024-09-12T03:16:09.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.005+0000 7f5362d9d640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f536415f830 0x7f5364153a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:09.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.005+0000 7f5361d9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5364158d90 0x7f53641f46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:09.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.006+0000 7f5362d9d640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f536415f830 0x7f5364153a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3301/0 says I am v2:172.21.15.45:45306/0 (socket says 172.21.15.45:45306) 2024-09-12T03:16:09.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.006+0000 7f5361d9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5364158d90 0x7f53641f46d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50924/0 (socket says 172.21.15.45:50924) 2024-09-12T03:16:09.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.006+0000 7f5362d9d640 1 -- 172.21.15.45:0/1576484401 learned_addr learned my addr 172.21.15.45:0/1576484401 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-12T03:16:09.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.006+0000 7f536259c640 1 --2- 172.21.15.45:0/1576484401 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f536406f260 0x7f5364159d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:09.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.006+0000 7f5362d9d640 1 -- 172.21.15.45:0/1576484401 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5364158d90 msgr2=0x7f53641f46d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:09.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.006+0000 7f5362d9d640 1 --2- 172.21.15.45:0/1576484401 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5364158d90 0x7f53641f46d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:09.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.006+0000 7f5362d9d640 1 -- 172.21.15.45:0/1576484401 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f536406f260 msgr2=0x7f5364159d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:09.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.006+0000 7f5362d9d640 1 --2- 172.21.15.45:0/1576484401 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f536406f260 0x7f5364159d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:09.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.006+0000 7f5362d9d640 1 -- 172.21.15.45:0/1576484401 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f536415a240 con 0x7f536415f830 2024-09-12T03:16:09.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.006+0000 7f536259c640 1 --2- 172.21.15.45:0/1576484401 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f536406f260 0x7f5364159d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:16:09.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.006+0000 7f5361d9b640 1 --2- 172.21.15.45:0/1576484401 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5364158d90 0x7f53641f46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:16:09.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.006+0000 7f5362d9d640 1 --2- 172.21.15.45:0/1576484401 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f536415f830 0x7f5364153a70 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f535400e850 tx=0x7f535400ed10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:09.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.006+0000 7f535affd640 1 -- 172.21.15.45:0/1576484401 <== mon.2 v2:172.21.15.45:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5354017070 con 0x7f536415f830 2024-09-12T03:16:09.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.006+0000 7f535affd640 1 -- 172.21.15.45:0/1576484401 <== mon.2 v2:172.21.15.45:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f535400c940 con 0x7f536415f830 2024-09-12T03:16:09.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.006+0000 7f5368d18640 1 -- 172.21.15.45:0/1576484401 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f536414a430 con 0x7f536415f830 2024-09-12T03:16:09.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.006+0000 7f535affd640 1 -- 172.21.15.45:0/1576484401 <== mon.2 v2:172.21.15.45:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f535400cd30 con 0x7f536415f830 2024-09-12T03:16:09.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.006+0000 7f5368d18640 1 -- 172.21.15.45:0/1576484401 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f5364155b20 con 0x7f536415f830 2024-09-12T03:16:09.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.006+0000 7f535affd640 1 -- 172.21.15.45:0/1576484401 <== mon.2 v2:172.21.15.45:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f535401a070 con 0x7f536415f830 2024-09-12T03:16:09.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.007+0000 7f535affd640 1 -- 172.21.15.45:0/1576484401 <== mon.2 v2:172.21.15.45:3301/0 5 ==== osd_map(5..5 src has 1..5) ==== 1070+0+0 (secure 0 0 0) 0x7f53540065c0 con 0x7f536415f830 2024-09-12T03:16:09.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.007+0000 7f5368d18640 1 -- 172.21.15.45:0/1576484401 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5364072900 con 0x7f536415f830 2024-09-12T03:16:09.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.011+0000 7f535affd640 1 -- 172.21.15.45:0/1576484401 <== mon.2 v2:172.21.15.45:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f536415a240 con 0x7f536415f830 2024-09-12T03:16:09.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.083+0000 7f5368d18640 1 -- 172.21.15.45:0/1576484401 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_command({"prefix": "osd new", "uuid": "d74d3f49-9c24-4b9f-98e2-3ecd7b5ef3b3", "id": 3} v 0) -- 0x7f53640b2870 con 0x7f536415f830 2024-09-12T03:16:09.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.088+0000 7f535affd640 1 -- 172.21.15.45:0/1576484401 <== mon.2 v2:172.21.15.45:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "d74d3f49-9c24-4b9f-98e2-3ecd7b5ef3b3", "id": 3}]=0 v6) ==== 112+0+2 (secure 0 0 0) 0x7f5354006020 con 0x7f536415f830 2024-09-12T03:16:09.088 INFO:teuthology.orchestra.run.smithi045.stdout:3 2024-09-12T03:16:09.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.089+0000 7f5368d18640 1 -- 172.21.15.45:0/1576484401 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f536415f830 msgr2=0x7f5364153a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:09.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.089+0000 7f5368d18640 1 --2- 172.21.15.45:0/1576484401 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f536415f830 0x7f5364153a70 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f535400e850 tx=0x7f535400ed10 comp rx=0 tx=0).stop 2024-09-12T03:16:09.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.089+0000 7f5368d18640 1 -- 172.21.15.45:0/1576484401 shutdown_connections 2024-09-12T03:16:09.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.089+0000 7f5368d18640 1 --2- 172.21.15.45:0/1576484401 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f536415f830 0x7f5364153a70 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:09.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.089+0000 7f5368d18640 1 --2- 172.21.15.45:0/1576484401 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5364158d90 0x7f53641f46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:09.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.089+0000 7f5368d18640 1 --2- 172.21.15.45:0/1576484401 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f536406f260 0x7f5364159d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:09.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.089+0000 7f5368d18640 1 -- 172.21.15.45:0/1576484401 >> 172.21.15.45:0/1576484401 conn(0x7f536406ca70 msgr2=0x7f53640b5dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:09.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.089+0000 7f5368d18640 1 -- 172.21.15.45:0/1576484401 shutdown_connections 2024-09-12T03:16:09.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.089+0000 7f5368d18640 1 -- 172.21.15.45:0/1576484401 wait complete. 2024-09-12T03:16:09.100 DEBUG:teuthology.orchestra.run.smithi045:> sudo ceph --cluster ceph osd new 2bf123f2-1b0d-41c4-aa3b-805bab267e59 4 2024-09-12T03:16:09.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.201+0000 7ff1f519f640 1 Processor -- start 2024-09-12T03:16:09.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.201+0000 7ff1f519f640 1 -- start start 2024-09-12T03:16:09.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.202+0000 7ff1f519f640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7ff1f00d11b0 0x7ff1f006f480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:09.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.202+0000 7ff1f519f640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ff1f00ceea0 0x7ff1f006c370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:09.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.202+0000 7ff1f519f640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff1f015cf20 con 0x7ff1f006bf50 2024-09-12T03:16:09.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.202+0000 7ff1f519f640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7ff1f015d200 con 0x7ff1f00d11b0 2024-09-12T03:16:09.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.202+0000 7ff1f519f640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7ff1f015d4e0 con 0x7ff1f00ceea0 2024-09-12T03:16:09.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.202+0000 7ff1ef59e640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ff1f00ceea0 0x7ff1f006c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:09.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.202+0000 7ff1ef59e640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ff1f00ceea0 0x7ff1f006c370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3301/0 says I am v2:172.21.15.45:45322/0 (socket says 172.21.15.45:45322) 2024-09-12T03:16:09.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.202+0000 7ff1ef59e640 1 -- 172.21.15.45:0/1024530738 learned_addr learned my addr 172.21.15.45:0/1024530738 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-12T03:16:09.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.202+0000 7ff1eed9d640 1 --2- 172.21.15.45:0/1024530738 >> v2:172.21.15.27:3300/0 conn(0x7ff1f00d11b0 0x7ff1f006f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:09.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.203+0000 7ff1ef59e640 1 -- 172.21.15.45:0/1024530738 >> v2:172.21.15.27:3300/0 conn(0x7ff1f00d11b0 msgr2=0x7ff1f006f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:09.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.203+0000 7ff1ef59e640 1 --2- 172.21.15.45:0/1024530738 >> v2:172.21.15.27:3300/0 conn(0x7ff1f00d11b0 0x7ff1f006f480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:09.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.203+0000 7ff1ef59e640 1 -- 172.21.15.45:0/1024530738 >> v1:172.21.15.27:6789/0 conn(0x7ff1f006bf50 legacy=0x7ff1f00ce790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:09.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.203+0000 7ff1ef59e640 1 -- 172.21.15.45:0/1024530738 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1f015d7c0 con 0x7ff1f00ceea0 2024-09-12T03:16:09.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.203+0000 7ff1ef59e640 1 --2- 172.21.15.45:0/1024530738 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ff1f00ceea0 0x7ff1f006c370 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7ff1e8009870 tx=0x7ff1e802a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e9365c263ae1d656 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:09.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.203+0000 7ff1edd9b640 1 -- 172.21.15.45:0/1024530738 <== mon.2 v2:172.21.15.45:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1e8037070 con 0x7ff1f00ceea0 2024-09-12T03:16:09.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.203+0000 7ff1edd9b640 1 -- 172.21.15.45:0/1024530738 <== mon.2 v2:172.21.15.45:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1e8032440 con 0x7ff1f00ceea0 2024-09-12T03:16:09.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.203+0000 7ff1edd9b640 1 -- 172.21.15.45:0/1024530738 <== mon.2 v2:172.21.15.45:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1e8032810 con 0x7ff1f00ceea0 2024-09-12T03:16:09.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.204+0000 7ff1f519f640 1 -- 172.21.15.45:0/1024530738 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ff1f00ceea0 msgr2=0x7ff1f006c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:09.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.204+0000 7ff1f519f640 1 --2- 172.21.15.45:0/1024530738 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ff1f00ceea0 0x7ff1f006c370 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7ff1e8009870 tx=0x7ff1e802a270 comp rx=0 tx=0).stop 2024-09-12T03:16:09.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.204+0000 7ff1f519f640 1 -- 172.21.15.45:0/1024530738 shutdown_connections 2024-09-12T03:16:09.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.204+0000 7ff1f519f640 1 --2- 172.21.15.45:0/1024530738 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ff1f00ceea0 0x7ff1f006c370 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:09.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.204+0000 7ff1f519f640 1 --2- 172.21.15.45:0/1024530738 >> v2:172.21.15.27:3300/0 conn(0x7ff1f00d11b0 0x7ff1f006f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:09.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.204+0000 7ff1f519f640 1 -- 172.21.15.45:0/1024530738 >> 172.21.15.45:0/1024530738 conn(0x7ff1f00b3370 msgr2=0x7ff1f00b3780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:09.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.204+0000 7ff1f519f640 1 -- 172.21.15.45:0/1024530738 shutdown_connections 2024-09-12T03:16:09.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.204+0000 7ff1f519f640 1 -- 172.21.15.45:0/1024530738 wait complete. 2024-09-12T03:16:09.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.206+0000 7ff1f519f640 1 Processor -- start 2024-09-12T03:16:09.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.206+0000 7ff1f519f640 1 -- start start 2024-09-12T03:16:09.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.206+0000 7ff1f519f640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff1f006bf50 0x7ff1f00ae8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:09.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.206+0000 7ff1f519f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff1f00ceea0 0x7ff1f0148fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:09.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.206+0000 7ff1f519f640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ff1f00d11b0 0x7ff1f00b4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:09.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.206+0000 7ff1f519f640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7ff1f006e470 con 0x7ff1f006bf50 2024-09-12T03:16:09.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.206+0000 7ff1f519f640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ff1f006e2f0 con 0x7ff1f00ceea0 2024-09-12T03:16:09.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.206+0000 7ff1f519f640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7ff1f006e5f0 con 0x7ff1f00d11b0 2024-09-12T03:16:09.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.207+0000 7ff1ef59e640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ff1f00d11b0 0x7ff1f00b4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:09.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.207+0000 7ff1ee59c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff1f00ceea0 0x7ff1f0148fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:09.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.207+0000 7ff1ef59e640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ff1f00d11b0 0x7ff1f00b4770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3301/0 says I am v2:172.21.15.45:45328/0 (socket says 172.21.15.45:45328) 2024-09-12T03:16:09.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.207+0000 7ff1ee59c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff1f00ceea0 0x7ff1f0148fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50940/0 (socket says 172.21.15.45:50940) 2024-09-12T03:16:09.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.207+0000 7ff1eed9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff1f006bf50 0x7ff1f00ae8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:09.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.207+0000 7ff1ef59e640 1 -- 172.21.15.45:0/1828739620 learned_addr learned my addr 172.21.15.45:0/1828739620 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-12T03:16:09.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.207+0000 7ff1ef59e640 1 -- 172.21.15.45:0/1828739620 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff1f00ceea0 msgr2=0x7ff1f0148fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:09.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.207+0000 7ff1ef59e640 1 --2- 172.21.15.45:0/1828739620 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff1f00ceea0 0x7ff1f0148fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:09.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.207+0000 7ff1ef59e640 1 -- 172.21.15.45:0/1828739620 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff1f006bf50 msgr2=0x7ff1f00ae8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:09.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.207+0000 7ff1ef59e640 1 --2- 172.21.15.45:0/1828739620 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff1f006bf50 0x7ff1f00ae8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:09.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.207+0000 7ff1ef59e640 1 -- 172.21.15.45:0/1828739620 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1f01547f0 con 0x7ff1f00d11b0 2024-09-12T03:16:09.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.207+0000 7ff1ee59c640 1 --2- 172.21.15.45:0/1828739620 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff1f00ceea0 0x7ff1f0148fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:16:09.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.207+0000 7ff1eed9d640 1 --2- 172.21.15.45:0/1828739620 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff1f006bf50 0x7ff1f00ae8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:16:09.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.207+0000 7ff1ef59e640 1 --2- 172.21.15.45:0/1828739620 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ff1f00d11b0 0x7ff1f00b4770 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7ff1e8009870 tx=0x7ff1e80046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:09.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.207+0000 7ff1e77fe640 1 -- 172.21.15.45:0/1828739620 <== mon.2 v2:172.21.15.45:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1e8041020 con 0x7ff1f00d11b0 2024-09-12T03:16:09.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.207+0000 7ff1e77fe640 1 -- 172.21.15.45:0/1828739620 <== mon.2 v2:172.21.15.45:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1e803d640 con 0x7ff1f00d11b0 2024-09-12T03:16:09.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.207+0000 7ff1e77fe640 1 -- 172.21.15.45:0/1828739620 <== mon.2 v2:172.21.15.45:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1e8037040 con 0x7ff1f00d11b0 2024-09-12T03:16:09.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.207+0000 7ff1f519f640 1 -- 172.21.15.45:0/1828739620 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1f00aefd0 con 0x7ff1f00d11b0 2024-09-12T03:16:09.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.207+0000 7ff1f519f640 1 -- 172.21.15.45:0/1828739620 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1f00af590 con 0x7ff1f00d11b0 2024-09-12T03:16:09.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.208+0000 7ff1e77fe640 1 -- 172.21.15.45:0/1828739620 <== mon.2 v2:172.21.15.45:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7ff1e8044020 con 0x7ff1f00d11b0 2024-09-12T03:16:09.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.208+0000 7ff1e77fe640 1 -- 172.21.15.45:0/1828739620 <== mon.2 v2:172.21.15.45:3301/0 5 ==== osd_map(6..6 src has 1..6) ==== 1185+0+0 (secure 0 0 0) 0x7ff1e8046b80 con 0x7ff1f00d11b0 2024-09-12T03:16:09.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.208+0000 7ff1f519f640 1 -- 172.21.15.45:0/1828739620 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1f006c370 con 0x7ff1f00d11b0 2024-09-12T03:16:09.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.210+0000 7ff1e77fe640 1 -- 172.21.15.45:0/1828739620 <== mon.2 v2:172.21.15.45:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7ff1e804b030 con 0x7ff1f00d11b0 2024-09-12T03:16:09.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.281+0000 7ff1f519f640 1 -- 172.21.15.45:0/1828739620 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_command({"prefix": "osd new", "uuid": "2bf123f2-1b0d-41c4-aa3b-805bab267e59", "id": 4} v 0) -- 0x7ff1f00b0fa0 con 0x7ff1f00d11b0 2024-09-12T03:16:09.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.287+0000 7ff1e77fe640 1 -- 172.21.15.45:0/1828739620 <== mon.2 v2:172.21.15.45:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "2bf123f2-1b0d-41c4-aa3b-805bab267e59", "id": 4}]=0 v7) ==== 112+0+2 (secure 0 0 0) 0x7ff1e8046e20 con 0x7ff1f00d11b0 2024-09-12T03:16:09.288 INFO:teuthology.orchestra.run.smithi045.stdout:4 2024-09-12T03:16:09.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.288+0000 7ff1f519f640 1 -- 172.21.15.45:0/1828739620 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ff1f00d11b0 msgr2=0x7ff1f00b4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:09.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.288+0000 7ff1f519f640 1 --2- 172.21.15.45:0/1828739620 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ff1f00d11b0 0x7ff1f00b4770 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7ff1e8009870 tx=0x7ff1e80046c0 comp rx=0 tx=0).stop 2024-09-12T03:16:09.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.288+0000 7ff1f519f640 1 -- 172.21.15.45:0/1828739620 shutdown_connections 2024-09-12T03:16:09.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.288+0000 7ff1f519f640 1 --2- 172.21.15.45:0/1828739620 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ff1f00d11b0 0x7ff1f00b4770 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:09.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.288+0000 7ff1f519f640 1 --2- 172.21.15.45:0/1828739620 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff1f00ceea0 0x7ff1f0148fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:09.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.288+0000 7ff1f519f640 1 --2- 172.21.15.45:0/1828739620 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff1f006bf50 0x7ff1f00ae8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:09.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.288+0000 7ff1f519f640 1 -- 172.21.15.45:0/1828739620 >> 172.21.15.45:0/1828739620 conn(0x7ff1f00b3370 msgr2=0x7ff1f00ce2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:09.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.288+0000 7ff1f519f640 1 -- 172.21.15.45:0/1828739620 shutdown_connections 2024-09-12T03:16:09.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.288+0000 7ff1f519f640 1 -- 172.21.15.45:0/1828739620 wait complete. 2024-09-12T03:16:09.298 DEBUG:teuthology.orchestra.run.smithi045:> sudo ceph --cluster ceph osd new c49ba5da-91c6-4a86-a97b-4aba89c253ec 5 2024-09-12T03:16:09.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.413+0000 7f6897b31640 1 Processor -- start 2024-09-12T03:16:09.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.414+0000 7f6897b31640 1 -- start start 2024-09-12T03:16:09.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.415+0000 7f6897b31640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f689006e1d0 0x7f6890169590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:09.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.415+0000 7f6897b31640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6890070220 con 0x7f68900b6ce0 2024-09-12T03:16:09.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.415+0000 7f6897b31640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f689006f180 con 0x7f68900b0530 2024-09-12T03:16:09.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.415+0000 7f6897b31640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f689006fd90 con 0x7f689006e1d0 2024-09-12T03:16:09.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.415+0000 7f68960ce640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f689006e1d0 0x7f6890169590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:09.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.415+0000 7f68950cc640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f68900b0530 0x7f689006dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:45054/0 (socket says 172.21.15.45:45054) 2024-09-12T03:16:09.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.415+0000 7f68950cc640 1 -- 172.21.15.45:0/3513994114 learned_addr learned my addr 172.21.15.45:0/3513994114 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-12T03:16:09.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.415+0000 7f68948cb640 1 -- 172.21.15.45:0/3513994114 <== mon.1 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1760928642 0 0) 0x7f689006f180 con 0x7f68900b0530 2024-09-12T03:16:09.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.415+0000 7f68960ce640 1 -- 172.21.15.45:0/3513994114 >> v1:172.21.15.45:6789/0 conn(0x7f68900b0530 legacy=0x7f689006dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:09.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.415+0000 7f68960ce640 1 -- 172.21.15.45:0/3513994114 >> v1:172.21.15.27:6789/0 conn(0x7f68900b6ce0 legacy=0x7f689015f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:09.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.415+0000 7f68960ce640 1 -- 172.21.15.45:0/3513994114 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f689006f460 con 0x7f689006e1d0 2024-09-12T03:16:09.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.416+0000 7f68960ce640 1 --2- 172.21.15.45:0/3513994114 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f689006e1d0 0x7f6890169590 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f6888009dc0 tx=0x7f688802a6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a8186f6118118ea7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:09.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.416+0000 7f68948cb640 1 -- 172.21.15.45:0/3513994114 <== mon.2 v2:172.21.15.45:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6888007740 con 0x7f689006e1d0 2024-09-12T03:16:09.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.416+0000 7f68948cb640 1 -- 172.21.15.45:0/3513994114 <== mon.2 v2:172.21.15.45:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68880078e0 con 0x7f689006e1d0 2024-09-12T03:16:09.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.416+0000 7f68948cb640 1 -- 172.21.15.45:0/3513994114 <== mon.2 v2:172.21.15.45:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6888007cb0 con 0x7f689006e1d0 2024-09-12T03:16:09.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.417+0000 7f6897b31640 1 -- 172.21.15.45:0/3513994114 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f689006e1d0 msgr2=0x7f6890169590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:09.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.417+0000 7f6897b31640 1 --2- 172.21.15.45:0/3513994114 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f689006e1d0 0x7f6890169590 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f6888009dc0 tx=0x7f688802a6d0 comp rx=0 tx=0).stop 2024-09-12T03:16:09.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.417+0000 7f6897b31640 1 -- 172.21.15.45:0/3513994114 shutdown_connections 2024-09-12T03:16:09.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.417+0000 7f6897b31640 1 --2- 172.21.15.45:0/3513994114 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f689006e1d0 0x7f6890169590 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:09.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.417+0000 7f6897b31640 1 -- 172.21.15.45:0/3513994114 >> 172.21.15.45:0/3513994114 conn(0x7f68900b0b50 msgr2=0x7f68900b0f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:09.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.417+0000 7f6897b31640 1 -- 172.21.15.45:0/3513994114 shutdown_connections 2024-09-12T03:16:09.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.417+0000 7f6897b31640 1 -- 172.21.15.45:0/3513994114 wait complete. 2024-09-12T03:16:09.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.418+0000 7f6897b31640 1 Processor -- start 2024-09-12T03:16:09.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.418+0000 7f6897b31640 1 -- start start 2024-09-12T03:16:09.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.419+0000 7f6897b31640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f689006e1d0 0x7f6890158a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:09.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.419+0000 7f6897b31640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f68900b0530 0x7f6890153b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:09.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.419+0000 7f6897b31640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f68900b6ce0 0x7f68900d0540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:09.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.419+0000 7f6897b31640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f68900af8e0 con 0x7f689006e1d0 2024-09-12T03:16:09.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.419+0000 7f6897b31640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f689006fa20 con 0x7f68900b6ce0 2024-09-12T03:16:09.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.419+0000 7f6897b31640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f6890169ad0 con 0x7f68900b0530 2024-09-12T03:16:09.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.419+0000 7f68950cc640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f68900b0530 0x7f6890153b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:09.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.419+0000 7f68960ce640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f68900b6ce0 0x7f68900d0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:09.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.419+0000 7f68958cd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f689006e1d0 0x7f6890158a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:09.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.419+0000 7f68950cc640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f68900b0530 0x7f6890153b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3301/0 says I am v2:172.21.15.45:45340/0 (socket says 172.21.15.45:45340) 2024-09-12T03:16:09.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.419+0000 7f68950cc640 1 -- 172.21.15.45:0/786710211 learned_addr learned my addr 172.21.15.45:0/786710211 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-12T03:16:09.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.419+0000 7f68950cc640 1 -- 172.21.15.45:0/786710211 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f68900b6ce0 msgr2=0x7f68900d0540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:09.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.419+0000 7f68950cc640 1 --2- 172.21.15.45:0/786710211 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f68900b6ce0 0x7f68900d0540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:09.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.419+0000 7f68950cc640 1 -- 172.21.15.45:0/786710211 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f689006e1d0 msgr2=0x7f6890158a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:09.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.419+0000 7f68950cc640 1 --2- 172.21.15.45:0/786710211 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f689006e1d0 0x7f6890158a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:09.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.419+0000 7f68950cc640 1 -- 172.21.15.45:0/786710211 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6890154b60 con 0x7f68900b0530 2024-09-12T03:16:09.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.419+0000 7f68950cc640 1 --2- 172.21.15.45:0/786710211 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f68900b0530 0x7f6890153b00 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f688000c4d0 tx=0x7f688000c990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:09.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.419+0000 7f68867fc640 1 -- 172.21.15.45:0/786710211 <== mon.2 v2:172.21.15.45:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6880018070 con 0x7f68900b0530 2024-09-12T03:16:09.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.419+0000 7f68867fc640 1 -- 172.21.15.45:0/786710211 <== mon.2 v2:172.21.15.45:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68800074e0 con 0x7f68900b0530 2024-09-12T03:16:09.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.419+0000 7f68867fc640 1 -- 172.21.15.45:0/786710211 <== mon.2 v2:172.21.15.45:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6880004a20 con 0x7f68900b0530 2024-09-12T03:16:09.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.420+0000 7f6897b31640 1 -- 172.21.15.45:0/786710211 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6890154040 con 0x7f68900b0530 2024-09-12T03:16:09.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.420+0000 7f6897b31640 1 -- 172.21.15.45:0/786710211 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f68901545a0 con 0x7f68900b0530 2024-09-12T03:16:09.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.420+0000 7f68867fc640 1 -- 172.21.15.45:0/786710211 <== mon.2 v2:172.21.15.45:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f6880004bc0 con 0x7f68900b0530 2024-09-12T03:16:09.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.420+0000 7f68867fc640 1 -- 172.21.15.45:0/786710211 <== mon.2 v2:172.21.15.45:3301/0 5 ==== osd_map(7..7 src has 1..7) ==== 1300+0+0 (secure 0 0 0) 0x7f688001b8e0 con 0x7f68900b0530 2024-09-12T03:16:09.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.420+0000 7f6897b31640 1 -- 172.21.15.45:0/786710211 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f689006dac0 con 0x7f68900b0530 2024-09-12T03:16:09.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.423+0000 7f68867fc640 1 -- 172.21.15.45:0/786710211 <== mon.2 v2:172.21.15.45:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f6890154040 con 0x7f68900b0530 2024-09-12T03:16:09.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.494+0000 7f6897b31640 1 -- 172.21.15.45:0/786710211 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_command({"prefix": "osd new", "uuid": "c49ba5da-91c6-4a86-a97b-4aba89c253ec", "id": 5} v 0) -- 0x7f68900b0310 con 0x7f68900b0530 2024-09-12T03:16:09.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.497+0000 7f68867fc640 1 -- 172.21.15.45:0/786710211 <== mon.2 v2:172.21.15.45:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "c49ba5da-91c6-4a86-a97b-4aba89c253ec", "id": 5}]=0 v8) ==== 112+0+2 (secure 0 0 0) 0x7f688001bb00 con 0x7f68900b0530 2024-09-12T03:16:09.496 INFO:teuthology.orchestra.run.smithi045.stdout:5 2024-09-12T03:16:09.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.497+0000 7f6897b31640 1 -- 172.21.15.45:0/786710211 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f68900b0530 msgr2=0x7f6890153b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:09.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.497+0000 7f6897b31640 1 --2- 172.21.15.45:0/786710211 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f68900b0530 0x7f6890153b00 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f688000c4d0 tx=0x7f688000c990 comp rx=0 tx=0).stop 2024-09-12T03:16:09.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.498+0000 7f6897b31640 1 -- 172.21.15.45:0/786710211 shutdown_connections 2024-09-12T03:16:09.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.498+0000 7f6897b31640 1 --2- 172.21.15.45:0/786710211 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f68900b6ce0 0x7f68900d0540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:09.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.498+0000 7f6897b31640 1 --2- 172.21.15.45:0/786710211 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f68900b0530 0x7f6890153b00 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:09.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.498+0000 7f6897b31640 1 --2- 172.21.15.45:0/786710211 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f689006e1d0 0x7f6890158a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:09.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.498+0000 7f6897b31640 1 -- 172.21.15.45:0/786710211 >> 172.21.15.45:0/786710211 conn(0x7f68900b0b50 msgr2=0x7f68900722b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:09.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.498+0000 7f6897b31640 1 -- 172.21.15.45:0/786710211 shutdown_connections 2024-09-12T03:16:09.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.498+0000 7f6897b31640 1 -- 172.21.15.45:0/786710211 wait complete. 2024-09-12T03:16:09.508 DEBUG:teuthology.orchestra.run.smithi045:> sudo ceph --cluster ceph osd new b0123475-89f7-4a34-9e33-d797b99dd9dc 6 2024-09-12T03:16:09.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.609+0000 7fe480703640 1 Processor -- start 2024-09-12T03:16:09.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.610+0000 7fe480703640 1 -- start start 2024-09-12T03:16:09.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.611+0000 7fe480703640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe4780b0530 0x7fe47806dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:09.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.611+0000 7fe480703640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe478070220 con 0x7fe4780b6ce0 2024-09-12T03:16:09.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.611+0000 7fe480703640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe47806f180 con 0x7fe47806e000 2024-09-12T03:16:09.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.611+0000 7fe480703640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fe47806fd90 con 0x7fe4780b0530 2024-09-12T03:16:09.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.611+0000 7fe47eca0640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fe47806e000 0x7fe4780ae8d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:45062/0 (socket says 172.21.15.45:45062) 2024-09-12T03:16:09.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.611+0000 7fe47eca0640 1 -- 172.21.15.45:0/3648333960 learned_addr learned my addr 172.21.15.45:0/3648333960 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-12T03:16:09.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.611+0000 7fe47dc9e640 1 --2- 172.21.15.45:0/3648333960 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe4780b0530 0x7fe47806dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:09.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.611+0000 7fe47d49d640 1 -- 172.21.15.45:0/3648333960 <== mon.1 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 718202476 0 0) 0x7fe47806f180 con 0x7fe47806e000 2024-09-12T03:16:09.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.611+0000 7fe47d49d640 1 -- 172.21.15.45:0/3648333960 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe464003520 con 0x7fe47806e000 2024-09-12T03:16:09.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.611+0000 7fe47dc9e640 1 -- 172.21.15.45:0/3648333960 >> v1:172.21.15.45:6789/0 conn(0x7fe47806e000 legacy=0x7fe4780ae8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:09.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.611+0000 7fe47dc9e640 1 -- 172.21.15.45:0/3648333960 >> v1:172.21.15.27:6789/0 conn(0x7fe4780b6ce0 legacy=0x7fe47815f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:09.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.611+0000 7fe47dc9e640 1 -- 172.21.15.45:0/3648333960 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe47806f460 con 0x7fe4780b0530 2024-09-12T03:16:09.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.612+0000 7fe47dc9e640 1 --2- 172.21.15.45:0/3648333960 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe4780b0530 0x7fe47806dac0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fe4680097d0 tx=0x7fe46802a1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6811f6c14677c9c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:09.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.612+0000 7fe47d49d640 1 -- 172.21.15.45:0/3648333960 <== mon.2 v2:172.21.15.45:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe468037070 con 0x7fe4780b0530 2024-09-12T03:16:09.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.612+0000 7fe47d49d640 1 -- 172.21.15.45:0/3648333960 <== mon.2 v2:172.21.15.45:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe468032440 con 0x7fe4780b0530 2024-09-12T03:16:09.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.612+0000 7fe47d49d640 1 -- 172.21.15.45:0/3648333960 <== mon.2 v2:172.21.15.45:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe468032810 con 0x7fe4780b0530 2024-09-12T03:16:09.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.613+0000 7fe480703640 1 -- 172.21.15.45:0/3648333960 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe4780b0530 msgr2=0x7fe47806dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:09.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.613+0000 7fe480703640 1 --2- 172.21.15.45:0/3648333960 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe4780b0530 0x7fe47806dac0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fe4680097d0 tx=0x7fe46802a1c0 comp rx=0 tx=0).stop 2024-09-12T03:16:09.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.613+0000 7fe480703640 1 -- 172.21.15.45:0/3648333960 shutdown_connections 2024-09-12T03:16:09.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.613+0000 7fe480703640 1 --2- 172.21.15.45:0/3648333960 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe4780b0530 0x7fe47806dac0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:09.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.613+0000 7fe480703640 1 -- 172.21.15.45:0/3648333960 >> 172.21.15.45:0/3648333960 conn(0x7fe4780b0b50 msgr2=0x7fe4780b0f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:09.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.613+0000 7fe480703640 1 -- 172.21.15.45:0/3648333960 shutdown_connections 2024-09-12T03:16:09.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.613+0000 7fe480703640 1 -- 172.21.15.45:0/3648333960 wait complete. 2024-09-12T03:16:09.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.614+0000 7fe480703640 1 Processor -- start 2024-09-12T03:16:09.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.614+0000 7fe480703640 1 -- start start 2024-09-12T03:16:09.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.615+0000 7fe480703640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe47806e000 0x7fe478161120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:09.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.615+0000 7fe480703640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe4780b0530 0x7fe47815f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:09.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.615+0000 7fe480703640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe4780b6ce0 0x7fe478154900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:09.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.615+0000 7fe480703640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fe4780aefe0 con 0x7fe47806e000 2024-09-12T03:16:09.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.615+0000 7fe480703640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe47806fa20 con 0x7fe4780b6ce0 2024-09-12T03:16:09.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.615+0000 7fe480703640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fe4780af160 con 0x7fe4780b0530 2024-09-12T03:16:09.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.615+0000 7fe47dc9e640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe4780b0530 0x7fe47815f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:09.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.615+0000 7fe47eca0640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe4780b6ce0 0x7fe478154900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:09.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.615+0000 7fe47dc9e640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe4780b0530 0x7fe47815f4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3301/0 says I am v2:172.21.15.45:45350/0 (socket says 172.21.15.45:45350) 2024-09-12T03:16:09.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.615+0000 7fe47dc9e640 1 -- 172.21.15.45:0/67955439 learned_addr learned my addr 172.21.15.45:0/67955439 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-12T03:16:09.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.615+0000 7fe47e49f640 1 --2- 172.21.15.45:0/67955439 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe47806e000 0x7fe478161120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:09.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.616+0000 7fe47dc9e640 1 -- 172.21.15.45:0/67955439 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe4780b6ce0 msgr2=0x7fe478154900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:09.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.616+0000 7fe47dc9e640 1 --2- 172.21.15.45:0/67955439 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe4780b6ce0 0x7fe478154900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:09.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.616+0000 7fe47dc9e640 1 -- 172.21.15.45:0/67955439 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe47806e000 msgr2=0x7fe478161120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:09.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.616+0000 7fe47dc9e640 1 --2- 172.21.15.45:0/67955439 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe47806e000 0x7fe478161120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:09.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.616+0000 7fe47dc9e640 1 -- 172.21.15.45:0/67955439 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe47815a330 con 0x7fe4780b0530 2024-09-12T03:16:09.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.616+0000 7fe47eca0640 1 --2- 172.21.15.45:0/67955439 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe4780b6ce0 0x7fe478154900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:16:09.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.616+0000 7fe47dc9e640 1 --2- 172.21.15.45:0/67955439 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe4780b0530 0x7fe47815f4b0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fe4680097d0 tx=0x7fe4680046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:09.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.616+0000 7fe46effd640 1 -- 172.21.15.45:0/67955439 <== mon.2 v2:172.21.15.45:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe468041070 con 0x7fe4780b0530 2024-09-12T03:16:09.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.616+0000 7fe46effd640 1 -- 172.21.15.45:0/67955439 <== mon.2 v2:172.21.15.45:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe46803c890 con 0x7fe4780b0530 2024-09-12T03:16:09.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.616+0000 7fe480703640 1 -- 172.21.15.45:0/67955439 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe47815fb20 con 0x7fe4780b0530 2024-09-12T03:16:09.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.616+0000 7fe46effd640 1 -- 172.21.15.45:0/67955439 <== mon.2 v2:172.21.15.45:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe468037040 con 0x7fe4780b0530 2024-09-12T03:16:09.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.616+0000 7fe480703640 1 -- 172.21.15.45:0/67955439 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4781600e0 con 0x7fe4780b0530 2024-09-12T03:16:09.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.616+0000 7fe46effd640 1 -- 172.21.15.45:0/67955439 <== mon.2 v2:172.21.15.45:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fe468044070 con 0x7fe4780b0530 2024-09-12T03:16:09.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.616+0000 7fe46effd640 1 -- 172.21.15.45:0/67955439 <== mon.2 v2:172.21.15.45:3301/0 5 ==== osd_map(8..8 src has 1..8) ==== 1415+0+0 (secure 0 0 0) 0x7fe46803d3f0 con 0x7fe4780b0530 2024-09-12T03:16:09.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.617+0000 7fe480703640 1 -- 172.21.15.45:0/67955439 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe47806dac0 con 0x7fe4780b0530 2024-09-12T03:16:09.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.621+0000 7fe46effd640 1 -- 172.21.15.45:0/67955439 <== mon.2 v2:172.21.15.45:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7fe46803cc80 con 0x7fe4780b0530 2024-09-12T03:16:09.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.693+0000 7fe480703640 1 -- 172.21.15.45:0/67955439 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_command({"prefix": "osd new", "uuid": "b0123475-89f7-4a34-9e33-d797b99dd9dc", "id": 6} v 0) -- 0x7fe4780cd6a0 con 0x7fe4780b0530 2024-09-12T03:16:09.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.698+0000 7fe46effd640 1 -- 172.21.15.45:0/67955439 <== mon.2 v2:172.21.15.45:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "b0123475-89f7-4a34-9e33-d797b99dd9dc", "id": 6}]=0 v9) ==== 112+0+2 (secure 0 0 0) 0x7fe468046450 con 0x7fe4780b0530 2024-09-12T03:16:09.698 INFO:teuthology.orchestra.run.smithi045.stdout:6 2024-09-12T03:16:09.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.699+0000 7fe480703640 1 -- 172.21.15.45:0/67955439 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe4780b0530 msgr2=0x7fe47815f4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:09.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.699+0000 7fe480703640 1 --2- 172.21.15.45:0/67955439 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe4780b0530 0x7fe47815f4b0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fe4680097d0 tx=0x7fe4680046c0 comp rx=0 tx=0).stop 2024-09-12T03:16:09.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.699+0000 7fe480703640 1 -- 172.21.15.45:0/67955439 shutdown_connections 2024-09-12T03:16:09.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.699+0000 7fe480703640 1 --2- 172.21.15.45:0/67955439 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe4780b6ce0 0x7fe478154900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:09.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.699+0000 7fe480703640 1 --2- 172.21.15.45:0/67955439 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe4780b0530 0x7fe47815f4b0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:09.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.699+0000 7fe480703640 1 --2- 172.21.15.45:0/67955439 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe47806e000 0x7fe478161120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:09.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.699+0000 7fe480703640 1 -- 172.21.15.45:0/67955439 >> 172.21.15.45:0/67955439 conn(0x7fe4780b0b50 msgr2=0x7fe47815cbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:09.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.699+0000 7fe480703640 1 -- 172.21.15.45:0/67955439 shutdown_connections 2024-09-12T03:16:09.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.699+0000 7fe480703640 1 -- 172.21.15.45:0/67955439 wait complete. 2024-09-12T03:16:09.709 DEBUG:teuthology.orchestra.run.smithi045:> sudo ceph --cluster ceph osd new 4dad773d-b7d8-46fc-8a10-a972aad27b29 7 2024-09-12T03:16:09.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.810+0000 7f3eac152640 1 Processor -- start 2024-09-12T03:16:09.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.811+0000 7f3eac152640 1 -- start start 2024-09-12T03:16:09.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.811+0000 7f3eac152640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f3ea40d0fa0 0x7f3ea40717a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:09.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.811+0000 7f3eac152640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3ea406d2b0 con 0x7f3ea40d1380 2024-09-12T03:16:09.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.812+0000 7f3eac152640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3ea406d530 con 0x7f3ea4071270 2024-09-12T03:16:09.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.812+0000 7f3eac152640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f3ea4071ce0 con 0x7f3ea40d0fa0 2024-09-12T03:16:09.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.812+0000 7f3ea96ed640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f3ea4071270 0x7f3ea415e400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:45072/0 (socket says 172.21.15.45:45072) 2024-09-12T03:16:09.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.812+0000 7f3ea96ed640 1 -- 172.21.15.45:0/3366632842 learned_addr learned my addr 172.21.15.45:0/3366632842 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-09-12T03:16:09.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.812+0000 7f3ea9eee640 1 --2- 172.21.15.45:0/3366632842 >> v2:172.21.15.27:3300/0 conn(0x7f3ea40d0fa0 0x7f3ea40717a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:09.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.812+0000 7f3ea8eec640 1 -- 172.21.15.45:0/3366632842 <== mon.1 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4231326225 0 0) 0x7f3ea406d530 con 0x7f3ea4071270 2024-09-12T03:16:09.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.812+0000 7f3ea8eec640 1 -- 172.21.15.45:0/3366632842 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3e90003520 con 0x7f3ea4071270 2024-09-12T03:16:09.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.812+0000 7f3ea8eec640 1 -- 172.21.15.45:0/3366632842 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 677888173 0 0) 0x7f3e9c001490 con 0x7f3ea40d1380 2024-09-12T03:16:09.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.812+0000 7f3ea8eec640 1 -- 172.21.15.45:0/3366632842 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3ea406d530 con 0x7f3ea40d1380 2024-09-12T03:16:09.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.812+0000 7f3ea9eee640 1 -- 172.21.15.45:0/3366632842 >> v1:172.21.15.45:6789/0 conn(0x7f3ea4071270 legacy=0x7f3ea415e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:09.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.812+0000 7f3ea9eee640 1 -- 172.21.15.45:0/3366632842 >> v1:172.21.15.27:6789/0 conn(0x7f3ea40d1380 legacy=0x7f3ea406dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:09.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.812+0000 7f3ea9eee640 1 -- 172.21.15.45:0/3366632842 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ea4070b70 con 0x7f3ea40d0fa0 2024-09-12T03:16:09.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.813+0000 7f3ea9eee640 1 --2- 172.21.15.45:0/3366632842 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3ea40d0fa0 0x7f3ea40717a0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f3ea00097a0 tx=0x7f3ea002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a43ca6aa82e550f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:09.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.813+0000 7f3ea8eec640 1 -- 172.21.15.45:0/3366632842 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ea0037070 con 0x7f3ea40d0fa0 2024-09-12T03:16:09.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.813+0000 7f3ea8eec640 1 -- 172.21.15.45:0/3366632842 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ea0032440 con 0x7f3ea40d0fa0 2024-09-12T03:16:09.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.813+0000 7f3ea8eec640 1 -- 172.21.15.45:0/3366632842 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ea0032810 con 0x7f3ea40d0fa0 2024-09-12T03:16:09.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.814+0000 7f3eac152640 1 -- 172.21.15.45:0/3366632842 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3ea40d0fa0 msgr2=0x7f3ea40717a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:09.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.814+0000 7f3eac152640 1 --2- 172.21.15.45:0/3366632842 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3ea40d0fa0 0x7f3ea40717a0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f3ea00097a0 tx=0x7f3ea002a230 comp rx=0 tx=0).stop 2024-09-12T03:16:09.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.814+0000 7f3eac152640 1 -- 172.21.15.45:0/3366632842 shutdown_connections 2024-09-12T03:16:09.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.814+0000 7f3eac152640 1 --2- 172.21.15.45:0/3366632842 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3ea40d0fa0 0x7f3ea40717a0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:09.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.814+0000 7f3eac152640 1 -- 172.21.15.45:0/3366632842 >> 172.21.15.45:0/3366632842 conn(0x7f3ea40b4320 msgr2=0x7f3ea40b4700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:09.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.814+0000 7f3eac152640 1 -- 172.21.15.45:0/3366632842 shutdown_connections 2024-09-12T03:16:09.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.814+0000 7f3eac152640 1 -- 172.21.15.45:0/3366632842 wait complete. 2024-09-12T03:16:09.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.815+0000 7f3eac152640 1 Processor -- start 2024-09-12T03:16:09.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.815+0000 7f3eac152640 1 -- start start 2024-09-12T03:16:09.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.816+0000 7f3eac152640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3ea4071270 0x7f3ea4143ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:09.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.816+0000 7f3eac152640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f3ea40d0fa0 0x7f3ea41442e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:09.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.816+0000 7f3eac152640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3ea40d1380 0x7f3ea41564b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:09.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.816+0000 7f3eac152640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f3ea40b5620 con 0x7f3ea40d1380 2024-09-12T03:16:09.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.816+0000 7f3eac152640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f3ea406d7f0 con 0x7f3ea4071270 2024-09-12T03:16:09.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.816+0000 7f3eac152640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f3ea40b57a0 con 0x7f3ea40d0fa0 2024-09-12T03:16:09.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.816+0000 7f3ea9eee640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3ea4071270 0x7f3ea4143ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:09.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.816+0000 7f3ea96ed640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f3ea40d0fa0 0x7f3ea41442e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:09.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.816+0000 7f3ea9eee640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3ea4071270 0x7f3ea4143ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50962/0 (socket says 172.21.15.45:50962) 2024-09-12T03:16:09.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.816+0000 7f3ea9eee640 1 -- 172.21.15.45:0/482167045 learned_addr learned my addr 172.21.15.45:0/482167045 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-12T03:16:09.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.816+0000 7f3eaa6ef640 1 --2- 172.21.15.45:0/482167045 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3ea40d1380 0x7f3ea41564b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:09.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.816+0000 7f3ea9eee640 1 -- 172.21.15.45:0/482167045 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f3ea40d0fa0 msgr2=0x7f3ea41442e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:09.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.816+0000 7f3ea9eee640 1 --2- 172.21.15.45:0/482167045 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f3ea40d0fa0 0x7f3ea41442e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:09.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.816+0000 7f3ea9eee640 1 -- 172.21.15.45:0/482167045 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3ea40d1380 msgr2=0x7f3ea41564b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:09.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.816+0000 7f3ea9eee640 1 --2- 172.21.15.45:0/482167045 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3ea40d1380 0x7f3ea41564b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:09.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.816+0000 7f3ea9eee640 1 -- 172.21.15.45:0/482167045 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ea415b900 con 0x7f3ea4071270 2024-09-12T03:16:09.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.816+0000 7f3ea9eee640 1 --2- 172.21.15.45:0/482167045 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3ea4071270 0x7f3ea4143ba0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f3ea00098c0 tx=0x7f3ea00046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:09.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.817+0000 7f3e967fc640 1 -- 172.21.15.45:0/482167045 <== mon.1 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ea0041030 con 0x7f3ea4071270 2024-09-12T03:16:09.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.817+0000 7f3e967fc640 1 -- 172.21.15.45:0/482167045 <== mon.1 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ea003d4e0 con 0x7f3ea4071270 2024-09-12T03:16:09.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.817+0000 7f3e967fc640 1 -- 172.21.15.45:0/482167045 <== mon.1 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ea0037040 con 0x7f3ea4071270 2024-09-12T03:16:09.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.817+0000 7f3eac152640 1 -- 172.21.15.45:0/482167045 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ea415bbe0 con 0x7f3ea4071270 2024-09-12T03:16:09.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.817+0000 7f3eac152640 1 -- 172.21.15.45:0/482167045 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ea4153f30 con 0x7f3ea4071270 2024-09-12T03:16:09.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.817+0000 7f3e967fc640 1 -- 172.21.15.45:0/482167045 <== mon.1 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f3ea0044070 con 0x7f3ea4071270 2024-09-12T03:16:09.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.817+0000 7f3e967fc640 1 -- 172.21.15.45:0/482167045 <== mon.1 v2:172.21.15.45:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1530+0+0 (secure 0 0 0) 0x7f3ea0046dd0 con 0x7f3ea4071270 2024-09-12T03:16:09.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.818+0000 7f3eac152640 1 -- 172.21.15.45:0/482167045 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ea40717a0 con 0x7f3ea4071270 2024-09-12T03:16:09.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.820+0000 7f3e967fc640 1 -- 172.21.15.45:0/482167045 <== mon.1 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f3ea0046730 con 0x7f3ea4071270 2024-09-12T03:16:09.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.891+0000 7f3eac152640 1 -- 172.21.15.45:0/482167045 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd new", "uuid": "4dad773d-b7d8-46fc-8a10-a972aad27b29", "id": 7} v 0) -- 0x7f3ea406e900 con 0x7f3ea4071270 2024-09-12T03:16:09.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.896+0000 7f3e967fc640 1 -- 172.21.15.45:0/482167045 <== mon.1 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "4dad773d-b7d8-46fc-8a10-a972aad27b29", "id": 7}]=0 v10) ==== 112+0+2 (secure 0 0 0) 0x7f3ea0046910 con 0x7f3ea4071270 2024-09-12T03:16:09.897 INFO:teuthology.orchestra.run.smithi045.stdout:7 2024-09-12T03:16:09.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.897+0000 7f3eac152640 1 -- 172.21.15.45:0/482167045 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3ea4071270 msgr2=0x7f3ea4143ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:09.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.897+0000 7f3eac152640 1 --2- 172.21.15.45:0/482167045 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3ea4071270 0x7f3ea4143ba0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f3ea00098c0 tx=0x7f3ea00046c0 comp rx=0 tx=0).stop 2024-09-12T03:16:09.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.897+0000 7f3eac152640 1 -- 172.21.15.45:0/482167045 shutdown_connections 2024-09-12T03:16:09.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.897+0000 7f3eac152640 1 --2- 172.21.15.45:0/482167045 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3ea40d1380 0x7f3ea41564b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:09.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.897+0000 7f3eac152640 1 --2- 172.21.15.45:0/482167045 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f3ea40d0fa0 0x7f3ea41442e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:09.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.897+0000 7f3eac152640 1 --2- 172.21.15.45:0/482167045 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3ea4071270 0x7f3ea4143ba0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:09.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.897+0000 7f3eac152640 1 -- 172.21.15.45:0/482167045 >> 172.21.15.45:0/482167045 conn(0x7f3ea40b4320 msgr2=0x7f3ea40b3290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:09.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.898+0000 7f3eac152640 1 -- 172.21.15.45:0/482167045 shutdown_connections 2024-09-12T03:16:09.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:09.898+0000 7f3eac152640 1 -- 172.21.15.45:0/482167045 wait complete. 2024-09-12T03:16:09.907 INFO:tasks.ceph.osd.0:Restarting daemon 2024-09-12T03:16:09.908 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-09-12T03:16:09.910 INFO:tasks.ceph.osd.0:Started 2024-09-12T03:16:09.910 INFO:tasks.ceph.osd.1:Restarting daemon 2024-09-12T03:16:09.910 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-09-12T03:16:09.912 INFO:tasks.ceph.osd.1:Started 2024-09-12T03:16:09.912 INFO:tasks.ceph.osd.2:Restarting daemon 2024-09-12T03:16:09.912 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-09-12T03:16:09.914 INFO:tasks.ceph.osd.2:Started 2024-09-12T03:16:09.914 INFO:tasks.ceph.osd.3:Restarting daemon 2024-09-12T03:16:09.915 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-09-12T03:16:09.917 INFO:tasks.ceph.osd.3:Started 2024-09-12T03:16:09.917 INFO:tasks.ceph.osd.4:Restarting daemon 2024-09-12T03:16:09.917 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-09-12T03:16:09.919 INFO:tasks.ceph.osd.4:Started 2024-09-12T03:16:09.919 INFO:tasks.ceph.osd.5:Restarting daemon 2024-09-12T03:16:09.919 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-09-12T03:16:09.921 INFO:tasks.ceph.osd.5:Started 2024-09-12T03:16:09.922 INFO:tasks.ceph.osd.6:Restarting daemon 2024-09-12T03:16:09.922 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-09-12T03:16:09.924 INFO:tasks.ceph.osd.6:Started 2024-09-12T03:16:09.924 INFO:tasks.ceph.osd.7:Restarting daemon 2024-09-12T03:16:09.924 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-09-12T03:16:09.926 INFO:tasks.ceph.osd.7:Started 2024-09-12T03:16:09.927 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-12T03:16:10.035 INFO:tasks.ceph.mgr.y.smithi027.stderr:2024-09-12T03:16:10.034+0000 7f32df11f040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-12T03:16:10.046 INFO:tasks.ceph.mgr.x.smithi045.stderr:2024-09-12T03:16:10.046+0000 7fc1f240d040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-12T03:16:10.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:10.052+0000 7f5c70877640 1 Processor -- start 2024-09-12T03:16:10.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:10.053+0000 7f5c70877640 1 -- start start 2024-09-12T03:16:10.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:10.054+0000 7f5c70877640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f5c6c144de0 0x7f5c6c0d0e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:10.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:10.054+0000 7f5c70877640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f5c6c15c450 0x7f5c6c0b50b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:10.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:10.054+0000 7f5c70877640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5c6c0b4610 con 0x7f5c6c1451c0 2024-09-12T03:16:10.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:10.054+0000 7f5c70877640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f5c6c15cbc0 con 0x7f5c6c15c450 2024-09-12T03:16:10.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:10.054+0000 7f5c70877640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f5c6c0b4c20 con 0x7f5c6c144de0 2024-09-12T03:16:10.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:10.054+0000 7f5c6ad9d640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f5c6c15c450 0x7f5c6c0b50b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:10.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:10.054+0000 7f5c63fff640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f5c6c1451c0 0x7f5c6c15bd40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:44258/0 (socket says 172.21.15.27:44258) 2024-09-12T03:16:10.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:10.054+0000 7f5c63fff640 1 -- 172.21.15.27:0/3354568998 learned_addr learned my addr 172.21.15.27:0/3354568998 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:16:10.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:10.054+0000 7f5c6a59c640 1 --2- 172.21.15.27:0/3354568998 >> v2:172.21.15.45:3300/0 conn(0x7f5c6c144de0 0x7f5c6c0d0e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:10.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:10.055+0000 7f5c6ad9d640 1 -- 172.21.15.27:0/3354568998 >> v2:172.21.15.45:3300/0 conn(0x7f5c6c144de0 msgr2=0x7f5c6c0d0e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:10.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:10.055+0000 7f5c6ad9d640 1 --2- 172.21.15.27:0/3354568998 >> v2:172.21.15.45:3300/0 conn(0x7f5c6c144de0 0x7f5c6c0d0e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:10.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:10.055+0000 7f5c6ad9d640 1 -- 172.21.15.27:0/3354568998 >> v1:172.21.15.27:6789/0 conn(0x7f5c6c1451c0 legacy=0x7f5c6c15bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:10.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:10.055+0000 7f5c6ad9d640 1 -- 172.21.15.27:0/3354568998 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c6c0d0710 con 0x7f5c6c15c450 2024-09-12T03:16:10.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:10.055+0000 7f5c6ad9d640 1 --2- 172.21.15.27:0/3354568998 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c6c15c450 0x7f5c6c0b50b0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f5c6c06d7e0 tx=0x7f5c6402c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=158690b914f4d835 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:10.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:10.055+0000 7f5c69d9b640 1 -- 172.21.15.27:0/3354568998 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c64039070 con 0x7f5c6c15c450 2024-09-12T03:16:10.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:10.055+0000 7f5c69d9b640 1 -- 172.21.15.27:0/3354568998 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c64009540 con 0x7f5c6c15c450 2024-09-12T03:16:10.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:10.055+0000 7f5c69d9b640 1 -- 172.21.15.27:0/3354568998 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c64009910 con 0x7f5c6c15c450 2024-09-12T03:16:10.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:10.056+0000 7f5c70877640 1 -- 172.21.15.27:0/3354568998 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c6c15c450 msgr2=0x7f5c6c0b50b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:10.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:10.056+0000 7f5c70877640 1 --2- 172.21.15.27:0/3354568998 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c6c15c450 0x7f5c6c0b50b0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f5c6c06d7e0 tx=0x7f5c6402c890 comp rx=0 tx=0).stop 2024-09-12T03:16:10.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:10.056+0000 7f5c70877640 1 -- 172.21.15.27:0/3354568998 shutdown_connections 2024-09-12T03:16:10.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:10.056+0000 7f5c70877640 1 --2- 172.21.15.27:0/3354568998 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c6c15c450 0x7f5c6c0b50b0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:10.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:10.056+0000 7f5c70877640 1 --2- 172.21.15.27:0/3354568998 >> v2:172.21.15.45:3300/0 conn(0x7f5c6c144de0 0x7f5c6c0d0e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:10.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:10.056+0000 7f5c70877640 1 -- 172.21.15.27:0/3354568998 >> 172.21.15.27:0/3354568998 conn(0x7f5c6c06d270 msgr2=0x7f5c6c0b3270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:10.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:10.056+0000 7f5c70877640 1 -- 172.21.15.27:0/3354568998 shutdown_connections 2024-09-12T03:16:10.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:10.056+0000 7f5c70877640 1 -- 172.21.15.27:0/3354568998 wait complete. 2024-09-12T03:16:10.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:10.058+0000 7f5c70877640 1 Processor -- start 2024-09-12T03:16:10.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:10.058+0000 7f5c70877640 1 -- start start 2024-09-12T03:16:10.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:10.059+0000 7f5c70877640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c6c144de0 0x7f5c6c0aa4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:10.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:10.059+0000 7f5c70877640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5c6c1451c0 0x7f5c6c15af70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:10.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:10.059+0000 7f5c70877640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f5c6c15c450 0x7f5c6c15f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:10.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:10.059+0000 7f5c70877640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f5c6c15cdc0 con 0x7f5c6c144de0 2024-09-12T03:16:10.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:10.059+0000 7f5c70877640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f5c6c0d09f0 con 0x7f5c6c1451c0 2024-09-12T03:16:10.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:10.059+0000 7f5c70877640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f5c6c15cf40 con 0x7f5c6c15c450 2024-09-12T03:16:10.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:10.059+0000 7f5c6a59c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c6c144de0 0x7f5c6c0aa4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:10.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:10.059+0000 7f5c6a59c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c6c144de0 0x7f5c6c0aa4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55444/0 (socket says 172.21.15.27:55444) 2024-09-12T03:16:10.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:10.059+0000 7f5c6a59c640 1 -- 172.21.15.27:0/579868944 learned_addr learned my addr 172.21.15.27:0/579868944 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:16:10.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:10.059+0000 7f5c63fff640 1 --2- 172.21.15.27:0/579868944 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5c6c1451c0 0x7f5c6c15af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:10.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:10.059+0000 7f5c6ad9d640 1 --2- 172.21.15.27:0/579868944 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f5c6c15c450 0x7f5c6c15f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:10.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:10.059+0000 7f5c63fff640 1 -- 172.21.15.27:0/579868944 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f5c6c15c450 msgr2=0x7f5c6c15f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:10.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:10.059+0000 7f5c63fff640 1 --2- 172.21.15.27:0/579868944 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f5c6c15c450 0x7f5c6c15f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:10.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:10.059+0000 7f5c63fff640 1 -- 172.21.15.27:0/579868944 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c6c144de0 msgr2=0x7f5c6c0aa4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:10.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:10.059+0000 7f5c63fff640 1 --2- 172.21.15.27:0/579868944 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c6c144de0 0x7f5c6c0aa4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:10.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:10.059+0000 7f5c63fff640 1 -- 172.21.15.27:0/579868944 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c6c0aaaa0 con 0x7f5c6c1451c0 2024-09-12T03:16:10.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:10.059+0000 7f5c6a59c640 1 --2- 172.21.15.27:0/579868944 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c6c144de0 0x7f5c6c0aa4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:16:10.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:10.059+0000 7f5c63fff640 1 --2- 172.21.15.27:0/579868944 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5c6c1451c0 0x7f5c6c15af70 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f5c54000f60 tx=0x7f5c5400c800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:10.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:10.059+0000 7f5c62ffd640 1 -- 172.21.15.27:0/579868944 <== mon.1 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c54017070 con 0x7f5c6c1451c0 2024-09-12T03:16:10.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:10.059+0000 7f5c62ffd640 1 -- 172.21.15.27:0/579868944 <== mon.1 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c54005ce0 con 0x7f5c6c1451c0 2024-09-12T03:16:10.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:10.060+0000 7f5c70877640 1 -- 172.21.15.27:0/579868944 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c6c0aad80 con 0x7f5c6c1451c0 2024-09-12T03:16:10.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:10.060+0000 7f5c62ffd640 1 -- 172.21.15.27:0/579868944 <== mon.1 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c54007310 con 0x7f5c6c1451c0 2024-09-12T03:16:10.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:10.060+0000 7f5c70877640 1 -- 172.21.15.27:0/579868944 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c6c0ab340 con 0x7f5c6c1451c0 2024-09-12T03:16:10.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:10.060+0000 7f5c62ffd640 1 -- 172.21.15.27:0/579868944 <== mon.1 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f5c5401a070 con 0x7f5c6c1451c0 2024-09-12T03:16:10.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:10.060+0000 7f5c62ffd640 1 -- 172.21.15.27:0/579868944 <== mon.1 v2:172.21.15.45:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f5c540074b0 con 0x7f5c6c1451c0 2024-09-12T03:16:10.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:10.061+0000 7f5c70877640 1 -- 172.21.15.27:0/579868944 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c6c0d0170 con 0x7f5c6c1451c0 2024-09-12T03:16:10.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:10.065+0000 7f5c62ffd640 1 -- 172.21.15.27:0/579868944 <== mon.1 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f5c54021020 con 0x7f5c6c1451c0 2024-09-12T03:16:10.094 INFO:tasks.ceph.mgr.y.smithi027.stderr:2024-09-12T03:16:10.093+0000 7f32df11f040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-12T03:16:10.103 INFO:tasks.ceph.mgr.x.smithi045.stderr:2024-09-12T03:16:10.103+0000 7fc1f240d040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-12T03:16:10.125 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:10.125+0000 7f5c70877640 1 -- 172.21.15.27:0/579868944 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5c6c141990 con 0x7f5c6c1451c0 2024-09-12T03:16:10.126 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:10.126+0000 7f5c62ffd640 1 -- 172.21.15.27:0/579868944 <== mon.1 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f5c5401c440 con 0x7f5c6c1451c0 2024-09-12T03:16:10.127 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:16:10.127 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":10,"fsid":"d2df504e-2647-4385-a5db-9fdf69f487a0","created":"2024-09-12T03:16:07.778937+0000","modified":"2024-09-12T03:16:09.893086+0000","last_up_change":"0.000000","last_in_change":"2024-09-12T03:16:09.893086+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"1851116c-1fb3-4dc9-9772-f90e2ef4c355","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"f38ae17d-73eb-4204-b99d-975fa806fb87","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"fb12dcc8-bdd0-42b8-b46d-8b10106224bb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"d74d3f49-9c24-4b9f-98e2-3ecd7b5ef3b3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"2bf123f2-1b0d-41c4-aa3b-805bab267e59","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"c49ba5da-91c6-4a86-a97b-4aba89c253ec","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"b0123475-89f7-4a34-9e33-d797b99dd9dc","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"4dad773d-b7d8-46fc-8a10-a972aad27b29","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T03:16:10.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:10.126+0000 7f5c70877640 1 -- 172.21.15.27:0/579868944 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5c6c1451c0 msgr2=0x7f5c6c15af70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:10.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:10.126+0000 7f5c70877640 1 --2- 172.21.15.27:0/579868944 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5c6c1451c0 0x7f5c6c15af70 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f5c54000f60 tx=0x7f5c5400c800 comp rx=0 tx=0).stop 2024-09-12T03:16:10.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:10.126+0000 7f5c70877640 1 -- 172.21.15.27:0/579868944 shutdown_connections 2024-09-12T03:16:10.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:10.126+0000 7f5c70877640 1 --2- 172.21.15.27:0/579868944 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f5c6c15c450 0x7f5c6c15f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:10.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:10.126+0000 7f5c70877640 1 --2- 172.21.15.27:0/579868944 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5c6c1451c0 0x7f5c6c15af70 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:10.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:10.126+0000 7f5c70877640 1 --2- 172.21.15.27:0/579868944 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c6c144de0 0x7f5c6c0aa4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:10.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:10.127+0000 7f5c70877640 1 -- 172.21.15.27:0/579868944 >> 172.21.15.27:0/579868944 conn(0x7f5c6c06d270 msgr2=0x7f5c6c0b2550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:10.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:10.127+0000 7f5c70877640 1 -- 172.21.15.27:0/579868944 shutdown_connections 2024-09-12T03:16:10.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:10.127+0000 7f5c70877640 1 -- 172.21.15.27:0/579868944 wait complete. 2024-09-12T03:16:10.138 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-09-12T03:16:10.138 INFO:tasks.ceph:Waiting for OSDs to come up 2024-09-12T03:16:10.485 INFO:tasks.ceph.mgr.y.smithi027.stderr:2024-09-12T03:16:10.484+0000 7f32df11f040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-12T03:16:10.490 INFO:tasks.ceph.mgr.x.smithi045.stderr:2024-09-12T03:16:10.491+0000 7fc1f240d040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-12T03:16:10.575 INFO:tasks.ceph.mgr.y.smithi027.stderr:2024-09-12T03:16:10.574+0000 7f32df11f040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-12T03:16:10.579 INFO:tasks.ceph.mgr.x.smithi045.stderr:2024-09-12T03:16:10.580+0000 7fc1f240d040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-12T03:16:10.688 INFO:tasks.ceph.mgr.y.smithi027.stderr:2024-09-12T03:16:10.687+0000 7f32df11f040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-12T03:16:10.691 INFO:tasks.ceph.mgr.x.smithi045.stderr:2024-09-12T03:16:10.691+0000 7fc1f240d040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-12T03:16:10.737 INFO:tasks.ceph.mgr.y.smithi027.stderr:2024-09-12T03:16:10.736+0000 7f32df11f040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-12T03:16:10.739 INFO:tasks.ceph.mgr.x.smithi045.stderr:2024-09-12T03:16:10.740+0000 7fc1f240d040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-12T03:16:10.857 INFO:tasks.ceph.mgr.y.smithi027.stderr:2024-09-12T03:16:10.856+0000 7f32df11f040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-12T03:16:10.859 INFO:tasks.ceph.mgr.x.smithi045.stderr:2024-09-12T03:16:10.860+0000 7fc1f240d040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-12T03:16:10.913 INFO:tasks.ceph.mgr.y.smithi027.stderr:2024-09-12T03:16:10.912+0000 7f32df11f040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-12T03:16:10.915 INFO:tasks.ceph.mgr.x.smithi045.stderr:2024-09-12T03:16:10.915+0000 7fc1f240d040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-12T03:16:10.944 DEBUG:teuthology.orchestra.run.smithi027:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-12T03:16:10.961 INFO:tasks.ceph.mgr.y.smithi027.stderr:2024-09-12T03:16:10.960+0000 7f32df11f040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-12T03:16:10.961 INFO:tasks.ceph.mgr.x.smithi045.stderr:2024-09-12T03:16:10.962+0000 7fc1f240d040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-12T03:16:11.038 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:11.037+0000 7f2eaf60a640 1 Processor -- start 2024-09-12T03:16:11.038 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:11.037+0000 7f2eaf60a640 1 -- start start 2024-09-12T03:16:11.039 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:11.038+0000 7f2eaf60a640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f2ea80ceeb0 0x7f2ea80d1630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:11.039 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:11.038+0000 7f2eaf60a640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f2ea80d1b70 0x7f2ea80d2f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:11.039 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:11.038+0000 7f2eaf60a640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2ea80cf3b0 con 0x7f2ea815e830 2024-09-12T03:16:11.039 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:11.038+0000 7f2eaf60a640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f2ea80cffc0 con 0x7f2ea80ceeb0 2024-09-12T03:16:11.040 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:11.038+0000 7f2eaf60a640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f2ea80d1fa0 con 0x7f2ea80d1b70 2024-09-12T03:16:11.040 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:11.038+0000 7f2eacba5640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f2ea80ceeb0 0x7f2ea80d1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:11.040 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:11.038+0000 7f2eacba5640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f2ea80ceeb0 0x7f2ea80d1630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55452/0 (socket says 172.21.15.27:55452) 2024-09-12T03:16:11.040 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:11.038+0000 7f2eacba5640 1 -- 172.21.15.27:0/916421424 learned_addr learned my addr 172.21.15.27:0/916421424 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:16:11.040 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:11.038+0000 7f2eadba7640 1 --2- 172.21.15.27:0/916421424 >> v2:172.21.15.45:3300/0 conn(0x7f2ea80d1b70 0x7f2ea80d2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:11.040 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:11.038+0000 7f2eacba5640 1 -- 172.21.15.27:0/916421424 >> v2:172.21.15.45:3300/0 conn(0x7f2ea80d1b70 msgr2=0x7f2ea80d2f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:11.040 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:11.038+0000 7f2eacba5640 1 --2- 172.21.15.27:0/916421424 >> v2:172.21.15.45:3300/0 conn(0x7f2ea80d1b70 0x7f2ea80d2f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:11.040 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:11.038+0000 7f2eacba5640 1 -- 172.21.15.27:0/916421424 >> v1:172.21.15.45:6789/0 conn(0x7f2ea815e830 legacy=0x7f2ea80b3990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:11.040 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:11.038+0000 7f2eacba5640 1 -- 172.21.15.27:0/916421424 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ea80cf690 con 0x7f2ea80ceeb0 2024-09-12T03:16:11.042 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:11.039+0000 7f2eacba5640 1 --2- 172.21.15.27:0/916421424 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2ea80ceeb0 0x7f2ea80d1630 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f2ea80cee40 tx=0x7f2e9802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9adb5cd083e9f028 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:11.042 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:11.039+0000 7f2e9ffff640 1 -- 172.21.15.27:0/916421424 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e98037070 con 0x7f2ea80ceeb0 2024-09-12T03:16:11.042 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:11.039+0000 7f2e9ffff640 1 -- 172.21.15.27:0/916421424 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e98032440 con 0x7f2ea80ceeb0 2024-09-12T03:16:11.042 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:11.039+0000 7f2e9ffff640 1 -- 172.21.15.27:0/916421424 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e98032810 con 0x7f2ea80ceeb0 2024-09-12T03:16:11.042 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:11.040+0000 7f2eaf60a640 1 -- 172.21.15.27:0/916421424 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2ea80ceeb0 msgr2=0x7f2ea80d1630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:11.042 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:11.040+0000 7f2eaf60a640 1 --2- 172.21.15.27:0/916421424 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2ea80ceeb0 0x7f2ea80d1630 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f2ea80cee40 tx=0x7f2e9802a270 comp rx=0 tx=0).stop 2024-09-12T03:16:11.043 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:11.040+0000 7f2eaf60a640 1 -- 172.21.15.27:0/916421424 shutdown_connections 2024-09-12T03:16:11.043 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:11.040+0000 7f2eaf60a640 1 --2- 172.21.15.27:0/916421424 >> v2:172.21.15.45:3300/0 conn(0x7f2ea80d1b70 0x7f2ea80d2f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:11.043 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:11.040+0000 7f2eaf60a640 1 --2- 172.21.15.27:0/916421424 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2ea80ceeb0 0x7f2ea80d1630 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:11.043 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:11.040+0000 7f2eaf60a640 1 -- 172.21.15.27:0/916421424 >> 172.21.15.27:0/916421424 conn(0x7f2ea8067ab0 msgr2=0x7f2ea8067ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:11.043 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:11.040+0000 7f2eaf60a640 1 -- 172.21.15.27:0/916421424 shutdown_connections 2024-09-12T03:16:11.043 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:11.040+0000 7f2eaf60a640 1 -- 172.21.15.27:0/916421424 wait complete. 2024-09-12T03:16:11.043 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:11.041+0000 7f2eaf60a640 1 Processor -- start 2024-09-12T03:16:11.043 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:11.041+0000 7f2eaf60a640 1 -- start start 2024-09-12T03:16:11.045 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:11.042+0000 7f2eaf60a640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2ea80ceeb0 0x7f2ea80cc180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:11.045 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:11.042+0000 7f2eaf60a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2ea80d1b70 0x7f2ea80cdc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:11.045 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:11.042+0000 7f2eaf60a640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2ea815e830 0x7f2ea80ce1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:11.046 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:11.042+0000 7f2eaf60a640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f2ea815ec10 con 0x7f2ea80ceeb0 2024-09-12T03:16:11.046 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:11.042+0000 7f2eaf60a640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f2ea80cfc50 con 0x7f2ea80d1b70 2024-09-12T03:16:11.046 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:11.042+0000 7f2eaf60a640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f2ea815ed90 con 0x7f2ea815e830 2024-09-12T03:16:11.046 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:11.042+0000 7f2ead3a6640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2ea80ceeb0 0x7f2ea80cc180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:11.046 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:11.042+0000 7f2ead3a6640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2ea80ceeb0 0x7f2ea80cc180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55464/0 (socket says 172.21.15.27:55464) 2024-09-12T03:16:11.046 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:11.042+0000 7f2ead3a6640 1 -- 172.21.15.27:0/3799922162 learned_addr learned my addr 172.21.15.27:0/3799922162 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:16:11.046 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:11.042+0000 7f2eacba5640 1 --2- 172.21.15.27:0/3799922162 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2ea80d1b70 0x7f2ea80cdc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:11.046 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:11.042+0000 7f2eadba7640 1 --2- 172.21.15.27:0/3799922162 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2ea815e830 0x7f2ea80ce1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:11.046 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:11.042+0000 7f2eacba5640 1 -- 172.21.15.27:0/3799922162 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2ea815e830 msgr2=0x7f2ea80ce1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:11.046 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:11.042+0000 7f2eacba5640 1 --2- 172.21.15.27:0/3799922162 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2ea815e830 0x7f2ea80ce1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:11.046 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:11.042+0000 7f2eacba5640 1 -- 172.21.15.27:0/3799922162 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2ea80ceeb0 msgr2=0x7f2ea80cc180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:11.046 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:11.042+0000 7f2eacba5640 1 --2- 172.21.15.27:0/3799922162 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2ea80ceeb0 0x7f2ea80cc180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:11.046 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:11.042+0000 7f2eacba5640 1 -- 172.21.15.27:0/3799922162 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ea80ccc80 con 0x7f2ea80d1b70 2024-09-12T03:16:11.047 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:11.042+0000 7f2ead3a6640 1 --2- 172.21.15.27:0/3799922162 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2ea80ceeb0 0x7f2ea80cc180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:16:11.047 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:11.042+0000 7f2eacba5640 1 --2- 172.21.15.27:0/3799922162 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2ea80d1b70 0x7f2ea80cdc60 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f2e98031000 tx=0x7f2e980046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:11.047 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:11.043+0000 7f2e9dffb640 1 -- 172.21.15.27:0/3799922162 <== mon.1 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e98041030 con 0x7f2ea80d1b70 2024-09-12T03:16:11.047 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:11.043+0000 7f2e9dffb640 1 -- 172.21.15.27:0/3799922162 <== mon.1 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e98030440 con 0x7f2ea80d1b70 2024-09-12T03:16:11.047 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:11.043+0000 7f2e9dffb640 1 -- 172.21.15.27:0/3799922162 <== mon.1 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e98037040 con 0x7f2ea80d1b70 2024-09-12T03:16:11.047 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:11.043+0000 7f2eaf60a640 1 -- 172.21.15.27:0/3799922162 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ea80ccf40 con 0x7f2ea80d1b70 2024-09-12T03:16:11.047 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:11.043+0000 7f2eaf60a640 1 -- 172.21.15.27:0/3799922162 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ea8150820 con 0x7f2ea80d1b70 2024-09-12T03:16:11.047 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:11.043+0000 7f2e9dffb640 1 -- 172.21.15.27:0/3799922162 <== mon.1 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f2e98044070 con 0x7f2ea80d1b70 2024-09-12T03:16:11.047 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:11.043+0000 7f2e9dffb640 1 -- 172.21.15.27:0/3799922162 <== mon.1 v2:172.21.15.45:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f2e9803dcd0 con 0x7f2ea80d1b70 2024-09-12T03:16:11.047 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:11.044+0000 7f2eaf60a640 1 -- 172.21.15.27:0/3799922162 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ea80d25e0 con 0x7f2ea80d1b70 2024-09-12T03:16:11.049 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:11.048+0000 7f2e9dffb640 1 -- 172.21.15.27:0/3799922162 <== mon.1 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f2e9803d600 con 0x7f2ea80d1b70 2024-09-12T03:16:11.080 INFO:tasks.ceph.mgr.x.smithi045.stderr:2024-09-12T03:16:11.080+0000 7fc1f240d040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-12T03:16:11.082 INFO:tasks.ceph.mgr.y.smithi027.stderr:2024-09-12T03:16:11.081+0000 7f32df11f040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-12T03:16:11.108 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:11.107+0000 7f2eaf60a640 1 -- 172.21.15.27:0/3799922162 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2ea81619e0 con 0x7f2ea80d1b70 2024-09-12T03:16:11.109 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:11.108+0000 7f2e9dffb640 1 -- 172.21.15.27:0/3799922162 <== mon.1 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f2e9803d7e0 con 0x7f2ea80d1b70 2024-09-12T03:16:11.109 INFO:teuthology.misc.health.smithi027.stdout: 2024-09-12T03:16:11.109 INFO:teuthology.misc.health.smithi027.stdout:{"epoch":10,"fsid":"d2df504e-2647-4385-a5db-9fdf69f487a0","created":"2024-09-12T03:16:07.778937+0000","modified":"2024-09-12T03:16:09.893086+0000","last_up_change":"0.000000","last_in_change":"2024-09-12T03:16:09.893086+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"1851116c-1fb3-4dc9-9772-f90e2ef4c355","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"f38ae17d-73eb-4204-b99d-975fa806fb87","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"fb12dcc8-bdd0-42b8-b46d-8b10106224bb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"d74d3f49-9c24-4b9f-98e2-3ecd7b5ef3b3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"2bf123f2-1b0d-41c4-aa3b-805bab267e59","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"c49ba5da-91c6-4a86-a97b-4aba89c253ec","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"b0123475-89f7-4a34-9e33-d797b99dd9dc","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"4dad773d-b7d8-46fc-8a10-a972aad27b29","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T03:16:11.110 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:11.109+0000 7f2eaf60a640 1 -- 172.21.15.27:0/3799922162 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2ea80d1b70 msgr2=0x7f2ea80cdc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:11.110 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:11.109+0000 7f2eaf60a640 1 --2- 172.21.15.27:0/3799922162 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2ea80d1b70 0x7f2ea80cdc60 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f2e98031000 tx=0x7f2e980046c0 comp rx=0 tx=0).stop 2024-09-12T03:16:11.110 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:11.109+0000 7f2eaf60a640 1 -- 172.21.15.27:0/3799922162 shutdown_connections 2024-09-12T03:16:11.110 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:11.109+0000 7f2eaf60a640 1 --2- 172.21.15.27:0/3799922162 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2ea815e830 0x7f2ea80ce1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:11.111 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:11.109+0000 7f2eaf60a640 1 --2- 172.21.15.27:0/3799922162 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2ea80d1b70 0x7f2ea80cdc60 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:11.111 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:11.109+0000 7f2eaf60a640 1 --2- 172.21.15.27:0/3799922162 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2ea80ceeb0 0x7f2ea80cc180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:11.111 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:11.109+0000 7f2eaf60a640 1 -- 172.21.15.27:0/3799922162 >> 172.21.15.27:0/3799922162 conn(0x7f2ea8067ab0 msgr2=0x7f2ea815efa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:11.111 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:11.109+0000 7f2eaf60a640 1 -- 172.21.15.27:0/3799922162 shutdown_connections 2024-09-12T03:16:11.111 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:11.109+0000 7f2eaf60a640 1 -- 172.21.15.27:0/3799922162 wait complete. 2024-09-12T03:16:11.121 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-09-12T03:16:11.133 INFO:tasks.ceph.mgr.x.smithi045.stderr:2024-09-12T03:16:11.133+0000 7fc1f240d040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-12T03:16:11.137 INFO:tasks.ceph.mgr.y.smithi027.stderr:2024-09-12T03:16:11.136+0000 7f32df11f040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-12T03:16:11.187 INFO:tasks.ceph.mgr.x.smithi045.stderr:2024-09-12T03:16:11.187+0000 7fc1f240d040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-12T03:16:11.191 INFO:tasks.ceph.mgr.y.smithi027.stderr:2024-09-12T03:16:11.190+0000 7f32df11f040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-12T03:16:11.257 INFO:tasks.ceph.mgr.x.smithi045.stderr:2024-09-12T03:16:11.258+0000 7fc1f240d040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-12T03:16:11.262 INFO:tasks.ceph.mgr.y.smithi027.stderr:2024-09-12T03:16:11.261+0000 7f32df11f040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-12T03:16:11.304 INFO:tasks.ceph.mgr.x.smithi045.stderr:2024-09-12T03:16:11.304+0000 7fc1f240d040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-12T03:16:11.309 INFO:tasks.ceph.mgr.y.smithi027.stderr:2024-09-12T03:16:11.309+0000 7f32df11f040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-12T03:16:11.350 INFO:tasks.ceph.mgr.x.smithi045.stderr:2024-09-12T03:16:11.350+0000 7fc1f240d040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-12T03:16:11.356 INFO:tasks.ceph.mgr.y.smithi027.stderr:2024-09-12T03:16:11.355+0000 7f32df11f040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-12T03:16:11.379 INFO:tasks.ceph.osd.6.smithi045.stderr:2024-09-12T03:16:11.379+0000 7f29058e0640 -1 Falling back to public interface 2024-09-12T03:16:11.620 INFO:tasks.ceph.osd.5.smithi045.stderr:2024-09-12T03:16:11.620+0000 7f2ec320b640 -1 Falling back to public interface 2024-09-12T03:16:11.646 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-12T03:16:11.645+0000 7f040ea17640 -1 Falling back to public interface 2024-09-12T03:16:11.653 INFO:tasks.ceph.osd.4.smithi045.stderr:2024-09-12T03:16:11.653+0000 7fe9bdc0e640 -1 Falling back to public interface 2024-09-12T03:16:11.879 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-12T03:16:11.878+0000 7fed9e323640 -1 Falling back to public interface 2024-09-12T03:16:11.880 INFO:tasks.ceph.osd.7.smithi045.stderr:2024-09-12T03:16:11.880+0000 7ff6490a5640 -1 Falling back to public interface 2024-09-12T03:16:11.884 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-12T03:16:11.883+0000 7fe81901c640 -1 Falling back to public interface 2024-09-12T03:16:11.901 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-12T03:16:11.900+0000 7f6d9f61b640 -1 Falling back to public interface 2024-09-12T03:16:12.039 INFO:tasks.ceph.mgr.x.smithi045.stderr:2024-09-12T03:16:12.039+0000 7fc1f240d040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-12T03:16:12.057 INFO:tasks.ceph.mgr.y.smithi027.stderr:2024-09-12T03:16:12.056+0000 7f32df11f040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-12T03:16:13.058 INFO:tasks.ceph.mgr.x.smithi045.stderr:2024-09-12T03:16:13.058+0000 7fc192071640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-09-12T03:16:13.058 INFO:tasks.ceph.mgr.x.smithi045.stderr:2024-09-12T03:16:13.058+0000 7fc192071640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-09-12T03:16:15.445 INFO:tasks.ceph.osd.6.smithi045.stderr:2024-09-12T03:16:15.444+0000 7f29058e0640 -1 osd.6 0 log_to_monitors true 2024-09-12T03:16:15.470 INFO:tasks.ceph.osd.4.smithi045.stderr:2024-09-12T03:16:15.470+0000 7fe9bdc0e640 -1 osd.4 0 log_to_monitors true 2024-09-12T03:16:15.472 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-12T03:16:15.471+0000 7f040ea17640 -1 osd.3 0 log_to_monitors true 2024-09-12T03:16:15.736 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-12T03:16:15.735+0000 7f6d9f61b640 -1 osd.2 0 log_to_monitors true 2024-09-12T03:16:15.872 INFO:tasks.ceph.osd.7.smithi045.stderr:2024-09-12T03:16:15.871+0000 7ff6490a5640 -1 osd.7 0 log_to_monitors true 2024-09-12T03:16:15.907 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-12T03:16:15.906+0000 7fe81901c640 -1 osd.0 0 log_to_monitors true 2024-09-12T03:16:16.113 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-12T03:16:16.112+0000 7fed9e323640 -1 osd.1 0 log_to_monitors true 2024-09-12T03:16:16.175 INFO:tasks.ceph.osd.5.smithi045.stderr:2024-09-12T03:16:16.174+0000 7f2ec320b640 -1 osd.5 0 log_to_monitors true 2024-09-12T03:16:17.929 DEBUG:teuthology.orchestra.run.smithi027:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-12T03:16:18.039 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:18.038+0000 7f5be91c4640 1 Processor -- start 2024-09-12T03:16:18.039 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:18.038+0000 7f5be91c4640 1 -- start start 2024-09-12T03:16:18.042 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:18.038+0000 7f5be91c4640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f5be415e830 0x7f5be40b3990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:18.042 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:18.038+0000 7f5be91c4640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f5be40ceeb0 0x7f5be40d1630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:18.042 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:18.038+0000 7f5be91c4640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5be40cf3b0 con 0x7f5be40d1b70 2024-09-12T03:16:18.042 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:18.038+0000 7f5be91c4640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f5be40cffc0 con 0x7f5be415e830 2024-09-12T03:16:18.042 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:18.038+0000 7f5be91c4640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f5be40d1fa0 con 0x7f5be40ceeb0 2024-09-12T03:16:18.042 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:18.039+0000 7f5be2d9d640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f5be415e830 0x7f5be40b3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:18.042 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:18.039+0000 7f5be359e640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f5be40d1b70 0x7f5be40b62e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:44264/0 (socket says 172.21.15.27:44264) 2024-09-12T03:16:18.042 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:18.039+0000 7f5be2d9d640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f5be415e830 0x7f5be40b3990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55524/0 (socket says 172.21.15.27:55524) 2024-09-12T03:16:18.042 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:18.039+0000 7f5be359e640 1 -- 172.21.15.27:0/2138577453 learned_addr learned my addr 172.21.15.27:0/2138577453 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:16:18.042 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:18.039+0000 7f5be259c640 1 --2- 172.21.15.27:0/2138577453 >> v2:172.21.15.45:3300/0 conn(0x7f5be40ceeb0 0x7f5be40d1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:18.042 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:18.039+0000 7f5be2d9d640 1 -- 172.21.15.27:0/2138577453 >> v2:172.21.15.45:3300/0 conn(0x7f5be40ceeb0 msgr2=0x7f5be40d1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:18.043 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:18.039+0000 7f5be2d9d640 1 --2- 172.21.15.27:0/2138577453 >> v2:172.21.15.45:3300/0 conn(0x7f5be40ceeb0 0x7f5be40d1630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:18.043 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:18.039+0000 7f5be2d9d640 1 -- 172.21.15.27:0/2138577453 >> v1:172.21.15.27:6789/0 conn(0x7f5be40d1b70 legacy=0x7f5be40b62e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:18.043 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:18.039+0000 7f5be1d9b640 1 -- 172.21.15.27:0/2138577453 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1625958942 0 0) 0x7f5be40cf3b0 con 0x7f5be40d1b70 2024-09-12T03:16:18.043 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:18.039+0000 7f5be2d9d640 1 -- 172.21.15.27:0/2138577453 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5be40cf690 con 0x7f5be415e830 2024-09-12T03:16:18.043 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:18.039+0000 7f5be259c640 1 --2- 172.21.15.27:0/2138577453 >> v2:172.21.15.45:3300/0 conn(0x7f5be40ceeb0 0x7f5be40d1630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:16:18.043 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:18.040+0000 7f5be2d9d640 1 --2- 172.21.15.27:0/2138577453 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5be415e830 0x7f5be40b3990 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f5be408d2e0 tx=0x7f5bd402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21eea7756eadecec server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:18.043 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:18.040+0000 7f5be1d9b640 1 -- 172.21.15.27:0/2138577453 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bd4037070 con 0x7f5be415e830 2024-09-12T03:16:18.043 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:18.040+0000 7f5be1d9b640 1 -- 172.21.15.27:0/2138577453 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5bd4032440 con 0x7f5be415e830 2024-09-12T03:16:18.043 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:18.040+0000 7f5be1d9b640 1 -- 172.21.15.27:0/2138577453 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bd4032810 con 0x7f5be415e830 2024-09-12T03:16:18.044 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:18.040+0000 7f5be91c4640 1 -- 172.21.15.27:0/2138577453 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5be415e830 msgr2=0x7f5be40b3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:18.044 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:18.040+0000 7f5be91c4640 1 --2- 172.21.15.27:0/2138577453 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5be415e830 0x7f5be40b3990 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f5be408d2e0 tx=0x7f5bd402a1a0 comp rx=0 tx=0).stop 2024-09-12T03:16:18.044 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:18.040+0000 7f5be91c4640 1 -- 172.21.15.27:0/2138577453 shutdown_connections 2024-09-12T03:16:18.044 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:18.040+0000 7f5be91c4640 1 --2- 172.21.15.27:0/2138577453 >> v2:172.21.15.45:3300/0 conn(0x7f5be40ceeb0 0x7f5be40d1630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:18.044 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:18.040+0000 7f5be91c4640 1 --2- 172.21.15.27:0/2138577453 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5be415e830 0x7f5be40b3990 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:18.044 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:18.040+0000 7f5be91c4640 1 -- 172.21.15.27:0/2138577453 >> 172.21.15.27:0/2138577453 conn(0x7f5be4067ab0 msgr2=0x7f5be4067ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:18.044 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:18.040+0000 7f5be91c4640 1 -- 172.21.15.27:0/2138577453 shutdown_connections 2024-09-12T03:16:18.044 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:18.041+0000 7f5be91c4640 1 -- 172.21.15.27:0/2138577453 wait complete. 2024-09-12T03:16:18.048 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:18.042+0000 7f5be91c4640 1 Processor -- start 2024-09-12T03:16:18.048 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:18.042+0000 7f5be91c4640 1 -- start start 2024-09-12T03:16:18.048 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:18.043+0000 7f5be91c4640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5be40ceeb0 0x7f5be414dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:18.048 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:18.043+0000 7f5be91c4640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f5be40d1b70 0x7f5be40cc980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:18.048 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:18.043+0000 7f5be91c4640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5be415e830 0x7f5be40ccec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:18.048 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:18.043+0000 7f5be91c4640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f5be40b3ed0 con 0x7f5be40ceeb0 2024-09-12T03:16:18.048 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:18.043+0000 7f5be91c4640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f5be40cfc50 con 0x7f5be415e830 2024-09-12T03:16:18.048 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:18.043+0000 7f5be91c4640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f5be415ec10 con 0x7f5be40d1b70 2024-09-12T03:16:18.048 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:18.043+0000 7f5be2d9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5be40ceeb0 0x7f5be414dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:18.049 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:18.043+0000 7f5be2d9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5be40ceeb0 0x7f5be414dc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55540/0 (socket says 172.21.15.27:55540) 2024-09-12T03:16:18.049 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:18.043+0000 7f5be2d9d640 1 -- 172.21.15.27:0/1840490783 learned_addr learned my addr 172.21.15.27:0/1840490783 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:16:18.049 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:18.043+0000 7f5be259c640 1 --2- 172.21.15.27:0/1840490783 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f5be40d1b70 0x7f5be40cc980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:18.049 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:18.043+0000 7f5be359e640 1 --2- 172.21.15.27:0/1840490783 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5be415e830 0x7f5be40ccec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:18.049 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:18.043+0000 7f5be2d9d640 1 -- 172.21.15.27:0/1840490783 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f5be40d1b70 msgr2=0x7f5be40cc980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:18.049 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:18.043+0000 7f5be2d9d640 1 --2- 172.21.15.27:0/1840490783 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f5be40d1b70 0x7f5be40cc980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:18.049 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:18.043+0000 7f5be2d9d640 1 -- 172.21.15.27:0/1840490783 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5be415e830 msgr2=0x7f5be40ccec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:18.049 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:18.043+0000 7f5be2d9d640 1 --2- 172.21.15.27:0/1840490783 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5be415e830 0x7f5be40ccec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:18.049 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:18.043+0000 7f5be2d9d640 1 -- 172.21.15.27:0/1840490783 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5be415dcb0 con 0x7f5be40ceeb0 2024-09-12T03:16:18.049 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:18.043+0000 7f5be359e640 1 --2- 172.21.15.27:0/1840490783 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5be415e830 0x7f5be40ccec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:16:18.049 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:18.043+0000 7f5be2d9d640 1 --2- 172.21.15.27:0/1840490783 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5be40ceeb0 0x7f5be414dc00 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f5bd4000c00 tx=0x7f5bd40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:18.049 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:18.043+0000 7f5bcb7fe640 1 -- 172.21.15.27:0/1840490783 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bd4037050 con 0x7f5be40ceeb0 2024-09-12T03:16:18.049 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:18.044+0000 7f5bcb7fe640 1 -- 172.21.15.27:0/1840490783 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5bd4038040 con 0x7f5be40ceeb0 2024-09-12T03:16:18.050 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:18.044+0000 7f5be91c4640 1 -- 172.21.15.27:0/1840490783 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5be415df90 con 0x7f5be40ceeb0 2024-09-12T03:16:18.050 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:18.044+0000 7f5bcb7fe640 1 -- 172.21.15.27:0/1840490783 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bd4006e10 con 0x7f5be40ceeb0 2024-09-12T03:16:18.050 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:18.044+0000 7f5be91c4640 1 -- 172.21.15.27:0/1840490783 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5be415b400 con 0x7f5be40ceeb0 2024-09-12T03:16:18.050 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:18.045+0000 7f5be91c4640 1 -- 172.21.15.27:0/1840490783 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5be40d1630 con 0x7f5be40ceeb0 2024-09-12T03:16:18.050 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:18.045+0000 7f5bcb7fe640 1 -- 172.21.15.27:0/1840490783 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 4) ==== 90442+0+0 (secure 0 0 0) 0x7f5bd4030980 con 0x7f5be40ceeb0 2024-09-12T03:16:18.050 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:18.045+0000 7f5bcb7fe640 1 --2- 172.21.15.27:0/1840490783 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f5bd006bda0 0x7f5bd006e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:18.050 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:18.046+0000 7f5bcb7fe640 1 -- 172.21.15.27:0/1840490783 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(12..12 src has 1..12) ==== 2199+0+0 (secure 0 0 0) 0x7f5bd40aa9c0 con 0x7f5be40ceeb0 2024-09-12T03:16:18.050 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:18.046+0000 7f5be259c640 1 --2- 172.21.15.27:0/1840490783 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f5bd006bda0 0x7f5bd006e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:18.050 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:18.046+0000 7f5be259c640 1 --2- 172.21.15.27:0/1840490783 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f5bd006bda0 0x7f5bd006e260 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f5be414e410 tx=0x7f5bd8008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:18.051 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:18.050+0000 7f5bcb7fe640 1 -- 172.21.15.27:0/1840490783 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f5bd407a290 con 0x7f5be40ceeb0 2024-09-12T03:16:18.310 INFO:tasks.ceph.osd.7.smithi045.stderr:2024-09-12T03:16:18.309+0000 7ff645848640 -1 osd.7 0 waiting for initial osdmap 2024-09-12T03:16:18.317 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:18.316+0000 7f5be91c4640 1 -- 172.21.15.27:0/1840490783 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5be40d25e0 con 0x7f5be40ceeb0 2024-09-12T03:16:18.318 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:18.317+0000 7f5bcb7fe640 1 -- 172.21.15.27:0/1840490783 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v13) ==== 74+0+7058 (secure 0 0 0) 0x7f5bd4079b90 con 0x7f5be40ceeb0 2024-09-12T03:16:18.318 INFO:teuthology.misc.health.smithi027.stdout: 2024-09-12T03:16:18.319 INFO:teuthology.misc.health.smithi027.stdout:{"epoch":13,"fsid":"d2df504e-2647-4385-a5db-9fdf69f487a0","created":"2024-09-12T03:16:07.778937+0000","modified":"2024-09-12T03:16:18.071133+0000","last_up_change":"0.000000","last_in_change":"2024-09-12T03:16:09.893086+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"1851116c-1fb3-4dc9-9772-f90e2ef4c355","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"f38ae17d-73eb-4204-b99d-975fa806fb87","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"fb12dcc8-bdd0-42b8-b46d-8b10106224bb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"d74d3f49-9c24-4b9f-98e2-3ecd7b5ef3b3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"2bf123f2-1b0d-41c4-aa3b-805bab267e59","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"c49ba5da-91c6-4a86-a97b-4aba89c253ec","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"b0123475-89f7-4a34-9e33-d797b99dd9dc","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"4dad773d-b7d8-46fc-8a10-a972aad27b29","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T03:16:18.322 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:18.321+0000 7f5be91c4640 1 -- 172.21.15.27:0/1840490783 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f5bd006bda0 msgr2=0x7f5bd006e260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:18.322 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:18.321+0000 7f5be91c4640 1 --2- 172.21.15.27:0/1840490783 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f5bd006bda0 0x7f5bd006e260 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f5be414e410 tx=0x7f5bd8008040 comp rx=0 tx=0).stop 2024-09-12T03:16:18.322 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:18.321+0000 7f5be91c4640 1 -- 172.21.15.27:0/1840490783 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5be40ceeb0 msgr2=0x7f5be414dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:18.322 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:18.321+0000 7f5be91c4640 1 --2- 172.21.15.27:0/1840490783 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5be40ceeb0 0x7f5be414dc00 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f5bd4000c00 tx=0x7f5bd40046c0 comp rx=0 tx=0).stop 2024-09-12T03:16:18.322 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:18.321+0000 7f5be91c4640 1 -- 172.21.15.27:0/1840490783 shutdown_connections 2024-09-12T03:16:18.322 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:18.321+0000 7f5be91c4640 1 --2- 172.21.15.27:0/1840490783 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f5bd006bda0 0x7f5bd006e260 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:18.323 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:18.321+0000 7f5be91c4640 1 --2- 172.21.15.27:0/1840490783 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5be415e830 0x7f5be40ccec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:18.323 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:18.321+0000 7f5be91c4640 1 --2- 172.21.15.27:0/1840490783 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f5be40d1b70 0x7f5be40cc980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:18.323 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:18.321+0000 7f5be91c4640 1 --2- 172.21.15.27:0/1840490783 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5be40ceeb0 0x7f5be414dc00 secure :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f5bd4000c00 tx=0x7f5bd40046c0 comp rx=0 tx=0).stop 2024-09-12T03:16:18.323 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:18.321+0000 7f5be91c4640 1 -- 172.21.15.27:0/1840490783 >> 172.21.15.27:0/1840490783 conn(0x7f5be4067ab0 msgr2=0x7f5be4067e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:18.323 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:18.321+0000 7f5be91c4640 1 -- 172.21.15.27:0/1840490783 shutdown_connections 2024-09-12T03:16:18.323 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:18.321+0000 7f5be91c4640 1 -- 172.21.15.27:0/1840490783 wait complete. 2024-09-12T03:16:18.327 INFO:tasks.ceph.osd.7.smithi045.stderr:2024-09-12T03:16:18.327+0000 7ff63ce43640 -1 osd.7 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T03:16:18.336 INFO:tasks.ceph.osd.6.smithi045.stderr:2024-09-12T03:16:18.335+0000 7f290104c640 -1 osd.6 0 waiting for initial osdmap 2024-09-12T03:16:18.337 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-09-12T03:16:18.350 INFO:tasks.ceph.osd.6.smithi045.stderr:2024-09-12T03:16:18.350+0000 7f28f967e640 -1 osd.6 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T03:16:18.366 INFO:tasks.ceph.osd.4.smithi045.stderr:2024-09-12T03:16:18.365+0000 7fe9b9d12640 -1 osd.4 0 waiting for initial osdmap 2024-09-12T03:16:18.386 INFO:tasks.ceph.osd.4.smithi045.stderr:2024-09-12T03:16:18.386+0000 7fe9b130d640 -1 osd.4 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T03:16:18.878 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-12T03:16:18.877+0000 7f040ac15640 -1 osd.3 0 waiting for initial osdmap 2024-09-12T03:16:18.891 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-12T03:16:18.890+0000 7f0402210640 -1 osd.3 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T03:16:18.915 INFO:tasks.ceph.osd.5.smithi045.stderr:2024-09-12T03:16:18.914+0000 7f2ebe40e640 -1 osd.5 0 waiting for initial osdmap 2024-09-12T03:16:18.918 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-12T03:16:18.916+0000 7f6d9a7e5640 -1 osd.2 0 waiting for initial osdmap 2024-09-12T03:16:18.928 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-12T03:16:18.927+0000 7f6d92e17640 -1 osd.2 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T03:16:18.929 INFO:tasks.ceph.osd.5.smithi045.stderr:2024-09-12T03:16:18.929+0000 7f2eb6a40640 -1 osd.5 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T03:16:18.964 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-12T03:16:18.963+0000 7fe81417e640 -1 osd.0 0 waiting for initial osdmap 2024-09-12T03:16:18.974 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-12T03:16:18.973+0000 7fe80c7b0640 -1 osd.0 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T03:16:19.194 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-12T03:16:19.193+0000 7fed99a8f640 -1 osd.1 0 waiting for initial osdmap 2024-09-12T03:16:19.205 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-12T03:16:19.205+0000 7fed920c1640 -1 osd.1 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T03:16:25.145 DEBUG:teuthology.orchestra.run.smithi027:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-12T03:16:25.250 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:25.250+0000 7f5f41796640 1 Processor -- start 2024-09-12T03:16:25.250 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:25.250+0000 7f5f41796640 1 -- start start 2024-09-12T03:16:25.252 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:25.251+0000 7f5f41796640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f5f3c15e720 0x7f5f3c070440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:25.252 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:25.251+0000 7f5f41796640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f5f3c162a10 0x7f5f3c06c820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:25.252 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:25.251+0000 7f5f41796640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5f3c070a70 con 0x7f5f3c0b4ad0 2024-09-12T03:16:25.252 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:25.251+0000 7f5f41796640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f5f3c070d50 con 0x7f5f3c15e720 2024-09-12T03:16:25.252 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:25.251+0000 7f5f41796640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f5f3c06bd40 con 0x7f5f3c162a10 2024-09-12T03:16:25.252 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:25.251+0000 7f5f3affd640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f5f3c15e720 0x7f5f3c070440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:25.252 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:25.251+0000 7f5f3affd640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f5f3c15e720 0x7f5f3c070440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36478/0 (socket says 172.21.15.27:36478) 2024-09-12T03:16:25.252 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:25.251+0000 7f5f3affd640 1 -- 172.21.15.27:0/2141339134 learned_addr learned my addr 172.21.15.27:0/2141339134 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:16:25.252 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:25.251+0000 7f5f3b7fe640 1 --2- 172.21.15.27:0/2141339134 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f5f3c162a10 0x7f5f3c06c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:25.253 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:25.251+0000 7f5f3affd640 1 -- 172.21.15.27:0/2141339134 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f5f3c162a10 msgr2=0x7f5f3c06c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:25.253 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:25.251+0000 7f5f3affd640 1 --2- 172.21.15.27:0/2141339134 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f5f3c162a10 0x7f5f3c06c820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:25.253 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:25.251+0000 7f5f3affd640 1 -- 172.21.15.27:0/2141339134 >> v1:172.21.15.45:6789/0 conn(0x7f5f3c0b4ad0 legacy=0x7f5f3c162300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:25.253 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:25.251+0000 7f5f3affd640 1 -- 172.21.15.27:0/2141339134 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f3c06c020 con 0x7f5f3c15e720 2024-09-12T03:16:25.253 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:25.251+0000 7f5f3b7fe640 1 --2- 172.21.15.27:0/2141339134 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f5f3c162a10 0x7f5f3c06c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:16:25.253 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:25.252+0000 7f5f3affd640 1 --2- 172.21.15.27:0/2141339134 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5f3c15e720 0x7f5f3c070440 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f5f2c009870 tx=0x7f5f2c02a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=112dc87364cf5f29 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:25.256 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:25.252+0000 7f5f3a7fc640 1 -- 172.21.15.27:0/2141339134 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f2c037070 con 0x7f5f3c15e720 2024-09-12T03:16:25.256 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:25.252+0000 7f5f3a7fc640 1 -- 172.21.15.27:0/2141339134 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f2c032440 con 0x7f5f3c15e720 2024-09-12T03:16:25.257 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:25.252+0000 7f5f3a7fc640 1 -- 172.21.15.27:0/2141339134 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f2c032810 con 0x7f5f3c15e720 2024-09-12T03:16:25.257 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:25.253+0000 7f5f41796640 1 -- 172.21.15.27:0/2141339134 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5f3c15e720 msgr2=0x7f5f3c070440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:25.257 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:25.253+0000 7f5f41796640 1 --2- 172.21.15.27:0/2141339134 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5f3c15e720 0x7f5f3c070440 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f5f2c009870 tx=0x7f5f2c02a1c0 comp rx=0 tx=0).stop 2024-09-12T03:16:25.257 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:25.253+0000 7f5f41796640 1 -- 172.21.15.27:0/2141339134 shutdown_connections 2024-09-12T03:16:25.257 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:25.253+0000 7f5f41796640 1 --2- 172.21.15.27:0/2141339134 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f5f3c162a10 0x7f5f3c06c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:25.257 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:25.253+0000 7f5f41796640 1 --2- 172.21.15.27:0/2141339134 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5f3c15e720 0x7f5f3c070440 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:25.257 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:25.253+0000 7f5f41796640 1 -- 172.21.15.27:0/2141339134 >> 172.21.15.27:0/2141339134 conn(0x7f5f3c09ba20 msgr2=0x7f5f3c0b4120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:25.257 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:25.253+0000 7f5f41796640 1 -- 172.21.15.27:0/2141339134 shutdown_connections 2024-09-12T03:16:25.257 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:25.253+0000 7f5f41796640 1 -- 172.21.15.27:0/2141339134 wait complete. 2024-09-12T03:16:25.257 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:25.254+0000 7f5f41796640 1 Processor -- start 2024-09-12T03:16:25.257 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:25.254+0000 7f5f41796640 1 -- start start 2024-09-12T03:16:25.258 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:25.255+0000 7f5f41796640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5f3c0b4ad0 0x7f5f3c0cbb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:25.258 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:25.255+0000 7f5f41796640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f5f3c15e720 0x7f5f3c153350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:25.258 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:25.255+0000 7f5f41796640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5f3c162a10 0x7f5f3c0b7da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:25.258 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:25.255+0000 7f5f41796640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f5f3c15eb00 con 0x7f5f3c162a10 2024-09-12T03:16:25.258 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:25.255+0000 7f5f41796640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f5f3c0b4f10 con 0x7f5f3c0b4ad0 2024-09-12T03:16:25.258 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:25.255+0000 7f5f41796640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f5f3c15ec80 con 0x7f5f3c15e720 2024-09-12T03:16:25.258 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:25.255+0000 7f5f3b7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5f3c162a10 0x7f5f3c0b7da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:25.258 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:25.255+0000 7f5f3b7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5f3c162a10 0x7f5f3c0b7da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36490/0 (socket says 172.21.15.27:36490) 2024-09-12T03:16:25.258 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:25.255+0000 7f5f3b7fe640 1 -- 172.21.15.27:0/3739352068 learned_addr learned my addr 172.21.15.27:0/3739352068 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:16:25.258 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:25.255+0000 7f5f3affd640 1 --2- 172.21.15.27:0/3739352068 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5f3c0b4ad0 0x7f5f3c0cbb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:25.258 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:25.255+0000 7f5f33fff640 1 --2- 172.21.15.27:0/3739352068 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f5f3c15e720 0x7f5f3c153350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:25.258 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:25.256+0000 7f5f3b7fe640 1 -- 172.21.15.27:0/3739352068 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f5f3c15e720 msgr2=0x7f5f3c153350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:25.258 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:25.256+0000 7f5f3b7fe640 1 --2- 172.21.15.27:0/3739352068 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f5f3c15e720 0x7f5f3c153350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:25.259 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:25.256+0000 7f5f3b7fe640 1 -- 172.21.15.27:0/3739352068 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5f3c0b4ad0 msgr2=0x7f5f3c0cbb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:25.259 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:25.256+0000 7f5f3b7fe640 1 --2- 172.21.15.27:0/3739352068 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5f3c0b4ad0 0x7f5f3c0cbb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:25.259 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:25.256+0000 7f5f3b7fe640 1 -- 172.21.15.27:0/3739352068 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f3c0caa90 con 0x7f5f3c162a10 2024-09-12T03:16:25.259 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:25.256+0000 7f5f3b7fe640 1 --2- 172.21.15.27:0/3739352068 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5f3c162a10 0x7f5f3c0b7da0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f5f3400ceb0 tx=0x7f5f3400a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:25.259 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:25.256+0000 7f5f337fe640 1 -- 172.21.15.27:0/3739352068 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f3401a070 con 0x7f5f3c162a10 2024-09-12T03:16:25.259 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:25.256+0000 7f5f337fe640 1 -- 172.21.15.27:0/3739352068 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f3400d840 con 0x7f5f3c162a10 2024-09-12T03:16:25.259 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:25.256+0000 7f5f41796640 1 -- 172.21.15.27:0/3739352068 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f3c0cad70 con 0x7f5f3c162a10 2024-09-12T03:16:25.259 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:25.256+0000 7f5f337fe640 1 -- 172.21.15.27:0/3739352068 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f3400dc30 con 0x7f5f3c162a10 2024-09-12T03:16:25.259 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:25.256+0000 7f5f41796640 1 -- 172.21.15.27:0/3739352068 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f3c0cb330 con 0x7f5f3c162a10 2024-09-12T03:16:25.260 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:25.257+0000 7f5f41796640 1 -- 172.21.15.27:0/3739352068 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f3c070440 con 0x7f5f3c162a10 2024-09-12T03:16:25.260 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:25.257+0000 7f5f337fe640 1 -- 172.21.15.27:0/3739352068 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5f3401d070 con 0x7f5f3c162a10 2024-09-12T03:16:25.260 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:25.258+0000 7f5f337fe640 1 --2- 172.21.15.27:0/3739352068 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f5f1c06be30 0x7f5f1c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:25.260 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:25.258+0000 7f5f337fe640 1 -- 172.21.15.27:0/3739352068 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7f5f3408b2d0 con 0x7f5f3c162a10 2024-09-12T03:16:25.260 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:25.258+0000 7f5f3affd640 1 --2- 172.21.15.27:0/3739352068 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f5f1c06be30 0x7f5f1c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:25.260 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:25.259+0000 7f5f3affd640 1 --2- 172.21.15.27:0/3739352068 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f5f1c06be30 0x7f5f1c06e2f0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f5f2c034000 tx=0x7f5f2c02e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:25.263 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:25.263+0000 7f5f337fe640 1 -- 172.21.15.27:0/3739352068 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f5f34059980 con 0x7f5f3c162a10 2024-09-12T03:16:25.440 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:25.440+0000 7f5f41796640 1 -- 172.21.15.27:0/3739352068 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5f3c0cfb70 con 0x7f5f3c162a10 2024-09-12T03:16:25.441 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:25.441+0000 7f5f337fe640 1 -- 172.21.15.27:0/3739352068 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) ==== 74+0+12848 (secure 0 0 0) 0x7f5f3405dc80 con 0x7f5f3c162a10 2024-09-12T03:16:25.441 INFO:teuthology.misc.health.smithi027.stdout: 2024-09-12T03:16:25.441 INFO:teuthology.misc.health.smithi027.stdout:{"epoch":17,"fsid":"d2df504e-2647-4385-a5db-9fdf69f487a0","created":"2024-09-12T03:16:07.778937+0000","modified":"2024-09-12T03:16:22.324345+0000","last_up_change":"2024-09-12T03:16:19.314778+0000","last_in_change":"2024-09-12T03:16:09.893086+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T03:16:20.097646+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1851116c-1fb3-4dc9-9772-f90e2ef4c355","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6817","nonce":3002321488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6819","nonce":3002321488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6823","nonce":3002321488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6821","nonce":3002321488}]},"public_addr":"172.21.15.27:6817/3002321488","cluster_addr":"172.21.15.27:6819/3002321488","heartbeat_back_addr":"172.21.15.27:6823/3002321488","heartbeat_front_addr":"172.21.15.27:6821/3002321488","state":["exists","up"]},{"osd":1,"uuid":"f38ae17d-73eb-4204-b99d-975fa806fb87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6809","nonce":1613102886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6811","nonce":1613102886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6815","nonce":1613102886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6813","nonce":1613102886}]},"public_addr":"172.21.15.27:6809/1613102886","cluster_addr":"172.21.15.27:6811/1613102886","heartbeat_back_addr":"172.21.15.27:6815/1613102886","heartbeat_front_addr":"172.21.15.27:6813/1613102886","state":["exists","up"]},{"osd":2,"uuid":"fb12dcc8-bdd0-42b8-b46d-8b10106224bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6825","nonce":3534723355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6827","nonce":3534723355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6831","nonce":3534723355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6829","nonce":3534723355}]},"public_addr":"172.21.15.27:6825/3534723355","cluster_addr":"172.21.15.27:6827/3534723355","heartbeat_back_addr":"172.21.15.27:6831/3534723355","heartbeat_front_addr":"172.21.15.27:6829/3534723355","state":["exists","up"]},{"osd":3,"uuid":"d74d3f49-9c24-4b9f-98e2-3ecd7b5ef3b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6801","nonce":80661598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6803","nonce":80661598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6807","nonce":80661598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6805","nonce":80661598}]},"public_addr":"172.21.15.27:6801/80661598","cluster_addr":"172.21.15.27:6803/80661598","heartbeat_back_addr":"172.21.15.27:6807/80661598","heartbeat_front_addr":"172.21.15.27:6805/80661598","state":["exists","up"]},{"osd":4,"uuid":"2bf123f2-1b0d-41c4-aa3b-805bab267e59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6817","nonce":3943189827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6819","nonce":3943189827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6823","nonce":3943189827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6821","nonce":3943189827}]},"public_addr":"172.21.15.45:6817/3943189827","cluster_addr":"172.21.15.45:6819/3943189827","heartbeat_back_addr":"172.21.15.45:6823/3943189827","heartbeat_front_addr":"172.21.15.45:6821/3943189827","state":["exists","up"]},{"osd":5,"uuid":"c49ba5da-91c6-4a86-a97b-4aba89c253ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6809","nonce":3620760330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6811","nonce":3620760330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6815","nonce":3620760330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6813","nonce":3620760330}]},"public_addr":"172.21.15.45:6809/3620760330","cluster_addr":"172.21.15.45:6811/3620760330","heartbeat_back_addr":"172.21.15.45:6815/3620760330","heartbeat_front_addr":"172.21.15.45:6813/3620760330","state":["exists","up"]},{"osd":6,"uuid":"b0123475-89f7-4a34-9e33-d797b99dd9dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6801","nonce":1989083510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6803","nonce":1989083510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6807","nonce":1989083510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6805","nonce":1989083510}]},"public_addr":"172.21.15.45:6801/1989083510","cluster_addr":"172.21.15.45:6803/1989083510","heartbeat_back_addr":"172.21.15.45:6807/1989083510","heartbeat_front_addr":"172.21.15.45:6805/1989083510","state":["exists","up"]},{"osd":7,"uuid":"4dad773d-b7d8-46fc-8a10-a972aad27b29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6825","nonce":1513300294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6827","nonce":1513300294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6831","nonce":1513300294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6829","nonce":1513300294}]},"public_addr":"172.21.15.45:6825/1513300294","cluster_addr":"172.21.15.45:6827/1513300294","heartbeat_back_addr":"172.21.15.45:6831/1513300294","heartbeat_front_addr":"172.21.15.45:6829/1513300294","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.881855+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:17.133252+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.735339+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.500742+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.477571+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:17.167667+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.470291+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.873743+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T03:16:25.443 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:25.443+0000 7f5f41796640 1 -- 172.21.15.27:0/3739352068 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f5f1c06be30 msgr2=0x7f5f1c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:25.443 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:25.443+0000 7f5f41796640 1 --2- 172.21.15.27:0/3739352068 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f5f1c06be30 0x7f5f1c06e2f0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f5f2c034000 tx=0x7f5f2c02e000 comp rx=0 tx=0).stop 2024-09-12T03:16:25.444 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:25.443+0000 7f5f41796640 1 -- 172.21.15.27:0/3739352068 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5f3c162a10 msgr2=0x7f5f3c0b7da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:25.444 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:25.443+0000 7f5f41796640 1 --2- 172.21.15.27:0/3739352068 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5f3c162a10 0x7f5f3c0b7da0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f5f3400ceb0 tx=0x7f5f3400a710 comp rx=0 tx=0).stop 2024-09-12T03:16:25.444 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:25.443+0000 7f5f41796640 1 -- 172.21.15.27:0/3739352068 shutdown_connections 2024-09-12T03:16:25.444 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:25.443+0000 7f5f41796640 1 --2- 172.21.15.27:0/3739352068 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f5f1c06be30 0x7f5f1c06e2f0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:25.444 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:25.443+0000 7f5f41796640 1 --2- 172.21.15.27:0/3739352068 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5f3c162a10 0x7f5f3c0b7da0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:25.444 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:25.443+0000 7f5f41796640 1 --2- 172.21.15.27:0/3739352068 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f5f3c15e720 0x7f5f3c153350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:25.444 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:25.443+0000 7f5f41796640 1 --2- 172.21.15.27:0/3739352068 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5f3c0b4ad0 0x7f5f3c0cbb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:25.444 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:25.443+0000 7f5f41796640 1 -- 172.21.15.27:0/3739352068 >> 172.21.15.27:0/3739352068 conn(0x7f5f3c09ba20 msgr2=0x7f5f3c0af500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:25.444 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:25.443+0000 7f5f41796640 1 -- 172.21.15.27:0/3739352068 shutdown_connections 2024-09-12T03:16:25.444 INFO:teuthology.misc.health.smithi027.stderr:2024-09-12T03:16:25.443+0000 7f5f41796640 1 -- 172.21.15.27:0/3739352068 wait complete. 2024-09-12T03:16:25.452 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-09-12T03:16:25.453 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-09-12T03:16:25.453 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-12T03:16:25.453 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-12T03:16:25.455 INFO:tasks.ceph.mds.a:Started 2024-09-12T03:16:25.455 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-12T03:16:25.456 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-12T03:16:25.457 INFO:tasks.ceph.mds.c:Started 2024-09-12T03:16:25.458 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-12T03:16:25.458 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-12T03:16:25.460 INFO:tasks.ceph.mds.b:Started 2024-09-12T03:16:25.461 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-12T03:16:25.461 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-12T03:16:25.463 INFO:tasks.ceph.mds.d:Started 2024-09-12T03:16:25.463 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-09-12T03:16:25.464 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-12T03:16:25.533 INFO:tasks.ceph.mds.a.smithi027.stdout:starting mds.a at 2024-09-12T03:16:25.533 INFO:tasks.ceph.mds.c.smithi027.stdout:starting mds.c at 2024-09-12T03:16:25.551 INFO:tasks.ceph.mds.b.smithi045.stdout:starting mds.b at 2024-09-12T03:16:25.551 INFO:tasks.ceph.mds.d.smithi045.stdout:starting mds.d at 2024-09-12T03:16:25.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:25.570+0000 7f0809b2a640 1 Processor -- start 2024-09-12T03:16:25.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:25.571+0000 7f0809b2a640 1 -- start start 2024-09-12T03:16:25.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:25.571+0000 7f0809b2a640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f080415eb90 0x7f08040d2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:25.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:25.571+0000 7f0809b2a640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f08040d1720 0x7f08040b8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:25.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:25.571+0000 7f0809b2a640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08040d3760 con 0x7f08040d5200 2024-09-12T03:16:25.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:25.572+0000 7f0809b2a640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f08040b7d20 con 0x7f080415eb90 2024-09-12T03:16:25.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:25.572+0000 7f0809b2a640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f08040d42e0 con 0x7f08040d1720 2024-09-12T03:16:25.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:25.572+0000 7f0802ffd640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f08040d5200 0x7f08040d1010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:51472/0 (socket says 172.21.15.27:51472) 2024-09-12T03:16:25.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:25.572+0000 7f0802ffd640 1 -- 172.21.15.27:0/1308342021 learned_addr learned my addr 172.21.15.27:0/1308342021 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:16:25.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:25.572+0000 7f08037fe640 1 --2- 172.21.15.27:0/1308342021 >> v2:172.21.15.27:3300/0 conn(0x7f080415eb90 0x7f08040d2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:25.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:25.572+0000 7f0803fff640 1 --2- 172.21.15.27:0/1308342021 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f08040d1720 0x7f08040b8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:25.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:25.572+0000 7f08037fe640 1 -- 172.21.15.27:0/1308342021 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f08040d1720 msgr2=0x7f08040b8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:25.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:25.572+0000 7f08037fe640 1 --2- 172.21.15.27:0/1308342021 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f08040d1720 0x7f08040b8190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:25.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:25.572+0000 7f08027fc640 1 -- 172.21.15.27:0/1308342021 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1637758823 0 0) 0x7f08040d3760 con 0x7f08040d5200 2024-09-12T03:16:25.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:25.572+0000 7f08037fe640 1 -- 172.21.15.27:0/1308342021 >> v1:172.21.15.27:6789/0 conn(0x7f08040d5200 legacy=0x7f08040d1010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:25.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:25.572+0000 7f08037fe640 1 -- 172.21.15.27:0/1308342021 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08040d3a40 con 0x7f080415eb90 2024-09-12T03:16:25.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:25.573+0000 7f08037fe640 1 --2- 172.21.15.27:0/1308342021 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f080415eb90 0x7f08040d2df0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f07ec0097a0 tx=0x7f07ec02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6daef56abc2a2909 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:25.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:25.573+0000 7f08027fc640 1 -- 172.21.15.27:0/1308342021 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07ec037070 con 0x7f080415eb90 2024-09-12T03:16:25.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:25.573+0000 7f08027fc640 1 -- 172.21.15.27:0/1308342021 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07ec032440 con 0x7f080415eb90 2024-09-12T03:16:25.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:25.573+0000 7f08027fc640 1 -- 172.21.15.27:0/1308342021 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07ec032810 con 0x7f080415eb90 2024-09-12T03:16:25.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:25.573+0000 7f0809b2a640 1 -- 172.21.15.27:0/1308342021 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f080415eb90 msgr2=0x7f08040d2df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:25.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:25.573+0000 7f0809b2a640 1 --2- 172.21.15.27:0/1308342021 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f080415eb90 0x7f08040d2df0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f07ec0097a0 tx=0x7f07ec02a180 comp rx=0 tx=0).stop 2024-09-12T03:16:25.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:25.573+0000 7f0809b2a640 1 -- 172.21.15.27:0/1308342021 shutdown_connections 2024-09-12T03:16:25.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:25.573+0000 7f0809b2a640 1 --2- 172.21.15.27:0/1308342021 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f08040d1720 0x7f08040b8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:25.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:25.573+0000 7f0809b2a640 1 --2- 172.21.15.27:0/1308342021 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f080415eb90 0x7f08040d2df0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:25.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:25.573+0000 7f0809b2a640 1 -- 172.21.15.27:0/1308342021 >> 172.21.15.27:0/1308342021 conn(0x7f080406dce0 msgr2=0x7f080406e0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:25.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:25.574+0000 7f0809b2a640 1 -- 172.21.15.27:0/1308342021 shutdown_connections 2024-09-12T03:16:25.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:25.574+0000 7f0809b2a640 1 -- 172.21.15.27:0/1308342021 wait complete. 2024-09-12T03:16:25.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:25.575+0000 7f0809b2a640 1 Processor -- start 2024-09-12T03:16:25.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:25.575+0000 7f0809b2a640 1 -- start start 2024-09-12T03:16:25.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:25.576+0000 7f0809b2a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f08040d1720 0x7f08041566a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:25.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:25.576+0000 7f0809b2a640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f08040d5200 0x7f080414bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:25.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:25.576+0000 7f0809b2a640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f080415eb90 0x7f0804157690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:25.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:25.576+0000 7f0809b2a640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f08040d3330 con 0x7f08040d5200 2024-09-12T03:16:25.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:25.576+0000 7f0809b2a640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f08040d4000 con 0x7f08040d1720 2024-09-12T03:16:25.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:25.576+0000 7f0809b2a640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f08040d34b0 con 0x7f080415eb90 2024-09-12T03:16:25.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:25.576+0000 7f0802ffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f08040d5200 0x7f080414bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:25.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:25.576+0000 7f0802ffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f08040d5200 0x7f080414bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36526/0 (socket says 172.21.15.27:36526) 2024-09-12T03:16:25.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:25.576+0000 7f0802ffd640 1 -- 172.21.15.27:0/2211411400 learned_addr learned my addr 172.21.15.27:0/2211411400 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:16:25.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:25.576+0000 7f08037fe640 1 --2- 172.21.15.27:0/2211411400 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f08040d1720 0x7f08041566a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:25.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:25.576+0000 7f0803fff640 1 --2- 172.21.15.27:0/2211411400 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f080415eb90 0x7f0804157690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:25.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:25.576+0000 7f0802ffd640 1 -- 172.21.15.27:0/2211411400 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f080415eb90 msgr2=0x7f0804157690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:25.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:25.576+0000 7f0802ffd640 1 --2- 172.21.15.27:0/2211411400 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f080415eb90 0x7f0804157690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:25.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:25.576+0000 7f0802ffd640 1 -- 172.21.15.27:0/2211411400 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f08040d1720 msgr2=0x7f08041566a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:25.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:25.576+0000 7f0802ffd640 1 --2- 172.21.15.27:0/2211411400 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f08040d1720 0x7f08041566a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:25.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:25.576+0000 7f0802ffd640 1 -- 172.21.15.27:0/2211411400 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f080414c1d0 con 0x7f08040d5200 2024-09-12T03:16:25.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:25.576+0000 7f0803fff640 1 --2- 172.21.15.27:0/2211411400 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f080415eb90 0x7f0804157690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:16:25.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:25.576+0000 7f0802ffd640 1 --2- 172.21.15.27:0/2211411400 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f08040d5200 0x7f080414bc90 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f07f400c460 tx=0x7f07f400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:25.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:25.576+0000 7f07fbfff640 1 -- 172.21.15.27:0/2211411400 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07f4018070 con 0x7f08040d5200 2024-09-12T03:16:25.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:25.576+0000 7f07fbfff640 1 -- 172.21.15.27:0/2211411400 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07f4007520 con 0x7f08040d5200 2024-09-12T03:16:25.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:25.576+0000 7f07fbfff640 1 -- 172.21.15.27:0/2211411400 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07f4004a20 con 0x7f08040d5200 2024-09-12T03:16:25.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:25.576+0000 7f0809b2a640 1 -- 172.21.15.27:0/2211411400 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0804157bd0 con 0x7f08040d5200 2024-09-12T03:16:25.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:25.576+0000 7f0809b2a640 1 -- 172.21.15.27:0/2211411400 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08040cbda0 con 0x7f08040d5200 2024-09-12T03:16:25.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:25.577+0000 7f0809b2a640 1 -- 172.21.15.27:0/2211411400 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f080415f000 con 0x7f08040d5200 2024-09-12T03:16:25.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:25.578+0000 7f07fbfff640 1 -- 172.21.15.27:0/2211411400 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f07f401b070 con 0x7f08040d5200 2024-09-12T03:16:25.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:25.578+0000 7f07fbfff640 1 --2- 172.21.15.27:0/2211411400 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f07e406be30 0x7f07e406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:25.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:25.578+0000 7f07fbfff640 1 -- 172.21.15.27:0/2211411400 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7f07f4089f10 con 0x7f08040d5200 2024-09-12T03:16:25.581 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:25.579+0000 7f08037fe640 1 --2- 172.21.15.27:0/2211411400 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f07e406be30 0x7f07e406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:25.581 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:25.579+0000 7f08037fe640 1 --2- 172.21.15.27:0/2211411400 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f07e406be30 0x7f07e406e2f0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f07ec034000 tx=0x7f07ec02e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:25.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:25.583+0000 7f07fbfff640 1 -- 172.21.15.27:0/2211411400 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f07f4058c10 con 0x7f08040d5200 2024-09-12T03:16:25.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:25.794+0000 7f0809b2a640 1 -- 172.21.15.27:0/2211411400 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f08040cb360 con 0x7f08040d5200 2024-09-12T03:16:25.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:25.794+0000 7f07fbfff640 1 -- 172.21.15.27:0/2211411400 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1434 (secure 0 0 0) 0x7f07f4058510 con 0x7f08040d5200 2024-09-12T03:16:25.795 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:16:25.795 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":2,"btime":"2024-09-12T03:16:25:538062+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4278,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6834/4084728235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":4084728235},{"type":"v1","addr":"172.21.15.27:6834","nonce":4084728235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-12T03:16:25.795 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 2 2024-09-12T03:16:25.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:25.796+0000 7f0809b2a640 1 -- 172.21.15.27:0/2211411400 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f07e406be30 msgr2=0x7f07e406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:25.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:25.796+0000 7f0809b2a640 1 --2- 172.21.15.27:0/2211411400 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f07e406be30 0x7f07e406e2f0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f07ec034000 tx=0x7f07ec02e000 comp rx=0 tx=0).stop 2024-09-12T03:16:25.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:25.796+0000 7f0809b2a640 1 -- 172.21.15.27:0/2211411400 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f08040d5200 msgr2=0x7f080414bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:25.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:25.796+0000 7f0809b2a640 1 --2- 172.21.15.27:0/2211411400 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f08040d5200 0x7f080414bc90 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f07f400c460 tx=0x7f07f400c920 comp rx=0 tx=0).stop 2024-09-12T03:16:25.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:25.796+0000 7f0809b2a640 1 -- 172.21.15.27:0/2211411400 shutdown_connections 2024-09-12T03:16:25.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:25.796+0000 7f0809b2a640 1 --2- 172.21.15.27:0/2211411400 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f07e406be30 0x7f07e406e2f0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:25.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:25.796+0000 7f0809b2a640 1 --2- 172.21.15.27:0/2211411400 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f080415eb90 0x7f0804157690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:25.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:25.796+0000 7f0809b2a640 1 --2- 172.21.15.27:0/2211411400 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f08040d5200 0x7f080414bc90 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:25.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:25.796+0000 7f0809b2a640 1 --2- 172.21.15.27:0/2211411400 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f08040d1720 0x7f08041566a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:25.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:25.796+0000 7f0809b2a640 1 -- 172.21.15.27:0/2211411400 >> 172.21.15.27:0/2211411400 conn(0x7f080406dce0 msgr2=0x7f08040b98d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:25.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:25.797+0000 7f0809b2a640 1 -- 172.21.15.27:0/2211411400 shutdown_connections 2024-09-12T03:16:25.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:25.797+0000 7f0809b2a640 1 -- 172.21.15.27:0/2211411400 wait complete. 2024-09-12T03:16:27.808 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-12T03:16:27.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:27.922+0000 7f54c488d640 1 Processor -- start 2024-09-12T03:16:27.923 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:27.923+0000 7f54c488d640 1 -- start start 2024-09-12T03:16:27.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:27.924+0000 7f54c488d640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f54c015c6c0 0x7f54c0073210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:27.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:27.924+0000 7f54c488d640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f54c00cfaf0 0x7f54c0073750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:27.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:27.924+0000 7f54c488d640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f54c00cf8a0 con 0x7f54c015c2e0 2024-09-12T03:16:27.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:27.924+0000 7f54c488d640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f54c0160eb0 con 0x7f54c00cfaf0 2024-09-12T03:16:27.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:27.924+0000 7f54c488d640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f54c01600f0 con 0x7f54c015c6c0 2024-09-12T03:16:27.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:27.924+0000 7f54bed9d640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f54c00cfaf0 0x7f54c0073750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:27.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:27.924+0000 7f54be59c640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f54c015c2e0 0x7f54c00cf070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:51486/0 (socket says 172.21.15.27:51486) 2024-09-12T03:16:27.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:27.924+0000 7f54bed9d640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f54c00cfaf0 0x7f54c0073750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36534/0 (socket says 172.21.15.27:36534) 2024-09-12T03:16:27.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:27.924+0000 7f54be59c640 1 -- 172.21.15.27:0/364249219 learned_addr learned my addr 172.21.15.27:0/364249219 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:16:27.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:27.924+0000 7f54bdd9b640 1 --2- 172.21.15.27:0/364249219 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f54c015c6c0 0x7f54c0073210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:27.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:27.924+0000 7f54bed9d640 1 -- 172.21.15.27:0/364249219 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f54c015c6c0 msgr2=0x7f54c0073210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:27.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:27.924+0000 7f54bed9d640 1 --2- 172.21.15.27:0/364249219 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f54c015c6c0 0x7f54c0073210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:27.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:27.924+0000 7f54bed9d640 1 -- 172.21.15.27:0/364249219 >> v1:172.21.15.27:6789/0 conn(0x7f54c015c2e0 legacy=0x7f54c00cf070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:27.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:27.924+0000 7f54bed9d640 1 -- 172.21.15.27:0/364249219 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54c01603d0 con 0x7f54c00cfaf0 2024-09-12T03:16:27.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:27.924+0000 7f54bd59a640 1 -- 172.21.15.27:0/364249219 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3559374901 0 0) 0x7f54c00cf8a0 con 0x7f54c015c2e0 2024-09-12T03:16:27.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:27.925+0000 7f54bed9d640 1 --2- 172.21.15.27:0/364249219 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f54c00cfaf0 0x7f54c0073750 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f54b8009d90 tx=0x7f54b802a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc512f4fbab10f9b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:27.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:27.925+0000 7f54bd59a640 1 -- 172.21.15.27:0/364249219 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54b8031600 con 0x7f54c00cfaf0 2024-09-12T03:16:27.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:27.925+0000 7f54bd59a640 1 -- 172.21.15.27:0/364249219 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54b80317a0 con 0x7f54c00cfaf0 2024-09-12T03:16:27.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:27.925+0000 7f54bd59a640 1 -- 172.21.15.27:0/364249219 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54b8031b70 con 0x7f54c00cfaf0 2024-09-12T03:16:27.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:27.926+0000 7f54c488d640 1 -- 172.21.15.27:0/364249219 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f54c00cfaf0 msgr2=0x7f54c0073750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:27.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:27.926+0000 7f54c488d640 1 --2- 172.21.15.27:0/364249219 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f54c00cfaf0 0x7f54c0073750 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f54b8009d90 tx=0x7f54b802a5d0 comp rx=0 tx=0).stop 2024-09-12T03:16:27.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:27.926+0000 7f54c488d640 1 -- 172.21.15.27:0/364249219 shutdown_connections 2024-09-12T03:16:27.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:27.926+0000 7f54c488d640 1 --2- 172.21.15.27:0/364249219 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f54c00cfaf0 0x7f54c0073750 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:27.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:27.926+0000 7f54c488d640 1 --2- 172.21.15.27:0/364249219 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f54c015c6c0 0x7f54c0073210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:27.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:27.926+0000 7f54c488d640 1 -- 172.21.15.27:0/364249219 >> 172.21.15.27:0/364249219 conn(0x7f54c006f2c0 msgr2=0x7f54c006f6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:27.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:27.926+0000 7f54c488d640 1 -- 172.21.15.27:0/364249219 shutdown_connections 2024-09-12T03:16:27.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:27.926+0000 7f54c488d640 1 -- 172.21.15.27:0/364249219 wait complete. 2024-09-12T03:16:27.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:27.927+0000 7f54c488d640 1 Processor -- start 2024-09-12T03:16:27.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:27.927+0000 7f54c488d640 1 -- start start 2024-09-12T03:16:27.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:27.928+0000 7f54c488d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f54c00cfaf0 0x7f54c01f2800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:27.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:27.928+0000 7f54c488d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f54c015c2e0 0x7f54c0154d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:27.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:27.928+0000 7f54c488d640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f54c015c6c0 0x7f54c015aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:27.930 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:27.928+0000 7f54c488d640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f54c00b5d40 con 0x7f54c015c2e0 2024-09-12T03:16:27.930 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:27.928+0000 7f54c488d640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f54c00b5bc0 con 0x7f54c00cfaf0 2024-09-12T03:16:27.930 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:27.928+0000 7f54c488d640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f54c00b5ec0 con 0x7f54c015c6c0 2024-09-12T03:16:27.930 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:27.928+0000 7f54bdd9b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f54c015c2e0 0x7f54c0154d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:27.930 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:27.928+0000 7f54bdd9b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f54c015c2e0 0x7f54c0154d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36548/0 (socket says 172.21.15.27:36548) 2024-09-12T03:16:27.930 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:27.928+0000 7f54bdd9b640 1 -- 172.21.15.27:0/3610642266 learned_addr learned my addr 172.21.15.27:0/3610642266 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:16:27.930 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:27.928+0000 7f54bed9d640 1 --2- 172.21.15.27:0/3610642266 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f54c015c6c0 0x7f54c015aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:27.930 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:27.928+0000 7f54be59c640 1 --2- 172.21.15.27:0/3610642266 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f54c00cfaf0 0x7f54c01f2800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:27.930 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:27.928+0000 7f54bdd9b640 1 -- 172.21.15.27:0/3610642266 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f54c015c6c0 msgr2=0x7f54c015aec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:27.930 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:27.928+0000 7f54bdd9b640 1 --2- 172.21.15.27:0/3610642266 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f54c015c6c0 0x7f54c015aec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:27.930 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:27.928+0000 7f54bdd9b640 1 -- 172.21.15.27:0/3610642266 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f54c00cfaf0 msgr2=0x7f54c01f2800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:27.930 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:27.928+0000 7f54bdd9b640 1 --2- 172.21.15.27:0/3610642266 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f54c00cfaf0 0x7f54c01f2800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:27.930 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:27.929+0000 7f54bdd9b640 1 -- 172.21.15.27:0/3610642266 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54c00ac390 con 0x7f54c015c2e0 2024-09-12T03:16:27.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:27.929+0000 7f54be59c640 1 --2- 172.21.15.27:0/3610642266 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f54c00cfaf0 0x7f54c01f2800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:16:27.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:27.929+0000 7f54bdd9b640 1 --2- 172.21.15.27:0/3610642266 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f54c015c2e0 0x7f54c0154d50 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f54b400e870 tx=0x7f54b400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:27.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:27.929+0000 7f54aeffd640 1 -- 172.21.15.27:0/3610642266 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54b4017070 con 0x7f54c015c2e0 2024-09-12T03:16:27.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:27.929+0000 7f54aeffd640 1 -- 172.21.15.27:0/3610642266 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54b400c940 con 0x7f54c015c2e0 2024-09-12T03:16:27.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:27.929+0000 7f54c488d640 1 -- 172.21.15.27:0/3610642266 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54c01f6410 con 0x7f54c015c2e0 2024-09-12T03:16:27.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:27.929+0000 7f54aeffd640 1 -- 172.21.15.27:0/3610642266 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54b400cd30 con 0x7f54c015c2e0 2024-09-12T03:16:27.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:27.929+0000 7f54c488d640 1 -- 172.21.15.27:0/3610642266 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54c01f6cb0 con 0x7f54c015c2e0 2024-09-12T03:16:27.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:27.930+0000 7f54c488d640 1 -- 172.21.15.27:0/3610642266 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54c01f6800 con 0x7f54c015c2e0 2024-09-12T03:16:27.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:27.930+0000 7f54aeffd640 1 -- 172.21.15.27:0/3610642266 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f54b401a070 con 0x7f54c015c2e0 2024-09-12T03:16:27.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:27.931+0000 7f54aeffd640 1 --2- 172.21.15.27:0/3610642266 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f54a006be60 0x7f54a006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:27.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:27.931+0000 7f54aeffd640 1 -- 172.21.15.27:0/3610642266 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7f54b408a930 con 0x7f54c015c2e0 2024-09-12T03:16:27.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:27.931+0000 7f54be59c640 1 --2- 172.21.15.27:0/3610642266 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f54a006be60 0x7f54a006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:27.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:27.932+0000 7f54be59c640 1 --2- 172.21.15.27:0/3610642266 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f54a006be60 0x7f54a006e320 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f54b0000f60 tx=0x7f54b0009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:27.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:27.935+0000 7f54aeffd640 1 -- 172.21.15.27:0/3610642266 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f54b40596c0 con 0x7f54c015c2e0 2024-09-12T03:16:28.147 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:28.147+0000 7f54c488d640 1 -- 172.21.15.27:0/3610642266 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f54c006fbd0 con 0x7f54c015c2e0 2024-09-12T03:16:28.148 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:28.148+0000 7f54aeffd640 1 -- 172.21.15.27:0/3610642266 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+3903 (secure 0 0 0) 0x7f54c006fbd0 con 0x7f54c015c2e0 2024-09-12T03:16:28.148 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:16:28.148 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":3,"btime":"2024-09-12T03:16:26:539170+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6836/682326693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":682326693},{"type":"v1","addr":"172.21.15.45:6836","nonce":682326693}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4278,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6834/4084728235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":4084728235},{"type":"v1","addr":"172.21.15.27:6834","nonce":4084728235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4281,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/2844240887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":2844240887},{"type":"v1","addr":"172.21.15.27:6835","nonce":2844240887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/1035926332","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6835","nonce":1035926332},{"type":"v1","addr":"172.21.15.45:6837","nonce":1035926332}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-09-12T03:16:28.148 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 3 2024-09-12T03:16:28.150 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:28.150+0000 7f54c488d640 1 -- 172.21.15.27:0/3610642266 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f54a006be60 msgr2=0x7f54a006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:28.150 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:28.150+0000 7f54c488d640 1 --2- 172.21.15.27:0/3610642266 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f54a006be60 0x7f54a006e320 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f54b0000f60 tx=0x7f54b0009040 comp rx=0 tx=0).stop 2024-09-12T03:16:28.150 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:28.150+0000 7f54c488d640 1 -- 172.21.15.27:0/3610642266 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f54c015c2e0 msgr2=0x7f54c0154d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:28.150 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:28.150+0000 7f54c488d640 1 --2- 172.21.15.27:0/3610642266 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f54c015c2e0 0x7f54c0154d50 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f54b400e870 tx=0x7f54b400ed30 comp rx=0 tx=0).stop 2024-09-12T03:16:28.150 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:28.150+0000 7f54c488d640 1 -- 172.21.15.27:0/3610642266 shutdown_connections 2024-09-12T03:16:28.150 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:28.150+0000 7f54c488d640 1 --2- 172.21.15.27:0/3610642266 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f54a006be60 0x7f54a006e320 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:28.150 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:28.150+0000 7f54c488d640 1 --2- 172.21.15.27:0/3610642266 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f54c015c6c0 0x7f54c015aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:28.151 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:28.150+0000 7f54c488d640 1 --2- 172.21.15.27:0/3610642266 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f54c015c2e0 0x7f54c0154d50 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:28.151 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:28.150+0000 7f54c488d640 1 --2- 172.21.15.27:0/3610642266 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f54c00cfaf0 0x7f54c01f2800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:28.151 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:28.150+0000 7f54c488d640 1 -- 172.21.15.27:0/3610642266 >> 172.21.15.27:0/3610642266 conn(0x7f54c006f2c0 msgr2=0x7f54c0157430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:28.151 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:28.150+0000 7f54c488d640 1 -- 172.21.15.27:0/3610642266 shutdown_connections 2024-09-12T03:16:28.151 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:28.150+0000 7f54c488d640 1 -- 172.21.15.27:0/3610642266 wait complete. 2024-09-12T03:16:28.160 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-12T03:16:28.262 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:28.262+0000 7f183cf54640 1 Processor -- start 2024-09-12T03:16:28.262 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:28.262+0000 7f183cf54640 1 -- start start 2024-09-12T03:16:28.265 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:28.263+0000 7f183cf54640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f183815eb00 0x7f18380cede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:28.266 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:28.263+0000 7f183cf54640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f18380d3dd0 0x7f18380b75e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:28.266 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:28.263+0000 7f183cf54640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f18380cf320 con 0x7f18380b7b20 2024-09-12T03:16:28.266 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:28.263+0000 7f183cf54640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f18380cfb80 con 0x7f183815eb00 2024-09-12T03:16:28.266 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:28.263+0000 7f183cf54640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f18380cf5c0 con 0x7f18380d3dd0 2024-09-12T03:16:28.266 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:28.263+0000 7f1836d9d640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f18380b7b20 0x7f18380b3500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:51494/0 (socket says 172.21.15.27:51494) 2024-09-12T03:16:28.266 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:28.263+0000 7f1836d9d640 1 -- 172.21.15.27:0/1233390502 learned_addr learned my addr 172.21.15.27:0/1233390502 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:16:28.266 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:28.263+0000 7f183659c640 1 --2- 172.21.15.27:0/1233390502 >> v2:172.21.15.45:3300/0 conn(0x7f183815eb00 0x7f18380cede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:28.266 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:28.263+0000 7f1835d9b640 1 --2- 172.21.15.27:0/1233390502 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f18380d3dd0 0x7f18380b75e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:28.266 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:28.263+0000 7f183559a640 1 -- 172.21.15.27:0/1233390502 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2764212918 0 0) 0x7f18380cf320 con 0x7f18380b7b20 2024-09-12T03:16:28.266 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:28.263+0000 7f183559a640 1 -- 172.21.15.27:0/1233390502 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1824003520 con 0x7f18380b7b20 2024-09-12T03:16:28.266 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:28.263+0000 7f183559a640 1 -- 172.21.15.27:0/1233390502 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3430155146 0 0) 0x7f1824003520 con 0x7f18380b7b20 2024-09-12T03:16:28.266 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:28.264+0000 7f183559a640 1 -- 172.21.15.27:0/1233390502 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f18380d3dd0 msgr2=0x7f18380b75e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:28.266 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:28.264+0000 7f183559a640 1 --2- 172.21.15.27:0/1233390502 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f18380d3dd0 0x7f18380b75e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:28.267 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:28.264+0000 7f183559a640 1 -- 172.21.15.27:0/1233390502 >> v2:172.21.15.45:3300/0 conn(0x7f183815eb00 msgr2=0x7f18380cede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:28.267 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:28.264+0000 7f183559a640 1 --2- 172.21.15.27:0/1233390502 >> v2:172.21.15.45:3300/0 conn(0x7f183815eb00 0x7f18380cede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:28.267 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:28.264+0000 7f183559a640 1 -- 172.21.15.27:0/1233390502 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18380cf8a0 con 0x7f18380b7b20 2024-09-12T03:16:28.267 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:28.264+0000 7f183559a640 1 -- 172.21.15.27:0/1233390502 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 847755213 0 0) 0x7f18300019f0 con 0x7f18380b7b20 2024-09-12T03:16:28.267 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:28.264+0000 7f183559a640 1 -- 172.21.15.27:0/1233390502 >> v1:172.21.15.27:6789/0 conn(0x7f18380b7b20 legacy=0x7f18380b3500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:28.267 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:28.264+0000 7f183559a640 1 --2- 172.21.15.27:0/1233390502 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f18240039b0 0x7f1824003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:28.267 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:28.264+0000 7f183559a640 1 --2- 172.21.15.27:0/1233390502 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1824004800 0x7f1824004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:28.267 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:28.264+0000 7f183559a640 1 --2- 172.21.15.27:0/1233390502 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f18380d3dd0 0x7f18240051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:28.267 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:28.264+0000 7f183559a640 1 -- 172.21.15.27:0/1233390502 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f18380cfb80 con 0x7f18240039b0 2024-09-12T03:16:28.267 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:28.264+0000 7f183559a640 1 -- 172.21.15.27:0/1233390502 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f18380cf5c0 con 0x7f1824004800 2024-09-12T03:16:28.267 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:28.264+0000 7f183559a640 1 -- 172.21.15.27:0/1233390502 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f182400b9c0 con 0x7f18380d3dd0 2024-09-12T03:16:28.267 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:28.264+0000 7f183659c640 1 --2- 172.21.15.27:0/1233390502 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f18240039b0 0x7f1824003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:28.267 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:28.264+0000 7f1835d9b640 1 --2- 172.21.15.27:0/1233390502 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1824004800 0x7f1824004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:28.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:28.264+0000 7f1836d9d640 1 --2- 172.21.15.27:0/1233390502 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f18380d3dd0 0x7f18240051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:28.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:28.264+0000 7f183659c640 1 -- 172.21.15.27:0/1233390502 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f18380d3dd0 msgr2=0x7f18240051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:28.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:28.264+0000 7f183659c640 1 --2- 172.21.15.27:0/1233390502 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f18380d3dd0 0x7f18240051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:28.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:28.264+0000 7f183659c640 1 -- 172.21.15.27:0/1233390502 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1824004800 msgr2=0x7f1824004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:28.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:28.264+0000 7f183659c640 1 --2- 172.21.15.27:0/1233390502 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1824004800 0x7f1824004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:28.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:28.264+0000 7f183659c640 1 -- 172.21.15.27:0/1233390502 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1824003520 con 0x7f18240039b0 2024-09-12T03:16:28.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:28.265+0000 7f183659c640 1 --2- 172.21.15.27:0/1233390502 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f18240039b0 0x7f1824003dd0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f1828009ac0 tx=0x7f1824005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9d44c68ba7a4b19 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:28.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:28.265+0000 7f183559a640 1 -- 172.21.15.27:0/1233390502 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1828036070 con 0x7f18240039b0 2024-09-12T03:16:28.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:28.265+0000 7f183559a640 1 -- 172.21.15.27:0/1233390502 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1828031400 con 0x7f18240039b0 2024-09-12T03:16:28.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:28.265+0000 7f183559a640 1 -- 172.21.15.27:0/1233390502 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1828009690 con 0x7f18240039b0 2024-09-12T03:16:28.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:28.265+0000 7f183cf54640 1 -- 172.21.15.27:0/1233390502 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f18240039b0 msgr2=0x7f1824003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:28.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:28.265+0000 7f183cf54640 1 --2- 172.21.15.27:0/1233390502 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f18240039b0 0x7f1824003dd0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f1828009ac0 tx=0x7f1824005830 comp rx=0 tx=0).stop 2024-09-12T03:16:28.269 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:28.265+0000 7f1836d9d640 1 -- 172.21.15.27:0/1233390502 reap_dead start 2024-09-12T03:16:28.269 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:28.265+0000 7f183cf54640 1 -- 172.21.15.27:0/1233390502 shutdown_connections 2024-09-12T03:16:28.269 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:28.265+0000 7f183cf54640 1 -- 172.21.15.27:0/1233390502 >> 172.21.15.27:0/1233390502 conn(0x7f18380d4790 msgr2=0x7f18380d4ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:28.269 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:28.265+0000 7f183cf54640 1 -- 172.21.15.27:0/1233390502 shutdown_connections 2024-09-12T03:16:28.269 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:28.265+0000 7f183cf54640 1 -- 172.21.15.27:0/1233390502 wait complete. 2024-09-12T03:16:28.272 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:28.267+0000 7f183cf54640 1 Processor -- start 2024-09-12T03:16:28.272 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:28.267+0000 7f183cf54640 1 -- start start 2024-09-12T03:16:28.272 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:28.268+0000 7f183cf54640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f18240039b0 0x7f1838152230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:28.272 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:28.268+0000 7f183cf54640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f18380ca660 0x7f18380c82c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:28.272 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:28.268+0000 7f183cf54640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f18380cda30 0x7f18380cdee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:28.272 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:28.268+0000 7f183cf54640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f18380b3c10 con 0x7f18240039b0 2024-09-12T03:16:28.272 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:28.268+0000 7f183cf54640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f18380d2cf0 con 0x7f18380cda30 2024-09-12T03:16:28.272 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:28.268+0000 7f183cf54640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f18380b3d90 con 0x7f18380ca660 2024-09-12T03:16:28.272 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:28.268+0000 7f183659c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f18240039b0 0x7f1838152230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:28.272 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:28.268+0000 7f183659c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f18240039b0 0x7f1838152230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36572/0 (socket says 172.21.15.27:36572) 2024-09-12T03:16:28.272 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:28.268+0000 7f183659c640 1 -- 172.21.15.27:0/3984775904 learned_addr learned my addr 172.21.15.27:0/3984775904 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:16:28.272 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:28.268+0000 7f1835d9b640 1 --2- 172.21.15.27:0/3984775904 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f18380ca660 0x7f18380c82c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:28.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:28.268+0000 7f1836d9d640 1 --2- 172.21.15.27:0/3984775904 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f18380cda30 0x7f18380cdee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:28.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:28.268+0000 7f183659c640 1 -- 172.21.15.27:0/3984775904 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f18380ca660 msgr2=0x7f18380c82c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:28.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:28.268+0000 7f183659c640 1 --2- 172.21.15.27:0/3984775904 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f18380ca660 0x7f18380c82c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:28.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:28.268+0000 7f183659c640 1 -- 172.21.15.27:0/3984775904 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f18380cda30 msgr2=0x7f18380cdee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:28.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:28.268+0000 7f183659c640 1 --2- 172.21.15.27:0/3984775904 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f18380cda30 0x7f18380cdee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:28.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:28.268+0000 7f183659c640 1 -- 172.21.15.27:0/3984775904 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18380c8a30 con 0x7f18240039b0 2024-09-12T03:16:28.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:28.268+0000 7f1836d9d640 1 --2- 172.21.15.27:0/3984775904 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f18380cda30 0x7f18380cdee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:16:28.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:28.268+0000 7f183659c640 1 --2- 172.21.15.27:0/3984775904 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f18240039b0 0x7f1838152230 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f1828034040 tx=0x7f1828004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:28.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:28.268+0000 7f182effd640 1 -- 172.21.15.27:0/3984775904 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1828040020 con 0x7f18240039b0 2024-09-12T03:16:28.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:28.268+0000 7f182effd640 1 -- 172.21.15.27:0/3984775904 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1828003950 con 0x7f18240039b0 2024-09-12T03:16:28.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:28.268+0000 7f183cf54640 1 -- 172.21.15.27:0/3984775904 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18380c8d10 con 0x7f18240039b0 2024-09-12T03:16:28.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:28.269+0000 7f182effd640 1 -- 172.21.15.27:0/3984775904 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1828036040 con 0x7f18240039b0 2024-09-12T03:16:28.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:28.269+0000 7f183cf54640 1 -- 172.21.15.27:0/3984775904 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18380b4220 con 0x7f18240039b0 2024-09-12T03:16:28.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:28.269+0000 7f183cf54640 1 -- 172.21.15.27:0/3984775904 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18380c9230 con 0x7f18240039b0 2024-09-12T03:16:28.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:28.270+0000 7f182effd640 1 -- 172.21.15.27:0/3984775904 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1828043020 con 0x7f18240039b0 2024-09-12T03:16:28.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:28.270+0000 7f182effd640 1 --2- 172.21.15.27:0/3984775904 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f181806be60 0x7f181806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:28.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:28.270+0000 7f182effd640 1 -- 172.21.15.27:0/3984775904 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7f18280aaa80 con 0x7f18240039b0 2024-09-12T03:16:28.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:28.271+0000 7f1835d9b640 1 --2- 172.21.15.27:0/3984775904 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f181806be60 0x7f181806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:28.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:28.271+0000 7f1835d9b640 1 --2- 172.21.15.27:0/3984775904 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f181806be60 0x7f181806e320 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f18380d3c80 tx=0x7f1820008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:28.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:28.275+0000 7f182effd640 1 -- 172.21.15.27:0/3984775904 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f182804a070 con 0x7f18240039b0 2024-09-12T03:16:28.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:28.453+0000 7f183cf54640 1 -- 172.21.15.27:0/3984775904 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f18380b75e0 con 0x7f18240039b0 2024-09-12T03:16:28.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:28.454+0000 7f182effd640 1 -- 172.21.15.27:0/3984775904 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v17) ==== 84+0+65 (secure 0 0 0) 0x7f18280791c0 con 0x7f18240039b0 2024-09-12T03:16:28.454 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:16:28.454 INFO:teuthology.orchestra.run.smithi027.stdout:[ 2024-09-12T03:16:28.454 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:16:28.454 INFO:teuthology.orchestra.run.smithi027.stdout: "poolnum": 1, 2024-09-12T03:16:28.454 INFO:teuthology.orchestra.run.smithi027.stdout: "poolname": ".mgr" 2024-09-12T03:16:28.454 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:16:28.454 INFO:teuthology.orchestra.run.smithi027.stdout:] 2024-09-12T03:16:28.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:28.456+0000 7f183cf54640 1 -- 172.21.15.27:0/3984775904 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f181806be60 msgr2=0x7f181806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:28.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:28.456+0000 7f183cf54640 1 --2- 172.21.15.27:0/3984775904 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f181806be60 0x7f181806e320 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f18380d3c80 tx=0x7f1820008040 comp rx=0 tx=0).stop 2024-09-12T03:16:28.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:28.456+0000 7f183cf54640 1 -- 172.21.15.27:0/3984775904 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f18240039b0 msgr2=0x7f1838152230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:28.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:28.456+0000 7f183cf54640 1 --2- 172.21.15.27:0/3984775904 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f18240039b0 0x7f1838152230 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f1828034040 tx=0x7f1828004730 comp rx=0 tx=0).stop 2024-09-12T03:16:28.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:28.456+0000 7f183cf54640 1 -- 172.21.15.27:0/3984775904 shutdown_connections 2024-09-12T03:16:28.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:28.456+0000 7f183cf54640 1 --2- 172.21.15.27:0/3984775904 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f181806be60 0x7f181806e320 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:28.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:28.456+0000 7f183cf54640 1 --2- 172.21.15.27:0/3984775904 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f18380cda30 0x7f18380cdee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:28.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:28.456+0000 7f183cf54640 1 --2- 172.21.15.27:0/3984775904 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f18380ca660 0x7f18380c82c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:28.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:28.456+0000 7f183cf54640 1 --2- 172.21.15.27:0/3984775904 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f18240039b0 0x7f1838152230 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:28.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:28.456+0000 7f183cf54640 1 -- 172.21.15.27:0/3984775904 >> 172.21.15.27:0/3984775904 conn(0x7f18380d4790 msgr2=0x7f183815dc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:28.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:28.456+0000 7f183cf54640 1 -- 172.21.15.27:0/3984775904 shutdown_connections 2024-09-12T03:16:28.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:28.456+0000 7f183cf54640 1 -- 172.21.15.27:0/3984775904 wait complete. 2024-09-12T03:16:28.466 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-12T03:16:28.467 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-12T03:16:28.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:28.573+0000 7faf214eb640 1 Processor -- start 2024-09-12T03:16:28.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:28.573+0000 7faf214eb640 1 -- start start 2024-09-12T03:16:28.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:28.574+0000 7faf214eb640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7faf1c15ec30 0x7faf1c0b3900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:28.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:28.574+0000 7faf214eb640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7faf1c06da00 0x7faf1c06deb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:28.577 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:28.574+0000 7faf214eb640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faf1c0ce5e0 con 0x7faf1c0af9d0 2024-09-12T03:16:28.577 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:28.574+0000 7faf214eb640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7faf1c0d02d0 con 0x7faf1c06da00 2024-09-12T03:16:28.577 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:28.574+0000 7faf214eb640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7faf1c06c650 con 0x7faf1c15ec30 2024-09-12T03:16:28.577 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:28.574+0000 7faf1a7fc640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7faf1c0af9d0 0x7faf1c0b3e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:51498/0 (socket says 172.21.15.27:51498) 2024-09-12T03:16:28.577 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:28.574+0000 7faf1a7fc640 1 -- 172.21.15.27:0/967570056 learned_addr learned my addr 172.21.15.27:0/967570056 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:16:28.577 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:28.574+0000 7faf19ffb640 1 -- 172.21.15.27:0/967570056 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2830533384 0 0) 0x7faf1c0ce5e0 con 0x7faf1c0af9d0 2024-09-12T03:16:28.577 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:28.574+0000 7faf19ffb640 1 -- 172.21.15.27:0/967570056 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7faf04003520 con 0x7faf1c0af9d0 2024-09-12T03:16:28.577 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:28.574+0000 7faf1affd640 1 --2- 172.21.15.27:0/967570056 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7faf1c15ec30 0x7faf1c0b3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:28.577 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:28.574+0000 7faf1b7fe640 1 --2- 172.21.15.27:0/967570056 >> v2:172.21.15.45:3300/0 conn(0x7faf1c06da00 0x7faf1c06deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:28.577 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:28.574+0000 7faf19ffb640 1 -- 172.21.15.27:0/967570056 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 519817516 0 0) 0x7faf04003520 con 0x7faf1c0af9d0 2024-09-12T03:16:28.577 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:28.574+0000 7faf19ffb640 1 -- 172.21.15.27:0/967570056 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7faf1c15ec30 msgr2=0x7faf1c0b3900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:28.577 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:28.574+0000 7faf19ffb640 1 --2- 172.21.15.27:0/967570056 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7faf1c15ec30 0x7faf1c0b3900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:28.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:28.574+0000 7faf19ffb640 1 -- 172.21.15.27:0/967570056 >> v2:172.21.15.45:3300/0 conn(0x7faf1c06da00 msgr2=0x7faf1c06deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:28.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:28.574+0000 7faf19ffb640 1 --2- 172.21.15.27:0/967570056 >> v2:172.21.15.45:3300/0 conn(0x7faf1c06da00 0x7faf1c06deb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:28.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:28.574+0000 7faf19ffb640 1 -- 172.21.15.27:0/967570056 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf1c0ceac0 con 0x7faf1c0af9d0 2024-09-12T03:16:28.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:28.574+0000 7faf19ffb640 1 -- 172.21.15.27:0/967570056 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 847755213 0 0) 0x7faf10001670 con 0x7faf1c0af9d0 2024-09-12T03:16:28.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:28.574+0000 7faf1b7fe640 1 --2- 172.21.15.27:0/967570056 >> v2:172.21.15.45:3300/0 conn(0x7faf1c06da00 0x7faf1c06deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:16:28.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:28.574+0000 7faf19ffb640 1 -- 172.21.15.27:0/967570056 >> v1:172.21.15.27:6789/0 conn(0x7faf1c0af9d0 legacy=0x7faf1c0b3e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:28.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:28.575+0000 7faf19ffb640 1 --2- 172.21.15.27:0/967570056 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faf040039b0 0x7faf04003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:28.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:28.575+0000 7faf19ffb640 1 --2- 172.21.15.27:0/967570056 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7faf1c15ec30 0x7faf04004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:28.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:28.575+0000 7faf19ffb640 1 --2- 172.21.15.27:0/967570056 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faf04004d40 0x7faf040051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:28.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:28.575+0000 7faf19ffb640 1 -- 172.21.15.27:0/967570056 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7faf1c0d02d0 con 0x7faf040039b0 2024-09-12T03:16:28.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:28.575+0000 7faf19ffb640 1 -- 172.21.15.27:0/967570056 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7faf1c06c650 con 0x7faf04004d40 2024-09-12T03:16:28.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:28.575+0000 7faf19ffb640 1 -- 172.21.15.27:0/967570056 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7faf0400dbd0 con 0x7faf1c15ec30 2024-09-12T03:16:28.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:28.575+0000 7faf1affd640 1 --2- 172.21.15.27:0/967570056 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faf040039b0 0x7faf04003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:28.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:28.575+0000 7faf1a7fc640 1 --2- 172.21.15.27:0/967570056 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7faf1c15ec30 0x7faf04004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:28.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:28.575+0000 7faf1b7fe640 1 --2- 172.21.15.27:0/967570056 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faf04004d40 0x7faf040051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:28.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:28.575+0000 7faf1affd640 1 -- 172.21.15.27:0/967570056 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7faf1c15ec30 msgr2=0x7faf04004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:28.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:28.575+0000 7faf1affd640 1 --2- 172.21.15.27:0/967570056 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7faf1c15ec30 0x7faf04004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:28.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:28.575+0000 7faf1affd640 1 -- 172.21.15.27:0/967570056 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faf04004d40 msgr2=0x7faf040051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:28.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:28.575+0000 7faf1affd640 1 --2- 172.21.15.27:0/967570056 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faf04004d40 0x7faf040051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:28.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:28.575+0000 7faf1affd640 1 -- 172.21.15.27:0/967570056 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf04003520 con 0x7faf040039b0 2024-09-12T03:16:28.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:28.576+0000 7faf1affd640 1 --2- 172.21.15.27:0/967570056 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faf040039b0 0x7faf04003dd0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7faf0c009af0 tx=0x7faf04005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37c2166aef93f86c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:28.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:28.576+0000 7faf19ffb640 1 -- 172.21.15.27:0/967570056 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf0c036070 con 0x7faf040039b0 2024-09-12T03:16:28.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:28.576+0000 7faf19ffb640 1 -- 172.21.15.27:0/967570056 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf0c031400 con 0x7faf040039b0 2024-09-12T03:16:28.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:28.576+0000 7faf19ffb640 1 -- 172.21.15.27:0/967570056 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf0c009690 con 0x7faf040039b0 2024-09-12T03:16:28.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:28.576+0000 7faf214eb640 1 -- 172.21.15.27:0/967570056 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faf040039b0 msgr2=0x7faf04003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:28.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:28.576+0000 7faf214eb640 1 --2- 172.21.15.27:0/967570056 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faf040039b0 0x7faf04003dd0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7faf0c009af0 tx=0x7faf04005830 comp rx=0 tx=0).stop 2024-09-12T03:16:28.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:28.576+0000 7faf1b7fe640 1 -- 172.21.15.27:0/967570056 reap_dead start 2024-09-12T03:16:28.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:28.576+0000 7faf214eb640 1 -- 172.21.15.27:0/967570056 shutdown_connections 2024-09-12T03:16:28.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:28.576+0000 7faf214eb640 1 -- 172.21.15.27:0/967570056 >> 172.21.15.27:0/967570056 conn(0x7faf1c06bb30 msgr2=0x7faf1c06bf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:28.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:28.576+0000 7faf214eb640 1 -- 172.21.15.27:0/967570056 shutdown_connections 2024-09-12T03:16:28.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:28.576+0000 7faf214eb640 1 -- 172.21.15.27:0/967570056 wait complete. 2024-09-12T03:16:28.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:28.578+0000 7faf214eb640 1 Processor -- start 2024-09-12T03:16:28.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:28.578+0000 7faf214eb640 1 -- start start 2024-09-12T03:16:28.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:28.579+0000 7faf214eb640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7faf040039b0 0x7faf1c15ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:28.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:28.579+0000 7faf214eb640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faf1c14f140 0x7faf1c143ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:28.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:28.579+0000 7faf214eb640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faf1c1f0790 0x7faf1c157660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:28.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:28.579+0000 7faf214eb640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7faf1c0cf110 con 0x7faf1c14f140 2024-09-12T03:16:28.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:28.579+0000 7faf214eb640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7faf1c0cef90 con 0x7faf1c1f0790 2024-09-12T03:16:28.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:28.579+0000 7faf214eb640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7faf1c0cf290 con 0x7faf040039b0 2024-09-12T03:16:28.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:28.579+0000 7faf1a7fc640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faf1c14f140 0x7faf1c143ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:28.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:28.579+0000 7faf1a7fc640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faf1c14f140 0x7faf1c143ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36594/0 (socket says 172.21.15.27:36594) 2024-09-12T03:16:28.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:28.579+0000 7faf1a7fc640 1 -- 172.21.15.27:0/1333392125 learned_addr learned my addr 172.21.15.27:0/1333392125 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:16:28.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:28.579+0000 7faf1affd640 1 --2- 172.21.15.27:0/1333392125 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7faf040039b0 0x7faf1c15ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:28.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:28.579+0000 7faf1b7fe640 1 --2- 172.21.15.27:0/1333392125 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faf1c1f0790 0x7faf1c157660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:28.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:28.579+0000 7faf1a7fc640 1 -- 172.21.15.27:0/1333392125 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7faf040039b0 msgr2=0x7faf1c15ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:28.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:28.579+0000 7faf1a7fc640 1 --2- 172.21.15.27:0/1333392125 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7faf040039b0 0x7faf1c15ae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:28.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:28.579+0000 7faf1a7fc640 1 -- 172.21.15.27:0/1333392125 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faf1c1f0790 msgr2=0x7faf1c157660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:28.584 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:28.579+0000 7faf1a7fc640 1 --2- 172.21.15.27:0/1333392125 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faf1c1f0790 0x7faf1c157660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:28.584 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:28.579+0000 7faf1a7fc640 1 -- 172.21.15.27:0/1333392125 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf1c144430 con 0x7faf1c14f140 2024-09-12T03:16:28.584 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:28.579+0000 7faf1a7fc640 1 --2- 172.21.15.27:0/1333392125 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faf1c14f140 0x7faf1c143ef0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7faf100073a0 tx=0x7faf10007860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:28.584 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:28.579+0000 7faf037fe640 1 -- 172.21.15.27:0/1333392125 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf1001a070 con 0x7faf1c14f140 2024-09-12T03:16:28.584 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:28.579+0000 7faf037fe640 1 -- 172.21.15.27:0/1333392125 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf1000f8e0 con 0x7faf1c14f140 2024-09-12T03:16:28.584 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:28.579+0000 7faf037fe640 1 -- 172.21.15.27:0/1333392125 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf1000fcb0 con 0x7faf1c14f140 2024-09-12T03:16:28.584 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:28.579+0000 7faf214eb640 1 -- 172.21.15.27:0/1333392125 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf1c15b570 con 0x7faf1c14f140 2024-09-12T03:16:28.584 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:28.579+0000 7faf214eb640 1 -- 172.21.15.27:0/1333392125 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf1c15bb30 con 0x7faf1c14f140 2024-09-12T03:16:28.584 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:28.580+0000 7faf214eb640 1 -- 172.21.15.27:0/1333392125 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf1c0695a0 con 0x7faf1c14f140 2024-09-12T03:16:28.584 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:28.581+0000 7faf037fe640 1 -- 172.21.15.27:0/1333392125 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7faf1001d070 con 0x7faf1c14f140 2024-09-12T03:16:28.584 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:28.581+0000 7faf037fe640 1 --2- 172.21.15.27:0/1333392125 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7faf0806be30 0x7faf0806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:28.584 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:28.581+0000 7faf037fe640 1 -- 172.21.15.27:0/1333392125 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7faf1008a360 con 0x7faf1c14f140 2024-09-12T03:16:28.584 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:28.581+0000 7faf1affd640 1 --2- 172.21.15.27:0/1333392125 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7faf0806be30 0x7faf0806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:28.584 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:28.582+0000 7faf1affd640 1 --2- 172.21.15.27:0/1333392125 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7faf0806be30 0x7faf0806e2f0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7faf0c033000 tx=0x7faf0c008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:28.585 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:28.585+0000 7faf037fe640 1 -- 172.21.15.27:0/1333392125 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7faf10058aa0 con 0x7faf1c14f140 2024-09-12T03:16:28.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:28.760+0000 7faf214eb640 1 -- 172.21.15.27:0/1333392125 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7faf1c15f010 con 0x7faf1c14f140 2024-09-12T03:16:29.552 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:29.552+0000 7faf037fe640 1 -- 172.21.15.27:0/1333392125 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v18) ==== 138+0+0 (secure 0 0 0) 0x7faf1005cda0 con 0x7faf1c14f140 2024-09-12T03:16:29.552 INFO:teuthology.orchestra.run.smithi027.stderr:pool 'cephfs_metadata' created 2024-09-12T03:16:29.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:29.555+0000 7faf214eb640 1 -- 172.21.15.27:0/1333392125 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7faf0806be30 msgr2=0x7faf0806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:29.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:29.555+0000 7faf214eb640 1 --2- 172.21.15.27:0/1333392125 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7faf0806be30 0x7faf0806e2f0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7faf0c033000 tx=0x7faf0c008030 comp rx=0 tx=0).stop 2024-09-12T03:16:29.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:29.555+0000 7faf214eb640 1 -- 172.21.15.27:0/1333392125 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faf1c14f140 msgr2=0x7faf1c143ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:29.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:29.555+0000 7faf214eb640 1 --2- 172.21.15.27:0/1333392125 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faf1c14f140 0x7faf1c143ef0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7faf100073a0 tx=0x7faf10007860 comp rx=0 tx=0).stop 2024-09-12T03:16:29.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:29.555+0000 7faf214eb640 1 -- 172.21.15.27:0/1333392125 shutdown_connections 2024-09-12T03:16:29.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:29.555+0000 7faf214eb640 1 --2- 172.21.15.27:0/1333392125 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7faf0806be30 0x7faf0806e2f0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:29.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:29.555+0000 7faf214eb640 1 --2- 172.21.15.27:0/1333392125 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faf1c1f0790 0x7faf1c157660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:29.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:29.555+0000 7faf214eb640 1 --2- 172.21.15.27:0/1333392125 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faf1c14f140 0x7faf1c143ef0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:29.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:29.555+0000 7faf214eb640 1 --2- 172.21.15.27:0/1333392125 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7faf040039b0 0x7faf1c15ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:29.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:29.555+0000 7faf214eb640 1 -- 172.21.15.27:0/1333392125 >> 172.21.15.27:0/1333392125 conn(0x7faf1c06bb30 msgr2=0x7faf1c0cf650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:29.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:29.555+0000 7faf214eb640 1 -- 172.21.15.27:0/1333392125 shutdown_connections 2024-09-12T03:16:29.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:29.555+0000 7faf214eb640 1 -- 172.21.15.27:0/1333392125 wait complete. 2024-09-12T03:16:29.571 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-12T03:16:29.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:29.684+0000 7f7e5414b640 1 Processor -- start 2024-09-12T03:16:29.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:29.685+0000 7f7e5414b640 1 -- start start 2024-09-12T03:16:29.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:29.685+0000 7f7e5414b640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f7e4c0d1190 0x7f7e4c0d15e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:29.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:29.685+0000 7f7e5414b640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7e4c0d1c70 con 0x7f7e4c15e9c0 2024-09-12T03:16:29.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:29.686+0000 7f7e5414b640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7e4c0d1f50 con 0x7f7e4c06dda0 2024-09-12T03:16:29.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:29.686+0000 7f7e5414b640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f7e4c0d2230 con 0x7f7e4c0d1190 2024-09-12T03:16:29.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:29.686+0000 7f7e516e6640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f7e4c15e9c0 0x7f7e4c160170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:51378/0 (socket says 172.21.15.27:51378) 2024-09-12T03:16:29.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:29.686+0000 7f7e516e6640 1 -- 172.21.15.27:0/1122803697 learned_addr learned my addr 172.21.15.27:0/1122803697 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:16:29.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:29.686+0000 7f7e50ee5640 1 -- 172.21.15.27:0/1122803697 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4042106281 0 0) 0x7f7e4c0d1c70 con 0x7f7e4c15e9c0 2024-09-12T03:16:29.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:29.686+0000 7f7e526e8640 1 --2- 172.21.15.27:0/1122803697 >> v2:172.21.15.45:3300/0 conn(0x7f7e4c0d1190 0x7f7e4c0d15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:29.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:29.686+0000 7f7e50ee5640 1 -- 172.21.15.27:0/1122803697 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7e38003520 con 0x7f7e4c15e9c0 2024-09-12T03:16:29.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:29.686+0000 7f7e50ee5640 1 -- 172.21.15.27:0/1122803697 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3386267322 0 0) 0x7f7e38003520 con 0x7f7e4c15e9c0 2024-09-12T03:16:29.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:29.686+0000 7f7e50ee5640 1 -- 172.21.15.27:0/1122803697 >> v2:172.21.15.45:3300/0 conn(0x7f7e4c0d1190 msgr2=0x7f7e4c0d15e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:29.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:29.686+0000 7f7e50ee5640 1 --2- 172.21.15.27:0/1122803697 >> v2:172.21.15.45:3300/0 conn(0x7f7e4c0d1190 0x7f7e4c0d15e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:29.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:29.686+0000 7f7e50ee5640 1 -- 172.21.15.27:0/1122803697 >> v1:172.21.15.45:6789/0 conn(0x7f7e4c06dda0 legacy=0x7f7e4c06e180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:29.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:29.686+0000 7f7e50ee5640 1 -- 172.21.15.27:0/1122803697 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e4c15efa0 con 0x7f7e4c15e9c0 2024-09-12T03:16:29.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:29.686+0000 7f7e50ee5640 1 -- 172.21.15.27:0/1122803697 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 847755213 0 0) 0x7f7e3c001670 con 0x7f7e4c15e9c0 2024-09-12T03:16:29.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:29.686+0000 7f7e50ee5640 1 -- 172.21.15.27:0/1122803697 >> v1:172.21.15.27:6789/0 conn(0x7f7e4c15e9c0 legacy=0x7f7e4c160170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:29.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:29.686+0000 7f7e50ee5640 1 --2- 172.21.15.27:0/1122803697 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f7e380039b0 0x7f7e38003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:29.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:29.686+0000 7f7e50ee5640 1 --2- 172.21.15.27:0/1122803697 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7e38004800 0x7f7e38004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:29.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:29.686+0000 7f7e50ee5640 1 --2- 172.21.15.27:0/1122803697 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7e380051c0 0x7f7e3800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:29.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:29.686+0000 7f7e50ee5640 1 -- 172.21.15.27:0/1122803697 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f7e4c0d2230 con 0x7f7e380051c0 2024-09-12T03:16:29.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:29.686+0000 7f7e50ee5640 1 -- 172.21.15.27:0/1122803697 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f7e3800c060 con 0x7f7e38004800 2024-09-12T03:16:29.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:29.686+0000 7f7e50ee5640 1 -- 172.21.15.27:0/1122803697 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f7e3800c260 con 0x7f7e380039b0 2024-09-12T03:16:29.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:29.686+0000 7f7e526e8640 1 --2- 172.21.15.27:0/1122803697 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7e380051c0 0x7f7e3800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:29.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:29.686+0000 7f7e516e6640 1 --2- 172.21.15.27:0/1122803697 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7e38004800 0x7f7e38004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:29.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:29.686+0000 7f7e51ee7640 1 --2- 172.21.15.27:0/1122803697 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f7e380039b0 0x7f7e38003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:29.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:29.687+0000 7f7e526e8640 1 -- 172.21.15.27:0/1122803697 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f7e380039b0 msgr2=0x7f7e38003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:29.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:29.687+0000 7f7e526e8640 1 --2- 172.21.15.27:0/1122803697 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f7e380039b0 0x7f7e38003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:29.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:29.687+0000 7f7e526e8640 1 -- 172.21.15.27:0/1122803697 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7e38004800 msgr2=0x7f7e38004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:29.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:29.687+0000 7f7e526e8640 1 --2- 172.21.15.27:0/1122803697 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7e38004800 0x7f7e38004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:29.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:29.687+0000 7f7e526e8640 1 -- 172.21.15.27:0/1122803697 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e38003520 con 0x7f7e380051c0 2024-09-12T03:16:29.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:29.687+0000 7f7e516e6640 1 --2- 172.21.15.27:0/1122803697 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7e38004800 0x7f7e38004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:16:29.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:29.687+0000 7f7e526e8640 1 -- 172.21.15.27:0/1122803697 reap_dead start 2024-09-12T03:16:29.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:29.687+0000 7f7e526e8640 1 --2- 172.21.15.27:0/1122803697 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7e380051c0 0x7f7e3800b9c0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f7e44009fc0 tx=0x7f7e38005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a58532f1019d7bb server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:29.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:29.688+0000 7f7e50ee5640 1 -- 172.21.15.27:0/1122803697 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e44035020 con 0x7f7e380051c0 2024-09-12T03:16:29.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:29.688+0000 7f7e50ee5640 1 -- 172.21.15.27:0/1122803697 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e44003bd0 con 0x7f7e380051c0 2024-09-12T03:16:29.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:29.688+0000 7f7e50ee5640 1 -- 172.21.15.27:0/1122803697 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e44009c10 con 0x7f7e380051c0 2024-09-12T03:16:29.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:29.688+0000 7f7e5414b640 1 -- 172.21.15.27:0/1122803697 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7e380051c0 msgr2=0x7f7e3800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:29.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:29.688+0000 7f7e5414b640 1 --2- 172.21.15.27:0/1122803697 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7e380051c0 0x7f7e3800b9c0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f7e44009fc0 tx=0x7f7e38005830 comp rx=0 tx=0).stop 2024-09-12T03:16:29.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:29.688+0000 7f7e5414b640 1 -- 172.21.15.27:0/1122803697 shutdown_connections 2024-09-12T03:16:29.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:29.688+0000 7f7e5414b640 1 --2- 172.21.15.27:0/1122803697 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7e380051c0 0x7f7e3800b9c0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:29.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:29.688+0000 7f7e5414b640 1 -- 172.21.15.27:0/1122803697 >> 172.21.15.27:0/1122803697 conn(0x7f7e4c06f320 msgr2=0x7f7e4c06f700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:29.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:29.688+0000 7f7e5414b640 1 -- 172.21.15.27:0/1122803697 shutdown_connections 2024-09-12T03:16:29.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:29.688+0000 7f7e5414b640 1 -- 172.21.15.27:0/1122803697 wait complete. 2024-09-12T03:16:29.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:29.689+0000 7f7e5414b640 1 Processor -- start 2024-09-12T03:16:29.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:29.689+0000 7f7e5414b640 1 -- start start 2024-09-12T03:16:29.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:29.690+0000 7f7e5414b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7e380051c0 0x7f7e4c149f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:29.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:29.690+0000 7f7e5414b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7e4c151560 0x7f7e4c1536c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:29.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:29.690+0000 7f7e5414b640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f7e4c151a50 0x7f7e4c153c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:29.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:29.690+0000 7f7e5414b640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f7e4c0d15e0 con 0x7f7e380051c0 2024-09-12T03:16:29.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:29.690+0000 7f7e5414b640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f7e4c06cf60 con 0x7f7e4c151560 2024-09-12T03:16:29.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:29.690+0000 7f7e5414b640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f7e4c0d1760 con 0x7f7e4c151a50 2024-09-12T03:16:29.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:29.690+0000 7f7e51ee7640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7e380051c0 0x7f7e4c149f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:29.695 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:29.690+0000 7f7e51ee7640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7e380051c0 0x7f7e4c149f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55400/0 (socket says 172.21.15.27:55400) 2024-09-12T03:16:29.695 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:29.690+0000 7f7e51ee7640 1 -- 172.21.15.27:0/3698501176 learned_addr learned my addr 172.21.15.27:0/3698501176 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:16:29.695 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:29.690+0000 7f7e526e8640 1 --2- 172.21.15.27:0/3698501176 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f7e4c151a50 0x7f7e4c153c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:29.695 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:29.690+0000 7f7e516e6640 1 --2- 172.21.15.27:0/3698501176 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7e4c151560 0x7f7e4c1536c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:29.695 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:29.690+0000 7f7e51ee7640 1 -- 172.21.15.27:0/3698501176 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f7e4c151a50 msgr2=0x7f7e4c153c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:29.695 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:29.690+0000 7f7e51ee7640 1 --2- 172.21.15.27:0/3698501176 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f7e4c151a50 0x7f7e4c153c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:29.695 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:29.690+0000 7f7e51ee7640 1 -- 172.21.15.27:0/3698501176 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7e4c151560 msgr2=0x7f7e4c1536c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:29.695 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:29.690+0000 7f7e51ee7640 1 --2- 172.21.15.27:0/3698501176 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7e4c151560 0x7f7e4c1536c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:29.695 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:29.691+0000 7f7e51ee7640 1 -- 172.21.15.27:0/3698501176 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e4c154de0 con 0x7f7e380051c0 2024-09-12T03:16:29.695 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:29.691+0000 7f7e51ee7640 1 --2- 172.21.15.27:0/3698501176 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7e380051c0 0x7f7e4c149f90 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f7e4800f620 tx=0x7f7e4800fae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:29.695 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:29.691+0000 7f7e427fc640 1 -- 172.21.15.27:0/3698501176 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e480052e0 con 0x7f7e380051c0 2024-09-12T03:16:29.695 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:29.691+0000 7f7e427fc640 1 -- 172.21.15.27:0/3698501176 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e48018070 con 0x7f7e380051c0 2024-09-12T03:16:29.695 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:29.691+0000 7f7e5414b640 1 -- 172.21.15.27:0/3698501176 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e4c15b1a0 con 0x7f7e380051c0 2024-09-12T03:16:29.695 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:29.691+0000 7f7e427fc640 1 -- 172.21.15.27:0/3698501176 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e48003810 con 0x7f7e380051c0 2024-09-12T03:16:29.696 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:29.691+0000 7f7e5414b640 1 -- 172.21.15.27:0/3698501176 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e4c154820 con 0x7f7e380051c0 2024-09-12T03:16:29.696 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:29.692+0000 7f7e5414b640 1 -- 172.21.15.27:0/3698501176 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e4c06e180 con 0x7f7e380051c0 2024-09-12T03:16:29.696 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:29.693+0000 7f7e427fc640 1 -- 172.21.15.27:0/3698501176 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7e480039b0 con 0x7f7e380051c0 2024-09-12T03:16:29.696 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:29.693+0000 7f7e427fc640 1 --2- 172.21.15.27:0/3698501176 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f7e3006be60 0x7f7e3006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:29.696 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:29.693+0000 7f7e427fc640 1 -- 172.21.15.27:0/3698501176 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 5227+0+0 (secure 0 0 0) 0x7f7e4808aae0 con 0x7f7e380051c0 2024-09-12T03:16:29.696 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:29.693+0000 7f7e516e6640 1 --2- 172.21.15.27:0/3698501176 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f7e3006be60 0x7f7e3006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:29.696 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:29.694+0000 7f7e516e6640 1 --2- 172.21.15.27:0/3698501176 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f7e3006be60 0x7f7e3006e320 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f7e4c151510 tx=0x7f7e3c009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:29.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:29.698+0000 7f7e427fc640 1 -- 172.21.15.27:0/3698501176 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f7e480595e0 con 0x7f7e380051c0 2024-09-12T03:16:29.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:29.878+0000 7f7e5414b640 1 -- 172.21.15.27:0/3698501176 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f7e4c06d260 con 0x7f7e380051c0 2024-09-12T03:16:30.410 DEBUG:teuthology.orchestra.run.smithi027:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-12T03:16:30.413 DEBUG:teuthology.orchestra.run.smithi045:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-12T03:16:30.558 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:30.557+0000 7f7e427fc640 1 -- 172.21.15.27:0/3698501176 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v19) ==== 170+0+0 (secure 0 0 0) 0x7f7e48058ee0 con 0x7f7e380051c0 2024-09-12T03:16:30.558 INFO:teuthology.orchestra.run.smithi027.stderr:pool 'cephfs_data' created 2024-09-12T03:16:30.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:30.561+0000 7f7e5414b640 1 -- 172.21.15.27:0/3698501176 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f7e3006be60 msgr2=0x7f7e3006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:30.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:30.561+0000 7f7e5414b640 1 --2- 172.21.15.27:0/3698501176 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f7e3006be60 0x7f7e3006e320 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f7e4c151510 tx=0x7f7e3c009040 comp rx=0 tx=0).stop 2024-09-12T03:16:30.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:30.561+0000 7f7e5414b640 1 -- 172.21.15.27:0/3698501176 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7e380051c0 msgr2=0x7f7e4c149f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:30.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:30.561+0000 7f7e5414b640 1 --2- 172.21.15.27:0/3698501176 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7e380051c0 0x7f7e4c149f90 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f7e4800f620 tx=0x7f7e4800fae0 comp rx=0 tx=0).stop 2024-09-12T03:16:30.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:30.561+0000 7f7e5414b640 1 -- 172.21.15.27:0/3698501176 shutdown_connections 2024-09-12T03:16:30.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:30.561+0000 7f7e5414b640 1 --2- 172.21.15.27:0/3698501176 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f7e3006be60 0x7f7e3006e320 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:30.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:30.561+0000 7f7e5414b640 1 --2- 172.21.15.27:0/3698501176 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f7e4c151a50 0x7f7e4c153c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:30.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:30.561+0000 7f7e5414b640 1 --2- 172.21.15.27:0/3698501176 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7e4c151560 0x7f7e4c1536c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:30.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:30.561+0000 7f7e5414b640 1 --2- 172.21.15.27:0/3698501176 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7e380051c0 0x7f7e4c149f90 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:30.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:30.561+0000 7f7e5414b640 1 -- 172.21.15.27:0/3698501176 >> 172.21.15.27:0/3698501176 conn(0x7f7e4c06f320 msgr2=0x7f7e4c0b6070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:30.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:30.561+0000 7f7e5414b640 1 -- 172.21.15.27:0/3698501176 shutdown_connections 2024-09-12T03:16:30.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:30.561+0000 7f7e5414b640 1 -- 172.21.15.27:0/3698501176 wait complete. 2024-09-12T03:16:30.584 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-12T03:16:30.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:30.689+0000 7fcc5bf86640 1 Processor -- start 2024-09-12T03:16:30.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:30.690+0000 7fcc5bf86640 1 -- start start 2024-09-12T03:16:30.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:30.690+0000 7fcc5bf86640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fcc5406cc40 0x7fcc5406d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:30.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:30.690+0000 7fcc5bf86640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fcc5406db30 0x7fcc5406df60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:30.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:30.690+0000 7fcc5bf86640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcc5406d5d0 con 0x7fcc5415e650 2024-09-12T03:16:30.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:30.690+0000 7fcc5bf86640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7fcc5406d850 con 0x7fcc5406db30 2024-09-12T03:16:30.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:30.690+0000 7fcc5bf86640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fcc540d1260 con 0x7fcc5406cc40 2024-09-12T03:16:30.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:30.691+0000 7fcc5a523640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fcc5406db30 0x7fcc5406df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:30.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:30.691+0000 7fcc59d22640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fcc5415e650 0x7fcc540d1d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:51380/0 (socket says 172.21.15.27:51380) 2024-09-12T03:16:30.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:30.691+0000 7fcc5a523640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fcc5406db30 0x7fcc5406df60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55412/0 (socket says 172.21.15.27:55412) 2024-09-12T03:16:30.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:30.691+0000 7fcc59d22640 1 -- 172.21.15.27:0/1322006535 learned_addr learned my addr 172.21.15.27:0/1322006535 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:16:30.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:30.691+0000 7fcc59521640 1 --2- 172.21.15.27:0/1322006535 >> v2:172.21.15.45:3300/0 conn(0x7fcc5406cc40 0x7fcc5406d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:30.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:30.691+0000 7fcc5a523640 1 -- 172.21.15.27:0/1322006535 >> v2:172.21.15.45:3300/0 conn(0x7fcc5406cc40 msgr2=0x7fcc5406d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:30.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:30.691+0000 7fcc5a523640 1 --2- 172.21.15.27:0/1322006535 >> v2:172.21.15.45:3300/0 conn(0x7fcc5406cc40 0x7fcc5406d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:30.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:30.691+0000 7fcc5a523640 1 -- 172.21.15.27:0/1322006535 >> v1:172.21.15.27:6789/0 conn(0x7fcc5415e650 legacy=0x7fcc540d1d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:30.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:30.691+0000 7fcc5a523640 1 -- 172.21.15.27:0/1322006535 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc540d1540 con 0x7fcc5406db30 2024-09-12T03:16:30.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:30.691+0000 7fcc58d20640 1 -- 172.21.15.27:0/1322006535 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1607484433 0 0) 0x7fcc50001110 con 0x7fcc5415e650 2024-09-12T03:16:30.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:30.692+0000 7fcc5a523640 1 --2- 172.21.15.27:0/1322006535 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcc5406db30 0x7fcc5406df60 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fcc54050f60 tx=0x7fcc4c02a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76f8b1e7a6eee758 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:30.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:30.692+0000 7fcc58d20640 1 -- 172.21.15.27:0/1322006535 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc4c007740 con 0x7fcc5406db30 2024-09-12T03:16:30.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:30.692+0000 7fcc58d20640 1 -- 172.21.15.27:0/1322006535 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc4c0078e0 con 0x7fcc5406db30 2024-09-12T03:16:30.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:30.692+0000 7fcc58d20640 1 -- 172.21.15.27:0/1322006535 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc4c007cb0 con 0x7fcc5406db30 2024-09-12T03:16:30.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:30.692+0000 7fcc5bf86640 1 -- 172.21.15.27:0/1322006535 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcc5406db30 msgr2=0x7fcc5406df60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:30.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:30.692+0000 7fcc5bf86640 1 --2- 172.21.15.27:0/1322006535 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcc5406db30 0x7fcc5406df60 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fcc54050f60 tx=0x7fcc4c02a5f0 comp rx=0 tx=0).stop 2024-09-12T03:16:30.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:30.692+0000 7fcc5bf86640 1 -- 172.21.15.27:0/1322006535 shutdown_connections 2024-09-12T03:16:30.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:30.692+0000 7fcc5bf86640 1 --2- 172.21.15.27:0/1322006535 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcc5406db30 0x7fcc5406df60 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:30.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:30.692+0000 7fcc5bf86640 1 --2- 172.21.15.27:0/1322006535 >> v2:172.21.15.45:3300/0 conn(0x7fcc5406cc40 0x7fcc5406d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:30.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:30.692+0000 7fcc5bf86640 1 -- 172.21.15.27:0/1322006535 >> 172.21.15.27:0/1322006535 conn(0x7fcc5406f320 msgr2=0x7fcc5406f700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:30.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:30.693+0000 7fcc5bf86640 1 -- 172.21.15.27:0/1322006535 shutdown_connections 2024-09-12T03:16:30.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:30.693+0000 7fcc5bf86640 1 -- 172.21.15.27:0/1322006535 wait complete. 2024-09-12T03:16:30.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:30.694+0000 7fcc5bf86640 1 Processor -- start 2024-09-12T03:16:30.695 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:30.694+0000 7fcc5bf86640 1 -- start start 2024-09-12T03:16:30.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:30.694+0000 7fcc5bf86640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcc5406cc40 0x7fcc541588e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:30.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:30.694+0000 7fcc5bf86640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fcc5406db30 0x7fcc540b3b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:30.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:30.694+0000 7fcc5bf86640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcc5415e650 0x7fcc541f4830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:30.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:30.694+0000 7fcc5bf86640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fcc5406e620 con 0x7fcc5415e650 2024-09-12T03:16:30.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:30.694+0000 7fcc5bf86640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fcc5406e4a0 con 0x7fcc5406cc40 2024-09-12T03:16:30.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:30.694+0000 7fcc5bf86640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fcc540b55b0 con 0x7fcc5406db30 2024-09-12T03:16:30.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:30.695+0000 7fcc5a523640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcc5415e650 0x7fcc541f4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:30.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:30.695+0000 7fcc5a523640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcc5415e650 0x7fcc541f4830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55416/0 (socket says 172.21.15.27:55416) 2024-09-12T03:16:30.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:30.695+0000 7fcc5a523640 1 -- 172.21.15.27:0/524286587 learned_addr learned my addr 172.21.15.27:0/524286587 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:16:30.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:30.695+0000 7fcc59d22640 1 --2- 172.21.15.27:0/524286587 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcc5406cc40 0x7fcc541588e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:30.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:30.695+0000 7fcc59521640 1 --2- 172.21.15.27:0/524286587 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fcc5406db30 0x7fcc540b3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:30.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:30.695+0000 7fcc5a523640 1 -- 172.21.15.27:0/524286587 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fcc5406db30 msgr2=0x7fcc540b3b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:30.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:30.695+0000 7fcc5a523640 1 --2- 172.21.15.27:0/524286587 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fcc5406db30 0x7fcc540b3b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:30.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:30.695+0000 7fcc5a523640 1 -- 172.21.15.27:0/524286587 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcc5406cc40 msgr2=0x7fcc541588e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:30.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:30.695+0000 7fcc5a523640 1 --2- 172.21.15.27:0/524286587 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcc5406cc40 0x7fcc541588e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:30.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:30.695+0000 7fcc5a523640 1 -- 172.21.15.27:0/524286587 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc54158ec0 con 0x7fcc5415e650 2024-09-12T03:16:30.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:30.695+0000 7fcc5a523640 1 --2- 172.21.15.27:0/524286587 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcc5415e650 0x7fcc541f4830 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fcc4c009e50 tx=0x7fcc4c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:30.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:30.695+0000 7fcc4a7fc640 1 -- 172.21.15.27:0/524286587 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc4c040070 con 0x7fcc5415e650 2024-09-12T03:16:30.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:30.695+0000 7fcc4a7fc640 1 -- 172.21.15.27:0/524286587 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc4c031bf0 con 0x7fcc5415e650 2024-09-12T03:16:30.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:30.695+0000 7fcc5bf86640 1 -- 172.21.15.27:0/524286587 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc541591a0 con 0x7fcc5415e650 2024-09-12T03:16:30.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:30.695+0000 7fcc4a7fc640 1 -- 172.21.15.27:0/524286587 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc4c009450 con 0x7fcc5415e650 2024-09-12T03:16:30.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:30.695+0000 7fcc5bf86640 1 -- 172.21.15.27:0/524286587 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc54159760 con 0x7fcc5415e650 2024-09-12T03:16:30.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:30.696+0000 7fcc5bf86640 1 -- 172.21.15.27:0/524286587 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc5406d090 con 0x7fcc5415e650 2024-09-12T03:16:30.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:30.697+0000 7fcc4a7fc640 1 -- 172.21.15.27:0/524286587 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fcc4c039040 con 0x7fcc5415e650 2024-09-12T03:16:30.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:30.697+0000 7fcc4a7fc640 1 --2- 172.21.15.27:0/524286587 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fcc3806be30 0x7fcc3806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:30.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:30.697+0000 7fcc4a7fc640 1 -- 172.21.15.27:0/524286587 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 5609+0+0 (secure 0 0 0) 0x7fcc4c0ac440 con 0x7fcc5415e650 2024-09-12T03:16:30.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:30.698+0000 7fcc59d22640 1 --2- 172.21.15.27:0/524286587 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fcc3806be30 0x7fcc3806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:30.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:30.698+0000 7fcc59d22640 1 --2- 172.21.15.27:0/524286587 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fcc3806be30 0x7fcc3806e2f0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fcc50007fd0 tx=0x7fcc50007de0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:30.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:30.702+0000 7fcc4a7fc640 1 -- 172.21.15.27:0/524286587 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fcc4c07a890 con 0x7fcc5415e650 2024-09-12T03:16:30.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:30.913+0000 7fcc5bf86640 1 -- 172.21.15.27:0/524286587 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fcc54069f80 con 0x7fcc5415e650 2024-09-12T03:16:31.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:31.559+0000 7fcc4a7fc640 1 -- 172.21.15.27:0/524286587 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) ==== 172+0+0 (secure 0 0 0) 0x7fcc4c07eb90 con 0x7fcc5415e650 2024-09-12T03:16:31.559 INFO:teuthology.orchestra.run.smithi027.stderr:new fs with metadata pool 2 and data pool 3 2024-09-12T03:16:31.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:31.561+0000 7fcc5bf86640 1 -- 172.21.15.27:0/524286587 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fcc3806be30 msgr2=0x7fcc3806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:31.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:31.561+0000 7fcc5bf86640 1 --2- 172.21.15.27:0/524286587 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fcc3806be30 0x7fcc3806e2f0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fcc50007fd0 tx=0x7fcc50007de0 comp rx=0 tx=0).stop 2024-09-12T03:16:31.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:31.561+0000 7fcc5bf86640 1 -- 172.21.15.27:0/524286587 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcc5415e650 msgr2=0x7fcc541f4830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:31.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:31.561+0000 7fcc5bf86640 1 --2- 172.21.15.27:0/524286587 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcc5415e650 0x7fcc541f4830 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fcc4c009e50 tx=0x7fcc4c004a40 comp rx=0 tx=0).stop 2024-09-12T03:16:31.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:31.561+0000 7fcc5bf86640 1 -- 172.21.15.27:0/524286587 shutdown_connections 2024-09-12T03:16:31.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:31.561+0000 7fcc5bf86640 1 --2- 172.21.15.27:0/524286587 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fcc3806be30 0x7fcc3806e2f0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:31.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:31.561+0000 7fcc5bf86640 1 --2- 172.21.15.27:0/524286587 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcc5415e650 0x7fcc541f4830 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:31.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:31.561+0000 7fcc5bf86640 1 --2- 172.21.15.27:0/524286587 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fcc5406db30 0x7fcc540b3b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:31.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:31.561+0000 7fcc5bf86640 1 --2- 172.21.15.27:0/524286587 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcc5406cc40 0x7fcc541588e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:31.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:31.561+0000 7fcc5bf86640 1 -- 172.21.15.27:0/524286587 >> 172.21.15.27:0/524286587 conn(0x7fcc5406f320 msgr2=0x7fcc540d2cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:31.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:31.561+0000 7fcc5bf86640 1 -- 172.21.15.27:0/524286587 shutdown_connections 2024-09-12T03:16:31.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:31.561+0000 7fcc5bf86640 1 -- 172.21.15.27:0/524286587 wait complete. 2024-09-12T03:16:31.585 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-12T03:16:31.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:31.692+0000 7fbcdf45c640 1 Processor -- start 2024-09-12T03:16:31.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:31.692+0000 7fbcdf45c640 1 -- start start 2024-09-12T03:16:31.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:31.693+0000 7fbcdf45c640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fbcd815a1f0 0x7fbcd806dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:31.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:31.693+0000 7fbcdf45c640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbcd8073b50 con 0x7fbcd806fcf0 2024-09-12T03:16:31.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:31.693+0000 7fbcdf45c640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbcd8073870 con 0x7fbcd8072fc0 2024-09-12T03:16:31.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:31.693+0000 7fbcdf45c640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fbcd80ce3e0 con 0x7fbcd815a1f0 2024-09-12T03:16:31.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:31.693+0000 7fbcdc9f7640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fbcd806fcf0 0x7fbcd80728b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:51386/0 (socket says 172.21.15.27:51386) 2024-09-12T03:16:31.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:31.693+0000 7fbcdc9f7640 1 -- 172.21.15.27:0/986730873 learned_addr learned my addr 172.21.15.27:0/986730873 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:16:31.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:31.693+0000 7fbcdd1f8640 1 --2- 172.21.15.27:0/986730873 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fbcd815a1f0 0x7fbcd806dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:31.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:31.694+0000 7fbcd7fff640 1 -- 172.21.15.27:0/986730873 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1863994934 0 0) 0x7fbcd8073b50 con 0x7fbcd806fcf0 2024-09-12T03:16:31.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:31.694+0000 7fbcd7fff640 1 -- 172.21.15.27:0/986730873 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbcc0003520 con 0x7fbcd806fcf0 2024-09-12T03:16:31.695 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:31.694+0000 7fbcd7fff640 1 -- 172.21.15.27:0/986730873 <== mon.1 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3682298 0 0) 0x7fbcd8073870 con 0x7fbcd8072fc0 2024-09-12T03:16:31.695 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:31.694+0000 7fbcd7fff640 1 -- 172.21.15.27:0/986730873 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbcd8073b50 con 0x7fbcd8072fc0 2024-09-12T03:16:31.695 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:31.694+0000 7fbcdd1f8640 1 -- 172.21.15.27:0/986730873 >> v1:172.21.15.45:6789/0 conn(0x7fbcd8072fc0 legacy=0x7fbcd815cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:31.695 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:31.694+0000 7fbcdd1f8640 1 -- 172.21.15.27:0/986730873 >> v1:172.21.15.27:6789/0 conn(0x7fbcd806fcf0 legacy=0x7fbcd80728b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:31.695 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:31.694+0000 7fbcdd1f8640 1 -- 172.21.15.27:0/986730873 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcd80ce6c0 con 0x7fbcd815a1f0 2024-09-12T03:16:31.696 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:31.694+0000 7fbcdd1f8640 1 --2- 172.21.15.27:0/986730873 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fbcd815a1f0 0x7fbcd806dac0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fbcd00097d0 tx=0x7fbcd002a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b1675f432da39587 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:31.696 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:31.695+0000 7fbcd7fff640 1 -- 172.21.15.27:0/986730873 <== mon.2 v2:172.21.15.45:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcd0037070 con 0x7fbcd815a1f0 2024-09-12T03:16:31.696 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:31.695+0000 7fbcd7fff640 1 -- 172.21.15.27:0/986730873 <== mon.2 v2:172.21.15.45:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbcd0032440 con 0x7fbcd815a1f0 2024-09-12T03:16:31.696 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:31.695+0000 7fbcd7fff640 1 -- 172.21.15.27:0/986730873 <== mon.2 v2:172.21.15.45:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcd00327f0 con 0x7fbcd815a1f0 2024-09-12T03:16:31.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:31.695+0000 7fbcdf45c640 1 -- 172.21.15.27:0/986730873 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fbcd815a1f0 msgr2=0x7fbcd806dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:31.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:31.695+0000 7fbcdf45c640 1 --2- 172.21.15.27:0/986730873 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fbcd815a1f0 0x7fbcd806dac0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fbcd00097d0 tx=0x7fbcd002a210 comp rx=0 tx=0).stop 2024-09-12T03:16:31.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:31.695+0000 7fbcdf45c640 1 -- 172.21.15.27:0/986730873 shutdown_connections 2024-09-12T03:16:31.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:31.695+0000 7fbcdf45c640 1 --2- 172.21.15.27:0/986730873 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fbcd815a1f0 0x7fbcd806dac0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:31.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:31.695+0000 7fbcdf45c640 1 -- 172.21.15.27:0/986730873 >> 172.21.15.27:0/986730873 conn(0x7fbcd806ca30 msgr2=0x7fbcd806ce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:31.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:31.695+0000 7fbcdf45c640 1 -- 172.21.15.27:0/986730873 shutdown_connections 2024-09-12T03:16:31.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:31.695+0000 7fbcdf45c640 1 -- 172.21.15.27:0/986730873 wait complete. 2024-09-12T03:16:31.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:31.696+0000 7fbcdf45c640 1 Processor -- start 2024-09-12T03:16:31.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:31.697+0000 7fbcdf45c640 1 -- start start 2024-09-12T03:16:31.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:31.697+0000 7fbcdf45c640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fbcd806fcf0 0x7fbcd814a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:31.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:31.697+0000 7fbcdf45c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbcd8072fc0 0x7fbcd8155860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:31.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:31.697+0000 7fbcdf45c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbcd815a1f0 0x7fbcd8158750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:31.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:31.697+0000 7fbcdf45c640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fbcd8070110 con 0x7fbcd815a1f0 2024-09-12T03:16:31.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:31.697+0000 7fbcdf45c640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fbcd8073520 con 0x7fbcd8072fc0 2024-09-12T03:16:31.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:31.697+0000 7fbcdf45c640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fbcd80b02a0 con 0x7fbcd806fcf0 2024-09-12T03:16:31.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:31.697+0000 7fbcdd9f9640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbcd815a1f0 0x7fbcd8158750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:31.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:31.697+0000 7fbcdd9f9640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbcd815a1f0 0x7fbcd8158750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55420/0 (socket says 172.21.15.27:55420) 2024-09-12T03:16:31.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:31.697+0000 7fbcdd9f9640 1 -- 172.21.15.27:0/2761662764 learned_addr learned my addr 172.21.15.27:0/2761662764 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:16:31.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:31.698+0000 7fbcdc9f7640 1 --2- 172.21.15.27:0/2761662764 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbcd8072fc0 0x7fbcd8155860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:31.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:31.698+0000 7fbcdd1f8640 1 --2- 172.21.15.27:0/2761662764 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fbcd806fcf0 0x7fbcd814a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:31.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:31.698+0000 7fbcdd9f9640 1 -- 172.21.15.27:0/2761662764 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fbcd806fcf0 msgr2=0x7fbcd814a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:31.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:31.698+0000 7fbcdd9f9640 1 --2- 172.21.15.27:0/2761662764 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fbcd806fcf0 0x7fbcd814a7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:31.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:31.698+0000 7fbcdd9f9640 1 -- 172.21.15.27:0/2761662764 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbcd8072fc0 msgr2=0x7fbcd8155860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:31.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:31.698+0000 7fbcdd9f9640 1 --2- 172.21.15.27:0/2761662764 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbcd8072fc0 0x7fbcd8155860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:31.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:31.698+0000 7fbcdd9f9640 1 -- 172.21.15.27:0/2761662764 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcd81561d0 con 0x7fbcd815a1f0 2024-09-12T03:16:31.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:31.698+0000 7fbcdd1f8640 1 --2- 172.21.15.27:0/2761662764 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fbcd806fcf0 0x7fbcd814a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:16:31.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:31.698+0000 7fbcdd9f9640 1 --2- 172.21.15.27:0/2761662764 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbcd815a1f0 0x7fbcd8158750 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fbcc800c760 tx=0x7fbcc800cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:31.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:31.698+0000 7fbcd5ffb640 1 -- 172.21.15.27:0/2761662764 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcc8003b70 con 0x7fbcd815a1f0 2024-09-12T03:16:31.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:31.698+0000 7fbcd5ffb640 1 -- 172.21.15.27:0/2761662764 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbcc8003d10 con 0x7fbcd815a1f0 2024-09-12T03:16:31.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:31.698+0000 7fbcdf45c640 1 -- 172.21.15.27:0/2761662764 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbcd81564b0 con 0x7fbcd815a1f0 2024-09-12T03:16:31.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:31.698+0000 7fbcd5ffb640 1 -- 172.21.15.27:0/2761662764 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcc800aa90 con 0x7fbcd815a1f0 2024-09-12T03:16:31.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:31.698+0000 7fbcdf45c640 1 -- 172.21.15.27:0/2761662764 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbcd8156a70 con 0x7fbcd815a1f0 2024-09-12T03:16:31.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:31.699+0000 7fbcdf45c640 1 -- 172.21.15.27:0/2761662764 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbcd806dac0 con 0x7fbcd815a1f0 2024-09-12T03:16:31.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:31.700+0000 7fbcd5ffb640 1 -- 172.21.15.27:0/2761662764 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fbcc800ac30 con 0x7fbcd815a1f0 2024-09-12T03:16:31.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:31.700+0000 7fbcd5ffb640 1 --2- 172.21.15.27:0/2761662764 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fbcc406be30 0x7fbcc406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:31.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:31.700+0000 7fbcd5ffb640 1 -- 172.21.15.27:0/2761662764 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fbcc808ad00 con 0x7fbcd815a1f0 2024-09-12T03:16:31.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:31.700+0000 7fbcdd1f8640 1 --2- 172.21.15.27:0/2761662764 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fbcc406be30 0x7fbcc406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:31.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:31.701+0000 7fbcdd1f8640 1 --2- 172.21.15.27:0/2761662764 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fbcc406be30 0x7fbcc406e2f0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fbcd0034000 tx=0x7fbcd002e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:31.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:31.704+0000 7fbcd5ffb640 1 -- 172.21.15.27:0/2761662764 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fbcc80596b0 con 0x7fbcd815a1f0 2024-09-12T03:16:31.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:31.880+0000 7fbcdf45c640 1 -- 172.21.15.27:0/2761662764 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbcd80726b0 con 0x7fbcd815a1f0 2024-09-12T03:16:31.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:31.882+0000 7fbcd5ffb640 1 -- 172.21.15.27:0/2761662764 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16630 (secure 0 0 0) 0x7fbcc8058fb0 con 0x7fbcd815a1f0 2024-09-12T03:16:31.882 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:16:31.882 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":20,"fsid":"d2df504e-2647-4385-a5db-9fdf69f487a0","created":"2024-09-12T03:16:07.778937+0000","modified":"2024-09-12T03:16:30.914204+0000","last_up_change":"2024-09-12T03:16:19.314778+0000","last_in_change":"2024-09-12T03:16:09.893086+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T03:16:20.097646+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-12T03:16:28.790791+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-12T03:16:29.909065+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1851116c-1fb3-4dc9-9772-f90e2ef4c355","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6817","nonce":3002321488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6819","nonce":3002321488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6823","nonce":3002321488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6821","nonce":3002321488}]},"public_addr":"172.21.15.27:6817/3002321488","cluster_addr":"172.21.15.27:6819/3002321488","heartbeat_back_addr":"172.21.15.27:6823/3002321488","heartbeat_front_addr":"172.21.15.27:6821/3002321488","state":["exists","up"]},{"osd":1,"uuid":"f38ae17d-73eb-4204-b99d-975fa806fb87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6809","nonce":1613102886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6811","nonce":1613102886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6815","nonce":1613102886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6813","nonce":1613102886}]},"public_addr":"172.21.15.27:6809/1613102886","cluster_addr":"172.21.15.27:6811/1613102886","heartbeat_back_addr":"172.21.15.27:6815/1613102886","heartbeat_front_addr":"172.21.15.27:6813/1613102886","state":["exists","up"]},{"osd":2,"uuid":"fb12dcc8-bdd0-42b8-b46d-8b10106224bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6825","nonce":3534723355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6827","nonce":3534723355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6831","nonce":3534723355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6829","nonce":3534723355}]},"public_addr":"172.21.15.27:6825/3534723355","cluster_addr":"172.21.15.27:6827/3534723355","heartbeat_back_addr":"172.21.15.27:6831/3534723355","heartbeat_front_addr":"172.21.15.27:6829/3534723355","state":["exists","up"]},{"osd":3,"uuid":"d74d3f49-9c24-4b9f-98e2-3ecd7b5ef3b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6801","nonce":80661598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6803","nonce":80661598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6807","nonce":80661598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6805","nonce":80661598}]},"public_addr":"172.21.15.27:6801/80661598","cluster_addr":"172.21.15.27:6803/80661598","heartbeat_back_addr":"172.21.15.27:6807/80661598","heartbeat_front_addr":"172.21.15.27:6805/80661598","state":["exists","up"]},{"osd":4,"uuid":"2bf123f2-1b0d-41c4-aa3b-805bab267e59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6817","nonce":3943189827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6819","nonce":3943189827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6823","nonce":3943189827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6821","nonce":3943189827}]},"public_addr":"172.21.15.45:6817/3943189827","cluster_addr":"172.21.15.45:6819/3943189827","heartbeat_back_addr":"172.21.15.45:6823/3943189827","heartbeat_front_addr":"172.21.15.45:6821/3943189827","state":["exists","up"]},{"osd":5,"uuid":"c49ba5da-91c6-4a86-a97b-4aba89c253ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6809","nonce":3620760330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6811","nonce":3620760330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6815","nonce":3620760330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6813","nonce":3620760330}]},"public_addr":"172.21.15.45:6809/3620760330","cluster_addr":"172.21.15.45:6811/3620760330","heartbeat_back_addr":"172.21.15.45:6815/3620760330","heartbeat_front_addr":"172.21.15.45:6813/3620760330","state":["exists","up"]},{"osd":6,"uuid":"b0123475-89f7-4a34-9e33-d797b99dd9dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6801","nonce":1989083510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6803","nonce":1989083510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6807","nonce":1989083510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6805","nonce":1989083510}]},"public_addr":"172.21.15.45:6801/1989083510","cluster_addr":"172.21.15.45:6803/1989083510","heartbeat_back_addr":"172.21.15.45:6807/1989083510","heartbeat_front_addr":"172.21.15.45:6805/1989083510","state":["exists","up"]},{"osd":7,"uuid":"4dad773d-b7d8-46fc-8a10-a972aad27b29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6825","nonce":1513300294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6827","nonce":1513300294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6831","nonce":1513300294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6829","nonce":1513300294}]},"public_addr":"172.21.15.45:6825/1513300294","cluster_addr":"172.21.15.45:6827/1513300294","heartbeat_back_addr":"172.21.15.45:6831/1513300294","heartbeat_front_addr":"172.21.15.45:6829/1513300294","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.881855+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:17.133252+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.735339+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.500742+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.477571+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:17.167667+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.470291+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.873743+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T03:16:31.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:31.883+0000 7fbcdf45c640 1 -- 172.21.15.27:0/2761662764 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fbcc406be30 msgr2=0x7fbcc406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:31.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:31.883+0000 7fbcdf45c640 1 --2- 172.21.15.27:0/2761662764 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fbcc406be30 0x7fbcc406e2f0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fbcd0034000 tx=0x7fbcd002e000 comp rx=0 tx=0).stop 2024-09-12T03:16:31.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:31.883+0000 7fbcdf45c640 1 -- 172.21.15.27:0/2761662764 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbcd815a1f0 msgr2=0x7fbcd8158750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:31.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:31.883+0000 7fbcdf45c640 1 --2- 172.21.15.27:0/2761662764 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbcd815a1f0 0x7fbcd8158750 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fbcc800c760 tx=0x7fbcc800cc20 comp rx=0 tx=0).stop 2024-09-12T03:16:31.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:31.883+0000 7fbcdf45c640 1 -- 172.21.15.27:0/2761662764 shutdown_connections 2024-09-12T03:16:31.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:31.884+0000 7fbcdf45c640 1 --2- 172.21.15.27:0/2761662764 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fbcc406be30 0x7fbcc406e2f0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:31.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:31.884+0000 7fbcdf45c640 1 --2- 172.21.15.27:0/2761662764 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbcd815a1f0 0x7fbcd8158750 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:31.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:31.884+0000 7fbcdf45c640 1 --2- 172.21.15.27:0/2761662764 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbcd8072fc0 0x7fbcd8155860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:31.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:31.884+0000 7fbcdf45c640 1 --2- 172.21.15.27:0/2761662764 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fbcd806fcf0 0x7fbcd814a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:31.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:31.884+0000 7fbcdf45c640 1 -- 172.21.15.27:0/2761662764 >> 172.21.15.27:0/2761662764 conn(0x7fbcd806ca30 msgr2=0x7fbcd80cff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:31.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:31.884+0000 7fbcdf45c640 1 -- 172.21.15.27:0/2761662764 shutdown_connections 2024-09-12T03:16:31.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:31.884+0000 7fbcdf45c640 1 -- 172.21.15.27:0/2761662764 wait complete. 2024-09-12T03:16:31.894 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-12T03:16:31.996 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:31.996+0000 7fcab6a23640 1 Processor -- start 2024-09-12T03:16:31.997 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:31.996+0000 7fcab6a23640 1 -- start start 2024-09-12T03:16:31.998 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:31.997+0000 7fcab6a23640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fcab00b6470 0x7fcab00d2710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:31.998 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:31.997+0000 7fcab6a23640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fcab00b6850 0x7fcab00d2c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:31.998 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:31.997+0000 7fcab6a23640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fcab00d3190 0x7fcab0161b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:31.998 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:31.997+0000 7fcab6a23640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7fcab00d1ec0 con 0x7fcab00b6850 2024-09-12T03:16:31.998 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:31.997+0000 7fcab6a23640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fcab00cf980 con 0x7fcab00b6470 2024-09-12T03:16:31.998 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:31.997+0000 7fcab6a23640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fcab00b79b0 con 0x7fcab00d3190 2024-09-12T03:16:31.998 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:31.997+0000 7fcaaf7fe640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fcab00b6850 0x7fcab00d2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:31.998 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:31.997+0000 7fcaaf7fe640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fcab00b6850 0x7fcab00d2c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55428/0 (socket says 172.21.15.27:55428) 2024-09-12T03:16:31.998 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:31.997+0000 7fcaaf7fe640 1 -- 172.21.15.27:0/2141222985 learned_addr learned my addr 172.21.15.27:0/2141222985 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:16:31.998 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:31.997+0000 7fcaaffff640 1 --2- 172.21.15.27:0/2141222985 >> v2:172.21.15.45:3300/0 conn(0x7fcab00b6470 0x7fcab00d2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:31.999 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:31.997+0000 7fcab4fc0640 1 --2- 172.21.15.27:0/2141222985 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fcab00d3190 0x7fcab0161b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:31.999 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:31.998+0000 7fcaaf7fe640 1 -- 172.21.15.27:0/2141222985 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fcab00d3190 msgr2=0x7fcab0161b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:31.999 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:31.998+0000 7fcaaf7fe640 1 --2- 172.21.15.27:0/2141222985 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fcab00d3190 0x7fcab0161b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:31.999 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:31.998+0000 7fcaaf7fe640 1 -- 172.21.15.27:0/2141222985 >> v2:172.21.15.45:3300/0 conn(0x7fcab00b6470 msgr2=0x7fcab00d2710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:31.999 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:31.998+0000 7fcaaf7fe640 1 --2- 172.21.15.27:0/2141222985 >> v2:172.21.15.45:3300/0 conn(0x7fcab00b6470 0x7fcab00d2710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:31.999 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:31.998+0000 7fcaaf7fe640 1 -- 172.21.15.27:0/2141222985 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcab00cb580 con 0x7fcab00b6850 2024-09-12T03:16:32.000 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:31.998+0000 7fcaaf7fe640 1 --2- 172.21.15.27:0/2141222985 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcab00b6850 0x7fcab00d2c50 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fcab0050fd0 tx=0x7fcaa402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d89b7e432e9278b0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:32.000 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:31.998+0000 7fcaaeffd640 1 -- 172.21.15.27:0/2141222985 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaa4037070 con 0x7fcab00b6850 2024-09-12T03:16:32.001 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:31.998+0000 7fcaaeffd640 1 -- 172.21.15.27:0/2141222985 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcaa4032440 con 0x7fcab00b6850 2024-09-12T03:16:32.001 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:31.999+0000 7fcaaeffd640 1 -- 172.21.15.27:0/2141222985 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaa4032810 con 0x7fcab00b6850 2024-09-12T03:16:32.001 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:31.999+0000 7fcab6a23640 1 -- 172.21.15.27:0/2141222985 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcab00b6850 msgr2=0x7fcab00d2c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:32.001 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:31.999+0000 7fcab6a23640 1 --2- 172.21.15.27:0/2141222985 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcab00b6850 0x7fcab00d2c50 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fcab0050fd0 tx=0x7fcaa402a1e0 comp rx=0 tx=0).stop 2024-09-12T03:16:32.001 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:31.999+0000 7fcab6a23640 1 -- 172.21.15.27:0/2141222985 shutdown_connections 2024-09-12T03:16:32.001 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:31.999+0000 7fcab6a23640 1 --2- 172.21.15.27:0/2141222985 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fcab00d3190 0x7fcab0161b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:32.001 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:31.999+0000 7fcab6a23640 1 --2- 172.21.15.27:0/2141222985 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcab00b6850 0x7fcab00d2c50 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:32.001 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:31.999+0000 7fcab6a23640 1 --2- 172.21.15.27:0/2141222985 >> v2:172.21.15.45:3300/0 conn(0x7fcab00b6470 0x7fcab00d2710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:32.001 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:31.999+0000 7fcab6a23640 1 -- 172.21.15.27:0/2141222985 >> 172.21.15.27:0/2141222985 conn(0x7fcab00d1200 msgr2=0x7fcab00d1610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:32.001 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:31.999+0000 7fcab6a23640 1 -- 172.21.15.27:0/2141222985 shutdown_connections 2024-09-12T03:16:32.001 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:31.999+0000 7fcab6a23640 1 -- 172.21.15.27:0/2141222985 wait complete. 2024-09-12T03:16:32.001 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:32.000+0000 7fcab6a23640 1 Processor -- start 2024-09-12T03:16:32.001 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:32.000+0000 7fcab6a23640 1 -- start start 2024-09-12T03:16:32.004 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:32.001+0000 7fcab6a23640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcab00b6470 0x7fcab00ce750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:32.004 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:32.001+0000 7fcab6a23640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fcab00b6850 0x7fcab00b19c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:32.004 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:32.001+0000 7fcab6a23640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcab00d3190 0x7fcab00b1f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:32.004 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:32.001+0000 7fcab6a23640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fcab006d9d0 con 0x7fcab00d3190 2024-09-12T03:16:32.004 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:32.001+0000 7fcab6a23640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fcab006d850 con 0x7fcab00b6470 2024-09-12T03:16:32.004 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:32.001+0000 7fcab6a23640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fcab00d3fa0 con 0x7fcab00b6850 2024-09-12T03:16:32.005 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:32.001+0000 7fcab4fc0640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcab00d3190 0x7fcab00b1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:32.005 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:32.001+0000 7fcab4fc0640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcab00d3190 0x7fcab00b1f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55436/0 (socket says 172.21.15.27:55436) 2024-09-12T03:16:32.005 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:32.001+0000 7fcab4fc0640 1 -- 172.21.15.27:0/1967701860 learned_addr learned my addr 172.21.15.27:0/1967701860 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:16:32.005 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:32.001+0000 7fcaaffff640 1 --2- 172.21.15.27:0/1967701860 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcab00b6470 0x7fcab00ce750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:32.005 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:32.001+0000 7fcaaf7fe640 1 --2- 172.21.15.27:0/1967701860 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fcab00b6850 0x7fcab00b19c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:32.005 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:32.002+0000 7fcab4fc0640 1 -- 172.21.15.27:0/1967701860 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fcab00b6850 msgr2=0x7fcab00b19c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:32.005 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:32.002+0000 7fcab4fc0640 1 --2- 172.21.15.27:0/1967701860 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fcab00b6850 0x7fcab00b19c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:32.005 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:32.002+0000 7fcab4fc0640 1 -- 172.21.15.27:0/1967701860 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcab00b6470 msgr2=0x7fcab00ce750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:32.005 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:32.002+0000 7fcab4fc0640 1 --2- 172.21.15.27:0/1967701860 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcab00b6470 0x7fcab00ce750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:32.005 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:32.002+0000 7fcab4fc0640 1 -- 172.21.15.27:0/1967701860 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcab01f8ca0 con 0x7fcab00d3190 2024-09-12T03:16:32.005 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:32.002+0000 7fcab4fc0640 1 --2- 172.21.15.27:0/1967701860 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcab00d3190 0x7fcab00b1f00 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fcaa8007e40 tx=0x7fcaa800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:32.005 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:32.002+0000 7fcaacff9640 1 -- 172.21.15.27:0/1967701860 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaa8014620 con 0x7fcab00d3190 2024-09-12T03:16:32.006 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:32.002+0000 7fcaacff9640 1 -- 172.21.15.27:0/1967701860 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcaa8019070 con 0x7fcab00d3190 2024-09-12T03:16:32.006 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:32.002+0000 7fcab6a23640 1 -- 172.21.15.27:0/1967701860 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcab00af4a0 con 0x7fcab00d3190 2024-09-12T03:16:32.006 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:32.002+0000 7fcaacff9640 1 -- 172.21.15.27:0/1967701860 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaa8014a20 con 0x7fcab00d3190 2024-09-12T03:16:32.006 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:32.002+0000 7fcab6a23640 1 -- 172.21.15.27:0/1967701860 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcab00afa60 con 0x7fcab00d3190 2024-09-12T03:16:32.006 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:32.003+0000 7fcab6a23640 1 -- 172.21.15.27:0/1967701860 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcab00d35c0 con 0x7fcab00d3190 2024-09-12T03:16:32.006 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:32.003+0000 7fcaacff9640 1 -- 172.21.15.27:0/1967701860 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fcaa8014bc0 con 0x7fcab00d3190 2024-09-12T03:16:32.006 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:32.004+0000 7fcaacff9640 1 --2- 172.21.15.27:0/1967701860 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fca9c06be30 0x7fca9c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:32.006 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:32.004+0000 7fcaaffff640 1 --2- 172.21.15.27:0/1967701860 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fca9c06be30 0x7fca9c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:32.006 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:32.004+0000 7fcaacff9640 1 -- 172.21.15.27:0/1967701860 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fcaa808b560 con 0x7fcab00d3190 2024-09-12T03:16:32.006 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:32.005+0000 7fcaaffff640 1 --2- 172.21.15.27:0/1967701860 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fca9c06be30 0x7fca9c06e2f0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fcaa0009a20 tx=0x7fcaa0009190 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:32.008 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:32.009+0000 7fcaacff9640 1 -- 172.21.15.27:0/1967701860 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fcaa8059950 con 0x7fcab00d3190 2024-09-12T03:16:32.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:32.188+0000 7fcab6a23640 1 -- 172.21.15.27:0/1967701860 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcab00ba700 con 0x7fcab00d3190 2024-09-12T03:16:32.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:32.189+0000 7fcaacff9640 1 -- 172.21.15.27:0/1967701860 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16630 (secure 0 0 0) 0x7fcaa805dc50 con 0x7fcab00d3190 2024-09-12T03:16:32.189 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:16:32.189 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":20,"fsid":"d2df504e-2647-4385-a5db-9fdf69f487a0","created":"2024-09-12T03:16:07.778937+0000","modified":"2024-09-12T03:16:30.914204+0000","last_up_change":"2024-09-12T03:16:19.314778+0000","last_in_change":"2024-09-12T03:16:09.893086+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T03:16:20.097646+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-12T03:16:28.790791+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-12T03:16:29.909065+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1851116c-1fb3-4dc9-9772-f90e2ef4c355","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6817","nonce":3002321488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6819","nonce":3002321488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6823","nonce":3002321488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6821","nonce":3002321488}]},"public_addr":"172.21.15.27:6817/3002321488","cluster_addr":"172.21.15.27:6819/3002321488","heartbeat_back_addr":"172.21.15.27:6823/3002321488","heartbeat_front_addr":"172.21.15.27:6821/3002321488","state":["exists","up"]},{"osd":1,"uuid":"f38ae17d-73eb-4204-b99d-975fa806fb87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6809","nonce":1613102886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6811","nonce":1613102886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6815","nonce":1613102886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6813","nonce":1613102886}]},"public_addr":"172.21.15.27:6809/1613102886","cluster_addr":"172.21.15.27:6811/1613102886","heartbeat_back_addr":"172.21.15.27:6815/1613102886","heartbeat_front_addr":"172.21.15.27:6813/1613102886","state":["exists","up"]},{"osd":2,"uuid":"fb12dcc8-bdd0-42b8-b46d-8b10106224bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6825","nonce":3534723355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6827","nonce":3534723355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6831","nonce":3534723355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6829","nonce":3534723355}]},"public_addr":"172.21.15.27:6825/3534723355","cluster_addr":"172.21.15.27:6827/3534723355","heartbeat_back_addr":"172.21.15.27:6831/3534723355","heartbeat_front_addr":"172.21.15.27:6829/3534723355","state":["exists","up"]},{"osd":3,"uuid":"d74d3f49-9c24-4b9f-98e2-3ecd7b5ef3b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6801","nonce":80661598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6803","nonce":80661598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6807","nonce":80661598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6805","nonce":80661598}]},"public_addr":"172.21.15.27:6801/80661598","cluster_addr":"172.21.15.27:6803/80661598","heartbeat_back_addr":"172.21.15.27:6807/80661598","heartbeat_front_addr":"172.21.15.27:6805/80661598","state":["exists","up"]},{"osd":4,"uuid":"2bf123f2-1b0d-41c4-aa3b-805bab267e59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6817","nonce":3943189827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6819","nonce":3943189827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6823","nonce":3943189827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6821","nonce":3943189827}]},"public_addr":"172.21.15.45:6817/3943189827","cluster_addr":"172.21.15.45:6819/3943189827","heartbeat_back_addr":"172.21.15.45:6823/3943189827","heartbeat_front_addr":"172.21.15.45:6821/3943189827","state":["exists","up"]},{"osd":5,"uuid":"c49ba5da-91c6-4a86-a97b-4aba89c253ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6809","nonce":3620760330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6811","nonce":3620760330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6815","nonce":3620760330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6813","nonce":3620760330}]},"public_addr":"172.21.15.45:6809/3620760330","cluster_addr":"172.21.15.45:6811/3620760330","heartbeat_back_addr":"172.21.15.45:6815/3620760330","heartbeat_front_addr":"172.21.15.45:6813/3620760330","state":["exists","up"]},{"osd":6,"uuid":"b0123475-89f7-4a34-9e33-d797b99dd9dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6801","nonce":1989083510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6803","nonce":1989083510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6807","nonce":1989083510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6805","nonce":1989083510}]},"public_addr":"172.21.15.45:6801/1989083510","cluster_addr":"172.21.15.45:6803/1989083510","heartbeat_back_addr":"172.21.15.45:6807/1989083510","heartbeat_front_addr":"172.21.15.45:6805/1989083510","state":["exists","up"]},{"osd":7,"uuid":"4dad773d-b7d8-46fc-8a10-a972aad27b29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6825","nonce":1513300294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6827","nonce":1513300294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6831","nonce":1513300294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6829","nonce":1513300294}]},"public_addr":"172.21.15.45:6825/1513300294","cluster_addr":"172.21.15.45:6827/1513300294","heartbeat_back_addr":"172.21.15.45:6831/1513300294","heartbeat_front_addr":"172.21.15.45:6829/1513300294","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.881855+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:17.133252+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.735339+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.500742+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.477571+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:17.167667+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.470291+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.873743+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T03:16:32.191 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:32.191+0000 7fcab6a23640 1 -- 172.21.15.27:0/1967701860 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fca9c06be30 msgr2=0x7fca9c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:32.191 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:32.191+0000 7fcab6a23640 1 --2- 172.21.15.27:0/1967701860 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fca9c06be30 0x7fca9c06e2f0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fcaa0009a20 tx=0x7fcaa0009190 comp rx=0 tx=0).stop 2024-09-12T03:16:32.191 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:32.191+0000 7fcab6a23640 1 -- 172.21.15.27:0/1967701860 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcab00d3190 msgr2=0x7fcab00b1f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:32.191 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:32.191+0000 7fcab6a23640 1 --2- 172.21.15.27:0/1967701860 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcab00d3190 0x7fcab00b1f00 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fcaa8007e40 tx=0x7fcaa800e4b0 comp rx=0 tx=0).stop 2024-09-12T03:16:32.191 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:32.191+0000 7fcab6a23640 1 -- 172.21.15.27:0/1967701860 shutdown_connections 2024-09-12T03:16:32.191 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:32.191+0000 7fcab6a23640 1 --2- 172.21.15.27:0/1967701860 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fca9c06be30 0x7fca9c06e2f0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:32.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:32.191+0000 7fcab6a23640 1 --2- 172.21.15.27:0/1967701860 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcab00d3190 0x7fcab00b1f00 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:32.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:32.191+0000 7fcab6a23640 1 --2- 172.21.15.27:0/1967701860 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fcab00b6850 0x7fcab00b19c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:32.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:32.191+0000 7fcab6a23640 1 --2- 172.21.15.27:0/1967701860 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcab00b6470 0x7fcab00ce750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:32.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:32.191+0000 7fcab6a23640 1 -- 172.21.15.27:0/1967701860 >> 172.21.15.27:0/1967701860 conn(0x7fcab00d1200 msgr2=0x7fcab00b9700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:32.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:32.191+0000 7fcab6a23640 1 -- 172.21.15.27:0/1967701860 shutdown_connections 2024-09-12T03:16:32.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:32.191+0000 7fcab6a23640 1 -- 172.21.15.27:0/1967701860 wait complete. 2024-09-12T03:16:32.202 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-12T03:16:32.306 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:32.306+0000 7f513348d640 1 Processor -- start 2024-09-12T03:16:32.306 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:32.306+0000 7f513348d640 1 -- start start 2024-09-12T03:16:32.307 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:32.307+0000 7f513348d640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f512c0d4040 0x7f512c0b7ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:32.307 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:32.307+0000 7f513348d640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f512c0d2930 con 0x7f512c0d3c20 2024-09-12T03:16:32.307 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:32.307+0000 7f513348d640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f512c0d2c10 con 0x7f512c0b6240 2024-09-12T03:16:32.308 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:32.307+0000 7f513348d640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f512c0d2ef0 con 0x7f512c0d4040 2024-09-12T03:16:32.308 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:32.307+0000 7f5131a2a640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f512c0d4040 0x7f512c0b7ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:32.308 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:32.307+0000 7f5130a28640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f512c0d3c20 0x7f512c0b77c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:51398/0 (socket says 172.21.15.27:51398) 2024-09-12T03:16:32.308 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:32.307+0000 7f5130a28640 1 -- 172.21.15.27:0/1996669370 learned_addr learned my addr 172.21.15.27:0/1996669370 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:16:32.308 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:32.307+0000 7f5131a2a640 1 -- 172.21.15.27:0/1996669370 >> v1:172.21.15.45:6789/0 conn(0x7f512c0b6240 legacy=0x7f512c0b6620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:32.308 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:32.307+0000 7f5131a2a640 1 -- 172.21.15.27:0/1996669370 >> v1:172.21.15.27:6789/0 conn(0x7f512c0d3c20 legacy=0x7f512c0b77c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:32.308 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:32.308+0000 7f5131a2a640 1 -- 172.21.15.27:0/1996669370 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f512c0d31d0 con 0x7f512c0d4040 2024-09-12T03:16:32.308 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:32.308+0000 7f5123fff640 1 -- 172.21.15.27:0/1996669370 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 66465976 0 0) 0x7f512c0d2930 con 0x7f512c0d3c20 2024-09-12T03:16:32.309 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:32.308+0000 7f5131a2a640 1 --2- 172.21.15.27:0/1996669370 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f512c0d4040 0x7f512c0b7ed0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f5124009d90 tx=0x7f512402a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=387570fd6ac51efd server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:32.309 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:32.308+0000 7f5123fff640 1 -- 172.21.15.27:0/1996669370 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5124007740 con 0x7f512c0d4040 2024-09-12T03:16:32.309 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:32.308+0000 7f5123fff640 1 -- 172.21.15.27:0/1996669370 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51240078e0 con 0x7f512c0d4040 2024-09-12T03:16:32.310 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:32.308+0000 7f5123fff640 1 -- 172.21.15.27:0/1996669370 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5124007cb0 con 0x7f512c0d4040 2024-09-12T03:16:32.310 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:32.309+0000 7f513348d640 1 -- 172.21.15.27:0/1996669370 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f512c0d4040 msgr2=0x7f512c0b7ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:32.310 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:32.309+0000 7f513348d640 1 --2- 172.21.15.27:0/1996669370 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f512c0d4040 0x7f512c0b7ed0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f5124009d90 tx=0x7f512402a730 comp rx=0 tx=0).stop 2024-09-12T03:16:32.310 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:32.309+0000 7f513348d640 1 -- 172.21.15.27:0/1996669370 shutdown_connections 2024-09-12T03:16:32.310 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:32.309+0000 7f513348d640 1 --2- 172.21.15.27:0/1996669370 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f512c0d4040 0x7f512c0b7ed0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:32.310 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:32.309+0000 7f513348d640 1 -- 172.21.15.27:0/1996669370 >> 172.21.15.27:0/1996669370 conn(0x7f512c0d1420 msgr2=0x7f512c0d1830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:32.310 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:32.309+0000 7f513348d640 1 -- 172.21.15.27:0/1996669370 shutdown_connections 2024-09-12T03:16:32.310 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:32.309+0000 7f513348d640 1 -- 172.21.15.27:0/1996669370 wait complete. 2024-09-12T03:16:32.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:32.310+0000 7f513348d640 1 Processor -- start 2024-09-12T03:16:32.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:32.310+0000 7f513348d640 1 -- start start 2024-09-12T03:16:32.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:32.311+0000 7f513348d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f512c0b6240 0x7f512c15d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:32.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:32.311+0000 7f513348d640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f512c0d3c20 0x7f512c0abc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:32.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:32.311+0000 7f513348d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f512c0d4040 0x7f512c161910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:32.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:32.311+0000 7f513348d640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f512c0cfdf0 con 0x7f512c0d4040 2024-09-12T03:16:32.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:32.311+0000 7f513348d640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f512c0cfc70 con 0x7f512c0b6240 2024-09-12T03:16:32.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:32.311+0000 7f513348d640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f512c0b8410 con 0x7f512c0d3c20 2024-09-12T03:16:32.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:32.311+0000 7f5131a2a640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f512c0d4040 0x7f512c161910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:32.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:32.311+0000 7f5131a2a640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f512c0d4040 0x7f512c161910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55460/0 (socket says 172.21.15.27:55460) 2024-09-12T03:16:32.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:32.311+0000 7f5131a2a640 1 -- 172.21.15.27:0/696097095 learned_addr learned my addr 172.21.15.27:0/696097095 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:16:32.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:32.311+0000 7f5131229640 1 --2- 172.21.15.27:0/696097095 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f512c0b6240 0x7f512c15d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:32.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:32.311+0000 7f5130a28640 1 --2- 172.21.15.27:0/696097095 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f512c0d3c20 0x7f512c0abc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:32.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:32.311+0000 7f5131a2a640 1 -- 172.21.15.27:0/696097095 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f512c0d3c20 msgr2=0x7f512c0abc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:32.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:32.311+0000 7f5131a2a640 1 --2- 172.21.15.27:0/696097095 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f512c0d3c20 0x7f512c0abc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:32.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:32.311+0000 7f5131a2a640 1 -- 172.21.15.27:0/696097095 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f512c0b6240 msgr2=0x7f512c15d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:32.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:32.311+0000 7f5131a2a640 1 --2- 172.21.15.27:0/696097095 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f512c0b6240 0x7f512c15d5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:32.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:32.311+0000 7f5131a2a640 1 -- 172.21.15.27:0/696097095 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f512c160870 con 0x7f512c0d4040 2024-09-12T03:16:32.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:32.312+0000 7f5131a2a640 1 --2- 172.21.15.27:0/696097095 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f512c0d4040 0x7f512c161910 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f5124009d90 tx=0x7f5124004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:32.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:32.312+0000 7f5121ffb640 1 -- 172.21.15.27:0/696097095 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f512403f070 con 0x7f512c0d4040 2024-09-12T03:16:32.314 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:32.312+0000 7f5121ffb640 1 -- 172.21.15.27:0/696097095 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5124031c90 con 0x7f512c0d4040 2024-09-12T03:16:32.314 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:32.312+0000 7f513348d640 1 -- 172.21.15.27:0/696097095 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f512c160b50 con 0x7f512c0d4040 2024-09-12T03:16:32.314 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:32.312+0000 7f5121ffb640 1 -- 172.21.15.27:0/696097095 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5124043630 con 0x7f512c0d4040 2024-09-12T03:16:32.314 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:32.312+0000 7f513348d640 1 -- 172.21.15.27:0/696097095 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f512c161110 con 0x7f512c0d4040 2024-09-12T03:16:32.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:32.313+0000 7f513348d640 1 -- 172.21.15.27:0/696097095 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f512c0b8770 con 0x7f512c0d4040 2024-09-12T03:16:32.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:32.313+0000 7f5121ffb640 1 -- 172.21.15.27:0/696097095 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f512403b410 con 0x7f512c0d4040 2024-09-12T03:16:32.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:32.314+0000 7f5121ffb640 1 --2- 172.21.15.27:0/696097095 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f511006be30 0x7f511006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:32.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:32.314+0000 7f5121ffb640 1 -- 172.21.15.27:0/696097095 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f51240ac440 con 0x7f512c0d4040 2024-09-12T03:16:32.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:32.314+0000 7f5131229640 1 --2- 172.21.15.27:0/696097095 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f511006be30 0x7f511006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:32.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:32.314+0000 7f5131229640 1 --2- 172.21.15.27:0/696097095 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f511006be30 0x7f511006e2f0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f5128003d90 tx=0x7f5128009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:32.318 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:32.318+0000 7f5121ffb640 1 -- 172.21.15.27:0/696097095 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f512407aeb0 con 0x7f512c0d4040 2024-09-12T03:16:32.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:32.530+0000 7f513348d640 1 -- 172.21.15.27:0/696097095 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f512c0b41c0 con 0x7f512c0d4040 2024-09-12T03:16:32.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:32.570+0000 7f5121ffb640 1 -- 172.21.15.27:0/696097095 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v5) ==== 118+0+0 (secure 0 0 0) 0x7f512407a7b0 con 0x7f512c0d4040 2024-09-12T03:16:32.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:32.572+0000 7f513348d640 1 -- 172.21.15.27:0/696097095 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f511006be30 msgr2=0x7f511006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:32.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:32.572+0000 7f513348d640 1 --2- 172.21.15.27:0/696097095 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f511006be30 0x7f511006e2f0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f5128003d90 tx=0x7f5128009040 comp rx=0 tx=0).stop 2024-09-12T03:16:32.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:32.572+0000 7f513348d640 1 -- 172.21.15.27:0/696097095 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f512c0d4040 msgr2=0x7f512c161910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:32.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:32.572+0000 7f513348d640 1 --2- 172.21.15.27:0/696097095 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f512c0d4040 0x7f512c161910 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f5124009d90 tx=0x7f5124004a40 comp rx=0 tx=0).stop 2024-09-12T03:16:32.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:32.572+0000 7f513348d640 1 -- 172.21.15.27:0/696097095 shutdown_connections 2024-09-12T03:16:32.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:32.572+0000 7f513348d640 1 --2- 172.21.15.27:0/696097095 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f511006be30 0x7f511006e2f0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:32.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:32.572+0000 7f513348d640 1 --2- 172.21.15.27:0/696097095 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f512c0d4040 0x7f512c161910 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:32.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:32.572+0000 7f513348d640 1 --2- 172.21.15.27:0/696097095 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f512c0d3c20 0x7f512c0abc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:32.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:32.572+0000 7f513348d640 1 --2- 172.21.15.27:0/696097095 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f512c0b6240 0x7f512c15d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:32.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:32.572+0000 7f513348d640 1 -- 172.21.15.27:0/696097095 >> 172.21.15.27:0/696097095 conn(0x7f512c0d1420 msgr2=0x7f512c0b75f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:32.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:32.572+0000 7f513348d640 1 -- 172.21.15.27:0/696097095 shutdown_connections 2024-09-12T03:16:32.574 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:32.572+0000 7f513348d640 1 -- 172.21.15.27:0/696097095 wait complete. 2024-09-12T03:16:32.584 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-12T03:16:32.585 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-12T03:16:32.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:32.689+0000 7febac72c640 1 Processor -- start 2024-09-12T03:16:32.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:32.689+0000 7febac72c640 1 -- start start 2024-09-12T03:16:32.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:32.690+0000 7febac72c640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7feba40d3790 0x7feba40d1da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:32.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:32.690+0000 7febac72c640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feba40d2e30 con 0x7feba4162040 2024-09-12T03:16:32.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:32.690+0000 7febac72c640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feba4162420 con 0x7feba406dfd0 2024-09-12T03:16:32.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:32.690+0000 7febac72c640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7feba40d30d0 con 0x7feba40d3790 2024-09-12T03:16:32.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:32.691+0000 7febaacc9640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7feba40d3790 0x7feba40d1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:32.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:32.691+0000 7febaa4c8640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7feba4162040 0x7feba406d8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:51402/0 (socket says 172.21.15.27:51402) 2024-09-12T03:16:32.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:32.691+0000 7febaa4c8640 1 -- 172.21.15.27:0/2623883829 learned_addr learned my addr 172.21.15.27:0/2623883829 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:16:32.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:32.691+0000 7febaacc9640 1 -- 172.21.15.27:0/2623883829 >> v1:172.21.15.45:6789/0 conn(0x7feba406dfd0 legacy=0x7feba40cf320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:32.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:32.691+0000 7febaacc9640 1 -- 172.21.15.27:0/2623883829 >> v1:172.21.15.27:6789/0 conn(0x7feba4162040 legacy=0x7feba406d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:32.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:32.691+0000 7feba94c6640 1 -- 172.21.15.27:0/2623883829 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3456281451 0 0) 0x7feba40d2e30 con 0x7feba4162040 2024-09-12T03:16:32.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:32.691+0000 7febaacc9640 1 -- 172.21.15.27:0/2623883829 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feba40d33b0 con 0x7feba40d3790 2024-09-12T03:16:32.693 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:32.692+0000 7febaacc9640 1 --2- 172.21.15.27:0/2623883829 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feba40d3790 0x7feba40d1da0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7feb9c0097a0 tx=0x7feb9c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=271ade968dac066a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:32.693 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:32.692+0000 7feba94c6640 1 -- 172.21.15.27:0/2623883829 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb9c037070 con 0x7feba40d3790 2024-09-12T03:16:32.693 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:32.692+0000 7feba94c6640 1 -- 172.21.15.27:0/2623883829 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb9c032440 con 0x7feba40d3790 2024-09-12T03:16:32.693 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:32.692+0000 7feba94c6640 1 -- 172.21.15.27:0/2623883829 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb9c032810 con 0x7feba40d3790 2024-09-12T03:16:32.693 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:32.692+0000 7febac72c640 1 -- 172.21.15.27:0/2623883829 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feba40d3790 msgr2=0x7feba40d1da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:32.693 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:32.692+0000 7febac72c640 1 --2- 172.21.15.27:0/2623883829 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feba40d3790 0x7feba40d1da0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7feb9c0097a0 tx=0x7feb9c02a1e0 comp rx=0 tx=0).stop 2024-09-12T03:16:32.693 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:32.692+0000 7febac72c640 1 -- 172.21.15.27:0/2623883829 shutdown_connections 2024-09-12T03:16:32.693 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:32.692+0000 7febac72c640 1 --2- 172.21.15.27:0/2623883829 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feba40d3790 0x7feba40d1da0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:32.693 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:32.692+0000 7febac72c640 1 -- 172.21.15.27:0/2623883829 >> 172.21.15.27:0/2623883829 conn(0x7feba40b6700 msgr2=0x7feba40b6b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:32.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:32.692+0000 7febac72c640 1 -- 172.21.15.27:0/2623883829 shutdown_connections 2024-09-12T03:16:32.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:32.692+0000 7febac72c640 1 -- 172.21.15.27:0/2623883829 wait complete. 2024-09-12T03:16:32.696 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:32.693+0000 7febac72c640 1 Processor -- start 2024-09-12T03:16:32.696 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:32.694+0000 7febac72c640 1 -- start start 2024-09-12T03:16:32.696 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:32.694+0000 7febac72c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feba406dfd0 0x7feba414d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:32.696 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:32.694+0000 7febac72c640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7feba40d3790 0x7feba414fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:32.696 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:32.694+0000 7febac72c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feba4162040 0x7feba40b9af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:32.696 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:32.694+0000 7febac72c640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7feba40d4150 con 0x7feba4162040 2024-09-12T03:16:32.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:32.694+0000 7febac72c640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7feba40d4530 con 0x7feba406dfd0 2024-09-12T03:16:32.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:32.694+0000 7febac72c640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7feba40d42d0 con 0x7feba40d3790 2024-09-12T03:16:32.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:32.694+0000 7febaacc9640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feba4162040 0x7feba40b9af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:32.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:32.694+0000 7febaacc9640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feba4162040 0x7feba40b9af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55478/0 (socket says 172.21.15.27:55478) 2024-09-12T03:16:32.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:32.694+0000 7febaacc9640 1 -- 172.21.15.27:0/1720693078 learned_addr learned my addr 172.21.15.27:0/1720693078 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:16:32.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:32.695+0000 7febaa4c8640 1 --2- 172.21.15.27:0/1720693078 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feba406dfd0 0x7feba414d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:32.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:32.695+0000 7feba9cc7640 1 --2- 172.21.15.27:0/1720693078 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7feba40d3790 0x7feba414fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:32.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:32.695+0000 7febaacc9640 1 -- 172.21.15.27:0/1720693078 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7feba40d3790 msgr2=0x7feba414fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:32.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:32.695+0000 7febaacc9640 1 --2- 172.21.15.27:0/1720693078 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7feba40d3790 0x7feba414fd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:32.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:32.695+0000 7febaacc9640 1 -- 172.21.15.27:0/1720693078 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feba406dfd0 msgr2=0x7feba414d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:32.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:32.695+0000 7febaacc9640 1 --2- 172.21.15.27:0/1720693078 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feba406dfd0 0x7feba414d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:32.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:32.695+0000 7febaacc9640 1 -- 172.21.15.27:0/1720693078 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feba41506e0 con 0x7feba4162040 2024-09-12T03:16:32.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:32.695+0000 7febaacc9640 1 --2- 172.21.15.27:0/1720693078 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feba4162040 0x7feba40b9af0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7feb9c0098c0 tx=0x7feb9c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:32.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:32.695+0000 7feba2ffd640 1 -- 172.21.15.27:0/1720693078 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb9c041070 con 0x7feba4162040 2024-09-12T03:16:32.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:32.695+0000 7feba2ffd640 1 -- 172.21.15.27:0/1720693078 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb9c03c870 con 0x7feba4162040 2024-09-12T03:16:32.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:32.695+0000 7febac72c640 1 -- 172.21.15.27:0/1720693078 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feba41509c0 con 0x7feba4162040 2024-09-12T03:16:32.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:32.695+0000 7feba2ffd640 1 -- 172.21.15.27:0/1720693078 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb9c037040 con 0x7feba4162040 2024-09-12T03:16:32.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:32.695+0000 7febac72c640 1 -- 172.21.15.27:0/1720693078 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feba414ddc0 con 0x7feba4162040 2024-09-12T03:16:32.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:32.696+0000 7febac72c640 1 -- 172.21.15.27:0/1720693078 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feba406e6e0 con 0x7feba4162040 2024-09-12T03:16:32.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:32.696+0000 7feba2ffd640 1 -- 172.21.15.27:0/1720693078 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7feb9c044070 con 0x7feba4162040 2024-09-12T03:16:32.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:32.697+0000 7feba2ffd640 1 --2- 172.21.15.27:0/1720693078 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7feb9006be30 0x7feb9006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:32.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:32.697+0000 7feba2ffd640 1 -- 172.21.15.27:0/1720693078 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7feb9c0aba00 con 0x7feba4162040 2024-09-12T03:16:32.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:32.697+0000 7febaa4c8640 1 --2- 172.21.15.27:0/1720693078 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7feb9006be30 0x7feb9006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:32.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:32.698+0000 7febaa4c8640 1 --2- 172.21.15.27:0/1720693078 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7feb9006be30 0x7feb9006e2f0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7feb94007ed0 tx=0x7feb94007e60 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:32.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:32.702+0000 7feba2ffd640 1 -- 172.21.15.27:0/1720693078 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7feb9c07a3b0 con 0x7feba4162040 2024-09-12T03:16:32.912 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:32.912+0000 7febac72c640 1 -- 172.21.15.27:0/1720693078 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7feba406dcd0 con 0x7feba4162040 2024-09-12T03:16:33.577 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:33.577+0000 7feba2ffd640 1 -- 172.21.15.27:0/1720693078 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v6) ==== 122+0+0 (secure 0 0 0) 0x7feb9c079cb0 con 0x7feba4162040 2024-09-12T03:16:33.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:33.580+0000 7febac72c640 1 -- 172.21.15.27:0/1720693078 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7feb9006be30 msgr2=0x7feb9006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:33.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:33.580+0000 7febac72c640 1 --2- 172.21.15.27:0/1720693078 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7feb9006be30 0x7feb9006e2f0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7feb94007ed0 tx=0x7feb94007e60 comp rx=0 tx=0).stop 2024-09-12T03:16:33.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:33.580+0000 7febac72c640 1 -- 172.21.15.27:0/1720693078 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feba4162040 msgr2=0x7feba40b9af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:33.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:33.580+0000 7febac72c640 1 --2- 172.21.15.27:0/1720693078 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feba4162040 0x7feba40b9af0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7feb9c0098c0 tx=0x7feb9c0046c0 comp rx=0 tx=0).stop 2024-09-12T03:16:33.581 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:33.580+0000 7febac72c640 1 -- 172.21.15.27:0/1720693078 shutdown_connections 2024-09-12T03:16:33.581 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:33.580+0000 7febac72c640 1 --2- 172.21.15.27:0/1720693078 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7feb9006be30 0x7feb9006e2f0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:33.581 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:33.580+0000 7febac72c640 1 --2- 172.21.15.27:0/1720693078 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feba4162040 0x7feba40b9af0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:33.581 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:33.580+0000 7febac72c640 1 --2- 172.21.15.27:0/1720693078 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7feba40d3790 0x7feba414fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:33.581 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:33.580+0000 7febac72c640 1 --2- 172.21.15.27:0/1720693078 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feba406dfd0 0x7feba414d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:33.581 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:33.580+0000 7febac72c640 1 -- 172.21.15.27:0/1720693078 >> 172.21.15.27:0/1720693078 conn(0x7feba40b6700 msgr2=0x7feba40d2310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:33.581 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:33.580+0000 7febac72c640 1 -- 172.21.15.27:0/1720693078 shutdown_connections 2024-09-12T03:16:33.581 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:33.580+0000 7febac72c640 1 -- 172.21.15.27:0/1720693078 wait complete. 2024-09-12T03:16:33.594 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-12T03:16:33.594 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-12T03:16:33.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:33.703+0000 7f405270a640 1 Processor -- start 2024-09-12T03:16:33.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:33.703+0000 7f405270a640 1 -- start start 2024-09-12T03:16:33.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:33.704+0000 7f405270a640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f404c0d1050 0x7f404c0cf810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:33.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:33.704+0000 7f405270a640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f404c0d3120 0x7f404c0d35b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:33.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:33.704+0000 7f405270a640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f404c0d1e60 con 0x7f404c0d52e0 2024-09-12T03:16:33.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:33.704+0000 7f405270a640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f404c0d2380 con 0x7f404c0d1050 2024-09-12T03:16:33.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:33.704+0000 7f405270a640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f404c0cfef0 con 0x7f404c0d3120 2024-09-12T03:16:33.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:33.704+0000 7f404bfff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f404c0d1050 0x7f404c0cf810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:33.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:33.704+0000 7f4043fff640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f404c0d52e0 0x7f404c0d1430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:51404/0 (socket says 172.21.15.27:51404) 2024-09-12T03:16:33.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:33.704+0000 7f4043fff640 1 -- 172.21.15.27:0/815589754 learned_addr learned my addr 172.21.15.27:0/815589754 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:16:33.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:33.704+0000 7f4050ca7640 1 --2- 172.21.15.27:0/815589754 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f404c0d3120 0x7f404c0d35b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:33.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:33.704+0000 7f404bfff640 1 -- 172.21.15.27:0/815589754 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f404c0d3120 msgr2=0x7f404c0d35b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:33.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:33.704+0000 7f404bfff640 1 --2- 172.21.15.27:0/815589754 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f404c0d3120 0x7f404c0d35b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:33.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:33.704+0000 7f404bfff640 1 -- 172.21.15.27:0/815589754 >> v1:172.21.15.27:6789/0 conn(0x7f404c0d52e0 legacy=0x7f404c0d1430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:33.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:33.704+0000 7f404b7fe640 1 -- 172.21.15.27:0/815589754 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 973673916 0 0) 0x7f404c0d1e60 con 0x7f404c0d52e0 2024-09-12T03:16:33.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:33.704+0000 7f404bfff640 1 -- 172.21.15.27:0/815589754 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f404c0d2640 con 0x7f404c0d1050 2024-09-12T03:16:33.707 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:33.705+0000 7f404bfff640 1 --2- 172.21.15.27:0/815589754 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f404c0d1050 0x7f404c0cf810 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f403c0097a0 tx=0x7f403c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=306541d9cdf84bf5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:33.707 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:33.705+0000 7f404b7fe640 1 -- 172.21.15.27:0/815589754 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f403c037070 con 0x7f404c0d1050 2024-09-12T03:16:33.707 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:33.705+0000 7f404b7fe640 1 -- 172.21.15.27:0/815589754 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f403c032440 con 0x7f404c0d1050 2024-09-12T03:16:33.707 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:33.705+0000 7f404b7fe640 1 -- 172.21.15.27:0/815589754 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f403c032810 con 0x7f404c0d1050 2024-09-12T03:16:33.707 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:33.706+0000 7f405270a640 1 -- 172.21.15.27:0/815589754 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f404c0d1050 msgr2=0x7f404c0cf810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:33.708 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:33.706+0000 7f405270a640 1 --2- 172.21.15.27:0/815589754 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f404c0d1050 0x7f404c0cf810 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f403c0097a0 tx=0x7f403c02a180 comp rx=0 tx=0).stop 2024-09-12T03:16:33.708 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:33.706+0000 7f405270a640 1 -- 172.21.15.27:0/815589754 shutdown_connections 2024-09-12T03:16:33.708 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:33.706+0000 7f405270a640 1 --2- 172.21.15.27:0/815589754 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f404c0d3120 0x7f404c0d35b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:33.708 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:33.706+0000 7f405270a640 1 --2- 172.21.15.27:0/815589754 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f404c0d1050 0x7f404c0cf810 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:33.708 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:33.706+0000 7f405270a640 1 -- 172.21.15.27:0/815589754 >> 172.21.15.27:0/815589754 conn(0x7f404c06ddf0 msgr2=0x7f404c06e200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:33.708 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:33.706+0000 7f405270a640 1 -- 172.21.15.27:0/815589754 shutdown_connections 2024-09-12T03:16:33.708 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:33.706+0000 7f405270a640 1 -- 172.21.15.27:0/815589754 wait complete. 2024-09-12T03:16:33.708 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:33.707+0000 7f405270a640 1 Processor -- start 2024-09-12T03:16:33.708 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:33.707+0000 7f405270a640 1 -- start start 2024-09-12T03:16:33.710 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:33.708+0000 7f405270a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f404c0d1050 0x7f404c15d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:33.711 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:33.708+0000 7f405270a640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f404c0d3120 0x7f404c0c9ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:33.711 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:33.708+0000 7f405270a640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f404c0d52e0 0x7f404c0ca3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:33.711 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:33.708+0000 7f405270a640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f404c0b7a50 con 0x7f404c0d52e0 2024-09-12T03:16:33.711 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:33.708+0000 7f405270a640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f404c0d2c00 con 0x7f404c0d1050 2024-09-12T03:16:33.711 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:33.708+0000 7f405270a640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f404c0b7bd0 con 0x7f404c0d3120 2024-09-12T03:16:33.711 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:33.708+0000 7f4050ca7640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f404c0d52e0 0x7f404c0ca3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:33.711 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:33.708+0000 7f4050ca7640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f404c0d52e0 0x7f404c0ca3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55510/0 (socket says 172.21.15.27:55510) 2024-09-12T03:16:33.711 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:33.708+0000 7f4050ca7640 1 -- 172.21.15.27:0/679691376 learned_addr learned my addr 172.21.15.27:0/679691376 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:16:33.711 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:33.708+0000 7f4050ca7640 1 -- 172.21.15.27:0/679691376 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f404c0d3120 msgr2=0x7f404c0c9ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:33.711 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:33.708+0000 7f4050ca7640 1 --2- 172.21.15.27:0/679691376 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f404c0d3120 0x7f404c0c9ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:33.711 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:33.708+0000 7f4050ca7640 1 -- 172.21.15.27:0/679691376 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f404c0d1050 msgr2=0x7f404c15d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:33.711 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:33.708+0000 7f404bfff640 1 --2- 172.21.15.27:0/679691376 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f404c0d1050 0x7f404c15d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:33.711 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:33.708+0000 7f4050ca7640 1 --2- 172.21.15.27:0/679691376 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f404c0d1050 0x7f404c15d3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:33.711 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:33.708+0000 7f4050ca7640 1 -- 172.21.15.27:0/679691376 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f404c0cba20 con 0x7f404c0d52e0 2024-09-12T03:16:33.712 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:33.709+0000 7f4050ca7640 1 --2- 172.21.15.27:0/679691376 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f404c0d52e0 0x7f404c0ca3e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f4044007e40 tx=0x7f404400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:33.712 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:33.709+0000 7f40497fa640 1 -- 172.21.15.27:0/679691376 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4044014620 con 0x7f404c0d52e0 2024-09-12T03:16:33.712 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:33.709+0000 7f40497fa640 1 -- 172.21.15.27:0/679691376 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4044019070 con 0x7f404c0d52e0 2024-09-12T03:16:33.712 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:33.709+0000 7f405270a640 1 -- 172.21.15.27:0/679691376 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f404c0cbd00 con 0x7f404c0d52e0 2024-09-12T03:16:33.712 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:33.709+0000 7f40497fa640 1 -- 172.21.15.27:0/679691376 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4044014a20 con 0x7f404c0d52e0 2024-09-12T03:16:33.712 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:33.709+0000 7f405270a640 1 -- 172.21.15.27:0/679691376 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f404c0cc2c0 con 0x7f404c0d52e0 2024-09-12T03:16:33.712 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:33.710+0000 7f405270a640 1 -- 172.21.15.27:0/679691376 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f404c15ec60 con 0x7f404c0d52e0 2024-09-12T03:16:33.712 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:33.710+0000 7f40497fa640 1 -- 172.21.15.27:0/679691376 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4044014bc0 con 0x7f404c0d52e0 2024-09-12T03:16:33.712 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:33.711+0000 7f40497fa640 1 --2- 172.21.15.27:0/679691376 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f402c06be30 0x7f402c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:33.712 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:33.711+0000 7f40497fa640 1 -- 172.21.15.27:0/679691376 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f404408b6d0 con 0x7f404c0d52e0 2024-09-12T03:16:33.712 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:33.711+0000 7f404bfff640 1 --2- 172.21.15.27:0/679691376 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f402c06be30 0x7f402c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:33.713 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:33.711+0000 7f404bfff640 1 --2- 172.21.15.27:0/679691376 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f402c06be30 0x7f402c06e2f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f403c034000 tx=0x7f403c02e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:33.715 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:33.715+0000 7f40497fa640 1 -- 172.21.15.27:0/679691376 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f404405a110 con 0x7f404c0d52e0 2024-09-12T03:16:33.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:33.928+0000 7f405270a640 1 -- 172.21.15.27:0/679691376 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f404c0d20a0 con 0x7f404c0d52e0 2024-09-12T03:16:33.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:33.928+0000 7f40497fa640 1 -- 172.21.15.27:0/679691376 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+5214 (secure 0 0 0) 0x7f4044059a10 con 0x7f404c0d52e0 2024-09-12T03:16:33.929 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:16:33.929 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":6,"btime":"2024-09-12T03:16:33:576418+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6836/682326693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":682326693},{"type":"v1","addr":"172.21.15.45:6836","nonce":682326693}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4278,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6834/4084728235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":4084728235},{"type":"v1","addr":"172.21.15.27:6834","nonce":4084728235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4281,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/2844240887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":2844240887},{"type":"v1","addr":"172.21.15.27:6835","nonce":2844240887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-12T03:16:30.914517+0000","modified":"2024-09-12T03:16:32.913742+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.45:6837/1035926332","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6835","nonce":1035926332},{"type":"v1","addr":"172.21.15.45:6837","nonce":1035926332}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4285,"qdb_cluster":[4285]},"id":1}]} 2024-09-12T03:16:33.929 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 6 2024-09-12T03:16:33.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:33.931+0000 7f405270a640 1 -- 172.21.15.27:0/679691376 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f402c06be30 msgr2=0x7f402c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:33.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:33.931+0000 7f405270a640 1 --2- 172.21.15.27:0/679691376 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f402c06be30 0x7f402c06e2f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f403c034000 tx=0x7f403c02e000 comp rx=0 tx=0).stop 2024-09-12T03:16:33.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:33.931+0000 7f405270a640 1 -- 172.21.15.27:0/679691376 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f404c0d52e0 msgr2=0x7f404c0ca3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:33.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:33.931+0000 7f405270a640 1 --2- 172.21.15.27:0/679691376 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f404c0d52e0 0x7f404c0ca3e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f4044007e40 tx=0x7f404400e4b0 comp rx=0 tx=0).stop 2024-09-12T03:16:33.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:33.931+0000 7f405270a640 1 -- 172.21.15.27:0/679691376 shutdown_connections 2024-09-12T03:16:33.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:33.931+0000 7f405270a640 1 --2- 172.21.15.27:0/679691376 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f402c06be30 0x7f402c06e2f0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:33.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:33.931+0000 7f405270a640 1 --2- 172.21.15.27:0/679691376 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f404c0d52e0 0x7f404c0ca3e0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:33.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:33.931+0000 7f405270a640 1 --2- 172.21.15.27:0/679691376 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f404c0d3120 0x7f404c0c9ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:33.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:33.931+0000 7f405270a640 1 --2- 172.21.15.27:0/679691376 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f404c0d1050 0x7f404c15d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:33.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:33.931+0000 7f405270a640 1 -- 172.21.15.27:0/679691376 >> 172.21.15.27:0/679691376 conn(0x7f404c06ddf0 msgr2=0x7f404c0d1430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:33.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:33.931+0000 7f405270a640 1 -- 172.21.15.27:0/679691376 shutdown_connections 2024-09-12T03:16:33.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:33.931+0000 7f405270a640 1 -- 172.21.15.27:0/679691376 wait complete. 2024-09-12T03:16:33.942 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-12T03:16:34.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.044+0000 7fb96281e640 1 Processor -- start 2024-09-12T03:16:34.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.044+0000 7fb96281e640 1 -- start start 2024-09-12T03:16:34.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.044+0000 7fb96281e640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fb95c15eb60 0x7fb95c0d1280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:34.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.044+0000 7fb96281e640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb95c0b6690 con 0x7fb95c0d17c0 2024-09-12T03:16:34.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.044+0000 7fb96281e640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb95c0b6970 con 0x7fb95c15e780 2024-09-12T03:16:34.046 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.044+0000 7fb96281e640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7fb95c0b7950 con 0x7fb95c15eb60 2024-09-12T03:16:34.046 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.045+0000 7fb953fff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fb95c15eb60 0x7fb95c0d1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:34.046 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.045+0000 7fb960dbb640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fb95c0d17c0 0x7fb95c0b3390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:51410/0 (socket says 172.21.15.27:51410) 2024-09-12T03:16:34.046 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.045+0000 7fb960dbb640 1 -- 172.21.15.27:0/2047346564 learned_addr learned my addr 172.21.15.27:0/2047346564 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:16:34.046 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.045+0000 7fb953fff640 1 -- 172.21.15.27:0/2047346564 >> v1:172.21.15.45:6789/0 conn(0x7fb95c15e780 legacy=0x7fb95c0d3e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:34.046 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.045+0000 7fb953fff640 1 -- 172.21.15.27:0/2047346564 >> v1:172.21.15.27:6789/0 conn(0x7fb95c0d17c0 legacy=0x7fb95c0b3390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:34.046 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.045+0000 7fb953fff640 1 -- 172.21.15.27:0/2047346564 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb95c0b6c50 con 0x7fb95c15eb60 2024-09-12T03:16:34.046 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.046+0000 7fb953fff640 1 --2- 172.21.15.27:0/2047346564 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb95c15eb60 0x7fb95c0d1280 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb9440097a0 tx=0x7fb94402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e4ac62285014e32 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:34.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.046+0000 7fb95b7fe640 1 -- 172.21.15.27:0/2047346564 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb944037070 con 0x7fb95c15eb60 2024-09-12T03:16:34.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.046+0000 7fb95b7fe640 1 -- 172.21.15.27:0/2047346564 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb944032440 con 0x7fb95c15eb60 2024-09-12T03:16:34.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.046+0000 7fb95b7fe640 1 -- 172.21.15.27:0/2047346564 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb944032810 con 0x7fb95c15eb60 2024-09-12T03:16:34.048 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.046+0000 7fb96281e640 1 -- 172.21.15.27:0/2047346564 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb95c15eb60 msgr2=0x7fb95c0d1280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:34.048 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.046+0000 7fb96281e640 1 --2- 172.21.15.27:0/2047346564 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb95c15eb60 0x7fb95c0d1280 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb9440097a0 tx=0x7fb94402a1e0 comp rx=0 tx=0).stop 2024-09-12T03:16:34.048 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.046+0000 7fb96281e640 1 -- 172.21.15.27:0/2047346564 shutdown_connections 2024-09-12T03:16:34.048 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.046+0000 7fb96281e640 1 --2- 172.21.15.27:0/2047346564 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb95c15eb60 0x7fb95c0d1280 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:34.048 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.046+0000 7fb96281e640 1 -- 172.21.15.27:0/2047346564 >> 172.21.15.27:0/2047346564 conn(0x7fb95c06da60 msgr2=0x7fb95c06de70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:34.048 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.046+0000 7fb96281e640 1 -- 172.21.15.27:0/2047346564 shutdown_connections 2024-09-12T03:16:34.048 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.046+0000 7fb96281e640 1 -- 172.21.15.27:0/2047346564 wait complete. 2024-09-12T03:16:34.048 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.048+0000 7fb96281e640 1 Processor -- start 2024-09-12T03:16:34.048 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.048+0000 7fb96281e640 1 -- start start 2024-09-12T03:16:34.050 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.048+0000 7fb96281e640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb95c0d17c0 0x7fb95c0c9840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:34.050 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.048+0000 7fb96281e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb95c15e780 0x7fb95c158fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:34.050 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.048+0000 7fb96281e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb95c15eb60 0x7fb95c0ab310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:34.050 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.048+0000 7fb96281e640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fb95c0d1bf0 con 0x7fb95c15eb60 2024-09-12T03:16:34.050 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.048+0000 7fb96281e640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb95c0b7210 con 0x7fb95c15e780 2024-09-12T03:16:34.050 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.048+0000 7fb96281e640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fb95c0b6340 con 0x7fb95c0d17c0 2024-09-12T03:16:34.050 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.049+0000 7fb960dbb640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb95c15eb60 0x7fb95c0ab310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:34.050 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.049+0000 7fb960dbb640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb95c15eb60 0x7fb95c0ab310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55522/0 (socket says 172.21.15.27:55522) 2024-09-12T03:16:34.050 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.049+0000 7fb960dbb640 1 -- 172.21.15.27:0/1293820862 learned_addr learned my addr 172.21.15.27:0/1293820862 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:16:34.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.049+0000 7fb953fff640 1 --2- 172.21.15.27:0/1293820862 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb95c15e780 0x7fb95c158fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:34.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.049+0000 7fb95bfff640 1 --2- 172.21.15.27:0/1293820862 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb95c0d17c0 0x7fb95c0c9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:34.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.049+0000 7fb960dbb640 1 -- 172.21.15.27:0/1293820862 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb95c0d17c0 msgr2=0x7fb95c0c9840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:34.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.049+0000 7fb960dbb640 1 --2- 172.21.15.27:0/1293820862 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb95c0d17c0 0x7fb95c0c9840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:34.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.049+0000 7fb960dbb640 1 -- 172.21.15.27:0/1293820862 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb95c15e780 msgr2=0x7fb95c158fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:34.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.049+0000 7fb960dbb640 1 --2- 172.21.15.27:0/1293820862 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb95c15e780 0x7fb95c158fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:34.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.049+0000 7fb960dbb640 1 -- 172.21.15.27:0/1293820862 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb95c0ca090 con 0x7fb95c15eb60 2024-09-12T03:16:34.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.049+0000 7fb960dbb640 1 --2- 172.21.15.27:0/1293820862 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb95c15eb60 0x7fb95c0ab310 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fb95400ca80 tx=0x7fb95400cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:34.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.049+0000 7fb9597fa640 1 -- 172.21.15.27:0/1293820862 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb954017070 con 0x7fb95c15eb60 2024-09-12T03:16:34.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.049+0000 7fb9597fa640 1 -- 172.21.15.27:0/1293820862 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb95400ac30 con 0x7fb95c15eb60 2024-09-12T03:16:34.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.049+0000 7fb9597fa640 1 -- 172.21.15.27:0/1293820862 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb954012600 con 0x7fb95c15eb60 2024-09-12T03:16:34.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.049+0000 7fb96281e640 1 -- 172.21.15.27:0/1293820862 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb95c0ca370 con 0x7fb95c15eb60 2024-09-12T03:16:34.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.049+0000 7fb96281e640 1 -- 172.21.15.27:0/1293820862 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb95c0c9d80 con 0x7fb95c15eb60 2024-09-12T03:16:34.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.050+0000 7fb96281e640 1 -- 172.21.15.27:0/1293820862 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb95c0d4610 con 0x7fb95c15eb60 2024-09-12T03:16:34.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.051+0000 7fb9597fa640 1 -- 172.21.15.27:0/1293820862 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb95401a070 con 0x7fb95c15eb60 2024-09-12T03:16:34.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.051+0000 7fb9597fa640 1 --2- 172.21.15.27:0/1293820862 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fb93c06be30 0x7fb93c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:34.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.051+0000 7fb9597fa640 1 -- 172.21.15.27:0/1293820862 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fb95408aa20 con 0x7fb95c15eb60 2024-09-12T03:16:34.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.052+0000 7fb95bfff640 1 --2- 172.21.15.27:0/1293820862 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fb93c06be30 0x7fb93c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:34.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.052+0000 7fb95bfff640 1 --2- 172.21.15.27:0/1293820862 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fb93c06be30 0x7fb93c06e2f0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fb94c004f60 tx=0x7fb94c007e30 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:34.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.056+0000 7fb9597fa640 1 -- 172.21.15.27:0/1293820862 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fb95400ae20 con 0x7fb95c15eb60 2024-09-12T03:16:34.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.232+0000 7fb96281e640 1 -- 172.21.15.27:0/1293820862 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb95c0d1280 con 0x7fb95c15eb60 2024-09-12T03:16:34.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.233+0000 7fb9597fa640 1 -- 172.21.15.27:0/1293820862 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16617 (secure 0 0 0) 0x7fb954058e10 con 0x7fb95c15eb60 2024-09-12T03:16:34.234 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:16:34.234 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":21,"fsid":"d2df504e-2647-4385-a5db-9fdf69f487a0","created":"2024-09-12T03:16:07.778937+0000","modified":"2024-09-12T03:16:32.570182+0000","last_up_change":"2024-09-12T03:16:19.314778+0000","last_in_change":"2024-09-12T03:16:09.893086+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T03:16:20.097646+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-12T03:16:28.790791+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-12T03:16:29.909065+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1851116c-1fb3-4dc9-9772-f90e2ef4c355","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6817","nonce":3002321488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6819","nonce":3002321488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6823","nonce":3002321488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6821","nonce":3002321488}]},"public_addr":"172.21.15.27:6817/3002321488","cluster_addr":"172.21.15.27:6819/3002321488","heartbeat_back_addr":"172.21.15.27:6823/3002321488","heartbeat_front_addr":"172.21.15.27:6821/3002321488","state":["exists","up"]},{"osd":1,"uuid":"f38ae17d-73eb-4204-b99d-975fa806fb87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6809","nonce":1613102886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6811","nonce":1613102886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6815","nonce":1613102886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6813","nonce":1613102886}]},"public_addr":"172.21.15.27:6809/1613102886","cluster_addr":"172.21.15.27:6811/1613102886","heartbeat_back_addr":"172.21.15.27:6815/1613102886","heartbeat_front_addr":"172.21.15.27:6813/1613102886","state":["exists","up"]},{"osd":2,"uuid":"fb12dcc8-bdd0-42b8-b46d-8b10106224bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6825","nonce":3534723355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6827","nonce":3534723355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6831","nonce":3534723355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6829","nonce":3534723355}]},"public_addr":"172.21.15.27:6825/3534723355","cluster_addr":"172.21.15.27:6827/3534723355","heartbeat_back_addr":"172.21.15.27:6831/3534723355","heartbeat_front_addr":"172.21.15.27:6829/3534723355","state":["exists","up"]},{"osd":3,"uuid":"d74d3f49-9c24-4b9f-98e2-3ecd7b5ef3b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6801","nonce":80661598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6803","nonce":80661598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6807","nonce":80661598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6805","nonce":80661598}]},"public_addr":"172.21.15.27:6801/80661598","cluster_addr":"172.21.15.27:6803/80661598","heartbeat_back_addr":"172.21.15.27:6807/80661598","heartbeat_front_addr":"172.21.15.27:6805/80661598","state":["exists","up"]},{"osd":4,"uuid":"2bf123f2-1b0d-41c4-aa3b-805bab267e59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6817","nonce":3943189827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6819","nonce":3943189827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6823","nonce":3943189827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6821","nonce":3943189827}]},"public_addr":"172.21.15.45:6817/3943189827","cluster_addr":"172.21.15.45:6819/3943189827","heartbeat_back_addr":"172.21.15.45:6823/3943189827","heartbeat_front_addr":"172.21.15.45:6821/3943189827","state":["exists","up"]},{"osd":5,"uuid":"c49ba5da-91c6-4a86-a97b-4aba89c253ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6809","nonce":3620760330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6811","nonce":3620760330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6815","nonce":3620760330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6813","nonce":3620760330}]},"public_addr":"172.21.15.45:6809/3620760330","cluster_addr":"172.21.15.45:6811/3620760330","heartbeat_back_addr":"172.21.15.45:6815/3620760330","heartbeat_front_addr":"172.21.15.45:6813/3620760330","state":["exists","up"]},{"osd":6,"uuid":"b0123475-89f7-4a34-9e33-d797b99dd9dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6801","nonce":1989083510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6803","nonce":1989083510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6807","nonce":1989083510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6805","nonce":1989083510}]},"public_addr":"172.21.15.45:6801/1989083510","cluster_addr":"172.21.15.45:6803/1989083510","heartbeat_back_addr":"172.21.15.45:6807/1989083510","heartbeat_front_addr":"172.21.15.45:6805/1989083510","state":["exists","up"]},{"osd":7,"uuid":"4dad773d-b7d8-46fc-8a10-a972aad27b29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6825","nonce":1513300294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6827","nonce":1513300294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6831","nonce":1513300294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6829","nonce":1513300294}]},"public_addr":"172.21.15.45:6825/1513300294","cluster_addr":"172.21.15.45:6827/1513300294","heartbeat_back_addr":"172.21.15.45:6831/1513300294","heartbeat_front_addr":"172.21.15.45:6829/1513300294","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.881855+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:17.133252+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.735339+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.500742+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.477571+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:17.167667+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.470291+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.873743+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T03:16:34.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.236+0000 7fb96281e640 1 -- 172.21.15.27:0/1293820862 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fb93c06be30 msgr2=0x7fb93c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:34.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.236+0000 7fb96281e640 1 --2- 172.21.15.27:0/1293820862 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fb93c06be30 0x7fb93c06e2f0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fb94c004f60 tx=0x7fb94c007e30 comp rx=0 tx=0).stop 2024-09-12T03:16:34.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.236+0000 7fb96281e640 1 -- 172.21.15.27:0/1293820862 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb95c15eb60 msgr2=0x7fb95c0ab310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:34.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.236+0000 7fb96281e640 1 --2- 172.21.15.27:0/1293820862 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb95c15eb60 0x7fb95c0ab310 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fb95400ca80 tx=0x7fb95400cf40 comp rx=0 tx=0).stop 2024-09-12T03:16:34.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.236+0000 7fb96281e640 1 -- 172.21.15.27:0/1293820862 shutdown_connections 2024-09-12T03:16:34.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.236+0000 7fb96281e640 1 --2- 172.21.15.27:0/1293820862 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fb93c06be30 0x7fb93c06e2f0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:34.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.236+0000 7fb96281e640 1 --2- 172.21.15.27:0/1293820862 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb95c15eb60 0x7fb95c0ab310 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:34.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.236+0000 7fb96281e640 1 --2- 172.21.15.27:0/1293820862 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb95c15e780 0x7fb95c158fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:34.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.236+0000 7fb96281e640 1 --2- 172.21.15.27:0/1293820862 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb95c0d17c0 0x7fb95c0c9840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:34.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.236+0000 7fb96281e640 1 -- 172.21.15.27:0/1293820862 >> 172.21.15.27:0/1293820862 conn(0x7fb95c06da60 msgr2=0x7fb95c0b8c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:34.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.236+0000 7fb96281e640 1 -- 172.21.15.27:0/1293820862 shutdown_connections 2024-09-12T03:16:34.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.236+0000 7fb96281e640 1 -- 172.21.15.27:0/1293820862 wait complete. 2024-09-12T03:16:34.247 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-12T03:16:34.247 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-12T03:16:34.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.349+0000 7f94f344c640 1 Processor -- start 2024-09-12T03:16:34.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.350+0000 7f94f344c640 1 -- start start 2024-09-12T03:16:34.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.350+0000 7f94f344c640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f94ec15eb90 0x7f94ec0d2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:34.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.350+0000 7f94f344c640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f94ec0d1720 0x7f94ec0b8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:34.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.350+0000 7f94f344c640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f94ec0d3760 con 0x7f94ec0d5200 2024-09-12T03:16:34.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.350+0000 7f94f344c640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f94ec0b7d20 con 0x7f94ec15eb90 2024-09-12T03:16:34.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.350+0000 7f94f344c640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f94ec0d42e0 con 0x7f94ec0d1720 2024-09-12T03:16:34.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.350+0000 7f94f11e8640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f94ec15eb90 0x7f94ec0d2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:34.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.350+0000 7f94f09e7640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f94ec0d5200 0x7f94ec0d1010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:51412/0 (socket says 172.21.15.27:51412) 2024-09-12T03:16:34.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.350+0000 7f94f09e7640 1 -- 172.21.15.27:0/1165065432 learned_addr learned my addr 172.21.15.27:0/1165065432 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:16:34.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.351+0000 7f94f19e9640 1 --2- 172.21.15.27:0/1165065432 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f94ec0d1720 0x7f94ec0b8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:34.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.351+0000 7f94f11e8640 1 -- 172.21.15.27:0/1165065432 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f94ec0d1720 msgr2=0x7f94ec0b8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:34.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.351+0000 7f94f11e8640 1 --2- 172.21.15.27:0/1165065432 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f94ec0d1720 0x7f94ec0b8190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:34.352 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.351+0000 7f94f11e8640 1 -- 172.21.15.27:0/1165065432 >> v1:172.21.15.27:6789/0 conn(0x7f94ec0d5200 legacy=0x7f94ec0d1010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:34.352 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.351+0000 7f94f11e8640 1 -- 172.21.15.27:0/1165065432 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94ec0d3a40 con 0x7f94ec15eb90 2024-09-12T03:16:34.353 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.351+0000 7f94f11e8640 1 --2- 172.21.15.27:0/1165065432 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f94ec15eb90 0x7f94ec0d2df0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f94e80097a0 tx=0x7f94e802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a18fef405b0b9b69 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:34.353 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.352+0000 7f94e3fff640 1 -- 172.21.15.27:0/1165065432 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94e8037070 con 0x7f94ec15eb90 2024-09-12T03:16:34.353 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.352+0000 7f94e3fff640 1 -- 172.21.15.27:0/1165065432 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f94e8032440 con 0x7f94ec15eb90 2024-09-12T03:16:34.353 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.352+0000 7f94e3fff640 1 -- 172.21.15.27:0/1165065432 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94e8032810 con 0x7f94ec15eb90 2024-09-12T03:16:34.353 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.352+0000 7f94f344c640 1 -- 172.21.15.27:0/1165065432 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f94ec15eb90 msgr2=0x7f94ec0d2df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:34.353 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.352+0000 7f94f344c640 1 --2- 172.21.15.27:0/1165065432 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f94ec15eb90 0x7f94ec0d2df0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f94e80097a0 tx=0x7f94e802a180 comp rx=0 tx=0).stop 2024-09-12T03:16:34.353 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.352+0000 7f94f344c640 1 -- 172.21.15.27:0/1165065432 shutdown_connections 2024-09-12T03:16:34.353 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.352+0000 7f94f344c640 1 --2- 172.21.15.27:0/1165065432 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f94ec0d1720 0x7f94ec0b8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:34.353 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.352+0000 7f94f344c640 1 --2- 172.21.15.27:0/1165065432 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f94ec15eb90 0x7f94ec0d2df0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:34.353 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.352+0000 7f94f344c640 1 -- 172.21.15.27:0/1165065432 >> 172.21.15.27:0/1165065432 conn(0x7f94ec06dce0 msgr2=0x7f94ec06e0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:34.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.352+0000 7f94f344c640 1 -- 172.21.15.27:0/1165065432 shutdown_connections 2024-09-12T03:16:34.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.352+0000 7f94f344c640 1 -- 172.21.15.27:0/1165065432 wait complete. 2024-09-12T03:16:34.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.353+0000 7f94f344c640 1 Processor -- start 2024-09-12T03:16:34.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.354+0000 7f94f344c640 1 -- start start 2024-09-12T03:16:34.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.354+0000 7f94f344c640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f94ec0d1720 0x7f94ec1501b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:34.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.354+0000 7f94f344c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f94ec0d5200 0x7f94ec1590a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:34.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.354+0000 7f94f344c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f94ec15eb90 0x7f94ec154e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:34.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.354+0000 7f94f344c640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f94ec0d3330 con 0x7f94ec0d5200 2024-09-12T03:16:34.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.354+0000 7f94f344c640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f94ec0d4000 con 0x7f94ec15eb90 2024-09-12T03:16:34.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.354+0000 7f94f344c640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f94ec0d34b0 con 0x7f94ec0d1720 2024-09-12T03:16:34.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.354+0000 7f94f09e7640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f94ec0d5200 0x7f94ec1590a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:34.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.354+0000 7f94f09e7640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f94ec0d5200 0x7f94ec1590a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55532/0 (socket says 172.21.15.27:55532) 2024-09-12T03:16:34.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.354+0000 7f94f09e7640 1 -- 172.21.15.27:0/3989680182 learned_addr learned my addr 172.21.15.27:0/3989680182 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:16:34.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.354+0000 7f94f11e8640 1 --2- 172.21.15.27:0/3989680182 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f94ec0d1720 0x7f94ec1501b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:34.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.354+0000 7f94f19e9640 1 --2- 172.21.15.27:0/3989680182 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f94ec15eb90 0x7f94ec154e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:34.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.355+0000 7f94f09e7640 1 -- 172.21.15.27:0/3989680182 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f94ec0d1720 msgr2=0x7f94ec1501b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:34.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.355+0000 7f94f09e7640 1 --2- 172.21.15.27:0/3989680182 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f94ec0d1720 0x7f94ec1501b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:34.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.355+0000 7f94f09e7640 1 -- 172.21.15.27:0/3989680182 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f94ec15eb90 msgr2=0x7f94ec154e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:34.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.355+0000 7f94f09e7640 1 --2- 172.21.15.27:0/3989680182 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f94ec15eb90 0x7f94ec154e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:34.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.355+0000 7f94f09e7640 1 -- 172.21.15.27:0/3989680182 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94ec159a10 con 0x7f94ec0d5200 2024-09-12T03:16:34.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.355+0000 7f94f09e7640 1 --2- 172.21.15.27:0/3989680182 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f94ec0d5200 0x7f94ec1590a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f94dc00c470 tx=0x7f94dc001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:34.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.355+0000 7f94e1ffb640 1 -- 172.21.15.27:0/3989680182 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94dc017070 con 0x7f94ec0d5200 2024-09-12T03:16:34.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.355+0000 7f94e1ffb640 1 -- 172.21.15.27:0/3989680182 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f94dc005ce0 con 0x7f94ec0d5200 2024-09-12T03:16:34.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.355+0000 7f94f344c640 1 -- 172.21.15.27:0/3989680182 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94ec0cc9a0 con 0x7f94ec0d5200 2024-09-12T03:16:34.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.355+0000 7f94e1ffb640 1 -- 172.21.15.27:0/3989680182 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94dc0072f0 con 0x7f94ec0d5200 2024-09-12T03:16:34.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.355+0000 7f94f344c640 1 -- 172.21.15.27:0/3989680182 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94ec0ccf60 con 0x7f94ec0d5200 2024-09-12T03:16:34.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.356+0000 7f94f344c640 1 -- 172.21.15.27:0/3989680182 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94ec15f850 con 0x7f94ec0d5200 2024-09-12T03:16:34.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.357+0000 7f94e1ffb640 1 -- 172.21.15.27:0/3989680182 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f94dc01a070 con 0x7f94ec0d5200 2024-09-12T03:16:34.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.357+0000 7f94e1ffb640 1 --2- 172.21.15.27:0/3989680182 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f94d006be30 0x7f94d006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:34.359 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.357+0000 7f94e1ffb640 1 -- 172.21.15.27:0/3989680182 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f94dc08a1b0 con 0x7f94ec0d5200 2024-09-12T03:16:34.359 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.357+0000 7f94f11e8640 1 --2- 172.21.15.27:0/3989680182 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f94d006be30 0x7f94d006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:34.359 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.358+0000 7f94f11e8640 1 --2- 172.21.15.27:0/3989680182 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f94d006be30 0x7f94d006e2f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f94e8034000 tx=0x7f94e802e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:34.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.361+0000 7f94e1ffb640 1 -- 172.21.15.27:0/3989680182 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f94dc058510 con 0x7f94ec0d5200 2024-09-12T03:16:34.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.535+0000 7f94f344c640 1 -- 172.21.15.27:0/3989680182 --> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f94ec160250 con 0x7f94d006be30 2024-09-12T03:16:34.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.542+0000 7f94e1ffb640 1 -- 172.21.15.27:0/3989680182 <== mgr.4102 v2:172.21.15.45:6832/3923821219 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+240051 (secure 0 0 0) 0x7f94ec160250 con 0x7f94d006be30 2024-09-12T03:16:34.545 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:16:34.546 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2024-09-12T03:16:34.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.545+0000 7f94f344c640 1 -- 172.21.15.27:0/3989680182 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f94d006be30 msgr2=0x7f94d006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:34.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.545+0000 7f94f344c640 1 --2- 172.21.15.27:0/3989680182 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f94d006be30 0x7f94d006e2f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f94e8034000 tx=0x7f94e802e000 comp rx=0 tx=0).stop 2024-09-12T03:16:34.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.545+0000 7f94f344c640 1 -- 172.21.15.27:0/3989680182 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f94ec0d5200 msgr2=0x7f94ec1590a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:34.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.545+0000 7f94f344c640 1 --2- 172.21.15.27:0/3989680182 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f94ec0d5200 0x7f94ec1590a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f94dc00c470 tx=0x7f94dc001e70 comp rx=0 tx=0).stop 2024-09-12T03:16:34.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.545+0000 7f94f344c640 1 -- 172.21.15.27:0/3989680182 shutdown_connections 2024-09-12T03:16:34.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.545+0000 7f94f344c640 1 --2- 172.21.15.27:0/3989680182 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f94d006be30 0x7f94d006e2f0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:34.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.545+0000 7f94f344c640 1 --2- 172.21.15.27:0/3989680182 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f94ec15eb90 0x7f94ec154e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:34.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.545+0000 7f94f344c640 1 --2- 172.21.15.27:0/3989680182 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f94ec0d5200 0x7f94ec1590a0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:34.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.545+0000 7f94f344c640 1 --2- 172.21.15.27:0/3989680182 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f94ec0d1720 0x7f94ec1501b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:34.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.545+0000 7f94f344c640 1 -- 172.21.15.27:0/3989680182 >> 172.21.15.27:0/3989680182 conn(0x7f94ec06dce0 msgr2=0x7f94ec161030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:34.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.545+0000 7f94f344c640 1 -- 172.21.15.27:0/3989680182 shutdown_connections 2024-09-12T03:16:34.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.545+0000 7f94f344c640 1 -- 172.21.15.27:0/3989680182 wait complete. 2024-09-12T03:16:34.555 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2024-09-12T03:16:34.053081+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":8,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235376,"kb_used_data":2500,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754739344,"statfs":{"total":773094113280,"available":772853088256,"internally_reserved":0,"allocated":2560000,"data_stored":1601789,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.499689"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T03:16:29.555565+0000","last_change":"2024-09-12T03:16:29.555565+0000","last_active":"2024-09-12T03:16:29.548038+0000","last_peered":"2024-09-12T03:16:29.548038+0000","last_clean":"2024-09-12T03:16:29.548038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:29.555565+0000","last_undegraded":"2024-09-12T03:16:29.555565+0000","last_fullsized":"2024-09-12T03:16:29.555565+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T03:16:29.558592+0000","last_change":"2024-09-12T03:16:29.558592+0000","last_active":"2024-09-12T03:16:29.548038+0000","last_peered":"2024-09-12T03:16:29.548038+0000","last_clean":"2024-09-12T03:16:29.548038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:29.558592+0000","last_undegraded":"2024-09-12T03:16:29.558592+0000","last_fullsized":"2024-09-12T03:16:29.558592+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T03:16:29.555565+0000","last_change":"2024-09-12T03:16:29.555565+0000","last_active":"2024-09-12T03:16:29.548038+0000","last_peered":"2024-09-12T03:16:29.548038+0000","last_clean":"2024-09-12T03:16:29.548038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:29.555565+0000","last_undegraded":"2024-09-12T03:16:29.555565+0000","last_fullsized":"2024-09-12T03:16:29.555565+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T03:16:29.560599+0000","last_change":"2024-09-12T03:16:29.560599+0000","last_active":"2024-09-12T03:16:29.548038+0000","last_peered":"2024-09-12T03:16:29.548038+0000","last_clean":"2024-09-12T03:16:29.548038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:29.560599+0000","last_undegraded":"2024-09-12T03:16:29.560599+0000","last_fullsized":"2024-09-12T03:16:29.560599+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T03:16:29.559533+0000","last_change":"2024-09-12T03:16:29.559533+0000","last_active":"2024-09-12T03:16:29.548038+0000","last_peered":"2024-09-12T03:16:29.548038+0000","last_clean":"2024-09-12T03:16:29.548038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:29.559533+0000","last_undegraded":"2024-09-12T03:16:29.559533+0000","last_fullsized":"2024-09-12T03:16:29.559533+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T03:16:29.560546+0000","last_change":"2024-09-12T03:16:29.560546+0000","last_active":"2024-09-12T03:16:29.548038+0000","last_peered":"2024-09-12T03:16:29.548038+0000","last_clean":"2024-09-12T03:16:29.548038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:29.560546+0000","last_undegraded":"2024-09-12T03:16:29.560546+0000","last_fullsized":"2024-09-12T03:16:29.560546+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T03:16:29.560488+0000","last_change":"2024-09-12T03:16:29.560488+0000","last_active":"2024-09-12T03:16:29.548038+0000","last_peered":"2024-09-12T03:16:29.548038+0000","last_clean":"2024-09-12T03:16:29.548038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:29.560488+0000","last_undegraded":"2024-09-12T03:16:29.560488+0000","last_fullsized":"2024-09-12T03:16:29.560488+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T03:16:29.558583+0000","last_change":"2024-09-12T03:16:29.558583+0000","last_active":"2024-09-12T03:16:29.548038+0000","last_peered":"2024-09-12T03:16:29.548038+0000","last_clean":"2024-09-12T03:16:29.548038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:29.558583+0000","last_undegraded":"2024-09-12T03:16:29.558583+0000","last_fullsized":"2024-09-12T03:16:29.558583+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T03:16:29.558598+0000","last_change":"2024-09-12T03:16:29.558598+0000","last_active":"2024-09-12T03:16:29.548038+0000","last_peered":"2024-09-12T03:16:29.548038+0000","last_clean":"2024-09-12T03:16:29.548038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:29.558598+0000","last_undegraded":"2024-09-12T03:16:29.558598+0000","last_fullsized":"2024-09-12T03:16:29.558598+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T03:16:29.561109+0000","last_change":"2024-09-12T03:16:29.561109+0000","last_active":"2024-09-12T03:16:29.548038+0000","last_peered":"2024-09-12T03:16:29.548038+0000","last_clean":"2024-09-12T03:16:29.548038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:29.561109+0000","last_undegraded":"2024-09-12T03:16:29.561109+0000","last_fullsized":"2024-09-12T03:16:29.561109+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T03:16:29.558613+0000","last_change":"2024-09-12T03:16:29.558613+0000","last_active":"2024-09-12T03:16:29.548038+0000","last_peered":"2024-09-12T03:16:29.548038+0000","last_clean":"2024-09-12T03:16:29.548038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:29.558613+0000","last_undegraded":"2024-09-12T03:16:29.558613+0000","last_fullsized":"2024-09-12T03:16:29.558613+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T03:16:29.563582+0000","last_change":"2024-09-12T03:16:29.563582+0000","last_active":"2024-09-12T03:16:29.548038+0000","last_peered":"2024-09-12T03:16:29.548038+0000","last_clean":"2024-09-12T03:16:29.548038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:29.563582+0000","last_undegraded":"2024-09-12T03:16:29.563582+0000","last_fullsized":"2024-09-12T03:16:29.563582+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T03:16:29.555666+0000","last_change":"2024-09-12T03:16:29.555666+0000","last_active":"2024-09-12T03:16:29.548038+0000","last_peered":"2024-09-12T03:16:29.548038+0000","last_clean":"2024-09-12T03:16:29.548038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:29.555666+0000","last_undegraded":"2024-09-12T03:16:29.555666+0000","last_fullsized":"2024-09-12T03:16:29.555666+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T03:16:29.563556+0000","last_change":"2024-09-12T03:16:29.563556+0000","last_active":"2024-09-12T03:16:29.548038+0000","last_peered":"2024-09-12T03:16:29.548038+0000","last_clean":"2024-09-12T03:16:29.548038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:29.563556+0000","last_undegraded":"2024-09-12T03:16:29.563556+0000","last_fullsized":"2024-09-12T03:16:29.563556+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T03:16:29.559421+0000","last_change":"2024-09-12T03:16:29.559421+0000","last_active":"2024-09-12T03:16:29.548038+0000","last_peered":"2024-09-12T03:16:29.548038+0000","last_clean":"2024-09-12T03:16:29.548038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:29.559421+0000","last_undegraded":"2024-09-12T03:16:29.559421+0000","last_fullsized":"2024-09-12T03:16:29.559421+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T03:16:29.563546+0000","last_change":"2024-09-12T03:16:29.563546+0000","last_active":"2024-09-12T03:16:29.548038+0000","last_peered":"2024-09-12T03:16:29.548038+0000","last_clean":"2024-09-12T03:16:29.548038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:29.563546+0000","last_undegraded":"2024-09-12T03:16:29.563546+0000","last_fullsized":"2024-09-12T03:16:29.563546+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":164,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-12T03:16:29.557520+0000","last_change":"2024-09-12T03:16:21.883446+0000","last_active":"2024-09-12T03:16:29.557520+0000","last_peered":"2024-09-12T03:16:29.557520+0000","last_clean":"2024-09-12T03:16:29.557520+0000","last_became_active":"2024-09-12T03:16:21.882207+0000","last_became_peered":"2024-09-12T03:16:21.882207+0000","last_unstale":"2024-09-12T03:16:29.557520+0000","last_undegraded":"2024-09-12T03:16:29.557520+0000","last_fullsized":"2024-09-12T03:16:29.557520+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:20.316426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:20.316426+0000","last_clean_scrub_stamp":"2024-09-12T03:16:20.316426+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:12:15.424320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T03:16:29.560957+0000","last_change":"2024-09-12T03:16:29.560957+0000","last_active":"2024-09-12T03:16:29.548038+0000","last_peered":"2024-09-12T03:16:29.548038+0000","last_clean":"2024-09-12T03:16:29.548038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:29.560957+0000","last_undegraded":"2024-09-12T03:16:29.560957+0000","last_fullsized":"2024-09-12T03:16:29.560957+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T03:16:29.556208+0000","last_change":"2024-09-12T03:16:29.556208+0000","last_active":"2024-09-12T03:16:29.548038+0000","last_peered":"2024-09-12T03:16:29.548038+0000","last_clean":"2024-09-12T03:16:29.548038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:29.556208+0000","last_undegraded":"2024-09-12T03:16:29.556208+0000","last_fullsized":"2024-09-12T03:16:29.556208+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T03:16:29.563487+0000","last_change":"2024-09-12T03:16:29.563487+0000","last_active":"2024-09-12T03:16:29.548038+0000","last_peered":"2024-09-12T03:16:29.548038+0000","last_clean":"2024-09-12T03:16:29.548038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:29.563487+0000","last_undegraded":"2024-09-12T03:16:29.563487+0000","last_fullsized":"2024-09-12T03:16:29.563487+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T03:16:29.560584+0000","last_change":"2024-09-12T03:16:29.560584+0000","last_active":"2024-09-12T03:16:29.548038+0000","last_peered":"2024-09-12T03:16:29.548038+0000","last_clean":"2024-09-12T03:16:29.548038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:29.560584+0000","last_undegraded":"2024-09-12T03:16:29.560584+0000","last_fullsized":"2024-09-12T03:16:29.560584+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T03:16:29.559139+0000","last_change":"2024-09-12T03:16:29.559139+0000","last_active":"2024-09-12T03:16:29.548038+0000","last_peered":"2024-09-12T03:16:29.548038+0000","last_clean":"2024-09-12T03:16:29.548038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:29.559139+0000","last_undegraded":"2024-09-12T03:16:29.559139+0000","last_fullsized":"2024-09-12T03:16:29.559139+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T03:16:29.555833+0000","last_change":"2024-09-12T03:16:29.555833+0000","last_active":"2024-09-12T03:16:29.548038+0000","last_peered":"2024-09-12T03:16:29.548038+0000","last_clean":"2024-09-12T03:16:29.548038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:29.555833+0000","last_undegraded":"2024-09-12T03:16:29.555833+0000","last_fullsized":"2024-09-12T03:16:29.555833+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T03:16:29.559271+0000","last_change":"2024-09-12T03:16:29.559271+0000","last_active":"2024-09-12T03:16:29.548038+0000","last_peered":"2024-09-12T03:16:29.548038+0000","last_clean":"2024-09-12T03:16:29.548038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:29.559271+0000","last_undegraded":"2024-09-12T03:16:29.559271+0000","last_fullsized":"2024-09-12T03:16:29.559271+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T03:16:29.558849+0000","last_change":"2024-09-12T03:16:29.558849+0000","last_active":"2024-09-12T03:16:29.548038+0000","last_peered":"2024-09-12T03:16:29.548038+0000","last_clean":"2024-09-12T03:16:29.548038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:29.558849+0000","last_undegraded":"2024-09-12T03:16:29.558849+0000","last_fullsized":"2024-09-12T03:16:29.558849+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T03:16:29.557099+0000","last_change":"2024-09-12T03:16:29.557099+0000","last_active":"2024-09-12T03:16:29.548038+0000","last_peered":"2024-09-12T03:16:29.548038+0000","last_clean":"2024-09-12T03:16:29.548038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:29.557099+0000","last_undegraded":"2024-09-12T03:16:29.557099+0000","last_fullsized":"2024-09-12T03:16:29.557099+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T03:16:29.559156+0000","last_change":"2024-09-12T03:16:29.559156+0000","last_active":"2024-09-12T03:16:29.548038+0000","last_peered":"2024-09-12T03:16:29.548038+0000","last_clean":"2024-09-12T03:16:29.548038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:29.559156+0000","last_undegraded":"2024-09-12T03:16:29.559156+0000","last_fullsized":"2024-09-12T03:16:29.559156+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T03:16:29.558951+0000","last_change":"2024-09-12T03:16:29.558951+0000","last_active":"2024-09-12T03:16:29.548038+0000","last_peered":"2024-09-12T03:16:29.548038+0000","last_clean":"2024-09-12T03:16:29.548038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:29.558951+0000","last_undegraded":"2024-09-12T03:16:29.558951+0000","last_fullsized":"2024-09-12T03:16:29.558951+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T03:16:29.555670+0000","last_change":"2024-09-12T03:16:29.555670+0000","last_active":"2024-09-12T03:16:29.548038+0000","last_peered":"2024-09-12T03:16:29.548038+0000","last_clean":"2024-09-12T03:16:29.548038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:29.555670+0000","last_undegraded":"2024-09-12T03:16:29.555670+0000","last_fullsized":"2024-09-12T03:16:29.555670+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T03:16:29.556203+0000","last_change":"2024-09-12T03:16:29.556203+0000","last_active":"2024-09-12T03:16:29.548038+0000","last_peered":"2024-09-12T03:16:29.548038+0000","last_clean":"2024-09-12T03:16:29.548038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:29.556203+0000","last_undegraded":"2024-09-12T03:16:29.556203+0000","last_fullsized":"2024-09-12T03:16:29.556203+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T03:16:29.556594+0000","last_change":"2024-09-12T03:16:29.556594+0000","last_active":"2024-09-12T03:16:29.548038+0000","last_peered":"2024-09-12T03:16:29.548038+0000","last_clean":"2024-09-12T03:16:29.548038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:29.556594+0000","last_undegraded":"2024-09-12T03:16:29.556594+0000","last_fullsized":"2024-09-12T03:16:29.556594+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T03:16:29.555746+0000","last_change":"2024-09-12T03:16:29.555746+0000","last_active":"2024-09-12T03:16:29.548038+0000","last_peered":"2024-09-12T03:16:29.548038+0000","last_clean":"2024-09-12T03:16:29.548038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:29.555746+0000","last_undegraded":"2024-09-12T03:16:29.555746+0000","last_fullsized":"2024-09-12T03:16:29.555746+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T03:16:29.558604+0000","last_change":"2024-09-12T03:16:29.558604+0000","last_active":"2024-09-12T03:16:29.548038+0000","last_peered":"2024-09-12T03:16:29.548038+0000","last_clean":"2024-09-12T03:16:29.548038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:29.558604+0000","last_undegraded":"2024-09-12T03:16:29.558604+0000","last_fullsized":"2024-09-12T03:16:29.558604+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542147,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31940,"kb_used_data":752,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339900,"statfs":{"total":96636764160,"available":96604057600,"internally_reserved":0,"allocated":770048,"data_stored":645181,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542149,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31352,"kb_used_data":160,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340488,"statfs":{"total":96636764160,"available":96604659712,"internally_reserved":0,"allocated":163840,"data_stored":48996,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542148,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31360,"kb_used_data":172,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340480,"statfs":{"total":96636764160,"available":96604651520,"internally_reserved":0,"allocated":176128,"data_stored":54813,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542147,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31352,"kb_used_data":160,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340488,"statfs":{"total":96636764160,"available":96604659712,"internally_reserved":0,"allocated":163840,"data_stored":48996,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542148,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27192,"kb_used_data":160,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344648,"statfs":{"total":96636764160,"available":96608919552,"internally_reserved":0,"allocated":163840,"data_stored":48996,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542148,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27200,"kb_used_data":172,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344640,"statfs":{"total":96636764160,"available":96608911360,"internally_reserved":0,"allocated":176128,"data_stored":54813,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542149,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27200,"kb_used_data":172,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344640,"statfs":{"total":96636764160,"available":96608911360,"internally_reserved":0,"allocated":176128,"data_stored":54813,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542149,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27780,"kb_used_data":752,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344060,"statfs":{"total":96636764160,"available":96608317440,"internally_reserved":0,"allocated":770048,"data_stored":645181,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T03:16:34.560 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-12T03:16:34.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.698+0000 7f0b16fe4640 1 Processor -- start 2024-09-12T03:16:34.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.698+0000 7f0b16fe4640 1 -- start start 2024-09-12T03:16:34.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.699+0000 7f0b16fe4640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f0b100b6470 0x7f0b100d2710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:34.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.699+0000 7f0b16fe4640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f0b100b6850 0x7f0b100d2c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:34.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.699+0000 7f0b16fe4640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0b100d1ec0 con 0x7f0b100d3190 2024-09-12T03:16:34.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.699+0000 7f0b16fe4640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f0b100cf980 con 0x7f0b100b6470 2024-09-12T03:16:34.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.699+0000 7f0b16fe4640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f0b100b79b0 con 0x7f0b100b6850 2024-09-12T03:16:34.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.699+0000 7f0b15581640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f0b100d3190 0x7f0b10161b80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:51424/0 (socket says 172.21.15.27:51424) 2024-09-12T03:16:34.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.699+0000 7f0b15581640 1 -- 172.21.15.27:0/675378364 learned_addr learned my addr 172.21.15.27:0/675378364 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:16:34.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.699+0000 7f0b0ffff640 1 --2- 172.21.15.27:0/675378364 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f0b100b6850 0x7f0b100d2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:34.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.699+0000 7f0b0f7fe640 1 -- 172.21.15.27:0/675378364 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3865132725 0 0) 0x7f0b100d1ec0 con 0x7f0b100d3190 2024-09-12T03:16:34.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.699+0000 7f0b14d80640 1 --2- 172.21.15.27:0/675378364 >> v2:172.21.15.45:3300/0 conn(0x7f0b100b6470 0x7f0b100d2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:34.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.699+0000 7f0b0f7fe640 1 -- 172.21.15.27:0/675378364 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0b00003520 con 0x7f0b100d3190 2024-09-12T03:16:34.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.699+0000 7f0b0f7fe640 1 -- 172.21.15.27:0/675378364 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2400696814 0 0) 0x7f0b00003520 con 0x7f0b100d3190 2024-09-12T03:16:34.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.699+0000 7f0b0f7fe640 1 -- 172.21.15.27:0/675378364 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f0b100b6850 msgr2=0x7f0b100d2c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:34.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.699+0000 7f0b0f7fe640 1 --2- 172.21.15.27:0/675378364 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f0b100b6850 0x7f0b100d2c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:34.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.699+0000 7f0b0f7fe640 1 -- 172.21.15.27:0/675378364 >> v2:172.21.15.45:3300/0 conn(0x7f0b100b6470 msgr2=0x7f0b100d2710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:34.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.699+0000 7f0b0f7fe640 1 --2- 172.21.15.27:0/675378364 >> v2:172.21.15.45:3300/0 conn(0x7f0b100b6470 0x7f0b100d2710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:34.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.699+0000 7f0b0f7fe640 1 -- 172.21.15.27:0/675378364 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b100cb580 con 0x7f0b100d3190 2024-09-12T03:16:34.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.699+0000 7f0b0f7fe640 1 -- 172.21.15.27:0/675378364 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 847755213 0 0) 0x7f0b080019f0 con 0x7f0b100d3190 2024-09-12T03:16:34.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.699+0000 7f0b0f7fe640 1 -- 172.21.15.27:0/675378364 >> v1:172.21.15.27:6789/0 conn(0x7f0b100d3190 legacy=0x7f0b10161b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:34.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.699+0000 7f0b0f7fe640 1 --2- 172.21.15.27:0/675378364 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f0b100b6850 0x7f0b000039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:34.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.699+0000 7f0b0f7fe640 1 --2- 172.21.15.27:0/675378364 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0b00003f30 0x7f0b00004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:34.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.700+0000 7f0b0f7fe640 1 --2- 172.21.15.27:0/675378364 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0b00004d40 0x7f0b000051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:34.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.700+0000 7f0b0f7fe640 1 -- 172.21.15.27:0/675378364 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f0b100cf980 con 0x7f0b00004d40 2024-09-12T03:16:34.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.700+0000 7f0b0f7fe640 1 -- 172.21.15.27:0/675378364 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f0b100b79b0 con 0x7f0b00003f30 2024-09-12T03:16:34.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.700+0000 7f0b0f7fe640 1 -- 172.21.15.27:0/675378364 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f0b0000b9c0 con 0x7f0b100b6850 2024-09-12T03:16:34.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.700+0000 7f0b15581640 1 --2- 172.21.15.27:0/675378364 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0b00004d40 0x7f0b000051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:34.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.700+0000 7f0b14d80640 1 --2- 172.21.15.27:0/675378364 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f0b100b6850 0x7f0b000039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:34.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.700+0000 7f0b0ffff640 1 --2- 172.21.15.27:0/675378364 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0b00003f30 0x7f0b00004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:34.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.700+0000 7f0b15581640 1 -- 172.21.15.27:0/675378364 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f0b100b6850 msgr2=0x7f0b000039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:34.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.700+0000 7f0b15581640 1 --2- 172.21.15.27:0/675378364 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f0b100b6850 0x7f0b000039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:34.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.700+0000 7f0b15581640 1 -- 172.21.15.27:0/675378364 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0b00003f30 msgr2=0x7f0b00004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:34.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.700+0000 7f0b15581640 1 --2- 172.21.15.27:0/675378364 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0b00003f30 0x7f0b00004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:34.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.700+0000 7f0b15581640 1 -- 172.21.15.27:0/675378364 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b00003520 con 0x7f0b00004d40 2024-09-12T03:16:34.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.700+0000 7f0b14d80640 1 --2- 172.21.15.27:0/675378364 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f0b100b6850 0x7f0b000039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:16:34.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.701+0000 7f0b15581640 1 --2- 172.21.15.27:0/675378364 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0b00004d40 0x7f0b000051c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f0b0800a8a0 tx=0x7f0b00005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6c1a6e1ac63501b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:34.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.701+0000 7f0b0f7fe640 1 -- 172.21.15.27:0/675378364 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b0800a4c0 con 0x7f0b00004d40 2024-09-12T03:16:34.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.701+0000 7f0b0f7fe640 1 -- 172.21.15.27:0/675378364 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b0802f070 con 0x7f0b00004d40 2024-09-12T03:16:34.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.701+0000 7f0b0f7fe640 1 -- 172.21.15.27:0/675378364 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b0802bd70 con 0x7f0b00004d40 2024-09-12T03:16:34.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.701+0000 7f0b16fe4640 1 -- 172.21.15.27:0/675378364 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0b00004d40 msgr2=0x7f0b000051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:34.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.701+0000 7f0b16fe4640 1 --2- 172.21.15.27:0/675378364 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0b00004d40 0x7f0b000051c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f0b0800a8a0 tx=0x7f0b00005830 comp rx=0 tx=0).stop 2024-09-12T03:16:34.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.701+0000 7f0b15581640 1 -- 172.21.15.27:0/675378364 reap_dead start 2024-09-12T03:16:34.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.701+0000 7f0b16fe4640 1 -- 172.21.15.27:0/675378364 shutdown_connections 2024-09-12T03:16:34.707 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.701+0000 7f0b16fe4640 1 -- 172.21.15.27:0/675378364 >> 172.21.15.27:0/675378364 conn(0x7f0b100d1200 msgr2=0x7f0b100d1610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:34.707 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.701+0000 7f0b16fe4640 1 -- 172.21.15.27:0/675378364 shutdown_connections 2024-09-12T03:16:34.707 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.701+0000 7f0b16fe4640 1 -- 172.21.15.27:0/675378364 wait complete. 2024-09-12T03:16:34.707 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.702+0000 7f0b16fe4640 1 Processor -- start 2024-09-12T03:16:34.707 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.703+0000 7f0b16fe4640 1 -- start start 2024-09-12T03:16:34.707 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.703+0000 7f0b16fe4640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0b00004d40 0x7f0b101f7970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:34.707 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.703+0000 7f0b16fe4640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f0b100b16b0 0x7f0b100b1b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:34.707 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.703+0000 7f0b16fe4640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0b101f4e80 0x7f0b101f5330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:34.707 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.703+0000 7f0b16fe4640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f0b1006d9d0 con 0x7f0b101f4e80 2024-09-12T03:16:34.707 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.703+0000 7f0b16fe4640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f0b1006d850 con 0x7f0b00004d40 2024-09-12T03:16:34.707 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.703+0000 7f0b16fe4640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f0b100d3fa0 con 0x7f0b100b16b0 2024-09-12T03:16:34.707 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.704+0000 7f0b15581640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0b101f4e80 0x7f0b101f5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:34.707 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.704+0000 7f0b15581640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0b101f4e80 0x7f0b101f5330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55548/0 (socket says 172.21.15.27:55548) 2024-09-12T03:16:34.707 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.704+0000 7f0b15581640 1 -- 172.21.15.27:0/3389092358 learned_addr learned my addr 172.21.15.27:0/3389092358 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:16:34.708 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.704+0000 7f0b0ffff640 1 --2- 172.21.15.27:0/3389092358 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f0b100b16b0 0x7f0b100b1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:34.708 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.704+0000 7f0b14d80640 1 --2- 172.21.15.27:0/3389092358 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0b00004d40 0x7f0b101f7970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:34.708 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.704+0000 7f0b15581640 1 -- 172.21.15.27:0/3389092358 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f0b100b16b0 msgr2=0x7f0b100b1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:34.708 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.704+0000 7f0b15581640 1 --2- 172.21.15.27:0/3389092358 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f0b100b16b0 0x7f0b100b1b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:34.708 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.704+0000 7f0b15581640 1 -- 172.21.15.27:0/3389092358 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0b00004d40 msgr2=0x7f0b101f7970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:34.708 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.704+0000 7f0b15581640 1 --2- 172.21.15.27:0/3389092358 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0b00004d40 0x7f0b101f7970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:34.708 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.704+0000 7f0b15581640 1 -- 172.21.15.27:0/3389092358 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b100b2380 con 0x7f0b101f4e80 2024-09-12T03:16:34.708 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.704+0000 7f0b15581640 1 --2- 172.21.15.27:0/3389092358 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0b101f4e80 0x7f0b101f5330 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f0b0800a9c0 tx=0x7f0b08001dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:34.708 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.704+0000 7f0b0d7fa640 1 -- 172.21.15.27:0/3389092358 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b08044070 con 0x7f0b101f4e80 2024-09-12T03:16:34.708 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.704+0000 7f0b0d7fa640 1 -- 172.21.15.27:0/3389092358 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b080019e0 con 0x7f0b101f4e80 2024-09-12T03:16:34.708 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.704+0000 7f0b16fe4640 1 -- 172.21.15.27:0/3389092358 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b100b2660 con 0x7f0b101f4e80 2024-09-12T03:16:34.708 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.704+0000 7f0b0d7fa640 1 -- 172.21.15.27:0/3389092358 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b0800a4c0 con 0x7f0b101f4e80 2024-09-12T03:16:34.708 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.704+0000 7f0b16fe4640 1 -- 172.21.15.27:0/3389092358 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b100b2c20 con 0x7f0b101f4e80 2024-09-12T03:16:34.708 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.705+0000 7f0b16fe4640 1 -- 172.21.15.27:0/3389092358 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b100d26f0 con 0x7f0b101f4e80 2024-09-12T03:16:34.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.706+0000 7f0b0d7fa640 1 -- 172.21.15.27:0/3389092358 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0b08047070 con 0x7f0b101f4e80 2024-09-12T03:16:34.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.706+0000 7f0b0d7fa640 1 --2- 172.21.15.27:0/3389092358 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f0af406be30 0x7f0af406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:34.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.706+0000 7f0b0d7fa640 1 -- 172.21.15.27:0/3389092358 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f0b080aaed0 con 0x7f0b101f4e80 2024-09-12T03:16:34.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.706+0000 7f0b14d80640 1 --2- 172.21.15.27:0/3389092358 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f0af406be30 0x7f0af406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:34.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.707+0000 7f0b14d80640 1 --2- 172.21.15.27:0/3389092358 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f0af406be30 0x7f0af406e2f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f0b1015f4b0 tx=0x7f0b04008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:34.710 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.711+0000 7f0b0d7fa640 1 -- 172.21.15.27:0/3389092358 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f0b100d26f0 con 0x7f0b101f4e80 2024-09-12T03:16:34.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.882+0000 7f0b16fe4640 1 -- 172.21.15.27:0/3389092358 --> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f0b100d3600 con 0x7f0af406be30 2024-09-12T03:16:34.888 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.888+0000 7f0b0d7fa640 1 -- 172.21.15.27:0/3389092358 <== mgr.4102 v2:172.21.15.45:6832/3923821219 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+240051 (secure 0 0 0) 0x7f0b100d3600 con 0x7f0af406be30 2024-09-12T03:16:34.891 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:16:34.892 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2024-09-12T03:16:34.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.891+0000 7f0b16fe4640 1 -- 172.21.15.27:0/3389092358 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f0af406be30 msgr2=0x7f0af406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:34.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.891+0000 7f0b16fe4640 1 --2- 172.21.15.27:0/3389092358 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f0af406be30 0x7f0af406e2f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f0b1015f4b0 tx=0x7f0b04008040 comp rx=0 tx=0).stop 2024-09-12T03:16:34.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.891+0000 7f0b16fe4640 1 -- 172.21.15.27:0/3389092358 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0b101f4e80 msgr2=0x7f0b101f5330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:34.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.891+0000 7f0b16fe4640 1 --2- 172.21.15.27:0/3389092358 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0b101f4e80 0x7f0b101f5330 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f0b0800a9c0 tx=0x7f0b08001dc0 comp rx=0 tx=0).stop 2024-09-12T03:16:34.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.892+0000 7f0b16fe4640 1 -- 172.21.15.27:0/3389092358 shutdown_connections 2024-09-12T03:16:34.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.892+0000 7f0b16fe4640 1 --2- 172.21.15.27:0/3389092358 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f0af406be30 0x7f0af406e2f0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:34.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.892+0000 7f0b16fe4640 1 --2- 172.21.15.27:0/3389092358 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0b101f4e80 0x7f0b101f5330 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:34.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.892+0000 7f0b16fe4640 1 --2- 172.21.15.27:0/3389092358 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f0b100b16b0 0x7f0b100b1b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:34.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.892+0000 7f0b16fe4640 1 --2- 172.21.15.27:0/3389092358 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0b00004d40 0x7f0b101f7970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:34.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.892+0000 7f0b16fe4640 1 -- 172.21.15.27:0/3389092358 >> 172.21.15.27:0/3389092358 conn(0x7f0b100d1200 msgr2=0x7f0b100ce7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:34.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.892+0000 7f0b16fe4640 1 -- 172.21.15.27:0/3389092358 shutdown_connections 2024-09-12T03:16:34.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:34.892+0000 7f0b16fe4640 1 -- 172.21.15.27:0/3389092358 wait complete. 2024-09-12T03:16:34.902 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2024-09-12T03:16:34.053081+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":8,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235376,"kb_used_data":2500,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754739344,"statfs":{"total":773094113280,"available":772853088256,"internally_reserved":0,"allocated":2560000,"data_stored":1601789,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.499689"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T03:16:29.555565+0000","last_change":"2024-09-12T03:16:29.555565+0000","last_active":"2024-09-12T03:16:29.548038+0000","last_peered":"2024-09-12T03:16:29.548038+0000","last_clean":"2024-09-12T03:16:29.548038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:29.555565+0000","last_undegraded":"2024-09-12T03:16:29.555565+0000","last_fullsized":"2024-09-12T03:16:29.555565+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T03:16:29.558592+0000","last_change":"2024-09-12T03:16:29.558592+0000","last_active":"2024-09-12T03:16:29.548038+0000","last_peered":"2024-09-12T03:16:29.548038+0000","last_clean":"2024-09-12T03:16:29.548038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:29.558592+0000","last_undegraded":"2024-09-12T03:16:29.558592+0000","last_fullsized":"2024-09-12T03:16:29.558592+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T03:16:29.555565+0000","last_change":"2024-09-12T03:16:29.555565+0000","last_active":"2024-09-12T03:16:29.548038+0000","last_peered":"2024-09-12T03:16:29.548038+0000","last_clean":"2024-09-12T03:16:29.548038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:29.555565+0000","last_undegraded":"2024-09-12T03:16:29.555565+0000","last_fullsized":"2024-09-12T03:16:29.555565+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T03:16:29.560599+0000","last_change":"2024-09-12T03:16:29.560599+0000","last_active":"2024-09-12T03:16:29.548038+0000","last_peered":"2024-09-12T03:16:29.548038+0000","last_clean":"2024-09-12T03:16:29.548038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:29.560599+0000","last_undegraded":"2024-09-12T03:16:29.560599+0000","last_fullsized":"2024-09-12T03:16:29.560599+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T03:16:29.559533+0000","last_change":"2024-09-12T03:16:29.559533+0000","last_active":"2024-09-12T03:16:29.548038+0000","last_peered":"2024-09-12T03:16:29.548038+0000","last_clean":"2024-09-12T03:16:29.548038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:29.559533+0000","last_undegraded":"2024-09-12T03:16:29.559533+0000","last_fullsized":"2024-09-12T03:16:29.559533+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T03:16:29.560546+0000","last_change":"2024-09-12T03:16:29.560546+0000","last_active":"2024-09-12T03:16:29.548038+0000","last_peered":"2024-09-12T03:16:29.548038+0000","last_clean":"2024-09-12T03:16:29.548038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:29.560546+0000","last_undegraded":"2024-09-12T03:16:29.560546+0000","last_fullsized":"2024-09-12T03:16:29.560546+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T03:16:29.560488+0000","last_change":"2024-09-12T03:16:29.560488+0000","last_active":"2024-09-12T03:16:29.548038+0000","last_peered":"2024-09-12T03:16:29.548038+0000","last_clean":"2024-09-12T03:16:29.548038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:29.560488+0000","last_undegraded":"2024-09-12T03:16:29.560488+0000","last_fullsized":"2024-09-12T03:16:29.560488+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T03:16:29.558583+0000","last_change":"2024-09-12T03:16:29.558583+0000","last_active":"2024-09-12T03:16:29.548038+0000","last_peered":"2024-09-12T03:16:29.548038+0000","last_clean":"2024-09-12T03:16:29.548038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:29.558583+0000","last_undegraded":"2024-09-12T03:16:29.558583+0000","last_fullsized":"2024-09-12T03:16:29.558583+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T03:16:29.558598+0000","last_change":"2024-09-12T03:16:29.558598+0000","last_active":"2024-09-12T03:16:29.548038+0000","last_peered":"2024-09-12T03:16:29.548038+0000","last_clean":"2024-09-12T03:16:29.548038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:29.558598+0000","last_undegraded":"2024-09-12T03:16:29.558598+0000","last_fullsized":"2024-09-12T03:16:29.558598+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T03:16:29.561109+0000","last_change":"2024-09-12T03:16:29.561109+0000","last_active":"2024-09-12T03:16:29.548038+0000","last_peered":"2024-09-12T03:16:29.548038+0000","last_clean":"2024-09-12T03:16:29.548038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:29.561109+0000","last_undegraded":"2024-09-12T03:16:29.561109+0000","last_fullsized":"2024-09-12T03:16:29.561109+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T03:16:29.558613+0000","last_change":"2024-09-12T03:16:29.558613+0000","last_active":"2024-09-12T03:16:29.548038+0000","last_peered":"2024-09-12T03:16:29.548038+0000","last_clean":"2024-09-12T03:16:29.548038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:29.558613+0000","last_undegraded":"2024-09-12T03:16:29.558613+0000","last_fullsized":"2024-09-12T03:16:29.558613+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T03:16:29.563582+0000","last_change":"2024-09-12T03:16:29.563582+0000","last_active":"2024-09-12T03:16:29.548038+0000","last_peered":"2024-09-12T03:16:29.548038+0000","last_clean":"2024-09-12T03:16:29.548038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:29.563582+0000","last_undegraded":"2024-09-12T03:16:29.563582+0000","last_fullsized":"2024-09-12T03:16:29.563582+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T03:16:29.555666+0000","last_change":"2024-09-12T03:16:29.555666+0000","last_active":"2024-09-12T03:16:29.548038+0000","last_peered":"2024-09-12T03:16:29.548038+0000","last_clean":"2024-09-12T03:16:29.548038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:29.555666+0000","last_undegraded":"2024-09-12T03:16:29.555666+0000","last_fullsized":"2024-09-12T03:16:29.555666+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T03:16:29.563556+0000","last_change":"2024-09-12T03:16:29.563556+0000","last_active":"2024-09-12T03:16:29.548038+0000","last_peered":"2024-09-12T03:16:29.548038+0000","last_clean":"2024-09-12T03:16:29.548038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:29.563556+0000","last_undegraded":"2024-09-12T03:16:29.563556+0000","last_fullsized":"2024-09-12T03:16:29.563556+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T03:16:29.559421+0000","last_change":"2024-09-12T03:16:29.559421+0000","last_active":"2024-09-12T03:16:29.548038+0000","last_peered":"2024-09-12T03:16:29.548038+0000","last_clean":"2024-09-12T03:16:29.548038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:29.559421+0000","last_undegraded":"2024-09-12T03:16:29.559421+0000","last_fullsized":"2024-09-12T03:16:29.559421+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T03:16:29.563546+0000","last_change":"2024-09-12T03:16:29.563546+0000","last_active":"2024-09-12T03:16:29.548038+0000","last_peered":"2024-09-12T03:16:29.548038+0000","last_clean":"2024-09-12T03:16:29.548038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:29.563546+0000","last_undegraded":"2024-09-12T03:16:29.563546+0000","last_fullsized":"2024-09-12T03:16:29.563546+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":164,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-12T03:16:29.557520+0000","last_change":"2024-09-12T03:16:21.883446+0000","last_active":"2024-09-12T03:16:29.557520+0000","last_peered":"2024-09-12T03:16:29.557520+0000","last_clean":"2024-09-12T03:16:29.557520+0000","last_became_active":"2024-09-12T03:16:21.882207+0000","last_became_peered":"2024-09-12T03:16:21.882207+0000","last_unstale":"2024-09-12T03:16:29.557520+0000","last_undegraded":"2024-09-12T03:16:29.557520+0000","last_fullsized":"2024-09-12T03:16:29.557520+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:20.316426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:20.316426+0000","last_clean_scrub_stamp":"2024-09-12T03:16:20.316426+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:12:15.424320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T03:16:29.560957+0000","last_change":"2024-09-12T03:16:29.560957+0000","last_active":"2024-09-12T03:16:29.548038+0000","last_peered":"2024-09-12T03:16:29.548038+0000","last_clean":"2024-09-12T03:16:29.548038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:29.560957+0000","last_undegraded":"2024-09-12T03:16:29.560957+0000","last_fullsized":"2024-09-12T03:16:29.560957+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T03:16:29.556208+0000","last_change":"2024-09-12T03:16:29.556208+0000","last_active":"2024-09-12T03:16:29.548038+0000","last_peered":"2024-09-12T03:16:29.548038+0000","last_clean":"2024-09-12T03:16:29.548038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:29.556208+0000","last_undegraded":"2024-09-12T03:16:29.556208+0000","last_fullsized":"2024-09-12T03:16:29.556208+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T03:16:29.563487+0000","last_change":"2024-09-12T03:16:29.563487+0000","last_active":"2024-09-12T03:16:29.548038+0000","last_peered":"2024-09-12T03:16:29.548038+0000","last_clean":"2024-09-12T03:16:29.548038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:29.563487+0000","last_undegraded":"2024-09-12T03:16:29.563487+0000","last_fullsized":"2024-09-12T03:16:29.563487+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T03:16:29.560584+0000","last_change":"2024-09-12T03:16:29.560584+0000","last_active":"2024-09-12T03:16:29.548038+0000","last_peered":"2024-09-12T03:16:29.548038+0000","last_clean":"2024-09-12T03:16:29.548038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:29.560584+0000","last_undegraded":"2024-09-12T03:16:29.560584+0000","last_fullsized":"2024-09-12T03:16:29.560584+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T03:16:29.559139+0000","last_change":"2024-09-12T03:16:29.559139+0000","last_active":"2024-09-12T03:16:29.548038+0000","last_peered":"2024-09-12T03:16:29.548038+0000","last_clean":"2024-09-12T03:16:29.548038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:29.559139+0000","last_undegraded":"2024-09-12T03:16:29.559139+0000","last_fullsized":"2024-09-12T03:16:29.559139+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T03:16:29.555833+0000","last_change":"2024-09-12T03:16:29.555833+0000","last_active":"2024-09-12T03:16:29.548038+0000","last_peered":"2024-09-12T03:16:29.548038+0000","last_clean":"2024-09-12T03:16:29.548038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:29.555833+0000","last_undegraded":"2024-09-12T03:16:29.555833+0000","last_fullsized":"2024-09-12T03:16:29.555833+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T03:16:29.559271+0000","last_change":"2024-09-12T03:16:29.559271+0000","last_active":"2024-09-12T03:16:29.548038+0000","last_peered":"2024-09-12T03:16:29.548038+0000","last_clean":"2024-09-12T03:16:29.548038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:29.559271+0000","last_undegraded":"2024-09-12T03:16:29.559271+0000","last_fullsized":"2024-09-12T03:16:29.559271+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T03:16:29.558849+0000","last_change":"2024-09-12T03:16:29.558849+0000","last_active":"2024-09-12T03:16:29.548038+0000","last_peered":"2024-09-12T03:16:29.548038+0000","last_clean":"2024-09-12T03:16:29.548038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:29.558849+0000","last_undegraded":"2024-09-12T03:16:29.558849+0000","last_fullsized":"2024-09-12T03:16:29.558849+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T03:16:29.557099+0000","last_change":"2024-09-12T03:16:29.557099+0000","last_active":"2024-09-12T03:16:29.548038+0000","last_peered":"2024-09-12T03:16:29.548038+0000","last_clean":"2024-09-12T03:16:29.548038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:29.557099+0000","last_undegraded":"2024-09-12T03:16:29.557099+0000","last_fullsized":"2024-09-12T03:16:29.557099+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T03:16:29.559156+0000","last_change":"2024-09-12T03:16:29.559156+0000","last_active":"2024-09-12T03:16:29.548038+0000","last_peered":"2024-09-12T03:16:29.548038+0000","last_clean":"2024-09-12T03:16:29.548038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:29.559156+0000","last_undegraded":"2024-09-12T03:16:29.559156+0000","last_fullsized":"2024-09-12T03:16:29.559156+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T03:16:29.558951+0000","last_change":"2024-09-12T03:16:29.558951+0000","last_active":"2024-09-12T03:16:29.548038+0000","last_peered":"2024-09-12T03:16:29.548038+0000","last_clean":"2024-09-12T03:16:29.548038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:29.558951+0000","last_undegraded":"2024-09-12T03:16:29.558951+0000","last_fullsized":"2024-09-12T03:16:29.558951+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T03:16:29.555670+0000","last_change":"2024-09-12T03:16:29.555670+0000","last_active":"2024-09-12T03:16:29.548038+0000","last_peered":"2024-09-12T03:16:29.548038+0000","last_clean":"2024-09-12T03:16:29.548038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:29.555670+0000","last_undegraded":"2024-09-12T03:16:29.555670+0000","last_fullsized":"2024-09-12T03:16:29.555670+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T03:16:29.556203+0000","last_change":"2024-09-12T03:16:29.556203+0000","last_active":"2024-09-12T03:16:29.548038+0000","last_peered":"2024-09-12T03:16:29.548038+0000","last_clean":"2024-09-12T03:16:29.548038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:29.556203+0000","last_undegraded":"2024-09-12T03:16:29.556203+0000","last_fullsized":"2024-09-12T03:16:29.556203+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T03:16:29.556594+0000","last_change":"2024-09-12T03:16:29.556594+0000","last_active":"2024-09-12T03:16:29.548038+0000","last_peered":"2024-09-12T03:16:29.548038+0000","last_clean":"2024-09-12T03:16:29.548038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:29.556594+0000","last_undegraded":"2024-09-12T03:16:29.556594+0000","last_fullsized":"2024-09-12T03:16:29.556594+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T03:16:29.555746+0000","last_change":"2024-09-12T03:16:29.555746+0000","last_active":"2024-09-12T03:16:29.548038+0000","last_peered":"2024-09-12T03:16:29.548038+0000","last_clean":"2024-09-12T03:16:29.548038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:29.555746+0000","last_undegraded":"2024-09-12T03:16:29.555746+0000","last_fullsized":"2024-09-12T03:16:29.555746+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T03:16:29.558604+0000","last_change":"2024-09-12T03:16:29.558604+0000","last_active":"2024-09-12T03:16:29.548038+0000","last_peered":"2024-09-12T03:16:29.548038+0000","last_clean":"2024-09-12T03:16:29.548038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:29.558604+0000","last_undegraded":"2024-09-12T03:16:29.558604+0000","last_fullsized":"2024-09-12T03:16:29.558604+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542147,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31940,"kb_used_data":752,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339900,"statfs":{"total":96636764160,"available":96604057600,"internally_reserved":0,"allocated":770048,"data_stored":645181,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542149,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31352,"kb_used_data":160,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340488,"statfs":{"total":96636764160,"available":96604659712,"internally_reserved":0,"allocated":163840,"data_stored":48996,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542148,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31360,"kb_used_data":172,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340480,"statfs":{"total":96636764160,"available":96604651520,"internally_reserved":0,"allocated":176128,"data_stored":54813,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542147,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31352,"kb_used_data":160,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340488,"statfs":{"total":96636764160,"available":96604659712,"internally_reserved":0,"allocated":163840,"data_stored":48996,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542148,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27192,"kb_used_data":160,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344648,"statfs":{"total":96636764160,"available":96608919552,"internally_reserved":0,"allocated":163840,"data_stored":48996,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542148,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27200,"kb_used_data":172,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344640,"statfs":{"total":96636764160,"available":96608911360,"internally_reserved":0,"allocated":176128,"data_stored":54813,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542149,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27200,"kb_used_data":172,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344640,"statfs":{"total":96636764160,"available":96608911360,"internally_reserved":0,"allocated":176128,"data_stored":54813,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542149,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27780,"kb_used_data":752,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344060,"statfs":{"total":96636764160,"available":96608317440,"internally_reserved":0,"allocated":770048,"data_stored":645181,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T03:16:34.906 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-12T03:16:35.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.045+0000 7f5a36303640 1 Processor -- start 2024-09-12T03:16:35.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.045+0000 7f5a36303640 1 -- start start 2024-09-12T03:16:35.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.046+0000 7f5a36303640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f5a300cda40 0x7f5a3015ca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:35.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.046+0000 7f5a36303640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5a300b4d70 con 0x7f5a30146010 2024-09-12T03:16:35.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.046+0000 7f5a36303640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5a3006d140 con 0x7f5a30145c30 2024-09-12T03:16:35.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.046+0000 7f5a36303640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f5a300b5050 con 0x7f5a300cda40 2024-09-12T03:16:35.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.046+0000 7f5a348a0640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f5a30146010 0x7f5a300b4040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:51432/0 (socket says 172.21.15.27:51432) 2024-09-12T03:16:35.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.046+0000 7f5a348a0640 1 -- 172.21.15.27:0/3654983125 learned_addr learned my addr 172.21.15.27:0/3654983125 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:16:35.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.046+0000 7f5a2f7fe640 1 --2- 172.21.15.27:0/3654983125 >> v2:172.21.15.45:3300/0 conn(0x7f5a300cda40 0x7f5a3015ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:35.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.046+0000 7f5a2effd640 1 -- 172.21.15.27:0/3654983125 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 474406156 0 0) 0x7f5a300b4d70 con 0x7f5a30146010 2024-09-12T03:16:35.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.046+0000 7f5a2effd640 1 -- 172.21.15.27:0/3654983125 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5a18003520 con 0x7f5a30146010 2024-09-12T03:16:35.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.046+0000 7f5a2f7fe640 1 -- 172.21.15.27:0/3654983125 >> v1:172.21.15.45:6789/0 conn(0x7f5a30145c30 legacy=0x7f5a300cd330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:35.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.046+0000 7f5a2f7fe640 1 -- 172.21.15.27:0/3654983125 >> v1:172.21.15.27:6789/0 conn(0x7f5a30146010 legacy=0x7f5a300b4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:35.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.046+0000 7f5a2f7fe640 1 -- 172.21.15.27:0/3654983125 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a3006d420 con 0x7f5a300cda40 2024-09-12T03:16:35.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.047+0000 7f5a2f7fe640 1 --2- 172.21.15.27:0/3654983125 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5a300cda40 0x7f5a3015ca00 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f5a240097a0 tx=0x7f5a2402a1a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aff06b63e275f891 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:35.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.047+0000 7f5a2effd640 1 -- 172.21.15.27:0/3654983125 <== mon.1 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a24037070 con 0x7f5a300cda40 2024-09-12T03:16:35.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.047+0000 7f5a2effd640 1 -- 172.21.15.27:0/3654983125 <== mon.1 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a24032440 con 0x7f5a300cda40 2024-09-12T03:16:35.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.047+0000 7f5a2effd640 1 -- 172.21.15.27:0/3654983125 <== mon.1 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a24032810 con 0x7f5a300cda40 2024-09-12T03:16:35.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.048+0000 7f5a36303640 1 -- 172.21.15.27:0/3654983125 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5a300cda40 msgr2=0x7f5a3015ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:35.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.048+0000 7f5a36303640 1 --2- 172.21.15.27:0/3654983125 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5a300cda40 0x7f5a3015ca00 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f5a240097a0 tx=0x7f5a2402a1a0 comp rx=0 tx=0).stop 2024-09-12T03:16:35.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.048+0000 7f5a36303640 1 -- 172.21.15.27:0/3654983125 shutdown_connections 2024-09-12T03:16:35.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.048+0000 7f5a36303640 1 --2- 172.21.15.27:0/3654983125 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5a300cda40 0x7f5a3015ca00 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:35.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.048+0000 7f5a36303640 1 -- 172.21.15.27:0/3654983125 >> 172.21.15.27:0/3654983125 conn(0x7f5a30062c60 msgr2=0x7f5a3006bb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:35.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.048+0000 7f5a36303640 1 -- 172.21.15.27:0/3654983125 shutdown_connections 2024-09-12T03:16:35.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.048+0000 7f5a36303640 1 -- 172.21.15.27:0/3654983125 wait complete. 2024-09-12T03:16:35.050 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.049+0000 7f5a36303640 1 Processor -- start 2024-09-12T03:16:35.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.049+0000 7f5a36303640 1 -- start start 2024-09-12T03:16:35.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.050+0000 7f5a36303640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5a300cda40 0x7f5a301450e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:35.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.050+0000 7f5a36303640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f5a30145c30 0x7f5a30158ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:35.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.050+0000 7f5a36303640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5a30146010 0x7f5a3013ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:35.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.050+0000 7f5a36303640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f5a300b48d0 con 0x7f5a300cda40 2024-09-12T03:16:35.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.050+0000 7f5a36303640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f5a300b4750 con 0x7f5a30146010 2024-09-12T03:16:35.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.050+0000 7f5a36303640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f5a300b4a50 con 0x7f5a30145c30 2024-09-12T03:16:35.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.050+0000 7f5a2ffff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5a300cda40 0x7f5a301450e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:35.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.050+0000 7f5a2ffff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5a300cda40 0x7f5a301450e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55564/0 (socket says 172.21.15.27:55564) 2024-09-12T03:16:35.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.050+0000 7f5a2ffff640 1 -- 172.21.15.27:0/2683071390 learned_addr learned my addr 172.21.15.27:0/2683071390 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:16:35.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.050+0000 7f5a2f7fe640 1 --2- 172.21.15.27:0/2683071390 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f5a30145c30 0x7f5a30158ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:35.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.050+0000 7f5a348a0640 1 --2- 172.21.15.27:0/2683071390 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5a30146010 0x7f5a3013ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:35.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.050+0000 7f5a2ffff640 1 -- 172.21.15.27:0/2683071390 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f5a30145c30 msgr2=0x7f5a30158ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:35.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.050+0000 7f5a2ffff640 1 --2- 172.21.15.27:0/2683071390 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f5a30145c30 0x7f5a30158ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:35.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.050+0000 7f5a2ffff640 1 -- 172.21.15.27:0/2683071390 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5a30146010 msgr2=0x7f5a3013ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:35.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.050+0000 7f5a2ffff640 1 --2- 172.21.15.27:0/2683071390 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5a30146010 0x7f5a3013ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:35.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.050+0000 7f5a2ffff640 1 -- 172.21.15.27:0/2683071390 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a300afb60 con 0x7f5a300cda40 2024-09-12T03:16:35.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.051+0000 7f5a2ffff640 1 --2- 172.21.15.27:0/2683071390 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5a300cda40 0x7f5a301450e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f5a2000c400 tx=0x7f5a2000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:35.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.051+0000 7f5a2cff9640 1 -- 172.21.15.27:0/2683071390 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a20017070 con 0x7f5a300cda40 2024-09-12T03:16:35.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.051+0000 7f5a2cff9640 1 -- 172.21.15.27:0/2683071390 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a200052d0 con 0x7f5a300cda40 2024-09-12T03:16:35.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.051+0000 7f5a2cff9640 1 -- 172.21.15.27:0/2683071390 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a20003830 con 0x7f5a300cda40 2024-09-12T03:16:35.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.051+0000 7f5a36303640 1 -- 172.21.15.27:0/2683071390 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a301441c0 con 0x7f5a300cda40 2024-09-12T03:16:35.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.051+0000 7f5a36303640 1 -- 172.21.15.27:0/2683071390 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a30144780 con 0x7f5a300cda40 2024-09-12T03:16:35.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.052+0000 7f5a36303640 1 -- 172.21.15.27:0/2683071390 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a3015ca00 con 0x7f5a300cda40 2024-09-12T03:16:35.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.052+0000 7f5a2cff9640 1 -- 172.21.15.27:0/2683071390 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5a2001a070 con 0x7f5a300cda40 2024-09-12T03:16:35.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.053+0000 7f5a2cff9640 1 --2- 172.21.15.27:0/2683071390 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f5a1c06be30 0x7f5a1c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:35.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.053+0000 7f5a2cff9640 1 -- 172.21.15.27:0/2683071390 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f5a2008a260 con 0x7f5a300cda40 2024-09-12T03:16:35.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.053+0000 7f5a2f7fe640 1 --2- 172.21.15.27:0/2683071390 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f5a1c06be30 0x7f5a1c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:35.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.054+0000 7f5a2f7fe640 1 --2- 172.21.15.27:0/2683071390 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f5a1c06be30 0x7f5a1c06e2f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f5a24034000 tx=0x7f5a2402e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:35.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.057+0000 7f5a2cff9640 1 -- 172.21.15.27:0/2683071390 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f5a20058b80 con 0x7f5a300cda40 2024-09-12T03:16:35.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.229+0000 7f5a36303640 1 -- 172.21.15.27:0/2683071390 --> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5a30075440 con 0x7f5a1c06be30 2024-09-12T03:16:35.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.235+0000 7f5a2cff9640 1 -- 172.21.15.27:0/2683071390 <== mgr.4102 v2:172.21.15.45:6832/3923821219 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+240051 (secure 0 0 0) 0x7f5a30075440 con 0x7f5a1c06be30 2024-09-12T03:16:35.238 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:16:35.239 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2024-09-12T03:16:35.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.238+0000 7f5a36303640 1 -- 172.21.15.27:0/2683071390 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f5a1c06be30 msgr2=0x7f5a1c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:35.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.238+0000 7f5a36303640 1 --2- 172.21.15.27:0/2683071390 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f5a1c06be30 0x7f5a1c06e2f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f5a24034000 tx=0x7f5a2402e000 comp rx=0 tx=0).stop 2024-09-12T03:16:35.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.238+0000 7f5a36303640 1 -- 172.21.15.27:0/2683071390 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5a300cda40 msgr2=0x7f5a301450e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:35.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.238+0000 7f5a36303640 1 --2- 172.21.15.27:0/2683071390 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5a300cda40 0x7f5a301450e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f5a2000c400 tx=0x7f5a2000c8c0 comp rx=0 tx=0).stop 2024-09-12T03:16:35.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.238+0000 7f5a36303640 1 -- 172.21.15.27:0/2683071390 shutdown_connections 2024-09-12T03:16:35.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.238+0000 7f5a36303640 1 --2- 172.21.15.27:0/2683071390 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f5a1c06be30 0x7f5a1c06e2f0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:35.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.238+0000 7f5a36303640 1 --2- 172.21.15.27:0/2683071390 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5a30146010 0x7f5a3013ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:35.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.238+0000 7f5a36303640 1 --2- 172.21.15.27:0/2683071390 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f5a30145c30 0x7f5a30158ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:35.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.238+0000 7f5a36303640 1 --2- 172.21.15.27:0/2683071390 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5a300cda40 0x7f5a301450e0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:35.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.238+0000 7f5a36303640 1 -- 172.21.15.27:0/2683071390 >> 172.21.15.27:0/2683071390 conn(0x7f5a30062c60 msgr2=0x7f5a300aaa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:35.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.239+0000 7f5a36303640 1 -- 172.21.15.27:0/2683071390 shutdown_connections 2024-09-12T03:16:35.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.239+0000 7f5a36303640 1 -- 172.21.15.27:0/2683071390 wait complete. 2024-09-12T03:16:35.249 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2024-09-12T03:16:34.053081+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":8,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235376,"kb_used_data":2500,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754739344,"statfs":{"total":773094113280,"available":772853088256,"internally_reserved":0,"allocated":2560000,"data_stored":1601789,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.499689"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T03:16:29.555565+0000","last_change":"2024-09-12T03:16:29.555565+0000","last_active":"2024-09-12T03:16:29.548038+0000","last_peered":"2024-09-12T03:16:29.548038+0000","last_clean":"2024-09-12T03:16:29.548038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:29.555565+0000","last_undegraded":"2024-09-12T03:16:29.555565+0000","last_fullsized":"2024-09-12T03:16:29.555565+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T03:16:29.558592+0000","last_change":"2024-09-12T03:16:29.558592+0000","last_active":"2024-09-12T03:16:29.548038+0000","last_peered":"2024-09-12T03:16:29.548038+0000","last_clean":"2024-09-12T03:16:29.548038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:29.558592+0000","last_undegraded":"2024-09-12T03:16:29.558592+0000","last_fullsized":"2024-09-12T03:16:29.558592+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T03:16:29.555565+0000","last_change":"2024-09-12T03:16:29.555565+0000","last_active":"2024-09-12T03:16:29.548038+0000","last_peered":"2024-09-12T03:16:29.548038+0000","last_clean":"2024-09-12T03:16:29.548038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:29.555565+0000","last_undegraded":"2024-09-12T03:16:29.555565+0000","last_fullsized":"2024-09-12T03:16:29.555565+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T03:16:29.560599+0000","last_change":"2024-09-12T03:16:29.560599+0000","last_active":"2024-09-12T03:16:29.548038+0000","last_peered":"2024-09-12T03:16:29.548038+0000","last_clean":"2024-09-12T03:16:29.548038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:29.560599+0000","last_undegraded":"2024-09-12T03:16:29.560599+0000","last_fullsized":"2024-09-12T03:16:29.560599+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T03:16:29.559533+0000","last_change":"2024-09-12T03:16:29.559533+0000","last_active":"2024-09-12T03:16:29.548038+0000","last_peered":"2024-09-12T03:16:29.548038+0000","last_clean":"2024-09-12T03:16:29.548038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:29.559533+0000","last_undegraded":"2024-09-12T03:16:29.559533+0000","last_fullsized":"2024-09-12T03:16:29.559533+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T03:16:29.560546+0000","last_change":"2024-09-12T03:16:29.560546+0000","last_active":"2024-09-12T03:16:29.548038+0000","last_peered":"2024-09-12T03:16:29.548038+0000","last_clean":"2024-09-12T03:16:29.548038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:29.560546+0000","last_undegraded":"2024-09-12T03:16:29.560546+0000","last_fullsized":"2024-09-12T03:16:29.560546+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T03:16:29.560488+0000","last_change":"2024-09-12T03:16:29.560488+0000","last_active":"2024-09-12T03:16:29.548038+0000","last_peered":"2024-09-12T03:16:29.548038+0000","last_clean":"2024-09-12T03:16:29.548038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:29.560488+0000","last_undegraded":"2024-09-12T03:16:29.560488+0000","last_fullsized":"2024-09-12T03:16:29.560488+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T03:16:29.558583+0000","last_change":"2024-09-12T03:16:29.558583+0000","last_active":"2024-09-12T03:16:29.548038+0000","last_peered":"2024-09-12T03:16:29.548038+0000","last_clean":"2024-09-12T03:16:29.548038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:29.558583+0000","last_undegraded":"2024-09-12T03:16:29.558583+0000","last_fullsized":"2024-09-12T03:16:29.558583+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T03:16:29.558598+0000","last_change":"2024-09-12T03:16:29.558598+0000","last_active":"2024-09-12T03:16:29.548038+0000","last_peered":"2024-09-12T03:16:29.548038+0000","last_clean":"2024-09-12T03:16:29.548038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:29.558598+0000","last_undegraded":"2024-09-12T03:16:29.558598+0000","last_fullsized":"2024-09-12T03:16:29.558598+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T03:16:29.561109+0000","last_change":"2024-09-12T03:16:29.561109+0000","last_active":"2024-09-12T03:16:29.548038+0000","last_peered":"2024-09-12T03:16:29.548038+0000","last_clean":"2024-09-12T03:16:29.548038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:29.561109+0000","last_undegraded":"2024-09-12T03:16:29.561109+0000","last_fullsized":"2024-09-12T03:16:29.561109+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T03:16:29.558613+0000","last_change":"2024-09-12T03:16:29.558613+0000","last_active":"2024-09-12T03:16:29.548038+0000","last_peered":"2024-09-12T03:16:29.548038+0000","last_clean":"2024-09-12T03:16:29.548038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:29.558613+0000","last_undegraded":"2024-09-12T03:16:29.558613+0000","last_fullsized":"2024-09-12T03:16:29.558613+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T03:16:29.563582+0000","last_change":"2024-09-12T03:16:29.563582+0000","last_active":"2024-09-12T03:16:29.548038+0000","last_peered":"2024-09-12T03:16:29.548038+0000","last_clean":"2024-09-12T03:16:29.548038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:29.563582+0000","last_undegraded":"2024-09-12T03:16:29.563582+0000","last_fullsized":"2024-09-12T03:16:29.563582+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T03:16:29.555666+0000","last_change":"2024-09-12T03:16:29.555666+0000","last_active":"2024-09-12T03:16:29.548038+0000","last_peered":"2024-09-12T03:16:29.548038+0000","last_clean":"2024-09-12T03:16:29.548038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:29.555666+0000","last_undegraded":"2024-09-12T03:16:29.555666+0000","last_fullsized":"2024-09-12T03:16:29.555666+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T03:16:29.563556+0000","last_change":"2024-09-12T03:16:29.563556+0000","last_active":"2024-09-12T03:16:29.548038+0000","last_peered":"2024-09-12T03:16:29.548038+0000","last_clean":"2024-09-12T03:16:29.548038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:29.563556+0000","last_undegraded":"2024-09-12T03:16:29.563556+0000","last_fullsized":"2024-09-12T03:16:29.563556+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T03:16:29.559421+0000","last_change":"2024-09-12T03:16:29.559421+0000","last_active":"2024-09-12T03:16:29.548038+0000","last_peered":"2024-09-12T03:16:29.548038+0000","last_clean":"2024-09-12T03:16:29.548038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:29.559421+0000","last_undegraded":"2024-09-12T03:16:29.559421+0000","last_fullsized":"2024-09-12T03:16:29.559421+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T03:16:29.563546+0000","last_change":"2024-09-12T03:16:29.563546+0000","last_active":"2024-09-12T03:16:29.548038+0000","last_peered":"2024-09-12T03:16:29.548038+0000","last_clean":"2024-09-12T03:16:29.548038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:29.563546+0000","last_undegraded":"2024-09-12T03:16:29.563546+0000","last_fullsized":"2024-09-12T03:16:29.563546+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":164,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-12T03:16:29.557520+0000","last_change":"2024-09-12T03:16:21.883446+0000","last_active":"2024-09-12T03:16:29.557520+0000","last_peered":"2024-09-12T03:16:29.557520+0000","last_clean":"2024-09-12T03:16:29.557520+0000","last_became_active":"2024-09-12T03:16:21.882207+0000","last_became_peered":"2024-09-12T03:16:21.882207+0000","last_unstale":"2024-09-12T03:16:29.557520+0000","last_undegraded":"2024-09-12T03:16:29.557520+0000","last_fullsized":"2024-09-12T03:16:29.557520+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:20.316426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:20.316426+0000","last_clean_scrub_stamp":"2024-09-12T03:16:20.316426+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:12:15.424320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T03:16:29.560957+0000","last_change":"2024-09-12T03:16:29.560957+0000","last_active":"2024-09-12T03:16:29.548038+0000","last_peered":"2024-09-12T03:16:29.548038+0000","last_clean":"2024-09-12T03:16:29.548038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:29.560957+0000","last_undegraded":"2024-09-12T03:16:29.560957+0000","last_fullsized":"2024-09-12T03:16:29.560957+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T03:16:29.556208+0000","last_change":"2024-09-12T03:16:29.556208+0000","last_active":"2024-09-12T03:16:29.548038+0000","last_peered":"2024-09-12T03:16:29.548038+0000","last_clean":"2024-09-12T03:16:29.548038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:29.556208+0000","last_undegraded":"2024-09-12T03:16:29.556208+0000","last_fullsized":"2024-09-12T03:16:29.556208+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T03:16:29.563487+0000","last_change":"2024-09-12T03:16:29.563487+0000","last_active":"2024-09-12T03:16:29.548038+0000","last_peered":"2024-09-12T03:16:29.548038+0000","last_clean":"2024-09-12T03:16:29.548038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:29.563487+0000","last_undegraded":"2024-09-12T03:16:29.563487+0000","last_fullsized":"2024-09-12T03:16:29.563487+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T03:16:29.560584+0000","last_change":"2024-09-12T03:16:29.560584+0000","last_active":"2024-09-12T03:16:29.548038+0000","last_peered":"2024-09-12T03:16:29.548038+0000","last_clean":"2024-09-12T03:16:29.548038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:29.560584+0000","last_undegraded":"2024-09-12T03:16:29.560584+0000","last_fullsized":"2024-09-12T03:16:29.560584+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T03:16:29.559139+0000","last_change":"2024-09-12T03:16:29.559139+0000","last_active":"2024-09-12T03:16:29.548038+0000","last_peered":"2024-09-12T03:16:29.548038+0000","last_clean":"2024-09-12T03:16:29.548038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:29.559139+0000","last_undegraded":"2024-09-12T03:16:29.559139+0000","last_fullsized":"2024-09-12T03:16:29.559139+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T03:16:29.555833+0000","last_change":"2024-09-12T03:16:29.555833+0000","last_active":"2024-09-12T03:16:29.548038+0000","last_peered":"2024-09-12T03:16:29.548038+0000","last_clean":"2024-09-12T03:16:29.548038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:29.555833+0000","last_undegraded":"2024-09-12T03:16:29.555833+0000","last_fullsized":"2024-09-12T03:16:29.555833+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T03:16:29.559271+0000","last_change":"2024-09-12T03:16:29.559271+0000","last_active":"2024-09-12T03:16:29.548038+0000","last_peered":"2024-09-12T03:16:29.548038+0000","last_clean":"2024-09-12T03:16:29.548038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:29.559271+0000","last_undegraded":"2024-09-12T03:16:29.559271+0000","last_fullsized":"2024-09-12T03:16:29.559271+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T03:16:29.558849+0000","last_change":"2024-09-12T03:16:29.558849+0000","last_active":"2024-09-12T03:16:29.548038+0000","last_peered":"2024-09-12T03:16:29.548038+0000","last_clean":"2024-09-12T03:16:29.548038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:29.558849+0000","last_undegraded":"2024-09-12T03:16:29.558849+0000","last_fullsized":"2024-09-12T03:16:29.558849+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T03:16:29.557099+0000","last_change":"2024-09-12T03:16:29.557099+0000","last_active":"2024-09-12T03:16:29.548038+0000","last_peered":"2024-09-12T03:16:29.548038+0000","last_clean":"2024-09-12T03:16:29.548038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:29.557099+0000","last_undegraded":"2024-09-12T03:16:29.557099+0000","last_fullsized":"2024-09-12T03:16:29.557099+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T03:16:29.559156+0000","last_change":"2024-09-12T03:16:29.559156+0000","last_active":"2024-09-12T03:16:29.548038+0000","last_peered":"2024-09-12T03:16:29.548038+0000","last_clean":"2024-09-12T03:16:29.548038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:29.559156+0000","last_undegraded":"2024-09-12T03:16:29.559156+0000","last_fullsized":"2024-09-12T03:16:29.559156+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T03:16:29.558951+0000","last_change":"2024-09-12T03:16:29.558951+0000","last_active":"2024-09-12T03:16:29.548038+0000","last_peered":"2024-09-12T03:16:29.548038+0000","last_clean":"2024-09-12T03:16:29.548038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:29.558951+0000","last_undegraded":"2024-09-12T03:16:29.558951+0000","last_fullsized":"2024-09-12T03:16:29.558951+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T03:16:29.555670+0000","last_change":"2024-09-12T03:16:29.555670+0000","last_active":"2024-09-12T03:16:29.548038+0000","last_peered":"2024-09-12T03:16:29.548038+0000","last_clean":"2024-09-12T03:16:29.548038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:29.555670+0000","last_undegraded":"2024-09-12T03:16:29.555670+0000","last_fullsized":"2024-09-12T03:16:29.555670+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T03:16:29.556203+0000","last_change":"2024-09-12T03:16:29.556203+0000","last_active":"2024-09-12T03:16:29.548038+0000","last_peered":"2024-09-12T03:16:29.548038+0000","last_clean":"2024-09-12T03:16:29.548038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:29.556203+0000","last_undegraded":"2024-09-12T03:16:29.556203+0000","last_fullsized":"2024-09-12T03:16:29.556203+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T03:16:29.556594+0000","last_change":"2024-09-12T03:16:29.556594+0000","last_active":"2024-09-12T03:16:29.548038+0000","last_peered":"2024-09-12T03:16:29.548038+0000","last_clean":"2024-09-12T03:16:29.548038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:29.556594+0000","last_undegraded":"2024-09-12T03:16:29.556594+0000","last_fullsized":"2024-09-12T03:16:29.556594+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T03:16:29.555746+0000","last_change":"2024-09-12T03:16:29.555746+0000","last_active":"2024-09-12T03:16:29.548038+0000","last_peered":"2024-09-12T03:16:29.548038+0000","last_clean":"2024-09-12T03:16:29.548038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:29.555746+0000","last_undegraded":"2024-09-12T03:16:29.555746+0000","last_fullsized":"2024-09-12T03:16:29.555746+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T03:16:29.558604+0000","last_change":"2024-09-12T03:16:29.558604+0000","last_active":"2024-09-12T03:16:29.548038+0000","last_peered":"2024-09-12T03:16:29.548038+0000","last_clean":"2024-09-12T03:16:29.548038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:29.558604+0000","last_undegraded":"2024-09-12T03:16:29.558604+0000","last_fullsized":"2024-09-12T03:16:29.558604+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542147,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31940,"kb_used_data":752,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339900,"statfs":{"total":96636764160,"available":96604057600,"internally_reserved":0,"allocated":770048,"data_stored":645181,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542149,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31352,"kb_used_data":160,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340488,"statfs":{"total":96636764160,"available":96604659712,"internally_reserved":0,"allocated":163840,"data_stored":48996,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542148,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31360,"kb_used_data":172,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340480,"statfs":{"total":96636764160,"available":96604651520,"internally_reserved":0,"allocated":176128,"data_stored":54813,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542147,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31352,"kb_used_data":160,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340488,"statfs":{"total":96636764160,"available":96604659712,"internally_reserved":0,"allocated":163840,"data_stored":48996,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542148,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27192,"kb_used_data":160,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344648,"statfs":{"total":96636764160,"available":96608919552,"internally_reserved":0,"allocated":163840,"data_stored":48996,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542148,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27200,"kb_used_data":172,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344640,"statfs":{"total":96636764160,"available":96608911360,"internally_reserved":0,"allocated":176128,"data_stored":54813,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542149,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27200,"kb_used_data":172,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344640,"statfs":{"total":96636764160,"available":96608911360,"internally_reserved":0,"allocated":176128,"data_stored":54813,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542149,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27780,"kb_used_data":752,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344060,"statfs":{"total":96636764160,"available":96608317440,"internally_reserved":0,"allocated":770048,"data_stored":645181,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T03:16:35.253 INFO:tasks.ceph.ceph_manager.ceph:PG 3.3f is not active+clean 2024-09-12T03:16:35.253 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.3f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T03:16:30.553126+0000', 'last_change': '2024-09-12T03:16:30.553126+0000', 'last_active': '2024-09-12T03:16:30.553126+0000', 'last_peered': '2024-09-12T03:16:30.553126+0000', 'last_clean': '2024-09-12T03:16:30.553126+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:16:30.553126+0000', 'last_undegraded': '2024-09-12T03:16:30.553126+0000', 'last_fullsized': '2024-09-12T03:16:30.553126+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:16:30.553126+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:16:30.553126+0000', 'last_clean_scrub_stamp': '2024-09-12T03:16:30.553126+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T03:16:35.253 INFO:tasks.ceph.ceph_manager.ceph:PG 3.3e is not active+clean 2024-09-12T03:16:35.253 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.3e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T03:16:30.553126+0000', 'last_change': '2024-09-12T03:16:30.553126+0000', 'last_active': '2024-09-12T03:16:30.553126+0000', 'last_peered': '2024-09-12T03:16:30.553126+0000', 'last_clean': '2024-09-12T03:16:30.553126+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:16:30.553126+0000', 'last_undegraded': '2024-09-12T03:16:30.553126+0000', 'last_fullsized': '2024-09-12T03:16:30.553126+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:16:30.553126+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:16:30.553126+0000', 'last_clean_scrub_stamp': '2024-09-12T03:16:30.553126+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T03:16:35.253 INFO:tasks.ceph.ceph_manager.ceph:PG 3.3d is not active+clean 2024-09-12T03:16:35.253 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.3d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T03:16:30.553126+0000', 'last_change': '2024-09-12T03:16:30.553126+0000', 'last_active': '2024-09-12T03:16:30.553126+0000', 'last_peered': '2024-09-12T03:16:30.553126+0000', 'last_clean': '2024-09-12T03:16:30.553126+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:16:30.553126+0000', 'last_undegraded': '2024-09-12T03:16:30.553126+0000', 'last_fullsized': '2024-09-12T03:16:30.553126+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:16:30.553126+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:16:30.553126+0000', 'last_clean_scrub_stamp': '2024-09-12T03:16:30.553126+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T03:16:35.253 INFO:tasks.ceph.ceph_manager.ceph:PG 3.3c is not active+clean 2024-09-12T03:16:35.253 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.3c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T03:16:30.553126+0000', 'last_change': '2024-09-12T03:16:30.553126+0000', 'last_active': '2024-09-12T03:16:30.553126+0000', 'last_peered': '2024-09-12T03:16:30.553126+0000', 'last_clean': '2024-09-12T03:16:30.553126+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:16:30.553126+0000', 'last_undegraded': '2024-09-12T03:16:30.553126+0000', 'last_fullsized': '2024-09-12T03:16:30.553126+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:16:30.553126+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:16:30.553126+0000', 'last_clean_scrub_stamp': '2024-09-12T03:16:30.553126+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T03:16:35.253 INFO:tasks.ceph.ceph_manager.ceph:PG 3.3b is not active+clean 2024-09-12T03:16:35.254 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T03:16:30.553126+0000', 'last_change': '2024-09-12T03:16:30.553126+0000', 'last_active': '2024-09-12T03:16:30.553126+0000', 'last_peered': '2024-09-12T03:16:30.553126+0000', 'last_clean': '2024-09-12T03:16:30.553126+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:16:30.553126+0000', 'last_undegraded': '2024-09-12T03:16:30.553126+0000', 'last_fullsized': '2024-09-12T03:16:30.553126+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:16:30.553126+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:16:30.553126+0000', 'last_clean_scrub_stamp': '2024-09-12T03:16:30.553126+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T03:16:35.254 INFO:tasks.ceph.ceph_manager.ceph:PG 3.3a is not active+clean 2024-09-12T03:16:35.254 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.3a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T03:16:30.553126+0000', 'last_change': '2024-09-12T03:16:30.553126+0000', 'last_active': '2024-09-12T03:16:30.553126+0000', 'last_peered': '2024-09-12T03:16:30.553126+0000', 'last_clean': '2024-09-12T03:16:30.553126+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:16:30.553126+0000', 'last_undegraded': '2024-09-12T03:16:30.553126+0000', 'last_fullsized': '2024-09-12T03:16:30.553126+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:16:30.553126+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:16:30.553126+0000', 'last_clean_scrub_stamp': '2024-09-12T03:16:30.553126+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T03:16:35.254 INFO:tasks.ceph.ceph_manager.ceph:PG 3.39 is not active+clean 2024-09-12T03:16:35.254 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.39', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T03:16:30.553126+0000', 'last_change': '2024-09-12T03:16:30.553126+0000', 'last_active': '2024-09-12T03:16:30.553126+0000', 'last_peered': '2024-09-12T03:16:30.553126+0000', 'last_clean': '2024-09-12T03:16:30.553126+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:16:30.553126+0000', 'last_undegraded': '2024-09-12T03:16:30.553126+0000', 'last_fullsized': '2024-09-12T03:16:30.553126+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:16:30.553126+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:16:30.553126+0000', 'last_clean_scrub_stamp': '2024-09-12T03:16:30.553126+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T03:16:35.254 INFO:tasks.ceph.ceph_manager.ceph:PG 3.38 is not active+clean 2024-09-12T03:16:35.254 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.38', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T03:16:30.553126+0000', 'last_change': '2024-09-12T03:16:30.553126+0000', 'last_active': '2024-09-12T03:16:30.553126+0000', 'last_peered': '2024-09-12T03:16:30.553126+0000', 'last_clean': '2024-09-12T03:16:30.553126+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:16:30.553126+0000', 'last_undegraded': '2024-09-12T03:16:30.553126+0000', 'last_fullsized': '2024-09-12T03:16:30.553126+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:16:30.553126+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:16:30.553126+0000', 'last_clean_scrub_stamp': '2024-09-12T03:16:30.553126+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T03:16:35.254 INFO:tasks.ceph.ceph_manager.ceph:PG 3.37 is not active+clean 2024-09-12T03:16:35.254 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.37', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T03:16:30.553126+0000', 'last_change': '2024-09-12T03:16:30.553126+0000', 'last_active': '2024-09-12T03:16:30.553126+0000', 'last_peered': '2024-09-12T03:16:30.553126+0000', 'last_clean': '2024-09-12T03:16:30.553126+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:16:30.553126+0000', 'last_undegraded': '2024-09-12T03:16:30.553126+0000', 'last_fullsized': '2024-09-12T03:16:30.553126+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:16:30.553126+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:16:30.553126+0000', 'last_clean_scrub_stamp': '2024-09-12T03:16:30.553126+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T03:16:35.254 INFO:tasks.ceph.ceph_manager.ceph:PG 3.36 is not active+clean 2024-09-12T03:16:35.254 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.36', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T03:16:30.553126+0000', 'last_change': '2024-09-12T03:16:30.553126+0000', 'last_active': '2024-09-12T03:16:30.553126+0000', 'last_peered': '2024-09-12T03:16:30.553126+0000', 'last_clean': '2024-09-12T03:16:30.553126+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:16:30.553126+0000', 'last_undegraded': '2024-09-12T03:16:30.553126+0000', 'last_fullsized': '2024-09-12T03:16:30.553126+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:16:30.553126+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:16:30.553126+0000', 'last_clean_scrub_stamp': '2024-09-12T03:16:30.553126+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T03:16:35.255 INFO:tasks.ceph.ceph_manager.ceph:PG 3.35 is not active+clean 2024-09-12T03:16:35.255 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.35', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T03:16:30.553126+0000', 'last_change': '2024-09-12T03:16:30.553126+0000', 'last_active': '2024-09-12T03:16:30.553126+0000', 'last_peered': '2024-09-12T03:16:30.553126+0000', 'last_clean': '2024-09-12T03:16:30.553126+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:16:30.553126+0000', 'last_undegraded': '2024-09-12T03:16:30.553126+0000', 'last_fullsized': '2024-09-12T03:16:30.553126+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:16:30.553126+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:16:30.553126+0000', 'last_clean_scrub_stamp': '2024-09-12T03:16:30.553126+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T03:16:35.255 INFO:tasks.ceph.ceph_manager.ceph:PG 3.34 is not active+clean 2024-09-12T03:16:35.255 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T03:16:30.553126+0000', 'last_change': '2024-09-12T03:16:30.553126+0000', 'last_active': '2024-09-12T03:16:30.553126+0000', 'last_peered': '2024-09-12T03:16:30.553126+0000', 'last_clean': '2024-09-12T03:16:30.553126+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:16:30.553126+0000', 'last_undegraded': '2024-09-12T03:16:30.553126+0000', 'last_fullsized': '2024-09-12T03:16:30.553126+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:16:30.553126+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:16:30.553126+0000', 'last_clean_scrub_stamp': '2024-09-12T03:16:30.553126+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T03:16:35.255 INFO:tasks.ceph.ceph_manager.ceph:PG 3.33 is not active+clean 2024-09-12T03:16:35.255 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.33', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T03:16:30.553126+0000', 'last_change': '2024-09-12T03:16:30.553126+0000', 'last_active': '2024-09-12T03:16:30.553126+0000', 'last_peered': '2024-09-12T03:16:30.553126+0000', 'last_clean': '2024-09-12T03:16:30.553126+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:16:30.553126+0000', 'last_undegraded': '2024-09-12T03:16:30.553126+0000', 'last_fullsized': '2024-09-12T03:16:30.553126+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:16:30.553126+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:16:30.553126+0000', 'last_clean_scrub_stamp': '2024-09-12T03:16:30.553126+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T03:16:35.255 INFO:tasks.ceph.ceph_manager.ceph:PG 3.32 is not active+clean 2024-09-12T03:16:35.255 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.32', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T03:16:30.553126+0000', 'last_change': '2024-09-12T03:16:30.553126+0000', 'last_active': '2024-09-12T03:16:30.553126+0000', 'last_peered': '2024-09-12T03:16:30.553126+0000', 'last_clean': '2024-09-12T03:16:30.553126+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:16:30.553126+0000', 'last_undegraded': '2024-09-12T03:16:30.553126+0000', 'last_fullsized': '2024-09-12T03:16:30.553126+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:16:30.553126+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:16:30.553126+0000', 'last_clean_scrub_stamp': '2024-09-12T03:16:30.553126+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T03:16:35.255 INFO:tasks.ceph.ceph_manager.ceph:PG 3.31 is not active+clean 2024-09-12T03:16:35.255 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.31', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T03:16:30.553126+0000', 'last_change': '2024-09-12T03:16:30.553126+0000', 'last_active': '2024-09-12T03:16:30.553126+0000', 'last_peered': '2024-09-12T03:16:30.553126+0000', 'last_clean': '2024-09-12T03:16:30.553126+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:16:30.553126+0000', 'last_undegraded': '2024-09-12T03:16:30.553126+0000', 'last_fullsized': '2024-09-12T03:16:30.553126+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:16:30.553126+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:16:30.553126+0000', 'last_clean_scrub_stamp': '2024-09-12T03:16:30.553126+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T03:16:35.255 INFO:tasks.ceph.ceph_manager.ceph:PG 3.30 is not active+clean 2024-09-12T03:16:35.255 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T03:16:30.553126+0000', 'last_change': '2024-09-12T03:16:30.553126+0000', 'last_active': '2024-09-12T03:16:30.553126+0000', 'last_peered': '2024-09-12T03:16:30.553126+0000', 'last_clean': '2024-09-12T03:16:30.553126+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:16:30.553126+0000', 'last_undegraded': '2024-09-12T03:16:30.553126+0000', 'last_fullsized': '2024-09-12T03:16:30.553126+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:16:30.553126+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:16:30.553126+0000', 'last_clean_scrub_stamp': '2024-09-12T03:16:30.553126+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T03:16:35.256 INFO:tasks.ceph.ceph_manager.ceph:PG 3.2f is not active+clean 2024-09-12T03:16:35.256 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.2f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T03:16:30.553126+0000', 'last_change': '2024-09-12T03:16:30.553126+0000', 'last_active': '2024-09-12T03:16:30.553126+0000', 'last_peered': '2024-09-12T03:16:30.553126+0000', 'last_clean': '2024-09-12T03:16:30.553126+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:16:30.553126+0000', 'last_undegraded': '2024-09-12T03:16:30.553126+0000', 'last_fullsized': '2024-09-12T03:16:30.553126+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:16:30.553126+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:16:30.553126+0000', 'last_clean_scrub_stamp': '2024-09-12T03:16:30.553126+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T03:16:35.256 INFO:tasks.ceph.ceph_manager.ceph:PG 3.2e is not active+clean 2024-09-12T03:16:35.256 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.2e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T03:16:30.553126+0000', 'last_change': '2024-09-12T03:16:30.553126+0000', 'last_active': '2024-09-12T03:16:30.553126+0000', 'last_peered': '2024-09-12T03:16:30.553126+0000', 'last_clean': '2024-09-12T03:16:30.553126+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:16:30.553126+0000', 'last_undegraded': '2024-09-12T03:16:30.553126+0000', 'last_fullsized': '2024-09-12T03:16:30.553126+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:16:30.553126+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:16:30.553126+0000', 'last_clean_scrub_stamp': '2024-09-12T03:16:30.553126+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T03:16:35.256 INFO:tasks.ceph.ceph_manager.ceph:PG 3.2d is not active+clean 2024-09-12T03:16:35.256 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.2d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T03:16:30.553126+0000', 'last_change': '2024-09-12T03:16:30.553126+0000', 'last_active': '2024-09-12T03:16:30.553126+0000', 'last_peered': '2024-09-12T03:16:30.553126+0000', 'last_clean': '2024-09-12T03:16:30.553126+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:16:30.553126+0000', 'last_undegraded': '2024-09-12T03:16:30.553126+0000', 'last_fullsized': '2024-09-12T03:16:30.553126+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:16:30.553126+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:16:30.553126+0000', 'last_clean_scrub_stamp': '2024-09-12T03:16:30.553126+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T03:16:35.256 INFO:tasks.ceph.ceph_manager.ceph:PG 3.2c is not active+clean 2024-09-12T03:16:35.256 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.2c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T03:16:30.553126+0000', 'last_change': '2024-09-12T03:16:30.553126+0000', 'last_active': '2024-09-12T03:16:30.553126+0000', 'last_peered': '2024-09-12T03:16:30.553126+0000', 'last_clean': '2024-09-12T03:16:30.553126+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:16:30.553126+0000', 'last_undegraded': '2024-09-12T03:16:30.553126+0000', 'last_fullsized': '2024-09-12T03:16:30.553126+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:16:30.553126+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:16:30.553126+0000', 'last_clean_scrub_stamp': '2024-09-12T03:16:30.553126+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T03:16:35.256 INFO:tasks.ceph.ceph_manager.ceph:PG 3.2b is not active+clean 2024-09-12T03:16:35.256 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T03:16:30.553126+0000', 'last_change': '2024-09-12T03:16:30.553126+0000', 'last_active': '2024-09-12T03:16:30.553126+0000', 'last_peered': '2024-09-12T03:16:30.553126+0000', 'last_clean': '2024-09-12T03:16:30.553126+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:16:30.553126+0000', 'last_undegraded': '2024-09-12T03:16:30.553126+0000', 'last_fullsized': '2024-09-12T03:16:30.553126+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:16:30.553126+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:16:30.553126+0000', 'last_clean_scrub_stamp': '2024-09-12T03:16:30.553126+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T03:16:35.256 INFO:tasks.ceph.ceph_manager.ceph:PG 3.2a is not active+clean 2024-09-12T03:16:35.257 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.2a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T03:16:30.553126+0000', 'last_change': '2024-09-12T03:16:30.553126+0000', 'last_active': '2024-09-12T03:16:30.553126+0000', 'last_peered': '2024-09-12T03:16:30.553126+0000', 'last_clean': '2024-09-12T03:16:30.553126+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:16:30.553126+0000', 'last_undegraded': '2024-09-12T03:16:30.553126+0000', 'last_fullsized': '2024-09-12T03:16:30.553126+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:16:30.553126+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:16:30.553126+0000', 'last_clean_scrub_stamp': '2024-09-12T03:16:30.553126+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T03:16:35.257 INFO:tasks.ceph.ceph_manager.ceph:PG 3.29 is not active+clean 2024-09-12T03:16:35.257 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.29', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T03:16:30.553126+0000', 'last_change': '2024-09-12T03:16:30.553126+0000', 'last_active': '2024-09-12T03:16:30.553126+0000', 'last_peered': '2024-09-12T03:16:30.553126+0000', 'last_clean': '2024-09-12T03:16:30.553126+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:16:30.553126+0000', 'last_undegraded': '2024-09-12T03:16:30.553126+0000', 'last_fullsized': '2024-09-12T03:16:30.553126+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:16:30.553126+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:16:30.553126+0000', 'last_clean_scrub_stamp': '2024-09-12T03:16:30.553126+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T03:16:35.257 INFO:tasks.ceph.ceph_manager.ceph:PG 3.28 is not active+clean 2024-09-12T03:16:35.257 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.28', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T03:16:30.553126+0000', 'last_change': '2024-09-12T03:16:30.553126+0000', 'last_active': '2024-09-12T03:16:30.553126+0000', 'last_peered': '2024-09-12T03:16:30.553126+0000', 'last_clean': '2024-09-12T03:16:30.553126+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:16:30.553126+0000', 'last_undegraded': '2024-09-12T03:16:30.553126+0000', 'last_fullsized': '2024-09-12T03:16:30.553126+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:16:30.553126+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:16:30.553126+0000', 'last_clean_scrub_stamp': '2024-09-12T03:16:30.553126+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T03:16:35.257 INFO:tasks.ceph.ceph_manager.ceph:PG 3.27 is not active+clean 2024-09-12T03:16:35.257 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.27', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T03:16:30.553126+0000', 'last_change': '2024-09-12T03:16:30.553126+0000', 'last_active': '2024-09-12T03:16:30.553126+0000', 'last_peered': '2024-09-12T03:16:30.553126+0000', 'last_clean': '2024-09-12T03:16:30.553126+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:16:30.553126+0000', 'last_undegraded': '2024-09-12T03:16:30.553126+0000', 'last_fullsized': '2024-09-12T03:16:30.553126+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:16:30.553126+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:16:30.553126+0000', 'last_clean_scrub_stamp': '2024-09-12T03:16:30.553126+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T03:16:35.257 INFO:tasks.ceph.ceph_manager.ceph:PG 3.26 is not active+clean 2024-09-12T03:16:35.257 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.26', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T03:16:30.553126+0000', 'last_change': '2024-09-12T03:16:30.553126+0000', 'last_active': '2024-09-12T03:16:30.553126+0000', 'last_peered': '2024-09-12T03:16:30.553126+0000', 'last_clean': '2024-09-12T03:16:30.553126+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:16:30.553126+0000', 'last_undegraded': '2024-09-12T03:16:30.553126+0000', 'last_fullsized': '2024-09-12T03:16:30.553126+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:16:30.553126+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:16:30.553126+0000', 'last_clean_scrub_stamp': '2024-09-12T03:16:30.553126+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T03:16:35.257 INFO:tasks.ceph.ceph_manager.ceph:PG 3.25 is not active+clean 2024-09-12T03:16:35.257 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.25', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T03:16:30.553126+0000', 'last_change': '2024-09-12T03:16:30.553126+0000', 'last_active': '2024-09-12T03:16:30.553126+0000', 'last_peered': '2024-09-12T03:16:30.553126+0000', 'last_clean': '2024-09-12T03:16:30.553126+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:16:30.553126+0000', 'last_undegraded': '2024-09-12T03:16:30.553126+0000', 'last_fullsized': '2024-09-12T03:16:30.553126+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:16:30.553126+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:16:30.553126+0000', 'last_clean_scrub_stamp': '2024-09-12T03:16:30.553126+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T03:16:35.258 INFO:tasks.ceph.ceph_manager.ceph:PG 3.24 is not active+clean 2024-09-12T03:16:35.258 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.24', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T03:16:30.553126+0000', 'last_change': '2024-09-12T03:16:30.553126+0000', 'last_active': '2024-09-12T03:16:30.553126+0000', 'last_peered': '2024-09-12T03:16:30.553126+0000', 'last_clean': '2024-09-12T03:16:30.553126+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:16:30.553126+0000', 'last_undegraded': '2024-09-12T03:16:30.553126+0000', 'last_fullsized': '2024-09-12T03:16:30.553126+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:16:30.553126+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:16:30.553126+0000', 'last_clean_scrub_stamp': '2024-09-12T03:16:30.553126+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T03:16:35.258 INFO:tasks.ceph.ceph_manager.ceph:PG 3.23 is not active+clean 2024-09-12T03:16:35.258 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.23', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T03:16:30.553126+0000', 'last_change': '2024-09-12T03:16:30.553126+0000', 'last_active': '2024-09-12T03:16:30.553126+0000', 'last_peered': '2024-09-12T03:16:30.553126+0000', 'last_clean': '2024-09-12T03:16:30.553126+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:16:30.553126+0000', 'last_undegraded': '2024-09-12T03:16:30.553126+0000', 'last_fullsized': '2024-09-12T03:16:30.553126+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:16:30.553126+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:16:30.553126+0000', 'last_clean_scrub_stamp': '2024-09-12T03:16:30.553126+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T03:16:35.258 INFO:tasks.ceph.ceph_manager.ceph:PG 3.22 is not active+clean 2024-09-12T03:16:35.258 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T03:16:30.553126+0000', 'last_change': '2024-09-12T03:16:30.553126+0000', 'last_active': '2024-09-12T03:16:30.553126+0000', 'last_peered': '2024-09-12T03:16:30.553126+0000', 'last_clean': '2024-09-12T03:16:30.553126+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:16:30.553126+0000', 'last_undegraded': '2024-09-12T03:16:30.553126+0000', 'last_fullsized': '2024-09-12T03:16:30.553126+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:16:30.553126+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:16:30.553126+0000', 'last_clean_scrub_stamp': '2024-09-12T03:16:30.553126+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T03:16:35.258 INFO:tasks.ceph.ceph_manager.ceph:PG 3.21 is not active+clean 2024-09-12T03:16:35.258 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.21', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T03:16:30.553126+0000', 'last_change': '2024-09-12T03:16:30.553126+0000', 'last_active': '2024-09-12T03:16:30.553126+0000', 'last_peered': '2024-09-12T03:16:30.553126+0000', 'last_clean': '2024-09-12T03:16:30.553126+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:16:30.553126+0000', 'last_undegraded': '2024-09-12T03:16:30.553126+0000', 'last_fullsized': '2024-09-12T03:16:30.553126+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:16:30.553126+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:16:30.553126+0000', 'last_clean_scrub_stamp': '2024-09-12T03:16:30.553126+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T03:16:35.258 INFO:tasks.ceph.ceph_manager.ceph:PG 3.20 is not active+clean 2024-09-12T03:16:35.258 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.20', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T03:16:30.553126+0000', 'last_change': '2024-09-12T03:16:30.553126+0000', 'last_active': '2024-09-12T03:16:30.553126+0000', 'last_peered': '2024-09-12T03:16:30.553126+0000', 'last_clean': '2024-09-12T03:16:30.553126+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:16:30.553126+0000', 'last_undegraded': '2024-09-12T03:16:30.553126+0000', 'last_fullsized': '2024-09-12T03:16:30.553126+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:16:30.553126+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:16:30.553126+0000', 'last_clean_scrub_stamp': '2024-09-12T03:16:30.553126+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T03:16:35.258 INFO:tasks.ceph.ceph_manager.ceph:PG 3.1a is not active+clean 2024-09-12T03:16:35.258 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T03:16:30.553126+0000', 'last_change': '2024-09-12T03:16:30.553126+0000', 'last_active': '2024-09-12T03:16:30.553126+0000', 'last_peered': '2024-09-12T03:16:30.553126+0000', 'last_clean': '2024-09-12T03:16:30.553126+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:16:30.553126+0000', 'last_undegraded': '2024-09-12T03:16:30.553126+0000', 'last_fullsized': '2024-09-12T03:16:30.553126+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:16:30.553126+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:16:30.553126+0000', 'last_clean_scrub_stamp': '2024-09-12T03:16:30.553126+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T03:16:35.259 INFO:tasks.ceph.ceph_manager.ceph:PG 2.1b is not active+clean 2024-09-12T03:16:35.259 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-09-12T03:16:29.555565+0000', 'last_change': '2024-09-12T03:16:29.555565+0000', 'last_active': '2024-09-12T03:16:29.548038+0000', 'last_peered': '2024-09-12T03:16:29.548038+0000', 'last_clean': '2024-09-12T03:16:29.548038+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:16:29.555565+0000', 'last_undegraded': '2024-09-12T03:16:29.555565+0000', 'last_fullsized': '2024-09-12T03:16:29.555565+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:16:29.548038+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:16:29.548038+0000', 'last_clean_scrub_stamp': '2024-09-12T03:16:29.548038+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-12T03:16:35.259 INFO:tasks.ceph.ceph_manager.ceph:PG 3.1b is not active+clean 2024-09-12T03:16:35.259 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T03:16:30.553126+0000', 'last_change': '2024-09-12T03:16:30.553126+0000', 'last_active': '2024-09-12T03:16:30.553126+0000', 'last_peered': '2024-09-12T03:16:30.553126+0000', 'last_clean': '2024-09-12T03:16:30.553126+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:16:30.553126+0000', 'last_undegraded': '2024-09-12T03:16:30.553126+0000', 'last_fullsized': '2024-09-12T03:16:30.553126+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:16:30.553126+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:16:30.553126+0000', 'last_clean_scrub_stamp': '2024-09-12T03:16:30.553126+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T03:16:35.259 INFO:tasks.ceph.ceph_manager.ceph:PG 2.1a is not active+clean 2024-09-12T03:16:35.259 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-09-12T03:16:29.558592+0000', 'last_change': '2024-09-12T03:16:29.558592+0000', 'last_active': '2024-09-12T03:16:29.548038+0000', 'last_peered': '2024-09-12T03:16:29.548038+0000', 'last_clean': '2024-09-12T03:16:29.548038+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:16:29.558592+0000', 'last_undegraded': '2024-09-12T03:16:29.558592+0000', 'last_fullsized': '2024-09-12T03:16:29.558592+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:16:29.548038+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:16:29.548038+0000', 'last_clean_scrub_stamp': '2024-09-12T03:16:29.548038+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-12T03:16:35.259 INFO:tasks.ceph.ceph_manager.ceph:PG 2.19 is not active+clean 2024-09-12T03:16:35.259 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-09-12T03:16:29.555565+0000', 'last_change': '2024-09-12T03:16:29.555565+0000', 'last_active': '2024-09-12T03:16:29.548038+0000', 'last_peered': '2024-09-12T03:16:29.548038+0000', 'last_clean': '2024-09-12T03:16:29.548038+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:16:29.555565+0000', 'last_undegraded': '2024-09-12T03:16:29.555565+0000', 'last_fullsized': '2024-09-12T03:16:29.555565+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:16:29.548038+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:16:29.548038+0000', 'last_clean_scrub_stamp': '2024-09-12T03:16:29.548038+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-12T03:16:35.259 INFO:tasks.ceph.ceph_manager.ceph:PG 3.18 is not active+clean 2024-09-12T03:16:35.259 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T03:16:30.553126+0000', 'last_change': '2024-09-12T03:16:30.553126+0000', 'last_active': '2024-09-12T03:16:30.553126+0000', 'last_peered': '2024-09-12T03:16:30.553126+0000', 'last_clean': '2024-09-12T03:16:30.553126+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:16:30.553126+0000', 'last_undegraded': '2024-09-12T03:16:30.553126+0000', 'last_fullsized': '2024-09-12T03:16:30.553126+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:16:30.553126+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:16:30.553126+0000', 'last_clean_scrub_stamp': '2024-09-12T03:16:30.553126+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T03:16:35.259 INFO:tasks.ceph.ceph_manager.ceph:PG 2.18 is not active+clean 2024-09-12T03:16:35.260 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-09-12T03:16:29.560599+0000', 'last_change': '2024-09-12T03:16:29.560599+0000', 'last_active': '2024-09-12T03:16:29.548038+0000', 'last_peered': '2024-09-12T03:16:29.548038+0000', 'last_clean': '2024-09-12T03:16:29.548038+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:16:29.560599+0000', 'last_undegraded': '2024-09-12T03:16:29.560599+0000', 'last_fullsized': '2024-09-12T03:16:29.560599+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:16:29.548038+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:16:29.548038+0000', 'last_clean_scrub_stamp': '2024-09-12T03:16:29.548038+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-12T03:16:35.260 INFO:tasks.ceph.ceph_manager.ceph:PG 3.19 is not active+clean 2024-09-12T03:16:35.260 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T03:16:30.553126+0000', 'last_change': '2024-09-12T03:16:30.553126+0000', 'last_active': '2024-09-12T03:16:30.553126+0000', 'last_peered': '2024-09-12T03:16:30.553126+0000', 'last_clean': '2024-09-12T03:16:30.553126+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:16:30.553126+0000', 'last_undegraded': '2024-09-12T03:16:30.553126+0000', 'last_fullsized': '2024-09-12T03:16:30.553126+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:16:30.553126+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:16:30.553126+0000', 'last_clean_scrub_stamp': '2024-09-12T03:16:30.553126+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T03:16:35.260 INFO:tasks.ceph.ceph_manager.ceph:PG 2.17 is not active+clean 2024-09-12T03:16:35.260 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-09-12T03:16:29.559533+0000', 'last_change': '2024-09-12T03:16:29.559533+0000', 'last_active': '2024-09-12T03:16:29.548038+0000', 'last_peered': '2024-09-12T03:16:29.548038+0000', 'last_clean': '2024-09-12T03:16:29.548038+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:16:29.559533+0000', 'last_undegraded': '2024-09-12T03:16:29.559533+0000', 'last_fullsized': '2024-09-12T03:16:29.559533+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:16:29.548038+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:16:29.548038+0000', 'last_clean_scrub_stamp': '2024-09-12T03:16:29.548038+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-12T03:16:35.260 INFO:tasks.ceph.ceph_manager.ceph:PG 3.16 is not active+clean 2024-09-12T03:16:35.260 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T03:16:30.553126+0000', 'last_change': '2024-09-12T03:16:30.553126+0000', 'last_active': '2024-09-12T03:16:30.553126+0000', 'last_peered': '2024-09-12T03:16:30.553126+0000', 'last_clean': '2024-09-12T03:16:30.553126+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:16:30.553126+0000', 'last_undegraded': '2024-09-12T03:16:30.553126+0000', 'last_fullsized': '2024-09-12T03:16:30.553126+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:16:30.553126+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:16:30.553126+0000', 'last_clean_scrub_stamp': '2024-09-12T03:16:30.553126+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T03:16:35.260 INFO:tasks.ceph.ceph_manager.ceph:PG 2.16 is not active+clean 2024-09-12T03:16:35.260 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-09-12T03:16:29.560546+0000', 'last_change': '2024-09-12T03:16:29.560546+0000', 'last_active': '2024-09-12T03:16:29.548038+0000', 'last_peered': '2024-09-12T03:16:29.548038+0000', 'last_clean': '2024-09-12T03:16:29.548038+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:16:29.560546+0000', 'last_undegraded': '2024-09-12T03:16:29.560546+0000', 'last_fullsized': '2024-09-12T03:16:29.560546+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:16:29.548038+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:16:29.548038+0000', 'last_clean_scrub_stamp': '2024-09-12T03:16:29.548038+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-12T03:16:35.260 INFO:tasks.ceph.ceph_manager.ceph:PG 3.17 is not active+clean 2024-09-12T03:16:35.260 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T03:16:30.553126+0000', 'last_change': '2024-09-12T03:16:30.553126+0000', 'last_active': '2024-09-12T03:16:30.553126+0000', 'last_peered': '2024-09-12T03:16:30.553126+0000', 'last_clean': '2024-09-12T03:16:30.553126+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:16:30.553126+0000', 'last_undegraded': '2024-09-12T03:16:30.553126+0000', 'last_fullsized': '2024-09-12T03:16:30.553126+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:16:30.553126+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:16:30.553126+0000', 'last_clean_scrub_stamp': '2024-09-12T03:16:30.553126+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T03:16:35.261 INFO:tasks.ceph.ceph_manager.ceph:PG 2.15 is not active+clean 2024-09-12T03:16:35.261 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-09-12T03:16:29.560488+0000', 'last_change': '2024-09-12T03:16:29.560488+0000', 'last_active': '2024-09-12T03:16:29.548038+0000', 'last_peered': '2024-09-12T03:16:29.548038+0000', 'last_clean': '2024-09-12T03:16:29.548038+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:16:29.560488+0000', 'last_undegraded': '2024-09-12T03:16:29.560488+0000', 'last_fullsized': '2024-09-12T03:16:29.560488+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:16:29.548038+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:16:29.548038+0000', 'last_clean_scrub_stamp': '2024-09-12T03:16:29.548038+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-12T03:16:35.261 INFO:tasks.ceph.ceph_manager.ceph:PG 3.14 is not active+clean 2024-09-12T03:16:35.261 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T03:16:30.553126+0000', 'last_change': '2024-09-12T03:16:30.553126+0000', 'last_active': '2024-09-12T03:16:30.553126+0000', 'last_peered': '2024-09-12T03:16:30.553126+0000', 'last_clean': '2024-09-12T03:16:30.553126+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:16:30.553126+0000', 'last_undegraded': '2024-09-12T03:16:30.553126+0000', 'last_fullsized': '2024-09-12T03:16:30.553126+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:16:30.553126+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:16:30.553126+0000', 'last_clean_scrub_stamp': '2024-09-12T03:16:30.553126+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T03:16:35.261 INFO:tasks.ceph.ceph_manager.ceph:PG 2.14 is not active+clean 2024-09-12T03:16:35.261 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-09-12T03:16:29.558583+0000', 'last_change': '2024-09-12T03:16:29.558583+0000', 'last_active': '2024-09-12T03:16:29.548038+0000', 'last_peered': '2024-09-12T03:16:29.548038+0000', 'last_clean': '2024-09-12T03:16:29.548038+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:16:29.558583+0000', 'last_undegraded': '2024-09-12T03:16:29.558583+0000', 'last_fullsized': '2024-09-12T03:16:29.558583+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:16:29.548038+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:16:29.548038+0000', 'last_clean_scrub_stamp': '2024-09-12T03:16:29.548038+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-12T03:16:35.261 INFO:tasks.ceph.ceph_manager.ceph:PG 3.15 is not active+clean 2024-09-12T03:16:35.261 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T03:16:30.553126+0000', 'last_change': '2024-09-12T03:16:30.553126+0000', 'last_active': '2024-09-12T03:16:30.553126+0000', 'last_peered': '2024-09-12T03:16:30.553126+0000', 'last_clean': '2024-09-12T03:16:30.553126+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:16:30.553126+0000', 'last_undegraded': '2024-09-12T03:16:30.553126+0000', 'last_fullsized': '2024-09-12T03:16:30.553126+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:16:30.553126+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:16:30.553126+0000', 'last_clean_scrub_stamp': '2024-09-12T03:16:30.553126+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T03:16:35.261 INFO:tasks.ceph.ceph_manager.ceph:PG 2.13 is not active+clean 2024-09-12T03:16:35.261 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-09-12T03:16:29.558598+0000', 'last_change': '2024-09-12T03:16:29.558598+0000', 'last_active': '2024-09-12T03:16:29.548038+0000', 'last_peered': '2024-09-12T03:16:29.548038+0000', 'last_clean': '2024-09-12T03:16:29.548038+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:16:29.558598+0000', 'last_undegraded': '2024-09-12T03:16:29.558598+0000', 'last_fullsized': '2024-09-12T03:16:29.558598+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:16:29.548038+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:16:29.548038+0000', 'last_clean_scrub_stamp': '2024-09-12T03:16:29.548038+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-12T03:16:35.262 INFO:tasks.ceph.ceph_manager.ceph:PG 3.12 is not active+clean 2024-09-12T03:16:35.262 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T03:16:30.553126+0000', 'last_change': '2024-09-12T03:16:30.553126+0000', 'last_active': '2024-09-12T03:16:30.553126+0000', 'last_peered': '2024-09-12T03:16:30.553126+0000', 'last_clean': '2024-09-12T03:16:30.553126+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:16:30.553126+0000', 'last_undegraded': '2024-09-12T03:16:30.553126+0000', 'last_fullsized': '2024-09-12T03:16:30.553126+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:16:30.553126+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:16:30.553126+0000', 'last_clean_scrub_stamp': '2024-09-12T03:16:30.553126+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T03:16:35.262 INFO:tasks.ceph.ceph_manager.ceph:PG 2.12 is not active+clean 2024-09-12T03:16:35.262 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-09-12T03:16:29.561109+0000', 'last_change': '2024-09-12T03:16:29.561109+0000', 'last_active': '2024-09-12T03:16:29.548038+0000', 'last_peered': '2024-09-12T03:16:29.548038+0000', 'last_clean': '2024-09-12T03:16:29.548038+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:16:29.561109+0000', 'last_undegraded': '2024-09-12T03:16:29.561109+0000', 'last_fullsized': '2024-09-12T03:16:29.561109+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:16:29.548038+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:16:29.548038+0000', 'last_clean_scrub_stamp': '2024-09-12T03:16:29.548038+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-12T03:16:35.262 INFO:tasks.ceph.ceph_manager.ceph:PG 3.13 is not active+clean 2024-09-12T03:16:35.262 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T03:16:30.553126+0000', 'last_change': '2024-09-12T03:16:30.553126+0000', 'last_active': '2024-09-12T03:16:30.553126+0000', 'last_peered': '2024-09-12T03:16:30.553126+0000', 'last_clean': '2024-09-12T03:16:30.553126+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:16:30.553126+0000', 'last_undegraded': '2024-09-12T03:16:30.553126+0000', 'last_fullsized': '2024-09-12T03:16:30.553126+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:16:30.553126+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:16:30.553126+0000', 'last_clean_scrub_stamp': '2024-09-12T03:16:30.553126+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T03:16:35.262 INFO:tasks.ceph.ceph_manager.ceph:PG 2.11 is not active+clean 2024-09-12T03:16:35.262 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-09-12T03:16:29.558613+0000', 'last_change': '2024-09-12T03:16:29.558613+0000', 'last_active': '2024-09-12T03:16:29.548038+0000', 'last_peered': '2024-09-12T03:16:29.548038+0000', 'last_clean': '2024-09-12T03:16:29.548038+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:16:29.558613+0000', 'last_undegraded': '2024-09-12T03:16:29.558613+0000', 'last_fullsized': '2024-09-12T03:16:29.558613+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:16:29.548038+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:16:29.548038+0000', 'last_clean_scrub_stamp': '2024-09-12T03:16:29.548038+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-12T03:16:35.262 INFO:tasks.ceph.ceph_manager.ceph:PG 3.10 is not active+clean 2024-09-12T03:16:35.262 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T03:16:30.553126+0000', 'last_change': '2024-09-12T03:16:30.553126+0000', 'last_active': '2024-09-12T03:16:30.553126+0000', 'last_peered': '2024-09-12T03:16:30.553126+0000', 'last_clean': '2024-09-12T03:16:30.553126+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:16:30.553126+0000', 'last_undegraded': '2024-09-12T03:16:30.553126+0000', 'last_fullsized': '2024-09-12T03:16:30.553126+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:16:30.553126+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:16:30.553126+0000', 'last_clean_scrub_stamp': '2024-09-12T03:16:30.553126+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T03:16:35.262 INFO:tasks.ceph.ceph_manager.ceph:PG 2.10 is not active+clean 2024-09-12T03:16:35.262 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-09-12T03:16:29.563582+0000', 'last_change': '2024-09-12T03:16:29.563582+0000', 'last_active': '2024-09-12T03:16:29.548038+0000', 'last_peered': '2024-09-12T03:16:29.548038+0000', 'last_clean': '2024-09-12T03:16:29.548038+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:16:29.563582+0000', 'last_undegraded': '2024-09-12T03:16:29.563582+0000', 'last_fullsized': '2024-09-12T03:16:29.563582+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:16:29.548038+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:16:29.548038+0000', 'last_clean_scrub_stamp': '2024-09-12T03:16:29.548038+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-12T03:16:35.263 INFO:tasks.ceph.ceph_manager.ceph:PG 3.11 is not active+clean 2024-09-12T03:16:35.263 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T03:16:30.553126+0000', 'last_change': '2024-09-12T03:16:30.553126+0000', 'last_active': '2024-09-12T03:16:30.553126+0000', 'last_peered': '2024-09-12T03:16:30.553126+0000', 'last_clean': '2024-09-12T03:16:30.553126+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:16:30.553126+0000', 'last_undegraded': '2024-09-12T03:16:30.553126+0000', 'last_fullsized': '2024-09-12T03:16:30.553126+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:16:30.553126+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:16:30.553126+0000', 'last_clean_scrub_stamp': '2024-09-12T03:16:30.553126+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T03:16:35.263 INFO:tasks.ceph.ceph_manager.ceph:PG 2.f is not active+clean 2024-09-12T03:16:35.263 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-09-12T03:16:29.555666+0000', 'last_change': '2024-09-12T03:16:29.555666+0000', 'last_active': '2024-09-12T03:16:29.548038+0000', 'last_peered': '2024-09-12T03:16:29.548038+0000', 'last_clean': '2024-09-12T03:16:29.548038+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:16:29.555666+0000', 'last_undegraded': '2024-09-12T03:16:29.555666+0000', 'last_fullsized': '2024-09-12T03:16:29.555666+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:16:29.548038+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:16:29.548038+0000', 'last_clean_scrub_stamp': '2024-09-12T03:16:29.548038+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-12T03:16:35.263 INFO:tasks.ceph.ceph_manager.ceph:PG 3.e is not active+clean 2024-09-12T03:16:35.263 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T03:16:30.553126+0000', 'last_change': '2024-09-12T03:16:30.553126+0000', 'last_active': '2024-09-12T03:16:30.553126+0000', 'last_peered': '2024-09-12T03:16:30.553126+0000', 'last_clean': '2024-09-12T03:16:30.553126+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:16:30.553126+0000', 'last_undegraded': '2024-09-12T03:16:30.553126+0000', 'last_fullsized': '2024-09-12T03:16:30.553126+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:16:30.553126+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:16:30.553126+0000', 'last_clean_scrub_stamp': '2024-09-12T03:16:30.553126+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T03:16:35.263 INFO:tasks.ceph.ceph_manager.ceph:PG 2.e is not active+clean 2024-09-12T03:16:35.263 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-09-12T03:16:29.563556+0000', 'last_change': '2024-09-12T03:16:29.563556+0000', 'last_active': '2024-09-12T03:16:29.548038+0000', 'last_peered': '2024-09-12T03:16:29.548038+0000', 'last_clean': '2024-09-12T03:16:29.548038+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:16:29.563556+0000', 'last_undegraded': '2024-09-12T03:16:29.563556+0000', 'last_fullsized': '2024-09-12T03:16:29.563556+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:16:29.548038+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:16:29.548038+0000', 'last_clean_scrub_stamp': '2024-09-12T03:16:29.548038+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-12T03:16:35.263 INFO:tasks.ceph.ceph_manager.ceph:PG 3.f is not active+clean 2024-09-12T03:16:35.263 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T03:16:30.553126+0000', 'last_change': '2024-09-12T03:16:30.553126+0000', 'last_active': '2024-09-12T03:16:30.553126+0000', 'last_peered': '2024-09-12T03:16:30.553126+0000', 'last_clean': '2024-09-12T03:16:30.553126+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:16:30.553126+0000', 'last_undegraded': '2024-09-12T03:16:30.553126+0000', 'last_fullsized': '2024-09-12T03:16:30.553126+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:16:30.553126+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:16:30.553126+0000', 'last_clean_scrub_stamp': '2024-09-12T03:16:30.553126+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T03:16:35.263 INFO:tasks.ceph.ceph_manager.ceph:PG 2.d is not active+clean 2024-09-12T03:16:35.264 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-09-12T03:16:29.559421+0000', 'last_change': '2024-09-12T03:16:29.559421+0000', 'last_active': '2024-09-12T03:16:29.548038+0000', 'last_peered': '2024-09-12T03:16:29.548038+0000', 'last_clean': '2024-09-12T03:16:29.548038+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:16:29.559421+0000', 'last_undegraded': '2024-09-12T03:16:29.559421+0000', 'last_fullsized': '2024-09-12T03:16:29.559421+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:16:29.548038+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:16:29.548038+0000', 'last_clean_scrub_stamp': '2024-09-12T03:16:29.548038+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-12T03:16:35.264 INFO:tasks.ceph.ceph_manager.ceph:PG 3.c is not active+clean 2024-09-12T03:16:35.264 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T03:16:30.553126+0000', 'last_change': '2024-09-12T03:16:30.553126+0000', 'last_active': '2024-09-12T03:16:30.553126+0000', 'last_peered': '2024-09-12T03:16:30.553126+0000', 'last_clean': '2024-09-12T03:16:30.553126+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:16:30.553126+0000', 'last_undegraded': '2024-09-12T03:16:30.553126+0000', 'last_fullsized': '2024-09-12T03:16:30.553126+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:16:30.553126+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:16:30.553126+0000', 'last_clean_scrub_stamp': '2024-09-12T03:16:30.553126+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T03:16:35.264 INFO:tasks.ceph.ceph_manager.ceph:PG 2.c is not active+clean 2024-09-12T03:16:35.264 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-09-12T03:16:29.563546+0000', 'last_change': '2024-09-12T03:16:29.563546+0000', 'last_active': '2024-09-12T03:16:29.548038+0000', 'last_peered': '2024-09-12T03:16:29.548038+0000', 'last_clean': '2024-09-12T03:16:29.548038+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:16:29.563546+0000', 'last_undegraded': '2024-09-12T03:16:29.563546+0000', 'last_fullsized': '2024-09-12T03:16:29.563546+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:16:29.548038+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:16:29.548038+0000', 'last_clean_scrub_stamp': '2024-09-12T03:16:29.548038+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-12T03:16:35.264 INFO:tasks.ceph.ceph_manager.ceph:PG 3.d is not active+clean 2024-09-12T03:16:35.264 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T03:16:30.553126+0000', 'last_change': '2024-09-12T03:16:30.553126+0000', 'last_active': '2024-09-12T03:16:30.553126+0000', 'last_peered': '2024-09-12T03:16:30.553126+0000', 'last_clean': '2024-09-12T03:16:30.553126+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:16:30.553126+0000', 'last_undegraded': '2024-09-12T03:16:30.553126+0000', 'last_fullsized': '2024-09-12T03:16:30.553126+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:16:30.553126+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:16:30.553126+0000', 'last_clean_scrub_stamp': '2024-09-12T03:16:30.553126+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T03:16:35.264 INFO:tasks.ceph.ceph_manager.ceph:PG 2.3 is not active+clean 2024-09-12T03:16:35.264 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-09-12T03:16:29.560957+0000', 'last_change': '2024-09-12T03:16:29.560957+0000', 'last_active': '2024-09-12T03:16:29.548038+0000', 'last_peered': '2024-09-12T03:16:29.548038+0000', 'last_clean': '2024-09-12T03:16:29.548038+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:16:29.560957+0000', 'last_undegraded': '2024-09-12T03:16:29.560957+0000', 'last_fullsized': '2024-09-12T03:16:29.560957+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:16:29.548038+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:16:29.548038+0000', 'last_clean_scrub_stamp': '2024-09-12T03:16:29.548038+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-12T03:16:35.264 INFO:tasks.ceph.ceph_manager.ceph:PG 3.2 is not active+clean 2024-09-12T03:16:35.264 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T03:16:30.553126+0000', 'last_change': '2024-09-12T03:16:30.553126+0000', 'last_active': '2024-09-12T03:16:30.553126+0000', 'last_peered': '2024-09-12T03:16:30.553126+0000', 'last_clean': '2024-09-12T03:16:30.553126+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:16:30.553126+0000', 'last_undegraded': '2024-09-12T03:16:30.553126+0000', 'last_fullsized': '2024-09-12T03:16:30.553126+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:16:30.553126+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:16:30.553126+0000', 'last_clean_scrub_stamp': '2024-09-12T03:16:30.553126+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T03:16:35.265 INFO:tasks.ceph.ceph_manager.ceph:PG 2.0 is not active+clean 2024-09-12T03:16:35.265 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-09-12T03:16:29.556208+0000', 'last_change': '2024-09-12T03:16:29.556208+0000', 'last_active': '2024-09-12T03:16:29.548038+0000', 'last_peered': '2024-09-12T03:16:29.548038+0000', 'last_clean': '2024-09-12T03:16:29.548038+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:16:29.556208+0000', 'last_undegraded': '2024-09-12T03:16:29.556208+0000', 'last_fullsized': '2024-09-12T03:16:29.556208+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:16:29.548038+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:16:29.548038+0000', 'last_clean_scrub_stamp': '2024-09-12T03:16:29.548038+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-12T03:16:35.265 INFO:tasks.ceph.ceph_manager.ceph:PG 3.1 is not active+clean 2024-09-12T03:16:35.265 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T03:16:30.553126+0000', 'last_change': '2024-09-12T03:16:30.553126+0000', 'last_active': '2024-09-12T03:16:30.553126+0000', 'last_peered': '2024-09-12T03:16:30.553126+0000', 'last_clean': '2024-09-12T03:16:30.553126+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:16:30.553126+0000', 'last_undegraded': '2024-09-12T03:16:30.553126+0000', 'last_fullsized': '2024-09-12T03:16:30.553126+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:16:30.553126+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:16:30.553126+0000', 'last_clean_scrub_stamp': '2024-09-12T03:16:30.553126+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T03:16:35.265 INFO:tasks.ceph.ceph_manager.ceph:PG 2.1 is not active+clean 2024-09-12T03:16:35.265 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-09-12T03:16:29.563487+0000', 'last_change': '2024-09-12T03:16:29.563487+0000', 'last_active': '2024-09-12T03:16:29.548038+0000', 'last_peered': '2024-09-12T03:16:29.548038+0000', 'last_clean': '2024-09-12T03:16:29.548038+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:16:29.563487+0000', 'last_undegraded': '2024-09-12T03:16:29.563487+0000', 'last_fullsized': '2024-09-12T03:16:29.563487+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:16:29.548038+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:16:29.548038+0000', 'last_clean_scrub_stamp': '2024-09-12T03:16:29.548038+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-12T03:16:35.265 INFO:tasks.ceph.ceph_manager.ceph:PG 3.0 is not active+clean 2024-09-12T03:16:35.265 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T03:16:30.553126+0000', 'last_change': '2024-09-12T03:16:30.553126+0000', 'last_active': '2024-09-12T03:16:30.553126+0000', 'last_peered': '2024-09-12T03:16:30.553126+0000', 'last_clean': '2024-09-12T03:16:30.553126+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:16:30.553126+0000', 'last_undegraded': '2024-09-12T03:16:30.553126+0000', 'last_fullsized': '2024-09-12T03:16:30.553126+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:16:30.553126+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:16:30.553126+0000', 'last_clean_scrub_stamp': '2024-09-12T03:16:30.553126+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T03:16:35.265 INFO:tasks.ceph.ceph_manager.ceph:PG 2.2 is not active+clean 2024-09-12T03:16:35.265 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-09-12T03:16:29.560584+0000', 'last_change': '2024-09-12T03:16:29.560584+0000', 'last_active': '2024-09-12T03:16:29.548038+0000', 'last_peered': '2024-09-12T03:16:29.548038+0000', 'last_clean': '2024-09-12T03:16:29.548038+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:16:29.560584+0000', 'last_undegraded': '2024-09-12T03:16:29.560584+0000', 'last_fullsized': '2024-09-12T03:16:29.560584+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:16:29.548038+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:16:29.548038+0000', 'last_clean_scrub_stamp': '2024-09-12T03:16:29.548038+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-12T03:16:35.265 INFO:tasks.ceph.ceph_manager.ceph:PG 3.3 is not active+clean 2024-09-12T03:16:35.266 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T03:16:30.553126+0000', 'last_change': '2024-09-12T03:16:30.553126+0000', 'last_active': '2024-09-12T03:16:30.553126+0000', 'last_peered': '2024-09-12T03:16:30.553126+0000', 'last_clean': '2024-09-12T03:16:30.553126+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:16:30.553126+0000', 'last_undegraded': '2024-09-12T03:16:30.553126+0000', 'last_fullsized': '2024-09-12T03:16:30.553126+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:16:30.553126+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:16:30.553126+0000', 'last_clean_scrub_stamp': '2024-09-12T03:16:30.553126+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T03:16:35.266 INFO:tasks.ceph.ceph_manager.ceph:PG 2.4 is not active+clean 2024-09-12T03:16:35.266 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-09-12T03:16:29.559139+0000', 'last_change': '2024-09-12T03:16:29.559139+0000', 'last_active': '2024-09-12T03:16:29.548038+0000', 'last_peered': '2024-09-12T03:16:29.548038+0000', 'last_clean': '2024-09-12T03:16:29.548038+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:16:29.559139+0000', 'last_undegraded': '2024-09-12T03:16:29.559139+0000', 'last_fullsized': '2024-09-12T03:16:29.559139+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:16:29.548038+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:16:29.548038+0000', 'last_clean_scrub_stamp': '2024-09-12T03:16:29.548038+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-12T03:16:35.266 INFO:tasks.ceph.ceph_manager.ceph:PG 3.5 is not active+clean 2024-09-12T03:16:35.266 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T03:16:30.553126+0000', 'last_change': '2024-09-12T03:16:30.553126+0000', 'last_active': '2024-09-12T03:16:30.553126+0000', 'last_peered': '2024-09-12T03:16:30.553126+0000', 'last_clean': '2024-09-12T03:16:30.553126+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:16:30.553126+0000', 'last_undegraded': '2024-09-12T03:16:30.553126+0000', 'last_fullsized': '2024-09-12T03:16:30.553126+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:16:30.553126+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:16:30.553126+0000', 'last_clean_scrub_stamp': '2024-09-12T03:16:30.553126+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T03:16:35.266 INFO:tasks.ceph.ceph_manager.ceph:PG 2.5 is not active+clean 2024-09-12T03:16:35.266 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-09-12T03:16:29.555833+0000', 'last_change': '2024-09-12T03:16:29.555833+0000', 'last_active': '2024-09-12T03:16:29.548038+0000', 'last_peered': '2024-09-12T03:16:29.548038+0000', 'last_clean': '2024-09-12T03:16:29.548038+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:16:29.555833+0000', 'last_undegraded': '2024-09-12T03:16:29.555833+0000', 'last_fullsized': '2024-09-12T03:16:29.555833+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:16:29.548038+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:16:29.548038+0000', 'last_clean_scrub_stamp': '2024-09-12T03:16:29.548038+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-12T03:16:35.266 INFO:tasks.ceph.ceph_manager.ceph:PG 3.4 is not active+clean 2024-09-12T03:16:35.266 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T03:16:30.553126+0000', 'last_change': '2024-09-12T03:16:30.553126+0000', 'last_active': '2024-09-12T03:16:30.553126+0000', 'last_peered': '2024-09-12T03:16:30.553126+0000', 'last_clean': '2024-09-12T03:16:30.553126+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:16:30.553126+0000', 'last_undegraded': '2024-09-12T03:16:30.553126+0000', 'last_fullsized': '2024-09-12T03:16:30.553126+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:16:30.553126+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:16:30.553126+0000', 'last_clean_scrub_stamp': '2024-09-12T03:16:30.553126+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T03:16:35.266 INFO:tasks.ceph.ceph_manager.ceph:PG 2.6 is not active+clean 2024-09-12T03:16:35.266 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-09-12T03:16:29.559271+0000', 'last_change': '2024-09-12T03:16:29.559271+0000', 'last_active': '2024-09-12T03:16:29.548038+0000', 'last_peered': '2024-09-12T03:16:29.548038+0000', 'last_clean': '2024-09-12T03:16:29.548038+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:16:29.559271+0000', 'last_undegraded': '2024-09-12T03:16:29.559271+0000', 'last_fullsized': '2024-09-12T03:16:29.559271+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:16:29.548038+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:16:29.548038+0000', 'last_clean_scrub_stamp': '2024-09-12T03:16:29.548038+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-12T03:16:35.267 INFO:tasks.ceph.ceph_manager.ceph:PG 3.7 is not active+clean 2024-09-12T03:16:35.267 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T03:16:30.553126+0000', 'last_change': '2024-09-12T03:16:30.553126+0000', 'last_active': '2024-09-12T03:16:30.553126+0000', 'last_peered': '2024-09-12T03:16:30.553126+0000', 'last_clean': '2024-09-12T03:16:30.553126+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:16:30.553126+0000', 'last_undegraded': '2024-09-12T03:16:30.553126+0000', 'last_fullsized': '2024-09-12T03:16:30.553126+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:16:30.553126+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:16:30.553126+0000', 'last_clean_scrub_stamp': '2024-09-12T03:16:30.553126+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T03:16:35.267 INFO:tasks.ceph.ceph_manager.ceph:PG 2.7 is not active+clean 2024-09-12T03:16:35.267 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-09-12T03:16:29.558849+0000', 'last_change': '2024-09-12T03:16:29.558849+0000', 'last_active': '2024-09-12T03:16:29.548038+0000', 'last_peered': '2024-09-12T03:16:29.548038+0000', 'last_clean': '2024-09-12T03:16:29.548038+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:16:29.558849+0000', 'last_undegraded': '2024-09-12T03:16:29.558849+0000', 'last_fullsized': '2024-09-12T03:16:29.558849+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:16:29.548038+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:16:29.548038+0000', 'last_clean_scrub_stamp': '2024-09-12T03:16:29.548038+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-12T03:16:35.267 INFO:tasks.ceph.ceph_manager.ceph:PG 3.6 is not active+clean 2024-09-12T03:16:35.267 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T03:16:30.553126+0000', 'last_change': '2024-09-12T03:16:30.553126+0000', 'last_active': '2024-09-12T03:16:30.553126+0000', 'last_peered': '2024-09-12T03:16:30.553126+0000', 'last_clean': '2024-09-12T03:16:30.553126+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:16:30.553126+0000', 'last_undegraded': '2024-09-12T03:16:30.553126+0000', 'last_fullsized': '2024-09-12T03:16:30.553126+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:16:30.553126+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:16:30.553126+0000', 'last_clean_scrub_stamp': '2024-09-12T03:16:30.553126+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T03:16:35.267 INFO:tasks.ceph.ceph_manager.ceph:PG 2.8 is not active+clean 2024-09-12T03:16:35.267 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-09-12T03:16:29.557099+0000', 'last_change': '2024-09-12T03:16:29.557099+0000', 'last_active': '2024-09-12T03:16:29.548038+0000', 'last_peered': '2024-09-12T03:16:29.548038+0000', 'last_clean': '2024-09-12T03:16:29.548038+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:16:29.557099+0000', 'last_undegraded': '2024-09-12T03:16:29.557099+0000', 'last_fullsized': '2024-09-12T03:16:29.557099+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:16:29.548038+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:16:29.548038+0000', 'last_clean_scrub_stamp': '2024-09-12T03:16:29.548038+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-12T03:16:35.267 INFO:tasks.ceph.ceph_manager.ceph:PG 3.9 is not active+clean 2024-09-12T03:16:35.267 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T03:16:30.553126+0000', 'last_change': '2024-09-12T03:16:30.553126+0000', 'last_active': '2024-09-12T03:16:30.553126+0000', 'last_peered': '2024-09-12T03:16:30.553126+0000', 'last_clean': '2024-09-12T03:16:30.553126+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:16:30.553126+0000', 'last_undegraded': '2024-09-12T03:16:30.553126+0000', 'last_fullsized': '2024-09-12T03:16:30.553126+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:16:30.553126+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:16:30.553126+0000', 'last_clean_scrub_stamp': '2024-09-12T03:16:30.553126+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T03:16:35.267 INFO:tasks.ceph.ceph_manager.ceph:PG 2.9 is not active+clean 2024-09-12T03:16:35.267 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-09-12T03:16:29.559156+0000', 'last_change': '2024-09-12T03:16:29.559156+0000', 'last_active': '2024-09-12T03:16:29.548038+0000', 'last_peered': '2024-09-12T03:16:29.548038+0000', 'last_clean': '2024-09-12T03:16:29.548038+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:16:29.559156+0000', 'last_undegraded': '2024-09-12T03:16:29.559156+0000', 'last_fullsized': '2024-09-12T03:16:29.559156+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:16:29.548038+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:16:29.548038+0000', 'last_clean_scrub_stamp': '2024-09-12T03:16:29.548038+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-12T03:16:35.268 INFO:tasks.ceph.ceph_manager.ceph:PG 3.8 is not active+clean 2024-09-12T03:16:35.268 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T03:16:30.553126+0000', 'last_change': '2024-09-12T03:16:30.553126+0000', 'last_active': '2024-09-12T03:16:30.553126+0000', 'last_peered': '2024-09-12T03:16:30.553126+0000', 'last_clean': '2024-09-12T03:16:30.553126+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:16:30.553126+0000', 'last_undegraded': '2024-09-12T03:16:30.553126+0000', 'last_fullsized': '2024-09-12T03:16:30.553126+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:16:30.553126+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:16:30.553126+0000', 'last_clean_scrub_stamp': '2024-09-12T03:16:30.553126+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T03:16:35.268 INFO:tasks.ceph.ceph_manager.ceph:PG 2.a is not active+clean 2024-09-12T03:16:35.268 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-09-12T03:16:29.558951+0000', 'last_change': '2024-09-12T03:16:29.558951+0000', 'last_active': '2024-09-12T03:16:29.548038+0000', 'last_peered': '2024-09-12T03:16:29.548038+0000', 'last_clean': '2024-09-12T03:16:29.548038+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:16:29.558951+0000', 'last_undegraded': '2024-09-12T03:16:29.558951+0000', 'last_fullsized': '2024-09-12T03:16:29.558951+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:16:29.548038+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:16:29.548038+0000', 'last_clean_scrub_stamp': '2024-09-12T03:16:29.548038+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-12T03:16:35.268 INFO:tasks.ceph.ceph_manager.ceph:PG 3.b is not active+clean 2024-09-12T03:16:35.268 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T03:16:30.553126+0000', 'last_change': '2024-09-12T03:16:30.553126+0000', 'last_active': '2024-09-12T03:16:30.553126+0000', 'last_peered': '2024-09-12T03:16:30.553126+0000', 'last_clean': '2024-09-12T03:16:30.553126+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:16:30.553126+0000', 'last_undegraded': '2024-09-12T03:16:30.553126+0000', 'last_fullsized': '2024-09-12T03:16:30.553126+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:16:30.553126+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:16:30.553126+0000', 'last_clean_scrub_stamp': '2024-09-12T03:16:30.553126+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T03:16:35.268 INFO:tasks.ceph.ceph_manager.ceph:PG 2.b is not active+clean 2024-09-12T03:16:35.268 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-09-12T03:16:29.555670+0000', 'last_change': '2024-09-12T03:16:29.555670+0000', 'last_active': '2024-09-12T03:16:29.548038+0000', 'last_peered': '2024-09-12T03:16:29.548038+0000', 'last_clean': '2024-09-12T03:16:29.548038+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:16:29.555670+0000', 'last_undegraded': '2024-09-12T03:16:29.555670+0000', 'last_fullsized': '2024-09-12T03:16:29.555670+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:16:29.548038+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:16:29.548038+0000', 'last_clean_scrub_stamp': '2024-09-12T03:16:29.548038+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-12T03:16:35.268 INFO:tasks.ceph.ceph_manager.ceph:PG 3.a is not active+clean 2024-09-12T03:16:35.268 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T03:16:30.553126+0000', 'last_change': '2024-09-12T03:16:30.553126+0000', 'last_active': '2024-09-12T03:16:30.553126+0000', 'last_peered': '2024-09-12T03:16:30.553126+0000', 'last_clean': '2024-09-12T03:16:30.553126+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:16:30.553126+0000', 'last_undegraded': '2024-09-12T03:16:30.553126+0000', 'last_fullsized': '2024-09-12T03:16:30.553126+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:16:30.553126+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:16:30.553126+0000', 'last_clean_scrub_stamp': '2024-09-12T03:16:30.553126+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T03:16:35.268 INFO:tasks.ceph.ceph_manager.ceph:PG 3.1d is not active+clean 2024-09-12T03:16:35.269 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T03:16:30.553126+0000', 'last_change': '2024-09-12T03:16:30.553126+0000', 'last_active': '2024-09-12T03:16:30.553126+0000', 'last_peered': '2024-09-12T03:16:30.553126+0000', 'last_clean': '2024-09-12T03:16:30.553126+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:16:30.553126+0000', 'last_undegraded': '2024-09-12T03:16:30.553126+0000', 'last_fullsized': '2024-09-12T03:16:30.553126+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:16:30.553126+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:16:30.553126+0000', 'last_clean_scrub_stamp': '2024-09-12T03:16:30.553126+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T03:16:35.269 INFO:tasks.ceph.ceph_manager.ceph:PG 2.1c is not active+clean 2024-09-12T03:16:35.269 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-09-12T03:16:29.556203+0000', 'last_change': '2024-09-12T03:16:29.556203+0000', 'last_active': '2024-09-12T03:16:29.548038+0000', 'last_peered': '2024-09-12T03:16:29.548038+0000', 'last_clean': '2024-09-12T03:16:29.548038+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:16:29.556203+0000', 'last_undegraded': '2024-09-12T03:16:29.556203+0000', 'last_fullsized': '2024-09-12T03:16:29.556203+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:16:29.548038+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:16:29.548038+0000', 'last_clean_scrub_stamp': '2024-09-12T03:16:29.548038+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-12T03:16:35.269 INFO:tasks.ceph.ceph_manager.ceph:PG 3.1c is not active+clean 2024-09-12T03:16:35.269 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T03:16:30.553126+0000', 'last_change': '2024-09-12T03:16:30.553126+0000', 'last_active': '2024-09-12T03:16:30.553126+0000', 'last_peered': '2024-09-12T03:16:30.553126+0000', 'last_clean': '2024-09-12T03:16:30.553126+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:16:30.553126+0000', 'last_undegraded': '2024-09-12T03:16:30.553126+0000', 'last_fullsized': '2024-09-12T03:16:30.553126+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:16:30.553126+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:16:30.553126+0000', 'last_clean_scrub_stamp': '2024-09-12T03:16:30.553126+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T03:16:35.269 INFO:tasks.ceph.ceph_manager.ceph:PG 2.1d is not active+clean 2024-09-12T03:16:35.269 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-09-12T03:16:29.556594+0000', 'last_change': '2024-09-12T03:16:29.556594+0000', 'last_active': '2024-09-12T03:16:29.548038+0000', 'last_peered': '2024-09-12T03:16:29.548038+0000', 'last_clean': '2024-09-12T03:16:29.548038+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:16:29.556594+0000', 'last_undegraded': '2024-09-12T03:16:29.556594+0000', 'last_fullsized': '2024-09-12T03:16:29.556594+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:16:29.548038+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:16:29.548038+0000', 'last_clean_scrub_stamp': '2024-09-12T03:16:29.548038+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-12T03:16:35.269 INFO:tasks.ceph.ceph_manager.ceph:PG 3.1f is not active+clean 2024-09-12T03:16:35.269 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T03:16:30.553126+0000', 'last_change': '2024-09-12T03:16:30.553126+0000', 'last_active': '2024-09-12T03:16:30.553126+0000', 'last_peered': '2024-09-12T03:16:30.553126+0000', 'last_clean': '2024-09-12T03:16:30.553126+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:16:30.553126+0000', 'last_undegraded': '2024-09-12T03:16:30.553126+0000', 'last_fullsized': '2024-09-12T03:16:30.553126+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:16:30.553126+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:16:30.553126+0000', 'last_clean_scrub_stamp': '2024-09-12T03:16:30.553126+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T03:16:35.269 INFO:tasks.ceph.ceph_manager.ceph:PG 2.1e is not active+clean 2024-09-12T03:16:35.269 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-09-12T03:16:29.555746+0000', 'last_change': '2024-09-12T03:16:29.555746+0000', 'last_active': '2024-09-12T03:16:29.548038+0000', 'last_peered': '2024-09-12T03:16:29.548038+0000', 'last_clean': '2024-09-12T03:16:29.548038+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:16:29.555746+0000', 'last_undegraded': '2024-09-12T03:16:29.555746+0000', 'last_fullsized': '2024-09-12T03:16:29.555746+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:16:29.548038+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:16:29.548038+0000', 'last_clean_scrub_stamp': '2024-09-12T03:16:29.548038+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-12T03:16:35.270 INFO:tasks.ceph.ceph_manager.ceph:PG 3.1e is not active+clean 2024-09-12T03:16:35.270 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T03:16:30.553126+0000', 'last_change': '2024-09-12T03:16:30.553126+0000', 'last_active': '2024-09-12T03:16:30.553126+0000', 'last_peered': '2024-09-12T03:16:30.553126+0000', 'last_clean': '2024-09-12T03:16:30.553126+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:16:30.553126+0000', 'last_undegraded': '2024-09-12T03:16:30.553126+0000', 'last_fullsized': '2024-09-12T03:16:30.553126+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:16:30.553126+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:16:30.553126+0000', 'last_clean_scrub_stamp': '2024-09-12T03:16:30.553126+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T03:16:35.270 INFO:tasks.ceph.ceph_manager.ceph:PG 2.1f is not active+clean 2024-09-12T03:16:35.270 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-09-12T03:16:29.558604+0000', 'last_change': '2024-09-12T03:16:29.558604+0000', 'last_active': '2024-09-12T03:16:29.548038+0000', 'last_peered': '2024-09-12T03:16:29.548038+0000', 'last_clean': '2024-09-12T03:16:29.548038+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:16:29.558604+0000', 'last_undegraded': '2024-09-12T03:16:29.558604+0000', 'last_fullsized': '2024-09-12T03:16:29.558604+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:16:29.548038+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:16:29.548038+0000', 'last_clean_scrub_stamp': '2024-09-12T03:16:29.548038+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-12T03:16:35.271 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-12T03:16:35.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.391+0000 7fcea9b5f640 1 Processor -- start 2024-09-12T03:16:35.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.391+0000 7fcea9b5f640 1 -- start start 2024-09-12T03:16:35.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.391+0000 7fcea9b5f640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fcea415eb90 0x7fcea40d2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:35.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.391+0000 7fcea9b5f640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fcea40d1720 0x7fcea40b8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:35.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.391+0000 7fcea9b5f640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcea40d3760 con 0x7fcea40d5200 2024-09-12T03:16:35.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.391+0000 7fcea9b5f640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fcea40b7d20 con 0x7fcea40d1720 2024-09-12T03:16:35.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.391+0000 7fcea9b5f640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fcea40d42e0 con 0x7fcea415eb90 2024-09-12T03:16:35.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.392+0000 7fcea3fff640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fcea40d1720 0x7fcea40b8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:35.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.392+0000 7fcea37fe640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fcea415eb90 0x7fcea40d2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:35.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.392+0000 7fcea37fe640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fcea415eb90 0x7fcea40d2df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3301/0 says I am v2:172.21.15.27:37036/0 (socket says 172.21.15.27:37036) 2024-09-12T03:16:35.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.392+0000 7fcea2ffd640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fcea40d5200 0x7fcea40d1010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.27:36688/0 (socket says 172.21.15.27:36688) 2024-09-12T03:16:35.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.392+0000 7fcea37fe640 1 -- 172.21.15.27:0/2046746037 learned_addr learned my addr 172.21.15.27:0/2046746037 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:16:35.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.392+0000 7fcea37fe640 1 -- 172.21.15.27:0/2046746037 >> v2:172.21.15.45:3300/0 conn(0x7fcea40d1720 msgr2=0x7fcea40b8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:35.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.392+0000 7fcea37fe640 1 --2- 172.21.15.27:0/2046746037 >> v2:172.21.15.45:3300/0 conn(0x7fcea40d1720 0x7fcea40b8190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:35.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.392+0000 7fcea37fe640 1 -- 172.21.15.27:0/2046746037 >> v1:172.21.15.45:6789/0 conn(0x7fcea40d5200 legacy=0x7fcea40d1010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:35.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.392+0000 7fcea37fe640 1 -- 172.21.15.27:0/2046746037 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcea40d3a40 con 0x7fcea415eb90 2024-09-12T03:16:35.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.392+0000 7fcea27fc640 1 -- 172.21.15.27:0/2046746037 <== mon.1 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3554013840 0 0) 0x7fcea40d3760 con 0x7fcea40d5200 2024-09-12T03:16:35.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.392+0000 7fcea3fff640 1 --2- 172.21.15.27:0/2046746037 >> v2:172.21.15.45:3300/0 conn(0x7fcea40d1720 0x7fcea40b8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:16:35.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.393+0000 7fcea37fe640 1 --2- 172.21.15.27:0/2046746037 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fcea415eb90 0x7fcea40d2df0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fce94009870 tx=0x7fce9402a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9361ef5d37ce6c40 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:35.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.393+0000 7fcea27fc640 1 -- 172.21.15.27:0/2046746037 <== mon.2 v2:172.21.15.45:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce94037070 con 0x7fcea415eb90 2024-09-12T03:16:35.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.394+0000 7fcea27fc640 1 -- 172.21.15.27:0/2046746037 <== mon.2 v2:172.21.15.45:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce94032440 con 0x7fcea415eb90 2024-09-12T03:16:35.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.394+0000 7fcea27fc640 1 -- 172.21.15.27:0/2046746037 <== mon.2 v2:172.21.15.45:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce94032810 con 0x7fcea415eb90 2024-09-12T03:16:35.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.394+0000 7fcea9b5f640 1 -- 172.21.15.27:0/2046746037 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fcea415eb90 msgr2=0x7fcea40d2df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:35.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.394+0000 7fcea9b5f640 1 --2- 172.21.15.27:0/2046746037 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fcea415eb90 0x7fcea40d2df0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fce94009870 tx=0x7fce9402a270 comp rx=0 tx=0).stop 2024-09-12T03:16:35.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.394+0000 7fcea9b5f640 1 -- 172.21.15.27:0/2046746037 shutdown_connections 2024-09-12T03:16:35.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.394+0000 7fcea9b5f640 1 --2- 172.21.15.27:0/2046746037 >> v2:172.21.15.45:3300/0 conn(0x7fcea40d1720 0x7fcea40b8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:35.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.394+0000 7fcea9b5f640 1 --2- 172.21.15.27:0/2046746037 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fcea415eb90 0x7fcea40d2df0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:35.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.394+0000 7fcea9b5f640 1 -- 172.21.15.27:0/2046746037 >> 172.21.15.27:0/2046746037 conn(0x7fcea406dce0 msgr2=0x7fcea406e0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:35.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.394+0000 7fcea9b5f640 1 -- 172.21.15.27:0/2046746037 shutdown_connections 2024-09-12T03:16:35.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.394+0000 7fcea9b5f640 1 -- 172.21.15.27:0/2046746037 wait complete. 2024-09-12T03:16:35.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.395+0000 7fcea9b5f640 1 Processor -- start 2024-09-12T03:16:35.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.395+0000 7fcea9b5f640 1 -- start start 2024-09-12T03:16:35.397 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.396+0000 7fcea9b5f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcea40d1720 0x7fcea40ce1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:35.397 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.396+0000 7fcea9b5f640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcea40d5200 0x7fcea4148e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:35.397 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.396+0000 7fcea9b5f640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fcea415eb90 0x7fcea4156f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:35.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.396+0000 7fcea9b5f640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fcea40d3330 con 0x7fcea40d5200 2024-09-12T03:16:35.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.396+0000 7fcea9b5f640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fcea40d4000 con 0x7fcea40d1720 2024-09-12T03:16:35.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.396+0000 7fcea9b5f640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fcea40d34b0 con 0x7fcea415eb90 2024-09-12T03:16:35.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.396+0000 7fcea2ffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcea40d5200 0x7fcea4148e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:35.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.396+0000 7fcea2ffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcea40d5200 0x7fcea4148e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55568/0 (socket says 172.21.15.27:55568) 2024-09-12T03:16:35.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.396+0000 7fcea2ffd640 1 -- 172.21.15.27:0/3112895982 learned_addr learned my addr 172.21.15.27:0/3112895982 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:16:35.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.397+0000 7fcea3fff640 1 --2- 172.21.15.27:0/3112895982 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fcea415eb90 0x7fcea4156f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:35.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.397+0000 7fcea37fe640 1 --2- 172.21.15.27:0/3112895982 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcea40d1720 0x7fcea40ce1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:35.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.397+0000 7fcea2ffd640 1 -- 172.21.15.27:0/3112895982 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fcea415eb90 msgr2=0x7fcea4156f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:35.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.397+0000 7fcea2ffd640 1 --2- 172.21.15.27:0/3112895982 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fcea415eb90 0x7fcea4156f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:35.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.397+0000 7fcea2ffd640 1 -- 172.21.15.27:0/3112895982 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcea40d1720 msgr2=0x7fcea40ce1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:35.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.397+0000 7fcea2ffd640 1 --2- 172.21.15.27:0/3112895982 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcea40d1720 0x7fcea40ce1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:35.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.397+0000 7fcea2ffd640 1 -- 172.21.15.27:0/3112895982 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcea41597c0 con 0x7fcea40d5200 2024-09-12T03:16:35.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.397+0000 7fcea2ffd640 1 --2- 172.21.15.27:0/3112895982 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcea40d5200 0x7fcea4148e50 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fce9800c470 tx=0x7fce9800c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:35.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.397+0000 7fce8bfff640 1 -- 172.21.15.27:0/3112895982 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce98018070 con 0x7fcea40d5200 2024-09-12T03:16:35.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.397+0000 7fce8bfff640 1 -- 172.21.15.27:0/3112895982 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce980074e0 con 0x7fcea40d5200 2024-09-12T03:16:35.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.397+0000 7fcea9b5f640 1 -- 172.21.15.27:0/3112895982 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcea4159aa0 con 0x7fcea40d5200 2024-09-12T03:16:35.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.397+0000 7fce8bfff640 1 -- 172.21.15.27:0/3112895982 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce98004a20 con 0x7fcea40d5200 2024-09-12T03:16:35.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.397+0000 7fcea9b5f640 1 -- 172.21.15.27:0/3112895982 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcea415a060 con 0x7fcea40d5200 2024-09-12T03:16:35.400 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.398+0000 7fcea9b5f640 1 -- 172.21.15.27:0/3112895982 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcea40c98e0 con 0x7fcea40d5200 2024-09-12T03:16:35.400 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.399+0000 7fce8bfff640 1 -- 172.21.15.27:0/3112895982 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fce9801b070 con 0x7fcea40d5200 2024-09-12T03:16:35.400 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.399+0000 7fce8bfff640 1 --2- 172.21.15.27:0/3112895982 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fce9006be30 0x7fce9006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:35.400 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.399+0000 7fce8bfff640 1 -- 172.21.15.27:0/3112895982 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fce9808a0f0 con 0x7fcea40d5200 2024-09-12T03:16:35.400 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.399+0000 7fcea37fe640 1 --2- 172.21.15.27:0/3112895982 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fce9006be30 0x7fce9006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:35.400 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.400+0000 7fcea37fe640 1 --2- 172.21.15.27:0/3112895982 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fce9006be30 0x7fce9006e2f0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fce94034000 tx=0x7fce9402e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:35.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.403+0000 7fce8bfff640 1 -- 172.21.15.27:0/3112895982 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fcea40c98e0 con 0x7fcea40d5200 2024-09-12T03:16:35.650 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.650+0000 7fcea9b5f640 1 -- 172.21.15.27:0/3112895982 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7fcea415f000 con 0x7fcea40d5200 2024-09-12T03:16:35.650 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.650+0000 7fce8bfff640 1 -- 172.21.15.27:0/3112895982 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1149 (secure 0 0 0) 0x7fce98058b30 con 0x7fcea40d5200 2024-09-12T03:16:35.650 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:16:35.650 INFO:teuthology.orchestra.run.smithi027.stdout:{"fsid":"d2df504e-2647-4385-a5db-9fdf69f487a0","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":27,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":21,"num_osds":8,"num_up_osds":8,"osd_up_since":1726110979,"num_in_osds":8,"osd_in_since":1726110969,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"unknown","count":64},{"state_name":"creating+peering","count":32},{"state_name":"active+clean","count":1}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":241025024,"bytes_avail":772853088256,"bytes_total":773094113280,"unknown_pgs_ratio":0.65979379415512085,"inactive_pgs_ratio":0.32989689707756042},"fsmap":{"epoch":6,"btime":"2024-09-12T03:16:33:576418+0000","id":1,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":1,"rank":0,"name":"d","status":"up:active","gid":4285}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":1,"modified":"2024-09-12T03:16:07.780038+0000","services":{}},"progress_events":{}} 2024-09-12T03:16:35.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.652+0000 7fcea9b5f640 1 -- 172.21.15.27:0/3112895982 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fce9006be30 msgr2=0x7fce9006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:35.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.652+0000 7fcea9b5f640 1 --2- 172.21.15.27:0/3112895982 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fce9006be30 0x7fce9006e2f0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fce94034000 tx=0x7fce9402e000 comp rx=0 tx=0).stop 2024-09-12T03:16:35.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.652+0000 7fcea9b5f640 1 -- 172.21.15.27:0/3112895982 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcea40d5200 msgr2=0x7fcea4148e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:35.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.652+0000 7fcea9b5f640 1 --2- 172.21.15.27:0/3112895982 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcea40d5200 0x7fcea4148e50 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fce9800c470 tx=0x7fce9800c930 comp rx=0 tx=0).stop 2024-09-12T03:16:35.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.652+0000 7fcea9b5f640 1 -- 172.21.15.27:0/3112895982 shutdown_connections 2024-09-12T03:16:35.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.652+0000 7fcea9b5f640 1 --2- 172.21.15.27:0/3112895982 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fce9006be30 0x7fce9006e2f0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:35.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.652+0000 7fcea9b5f640 1 --2- 172.21.15.27:0/3112895982 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fcea415eb90 0x7fcea4156f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:35.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.652+0000 7fcea9b5f640 1 --2- 172.21.15.27:0/3112895982 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcea40d5200 0x7fcea4148e50 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:35.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.652+0000 7fcea9b5f640 1 --2- 172.21.15.27:0/3112895982 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcea40d1720 0x7fcea40ce1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:35.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.652+0000 7fcea9b5f640 1 -- 172.21.15.27:0/3112895982 >> 172.21.15.27:0/3112895982 conn(0x7fcea406dce0 msgr2=0x7fcea40d24a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:35.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.653+0000 7fcea9b5f640 1 -- 172.21.15.27:0/3112895982 shutdown_connections 2024-09-12T03:16:35.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.653+0000 7fcea9b5f640 1 -- 172.21.15.27:0/3112895982 wait complete. 2024-09-12T03:16:35.663 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-09-12T03:16:35.664 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-12T03:16:35.766 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.765+0000 7fddbdfc6640 1 Processor -- start 2024-09-12T03:16:35.766 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.766+0000 7fddbdfc6640 1 -- start start 2024-09-12T03:16:35.767 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.766+0000 7fddbdfc6640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fddb815eb90 0x7fddb80d2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:35.767 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.766+0000 7fddbdfc6640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fddb80d1720 0x7fddb80b8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:35.767 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.766+0000 7fddbdfc6640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fddb80d3760 con 0x7fddb80d5200 2024-09-12T03:16:35.767 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.766+0000 7fddbdfc6640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fddb80b7d20 con 0x7fddb80d1720 2024-09-12T03:16:35.767 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.766+0000 7fddbdfc6640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fddb80d42e0 con 0x7fddb815eb90 2024-09-12T03:16:35.767 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.767+0000 7fddb77fe640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fddb815eb90 0x7fddb80d2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:35.768 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.767+0000 7fddb7fff640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fddb80d1720 0x7fddb80b8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:35.768 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.767+0000 7fddb77fe640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fddb815eb90 0x7fddb80d2df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3301/0 says I am v2:172.21.15.27:37046/0 (socket says 172.21.15.27:37046) 2024-09-12T03:16:35.768 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.767+0000 7fddb77fe640 1 -- 172.21.15.27:0/2076274803 learned_addr learned my addr 172.21.15.27:0/2076274803 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:16:35.768 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.767+0000 7fddb6ffd640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fddb80d5200 0x7fddb80d1010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.27:36694/0 (socket says 172.21.15.27:36694) 2024-09-12T03:16:35.768 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.767+0000 7fddb77fe640 1 -- 172.21.15.27:0/2076274803 >> v2:172.21.15.45:3300/0 conn(0x7fddb80d1720 msgr2=0x7fddb80b8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:35.768 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.767+0000 7fddb77fe640 1 --2- 172.21.15.27:0/2076274803 >> v2:172.21.15.45:3300/0 conn(0x7fddb80d1720 0x7fddb80b8190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:35.768 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.767+0000 7fddb77fe640 1 -- 172.21.15.27:0/2076274803 >> v1:172.21.15.45:6789/0 conn(0x7fddb80d5200 legacy=0x7fddb80d1010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:35.768 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.767+0000 7fddb77fe640 1 -- 172.21.15.27:0/2076274803 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddb80d3a40 con 0x7fddb815eb90 2024-09-12T03:16:35.768 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.767+0000 7fddb7fff640 1 --2- 172.21.15.27:0/2076274803 >> v2:172.21.15.45:3300/0 conn(0x7fddb80d1720 0x7fddb80b8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:16:35.769 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.768+0000 7fddb77fe640 1 --2- 172.21.15.27:0/2076274803 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fddb815eb90 0x7fddb80d2df0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fdda8009e50 tx=0x7fdda802a710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5be9b8242acb4cee server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:35.770 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.768+0000 7fddb67fc640 1 -- 172.21.15.27:0/2076274803 <== mon.2 v2:172.21.15.45:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdda8036070 con 0x7fddb815eb90 2024-09-12T03:16:35.770 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.768+0000 7fddb67fc640 1 -- 172.21.15.27:0/2076274803 <== mon.2 v2:172.21.15.45:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdda8031600 con 0x7fddb815eb90 2024-09-12T03:16:35.770 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.768+0000 7fddb67fc640 1 -- 172.21.15.27:0/2076274803 <== mon.2 v2:172.21.15.45:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdda80319d0 con 0x7fddb815eb90 2024-09-12T03:16:35.770 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.769+0000 7fddbdfc6640 1 -- 172.21.15.27:0/2076274803 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fddb815eb90 msgr2=0x7fddb80d2df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:35.770 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.769+0000 7fddbdfc6640 1 --2- 172.21.15.27:0/2076274803 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fddb815eb90 0x7fddb80d2df0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fdda8009e50 tx=0x7fdda802a710 comp rx=0 tx=0).stop 2024-09-12T03:16:35.770 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.769+0000 7fddbdfc6640 1 -- 172.21.15.27:0/2076274803 shutdown_connections 2024-09-12T03:16:35.770 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.769+0000 7fddbdfc6640 1 --2- 172.21.15.27:0/2076274803 >> v2:172.21.15.45:3300/0 conn(0x7fddb80d1720 0x7fddb80b8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:35.770 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.769+0000 7fddbdfc6640 1 --2- 172.21.15.27:0/2076274803 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fddb815eb90 0x7fddb80d2df0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:35.770 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.769+0000 7fddbdfc6640 1 -- 172.21.15.27:0/2076274803 >> 172.21.15.27:0/2076274803 conn(0x7fddb806dce0 msgr2=0x7fddb806e0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:35.770 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.769+0000 7fddbdfc6640 1 -- 172.21.15.27:0/2076274803 shutdown_connections 2024-09-12T03:16:35.770 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.769+0000 7fddbdfc6640 1 -- 172.21.15.27:0/2076274803 wait complete. 2024-09-12T03:16:35.770 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.770+0000 7fddbdfc6640 1 Processor -- start 2024-09-12T03:16:35.772 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.770+0000 7fddbdfc6640 1 -- start start 2024-09-12T03:16:35.772 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.771+0000 7fddbdfc6640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fddb80d1720 0x7fddb8159260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:35.772 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.771+0000 7fddbdfc6640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fddb80d5200 0x7fddb815abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:35.772 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.771+0000 7fddbdfc6640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fddb815eb90 0x7fddb814eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:35.772 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.771+0000 7fddbdfc6640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fddb80d3330 con 0x7fddb80d5200 2024-09-12T03:16:35.773 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.771+0000 7fddbdfc6640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fddb80d4000 con 0x7fddb80d1720 2024-09-12T03:16:35.773 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.771+0000 7fddbdfc6640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fddb80d34b0 con 0x7fddb815eb90 2024-09-12T03:16:35.773 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.771+0000 7fddb6ffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fddb80d5200 0x7fddb815abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:35.773 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.771+0000 7fddb6ffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fddb80d5200 0x7fddb815abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55578/0 (socket says 172.21.15.27:55578) 2024-09-12T03:16:35.773 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.771+0000 7fddb6ffd640 1 -- 172.21.15.27:0/2474254021 learned_addr learned my addr 172.21.15.27:0/2474254021 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:16:35.773 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.771+0000 7fddb7fff640 1 --2- 172.21.15.27:0/2474254021 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fddb815eb90 0x7fddb814eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:35.773 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.771+0000 7fddb77fe640 1 --2- 172.21.15.27:0/2474254021 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fddb80d1720 0x7fddb8159260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:35.773 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.771+0000 7fddb6ffd640 1 -- 172.21.15.27:0/2474254021 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fddb815eb90 msgr2=0x7fddb814eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:35.773 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.771+0000 7fddb6ffd640 1 --2- 172.21.15.27:0/2474254021 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fddb815eb90 0x7fddb814eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:35.773 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.771+0000 7fddb6ffd640 1 -- 172.21.15.27:0/2474254021 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fddb80d1720 msgr2=0x7fddb8159260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:35.773 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.771+0000 7fddb6ffd640 1 --2- 172.21.15.27:0/2474254021 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fddb80d1720 0x7fddb8159260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:35.773 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.771+0000 7fddb6ffd640 1 -- 172.21.15.27:0/2474254021 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddb815b110 con 0x7fddb80d5200 2024-09-12T03:16:35.773 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.772+0000 7fddb6ffd640 1 --2- 172.21.15.27:0/2474254021 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fddb80d5200 0x7fddb815abd0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fddac00c420 tx=0x7fddac00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:35.773 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.772+0000 7fdd9ffff640 1 -- 172.21.15.27:0/2474254021 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddac0074e0 con 0x7fddb80d5200 2024-09-12T03:16:35.774 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.772+0000 7fdd9ffff640 1 -- 172.21.15.27:0/2474254021 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fddac007680 con 0x7fddb80d5200 2024-09-12T03:16:35.774 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.772+0000 7fdd9ffff640 1 -- 172.21.15.27:0/2474254021 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddac004a20 con 0x7fddb80d5200 2024-09-12T03:16:35.774 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.772+0000 7fddbdfc6640 1 -- 172.21.15.27:0/2474254021 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fddb8159940 con 0x7fddb80d5200 2024-09-12T03:16:35.774 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.772+0000 7fddbdfc6640 1 -- 172.21.15.27:0/2474254021 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fddb8159f00 con 0x7fddb80d5200 2024-09-12T03:16:35.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.773+0000 7fddbdfc6640 1 -- 172.21.15.27:0/2474254021 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fddb815f090 con 0x7fddb80d5200 2024-09-12T03:16:35.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.773+0000 7fdd9ffff640 1 -- 172.21.15.27:0/2474254021 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fddac01a020 con 0x7fddb80d5200 2024-09-12T03:16:35.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.774+0000 7fdd9ffff640 1 --2- 172.21.15.27:0/2474254021 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fdda406be30 0x7fdda406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:35.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.774+0000 7fdd9ffff640 1 -- 172.21.15.27:0/2474254021 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fddac08a4b0 con 0x7fddb80d5200 2024-09-12T03:16:35.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.774+0000 7fddb77fe640 1 --2- 172.21.15.27:0/2474254021 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fdda406be30 0x7fdda406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:35.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.775+0000 7fddb77fe640 1 --2- 172.21.15.27:0/2474254021 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fdda406be30 0x7fdda406e2f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fdda8033000 tx=0x7fdda8009000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:35.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.778+0000 7fdd9ffff640 1 -- 172.21.15.27:0/2474254021 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fddac0588a0 con 0x7fddb80d5200 2024-09-12T03:16:35.950 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.950+0000 7fddbdfc6640 1 -- 172.21.15.27:0/2474254021 --> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fddb815e340 con 0x7fdda406be30 2024-09-12T03:16:35.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.956+0000 7fdd9ffff640 1 -- 172.21.15.27:0/2474254021 <== mgr.4102 v2:172.21.15.45:6832/3923821219 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+240051 (secure 0 0 0) 0x7fddb815e340 con 0x7fdda406be30 2024-09-12T03:16:35.959 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:16:35.960 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2024-09-12T03:16:35.962 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.959+0000 7fddbdfc6640 1 -- 172.21.15.27:0/2474254021 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fdda406be30 msgr2=0x7fdda406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:35.962 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.959+0000 7fddbdfc6640 1 --2- 172.21.15.27:0/2474254021 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fdda406be30 0x7fdda406e2f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fdda8033000 tx=0x7fdda8009000 comp rx=0 tx=0).stop 2024-09-12T03:16:35.962 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.959+0000 7fddbdfc6640 1 -- 172.21.15.27:0/2474254021 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fddb80d5200 msgr2=0x7fddb815abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:35.962 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.959+0000 7fddbdfc6640 1 --2- 172.21.15.27:0/2474254021 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fddb80d5200 0x7fddb815abd0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fddac00c420 tx=0x7fddac00c8e0 comp rx=0 tx=0).stop 2024-09-12T03:16:35.963 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.959+0000 7fddbdfc6640 1 -- 172.21.15.27:0/2474254021 shutdown_connections 2024-09-12T03:16:35.963 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.959+0000 7fddbdfc6640 1 --2- 172.21.15.27:0/2474254021 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fdda406be30 0x7fdda406e2f0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:35.963 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.959+0000 7fddbdfc6640 1 --2- 172.21.15.27:0/2474254021 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fddb815eb90 0x7fddb814eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:35.963 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.959+0000 7fddbdfc6640 1 --2- 172.21.15.27:0/2474254021 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fddb80d5200 0x7fddb815abd0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:35.963 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.959+0000 7fddbdfc6640 1 --2- 172.21.15.27:0/2474254021 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fddb80d1720 0x7fddb8159260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:35.963 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.959+0000 7fddbdfc6640 1 -- 172.21.15.27:0/2474254021 >> 172.21.15.27:0/2474254021 conn(0x7fddb806dce0 msgr2=0x7fddb80cfcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:35.963 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.959+0000 7fddbdfc6640 1 -- 172.21.15.27:0/2474254021 shutdown_connections 2024-09-12T03:16:35.963 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:35.959+0000 7fddbdfc6640 1 -- 172.21.15.27:0/2474254021 wait complete. 2024-09-12T03:16:35.970 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2024-09-12T03:16:34.053081+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":8,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235376,"kb_used_data":2500,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754739344,"statfs":{"total":773094113280,"available":772853088256,"internally_reserved":0,"allocated":2560000,"data_stored":1601789,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.499689"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T03:16:29.555565+0000","last_change":"2024-09-12T03:16:29.555565+0000","last_active":"2024-09-12T03:16:29.548038+0000","last_peered":"2024-09-12T03:16:29.548038+0000","last_clean":"2024-09-12T03:16:29.548038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:29.555565+0000","last_undegraded":"2024-09-12T03:16:29.555565+0000","last_fullsized":"2024-09-12T03:16:29.555565+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T03:16:29.558592+0000","last_change":"2024-09-12T03:16:29.558592+0000","last_active":"2024-09-12T03:16:29.548038+0000","last_peered":"2024-09-12T03:16:29.548038+0000","last_clean":"2024-09-12T03:16:29.548038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:29.558592+0000","last_undegraded":"2024-09-12T03:16:29.558592+0000","last_fullsized":"2024-09-12T03:16:29.558592+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T03:16:29.555565+0000","last_change":"2024-09-12T03:16:29.555565+0000","last_active":"2024-09-12T03:16:29.548038+0000","last_peered":"2024-09-12T03:16:29.548038+0000","last_clean":"2024-09-12T03:16:29.548038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:29.555565+0000","last_undegraded":"2024-09-12T03:16:29.555565+0000","last_fullsized":"2024-09-12T03:16:29.555565+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T03:16:29.560599+0000","last_change":"2024-09-12T03:16:29.560599+0000","last_active":"2024-09-12T03:16:29.548038+0000","last_peered":"2024-09-12T03:16:29.548038+0000","last_clean":"2024-09-12T03:16:29.548038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:29.560599+0000","last_undegraded":"2024-09-12T03:16:29.560599+0000","last_fullsized":"2024-09-12T03:16:29.560599+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T03:16:29.559533+0000","last_change":"2024-09-12T03:16:29.559533+0000","last_active":"2024-09-12T03:16:29.548038+0000","last_peered":"2024-09-12T03:16:29.548038+0000","last_clean":"2024-09-12T03:16:29.548038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:29.559533+0000","last_undegraded":"2024-09-12T03:16:29.559533+0000","last_fullsized":"2024-09-12T03:16:29.559533+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T03:16:29.560546+0000","last_change":"2024-09-12T03:16:29.560546+0000","last_active":"2024-09-12T03:16:29.548038+0000","last_peered":"2024-09-12T03:16:29.548038+0000","last_clean":"2024-09-12T03:16:29.548038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:29.560546+0000","last_undegraded":"2024-09-12T03:16:29.560546+0000","last_fullsized":"2024-09-12T03:16:29.560546+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T03:16:29.560488+0000","last_change":"2024-09-12T03:16:29.560488+0000","last_active":"2024-09-12T03:16:29.548038+0000","last_peered":"2024-09-12T03:16:29.548038+0000","last_clean":"2024-09-12T03:16:29.548038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:29.560488+0000","last_undegraded":"2024-09-12T03:16:29.560488+0000","last_fullsized":"2024-09-12T03:16:29.560488+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T03:16:29.558583+0000","last_change":"2024-09-12T03:16:29.558583+0000","last_active":"2024-09-12T03:16:29.548038+0000","last_peered":"2024-09-12T03:16:29.548038+0000","last_clean":"2024-09-12T03:16:29.548038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:29.558583+0000","last_undegraded":"2024-09-12T03:16:29.558583+0000","last_fullsized":"2024-09-12T03:16:29.558583+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T03:16:29.558598+0000","last_change":"2024-09-12T03:16:29.558598+0000","last_active":"2024-09-12T03:16:29.548038+0000","last_peered":"2024-09-12T03:16:29.548038+0000","last_clean":"2024-09-12T03:16:29.548038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:29.558598+0000","last_undegraded":"2024-09-12T03:16:29.558598+0000","last_fullsized":"2024-09-12T03:16:29.558598+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T03:16:29.561109+0000","last_change":"2024-09-12T03:16:29.561109+0000","last_active":"2024-09-12T03:16:29.548038+0000","last_peered":"2024-09-12T03:16:29.548038+0000","last_clean":"2024-09-12T03:16:29.548038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:29.561109+0000","last_undegraded":"2024-09-12T03:16:29.561109+0000","last_fullsized":"2024-09-12T03:16:29.561109+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T03:16:29.558613+0000","last_change":"2024-09-12T03:16:29.558613+0000","last_active":"2024-09-12T03:16:29.548038+0000","last_peered":"2024-09-12T03:16:29.548038+0000","last_clean":"2024-09-12T03:16:29.548038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:29.558613+0000","last_undegraded":"2024-09-12T03:16:29.558613+0000","last_fullsized":"2024-09-12T03:16:29.558613+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T03:16:29.563582+0000","last_change":"2024-09-12T03:16:29.563582+0000","last_active":"2024-09-12T03:16:29.548038+0000","last_peered":"2024-09-12T03:16:29.548038+0000","last_clean":"2024-09-12T03:16:29.548038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:29.563582+0000","last_undegraded":"2024-09-12T03:16:29.563582+0000","last_fullsized":"2024-09-12T03:16:29.563582+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T03:16:29.555666+0000","last_change":"2024-09-12T03:16:29.555666+0000","last_active":"2024-09-12T03:16:29.548038+0000","last_peered":"2024-09-12T03:16:29.548038+0000","last_clean":"2024-09-12T03:16:29.548038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:29.555666+0000","last_undegraded":"2024-09-12T03:16:29.555666+0000","last_fullsized":"2024-09-12T03:16:29.555666+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T03:16:29.563556+0000","last_change":"2024-09-12T03:16:29.563556+0000","last_active":"2024-09-12T03:16:29.548038+0000","last_peered":"2024-09-12T03:16:29.548038+0000","last_clean":"2024-09-12T03:16:29.548038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:29.563556+0000","last_undegraded":"2024-09-12T03:16:29.563556+0000","last_fullsized":"2024-09-12T03:16:29.563556+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T03:16:29.559421+0000","last_change":"2024-09-12T03:16:29.559421+0000","last_active":"2024-09-12T03:16:29.548038+0000","last_peered":"2024-09-12T03:16:29.548038+0000","last_clean":"2024-09-12T03:16:29.548038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:29.559421+0000","last_undegraded":"2024-09-12T03:16:29.559421+0000","last_fullsized":"2024-09-12T03:16:29.559421+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T03:16:29.563546+0000","last_change":"2024-09-12T03:16:29.563546+0000","last_active":"2024-09-12T03:16:29.548038+0000","last_peered":"2024-09-12T03:16:29.548038+0000","last_clean":"2024-09-12T03:16:29.548038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:29.563546+0000","last_undegraded":"2024-09-12T03:16:29.563546+0000","last_fullsized":"2024-09-12T03:16:29.563546+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":164,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-12T03:16:29.557520+0000","last_change":"2024-09-12T03:16:21.883446+0000","last_active":"2024-09-12T03:16:29.557520+0000","last_peered":"2024-09-12T03:16:29.557520+0000","last_clean":"2024-09-12T03:16:29.557520+0000","last_became_active":"2024-09-12T03:16:21.882207+0000","last_became_peered":"2024-09-12T03:16:21.882207+0000","last_unstale":"2024-09-12T03:16:29.557520+0000","last_undegraded":"2024-09-12T03:16:29.557520+0000","last_fullsized":"2024-09-12T03:16:29.557520+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:20.316426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:20.316426+0000","last_clean_scrub_stamp":"2024-09-12T03:16:20.316426+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:12:15.424320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T03:16:29.560957+0000","last_change":"2024-09-12T03:16:29.560957+0000","last_active":"2024-09-12T03:16:29.548038+0000","last_peered":"2024-09-12T03:16:29.548038+0000","last_clean":"2024-09-12T03:16:29.548038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:29.560957+0000","last_undegraded":"2024-09-12T03:16:29.560957+0000","last_fullsized":"2024-09-12T03:16:29.560957+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T03:16:29.556208+0000","last_change":"2024-09-12T03:16:29.556208+0000","last_active":"2024-09-12T03:16:29.548038+0000","last_peered":"2024-09-12T03:16:29.548038+0000","last_clean":"2024-09-12T03:16:29.548038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:29.556208+0000","last_undegraded":"2024-09-12T03:16:29.556208+0000","last_fullsized":"2024-09-12T03:16:29.556208+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T03:16:29.563487+0000","last_change":"2024-09-12T03:16:29.563487+0000","last_active":"2024-09-12T03:16:29.548038+0000","last_peered":"2024-09-12T03:16:29.548038+0000","last_clean":"2024-09-12T03:16:29.548038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:29.563487+0000","last_undegraded":"2024-09-12T03:16:29.563487+0000","last_fullsized":"2024-09-12T03:16:29.563487+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T03:16:29.560584+0000","last_change":"2024-09-12T03:16:29.560584+0000","last_active":"2024-09-12T03:16:29.548038+0000","last_peered":"2024-09-12T03:16:29.548038+0000","last_clean":"2024-09-12T03:16:29.548038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:29.560584+0000","last_undegraded":"2024-09-12T03:16:29.560584+0000","last_fullsized":"2024-09-12T03:16:29.560584+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T03:16:29.559139+0000","last_change":"2024-09-12T03:16:29.559139+0000","last_active":"2024-09-12T03:16:29.548038+0000","last_peered":"2024-09-12T03:16:29.548038+0000","last_clean":"2024-09-12T03:16:29.548038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:29.559139+0000","last_undegraded":"2024-09-12T03:16:29.559139+0000","last_fullsized":"2024-09-12T03:16:29.559139+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T03:16:29.555833+0000","last_change":"2024-09-12T03:16:29.555833+0000","last_active":"2024-09-12T03:16:29.548038+0000","last_peered":"2024-09-12T03:16:29.548038+0000","last_clean":"2024-09-12T03:16:29.548038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:29.555833+0000","last_undegraded":"2024-09-12T03:16:29.555833+0000","last_fullsized":"2024-09-12T03:16:29.555833+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T03:16:29.559271+0000","last_change":"2024-09-12T03:16:29.559271+0000","last_active":"2024-09-12T03:16:29.548038+0000","last_peered":"2024-09-12T03:16:29.548038+0000","last_clean":"2024-09-12T03:16:29.548038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:29.559271+0000","last_undegraded":"2024-09-12T03:16:29.559271+0000","last_fullsized":"2024-09-12T03:16:29.559271+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T03:16:29.558849+0000","last_change":"2024-09-12T03:16:29.558849+0000","last_active":"2024-09-12T03:16:29.548038+0000","last_peered":"2024-09-12T03:16:29.548038+0000","last_clean":"2024-09-12T03:16:29.548038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:29.558849+0000","last_undegraded":"2024-09-12T03:16:29.558849+0000","last_fullsized":"2024-09-12T03:16:29.558849+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T03:16:29.557099+0000","last_change":"2024-09-12T03:16:29.557099+0000","last_active":"2024-09-12T03:16:29.548038+0000","last_peered":"2024-09-12T03:16:29.548038+0000","last_clean":"2024-09-12T03:16:29.548038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:29.557099+0000","last_undegraded":"2024-09-12T03:16:29.557099+0000","last_fullsized":"2024-09-12T03:16:29.557099+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T03:16:29.559156+0000","last_change":"2024-09-12T03:16:29.559156+0000","last_active":"2024-09-12T03:16:29.548038+0000","last_peered":"2024-09-12T03:16:29.548038+0000","last_clean":"2024-09-12T03:16:29.548038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:29.559156+0000","last_undegraded":"2024-09-12T03:16:29.559156+0000","last_fullsized":"2024-09-12T03:16:29.559156+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T03:16:29.558951+0000","last_change":"2024-09-12T03:16:29.558951+0000","last_active":"2024-09-12T03:16:29.548038+0000","last_peered":"2024-09-12T03:16:29.548038+0000","last_clean":"2024-09-12T03:16:29.548038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:29.558951+0000","last_undegraded":"2024-09-12T03:16:29.558951+0000","last_fullsized":"2024-09-12T03:16:29.558951+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T03:16:29.555670+0000","last_change":"2024-09-12T03:16:29.555670+0000","last_active":"2024-09-12T03:16:29.548038+0000","last_peered":"2024-09-12T03:16:29.548038+0000","last_clean":"2024-09-12T03:16:29.548038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:29.555670+0000","last_undegraded":"2024-09-12T03:16:29.555670+0000","last_fullsized":"2024-09-12T03:16:29.555670+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T03:16:29.556203+0000","last_change":"2024-09-12T03:16:29.556203+0000","last_active":"2024-09-12T03:16:29.548038+0000","last_peered":"2024-09-12T03:16:29.548038+0000","last_clean":"2024-09-12T03:16:29.548038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:29.556203+0000","last_undegraded":"2024-09-12T03:16:29.556203+0000","last_fullsized":"2024-09-12T03:16:29.556203+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T03:16:29.556594+0000","last_change":"2024-09-12T03:16:29.556594+0000","last_active":"2024-09-12T03:16:29.548038+0000","last_peered":"2024-09-12T03:16:29.548038+0000","last_clean":"2024-09-12T03:16:29.548038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:29.556594+0000","last_undegraded":"2024-09-12T03:16:29.556594+0000","last_fullsized":"2024-09-12T03:16:29.556594+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T03:16:29.555746+0000","last_change":"2024-09-12T03:16:29.555746+0000","last_active":"2024-09-12T03:16:29.548038+0000","last_peered":"2024-09-12T03:16:29.548038+0000","last_clean":"2024-09-12T03:16:29.548038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:29.555746+0000","last_undegraded":"2024-09-12T03:16:29.555746+0000","last_fullsized":"2024-09-12T03:16:29.555746+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:16:30.553126+0000","last_change":"2024-09-12T03:16:30.553126+0000","last_active":"2024-09-12T03:16:30.553126+0000","last_peered":"2024-09-12T03:16:30.553126+0000","last_clean":"2024-09-12T03:16:30.553126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:30.553126+0000","last_undegraded":"2024-09-12T03:16:30.553126+0000","last_fullsized":"2024-09-12T03:16:30.553126+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T03:16:29.558604+0000","last_change":"2024-09-12T03:16:29.558604+0000","last_active":"2024-09-12T03:16:29.548038+0000","last_peered":"2024-09-12T03:16:29.548038+0000","last_clean":"2024-09-12T03:16:29.548038+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:16:29.558604+0000","last_undegraded":"2024-09-12T03:16:29.558604+0000","last_fullsized":"2024-09-12T03:16:29.558604+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542147,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31940,"kb_used_data":752,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339900,"statfs":{"total":96636764160,"available":96604057600,"internally_reserved":0,"allocated":770048,"data_stored":645181,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542149,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31352,"kb_used_data":160,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340488,"statfs":{"total":96636764160,"available":96604659712,"internally_reserved":0,"allocated":163840,"data_stored":48996,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542148,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31360,"kb_used_data":172,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340480,"statfs":{"total":96636764160,"available":96604651520,"internally_reserved":0,"allocated":176128,"data_stored":54813,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542147,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31352,"kb_used_data":160,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340488,"statfs":{"total":96636764160,"available":96604659712,"internally_reserved":0,"allocated":163840,"data_stored":48996,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542148,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27192,"kb_used_data":160,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344648,"statfs":{"total":96636764160,"available":96608919552,"internally_reserved":0,"allocated":163840,"data_stored":48996,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542148,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27200,"kb_used_data":172,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344640,"statfs":{"total":96636764160,"available":96608911360,"internally_reserved":0,"allocated":176128,"data_stored":54813,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542149,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27200,"kb_used_data":172,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344640,"statfs":{"total":96636764160,"available":96608911360,"internally_reserved":0,"allocated":176128,"data_stored":54813,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542149,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27780,"kb_used_data":752,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344060,"statfs":{"total":96636764160,"available":96608317440,"internally_reserved":0,"allocated":770048,"data_stored":645181,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T03:16:37.546 INFO:tasks.ceph.mds.d.smithi045.stderr:2024-09-12T03:16:37.545+0000 7f8cfe70b640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-12T03:16:38.975 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-12T03:16:39.090 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.090+0000 7fd952e82640 1 Processor -- start 2024-09-12T03:16:39.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.090+0000 7fd952e82640 1 -- start start 2024-09-12T03:16:39.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.091+0000 7fd952e82640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fd94c15eb90 0x7fd94c0d2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:39.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.091+0000 7fd952e82640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd94c0d3760 con 0x7fd94c0d5200 2024-09-12T03:16:39.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.091+0000 7fd952e82640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd94c0d3a40 con 0x7fd94c0d1720 2024-09-12T03:16:39.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.091+0000 7fd952e82640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fd94c0b7d20 con 0x7fd94c15eb90 2024-09-12T03:16:39.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.091+0000 7fd94bfff640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fd94c0d5200 0x7fd94c0d1010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:51434/0 (socket says 172.21.15.27:51434) 2024-09-12T03:16:39.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.091+0000 7fd94bfff640 1 -- 172.21.15.27:0/1748594751 learned_addr learned my addr 172.21.15.27:0/1748594751 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:16:39.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.092+0000 7fd950c1e640 1 --2- 172.21.15.27:0/1748594751 >> v2:172.21.15.45:3300/0 conn(0x7fd94c15eb90 0x7fd94c0d2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:39.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.092+0000 7fd94b7fe640 1 -- 172.21.15.27:0/1748594751 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 979596383 0 0) 0x7fd94c0d3760 con 0x7fd94c0d5200 2024-09-12T03:16:39.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.092+0000 7fd94b7fe640 1 -- 172.21.15.27:0/1748594751 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd93c003520 con 0x7fd94c0d5200 2024-09-12T03:16:39.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.092+0000 7fd94b7fe640 1 -- 172.21.15.27:0/1748594751 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2785208993 0 0) 0x7fd93c003520 con 0x7fd94c0d5200 2024-09-12T03:16:39.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.092+0000 7fd94b7fe640 1 -- 172.21.15.27:0/1748594751 >> v2:172.21.15.45:3300/0 conn(0x7fd94c15eb90 msgr2=0x7fd94c0d2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:39.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.092+0000 7fd94b7fe640 1 --2- 172.21.15.27:0/1748594751 >> v2:172.21.15.45:3300/0 conn(0x7fd94c15eb90 0x7fd94c0d2df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:39.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.092+0000 7fd94b7fe640 1 -- 172.21.15.27:0/1748594751 >> v1:172.21.15.45:6789/0 conn(0x7fd94c0d1720 legacy=0x7fd94c0b8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:39.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.092+0000 7fd94b7fe640 1 -- 172.21.15.27:0/1748594751 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd94c0d3d20 con 0x7fd94c0d5200 2024-09-12T03:16:39.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.092+0000 7fd94b7fe640 1 -- 172.21.15.27:0/1748594751 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 847755213 0 0) 0x7fd938001670 con 0x7fd94c0d5200 2024-09-12T03:16:39.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.092+0000 7fd94b7fe640 1 -- 172.21.15.27:0/1748594751 >> v1:172.21.15.27:6789/0 conn(0x7fd94c0d5200 legacy=0x7fd94c0d1010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:39.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.092+0000 7fd94b7fe640 1 --2- 172.21.15.27:0/1748594751 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fd93c0039b0 0x7fd93c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:39.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.092+0000 7fd94b7fe640 1 --2- 172.21.15.27:0/1748594751 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd93c004800 0x7fd93c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:39.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.092+0000 7fd94b7fe640 1 --2- 172.21.15.27:0/1748594751 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd93c0051c0 0x7fd93c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:39.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.092+0000 7fd94b7fe640 1 -- 172.21.15.27:0/1748594751 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fd93c00c030 con 0x7fd93c004800 2024-09-12T03:16:39.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.092+0000 7fd94b7fe640 1 -- 172.21.15.27:0/1748594751 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fd93c00c1e0 con 0x7fd93c0051c0 2024-09-12T03:16:39.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.092+0000 7fd94b7fe640 1 -- 172.21.15.27:0/1748594751 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fd93c00c3e0 con 0x7fd93c0039b0 2024-09-12T03:16:39.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.092+0000 7fd94bfff640 1 --2- 172.21.15.27:0/1748594751 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd93c004800 0x7fd93c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:39.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.092+0000 7fd950c1e640 1 --2- 172.21.15.27:0/1748594751 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fd93c0039b0 0x7fd93c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:39.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.092+0000 7fd95141f640 1 --2- 172.21.15.27:0/1748594751 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd93c0051c0 0x7fd93c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:39.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.092+0000 7fd94bfff640 1 -- 172.21.15.27:0/1748594751 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fd93c0039b0 msgr2=0x7fd93c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:39.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.092+0000 7fd94bfff640 1 --2- 172.21.15.27:0/1748594751 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fd93c0039b0 0x7fd93c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:39.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.092+0000 7fd94bfff640 1 -- 172.21.15.27:0/1748594751 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd93c0051c0 msgr2=0x7fd93c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:39.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.092+0000 7fd94bfff640 1 --2- 172.21.15.27:0/1748594751 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd93c0051c0 0x7fd93c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:39.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.092+0000 7fd95141f640 1 -- 172.21.15.27:0/1748594751 reap_dead start 2024-09-12T03:16:39.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.092+0000 7fd94bfff640 1 -- 172.21.15.27:0/1748594751 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd93c003520 con 0x7fd93c004800 2024-09-12T03:16:39.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.093+0000 7fd94bfff640 1 --2- 172.21.15.27:0/1748594751 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd93c004800 0x7fd93c004c80 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fd93800c210 tx=0x7fd93c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be8b097e3a0f4ab5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:39.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.093+0000 7fd94b7fe640 1 -- 172.21.15.27:0/1748594751 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd938007cf0 con 0x7fd93c004800 2024-09-12T03:16:39.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.093+0000 7fd94b7fe640 1 -- 172.21.15.27:0/1748594751 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd938005af0 con 0x7fd93c004800 2024-09-12T03:16:39.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.093+0000 7fd94b7fe640 1 -- 172.21.15.27:0/1748594751 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd938007070 con 0x7fd93c004800 2024-09-12T03:16:39.100 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.094+0000 7fd952e82640 1 -- 172.21.15.27:0/1748594751 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd93c004800 msgr2=0x7fd93c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:39.100 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.094+0000 7fd952e82640 1 --2- 172.21.15.27:0/1748594751 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd93c004800 0x7fd93c004c80 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fd93800c210 tx=0x7fd93c005830 comp rx=0 tx=0).stop 2024-09-12T03:16:39.100 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.094+0000 7fd952e82640 1 -- 172.21.15.27:0/1748594751 shutdown_connections 2024-09-12T03:16:39.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.094+0000 7fd952e82640 1 --2- 172.21.15.27:0/1748594751 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd93c004800 0x7fd93c004c80 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:39.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.094+0000 7fd952e82640 1 -- 172.21.15.27:0/1748594751 >> 172.21.15.27:0/1748594751 conn(0x7fd94c06dce0 msgr2=0x7fd94c06e0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:39.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.094+0000 7fd952e82640 1 -- 172.21.15.27:0/1748594751 shutdown_connections 2024-09-12T03:16:39.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.094+0000 7fd952e82640 1 -- 172.21.15.27:0/1748594751 wait complete. 2024-09-12T03:16:39.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.095+0000 7fd952e82640 1 Processor -- start 2024-09-12T03:16:39.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.095+0000 7fd952e82640 1 -- start start 2024-09-12T03:16:39.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.096+0000 7fd952e82640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd93c004800 0x7fd94c1e8b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:39.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.096+0000 7fd952e82640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd94c15ac00 0x7fd94c1e7290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:39.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.096+0000 7fd952e82640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fd94c15a420 0x7fd94c1603d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:39.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.096+0000 7fd952e82640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fd94c15ef70 con 0x7fd93c004800 2024-09-12T03:16:39.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.096+0000 7fd952e82640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fd94c0d42e0 con 0x7fd94c15ac00 2024-09-12T03:16:39.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.096+0000 7fd952e82640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fd94c15f0f0 con 0x7fd94c15a420 2024-09-12T03:16:39.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.096+0000 7fd950c1e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd93c004800 0x7fd94c1e8b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:39.102 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.096+0000 7fd950c1e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd93c004800 0x7fd94c1e8b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55600/0 (socket says 172.21.15.27:55600) 2024-09-12T03:16:39.102 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.096+0000 7fd950c1e640 1 -- 172.21.15.27:0/3616935778 learned_addr learned my addr 172.21.15.27:0/3616935778 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:16:39.102 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.096+0000 7fd95141f640 1 --2- 172.21.15.27:0/3616935778 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fd94c15a420 0x7fd94c1603d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:39.102 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.097+0000 7fd94bfff640 1 --2- 172.21.15.27:0/3616935778 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd94c15ac00 0x7fd94c1e7290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:39.102 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.097+0000 7fd950c1e640 1 -- 172.21.15.27:0/3616935778 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fd94c15a420 msgr2=0x7fd94c1603d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:39.102 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.097+0000 7fd950c1e640 1 --2- 172.21.15.27:0/3616935778 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fd94c15a420 0x7fd94c1603d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:39.102 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.097+0000 7fd950c1e640 1 -- 172.21.15.27:0/3616935778 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd94c15ac00 msgr2=0x7fd94c1e7290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:39.102 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.097+0000 7fd950c1e640 1 --2- 172.21.15.27:0/3616935778 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd94c15ac00 0x7fd94c1e7290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:39.102 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.097+0000 7fd950c1e640 1 -- 172.21.15.27:0/3616935778 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd94c1e77d0 con 0x7fd93c004800 2024-09-12T03:16:39.102 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.097+0000 7fd95141f640 1 --2- 172.21.15.27:0/3616935778 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fd94c15a420 0x7fd94c1603d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:16:39.102 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.097+0000 7fd950c1e640 1 --2- 172.21.15.27:0/3616935778 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd93c004800 0x7fd94c1e8b10 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fd940011890 tx=0x7fd940011d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:39.102 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.097+0000 7fd9497fa640 1 -- 172.21.15.27:0/3616935778 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd940019070 con 0x7fd93c004800 2024-09-12T03:16:39.102 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.097+0000 7fd9497fa640 1 -- 172.21.15.27:0/3616935778 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd940004240 con 0x7fd93c004800 2024-09-12T03:16:39.102 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.097+0000 7fd952e82640 1 -- 172.21.15.27:0/3616935778 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd94c1e7a70 con 0x7fd93c004800 2024-09-12T03:16:39.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.097+0000 7fd9497fa640 1 -- 172.21.15.27:0/3616935778 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd94000f7c0 con 0x7fd93c004800 2024-09-12T03:16:39.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.097+0000 7fd952e82640 1 -- 172.21.15.27:0/3616935778 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd94c1e8030 con 0x7fd93c004800 2024-09-12T03:16:39.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.098+0000 7fd952e82640 1 -- 172.21.15.27:0/3616935778 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd94c0cfc40 con 0x7fd93c004800 2024-09-12T03:16:39.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.099+0000 7fd9497fa640 1 -- 172.21.15.27:0/3616935778 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd94001c070 con 0x7fd93c004800 2024-09-12T03:16:39.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.099+0000 7fd9497fa640 1 --2- 172.21.15.27:0/3616935778 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fd93006be30 0x7fd93006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:39.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.099+0000 7fd9497fa640 1 -- 172.21.15.27:0/3616935778 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fd94008b220 con 0x7fd93c004800 2024-09-12T03:16:39.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.099+0000 7fd94bfff640 1 --2- 172.21.15.27:0/3616935778 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fd93006be30 0x7fd93006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:39.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.100+0000 7fd94bfff640 1 --2- 172.21.15.27:0/3616935778 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fd93006be30 0x7fd93006e2f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fd938032000 tx=0x7fd938009030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:39.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.104+0000 7fd9497fa640 1 -- 172.21.15.27:0/3616935778 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fd94c0cfc40 con 0x7fd93c004800 2024-09-12T03:16:39.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.275+0000 7fd952e82640 1 -- 172.21.15.27:0/3616935778 --> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd94c0ce980 con 0x7fd93006be30 2024-09-12T03:16:39.282 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.282+0000 7fd9497fa640 1 -- 172.21.15.27:0/3616935778 <== mgr.4102 v2:172.21.15.45:6832/3923821219 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251948 (secure 0 0 0) 0x7fd94c0ce980 con 0x7fd93006be30 2024-09-12T03:16:39.285 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:16:39.286 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2024-09-12T03:16:39.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.285+0000 7fd952e82640 1 -- 172.21.15.27:0/3616935778 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fd93006be30 msgr2=0x7fd93006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:39.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.285+0000 7fd952e82640 1 --2- 172.21.15.27:0/3616935778 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fd93006be30 0x7fd93006e2f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fd938032000 tx=0x7fd938009030 comp rx=0 tx=0).stop 2024-09-12T03:16:39.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.285+0000 7fd952e82640 1 -- 172.21.15.27:0/3616935778 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd93c004800 msgr2=0x7fd94c1e8b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:39.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.285+0000 7fd952e82640 1 --2- 172.21.15.27:0/3616935778 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd93c004800 0x7fd94c1e8b10 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fd940011890 tx=0x7fd940011d50 comp rx=0 tx=0).stop 2024-09-12T03:16:39.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.285+0000 7fd952e82640 1 -- 172.21.15.27:0/3616935778 shutdown_connections 2024-09-12T03:16:39.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.285+0000 7fd952e82640 1 --2- 172.21.15.27:0/3616935778 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fd93006be30 0x7fd93006e2f0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:39.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.285+0000 7fd952e82640 1 --2- 172.21.15.27:0/3616935778 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fd94c15a420 0x7fd94c1603d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:39.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.285+0000 7fd952e82640 1 --2- 172.21.15.27:0/3616935778 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd94c15ac00 0x7fd94c1e7290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:39.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.285+0000 7fd952e82640 1 --2- 172.21.15.27:0/3616935778 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd93c004800 0x7fd94c1e8b10 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:39.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.285+0000 7fd952e82640 1 -- 172.21.15.27:0/3616935778 >> 172.21.15.27:0/3616935778 conn(0x7fd94c06dce0 msgr2=0x7fd94c0b8050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:39.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.285+0000 7fd952e82640 1 -- 172.21.15.27:0/3616935778 shutdown_connections 2024-09-12T03:16:39.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.285+0000 7fd952e82640 1 -- 172.21.15.27:0/3616935778 wait complete. 2024-09-12T03:16:39.296 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2024-09-12T03:16:38.054051+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235760,"kb_used_data":2888,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738960,"statfs":{"total":773094113280,"available":772852695040,"internally_reserved":0,"allocated":2957312,"data_stored":1778860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":32,"apply_latency_ms":32,"commit_latency_ns":32000000,"apply_latency_ns":32000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.494508"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:33.384323+0000","last_change":"2024-09-12T03:16:31.583498+0000","last_active":"2024-09-12T03:16:33.384323+0000","last_peered":"2024-09-12T03:16:33.384323+0000","last_clean":"2024-09-12T03:16:33.384323+0000","last_became_active":"2024-09-12T03:16:31.583196+0000","last_became_peered":"2024-09-12T03:16:31.583196+0000","last_unstale":"2024-09-12T03:16:33.384323+0000","last_undegraded":"2024-09-12T03:16:33.384323+0000","last_fullsized":"2024-09-12T03:16:33.384323+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:08:48.647955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:32.578394+0000","last_change":"2024-09-12T03:16:31.584109+0000","last_active":"2024-09-12T03:16:32.578394+0000","last_peered":"2024-09-12T03:16:32.578394+0000","last_clean":"2024-09-12T03:16:32.578394+0000","last_became_active":"2024-09-12T03:16:31.583836+0000","last_became_peered":"2024-09-12T03:16:31.583836+0000","last_unstale":"2024-09-12T03:16:32.578394+0000","last_undegraded":"2024-09-12T03:16:32.578394+0000","last_fullsized":"2024-09-12T03:16:32.578394+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:27:20.447453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:33.384286+0000","last_change":"2024-09-12T03:16:31.574829+0000","last_active":"2024-09-12T03:16:33.384286+0000","last_peered":"2024-09-12T03:16:33.384286+0000","last_clean":"2024-09-12T03:16:33.384286+0000","last_became_active":"2024-09-12T03:16:31.574523+0000","last_became_peered":"2024-09-12T03:16:31.574523+0000","last_unstale":"2024-09-12T03:16:33.384286+0000","last_undegraded":"2024-09-12T03:16:33.384286+0000","last_fullsized":"2024-09-12T03:16:33.384286+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:33:01.912717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:32.580588+0000","last_change":"2024-09-12T03:16:31.585853+0000","last_active":"2024-09-12T03:16:32.580588+0000","last_peered":"2024-09-12T03:16:32.580588+0000","last_clean":"2024-09-12T03:16:32.580588+0000","last_became_active":"2024-09-12T03:16:31.583206+0000","last_became_peered":"2024-09-12T03:16:31.583206+0000","last_unstale":"2024-09-12T03:16:32.580588+0000","last_undegraded":"2024-09-12T03:16:32.580588+0000","last_fullsized":"2024-09-12T03:16:32.580588+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:05:29.506554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:33.383913+0000","last_change":"2024-09-12T03:16:31.582095+0000","last_active":"2024-09-12T03:16:33.383913+0000","last_peered":"2024-09-12T03:16:33.383913+0000","last_clean":"2024-09-12T03:16:33.383913+0000","last_became_active":"2024-09-12T03:16:31.581414+0000","last_became_peered":"2024-09-12T03:16:31.581414+0000","last_unstale":"2024-09-12T03:16:33.383913+0000","last_undegraded":"2024-09-12T03:16:33.383913+0000","last_fullsized":"2024-09-12T03:16:33.383913+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:44:06.896638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:32.579523+0000","last_change":"2024-09-12T03:16:31.584676+0000","last_active":"2024-09-12T03:16:32.579523+0000","last_peered":"2024-09-12T03:16:32.579523+0000","last_clean":"2024-09-12T03:16:32.579523+0000","last_became_active":"2024-09-12T03:16:31.584392+0000","last_became_peered":"2024-09-12T03:16:31.584392+0000","last_unstale":"2024-09-12T03:16:32.579523+0000","last_undegraded":"2024-09-12T03:16:32.579523+0000","last_fullsized":"2024-09-12T03:16:32.579523+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:03:19.355158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:33.383478+0000","last_change":"2024-09-12T03:16:31.587169+0000","last_active":"2024-09-12T03:16:33.383478+0000","last_peered":"2024-09-12T03:16:33.383478+0000","last_clean":"2024-09-12T03:16:33.383478+0000","last_became_active":"2024-09-12T03:16:31.586884+0000","last_became_peered":"2024-09-12T03:16:31.586884+0000","last_unstale":"2024-09-12T03:16:33.383478+0000","last_undegraded":"2024-09-12T03:16:33.383478+0000","last_fullsized":"2024-09-12T03:16:33.383478+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:35:22.374781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:33.383041+0000","last_change":"2024-09-12T03:16:31.592684+0000","last_active":"2024-09-12T03:16:33.383041+0000","last_peered":"2024-09-12T03:16:33.383041+0000","last_clean":"2024-09-12T03:16:33.383041+0000","last_became_active":"2024-09-12T03:16:31.592374+0000","last_became_peered":"2024-09-12T03:16:31.592374+0000","last_unstale":"2024-09-12T03:16:33.383041+0000","last_undegraded":"2024-09-12T03:16:33.383041+0000","last_fullsized":"2024-09-12T03:16:33.383041+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:18:57.036317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:33.384083+0000","last_change":"2024-09-12T03:16:31.590609+0000","last_active":"2024-09-12T03:16:33.384083+0000","last_peered":"2024-09-12T03:16:33.384083+0000","last_clean":"2024-09-12T03:16:33.384083+0000","last_became_active":"2024-09-12T03:16:31.590320+0000","last_became_peered":"2024-09-12T03:16:31.590320+0000","last_unstale":"2024-09-12T03:16:33.384083+0000","last_undegraded":"2024-09-12T03:16:33.384083+0000","last_fullsized":"2024-09-12T03:16:33.384083+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:56:03.306616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:33.384773+0000","last_change":"2024-09-12T03:16:31.584375+0000","last_active":"2024-09-12T03:16:33.384773+0000","last_peered":"2024-09-12T03:16:33.384773+0000","last_clean":"2024-09-12T03:16:33.384773+0000","last_became_active":"2024-09-12T03:16:31.581766+0000","last_became_peered":"2024-09-12T03:16:31.581766+0000","last_unstale":"2024-09-12T03:16:33.384773+0000","last_undegraded":"2024-09-12T03:16:33.384773+0000","last_fullsized":"2024-09-12T03:16:33.384773+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:23:22.497643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:33.384404+0000","last_change":"2024-09-12T03:16:31.581788+0000","last_active":"2024-09-12T03:16:33.384404+0000","last_peered":"2024-09-12T03:16:33.384404+0000","last_clean":"2024-09-12T03:16:33.384404+0000","last_became_active":"2024-09-12T03:16:31.581516+0000","last_became_peered":"2024-09-12T03:16:31.581516+0000","last_unstale":"2024-09-12T03:16:33.384404+0000","last_undegraded":"2024-09-12T03:16:33.384404+0000","last_fullsized":"2024-09-12T03:16:33.384404+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:52:47.384994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:32.580659+0000","last_change":"2024-09-12T03:16:31.579564+0000","last_active":"2024-09-12T03:16:32.580659+0000","last_peered":"2024-09-12T03:16:32.580659+0000","last_clean":"2024-09-12T03:16:32.580659+0000","last_became_active":"2024-09-12T03:16:31.579247+0000","last_became_peered":"2024-09-12T03:16:31.579247+0000","last_unstale":"2024-09-12T03:16:32.580659+0000","last_undegraded":"2024-09-12T03:16:32.580659+0000","last_fullsized":"2024-09-12T03:16:32.580659+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:19:43.151205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:33.383395+0000","last_change":"2024-09-12T03:16:31.580809+0000","last_active":"2024-09-12T03:16:33.383395+0000","last_peered":"2024-09-12T03:16:33.383395+0000","last_clean":"2024-09-12T03:16:33.383395+0000","last_became_active":"2024-09-12T03:16:31.580520+0000","last_became_peered":"2024-09-12T03:16:31.580520+0000","last_unstale":"2024-09-12T03:16:33.383395+0000","last_undegraded":"2024-09-12T03:16:33.383395+0000","last_fullsized":"2024-09-12T03:16:33.383395+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:52:47.384994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:33.383780+0000","last_change":"2024-09-12T03:16:31.589461+0000","last_active":"2024-09-12T03:16:33.383780+0000","last_peered":"2024-09-12T03:16:33.383780+0000","last_clean":"2024-09-12T03:16:33.383780+0000","last_became_active":"2024-09-12T03:16:31.587058+0000","last_became_peered":"2024-09-12T03:16:31.587058+0000","last_unstale":"2024-09-12T03:16:33.383780+0000","last_undegraded":"2024-09-12T03:16:33.383780+0000","last_fullsized":"2024-09-12T03:16:33.383780+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:27:08.819051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:32.578055+0000","last_change":"2024-09-12T03:16:31.589724+0000","last_active":"2024-09-12T03:16:32.578055+0000","last_peered":"2024-09-12T03:16:32.578055+0000","last_clean":"2024-09-12T03:16:32.578055+0000","last_became_active":"2024-09-12T03:16:31.589452+0000","last_became_peered":"2024-09-12T03:16:31.589452+0000","last_unstale":"2024-09-12T03:16:32.578055+0000","last_undegraded":"2024-09-12T03:16:32.578055+0000","last_fullsized":"2024-09-12T03:16:32.578055+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:30:37.571167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:32.579421+0000","last_change":"2024-09-12T03:16:31.574474+0000","last_active":"2024-09-12T03:16:32.579421+0000","last_peered":"2024-09-12T03:16:32.579421+0000","last_clean":"2024-09-12T03:16:32.579421+0000","last_became_active":"2024-09-12T03:16:31.574190+0000","last_became_peered":"2024-09-12T03:16:31.574190+0000","last_unstale":"2024-09-12T03:16:32.579421+0000","last_undegraded":"2024-09-12T03:16:32.579421+0000","last_fullsized":"2024-09-12T03:16:32.579421+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:27:20.447453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:32.580359+0000","last_change":"2024-09-12T03:16:31.589878+0000","last_active":"2024-09-12T03:16:32.580359+0000","last_peered":"2024-09-12T03:16:32.580359+0000","last_clean":"2024-09-12T03:16:32.580359+0000","last_became_active":"2024-09-12T03:16:31.589603+0000","last_became_peered":"2024-09-12T03:16:31.589603+0000","last_unstale":"2024-09-12T03:16:32.580359+0000","last_undegraded":"2024-09-12T03:16:32.580359+0000","last_fullsized":"2024-09-12T03:16:32.580359+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:39:25.162423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:33.341837+0000","last_change":"2024-09-12T03:16:31.579526+0000","last_active":"2024-09-12T03:16:33.341837+0000","last_peered":"2024-09-12T03:16:33.341837+0000","last_clean":"2024-09-12T03:16:33.341837+0000","last_became_active":"2024-09-12T03:16:31.579217+0000","last_became_peered":"2024-09-12T03:16:31.579217+0000","last_unstale":"2024-09-12T03:16:33.341837+0000","last_undegraded":"2024-09-12T03:16:33.341837+0000","last_fullsized":"2024-09-12T03:16:33.341837+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:49:09.388955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:33.384367+0000","last_change":"2024-09-12T03:16:31.588216+0000","last_active":"2024-09-12T03:16:33.384367+0000","last_peered":"2024-09-12T03:16:33.384367+0000","last_clean":"2024-09-12T03:16:33.384367+0000","last_became_active":"2024-09-12T03:16:31.587499+0000","last_became_peered":"2024-09-12T03:16:31.587499+0000","last_unstale":"2024-09-12T03:16:33.384367+0000","last_undegraded":"2024-09-12T03:16:33.384367+0000","last_fullsized":"2024-09-12T03:16:33.384367+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:07:30.199995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:33.384562+0000","last_change":"2024-09-12T03:16:31.589828+0000","last_active":"2024-09-12T03:16:33.384562+0000","last_peered":"2024-09-12T03:16:33.384562+0000","last_clean":"2024-09-12T03:16:33.384562+0000","last_became_active":"2024-09-12T03:16:31.589341+0000","last_became_peered":"2024-09-12T03:16:31.589341+0000","last_unstale":"2024-09-12T03:16:33.384562+0000","last_undegraded":"2024-09-12T03:16:33.384562+0000","last_fullsized":"2024-09-12T03:16:33.384562+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:05:13.773162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:33.384136+0000","last_change":"2024-09-12T03:16:31.586744+0000","last_active":"2024-09-12T03:16:33.384136+0000","last_peered":"2024-09-12T03:16:33.384136+0000","last_clean":"2024-09-12T03:16:33.384136+0000","last_became_active":"2024-09-12T03:16:31.586462+0000","last_became_peered":"2024-09-12T03:16:31.586462+0000","last_unstale":"2024-09-12T03:16:33.384136+0000","last_undegraded":"2024-09-12T03:16:33.384136+0000","last_fullsized":"2024-09-12T03:16:33.384136+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:52:47.384994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:32.579721+0000","last_change":"2024-09-12T03:16:31.583270+0000","last_active":"2024-09-12T03:16:32.579721+0000","last_peered":"2024-09-12T03:16:32.579721+0000","last_clean":"2024-09-12T03:16:32.579721+0000","last_became_active":"2024-09-12T03:16:31.582948+0000","last_became_peered":"2024-09-12T03:16:31.582948+0000","last_unstale":"2024-09-12T03:16:32.579721+0000","last_undegraded":"2024-09-12T03:16:32.579721+0000","last_fullsized":"2024-09-12T03:16:32.579721+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:05:55.585173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:33.383081+0000","last_change":"2024-09-12T03:16:31.590127+0000","last_active":"2024-09-12T03:16:33.383081+0000","last_peered":"2024-09-12T03:16:33.383081+0000","last_clean":"2024-09-12T03:16:33.383081+0000","last_became_active":"2024-09-12T03:16:31.589847+0000","last_became_peered":"2024-09-12T03:16:31.589847+0000","last_unstale":"2024-09-12T03:16:33.383081+0000","last_undegraded":"2024-09-12T03:16:33.383081+0000","last_fullsized":"2024-09-12T03:16:33.383081+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:27:08.819051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:32.579620+0000","last_change":"2024-09-12T03:16:31.578711+0000","last_active":"2024-09-12T03:16:32.579620+0000","last_peered":"2024-09-12T03:16:32.579620+0000","last_clean":"2024-09-12T03:16:32.579620+0000","last_became_active":"2024-09-12T03:16:31.578421+0000","last_became_peered":"2024-09-12T03:16:31.578421+0000","last_unstale":"2024-09-12T03:16:32.579620+0000","last_undegraded":"2024-09-12T03:16:32.579620+0000","last_fullsized":"2024-09-12T03:16:32.579620+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:21:32.875776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:33.383880+0000","last_change":"2024-09-12T03:16:31.589412+0000","last_active":"2024-09-12T03:16:33.383880+0000","last_peered":"2024-09-12T03:16:33.383880+0000","last_clean":"2024-09-12T03:16:33.383880+0000","last_became_active":"2024-09-12T03:16:31.589147+0000","last_became_peered":"2024-09-12T03:16:31.589147+0000","last_unstale":"2024-09-12T03:16:33.383880+0000","last_undegraded":"2024-09-12T03:16:33.383880+0000","last_fullsized":"2024-09-12T03:16:33.383880+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:44:02.835393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:33.384825+0000","last_change":"2024-09-12T03:16:31.586179+0000","last_active":"2024-09-12T03:16:33.384825+0000","last_peered":"2024-09-12T03:16:33.384825+0000","last_clean":"2024-09-12T03:16:33.384825+0000","last_became_active":"2024-09-12T03:16:31.585793+0000","last_became_peered":"2024-09-12T03:16:31.585793+0000","last_unstale":"2024-09-12T03:16:33.384825+0000","last_undegraded":"2024-09-12T03:16:33.384825+0000","last_fullsized":"2024-09-12T03:16:33.384825+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:46:08.748138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:32.578142+0000","last_change":"2024-09-12T03:16:31.578743+0000","last_active":"2024-09-12T03:16:32.578142+0000","last_peered":"2024-09-12T03:16:32.578142+0000","last_clean":"2024-09-12T03:16:32.578142+0000","last_became_active":"2024-09-12T03:16:31.578450+0000","last_became_peered":"2024-09-12T03:16:31.578450+0000","last_unstale":"2024-09-12T03:16:32.578142+0000","last_undegraded":"2024-09-12T03:16:32.578142+0000","last_fullsized":"2024-09-12T03:16:32.578142+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:05:01.246614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:33.383793+0000","last_change":"2024-09-12T03:16:31.590186+0000","last_active":"2024-09-12T03:16:33.383793+0000","last_peered":"2024-09-12T03:16:33.383793+0000","last_clean":"2024-09-12T03:16:33.383793+0000","last_became_active":"2024-09-12T03:16:31.589895+0000","last_became_peered":"2024-09-12T03:16:31.589895+0000","last_unstale":"2024-09-12T03:16:33.383793+0000","last_undegraded":"2024-09-12T03:16:33.383793+0000","last_fullsized":"2024-09-12T03:16:33.383793+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:33:01.912717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:33.384305+0000","last_change":"2024-09-12T03:16:31.577171+0000","last_active":"2024-09-12T03:16:33.384305+0000","last_peered":"2024-09-12T03:16:33.384305+0000","last_clean":"2024-09-12T03:16:33.384305+0000","last_became_active":"2024-09-12T03:16:31.576882+0000","last_became_peered":"2024-09-12T03:16:31.576882+0000","last_unstale":"2024-09-12T03:16:33.384305+0000","last_undegraded":"2024-09-12T03:16:33.384305+0000","last_fullsized":"2024-09-12T03:16:33.384305+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:44:02.835393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:32.579941+0000","last_change":"2024-09-12T03:16:31.578670+0000","last_active":"2024-09-12T03:16:32.579941+0000","last_peered":"2024-09-12T03:16:32.579941+0000","last_clean":"2024-09-12T03:16:32.579941+0000","last_became_active":"2024-09-12T03:16:31.578379+0000","last_became_peered":"2024-09-12T03:16:31.578379+0000","last_unstale":"2024-09-12T03:16:32.579941+0000","last_undegraded":"2024-09-12T03:16:32.579941+0000","last_fullsized":"2024-09-12T03:16:32.579941+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:22:01.431497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:33.341761+0000","last_change":"2024-09-12T03:16:31.589511+0000","last_active":"2024-09-12T03:16:33.341761+0000","last_peered":"2024-09-12T03:16:33.341761+0000","last_clean":"2024-09-12T03:16:33.341761+0000","last_became_active":"2024-09-12T03:16:31.589230+0000","last_became_peered":"2024-09-12T03:16:31.589230+0000","last_unstale":"2024-09-12T03:16:33.341761+0000","last_undegraded":"2024-09-12T03:16:33.341761+0000","last_fullsized":"2024-09-12T03:16:33.341761+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:54:17.604800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:33.383613+0000","last_change":"2024-09-12T03:16:31.585042+0000","last_active":"2024-09-12T03:16:33.383613+0000","last_peered":"2024-09-12T03:16:33.383613+0000","last_clean":"2024-09-12T03:16:33.383613+0000","last_became_active":"2024-09-12T03:16:31.584495+0000","last_became_peered":"2024-09-12T03:16:31.584495+0000","last_unstale":"2024-09-12T03:16:33.383613+0000","last_undegraded":"2024-09-12T03:16:33.383613+0000","last_fullsized":"2024-09-12T03:16:33.383613+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:27:31.919800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:32.580182+0000","last_change":"2024-09-12T03:16:31.588040+0000","last_active":"2024-09-12T03:16:32.580182+0000","last_peered":"2024-09-12T03:16:32.580182+0000","last_clean":"2024-09-12T03:16:32.580182+0000","last_became_active":"2024-09-12T03:16:31.587757+0000","last_became_peered":"2024-09-12T03:16:31.587757+0000","last_unstale":"2024-09-12T03:16:32.580182+0000","last_undegraded":"2024-09-12T03:16:32.580182+0000","last_fullsized":"2024-09-12T03:16:32.580182+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:01:36.472991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:33.383460+0000","last_change":"2024-09-12T03:16:30.583423+0000","last_active":"2024-09-12T03:16:33.383460+0000","last_peered":"2024-09-12T03:16:33.383460+0000","last_clean":"2024-09-12T03:16:33.383460+0000","last_became_active":"2024-09-12T03:16:30.583160+0000","last_became_peered":"2024-09-12T03:16:30.583160+0000","last_unstale":"2024-09-12T03:16:33.383460+0000","last_undegraded":"2024-09-12T03:16:33.383460+0000","last_fullsized":"2024-09-12T03:16:33.383460+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:19:47.998143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:33.384360+0000","last_change":"2024-09-12T03:16:31.591565+0000","last_active":"2024-09-12T03:16:33.384360+0000","last_peered":"2024-09-12T03:16:33.384360+0000","last_clean":"2024-09-12T03:16:33.384360+0000","last_became_active":"2024-09-12T03:16:31.591236+0000","last_became_peered":"2024-09-12T03:16:31.591236+0000","last_unstale":"2024-09-12T03:16:33.384360+0000","last_undegraded":"2024-09-12T03:16:33.384360+0000","last_fullsized":"2024-09-12T03:16:33.384360+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:35:22.374781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:33.383956+0000","last_change":"2024-09-12T03:16:30.575856+0000","last_active":"2024-09-12T03:16:33.383956+0000","last_peered":"2024-09-12T03:16:33.383956+0000","last_clean":"2024-09-12T03:16:33.383956+0000","last_became_active":"2024-09-12T03:16:30.575512+0000","last_became_peered":"2024-09-12T03:16:30.575512+0000","last_unstale":"2024-09-12T03:16:33.383956+0000","last_undegraded":"2024-09-12T03:16:33.383956+0000","last_fullsized":"2024-09-12T03:16:33.383956+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:13:33.665229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:33.383136+0000","last_change":"2024-09-12T03:16:30.580131+0000","last_active":"2024-09-12T03:16:33.383136+0000","last_peered":"2024-09-12T03:16:33.383136+0000","last_clean":"2024-09-12T03:16:33.383136+0000","last_became_active":"2024-09-12T03:16:30.579760+0000","last_became_peered":"2024-09-12T03:16:30.579760+0000","last_unstale":"2024-09-12T03:16:33.383136+0000","last_undegraded":"2024-09-12T03:16:33.383136+0000","last_fullsized":"2024-09-12T03:16:33.383136+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:29:14.889397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:33.383040+0000","last_change":"2024-09-12T03:16:31.581185+0000","last_active":"2024-09-12T03:16:33.383040+0000","last_peered":"2024-09-12T03:16:33.383040+0000","last_clean":"2024-09-12T03:16:33.383040+0000","last_became_active":"2024-09-12T03:16:31.580425+0000","last_became_peered":"2024-09-12T03:16:31.580425+0000","last_unstale":"2024-09-12T03:16:33.383040+0000","last_undegraded":"2024-09-12T03:16:33.383040+0000","last_fullsized":"2024-09-12T03:16:33.383040+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:12:47.905772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:32.579685+0000","last_change":"2024-09-12T03:16:30.571998+0000","last_active":"2024-09-12T03:16:32.579685+0000","last_peered":"2024-09-12T03:16:32.579685+0000","last_clean":"2024-09-12T03:16:32.579685+0000","last_became_active":"2024-09-12T03:16:30.571673+0000","last_became_peered":"2024-09-12T03:16:30.571673+0000","last_unstale":"2024-09-12T03:16:32.579685+0000","last_undegraded":"2024-09-12T03:16:32.579685+0000","last_fullsized":"2024-09-12T03:16:32.579685+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:19:17.746405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:32.578040+0000","last_change":"2024-09-12T03:16:31.587700+0000","last_active":"2024-09-12T03:16:32.578040+0000","last_peered":"2024-09-12T03:16:32.578040+0000","last_clean":"2024-09-12T03:16:32.578040+0000","last_became_active":"2024-09-12T03:16:31.587296+0000","last_became_peered":"2024-09-12T03:16:31.587296+0000","last_unstale":"2024-09-12T03:16:32.578040+0000","last_undegraded":"2024-09-12T03:16:32.578040+0000","last_fullsized":"2024-09-12T03:16:32.578040+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:28:08.918294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:33.384131+0000","last_change":"2024-09-12T03:16:30.581517+0000","last_active":"2024-09-12T03:16:33.384131+0000","last_peered":"2024-09-12T03:16:33.384131+0000","last_clean":"2024-09-12T03:16:33.384131+0000","last_became_active":"2024-09-12T03:16:30.581107+0000","last_became_peered":"2024-09-12T03:16:30.581107+0000","last_unstale":"2024-09-12T03:16:33.384131+0000","last_undegraded":"2024-09-12T03:16:33.384131+0000","last_fullsized":"2024-09-12T03:16:33.384131+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:01:35.467902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:32.580094+0000","last_change":"2024-09-12T03:16:31.588465+0000","last_active":"2024-09-12T03:16:32.580094+0000","last_peered":"2024-09-12T03:16:32.580094+0000","last_clean":"2024-09-12T03:16:32.580094+0000","last_became_active":"2024-09-12T03:16:31.588150+0000","last_became_peered":"2024-09-12T03:16:31.588150+0000","last_unstale":"2024-09-12T03:16:32.580094+0000","last_undegraded":"2024-09-12T03:16:32.580094+0000","last_fullsized":"2024-09-12T03:16:32.580094+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:04:42.975000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:32.580148+0000","last_change":"2024-09-12T03:16:30.578195+0000","last_active":"2024-09-12T03:16:32.580148+0000","last_peered":"2024-09-12T03:16:32.580148+0000","last_clean":"2024-09-12T03:16:32.580148+0000","last_became_active":"2024-09-12T03:16:30.577878+0000","last_became_peered":"2024-09-12T03:16:30.577878+0000","last_unstale":"2024-09-12T03:16:32.580148+0000","last_undegraded":"2024-09-12T03:16:32.580148+0000","last_fullsized":"2024-09-12T03:16:32.580148+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:19:07.178848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:33.384177+0000","last_change":"2024-09-12T03:16:31.584084+0000","last_active":"2024-09-12T03:16:33.384177+0000","last_peered":"2024-09-12T03:16:33.384177+0000","last_clean":"2024-09-12T03:16:33.384177+0000","last_became_active":"2024-09-12T03:16:31.583539+0000","last_became_peered":"2024-09-12T03:16:31.583539+0000","last_unstale":"2024-09-12T03:16:33.384177+0000","last_undegraded":"2024-09-12T03:16:33.384177+0000","last_fullsized":"2024-09-12T03:16:33.384177+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:23:22.497643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:32.578203+0000","last_change":"2024-09-12T03:16:30.572959+0000","last_active":"2024-09-12T03:16:32.578203+0000","last_peered":"2024-09-12T03:16:32.578203+0000","last_clean":"2024-09-12T03:16:32.578203+0000","last_became_active":"2024-09-12T03:16:30.572692+0000","last_became_peered":"2024-09-12T03:16:30.572692+0000","last_unstale":"2024-09-12T03:16:32.578203+0000","last_undegraded":"2024-09-12T03:16:32.578203+0000","last_fullsized":"2024-09-12T03:16:32.578203+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:21:08.051836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:32.580840+0000","last_change":"2024-09-12T03:16:31.590092+0000","last_active":"2024-09-12T03:16:32.580840+0000","last_peered":"2024-09-12T03:16:32.580840+0000","last_clean":"2024-09-12T03:16:32.580840+0000","last_became_active":"2024-09-12T03:16:31.589831+0000","last_became_peered":"2024-09-12T03:16:31.589831+0000","last_unstale":"2024-09-12T03:16:32.580840+0000","last_undegraded":"2024-09-12T03:16:32.580840+0000","last_fullsized":"2024-09-12T03:16:32.580840+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:54:17.604800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"20'6","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:33.384460+0000","last_change":"2024-09-12T03:16:30.575948+0000","last_active":"2024-09-12T03:16:33.384460+0000","last_peered":"2024-09-12T03:16:33.384460+0000","last_clean":"2024-09-12T03:16:33.384460+0000","last_became_active":"2024-09-12T03:16:30.575651+0000","last_became_peered":"2024-09-12T03:16:30.575651+0000","last_unstale":"2024-09-12T03:16:33.384460+0000","last_undegraded":"2024-09-12T03:16:33.384460+0000","last_fullsized":"2024-09-12T03:16:33.384460+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:44:10.172361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:33.383628+0000","last_change":"2024-09-12T03:16:31.587604+0000","last_active":"2024-09-12T03:16:33.383628+0000","last_peered":"2024-09-12T03:16:33.383628+0000","last_clean":"2024-09-12T03:16:33.383628+0000","last_became_active":"2024-09-12T03:16:31.581590+0000","last_became_peered":"2024-09-12T03:16:31.581590+0000","last_unstale":"2024-09-12T03:16:33.383628+0000","last_undegraded":"2024-09-12T03:16:33.383628+0000","last_fullsized":"2024-09-12T03:16:33.383628+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:40:23.693285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:33.384342+0000","last_change":"2024-09-12T03:16:30.580151+0000","last_active":"2024-09-12T03:16:33.384342+0000","last_peered":"2024-09-12T03:16:33.384342+0000","last_clean":"2024-09-12T03:16:33.384342+0000","last_became_active":"2024-09-12T03:16:30.579859+0000","last_became_peered":"2024-09-12T03:16:30.579859+0000","last_unstale":"2024-09-12T03:16:33.384342+0000","last_undegraded":"2024-09-12T03:16:33.384342+0000","last_fullsized":"2024-09-12T03:16:33.384342+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:44:10.172361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:33.383933+0000","last_change":"2024-09-12T03:16:31.580748+0000","last_active":"2024-09-12T03:16:33.383933+0000","last_peered":"2024-09-12T03:16:33.383933+0000","last_clean":"2024-09-12T03:16:33.383933+0000","last_became_active":"2024-09-12T03:16:31.580434+0000","last_became_peered":"2024-09-12T03:16:31.580434+0000","last_unstale":"2024-09-12T03:16:33.383933+0000","last_undegraded":"2024-09-12T03:16:33.383933+0000","last_fullsized":"2024-09-12T03:16:33.383933+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:27:31.919800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:32.580017+0000","last_change":"2024-09-12T03:16:30.570604+0000","last_active":"2024-09-12T03:16:32.580017+0000","last_peered":"2024-09-12T03:16:32.580017+0000","last_clean":"2024-09-12T03:16:32.580017+0000","last_became_active":"2024-09-12T03:16:30.570227+0000","last_became_peered":"2024-09-12T03:16:30.570227+0000","last_unstale":"2024-09-12T03:16:32.580017+0000","last_undegraded":"2024-09-12T03:16:32.580017+0000","last_fullsized":"2024-09-12T03:16:32.580017+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:55:03.313892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:33.383958+0000","last_change":"2024-09-12T03:16:31.592248+0000","last_active":"2024-09-12T03:16:33.383958+0000","last_peered":"2024-09-12T03:16:33.383958+0000","last_clean":"2024-09-12T03:16:33.383958+0000","last_became_active":"2024-09-12T03:16:31.591836+0000","last_became_peered":"2024-09-12T03:16:31.591836+0000","last_unstale":"2024-09-12T03:16:33.383958+0000","last_undegraded":"2024-09-12T03:16:33.383958+0000","last_fullsized":"2024-09-12T03:16:33.383958+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:05:17.785019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:33.383324+0000","last_change":"2024-09-12T03:16:30.578648+0000","last_active":"2024-09-12T03:16:33.383324+0000","last_peered":"2024-09-12T03:16:33.383324+0000","last_clean":"2024-09-12T03:16:33.383324+0000","last_became_active":"2024-09-12T03:16:30.578343+0000","last_became_peered":"2024-09-12T03:16:30.578343+0000","last_unstale":"2024-09-12T03:16:33.383324+0000","last_undegraded":"2024-09-12T03:16:33.383324+0000","last_fullsized":"2024-09-12T03:16:33.383324+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:29:14.889397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:33.383440+0000","last_change":"2024-09-12T03:16:31.584802+0000","last_active":"2024-09-12T03:16:33.383440+0000","last_peered":"2024-09-12T03:16:33.383440+0000","last_clean":"2024-09-12T03:16:33.383440+0000","last_became_active":"2024-09-12T03:16:31.584124+0000","last_became_peered":"2024-09-12T03:16:31.584124+0000","last_unstale":"2024-09-12T03:16:33.383440+0000","last_undegraded":"2024-09-12T03:16:33.383440+0000","last_fullsized":"2024-09-12T03:16:33.383440+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:27:08.819051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":22,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:33.341548+0000","last_change":"2024-09-12T03:16:30.574668+0000","last_active":"2024-09-12T03:16:33.341548+0000","last_peered":"2024-09-12T03:16:33.341548+0000","last_clean":"2024-09-12T03:16:33.341548+0000","last_became_active":"2024-09-12T03:16:30.574373+0000","last_became_peered":"2024-09-12T03:16:30.574373+0000","last_unstale":"2024-09-12T03:16:33.341548+0000","last_undegraded":"2024-09-12T03:16:33.341548+0000","last_fullsized":"2024-09-12T03:16:33.341548+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:19:47.998143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:33.383657+0000","last_change":"2024-09-12T03:16:31.592520+0000","last_active":"2024-09-12T03:16:33.383657+0000","last_peered":"2024-09-12T03:16:33.383657+0000","last_clean":"2024-09-12T03:16:33.383657+0000","last_became_active":"2024-09-12T03:16:31.591766+0000","last_became_peered":"2024-09-12T03:16:31.591766+0000","last_unstale":"2024-09-12T03:16:33.383657+0000","last_undegraded":"2024-09-12T03:16:33.383657+0000","last_fullsized":"2024-09-12T03:16:33.383657+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:22:52.960561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:32.579754+0000","last_change":"2024-09-12T03:16:30.582087+0000","last_active":"2024-09-12T03:16:32.579754+0000","last_peered":"2024-09-12T03:16:32.579754+0000","last_clean":"2024-09-12T03:16:32.579754+0000","last_became_active":"2024-09-12T03:16:30.581622+0000","last_became_peered":"2024-09-12T03:16:30.581622+0000","last_unstale":"2024-09-12T03:16:32.579754+0000","last_undegraded":"2024-09-12T03:16:32.579754+0000","last_fullsized":"2024-09-12T03:16:32.579754+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:58:54.372496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:33.384324+0000","last_change":"2024-09-12T03:16:31.591374+0000","last_active":"2024-09-12T03:16:33.384324+0000","last_peered":"2024-09-12T03:16:33.384324+0000","last_clean":"2024-09-12T03:16:33.384324+0000","last_became_active":"2024-09-12T03:16:31.591092+0000","last_became_peered":"2024-09-12T03:16:31.591092+0000","last_unstale":"2024-09-12T03:16:33.384324+0000","last_undegraded":"2024-09-12T03:16:33.384324+0000","last_fullsized":"2024-09-12T03:16:33.384324+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:08:36.235105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:33.341748+0000","last_change":"2024-09-12T03:16:30.570924+0000","last_active":"2024-09-12T03:16:33.341748+0000","last_peered":"2024-09-12T03:16:33.341748+0000","last_clean":"2024-09-12T03:16:33.341748+0000","last_became_active":"2024-09-12T03:16:30.570636+0000","last_became_peered":"2024-09-12T03:16:30.570636+0000","last_unstale":"2024-09-12T03:16:33.341748+0000","last_undegraded":"2024-09-12T03:16:33.341748+0000","last_fullsized":"2024-09-12T03:16:33.341748+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:29:14.889397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:33.384281+0000","last_change":"2024-09-12T03:16:31.592777+0000","last_active":"2024-09-12T03:16:33.384281+0000","last_peered":"2024-09-12T03:16:33.384281+0000","last_clean":"2024-09-12T03:16:33.384281+0000","last_became_active":"2024-09-12T03:16:31.591855+0000","last_became_peered":"2024-09-12T03:16:31.591855+0000","last_unstale":"2024-09-12T03:16:33.384281+0000","last_undegraded":"2024-09-12T03:16:33.384281+0000","last_fullsized":"2024-09-12T03:16:33.384281+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:39:49.575672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:32.578281+0000","last_change":"2024-09-12T03:16:30.574066+0000","last_active":"2024-09-12T03:16:32.578281+0000","last_peered":"2024-09-12T03:16:32.578281+0000","last_clean":"2024-09-12T03:16:32.578281+0000","last_became_active":"2024-09-12T03:16:30.573756+0000","last_became_peered":"2024-09-12T03:16:30.573756+0000","last_unstale":"2024-09-12T03:16:32.578281+0000","last_undegraded":"2024-09-12T03:16:32.578281+0000","last_fullsized":"2024-09-12T03:16:32.578281+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:19:17.746405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:32.580747+0000","last_change":"2024-09-12T03:16:31.583553+0000","last_active":"2024-09-12T03:16:32.580747+0000","last_peered":"2024-09-12T03:16:32.580747+0000","last_clean":"2024-09-12T03:16:32.580747+0000","last_became_active":"2024-09-12T03:16:31.582827+0000","last_became_peered":"2024-09-12T03:16:31.582827+0000","last_unstale":"2024-09-12T03:16:32.580747+0000","last_undegraded":"2024-09-12T03:16:32.580747+0000","last_fullsized":"2024-09-12T03:16:32.580747+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:54:20.264625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:33.341655+0000","last_change":"2024-09-12T03:16:30.573483+0000","last_active":"2024-09-12T03:16:33.341655+0000","last_peered":"2024-09-12T03:16:33.341655+0000","last_clean":"2024-09-12T03:16:33.341655+0000","last_became_active":"2024-09-12T03:16:30.573180+0000","last_became_peered":"2024-09-12T03:16:30.573180+0000","last_unstale":"2024-09-12T03:16:33.341655+0000","last_undegraded":"2024-09-12T03:16:33.341655+0000","last_fullsized":"2024-09-12T03:16:33.341655+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:13:33.665229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:33.383685+0000","last_change":"2024-09-12T03:16:31.590008+0000","last_active":"2024-09-12T03:16:33.383685+0000","last_peered":"2024-09-12T03:16:33.383685+0000","last_clean":"2024-09-12T03:16:33.383685+0000","last_became_active":"2024-09-12T03:16:31.589738+0000","last_became_peered":"2024-09-12T03:16:31.589738+0000","last_unstale":"2024-09-12T03:16:33.383685+0000","last_undegraded":"2024-09-12T03:16:33.383685+0000","last_fullsized":"2024-09-12T03:16:33.383685+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:31:17.482655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":170,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:33.384272+0000","last_change":"2024-09-12T03:16:21.883446+0000","last_active":"2024-09-12T03:16:33.384272+0000","last_peered":"2024-09-12T03:16:33.384272+0000","last_clean":"2024-09-12T03:16:33.384272+0000","last_became_active":"2024-09-12T03:16:21.882207+0000","last_became_peered":"2024-09-12T03:16:21.882207+0000","last_unstale":"2024-09-12T03:16:33.384272+0000","last_undegraded":"2024-09-12T03:16:33.384272+0000","last_fullsized":"2024-09-12T03:16:33.384272+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:20.316426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:20.316426+0000","last_clean_scrub_stamp":"2024-09-12T03:16:20.316426+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:12:15.424320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:32.579477+0000","last_change":"2024-09-12T03:16:30.573340+0000","last_active":"2024-09-12T03:16:32.579477+0000","last_peered":"2024-09-12T03:16:32.579477+0000","last_clean":"2024-09-12T03:16:32.579477+0000","last_became_active":"2024-09-12T03:16:30.572756+0000","last_became_peered":"2024-09-12T03:16:30.572756+0000","last_unstale":"2024-09-12T03:16:32.579477+0000","last_undegraded":"2024-09-12T03:16:32.579477+0000","last_fullsized":"2024-09-12T03:16:32.579477+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:21:08.051836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:33.383593+0000","last_change":"2024-09-12T03:16:31.592850+0000","last_active":"2024-09-12T03:16:33.383593+0000","last_peered":"2024-09-12T03:16:33.383593+0000","last_clean":"2024-09-12T03:16:33.383593+0000","last_became_active":"2024-09-12T03:16:31.592505+0000","last_became_peered":"2024-09-12T03:16:31.592505+0000","last_unstale":"2024-09-12T03:16:33.383593+0000","last_undegraded":"2024-09-12T03:16:33.383593+0000","last_fullsized":"2024-09-12T03:16:33.383593+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:23:22.497643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:33.384154+0000","last_change":"2024-09-12T03:16:30.584842+0000","last_active":"2024-09-12T03:16:33.384154+0000","last_peered":"2024-09-12T03:16:33.384154+0000","last_clean":"2024-09-12T03:16:33.384154+0000","last_became_active":"2024-09-12T03:16:30.584516+0000","last_became_peered":"2024-09-12T03:16:30.584516+0000","last_unstale":"2024-09-12T03:16:33.384154+0000","last_undegraded":"2024-09-12T03:16:33.384154+0000","last_fullsized":"2024-09-12T03:16:33.384154+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:44:15.389117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:33.383582+0000","last_change":"2024-09-12T03:16:31.570586+0000","last_active":"2024-09-12T03:16:33.383582+0000","last_peered":"2024-09-12T03:16:33.383582+0000","last_clean":"2024-09-12T03:16:33.383582+0000","last_became_active":"2024-09-12T03:16:31.570210+0000","last_became_peered":"2024-09-12T03:16:31.570210+0000","last_unstale":"2024-09-12T03:16:33.383582+0000","last_undegraded":"2024-09-12T03:16:33.383582+0000","last_fullsized":"2024-09-12T03:16:33.383582+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:12:47.905772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:33.341721+0000","last_change":"2024-09-12T03:16:30.569374+0000","last_active":"2024-09-12T03:16:33.341721+0000","last_peered":"2024-09-12T03:16:33.341721+0000","last_clean":"2024-09-12T03:16:33.341721+0000","last_became_active":"2024-09-12T03:16:30.569053+0000","last_became_peered":"2024-09-12T03:16:30.569053+0000","last_unstale":"2024-09-12T03:16:33.341721+0000","last_undegraded":"2024-09-12T03:16:33.341721+0000","last_fullsized":"2024-09-12T03:16:33.341721+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:44:10.172361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:32.578285+0000","last_change":"2024-09-12T03:16:31.587841+0000","last_active":"2024-09-12T03:16:32.578285+0000","last_peered":"2024-09-12T03:16:32.578285+0000","last_clean":"2024-09-12T03:16:32.578285+0000","last_became_active":"2024-09-12T03:16:31.587555+0000","last_became_peered":"2024-09-12T03:16:31.587555+0000","last_unstale":"2024-09-12T03:16:32.578285+0000","last_undegraded":"2024-09-12T03:16:32.578285+0000","last_fullsized":"2024-09-12T03:16:32.578285+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:02:33.249591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:32.580124+0000","last_change":"2024-09-12T03:16:30.576697+0000","last_active":"2024-09-12T03:16:32.580124+0000","last_peered":"2024-09-12T03:16:32.580124+0000","last_clean":"2024-09-12T03:16:32.580124+0000","last_became_active":"2024-09-12T03:16:30.574772+0000","last_became_peered":"2024-09-12T03:16:30.574772+0000","last_unstale":"2024-09-12T03:16:32.580124+0000","last_undegraded":"2024-09-12T03:16:32.580124+0000","last_fullsized":"2024-09-12T03:16:32.580124+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:36:06.512797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:32.580280+0000","last_change":"2024-09-12T03:16:31.581253+0000","last_active":"2024-09-12T03:16:32.580280+0000","last_peered":"2024-09-12T03:16:32.580280+0000","last_clean":"2024-09-12T03:16:32.580280+0000","last_became_active":"2024-09-12T03:16:31.580992+0000","last_became_peered":"2024-09-12T03:16:31.580992+0000","last_unstale":"2024-09-12T03:16:32.580280+0000","last_undegraded":"2024-09-12T03:16:32.580280+0000","last_fullsized":"2024-09-12T03:16:32.580280+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:23:14.503630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:32.578229+0000","last_change":"2024-09-12T03:16:30.580488+0000","last_active":"2024-09-12T03:16:32.578229+0000","last_peered":"2024-09-12T03:16:32.578229+0000","last_clean":"2024-09-12T03:16:32.578229+0000","last_became_active":"2024-09-12T03:16:30.580231+0000","last_became_peered":"2024-09-12T03:16:30.580231+0000","last_unstale":"2024-09-12T03:16:32.578229+0000","last_undegraded":"2024-09-12T03:16:32.578229+0000","last_fullsized":"2024-09-12T03:16:32.578229+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:55:03.313892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:32.580115+0000","last_change":"2024-09-12T03:16:31.582097+0000","last_active":"2024-09-12T03:16:32.580115+0000","last_peered":"2024-09-12T03:16:32.580115+0000","last_clean":"2024-09-12T03:16:32.580115+0000","last_became_active":"2024-09-12T03:16:31.580258+0000","last_became_peered":"2024-09-12T03:16:31.580258+0000","last_unstale":"2024-09-12T03:16:32.580115+0000","last_undegraded":"2024-09-12T03:16:32.580115+0000","last_fullsized":"2024-09-12T03:16:32.580115+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:13:00.761714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:33.384050+0000","last_change":"2024-09-12T03:16:30.586287+0000","last_active":"2024-09-12T03:16:33.384050+0000","last_peered":"2024-09-12T03:16:33.384050+0000","last_clean":"2024-09-12T03:16:33.384050+0000","last_became_active":"2024-09-12T03:16:30.585919+0000","last_became_peered":"2024-09-12T03:16:30.585919+0000","last_unstale":"2024-09-12T03:16:33.384050+0000","last_undegraded":"2024-09-12T03:16:33.384050+0000","last_fullsized":"2024-09-12T03:16:33.384050+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:04:51.773994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:32.578331+0000","last_change":"2024-09-12T03:16:31.587128+0000","last_active":"2024-09-12T03:16:32.578331+0000","last_peered":"2024-09-12T03:16:32.578331+0000","last_clean":"2024-09-12T03:16:32.578331+0000","last_became_active":"2024-09-12T03:16:31.586831+0000","last_became_peered":"2024-09-12T03:16:31.586831+0000","last_unstale":"2024-09-12T03:16:32.578331+0000","last_undegraded":"2024-09-12T03:16:32.578331+0000","last_fullsized":"2024-09-12T03:16:32.578331+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:59:21.255153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:32.578379+0000","last_change":"2024-09-12T03:16:30.577847+0000","last_active":"2024-09-12T03:16:32.578379+0000","last_peered":"2024-09-12T03:16:32.578379+0000","last_clean":"2024-09-12T03:16:32.578379+0000","last_became_active":"2024-09-12T03:16:30.577552+0000","last_became_peered":"2024-09-12T03:16:30.577552+0000","last_unstale":"2024-09-12T03:16:32.578379+0000","last_undegraded":"2024-09-12T03:16:32.578379+0000","last_fullsized":"2024-09-12T03:16:32.578379+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:36:06.512797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:33.383991+0000","last_change":"2024-09-12T03:16:31.586339+0000","last_active":"2024-09-12T03:16:33.383991+0000","last_peered":"2024-09-12T03:16:33.383991+0000","last_clean":"2024-09-12T03:16:33.383991+0000","last_became_active":"2024-09-12T03:16:31.586009+0000","last_became_peered":"2024-09-12T03:16:31.586009+0000","last_unstale":"2024-09-12T03:16:33.383991+0000","last_undegraded":"2024-09-12T03:16:33.383991+0000","last_fullsized":"2024-09-12T03:16:33.383991+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:35:22.374781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:33.384683+0000","last_change":"2024-09-12T03:16:30.583723+0000","last_active":"2024-09-12T03:16:33.384683+0000","last_peered":"2024-09-12T03:16:33.384683+0000","last_clean":"2024-09-12T03:16:33.384683+0000","last_became_active":"2024-09-12T03:16:30.582911+0000","last_became_peered":"2024-09-12T03:16:30.582911+0000","last_unstale":"2024-09-12T03:16:33.384683+0000","last_undegraded":"2024-09-12T03:16:33.384683+0000","last_fullsized":"2024-09-12T03:16:33.384683+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:23:13.498541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:33.384195+0000","last_change":"2024-09-12T03:16:31.589322+0000","last_active":"2024-09-12T03:16:33.384195+0000","last_peered":"2024-09-12T03:16:33.384195+0000","last_clean":"2024-09-12T03:16:33.384195+0000","last_became_active":"2024-09-12T03:16:31.588464+0000","last_became_peered":"2024-09-12T03:16:31.588464+0000","last_unstale":"2024-09-12T03:16:33.384195+0000","last_undegraded":"2024-09-12T03:16:33.384195+0000","last_fullsized":"2024-09-12T03:16:33.384195+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:46:08.748138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:33.383860+0000","last_change":"2024-09-12T03:16:30.583438+0000","last_active":"2024-09-12T03:16:33.383860+0000","last_peered":"2024-09-12T03:16:33.383860+0000","last_clean":"2024-09-12T03:16:33.383860+0000","last_became_active":"2024-09-12T03:16:30.583067+0000","last_became_peered":"2024-09-12T03:16:30.583067+0000","last_unstale":"2024-09-12T03:16:33.383860+0000","last_undegraded":"2024-09-12T03:16:33.383860+0000","last_fullsized":"2024-09-12T03:16:33.383860+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:54:16.599711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:32.581034+0000","last_change":"2024-09-12T03:16:31.578365+0000","last_active":"2024-09-12T03:16:32.581034+0000","last_peered":"2024-09-12T03:16:32.581034+0000","last_clean":"2024-09-12T03:16:32.581034+0000","last_became_active":"2024-09-12T03:16:31.577424+0000","last_became_peered":"2024-09-12T03:16:31.577424+0000","last_unstale":"2024-09-12T03:16:32.581034+0000","last_undegraded":"2024-09-12T03:16:32.581034+0000","last_fullsized":"2024-09-12T03:16:32.581034+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:49:09.388955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:32.578103+0000","last_change":"2024-09-12T03:16:30.574028+0000","last_active":"2024-09-12T03:16:32.578103+0000","last_peered":"2024-09-12T03:16:32.578103+0000","last_clean":"2024-09-12T03:16:32.578103+0000","last_became_active":"2024-09-12T03:16:30.573742+0000","last_became_peered":"2024-09-12T03:16:30.573742+0000","last_unstale":"2024-09-12T03:16:32.578103+0000","last_undegraded":"2024-09-12T03:16:32.578103+0000","last_fullsized":"2024-09-12T03:16:32.578103+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:19:07.178848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:33.383373+0000","last_change":"2024-09-12T03:16:31.584222+0000","last_active":"2024-09-12T03:16:33.383373+0000","last_peered":"2024-09-12T03:16:33.383373+0000","last_clean":"2024-09-12T03:16:33.383373+0000","last_became_active":"2024-09-12T03:16:31.583947+0000","last_became_peered":"2024-09-12T03:16:31.583947+0000","last_unstale":"2024-09-12T03:16:33.383373+0000","last_undegraded":"2024-09-12T03:16:33.383373+0000","last_fullsized":"2024-09-12T03:16:33.383373+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:27:31.919800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:32.578127+0000","last_change":"2024-09-12T03:16:30.572123+0000","last_active":"2024-09-12T03:16:32.578127+0000","last_peered":"2024-09-12T03:16:32.578127+0000","last_clean":"2024-09-12T03:16:32.578127+0000","last_became_active":"2024-09-12T03:16:30.571600+0000","last_became_peered":"2024-09-12T03:16:30.571600+0000","last_unstale":"2024-09-12T03:16:32.578127+0000","last_undegraded":"2024-09-12T03:16:32.578127+0000","last_fullsized":"2024-09-12T03:16:32.578127+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:25:21.686984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:33.383482+0000","last_change":"2024-09-12T03:16:31.581216+0000","last_active":"2024-09-12T03:16:33.383482+0000","last_peered":"2024-09-12T03:16:33.383482+0000","last_clean":"2024-09-12T03:16:33.383482+0000","last_became_active":"2024-09-12T03:16:31.580663+0000","last_became_peered":"2024-09-12T03:16:31.580663+0000","last_unstale":"2024-09-12T03:16:33.383482+0000","last_undegraded":"2024-09-12T03:16:33.383482+0000","last_fullsized":"2024-09-12T03:16:33.383482+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:05:13.773162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:33.384021+0000","last_change":"2024-09-12T03:16:30.584019+0000","last_active":"2024-09-12T03:16:33.384021+0000","last_peered":"2024-09-12T03:16:33.384021+0000","last_clean":"2024-09-12T03:16:33.384021+0000","last_became_active":"2024-09-12T03:16:30.583213+0000","last_became_peered":"2024-09-12T03:16:30.583213+0000","last_unstale":"2024-09-12T03:16:33.384021+0000","last_undegraded":"2024-09-12T03:16:33.384021+0000","last_fullsized":"2024-09-12T03:16:33.384021+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:14:33.816306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:33.383420+0000","last_change":"2024-09-12T03:16:31.590094+0000","last_active":"2024-09-12T03:16:33.383420+0000","last_peered":"2024-09-12T03:16:33.383420+0000","last_clean":"2024-09-12T03:16:33.383420+0000","last_became_active":"2024-09-12T03:16:31.589818+0000","last_became_peered":"2024-09-12T03:16:31.589818+0000","last_unstale":"2024-09-12T03:16:33.383420+0000","last_undegraded":"2024-09-12T03:16:33.383420+0000","last_fullsized":"2024-09-12T03:16:33.383420+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:12:47.905772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:32.580177+0000","last_change":"2024-09-12T03:16:31.585182+0000","last_active":"2024-09-12T03:16:32.580177+0000","last_peered":"2024-09-12T03:16:32.580177+0000","last_clean":"2024-09-12T03:16:32.580177+0000","last_became_active":"2024-09-12T03:16:31.584871+0000","last_became_peered":"2024-09-12T03:16:31.584871+0000","last_unstale":"2024-09-12T03:16:32.580177+0000","last_undegraded":"2024-09-12T03:16:32.580177+0000","last_fullsized":"2024-09-12T03:16:32.580177+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:48:52.581642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:33.383721+0000","last_change":"2024-09-12T03:16:30.583526+0000","last_active":"2024-09-12T03:16:33.383721+0000","last_peered":"2024-09-12T03:16:33.383721+0000","last_clean":"2024-09-12T03:16:33.383721+0000","last_became_active":"2024-09-12T03:16:30.583166+0000","last_became_peered":"2024-09-12T03:16:30.583166+0000","last_unstale":"2024-09-12T03:16:33.383721+0000","last_undegraded":"2024-09-12T03:16:33.383721+0000","last_fullsized":"2024-09-12T03:16:33.383721+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:19:42.146116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:32.580666+0000","last_change":"2024-09-12T03:16:31.585324+0000","last_active":"2024-09-12T03:16:32.580666+0000","last_peered":"2024-09-12T03:16:32.580666+0000","last_clean":"2024-09-12T03:16:32.580666+0000","last_became_active":"2024-09-12T03:16:31.584963+0000","last_became_peered":"2024-09-12T03:16:31.584963+0000","last_unstale":"2024-09-12T03:16:32.580666+0000","last_undegraded":"2024-09-12T03:16:32.580666+0000","last_fullsized":"2024-09-12T03:16:32.580666+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:34:35.319063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:33.383711+0000","last_change":"2024-09-12T03:16:30.583739+0000","last_active":"2024-09-12T03:16:33.383711+0000","last_peered":"2024-09-12T03:16:33.383711+0000","last_clean":"2024-09-12T03:16:33.383711+0000","last_became_active":"2024-09-12T03:16:30.583397+0000","last_became_peered":"2024-09-12T03:16:30.583397+0000","last_unstale":"2024-09-12T03:16:33.383711+0000","last_undegraded":"2024-09-12T03:16:33.383711+0000","last_fullsized":"2024-09-12T03:16:33.383711+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:49:08.383866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:33.384172+0000","last_change":"2024-09-12T03:16:31.583861+0000","last_active":"2024-09-12T03:16:33.384172+0000","last_peered":"2024-09-12T03:16:33.384172+0000","last_clean":"2024-09-12T03:16:33.384172+0000","last_became_active":"2024-09-12T03:16:31.583134+0000","last_became_peered":"2024-09-12T03:16:31.583134+0000","last_unstale":"2024-09-12T03:16:33.384172+0000","last_undegraded":"2024-09-12T03:16:33.384172+0000","last_fullsized":"2024-09-12T03:16:33.384172+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:05:13.773162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:33.383731+0000","last_change":"2024-09-12T03:16:30.580413+0000","last_active":"2024-09-12T03:16:33.383731+0000","last_peered":"2024-09-12T03:16:33.383731+0000","last_clean":"2024-09-12T03:16:33.383731+0000","last_became_active":"2024-09-12T03:16:30.580133+0000","last_became_peered":"2024-09-12T03:16:30.580133+0000","last_unstale":"2024-09-12T03:16:33.383731+0000","last_undegraded":"2024-09-12T03:16:33.383731+0000","last_fullsized":"2024-09-12T03:16:33.383731+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:12:12.227072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:33.383784+0000","last_change":"2024-09-12T03:16:31.580830+0000","last_active":"2024-09-12T03:16:33.383784+0000","last_peered":"2024-09-12T03:16:33.383784+0000","last_clean":"2024-09-12T03:16:33.383784+0000","last_became_active":"2024-09-12T03:16:31.580548+0000","last_became_peered":"2024-09-12T03:16:31.580548+0000","last_unstale":"2024-09-12T03:16:33.383784+0000","last_undegraded":"2024-09-12T03:16:33.383784+0000","last_fullsized":"2024-09-12T03:16:33.383784+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:46:08.748138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"20'4","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:33.384309+0000","last_change":"2024-09-12T03:16:30.584481+0000","last_active":"2024-09-12T03:16:33.384309+0000","last_peered":"2024-09-12T03:16:33.384309+0000","last_clean":"2024-09-12T03:16:33.384309+0000","last_became_active":"2024-09-12T03:16:30.584234+0000","last_became_peered":"2024-09-12T03:16:30.584234+0000","last_unstale":"2024-09-12T03:16:33.384309+0000","last_undegraded":"2024-09-12T03:16:33.384309+0000","last_fullsized":"2024-09-12T03:16:33.384309+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:29:14.889397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542148,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31980,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339860,"statfs":{"total":96636764160,"available":96604016640,"internally_reserved":0,"allocated":811008,"data_stored":665113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542150,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31408,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340432,"statfs":{"total":96636764160,"available":96604602368,"internally_reserved":0,"allocated":225280,"data_stored":75775,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542149,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31404,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340436,"statfs":{"total":96636764160,"available":96604606464,"internally_reserved":0,"allocated":217088,"data_stored":74253,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542148,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31408,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340432,"statfs":{"total":96636764160,"available":96604602368,"internally_reserved":0,"allocated":225280,"data_stored":74289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542149,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":225280,"data_stored":75775,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542149,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":212992,"data_stored":74163,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542150,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":221184,"data_stored":74323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542150,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27828,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344012,"statfs":{"total":96636764160,"available":96608268288,"internally_reserved":0,"allocated":819200,"data_stored":665169,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T03:16:39.301 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-12T03:16:39.302 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-12T03:16:39.302 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-09-12T03:16:39.302 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-09-12T03:16:39.303 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-09-12T03:16:39.439 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.439+0000 7f0a24de8640 1 Processor -- start 2024-09-12T03:16:39.439 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.439+0000 7f0a24de8640 1 -- start start 2024-09-12T03:16:39.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.440+0000 7f0a24de8640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f0a2015e5f0 0x7f0a20073250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:39.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.440+0000 7f0a24de8640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f0a20162a30 0x7f0a2015e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:39.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.440+0000 7f0a24de8640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0a20073bb0 con 0x7f0a20073790 2024-09-12T03:16:39.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.440+0000 7f0a24de8640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f0a200d0da0 con 0x7f0a20162a30 2024-09-12T03:16:39.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.440+0000 7f0a24de8640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f0a200d2240 con 0x7f0a2015e5f0 2024-09-12T03:16:39.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.440+0000 7f0a1ed9d640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f0a20162a30 0x7f0a2015e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:39.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.440+0000 7f0a1ed9d640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f0a20162a30 0x7f0a2015e9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33096/0 (socket says 172.21.15.27:33096) 2024-09-12T03:16:39.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.440+0000 7f0a1ed9d640 1 -- 172.21.15.27:0/3503100523 learned_addr learned my addr 172.21.15.27:0/3503100523 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:16:39.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.440+0000 7f0a1e59c640 1 --2- 172.21.15.27:0/3503100523 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f0a2015e5f0 0x7f0a20073250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:39.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.441+0000 7f0a1ed9d640 1 -- 172.21.15.27:0/3503100523 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f0a2015e5f0 msgr2=0x7f0a20073250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:39.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.441+0000 7f0a1ed9d640 1 --2- 172.21.15.27:0/3503100523 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f0a2015e5f0 0x7f0a20073250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:39.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.441+0000 7f0a1ed9d640 1 -- 172.21.15.27:0/3503100523 >> v1:172.21.15.45:6789/0 conn(0x7f0a20073790 legacy=0x7f0a20162320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:39.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.441+0000 7f0a1ed9d640 1 -- 172.21.15.27:0/3503100523 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a20163090 con 0x7f0a20162a30 2024-09-12T03:16:39.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.441+0000 7f0a1ed9d640 1 --2- 172.21.15.27:0/3503100523 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0a20162a30 0x7f0a2015e9d0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f0a10009e50 tx=0x7f0a1002a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b1a41880520493c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:39.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.441+0000 7f0a1d59a640 1 -- 172.21.15.27:0/3503100523 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a10036070 con 0x7f0a20162a30 2024-09-12T03:16:39.447 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.442+0000 7f0a1d59a640 1 -- 172.21.15.27:0/3503100523 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a10031600 con 0x7f0a20162a30 2024-09-12T03:16:39.447 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.442+0000 7f0a1d59a640 1 -- 172.21.15.27:0/3503100523 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a100319d0 con 0x7f0a20162a30 2024-09-12T03:16:39.447 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.442+0000 7f0a24de8640 1 -- 172.21.15.27:0/3503100523 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0a20162a30 msgr2=0x7f0a2015e9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:39.447 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.442+0000 7f0a24de8640 1 --2- 172.21.15.27:0/3503100523 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0a20162a30 0x7f0a2015e9d0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f0a10009e50 tx=0x7f0a1002a710 comp rx=0 tx=0).stop 2024-09-12T03:16:39.447 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.442+0000 7f0a24de8640 1 -- 172.21.15.27:0/3503100523 shutdown_connections 2024-09-12T03:16:39.447 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.442+0000 7f0a24de8640 1 --2- 172.21.15.27:0/3503100523 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0a20162a30 0x7f0a2015e9d0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:39.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.442+0000 7f0a24de8640 1 --2- 172.21.15.27:0/3503100523 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f0a2015e5f0 0x7f0a20073250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:39.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.442+0000 7f0a24de8640 1 -- 172.21.15.27:0/3503100523 >> 172.21.15.27:0/3503100523 conn(0x7f0a2006f300 msgr2=0x7f0a2006f710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:39.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.442+0000 7f0a24de8640 1 -- 172.21.15.27:0/3503100523 shutdown_connections 2024-09-12T03:16:39.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.442+0000 7f0a24de8640 1 -- 172.21.15.27:0/3503100523 wait complete. 2024-09-12T03:16:39.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.444+0000 7f0a24de8640 1 Processor -- start 2024-09-12T03:16:39.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.444+0000 7f0a24de8640 1 -- start start 2024-09-12T03:16:39.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.444+0000 7f0a24de8640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0a20073790 0x7f0a2014fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:39.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.444+0000 7f0a24de8640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0a2015e5f0 0x7f0a20152560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:39.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.444+0000 7f0a24de8640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f0a20162a30 0x7f0a20152aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:39.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.444+0000 7f0a24de8640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f0a200d0a30 con 0x7f0a20073790 2024-09-12T03:16:39.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.444+0000 7f0a24de8640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f0a200d0750 con 0x7f0a2015e5f0 2024-09-12T03:16:39.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.444+0000 7f0a24de8640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f0a200b5620 con 0x7f0a20162a30 2024-09-12T03:16:39.449 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.444+0000 7f0a1e59c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0a20073790 0x7f0a2014fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:39.449 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.444+0000 7f0a1e59c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0a20073790 0x7f0a2014fba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33098/0 (socket says 172.21.15.27:33098) 2024-09-12T03:16:39.449 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.444+0000 7f0a1e59c640 1 -- 172.21.15.27:0/2209598710 learned_addr learned my addr 172.21.15.27:0/2209598710 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:16:39.449 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.445+0000 7f0a1ed9d640 1 --2- 172.21.15.27:0/2209598710 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f0a20162a30 0x7f0a20152aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:39.449 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.445+0000 7f0a1dd9b640 1 --2- 172.21.15.27:0/2209598710 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0a2015e5f0 0x7f0a20152560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:39.449 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.445+0000 7f0a1e59c640 1 -- 172.21.15.27:0/2209598710 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f0a20162a30 msgr2=0x7f0a20152aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:39.449 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.445+0000 7f0a1e59c640 1 --2- 172.21.15.27:0/2209598710 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f0a20162a30 0x7f0a20152aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:39.449 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.445+0000 7f0a1e59c640 1 -- 172.21.15.27:0/2209598710 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0a2015e5f0 msgr2=0x7f0a20152560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:39.449 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.445+0000 7f0a1e59c640 1 --2- 172.21.15.27:0/2209598710 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0a2015e5f0 0x7f0a20152560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:39.449 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.445+0000 7f0a1e59c640 1 -- 172.21.15.27:0/2209598710 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a2015d830 con 0x7f0a20073790 2024-09-12T03:16:39.449 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.445+0000 7f0a1ed9d640 1 --2- 172.21.15.27:0/2209598710 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f0a20162a30 0x7f0a20152aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:16:39.449 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.445+0000 7f0a1e59c640 1 --2- 172.21.15.27:0/2209598710 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0a20073790 0x7f0a2014fba0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f0a1800e9b0 tx=0x7f0a1800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:39.449 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.445+0000 7f0a16ffd640 1 -- 172.21.15.27:0/2209598710 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a18017070 con 0x7f0a20073790 2024-09-12T03:16:39.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.445+0000 7f0a16ffd640 1 -- 172.21.15.27:0/2209598710 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a1800c9a0 con 0x7f0a20073790 2024-09-12T03:16:39.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.445+0000 7f0a24de8640 1 -- 172.21.15.27:0/2209598710 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a2015db10 con 0x7f0a20073790 2024-09-12T03:16:39.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.445+0000 7f0a16ffd640 1 -- 172.21.15.27:0/2209598710 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a1800cd90 con 0x7f0a20073790 2024-09-12T03:16:39.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.445+0000 7f0a24de8640 1 -- 172.21.15.27:0/2209598710 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a2013fb30 con 0x7f0a20073790 2024-09-12T03:16:39.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.446+0000 7f0a24de8640 1 -- 172.21.15.27:0/2209598710 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a201599d0 con 0x7f0a20073790 2024-09-12T03:16:39.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.447+0000 7f0a16ffd640 1 -- 172.21.15.27:0/2209598710 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0a1801a070 con 0x7f0a20073790 2024-09-12T03:16:39.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.447+0000 7f0a16ffd640 1 --2- 172.21.15.27:0/2209598710 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f0a0006be60 0x7f0a0006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:39.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.447+0000 7f0a16ffd640 1 -- 172.21.15.27:0/2209598710 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f0a1808b270 con 0x7f0a20073790 2024-09-12T03:16:39.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.447+0000 7f0a1dd9b640 1 --2- 172.21.15.27:0/2209598710 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f0a0006be60 0x7f0a0006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:39.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.448+0000 7f0a1dd9b640 1 --2- 172.21.15.27:0/2209598710 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f0a0006be60 0x7f0a0006e320 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f0a201612a0 tx=0x7f0a08009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:39.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.452+0000 7f0a16ffd640 1 -- 172.21.15.27:0/2209598710 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f0a18059b90 con 0x7f0a20073790 2024-09-12T03:16:39.679 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.679+0000 7f0a24de8640 1 -- 172.21.15.27:0/2209598710 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f0a20159410 con 0x7f0a20073790 2024-09-12T03:16:39.682 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.682+0000 7f0a16ffd640 1 -- 172.21.15.27:0/2209598710 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) ==== 74+0+172938 (secure 0 0 0) 0x7f0a18059490 con 0x7f0a20073790 2024-09-12T03:16:39.682 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:16:39.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.684+0000 7f0a24de8640 1 -- 172.21.15.27:0/2209598710 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f0a0006be60 msgr2=0x7f0a0006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:39.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.684+0000 7f0a24de8640 1 --2- 172.21.15.27:0/2209598710 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f0a0006be60 0x7f0a0006e320 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f0a201612a0 tx=0x7f0a08009040 comp rx=0 tx=0).stop 2024-09-12T03:16:39.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.684+0000 7f0a24de8640 1 -- 172.21.15.27:0/2209598710 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0a20073790 msgr2=0x7f0a2014fba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:39.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.684+0000 7f0a24de8640 1 --2- 172.21.15.27:0/2209598710 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0a20073790 0x7f0a2014fba0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f0a1800e9b0 tx=0x7f0a1800ee70 comp rx=0 tx=0).stop 2024-09-12T03:16:39.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.684+0000 7f0a24de8640 1 -- 172.21.15.27:0/2209598710 shutdown_connections 2024-09-12T03:16:39.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.684+0000 7f0a24de8640 1 --2- 172.21.15.27:0/2209598710 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f0a0006be60 0x7f0a0006e320 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:39.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.684+0000 7f0a24de8640 1 --2- 172.21.15.27:0/2209598710 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f0a20162a30 0x7f0a20152aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:39.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.684+0000 7f0a24de8640 1 --2- 172.21.15.27:0/2209598710 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0a2015e5f0 0x7f0a20152560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:39.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.684+0000 7f0a24de8640 1 --2- 172.21.15.27:0/2209598710 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0a20073790 0x7f0a2014fba0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:39.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.684+0000 7f0a24de8640 1 -- 172.21.15.27:0/2209598710 >> 172.21.15.27:0/2209598710 conn(0x7f0a2006f300 msgr2=0x7f0a2015efb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:39.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.684+0000 7f0a24de8640 1 -- 172.21.15.27:0/2209598710 shutdown_connections 2024-09-12T03:16:39.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.684+0000 7f0a24de8640 1 -- 172.21.15.27:0/2209598710 wait complete. 2024-09-12T03:16:39.695 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":5,"flags":0,"active_gid":4102,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":3923821219},{"type":"v1","addr":"172.21.15.45:6833","nonce":3923821219}]},"active_addr":"172.21.15.45:6833/3923821219","active_change":"2024-09-12T03:16:12.041759+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4113,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.45:0","nonce":3383141969}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.45:0","nonce":1945166272}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.45:0","nonce":687436879}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.45:0","nonce":4216551134}]}]} 2024-09-12T03:16:39.701 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-09-12T03:16:39.701 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-12T03:16:39.701 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-12T03:16:39.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.803+0000 7f2dc2d66640 1 Processor -- start 2024-09-12T03:16:39.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.803+0000 7f2dc2d66640 1 -- start start 2024-09-12T03:16:39.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.804+0000 7f2dc2d66640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f2dbc0d52e0 0x7f2dbc0cf810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:39.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.804+0000 7f2dc2d66640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2dbc0d3120 0x7f2dbc0d35b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:39.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.804+0000 7f2dc2d66640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2dbc0d1e60 con 0x7f2dbc0d1050 2024-09-12T03:16:39.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.804+0000 7f2dc2d66640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f2dbc0d2380 con 0x7f2dbc0d52e0 2024-09-12T03:16:39.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.804+0000 7f2dc2d66640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f2dbc0cfef0 con 0x7f2dbc0d3120 2024-09-12T03:16:39.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.804+0000 7f2dbbfff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f2dbc0d52e0 0x7f2dbc0cf810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:39.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.804+0000 7f2dc0b02640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f2dbc0d1050 0x7f2dbc0d1430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:39148/0 (socket says 172.21.15.27:39148) 2024-09-12T03:16:39.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.804+0000 7f2dc0b02640 1 -- 172.21.15.27:0/1732358792 learned_addr learned my addr 172.21.15.27:0/1732358792 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:16:39.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.805+0000 7f2dc1303640 1 --2- 172.21.15.27:0/1732358792 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2dbc0d3120 0x7f2dbc0d35b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:39.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.805+0000 7f2dbbfff640 1 -- 172.21.15.27:0/1732358792 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2dbc0d3120 msgr2=0x7f2dbc0d35b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:39.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.805+0000 7f2dbbfff640 1 --2- 172.21.15.27:0/1732358792 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2dbc0d3120 0x7f2dbc0d35b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:39.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.805+0000 7f2dbbfff640 1 -- 172.21.15.27:0/1732358792 >> v1:172.21.15.27:6789/0 conn(0x7f2dbc0d1050 legacy=0x7f2dbc0d1430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:39.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.805+0000 7f2dbbfff640 1 -- 172.21.15.27:0/1732358792 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2dbc0d2640 con 0x7f2dbc0d52e0 2024-09-12T03:16:39.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.805+0000 7f2dbbfff640 1 --2- 172.21.15.27:0/1732358792 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2dbc0d52e0 0x7f2dbc0cf810 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f2dac009d90 tx=0x7f2dac02a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0d40b845f33887f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:39.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.806+0000 7f2dbb7fe640 1 -- 172.21.15.27:0/1732358792 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dac007740 con 0x7f2dbc0d52e0 2024-09-12T03:16:39.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.806+0000 7f2dbb7fe640 1 -- 172.21.15.27:0/1732358792 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2dac0078e0 con 0x7f2dbc0d52e0 2024-09-12T03:16:39.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.806+0000 7f2dbb7fe640 1 -- 172.21.15.27:0/1732358792 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dac007cb0 con 0x7f2dbc0d52e0 2024-09-12T03:16:39.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.806+0000 7f2dc2d66640 1 -- 172.21.15.27:0/1732358792 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2dbc0d52e0 msgr2=0x7f2dbc0cf810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:39.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.806+0000 7f2dc2d66640 1 --2- 172.21.15.27:0/1732358792 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2dbc0d52e0 0x7f2dbc0cf810 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f2dac009d90 tx=0x7f2dac02a660 comp rx=0 tx=0).stop 2024-09-12T03:16:39.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.806+0000 7f2dc2d66640 1 -- 172.21.15.27:0/1732358792 shutdown_connections 2024-09-12T03:16:39.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.806+0000 7f2dc2d66640 1 --2- 172.21.15.27:0/1732358792 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2dbc0d3120 0x7f2dbc0d35b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:39.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.806+0000 7f2dc2d66640 1 --2- 172.21.15.27:0/1732358792 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2dbc0d52e0 0x7f2dbc0cf810 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:39.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.806+0000 7f2dc2d66640 1 -- 172.21.15.27:0/1732358792 >> 172.21.15.27:0/1732358792 conn(0x7f2dbc06ddf0 msgr2=0x7f2dbc06e200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:39.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.806+0000 7f2dc2d66640 1 -- 172.21.15.27:0/1732358792 shutdown_connections 2024-09-12T03:16:39.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.806+0000 7f2dc2d66640 1 -- 172.21.15.27:0/1732358792 wait complete. 2024-09-12T03:16:39.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.807+0000 7f2dc2d66640 1 Processor -- start 2024-09-12T03:16:39.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.807+0000 7f2dc2d66640 1 -- start start 2024-09-12T03:16:39.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.808+0000 7f2dc2d66640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2dbc0d1050 0x7f2dbc1f8040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:39.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.808+0000 7f2dc2d66640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2dbc0d3120 0x7f2dbc1f8580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:39.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.808+0000 7f2dc2d66640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2dbc0d52e0 0x7f2dbc0cd8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:39.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.808+0000 7f2dc2d66640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f2dbc0b7a50 con 0x7f2dbc0d3120 2024-09-12T03:16:39.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.808+0000 7f2dc2d66640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f2dbc0d2c00 con 0x7f2dbc0d52e0 2024-09-12T03:16:39.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.808+0000 7f2dc2d66640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f2dbc0b7bd0 con 0x7f2dbc0d1050 2024-09-12T03:16:39.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.808+0000 7f2dbbfff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2dbc0d3120 0x7f2dbc1f8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:39.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.808+0000 7f2dbbfff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2dbc0d3120 0x7f2dbc1f8580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33118/0 (socket says 172.21.15.27:33118) 2024-09-12T03:16:39.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.808+0000 7f2dbbfff640 1 -- 172.21.15.27:0/4122615812 learned_addr learned my addr 172.21.15.27:0/4122615812 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:16:39.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.809+0000 7f2dc0b02640 1 --2- 172.21.15.27:0/4122615812 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2dbc0d1050 0x7f2dbc1f8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:39.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.809+0000 7f2dc1303640 1 --2- 172.21.15.27:0/4122615812 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2dbc0d52e0 0x7f2dbc0cd8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:39.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.809+0000 7f2dbbfff640 1 -- 172.21.15.27:0/4122615812 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2dbc0d1050 msgr2=0x7f2dbc1f8040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:39.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.809+0000 7f2dbbfff640 1 --2- 172.21.15.27:0/4122615812 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2dbc0d1050 0x7f2dbc1f8040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:39.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.809+0000 7f2dbbfff640 1 -- 172.21.15.27:0/4122615812 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2dbc0d52e0 msgr2=0x7f2dbc0cd8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:39.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.809+0000 7f2dbbfff640 1 --2- 172.21.15.27:0/4122615812 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2dbc0d52e0 0x7f2dbc0cd8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:39.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.809+0000 7f2dbbfff640 1 -- 172.21.15.27:0/4122615812 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2dbc0c8e90 con 0x7f2dbc0d3120 2024-09-12T03:16:39.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.809+0000 7f2dbbfff640 1 --2- 172.21.15.27:0/4122615812 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2dbc0d3120 0x7f2dbc1f8580 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f2dac009d90 tx=0x7f2dac004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:39.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.809+0000 7f2db97fa640 1 -- 172.21.15.27:0/4122615812 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dac03f070 con 0x7f2dbc0d3120 2024-09-12T03:16:39.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.809+0000 7f2db97fa640 1 -- 172.21.15.27:0/4122615812 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2dac031c40 con 0x7f2dbc0d3120 2024-09-12T03:16:39.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.809+0000 7f2dc2d66640 1 -- 172.21.15.27:0/4122615812 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2dbc0c9170 con 0x7f2dbc0d3120 2024-09-12T03:16:39.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.809+0000 7f2db97fa640 1 -- 172.21.15.27:0/4122615812 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dac044460 con 0x7f2dbc0d3120 2024-09-12T03:16:39.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.809+0000 7f2dc2d66640 1 -- 172.21.15.27:0/4122615812 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2dbc0c9730 con 0x7f2dbc0d3120 2024-09-12T03:16:39.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.810+0000 7f2dc2d66640 1 -- 172.21.15.27:0/4122615812 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2dbc15ec60 con 0x7f2dbc0d3120 2024-09-12T03:16:39.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.811+0000 7f2db97fa640 1 -- 172.21.15.27:0/4122615812 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2dac044600 con 0x7f2dbc0d3120 2024-09-12T03:16:39.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.811+0000 7f2db97fa640 1 --2- 172.21.15.27:0/4122615812 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f2d9c06be30 0x7f2d9c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:39.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.811+0000 7f2db97fa640 1 -- 172.21.15.27:0/4122615812 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f2dac0ac5f0 con 0x7f2dbc0d3120 2024-09-12T03:16:39.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.812+0000 7f2dc0b02640 1 --2- 172.21.15.27:0/4122615812 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f2d9c06be30 0x7f2d9c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:39.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.812+0000 7f2dc0b02640 1 --2- 172.21.15.27:0/4122615812 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f2d9c06be30 0x7f2d9c06e2f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f2db4007fd0 tx=0x7f2db4007e80 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:39.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.816+0000 7f2db97fa640 1 -- 172.21.15.27:0/4122615812 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f2dac07a950 con 0x7f2dbc0d3120 2024-09-12T03:16:39.992 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.992+0000 7f2dc2d66640 1 -- 172.21.15.27:0/4122615812 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2dbc0d20a0 con 0x7f2dbc0d3120 2024-09-12T03:16:39.993 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.993+0000 7f2db97fa640 1 -- 172.21.15.27:0/4122615812 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16617 (secure 0 0 0) 0x7f2dac07ec50 con 0x7f2dbc0d3120 2024-09-12T03:16:39.994 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:16:39.994 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":21,"fsid":"d2df504e-2647-4385-a5db-9fdf69f487a0","created":"2024-09-12T03:16:07.778937+0000","modified":"2024-09-12T03:16:32.570182+0000","last_up_change":"2024-09-12T03:16:19.314778+0000","last_in_change":"2024-09-12T03:16:09.893086+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T03:16:20.097646+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-12T03:16:28.790791+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-12T03:16:29.909065+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1851116c-1fb3-4dc9-9772-f90e2ef4c355","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6817","nonce":3002321488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6819","nonce":3002321488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6823","nonce":3002321488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6821","nonce":3002321488}]},"public_addr":"172.21.15.27:6817/3002321488","cluster_addr":"172.21.15.27:6819/3002321488","heartbeat_back_addr":"172.21.15.27:6823/3002321488","heartbeat_front_addr":"172.21.15.27:6821/3002321488","state":["exists","up"]},{"osd":1,"uuid":"f38ae17d-73eb-4204-b99d-975fa806fb87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6809","nonce":1613102886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6811","nonce":1613102886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6815","nonce":1613102886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6813","nonce":1613102886}]},"public_addr":"172.21.15.27:6809/1613102886","cluster_addr":"172.21.15.27:6811/1613102886","heartbeat_back_addr":"172.21.15.27:6815/1613102886","heartbeat_front_addr":"172.21.15.27:6813/1613102886","state":["exists","up"]},{"osd":2,"uuid":"fb12dcc8-bdd0-42b8-b46d-8b10106224bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6825","nonce":3534723355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6827","nonce":3534723355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6831","nonce":3534723355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6829","nonce":3534723355}]},"public_addr":"172.21.15.27:6825/3534723355","cluster_addr":"172.21.15.27:6827/3534723355","heartbeat_back_addr":"172.21.15.27:6831/3534723355","heartbeat_front_addr":"172.21.15.27:6829/3534723355","state":["exists","up"]},{"osd":3,"uuid":"d74d3f49-9c24-4b9f-98e2-3ecd7b5ef3b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6801","nonce":80661598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6803","nonce":80661598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6807","nonce":80661598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6805","nonce":80661598}]},"public_addr":"172.21.15.27:6801/80661598","cluster_addr":"172.21.15.27:6803/80661598","heartbeat_back_addr":"172.21.15.27:6807/80661598","heartbeat_front_addr":"172.21.15.27:6805/80661598","state":["exists","up"]},{"osd":4,"uuid":"2bf123f2-1b0d-41c4-aa3b-805bab267e59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6817","nonce":3943189827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6819","nonce":3943189827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6823","nonce":3943189827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6821","nonce":3943189827}]},"public_addr":"172.21.15.45:6817/3943189827","cluster_addr":"172.21.15.45:6819/3943189827","heartbeat_back_addr":"172.21.15.45:6823/3943189827","heartbeat_front_addr":"172.21.15.45:6821/3943189827","state":["exists","up"]},{"osd":5,"uuid":"c49ba5da-91c6-4a86-a97b-4aba89c253ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6809","nonce":3620760330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6811","nonce":3620760330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6815","nonce":3620760330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6813","nonce":3620760330}]},"public_addr":"172.21.15.45:6809/3620760330","cluster_addr":"172.21.15.45:6811/3620760330","heartbeat_back_addr":"172.21.15.45:6815/3620760330","heartbeat_front_addr":"172.21.15.45:6813/3620760330","state":["exists","up"]},{"osd":6,"uuid":"b0123475-89f7-4a34-9e33-d797b99dd9dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6801","nonce":1989083510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6803","nonce":1989083510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6807","nonce":1989083510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6805","nonce":1989083510}]},"public_addr":"172.21.15.45:6801/1989083510","cluster_addr":"172.21.15.45:6803/1989083510","heartbeat_back_addr":"172.21.15.45:6807/1989083510","heartbeat_front_addr":"172.21.15.45:6805/1989083510","state":["exists","up"]},{"osd":7,"uuid":"4dad773d-b7d8-46fc-8a10-a972aad27b29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6825","nonce":1513300294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6827","nonce":1513300294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6831","nonce":1513300294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6829","nonce":1513300294}]},"public_addr":"172.21.15.45:6825/1513300294","cluster_addr":"172.21.15.45:6827/1513300294","heartbeat_back_addr":"172.21.15.45:6831/1513300294","heartbeat_front_addr":"172.21.15.45:6829/1513300294","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.881855+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:17.133252+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.735339+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.500742+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.477571+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:17.167667+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.470291+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.873743+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T03:16:39.995 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.995+0000 7f2dc2d66640 1 -- 172.21.15.27:0/4122615812 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f2d9c06be30 msgr2=0x7f2d9c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:39.996 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.995+0000 7f2dc2d66640 1 --2- 172.21.15.27:0/4122615812 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f2d9c06be30 0x7f2d9c06e2f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f2db4007fd0 tx=0x7f2db4007e80 comp rx=0 tx=0).stop 2024-09-12T03:16:39.996 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.995+0000 7f2dc2d66640 1 -- 172.21.15.27:0/4122615812 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2dbc0d3120 msgr2=0x7f2dbc1f8580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:39.996 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.995+0000 7f2dc2d66640 1 --2- 172.21.15.27:0/4122615812 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2dbc0d3120 0x7f2dbc1f8580 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f2dac009d90 tx=0x7f2dac004a40 comp rx=0 tx=0).stop 2024-09-12T03:16:39.996 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.995+0000 7f2dc2d66640 1 -- 172.21.15.27:0/4122615812 shutdown_connections 2024-09-12T03:16:39.996 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.995+0000 7f2dc2d66640 1 --2- 172.21.15.27:0/4122615812 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f2d9c06be30 0x7f2d9c06e2f0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:39.996 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.995+0000 7f2dc2d66640 1 --2- 172.21.15.27:0/4122615812 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2dbc0d52e0 0x7f2dbc0cd8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:39.996 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.995+0000 7f2dc2d66640 1 --2- 172.21.15.27:0/4122615812 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2dbc0d3120 0x7f2dbc1f8580 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:39.996 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.995+0000 7f2dc2d66640 1 --2- 172.21.15.27:0/4122615812 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2dbc0d1050 0x7f2dbc1f8040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:39.996 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.995+0000 7f2dc2d66640 1 -- 172.21.15.27:0/4122615812 >> 172.21.15.27:0/4122615812 conn(0x7f2dbc06ddf0 msgr2=0x7f2dbc0b5a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:39.996 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.995+0000 7f2dc2d66640 1 -- 172.21.15.27:0/4122615812 shutdown_connections 2024-09-12T03:16:39.996 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:39.995+0000 7f2dc2d66640 1 -- 172.21.15.27:0/4122615812 wait complete. 2024-09-12T03:16:40.007 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-12T03:16:40.007 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-12T03:16:40.109 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.109+0000 7f8e2a52a640 1 Processor -- start 2024-09-12T03:16:40.110 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.109+0000 7f8e2a52a640 1 -- start start 2024-09-12T03:16:40.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.110+0000 7f8e2a52a640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f8e2415e660 0x7f8e24073210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:40.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.110+0000 7f8e2a52a640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f8e24071150 0x7f8e24073750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:40.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.110+0000 7f8e2a52a640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8e2415ea40 con 0x7f8e240cf3c0 2024-09-12T03:16:40.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.110+0000 7f8e2a52a640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f8e240b5a20 con 0x7f8e24071150 2024-09-12T03:16:40.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.110+0000 7f8e2a52a640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f8e2406d240 con 0x7f8e2415e660 2024-09-12T03:16:40.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.110+0000 7f8e28ac7640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f8e24071150 0x7f8e24073750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:40.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.110+0000 7f8e237fe640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f8e240cf3c0 0x7f8e24070a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:39150/0 (socket says 172.21.15.27:39150) 2024-09-12T03:16:40.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.110+0000 7f8e237fe640 1 -- 172.21.15.27:0/1350793961 learned_addr learned my addr 172.21.15.27:0/1350793961 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:16:40.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.111+0000 7f8e23fff640 1 --2- 172.21.15.27:0/1350793961 >> v2:172.21.15.45:3300/0 conn(0x7f8e2415e660 0x7f8e24073210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:40.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.111+0000 7f8e28ac7640 1 -- 172.21.15.27:0/1350793961 >> v2:172.21.15.45:3300/0 conn(0x7f8e2415e660 msgr2=0x7f8e24073210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:40.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.111+0000 7f8e28ac7640 1 --2- 172.21.15.27:0/1350793961 >> v2:172.21.15.45:3300/0 conn(0x7f8e2415e660 0x7f8e24073210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:40.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.111+0000 7f8e28ac7640 1 -- 172.21.15.27:0/1350793961 >> v1:172.21.15.27:6789/0 conn(0x7f8e240cf3c0 legacy=0x7f8e24070a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:40.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.111+0000 7f8e22ffd640 1 -- 172.21.15.27:0/1350793961 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 631325332 0 0) 0x7f8e2415ea40 con 0x7f8e240cf3c0 2024-09-12T03:16:40.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.111+0000 7f8e28ac7640 1 -- 172.21.15.27:0/1350793961 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e24071970 con 0x7f8e24071150 2024-09-12T03:16:40.113 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.111+0000 7f8e23fff640 1 --2- 172.21.15.27:0/1350793961 >> v2:172.21.15.45:3300/0 conn(0x7f8e2415e660 0x7f8e24073210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:16:40.113 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.111+0000 7f8e28ac7640 1 --2- 172.21.15.27:0/1350793961 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8e24071150 0x7f8e24073750 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f8e240754f0 tx=0x7f8e1c02a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9a5699e8f0f85 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:40.113 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.112+0000 7f8e22ffd640 1 -- 172.21.15.27:0/1350793961 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e1c007740 con 0x7f8e24071150 2024-09-12T03:16:40.113 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.112+0000 7f8e22ffd640 1 -- 172.21.15.27:0/1350793961 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e1c0078e0 con 0x7f8e24071150 2024-09-12T03:16:40.113 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.112+0000 7f8e22ffd640 1 -- 172.21.15.27:0/1350793961 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e1c007cb0 con 0x7f8e24071150 2024-09-12T03:16:40.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.112+0000 7f8e2a52a640 1 -- 172.21.15.27:0/1350793961 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8e24071150 msgr2=0x7f8e24073750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:40.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.112+0000 7f8e2a52a640 1 --2- 172.21.15.27:0/1350793961 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8e24071150 0x7f8e24073750 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f8e240754f0 tx=0x7f8e1c02a680 comp rx=0 tx=0).stop 2024-09-12T03:16:40.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.112+0000 7f8e2a52a640 1 -- 172.21.15.27:0/1350793961 shutdown_connections 2024-09-12T03:16:40.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.112+0000 7f8e2a52a640 1 --2- 172.21.15.27:0/1350793961 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8e24071150 0x7f8e24073750 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:40.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.112+0000 7f8e2a52a640 1 --2- 172.21.15.27:0/1350793961 >> v2:172.21.15.45:3300/0 conn(0x7f8e2415e660 0x7f8e24073210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:40.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.112+0000 7f8e2a52a640 1 -- 172.21.15.27:0/1350793961 >> 172.21.15.27:0/1350793961 conn(0x7f8e2406f2c0 msgr2=0x7f8e2406f6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:40.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.112+0000 7f8e2a52a640 1 -- 172.21.15.27:0/1350793961 shutdown_connections 2024-09-12T03:16:40.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.112+0000 7f8e2a52a640 1 -- 172.21.15.27:0/1350793961 wait complete. 2024-09-12T03:16:40.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.113+0000 7f8e2a52a640 1 Processor -- start 2024-09-12T03:16:40.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.114+0000 7f8e2a52a640 1 -- start start 2024-09-12T03:16:40.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.114+0000 7f8e2a52a640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f8e24071150 0x7f8e2415a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:40.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.114+0000 7f8e2a52a640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8e240cf3c0 0x7f8e2414e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:40.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.114+0000 7f8e2a52a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8e2415e660 0x7f8e24154200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:40.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.114+0000 7f8e2a52a640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f8e2406d7c0 con 0x7f8e240cf3c0 2024-09-12T03:16:40.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.114+0000 7f8e2a52a640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f8e2406d4e0 con 0x7f8e2415e660 2024-09-12T03:16:40.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.114+0000 7f8e2a52a640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f8e241681e0 con 0x7f8e24071150 2024-09-12T03:16:40.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.114+0000 7f8e237fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8e240cf3c0 0x7f8e2414e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:40.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.114+0000 7f8e237fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8e240cf3c0 0x7f8e2414e1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33142/0 (socket says 172.21.15.27:33142) 2024-09-12T03:16:40.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.114+0000 7f8e237fe640 1 -- 172.21.15.27:0/3407795023 learned_addr learned my addr 172.21.15.27:0/3407795023 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:16:40.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.115+0000 7f8e23fff640 1 --2- 172.21.15.27:0/3407795023 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f8e24071150 0x7f8e2415a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:40.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.115+0000 7f8e28ac7640 1 --2- 172.21.15.27:0/3407795023 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8e2415e660 0x7f8e24154200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:40.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.115+0000 7f8e237fe640 1 -- 172.21.15.27:0/3407795023 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f8e24071150 msgr2=0x7f8e2415a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:40.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.115+0000 7f8e237fe640 1 --2- 172.21.15.27:0/3407795023 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f8e24071150 0x7f8e2415a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:40.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.115+0000 7f8e237fe640 1 -- 172.21.15.27:0/3407795023 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8e2415e660 msgr2=0x7f8e24154200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:40.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.115+0000 7f8e237fe640 1 --2- 172.21.15.27:0/3407795023 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8e2415e660 0x7f8e24154200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:40.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.115+0000 7f8e237fe640 1 -- 172.21.15.27:0/3407795023 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e24152400 con 0x7f8e240cf3c0 2024-09-12T03:16:40.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.115+0000 7f8e237fe640 1 --2- 172.21.15.27:0/3407795023 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8e240cf3c0 0x7f8e2414e1a0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f8e1800c4e0 tx=0x7f8e1800c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:40.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.115+0000 7f8e20ff9640 1 -- 172.21.15.27:0/3407795023 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e18018070 con 0x7f8e240cf3c0 2024-09-12T03:16:40.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.115+0000 7f8e20ff9640 1 -- 172.21.15.27:0/3407795023 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e18007500 con 0x7f8e240cf3c0 2024-09-12T03:16:40.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.115+0000 7f8e2a52a640 1 -- 172.21.15.27:0/3407795023 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e241526e0 con 0x7f8e240cf3c0 2024-09-12T03:16:40.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.115+0000 7f8e20ff9640 1 -- 172.21.15.27:0/3407795023 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e18004a20 con 0x7f8e240cf3c0 2024-09-12T03:16:40.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.115+0000 7f8e2a52a640 1 -- 172.21.15.27:0/3407795023 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e24152ca0 con 0x7f8e240cf3c0 2024-09-12T03:16:40.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.116+0000 7f8e2a52a640 1 -- 172.21.15.27:0/3407795023 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e24070a40 con 0x7f8e240cf3c0 2024-09-12T03:16:40.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.117+0000 7f8e20ff9640 1 -- 172.21.15.27:0/3407795023 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8e1801b070 con 0x7f8e240cf3c0 2024-09-12T03:16:40.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.117+0000 7f8e20ff9640 1 --2- 172.21.15.27:0/3407795023 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f8e1006be30 0x7f8e1006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:40.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.117+0000 7f8e20ff9640 1 -- 172.21.15.27:0/3407795023 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f8e1808a130 con 0x7f8e240cf3c0 2024-09-12T03:16:40.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.117+0000 7f8e23fff640 1 --2- 172.21.15.27:0/3407795023 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f8e1006be30 0x7f8e1006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:40.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.118+0000 7f8e23fff640 1 --2- 172.21.15.27:0/3407795023 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f8e1006be30 0x7f8e1006e2f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f8e1400a950 tx=0x7f8e14008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:40.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.121+0000 7f8e20ff9640 1 -- 172.21.15.27:0/3407795023 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f8e18058c30 con 0x7f8e240cf3c0 2024-09-12T03:16:40.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.299+0000 7f8e2a52a640 1 -- 172.21.15.27:0/3407795023 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8e240b3f90 con 0x7f8e240cf3c0 2024-09-12T03:16:40.300 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.300+0000 7f8e20ff9640 1 -- 172.21.15.27:0/3407795023 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16617 (secure 0 0 0) 0x7f8e18058530 con 0x7f8e240cf3c0 2024-09-12T03:16:40.300 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:16:40.300 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":21,"fsid":"d2df504e-2647-4385-a5db-9fdf69f487a0","created":"2024-09-12T03:16:07.778937+0000","modified":"2024-09-12T03:16:32.570182+0000","last_up_change":"2024-09-12T03:16:19.314778+0000","last_in_change":"2024-09-12T03:16:09.893086+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T03:16:20.097646+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-12T03:16:28.790791+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-12T03:16:29.909065+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1851116c-1fb3-4dc9-9772-f90e2ef4c355","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6817","nonce":3002321488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6819","nonce":3002321488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6823","nonce":3002321488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6821","nonce":3002321488}]},"public_addr":"172.21.15.27:6817/3002321488","cluster_addr":"172.21.15.27:6819/3002321488","heartbeat_back_addr":"172.21.15.27:6823/3002321488","heartbeat_front_addr":"172.21.15.27:6821/3002321488","state":["exists","up"]},{"osd":1,"uuid":"f38ae17d-73eb-4204-b99d-975fa806fb87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6809","nonce":1613102886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6811","nonce":1613102886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6815","nonce":1613102886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6813","nonce":1613102886}]},"public_addr":"172.21.15.27:6809/1613102886","cluster_addr":"172.21.15.27:6811/1613102886","heartbeat_back_addr":"172.21.15.27:6815/1613102886","heartbeat_front_addr":"172.21.15.27:6813/1613102886","state":["exists","up"]},{"osd":2,"uuid":"fb12dcc8-bdd0-42b8-b46d-8b10106224bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6825","nonce":3534723355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6827","nonce":3534723355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6831","nonce":3534723355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6829","nonce":3534723355}]},"public_addr":"172.21.15.27:6825/3534723355","cluster_addr":"172.21.15.27:6827/3534723355","heartbeat_back_addr":"172.21.15.27:6831/3534723355","heartbeat_front_addr":"172.21.15.27:6829/3534723355","state":["exists","up"]},{"osd":3,"uuid":"d74d3f49-9c24-4b9f-98e2-3ecd7b5ef3b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6801","nonce":80661598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6803","nonce":80661598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6807","nonce":80661598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6805","nonce":80661598}]},"public_addr":"172.21.15.27:6801/80661598","cluster_addr":"172.21.15.27:6803/80661598","heartbeat_back_addr":"172.21.15.27:6807/80661598","heartbeat_front_addr":"172.21.15.27:6805/80661598","state":["exists","up"]},{"osd":4,"uuid":"2bf123f2-1b0d-41c4-aa3b-805bab267e59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6817","nonce":3943189827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6819","nonce":3943189827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6823","nonce":3943189827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6821","nonce":3943189827}]},"public_addr":"172.21.15.45:6817/3943189827","cluster_addr":"172.21.15.45:6819/3943189827","heartbeat_back_addr":"172.21.15.45:6823/3943189827","heartbeat_front_addr":"172.21.15.45:6821/3943189827","state":["exists","up"]},{"osd":5,"uuid":"c49ba5da-91c6-4a86-a97b-4aba89c253ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6809","nonce":3620760330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6811","nonce":3620760330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6815","nonce":3620760330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6813","nonce":3620760330}]},"public_addr":"172.21.15.45:6809/3620760330","cluster_addr":"172.21.15.45:6811/3620760330","heartbeat_back_addr":"172.21.15.45:6815/3620760330","heartbeat_front_addr":"172.21.15.45:6813/3620760330","state":["exists","up"]},{"osd":6,"uuid":"b0123475-89f7-4a34-9e33-d797b99dd9dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6801","nonce":1989083510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6803","nonce":1989083510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6807","nonce":1989083510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6805","nonce":1989083510}]},"public_addr":"172.21.15.45:6801/1989083510","cluster_addr":"172.21.15.45:6803/1989083510","heartbeat_back_addr":"172.21.15.45:6807/1989083510","heartbeat_front_addr":"172.21.15.45:6805/1989083510","state":["exists","up"]},{"osd":7,"uuid":"4dad773d-b7d8-46fc-8a10-a972aad27b29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6825","nonce":1513300294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6827","nonce":1513300294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6831","nonce":1513300294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6829","nonce":1513300294}]},"public_addr":"172.21.15.45:6825/1513300294","cluster_addr":"172.21.15.45:6827/1513300294","heartbeat_back_addr":"172.21.15.45:6831/1513300294","heartbeat_front_addr":"172.21.15.45:6829/1513300294","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.881855+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:17.133252+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.735339+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.500742+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.477571+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:17.167667+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.470291+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.873743+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T03:16:40.302 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.302+0000 7f8e2a52a640 1 -- 172.21.15.27:0/3407795023 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f8e1006be30 msgr2=0x7f8e1006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:40.303 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.302+0000 7f8e2a52a640 1 --2- 172.21.15.27:0/3407795023 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f8e1006be30 0x7f8e1006e2f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f8e1400a950 tx=0x7f8e14008040 comp rx=0 tx=0).stop 2024-09-12T03:16:40.303 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.302+0000 7f8e2a52a640 1 -- 172.21.15.27:0/3407795023 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8e240cf3c0 msgr2=0x7f8e2414e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:40.303 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.302+0000 7f8e2a52a640 1 --2- 172.21.15.27:0/3407795023 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8e240cf3c0 0x7f8e2414e1a0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f8e1800c4e0 tx=0x7f8e1800c9a0 comp rx=0 tx=0).stop 2024-09-12T03:16:40.303 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.302+0000 7f8e2a52a640 1 -- 172.21.15.27:0/3407795023 shutdown_connections 2024-09-12T03:16:40.303 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.302+0000 7f8e2a52a640 1 --2- 172.21.15.27:0/3407795023 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f8e1006be30 0x7f8e1006e2f0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:40.303 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.302+0000 7f8e2a52a640 1 --2- 172.21.15.27:0/3407795023 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8e2415e660 0x7f8e24154200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:40.303 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.302+0000 7f8e2a52a640 1 --2- 172.21.15.27:0/3407795023 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8e240cf3c0 0x7f8e2414e1a0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:40.303 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.302+0000 7f8e2a52a640 1 --2- 172.21.15.27:0/3407795023 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f8e24071150 0x7f8e2415a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:40.303 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.302+0000 7f8e2a52a640 1 -- 172.21.15.27:0/3407795023 >> 172.21.15.27:0/3407795023 conn(0x7f8e2406f2c0 msgr2=0x7f8e24157eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:40.303 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.302+0000 7f8e2a52a640 1 -- 172.21.15.27:0/3407795023 shutdown_connections 2024-09-12T03:16:40.303 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.302+0000 7f8e2a52a640 1 -- 172.21.15.27:0/3407795023 wait complete. 2024-09-12T03:16:40.314 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-12T03:16:40.314 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-12T03:16:40.315 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-12T03:16:40.315 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-12T03:16:40.316 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-09-12T03:16:40.316 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-09-12T03:16:40.316 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-09-12T03:16:40.317 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-09-12T03:16:40.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.491+0000 7f8d85c7b640 1 Processor -- start 2024-09-12T03:16:40.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.491+0000 7f8d85c7b640 1 -- start start 2024-09-12T03:16:40.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.492+0000 7f3c8f59e640 1 Processor -- start 2024-09-12T03:16:40.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.492+0000 7f3c8f59e640 1 -- start start 2024-09-12T03:16:40.494 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.494+0000 7fc58b3c6640 1 Processor -- start 2024-09-12T03:16:40.494 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.494+0000 7fc58b3c6640 1 -- start start 2024-09-12T03:16:40.495 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.493+0000 7fe9b6b98640 1 Processor -- start 2024-09-12T03:16:40.495 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.493+0000 7fe9b6b98640 1 -- start start 2024-09-12T03:16:40.495 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.494+0000 7f3c8f59e640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f3c900cd520 0x7f3c90070fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:40.495 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.494+0000 7f3c8f59e640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f3c9006e980 0x7f3c90162e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:40.495 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.494+0000 7f3c8f59e640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3c90071670 con 0x7f3c900cd140 2024-09-12T03:16:40.495 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.494+0000 7f3c8f59e640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f3c900b34f0 con 0x7f3c900cd520 2024-09-12T03:16:40.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.494+0000 7f3c8f59e640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f3c9006ee30 con 0x7f3c9006e980 2024-09-12T03:16:40.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.494+0000 7f3c8e59c640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f3c900cd140 0x7f3c90162700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.27:38894/0 (socket says 172.21.15.27:38894) 2024-09-12T03:16:40.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.494+0000 7f3c8e59c640 1 -- 172.21.15.27:0/3620354981 learned_addr learned my addr 172.21.15.27:0/3620354981 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:16:40.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.494+0000 7f3c8ed9d640 1 --2- 172.21.15.27:0/3620354981 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f3c9006e980 0x7f3c90162e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:40.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.492+0000 7f8d85c7b640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f8d8006d010 0x7f8d800afc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:40.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.492+0000 7f8d85c7b640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f8d80072930 0x7f8d80071270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:40.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.492+0000 7f8d85c7b640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8d800b5a10 con 0x7f8d8006d3f0 2024-09-12T03:16:40.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.492+0000 7f8d85c7b640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f8d80071890 con 0x7f8d80072930 2024-09-12T03:16:40.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.492+0000 7f8d85c7b640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f8d80071b70 con 0x7f8d8006d010 2024-09-12T03:16:40.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.492+0000 7f8d7ffff640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f8d8006d3f0 0x7f8d800b4720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:39164/0 (socket says 172.21.15.27:39164) 2024-09-12T03:16:40.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.492+0000 7f8d7ffff640 1 -- 172.21.15.27:0/4253355514 learned_addr learned my addr 172.21.15.27:0/4253355514 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:16:40.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.493+0000 7f8d7effd640 1 --2- 172.21.15.27:0/4253355514 >> v2:172.21.15.45:3300/0 conn(0x7f8d80072930 0x7f8d80071270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:40.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.493+0000 7f8d7e7fc640 1 -- 172.21.15.27:0/4253355514 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3413141129 0 0) 0x7f8d800b5a10 con 0x7f8d8006d3f0 2024-09-12T03:16:40.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.493+0000 7f8d7e7fc640 1 -- 172.21.15.27:0/4253355514 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8d68003520 con 0x7f8d8006d3f0 2024-09-12T03:16:40.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.493+0000 7f8d7effd640 1 -- 172.21.15.27:0/4253355514 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f8d8006d010 msgr2=0x7f8d800afc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:40.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.493+0000 7f8d7effd640 1 --2- 172.21.15.27:0/4253355514 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f8d8006d010 0x7f8d800afc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:40.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.493+0000 7f8d7effd640 1 -- 172.21.15.27:0/4253355514 >> v1:172.21.15.27:6789/0 conn(0x7f8d8006d3f0 legacy=0x7f8d800b4720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:40.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.493+0000 7f8d7effd640 1 -- 172.21.15.27:0/4253355514 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d80071e50 con 0x7f8d80072930 2024-09-12T03:16:40.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.493+0000 7f4a8dc7b640 1 Processor -- start 2024-09-12T03:16:40.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.493+0000 7f4a8dc7b640 1 -- start start 2024-09-12T03:16:40.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.494+0000 7fe9b6b98640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fe9b0162c00 0x7fe9b0073250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:40.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.494+0000 7fe9b6b98640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe9b0073790 0x7fe9b006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:40.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.495+0000 7fe9b6b98640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe9b0162fe0 con 0x7fe9b006e830 2024-09-12T03:16:40.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.495+0000 7fe9b6b98640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fe9b00d1ca0 con 0x7fe9b0162c00 2024-09-12T03:16:40.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.495+0000 7fe9b6b98640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fe9b0073bd0 con 0x7fe9b0073790 2024-09-12T03:16:40.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.495+0000 7fe9b4934640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fe9b0162c00 0x7fe9b0073250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:40.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.495+0000 7fe9affff640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe9b0073790 0x7fe9b006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:40.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.495+0000 7fe9b5135640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fe9b006e830 0x7fe9b01666c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:39176/0 (socket says 172.21.15.27:39176) 2024-09-12T03:16:40.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.495+0000 7fe9b5135640 1 -- 172.21.15.27:0/917348323 learned_addr learned my addr 172.21.15.27:0/917348323 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:16:40.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.495+0000 7f8d7effd640 1 --2- 172.21.15.27:0/4253355514 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8d80072930 0x7f8d80071270 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f8d740097a0 tx=0x7f8d7402c3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3d19228705efd1fe server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:40.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.495+0000 7f4a8dc7b640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4a880d4230 0x7f4a880b9240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:40.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.495+0000 7f4a8cc79640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f4a8806c160 0x7f4a880cff60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.27:38898/0 (socket says 172.21.15.27:38898) 2024-09-12T03:16:40.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.495+0000 7f4a8cc79640 1 -- 172.21.15.27:0/1156939991 learned_addr learned my addr 172.21.15.27:0/1156939991 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:16:40.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.495+0000 7f4a8dc7b640 1 --2- 172.21.15.27:0/1156939991 >> v2:172.21.15.27:3300/0 conn(0x7f4a880b9780 0x7f4a880d2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:40.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.495+0000 7f4a8dc7b640 1 -- 172.21.15.27:0/1156939991 --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4a881630d0 con 0x7f4a8806c160 2024-09-12T03:16:40.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.495+0000 7f9ea21ac640 1 Processor -- start 2024-09-12T03:16:40.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.494+0000 7f2f84ef8640 1 Processor -- start 2024-09-12T03:16:40.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.495+0000 7f76b5645640 1 Processor -- start 2024-09-12T03:16:40.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.495+0000 7f4a8dc7b640 1 -- 172.21.15.27:0/1156939991 --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f4a880d07e0 con 0x7f4a880b9780 2024-09-12T03:16:40.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.495+0000 7f4a8dc7b640 1 -- 172.21.15.27:0/1156939991 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f4a880b8a40 con 0x7f4a880d4230 2024-09-12T03:16:40.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.495+0000 7f4a8d47a640 1 --2- 172.21.15.27:0/1156939991 >> v2:172.21.15.27:3300/0 conn(0x7f4a880b9780 0x7f4a880d2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:40.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.494+0000 7f3c8dd9b640 1 --2- 172.21.15.27:0/3620354981 >> v2:172.21.15.45:3300/0 conn(0x7f3c900cd520 0x7f3c90070fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:40.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.494+0000 7f3c8ed9d640 1 -- 172.21.15.27:0/3620354981 >> v2:172.21.15.45:3300/0 conn(0x7f3c900cd520 msgr2=0x7f3c90070fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:40.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.494+0000 7f3c8ed9d640 1 --2- 172.21.15.27:0/3620354981 >> v2:172.21.15.45:3300/0 conn(0x7f3c900cd520 0x7f3c90070fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:40.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.494+0000 7f3c8ed9d640 1 -- 172.21.15.27:0/3620354981 >> v1:172.21.15.45:6789/0 conn(0x7f3c900cd140 legacy=0x7f3c90162700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:40.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.495+0000 7f76b5645640 1 -- start start 2024-09-12T03:16:40.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.494+0000 7f3c8ed9d640 1 -- 172.21.15.27:0/3620354981 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c901669e0 con 0x7f3c9006e980 2024-09-12T03:16:40.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.495+0000 7f4a87fff640 1 --2- 172.21.15.27:0/1156939991 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4a880d4230 0x7f4a880b9240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:40.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.496+0000 7f4a877fe640 1 -- 172.21.15.27:0/1156939991 <== mon.1 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2565178057 0 0) 0x7f4a881630d0 con 0x7f4a8806c160 2024-09-12T03:16:40.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.496+0000 7f8d7e7fc640 1 -- 172.21.15.27:0/4253355514 <== mon.1 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d74039070 con 0x7f8d80072930 2024-09-12T03:16:40.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.496+0000 7f8d7e7fc640 1 -- 172.21.15.27:0/4253355514 <== mon.1 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d74034640 con 0x7f8d80072930 2024-09-12T03:16:40.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.496+0000 7f4a877fe640 1 -- 172.21.15.27:0/1156939991 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4a74003520 con 0x7f4a8806c160 2024-09-12T03:16:40.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.496+0000 7fe9affff640 1 -- 172.21.15.27:0/917348323 >> v2:172.21.15.45:3300/0 conn(0x7fe9b0162c00 msgr2=0x7fe9b0073250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:40.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.496+0000 7fe9affff640 1 --2- 172.21.15.27:0/917348323 >> v2:172.21.15.45:3300/0 conn(0x7fe9b0162c00 0x7fe9b0073250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:40.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.496+0000 7fe9affff640 1 -- 172.21.15.27:0/917348323 >> v1:172.21.15.27:6789/0 conn(0x7fe9b006e830 legacy=0x7fe9b01666c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:40.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.496+0000 7fe9affff640 1 -- 172.21.15.27:0/917348323 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9b00d1090 con 0x7fe9b0073790 2024-09-12T03:16:40.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.496+0000 7fc58b3c6640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fc5840d1410 0x7fc5840d0470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:40.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.496+0000 7fc58b3c6640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc5840d09b0 0x7fc584071480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:40.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.496+0000 7fc58b3c6640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc5840b7af0 con 0x7fc5840719c0 2024-09-12T03:16:40.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.496+0000 7fc58b3c6640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7fc5840b7250 con 0x7fc5840d1410 2024-09-12T03:16:40.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.496+0000 7fc58b3c6640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fc5840b7530 con 0x7fc5840d09b0 2024-09-12T03:16:40.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.496+0000 7fc589963640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fc5840719c0 0x7fc584073280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.27:38902/0 (socket says 172.21.15.27:38902) 2024-09-12T03:16:40.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.496+0000 7fc589963640 1 -- 172.21.15.27:0/3507186999 learned_addr learned my addr 172.21.15.27:0/3507186999 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:16:40.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.496+0000 7fc589162640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fc5840d1410 0x7fc5840d0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:40.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.496+0000 7f3c8ed9d640 1 --2- 172.21.15.27:0/3620354981 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f3c9006e980 0x7f3c90162e10 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f3c88009d90 tx=0x7f3c8802a630 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d0089236054f4b7d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:40.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.496+0000 7f3c8d59a640 1 -- 172.21.15.27:0/3620354981 <== mon.2 v2:172.21.15.45:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c88007740 con 0x7f3c9006e980 2024-09-12T03:16:40.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.496+0000 7f3c8d59a640 1 -- 172.21.15.27:0/3620354981 <== mon.2 v2:172.21.15.45:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c880078e0 con 0x7f3c9006e980 2024-09-12T03:16:40.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.496+0000 7f4a877fe640 1 -- 172.21.15.27:0/1156939991 <== mon.1 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 598187497 0 0) 0x7f4a74003520 con 0x7f4a8806c160 2024-09-12T03:16:40.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.496+0000 7f4a8d47a640 1 -- 172.21.15.27:0/1156939991 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4a880d4230 msgr2=0x7f4a880b9240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:40.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.496+0000 7f4a8d47a640 1 --2- 172.21.15.27:0/1156939991 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4a880d4230 0x7f4a880b9240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:40.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.496+0000 7f4a8d47a640 1 -- 172.21.15.27:0/1156939991 >> v1:172.21.15.45:6789/0 conn(0x7f4a8806c160 legacy=0x7f4a880cff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:40.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.496+0000 7f4a8d47a640 1 -- 172.21.15.27:0/1156939991 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a880b87d0 con 0x7f4a880b9780 2024-09-12T03:16:40.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.495+0000 7f9ea21ac640 1 -- start start 2024-09-12T03:16:40.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.496+0000 7f2f84ef8640 1 -- start start 2024-09-12T03:16:40.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.497+0000 7f8d85c7b640 1 -- 172.21.15.27:0/4253355514 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8d80072930 msgr2=0x7f8d80071270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:40.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.497+0000 7f8d85c7b640 1 --2- 172.21.15.27:0/4253355514 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8d80072930 0x7f8d80071270 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f8d740097a0 tx=0x7f8d7402c3b0 comp rx=0 tx=0).stop 2024-09-12T03:16:40.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.497+0000 7f76b5645640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f76b015e4a0 0x7f76b00b3ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:40.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.497+0000 7f9ea21ac640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f9e9c0b65e0 0x7f9e9c162760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:40.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.497+0000 7f9ea21ac640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f9e9c166fd0 0x7f9e9c162ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:40.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.497+0000 7f9ea21ac640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9e9c071a00 con 0x7f9e9c0b6200 2024-09-12T03:16:40.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.497+0000 7f9ea21ac640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f9e9c070960 con 0x7f9e9c0b65e0 2024-09-12T03:16:40.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.497+0000 7f9ea21ac640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f9e9c070c40 con 0x7f9e9c166fd0 2024-09-12T03:16:40.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.496+0000 7fe9b4934640 1 --2- 172.21.15.27:0/917348323 >> v2:172.21.15.45:3300/0 conn(0x7fe9b0162c00 0x7fe9b0073250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:16:40.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.497+0000 7fe9affff640 1 --2- 172.21.15.27:0/917348323 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe9b0073790 0x7fe9b006e2f0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fe9a0009d90 tx=0x7fe9a002a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=49daa979bdef7bf9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:40.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.497+0000 7f8d85c7b640 1 -- 172.21.15.27:0/4253355514 shutdown_connections 2024-09-12T03:16:40.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.497+0000 7f8d85c7b640 1 --2- 172.21.15.27:0/4253355514 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8d80072930 0x7f8d80071270 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:40.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.497+0000 7f8d85c7b640 1 --2- 172.21.15.27:0/4253355514 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f8d8006d010 0x7f8d800afc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:40.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.497+0000 7f8d85c7b640 1 -- 172.21.15.27:0/4253355514 >> 172.21.15.27:0/4253355514 conn(0x7f8d8006bd10 msgr2=0x7f8d8006c0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:40.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.497+0000 7f9ea09a9640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f9e9c0b65e0 0x7f9e9c162760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:40.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.497+0000 7fe9af7fe640 1 -- 172.21.15.27:0/917348323 <== mon.2 v2:172.21.15.45:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9a0007740 con 0x7fe9b0073790 2024-09-12T03:16:40.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.497+0000 7fe9af7fe640 1 -- 172.21.15.27:0/917348323 <== mon.2 v2:172.21.15.45:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9a00078e0 con 0x7fe9b0073790 2024-09-12T03:16:40.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.497+0000 7f76b5645640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f76b00afff0 0x7f76b00726d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:40.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.497+0000 7f76b5645640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f76b0071630 con 0x7f76b00b43e0 2024-09-12T03:16:40.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.497+0000 7f76b5645640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f76b0071910 con 0x7f76b015e4a0 2024-09-12T03:16:40.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.497+0000 7f76b5645640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f76b0071bf0 con 0x7f76b00afff0 2024-09-12T03:16:40.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.497+0000 7f76aeffd640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f76b015e4a0 0x7f76b00b3ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:40.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.497+0000 7f76affff640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f76b00afff0 0x7f76b00726d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:40.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.497+0000 7f9ea09a9640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f9e9c0b65e0 0x7f9e9c162760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33162/0 (socket says 172.21.15.27:33162) 2024-09-12T03:16:40.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.497+0000 7f9ea09a9640 1 -- 172.21.15.27:0/1281771360 learned_addr learned my addr 172.21.15.27:0/1281771360 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:16:40.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.497+0000 7f76ae7fc640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f76b00b43e0 0x7f76b00af8e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:39182/0 (socket says 172.21.15.27:39182) 2024-09-12T03:16:40.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.497+0000 7f76ae7fc640 1 -- 172.21.15.27:0/2942081252 learned_addr learned my addr 172.21.15.27:0/2942081252 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:16:40.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.497+0000 7fe9af7fe640 1 -- 172.21.15.27:0/917348323 <== mon.2 v2:172.21.15.45:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9a0007cb0 con 0x7fe9b0073790 2024-09-12T03:16:40.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.497+0000 7f3c8f59e640 1 -- 172.21.15.27:0/3620354981 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f3c9006e980 msgr2=0x7f3c90162e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:40.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.497+0000 7f3c8f59e640 1 --2- 172.21.15.27:0/3620354981 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f3c9006e980 0x7f3c90162e10 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f3c88009d90 tx=0x7f3c8802a630 comp rx=0 tx=0).stop 2024-09-12T03:16:40.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.496+0000 7fc588961640 1 --2- 172.21.15.27:0/3507186999 >> v2:172.21.15.45:3300/0 conn(0x7fc5840d09b0 0x7fc584071480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:40.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.497+0000 7fc57bfff640 1 -- 172.21.15.27:0/3507186999 <== mon.1 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2051380365 0 0) 0x7fc5840b7af0 con 0x7fc5840719c0 2024-09-12T03:16:40.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.497+0000 7fc57bfff640 1 -- 172.21.15.27:0/3507186999 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc570003520 con 0x7fc5840719c0 2024-09-12T03:16:40.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.497+0000 7fc588961640 1 -- 172.21.15.27:0/3507186999 >> v2:172.21.15.27:3300/0 conn(0x7fc5840d1410 msgr2=0x7fc5840d0470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:40.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.497+0000 7fc588961640 1 --2- 172.21.15.27:0/3507186999 >> v2:172.21.15.27:3300/0 conn(0x7fc5840d1410 0x7fc5840d0470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:40.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.497+0000 7fc588961640 1 -- 172.21.15.27:0/3507186999 >> v1:172.21.15.45:6789/0 conn(0x7fc5840719c0 legacy=0x7fc584073280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:40.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.497+0000 7fc588961640 1 -- 172.21.15.27:0/3507186999 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5840b7810 con 0x7fc5840d09b0 2024-09-12T03:16:40.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.497+0000 7fc589162640 1 --2- 172.21.15.27:0/3507186999 >> v2:172.21.15.27:3300/0 conn(0x7fc5840d1410 0x7fc5840d0470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:16:40.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.498+0000 7f9ea19ab640 1 --2- 172.21.15.27:0/1281771360 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f9e9c166fd0 0x7f9e9c162ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:40.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.498+0000 7f8d85c7b640 1 -- 172.21.15.27:0/4253355514 shutdown_connections 2024-09-12T03:16:40.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.498+0000 7f76aeffd640 1 -- 172.21.15.27:0/2942081252 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f76b00afff0 msgr2=0x7f76b00726d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:40.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.498+0000 7f76aeffd640 1 --2- 172.21.15.27:0/2942081252 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f76b00afff0 0x7f76b00726d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:40.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.498+0000 7f76aeffd640 1 -- 172.21.15.27:0/2942081252 >> v1:172.21.15.27:6789/0 conn(0x7f76b00b43e0 legacy=0x7f76b00af8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:40.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.498+0000 7f76aeffd640 1 -- 172.21.15.27:0/2942081252 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76b0071ed0 con 0x7f76b015e4a0 2024-09-12T03:16:40.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.498+0000 7f76adffb640 1 -- 172.21.15.27:0/2942081252 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 269928809 0 0) 0x7f76b0071630 con 0x7f76b00b43e0 2024-09-12T03:16:40.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.498+0000 7f9e93fff640 1 -- 172.21.15.27:0/1281771360 <== mon.1 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1366834532 0 0) 0x7f9e9c071a00 con 0x7f9e9c0b6200 2024-09-12T03:16:40.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.498+0000 7f3c8f59e640 1 -- 172.21.15.27:0/3620354981 shutdown_connections 2024-09-12T03:16:40.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.498+0000 7f3c8f59e640 1 --2- 172.21.15.27:0/3620354981 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f3c9006e980 0x7f3c90162e10 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:40.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.498+0000 7f3c8f59e640 1 --2- 172.21.15.27:0/3620354981 >> v2:172.21.15.45:3300/0 conn(0x7f3c900cd520 0x7f3c90070fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:40.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.498+0000 7f3c8f59e640 1 -- 172.21.15.27:0/3620354981 >> 172.21.15.27:0/3620354981 conn(0x7f3c90062d50 msgr2=0x7f3c9006bb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:40.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.498+0000 7f9e93fff640 1 -- 172.21.15.27:0/1281771360 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9e88003520 con 0x7f9e9c0b6200 2024-09-12T03:16:40.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.498+0000 7f2f84ef8640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2f8015e7f0 0x7f2f800b5ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:40.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.498+0000 7fc588961640 1 --2- 172.21.15.27:0/3507186999 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc5840d09b0 0x7fc584071480 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fc584153c20 tx=0x7fc57402a1a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dfe52cb57e23431d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:40.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.498+0000 7f2f84ef8640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f2f800b6b90 0x7f2f800b6530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:40.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.498+0000 7f2f84ef8640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2f800b7980 con 0x7f2f800d20b0 2024-09-12T03:16:40.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.498+0000 7f2f84ef8640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f2f800d3f30 con 0x7f2f800b6b90 2024-09-12T03:16:40.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.498+0000 7f2f84ef8640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f2f800d33b0 con 0x7f2f8015e7f0 2024-09-12T03:16:40.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.498+0000 7f2f7ed9d640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f2f800d20b0 0x7f2f800b2f80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.27:38920/0 (socket says 172.21.15.27:38920) 2024-09-12T03:16:40.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.498+0000 7f2f7ed9d640 1 -- 172.21.15.27:0/428380219 learned_addr learned my addr 172.21.15.27:0/428380219 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:16:40.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.498+0000 7f2f7e59c640 1 --2- 172.21.15.27:0/428380219 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2f8015e7f0 0x7f2f800b5ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:40.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.499+0000 7f8d85c7b640 1 -- 172.21.15.27:0/4253355514 wait complete. 2024-09-12T03:16:40.519 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.498+0000 7fc57bfff640 1 -- 172.21.15.27:0/3507186999 <== mon.1 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc574037070 con 0x7fc5840d09b0 2024-09-12T03:16:40.519 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.499+0000 7fc57bfff640 1 -- 172.21.15.27:0/3507186999 <== mon.1 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc574032440 con 0x7fc5840d09b0 2024-09-12T03:16:40.519 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.499+0000 7fc57bfff640 1 -- 172.21.15.27:0/3507186999 <== mon.1 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc574032810 con 0x7fc5840d09b0 2024-09-12T03:16:40.519 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.498+0000 7f9ea19ab640 1 -- 172.21.15.27:0/1281771360 >> v2:172.21.15.27:3300/0 conn(0x7f9e9c0b65e0 msgr2=0x7f9e9c162760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:40.519 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.498+0000 7f9ea19ab640 1 --2- 172.21.15.27:0/1281771360 >> v2:172.21.15.27:3300/0 conn(0x7f9e9c0b65e0 0x7f9e9c162760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:40.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.498+0000 7f9ea19ab640 1 -- 172.21.15.27:0/1281771360 >> v1:172.21.15.45:6789/0 conn(0x7f9e9c0b6200 legacy=0x7f9e9c0b19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:40.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.498+0000 7f9ea19ab640 1 -- 172.21.15.27:0/1281771360 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e9c070f20 con 0x7f9e9c166fd0 2024-09-12T03:16:40.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.499+0000 7f2f7d59a640 1 -- 172.21.15.27:0/428380219 <== mon.1 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2858249041 0 0) 0x7f2f800b7980 con 0x7f2f800d20b0 2024-09-12T03:16:40.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.499+0000 7f2f7d59a640 1 -- 172.21.15.27:0/428380219 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2f68003520 con 0x7f2f800d20b0 2024-09-12T03:16:40.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.499+0000 7f76aeffd640 1 --2- 172.21.15.27:0/2942081252 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f76b015e4a0 0x7f76b00b3ea0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f76a00097a0 tx=0x7f76a002a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ef2c6230d9173ca2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:40.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.499+0000 7f2f7e59c640 1 -- 172.21.15.27:0/428380219 >> v2:172.21.15.45:3300/0 conn(0x7f2f800b6b90 msgr2=0x7f2f800b6530 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-12T03:16:40.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.499+0000 7f2f7e59c640 1 --2- 172.21.15.27:0/428380219 >> v2:172.21.15.45:3300/0 conn(0x7f2f800b6b90 0x7f2f800b6530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:40.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.499+0000 7f2f7e59c640 1 -- 172.21.15.27:0/428380219 >> v1:172.21.15.45:6789/0 conn(0x7f2f800d20b0 legacy=0x7f2f800b2f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:40.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.499+0000 7f2f7e59c640 1 -- 172.21.15.27:0/428380219 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f800d3690 con 0x7f2f8015e7f0 2024-09-12T03:16:40.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.498+0000 7fe9b6b98640 1 -- 172.21.15.27:0/917348323 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe9b0073790 msgr2=0x7fe9b006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:40.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.498+0000 7fe9b6b98640 1 --2- 172.21.15.27:0/917348323 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe9b0073790 0x7fe9b006e2f0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fe9a0009d90 tx=0x7fe9a002a660 comp rx=0 tx=0).stop 2024-09-12T03:16:40.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.498+0000 7f4a8d47a640 1 --2- 172.21.15.27:0/1156939991 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4a880b9780 0x7f4a880d2270 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f4a80009d90 tx=0x7f4a8002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f7ea19c874e80c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:40.522 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.499+0000 7f4a877fe640 1 -- 172.21.15.27:0/1156939991 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a70001670 con 0x7f4a880b9780 2024-09-12T03:16:40.522 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.499+0000 7f4a877fe640 1 -- 172.21.15.27:0/1156939991 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a80007540 con 0x7f4a880b9780 2024-09-12T03:16:40.522 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.499+0000 7f4a877fe640 1 -- 172.21.15.27:0/1156939991 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a70001670 con 0x7f4a880b9780 2024-09-12T03:16:40.522 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.499+0000 7f3c8f59e640 1 -- 172.21.15.27:0/3620354981 shutdown_connections 2024-09-12T03:16:40.522 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.499+0000 7f3c8f59e640 1 -- 172.21.15.27:0/3620354981 wait complete. 2024-09-12T03:16:40.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.499+0000 7f76adffb640 1 -- 172.21.15.27:0/2942081252 <== mon.1 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76a0037070 con 0x7f76b015e4a0 2024-09-12T03:16:40.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.499+0000 7f76adffb640 1 -- 172.21.15.27:0/2942081252 <== mon.1 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76a0032440 con 0x7f76b015e4a0 2024-09-12T03:16:40.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.499+0000 7f76adffb640 1 -- 172.21.15.27:0/2942081252 <== mon.1 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76a0032810 con 0x7f76b015e4a0 2024-09-12T03:16:40.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.499+0000 7fe9b6b98640 1 -- 172.21.15.27:0/917348323 shutdown_connections 2024-09-12T03:16:40.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.499+0000 7fe9b6b98640 1 --2- 172.21.15.27:0/917348323 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe9b0073790 0x7fe9b006e2f0 secure :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fe9a0009d90 tx=0x7fe9a002a660 comp rx=0 tx=0).stop 2024-09-12T03:16:40.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.499+0000 7fe9b6b98640 1 --2- 172.21.15.27:0/917348323 >> v2:172.21.15.45:3300/0 conn(0x7fe9b0162c00 0x7fe9b0073250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:40.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.499+0000 7fe9b6b98640 1 -- 172.21.15.27:0/917348323 >> 172.21.15.27:0/917348323 conn(0x7fe9b006f300 msgr2=0x7fe9b006f710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:40.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.500+0000 7fc58b3c6640 1 -- 172.21.15.27:0/3507186999 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc5840d09b0 msgr2=0x7fc584071480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:40.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.500+0000 7fe9b6b98640 1 -- 172.21.15.27:0/917348323 shutdown_connections 2024-09-12T03:16:40.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.500+0000 7fc58b3c6640 1 --2- 172.21.15.27:0/3507186999 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc5840d09b0 0x7fc584071480 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fc584153c20 tx=0x7fc57402a1a0 comp rx=0 tx=0).stop 2024-09-12T03:16:40.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.500+0000 7fc58b3c6640 1 -- 172.21.15.27:0/3507186999 shutdown_connections 2024-09-12T03:16:40.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.500+0000 7fc58b3c6640 1 --2- 172.21.15.27:0/3507186999 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc5840d09b0 0x7fc584071480 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:40.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.500+0000 7fc58b3c6640 1 --2- 172.21.15.27:0/3507186999 >> v2:172.21.15.27:3300/0 conn(0x7fc5840d1410 0x7fc5840d0470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:40.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.500+0000 7fc58b3c6640 1 -- 172.21.15.27:0/3507186999 >> 172.21.15.27:0/3507186999 conn(0x7fc5840b4320 msgr2=0x7fc5840b4700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:40.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.500+0000 7f9ea19ab640 1 --2- 172.21.15.27:0/1281771360 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f9e9c166fd0 0x7f9e9c162ca0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f9e980097a0 tx=0x7f9e9802a0f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bb55bc0c86058e76 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:40.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.500+0000 7f4a8dc7b640 1 -- 172.21.15.27:0/1156939991 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4a880b9780 msgr2=0x7f4a880d2270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:40.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.500+0000 7f4a8dc7b640 1 --2- 172.21.15.27:0/1156939991 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4a880b9780 0x7f4a880d2270 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f4a80009d90 tx=0x7f4a8002a6a0 comp rx=0 tx=0).stop 2024-09-12T03:16:40.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.500+0000 7f2f7e59c640 1 --2- 172.21.15.27:0/428380219 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2f8015e7f0 0x7f2f800b5ff0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f2f700097a0 tx=0x7f2f7002a1a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=547e432888de8079 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:40.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.500+0000 7fe9b6b98640 1 -- 172.21.15.27:0/917348323 wait complete. 2024-09-12T03:16:40.528 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.500+0000 7fc58b3c6640 1 -- 172.21.15.27:0/3507186999 shutdown_connections 2024-09-12T03:16:40.528 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.500+0000 7f9e93fff640 1 -- 172.21.15.27:0/1281771360 <== mon.2 v2:172.21.15.45:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e98037070 con 0x7f9e9c166fd0 2024-09-12T03:16:40.528 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.500+0000 7f9e93fff640 1 -- 172.21.15.27:0/1281771360 <== mon.2 v2:172.21.15.45:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e9802ae40 con 0x7f9e9c166fd0 2024-09-12T03:16:40.528 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.500+0000 7f9e93fff640 1 -- 172.21.15.27:0/1281771360 <== mon.2 v2:172.21.15.45:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e98032660 con 0x7f9e9c166fd0 2024-09-12T03:16:40.528 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.500+0000 7f8d85c7b640 1 Processor -- start 2024-09-12T03:16:40.529 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.500+0000 7f76b5645640 1 -- 172.21.15.27:0/2942081252 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f76b015e4a0 msgr2=0x7f76b00b3ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:40.529 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.500+0000 7f76b5645640 1 --2- 172.21.15.27:0/2942081252 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f76b015e4a0 0x7f76b00b3ea0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f76a00097a0 tx=0x7f76a002a180 comp rx=0 tx=0).stop 2024-09-12T03:16:40.529 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.500+0000 7f2f7d59a640 1 -- 172.21.15.27:0/428380219 <== mon.2 v2:172.21.15.45:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f70037070 con 0x7f2f8015e7f0 2024-09-12T03:16:40.529 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.500+0000 7f2f7d59a640 1 -- 172.21.15.27:0/428380219 <== mon.2 v2:172.21.15.45:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f70032440 con 0x7f2f8015e7f0 2024-09-12T03:16:40.529 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.500+0000 7f2f7d59a640 1 -- 172.21.15.27:0/428380219 <== mon.2 v2:172.21.15.45:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f70032810 con 0x7f2f8015e7f0 2024-09-12T03:16:40.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.500+0000 7f76b5645640 1 -- 172.21.15.27:0/2942081252 shutdown_connections 2024-09-12T03:16:40.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.500+0000 7f76b5645640 1 --2- 172.21.15.27:0/2942081252 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f76b00afff0 0x7f76b00726d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:40.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.500+0000 7f76b5645640 1 --2- 172.21.15.27:0/2942081252 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f76b015e4a0 0x7f76b00b3ea0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:40.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.500+0000 7f76b5645640 1 -- 172.21.15.27:0/2942081252 >> 172.21.15.27:0/2942081252 conn(0x7f76b00b2570 msgr2=0x7f76b00b2980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:40.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.501+0000 7f76b5645640 1 -- 172.21.15.27:0/2942081252 shutdown_connections 2024-09-12T03:16:40.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.501+0000 7f3c8f59e640 1 Processor -- start 2024-09-12T03:16:40.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.501+0000 7f76b5645640 1 -- 172.21.15.27:0/2942081252 wait complete. 2024-09-12T03:16:40.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.501+0000 7f3c8f59e640 1 -- start start 2024-09-12T03:16:40.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.501+0000 7fc58b3c6640 1 -- 172.21.15.27:0/3507186999 wait complete. 2024-09-12T03:16:40.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.501+0000 7f8d85c7b640 1 -- start start 2024-09-12T03:16:40.532 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.500+0000 7f4a8dc7b640 1 -- 172.21.15.27:0/1156939991 shutdown_connections 2024-09-12T03:16:40.532 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.500+0000 7f4a8dc7b640 1 --2- 172.21.15.27:0/1156939991 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4a880b9780 0x7f4a880d2270 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:40.532 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.500+0000 7f4a8dc7b640 1 --2- 172.21.15.27:0/1156939991 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4a880d4230 0x7f4a880b9240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:40.532 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.500+0000 7f4a8dc7b640 1 -- 172.21.15.27:0/1156939991 >> 172.21.15.27:0/1156939991 conn(0x7f4a880d46a0 msgr2=0x7f4a880d4ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:40.532 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.501+0000 7f4a8dc7b640 1 -- 172.21.15.27:0/1156939991 shutdown_connections 2024-09-12T03:16:40.532 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.501+0000 7f4a8dc7b640 1 -- 172.21.15.27:0/1156939991 wait complete. 2024-09-12T03:16:40.532 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.502+0000 7f2f84ef8640 1 -- 172.21.15.27:0/428380219 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2f8015e7f0 msgr2=0x7f2f800b5ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:40.532 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.502+0000 7f2f84ef8640 1 --2- 172.21.15.27:0/428380219 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2f8015e7f0 0x7f2f800b5ff0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f2f700097a0 tx=0x7f2f7002a1a0 comp rx=0 tx=0).stop 2024-09-12T03:16:40.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.501+0000 7f9ea21ac640 1 -- 172.21.15.27:0/1281771360 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f9e9c166fd0 msgr2=0x7f9e9c162ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:40.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.501+0000 7f9ea21ac640 1 --2- 172.21.15.27:0/1281771360 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f9e9c166fd0 0x7f9e9c162ca0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f9e980097a0 tx=0x7f9e9802a0f0 comp rx=0 tx=0).stop 2024-09-12T03:16:40.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.501+0000 7f9ea21ac640 1 -- 172.21.15.27:0/1281771360 shutdown_connections 2024-09-12T03:16:40.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.501+0000 7f9ea21ac640 1 --2- 172.21.15.27:0/1281771360 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f9e9c166fd0 0x7f9e9c162ca0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:40.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.501+0000 7f9ea21ac640 1 --2- 172.21.15.27:0/1281771360 >> v2:172.21.15.27:3300/0 conn(0x7f9e9c0b65e0 0x7f9e9c162760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:40.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.501+0000 7f9ea21ac640 1 -- 172.21.15.27:0/1281771360 >> 172.21.15.27:0/1281771360 conn(0x7f9e9c06f320 msgr2=0x7f9e9c06f730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:40.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.502+0000 7fe9b6b98640 1 Processor -- start 2024-09-12T03:16:40.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.502+0000 7f2f84ef8640 1 -- 172.21.15.27:0/428380219 shutdown_connections 2024-09-12T03:16:40.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.502+0000 7f2f84ef8640 1 --2- 172.21.15.27:0/428380219 >> v2:172.21.15.45:3300/0 conn(0x7f2f800b6b90 0x7f2f800b6530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:40.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.502+0000 7f2f84ef8640 1 --2- 172.21.15.27:0/428380219 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2f8015e7f0 0x7f2f800b5ff0 secure :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f2f700097a0 tx=0x7f2f7002a1a0 comp rx=0 tx=0).stop 2024-09-12T03:16:40.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.502+0000 7f9ea21ac640 1 -- 172.21.15.27:0/1281771360 shutdown_connections 2024-09-12T03:16:40.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.502+0000 7f2f84ef8640 1 -- 172.21.15.27:0/428380219 >> 172.21.15.27:0/428380219 conn(0x7f2f800d4570 msgr2=0x7f2f800d4980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:40.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.502+0000 7f8d85c7b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8d8006d010 0x7f8d801ff9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:40.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.502+0000 7f8d85c7b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8d8006d3f0 0x7f8d801f6cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:40.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.502+0000 7f8d85c7b640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f8d80072930 0x7f8d801f71f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:40.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.502+0000 7fe9b6b98640 1 -- start start 2024-09-12T03:16:40.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.502+0000 7f3c8f59e640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f3c9006e980 0x7f3c90157a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:40.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.502+0000 7f8d85c7b640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f8d8006d8a0 con 0x7f8d8006d3f0 2024-09-12T03:16:40.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.502+0000 7f8d85c7b640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f8d80072410 con 0x7f8d8006d010 2024-09-12T03:16:40.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.502+0000 7f8d85c7b640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f8d8006dd80 con 0x7f8d80072930 2024-09-12T03:16:40.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.502+0000 7f8d7f7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8d8006d010 0x7f8d801ff9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:40.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.502+0000 7f3c8f59e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3c900cd140 0x7f3c901590b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:40.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.502+0000 7f3c8f59e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3c900cd520 0x7f3c901595f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:40.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.502+0000 7f8d7f7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8d8006d010 0x7f8d801ff9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.27:60832/0 (socket says 172.21.15.27:60832) 2024-09-12T03:16:40.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.502+0000 7f8d7f7fe640 1 -- 172.21.15.27:0/1022859362 learned_addr learned my addr 172.21.15.27:0/1022859362 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:16:40.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.502+0000 7f8d7ffff640 1 --2- 172.21.15.27:0/1022859362 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f8d80072930 0x7f8d801f71f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:40.538 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.502+0000 7f3c8e59c640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f3c9006e980 0x7f3c90157a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:40.538 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.502+0000 7f3c8e59c640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f3c9006e980 0x7f3c90157a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3301/0 says I am v2:172.21.15.27:56288/0 (socket says 172.21.15.27:56288) 2024-09-12T03:16:40.538 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.503+0000 7fc58b3c6640 1 Processor -- start 2024-09-12T03:16:40.538 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.503+0000 7f2f84ef8640 1 -- 172.21.15.27:0/428380219 shutdown_connections 2024-09-12T03:16:40.538 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.503+0000 7f8d7ffff640 1 -- 172.21.15.27:0/1022859362 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8d8006d010 msgr2=0x7f8d801ff9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:40.539 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.503+0000 7f8d7ffff640 1 --2- 172.21.15.27:0/1022859362 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8d8006d010 0x7f8d801ff9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:40.539 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.503+0000 7f8d7ffff640 1 -- 172.21.15.27:0/1022859362 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8d8006d3f0 msgr2=0x7f8d801f6cb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-12T03:16:40.539 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.503+0000 7f3c8ed9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3c900cd520 0x7f3c901595f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:40.539 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.503+0000 7f3c8dd9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3c900cd140 0x7f3c901590b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:40.539 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.503+0000 7f3c8dd9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3c900cd140 0x7f3c901590b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.27:60836/0 (socket says 172.21.15.27:60836) 2024-09-12T03:16:40.539 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.503+0000 7f76b5645640 1 Processor -- start 2024-09-12T03:16:40.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.503+0000 7f3c8ed9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3c900cd520 0x7f3c901595f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33176/0 (socket says 172.21.15.27:33176) 2024-09-12T03:16:40.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.503+0000 7f3c8e59c640 1 -- 172.21.15.27:0/2937087087 learned_addr learned my addr 172.21.15.27:0/2937087087 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:16:40.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.503+0000 7f3c8f59e640 1 -- 172.21.15.27:0/2937087087 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f3c90167280 con 0x7f3c900cd520 2024-09-12T03:16:40.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.503+0000 7f76b5645640 1 -- start start 2024-09-12T03:16:40.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.503+0000 7f3c8f59e640 1 -- 172.21.15.27:0/2937087087 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f3c90166fa0 con 0x7f3c900cd140 2024-09-12T03:16:40.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.503+0000 7fc58b3c6640 1 -- start start 2024-09-12T03:16:40.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.503+0000 7f3c8f59e640 1 -- 172.21.15.27:0/2937087087 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f3c90167780 con 0x7f3c9006e980 2024-09-12T03:16:40.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.503+0000 7f8d7effd640 1 --2- 172.21.15.27:0/1022859362 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8d8006d3f0 0x7f8d801f6cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:40.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.502+0000 7f9ea21ac640 1 -- 172.21.15.27:0/1281771360 wait complete. 2024-09-12T03:16:40.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.503+0000 7f8d7ffff640 1 --2- 172.21.15.27:0/1022859362 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8d8006d3f0 0x7f8d801f6cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:40.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.503+0000 7f8d7ffff640 1 -- 172.21.15.27:0/1022859362 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d801f79a0 con 0x7f8d80072930 2024-09-12T03:16:40.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.503+0000 7f8d7effd640 1 --2- 172.21.15.27:0/1022859362 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8d8006d3f0 0x7f8d801f6cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:16:40.544 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.504+0000 7f4a8dc7b640 1 Processor -- start 2024-09-12T03:16:40.544 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.504+0000 7fe9b6b98640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe9b006e830 0x7fe9b00cd0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:40.544 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.504+0000 7f9ea21ac640 1 Processor -- start 2024-09-12T03:16:40.545 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.504+0000 7f3c8e59c640 1 -- 172.21.15.27:0/2937087087 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3c900cd140 msgr2=0x7f3c901590b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:40.545 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.504+0000 7f3c8e59c640 1 --2- 172.21.15.27:0/2937087087 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3c900cd140 0x7f3c901590b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:40.545 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.504+0000 7f3c8e59c640 1 -- 172.21.15.27:0/2937087087 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3c900cd520 msgr2=0x7f3c901595f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:40.545 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.504+0000 7fe9b6b98640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe9b0162c00 0x7fe9b0150730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:40.545 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.504+0000 7f3c8e59c640 1 --2- 172.21.15.27:0/2937087087 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3c900cd520 0x7f3c901595f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:40.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.504+0000 7f4a8dc7b640 1 -- start start 2024-09-12T03:16:40.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.504+0000 7f8d7ffff640 1 --2- 172.21.15.27:0/1022859362 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f8d80072930 0x7f8d801f71f0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f8d7800ca70 tx=0x7f8d7800cf30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:40.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.504+0000 7fe9b6b98640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe9b0166420 0x7fe9b01668d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:40.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.504+0000 7f9ea21ac640 1 -- start start 2024-09-12T03:16:40.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.504+0000 7fe9b4934640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe9b006e830 0x7fe9b00cd0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:40.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.504+0000 7fe9b4934640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe9b006e830 0x7fe9b00cd0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.27:60850/0 (socket says 172.21.15.27:60850) 2024-09-12T03:16:40.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.504+0000 7f2f84ef8640 1 -- 172.21.15.27:0/428380219 wait complete. 2024-09-12T03:16:40.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.504+0000 7fe9b4934640 1 -- 172.21.15.27:0/1405050804 learned_addr learned my addr 172.21.15.27:0/1405050804 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:16:40.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.504+0000 7fe9affff640 1 --2- 172.21.15.27:0/1405050804 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe9b0162c00 0x7fe9b0150730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:40.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.504+0000 7f8d67fff640 1 -- 172.21.15.27:0/1022859362 <== mon.2 v2:172.21.15.45:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d78017070 con 0x7f8d80072930 2024-09-12T03:16:40.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.504+0000 7f3c8e59c640 1 -- 172.21.15.27:0/2937087087 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c90157f80 con 0x7f3c9006e980 2024-09-12T03:16:40.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.504+0000 7fe9b5135640 1 --2- 172.21.15.27:0/1405050804 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe9b0166420 0x7fe9b01668d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:40.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.504+0000 7fc58b3c6640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc5840719c0 0x7fc58414b830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:40.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.504+0000 7fc58b3c6640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc5840d09b0 0x7fc5840ad0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:40.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.504+0000 7fc58b3c6640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fc5840d1410 0x7fc58414a140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:40.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.504+0000 7f3c8ed9d640 1 --2- 172.21.15.27:0/2937087087 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3c900cd520 0x7f3c901595f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:16:40.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.505+0000 7fc58b3c6640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fc5840b6e90 con 0x7fc5840d09b0 2024-09-12T03:16:40.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.505+0000 7f76b5645640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f76b00afff0 0x7f76b00cf850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:40.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.505+0000 7fc58b3c6640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc5840d17f0 con 0x7fc5840719c0 2024-09-12T03:16:40.551 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.504+0000 7f8d85c7b640 1 -- 172.21.15.27:0/1022859362 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d801f5ed0 con 0x7f8d80072930 2024-09-12T03:16:40.551 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.504+0000 7f8d85c7b640 1 -- 172.21.15.27:0/1022859362 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d801ff1d0 con 0x7f8d80072930 2024-09-12T03:16:40.551 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.505+0000 7fc58b3c6640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fc5840b7010 con 0x7fc5840d1410 2024-09-12T03:16:40.551 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.505+0000 7f8d67fff640 1 -- 172.21.15.27:0/1022859362 <== mon.2 v2:172.21.15.45:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d7800ab70 con 0x7f8d80072930 2024-09-12T03:16:40.552 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.505+0000 7fe9b6b98640 1 -- 172.21.15.27:0/1405050804 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fe9b00d1650 con 0x7fe9b0162c00 2024-09-12T03:16:40.552 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.505+0000 7fe9b6b98640 1 -- 172.21.15.27:0/1405050804 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe9b00b6630 con 0x7fe9b006e830 2024-09-12T03:16:40.552 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.505+0000 7f8d67fff640 1 -- 172.21.15.27:0/1022859362 <== mon.2 v2:172.21.15.45:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d78012400 con 0x7f8d80072930 2024-09-12T03:16:40.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.505+0000 7fe9b6b98640 1 -- 172.21.15.27:0/1405050804 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fe9b00d1930 con 0x7fe9b0166420 2024-09-12T03:16:40.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.505+0000 7fc589963640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fc5840d1410 0x7fc58414a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:40.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.505+0000 7f3c8e59c640 1 --2- 172.21.15.27:0/2937087087 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f3c9006e980 0x7f3c90157a40 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f3c8000c460 tx=0x7f3c8000c920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:40.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.505+0000 7f4a8dc7b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4a8806c160 0x7f4a8815de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:40.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.505+0000 7f9ea21ac640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9e9c0b6200 0x7f9e9c15c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:40.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.505+0000 7fc589162640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc5840719c0 0x7fc58414b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:40.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.505+0000 7fc589162640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc5840719c0 0x7fc58414b830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.27:60860/0 (socket says 172.21.15.27:60860) 2024-09-12T03:16:40.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.505+0000 7fc589162640 1 -- 172.21.15.27:0/666499063 learned_addr learned my addr 172.21.15.27:0/666499063 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:16:40.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.505+0000 7f4a8dc7b640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4a880d4230 0x7f4a880cd140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:40.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.505+0000 7f4a8cc79640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4a8806c160 0x7f4a8815de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:40.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.505+0000 7f9ea21ac640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f9e9c0b65e0 0x7f9e9c0d1850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:40.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.505+0000 7f9ea21ac640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9e9c166fd0 0x7f9e9c166250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:40.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.505+0000 7f76b5645640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f76b00b43e0 0x7f76b015ea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:40.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.505+0000 7f76b5645640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f76b015e4a0 0x7f76b015efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:40.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.505+0000 7f76b5645640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f76b006d4f0 con 0x7f76b00afff0 2024-09-12T03:16:40.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.505+0000 7f76b5645640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f76b015e880 con 0x7f76b015e4a0 2024-09-12T03:16:40.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.505+0000 7f76b5645640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f76b006d670 con 0x7f76b00b43e0 2024-09-12T03:16:40.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.505+0000 7f76aeffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f76b00afff0 0x7f76b00cf850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:40.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.505+0000 7f76aeffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f76b00afff0 0x7f76b00cf850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33204/0 (socket says 172.21.15.27:33204) 2024-09-12T03:16:40.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.505+0000 7f76aeffd640 1 -- 172.21.15.27:0/15777388 learned_addr learned my addr 172.21.15.27:0/15777388 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:16:40.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.505+0000 7f4a8cc79640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4a8806c160 0x7f4a8815de30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.27:60868/0 (socket says 172.21.15.27:60868) 2024-09-12T03:16:40.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.506+0000 7f4a87fff640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4a880d4230 0x7f4a880cd140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:40.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.506+0000 7f4a8dc7b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4a8815af90 0x7f4a880cd680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:40.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.506+0000 7f9ea09a9640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f9e9c0b65e0 0x7f9e9c0d1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:40.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.506+0000 7f9ea09a9640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f9e9c0b65e0 0x7f9e9c0d1850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3301/0 says I am v2:172.21.15.27:56326/0 (socket says 172.21.15.27:56326) 2024-09-12T03:16:40.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.506+0000 7f9ea11aa640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9e9c0b6200 0x7f9e9c15c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:40.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.506+0000 7f9ea09a9640 1 -- 172.21.15.27:0/2997252703 learned_addr learned my addr 172.21.15.27:0/2997252703 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:16:40.558 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.506+0000 7f9ea21ac640 1 -- 172.21.15.27:0/2997252703 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f9e9c06dc50 con 0x7f9e9c0b6200 2024-09-12T03:16:40.558 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.506+0000 7f9ea21ac640 1 -- 172.21.15.27:0/2997252703 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f9e9c0714e0 con 0x7f9e9c166fd0 2024-09-12T03:16:40.558 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.506+0000 7f9ea21ac640 1 -- 172.21.15.27:0/2997252703 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f9e9c1631e0 con 0x7f9e9c0b65e0 2024-09-12T03:16:40.558 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.506+0000 7f9ea19ab640 1 --2- 172.21.15.27:0/2997252703 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9e9c166fd0 0x7f9e9c166250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:40.558 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.506+0000 7f4a87fff640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4a880d4230 0x7f4a880cd140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3301/0 says I am v2:172.21.15.27:56316/0 (socket says 172.21.15.27:56316) 2024-09-12T03:16:40.558 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.506+0000 7f4a8cc79640 1 -- 172.21.15.27:0/273501929 learned_addr learned my addr 172.21.15.27:0/273501929 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:16:40.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.506+0000 7f76affff640 1 --2- 172.21.15.27:0/15777388 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f76b015e4a0 0x7f76b015efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:40.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.506+0000 7f4a8dc7b640 1 -- 172.21.15.27:0/273501929 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f4a880d0c40 con 0x7f4a8815af90 2024-09-12T03:16:40.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.506+0000 7f4a8dc7b640 1 -- 172.21.15.27:0/273501929 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f4a880d0ac0 con 0x7f4a8806c160 2024-09-12T03:16:40.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.506+0000 7f4a8dc7b640 1 -- 172.21.15.27:0/273501929 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f4a8806c540 con 0x7f4a880d4230 2024-09-12T03:16:40.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.505+0000 7fe9affff640 1 -- 172.21.15.27:0/1405050804 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe9b0166420 msgr2=0x7fe9b01668d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:40.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.505+0000 7fe9affff640 1 --2- 172.21.15.27:0/1405050804 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe9b0166420 0x7fe9b01668d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:40.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.505+0000 7fe9affff640 1 -- 172.21.15.27:0/1405050804 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe9b006e830 msgr2=0x7fe9b00cd0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:40.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.505+0000 7fe9affff640 1 --2- 172.21.15.27:0/1405050804 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe9b006e830 0x7fe9b00cd0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:40.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.505+0000 7fe9affff640 1 -- 172.21.15.27:0/1405050804 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9b00cc020 con 0x7fe9b0162c00 2024-09-12T03:16:40.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.505+0000 7fe9affff640 1 --2- 172.21.15.27:0/1405050804 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe9b0162c00 0x7fe9b0150730 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fe9a0031000 tx=0x7fe9a0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:40.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.507+0000 7fe9ad7fa640 1 -- 172.21.15.27:0/1405050804 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9a0040070 con 0x7fe9b0162c00 2024-09-12T03:16:40.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.507+0000 7fe9ad7fa640 1 -- 172.21.15.27:0/1405050804 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9a0031bc0 con 0x7fe9b0162c00 2024-09-12T03:16:40.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.506+0000 7f3c7effd640 1 -- 172.21.15.27:0/2937087087 <== mon.2 v2:172.21.15.45:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c80005ce0 con 0x7f3c9006e980 2024-09-12T03:16:40.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.506+0000 7f3c7effd640 1 -- 172.21.15.27:0/2937087087 <== mon.2 v2:172.21.15.45:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c80007070 con 0x7f3c9006e980 2024-09-12T03:16:40.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.506+0000 7f76ae7fc640 1 --2- 172.21.15.27:0/15777388 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f76b00b43e0 0x7f76b015ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:40.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.506+0000 7f76aeffd640 1 -- 172.21.15.27:0/15777388 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f76b00b43e0 msgr2=0x7f76b015ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:40.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.506+0000 7f76aeffd640 1 --2- 172.21.15.27:0/15777388 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f76b00b43e0 0x7f76b015ea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:40.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.506+0000 7f76aeffd640 1 -- 172.21.15.27:0/15777388 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f76b015e4a0 msgr2=0x7f76b015efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:40.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.506+0000 7f76aeffd640 1 --2- 172.21.15.27:0/15777388 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f76b015e4a0 0x7f76b015efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:40.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.506+0000 7f76aeffd640 1 -- 172.21.15.27:0/15777388 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76b00cfd90 con 0x7f76b00afff0 2024-09-12T03:16:40.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.506+0000 7f4a8d47a640 1 --2- 172.21.15.27:0/273501929 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4a8815af90 0x7f4a880cd680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:40.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.507+0000 7f4a8cc79640 1 -- 172.21.15.27:0/273501929 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4a880d4230 msgr2=0x7f4a880cd140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:40.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.507+0000 7f4a8cc79640 1 --2- 172.21.15.27:0/273501929 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4a880d4230 0x7f4a880cd140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:40.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.507+0000 7f4a8cc79640 1 -- 172.21.15.27:0/273501929 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4a8815af90 msgr2=0x7f4a880cd680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:40.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.507+0000 7f4a8cc79640 1 --2- 172.21.15.27:0/273501929 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4a8815af90 0x7f4a880cd680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:40.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.507+0000 7f4a8cc79640 1 -- 172.21.15.27:0/273501929 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a881f8b30 con 0x7f4a8806c160 2024-09-12T03:16:40.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.507+0000 7f4a8d47a640 1 --2- 172.21.15.27:0/273501929 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4a8815af90 0x7f4a880cd680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:16:40.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.507+0000 7f4a8cc79640 1 --2- 172.21.15.27:0/273501929 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4a8806c160 0x7f4a8815de30 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f4a7000cac0 tx=0x7f4a7000cf80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:40.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.506+0000 7f2f84ef8640 1 Processor -- start 2024-09-12T03:16:40.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.506+0000 7f2f84ef8640 1 -- start start 2024-09-12T03:16:40.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.507+0000 7f9ea11aa640 1 -- 172.21.15.27:0/2997252703 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f9e9c0b65e0 msgr2=0x7f9e9c0d1850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:40.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.507+0000 7f9ea11aa640 1 --2- 172.21.15.27:0/2997252703 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f9e9c0b65e0 0x7f9e9c0d1850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:40.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.507+0000 7f9ea11aa640 1 -- 172.21.15.27:0/2997252703 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9e9c166fd0 msgr2=0x7f9e9c166250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:40.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.507+0000 7f9ea11aa640 1 --2- 172.21.15.27:0/2997252703 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9e9c166fd0 0x7f9e9c166250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:40.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.507+0000 7f9ea11aa640 1 -- 172.21.15.27:0/2997252703 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e9c15cbb0 con 0x7f9e9c0b6200 2024-09-12T03:16:40.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.507+0000 7f9ea19ab640 1 --2- 172.21.15.27:0/2997252703 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9e9c166fd0 0x7f9e9c166250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:16:40.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.507+0000 7f9ea11aa640 1 --2- 172.21.15.27:0/2997252703 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9e9c0b6200 0x7f9e9c15c670 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f9e8c00c400 tx=0x7f9e8c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:40.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.507+0000 7f3c7effd640 1 -- 172.21.15.27:0/2937087087 <== mon.2 v2:172.21.15.45:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c80007460 con 0x7f3c9006e980 2024-09-12T03:16:40.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.507+0000 7f9e91ffb640 1 -- 172.21.15.27:0/2997252703 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e8c017070 con 0x7f9e9c0b6200 2024-09-12T03:16:40.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.507+0000 7f2f84ef8640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2f800b6b90 0x7f2f8015ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:40.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.507+0000 7f9e91ffb640 1 -- 172.21.15.27:0/2997252703 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e8c0052d0 con 0x7f9e9c0b6200 2024-09-12T03:16:40.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.506+0000 7fc588961640 1 --2- 172.21.15.27:0/666499063 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc5840d09b0 0x7fc5840ad0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:40.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.506+0000 7fc588961640 1 -- 172.21.15.27:0/666499063 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fc5840d1410 msgr2=0x7fc58414a140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:40.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.506+0000 7fc588961640 1 --2- 172.21.15.27:0/666499063 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fc5840d1410 0x7fc58414a140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:40.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.506+0000 7fc588961640 1 -- 172.21.15.27:0/666499063 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc5840719c0 msgr2=0x7fc58414b830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:40.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.506+0000 7fc588961640 1 --2- 172.21.15.27:0/666499063 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc5840719c0 0x7fc58414b830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:40.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.506+0000 7fc588961640 1 -- 172.21.15.27:0/666499063 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc58414be20 con 0x7fc5840d09b0 2024-09-12T03:16:40.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.506+0000 7fc589963640 1 --2- 172.21.15.27:0/666499063 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fc5840d1410 0x7fc58414a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:16:40.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.506+0000 7fc588961640 1 --2- 172.21.15.27:0/666499063 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc5840d09b0 0x7fc5840ad0c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fc574031000 tx=0x7fc5740046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:40.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.507+0000 7fc579ffb640 1 -- 172.21.15.27:0/666499063 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc574037050 con 0x7fc5840d09b0 2024-09-12T03:16:40.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.507+0000 7fc579ffb640 1 -- 172.21.15.27:0/666499063 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc574043070 con 0x7fc5840d09b0 2024-09-12T03:16:40.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.507+0000 7fc579ffb640 1 -- 172.21.15.27:0/666499063 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc574030b70 con 0x7fc5840d09b0 2024-09-12T03:16:40.567 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.507+0000 7fc58b3c6640 1 -- 172.21.15.27:0/666499063 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc58414c100 con 0x7fc5840d09b0 2024-09-12T03:16:40.567 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.507+0000 7fc58b3c6640 1 -- 172.21.15.27:0/666499063 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc58414c6c0 con 0x7fc5840d09b0 2024-09-12T03:16:40.567 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.508+0000 7f2f7e59c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2f800b6b90 0x7f2f8015ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:40.567 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.508+0000 7f2f7e59c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2f800b6b90 0x7f2f8015ae80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33244/0 (socket says 172.21.15.27:33244) 2024-09-12T03:16:40.567 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.508+0000 7f2f7e59c640 1 -- 172.21.15.27:0/1763082757 learned_addr learned my addr 172.21.15.27:0/1763082757 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:16:40.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.508+0000 7f2f84ef8640 1 --2- 172.21.15.27:0/1763082757 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2f800d20b0 0x7f2f80071270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:40.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.508+0000 7f2f84ef8640 1 --2- 172.21.15.27:0/1763082757 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2f8015b3c0 0x7f2f800717b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:40.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.508+0000 7f2f84ef8640 1 -- 172.21.15.27:0/1763082757 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f2f8015ebd0 con 0x7f2f800b6b90 2024-09-12T03:16:40.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.508+0000 7f2f84ef8640 1 -- 172.21.15.27:0/1763082757 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f2f800d3c50 con 0x7f2f8015b3c0 2024-09-12T03:16:40.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.507+0000 7f76aeffd640 1 --2- 172.21.15.27:0/15777388 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f76b00afff0 0x7f76b00cf850 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f76a00098c0 tx=0x7f76a00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:40.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.508+0000 7f2f84ef8640 1 -- 172.21.15.27:0/1763082757 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f2f8015c8f0 con 0x7f2f800d20b0 2024-09-12T03:16:40.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.508+0000 7f3c8f59e640 1 -- 172.21.15.27:0/2937087087 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c90158490 con 0x7f3c9006e980 2024-09-12T03:16:40.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.508+0000 7fe9ad7fa640 1 -- 172.21.15.27:0/1405050804 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9a0045460 con 0x7fe9b0162c00 2024-09-12T03:16:40.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.508+0000 7f9ea21ac640 1 -- 172.21.15.27:0/2997252703 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e9c15e7d0 con 0x7f9e9c0b6200 2024-09-12T03:16:40.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.508+0000 7fe9b6b98640 1 -- 172.21.15.27:0/1405050804 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9b00cc300 con 0x7fe9b0162c00 2024-09-12T03:16:40.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.508+0000 7f3c8f59e640 1 -- 172.21.15.27:0/2937087087 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c900aab70 con 0x7f3c9006e980 2024-09-12T03:16:40.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.508+0000 7f2f7dd9b640 1 --2- 172.21.15.27:0/1763082757 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2f800d20b0 0x7f2f80071270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:40.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.508+0000 7f2f7ed9d640 1 --2- 172.21.15.27:0/1763082757 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2f8015b3c0 0x7f2f800717b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:40.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.508+0000 7fe9b6b98640 1 -- 172.21.15.27:0/1405050804 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9b00cc8c0 con 0x7fe9b0162c00 2024-09-12T03:16:40.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.508+0000 7f9ea21ac640 1 -- 172.21.15.27:0/2997252703 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e9c15ed90 con 0x7f9e9c0b6200 2024-09-12T03:16:40.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.508+0000 7f9e91ffb640 1 -- 172.21.15.27:0/2997252703 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e8c017070 con 0x7f9e9c0b6200 2024-09-12T03:16:40.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.509+0000 7f8d85c7b640 1 -- 172.21.15.27:0/1022859362 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8d60001030 con 0x7f8d80072930 2024-09-12T03:16:40.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.508+0000 7f4a857fa640 1 -- 172.21.15.27:0/273501929 <== mon.1 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a70017070 con 0x7f4a8806c160 2024-09-12T03:16:40.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.508+0000 7f4a857fa640 1 -- 172.21.15.27:0/273501929 <== mon.1 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a70005ce0 con 0x7f4a8806c160 2024-09-12T03:16:40.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.508+0000 7f4a857fa640 1 -- 172.21.15.27:0/273501929 <== mon.1 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a70017070 con 0x7f4a8806c160 2024-09-12T03:16:40.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.509+0000 7f2f7dd9b640 1 -- 172.21.15.27:0/1763082757 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2f8015b3c0 msgr2=0x7f2f800717b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:40.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.509+0000 7f2f7dd9b640 1 --2- 172.21.15.27:0/1763082757 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2f8015b3c0 0x7f2f800717b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:40.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.509+0000 7f4a8dc7b640 1 -- 172.21.15.27:0/273501929 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a881f8e10 con 0x7f4a8806c160 2024-09-12T03:16:40.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.509+0000 7f2f7dd9b640 1 -- 172.21.15.27:0/1763082757 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2f800b6b90 msgr2=0x7f2f8015ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:40.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.509+0000 7f2f7dd9b640 1 --2- 172.21.15.27:0/1763082757 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2f800b6b90 0x7f2f8015ae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:40.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.509+0000 7f2f7dd9b640 1 -- 172.21.15.27:0/1763082757 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f80156070 con 0x7f2f800d20b0 2024-09-12T03:16:40.574 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.509+0000 7f4a8dc7b640 1 -- 172.21.15.27:0/273501929 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a881f93d0 con 0x7f4a8806c160 2024-09-12T03:16:40.574 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.509+0000 7f8d67fff640 1 -- 172.21.15.27:0/1022859362 <== mon.2 v2:172.21.15.45:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8d7801a070 con 0x7f8d80072930 2024-09-12T03:16:40.574 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.509+0000 7f8d67fff640 1 --2- 172.21.15.27:0/1022859362 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f8d6c06bf00 0x7f8d6c06e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:40.574 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.510+0000 7f3c7cff9640 1 -- 172.21.15.27:0/2937087087 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f3c68001030 con 0x7f3c9006e980 2024-09-12T03:16:40.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.510+0000 7fe996ffd640 1 -- 172.21.15.27:0/1405050804 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe988001030 con 0x7fe9b0162c00 2024-09-12T03:16:40.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.509+0000 7fc58b3c6640 1 -- 172.21.15.27:0/666499063 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc584071480 con 0x7fc5840d09b0 2024-09-12T03:16:40.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.510+0000 7f8d7f7fe640 1 --2- 172.21.15.27:0/1022859362 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f8d6c06bf00 0x7f8d6c06e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:40.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.510+0000 7f8d67fff640 1 -- 172.21.15.27:0/1022859362 <== mon.2 v2:172.21.15.45:3301/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f8d7808ab80 con 0x7f8d80072930 2024-09-12T03:16:40.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.510+0000 7f2f7dd9b640 1 --2- 172.21.15.27:0/1763082757 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2f800d20b0 0x7f2f80071270 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f2f74009fd0 tx=0x7f2f7400b870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:40.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.510+0000 7f9ea21ac640 1 -- 172.21.15.27:0/2997252703 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9e9c0b19f0 con 0x7f9e9c0b6200 2024-09-12T03:16:40.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.510+0000 7f8d67fff640 1 --2- 172.21.15.27:0/1022859362 >> [v2:172.21.15.27:6800/80661598,v1:172.21.15.27:6801/80661598] conn(0x7f8d6c074120 0x7f8d6c076540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:40.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.510+0000 7f2f6effd640 1 -- 172.21.15.27:0/1763082757 <== mon.2 v2:172.21.15.45:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f74015020 con 0x7f2f800d20b0 2024-09-12T03:16:40.577 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.510+0000 7f8d67fff640 1 -- 172.21.15.27:0/1022859362 --> [v2:172.21.15.27:6800/80661598,v1:172.21.15.27:6801/80661598] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8d6c076bf0 con 0x7f8d6c074120 2024-09-12T03:16:40.577 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.510+0000 7f8d67fff640 1 -- 172.21.15.27:0/1022859362 <== mon.2 v2:172.21.15.45:3301/0 6 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f8d78091050 con 0x7f8d80072930 2024-09-12T03:16:40.577 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.510+0000 7f2f6effd640 1 -- 172.21.15.27:0/1763082757 <== mon.2 v2:172.21.15.45:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f74011850 con 0x7f2f800d20b0 2024-09-12T03:16:40.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.510+0000 7f8d7f7fe640 1 --2- 172.21.15.27:0/1022859362 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f8d6c06bf00 0x7f8d6c06e3c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f8d80200080 tx=0x7f8d70008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:40.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.511+0000 7fc579ffb640 1 -- 172.21.15.27:0/666499063 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc574030d10 con 0x7fc5840d09b0 2024-09-12T03:16:40.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.511+0000 7f8d7effd640 1 --2- 172.21.15.27:0/1022859362 >> [v2:172.21.15.27:6800/80661598,v1:172.21.15.27:6801/80661598] conn(0x7f8d6c074120 0x7f8d6c076540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:40.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.511+0000 7f2f6effd640 1 -- 172.21.15.27:0/1763082757 <== mon.2 v2:172.21.15.45:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f74011c40 con 0x7f2f800d20b0 2024-09-12T03:16:40.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.511+0000 7f2f84ef8640 1 -- 172.21.15.27:0/1763082757 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f8015b870 con 0x7f2f800d20b0 2024-09-12T03:16:40.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.511+0000 7f2f84ef8640 1 -- 172.21.15.27:0/1763082757 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f8015be10 con 0x7f2f800d20b0 2024-09-12T03:16:40.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.511+0000 7f8d7effd640 1 --2- 172.21.15.27:0/1022859362 >> [v2:172.21.15.27:6800/80661598,v1:172.21.15.27:6801/80661598] conn(0x7f8d6c074120 0x7f8d6c076540 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:40.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.511+0000 7fc579ffb640 1 --2- 172.21.15.27:0/666499063 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fc56806be30 0x7fc56806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:40.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.511+0000 7fc579ffb640 1 -- 172.21.15.27:0/666499063 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fc5740abb90 con 0x7fc5840d09b0 2024-09-12T03:16:40.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.511+0000 7fc579ffb640 1 --2- 172.21.15.27:0/666499063 >> [v2:172.21.15.27:6816/3002321488,v1:172.21.15.27:6817/3002321488] conn(0x7fc568073fd0 0x7fc5680763f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:40.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.511+0000 7fc579ffb640 1 -- 172.21.15.27:0/666499063 --> [v2:172.21.15.27:6816/3002321488,v1:172.21.15.27:6817/3002321488] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc574007e40 con 0x7fc568073fd0 2024-09-12T03:16:40.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.511+0000 7f3c7effd640 1 -- 172.21.15.27:0/2937087087 <== mon.2 v2:172.21.15.45:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3c80007600 con 0x7f3c9006e980 2024-09-12T03:16:40.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.512+0000 7f3c7effd640 1 --2- 172.21.15.27:0/2937087087 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f3c7006bf00 0x7f3c7006e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:40.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.512+0000 7fc579ffb640 1 -- 172.21.15.27:0/666499063 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fc5740760d0 con 0x7fc5840d09b0 2024-09-12T03:16:40.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.512+0000 7fc589963640 1 --2- 172.21.15.27:0/666499063 >> [v2:172.21.15.27:6816/3002321488,v1:172.21.15.27:6817/3002321488] conn(0x7fc568073fd0 0x7fc5680763f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:40.581 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.512+0000 7f3c7effd640 1 -- 172.21.15.27:0/2937087087 <== mon.2 v2:172.21.15.45:3301/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f3c8008aa00 con 0x7f3c9006e980 2024-09-12T03:16:40.581 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.510+0000 7f4a8dc7b640 1 -- 172.21.15.27:0/273501929 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4a880ca070 con 0x7f4a8806c160 2024-09-12T03:16:40.581 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.511+0000 7f4a857fa640 1 -- 172.21.15.27:0/273501929 <== mon.1 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4a7001a070 con 0x7f4a8806c160 2024-09-12T03:16:40.581 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.512+0000 7f4a857fa640 1 --2- 172.21.15.27:0/273501929 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f4a6806c230 0x7f4a6806e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:40.581 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.512+0000 7f4a857fa640 1 -- 172.21.15.27:0/273501929 <== mon.1 v2:172.21.15.45:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f4a70005ae0 con 0x7f4a8806c160 2024-09-12T03:16:40.581 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.512+0000 7f4a857fa640 1 --2- 172.21.15.27:0/273501929 >> [v2:172.21.15.45:6808/3620760330,v1:172.21.15.45:6809/3620760330] conn(0x7f4a680743d0 0x7f4a680767f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:40.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.512+0000 7f4a857fa640 1 -- 172.21.15.27:0/273501929 --> [v2:172.21.15.45:6808/3620760330,v1:172.21.15.45:6809/3620760330] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4a68076ea0 con 0x7f4a680743d0 2024-09-12T03:16:40.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.512+0000 7f4a857fa640 1 -- 172.21.15.27:0/273501929 <== mon.1 v2:172.21.15.45:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f4a7008c4e0 con 0x7f4a8806c160 2024-09-12T03:16:40.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.512+0000 7f76977fe640 1 -- 172.21.15.27:0/15777388 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76a0037040 con 0x7f76b00afff0 2024-09-12T03:16:40.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.512+0000 7f4a87fff640 1 --2- 172.21.15.27:0/273501929 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f4a6806c230 0x7f4a6806e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:40.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.512+0000 7f76b5645640 1 -- 172.21.15.27:0/15777388 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76b01610f0 con 0x7f76b00afff0 2024-09-12T03:16:40.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.512+0000 7fc589963640 1 --2- 172.21.15.27:0/666499063 >> [v2:172.21.15.27:6816/3002321488,v1:172.21.15.27:6817/3002321488] conn(0x7fc568073fd0 0x7fc5680763f0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:40.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.513+0000 7fc589162640 1 --2- 172.21.15.27:0/666499063 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fc56806be30 0x7fc56806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:40.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.512+0000 7f8d67fff640 1 -- 172.21.15.27:0/1022859362 <== osd.3 v2:172.21.15.27:6800/80661598 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8d6c076bf0 con 0x7f8d6c074120 2024-09-12T03:16:40.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.513+0000 7f4a8d47a640 1 --2- 172.21.15.27:0/273501929 >> [v2:172.21.15.45:6808/3620760330,v1:172.21.15.45:6809/3620760330] conn(0x7f4a680743d0 0x7f4a680767f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:40.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.513+0000 7f2f84ef8640 1 -- 172.21.15.27:0/1763082757 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f2f800d06f0 con 0x7f2f800d20b0 2024-09-12T03:16:40.584 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.513+0000 7f9e91ffb640 1 -- 172.21.15.27:0/2997252703 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9e8c01a070 con 0x7f9e9c0b6200 2024-09-12T03:16:40.584 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.513+0000 7f9e91ffb640 1 --2- 172.21.15.27:0/2997252703 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f9e8006c230 0x7f9e8006e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:40.584 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.513+0000 7f9ea09a9640 1 --2- 172.21.15.27:0/2997252703 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f9e8006c230 0x7f9e8006e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:40.584 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.513+0000 7f76977fe640 1 -- 172.21.15.27:0/15777388 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76a0042070 con 0x7f76b00afff0 2024-09-12T03:16:40.585 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.514+0000 7fc589162640 1 --2- 172.21.15.27:0/666499063 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fc56806be30 0x7fc56806e2f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc580006d40 tx=0x7fc580008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:40.585 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.514+0000 7f9e91ffb640 1 -- 172.21.15.27:0/2997252703 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f9e8c0050d0 con 0x7f9e9c0b6200 2024-09-12T03:16:40.585 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.514+0000 7f9e91ffb640 1 --2- 172.21.15.27:0/2997252703 >> [v2:172.21.15.45:6824/1513300294,v1:172.21.15.45:6825/1513300294] conn(0x7f9e80074f60 0x7f9e80077400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:40.585 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.514+0000 7f9e91ffb640 1 -- 172.21.15.27:0/2997252703 --> [v2:172.21.15.45:6824/1513300294,v1:172.21.15.45:6825/1513300294] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9e80077ad0 con 0x7f9e80074f60 2024-09-12T03:16:40.585 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.514+0000 7fc579ffb640 1 -- 172.21.15.27:0/666499063 <== osd.0 v2:172.21.15.27:6816/3002321488 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc574007e40 con 0x7fc568073fd0 2024-09-12T03:16:40.586 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.512+0000 7fe9ad7fa640 1 -- 172.21.15.27:0/1405050804 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe9a0039040 con 0x7fe9b0162c00 2024-09-12T03:16:40.586 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.513+0000 7fe9ad7fa640 1 --2- 172.21.15.27:0/1405050804 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fe99c06be80 0x7fe99c06e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:40.586 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.513+0000 7fe9ad7fa640 1 -- 172.21.15.27:0/1405050804 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fe9a00aca90 con 0x7fe9b0162c00 2024-09-12T03:16:40.586 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.513+0000 7fe9b4934640 1 --2- 172.21.15.27:0/1405050804 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fe99c06be80 0x7fe99c06e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:40.586 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.513+0000 7fe9ad7fa640 1 --2- 172.21.15.27:0/1405050804 >> [v2:172.21.15.27:6808/1613102886,v1:172.21.15.27:6809/1613102886] conn(0x7fe99c0740a0 0x7fe99c0764c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:40.586 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.513+0000 7fe9ad7fa640 1 -- 172.21.15.27:0/1405050804 --> [v2:172.21.15.27:6808/1613102886,v1:172.21.15.27:6809/1613102886] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe9a000ae80 con 0x7fe99c0740a0 2024-09-12T03:16:40.586 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.513+0000 7fe9ad7fa640 1 -- 172.21.15.27:0/1405050804 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fe9a00acf70 con 0x7fe9b0162c00 2024-09-12T03:16:40.586 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.514+0000 7fe9b5135640 1 --2- 172.21.15.27:0/1405050804 >> [v2:172.21.15.27:6808/1613102886,v1:172.21.15.27:6809/1613102886] conn(0x7fe99c0740a0 0x7fe99c0764c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:40.587 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.514+0000 7fe9b4934640 1 --2- 172.21.15.27:0/1405050804 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fe99c06be80 0x7fe99c06e340 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fe9a4009a10 tx=0x7fe9a4008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:40.587 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.514+0000 7f76b5645640 1 -- 172.21.15.27:0/15777388 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76b01616b0 con 0x7f76b00afff0 2024-09-12T03:16:40.587 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.514+0000 7fe9b5135640 1 --2- 172.21.15.27:0/1405050804 >> [v2:172.21.15.27:6808/1613102886,v1:172.21.15.27:6809/1613102886] conn(0x7fe99c0740a0 0x7fe99c0764c0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:40.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.512+0000 7f3c8dd9b640 1 --2- 172.21.15.27:0/2937087087 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f3c7006bf00 0x7f3c7006e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:40.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.512+0000 7f3c7effd640 1 --2- 172.21.15.27:0/2937087087 >> [v2:172.21.15.27:6824/3534723355,v1:172.21.15.27:6825/3534723355] conn(0x7f3c70074120 0x7f3c70076540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:40.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.512+0000 7f3c7effd640 1 -- 172.21.15.27:0/2937087087 --> [v2:172.21.15.27:6824/3534723355,v1:172.21.15.27:6825/3534723355] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3c80007070 con 0x7f3c70074120 2024-09-12T03:16:40.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.512+0000 7f3c7effd640 1 -- 172.21.15.27:0/2937087087 <== mon.2 v2:172.21.15.45:3301/0 6 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f3c8008f2a0 con 0x7f3c9006e980 2024-09-12T03:16:40.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.513+0000 7f3c8dd9b640 1 --2- 172.21.15.27:0/2937087087 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f3c7006bf00 0x7f3c7006e3c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f3c84006fd0 tx=0x7f3c84008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:40.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.514+0000 7f4a87fff640 1 --2- 172.21.15.27:0/273501929 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f4a6806c230 0x7f4a6806e6f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f4a880c8060 tx=0x7f4a78009190 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:40.589 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.514+0000 7f3c8ed9d640 1 --2- 172.21.15.27:0/2937087087 >> [v2:172.21.15.27:6824/3534723355,v1:172.21.15.27:6825/3534723355] conn(0x7f3c70074120 0x7f3c70076540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:40.589 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.514+0000 7f76977fe640 1 -- 172.21.15.27:0/15777388 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76a0030940 con 0x7f76b00afff0 2024-09-12T03:16:40.589 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.515+0000 7f4a8d47a640 1 --2- 172.21.15.27:0/273501929 >> [v2:172.21.15.45:6808/3620760330,v1:172.21.15.45:6809/3620760330] conn(0x7f4a680743d0 0x7f4a680767f0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:40.589 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.515+0000 7f3c8ed9d640 1 --2- 172.21.15.27:0/2937087087 >> [v2:172.21.15.27:6824/3534723355,v1:172.21.15.27:6825/3534723355] conn(0x7f3c70074120 0x7f3c70076540 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:40.590 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.515+0000 7f76957fa640 1 -- 172.21.15.27:0/15777388 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7688001030 con 0x7f76b00afff0 2024-09-12T03:16:40.591 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.514+0000 7f9ea19ab640 1 --2- 172.21.15.27:0/2997252703 >> [v2:172.21.15.45:6824/1513300294,v1:172.21.15.45:6825/1513300294] conn(0x7f9e80074f60 0x7f9e80077400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:40.591 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.514+0000 7f9ea09a9640 1 --2- 172.21.15.27:0/2997252703 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f9e8006c230 0x7f9e8006e6f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f9e9c0cdaa0 tx=0x7f9e9400b040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:40.591 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.515+0000 7f9e91ffb640 1 -- 172.21.15.27:0/2997252703 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f9e8c054810 con 0x7f9e9c0b6200 2024-09-12T03:16:40.591 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.515+0000 7f2f6effd640 1 -- 172.21.15.27:0/1763082757 <== mon.2 v2:172.21.15.45:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2f74018020 con 0x7f2f800d20b0 2024-09-12T03:16:40.591 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.515+0000 7f9ea19ab640 1 --2- 172.21.15.27:0/2997252703 >> [v2:172.21.15.45:6824/1513300294,v1:172.21.15.45:6825/1513300294] conn(0x7f9e80074f60 0x7f9e80077400 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:40.592 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.515+0000 7fe9ad7fa640 1 -- 172.21.15.27:0/1405050804 <== osd.1 v2:172.21.15.27:6808/1613102886 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe9a000ae80 con 0x7fe99c0740a0 2024-09-12T03:16:40.592 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.516+0000 7f4a857fa640 1 -- 172.21.15.27:0/273501929 <== osd.5 v2:172.21.15.45:6808/3620760330 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4a68076ea0 con 0x7f4a680743d0 2024-09-12T03:16:40.592 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.516+0000 7f2f6effd640 1 --2- 172.21.15.27:0/1763082757 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f2f6006bf00 0x7f2f6006e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:40.593 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.516+0000 7f3c7effd640 1 -- 172.21.15.27:0/2937087087 <== osd.2 v2:172.21.15.27:6824/3534723355 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3c80007070 con 0x7f3c70074120 2024-09-12T03:16:40.593 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.516+0000 7f2f7e59c640 1 --2- 172.21.15.27:0/1763082757 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f2f6006bf00 0x7f2f6006e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:40.593 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.516+0000 7f2f6effd640 1 -- 172.21.15.27:0/1763082757 <== mon.2 v2:172.21.15.45:3301/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f2f7408ae50 con 0x7f2f800d20b0 2024-09-12T03:16:40.593 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.517+0000 7f9e91ffb640 1 -- 172.21.15.27:0/2997252703 <== osd.7 v2:172.21.15.45:6824/1513300294 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9e80077ad0 con 0x7f9e80074f60 2024-09-12T03:16:40.594 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.516+0000 7f76977fe640 1 -- 172.21.15.27:0/15777388 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f76a0030ae0 con 0x7f76b00afff0 2024-09-12T03:16:40.594 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.517+0000 7f2f6effd640 1 --2- 172.21.15.27:0/1763082757 >> [v2:172.21.15.45:6800/1989083510,v1:172.21.15.45:6801/1989083510] conn(0x7f2f60074120 0x7f2f60076540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:40.594 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.517+0000 7f76977fe640 1 --2- 172.21.15.27:0/15777388 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f769c06bf00 0x7f769c06e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:40.594 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.517+0000 7f2f7e59c640 1 --2- 172.21.15.27:0/1763082757 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f2f6006bf00 0x7f2f6006e3c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f2f7002a1e0 tx=0x7f2f7002e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:40.595 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.517+0000 7f76977fe640 1 -- 172.21.15.27:0/15777388 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f76a00ab7b0 con 0x7f76b00afff0 2024-09-12T03:16:40.595 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.517+0000 7f2f7ed9d640 1 --2- 172.21.15.27:0/1763082757 >> [v2:172.21.15.45:6800/1989083510,v1:172.21.15.45:6801/1989083510] conn(0x7f2f60074120 0x7f2f60076540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:40.595 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.517+0000 7f76977fe640 1 --2- 172.21.15.27:0/15777388 >> [v2:172.21.15.45:6816/3943189827,v1:172.21.15.45:6817/3943189827] conn(0x7f769c074120 0x7f769c076540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:40.595 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.517+0000 7f76977fe640 1 -- 172.21.15.27:0/15777388 --> [v2:172.21.15.45:6816/3943189827,v1:172.21.15.45:6817/3943189827] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f76a0032440 con 0x7f769c074120 2024-09-12T03:16:40.595 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.517+0000 7f76977fe640 1 -- 172.21.15.27:0/15777388 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f76a00b2050 con 0x7f76b00afff0 2024-09-12T03:16:40.596 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.517+0000 7f2f6effd640 1 -- 172.21.15.27:0/1763082757 --> [v2:172.21.15.45:6800/1989083510,v1:172.21.15.45:6801/1989083510] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2f7400fe50 con 0x7f2f60074120 2024-09-12T03:16:40.596 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.517+0000 7f2f6effd640 1 -- 172.21.15.27:0/1763082757 <== mon.2 v2:172.21.15.45:3301/0 6 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f2f7408b240 con 0x7f2f800d20b0 2024-09-12T03:16:40.596 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.518+0000 7f76affff640 1 --2- 172.21.15.27:0/15777388 >> [v2:172.21.15.45:6816/3943189827,v1:172.21.15.45:6817/3943189827] conn(0x7f769c074120 0x7f769c076540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:40.597 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.518+0000 7f76ae7fc640 1 --2- 172.21.15.27:0/15777388 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f769c06bf00 0x7f769c06e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:40.597 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.518+0000 7f2f7ed9d640 1 --2- 172.21.15.27:0/1763082757 >> [v2:172.21.15.45:6800/1989083510,v1:172.21.15.45:6801/1989083510] conn(0x7f2f60074120 0x7f2f60076540 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:40.597 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.518+0000 7f76ae7fc640 1 --2- 172.21.15.27:0/15777388 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f769c06bf00 0x7f769c06e3c0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f76b01e96d0 tx=0x7f76a4007b90 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:40.597 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.518+0000 7f76affff640 1 --2- 172.21.15.27:0/15777388 >> [v2:172.21.15.45:6816/3943189827,v1:172.21.15.45:6817/3943189827] conn(0x7f769c074120 0x7f769c076540 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:40.598 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.519+0000 7f2f6effd640 1 -- 172.21.15.27:0/1763082757 <== osd.6 v2:172.21.15.45:6800/1989083510 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2f7400fe50 con 0x7f2f60074120 2024-09-12T03:16:40.598 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.519+0000 7f76977fe640 1 -- 172.21.15.27:0/15777388 <== osd.4 v2:172.21.15.45:6816/3943189827 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f76a0032440 con 0x7f769c074120 2024-09-12T03:16:40.600 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.544+0000 7f8d85c7b640 1 -- 172.21.15.27:0/1022859362 --> [v2:172.21.15.27:6800/80661598,v1:172.21.15.27:6801/80661598] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f8d60002dd0 con 0x7f8d6c074120 2024-09-12T03:16:40.600 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.544+0000 7f8d67fff640 1 -- 172.21.15.27:0/1022859362 <== osd.3 v2:172.21.15.27:6800/80661598 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f8d60002dd0 con 0x7f8d6c074120 2024-09-12T03:16:40.600 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.545+0000 7f8d65ffb640 1 -- 172.21.15.27:0/1022859362 >> [v2:172.21.15.27:6800/80661598,v1:172.21.15.27:6801/80661598] conn(0x7f8d6c074120 msgr2=0x7f8d6c076540 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:40.600 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.545+0000 7f8d65ffb640 1 --2- 172.21.15.27:0/1022859362 >> [v2:172.21.15.27:6800/80661598,v1:172.21.15.27:6801/80661598] conn(0x7f8d6c074120 0x7f8d6c076540 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:40.600 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.545+0000 7f8d65ffb640 1 -- 172.21.15.27:0/1022859362 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f8d6c06bf00 msgr2=0x7f8d6c06e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:40.600 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.545+0000 7f8d65ffb640 1 --2- 172.21.15.27:0/1022859362 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f8d6c06bf00 0x7f8d6c06e3c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f8d80200080 tx=0x7f8d70008040 comp rx=0 tx=0).stop 2024-09-12T03:16:40.600 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.545+0000 7f8d65ffb640 1 -- 172.21.15.27:0/1022859362 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f8d80072930 msgr2=0x7f8d801f71f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:40.600 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.545+0000 7f8d65ffb640 1 --2- 172.21.15.27:0/1022859362 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f8d80072930 0x7f8d801f71f0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f8d7800ca70 tx=0x7f8d7800cf30 comp rx=0 tx=0).stop 2024-09-12T03:16:40.600 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.546+0000 7f8d7ffff640 1 -- 172.21.15.27:0/1022859362 reap_dead start 2024-09-12T03:16:40.601 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.546+0000 7f8d65ffb640 1 -- 172.21.15.27:0/1022859362 shutdown_connections 2024-09-12T03:16:40.601 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.546+0000 7f8d65ffb640 1 -- 172.21.15.27:0/1022859362 >> 172.21.15.27:0/1022859362 conn(0x7f8d8006bd10 msgr2=0x7f8d800b1500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:40.601 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.546+0000 7f8d65ffb640 1 -- 172.21.15.27:0/1022859362 shutdown_connections 2024-09-12T03:16:40.601 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.546+0000 7f8d65ffb640 1 -- 172.21.15.27:0/1022859362 wait complete. 2024-09-12T03:16:40.601 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.547+0000 7f4a8dc7b640 1 -- 172.21.15.27:0/273501929 --> [v2:172.21.15.45:6808/3620760330,v1:172.21.15.45:6809/3620760330] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f4a880b9240 con 0x7f4a680743d0 2024-09-12T03:16:40.601 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.547+0000 7fc58b3c6640 1 -- 172.21.15.27:0/666499063 --> [v2:172.21.15.27:6816/3002321488,v1:172.21.15.27:6817/3002321488] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fc5840d1d30 con 0x7fc568073fd0 2024-09-12T03:16:40.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.547+0000 7fc579ffb640 1 -- 172.21.15.27:0/666499063 <== osd.0 v2:172.21.15.27:6816/3002321488 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fc5840d1d30 con 0x7fc568073fd0 2024-09-12T03:16:40.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.547+0000 7fe996ffd640 1 -- 172.21.15.27:0/1405050804 --> [v2:172.21.15.27:6808/1613102886,v1:172.21.15.27:6809/1613102886] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fe988002dd0 con 0x7fe99c0740a0 2024-09-12T03:16:40.603 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.548+0000 7fe9ad7fa640 1 -- 172.21.15.27:0/1405050804 <== osd.1 v2:172.21.15.27:6808/1613102886 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fe988002dd0 con 0x7fe99c0740a0 2024-09-12T03:16:40.603 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.548+0000 7f4a857fa640 1 -- 172.21.15.27:0/273501929 <== osd.5 v2:172.21.15.45:6808/3620760330 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f4a880b9240 con 0x7f4a680743d0 2024-09-12T03:16:40.603 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.548+0000 7f3c7cff9640 1 -- 172.21.15.27:0/2937087087 --> [v2:172.21.15.27:6824/3534723355,v1:172.21.15.27:6825/3534723355] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f3c68002dd0 con 0x7f3c70074120 2024-09-12T03:16:40.604 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.548+0000 7fc5677fe640 1 -- 172.21.15.27:0/666499063 >> [v2:172.21.15.27:6816/3002321488,v1:172.21.15.27:6817/3002321488] conn(0x7fc568073fd0 msgr2=0x7fc5680763f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:40.604 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.548+0000 7fc5677fe640 1 --2- 172.21.15.27:0/666499063 >> [v2:172.21.15.27:6816/3002321488,v1:172.21.15.27:6817/3002321488] conn(0x7fc568073fd0 0x7fc5680763f0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:40.604 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.548+0000 7f3c7effd640 1 -- 172.21.15.27:0/2937087087 <== osd.2 v2:172.21.15.27:6824/3534723355 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f3c68002dd0 con 0x7f3c70074120 2024-09-12T03:16:40.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.548+0000 7fe996ffd640 1 -- 172.21.15.27:0/1405050804 >> [v2:172.21.15.27:6808/1613102886,v1:172.21.15.27:6809/1613102886] conn(0x7fe99c0740a0 msgr2=0x7fe99c0764c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:40.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.549+0000 7fe996ffd640 1 --2- 172.21.15.27:0/1405050804 >> [v2:172.21.15.27:6808/1613102886,v1:172.21.15.27:6809/1613102886] conn(0x7fe99c0740a0 0x7fe99c0764c0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:40.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.549+0000 7f9ea21ac640 1 -- 172.21.15.27:0/2997252703 --> [v2:172.21.15.45:6824/1513300294,v1:172.21.15.45:6825/1513300294] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f9e9c0d1fe0 con 0x7f9e80074f60 2024-09-12T03:16:40.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.549+0000 7f4a7effd640 1 -- 172.21.15.27:0/273501929 >> [v2:172.21.15.45:6808/3620760330,v1:172.21.15.45:6809/3620760330] conn(0x7f4a680743d0 msgr2=0x7f4a680767f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:40.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.549+0000 7f4a7effd640 1 --2- 172.21.15.27:0/273501929 >> [v2:172.21.15.45:6808/3620760330,v1:172.21.15.45:6809/3620760330] conn(0x7f4a680743d0 0x7f4a680767f0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:40.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.548+0000 7fc5677fe640 1 -- 172.21.15.27:0/666499063 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fc56806be30 msgr2=0x7fc56806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:40.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.548+0000 7fc5677fe640 1 --2- 172.21.15.27:0/666499063 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fc56806be30 0x7fc56806e2f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc580006d40 tx=0x7fc580008040 comp rx=0 tx=0).stop 2024-09-12T03:16:40.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.548+0000 7fc5677fe640 1 -- 172.21.15.27:0/666499063 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc5840d09b0 msgr2=0x7fc5840ad0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:40.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.548+0000 7fc5677fe640 1 --2- 172.21.15.27:0/666499063 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc5840d09b0 0x7fc5840ad0c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fc574031000 tx=0x7fc5740046c0 comp rx=0 tx=0).stop 2024-09-12T03:16:40.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.549+0000 7fc589963640 1 -- 172.21.15.27:0/666499063 reap_dead start 2024-09-12T03:16:40.607 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.549+0000 7fe996ffd640 1 -- 172.21.15.27:0/1405050804 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fe99c06be80 msgr2=0x7fe99c06e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:40.607 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.549+0000 7f4a7effd640 1 -- 172.21.15.27:0/273501929 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f4a6806c230 msgr2=0x7f4a6806e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:40.607 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.549+0000 7f4a7effd640 1 --2- 172.21.15.27:0/273501929 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f4a6806c230 0x7f4a6806e6f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f4a880c8060 tx=0x7f4a78009190 comp rx=0 tx=0).stop 2024-09-12T03:16:40.607 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.549+0000 7f4a7effd640 1 -- 172.21.15.27:0/273501929 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4a8806c160 msgr2=0x7f4a8815de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:40.607 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.549+0000 7f4a7effd640 1 --2- 172.21.15.27:0/273501929 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4a8806c160 0x7f4a8815de30 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f4a7000cac0 tx=0x7f4a7000cf80 comp rx=0 tx=0).stop 2024-09-12T03:16:40.607 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.549+0000 7fe996ffd640 1 --2- 172.21.15.27:0/1405050804 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fe99c06be80 0x7fe99c06e340 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fe9a4009a10 tx=0x7fe9a4008040 comp rx=0 tx=0).stop 2024-09-12T03:16:40.608 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.549+0000 7fc5677fe640 1 -- 172.21.15.27:0/666499063 shutdown_connections 2024-09-12T03:16:40.608 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.549+0000 7fc5677fe640 1 -- 172.21.15.27:0/666499063 >> 172.21.15.27:0/666499063 conn(0x7fc5840b4320 msgr2=0x7fc58415dd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:40.608 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.549+0000 7fe996ffd640 1 -- 172.21.15.27:0/1405050804 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe9b0162c00 msgr2=0x7fe9b0150730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:40.608 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.549+0000 7fe996ffd640 1 --2- 172.21.15.27:0/1405050804 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe9b0162c00 0x7fe9b0150730 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fe9a0031000 tx=0x7fe9a0004a40 comp rx=0 tx=0).stop 2024-09-12T03:16:40.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.549+0000 7fe9b5135640 1 -- 172.21.15.27:0/1405050804 reap_dead start 2024-09-12T03:16:40.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.549+0000 7f4a8d47a640 1 -- 172.21.15.27:0/273501929 reap_dead start 2024-09-12T03:16:40.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.549+0000 7f4a7effd640 1 -- 172.21.15.27:0/273501929 shutdown_connections 2024-09-12T03:16:40.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.549+0000 7f4a7effd640 1 -- 172.21.15.27:0/273501929 >> 172.21.15.27:0/273501929 conn(0x7f4a880d46a0 msgr2=0x7f4a880c9960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:40.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.549+0000 7fc5677fe640 1 -- 172.21.15.27:0/666499063 shutdown_connections 2024-09-12T03:16:40.610 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.549+0000 7fc5677fe640 1 -- 172.21.15.27:0/666499063 wait complete. 2024-09-12T03:16:40.610 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.549+0000 7fe996ffd640 1 -- 172.21.15.27:0/1405050804 shutdown_connections 2024-09-12T03:16:40.610 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.549+0000 7fe996ffd640 1 -- 172.21.15.27:0/1405050804 >> 172.21.15.27:0/1405050804 conn(0x7fe9b006f300 msgr2=0x7fe9b01633d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:40.610 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.549+0000 7f3c7cff9640 1 -- 172.21.15.27:0/2937087087 >> [v2:172.21.15.27:6824/3534723355,v1:172.21.15.27:6825/3534723355] conn(0x7f3c70074120 msgr2=0x7f3c70076540 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:40.610 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.549+0000 7f3c7cff9640 1 --2- 172.21.15.27:0/2937087087 >> [v2:172.21.15.27:6824/3534723355,v1:172.21.15.27:6825/3534723355] conn(0x7f3c70074120 0x7f3c70076540 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:40.611 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.549+0000 7fe996ffd640 1 -- 172.21.15.27:0/1405050804 shutdown_connections 2024-09-12T03:16:40.611 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.549+0000 7fe996ffd640 1 -- 172.21.15.27:0/1405050804 wait complete. 2024-09-12T03:16:40.611 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.549+0000 7f9e91ffb640 1 -- 172.21.15.27:0/2997252703 <== osd.7 v2:172.21.15.45:6824/1513300294 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f9e9c0d1fe0 con 0x7f9e80074f60 2024-09-12T03:16:40.612 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.549+0000 7f4a7effd640 1 -- 172.21.15.27:0/273501929 shutdown_connections 2024-09-12T03:16:40.612 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.549+0000 7f4a7effd640 1 -- 172.21.15.27:0/273501929 wait complete. 2024-09-12T03:16:40.612 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.550+0000 7f3c7cff9640 1 -- 172.21.15.27:0/2937087087 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f3c7006bf00 msgr2=0x7f3c7006e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:40.612 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.550+0000 7f3c7cff9640 1 --2- 172.21.15.27:0/2937087087 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f3c7006bf00 0x7f3c7006e3c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f3c84006fd0 tx=0x7f3c84008040 comp rx=0 tx=0).stop 2024-09-12T03:16:40.612 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.550+0000 7f3c7cff9640 1 -- 172.21.15.27:0/2937087087 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f3c9006e980 msgr2=0x7f3c90157a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:40.612 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.550+0000 7f3c7cff9640 1 --2- 172.21.15.27:0/2937087087 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f3c9006e980 0x7f3c90157a40 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f3c8000c460 tx=0x7f3c8000c920 comp rx=0 tx=0).stop 2024-09-12T03:16:40.613 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.550+0000 7f3c8ed9d640 1 -- 172.21.15.27:0/2937087087 reap_dead start 2024-09-12T03:16:40.613 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.550+0000 7f3c7cff9640 1 -- 172.21.15.27:0/2937087087 shutdown_connections 2024-09-12T03:16:40.613 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.550+0000 7f3c7cff9640 1 -- 172.21.15.27:0/2937087087 >> 172.21.15.27:0/2937087087 conn(0x7f3c90062d50 msgr2=0x7f3c9015d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:40.613 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.550+0000 7f3c7cff9640 1 -- 172.21.15.27:0/2937087087 shutdown_connections 2024-09-12T03:16:40.613 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.550+0000 7f3c7cff9640 1 -- 172.21.15.27:0/2937087087 wait complete. 2024-09-12T03:16:40.614 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.550+0000 7f9ea21ac640 1 -- 172.21.15.27:0/2997252703 >> [v2:172.21.15.45:6824/1513300294,v1:172.21.15.45:6825/1513300294] conn(0x7f9e80074f60 msgr2=0x7f9e80077400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:40.614 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.550+0000 7f9ea21ac640 1 --2- 172.21.15.27:0/2997252703 >> [v2:172.21.15.45:6824/1513300294,v1:172.21.15.45:6825/1513300294] conn(0x7f9e80074f60 0x7f9e80077400 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:40.614 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.551+0000 7f9ea21ac640 1 -- 172.21.15.27:0/2997252703 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f9e8006c230 msgr2=0x7f9e8006e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:40.614 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.551+0000 7f9ea21ac640 1 --2- 172.21.15.27:0/2997252703 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f9e8006c230 0x7f9e8006e6f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f9e9c0cdaa0 tx=0x7f9e9400b040 comp rx=0 tx=0).stop 2024-09-12T03:16:40.614 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.551+0000 7f9ea21ac640 1 -- 172.21.15.27:0/2997252703 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9e9c0b6200 msgr2=0x7f9e9c15c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:40.614 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.551+0000 7f9ea21ac640 1 --2- 172.21.15.27:0/2997252703 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9e9c0b6200 0x7f9e9c15c670 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f9e8c00c400 tx=0x7f9e8c00c8c0 comp rx=0 tx=0).stop 2024-09-12T03:16:40.615 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.551+0000 7f9ea19ab640 1 -- 172.21.15.27:0/2997252703 reap_dead start 2024-09-12T03:16:40.615 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.551+0000 7f76957fa640 1 -- 172.21.15.27:0/15777388 --> [v2:172.21.15.45:6816/3943189827,v1:172.21.15.45:6817/3943189827] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f7688002dd0 con 0x7f769c074120 2024-09-12T03:16:40.615 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.551+0000 7f9ea21ac640 1 -- 172.21.15.27:0/2997252703 shutdown_connections 2024-09-12T03:16:40.615 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.551+0000 7f9ea21ac640 1 -- 172.21.15.27:0/2997252703 >> 172.21.15.27:0/2997252703 conn(0x7f9e9c06f320 msgr2=0x7f9e9c0b3080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:40.615 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.551+0000 7f9ea21ac640 1 -- 172.21.15.27:0/2997252703 shutdown_connections 2024-09-12T03:16:40.616 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.551+0000 7f2f84ef8640 1 -- 172.21.15.27:0/1763082757 --> [v2:172.21.15.45:6800/1989083510,v1:172.21.15.45:6801/1989083510] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f2f800b5ff0 con 0x7f2f60074120 2024-09-12T03:16:40.616 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.551+0000 7f9ea21ac640 1 -- 172.21.15.27:0/2997252703 wait complete. 2024-09-12T03:16:40.616 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.552+0000 7f76977fe640 1 -- 172.21.15.27:0/15777388 <== osd.4 v2:172.21.15.45:6816/3943189827 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f7688002dd0 con 0x7f769c074120 2024-09-12T03:16:40.616 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.552+0000 7f2f6effd640 1 -- 172.21.15.27:0/1763082757 <== osd.6 v2:172.21.15.45:6800/1989083510 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f2f800b5ff0 con 0x7f2f60074120 2024-09-12T03:16:40.617 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.552+0000 7f2f84ef8640 1 -- 172.21.15.27:0/1763082757 >> [v2:172.21.15.45:6800/1989083510,v1:172.21.15.45:6801/1989083510] conn(0x7f2f60074120 msgr2=0x7f2f60076540 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:40.617 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.552+0000 7f2f84ef8640 1 --2- 172.21.15.27:0/1763082757 >> [v2:172.21.15.45:6800/1989083510,v1:172.21.15.45:6801/1989083510] conn(0x7f2f60074120 0x7f2f60076540 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:40.617 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.553+0000 7f76b5645640 1 -- 172.21.15.27:0/15777388 >> [v2:172.21.15.45:6816/3943189827,v1:172.21.15.45:6817/3943189827] conn(0x7f769c074120 msgr2=0x7f769c076540 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:40.617 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.553+0000 7f76b5645640 1 --2- 172.21.15.27:0/15777388 >> [v2:172.21.15.45:6816/3943189827,v1:172.21.15.45:6817/3943189827] conn(0x7f769c074120 0x7f769c076540 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:40.618 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.553+0000 7f2f84ef8640 1 -- 172.21.15.27:0/1763082757 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f2f6006bf00 msgr2=0x7f2f6006e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:40.618 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.553+0000 7f2f84ef8640 1 --2- 172.21.15.27:0/1763082757 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f2f6006bf00 0x7f2f6006e3c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f2f7002a1e0 tx=0x7f2f7002e000 comp rx=0 tx=0).stop 2024-09-12T03:16:40.618 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.553+0000 7f2f84ef8640 1 -- 172.21.15.27:0/1763082757 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2f800d20b0 msgr2=0x7f2f80071270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:40.618 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.553+0000 7f2f84ef8640 1 --2- 172.21.15.27:0/1763082757 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2f800d20b0 0x7f2f80071270 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f2f74009fd0 tx=0x7f2f7400b870 comp rx=0 tx=0).stop 2024-09-12T03:16:40.618 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.553+0000 7f2f7ed9d640 1 -- 172.21.15.27:0/1763082757 reap_dead start 2024-09-12T03:16:40.619 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.553+0000 7f2f84ef8640 1 -- 172.21.15.27:0/1763082757 shutdown_connections 2024-09-12T03:16:40.619 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.553+0000 7f2f84ef8640 1 -- 172.21.15.27:0/1763082757 >> 172.21.15.27:0/1763082757 conn(0x7f2f800d4570 msgr2=0x7f2f800d4950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:40.619 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.553+0000 7f2f84ef8640 1 -- 172.21.15.27:0/1763082757 shutdown_connections 2024-09-12T03:16:40.619 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.553+0000 7f2f84ef8640 1 -- 172.21.15.27:0/1763082757 wait complete. 2024-09-12T03:16:40.620 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.553+0000 7f76b5645640 1 -- 172.21.15.27:0/15777388 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f769c06bf00 msgr2=0x7f769c06e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:40.620 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.554+0000 7f76b5645640 1 --2- 172.21.15.27:0/15777388 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f769c06bf00 0x7f769c06e3c0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f76b01e96d0 tx=0x7f76a4007b90 comp rx=0 tx=0).stop 2024-09-12T03:16:40.620 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.554+0000 7f76b5645640 1 -- 172.21.15.27:0/15777388 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f76b00afff0 msgr2=0x7f76b00cf850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:40.620 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.554+0000 7f76b5645640 1 --2- 172.21.15.27:0/15777388 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f76b00afff0 0x7f76b00cf850 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f76a00098c0 tx=0x7f76a00046c0 comp rx=0 tx=0).stop 2024-09-12T03:16:40.620 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.554+0000 7f76affff640 1 -- 172.21.15.27:0/15777388 reap_dead start 2024-09-12T03:16:40.620 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.554+0000 7f76b5645640 1 -- 172.21.15.27:0/15777388 shutdown_connections 2024-09-12T03:16:40.621 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.554+0000 7f76b5645640 1 -- 172.21.15.27:0/15777388 >> 172.21.15.27:0/15777388 conn(0x7f76b00b2570 msgr2=0x7f76b00b06a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:40.621 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.557+0000 7f76b5645640 1 -- 172.21.15.27:0/15777388 shutdown_connections 2024-09-12T03:16:40.621 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.557+0000 7f76b5645640 1 -- 172.21.15.27:0/15777388 wait complete. 2024-09-12T03:16:40.621 INFO:teuthology.orchestra.run.smithi027.stdout:60129542151 2024-09-12T03:16:40.622 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-12T03:16:40.622 INFO:teuthology.orchestra.run.smithi027.stdout:60129542152 2024-09-12T03:16:40.623 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-12T03:16:40.623 INFO:teuthology.orchestra.run.smithi027.stdout:60129542152 2024-09-12T03:16:40.624 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-12T03:16:40.625 INFO:teuthology.orchestra.run.smithi027.stdout:60129542151 2024-09-12T03:16:40.625 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-12T03:16:40.626 INFO:teuthology.orchestra.run.smithi027.stdout:60129542151 2024-09-12T03:16:40.626 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-12T03:16:40.627 INFO:teuthology.orchestra.run.smithi027.stdout:60129542152 2024-09-12T03:16:40.627 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-12T03:16:40.628 INFO:teuthology.orchestra.run.smithi027.stdout:60129542150 2024-09-12T03:16:40.628 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-12T03:16:40.629 INFO:teuthology.orchestra.run.smithi027.stdout:60129542150 2024-09-12T03:16:40.630 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-12T03:16:40.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.837+0000 7f0230080640 1 Processor -- start 2024-09-12T03:16:40.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.838+0000 7fb032442640 1 Processor -- start 2024-09-12T03:16:40.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.837+0000 7f0230080640 1 -- start start 2024-09-12T03:16:40.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.838+0000 7f0230080640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f022806e970 0x7f02280cd750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:40.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.839+0000 7f75b0cb7640 1 Processor -- start 2024-09-12T03:16:40.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.839+0000 7fb032442640 1 -- start start 2024-09-12T03:16:40.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.838+0000 7f0230080640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f022806d0a0 0x7f022806d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:40.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.838+0000 7f0230080640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f022806de60 con 0x7f022815db00 2024-09-12T03:16:40.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.838+0000 7f0230080640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f0228071fe0 con 0x7f022806e970 2024-09-12T03:16:40.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.838+0000 7f0230080640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f0228071990 con 0x7f022806d0a0 2024-09-12T03:16:40.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.839+0000 7f022de1c640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f022815db00 0x7f022806e260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.27:38924/0 (socket says 172.21.15.27:38924) 2024-09-12T03:16:40.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.839+0000 7f022de1c640 1 -- 172.21.15.27:0/905792217 learned_addr learned my addr 172.21.15.27:0/905792217 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:16:40.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.839+0000 7f022d61b640 1 --2- 172.21.15.27:0/905792217 >> v2:172.21.15.45:3300/0 conn(0x7f022806e970 0x7f02280cd750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:40.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.839+0000 7f022d61b640 1 -- 172.21.15.27:0/905792217 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f022806d0a0 msgr2=0x7f022806d4f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-12T03:16:40.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.839+0000 7f022d61b640 1 --2- 172.21.15.27:0/905792217 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f022806d0a0 0x7f022806d4f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:40.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.839+0000 7f022d61b640 1 -- 172.21.15.27:0/905792217 >> v1:172.21.15.45:6789/0 conn(0x7f022815db00 legacy=0x7f022806e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:40.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.839+0000 7f022d61b640 1 -- 172.21.15.27:0/905792217 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0228071c70 con 0x7f022806e970 2024-09-12T03:16:40.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.841+0000 7f022d61b640 1 --2- 172.21.15.27:0/905792217 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f022806e970 0x7f02280cd750 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f022000bd20 tx=0x7f0220032c90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d583902a8ad0d749 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:40.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.841+0000 7f022ce1a640 1 -- 172.21.15.27:0/905792217 <== mon.1 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0220039070 con 0x7f022806e970 2024-09-12T03:16:40.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.841+0000 7f022ce1a640 1 -- 172.21.15.27:0/905792217 <== mon.1 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0220033920 con 0x7f022806e970 2024-09-12T03:16:40.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.841+0000 7f022ce1a640 1 -- 172.21.15.27:0/905792217 <== mon.1 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0220033d10 con 0x7f022806e970 2024-09-12T03:16:40.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.840+0000 7fb032442640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fb02c15a270 0x7fb02c06ee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:40.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.840+0000 7fb032442640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb02c06d950 0x7fb02c0cc900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:40.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.840+0000 7fb032442640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb02c0b3c60 con 0x7fb02c0cfde0 2024-09-12T03:16:40.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.840+0000 7fb032442640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7fb02c0b3f40 con 0x7fb02c15a270 2024-09-12T03:16:40.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.840+0000 7fb032442640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fb02c0b4220 con 0x7fb02c06d950 2024-09-12T03:16:40.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.840+0000 7fb031440640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fb02c15a270 0x7fb02c06ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:40.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.840+0000 7fb031440640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fb02c15a270 0x7fb02c06ee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33246/0 (socket says 172.21.15.27:33246) 2024-09-12T03:16:40.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.840+0000 7fb031440640 1 -- 172.21.15.27:0/3277541378 learned_addr learned my addr 172.21.15.27:0/3277541378 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:16:40.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.841+0000 7fb031440640 1 -- 172.21.15.27:0/3277541378 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb02c06d950 msgr2=0x7fb02c0cc900 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-12T03:16:40.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.841+0000 7fb031440640 1 --2- 172.21.15.27:0/3277541378 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb02c06d950 0x7fb02c0cc900 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:40.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.841+0000 7fb031440640 1 -- 172.21.15.27:0/3277541378 >> v1:172.21.15.45:6789/0 conn(0x7fb02c0cfde0 legacy=0x7fb02c06d240 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-12T03:16:40.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.841+0000 7fb031440640 1 -- 172.21.15.27:0/3277541378 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb02c0b4500 con 0x7fb02c15a270 2024-09-12T03:16:40.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.840+0000 7f75b0cb7640 1 -- start start 2024-09-12T03:16:40.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.841+0000 7f75b0cb7640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f75ac071030 0x7f75ac0cd750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:40.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.842+0000 7f75b0cb7640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f75ac15db00 0x7f75ac06d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:40.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.842+0000 7f75b0cb7640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f75ac06d5e0 0x7f75ac06e260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:40.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.842+0000 7f75b0cb7640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f75ac071fe0 con 0x7f75ac071030 2024-09-12T03:16:40.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.842+0000 7f75b0cb7640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f75ac06de60 con 0x7f75ac06d5e0 2024-09-12T03:16:40.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.842+0000 7f75b0cb7640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f75ac071990 con 0x7f75ac15db00 2024-09-12T03:16:40.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.842+0000 7f0e67fff640 1 Processor -- start 2024-09-12T03:16:40.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.842+0000 7f6327eac640 1 Processor -- start 2024-09-12T03:16:40.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.842+0000 7f75a9d9b640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f75ac15db00 0x7f75ac06d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:40.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.842+0000 7f75a9d9b640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f75ac15db00 0x7f75ac06d0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3301/0 says I am v2:172.21.15.27:56360/0 (socket says 172.21.15.27:56360) 2024-09-12T03:16:40.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.842+0000 7f75a9d9b640 1 -- 172.21.15.27:0/771837467 learned_addr learned my addr 172.21.15.27:0/771837467 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:16:40.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.842+0000 7f0230080640 1 -- 172.21.15.27:0/905792217 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f022806e970 msgr2=0x7f02280cd750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:40.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.842+0000 7f0e67fff640 1 -- start start 2024-09-12T03:16:40.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.842+0000 7f75aad9d640 1 --2- 172.21.15.27:0/771837467 >> v2:172.21.15.45:3300/0 conn(0x7f75ac06d5e0 0x7f75ac06e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:40.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.842+0000 7f0230080640 1 --2- 172.21.15.27:0/905792217 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f022806e970 0x7f02280cd750 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f022000bd20 tx=0x7f0220032c90 comp rx=0 tx=0).stop 2024-09-12T03:16:40.847 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.842+0000 7f6327eac640 1 -- start start 2024-09-12T03:16:40.847 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.842+0000 7f0230080640 1 -- 172.21.15.27:0/905792217 shutdown_connections 2024-09-12T03:16:40.847 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.842+0000 7f0230080640 1 --2- 172.21.15.27:0/905792217 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f022806d0a0 0x7f022806d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:40.847 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.842+0000 7f0230080640 1 --2- 172.21.15.27:0/905792217 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f022806e970 0x7f02280cd750 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:40.847 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.842+0000 7f0230080640 1 -- 172.21.15.27:0/905792217 >> 172.21.15.27:0/905792217 conn(0x7f022806bd10 msgr2=0x7f022806c0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:40.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.842+0000 7fb031440640 1 --2- 172.21.15.27:0/3277541378 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb02c15a270 0x7fb02c06ee50 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fb028009b00 tx=0x7fb02802a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbb5b788df758a0c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:40.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.842+0000 7fb023fff640 1 -- 172.21.15.27:0/3277541378 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb028036070 con 0x7fb02c15a270 2024-09-12T03:16:40.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.842+0000 7f8cf6d33640 1 Processor -- start 2024-09-12T03:16:40.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.842+0000 7f8cf6d33640 1 -- start start 2024-09-12T03:16:40.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.843+0000 7fb023fff640 1 -- 172.21.15.27:0/3277541378 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb028031600 con 0x7fb02c15a270 2024-09-12T03:16:40.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.843+0000 7f75aa59c640 1 --2- 172.21.15.27:0/771837467 >> v2:172.21.15.27:3300/0 conn(0x7f75ac071030 0x7f75ac0cd750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:40.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.843+0000 7f0230080640 1 -- 172.21.15.27:0/905792217 shutdown_connections 2024-09-12T03:16:40.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.843+0000 7f0230080640 1 -- 172.21.15.27:0/905792217 wait complete. 2024-09-12T03:16:40.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.843+0000 7f75a9d9b640 1 -- 172.21.15.27:0/771837467 >> v2:172.21.15.45:3300/0 conn(0x7f75ac06d5e0 msgr2=0x7f75ac06e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:40.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.843+0000 7f75a9d9b640 1 --2- 172.21.15.27:0/771837467 >> v2:172.21.15.45:3300/0 conn(0x7f75ac06d5e0 0x7f75ac06e260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:40.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.843+0000 7f75a9d9b640 1 -- 172.21.15.27:0/771837467 >> v2:172.21.15.27:3300/0 conn(0x7f75ac071030 msgr2=0x7f75ac0cd750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:40.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.843+0000 7f75a9d9b640 1 --2- 172.21.15.27:0/771837467 >> v2:172.21.15.27:3300/0 conn(0x7f75ac071030 0x7f75ac0cd750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:40.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.843+0000 7f75a9d9b640 1 -- 172.21.15.27:0/771837467 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75ac071c70 con 0x7f75ac15db00 2024-09-12T03:16:40.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.843+0000 7f75aa59c640 1 --2- 172.21.15.27:0/771837467 >> v2:172.21.15.27:3300/0 conn(0x7f75ac071030 0x7f75ac0cd750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:16:40.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.843+0000 7f1b70c5e640 1 Processor -- start 2024-09-12T03:16:40.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.843+0000 7f2635e1e640 1 Processor -- start 2024-09-12T03:16:40.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.843+0000 7f2635e1e640 1 -- start start 2024-09-12T03:16:40.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.844+0000 7fb032442640 1 -- 172.21.15.27:0/3277541378 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb02c15a270 msgr2=0x7fb02c06ee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:40.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.844+0000 7fb032442640 1 --2- 172.21.15.27:0/3277541378 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb02c15a270 0x7fb02c06ee50 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fb028009b00 tx=0x7fb02802a420 comp rx=0 tx=0).stop 2024-09-12T03:16:40.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.844+0000 7f8cf6d33640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f8cf015d070 0x7f8cf00b53c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:40.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.844+0000 7f8cf6d33640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f8cf0163cf0 0x7f8cf006dde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:40.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.844+0000 7f8cf6d33640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8cf006e830 con 0x7f8cf006e320 2024-09-12T03:16:40.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.844+0000 7f8cf6d33640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f8cf006eb10 con 0x7f8cf0163cf0 2024-09-12T03:16:40.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.844+0000 7f8cf6d33640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f8cf00b4690 con 0x7f8cf015d070 2024-09-12T03:16:40.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.844+0000 7f8cf52d0640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f8cf006e320 0x7f8cf006d010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.27:38934/0 (socket says 172.21.15.27:38934) 2024-09-12T03:16:40.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.844+0000 7f8cf52d0640 1 -- 172.21.15.27:0/935242691 learned_addr learned my addr 172.21.15.27:0/935242691 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:16:40.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.845+0000 7f8ceffff640 1 --2- 172.21.15.27:0/935242691 >> v2:172.21.15.27:3300/0 conn(0x7f8cf0163cf0 0x7f8cf006dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:40.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.845+0000 7f8cef7fe640 1 -- 172.21.15.27:0/935242691 <== mon.1 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2341348988 0 0) 0x7f8cf006e830 con 0x7f8cf006e320 2024-09-12T03:16:40.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.843+0000 7f0e67fff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f0e60007190 0x7f0e600581c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:40.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.843+0000 7f0e67fff640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f0e60037bd0 0x7f0e60057230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:40.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.843+0000 7f0e67fff640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0e600380d0 con 0x7f0e600ea950 2024-09-12T03:16:40.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.843+0000 7f0e67fff640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f0e60056da0 con 0x7f0e60007190 2024-09-12T03:16:40.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.843+0000 7f0e67fff640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f0e60056750 con 0x7f0e60037bd0 2024-09-12T03:16:40.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.843+0000 7f0e66ffd640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f0e60007190 0x7f0e600581c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:40.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.843+0000 7f0e66ffd640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f0e60007190 0x7f0e600581c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33272/0 (socket says 172.21.15.27:33272) 2024-09-12T03:16:40.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.843+0000 7f0e66ffd640 1 -- 172.21.15.27:0/2777232213 learned_addr learned my addr 172.21.15.27:0/2777232213 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:16:40.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.844+0000 7f0e66ffd640 1 -- 172.21.15.27:0/2777232213 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f0e60037bd0 msgr2=0x7f0e60057230 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-12T03:16:40.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.844+0000 7f0e66ffd640 1 --2- 172.21.15.27:0/2777232213 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f0e60037bd0 0x7f0e60057230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:40.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.844+0000 7f0e66ffd640 1 -- 172.21.15.27:0/2777232213 >> v1:172.21.15.27:6789/0 conn(0x7f0e600ea950 legacy=0x7f0e60005dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:40.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.844+0000 7f0e66ffd640 1 -- 172.21.15.27:0/2777232213 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e60056a30 con 0x7f0e60007190 2024-09-12T03:16:40.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.844+0000 7f6327eac640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f631801db50 0x7f631800ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:40.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.844+0000 7f6327eac640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f6318006c40 0x7f6318050120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:40.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.844+0000 7f6327eac640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6318009ed0 con 0x7f6318005ff0 2024-09-12T03:16:40.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.844+0000 7f6327eac640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f631800b1d0 con 0x7f6318006c40 2024-09-12T03:16:40.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.844+0000 7f6327eac640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f631800a800 con 0x7f631801db50 2024-09-12T03:16:40.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.845+0000 7f6325447640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f6318006c40 0x7f6318050120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:40.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.845+0000 7f6325447640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f6318006c40 0x7f6318050120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33286/0 (socket says 172.21.15.27:33286) 2024-09-12T03:16:40.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.845+0000 7f6325447640 1 -- 172.21.15.27:0/4086664624 learned_addr learned my addr 172.21.15.27:0/4086664624 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:16:40.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.845+0000 7f8cef7fe640 1 -- 172.21.15.27:0/935242691 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8cd8003520 con 0x7f8cf006e320 2024-09-12T03:16:40.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.845+0000 7f2635e1e640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f2628004a60 0x7f262800bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:40.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.845+0000 7f2635e1e640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2628055840 con 0x7f262800a460 2024-09-12T03:16:40.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.845+0000 7fb032442640 1 -- 172.21.15.27:0/3277541378 shutdown_connections 2024-09-12T03:16:40.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.845+0000 7fb032442640 1 --2- 172.21.15.27:0/3277541378 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb02c06d950 0x7fb02c0cc900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:40.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.845+0000 7fb032442640 1 --2- 172.21.15.27:0/3277541378 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb02c15a270 0x7fb02c06ee50 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:40.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.845+0000 7fb032442640 1 -- 172.21.15.27:0/3277541378 >> 172.21.15.27:0/3277541378 conn(0x7fb02c0d0210 msgr2=0x7fb02c0d0620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:40.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.845+0000 7f2635e1e640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2628006ca0 con 0x7f2628037040 2024-09-12T03:16:40.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.845+0000 7f2635e1e640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f2628007210 con 0x7f2628004a60 2024-09-12T03:16:40.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.845+0000 7f0230080640 1 Processor -- start 2024-09-12T03:16:40.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.845+0000 7f8cef7fe640 1 -- 172.21.15.27:0/935242691 <== mon.1 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 714139681 0 0) 0x7f8cd8003520 con 0x7f8cf006e320 2024-09-12T03:16:40.860 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.844+0000 7f75a9d9b640 1 --2- 172.21.15.27:0/771837467 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f75ac15db00 0x7f75ac06d0a0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f75ac0b3a50 tx=0x7f759c02a1e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b73382e1d0e04470 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:40.860 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.845+0000 7f75a959a640 1 -- 172.21.15.27:0/771837467 <== mon.2 v2:172.21.15.45:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f759c037070 con 0x7f75ac15db00 2024-09-12T03:16:40.860 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.845+0000 7f75a959a640 1 -- 172.21.15.27:0/771837467 <== mon.2 v2:172.21.15.45:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f759c032440 con 0x7f75ac15db00 2024-09-12T03:16:40.860 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.845+0000 7f8cef7fe640 1 -- 172.21.15.27:0/935242691 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f8cf015d070 msgr2=0x7f8cf00b53c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:40.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.845+0000 7f75a959a640 1 -- 172.21.15.27:0/771837467 <== mon.2 v2:172.21.15.45:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f759c032810 con 0x7f75ac15db00 2024-09-12T03:16:40.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.845+0000 7f8cef7fe640 1 --2- 172.21.15.27:0/935242691 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f8cf015d070 0x7f8cf00b53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:40.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.844+0000 7f1b70c5e640 1 -- start start 2024-09-12T03:16:40.862 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.845+0000 7f8cef7fe640 1 -- 172.21.15.27:0/935242691 >> v2:172.21.15.27:3300/0 conn(0x7f8cf0163cf0 msgr2=0x7f8cf006dde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:40.862 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.845+0000 7f8cef7fe640 1 --2- 172.21.15.27:0/935242691 >> v2:172.21.15.27:3300/0 conn(0x7f8cf0163cf0 0x7f8cf006dde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:40.862 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.845+0000 7f1b70c5e640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f1b6c0b53a0 0x7f1b6c0b5810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:40.862 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.845+0000 7f6325c48640 1 --2- 172.21.15.27:0/4086664624 >> v2:172.21.15.45:3300/0 conn(0x7f631801db50 0x7f631800ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:40.862 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.845+0000 7f8cef7fe640 1 -- 172.21.15.27:0/935242691 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cf00b48e0 con 0x7f8cf006e320 2024-09-12T03:16:40.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.845+0000 7f1b70c5e640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f1b6c06dbc0 0x7f1b6c06e030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:40.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.845+0000 7f1b70c5e640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1b6c0b4010 con 0x7f1b6c159990 2024-09-12T03:16:40.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.845+0000 7f1b70c5e640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f1b6c0701e0 con 0x7f1b6c0b53a0 2024-09-12T03:16:40.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.845+0000 7f1b70c5e640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f1b6c0b5d50 con 0x7f1b6c06dbc0 2024-09-12T03:16:40.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.845+0000 7f0230080640 1 -- start start 2024-09-12T03:16:40.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.845+0000 7f262f7fe640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f262800a460 0x7f26280396d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:39196/0 (socket says 172.21.15.27:39196) 2024-09-12T03:16:40.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.845+0000 7f262f7fe640 1 -- 172.21.15.27:0/3721016814 learned_addr learned my addr 172.21.15.27:0/3721016814 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:16:40.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.845+0000 7fb032442640 1 -- 172.21.15.27:0/3277541378 shutdown_connections 2024-09-12T03:16:40.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.845+0000 7fb032442640 1 -- 172.21.15.27:0/3277541378 wait complete. 2024-09-12T03:16:40.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.845+0000 7f8cef7fe640 1 -- 172.21.15.27:0/935242691 <== mon.1 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 847755213 0 0) 0x7f8ce80019f0 con 0x7f8cf006e320 2024-09-12T03:16:40.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.845+0000 7f262effd640 1 --2- 172.21.15.27:0/3721016814 >> v2:172.21.15.27:3300/0 conn(0x7f2628004a60 0x7f262800bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:40.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.845+0000 7f262effd640 1 -- 172.21.15.27:0/3721016814 >> v1:172.21.15.45:6789/0 conn(0x7f2628037040 legacy=0x7f2628055130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:40.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.845+0000 7f8cef7fe640 1 -- 172.21.15.27:0/935242691 >> v1:172.21.15.45:6789/0 conn(0x7f8cf006e320 legacy=0x7f8cf006d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:40.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.845+0000 7f262effd640 1 -- 172.21.15.27:0/3721016814 >> v1:172.21.15.27:6789/0 conn(0x7f262800a460 legacy=0x7f26280396d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:40.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.845+0000 7f6325447640 1 -- 172.21.15.27:0/4086664624 >> v2:172.21.15.45:3300/0 conn(0x7f631801db50 msgr2=0x7f631800ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:40.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.846+0000 7f6325447640 1 --2- 172.21.15.27:0/4086664624 >> v2:172.21.15.45:3300/0 conn(0x7f631801db50 0x7f631800ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:40.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.846+0000 7f6325447640 1 -- 172.21.15.27:0/4086664624 >> v1:172.21.15.27:6789/0 conn(0x7f6318005ff0 legacy=0x7f631801c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:40.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.846+0000 7f1b6a59c640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f1b6c159990 0x7f1b6c06fad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.27:38956/0 (socket says 172.21.15.27:38956) 2024-09-12T03:16:40.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.846+0000 7f8cef7fe640 1 --2- 172.21.15.27:0/935242691 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8cd80039b0 0x7f8cd8003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:40.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.846+0000 7f1b6a59c640 1 -- 172.21.15.27:0/1720458839 learned_addr learned my addr 172.21.15.27:0/1720458839 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:16:40.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.846+0000 7f8cef7fe640 1 --2- 172.21.15.27:0/935242691 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8cd80043b0 0x7f8cd8004880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:40.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.846+0000 7f6325447640 1 -- 172.21.15.27:0/4086664624 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f631800a1b0 con 0x7f6318006c40 2024-09-12T03:16:40.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.846+0000 7f1b6ad9d640 1 --2- 172.21.15.27:0/1720458839 >> v2:172.21.15.45:3300/0 conn(0x7f1b6c06dbc0 0x7f1b6c06e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:40.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.846+0000 7f8cef7fe640 1 --2- 172.21.15.27:0/935242691 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f8cf015d070 0x7f8cd8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:40.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.846+0000 7f8cef7fe640 1 -- 172.21.15.27:0/935242691 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f8cf006eb10 con 0x7f8cd80039b0 2024-09-12T03:16:40.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.846+0000 7f1b6959a640 1 -- 172.21.15.27:0/1720458839 <== mon.1 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3902699730 0 0) 0x7f1b6c0b4010 con 0x7f1b6c159990 2024-09-12T03:16:40.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.846+0000 7f1b6959a640 1 -- 172.21.15.27:0/1720458839 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1b54003520 con 0x7f1b6c159990 2024-09-12T03:16:40.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.846+0000 7f8cf4acf640 1 --2- 172.21.15.27:0/935242691 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8cd80039b0 0x7f8cd8003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:40.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.846+0000 7f8cf52d0640 1 --2- 172.21.15.27:0/935242691 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f8cf015d070 0x7f8cd8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:40.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.846+0000 7f1b69d9b640 1 --2- 172.21.15.27:0/1720458839 >> v2:172.21.15.27:3300/0 conn(0x7f1b6c0b53a0 0x7f1b6c0b5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:40.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.846+0000 7f8cef7fe640 1 -- 172.21.15.27:0/935242691 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f8cf00b4690 con 0x7f8cd80043b0 2024-09-12T03:16:40.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.846+0000 7f8cef7fe640 1 -- 172.21.15.27:0/935242691 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f8cd800baf0 con 0x7f8cf015d070 2024-09-12T03:16:40.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.846+0000 7f1b6ad9d640 1 -- 172.21.15.27:0/1720458839 >> v2:172.21.15.27:3300/0 conn(0x7f1b6c0b53a0 msgr2=0x7f1b6c0b5810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:40.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.846+0000 7f1b6ad9d640 1 --2- 172.21.15.27:0/1720458839 >> v2:172.21.15.27:3300/0 conn(0x7f1b6c0b53a0 0x7f1b6c0b5810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:40.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.846+0000 7f1b6ad9d640 1 -- 172.21.15.27:0/1720458839 >> v1:172.21.15.45:6789/0 conn(0x7f1b6c159990 legacy=0x7f1b6c06fad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:40.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.846+0000 7f1b6ad9d640 1 -- 172.21.15.27:0/1720458839 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b6c0b42f0 con 0x7f1b6c06dbc0 2024-09-12T03:16:40.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.846+0000 7f1b69d9b640 1 --2- 172.21.15.27:0/1720458839 >> v2:172.21.15.27:3300/0 conn(0x7f1b6c0b53a0 0x7f1b6c0b5810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:16:40.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.846+0000 7f6325c48640 1 --2- 172.21.15.27:0/4086664624 >> v2:172.21.15.45:3300/0 conn(0x7f631801db50 0x7f631800ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:16:40.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.846+0000 7f75b0cb7640 1 -- 172.21.15.27:0/771837467 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f75ac15db00 msgr2=0x7f75ac06d0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:40.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.846+0000 7f75b0cb7640 1 --2- 172.21.15.27:0/771837467 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f75ac15db00 0x7f75ac06d0a0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f75ac0b3a50 tx=0x7f759c02a1e0 comp rx=0 tx=0).stop 2024-09-12T03:16:40.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.846+0000 7f262effd640 1 -- 172.21.15.27:0/3721016814 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2628007460 con 0x7f2628004a60 2024-09-12T03:16:40.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.847+0000 7f75b0cb7640 1 -- 172.21.15.27:0/771837467 shutdown_connections 2024-09-12T03:16:40.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.847+0000 7f75b0cb7640 1 --2- 172.21.15.27:0/771837467 >> v2:172.21.15.45:3300/0 conn(0x7f75ac06d5e0 0x7f75ac06e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:40.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.847+0000 7f75b0cb7640 1 --2- 172.21.15.27:0/771837467 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f75ac15db00 0x7f75ac06d0a0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:40.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.847+0000 7f75b0cb7640 1 --2- 172.21.15.27:0/771837467 >> v2:172.21.15.27:3300/0 conn(0x7f75ac071030 0x7f75ac0cd750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:40.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.847+0000 7f75b0cb7640 1 -- 172.21.15.27:0/771837467 >> 172.21.15.27:0/771837467 conn(0x7f75ac06bd10 msgr2=0x7f75ac06c0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:40.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.847+0000 7f0230080640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f022806d0a0 0x7f0228157070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:40.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.847+0000 7f0230080640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f022806e970 0x7f0228158a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:40.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.847+0000 7f0230080640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f022815db00 0x7f0228158fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:40.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.847+0000 7f022de1c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f022806d0a0 0x7f0228157070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:40.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.847+0000 7f022de1c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f022806d0a0 0x7f0228157070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.27:60924/0 (socket says 172.21.15.27:60924) 2024-09-12T03:16:40.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.847+0000 7f022d61b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f022806e970 0x7f0228158a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:40.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.847+0000 7f022d61b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f022806e970 0x7f0228158a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33314/0 (socket says 172.21.15.27:33314) 2024-09-12T03:16:40.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.847+0000 7f8cf4acf640 1 -- 172.21.15.27:0/935242691 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f8cf015d070 msgr2=0x7f8cd8005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:40.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.847+0000 7f8cf4acf640 1 --2- 172.21.15.27:0/935242691 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f8cf015d070 0x7f8cd8005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:40.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.847+0000 7f0230080640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f022806dbb0 con 0x7f022806e970 2024-09-12T03:16:40.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.847+0000 7f022de1c640 1 -- 172.21.15.27:0/2507381626 learned_addr learned my addr 172.21.15.27:0/2507381626 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:16:40.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.847+0000 7f0230080640 1 -- 172.21.15.27:0/2507381626 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f022806da30 con 0x7f022806d0a0 2024-09-12T03:16:40.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.847+0000 7f0230080640 1 -- 172.21.15.27:0/2507381626 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f0228071030 con 0x7f022815db00 2024-09-12T03:16:40.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.847+0000 7f8cf4acf640 1 -- 172.21.15.27:0/935242691 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8cd80043b0 msgr2=0x7f8cd8004880 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-12T03:16:40.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.847+0000 7f8cf4acf640 1 --2- 172.21.15.27:0/935242691 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8cd80043b0 0x7f8cd8004880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:40.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.847+0000 7f1b6ad9d640 1 --2- 172.21.15.27:0/1720458839 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1b6c06dbc0 0x7f1b6c06e030 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f1b64009d90 tx=0x7f1b6402a590 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e57897d98a93eee3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:40.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.847+0000 7f022e61d640 1 --2- 172.21.15.27:0/2507381626 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f022815db00 0x7f0228158fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:40.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.847+0000 7fb032442640 1 Processor -- start 2024-09-12T03:16:40.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.847+0000 7f8cf4acf640 1 -- 172.21.15.27:0/935242691 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cd8003520 con 0x7f8cd80039b0 2024-09-12T03:16:40.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.847+0000 7f6325447640 1 --2- 172.21.15.27:0/4086664624 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6318006c40 0x7f6318050120 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f632006afe0 tx=0x7f63200a6160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8dbdbdc5111dbfd server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:40.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.847+0000 7f75b0cb7640 1 -- 172.21.15.27:0/771837467 shutdown_connections 2024-09-12T03:16:40.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.847+0000 7f75b0cb7640 1 -- 172.21.15.27:0/771837467 wait complete. 2024-09-12T03:16:40.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.847+0000 7f262effd640 1 --2- 172.21.15.27:0/3721016814 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2628004a60 0x7f262800bbf0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f2630000b60 tx=0x7f26300aa5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfc03721c9c927b5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:40.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.848+0000 7f022de1c640 1 -- 172.21.15.27:0/2507381626 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f022815db00 msgr2=0x7f0228158fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:40.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.848+0000 7f022de1c640 1 --2- 172.21.15.27:0/2507381626 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f022815db00 0x7f0228158fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:40.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.848+0000 7f022de1c640 1 -- 172.21.15.27:0/2507381626 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f022806e970 msgr2=0x7f0228158a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:40.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.848+0000 7f022de1c640 1 --2- 172.21.15.27:0/2507381626 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f022806e970 0x7f0228158a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:40.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.848+0000 7f022de1c640 1 -- 172.21.15.27:0/2507381626 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02280ae210 con 0x7f022806d0a0 2024-09-12T03:16:40.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.848+0000 7f1b6959a640 1 -- 172.21.15.27:0/1720458839 <== mon.1 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b64007740 con 0x7f1b6c06dbc0 2024-09-12T03:16:40.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.848+0000 7f1b6959a640 1 -- 172.21.15.27:0/1720458839 <== mon.1 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b640078e0 con 0x7f1b6c06dbc0 2024-09-12T03:16:40.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.848+0000 7f6324c46640 1 -- 172.21.15.27:0/4086664624 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63200b3070 con 0x7f6318006c40 2024-09-12T03:16:40.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.848+0000 7f6324c46640 1 -- 172.21.15.27:0/4086664624 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63200ae440 con 0x7f6318006c40 2024-09-12T03:16:40.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.848+0000 7f022de1c640 1 --2- 172.21.15.27:0/2507381626 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f022806d0a0 0x7f0228157070 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f022400b780 tx=0x7f022400bc40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:40.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.848+0000 7f262e7fc640 1 -- 172.21.15.27:0/3721016814 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26300b1600 con 0x7f2628004a60 2024-09-12T03:16:40.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.848+0000 7f262e7fc640 1 -- 172.21.15.27:0/3721016814 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26300b17a0 con 0x7f2628004a60 2024-09-12T03:16:40.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.848+0000 7f262e7fc640 1 -- 172.21.15.27:0/3721016814 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26300b1c10 con 0x7f2628004a60 2024-09-12T03:16:40.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.848+0000 7f6324c46640 1 -- 172.21.15.27:0/4086664624 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63200ae8b0 con 0x7f6318006c40 2024-09-12T03:16:40.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.848+0000 7f021e7fc640 1 -- 172.21.15.27:0/2507381626 <== mon.1 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02240060d0 con 0x7f022806d0a0 2024-09-12T03:16:40.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.849+0000 7f1b6959a640 1 -- 172.21.15.27:0/1720458839 <== mon.1 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b64007cb0 con 0x7f1b6c06dbc0 2024-09-12T03:16:40.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.849+0000 7f1b70c5e640 1 -- 172.21.15.27:0/1720458839 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1b6c06dbc0 msgr2=0x7f1b6c06e030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:40.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.849+0000 7f1b70c5e640 1 --2- 172.21.15.27:0/1720458839 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1b6c06dbc0 0x7f1b6c06e030 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f1b64009d90 tx=0x7f1b6402a590 comp rx=0 tx=0).stop 2024-09-12T03:16:40.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.848+0000 7f0230080640 1 -- 172.21.15.27:0/2507381626 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02280ae4f0 con 0x7f022806d0a0 2024-09-12T03:16:40.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.848+0000 7f021e7fc640 1 -- 172.21.15.27:0/2507381626 <== mon.1 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0224010070 con 0x7f022806d0a0 2024-09-12T03:16:40.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.848+0000 7f021e7fc640 1 -- 172.21.15.27:0/2507381626 <== mon.1 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02240064b0 con 0x7f022806d0a0 2024-09-12T03:16:40.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.848+0000 7f0230080640 1 -- 172.21.15.27:0/2507381626 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02280ab6d0 con 0x7f022806d0a0 2024-09-12T03:16:40.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.849+0000 7f2635e1e640 1 -- 172.21.15.27:0/3721016814 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2628004a60 msgr2=0x7f262800bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:40.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.849+0000 7f2635e1e640 1 --2- 172.21.15.27:0/3721016814 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2628004a60 0x7f262800bbf0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f2630000b60 tx=0x7f26300aa5d0 comp rx=0 tx=0).stop 2024-09-12T03:16:40.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.847+0000 7fb032442640 1 -- start start 2024-09-12T03:16:40.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.849+0000 7f8cf4acf640 1 --2- 172.21.15.27:0/935242691 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8cd80039b0 0x7f8cd8003d90 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f8ce0009860 tx=0x7f8cd8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=208c9756da15a730 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:40.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.847+0000 7f0e66ffd640 1 --2- 172.21.15.27:0/2777232213 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e60007190 0x7f0e600581c0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f0e5c009870 tx=0x7f0e5c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19147a40ba7af52b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:40.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.848+0000 7f0e65ffb640 1 -- 172.21.15.27:0/2777232213 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e5c037070 con 0x7f0e60007190 2024-09-12T03:16:40.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.848+0000 7f0e65ffb640 1 -- 172.21.15.27:0/2777232213 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e5c02ae10 con 0x7f0e60007190 2024-09-12T03:16:40.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.848+0000 7f0e65ffb640 1 -- 172.21.15.27:0/2777232213 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e5c032660 con 0x7f0e60007190 2024-09-12T03:16:40.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.849+0000 7f8cef7fe640 1 -- 172.21.15.27:0/935242691 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ce0037070 con 0x7f8cd80039b0 2024-09-12T03:16:40.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.849+0000 7f1b70c5e640 1 -- 172.21.15.27:0/1720458839 shutdown_connections 2024-09-12T03:16:40.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.849+0000 7f1b70c5e640 1 --2- 172.21.15.27:0/1720458839 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1b6c06dbc0 0x7f1b6c06e030 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:40.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.849+0000 7f1b70c5e640 1 --2- 172.21.15.27:0/1720458839 >> v2:172.21.15.27:3300/0 conn(0x7f1b6c0b53a0 0x7f1b6c0b5810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:40.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.849+0000 7f1b70c5e640 1 -- 172.21.15.27:0/1720458839 >> 172.21.15.27:0/1720458839 conn(0x7f1b6c0b0b50 msgr2=0x7f1b6c0b0f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:40.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.849+0000 7f2635e1e640 1 -- 172.21.15.27:0/3721016814 shutdown_connections 2024-09-12T03:16:40.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.849+0000 7f2635e1e640 1 --2- 172.21.15.27:0/3721016814 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2628004a60 0x7f262800bbf0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:40.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.849+0000 7f2635e1e640 1 -- 172.21.15.27:0/3721016814 >> 172.21.15.27:0/3721016814 conn(0x7f262801ddb0 msgr2=0x7f2628054a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:40.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.849+0000 7f6327eac640 1 -- 172.21.15.27:0/4086664624 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6318006c40 msgr2=0x7f6318050120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:40.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.849+0000 7f6327eac640 1 --2- 172.21.15.27:0/4086664624 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6318006c40 0x7f6318050120 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f632006afe0 tx=0x7f63200a6160 comp rx=0 tx=0).stop 2024-09-12T03:16:40.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.849+0000 7f8cef7fe640 1 -- 172.21.15.27:0/935242691 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ce002aa80 con 0x7f8cd80039b0 2024-09-12T03:16:40.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.849+0000 7f1b70c5e640 1 -- 172.21.15.27:0/1720458839 shutdown_connections 2024-09-12T03:16:40.886 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.849+0000 7f75b0cb7640 1 Processor -- start 2024-09-12T03:16:40.886 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.850+0000 7f0e67fff640 1 -- 172.21.15.27:0/2777232213 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e60007190 msgr2=0x7f0e600581c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:40.886 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.850+0000 7f75b0cb7640 1 -- start start 2024-09-12T03:16:40.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.850+0000 7f0e67fff640 1 --2- 172.21.15.27:0/2777232213 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e60007190 0x7f0e600581c0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f0e5c009870 tx=0x7f0e5c02a180 comp rx=0 tx=0).stop 2024-09-12T03:16:40.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.850+0000 7fb032442640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb02c06d950 0x7fb02c14df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:40.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.850+0000 7fb032442640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb02c0cfde0 0x7fb02c1566a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:40.888 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.850+0000 7f0e67fff640 1 -- 172.21.15.27:0/2777232213 shutdown_connections 2024-09-12T03:16:40.888 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.850+0000 7f0e67fff640 1 --2- 172.21.15.27:0/2777232213 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f0e60037bd0 0x7f0e60057230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:40.888 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.850+0000 7f0e67fff640 1 --2- 172.21.15.27:0/2777232213 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e60007190 0x7f0e600581c0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:40.888 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.850+0000 7f0e67fff640 1 -- 172.21.15.27:0/2777232213 >> 172.21.15.27:0/2777232213 conn(0x7f0e6003a4f0 msgr2=0x7f0e6003a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:40.888 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.850+0000 7fb032442640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb02c15a270 0x7fb02c158ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:40.889 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.850+0000 7fb032442640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fb02c0d0d30 con 0x7fb02c06d950 2024-09-12T03:16:40.889 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.850+0000 7f2635e1e640 1 -- 172.21.15.27:0/3721016814 shutdown_connections 2024-09-12T03:16:40.889 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.850+0000 7f2635e1e640 1 -- 172.21.15.27:0/3721016814 wait complete. 2024-09-12T03:16:40.889 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.850+0000 7fb032442640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb02c06f390 con 0x7fb02c15a270 2024-09-12T03:16:40.889 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.850+0000 7fb032442640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fb02c0d0eb0 con 0x7fb02c0cfde0 2024-09-12T03:16:40.890 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.850+0000 7f8cf6d33640 1 -- 172.21.15.27:0/935242691 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8cd80039b0 msgr2=0x7f8cd8003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:40.890 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.850+0000 7f8cf6d33640 1 --2- 172.21.15.27:0/935242691 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8cd80039b0 0x7f8cd8003d90 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f8ce0009860 tx=0x7f8cd8005830 comp rx=0 tx=0).stop 2024-09-12T03:16:40.890 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.850+0000 7f8cf52d0640 1 -- 172.21.15.27:0/935242691 reap_dead start 2024-09-12T03:16:40.890 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.849+0000 7f1b70c5e640 1 -- 172.21.15.27:0/1720458839 wait complete. 2024-09-12T03:16:40.890 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.850+0000 7fb031440640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb02c06d950 0x7fb02c14df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:40.891 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.850+0000 7f6327eac640 1 -- 172.21.15.27:0/4086664624 shutdown_connections 2024-09-12T03:16:40.891 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.850+0000 7f6327eac640 1 --2- 172.21.15.27:0/4086664624 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6318006c40 0x7f6318050120 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:40.891 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.850+0000 7f6327eac640 1 --2- 172.21.15.27:0/4086664624 >> v2:172.21.15.45:3300/0 conn(0x7f631801db50 0x7f631800ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:40.891 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.850+0000 7f6327eac640 1 -- 172.21.15.27:0/4086664624 >> 172.21.15.27:0/4086664624 conn(0x7f63180360c0 msgr2=0x7f63180364d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:40.891 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.850+0000 7f8cf6d33640 1 -- 172.21.15.27:0/935242691 shutdown_connections 2024-09-12T03:16:40.892 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.850+0000 7f8cf6d33640 1 -- 172.21.15.27:0/935242691 >> 172.21.15.27:0/935242691 conn(0x7f8cf006bd10 msgr2=0x7f8cf006c0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:40.892 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.850+0000 7fb030c3f640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb02c0cfde0 0x7fb02c1566a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:40.892 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.850+0000 7fb030c3f640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb02c0cfde0 0x7fb02c1566a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3301/0 says I am v2:172.21.15.27:56394/0 (socket says 172.21.15.27:56394) 2024-09-12T03:16:40.892 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.850+0000 7fb030c3f640 1 -- 172.21.15.27:0/3478031545 learned_addr learned my addr 172.21.15.27:0/3478031545 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:16:40.892 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.851+0000 7fb030c3f640 1 -- 172.21.15.27:0/3478031545 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb02c15a270 msgr2=0x7fb02c158ea0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-12T03:16:40.892 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.851+0000 7fb030c3f640 1 --2- 172.21.15.27:0/3478031545 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb02c15a270 0x7fb02c158ea0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:40.892 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.851+0000 7fb030c3f640 1 -- 172.21.15.27:0/3478031545 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb02c06d950 msgr2=0x7fb02c14df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:40.892 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.851+0000 7fb030c3f640 1 --2- 172.21.15.27:0/3478031545 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb02c06d950 0x7fb02c14df50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:40.892 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.851+0000 7fb030c3f640 1 -- 172.21.15.27:0/3478031545 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb02c14e490 con 0x7fb02c0cfde0 2024-09-12T03:16:40.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.851+0000 7f0e67fff640 1 -- 172.21.15.27:0/2777232213 shutdown_connections 2024-09-12T03:16:40.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.851+0000 7f0e67fff640 1 -- 172.21.15.27:0/2777232213 wait complete. 2024-09-12T03:16:40.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.851+0000 7f0230080640 1 -- 172.21.15.27:0/2507381626 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0210005290 con 0x7f022806d0a0 2024-09-12T03:16:40.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.851+0000 7fb031440640 1 --2- 172.21.15.27:0/3478031545 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb02c06d950 0x7fb02c14df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:16:40.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.851+0000 7f021e7fc640 1 -- 172.21.15.27:0/2507381626 <== mon.1 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0224006650 con 0x7f022806d0a0 2024-09-12T03:16:40.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.851+0000 7fb030c3f640 1 --2- 172.21.15.27:0/3478031545 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb02c0cfde0 0x7fb02c1566a0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fb01c00b450 tx=0x7fb01c00b910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:40.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.851+0000 7f75b0cb7640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f75ac06d5e0 0x7f75ac1503e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:40.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.851+0000 7fb021ffb640 1 -- 172.21.15.27:0/3478031545 <== mon.2 v2:172.21.15.45:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb01c015020 con 0x7fb02c0cfde0 2024-09-12T03:16:40.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.851+0000 7fb032442640 1 -- 172.21.15.27:0/3478031545 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb02c156be0 con 0x7fb02c0cfde0 2024-09-12T03:16:40.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.851+0000 7f75b0cb7640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f75ac071030 0x7f75ac1524e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:40.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.851+0000 7f75b0cb7640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f75ac15db00 0x7f75ac152a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:40.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.851+0000 7f6327eac640 1 -- 172.21.15.27:0/4086664624 shutdown_connections 2024-09-12T03:16:40.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.851+0000 7f6327eac640 1 -- 172.21.15.27:0/4086664624 wait complete. 2024-09-12T03:16:40.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.852+0000 7fb032442640 1 -- 172.21.15.27:0/3478031545 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fb02c1583c0 con 0x7fb02c0cfde0 2024-09-12T03:16:40.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.852+0000 7f021e7fc640 1 --2- 172.21.15.27:0/2507381626 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f020c06be30 0x7f020c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:40.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.852+0000 7fb021ffb640 1 -- 172.21.15.27:0/3478031545 <== mon.2 v2:172.21.15.45:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb01c011890 con 0x7fb02c0cfde0 2024-09-12T03:16:40.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.852+0000 7f8cf6d33640 1 -- 172.21.15.27:0/935242691 shutdown_connections 2024-09-12T03:16:40.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.852+0000 7f2635e1e640 1 Processor -- start 2024-09-12T03:16:40.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.852+0000 7f021e7fc640 1 -- 172.21.15.27:0/2507381626 <== mon.1 v2:172.21.15.45:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f022408a630 con 0x7f022806d0a0 2024-09-12T03:16:40.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.852+0000 7f2635e1e640 1 -- start start 2024-09-12T03:16:40.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.852+0000 7f022d61b640 1 --2- 172.21.15.27:0/2507381626 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f020c06be30 0x7f020c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:40.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.852+0000 7f1b70c5e640 1 Processor -- start 2024-09-12T03:16:40.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.852+0000 7f1b70c5e640 1 -- start start 2024-09-12T03:16:40.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.852+0000 7fb021ffb640 1 -- 172.21.15.27:0/3478031545 <== mon.2 v2:172.21.15.45:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb01c011c80 con 0x7fb02c0cfde0 2024-09-12T03:16:40.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.851+0000 7f75aa59c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f75ac06d5e0 0x7f75ac1503e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:40.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.852+0000 7f75aa59c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f75ac06d5e0 0x7f75ac1503e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.27:60926/0 (socket says 172.21.15.27:60926) 2024-09-12T03:16:40.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.852+0000 7f75aa59c640 1 -- 172.21.15.27:0/775156009 learned_addr learned my addr 172.21.15.27:0/775156009 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:16:40.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.852+0000 7f75b0cb7640 1 -- 172.21.15.27:0/775156009 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f75ac06db90 con 0x7f75ac15db00 2024-09-12T03:16:40.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.852+0000 7f75b0cb7640 1 -- 172.21.15.27:0/775156009 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f75ac06da10 con 0x7f75ac06d5e0 2024-09-12T03:16:40.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.852+0000 7f75b0cb7640 1 -- 172.21.15.27:0/775156009 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f75ac06e7a0 con 0x7f75ac071030 2024-09-12T03:16:40.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.852+0000 7f75aad9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f75ac15db00 0x7f75ac152a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:40.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.852+0000 7f75a9d9b640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f75ac071030 0x7f75ac1524e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:40.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.853+0000 7f75aa59c640 1 -- 172.21.15.27:0/775156009 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f75ac071030 msgr2=0x7f75ac1524e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:40.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.853+0000 7f75aa59c640 1 --2- 172.21.15.27:0/775156009 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f75ac071030 0x7f75ac1524e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:40.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.853+0000 7f75aa59c640 1 -- 172.21.15.27:0/775156009 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f75ac15db00 msgr2=0x7f75ac152a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:40.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.853+0000 7f75aa59c640 1 --2- 172.21.15.27:0/775156009 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f75ac15db00 0x7f75ac152a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:40.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.853+0000 7f75aa59c640 1 -- 172.21.15.27:0/775156009 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75ac14d740 con 0x7f75ac06d5e0 2024-09-12T03:16:40.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.853+0000 7f75a9d9b640 1 --2- 172.21.15.27:0/775156009 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f75ac071030 0x7f75ac1524e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:16:40.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.853+0000 7f75aa59c640 1 --2- 172.21.15.27:0/775156009 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f75ac06d5e0 0x7f75ac1503e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f75a400cf60 tx=0x7f75a400cf90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:40.902 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.853+0000 7f022d61b640 1 --2- 172.21.15.27:0/2507381626 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f020c06be30 0x7f020c06e2f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f022003e000 tx=0x7f0220039000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:40.902 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.853+0000 7f0e67fff640 1 Processor -- start 2024-09-12T03:16:40.902 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.853+0000 7f0e67fff640 1 -- start start 2024-09-12T03:16:40.902 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.853+0000 7fb032442640 1 -- 172.21.15.27:0/3478031545 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb00c005290 con 0x7fb02c0cfde0 2024-09-12T03:16:40.902 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.853+0000 7f8cf6d33640 1 -- 172.21.15.27:0/935242691 wait complete. 2024-09-12T03:16:40.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.853+0000 7f2635e1e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2628004a60 0x7f26280d9000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:40.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.854+0000 7f2635e1e640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f262800a460 0x7f26280e11c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:40.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.854+0000 7f262f7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2628004a60 0x7f26280d9000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:40.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.854+0000 7f262f7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2628004a60 0x7f26280d9000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33334/0 (socket says 172.21.15.27:33334) 2024-09-12T03:16:40.904 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.854+0000 7f6327eac640 1 Processor -- start 2024-09-12T03:16:40.904 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.854+0000 7f2635e1e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2628037040 0x7f262804d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:40.904 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.854+0000 7f6327eac640 1 -- start start 2024-09-12T03:16:40.904 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.854+0000 7f1b70c5e640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1b6c06dbc0 0x7f1b6c152e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:40.905 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.854+0000 7f2635e1e640 1 -- 172.21.15.27:0/647522573 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f2628006a60 con 0x7f2628004a60 2024-09-12T03:16:40.905 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.854+0000 7f262f7fe640 1 -- 172.21.15.27:0/647522573 learned_addr learned my addr 172.21.15.27:0/647522573 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:16:40.905 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.854+0000 7f2635e1e640 1 -- 172.21.15.27:0/647522573 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f26280068e0 con 0x7f2628037040 2024-09-12T03:16:40.905 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.854+0000 7f2635e1e640 1 -- 172.21.15.27:0/647522573 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f26280dec10 con 0x7f262800a460 2024-09-12T03:16:40.905 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.854+0000 7f262effd640 1 --2- 172.21.15.27:0/647522573 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f262800a460 0x7f26280e11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:40.905 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.854+0000 7f1b70c5e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1b6c0b53a0 0x7f1b6c1558c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:40.906 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.854+0000 7f262ffff640 1 --2- 172.21.15.27:0/647522573 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2628037040 0x7f262804d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:40.906 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.854+0000 7fb021ffb640 1 -- 172.21.15.27:0/3478031545 <== mon.2 v2:172.21.15.45:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb01c018020 con 0x7fb02c0cfde0 2024-09-12T03:16:40.906 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.854+0000 7f1b70c5e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1b6c159990 0x7f1b6c155e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:40.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.854+0000 7f262effd640 1 -- 172.21.15.27:0/647522573 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2628037040 msgr2=0x7f262804d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:40.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.854+0000 7f262effd640 1 --2- 172.21.15.27:0/647522573 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2628037040 0x7f262804d5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:40.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.854+0000 7f262effd640 1 -- 172.21.15.27:0/647522573 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2628004a60 msgr2=0x7f26280d9000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:40.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.854+0000 7f1b6a59c640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1b6c06dbc0 0x7f1b6c152e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:40.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.854+0000 7f1b6a59c640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1b6c06dbc0 0x7f1b6c152e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3301/0 says I am v2:172.21.15.27:56434/0 (socket says 172.21.15.27:56434) 2024-09-12T03:16:40.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.854+0000 7f262effd640 1 --2- 172.21.15.27:0/647522573 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2628004a60 0x7f26280d9000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:40.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.854+0000 7f262effd640 1 -- 172.21.15.27:0/647522573 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26280e1e30 con 0x7f262800a460 2024-09-12T03:16:40.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.854+0000 7f262ffff640 1 --2- 172.21.15.27:0/647522573 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2628037040 0x7f262804d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:16:40.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.854+0000 7f0e67fff640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f0e60007190 0x7f0e600e3130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:40.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.854+0000 7f1b6a59c640 1 -- 172.21.15.27:0/116456440 learned_addr learned my addr 172.21.15.27:0/116456440 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:16:40.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.854+0000 7f1b69d9b640 1 --2- 172.21.15.27:0/116456440 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1b6c0b53a0 0x7f1b6c1558c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:40.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.855+0000 7f1b70c5e640 1 -- 172.21.15.27:0/116456440 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f1b6c159d70 con 0x7f1b6c159990 2024-09-12T03:16:40.909 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.855+0000 7f0e67fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e60037bd0 0x7f0e6003dc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:40.909 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.855+0000 7f1b6ad9d640 1 --2- 172.21.15.27:0/116456440 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1b6c159990 0x7f1b6c155e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:40.909 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.855+0000 7f1b70c5e640 1 -- 172.21.15.27:0/116456440 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f1b6c0b48b0 con 0x7f1b6c0b53a0 2024-09-12T03:16:40.909 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.855+0000 7f262effd640 1 --2- 172.21.15.27:0/647522573 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f262800a460 0x7f26280e11c0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f2630069370 tx=0x7f26300ac000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:40.910 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.855+0000 7f0e66ffd640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f0e60007190 0x7f0e600e3130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:40.910 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.855+0000 7fb021ffb640 1 --2- 172.21.15.27:0/3478031545 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fb01006be30 0x7fb01006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:40.910 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.855+0000 7f0e66ffd640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f0e60007190 0x7f0e600e3130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3301/0 says I am v2:172.21.15.27:56442/0 (socket says 172.21.15.27:56442) 2024-09-12T03:16:40.911 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.855+0000 7f8cf6d33640 1 Processor -- start 2024-09-12T03:16:40.911 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.855+0000 7f0e667fc640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e60037bd0 0x7f0e6003dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:40.911 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.855+0000 7f0e667fc640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e60037bd0 0x7f0e6003dc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33358/0 (socket says 172.21.15.27:33358) 2024-09-12T03:16:40.911 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.855+0000 7fb021ffb640 1 -- 172.21.15.27:0/3478031545 <== mon.2 v2:172.21.15.45:3301/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fb01c08afc0 con 0x7fb02c0cfde0 2024-09-12T03:16:40.912 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.855+0000 7f1b70c5e640 1 -- 172.21.15.27:0/116456440 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f1b6c1f9210 con 0x7f1b6c06dbc0 2024-09-12T03:16:40.912 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.855+0000 7f8cf6d33640 1 -- start start 2024-09-12T03:16:40.912 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.855+0000 7fb031440640 1 --2- 172.21.15.27:0/3478031545 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fb01006be30 0x7fb01006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:40.912 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.855+0000 7f0e66ffd640 1 -- 172.21.15.27:0/1957765297 learned_addr learned my addr 172.21.15.27:0/1957765297 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:16:40.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.855+0000 7f6327eac640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f6318005ff0 0x7f63180c53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:40.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.855+0000 7f1b6a59c640 1 -- 172.21.15.27:0/116456440 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1b6c0b53a0 msgr2=0x7f1b6c1558c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:40.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.855+0000 7f1b6a59c640 1 --2- 172.21.15.27:0/116456440 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1b6c0b53a0 0x7f1b6c1558c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:40.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.855+0000 7f1b6a59c640 1 -- 172.21.15.27:0/116456440 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1b6c159990 msgr2=0x7f1b6c155e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:40.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.855+0000 7f1b6a59c640 1 --2- 172.21.15.27:0/116456440 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1b6c159990 0x7f1b6c155e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:40.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.856+0000 7f6327eac640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f631801db50 0x7f6318031540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:40.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.855+0000 7f1b6a59c640 1 -- 172.21.15.27:0/116456440 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b6c157c90 con 0x7f1b6c06dbc0 2024-09-12T03:16:40.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.856+0000 7f0e67fff640 1 --2- 172.21.15.27:0/1957765297 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0e600ea950 0x7f0e600e3c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:40.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.856+0000 7f6327eac640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f63180d88b0 0x7f63180d8d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:40.915 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.856+0000 7f6327eac640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f63180507e0 con 0x7f631801db50 2024-09-12T03:16:40.915 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.856+0000 7fb031440640 1 --2- 172.21.15.27:0/3478031545 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fb01006be30 0x7fb01006e2f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fb028033000 tx=0x7fb028008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:40.915 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.856+0000 7f6327eac640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f6318050660 con 0x7f63180d88b0 2024-09-12T03:16:40.915 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.856+0000 7f6327eac640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f631800bbf0 con 0x7f6318005ff0 2024-09-12T03:16:40.915 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.856+0000 7f6325c48640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f6318005ff0 0x7f63180c53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:40.915 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.856+0000 7f6325447640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f631801db50 0x7f6318031540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:40.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.856+0000 7f6325447640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f631801db50 0x7f6318031540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33364/0 (socket says 172.21.15.27:33364) 2024-09-12T03:16:40.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.856+0000 7f6325447640 1 -- 172.21.15.27:0/1880598132 learned_addr learned my addr 172.21.15.27:0/1880598132 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:16:40.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.856+0000 7f1b6a59c640 1 --2- 172.21.15.27:0/116456440 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1b6c06dbc0 0x7f1b6c152e90 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f1b5c00c410 tx=0x7f1b5c00c8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:40.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.856+0000 7f6326449640 1 --2- 172.21.15.27:0/1880598132 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f63180d88b0 0x7f63180d8d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:40.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.856+0000 7f6325447640 1 -- 172.21.15.27:0/1880598132 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f6318005ff0 msgr2=0x7f63180c53c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:40.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.856+0000 7f6325447640 1 --2- 172.21.15.27:0/1880598132 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f6318005ff0 0x7f63180c53c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:40.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.856+0000 7f6325447640 1 -- 172.21.15.27:0/1880598132 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f63180d88b0 msgr2=0x7f63180d8d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:40.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.856+0000 7f6325447640 1 --2- 172.21.15.27:0/1880598132 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f63180d88b0 0x7f63180d8d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:40.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.856+0000 7f6325447640 1 -- 172.21.15.27:0/1880598132 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63180de780 con 0x7f631801db50 2024-09-12T03:16:40.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.856+0000 7f6325447640 1 --2- 172.21.15.27:0/1880598132 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f631801db50 0x7f6318031540 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f6320051220 tx=0x7f63200a8070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:40.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.856+0000 7f0e677fe640 1 --2- 172.21.15.27:0/1957765297 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0e600ea950 0x7f0e600e3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:40.918 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.856+0000 7f8cf6d33640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8cd80039b0 0x7f8cf0162030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:40.918 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.856+0000 7f63167fc640 1 -- 172.21.15.27:0/1880598132 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63200b3050 con 0x7f631801db50 2024-09-12T03:16:40.918 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.856+0000 7f63167fc640 1 -- 172.21.15.27:0/1880598132 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63200b4040 con 0x7f631801db50 2024-09-12T03:16:40.918 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.856+0000 7f63167fc640 1 -- 172.21.15.27:0/1880598132 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63200adbc0 con 0x7f631801db50 2024-09-12T03:16:40.918 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.856+0000 7f8cf6d33640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8cf00cc200 0x7f8cf014e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:40.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.856+0000 7f0e67fff640 1 -- 172.21.15.27:0/1957765297 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f0e60007570 con 0x7f0e60037bd0 2024-09-12T03:16:40.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.856+0000 7f0e67fff640 1 -- 172.21.15.27:0/1957765297 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f0e60057770 con 0x7f0e600ea950 2024-09-12T03:16:40.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.856+0000 7f0e67fff640 1 -- 172.21.15.27:0/1957765297 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f0e600076f0 con 0x7f0e60007190 2024-09-12T03:16:40.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.856+0000 7f6327eac640 1 -- 172.21.15.27:0/1880598132 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63180dea60 con 0x7f631801db50 2024-09-12T03:16:40.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.856+0000 7f8cf6d33640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f8cf00ca980 0x7f8cf00c7d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:40.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.857+0000 7f8cf6d33640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f8cf015d450 con 0x7f8cf00cc200 2024-09-12T03:16:40.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.857+0000 7f6327eac640 1 -- 172.21.15.27:0/1880598132 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63180df020 con 0x7f631801db50 2024-09-12T03:16:40.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.857+0000 7f8cf6d33640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f8cf00b4ea0 con 0x7f8cd80039b0 2024-09-12T03:16:40.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.857+0000 7f8cf4acf640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8cd80039b0 0x7f8cf0162030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:40.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.857+0000 7f8cf4acf640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8cd80039b0 0x7f8cf0162030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.27:60964/0 (socket says 172.21.15.27:60964) 2024-09-12T03:16:40.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.857+0000 7f8cf4acf640 1 -- 172.21.15.27:0/138775458 learned_addr learned my addr 172.21.15.27:0/138775458 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:16:40.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.857+0000 7f8cf6d33640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f8cf006d720 con 0x7f8cf00ca980 2024-09-12T03:16:40.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.857+0000 7f8cf52d0640 1 --2- 172.21.15.27:0/138775458 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f8cf00ca980 0x7f8cf00c7d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:40.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.857+0000 7f0e667fc640 1 -- 172.21.15.27:0/1957765297 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f0e60007190 msgr2=0x7f0e600e3130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:40.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.857+0000 7f759affd640 1 -- 172.21.15.27:0/775156009 <== mon.1 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75a401a070 con 0x7f75ac06d5e0 2024-09-12T03:16:40.923 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.857+0000 7f0e667fc640 1 --2- 172.21.15.27:0/1957765297 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f0e60007190 0x7f0e600e3130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:40.923 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.857+0000 7f0e667fc640 1 -- 172.21.15.27:0/1957765297 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0e600ea950 msgr2=0x7f0e600e3c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:40.923 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.857+0000 7f0e667fc640 1 --2- 172.21.15.27:0/1957765297 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0e600ea950 0x7f0e600e3c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:40.923 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.857+0000 7f0e667fc640 1 -- 172.21.15.27:0/1957765297 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e600e4d80 con 0x7f0e60037bd0 2024-09-12T03:16:40.923 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.857+0000 7f8ceffff640 1 --2- 172.21.15.27:0/138775458 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8cf00cc200 0x7f8cf014e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:40.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.857+0000 7f759affd640 1 -- 172.21.15.27:0/775156009 <== mon.1 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75a400d840 con 0x7f75ac06d5e0 2024-09-12T03:16:40.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.857+0000 7f759affd640 1 -- 172.21.15.27:0/775156009 <== mon.1 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75a400dbf0 con 0x7f75ac06d5e0 2024-09-12T03:16:40.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.857+0000 7f75b0cb7640 1 -- 172.21.15.27:0/775156009 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75ac14da20 con 0x7f75ac06d5e0 2024-09-12T03:16:40.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.857+0000 7f75b0cb7640 1 -- 172.21.15.27:0/775156009 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75ac14dfe0 con 0x7f75ac06d5e0 2024-09-12T03:16:40.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.857+0000 7f021e7fc640 1 -- 172.21.15.27:0/2507381626 <== mon.1 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f0224058fd0 con 0x7f022806d0a0 2024-09-12T03:16:40.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.857+0000 7f8cf52d0640 1 -- 172.21.15.27:0/138775458 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8cd80039b0 msgr2=0x7f8cf0162030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:40.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.857+0000 7f8cf52d0640 1 --2- 172.21.15.27:0/138775458 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8cd80039b0 0x7f8cf0162030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:40.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.857+0000 7f8cf52d0640 1 -- 172.21.15.27:0/138775458 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8cf00cc200 msgr2=0x7f8cf014e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:40.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.857+0000 7f8cf52d0640 1 --2- 172.21.15.27:0/138775458 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8cf00cc200 0x7f8cf014e670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:40.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.858+0000 7f8cf52d0640 1 -- 172.21.15.27:0/138775458 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cf00cb160 con 0x7f8cf00ca980 2024-09-12T03:16:40.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.858+0000 7f8cf52d0640 1 --2- 172.21.15.27:0/138775458 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f8cf00ca980 0x7f8cf00c7d30 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f8cf0073210 tx=0x7f8ce8001d70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:40.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.857+0000 7f0e667fc640 1 --2- 172.21.15.27:0/1957765297 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e60037bd0 0x7f0e6003dc60 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f0e5800b750 tx=0x7f0e5800bc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:40.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.858+0000 7f0e4f7fe640 1 -- 172.21.15.27:0/1957765297 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e58019070 con 0x7f0e60037bd0 2024-09-12T03:16:40.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.858+0000 7f6327eac640 1 -- 172.21.15.27:0/1880598132 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6318050120 con 0x7f631801db50 2024-09-12T03:16:40.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.858+0000 7f0e67fff640 1 -- 172.21.15.27:0/1957765297 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e6003e1a0 con 0x7f0e60037bd0 2024-09-12T03:16:40.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.858+0000 7f0e67fff640 1 -- 172.21.15.27:0/1957765297 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e600e2930 con 0x7f0e60037bd0 2024-09-12T03:16:40.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.858+0000 7f0e4f7fe640 1 -- 172.21.15.27:0/1957765297 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e5800c720 con 0x7f0e60037bd0 2024-09-12T03:16:40.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.858+0000 7f0e4f7fe640 1 -- 172.21.15.27:0/1957765297 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e5800cb30 con 0x7f0e60037bd0 2024-09-12T03:16:40.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.859+0000 7f75b0cb7640 1 -- 172.21.15.27:0/775156009 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75ac06eb90 con 0x7f75ac06d5e0 2024-09-12T03:16:40.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.859+0000 7f63167fc640 1 -- 172.21.15.27:0/1880598132 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f63200c1020 con 0x7f631801db50 2024-09-12T03:16:40.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.859+0000 7f0e67fff640 1 -- 172.21.15.27:0/1957765297 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e48005290 con 0x7f0e60037bd0 2024-09-12T03:16:40.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.860+0000 7f63167fc640 1 --2- 172.21.15.27:0/1880598132 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f630406be30 0x7f630406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:40.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.860+0000 7f261bfff640 1 -- 172.21.15.27:0/647522573 <== mon.2 v2:172.21.15.45:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26300afc10 con 0x7f262800a460 2024-09-12T03:16:40.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.860+0000 7f261bfff640 1 -- 172.21.15.27:0/647522573 <== mon.2 v2:172.21.15.45:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26300afdb0 con 0x7f262800a460 2024-09-12T03:16:40.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.860+0000 7f261bfff640 1 -- 172.21.15.27:0/647522573 <== mon.2 v2:172.21.15.45:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26300c6680 con 0x7f262800a460 2024-09-12T03:16:40.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.860+0000 7f2635e1e640 1 -- 172.21.15.27:0/647522573 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26280e1b50 con 0x7f262800a460 2024-09-12T03:16:40.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.860+0000 7f8ced7fa640 1 -- 172.21.15.27:0/138775458 <== mon.2 v2:172.21.15.45:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ce80088c0 con 0x7f8cf00ca980 2024-09-12T03:16:40.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.860+0000 7f8cf6d33640 1 -- 172.21.15.27:0/138775458 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8cf00cb440 con 0x7f8cf00ca980 2024-09-12T03:16:40.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.860+0000 7f8ced7fa640 1 -- 172.21.15.27:0/138775458 <== mon.2 v2:172.21.15.45:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ce8019070 con 0x7f8cf00ca980 2024-09-12T03:16:40.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.860+0000 7f8cf6d33640 1 -- 172.21.15.27:0/138775458 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f8cf00cba00 con 0x7f8cf00ca980 2024-09-12T03:16:40.930 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.860+0000 7f2635e1e640 1 -- 172.21.15.27:0/647522573 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f262804cdf0 con 0x7f262800a460 2024-09-12T03:16:40.930 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.860+0000 7f1b5affd640 1 -- 172.21.15.27:0/116456440 <== mon.2 v2:172.21.15.45:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b5c017070 con 0x7f1b6c06dbc0 2024-09-12T03:16:40.930 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.861+0000 7f0e4f7fe640 1 -- 172.21.15.27:0/1957765297 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0e5801c070 con 0x7f0e60037bd0 2024-09-12T03:16:40.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.861+0000 7f8ced7fa640 1 -- 172.21.15.27:0/138775458 <== mon.2 v2:172.21.15.45:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ce80143f0 con 0x7f8cf00ca980 2024-09-12T03:16:40.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.861+0000 7fb021ffb640 1 -- 172.21.15.27:0/3478031545 <== mon.2 v2:172.21.15.45:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fb01c01f020 con 0x7fb02c0cfde0 2024-09-12T03:16:40.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.860+0000 7f1b5affd640 1 -- 172.21.15.27:0/116456440 <== mon.2 v2:172.21.15.45:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b5c005ce0 con 0x7f1b6c06dbc0 2024-09-12T03:16:40.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.860+0000 7f1b5affd640 1 -- 172.21.15.27:0/116456440 <== mon.2 v2:172.21.15.45:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b5c0072f0 con 0x7f1b6c06dbc0 2024-09-12T03:16:40.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.861+0000 7f1b70c5e640 1 -- 172.21.15.27:0/116456440 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b6c157f70 con 0x7f1b6c06dbc0 2024-09-12T03:16:40.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.861+0000 7f1b70c5e640 1 -- 172.21.15.27:0/116456440 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b6c158770 con 0x7f1b6c06dbc0 2024-09-12T03:16:40.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.861+0000 7f0e4f7fe640 1 --2- 172.21.15.27:0/1957765297 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f0e5406be60 0x7f0e5406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:40.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.862+0000 7f63167fc640 1 -- 172.21.15.27:0/1880598132 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f6320126fc0 con 0x7f631801db50 2024-09-12T03:16:40.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.862+0000 7f0e4f7fe640 1 -- 172.21.15.27:0/1957765297 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f0e5808a210 con 0x7f0e60037bd0 2024-09-12T03:16:40.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.861+0000 7f759affd640 1 -- 172.21.15.27:0/775156009 <== mon.1 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f75a401d070 con 0x7f75ac06d5e0 2024-09-12T03:16:40.934 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.862+0000 7f759affd640 1 --2- 172.21.15.27:0/775156009 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f758c06be30 0x7f758c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:40.934 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.862+0000 7f75a9d9b640 1 --2- 172.21.15.27:0/775156009 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f758c06be30 0x7f758c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:40.934 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.862+0000 7f759affd640 1 -- 172.21.15.27:0/775156009 <== mon.1 v2:172.21.15.45:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f75a408b6d0 con 0x7f75ac06d5e0 2024-09-12T03:16:40.934 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.863+0000 7f6325c48640 1 --2- 172.21.15.27:0/1880598132 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f630406be30 0x7f630406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:40.934 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.863+0000 7f1b70c5e640 1 -- 172.21.15.27:0/116456440 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b6c158940 con 0x7f1b6c06dbc0 2024-09-12T03:16:40.934 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.863+0000 7f0e66ffd640 1 --2- 172.21.15.27:0/1957765297 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f0e5406be60 0x7f0e5406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:40.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.863+0000 7f8ced7fa640 1 -- 172.21.15.27:0/138775458 <== mon.2 v2:172.21.15.45:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8ce8014590 con 0x7f8cf00ca980 2024-09-12T03:16:40.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.864+0000 7f75a9d9b640 1 --2- 172.21.15.27:0/775156009 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f758c06be30 0x7f758c06e2f0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f759c034000 tx=0x7f759c02e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:40.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.864+0000 7f8ced7fa640 1 --2- 172.21.15.27:0/138775458 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f8cdc06be60 0x7f8cdc06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:40.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.864+0000 7f8ced7fa640 1 -- 172.21.15.27:0/138775458 <== mon.2 v2:172.21.15.45:3301/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f8ce808b4e0 con 0x7f8cf00ca980 2024-09-12T03:16:40.936 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.864+0000 7f2635e1e640 1 -- 172.21.15.27:0/647522573 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2610005290 con 0x7f262800a460 2024-09-12T03:16:40.936 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.864+0000 7f6325c48640 1 --2- 172.21.15.27:0/1880598132 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f630406be30 0x7f630406e2f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f6310007220 tx=0x7f6310011040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:40.936 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.864+0000 7f1b5affd640 1 -- 172.21.15.27:0/116456440 <== mon.2 v2:172.21.15.45:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1b5c01a070 con 0x7f1b6c06dbc0 2024-09-12T03:16:40.936 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.865+0000 7f0e66ffd640 1 --2- 172.21.15.27:0/1957765297 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f0e5406be60 0x7f0e5406e320 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f0e5c000c00 tx=0x7f0e5c02e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:40.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.865+0000 7f261bfff640 1 -- 172.21.15.27:0/647522573 <== mon.2 v2:172.21.15.45:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f26300c6820 con 0x7f262800a460 2024-09-12T03:16:40.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.865+0000 7f8cf4acf640 1 --2- 172.21.15.27:0/138775458 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f8cdc06be60 0x7f8cdc06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:40.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.865+0000 7f1b5affd640 1 --2- 172.21.15.27:0/116456440 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f1b4c06bf00 0x7f1b4c06e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:40.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.865+0000 7f1b5affd640 1 -- 172.21.15.27:0/116456440 <== mon.2 v2:172.21.15.45:3301/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f1b5c08a280 con 0x7f1b6c06dbc0 2024-09-12T03:16:40.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.865+0000 7f1b69d9b640 1 --2- 172.21.15.27:0/116456440 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f1b4c06bf00 0x7f1b4c06e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:40.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.865+0000 7f8cf6d33640 1 -- 172.21.15.27:0/138775458 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8cf006e080 con 0x7f8cf00ca980 2024-09-12T03:16:40.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.865+0000 7f8cf4acf640 1 --2- 172.21.15.27:0/138775458 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f8cdc06be60 0x7f8cdc06e320 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f8ce002ae40 tx=0x7f8ce002e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:40.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.865+0000 7f261bfff640 1 --2- 172.21.15.27:0/647522573 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f261406bf00 0x7f261406e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:40.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.866+0000 7f261bfff640 1 -- 172.21.15.27:0/647522573 <== mon.2 v2:172.21.15.45:3301/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f263012c2a0 con 0x7f262800a460 2024-09-12T03:16:40.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.866+0000 7f262f7fe640 1 --2- 172.21.15.27:0/647522573 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f261406bf00 0x7f261406e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:40.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.866+0000 7f1b69d9b640 1 --2- 172.21.15.27:0/116456440 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f1b4c06bf00 0x7f1b4c06e3c0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f1b6c1550a0 tx=0x7f1b60008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:40.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.866+0000 7f759affd640 1 -- 172.21.15.27:0/775156009 <== mon.1 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f75a4059990 con 0x7f75ac06d5e0 2024-09-12T03:16:40.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.866+0000 7f262f7fe640 1 --2- 172.21.15.27:0/647522573 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f261406bf00 0x7f261406e3c0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f261c0010e0 tx=0x7f261c009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:40.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.867+0000 7f63167fc640 1 -- 172.21.15.27:0/1880598132 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f63200f5a80 con 0x7f631801db50 2024-09-12T03:16:40.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.869+0000 7f1b5affd640 1 -- 172.21.15.27:0/116456440 <== mon.2 v2:172.21.15.45:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f1b5c058680 con 0x7f1b6c06dbc0 2024-09-12T03:16:40.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.871+0000 7f0e4f7fe640 1 -- 172.21.15.27:0/1957765297 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f0e600e4d80 con 0x7f0e60037bd0 2024-09-12T03:16:40.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.874+0000 7f261bfff640 1 -- 172.21.15.27:0/647522573 <== mon.2 v2:172.21.15.45:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f26300facc0 con 0x7f262800a460 2024-09-12T03:16:40.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:40.878+0000 7f8ced7fa640 1 -- 172.21.15.27:0/138775458 <== mon.2 v2:172.21.15.45:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f8ce80597a0 con 0x7f8cf00ca980 2024-09-12T03:16:41.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:41.248+0000 7f0230080640 1 -- 172.21.15.27:0/2507381626 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f0210005580 con 0x7f022806d0a0 2024-09-12T03:16:41.248 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:41.248+0000 7f021e7fc640 1 -- 172.21.15.27:0/2507381626 <== mon.1 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f02240588d0 con 0x7f022806d0a0 2024-09-12T03:16:41.248 INFO:teuthology.orchestra.run.smithi027.stdout:60129542150 2024-09-12T03:16:41.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:41.252+0000 7f020bfff640 1 -- 172.21.15.27:0/2507381626 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f020c06be30 msgr2=0x7f020c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:41.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:41.252+0000 7f020bfff640 1 --2- 172.21.15.27:0/2507381626 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f020c06be30 0x7f020c06e2f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f022003e000 tx=0x7f0220039000 comp rx=0 tx=0).stop 2024-09-12T03:16:41.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:41.252+0000 7f020bfff640 1 -- 172.21.15.27:0/2507381626 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f022806d0a0 msgr2=0x7f0228157070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:41.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:41.252+0000 7f020bfff640 1 --2- 172.21.15.27:0/2507381626 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f022806d0a0 0x7f0228157070 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f022400b780 tx=0x7f022400bc40 comp rx=0 tx=0).stop 2024-09-12T03:16:41.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:41.252+0000 7f020bfff640 1 -- 172.21.15.27:0/2507381626 shutdown_connections 2024-09-12T03:16:41.253 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:41.252+0000 7f020bfff640 1 --2- 172.21.15.27:0/2507381626 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f020c06be30 0x7f020c06e2f0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:41.253 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:41.252+0000 7f020bfff640 1 --2- 172.21.15.27:0/2507381626 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f022815db00 0x7f0228158fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:41.253 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:41.252+0000 7f020bfff640 1 --2- 172.21.15.27:0/2507381626 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f022806e970 0x7f0228158a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:41.253 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:41.252+0000 7f020bfff640 1 --2- 172.21.15.27:0/2507381626 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f022806d0a0 0x7f0228157070 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:41.253 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:41.253+0000 7f020bfff640 1 -- 172.21.15.27:0/2507381626 >> 172.21.15.27:0/2507381626 conn(0x7f022806bd10 msgr2=0x7f0228072470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:41.253 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:41.253+0000 7f020bfff640 1 -- 172.21.15.27:0/2507381626 shutdown_connections 2024-09-12T03:16:41.253 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:41.253+0000 7f020bfff640 1 -- 172.21.15.27:0/2507381626 wait complete. 2024-09-12T03:16:41.269 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542152 got 60129542150 for osd.1 2024-09-12T03:16:41.269 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:41.269+0000 7fb032442640 1 -- 172.21.15.27:0/3478031545 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fb00c005580 con 0x7fb02c0cfde0 2024-09-12T03:16:41.270 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:41.270+0000 7fb021ffb640 1 -- 172.21.15.27:0/3478031545 <== mon.2 v2:172.21.15.45:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fb01c00d040 con 0x7fb02c0cfde0 2024-09-12T03:16:41.270 INFO:teuthology.orchestra.run.smithi027.stdout:60129542149 2024-09-12T03:16:41.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:41.273+0000 7fb0177fe640 1 -- 172.21.15.27:0/3478031545 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fb01006be30 msgr2=0x7fb01006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:41.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:41.273+0000 7fb0177fe640 1 --2- 172.21.15.27:0/3478031545 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fb01006be30 0x7fb01006e2f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fb028033000 tx=0x7fb028008030 comp rx=0 tx=0).stop 2024-09-12T03:16:41.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:41.273+0000 7fb0177fe640 1 -- 172.21.15.27:0/3478031545 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb02c0cfde0 msgr2=0x7fb02c1566a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:41.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:41.273+0000 7fb0177fe640 1 --2- 172.21.15.27:0/3478031545 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb02c0cfde0 0x7fb02c1566a0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fb01c00b450 tx=0x7fb01c00b910 comp rx=0 tx=0).stop 2024-09-12T03:16:41.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:41.273+0000 7fb0177fe640 1 -- 172.21.15.27:0/3478031545 shutdown_connections 2024-09-12T03:16:41.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:41.273+0000 7fb0177fe640 1 --2- 172.21.15.27:0/3478031545 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fb01006be30 0x7fb01006e2f0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:41.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:41.273+0000 7fb0177fe640 1 --2- 172.21.15.27:0/3478031545 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb02c15a270 0x7fb02c158ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:41.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:41.273+0000 7fb0177fe640 1 --2- 172.21.15.27:0/3478031545 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb02c0cfde0 0x7fb02c1566a0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:41.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:41.273+0000 7fb0177fe640 1 --2- 172.21.15.27:0/3478031545 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb02c06d950 0x7fb02c14df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:41.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:41.273+0000 7fb0177fe640 1 -- 172.21.15.27:0/3478031545 >> 172.21.15.27:0/3478031545 conn(0x7fb02c0d0210 msgr2=0x7fb02c073550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:41.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:41.273+0000 7fb0177fe640 1 -- 172.21.15.27:0/3478031545 shutdown_connections 2024-09-12T03:16:41.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:41.273+0000 7fb0177fe640 1 -- 172.21.15.27:0/3478031545 wait complete. 2024-09-12T03:16:41.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:41.275+0000 7f8cf6d33640 1 -- 172.21.15.27:0/138775458 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f8cf0165120 con 0x7f8cf00ca980 2024-09-12T03:16:41.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:41.275+0000 7f75b0cb7640 1 -- 172.21.15.27:0/775156009 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f75ac0b1cd0 con 0x7f75ac06d5e0 2024-09-12T03:16:41.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:41.275+0000 7f8ced7fa640 1 -- 172.21.15.27:0/138775458 <== mon.2 v2:172.21.15.45:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f8ce805daa0 con 0x7f8cf00ca980 2024-09-12T03:16:41.276 INFO:teuthology.orchestra.run.smithi027.stdout:60129542150 2024-09-12T03:16:41.276 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:41.276+0000 7f759affd640 1 -- 172.21.15.27:0/775156009 <== mon.1 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f75a405ee20 con 0x7f75ac06d5e0 2024-09-12T03:16:41.276 INFO:teuthology.orchestra.run.smithi027.stdout:60129542150 2024-09-12T03:16:41.278 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:41.278+0000 7f8cf6d33640 1 -- 172.21.15.27:0/138775458 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f8cdc06be60 msgr2=0x7f8cdc06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:41.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:41.278+0000 7f8cf6d33640 1 --2- 172.21.15.27:0/138775458 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f8cdc06be60 0x7f8cdc06e320 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f8ce002ae40 tx=0x7f8ce002e000 comp rx=0 tx=0).stop 2024-09-12T03:16:41.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:41.278+0000 7f8cf6d33640 1 -- 172.21.15.27:0/138775458 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f8cf00ca980 msgr2=0x7f8cf00c7d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:41.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:41.278+0000 7f8cf6d33640 1 --2- 172.21.15.27:0/138775458 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f8cf00ca980 0x7f8cf00c7d30 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f8cf0073210 tx=0x7f8ce8001d70 comp rx=0 tx=0).stop 2024-09-12T03:16:41.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:41.278+0000 7f8cf6d33640 1 -- 172.21.15.27:0/138775458 shutdown_connections 2024-09-12T03:16:41.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:41.278+0000 7f8cf6d33640 1 --2- 172.21.15.27:0/138775458 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f8cdc06be60 0x7f8cdc06e320 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:41.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:41.278+0000 7f8cf6d33640 1 --2- 172.21.15.27:0/138775458 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f8cf00ca980 0x7f8cf00c7d30 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:41.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:41.278+0000 7f8cf6d33640 1 --2- 172.21.15.27:0/138775458 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8cf00cc200 0x7f8cf014e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:41.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:41.278+0000 7f8cf6d33640 1 --2- 172.21.15.27:0/138775458 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8cd80039b0 0x7f8cf0162030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:41.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:41.278+0000 7f8cf6d33640 1 -- 172.21.15.27:0/138775458 >> 172.21.15.27:0/138775458 conn(0x7f8cf006bd10 msgr2=0x7f8cf0165b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:41.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:41.278+0000 7f8cf6d33640 1 -- 172.21.15.27:0/138775458 shutdown_connections 2024-09-12T03:16:41.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:41.278+0000 7f8cf6d33640 1 -- 172.21.15.27:0/138775458 wait complete. 2024-09-12T03:16:41.281 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:41.280+0000 7f75b0cb7640 1 -- 172.21.15.27:0/775156009 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f758c06be30 msgr2=0x7f758c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:41.281 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:41.280+0000 7f75b0cb7640 1 --2- 172.21.15.27:0/775156009 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f758c06be30 0x7f758c06e2f0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f759c034000 tx=0x7f759c02e000 comp rx=0 tx=0).stop 2024-09-12T03:16:41.281 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:41.280+0000 7f75b0cb7640 1 -- 172.21.15.27:0/775156009 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f75ac06d5e0 msgr2=0x7f75ac1503e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:41.281 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:41.280+0000 7f75b0cb7640 1 --2- 172.21.15.27:0/775156009 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f75ac06d5e0 0x7f75ac1503e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f75a400cf60 tx=0x7f75a400cf90 comp rx=0 tx=0).stop 2024-09-12T03:16:41.281 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:41.280+0000 7f75b0cb7640 1 -- 172.21.15.27:0/775156009 shutdown_connections 2024-09-12T03:16:41.281 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:41.280+0000 7f75b0cb7640 1 --2- 172.21.15.27:0/775156009 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f758c06be30 0x7f758c06e2f0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:41.281 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:41.280+0000 7f75b0cb7640 1 --2- 172.21.15.27:0/775156009 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f75ac15db00 0x7f75ac152a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:41.281 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:41.280+0000 7f75b0cb7640 1 --2- 172.21.15.27:0/775156009 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f75ac071030 0x7f75ac1524e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:41.281 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:41.280+0000 7f75b0cb7640 1 --2- 172.21.15.27:0/775156009 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f75ac06d5e0 0x7f75ac1503e0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:41.281 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:41.280+0000 7f75b0cb7640 1 -- 172.21.15.27:0/775156009 >> 172.21.15.27:0/775156009 conn(0x7f75ac06bd10 msgr2=0x7f75ac0b0c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:41.281 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:41.280+0000 7f75b0cb7640 1 -- 172.21.15.27:0/775156009 shutdown_connections 2024-09-12T03:16:41.281 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:41.280+0000 7f75b0cb7640 1 -- 172.21.15.27:0/775156009 wait complete. 2024-09-12T03:16:41.282 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:41.281+0000 7f6327eac640 1 -- 172.21.15.27:0/1880598132 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f63180df1f0 con 0x7f631801db50 2024-09-12T03:16:41.282 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:41.282+0000 7f1b70c5e640 1 -- 172.21.15.27:0/116456440 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f1b6c062830 con 0x7f1b6c06dbc0 2024-09-12T03:16:41.282 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:41.282+0000 7f63167fc640 1 -- 172.21.15.27:0/1880598132 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f63200f5380 con 0x7f631801db50 2024-09-12T03:16:41.283 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:41.282+0000 7f1b5affd640 1 -- 172.21.15.27:0/116456440 <== mon.2 v2:172.21.15.45:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f1b5c05c980 con 0x7f1b6c06dbc0 2024-09-12T03:16:41.283 INFO:teuthology.orchestra.run.smithi027.stdout:60129542148 2024-09-12T03:16:41.283 INFO:teuthology.orchestra.run.smithi027.stdout:60129542149 2024-09-12T03:16:41.284 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:41.284+0000 7f0e67fff640 1 -- 172.21.15.27:0/1957765297 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f0e48005580 con 0x7f0e60037bd0 2024-09-12T03:16:41.284 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:41.285+0000 7f0e4f7fe640 1 -- 172.21.15.27:0/1957765297 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f0e580585a0 con 0x7f0e60037bd0 2024-09-12T03:16:41.285 INFO:teuthology.orchestra.run.smithi027.stdout:60129542148 2024-09-12T03:16:41.286 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:41.286+0000 7f6327eac640 1 -- 172.21.15.27:0/1880598132 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f630406be30 msgr2=0x7f630406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:41.286 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:41.286+0000 7f6327eac640 1 --2- 172.21.15.27:0/1880598132 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f630406be30 0x7f630406e2f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f6310007220 tx=0x7f6310011040 comp rx=0 tx=0).stop 2024-09-12T03:16:41.286 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:41.286+0000 7f1b70c5e640 1 -- 172.21.15.27:0/116456440 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f1b4c06bf00 msgr2=0x7f1b4c06e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:41.286 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:41.286+0000 7f1b70c5e640 1 --2- 172.21.15.27:0/116456440 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f1b4c06bf00 0x7f1b4c06e3c0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f1b6c1550a0 tx=0x7f1b60008040 comp rx=0 tx=0).stop 2024-09-12T03:16:41.286 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:41.286+0000 7f1b70c5e640 1 -- 172.21.15.27:0/116456440 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1b6c06dbc0 msgr2=0x7f1b6c152e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:41.286 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:41.286+0000 7f1b70c5e640 1 --2- 172.21.15.27:0/116456440 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1b6c06dbc0 0x7f1b6c152e90 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f1b5c00c410 tx=0x7f1b5c00c8d0 comp rx=0 tx=0).stop 2024-09-12T03:16:41.286 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:41.286+0000 7f6327eac640 1 -- 172.21.15.27:0/1880598132 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f631801db50 msgr2=0x7f6318031540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:41.287 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:41.286+0000 7f6327eac640 1 --2- 172.21.15.27:0/1880598132 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f631801db50 0x7f6318031540 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f6320051220 tx=0x7f63200a8070 comp rx=0 tx=0).stop 2024-09-12T03:16:41.287 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:41.286+0000 7f2635e1e640 1 -- 172.21.15.27:0/647522573 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f2610005580 con 0x7f262800a460 2024-09-12T03:16:41.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:41.286+0000 7f6327eac640 1 -- 172.21.15.27:0/1880598132 shutdown_connections 2024-09-12T03:16:41.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:41.286+0000 7f6327eac640 1 --2- 172.21.15.27:0/1880598132 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f630406be30 0x7f630406e2f0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:41.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:41.286+0000 7f6327eac640 1 --2- 172.21.15.27:0/1880598132 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f63180d88b0 0x7f63180d8d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:41.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:41.286+0000 7f6327eac640 1 --2- 172.21.15.27:0/1880598132 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f631801db50 0x7f6318031540 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:41.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:41.286+0000 7f6327eac640 1 --2- 172.21.15.27:0/1880598132 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f6318005ff0 0x7f63180c53c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:41.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:41.286+0000 7f6327eac640 1 -- 172.21.15.27:0/1880598132 >> 172.21.15.27:0/1880598132 conn(0x7f63180360c0 msgr2=0x7f63180dda00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:41.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:41.286+0000 7f1b70c5e640 1 -- 172.21.15.27:0/116456440 shutdown_connections 2024-09-12T03:16:41.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:41.286+0000 7f1b70c5e640 1 --2- 172.21.15.27:0/116456440 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f1b4c06bf00 0x7f1b4c06e3c0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:41.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:41.286+0000 7f1b70c5e640 1 --2- 172.21.15.27:0/116456440 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1b6c159990 0x7f1b6c155e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:41.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:41.286+0000 7f1b70c5e640 1 --2- 172.21.15.27:0/116456440 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1b6c0b53a0 0x7f1b6c1558c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:41.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:41.286+0000 7f1b70c5e640 1 --2- 172.21.15.27:0/116456440 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1b6c06dbc0 0x7f1b6c152e90 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:41.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:41.286+0000 7f1b70c5e640 1 -- 172.21.15.27:0/116456440 >> 172.21.15.27:0/116456440 conn(0x7f1b6c0b0b50 msgr2=0x7f1b6c163060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:41.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:41.286+0000 7f6327eac640 1 -- 172.21.15.27:0/1880598132 shutdown_connections 2024-09-12T03:16:41.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:41.286+0000 7f1b70c5e640 1 -- 172.21.15.27:0/116456440 shutdown_connections 2024-09-12T03:16:41.290 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:41.287+0000 7f6327eac640 1 -- 172.21.15.27:0/1880598132 wait complete. 2024-09-12T03:16:41.290 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:41.287+0000 7f261bfff640 1 -- 172.21.15.27:0/647522573 <== mon.2 v2:172.21.15.45:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f26300fa5c0 con 0x7f262800a460 2024-09-12T03:16:41.290 INFO:teuthology.orchestra.run.smithi027.stdout:60129542149 2024-09-12T03:16:41.290 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:41.287+0000 7f1b70c5e640 1 -- 172.21.15.27:0/116456440 wait complete. 2024-09-12T03:16:41.291 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:41.288+0000 7f0e67fff640 1 -- 172.21.15.27:0/1957765297 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f0e5406be60 msgr2=0x7f0e5406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:41.291 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:41.288+0000 7f0e67fff640 1 --2- 172.21.15.27:0/1957765297 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f0e5406be60 0x7f0e5406e320 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f0e5c000c00 tx=0x7f0e5c02e000 comp rx=0 tx=0).stop 2024-09-12T03:16:41.291 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:41.289+0000 7f0e67fff640 1 -- 172.21.15.27:0/1957765297 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e60037bd0 msgr2=0x7f0e6003dc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:41.291 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:41.289+0000 7f0e67fff640 1 --2- 172.21.15.27:0/1957765297 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e60037bd0 0x7f0e6003dc60 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f0e5800b750 tx=0x7f0e5800bc10 comp rx=0 tx=0).stop 2024-09-12T03:16:41.291 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:41.290+0000 7f2635e1e640 1 -- 172.21.15.27:0/647522573 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f261406bf00 msgr2=0x7f261406e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:41.291 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:41.290+0000 7f2635e1e640 1 --2- 172.21.15.27:0/647522573 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f261406bf00 0x7f261406e3c0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f261c0010e0 tx=0x7f261c009040 comp rx=0 tx=0).stop 2024-09-12T03:16:41.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:41.290+0000 7f2635e1e640 1 -- 172.21.15.27:0/647522573 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f262800a460 msgr2=0x7f26280e11c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:41.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:41.290+0000 7f2635e1e640 1 --2- 172.21.15.27:0/647522573 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f262800a460 0x7f26280e11c0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f2630069370 tx=0x7f26300ac000 comp rx=0 tx=0).stop 2024-09-12T03:16:41.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:41.289+0000 7f0e67fff640 1 -- 172.21.15.27:0/1957765297 shutdown_connections 2024-09-12T03:16:41.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:41.289+0000 7f0e67fff640 1 --2- 172.21.15.27:0/1957765297 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f0e5406be60 0x7f0e5406e320 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:41.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:41.289+0000 7f0e67fff640 1 --2- 172.21.15.27:0/1957765297 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0e600ea950 0x7f0e600e3c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:41.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:41.289+0000 7f0e67fff640 1 --2- 172.21.15.27:0/1957765297 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e60037bd0 0x7f0e6003dc60 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:41.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:41.289+0000 7f0e67fff640 1 --2- 172.21.15.27:0/1957765297 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f0e60007190 0x7f0e600e3130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:41.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:41.289+0000 7f0e67fff640 1 -- 172.21.15.27:0/1957765297 >> 172.21.15.27:0/1957765297 conn(0x7f0e6003a4f0 msgr2=0x7f0e600e4f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:41.293 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:41.290+0000 7f2635e1e640 1 -- 172.21.15.27:0/647522573 shutdown_connections 2024-09-12T03:16:41.293 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:41.290+0000 7f2635e1e640 1 --2- 172.21.15.27:0/647522573 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f261406bf00 0x7f261406e3c0 secure :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f261c0010e0 tx=0x7f261c009040 comp rx=0 tx=0).stop 2024-09-12T03:16:41.293 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:41.290+0000 7f2635e1e640 1 --2- 172.21.15.27:0/647522573 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2628037040 0x7f262804d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:41.293 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:41.290+0000 7f2635e1e640 1 --2- 172.21.15.27:0/647522573 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f262800a460 0x7f26280e11c0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:41.293 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:41.290+0000 7f2635e1e640 1 --2- 172.21.15.27:0/647522573 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2628004a60 0x7f26280d9000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:41.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:41.290+0000 7f2635e1e640 1 -- 172.21.15.27:0/647522573 >> 172.21.15.27:0/647522573 conn(0x7f262801ddb0 msgr2=0x7f2628032d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:41.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:41.290+0000 7f0e67fff640 1 -- 172.21.15.27:0/1957765297 shutdown_connections 2024-09-12T03:16:41.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:41.290+0000 7f2635e1e640 1 -- 172.21.15.27:0/647522573 shutdown_connections 2024-09-12T03:16:41.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:41.290+0000 7f2635e1e640 1 -- 172.21.15.27:0/647522573 wait complete. 2024-09-12T03:16:41.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:41.290+0000 7f0e67fff640 1 -- 172.21.15.27:0/1957765297 wait complete. 2024-09-12T03:16:41.295 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542151 got 60129542149 for osd.2 2024-09-12T03:16:41.297 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542152 got 60129542150 for osd.6 2024-09-12T03:16:41.302 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542152 got 60129542150 for osd.0 2024-09-12T03:16:41.304 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542151 got 60129542149 for osd.3 2024-09-12T03:16:41.306 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542148 for osd.4 2024-09-12T03:16:41.310 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542148 for osd.7 2024-09-12T03:16:41.311 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542151 got 60129542149 for osd.5 2024-09-12T03:16:42.270 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-12T03:16:42.296 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-12T03:16:42.298 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-12T03:16:42.303 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-12T03:16:42.306 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-12T03:16:42.307 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-12T03:16:42.311 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-12T03:16:42.311 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-12T03:16:42.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.440+0000 7fd80cd1f640 1 Processor -- start 2024-09-12T03:16:42.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.440+0000 7fd80cd1f640 1 -- start start 2024-09-12T03:16:42.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.442+0000 7fd80cd1f640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fd8080cfde0 0x7fd80806d240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:42.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.442+0000 7fd80cd1f640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fd8080b4d00 0x7fd80806d780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:42.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.442+0000 7fd80cd1f640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd8080b3c60 con 0x7fd80815a270 2024-09-12T03:16:42.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.442+0000 7fd80cd1f640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fd8080b3f40 con 0x7fd8080b4d00 2024-09-12T03:16:42.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.442+0000 7fd80cd1f640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fd8080b4220 con 0x7fd8080cfde0 2024-09-12T03:16:42.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.442+0000 7fd806ffd640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fd8080cfde0 0x7fd80806d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:42.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.442+0000 7fd806ffd640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fd8080cfde0 0x7fd80806d240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3301/0 says I am v2:172.21.15.27:56460/0 (socket says 172.21.15.27:56460) 2024-09-12T03:16:42.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.442+0000 7fd806ffd640 1 -- 172.21.15.27:0/4021296159 learned_addr learned my addr 172.21.15.27:0/4021296159 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:16:42.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.443+0000 7fd806ffd640 1 -- 172.21.15.27:0/4021296159 >> v2:172.21.15.45:3300/0 conn(0x7fd8080b4d00 msgr2=0x7fd80806d780 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-12T03:16:42.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.443+0000 7fd807fff640 1 --2- 172.21.15.27:0/4021296159 >> v2:172.21.15.45:3300/0 conn(0x7fd8080b4d00 0x7fd80806d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:42.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.443+0000 7fd806ffd640 1 --2- 172.21.15.27:0/4021296159 >> v2:172.21.15.45:3300/0 conn(0x7fd8080b4d00 0x7fd80806d780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:42.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.443+0000 7fd806ffd640 1 -- 172.21.15.27:0/4021296159 >> v1:172.21.15.45:6789/0 conn(0x7fd80815a270 legacy=0x7fd80806ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:42.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.443+0000 7fd806ffd640 1 -- 172.21.15.27:0/4021296159 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8080b4500 con 0x7fd8080cfde0 2024-09-12T03:16:42.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.444+0000 7fd807fff640 1 --2- 172.21.15.27:0/4021296159 >> v2:172.21.15.45:3300/0 conn(0x7fd8080b4d00 0x7fd80806d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:16:42.446 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.444+0000 7fd806ffd640 1 --2- 172.21.15.27:0/4021296159 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fd8080cfde0 0x7fd80806d240 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fd80000d370 tx=0x7fd80002c940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6f41865cb0b3e69a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:42.446 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.446+0000 7fd8067fc640 1 -- 172.21.15.27:0/4021296159 <== mon.2 v2:172.21.15.45:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd80003b070 con 0x7fd8080cfde0 2024-09-12T03:16:42.447 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.446+0000 7fd8067fc640 1 -- 172.21.15.27:0/4021296159 <== mon.2 v2:172.21.15.45:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8000336f0 con 0x7fd8080cfde0 2024-09-12T03:16:42.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.447+0000 7fd80cd1f640 1 -- 172.21.15.27:0/4021296159 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fd8080cfde0 msgr2=0x7fd80806d240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:42.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.447+0000 7fd80cd1f640 1 --2- 172.21.15.27:0/4021296159 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fd8080cfde0 0x7fd80806d240 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fd80000d370 tx=0x7fd80002c940 comp rx=0 tx=0).stop 2024-09-12T03:16:42.449 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.447+0000 7fd80cd1f640 1 -- 172.21.15.27:0/4021296159 shutdown_connections 2024-09-12T03:16:42.449 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.447+0000 7fd80cd1f640 1 --2- 172.21.15.27:0/4021296159 >> v2:172.21.15.45:3300/0 conn(0x7fd8080b4d00 0x7fd80806d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:42.449 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.447+0000 7fd80cd1f640 1 --2- 172.21.15.27:0/4021296159 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fd8080cfde0 0x7fd80806d240 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:42.449 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.447+0000 7fd80cd1f640 1 -- 172.21.15.27:0/4021296159 >> 172.21.15.27:0/4021296159 conn(0x7fd8080d0210 msgr2=0x7fd8080d0620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:42.449 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.448+0000 7fd80cd1f640 1 -- 172.21.15.27:0/4021296159 shutdown_connections 2024-09-12T03:16:42.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.448+0000 7fd80cd1f640 1 -- 172.21.15.27:0/4021296159 wait complete. 2024-09-12T03:16:42.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.450+0000 7fd80cd1f640 1 Processor -- start 2024-09-12T03:16:42.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.450+0000 7fd80cd1f640 1 -- start start 2024-09-12T03:16:42.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.452+0000 7fd80cd1f640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fd8080b4d00 0x7fd8080aa7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:42.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.452+0000 7fd80cd1f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd8080cfde0 0x7fd8080aae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:42.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.452+0000 7fd80cd1f640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd80815a270 0x7fd808155a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:42.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.452+0000 7fd8077fe640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fd8080b4d00 0x7fd8080aa7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:42.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.452+0000 7fd8077fe640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fd8080b4d00 0x7fd8080aa7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3301/0 says I am v2:172.21.15.27:56464/0 (socket says 172.21.15.27:56464) 2024-09-12T03:16:42.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.453+0000 7fd806ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd8080cfde0 0x7fd8080aae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:42.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.453+0000 7fd806ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd8080cfde0 0x7fd8080aae90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.27:60976/0 (socket says 172.21.15.27:60976) 2024-09-12T03:16:42.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.453+0000 7fd8077fe640 1 -- 172.21.15.27:0/1980670100 learned_addr learned my addr 172.21.15.27:0/1980670100 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:16:42.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.453+0000 7fd80cd1f640 1 -- 172.21.15.27:0/1980670100 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fd8080d0eb0 con 0x7fd80815a270 2024-09-12T03:16:42.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.453+0000 7fd80cd1f640 1 -- 172.21.15.27:0/1980670100 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fd8080d0d30 con 0x7fd8080cfde0 2024-09-12T03:16:42.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.453+0000 7fd80cd1f640 1 -- 172.21.15.27:0/1980670100 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fd80815a650 con 0x7fd8080b4d00 2024-09-12T03:16:42.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.453+0000 7fd807fff640 1 --2- 172.21.15.27:0/1980670100 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd80815a270 0x7fd808155a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:42.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.454+0000 7fd8077fe640 1 -- 172.21.15.27:0/1980670100 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd8080cfde0 msgr2=0x7fd8080aae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:42.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.454+0000 7fd8077fe640 1 --2- 172.21.15.27:0/1980670100 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd8080cfde0 0x7fd8080aae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:42.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.454+0000 7fd8077fe640 1 -- 172.21.15.27:0/1980670100 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd80815a270 msgr2=0x7fd808155a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:42.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.454+0000 7fd8077fe640 1 --2- 172.21.15.27:0/1980670100 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd80815a270 0x7fd808155a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:42.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.454+0000 7fd8077fe640 1 -- 172.21.15.27:0/1980670100 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8081f64c0 con 0x7fd8080b4d00 2024-09-12T03:16:42.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.459+0000 7fd8077fe640 1 --2- 172.21.15.27:0/1980670100 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fd8080b4d00 0x7fd8080aa7d0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fd7f800c780 tx=0x7fd7f800cc40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:42.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.459+0000 7fd7effff640 1 -- 172.21.15.27:0/1980670100 <== mon.2 v2:172.21.15.45:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7f8017070 con 0x7fd8080b4d00 2024-09-12T03:16:42.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.459+0000 7fd80cd1f640 1 -- 172.21.15.27:0/1980670100 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8081f67a0 con 0x7fd8080b4d00 2024-09-12T03:16:42.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.459+0000 7fd80cd1f640 1 -- 172.21.15.27:0/1980670100 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8080ab600 con 0x7fd8080b4d00 2024-09-12T03:16:42.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.461+0000 7fd7effff640 1 -- 172.21.15.27:0/1980670100 <== mon.2 v2:172.21.15.45:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7f8005270 con 0x7fd8080b4d00 2024-09-12T03:16:42.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.461+0000 7fd7effff640 1 -- 172.21.15.27:0/1980670100 <== mon.2 v2:172.21.15.45:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7f800a880 con 0x7fd8080b4d00 2024-09-12T03:16:42.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.463+0000 7fd7effff640 1 -- 172.21.15.27:0/1980670100 <== mon.2 v2:172.21.15.45:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd7f801a070 con 0x7fd8080b4d00 2024-09-12T03:16:42.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.464+0000 7fd7effff640 1 --2- 172.21.15.27:0/1980670100 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fd7f406bf00 0x7fd7f406e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:42.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.464+0000 7fd7effff640 1 -- 172.21.15.27:0/1980670100 <== mon.2 v2:172.21.15.45:3301/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fd7f808a300 con 0x7fd8080b4d00 2024-09-12T03:16:42.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.464+0000 7fd806ffd640 1 --2- 172.21.15.27:0/1980670100 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fd7f406bf00 0x7fd7f406e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:42.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.465+0000 7fd806ffd640 1 --2- 172.21.15.27:0/1980670100 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fd7f406bf00 0x7fd7f406e3c0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fd800038000 tx=0x7fd800031000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:42.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.465+0000 7fd80cd1f640 1 -- 172.21.15.27:0/1980670100 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7e8005290 con 0x7fd8080b4d00 2024-09-12T03:16:42.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.471+0000 7fd7effff640 1 -- 172.21.15.27:0/1980670100 <== mon.2 v2:172.21.15.45:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fd7f80586d0 con 0x7fd8080b4d00 2024-09-12T03:16:42.493 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.492+0000 7f1692cf1640 1 Processor -- start 2024-09-12T03:16:42.494 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.493+0000 7f1692cf1640 1 -- start start 2024-09-12T03:16:42.495 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.494+0000 7f1692cf1640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f168c159020 0x7f168c15c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:42.495 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.494+0000 7f1692cf1640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f168c0720c0 0x7f168c072530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:42.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.494+0000 7f1690a8d640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f168c159020 0x7f168c15c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:42.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.495+0000 7f1690a8d640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f168c159020 0x7f168c15c4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.27:60986/0 (socket says 172.21.15.27:60986) 2024-09-12T03:16:42.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.495+0000 7f1690a8d640 1 -- 172.21.15.27:0/1054959496 learned_addr learned my addr 172.21.15.27:0/1054959496 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:16:42.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.495+0000 7f905d81e640 1 Processor -- start 2024-09-12T03:16:42.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.495+0000 7f1692cf1640 1 -- 172.21.15.27:0/1054959496 --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f168c15c9f0 con 0x7f168c15f020 2024-09-12T03:16:42.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.495+0000 7f168bfff640 1 --2- 172.21.15.27:0/1054959496 >> v2:172.21.15.27:3300/0 conn(0x7f168c0720c0 0x7f168c072530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:42.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.496+0000 7f1692cf1640 1 -- 172.21.15.27:0/1054959496 --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f168c072a70 con 0x7f168c0720c0 2024-09-12T03:16:42.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.496+0000 7f168b7fe640 1 -- 172.21.15.27:0/1054959496 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2528520200 0 0) 0x7f1684001490 con 0x7f168c15f020 2024-09-12T03:16:42.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.496+0000 7f1692cf1640 1 -- 172.21.15.27:0/1054959496 --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f168c06e480 con 0x7f168c159020 2024-09-12T03:16:42.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.496+0000 7f168b7fe640 1 -- 172.21.15.27:0/1054959496 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f167c003520 con 0x7f168c15f020 2024-09-12T03:16:42.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.496+0000 7f1690a8d640 1 -- 172.21.15.27:0/1054959496 >> v2:172.21.15.27:3300/0 conn(0x7f168c0720c0 msgr2=0x7f168c072530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:42.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.495+0000 7f905d81e640 1 -- start start 2024-09-12T03:16:42.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.496+0000 7f168b7fe640 1 -- 172.21.15.27:0/1054959496 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1901674609 0 0) 0x7f167c003520 con 0x7f168c15f020 2024-09-12T03:16:42.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.496+0000 7f905d81e640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f905815b360 0x7f905806e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:42.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.496+0000 7f905d81e640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f90580b59f0 con 0x7f905806e5d0 2024-09-12T03:16:42.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.496+0000 7f905d81e640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f905815b740 con 0x7f90580b3cd0 2024-09-12T03:16:42.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.496+0000 7f905d81e640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f905806ec30 con 0x7f905815b360 2024-09-12T03:16:42.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.496+0000 7f1690a8d640 1 --2- 172.21.15.27:0/1054959496 >> v2:172.21.15.27:3300/0 conn(0x7f168c0720c0 0x7f168c072530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:42.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.496+0000 7f1690a8d640 1 -- 172.21.15.27:0/1054959496 >> v1:172.21.15.27:6789/0 conn(0x7f168c15f020 legacy=0x7f168c15f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:42.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.497+0000 7f9681a8d640 1 Processor -- start 2024-09-12T03:16:42.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.496+0000 7f1690a8d640 1 -- 172.21.15.27:0/1054959496 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f168c06e760 con 0x7f168c159020 2024-09-12T03:16:42.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.497+0000 7f9681a8d640 1 -- start start 2024-09-12T03:16:42.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.497+0000 7f168bfff640 1 --2- 172.21.15.27:0/1054959496 >> v2:172.21.15.27:3300/0 conn(0x7f168c0720c0 0x7f168c072530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:16:42.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.497+0000 7f90567fc640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f905806e5d0 0x7f90580b35c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:39220/0 (socket says 172.21.15.27:39220) 2024-09-12T03:16:42.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.498+0000 7f90567fc640 1 -- 172.21.15.27:0/1589370744 learned_addr learned my addr 172.21.15.27:0/1589370744 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:16:42.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.498+0000 7fe60359e640 1 Processor -- start 2024-09-12T03:16:42.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.498+0000 7f9055ffb640 1 -- 172.21.15.27:0/1589370744 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4202493297 0 0) 0x7f9048001110 con 0x7f905806e5d0 2024-09-12T03:16:42.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.498+0000 7f9055ffb640 1 -- 172.21.15.27:0/1589370744 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9044003520 con 0x7f905806e5d0 2024-09-12T03:16:42.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.498+0000 7f9055ffb640 1 -- 172.21.15.27:0/1589370744 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4242640580 0 0) 0x7f9044003520 con 0x7f905806e5d0 2024-09-12T03:16:42.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.498+0000 7f9055ffb640 1 -- 172.21.15.27:0/1589370744 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f905815b360 msgr2=0x7f905806e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:42.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.498+0000 7f9055ffb640 1 --2- 172.21.15.27:0/1589370744 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f905815b360 0x7f905806e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:42.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.498+0000 7f9055ffb640 1 -- 172.21.15.27:0/1589370744 >> v1:172.21.15.45:6789/0 conn(0x7f90580b3cd0 legacy=0x7f905815eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:42.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.498+0000 7f9055ffb640 1 -- 172.21.15.27:0/1589370744 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f905815b9e0 con 0x7f905806e5d0 2024-09-12T03:16:42.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.498+0000 7f9055ffb640 1 -- 172.21.15.27:0/1589370744 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 847755213 0 0) 0x7f9048001840 con 0x7f905806e5d0 2024-09-12T03:16:42.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.498+0000 7f9055ffb640 1 -- 172.21.15.27:0/1589370744 >> v1:172.21.15.27:6789/0 conn(0x7f905806e5d0 legacy=0x7f90580b35c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:42.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.498+0000 7f9681a8d640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f967c15b360 0x7f967c06e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:42.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.498+0000 7f9055ffb640 1 --2- 172.21.15.27:0/1589370744 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f90440039e0 0x7f9044003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:42.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.498+0000 7f9055ffb640 1 --2- 172.21.15.27:0/1589370744 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f9044004800 0x7f9044004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:42.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.498+0000 7f9055ffb640 1 --2- 172.21.15.27:0/1589370744 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f90440052e0 0x7f904400dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:42.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.498+0000 7f9055ffb640 1 -- 172.21.15.27:0/1589370744 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f905806ec30 con 0x7f90440039e0 2024-09-12T03:16:42.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.498+0000 7f9055ffb640 1 -- 172.21.15.27:0/1589370744 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f904400e3f0 con 0x7f90440052e0 2024-09-12T03:16:42.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.498+0000 7f9055ffb640 1 -- 172.21.15.27:0/1589370744 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f904400e5f0 con 0x7f9044004800 2024-09-12T03:16:42.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.498+0000 7f9681a8d640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f967c06e5d0 0x7f967c0b35c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:42.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.498+0000 7fe60359e640 1 -- start start 2024-09-12T03:16:42.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.499+0000 7f1690a8d640 1 --2- 172.21.15.27:0/1054959496 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f168c159020 0x7f168c15c4b0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f168c071db0 tx=0x7f168002a1a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2793a8137fb13c08 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:42.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.499+0000 7f90567fc640 1 --2- 172.21.15.27:0/1589370744 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f9044004800 0x7f9044004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:42.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.499+0000 7f9681a8d640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f967c0b59f0 con 0x7f967c0b3b00 2024-09-12T03:16:42.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.499+0000 7f967b7fe640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f967c15b360 0x7f967c06e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:42.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.499+0000 7f967b7fe640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f967c15b360 0x7f967c06e090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.27:32770/0 (socket says 172.21.15.27:32770) 2024-09-12T03:16:42.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.499+0000 7f168b7fe640 1 -- 172.21.15.27:0/1054959496 <== mon.1 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1680037070 con 0x7f168c159020 2024-09-12T03:16:42.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.499+0000 7f967b7fe640 1 -- 172.21.15.27:0/2760625750 learned_addr learned my addr 172.21.15.27:0/2760625750 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:16:42.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.499+0000 7f9681a8d640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f967c15b740 con 0x7f967c15b360 2024-09-12T03:16:42.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.499+0000 7f967affd640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f967c06e5d0 0x7f967c0b35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:42.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.499+0000 7f168b7fe640 1 -- 172.21.15.27:0/1054959496 <== mon.1 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f168002ac30 con 0x7f168c159020 2024-09-12T03:16:42.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.499+0000 7f168b7fe640 1 -- 172.21.15.27:0/1054959496 <== mon.1 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1684001bc0 con 0x7f168c159020 2024-09-12T03:16:42.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.499+0000 7f9681a8d640 1 -- 172.21.15.27:0/2760625750 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f967c06ec30 con 0x7f967c06e5d0 2024-09-12T03:16:42.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.499+0000 7f90567fc640 1 -- 172.21.15.27:0/1589370744 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f90440052e0 msgr2=0x7f904400dbd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-12T03:16:42.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.499+0000 7f90567fc640 1 --2- 172.21.15.27:0/1589370744 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f90440052e0 0x7f904400dbd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:42.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.499+0000 7f90567fc640 1 -- 172.21.15.27:0/1589370744 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f90440039e0 msgr2=0x7f9044003de0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-12T03:16:42.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.499+0000 7f90567fc640 1 --2- 172.21.15.27:0/1589370744 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f90440039e0 0x7f9044003de0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:42.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.499+0000 7f90567fc640 1 -- 172.21.15.27:0/1589370744 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9044003520 con 0x7f9044004800 2024-09-12T03:16:42.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.499+0000 7f967affd640 1 -- 172.21.15.27:0/2760625750 >> v2:172.21.15.45:3300/0 conn(0x7f967c15b360 msgr2=0x7f967c06e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:42.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.499+0000 7f967affd640 1 --2- 172.21.15.27:0/2760625750 >> v2:172.21.15.45:3300/0 conn(0x7f967c15b360 0x7f967c06e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:42.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.499+0000 7f967affd640 1 -- 172.21.15.27:0/2760625750 >> v1:172.21.15.27:6789/0 conn(0x7f967c0b3b00 legacy=0x7f967c072120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:42.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.499+0000 7f967affd640 1 -- 172.21.15.27:0/2760625750 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f967c15b9e0 con 0x7f967c06e5d0 2024-09-12T03:16:42.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.499+0000 7fe46d62e640 1 Processor -- start 2024-09-12T03:16:42.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.500+0000 7fe60359e640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fe5fc036470 0x7fe5fc036890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:42.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.500+0000 7fe60359e640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fe5fc00ad40 0x7fe5fc00b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:42.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.500+0000 7fe60359e640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe5fc036dd0 con 0x7fe5fc0541f0 2024-09-12T03:16:42.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.500+0000 7fe60359e640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7fe5fc03a900 con 0x7fe5fc00ad40 2024-09-12T03:16:42.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.500+0000 7fe60359e640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fe5fc004c00 con 0x7fe5fc036470 2024-09-12T03:16:42.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.499+0000 7fe46d62e640 1 -- start start 2024-09-12T03:16:42.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.500+0000 7f1692cf1640 1 -- 172.21.15.27:0/1054959496 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f168c159020 msgr2=0x7f168c15c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:42.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.500+0000 7fe601d9b640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fe5fc036470 0x7fe5fc036890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:42.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.500+0000 7fe601d9b640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fe5fc036470 0x7fe5fc036890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.27:32784/0 (socket says 172.21.15.27:32784) 2024-09-12T03:16:42.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.500+0000 7fe601d9b640 1 -- 172.21.15.27:0/308011220 learned_addr learned my addr 172.21.15.27:0/308011220 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:16:42.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.500+0000 7f1692cf1640 1 --2- 172.21.15.27:0/1054959496 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f168c159020 0x7f168c15c4b0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f168c071db0 tx=0x7f168002a1a0 comp rx=0 tx=0).stop 2024-09-12T03:16:42.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.500+0000 7f967b7fe640 1 --2- 172.21.15.27:0/2760625750 >> v2:172.21.15.45:3300/0 conn(0x7f967c15b360 0x7f967c06e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:16:42.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.500+0000 7f63833bd640 1 Processor -- start 2024-09-12T03:16:42.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.500+0000 7f1692cf1640 1 -- 172.21.15.27:0/1054959496 shutdown_connections 2024-09-12T03:16:42.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.500+0000 7f1692cf1640 1 --2- 172.21.15.27:0/1054959496 >> v2:172.21.15.27:3300/0 conn(0x7f168c0720c0 0x7f168c072530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:42.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.500+0000 7f1692cf1640 1 --2- 172.21.15.27:0/1054959496 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f168c159020 0x7f168c15c4b0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:42.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.500+0000 7f1692cf1640 1 -- 172.21.15.27:0/1054959496 >> 172.21.15.27:0/1054959496 conn(0x7f168c06cc40 msgr2=0x7f168c06d050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:42.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.500+0000 7f63833bd640 1 -- start start 2024-09-12T03:16:42.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.501+0000 7f1692cf1640 1 -- 172.21.15.27:0/1054959496 shutdown_connections 2024-09-12T03:16:42.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.501+0000 7f1692cf1640 1 -- 172.21.15.27:0/1054959496 wait complete. 2024-09-12T03:16:42.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.501+0000 7fe601d9b640 1 -- 172.21.15.27:0/308011220 >> v2:172.21.15.27:3300/0 conn(0x7fe5fc00ad40 msgr2=0x7fe5fc00b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:42.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.501+0000 7fe601d9b640 1 --2- 172.21.15.27:0/308011220 >> v2:172.21.15.27:3300/0 conn(0x7fe5fc00ad40 0x7fe5fc00b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:42.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.501+0000 7fe601d9b640 1 -- 172.21.15.27:0/308011220 >> v1:172.21.15.45:6789/0 conn(0x7fe5fc0541f0 legacy=0x7fe5fc03a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:42.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.501+0000 7fe601d9b640 1 -- 172.21.15.27:0/308011220 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5fc047af0 con 0x7fe5fc036470 2024-09-12T03:16:42.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.501+0000 7f967affd640 1 --2- 172.21.15.27:0/2760625750 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f967c06e5d0 0x7f967c0b35c0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f966c0097a0 tx=0x7f966c02c2c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cc25b3444da4ff8c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:42.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.501+0000 7fe46d62e640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe46815eb90 0x7fe4680d2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:42.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.501+0000 7fe46d62e640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fe4680d1720 0x7fe4680b8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:42.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.501+0000 7fe46d62e640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe4680d3760 con 0x7fe4680d5200 2024-09-12T03:16:42.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.501+0000 7fe46d62e640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7fe4680b7d20 con 0x7fe4680d1720 2024-09-12T03:16:42.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.501+0000 7fe46d62e640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fe4680d42e0 con 0x7fe46815eb90 2024-09-12T03:16:42.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.501+0000 7f90567fc640 1 --2- 172.21.15.27:0/1589370744 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f9044004800 0x7f9044004c80 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f904800c3b0 tx=0x7f9044005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bd73c40980ca56c4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:42.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.501+0000 7fe4677fe640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fe4680d1720 0x7fe4680b8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:42.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.501+0000 7fe466ffd640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe46815eb90 0x7fe4680d2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:42.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.501+0000 7fe4677fe640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fe4680d1720 0x7fe4680b8190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33388/0 (socket says 172.21.15.27:33388) 2024-09-12T03:16:42.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.501+0000 7fe4677fe640 1 -- 172.21.15.27:0/3701491061 learned_addr learned my addr 172.21.15.27:0/3701491061 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:16:42.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.501+0000 7f967a7fc640 1 -- 172.21.15.27:0/2760625750 <== mon.2 v2:172.21.15.45:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f966c039070 con 0x7f967c06e5d0 2024-09-12T03:16:42.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.501+0000 7f9055ffb640 1 -- 172.21.15.27:0/1589370744 <== mon.2 v2:172.21.15.45:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f904800c0e0 con 0x7f9044004800 2024-09-12T03:16:42.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.501+0000 7f967a7fc640 1 -- 172.21.15.27:0/2760625750 <== mon.2 v2:172.21.15.45:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f966c034440 con 0x7f967c06e5d0 2024-09-12T03:16:42.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.501+0000 7f9055ffb640 1 -- 172.21.15.27:0/1589370744 <== mon.2 v2:172.21.15.45:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9048034070 con 0x7f9044004800 2024-09-12T03:16:42.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.501+0000 7fe4677fe640 1 -- 172.21.15.27:0/3701491061 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe46815eb90 msgr2=0x7fe4680d2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:42.519 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.501+0000 7fe4677fe640 1 --2- 172.21.15.27:0/3701491061 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe46815eb90 0x7fe4680d2df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:42.519 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.501+0000 7fe4677fe640 1 -- 172.21.15.27:0/3701491061 >> v1:172.21.15.45:6789/0 conn(0x7fe4680d5200 legacy=0x7fe4680d1010 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-12T03:16:42.519 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.501+0000 7fe4677fe640 1 -- 172.21.15.27:0/3701491061 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4680d3a40 con 0x7fe4680d1720 2024-09-12T03:16:42.519 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.501+0000 7f9055ffb640 1 -- 172.21.15.27:0/1589370744 <== mon.2 v2:172.21.15.45:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9048005d90 con 0x7f9044004800 2024-09-12T03:16:42.519 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.502+0000 7f967a7fc640 1 -- 172.21.15.27:0/2760625750 <== mon.2 v2:172.21.15.45:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f966c034810 con 0x7f967c06e5d0 2024-09-12T03:16:42.519 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.502+0000 7f9681a8d640 1 -- 172.21.15.27:0/2760625750 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f967c06e5d0 msgr2=0x7f967c0b35c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:42.519 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.502+0000 7f9681a8d640 1 --2- 172.21.15.27:0/2760625750 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f967c06e5d0 0x7f967c0b35c0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f966c0097a0 tx=0x7f966c02c2c0 comp rx=0 tx=0).stop 2024-09-12T03:16:42.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.502+0000 7f9681a8d640 1 -- 172.21.15.27:0/2760625750 shutdown_connections 2024-09-12T03:16:42.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.502+0000 7f9681a8d640 1 --2- 172.21.15.27:0/2760625750 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f967c06e5d0 0x7f967c0b35c0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:42.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.502+0000 7f9681a8d640 1 --2- 172.21.15.27:0/2760625750 >> v2:172.21.15.45:3300/0 conn(0x7f967c15b360 0x7f967c06e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:42.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.502+0000 7f9681a8d640 1 -- 172.21.15.27:0/2760625750 >> 172.21.15.27:0/2760625750 conn(0x7f967c06ce20 msgr2=0x7f967c06d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:42.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.502+0000 7f905d81e640 1 -- 172.21.15.27:0/1589370744 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f9044004800 msgr2=0x7f9044004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:42.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.502+0000 7f9681a8d640 1 -- 172.21.15.27:0/2760625750 shutdown_connections 2024-09-12T03:16:42.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.502+0000 7fe601d9b640 1 --2- 172.21.15.27:0/308011220 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe5fc036470 0x7fe5fc036890 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fe5fc00dd90 tx=0x7fe5f802a2a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b345f4921e622d48 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:42.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.502+0000 7f905d81e640 1 --2- 172.21.15.27:0/1589370744 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f9044004800 0x7f9044004c80 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f904800c3b0 tx=0x7f9044005830 comp rx=0 tx=0).stop 2024-09-12T03:16:42.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.502+0000 7f90577fe640 1 -- 172.21.15.27:0/1589370744 reap_dead start 2024-09-12T03:16:42.522 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.502+0000 7fe60159a640 1 -- 172.21.15.27:0/308011220 <== mon.1 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5f8037070 con 0x7fe5fc036470 2024-09-12T03:16:42.522 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.503+0000 7fe60159a640 1 -- 172.21.15.27:0/308011220 <== mon.1 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5f8032440 con 0x7fe5fc036470 2024-09-12T03:16:42.522 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.503+0000 7fe60159a640 1 -- 172.21.15.27:0/308011220 <== mon.1 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5f8032810 con 0x7fe5fc036470 2024-09-12T03:16:42.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.503+0000 7f1692cf1640 1 Processor -- start 2024-09-12T03:16:42.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.503+0000 7f905d81e640 1 -- 172.21.15.27:0/1589370744 shutdown_connections 2024-09-12T03:16:42.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.503+0000 7f905d81e640 1 -- 172.21.15.27:0/1589370744 >> 172.21.15.27:0/1589370744 conn(0x7f905806ce20 msgr2=0x7f905806d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:42.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.503+0000 7f1692cf1640 1 -- start start 2024-09-12T03:16:42.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.503+0000 7f63833bd640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f637c15af80 0x7f637c06d010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:42.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.503+0000 7f905d81e640 1 -- 172.21.15.27:0/1589370744 shutdown_connections 2024-09-12T03:16:42.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.503+0000 7f63833bd640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f637c0b4570 0x7f637c1642c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:42.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.503+0000 7f6380958640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f637c06d550 0x7f637c15b360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:39242/0 (socket says 172.21.15.27:39242) 2024-09-12T03:16:42.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.503+0000 7fe60359e640 1 -- 172.21.15.27:0/308011220 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe5fc036470 msgr2=0x7fe5fc036890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:42.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.503+0000 7fe60359e640 1 --2- 172.21.15.27:0/308011220 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe5fc036470 0x7fe5fc036890 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fe5fc00dd90 tx=0x7fe5f802a2a0 comp rx=0 tx=0).stop 2024-09-12T03:16:42.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.503+0000 7f63833bd640 1 -- 172.21.15.27:0/1777875661 --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f637c06da60 con 0x7f637c06d550 2024-09-12T03:16:42.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.503+0000 7f6380958640 1 -- 172.21.15.27:0/1777875661 learned_addr learned my addr 172.21.15.27:0/1777875661 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:16:42.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.504+0000 7f6381159640 1 --2- 172.21.15.27:0/1777875661 >> v2:172.21.15.45:3300/0 conn(0x7f637c15af80 0x7f637c06d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:42.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.504+0000 7fe60359e640 1 -- 172.21.15.27:0/308011220 shutdown_connections 2024-09-12T03:16:42.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.504+0000 7fe60359e640 1 --2- 172.21.15.27:0/308011220 >> v2:172.21.15.27:3300/0 conn(0x7fe5fc00ad40 0x7fe5fc00b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:42.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.504+0000 7fe60359e640 1 --2- 172.21.15.27:0/308011220 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe5fc036470 0x7fe5fc036890 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:42.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.504+0000 7fe60359e640 1 -- 172.21.15.27:0/308011220 >> 172.21.15.27:0/308011220 conn(0x7fe5fc01dd80 msgr2=0x7fe5fc0549f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:42.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.504+0000 7fe60359e640 1 -- 172.21.15.27:0/308011220 shutdown_connections 2024-09-12T03:16:42.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.504+0000 7f52bdcb7640 1 Processor -- start 2024-09-12T03:16:42.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.504+0000 7f52bdcb7640 1 -- start start 2024-09-12T03:16:42.528 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.504+0000 7f638195a640 1 --2- 172.21.15.27:0/1777875661 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f637c0b4570 0x7f637c1642c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:42.528 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.504+0000 7fe60359e640 1 -- 172.21.15.27:0/308011220 wait complete. 2024-09-12T03:16:42.529 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.504+0000 7f63833bd640 1 -- 172.21.15.27:0/1777875661 --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f637c06dd40 con 0x7f637c15af80 2024-09-12T03:16:42.529 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.504+0000 7f63833bd640 1 -- 172.21.15.27:0/1777875661 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f637c06e020 con 0x7f637c0b4570 2024-09-12T03:16:42.529 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.502+0000 7f9681a8d640 1 -- 172.21.15.27:0/2760625750 wait complete. 2024-09-12T03:16:42.529 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.503+0000 7f905d81e640 1 -- 172.21.15.27:0/1589370744 wait complete. 2024-09-12T03:16:42.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.504+0000 7f9681a8d640 1 Processor -- start 2024-09-12T03:16:42.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.503+0000 7fe4677fe640 1 --2- 172.21.15.27:0/3701491061 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe4680d1720 0x7fe4680b8190 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fe460009e50 tx=0x7fe46002a6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1c4efbe099e3a0c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:42.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.504+0000 7fe465ffb640 1 -- 172.21.15.27:0/3701491061 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe460036070 con 0x7fe4680d1720 2024-09-12T03:16:42.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.505+0000 7f9681a8d640 1 -- start start 2024-09-12T03:16:42.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.505+0000 7fe465ffb640 1 -- 172.21.15.27:0/3701491061 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe460031600 con 0x7fe4680d1720 2024-09-12T03:16:42.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.505+0000 7fe465ffb640 1 -- 172.21.15.27:0/3701491061 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4600319d0 con 0x7fe4680d1720 2024-09-12T03:16:42.532 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.505+0000 7f1692cf1640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f168c0720c0 0x7f168c1555e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:42.532 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.505+0000 7f1692cf1640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f168c159020 0x7f168c0b5e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:42.532 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.505+0000 7f1692cf1640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f168c15f020 0x7f168c0b6340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:42.532 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.505+0000 7f1692cf1640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f168c06f000 con 0x7f168c159020 2024-09-12T03:16:42.532 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.505+0000 7f1692cf1640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f168c06ed20 con 0x7f168c15f020 2024-09-12T03:16:42.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.505+0000 7f52bdcb7640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f52b815b250 0x7f52b80b57a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:42.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.505+0000 7f52bdcb7640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f52b8071540 0x7f52b8072e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:42.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.505+0000 7f52bdcb7640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f52b8073370 0x7f52b80cd450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:42.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.505+0000 7f52bdcb7640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f52b80737e0 con 0x7f52b815b250 2024-09-12T03:16:42.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.505+0000 7f52bdcb7640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f52b80b5310 con 0x7f52b8071540 2024-09-12T03:16:42.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.505+0000 7f52bdcb7640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f52b80b5ce0 con 0x7f52b8073370 2024-09-12T03:16:42.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.506+0000 7f1692cf1640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f168c159400 con 0x7f168c0720c0 2024-09-12T03:16:42.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.506+0000 7fe46d62e640 1 -- 172.21.15.27:0/3701491061 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe4680d1720 msgr2=0x7fe4680b8190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:42.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.506+0000 7fe46d62e640 1 --2- 172.21.15.27:0/3701491061 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe4680d1720 0x7fe4680b8190 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fe460009e50 tx=0x7fe46002a6d0 comp rx=0 tx=0).stop 2024-09-12T03:16:42.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.505+0000 7f6381159640 1 -- 172.21.15.27:0/1777875661 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f637c0b4570 msgr2=0x7f637c1642c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:42.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.506+0000 7f52b7fff640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f52b8073370 0x7f52b80cd450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:42.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.506+0000 7f52b7fff640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f52b8073370 0x7f52b80cd450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3301/0 says I am v2:172.21.15.27:56530/0 (socket says 172.21.15.27:56530) 2024-09-12T03:16:42.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.506+0000 7f52b7fff640 1 -- 172.21.15.27:0/1672190761 learned_addr learned my addr 172.21.15.27:0/1672190761 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:16:42.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.505+0000 7f6381159640 1 --2- 172.21.15.27:0/1777875661 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f637c0b4570 0x7f637c1642c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:42.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.506+0000 7fe46d62e640 1 -- 172.21.15.27:0/3701491061 shutdown_connections 2024-09-12T03:16:42.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.506+0000 7fe46d62e640 1 --2- 172.21.15.27:0/3701491061 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe4680d1720 0x7fe4680b8190 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:42.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.506+0000 7fe46d62e640 1 --2- 172.21.15.27:0/3701491061 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe46815eb90 0x7fe4680d2df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:42.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.506+0000 7fe46d62e640 1 -- 172.21.15.27:0/3701491061 >> 172.21.15.27:0/3701491061 conn(0x7fe46806dce0 msgr2=0x7fe46806e0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:42.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.506+0000 7f52aeffd640 1 --2- 172.21.15.27:0/1672190761 >> v2:172.21.15.45:3300/0 conn(0x7f52b8071540 0x7f52b8072e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:42.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.505+0000 7f6381159640 1 -- 172.21.15.27:0/1777875661 >> v1:172.21.15.27:6789/0 conn(0x7f637c06d550 legacy=0x7f637c15b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:42.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.506+0000 7f52b77fe640 1 --2- 172.21.15.27:0/1672190761 >> v2:172.21.15.27:3300/0 conn(0x7f52b815b250 0x7f52b80b57a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:42.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.506+0000 7f9681a8d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f967c06e5d0 0x7f967c155640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:42.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.505+0000 7f6381159640 1 -- 172.21.15.27:0/1777875661 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f637c06e300 con 0x7f637c15af80 2024-09-12T03:16:42.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.506+0000 7fe60359e640 1 Processor -- start 2024-09-12T03:16:42.538 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.506+0000 7f168bfff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f168c159020 0x7f168c0b5e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:42.538 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.506+0000 7f168bfff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f168c159020 0x7f168c0b5e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33390/0 (socket says 172.21.15.27:33390) 2024-09-12T03:16:42.538 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.506+0000 7f168bfff640 1 -- 172.21.15.27:0/2577237892 learned_addr learned my addr 172.21.15.27:0/2577237892 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:16:42.538 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.506+0000 7f169128e640 1 --2- 172.21.15.27:0/2577237892 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f168c15f020 0x7f168c0b6340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:42.538 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.506+0000 7f1690a8d640 1 --2- 172.21.15.27:0/2577237892 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f168c0720c0 0x7f168c1555e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:42.538 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.506+0000 7fe60359e640 1 -- start start 2024-09-12T03:16:42.539 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.506+0000 7f52b7fff640 1 -- 172.21.15.27:0/1672190761 >> v2:172.21.15.45:3300/0 conn(0x7f52b8071540 msgr2=0x7f52b8072e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:42.539 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.506+0000 7f52b7fff640 1 --2- 172.21.15.27:0/1672190761 >> v2:172.21.15.45:3300/0 conn(0x7f52b8071540 0x7f52b8072e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:42.539 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.506+0000 7f52b7fff640 1 -- 172.21.15.27:0/1672190761 >> v2:172.21.15.27:3300/0 conn(0x7f52b815b250 msgr2=0x7f52b80b57a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:42.539 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.506+0000 7f52b7fff640 1 --2- 172.21.15.27:0/1672190761 >> v2:172.21.15.27:3300/0 conn(0x7f52b815b250 0x7f52b80b57a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:42.539 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.506+0000 7f52b7fff640 1 -- 172.21.15.27:0/1672190761 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52b806d8b0 con 0x7f52b8073370 2024-09-12T03:16:42.539 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.506+0000 7fe46d62e640 1 -- 172.21.15.27:0/3701491061 shutdown_connections 2024-09-12T03:16:42.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.506+0000 7fe46d62e640 1 -- 172.21.15.27:0/3701491061 wait complete. 2024-09-12T03:16:42.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.506+0000 7f6381159640 1 --2- 172.21.15.27:0/1777875661 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f637c15af80 0x7f637c06d010 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f637c0b32f0 tx=0x7f637802c370 comp rx=0 tx=0).ready entity=mon.1 client_cookie=65f3ae9faacdc96d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:42.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.505+0000 7f905d81e640 1 Processor -- start 2024-09-12T03:16:42.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.507+0000 7f6373fff640 1 -- 172.21.15.27:0/1777875661 <== mon.1 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6378039070 con 0x7f637c15af80 2024-09-12T03:16:42.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.507+0000 7f6373fff640 1 -- 172.21.15.27:0/1777875661 <== mon.1 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6378034640 con 0x7f637c15af80 2024-09-12T03:16:42.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.506+0000 7f967b7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f967c06e5d0 0x7f967c155640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:42.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.506+0000 7f9681a8d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f967c0b3b00 0x7f967c154980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:42.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.506+0000 7f9681a8d640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f967c15b360 0x7f967c152b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:42.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.506+0000 7f9681a8d640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f967c15c7a0 con 0x7f967c06e5d0 2024-09-12T03:16:42.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.506+0000 7f967b7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f967c06e5d0 0x7f967c155640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33400/0 (socket says 172.21.15.27:33400) 2024-09-12T03:16:42.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.506+0000 7f967b7fe640 1 -- 172.21.15.27:0/4197418751 learned_addr learned my addr 172.21.15.27:0/4197418751 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:16:42.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.506+0000 7f9681a8d640 1 -- 172.21.15.27:0/4197418751 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f967c15bfa0 con 0x7f967c0b3b00 2024-09-12T03:16:42.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.506+0000 7f9681a8d640 1 -- 172.21.15.27:0/4197418751 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f967c06faf0 con 0x7f967c15b360 2024-09-12T03:16:42.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.507+0000 7f967affd640 1 --2- 172.21.15.27:0/4197418751 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f967c0b3b00 0x7f967c154980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:42.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.507+0000 7f168bfff640 1 -- 172.21.15.27:0/2577237892 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f168c0720c0 msgr2=0x7f168c1555e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:42.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.507+0000 7f168bfff640 1 --2- 172.21.15.27:0/2577237892 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f168c0720c0 0x7f168c1555e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:42.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.507+0000 7f168bfff640 1 -- 172.21.15.27:0/2577237892 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f168c15f020 msgr2=0x7f168c0b6340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:42.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.507+0000 7f168bfff640 1 --2- 172.21.15.27:0/2577237892 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f168c15f020 0x7f168c0b6340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:42.543 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.507+0000 7f168bfff640 1 -- 172.21.15.27:0/2577237892 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f168c2014e0 con 0x7f168c159020 2024-09-12T03:16:42.543 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.507+0000 7f52b7fff640 1 --2- 172.21.15.27:0/1672190761 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f52b8073370 0x7f52b80cd450 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f52b8071440 tx=0x7f52b002c8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=502ff1ec36a90b11 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:42.543 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.507+0000 7f967affd640 1 -- 172.21.15.27:0/4197418751 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f967c15b360 msgr2=0x7f967c152b00 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-12T03:16:42.543 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.507+0000 7f967affd640 1 --2- 172.21.15.27:0/4197418751 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f967c15b360 0x7f967c152b00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:42.543 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.507+0000 7f967affd640 1 -- 172.21.15.27:0/4197418751 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f967c06e5d0 msgr2=0x7f967c155640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:42.544 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.507+0000 7fe60359e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe5fc00ad40 0x7fe5fc033280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:42.544 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.507+0000 7f6373fff640 1 -- 172.21.15.27:0/1777875661 <== mon.1 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6378039070 con 0x7f637c15af80 2024-09-12T03:16:42.544 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.507+0000 7f52b6ffd640 1 -- 172.21.15.27:0/1672190761 <== mon.2 v2:172.21.15.45:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52b0033640 con 0x7f52b8073370 2024-09-12T03:16:42.545 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.507+0000 7fe60359e640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe5fc036470 0x7fe5fc04af70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:42.545 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.508+0000 7fe60359e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe5fc0541f0 0x7fe5fc048420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:42.545 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.508+0000 7f63833bd640 1 -- 172.21.15.27:0/1777875661 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f637c15af80 msgr2=0x7f637c06d010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:42.545 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.508+0000 7fe60359e640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fe5fc007290 con 0x7fe5fc0541f0 2024-09-12T03:16:42.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.508+0000 7f63833bd640 1 --2- 172.21.15.27:0/1777875661 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f637c15af80 0x7f637c06d010 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f637c0b32f0 tx=0x7f637802c370 comp rx=0 tx=0).stop 2024-09-12T03:16:42.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.508+0000 7fe60359e640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe5fc0480b0 con 0x7fe5fc00ad40 2024-09-12T03:16:42.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.508+0000 7fe60359e640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fe5fc052530 con 0x7fe5fc036470 2024-09-12T03:16:42.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.507+0000 7f52b6ffd640 1 -- 172.21.15.27:0/1672190761 <== mon.2 v2:172.21.15.45:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52b0038070 con 0x7f52b8073370 2024-09-12T03:16:42.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.508+0000 7f63833bd640 1 -- 172.21.15.27:0/1777875661 shutdown_connections 2024-09-12T03:16:42.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.508+0000 7f63833bd640 1 --2- 172.21.15.27:0/1777875661 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f637c0b4570 0x7f637c1642c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:42.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.508+0000 7f63833bd640 1 --2- 172.21.15.27:0/1777875661 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f637c15af80 0x7f637c06d010 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:42.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.508+0000 7f63833bd640 1 -- 172.21.15.27:0/1777875661 >> 172.21.15.27:0/1777875661 conn(0x7f637c062c60 msgr2=0x7f637c06bb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:42.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.508+0000 7f168bfff640 1 --2- 172.21.15.27:0/2577237892 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f168c159020 0x7f168c0b5e00 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f167800e7e0 tx=0x7f167800eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:42.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.507+0000 7f967affd640 1 --2- 172.21.15.27:0/4197418751 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f967c06e5d0 0x7f967c155640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:42.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.507+0000 7f967affd640 1 -- 172.21.15.27:0/4197418751 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f967c157760 con 0x7f967c0b3b00 2024-09-12T03:16:42.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.507+0000 7f967affd640 1 --2- 172.21.15.27:0/4197418751 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f967c0b3b00 0x7f967c154980 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f966c033000 tx=0x7f966c0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:42.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.508+0000 7f9663fff640 1 -- 172.21.15.27:0/4197418751 <== mon.1 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f966c039040 con 0x7f967c0b3b00 2024-09-12T03:16:42.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.508+0000 7f905d81e640 1 -- start start 2024-09-12T03:16:42.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.508+0000 7f9663fff640 1 -- 172.21.15.27:0/4197418751 <== mon.1 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f966c0326f0 con 0x7f967c0b3b00 2024-09-12T03:16:42.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.508+0000 7f16897fa640 1 -- 172.21.15.27:0/2577237892 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f167800f040 con 0x7f168c159020 2024-09-12T03:16:42.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.508+0000 7f9663fff640 1 -- 172.21.15.27:0/4197418751 <== mon.1 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f966c032ae0 con 0x7f967c0b3b00 2024-09-12T03:16:42.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.508+0000 7f16897fa640 1 -- 172.21.15.27:0/2577237892 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1678004030 con 0x7f168c159020 2024-09-12T03:16:42.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.508+0000 7fe46d62e640 1 Processor -- start 2024-09-12T03:16:42.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.508+0000 7f1692cf1640 1 -- 172.21.15.27:0/2577237892 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f168c2017c0 con 0x7f168c159020 2024-09-12T03:16:42.551 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.508+0000 7fe602d9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe5fc0541f0 0x7fe5fc048420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:42.551 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.508+0000 7fe46d62e640 1 -- start start 2024-09-12T03:16:42.551 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.508+0000 7fe602d9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe5fc0541f0 0x7fe5fc048420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33416/0 (socket says 172.21.15.27:33416) 2024-09-12T03:16:42.552 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.508+0000 7fe602d9d640 1 -- 172.21.15.27:0/1324083911 learned_addr learned my addr 172.21.15.27:0/1324083911 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:16:42.552 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.509+0000 7f905d81e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f905814cfd0 0x7f905815de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:42.552 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.509+0000 7f905d81e640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f905815e3a0 0x7f9058157d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:42.552 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.509+0000 7f9681a8d640 1 -- 172.21.15.27:0/4197418751 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f967c157a40 con 0x7f967c0b3b00 2024-09-12T03:16:42.552 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.509+0000 7f9681a8d640 1 -- 172.21.15.27:0/4197418751 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f967c1582e0 con 0x7f967c0b3b00 2024-09-12T03:16:42.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.509+0000 7fe602d9d640 1 -- 172.21.15.27:0/1324083911 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe5fc036470 msgr2=0x7fe5fc04af70 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-12T03:16:42.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.509+0000 7fe602d9d640 1 --2- 172.21.15.27:0/1324083911 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe5fc036470 0x7fe5fc04af70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:42.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.509+0000 7fe602d9d640 1 -- 172.21.15.27:0/1324083911 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe5fc00ad40 msgr2=0x7fe5fc033280 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-12T03:16:42.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.509+0000 7fe602d9d640 1 --2- 172.21.15.27:0/1324083911 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe5fc00ad40 0x7fe5fc033280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:42.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.509+0000 7fe602d9d640 1 -- 172.21.15.27:0/1324083911 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5fc0321e0 con 0x7fe5fc0541f0 2024-09-12T03:16:42.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.509+0000 7fe602d9d640 1 --2- 172.21.15.27:0/1324083911 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe5fc0541f0 0x7fe5fc048420 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fe60407bb90 tx=0x7fe6040697a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:42.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.508+0000 7f52b6ffd640 1 -- 172.21.15.27:0/1672190761 <== mon.2 v2:172.21.15.45:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52b0033a00 con 0x7f52b8073370 2024-09-12T03:16:42.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.508+0000 7f52bdcb7640 1 -- 172.21.15.27:0/1672190761 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f52b8073370 msgr2=0x7f52b80cd450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:42.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.508+0000 7f52bdcb7640 1 --2- 172.21.15.27:0/1672190761 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f52b8073370 0x7f52b80cd450 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f52b8071440 tx=0x7f52b002c8b0 comp rx=0 tx=0).stop 2024-09-12T03:16:42.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.509+0000 7f52bdcb7640 1 -- 172.21.15.27:0/1672190761 shutdown_connections 2024-09-12T03:16:42.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.509+0000 7f52bdcb7640 1 --2- 172.21.15.27:0/1672190761 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f52b8073370 0x7f52b80cd450 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:42.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.509+0000 7f52bdcb7640 1 --2- 172.21.15.27:0/1672190761 >> v2:172.21.15.45:3300/0 conn(0x7f52b8071540 0x7f52b8072e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:42.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.509+0000 7f52bdcb7640 1 --2- 172.21.15.27:0/1672190761 >> v2:172.21.15.27:3300/0 conn(0x7f52b815b250 0x7f52b80b57a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:42.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.509+0000 7f52bdcb7640 1 -- 172.21.15.27:0/1672190761 >> 172.21.15.27:0/1672190761 conn(0x7f52b80b2500 msgr2=0x7f52b80b28e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:42.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.510+0000 7f16897fa640 1 -- 172.21.15.27:0/2577237892 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1678004400 con 0x7f168c159020 2024-09-12T03:16:42.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.510+0000 7f905d81e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f90581f0850 0x7f9058158290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:42.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.509+0000 7f63833bd640 1 -- 172.21.15.27:0/1777875661 shutdown_connections 2024-09-12T03:16:42.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.510+0000 7f905d81e640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f905815c7a0 con 0x7f905814cfd0 2024-09-12T03:16:42.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.510+0000 7f63833bd640 1 -- 172.21.15.27:0/1777875661 wait complete. 2024-09-12T03:16:42.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.510+0000 7f90567fc640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f905815e3a0 0x7f9058157d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:42.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.510+0000 7fe5f2ffd640 1 -- 172.21.15.27:0/1324083911 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6040504d0 con 0x7fe5fc0541f0 2024-09-12T03:16:42.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.510+0000 7f9056ffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f905814cfd0 0x7f905815de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:42.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.510+0000 7f90567fc640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f905815e3a0 0x7f9058157d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3301/0 says I am v2:172.21.15.27:56544/0 (socket says 172.21.15.27:56544) 2024-09-12T03:16:42.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.510+0000 7fe5f2ffd640 1 -- 172.21.15.27:0/1324083911 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe604074260 con 0x7fe5fc0541f0 2024-09-12T03:16:42.558 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.510+0000 7f90567fc640 1 -- 172.21.15.27:0/3331662948 learned_addr learned my addr 172.21.15.27:0/3331662948 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:16:42.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.510+0000 7fe60359e640 1 -- 172.21.15.27:0/1324083911 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5fc0324c0 con 0x7fe5fc0541f0 2024-09-12T03:16:42.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.510+0000 7fe46d62e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe4680d1720 0x7fe46815cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:42.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.510+0000 7f90577fe640 1 --2- 172.21.15.27:0/3331662948 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f90581f0850 0x7f9058158290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:42.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.510+0000 7fe46d62e640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe4680d5200 0x7fe4681e9a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:42.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.510+0000 7fe60359e640 1 -- 172.21.15.27:0/1324083911 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5fc032a80 con 0x7fe5fc0541f0 2024-09-12T03:16:42.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.510+0000 7f905d81e640 1 -- 172.21.15.27:0/3331662948 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f905815bfa0 con 0x7f90581f0850 2024-09-12T03:16:42.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.510+0000 7fe466ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe4680d1720 0x7fe46815cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:42.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.510+0000 7f905d81e640 1 -- 172.21.15.27:0/3331662948 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f905806fb10 con 0x7f905815e3a0 2024-09-12T03:16:42.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.510+0000 7fe466ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe4680d1720 0x7fe46815cd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.27:32834/0 (socket says 172.21.15.27:32834) 2024-09-12T03:16:42.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.510+0000 7f52bdcb7640 1 -- 172.21.15.27:0/1672190761 shutdown_connections 2024-09-12T03:16:42.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.510+0000 7fe466ffd640 1 -- 172.21.15.27:0/3182837971 learned_addr learned my addr 172.21.15.27:0/3182837971 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:16:42.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.511+0000 7fe4667fc640 1 --2- 172.21.15.27:0/3182837971 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe4680d5200 0x7fe4681e9a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:42.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.511+0000 7fe46d62e640 1 --2- 172.21.15.27:0/3182837971 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe46815eb90 0x7fe4680ab210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:42.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.511+0000 7f1692cf1640 1 -- 172.21.15.27:0/2577237892 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f168c201d80 con 0x7f168c159020 2024-09-12T03:16:42.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.511+0000 7f16897fa640 1 -- 172.21.15.27:0/2577237892 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f167800c6d0 con 0x7f168c159020 2024-09-12T03:16:42.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.511+0000 7f90567fc640 1 -- 172.21.15.27:0/3331662948 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f90581f0850 msgr2=0x7f9058158290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:42.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.511+0000 7f90567fc640 1 --2- 172.21.15.27:0/3331662948 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f90581f0850 0x7f9058158290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:42.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.511+0000 7f90567fc640 1 -- 172.21.15.27:0/3331662948 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f905814cfd0 msgr2=0x7f905815de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:42.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.511+0000 7f90567fc640 1 --2- 172.21.15.27:0/3331662948 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f905814cfd0 0x7f905815de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:42.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.511+0000 7f90567fc640 1 -- 172.21.15.27:0/3331662948 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90581f0da0 con 0x7f905815e3a0 2024-09-12T03:16:42.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.511+0000 7f90577fe640 1 --2- 172.21.15.27:0/3331662948 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f90581f0850 0x7f9058158290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:16:42.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.511+0000 7f9056ffd640 1 --2- 172.21.15.27:0/3331662948 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f905814cfd0 0x7f905815de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:16:42.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.510+0000 7f9661ffb640 1 -- 172.21.15.27:0/4197418751 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9654005290 con 0x7f967c0b3b00 2024-09-12T03:16:42.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.512+0000 7fe5f2ffd640 1 -- 172.21.15.27:0/1324083911 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6040953d0 con 0x7fe5fc0541f0 2024-09-12T03:16:42.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.512+0000 7f9663fff640 1 -- 172.21.15.27:0/4197418751 <== mon.1 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f966c044020 con 0x7f967c0b3b00 2024-09-12T03:16:42.567 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.512+0000 7f90567fc640 1 --2- 172.21.15.27:0/3331662948 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f905815e3a0 0x7f9058157d50 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f904802af60 tx=0x7f905815bbb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:42.567 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.512+0000 7f904ffff640 1 -- 172.21.15.27:0/3331662948 <== mon.2 v2:172.21.15.45:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9048004e60 con 0x7f905815e3a0 2024-09-12T03:16:42.567 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.512+0000 7f63833bd640 1 Processor -- start 2024-09-12T03:16:42.567 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.512+0000 7f904ffff640 1 -- 172.21.15.27:0/3331662948 <== mon.2 v2:172.21.15.45:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f904803f070 con 0x7f905815e3a0 2024-09-12T03:16:42.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.512+0000 7f904ffff640 1 -- 172.21.15.27:0/3331662948 <== mon.2 v2:172.21.15.45:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90480074d0 con 0x7f905815e3a0 2024-09-12T03:16:42.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.512+0000 7f16897fa640 1 --2- 172.21.15.27:0/2577237892 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f167006bf00 0x7f167006e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:42.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.512+0000 7f1690a8d640 1 --2- 172.21.15.27:0/2577237892 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f167006bf00 0x7f167006e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:42.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.513+0000 7f16897fa640 1 -- 172.21.15.27:0/2577237892 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f1678089fe0 con 0x7f168c159020 2024-09-12T03:16:42.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.513+0000 7f1692cf1640 1 -- 172.21.15.27:0/2577237892 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f168c06dc50 con 0x7f168c159020 2024-09-12T03:16:42.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.512+0000 7f905d81e640 1 -- 172.21.15.27:0/3331662948 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90581f1080 con 0x7f905815e3a0 2024-09-12T03:16:42.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.512+0000 7f905d81e640 1 -- 172.21.15.27:0/3331662948 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f90581f1640 con 0x7f905815e3a0 2024-09-12T03:16:42.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.513+0000 7fe5f2ffd640 1 -- 172.21.15.27:0/1324083911 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe604095630 con 0x7fe5fc0541f0 2024-09-12T03:16:42.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.512+0000 7f63833bd640 1 -- start start 2024-09-12T03:16:42.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.513+0000 7fe5f2ffd640 1 --2- 172.21.15.27:0/1324083911 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fe5e406bf00 0x7fe5e406e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:42.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.514+0000 7f63833bd640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f637c06d550 0x7f637c0ade50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:42.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.514+0000 7fe5f2ffd640 1 -- 172.21.15.27:0/1324083911 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fe6041069b0 con 0x7fe5fc0541f0 2024-09-12T03:16:42.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.514+0000 7fe60259c640 1 --2- 172.21.15.27:0/1324083911 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fe5e406bf00 0x7fe5e406e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:42.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.514+0000 7f63833bd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f637c0b4570 0x7f637c14c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:42.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.514+0000 7fe60359e640 1 -- 172.21.15.27:0/1324083911 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5e8005290 con 0x7fe5fc0541f0 2024-09-12T03:16:42.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.514+0000 7f6381159640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f637c06d550 0x7f637c0ade50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:42.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.514+0000 7f6381159640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f637c06d550 0x7f637c0ade50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3301/0 says I am v2:172.21.15.27:56572/0 (socket says 172.21.15.27:56572) 2024-09-12T03:16:42.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.514+0000 7f6381159640 1 -- 172.21.15.27:0/2306694138 learned_addr learned my addr 172.21.15.27:0/2306694138 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:16:42.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.514+0000 7f6380958640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f637c0b4570 0x7f637c14c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:42.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.514+0000 7f63833bd640 1 --2- 172.21.15.27:0/2306694138 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f637c15af80 0x7f637c0b2240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:42.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.514+0000 7fe4677fe640 1 --2- 172.21.15.27:0/3182837971 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe46815eb90 0x7fe4680ab210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:42.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.514+0000 7fe46d62e640 1 -- 172.21.15.27:0/3182837971 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fe4680d3330 con 0x7fe46815eb90 2024-09-12T03:16:42.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.514+0000 7fe46d62e640 1 -- 172.21.15.27:0/3182837971 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe4680d4000 con 0x7fe4680d1720 2024-09-12T03:16:42.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.515+0000 7fe46d62e640 1 -- 172.21.15.27:0/3182837971 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fe4680d34b0 con 0x7fe4680d5200 2024-09-12T03:16:42.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.515+0000 7f63833bd640 1 -- 172.21.15.27:0/2306694138 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f637c072320 con 0x7f637c0b4570 2024-09-12T03:16:42.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.515+0000 7f63833bd640 1 -- 172.21.15.27:0/2306694138 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f637c15ba70 con 0x7f637c15af80 2024-09-12T03:16:42.574 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.515+0000 7f904ffff640 1 -- 172.21.15.27:0/3331662948 <== mon.2 v2:172.21.15.45:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9048007670 con 0x7f905815e3a0 2024-09-12T03:16:42.574 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.515+0000 7f638195a640 1 --2- 172.21.15.27:0/2306694138 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f637c15af80 0x7f637c0b2240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:42.574 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.515+0000 7f63833bd640 1 -- 172.21.15.27:0/2306694138 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f637c0724a0 con 0x7f637c06d550 2024-09-12T03:16:42.574 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.515+0000 7f904ffff640 1 --2- 172.21.15.27:0/3331662948 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f903806beb0 0x7f903806e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:42.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.514+0000 7f9663fff640 1 --2- 172.21.15.27:0/4197418751 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f966806be60 0x7f966806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:42.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.515+0000 7f6381159640 1 -- 172.21.15.27:0/2306694138 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f637c15af80 msgr2=0x7f637c0b2240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:42.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.515+0000 7f6381159640 1 --2- 172.21.15.27:0/2306694138 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f637c15af80 0x7f637c0b2240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:42.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.515+0000 7f6381159640 1 -- 172.21.15.27:0/2306694138 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f637c0b4570 msgr2=0x7f637c14c5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:42.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.515+0000 7f6381159640 1 --2- 172.21.15.27:0/2306694138 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f637c0b4570 0x7f637c14c5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:42.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.515+0000 7f6381159640 1 -- 172.21.15.27:0/2306694138 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f637c14cae0 con 0x7f637c06d550 2024-09-12T03:16:42.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.515+0000 7f638195a640 1 --2- 172.21.15.27:0/2306694138 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f637c15af80 0x7f637c0b2240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:16:42.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.516+0000 7f1690a8d640 1 --2- 172.21.15.27:0/2577237892 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f167006bf00 0x7f167006e3c0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f1680034000 tx=0x7f168002e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:42.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.514+0000 7fe60259c640 1 --2- 172.21.15.27:0/1324083911 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fe5e406bf00 0x7fe5e406e3c0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fe5f40014d0 tx=0x7fe5f4007970 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:42.577 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.516+0000 7f6381159640 1 --2- 172.21.15.27:0/2306694138 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f637c06d550 0x7f637c0ade50 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f637802c3b0 tx=0x7f63780046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:42.577 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.516+0000 7f6380958640 1 --2- 172.21.15.27:0/2306694138 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f637c0b4570 0x7f637c14c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:16:42.577 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.516+0000 7f6371ffb640 1 -- 172.21.15.27:0/2306694138 <== mon.2 v2:172.21.15.45:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6378045070 con 0x7f637c06d550 2024-09-12T03:16:42.577 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.516+0000 7f6371ffb640 1 -- 172.21.15.27:0/2306694138 <== mon.2 v2:172.21.15.45:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f637803ea70 con 0x7f637c06d550 2024-09-12T03:16:42.577 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.516+0000 7f6371ffb640 1 -- 172.21.15.27:0/2306694138 <== mon.2 v2:172.21.15.45:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6378039040 con 0x7f637c06d550 2024-09-12T03:16:42.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.515+0000 7fe4677fe640 1 -- 172.21.15.27:0/3182837971 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe4680d5200 msgr2=0x7fe4681e9a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:42.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.515+0000 7fe4677fe640 1 --2- 172.21.15.27:0/3182837971 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe4680d5200 0x7fe4681e9a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:42.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.515+0000 7fe4677fe640 1 -- 172.21.15.27:0/3182837971 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe4680d1720 msgr2=0x7fe46815cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:42.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.515+0000 7fe4677fe640 1 --2- 172.21.15.27:0/3182837971 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe4680d1720 0x7fe46815cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:42.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.515+0000 7fe4677fe640 1 -- 172.21.15.27:0/3182837971 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe46815d7a0 con 0x7fe46815eb90 2024-09-12T03:16:42.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.516+0000 7fe4677fe640 1 --2- 172.21.15.27:0/3182837971 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe46815eb90 0x7fe4680ab210 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fe4680d4d50 tx=0x7fe460004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:42.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.516+0000 7f63833bd640 1 -- 172.21.15.27:0/2306694138 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f637c14b500 con 0x7f637c06d550 2024-09-12T03:16:42.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.516+0000 7fe4667fc640 1 --2- 172.21.15.27:0/3182837971 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe4680d5200 0x7fe4681e9a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:16:42.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.515+0000 7f904ffff640 1 -- 172.21.15.27:0/3331662948 <== mon.2 v2:172.21.15.45:3301/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f90480abdf0 con 0x7f905815e3a0 2024-09-12T03:16:42.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.514+0000 7f9663fff640 1 -- 172.21.15.27:0/4197418751 <== mon.1 v2:172.21.15.45:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f966c0aac70 con 0x7f967c0b3b00 2024-09-12T03:16:42.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.516+0000 7f967b7fe640 1 --2- 172.21.15.27:0/4197418751 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f966806be60 0x7f966806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:42.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.515+0000 7f9056ffd640 1 --2- 172.21.15.27:0/3331662948 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f903806beb0 0x7f903806e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:42.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.516+0000 7f905d81e640 1 -- 172.21.15.27:0/3331662948 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90581f1810 con 0x7f905815e3a0 2024-09-12T03:16:42.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.516+0000 7f63833bd640 1 -- 172.21.15.27:0/2306694138 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f637c14bac0 con 0x7f637c06d550 2024-09-12T03:16:42.581 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.517+0000 7f967b7fe640 1 --2- 172.21.15.27:0/4197418751 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f966806be60 0x7f966806e320 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f967c15a310 tx=0x7f9674009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:42.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.512+0000 7f52bdcb7640 1 -- 172.21.15.27:0/1672190761 wait complete. 2024-09-12T03:16:42.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.514+0000 7f52bdcb7640 1 Processor -- start 2024-09-12T03:16:42.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.515+0000 7f52bdcb7640 1 -- start start 2024-09-12T03:16:42.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.516+0000 7f52bdcb7640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f52b8073370 0x7f52b814ba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:42.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.516+0000 7f52bdcb7640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f52b815b250 0x7f52b8153740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:42.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.516+0000 7f52bdcb7640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f52b815a890 0x7f52b8153c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:42.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.516+0000 7f52b77fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f52b8073370 0x7f52b814ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:42.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.516+0000 7f52b77fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f52b8073370 0x7f52b814ba00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.27:32852/0 (socket says 172.21.15.27:32852) 2024-09-12T03:16:42.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.516+0000 7f52b7fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f52b815a890 0x7f52b8153c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:42.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.516+0000 7f52b7fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f52b815a890 0x7f52b8153c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33440/0 (socket says 172.21.15.27:33440) 2024-09-12T03:16:42.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.516+0000 7f52b77fe640 1 -- 172.21.15.27:0/2011150453 learned_addr learned my addr 172.21.15.27:0/2011150453 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:16:42.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.516+0000 7f52bdcb7640 1 -- 172.21.15.27:0/2011150453 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f52b806d4f0 con 0x7f52b815a890 2024-09-12T03:16:42.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.516+0000 7f52bdcb7640 1 -- 172.21.15.27:0/2011150453 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f52b80b4fa0 con 0x7f52b8073370 2024-09-12T03:16:42.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.516+0000 7f52bdcb7640 1 -- 172.21.15.27:0/2011150453 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f52b806d670 con 0x7f52b815b250 2024-09-12T03:16:42.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.517+0000 7f52aeffd640 1 --2- 172.21.15.27:0/2011150453 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f52b815b250 0x7f52b8153740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:42.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.517+0000 7f52b7fff640 1 -- 172.21.15.27:0/2011150453 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f52b815b250 msgr2=0x7f52b8153740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:42.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.517+0000 7f52b7fff640 1 --2- 172.21.15.27:0/2011150453 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f52b815b250 0x7f52b8153740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:42.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.517+0000 7f52b7fff640 1 -- 172.21.15.27:0/2011150453 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f52b8073370 msgr2=0x7f52b814ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:42.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.517+0000 7f52b7fff640 1 --2- 172.21.15.27:0/2011150453 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f52b8073370 0x7f52b814ba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:42.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.517+0000 7f52b7fff640 1 -- 172.21.15.27:0/2011150453 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52b815c850 con 0x7f52b815a890 2024-09-12T03:16:42.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.517+0000 7f52aeffd640 1 --2- 172.21.15.27:0/2011150453 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f52b815b250 0x7f52b8153740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:16:42.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.517+0000 7f52b7fff640 1 --2- 172.21.15.27:0/2011150453 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f52b815a890 0x7f52b8153c80 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f52b0034000 tx=0x7f52b0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:42.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.517+0000 7f9663fff640 1 -- 172.21.15.27:0/4197418751 <== mon.1 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f967c0b59f0 con 0x7f967c0b3b00 2024-09-12T03:16:42.584 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.517+0000 7f63833bd640 1 -- 172.21.15.27:0/2306694138 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f637c06f120 con 0x7f637c06d550 2024-09-12T03:16:42.584 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.518+0000 7f9056ffd640 1 --2- 172.21.15.27:0/3331662948 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f903806beb0 0x7f903806e370 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f905806dbc0 tx=0x7f9040008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:42.585 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.518+0000 7f6371ffb640 1 -- 172.21.15.27:0/2306694138 <== mon.2 v2:172.21.15.45:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6378051350 con 0x7f637c06d550 2024-09-12T03:16:42.585 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.518+0000 7f6371ffb640 1 --2- 172.21.15.27:0/2306694138 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f636006be30 0x7f636006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:42.585 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.519+0000 7f6380958640 1 --2- 172.21.15.27:0/2306694138 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f636006be30 0x7f636006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:42.585 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.519+0000 7f6380958640 1 --2- 172.21.15.27:0/2306694138 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f636006be30 0x7f636006e2f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f637c15e1c0 tx=0x7f6374009870 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:42.585 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.522+0000 7f6371ffb640 1 -- 172.21.15.27:0/2306694138 <== mon.2 v2:172.21.15.45:3301/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f63780ad530 con 0x7f637c06d550 2024-09-12T03:16:42.585 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.522+0000 7f904ffff640 1 -- 172.21.15.27:0/3331662948 <== mon.2 v2:172.21.15.45:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f9048049060 con 0x7f905815e3a0 2024-09-12T03:16:42.586 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.522+0000 7f16897fa640 1 -- 172.21.15.27:0/2577237892 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f1678017020 con 0x7f168c159020 2024-09-12T03:16:42.586 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.529+0000 7f6371ffb640 1 -- 172.21.15.27:0/2306694138 <== mon.2 v2:172.21.15.45:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f637807d1c0 con 0x7f637c06d550 2024-09-12T03:16:42.586 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.531+0000 7fe44f7fe640 1 -- 172.21.15.27:0/3182837971 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe460036040 con 0x7fe46815eb90 2024-09-12T03:16:42.586 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.532+0000 7fe44f7fe640 1 -- 172.21.15.27:0/3182837971 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe460040a70 con 0x7fe46815eb90 2024-09-12T03:16:42.586 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.532+0000 7fe44f7fe640 1 -- 172.21.15.27:0/3182837971 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe460040e20 con 0x7fe46815eb90 2024-09-12T03:16:42.587 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.532+0000 7fe5f2ffd640 1 -- 172.21.15.27:0/1324083911 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fe6040d53d0 con 0x7fe5fc0541f0 2024-09-12T03:16:42.587 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.532+0000 7fe46d62e640 1 -- 172.21.15.27:0/3182837971 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4681e89c0 con 0x7fe46815eb90 2024-09-12T03:16:42.587 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.532+0000 7fe46d62e640 1 -- 172.21.15.27:0/3182837971 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4681e8f80 con 0x7fe46815eb90 2024-09-12T03:16:42.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.533+0000 7f52b4ff9640 1 -- 172.21.15.27:0/2011150453 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52b0043070 con 0x7f52b815a890 2024-09-12T03:16:42.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.533+0000 7f52b4ff9640 1 -- 172.21.15.27:0/2011150453 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52b003c8f0 con 0x7f52b815a890 2024-09-12T03:16:42.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.533+0000 7fe44d7fa640 1 -- 172.21.15.27:0/3182837971 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe468160210 con 0x7fe46815eb90 2024-09-12T03:16:42.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.535+0000 7fe44f7fe640 1 -- 172.21.15.27:0/3182837971 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe460008930 con 0x7fe46815eb90 2024-09-12T03:16:42.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.536+0000 7fe44f7fe640 1 --2- 172.21.15.27:0/3182837971 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fe45406bf00 0x7fe45406e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:42.589 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.534+0000 7f52b4ff9640 1 -- 172.21.15.27:0/2011150453 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52b003cca0 con 0x7f52b815a890 2024-09-12T03:16:42.589 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.534+0000 7f52bdcb7640 1 -- 172.21.15.27:0/2011150453 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52b815ba90 con 0x7f52b815a890 2024-09-12T03:16:42.589 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.536+0000 7fe44f7fe640 1 -- 172.21.15.27:0/3182837971 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fe4600b4320 con 0x7fe46815eb90 2024-09-12T03:16:42.589 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.534+0000 7f52bdcb7640 1 -- 172.21.15.27:0/2011150453 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52b815c050 con 0x7f52b815a890 2024-09-12T03:16:42.589 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.536+0000 7fe466ffd640 1 --2- 172.21.15.27:0/3182837971 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fe45406bf00 0x7fe45406e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:42.590 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.537+0000 7f52adffb640 1 -- 172.21.15.27:0/2011150453 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52b8152bd0 con 0x7f52b815a890 2024-09-12T03:16:42.590 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.537+0000 7fe466ffd640 1 --2- 172.21.15.27:0/3182837971 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fe45406bf00 0x7fe45406e3c0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fe45800a990 tx=0x7fe458008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:42.590 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.543+0000 7fe44f7fe640 1 -- 172.21.15.27:0/3182837971 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fe460082d40 con 0x7fe46815eb90 2024-09-12T03:16:42.592 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.549+0000 7f52b4ff9640 1 -- 172.21.15.27:0/2011150453 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f52b0046070 con 0x7f52b815a890 2024-09-12T03:16:42.592 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.549+0000 7f52b4ff9640 1 --2- 172.21.15.27:0/2011150453 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f529806bf00 0x7f529806e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:42.592 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.550+0000 7f52b4ff9640 1 -- 172.21.15.27:0/2011150453 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f52b00abf10 con 0x7f52b815a890 2024-09-12T03:16:42.592 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.551+0000 7f52b77fe640 1 --2- 172.21.15.27:0/2011150453 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f529806bf00 0x7f529806e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:42.593 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.553+0000 7f52b77fe640 1 --2- 172.21.15.27:0/2011150453 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f529806bf00 0x7f529806e3c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f52b80cdc10 tx=0x7f52a80098d0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:42.593 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.553+0000 7f52b4ff9640 1 -- 172.21.15.27:0/2011150453 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f52b004d030 con 0x7f52b815a890 2024-09-12T03:16:42.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.854+0000 7fd80cd1f640 1 -- 172.21.15.27:0/1980670100 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7fd7e8005580 con 0x7fd8080b4d00 2024-09-12T03:16:42.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.855+0000 7fd7effff640 1 -- 172.21.15.27:0/1980670100 <== mon.2 v2:172.21.15.45:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fd7f805c9d0 con 0x7fd8080b4d00 2024-09-12T03:16:42.856 INFO:teuthology.orchestra.run.smithi027.stdout:60129542152 2024-09-12T03:16:42.860 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.858+0000 7fd7edffb640 1 -- 172.21.15.27:0/1980670100 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fd7f406bf00 msgr2=0x7fd7f406e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:42.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.858+0000 7fd7edffb640 1 --2- 172.21.15.27:0/1980670100 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fd7f406bf00 0x7fd7f406e3c0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fd800038000 tx=0x7fd800031000 comp rx=0 tx=0).stop 2024-09-12T03:16:42.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.858+0000 7fd7edffb640 1 -- 172.21.15.27:0/1980670100 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fd8080b4d00 msgr2=0x7fd8080aa7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:42.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.858+0000 7fd7edffb640 1 --2- 172.21.15.27:0/1980670100 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fd8080b4d00 0x7fd8080aa7d0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fd7f800c780 tx=0x7fd7f800cc40 comp rx=0 tx=0).stop 2024-09-12T03:16:42.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.859+0000 7fd7edffb640 1 -- 172.21.15.27:0/1980670100 shutdown_connections 2024-09-12T03:16:42.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.859+0000 7fd7edffb640 1 --2- 172.21.15.27:0/1980670100 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fd7f406bf00 0x7fd7f406e3c0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:42.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.859+0000 7fd7edffb640 1 --2- 172.21.15.27:0/1980670100 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd80815a270 0x7fd808155a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:42.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.859+0000 7fd7edffb640 1 --2- 172.21.15.27:0/1980670100 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd8080cfde0 0x7fd8080aae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:42.862 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.859+0000 7fd7edffb640 1 --2- 172.21.15.27:0/1980670100 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fd8080b4d00 0x7fd8080aa7d0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:42.862 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.859+0000 7fd7edffb640 1 -- 172.21.15.27:0/1980670100 >> 172.21.15.27:0/1980670100 conn(0x7fd8080d0210 msgr2=0x7fd8081583c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:42.862 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.859+0000 7fd7edffb640 1 -- 172.21.15.27:0/1980670100 shutdown_connections 2024-09-12T03:16:42.862 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.859+0000 7fd7edffb640 1 -- 172.21.15.27:0/1980670100 wait complete. 2024-09-12T03:16:42.876 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542152 got 60129542152 for osd.1 2024-09-12T03:16:42.876 DEBUG:teuthology.parallel:result is None 2024-09-12T03:16:42.910 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.909+0000 7f9661ffb640 1 -- 172.21.15.27:0/4197418751 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f9654005580 con 0x7f967c0b3b00 2024-09-12T03:16:42.910 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.908+0000 7fe44d7fa640 1 -- 172.21.15.27:0/3182837971 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7fe4680b9240 con 0x7fe46815eb90 2024-09-12T03:16:42.910 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.909+0000 7fe44f7fe640 1 -- 172.21.15.27:0/3182837971 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fe460082640 con 0x7fe46815eb90 2024-09-12T03:16:42.911 INFO:teuthology.orchestra.run.smithi027.stdout:60129542150 2024-09-12T03:16:42.911 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.909+0000 7f9663fff640 1 -- 172.21.15.27:0/4197418751 <== mon.1 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f966c079610 con 0x7f967c0b3b00 2024-09-12T03:16:42.911 INFO:teuthology.orchestra.run.smithi027.stdout:60129542152 2024-09-12T03:16:42.912 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.911+0000 7fe44d7fa640 1 -- 172.21.15.27:0/3182837971 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fe45406bf00 msgr2=0x7fe45406e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:42.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.911+0000 7fe44d7fa640 1 --2- 172.21.15.27:0/3182837971 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fe45406bf00 0x7fe45406e3c0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fe45800a990 tx=0x7fe458008040 comp rx=0 tx=0).stop 2024-09-12T03:16:42.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.911+0000 7fe44d7fa640 1 -- 172.21.15.27:0/3182837971 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe46815eb90 msgr2=0x7fe4680ab210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:42.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.911+0000 7fe44d7fa640 1 --2- 172.21.15.27:0/3182837971 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe46815eb90 0x7fe4680ab210 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fe4680d4d50 tx=0x7fe460004a40 comp rx=0 tx=0).stop 2024-09-12T03:16:42.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.911+0000 7fe44d7fa640 1 -- 172.21.15.27:0/3182837971 shutdown_connections 2024-09-12T03:16:42.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.911+0000 7fe44d7fa640 1 --2- 172.21.15.27:0/3182837971 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fe45406bf00 0x7fe45406e3c0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:42.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.911+0000 7fe44d7fa640 1 --2- 172.21.15.27:0/3182837971 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe46815eb90 0x7fe4680ab210 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:42.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.911+0000 7fe44d7fa640 1 --2- 172.21.15.27:0/3182837971 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe4680d5200 0x7fe4681e9a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:42.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.911+0000 7fe44d7fa640 1 --2- 172.21.15.27:0/3182837971 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe4680d1720 0x7fe46815cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:42.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.911+0000 7fe44d7fa640 1 -- 172.21.15.27:0/3182837971 >> 172.21.15.27:0/3182837971 conn(0x7fe46806dce0 msgr2=0x7fe468160c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:42.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.912+0000 7fe44d7fa640 1 -- 172.21.15.27:0/3182837971 shutdown_connections 2024-09-12T03:16:42.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.912+0000 7fe44d7fa640 1 -- 172.21.15.27:0/3182837971 wait complete. 2024-09-12T03:16:42.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.912+0000 7f9661ffb640 1 -- 172.21.15.27:0/4197418751 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f966806be60 msgr2=0x7f966806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:42.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.912+0000 7f9661ffb640 1 --2- 172.21.15.27:0/4197418751 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f966806be60 0x7f966806e320 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f967c15a310 tx=0x7f9674009040 comp rx=0 tx=0).stop 2024-09-12T03:16:42.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.912+0000 7f9661ffb640 1 -- 172.21.15.27:0/4197418751 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f967c0b3b00 msgr2=0x7f967c154980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:42.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.912+0000 7f9661ffb640 1 --2- 172.21.15.27:0/4197418751 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f967c0b3b00 0x7f967c154980 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f966c033000 tx=0x7f966c0046c0 comp rx=0 tx=0).stop 2024-09-12T03:16:42.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.913+0000 7f9661ffb640 1 -- 172.21.15.27:0/4197418751 shutdown_connections 2024-09-12T03:16:42.915 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.913+0000 7f9661ffb640 1 --2- 172.21.15.27:0/4197418751 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f966806be60 0x7f966806e320 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:42.915 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.913+0000 7f9661ffb640 1 --2- 172.21.15.27:0/4197418751 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f967c15b360 0x7f967c152b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:42.915 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.913+0000 7f9661ffb640 1 --2- 172.21.15.27:0/4197418751 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f967c0b3b00 0x7f967c154980 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:42.915 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.913+0000 7f9661ffb640 1 --2- 172.21.15.27:0/4197418751 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f967c06e5d0 0x7f967c155640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:42.915 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.913+0000 7f9661ffb640 1 -- 172.21.15.27:0/4197418751 >> 172.21.15.27:0/4197418751 conn(0x7f967c06ce20 msgr2=0x7f967c06c310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:42.915 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.913+0000 7f9661ffb640 1 -- 172.21.15.27:0/4197418751 shutdown_connections 2024-09-12T03:16:42.915 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.915+0000 7f9661ffb640 1 -- 172.21.15.27:0/4197418751 wait complete. 2024-09-12T03:16:42.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.924+0000 7f1692cf1640 1 -- 172.21.15.27:0/2577237892 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f168c0cf8b0 con 0x7f168c159020 2024-09-12T03:16:42.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.924+0000 7f16897fa640 1 -- 172.21.15.27:0/2577237892 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f1678058330 con 0x7f168c159020 2024-09-12T03:16:42.925 INFO:teuthology.orchestra.run.smithi027.stdout:60129542151 2024-09-12T03:16:42.929 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542150 for osd.7 2024-09-12T03:16:42.930 DEBUG:teuthology.parallel:result is None 2024-09-12T03:16:42.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.928+0000 7f1692cf1640 1 -- 172.21.15.27:0/2577237892 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f167006bf00 msgr2=0x7f167006e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:42.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.928+0000 7f1692cf1640 1 --2- 172.21.15.27:0/2577237892 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f167006bf00 0x7f167006e3c0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f1680034000 tx=0x7f168002e000 comp rx=0 tx=0).stop 2024-09-12T03:16:42.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.928+0000 7f1692cf1640 1 -- 172.21.15.27:0/2577237892 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f168c159020 msgr2=0x7f168c0b5e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:42.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.928+0000 7f1692cf1640 1 --2- 172.21.15.27:0/2577237892 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f168c159020 0x7f168c0b5e00 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f167800e7e0 tx=0x7f167800eca0 comp rx=0 tx=0).stop 2024-09-12T03:16:42.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.928+0000 7f1692cf1640 1 -- 172.21.15.27:0/2577237892 shutdown_connections 2024-09-12T03:16:42.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.928+0000 7f1692cf1640 1 --2- 172.21.15.27:0/2577237892 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f167006bf00 0x7f167006e3c0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:42.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.928+0000 7f1692cf1640 1 --2- 172.21.15.27:0/2577237892 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f168c15f020 0x7f168c0b6340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:42.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.928+0000 7f1692cf1640 1 --2- 172.21.15.27:0/2577237892 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f168c159020 0x7f168c0b5e00 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:42.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.928+0000 7f1692cf1640 1 --2- 172.21.15.27:0/2577237892 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f168c0720c0 0x7f168c1555e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:42.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.928+0000 7f1692cf1640 1 -- 172.21.15.27:0/2577237892 >> 172.21.15.27:0/2577237892 conn(0x7f168c06cc40 msgr2=0x7f168c0aeb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:42.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.929+0000 7f1692cf1640 1 -- 172.21.15.27:0/2577237892 shutdown_connections 2024-09-12T03:16:42.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.929+0000 7f1692cf1640 1 -- 172.21.15.27:0/2577237892 wait complete. 2024-09-12T03:16:42.934 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542152 got 60129542152 for osd.0 2024-09-12T03:16:42.934 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.932+0000 7f905d81e640 1 -- 172.21.15.27:0/3331662948 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f905806e090 con 0x7f905815e3a0 2024-09-12T03:16:42.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.933+0000 7f904ffff640 1 -- 172.21.15.27:0/3331662948 <== mon.2 v2:172.21.15.45:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f90580b59f0 con 0x7f905815e3a0 2024-09-12T03:16:42.935 DEBUG:teuthology.parallel:result is None 2024-09-12T03:16:42.935 INFO:teuthology.orchestra.run.smithi027.stdout:60129542152 2024-09-12T03:16:42.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.936+0000 7f905d81e640 1 -- 172.21.15.27:0/3331662948 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f903806beb0 msgr2=0x7f903806e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:42.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.936+0000 7f905d81e640 1 --2- 172.21.15.27:0/3331662948 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f903806beb0 0x7f903806e370 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f905806dbc0 tx=0x7f9040008040 comp rx=0 tx=0).stop 2024-09-12T03:16:42.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.936+0000 7f905d81e640 1 -- 172.21.15.27:0/3331662948 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f905815e3a0 msgr2=0x7f9058157d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:42.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.936+0000 7f905d81e640 1 --2- 172.21.15.27:0/3331662948 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f905815e3a0 0x7f9058157d50 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f904802af60 tx=0x7f905815bbb0 comp rx=0 tx=0).stop 2024-09-12T03:16:42.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.936+0000 7f905d81e640 1 -- 172.21.15.27:0/3331662948 shutdown_connections 2024-09-12T03:16:42.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.936+0000 7f905d81e640 1 --2- 172.21.15.27:0/3331662948 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f903806beb0 0x7f903806e370 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:42.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.936+0000 7f905d81e640 1 --2- 172.21.15.27:0/3331662948 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f90581f0850 0x7f9058158290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:42.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.936+0000 7f905d81e640 1 --2- 172.21.15.27:0/3331662948 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f905815e3a0 0x7f9058157d50 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:42.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.936+0000 7f905d81e640 1 --2- 172.21.15.27:0/3331662948 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f905814cfd0 0x7f905815de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:42.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.936+0000 7f905d81e640 1 -- 172.21.15.27:0/3331662948 >> 172.21.15.27:0/3331662948 conn(0x7f905806ce20 msgr2=0x7f90580cd800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:42.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.937+0000 7f905d81e640 1 -- 172.21.15.27:0/3331662948 shutdown_connections 2024-09-12T03:16:42.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.937+0000 7f905d81e640 1 -- 172.21.15.27:0/3331662948 wait complete. 2024-09-12T03:16:42.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.940+0000 7f52adffb640 1 -- 172.21.15.27:0/2011150453 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f52b81528f0 con 0x7f52b815a890 2024-09-12T03:16:42.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.940+0000 7f52b4ff9640 1 -- 172.21.15.27:0/2011150453 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f52b007aa50 con 0x7f52b815a890 2024-09-12T03:16:42.941 INFO:teuthology.orchestra.run.smithi027.stdout:60129542150 2024-09-12T03:16:42.942 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.941+0000 7f63833bd640 1 -- 172.21.15.27:0/2306694138 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f637c159980 con 0x7f637c06d550 2024-09-12T03:16:42.943 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542151 got 60129542151 for osd.2 2024-09-12T03:16:42.943 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.942+0000 7f6371ffb640 1 -- 172.21.15.27:0/2306694138 <== mon.2 v2:172.21.15.45:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f63780814c0 con 0x7f637c06d550 2024-09-12T03:16:42.943 INFO:teuthology.orchestra.run.smithi027.stdout:60129542151 2024-09-12T03:16:42.944 DEBUG:teuthology.parallel:result is None 2024-09-12T03:16:42.944 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.943+0000 7fe60359e640 1 -- 172.21.15.27:0/1324083911 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7fe5e8005580 con 0x7fe5fc0541f0 2024-09-12T03:16:42.944 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.943+0000 7f52adffb640 1 -- 172.21.15.27:0/2011150453 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f529806bf00 msgr2=0x7f529806e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:42.944 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.943+0000 7f52adffb640 1 --2- 172.21.15.27:0/2011150453 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f529806bf00 0x7f529806e3c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f52b80cdc10 tx=0x7f52a80098d0 comp rx=0 tx=0).stop 2024-09-12T03:16:42.945 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.943+0000 7f52adffb640 1 -- 172.21.15.27:0/2011150453 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f52b815a890 msgr2=0x7f52b8153c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:42.945 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.943+0000 7f52adffb640 1 --2- 172.21.15.27:0/2011150453 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f52b815a890 0x7f52b8153c80 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f52b0034000 tx=0x7f52b0004730 comp rx=0 tx=0).stop 2024-09-12T03:16:42.945 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.943+0000 7f52adffb640 1 -- 172.21.15.27:0/2011150453 shutdown_connections 2024-09-12T03:16:42.945 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.943+0000 7f52adffb640 1 --2- 172.21.15.27:0/2011150453 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f529806bf00 0x7f529806e3c0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:42.945 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.943+0000 7f52adffb640 1 --2- 172.21.15.27:0/2011150453 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f52b815a890 0x7f52b8153c80 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:42.945 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.943+0000 7f52adffb640 1 --2- 172.21.15.27:0/2011150453 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f52b815b250 0x7f52b8153740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:42.945 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.943+0000 7f52adffb640 1 --2- 172.21.15.27:0/2011150453 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f52b8073370 0x7f52b814ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:42.945 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.943+0000 7f52adffb640 1 -- 172.21.15.27:0/2011150453 >> 172.21.15.27:0/2011150453 conn(0x7f52b80b2500 msgr2=0x7f52b80cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:42.945 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.943+0000 7f52adffb640 1 -- 172.21.15.27:0/2011150453 shutdown_connections 2024-09-12T03:16:42.945 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.943+0000 7f52adffb640 1 -- 172.21.15.27:0/2011150453 wait complete. 2024-09-12T03:16:42.945 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.943+0000 7fe5f2ffd640 1 -- 172.21.15.27:0/1324083911 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fe6040d4cd0 con 0x7fe5fc0541f0 2024-09-12T03:16:42.946 INFO:teuthology.orchestra.run.smithi027.stdout:60129542151 2024-09-12T03:16:42.947 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.945+0000 7f63833bd640 1 -- 172.21.15.27:0/2306694138 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f636006be30 msgr2=0x7f636006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:42.947 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.945+0000 7f63833bd640 1 --2- 172.21.15.27:0/2306694138 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f636006be30 0x7f636006e2f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f637c15e1c0 tx=0x7f6374009870 comp rx=0 tx=0).stop 2024-09-12T03:16:42.947 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.945+0000 7f63833bd640 1 -- 172.21.15.27:0/2306694138 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f637c06d550 msgr2=0x7f637c0ade50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:42.947 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.945+0000 7f63833bd640 1 --2- 172.21.15.27:0/2306694138 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f637c06d550 0x7f637c0ade50 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f637802c3b0 tx=0x7f63780046c0 comp rx=0 tx=0).stop 2024-09-12T03:16:42.947 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.946+0000 7f63833bd640 1 -- 172.21.15.27:0/2306694138 shutdown_connections 2024-09-12T03:16:42.947 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.946+0000 7f63833bd640 1 --2- 172.21.15.27:0/2306694138 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f636006be30 0x7f636006e2f0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:42.947 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.946+0000 7f63833bd640 1 --2- 172.21.15.27:0/2306694138 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f637c15af80 0x7f637c0b2240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:42.947 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.946+0000 7f63833bd640 1 --2- 172.21.15.27:0/2306694138 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f637c0b4570 0x7f637c14c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:42.947 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.946+0000 7f63833bd640 1 --2- 172.21.15.27:0/2306694138 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f637c06d550 0x7f637c0ade50 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:42.947 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.946+0000 7f63833bd640 1 -- 172.21.15.27:0/2306694138 >> 172.21.15.27:0/2306694138 conn(0x7f637c062c60 msgr2=0x7f637c0727e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:42.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.946+0000 7f63833bd640 1 -- 172.21.15.27:0/2306694138 shutdown_connections 2024-09-12T03:16:42.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.946+0000 7f63833bd640 1 -- 172.21.15.27:0/2306694138 wait complete. 2024-09-12T03:16:42.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.946+0000 7fe60359e640 1 -- 172.21.15.27:0/1324083911 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fe5e406bf00 msgr2=0x7fe5e406e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:42.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.946+0000 7fe60359e640 1 --2- 172.21.15.27:0/1324083911 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fe5e406bf00 0x7fe5e406e3c0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fe5f40014d0 tx=0x7fe5f4007970 comp rx=0 tx=0).stop 2024-09-12T03:16:42.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.946+0000 7fe60359e640 1 -- 172.21.15.27:0/1324083911 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe5fc0541f0 msgr2=0x7fe5fc048420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:42.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.946+0000 7fe60359e640 1 --2- 172.21.15.27:0/1324083911 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe5fc0541f0 0x7fe5fc048420 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fe60407bb90 tx=0x7fe6040697a0 comp rx=0 tx=0).stop 2024-09-12T03:16:42.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.946+0000 7fe60359e640 1 -- 172.21.15.27:0/1324083911 shutdown_connections 2024-09-12T03:16:42.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.946+0000 7fe60359e640 1 --2- 172.21.15.27:0/1324083911 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fe5e406bf00 0x7fe5e406e3c0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:42.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.946+0000 7fe60359e640 1 --2- 172.21.15.27:0/1324083911 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe5fc0541f0 0x7fe5fc048420 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:42.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.946+0000 7fe60359e640 1 --2- 172.21.15.27:0/1324083911 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe5fc036470 0x7fe5fc04af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:42.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.946+0000 7fe60359e640 1 --2- 172.21.15.27:0/1324083911 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe5fc00ad40 0x7fe5fc033280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:42.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.946+0000 7fe60359e640 1 -- 172.21.15.27:0/1324083911 >> 172.21.15.27:0/1324083911 conn(0x7fe5fc01dd80 msgr2=0x7fe5fc035ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:42.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.946+0000 7fe60359e640 1 -- 172.21.15.27:0/1324083911 shutdown_connections 2024-09-12T03:16:42.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:42.946+0000 7fe60359e640 1 -- 172.21.15.27:0/1324083911 wait complete. 2024-09-12T03:16:42.952 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542152 got 60129542152 for osd.6 2024-09-12T03:16:42.952 DEBUG:teuthology.parallel:result is None 2024-09-12T03:16:42.959 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542150 for osd.4 2024-09-12T03:16:42.959 DEBUG:teuthology.parallel:result is None 2024-09-12T03:16:42.960 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542151 got 60129542151 for osd.5 2024-09-12T03:16:42.961 DEBUG:teuthology.parallel:result is None 2024-09-12T03:16:42.961 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542151 got 60129542151 for osd.3 2024-09-12T03:16:42.962 DEBUG:teuthology.parallel:result is None 2024-09-12T03:16:42.962 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-12T03:16:42.962 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-12T03:16:43.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.102+0000 7ff21559f640 1 Processor -- start 2024-09-12T03:16:43.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.102+0000 7ff21559f640 1 -- start start 2024-09-12T03:16:43.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.103+0000 7ff21559f640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ff21006c160 0x7ff21006c540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:43.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.103+0000 7ff21559f640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff2101634f0 con 0x7ff2100d4090 2024-09-12T03:16:43.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.103+0000 7ff21559f640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff2100d3920 con 0x7ff21006ca80 2024-09-12T03:16:43.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.103+0000 7ff21559f640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7ff2100d54d0 con 0x7ff21006c160 2024-09-12T03:16:43.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.103+0000 7ff214d9e640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7ff2100d4090 0x7ff210162de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:39244/0 (socket says 172.21.15.27:39244) 2024-09-12T03:16:43.108 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.103+0000 7ff214d9e640 1 -- 172.21.15.27:0/4155158902 learned_addr learned my addr 172.21.15.27:0/4155158902 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:16:43.108 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.103+0000 7ff20ffff640 1 --2- 172.21.15.27:0/4155158902 >> v2:172.21.15.45:3300/0 conn(0x7ff21006c160 0x7ff21006c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:43.108 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.103+0000 7ff20effd640 1 -- 172.21.15.27:0/4155158902 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1673394448 0 0) 0x7ff2101634f0 con 0x7ff2100d4090 2024-09-12T03:16:43.108 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.103+0000 7ff20effd640 1 -- 172.21.15.27:0/4155158902 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff1f8003520 con 0x7ff2100d4090 2024-09-12T03:16:43.108 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.104+0000 7ff20effd640 1 -- 172.21.15.27:0/4155158902 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2736736875 0 0) 0x7ff1f8003520 con 0x7ff2100d4090 2024-09-12T03:16:43.108 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.104+0000 7ff20effd640 1 -- 172.21.15.27:0/4155158902 >> v2:172.21.15.45:3300/0 conn(0x7ff21006c160 msgr2=0x7ff21006c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:43.108 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.104+0000 7ff20effd640 1 --2- 172.21.15.27:0/4155158902 >> v2:172.21.15.45:3300/0 conn(0x7ff21006c160 0x7ff21006c540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:43.108 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.104+0000 7ff20effd640 1 -- 172.21.15.27:0/4155158902 >> v1:172.21.15.45:6789/0 conn(0x7ff21006ca80 legacy=0x7ff2101626d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:43.108 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.104+0000 7ff20effd640 1 -- 172.21.15.27:0/4155158902 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2100d3360 con 0x7ff2100d4090 2024-09-12T03:16:43.109 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.104+0000 7ff20effd640 1 -- 172.21.15.27:0/4155158902 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 847755213 0 0) 0x7ff2080019f0 con 0x7ff2100d4090 2024-09-12T03:16:43.109 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.104+0000 7ff20effd640 1 -- 172.21.15.27:0/4155158902 >> v1:172.21.15.27:6789/0 conn(0x7ff2100d4090 legacy=0x7ff210162de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:43.109 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.104+0000 7ff20effd640 1 --2- 172.21.15.27:0/4155158902 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ff1f8003a90 0x7ff1f8003eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:43.109 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.104+0000 7ff20effd640 1 --2- 172.21.15.27:0/4155158902 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff1f8004800 0x7ff1f8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:43.109 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.104+0000 7ff20effd640 1 --2- 172.21.15.27:0/4155158902 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff1f80053c0 0x7ff1f800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:43.109 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.104+0000 7ff20effd640 1 -- 172.21.15.27:0/4155158902 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7ff2100d54d0 con 0x7ff1f8004800 2024-09-12T03:16:43.109 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.104+0000 7ff20effd640 1 -- 172.21.15.27:0/4155158902 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ff1f800c280 con 0x7ff1f80053c0 2024-09-12T03:16:43.109 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.104+0000 7ff20effd640 1 -- 172.21.15.27:0/4155158902 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7ff1f800c480 con 0x7ff1f8003a90 2024-09-12T03:16:43.109 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.104+0000 7ff20f7fe640 1 --2- 172.21.15.27:0/4155158902 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff1f8004800 0x7ff1f8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:43.109 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.104+0000 7ff20ffff640 1 --2- 172.21.15.27:0/4155158902 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ff1f8003a90 0x7ff1f8003eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:43.109 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.104+0000 7ff214d9e640 1 --2- 172.21.15.27:0/4155158902 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff1f80053c0 0x7ff1f800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:43.109 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.104+0000 7ff20f7fe640 1 -- 172.21.15.27:0/4155158902 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ff1f8003a90 msgr2=0x7ff1f8003eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:43.109 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.104+0000 7ff20f7fe640 1 --2- 172.21.15.27:0/4155158902 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ff1f8003a90 0x7ff1f8003eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:43.110 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.104+0000 7ff20f7fe640 1 -- 172.21.15.27:0/4155158902 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff1f80053c0 msgr2=0x7ff1f800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:43.110 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.104+0000 7ff20f7fe640 1 --2- 172.21.15.27:0/4155158902 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff1f80053c0 0x7ff1f800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:43.110 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.104+0000 7ff20f7fe640 1 -- 172.21.15.27:0/4155158902 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1f8003520 con 0x7ff1f8004800 2024-09-12T03:16:43.110 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.104+0000 7ff214d9e640 1 -- 172.21.15.27:0/4155158902 reap_dead start 2024-09-12T03:16:43.110 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.105+0000 7ff20f7fe640 1 --2- 172.21.15.27:0/4155158902 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff1f8004800 0x7ff1f8004c80 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7ff2100c8080 tx=0x7ff1f8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b63001d4bad34ea4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:43.110 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.105+0000 7ff20effd640 1 -- 172.21.15.27:0/4155158902 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff204001370 con 0x7ff1f8004800 2024-09-12T03:16:43.110 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.105+0000 7ff20effd640 1 -- 172.21.15.27:0/4155158902 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2040052d0 con 0x7ff1f8004800 2024-09-12T03:16:43.110 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.105+0000 7ff20effd640 1 -- 172.21.15.27:0/4155158902 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff20402ab70 con 0x7ff1f8004800 2024-09-12T03:16:43.110 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.106+0000 7ff21559f640 1 -- 172.21.15.27:0/4155158902 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff1f8004800 msgr2=0x7ff1f8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:43.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.106+0000 7ff21559f640 1 --2- 172.21.15.27:0/4155158902 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff1f8004800 0x7ff1f8004c80 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7ff2100c8080 tx=0x7ff1f8005830 comp rx=0 tx=0).stop 2024-09-12T03:16:43.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.106+0000 7ff21559f640 1 -- 172.21.15.27:0/4155158902 shutdown_connections 2024-09-12T03:16:43.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.106+0000 7ff21559f640 1 --2- 172.21.15.27:0/4155158902 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff1f8004800 0x7ff1f8004c80 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:43.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.106+0000 7ff21559f640 1 -- 172.21.15.27:0/4155158902 >> 172.21.15.27:0/4155158902 conn(0x7ff2100d45e0 msgr2=0x7ff2100d49f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:43.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.106+0000 7ff21559f640 1 -- 172.21.15.27:0/4155158902 shutdown_connections 2024-09-12T03:16:43.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.106+0000 7ff21559f640 1 -- 172.21.15.27:0/4155158902 wait complete. 2024-09-12T03:16:43.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.107+0000 7ff21559f640 1 Processor -- start 2024-09-12T03:16:43.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.107+0000 7ff21559f640 1 -- start start 2024-09-12T03:16:43.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.108+0000 7ff21559f640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ff1f8004800 0x7ff21015e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:43.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.108+0000 7ff21559f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff2100cd5c0 0x7ff2100ac610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:43.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.108+0000 7ff21559f640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff2100cda40 0x7ff2100acb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:43.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.108+0000 7ff21559f640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7ff2100cf830 con 0x7ff2100cda40 2024-09-12T03:16:43.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.108+0000 7ff21559f640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ff2100cf550 con 0x7ff2100cd5c0 2024-09-12T03:16:43.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.108+0000 7ff21559f640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7ff21006c540 con 0x7ff1f8004800 2024-09-12T03:16:43.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.108+0000 7ff214d9e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff2100cda40 0x7ff2100acb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:43.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.108+0000 7ff214d9e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff2100cda40 0x7ff2100acb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33456/0 (socket says 172.21.15.27:33456) 2024-09-12T03:16:43.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.108+0000 7ff214d9e640 1 -- 172.21.15.27:0/1348691116 learned_addr learned my addr 172.21.15.27:0/1348691116 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:16:43.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.108+0000 7ff20ffff640 1 --2- 172.21.15.27:0/1348691116 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ff1f8004800 0x7ff21015e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:43.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.108+0000 7ff20f7fe640 1 --2- 172.21.15.27:0/1348691116 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff2100cd5c0 0x7ff2100ac610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:43.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.108+0000 7ff214d9e640 1 -- 172.21.15.27:0/1348691116 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ff1f8004800 msgr2=0x7ff21015e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:43.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.108+0000 7ff214d9e640 1 --2- 172.21.15.27:0/1348691116 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ff1f8004800 0x7ff21015e460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:43.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.108+0000 7ff214d9e640 1 -- 172.21.15.27:0/1348691116 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff2100cd5c0 msgr2=0x7ff2100ac610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:43.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.108+0000 7ff214d9e640 1 --2- 172.21.15.27:0/1348691116 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff2100cd5c0 0x7ff2100ac610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:43.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.108+0000 7ff214d9e640 1 -- 172.21.15.27:0/1348691116 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2100cc310 con 0x7ff2100cda40 2024-09-12T03:16:43.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.108+0000 7ff20ffff640 1 --2- 172.21.15.27:0/1348691116 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ff1f8004800 0x7ff21015e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:16:43.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.108+0000 7ff214d9e640 1 --2- 172.21.15.27:0/1348691116 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff2100cda40 0x7ff2100acb50 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7ff208000f80 tx=0x7ff208007a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:43.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.109+0000 7ff20cff9640 1 -- 172.21.15.27:0/1348691116 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff20800ab00 con 0x7ff2100cda40 2024-09-12T03:16:43.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.109+0000 7ff20cff9640 1 -- 172.21.15.27:0/1348691116 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff208019070 con 0x7ff2100cda40 2024-09-12T03:16:43.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.109+0000 7ff21559f640 1 -- 172.21.15.27:0/1348691116 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2100cc5f0 con 0x7ff2100cda40 2024-09-12T03:16:43.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.109+0000 7ff20cff9640 1 -- 172.21.15.27:0/1348691116 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2080133f0 con 0x7ff2100cda40 2024-09-12T03:16:43.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.109+0000 7ff21559f640 1 -- 172.21.15.27:0/1348691116 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2100ccbb0 con 0x7ff2100cda40 2024-09-12T03:16:43.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.110+0000 7ff21559f640 1 -- 172.21.15.27:0/1348691116 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2101626d0 con 0x7ff2100cda40 2024-09-12T03:16:43.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.110+0000 7ff20cff9640 1 -- 172.21.15.27:0/1348691116 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff208013590 con 0x7ff2100cda40 2024-09-12T03:16:43.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.111+0000 7ff20cff9640 1 --2- 172.21.15.27:0/1348691116 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7ff1fc06be30 0x7ff1fc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:43.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.111+0000 7ff20cff9640 1 -- 172.21.15.27:0/1348691116 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7ff20808b310 con 0x7ff2100cda40 2024-09-12T03:16:43.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.111+0000 7ff20ffff640 1 --2- 172.21.15.27:0/1348691116 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7ff1fc06be30 0x7ff1fc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:43.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.111+0000 7ff20ffff640 1 --2- 172.21.15.27:0/1348691116 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7ff1fc06be30 0x7ff1fc06e2f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7ff200009bc0 tx=0x7ff200008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:43.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.115+0000 7ff20cff9640 1 -- 172.21.15.27:0/1348691116 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7ff208059cc0 con 0x7ff2100cda40 2024-09-12T03:16:43.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.288+0000 7ff21559f640 1 -- 172.21.15.27:0/1348691116 --> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff2100cc0c0 con 0x7ff1fc06be30 2024-09-12T03:16:43.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.294+0000 7ff20cff9640 1 -- 172.21.15.27:0/1348691116 <== mgr.4102 v2:172.21.15.45:6832/3923821219 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251837 (secure 0 0 0) 0x7ff2100cc0c0 con 0x7ff1fc06be30 2024-09-12T03:16:43.298 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:16:43.299 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2024-09-12T03:16:43.301 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.297+0000 7ff21559f640 1 -- 172.21.15.27:0/1348691116 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7ff1fc06be30 msgr2=0x7ff1fc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:43.301 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.297+0000 7ff21559f640 1 --2- 172.21.15.27:0/1348691116 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7ff1fc06be30 0x7ff1fc06e2f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7ff200009bc0 tx=0x7ff200008040 comp rx=0 tx=0).stop 2024-09-12T03:16:43.301 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.297+0000 7ff21559f640 1 -- 172.21.15.27:0/1348691116 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff2100cda40 msgr2=0x7ff2100acb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:43.301 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.297+0000 7ff21559f640 1 --2- 172.21.15.27:0/1348691116 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff2100cda40 0x7ff2100acb50 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7ff208000f80 tx=0x7ff208007a90 comp rx=0 tx=0).stop 2024-09-12T03:16:43.301 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.297+0000 7ff21559f640 1 -- 172.21.15.27:0/1348691116 shutdown_connections 2024-09-12T03:16:43.301 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.297+0000 7ff21559f640 1 --2- 172.21.15.27:0/1348691116 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7ff1fc06be30 0x7ff1fc06e2f0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:43.302 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.297+0000 7ff21559f640 1 --2- 172.21.15.27:0/1348691116 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff2100cda40 0x7ff2100acb50 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:43.302 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.297+0000 7ff21559f640 1 --2- 172.21.15.27:0/1348691116 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff2100cd5c0 0x7ff2100ac610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:43.302 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.297+0000 7ff21559f640 1 --2- 172.21.15.27:0/1348691116 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ff1f8004800 0x7ff21015e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:43.302 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.297+0000 7ff21559f640 1 -- 172.21.15.27:0/1348691116 >> 172.21.15.27:0/1348691116 conn(0x7ff2100d45e0 msgr2=0x7ff2100b9a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:43.302 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.297+0000 7ff21559f640 1 -- 172.21.15.27:0/1348691116 shutdown_connections 2024-09-12T03:16:43.302 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.297+0000 7ff21559f640 1 -- 172.21.15.27:0/1348691116 wait complete. 2024-09-12T03:16:43.309 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2024-09-12T03:16:42.054875+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235760,"kb_used_data":2888,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738960,"statfs":{"total":773094113280,"available":772852695040,"internally_reserved":0,"allocated":2957312,"data_stored":1778860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.002251"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:33.384323+0000","last_change":"2024-09-12T03:16:31.583498+0000","last_active":"2024-09-12T03:16:33.384323+0000","last_peered":"2024-09-12T03:16:33.384323+0000","last_clean":"2024-09-12T03:16:33.384323+0000","last_became_active":"2024-09-12T03:16:31.583196+0000","last_became_peered":"2024-09-12T03:16:31.583196+0000","last_unstale":"2024-09-12T03:16:33.384323+0000","last_undegraded":"2024-09-12T03:16:33.384323+0000","last_fullsized":"2024-09-12T03:16:33.384323+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:08:48.647955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:32.578394+0000","last_change":"2024-09-12T03:16:31.584109+0000","last_active":"2024-09-12T03:16:32.578394+0000","last_peered":"2024-09-12T03:16:32.578394+0000","last_clean":"2024-09-12T03:16:32.578394+0000","last_became_active":"2024-09-12T03:16:31.583836+0000","last_became_peered":"2024-09-12T03:16:31.583836+0000","last_unstale":"2024-09-12T03:16:32.578394+0000","last_undegraded":"2024-09-12T03:16:32.578394+0000","last_fullsized":"2024-09-12T03:16:32.578394+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:27:20.447453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:33.384286+0000","last_change":"2024-09-12T03:16:31.574829+0000","last_active":"2024-09-12T03:16:33.384286+0000","last_peered":"2024-09-12T03:16:33.384286+0000","last_clean":"2024-09-12T03:16:33.384286+0000","last_became_active":"2024-09-12T03:16:31.574523+0000","last_became_peered":"2024-09-12T03:16:31.574523+0000","last_unstale":"2024-09-12T03:16:33.384286+0000","last_undegraded":"2024-09-12T03:16:33.384286+0000","last_fullsized":"2024-09-12T03:16:33.384286+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:33:01.912717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:32.580588+0000","last_change":"2024-09-12T03:16:31.585853+0000","last_active":"2024-09-12T03:16:32.580588+0000","last_peered":"2024-09-12T03:16:32.580588+0000","last_clean":"2024-09-12T03:16:32.580588+0000","last_became_active":"2024-09-12T03:16:31.583206+0000","last_became_peered":"2024-09-12T03:16:31.583206+0000","last_unstale":"2024-09-12T03:16:32.580588+0000","last_undegraded":"2024-09-12T03:16:32.580588+0000","last_fullsized":"2024-09-12T03:16:32.580588+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:05:29.506554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:33.383913+0000","last_change":"2024-09-12T03:16:31.582095+0000","last_active":"2024-09-12T03:16:33.383913+0000","last_peered":"2024-09-12T03:16:33.383913+0000","last_clean":"2024-09-12T03:16:33.383913+0000","last_became_active":"2024-09-12T03:16:31.581414+0000","last_became_peered":"2024-09-12T03:16:31.581414+0000","last_unstale":"2024-09-12T03:16:33.383913+0000","last_undegraded":"2024-09-12T03:16:33.383913+0000","last_fullsized":"2024-09-12T03:16:33.383913+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:44:06.896638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:32.579523+0000","last_change":"2024-09-12T03:16:31.584676+0000","last_active":"2024-09-12T03:16:32.579523+0000","last_peered":"2024-09-12T03:16:32.579523+0000","last_clean":"2024-09-12T03:16:32.579523+0000","last_became_active":"2024-09-12T03:16:31.584392+0000","last_became_peered":"2024-09-12T03:16:31.584392+0000","last_unstale":"2024-09-12T03:16:32.579523+0000","last_undegraded":"2024-09-12T03:16:32.579523+0000","last_fullsized":"2024-09-12T03:16:32.579523+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:03:19.355158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:33.383478+0000","last_change":"2024-09-12T03:16:31.587169+0000","last_active":"2024-09-12T03:16:33.383478+0000","last_peered":"2024-09-12T03:16:33.383478+0000","last_clean":"2024-09-12T03:16:33.383478+0000","last_became_active":"2024-09-12T03:16:31.586884+0000","last_became_peered":"2024-09-12T03:16:31.586884+0000","last_unstale":"2024-09-12T03:16:33.383478+0000","last_undegraded":"2024-09-12T03:16:33.383478+0000","last_fullsized":"2024-09-12T03:16:33.383478+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:35:22.374781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:33.383041+0000","last_change":"2024-09-12T03:16:31.592684+0000","last_active":"2024-09-12T03:16:33.383041+0000","last_peered":"2024-09-12T03:16:33.383041+0000","last_clean":"2024-09-12T03:16:33.383041+0000","last_became_active":"2024-09-12T03:16:31.592374+0000","last_became_peered":"2024-09-12T03:16:31.592374+0000","last_unstale":"2024-09-12T03:16:33.383041+0000","last_undegraded":"2024-09-12T03:16:33.383041+0000","last_fullsized":"2024-09-12T03:16:33.383041+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:18:57.036317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:33.384083+0000","last_change":"2024-09-12T03:16:31.590609+0000","last_active":"2024-09-12T03:16:33.384083+0000","last_peered":"2024-09-12T03:16:33.384083+0000","last_clean":"2024-09-12T03:16:33.384083+0000","last_became_active":"2024-09-12T03:16:31.590320+0000","last_became_peered":"2024-09-12T03:16:31.590320+0000","last_unstale":"2024-09-12T03:16:33.384083+0000","last_undegraded":"2024-09-12T03:16:33.384083+0000","last_fullsized":"2024-09-12T03:16:33.384083+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:56:03.306616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:33.384773+0000","last_change":"2024-09-12T03:16:31.584375+0000","last_active":"2024-09-12T03:16:33.384773+0000","last_peered":"2024-09-12T03:16:33.384773+0000","last_clean":"2024-09-12T03:16:33.384773+0000","last_became_active":"2024-09-12T03:16:31.581766+0000","last_became_peered":"2024-09-12T03:16:31.581766+0000","last_unstale":"2024-09-12T03:16:33.384773+0000","last_undegraded":"2024-09-12T03:16:33.384773+0000","last_fullsized":"2024-09-12T03:16:33.384773+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:23:22.497643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:33.384404+0000","last_change":"2024-09-12T03:16:31.581788+0000","last_active":"2024-09-12T03:16:33.384404+0000","last_peered":"2024-09-12T03:16:33.384404+0000","last_clean":"2024-09-12T03:16:33.384404+0000","last_became_active":"2024-09-12T03:16:31.581516+0000","last_became_peered":"2024-09-12T03:16:31.581516+0000","last_unstale":"2024-09-12T03:16:33.384404+0000","last_undegraded":"2024-09-12T03:16:33.384404+0000","last_fullsized":"2024-09-12T03:16:33.384404+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:52:47.384994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:32.580659+0000","last_change":"2024-09-12T03:16:31.579564+0000","last_active":"2024-09-12T03:16:32.580659+0000","last_peered":"2024-09-12T03:16:32.580659+0000","last_clean":"2024-09-12T03:16:32.580659+0000","last_became_active":"2024-09-12T03:16:31.579247+0000","last_became_peered":"2024-09-12T03:16:31.579247+0000","last_unstale":"2024-09-12T03:16:32.580659+0000","last_undegraded":"2024-09-12T03:16:32.580659+0000","last_fullsized":"2024-09-12T03:16:32.580659+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:19:43.151205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:33.383395+0000","last_change":"2024-09-12T03:16:31.580809+0000","last_active":"2024-09-12T03:16:33.383395+0000","last_peered":"2024-09-12T03:16:33.383395+0000","last_clean":"2024-09-12T03:16:33.383395+0000","last_became_active":"2024-09-12T03:16:31.580520+0000","last_became_peered":"2024-09-12T03:16:31.580520+0000","last_unstale":"2024-09-12T03:16:33.383395+0000","last_undegraded":"2024-09-12T03:16:33.383395+0000","last_fullsized":"2024-09-12T03:16:33.383395+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:52:47.384994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:33.383780+0000","last_change":"2024-09-12T03:16:31.589461+0000","last_active":"2024-09-12T03:16:33.383780+0000","last_peered":"2024-09-12T03:16:33.383780+0000","last_clean":"2024-09-12T03:16:33.383780+0000","last_became_active":"2024-09-12T03:16:31.587058+0000","last_became_peered":"2024-09-12T03:16:31.587058+0000","last_unstale":"2024-09-12T03:16:33.383780+0000","last_undegraded":"2024-09-12T03:16:33.383780+0000","last_fullsized":"2024-09-12T03:16:33.383780+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:27:08.819051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:32.578055+0000","last_change":"2024-09-12T03:16:31.589724+0000","last_active":"2024-09-12T03:16:32.578055+0000","last_peered":"2024-09-12T03:16:32.578055+0000","last_clean":"2024-09-12T03:16:32.578055+0000","last_became_active":"2024-09-12T03:16:31.589452+0000","last_became_peered":"2024-09-12T03:16:31.589452+0000","last_unstale":"2024-09-12T03:16:32.578055+0000","last_undegraded":"2024-09-12T03:16:32.578055+0000","last_fullsized":"2024-09-12T03:16:32.578055+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:30:37.571167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:32.579421+0000","last_change":"2024-09-12T03:16:31.574474+0000","last_active":"2024-09-12T03:16:32.579421+0000","last_peered":"2024-09-12T03:16:32.579421+0000","last_clean":"2024-09-12T03:16:32.579421+0000","last_became_active":"2024-09-12T03:16:31.574190+0000","last_became_peered":"2024-09-12T03:16:31.574190+0000","last_unstale":"2024-09-12T03:16:32.579421+0000","last_undegraded":"2024-09-12T03:16:32.579421+0000","last_fullsized":"2024-09-12T03:16:32.579421+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:27:20.447453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:32.580359+0000","last_change":"2024-09-12T03:16:31.589878+0000","last_active":"2024-09-12T03:16:32.580359+0000","last_peered":"2024-09-12T03:16:32.580359+0000","last_clean":"2024-09-12T03:16:32.580359+0000","last_became_active":"2024-09-12T03:16:31.589603+0000","last_became_peered":"2024-09-12T03:16:31.589603+0000","last_unstale":"2024-09-12T03:16:32.580359+0000","last_undegraded":"2024-09-12T03:16:32.580359+0000","last_fullsized":"2024-09-12T03:16:32.580359+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:39:25.162423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:33.341837+0000","last_change":"2024-09-12T03:16:31.579526+0000","last_active":"2024-09-12T03:16:33.341837+0000","last_peered":"2024-09-12T03:16:33.341837+0000","last_clean":"2024-09-12T03:16:33.341837+0000","last_became_active":"2024-09-12T03:16:31.579217+0000","last_became_peered":"2024-09-12T03:16:31.579217+0000","last_unstale":"2024-09-12T03:16:33.341837+0000","last_undegraded":"2024-09-12T03:16:33.341837+0000","last_fullsized":"2024-09-12T03:16:33.341837+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:49:09.388955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:33.384367+0000","last_change":"2024-09-12T03:16:31.588216+0000","last_active":"2024-09-12T03:16:33.384367+0000","last_peered":"2024-09-12T03:16:33.384367+0000","last_clean":"2024-09-12T03:16:33.384367+0000","last_became_active":"2024-09-12T03:16:31.587499+0000","last_became_peered":"2024-09-12T03:16:31.587499+0000","last_unstale":"2024-09-12T03:16:33.384367+0000","last_undegraded":"2024-09-12T03:16:33.384367+0000","last_fullsized":"2024-09-12T03:16:33.384367+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:07:30.199995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:33.384562+0000","last_change":"2024-09-12T03:16:31.589828+0000","last_active":"2024-09-12T03:16:33.384562+0000","last_peered":"2024-09-12T03:16:33.384562+0000","last_clean":"2024-09-12T03:16:33.384562+0000","last_became_active":"2024-09-12T03:16:31.589341+0000","last_became_peered":"2024-09-12T03:16:31.589341+0000","last_unstale":"2024-09-12T03:16:33.384562+0000","last_undegraded":"2024-09-12T03:16:33.384562+0000","last_fullsized":"2024-09-12T03:16:33.384562+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:05:13.773162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:33.384136+0000","last_change":"2024-09-12T03:16:31.586744+0000","last_active":"2024-09-12T03:16:33.384136+0000","last_peered":"2024-09-12T03:16:33.384136+0000","last_clean":"2024-09-12T03:16:33.384136+0000","last_became_active":"2024-09-12T03:16:31.586462+0000","last_became_peered":"2024-09-12T03:16:31.586462+0000","last_unstale":"2024-09-12T03:16:33.384136+0000","last_undegraded":"2024-09-12T03:16:33.384136+0000","last_fullsized":"2024-09-12T03:16:33.384136+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:52:47.384994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:32.579721+0000","last_change":"2024-09-12T03:16:31.583270+0000","last_active":"2024-09-12T03:16:32.579721+0000","last_peered":"2024-09-12T03:16:32.579721+0000","last_clean":"2024-09-12T03:16:32.579721+0000","last_became_active":"2024-09-12T03:16:31.582948+0000","last_became_peered":"2024-09-12T03:16:31.582948+0000","last_unstale":"2024-09-12T03:16:32.579721+0000","last_undegraded":"2024-09-12T03:16:32.579721+0000","last_fullsized":"2024-09-12T03:16:32.579721+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:05:55.585173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:33.383081+0000","last_change":"2024-09-12T03:16:31.590127+0000","last_active":"2024-09-12T03:16:33.383081+0000","last_peered":"2024-09-12T03:16:33.383081+0000","last_clean":"2024-09-12T03:16:33.383081+0000","last_became_active":"2024-09-12T03:16:31.589847+0000","last_became_peered":"2024-09-12T03:16:31.589847+0000","last_unstale":"2024-09-12T03:16:33.383081+0000","last_undegraded":"2024-09-12T03:16:33.383081+0000","last_fullsized":"2024-09-12T03:16:33.383081+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:27:08.819051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:32.579620+0000","last_change":"2024-09-12T03:16:31.578711+0000","last_active":"2024-09-12T03:16:32.579620+0000","last_peered":"2024-09-12T03:16:32.579620+0000","last_clean":"2024-09-12T03:16:32.579620+0000","last_became_active":"2024-09-12T03:16:31.578421+0000","last_became_peered":"2024-09-12T03:16:31.578421+0000","last_unstale":"2024-09-12T03:16:32.579620+0000","last_undegraded":"2024-09-12T03:16:32.579620+0000","last_fullsized":"2024-09-12T03:16:32.579620+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:21:32.875776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:33.383880+0000","last_change":"2024-09-12T03:16:31.589412+0000","last_active":"2024-09-12T03:16:33.383880+0000","last_peered":"2024-09-12T03:16:33.383880+0000","last_clean":"2024-09-12T03:16:33.383880+0000","last_became_active":"2024-09-12T03:16:31.589147+0000","last_became_peered":"2024-09-12T03:16:31.589147+0000","last_unstale":"2024-09-12T03:16:33.383880+0000","last_undegraded":"2024-09-12T03:16:33.383880+0000","last_fullsized":"2024-09-12T03:16:33.383880+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:44:02.835393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:33.384825+0000","last_change":"2024-09-12T03:16:31.586179+0000","last_active":"2024-09-12T03:16:33.384825+0000","last_peered":"2024-09-12T03:16:33.384825+0000","last_clean":"2024-09-12T03:16:33.384825+0000","last_became_active":"2024-09-12T03:16:31.585793+0000","last_became_peered":"2024-09-12T03:16:31.585793+0000","last_unstale":"2024-09-12T03:16:33.384825+0000","last_undegraded":"2024-09-12T03:16:33.384825+0000","last_fullsized":"2024-09-12T03:16:33.384825+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:46:08.748138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:32.578142+0000","last_change":"2024-09-12T03:16:31.578743+0000","last_active":"2024-09-12T03:16:32.578142+0000","last_peered":"2024-09-12T03:16:32.578142+0000","last_clean":"2024-09-12T03:16:32.578142+0000","last_became_active":"2024-09-12T03:16:31.578450+0000","last_became_peered":"2024-09-12T03:16:31.578450+0000","last_unstale":"2024-09-12T03:16:32.578142+0000","last_undegraded":"2024-09-12T03:16:32.578142+0000","last_fullsized":"2024-09-12T03:16:32.578142+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:05:01.246614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:33.383793+0000","last_change":"2024-09-12T03:16:31.590186+0000","last_active":"2024-09-12T03:16:33.383793+0000","last_peered":"2024-09-12T03:16:33.383793+0000","last_clean":"2024-09-12T03:16:33.383793+0000","last_became_active":"2024-09-12T03:16:31.589895+0000","last_became_peered":"2024-09-12T03:16:31.589895+0000","last_unstale":"2024-09-12T03:16:33.383793+0000","last_undegraded":"2024-09-12T03:16:33.383793+0000","last_fullsized":"2024-09-12T03:16:33.383793+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:33:01.912717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:33.384305+0000","last_change":"2024-09-12T03:16:31.577171+0000","last_active":"2024-09-12T03:16:33.384305+0000","last_peered":"2024-09-12T03:16:33.384305+0000","last_clean":"2024-09-12T03:16:33.384305+0000","last_became_active":"2024-09-12T03:16:31.576882+0000","last_became_peered":"2024-09-12T03:16:31.576882+0000","last_unstale":"2024-09-12T03:16:33.384305+0000","last_undegraded":"2024-09-12T03:16:33.384305+0000","last_fullsized":"2024-09-12T03:16:33.384305+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:44:02.835393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:32.579941+0000","last_change":"2024-09-12T03:16:31.578670+0000","last_active":"2024-09-12T03:16:32.579941+0000","last_peered":"2024-09-12T03:16:32.579941+0000","last_clean":"2024-09-12T03:16:32.579941+0000","last_became_active":"2024-09-12T03:16:31.578379+0000","last_became_peered":"2024-09-12T03:16:31.578379+0000","last_unstale":"2024-09-12T03:16:32.579941+0000","last_undegraded":"2024-09-12T03:16:32.579941+0000","last_fullsized":"2024-09-12T03:16:32.579941+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:22:01.431497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:33.341761+0000","last_change":"2024-09-12T03:16:31.589511+0000","last_active":"2024-09-12T03:16:33.341761+0000","last_peered":"2024-09-12T03:16:33.341761+0000","last_clean":"2024-09-12T03:16:33.341761+0000","last_became_active":"2024-09-12T03:16:31.589230+0000","last_became_peered":"2024-09-12T03:16:31.589230+0000","last_unstale":"2024-09-12T03:16:33.341761+0000","last_undegraded":"2024-09-12T03:16:33.341761+0000","last_fullsized":"2024-09-12T03:16:33.341761+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:54:17.604800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:33.383613+0000","last_change":"2024-09-12T03:16:31.585042+0000","last_active":"2024-09-12T03:16:33.383613+0000","last_peered":"2024-09-12T03:16:33.383613+0000","last_clean":"2024-09-12T03:16:33.383613+0000","last_became_active":"2024-09-12T03:16:31.584495+0000","last_became_peered":"2024-09-12T03:16:31.584495+0000","last_unstale":"2024-09-12T03:16:33.383613+0000","last_undegraded":"2024-09-12T03:16:33.383613+0000","last_fullsized":"2024-09-12T03:16:33.383613+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:27:31.919800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:32.580182+0000","last_change":"2024-09-12T03:16:31.588040+0000","last_active":"2024-09-12T03:16:32.580182+0000","last_peered":"2024-09-12T03:16:32.580182+0000","last_clean":"2024-09-12T03:16:32.580182+0000","last_became_active":"2024-09-12T03:16:31.587757+0000","last_became_peered":"2024-09-12T03:16:31.587757+0000","last_unstale":"2024-09-12T03:16:32.580182+0000","last_undegraded":"2024-09-12T03:16:32.580182+0000","last_fullsized":"2024-09-12T03:16:32.580182+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:01:36.472991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:33.383460+0000","last_change":"2024-09-12T03:16:30.583423+0000","last_active":"2024-09-12T03:16:33.383460+0000","last_peered":"2024-09-12T03:16:33.383460+0000","last_clean":"2024-09-12T03:16:33.383460+0000","last_became_active":"2024-09-12T03:16:30.583160+0000","last_became_peered":"2024-09-12T03:16:30.583160+0000","last_unstale":"2024-09-12T03:16:33.383460+0000","last_undegraded":"2024-09-12T03:16:33.383460+0000","last_fullsized":"2024-09-12T03:16:33.383460+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:19:47.998143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:33.384360+0000","last_change":"2024-09-12T03:16:31.591565+0000","last_active":"2024-09-12T03:16:33.384360+0000","last_peered":"2024-09-12T03:16:33.384360+0000","last_clean":"2024-09-12T03:16:33.384360+0000","last_became_active":"2024-09-12T03:16:31.591236+0000","last_became_peered":"2024-09-12T03:16:31.591236+0000","last_unstale":"2024-09-12T03:16:33.384360+0000","last_undegraded":"2024-09-12T03:16:33.384360+0000","last_fullsized":"2024-09-12T03:16:33.384360+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:35:22.374781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:33.383956+0000","last_change":"2024-09-12T03:16:30.575856+0000","last_active":"2024-09-12T03:16:33.383956+0000","last_peered":"2024-09-12T03:16:33.383956+0000","last_clean":"2024-09-12T03:16:33.383956+0000","last_became_active":"2024-09-12T03:16:30.575512+0000","last_became_peered":"2024-09-12T03:16:30.575512+0000","last_unstale":"2024-09-12T03:16:33.383956+0000","last_undegraded":"2024-09-12T03:16:33.383956+0000","last_fullsized":"2024-09-12T03:16:33.383956+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:13:33.665229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:33.383136+0000","last_change":"2024-09-12T03:16:30.580131+0000","last_active":"2024-09-12T03:16:33.383136+0000","last_peered":"2024-09-12T03:16:33.383136+0000","last_clean":"2024-09-12T03:16:33.383136+0000","last_became_active":"2024-09-12T03:16:30.579760+0000","last_became_peered":"2024-09-12T03:16:30.579760+0000","last_unstale":"2024-09-12T03:16:33.383136+0000","last_undegraded":"2024-09-12T03:16:33.383136+0000","last_fullsized":"2024-09-12T03:16:33.383136+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:29:14.889397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:33.383040+0000","last_change":"2024-09-12T03:16:31.581185+0000","last_active":"2024-09-12T03:16:33.383040+0000","last_peered":"2024-09-12T03:16:33.383040+0000","last_clean":"2024-09-12T03:16:33.383040+0000","last_became_active":"2024-09-12T03:16:31.580425+0000","last_became_peered":"2024-09-12T03:16:31.580425+0000","last_unstale":"2024-09-12T03:16:33.383040+0000","last_undegraded":"2024-09-12T03:16:33.383040+0000","last_fullsized":"2024-09-12T03:16:33.383040+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:12:47.905772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:32.579685+0000","last_change":"2024-09-12T03:16:30.571998+0000","last_active":"2024-09-12T03:16:32.579685+0000","last_peered":"2024-09-12T03:16:32.579685+0000","last_clean":"2024-09-12T03:16:32.579685+0000","last_became_active":"2024-09-12T03:16:30.571673+0000","last_became_peered":"2024-09-12T03:16:30.571673+0000","last_unstale":"2024-09-12T03:16:32.579685+0000","last_undegraded":"2024-09-12T03:16:32.579685+0000","last_fullsized":"2024-09-12T03:16:32.579685+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:19:17.746405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:32.578040+0000","last_change":"2024-09-12T03:16:31.587700+0000","last_active":"2024-09-12T03:16:32.578040+0000","last_peered":"2024-09-12T03:16:32.578040+0000","last_clean":"2024-09-12T03:16:32.578040+0000","last_became_active":"2024-09-12T03:16:31.587296+0000","last_became_peered":"2024-09-12T03:16:31.587296+0000","last_unstale":"2024-09-12T03:16:32.578040+0000","last_undegraded":"2024-09-12T03:16:32.578040+0000","last_fullsized":"2024-09-12T03:16:32.578040+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:28:08.918294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:33.384131+0000","last_change":"2024-09-12T03:16:30.581517+0000","last_active":"2024-09-12T03:16:33.384131+0000","last_peered":"2024-09-12T03:16:33.384131+0000","last_clean":"2024-09-12T03:16:33.384131+0000","last_became_active":"2024-09-12T03:16:30.581107+0000","last_became_peered":"2024-09-12T03:16:30.581107+0000","last_unstale":"2024-09-12T03:16:33.384131+0000","last_undegraded":"2024-09-12T03:16:33.384131+0000","last_fullsized":"2024-09-12T03:16:33.384131+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:01:35.467902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:32.580094+0000","last_change":"2024-09-12T03:16:31.588465+0000","last_active":"2024-09-12T03:16:32.580094+0000","last_peered":"2024-09-12T03:16:32.580094+0000","last_clean":"2024-09-12T03:16:32.580094+0000","last_became_active":"2024-09-12T03:16:31.588150+0000","last_became_peered":"2024-09-12T03:16:31.588150+0000","last_unstale":"2024-09-12T03:16:32.580094+0000","last_undegraded":"2024-09-12T03:16:32.580094+0000","last_fullsized":"2024-09-12T03:16:32.580094+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:04:42.975000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:32.580148+0000","last_change":"2024-09-12T03:16:30.578195+0000","last_active":"2024-09-12T03:16:32.580148+0000","last_peered":"2024-09-12T03:16:32.580148+0000","last_clean":"2024-09-12T03:16:32.580148+0000","last_became_active":"2024-09-12T03:16:30.577878+0000","last_became_peered":"2024-09-12T03:16:30.577878+0000","last_unstale":"2024-09-12T03:16:32.580148+0000","last_undegraded":"2024-09-12T03:16:32.580148+0000","last_fullsized":"2024-09-12T03:16:32.580148+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:19:07.178848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:33.384177+0000","last_change":"2024-09-12T03:16:31.584084+0000","last_active":"2024-09-12T03:16:33.384177+0000","last_peered":"2024-09-12T03:16:33.384177+0000","last_clean":"2024-09-12T03:16:33.384177+0000","last_became_active":"2024-09-12T03:16:31.583539+0000","last_became_peered":"2024-09-12T03:16:31.583539+0000","last_unstale":"2024-09-12T03:16:33.384177+0000","last_undegraded":"2024-09-12T03:16:33.384177+0000","last_fullsized":"2024-09-12T03:16:33.384177+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:23:22.497643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:32.578203+0000","last_change":"2024-09-12T03:16:30.572959+0000","last_active":"2024-09-12T03:16:32.578203+0000","last_peered":"2024-09-12T03:16:32.578203+0000","last_clean":"2024-09-12T03:16:32.578203+0000","last_became_active":"2024-09-12T03:16:30.572692+0000","last_became_peered":"2024-09-12T03:16:30.572692+0000","last_unstale":"2024-09-12T03:16:32.578203+0000","last_undegraded":"2024-09-12T03:16:32.578203+0000","last_fullsized":"2024-09-12T03:16:32.578203+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:21:08.051836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:32.580840+0000","last_change":"2024-09-12T03:16:31.590092+0000","last_active":"2024-09-12T03:16:32.580840+0000","last_peered":"2024-09-12T03:16:32.580840+0000","last_clean":"2024-09-12T03:16:32.580840+0000","last_became_active":"2024-09-12T03:16:31.589831+0000","last_became_peered":"2024-09-12T03:16:31.589831+0000","last_unstale":"2024-09-12T03:16:32.580840+0000","last_undegraded":"2024-09-12T03:16:32.580840+0000","last_fullsized":"2024-09-12T03:16:32.580840+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:54:17.604800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"20'6","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:33.384460+0000","last_change":"2024-09-12T03:16:30.575948+0000","last_active":"2024-09-12T03:16:33.384460+0000","last_peered":"2024-09-12T03:16:33.384460+0000","last_clean":"2024-09-12T03:16:33.384460+0000","last_became_active":"2024-09-12T03:16:30.575651+0000","last_became_peered":"2024-09-12T03:16:30.575651+0000","last_unstale":"2024-09-12T03:16:33.384460+0000","last_undegraded":"2024-09-12T03:16:33.384460+0000","last_fullsized":"2024-09-12T03:16:33.384460+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:44:10.172361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:33.383628+0000","last_change":"2024-09-12T03:16:31.587604+0000","last_active":"2024-09-12T03:16:33.383628+0000","last_peered":"2024-09-12T03:16:33.383628+0000","last_clean":"2024-09-12T03:16:33.383628+0000","last_became_active":"2024-09-12T03:16:31.581590+0000","last_became_peered":"2024-09-12T03:16:31.581590+0000","last_unstale":"2024-09-12T03:16:33.383628+0000","last_undegraded":"2024-09-12T03:16:33.383628+0000","last_fullsized":"2024-09-12T03:16:33.383628+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:40:23.693285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:33.384342+0000","last_change":"2024-09-12T03:16:30.580151+0000","last_active":"2024-09-12T03:16:33.384342+0000","last_peered":"2024-09-12T03:16:33.384342+0000","last_clean":"2024-09-12T03:16:33.384342+0000","last_became_active":"2024-09-12T03:16:30.579859+0000","last_became_peered":"2024-09-12T03:16:30.579859+0000","last_unstale":"2024-09-12T03:16:33.384342+0000","last_undegraded":"2024-09-12T03:16:33.384342+0000","last_fullsized":"2024-09-12T03:16:33.384342+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:44:10.172361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:33.383933+0000","last_change":"2024-09-12T03:16:31.580748+0000","last_active":"2024-09-12T03:16:33.383933+0000","last_peered":"2024-09-12T03:16:33.383933+0000","last_clean":"2024-09-12T03:16:33.383933+0000","last_became_active":"2024-09-12T03:16:31.580434+0000","last_became_peered":"2024-09-12T03:16:31.580434+0000","last_unstale":"2024-09-12T03:16:33.383933+0000","last_undegraded":"2024-09-12T03:16:33.383933+0000","last_fullsized":"2024-09-12T03:16:33.383933+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:27:31.919800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:32.580017+0000","last_change":"2024-09-12T03:16:30.570604+0000","last_active":"2024-09-12T03:16:32.580017+0000","last_peered":"2024-09-12T03:16:32.580017+0000","last_clean":"2024-09-12T03:16:32.580017+0000","last_became_active":"2024-09-12T03:16:30.570227+0000","last_became_peered":"2024-09-12T03:16:30.570227+0000","last_unstale":"2024-09-12T03:16:32.580017+0000","last_undegraded":"2024-09-12T03:16:32.580017+0000","last_fullsized":"2024-09-12T03:16:32.580017+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:55:03.313892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:33.383958+0000","last_change":"2024-09-12T03:16:31.592248+0000","last_active":"2024-09-12T03:16:33.383958+0000","last_peered":"2024-09-12T03:16:33.383958+0000","last_clean":"2024-09-12T03:16:33.383958+0000","last_became_active":"2024-09-12T03:16:31.591836+0000","last_became_peered":"2024-09-12T03:16:31.591836+0000","last_unstale":"2024-09-12T03:16:33.383958+0000","last_undegraded":"2024-09-12T03:16:33.383958+0000","last_fullsized":"2024-09-12T03:16:33.383958+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:05:17.785019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:33.383324+0000","last_change":"2024-09-12T03:16:30.578648+0000","last_active":"2024-09-12T03:16:33.383324+0000","last_peered":"2024-09-12T03:16:33.383324+0000","last_clean":"2024-09-12T03:16:33.383324+0000","last_became_active":"2024-09-12T03:16:30.578343+0000","last_became_peered":"2024-09-12T03:16:30.578343+0000","last_unstale":"2024-09-12T03:16:33.383324+0000","last_undegraded":"2024-09-12T03:16:33.383324+0000","last_fullsized":"2024-09-12T03:16:33.383324+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:29:14.889397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:33.383440+0000","last_change":"2024-09-12T03:16:31.584802+0000","last_active":"2024-09-12T03:16:33.383440+0000","last_peered":"2024-09-12T03:16:33.383440+0000","last_clean":"2024-09-12T03:16:33.383440+0000","last_became_active":"2024-09-12T03:16:31.584124+0000","last_became_peered":"2024-09-12T03:16:31.584124+0000","last_unstale":"2024-09-12T03:16:33.383440+0000","last_undegraded":"2024-09-12T03:16:33.383440+0000","last_fullsized":"2024-09-12T03:16:33.383440+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:27:08.819051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":22,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:33.341548+0000","last_change":"2024-09-12T03:16:30.574668+0000","last_active":"2024-09-12T03:16:33.341548+0000","last_peered":"2024-09-12T03:16:33.341548+0000","last_clean":"2024-09-12T03:16:33.341548+0000","last_became_active":"2024-09-12T03:16:30.574373+0000","last_became_peered":"2024-09-12T03:16:30.574373+0000","last_unstale":"2024-09-12T03:16:33.341548+0000","last_undegraded":"2024-09-12T03:16:33.341548+0000","last_fullsized":"2024-09-12T03:16:33.341548+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:19:47.998143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:33.383657+0000","last_change":"2024-09-12T03:16:31.592520+0000","last_active":"2024-09-12T03:16:33.383657+0000","last_peered":"2024-09-12T03:16:33.383657+0000","last_clean":"2024-09-12T03:16:33.383657+0000","last_became_active":"2024-09-12T03:16:31.591766+0000","last_became_peered":"2024-09-12T03:16:31.591766+0000","last_unstale":"2024-09-12T03:16:33.383657+0000","last_undegraded":"2024-09-12T03:16:33.383657+0000","last_fullsized":"2024-09-12T03:16:33.383657+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:22:52.960561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:32.579754+0000","last_change":"2024-09-12T03:16:30.582087+0000","last_active":"2024-09-12T03:16:32.579754+0000","last_peered":"2024-09-12T03:16:32.579754+0000","last_clean":"2024-09-12T03:16:32.579754+0000","last_became_active":"2024-09-12T03:16:30.581622+0000","last_became_peered":"2024-09-12T03:16:30.581622+0000","last_unstale":"2024-09-12T03:16:32.579754+0000","last_undegraded":"2024-09-12T03:16:32.579754+0000","last_fullsized":"2024-09-12T03:16:32.579754+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:58:54.372496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:33.384324+0000","last_change":"2024-09-12T03:16:31.591374+0000","last_active":"2024-09-12T03:16:33.384324+0000","last_peered":"2024-09-12T03:16:33.384324+0000","last_clean":"2024-09-12T03:16:33.384324+0000","last_became_active":"2024-09-12T03:16:31.591092+0000","last_became_peered":"2024-09-12T03:16:31.591092+0000","last_unstale":"2024-09-12T03:16:33.384324+0000","last_undegraded":"2024-09-12T03:16:33.384324+0000","last_fullsized":"2024-09-12T03:16:33.384324+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:08:36.235105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:33.341748+0000","last_change":"2024-09-12T03:16:30.570924+0000","last_active":"2024-09-12T03:16:33.341748+0000","last_peered":"2024-09-12T03:16:33.341748+0000","last_clean":"2024-09-12T03:16:33.341748+0000","last_became_active":"2024-09-12T03:16:30.570636+0000","last_became_peered":"2024-09-12T03:16:30.570636+0000","last_unstale":"2024-09-12T03:16:33.341748+0000","last_undegraded":"2024-09-12T03:16:33.341748+0000","last_fullsized":"2024-09-12T03:16:33.341748+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:29:14.889397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:33.384281+0000","last_change":"2024-09-12T03:16:31.592777+0000","last_active":"2024-09-12T03:16:33.384281+0000","last_peered":"2024-09-12T03:16:33.384281+0000","last_clean":"2024-09-12T03:16:33.384281+0000","last_became_active":"2024-09-12T03:16:31.591855+0000","last_became_peered":"2024-09-12T03:16:31.591855+0000","last_unstale":"2024-09-12T03:16:33.384281+0000","last_undegraded":"2024-09-12T03:16:33.384281+0000","last_fullsized":"2024-09-12T03:16:33.384281+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:39:49.575672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:32.578281+0000","last_change":"2024-09-12T03:16:30.574066+0000","last_active":"2024-09-12T03:16:32.578281+0000","last_peered":"2024-09-12T03:16:32.578281+0000","last_clean":"2024-09-12T03:16:32.578281+0000","last_became_active":"2024-09-12T03:16:30.573756+0000","last_became_peered":"2024-09-12T03:16:30.573756+0000","last_unstale":"2024-09-12T03:16:32.578281+0000","last_undegraded":"2024-09-12T03:16:32.578281+0000","last_fullsized":"2024-09-12T03:16:32.578281+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:19:17.746405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:32.580747+0000","last_change":"2024-09-12T03:16:31.583553+0000","last_active":"2024-09-12T03:16:32.580747+0000","last_peered":"2024-09-12T03:16:32.580747+0000","last_clean":"2024-09-12T03:16:32.580747+0000","last_became_active":"2024-09-12T03:16:31.582827+0000","last_became_peered":"2024-09-12T03:16:31.582827+0000","last_unstale":"2024-09-12T03:16:32.580747+0000","last_undegraded":"2024-09-12T03:16:32.580747+0000","last_fullsized":"2024-09-12T03:16:32.580747+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:54:20.264625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:33.341655+0000","last_change":"2024-09-12T03:16:30.573483+0000","last_active":"2024-09-12T03:16:33.341655+0000","last_peered":"2024-09-12T03:16:33.341655+0000","last_clean":"2024-09-12T03:16:33.341655+0000","last_became_active":"2024-09-12T03:16:30.573180+0000","last_became_peered":"2024-09-12T03:16:30.573180+0000","last_unstale":"2024-09-12T03:16:33.341655+0000","last_undegraded":"2024-09-12T03:16:33.341655+0000","last_fullsized":"2024-09-12T03:16:33.341655+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:13:33.665229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:33.383685+0000","last_change":"2024-09-12T03:16:31.590008+0000","last_active":"2024-09-12T03:16:33.383685+0000","last_peered":"2024-09-12T03:16:33.383685+0000","last_clean":"2024-09-12T03:16:33.383685+0000","last_became_active":"2024-09-12T03:16:31.589738+0000","last_became_peered":"2024-09-12T03:16:31.589738+0000","last_unstale":"2024-09-12T03:16:33.383685+0000","last_undegraded":"2024-09-12T03:16:33.383685+0000","last_fullsized":"2024-09-12T03:16:33.383685+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:31:17.482655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":170,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:33.384272+0000","last_change":"2024-09-12T03:16:21.883446+0000","last_active":"2024-09-12T03:16:33.384272+0000","last_peered":"2024-09-12T03:16:33.384272+0000","last_clean":"2024-09-12T03:16:33.384272+0000","last_became_active":"2024-09-12T03:16:21.882207+0000","last_became_peered":"2024-09-12T03:16:21.882207+0000","last_unstale":"2024-09-12T03:16:33.384272+0000","last_undegraded":"2024-09-12T03:16:33.384272+0000","last_fullsized":"2024-09-12T03:16:33.384272+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:20.316426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:20.316426+0000","last_clean_scrub_stamp":"2024-09-12T03:16:20.316426+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:12:15.424320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:32.579477+0000","last_change":"2024-09-12T03:16:30.573340+0000","last_active":"2024-09-12T03:16:32.579477+0000","last_peered":"2024-09-12T03:16:32.579477+0000","last_clean":"2024-09-12T03:16:32.579477+0000","last_became_active":"2024-09-12T03:16:30.572756+0000","last_became_peered":"2024-09-12T03:16:30.572756+0000","last_unstale":"2024-09-12T03:16:32.579477+0000","last_undegraded":"2024-09-12T03:16:32.579477+0000","last_fullsized":"2024-09-12T03:16:32.579477+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:21:08.051836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:33.383593+0000","last_change":"2024-09-12T03:16:31.592850+0000","last_active":"2024-09-12T03:16:33.383593+0000","last_peered":"2024-09-12T03:16:33.383593+0000","last_clean":"2024-09-12T03:16:33.383593+0000","last_became_active":"2024-09-12T03:16:31.592505+0000","last_became_peered":"2024-09-12T03:16:31.592505+0000","last_unstale":"2024-09-12T03:16:33.383593+0000","last_undegraded":"2024-09-12T03:16:33.383593+0000","last_fullsized":"2024-09-12T03:16:33.383593+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:23:22.497643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:33.384154+0000","last_change":"2024-09-12T03:16:30.584842+0000","last_active":"2024-09-12T03:16:33.384154+0000","last_peered":"2024-09-12T03:16:33.384154+0000","last_clean":"2024-09-12T03:16:33.384154+0000","last_became_active":"2024-09-12T03:16:30.584516+0000","last_became_peered":"2024-09-12T03:16:30.584516+0000","last_unstale":"2024-09-12T03:16:33.384154+0000","last_undegraded":"2024-09-12T03:16:33.384154+0000","last_fullsized":"2024-09-12T03:16:33.384154+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:44:15.389117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:33.383582+0000","last_change":"2024-09-12T03:16:31.570586+0000","last_active":"2024-09-12T03:16:33.383582+0000","last_peered":"2024-09-12T03:16:33.383582+0000","last_clean":"2024-09-12T03:16:33.383582+0000","last_became_active":"2024-09-12T03:16:31.570210+0000","last_became_peered":"2024-09-12T03:16:31.570210+0000","last_unstale":"2024-09-12T03:16:33.383582+0000","last_undegraded":"2024-09-12T03:16:33.383582+0000","last_fullsized":"2024-09-12T03:16:33.383582+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:12:47.905772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:33.341721+0000","last_change":"2024-09-12T03:16:30.569374+0000","last_active":"2024-09-12T03:16:33.341721+0000","last_peered":"2024-09-12T03:16:33.341721+0000","last_clean":"2024-09-12T03:16:33.341721+0000","last_became_active":"2024-09-12T03:16:30.569053+0000","last_became_peered":"2024-09-12T03:16:30.569053+0000","last_unstale":"2024-09-12T03:16:33.341721+0000","last_undegraded":"2024-09-12T03:16:33.341721+0000","last_fullsized":"2024-09-12T03:16:33.341721+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:44:10.172361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:32.578285+0000","last_change":"2024-09-12T03:16:31.587841+0000","last_active":"2024-09-12T03:16:32.578285+0000","last_peered":"2024-09-12T03:16:32.578285+0000","last_clean":"2024-09-12T03:16:32.578285+0000","last_became_active":"2024-09-12T03:16:31.587555+0000","last_became_peered":"2024-09-12T03:16:31.587555+0000","last_unstale":"2024-09-12T03:16:32.578285+0000","last_undegraded":"2024-09-12T03:16:32.578285+0000","last_fullsized":"2024-09-12T03:16:32.578285+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:02:33.249591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:32.580124+0000","last_change":"2024-09-12T03:16:30.576697+0000","last_active":"2024-09-12T03:16:32.580124+0000","last_peered":"2024-09-12T03:16:32.580124+0000","last_clean":"2024-09-12T03:16:32.580124+0000","last_became_active":"2024-09-12T03:16:30.574772+0000","last_became_peered":"2024-09-12T03:16:30.574772+0000","last_unstale":"2024-09-12T03:16:32.580124+0000","last_undegraded":"2024-09-12T03:16:32.580124+0000","last_fullsized":"2024-09-12T03:16:32.580124+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:36:06.512797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:32.580280+0000","last_change":"2024-09-12T03:16:31.581253+0000","last_active":"2024-09-12T03:16:32.580280+0000","last_peered":"2024-09-12T03:16:32.580280+0000","last_clean":"2024-09-12T03:16:32.580280+0000","last_became_active":"2024-09-12T03:16:31.580992+0000","last_became_peered":"2024-09-12T03:16:31.580992+0000","last_unstale":"2024-09-12T03:16:32.580280+0000","last_undegraded":"2024-09-12T03:16:32.580280+0000","last_fullsized":"2024-09-12T03:16:32.580280+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:23:14.503630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:32.578229+0000","last_change":"2024-09-12T03:16:30.580488+0000","last_active":"2024-09-12T03:16:32.578229+0000","last_peered":"2024-09-12T03:16:32.578229+0000","last_clean":"2024-09-12T03:16:32.578229+0000","last_became_active":"2024-09-12T03:16:30.580231+0000","last_became_peered":"2024-09-12T03:16:30.580231+0000","last_unstale":"2024-09-12T03:16:32.578229+0000","last_undegraded":"2024-09-12T03:16:32.578229+0000","last_fullsized":"2024-09-12T03:16:32.578229+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:55:03.313892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:32.580115+0000","last_change":"2024-09-12T03:16:31.582097+0000","last_active":"2024-09-12T03:16:32.580115+0000","last_peered":"2024-09-12T03:16:32.580115+0000","last_clean":"2024-09-12T03:16:32.580115+0000","last_became_active":"2024-09-12T03:16:31.580258+0000","last_became_peered":"2024-09-12T03:16:31.580258+0000","last_unstale":"2024-09-12T03:16:32.580115+0000","last_undegraded":"2024-09-12T03:16:32.580115+0000","last_fullsized":"2024-09-12T03:16:32.580115+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:13:00.761714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:33.384050+0000","last_change":"2024-09-12T03:16:30.586287+0000","last_active":"2024-09-12T03:16:33.384050+0000","last_peered":"2024-09-12T03:16:33.384050+0000","last_clean":"2024-09-12T03:16:33.384050+0000","last_became_active":"2024-09-12T03:16:30.585919+0000","last_became_peered":"2024-09-12T03:16:30.585919+0000","last_unstale":"2024-09-12T03:16:33.384050+0000","last_undegraded":"2024-09-12T03:16:33.384050+0000","last_fullsized":"2024-09-12T03:16:33.384050+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:04:51.773994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:32.578331+0000","last_change":"2024-09-12T03:16:31.587128+0000","last_active":"2024-09-12T03:16:32.578331+0000","last_peered":"2024-09-12T03:16:32.578331+0000","last_clean":"2024-09-12T03:16:32.578331+0000","last_became_active":"2024-09-12T03:16:31.586831+0000","last_became_peered":"2024-09-12T03:16:31.586831+0000","last_unstale":"2024-09-12T03:16:32.578331+0000","last_undegraded":"2024-09-12T03:16:32.578331+0000","last_fullsized":"2024-09-12T03:16:32.578331+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:59:21.255153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:32.578379+0000","last_change":"2024-09-12T03:16:30.577847+0000","last_active":"2024-09-12T03:16:32.578379+0000","last_peered":"2024-09-12T03:16:32.578379+0000","last_clean":"2024-09-12T03:16:32.578379+0000","last_became_active":"2024-09-12T03:16:30.577552+0000","last_became_peered":"2024-09-12T03:16:30.577552+0000","last_unstale":"2024-09-12T03:16:32.578379+0000","last_undegraded":"2024-09-12T03:16:32.578379+0000","last_fullsized":"2024-09-12T03:16:32.578379+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:36:06.512797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:33.383991+0000","last_change":"2024-09-12T03:16:31.586339+0000","last_active":"2024-09-12T03:16:33.383991+0000","last_peered":"2024-09-12T03:16:33.383991+0000","last_clean":"2024-09-12T03:16:33.383991+0000","last_became_active":"2024-09-12T03:16:31.586009+0000","last_became_peered":"2024-09-12T03:16:31.586009+0000","last_unstale":"2024-09-12T03:16:33.383991+0000","last_undegraded":"2024-09-12T03:16:33.383991+0000","last_fullsized":"2024-09-12T03:16:33.383991+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:35:22.374781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:33.384683+0000","last_change":"2024-09-12T03:16:30.583723+0000","last_active":"2024-09-12T03:16:33.384683+0000","last_peered":"2024-09-12T03:16:33.384683+0000","last_clean":"2024-09-12T03:16:33.384683+0000","last_became_active":"2024-09-12T03:16:30.582911+0000","last_became_peered":"2024-09-12T03:16:30.582911+0000","last_unstale":"2024-09-12T03:16:33.384683+0000","last_undegraded":"2024-09-12T03:16:33.384683+0000","last_fullsized":"2024-09-12T03:16:33.384683+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:23:13.498541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:33.384195+0000","last_change":"2024-09-12T03:16:31.589322+0000","last_active":"2024-09-12T03:16:33.384195+0000","last_peered":"2024-09-12T03:16:33.384195+0000","last_clean":"2024-09-12T03:16:33.384195+0000","last_became_active":"2024-09-12T03:16:31.588464+0000","last_became_peered":"2024-09-12T03:16:31.588464+0000","last_unstale":"2024-09-12T03:16:33.384195+0000","last_undegraded":"2024-09-12T03:16:33.384195+0000","last_fullsized":"2024-09-12T03:16:33.384195+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:46:08.748138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:33.383860+0000","last_change":"2024-09-12T03:16:30.583438+0000","last_active":"2024-09-12T03:16:33.383860+0000","last_peered":"2024-09-12T03:16:33.383860+0000","last_clean":"2024-09-12T03:16:33.383860+0000","last_became_active":"2024-09-12T03:16:30.583067+0000","last_became_peered":"2024-09-12T03:16:30.583067+0000","last_unstale":"2024-09-12T03:16:33.383860+0000","last_undegraded":"2024-09-12T03:16:33.383860+0000","last_fullsized":"2024-09-12T03:16:33.383860+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:54:16.599711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:32.581034+0000","last_change":"2024-09-12T03:16:31.578365+0000","last_active":"2024-09-12T03:16:32.581034+0000","last_peered":"2024-09-12T03:16:32.581034+0000","last_clean":"2024-09-12T03:16:32.581034+0000","last_became_active":"2024-09-12T03:16:31.577424+0000","last_became_peered":"2024-09-12T03:16:31.577424+0000","last_unstale":"2024-09-12T03:16:32.581034+0000","last_undegraded":"2024-09-12T03:16:32.581034+0000","last_fullsized":"2024-09-12T03:16:32.581034+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:49:09.388955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:32.578103+0000","last_change":"2024-09-12T03:16:30.574028+0000","last_active":"2024-09-12T03:16:32.578103+0000","last_peered":"2024-09-12T03:16:32.578103+0000","last_clean":"2024-09-12T03:16:32.578103+0000","last_became_active":"2024-09-12T03:16:30.573742+0000","last_became_peered":"2024-09-12T03:16:30.573742+0000","last_unstale":"2024-09-12T03:16:32.578103+0000","last_undegraded":"2024-09-12T03:16:32.578103+0000","last_fullsized":"2024-09-12T03:16:32.578103+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:19:07.178848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:33.383373+0000","last_change":"2024-09-12T03:16:31.584222+0000","last_active":"2024-09-12T03:16:33.383373+0000","last_peered":"2024-09-12T03:16:33.383373+0000","last_clean":"2024-09-12T03:16:33.383373+0000","last_became_active":"2024-09-12T03:16:31.583947+0000","last_became_peered":"2024-09-12T03:16:31.583947+0000","last_unstale":"2024-09-12T03:16:33.383373+0000","last_undegraded":"2024-09-12T03:16:33.383373+0000","last_fullsized":"2024-09-12T03:16:33.383373+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:27:31.919800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:32.578127+0000","last_change":"2024-09-12T03:16:30.572123+0000","last_active":"2024-09-12T03:16:32.578127+0000","last_peered":"2024-09-12T03:16:32.578127+0000","last_clean":"2024-09-12T03:16:32.578127+0000","last_became_active":"2024-09-12T03:16:30.571600+0000","last_became_peered":"2024-09-12T03:16:30.571600+0000","last_unstale":"2024-09-12T03:16:32.578127+0000","last_undegraded":"2024-09-12T03:16:32.578127+0000","last_fullsized":"2024-09-12T03:16:32.578127+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:25:21.686984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:33.383482+0000","last_change":"2024-09-12T03:16:31.581216+0000","last_active":"2024-09-12T03:16:33.383482+0000","last_peered":"2024-09-12T03:16:33.383482+0000","last_clean":"2024-09-12T03:16:33.383482+0000","last_became_active":"2024-09-12T03:16:31.580663+0000","last_became_peered":"2024-09-12T03:16:31.580663+0000","last_unstale":"2024-09-12T03:16:33.383482+0000","last_undegraded":"2024-09-12T03:16:33.383482+0000","last_fullsized":"2024-09-12T03:16:33.383482+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:05:13.773162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:33.384021+0000","last_change":"2024-09-12T03:16:30.584019+0000","last_active":"2024-09-12T03:16:33.384021+0000","last_peered":"2024-09-12T03:16:33.384021+0000","last_clean":"2024-09-12T03:16:33.384021+0000","last_became_active":"2024-09-12T03:16:30.583213+0000","last_became_peered":"2024-09-12T03:16:30.583213+0000","last_unstale":"2024-09-12T03:16:33.384021+0000","last_undegraded":"2024-09-12T03:16:33.384021+0000","last_fullsized":"2024-09-12T03:16:33.384021+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:14:33.816306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:33.383420+0000","last_change":"2024-09-12T03:16:31.590094+0000","last_active":"2024-09-12T03:16:33.383420+0000","last_peered":"2024-09-12T03:16:33.383420+0000","last_clean":"2024-09-12T03:16:33.383420+0000","last_became_active":"2024-09-12T03:16:31.589818+0000","last_became_peered":"2024-09-12T03:16:31.589818+0000","last_unstale":"2024-09-12T03:16:33.383420+0000","last_undegraded":"2024-09-12T03:16:33.383420+0000","last_fullsized":"2024-09-12T03:16:33.383420+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:12:47.905772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:32.580177+0000","last_change":"2024-09-12T03:16:31.585182+0000","last_active":"2024-09-12T03:16:32.580177+0000","last_peered":"2024-09-12T03:16:32.580177+0000","last_clean":"2024-09-12T03:16:32.580177+0000","last_became_active":"2024-09-12T03:16:31.584871+0000","last_became_peered":"2024-09-12T03:16:31.584871+0000","last_unstale":"2024-09-12T03:16:32.580177+0000","last_undegraded":"2024-09-12T03:16:32.580177+0000","last_fullsized":"2024-09-12T03:16:32.580177+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:48:52.581642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:33.383721+0000","last_change":"2024-09-12T03:16:30.583526+0000","last_active":"2024-09-12T03:16:33.383721+0000","last_peered":"2024-09-12T03:16:33.383721+0000","last_clean":"2024-09-12T03:16:33.383721+0000","last_became_active":"2024-09-12T03:16:30.583166+0000","last_became_peered":"2024-09-12T03:16:30.583166+0000","last_unstale":"2024-09-12T03:16:33.383721+0000","last_undegraded":"2024-09-12T03:16:33.383721+0000","last_fullsized":"2024-09-12T03:16:33.383721+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:19:42.146116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:32.580666+0000","last_change":"2024-09-12T03:16:31.585324+0000","last_active":"2024-09-12T03:16:32.580666+0000","last_peered":"2024-09-12T03:16:32.580666+0000","last_clean":"2024-09-12T03:16:32.580666+0000","last_became_active":"2024-09-12T03:16:31.584963+0000","last_became_peered":"2024-09-12T03:16:31.584963+0000","last_unstale":"2024-09-12T03:16:32.580666+0000","last_undegraded":"2024-09-12T03:16:32.580666+0000","last_fullsized":"2024-09-12T03:16:32.580666+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:34:35.319063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:33.383711+0000","last_change":"2024-09-12T03:16:30.583739+0000","last_active":"2024-09-12T03:16:33.383711+0000","last_peered":"2024-09-12T03:16:33.383711+0000","last_clean":"2024-09-12T03:16:33.383711+0000","last_became_active":"2024-09-12T03:16:30.583397+0000","last_became_peered":"2024-09-12T03:16:30.583397+0000","last_unstale":"2024-09-12T03:16:33.383711+0000","last_undegraded":"2024-09-12T03:16:33.383711+0000","last_fullsized":"2024-09-12T03:16:33.383711+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:49:08.383866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:33.384172+0000","last_change":"2024-09-12T03:16:31.583861+0000","last_active":"2024-09-12T03:16:33.384172+0000","last_peered":"2024-09-12T03:16:33.384172+0000","last_clean":"2024-09-12T03:16:33.384172+0000","last_became_active":"2024-09-12T03:16:31.583134+0000","last_became_peered":"2024-09-12T03:16:31.583134+0000","last_unstale":"2024-09-12T03:16:33.384172+0000","last_undegraded":"2024-09-12T03:16:33.384172+0000","last_fullsized":"2024-09-12T03:16:33.384172+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:05:13.773162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:33.383731+0000","last_change":"2024-09-12T03:16:30.580413+0000","last_active":"2024-09-12T03:16:33.383731+0000","last_peered":"2024-09-12T03:16:33.383731+0000","last_clean":"2024-09-12T03:16:33.383731+0000","last_became_active":"2024-09-12T03:16:30.580133+0000","last_became_peered":"2024-09-12T03:16:30.580133+0000","last_unstale":"2024-09-12T03:16:33.383731+0000","last_undegraded":"2024-09-12T03:16:33.383731+0000","last_fullsized":"2024-09-12T03:16:33.383731+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:12:12.227072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:33.383784+0000","last_change":"2024-09-12T03:16:31.580830+0000","last_active":"2024-09-12T03:16:33.383784+0000","last_peered":"2024-09-12T03:16:33.383784+0000","last_clean":"2024-09-12T03:16:33.383784+0000","last_became_active":"2024-09-12T03:16:31.580548+0000","last_became_peered":"2024-09-12T03:16:31.580548+0000","last_unstale":"2024-09-12T03:16:33.383784+0000","last_undegraded":"2024-09-12T03:16:33.383784+0000","last_fullsized":"2024-09-12T03:16:33.383784+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:46:08.748138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"20'4","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:33.384309+0000","last_change":"2024-09-12T03:16:30.584481+0000","last_active":"2024-09-12T03:16:33.384309+0000","last_peered":"2024-09-12T03:16:33.384309+0000","last_clean":"2024-09-12T03:16:33.384309+0000","last_became_active":"2024-09-12T03:16:30.584234+0000","last_became_peered":"2024-09-12T03:16:30.584234+0000","last_unstale":"2024-09-12T03:16:33.384309+0000","last_undegraded":"2024-09-12T03:16:33.384309+0000","last_fullsized":"2024-09-12T03:16:33.384309+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:29:14.889397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542150,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31980,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339860,"statfs":{"total":96636764160,"available":96604016640,"internally_reserved":0,"allocated":811008,"data_stored":665113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542152,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31408,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340432,"statfs":{"total":96636764160,"available":96604602368,"internally_reserved":0,"allocated":225280,"data_stored":75775,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542151,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31404,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340436,"statfs":{"total":96636764160,"available":96604606464,"internally_reserved":0,"allocated":217088,"data_stored":74253,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542150,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31408,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340432,"statfs":{"total":96636764160,"available":96604602368,"internally_reserved":0,"allocated":225280,"data_stored":74289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542151,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":225280,"data_stored":75775,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542151,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":212992,"data_stored":74163,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542152,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":221184,"data_stored":74323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542152,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27828,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344012,"statfs":{"total":96636764160,"available":96608268288,"internally_reserved":0,"allocated":819200,"data_stored":665169,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T03:16:43.313 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-12T03:16:43.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.450+0000 7fd65f313640 1 Processor -- start 2024-09-12T03:16:43.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.450+0000 7fd65f313640 1 -- start start 2024-09-12T03:16:43.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.451+0000 7fd65f313640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fd6580d5200 0x7fd6580d1010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:43.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.451+0000 7fd65f313640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd6580d3760 con 0x7fd65815eb90 2024-09-12T03:16:43.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.451+0000 7fd65f313640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd6580d3a40 con 0x7fd6580d1550 2024-09-12T03:16:43.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.451+0000 7fd65f313640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fd6580b7d20 con 0x7fd6580d5200 2024-09-12T03:16:43.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.451+0000 7fd65d0af640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fd65815eb90 0x7fd6580d2df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:39254/0 (socket says 172.21.15.27:39254) 2024-09-12T03:16:43.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.451+0000 7fd65d0af640 1 -- 172.21.15.27:0/581507970 learned_addr learned my addr 172.21.15.27:0/581507970 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:16:43.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.452+0000 7fd657fff640 1 -- 172.21.15.27:0/581507970 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 532873184 0 0) 0x7fd6580d3760 con 0x7fd65815eb90 2024-09-12T03:16:43.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.452+0000 7fd657fff640 1 -- 172.21.15.27:0/581507970 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd640003520 con 0x7fd65815eb90 2024-09-12T03:16:43.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.452+0000 7fd65c8ae640 1 --2- 172.21.15.27:0/581507970 >> v2:172.21.15.45:3300/0 conn(0x7fd6580d5200 0x7fd6580d1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:43.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.452+0000 7fd657fff640 1 -- 172.21.15.27:0/581507970 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4151493498 0 0) 0x7fd640003520 con 0x7fd65815eb90 2024-09-12T03:16:43.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.452+0000 7fd657fff640 1 -- 172.21.15.27:0/581507970 >> v2:172.21.15.45:3300/0 conn(0x7fd6580d5200 msgr2=0x7fd6580d1010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:43.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.452+0000 7fd657fff640 1 --2- 172.21.15.27:0/581507970 >> v2:172.21.15.45:3300/0 conn(0x7fd6580d5200 0x7fd6580d1010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:43.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.452+0000 7fd657fff640 1 -- 172.21.15.27:0/581507970 >> v1:172.21.15.45:6789/0 conn(0x7fd6580d1550 legacy=0x7fd6580b8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:43.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.452+0000 7fd657fff640 1 -- 172.21.15.27:0/581507970 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6580d3d20 con 0x7fd65815eb90 2024-09-12T03:16:43.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.452+0000 7fd657fff640 1 -- 172.21.15.27:0/581507970 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 847755213 0 0) 0x7fd6480019f0 con 0x7fd65815eb90 2024-09-12T03:16:43.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.452+0000 7fd657fff640 1 -- 172.21.15.27:0/581507970 >> v1:172.21.15.27:6789/0 conn(0x7fd65815eb90 legacy=0x7fd6580d2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:43.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.452+0000 7fd657fff640 1 --2- 172.21.15.27:0/581507970 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fd6400039b0 0x7fd640003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:43.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.452+0000 7fd657fff640 1 --2- 172.21.15.27:0/581507970 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd640004800 0x7fd640004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:43.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.452+0000 7fd657fff640 1 --2- 172.21.15.27:0/581507970 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd6400051c0 0x7fd64000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:43.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.452+0000 7fd657fff640 1 -- 172.21.15.27:0/581507970 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fd64000c0e0 con 0x7fd6400051c0 2024-09-12T03:16:43.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.452+0000 7fd657fff640 1 -- 172.21.15.27:0/581507970 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fd64000c290 con 0x7fd640004800 2024-09-12T03:16:43.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.452+0000 7fd657fff640 1 -- 172.21.15.27:0/581507970 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fd64000c490 con 0x7fd6400039b0 2024-09-12T03:16:43.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.452+0000 7fd65d8b0640 1 --2- 172.21.15.27:0/581507970 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd6400051c0 0x7fd64000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:43.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.452+0000 7fd65d0af640 1 --2- 172.21.15.27:0/581507970 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fd6400039b0 0x7fd640003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:43.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.452+0000 7fd65c8ae640 1 --2- 172.21.15.27:0/581507970 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd640004800 0x7fd640004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:43.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.452+0000 7fd65d8b0640 1 -- 172.21.15.27:0/581507970 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fd6400039b0 msgr2=0x7fd640003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:43.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.452+0000 7fd65d8b0640 1 --2- 172.21.15.27:0/581507970 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fd6400039b0 0x7fd640003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:43.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.452+0000 7fd65d8b0640 1 -- 172.21.15.27:0/581507970 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd640004800 msgr2=0x7fd640004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:43.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.452+0000 7fd65d8b0640 1 --2- 172.21.15.27:0/581507970 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd640004800 0x7fd640004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:43.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.453+0000 7fd65d8b0640 1 -- 172.21.15.27:0/581507970 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd640003520 con 0x7fd6400051c0 2024-09-12T03:16:43.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.453+0000 7fd65d8b0640 1 -- 172.21.15.27:0/581507970 reap_dead start 2024-09-12T03:16:43.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.453+0000 7fd65d8b0640 1 --2- 172.21.15.27:0/581507970 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd6400051c0 0x7fd64000b9c0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fd650001750 tx=0x7fd640005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2fe29884cbca658 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:43.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.453+0000 7fd657fff640 1 -- 172.21.15.27:0/581507970 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd650001370 con 0x7fd6400051c0 2024-09-12T03:16:43.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.453+0000 7fd657fff640 1 -- 172.21.15.27:0/581507970 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd650005310 con 0x7fd6400051c0 2024-09-12T03:16:43.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.453+0000 7fd657fff640 1 -- 172.21.15.27:0/581507970 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd65002abe0 con 0x7fd6400051c0 2024-09-12T03:16:43.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.454+0000 7fd65f313640 1 -- 172.21.15.27:0/581507970 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd6400051c0 msgr2=0x7fd64000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:43.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.454+0000 7fd65f313640 1 --2- 172.21.15.27:0/581507970 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd6400051c0 0x7fd64000b9c0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fd650001750 tx=0x7fd640005830 comp rx=0 tx=0).stop 2024-09-12T03:16:43.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.454+0000 7fd65f313640 1 -- 172.21.15.27:0/581507970 shutdown_connections 2024-09-12T03:16:43.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.454+0000 7fd65f313640 1 --2- 172.21.15.27:0/581507970 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd6400051c0 0x7fd64000b9c0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:43.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.454+0000 7fd65f313640 1 -- 172.21.15.27:0/581507970 >> 172.21.15.27:0/581507970 conn(0x7fd65806dce0 msgr2=0x7fd65806e0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:43.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.454+0000 7fd65f313640 1 -- 172.21.15.27:0/581507970 shutdown_connections 2024-09-12T03:16:43.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.454+0000 7fd65f313640 1 -- 172.21.15.27:0/581507970 wait complete. 2024-09-12T03:16:43.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.455+0000 7fd65f313640 1 Processor -- start 2024-09-12T03:16:43.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.455+0000 7fd65f313640 1 -- start start 2024-09-12T03:16:43.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.456+0000 7fd65f313640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd6400051c0 0x7fd658151d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:43.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.456+0000 7fd65f313640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fd65815ae40 0x7fd6580ab210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:43.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.456+0000 7fd65f313640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd6580cdba0 0x7fd6580ce030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:43.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.456+0000 7fd65f313640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fd6580d1960 con 0x7fd6400051c0 2024-09-12T03:16:43.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.456+0000 7fd65f313640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fd6580d42e0 con 0x7fd6580cdba0 2024-09-12T03:16:43.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.456+0000 7fd65f313640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fd65815ef70 con 0x7fd65815ae40 2024-09-12T03:16:43.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.456+0000 7fd65d0af640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd6400051c0 0x7fd658151d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:43.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.456+0000 7fd65d0af640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd6400051c0 0x7fd658151d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33466/0 (socket says 172.21.15.27:33466) 2024-09-12T03:16:43.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.456+0000 7fd65d0af640 1 -- 172.21.15.27:0/494741189 learned_addr learned my addr 172.21.15.27:0/494741189 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:16:43.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.456+0000 7fd65c8ae640 1 --2- 172.21.15.27:0/494741189 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fd65815ae40 0x7fd6580ab210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:43.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.456+0000 7fd65d8b0640 1 --2- 172.21.15.27:0/494741189 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd6580cdba0 0x7fd6580ce030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:43.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.456+0000 7fd65d0af640 1 -- 172.21.15.27:0/494741189 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fd65815ae40 msgr2=0x7fd6580ab210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:43.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.456+0000 7fd65d0af640 1 --2- 172.21.15.27:0/494741189 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fd65815ae40 0x7fd6580ab210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:43.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.456+0000 7fd65d0af640 1 -- 172.21.15.27:0/494741189 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd6580cdba0 msgr2=0x7fd6580ce030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:43.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.456+0000 7fd65d0af640 1 --2- 172.21.15.27:0/494741189 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd6580cdba0 0x7fd6580ce030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:43.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.456+0000 7fd65d0af640 1 -- 172.21.15.27:0/494741189 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6580aba40 con 0x7fd6400051c0 2024-09-12T03:16:43.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.456+0000 7fd65c8ae640 1 --2- 172.21.15.27:0/494741189 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fd65815ae40 0x7fd6580ab210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:16:43.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.457+0000 7fd65d0af640 1 --2- 172.21.15.27:0/494741189 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd6400051c0 0x7fd658151d10 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fd648007a20 tx=0x7fd64800a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:43.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.457+0000 7fd655ffb640 1 -- 172.21.15.27:0/494741189 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd648017020 con 0x7fd6400051c0 2024-09-12T03:16:43.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.457+0000 7fd655ffb640 1 -- 172.21.15.27:0/494741189 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6480133f0 con 0x7fd6400051c0 2024-09-12T03:16:43.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.457+0000 7fd65f313640 1 -- 172.21.15.27:0/494741189 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6580abd20 con 0x7fd6400051c0 2024-09-12T03:16:43.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.457+0000 7fd655ffb640 1 -- 172.21.15.27:0/494741189 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6480137e0 con 0x7fd6400051c0 2024-09-12T03:16:43.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.457+0000 7fd65f313640 1 -- 172.21.15.27:0/494741189 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6580ac2e0 con 0x7fd6400051c0 2024-09-12T03:16:43.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.458+0000 7fd65f313640 1 -- 172.21.15.27:0/494741189 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd65815e340 con 0x7fd6400051c0 2024-09-12T03:16:43.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.458+0000 7fd655ffb640 1 -- 172.21.15.27:0/494741189 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd64801a020 con 0x7fd6400051c0 2024-09-12T03:16:43.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.459+0000 7fd655ffb640 1 --2- 172.21.15.27:0/494741189 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fd64406be30 0x7fd64406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:43.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.459+0000 7fd655ffb640 1 -- 172.21.15.27:0/494741189 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fd64808b8e0 con 0x7fd6400051c0 2024-09-12T03:16:43.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.459+0000 7fd65c8ae640 1 --2- 172.21.15.27:0/494741189 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fd64406be30 0x7fd64406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:43.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.459+0000 7fd65c8ae640 1 --2- 172.21.15.27:0/494741189 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fd64406be30 0x7fd64406e2f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fd64c006fd0 tx=0x7fd64c008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:43.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.463+0000 7fd655ffb640 1 -- 172.21.15.27:0/494741189 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fd648021020 con 0x7fd6400051c0 2024-09-12T03:16:43.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.634+0000 7fd65f313640 1 -- 172.21.15.27:0/494741189 --> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd6580b8370 con 0x7fd64406be30 2024-09-12T03:16:43.641 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.640+0000 7fd655ffb640 1 -- 172.21.15.27:0/494741189 <== mgr.4102 v2:172.21.15.45:6832/3923821219 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251837 (secure 0 0 0) 0x7fd6580b8370 con 0x7fd64406be30 2024-09-12T03:16:43.645 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:16:43.646 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2024-09-12T03:16:43.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.644+0000 7fd65f313640 1 -- 172.21.15.27:0/494741189 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fd64406be30 msgr2=0x7fd64406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:43.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.644+0000 7fd65f313640 1 --2- 172.21.15.27:0/494741189 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fd64406be30 0x7fd64406e2f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fd64c006fd0 tx=0x7fd64c008040 comp rx=0 tx=0).stop 2024-09-12T03:16:43.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.644+0000 7fd65f313640 1 -- 172.21.15.27:0/494741189 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd6400051c0 msgr2=0x7fd658151d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:43.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.644+0000 7fd65f313640 1 --2- 172.21.15.27:0/494741189 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd6400051c0 0x7fd658151d10 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fd648007a20 tx=0x7fd64800a6f0 comp rx=0 tx=0).stop 2024-09-12T03:16:43.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.644+0000 7fd65f313640 1 -- 172.21.15.27:0/494741189 shutdown_connections 2024-09-12T03:16:43.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.644+0000 7fd65f313640 1 --2- 172.21.15.27:0/494741189 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fd64406be30 0x7fd64406e2f0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:43.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.644+0000 7fd65f313640 1 --2- 172.21.15.27:0/494741189 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd6580cdba0 0x7fd6580ce030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:43.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.644+0000 7fd65f313640 1 --2- 172.21.15.27:0/494741189 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fd65815ae40 0x7fd6580ab210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:43.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.644+0000 7fd65f313640 1 --2- 172.21.15.27:0/494741189 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd6400051c0 0x7fd658151d10 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:43.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.644+0000 7fd65f313640 1 -- 172.21.15.27:0/494741189 >> 172.21.15.27:0/494741189 conn(0x7fd65806dce0 msgr2=0x7fd658160590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:43.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.644+0000 7fd65f313640 1 -- 172.21.15.27:0/494741189 shutdown_connections 2024-09-12T03:16:43.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.644+0000 7fd65f313640 1 -- 172.21.15.27:0/494741189 wait complete. 2024-09-12T03:16:43.655 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2024-09-12T03:16:42.054875+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235760,"kb_used_data":2888,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738960,"statfs":{"total":773094113280,"available":772852695040,"internally_reserved":0,"allocated":2957312,"data_stored":1778860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.002251"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:33.384323+0000","last_change":"2024-09-12T03:16:31.583498+0000","last_active":"2024-09-12T03:16:33.384323+0000","last_peered":"2024-09-12T03:16:33.384323+0000","last_clean":"2024-09-12T03:16:33.384323+0000","last_became_active":"2024-09-12T03:16:31.583196+0000","last_became_peered":"2024-09-12T03:16:31.583196+0000","last_unstale":"2024-09-12T03:16:33.384323+0000","last_undegraded":"2024-09-12T03:16:33.384323+0000","last_fullsized":"2024-09-12T03:16:33.384323+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:08:48.647955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:32.578394+0000","last_change":"2024-09-12T03:16:31.584109+0000","last_active":"2024-09-12T03:16:32.578394+0000","last_peered":"2024-09-12T03:16:32.578394+0000","last_clean":"2024-09-12T03:16:32.578394+0000","last_became_active":"2024-09-12T03:16:31.583836+0000","last_became_peered":"2024-09-12T03:16:31.583836+0000","last_unstale":"2024-09-12T03:16:32.578394+0000","last_undegraded":"2024-09-12T03:16:32.578394+0000","last_fullsized":"2024-09-12T03:16:32.578394+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:27:20.447453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:33.384286+0000","last_change":"2024-09-12T03:16:31.574829+0000","last_active":"2024-09-12T03:16:33.384286+0000","last_peered":"2024-09-12T03:16:33.384286+0000","last_clean":"2024-09-12T03:16:33.384286+0000","last_became_active":"2024-09-12T03:16:31.574523+0000","last_became_peered":"2024-09-12T03:16:31.574523+0000","last_unstale":"2024-09-12T03:16:33.384286+0000","last_undegraded":"2024-09-12T03:16:33.384286+0000","last_fullsized":"2024-09-12T03:16:33.384286+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:33:01.912717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:32.580588+0000","last_change":"2024-09-12T03:16:31.585853+0000","last_active":"2024-09-12T03:16:32.580588+0000","last_peered":"2024-09-12T03:16:32.580588+0000","last_clean":"2024-09-12T03:16:32.580588+0000","last_became_active":"2024-09-12T03:16:31.583206+0000","last_became_peered":"2024-09-12T03:16:31.583206+0000","last_unstale":"2024-09-12T03:16:32.580588+0000","last_undegraded":"2024-09-12T03:16:32.580588+0000","last_fullsized":"2024-09-12T03:16:32.580588+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:05:29.506554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:33.383913+0000","last_change":"2024-09-12T03:16:31.582095+0000","last_active":"2024-09-12T03:16:33.383913+0000","last_peered":"2024-09-12T03:16:33.383913+0000","last_clean":"2024-09-12T03:16:33.383913+0000","last_became_active":"2024-09-12T03:16:31.581414+0000","last_became_peered":"2024-09-12T03:16:31.581414+0000","last_unstale":"2024-09-12T03:16:33.383913+0000","last_undegraded":"2024-09-12T03:16:33.383913+0000","last_fullsized":"2024-09-12T03:16:33.383913+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:44:06.896638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:32.579523+0000","last_change":"2024-09-12T03:16:31.584676+0000","last_active":"2024-09-12T03:16:32.579523+0000","last_peered":"2024-09-12T03:16:32.579523+0000","last_clean":"2024-09-12T03:16:32.579523+0000","last_became_active":"2024-09-12T03:16:31.584392+0000","last_became_peered":"2024-09-12T03:16:31.584392+0000","last_unstale":"2024-09-12T03:16:32.579523+0000","last_undegraded":"2024-09-12T03:16:32.579523+0000","last_fullsized":"2024-09-12T03:16:32.579523+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:03:19.355158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:33.383478+0000","last_change":"2024-09-12T03:16:31.587169+0000","last_active":"2024-09-12T03:16:33.383478+0000","last_peered":"2024-09-12T03:16:33.383478+0000","last_clean":"2024-09-12T03:16:33.383478+0000","last_became_active":"2024-09-12T03:16:31.586884+0000","last_became_peered":"2024-09-12T03:16:31.586884+0000","last_unstale":"2024-09-12T03:16:33.383478+0000","last_undegraded":"2024-09-12T03:16:33.383478+0000","last_fullsized":"2024-09-12T03:16:33.383478+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:35:22.374781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:33.383041+0000","last_change":"2024-09-12T03:16:31.592684+0000","last_active":"2024-09-12T03:16:33.383041+0000","last_peered":"2024-09-12T03:16:33.383041+0000","last_clean":"2024-09-12T03:16:33.383041+0000","last_became_active":"2024-09-12T03:16:31.592374+0000","last_became_peered":"2024-09-12T03:16:31.592374+0000","last_unstale":"2024-09-12T03:16:33.383041+0000","last_undegraded":"2024-09-12T03:16:33.383041+0000","last_fullsized":"2024-09-12T03:16:33.383041+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:18:57.036317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:33.384083+0000","last_change":"2024-09-12T03:16:31.590609+0000","last_active":"2024-09-12T03:16:33.384083+0000","last_peered":"2024-09-12T03:16:33.384083+0000","last_clean":"2024-09-12T03:16:33.384083+0000","last_became_active":"2024-09-12T03:16:31.590320+0000","last_became_peered":"2024-09-12T03:16:31.590320+0000","last_unstale":"2024-09-12T03:16:33.384083+0000","last_undegraded":"2024-09-12T03:16:33.384083+0000","last_fullsized":"2024-09-12T03:16:33.384083+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:56:03.306616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:33.384773+0000","last_change":"2024-09-12T03:16:31.584375+0000","last_active":"2024-09-12T03:16:33.384773+0000","last_peered":"2024-09-12T03:16:33.384773+0000","last_clean":"2024-09-12T03:16:33.384773+0000","last_became_active":"2024-09-12T03:16:31.581766+0000","last_became_peered":"2024-09-12T03:16:31.581766+0000","last_unstale":"2024-09-12T03:16:33.384773+0000","last_undegraded":"2024-09-12T03:16:33.384773+0000","last_fullsized":"2024-09-12T03:16:33.384773+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:23:22.497643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:33.384404+0000","last_change":"2024-09-12T03:16:31.581788+0000","last_active":"2024-09-12T03:16:33.384404+0000","last_peered":"2024-09-12T03:16:33.384404+0000","last_clean":"2024-09-12T03:16:33.384404+0000","last_became_active":"2024-09-12T03:16:31.581516+0000","last_became_peered":"2024-09-12T03:16:31.581516+0000","last_unstale":"2024-09-12T03:16:33.384404+0000","last_undegraded":"2024-09-12T03:16:33.384404+0000","last_fullsized":"2024-09-12T03:16:33.384404+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:52:47.384994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:32.580659+0000","last_change":"2024-09-12T03:16:31.579564+0000","last_active":"2024-09-12T03:16:32.580659+0000","last_peered":"2024-09-12T03:16:32.580659+0000","last_clean":"2024-09-12T03:16:32.580659+0000","last_became_active":"2024-09-12T03:16:31.579247+0000","last_became_peered":"2024-09-12T03:16:31.579247+0000","last_unstale":"2024-09-12T03:16:32.580659+0000","last_undegraded":"2024-09-12T03:16:32.580659+0000","last_fullsized":"2024-09-12T03:16:32.580659+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:19:43.151205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:33.383395+0000","last_change":"2024-09-12T03:16:31.580809+0000","last_active":"2024-09-12T03:16:33.383395+0000","last_peered":"2024-09-12T03:16:33.383395+0000","last_clean":"2024-09-12T03:16:33.383395+0000","last_became_active":"2024-09-12T03:16:31.580520+0000","last_became_peered":"2024-09-12T03:16:31.580520+0000","last_unstale":"2024-09-12T03:16:33.383395+0000","last_undegraded":"2024-09-12T03:16:33.383395+0000","last_fullsized":"2024-09-12T03:16:33.383395+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:52:47.384994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:33.383780+0000","last_change":"2024-09-12T03:16:31.589461+0000","last_active":"2024-09-12T03:16:33.383780+0000","last_peered":"2024-09-12T03:16:33.383780+0000","last_clean":"2024-09-12T03:16:33.383780+0000","last_became_active":"2024-09-12T03:16:31.587058+0000","last_became_peered":"2024-09-12T03:16:31.587058+0000","last_unstale":"2024-09-12T03:16:33.383780+0000","last_undegraded":"2024-09-12T03:16:33.383780+0000","last_fullsized":"2024-09-12T03:16:33.383780+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:27:08.819051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:32.578055+0000","last_change":"2024-09-12T03:16:31.589724+0000","last_active":"2024-09-12T03:16:32.578055+0000","last_peered":"2024-09-12T03:16:32.578055+0000","last_clean":"2024-09-12T03:16:32.578055+0000","last_became_active":"2024-09-12T03:16:31.589452+0000","last_became_peered":"2024-09-12T03:16:31.589452+0000","last_unstale":"2024-09-12T03:16:32.578055+0000","last_undegraded":"2024-09-12T03:16:32.578055+0000","last_fullsized":"2024-09-12T03:16:32.578055+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:30:37.571167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:32.579421+0000","last_change":"2024-09-12T03:16:31.574474+0000","last_active":"2024-09-12T03:16:32.579421+0000","last_peered":"2024-09-12T03:16:32.579421+0000","last_clean":"2024-09-12T03:16:32.579421+0000","last_became_active":"2024-09-12T03:16:31.574190+0000","last_became_peered":"2024-09-12T03:16:31.574190+0000","last_unstale":"2024-09-12T03:16:32.579421+0000","last_undegraded":"2024-09-12T03:16:32.579421+0000","last_fullsized":"2024-09-12T03:16:32.579421+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:27:20.447453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:32.580359+0000","last_change":"2024-09-12T03:16:31.589878+0000","last_active":"2024-09-12T03:16:32.580359+0000","last_peered":"2024-09-12T03:16:32.580359+0000","last_clean":"2024-09-12T03:16:32.580359+0000","last_became_active":"2024-09-12T03:16:31.589603+0000","last_became_peered":"2024-09-12T03:16:31.589603+0000","last_unstale":"2024-09-12T03:16:32.580359+0000","last_undegraded":"2024-09-12T03:16:32.580359+0000","last_fullsized":"2024-09-12T03:16:32.580359+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:39:25.162423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:33.341837+0000","last_change":"2024-09-12T03:16:31.579526+0000","last_active":"2024-09-12T03:16:33.341837+0000","last_peered":"2024-09-12T03:16:33.341837+0000","last_clean":"2024-09-12T03:16:33.341837+0000","last_became_active":"2024-09-12T03:16:31.579217+0000","last_became_peered":"2024-09-12T03:16:31.579217+0000","last_unstale":"2024-09-12T03:16:33.341837+0000","last_undegraded":"2024-09-12T03:16:33.341837+0000","last_fullsized":"2024-09-12T03:16:33.341837+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:49:09.388955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:33.384367+0000","last_change":"2024-09-12T03:16:31.588216+0000","last_active":"2024-09-12T03:16:33.384367+0000","last_peered":"2024-09-12T03:16:33.384367+0000","last_clean":"2024-09-12T03:16:33.384367+0000","last_became_active":"2024-09-12T03:16:31.587499+0000","last_became_peered":"2024-09-12T03:16:31.587499+0000","last_unstale":"2024-09-12T03:16:33.384367+0000","last_undegraded":"2024-09-12T03:16:33.384367+0000","last_fullsized":"2024-09-12T03:16:33.384367+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:07:30.199995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:33.384562+0000","last_change":"2024-09-12T03:16:31.589828+0000","last_active":"2024-09-12T03:16:33.384562+0000","last_peered":"2024-09-12T03:16:33.384562+0000","last_clean":"2024-09-12T03:16:33.384562+0000","last_became_active":"2024-09-12T03:16:31.589341+0000","last_became_peered":"2024-09-12T03:16:31.589341+0000","last_unstale":"2024-09-12T03:16:33.384562+0000","last_undegraded":"2024-09-12T03:16:33.384562+0000","last_fullsized":"2024-09-12T03:16:33.384562+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:05:13.773162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:33.384136+0000","last_change":"2024-09-12T03:16:31.586744+0000","last_active":"2024-09-12T03:16:33.384136+0000","last_peered":"2024-09-12T03:16:33.384136+0000","last_clean":"2024-09-12T03:16:33.384136+0000","last_became_active":"2024-09-12T03:16:31.586462+0000","last_became_peered":"2024-09-12T03:16:31.586462+0000","last_unstale":"2024-09-12T03:16:33.384136+0000","last_undegraded":"2024-09-12T03:16:33.384136+0000","last_fullsized":"2024-09-12T03:16:33.384136+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:52:47.384994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:32.579721+0000","last_change":"2024-09-12T03:16:31.583270+0000","last_active":"2024-09-12T03:16:32.579721+0000","last_peered":"2024-09-12T03:16:32.579721+0000","last_clean":"2024-09-12T03:16:32.579721+0000","last_became_active":"2024-09-12T03:16:31.582948+0000","last_became_peered":"2024-09-12T03:16:31.582948+0000","last_unstale":"2024-09-12T03:16:32.579721+0000","last_undegraded":"2024-09-12T03:16:32.579721+0000","last_fullsized":"2024-09-12T03:16:32.579721+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:05:55.585173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:33.383081+0000","last_change":"2024-09-12T03:16:31.590127+0000","last_active":"2024-09-12T03:16:33.383081+0000","last_peered":"2024-09-12T03:16:33.383081+0000","last_clean":"2024-09-12T03:16:33.383081+0000","last_became_active":"2024-09-12T03:16:31.589847+0000","last_became_peered":"2024-09-12T03:16:31.589847+0000","last_unstale":"2024-09-12T03:16:33.383081+0000","last_undegraded":"2024-09-12T03:16:33.383081+0000","last_fullsized":"2024-09-12T03:16:33.383081+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:27:08.819051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:32.579620+0000","last_change":"2024-09-12T03:16:31.578711+0000","last_active":"2024-09-12T03:16:32.579620+0000","last_peered":"2024-09-12T03:16:32.579620+0000","last_clean":"2024-09-12T03:16:32.579620+0000","last_became_active":"2024-09-12T03:16:31.578421+0000","last_became_peered":"2024-09-12T03:16:31.578421+0000","last_unstale":"2024-09-12T03:16:32.579620+0000","last_undegraded":"2024-09-12T03:16:32.579620+0000","last_fullsized":"2024-09-12T03:16:32.579620+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:21:32.875776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:33.383880+0000","last_change":"2024-09-12T03:16:31.589412+0000","last_active":"2024-09-12T03:16:33.383880+0000","last_peered":"2024-09-12T03:16:33.383880+0000","last_clean":"2024-09-12T03:16:33.383880+0000","last_became_active":"2024-09-12T03:16:31.589147+0000","last_became_peered":"2024-09-12T03:16:31.589147+0000","last_unstale":"2024-09-12T03:16:33.383880+0000","last_undegraded":"2024-09-12T03:16:33.383880+0000","last_fullsized":"2024-09-12T03:16:33.383880+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:44:02.835393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:33.384825+0000","last_change":"2024-09-12T03:16:31.586179+0000","last_active":"2024-09-12T03:16:33.384825+0000","last_peered":"2024-09-12T03:16:33.384825+0000","last_clean":"2024-09-12T03:16:33.384825+0000","last_became_active":"2024-09-12T03:16:31.585793+0000","last_became_peered":"2024-09-12T03:16:31.585793+0000","last_unstale":"2024-09-12T03:16:33.384825+0000","last_undegraded":"2024-09-12T03:16:33.384825+0000","last_fullsized":"2024-09-12T03:16:33.384825+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:46:08.748138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:32.578142+0000","last_change":"2024-09-12T03:16:31.578743+0000","last_active":"2024-09-12T03:16:32.578142+0000","last_peered":"2024-09-12T03:16:32.578142+0000","last_clean":"2024-09-12T03:16:32.578142+0000","last_became_active":"2024-09-12T03:16:31.578450+0000","last_became_peered":"2024-09-12T03:16:31.578450+0000","last_unstale":"2024-09-12T03:16:32.578142+0000","last_undegraded":"2024-09-12T03:16:32.578142+0000","last_fullsized":"2024-09-12T03:16:32.578142+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:05:01.246614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:33.383793+0000","last_change":"2024-09-12T03:16:31.590186+0000","last_active":"2024-09-12T03:16:33.383793+0000","last_peered":"2024-09-12T03:16:33.383793+0000","last_clean":"2024-09-12T03:16:33.383793+0000","last_became_active":"2024-09-12T03:16:31.589895+0000","last_became_peered":"2024-09-12T03:16:31.589895+0000","last_unstale":"2024-09-12T03:16:33.383793+0000","last_undegraded":"2024-09-12T03:16:33.383793+0000","last_fullsized":"2024-09-12T03:16:33.383793+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:33:01.912717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:33.384305+0000","last_change":"2024-09-12T03:16:31.577171+0000","last_active":"2024-09-12T03:16:33.384305+0000","last_peered":"2024-09-12T03:16:33.384305+0000","last_clean":"2024-09-12T03:16:33.384305+0000","last_became_active":"2024-09-12T03:16:31.576882+0000","last_became_peered":"2024-09-12T03:16:31.576882+0000","last_unstale":"2024-09-12T03:16:33.384305+0000","last_undegraded":"2024-09-12T03:16:33.384305+0000","last_fullsized":"2024-09-12T03:16:33.384305+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:44:02.835393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:32.579941+0000","last_change":"2024-09-12T03:16:31.578670+0000","last_active":"2024-09-12T03:16:32.579941+0000","last_peered":"2024-09-12T03:16:32.579941+0000","last_clean":"2024-09-12T03:16:32.579941+0000","last_became_active":"2024-09-12T03:16:31.578379+0000","last_became_peered":"2024-09-12T03:16:31.578379+0000","last_unstale":"2024-09-12T03:16:32.579941+0000","last_undegraded":"2024-09-12T03:16:32.579941+0000","last_fullsized":"2024-09-12T03:16:32.579941+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:22:01.431497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:33.341761+0000","last_change":"2024-09-12T03:16:31.589511+0000","last_active":"2024-09-12T03:16:33.341761+0000","last_peered":"2024-09-12T03:16:33.341761+0000","last_clean":"2024-09-12T03:16:33.341761+0000","last_became_active":"2024-09-12T03:16:31.589230+0000","last_became_peered":"2024-09-12T03:16:31.589230+0000","last_unstale":"2024-09-12T03:16:33.341761+0000","last_undegraded":"2024-09-12T03:16:33.341761+0000","last_fullsized":"2024-09-12T03:16:33.341761+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:54:17.604800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:33.383613+0000","last_change":"2024-09-12T03:16:31.585042+0000","last_active":"2024-09-12T03:16:33.383613+0000","last_peered":"2024-09-12T03:16:33.383613+0000","last_clean":"2024-09-12T03:16:33.383613+0000","last_became_active":"2024-09-12T03:16:31.584495+0000","last_became_peered":"2024-09-12T03:16:31.584495+0000","last_unstale":"2024-09-12T03:16:33.383613+0000","last_undegraded":"2024-09-12T03:16:33.383613+0000","last_fullsized":"2024-09-12T03:16:33.383613+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:27:31.919800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:32.580182+0000","last_change":"2024-09-12T03:16:31.588040+0000","last_active":"2024-09-12T03:16:32.580182+0000","last_peered":"2024-09-12T03:16:32.580182+0000","last_clean":"2024-09-12T03:16:32.580182+0000","last_became_active":"2024-09-12T03:16:31.587757+0000","last_became_peered":"2024-09-12T03:16:31.587757+0000","last_unstale":"2024-09-12T03:16:32.580182+0000","last_undegraded":"2024-09-12T03:16:32.580182+0000","last_fullsized":"2024-09-12T03:16:32.580182+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:01:36.472991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:33.383460+0000","last_change":"2024-09-12T03:16:30.583423+0000","last_active":"2024-09-12T03:16:33.383460+0000","last_peered":"2024-09-12T03:16:33.383460+0000","last_clean":"2024-09-12T03:16:33.383460+0000","last_became_active":"2024-09-12T03:16:30.583160+0000","last_became_peered":"2024-09-12T03:16:30.583160+0000","last_unstale":"2024-09-12T03:16:33.383460+0000","last_undegraded":"2024-09-12T03:16:33.383460+0000","last_fullsized":"2024-09-12T03:16:33.383460+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:19:47.998143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:33.384360+0000","last_change":"2024-09-12T03:16:31.591565+0000","last_active":"2024-09-12T03:16:33.384360+0000","last_peered":"2024-09-12T03:16:33.384360+0000","last_clean":"2024-09-12T03:16:33.384360+0000","last_became_active":"2024-09-12T03:16:31.591236+0000","last_became_peered":"2024-09-12T03:16:31.591236+0000","last_unstale":"2024-09-12T03:16:33.384360+0000","last_undegraded":"2024-09-12T03:16:33.384360+0000","last_fullsized":"2024-09-12T03:16:33.384360+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:35:22.374781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:33.383956+0000","last_change":"2024-09-12T03:16:30.575856+0000","last_active":"2024-09-12T03:16:33.383956+0000","last_peered":"2024-09-12T03:16:33.383956+0000","last_clean":"2024-09-12T03:16:33.383956+0000","last_became_active":"2024-09-12T03:16:30.575512+0000","last_became_peered":"2024-09-12T03:16:30.575512+0000","last_unstale":"2024-09-12T03:16:33.383956+0000","last_undegraded":"2024-09-12T03:16:33.383956+0000","last_fullsized":"2024-09-12T03:16:33.383956+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:13:33.665229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:33.383136+0000","last_change":"2024-09-12T03:16:30.580131+0000","last_active":"2024-09-12T03:16:33.383136+0000","last_peered":"2024-09-12T03:16:33.383136+0000","last_clean":"2024-09-12T03:16:33.383136+0000","last_became_active":"2024-09-12T03:16:30.579760+0000","last_became_peered":"2024-09-12T03:16:30.579760+0000","last_unstale":"2024-09-12T03:16:33.383136+0000","last_undegraded":"2024-09-12T03:16:33.383136+0000","last_fullsized":"2024-09-12T03:16:33.383136+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:29:14.889397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:33.383040+0000","last_change":"2024-09-12T03:16:31.581185+0000","last_active":"2024-09-12T03:16:33.383040+0000","last_peered":"2024-09-12T03:16:33.383040+0000","last_clean":"2024-09-12T03:16:33.383040+0000","last_became_active":"2024-09-12T03:16:31.580425+0000","last_became_peered":"2024-09-12T03:16:31.580425+0000","last_unstale":"2024-09-12T03:16:33.383040+0000","last_undegraded":"2024-09-12T03:16:33.383040+0000","last_fullsized":"2024-09-12T03:16:33.383040+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:12:47.905772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:32.579685+0000","last_change":"2024-09-12T03:16:30.571998+0000","last_active":"2024-09-12T03:16:32.579685+0000","last_peered":"2024-09-12T03:16:32.579685+0000","last_clean":"2024-09-12T03:16:32.579685+0000","last_became_active":"2024-09-12T03:16:30.571673+0000","last_became_peered":"2024-09-12T03:16:30.571673+0000","last_unstale":"2024-09-12T03:16:32.579685+0000","last_undegraded":"2024-09-12T03:16:32.579685+0000","last_fullsized":"2024-09-12T03:16:32.579685+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:19:17.746405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:32.578040+0000","last_change":"2024-09-12T03:16:31.587700+0000","last_active":"2024-09-12T03:16:32.578040+0000","last_peered":"2024-09-12T03:16:32.578040+0000","last_clean":"2024-09-12T03:16:32.578040+0000","last_became_active":"2024-09-12T03:16:31.587296+0000","last_became_peered":"2024-09-12T03:16:31.587296+0000","last_unstale":"2024-09-12T03:16:32.578040+0000","last_undegraded":"2024-09-12T03:16:32.578040+0000","last_fullsized":"2024-09-12T03:16:32.578040+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:28:08.918294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:33.384131+0000","last_change":"2024-09-12T03:16:30.581517+0000","last_active":"2024-09-12T03:16:33.384131+0000","last_peered":"2024-09-12T03:16:33.384131+0000","last_clean":"2024-09-12T03:16:33.384131+0000","last_became_active":"2024-09-12T03:16:30.581107+0000","last_became_peered":"2024-09-12T03:16:30.581107+0000","last_unstale":"2024-09-12T03:16:33.384131+0000","last_undegraded":"2024-09-12T03:16:33.384131+0000","last_fullsized":"2024-09-12T03:16:33.384131+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:01:35.467902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:32.580094+0000","last_change":"2024-09-12T03:16:31.588465+0000","last_active":"2024-09-12T03:16:32.580094+0000","last_peered":"2024-09-12T03:16:32.580094+0000","last_clean":"2024-09-12T03:16:32.580094+0000","last_became_active":"2024-09-12T03:16:31.588150+0000","last_became_peered":"2024-09-12T03:16:31.588150+0000","last_unstale":"2024-09-12T03:16:32.580094+0000","last_undegraded":"2024-09-12T03:16:32.580094+0000","last_fullsized":"2024-09-12T03:16:32.580094+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:04:42.975000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:32.580148+0000","last_change":"2024-09-12T03:16:30.578195+0000","last_active":"2024-09-12T03:16:32.580148+0000","last_peered":"2024-09-12T03:16:32.580148+0000","last_clean":"2024-09-12T03:16:32.580148+0000","last_became_active":"2024-09-12T03:16:30.577878+0000","last_became_peered":"2024-09-12T03:16:30.577878+0000","last_unstale":"2024-09-12T03:16:32.580148+0000","last_undegraded":"2024-09-12T03:16:32.580148+0000","last_fullsized":"2024-09-12T03:16:32.580148+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:19:07.178848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:33.384177+0000","last_change":"2024-09-12T03:16:31.584084+0000","last_active":"2024-09-12T03:16:33.384177+0000","last_peered":"2024-09-12T03:16:33.384177+0000","last_clean":"2024-09-12T03:16:33.384177+0000","last_became_active":"2024-09-12T03:16:31.583539+0000","last_became_peered":"2024-09-12T03:16:31.583539+0000","last_unstale":"2024-09-12T03:16:33.384177+0000","last_undegraded":"2024-09-12T03:16:33.384177+0000","last_fullsized":"2024-09-12T03:16:33.384177+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:23:22.497643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:32.578203+0000","last_change":"2024-09-12T03:16:30.572959+0000","last_active":"2024-09-12T03:16:32.578203+0000","last_peered":"2024-09-12T03:16:32.578203+0000","last_clean":"2024-09-12T03:16:32.578203+0000","last_became_active":"2024-09-12T03:16:30.572692+0000","last_became_peered":"2024-09-12T03:16:30.572692+0000","last_unstale":"2024-09-12T03:16:32.578203+0000","last_undegraded":"2024-09-12T03:16:32.578203+0000","last_fullsized":"2024-09-12T03:16:32.578203+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:21:08.051836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:32.580840+0000","last_change":"2024-09-12T03:16:31.590092+0000","last_active":"2024-09-12T03:16:32.580840+0000","last_peered":"2024-09-12T03:16:32.580840+0000","last_clean":"2024-09-12T03:16:32.580840+0000","last_became_active":"2024-09-12T03:16:31.589831+0000","last_became_peered":"2024-09-12T03:16:31.589831+0000","last_unstale":"2024-09-12T03:16:32.580840+0000","last_undegraded":"2024-09-12T03:16:32.580840+0000","last_fullsized":"2024-09-12T03:16:32.580840+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:54:17.604800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"20'6","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:33.384460+0000","last_change":"2024-09-12T03:16:30.575948+0000","last_active":"2024-09-12T03:16:33.384460+0000","last_peered":"2024-09-12T03:16:33.384460+0000","last_clean":"2024-09-12T03:16:33.384460+0000","last_became_active":"2024-09-12T03:16:30.575651+0000","last_became_peered":"2024-09-12T03:16:30.575651+0000","last_unstale":"2024-09-12T03:16:33.384460+0000","last_undegraded":"2024-09-12T03:16:33.384460+0000","last_fullsized":"2024-09-12T03:16:33.384460+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:44:10.172361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:33.383628+0000","last_change":"2024-09-12T03:16:31.587604+0000","last_active":"2024-09-12T03:16:33.383628+0000","last_peered":"2024-09-12T03:16:33.383628+0000","last_clean":"2024-09-12T03:16:33.383628+0000","last_became_active":"2024-09-12T03:16:31.581590+0000","last_became_peered":"2024-09-12T03:16:31.581590+0000","last_unstale":"2024-09-12T03:16:33.383628+0000","last_undegraded":"2024-09-12T03:16:33.383628+0000","last_fullsized":"2024-09-12T03:16:33.383628+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:40:23.693285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:33.384342+0000","last_change":"2024-09-12T03:16:30.580151+0000","last_active":"2024-09-12T03:16:33.384342+0000","last_peered":"2024-09-12T03:16:33.384342+0000","last_clean":"2024-09-12T03:16:33.384342+0000","last_became_active":"2024-09-12T03:16:30.579859+0000","last_became_peered":"2024-09-12T03:16:30.579859+0000","last_unstale":"2024-09-12T03:16:33.384342+0000","last_undegraded":"2024-09-12T03:16:33.384342+0000","last_fullsized":"2024-09-12T03:16:33.384342+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:44:10.172361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:33.383933+0000","last_change":"2024-09-12T03:16:31.580748+0000","last_active":"2024-09-12T03:16:33.383933+0000","last_peered":"2024-09-12T03:16:33.383933+0000","last_clean":"2024-09-12T03:16:33.383933+0000","last_became_active":"2024-09-12T03:16:31.580434+0000","last_became_peered":"2024-09-12T03:16:31.580434+0000","last_unstale":"2024-09-12T03:16:33.383933+0000","last_undegraded":"2024-09-12T03:16:33.383933+0000","last_fullsized":"2024-09-12T03:16:33.383933+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:27:31.919800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:32.580017+0000","last_change":"2024-09-12T03:16:30.570604+0000","last_active":"2024-09-12T03:16:32.580017+0000","last_peered":"2024-09-12T03:16:32.580017+0000","last_clean":"2024-09-12T03:16:32.580017+0000","last_became_active":"2024-09-12T03:16:30.570227+0000","last_became_peered":"2024-09-12T03:16:30.570227+0000","last_unstale":"2024-09-12T03:16:32.580017+0000","last_undegraded":"2024-09-12T03:16:32.580017+0000","last_fullsized":"2024-09-12T03:16:32.580017+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:55:03.313892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:33.383958+0000","last_change":"2024-09-12T03:16:31.592248+0000","last_active":"2024-09-12T03:16:33.383958+0000","last_peered":"2024-09-12T03:16:33.383958+0000","last_clean":"2024-09-12T03:16:33.383958+0000","last_became_active":"2024-09-12T03:16:31.591836+0000","last_became_peered":"2024-09-12T03:16:31.591836+0000","last_unstale":"2024-09-12T03:16:33.383958+0000","last_undegraded":"2024-09-12T03:16:33.383958+0000","last_fullsized":"2024-09-12T03:16:33.383958+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:05:17.785019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:33.383324+0000","last_change":"2024-09-12T03:16:30.578648+0000","last_active":"2024-09-12T03:16:33.383324+0000","last_peered":"2024-09-12T03:16:33.383324+0000","last_clean":"2024-09-12T03:16:33.383324+0000","last_became_active":"2024-09-12T03:16:30.578343+0000","last_became_peered":"2024-09-12T03:16:30.578343+0000","last_unstale":"2024-09-12T03:16:33.383324+0000","last_undegraded":"2024-09-12T03:16:33.383324+0000","last_fullsized":"2024-09-12T03:16:33.383324+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:29:14.889397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:33.383440+0000","last_change":"2024-09-12T03:16:31.584802+0000","last_active":"2024-09-12T03:16:33.383440+0000","last_peered":"2024-09-12T03:16:33.383440+0000","last_clean":"2024-09-12T03:16:33.383440+0000","last_became_active":"2024-09-12T03:16:31.584124+0000","last_became_peered":"2024-09-12T03:16:31.584124+0000","last_unstale":"2024-09-12T03:16:33.383440+0000","last_undegraded":"2024-09-12T03:16:33.383440+0000","last_fullsized":"2024-09-12T03:16:33.383440+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:27:08.819051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":22,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:33.341548+0000","last_change":"2024-09-12T03:16:30.574668+0000","last_active":"2024-09-12T03:16:33.341548+0000","last_peered":"2024-09-12T03:16:33.341548+0000","last_clean":"2024-09-12T03:16:33.341548+0000","last_became_active":"2024-09-12T03:16:30.574373+0000","last_became_peered":"2024-09-12T03:16:30.574373+0000","last_unstale":"2024-09-12T03:16:33.341548+0000","last_undegraded":"2024-09-12T03:16:33.341548+0000","last_fullsized":"2024-09-12T03:16:33.341548+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:19:47.998143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:33.383657+0000","last_change":"2024-09-12T03:16:31.592520+0000","last_active":"2024-09-12T03:16:33.383657+0000","last_peered":"2024-09-12T03:16:33.383657+0000","last_clean":"2024-09-12T03:16:33.383657+0000","last_became_active":"2024-09-12T03:16:31.591766+0000","last_became_peered":"2024-09-12T03:16:31.591766+0000","last_unstale":"2024-09-12T03:16:33.383657+0000","last_undegraded":"2024-09-12T03:16:33.383657+0000","last_fullsized":"2024-09-12T03:16:33.383657+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:22:52.960561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:32.579754+0000","last_change":"2024-09-12T03:16:30.582087+0000","last_active":"2024-09-12T03:16:32.579754+0000","last_peered":"2024-09-12T03:16:32.579754+0000","last_clean":"2024-09-12T03:16:32.579754+0000","last_became_active":"2024-09-12T03:16:30.581622+0000","last_became_peered":"2024-09-12T03:16:30.581622+0000","last_unstale":"2024-09-12T03:16:32.579754+0000","last_undegraded":"2024-09-12T03:16:32.579754+0000","last_fullsized":"2024-09-12T03:16:32.579754+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:58:54.372496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:33.384324+0000","last_change":"2024-09-12T03:16:31.591374+0000","last_active":"2024-09-12T03:16:33.384324+0000","last_peered":"2024-09-12T03:16:33.384324+0000","last_clean":"2024-09-12T03:16:33.384324+0000","last_became_active":"2024-09-12T03:16:31.591092+0000","last_became_peered":"2024-09-12T03:16:31.591092+0000","last_unstale":"2024-09-12T03:16:33.384324+0000","last_undegraded":"2024-09-12T03:16:33.384324+0000","last_fullsized":"2024-09-12T03:16:33.384324+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:08:36.235105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:33.341748+0000","last_change":"2024-09-12T03:16:30.570924+0000","last_active":"2024-09-12T03:16:33.341748+0000","last_peered":"2024-09-12T03:16:33.341748+0000","last_clean":"2024-09-12T03:16:33.341748+0000","last_became_active":"2024-09-12T03:16:30.570636+0000","last_became_peered":"2024-09-12T03:16:30.570636+0000","last_unstale":"2024-09-12T03:16:33.341748+0000","last_undegraded":"2024-09-12T03:16:33.341748+0000","last_fullsized":"2024-09-12T03:16:33.341748+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:29:14.889397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:33.384281+0000","last_change":"2024-09-12T03:16:31.592777+0000","last_active":"2024-09-12T03:16:33.384281+0000","last_peered":"2024-09-12T03:16:33.384281+0000","last_clean":"2024-09-12T03:16:33.384281+0000","last_became_active":"2024-09-12T03:16:31.591855+0000","last_became_peered":"2024-09-12T03:16:31.591855+0000","last_unstale":"2024-09-12T03:16:33.384281+0000","last_undegraded":"2024-09-12T03:16:33.384281+0000","last_fullsized":"2024-09-12T03:16:33.384281+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:39:49.575672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:32.578281+0000","last_change":"2024-09-12T03:16:30.574066+0000","last_active":"2024-09-12T03:16:32.578281+0000","last_peered":"2024-09-12T03:16:32.578281+0000","last_clean":"2024-09-12T03:16:32.578281+0000","last_became_active":"2024-09-12T03:16:30.573756+0000","last_became_peered":"2024-09-12T03:16:30.573756+0000","last_unstale":"2024-09-12T03:16:32.578281+0000","last_undegraded":"2024-09-12T03:16:32.578281+0000","last_fullsized":"2024-09-12T03:16:32.578281+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:19:17.746405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:32.580747+0000","last_change":"2024-09-12T03:16:31.583553+0000","last_active":"2024-09-12T03:16:32.580747+0000","last_peered":"2024-09-12T03:16:32.580747+0000","last_clean":"2024-09-12T03:16:32.580747+0000","last_became_active":"2024-09-12T03:16:31.582827+0000","last_became_peered":"2024-09-12T03:16:31.582827+0000","last_unstale":"2024-09-12T03:16:32.580747+0000","last_undegraded":"2024-09-12T03:16:32.580747+0000","last_fullsized":"2024-09-12T03:16:32.580747+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:54:20.264625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:33.341655+0000","last_change":"2024-09-12T03:16:30.573483+0000","last_active":"2024-09-12T03:16:33.341655+0000","last_peered":"2024-09-12T03:16:33.341655+0000","last_clean":"2024-09-12T03:16:33.341655+0000","last_became_active":"2024-09-12T03:16:30.573180+0000","last_became_peered":"2024-09-12T03:16:30.573180+0000","last_unstale":"2024-09-12T03:16:33.341655+0000","last_undegraded":"2024-09-12T03:16:33.341655+0000","last_fullsized":"2024-09-12T03:16:33.341655+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:13:33.665229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:33.383685+0000","last_change":"2024-09-12T03:16:31.590008+0000","last_active":"2024-09-12T03:16:33.383685+0000","last_peered":"2024-09-12T03:16:33.383685+0000","last_clean":"2024-09-12T03:16:33.383685+0000","last_became_active":"2024-09-12T03:16:31.589738+0000","last_became_peered":"2024-09-12T03:16:31.589738+0000","last_unstale":"2024-09-12T03:16:33.383685+0000","last_undegraded":"2024-09-12T03:16:33.383685+0000","last_fullsized":"2024-09-12T03:16:33.383685+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:31:17.482655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":170,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:33.384272+0000","last_change":"2024-09-12T03:16:21.883446+0000","last_active":"2024-09-12T03:16:33.384272+0000","last_peered":"2024-09-12T03:16:33.384272+0000","last_clean":"2024-09-12T03:16:33.384272+0000","last_became_active":"2024-09-12T03:16:21.882207+0000","last_became_peered":"2024-09-12T03:16:21.882207+0000","last_unstale":"2024-09-12T03:16:33.384272+0000","last_undegraded":"2024-09-12T03:16:33.384272+0000","last_fullsized":"2024-09-12T03:16:33.384272+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:20.316426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:20.316426+0000","last_clean_scrub_stamp":"2024-09-12T03:16:20.316426+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:12:15.424320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:32.579477+0000","last_change":"2024-09-12T03:16:30.573340+0000","last_active":"2024-09-12T03:16:32.579477+0000","last_peered":"2024-09-12T03:16:32.579477+0000","last_clean":"2024-09-12T03:16:32.579477+0000","last_became_active":"2024-09-12T03:16:30.572756+0000","last_became_peered":"2024-09-12T03:16:30.572756+0000","last_unstale":"2024-09-12T03:16:32.579477+0000","last_undegraded":"2024-09-12T03:16:32.579477+0000","last_fullsized":"2024-09-12T03:16:32.579477+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:21:08.051836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:33.383593+0000","last_change":"2024-09-12T03:16:31.592850+0000","last_active":"2024-09-12T03:16:33.383593+0000","last_peered":"2024-09-12T03:16:33.383593+0000","last_clean":"2024-09-12T03:16:33.383593+0000","last_became_active":"2024-09-12T03:16:31.592505+0000","last_became_peered":"2024-09-12T03:16:31.592505+0000","last_unstale":"2024-09-12T03:16:33.383593+0000","last_undegraded":"2024-09-12T03:16:33.383593+0000","last_fullsized":"2024-09-12T03:16:33.383593+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:23:22.497643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:33.384154+0000","last_change":"2024-09-12T03:16:30.584842+0000","last_active":"2024-09-12T03:16:33.384154+0000","last_peered":"2024-09-12T03:16:33.384154+0000","last_clean":"2024-09-12T03:16:33.384154+0000","last_became_active":"2024-09-12T03:16:30.584516+0000","last_became_peered":"2024-09-12T03:16:30.584516+0000","last_unstale":"2024-09-12T03:16:33.384154+0000","last_undegraded":"2024-09-12T03:16:33.384154+0000","last_fullsized":"2024-09-12T03:16:33.384154+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:44:15.389117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:33.383582+0000","last_change":"2024-09-12T03:16:31.570586+0000","last_active":"2024-09-12T03:16:33.383582+0000","last_peered":"2024-09-12T03:16:33.383582+0000","last_clean":"2024-09-12T03:16:33.383582+0000","last_became_active":"2024-09-12T03:16:31.570210+0000","last_became_peered":"2024-09-12T03:16:31.570210+0000","last_unstale":"2024-09-12T03:16:33.383582+0000","last_undegraded":"2024-09-12T03:16:33.383582+0000","last_fullsized":"2024-09-12T03:16:33.383582+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:12:47.905772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:33.341721+0000","last_change":"2024-09-12T03:16:30.569374+0000","last_active":"2024-09-12T03:16:33.341721+0000","last_peered":"2024-09-12T03:16:33.341721+0000","last_clean":"2024-09-12T03:16:33.341721+0000","last_became_active":"2024-09-12T03:16:30.569053+0000","last_became_peered":"2024-09-12T03:16:30.569053+0000","last_unstale":"2024-09-12T03:16:33.341721+0000","last_undegraded":"2024-09-12T03:16:33.341721+0000","last_fullsized":"2024-09-12T03:16:33.341721+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:44:10.172361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:32.578285+0000","last_change":"2024-09-12T03:16:31.587841+0000","last_active":"2024-09-12T03:16:32.578285+0000","last_peered":"2024-09-12T03:16:32.578285+0000","last_clean":"2024-09-12T03:16:32.578285+0000","last_became_active":"2024-09-12T03:16:31.587555+0000","last_became_peered":"2024-09-12T03:16:31.587555+0000","last_unstale":"2024-09-12T03:16:32.578285+0000","last_undegraded":"2024-09-12T03:16:32.578285+0000","last_fullsized":"2024-09-12T03:16:32.578285+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:02:33.249591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:32.580124+0000","last_change":"2024-09-12T03:16:30.576697+0000","last_active":"2024-09-12T03:16:32.580124+0000","last_peered":"2024-09-12T03:16:32.580124+0000","last_clean":"2024-09-12T03:16:32.580124+0000","last_became_active":"2024-09-12T03:16:30.574772+0000","last_became_peered":"2024-09-12T03:16:30.574772+0000","last_unstale":"2024-09-12T03:16:32.580124+0000","last_undegraded":"2024-09-12T03:16:32.580124+0000","last_fullsized":"2024-09-12T03:16:32.580124+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:36:06.512797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:32.580280+0000","last_change":"2024-09-12T03:16:31.581253+0000","last_active":"2024-09-12T03:16:32.580280+0000","last_peered":"2024-09-12T03:16:32.580280+0000","last_clean":"2024-09-12T03:16:32.580280+0000","last_became_active":"2024-09-12T03:16:31.580992+0000","last_became_peered":"2024-09-12T03:16:31.580992+0000","last_unstale":"2024-09-12T03:16:32.580280+0000","last_undegraded":"2024-09-12T03:16:32.580280+0000","last_fullsized":"2024-09-12T03:16:32.580280+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:23:14.503630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:32.578229+0000","last_change":"2024-09-12T03:16:30.580488+0000","last_active":"2024-09-12T03:16:32.578229+0000","last_peered":"2024-09-12T03:16:32.578229+0000","last_clean":"2024-09-12T03:16:32.578229+0000","last_became_active":"2024-09-12T03:16:30.580231+0000","last_became_peered":"2024-09-12T03:16:30.580231+0000","last_unstale":"2024-09-12T03:16:32.578229+0000","last_undegraded":"2024-09-12T03:16:32.578229+0000","last_fullsized":"2024-09-12T03:16:32.578229+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:55:03.313892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:32.580115+0000","last_change":"2024-09-12T03:16:31.582097+0000","last_active":"2024-09-12T03:16:32.580115+0000","last_peered":"2024-09-12T03:16:32.580115+0000","last_clean":"2024-09-12T03:16:32.580115+0000","last_became_active":"2024-09-12T03:16:31.580258+0000","last_became_peered":"2024-09-12T03:16:31.580258+0000","last_unstale":"2024-09-12T03:16:32.580115+0000","last_undegraded":"2024-09-12T03:16:32.580115+0000","last_fullsized":"2024-09-12T03:16:32.580115+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:13:00.761714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:33.384050+0000","last_change":"2024-09-12T03:16:30.586287+0000","last_active":"2024-09-12T03:16:33.384050+0000","last_peered":"2024-09-12T03:16:33.384050+0000","last_clean":"2024-09-12T03:16:33.384050+0000","last_became_active":"2024-09-12T03:16:30.585919+0000","last_became_peered":"2024-09-12T03:16:30.585919+0000","last_unstale":"2024-09-12T03:16:33.384050+0000","last_undegraded":"2024-09-12T03:16:33.384050+0000","last_fullsized":"2024-09-12T03:16:33.384050+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:04:51.773994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:32.578331+0000","last_change":"2024-09-12T03:16:31.587128+0000","last_active":"2024-09-12T03:16:32.578331+0000","last_peered":"2024-09-12T03:16:32.578331+0000","last_clean":"2024-09-12T03:16:32.578331+0000","last_became_active":"2024-09-12T03:16:31.586831+0000","last_became_peered":"2024-09-12T03:16:31.586831+0000","last_unstale":"2024-09-12T03:16:32.578331+0000","last_undegraded":"2024-09-12T03:16:32.578331+0000","last_fullsized":"2024-09-12T03:16:32.578331+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:59:21.255153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:32.578379+0000","last_change":"2024-09-12T03:16:30.577847+0000","last_active":"2024-09-12T03:16:32.578379+0000","last_peered":"2024-09-12T03:16:32.578379+0000","last_clean":"2024-09-12T03:16:32.578379+0000","last_became_active":"2024-09-12T03:16:30.577552+0000","last_became_peered":"2024-09-12T03:16:30.577552+0000","last_unstale":"2024-09-12T03:16:32.578379+0000","last_undegraded":"2024-09-12T03:16:32.578379+0000","last_fullsized":"2024-09-12T03:16:32.578379+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:36:06.512797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:33.383991+0000","last_change":"2024-09-12T03:16:31.586339+0000","last_active":"2024-09-12T03:16:33.383991+0000","last_peered":"2024-09-12T03:16:33.383991+0000","last_clean":"2024-09-12T03:16:33.383991+0000","last_became_active":"2024-09-12T03:16:31.586009+0000","last_became_peered":"2024-09-12T03:16:31.586009+0000","last_unstale":"2024-09-12T03:16:33.383991+0000","last_undegraded":"2024-09-12T03:16:33.383991+0000","last_fullsized":"2024-09-12T03:16:33.383991+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:35:22.374781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:33.384683+0000","last_change":"2024-09-12T03:16:30.583723+0000","last_active":"2024-09-12T03:16:33.384683+0000","last_peered":"2024-09-12T03:16:33.384683+0000","last_clean":"2024-09-12T03:16:33.384683+0000","last_became_active":"2024-09-12T03:16:30.582911+0000","last_became_peered":"2024-09-12T03:16:30.582911+0000","last_unstale":"2024-09-12T03:16:33.384683+0000","last_undegraded":"2024-09-12T03:16:33.384683+0000","last_fullsized":"2024-09-12T03:16:33.384683+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:23:13.498541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:33.384195+0000","last_change":"2024-09-12T03:16:31.589322+0000","last_active":"2024-09-12T03:16:33.384195+0000","last_peered":"2024-09-12T03:16:33.384195+0000","last_clean":"2024-09-12T03:16:33.384195+0000","last_became_active":"2024-09-12T03:16:31.588464+0000","last_became_peered":"2024-09-12T03:16:31.588464+0000","last_unstale":"2024-09-12T03:16:33.384195+0000","last_undegraded":"2024-09-12T03:16:33.384195+0000","last_fullsized":"2024-09-12T03:16:33.384195+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:46:08.748138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:33.383860+0000","last_change":"2024-09-12T03:16:30.583438+0000","last_active":"2024-09-12T03:16:33.383860+0000","last_peered":"2024-09-12T03:16:33.383860+0000","last_clean":"2024-09-12T03:16:33.383860+0000","last_became_active":"2024-09-12T03:16:30.583067+0000","last_became_peered":"2024-09-12T03:16:30.583067+0000","last_unstale":"2024-09-12T03:16:33.383860+0000","last_undegraded":"2024-09-12T03:16:33.383860+0000","last_fullsized":"2024-09-12T03:16:33.383860+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:54:16.599711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:32.581034+0000","last_change":"2024-09-12T03:16:31.578365+0000","last_active":"2024-09-12T03:16:32.581034+0000","last_peered":"2024-09-12T03:16:32.581034+0000","last_clean":"2024-09-12T03:16:32.581034+0000","last_became_active":"2024-09-12T03:16:31.577424+0000","last_became_peered":"2024-09-12T03:16:31.577424+0000","last_unstale":"2024-09-12T03:16:32.581034+0000","last_undegraded":"2024-09-12T03:16:32.581034+0000","last_fullsized":"2024-09-12T03:16:32.581034+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:49:09.388955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:32.578103+0000","last_change":"2024-09-12T03:16:30.574028+0000","last_active":"2024-09-12T03:16:32.578103+0000","last_peered":"2024-09-12T03:16:32.578103+0000","last_clean":"2024-09-12T03:16:32.578103+0000","last_became_active":"2024-09-12T03:16:30.573742+0000","last_became_peered":"2024-09-12T03:16:30.573742+0000","last_unstale":"2024-09-12T03:16:32.578103+0000","last_undegraded":"2024-09-12T03:16:32.578103+0000","last_fullsized":"2024-09-12T03:16:32.578103+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:19:07.178848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:33.383373+0000","last_change":"2024-09-12T03:16:31.584222+0000","last_active":"2024-09-12T03:16:33.383373+0000","last_peered":"2024-09-12T03:16:33.383373+0000","last_clean":"2024-09-12T03:16:33.383373+0000","last_became_active":"2024-09-12T03:16:31.583947+0000","last_became_peered":"2024-09-12T03:16:31.583947+0000","last_unstale":"2024-09-12T03:16:33.383373+0000","last_undegraded":"2024-09-12T03:16:33.383373+0000","last_fullsized":"2024-09-12T03:16:33.383373+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:27:31.919800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:32.578127+0000","last_change":"2024-09-12T03:16:30.572123+0000","last_active":"2024-09-12T03:16:32.578127+0000","last_peered":"2024-09-12T03:16:32.578127+0000","last_clean":"2024-09-12T03:16:32.578127+0000","last_became_active":"2024-09-12T03:16:30.571600+0000","last_became_peered":"2024-09-12T03:16:30.571600+0000","last_unstale":"2024-09-12T03:16:32.578127+0000","last_undegraded":"2024-09-12T03:16:32.578127+0000","last_fullsized":"2024-09-12T03:16:32.578127+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:25:21.686984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:33.383482+0000","last_change":"2024-09-12T03:16:31.581216+0000","last_active":"2024-09-12T03:16:33.383482+0000","last_peered":"2024-09-12T03:16:33.383482+0000","last_clean":"2024-09-12T03:16:33.383482+0000","last_became_active":"2024-09-12T03:16:31.580663+0000","last_became_peered":"2024-09-12T03:16:31.580663+0000","last_unstale":"2024-09-12T03:16:33.383482+0000","last_undegraded":"2024-09-12T03:16:33.383482+0000","last_fullsized":"2024-09-12T03:16:33.383482+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:05:13.773162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:33.384021+0000","last_change":"2024-09-12T03:16:30.584019+0000","last_active":"2024-09-12T03:16:33.384021+0000","last_peered":"2024-09-12T03:16:33.384021+0000","last_clean":"2024-09-12T03:16:33.384021+0000","last_became_active":"2024-09-12T03:16:30.583213+0000","last_became_peered":"2024-09-12T03:16:30.583213+0000","last_unstale":"2024-09-12T03:16:33.384021+0000","last_undegraded":"2024-09-12T03:16:33.384021+0000","last_fullsized":"2024-09-12T03:16:33.384021+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:14:33.816306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:33.383420+0000","last_change":"2024-09-12T03:16:31.590094+0000","last_active":"2024-09-12T03:16:33.383420+0000","last_peered":"2024-09-12T03:16:33.383420+0000","last_clean":"2024-09-12T03:16:33.383420+0000","last_became_active":"2024-09-12T03:16:31.589818+0000","last_became_peered":"2024-09-12T03:16:31.589818+0000","last_unstale":"2024-09-12T03:16:33.383420+0000","last_undegraded":"2024-09-12T03:16:33.383420+0000","last_fullsized":"2024-09-12T03:16:33.383420+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:12:47.905772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:32.580177+0000","last_change":"2024-09-12T03:16:31.585182+0000","last_active":"2024-09-12T03:16:32.580177+0000","last_peered":"2024-09-12T03:16:32.580177+0000","last_clean":"2024-09-12T03:16:32.580177+0000","last_became_active":"2024-09-12T03:16:31.584871+0000","last_became_peered":"2024-09-12T03:16:31.584871+0000","last_unstale":"2024-09-12T03:16:32.580177+0000","last_undegraded":"2024-09-12T03:16:32.580177+0000","last_fullsized":"2024-09-12T03:16:32.580177+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:48:52.581642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:33.383721+0000","last_change":"2024-09-12T03:16:30.583526+0000","last_active":"2024-09-12T03:16:33.383721+0000","last_peered":"2024-09-12T03:16:33.383721+0000","last_clean":"2024-09-12T03:16:33.383721+0000","last_became_active":"2024-09-12T03:16:30.583166+0000","last_became_peered":"2024-09-12T03:16:30.583166+0000","last_unstale":"2024-09-12T03:16:33.383721+0000","last_undegraded":"2024-09-12T03:16:33.383721+0000","last_fullsized":"2024-09-12T03:16:33.383721+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:19:42.146116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:32.580666+0000","last_change":"2024-09-12T03:16:31.585324+0000","last_active":"2024-09-12T03:16:32.580666+0000","last_peered":"2024-09-12T03:16:32.580666+0000","last_clean":"2024-09-12T03:16:32.580666+0000","last_became_active":"2024-09-12T03:16:31.584963+0000","last_became_peered":"2024-09-12T03:16:31.584963+0000","last_unstale":"2024-09-12T03:16:32.580666+0000","last_undegraded":"2024-09-12T03:16:32.580666+0000","last_fullsized":"2024-09-12T03:16:32.580666+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:34:35.319063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:33.383711+0000","last_change":"2024-09-12T03:16:30.583739+0000","last_active":"2024-09-12T03:16:33.383711+0000","last_peered":"2024-09-12T03:16:33.383711+0000","last_clean":"2024-09-12T03:16:33.383711+0000","last_became_active":"2024-09-12T03:16:30.583397+0000","last_became_peered":"2024-09-12T03:16:30.583397+0000","last_unstale":"2024-09-12T03:16:33.383711+0000","last_undegraded":"2024-09-12T03:16:33.383711+0000","last_fullsized":"2024-09-12T03:16:33.383711+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:49:08.383866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:33.384172+0000","last_change":"2024-09-12T03:16:31.583861+0000","last_active":"2024-09-12T03:16:33.384172+0000","last_peered":"2024-09-12T03:16:33.384172+0000","last_clean":"2024-09-12T03:16:33.384172+0000","last_became_active":"2024-09-12T03:16:31.583134+0000","last_became_peered":"2024-09-12T03:16:31.583134+0000","last_unstale":"2024-09-12T03:16:33.384172+0000","last_undegraded":"2024-09-12T03:16:33.384172+0000","last_fullsized":"2024-09-12T03:16:33.384172+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:05:13.773162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:33.383731+0000","last_change":"2024-09-12T03:16:30.580413+0000","last_active":"2024-09-12T03:16:33.383731+0000","last_peered":"2024-09-12T03:16:33.383731+0000","last_clean":"2024-09-12T03:16:33.383731+0000","last_became_active":"2024-09-12T03:16:30.580133+0000","last_became_peered":"2024-09-12T03:16:30.580133+0000","last_unstale":"2024-09-12T03:16:33.383731+0000","last_undegraded":"2024-09-12T03:16:33.383731+0000","last_fullsized":"2024-09-12T03:16:33.383731+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:12:12.227072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:33.383784+0000","last_change":"2024-09-12T03:16:31.580830+0000","last_active":"2024-09-12T03:16:33.383784+0000","last_peered":"2024-09-12T03:16:33.383784+0000","last_clean":"2024-09-12T03:16:33.383784+0000","last_became_active":"2024-09-12T03:16:31.580548+0000","last_became_peered":"2024-09-12T03:16:31.580548+0000","last_unstale":"2024-09-12T03:16:33.383784+0000","last_undegraded":"2024-09-12T03:16:33.383784+0000","last_fullsized":"2024-09-12T03:16:33.383784+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:30.553126+0000","last_clean_scrub_stamp":"2024-09-12T03:16:30.553126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:46:08.748138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"20'4","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T03:16:33.384309+0000","last_change":"2024-09-12T03:16:30.584481+0000","last_active":"2024-09-12T03:16:33.384309+0000","last_peered":"2024-09-12T03:16:33.384309+0000","last_clean":"2024-09-12T03:16:33.384309+0000","last_became_active":"2024-09-12T03:16:30.584234+0000","last_became_peered":"2024-09-12T03:16:30.584234+0000","last_unstale":"2024-09-12T03:16:33.384309+0000","last_undegraded":"2024-09-12T03:16:33.384309+0000","last_fullsized":"2024-09-12T03:16:33.384309+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:29.548038+0000","last_clean_scrub_stamp":"2024-09-12T03:16:29.548038+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:29:14.889397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542150,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31980,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339860,"statfs":{"total":96636764160,"available":96604016640,"internally_reserved":0,"allocated":811008,"data_stored":665113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542152,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31408,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340432,"statfs":{"total":96636764160,"available":96604602368,"internally_reserved":0,"allocated":225280,"data_stored":75775,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542151,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31404,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340436,"statfs":{"total":96636764160,"available":96604606464,"internally_reserved":0,"allocated":217088,"data_stored":74253,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542150,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31408,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340432,"statfs":{"total":96636764160,"available":96604602368,"internally_reserved":0,"allocated":225280,"data_stored":74289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542151,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":225280,"data_stored":75775,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542151,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":212992,"data_stored":74163,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542152,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":221184,"data_stored":74323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542152,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27828,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344012,"statfs":{"total":96636764160,"available":96608268288,"internally_reserved":0,"allocated":819200,"data_stored":665169,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T03:16:43.659 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-12T03:16:43.660 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-12T03:16:43.660 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-09-12T03:16:43.660 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-12T03:16:43.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.796+0000 7f098a8a4640 1 Processor -- start 2024-09-12T03:16:43.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.796+0000 7f098a8a4640 1 -- start start 2024-09-12T03:16:43.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.797+0000 7f098a8a4640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f098415e5f0 0x7f0984073250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:43.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.797+0000 7f098a8a4640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f09840d2240 con 0x7f0984073790 2024-09-12T03:16:43.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.797+0000 7f098a8a4640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0984163090 con 0x7f0984162a30 2024-09-12T03:16:43.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.797+0000 7f098a8a4640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f0984162e60 con 0x7f098415e5f0 2024-09-12T03:16:43.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.797+0000 7f0983fff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f098415e5f0 0x7f0984073250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:43.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.797+0000 7f09837fe640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f0984073790 0x7f0984162320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:39256/0 (socket says 172.21.15.27:39256) 2024-09-12T03:16:43.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.797+0000 7f09837fe640 1 -- 172.21.15.27:0/682363037 learned_addr learned my addr 172.21.15.27:0/682363037 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:16:43.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.797+0000 7f0983fff640 1 -- 172.21.15.27:0/682363037 >> v1:172.21.15.45:6789/0 conn(0x7f0984162a30 legacy=0x7f098415e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:43.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.797+0000 7f0983fff640 1 -- 172.21.15.27:0/682363037 >> v1:172.21.15.27:6789/0 conn(0x7f0984073790 legacy=0x7f0984162320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:43.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.797+0000 7f0983fff640 1 -- 172.21.15.27:0/682363037 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09840d0470 con 0x7f098415e5f0 2024-09-12T03:16:43.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.797+0000 7f0982ffd640 1 -- 172.21.15.27:0/682363037 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2884568124 0 0) 0x7f09840d2240 con 0x7f0984073790 2024-09-12T03:16:43.801 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.798+0000 7f0983fff640 1 --2- 172.21.15.27:0/682363037 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f098415e5f0 0x7f0984073250 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f09740097a0 tx=0x7f097402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=374c39337d4e9f68 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:43.801 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.798+0000 7f0982ffd640 1 -- 172.21.15.27:0/682363037 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0974037070 con 0x7f098415e5f0 2024-09-12T03:16:43.801 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.798+0000 7f0982ffd640 1 -- 172.21.15.27:0/682363037 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0974032440 con 0x7f098415e5f0 2024-09-12T03:16:43.801 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.798+0000 7f0982ffd640 1 -- 172.21.15.27:0/682363037 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0974032810 con 0x7f098415e5f0 2024-09-12T03:16:43.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.799+0000 7f098a8a4640 1 -- 172.21.15.27:0/682363037 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f098415e5f0 msgr2=0x7f0984073250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:43.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.799+0000 7f098a8a4640 1 --2- 172.21.15.27:0/682363037 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f098415e5f0 0x7f0984073250 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f09740097a0 tx=0x7f097402a1e0 comp rx=0 tx=0).stop 2024-09-12T03:16:43.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.799+0000 7f098a8a4640 1 -- 172.21.15.27:0/682363037 shutdown_connections 2024-09-12T03:16:43.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.799+0000 7f098a8a4640 1 --2- 172.21.15.27:0/682363037 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f098415e5f0 0x7f0984073250 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:43.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.799+0000 7f098a8a4640 1 -- 172.21.15.27:0/682363037 >> 172.21.15.27:0/682363037 conn(0x7f098406f300 msgr2=0x7f098406f710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:43.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.799+0000 7f098a8a4640 1 -- 172.21.15.27:0/682363037 shutdown_connections 2024-09-12T03:16:43.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.799+0000 7f098a8a4640 1 -- 172.21.15.27:0/682363037 wait complete. 2024-09-12T03:16:43.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.800+0000 7f098a8a4640 1 Processor -- start 2024-09-12T03:16:43.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.800+0000 7f098a8a4640 1 -- start start 2024-09-12T03:16:43.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.801+0000 7f098a8a4640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0984073790 0x7f0984148e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:43.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.801+0000 7f098a8a4640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f098415e5f0 0x7f098415cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:43.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.801+0000 7f098a8a4640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0984162a30 0x7f098415d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:43.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.801+0000 7f098a8a4640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f09840d0da0 con 0x7f0984162a30 2024-09-12T03:16:43.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.801+0000 7f098a8a4640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f0984073be0 con 0x7f0984073790 2024-09-12T03:16:43.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.801+0000 7f098a8a4640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f09840d0a30 con 0x7f098415e5f0 2024-09-12T03:16:43.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.801+0000 7f0988e41640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0984162a30 0x7f098415d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:43.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.801+0000 7f0988e41640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0984162a30 0x7f098415d2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33482/0 (socket says 172.21.15.27:33482) 2024-09-12T03:16:43.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.801+0000 7f0988e41640 1 -- 172.21.15.27:0/1440448406 learned_addr learned my addr 172.21.15.27:0/1440448406 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:16:43.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.801+0000 7f0983fff640 1 --2- 172.21.15.27:0/1440448406 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0984073790 0x7f0984148e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:43.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.801+0000 7f09837fe640 1 --2- 172.21.15.27:0/1440448406 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f098415e5f0 0x7f098415cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:43.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.801+0000 7f0988e41640 1 -- 172.21.15.27:0/1440448406 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f098415e5f0 msgr2=0x7f098415cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:43.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.801+0000 7f0988e41640 1 --2- 172.21.15.27:0/1440448406 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f098415e5f0 0x7f098415cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:43.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.801+0000 7f0988e41640 1 -- 172.21.15.27:0/1440448406 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0984073790 msgr2=0x7f0984148e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:43.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.801+0000 7f0988e41640 1 --2- 172.21.15.27:0/1440448406 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0984073790 0x7f0984148e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:43.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.801+0000 7f0988e41640 1 -- 172.21.15.27:0/1440448406 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f098415b430 con 0x7f0984162a30 2024-09-12T03:16:43.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.801+0000 7f0983fff640 1 --2- 172.21.15.27:0/1440448406 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0984073790 0x7f0984148e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:16:43.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.801+0000 7f0988e41640 1 --2- 172.21.15.27:0/1440448406 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0984162a30 0x7f098415d2c0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f097c00b970 tx=0x7f097c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:43.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.802+0000 7f0980ff9640 1 -- 172.21.15.27:0/1440448406 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f097c019070 con 0x7f0984162a30 2024-09-12T03:16:43.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.802+0000 7f0980ff9640 1 -- 172.21.15.27:0/1440448406 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f097c00cb40 con 0x7f0984162a30 2024-09-12T03:16:43.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.802+0000 7f098a8a4640 1 -- 172.21.15.27:0/1440448406 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f098415b710 con 0x7f0984162a30 2024-09-12T03:16:43.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.802+0000 7f0980ff9640 1 -- 172.21.15.27:0/1440448406 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f097c014440 con 0x7f0984162a30 2024-09-12T03:16:43.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.802+0000 7f098a8a4640 1 -- 172.21.15.27:0/1440448406 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f098415bcd0 con 0x7f0984162a30 2024-09-12T03:16:43.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.803+0000 7f098a8a4640 1 -- 172.21.15.27:0/1440448406 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0984159eb0 con 0x7f0984162a30 2024-09-12T03:16:43.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.803+0000 7f0980ff9640 1 -- 172.21.15.27:0/1440448406 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f097c01c070 con 0x7f0984162a30 2024-09-12T03:16:43.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.804+0000 7f0980ff9640 1 --2- 172.21.15.27:0/1440448406 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f097006be30 0x7f097006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:43.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.804+0000 7f0980ff9640 1 -- 172.21.15.27:0/1440448406 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f097c08aa30 con 0x7f0984162a30 2024-09-12T03:16:43.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.804+0000 7f0983fff640 1 --2- 172.21.15.27:0/1440448406 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f097006be30 0x7f097006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:43.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.805+0000 7f0983fff640 1 --2- 172.21.15.27:0/1440448406 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f097006be30 0x7f097006e2f0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f0974034000 tx=0x7f097402e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:43.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:43.808+0000 7f0980ff9640 1 -- 172.21.15.27:0/1440448406 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f097c059470 con 0x7f0984162a30 2024-09-12T03:16:44.021 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.020+0000 7f098a8a4640 1 -- 172.21.15.27:0/1440448406 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f09841598f0 con 0x7f0984162a30 2024-09-12T03:16:44.021 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.020+0000 7f0980ff9640 1 -- 172.21.15.27:0/1440448406 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f097c058d70 con 0x7f0984162a30 2024-09-12T03:16:44.021 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:16:44.021 INFO:teuthology.orchestra.run.smithi027.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-12T03:16:44.023 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.022+0000 7f098a8a4640 1 -- 172.21.15.27:0/1440448406 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f097006be30 msgr2=0x7f097006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:44.023 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.022+0000 7f098a8a4640 1 --2- 172.21.15.27:0/1440448406 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f097006be30 0x7f097006e2f0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f0974034000 tx=0x7f097402e000 comp rx=0 tx=0).stop 2024-09-12T03:16:44.023 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.022+0000 7f098a8a4640 1 -- 172.21.15.27:0/1440448406 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0984162a30 msgr2=0x7f098415d2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:44.023 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.022+0000 7f098a8a4640 1 --2- 172.21.15.27:0/1440448406 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0984162a30 0x7f098415d2c0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f097c00b970 tx=0x7f097c00be30 comp rx=0 tx=0).stop 2024-09-12T03:16:44.023 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.022+0000 7f098a8a4640 1 -- 172.21.15.27:0/1440448406 shutdown_connections 2024-09-12T03:16:44.024 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.022+0000 7f098a8a4640 1 --2- 172.21.15.27:0/1440448406 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f097006be30 0x7f097006e2f0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:44.024 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.022+0000 7f098a8a4640 1 --2- 172.21.15.27:0/1440448406 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0984162a30 0x7f098415d2c0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:44.024 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.022+0000 7f098a8a4640 1 --2- 172.21.15.27:0/1440448406 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f098415e5f0 0x7f098415cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:44.024 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.022+0000 7f098a8a4640 1 --2- 172.21.15.27:0/1440448406 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0984073790 0x7f0984148e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:44.024 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.022+0000 7f098a8a4640 1 -- 172.21.15.27:0/1440448406 >> 172.21.15.27:0/1440448406 conn(0x7f098406f300 msgr2=0x7f09840b6c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:44.024 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.022+0000 7f098a8a4640 1 -- 172.21.15.27:0/1440448406 shutdown_connections 2024-09-12T03:16:44.024 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.022+0000 7f098a8a4640 1 -- 172.21.15.27:0/1440448406 wait complete. 2024-09-12T03:16:44.034 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-09-12T03:16:44.034 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-12T03:16:44.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.135+0000 7f83fe70a640 1 Processor -- start 2024-09-12T03:16:44.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.136+0000 7f83fe70a640 1 -- start start 2024-09-12T03:16:44.138 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.136+0000 7f83fe70a640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f83f80d5200 0x7f83f80d1010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:44.138 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.136+0000 7f83fe70a640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f83f80d1550 0x7f83f80b8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:44.138 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.136+0000 7f83fe70a640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f83f80d3760 con 0x7f83f815eb90 2024-09-12T03:16:44.138 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.136+0000 7f83fe70a640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f83f80b7d20 con 0x7f83f80d5200 2024-09-12T03:16:44.138 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.136+0000 7f83fe70a640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f83f80d42e0 con 0x7f83f80d1550 2024-09-12T03:16:44.138 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.136+0000 7f83f7fff640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f83f815eb90 0x7f83f80d2df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:39272/0 (socket says 172.21.15.27:39272) 2024-09-12T03:16:44.138 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.136+0000 7f83f7fff640 1 -- 172.21.15.27:0/3267351533 learned_addr learned my addr 172.21.15.27:0/3267351533 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:16:44.138 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.137+0000 7f83f77fe640 1 --2- 172.21.15.27:0/3267351533 >> v2:172.21.15.27:3300/0 conn(0x7f83f80d5200 0x7f83f80d1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:44.138 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.137+0000 7f83fcca7640 1 --2- 172.21.15.27:0/3267351533 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f83f80d1550 0x7f83f80b8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:44.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.137+0000 7f83f6ffd640 1 -- 172.21.15.27:0/3267351533 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 658600082 0 0) 0x7f83f80d3760 con 0x7f83f815eb90 2024-09-12T03:16:44.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.137+0000 7f83f6ffd640 1 -- 172.21.15.27:0/3267351533 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f83e0003520 con 0x7f83f815eb90 2024-09-12T03:16:44.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.137+0000 7f83f77fe640 1 -- 172.21.15.27:0/3267351533 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f83f80d1550 msgr2=0x7f83f80b8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:44.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.137+0000 7f83f77fe640 1 --2- 172.21.15.27:0/3267351533 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f83f80d1550 0x7f83f80b8190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:44.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.137+0000 7f83f77fe640 1 -- 172.21.15.27:0/3267351533 >> v1:172.21.15.27:6789/0 conn(0x7f83f815eb90 legacy=0x7f83f80d2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:44.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.137+0000 7f83f77fe640 1 -- 172.21.15.27:0/3267351533 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83f80d3a40 con 0x7f83f80d5200 2024-09-12T03:16:44.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.137+0000 7f83fcca7640 1 --2- 172.21.15.27:0/3267351533 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f83f80d1550 0x7f83f80b8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:16:44.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.138+0000 7f83f77fe640 1 --2- 172.21.15.27:0/3267351533 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f83f80d5200 0x7f83f80d1010 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f83ec0097a0 tx=0x7f83ec02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=832ee74c6bed1212 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:44.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.138+0000 7f83f6ffd640 1 -- 172.21.15.27:0/3267351533 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83ec037070 con 0x7f83f80d5200 2024-09-12T03:16:44.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.138+0000 7f83f6ffd640 1 -- 172.21.15.27:0/3267351533 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83ec032440 con 0x7f83f80d5200 2024-09-12T03:16:44.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.138+0000 7f83f6ffd640 1 -- 172.21.15.27:0/3267351533 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83ec0327f0 con 0x7f83f80d5200 2024-09-12T03:16:44.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.138+0000 7f83fe70a640 1 -- 172.21.15.27:0/3267351533 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f83f80d5200 msgr2=0x7f83f80d1010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:44.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.138+0000 7f83fe70a640 1 --2- 172.21.15.27:0/3267351533 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f83f80d5200 0x7f83f80d1010 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f83ec0097a0 tx=0x7f83ec02a180 comp rx=0 tx=0).stop 2024-09-12T03:16:44.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.138+0000 7f83fe70a640 1 -- 172.21.15.27:0/3267351533 shutdown_connections 2024-09-12T03:16:44.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.138+0000 7f83fe70a640 1 --2- 172.21.15.27:0/3267351533 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f83f80d1550 0x7f83f80b8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:44.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.138+0000 7f83fe70a640 1 --2- 172.21.15.27:0/3267351533 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f83f80d5200 0x7f83f80d1010 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:44.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.138+0000 7f83fe70a640 1 -- 172.21.15.27:0/3267351533 >> 172.21.15.27:0/3267351533 conn(0x7f83f806dce0 msgr2=0x7f83f806e0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:44.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.138+0000 7f83fe70a640 1 -- 172.21.15.27:0/3267351533 shutdown_connections 2024-09-12T03:16:44.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.138+0000 7f83fe70a640 1 -- 172.21.15.27:0/3267351533 wait complete. 2024-09-12T03:16:44.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.140+0000 7f83fe70a640 1 Processor -- start 2024-09-12T03:16:44.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.140+0000 7f83fe70a640 1 -- start start 2024-09-12T03:16:44.143 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.140+0000 7f83fe70a640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f83f80d1550 0x7f83f8158c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:44.143 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.140+0000 7f83fe70a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f83f80d5200 0x7f83f80cb100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:44.143 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.140+0000 7f83fe70a640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f83f815eb90 0x7f83f81f4300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:44.143 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.140+0000 7f83fe70a640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f83f80d4000 con 0x7f83f80d1550 2024-09-12T03:16:44.144 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.140+0000 7f83fe70a640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f83f80d3500 con 0x7f83f80d5200 2024-09-12T03:16:44.144 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.140+0000 7f83fe70a640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f83f80d19a0 con 0x7f83f815eb90 2024-09-12T03:16:44.144 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.141+0000 7f83f7fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f83f80d1550 0x7f83f8158c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:44.144 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.141+0000 7f83f7fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f83f80d1550 0x7f83f8158c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33490/0 (socket says 172.21.15.27:33490) 2024-09-12T03:16:44.144 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.141+0000 7f83f7fff640 1 -- 172.21.15.27:0/3638315399 learned_addr learned my addr 172.21.15.27:0/3638315399 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:16:44.144 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.141+0000 7f83fcca7640 1 --2- 172.21.15.27:0/3638315399 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f83f815eb90 0x7f83f81f4300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:44.144 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.141+0000 7f83f77fe640 1 --2- 172.21.15.27:0/3638315399 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f83f80d5200 0x7f83f80cb100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:44.144 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.141+0000 7f83f7fff640 1 -- 172.21.15.27:0/3638315399 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f83f815eb90 msgr2=0x7f83f81f4300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:44.144 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.141+0000 7f83f7fff640 1 --2- 172.21.15.27:0/3638315399 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f83f815eb90 0x7f83f81f4300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:44.144 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.141+0000 7f83f7fff640 1 -- 172.21.15.27:0/3638315399 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f83f80d5200 msgr2=0x7f83f80cb100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:44.144 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.141+0000 7f83f7fff640 1 --2- 172.21.15.27:0/3638315399 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f83f80d5200 0x7f83f80cb100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:44.144 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.141+0000 7f83f7fff640 1 -- 172.21.15.27:0/3638315399 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83f80cb640 con 0x7f83f80d1550 2024-09-12T03:16:44.145 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.141+0000 7f83f7fff640 1 --2- 172.21.15.27:0/3638315399 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f83f80d1550 0x7f83f8158c10 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f83e800c390 tx=0x7f83e800c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:44.145 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.141+0000 7f83f4ff9640 1 -- 172.21.15.27:0/3638315399 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83e8017070 con 0x7f83f80d1550 2024-09-12T03:16:44.145 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.141+0000 7f83f4ff9640 1 -- 172.21.15.27:0/3638315399 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83e8005ce0 con 0x7f83f80d1550 2024-09-12T03:16:44.145 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.141+0000 7f83fe70a640 1 -- 172.21.15.27:0/3638315399 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83f81592a0 con 0x7f83f80d1550 2024-09-12T03:16:44.145 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.141+0000 7f83f4ff9640 1 -- 172.21.15.27:0/3638315399 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83e80072f0 con 0x7f83f80d1550 2024-09-12T03:16:44.145 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.141+0000 7f83fe70a640 1 -- 172.21.15.27:0/3638315399 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83f8159860 con 0x7f83f80d1550 2024-09-12T03:16:44.145 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.142+0000 7f83fe70a640 1 -- 172.21.15.27:0/3638315399 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83f8159aa0 con 0x7f83f80d1550 2024-09-12T03:16:44.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.143+0000 7f83f4ff9640 1 -- 172.21.15.27:0/3638315399 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f83e801a070 con 0x7f83f80d1550 2024-09-12T03:16:44.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.143+0000 7f83f4ff9640 1 --2- 172.21.15.27:0/3638315399 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f83e406be30 0x7f83e406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:44.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.143+0000 7f83f4ff9640 1 -- 172.21.15.27:0/3638315399 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f83e808a1f0 con 0x7f83f80d1550 2024-09-12T03:16:44.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.144+0000 7f83f77fe640 1 --2- 172.21.15.27:0/3638315399 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f83e406be30 0x7f83e406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:44.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.144+0000 7f83f77fe640 1 --2- 172.21.15.27:0/3638315399 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f83e406be30 0x7f83e406e2f0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f83ec034000 tx=0x7f83ec02e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:44.148 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.148+0000 7f83f4ff9640 1 -- 172.21.15.27:0/3638315399 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f83f8159aa0 con 0x7f83f80d1550 2024-09-12T03:16:44.359 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.358+0000 7f83fe70a640 1 -- 172.21.15.27:0/3638315399 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f83f81601d0 con 0x7f83f80d1550 2024-09-12T03:16:44.360 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.359+0000 7f83f4ff9640 1 -- 172.21.15.27:0/3638315399 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+5214 (secure 0 0 0) 0x7f83e8058550 con 0x7f83f80d1550 2024-09-12T03:16:44.360 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:16:44.360 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":6,"btime":"2024-09-12T03:16:33:576418+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6836/682326693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":682326693},{"type":"v1","addr":"172.21.15.45:6836","nonce":682326693}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4278,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6834/4084728235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":4084728235},{"type":"v1","addr":"172.21.15.27:6834","nonce":4084728235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4281,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/2844240887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":2844240887},{"type":"v1","addr":"172.21.15.27:6835","nonce":2844240887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-12T03:16:30.914517+0000","modified":"2024-09-12T03:16:32.913742+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.45:6837/1035926332","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6835","nonce":1035926332},{"type":"v1","addr":"172.21.15.45:6837","nonce":1035926332}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4285,"qdb_cluster":[4285]},"id":1}]} 2024-09-12T03:16:44.361 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 6 2024-09-12T03:16:44.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.361+0000 7f83fe70a640 1 -- 172.21.15.27:0/3638315399 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f83e406be30 msgr2=0x7f83e406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:44.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.361+0000 7f83fe70a640 1 --2- 172.21.15.27:0/3638315399 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f83e406be30 0x7f83e406e2f0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f83ec034000 tx=0x7f83ec02e000 comp rx=0 tx=0).stop 2024-09-12T03:16:44.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.361+0000 7f83fe70a640 1 -- 172.21.15.27:0/3638315399 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f83f80d1550 msgr2=0x7f83f8158c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:44.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.361+0000 7f83fe70a640 1 --2- 172.21.15.27:0/3638315399 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f83f80d1550 0x7f83f8158c10 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f83e800c390 tx=0x7f83e800c850 comp rx=0 tx=0).stop 2024-09-12T03:16:44.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.361+0000 7f83fe70a640 1 -- 172.21.15.27:0/3638315399 shutdown_connections 2024-09-12T03:16:44.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.361+0000 7f83fe70a640 1 --2- 172.21.15.27:0/3638315399 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f83e406be30 0x7f83e406e2f0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:44.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.361+0000 7f83fe70a640 1 --2- 172.21.15.27:0/3638315399 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f83f815eb90 0x7f83f81f4300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:44.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.361+0000 7f83fe70a640 1 --2- 172.21.15.27:0/3638315399 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f83f80d5200 0x7f83f80cb100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:44.363 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.361+0000 7f83fe70a640 1 --2- 172.21.15.27:0/3638315399 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f83f80d1550 0x7f83f8158c10 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:44.363 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.361+0000 7f83fe70a640 1 -- 172.21.15.27:0/3638315399 >> 172.21.15.27:0/3638315399 conn(0x7f83f806dce0 msgr2=0x7f83f8161b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:44.363 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.361+0000 7f83fe70a640 1 -- 172.21.15.27:0/3638315399 shutdown_connections 2024-09-12T03:16:44.363 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.361+0000 7f83fe70a640 1 -- 172.21.15.27:0/3638315399 wait complete. 2024-09-12T03:16:44.373 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-12T03:16:44.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.473+0000 7f7eab4f1640 1 Processor -- start 2024-09-12T03:16:44.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.474+0000 7f7eab4f1640 1 -- start start 2024-09-12T03:16:44.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.475+0000 7f7eab4f1640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f7ea415eb90 0x7f7ea40d2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:44.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.475+0000 7f7eab4f1640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f7ea40d5200 0x7f7ea40d1010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:44.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.475+0000 7f7eab4f1640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7ea40d19e0 con 0x7f7ea40d1550 2024-09-12T03:16:44.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.475+0000 7f7eab4f1640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f7ea40b7d20 con 0x7f7ea40d5200 2024-09-12T03:16:44.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.475+0000 7f7eab4f1640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f7ea40d42e0 con 0x7f7ea415eb90 2024-09-12T03:16:44.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.475+0000 7f7ea8a8c640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f7ea40d5200 0x7f7ea40d1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:44.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.475+0000 7f7ea8a8c640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f7ea40d5200 0x7f7ea40d1010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33504/0 (socket says 172.21.15.27:33504) 2024-09-12T03:16:44.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.475+0000 7f7ea8a8c640 1 -- 172.21.15.27:0/49722201 learned_addr learned my addr 172.21.15.27:0/49722201 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:16:44.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.475+0000 7f7ea928d640 1 --2- 172.21.15.27:0/49722201 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f7ea415eb90 0x7f7ea40d2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:44.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.475+0000 7f7ea8a8c640 1 -- 172.21.15.27:0/49722201 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f7ea415eb90 msgr2=0x7f7ea40d2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:44.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.475+0000 7f7ea8a8c640 1 --2- 172.21.15.27:0/49722201 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f7ea415eb90 0x7f7ea40d2df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:44.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.475+0000 7f7ea8a8c640 1 -- 172.21.15.27:0/49722201 >> v1:172.21.15.45:6789/0 conn(0x7f7ea40d1550 legacy=0x7f7ea40cf200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:44.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.475+0000 7f7ea8a8c640 1 -- 172.21.15.27:0/49722201 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ea40d3760 con 0x7f7ea40d5200 2024-09-12T03:16:44.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.476+0000 7f7ea8a8c640 1 --2- 172.21.15.27:0/49722201 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7ea40d5200 0x7f7ea40d1010 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f7e98009870 tx=0x7f7e9802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d4f3580014656b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:44.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.476+0000 7f7e97fff640 1 -- 172.21.15.27:0/49722201 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e98037070 con 0x7f7ea40d5200 2024-09-12T03:16:44.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.476+0000 7f7e97fff640 1 -- 172.21.15.27:0/49722201 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e98032440 con 0x7f7ea40d5200 2024-09-12T03:16:44.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.476+0000 7f7e97fff640 1 -- 172.21.15.27:0/49722201 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e98032810 con 0x7f7ea40d5200 2024-09-12T03:16:44.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.476+0000 7f7eab4f1640 1 -- 172.21.15.27:0/49722201 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7ea40d5200 msgr2=0x7f7ea40d1010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:44.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.476+0000 7f7eab4f1640 1 --2- 172.21.15.27:0/49722201 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7ea40d5200 0x7f7ea40d1010 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f7e98009870 tx=0x7f7e9802a270 comp rx=0 tx=0).stop 2024-09-12T03:16:44.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.476+0000 7f7eab4f1640 1 -- 172.21.15.27:0/49722201 shutdown_connections 2024-09-12T03:16:44.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.476+0000 7f7eab4f1640 1 --2- 172.21.15.27:0/49722201 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7ea40d5200 0x7f7ea40d1010 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:44.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.477+0000 7f7eab4f1640 1 --2- 172.21.15.27:0/49722201 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f7ea415eb90 0x7f7ea40d2df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:44.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.477+0000 7f7eab4f1640 1 -- 172.21.15.27:0/49722201 >> 172.21.15.27:0/49722201 conn(0x7f7ea406dce0 msgr2=0x7f7ea406e0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:44.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.477+0000 7f7eab4f1640 1 -- 172.21.15.27:0/49722201 shutdown_connections 2024-09-12T03:16:44.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.477+0000 7f7eab4f1640 1 -- 172.21.15.27:0/49722201 wait complete. 2024-09-12T03:16:44.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.478+0000 7f7eab4f1640 1 Processor -- start 2024-09-12T03:16:44.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.478+0000 7f7eab4f1640 1 -- start start 2024-09-12T03:16:44.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.479+0000 7f7eab4f1640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f7ea40d1550 0x7f7ea41f3f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:44.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.479+0000 7f7eab4f1640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7ea40d5200 0x7f7ea40ca120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:44.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.479+0000 7f7eab4f1640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7ea415eb90 0x7f7ea40cdab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:44.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.479+0000 7f7eab4f1640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f7ea40d4000 con 0x7f7ea40d5200 2024-09-12T03:16:44.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.479+0000 7f7eab4f1640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f7ea40d3d20 con 0x7f7ea415eb90 2024-09-12T03:16:44.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.479+0000 7f7eab4f1640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f7ea40d3330 con 0x7f7ea40d1550 2024-09-12T03:16:44.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.479+0000 7f7ea8a8c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7ea40d5200 0x7f7ea40ca120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:44.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.479+0000 7f7ea8a8c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7ea40d5200 0x7f7ea40ca120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33516/0 (socket says 172.21.15.27:33516) 2024-09-12T03:16:44.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.479+0000 7f7ea8a8c640 1 -- 172.21.15.27:0/275686858 learned_addr learned my addr 172.21.15.27:0/275686858 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:16:44.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.479+0000 7f7ea928d640 1 --2- 172.21.15.27:0/275686858 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f7ea40d1550 0x7f7ea41f3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:44.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.479+0000 7f7ea9a8e640 1 --2- 172.21.15.27:0/275686858 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7ea415eb90 0x7f7ea40cdab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:44.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.479+0000 7f7ea8a8c640 1 -- 172.21.15.27:0/275686858 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f7ea40d1550 msgr2=0x7f7ea41f3f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:44.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.479+0000 7f7ea8a8c640 1 --2- 172.21.15.27:0/275686858 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f7ea40d1550 0x7f7ea41f3f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:44.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.479+0000 7f7ea8a8c640 1 -- 172.21.15.27:0/275686858 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7ea415eb90 msgr2=0x7f7ea40cdab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:44.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.479+0000 7f7ea8a8c640 1 --2- 172.21.15.27:0/275686858 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7ea415eb90 0x7f7ea40cdab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:44.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.479+0000 7f7ea8a8c640 1 -- 172.21.15.27:0/275686858 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ea40cab00 con 0x7f7ea40d5200 2024-09-12T03:16:44.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.479+0000 7f7ea9a8e640 1 --2- 172.21.15.27:0/275686858 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7ea415eb90 0x7f7ea40cdab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:16:44.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.479+0000 7f7ea8a8c640 1 --2- 172.21.15.27:0/275686858 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7ea40d5200 0x7f7ea40ca120 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f7e98000c00 tx=0x7f7e980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:44.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.479+0000 7f7e95ffb640 1 -- 172.21.15.27:0/275686858 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e98037040 con 0x7f7ea40d5200 2024-09-12T03:16:44.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.479+0000 7f7e95ffb640 1 -- 172.21.15.27:0/275686858 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e98042070 con 0x7f7ea40d5200 2024-09-12T03:16:44.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.479+0000 7f7eab4f1640 1 -- 172.21.15.27:0/275686858 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ea40cade0 con 0x7f7ea40d5200 2024-09-12T03:16:44.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.479+0000 7f7e95ffb640 1 -- 172.21.15.27:0/275686858 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e98030880 con 0x7f7ea40d5200 2024-09-12T03:16:44.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.480+0000 7f7eab4f1640 1 -- 172.21.15.27:0/275686858 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ea40cb3a0 con 0x7f7ea40d5200 2024-09-12T03:16:44.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.480+0000 7f7eab4f1640 1 -- 172.21.15.27:0/275686858 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ea415ef70 con 0x7f7ea40d5200 2024-09-12T03:16:44.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.481+0000 7f7e95ffb640 1 -- 172.21.15.27:0/275686858 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7e98030a20 con 0x7f7ea40d5200 2024-09-12T03:16:44.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.481+0000 7f7e95ffb640 1 --2- 172.21.15.27:0/275686858 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f7e8806be30 0x7f7e8806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:44.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.482+0000 7f7e95ffb640 1 -- 172.21.15.27:0/275686858 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f7e980ab5a0 con 0x7f7ea40d5200 2024-09-12T03:16:44.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.482+0000 7f7ea928d640 1 --2- 172.21.15.27:0/275686858 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f7e8806be30 0x7f7e8806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:44.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.482+0000 7f7ea928d640 1 --2- 172.21.15.27:0/275686858 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f7e8806be30 0x7f7e8806e2f0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f7ea0009aa0 tx=0x7f7ea0009190 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:44.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.486+0000 7f7e95ffb640 1 -- 172.21.15.27:0/275686858 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f7e98079fe0 con 0x7f7ea40d5200 2024-09-12T03:16:44.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.696+0000 7f7eab4f1640 1 -- 172.21.15.27:0/275686858 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7ea40b9240 con 0x7f7ea40d5200 2024-09-12T03:16:44.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.697+0000 7f7e95ffb640 1 -- 172.21.15.27:0/275686858 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+5214 (secure 0 0 0) 0x7f7e9807ee20 con 0x7f7ea40d5200 2024-09-12T03:16:44.698 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:16:44.698 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":6,"btime":"2024-09-12T03:16:33:576418+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6836/682326693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":682326693},{"type":"v1","addr":"172.21.15.45:6836","nonce":682326693}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4278,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6834/4084728235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":4084728235},{"type":"v1","addr":"172.21.15.27:6834","nonce":4084728235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4281,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/2844240887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":2844240887},{"type":"v1","addr":"172.21.15.27:6835","nonce":2844240887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-12T03:16:30.914517+0000","modified":"2024-09-12T03:16:32.913742+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.45:6837/1035926332","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6835","nonce":1035926332},{"type":"v1","addr":"172.21.15.45:6837","nonce":1035926332}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4285,"qdb_cluster":[4285]},"id":1}]} 2024-09-12T03:16:44.698 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 6 2024-09-12T03:16:44.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.699+0000 7f7eab4f1640 1 -- 172.21.15.27:0/275686858 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f7e8806be30 msgr2=0x7f7e8806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:44.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.699+0000 7f7eab4f1640 1 --2- 172.21.15.27:0/275686858 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f7e8806be30 0x7f7e8806e2f0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f7ea0009aa0 tx=0x7f7ea0009190 comp rx=0 tx=0).stop 2024-09-12T03:16:44.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.699+0000 7f7eab4f1640 1 -- 172.21.15.27:0/275686858 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7ea40d5200 msgr2=0x7f7ea40ca120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:44.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.699+0000 7f7eab4f1640 1 --2- 172.21.15.27:0/275686858 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7ea40d5200 0x7f7ea40ca120 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f7e98000c00 tx=0x7f7e980046c0 comp rx=0 tx=0).stop 2024-09-12T03:16:44.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.699+0000 7f7eab4f1640 1 -- 172.21.15.27:0/275686858 shutdown_connections 2024-09-12T03:16:44.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.699+0000 7f7eab4f1640 1 --2- 172.21.15.27:0/275686858 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f7e8806be30 0x7f7e8806e2f0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:44.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.699+0000 7f7eab4f1640 1 --2- 172.21.15.27:0/275686858 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7ea415eb90 0x7f7ea40cdab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:44.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.699+0000 7f7eab4f1640 1 --2- 172.21.15.27:0/275686858 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7ea40d5200 0x7f7ea40ca120 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:44.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.699+0000 7f7eab4f1640 1 --2- 172.21.15.27:0/275686858 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f7ea40d1550 0x7f7ea41f3f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:44.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.699+0000 7f7eab4f1640 1 -- 172.21.15.27:0/275686858 >> 172.21.15.27:0/275686858 conn(0x7f7ea406dce0 msgr2=0x7f7ea40d1f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:44.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.699+0000 7f7eab4f1640 1 -- 172.21.15.27:0/275686858 shutdown_connections 2024-09-12T03:16:44.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.699+0000 7f7eab4f1640 1 -- 172.21.15.27:0/275686858 wait complete. 2024-09-12T03:16:44.711 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-12T03:16:44.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.812+0000 7f70e4318640 1 Processor -- start 2024-09-12T03:16:44.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.812+0000 7f70e4318640 1 -- start start 2024-09-12T03:16:44.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.813+0000 7f70e4318640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f70dc0cf810 0x7f70dc0d3830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:44.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.813+0000 7f70e4318640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f70dc0d1e60 con 0x7f70dc0d52e0 2024-09-12T03:16:44.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.813+0000 7f70e4318640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f70dc0d2380 con 0x7f70dc0d1050 2024-09-12T03:16:44.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.813+0000 7f70e4318640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f70dc0cfca0 con 0x7f70dc0cf810 2024-09-12T03:16:44.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.813+0000 7f70e18b3640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f70dc0d52e0 0x7f70dc0d3120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:39280/0 (socket says 172.21.15.27:39280) 2024-09-12T03:16:44.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.813+0000 7f70e18b3640 1 -- 172.21.15.27:0/1013105816 learned_addr learned my addr 172.21.15.27:0/1013105816 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:16:44.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.813+0000 7f70e10b2640 1 -- 172.21.15.27:0/1013105816 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1887177040 0 0) 0x7f70dc0d1e60 con 0x7f70dc0d52e0 2024-09-12T03:16:44.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.813+0000 7f70e28b5640 1 --2- 172.21.15.27:0/1013105816 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f70dc0cf810 0x7f70dc0d3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:44.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.813+0000 7f70e10b2640 1 -- 172.21.15.27:0/1013105816 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f70c8003520 con 0x7f70dc0d52e0 2024-09-12T03:16:44.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.813+0000 7f70e10b2640 1 -- 172.21.15.27:0/1013105816 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3901103002 0 0) 0x7f70c8003520 con 0x7f70dc0d52e0 2024-09-12T03:16:44.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.813+0000 7f70e10b2640 1 -- 172.21.15.27:0/1013105816 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f70dc0cf810 msgr2=0x7f70dc0d3830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:44.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.813+0000 7f70e10b2640 1 --2- 172.21.15.27:0/1013105816 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f70dc0cf810 0x7f70dc0d3830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:44.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.813+0000 7f70e10b2640 1 -- 172.21.15.27:0/1013105816 >> v1:172.21.15.45:6789/0 conn(0x7f70dc0d1050 legacy=0x7f70dc0d1430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:44.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.813+0000 7f70e10b2640 1 -- 172.21.15.27:0/1013105816 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70dc0cfef0 con 0x7f70dc0d52e0 2024-09-12T03:16:44.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.813+0000 7f70e10b2640 1 -- 172.21.15.27:0/1013105816 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 847755213 0 0) 0x7f70cc001670 con 0x7f70dc0d52e0 2024-09-12T03:16:44.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.813+0000 7f70e10b2640 1 -- 172.21.15.27:0/1013105816 >> v1:172.21.15.27:6789/0 conn(0x7f70dc0d52e0 legacy=0x7f70dc0d3120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:44.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.813+0000 7f70e10b2640 1 --2- 172.21.15.27:0/1013105816 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f70c80039e0 0x7f70c8003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:44.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.813+0000 7f70e10b2640 1 --2- 172.21.15.27:0/1013105816 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f70dc0cf810 0x7f70c8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:44.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.813+0000 7f70e10b2640 1 --2- 172.21.15.27:0/1013105816 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f70c8004d40 0x7f70c80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:44.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.813+0000 7f70e10b2640 1 -- 172.21.15.27:0/1013105816 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f70dc0cfca0 con 0x7f70c80039e0 2024-09-12T03:16:44.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.813+0000 7f70e10b2640 1 -- 172.21.15.27:0/1013105816 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f70c800baa0 con 0x7f70c8004d40 2024-09-12T03:16:44.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.813+0000 7f70e10b2640 1 -- 172.21.15.27:0/1013105816 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f70c800bca0 con 0x7f70dc0cf810 2024-09-12T03:16:44.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.813+0000 7f70e20b4640 1 --2- 172.21.15.27:0/1013105816 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f70c80039e0 0x7f70c8003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:44.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.814+0000 7f70e18b3640 1 --2- 172.21.15.27:0/1013105816 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f70dc0cf810 0x7f70c8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:44.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.814+0000 7f70e28b5640 1 --2- 172.21.15.27:0/1013105816 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f70c8004d40 0x7f70c80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:44.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.814+0000 7f70e20b4640 1 -- 172.21.15.27:0/1013105816 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f70dc0cf810 msgr2=0x7f70c8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:44.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.814+0000 7f70e20b4640 1 --2- 172.21.15.27:0/1013105816 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f70dc0cf810 0x7f70c8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:44.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.814+0000 7f70e20b4640 1 -- 172.21.15.27:0/1013105816 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f70c8004d40 msgr2=0x7f70c80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:44.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.814+0000 7f70e20b4640 1 --2- 172.21.15.27:0/1013105816 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f70c8004d40 0x7f70c80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:44.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.814+0000 7f70e20b4640 1 -- 172.21.15.27:0/1013105816 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70c8003520 con 0x7f70c80039e0 2024-09-12T03:16:44.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.815+0000 7f70e20b4640 1 --2- 172.21.15.27:0/1013105816 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f70c80039e0 0x7f70c8003e00 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f70d8001720 tx=0x7f70c8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d568c503cd1d6a8d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:44.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.815+0000 7f70e10b2640 1 -- 172.21.15.27:0/1013105816 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70d8001370 con 0x7f70c80039e0 2024-09-12T03:16:44.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.815+0000 7f70e10b2640 1 -- 172.21.15.27:0/1013105816 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70d80052d0 con 0x7f70c80039e0 2024-09-12T03:16:44.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.815+0000 7f70e10b2640 1 -- 172.21.15.27:0/1013105816 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70d802abc0 con 0x7f70c80039e0 2024-09-12T03:16:44.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.815+0000 7f70e4318640 1 -- 172.21.15.27:0/1013105816 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f70c80039e0 msgr2=0x7f70c8003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:44.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.815+0000 7f70e4318640 1 --2- 172.21.15.27:0/1013105816 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f70c80039e0 0x7f70c8003e00 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f70d8001720 tx=0x7f70c8005830 comp rx=0 tx=0).stop 2024-09-12T03:16:44.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.815+0000 7f70e28b5640 1 -- 172.21.15.27:0/1013105816 reap_dead start 2024-09-12T03:16:44.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.815+0000 7f70e4318640 1 -- 172.21.15.27:0/1013105816 shutdown_connections 2024-09-12T03:16:44.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.815+0000 7f70e4318640 1 -- 172.21.15.27:0/1013105816 >> 172.21.15.27:0/1013105816 conn(0x7f70dc06ddf0 msgr2=0x7f70dc06e200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:44.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.815+0000 7f70e4318640 1 -- 172.21.15.27:0/1013105816 shutdown_connections 2024-09-12T03:16:44.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.815+0000 7f70e4318640 1 -- 172.21.15.27:0/1013105816 wait complete. 2024-09-12T03:16:44.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.816+0000 7f70e4318640 1 Processor -- start 2024-09-12T03:16:44.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.817+0000 7f70e4318640 1 -- start start 2024-09-12T03:16:44.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.817+0000 7f70e4318640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f70c80039e0 0x7f70dc149630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:44.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.817+0000 7f70e4318640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f70dc14bf90 0x7f70dc14c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:44.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.817+0000 7f70e4318640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f70dc149fe0 0x7f70dc14a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:44.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.817+0000 7f70e4318640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f70dc0d2c00 con 0x7f70dc14bf90 2024-09-12T03:16:44.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.817+0000 7f70e4318640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f70dc0d2920 con 0x7f70dc149fe0 2024-09-12T03:16:44.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.817+0000 7f70e4318640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f70dc0b7a50 con 0x7f70c80039e0 2024-09-12T03:16:44.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.817+0000 7f70e18b3640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f70dc14bf90 0x7f70dc14c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:44.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.817+0000 7f70e18b3640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f70dc14bf90 0x7f70dc14c3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33532/0 (socket says 172.21.15.27:33532) 2024-09-12T03:16:44.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.817+0000 7f70e18b3640 1 -- 172.21.15.27:0/2110353913 learned_addr learned my addr 172.21.15.27:0/2110353913 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:16:44.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.817+0000 7f70e20b4640 1 --2- 172.21.15.27:0/2110353913 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f70c80039e0 0x7f70dc149630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:44.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.817+0000 7f70e28b5640 1 --2- 172.21.15.27:0/2110353913 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f70dc149fe0 0x7f70dc14a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:44.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.818+0000 7f70e18b3640 1 -- 172.21.15.27:0/2110353913 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f70c80039e0 msgr2=0x7f70dc149630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:44.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.818+0000 7f70e18b3640 1 --2- 172.21.15.27:0/2110353913 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f70c80039e0 0x7f70dc149630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:44.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.818+0000 7f70e18b3640 1 -- 172.21.15.27:0/2110353913 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f70dc149fe0 msgr2=0x7f70dc14a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:44.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.818+0000 7f70e18b3640 1 --2- 172.21.15.27:0/2110353913 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f70dc149fe0 0x7f70dc14a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:44.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.818+0000 7f70e18b3640 1 -- 172.21.15.27:0/2110353913 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70dc14ad30 con 0x7f70dc14bf90 2024-09-12T03:16:44.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.818+0000 7f70e18b3640 1 --2- 172.21.15.27:0/2110353913 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f70dc14bf90 0x7f70dc14c3f0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f70cc0072c0 tx=0x7f70cc007780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:44.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.818+0000 7f70d2ffd640 1 -- 172.21.15.27:0/2110353913 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70cc01a070 con 0x7f70dc14bf90 2024-09-12T03:16:44.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.818+0000 7f70d2ffd640 1 -- 172.21.15.27:0/2110353913 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70cc00f960 con 0x7f70dc14bf90 2024-09-12T03:16:44.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.818+0000 7f70e4318640 1 -- 172.21.15.27:0/2110353913 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70dc14b010 con 0x7f70dc14bf90 2024-09-12T03:16:44.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.818+0000 7f70d2ffd640 1 -- 172.21.15.27:0/2110353913 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70cc00fd30 con 0x7f70dc14bf90 2024-09-12T03:16:44.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.818+0000 7f70e4318640 1 -- 172.21.15.27:0/2110353913 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70dc14b5d0 con 0x7f70dc14bf90 2024-09-12T03:16:44.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.819+0000 7f70e4318640 1 -- 172.21.15.27:0/2110353913 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70dc0d20a0 con 0x7f70dc14bf90 2024-09-12T03:16:44.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.820+0000 7f70d2ffd640 1 -- 172.21.15.27:0/2110353913 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f70cc01d070 con 0x7f70dc14bf90 2024-09-12T03:16:44.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.820+0000 7f70d2ffd640 1 --2- 172.21.15.27:0/2110353913 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f70c006be30 0x7f70c006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:44.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.820+0000 7f70d2ffd640 1 -- 172.21.15.27:0/2110353913 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f70cc08a6f0 con 0x7f70dc14bf90 2024-09-12T03:16:44.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.820+0000 7f70e20b4640 1 --2- 172.21.15.27:0/2110353913 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f70c006be30 0x7f70c006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:44.826 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.821+0000 7f70e20b4640 1 --2- 172.21.15.27:0/2110353913 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f70c006be30 0x7f70c006e2f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f70d8032000 tx=0x7f70d802e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:44.826 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:44.824+0000 7f70d2ffd640 1 -- 172.21.15.27:0/2110353913 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f70cc058ae0 con 0x7f70dc14bf90 2024-09-12T03:16:45.001 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.000+0000 7f70e4318640 1 -- 172.21.15.27:0/2110353913 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f70dc0b6ef0 con 0x7f70dc14bf90 2024-09-12T03:16:45.002 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.001+0000 7f70d2ffd640 1 -- 172.21.15.27:0/2110353913 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16617 (secure 0 0 0) 0x7f70cc05cde0 con 0x7f70dc14bf90 2024-09-12T03:16:45.003 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:16:45.003 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":21,"fsid":"d2df504e-2647-4385-a5db-9fdf69f487a0","created":"2024-09-12T03:16:07.778937+0000","modified":"2024-09-12T03:16:32.570182+0000","last_up_change":"2024-09-12T03:16:19.314778+0000","last_in_change":"2024-09-12T03:16:09.893086+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T03:16:20.097646+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-12T03:16:28.790791+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-12T03:16:29.909065+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1851116c-1fb3-4dc9-9772-f90e2ef4c355","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6817","nonce":3002321488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6819","nonce":3002321488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6823","nonce":3002321488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6821","nonce":3002321488}]},"public_addr":"172.21.15.27:6817/3002321488","cluster_addr":"172.21.15.27:6819/3002321488","heartbeat_back_addr":"172.21.15.27:6823/3002321488","heartbeat_front_addr":"172.21.15.27:6821/3002321488","state":["exists","up"]},{"osd":1,"uuid":"f38ae17d-73eb-4204-b99d-975fa806fb87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6809","nonce":1613102886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6811","nonce":1613102886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6815","nonce":1613102886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6813","nonce":1613102886}]},"public_addr":"172.21.15.27:6809/1613102886","cluster_addr":"172.21.15.27:6811/1613102886","heartbeat_back_addr":"172.21.15.27:6815/1613102886","heartbeat_front_addr":"172.21.15.27:6813/1613102886","state":["exists","up"]},{"osd":2,"uuid":"fb12dcc8-bdd0-42b8-b46d-8b10106224bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6825","nonce":3534723355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6827","nonce":3534723355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6831","nonce":3534723355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6829","nonce":3534723355}]},"public_addr":"172.21.15.27:6825/3534723355","cluster_addr":"172.21.15.27:6827/3534723355","heartbeat_back_addr":"172.21.15.27:6831/3534723355","heartbeat_front_addr":"172.21.15.27:6829/3534723355","state":["exists","up"]},{"osd":3,"uuid":"d74d3f49-9c24-4b9f-98e2-3ecd7b5ef3b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6801","nonce":80661598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6803","nonce":80661598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6807","nonce":80661598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6805","nonce":80661598}]},"public_addr":"172.21.15.27:6801/80661598","cluster_addr":"172.21.15.27:6803/80661598","heartbeat_back_addr":"172.21.15.27:6807/80661598","heartbeat_front_addr":"172.21.15.27:6805/80661598","state":["exists","up"]},{"osd":4,"uuid":"2bf123f2-1b0d-41c4-aa3b-805bab267e59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6817","nonce":3943189827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6819","nonce":3943189827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6823","nonce":3943189827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6821","nonce":3943189827}]},"public_addr":"172.21.15.45:6817/3943189827","cluster_addr":"172.21.15.45:6819/3943189827","heartbeat_back_addr":"172.21.15.45:6823/3943189827","heartbeat_front_addr":"172.21.15.45:6821/3943189827","state":["exists","up"]},{"osd":5,"uuid":"c49ba5da-91c6-4a86-a97b-4aba89c253ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6809","nonce":3620760330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6811","nonce":3620760330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6815","nonce":3620760330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6813","nonce":3620760330}]},"public_addr":"172.21.15.45:6809/3620760330","cluster_addr":"172.21.15.45:6811/3620760330","heartbeat_back_addr":"172.21.15.45:6815/3620760330","heartbeat_front_addr":"172.21.15.45:6813/3620760330","state":["exists","up"]},{"osd":6,"uuid":"b0123475-89f7-4a34-9e33-d797b99dd9dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6801","nonce":1989083510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6803","nonce":1989083510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6807","nonce":1989083510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6805","nonce":1989083510}]},"public_addr":"172.21.15.45:6801/1989083510","cluster_addr":"172.21.15.45:6803/1989083510","heartbeat_back_addr":"172.21.15.45:6807/1989083510","heartbeat_front_addr":"172.21.15.45:6805/1989083510","state":["exists","up"]},{"osd":7,"uuid":"4dad773d-b7d8-46fc-8a10-a972aad27b29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6825","nonce":1513300294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6827","nonce":1513300294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6831","nonce":1513300294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6829","nonce":1513300294}]},"public_addr":"172.21.15.45:6825/1513300294","cluster_addr":"172.21.15.45:6827/1513300294","heartbeat_back_addr":"172.21.15.45:6831/1513300294","heartbeat_front_addr":"172.21.15.45:6829/1513300294","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.881855+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:17.133252+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.735339+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.500742+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.477571+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:17.167667+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.470291+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.873743+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T03:16:45.005 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.004+0000 7f70e4318640 1 -- 172.21.15.27:0/2110353913 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f70c006be30 msgr2=0x7f70c006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:45.006 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.004+0000 7f70e4318640 1 --2- 172.21.15.27:0/2110353913 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f70c006be30 0x7f70c006e2f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f70d8032000 tx=0x7f70d802e000 comp rx=0 tx=0).stop 2024-09-12T03:16:45.006 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.004+0000 7f70e4318640 1 -- 172.21.15.27:0/2110353913 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f70dc14bf90 msgr2=0x7f70dc14c3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:45.006 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.004+0000 7f70e4318640 1 --2- 172.21.15.27:0/2110353913 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f70dc14bf90 0x7f70dc14c3f0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f70cc0072c0 tx=0x7f70cc007780 comp rx=0 tx=0).stop 2024-09-12T03:16:45.006 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.004+0000 7f70e4318640 1 -- 172.21.15.27:0/2110353913 shutdown_connections 2024-09-12T03:16:45.006 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.004+0000 7f70e4318640 1 --2- 172.21.15.27:0/2110353913 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f70c006be30 0x7f70c006e2f0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:45.006 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.004+0000 7f70e4318640 1 --2- 172.21.15.27:0/2110353913 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f70dc149fe0 0x7f70dc14a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:45.006 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.004+0000 7f70e4318640 1 --2- 172.21.15.27:0/2110353913 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f70dc14bf90 0x7f70dc14c3f0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:45.006 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.004+0000 7f70e4318640 1 --2- 172.21.15.27:0/2110353913 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f70c80039e0 0x7f70dc149630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:45.006 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.004+0000 7f70e4318640 1 -- 172.21.15.27:0/2110353913 >> 172.21.15.27:0/2110353913 conn(0x7f70dc06ddf0 msgr2=0x7f70dc06e200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:45.006 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.004+0000 7f70e4318640 1 -- 172.21.15.27:0/2110353913 shutdown_connections 2024-09-12T03:16:45.006 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.004+0000 7f70e4318640 1 -- 172.21.15.27:0/2110353913 wait complete. 2024-09-12T03:16:45.016 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-12T03:16:45.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.117+0000 7ffbeec1e640 1 Processor -- start 2024-09-12T03:16:45.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.117+0000 7ffbeec1e640 1 -- start start 2024-09-12T03:16:45.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.118+0000 7ffbeec1e640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ffbe80b1ca0 0x7ffbe815f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:45.122 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.118+0000 7ffbeec1e640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffbe815fd70 con 0x7ffbe80cf660 2024-09-12T03:16:45.122 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.118+0000 7ffbeec1e640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffbe806cc40 con 0x7ffbe806f360 2024-09-12T03:16:45.122 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.118+0000 7ffbeec1e640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7ffbe80d22f0 con 0x7ffbe80b1ca0 2024-09-12T03:16:45.122 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.118+0000 7ffbed1bb640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7ffbe80cf660 0x7ffbe80d2780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:39284/0 (socket says 172.21.15.27:39284) 2024-09-12T03:16:45.122 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.118+0000 7ffbed1bb640 1 -- 172.21.15.27:0/2026922006 learned_addr learned my addr 172.21.15.27:0/2026922006 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:16:45.122 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.118+0000 7ffbec9ba640 1 --2- 172.21.15.27:0/2026922006 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ffbe80b1ca0 0x7ffbe815f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:45.122 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.118+0000 7ffbdf7fe640 1 -- 172.21.15.27:0/2026922006 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2277260835 0 0) 0x7ffbe0001490 con 0x7ffbe80cf660 2024-09-12T03:16:45.123 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.118+0000 7ffbdf7fe640 1 -- 172.21.15.27:0/2026922006 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ffbd0003520 con 0x7ffbe80cf660 2024-09-12T03:16:45.123 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.118+0000 7ffbdf7fe640 1 -- 172.21.15.27:0/2026922006 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2961069843 0 0) 0x7ffbd0003520 con 0x7ffbe80cf660 2024-09-12T03:16:45.123 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.118+0000 7ffbdf7fe640 1 -- 172.21.15.27:0/2026922006 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ffbe80b1ca0 msgr2=0x7ffbe815f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:45.123 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.118+0000 7ffbdf7fe640 1 --2- 172.21.15.27:0/2026922006 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ffbe80b1ca0 0x7ffbe815f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:45.123 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.118+0000 7ffbdf7fe640 1 -- 172.21.15.27:0/2026922006 >> v1:172.21.15.45:6789/0 conn(0x7ffbe806f360 legacy=0x7ffbe80cef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:45.123 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.119+0000 7ffbdf7fe640 1 -- 172.21.15.27:0/2026922006 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbe80d1ce0 con 0x7ffbe80cf660 2024-09-12T03:16:45.123 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.119+0000 7ffbdf7fe640 1 -- 172.21.15.27:0/2026922006 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 847755213 0 0) 0x7ffbe0001bc0 con 0x7ffbe80cf660 2024-09-12T03:16:45.123 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.119+0000 7ffbdf7fe640 1 -- 172.21.15.27:0/2026922006 >> v1:172.21.15.27:6789/0 conn(0x7ffbe80cf660 legacy=0x7ffbe80d2780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:45.123 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.119+0000 7ffbdf7fe640 1 --2- 172.21.15.27:0/2026922006 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffbd00039e0 0x7ffbd0003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:45.123 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.119+0000 7ffbdf7fe640 1 --2- 172.21.15.27:0/2026922006 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ffbd00043b0 0x7ffbd0004880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:45.124 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.119+0000 7ffbdf7fe640 1 --2- 172.21.15.27:0/2026922006 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffbd0005030 0x7ffbd000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:45.124 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.119+0000 7ffbdf7fe640 1 -- 172.21.15.27:0/2026922006 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7ffbe80d22f0 con 0x7ffbd0005030 2024-09-12T03:16:45.124 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.119+0000 7ffbdf7fe640 1 -- 172.21.15.27:0/2026922006 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ffbd000bf60 con 0x7ffbd00039e0 2024-09-12T03:16:45.124 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.119+0000 7ffbdf7fe640 1 -- 172.21.15.27:0/2026922006 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7ffbd000c160 con 0x7ffbd00043b0 2024-09-12T03:16:45.124 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.119+0000 7ffbec9ba640 1 --2- 172.21.15.27:0/2026922006 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ffbe80b1ca0 0x7ffbe815f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:16:45.124 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.119+0000 7ffbed1bb640 1 --2- 172.21.15.27:0/2026922006 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffbd0005030 0x7ffbd000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:45.124 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.119+0000 7ffbdffff640 1 --2- 172.21.15.27:0/2026922006 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ffbd00043b0 0x7ffbd0004880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:45.124 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.119+0000 7ffbec9ba640 1 --2- 172.21.15.27:0/2026922006 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffbd00039e0 0x7ffbd0003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:45.124 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.119+0000 7ffbed1bb640 1 -- 172.21.15.27:0/2026922006 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ffbd00043b0 msgr2=0x7ffbd0004880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:45.124 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.119+0000 7ffbed1bb640 1 --2- 172.21.15.27:0/2026922006 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ffbd00043b0 0x7ffbd0004880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:45.124 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.119+0000 7ffbed1bb640 1 -- 172.21.15.27:0/2026922006 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffbd00039e0 msgr2=0x7ffbd0003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:45.124 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.119+0000 7ffbed1bb640 1 --2- 172.21.15.27:0/2026922006 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffbd00039e0 0x7ffbd0003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:45.125 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.119+0000 7ffbed1bb640 1 -- 172.21.15.27:0/2026922006 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbd0003520 con 0x7ffbd0005030 2024-09-12T03:16:45.125 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.119+0000 7ffbec9ba640 1 --2- 172.21.15.27:0/2026922006 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffbd00039e0 0x7ffbd0003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:16:45.125 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.120+0000 7ffbed1bb640 1 --2- 172.21.15.27:0/2026922006 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffbd0005030 0x7ffbd000b9c0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7ffbe80730a0 tx=0x7ffbd0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7641e321e670958 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:45.125 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.120+0000 7ffbdf7fe640 1 -- 172.21.15.27:0/2026922006 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbe000a600 con 0x7ffbd0005030 2024-09-12T03:16:45.125 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.120+0000 7ffbdf7fe640 1 -- 172.21.15.27:0/2026922006 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffbe002f070 con 0x7ffbd0005030 2024-09-12T03:16:45.125 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.120+0000 7ffbdf7fe640 1 -- 172.21.15.27:0/2026922006 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbe002bdd0 con 0x7ffbd0005030 2024-09-12T03:16:45.125 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.120+0000 7ffbeec1e640 1 -- 172.21.15.27:0/2026922006 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffbd0005030 msgr2=0x7ffbd000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:45.126 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.120+0000 7ffbeec1e640 1 --2- 172.21.15.27:0/2026922006 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffbd0005030 0x7ffbd000b9c0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7ffbe80730a0 tx=0x7ffbd0005830 comp rx=0 tx=0).stop 2024-09-12T03:16:45.126 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.120+0000 7ffbed1bb640 1 -- 172.21.15.27:0/2026922006 reap_dead start 2024-09-12T03:16:45.126 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.120+0000 7ffbeec1e640 1 -- 172.21.15.27:0/2026922006 shutdown_connections 2024-09-12T03:16:45.126 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.120+0000 7ffbeec1e640 1 -- 172.21.15.27:0/2026922006 >> 172.21.15.27:0/2026922006 conn(0x7ffbe8073800 msgr2=0x7ffbe809ba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:45.126 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.120+0000 7ffbeec1e640 1 -- 172.21.15.27:0/2026922006 shutdown_connections 2024-09-12T03:16:45.126 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.121+0000 7ffbeec1e640 1 -- 172.21.15.27:0/2026922006 wait complete. 2024-09-12T03:16:45.129 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.122+0000 7ffbeec1e640 1 Processor -- start 2024-09-12T03:16:45.129 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.122+0000 7ffbeec1e640 1 -- start start 2024-09-12T03:16:45.129 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.123+0000 7ffbeec1e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffbd0005030 0x7ffbe81478f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:45.129 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.123+0000 7ffbeec1e640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ffbe81572d0 0x7ffbe8148110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:45.129 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.123+0000 7ffbeec1e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffbe814a160 0x7ffbe80ce080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:45.129 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.123+0000 7ffbeec1e640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7ffbe806d1c0 con 0x7ffbe814a160 2024-09-12T03:16:45.129 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.123+0000 7ffbeec1e640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ffbe80d1f80 con 0x7ffbd0005030 2024-09-12T03:16:45.130 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.123+0000 7ffbeec1e640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7ffbe806d340 con 0x7ffbe81572d0 2024-09-12T03:16:45.130 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.123+0000 7ffbed1bb640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffbe814a160 0x7ffbe80ce080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:45.130 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.123+0000 7ffbed1bb640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffbe814a160 0x7ffbe80ce080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33550/0 (socket says 172.21.15.27:33550) 2024-09-12T03:16:45.130 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.123+0000 7ffbed1bb640 1 -- 172.21.15.27:0/254682591 learned_addr learned my addr 172.21.15.27:0/254682591 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:16:45.130 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.123+0000 7ffbec9ba640 1 --2- 172.21.15.27:0/254682591 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffbd0005030 0x7ffbe81478f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:45.130 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.123+0000 7ffbdffff640 1 --2- 172.21.15.27:0/254682591 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ffbe81572d0 0x7ffbe8148110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:45.130 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.123+0000 7ffbed1bb640 1 -- 172.21.15.27:0/254682591 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ffbe81572d0 msgr2=0x7ffbe8148110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:45.130 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.123+0000 7ffbed1bb640 1 --2- 172.21.15.27:0/254682591 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ffbe81572d0 0x7ffbe8148110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:45.130 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.123+0000 7ffbed1bb640 1 -- 172.21.15.27:0/254682591 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffbd0005030 msgr2=0x7ffbe81478f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:45.130 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.123+0000 7ffbed1bb640 1 --2- 172.21.15.27:0/254682591 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffbd0005030 0x7ffbe81478f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:45.130 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.123+0000 7ffbed1bb640 1 -- 172.21.15.27:0/254682591 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbe8147e30 con 0x7ffbe814a160 2024-09-12T03:16:45.130 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.123+0000 7ffbec9ba640 1 --2- 172.21.15.27:0/254682591 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffbd0005030 0x7ffbe81478f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:16:45.130 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.123+0000 7ffbed1bb640 1 --2- 172.21.15.27:0/254682591 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffbe814a160 0x7ffbe80ce080 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7ffbe002b620 tx=0x7ffbe002d070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:45.131 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.123+0000 7ffbdd7fa640 1 -- 172.21.15.27:0/254682591 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbe000a600 con 0x7ffbe814a160 2024-09-12T03:16:45.131 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.123+0000 7ffbdd7fa640 1 -- 172.21.15.27:0/254682591 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffbe0001490 con 0x7ffbe814a160 2024-09-12T03:16:45.131 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.123+0000 7ffbeec1e640 1 -- 172.21.15.27:0/254682591 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffbe8148a50 con 0x7ffbe814a160 2024-09-12T03:16:45.131 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.123+0000 7ffbdd7fa640 1 -- 172.21.15.27:0/254682591 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbe0034670 con 0x7ffbe814a160 2024-09-12T03:16:45.131 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.123+0000 7ffbeec1e640 1 -- 172.21.15.27:0/254682591 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffbe814ae40 con 0x7ffbe814a160 2024-09-12T03:16:45.131 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.124+0000 7ffbeec1e640 1 -- 172.21.15.27:0/254682591 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffbe815e9e0 con 0x7ffbe814a160 2024-09-12T03:16:45.131 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.125+0000 7ffbdd7fa640 1 -- 172.21.15.27:0/254682591 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ffbe0034810 con 0x7ffbe814a160 2024-09-12T03:16:45.131 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.125+0000 7ffbdd7fa640 1 --2- 172.21.15.27:0/254682591 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7ffbd406be60 0x7ffbd406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:45.131 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.125+0000 7ffbdd7fa640 1 -- 172.21.15.27:0/254682591 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7ffbe00ab1b0 con 0x7ffbe814a160 2024-09-12T03:16:45.131 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.125+0000 7ffbec9ba640 1 --2- 172.21.15.27:0/254682591 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7ffbd406be60 0x7ffbd406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:45.131 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.126+0000 7ffbec9ba640 1 --2- 172.21.15.27:0/254682591 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7ffbd406be60 0x7ffbd406e320 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7ffbe8073ed0 tx=0x7ffbe4008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:45.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.130+0000 7ffbdd7fa640 1 -- 172.21.15.27:0/254682591 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7ffbe815e9e0 con 0x7ffbe814a160 2024-09-12T03:16:45.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.340+0000 7ffbeec1e640 1 -- 172.21.15.27:0/254682591 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ffbe80cfc40 con 0x7ffbe814a160 2024-09-12T03:16:45.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.340+0000 7ffbdd7fa640 1 -- 172.21.15.27:0/254682591 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+5214 (secure 0 0 0) 0x7ffbe8147e30 con 0x7ffbe814a160 2024-09-12T03:16:45.341 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:16:45.342 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":6,"btime":"2024-09-12T03:16:33:576418+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6836/682326693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":682326693},{"type":"v1","addr":"172.21.15.45:6836","nonce":682326693}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4278,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6834/4084728235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":4084728235},{"type":"v1","addr":"172.21.15.27:6834","nonce":4084728235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4281,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/2844240887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":2844240887},{"type":"v1","addr":"172.21.15.27:6835","nonce":2844240887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-12T03:16:30.914517+0000","modified":"2024-09-12T03:16:32.913742+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.45:6837/1035926332","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6835","nonce":1035926332},{"type":"v1","addr":"172.21.15.45:6837","nonce":1035926332}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4285,"qdb_cluster":[4285]},"id":1}]} 2024-09-12T03:16:45.342 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 6 2024-09-12T03:16:45.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.342+0000 7ffbeec1e640 1 -- 172.21.15.27:0/254682591 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7ffbd406be60 msgr2=0x7ffbd406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:45.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.342+0000 7ffbeec1e640 1 --2- 172.21.15.27:0/254682591 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7ffbd406be60 0x7ffbd406e320 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7ffbe8073ed0 tx=0x7ffbe4008040 comp rx=0 tx=0).stop 2024-09-12T03:16:45.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.342+0000 7ffbeec1e640 1 -- 172.21.15.27:0/254682591 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffbe814a160 msgr2=0x7ffbe80ce080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:45.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.342+0000 7ffbeec1e640 1 --2- 172.21.15.27:0/254682591 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffbe814a160 0x7ffbe80ce080 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7ffbe002b620 tx=0x7ffbe002d070 comp rx=0 tx=0).stop 2024-09-12T03:16:45.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.342+0000 7ffbeec1e640 1 -- 172.21.15.27:0/254682591 shutdown_connections 2024-09-12T03:16:45.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.342+0000 7ffbeec1e640 1 --2- 172.21.15.27:0/254682591 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7ffbd406be60 0x7ffbd406e320 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:45.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.342+0000 7ffbeec1e640 1 --2- 172.21.15.27:0/254682591 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffbe814a160 0x7ffbe80ce080 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:45.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.342+0000 7ffbeec1e640 1 --2- 172.21.15.27:0/254682591 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ffbe81572d0 0x7ffbe8148110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:45.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.342+0000 7ffbeec1e640 1 --2- 172.21.15.27:0/254682591 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffbd0005030 0x7ffbe81478f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:45.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.342+0000 7ffbeec1e640 1 -- 172.21.15.27:0/254682591 >> 172.21.15.27:0/254682591 conn(0x7ffbe8073800 msgr2=0x7ffbe80b31e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:45.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.342+0000 7ffbeec1e640 1 -- 172.21.15.27:0/254682591 shutdown_connections 2024-09-12T03:16:45.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.342+0000 7ffbeec1e640 1 -- 172.21.15.27:0/254682591 wait complete. 2024-09-12T03:16:45.757 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 6, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-12T03:16:30.914517+0000', 'modified': '2024-09-12T03:16:32.913742+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4285}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4285': {'gid': 4285, 'name': 'd', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.45:6837/1035926332', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.45:6835', 'nonce': 1035926332}, {'type': 'v1', 'addr': '172.21.15.45:6837', 'nonce': 1035926332}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4285, 'qdb_cluster': [4285]} 2024-09-12T03:16:45.757 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-12T03:16:45.757 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-09-12T03:16:45.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.862+0000 7fde4de27640 1 Processor -- start 2024-09-12T03:16:45.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.862+0000 7fde4de27640 1 -- start start 2024-09-12T03:16:45.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.863+0000 7fde4de27640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fde48145090 0x7fde480affb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:45.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.863+0000 7fde4de27640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fde4806bb30 0x7fde4806bf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:45.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.863+0000 7fde4de27640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fde4815f260 con 0x7fde480ae8d0 2024-09-12T03:16:45.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.863+0000 7fde4de27640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fde4806c4b0 con 0x7fde4806bb30 2024-09-12T03:16:45.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.863+0000 7fde4de27640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fde4815f540 con 0x7fde48145090 2024-09-12T03:16:45.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.863+0000 7fde477fe640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fde48145090 0x7fde480affb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:45.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.863+0000 7fde46ffd640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fde4806bb30 0x7fde4806bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:45.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.863+0000 7fde47fff640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fde480ae8d0 0x7fde480aed20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.27:39016/0 (socket says 172.21.15.27:39016) 2024-09-12T03:16:45.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.863+0000 7fde47fff640 1 -- 172.21.15.27:0/1037066462 learned_addr learned my addr 172.21.15.27:0/1037066462 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:16:45.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.863+0000 7fde477fe640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fde48145090 0x7fde480affb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3301/0 says I am v2:172.21.15.27:56758/0 (socket says 172.21.15.27:56758) 2024-09-12T03:16:45.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.863+0000 7fde477fe640 1 -- 172.21.15.27:0/1037066462 >> v2:172.21.15.45:3300/0 conn(0x7fde4806bb30 msgr2=0x7fde4806bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:45.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.863+0000 7fde477fe640 1 --2- 172.21.15.27:0/1037066462 >> v2:172.21.15.45:3300/0 conn(0x7fde4806bb30 0x7fde4806bf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:45.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.864+0000 7fde477fe640 1 -- 172.21.15.27:0/1037066462 >> v1:172.21.15.45:6789/0 conn(0x7fde480ae8d0 legacy=0x7fde480aed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:45.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.864+0000 7fde477fe640 1 -- 172.21.15.27:0/1037066462 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde4815f820 con 0x7fde48145090 2024-09-12T03:16:45.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.864+0000 7fde46ffd640 1 --2- 172.21.15.27:0/1037066462 >> v2:172.21.15.45:3300/0 conn(0x7fde4806bb30 0x7fde4806bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:16:45.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.864+0000 7fde477fe640 1 --2- 172.21.15.27:0/1037066462 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fde48145090 0x7fde480affb0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fde380097a0 tx=0x7fde3802a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=26a79cc371ac05f6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:45.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.865+0000 7fde467fc640 1 -- 172.21.15.27:0/1037066462 <== mon.2 v2:172.21.15.45:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde38037070 con 0x7fde48145090 2024-09-12T03:16:45.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.865+0000 7fde467fc640 1 -- 172.21.15.27:0/1037066462 <== mon.2 v2:172.21.15.45:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde3802ae50 con 0x7fde48145090 2024-09-12T03:16:45.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.865+0000 7fde467fc640 1 -- 172.21.15.27:0/1037066462 <== mon.2 v2:172.21.15.45:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde38032660 con 0x7fde48145090 2024-09-12T03:16:45.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.865+0000 7fde4de27640 1 -- 172.21.15.27:0/1037066462 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fde48145090 msgr2=0x7fde480affb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:45.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.865+0000 7fde4de27640 1 --2- 172.21.15.27:0/1037066462 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fde48145090 0x7fde480affb0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fde380097a0 tx=0x7fde3802a100 comp rx=0 tx=0).stop 2024-09-12T03:16:45.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.865+0000 7fde4de27640 1 -- 172.21.15.27:0/1037066462 shutdown_connections 2024-09-12T03:16:45.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.865+0000 7fde4de27640 1 --2- 172.21.15.27:0/1037066462 >> v2:172.21.15.45:3300/0 conn(0x7fde4806bb30 0x7fde4806bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:45.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.865+0000 7fde4de27640 1 --2- 172.21.15.27:0/1037066462 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fde48145090 0x7fde480affb0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:45.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.865+0000 7fde4de27640 1 -- 172.21.15.27:0/1037066462 >> 172.21.15.27:0/1037066462 conn(0x7fde4806d270 msgr2=0x7fde480b3270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:45.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.865+0000 7fde4de27640 1 -- 172.21.15.27:0/1037066462 shutdown_connections 2024-09-12T03:16:45.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.865+0000 7fde4de27640 1 -- 172.21.15.27:0/1037066462 wait complete. 2024-09-12T03:16:45.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.866+0000 7fde4de27640 1 Processor -- start 2024-09-12T03:16:45.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.866+0000 7fde4de27640 1 -- start start 2024-09-12T03:16:45.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.867+0000 7fde4de27640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fde4806bb30 0x7fde4813aff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:45.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.867+0000 7fde4de27640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fde480ae8d0 0x7fde4813e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:45.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.867+0000 7fde4de27640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fde48145090 0x7fde481433f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:45.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.867+0000 7fde4de27640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fde4815fde0 con 0x7fde4806bb30 2024-09-12T03:16:45.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.867+0000 7fde4de27640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fde4815ef50 con 0x7fde48145090 2024-09-12T03:16:45.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.867+0000 7fde4de27640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fde480b04f0 con 0x7fde480ae8d0 2024-09-12T03:16:45.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.867+0000 7fde477fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fde4806bb30 0x7fde4813aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:45.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.867+0000 7fde477fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fde4806bb30 0x7fde4813aff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33556/0 (socket says 172.21.15.27:33556) 2024-09-12T03:16:45.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.867+0000 7fde477fe640 1 -- 172.21.15.27:0/1387193990 learned_addr learned my addr 172.21.15.27:0/1387193990 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:16:45.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.868+0000 7fde46ffd640 1 --2- 172.21.15.27:0/1387193990 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fde480ae8d0 0x7fde4813e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:45.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.868+0000 7fde47fff640 1 --2- 172.21.15.27:0/1387193990 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fde48145090 0x7fde481433f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:45.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.868+0000 7fde477fe640 1 -- 172.21.15.27:0/1387193990 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fde480ae8d0 msgr2=0x7fde4813e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:45.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.868+0000 7fde477fe640 1 --2- 172.21.15.27:0/1387193990 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fde480ae8d0 0x7fde4813e4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:45.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.868+0000 7fde477fe640 1 -- 172.21.15.27:0/1387193990 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fde48145090 msgr2=0x7fde481433f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:45.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.868+0000 7fde477fe640 1 --2- 172.21.15.27:0/1387193990 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fde48145090 0x7fde481433f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:45.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.868+0000 7fde477fe640 1 -- 172.21.15.27:0/1387193990 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde48142b00 con 0x7fde4806bb30 2024-09-12T03:16:45.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.868+0000 7fde46ffd640 1 --2- 172.21.15.27:0/1387193990 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fde480ae8d0 0x7fde4813e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:16:45.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.868+0000 7fde47fff640 1 --2- 172.21.15.27:0/1387193990 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fde48145090 0x7fde481433f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:16:45.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.868+0000 7fde477fe640 1 --2- 172.21.15.27:0/1387193990 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fde4806bb30 0x7fde4813aff0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fde38035040 tx=0x7fde380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:45.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.868+0000 7fde2ffff640 1 -- 172.21.15.27:0/1387193990 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde38041070 con 0x7fde4806bb30 2024-09-12T03:16:45.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.868+0000 7fde2ffff640 1 -- 172.21.15.27:0/1387193990 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde3803c460 con 0x7fde4806bb30 2024-09-12T03:16:45.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.868+0000 7fde4de27640 1 -- 172.21.15.27:0/1387193990 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde48142540 con 0x7fde4806bb30 2024-09-12T03:16:45.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.868+0000 7fde2ffff640 1 -- 172.21.15.27:0/1387193990 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde38037040 con 0x7fde4806bb30 2024-09-12T03:16:45.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.868+0000 7fde4de27640 1 -- 172.21.15.27:0/1387193990 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde4813b8e0 con 0x7fde4806bb30 2024-09-12T03:16:45.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.870+0000 7fde2ffff640 1 -- 172.21.15.27:0/1387193990 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fde3803a070 con 0x7fde4806bb30 2024-09-12T03:16:45.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.870+0000 7fde2ffff640 1 --2- 172.21.15.27:0/1387193990 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fde3406be30 0x7fde3406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:45.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.870+0000 7fde2ffff640 1 -- 172.21.15.27:0/1387193990 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fde380ab030 con 0x7fde4806bb30 2024-09-12T03:16:45.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.871+0000 7fde46ffd640 1 --2- 172.21.15.27:0/1387193990 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fde3406be30 0x7fde3406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:45.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.871+0000 7fde46ffd640 1 --2- 172.21.15.27:0/1387193990 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fde3406be30 0x7fde3406e2f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fde4813db70 tx=0x7fde3c008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:45.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.871+0000 7fde50bc5740 1 Processor -- start 2024-09-12T03:16:45.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.871+0000 7fde50bc5740 1 -- start start 2024-09-12T03:16:45.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.872+0000 7fde50bc5740 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x55badab1e2c0 0x55badab63a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:45.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.872+0000 7fde50bc5740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55badab1ba80 0x55badab1e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:45.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.872+0000 7fde50bc5740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55badab1fa50 0x55badab188b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:45.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.872+0000 7fde50bc5740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55badaa80520 con 0x55badab1ba80 2024-09-12T03:16:45.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.872+0000 7fde50bc5740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55badaa283e0 con 0x55badab1fa50 2024-09-12T03:16:45.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.872+0000 7fde50bc5740 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x55badaa7c4e0 con 0x55badab1e2c0 2024-09-12T03:16:45.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.872+0000 7fde46ffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55badab1ba80 0x55badab1e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:45.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.872+0000 7fde46ffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55badab1ba80 0x55badab1e6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33568/0 (socket says 172.21.15.27:33568) 2024-09-12T03:16:45.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.872+0000 7fde46ffd640 1 -- 172.21.15.27:0/1294686645 learned_addr learned my addr 172.21.15.27:0/1294686645 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:16:45.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.872+0000 7fde477fe640 1 --2- 172.21.15.27:0/1294686645 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x55badab1e2c0 0x55badab63a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:45.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.872+0000 7fde47fff640 1 --2- 172.21.15.27:0/1294686645 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55badab1fa50 0x55badab188b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:45.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.872+0000 7fde46ffd640 1 -- 172.21.15.27:0/1294686645 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x55badab1e2c0 msgr2=0x55badab63a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:45.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.872+0000 7fde46ffd640 1 --2- 172.21.15.27:0/1294686645 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x55badab1e2c0 0x55badab63a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:45.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.872+0000 7fde46ffd640 1 -- 172.21.15.27:0/1294686645 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55badab1fa50 msgr2=0x55badab188b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:45.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.872+0000 7fde46ffd640 1 --2- 172.21.15.27:0/1294686645 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55badab1fa50 0x55badab188b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:45.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.872+0000 7fde46ffd640 1 -- 172.21.15.27:0/1294686645 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55badaa432f0 con 0x55badab1ba80 2024-09-12T03:16:45.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.872+0000 7fde47fff640 1 --2- 172.21.15.27:0/1294686645 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55badab1fa50 0x55badab188b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:16:45.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.873+0000 7fde46ffd640 1 --2- 172.21.15.27:0/1294686645 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55badab1ba80 0x55badab1e6a0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fde3c010a90 tx=0x7fde3c013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d0f07b9b74de03b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:45.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.873+0000 7fde2d7fa640 1 -- 172.21.15.27:0/1294686645 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde3c01c050 con 0x55badab1ba80 2024-09-12T03:16:45.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.873+0000 7fde2d7fa640 1 -- 172.21.15.27:0/1294686645 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde3c026070 con 0x55badab1ba80 2024-09-12T03:16:45.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.873+0000 7fde2d7fa640 1 -- 172.21.15.27:0/1294686645 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde3c00f040 con 0x55badab1ba80 2024-09-12T03:16:45.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.873+0000 7fde50bc5740 1 -- 172.21.15.27:0/1294686645 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55badab1ba80 msgr2=0x55badab1e6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:45.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.873+0000 7fde50bc5740 1 --2- 172.21.15.27:0/1294686645 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55badab1ba80 0x55badab1e6a0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fde3c010a90 tx=0x7fde3c013000 comp rx=0 tx=0).stop 2024-09-12T03:16:45.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.874+0000 7fde50bc5740 1 -- 172.21.15.27:0/1294686645 shutdown_connections 2024-09-12T03:16:45.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.874+0000 7fde50bc5740 1 --2- 172.21.15.27:0/1294686645 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55badab1fa50 0x55badab188b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:45.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.874+0000 7fde50bc5740 1 --2- 172.21.15.27:0/1294686645 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55badab1ba80 0x55badab1e6a0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:45.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.874+0000 7fde50bc5740 1 --2- 172.21.15.27:0/1294686645 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x55badab1e2c0 0x55badab63a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:45.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.874+0000 7fde50bc5740 1 -- 172.21.15.27:0/1294686645 >> 172.21.15.27:0/1294686645 conn(0x55badab1aa80 msgr2=0x55badab62f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:45.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.874+0000 7fde50bc5740 1 -- 172.21.15.27:0/1294686645 shutdown_connections 2024-09-12T03:16:45.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.874+0000 7fde50bc5740 1 -- 172.21.15.27:0/1294686645 wait complete. 2024-09-12T03:16:45.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.874+0000 7fde50bc5740 1 -- start start 2024-09-12T03:16:45.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.874+0000 7fde50bc5740 1 Processor -- start 2024-09-12T03:16:45.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.875+0000 7fde50bc5740 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x55badab1e2c0 0x55badab3e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:45.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.875+0000 7fde50bc5740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55badab1fa50 0x55badab2cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:45.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.875+0000 7fde50bc5740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55badab3ec30 0x55badab2d220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:45.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.875+0000 7fde50bc5740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55badab19f20 con 0x55badab1fa50 2024-09-12T03:16:45.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.875+0000 7fde50bc5740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55badaa09450 con 0x55badab3ec30 2024-09-12T03:16:45.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.875+0000 7fde50bc5740 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x55badab64290 con 0x55badab1e2c0 2024-09-12T03:16:45.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.875+0000 7fde50bc5740 10 client.0 _ll_register_callbacks cb 0x55bada792ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:16:45.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.875+0000 7fde46ffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55badab1fa50 0x55badab2cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:45.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.875+0000 7fde46ffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55badab1fa50 0x55badab2cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33572/0 (socket says 172.21.15.27:33572) 2024-09-12T03:16:45.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.875+0000 7fde46ffd640 1 -- 172.21.15.27:0/6871255 learned_addr learned my addr 172.21.15.27:0/6871255 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:16:45.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.875+0000 7fde477fe640 1 --2- 172.21.15.27:0/6871255 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x55badab1e2c0 0x55badab3e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:45.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.875+0000 7fde47fff640 1 --2- 172.21.15.27:0/6871255 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55badab3ec30 0x55badab2d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:45.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.875+0000 7fde46ffd640 1 -- 172.21.15.27:0/6871255 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x55badab1e2c0 msgr2=0x55badab3e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:45.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.875+0000 7fde46ffd640 1 --2- 172.21.15.27:0/6871255 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x55badab1e2c0 0x55badab3e6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:45.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.875+0000 7fde46ffd640 1 -- 172.21.15.27:0/6871255 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55badab3ec30 msgr2=0x55badab2d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:45.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.875+0000 7fde46ffd640 1 --2- 172.21.15.27:0/6871255 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55badab3ec30 0x55badab2d220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:45.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.875+0000 7fde46ffd640 1 -- 172.21.15.27:0/6871255 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bada7932d0 con 0x55badab1fa50 2024-09-12T03:16:45.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.875+0000 7fde47fff640 1 --2- 172.21.15.27:0/6871255 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55badab3ec30 0x55badab2d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:16:45.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.876+0000 7fde46ffd640 1 --2- 172.21.15.27:0/6871255 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55badab1fa50 0x55badab2cce0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fde3c011010 tx=0x7fde3c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=169d43ad03176fdb server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:45.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.876+0000 7fde267fc640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-12T03:16:45.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.876+0000 7fde267fc640 1 -- 172.21.15.27:0/6871255 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde3c030070 con 0x55badab1fa50 2024-09-12T03:16:45.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.876+0000 7fde267fc640 1 -- 172.21.15.27:0/6871255 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fde3c021c80 con 0x55badab1fa50 2024-09-12T03:16:45.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.876+0000 7fde50bc5740 1 -- 172.21.15.27:0/6871255 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fde3c026070 con 0x55badab1fa50 2024-09-12T03:16:45.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.876+0000 7fde267fc640 1 -- 172.21.15.27:0/6871255 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=6) ==== 24+0+0 (secure 0 0 0) 0x7fde3c01c020 con 0x55badab1fa50 2024-09-12T03:16:45.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.876+0000 7fde50bc5740 10 client.4623 fetch_fsmap learned FSMap version 6 2024-09-12T03:16:45.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.876+0000 7fde50bc5740 1 -- 172.21.15.27:0/6871255 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=6,monmap=2+}) -- 0x55bada85d6b0 con 0x55badab1fa50 2024-09-12T03:16:45.886 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.876+0000 7fde267fc640 1 -- 172.21.15.27:0/6871255 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde3c026070 con 0x55badab1fa50 2024-09-12T03:16:45.886 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.876+0000 7fde267fc640 1 -- 172.21.15.27:0/6871255 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 6) ==== 3502+0+0 (secure 0 0 0) 0x7fde3c016420 con 0x55badab1fa50 2024-09-12T03:16:45.886 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.876+0000 7fde50bc5740 10 client.4623 fetch_fsmap finished waiting for FSMap version 6 2024-09-12T03:16:45.886 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.876+0000 7fde50bc5740 10 client.4623 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-12T03:16:45.886 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.876+0000 7fde50bc5740 20 client.4623 populate_metadata read hostname 'smithi027' 2024-09-12T03:16:45.886 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.876+0000 7fde50bc5740 1 --2- 172.21.15.27:0/6871255 >> [v2:172.21.15.45:6835/1035926332,v1:172.21.15.45:6837/1035926332] conn(0x55badab17cb0 0x55badab80750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:45.886 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.876+0000 7fde50bc5740 4 client.4623 mds_command: new command op to 4285 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:16:45.886 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.876+0000 7fde50bc5740 1 -- 172.21.15.27:0/6871255 --> [v2:172.21.15.45:6835/1035926332,v1:172.21.15.45:6837/1035926332] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55badaa8dc50 con 0x55badab17cb0 2024-09-12T03:16:45.886 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.876+0000 7fde477fe640 1 --2- 172.21.15.27:0/6871255 >> [v2:172.21.15.45:6835/1035926332,v1:172.21.15.45:6837/1035926332] conn(0x55badab17cb0 0x55badab80750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:45.886 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.877+0000 7fde477fe640 1 --2- 172.21.15.27:0/6871255 >> [v2:172.21.15.45:6835/1035926332,v1:172.21.15.45:6837/1035926332] conn(0x55badab17cb0 0x55badab80750 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b9509bacd0f95249 server_cookie=ad260ae4049490b4 in_seq=0 out_seq=0 2024-09-12T03:16:45.886 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.877+0000 7fde267fc640 10 client.4623 ms_handle_connect on v2:172.21.15.45:6835/1035926332 2024-09-12T03:16:45.886 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.878+0000 7fde267fc640 1 -- 172.21.15.27:0/6871255 <== mds.0 v2:172.21.15.45:6835/1035926332 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55badaa8dc50 con 0x55badab17cb0 2024-09-12T03:16:45.886 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.878+0000 7fde267fc640 10 client.4623 handle_command_reply: tid=0 2024-09-12T03:16:45.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.878+0000 7fde50bc5740 1 client.4623 shutdown 2024-09-12T03:16:45.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.878+0000 7fde50bc5740 1 -- 172.21.15.27:0/6871255 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55badab1fa50 msgr2=0x55badab2cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:45.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.878+0000 7fde50bc5740 1 --2- 172.21.15.27:0/6871255 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55badab1fa50 0x55badab2cce0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fde3c011010 tx=0x7fde3c0046c0 comp rx=0 tx=0).stop 2024-09-12T03:16:45.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.878+0000 7fde50bc5740 1 -- 172.21.15.27:0/6871255 shutdown_connections 2024-09-12T03:16:45.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.878+0000 7fde50bc5740 1 --2- 172.21.15.27:0/6871255 >> [v2:172.21.15.45:6835/1035926332,v1:172.21.15.45:6837/1035926332] conn(0x55badab17cb0 0x55badab80750 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:45.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.878+0000 7fde50bc5740 1 --2- 172.21.15.27:0/6871255 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55badab3ec30 0x55badab2d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:45.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.878+0000 7fde50bc5740 1 --2- 172.21.15.27:0/6871255 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55badab1fa50 0x55badab2cce0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:45.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.878+0000 7fde50bc5740 1 --2- 172.21.15.27:0/6871255 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x55badab1e2c0 0x55badab3e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:45.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.878+0000 7fde267fc640 0 client.4623 ms_handle_reset on v2:172.21.15.45:6835/1035926332 2024-09-12T03:16:45.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.878+0000 7fde50bc5740 1 -- 172.21.15.27:0/6871255 >> 172.21.15.27:0/6871255 conn(0x55badab1ba80 msgr2=0x55badab66160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:45.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.878+0000 7fde50bc5740 1 -- 172.21.15.27:0/6871255 shutdown_connections 2024-09-12T03:16:45.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.878+0000 7fde50bc5740 1 -- 172.21.15.27:0/6871255 wait complete. 2024-09-12T03:16:45.889 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.878+0000 7fde50bc5740 20 client.4623 trim_cache size 0 max 16384 2024-09-12T03:16:45.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.897+0000 7fde50bc5740 1 Processor -- start 2024-09-12T03:16:45.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.897+0000 7fde50bc5740 1 -- start start 2024-09-12T03:16:45.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.897+0000 7fde50bc5740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55badab1e2c0 0x55badabdac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:45.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.897+0000 7fde50bc5740 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x55badab1fa50 0x55badab75670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:45.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.898+0000 7fde50bc5740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55badab3ec30 0x55badab75bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:45.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.898+0000 7fde50bc5740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fde481ffdf0 con 0x55badab3ec30 2024-09-12T03:16:45.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.898+0000 7fde50bc5740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fde481f9060 con 0x55badab1e2c0 2024-09-12T03:16:45.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.898+0000 7fde50bc5740 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fde481ff7a0 con 0x55badab1fa50 2024-09-12T03:16:45.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.898+0000 7fde47fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55badab3ec30 0x55badab75bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:45.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.898+0000 7fde47fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55badab3ec30 0x55badab75bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33588/0 (socket says 172.21.15.27:33588) 2024-09-12T03:16:45.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.898+0000 7fde47fff640 1 -- 172.21.15.27:0/54309963 learned_addr learned my addr 172.21.15.27:0/54309963 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:16:45.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.898+0000 7fde46ffd640 1 --2- 172.21.15.27:0/54309963 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x55badab1fa50 0x55badab75670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:45.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.898+0000 7fde477fe640 1 --2- 172.21.15.27:0/54309963 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55badab1e2c0 0x55badabdac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:45.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.898+0000 7fde47fff640 1 -- 172.21.15.27:0/54309963 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x55badab1fa50 msgr2=0x55badab75670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:45.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.898+0000 7fde47fff640 1 --2- 172.21.15.27:0/54309963 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x55badab1fa50 0x55badab75670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:45.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.898+0000 7fde47fff640 1 -- 172.21.15.27:0/54309963 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55badab1e2c0 msgr2=0x55badabdac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:45.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.898+0000 7fde47fff640 1 --2- 172.21.15.27:0/54309963 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55badab1e2c0 0x55badabdac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:45.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.898+0000 7fde47fff640 1 -- 172.21.15.27:0/54309963 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55badab48e60 con 0x55badab3ec30 2024-09-12T03:16:45.902 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.898+0000 7fde46ffd640 1 --2- 172.21.15.27:0/54309963 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x55badab1fa50 0x55badab75670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:16:45.902 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.898+0000 7fde47fff640 1 --2- 172.21.15.27:0/54309963 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55badab3ec30 0x55badab75bb0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fde40001ce0 tx=0x7fde400021a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec76ab73940d6e93 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:45.902 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.898+0000 7fde2cff9640 1 -- 172.21.15.27:0/54309963 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde40007a20 con 0x55badab3ec30 2024-09-12T03:16:45.902 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.898+0000 7fde2cff9640 1 -- 172.21.15.27:0/54309963 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde40015430 con 0x55badab3ec30 2024-09-12T03:16:45.902 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.898+0000 7fde2cff9640 1 -- 172.21.15.27:0/54309963 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde40015620 con 0x55badab3ec30 2024-09-12T03:16:45.902 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.899+0000 7fde50bc5740 1 -- 172.21.15.27:0/54309963 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55badab3ec30 msgr2=0x55badab75bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:45.902 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.899+0000 7fde50bc5740 1 --2- 172.21.15.27:0/54309963 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55badab3ec30 0x55badab75bb0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fde40001ce0 tx=0x7fde400021a0 comp rx=0 tx=0).stop 2024-09-12T03:16:45.902 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.899+0000 7fde50bc5740 1 -- 172.21.15.27:0/54309963 shutdown_connections 2024-09-12T03:16:45.902 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.899+0000 7fde50bc5740 1 --2- 172.21.15.27:0/54309963 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55badab3ec30 0x55badab75bb0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:45.902 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.899+0000 7fde50bc5740 1 --2- 172.21.15.27:0/54309963 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x55badab1fa50 0x55badab75670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:45.902 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.899+0000 7fde50bc5740 1 --2- 172.21.15.27:0/54309963 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55badab1e2c0 0x55badabdac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:45.902 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.899+0000 7fde50bc5740 1 -- 172.21.15.27:0/54309963 >> 172.21.15.27:0/54309963 conn(0x55badab1ba80 msgr2=0x55badab3e310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:45.902 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.899+0000 7fde50bc5740 1 -- 172.21.15.27:0/54309963 shutdown_connections 2024-09-12T03:16:45.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.899+0000 7fde50bc5740 1 -- 172.21.15.27:0/54309963 wait complete. 2024-09-12T03:16:45.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.900+0000 7fde50bc5740 1 -- start start 2024-09-12T03:16:45.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.900+0000 7fde50bc5740 1 Processor -- start 2024-09-12T03:16:45.906 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.900+0000 7fde50bc5740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55badab1e2c0 0x55badabd4770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:45.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.900+0000 7fde50bc5740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55badab1fa50 0x55badab75670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:45.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.900+0000 7fde50bc5740 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x55badab3ec30 0x55badab75bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:45.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.900+0000 7fde50bc5740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55badab196f0 con 0x55badab1e2c0 2024-09-12T03:16:45.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.900+0000 7fde50bc5740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55badab2d990 con 0x55badab1fa50 2024-09-12T03:16:45.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.900+0000 7fde50bc5740 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x55badab40c30 con 0x55badab3ec30 2024-09-12T03:16:45.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.900+0000 7fde50bc5740 10 client.0 _ll_register_callbacks cb 0x7fde38009400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:16:45.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.900+0000 7fde477fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55badab1e2c0 0x55badabd4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:45.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.900+0000 7fde477fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55badab1e2c0 0x55badabd4770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33590/0 (socket says 172.21.15.27:33590) 2024-09-12T03:16:45.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.900+0000 7fde477fe640 1 -- 172.21.15.27:0/3344455398 learned_addr learned my addr 172.21.15.27:0/3344455398 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:16:45.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.901+0000 7fde477fe640 1 -- 172.21.15.27:0/3344455398 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x55badab3ec30 msgr2=0x55badab75bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:45.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.901+0000 7fde46ffd640 1 --2- 172.21.15.27:0/3344455398 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55badab1fa50 0x55badab75670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:45.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.901+0000 7fde47fff640 1 --2- 172.21.15.27:0/3344455398 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x55badab3ec30 0x55badab75bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:45.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.901+0000 7fde477fe640 1 --2- 172.21.15.27:0/3344455398 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x55badab3ec30 0x55badab75bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:45.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.901+0000 7fde477fe640 1 -- 172.21.15.27:0/3344455398 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55badab1fa50 msgr2=0x55badab75670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:45.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.901+0000 7fde477fe640 1 --2- 172.21.15.27:0/3344455398 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55badab1fa50 0x55badab75670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:45.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.901+0000 7fde477fe640 1 -- 172.21.15.27:0/3344455398 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55badab66500 con 0x55badab1e2c0 2024-09-12T03:16:45.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.901+0000 7fde477fe640 1 --2- 172.21.15.27:0/3344455398 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55badab1e2c0 0x55badabd4770 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fde38079070 tx=0x7fde38008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d647ead63df911cb server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:45.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.901+0000 7fde267fc640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-12T03:16:45.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.901+0000 7fde267fc640 1 -- 172.21.15.27:0/3344455398 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde380a6020 con 0x55badab1e2c0 2024-09-12T03:16:45.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.901+0000 7fde267fc640 1 -- 172.21.15.27:0/3344455398 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fde38081ac0 con 0x55badab1e2c0 2024-09-12T03:16:45.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.901+0000 7fde50bc5740 1 -- 172.21.15.27:0/3344455398 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fde40015430 con 0x55badab1e2c0 2024-09-12T03:16:45.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.901+0000 7fde267fc640 1 -- 172.21.15.27:0/3344455398 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=6) ==== 24+0+0 (secure 0 0 0) 0x7fde38098050 con 0x55badab1e2c0 2024-09-12T03:16:45.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.901+0000 7fde50bc5740 10 client.4629 fetch_fsmap learned FSMap version 6 2024-09-12T03:16:45.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.901+0000 7fde50bc5740 1 -- 172.21.15.27:0/3344455398 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=6,monmap=2+}) -- 0x55badabd4cb0 con 0x55badab1e2c0 2024-09-12T03:16:45.909 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.901+0000 7fde267fc640 1 -- 172.21.15.27:0/3344455398 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde40015430 con 0x55badab1e2c0 2024-09-12T03:16:45.909 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.901+0000 7fde267fc640 1 -- 172.21.15.27:0/3344455398 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 6) ==== 3502+0+0 (secure 0 0 0) 0x7fde380981f0 con 0x55badab1e2c0 2024-09-12T03:16:45.909 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.901+0000 7fde50bc5740 10 client.4629 fetch_fsmap finished waiting for FSMap version 6 2024-09-12T03:16:45.909 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.901+0000 7fde50bc5740 10 client.4629 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-12T03:16:45.909 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.901+0000 7fde50bc5740 20 client.4629 populate_metadata read hostname 'smithi027' 2024-09-12T03:16:45.909 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.901+0000 7fde50bc5740 1 --2- 172.21.15.27:0/3344455398 >> [v2:172.21.15.45:6835/1035926332,v1:172.21.15.45:6837/1035926332] conn(0x55badabd5790 0x55badab768c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:45.909 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.901+0000 7fde50bc5740 4 client.4629 mds_command: new command op to 4285 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-12T03:16:45.909 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.901+0000 7fde50bc5740 1 -- 172.21.15.27:0/3344455398 --> [v2:172.21.15.45:6835/1035926332,v1:172.21.15.45:6837/1035926332] -- command(tid 0: {"prefix": "status"}) -- 0x55badaa87fa0 con 0x55badabd5790 2024-09-12T03:16:45.909 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.902+0000 7fde46ffd640 1 --2- 172.21.15.27:0/3344455398 >> [v2:172.21.15.45:6835/1035926332,v1:172.21.15.45:6837/1035926332] conn(0x55badabd5790 0x55badab768c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:45.909 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.902+0000 7fde46ffd640 1 --2- 172.21.15.27:0/3344455398 >> [v2:172.21.15.45:6835/1035926332,v1:172.21.15.45:6837/1035926332] conn(0x55badabd5790 0x55badab768c0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=56e3d63134789e89 server_cookie=63401d07ff652a5d in_seq=0 out_seq=0 2024-09-12T03:16:45.909 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.902+0000 7fde267fc640 10 client.4629 ms_handle_connect on v2:172.21.15.45:6835/1035926332 2024-09-12T03:16:45.909 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.902+0000 7fde267fc640 1 -- 172.21.15.27:0/3344455398 <== mds.0 v2:172.21.15.45:6835/1035926332 1 ==== command_reply(tid 0: 0 ) ==== 8+0+318 (crc 0 0 0) 0x55badaa87fa0 con 0x55badabd5790 2024-09-12T03:16:45.910 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.902+0000 7fde267fc640 10 client.4629 handle_command_reply: tid=0 2024-09-12T03:16:45.910 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.902+0000 7fde50bc5740 1 client.4629 shutdown 2024-09-12T03:16:45.910 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.902+0000 7fde50bc5740 1 -- 172.21.15.27:0/3344455398 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55badab1e2c0 msgr2=0x55badabd4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:45.910 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.902+0000 7fde50bc5740 1 --2- 172.21.15.27:0/3344455398 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55badab1e2c0 0x55badabd4770 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fde38079070 tx=0x7fde38008040 comp rx=0 tx=0).stop 2024-09-12T03:16:45.910 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.902+0000 7fde50bc5740 1 -- 172.21.15.27:0/3344455398 shutdown_connections 2024-09-12T03:16:45.910 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.902+0000 7fde50bc5740 1 --2- 172.21.15.27:0/3344455398 >> [v2:172.21.15.45:6835/1035926332,v1:172.21.15.45:6837/1035926332] conn(0x55badabd5790 0x55badab768c0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:45.910 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.902+0000 7fde50bc5740 1 --2- 172.21.15.27:0/3344455398 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x55badab3ec30 0x55badab75bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:45.910 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.902+0000 7fde50bc5740 1 --2- 172.21.15.27:0/3344455398 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55badab1fa50 0x55badab75670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:45.910 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.902+0000 7fde267fc640 0 client.4629 ms_handle_reset on v2:172.21.15.45:6835/1035926332 2024-09-12T03:16:45.910 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.902+0000 7fde50bc5740 1 --2- 172.21.15.27:0/3344455398 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55badab1e2c0 0x55badabd4770 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:45.910 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.902+0000 7fde50bc5740 1 -- 172.21.15.27:0/3344455398 >> 172.21.15.27:0/3344455398 conn(0x55badab1ba80 msgr2=0x55badab62ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:45.910 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.902+0000 7fde50bc5740 1 -- 172.21.15.27:0/3344455398 shutdown_connections 2024-09-12T03:16:45.910 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.902+0000 7fde50bc5740 1 -- 172.21.15.27:0/3344455398 wait complete. 2024-09-12T03:16:45.911 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.903+0000 7fde50bc5740 20 client.4629 trim_cache size 0 max 16384 2024-09-12T03:16:45.911 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-12T03:16:45.911 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_fsid": "d2df504e-2647-4385-a5db-9fdf69f487a0", 2024-09-12T03:16:45.911 INFO:teuthology.orchestra.run.smithi027.stdout: "whoami": 0, 2024-09-12T03:16:45.911 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 4285, 2024-09-12T03:16:45.911 INFO:teuthology.orchestra.run.smithi027.stdout: "want_state": "up:active", 2024-09-12T03:16:45.911 INFO:teuthology.orchestra.run.smithi027.stdout: "state": "up:active", 2024-09-12T03:16:45.911 INFO:teuthology.orchestra.run.smithi027.stdout: "fs_name": "cephfs", 2024-09-12T03:16:45.911 INFO:teuthology.orchestra.run.smithi027.stdout: "rank_uptime": 14.343284076, 2024-09-12T03:16:45.911 INFO:teuthology.orchestra.run.smithi027.stdout: "mdsmap_epoch": 6, 2024-09-12T03:16:45.912 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch": 21, 2024-09-12T03:16:45.912 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch_barrier": 21, 2024-09-12T03:16:45.912 INFO:teuthology.orchestra.run.smithi027.stdout: "uptime": 20.349116706 2024-09-12T03:16:45.912 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-12T03:16:45.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.903+0000 7fde4de27640 1 -- 172.21.15.27:0/1387193990 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fde3406be30 msgr2=0x7fde3406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:45.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.903+0000 7fde4de27640 1 --2- 172.21.15.27:0/1387193990 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fde3406be30 0x7fde3406e2f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fde4813db70 tx=0x7fde3c008040 comp rx=0 tx=0).stop 2024-09-12T03:16:45.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.903+0000 7fde4de27640 1 -- 172.21.15.27:0/1387193990 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fde4806bb30 msgr2=0x7fde4813aff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:45.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.903+0000 7fde4de27640 1 --2- 172.21.15.27:0/1387193990 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fde4806bb30 0x7fde4813aff0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fde38035040 tx=0x7fde380046c0 comp rx=0 tx=0).stop 2024-09-12T03:16:45.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.903+0000 7fde4de27640 1 -- 172.21.15.27:0/1387193990 shutdown_connections 2024-09-12T03:16:45.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.903+0000 7fde4de27640 1 --2- 172.21.15.27:0/1387193990 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fde3406be30 0x7fde3406e2f0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:45.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.903+0000 7fde4de27640 1 --2- 172.21.15.27:0/1387193990 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fde48145090 0x7fde481433f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:45.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.903+0000 7fde4de27640 1 --2- 172.21.15.27:0/1387193990 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fde480ae8d0 0x7fde4813e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:45.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.903+0000 7fde4de27640 1 --2- 172.21.15.27:0/1387193990 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fde4806bb30 0x7fde4813aff0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:45.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.903+0000 7fde4de27640 1 -- 172.21.15.27:0/1387193990 >> 172.21.15.27:0/1387193990 conn(0x7fde4806d270 msgr2=0x7fde48160840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:45.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.903+0000 7fde4de27640 1 -- 172.21.15.27:0/1387193990 shutdown_connections 2024-09-12T03:16:45.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:45.903+0000 7fde4de27640 1 -- 172.21.15.27:0/1387193990 wait complete. 2024-09-12T03:16:45.915 INFO:teuthology.run_tasks:Running task sequential... 2024-09-12T03:16:45.926 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-09-12T03:16:45.927 INFO:teuthology.task.sequential:In sequential, running task print... 2024-09-12T03:16:45.927 INFO:teuthology.task.print:Enabling mgr modules 2024-09-12T03:16:45.927 INFO:teuthology.run_tasks:Running task kclient... 2024-09-12T03:16:45.980 INFO:tasks.kclient:Mounting kernel clients... 2024-09-12T03:16:45.980 INFO:tasks.kclient:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2024-09-12T03:16:45.980 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:16:45.980 DEBUG:teuthology.orchestra.run.smithi027:> ip netns list 2024-09-12T03:16:45.997 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:16:45.997 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip link delete ceph-brx 2024-09-12T03:16:46.064 INFO:teuthology.orchestra.run.smithi027.stderr:Cannot find device "ceph-brx" 2024-09-12T03:16:46.066 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T03:16:46.066 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:16:46.066 DEBUG:teuthology.orchestra.run.smithi027:> ip netns list 2024-09-12T03:16:46.123 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:16:46.123 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip link delete ceph-brx 2024-09-12T03:16:46.189 INFO:teuthology.orchestra.run.smithi027.stderr:Cannot find device "ceph-brx" 2024-09-12T03:16:46.190 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T03:16:46.191 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:16:46.191 DEBUG:teuthology.orchestra.run.smithi045:> ip netns list 2024-09-12T03:16:46.211 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:16:46.211 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link delete ceph-brx 2024-09-12T03:16:46.285 INFO:teuthology.orchestra.run.smithi045.stderr:Cannot find device "ceph-brx" 2024-09-12T03:16:46.286 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T03:16:46.287 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:16:46.287 DEBUG:teuthology.orchestra.run.smithi045:> ip netns list 2024-09-12T03:16:46.346 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:16:46.346 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link delete ceph-brx 2024-09-12T03:16:46.416 INFO:teuthology.orchestra.run.smithi045.stderr:Cannot find device "ceph-brx" 2024-09-12T03:16:46.418 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T03:16:46.418 INFO:tasks.kclient:client.0 config is {'syntax': 'v2'} 2024-09-12T03:16:46.418 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-12T03:16:46.418 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-12T03:16:46.419 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-12T03:16:46.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:46.520+0000 7f59d155b640 1 Processor -- start 2024-09-12T03:16:46.522 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:46.520+0000 7f59d155b640 1 -- start start 2024-09-12T03:16:46.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:46.521+0000 7f59d155b640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f59cc06f0c0 0x7f59cc0b3560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:46.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:46.521+0000 7f59d155b640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f59cc0ce410 con 0x7f59cc15b2b0 2024-09-12T03:16:46.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:46.521+0000 7f59d155b640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f59cc0732c0 con 0x7f59cc072d70 2024-09-12T03:16:46.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:46.521+0000 7f59d155b640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f59cc0720a0 con 0x7f59cc06f0c0 2024-09-12T03:16:46.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:46.521+0000 7f59caffd640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f59cc15b2b0 0x7f59cc072430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:39296/0 (socket says 172.21.15.27:39296) 2024-09-12T03:16:46.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:46.521+0000 7f59caffd640 1 -- 172.21.15.27:0/1237063047 learned_addr learned my addr 172.21.15.27:0/1237063047 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:16:46.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:46.522+0000 7f59cb7fe640 1 --2- 172.21.15.27:0/1237063047 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f59cc06f0c0 0x7f59cc0b3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:46.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:46.522+0000 7f59c9ffb640 1 -- 172.21.15.27:0/1237063047 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3545323489 0 0) 0x7f59c4001110 con 0x7f59cc15b2b0 2024-09-12T03:16:46.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:46.522+0000 7f59c9ffb640 1 -- 172.21.15.27:0/1237063047 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f59b8003520 con 0x7f59cc15b2b0 2024-09-12T03:16:46.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:46.522+0000 7f59c9ffb640 1 -- 172.21.15.27:0/1237063047 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1584783849 0 0) 0x7f59b8003520 con 0x7f59cc15b2b0 2024-09-12T03:16:46.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:46.522+0000 7f59c9ffb640 1 -- 172.21.15.27:0/1237063047 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f59cc06f0c0 msgr2=0x7f59cc0b3560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:46.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:46.522+0000 7f59c9ffb640 1 --2- 172.21.15.27:0/1237063047 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f59cc06f0c0 0x7f59cc0b3560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:46.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:46.522+0000 7f59c9ffb640 1 -- 172.21.15.27:0/1237063047 >> v1:172.21.15.45:6789/0 conn(0x7f59cc072d70 legacy=0x7f59cc06e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:46.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:46.522+0000 7f59c9ffb640 1 -- 172.21.15.27:0/1237063047 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59cc071ae0 con 0x7f59cc15b2b0 2024-09-12T03:16:46.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:46.522+0000 7f59c9ffb640 1 -- 172.21.15.27:0/1237063047 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 847755213 0 0) 0x7f59c4001860 con 0x7f59cc15b2b0 2024-09-12T03:16:46.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:46.522+0000 7f59c9ffb640 1 -- 172.21.15.27:0/1237063047 >> v1:172.21.15.27:6789/0 conn(0x7f59cc15b2b0 legacy=0x7f59cc072430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:46.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:46.522+0000 7f59c9ffb640 1 --2- 172.21.15.27:0/1237063047 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f59b80039e0 0x7f59b8003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:46.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:46.522+0000 7f59c9ffb640 1 --2- 172.21.15.27:0/1237063047 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f59cc06f0c0 0x7f59b8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:46.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:46.522+0000 7f59c9ffb640 1 --2- 172.21.15.27:0/1237063047 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f59b8004d40 0x7f59b80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:46.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:46.522+0000 7f59c9ffb640 1 -- 172.21.15.27:0/1237063047 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f59cc0720a0 con 0x7f59b80039e0 2024-09-12T03:16:46.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:46.522+0000 7f59c9ffb640 1 -- 172.21.15.27:0/1237063047 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f59b800baa0 con 0x7f59b8004d40 2024-09-12T03:16:46.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:46.522+0000 7f59c9ffb640 1 -- 172.21.15.27:0/1237063047 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f59b800bca0 con 0x7f59cc06f0c0 2024-09-12T03:16:46.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:46.522+0000 7f59caffd640 1 --2- 172.21.15.27:0/1237063047 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f59b80039e0 0x7f59b8003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:46.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:46.522+0000 7f59ca7fc640 1 --2- 172.21.15.27:0/1237063047 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f59cc06f0c0 0x7f59b8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:46.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:46.522+0000 7f59cb7fe640 1 --2- 172.21.15.27:0/1237063047 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f59b8004d40 0x7f59b80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:46.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:46.522+0000 7f59caffd640 1 -- 172.21.15.27:0/1237063047 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f59cc06f0c0 msgr2=0x7f59b8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:46.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:46.522+0000 7f59caffd640 1 --2- 172.21.15.27:0/1237063047 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f59cc06f0c0 0x7f59b8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:46.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:46.522+0000 7f59caffd640 1 -- 172.21.15.27:0/1237063047 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f59b8004d40 msgr2=0x7f59b80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:46.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:46.522+0000 7f59caffd640 1 --2- 172.21.15.27:0/1237063047 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f59b8004d40 0x7f59b80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:46.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:46.522+0000 7f59caffd640 1 -- 172.21.15.27:0/1237063047 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59b8003520 con 0x7f59b80039e0 2024-09-12T03:16:46.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:46.522+0000 7f59ca7fc640 1 --2- 172.21.15.27:0/1237063047 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f59cc06f0c0 0x7f59b8004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:16:46.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:46.523+0000 7f59caffd640 1 --2- 172.21.15.27:0/1237063047 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f59b80039e0 0x7f59b8003e00 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f59cc0723c0 tx=0x7f59b8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70c3ff0364283d6d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:46.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:46.523+0000 7f59c9ffb640 1 -- 172.21.15.27:0/1237063047 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59c400c0e0 con 0x7f59b80039e0 2024-09-12T03:16:46.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:46.523+0000 7f59c9ffb640 1 -- 172.21.15.27:0/1237063047 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59c4034070 con 0x7f59b80039e0 2024-09-12T03:16:46.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:46.523+0000 7f59c9ffb640 1 -- 172.21.15.27:0/1237063047 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59c4005d30 con 0x7f59b80039e0 2024-09-12T03:16:46.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:46.524+0000 7f59d155b640 1 -- 172.21.15.27:0/1237063047 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f59b80039e0 msgr2=0x7f59b8003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:46.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:46.524+0000 7f59d155b640 1 --2- 172.21.15.27:0/1237063047 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f59b80039e0 0x7f59b8003e00 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f59cc0723c0 tx=0x7f59b8005830 comp rx=0 tx=0).stop 2024-09-12T03:16:46.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:46.524+0000 7f59cb7fe640 1 -- 172.21.15.27:0/1237063047 reap_dead start 2024-09-12T03:16:46.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:46.524+0000 7f59d155b640 1 -- 172.21.15.27:0/1237063047 shutdown_connections 2024-09-12T03:16:46.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:46.524+0000 7f59d155b640 1 -- 172.21.15.27:0/1237063047 >> 172.21.15.27:0/1237063047 conn(0x7f59cc062c60 msgr2=0x7f59cc06bb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:46.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:46.524+0000 7f59d155b640 1 -- 172.21.15.27:0/1237063047 shutdown_connections 2024-09-12T03:16:46.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:46.524+0000 7f59d155b640 1 -- 172.21.15.27:0/1237063047 wait complete. 2024-09-12T03:16:46.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:46.525+0000 7f59d155b640 1 Processor -- start 2024-09-12T03:16:46.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:46.525+0000 7f59d155b640 1 -- start start 2024-09-12T03:16:46.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:46.526+0000 7f59d155b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f59b80039e0 0x7f59cc0ab490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:46.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:46.526+0000 7f59d155b640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f59cc159de0 0x7f59cc0b2050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:46.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:46.526+0000 7f59d155b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f59cc0aa4c0 0x7f59cc146740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:46.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:46.526+0000 7f59d155b640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f59cc15b690 con 0x7f59b80039e0 2024-09-12T03:16:46.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:46.526+0000 7f59d155b640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f59cc0ce690 con 0x7f59cc0aa4c0 2024-09-12T03:16:46.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:46.526+0000 7f59d155b640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f59cc15b810 con 0x7f59cc159de0 2024-09-12T03:16:46.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:46.526+0000 7f59caffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f59b80039e0 0x7f59cc0ab490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:46.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:46.526+0000 7f59caffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f59b80039e0 0x7f59cc0ab490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33604/0 (socket says 172.21.15.27:33604) 2024-09-12T03:16:46.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:46.526+0000 7f59caffd640 1 -- 172.21.15.27:0/1843114418 learned_addr learned my addr 172.21.15.27:0/1843114418 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:16:46.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:46.526+0000 7f59cb7fe640 1 --2- 172.21.15.27:0/1843114418 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f59cc0aa4c0 0x7f59cc146740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:46.532 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:46.526+0000 7f59ca7fc640 1 --2- 172.21.15.27:0/1843114418 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f59cc159de0 0x7f59cc0b2050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:46.532 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:46.526+0000 7f59caffd640 1 -- 172.21.15.27:0/1843114418 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f59cc159de0 msgr2=0x7f59cc0b2050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:46.532 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:46.526+0000 7f59caffd640 1 --2- 172.21.15.27:0/1843114418 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f59cc159de0 0x7f59cc0b2050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:46.532 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:46.526+0000 7f59caffd640 1 -- 172.21.15.27:0/1843114418 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f59cc0aa4c0 msgr2=0x7f59cc146740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:46.532 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:46.526+0000 7f59caffd640 1 --2- 172.21.15.27:0/1843114418 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f59cc0aa4c0 0x7f59cc146740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:46.532 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:46.526+0000 7f59caffd640 1 -- 172.21.15.27:0/1843114418 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59cc159020 con 0x7f59b80039e0 2024-09-12T03:16:46.532 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:46.526+0000 7f59cb7fe640 1 --2- 172.21.15.27:0/1843114418 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f59cc0aa4c0 0x7f59cc146740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:16:46.532 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:46.526+0000 7f59caffd640 1 --2- 172.21.15.27:0/1843114418 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f59b80039e0 0x7f59cc0ab490 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f59c4001b60 tx=0x7f59cc071cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:46.532 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:46.527+0000 7f59c37fe640 1 -- 172.21.15.27:0/1843114418 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59c4007270 con 0x7f59b80039e0 2024-09-12T03:16:46.532 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:46.527+0000 7f59c37fe640 1 -- 172.21.15.27:0/1843114418 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59c4007430 con 0x7f59b80039e0 2024-09-12T03:16:46.532 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:46.527+0000 7f59d155b640 1 -- 172.21.15.27:0/1843114418 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59cc159300 con 0x7f59b80039e0 2024-09-12T03:16:46.532 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:46.527+0000 7f59c37fe640 1 -- 172.21.15.27:0/1843114418 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59c4007820 con 0x7f59b80039e0 2024-09-12T03:16:46.532 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:46.527+0000 7f59d155b640 1 -- 172.21.15.27:0/1843114418 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59cc0b2590 con 0x7f59b80039e0 2024-09-12T03:16:46.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:46.528+0000 7f59d155b640 1 -- 172.21.15.27:0/1843114418 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59cc15e940 con 0x7f59b80039e0 2024-09-12T03:16:46.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:46.528+0000 7f59c37fe640 1 -- 172.21.15.27:0/1843114418 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f59c402e070 con 0x7f59b80039e0 2024-09-12T03:16:46.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:46.529+0000 7f59c37fe640 1 --2- 172.21.15.27:0/1843114418 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f59ac06be60 0x7f59ac06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:46.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:46.529+0000 7f59c37fe640 1 -- 172.21.15.27:0/1843114418 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f59c40aba20 con 0x7f59b80039e0 2024-09-12T03:16:46.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:46.529+0000 7f59ca7fc640 1 --2- 172.21.15.27:0/1843114418 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f59ac06be60 0x7f59ac06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:46.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:46.529+0000 7f59ca7fc640 1 --2- 172.21.15.27:0/1843114418 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f59ac06be60 0x7f59ac06e320 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f59cc0b3280 tx=0x7f59b4009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:46.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:46.533+0000 7f59c37fe640 1 -- 172.21.15.27:0/1843114418 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f59cc15e940 con 0x7f59b80039e0 2024-09-12T03:16:46.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:46.749+0000 7f59d155b640 1 -- 172.21.15.27:0/1843114418 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f59cc0b3560 con 0x7f59b80039e0 2024-09-12T03:16:46.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:46.750+0000 7f59c37fe640 1 -- 172.21.15.27:0/1843114418 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v6) ==== 53+0+73 (secure 0 0 0) 0x7f59c402c040 con 0x7f59b80039e0 2024-09-12T03:16:46.751 INFO:teuthology.orchestra.run.smithi027.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-12T03:16:46.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:46.752+0000 7f59d155b640 1 -- 172.21.15.27:0/1843114418 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f59ac06be60 msgr2=0x7f59ac06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:46.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:46.752+0000 7f59d155b640 1 --2- 172.21.15.27:0/1843114418 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f59ac06be60 0x7f59ac06e320 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f59cc0b3280 tx=0x7f59b4009040 comp rx=0 tx=0).stop 2024-09-12T03:16:46.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:46.752+0000 7f59d155b640 1 -- 172.21.15.27:0/1843114418 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f59b80039e0 msgr2=0x7f59cc0ab490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:46.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:46.752+0000 7f59d155b640 1 --2- 172.21.15.27:0/1843114418 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f59b80039e0 0x7f59cc0ab490 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f59c4001b60 tx=0x7f59cc071cb0 comp rx=0 tx=0).stop 2024-09-12T03:16:46.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:46.752+0000 7f59d155b640 1 -- 172.21.15.27:0/1843114418 shutdown_connections 2024-09-12T03:16:46.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:46.752+0000 7f59d155b640 1 --2- 172.21.15.27:0/1843114418 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f59ac06be60 0x7f59ac06e320 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:46.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:46.752+0000 7f59d155b640 1 --2- 172.21.15.27:0/1843114418 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f59cc0aa4c0 0x7f59cc146740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:46.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:46.752+0000 7f59d155b640 1 --2- 172.21.15.27:0/1843114418 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f59cc159de0 0x7f59cc0b2050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:46.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:46.752+0000 7f59d155b640 1 --2- 172.21.15.27:0/1843114418 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f59b80039e0 0x7f59cc0ab490 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:46.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:46.752+0000 7f59d155b640 1 -- 172.21.15.27:0/1843114418 >> 172.21.15.27:0/1843114418 conn(0x7f59cc062c60 msgr2=0x7f59cc0b5150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:46.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:46.752+0000 7f59d155b640 1 -- 172.21.15.27:0/1843114418 shutdown_connections 2024-09-12T03:16:46.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:46.752+0000 7f59d155b640 1 -- 172.21.15.27:0/1843114418 wait complete. 2024-09-12T03:16:46.763 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-12T03:16:46.763 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-12T03:16:46.763 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi027.front.sepia.ceph.com 2024-09-12T03:16:46.763 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-12T03:16:46.764 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-12T03:16:46.764 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-12T03:16:46.764 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-12T03:16:46.764 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-12T03:16:46.764 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-12T03:16:46.764 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:16:46.764 DEBUG:teuthology.orchestra.run.smithi027:> ip addr 2024-09-12T03:16:46.780 INFO:teuthology.orchestra.run.smithi027.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-12T03:16:46.780 INFO:teuthology.orchestra.run.smithi027.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-12T03:16:46.780 INFO:teuthology.orchestra.run.smithi027.stdout: inet 127.0.0.1/8 scope host lo 2024-09-12T03:16:46.780 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:16:46.781 INFO:teuthology.orchestra.run.smithi027.stdout: inet6 ::1/128 scope host 2024-09-12T03:16:46.781 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:16:46.781 INFO:teuthology.orchestra.run.smithi027.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-12T03:16:46.781 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether 0c:c4:7a:6b:fe:c8 brd ff:ff:ff:ff:ff:ff 2024-09-12T03:16:46.781 INFO:teuthology.orchestra.run.smithi027.stdout: altname enp7s0f0 2024-09-12T03:16:46.781 INFO:teuthology.orchestra.run.smithi027.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-12T03:16:46.781 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether 0c:c4:7a:6b:fe:c9 brd ff:ff:ff:ff:ff:ff 2024-09-12T03:16:46.781 INFO:teuthology.orchestra.run.smithi027.stdout: altname enp7s0f1 2024-09-12T03:16:46.781 INFO:teuthology.orchestra.run.smithi027.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-12T03:16:46.781 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether ac:1f:6b:f8:39:5a brd ff:ff:ff:ff:ff:ff 2024-09-12T03:16:46.781 INFO:teuthology.orchestra.run.smithi027.stdout: altname enp1s0f0 2024-09-12T03:16:46.781 INFO:teuthology.orchestra.run.smithi027.stdout: inet 172.21.15.27/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-12T03:16:46.782 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft 42782sec preferred_lft 42782sec 2024-09-12T03:16:46.782 INFO:teuthology.orchestra.run.smithi027.stdout: inet6 fe80::ae1f:6bff:fef8:395a/64 scope link 2024-09-12T03:16:46.782 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:16:46.782 INFO:teuthology.orchestra.run.smithi027.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-12T03:16:46.782 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether ac:1f:6b:f8:39:5b brd ff:ff:ff:ff:ff:ff 2024-09-12T03:16:46.782 INFO:teuthology.orchestra.run.smithi027.stdout: altname enp1s0f1 2024-09-12T03:16:46.782 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-12T03:16:46.782 DEBUG:teuthology.orchestra.run.smithi027:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-12T03:16:46.783 DEBUG:teuthology.orchestra.run.smithi027:> set -e 2024-09-12T03:16:46.783 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip link add name ceph-brx type bridge 2024-09-12T03:16:46.783 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip addr flush dev ceph-brx 2024-09-12T03:16:46.783 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip link set ceph-brx up 2024-09-12T03:16:46.783 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-12T03:16:46.783 DEBUG:teuthology.orchestra.run.smithi027:> ') 2024-09-12T03:16:46.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:46 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T03:16:46.981 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T03:16:46.988 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:16:46.988 DEBUG:teuthology.orchestra.run.smithi027:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-12T03:16:47.014 INFO:teuthology.orchestra.run.smithi027.stdout:1 2024-09-12T03:16:47.016 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:16:47.016 DEBUG:teuthology.orchestra.run.smithi027:> route 2024-09-12T03:16:47.088 INFO:teuthology.orchestra.run.smithi027.stdout:Kernel IP routing table 2024-09-12T03:16:47.088 INFO:teuthology.orchestra.run.smithi027.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-12T03:16:47.088 INFO:teuthology.orchestra.run.smithi027.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-12T03:16:47.088 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-12T03:16:47.089 INFO:teuthology.orchestra.run.smithi027.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-12T03:16:47.089 DEBUG:teuthology.orchestra.run.smithi027:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-12T03:16:47.089 DEBUG:teuthology.orchestra.run.smithi027:> set -e 2024-09-12T03:16:47.089 DEBUG:teuthology.orchestra.run.smithi027:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-12T03:16:47.089 DEBUG:teuthology.orchestra.run.smithi027:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-12T03:16:47.089 DEBUG:teuthology.orchestra.run.smithi027:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-12T03:16:47.089 DEBUG:teuthology.orchestra.run.smithi027:> ') 2024-09-12T03:16:47.178 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:47 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T03:16:47.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T03:16:47.397 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:16:47.397 DEBUG:teuthology.orchestra.run.smithi027:> ip netns list 2024-09-12T03:16:47.413 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:16:47.413 DEBUG:teuthology.orchestra.run.smithi027:> ip netns list-id 2024-09-12T03:16:47.469 DEBUG:teuthology.orchestra.run.smithi027:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-12T03:16:47.470 DEBUG:teuthology.orchestra.run.smithi027:> set -e 2024-09-12T03:16:47.470 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-12T03:16:47.470 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-09-12T03:16:47.470 DEBUG:teuthology.orchestra.run.smithi027:> ') 2024-09-12T03:16:47.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:47 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T03:16:47.584 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T03:16:47.591 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-09-12T03:16:47.591 DEBUG:teuthology.orchestra.run.smithi027:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-12T03:16:47.591 DEBUG:teuthology.orchestra.run.smithi027:> set -e 2024-09-12T03:16:47.591 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-09-12T03:16:47.591 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-12T03:16:47.591 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-09-12T03:16:47.591 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-09-12T03:16:47.591 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-09-12T03:16:47.592 DEBUG:teuthology.orchestra.run.smithi027:> ') 2024-09-12T03:16:47.642 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:47 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T03:16:47.711 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T03:16:47.717 DEBUG:teuthology.orchestra.run.smithi027:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-12T03:16:47.717 DEBUG:teuthology.orchestra.run.smithi027:> set -e 2024-09-12T03:16:47.717 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip link set brx.0 up 2024-09-12T03:16:47.717 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip link set dev brx.0 master ceph-brx 2024-09-12T03:16:47.717 DEBUG:teuthology.orchestra.run.smithi027:> ') 2024-09-12T03:16:47.767 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:47 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T03:16:47.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T03:16:47.798 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-12T03:16:47.799 DEBUG:teuthology.orchestra.run.smithi027:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-12T03:16:47.815 INFO:teuthology.orchestra.run.smithi027.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-12T03:16:47.815 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-12T03:16:47.815 DEBUG:teuthology.orchestra.run.smithi027:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-12T03:16:47.870 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-09-12T03:16:47.871 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:16:47.871 DEBUG:teuthology.orchestra.run.smithi027:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-12T03:16:48.007 INFO:teuthology.orchestra.run.smithi027.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-12T03:16:48.007 INFO:teuthology.orchestra.run.smithi027.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-12T03:16:48.128 INFO:teuthology.orchestra.run.smithi027.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-12T03:16:48.128 INFO:teuthology.orchestra.run.smithi027.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-12T03:16:48.128 INFO:teuthology.orchestra.run.smithi027.stdout:mount.ceph: resolved to: "172.21.15.27:3300,172.21.15.45:3300,172.21.15.45:3301" 2024-09-12T03:16:48.129 INFO:teuthology.orchestra.run.smithi027.stdout:mount.ceph: trying mount with new device syntax: 0@d2df504e-2647-4385-a5db-9fdf69f487a0.cephfs=/ 2024-09-12T03:16:48.129 INFO:teuthology.orchestra.run.smithi027.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.27:3300/172.21.15.45:3300/172.21.15.45:3301" will pass to kernel 2024-09-12T03:16:48.130 INFO:teuthology.orchestra.run.smithi027.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-12T03:16:48.130 INFO:teuthology.orchestra.run.smithi027.stdout: You just mounted a file system that supports labels which does not 2024-09-12T03:16:48.130 INFO:teuthology.orchestra.run.smithi027.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-12T03:16:48.130 INFO:teuthology.orchestra.run.smithi027.stdout: applications will generate AVC messages and not be allowed access to 2024-09-12T03:16:48.130 INFO:teuthology.orchestra.run.smithi027.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-12T03:16:48.132 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-12T03:16:48.132 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:16:48.132 DEBUG:teuthology.orchestra.run.smithi027:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-12T03:16:48.158 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-12T03:16:48.251 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:48 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-12T03:16:48.257 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T03:16:48.262 INFO:teuthology.orchestra.run.smithi027.stdout:client4638 2024-09-12T03:16:48.263 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-12T03:16:48.263 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T03:16:48.279 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-12T03:16:48.371 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:48 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-12T03:16:48.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T03:16:48.383 INFO:teuthology.orchestra.run.smithi027.stdout:client4638 2024-09-12T03:16:48.384 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-12T03:16:48.434 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:48 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-12T03:16:48.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T03:16:48.446 INFO:teuthology.orchestra.run.smithi027.stdout:client4638 2024-09-12T03:16:48.446 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/d2df504e-2647-4385-a5db-9fdf69f487a0.client4638/status') 2024-09-12T03:16:48.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:48 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T03:16:48.511 INFO:teuthology.orchestra.run.smithi027.stdout:instance: client.4638 (3)192.168.0.1:0/2741028735 2024-09-12T03:16:48.511 INFO:teuthology.orchestra.run.smithi027.stdout:blocklisted: false 2024-09-12T03:16:48.512 INFO:teuthology.orchestra.run.smithi027.stderr:0+1 records in 2024-09-12T03:16:48.512 INFO:teuthology.orchestra.run.smithi027.stderr:0+1 records out 2024-09-12T03:16:48.512 INFO:teuthology.orchestra.run.smithi027.stderr:69 bytes copied, 2.5106e-05 s, 2.7 MB/s 2024-09-12T03:16:48.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T03:16:48.519 INFO:tasks.kclient:client.2 config is {'syntax': 'v2'} 2024-09-12T03:16:48.519 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-12T03:16:48.519 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-09-12T03:16:48.519 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-12T03:16:48.622 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:48.621+0000 7fdee1fe8640 1 Processor -- start 2024-09-12T03:16:48.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:48.621+0000 7fdee1fe8640 1 -- start start 2024-09-12T03:16:48.624 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:48.622+0000 7fdee1fe8640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fdedc0d52e0 0x7fdedc0cf810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:48.624 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:48.622+0000 7fdee1fe8640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdedc0d1e60 con 0x7fdedc0d3120 2024-09-12T03:16:48.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:48.622+0000 7fdee1fe8640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdedc0d2380 con 0x7fdedc0d1050 2024-09-12T03:16:48.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:48.622+0000 7fdee1fe8640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fdedc0cfef0 con 0x7fdedc0d52e0 2024-09-12T03:16:48.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:48.622+0000 7fdedbfff640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fdedc0d3120 0x7fdedc0d35b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:39306/0 (socket says 172.21.15.27:39306) 2024-09-12T03:16:48.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:48.622+0000 7fdedbfff640 1 -- 172.21.15.27:0/159004410 learned_addr learned my addr 172.21.15.27:0/159004410 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:16:48.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:48.623+0000 7fdedaffd640 1 --2- 172.21.15.27:0/159004410 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fdedc0d52e0 0x7fdedc0cf810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:48.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:48.623+0000 7fdeda7fc640 1 -- 172.21.15.27:0/159004410 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3234642085 0 0) 0x7fdedc0d1e60 con 0x7fdedc0d3120 2024-09-12T03:16:48.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:48.623+0000 7fdeda7fc640 1 -- 172.21.15.27:0/159004410 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdec4003520 con 0x7fdedc0d3120 2024-09-12T03:16:48.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:48.623+0000 7fdeda7fc640 1 -- 172.21.15.27:0/159004410 <== mon.1 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2674480192 0 0) 0x7fdedc0d2380 con 0x7fdedc0d1050 2024-09-12T03:16:48.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:48.623+0000 7fdeda7fc640 1 -- 172.21.15.27:0/159004410 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdedc0d1e60 con 0x7fdedc0d1050 2024-09-12T03:16:48.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:48.623+0000 7fdedaffd640 1 -- 172.21.15.27:0/159004410 >> v1:172.21.15.45:6789/0 conn(0x7fdedc0d1050 legacy=0x7fdedc0d1430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:48.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:48.623+0000 7fdedaffd640 1 -- 172.21.15.27:0/159004410 >> v1:172.21.15.27:6789/0 conn(0x7fdedc0d3120 legacy=0x7fdedc0d35b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:48.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:48.623+0000 7fdeda7fc640 1 -- 172.21.15.27:0/159004410 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3634072794 0 0) 0x7fdec4003520 con 0x7fdedc0d3120 2024-09-12T03:16:48.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:48.623+0000 7fdedaffd640 1 -- 172.21.15.27:0/159004410 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdedc0d2640 con 0x7fdedc0d52e0 2024-09-12T03:16:48.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:48.624+0000 7fdedaffd640 1 --2- 172.21.15.27:0/159004410 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fdedc0d52e0 0x7fdedc0cf810 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fded00097d0 tx=0x7fded002a1a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=52abd56e53e498d7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:48.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:48.624+0000 7fdeda7fc640 1 -- 172.21.15.27:0/159004410 <== mon.2 v2:172.21.15.45:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fded0037070 con 0x7fdedc0d52e0 2024-09-12T03:16:48.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:48.624+0000 7fdeda7fc640 1 -- 172.21.15.27:0/159004410 <== mon.2 v2:172.21.15.45:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fded002ac30 con 0x7fdedc0d52e0 2024-09-12T03:16:48.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:48.624+0000 7fdeda7fc640 1 -- 172.21.15.27:0/159004410 <== mon.2 v2:172.21.15.45:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fded40019f0 con 0x7fdedc0d52e0 2024-09-12T03:16:48.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:48.624+0000 7fdee1fe8640 1 -- 172.21.15.27:0/159004410 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fdedc0d52e0 msgr2=0x7fdedc0cf810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:48.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:48.624+0000 7fdee1fe8640 1 --2- 172.21.15.27:0/159004410 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fdedc0d52e0 0x7fdedc0cf810 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fded00097d0 tx=0x7fded002a1a0 comp rx=0 tx=0).stop 2024-09-12T03:16:48.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:48.624+0000 7fdee1fe8640 1 -- 172.21.15.27:0/159004410 shutdown_connections 2024-09-12T03:16:48.628 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:48.624+0000 7fdee1fe8640 1 --2- 172.21.15.27:0/159004410 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fdedc0d52e0 0x7fdedc0cf810 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:48.628 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:48.624+0000 7fdee1fe8640 1 -- 172.21.15.27:0/159004410 >> 172.21.15.27:0/159004410 conn(0x7fdedc06ddf0 msgr2=0x7fdedc06e200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:48.628 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:48.625+0000 7fdee1fe8640 1 -- 172.21.15.27:0/159004410 shutdown_connections 2024-09-12T03:16:48.628 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:48.625+0000 7fdee1fe8640 1 -- 172.21.15.27:0/159004410 wait complete. 2024-09-12T03:16:48.628 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:48.626+0000 7fdee1fe8640 1 Processor -- start 2024-09-12T03:16:48.628 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:48.626+0000 7fdee1fe8640 1 -- start start 2024-09-12T03:16:48.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:48.627+0000 7fdee1fe8640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdedc0d1050 0x7fdedc15ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:48.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:48.627+0000 7fdee1fe8640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fdedc0d3120 0x7fdedc162940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:48.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:48.627+0000 7fdee1fe8640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdedc0d52e0 0x7fdedc0ccfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:48.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:48.627+0000 7fdee1fe8640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fdedc0b7a50 con 0x7fdedc0d1050 2024-09-12T03:16:48.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:48.627+0000 7fdee1fe8640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fdedc0d2c00 con 0x7fdedc0d52e0 2024-09-12T03:16:48.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:48.627+0000 7fdee1fe8640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fdedc0b7bd0 con 0x7fdedc0d3120 2024-09-12T03:16:48.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:48.627+0000 7fdedb7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdedc0d1050 0x7fdedc15ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:48.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:48.627+0000 7fdedb7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdedc0d1050 0x7fdedc15ee70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33620/0 (socket says 172.21.15.27:33620) 2024-09-12T03:16:48.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:48.627+0000 7fdedb7fe640 1 -- 172.21.15.27:0/390123050 learned_addr learned my addr 172.21.15.27:0/390123050 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:16:48.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:48.627+0000 7fdedaffd640 1 --2- 172.21.15.27:0/390123050 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fdedc0d3120 0x7fdedc162940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:48.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:48.627+0000 7fdedbfff640 1 --2- 172.21.15.27:0/390123050 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdedc0d52e0 0x7fdedc0ccfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:48.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:48.627+0000 7fdedb7fe640 1 -- 172.21.15.27:0/390123050 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fdedc0d3120 msgr2=0x7fdedc162940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:48.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:48.627+0000 7fdedb7fe640 1 --2- 172.21.15.27:0/390123050 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fdedc0d3120 0x7fdedc162940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:48.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:48.627+0000 7fdedb7fe640 1 -- 172.21.15.27:0/390123050 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdedc0d52e0 msgr2=0x7fdedc0ccfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:48.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:48.627+0000 7fdedb7fe640 1 --2- 172.21.15.27:0/390123050 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdedc0d52e0 0x7fdedc0ccfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:48.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:48.627+0000 7fdedb7fe640 1 -- 172.21.15.27:0/390123050 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdedc15f490 con 0x7fdedc0d1050 2024-09-12T03:16:48.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:48.627+0000 7fdedb7fe640 1 --2- 172.21.15.27:0/390123050 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdedc0d1050 0x7fdedc15ee70 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fdecc00c490 tx=0x7fdecc00c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:48.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:48.627+0000 7fdec3fff640 1 -- 172.21.15.27:0/390123050 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdecc0052d0 con 0x7fdedc0d1050 2024-09-12T03:16:48.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:48.627+0000 7fdec3fff640 1 -- 172.21.15.27:0/390123050 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdecc005470 con 0x7fdedc0d1050 2024-09-12T03:16:48.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:48.628+0000 7fdee1fe8640 1 -- 172.21.15.27:0/390123050 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdedc161e60 con 0x7fdedc0d1050 2024-09-12T03:16:48.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:48.628+0000 7fdec3fff640 1 -- 172.21.15.27:0/390123050 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdecc012850 con 0x7fdedc0d1050 2024-09-12T03:16:48.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:48.628+0000 7fdee1fe8640 1 -- 172.21.15.27:0/390123050 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdedc162420 con 0x7fdedc0d1050 2024-09-12T03:16:48.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:48.629+0000 7fdee1fe8640 1 -- 172.21.15.27:0/390123050 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdedc15ec60 con 0x7fdedc0d1050 2024-09-12T03:16:48.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:48.629+0000 7fdec3fff640 1 -- 172.21.15.27:0/390123050 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fdecc0129f0 con 0x7fdedc0d1050 2024-09-12T03:16:48.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:48.629+0000 7fdec3fff640 1 --2- 172.21.15.27:0/390123050 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fdec806be30 0x7fdec806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:48.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:48.629+0000 7fdec3fff640 1 -- 172.21.15.27:0/390123050 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fdecc08a9c0 con 0x7fdedc0d1050 2024-09-12T03:16:48.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:48.630+0000 7fdedaffd640 1 --2- 172.21.15.27:0/390123050 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fdec806be30 0x7fdec806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:48.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:48.630+0000 7fdedaffd640 1 --2- 172.21.15.27:0/390123050 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fdec806be30 0x7fdec806e2f0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fded0034000 tx=0x7fded002e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:48.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:48.634+0000 7fdec3fff640 1 -- 172.21.15.27:0/390123050 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fdecc08ce20 con 0x7fdedc0d1050 2024-09-12T03:16:48.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:48.844+0000 7fdee1fe8640 1 -- 172.21.15.27:0/390123050 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fdedc0b8310 con 0x7fdedc0d1050 2024-09-12T03:16:48.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:48.845+0000 7fdec3fff640 1 -- 172.21.15.27:0/390123050 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v6) ==== 53+0+73 (secure 0 0 0) 0x7fdecc05f020 con 0x7fdedc0d1050 2024-09-12T03:16:48.846 INFO:teuthology.orchestra.run.smithi027.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-12T03:16:48.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:48.847+0000 7fdee1fe8640 1 -- 172.21.15.27:0/390123050 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fdec806be30 msgr2=0x7fdec806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:48.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:48.847+0000 7fdee1fe8640 1 --2- 172.21.15.27:0/390123050 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fdec806be30 0x7fdec806e2f0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fded0034000 tx=0x7fded002e000 comp rx=0 tx=0).stop 2024-09-12T03:16:48.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:48.847+0000 7fdee1fe8640 1 -- 172.21.15.27:0/390123050 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdedc0d1050 msgr2=0x7fdedc15ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:48.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:48.847+0000 7fdee1fe8640 1 --2- 172.21.15.27:0/390123050 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdedc0d1050 0x7fdedc15ee70 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fdecc00c490 tx=0x7fdecc00c950 comp rx=0 tx=0).stop 2024-09-12T03:16:48.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:48.847+0000 7fdee1fe8640 1 -- 172.21.15.27:0/390123050 shutdown_connections 2024-09-12T03:16:48.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:48.847+0000 7fdee1fe8640 1 --2- 172.21.15.27:0/390123050 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fdec806be30 0x7fdec806e2f0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:48.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:48.847+0000 7fdee1fe8640 1 --2- 172.21.15.27:0/390123050 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdedc0d52e0 0x7fdedc0ccfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:48.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:48.847+0000 7fdee1fe8640 1 --2- 172.21.15.27:0/390123050 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fdedc0d3120 0x7fdedc162940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:48.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:48.847+0000 7fdee1fe8640 1 --2- 172.21.15.27:0/390123050 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdedc0d1050 0x7fdedc15ee70 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:48.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:48.847+0000 7fdee1fe8640 1 -- 172.21.15.27:0/390123050 >> 172.21.15.27:0/390123050 conn(0x7fdedc06ddf0 msgr2=0x7fdedc0b62a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:48.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:48.847+0000 7fdee1fe8640 1 -- 172.21.15.27:0/390123050 shutdown_connections 2024-09-12T03:16:48.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:48.847+0000 7fdee1fe8640 1 -- 172.21.15.27:0/390123050 wait complete. 2024-09-12T03:16:48.859 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-12T03:16:48.859 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-12T03:16:48.859 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi027.front.sepia.ceph.com 2024-09-12T03:16:48.859 INFO:tasks.cephfs.mount:self.client.name = client.2 2024-09-12T03:16:48.859 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-09-12T03:16:48.860 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-12T03:16:48.860 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-12T03:16:48.860 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-12T03:16:48.860 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2024-09-12T03:16:48.860 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:16:48.861 DEBUG:teuthology.orchestra.run.smithi027:> ip addr 2024-09-12T03:16:48.877 INFO:teuthology.orchestra.run.smithi027.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-12T03:16:48.877 INFO:teuthology.orchestra.run.smithi027.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-12T03:16:48.877 INFO:teuthology.orchestra.run.smithi027.stdout: inet 127.0.0.1/8 scope host lo 2024-09-12T03:16:48.877 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:16:48.877 INFO:teuthology.orchestra.run.smithi027.stdout: inet6 ::1/128 scope host 2024-09-12T03:16:48.877 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:16:48.878 INFO:teuthology.orchestra.run.smithi027.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-12T03:16:48.878 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether 0c:c4:7a:6b:fe:c8 brd ff:ff:ff:ff:ff:ff 2024-09-12T03:16:48.878 INFO:teuthology.orchestra.run.smithi027.stdout: altname enp7s0f0 2024-09-12T03:16:48.878 INFO:teuthology.orchestra.run.smithi027.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-12T03:16:48.878 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether 0c:c4:7a:6b:fe:c9 brd ff:ff:ff:ff:ff:ff 2024-09-12T03:16:48.878 INFO:teuthology.orchestra.run.smithi027.stdout: altname enp7s0f1 2024-09-12T03:16:48.878 INFO:teuthology.orchestra.run.smithi027.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-12T03:16:48.878 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether ac:1f:6b:f8:39:5a brd ff:ff:ff:ff:ff:ff 2024-09-12T03:16:48.878 INFO:teuthology.orchestra.run.smithi027.stdout: altname enp1s0f0 2024-09-12T03:16:48.878 INFO:teuthology.orchestra.run.smithi027.stdout: inet 172.21.15.27/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-12T03:16:48.878 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft 42780sec preferred_lft 42780sec 2024-09-12T03:16:48.878 INFO:teuthology.orchestra.run.smithi027.stdout: inet6 fe80::ae1f:6bff:fef8:395a/64 scope link 2024-09-12T03:16:48.878 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:16:48.879 INFO:teuthology.orchestra.run.smithi027.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-12T03:16:48.879 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether ac:1f:6b:f8:39:5b brd ff:ff:ff:ff:ff:ff 2024-09-12T03:16:48.879 INFO:teuthology.orchestra.run.smithi027.stdout: altname enp1s0f1 2024-09-12T03:16:48.879 INFO:teuthology.orchestra.run.smithi027.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-12T03:16:48.879 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether 42:40:d7:5c:7a:8e brd ff:ff:ff:ff:ff:ff 2024-09-12T03:16:48.879 INFO:teuthology.orchestra.run.smithi027.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-12T03:16:48.879 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:16:48.879 INFO:teuthology.orchestra.run.smithi027.stdout: inet6 fe80::c84b:19ff:fef0:ca74/64 scope link 2024-09-12T03:16:48.879 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:16:48.879 INFO:teuthology.orchestra.run.smithi027.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-12T03:16:48.879 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether 42:40:d7:5c:7a:8e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-12T03:16:48.879 INFO:teuthology.orchestra.run.smithi027.stdout: inet6 fe80::4040:d7ff:fe5c:7a8e/64 scope link tentative 2024-09-12T03:16:48.879 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:16:48.880 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:16:48.880 DEBUG:teuthology.orchestra.run.smithi027:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-12T03:16:48.942 INFO:teuthology.orchestra.run.smithi027.stdout:1 2024-09-12T03:16:48.943 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:16:48.943 DEBUG:teuthology.orchestra.run.smithi027:> route 2024-09-12T03:16:49.000 INFO:teuthology.orchestra.run.smithi027.stdout:Kernel IP routing table 2024-09-12T03:16:49.000 INFO:teuthology.orchestra.run.smithi027.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-12T03:16:49.001 INFO:teuthology.orchestra.run.smithi027.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-12T03:16:49.001 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-12T03:16:49.001 INFO:teuthology.orchestra.run.smithi027.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-12T03:16:49.001 DEBUG:teuthology.orchestra.run.smithi027:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-12T03:16:49.001 DEBUG:teuthology.orchestra.run.smithi027:> set -e 2024-09-12T03:16:49.001 DEBUG:teuthology.orchestra.run.smithi027:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-12T03:16:49.001 DEBUG:teuthology.orchestra.run.smithi027:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-12T03:16:49.001 DEBUG:teuthology.orchestra.run.smithi027:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-12T03:16:49.002 DEBUG:teuthology.orchestra.run.smithi027:> ') 2024-09-12T03:16:49.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:49 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T03:16:49.124 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T03:16:49.131 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:16:49.131 DEBUG:teuthology.orchestra.run.smithi027:> ip netns list 2024-09-12T03:16:49.147 INFO:teuthology.orchestra.run.smithi027.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-12T03:16:49.147 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:16:49.148 DEBUG:teuthology.orchestra.run.smithi027:> ip netns list-id 2024-09-12T03:16:49.204 INFO:teuthology.orchestra.run.smithi027.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2024-09-12T03:16:49.204 DEBUG:teuthology.orchestra.run.smithi027:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-12T03:16:49.204 DEBUG:teuthology.orchestra.run.smithi027:> set -e 2024-09-12T03:16:49.204 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-12T03:16:49.204 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2024-09-12T03:16:49.205 DEBUG:teuthology.orchestra.run.smithi027:> ') 2024-09-12T03:16:49.293 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:49 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T03:16:49.316 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T03:16:49.322 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:16:49.323 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-09-12T03:16:49.348 INFO:teuthology.orchestra.run.smithi027.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-12T03:16:49.348 INFO:teuthology.orchestra.run.smithi027.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-12T03:16:49.349 INFO:teuthology.orchestra.run.smithi027.stdout: inet 127.0.0.1/8 scope host lo 2024-09-12T03:16:49.349 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:16:49.349 INFO:teuthology.orchestra.run.smithi027.stdout: inet6 ::1/128 scope host 2024-09-12T03:16:49.349 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:16:49.349 INFO:teuthology.orchestra.run.smithi027.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-12T03:16:49.349 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether 1e:6d:6f:87:db:c9 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-09-12T03:16:49.349 INFO:teuthology.orchestra.run.smithi027.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-09-12T03:16:49.349 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:16:49.349 INFO:teuthology.orchestra.run.smithi027.stdout: inet6 fe80::1c6d:6fff:fe87:dbc9/64 scope link 2024-09-12T03:16:49.349 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:16:49.350 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:16:49.350 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-09-12T03:16:49.417 INFO:teuthology.orchestra.run.smithi027.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-12T03:16:49.417 INFO:teuthology.orchestra.run.smithi027.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-12T03:16:49.417 INFO:teuthology.orchestra.run.smithi027.stdout: inet 127.0.0.1/8 scope host lo 2024-09-12T03:16:49.417 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:16:49.417 INFO:teuthology.orchestra.run.smithi027.stdout: inet6 ::1/128 scope host 2024-09-12T03:16:49.417 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:16:49.417 INFO:teuthology.orchestra.run.smithi027.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-12T03:16:49.417 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether 1e:6d:6f:87:db:c9 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-09-12T03:16:49.417 INFO:teuthology.orchestra.run.smithi027.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-09-12T03:16:49.418 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:16:49.418 INFO:teuthology.orchestra.run.smithi027.stdout: inet6 fe80::1c6d:6fff:fe87:dbc9/64 scope link 2024-09-12T03:16:49.418 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:16:49.419 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2024-09-12T03:16:49.419 DEBUG:teuthology.orchestra.run.smithi027:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-12T03:16:49.419 DEBUG:teuthology.orchestra.run.smithi027:> set -e 2024-09-12T03:16:49.419 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2024-09-12T03:16:49.419 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-09-12T03:16:49.420 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2024-09-12T03:16:49.420 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2024-09-12T03:16:49.420 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2024-09-12T03:16:49.420 DEBUG:teuthology.orchestra.run.smithi027:> ') 2024-09-12T03:16:49.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:49 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T03:16:49.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T03:16:49.575 DEBUG:teuthology.orchestra.run.smithi027:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-12T03:16:49.575 DEBUG:teuthology.orchestra.run.smithi027:> set -e 2024-09-12T03:16:49.575 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip link set brx.1 up 2024-09-12T03:16:49.575 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip link set dev brx.1 master ceph-brx 2024-09-12T03:16:49.575 DEBUG:teuthology.orchestra.run.smithi027:> ') 2024-09-12T03:16:49.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:49 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T03:16:49.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T03:16:49.655 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-12T03:16:49.655 DEBUG:teuthology.orchestra.run.smithi027:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2024-09-12T03:16:49.683 INFO:teuthology.orchestra.run.smithi027.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2024-09-12T03:16:49.683 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-12T03:16:49.683 DEBUG:teuthology.orchestra.run.smithi027:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2024-09-12T03:16:49.738 INFO:tasks.cephfs.kernel_mount:mounting using device: 2@.cephfs=/ 2024-09-12T03:16:49.738 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:16:49.738 DEBUG:teuthology.orchestra.run.smithi027:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 2@.cephfs=/ /home/ubuntu/cephtest/mnt.2 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-12T03:16:49.823 INFO:teuthology.orchestra.run.smithi027.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-12T03:16:49.824 INFO:teuthology.orchestra.run.smithi027.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-12T03:16:49.844 INFO:teuthology.orchestra.run.smithi027.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-12T03:16:49.844 INFO:teuthology.orchestra.run.smithi027.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-12T03:16:49.845 INFO:teuthology.orchestra.run.smithi027.stdout:mount.ceph: resolved to: "172.21.15.27:3300,172.21.15.45:3300,172.21.15.45:3301" 2024-09-12T03:16:49.845 INFO:teuthology.orchestra.run.smithi027.stdout:mount.ceph: trying mount with new device syntax: 2@d2df504e-2647-4385-a5db-9fdf69f487a0.cephfs=/ 2024-09-12T03:16:49.845 INFO:teuthology.orchestra.run.smithi027.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,ms_mode=prefer-crc,key=2,mon_addr=172.21.15.27:3300/172.21.15.45:3300/172.21.15.45:3301" will pass to kernel 2024-09-12T03:16:49.845 INFO:teuthology.orchestra.run.smithi027.stdout:mount: /home/ubuntu/cephtest/mnt.2 does not contain SELinux labels. 2024-09-12T03:16:49.845 INFO:teuthology.orchestra.run.smithi027.stdout: You just mounted a file system that supports labels which does not 2024-09-12T03:16:49.845 INFO:teuthology.orchestra.run.smithi027.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-12T03:16:49.845 INFO:teuthology.orchestra.run.smithi027.stdout: applications will generate AVC messages and not be allowed access to 2024-09-12T03:16:49.846 INFO:teuthology.orchestra.run.smithi027.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-12T03:16:49.847 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-12T03:16:49.847 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:16:49.847 DEBUG:teuthology.orchestra.run.smithi027:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2024-09-12T03:16:49.873 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-12T03:16:49.963 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:49 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-12T03:16:49.964 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T03:16:49.970 INFO:teuthology.orchestra.run.smithi027.stdout:client4647 2024-09-12T03:16:49.972 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-12T03:16:49.972 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T03:16:49.989 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-12T03:16:50.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:50 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-12T03:16:50.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T03:16:50.086 INFO:teuthology.orchestra.run.smithi027.stdout:client4647 2024-09-12T03:16:50.087 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-12T03:16:50.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:50 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-12T03:16:50.138 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T03:16:50.144 INFO:teuthology.orchestra.run.smithi027.stdout:client4647 2024-09-12T03:16:50.144 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/d2df504e-2647-4385-a5db-9fdf69f487a0.client4647/status') 2024-09-12T03:16:50.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:50 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T03:16:50.204 INFO:teuthology.orchestra.run.smithi027.stdout:instance: client.4647 (3)192.168.0.2:0/4176229498 2024-09-12T03:16:50.204 INFO:teuthology.orchestra.run.smithi027.stdout:blocklisted: false 2024-09-12T03:16:50.204 INFO:teuthology.orchestra.run.smithi027.stderr:0+1 records in 2024-09-12T03:16:50.204 INFO:teuthology.orchestra.run.smithi027.stderr:0+1 records out 2024-09-12T03:16:50.204 INFO:teuthology.orchestra.run.smithi027.stderr:69 bytes copied, 2.3786e-05 s, 2.9 MB/s 2024-09-12T03:16:50.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T03:16:50.211 INFO:tasks.kclient:client.1 config is {'syntax': 'v2'} 2024-09-12T03:16:50.211 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-12T03:16:50.211 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-12T03:16:50.211 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-12T03:16:50.314 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:50.312+0000 7f78af20e640 1 Processor -- start 2024-09-12T03:16:50.314 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:50.312+0000 7f78af20e640 1 -- start start 2024-09-12T03:16:50.320 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:50.313+0000 7f78af20e640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f78a80d5200 0x7f78a80d1010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:50.321 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:50.313+0000 7f78af20e640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f78a80d3760 con 0x7f78a80d1550 2024-09-12T03:16:50.321 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:50.313+0000 7f78af20e640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f78a80d3a40 con 0x7f78a815eb90 2024-09-12T03:16:50.321 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:50.313+0000 7f78af20e640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f78a80b7d20 con 0x7f78a80d5200 2024-09-12T03:16:50.321 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:50.313+0000 7f78ad7ab640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f78a80d1550 0x7f78a80b8190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:46496/0 (socket says 172.21.15.27:46496) 2024-09-12T03:16:50.321 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:50.313+0000 7f78ad7ab640 1 -- 172.21.15.27:0/3171499179 learned_addr learned my addr 172.21.15.27:0/3171499179 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:16:50.321 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:50.314+0000 7f78a7fff640 1 --2- 172.21.15.27:0/3171499179 >> v2:172.21.15.45:3300/0 conn(0x7f78a80d5200 0x7f78a80d1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:50.321 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:50.314+0000 7f78a77fe640 1 -- 172.21.15.27:0/3171499179 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 982914143 0 0) 0x7f78a80d3760 con 0x7f78a80d1550 2024-09-12T03:16:50.321 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:50.314+0000 7f78a77fe640 1 -- 172.21.15.27:0/3171499179 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7894003520 con 0x7f78a80d1550 2024-09-12T03:16:50.322 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:50.314+0000 7f78a77fe640 1 -- 172.21.15.27:0/3171499179 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2117142024 0 0) 0x7f7894003520 con 0x7f78a80d1550 2024-09-12T03:16:50.322 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:50.314+0000 7f78a77fe640 1 -- 172.21.15.27:0/3171499179 >> v2:172.21.15.45:3300/0 conn(0x7f78a80d5200 msgr2=0x7f78a80d1010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:50.322 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:50.314+0000 7f78a77fe640 1 --2- 172.21.15.27:0/3171499179 >> v2:172.21.15.45:3300/0 conn(0x7f78a80d5200 0x7f78a80d1010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:50.322 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:50.314+0000 7f78a77fe640 1 -- 172.21.15.27:0/3171499179 >> v1:172.21.15.45:6789/0 conn(0x7f78a815eb90 legacy=0x7f78a80d2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:50.322 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:50.314+0000 7f78a77fe640 1 -- 172.21.15.27:0/3171499179 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78a80d3d20 con 0x7f78a80d1550 2024-09-12T03:16:50.322 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:50.314+0000 7f78a77fe640 1 -- 172.21.15.27:0/3171499179 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 847755213 0 0) 0x7f7890001670 con 0x7f78a80d1550 2024-09-12T03:16:50.322 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:50.314+0000 7f78a77fe640 1 -- 172.21.15.27:0/3171499179 >> v1:172.21.15.27:6789/0 conn(0x7f78a80d1550 legacy=0x7f78a80b8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:50.322 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:50.314+0000 7f78a77fe640 1 --2- 172.21.15.27:0/3171499179 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f78940039b0 0x7f7894003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:50.322 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:50.314+0000 7f78a77fe640 1 --2- 172.21.15.27:0/3171499179 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7894004800 0x7f7894004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:50.322 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:50.314+0000 7f78a77fe640 1 --2- 172.21.15.27:0/3171499179 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f78940051c0 0x7f789400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:50.322 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:50.314+0000 7f78a77fe640 1 -- 172.21.15.27:0/3171499179 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f789400c030 con 0x7f7894004800 2024-09-12T03:16:50.323 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:50.314+0000 7f78a77fe640 1 -- 172.21.15.27:0/3171499179 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f789400c1e0 con 0x7f78940039b0 2024-09-12T03:16:50.323 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:50.314+0000 7f78a77fe640 1 -- 172.21.15.27:0/3171499179 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f789400c3e0 con 0x7f78940051c0 2024-09-12T03:16:50.323 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:50.314+0000 7f78a7fff640 1 --2- 172.21.15.27:0/3171499179 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7894004800 0x7f7894004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:50.323 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:50.314+0000 7f78acfaa640 1 --2- 172.21.15.27:0/3171499179 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f78940039b0 0x7f7894003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:50.323 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:50.314+0000 7f78ad7ab640 1 --2- 172.21.15.27:0/3171499179 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f78940051c0 0x7f789400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:50.323 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:50.314+0000 7f78a7fff640 1 -- 172.21.15.27:0/3171499179 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f78940051c0 msgr2=0x7f789400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:50.323 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:50.314+0000 7f78a7fff640 1 --2- 172.21.15.27:0/3171499179 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f78940051c0 0x7f789400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:50.323 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:50.314+0000 7f78a7fff640 1 -- 172.21.15.27:0/3171499179 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f78940039b0 msgr2=0x7f7894003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:50.323 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:50.314+0000 7f78a7fff640 1 --2- 172.21.15.27:0/3171499179 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f78940039b0 0x7f7894003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:50.324 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:50.315+0000 7f78a7fff640 1 -- 172.21.15.27:0/3171499179 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7894003520 con 0x7f7894004800 2024-09-12T03:16:50.324 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:50.315+0000 7f78acfaa640 1 --2- 172.21.15.27:0/3171499179 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f78940039b0 0x7f7894003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:16:50.324 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:50.315+0000 7f78ad7ab640 1 -- 172.21.15.27:0/3171499179 reap_dead start 2024-09-12T03:16:50.324 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:50.315+0000 7f78a7fff640 1 --2- 172.21.15.27:0/3171499179 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7894004800 0x7f7894004c80 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f7898009a90 tx=0x7f7894005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf65c44e86e24f6b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:50.324 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:50.315+0000 7f78a77fe640 1 -- 172.21.15.27:0/3171499179 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7898036070 con 0x7f7894004800 2024-09-12T03:16:50.324 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:50.315+0000 7f78a77fe640 1 -- 172.21.15.27:0/3171499179 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7898031400 con 0x7f7894004800 2024-09-12T03:16:50.324 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:50.315+0000 7f78a77fe640 1 -- 172.21.15.27:0/3171499179 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7898009690 con 0x7f7894004800 2024-09-12T03:16:50.324 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:50.316+0000 7f78af20e640 1 -- 172.21.15.27:0/3171499179 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7894004800 msgr2=0x7f7894004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:50.324 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:50.316+0000 7f78af20e640 1 --2- 172.21.15.27:0/3171499179 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7894004800 0x7f7894004c80 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f7898009a90 tx=0x7f7894005830 comp rx=0 tx=0).stop 2024-09-12T03:16:50.325 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:50.316+0000 7f78af20e640 1 -- 172.21.15.27:0/3171499179 shutdown_connections 2024-09-12T03:16:50.325 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:50.316+0000 7f78af20e640 1 --2- 172.21.15.27:0/3171499179 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7894004800 0x7f7894004c80 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:50.325 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:50.316+0000 7f78af20e640 1 -- 172.21.15.27:0/3171499179 >> 172.21.15.27:0/3171499179 conn(0x7f78a806dce0 msgr2=0x7f78a806e0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:50.325 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:50.316+0000 7f78af20e640 1 -- 172.21.15.27:0/3171499179 shutdown_connections 2024-09-12T03:16:50.325 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:50.316+0000 7f78af20e640 1 -- 172.21.15.27:0/3171499179 wait complete. 2024-09-12T03:16:50.325 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:50.317+0000 7f78af20e640 1 Processor -- start 2024-09-12T03:16:50.325 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:50.317+0000 7f78af20e640 1 -- start start 2024-09-12T03:16:50.325 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:50.318+0000 7f78af20e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7894004800 0x7f78a814cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:50.325 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:50.318+0000 7f78af20e640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f78a815a3b0 0x7f78a815cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:50.325 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:50.318+0000 7f78af20e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f78a815d530 0x7f78a81585e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:50.325 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:50.318+0000 7f78af20e640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f78a80d1960 con 0x7f7894004800 2024-09-12T03:16:50.325 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:50.318+0000 7f78af20e640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f78a80d42e0 con 0x7f78a815d530 2024-09-12T03:16:50.326 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:50.318+0000 7f78af20e640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f78a815ef70 con 0x7f78a815a3b0 2024-09-12T03:16:50.326 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:50.318+0000 7f78acfaa640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7894004800 0x7f78a814cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:50.326 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:50.318+0000 7f78acfaa640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7894004800 0x7f78a814cec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59048/0 (socket says 172.21.15.27:59048) 2024-09-12T03:16:50.326 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:50.318+0000 7f78acfaa640 1 -- 172.21.15.27:0/4128138939 learned_addr learned my addr 172.21.15.27:0/4128138939 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:16:50.326 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:50.318+0000 7f78ad7ab640 1 --2- 172.21.15.27:0/4128138939 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f78a815d530 0x7f78a81585e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:50.326 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:50.318+0000 7f78a7fff640 1 --2- 172.21.15.27:0/4128138939 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f78a815a3b0 0x7f78a815cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:50.326 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:50.318+0000 7f78acfaa640 1 -- 172.21.15.27:0/4128138939 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f78a815a3b0 msgr2=0x7f78a815cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:50.326 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:50.318+0000 7f78acfaa640 1 --2- 172.21.15.27:0/4128138939 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f78a815a3b0 0x7f78a815cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:50.326 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:50.318+0000 7f78acfaa640 1 -- 172.21.15.27:0/4128138939 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f78a815d530 msgr2=0x7f78a81585e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:50.326 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:50.318+0000 7f78acfaa640 1 --2- 172.21.15.27:0/4128138939 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f78a815d530 0x7f78a81585e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:50.327 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:50.318+0000 7f78acfaa640 1 -- 172.21.15.27:0/4128138939 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78a8159310 con 0x7f7894004800 2024-09-12T03:16:50.327 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:50.318+0000 7f78ad7ab640 1 --2- 172.21.15.27:0/4128138939 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f78a815d530 0x7f78a81585e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:16:50.327 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:50.318+0000 7f78a7fff640 1 --2- 172.21.15.27:0/4128138939 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f78a815a3b0 0x7f78a815cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:16:50.327 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:50.319+0000 7f78acfaa640 1 --2- 172.21.15.27:0/4128138939 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7894004800 0x7f78a814cec0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f78a000eaf0 tx=0x7f78a0002060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:50.327 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:50.319+0000 7f78a57fa640 1 -- 172.21.15.27:0/4128138939 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78a0003b80 con 0x7f7894004800 2024-09-12T03:16:50.327 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:50.319+0000 7f78a57fa640 1 -- 172.21.15.27:0/4128138939 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f78a0018070 con 0x7f7894004800 2024-09-12T03:16:50.327 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:50.319+0000 7f78af20e640 1 -- 172.21.15.27:0/4128138939 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78a81595f0 con 0x7f7894004800 2024-09-12T03:16:50.327 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:50.319+0000 7f78a57fa640 1 -- 172.21.15.27:0/4128138939 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78a000c780 con 0x7f7894004800 2024-09-12T03:16:50.327 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:50.319+0000 7f78af20e640 1 -- 172.21.15.27:0/4128138939 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78a8159bb0 con 0x7f7894004800 2024-09-12T03:16:50.327 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:50.320+0000 7f78af20e640 1 -- 172.21.15.27:0/4128138939 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78a80d1010 con 0x7f7894004800 2024-09-12T03:16:50.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:50.320+0000 7f78a57fa640 1 -- 172.21.15.27:0/4128138939 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f78a000c920 con 0x7f7894004800 2024-09-12T03:16:50.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:50.321+0000 7f78a57fa640 1 --2- 172.21.15.27:0/4128138939 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f788806be30 0x7f788806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:50.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:50.321+0000 7f78a57fa640 1 -- 172.21.15.27:0/4128138939 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f78a008aef0 con 0x7f7894004800 2024-09-12T03:16:50.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:50.321+0000 7f78a7fff640 1 --2- 172.21.15.27:0/4128138939 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f788806be30 0x7f788806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:50.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:50.322+0000 7f78a7fff640 1 --2- 172.21.15.27:0/4128138939 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f788806be30 0x7f788806e2f0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f7898033000 tx=0x7f7898008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:50.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:50.325+0000 7f78a57fa640 1 -- 172.21.15.27:0/4128138939 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f78a0059250 con 0x7f7894004800 2024-09-12T03:16:50.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:50.536+0000 7f78af20e640 1 -- 172.21.15.27:0/4128138939 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f78a815d910 con 0x7f7894004800 2024-09-12T03:16:50.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:50.536+0000 7f78a57fa640 1 -- 172.21.15.27:0/4128138939 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v6) ==== 53+0+73 (secure 0 0 0) 0x7f78a005d550 con 0x7f7894004800 2024-09-12T03:16:50.538 INFO:teuthology.orchestra.run.smithi027.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-12T03:16:50.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:50.538+0000 7f78af20e640 1 -- 172.21.15.27:0/4128138939 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f788806be30 msgr2=0x7f788806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:50.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:50.539+0000 7f78af20e640 1 --2- 172.21.15.27:0/4128138939 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f788806be30 0x7f788806e2f0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f7898033000 tx=0x7f7898008030 comp rx=0 tx=0).stop 2024-09-12T03:16:50.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:50.539+0000 7f78af20e640 1 -- 172.21.15.27:0/4128138939 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7894004800 msgr2=0x7f78a814cec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:50.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:50.539+0000 7f78af20e640 1 --2- 172.21.15.27:0/4128138939 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7894004800 0x7f78a814cec0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f78a000eaf0 tx=0x7f78a0002060 comp rx=0 tx=0).stop 2024-09-12T03:16:50.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:50.539+0000 7f78af20e640 1 -- 172.21.15.27:0/4128138939 shutdown_connections 2024-09-12T03:16:50.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:50.539+0000 7f78af20e640 1 --2- 172.21.15.27:0/4128138939 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f788806be30 0x7f788806e2f0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:50.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:50.539+0000 7f78af20e640 1 --2- 172.21.15.27:0/4128138939 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f78a815d530 0x7f78a81585e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:50.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:50.539+0000 7f78af20e640 1 --2- 172.21.15.27:0/4128138939 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f78a815a3b0 0x7f78a815cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:50.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:50.539+0000 7f78af20e640 1 --2- 172.21.15.27:0/4128138939 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7894004800 0x7f78a814cec0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:50.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:50.539+0000 7f78af20e640 1 -- 172.21.15.27:0/4128138939 >> 172.21.15.27:0/4128138939 conn(0x7f78a806dce0 msgr2=0x7f78a80b9460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:50.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:50.539+0000 7f78af20e640 1 -- 172.21.15.27:0/4128138939 shutdown_connections 2024-09-12T03:16:50.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:50.539+0000 7f78af20e640 1 -- 172.21.15.27:0/4128138939 wait complete. 2024-09-12T03:16:50.550 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-12T03:16:50.551 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-12T03:16:50.551 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi045.front.sepia.ceph.com 2024-09-12T03:16:50.551 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-12T03:16:50.551 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-12T03:16:50.551 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-12T03:16:50.551 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-12T03:16:50.552 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-12T03:16:50.552 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-12T03:16:50.552 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:16:50.552 DEBUG:teuthology.orchestra.run.smithi045:> ip addr 2024-09-12T03:16:50.572 INFO:teuthology.orchestra.run.smithi045.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-12T03:16:50.573 INFO:teuthology.orchestra.run.smithi045.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-12T03:16:50.573 INFO:teuthology.orchestra.run.smithi045.stdout: inet 127.0.0.1/8 scope host lo 2024-09-12T03:16:50.573 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:16:50.573 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 ::1/128 scope host 2024-09-12T03:16:50.573 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:16:50.574 INFO:teuthology.orchestra.run.smithi045.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-12T03:16:50.574 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:6b:fd:a6 brd ff:ff:ff:ff:ff:ff 2024-09-12T03:16:50.574 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp7s0f0 2024-09-12T03:16:50.574 INFO:teuthology.orchestra.run.smithi045.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-12T03:16:50.574 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:6b:fd:a7 brd ff:ff:ff:ff:ff:ff 2024-09-12T03:16:50.575 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp7s0f1 2024-09-12T03:16:50.575 INFO:teuthology.orchestra.run.smithi045.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-12T03:16:50.575 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:bd:12:be brd ff:ff:ff:ff:ff:ff 2024-09-12T03:16:50.575 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp1s0f0 2024-09-12T03:16:50.575 INFO:teuthology.orchestra.run.smithi045.stdout: inet 172.21.15.45/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-12T03:16:50.575 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft 42723sec preferred_lft 42723sec 2024-09-12T03:16:50.576 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::ec4:7aff:febd:12be/64 scope link 2024-09-12T03:16:50.576 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:16:50.576 INFO:teuthology.orchestra.run.smithi045.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-12T03:16:50.576 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:bd:12:bf brd ff:ff:ff:ff:ff:ff 2024-09-12T03:16:50.577 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp1s0f1 2024-09-12T03:16:50.577 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-12T03:16:50.577 DEBUG:teuthology.orchestra.run.smithi045:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-12T03:16:50.577 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-09-12T03:16:50.577 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link add name ceph-brx type bridge 2024-09-12T03:16:50.578 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip addr flush dev ceph-brx 2024-09-12T03:16:50.578 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link set ceph-brx up 2024-09-12T03:16:50.578 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-12T03:16:50.578 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-09-12T03:16:50.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:50 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T03:16:50.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T03:16:50.797 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:16:50.797 DEBUG:teuthology.orchestra.run.smithi045:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-12T03:16:50.823 INFO:teuthology.orchestra.run.smithi045.stdout:1 2024-09-12T03:16:50.825 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:16:50.825 DEBUG:teuthology.orchestra.run.smithi045:> route 2024-09-12T03:16:50.896 INFO:teuthology.orchestra.run.smithi045.stdout:Kernel IP routing table 2024-09-12T03:16:50.896 INFO:teuthology.orchestra.run.smithi045.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-12T03:16:50.896 INFO:teuthology.orchestra.run.smithi045.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-12T03:16:50.896 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-12T03:16:50.896 INFO:teuthology.orchestra.run.smithi045.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-12T03:16:50.897 DEBUG:teuthology.orchestra.run.smithi045:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-12T03:16:50.897 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-09-12T03:16:50.897 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-12T03:16:50.897 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-12T03:16:50.897 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-12T03:16:50.897 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-09-12T03:16:50.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:50 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T03:16:51.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T03:16:51.177 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:16:51.177 DEBUG:teuthology.orchestra.run.smithi045:> ip netns list 2024-09-12T03:16:51.194 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:16:51.194 DEBUG:teuthology.orchestra.run.smithi045:> ip netns list-id 2024-09-12T03:16:51.250 DEBUG:teuthology.orchestra.run.smithi045:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-12T03:16:51.250 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-09-12T03:16:51.250 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-12T03:16:51.250 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-09-12T03:16:51.250 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-09-12T03:16:51.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:51 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T03:16:51.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T03:16:51.373 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-09-12T03:16:51.373 DEBUG:teuthology.orchestra.run.smithi045:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-12T03:16:51.373 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-09-12T03:16:51.373 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-09-12T03:16:51.374 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-12T03:16:51.374 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-09-12T03:16:51.374 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-09-12T03:16:51.374 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-09-12T03:16:51.374 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-09-12T03:16:51.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:51 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T03:16:51.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T03:16:51.512 DEBUG:teuthology.orchestra.run.smithi045:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-12T03:16:51.512 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-09-12T03:16:51.512 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link set brx.0 up 2024-09-12T03:16:51.512 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link set dev brx.0 master ceph-brx 2024-09-12T03:16:51.512 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-09-12T03:16:51.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:51 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T03:16:51.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T03:16:51.591 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-12T03:16:51.591 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-12T03:16:51.620 INFO:teuthology.orchestra.run.smithi045.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-12T03:16:51.621 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-12T03:16:51.621 DEBUG:teuthology.orchestra.run.smithi045:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-12T03:16:51.676 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2024-09-12T03:16:51.677 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:16:51.677 DEBUG:teuthology.orchestra.run.smithi045:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-12T03:16:51.779 INFO:teuthology.orchestra.run.smithi045.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-12T03:16:51.780 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-12T03:16:51.912 INFO:teuthology.orchestra.run.smithi045.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-12T03:16:51.912 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-12T03:16:51.913 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: resolved to: "172.21.15.27:3300,172.21.15.45:3300,172.21.15.45:3301" 2024-09-12T03:16:51.913 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: trying mount with new device syntax: 1@d2df504e-2647-4385-a5db-9fdf69f487a0.cephfs=/ 2024-09-12T03:16:51.913 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.27:3300/172.21.15.45:3300/172.21.15.45:3301" will pass to kernel 2024-09-12T03:16:51.914 INFO:teuthology.orchestra.run.smithi045.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-09-12T03:16:51.914 INFO:teuthology.orchestra.run.smithi045.stdout: You just mounted a file system that supports labels which does not 2024-09-12T03:16:51.914 INFO:teuthology.orchestra.run.smithi045.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-12T03:16:51.914 INFO:teuthology.orchestra.run.smithi045.stdout: applications will generate AVC messages and not be allowed access to 2024-09-12T03:16:51.914 INFO:teuthology.orchestra.run.smithi045.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-12T03:16:51.916 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-12T03:16:51.916 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:16:51.916 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-12T03:16:51.942 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-12T03:16:52.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:52 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-12T03:16:52.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T03:16:52.040 INFO:teuthology.orchestra.run.smithi045.stdout:client4484 2024-09-12T03:16:52.040 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-12T03:16:52.040 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T03:16:52.056 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-12T03:16:52.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:52 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-12T03:16:52.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T03:16:52.153 INFO:teuthology.orchestra.run.smithi045.stdout:client4484 2024-09-12T03:16:52.154 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-12T03:16:52.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:52 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-12T03:16:52.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T03:16:52.211 INFO:teuthology.orchestra.run.smithi045.stdout:client4484 2024-09-12T03:16:52.211 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/d2df504e-2647-4385-a5db-9fdf69f487a0.client4484/status') 2024-09-12T03:16:52.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:52 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T03:16:52.273 INFO:teuthology.orchestra.run.smithi045.stdout:instance: client.4484 (3)192.168.0.1:0/2287088535 2024-09-12T03:16:52.273 INFO:teuthology.orchestra.run.smithi045.stdout:blocklisted: false 2024-09-12T03:16:52.273 INFO:teuthology.orchestra.run.smithi045.stderr:0+1 records in 2024-09-12T03:16:52.273 INFO:teuthology.orchestra.run.smithi045.stderr:0+1 records out 2024-09-12T03:16:52.273 INFO:teuthology.orchestra.run.smithi045.stderr:69 bytes copied, 2.5062e-05 s, 2.8 MB/s 2024-09-12T03:16:52.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T03:16:52.280 INFO:tasks.kclient:client.3 config is {'syntax': 'v2'} 2024-09-12T03:16:52.280 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-12T03:16:52.280 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-09-12T03:16:52.280 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-12T03:16:52.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:52.395+0000 7f399282e640 1 Processor -- start 2024-09-12T03:16:52.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:52.395+0000 7f399282e640 1 -- start start 2024-09-12T03:16:52.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:52.396+0000 7f399282e640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f398c0cef50 0x7f398c0cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:52.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:52.396+0000 7f399282e640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f398c06fa00 con 0x7f398c070e80 2024-09-12T03:16:52.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:52.396+0000 7f399282e640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f398c0cf870 con 0x7f398c0d0b50 2024-09-12T03:16:52.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:52.396+0000 7f399282e640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f398c15ca90 con 0x7f398c0cef50 2024-09-12T03:16:52.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:52.396+0000 7f398b7fe640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f398c070e80 0x7f398c0d0440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:46510/0 (socket says 172.21.15.27:46510) 2024-09-12T03:16:52.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:52.396+0000 7f398b7fe640 1 -- 172.21.15.27:0/3673995917 learned_addr learned my addr 172.21.15.27:0/3673995917 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:16:52.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:52.396+0000 7f398bfff640 1 --2- 172.21.15.27:0/3673995917 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f398c0cef50 0x7f398c0cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:52.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:52.396+0000 7f398affd640 1 -- 172.21.15.27:0/3673995917 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2228173523 0 0) 0x7f398c06fa00 con 0x7f398c070e80 2024-09-12T03:16:52.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:52.396+0000 7f398affd640 1 -- 172.21.15.27:0/3673995917 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3974003520 con 0x7f398c070e80 2024-09-12T03:16:52.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:52.396+0000 7f398affd640 1 -- 172.21.15.27:0/3673995917 <== mon.1 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1095849526 0 0) 0x7f398c0cf870 con 0x7f398c0d0b50 2024-09-12T03:16:52.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:52.396+0000 7f398affd640 1 -- 172.21.15.27:0/3673995917 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f398c06fa00 con 0x7f398c0d0b50 2024-09-12T03:16:52.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:52.396+0000 7f398bfff640 1 -- 172.21.15.27:0/3673995917 >> v1:172.21.15.45:6789/0 conn(0x7f398c0d0b50 legacy=0x7f398c0d0fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:52.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:52.396+0000 7f398affd640 1 -- 172.21.15.27:0/3673995917 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4263166708 0 0) 0x7f3974003520 con 0x7f398c070e80 2024-09-12T03:16:52.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:52.396+0000 7f398bfff640 1 -- 172.21.15.27:0/3673995917 >> v1:172.21.15.27:6789/0 conn(0x7f398c070e80 legacy=0x7f398c0d0440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:52.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:52.396+0000 7f398bfff640 1 -- 172.21.15.27:0/3673995917 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f398c0d16f0 con 0x7f398c0cef50 2024-09-12T03:16:52.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:52.397+0000 7f398bfff640 1 --2- 172.21.15.27:0/3673995917 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f398c0cef50 0x7f398c0cf330 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f397c0097d0 tx=0x7f397c02a080 comp rx=0 tx=0).ready entity=mon.2 client_cookie=df0a27e7ab5eb656 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:52.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:52.397+0000 7f398affd640 1 -- 172.21.15.27:0/3673995917 <== mon.2 v2:172.21.15.45:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f397c037070 con 0x7f398c0cef50 2024-09-12T03:16:52.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:52.397+0000 7f398affd640 1 -- 172.21.15.27:0/3673995917 <== mon.2 v2:172.21.15.45:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f397c02ab10 con 0x7f398c0cef50 2024-09-12T03:16:52.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:52.397+0000 7f398affd640 1 -- 172.21.15.27:0/3673995917 <== mon.2 v2:172.21.15.45:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3980001670 con 0x7f398c0cef50 2024-09-12T03:16:52.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:52.398+0000 7f399282e640 1 -- 172.21.15.27:0/3673995917 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f398c0cef50 msgr2=0x7f398c0cf330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:52.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:52.398+0000 7f399282e640 1 --2- 172.21.15.27:0/3673995917 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f398c0cef50 0x7f398c0cf330 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f397c0097d0 tx=0x7f397c02a080 comp rx=0 tx=0).stop 2024-09-12T03:16:52.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:52.398+0000 7f399282e640 1 -- 172.21.15.27:0/3673995917 shutdown_connections 2024-09-12T03:16:52.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:52.398+0000 7f399282e640 1 --2- 172.21.15.27:0/3673995917 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f398c0cef50 0x7f398c0cf330 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:52.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:52.398+0000 7f399282e640 1 -- 172.21.15.27:0/3673995917 >> 172.21.15.27:0/3673995917 conn(0x7f398c073800 msgr2=0x7f398c09ba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:52.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:52.398+0000 7f399282e640 1 -- 172.21.15.27:0/3673995917 shutdown_connections 2024-09-12T03:16:52.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:52.398+0000 7f399282e640 1 -- 172.21.15.27:0/3673995917 wait complete. 2024-09-12T03:16:52.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:52.399+0000 7f399282e640 1 Processor -- start 2024-09-12T03:16:52.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:52.400+0000 7f399282e640 1 -- start start 2024-09-12T03:16:52.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:52.400+0000 7f399282e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f398c070e80 0x7f398c0cb850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:52.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:52.400+0000 7f399282e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f398c0cef50 0x7f398c15e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:52.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:52.400+0000 7f399282e640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f398c0d0b50 0x7f398c15e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:52.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:52.400+0000 7f399282e640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f398c0d2d10 con 0x7f398c070e80 2024-09-12T03:16:52.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:52.400+0000 7f399282e640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f398c0d2a30 con 0x7f398c0cef50 2024-09-12T03:16:52.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:52.400+0000 7f399282e640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f398c06f360 con 0x7f398c0d0b50 2024-09-12T03:16:52.405 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:52.400+0000 7f398bfff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f398c070e80 0x7f398c0cb850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:52.405 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:52.400+0000 7f398bfff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f398c070e80 0x7f398c0cb850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59056/0 (socket says 172.21.15.27:59056) 2024-09-12T03:16:52.405 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:52.400+0000 7f398bfff640 1 -- 172.21.15.27:0/2880217078 learned_addr learned my addr 172.21.15.27:0/2880217078 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:16:52.405 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:52.400+0000 7f398b7fe640 1 --2- 172.21.15.27:0/2880217078 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f398c0cef50 0x7f398c15e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:52.405 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:52.400+0000 7f3990dcb640 1 --2- 172.21.15.27:0/2880217078 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f398c0d0b50 0x7f398c15e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:52.405 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:52.401+0000 7f398bfff640 1 -- 172.21.15.27:0/2880217078 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f398c0d0b50 msgr2=0x7f398c15e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:52.405 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:52.401+0000 7f398bfff640 1 --2- 172.21.15.27:0/2880217078 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f398c0d0b50 0x7f398c15e9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:52.405 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:52.401+0000 7f398bfff640 1 -- 172.21.15.27:0/2880217078 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f398c0cef50 msgr2=0x7f398c15e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:52.405 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:52.401+0000 7f398bfff640 1 --2- 172.21.15.27:0/2880217078 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f398c0cef50 0x7f398c15e470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:52.405 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:52.401+0000 7f398bfff640 1 -- 172.21.15.27:0/2880217078 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f398c15d9d0 con 0x7f398c070e80 2024-09-12T03:16:52.405 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:52.401+0000 7f398bfff640 1 --2- 172.21.15.27:0/2880217078 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f398c070e80 0x7f398c0cb850 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f397c035040 tx=0x7f397c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:52.405 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:52.401+0000 7f3988ff9640 1 -- 172.21.15.27:0/2880217078 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f397c037050 con 0x7f398c070e80 2024-09-12T03:16:52.405 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:52.401+0000 7f3988ff9640 1 -- 172.21.15.27:0/2880217078 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f397c038040 con 0x7f398c070e80 2024-09-12T03:16:52.406 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:52.401+0000 7f399282e640 1 -- 172.21.15.27:0/2880217078 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f398c15f730 con 0x7f398c070e80 2024-09-12T03:16:52.406 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:52.401+0000 7f3988ff9640 1 -- 172.21.15.27:0/2880217078 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f397c003b20 con 0x7f398c070e80 2024-09-12T03:16:52.406 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:52.401+0000 7f399282e640 1 -- 172.21.15.27:0/2880217078 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f398c15dc70 con 0x7f398c070e80 2024-09-12T03:16:52.406 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:52.402+0000 7f399282e640 1 -- 172.21.15.27:0/2880217078 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f398c15fc50 con 0x7f398c070e80 2024-09-12T03:16:52.407 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:52.403+0000 7f3988ff9640 1 -- 172.21.15.27:0/2880217078 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f397c045020 con 0x7f398c070e80 2024-09-12T03:16:52.407 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:52.403+0000 7f3988ff9640 1 --2- 172.21.15.27:0/2880217078 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f397806be60 0x7f397806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:52.407 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:52.403+0000 7f3988ff9640 1 -- 172.21.15.27:0/2880217078 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f397c0aa9e0 con 0x7f398c070e80 2024-09-12T03:16:52.407 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:52.403+0000 7f398b7fe640 1 --2- 172.21.15.27:0/2880217078 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f397806be60 0x7f397806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:52.407 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:52.404+0000 7f398b7fe640 1 --2- 172.21.15.27:0/2880217078 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f397806be60 0x7f397806e320 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f398c0afae0 tx=0x7f39800019b0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:52.408 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:52.407+0000 7f3988ff9640 1 -- 172.21.15.27:0/2880217078 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f397c04c020 con 0x7f398c070e80 2024-09-12T03:16:52.620 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:52.619+0000 7f399282e640 1 -- 172.21.15.27:0/2880217078 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f398c0712a0 con 0x7f398c070e80 2024-09-12T03:16:52.620 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:52.619+0000 7f3988ff9640 1 -- 172.21.15.27:0/2880217078 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v6) ==== 53+0+73 (secure 0 0 0) 0x7f397c078e70 con 0x7f398c070e80 2024-09-12T03:16:52.620 INFO:teuthology.orchestra.run.smithi027.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-12T03:16:52.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:52.621+0000 7f399282e640 1 -- 172.21.15.27:0/2880217078 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f397806be60 msgr2=0x7f397806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:52.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:52.621+0000 7f399282e640 1 --2- 172.21.15.27:0/2880217078 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f397806be60 0x7f397806e320 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f398c0afae0 tx=0x7f39800019b0 comp rx=0 tx=0).stop 2024-09-12T03:16:52.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:52.621+0000 7f399282e640 1 -- 172.21.15.27:0/2880217078 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f398c070e80 msgr2=0x7f398c0cb850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:52.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:52.621+0000 7f399282e640 1 --2- 172.21.15.27:0/2880217078 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f398c070e80 0x7f398c0cb850 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f397c035040 tx=0x7f397c0046c0 comp rx=0 tx=0).stop 2024-09-12T03:16:52.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:52.622+0000 7f399282e640 1 -- 172.21.15.27:0/2880217078 shutdown_connections 2024-09-12T03:16:52.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:52.622+0000 7f399282e640 1 --2- 172.21.15.27:0/2880217078 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f397806be60 0x7f397806e320 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:52.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:52.622+0000 7f399282e640 1 --2- 172.21.15.27:0/2880217078 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f398c0d0b50 0x7f398c15e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:52.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:52.622+0000 7f399282e640 1 --2- 172.21.15.27:0/2880217078 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f398c0cef50 0x7f398c15e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:52.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:52.622+0000 7f399282e640 1 --2- 172.21.15.27:0/2880217078 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f398c070e80 0x7f398c0cb850 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:52.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:52.622+0000 7f399282e640 1 -- 172.21.15.27:0/2880217078 >> 172.21.15.27:0/2880217078 conn(0x7f398c073800 msgr2=0x7f398c0b6490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:52.624 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:52.622+0000 7f399282e640 1 -- 172.21.15.27:0/2880217078 shutdown_connections 2024-09-12T03:16:52.624 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:52.622+0000 7f399282e640 1 -- 172.21.15.27:0/2880217078 wait complete. 2024-09-12T03:16:52.635 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-12T03:16:52.635 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-12T03:16:52.635 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi045.front.sepia.ceph.com 2024-09-12T03:16:52.635 INFO:tasks.cephfs.mount:self.client.name = client.3 2024-09-12T03:16:52.635 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-09-12T03:16:52.635 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-12T03:16:52.635 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-12T03:16:52.635 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-12T03:16:52.635 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2024-09-12T03:16:52.636 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:16:52.636 DEBUG:teuthology.orchestra.run.smithi045:> ip addr 2024-09-12T03:16:52.653 INFO:teuthology.orchestra.run.smithi045.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-12T03:16:52.654 INFO:teuthology.orchestra.run.smithi045.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-12T03:16:52.654 INFO:teuthology.orchestra.run.smithi045.stdout: inet 127.0.0.1/8 scope host lo 2024-09-12T03:16:52.654 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:16:52.654 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 ::1/128 scope host 2024-09-12T03:16:52.654 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:16:52.654 INFO:teuthology.orchestra.run.smithi045.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-12T03:16:52.654 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:6b:fd:a6 brd ff:ff:ff:ff:ff:ff 2024-09-12T03:16:52.654 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp7s0f0 2024-09-12T03:16:52.654 INFO:teuthology.orchestra.run.smithi045.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-12T03:16:52.654 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:6b:fd:a7 brd ff:ff:ff:ff:ff:ff 2024-09-12T03:16:52.654 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp7s0f1 2024-09-12T03:16:52.654 INFO:teuthology.orchestra.run.smithi045.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-12T03:16:52.654 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:bd:12:be brd ff:ff:ff:ff:ff:ff 2024-09-12T03:16:52.655 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp1s0f0 2024-09-12T03:16:52.655 INFO:teuthology.orchestra.run.smithi045.stdout: inet 172.21.15.45/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-12T03:16:52.655 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft 42721sec preferred_lft 42721sec 2024-09-12T03:16:52.655 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::ec4:7aff:febd:12be/64 scope link 2024-09-12T03:16:52.655 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:16:52.655 INFO:teuthology.orchestra.run.smithi045.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-12T03:16:52.655 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:bd:12:bf brd ff:ff:ff:ff:ff:ff 2024-09-12T03:16:52.655 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp1s0f1 2024-09-12T03:16:52.655 INFO:teuthology.orchestra.run.smithi045.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-12T03:16:52.655 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 5a:ec:d7:40:ef:75 brd ff:ff:ff:ff:ff:ff 2024-09-12T03:16:52.655 INFO:teuthology.orchestra.run.smithi045.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-12T03:16:52.655 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:16:52.655 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::d8c2:5dff:fe26:78c8/64 scope link 2024-09-12T03:16:52.655 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:16:52.656 INFO:teuthology.orchestra.run.smithi045.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-12T03:16:52.656 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 5a:ec:d7:40:ef:75 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-12T03:16:52.656 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::58ec:d7ff:fe40:ef75/64 scope link tentative 2024-09-12T03:16:52.656 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:16:52.656 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:16:52.656 DEBUG:teuthology.orchestra.run.smithi045:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-12T03:16:52.719 INFO:teuthology.orchestra.run.smithi045.stdout:1 2024-09-12T03:16:52.721 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:16:52.721 DEBUG:teuthology.orchestra.run.smithi045:> route 2024-09-12T03:16:52.779 INFO:teuthology.orchestra.run.smithi045.stdout:Kernel IP routing table 2024-09-12T03:16:52.779 INFO:teuthology.orchestra.run.smithi045.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-12T03:16:52.780 INFO:teuthology.orchestra.run.smithi045.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-12T03:16:52.780 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-12T03:16:52.780 INFO:teuthology.orchestra.run.smithi045.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-12T03:16:52.780 DEBUG:teuthology.orchestra.run.smithi045:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-12T03:16:52.780 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-09-12T03:16:52.780 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-12T03:16:52.780 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-12T03:16:52.781 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-12T03:16:52.781 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-09-12T03:16:52.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:52 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T03:16:52.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T03:16:52.910 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:16:52.910 DEBUG:teuthology.orchestra.run.smithi045:> ip netns list 2024-09-12T03:16:52.926 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-12T03:16:52.927 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:16:52.927 DEBUG:teuthology.orchestra.run.smithi045:> ip netns list-id 2024-09-12T03:16:52.982 INFO:teuthology.orchestra.run.smithi045.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2024-09-12T03:16:52.982 DEBUG:teuthology.orchestra.run.smithi045:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-12T03:16:52.982 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-09-12T03:16:52.982 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-12T03:16:52.982 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2024-09-12T03:16:52.982 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-09-12T03:16:53.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:53 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T03:16:53.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T03:16:53.103 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:16:53.103 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-09-12T03:16:53.128 INFO:teuthology.orchestra.run.smithi045.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-12T03:16:53.128 INFO:teuthology.orchestra.run.smithi045.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-12T03:16:53.128 INFO:teuthology.orchestra.run.smithi045.stdout: inet 127.0.0.1/8 scope host lo 2024-09-12T03:16:53.128 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:16:53.128 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 ::1/128 scope host 2024-09-12T03:16:53.128 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:16:53.129 INFO:teuthology.orchestra.run.smithi045.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-12T03:16:53.129 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 26:6f:4a:79:f7:69 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-09-12T03:16:53.129 INFO:teuthology.orchestra.run.smithi045.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-09-12T03:16:53.129 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:16:53.129 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::246f:4aff:fe79:f769/64 scope link 2024-09-12T03:16:53.129 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:16:53.130 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:16:53.130 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-09-12T03:16:53.196 INFO:teuthology.orchestra.run.smithi045.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-12T03:16:53.196 INFO:teuthology.orchestra.run.smithi045.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-12T03:16:53.196 INFO:teuthology.orchestra.run.smithi045.stdout: inet 127.0.0.1/8 scope host lo 2024-09-12T03:16:53.197 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:16:53.197 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 ::1/128 scope host 2024-09-12T03:16:53.197 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:16:53.197 INFO:teuthology.orchestra.run.smithi045.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-12T03:16:53.197 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 26:6f:4a:79:f7:69 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-09-12T03:16:53.197 INFO:teuthology.orchestra.run.smithi045.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-09-12T03:16:53.197 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:16:53.197 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::246f:4aff:fe79:f769/64 scope link 2024-09-12T03:16:53.197 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:16:53.198 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2024-09-12T03:16:53.199 DEBUG:teuthology.orchestra.run.smithi045:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-12T03:16:53.199 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-09-12T03:16:53.199 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2024-09-12T03:16:53.199 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-09-12T03:16:53.199 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2024-09-12T03:16:53.199 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2024-09-12T03:16:53.199 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2024-09-12T03:16:53.199 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-09-12T03:16:53.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:53 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T03:16:53.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T03:16:53.355 DEBUG:teuthology.orchestra.run.smithi045:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-12T03:16:53.355 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-09-12T03:16:53.355 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link set brx.1 up 2024-09-12T03:16:53.355 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link set dev brx.1 master ceph-brx 2024-09-12T03:16:53.355 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-09-12T03:16:53.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:53 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T03:16:53.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T03:16:53.435 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-12T03:16:53.435 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2024-09-12T03:16:53.453 INFO:teuthology.orchestra.run.smithi045.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2024-09-12T03:16:53.453 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-12T03:16:53.453 DEBUG:teuthology.orchestra.run.smithi045:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2024-09-12T03:16:53.508 INFO:tasks.cephfs.kernel_mount:mounting using device: 3@.cephfs=/ 2024-09-12T03:16:53.508 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:16:53.509 DEBUG:teuthology.orchestra.run.smithi045:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 3@.cephfs=/ /home/ubuntu/cephtest/mnt.3 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-12T03:16:53.594 INFO:teuthology.orchestra.run.smithi045.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-12T03:16:53.594 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-12T03:16:53.601 INFO:teuthology.orchestra.run.smithi045.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-12T03:16:53.601 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-12T03:16:53.601 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: resolved to: "172.21.15.27:3300,172.21.15.45:3300,172.21.15.45:3301" 2024-09-12T03:16:53.601 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: trying mount with new device syntax: 3@d2df504e-2647-4385-a5db-9fdf69f487a0.cephfs=/ 2024-09-12T03:16:53.601 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,ms_mode=prefer-crc,key=3,mon_addr=172.21.15.27:3300/172.21.15.45:3300/172.21.15.45:3301" will pass to kernel 2024-09-12T03:16:53.602 INFO:teuthology.orchestra.run.smithi045.stdout:mount: /home/ubuntu/cephtest/mnt.3 does not contain SELinux labels. 2024-09-12T03:16:53.602 INFO:teuthology.orchestra.run.smithi045.stdout: You just mounted a file system that supports labels which does not 2024-09-12T03:16:53.602 INFO:teuthology.orchestra.run.smithi045.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-12T03:16:53.602 INFO:teuthology.orchestra.run.smithi045.stdout: applications will generate AVC messages and not be allowed access to 2024-09-12T03:16:53.602 INFO:teuthology.orchestra.run.smithi045.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-12T03:16:53.604 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-12T03:16:53.604 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:16:53.604 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2024-09-12T03:16:53.630 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-12T03:16:53.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:53 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-12T03:16:53.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T03:16:53.727 INFO:teuthology.orchestra.run.smithi045.stdout:client4606 2024-09-12T03:16:53.728 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-12T03:16:53.728 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T03:16:53.744 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-12T03:16:53.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:53 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-12T03:16:53.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T03:16:53.841 INFO:teuthology.orchestra.run.smithi045.stdout:client4606 2024-09-12T03:16:53.841 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-12T03:16:53.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:53 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-12T03:16:53.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T03:16:53.898 INFO:teuthology.orchestra.run.smithi045.stdout:client4606 2024-09-12T03:16:53.898 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/d2df504e-2647-4385-a5db-9fdf69f487a0.client4606/status') 2024-09-12T03:16:53.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:53 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T03:16:53.957 INFO:teuthology.orchestra.run.smithi045.stdout:instance: client.4606 (3)192.168.0.2:0/3274206049 2024-09-12T03:16:53.957 INFO:teuthology.orchestra.run.smithi045.stdout:blocklisted: false 2024-09-12T03:16:53.957 INFO:teuthology.orchestra.run.smithi045.stderr:0+1 records in 2024-09-12T03:16:53.957 INFO:teuthology.orchestra.run.smithi045.stderr:0+1 records out 2024-09-12T03:16:53.957 INFO:teuthology.orchestra.run.smithi045.stderr:69 bytes copied, 2.638e-05 s, 2.6 MB/s 2024-09-12T03:16:53.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:16:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T03:16:53.964 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2024-09-12T03:16:54.293 INFO:tasks.cephfs_test_runner:Starting test: test_binary_metafile_on_legacy_to_v1_upgrade (tasks.cephfs.test_volumes.TestMisc) Validate binary .meta file on legacy subvol root doesn't break the system 2024-09-12T03:16:54.293 INFO:tasks.cephfs_test_runner:test_binary_metafile_on_legacy_to_v1_upgrade (tasks.cephfs.test_volumes.TestMisc) 2024-09-12T03:16:54.293 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-12T03:16:54.409 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:54.408+0000 7fa5e032a640 1 Processor -- start 2024-09-12T03:16:54.409 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:54.408+0000 7fa5e032a640 1 -- start start 2024-09-12T03:16:54.410 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:54.409+0000 7fa5e032a640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fa5d815b2b0 0x7fa5d80b3560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:54.410 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:54.409+0000 7fa5e032a640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa5d8072d70 0x7fa5d8072430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:54.410 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:54.409+0000 7fa5e032a640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa5d80ce410 con 0x7fa5d806e9b0 2024-09-12T03:16:54.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:54.409+0000 7fa5e032a640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7fa5d80720a0 con 0x7fa5d815b2b0 2024-09-12T03:16:54.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:54.409+0000 7fa5e032a640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fa5d8072970 con 0x7fa5d8072d70 2024-09-12T03:16:54.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:54.409+0000 7fa5de0c6640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fa5d815b2b0 0x7fa5d80b3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:54.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:54.409+0000 7fa5de0c6640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fa5d815b2b0 0x7fa5d80b3560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59068/0 (socket says 172.21.15.27:59068) 2024-09-12T03:16:54.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:54.409+0000 7fa5de0c6640 1 -- 172.21.15.27:0/3437345455 learned_addr learned my addr 172.21.15.27:0/3437345455 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:16:54.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:54.409+0000 7fa5dd8c5640 1 --2- 172.21.15.27:0/3437345455 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa5d8072d70 0x7fa5d8072430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:54.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:54.409+0000 7fa5de0c6640 1 -- 172.21.15.27:0/3437345455 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa5d8072d70 msgr2=0x7fa5d8072430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:54.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:54.409+0000 7fa5de0c6640 1 --2- 172.21.15.27:0/3437345455 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa5d8072d70 0x7fa5d8072430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:54.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:54.409+0000 7fa5de0c6640 1 -- 172.21.15.27:0/3437345455 >> v1:172.21.15.27:6789/0 conn(0x7fa5d806e9b0 legacy=0x7fa5d80cebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:54.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:54.409+0000 7fa5de0c6640 1 -- 172.21.15.27:0/3437345455 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5d80732c0 con 0x7fa5d815b2b0 2024-09-12T03:16:54.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:54.409+0000 7fa5dd0c4640 1 -- 172.21.15.27:0/3437345455 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2758742091 0 0) 0x7fa5d0001490 con 0x7fa5d806e9b0 2024-09-12T03:16:54.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:54.409+0000 7fa5dd8c5640 1 --2- 172.21.15.27:0/3437345455 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa5d8072d70 0x7fa5d8072430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:16:54.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:54.410+0000 7fa5de0c6640 1 --2- 172.21.15.27:0/3437345455 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa5d815b2b0 0x7fa5d80b3560 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fa5d400bb10 tx=0x7fa5d402c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70118c295fcf88f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:54.413 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:54.410+0000 7fa5dd0c4640 1 -- 172.21.15.27:0/3437345455 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5d4036020 con 0x7fa5d815b2b0 2024-09-12T03:16:54.413 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:54.410+0000 7fa5dd0c4640 1 -- 172.21.15.27:0/3437345455 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5d4032820 con 0x7fa5d815b2b0 2024-09-12T03:16:54.413 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:54.410+0000 7fa5dd0c4640 1 -- 172.21.15.27:0/3437345455 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5d4032bf0 con 0x7fa5d815b2b0 2024-09-12T03:16:54.413 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:54.411+0000 7fa5e032a640 1 -- 172.21.15.27:0/3437345455 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa5d815b2b0 msgr2=0x7fa5d80b3560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:54.413 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:54.411+0000 7fa5e032a640 1 --2- 172.21.15.27:0/3437345455 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa5d815b2b0 0x7fa5d80b3560 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fa5d400bb10 tx=0x7fa5d402c920 comp rx=0 tx=0).stop 2024-09-12T03:16:54.414 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:54.411+0000 7fa5e032a640 1 -- 172.21.15.27:0/3437345455 shutdown_connections 2024-09-12T03:16:54.414 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:54.411+0000 7fa5e032a640 1 --2- 172.21.15.27:0/3437345455 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa5d8072d70 0x7fa5d8072430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:54.414 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:54.411+0000 7fa5e032a640 1 --2- 172.21.15.27:0/3437345455 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa5d815b2b0 0x7fa5d80b3560 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:54.414 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:54.411+0000 7fa5e032a640 1 -- 172.21.15.27:0/3437345455 >> 172.21.15.27:0/3437345455 conn(0x7fa5d8062c60 msgr2=0x7fa5d806bb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:54.414 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:54.411+0000 7fa5e032a640 1 -- 172.21.15.27:0/3437345455 shutdown_connections 2024-09-12T03:16:54.415 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:54.411+0000 7fa5e032a640 1 -- 172.21.15.27:0/3437345455 wait complete. 2024-09-12T03:16:54.420 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:54.412+0000 7fa5e032a640 1 Processor -- start 2024-09-12T03:16:54.421 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:54.412+0000 7fa5e032a640 1 -- start start 2024-09-12T03:16:54.421 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:54.413+0000 7fa5e032a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa5d806e9b0 0x7fa5d81f37e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:54.421 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:54.413+0000 7fa5e032a640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa5d8072d70 0x7fa5d80aa4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:54.421 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:54.413+0000 7fa5e032a640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa5d815b2b0 0x7fa5d81ecfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:54.421 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:54.413+0000 7fa5e032a640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fa5d80ce690 con 0x7fa5d815b2b0 2024-09-12T03:16:54.422 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:54.413+0000 7fa5e032a640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa5d8071dc0 con 0x7fa5d806e9b0 2024-09-12T03:16:54.422 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:54.413+0000 7fa5e032a640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fa5d815b690 con 0x7fa5d8072d70 2024-09-12T03:16:54.422 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:54.413+0000 7fa5de8c7640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa5d815b2b0 0x7fa5d81ecfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:54.422 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:54.413+0000 7fa5de8c7640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa5d815b2b0 0x7fa5d81ecfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59072/0 (socket says 172.21.15.27:59072) 2024-09-12T03:16:54.422 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:54.413+0000 7fa5de8c7640 1 -- 172.21.15.27:0/4147269668 learned_addr learned my addr 172.21.15.27:0/4147269668 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:16:54.423 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:54.413+0000 7fa5dd8c5640 1 --2- 172.21.15.27:0/4147269668 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa5d8072d70 0x7fa5d80aa4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:54.423 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:54.413+0000 7fa5de0c6640 1 --2- 172.21.15.27:0/4147269668 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa5d806e9b0 0x7fa5d81f37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:54.423 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:54.414+0000 7fa5de8c7640 1 -- 172.21.15.27:0/4147269668 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa5d8072d70 msgr2=0x7fa5d80aa4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:54.423 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:54.414+0000 7fa5de8c7640 1 --2- 172.21.15.27:0/4147269668 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa5d8072d70 0x7fa5d80aa4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:54.423 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:54.414+0000 7fa5de8c7640 1 -- 172.21.15.27:0/4147269668 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa5d806e9b0 msgr2=0x7fa5d81f37e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:54.423 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:54.414+0000 7fa5de8c7640 1 --2- 172.21.15.27:0/4147269668 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa5d806e9b0 0x7fa5d81f37e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:54.424 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:54.414+0000 7fa5de8c7640 1 -- 172.21.15.27:0/4147269668 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5d80aaf40 con 0x7fa5d815b2b0 2024-09-12T03:16:54.424 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:54.414+0000 7fa5de0c6640 1 --2- 172.21.15.27:0/4147269668 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa5d806e9b0 0x7fa5d81f37e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:16:54.424 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:54.414+0000 7fa5de8c7640 1 --2- 172.21.15.27:0/4147269668 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa5d815b2b0 0x7fa5d81ecfb0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fa5d000c7f0 tx=0x7fa5d000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:54.424 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:54.414+0000 7fa5ceffd640 1 -- 172.21.15.27:0/4147269668 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5d0003da0 con 0x7fa5d815b2b0 2024-09-12T03:16:54.424 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:54.414+0000 7fa5ceffd640 1 -- 172.21.15.27:0/4147269668 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5d0005070 con 0x7fa5d815b2b0 2024-09-12T03:16:54.425 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:54.414+0000 7fa5ceffd640 1 -- 172.21.15.27:0/4147269668 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5d0005460 con 0x7fa5d815b2b0 2024-09-12T03:16:54.425 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:54.414+0000 7fa5e032a640 1 -- 172.21.15.27:0/4147269668 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5d80ab220 con 0x7fa5d815b2b0 2024-09-12T03:16:54.425 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:54.414+0000 7fa5e032a640 1 -- 172.21.15.27:0/4147269668 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5d80ab7e0 con 0x7fa5d815b2b0 2024-09-12T03:16:54.425 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:54.415+0000 7fa5e032a640 1 -- 172.21.15.27:0/4147269668 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5d80abcc0 con 0x7fa5d815b2b0 2024-09-12T03:16:54.425 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:54.416+0000 7fa5ceffd640 1 -- 172.21.15.27:0/4147269668 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa5d0005600 con 0x7fa5d815b2b0 2024-09-12T03:16:54.426 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:54.416+0000 7fa5ceffd640 1 --2- 172.21.15.27:0/4147269668 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fa5bc06be30 0x7fa5bc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:54.426 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:54.416+0000 7fa5ceffd640 1 -- 172.21.15.27:0/4147269668 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fa5d008acc0 con 0x7fa5d815b2b0 2024-09-12T03:16:54.426 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:54.416+0000 7fa5de0c6640 1 --2- 172.21.15.27:0/4147269668 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fa5bc06be30 0x7fa5bc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:54.426 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:54.417+0000 7fa5de0c6640 1 --2- 172.21.15.27:0/4147269668 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fa5bc06be30 0x7fa5bc06e2f0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fa5d4037000 tx=0x7fa5d4032000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:54.427 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:54.421+0000 7fa5ceffd640 1 -- 172.21.15.27:0/4147269668 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fa5d80abcc0 con 0x7fa5d815b2b0 2024-09-12T03:16:54.600 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:54.599+0000 7fa5e032a640 1 -- 172.21.15.27:0/4147269668 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa5d8072430 con 0x7fa5d815b2b0 2024-09-12T03:16:54.601 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:54.600+0000 7fa5ceffd640 1 -- 172.21.15.27:0/4147269668 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16617 (secure 0 0 0) 0x7fa5d80ce410 con 0x7fa5d815b2b0 2024-09-12T03:16:54.601 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:16:54.601 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":21,"fsid":"d2df504e-2647-4385-a5db-9fdf69f487a0","created":"2024-09-12T03:16:07.778937+0000","modified":"2024-09-12T03:16:32.570182+0000","last_up_change":"2024-09-12T03:16:19.314778+0000","last_in_change":"2024-09-12T03:16:09.893086+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T03:16:20.097646+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-12T03:16:28.790791+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-12T03:16:29.909065+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1851116c-1fb3-4dc9-9772-f90e2ef4c355","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6817","nonce":3002321488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6819","nonce":3002321488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6823","nonce":3002321488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6821","nonce":3002321488}]},"public_addr":"172.21.15.27:6817/3002321488","cluster_addr":"172.21.15.27:6819/3002321488","heartbeat_back_addr":"172.21.15.27:6823/3002321488","heartbeat_front_addr":"172.21.15.27:6821/3002321488","state":["exists","up"]},{"osd":1,"uuid":"f38ae17d-73eb-4204-b99d-975fa806fb87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6809","nonce":1613102886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6811","nonce":1613102886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6815","nonce":1613102886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6813","nonce":1613102886}]},"public_addr":"172.21.15.27:6809/1613102886","cluster_addr":"172.21.15.27:6811/1613102886","heartbeat_back_addr":"172.21.15.27:6815/1613102886","heartbeat_front_addr":"172.21.15.27:6813/1613102886","state":["exists","up"]},{"osd":2,"uuid":"fb12dcc8-bdd0-42b8-b46d-8b10106224bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6825","nonce":3534723355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6827","nonce":3534723355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6831","nonce":3534723355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6829","nonce":3534723355}]},"public_addr":"172.21.15.27:6825/3534723355","cluster_addr":"172.21.15.27:6827/3534723355","heartbeat_back_addr":"172.21.15.27:6831/3534723355","heartbeat_front_addr":"172.21.15.27:6829/3534723355","state":["exists","up"]},{"osd":3,"uuid":"d74d3f49-9c24-4b9f-98e2-3ecd7b5ef3b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6801","nonce":80661598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6803","nonce":80661598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6807","nonce":80661598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6805","nonce":80661598}]},"public_addr":"172.21.15.27:6801/80661598","cluster_addr":"172.21.15.27:6803/80661598","heartbeat_back_addr":"172.21.15.27:6807/80661598","heartbeat_front_addr":"172.21.15.27:6805/80661598","state":["exists","up"]},{"osd":4,"uuid":"2bf123f2-1b0d-41c4-aa3b-805bab267e59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6817","nonce":3943189827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6819","nonce":3943189827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6823","nonce":3943189827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6821","nonce":3943189827}]},"public_addr":"172.21.15.45:6817/3943189827","cluster_addr":"172.21.15.45:6819/3943189827","heartbeat_back_addr":"172.21.15.45:6823/3943189827","heartbeat_front_addr":"172.21.15.45:6821/3943189827","state":["exists","up"]},{"osd":5,"uuid":"c49ba5da-91c6-4a86-a97b-4aba89c253ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6809","nonce":3620760330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6811","nonce":3620760330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6815","nonce":3620760330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6813","nonce":3620760330}]},"public_addr":"172.21.15.45:6809/3620760330","cluster_addr":"172.21.15.45:6811/3620760330","heartbeat_back_addr":"172.21.15.45:6815/3620760330","heartbeat_front_addr":"172.21.15.45:6813/3620760330","state":["exists","up"]},{"osd":6,"uuid":"b0123475-89f7-4a34-9e33-d797b99dd9dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6801","nonce":1989083510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6803","nonce":1989083510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6807","nonce":1989083510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6805","nonce":1989083510}]},"public_addr":"172.21.15.45:6801/1989083510","cluster_addr":"172.21.15.45:6803/1989083510","heartbeat_back_addr":"172.21.15.45:6807/1989083510","heartbeat_front_addr":"172.21.15.45:6805/1989083510","state":["exists","up"]},{"osd":7,"uuid":"4dad773d-b7d8-46fc-8a10-a972aad27b29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6825","nonce":1513300294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6827","nonce":1513300294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6831","nonce":1513300294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6829","nonce":1513300294}]},"public_addr":"172.21.15.45:6825/1513300294","cluster_addr":"172.21.15.45:6827/1513300294","heartbeat_back_addr":"172.21.15.45:6831/1513300294","heartbeat_front_addr":"172.21.15.45:6829/1513300294","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.881855+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:17.133252+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.735339+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.500742+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.477571+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:17.167667+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.470291+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.873743+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T03:16:54.603 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:54.602+0000 7fa5e032a640 1 -- 172.21.15.27:0/4147269668 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fa5bc06be30 msgr2=0x7fa5bc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:54.603 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:54.602+0000 7fa5e032a640 1 --2- 172.21.15.27:0/4147269668 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fa5bc06be30 0x7fa5bc06e2f0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fa5d4037000 tx=0x7fa5d4032000 comp rx=0 tx=0).stop 2024-09-12T03:16:54.603 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:54.602+0000 7fa5e032a640 1 -- 172.21.15.27:0/4147269668 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa5d815b2b0 msgr2=0x7fa5d81ecfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:54.603 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:54.602+0000 7fa5e032a640 1 --2- 172.21.15.27:0/4147269668 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa5d815b2b0 0x7fa5d81ecfb0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fa5d000c7f0 tx=0x7fa5d000ccb0 comp rx=0 tx=0).stop 2024-09-12T03:16:54.603 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:54.602+0000 7fa5e032a640 1 -- 172.21.15.27:0/4147269668 shutdown_connections 2024-09-12T03:16:54.603 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:54.602+0000 7fa5e032a640 1 --2- 172.21.15.27:0/4147269668 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fa5bc06be30 0x7fa5bc06e2f0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:54.604 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:54.602+0000 7fa5e032a640 1 --2- 172.21.15.27:0/4147269668 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa5d815b2b0 0x7fa5d81ecfb0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:54.604 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:54.602+0000 7fa5e032a640 1 --2- 172.21.15.27:0/4147269668 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa5d8072d70 0x7fa5d80aa4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:54.604 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:54.602+0000 7fa5e032a640 1 --2- 172.21.15.27:0/4147269668 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa5d806e9b0 0x7fa5d81f37e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:54.604 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:54.602+0000 7fa5e032a640 1 -- 172.21.15.27:0/4147269668 >> 172.21.15.27:0/4147269668 conn(0x7fa5d8062c60 msgr2=0x7fa5d80b3d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:54.604 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:54.602+0000 7fa5e032a640 1 -- 172.21.15.27:0/4147269668 shutdown_connections 2024-09-12T03:16:54.604 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:54.602+0000 7fa5e032a640 1 -- 172.21.15.27:0/4147269668 wait complete. 2024-09-12T03:16:54.614 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-12T03:16:20.097646+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-12T03:16:28.790791+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-12T03:16:29.909065+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-12T03:16:54.614 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-12T03:16:54.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:54.719+0000 7fba4129b640 1 Processor -- start 2024-09-12T03:16:54.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:54.719+0000 7fba4129b640 1 -- start start 2024-09-12T03:16:54.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:54.720+0000 7fba4129b640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fba3c0b6240 0x7fba3c0b6620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:54.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:54.720+0000 7fba4129b640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fba3c0b87d0 0x7fba3c0d4040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:54.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:54.720+0000 7fba4129b640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fba3c0b6b60 con 0x7fba3c0d3c20 2024-09-12T03:16:54.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:54.720+0000 7fba4129b640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7fba3c0d2930 con 0x7fba3c0b6240 2024-09-12T03:16:54.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:54.720+0000 7fba4129b640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fba3c0d2c10 con 0x7fba3c0b87d0 2024-09-12T03:16:54.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:54.720+0000 7fba3affd640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fba3c0b6240 0x7fba3c0b6620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:54.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:54.720+0000 7fba3affd640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fba3c0b6240 0x7fba3c0b6620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59088/0 (socket says 172.21.15.27:59088) 2024-09-12T03:16:54.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:54.720+0000 7fba327fc640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fba3c0d3c20 0x7fba3c06d960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:46542/0 (socket says 172.21.15.27:46542) 2024-09-12T03:16:54.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:54.720+0000 7fba3affd640 1 -- 172.21.15.27:0/3977247374 learned_addr learned my addr 172.21.15.27:0/3977247374 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:16:54.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:54.720+0000 7fba3affd640 1 -- 172.21.15.27:0/3977247374 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fba3c0b87d0 msgr2=0x7fba3c0d4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:54.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:54.720+0000 7fba3affd640 1 --2- 172.21.15.27:0/3977247374 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fba3c0b87d0 0x7fba3c0d4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:54.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:54.720+0000 7fba3affd640 1 -- 172.21.15.27:0/3977247374 >> v1:172.21.15.27:6789/0 conn(0x7fba3c0d3c20 legacy=0x7fba3c06d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:54.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:54.720+0000 7fba3affd640 1 -- 172.21.15.27:0/3977247374 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba3c0d2ef0 con 0x7fba3c0b6240 2024-09-12T03:16:54.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:54.720+0000 7fba3a7fc640 1 -- 172.21.15.27:0/3977247374 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 903911739 0 0) 0x7fba3c0b6b60 con 0x7fba3c0d3c20 2024-09-12T03:16:54.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:54.721+0000 7fba3affd640 1 --2- 172.21.15.27:0/3977247374 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fba3c0b6240 0x7fba3c0b6620 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fba2c009870 tx=0x7fba2c02a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aeb6be5f044064dd server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:54.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:54.721+0000 7fba3a7fc640 1 -- 172.21.15.27:0/3977247374 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba2c037070 con 0x7fba3c0b6240 2024-09-12T03:16:54.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:54.721+0000 7fba3a7fc640 1 -- 172.21.15.27:0/3977247374 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba2c02ae50 con 0x7fba3c0b6240 2024-09-12T03:16:54.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:54.721+0000 7fba3a7fc640 1 -- 172.21.15.27:0/3977247374 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba2c032860 con 0x7fba3c0b6240 2024-09-12T03:16:54.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:54.721+0000 7fba4129b640 1 -- 172.21.15.27:0/3977247374 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fba3c0b6240 msgr2=0x7fba3c0b6620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:54.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:54.721+0000 7fba4129b640 1 --2- 172.21.15.27:0/3977247374 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fba3c0b6240 0x7fba3c0b6620 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fba2c009870 tx=0x7fba2c02a300 comp rx=0 tx=0).stop 2024-09-12T03:16:54.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:54.722+0000 7fba4129b640 1 -- 172.21.15.27:0/3977247374 shutdown_connections 2024-09-12T03:16:54.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:54.722+0000 7fba4129b640 1 --2- 172.21.15.27:0/3977247374 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fba3c0b87d0 0x7fba3c0d4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:54.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:54.722+0000 7fba4129b640 1 --2- 172.21.15.27:0/3977247374 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fba3c0b6240 0x7fba3c0b6620 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:54.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:54.722+0000 7fba4129b640 1 -- 172.21.15.27:0/3977247374 >> 172.21.15.27:0/3977247374 conn(0x7fba3c0d1420 msgr2=0x7fba3c0d1830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:54.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:54.722+0000 7fba4129b640 1 -- 172.21.15.27:0/3977247374 shutdown_connections 2024-09-12T03:16:54.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:54.722+0000 7fba4129b640 1 -- 172.21.15.27:0/3977247374 wait complete. 2024-09-12T03:16:54.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:54.723+0000 7fba4129b640 1 Processor -- start 2024-09-12T03:16:54.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:54.723+0000 7fba4129b640 1 -- start start 2024-09-12T03:16:54.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:54.724+0000 7fba4129b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fba3c0b6240 0x7fba3c1f44b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:54.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:54.724+0000 7fba4129b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fba3c0b87d0 0x7fba3c15d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:54.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:54.724+0000 7fba4129b640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fba3c0d3c20 0x7fba3c1e8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:54.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:54.724+0000 7fba4129b640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fba3c15ece0 con 0x7fba3c0b6240 2024-09-12T03:16:54.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:54.724+0000 7fba4129b640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fba3c15eb60 con 0x7fba3c0b87d0 2024-09-12T03:16:54.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:54.724+0000 7fba4129b640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fba3c0cfc70 con 0x7fba3c0d3c20 2024-09-12T03:16:54.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:54.724+0000 7fba3affd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fba3c0b6240 0x7fba3c1f44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:54.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:54.724+0000 7fba3affd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fba3c0b6240 0x7fba3c1f44b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59102/0 (socket says 172.21.15.27:59102) 2024-09-12T03:16:54.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:54.724+0000 7fba3affd640 1 -- 172.21.15.27:0/4229274004 learned_addr learned my addr 172.21.15.27:0/4229274004 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:16:54.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:54.724+0000 7fba327fc640 1 --2- 172.21.15.27:0/4229274004 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fba3c0b87d0 0x7fba3c15d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:54.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:54.724+0000 7fba3b7fe640 1 --2- 172.21.15.27:0/4229274004 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fba3c0d3c20 0x7fba3c1e8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:54.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:54.724+0000 7fba3affd640 1 -- 172.21.15.27:0/4229274004 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fba3c0d3c20 msgr2=0x7fba3c1e8a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:54.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:54.724+0000 7fba3affd640 1 --2- 172.21.15.27:0/4229274004 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fba3c0d3c20 0x7fba3c1e8a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:54.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:54.724+0000 7fba3affd640 1 -- 172.21.15.27:0/4229274004 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fba3c0b87d0 msgr2=0x7fba3c15d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:54.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:54.724+0000 7fba3affd640 1 --2- 172.21.15.27:0/4229274004 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fba3c0b87d0 0x7fba3c15d8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:54.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:54.724+0000 7fba3affd640 1 -- 172.21.15.27:0/4229274004 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba3c1f4d00 con 0x7fba3c0b6240 2024-09-12T03:16:54.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:54.724+0000 7fba3affd640 1 --2- 172.21.15.27:0/4229274004 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fba3c0b6240 0x7fba3c1f44b0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fba2c0097c0 tx=0x7fba2c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:54.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:54.725+0000 7fba33fff640 1 -- 172.21.15.27:0/4229274004 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba2c042070 con 0x7fba3c0b6240 2024-09-12T03:16:54.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:54.725+0000 7fba33fff640 1 -- 172.21.15.27:0/4229274004 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba2c03e870 con 0x7fba3c0b6240 2024-09-12T03:16:54.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:54.725+0000 7fba4129b640 1 -- 172.21.15.27:0/4229274004 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba3c1f4fe0 con 0x7fba3c0b6240 2024-09-12T03:16:54.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:54.725+0000 7fba33fff640 1 -- 172.21.15.27:0/4229274004 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba2c037040 con 0x7fba3c0b6240 2024-09-12T03:16:54.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:54.725+0000 7fba4129b640 1 -- 172.21.15.27:0/4229274004 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba3c0cbaa0 con 0x7fba3c0b6240 2024-09-12T03:16:54.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:54.726+0000 7fba4129b640 1 -- 172.21.15.27:0/4229274004 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba3c0d4040 con 0x7fba3c0b6240 2024-09-12T03:16:54.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:54.726+0000 7fba33fff640 1 -- 172.21.15.27:0/4229274004 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fba2c045070 con 0x7fba3c0b6240 2024-09-12T03:16:54.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:54.727+0000 7fba33fff640 1 --2- 172.21.15.27:0/4229274004 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fba1c06be30 0x7fba1c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:54.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:54.727+0000 7fba33fff640 1 -- 172.21.15.27:0/4229274004 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fba2c0ab780 con 0x7fba3c0b6240 2024-09-12T03:16:54.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:54.727+0000 7fba327fc640 1 --2- 172.21.15.27:0/4229274004 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fba1c06be30 0x7fba1c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:54.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:54.727+0000 7fba327fc640 1 --2- 172.21.15.27:0/4229274004 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fba1c06be30 0x7fba1c06e2f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fba3c0b6970 tx=0x7fba24007c00 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:54.732 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:54.731+0000 7fba33fff640 1 -- 172.21.15.27:0/4229274004 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fba2c07a1c0 con 0x7fba3c0b6240 2024-09-12T03:16:54.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:54.905+0000 7fba4129b640 1 -- 172.21.15.27:0/4229274004 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fba3c0b7190 con 0x7fba3c0b6240 2024-09-12T03:16:54.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:54.906+0000 7fba33fff640 1 -- 172.21.15.27:0/4229274004 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) ==== 93+0+10 (secure 0 0 0) 0x7fba2c079ac0 con 0x7fba3c0b6240 2024-09-12T03:16:54.907 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 1 2024-09-12T03:16:54.909 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:54.908+0000 7fba4129b640 1 -- 172.21.15.27:0/4229274004 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fba1c06be30 msgr2=0x7fba1c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:54.909 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:54.908+0000 7fba4129b640 1 --2- 172.21.15.27:0/4229274004 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fba1c06be30 0x7fba1c06e2f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fba3c0b6970 tx=0x7fba24007c00 comp rx=0 tx=0).stop 2024-09-12T03:16:54.909 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:54.908+0000 7fba4129b640 1 -- 172.21.15.27:0/4229274004 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fba3c0b6240 msgr2=0x7fba3c1f44b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:54.910 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:54.908+0000 7fba4129b640 1 --2- 172.21.15.27:0/4229274004 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fba3c0b6240 0x7fba3c1f44b0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fba2c0097c0 tx=0x7fba2c0046c0 comp rx=0 tx=0).stop 2024-09-12T03:16:54.910 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:54.908+0000 7fba4129b640 1 -- 172.21.15.27:0/4229274004 shutdown_connections 2024-09-12T03:16:54.910 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:54.908+0000 7fba4129b640 1 --2- 172.21.15.27:0/4229274004 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fba1c06be30 0x7fba1c06e2f0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:54.910 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:54.908+0000 7fba4129b640 1 --2- 172.21.15.27:0/4229274004 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fba3c0d3c20 0x7fba3c1e8a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:54.910 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:54.908+0000 7fba4129b640 1 --2- 172.21.15.27:0/4229274004 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fba3c0b87d0 0x7fba3c15d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:54.910 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:54.908+0000 7fba4129b640 1 --2- 172.21.15.27:0/4229274004 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fba3c0b6240 0x7fba3c1f44b0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:54.910 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:54.908+0000 7fba4129b640 1 -- 172.21.15.27:0/4229274004 >> 172.21.15.27:0/4229274004 conn(0x7fba3c0d1420 msgr2=0x7fba3c0b5a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:54.910 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:54.908+0000 7fba4129b640 1 -- 172.21.15.27:0/4229274004 shutdown_connections 2024-09-12T03:16:54.910 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:54.908+0000 7fba4129b640 1 -- 172.21.15.27:0/4229274004 wait complete. 2024-09-12T03:16:54.920 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-12T03:16:55.026 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.025+0000 7f19f51d6640 1 Processor -- start 2024-09-12T03:16:55.026 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.025+0000 7f19f51d6640 1 -- start start 2024-09-12T03:16:55.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.026+0000 7f19f51d6640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f19f015e990 0x7f19f00d1d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:55.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.026+0000 7f19f51d6640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f19f00d2720 con 0x7f19f00b6200 2024-09-12T03:16:55.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.026+0000 7f19f51d6640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f19f00cf910 con 0x7f19f00b6d50 2024-09-12T03:16:55.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.026+0000 7f19f51d6640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f19f00cfbf0 con 0x7f19f015e990 2024-09-12T03:16:55.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.026+0000 7f19e7fff640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f19f00b6200 0x7f19f00b6640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:46554/0 (socket says 172.21.15.27:46554) 2024-09-12T03:16:55.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.026+0000 7f19e7fff640 1 -- 172.21.15.27:0/190080937 learned_addr learned my addr 172.21.15.27:0/190080937 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:16:55.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.026+0000 7f19eed9d640 1 --2- 172.21.15.27:0/190080937 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f19f015e990 0x7f19f00d1d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:55.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.026+0000 7f19ee59c640 1 -- 172.21.15.27:0/190080937 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 447902466 0 0) 0x7f19f00d2720 con 0x7f19f00b6200 2024-09-12T03:16:55.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.026+0000 7f19ee59c640 1 -- 172.21.15.27:0/190080937 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f19dc003520 con 0x7f19f00b6200 2024-09-12T03:16:55.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.026+0000 7f19ee59c640 1 -- 172.21.15.27:0/190080937 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 685908690 0 0) 0x7f19dc003520 con 0x7f19f00b6200 2024-09-12T03:16:55.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.026+0000 7f19ee59c640 1 -- 172.21.15.27:0/190080937 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f19f015e990 msgr2=0x7f19f00d1d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:55.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.026+0000 7f19ee59c640 1 --2- 172.21.15.27:0/190080937 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f19f015e990 0x7f19f00d1d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:55.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.026+0000 7f19ee59c640 1 -- 172.21.15.27:0/190080937 >> v1:172.21.15.45:6789/0 conn(0x7f19f00b6d50 legacy=0x7f19f00d2f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:55.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.026+0000 7f19ee59c640 1 -- 172.21.15.27:0/190080937 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19f00d29e0 con 0x7f19f00b6200 2024-09-12T03:16:55.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.027+0000 7f19ee59c640 1 -- 172.21.15.27:0/190080937 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 847755213 0 0) 0x7f19d8001670 con 0x7f19f00b6200 2024-09-12T03:16:55.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.027+0000 7f19ee59c640 1 -- 172.21.15.27:0/190080937 >> v1:172.21.15.27:6789/0 conn(0x7f19f00b6200 legacy=0x7f19f00b6640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:55.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.027+0000 7f19ee59c640 1 --2- 172.21.15.27:0/190080937 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f19dc0039e0 0x7f19dc003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:55.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.027+0000 7f19ee59c640 1 --2- 172.21.15.27:0/190080937 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f19f015e990 0x7f19dc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:55.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.027+0000 7f19ee59c640 1 --2- 172.21.15.27:0/190080937 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f19dc004d40 0x7f19dc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:55.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.027+0000 7f19ee59c640 1 -- 172.21.15.27:0/190080937 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f19f00cfbf0 con 0x7f19dc0039e0 2024-09-12T03:16:55.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.027+0000 7f19ee59c640 1 -- 172.21.15.27:0/190080937 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f19dc00bb80 con 0x7f19dc004d40 2024-09-12T03:16:55.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.027+0000 7f19ee59c640 1 -- 172.21.15.27:0/190080937 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f19dc00bd80 con 0x7f19f015e990 2024-09-12T03:16:55.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.027+0000 7f19eed9d640 1 --2- 172.21.15.27:0/190080937 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f19dc0039e0 0x7f19dc003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:55.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.027+0000 7f19e7fff640 1 --2- 172.21.15.27:0/190080937 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f19f015e990 0x7f19dc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:55.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.027+0000 7f19ef59e640 1 --2- 172.21.15.27:0/190080937 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f19dc004d40 0x7f19dc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:55.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.027+0000 7f19eed9d640 1 -- 172.21.15.27:0/190080937 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f19f015e990 msgr2=0x7f19dc004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:55.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.027+0000 7f19eed9d640 1 --2- 172.21.15.27:0/190080937 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f19f015e990 0x7f19dc004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:55.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.027+0000 7f19eed9d640 1 -- 172.21.15.27:0/190080937 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f19dc004d40 msgr2=0x7f19dc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:55.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.027+0000 7f19eed9d640 1 --2- 172.21.15.27:0/190080937 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f19dc004d40 0x7f19dc0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:55.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.027+0000 7f19eed9d640 1 -- 172.21.15.27:0/190080937 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19dc003520 con 0x7f19dc0039e0 2024-09-12T03:16:55.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.027+0000 7f19ef59e640 1 --2- 172.21.15.27:0/190080937 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f19dc004d40 0x7f19dc0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:16:55.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.028+0000 7f19eed9d640 1 --2- 172.21.15.27:0/190080937 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f19dc0039e0 0x7f19dc003e00 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f19e0009ac0 tx=0x7f19dc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d31758b1f7262278 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:55.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.028+0000 7f19ee59c640 1 -- 172.21.15.27:0/190080937 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19e0036070 con 0x7f19dc0039e0 2024-09-12T03:16:55.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.028+0000 7f19ee59c640 1 -- 172.21.15.27:0/190080937 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19e0031400 con 0x7f19dc0039e0 2024-09-12T03:16:55.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.028+0000 7f19ee59c640 1 -- 172.21.15.27:0/190080937 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19e0009690 con 0x7f19dc0039e0 2024-09-12T03:16:55.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.028+0000 7f19f51d6640 1 -- 172.21.15.27:0/190080937 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f19dc0039e0 msgr2=0x7f19dc003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:55.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.028+0000 7f19f51d6640 1 --2- 172.21.15.27:0/190080937 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f19dc0039e0 0x7f19dc003e00 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f19e0009ac0 tx=0x7f19dc005830 comp rx=0 tx=0).stop 2024-09-12T03:16:55.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.028+0000 7f19ef59e640 1 -- 172.21.15.27:0/190080937 reap_dead start 2024-09-12T03:16:55.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.028+0000 7f19f51d6640 1 -- 172.21.15.27:0/190080937 shutdown_connections 2024-09-12T03:16:55.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.028+0000 7f19f51d6640 1 -- 172.21.15.27:0/190080937 >> 172.21.15.27:0/190080937 conn(0x7f19f00d1200 msgr2=0x7f19f00d1610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:55.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.028+0000 7f19f51d6640 1 -- 172.21.15.27:0/190080937 shutdown_connections 2024-09-12T03:16:55.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.028+0000 7f19f51d6640 1 -- 172.21.15.27:0/190080937 wait complete. 2024-09-12T03:16:55.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.030+0000 7f19f51d6640 1 Processor -- start 2024-09-12T03:16:55.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.030+0000 7f19f51d6640 1 -- start start 2024-09-12T03:16:55.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.030+0000 7f19f51d6640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f19dc0039e0 0x7f19f0150150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:55.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.030+0000 7f19f51d6640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f19f01537d0 0x7f19f014cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:55.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.031+0000 7f19f51d6640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f19f014d710 0x7f19f00ada20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:55.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.031+0000 7f19f51d6640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f19f00d0050 con 0x7f19dc0039e0 2024-09-12T03:16:55.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.031+0000 7f19f51d6640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f19f00cfed0 con 0x7f19f014d710 2024-09-12T03:16:55.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.031+0000 7f19f51d6640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f19f00d1d20 con 0x7f19f01537d0 2024-09-12T03:16:55.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.031+0000 7f19eed9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f19dc0039e0 0x7f19f0150150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:55.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.031+0000 7f19eed9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f19dc0039e0 0x7f19f0150150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59114/0 (socket says 172.21.15.27:59114) 2024-09-12T03:16:55.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.031+0000 7f19eed9d640 1 -- 172.21.15.27:0/191052161 learned_addr learned my addr 172.21.15.27:0/191052161 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:16:55.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.031+0000 7f19e7fff640 1 --2- 172.21.15.27:0/191052161 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f19f01537d0 0x7f19f014cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:55.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.031+0000 7f19ef59e640 1 --2- 172.21.15.27:0/191052161 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f19f014d710 0x7f19f00ada20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:55.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.031+0000 7f19eed9d640 1 -- 172.21.15.27:0/191052161 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f19f01537d0 msgr2=0x7f19f014cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:55.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.031+0000 7f19eed9d640 1 --2- 172.21.15.27:0/191052161 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f19f01537d0 0x7f19f014cf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:55.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.031+0000 7f19eed9d640 1 -- 172.21.15.27:0/191052161 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f19f014d710 msgr2=0x7f19f00ada20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:55.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.031+0000 7f19eed9d640 1 --2- 172.21.15.27:0/191052161 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f19f014d710 0x7f19f00ada20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:55.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.031+0000 7f19eed9d640 1 -- 172.21.15.27:0/191052161 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19f00ccb70 con 0x7f19dc0039e0 2024-09-12T03:16:55.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.031+0000 7f19eed9d640 1 --2- 172.21.15.27:0/191052161 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f19dc0039e0 0x7f19f0150150 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f19e0009ac0 tx=0x7f19e0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:55.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.031+0000 7f19e77fe640 1 -- 172.21.15.27:0/191052161 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19e0040020 con 0x7f19dc0039e0 2024-09-12T03:16:55.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.031+0000 7f19e77fe640 1 -- 172.21.15.27:0/191052161 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19e0003bc0 con 0x7f19dc0039e0 2024-09-12T03:16:55.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.031+0000 7f19f51d6640 1 -- 172.21.15.27:0/191052161 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19f00cce50 con 0x7f19dc0039e0 2024-09-12T03:16:55.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.031+0000 7f19e77fe640 1 -- 172.21.15.27:0/191052161 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19e0036040 con 0x7f19dc0039e0 2024-09-12T03:16:55.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.031+0000 7f19f51d6640 1 -- 172.21.15.27:0/191052161 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19f00cd410 con 0x7f19dc0039e0 2024-09-12T03:16:55.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.032+0000 7f19f51d6640 1 -- 172.21.15.27:0/191052161 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19f00693d0 con 0x7f19dc0039e0 2024-09-12T03:16:55.037 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.033+0000 7f19e77fe640 1 -- 172.21.15.27:0/191052161 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f19e0043020 con 0x7f19dc0039e0 2024-09-12T03:16:55.037 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.033+0000 7f19e77fe640 1 --2- 172.21.15.27:0/191052161 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f19d006be60 0x7f19d006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:55.037 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.033+0000 7f19e77fe640 1 -- 172.21.15.27:0/191052161 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f19e00aad70 con 0x7f19dc0039e0 2024-09-12T03:16:55.037 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.034+0000 7f19e7fff640 1 --2- 172.21.15.27:0/191052161 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f19d006be60 0x7f19d006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:55.037 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.034+0000 7f19e7fff640 1 --2- 172.21.15.27:0/191052161 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f19d006be60 0x7f19d006e320 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f19f0161b20 tx=0x7f19d80078f0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:55.038 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.038+0000 7f19e77fe640 1 -- 172.21.15.27:0/191052161 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f19e004a070 con 0x7f19dc0039e0 2024-09-12T03:16:55.212 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.211+0000 7f19f51d6640 1 -- 172.21.15.27:0/191052161 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f19f00d2110 con 0x7f19dc0039e0 2024-09-12T03:16:55.212 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.211+0000 7f19e77fe640 1 -- 172.21.15.27:0/191052161 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) ==== 104+0+11 (secure 0 0 0) 0x7f19e00044e0 con 0x7f19dc0039e0 2024-09-12T03:16:55.213 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 32 2024-09-12T03:16:55.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.213+0000 7f19f51d6640 1 -- 172.21.15.27:0/191052161 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f19d006be60 msgr2=0x7f19d006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:55.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.213+0000 7f19f51d6640 1 --2- 172.21.15.27:0/191052161 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f19d006be60 0x7f19d006e320 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f19f0161b20 tx=0x7f19d80078f0 comp rx=0 tx=0).stop 2024-09-12T03:16:55.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.213+0000 7f19f51d6640 1 -- 172.21.15.27:0/191052161 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f19dc0039e0 msgr2=0x7f19f0150150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:55.215 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.213+0000 7f19f51d6640 1 --2- 172.21.15.27:0/191052161 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f19dc0039e0 0x7f19f0150150 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f19e0009ac0 tx=0x7f19e0004730 comp rx=0 tx=0).stop 2024-09-12T03:16:55.215 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.213+0000 7f19f51d6640 1 -- 172.21.15.27:0/191052161 shutdown_connections 2024-09-12T03:16:55.215 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.213+0000 7f19f51d6640 1 --2- 172.21.15.27:0/191052161 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f19d006be60 0x7f19d006e320 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:55.215 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.213+0000 7f19f51d6640 1 --2- 172.21.15.27:0/191052161 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f19f014d710 0x7f19f00ada20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:55.215 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.213+0000 7f19f51d6640 1 --2- 172.21.15.27:0/191052161 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f19f01537d0 0x7f19f014cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:55.215 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.213+0000 7f19f51d6640 1 --2- 172.21.15.27:0/191052161 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f19dc0039e0 0x7f19f0150150 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:55.215 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.213+0000 7f19f51d6640 1 -- 172.21.15.27:0/191052161 >> 172.21.15.27:0/191052161 conn(0x7f19f00d1200 msgr2=0x7f19f00b7970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:55.215 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.213+0000 7f19f51d6640 1 -- 172.21.15.27:0/191052161 shutdown_connections 2024-09-12T03:16:55.215 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.213+0000 7f19f51d6640 1 -- 172.21.15.27:0/191052161 wait complete. 2024-09-12T03:16:55.225 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-12T03:16:55.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.329+0000 7f5077047640 1 Processor -- start 2024-09-12T03:16:55.331 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.330+0000 7f5077047640 1 -- start start 2024-09-12T03:16:55.332 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.330+0000 7f5077047640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f50700737e0 0x7f50700b6930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:55.332 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.330+0000 7f5077047640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f50700b55b0 con 0x7f50700b4950 2024-09-12T03:16:55.332 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.330+0000 7f5077047640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f50700b5890 con 0x7f507015c6e0 2024-09-12T03:16:55.332 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.330+0000 7f5077047640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f50700b5b70 con 0x7f50700737e0 2024-09-12T03:16:55.332 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.331+0000 7f506ffff640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f50700b4950 0x7f50700d03b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:46570/0 (socket says 172.21.15.27:46570) 2024-09-12T03:16:55.332 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.331+0000 7f506ffff640 1 -- 172.21.15.27:0/3833542890 learned_addr learned my addr 172.21.15.27:0/3833542890 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:16:55.332 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.331+0000 7f50755e4640 1 --2- 172.21.15.27:0/3833542890 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f50700737e0 0x7f50700b6930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:55.332 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.331+0000 7f506f7fe640 1 -- 172.21.15.27:0/3833542890 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2522170441 0 0) 0x7f50700b55b0 con 0x7f50700b4950 2024-09-12T03:16:55.333 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.331+0000 7f506f7fe640 1 -- 172.21.15.27:0/3833542890 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5060003520 con 0x7f50700b4950 2024-09-12T03:16:55.333 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.331+0000 7f50755e4640 1 -- 172.21.15.27:0/3833542890 >> v1:172.21.15.45:6789/0 conn(0x7f507015c6e0 legacy=0x7f50700b4240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:55.333 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.331+0000 7f50755e4640 1 -- 172.21.15.27:0/3833542890 >> v1:172.21.15.27:6789/0 conn(0x7f50700b4950 legacy=0x7f50700d03b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:55.333 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.331+0000 7f506f7fe640 1 -- 172.21.15.27:0/3833542890 <== mon.1 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3093876836 0 0) 0x7f50700b5890 con 0x7f507015c6e0 2024-09-12T03:16:55.333 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.331+0000 7f50755e4640 1 -- 172.21.15.27:0/3833542890 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50700b5e50 con 0x7f50700737e0 2024-09-12T03:16:55.334 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.332+0000 7f50755e4640 1 --2- 172.21.15.27:0/3833542890 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f50700737e0 0x7f50700b6930 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f5068009dc0 tx=0x7f506802a6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ffe9e240286640c9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:55.334 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.332+0000 7f506f7fe640 1 -- 172.21.15.27:0/3833542890 <== mon.2 v2:172.21.15.45:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5068007740 con 0x7f50700737e0 2024-09-12T03:16:55.334 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.332+0000 7f506f7fe640 1 -- 172.21.15.27:0/3833542890 <== mon.2 v2:172.21.15.45:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50680078e0 con 0x7f50700737e0 2024-09-12T03:16:55.334 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.332+0000 7f506f7fe640 1 -- 172.21.15.27:0/3833542890 <== mon.2 v2:172.21.15.45:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5068007c90 con 0x7f50700737e0 2024-09-12T03:16:55.334 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.333+0000 7f5077047640 1 -- 172.21.15.27:0/3833542890 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f50700737e0 msgr2=0x7f50700b6930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:55.334 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.333+0000 7f5077047640 1 --2- 172.21.15.27:0/3833542890 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f50700737e0 0x7f50700b6930 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f5068009dc0 tx=0x7f506802a6d0 comp rx=0 tx=0).stop 2024-09-12T03:16:55.334 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.333+0000 7f5077047640 1 -- 172.21.15.27:0/3833542890 shutdown_connections 2024-09-12T03:16:55.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.333+0000 7f5077047640 1 --2- 172.21.15.27:0/3833542890 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f50700737e0 0x7f50700b6930 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:55.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.333+0000 7f5077047640 1 -- 172.21.15.27:0/3833542890 >> 172.21.15.27:0/3833542890 conn(0x7f5070062630 msgr2=0x7f5070062a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:55.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.333+0000 7f5077047640 1 -- 172.21.15.27:0/3833542890 shutdown_connections 2024-09-12T03:16:55.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.333+0000 7f5077047640 1 -- 172.21.15.27:0/3833542890 wait complete. 2024-09-12T03:16:55.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.334+0000 7f5077047640 1 Processor -- start 2024-09-12T03:16:55.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.334+0000 7f5077047640 1 -- start start 2024-09-12T03:16:55.337 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.335+0000 7f5077047640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f50700737e0 0x7f507015e520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:55.337 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.335+0000 7f5077047640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f50700b4950 0x7f5070152c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:55.337 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.335+0000 7f5077047640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f507015c6e0 0x7f507014f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:55.337 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.335+0000 7f5077047640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f50700d0ac0 con 0x7f50700737e0 2024-09-12T03:16:55.338 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.335+0000 7f5077047640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f50700b6410 con 0x7f50700b4950 2024-09-12T03:16:55.338 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.335+0000 7f5077047640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f50700d0c40 con 0x7f507015c6e0 2024-09-12T03:16:55.338 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.335+0000 7f5074de3640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f50700737e0 0x7f507015e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:55.338 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.335+0000 7f5074de3640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f50700737e0 0x7f507015e520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59126/0 (socket says 172.21.15.27:59126) 2024-09-12T03:16:55.338 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.335+0000 7f5074de3640 1 -- 172.21.15.27:0/2749021429 learned_addr learned my addr 172.21.15.27:0/2749021429 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:16:55.338 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.335+0000 7f50755e4640 1 --2- 172.21.15.27:0/2749021429 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f507015c6e0 0x7f507014f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:55.338 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.335+0000 7f5074de3640 1 -- 172.21.15.27:0/2749021429 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f507015c6e0 msgr2=0x7f507014f9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:55.338 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.335+0000 7f5074de3640 1 --2- 172.21.15.27:0/2749021429 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f507015c6e0 0x7f507014f9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:55.338 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.335+0000 7f5074de3640 1 -- 172.21.15.27:0/2749021429 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f50700b4950 msgr2=0x7f5070152c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:55.338 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.335+0000 7f506ffff640 1 --2- 172.21.15.27:0/2749021429 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f50700b4950 0x7f5070152c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:55.338 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.335+0000 7f5074de3640 1 --2- 172.21.15.27:0/2749021429 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f50700b4950 0x7f5070152c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:55.338 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.335+0000 7f5074de3640 1 -- 172.21.15.27:0/2749021429 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f507015f920 con 0x7f50700737e0 2024-09-12T03:16:55.338 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.335+0000 7f5074de3640 1 --2- 172.21.15.27:0/2749021429 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f50700737e0 0x7f507015e520 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f506400c410 tx=0x7f506400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:55.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.336+0000 7f506d7fa640 1 -- 172.21.15.27:0/2749021429 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5064017070 con 0x7f50700737e0 2024-09-12T03:16:55.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.336+0000 7f506d7fa640 1 -- 172.21.15.27:0/2749021429 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50640035f0 con 0x7f50700737e0 2024-09-12T03:16:55.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.336+0000 7f5077047640 1 -- 172.21.15.27:0/2749021429 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f507013ff50 con 0x7f50700737e0 2024-09-12T03:16:55.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.336+0000 7f506d7fa640 1 -- 172.21.15.27:0/2749021429 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50640039e0 con 0x7f50700737e0 2024-09-12T03:16:55.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.336+0000 7f5077047640 1 -- 172.21.15.27:0/2749021429 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5070140510 con 0x7f50700737e0 2024-09-12T03:16:55.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.337+0000 7f5077047640 1 -- 172.21.15.27:0/2749021429 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50700b6930 con 0x7f50700737e0 2024-09-12T03:16:55.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.337+0000 7f506d7fa640 1 -- 172.21.15.27:0/2749021429 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f506401a070 con 0x7f50700737e0 2024-09-12T03:16:55.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.338+0000 7f506d7fa640 1 --2- 172.21.15.27:0/2749021429 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f505406be30 0x7f505406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:55.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.338+0000 7f506d7fa640 1 -- 172.21.15.27:0/2749021429 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f506408a2b0 con 0x7f50700737e0 2024-09-12T03:16:55.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.338+0000 7f506ffff640 1 --2- 172.21.15.27:0/2749021429 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f505406be30 0x7f505406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:55.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.339+0000 7f506ffff640 1 --2- 172.21.15.27:0/2749021429 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f505406be30 0x7f505406e2f0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f50700b18a0 tx=0x7f505c009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:55.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.342+0000 7f506d7fa640 1 -- 172.21.15.27:0/2749021429 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f5064058c70 con 0x7f50700737e0 2024-09-12T03:16:55.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.520+0000 7f5077047640 1 -- 172.21.15.27:0/2749021429 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f5070062bb0 con 0x7f50700737e0 2024-09-12T03:16:55.522 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.520+0000 7f506d7fa640 1 -- 172.21.15.27:0/2749021429 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) ==== 100+0+11 (secure 0 0 0) 0x7f5064058570 con 0x7f50700737e0 2024-09-12T03:16:55.522 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 64 2024-09-12T03:16:55.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.523+0000 7f5077047640 1 -- 172.21.15.27:0/2749021429 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f505406be30 msgr2=0x7f505406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:55.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.523+0000 7f5077047640 1 --2- 172.21.15.27:0/2749021429 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f505406be30 0x7f505406e2f0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f50700b18a0 tx=0x7f505c009040 comp rx=0 tx=0).stop 2024-09-12T03:16:55.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.523+0000 7f5077047640 1 -- 172.21.15.27:0/2749021429 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f50700737e0 msgr2=0x7f507015e520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:55.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.523+0000 7f5077047640 1 --2- 172.21.15.27:0/2749021429 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f50700737e0 0x7f507015e520 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f506400c410 tx=0x7f506400c8d0 comp rx=0 tx=0).stop 2024-09-12T03:16:55.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.523+0000 7f5077047640 1 -- 172.21.15.27:0/2749021429 shutdown_connections 2024-09-12T03:16:55.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.523+0000 7f5077047640 1 --2- 172.21.15.27:0/2749021429 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f505406be30 0x7f505406e2f0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:55.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.523+0000 7f5077047640 1 --2- 172.21.15.27:0/2749021429 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f507015c6e0 0x7f507014f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:55.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.523+0000 7f5077047640 1 --2- 172.21.15.27:0/2749021429 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f50700b4950 0x7f5070152c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:55.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.523+0000 7f5077047640 1 --2- 172.21.15.27:0/2749021429 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f50700737e0 0x7f507015e520 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:55.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.523+0000 7f5077047640 1 -- 172.21.15.27:0/2749021429 >> 172.21.15.27:0/2749021429 conn(0x7f5070062630 msgr2=0x7f507015ac80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:55.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.523+0000 7f5077047640 1 -- 172.21.15.27:0/2749021429 shutdown_connections 2024-09-12T03:16:55.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.523+0000 7f5077047640 1 -- 172.21.15.27:0/2749021429 wait complete. 2024-09-12T03:16:55.535 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestMisc.test_binary_metafile_on_legacy_to_v1_upgrade' 2024-09-12T03:16:55.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.639+0000 7f2367af1640 1 Processor -- start 2024-09-12T03:16:55.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.640+0000 7f2367af1640 1 -- start start 2024-09-12T03:16:55.642 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.640+0000 7f2367af1640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f236015eb90 0x7f23600d2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:55.642 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.640+0000 7f2367af1640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f23600d1720 0x7f23600b8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:55.642 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.640+0000 7f2367af1640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f23600d3760 con 0x7f23600d5200 2024-09-12T03:16:55.642 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.640+0000 7f2367af1640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f23600b7d20 con 0x7f23600d1720 2024-09-12T03:16:55.642 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.640+0000 7f2367af1640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f23600d42e0 con 0x7f236015eb90 2024-09-12T03:16:55.642 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.640+0000 7f236608e640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f23600d1720 0x7f23600b8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:55.642 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.640+0000 7f236508c640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f23600d5200 0x7f23600d1010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:46586/0 (socket says 172.21.15.27:46586) 2024-09-12T03:16:55.642 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.640+0000 7f236508c640 1 -- 172.21.15.27:0/2027973916 learned_addr learned my addr 172.21.15.27:0/2027973916 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:16:55.642 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.641+0000 7f236588d640 1 --2- 172.21.15.27:0/2027973916 >> v2:172.21.15.45:3300/0 conn(0x7f236015eb90 0x7f23600d2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:55.642 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.641+0000 7f236608e640 1 -- 172.21.15.27:0/2027973916 >> v2:172.21.15.45:3300/0 conn(0x7f236015eb90 msgr2=0x7f23600d2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:55.643 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.641+0000 7f236608e640 1 --2- 172.21.15.27:0/2027973916 >> v2:172.21.15.45:3300/0 conn(0x7f236015eb90 0x7f23600d2df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:55.643 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.641+0000 7f236608e640 1 -- 172.21.15.27:0/2027973916 >> v1:172.21.15.27:6789/0 conn(0x7f23600d5200 legacy=0x7f23600d1010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:55.643 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.641+0000 7f236608e640 1 -- 172.21.15.27:0/2027973916 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23600d3a40 con 0x7f23600d1720 2024-09-12T03:16:55.644 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.641+0000 7f236608e640 1 --2- 172.21.15.27:0/2027973916 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f23600d1720 0x7f23600b8190 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f2360075560 tx=0x7f235002a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=717110e8265757ab server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:55.644 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.642+0000 7f236488b640 1 -- 172.21.15.27:0/2027973916 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2350007740 con 0x7f23600d1720 2024-09-12T03:16:55.644 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.642+0000 7f236488b640 1 -- 172.21.15.27:0/2027973916 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23500078e0 con 0x7f23600d1720 2024-09-12T03:16:55.644 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.642+0000 7f236488b640 1 -- 172.21.15.27:0/2027973916 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2350007cb0 con 0x7f23600d1720 2024-09-12T03:16:55.644 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.642+0000 7f2367af1640 1 -- 172.21.15.27:0/2027973916 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f23600d1720 msgr2=0x7f23600b8190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:55.644 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.642+0000 7f2367af1640 1 --2- 172.21.15.27:0/2027973916 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f23600d1720 0x7f23600b8190 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f2360075560 tx=0x7f235002a680 comp rx=0 tx=0).stop 2024-09-12T03:16:55.644 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.642+0000 7f2367af1640 1 -- 172.21.15.27:0/2027973916 shutdown_connections 2024-09-12T03:16:55.645 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.642+0000 7f2367af1640 1 --2- 172.21.15.27:0/2027973916 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f23600d1720 0x7f23600b8190 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:55.645 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.642+0000 7f2367af1640 1 --2- 172.21.15.27:0/2027973916 >> v2:172.21.15.45:3300/0 conn(0x7f236015eb90 0x7f23600d2df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:55.645 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.642+0000 7f2367af1640 1 -- 172.21.15.27:0/2027973916 >> 172.21.15.27:0/2027973916 conn(0x7f236006dce0 msgr2=0x7f236006e0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:55.645 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.642+0000 7f2367af1640 1 -- 172.21.15.27:0/2027973916 shutdown_connections 2024-09-12T03:16:55.645 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.642+0000 7f2367af1640 1 -- 172.21.15.27:0/2027973916 wait complete. 2024-09-12T03:16:55.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.643+0000 7f2367af1640 1 Processor -- start 2024-09-12T03:16:55.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.644+0000 7f2367af1640 1 -- start start 2024-09-12T03:16:55.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.644+0000 7f2367af1640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f23600d1720 0x7f23601512c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:55.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.644+0000 7f2367af1640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f23600d5200 0x7f23601567d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:55.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.644+0000 7f2367af1640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f236015eb90 0x7f236015c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:55.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.644+0000 7f2367af1640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f23600d3330 con 0x7f23600d1720 2024-09-12T03:16:55.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.644+0000 7f2367af1640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f23600d4000 con 0x7f23600d5200 2024-09-12T03:16:55.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.644+0000 7f2367af1640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f23600d34b0 con 0x7f236015eb90 2024-09-12T03:16:55.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.644+0000 7f236588d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f23600d1720 0x7f23601512c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:55.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.644+0000 7f236588d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f23600d1720 0x7f23601512c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59134/0 (socket says 172.21.15.27:59134) 2024-09-12T03:16:55.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.644+0000 7f236588d640 1 -- 172.21.15.27:0/1484110082 learned_addr learned my addr 172.21.15.27:0/1484110082 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:16:55.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.645+0000 7f236608e640 1 --2- 172.21.15.27:0/1484110082 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f236015eb90 0x7f236015c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:55.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.645+0000 7f236508c640 1 --2- 172.21.15.27:0/1484110082 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f23600d5200 0x7f23601567d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:55.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.645+0000 7f236588d640 1 -- 172.21.15.27:0/1484110082 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f236015eb90 msgr2=0x7f236015c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:55.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.645+0000 7f236588d640 1 --2- 172.21.15.27:0/1484110082 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f236015eb90 0x7f236015c9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:55.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.645+0000 7f236588d640 1 -- 172.21.15.27:0/1484110082 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f23600d5200 msgr2=0x7f23601567d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:55.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.645+0000 7f236588d640 1 --2- 172.21.15.27:0/1484110082 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f23600d5200 0x7f23601567d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:55.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.645+0000 7f236588d640 1 -- 172.21.15.27:0/1484110082 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2360155a10 con 0x7f23600d1720 2024-09-12T03:16:55.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.645+0000 7f236608e640 1 --2- 172.21.15.27:0/1484110082 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f236015eb90 0x7f236015c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:16:55.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.645+0000 7f236508c640 1 --2- 172.21.15.27:0/1484110082 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f23600d5200 0x7f23601567d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:16:55.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.645+0000 7f236588d640 1 --2- 172.21.15.27:0/1484110082 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f23600d1720 0x7f23601512c0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f235800d800 tx=0x7f235800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:55.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.645+0000 7f235e7fc640 1 -- 172.21.15.27:0/1484110082 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f235800b6c0 con 0x7f23600d1720 2024-09-12T03:16:55.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.645+0000 7f235e7fc640 1 -- 172.21.15.27:0/1484110082 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2358019070 con 0x7f23600d1720 2024-09-12T03:16:55.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.645+0000 7f2367af1640 1 -- 172.21.15.27:0/1484110082 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2360155cf0 con 0x7f23600d1720 2024-09-12T03:16:55.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.645+0000 7f235e7fc640 1 -- 172.21.15.27:0/1484110082 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f235800ba80 con 0x7f23600d1720 2024-09-12T03:16:55.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.645+0000 7f2367af1640 1 -- 172.21.15.27:0/1484110082 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23601562b0 con 0x7f23600d1720 2024-09-12T03:16:55.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.646+0000 7f2367af1640 1 -- 172.21.15.27:0/1484110082 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23600693d0 con 0x7f23600d1720 2024-09-12T03:16:55.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.646+0000 7f235e7fc640 1 -- 172.21.15.27:0/1484110082 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f235800bc20 con 0x7f23600d1720 2024-09-12T03:16:55.650 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.647+0000 7f235e7fc640 1 --2- 172.21.15.27:0/1484110082 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f234c06be30 0x7f234c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:55.650 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.647+0000 7f235e7fc640 1 -- 172.21.15.27:0/1484110082 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f235808b3c0 con 0x7f23600d1720 2024-09-12T03:16:55.650 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.647+0000 7f236508c640 1 --2- 172.21.15.27:0/1484110082 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f234c06be30 0x7f234c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:55.650 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.648+0000 7f236508c640 1 --2- 172.21.15.27:0/1484110082 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f234c06be30 0x7f234c06e2f0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f2360158390 tx=0x7f2354009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:55.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.651+0000 7f235e7fc640 1 -- 172.21.15.27:0/1484110082 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f23600693d0 con 0x7f23600d1720 2024-09-12T03:16:55.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:55.895+0000 7f2367af1640 1 -- 172.21.15.27:0/1484110082 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_binary_metafile_on_legacy_to_v1_upgrade"]} v 0) -- 0x7f236015f090 con 0x7f23600d1720 2024-09-12T03:16:56.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:56.634+0000 7f235e7fc640 1 -- 172.21.15.27:0/1484110082 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_binary_metafile_on_legacy_to_v1_upgrade"]}]=0 v45) ==== 161+0+0 (secure 0 0 0) 0x7f235801c360 con 0x7f23600d1720 2024-09-12T03:16:56.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:56.636+0000 7f2367af1640 1 -- 172.21.15.27:0/1484110082 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f234c06be30 msgr2=0x7f234c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:56.638 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:56.636+0000 7f2367af1640 1 --2- 172.21.15.27:0/1484110082 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f234c06be30 0x7f234c06e2f0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f2360158390 tx=0x7f2354009040 comp rx=0 tx=0).stop 2024-09-12T03:16:56.638 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:56.636+0000 7f2367af1640 1 -- 172.21.15.27:0/1484110082 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f23600d1720 msgr2=0x7f23601512c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:56.638 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:56.636+0000 7f2367af1640 1 --2- 172.21.15.27:0/1484110082 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f23600d1720 0x7f23601512c0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f235800d800 tx=0x7f235800dcc0 comp rx=0 tx=0).stop 2024-09-12T03:16:56.638 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:56.636+0000 7f2367af1640 1 -- 172.21.15.27:0/1484110082 shutdown_connections 2024-09-12T03:16:56.638 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:56.636+0000 7f2367af1640 1 --2- 172.21.15.27:0/1484110082 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f234c06be30 0x7f234c06e2f0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:56.638 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:56.636+0000 7f2367af1640 1 --2- 172.21.15.27:0/1484110082 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f236015eb90 0x7f236015c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:56.638 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:56.636+0000 7f2367af1640 1 --2- 172.21.15.27:0/1484110082 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f23600d5200 0x7f23601567d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:56.638 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:56.636+0000 7f2367af1640 1 --2- 172.21.15.27:0/1484110082 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f23600d1720 0x7f23601512c0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:56.638 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:56.636+0000 7f2367af1640 1 -- 172.21.15.27:0/1484110082 >> 172.21.15.27:0/1484110082 conn(0x7f236006dce0 msgr2=0x7f23600d1010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:56.638 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:56.636+0000 7f2367af1640 1 -- 172.21.15.27:0/1484110082 shutdown_connections 2024-09-12T03:16:56.639 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:56.636+0000 7f2367af1640 1 -- 172.21.15.27:0/1484110082 wait complete. 2024-09-12T03:16:56.650 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-12T03:16:56.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:56.758+0000 7efc91bfd640 1 Processor -- start 2024-09-12T03:16:56.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:56.758+0000 7efc91bfd640 1 -- start start 2024-09-12T03:16:56.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:56.759+0000 7efc91bfd640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7efc8c15a270 0x7efc8c06cff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:56.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:56.759+0000 7efc91bfd640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efc8c0ce650 con 0x7efc8c06d530 2024-09-12T03:16:56.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:56.759+0000 7efc91bfd640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efc8c0ce930 con 0x7efc8c0cfb20 2024-09-12T03:16:56.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:56.759+0000 7efc91bfd640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7efc8c0cef80 con 0x7efc8c15a270 2024-09-12T03:16:56.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:56.759+0000 7efc8affd640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7efc8c06d530 0x7efc8c0cf410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:46594/0 (socket says 172.21.15.27:46594) 2024-09-12T03:16:56.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:56.759+0000 7efc8affd640 1 -- 172.21.15.27:0/1421572582 learned_addr learned my addr 172.21.15.27:0/1421572582 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:16:56.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:56.759+0000 7efc8a7fc640 1 -- 172.21.15.27:0/1421572582 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2586691706 0 0) 0x7efc8c0ce650 con 0x7efc8c06d530 2024-09-12T03:16:56.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:56.759+0000 7efc8a7fc640 1 -- 172.21.15.27:0/1421572582 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efc74003520 con 0x7efc8c06d530 2024-09-12T03:16:56.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:56.759+0000 7efc8b7fe640 1 --2- 172.21.15.27:0/1421572582 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7efc8c15a270 0x7efc8c06cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:56.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:56.759+0000 7efc8a7fc640 1 -- 172.21.15.27:0/1421572582 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 265498775 0 0) 0x7efc74003520 con 0x7efc8c06d530 2024-09-12T03:16:56.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:56.759+0000 7efc8a7fc640 1 -- 172.21.15.27:0/1421572582 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7efc8c15a270 msgr2=0x7efc8c06cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:56.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:56.759+0000 7efc8a7fc640 1 --2- 172.21.15.27:0/1421572582 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7efc8c15a270 0x7efc8c06cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:56.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:56.759+0000 7efc8a7fc640 1 -- 172.21.15.27:0/1421572582 >> v1:172.21.15.45:6789/0 conn(0x7efc8c0cfb20 legacy=0x7efc8c0726a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:56.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:56.759+0000 7efc8a7fc640 1 -- 172.21.15.27:0/1421572582 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc8c0cec10 con 0x7efc8c06d530 2024-09-12T03:16:56.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:56.759+0000 7efc8a7fc640 1 -- 172.21.15.27:0/1421572582 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 847755213 0 0) 0x7efc80001670 con 0x7efc8c06d530 2024-09-12T03:16:56.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:56.760+0000 7efc8a7fc640 1 -- 172.21.15.27:0/1421572582 >> v1:172.21.15.27:6789/0 conn(0x7efc8c06d530 legacy=0x7efc8c0cf410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:56.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:56.760+0000 7efc8a7fc640 1 --2- 172.21.15.27:0/1421572582 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efc740039e0 0x7efc74003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:56.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:56.760+0000 7efc8a7fc640 1 --2- 172.21.15.27:0/1421572582 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7efc8c15a270 0x7efc74004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:56.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:56.760+0000 7efc8a7fc640 1 --2- 172.21.15.27:0/1421572582 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efc74004d40 0x7efc740051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:56.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:56.760+0000 7efc8a7fc640 1 -- 172.21.15.27:0/1421572582 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7efc8c0cef80 con 0x7efc740039e0 2024-09-12T03:16:56.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:56.760+0000 7efc8a7fc640 1 -- 172.21.15.27:0/1421572582 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7efc7400ba70 con 0x7efc74004d40 2024-09-12T03:16:56.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:56.760+0000 7efc8a7fc640 1 -- 172.21.15.27:0/1421572582 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7efc7400bc70 con 0x7efc8c15a270 2024-09-12T03:16:56.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:56.760+0000 7efc8b7fe640 1 --2- 172.21.15.27:0/1421572582 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efc740039e0 0x7efc74003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:56.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:56.760+0000 7efc8affd640 1 --2- 172.21.15.27:0/1421572582 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7efc8c15a270 0x7efc74004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:56.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:56.760+0000 7efc8bfff640 1 --2- 172.21.15.27:0/1421572582 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efc74004d40 0x7efc740051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:56.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:56.760+0000 7efc8b7fe640 1 -- 172.21.15.27:0/1421572582 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7efc8c15a270 msgr2=0x7efc74004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:56.764 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:56.760+0000 7efc8b7fe640 1 --2- 172.21.15.27:0/1421572582 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7efc8c15a270 0x7efc74004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:56.764 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:56.760+0000 7efc8b7fe640 1 -- 172.21.15.27:0/1421572582 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efc74004d40 msgr2=0x7efc740051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:56.764 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:56.760+0000 7efc8b7fe640 1 --2- 172.21.15.27:0/1421572582 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efc74004d40 0x7efc740051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:56.764 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:56.760+0000 7efc8b7fe640 1 -- 172.21.15.27:0/1421572582 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc74003520 con 0x7efc740039e0 2024-09-12T03:16:56.764 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:56.760+0000 7efc8affd640 1 --2- 172.21.15.27:0/1421572582 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7efc8c15a270 0x7efc74004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:16:56.764 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:56.761+0000 7efc8b7fe640 1 --2- 172.21.15.27:0/1421572582 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efc740039e0 0x7efc74003e00 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7efc7c009a90 tx=0x7efc74005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0ff83b86a47462c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:56.764 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:56.761+0000 7efc8a7fc640 1 -- 172.21.15.27:0/1421572582 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc7c036070 con 0x7efc740039e0 2024-09-12T03:16:56.764 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:56.761+0000 7efc8a7fc640 1 -- 172.21.15.27:0/1421572582 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc7c031400 con 0x7efc740039e0 2024-09-12T03:16:56.764 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:56.761+0000 7efc8a7fc640 1 -- 172.21.15.27:0/1421572582 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc7c009690 con 0x7efc740039e0 2024-09-12T03:16:56.767 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:56.761+0000 7efc91bfd640 1 -- 172.21.15.27:0/1421572582 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efc740039e0 msgr2=0x7efc74003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:56.767 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:56.761+0000 7efc91bfd640 1 --2- 172.21.15.27:0/1421572582 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efc740039e0 0x7efc74003e00 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7efc7c009a90 tx=0x7efc74005830 comp rx=0 tx=0).stop 2024-09-12T03:16:56.767 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:56.761+0000 7efc8bfff640 1 -- 172.21.15.27:0/1421572582 reap_dead start 2024-09-12T03:16:56.767 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:56.761+0000 7efc91bfd640 1 -- 172.21.15.27:0/1421572582 shutdown_connections 2024-09-12T03:16:56.767 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:56.761+0000 7efc91bfd640 1 -- 172.21.15.27:0/1421572582 >> 172.21.15.27:0/1421572582 conn(0x7efc8c06bb30 msgr2=0x7efc8c06bf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:56.767 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:56.761+0000 7efc91bfd640 1 -- 172.21.15.27:0/1421572582 shutdown_connections 2024-09-12T03:16:56.767 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:56.761+0000 7efc91bfd640 1 -- 172.21.15.27:0/1421572582 wait complete. 2024-09-12T03:16:56.767 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:56.763+0000 7efc91bfd640 1 Processor -- start 2024-09-12T03:16:56.767 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:56.763+0000 7efc91bfd640 1 -- start start 2024-09-12T03:16:56.767 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:56.764+0000 7efc91bfd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efc740039e0 0x7efc8c0b5030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:56.768 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:56.764+0000 7efc91bfd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efc8c1577d0 0x7efc8c0aeb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:56.768 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:56.764+0000 7efc91bfd640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7efc8c1f47a0 0x7efc8c0b3e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:56.768 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:56.764+0000 7efc91bfd640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7efc8c06c7d0 con 0x7efc8c1577d0 2024-09-12T03:16:56.768 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:56.764+0000 7efc91bfd640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7efc8c06c650 con 0x7efc740039e0 2024-09-12T03:16:56.768 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:56.764+0000 7efc91bfd640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7efc8c06c950 con 0x7efc8c1f47a0 2024-09-12T03:16:56.768 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:56.764+0000 7efc8affd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efc8c1577d0 0x7efc8c0aeb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:56.768 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:56.764+0000 7efc8affd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efc8c1577d0 0x7efc8c0aeb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59160/0 (socket says 172.21.15.27:59160) 2024-09-12T03:16:56.768 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:56.764+0000 7efc8affd640 1 -- 172.21.15.27:0/622087519 learned_addr learned my addr 172.21.15.27:0/622087519 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:16:56.768 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:56.764+0000 7efc8b7fe640 1 --2- 172.21.15.27:0/622087519 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efc740039e0 0x7efc8c0b5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:56.768 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:56.764+0000 7efc8bfff640 1 --2- 172.21.15.27:0/622087519 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7efc8c1f47a0 0x7efc8c0b3e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:56.768 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:56.764+0000 7efc8affd640 1 -- 172.21.15.27:0/622087519 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7efc8c1f47a0 msgr2=0x7efc8c0b3e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:56.768 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:56.764+0000 7efc8affd640 1 --2- 172.21.15.27:0/622087519 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7efc8c1f47a0 0x7efc8c0b3e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:56.769 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:56.764+0000 7efc8affd640 1 -- 172.21.15.27:0/622087519 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efc740039e0 msgr2=0x7efc8c0b5030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:56.769 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:56.764+0000 7efc8affd640 1 --2- 172.21.15.27:0/622087519 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efc740039e0 0x7efc8c0b5030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:56.769 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:56.764+0000 7efc8affd640 1 -- 172.21.15.27:0/622087519 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc8c1595a0 con 0x7efc8c1577d0 2024-09-12T03:16:56.769 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:56.764+0000 7efc8affd640 1 --2- 172.21.15.27:0/622087519 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efc8c1577d0 0x7efc8c0aeb40 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7efc8000e820 tx=0x7efc8000ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:56.769 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:56.764+0000 7efc73fff640 1 -- 172.21.15.27:0/622087519 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc8000c8d0 con 0x7efc8c1577d0 2024-09-12T03:16:56.769 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:56.764+0000 7efc73fff640 1 -- 172.21.15.27:0/622087519 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc80018070 con 0x7efc8c1577d0 2024-09-12T03:16:56.769 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:56.764+0000 7efc73fff640 1 -- 172.21.15.27:0/622087519 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc8000cc90 con 0x7efc8c1577d0 2024-09-12T03:16:56.769 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:56.764+0000 7efc91bfd640 1 -- 172.21.15.27:0/622087519 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc8c159880 con 0x7efc8c1577d0 2024-09-12T03:16:56.769 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:56.764+0000 7efc91bfd640 1 -- 172.21.15.27:0/622087519 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc8c0b5570 con 0x7efc8c1577d0 2024-09-12T03:16:56.769 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:56.765+0000 7efc91bfd640 1 -- 172.21.15.27:0/622087519 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc8c15d560 con 0x7efc8c1577d0 2024-09-12T03:16:56.769 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:56.766+0000 7efc73fff640 1 -- 172.21.15.27:0/622087519 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7efc8001d460 con 0x7efc8c1577d0 2024-09-12T03:16:56.770 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:56.766+0000 7efc73fff640 1 --2- 172.21.15.27:0/622087519 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7efc7806be30 0x7efc7806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:56.770 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:56.767+0000 7efc73fff640 1 -- 172.21.15.27:0/622087519 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7efc8008ac00 con 0x7efc8c1577d0 2024-09-12T03:16:56.770 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:56.767+0000 7efc8b7fe640 1 --2- 172.21.15.27:0/622087519 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7efc7806be30 0x7efc7806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:56.770 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:56.767+0000 7efc8b7fe640 1 --2- 172.21.15.27:0/622087519 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7efc7806be30 0x7efc7806e2f0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7efc7c033000 tx=0x7efc7c008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:56.771 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:56.771+0000 7efc73fff640 1 -- 172.21.15.27:0/622087519 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7efc80091080 con 0x7efc8c1577d0 2024-09-12T03:16:56.947 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:56.946+0000 7efc91bfd640 1 -- 172.21.15.27:0/622087519 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7efc8c0ccdb0 con 0x7efc8c1577d0 2024-09-12T03:16:56.950 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:56.949+0000 7efc73fff640 1 -- 172.21.15.27:0/622087519 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) ==== 122+0+0 (secure 0 0 0) 0x7efc80058f60 con 0x7efc8c1577d0 2024-09-12T03:16:56.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:56.952+0000 7efc91bfd640 1 -- 172.21.15.27:0/622087519 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7efc7806be30 msgr2=0x7efc7806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:56.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:56.952+0000 7efc91bfd640 1 --2- 172.21.15.27:0/622087519 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7efc7806be30 0x7efc7806e2f0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7efc7c033000 tx=0x7efc7c008030 comp rx=0 tx=0).stop 2024-09-12T03:16:56.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:56.952+0000 7efc91bfd640 1 -- 172.21.15.27:0/622087519 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efc8c1577d0 msgr2=0x7efc8c0aeb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:56.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:56.952+0000 7efc91bfd640 1 --2- 172.21.15.27:0/622087519 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efc8c1577d0 0x7efc8c0aeb40 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7efc8000e820 tx=0x7efc8000ece0 comp rx=0 tx=0).stop 2024-09-12T03:16:56.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:56.952+0000 7efc91bfd640 1 -- 172.21.15.27:0/622087519 shutdown_connections 2024-09-12T03:16:56.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:56.952+0000 7efc91bfd640 1 --2- 172.21.15.27:0/622087519 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7efc7806be30 0x7efc7806e2f0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:56.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:56.952+0000 7efc91bfd640 1 --2- 172.21.15.27:0/622087519 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7efc8c1f47a0 0x7efc8c0b3e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:56.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:56.952+0000 7efc91bfd640 1 --2- 172.21.15.27:0/622087519 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efc8c1577d0 0x7efc8c0aeb40 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:56.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:56.952+0000 7efc91bfd640 1 --2- 172.21.15.27:0/622087519 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efc740039e0 0x7efc8c0b5030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:56.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:56.952+0000 7efc91bfd640 1 -- 172.21.15.27:0/622087519 >> 172.21.15.27:0/622087519 conn(0x7efc8c06bb30 msgr2=0x7efc8c0b1b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:56.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:56.952+0000 7efc91bfd640 1 -- 172.21.15.27:0/622087519 shutdown_connections 2024-09-12T03:16:56.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:56.952+0000 7efc91bfd640 1 -- 172.21.15.27:0/622087519 wait complete. 2024-09-12T03:16:56.964 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-12T03:16:56.964 DEBUG:teuthology.orchestra.run.smithi027:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-12T03:16:57.003 DEBUG:teuthology.orchestra.run.smithi045:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-12T03:16:57.057 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-12T03:16:57.058 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-12T03:16:57.058 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T03:16:57.074 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-12T03:16:57.075 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T03:16:57.131 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-12T03:16:57.131 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T03:16:57.187 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-12T03:16:57.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:57.328+0000 7f882faf1640 1 Processor -- start 2024-09-12T03:16:57.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:57.329+0000 7f882faf1640 1 -- start start 2024-09-12T03:16:57.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:57.329+0000 7f882faf1640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f8828072ba0 0x7f882806d010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:57.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:57.330+0000 7f882faf1640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f882806daf0 con 0x7f8828071be0 2024-09-12T03:16:57.331 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:57.330+0000 7f882faf1640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f882806ddd0 con 0x7f882806d550 2024-09-12T03:16:57.331 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:57.330+0000 7f882faf1640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f882806e0b0 con 0x7f8828072ba0 2024-09-12T03:16:57.331 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:57.330+0000 7f882d88d640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f8828071be0 0x7f882815a8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:46602/0 (socket says 172.21.15.27:46602) 2024-09-12T03:16:57.331 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:57.330+0000 7f882d88d640 1 -- 172.21.15.27:0/2156900946 learned_addr learned my addr 172.21.15.27:0/2156900946 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:16:57.333 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:57.330+0000 7f882c88b640 1 -- 172.21.15.27:0/2156900946 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2843846563 0 0) 0x7f882806daf0 con 0x7f8828071be0 2024-09-12T03:16:57.333 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:57.330+0000 7f882c88b640 1 -- 172.21.15.27:0/2156900946 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8810003520 con 0x7f8828071be0 2024-09-12T03:16:57.333 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:57.330+0000 7f882d08c640 1 --2- 172.21.15.27:0/2156900946 >> v2:172.21.15.45:3300/0 conn(0x7f8828072ba0 0x7f882806d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:57.334 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:57.330+0000 7f882c88b640 1 -- 172.21.15.27:0/2156900946 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3040974018 0 0) 0x7f8810003520 con 0x7f8828071be0 2024-09-12T03:16:57.334 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:57.330+0000 7f882c88b640 1 -- 172.21.15.27:0/2156900946 >> v2:172.21.15.45:3300/0 conn(0x7f8828072ba0 msgr2=0x7f882806d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:57.334 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:57.330+0000 7f882c88b640 1 --2- 172.21.15.27:0/2156900946 >> v2:172.21.15.45:3300/0 conn(0x7f8828072ba0 0x7f882806d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:57.334 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:57.330+0000 7f882c88b640 1 -- 172.21.15.27:0/2156900946 >> v1:172.21.15.45:6789/0 conn(0x7f882806d550 legacy=0x7f88280cdb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:57.334 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:57.330+0000 7f882c88b640 1 -- 172.21.15.27:0/2156900946 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f882806e390 con 0x7f8828071be0 2024-09-12T03:16:57.334 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:57.330+0000 7f882c88b640 1 -- 172.21.15.27:0/2156900946 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 847755213 0 0) 0x7f88180019f0 con 0x7f8828071be0 2024-09-12T03:16:57.334 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:57.330+0000 7f882c88b640 1 -- 172.21.15.27:0/2156900946 >> v1:172.21.15.27:6789/0 conn(0x7f8828071be0 legacy=0x7f882815a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:57.334 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:57.330+0000 7f882c88b640 1 --2- 172.21.15.27:0/2156900946 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f88100039b0 0x7f8810003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:57.334 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:57.330+0000 7f882c88b640 1 --2- 172.21.15.27:0/2156900946 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8810004800 0x7f8810004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:57.334 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:57.330+0000 7f882c88b640 1 --2- 172.21.15.27:0/2156900946 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f88100051c0 0x7f881000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:57.334 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:57.330+0000 7f882c88b640 1 -- 172.21.15.27:0/2156900946 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f882806e0b0 con 0x7f8810004800 2024-09-12T03:16:57.334 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:57.330+0000 7f882c88b640 1 -- 172.21.15.27:0/2156900946 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f881000c060 con 0x7f88100039b0 2024-09-12T03:16:57.334 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:57.330+0000 7f882c88b640 1 -- 172.21.15.27:0/2156900946 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f881000c260 con 0x7f88100051c0 2024-09-12T03:16:57.334 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:57.330+0000 7f882d08c640 1 --2- 172.21.15.27:0/2156900946 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8810004800 0x7f8810004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:57.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:57.331+0000 7f882e08e640 1 --2- 172.21.15.27:0/2156900946 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f88100051c0 0x7f881000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:57.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:57.331+0000 7f882d88d640 1 --2- 172.21.15.27:0/2156900946 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f88100039b0 0x7f8810003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:57.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:57.331+0000 7f882d08c640 1 -- 172.21.15.27:0/2156900946 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f88100051c0 msgr2=0x7f881000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:57.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:57.331+0000 7f882d08c640 1 --2- 172.21.15.27:0/2156900946 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f88100051c0 0x7f881000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:57.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:57.331+0000 7f882d08c640 1 -- 172.21.15.27:0/2156900946 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f88100039b0 msgr2=0x7f8810003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:57.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:57.331+0000 7f882d08c640 1 --2- 172.21.15.27:0/2156900946 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f88100039b0 0x7f8810003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:57.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:57.331+0000 7f882d08c640 1 -- 172.21.15.27:0/2156900946 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8810003520 con 0x7f8810004800 2024-09-12T03:16:57.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:57.331+0000 7f882e08e640 1 -- 172.21.15.27:0/2156900946 reap_dead start 2024-09-12T03:16:57.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:57.331+0000 7f882d88d640 1 --2- 172.21.15.27:0/2156900946 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f88100039b0 0x7f8810003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:16:57.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:57.332+0000 7f882d08c640 1 --2- 172.21.15.27:0/2156900946 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8810004800 0x7f8810004c80 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f881c009ac0 tx=0x7f8810005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47441ebc756d6cb server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:57.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:57.332+0000 7f882c88b640 1 -- 172.21.15.27:0/2156900946 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f881c036070 con 0x7f8810004800 2024-09-12T03:16:57.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:57.332+0000 7f882c88b640 1 -- 172.21.15.27:0/2156900946 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f881c031400 con 0x7f8810004800 2024-09-12T03:16:57.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:57.332+0000 7f882c88b640 1 -- 172.21.15.27:0/2156900946 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f881c009690 con 0x7f8810004800 2024-09-12T03:16:57.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:57.332+0000 7f882faf1640 1 -- 172.21.15.27:0/2156900946 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8810004800 msgr2=0x7f8810004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:57.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:57.332+0000 7f882faf1640 1 --2- 172.21.15.27:0/2156900946 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8810004800 0x7f8810004c80 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f881c009ac0 tx=0x7f8810005830 comp rx=0 tx=0).stop 2024-09-12T03:16:57.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:57.332+0000 7f882faf1640 1 -- 172.21.15.27:0/2156900946 shutdown_connections 2024-09-12T03:16:57.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:57.332+0000 7f882faf1640 1 --2- 172.21.15.27:0/2156900946 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8810004800 0x7f8810004c80 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:57.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:57.332+0000 7f882faf1640 1 -- 172.21.15.27:0/2156900946 >> 172.21.15.27:0/2156900946 conn(0x7f8828062c60 msgr2=0x7f882806bb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:57.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:57.332+0000 7f882faf1640 1 -- 172.21.15.27:0/2156900946 shutdown_connections 2024-09-12T03:16:57.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:57.332+0000 7f882faf1640 1 -- 172.21.15.27:0/2156900946 wait complete. 2024-09-12T03:16:57.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:57.334+0000 7f882faf1640 1 Processor -- start 2024-09-12T03:16:57.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:57.334+0000 7f882faf1640 1 -- start start 2024-09-12T03:16:57.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:57.334+0000 7f882faf1640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8810004800 0x7f88281f6130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:57.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:57.334+0000 7f882faf1640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f882815d360 0x7f8828144ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:57.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:57.334+0000 7f882faf1640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8828145020 0x7f88281f1660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:57.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:57.334+0000 7f882faf1640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f882806d010 con 0x7f8810004800 2024-09-12T03:16:57.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:57.334+0000 7f882faf1640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f88280b3770 con 0x7f8828145020 2024-09-12T03:16:57.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:57.334+0000 7f882faf1640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f882806d190 con 0x7f882815d360 2024-09-12T03:16:57.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:57.334+0000 7f882d88d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8810004800 0x7f88281f6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:57.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:57.334+0000 7f882d88d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8810004800 0x7f88281f6130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59178/0 (socket says 172.21.15.27:59178) 2024-09-12T03:16:57.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:57.334+0000 7f882d88d640 1 -- 172.21.15.27:0/1098813221 learned_addr learned my addr 172.21.15.27:0/1098813221 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:16:57.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:57.335+0000 7f882d08c640 1 --2- 172.21.15.27:0/1098813221 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f882815d360 0x7f8828144ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:57.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:57.335+0000 7f882e08e640 1 --2- 172.21.15.27:0/1098813221 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8828145020 0x7f88281f1660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:57.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:57.335+0000 7f882d88d640 1 -- 172.21.15.27:0/1098813221 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f882815d360 msgr2=0x7f8828144ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:57.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:57.335+0000 7f882d88d640 1 --2- 172.21.15.27:0/1098813221 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f882815d360 0x7f8828144ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:57.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:57.335+0000 7f882d88d640 1 -- 172.21.15.27:0/1098813221 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8828145020 msgr2=0x7f88281f1660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:57.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:57.335+0000 7f882d88d640 1 --2- 172.21.15.27:0/1098813221 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8828145020 0x7f88281f1660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:57.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:57.335+0000 7f882d88d640 1 -- 172.21.15.27:0/1098813221 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88281f6670 con 0x7f8810004800 2024-09-12T03:16:57.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:57.335+0000 7f882d88d640 1 --2- 172.21.15.27:0/1098813221 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8810004800 0x7f88281f6130 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f88180079e0 tx=0x7f881800e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:57.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:57.335+0000 7f88267fc640 1 -- 172.21.15.27:0/1098813221 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f881800edf0 con 0x7f8810004800 2024-09-12T03:16:57.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:57.335+0000 7f88267fc640 1 -- 172.21.15.27:0/1098813221 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8818018070 con 0x7f8810004800 2024-09-12T03:16:57.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:57.335+0000 7f882faf1640 1 -- 172.21.15.27:0/1098813221 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f882815c2c0 con 0x7f8810004800 2024-09-12T03:16:57.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:57.335+0000 7f88267fc640 1 -- 172.21.15.27:0/1098813221 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8818013620 con 0x7f8810004800 2024-09-12T03:16:57.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:57.335+0000 7f882faf1640 1 -- 172.21.15.27:0/1098813221 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f882815c880 con 0x7f8810004800 2024-09-12T03:16:57.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:57.336+0000 7f882faf1640 1 -- 172.21.15.27:0/1098813221 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f882815a8c0 con 0x7f8810004800 2024-09-12T03:16:57.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:57.337+0000 7f88267fc640 1 -- 172.21.15.27:0/1098813221 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f88180137c0 con 0x7f8810004800 2024-09-12T03:16:57.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:57.337+0000 7f88267fc640 1 --2- 172.21.15.27:0/1098813221 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f881406be30 0x7f881406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:57.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:57.337+0000 7f88267fc640 1 -- 172.21.15.27:0/1098813221 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f881808b340 con 0x7f8810004800 2024-09-12T03:16:57.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:57.337+0000 7f882d08c640 1 --2- 172.21.15.27:0/1098813221 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f881406be30 0x7f881406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:57.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:57.338+0000 7f882d08c640 1 --2- 172.21.15.27:0/1098813221 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f881406be30 0x7f881406e2f0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f881c033000 tx=0x7f881c008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:57.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:57.341+0000 7f88267fc640 1 -- 172.21.15.27:0/1098813221 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f88180596a0 con 0x7f8810004800 2024-09-12T03:16:57.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:57.530+0000 7f882faf1640 1 -- 172.21.15.27:0/1098813221 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f88280b4d70 con 0x7f8810004800 2024-09-12T03:16:57.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:57.530+0000 7f88267fc640 1 -- 172.21.15.27:0/1098813221 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v21) ==== 80+0+0 (secure 0 0 0) 0x7f881805d9a0 con 0x7f8810004800 2024-09-12T03:16:57.531 INFO:teuthology.orchestra.run.smithi027.stderr:listed 0 entries 2024-09-12T03:16:57.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:57.532+0000 7f882faf1640 1 -- 172.21.15.27:0/1098813221 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f881406be30 msgr2=0x7f881406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:57.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:57.532+0000 7f882faf1640 1 --2- 172.21.15.27:0/1098813221 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f881406be30 0x7f881406e2f0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f881c033000 tx=0x7f881c008030 comp rx=0 tx=0).stop 2024-09-12T03:16:57.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:57.532+0000 7f882faf1640 1 -- 172.21.15.27:0/1098813221 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8810004800 msgr2=0x7f88281f6130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:57.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:57.532+0000 7f882faf1640 1 --2- 172.21.15.27:0/1098813221 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8810004800 0x7f88281f6130 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f88180079e0 tx=0x7f881800e530 comp rx=0 tx=0).stop 2024-09-12T03:16:57.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:57.532+0000 7f882faf1640 1 -- 172.21.15.27:0/1098813221 shutdown_connections 2024-09-12T03:16:57.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:57.532+0000 7f882faf1640 1 --2- 172.21.15.27:0/1098813221 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f881406be30 0x7f881406e2f0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:57.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:57.532+0000 7f882faf1640 1 --2- 172.21.15.27:0/1098813221 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8828145020 0x7f88281f1660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:57.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:57.532+0000 7f882faf1640 1 --2- 172.21.15.27:0/1098813221 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f882815d360 0x7f8828144ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:57.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:57.532+0000 7f882faf1640 1 --2- 172.21.15.27:0/1098813221 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8810004800 0x7f88281f6130 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:57.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:57.532+0000 7f882faf1640 1 -- 172.21.15.27:0/1098813221 >> 172.21.15.27:0/1098813221 conn(0x7f8828062c60 msgr2=0x7f882814ffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:57.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:57.532+0000 7f882faf1640 1 -- 172.21.15.27:0/1098813221 shutdown_connections 2024-09-12T03:16:57.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:57.532+0000 7f882faf1640 1 -- 172.21.15.27:0/1098813221 wait complete. 2024-09-12T03:16:57.545 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-12T03:16:57.545 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:16:57.548 DEBUG:teuthology.orchestra.run.smithi027:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2024-09-12T03:16:57.576 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi027.front.sepia.ceph.com 2024-09-12T03:16:57.576 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:16:57.576 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-12T03:16:57.646 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-12T03:16:57.646 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T03:16:57.666 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-12T03:16:57.666 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T03:16:57.726 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-12T03:16:57.726 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T03:16:57.785 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-12T03:16:57.888 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:57.888+0000 7f7327e77640 1 Processor -- start 2024-09-12T03:16:57.889 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:57.888+0000 7f7327e77640 1 -- start start 2024-09-12T03:16:57.890 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:57.888+0000 7f7327e77640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f732015a8c0 0x7f73200cdb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:57.890 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:57.888+0000 7f7327e77640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f7320071be0 0x7f732006e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:57.890 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:57.888+0000 7f7327e77640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f732006daf0 con 0x7f732015aca0 2024-09-12T03:16:57.890 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:57.888+0000 7f7327e77640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f732006d8a0 con 0x7f7320071be0 2024-09-12T03:16:57.890 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:57.889+0000 7f7327e77640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f732006ddd0 con 0x7f732015a8c0 2024-09-12T03:16:57.891 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:57.889+0000 7f7325412640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f7320071be0 0x7f732006e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:57.891 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:57.889+0000 7f7325412640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f7320071be0 0x7f732006e8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59182/0 (socket says 172.21.15.27:59182) 2024-09-12T03:16:57.891 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:57.889+0000 7f7325412640 1 -- 172.21.15.27:0/244563431 learned_addr learned my addr 172.21.15.27:0/244563431 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:16:57.891 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:57.889+0000 7f7325c13640 1 --2- 172.21.15.27:0/244563431 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f732015a8c0 0x7f73200cdb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:57.891 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:57.889+0000 7f7325412640 1 -- 172.21.15.27:0/244563431 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f732015a8c0 msgr2=0x7f73200cdb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:57.891 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:57.889+0000 7f7325412640 1 --2- 172.21.15.27:0/244563431 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f732015a8c0 0x7f73200cdb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:57.891 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:57.889+0000 7f7325412640 1 -- 172.21.15.27:0/244563431 >> v1:172.21.15.45:6789/0 conn(0x7f732015aca0 legacy=0x7f73200b4710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:57.891 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:57.889+0000 7f7325412640 1 -- 172.21.15.27:0/244563431 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f732006e0b0 con 0x7f7320071be0 2024-09-12T03:16:57.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:57.890+0000 7f7325412640 1 --2- 172.21.15.27:0/244563431 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7320071be0 0x7f732006e8b0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f731000bb10 tx=0x7f731002c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eccc576a83d08951 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:57.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:57.890+0000 7f7324c11640 1 -- 172.21.15.27:0/244563431 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7310032790 con 0x7f7320071be0 2024-09-12T03:16:57.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:57.890+0000 7f7324c11640 1 -- 172.21.15.27:0/244563431 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7310038070 con 0x7f7320071be0 2024-09-12T03:16:57.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:57.890+0000 7f7324c11640 1 -- 172.21.15.27:0/244563431 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7310032b50 con 0x7f7320071be0 2024-09-12T03:16:57.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:57.890+0000 7f7327e77640 1 -- 172.21.15.27:0/244563431 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7320071be0 msgr2=0x7f732006e8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:57.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:57.890+0000 7f7327e77640 1 --2- 172.21.15.27:0/244563431 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7320071be0 0x7f732006e8b0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f731000bb10 tx=0x7f731002c8d0 comp rx=0 tx=0).stop 2024-09-12T03:16:57.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:57.890+0000 7f7327e77640 1 -- 172.21.15.27:0/244563431 shutdown_connections 2024-09-12T03:16:57.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:57.890+0000 7f7327e77640 1 --2- 172.21.15.27:0/244563431 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7320071be0 0x7f732006e8b0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:57.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:57.890+0000 7f7327e77640 1 --2- 172.21.15.27:0/244563431 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f732015a8c0 0x7f73200cdb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:57.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:57.890+0000 7f7327e77640 1 -- 172.21.15.27:0/244563431 >> 172.21.15.27:0/244563431 conn(0x7f7320062c60 msgr2=0x7f732006bb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:57.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:57.891+0000 7f7327e77640 1 -- 172.21.15.27:0/244563431 shutdown_connections 2024-09-12T03:16:57.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:57.891+0000 7f7327e77640 1 -- 172.21.15.27:0/244563431 wait complete. 2024-09-12T03:16:57.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:57.892+0000 7f7327e77640 1 Processor -- start 2024-09-12T03:16:57.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:57.892+0000 7f7327e77640 1 -- start start 2024-09-12T03:16:57.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:57.892+0000 7f7327e77640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7320071be0 0x7f7320148920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:57.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:57.892+0000 7f7327e77640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f732015a8c0 0x7f732015ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:57.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:57.892+0000 7f7327e77640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f732015aca0 0x7f732015d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:57.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:57.892+0000 7f7327e77640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f73200b36e0 con 0x7f732015a8c0 2024-09-12T03:16:57.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:57.892+0000 7f7327e77640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f73200b3560 con 0x7f7320071be0 2024-09-12T03:16:57.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:57.892+0000 7f7327e77640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f73200b4e20 con 0x7f732015aca0 2024-09-12T03:16:57.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:57.893+0000 7f7325412640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f732015a8c0 0x7f732015ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:57.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:57.893+0000 7f7325412640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f732015a8c0 0x7f732015ce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59196/0 (socket says 172.21.15.27:59196) 2024-09-12T03:16:57.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:57.893+0000 7f7325412640 1 -- 172.21.15.27:0/1120609714 learned_addr learned my addr 172.21.15.27:0/1120609714 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:16:57.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:57.893+0000 7f7325c13640 1 --2- 172.21.15.27:0/1120609714 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7320071be0 0x7f7320148920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:57.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:57.893+0000 7f7326414640 1 --2- 172.21.15.27:0/1120609714 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f732015aca0 0x7f732015d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:57.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:57.893+0000 7f7325412640 1 -- 172.21.15.27:0/1120609714 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f732015aca0 msgr2=0x7f732015d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:57.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:57.893+0000 7f7325412640 1 --2- 172.21.15.27:0/1120609714 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f732015aca0 0x7f732015d3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:57.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:57.893+0000 7f7325412640 1 -- 172.21.15.27:0/1120609714 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7320071be0 msgr2=0x7f7320148920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:57.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:57.893+0000 7f7325412640 1 --2- 172.21.15.27:0/1120609714 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7320071be0 0x7f7320148920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:57.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:57.893+0000 7f7325412640 1 -- 172.21.15.27:0/1120609714 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73201efec0 con 0x7f732015a8c0 2024-09-12T03:16:57.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:57.893+0000 7f7325412640 1 --2- 172.21.15.27:0/1120609714 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f732015a8c0 0x7f732015ce90 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f731000b2c0 tx=0x7f73100046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:57.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:57.893+0000 7f73167fc640 1 -- 172.21.15.27:0/1120609714 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7310033620 con 0x7f732015a8c0 2024-09-12T03:16:57.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:57.893+0000 7f73167fc640 1 -- 172.21.15.27:0/1120609714 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73100337c0 con 0x7f732015a8c0 2024-09-12T03:16:57.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:57.893+0000 7f73167fc640 1 -- 172.21.15.27:0/1120609714 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7310033bb0 con 0x7f732015a8c0 2024-09-12T03:16:57.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:57.893+0000 7f7327e77640 1 -- 172.21.15.27:0/1120609714 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73201f01a0 con 0x7f732015a8c0 2024-09-12T03:16:57.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:57.893+0000 7f7327e77640 1 -- 172.21.15.27:0/1120609714 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73201f0760 con 0x7f732015a8c0 2024-09-12T03:16:57.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:57.894+0000 7f7327e77640 1 -- 172.21.15.27:0/1120609714 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7320072ba0 con 0x7f732015a8c0 2024-09-12T03:16:57.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:57.895+0000 7f73167fc640 1 -- 172.21.15.27:0/1120609714 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f731004d060 con 0x7f732015a8c0 2024-09-12T03:16:57.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:57.895+0000 7f73167fc640 1 --2- 172.21.15.27:0/1120609714 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f730406be30 0x7f730406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:57.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:57.895+0000 7f73167fc640 1 -- 172.21.15.27:0/1120609714 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f73100adcb0 con 0x7f732015a8c0 2024-09-12T03:16:57.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:57.895+0000 7f7325c13640 1 --2- 172.21.15.27:0/1120609714 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f730406be30 0x7f730406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:57.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:57.896+0000 7f7325c13640 1 --2- 172.21.15.27:0/1120609714 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f730406be30 0x7f730406e2f0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f73200b1f60 tx=0x7f731c008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:57.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:57.900+0000 7f73167fc640 1 -- 172.21.15.27:0/1120609714 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f7310041070 con 0x7f732015a8c0 2024-09-12T03:16:58.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:58.087+0000 7f7327e77640 1 -- 172.21.15.27:0/1120609714 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f73200b1fb0 con 0x7f732015a8c0 2024-09-12T03:16:58.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:58.087+0000 7f73167fc640 1 -- 172.21.15.27:0/1120609714 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v21) ==== 80+0+0 (secure 0 0 0) 0x7f731007c1c0 con 0x7f732015a8c0 2024-09-12T03:16:58.089 INFO:teuthology.orchestra.run.smithi027.stderr:listed 0 entries 2024-09-12T03:16:58.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:58.090+0000 7f7327e77640 1 -- 172.21.15.27:0/1120609714 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f730406be30 msgr2=0x7f730406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:58.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:58.090+0000 7f7327e77640 1 --2- 172.21.15.27:0/1120609714 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f730406be30 0x7f730406e2f0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f73200b1f60 tx=0x7f731c008040 comp rx=0 tx=0).stop 2024-09-12T03:16:58.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:58.090+0000 7f7327e77640 1 -- 172.21.15.27:0/1120609714 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f732015a8c0 msgr2=0x7f732015ce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:58.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:58.090+0000 7f7327e77640 1 --2- 172.21.15.27:0/1120609714 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f732015a8c0 0x7f732015ce90 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f731000b2c0 tx=0x7f73100046c0 comp rx=0 tx=0).stop 2024-09-12T03:16:58.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:58.090+0000 7f7327e77640 1 -- 172.21.15.27:0/1120609714 shutdown_connections 2024-09-12T03:16:58.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:58.090+0000 7f7327e77640 1 --2- 172.21.15.27:0/1120609714 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f730406be30 0x7f730406e2f0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:58.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:58.090+0000 7f7327e77640 1 --2- 172.21.15.27:0/1120609714 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f732015aca0 0x7f732015d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:58.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:58.090+0000 7f7327e77640 1 --2- 172.21.15.27:0/1120609714 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f732015a8c0 0x7f732015ce90 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:58.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:58.090+0000 7f7327e77640 1 --2- 172.21.15.27:0/1120609714 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7320071be0 0x7f7320148920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:58.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:58.090+0000 7f7327e77640 1 -- 172.21.15.27:0/1120609714 >> 172.21.15.27:0/1120609714 conn(0x7f7320062c60 msgr2=0x7f732015bd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:58.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:58.090+0000 7f7327e77640 1 -- 172.21.15.27:0/1120609714 shutdown_connections 2024-09-12T03:16:58.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:58.090+0000 7f7327e77640 1 -- 172.21.15.27:0/1120609714 wait complete. 2024-09-12T03:16:58.101 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-09-12T03:16:58.101 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:16:58.102 DEBUG:teuthology.orchestra.run.smithi045:> sudo umount /home/ubuntu/cephtest/mnt.1 -f 2024-09-12T03:16:58.135 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi045.front.sepia.ceph.com 2024-09-12T03:16:58.135 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:16:58.136 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-12T03:16:58.209 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-12T03:16:58.209 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T03:16:58.227 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-12T03:16:58.227 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T03:16:58.283 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-12T03:16:58.283 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T03:16:58.339 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-12T03:16:58.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:58.482+0000 7f57e24b7640 1 Processor -- start 2024-09-12T03:16:58.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:58.482+0000 7f57e24b7640 1 -- start start 2024-09-12T03:16:58.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:58.483+0000 7f57e24b7640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f57dc15e780 0x7f57dc0d3e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:58.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:58.483+0000 7f57e24b7640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f57dc0d43b0 0x7f57dc0d1280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:58.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:58.483+0000 7f57e24b7640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f57dc15eb60 0x7f57dc0d17c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:58.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:58.483+0000 7f57e24b7640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f57dc0b7950 con 0x7f57dc15e780 2024-09-12T03:16:58.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:58.483+0000 7f57e24b7640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f57dc0b7210 con 0x7f57dc15eb60 2024-09-12T03:16:58.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:58.483+0000 7f57e24b7640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f57dc0b6690 con 0x7f57dc0d43b0 2024-09-12T03:16:58.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:58.483+0000 7f57dbfff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f57dc15e780 0x7f57dc0d3e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:58.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:58.483+0000 7f57dbfff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f57dc15e780 0x7f57dc0d3e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59202/0 (socket says 172.21.15.27:59202) 2024-09-12T03:16:58.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:58.483+0000 7f57dbfff640 1 -- 172.21.15.27:0/2889947336 learned_addr learned my addr 172.21.15.27:0/2889947336 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:16:58.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:58.483+0000 7f57e0a54640 1 --2- 172.21.15.27:0/2889947336 >> v2:172.21.15.45:3300/0 conn(0x7f57dc15eb60 0x7f57dc0d17c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:58.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:58.483+0000 7f57db7fe640 1 --2- 172.21.15.27:0/2889947336 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f57dc0d43b0 0x7f57dc0d1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:58.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:58.483+0000 7f57dbfff640 1 -- 172.21.15.27:0/2889947336 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f57dc0d43b0 msgr2=0x7f57dc0d1280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:58.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:58.483+0000 7f57dbfff640 1 --2- 172.21.15.27:0/2889947336 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f57dc0d43b0 0x7f57dc0d1280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:58.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:58.483+0000 7f57dbfff640 1 -- 172.21.15.27:0/2889947336 >> v2:172.21.15.45:3300/0 conn(0x7f57dc15eb60 msgr2=0x7f57dc0d17c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:58.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:58.483+0000 7f57dbfff640 1 --2- 172.21.15.27:0/2889947336 >> v2:172.21.15.45:3300/0 conn(0x7f57dc15eb60 0x7f57dc0d17c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:58.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:58.483+0000 7f57dbfff640 1 -- 172.21.15.27:0/2889947336 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57dc0b6970 con 0x7f57dc15e780 2024-09-12T03:16:58.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:58.483+0000 7f57e0a54640 1 --2- 172.21.15.27:0/2889947336 >> v2:172.21.15.45:3300/0 conn(0x7f57dc15eb60 0x7f57dc0d17c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:16:58.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:58.483+0000 7f57db7fe640 1 --2- 172.21.15.27:0/2889947336 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f57dc0d43b0 0x7f57dc0d1280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:16:58.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:58.484+0000 7f57dbfff640 1 --2- 172.21.15.27:0/2889947336 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f57dc15e780 0x7f57dc0d3e70 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f57dc069cd0 tx=0x7f57cc02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1b7b18fbef97ba5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:58.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:58.484+0000 7f57daffd640 1 -- 172.21.15.27:0/2889947336 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57cc037070 con 0x7f57dc15e780 2024-09-12T03:16:58.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:58.484+0000 7f57daffd640 1 -- 172.21.15.27:0/2889947336 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57cc032440 con 0x7f57dc15e780 2024-09-12T03:16:58.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:58.484+0000 7f57daffd640 1 -- 172.21.15.27:0/2889947336 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57cc032810 con 0x7f57dc15e780 2024-09-12T03:16:58.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:58.484+0000 7f57e24b7640 1 -- 172.21.15.27:0/2889947336 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f57dc15e780 msgr2=0x7f57dc0d3e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:58.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:58.485+0000 7f57e24b7640 1 --2- 172.21.15.27:0/2889947336 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f57dc15e780 0x7f57dc0d3e70 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f57dc069cd0 tx=0x7f57cc02a1e0 comp rx=0 tx=0).stop 2024-09-12T03:16:58.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:58.485+0000 7f57e24b7640 1 -- 172.21.15.27:0/2889947336 shutdown_connections 2024-09-12T03:16:58.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:58.485+0000 7f57e24b7640 1 --2- 172.21.15.27:0/2889947336 >> v2:172.21.15.45:3300/0 conn(0x7f57dc15eb60 0x7f57dc0d17c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:58.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:58.485+0000 7f57e24b7640 1 --2- 172.21.15.27:0/2889947336 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f57dc0d43b0 0x7f57dc0d1280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:58.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:58.485+0000 7f57e24b7640 1 --2- 172.21.15.27:0/2889947336 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f57dc15e780 0x7f57dc0d3e70 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:58.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:58.485+0000 7f57e24b7640 1 -- 172.21.15.27:0/2889947336 >> 172.21.15.27:0/2889947336 conn(0x7f57dc06da60 msgr2=0x7f57dc06de70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:58.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:58.485+0000 7f57e24b7640 1 -- 172.21.15.27:0/2889947336 shutdown_connections 2024-09-12T03:16:58.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:58.485+0000 7f57e24b7640 1 -- 172.21.15.27:0/2889947336 wait complete. 2024-09-12T03:16:58.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:58.486+0000 7f57e24b7640 1 Processor -- start 2024-09-12T03:16:58.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:58.486+0000 7f57e24b7640 1 -- start start 2024-09-12T03:16:58.490 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:58.487+0000 7f57e24b7640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f57dc0d43b0 0x7f57dc1f4510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:58.490 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:58.487+0000 7f57e24b7640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f57dc15e780 0x7f57dc0ca930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:58.490 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:58.487+0000 7f57e24b7640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f57dc15eb60 0x7f57dc1484b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:58.490 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:58.487+0000 7f57e24b7640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f57dc0b6340 con 0x7f57dc15e780 2024-09-12T03:16:58.490 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:58.487+0000 7f57e24b7640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f57dc0b6f30 con 0x7f57dc0d43b0 2024-09-12T03:16:58.490 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:58.487+0000 7f57e24b7640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f57dc0b3390 con 0x7f57dc15eb60 2024-09-12T03:16:58.490 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:58.487+0000 7f57db7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f57dc15e780 0x7f57dc0ca930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:58.490 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:58.487+0000 7f57db7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f57dc15e780 0x7f57dc0ca930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59208/0 (socket says 172.21.15.27:59208) 2024-09-12T03:16:58.490 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:58.487+0000 7f57db7fe640 1 -- 172.21.15.27:0/3459334424 learned_addr learned my addr 172.21.15.27:0/3459334424 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:16:58.490 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:58.487+0000 7f57dbfff640 1 --2- 172.21.15.27:0/3459334424 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f57dc0d43b0 0x7f57dc1f4510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:58.490 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:58.487+0000 7f57e0a54640 1 --2- 172.21.15.27:0/3459334424 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f57dc15eb60 0x7f57dc1484b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:58.490 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:58.487+0000 7f57db7fe640 1 -- 172.21.15.27:0/3459334424 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f57dc15eb60 msgr2=0x7f57dc1484b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:58.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:58.487+0000 7f57db7fe640 1 --2- 172.21.15.27:0/3459334424 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f57dc15eb60 0x7f57dc1484b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:58.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:58.487+0000 7f57db7fe640 1 -- 172.21.15.27:0/3459334424 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f57dc0d43b0 msgr2=0x7f57dc1f4510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:58.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:58.487+0000 7f57db7fe640 1 --2- 172.21.15.27:0/3459334424 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f57dc0d43b0 0x7f57dc1f4510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:58.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:58.487+0000 7f57db7fe640 1 -- 172.21.15.27:0/3459334424 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57dc155030 con 0x7f57dc15e780 2024-09-12T03:16:58.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:58.488+0000 7f57db7fe640 1 --2- 172.21.15.27:0/3459334424 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f57dc15e780 0x7f57dc0ca930 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f57d000e810 tx=0x7f57d000ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:58.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:58.488+0000 7f57d8ff9640 1 -- 172.21.15.27:0/3459334424 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57d000f040 con 0x7f57dc15e780 2024-09-12T03:16:58.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:58.488+0000 7f57d8ff9640 1 -- 172.21.15.27:0/3459334424 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57d0004030 con 0x7f57dc15e780 2024-09-12T03:16:58.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:58.488+0000 7f57e24b7640 1 -- 172.21.15.27:0/3459334424 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57dc155310 con 0x7f57dc15e780 2024-09-12T03:16:58.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:58.488+0000 7f57d8ff9640 1 -- 172.21.15.27:0/3459334424 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57d0004400 con 0x7f57dc15e780 2024-09-12T03:16:58.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:58.488+0000 7f57e24b7640 1 -- 172.21.15.27:0/3459334424 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57dc1558d0 con 0x7f57dc15e780 2024-09-12T03:16:58.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:58.489+0000 7f57e24b7640 1 -- 172.21.15.27:0/3459334424 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57dc0d1a70 con 0x7f57dc15e780 2024-09-12T03:16:58.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:58.489+0000 7f57d8ff9640 1 -- 172.21.15.27:0/3459334424 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f57d000c6d0 con 0x7f57dc15e780 2024-09-12T03:16:58.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:58.490+0000 7f57d8ff9640 1 --2- 172.21.15.27:0/3459334424 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f57c806be30 0x7f57c806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:58.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:58.490+0000 7f57d8ff9640 1 -- 172.21.15.27:0/3459334424 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f57d008aa90 con 0x7f57dc15e780 2024-09-12T03:16:58.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:58.490+0000 7f57dbfff640 1 --2- 172.21.15.27:0/3459334424 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f57c806be30 0x7f57c806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:58.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:58.490+0000 7f57dbfff640 1 --2- 172.21.15.27:0/3459334424 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f57c806be30 0x7f57c806e2f0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f57cc034000 tx=0x7f57cc02e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:58.494 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:58.494+0000 7f57d8ff9640 1 -- 172.21.15.27:0/3459334424 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f57dc0d1a70 con 0x7f57dc15e780 2024-09-12T03:16:58.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:58.683+0000 7f57e24b7640 1 -- 172.21.15.27:0/3459334424 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f57dc0d3e70 con 0x7f57dc15e780 2024-09-12T03:16:58.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:58.683+0000 7f57d8ff9640 1 -- 172.21.15.27:0/3459334424 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v21) ==== 80+0+0 (secure 0 0 0) 0x7f57d0017020 con 0x7f57dc15e780 2024-09-12T03:16:58.683 INFO:teuthology.orchestra.run.smithi027.stderr:listed 0 entries 2024-09-12T03:16:58.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:58.685+0000 7f57e24b7640 1 -- 172.21.15.27:0/3459334424 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f57c806be30 msgr2=0x7f57c806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:58.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:58.685+0000 7f57e24b7640 1 --2- 172.21.15.27:0/3459334424 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f57c806be30 0x7f57c806e2f0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f57cc034000 tx=0x7f57cc02e000 comp rx=0 tx=0).stop 2024-09-12T03:16:58.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:58.685+0000 7f57e24b7640 1 -- 172.21.15.27:0/3459334424 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f57dc15e780 msgr2=0x7f57dc0ca930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:58.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:58.685+0000 7f57e24b7640 1 --2- 172.21.15.27:0/3459334424 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f57dc15e780 0x7f57dc0ca930 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f57d000e810 tx=0x7f57d000ecd0 comp rx=0 tx=0).stop 2024-09-12T03:16:58.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:58.685+0000 7f57e24b7640 1 -- 172.21.15.27:0/3459334424 shutdown_connections 2024-09-12T03:16:58.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:58.685+0000 7f57e24b7640 1 --2- 172.21.15.27:0/3459334424 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f57c806be30 0x7f57c806e2f0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:58.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:58.685+0000 7f57e24b7640 1 --2- 172.21.15.27:0/3459334424 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f57dc15eb60 0x7f57dc1484b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:58.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:58.685+0000 7f57e24b7640 1 --2- 172.21.15.27:0/3459334424 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f57dc15e780 0x7f57dc0ca930 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:58.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:58.685+0000 7f57e24b7640 1 --2- 172.21.15.27:0/3459334424 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f57dc0d43b0 0x7f57dc1f4510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:58.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:58.685+0000 7f57e24b7640 1 -- 172.21.15.27:0/3459334424 >> 172.21.15.27:0/3459334424 conn(0x7f57dc06da60 msgr2=0x7f57dc0b7d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:58.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:58.686+0000 7f57e24b7640 1 -- 172.21.15.27:0/3459334424 shutdown_connections 2024-09-12T03:16:58.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:58.686+0000 7f57e24b7640 1 -- 172.21.15.27:0/3459334424 wait complete. 2024-09-12T03:16:58.697 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.2... 2024-09-12T03:16:58.697 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:16:58.697 DEBUG:teuthology.orchestra.run.smithi027:> sudo umount /home/ubuntu/cephtest/mnt.2 -f 2024-09-12T03:16:58.724 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi027.front.sepia.ceph.com 2024-09-12T03:16:58.724 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:16:58.724 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-09-12T03:16:58.781 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-12T03:16:58.781 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T03:16:58.801 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-12T03:16:58.801 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T03:16:58.861 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-12T03:16:58.862 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T03:16:58.921 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-12T03:16:59.026 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.026+0000 7fa00eb2a640 1 Processor -- start 2024-09-12T03:16:59.026 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.026+0000 7fa00eb2a640 1 -- start start 2024-09-12T03:16:59.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.027+0000 7fa00eb2a640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa00815e780 0x7fa0080d3e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:59.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.027+0000 7fa00eb2a640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa0080d43b0 0x7fa0080d1280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:59.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.027+0000 7fa00eb2a640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fa00815eb60 0x7fa0080d17c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:59.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.027+0000 7fa00eb2a640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7fa0080b7950 con 0x7fa00815eb60 2024-09-12T03:16:59.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.027+0000 7fa00eb2a640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fa0080b7210 con 0x7fa00815e780 2024-09-12T03:16:59.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.027+0000 7fa00eb2a640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fa0080b6690 con 0x7fa0080d43b0 2024-09-12T03:16:59.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.027+0000 7fa00d0c7640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fa00815eb60 0x7fa0080d17c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:59.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.027+0000 7fa00d0c7640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fa00815eb60 0x7fa0080d17c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59218/0 (socket says 172.21.15.27:59218) 2024-09-12T03:16:59.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.027+0000 7fa00d0c7640 1 -- 172.21.15.27:0/3480369092 learned_addr learned my addr 172.21.15.27:0/3480369092 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:16:59.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.027+0000 7fa00c8c6640 1 --2- 172.21.15.27:0/3480369092 >> v2:172.21.15.45:3300/0 conn(0x7fa00815e780 0x7fa0080d3e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:59.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.027+0000 7fa00d0c7640 1 -- 172.21.15.27:0/3480369092 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa0080d43b0 msgr2=0x7fa0080d1280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:59.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.027+0000 7fa007fff640 1 --2- 172.21.15.27:0/3480369092 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa0080d43b0 0x7fa0080d1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:59.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.027+0000 7fa00d0c7640 1 --2- 172.21.15.27:0/3480369092 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa0080d43b0 0x7fa0080d1280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:59.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.027+0000 7fa00d0c7640 1 -- 172.21.15.27:0/3480369092 >> v2:172.21.15.45:3300/0 conn(0x7fa00815e780 msgr2=0x7fa0080d3e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:59.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.027+0000 7fa00d0c7640 1 --2- 172.21.15.27:0/3480369092 >> v2:172.21.15.45:3300/0 conn(0x7fa00815e780 0x7fa0080d3e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:59.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.027+0000 7fa00d0c7640 1 -- 172.21.15.27:0/3480369092 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0080b6970 con 0x7fa00815eb60 2024-09-12T03:16:59.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.027+0000 7fa00c8c6640 1 --2- 172.21.15.27:0/3480369092 >> v2:172.21.15.45:3300/0 conn(0x7fa00815e780 0x7fa0080d3e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:16:59.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.028+0000 7fa00d0c7640 1 --2- 172.21.15.27:0/3480369092 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa00815eb60 0x7fa0080d17c0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fa00806eae0 tx=0x7f9ff002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=452445ad8c257be server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:59.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.028+0000 7fa0077fe640 1 -- 172.21.15.27:0/3480369092 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ff0037070 con 0x7fa00815eb60 2024-09-12T03:16:59.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.028+0000 7fa0077fe640 1 -- 172.21.15.27:0/3480369092 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ff0032440 con 0x7fa00815eb60 2024-09-12T03:16:59.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.028+0000 7fa0077fe640 1 -- 172.21.15.27:0/3480369092 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ff0032810 con 0x7fa00815eb60 2024-09-12T03:16:59.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.028+0000 7fa00eb2a640 1 -- 172.21.15.27:0/3480369092 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa00815eb60 msgr2=0x7fa0080d17c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:59.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.028+0000 7fa00eb2a640 1 --2- 172.21.15.27:0/3480369092 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa00815eb60 0x7fa0080d17c0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fa00806eae0 tx=0x7f9ff002a1e0 comp rx=0 tx=0).stop 2024-09-12T03:16:59.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.029+0000 7fa00eb2a640 1 -- 172.21.15.27:0/3480369092 shutdown_connections 2024-09-12T03:16:59.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.029+0000 7fa00eb2a640 1 --2- 172.21.15.27:0/3480369092 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa00815eb60 0x7fa0080d17c0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:59.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.029+0000 7fa00eb2a640 1 --2- 172.21.15.27:0/3480369092 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa0080d43b0 0x7fa0080d1280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:59.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.029+0000 7fa00eb2a640 1 --2- 172.21.15.27:0/3480369092 >> v2:172.21.15.45:3300/0 conn(0x7fa00815e780 0x7fa0080d3e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:59.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.029+0000 7fa00eb2a640 1 -- 172.21.15.27:0/3480369092 >> 172.21.15.27:0/3480369092 conn(0x7fa00806da60 msgr2=0x7fa00806de70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:59.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.029+0000 7fa00eb2a640 1 -- 172.21.15.27:0/3480369092 shutdown_connections 2024-09-12T03:16:59.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.029+0000 7fa00eb2a640 1 -- 172.21.15.27:0/3480369092 wait complete. 2024-09-12T03:16:59.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.030+0000 7fa00eb2a640 1 Processor -- start 2024-09-12T03:16:59.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.030+0000 7fa00eb2a640 1 -- start start 2024-09-12T03:16:59.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.031+0000 7fa00eb2a640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa0080d43b0 0x7fa00814aca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:59.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.031+0000 7fa00eb2a640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa00815e780 0x7fa008161aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:59.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.031+0000 7fa00eb2a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa00815eb60 0x7fa008156cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:59.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.031+0000 7fa00eb2a640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fa0080b6340 con 0x7fa0080d43b0 2024-09-12T03:16:59.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.031+0000 7fa00eb2a640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa0080b6f30 con 0x7fa00815eb60 2024-09-12T03:16:59.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.031+0000 7fa00eb2a640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fa0080b3390 con 0x7fa00815e780 2024-09-12T03:16:59.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.031+0000 7fa00c8c6640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa0080d43b0 0x7fa00814aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:59.034 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.031+0000 7fa00c8c6640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa0080d43b0 0x7fa00814aca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59220/0 (socket says 172.21.15.27:59220) 2024-09-12T03:16:59.034 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.031+0000 7fa00c8c6640 1 -- 172.21.15.27:0/1267378622 learned_addr learned my addr 172.21.15.27:0/1267378622 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:16:59.034 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.031+0000 7fa00d0c7640 1 --2- 172.21.15.27:0/1267378622 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa00815eb60 0x7fa008156cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:59.034 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.031+0000 7fa007fff640 1 --2- 172.21.15.27:0/1267378622 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa00815e780 0x7fa008161aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:59.034 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.031+0000 7fa00c8c6640 1 -- 172.21.15.27:0/1267378622 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa00815e780 msgr2=0x7fa008161aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:59.034 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.031+0000 7fa00c8c6640 1 --2- 172.21.15.27:0/1267378622 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa00815e780 0x7fa008161aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:59.034 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.031+0000 7fa00c8c6640 1 -- 172.21.15.27:0/1267378622 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa00815eb60 msgr2=0x7fa008156cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:59.034 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.031+0000 7fa00c8c6640 1 --2- 172.21.15.27:0/1267378622 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa00815eb60 0x7fa008156cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:59.034 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.031+0000 7fa00c8c6640 1 -- 172.21.15.27:0/1267378622 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa00814b1e0 con 0x7fa0080d43b0 2024-09-12T03:16:59.034 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.031+0000 7fa007fff640 1 --2- 172.21.15.27:0/1267378622 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa00815e780 0x7fa008161aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:16:59.034 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.031+0000 7fa00c8c6640 1 --2- 172.21.15.27:0/1267378622 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa0080d43b0 0x7fa00814aca0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fa00000cf60 tx=0x7fa00000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:59.034 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.031+0000 7fa0057fa640 1 -- 172.21.15.27:0/1267378622 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa00001a070 con 0x7fa0080d43b0 2024-09-12T03:16:59.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.031+0000 7fa0057fa640 1 -- 172.21.15.27:0/1267378622 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa00000d840 con 0x7fa0080d43b0 2024-09-12T03:16:59.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.031+0000 7fa00eb2a640 1 -- 172.21.15.27:0/1267378622 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa008156150 con 0x7fa0080d43b0 2024-09-12T03:16:59.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.031+0000 7fa0057fa640 1 -- 172.21.15.27:0/1267378622 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa00000dc10 con 0x7fa0080d43b0 2024-09-12T03:16:59.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.032+0000 7fa00eb2a640 1 -- 172.21.15.27:0/1267378622 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa00815f3a0 con 0x7fa0080d43b0 2024-09-12T03:16:59.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.032+0000 7fa00eb2a640 1 -- 172.21.15.27:0/1267378622 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0080b5d20 con 0x7fa0080d43b0 2024-09-12T03:16:59.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.033+0000 7fa0057fa640 1 -- 172.21.15.27:0/1267378622 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa00001d070 con 0x7fa0080d43b0 2024-09-12T03:16:59.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.033+0000 7fa0057fa640 1 --2- 172.21.15.27:0/1267378622 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f9fe806be30 0x7f9fe806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:59.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.033+0000 7fa0057fa640 1 -- 172.21.15.27:0/1267378622 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fa00008b5b0 con 0x7fa0080d43b0 2024-09-12T03:16:59.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.034+0000 7fa007fff640 1 --2- 172.21.15.27:0/1267378622 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f9fe806be30 0x7f9fe806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:59.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.034+0000 7fa007fff640 1 --2- 172.21.15.27:0/1267378622 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f9fe806be30 0x7f9fe806e2f0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fa00815d950 tx=0x7f9ff8008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:59.037 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.038+0000 7fa0057fa640 1 -- 172.21.15.27:0/1267378622 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fa0000599a0 con 0x7fa0080d43b0 2024-09-12T03:16:59.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.226+0000 7fa00eb2a640 1 -- 172.21.15.27:0/1267378622 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fa0080d3e20 con 0x7fa0080d43b0 2024-09-12T03:16:59.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.226+0000 7fa0057fa640 1 -- 172.21.15.27:0/1267378622 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v21) ==== 80+0+0 (secure 0 0 0) 0x7fa00005dca0 con 0x7fa0080d43b0 2024-09-12T03:16:59.226 INFO:teuthology.orchestra.run.smithi027.stderr:listed 0 entries 2024-09-12T03:16:59.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.229+0000 7fa00eb2a640 1 -- 172.21.15.27:0/1267378622 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f9fe806be30 msgr2=0x7f9fe806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:59.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.229+0000 7fa00eb2a640 1 --2- 172.21.15.27:0/1267378622 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f9fe806be30 0x7f9fe806e2f0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fa00815d950 tx=0x7f9ff8008040 comp rx=0 tx=0).stop 2024-09-12T03:16:59.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.229+0000 7fa00eb2a640 1 -- 172.21.15.27:0/1267378622 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa0080d43b0 msgr2=0x7fa00814aca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:59.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.229+0000 7fa00eb2a640 1 --2- 172.21.15.27:0/1267378622 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa0080d43b0 0x7fa00814aca0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fa00000cf60 tx=0x7fa00000cf90 comp rx=0 tx=0).stop 2024-09-12T03:16:59.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.229+0000 7fa00eb2a640 1 -- 172.21.15.27:0/1267378622 shutdown_connections 2024-09-12T03:16:59.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.229+0000 7fa00eb2a640 1 --2- 172.21.15.27:0/1267378622 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f9fe806be30 0x7f9fe806e2f0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:59.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.229+0000 7fa00eb2a640 1 --2- 172.21.15.27:0/1267378622 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa00815eb60 0x7fa008156cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:59.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.229+0000 7fa00eb2a640 1 --2- 172.21.15.27:0/1267378622 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa00815e780 0x7fa008161aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:59.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.229+0000 7fa00eb2a640 1 --2- 172.21.15.27:0/1267378622 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa0080d43b0 0x7fa00814aca0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:59.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.229+0000 7fa00eb2a640 1 -- 172.21.15.27:0/1267378622 >> 172.21.15.27:0/1267378622 conn(0x7fa00806da60 msgr2=0x7fa0080b3cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:59.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.229+0000 7fa00eb2a640 1 -- 172.21.15.27:0/1267378622 shutdown_connections 2024-09-12T03:16:59.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.229+0000 7fa00eb2a640 1 -- 172.21.15.27:0/1267378622 wait complete. 2024-09-12T03:16:59.240 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.3... 2024-09-12T03:16:59.240 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:16:59.240 DEBUG:teuthology.orchestra.run.smithi045:> sudo umount /home/ubuntu/cephtest/mnt.3 -f 2024-09-12T03:16:59.273 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi045.front.sepia.ceph.com 2024-09-12T03:16:59.273 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:16:59.273 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-09-12T03:16:59.333 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-12T03:16:59.333 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-12T03:16:59.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.435+0000 7f8b76949640 1 Processor -- start 2024-09-12T03:16:59.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.435+0000 7f8b76949640 1 -- start start 2024-09-12T03:16:59.438 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.436+0000 7f8b76949640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f8b7006f360 0x7f8b700cef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:59.438 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.436+0000 7f8b76949640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8b7006cc40 con 0x7f8b700cf490 2024-09-12T03:16:59.438 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.436+0000 7f8b76949640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8b700d1ce0 con 0x7f8b700b1ca0 2024-09-12T03:16:59.438 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.436+0000 7f8b76949640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f8b700cf900 con 0x7f8b7006f360 2024-09-12T03:16:59.439 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.436+0000 7f8b74ee6640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f8b700cf490 0x7f8b700d2780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:44280/0 (socket says 172.21.15.27:44280) 2024-09-12T03:16:59.439 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.436+0000 7f8b74ee6640 1 -- 172.21.15.27:0/315487009 learned_addr learned my addr 172.21.15.27:0/315487009 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:16:59.439 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.437+0000 7f8b6effd640 1 -- 172.21.15.27:0/315487009 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2303623245 0 0) 0x7f8b7006cc40 con 0x7f8b700cf490 2024-09-12T03:16:59.439 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.437+0000 7f8b6effd640 1 -- 172.21.15.27:0/315487009 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8b58003520 con 0x7f8b700cf490 2024-09-12T03:16:59.439 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.437+0000 7f8b6f7fe640 1 --2- 172.21.15.27:0/315487009 >> v2:172.21.15.45:3300/0 conn(0x7f8b7006f360 0x7f8b700cef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:59.439 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.437+0000 7f8b6effd640 1 -- 172.21.15.27:0/315487009 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3106778911 0 0) 0x7f8b58003520 con 0x7f8b700cf490 2024-09-12T03:16:59.439 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.437+0000 7f8b6effd640 1 -- 172.21.15.27:0/315487009 >> v2:172.21.15.45:3300/0 conn(0x7f8b7006f360 msgr2=0x7f8b700cef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:59.439 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.437+0000 7f8b6effd640 1 --2- 172.21.15.27:0/315487009 >> v2:172.21.15.45:3300/0 conn(0x7f8b7006f360 0x7f8b700cef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:59.439 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.437+0000 7f8b6effd640 1 -- 172.21.15.27:0/315487009 >> v1:172.21.15.45:6789/0 conn(0x7f8b700b1ca0 legacy=0x7f8b7015f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:59.439 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.437+0000 7f8b6effd640 1 -- 172.21.15.27:0/315487009 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b7006cee0 con 0x7f8b700cf490 2024-09-12T03:16:59.439 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.437+0000 7f8b6effd640 1 -- 172.21.15.27:0/315487009 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 847755213 0 0) 0x7f8b680019f0 con 0x7f8b700cf490 2024-09-12T03:16:59.439 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.437+0000 7f8b6effd640 1 -- 172.21.15.27:0/315487009 >> v1:172.21.15.27:6789/0 conn(0x7f8b700cf490 legacy=0x7f8b700d2780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:59.439 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.437+0000 7f8b6effd640 1 --2- 172.21.15.27:0/315487009 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f8b580039b0 0x7f8b58003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:59.440 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.437+0000 7f8b6effd640 1 --2- 172.21.15.27:0/315487009 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8b58004800 0x7f8b58004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:59.440 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.437+0000 7f8b6effd640 1 --2- 172.21.15.27:0/315487009 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8b58005210 0x7f8b5800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:59.440 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.437+0000 7f8b6effd640 1 -- 172.21.15.27:0/315487009 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f8b700cf900 con 0x7f8b58004800 2024-09-12T03:16:59.440 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.437+0000 7f8b6effd640 1 -- 172.21.15.27:0/315487009 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f8b5800c140 con 0x7f8b58005210 2024-09-12T03:16:59.440 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.437+0000 7f8b6effd640 1 -- 172.21.15.27:0/315487009 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f8b5800c340 con 0x7f8b580039b0 2024-09-12T03:16:59.440 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.437+0000 7f8b6f7fe640 1 --2- 172.21.15.27:0/315487009 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8b58004800 0x7f8b58004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:59.440 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.437+0000 7f8b6ffff640 1 --2- 172.21.15.27:0/315487009 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f8b580039b0 0x7f8b58003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:59.440 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.437+0000 7f8b74ee6640 1 --2- 172.21.15.27:0/315487009 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8b58005210 0x7f8b5800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:59.440 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.437+0000 7f8b6f7fe640 1 -- 172.21.15.27:0/315487009 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f8b580039b0 msgr2=0x7f8b58003df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:59.440 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.437+0000 7f8b6f7fe640 1 --2- 172.21.15.27:0/315487009 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f8b580039b0 0x7f8b58003df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:59.440 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.437+0000 7f8b6f7fe640 1 -- 172.21.15.27:0/315487009 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8b58005210 msgr2=0x7f8b5800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:59.440 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.437+0000 7f8b6f7fe640 1 --2- 172.21.15.27:0/315487009 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8b58005210 0x7f8b5800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:59.440 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.437+0000 7f8b6f7fe640 1 -- 172.21.15.27:0/315487009 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b58003520 con 0x7f8b58004800 2024-09-12T03:16:59.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.437+0000 7f8b74ee6640 1 --2- 172.21.15.27:0/315487009 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8b58005210 0x7f8b5800b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:16:59.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.437+0000 7f8b74ee6640 1 -- 172.21.15.27:0/315487009 reap_dead start 2024-09-12T03:16:59.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.438+0000 7f8b6f7fe640 1 --2- 172.21.15.27:0/315487009 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8b58004800 0x7f8b58004c80 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f8b6400ba30 tx=0x7f8b58005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9df4b1c85fb76b06 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:59.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.438+0000 7f8b6effd640 1 -- 172.21.15.27:0/315487009 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b64030070 con 0x7f8b58004800 2024-09-12T03:16:59.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.438+0000 7f8b6effd640 1 -- 172.21.15.27:0/315487009 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b6402ade0 con 0x7f8b58004800 2024-09-12T03:16:59.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.438+0000 7f8b6effd640 1 -- 172.21.15.27:0/315487009 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b6400b600 con 0x7f8b58004800 2024-09-12T03:16:59.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.439+0000 7f8b76949640 1 -- 172.21.15.27:0/315487009 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8b58004800 msgr2=0x7f8b58004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:59.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.439+0000 7f8b76949640 1 --2- 172.21.15.27:0/315487009 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8b58004800 0x7f8b58004c80 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f8b6400ba30 tx=0x7f8b58005830 comp rx=0 tx=0).stop 2024-09-12T03:16:59.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.439+0000 7f8b76949640 1 -- 172.21.15.27:0/315487009 shutdown_connections 2024-09-12T03:16:59.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.439+0000 7f8b76949640 1 --2- 172.21.15.27:0/315487009 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8b58004800 0x7f8b58004c80 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:59.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.439+0000 7f8b76949640 1 -- 172.21.15.27:0/315487009 >> 172.21.15.27:0/315487009 conn(0x7f8b70073800 msgr2=0x7f8b7009ba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:59.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.439+0000 7f8b76949640 1 -- 172.21.15.27:0/315487009 shutdown_connections 2024-09-12T03:16:59.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.439+0000 7f8b76949640 1 -- 172.21.15.27:0/315487009 wait complete. 2024-09-12T03:16:59.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.440+0000 7f8b76949640 1 Processor -- start 2024-09-12T03:16:59.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.440+0000 7f8b76949640 1 -- start start 2024-09-12T03:16:59.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.441+0000 7f8b76949640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f8b58004800 0x7f8b7015fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:59.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.441+0000 7f8b76949640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8b7014d720 0x7f8b70150150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:59.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.441+0000 7f8b76949640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8b7014dba0 0x7f8b70150690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:59.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.441+0000 7f8b76949640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f8b7006d1c0 con 0x7f8b7014dba0 2024-09-12T03:16:59.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.441+0000 7f8b76949640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f8b700d22f0 con 0x7f8b7014d720 2024-09-12T03:16:59.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.441+0000 7f8b76949640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f8b7006d340 con 0x7f8b58004800 2024-09-12T03:16:59.446 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.441+0000 7f8b74ee6640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8b7014dba0 0x7f8b70150690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:59.446 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.441+0000 7f8b74ee6640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8b7014dba0 0x7f8b70150690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57962/0 (socket says 172.21.15.27:57962) 2024-09-12T03:16:59.446 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.441+0000 7f8b74ee6640 1 -- 172.21.15.27:0/1658144722 learned_addr learned my addr 172.21.15.27:0/1658144722 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:16:59.446 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.441+0000 7f8b6f7fe640 1 --2- 172.21.15.27:0/1658144722 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8b7014d720 0x7f8b70150150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:59.446 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.441+0000 7f8b6ffff640 1 --2- 172.21.15.27:0/1658144722 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f8b58004800 0x7f8b7015fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:59.446 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.441+0000 7f8b74ee6640 1 -- 172.21.15.27:0/1658144722 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f8b58004800 msgr2=0x7f8b7015fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:59.446 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.441+0000 7f8b74ee6640 1 --2- 172.21.15.27:0/1658144722 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f8b58004800 0x7f8b7015fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:59.446 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.441+0000 7f8b74ee6640 1 -- 172.21.15.27:0/1658144722 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8b7014d720 msgr2=0x7f8b70150150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:59.446 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.441+0000 7f8b74ee6640 1 --2- 172.21.15.27:0/1658144722 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8b7014d720 0x7f8b70150150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:59.446 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.441+0000 7f8b74ee6640 1 -- 172.21.15.27:0/1658144722 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b7014e0c0 con 0x7f8b7014dba0 2024-09-12T03:16:59.446 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.442+0000 7f8b74ee6640 1 --2- 172.21.15.27:0/1658144722 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8b7014dba0 0x7f8b70150690 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f8b700cf870 tx=0x7f8b6800e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:59.446 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.442+0000 7f8b6cff9640 1 -- 172.21.15.27:0/1658144722 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b6800eb90 con 0x7f8b7014dba0 2024-09-12T03:16:59.447 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.442+0000 7f8b6cff9640 1 -- 172.21.15.27:0/1658144722 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b68018070 con 0x7f8b7014dba0 2024-09-12T03:16:59.447 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.442+0000 7f8b76949640 1 -- 172.21.15.27:0/1658144722 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b70160390 con 0x7f8b7014dba0 2024-09-12T03:16:59.447 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.442+0000 7f8b6cff9640 1 -- 172.21.15.27:0/1658144722 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b68013420 con 0x7f8b7014dba0 2024-09-12T03:16:59.447 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.442+0000 7f8b76949640 1 -- 172.21.15.27:0/1658144722 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b70160950 con 0x7f8b7014dba0 2024-09-12T03:16:59.447 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.443+0000 7f8b76949640 1 -- 172.21.15.27:0/1658144722 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b7014e290 con 0x7f8b7014dba0 2024-09-12T03:16:59.447 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.443+0000 7f8b6cff9640 1 -- 172.21.15.27:0/1658144722 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8b680135c0 con 0x7f8b7014dba0 2024-09-12T03:16:59.447 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.444+0000 7f8b6cff9640 1 --2- 172.21.15.27:0/1658144722 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f8b5c06be60 0x7f8b5c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:59.447 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.444+0000 7f8b6cff9640 1 -- 172.21.15.27:0/1658144722 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f8b6808b020 con 0x7f8b7014dba0 2024-09-12T03:16:59.447 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.444+0000 7f8b6ffff640 1 --2- 172.21.15.27:0/1658144722 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f8b5c06be60 0x7f8b5c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:59.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.445+0000 7f8b6ffff640 1 --2- 172.21.15.27:0/1658144722 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f8b5c06be60 0x7f8b5c06e320 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f8b60000c00 tx=0x7f8b60009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:59.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.448+0000 7f8b6cff9640 1 -- 172.21.15.27:0/1658144722 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f8b680599d0 con 0x7f8b7014dba0 2024-09-12T03:16:59.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.661+0000 7f8b76949640 1 -- 172.21.15.27:0/1658144722 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8b700cfc40 con 0x7f8b7014dba0 2024-09-12T03:16:59.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.661+0000 7f8b6cff9640 1 -- 172.21.15.27:0/1658144722 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+5214 (secure 0 0 0) 0x7f8b680592d0 con 0x7f8b7014dba0 2024-09-12T03:16:59.662 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:16:59.662 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":6,"btime":"2024-09-12T03:16:33:576418+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6836/682326693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":682326693},{"type":"v1","addr":"172.21.15.45:6836","nonce":682326693}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4278,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6834/4084728235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":4084728235},{"type":"v1","addr":"172.21.15.27:6834","nonce":4084728235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4281,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/2844240887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":2844240887},{"type":"v1","addr":"172.21.15.27:6835","nonce":2844240887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-12T03:16:30.914517+0000","modified":"2024-09-12T03:16:32.913742+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.45:6837/1035926332","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6835","nonce":1035926332},{"type":"v1","addr":"172.21.15.45:6837","nonce":1035926332}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4285,"qdb_cluster":[4285]},"id":1}]} 2024-09-12T03:16:59.662 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 6 2024-09-12T03:16:59.664 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.663+0000 7f8b76949640 1 -- 172.21.15.27:0/1658144722 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f8b5c06be60 msgr2=0x7f8b5c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:59.664 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.663+0000 7f8b76949640 1 --2- 172.21.15.27:0/1658144722 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f8b5c06be60 0x7f8b5c06e320 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f8b60000c00 tx=0x7f8b60009040 comp rx=0 tx=0).stop 2024-09-12T03:16:59.664 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.663+0000 7f8b76949640 1 -- 172.21.15.27:0/1658144722 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8b7014dba0 msgr2=0x7f8b70150690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:59.664 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.663+0000 7f8b76949640 1 --2- 172.21.15.27:0/1658144722 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8b7014dba0 0x7f8b70150690 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f8b700cf870 tx=0x7f8b6800e530 comp rx=0 tx=0).stop 2024-09-12T03:16:59.664 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.663+0000 7f8b76949640 1 -- 172.21.15.27:0/1658144722 shutdown_connections 2024-09-12T03:16:59.664 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.663+0000 7f8b76949640 1 --2- 172.21.15.27:0/1658144722 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f8b5c06be60 0x7f8b5c06e320 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:59.665 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.663+0000 7f8b76949640 1 --2- 172.21.15.27:0/1658144722 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8b7014dba0 0x7f8b70150690 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:59.665 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.663+0000 7f8b76949640 1 --2- 172.21.15.27:0/1658144722 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8b7014d720 0x7f8b70150150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:59.665 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.663+0000 7f8b76949640 1 --2- 172.21.15.27:0/1658144722 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f8b58004800 0x7f8b7015fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:59.665 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.663+0000 7f8b76949640 1 -- 172.21.15.27:0/1658144722 >> 172.21.15.27:0/1658144722 conn(0x7f8b70073800 msgr2=0x7f8b700b73b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:59.665 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.663+0000 7f8b76949640 1 -- 172.21.15.27:0/1658144722 shutdown_connections 2024-09-12T03:16:59.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.663+0000 7f8b76949640 1 -- 172.21.15.27:0/1658144722 wait complete. 2024-09-12T03:16:59.674 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-12T03:16:59.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.776+0000 7ff8fa8d0640 1 Processor -- start 2024-09-12T03:16:59.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.776+0000 7ff8fa8d0640 1 -- start start 2024-09-12T03:16:59.780 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.777+0000 7ff8fa8d0640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ff8f40b1ca0 0x7ff8f415f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:59.780 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.777+0000 7ff8fa8d0640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ff8f40cf660 0x7ff8f40d2780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:59.780 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.777+0000 7ff8fa8d0640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff8f415fd70 con 0x7ff8f406f360 2024-09-12T03:16:59.780 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.777+0000 7ff8fa8d0640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7ff8f40d22f0 con 0x7ff8f40b1ca0 2024-09-12T03:16:59.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.777+0000 7ff8fa8d0640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7ff8f406cc40 con 0x7ff8f40cf660 2024-09-12T03:16:59.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.777+0000 7ff8f37fe640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7ff8f406f360 0x7ff8f40cef50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:44292/0 (socket says 172.21.15.27:44292) 2024-09-12T03:16:59.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.777+0000 7ff8f37fe640 1 -- 172.21.15.27:0/3882290632 learned_addr learned my addr 172.21.15.27:0/3882290632 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:16:59.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.778+0000 7ff8f2ffd640 1 -- 172.21.15.27:0/3882290632 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3875971717 0 0) 0x7ff8e8001110 con 0x7ff8f406f360 2024-09-12T03:16:59.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.778+0000 7ff8f2ffd640 1 -- 172.21.15.27:0/3882290632 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff8dc003520 con 0x7ff8f406f360 2024-09-12T03:16:59.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.778+0000 7ff8f8e6d640 1 --2- 172.21.15.27:0/3882290632 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ff8f40cf660 0x7ff8f40d2780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:59.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.778+0000 7ff8f3fff640 1 --2- 172.21.15.27:0/3882290632 >> v2:172.21.15.45:3300/0 conn(0x7ff8f40b1ca0 0x7ff8f415f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:59.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.778+0000 7ff8f2ffd640 1 -- 172.21.15.27:0/3882290632 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 161047473 0 0) 0x7ff8dc003520 con 0x7ff8f406f360 2024-09-12T03:16:59.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.778+0000 7ff8f2ffd640 1 -- 172.21.15.27:0/3882290632 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ff8f40cf660 msgr2=0x7ff8f40d2780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:59.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.778+0000 7ff8f2ffd640 1 --2- 172.21.15.27:0/3882290632 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ff8f40cf660 0x7ff8f40d2780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:59.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.778+0000 7ff8f2ffd640 1 -- 172.21.15.27:0/3882290632 >> v2:172.21.15.45:3300/0 conn(0x7ff8f40b1ca0 msgr2=0x7ff8f415f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:59.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.778+0000 7ff8f2ffd640 1 --2- 172.21.15.27:0/3882290632 >> v2:172.21.15.45:3300/0 conn(0x7ff8f40b1ca0 0x7ff8f415f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:59.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.778+0000 7ff8f2ffd640 1 -- 172.21.15.27:0/3882290632 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8f40d1ce0 con 0x7ff8f406f360 2024-09-12T03:16:59.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.778+0000 7ff8f2ffd640 1 -- 172.21.15.27:0/3882290632 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 847755213 0 0) 0x7ff8e8001840 con 0x7ff8f406f360 2024-09-12T03:16:59.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.778+0000 7ff8f2ffd640 1 -- 172.21.15.27:0/3882290632 >> v1:172.21.15.27:6789/0 conn(0x7ff8f406f360 legacy=0x7ff8f40cef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:59.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.778+0000 7ff8f2ffd640 1 --2- 172.21.15.27:0/3882290632 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff8dc0039b0 0x7ff8dc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:59.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.778+0000 7ff8f2ffd640 1 --2- 172.21.15.27:0/3882290632 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff8dc004800 0x7ff8dc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:59.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.778+0000 7ff8f2ffd640 1 --2- 172.21.15.27:0/3882290632 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ff8f40cf660 0x7ff8dc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:59.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.778+0000 7ff8f2ffd640 1 -- 172.21.15.27:0/3882290632 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7ff8f40d22f0 con 0x7ff8dc0039b0 2024-09-12T03:16:59.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.778+0000 7ff8f2ffd640 1 -- 172.21.15.27:0/3882290632 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ff8f406cc40 con 0x7ff8dc004800 2024-09-12T03:16:59.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.778+0000 7ff8f2ffd640 1 -- 172.21.15.27:0/3882290632 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7ff8dc00b9c0 con 0x7ff8f40cf660 2024-09-12T03:16:59.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.778+0000 7ff8f3fff640 1 --2- 172.21.15.27:0/3882290632 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff8dc0039b0 0x7ff8dc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:59.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.778+0000 7ff8f37fe640 1 --2- 172.21.15.27:0/3882290632 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff8dc004800 0x7ff8dc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:59.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.778+0000 7ff8f8e6d640 1 --2- 172.21.15.27:0/3882290632 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ff8f40cf660 0x7ff8dc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:59.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.778+0000 7ff8f3fff640 1 -- 172.21.15.27:0/3882290632 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ff8f40cf660 msgr2=0x7ff8dc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:59.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.778+0000 7ff8f3fff640 1 --2- 172.21.15.27:0/3882290632 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ff8f40cf660 0x7ff8dc0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:59.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.778+0000 7ff8f3fff640 1 -- 172.21.15.27:0/3882290632 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff8dc004800 msgr2=0x7ff8dc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:59.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.778+0000 7ff8f3fff640 1 --2- 172.21.15.27:0/3882290632 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff8dc004800 0x7ff8dc004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:59.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.778+0000 7ff8f3fff640 1 -- 172.21.15.27:0/3882290632 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8dc003520 con 0x7ff8dc0039b0 2024-09-12T03:16:59.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.779+0000 7ff8f3fff640 1 --2- 172.21.15.27:0/3882290632 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff8dc0039b0 0x7ff8dc003dd0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7ff8e4009ac0 tx=0x7ff8dc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c844d9791f3a541 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:59.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.779+0000 7ff8f2ffd640 1 -- 172.21.15.27:0/3882290632 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8e4036070 con 0x7ff8dc0039b0 2024-09-12T03:16:59.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.779+0000 7ff8f2ffd640 1 -- 172.21.15.27:0/3882290632 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8e4031400 con 0x7ff8dc0039b0 2024-09-12T03:16:59.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.779+0000 7ff8f2ffd640 1 -- 172.21.15.27:0/3882290632 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8e4009690 con 0x7ff8dc0039b0 2024-09-12T03:16:59.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.780+0000 7ff8fa8d0640 1 -- 172.21.15.27:0/3882290632 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff8dc0039b0 msgr2=0x7ff8dc003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:59.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.780+0000 7ff8fa8d0640 1 --2- 172.21.15.27:0/3882290632 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff8dc0039b0 0x7ff8dc003dd0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7ff8e4009ac0 tx=0x7ff8dc005830 comp rx=0 tx=0).stop 2024-09-12T03:16:59.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.780+0000 7ff8f8e6d640 1 -- 172.21.15.27:0/3882290632 reap_dead start 2024-09-12T03:16:59.784 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.780+0000 7ff8fa8d0640 1 -- 172.21.15.27:0/3882290632 shutdown_connections 2024-09-12T03:16:59.784 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.780+0000 7ff8fa8d0640 1 -- 172.21.15.27:0/3882290632 >> 172.21.15.27:0/3882290632 conn(0x7ff8f4073800 msgr2=0x7ff8f409ba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:59.784 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.780+0000 7ff8fa8d0640 1 -- 172.21.15.27:0/3882290632 shutdown_connections 2024-09-12T03:16:59.784 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.780+0000 7ff8fa8d0640 1 -- 172.21.15.27:0/3882290632 wait complete. 2024-09-12T03:16:59.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.781+0000 7ff8fa8d0640 1 Processor -- start 2024-09-12T03:16:59.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.781+0000 7ff8fa8d0640 1 -- start start 2024-09-12T03:16:59.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.782+0000 7ff8fa8d0640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff8dc0039b0 0x7ff8f41f4130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:59.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.782+0000 7ff8fa8d0640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff8f415bd70 0x7ff8f40b5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:59.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.782+0000 7ff8fa8d0640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ff8f415c1f0 0x7ff8f4159880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:59.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.782+0000 7ff8fa8d0640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7ff8f406d1c0 con 0x7ff8f415bd70 2024-09-12T03:16:59.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.782+0000 7ff8fa8d0640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ff8f40d1f80 con 0x7ff8dc0039b0 2024-09-12T03:16:59.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.782+0000 7ff8fa8d0640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7ff8f406d340 con 0x7ff8f415c1f0 2024-09-12T03:16:59.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.782+0000 7ff8f37fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff8f415bd70 0x7ff8f40b5ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:59.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.782+0000 7ff8f37fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff8f415bd70 0x7ff8f40b5ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57986/0 (socket says 172.21.15.27:57986) 2024-09-12T03:16:59.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.782+0000 7ff8f37fe640 1 -- 172.21.15.27:0/1149892950 learned_addr learned my addr 172.21.15.27:0/1149892950 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:16:59.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.782+0000 7ff8f8e6d640 1 --2- 172.21.15.27:0/1149892950 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ff8f415c1f0 0x7ff8f4159880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:59.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.782+0000 7ff8f3fff640 1 --2- 172.21.15.27:0/1149892950 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff8dc0039b0 0x7ff8f41f4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:59.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.782+0000 7ff8f37fe640 1 -- 172.21.15.27:0/1149892950 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ff8f415c1f0 msgr2=0x7ff8f4159880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:59.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.782+0000 7ff8f37fe640 1 --2- 172.21.15.27:0/1149892950 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ff8f415c1f0 0x7ff8f4159880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:59.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.782+0000 7ff8f37fe640 1 -- 172.21.15.27:0/1149892950 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff8dc0039b0 msgr2=0x7ff8f41f4130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:59.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.782+0000 7ff8f37fe640 1 --2- 172.21.15.27:0/1149892950 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff8dc0039b0 0x7ff8f41f4130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:59.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.782+0000 7ff8f37fe640 1 -- 172.21.15.27:0/1149892950 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8f41f4670 con 0x7ff8f415bd70 2024-09-12T03:16:59.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.782+0000 7ff8f3fff640 1 --2- 172.21.15.27:0/1149892950 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff8dc0039b0 0x7ff8f41f4130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:16:59.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.783+0000 7ff8f37fe640 1 --2- 172.21.15.27:0/1149892950 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff8f415bd70 0x7ff8f40b5ef0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7ff8e8001c40 tx=0x7ff8e80077a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:59.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.783+0000 7ff8f0ff9640 1 -- 172.21.15.27:0/1149892950 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8e800f960 con 0x7ff8f415bd70 2024-09-12T03:16:59.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.783+0000 7ff8f0ff9640 1 -- 172.21.15.27:0/1149892950 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8e8018070 con 0x7ff8f415bd70 2024-09-12T03:16:59.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.783+0000 7ff8f0ff9640 1 -- 172.21.15.27:0/1149892950 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8e800fd80 con 0x7ff8f415bd70 2024-09-12T03:16:59.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.783+0000 7ff8fa8d0640 1 -- 172.21.15.27:0/1149892950 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8f415dc00 con 0x7ff8f415bd70 2024-09-12T03:16:59.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.783+0000 7ff8fa8d0640 1 -- 172.21.15.27:0/1149892950 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8f415e1c0 con 0x7ff8f415bd70 2024-09-12T03:16:59.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.784+0000 7ff8fa8d0640 1 -- 172.21.15.27:0/1149892950 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8f406e060 con 0x7ff8f415bd70 2024-09-12T03:16:59.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.784+0000 7ff8f0ff9640 1 -- 172.21.15.27:0/1149892950 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff8e801d460 con 0x7ff8f415bd70 2024-09-12T03:16:59.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.785+0000 7ff8f0ff9640 1 --2- 172.21.15.27:0/1149892950 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7ff8e006be30 0x7ff8e006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:59.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.785+0000 7ff8f0ff9640 1 -- 172.21.15.27:0/1149892950 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7ff8e808aef0 con 0x7ff8f415bd70 2024-09-12T03:16:59.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.785+0000 7ff8f3fff640 1 --2- 172.21.15.27:0/1149892950 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7ff8e006be30 0x7ff8e006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:59.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.785+0000 7ff8f3fff640 1 --2- 172.21.15.27:0/1149892950 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7ff8e006be30 0x7ff8e006e2f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7ff8e4033000 tx=0x7ff8e4008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:59.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:16:59.789+0000 7ff8f0ff9640 1 -- 172.21.15.27:0/1149892950 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7ff8f406e060 con 0x7ff8f415bd70 2024-09-12T03:17:00.001 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.000+0000 7ff8fa8d0640 1 -- 172.21.15.27:0/1149892950 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff8f41f7720 con 0x7ff8f415bd70 2024-09-12T03:17:00.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.001+0000 7ff8f0ff9640 1 -- 172.21.15.27:0/1149892950 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+5214 (secure 0 0 0) 0x7ff8f41f7720 con 0x7ff8f415bd70 2024-09-12T03:17:00.093 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:17:00.093 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":6,"btime":"2024-09-12T03:16:33:576418+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6836/682326693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":682326693},{"type":"v1","addr":"172.21.15.45:6836","nonce":682326693}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4278,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6834/4084728235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":4084728235},{"type":"v1","addr":"172.21.15.27:6834","nonce":4084728235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4281,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/2844240887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":2844240887},{"type":"v1","addr":"172.21.15.27:6835","nonce":2844240887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-12T03:16:30.914517+0000","modified":"2024-09-12T03:16:32.913742+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.45:6837/1035926332","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6835","nonce":1035926332},{"type":"v1","addr":"172.21.15.45:6837","nonce":1035926332}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4285,"qdb_cluster":[4285]},"id":1}]} 2024-09-12T03:17:00.094 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 6 2024-09-12T03:17:00.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.003+0000 7ff8fa8d0640 1 -- 172.21.15.27:0/1149892950 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7ff8e006be30 msgr2=0x7ff8e006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:00.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.003+0000 7ff8fa8d0640 1 --2- 172.21.15.27:0/1149892950 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7ff8e006be30 0x7ff8e006e2f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7ff8e4033000 tx=0x7ff8e4008030 comp rx=0 tx=0).stop 2024-09-12T03:17:00.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.003+0000 7ff8fa8d0640 1 -- 172.21.15.27:0/1149892950 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff8f415bd70 msgr2=0x7ff8f40b5ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:00.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.003+0000 7ff8fa8d0640 1 --2- 172.21.15.27:0/1149892950 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff8f415bd70 0x7ff8f40b5ef0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7ff8e8001c40 tx=0x7ff8e80077a0 comp rx=0 tx=0).stop 2024-09-12T03:17:00.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.003+0000 7ff8fa8d0640 1 -- 172.21.15.27:0/1149892950 shutdown_connections 2024-09-12T03:17:00.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.003+0000 7ff8fa8d0640 1 --2- 172.21.15.27:0/1149892950 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7ff8e006be30 0x7ff8e006e2f0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:00.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.003+0000 7ff8fa8d0640 1 --2- 172.21.15.27:0/1149892950 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ff8f415c1f0 0x7ff8f4159880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:00.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.003+0000 7ff8fa8d0640 1 --2- 172.21.15.27:0/1149892950 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff8f415bd70 0x7ff8f40b5ef0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:00.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.003+0000 7ff8fa8d0640 1 --2- 172.21.15.27:0/1149892950 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff8dc0039b0 0x7ff8f41f4130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:00.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.003+0000 7ff8fa8d0640 1 -- 172.21.15.27:0/1149892950 >> 172.21.15.27:0/1149892950 conn(0x7ff8f4073800 msgr2=0x7ff8f4161090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:00.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.003+0000 7ff8fa8d0640 1 -- 172.21.15.27:0/1149892950 shutdown_connections 2024-09-12T03:17:00.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.003+0000 7ff8fa8d0640 1 -- 172.21.15.27:0/1149892950 wait complete. 2024-09-12T03:17:00.096 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-12T03:17:00.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.198+0000 7f6929206640 1 Processor -- start 2024-09-12T03:17:00.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.198+0000 7f6929206640 1 -- start start 2024-09-12T03:17:00.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.199+0000 7f6929206640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f69240d5200 0x7f69240d1010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:00.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.199+0000 7f6929206640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f69240d1550 0x7f69240b8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:00.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.199+0000 7f6929206640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f69240d3760 con 0x7f692415eb90 2024-09-12T03:17:00.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.199+0000 7f6929206640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f69240b7d20 con 0x7f69240d5200 2024-09-12T03:17:00.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.199+0000 7f6929206640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f69240d42e0 con 0x7f69240d1550 2024-09-12T03:17:00.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.199+0000 7f6922d9d640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f692415eb90 0x7f69240d2df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:44306/0 (socket says 172.21.15.27:44306) 2024-09-12T03:17:00.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.199+0000 7f6922d9d640 1 -- 172.21.15.27:0/3871957563 learned_addr learned my addr 172.21.15.27:0/3871957563 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:17:00.208 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.200+0000 7f6921d9b640 1 -- 172.21.15.27:0/3871957563 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3787158489 0 0) 0x7f69240d3760 con 0x7f692415eb90 2024-09-12T03:17:00.208 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.200+0000 7f6921d9b640 1 -- 172.21.15.27:0/3871957563 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6910003520 con 0x7f692415eb90 2024-09-12T03:17:00.208 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.200+0000 7f692359e640 1 --2- 172.21.15.27:0/3871957563 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f69240d1550 0x7f69240b8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:00.208 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.200+0000 7f692259c640 1 --2- 172.21.15.27:0/3871957563 >> v2:172.21.15.45:3300/0 conn(0x7f69240d5200 0x7f69240d1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:00.208 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.200+0000 7f6921d9b640 1 -- 172.21.15.27:0/3871957563 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4000983984 0 0) 0x7f6910003520 con 0x7f692415eb90 2024-09-12T03:17:00.208 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.200+0000 7f6921d9b640 1 -- 172.21.15.27:0/3871957563 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f69240d1550 msgr2=0x7f69240b8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:00.208 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.200+0000 7f6921d9b640 1 --2- 172.21.15.27:0/3871957563 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f69240d1550 0x7f69240b8190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:00.208 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.200+0000 7f6921d9b640 1 -- 172.21.15.27:0/3871957563 >> v2:172.21.15.45:3300/0 conn(0x7f69240d5200 msgr2=0x7f69240d1010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:00.208 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.200+0000 7f6921d9b640 1 --2- 172.21.15.27:0/3871957563 >> v2:172.21.15.45:3300/0 conn(0x7f69240d5200 0x7f69240d1010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:00.208 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.200+0000 7f6921d9b640 1 -- 172.21.15.27:0/3871957563 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69240d3a40 con 0x7f692415eb90 2024-09-12T03:17:00.208 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.200+0000 7f6921d9b640 1 -- 172.21.15.27:0/3871957563 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 847755213 0 0) 0x7f69140019f0 con 0x7f692415eb90 2024-09-12T03:17:00.208 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.200+0000 7f6921d9b640 1 -- 172.21.15.27:0/3871957563 >> v1:172.21.15.27:6789/0 conn(0x7f692415eb90 legacy=0x7f69240d2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:00.209 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.200+0000 7f6921d9b640 1 --2- 172.21.15.27:0/3871957563 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f69100039b0 0x7f6910003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:00.209 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.200+0000 7f6921d9b640 1 --2- 172.21.15.27:0/3871957563 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f69240d1550 0x7f6910004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:00.209 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.200+0000 7f6921d9b640 1 --2- 172.21.15.27:0/3871957563 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6910004d40 0x7f69100051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:00.209 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.200+0000 7f6921d9b640 1 -- 172.21.15.27:0/3871957563 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f69240d42e0 con 0x7f6910004d40 2024-09-12T03:17:00.209 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.200+0000 7f6921d9b640 1 -- 172.21.15.27:0/3871957563 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f691000b9c0 con 0x7f69100039b0 2024-09-12T03:17:00.209 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.200+0000 7f6921d9b640 1 -- 172.21.15.27:0/3871957563 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f691000bb40 con 0x7f69240d1550 2024-09-12T03:17:00.209 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.200+0000 7f692359e640 1 --2- 172.21.15.27:0/3871957563 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6910004d40 0x7f69100051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:00.209 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.200+0000 7f6922d9d640 1 --2- 172.21.15.27:0/3871957563 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f69100039b0 0x7f6910003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:00.209 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.200+0000 7f692259c640 1 --2- 172.21.15.27:0/3871957563 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f69240d1550 0x7f6910004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:00.209 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.200+0000 7f692359e640 1 -- 172.21.15.27:0/3871957563 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f69240d1550 msgr2=0x7f6910004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:00.209 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.200+0000 7f692359e640 1 --2- 172.21.15.27:0/3871957563 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f69240d1550 0x7f6910004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:00.209 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.200+0000 7f692359e640 1 -- 172.21.15.27:0/3871957563 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f69100039b0 msgr2=0x7f6910003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:00.209 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.200+0000 7f692359e640 1 --2- 172.21.15.27:0/3871957563 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f69100039b0 0x7f6910003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:00.210 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.200+0000 7f692359e640 1 -- 172.21.15.27:0/3871957563 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6910003520 con 0x7f6910004d40 2024-09-12T03:17:00.210 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.201+0000 7f692359e640 1 --2- 172.21.15.27:0/3871957563 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6910004d40 0x7f69100051c0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f691c009ac0 tx=0x7f6910005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2df8b31f8db86b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:00.210 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.201+0000 7f6921d9b640 1 -- 172.21.15.27:0/3871957563 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f691c036070 con 0x7f6910004d40 2024-09-12T03:17:00.210 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.201+0000 7f6921d9b640 1 -- 172.21.15.27:0/3871957563 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f691c031400 con 0x7f6910004d40 2024-09-12T03:17:00.210 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.201+0000 7f6921d9b640 1 -- 172.21.15.27:0/3871957563 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f691c009690 con 0x7f6910004d40 2024-09-12T03:17:00.210 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.202+0000 7f6929206640 1 -- 172.21.15.27:0/3871957563 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6910004d40 msgr2=0x7f69100051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:00.210 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.202+0000 7f6929206640 1 --2- 172.21.15.27:0/3871957563 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6910004d40 0x7f69100051c0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f691c009ac0 tx=0x7f6910005830 comp rx=0 tx=0).stop 2024-09-12T03:17:00.210 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.202+0000 7f692359e640 1 -- 172.21.15.27:0/3871957563 reap_dead start 2024-09-12T03:17:00.210 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.202+0000 7f6929206640 1 -- 172.21.15.27:0/3871957563 shutdown_connections 2024-09-12T03:17:00.210 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.202+0000 7f6929206640 1 -- 172.21.15.27:0/3871957563 >> 172.21.15.27:0/3871957563 conn(0x7f692406dce0 msgr2=0x7f692406e0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:00.210 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.202+0000 7f6929206640 1 -- 172.21.15.27:0/3871957563 shutdown_connections 2024-09-12T03:17:00.210 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.202+0000 7f6929206640 1 -- 172.21.15.27:0/3871957563 wait complete. 2024-09-12T03:17:00.210 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.203+0000 7f6929206640 1 Processor -- start 2024-09-12T03:17:00.210 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.203+0000 7f6929206640 1 -- start start 2024-09-12T03:17:00.211 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.204+0000 7f6929206640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6910004d40 0x7f69241f7740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:00.211 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.204+0000 7f6929206640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f69240c9390 0x7f69240c9810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:00.211 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.204+0000 7f6929206640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f692414f850 0x7f692414fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:00.211 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.204+0000 7f6929206640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f69240d4000 con 0x7f6910004d40 2024-09-12T03:17:00.211 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.204+0000 7f6929206640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f69240d3500 con 0x7f692414f850 2024-09-12T03:17:00.211 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.204+0000 7f6929206640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f69240d19a0 con 0x7f69240c9390 2024-09-12T03:17:00.211 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.204+0000 7f6922d9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6910004d40 0x7f69241f7740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:00.211 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.204+0000 7f6922d9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6910004d40 0x7f69241f7740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58002/0 (socket says 172.21.15.27:58002) 2024-09-12T03:17:00.211 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.204+0000 7f6922d9d640 1 -- 172.21.15.27:0/3445534700 learned_addr learned my addr 172.21.15.27:0/3445534700 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:17:00.211 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.204+0000 7f692259c640 1 --2- 172.21.15.27:0/3445534700 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f69240c9390 0x7f69240c9810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:00.211 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.204+0000 7f692359e640 1 --2- 172.21.15.27:0/3445534700 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f692414f850 0x7f692414fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:00.211 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.205+0000 7f6922d9d640 1 -- 172.21.15.27:0/3445534700 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f69240c9390 msgr2=0x7f69240c9810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:00.211 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.205+0000 7f6922d9d640 1 --2- 172.21.15.27:0/3445534700 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f69240c9390 0x7f69240c9810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:00.212 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.205+0000 7f6922d9d640 1 -- 172.21.15.27:0/3445534700 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f692414f850 msgr2=0x7f692414fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:00.212 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.205+0000 7f6922d9d640 1 --2- 172.21.15.27:0/3445534700 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f692414f850 0x7f692414fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:00.212 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.205+0000 7f6922d9d640 1 -- 172.21.15.27:0/3445534700 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69240ca0c0 con 0x7f6910004d40 2024-09-12T03:17:00.212 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.205+0000 7f6922d9d640 1 --2- 172.21.15.27:0/3445534700 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6910004d40 0x7f69241f7740 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f6914007a20 tx=0x7f691400a750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:00.212 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.205+0000 7f691b7fe640 1 -- 172.21.15.27:0/3445534700 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6914017020 con 0x7f6910004d40 2024-09-12T03:17:00.212 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.205+0000 7f691b7fe640 1 -- 172.21.15.27:0/3445534700 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6914013600 con 0x7f6910004d40 2024-09-12T03:17:00.212 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.205+0000 7f691b7fe640 1 -- 172.21.15.27:0/3445534700 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f691400ae60 con 0x7f6910004d40 2024-09-12T03:17:00.212 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.205+0000 7f6929206640 1 -- 172.21.15.27:0/3445534700 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69240ca3a0 con 0x7f6910004d40 2024-09-12T03:17:00.212 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.205+0000 7f6929206640 1 -- 172.21.15.27:0/3445534700 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69240ca960 con 0x7f6910004d40 2024-09-12T03:17:00.212 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.206+0000 7f6929206640 1 -- 172.21.15.27:0/3445534700 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f692415f000 con 0x7f6910004d40 2024-09-12T03:17:00.212 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.207+0000 7f691b7fe640 1 -- 172.21.15.27:0/3445534700 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f691401a020 con 0x7f6910004d40 2024-09-12T03:17:00.212 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.207+0000 7f691b7fe640 1 --2- 172.21.15.27:0/3445534700 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f690406be60 0x7f690406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:00.212 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.207+0000 7f691b7fe640 1 -- 172.21.15.27:0/3445534700 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f691408b8d0 con 0x7f6910004d40 2024-09-12T03:17:00.212 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.207+0000 7f692259c640 1 --2- 172.21.15.27:0/3445534700 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f690406be60 0x7f690406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:00.213 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.208+0000 7f692259c640 1 --2- 172.21.15.27:0/3445534700 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f690406be60 0x7f690406e320 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f69240c7f00 tx=0x7f690c008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:00.213 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.211+0000 7f691b7fe640 1 -- 172.21.15.27:0/3445534700 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f6914021020 con 0x7f6910004d40 2024-09-12T03:17:00.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.387+0000 7f6929206640 1 -- 172.21.15.27:0/3445534700 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f69240cad80 con 0x7f6910004d40 2024-09-12T03:17:00.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.388+0000 7f691b7fe640 1 -- 172.21.15.27:0/3445534700 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16617 (secure 0 0 0) 0x7f6914059c30 con 0x7f6910004d40 2024-09-12T03:17:00.389 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:17:00.389 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":21,"fsid":"d2df504e-2647-4385-a5db-9fdf69f487a0","created":"2024-09-12T03:16:07.778937+0000","modified":"2024-09-12T03:16:32.570182+0000","last_up_change":"2024-09-12T03:16:19.314778+0000","last_in_change":"2024-09-12T03:16:09.893086+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T03:16:20.097646+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-12T03:16:28.790791+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-12T03:16:29.909065+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1851116c-1fb3-4dc9-9772-f90e2ef4c355","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6817","nonce":3002321488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6819","nonce":3002321488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6823","nonce":3002321488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6821","nonce":3002321488}]},"public_addr":"172.21.15.27:6817/3002321488","cluster_addr":"172.21.15.27:6819/3002321488","heartbeat_back_addr":"172.21.15.27:6823/3002321488","heartbeat_front_addr":"172.21.15.27:6821/3002321488","state":["exists","up"]},{"osd":1,"uuid":"f38ae17d-73eb-4204-b99d-975fa806fb87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6809","nonce":1613102886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6811","nonce":1613102886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6815","nonce":1613102886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6813","nonce":1613102886}]},"public_addr":"172.21.15.27:6809/1613102886","cluster_addr":"172.21.15.27:6811/1613102886","heartbeat_back_addr":"172.21.15.27:6815/1613102886","heartbeat_front_addr":"172.21.15.27:6813/1613102886","state":["exists","up"]},{"osd":2,"uuid":"fb12dcc8-bdd0-42b8-b46d-8b10106224bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6825","nonce":3534723355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6827","nonce":3534723355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6831","nonce":3534723355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6829","nonce":3534723355}]},"public_addr":"172.21.15.27:6825/3534723355","cluster_addr":"172.21.15.27:6827/3534723355","heartbeat_back_addr":"172.21.15.27:6831/3534723355","heartbeat_front_addr":"172.21.15.27:6829/3534723355","state":["exists","up"]},{"osd":3,"uuid":"d74d3f49-9c24-4b9f-98e2-3ecd7b5ef3b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6801","nonce":80661598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6803","nonce":80661598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6807","nonce":80661598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6805","nonce":80661598}]},"public_addr":"172.21.15.27:6801/80661598","cluster_addr":"172.21.15.27:6803/80661598","heartbeat_back_addr":"172.21.15.27:6807/80661598","heartbeat_front_addr":"172.21.15.27:6805/80661598","state":["exists","up"]},{"osd":4,"uuid":"2bf123f2-1b0d-41c4-aa3b-805bab267e59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6817","nonce":3943189827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6819","nonce":3943189827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6823","nonce":3943189827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6821","nonce":3943189827}]},"public_addr":"172.21.15.45:6817/3943189827","cluster_addr":"172.21.15.45:6819/3943189827","heartbeat_back_addr":"172.21.15.45:6823/3943189827","heartbeat_front_addr":"172.21.15.45:6821/3943189827","state":["exists","up"]},{"osd":5,"uuid":"c49ba5da-91c6-4a86-a97b-4aba89c253ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6809","nonce":3620760330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6811","nonce":3620760330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6815","nonce":3620760330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6813","nonce":3620760330}]},"public_addr":"172.21.15.45:6809/3620760330","cluster_addr":"172.21.15.45:6811/3620760330","heartbeat_back_addr":"172.21.15.45:6815/3620760330","heartbeat_front_addr":"172.21.15.45:6813/3620760330","state":["exists","up"]},{"osd":6,"uuid":"b0123475-89f7-4a34-9e33-d797b99dd9dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6801","nonce":1989083510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6803","nonce":1989083510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6807","nonce":1989083510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6805","nonce":1989083510}]},"public_addr":"172.21.15.45:6801/1989083510","cluster_addr":"172.21.15.45:6803/1989083510","heartbeat_back_addr":"172.21.15.45:6807/1989083510","heartbeat_front_addr":"172.21.15.45:6805/1989083510","state":["exists","up"]},{"osd":7,"uuid":"4dad773d-b7d8-46fc-8a10-a972aad27b29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6825","nonce":1513300294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6827","nonce":1513300294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6831","nonce":1513300294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6829","nonce":1513300294}]},"public_addr":"172.21.15.45:6825/1513300294","cluster_addr":"172.21.15.45:6827/1513300294","heartbeat_back_addr":"172.21.15.45:6831/1513300294","heartbeat_front_addr":"172.21.15.45:6829/1513300294","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.881855+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:17.133252+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.735339+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.500742+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.477571+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:17.167667+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.470291+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.873743+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T03:17:00.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.391+0000 7f6929206640 1 -- 172.21.15.27:0/3445534700 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f690406be60 msgr2=0x7f690406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:00.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.391+0000 7f6929206640 1 --2- 172.21.15.27:0/3445534700 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f690406be60 0x7f690406e320 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f69240c7f00 tx=0x7f690c008040 comp rx=0 tx=0).stop 2024-09-12T03:17:00.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.391+0000 7f6929206640 1 -- 172.21.15.27:0/3445534700 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6910004d40 msgr2=0x7f69241f7740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:00.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.391+0000 7f6929206640 1 --2- 172.21.15.27:0/3445534700 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6910004d40 0x7f69241f7740 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f6914007a20 tx=0x7f691400a750 comp rx=0 tx=0).stop 2024-09-12T03:17:00.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.391+0000 7f6929206640 1 -- 172.21.15.27:0/3445534700 shutdown_connections 2024-09-12T03:17:00.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.391+0000 7f6929206640 1 --2- 172.21.15.27:0/3445534700 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f690406be60 0x7f690406e320 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:00.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.391+0000 7f6929206640 1 --2- 172.21.15.27:0/3445534700 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f692414f850 0x7f692414fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:00.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.391+0000 7f6929206640 1 --2- 172.21.15.27:0/3445534700 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f69240c9390 0x7f69240c9810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:00.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.391+0000 7f6929206640 1 --2- 172.21.15.27:0/3445534700 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6910004d40 0x7f69241f7740 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:00.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.391+0000 7f6929206640 1 -- 172.21.15.27:0/3445534700 >> 172.21.15.27:0/3445534700 conn(0x7f692406dce0 msgr2=0x7f69240b9910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:00.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.391+0000 7f6929206640 1 -- 172.21.15.27:0/3445534700 shutdown_connections 2024-09-12T03:17:00.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.391+0000 7f6929206640 1 -- 172.21.15.27:0/3445534700 wait complete. 2024-09-12T03:17:00.402 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-12T03:17:00.402 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-12T03:17:00.483 DEBUG:teuthology.orchestra.run.smithi027:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-12T03:17:00.486 DEBUG:teuthology.orchestra.run.smithi045:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-12T03:17:00.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.505+0000 7fb29a623640 1 Processor -- start 2024-09-12T03:17:00.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.505+0000 7fb29a623640 1 -- start start 2024-09-12T03:17:00.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.506+0000 7fb29a623640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb29406e7a0 0x7fb2940b35c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:00.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.506+0000 7fb29a623640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb2940b59f0 con 0x7fb29415b360 2024-09-12T03:17:00.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.506+0000 7fb29a623640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb29415b740 con 0x7fb2940b3b00 2024-09-12T03:17:00.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.506+0000 7fb29a623640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fb2940b3f50 con 0x7fb29406e7a0 2024-09-12T03:17:00.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.506+0000 7fb293fff640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fb29415b360 0x7fb29406e090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:44312/0 (socket says 172.21.15.27:44312) 2024-09-12T03:17:00.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.506+0000 7fb293fff640 1 -- 172.21.15.27:0/1077944816 learned_addr learned my addr 172.21.15.27:0/1077944816 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:17:00.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.506+0000 7fb28bfff640 1 --2- 172.21.15.27:0/1077944816 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb29406e7a0 0x7fb2940b35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:00.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.506+0000 7fb2937fe640 1 -- 172.21.15.27:0/1077944816 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1700166320 0 0) 0x7fb284001110 con 0x7fb29415b360 2024-09-12T03:17:00.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.506+0000 7fb2937fe640 1 -- 172.21.15.27:0/1077944816 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb280005730 con 0x7fb29415b360 2024-09-12T03:17:00.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.506+0000 7fb2937fe640 1 -- 172.21.15.27:0/1077944816 <== mon.1 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3398950111 0 0) 0x7fb29415b740 con 0x7fb2940b3b00 2024-09-12T03:17:00.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.506+0000 7fb2937fe640 1 -- 172.21.15.27:0/1077944816 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb284001110 con 0x7fb2940b3b00 2024-09-12T03:17:00.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.506+0000 7fb28bfff640 1 -- 172.21.15.27:0/1077944816 >> v1:172.21.15.45:6789/0 conn(0x7fb2940b3b00 legacy=0x7fb29415eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:00.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.506+0000 7fb28bfff640 1 -- 172.21.15.27:0/1077944816 >> v1:172.21.15.27:6789/0 conn(0x7fb29415b360 legacy=0x7fb29406e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:00.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.506+0000 7fb28bfff640 1 -- 172.21.15.27:0/1077944816 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb29406ec30 con 0x7fb29406e7a0 2024-09-12T03:17:00.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.507+0000 7fb28bfff640 1 --2- 172.21.15.27:0/1077944816 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb29406e7a0 0x7fb2940b35c0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fb27c0097d0 tx=0x7fb27c02a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=26828c26ef752313 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:00.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.508+0000 7fb2937fe640 1 -- 172.21.15.27:0/1077944816 <== mon.2 v2:172.21.15.45:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb27c037070 con 0x7fb29406e7a0 2024-09-12T03:17:00.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.508+0000 7fb2937fe640 1 -- 172.21.15.27:0/1077944816 <== mon.2 v2:172.21.15.45:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb27c032440 con 0x7fb29406e7a0 2024-09-12T03:17:00.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.508+0000 7fb2937fe640 1 -- 172.21.15.27:0/1077944816 <== mon.2 v2:172.21.15.45:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb27c032810 con 0x7fb29406e7a0 2024-09-12T03:17:00.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.508+0000 7fb29a623640 1 -- 172.21.15.27:0/1077944816 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb29406e7a0 msgr2=0x7fb2940b35c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:00.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.508+0000 7fb29a623640 1 --2- 172.21.15.27:0/1077944816 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb29406e7a0 0x7fb2940b35c0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fb27c0097d0 tx=0x7fb27c02a180 comp rx=0 tx=0).stop 2024-09-12T03:17:00.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.508+0000 7fb29a623640 1 -- 172.21.15.27:0/1077944816 shutdown_connections 2024-09-12T03:17:00.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.508+0000 7fb29a623640 1 --2- 172.21.15.27:0/1077944816 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb29406e7a0 0x7fb2940b35c0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:00.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.508+0000 7fb29a623640 1 -- 172.21.15.27:0/1077944816 >> 172.21.15.27:0/1077944816 conn(0x7fb29406ce20 msgr2=0x7fb29406d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:00.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.508+0000 7fb29a623640 1 -- 172.21.15.27:0/1077944816 shutdown_connections 2024-09-12T03:17:00.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.508+0000 7fb29a623640 1 -- 172.21.15.27:0/1077944816 wait complete. 2024-09-12T03:17:00.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.509+0000 7fb29a623640 1 Processor -- start 2024-09-12T03:17:00.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.510+0000 7fb29a623640 1 -- start start 2024-09-12T03:17:00.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.510+0000 7fb29a623640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb2940b3b00 0x7fb29414bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:00.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.510+0000 7fb29a623640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb29415b360 0x7fb2940ca8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:00.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.510+0000 7fb29a623640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb294158950 0x7fb29415df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:00.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.510+0000 7fb29a623640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fb29415bfa0 con 0x7fb294158950 2024-09-12T03:17:00.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.510+0000 7fb29a623640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb29415bcc0 con 0x7fb2940b3b00 2024-09-12T03:17:00.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.510+0000 7fb29a623640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fb29415c7a0 con 0x7fb29415b360 2024-09-12T03:17:00.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.510+0000 7fb298bc0640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb294158950 0x7fb29415df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:00.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.510+0000 7fb298bc0640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb294158950 0x7fb29415df50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58014/0 (socket says 172.21.15.27:58014) 2024-09-12T03:17:00.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.510+0000 7fb298bc0640 1 -- 172.21.15.27:0/2672216432 learned_addr learned my addr 172.21.15.27:0/2672216432 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:17:00.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.510+0000 7fb293fff640 1 --2- 172.21.15.27:0/2672216432 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb2940b3b00 0x7fb29414bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:00.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.511+0000 7fb28bfff640 1 --2- 172.21.15.27:0/2672216432 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb29415b360 0x7fb2940ca8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:00.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.511+0000 7fb298bc0640 1 -- 172.21.15.27:0/2672216432 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb29415b360 msgr2=0x7fb2940ca8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:00.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.511+0000 7fb298bc0640 1 --2- 172.21.15.27:0/2672216432 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb29415b360 0x7fb2940ca8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:00.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.511+0000 7fb298bc0640 1 -- 172.21.15.27:0/2672216432 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb2940b3b00 msgr2=0x7fb29414bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:00.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.511+0000 7fb298bc0640 1 --2- 172.21.15.27:0/2672216432 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb2940b3b00 0x7fb29414bdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:00.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.511+0000 7fb298bc0640 1 -- 172.21.15.27:0/2672216432 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2941f4980 con 0x7fb294158950 2024-09-12T03:17:00.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.511+0000 7fb298bc0640 1 --2- 172.21.15.27:0/2672216432 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb294158950 0x7fb29415df50 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fb28c00c7c0 tx=0x7fb28c00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:00.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.511+0000 7fb2917fa640 1 -- 172.21.15.27:0/2672216432 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb28c003b70 con 0x7fb294158950 2024-09-12T03:17:00.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.511+0000 7fb2917fa640 1 -- 172.21.15.27:0/2672216432 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb28c003d10 con 0x7fb294158950 2024-09-12T03:17:00.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.511+0000 7fb2917fa640 1 -- 172.21.15.27:0/2672216432 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb28c00aa50 con 0x7fb294158950 2024-09-12T03:17:00.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.511+0000 7fb29a623640 1 -- 172.21.15.27:0/2672216432 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2941f4c60 con 0x7fb294158950 2024-09-12T03:17:00.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.511+0000 7fb29a623640 1 -- 172.21.15.27:0/2672216432 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2940cc450 con 0x7fb294158950 2024-09-12T03:17:00.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.512+0000 7fb29a623640 1 -- 172.21.15.27:0/2672216432 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2941f4e30 con 0x7fb294158950 2024-09-12T03:17:00.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.513+0000 7fb2917fa640 1 -- 172.21.15.27:0/2672216432 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb28c00abf0 con 0x7fb294158950 2024-09-12T03:17:00.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.513+0000 7fb2917fa640 1 --2- 172.21.15.27:0/2672216432 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fb27406be30 0x7fb27406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:00.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.513+0000 7fb2917fa640 1 -- 172.21.15.27:0/2672216432 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fb28c08ad30 con 0x7fb294158950 2024-09-12T03:17:00.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.513+0000 7fb293fff640 1 --2- 172.21.15.27:0/2672216432 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fb27406be30 0x7fb27406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:00.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.514+0000 7fb293fff640 1 --2- 172.21.15.27:0/2672216432 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fb27406be30 0x7fb27406e2f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fb2840013b0 tx=0x7fb284007d30 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:00.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.517+0000 7fb2917fa640 1 -- 172.21.15.27:0/2672216432 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fb28c059090 con 0x7fb294158950 2024-09-12T03:17:00.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.728+0000 7fb29a623640 1 -- 172.21.15.27:0/2672216432 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb2940cc680 con 0x7fb294158950 2024-09-12T03:17:00.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.729+0000 7fb2917fa640 1 -- 172.21.15.27:0/2672216432 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+5214 (secure 0 0 0) 0x7fb2940cc680 con 0x7fb294158950 2024-09-12T03:17:00.729 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:17:00.729 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":6,"btime":"2024-09-12T03:16:33:576418+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6836/682326693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":682326693},{"type":"v1","addr":"172.21.15.45:6836","nonce":682326693}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4278,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6834/4084728235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":4084728235},{"type":"v1","addr":"172.21.15.27:6834","nonce":4084728235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4281,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/2844240887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":2844240887},{"type":"v1","addr":"172.21.15.27:6835","nonce":2844240887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-12T03:16:30.914517+0000","modified":"2024-09-12T03:16:32.913742+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.45:6837/1035926332","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6835","nonce":1035926332},{"type":"v1","addr":"172.21.15.45:6837","nonce":1035926332}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4285,"qdb_cluster":[4285]},"id":1}]} 2024-09-12T03:17:00.730 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 6 2024-09-12T03:17:00.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.731+0000 7fb29a623640 1 -- 172.21.15.27:0/2672216432 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fb27406be30 msgr2=0x7fb27406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:00.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.731+0000 7fb29a623640 1 --2- 172.21.15.27:0/2672216432 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fb27406be30 0x7fb27406e2f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fb2840013b0 tx=0x7fb284007d30 comp rx=0 tx=0).stop 2024-09-12T03:17:00.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.731+0000 7fb29a623640 1 -- 172.21.15.27:0/2672216432 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb294158950 msgr2=0x7fb29415df50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:00.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.731+0000 7fb29a623640 1 --2- 172.21.15.27:0/2672216432 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb294158950 0x7fb29415df50 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fb28c00c7c0 tx=0x7fb28c00cc80 comp rx=0 tx=0).stop 2024-09-12T03:17:00.732 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.731+0000 7fb29a623640 1 -- 172.21.15.27:0/2672216432 shutdown_connections 2024-09-12T03:17:00.732 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.731+0000 7fb29a623640 1 --2- 172.21.15.27:0/2672216432 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fb27406be30 0x7fb27406e2f0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:00.732 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.731+0000 7fb29a623640 1 --2- 172.21.15.27:0/2672216432 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb294158950 0x7fb29415df50 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:00.732 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.731+0000 7fb29a623640 1 --2- 172.21.15.27:0/2672216432 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb29415b360 0x7fb2940ca8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:00.732 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.731+0000 7fb29a623640 1 --2- 172.21.15.27:0/2672216432 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb2940b3b00 0x7fb29414bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:00.732 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.731+0000 7fb29a623640 1 -- 172.21.15.27:0/2672216432 >> 172.21.15.27:0/2672216432 conn(0x7fb29406ce20 msgr2=0x7fb2940d1d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:00.732 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.731+0000 7fb29a623640 1 -- 172.21.15.27:0/2672216432 shutdown_connections 2024-09-12T03:17:00.732 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.731+0000 7fb29a623640 1 -- 172.21.15.27:0/2672216432 wait complete. 2024-09-12T03:17:00.742 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-12T03:17:00.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.844+0000 7eff8528d640 1 Processor -- start 2024-09-12T03:17:00.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.844+0000 7eff8528d640 1 -- start start 2024-09-12T03:17:00.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.845+0000 7eff8528d640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7eff8015e660 0x7eff80073210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:00.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.845+0000 7eff8528d640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7eff8015ea40 con 0x7eff800cf3c0 2024-09-12T03:17:00.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.845+0000 7eff8528d640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7eff8006d240 con 0x7eff80071150 2024-09-12T03:17:00.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.845+0000 7eff8528d640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7eff800b5a20 con 0x7eff8015e660 2024-09-12T03:17:00.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.845+0000 7eff7effd640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7eff8015e660 0x7eff80073210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:00.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.845+0000 7eff7e7fc640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7eff800cf3c0 0x7eff80070a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:44328/0 (socket says 172.21.15.27:44328) 2024-09-12T03:17:00.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.845+0000 7eff7e7fc640 1 -- 172.21.15.27:0/2273627884 learned_addr learned my addr 172.21.15.27:0/2273627884 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:17:00.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.846+0000 7eff7effd640 1 -- 172.21.15.27:0/2273627884 >> v1:172.21.15.45:6789/0 conn(0x7eff80071150 legacy=0x7eff801681e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:00.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.846+0000 7eff7effd640 1 -- 172.21.15.27:0/2273627884 >> v1:172.21.15.27:6789/0 conn(0x7eff800cf3c0 legacy=0x7eff80070a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:00.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.846+0000 7eff7effd640 1 -- 172.21.15.27:0/2273627884 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff80071970 con 0x7eff8015e660 2024-09-12T03:17:00.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.846+0000 7eff7dffb640 1 -- 172.21.15.27:0/2273627884 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1444742734 0 0) 0x7eff8015ea40 con 0x7eff800cf3c0 2024-09-12T03:17:00.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.846+0000 7eff7effd640 1 --2- 172.21.15.27:0/2273627884 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7eff8015e660 0x7eff80073210 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7eff700097a0 tx=0x7eff7002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13762ccf99177818 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:00.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.847+0000 7eff7dffb640 1 -- 172.21.15.27:0/2273627884 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff70037070 con 0x7eff8015e660 2024-09-12T03:17:00.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.847+0000 7eff7dffb640 1 -- 172.21.15.27:0/2273627884 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff70032440 con 0x7eff8015e660 2024-09-12T03:17:00.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.847+0000 7eff7dffb640 1 -- 172.21.15.27:0/2273627884 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff70032810 con 0x7eff8015e660 2024-09-12T03:17:00.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.847+0000 7eff8528d640 1 -- 172.21.15.27:0/2273627884 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7eff8015e660 msgr2=0x7eff80073210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:00.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.847+0000 7eff8528d640 1 --2- 172.21.15.27:0/2273627884 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7eff8015e660 0x7eff80073210 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7eff700097a0 tx=0x7eff7002a230 comp rx=0 tx=0).stop 2024-09-12T03:17:00.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.847+0000 7eff8528d640 1 -- 172.21.15.27:0/2273627884 shutdown_connections 2024-09-12T03:17:00.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.847+0000 7eff8528d640 1 --2- 172.21.15.27:0/2273627884 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7eff8015e660 0x7eff80073210 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:00.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.847+0000 7eff8528d640 1 -- 172.21.15.27:0/2273627884 >> 172.21.15.27:0/2273627884 conn(0x7eff8006f2c0 msgr2=0x7eff8006f6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:00.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.847+0000 7eff8528d640 1 -- 172.21.15.27:0/2273627884 shutdown_connections 2024-09-12T03:17:00.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.847+0000 7eff8528d640 1 -- 172.21.15.27:0/2273627884 wait complete. 2024-09-12T03:17:00.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.849+0000 7eff8528d640 1 Processor -- start 2024-09-12T03:17:00.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.849+0000 7eff8528d640 1 -- start start 2024-09-12T03:17:00.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.849+0000 7eff8528d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7eff80071150 0x7eff8015d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:00.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.849+0000 7eff8528d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7eff800cf3c0 0x7eff8015fff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:00.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.849+0000 7eff8528d640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7eff8015e660 0x7eff800cd0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:00.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.849+0000 7eff8528d640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7eff8006d7c0 con 0x7eff800cf3c0 2024-09-12T03:17:00.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.849+0000 7eff8528d640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7eff8006d4e0 con 0x7eff80071150 2024-09-12T03:17:00.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.849+0000 7eff8528d640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7eff80073750 con 0x7eff8015e660 2024-09-12T03:17:00.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.849+0000 7eff7e7fc640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7eff800cf3c0 0x7eff8015fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:00.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.849+0000 7eff7e7fc640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7eff800cf3c0 0x7eff8015fff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58032/0 (socket says 172.21.15.27:58032) 2024-09-12T03:17:00.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.849+0000 7eff7e7fc640 1 -- 172.21.15.27:0/612845474 learned_addr learned my addr 172.21.15.27:0/612845474 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:17:00.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.850+0000 7eff7effd640 1 --2- 172.21.15.27:0/612845474 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7eff80071150 0x7eff8015d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:00.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.850+0000 7eff7f7fe640 1 --2- 172.21.15.27:0/612845474 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7eff8015e660 0x7eff800cd0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:00.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.850+0000 7eff7e7fc640 1 -- 172.21.15.27:0/612845474 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7eff8015e660 msgr2=0x7eff800cd0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:00.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.850+0000 7eff7e7fc640 1 --2- 172.21.15.27:0/612845474 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7eff8015e660 0x7eff800cd0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:00.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.850+0000 7eff7e7fc640 1 -- 172.21.15.27:0/612845474 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7eff80071150 msgr2=0x7eff8015d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:00.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.850+0000 7eff7e7fc640 1 --2- 172.21.15.27:0/612845474 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7eff80071150 0x7eff8015d590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:00.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.850+0000 7eff7e7fc640 1 -- 172.21.15.27:0/612845474 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff8015c100 con 0x7eff800cf3c0 2024-09-12T03:17:00.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.850+0000 7eff7e7fc640 1 --2- 172.21.15.27:0/612845474 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7eff800cf3c0 0x7eff8015fff0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7eff7400c460 tx=0x7eff7400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:00.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.850+0000 7eff677fe640 1 -- 172.21.15.27:0/612845474 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff74018070 con 0x7eff800cf3c0 2024-09-12T03:17:00.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.850+0000 7eff677fe640 1 -- 172.21.15.27:0/612845474 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff74007520 con 0x7eff800cf3c0 2024-09-12T03:17:00.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.850+0000 7eff677fe640 1 -- 172.21.15.27:0/612845474 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff74004a20 con 0x7eff800cf3c0 2024-09-12T03:17:00.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.850+0000 7eff8528d640 1 -- 172.21.15.27:0/612845474 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff80160b20 con 0x7eff800cf3c0 2024-09-12T03:17:00.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.850+0000 7eff8528d640 1 -- 172.21.15.27:0/612845474 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff801610e0 con 0x7eff800cf3c0 2024-09-12T03:17:00.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.851+0000 7eff8528d640 1 -- 172.21.15.27:0/612845474 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff80073b10 con 0x7eff800cf3c0 2024-09-12T03:17:00.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.852+0000 7eff677fe640 1 -- 172.21.15.27:0/612845474 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7eff7401b070 con 0x7eff800cf3c0 2024-09-12T03:17:00.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.852+0000 7eff677fe640 1 --2- 172.21.15.27:0/612845474 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7eff6c06be30 0x7eff6c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:00.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.852+0000 7eff677fe640 1 -- 172.21.15.27:0/612845474 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7eff7408a240 con 0x7eff800cf3c0 2024-09-12T03:17:00.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.852+0000 7eff7effd640 1 --2- 172.21.15.27:0/612845474 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7eff6c06be30 0x7eff6c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:00.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.853+0000 7eff7effd640 1 --2- 172.21.15.27:0/612845474 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7eff6c06be30 0x7eff6c06e2f0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7eff70034000 tx=0x7eff7002e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:00.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:00.856+0000 7eff677fe640 1 -- 172.21.15.27:0/612845474 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7eff74058510 con 0x7eff800cf3c0 2024-09-12T03:17:01.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.067+0000 7eff8528d640 1 -- 172.21.15.27:0/612845474 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7eff80160760 con 0x7eff800cf3c0 2024-09-12T03:17:01.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.068+0000 7eff677fe640 1 -- 172.21.15.27:0/612845474 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+5214 (secure 0 0 0) 0x7eff7405c810 con 0x7eff800cf3c0 2024-09-12T03:17:01.068 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:17:01.068 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":6,"btime":"2024-09-12T03:16:33:576418+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6836/682326693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":682326693},{"type":"v1","addr":"172.21.15.45:6836","nonce":682326693}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4278,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6834/4084728235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":4084728235},{"type":"v1","addr":"172.21.15.27:6834","nonce":4084728235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4281,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/2844240887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":2844240887},{"type":"v1","addr":"172.21.15.27:6835","nonce":2844240887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-12T03:16:30.914517+0000","modified":"2024-09-12T03:16:32.913742+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.45:6837/1035926332","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6835","nonce":1035926332},{"type":"v1","addr":"172.21.15.45:6837","nonce":1035926332}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4285,"qdb_cluster":[4285]},"id":1}]} 2024-09-12T03:17:01.068 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 6 2024-09-12T03:17:01.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.070+0000 7eff8528d640 1 -- 172.21.15.27:0/612845474 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7eff6c06be30 msgr2=0x7eff6c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:01.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.070+0000 7eff8528d640 1 --2- 172.21.15.27:0/612845474 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7eff6c06be30 0x7eff6c06e2f0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7eff70034000 tx=0x7eff7002e000 comp rx=0 tx=0).stop 2024-09-12T03:17:01.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.070+0000 7eff8528d640 1 -- 172.21.15.27:0/612845474 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7eff800cf3c0 msgr2=0x7eff8015fff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:01.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.070+0000 7eff8528d640 1 --2- 172.21.15.27:0/612845474 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7eff800cf3c0 0x7eff8015fff0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7eff7400c460 tx=0x7eff7400c920 comp rx=0 tx=0).stop 2024-09-12T03:17:01.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.070+0000 7eff8528d640 1 -- 172.21.15.27:0/612845474 shutdown_connections 2024-09-12T03:17:01.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.070+0000 7eff8528d640 1 --2- 172.21.15.27:0/612845474 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7eff6c06be30 0x7eff6c06e2f0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:01.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.070+0000 7eff8528d640 1 --2- 172.21.15.27:0/612845474 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7eff8015e660 0x7eff800cd0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:01.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.070+0000 7eff8528d640 1 --2- 172.21.15.27:0/612845474 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7eff800cf3c0 0x7eff8015fff0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:01.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.070+0000 7eff8528d640 1 --2- 172.21.15.27:0/612845474 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7eff80071150 0x7eff8015d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:01.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.070+0000 7eff8528d640 1 -- 172.21.15.27:0/612845474 >> 172.21.15.27:0/612845474 conn(0x7eff8006f2c0 msgr2=0x7eff800d03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:01.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.070+0000 7eff8528d640 1 -- 172.21.15.27:0/612845474 shutdown_connections 2024-09-12T03:17:01.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.070+0000 7eff8528d640 1 -- 172.21.15.27:0/612845474 wait complete. 2024-09-12T03:17:01.081 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-12T03:17:01.183 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.183+0000 7f58c605e640 1 Processor -- start 2024-09-12T03:17:01.183 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.183+0000 7f58c605e640 1 -- start start 2024-09-12T03:17:01.184 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.184+0000 7f58c605e640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f58c00b6470 0x7f58c00d2710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:01.184 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.184+0000 7f58c605e640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f58c00b6850 0x7f58c00d2c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:01.184 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.184+0000 7f58c605e640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f58c00d3190 0x7f58c0161b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:01.184 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.184+0000 7f58c605e640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f58c00d1ec0 con 0x7f58c00d3190 2024-09-12T03:17:01.185 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.184+0000 7f58c605e640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f58c00cf980 con 0x7f58c00b6470 2024-09-12T03:17:01.185 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.184+0000 7f58c605e640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f58c00b79b0 con 0x7f58c00b6850 2024-09-12T03:17:01.185 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.184+0000 7f58bffff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f58c00d3190 0x7f58c0161b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:01.185 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.184+0000 7f58bffff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f58c00d3190 0x7f58c0161b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58038/0 (socket says 172.21.15.27:58038) 2024-09-12T03:17:01.185 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.184+0000 7f58bffff640 1 -- 172.21.15.27:0/1826133011 learned_addr learned my addr 172.21.15.27:0/1826133011 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:17:01.185 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.184+0000 7f58bf7fe640 1 --2- 172.21.15.27:0/1826133011 >> v2:172.21.15.45:3300/0 conn(0x7f58c00b6470 0x7f58c00d2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:01.185 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.184+0000 7f58beffd640 1 --2- 172.21.15.27:0/1826133011 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f58c00b6850 0x7f58c00d2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:01.185 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.184+0000 7f58bffff640 1 -- 172.21.15.27:0/1826133011 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f58c00b6850 msgr2=0x7f58c00d2c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:01.185 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.184+0000 7f58bffff640 1 --2- 172.21.15.27:0/1826133011 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f58c00b6850 0x7f58c00d2c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:01.185 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.184+0000 7f58bffff640 1 -- 172.21.15.27:0/1826133011 >> v2:172.21.15.45:3300/0 conn(0x7f58c00b6470 msgr2=0x7f58c00d2710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:01.185 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.184+0000 7f58bffff640 1 --2- 172.21.15.27:0/1826133011 >> v2:172.21.15.45:3300/0 conn(0x7f58c00b6470 0x7f58c00d2710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:01.185 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.184+0000 7f58bffff640 1 -- 172.21.15.27:0/1826133011 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58c00cb580 con 0x7f58c00d3190 2024-09-12T03:17:01.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.185+0000 7f58bffff640 1 --2- 172.21.15.27:0/1826133011 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f58c00d3190 0x7f58c0161b80 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f58c0050fd0 tx=0x7f58b802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33ee42caef0de83f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:01.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.185+0000 7f58be7fc640 1 -- 172.21.15.27:0/1826133011 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58b8036070 con 0x7f58c00d3190 2024-09-12T03:17:01.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.185+0000 7f58be7fc640 1 -- 172.21.15.27:0/1826133011 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58b8031600 con 0x7f58c00d3190 2024-09-12T03:17:01.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.185+0000 7f58be7fc640 1 -- 172.21.15.27:0/1826133011 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58b80319d0 con 0x7f58c00d3190 2024-09-12T03:17:01.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.186+0000 7f58c605e640 1 -- 172.21.15.27:0/1826133011 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f58c00d3190 msgr2=0x7f58c0161b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:01.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.186+0000 7f58c605e640 1 --2- 172.21.15.27:0/1826133011 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f58c00d3190 0x7f58c0161b80 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f58c0050fd0 tx=0x7f58b802a6a0 comp rx=0 tx=0).stop 2024-09-12T03:17:01.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.186+0000 7f58c605e640 1 -- 172.21.15.27:0/1826133011 shutdown_connections 2024-09-12T03:17:01.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.186+0000 7f58c605e640 1 --2- 172.21.15.27:0/1826133011 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f58c00d3190 0x7f58c0161b80 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:01.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.186+0000 7f58c605e640 1 --2- 172.21.15.27:0/1826133011 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f58c00b6850 0x7f58c00d2c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:01.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.186+0000 7f58c605e640 1 --2- 172.21.15.27:0/1826133011 >> v2:172.21.15.45:3300/0 conn(0x7f58c00b6470 0x7f58c00d2710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:01.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.186+0000 7f58c605e640 1 -- 172.21.15.27:0/1826133011 >> 172.21.15.27:0/1826133011 conn(0x7f58c00d1200 msgr2=0x7f58c00d1610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:01.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.186+0000 7f58c605e640 1 -- 172.21.15.27:0/1826133011 shutdown_connections 2024-09-12T03:17:01.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.186+0000 7f58c605e640 1 -- 172.21.15.27:0/1826133011 wait complete. 2024-09-12T03:17:01.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.187+0000 7f58c605e640 1 Processor -- start 2024-09-12T03:17:01.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.187+0000 7f58c605e640 1 -- start start 2024-09-12T03:17:01.190 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.188+0000 7f58c605e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f58c00b6470 0x7f58c00cf090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:01.190 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.188+0000 7f58c605e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f58c00b6850 0x7f58c015b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:01.190 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.188+0000 7f58c605e640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f58c00d3190 0x7f58c00ac9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:01.191 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.188+0000 7f58c605e640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f58c006d9d0 con 0x7f58c00b6470 2024-09-12T03:17:01.191 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.188+0000 7f58c605e640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f58c006d850 con 0x7f58c00b6850 2024-09-12T03:17:01.191 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.188+0000 7f58c605e640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f58c00d3fa0 con 0x7f58c00d3190 2024-09-12T03:17:01.191 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.188+0000 7f58bf7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f58c00b6470 0x7f58c00cf090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:01.191 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.188+0000 7f58bf7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f58c00b6470 0x7f58c00cf090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58048/0 (socket says 172.21.15.27:58048) 2024-09-12T03:17:01.191 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.188+0000 7f58bf7fe640 1 -- 172.21.15.27:0/1222276066 learned_addr learned my addr 172.21.15.27:0/1222276066 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:17:01.191 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.188+0000 7f58beffd640 1 --2- 172.21.15.27:0/1222276066 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f58c00b6850 0x7f58c015b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:01.191 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.188+0000 7f58bffff640 1 --2- 172.21.15.27:0/1222276066 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f58c00d3190 0x7f58c00ac9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:01.191 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.188+0000 7f58bf7fe640 1 -- 172.21.15.27:0/1222276066 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f58c00d3190 msgr2=0x7f58c00ac9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:01.191 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.188+0000 7f58bf7fe640 1 --2- 172.21.15.27:0/1222276066 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f58c00d3190 0x7f58c00ac9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:01.191 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.188+0000 7f58bf7fe640 1 -- 172.21.15.27:0/1222276066 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f58c00b6850 msgr2=0x7f58c015b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:01.191 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.188+0000 7f58bf7fe640 1 --2- 172.21.15.27:0/1222276066 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f58c00b6850 0x7f58c015b610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:01.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.188+0000 7f58bf7fe640 1 -- 172.21.15.27:0/1222276066 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58c00abcb0 con 0x7f58c00b6470 2024-09-12T03:17:01.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.188+0000 7f58bffff640 1 --2- 172.21.15.27:0/1222276066 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f58c00d3190 0x7f58c00ac9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:17:01.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.188+0000 7f58bf7fe640 1 --2- 172.21.15.27:0/1222276066 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f58c00b6470 0x7f58c00cf090 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f58b000c570 tx=0x7f58b000ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:01.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.188+0000 7f58a7fff640 1 -- 172.21.15.27:0/1222276066 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58b0015070 con 0x7f58c00b6470 2024-09-12T03:17:01.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.188+0000 7f58a7fff640 1 -- 172.21.15.27:0/1222276066 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58b0011800 con 0x7f58c00b6470 2024-09-12T03:17:01.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.189+0000 7f58c605e640 1 -- 172.21.15.27:0/1222276066 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58c00abf90 con 0x7f58c00b6470 2024-09-12T03:17:01.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.189+0000 7f58a7fff640 1 -- 172.21.15.27:0/1222276066 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58b0011bf0 con 0x7f58c00b6470 2024-09-12T03:17:01.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.189+0000 7f58c605e640 1 -- 172.21.15.27:0/1222276066 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58c00ac550 con 0x7f58c00b6470 2024-09-12T03:17:01.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.190+0000 7f58c605e640 1 -- 172.21.15.27:0/1222276066 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58c00d3600 con 0x7f58c00b6470 2024-09-12T03:17:01.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.190+0000 7f58a7fff640 1 -- 172.21.15.27:0/1222276066 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f58b0018070 con 0x7f58c00b6470 2024-09-12T03:17:01.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.190+0000 7f58a7fff640 1 --2- 172.21.15.27:0/1222276066 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f58ac06be30 0x7f58ac06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:01.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.191+0000 7f58a7fff640 1 -- 172.21.15.27:0/1222276066 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f58b008ab60 con 0x7f58c00b6470 2024-09-12T03:17:01.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.191+0000 7f58beffd640 1 --2- 172.21.15.27:0/1222276066 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f58ac06be30 0x7f58ac06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:01.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.191+0000 7f58beffd640 1 --2- 172.21.15.27:0/1222276066 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f58ac06be30 0x7f58ac06e2f0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f58c00b4270 tx=0x7f58b4008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:01.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.195+0000 7f58a7fff640 1 -- 172.21.15.27:0/1222276066 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f58b0058f50 con 0x7f58c00b6470 2024-09-12T03:17:01.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.374+0000 7f58c605e640 1 -- 172.21.15.27:0/1222276066 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f58c00cc380 con 0x7f58c00b6470 2024-09-12T03:17:01.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.375+0000 7f58a7fff640 1 -- 172.21.15.27:0/1222276066 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16617 (secure 0 0 0) 0x7f58b005d250 con 0x7f58c00b6470 2024-09-12T03:17:01.376 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:17:01.377 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":21,"fsid":"d2df504e-2647-4385-a5db-9fdf69f487a0","created":"2024-09-12T03:16:07.778937+0000","modified":"2024-09-12T03:16:32.570182+0000","last_up_change":"2024-09-12T03:16:19.314778+0000","last_in_change":"2024-09-12T03:16:09.893086+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T03:16:20.097646+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-12T03:16:28.790791+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-12T03:16:29.909065+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1851116c-1fb3-4dc9-9772-f90e2ef4c355","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6817","nonce":3002321488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6819","nonce":3002321488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6823","nonce":3002321488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6821","nonce":3002321488}]},"public_addr":"172.21.15.27:6817/3002321488","cluster_addr":"172.21.15.27:6819/3002321488","heartbeat_back_addr":"172.21.15.27:6823/3002321488","heartbeat_front_addr":"172.21.15.27:6821/3002321488","state":["exists","up"]},{"osd":1,"uuid":"f38ae17d-73eb-4204-b99d-975fa806fb87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6809","nonce":1613102886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6811","nonce":1613102886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6815","nonce":1613102886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6813","nonce":1613102886}]},"public_addr":"172.21.15.27:6809/1613102886","cluster_addr":"172.21.15.27:6811/1613102886","heartbeat_back_addr":"172.21.15.27:6815/1613102886","heartbeat_front_addr":"172.21.15.27:6813/1613102886","state":["exists","up"]},{"osd":2,"uuid":"fb12dcc8-bdd0-42b8-b46d-8b10106224bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6825","nonce":3534723355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6827","nonce":3534723355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6831","nonce":3534723355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6829","nonce":3534723355}]},"public_addr":"172.21.15.27:6825/3534723355","cluster_addr":"172.21.15.27:6827/3534723355","heartbeat_back_addr":"172.21.15.27:6831/3534723355","heartbeat_front_addr":"172.21.15.27:6829/3534723355","state":["exists","up"]},{"osd":3,"uuid":"d74d3f49-9c24-4b9f-98e2-3ecd7b5ef3b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6801","nonce":80661598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6803","nonce":80661598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6807","nonce":80661598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6805","nonce":80661598}]},"public_addr":"172.21.15.27:6801/80661598","cluster_addr":"172.21.15.27:6803/80661598","heartbeat_back_addr":"172.21.15.27:6807/80661598","heartbeat_front_addr":"172.21.15.27:6805/80661598","state":["exists","up"]},{"osd":4,"uuid":"2bf123f2-1b0d-41c4-aa3b-805bab267e59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6817","nonce":3943189827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6819","nonce":3943189827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6823","nonce":3943189827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6821","nonce":3943189827}]},"public_addr":"172.21.15.45:6817/3943189827","cluster_addr":"172.21.15.45:6819/3943189827","heartbeat_back_addr":"172.21.15.45:6823/3943189827","heartbeat_front_addr":"172.21.15.45:6821/3943189827","state":["exists","up"]},{"osd":5,"uuid":"c49ba5da-91c6-4a86-a97b-4aba89c253ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6809","nonce":3620760330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6811","nonce":3620760330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6815","nonce":3620760330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6813","nonce":3620760330}]},"public_addr":"172.21.15.45:6809/3620760330","cluster_addr":"172.21.15.45:6811/3620760330","heartbeat_back_addr":"172.21.15.45:6815/3620760330","heartbeat_front_addr":"172.21.15.45:6813/3620760330","state":["exists","up"]},{"osd":6,"uuid":"b0123475-89f7-4a34-9e33-d797b99dd9dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6801","nonce":1989083510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6803","nonce":1989083510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6807","nonce":1989083510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6805","nonce":1989083510}]},"public_addr":"172.21.15.45:6801/1989083510","cluster_addr":"172.21.15.45:6803/1989083510","heartbeat_back_addr":"172.21.15.45:6807/1989083510","heartbeat_front_addr":"172.21.15.45:6805/1989083510","state":["exists","up"]},{"osd":7,"uuid":"4dad773d-b7d8-46fc-8a10-a972aad27b29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6825","nonce":1513300294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6827","nonce":1513300294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6831","nonce":1513300294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6829","nonce":1513300294}]},"public_addr":"172.21.15.45:6825/1513300294","cluster_addr":"172.21.15.45:6827/1513300294","heartbeat_back_addr":"172.21.15.45:6831/1513300294","heartbeat_front_addr":"172.21.15.45:6829/1513300294","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.881855+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:17.133252+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.735339+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.500742+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.477571+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:17.167667+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.470291+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.873743+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T03:17:01.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.377+0000 7f58c605e640 1 -- 172.21.15.27:0/1222276066 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f58ac06be30 msgr2=0x7f58ac06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:01.379 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.377+0000 7f58c605e640 1 --2- 172.21.15.27:0/1222276066 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f58ac06be30 0x7f58ac06e2f0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f58c00b4270 tx=0x7f58b4008040 comp rx=0 tx=0).stop 2024-09-12T03:17:01.379 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.377+0000 7f58c605e640 1 -- 172.21.15.27:0/1222276066 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f58c00b6470 msgr2=0x7f58c00cf090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:01.379 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.377+0000 7f58c605e640 1 --2- 172.21.15.27:0/1222276066 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f58c00b6470 0x7f58c00cf090 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f58b000c570 tx=0x7f58b000ca30 comp rx=0 tx=0).stop 2024-09-12T03:17:01.379 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.377+0000 7f58c605e640 1 -- 172.21.15.27:0/1222276066 shutdown_connections 2024-09-12T03:17:01.379 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.377+0000 7f58c605e640 1 --2- 172.21.15.27:0/1222276066 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f58ac06be30 0x7f58ac06e2f0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:01.379 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.377+0000 7f58c605e640 1 --2- 172.21.15.27:0/1222276066 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f58c00d3190 0x7f58c00ac9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:01.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.377+0000 7f58c605e640 1 --2- 172.21.15.27:0/1222276066 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f58c00b6850 0x7f58c015b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:01.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.377+0000 7f58c605e640 1 --2- 172.21.15.27:0/1222276066 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f58c00b6470 0x7f58c00cf090 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:01.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.377+0000 7f58c605e640 1 -- 172.21.15.27:0/1222276066 >> 172.21.15.27:0/1222276066 conn(0x7f58c00d1200 msgr2=0x7f58c00d2710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:01.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.377+0000 7f58c605e640 1 -- 172.21.15.27:0/1222276066 shutdown_connections 2024-09-12T03:17:01.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.378+0000 7f58c605e640 1 -- 172.21.15.27:0/1222276066 wait complete. 2024-09-12T03:17:01.389 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-12T03:17:01.493 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.493+0000 7f7121853640 1 Processor -- start 2024-09-12T03:17:01.494 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.493+0000 7f7121853640 1 -- start start 2024-09-12T03:17:01.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.494+0000 7f7121853640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f711c073250 0x7f711c0736c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:01.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.494+0000 7f7121853640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f711c15cbe0 0x7f711c15cff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:01.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.494+0000 7f7121853640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f711c0d1e30 con 0x7f711c146020 2024-09-12T03:17:01.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.494+0000 7f7121853640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f711c073c00 con 0x7f711c073250 2024-09-12T03:17:01.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.494+0000 7f7121853640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f711c146400 con 0x7f711c15cbe0 2024-09-12T03:17:01.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.494+0000 7f711a7fc640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f711c073250 0x7f711c0736c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:01.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.494+0000 7f711a7fc640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f711c073250 0x7f711c0736c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58058/0 (socket says 172.21.15.27:58058) 2024-09-12T03:17:01.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.494+0000 7f711a7fc640 1 -- 172.21.15.27:0/902895375 learned_addr learned my addr 172.21.15.27:0/902895375 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:17:01.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.494+0000 7f711b7fe640 1 --2- 172.21.15.27:0/902895375 >> v2:172.21.15.45:3300/0 conn(0x7f711c15cbe0 0x7f711c15cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:01.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.494+0000 7f711a7fc640 1 -- 172.21.15.27:0/902895375 >> v2:172.21.15.45:3300/0 conn(0x7f711c15cbe0 msgr2=0x7f711c15cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:01.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.494+0000 7f711a7fc640 1 --2- 172.21.15.27:0/902895375 >> v2:172.21.15.45:3300/0 conn(0x7f711c15cbe0 0x7f711c15cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:01.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.494+0000 7f711a7fc640 1 -- 172.21.15.27:0/902895375 >> v1:172.21.15.45:6789/0 conn(0x7f711c146020 legacy=0x7f711c0d1720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:01.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.494+0000 7f711a7fc640 1 -- 172.21.15.27:0/902895375 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f711c0d2100 con 0x7f711c073250 2024-09-12T03:17:01.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.495+0000 7f711a7fc640 1 --2- 172.21.15.27:0/902895375 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f711c073250 0x7f711c0736c0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f711c0b1b30 tx=0x7f711002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77959c962061c844 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:01.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.495+0000 7f7119ffb640 1 -- 172.21.15.27:0/902895375 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7110037070 con 0x7f711c073250 2024-09-12T03:17:01.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.495+0000 7f7119ffb640 1 -- 172.21.15.27:0/902895375 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7110032440 con 0x7f711c073250 2024-09-12T03:17:01.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.495+0000 7f7119ffb640 1 -- 172.21.15.27:0/902895375 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7110032810 con 0x7f711c073250 2024-09-12T03:17:01.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.496+0000 7f7121853640 1 -- 172.21.15.27:0/902895375 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f711c073250 msgr2=0x7f711c0736c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:01.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.496+0000 7f7121853640 1 --2- 172.21.15.27:0/902895375 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f711c073250 0x7f711c0736c0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f711c0b1b30 tx=0x7f711002a270 comp rx=0 tx=0).stop 2024-09-12T03:17:01.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.496+0000 7f7121853640 1 -- 172.21.15.27:0/902895375 shutdown_connections 2024-09-12T03:17:01.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.496+0000 7f7121853640 1 --2- 172.21.15.27:0/902895375 >> v2:172.21.15.45:3300/0 conn(0x7f711c15cbe0 0x7f711c15cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:01.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.496+0000 7f7121853640 1 --2- 172.21.15.27:0/902895375 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f711c073250 0x7f711c0736c0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:01.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.496+0000 7f7121853640 1 -- 172.21.15.27:0/902895375 >> 172.21.15.27:0/902895375 conn(0x7f711c06f300 msgr2=0x7f711c06f710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:01.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.496+0000 7f7121853640 1 -- 172.21.15.27:0/902895375 shutdown_connections 2024-09-12T03:17:01.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.496+0000 7f7121853640 1 -- 172.21.15.27:0/902895375 wait complete. 2024-09-12T03:17:01.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.497+0000 7f7121853640 1 Processor -- start 2024-09-12T03:17:01.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.497+0000 7f7121853640 1 -- start start 2024-09-12T03:17:01.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.498+0000 7f7121853640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f711c073250 0x7f711c13a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:01.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.498+0000 7f7121853640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f711c146020 0x7f711c140690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:01.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.498+0000 7f7121853640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f711c15cbe0 0x7f711c1f4f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:01.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.498+0000 7f7121853640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f711c0b7570 con 0x7f711c073250 2024-09-12T03:17:01.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.498+0000 7f7121853640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f711c0b7290 con 0x7f711c146020 2024-09-12T03:17:01.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.498+0000 7f7121853640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f711c0b6c60 con 0x7f711c15cbe0 2024-09-12T03:17:01.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.498+0000 7f711affd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f711c073250 0x7f711c13a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:01.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.498+0000 7f711affd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f711c073250 0x7f711c13a7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58062/0 (socket says 172.21.15.27:58062) 2024-09-12T03:17:01.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.498+0000 7f711affd640 1 -- 172.21.15.27:0/2532370973 learned_addr learned my addr 172.21.15.27:0/2532370973 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:17:01.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.498+0000 7f711affd640 1 -- 172.21.15.27:0/2532370973 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f711c15cbe0 msgr2=0x7f711c1f4f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:01.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.498+0000 7f711affd640 1 --2- 172.21.15.27:0/2532370973 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f711c15cbe0 0x7f711c1f4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:01.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.498+0000 7f711affd640 1 -- 172.21.15.27:0/2532370973 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f711c146020 msgr2=0x7f711c140690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:01.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.498+0000 7f711affd640 1 --2- 172.21.15.27:0/2532370973 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f711c146020 0x7f711c140690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:01.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.498+0000 7f711affd640 1 -- 172.21.15.27:0/2532370973 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f711c13eba0 con 0x7f711c073250 2024-09-12T03:17:01.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.499+0000 7f711affd640 1 --2- 172.21.15.27:0/2532370973 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f711c073250 0x7f711c13a7f0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f710c00c400 tx=0x7f710c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:01.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.499+0000 7f7103fff640 1 -- 172.21.15.27:0/2532370973 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f710c0052d0 con 0x7f711c073250 2024-09-12T03:17:01.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.499+0000 7f7103fff640 1 -- 172.21.15.27:0/2532370973 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f710c005490 con 0x7f711c073250 2024-09-12T03:17:01.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.499+0000 7f7121853640 1 -- 172.21.15.27:0/2532370973 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f711c13ee80 con 0x7f711c073250 2024-09-12T03:17:01.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.499+0000 7f7103fff640 1 -- 172.21.15.27:0/2532370973 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f710c007260 con 0x7f711c073250 2024-09-12T03:17:01.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.499+0000 7f7121853640 1 -- 172.21.15.27:0/2532370973 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f711c13f440 con 0x7f711c073250 2024-09-12T03:17:01.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.500+0000 7f7121853640 1 -- 172.21.15.27:0/2532370973 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f711c15eac0 con 0x7f711c073250 2024-09-12T03:17:01.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.500+0000 7f7103fff640 1 -- 172.21.15.27:0/2532370973 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f710c007400 con 0x7f711c073250 2024-09-12T03:17:01.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.501+0000 7f7103fff640 1 --2- 172.21.15.27:0/2532370973 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f710806be30 0x7f710806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:01.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.501+0000 7f7103fff640 1 -- 172.21.15.27:0/2532370973 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f710c08a3a0 con 0x7f711c073250 2024-09-12T03:17:01.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.501+0000 7f711a7fc640 1 --2- 172.21.15.27:0/2532370973 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f710806be30 0x7f710806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:01.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.502+0000 7f711a7fc640 1 --2- 172.21.15.27:0/2532370973 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f710806be30 0x7f710806e2f0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f7110034000 tx=0x7f711002c000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:01.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.505+0000 7f7103fff640 1 -- 172.21.15.27:0/2532370973 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f710c058d50 con 0x7f711c073250 2024-09-12T03:17:01.718 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.718+0000 7f7121853640 1 -- 172.21.15.27:0/2532370973 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f711c069790 con 0x7f711c073250 2024-09-12T03:17:01.719 INFO:tasks.ceph.mon.a.smithi027.stderr:2024-09-12T03:17:01.719+0000 7f959bfd9640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-12T03:17:01.720 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.720+0000 7f7103fff640 1 -- 172.21.15.27:0/2532370973 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v7) ==== 161+0+0 (secure 0 0 0) 0x7f710c058650 con 0x7f711c073250 2024-09-12T03:17:01.720 INFO:teuthology.orchestra.run.smithi027.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-12T03:17:01.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.722+0000 7f7121853640 1 -- 172.21.15.27:0/2532370973 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f710806be30 msgr2=0x7f710806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:01.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.722+0000 7f7121853640 1 --2- 172.21.15.27:0/2532370973 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f710806be30 0x7f710806e2f0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f7110034000 tx=0x7f711002c000 comp rx=0 tx=0).stop 2024-09-12T03:17:01.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.722+0000 7f7121853640 1 -- 172.21.15.27:0/2532370973 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f711c073250 msgr2=0x7f711c13a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:01.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.722+0000 7f7121853640 1 --2- 172.21.15.27:0/2532370973 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f711c073250 0x7f711c13a7f0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f710c00c400 tx=0x7f710c00c8c0 comp rx=0 tx=0).stop 2024-09-12T03:17:01.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.722+0000 7f7121853640 1 -- 172.21.15.27:0/2532370973 shutdown_connections 2024-09-12T03:17:01.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.722+0000 7f7121853640 1 --2- 172.21.15.27:0/2532370973 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f710806be30 0x7f710806e2f0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:01.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.722+0000 7f7121853640 1 --2- 172.21.15.27:0/2532370973 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f711c15cbe0 0x7f711c1f4f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:01.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.722+0000 7f7121853640 1 --2- 172.21.15.27:0/2532370973 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f711c146020 0x7f711c140690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:01.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.722+0000 7f7121853640 1 --2- 172.21.15.27:0/2532370973 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f711c073250 0x7f711c13a7f0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:01.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.723+0000 7f7121853640 1 -- 172.21.15.27:0/2532370973 >> 172.21.15.27:0/2532370973 conn(0x7f711c06f300 msgr2=0x7f711c15efc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:01.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.723+0000 7f7121853640 1 -- 172.21.15.27:0/2532370973 shutdown_connections 2024-09-12T03:17:01.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.723+0000 7f7121853640 1 -- 172.21.15.27:0/2532370973 wait complete. 2024-09-12T03:17:01.728 INFO:tasks.ceph.mds.d.smithi045.stderr: -6658> 2024-09-12T03:16:37.545+0000 7f8cfe70b640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-12T03:17:01.734 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-12T03:17:01.763 INFO:tasks.ceph.mds.d.smithi045.stdout:starting mds.d at 2024-09-12T03:17:01.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.840+0000 7f4953ced640 1 Processor -- start 2024-09-12T03:17:01.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.840+0000 7f4953ced640 1 -- start start 2024-09-12T03:17:01.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.841+0000 7f4953ced640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f494c0b6240 0x7f494c0b6620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:01.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.841+0000 7f4953ced640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f494c0d3c20 0x7f494c0d4040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:01.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.841+0000 7f4953ced640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f494c0b6b60 con 0x7f494c06d960 2024-09-12T03:17:01.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.841+0000 7f4953ced640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f494c0d2930 con 0x7f494c0d3c20 2024-09-12T03:17:01.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.841+0000 7f4953ced640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f494c0d2c10 con 0x7f494c0b6240 2024-09-12T03:17:01.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.841+0000 7f4951288640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f494c0d3c20 0x7f494c0d4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:01.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.841+0000 7f495228a640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f494c06d960 0x7f494c06de10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:44336/0 (socket says 172.21.15.27:44336) 2024-09-12T03:17:01.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.841+0000 7f495228a640 1 -- 172.21.15.27:0/3657732196 learned_addr learned my addr 172.21.15.27:0/3657732196 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:17:01.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.841+0000 7f4951a89640 1 --2- 172.21.15.27:0/3657732196 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f494c0b6240 0x7f494c0b6620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:01.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.841+0000 7f4951288640 1 -- 172.21.15.27:0/3657732196 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f494c0b6240 msgr2=0x7f494c0b6620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:01.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.841+0000 7f4951288640 1 --2- 172.21.15.27:0/3657732196 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f494c0b6240 0x7f494c0b6620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:01.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.841+0000 7f4951288640 1 -- 172.21.15.27:0/3657732196 >> v1:172.21.15.27:6789/0 conn(0x7f494c06d960 legacy=0x7f494c06de10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:01.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.841+0000 7f4951288640 1 -- 172.21.15.27:0/3657732196 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f494c0d2ef0 con 0x7f494c0d3c20 2024-09-12T03:17:01.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.841+0000 7f4950a87640 1 -- 172.21.15.27:0/3657732196 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1890918327 0 0) 0x7f494c0b6b60 con 0x7f494c06d960 2024-09-12T03:17:01.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.842+0000 7f4951288640 1 --2- 172.21.15.27:0/3657732196 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f494c0d3c20 0x7f494c0d4040 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f49480097a0 tx=0x7f494802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb7f6cfce79dc5a6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:01.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.842+0000 7f4950a87640 1 -- 172.21.15.27:0/3657732196 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4948037070 con 0x7f494c0d3c20 2024-09-12T03:17:01.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.842+0000 7f4950a87640 1 -- 172.21.15.27:0/3657732196 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4948032440 con 0x7f494c0d3c20 2024-09-12T03:17:01.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.842+0000 7f4950a87640 1 -- 172.21.15.27:0/3657732196 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4948032810 con 0x7f494c0d3c20 2024-09-12T03:17:01.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.843+0000 7f4953ced640 1 -- 172.21.15.27:0/3657732196 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f494c0d3c20 msgr2=0x7f494c0d4040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:01.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.843+0000 7f4953ced640 1 --2- 172.21.15.27:0/3657732196 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f494c0d3c20 0x7f494c0d4040 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f49480097a0 tx=0x7f494802a210 comp rx=0 tx=0).stop 2024-09-12T03:17:01.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.843+0000 7f4953ced640 1 -- 172.21.15.27:0/3657732196 shutdown_connections 2024-09-12T03:17:01.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.843+0000 7f4953ced640 1 --2- 172.21.15.27:0/3657732196 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f494c0d3c20 0x7f494c0d4040 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:01.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.843+0000 7f4953ced640 1 --2- 172.21.15.27:0/3657732196 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f494c0b6240 0x7f494c0b6620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:01.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.843+0000 7f4953ced640 1 -- 172.21.15.27:0/3657732196 >> 172.21.15.27:0/3657732196 conn(0x7f494c0d1420 msgr2=0x7f494c0d1830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:01.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.843+0000 7f4953ced640 1 -- 172.21.15.27:0/3657732196 shutdown_connections 2024-09-12T03:17:01.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.843+0000 7f4953ced640 1 -- 172.21.15.27:0/3657732196 wait complete. 2024-09-12T03:17:01.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.844+0000 7f4953ced640 1 Processor -- start 2024-09-12T03:17:01.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.844+0000 7f4953ced640 1 -- start start 2024-09-12T03:17:01.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.845+0000 7f4953ced640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f494c06d960 0x7f494c1f1550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:01.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.845+0000 7f4953ced640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f494c0b6240 0x7f494c1f4820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:01.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.845+0000 7f4953ced640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f494c0d3c20 0x7f494c0c82f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:01.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.845+0000 7f4953ced640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f494c15ece0 con 0x7f494c0d3c20 2024-09-12T03:17:01.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.845+0000 7f4953ced640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f494c15eb60 con 0x7f494c06d960 2024-09-12T03:17:01.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.845+0000 7f4953ced640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f494c0cfc70 con 0x7f494c0b6240 2024-09-12T03:17:01.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.845+0000 7f495228a640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f494c0d3c20 0x7f494c0c82f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:01.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.845+0000 7f495228a640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f494c0d3c20 0x7f494c0c82f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58076/0 (socket says 172.21.15.27:58076) 2024-09-12T03:17:01.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.845+0000 7f495228a640 1 -- 172.21.15.27:0/2517901332 learned_addr learned my addr 172.21.15.27:0/2517901332 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:17:01.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.845+0000 7f4951a89640 1 --2- 172.21.15.27:0/2517901332 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f494c06d960 0x7f494c1f1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:01.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.845+0000 7f4951288640 1 --2- 172.21.15.27:0/2517901332 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f494c0b6240 0x7f494c1f4820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:01.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.845+0000 7f495228a640 1 -- 172.21.15.27:0/2517901332 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f494c0b6240 msgr2=0x7f494c1f4820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:01.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.845+0000 7f495228a640 1 --2- 172.21.15.27:0/2517901332 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f494c0b6240 0x7f494c1f4820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:01.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.845+0000 7f495228a640 1 -- 172.21.15.27:0/2517901332 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f494c06d960 msgr2=0x7f494c1f1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:01.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.845+0000 7f495228a640 1 --2- 172.21.15.27:0/2517901332 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f494c06d960 0x7f494c1f1550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:01.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.845+0000 7f495228a640 1 -- 172.21.15.27:0/2517901332 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f494c1f1a90 con 0x7f494c0d3c20 2024-09-12T03:17:01.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.845+0000 7f495228a640 1 --2- 172.21.15.27:0/2517901332 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f494c0d3c20 0x7f494c0c82f0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f494400c7d0 tx=0x7f494400cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:01.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.845+0000 7f49427fc640 1 -- 172.21.15.27:0/2517901332 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4944018070 con 0x7f494c0d3c20 2024-09-12T03:17:01.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.845+0000 7f49427fc640 1 -- 172.21.15.27:0/2517901332 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f494400a640 con 0x7f494c0d3c20 2024-09-12T03:17:01.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.845+0000 7f4953ced640 1 -- 172.21.15.27:0/2517901332 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f494c0ca660 con 0x7f494c0d3c20 2024-09-12T03:17:01.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.846+0000 7f49427fc640 1 -- 172.21.15.27:0/2517901332 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f494400aa30 con 0x7f494c0d3c20 2024-09-12T03:17:01.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.846+0000 7f4953ced640 1 -- 172.21.15.27:0/2517901332 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f494c0cac20 con 0x7f494c0d3c20 2024-09-12T03:17:01.860 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.846+0000 7f4953ced640 1 -- 172.21.15.27:0/2517901332 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f494c0d4040 con 0x7f494c0d3c20 2024-09-12T03:17:01.860 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.847+0000 7f49427fc640 1 -- 172.21.15.27:0/2517901332 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f494401b070 con 0x7f494c0d3c20 2024-09-12T03:17:01.860 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.847+0000 7f49427fc640 1 --2- 172.21.15.27:0/2517901332 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f493006be30 0x7f493006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:01.860 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.848+0000 7f49427fc640 1 -- 172.21.15.27:0/2517901332 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5795+0+0 (secure 0 0 0) 0x7f494408a460 con 0x7f494c0d3c20 2024-09-12T03:17:01.860 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.848+0000 7f4951a89640 1 --2- 172.21.15.27:0/2517901332 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f493006be30 0x7f493006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:01.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.848+0000 7f4951a89640 1 --2- 172.21.15.27:0/2517901332 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f493006be30 0x7f493006e2f0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f493c006fd0 tx=0x7f493c008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:01.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:01.852+0000 7f49427fc640 1 -- 172.21.15.27:0/2517901332 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f4944058db0 con 0x7f494c0d3c20 2024-09-12T03:17:02.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:02.066+0000 7f4953ced640 1 -- 172.21.15.27:0/2517901332 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f494c0b6960 con 0x7f494c0d3c20 2024-09-12T03:17:02.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:02.724+0000 7f49427fc640 1 -- 172.21.15.27:0/2517901332 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v8) ==== 104+0+0 (secure 0 0 0) 0x7f49440586b0 con 0x7f494c0d3c20 2024-09-12T03:17:02.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:02.727+0000 7f4953ced640 1 -- 172.21.15.27:0/2517901332 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f493006be30 msgr2=0x7f493006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:02.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:02.727+0000 7f4953ced640 1 --2- 172.21.15.27:0/2517901332 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f493006be30 0x7f493006e2f0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f493c006fd0 tx=0x7f493c008040 comp rx=0 tx=0).stop 2024-09-12T03:17:02.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:02.727+0000 7f4953ced640 1 -- 172.21.15.27:0/2517901332 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f494c0d3c20 msgr2=0x7f494c0c82f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:02.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:02.727+0000 7f4953ced640 1 --2- 172.21.15.27:0/2517901332 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f494c0d3c20 0x7f494c0c82f0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f494400c7d0 tx=0x7f494400cc90 comp rx=0 tx=0).stop 2024-09-12T03:17:02.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:02.727+0000 7f4953ced640 1 -- 172.21.15.27:0/2517901332 shutdown_connections 2024-09-12T03:17:02.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:02.727+0000 7f4953ced640 1 --2- 172.21.15.27:0/2517901332 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f493006be30 0x7f493006e2f0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:02.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:02.727+0000 7f4953ced640 1 --2- 172.21.15.27:0/2517901332 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f494c0d3c20 0x7f494c0c82f0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:02.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:02.727+0000 7f4953ced640 1 --2- 172.21.15.27:0/2517901332 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f494c0b6240 0x7f494c1f4820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:02.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:02.727+0000 7f4953ced640 1 --2- 172.21.15.27:0/2517901332 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f494c06d960 0x7f494c1f1550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:02.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:02.727+0000 7f4953ced640 1 -- 172.21.15.27:0/2517901332 >> 172.21.15.27:0/2517901332 conn(0x7f494c0d1420 msgr2=0x7f494c0d1800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:02.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:02.727+0000 7f4953ced640 1 -- 172.21.15.27:0/2517901332 shutdown_connections 2024-09-12T03:17:02.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:02.727+0000 7f4953ced640 1 -- 172.21.15.27:0/2517901332 wait complete. 2024-09-12T03:17:02.741 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-12T03:17:02.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:02.854+0000 7f45f9cba640 1 Processor -- start 2024-09-12T03:17:02.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:02.855+0000 7f45f9cba640 1 -- start start 2024-09-12T03:17:02.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:02.855+0000 7f45f9cba640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f45f4158df0 0x7f45f40b5150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:02.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:02.855+0000 7f45f9cba640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f45f4072d50 0x7f45f406ff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:02.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:02.855+0000 7f45f9cba640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f45f415c890 con 0x7f45f415c420 2024-09-12T03:17:02.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:02.855+0000 7f45f9cba640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f45f4070470 con 0x7f45f4158df0 2024-09-12T03:17:02.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:02.855+0000 7f45f9cba640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f45f40706d0 con 0x7f45f4072d50 2024-09-12T03:17:02.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:02.856+0000 7f45f37fe640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f45f4158df0 0x7f45f40b5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:02.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:02.856+0000 7f45f2ffd640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f45f415c420 0x7f45f4072640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:44342/0 (socket says 172.21.15.27:44342) 2024-09-12T03:17:02.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:02.856+0000 7f45f2ffd640 1 -- 172.21.15.27:0/3733966371 learned_addr learned my addr 172.21.15.27:0/3733966371 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:17:02.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:02.856+0000 7f45f3fff640 1 --2- 172.21.15.27:0/3733966371 >> v2:172.21.15.45:3300/0 conn(0x7f45f4072d50 0x7f45f406ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:02.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:02.856+0000 7f45f37fe640 1 -- 172.21.15.27:0/3733966371 >> v2:172.21.15.45:3300/0 conn(0x7f45f4072d50 msgr2=0x7f45f406ff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:02.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:02.856+0000 7f45f37fe640 1 --2- 172.21.15.27:0/3733966371 >> v2:172.21.15.45:3300/0 conn(0x7f45f4072d50 0x7f45f406ff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:02.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:02.856+0000 7f45f37fe640 1 -- 172.21.15.27:0/3733966371 >> v1:172.21.15.27:6789/0 conn(0x7f45f415c420 legacy=0x7f45f4072640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:02.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:02.856+0000 7f45f37fe640 1 -- 172.21.15.27:0/3733966371 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45f40709b0 con 0x7f45f4158df0 2024-09-12T03:17:02.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:02.856+0000 7f45f27fc640 1 -- 172.21.15.27:0/3733966371 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3737212121 0 0) 0x7f45f415c890 con 0x7f45f415c420 2024-09-12T03:17:02.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:02.857+0000 7f45f37fe640 1 --2- 172.21.15.27:0/3733966371 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f45f4158df0 0x7f45f40b5150 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f45f4070cc0 tx=0x7f45e402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47d881030e9b2429 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:02.862 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:02.857+0000 7f45f27fc640 1 -- 172.21.15.27:0/3733966371 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45e4037070 con 0x7f45f4158df0 2024-09-12T03:17:02.862 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:02.857+0000 7f45f27fc640 1 -- 172.21.15.27:0/3733966371 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45e4032440 con 0x7f45f4158df0 2024-09-12T03:17:02.862 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:02.857+0000 7f45f27fc640 1 -- 172.21.15.27:0/3733966371 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45e4032810 con 0x7f45f4158df0 2024-09-12T03:17:02.862 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:02.857+0000 7f45f9cba640 1 -- 172.21.15.27:0/3733966371 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f45f4158df0 msgr2=0x7f45f40b5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:02.862 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:02.857+0000 7f45f9cba640 1 --2- 172.21.15.27:0/3733966371 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f45f4158df0 0x7f45f40b5150 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f45f4070cc0 tx=0x7f45e402a1a0 comp rx=0 tx=0).stop 2024-09-12T03:17:02.862 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:02.857+0000 7f45f9cba640 1 -- 172.21.15.27:0/3733966371 shutdown_connections 2024-09-12T03:17:02.862 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:02.857+0000 7f45f9cba640 1 --2- 172.21.15.27:0/3733966371 >> v2:172.21.15.45:3300/0 conn(0x7f45f4072d50 0x7f45f406ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:02.862 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:02.857+0000 7f45f9cba640 1 --2- 172.21.15.27:0/3733966371 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f45f4158df0 0x7f45f40b5150 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:02.862 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:02.857+0000 7f45f9cba640 1 -- 172.21.15.27:0/3733966371 >> 172.21.15.27:0/3733966371 conn(0x7f45f406d060 msgr2=0x7f45f406d470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:02.862 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:02.857+0000 7f45f9cba640 1 -- 172.21.15.27:0/3733966371 shutdown_connections 2024-09-12T03:17:02.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:02.858+0000 7f45f9cba640 1 -- 172.21.15.27:0/3733966371 wait complete. 2024-09-12T03:17:02.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:02.859+0000 7f45f9cba640 1 Processor -- start 2024-09-12T03:17:02.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:02.859+0000 7f45f9cba640 1 -- start start 2024-09-12T03:17:02.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:02.860+0000 7f45f9cba640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f45f4072d50 0x7f45f414a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:02.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:02.860+0000 7f45f9cba640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f45f4158df0 0x7f45f40adc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:02.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:02.860+0000 7f45f9cba640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f45f415c420 0x7f45f40aeac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:02.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:02.860+0000 7f45f9cba640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f45f40b4c30 con 0x7f45f4158df0 2024-09-12T03:17:02.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:02.860+0000 7f45f9cba640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f45f40b4950 con 0x7f45f415c420 2024-09-12T03:17:02.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:02.860+0000 7f45f9cba640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f45f41591d0 con 0x7f45f4072d50 2024-09-12T03:17:02.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:02.860+0000 7f45f2ffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f45f4158df0 0x7f45f40adc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:02.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:02.860+0000 7f45f2ffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f45f4158df0 0x7f45f40adc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58096/0 (socket says 172.21.15.27:58096) 2024-09-12T03:17:02.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:02.860+0000 7f45f2ffd640 1 -- 172.21.15.27:0/1198420458 learned_addr learned my addr 172.21.15.27:0/1198420458 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:17:02.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:02.860+0000 7f45f37fe640 1 --2- 172.21.15.27:0/1198420458 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f45f4072d50 0x7f45f414a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:02.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:02.860+0000 7f45f3fff640 1 --2- 172.21.15.27:0/1198420458 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f45f415c420 0x7f45f40aeac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:02.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:02.860+0000 7f45f2ffd640 1 -- 172.21.15.27:0/1198420458 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f45f4072d50 msgr2=0x7f45f414a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:02.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:02.860+0000 7f45f2ffd640 1 --2- 172.21.15.27:0/1198420458 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f45f4072d50 0x7f45f414a050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:02.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:02.860+0000 7f45f2ffd640 1 -- 172.21.15.27:0/1198420458 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f45f415c420 msgr2=0x7f45f40aeac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:02.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:02.860+0000 7f45f2ffd640 1 --2- 172.21.15.27:0/1198420458 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f45f415c420 0x7f45f40aeac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:02.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:02.860+0000 7f45f2ffd640 1 -- 172.21.15.27:0/1198420458 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45f40acbe0 con 0x7f45f4158df0 2024-09-12T03:17:02.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:02.860+0000 7f45f2ffd640 1 --2- 172.21.15.27:0/1198420458 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f45f4158df0 0x7f45f40adc80 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f45e800c460 tx=0x7f45e800c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:02.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:02.860+0000 7f45dbfff640 1 -- 172.21.15.27:0/1198420458 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45e8018070 con 0x7f45f4158df0 2024-09-12T03:17:02.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:02.860+0000 7f45dbfff640 1 -- 172.21.15.27:0/1198420458 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45e8007500 con 0x7f45f4158df0 2024-09-12T03:17:02.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:02.860+0000 7f45f9cba640 1 -- 172.21.15.27:0/1198420458 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45f40acec0 con 0x7f45f4158df0 2024-09-12T03:17:02.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:02.860+0000 7f45dbfff640 1 -- 172.21.15.27:0/1198420458 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45e8004a20 con 0x7f45f4158df0 2024-09-12T03:17:02.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:02.861+0000 7f45f9cba640 1 -- 172.21.15.27:0/1198420458 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45f40ad480 con 0x7f45f4158df0 2024-09-12T03:17:02.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:02.861+0000 7f45f9cba640 1 -- 172.21.15.27:0/1198420458 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45f406ff30 con 0x7f45f4158df0 2024-09-12T03:17:02.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:02.862+0000 7f45dbfff640 1 -- 172.21.15.27:0/1198420458 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f45e8004bc0 con 0x7f45f4158df0 2024-09-12T03:17:02.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:02.862+0000 7f45dbfff640 1 --2- 172.21.15.27:0/1198420458 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f45e006be60 0x7f45e006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:02.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:02.863+0000 7f45dbfff640 1 -- 172.21.15.27:0/1198420458 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5795+0+0 (secure 0 0 0) 0x7f45e808a190 con 0x7f45f4158df0 2024-09-12T03:17:02.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:02.863+0000 7f45f37fe640 1 --2- 172.21.15.27:0/1198420458 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f45e006be60 0x7f45e006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:02.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:02.863+0000 7f45f37fe640 1 --2- 172.21.15.27:0/1198420458 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f45e006be60 0x7f45e006e320 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f45e4034000 tx=0x7f45e402e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:02.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:02.867+0000 7f45dbfff640 1 -- 172.21.15.27:0/1198420458 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f45e8058410 con 0x7f45f4158df0 2024-09-12T03:17:03.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:03.041+0000 7f45f9cba640 1 -- 172.21.15.27:0/1198420458 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f45f40b5ad0 con 0x7f45f4158df0 2024-09-12T03:17:03.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:03.726+0000 7f45dbfff640 1 -- 172.21.15.27:0/1198420458 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v23) ==== 181+0+0 (secure 0 0 0) 0x7f45e805c710 con 0x7f45f4158df0 2024-09-12T03:17:03.726 INFO:teuthology.orchestra.run.smithi027.stderr:pool 'cephfs_metadata' removed 2024-09-12T03:17:03.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:03.729+0000 7f45f9cba640 1 -- 172.21.15.27:0/1198420458 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f45e006be60 msgr2=0x7f45e006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:03.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:03.729+0000 7f45f9cba640 1 --2- 172.21.15.27:0/1198420458 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f45e006be60 0x7f45e006e320 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f45e4034000 tx=0x7f45e402e000 comp rx=0 tx=0).stop 2024-09-12T03:17:03.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:03.729+0000 7f45f9cba640 1 -- 172.21.15.27:0/1198420458 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f45f4158df0 msgr2=0x7f45f40adc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:03.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:03.729+0000 7f45f9cba640 1 --2- 172.21.15.27:0/1198420458 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f45f4158df0 0x7f45f40adc80 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f45e800c460 tx=0x7f45e800c920 comp rx=0 tx=0).stop 2024-09-12T03:17:03.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:03.729+0000 7f45f9cba640 1 -- 172.21.15.27:0/1198420458 shutdown_connections 2024-09-12T03:17:03.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:03.729+0000 7f45f9cba640 1 --2- 172.21.15.27:0/1198420458 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f45e006be60 0x7f45e006e320 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:03.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:03.729+0000 7f45f9cba640 1 --2- 172.21.15.27:0/1198420458 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f45f415c420 0x7f45f40aeac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:03.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:03.729+0000 7f45f9cba640 1 --2- 172.21.15.27:0/1198420458 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f45f4158df0 0x7f45f40adc80 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:03.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:03.729+0000 7f45f9cba640 1 --2- 172.21.15.27:0/1198420458 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f45f4072d50 0x7f45f414a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:03.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:03.729+0000 7f45f9cba640 1 -- 172.21.15.27:0/1198420458 >> 172.21.15.27:0/1198420458 conn(0x7f45f406d060 msgr2=0x7f45f415f7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:03.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:03.730+0000 7f45f9cba640 1 -- 172.21.15.27:0/1198420458 shutdown_connections 2024-09-12T03:17:03.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:03.730+0000 7f45f9cba640 1 -- 172.21.15.27:0/1198420458 wait complete. 2024-09-12T03:17:03.744 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-12T03:17:03.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:03.855+0000 7fbae6df8640 1 Processor -- start 2024-09-12T03:17:03.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:03.855+0000 7fbae6df8640 1 -- start start 2024-09-12T03:17:03.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:03.856+0000 7fbae6df8640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fbae00d4040 0x7fbae00b7ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:03.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:03.856+0000 7fbae6df8640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbae00d2930 con 0x7fbae00b6240 2024-09-12T03:17:03.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:03.856+0000 7fbae6df8640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbae00d2c10 con 0x7fbae00d3c20 2024-09-12T03:17:03.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:03.856+0000 7fbae6df8640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fbae00d2ef0 con 0x7fbae00d4040 2024-09-12T03:17:03.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:03.856+0000 7fbae4b94640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fbae00b6240 0x7fbae00b6620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:44352/0 (socket says 172.21.15.27:44352) 2024-09-12T03:17:03.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:03.856+0000 7fbae4b94640 1 -- 172.21.15.27:0/2040839203 learned_addr learned my addr 172.21.15.27:0/2040839203 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:17:03.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:03.856+0000 7fbae5395640 1 --2- 172.21.15.27:0/2040839203 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fbae00d4040 0x7fbae00b7ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:03.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:03.857+0000 7fbadf7fe640 1 -- 172.21.15.27:0/2040839203 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3943025114 0 0) 0x7fbae00d2930 con 0x7fbae00b6240 2024-09-12T03:17:03.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:03.857+0000 7fbadf7fe640 1 -- 172.21.15.27:0/2040839203 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbac8003520 con 0x7fbae00b6240 2024-09-12T03:17:03.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:03.857+0000 7fbadf7fe640 1 -- 172.21.15.27:0/2040839203 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4136513354 0 0) 0x7fbac8003520 con 0x7fbae00b6240 2024-09-12T03:17:03.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:03.857+0000 7fbadf7fe640 1 -- 172.21.15.27:0/2040839203 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fbae00d4040 msgr2=0x7fbae00b7ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:03.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:03.857+0000 7fbadf7fe640 1 --2- 172.21.15.27:0/2040839203 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fbae00d4040 0x7fbae00b7ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:03.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:03.857+0000 7fbadf7fe640 1 -- 172.21.15.27:0/2040839203 >> v1:172.21.15.45:6789/0 conn(0x7fbae00d3c20 legacy=0x7fbae00b77c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:03.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:03.857+0000 7fbadf7fe640 1 -- 172.21.15.27:0/2040839203 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbae00d31d0 con 0x7fbae00b6240 2024-09-12T03:17:03.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:03.857+0000 7fbadf7fe640 1 -- 172.21.15.27:0/2040839203 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 847755213 0 0) 0x7fbad0001670 con 0x7fbae00b6240 2024-09-12T03:17:03.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:03.857+0000 7fbadf7fe640 1 -- 172.21.15.27:0/2040839203 >> v1:172.21.15.27:6789/0 conn(0x7fbae00b6240 legacy=0x7fbae00b6620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:03.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:03.857+0000 7fbadf7fe640 1 --2- 172.21.15.27:0/2040839203 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbac80039e0 0x7fbac8003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:03.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:03.857+0000 7fbadf7fe640 1 --2- 172.21.15.27:0/2040839203 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fbae00d4040 0x7fbac8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:03.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:03.857+0000 7fbadf7fe640 1 --2- 172.21.15.27:0/2040839203 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbac8004d40 0x7fbac80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:03.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:03.857+0000 7fbadf7fe640 1 -- 172.21.15.27:0/2040839203 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fbae00d2ef0 con 0x7fbac8004d40 2024-09-12T03:17:03.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:03.857+0000 7fbadf7fe640 1 -- 172.21.15.27:0/2040839203 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fbac800bb30 con 0x7fbac80039e0 2024-09-12T03:17:03.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:03.857+0000 7fbadf7fe640 1 -- 172.21.15.27:0/2040839203 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fbac800bd30 con 0x7fbae00d4040 2024-09-12T03:17:03.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:03.857+0000 7fbae5395640 1 --2- 172.21.15.27:0/2040839203 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbac8004d40 0x7fbac80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:03.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:03.857+0000 7fbae4b94640 1 --2- 172.21.15.27:0/2040839203 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbac80039e0 0x7fbac8003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:03.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:03.857+0000 7fbadffff640 1 --2- 172.21.15.27:0/2040839203 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fbae00d4040 0x7fbac8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:03.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:03.857+0000 7fbae5395640 1 -- 172.21.15.27:0/2040839203 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fbae00d4040 msgr2=0x7fbac8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:03.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:03.857+0000 7fbae5395640 1 --2- 172.21.15.27:0/2040839203 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fbae00d4040 0x7fbac8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:03.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:03.857+0000 7fbae5395640 1 -- 172.21.15.27:0/2040839203 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbac80039e0 msgr2=0x7fbac8003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:03.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:03.857+0000 7fbae5395640 1 --2- 172.21.15.27:0/2040839203 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbac80039e0 0x7fbac8003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:03.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:03.857+0000 7fbae5395640 1 -- 172.21.15.27:0/2040839203 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbac8003520 con 0x7fbac8004d40 2024-09-12T03:17:03.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:03.857+0000 7fbadffff640 1 --2- 172.21.15.27:0/2040839203 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fbae00d4040 0x7fbac8004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:17:03.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:03.858+0000 7fbae5395640 1 --2- 172.21.15.27:0/2040839203 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbac8004d40 0x7fbac80051c0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fbad8009f90 tx=0x7fbac8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46675d203e899a10 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:03.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:03.858+0000 7fbadf7fe640 1 -- 172.21.15.27:0/2040839203 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbad8009c10 con 0x7fbac8004d40 2024-09-12T03:17:03.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:03.858+0000 7fbadf7fe640 1 -- 172.21.15.27:0/2040839203 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbad8036070 con 0x7fbac8004d40 2024-09-12T03:17:03.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:03.858+0000 7fbadf7fe640 1 -- 172.21.15.27:0/2040839203 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbad8031850 con 0x7fbac8004d40 2024-09-12T03:17:03.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:03.859+0000 7fbae6df8640 1 -- 172.21.15.27:0/2040839203 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbac8004d40 msgr2=0x7fbac80051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:03.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:03.859+0000 7fbae6df8640 1 --2- 172.21.15.27:0/2040839203 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbac8004d40 0x7fbac80051c0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fbad8009f90 tx=0x7fbac8005830 comp rx=0 tx=0).stop 2024-09-12T03:17:03.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:03.859+0000 7fbae5395640 1 -- 172.21.15.27:0/2040839203 reap_dead start 2024-09-12T03:17:03.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:03.859+0000 7fbae6df8640 1 -- 172.21.15.27:0/2040839203 shutdown_connections 2024-09-12T03:17:03.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:03.859+0000 7fbae6df8640 1 -- 172.21.15.27:0/2040839203 >> 172.21.15.27:0/2040839203 conn(0x7fbae00d1420 msgr2=0x7fbae00d1830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:03.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:03.859+0000 7fbae6df8640 1 -- 172.21.15.27:0/2040839203 shutdown_connections 2024-09-12T03:17:03.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:03.859+0000 7fbae6df8640 1 -- 172.21.15.27:0/2040839203 wait complete. 2024-09-12T03:17:03.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:03.860+0000 7fbae6df8640 1 Processor -- start 2024-09-12T03:17:03.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:03.860+0000 7fbae6df8640 1 -- start start 2024-09-12T03:17:03.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:03.861+0000 7fbae6df8640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fbae01f3f10 0x7fbae00ab2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:03.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:03.861+0000 7fbae6df8640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbae00ab7e0 0x7fbae00ccae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:03.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:03.861+0000 7fbae6df8640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbae00cd020 0x7fbae01f42f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:03.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:03.861+0000 7fbae6df8640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fbae00cfdf0 con 0x7fbae00cd020 2024-09-12T03:17:03.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:03.861+0000 7fbae6df8640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fbae00cfc70 con 0x7fbae00ab7e0 2024-09-12T03:17:03.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:03.861+0000 7fbae6df8640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fbae00b7ed0 con 0x7fbae01f3f10 2024-09-12T03:17:03.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:03.861+0000 7fbae5395640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbae00cd020 0x7fbae01f42f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:03.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:03.861+0000 7fbae5395640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbae00cd020 0x7fbae01f42f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58116/0 (socket says 172.21.15.27:58116) 2024-09-12T03:17:03.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:03.861+0000 7fbae5395640 1 -- 172.21.15.27:0/863262910 learned_addr learned my addr 172.21.15.27:0/863262910 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:17:03.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:03.861+0000 7fbae4b94640 1 --2- 172.21.15.27:0/863262910 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fbae01f3f10 0x7fbae00ab2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:03.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:03.861+0000 7fbadffff640 1 --2- 172.21.15.27:0/863262910 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbae00ab7e0 0x7fbae00ccae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:03.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:03.861+0000 7fbae5395640 1 -- 172.21.15.27:0/863262910 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fbae01f3f10 msgr2=0x7fbae00ab2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:03.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:03.861+0000 7fbae5395640 1 --2- 172.21.15.27:0/863262910 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fbae01f3f10 0x7fbae00ab2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:03.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:03.861+0000 7fbae5395640 1 -- 172.21.15.27:0/863262910 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbae00ab7e0 msgr2=0x7fbae00ccae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:03.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:03.861+0000 7fbae5395640 1 --2- 172.21.15.27:0/863262910 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbae00ab7e0 0x7fbae00ccae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:03.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:03.861+0000 7fbae5395640 1 -- 172.21.15.27:0/863262910 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbae00cd840 con 0x7fbae00cd020 2024-09-12T03:17:03.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:03.861+0000 7fbadffff640 1 --2- 172.21.15.27:0/863262910 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbae00ab7e0 0x7fbae00ccae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:17:03.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:03.862+0000 7fbae5395640 1 --2- 172.21.15.27:0/863262910 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbae00cd020 0x7fbae01f42f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fbad8033040 tx=0x7fbad8009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:03.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:03.862+0000 7fbadd7fa640 1 -- 172.21.15.27:0/863262910 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbad8009c10 con 0x7fbae00cd020 2024-09-12T03:17:03.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:03.862+0000 7fbadd7fa640 1 -- 172.21.15.27:0/863262910 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbad8003990 con 0x7fbae00cd020 2024-09-12T03:17:03.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:03.862+0000 7fbadd7fa640 1 -- 172.21.15.27:0/863262910 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbad8003d80 con 0x7fbae00cd020 2024-09-12T03:17:03.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:03.862+0000 7fbae6df8640 1 -- 172.21.15.27:0/863262910 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbae00cdb20 con 0x7fbae00cd020 2024-09-12T03:17:03.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:03.862+0000 7fbae6df8640 1 -- 172.21.15.27:0/863262910 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbae00ce0e0 con 0x7fbae00cd020 2024-09-12T03:17:03.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:03.863+0000 7fbae6df8640 1 -- 172.21.15.27:0/863262910 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbae00b6620 con 0x7fbae00cd020 2024-09-12T03:17:03.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:03.863+0000 7fbadd7fa640 1 -- 172.21.15.27:0/863262910 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fbad8005d10 con 0x7fbae00cd020 2024-09-12T03:17:03.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:03.864+0000 7fbadd7fa640 1 --2- 172.21.15.27:0/863262910 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fbacc06be60 0x7fbacc06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:03.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:03.864+0000 7fbadd7fa640 1 -- 172.21.15.27:0/863262910 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5357+0+0 (secure 0 0 0) 0x7fbad80abea0 con 0x7fbae00cd020 2024-09-12T03:17:03.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:03.864+0000 7fbae4b94640 1 --2- 172.21.15.27:0/863262910 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fbacc06be60 0x7fbacc06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:03.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:03.865+0000 7fbae4b94640 1 --2- 172.21.15.27:0/863262910 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fbacc06be60 0x7fbacc06e320 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fbae014e510 tx=0x7fbad0000ed0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:03.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:03.868+0000 7fbadd7fa640 1 -- 172.21.15.27:0/863262910 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fbad804a060 con 0x7fbae00cd020 2024-09-12T03:17:04.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:04.044+0000 7fbae6df8640 1 -- 172.21.15.27:0/863262910 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fbae00abbc0 con 0x7fbae00cd020 2024-09-12T03:17:04.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:04.730+0000 7fbadd7fa640 1 -- 172.21.15.27:0/863262910 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v24) ==== 169+0+0 (secure 0 0 0) 0x7fbad802e050 con 0x7fbae00cd020 2024-09-12T03:17:04.731 INFO:teuthology.orchestra.run.smithi027.stderr:pool 'cephfs_data' removed 2024-09-12T03:17:04.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:04.735+0000 7fbae6df8640 1 -- 172.21.15.27:0/863262910 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fbacc06be60 msgr2=0x7fbacc06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:04.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:04.735+0000 7fbae6df8640 1 --2- 172.21.15.27:0/863262910 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fbacc06be60 0x7fbacc06e320 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fbae014e510 tx=0x7fbad0000ed0 comp rx=0 tx=0).stop 2024-09-12T03:17:04.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:04.735+0000 7fbae6df8640 1 -- 172.21.15.27:0/863262910 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbae00cd020 msgr2=0x7fbae01f42f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:04.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:04.735+0000 7fbae6df8640 1 --2- 172.21.15.27:0/863262910 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbae00cd020 0x7fbae01f42f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fbad8033040 tx=0x7fbad8009750 comp rx=0 tx=0).stop 2024-09-12T03:17:04.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:04.735+0000 7fbae6df8640 1 -- 172.21.15.27:0/863262910 shutdown_connections 2024-09-12T03:17:04.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:04.735+0000 7fbae6df8640 1 --2- 172.21.15.27:0/863262910 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fbacc06be60 0x7fbacc06e320 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:04.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:04.735+0000 7fbae6df8640 1 --2- 172.21.15.27:0/863262910 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbae00cd020 0x7fbae01f42f0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:04.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:04.735+0000 7fbae6df8640 1 --2- 172.21.15.27:0/863262910 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbae00ab7e0 0x7fbae00ccae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:04.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:04.735+0000 7fbae6df8640 1 --2- 172.21.15.27:0/863262910 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fbae01f3f10 0x7fbae00ab2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:04.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:04.735+0000 7fbae6df8640 1 -- 172.21.15.27:0/863262910 >> 172.21.15.27:0/863262910 conn(0x7fbae00d1420 msgr2=0x7fbae00b4120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:04.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:04.735+0000 7fbae6df8640 1 -- 172.21.15.27:0/863262910 shutdown_connections 2024-09-12T03:17:04.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:04.735+0000 7fbae6df8640 1 -- 172.21.15.27:0/863262910 wait complete. 2024-09-12T03:17:04.749 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-12T03:17:04.750 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-12T03:17:04.750 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-12T03:17:04.750 INFO:teuthology.orchestra.run:waiting for 300 2024-09-12T03:17:04.750 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-12T03:17:04.750 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-12T03:17:04.750 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-12T03:17:04.751 INFO:teuthology.orchestra.run:waiting for 300 2024-09-12T03:17:04.751 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-12T03:17:04.751 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-12T03:17:04.751 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-12T03:17:04.751 INFO:teuthology.orchestra.run:waiting for 300 2024-09-12T03:17:04.751 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-12T03:17:04.751 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-12T03:17:04.752 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-12T03:17:04.752 INFO:teuthology.orchestra.run:waiting for 300 2024-09-12T03:17:04.761 INFO:tasks.ceph.mds.a:Stopped 2024-09-12T03:17:04.761 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-12T03:17:04.762 INFO:tasks.ceph.mds.c:Stopped 2024-09-12T03:17:04.762 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-12T03:17:04.763 INFO:tasks.ceph.mds.b:Stopped 2024-09-12T03:17:04.763 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-12T03:17:04.764 INFO:tasks.ceph.mds.d:Stopped 2024-09-12T03:17:04.764 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-12T03:17:04.803 INFO:tasks.ceph.mds.a:Started 2024-09-12T03:17:04.804 DEBUG:teuthology.parallel:result is None 2024-09-12T03:17:04.804 INFO:tasks.ceph.mds.c:Started 2024-09-12T03:17:04.804 DEBUG:teuthology.parallel:result is None 2024-09-12T03:17:04.806 INFO:tasks.ceph.mds.b:Started 2024-09-12T03:17:04.807 DEBUG:teuthology.parallel:result is None 2024-09-12T03:17:04.807 INFO:tasks.ceph.mds.d:Started 2024-09-12T03:17:04.807 DEBUG:teuthology.parallel:result is None 2024-09-12T03:17:04.808 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-12T03:17:04.884 INFO:tasks.ceph.mds.a.smithi027.stdout:starting mds.a at 2024-09-12T03:17:04.884 INFO:tasks.ceph.mds.c.smithi027.stdout:starting mds.c at 2024-09-12T03:17:04.891 INFO:tasks.ceph.mds.d.smithi045.stdout:starting mds.d at 2024-09-12T03:17:04.891 INFO:tasks.ceph.mds.b.smithi045.stdout:starting mds.b at 2024-09-12T03:17:04.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:04.917+0000 7fca790e6640 1 Processor -- start 2024-09-12T03:17:04.918 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:04.917+0000 7fca790e6640 1 -- start start 2024-09-12T03:17:04.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:04.918+0000 7fca790e6640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fca7415c2e0 0x7fca740cf070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:04.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:04.918+0000 7fca790e6640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fca7415c6c0 0x7fca74073210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:04.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:04.918+0000 7fca790e6640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fca740cfaf0 0x7fca74073750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:04.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:04.919+0000 7fca7359e640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fca740cfaf0 0x7fca74073750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:04.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:04.919+0000 7fca7359e640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fca740cfaf0 0x7fca74073750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58160/0 (socket says 172.21.15.27:58160) 2024-09-12T03:17:04.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:04.919+0000 7fca7359e640 1 -- 172.21.15.27:0/2335764671 learned_addr learned my addr 172.21.15.27:0/2335764671 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:17:04.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:04.919+0000 7fca790e6640 1 -- 172.21.15.27:0/2335764671 --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7fca74160eb0 con 0x7fca740cfaf0 2024-09-12T03:17:04.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:04.919+0000 7fca790e6640 1 -- 172.21.15.27:0/2335764671 --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fca740cf8a0 con 0x7fca7415c2e0 2024-09-12T03:17:04.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:04.919+0000 7fca790e6640 1 -- 172.21.15.27:0/2335764671 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fca741600f0 con 0x7fca7415c6c0 2024-09-12T03:17:04.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:04.919+0000 7fca7259c640 1 --2- 172.21.15.27:0/2335764671 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fca7415c6c0 0x7fca74073210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:04.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:04.919+0000 7fca72d9d640 1 --2- 172.21.15.27:0/2335764671 >> v2:172.21.15.45:3300/0 conn(0x7fca7415c2e0 0x7fca740cf070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:04.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:04.919+0000 7fca7359e640 1 -- 172.21.15.27:0/2335764671 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fca7415c6c0 msgr2=0x7fca74073210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:04.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:04.919+0000 7fca7359e640 1 --2- 172.21.15.27:0/2335764671 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fca7415c6c0 0x7fca74073210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:04.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:04.919+0000 7fca7359e640 1 -- 172.21.15.27:0/2335764671 >> v2:172.21.15.45:3300/0 conn(0x7fca7415c2e0 msgr2=0x7fca740cf070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:04.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:04.919+0000 7fca7359e640 1 --2- 172.21.15.27:0/2335764671 >> v2:172.21.15.45:3300/0 conn(0x7fca7415c2e0 0x7fca740cf070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:04.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:04.919+0000 7fca7359e640 1 -- 172.21.15.27:0/2335764671 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca741603d0 con 0x7fca740cfaf0 2024-09-12T03:17:04.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:04.920+0000 7fca7359e640 1 --2- 172.21.15.27:0/2335764671 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fca740cfaf0 0x7fca74073750 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fca7406f160 tx=0x7fca6c02a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=562032155ef0d9a8 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:04.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:04.920+0000 7fca71d9b640 1 -- 172.21.15.27:0/2335764671 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca6c036070 con 0x7fca740cfaf0 2024-09-12T03:17:04.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:04.920+0000 7fca71d9b640 1 -- 172.21.15.27:0/2335764671 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca6c031600 con 0x7fca740cfaf0 2024-09-12T03:17:04.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:04.920+0000 7fca71d9b640 1 -- 172.21.15.27:0/2335764671 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca6c0319d0 con 0x7fca740cfaf0 2024-09-12T03:17:04.923 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:04.920+0000 7fca790e6640 1 -- 172.21.15.27:0/2335764671 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fca740cfaf0 msgr2=0x7fca74073750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:04.923 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:04.920+0000 7fca790e6640 1 --2- 172.21.15.27:0/2335764671 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fca740cfaf0 0x7fca74073750 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fca7406f160 tx=0x7fca6c02a610 comp rx=0 tx=0).stop 2024-09-12T03:17:04.923 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:04.921+0000 7fca790e6640 1 -- 172.21.15.27:0/2335764671 shutdown_connections 2024-09-12T03:17:04.923 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:04.921+0000 7fca790e6640 1 --2- 172.21.15.27:0/2335764671 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fca740cfaf0 0x7fca74073750 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:04.923 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:04.921+0000 7fca790e6640 1 --2- 172.21.15.27:0/2335764671 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fca7415c6c0 0x7fca74073210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:04.923 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:04.921+0000 7fca790e6640 1 --2- 172.21.15.27:0/2335764671 >> v2:172.21.15.45:3300/0 conn(0x7fca7415c2e0 0x7fca740cf070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:04.923 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:04.921+0000 7fca790e6640 1 -- 172.21.15.27:0/2335764671 >> 172.21.15.27:0/2335764671 conn(0x7fca7406f2c0 msgr2=0x7fca7406f6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:04.923 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:04.921+0000 7fca790e6640 1 -- 172.21.15.27:0/2335764671 shutdown_connections 2024-09-12T03:17:04.923 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:04.921+0000 7fca790e6640 1 -- 172.21.15.27:0/2335764671 wait complete. 2024-09-12T03:17:04.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:04.922+0000 7fca790e6640 1 Processor -- start 2024-09-12T03:17:04.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:04.922+0000 7fca790e6640 1 -- start start 2024-09-12T03:17:04.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:04.923+0000 7fca790e6640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fca740cfaf0 0x7fca741f1950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:04.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:04.923+0000 7fca790e6640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fca7415c2e0 0x7fca740acb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:04.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:04.923+0000 7fca790e6640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fca7415c6c0 0x7fca740ade40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:04.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:04.923+0000 7fca790e6640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fca74070a40 con 0x7fca7415c2e0 2024-09-12T03:17:04.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:04.923+0000 7fca790e6640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fca740cf5b0 con 0x7fca7415c6c0 2024-09-12T03:17:04.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:04.923+0000 7fca790e6640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fca74070bc0 con 0x7fca740cfaf0 2024-09-12T03:17:04.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:04.923+0000 7fca7259c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fca7415c2e0 0x7fca740acb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:04.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:04.923+0000 7fca7259c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fca7415c2e0 0x7fca740acb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58168/0 (socket says 172.21.15.27:58168) 2024-09-12T03:17:04.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:04.923+0000 7fca7259c640 1 -- 172.21.15.27:0/3339064371 learned_addr learned my addr 172.21.15.27:0/3339064371 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:17:04.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:04.923+0000 7fca72d9d640 1 --2- 172.21.15.27:0/3339064371 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fca740cfaf0 0x7fca741f1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:04.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:04.923+0000 7fca7359e640 1 --2- 172.21.15.27:0/3339064371 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fca7415c6c0 0x7fca740ade40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:04.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:04.923+0000 7fca7259c640 1 -- 172.21.15.27:0/3339064371 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fca740cfaf0 msgr2=0x7fca741f1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:04.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:04.923+0000 7fca7259c640 1 --2- 172.21.15.27:0/3339064371 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fca740cfaf0 0x7fca741f1950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:04.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:04.923+0000 7fca7259c640 1 -- 172.21.15.27:0/3339064371 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fca7415c6c0 msgr2=0x7fca740ade40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:04.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:04.923+0000 7fca7259c640 1 --2- 172.21.15.27:0/3339064371 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fca7415c6c0 0x7fca740ade40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:04.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:04.923+0000 7fca7259c640 1 -- 172.21.15.27:0/3339064371 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca740ae790 con 0x7fca7415c2e0 2024-09-12T03:17:04.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:04.924+0000 7fca7259c640 1 --2- 172.21.15.27:0/3339064371 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fca7415c2e0 0x7fca740acb50 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fca5c00e870 tx=0x7fca5c00ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:04.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:04.924+0000 7fca6b7fe640 1 -- 172.21.15.27:0/3339064371 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca5c017070 con 0x7fca7415c2e0 2024-09-12T03:17:04.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:04.924+0000 7fca6b7fe640 1 -- 172.21.15.27:0/3339064371 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca5c00c940 con 0x7fca7415c2e0 2024-09-12T03:17:04.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:04.924+0000 7fca790e6640 1 -- 172.21.15.27:0/3339064371 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca740ac070 con 0x7fca7415c2e0 2024-09-12T03:17:04.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:04.924+0000 7fca6b7fe640 1 -- 172.21.15.27:0/3339064371 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca5c00cd30 con 0x7fca7415c2e0 2024-09-12T03:17:04.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:04.924+0000 7fca790e6640 1 -- 172.21.15.27:0/3339064371 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca740ac630 con 0x7fca7415c2e0 2024-09-12T03:17:04.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:04.925+0000 7fca790e6640 1 -- 172.21.15.27:0/3339064371 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca74158580 con 0x7fca7415c2e0 2024-09-12T03:17:04.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:04.926+0000 7fca6b7fe640 1 -- 172.21.15.27:0/3339064371 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fca5c01a070 con 0x7fca7415c2e0 2024-09-12T03:17:04.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:04.926+0000 7fca6b7fe640 1 --2- 172.21.15.27:0/3339064371 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fca5406be30 0x7fca5406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:04.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:04.926+0000 7fca6b7fe640 1 -- 172.21.15.27:0/3339064371 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 4943+0+0 (secure 0 0 0) 0x7fca5c08ada0 con 0x7fca7415c2e0 2024-09-12T03:17:04.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:04.926+0000 7fca72d9d640 1 --2- 172.21.15.27:0/3339064371 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fca5406be30 0x7fca5406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:04.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:04.927+0000 7fca72d9d640 1 --2- 172.21.15.27:0/3339064371 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fca5406be30 0x7fca5406e2f0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fca64009ac0 tx=0x7fca64009190 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:04.930 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:04.931+0000 7fca6b7fe640 1 -- 172.21.15.27:0/3339064371 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fca5c059370 con 0x7fca7415c2e0 2024-09-12T03:17:05.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:05.115+0000 7fca790e6640 1 -- 172.21.15.27:0/3339064371 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7fca741582a0 con 0x7fca7415c2e0 2024-09-12T03:17:05.740 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:05.740+0000 7fca6b7fe640 1 -- 172.21.15.27:0/3339064371 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v26) ==== 97+0+0 (secure 0 0 0) 0x7fca5c05d670 con 0x7fca7415c2e0 2024-09-12T03:17:05.740 INFO:teuthology.orchestra.run.smithi027.stderr: removed all blocklist entries 2024-09-12T03:17:05.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:05.742+0000 7fca790e6640 1 -- 172.21.15.27:0/3339064371 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fca5406be30 msgr2=0x7fca5406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:05.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:05.742+0000 7fca790e6640 1 --2- 172.21.15.27:0/3339064371 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fca5406be30 0x7fca5406e2f0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fca64009ac0 tx=0x7fca64009190 comp rx=0 tx=0).stop 2024-09-12T03:17:05.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:05.742+0000 7fca790e6640 1 -- 172.21.15.27:0/3339064371 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fca7415c2e0 msgr2=0x7fca740acb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:05.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:05.742+0000 7fca790e6640 1 --2- 172.21.15.27:0/3339064371 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fca7415c2e0 0x7fca740acb50 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fca5c00e870 tx=0x7fca5c00ed30 comp rx=0 tx=0).stop 2024-09-12T03:17:05.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:05.743+0000 7fca790e6640 1 -- 172.21.15.27:0/3339064371 shutdown_connections 2024-09-12T03:17:05.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:05.743+0000 7fca790e6640 1 --2- 172.21.15.27:0/3339064371 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fca5406be30 0x7fca5406e2f0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:05.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:05.743+0000 7fca790e6640 1 --2- 172.21.15.27:0/3339064371 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fca7415c6c0 0x7fca740ade40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:05.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:05.743+0000 7fca790e6640 1 --2- 172.21.15.27:0/3339064371 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fca7415c2e0 0x7fca740acb50 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:05.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:05.743+0000 7fca790e6640 1 --2- 172.21.15.27:0/3339064371 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fca740cfaf0 0x7fca741f1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:05.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:05.743+0000 7fca790e6640 1 -- 172.21.15.27:0/3339064371 >> 172.21.15.27:0/3339064371 conn(0x7fca7406f2c0 msgr2=0x7fca74159870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:05.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:05.743+0000 7fca790e6640 1 -- 172.21.15.27:0/3339064371 shutdown_connections 2024-09-12T03:17:05.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:05.743+0000 7fca790e6640 1 -- 172.21.15.27:0/3339064371 wait complete. 2024-09-12T03:17:05.755 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-12T03:17:05.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:05.861+0000 7f2d802b7640 1 Processor -- start 2024-09-12T03:17:05.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:05.861+0000 7f2d802b7640 1 -- start start 2024-09-12T03:17:05.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:05.862+0000 7f2d802b7640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f2d780b1ca0 0x7f2d7815f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:05.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:05.862+0000 7f2d802b7640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f2d780cf660 0x7f2d780d2780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:05.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:05.862+0000 7f2d802b7640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d7815fd70 con 0x7f2d7806f360 2024-09-12T03:17:05.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:05.862+0000 7f2d802b7640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f2d780d22f0 con 0x7f2d780b1ca0 2024-09-12T03:17:05.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:05.862+0000 7f2d802b7640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f2d7806cc40 con 0x7f2d780cf660 2024-09-12T03:17:05.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:05.862+0000 7f2d7e053640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f2d780b1ca0 0x7f2d7815f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:05.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:05.862+0000 7f2d7e053640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f2d780b1ca0 0x7f2d7815f830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58170/0 (socket says 172.21.15.27:58170) 2024-09-12T03:17:05.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:05.862+0000 7f2d7e053640 1 -- 172.21.15.27:0/758573768 learned_addr learned my addr 172.21.15.27:0/758573768 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:17:05.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:05.862+0000 7f2d7e854640 1 --2- 172.21.15.27:0/758573768 >> v2:172.21.15.45:3300/0 conn(0x7f2d780cf660 0x7f2d780d2780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:05.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:05.862+0000 7f2d7e053640 1 -- 172.21.15.27:0/758573768 >> v2:172.21.15.45:3300/0 conn(0x7f2d780cf660 msgr2=0x7f2d780d2780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:05.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:05.862+0000 7f2d7e053640 1 --2- 172.21.15.27:0/758573768 >> v2:172.21.15.45:3300/0 conn(0x7f2d780cf660 0x7f2d780d2780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:05.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:05.862+0000 7f2d7e053640 1 -- 172.21.15.27:0/758573768 >> v1:172.21.15.45:6789/0 conn(0x7f2d7806f360 legacy=0x7f2d780cef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:05.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:05.862+0000 7f2d7e053640 1 -- 172.21.15.27:0/758573768 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d780d1ce0 con 0x7f2d780b1ca0 2024-09-12T03:17:05.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:05.863+0000 7f2d7e053640 1 --2- 172.21.15.27:0/758573768 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2d780b1ca0 0x7f2d7815f830 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f2d78069cf0 tx=0x7f2d6c02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67f1dfc6d16ef5be server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:05.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:05.863+0000 7f2d7d051640 1 -- 172.21.15.27:0/758573768 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d6c037070 con 0x7f2d780b1ca0 2024-09-12T03:17:05.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:05.863+0000 7f2d7d051640 1 -- 172.21.15.27:0/758573768 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d6c032440 con 0x7f2d780b1ca0 2024-09-12T03:17:05.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:05.863+0000 7f2d7d051640 1 -- 172.21.15.27:0/758573768 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d6c032810 con 0x7f2d780b1ca0 2024-09-12T03:17:05.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:05.864+0000 7f2d802b7640 1 -- 172.21.15.27:0/758573768 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2d780b1ca0 msgr2=0x7f2d7815f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:05.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:05.864+0000 7f2d802b7640 1 --2- 172.21.15.27:0/758573768 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2d780b1ca0 0x7f2d7815f830 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f2d78069cf0 tx=0x7f2d6c02a230 comp rx=0 tx=0).stop 2024-09-12T03:17:05.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:05.864+0000 7f2d802b7640 1 -- 172.21.15.27:0/758573768 shutdown_connections 2024-09-12T03:17:05.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:05.864+0000 7f2d802b7640 1 --2- 172.21.15.27:0/758573768 >> v2:172.21.15.45:3300/0 conn(0x7f2d780cf660 0x7f2d780d2780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:05.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:05.864+0000 7f2d802b7640 1 --2- 172.21.15.27:0/758573768 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2d780b1ca0 0x7f2d7815f830 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:05.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:05.864+0000 7f2d802b7640 1 -- 172.21.15.27:0/758573768 >> 172.21.15.27:0/758573768 conn(0x7f2d78073800 msgr2=0x7f2d7809ba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:05.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:05.864+0000 7f2d802b7640 1 -- 172.21.15.27:0/758573768 shutdown_connections 2024-09-12T03:17:05.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:05.864+0000 7f2d802b7640 1 -- 172.21.15.27:0/758573768 wait complete. 2024-09-12T03:17:05.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:05.865+0000 7f2d802b7640 1 Processor -- start 2024-09-12T03:17:05.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:05.865+0000 7f2d802b7640 1 -- start start 2024-09-12T03:17:05.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:05.866+0000 7f2d802b7640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2d7806f360 0x7f2d7815d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:05.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:05.866+0000 7f2d802b7640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2d780b1ca0 0x7f2d78156190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:05.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:05.866+0000 7f2d802b7640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2d780cf660 0x7f2d7815ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:05.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:05.866+0000 7f2d802b7640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f2d7806d1c0 con 0x7f2d780cf660 2024-09-12T03:17:05.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:05.866+0000 7f2d802b7640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f2d780d1f80 con 0x7f2d780b1ca0 2024-09-12T03:17:05.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:05.866+0000 7f2d802b7640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f2d7806d340 con 0x7f2d7806f360 2024-09-12T03:17:05.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:05.866+0000 7f2d7e854640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2d780cf660 0x7f2d7815ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:05.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:05.866+0000 7f2d7e854640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2d780cf660 0x7f2d7815ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58174/0 (socket says 172.21.15.27:58174) 2024-09-12T03:17:05.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:05.866+0000 7f2d7e854640 1 -- 172.21.15.27:0/1241014674 learned_addr learned my addr 172.21.15.27:0/1241014674 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:17:05.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:05.866+0000 7f2d7d852640 1 --2- 172.21.15.27:0/1241014674 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2d780b1ca0 0x7f2d78156190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:05.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:05.866+0000 7f2d7e854640 1 -- 172.21.15.27:0/1241014674 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2d7806f360 msgr2=0x7f2d7815d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:05.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:05.866+0000 7f2d7e854640 1 --2- 172.21.15.27:0/1241014674 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2d7806f360 0x7f2d7815d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:05.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:05.866+0000 7f2d7e854640 1 -- 172.21.15.27:0/1241014674 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2d780b1ca0 msgr2=0x7f2d78156190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:05.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:05.866+0000 7f2d7e854640 1 --2- 172.21.15.27:0/1241014674 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2d780b1ca0 0x7f2d78156190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:05.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:05.866+0000 7f2d7e854640 1 -- 172.21.15.27:0/1241014674 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d7815b520 con 0x7f2d780cf660 2024-09-12T03:17:05.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:05.867+0000 7f2d7e854640 1 --2- 172.21.15.27:0/1241014674 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2d780cf660 0x7f2d7815ab40 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f2d70007d40 tx=0x7f2d7000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:05.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:05.867+0000 7f2d6affd640 1 -- 172.21.15.27:0/1241014674 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d70014420 con 0x7f2d780cf660 2024-09-12T03:17:05.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:05.867+0000 7f2d6affd640 1 -- 172.21.15.27:0/1241014674 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d70019070 con 0x7f2d780cf660 2024-09-12T03:17:05.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:05.867+0000 7f2d802b7640 1 -- 172.21.15.27:0/1241014674 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d7815c720 con 0x7f2d780cf660 2024-09-12T03:17:05.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:05.867+0000 7f2d6affd640 1 -- 172.21.15.27:0/1241014674 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d70014820 con 0x7f2d780cf660 2024-09-12T03:17:05.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:05.867+0000 7f2d802b7640 1 -- 172.21.15.27:0/1241014674 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d7815cce0 con 0x7f2d780cf660 2024-09-12T03:17:05.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:05.868+0000 7f2d802b7640 1 -- 172.21.15.27:0/1241014674 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d78161a90 con 0x7f2d780cf660 2024-09-12T03:17:05.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:05.868+0000 7f2d6affd640 1 -- 172.21.15.27:0/1241014674 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2d700149c0 con 0x7f2d780cf660 2024-09-12T03:17:05.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:05.869+0000 7f2d6affd640 1 --2- 172.21.15.27:0/1241014674 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f2d5c06be30 0x7f2d5c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:05.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:05.869+0000 7f2d6affd640 1 -- 172.21.15.27:0/1241014674 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 4857+0+0 (secure 0 0 0) 0x7f2d7008b2b0 con 0x7f2d780cf660 2024-09-12T03:17:05.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:05.869+0000 7f2d7e053640 1 --2- 172.21.15.27:0/1241014674 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f2d5c06be30 0x7f2d5c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:05.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:05.870+0000 7f2d7e053640 1 --2- 172.21.15.27:0/1241014674 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f2d5c06be30 0x7f2d5c06e2f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f2d6c034000 tx=0x7f2d6c02c000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:05.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:05.873+0000 7f2d6affd640 1 -- 172.21.15.27:0/1241014674 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f2d78161a90 con 0x7f2d780cf660 2024-09-12T03:17:06.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.116+0000 7f2d802b7640 1 -- 172.21.15.27:0/1241014674 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f2d780cfc40 con 0x7f2d780cf660 2024-09-12T03:17:06.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.116+0000 7f2d6affd640 1 -- 172.21.15.27:0/1241014674 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v4) ==== 80+0+6645 (secure 0 0 0) 0x7f2d70059fb0 con 0x7f2d780cf660 2024-09-12T03:17:06.117 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:17:06.117 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-12T03:17:06.117 INFO:teuthology.orchestra.run.smithi027.stdout: "auth_dump": [ 2024-09-12T03:17:06.117 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:17:06.117 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "mds.a", 2024-09-12T03:17:06.117 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDUXOJmIhxjNxAAaGg8SwtW6NRe/lIgNCV20w==", 2024-09-12T03:17:06.117 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:17:06.118 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow", 2024-09-12T03:17:06.118 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow *", 2024-09-12T03:17:06.118 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow *", 2024-09-12T03:17:06.118 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2024-09-12T03:17:06.118 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:17:06.118 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:17:06.118 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:17:06.118 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "mds.b", 2024-09-12T03:17:06.118 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDVXOJm63kWCBAAzjfOsBsWUI+IAsuZ49yEkw==", 2024-09-12T03:17:06.118 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:17:06.118 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow", 2024-09-12T03:17:06.119 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow *", 2024-09-12T03:17:06.119 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow *", 2024-09-12T03:17:06.119 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2024-09-12T03:17:06.119 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:17:06.119 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:17:06.119 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:17:06.119 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "mds.c", 2024-09-12T03:17:06.119 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDVXOJm0sX7AhAAdynCcho2a/AgFF9Cw3+vAw==", 2024-09-12T03:17:06.119 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:17:06.119 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow", 2024-09-12T03:17:06.119 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow *", 2024-09-12T03:17:06.120 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow *", 2024-09-12T03:17:06.120 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2024-09-12T03:17:06.120 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:17:06.120 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:17:06.120 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:17:06.120 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "mds.d", 2024-09-12T03:17:06.120 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDVXOJm7Dd8DxAANHYmvKgpRZ6sVfXDSgPHWw==", 2024-09-12T03:17:06.120 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:17:06.120 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow", 2024-09-12T03:17:06.120 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow *", 2024-09-12T03:17:06.120 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow *", 2024-09-12T03:17:06.121 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2024-09-12T03:17:06.121 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:17:06.121 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:17:06.121 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:17:06.121 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "osd.0", 2024-09-12T03:17:06.121 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDYXOJmJTVYLxAAmo8kxRjaiwTT44dZLZlq4A==", 2024-09-12T03:17:06.121 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:17:06.121 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow profile osd", 2024-09-12T03:17:06.121 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile osd", 2024-09-12T03:17:06.122 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2024-09-12T03:17:06.122 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:17:06.122 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:17:06.122 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:17:06.122 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "osd.1", 2024-09-12T03:17:06.122 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDcXOJmYV0BFBAAmoGX9gf9yLoE/qkWFnv9tg==", 2024-09-12T03:17:06.122 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:17:06.122 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow profile osd", 2024-09-12T03:17:06.122 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile osd", 2024-09-12T03:17:06.122 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2024-09-12T03:17:06.123 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:17:06.123 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:17:06.123 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:17:06.123 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "osd.2", 2024-09-12T03:17:06.123 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDfXOJmEetiJRAAXk+AvfTfhjsB6T2UB4Z2/w==", 2024-09-12T03:17:06.123 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:17:06.123 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow profile osd", 2024-09-12T03:17:06.123 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile osd", 2024-09-12T03:17:06.123 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2024-09-12T03:17:06.123 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:17:06.123 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:17:06.124 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:17:06.124 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "osd.3", 2024-09-12T03:17:06.124 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDiXOJmrQbPJxAAQsi2B5a1LnPTPGZVdo7j7w==", 2024-09-12T03:17:06.124 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:17:06.124 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow profile osd", 2024-09-12T03:17:06.124 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile osd", 2024-09-12T03:17:06.124 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2024-09-12T03:17:06.124 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:17:06.124 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:17:06.124 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:17:06.124 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "osd.4", 2024-09-12T03:17:06.124 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDnXOJmoMr1NxAAv3u2q8hYt9w//9gz+N7UAQ==", 2024-09-12T03:17:06.124 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:17:06.124 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow profile osd", 2024-09-12T03:17:06.125 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile osd", 2024-09-12T03:17:06.125 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2024-09-12T03:17:06.125 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:17:06.125 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:17:06.125 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:17:06.125 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "osd.5", 2024-09-12T03:17:06.125 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDqXOJmxpUPOxAAxLH6RJsyLOZ2XMfzM0fQ1g==", 2024-09-12T03:17:06.125 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:17:06.125 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow profile osd", 2024-09-12T03:17:06.125 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile osd", 2024-09-12T03:17:06.125 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2024-09-12T03:17:06.125 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:17:06.125 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:17:06.125 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:17:06.126 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "osd.6", 2024-09-12T03:17:06.126 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDuXOJmuP0+LxAAMHoz5eSO81hPtkdBIS2cig==", 2024-09-12T03:17:06.126 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:17:06.126 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow profile osd", 2024-09-12T03:17:06.126 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile osd", 2024-09-12T03:17:06.126 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2024-09-12T03:17:06.126 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:17:06.126 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:17:06.126 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:17:06.126 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "osd.7", 2024-09-12T03:17:06.126 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDxXOJmuIC/MRAAPDM+lAHqdoQyGVODwjFp+Q==", 2024-09-12T03:17:06.126 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:17:06.126 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow profile osd", 2024-09-12T03:17:06.126 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile osd", 2024-09-12T03:17:06.127 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2024-09-12T03:17:06.127 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:17:06.127 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:17:06.127 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:17:06.127 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "client.0", 2024-09-12T03:17:06.127 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDVXOJmLm+5ExAAECM4CKfuVElmp/n0b02mXg==", 2024-09-12T03:17:06.127 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:17:06.127 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow", 2024-09-12T03:17:06.127 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow r", 2024-09-12T03:17:06.127 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow rw", 2024-09-12T03:17:06.127 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow rwx" 2024-09-12T03:17:06.127 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:17:06.127 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:17:06.127 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:17:06.128 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "client.1", 2024-09-12T03:17:06.128 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDVXOJm61/FHBAALCQouXySfQeZ5DESGmqkzA==", 2024-09-12T03:17:06.128 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:17:06.128 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow", 2024-09-12T03:17:06.128 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow r", 2024-09-12T03:17:06.128 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow rw", 2024-09-12T03:17:06.128 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow rwx" 2024-09-12T03:17:06.128 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:17:06.128 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:17:06.128 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:17:06.128 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "client.2", 2024-09-12T03:17:06.128 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDVXOJmq/ZkGRAAOzmQzEIhb4b/YTxayc34Bw==", 2024-09-12T03:17:06.128 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:17:06.128 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow", 2024-09-12T03:17:06.129 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow r", 2024-09-12T03:17:06.129 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow rw", 2024-09-12T03:17:06.129 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow rwx" 2024-09-12T03:17:06.129 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:17:06.129 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:17:06.129 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:17:06.129 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "client.3", 2024-09-12T03:17:06.129 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDVXOJm+DlzIhAATQKZe+Zyk0MM8efTa6g9PA==", 2024-09-12T03:17:06.129 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:17:06.129 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow", 2024-09-12T03:17:06.129 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow r", 2024-09-12T03:17:06.129 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow rw", 2024-09-12T03:17:06.129 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow rwx" 2024-09-12T03:17:06.129 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:17:06.130 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:17:06.130 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:17:06.130 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "client.admin", 2024-09-12T03:17:06.130 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDUXOJm+JGWGhAApTQw9RtJWd/J5QzQaUFXvg==", 2024-09-12T03:17:06.130 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:17:06.130 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow *", 2024-09-12T03:17:06.130 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow *", 2024-09-12T03:17:06.130 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow *", 2024-09-12T03:17:06.130 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2024-09-12T03:17:06.130 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:17:06.130 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:17:06.130 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:17:06.130 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "client.bootstrap-mds", 2024-09-12T03:17:06.130 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQD3XOJmaCp0LhAA478FS9mE26cs+/VYW/IGTQ==", 2024-09-12T03:17:06.131 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:17:06.131 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile bootstrap-mds" 2024-09-12T03:17:06.131 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:17:06.131 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:17:06.131 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:17:06.131 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "client.bootstrap-mgr", 2024-09-12T03:17:06.131 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQD3XOJmuTJ0LhAA4NB0hNLNVsf8lGSRmtcUIA==", 2024-09-12T03:17:06.131 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:17:06.131 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile bootstrap-mgr" 2024-09-12T03:17:06.131 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:17:06.131 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:17:06.131 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:17:06.131 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "client.bootstrap-osd", 2024-09-12T03:17:06.131 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQD3XOJmWTp0LhAA1KxilDa70GILLMZIANBoMA==", 2024-09-12T03:17:06.132 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:17:06.132 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile bootstrap-osd" 2024-09-12T03:17:06.132 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:17:06.132 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:17:06.132 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:17:06.132 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "client.bootstrap-rbd", 2024-09-12T03:17:06.132 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQD3XOJm1EF0LhAAVerqReoWMTuLelASSvjVug==", 2024-09-12T03:17:06.132 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:17:06.132 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile bootstrap-rbd" 2024-09-12T03:17:06.132 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:17:06.132 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:17:06.132 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:17:06.132 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "client.bootstrap-rbd-mirror", 2024-09-12T03:17:06.133 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQD3XOJmU0l0LhAA3TjgiEH3czDvL4v7VlYxCg==", 2024-09-12T03:17:06.133 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:17:06.133 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2024-09-12T03:17:06.133 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:17:06.133 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:17:06.133 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:17:06.133 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "client.bootstrap-rgw", 2024-09-12T03:17:06.133 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQD3XOJmtlB0LhAAv8UwfFxPJ2PYfJI151+Eig==", 2024-09-12T03:17:06.133 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:17:06.133 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile bootstrap-rgw" 2024-09-12T03:17:06.133 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:17:06.133 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:17:06.133 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:17:06.133 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "mgr.x", 2024-09-12T03:17:06.134 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDUXOJmZcsONBAAMTY2KykuHaEkv1t6Vz5KjA==", 2024-09-12T03:17:06.134 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:17:06.134 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow *", 2024-09-12T03:17:06.134 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile mgr", 2024-09-12T03:17:06.134 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2024-09-12T03:17:06.134 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:17:06.134 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:17:06.134 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:17:06.134 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "mgr.y", 2024-09-12T03:17:06.134 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDUXOJmJOgZMBAAQ/JTxfhox7HgxBMtsvUCBg==", 2024-09-12T03:17:06.134 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:17:06.134 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow *", 2024-09-12T03:17:06.134 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile mgr", 2024-09-12T03:17:06.134 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2024-09-12T03:17:06.135 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:17:06.135 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:17:06.135 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-12T03:17:06.135 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-12T03:17:06.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.119+0000 7f2d802b7640 1 -- 172.21.15.27:0/1241014674 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f2d5c06be30 msgr2=0x7f2d5c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:06.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.119+0000 7f2d802b7640 1 --2- 172.21.15.27:0/1241014674 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f2d5c06be30 0x7f2d5c06e2f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f2d6c034000 tx=0x7f2d6c02c000 comp rx=0 tx=0).stop 2024-09-12T03:17:06.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.119+0000 7f2d802b7640 1 -- 172.21.15.27:0/1241014674 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2d780cf660 msgr2=0x7f2d7815ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:06.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.119+0000 7f2d802b7640 1 --2- 172.21.15.27:0/1241014674 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2d780cf660 0x7f2d7815ab40 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f2d70007d40 tx=0x7f2d7000e4b0 comp rx=0 tx=0).stop 2024-09-12T03:17:06.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.119+0000 7f2d802b7640 1 -- 172.21.15.27:0/1241014674 shutdown_connections 2024-09-12T03:17:06.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.119+0000 7f2d802b7640 1 --2- 172.21.15.27:0/1241014674 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f2d5c06be30 0x7f2d5c06e2f0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:06.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.119+0000 7f2d802b7640 1 --2- 172.21.15.27:0/1241014674 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2d780cf660 0x7f2d7815ab40 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:06.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.119+0000 7f2d802b7640 1 --2- 172.21.15.27:0/1241014674 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2d780b1ca0 0x7f2d78156190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:06.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.119+0000 7f2d802b7640 1 --2- 172.21.15.27:0/1241014674 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2d7806f360 0x7f2d7815d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:06.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.119+0000 7f2d802b7640 1 -- 172.21.15.27:0/1241014674 >> 172.21.15.27:0/1241014674 conn(0x7f2d78073800 msgr2=0x7f2d780b3250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:06.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.119+0000 7f2d802b7640 1 -- 172.21.15.27:0/1241014674 shutdown_connections 2024-09-12T03:17:06.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.119+0000 7f2d802b7640 1 -- 172.21.15.27:0/1241014674 wait complete. 2024-09-12T03:17:06.138 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2024-09-12T03:17:06.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.242+0000 7f3a1ec1e640 1 Processor -- start 2024-09-12T03:17:06.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.242+0000 7f3a1ec1e640 1 -- start start 2024-09-12T03:17:06.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.243+0000 7f3a1ec1e640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f3a1815eb60 0x7f3a180d1280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:06.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.243+0000 7f3a1ec1e640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3a180b6690 con 0x7f3a180d17c0 2024-09-12T03:17:06.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.243+0000 7f3a1ec1e640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3a180b6970 con 0x7f3a1815e780 2024-09-12T03:17:06.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.243+0000 7f3a1ec1e640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f3a180d1c30 con 0x7f3a1815eb60 2024-09-12T03:17:06.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.243+0000 7f3a1d1bb640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f3a180d17c0 0x7f3a180b3390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:44362/0 (socket says 172.21.15.27:44362) 2024-09-12T03:17:06.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.243+0000 7f3a1d1bb640 1 -- 172.21.15.27:0/2272094110 learned_addr learned my addr 172.21.15.27:0/2272094110 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:17:06.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.243+0000 7f3a17fff640 1 --2- 172.21.15.27:0/2272094110 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f3a1815eb60 0x7f3a180d1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:06.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.243+0000 7f3a177fe640 1 -- 172.21.15.27:0/2272094110 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 336215398 0 0) 0x7f3a180b6690 con 0x7f3a180d17c0 2024-09-12T03:17:06.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.244+0000 7f3a177fe640 1 -- 172.21.15.27:0/2272094110 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3a00003520 con 0x7f3a180d17c0 2024-09-12T03:17:06.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.244+0000 7f3a177fe640 1 -- 172.21.15.27:0/2272094110 <== mon.1 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1826092912 0 0) 0x7f3a180b6970 con 0x7f3a1815e780 2024-09-12T03:17:06.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.244+0000 7f3a177fe640 1 -- 172.21.15.27:0/2272094110 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3a180b6690 con 0x7f3a1815e780 2024-09-12T03:17:06.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.244+0000 7f3a177fe640 1 -- 172.21.15.27:0/2272094110 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3556157473 0 0) 0x7f3a00003520 con 0x7f3a180d17c0 2024-09-12T03:17:06.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.244+0000 7f3a17fff640 1 -- 172.21.15.27:0/2272094110 >> v1:172.21.15.45:6789/0 conn(0x7f3a1815e780 legacy=0x7f3a180d3e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:06.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.244+0000 7f3a17fff640 1 -- 172.21.15.27:0/2272094110 >> v1:172.21.15.27:6789/0 conn(0x7f3a180d17c0 legacy=0x7f3a180b3390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:06.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.244+0000 7f3a17fff640 1 -- 172.21.15.27:0/2272094110 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a180b6c50 con 0x7f3a1815eb60 2024-09-12T03:17:06.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.245+0000 7f3a17fff640 1 --2- 172.21.15.27:0/2272094110 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f3a1815eb60 0x7f3a180d1280 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f3a0c0097d0 tx=0x7f3a0c02a1a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a0f0d5e74315bc0c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:06.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.245+0000 7f3a177fe640 1 -- 172.21.15.27:0/2272094110 <== mon.2 v2:172.21.15.45:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a0c037070 con 0x7f3a1815eb60 2024-09-12T03:17:06.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.245+0000 7f3a177fe640 1 -- 172.21.15.27:0/2272094110 <== mon.2 v2:172.21.15.45:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a0c02ac30 con 0x7f3a1815eb60 2024-09-12T03:17:06.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.245+0000 7f3a177fe640 1 -- 172.21.15.27:0/2272094110 <== mon.2 v2:172.21.15.45:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a100019f0 con 0x7f3a1815eb60 2024-09-12T03:17:06.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.245+0000 7f3a1ec1e640 1 -- 172.21.15.27:0/2272094110 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f3a1815eb60 msgr2=0x7f3a180d1280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:06.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.245+0000 7f3a1ec1e640 1 --2- 172.21.15.27:0/2272094110 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f3a1815eb60 0x7f3a180d1280 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f3a0c0097d0 tx=0x7f3a0c02a1a0 comp rx=0 tx=0).stop 2024-09-12T03:17:06.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.245+0000 7f3a1ec1e640 1 -- 172.21.15.27:0/2272094110 shutdown_connections 2024-09-12T03:17:06.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.245+0000 7f3a1ec1e640 1 --2- 172.21.15.27:0/2272094110 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f3a1815eb60 0x7f3a180d1280 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:06.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.245+0000 7f3a1ec1e640 1 -- 172.21.15.27:0/2272094110 >> 172.21.15.27:0/2272094110 conn(0x7f3a1806da60 msgr2=0x7f3a1806de70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:06.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.245+0000 7f3a1ec1e640 1 -- 172.21.15.27:0/2272094110 shutdown_connections 2024-09-12T03:17:06.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.245+0000 7f3a1ec1e640 1 -- 172.21.15.27:0/2272094110 wait complete. 2024-09-12T03:17:06.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.247+0000 7f3a1ec1e640 1 Processor -- start 2024-09-12T03:17:06.248 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.247+0000 7f3a1ec1e640 1 -- start start 2024-09-12T03:17:06.249 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.248+0000 7f3a1ec1e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3a180d17c0 0x7f3a180ce180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:06.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.248+0000 7f3a1ec1e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3a1815e780 0x7f3a180cb990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:06.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.248+0000 7f3a1ec1e640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f3a1815eb60 0x7f3a180c9460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:06.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.248+0000 7f3a1ec1e640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f3a180b7210 con 0x7f3a180d17c0 2024-09-12T03:17:06.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.248+0000 7f3a1ec1e640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f3a180b6340 con 0x7f3a1815e780 2024-09-12T03:17:06.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.248+0000 7f3a1ec1e640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f3a180d4580 con 0x7f3a1815eb60 2024-09-12T03:17:06.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.248+0000 7f3a1c9ba640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3a180d17c0 0x7f3a180ce180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:06.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.248+0000 7f3a1c9ba640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3a180d17c0 0x7f3a180ce180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58178/0 (socket says 172.21.15.27:58178) 2024-09-12T03:17:06.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.248+0000 7f3a1c9ba640 1 -- 172.21.15.27:0/1047830387 learned_addr learned my addr 172.21.15.27:0/1047830387 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:17:06.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.248+0000 7f3a17fff640 1 --2- 172.21.15.27:0/1047830387 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3a1815e780 0x7f3a180cb990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:06.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.248+0000 7f3a1d1bb640 1 --2- 172.21.15.27:0/1047830387 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f3a1815eb60 0x7f3a180c9460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:06.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.248+0000 7f3a1c9ba640 1 -- 172.21.15.27:0/1047830387 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f3a1815eb60 msgr2=0x7f3a180c9460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:06.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.248+0000 7f3a1c9ba640 1 --2- 172.21.15.27:0/1047830387 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f3a1815eb60 0x7f3a180c9460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:06.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.248+0000 7f3a1c9ba640 1 -- 172.21.15.27:0/1047830387 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3a1815e780 msgr2=0x7f3a180cb990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:06.251 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.248+0000 7f3a1c9ba640 1 --2- 172.21.15.27:0/1047830387 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3a1815e780 0x7f3a180cb990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:06.251 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.248+0000 7f3a1c9ba640 1 -- 172.21.15.27:0/1047830387 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a180cc2e0 con 0x7f3a180d17c0 2024-09-12T03:17:06.251 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.248+0000 7f3a1d1bb640 1 --2- 172.21.15.27:0/1047830387 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f3a1815eb60 0x7f3a180c9460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:17:06.251 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.248+0000 7f3a1c9ba640 1 --2- 172.21.15.27:0/1047830387 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3a180d17c0 0x7f3a180ce180 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f3a0800c490 tx=0x7f3a0800c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:06.251 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.248+0000 7f3a157fa640 1 -- 172.21.15.27:0/1047830387 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a080052d0 con 0x7f3a180d17c0 2024-09-12T03:17:06.251 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.248+0000 7f3a157fa640 1 -- 172.21.15.27:0/1047830387 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a08005470 con 0x7f3a180d17c0 2024-09-12T03:17:06.251 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.248+0000 7f3a1ec1e640 1 -- 172.21.15.27:0/1047830387 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a180cc5c0 con 0x7f3a180d17c0 2024-09-12T03:17:06.251 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.248+0000 7f3a157fa640 1 -- 172.21.15.27:0/1047830387 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a08012850 con 0x7f3a180d17c0 2024-09-12T03:17:06.251 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.248+0000 7f3a1ec1e640 1 -- 172.21.15.27:0/1047830387 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a180cc8a0 con 0x7f3a180d17c0 2024-09-12T03:17:06.251 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.249+0000 7f3a1ec1e640 1 -- 172.21.15.27:0/1047830387 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a180d1280 con 0x7f3a180d17c0 2024-09-12T03:17:06.251 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.250+0000 7f3a157fa640 1 -- 172.21.15.27:0/1047830387 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3a080129f0 con 0x7f3a180d17c0 2024-09-12T03:17:06.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.250+0000 7f3a157fa640 1 --2- 172.21.15.27:0/1047830387 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f3a0406be60 0x7f3a0406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:06.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.250+0000 7f3a157fa640 1 -- 172.21.15.27:0/1047830387 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 4857+0+0 (secure 0 0 0) 0x7f3a0808a670 con 0x7f3a180d17c0 2024-09-12T03:17:06.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.251+0000 7f3a17fff640 1 --2- 172.21.15.27:0/1047830387 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f3a0406be60 0x7f3a0406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:06.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.251+0000 7f3a17fff640 1 --2- 172.21.15.27:0/1047830387 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f3a0406be60 0x7f3a0406e320 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f3a0c034000 tx=0x7f3a0c02e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:06.254 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.255+0000 7f3a157fa640 1 -- 172.21.15.27:0/1047830387 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f3a08059400 con 0x7f3a180d17c0 2024-09-12T03:17:06.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.499+0000 7f3a1ec1e640 1 -- 172.21.15.27:0/1047830387 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) -- 0x7f3a180c9080 con 0x7f3a180d17c0 2024-09-12T03:17:06.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.501+0000 7f3a157fa640 1 -- 172.21.15.27:0/1047830387 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v5) ==== 90+0+0 (secure 0 0 0) 0x7f3a08058d00 con 0x7f3a180d17c0 2024-09-12T03:17:06.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.503+0000 7f3a1ec1e640 1 -- 172.21.15.27:0/1047830387 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f3a0406be60 msgr2=0x7f3a0406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:06.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.503+0000 7f3a1ec1e640 1 --2- 172.21.15.27:0/1047830387 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f3a0406be60 0x7f3a0406e320 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f3a0c034000 tx=0x7f3a0c02e000 comp rx=0 tx=0).stop 2024-09-12T03:17:06.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.503+0000 7f3a1ec1e640 1 -- 172.21.15.27:0/1047830387 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3a180d17c0 msgr2=0x7f3a180ce180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:06.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.503+0000 7f3a1ec1e640 1 --2- 172.21.15.27:0/1047830387 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3a180d17c0 0x7f3a180ce180 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f3a0800c490 tx=0x7f3a0800c950 comp rx=0 tx=0).stop 2024-09-12T03:17:06.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.503+0000 7f3a1ec1e640 1 -- 172.21.15.27:0/1047830387 shutdown_connections 2024-09-12T03:17:06.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.503+0000 7f3a1ec1e640 1 --2- 172.21.15.27:0/1047830387 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f3a0406be60 0x7f3a0406e320 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:06.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.503+0000 7f3a1ec1e640 1 --2- 172.21.15.27:0/1047830387 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f3a1815eb60 0x7f3a180c9460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:06.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.503+0000 7f3a1ec1e640 1 --2- 172.21.15.27:0/1047830387 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3a1815e780 0x7f3a180cb990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:06.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.503+0000 7f3a1ec1e640 1 --2- 172.21.15.27:0/1047830387 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3a180d17c0 0x7f3a180ce180 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:06.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.503+0000 7f3a1ec1e640 1 -- 172.21.15.27:0/1047830387 >> 172.21.15.27:0/1047830387 conn(0x7f3a1806da60 msgr2=0x7f3a1815f000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:06.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.503+0000 7f3a1ec1e640 1 -- 172.21.15.27:0/1047830387 shutdown_connections 2024-09-12T03:17:06.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.503+0000 7f3a1ec1e640 1 -- 172.21.15.27:0/1047830387 wait complete. 2024-09-12T03:17:06.514 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2024-09-12T03:17:06.618 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.618+0000 7f7ea88a6640 1 Processor -- start 2024-09-12T03:17:06.618 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.618+0000 7f7ea88a6640 1 -- start start 2024-09-12T03:17:06.620 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.619+0000 7f7ea88a6640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f7ea40b1ca0 0x7f7ea415f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:06.620 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.619+0000 7f7ea88a6640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f7ea40cf660 0x7f7ea40d2780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:06.620 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.619+0000 7f7ea88a6640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7ea415fd70 con 0x7f7ea406f360 2024-09-12T03:17:06.620 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.619+0000 7f7ea88a6640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f7ea40d22f0 con 0x7f7ea40cf660 2024-09-12T03:17:06.620 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.619+0000 7f7ea88a6640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f7ea406cc40 con 0x7f7ea40b1ca0 2024-09-12T03:17:06.620 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.619+0000 7f7ea1d9b640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f7ea406f360 0x7f7ea40cef50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:44366/0 (socket says 172.21.15.27:44366) 2024-09-12T03:17:06.620 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.619+0000 7f7ea1d9b640 1 -- 172.21.15.27:0/2440162831 learned_addr learned my addr 172.21.15.27:0/2440162831 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:17:06.620 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.619+0000 7f7ea2d9d640 1 --2- 172.21.15.27:0/2440162831 >> v2:172.21.15.45:3300/0 conn(0x7f7ea40cf660 0x7f7ea40d2780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:06.621 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.619+0000 7f7ea259c640 1 --2- 172.21.15.27:0/2440162831 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f7ea40b1ca0 0x7f7ea415f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:06.621 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.619+0000 7f7ea159a640 1 -- 172.21.15.27:0/2440162831 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3755591736 0 0) 0x7f7e98001110 con 0x7f7ea406f360 2024-09-12T03:17:06.621 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.620+0000 7f7ea159a640 1 -- 172.21.15.27:0/2440162831 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7e8c003520 con 0x7f7ea406f360 2024-09-12T03:17:06.621 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.620+0000 7f7ea2d9d640 1 -- 172.21.15.27:0/2440162831 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f7ea40b1ca0 msgr2=0x7f7ea415f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:06.621 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.620+0000 7f7ea2d9d640 1 --2- 172.21.15.27:0/2440162831 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f7ea40b1ca0 0x7f7ea415f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:06.621 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.620+0000 7f7ea2d9d640 1 -- 172.21.15.27:0/2440162831 >> v1:172.21.15.27:6789/0 conn(0x7f7ea406f360 legacy=0x7f7ea40cef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:06.621 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.620+0000 7f7ea2d9d640 1 -- 172.21.15.27:0/2440162831 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ea40d1ce0 con 0x7f7ea40cf660 2024-09-12T03:17:06.622 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.620+0000 7f7ea2d9d640 1 --2- 172.21.15.27:0/2440162831 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7ea40cf660 0x7f7ea40d2780 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f7e9c009dc0 tx=0x7f7e9c02a680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=99830efb708f7035 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:06.622 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.621+0000 7f7ea159a640 1 -- 172.21.15.27:0/2440162831 <== mon.1 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e9c02ae60 con 0x7f7ea40cf660 2024-09-12T03:17:06.622 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.621+0000 7f7ea159a640 1 -- 172.21.15.27:0/2440162831 <== mon.1 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e9c007740 con 0x7f7ea40cf660 2024-09-12T03:17:06.622 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.621+0000 7f7ea159a640 1 -- 172.21.15.27:0/2440162831 <== mon.1 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e9c007b10 con 0x7f7ea40cf660 2024-09-12T03:17:06.622 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.621+0000 7f7ea88a6640 1 -- 172.21.15.27:0/2440162831 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7ea40cf660 msgr2=0x7f7ea40d2780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:06.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.621+0000 7f7ea88a6640 1 --2- 172.21.15.27:0/2440162831 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7ea40cf660 0x7f7ea40d2780 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f7e9c009dc0 tx=0x7f7e9c02a680 comp rx=0 tx=0).stop 2024-09-12T03:17:06.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.622+0000 7f7ea88a6640 1 -- 172.21.15.27:0/2440162831 shutdown_connections 2024-09-12T03:17:06.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.622+0000 7f7ea88a6640 1 --2- 172.21.15.27:0/2440162831 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7ea40cf660 0x7f7ea40d2780 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:06.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.622+0000 7f7ea88a6640 1 --2- 172.21.15.27:0/2440162831 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f7ea40b1ca0 0x7f7ea415f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:06.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.622+0000 7f7ea88a6640 1 -- 172.21.15.27:0/2440162831 >> 172.21.15.27:0/2440162831 conn(0x7f7ea4073800 msgr2=0x7f7ea409ba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:06.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.622+0000 7f7ea88a6640 1 -- 172.21.15.27:0/2440162831 shutdown_connections 2024-09-12T03:17:06.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.622+0000 7f7ea88a6640 1 -- 172.21.15.27:0/2440162831 wait complete. 2024-09-12T03:17:06.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.623+0000 7f7ea88a6640 1 Processor -- start 2024-09-12T03:17:06.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.623+0000 7f7ea88a6640 1 -- start start 2024-09-12T03:17:06.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.624+0000 7f7ea88a6640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7ea406f360 0x7f7ea415e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:06.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.624+0000 7f7ea88a6640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f7ea40b1ca0 0x7f7ea40ce250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:06.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.624+0000 7f7ea88a6640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7ea40cf660 0x7f7ea40701d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:06.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.624+0000 7f7ea88a6640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f7ea406d1c0 con 0x7f7ea406f360 2024-09-12T03:17:06.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.624+0000 7f7ea88a6640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f7ea40d1f80 con 0x7f7ea40cf660 2024-09-12T03:17:06.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.624+0000 7f7ea88a6640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f7ea406d340 con 0x7f7ea40b1ca0 2024-09-12T03:17:06.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.624+0000 7f7ea259c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7ea406f360 0x7f7ea415e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:06.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.624+0000 7f7ea259c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7ea406f360 0x7f7ea415e5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58190/0 (socket says 172.21.15.27:58190) 2024-09-12T03:17:06.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.624+0000 7f7ea259c640 1 -- 172.21.15.27:0/1320463579 learned_addr learned my addr 172.21.15.27:0/1320463579 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:17:06.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.624+0000 7f7ea1d9b640 1 --2- 172.21.15.27:0/1320463579 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f7ea40b1ca0 0x7f7ea40ce250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:06.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.624+0000 7f7ea2d9d640 1 --2- 172.21.15.27:0/1320463579 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7ea40cf660 0x7f7ea40701d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:06.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.624+0000 7f7ea259c640 1 -- 172.21.15.27:0/1320463579 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f7ea40b1ca0 msgr2=0x7f7ea40ce250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:06.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.624+0000 7f7ea259c640 1 --2- 172.21.15.27:0/1320463579 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f7ea40b1ca0 0x7f7ea40ce250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:06.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.624+0000 7f7ea259c640 1 -- 172.21.15.27:0/1320463579 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7ea40cf660 msgr2=0x7f7ea40701d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:06.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.624+0000 7f7ea259c640 1 --2- 172.21.15.27:0/1320463579 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7ea40cf660 0x7f7ea40701d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:06.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.624+0000 7f7ea259c640 1 -- 172.21.15.27:0/1320463579 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ea415d550 con 0x7f7ea406f360 2024-09-12T03:17:06.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.624+0000 7f7ea259c640 1 --2- 172.21.15.27:0/1320463579 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7ea406f360 0x7f7ea415e5d0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f7e9400e9b0 tx=0x7f7e9400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:06.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.625+0000 7f7e92ffd640 1 -- 172.21.15.27:0/1320463579 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e94017070 con 0x7f7ea406f360 2024-09-12T03:17:06.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.625+0000 7f7e92ffd640 1 -- 172.21.15.27:0/1320463579 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e9400c9a0 con 0x7f7ea406f360 2024-09-12T03:17:06.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.625+0000 7f7ea88a6640 1 -- 172.21.15.27:0/1320463579 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ea415d830 con 0x7f7ea406f360 2024-09-12T03:17:06.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.625+0000 7f7e92ffd640 1 -- 172.21.15.27:0/1320463579 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e9400cd90 con 0x7f7ea406f360 2024-09-12T03:17:06.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.625+0000 7f7ea88a6640 1 -- 172.21.15.27:0/1320463579 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ea415ddf0 con 0x7f7ea406f360 2024-09-12T03:17:06.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.626+0000 7f7ea88a6640 1 -- 172.21.15.27:0/1320463579 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ea40cfc40 con 0x7f7ea406f360 2024-09-12T03:17:06.628 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.626+0000 7f7e92ffd640 1 -- 172.21.15.27:0/1320463579 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7e9401a070 con 0x7f7ea406f360 2024-09-12T03:17:06.628 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.627+0000 7f7e92ffd640 1 --2- 172.21.15.27:0/1320463579 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f7e8406be30 0x7f7e8406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:06.628 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.627+0000 7f7e92ffd640 1 -- 172.21.15.27:0/1320463579 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 4857+0+0 (secure 0 0 0) 0x7f7e9408ae40 con 0x7f7ea406f360 2024-09-12T03:17:06.628 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.627+0000 7f7ea1d9b640 1 --2- 172.21.15.27:0/1320463579 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f7e8406be30 0x7f7e8406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:06.628 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.628+0000 7f7ea1d9b640 1 --2- 172.21.15.27:0/1320463579 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f7e8406be30 0x7f7e8406e2f0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f7e98005880 tx=0x7f7e98009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:06.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.631+0000 7f7e92ffd640 1 -- 172.21.15.27:0/1320463579 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f7e94059500 con 0x7f7ea406f360 2024-09-12T03:17:06.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.874+0000 7f7ea88a6640 1 -- 172.21.15.27:0/1320463579 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) -- 0x7f7ea4161750 con 0x7f7ea406f360 2024-09-12T03:17:06.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.876+0000 7f7e92ffd640 1 -- 172.21.15.27:0/1320463579 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v6) ==== 90+0+0 (secure 0 0 0) 0x7f7e9405d800 con 0x7f7ea406f360 2024-09-12T03:17:06.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.878+0000 7f7ea88a6640 1 -- 172.21.15.27:0/1320463579 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f7e8406be30 msgr2=0x7f7e8406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:06.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.878+0000 7f7ea88a6640 1 --2- 172.21.15.27:0/1320463579 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f7e8406be30 0x7f7e8406e2f0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f7e98005880 tx=0x7f7e98009040 comp rx=0 tx=0).stop 2024-09-12T03:17:06.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.878+0000 7f7ea88a6640 1 -- 172.21.15.27:0/1320463579 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7ea406f360 msgr2=0x7f7ea415e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:06.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.878+0000 7f7ea88a6640 1 --2- 172.21.15.27:0/1320463579 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7ea406f360 0x7f7ea415e5d0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f7e9400e9b0 tx=0x7f7e9400ee70 comp rx=0 tx=0).stop 2024-09-12T03:17:06.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.878+0000 7f7ea88a6640 1 -- 172.21.15.27:0/1320463579 shutdown_connections 2024-09-12T03:17:06.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.878+0000 7f7ea88a6640 1 --2- 172.21.15.27:0/1320463579 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f7e8406be30 0x7f7e8406e2f0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:06.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.878+0000 7f7ea88a6640 1 --2- 172.21.15.27:0/1320463579 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7ea40cf660 0x7f7ea40701d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:06.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.878+0000 7f7ea88a6640 1 --2- 172.21.15.27:0/1320463579 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f7ea40b1ca0 0x7f7ea40ce250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:06.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.878+0000 7f7ea88a6640 1 --2- 172.21.15.27:0/1320463579 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7ea406f360 0x7f7ea415e5d0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:06.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.878+0000 7f7ea88a6640 1 -- 172.21.15.27:0/1320463579 >> 172.21.15.27:0/1320463579 conn(0x7f7ea4073800 msgr2=0x7f7ea415bd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:06.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.878+0000 7f7ea88a6640 1 -- 172.21.15.27:0/1320463579 shutdown_connections 2024-09-12T03:17:06.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.878+0000 7f7ea88a6640 1 -- 172.21.15.27:0/1320463579 wait complete. 2024-09-12T03:17:06.889 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2024-09-12T03:17:06.993 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.993+0000 7fb78574b640 1 Processor -- start 2024-09-12T03:17:06.993 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.993+0000 7fb78574b640 1 -- start start 2024-09-12T03:17:06.994 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.994+0000 7fb78574b640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb78015b2b0 0x7fb7800b3560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:06.994 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.994+0000 7fb78574b640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fb780072d70 0x7fb780072430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:06.995 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.994+0000 7fb78574b640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb7800ce410 con 0x7fb78006e9b0 2024-09-12T03:17:06.995 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.994+0000 7fb78574b640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fb7800720a0 con 0x7fb780072d70 2024-09-12T03:17:06.995 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.994+0000 7fb78574b640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fb780072970 con 0x7fb78015b2b0 2024-09-12T03:17:06.995 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.994+0000 7fb77effd640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb78015b2b0 0x7fb7800b3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:06.995 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.994+0000 7fb77e7fc640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fb780072d70 0x7fb780072430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:06.995 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.994+0000 7fb77effd640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb78015b2b0 0x7fb7800b3560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3301/0 says I am v2:172.21.15.27:58152/0 (socket says 172.21.15.27:58152) 2024-09-12T03:17:06.995 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.994+0000 7fb77effd640 1 -- 172.21.15.27:0/3873683727 learned_addr learned my addr 172.21.15.27:0/3873683727 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:17:06.995 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.995+0000 7fb77e7fc640 1 -- 172.21.15.27:0/3873683727 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb78015b2b0 msgr2=0x7fb7800b3560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:06.995 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.995+0000 7fb77e7fc640 1 --2- 172.21.15.27:0/3873683727 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb78015b2b0 0x7fb7800b3560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:06.995 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.995+0000 7fb77e7fc640 1 -- 172.21.15.27:0/3873683727 >> v1:172.21.15.45:6789/0 conn(0x7fb78006e9b0 legacy=0x7fb7800cebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:06.995 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.995+0000 7fb77e7fc640 1 -- 172.21.15.27:0/3873683727 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7800732c0 con 0x7fb780072d70 2024-09-12T03:17:06.995 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.995+0000 7fb77effd640 1 --2- 172.21.15.27:0/3873683727 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb78015b2b0 0x7fb7800b3560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:17:06.997 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.995+0000 7fb77e7fc640 1 --2- 172.21.15.27:0/3873683727 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb780072d70 0x7fb780072430 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fb7740097a0 tx=0x7fb77402a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4644694451a58854 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:06.997 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.995+0000 7fb77dffb640 1 -- 172.21.15.27:0/3873683727 <== mon.1 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb774037070 con 0x7fb780072d70 2024-09-12T03:17:06.997 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.996+0000 7fb77dffb640 1 -- 172.21.15.27:0/3873683727 <== mon.1 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb774032440 con 0x7fb780072d70 2024-09-12T03:17:06.997 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.996+0000 7fb77dffb640 1 -- 172.21.15.27:0/3873683727 <== mon.1 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb774032810 con 0x7fb780072d70 2024-09-12T03:17:06.997 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.996+0000 7fb78574b640 1 -- 172.21.15.27:0/3873683727 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb780072d70 msgr2=0x7fb780072430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:06.997 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.996+0000 7fb78574b640 1 --2- 172.21.15.27:0/3873683727 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb780072d70 0x7fb780072430 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fb7740097a0 tx=0x7fb77402a180 comp rx=0 tx=0).stop 2024-09-12T03:17:06.997 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.996+0000 7fb78574b640 1 -- 172.21.15.27:0/3873683727 shutdown_connections 2024-09-12T03:17:06.997 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.996+0000 7fb78574b640 1 --2- 172.21.15.27:0/3873683727 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb780072d70 0x7fb780072430 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:06.997 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.996+0000 7fb78574b640 1 --2- 172.21.15.27:0/3873683727 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb78015b2b0 0x7fb7800b3560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:06.997 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.996+0000 7fb78574b640 1 -- 172.21.15.27:0/3873683727 >> 172.21.15.27:0/3873683727 conn(0x7fb780062c60 msgr2=0x7fb78006bb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:06.997 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.996+0000 7fb78574b640 1 -- 172.21.15.27:0/3873683727 shutdown_connections 2024-09-12T03:17:06.998 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.996+0000 7fb78574b640 1 -- 172.21.15.27:0/3873683727 wait complete. 2024-09-12T03:17:06.999 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.997+0000 7fb78574b640 1 Processor -- start 2024-09-12T03:17:07.000 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.998+0000 7fb78574b640 1 -- start start 2024-09-12T03:17:07.000 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.998+0000 7fb78574b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb78006e9b0 0x7fb780157330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:07.000 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.998+0000 7fb78574b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb780072d70 0x7fb780150ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:07.000 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.998+0000 7fb78574b640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb78015b2b0 0x7fb7801e6020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:07.000 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.998+0000 7fb78574b640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fb7800ce690 con 0x7fb78006e9b0 2024-09-12T03:17:07.000 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.998+0000 7fb78574b640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb780071dc0 con 0x7fb780072d70 2024-09-12T03:17:07.000 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.998+0000 7fb78574b640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fb78015b690 con 0x7fb78015b2b0 2024-09-12T03:17:07.000 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.998+0000 7fb77effd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb78006e9b0 0x7fb780157330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:07.000 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.998+0000 7fb77effd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb78006e9b0 0x7fb780157330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58194/0 (socket says 172.21.15.27:58194) 2024-09-12T03:17:07.000 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.998+0000 7fb77effd640 1 -- 172.21.15.27:0/665362570 learned_addr learned my addr 172.21.15.27:0/665362570 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:17:07.000 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.999+0000 7fb77e7fc640 1 --2- 172.21.15.27:0/665362570 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb780072d70 0x7fb780150ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:07.000 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.999+0000 7fb77f7fe640 1 --2- 172.21.15.27:0/665362570 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb78015b2b0 0x7fb7801e6020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:07.001 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.999+0000 7fb77effd640 1 -- 172.21.15.27:0/665362570 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb78015b2b0 msgr2=0x7fb7801e6020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:07.001 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.999+0000 7fb77effd640 1 --2- 172.21.15.27:0/665362570 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb78015b2b0 0x7fb7801e6020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:07.001 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.999+0000 7fb77effd640 1 -- 172.21.15.27:0/665362570 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb780072d70 msgr2=0x7fb780150ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:07.001 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.999+0000 7fb77effd640 1 --2- 172.21.15.27:0/665362570 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb780072d70 0x7fb780150ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:07.001 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.999+0000 7fb77effd640 1 -- 172.21.15.27:0/665362570 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb780151020 con 0x7fb78006e9b0 2024-09-12T03:17:07.001 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.999+0000 7fb77effd640 1 --2- 172.21.15.27:0/665362570 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb78006e9b0 0x7fb780157330 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fb770010e00 tx=0x7fb77000e730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:07.001 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.999+0000 7fb7677fe640 1 -- 172.21.15.27:0/665362570 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb77001e070 con 0x7fb78006e9b0 2024-09-12T03:17:07.001 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.999+0000 7fb7677fe640 1 -- 172.21.15.27:0/665362570 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb770011860 con 0x7fb78006e9b0 2024-09-12T03:17:07.001 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.999+0000 7fb78574b640 1 -- 172.21.15.27:0/665362570 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb780155320 con 0x7fb78006e9b0 2024-09-12T03:17:07.001 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.999+0000 7fb7677fe640 1 -- 172.21.15.27:0/665362570 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb770011c10 con 0x7fb78006e9b0 2024-09-12T03:17:07.001 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:06.999+0000 7fb78574b640 1 -- 172.21.15.27:0/665362570 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7801558e0 con 0x7fb78006e9b0 2024-09-12T03:17:07.002 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:07.000+0000 7fb78574b640 1 -- 172.21.15.27:0/665362570 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb780072430 con 0x7fb78006e9b0 2024-09-12T03:17:07.002 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:07.001+0000 7fb7677fe640 1 -- 172.21.15.27:0/665362570 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb770021070 con 0x7fb78006e9b0 2024-09-12T03:17:07.002 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:07.001+0000 7fb7677fe640 1 --2- 172.21.15.27:0/665362570 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fb76c06be30 0x7fb76c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:07.002 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:07.001+0000 7fb7677fe640 1 -- 172.21.15.27:0/665362570 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 4857+0+0 (secure 0 0 0) 0x7fb77008d550 con 0x7fb78006e9b0 2024-09-12T03:17:07.002 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:07.001+0000 7fb77e7fc640 1 --2- 172.21.15.27:0/665362570 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fb76c06be30 0x7fb76c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:07.002 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:07.002+0000 7fb77e7fc640 1 --2- 172.21.15.27:0/665362570 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fb76c06be30 0x7fb76c06e2f0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fb774034000 tx=0x7fb77402e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:07.005 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:07.005+0000 7fb7677fe640 1 -- 172.21.15.27:0/665362570 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fb780151020 con 0x7fb78006e9b0 2024-09-12T03:17:07.248 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:07.248+0000 7fb78574b640 1 -- 172.21.15.27:0/665362570 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) -- 0x7fb78006f0f0 con 0x7fb78006e9b0 2024-09-12T03:17:07.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:07.250+0000 7fb7677fe640 1 -- 172.21.15.27:0/665362570 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v7) ==== 90+0+0 (secure 0 0 0) 0x7fb77005bc00 con 0x7fb78006e9b0 2024-09-12T03:17:07.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:07.252+0000 7fb78574b640 1 -- 172.21.15.27:0/665362570 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fb76c06be30 msgr2=0x7fb76c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:07.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:07.252+0000 7fb78574b640 1 --2- 172.21.15.27:0/665362570 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fb76c06be30 0x7fb76c06e2f0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fb774034000 tx=0x7fb77402e000 comp rx=0 tx=0).stop 2024-09-12T03:17:07.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:07.252+0000 7fb78574b640 1 -- 172.21.15.27:0/665362570 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb78006e9b0 msgr2=0x7fb780157330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:07.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:07.252+0000 7fb78574b640 1 --2- 172.21.15.27:0/665362570 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb78006e9b0 0x7fb780157330 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fb770010e00 tx=0x7fb77000e730 comp rx=0 tx=0).stop 2024-09-12T03:17:07.253 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:07.252+0000 7fb78574b640 1 -- 172.21.15.27:0/665362570 shutdown_connections 2024-09-12T03:17:07.253 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:07.252+0000 7fb78574b640 1 --2- 172.21.15.27:0/665362570 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fb76c06be30 0x7fb76c06e2f0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:07.253 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:07.252+0000 7fb78574b640 1 --2- 172.21.15.27:0/665362570 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb78015b2b0 0x7fb7801e6020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:07.253 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:07.252+0000 7fb78574b640 1 --2- 172.21.15.27:0/665362570 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb780072d70 0x7fb780150ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:07.253 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:07.252+0000 7fb78574b640 1 --2- 172.21.15.27:0/665362570 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb78006e9b0 0x7fb780157330 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:07.253 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:07.252+0000 7fb78574b640 1 -- 172.21.15.27:0/665362570 >> 172.21.15.27:0/665362570 conn(0x7fb780062c60 msgr2=0x7fb7800b1410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:07.253 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:07.252+0000 7fb78574b640 1 -- 172.21.15.27:0/665362570 shutdown_connections 2024-09-12T03:17:07.253 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:07.252+0000 7fb78574b640 1 -- 172.21.15.27:0/665362570 wait complete. 2024-09-12T03:17:07.263 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2024-09-12T03:17:07.367 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:07.366+0000 7f3c0c077640 1 Processor -- start 2024-09-12T03:17:07.367 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:07.367+0000 7f3c0c077640 1 -- start start 2024-09-12T03:17:07.368 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:07.367+0000 7f3c0c077640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f3c040ae8d0 0x7f3c040728b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:07.368 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:07.368+0000 7f3c0c077640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f3c04072df0 0x7f3c0406e930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:07.368 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:07.368+0000 7f3c0c077640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3c040aee90 con 0x7f3c040b5a70 2024-09-12T03:17:07.369 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:07.368+0000 7f3c0c077640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f3c04073240 con 0x7f3c04072df0 2024-09-12T03:17:07.369 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:07.368+0000 7f3c0c077640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f3c040af170 con 0x7f3c040ae8d0 2024-09-12T03:17:07.369 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:07.368+0000 7f3c0a614640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f3c04072df0 0x7f3c0406e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:07.369 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:07.368+0000 7f3c09e13640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f3c040b5a70 0x7f3c0415f600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:44380/0 (socket says 172.21.15.27:44380) 2024-09-12T03:17:07.369 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:07.368+0000 7f3c09e13640 1 -- 172.21.15.27:0/539547748 learned_addr learned my addr 172.21.15.27:0/539547748 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:17:07.369 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:07.368+0000 7f3c09612640 1 --2- 172.21.15.27:0/539547748 >> v2:172.21.15.45:3300/0 conn(0x7f3c040ae8d0 0x7f3c040728b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:07.369 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:07.368+0000 7f3c0a614640 1 -- 172.21.15.27:0/539547748 >> v2:172.21.15.45:3300/0 conn(0x7f3c040ae8d0 msgr2=0x7f3c040728b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:07.369 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:07.368+0000 7f3c0a614640 1 --2- 172.21.15.27:0/539547748 >> v2:172.21.15.45:3300/0 conn(0x7f3c040ae8d0 0x7f3c040728b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:07.369 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:07.368+0000 7f3c0a614640 1 -- 172.21.15.27:0/539547748 >> v1:172.21.15.27:6789/0 conn(0x7f3c040b5a70 legacy=0x7f3c0415f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:07.369 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:07.368+0000 7f3c0a614640 1 -- 172.21.15.27:0/539547748 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c040af450 con 0x7f3c04072df0 2024-09-12T03:17:07.369 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:07.368+0000 7f3c08e11640 1 -- 172.21.15.27:0/539547748 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2780774986 0 0) 0x7f3c040aee90 con 0x7f3c040b5a70 2024-09-12T03:17:07.369 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:07.368+0000 7f3c09612640 1 --2- 172.21.15.27:0/539547748 >> v2:172.21.15.45:3300/0 conn(0x7f3c040ae8d0 0x7f3c040728b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:17:07.370 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:07.369+0000 7f3c0a614640 1 --2- 172.21.15.27:0/539547748 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3c04072df0 0x7f3c0406e930 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f3c04062840 tx=0x7f3bfc02a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd4fb149c0c590e4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:07.371 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:07.369+0000 7f3c08e11640 1 -- 172.21.15.27:0/539547748 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bfc037070 con 0x7f3c04072df0 2024-09-12T03:17:07.371 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:07.369+0000 7f3c08e11640 1 -- 172.21.15.27:0/539547748 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3bfc032440 con 0x7f3c04072df0 2024-09-12T03:17:07.371 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:07.369+0000 7f3c08e11640 1 -- 172.21.15.27:0/539547748 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bfc02ae60 con 0x7f3c04072df0 2024-09-12T03:17:07.371 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:07.369+0000 7f3c0c077640 1 -- 172.21.15.27:0/539547748 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3c04072df0 msgr2=0x7f3c0406e930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:07.371 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:07.369+0000 7f3c0c077640 1 --2- 172.21.15.27:0/539547748 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3c04072df0 0x7f3c0406e930 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f3c04062840 tx=0x7f3bfc02a110 comp rx=0 tx=0).stop 2024-09-12T03:17:07.371 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:07.369+0000 7f3c0c077640 1 -- 172.21.15.27:0/539547748 shutdown_connections 2024-09-12T03:17:07.371 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:07.369+0000 7f3c0c077640 1 --2- 172.21.15.27:0/539547748 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3c04072df0 0x7f3c0406e930 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:07.371 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:07.369+0000 7f3c0c077640 1 --2- 172.21.15.27:0/539547748 >> v2:172.21.15.45:3300/0 conn(0x7f3c040ae8d0 0x7f3c040728b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:07.371 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:07.369+0000 7f3c0c077640 1 -- 172.21.15.27:0/539547748 >> 172.21.15.27:0/539547748 conn(0x7f3c0406ca30 msgr2=0x7f3c0406ce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:07.372 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:07.370+0000 7f3c0c077640 1 -- 172.21.15.27:0/539547748 shutdown_connections 2024-09-12T03:17:07.372 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:07.370+0000 7f3c0c077640 1 -- 172.21.15.27:0/539547748 wait complete. 2024-09-12T03:17:07.372 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:07.371+0000 7f3c0c077640 1 Processor -- start 2024-09-12T03:17:07.372 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:07.371+0000 7f3c0c077640 1 -- start start 2024-09-12T03:17:07.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:07.371+0000 7f3c0c077640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3c04072df0 0x7f3c0415b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:07.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:07.372+0000 7f3c0c077640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f3c040ae8d0 0x7f3c040ce340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:07.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:07.372+0000 7f3c0c077640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3c040b5a70 0x7f3c04156430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:07.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:07.372+0000 7f3c0c077640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f3c0406eff0 con 0x7f3c04072df0 2024-09-12T03:17:07.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:07.372+0000 7f3c0c077640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f3c0406ee70 con 0x7f3c040b5a70 2024-09-12T03:17:07.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:07.372+0000 7f3c0c077640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f3c0406f170 con 0x7f3c040ae8d0 2024-09-12T03:17:07.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:07.372+0000 7f3c09e13640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3c04072df0 0x7f3c0415b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:07.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:07.372+0000 7f3c09e13640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3c04072df0 0x7f3c0415b480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58212/0 (socket says 172.21.15.27:58212) 2024-09-12T03:17:07.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:07.372+0000 7f3c09e13640 1 -- 172.21.15.27:0/1038283600 learned_addr learned my addr 172.21.15.27:0/1038283600 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:17:07.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:07.372+0000 7f3c0a614640 1 --2- 172.21.15.27:0/1038283600 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3c040b5a70 0x7f3c04156430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:07.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:07.372+0000 7f3c09612640 1 --2- 172.21.15.27:0/1038283600 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f3c040ae8d0 0x7f3c040ce340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:07.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:07.372+0000 7f3c09e13640 1 -- 172.21.15.27:0/1038283600 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f3c040ae8d0 msgr2=0x7f3c040ce340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:07.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:07.372+0000 7f3c09e13640 1 --2- 172.21.15.27:0/1038283600 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f3c040ae8d0 0x7f3c040ce340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:07.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:07.372+0000 7f3c09e13640 1 -- 172.21.15.27:0/1038283600 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3c040b5a70 msgr2=0x7f3c04156430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:07.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:07.372+0000 7f3c09e13640 1 --2- 172.21.15.27:0/1038283600 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3c040b5a70 0x7f3c04156430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:07.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:07.372+0000 7f3c09e13640 1 -- 172.21.15.27:0/1038283600 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c0415bb10 con 0x7f3c04072df0 2024-09-12T03:17:07.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:07.372+0000 7f3c09612640 1 --2- 172.21.15.27:0/1038283600 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f3c040ae8d0 0x7f3c040ce340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:17:07.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:07.372+0000 7f3c09e13640 1 --2- 172.21.15.27:0/1038283600 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3c04072df0 0x7f3c0415b480 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f3bf400c7c0 tx=0x7f3bf400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:07.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:07.372+0000 7f3c027fc640 1 -- 172.21.15.27:0/1038283600 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bf4018070 con 0x7f3c04072df0 2024-09-12T03:17:07.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:07.372+0000 7f3c027fc640 1 -- 172.21.15.27:0/1038283600 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3bf400a640 con 0x7f3c04072df0 2024-09-12T03:17:07.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:07.372+0000 7f3c0c077640 1 -- 172.21.15.27:0/1038283600 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c0415bdf0 con 0x7f3c04072df0 2024-09-12T03:17:07.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:07.372+0000 7f3c027fc640 1 -- 172.21.15.27:0/1038283600 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bf400aa30 con 0x7f3c04072df0 2024-09-12T03:17:07.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:07.372+0000 7f3c0c077640 1 -- 172.21.15.27:0/1038283600 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c040ced40 con 0x7f3c04072df0 2024-09-12T03:17:07.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:07.373+0000 7f3c0c077640 1 -- 172.21.15.27:0/1038283600 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c0406e910 con 0x7f3c04072df0 2024-09-12T03:17:07.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:07.374+0000 7f3c027fc640 1 -- 172.21.15.27:0/1038283600 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3bf401b070 con 0x7f3c04072df0 2024-09-12T03:17:07.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:07.374+0000 7f3c027fc640 1 --2- 172.21.15.27:0/1038283600 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f3bf006be60 0x7f3bf006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:07.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:07.374+0000 7f3c027fc640 1 -- 172.21.15.27:0/1038283600 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 4857+0+0 (secure 0 0 0) 0x7f3bf408a140 con 0x7f3c04072df0 2024-09-12T03:17:07.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:07.375+0000 7f3c09612640 1 --2- 172.21.15.27:0/1038283600 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f3bf006be60 0x7f3bf006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:07.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:07.375+0000 7f3c09612640 1 --2- 172.21.15.27:0/1038283600 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f3bf006be60 0x7f3bf006e320 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f3bf8006fd0 tx=0x7f3bf8008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:07.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:07.378+0000 7f3c027fc640 1 -- 172.21.15.27:0/1038283600 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f3bf4058e40 con 0x7f3c04072df0 2024-09-12T03:17:07.620 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:07.620+0000 7f3c0c077640 1 -- 172.21.15.27:0/1038283600 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) -- 0x7f3c04162fe0 con 0x7f3c04072df0 2024-09-12T03:17:07.622 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:07.622+0000 7f3c027fc640 1 -- 172.21.15.27:0/1038283600 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v8) ==== 90+0+0 (secure 0 0 0) 0x7f3bf4058740 con 0x7f3c04072df0 2024-09-12T03:17:07.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:07.625+0000 7f3c0c077640 1 -- 172.21.15.27:0/1038283600 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f3bf006be60 msgr2=0x7f3bf006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:07.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:07.625+0000 7f3c0c077640 1 --2- 172.21.15.27:0/1038283600 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f3bf006be60 0x7f3bf006e320 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f3bf8006fd0 tx=0x7f3bf8008040 comp rx=0 tx=0).stop 2024-09-12T03:17:07.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:07.625+0000 7f3c0c077640 1 -- 172.21.15.27:0/1038283600 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3c04072df0 msgr2=0x7f3c0415b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:07.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:07.625+0000 7f3c0c077640 1 --2- 172.21.15.27:0/1038283600 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3c04072df0 0x7f3c0415b480 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f3bf400c7c0 tx=0x7f3bf400cc80 comp rx=0 tx=0).stop 2024-09-12T03:17:07.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:07.625+0000 7f3c0c077640 1 -- 172.21.15.27:0/1038283600 shutdown_connections 2024-09-12T03:17:07.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:07.625+0000 7f3c0c077640 1 --2- 172.21.15.27:0/1038283600 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f3bf006be60 0x7f3bf006e320 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:07.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:07.625+0000 7f3c0c077640 1 --2- 172.21.15.27:0/1038283600 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3c040b5a70 0x7f3c04156430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:07.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:07.625+0000 7f3c0c077640 1 --2- 172.21.15.27:0/1038283600 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f3c040ae8d0 0x7f3c040ce340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:07.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:07.625+0000 7f3c0c077640 1 --2- 172.21.15.27:0/1038283600 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3c04072df0 0x7f3c0415b480 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:07.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:07.625+0000 7f3c0c077640 1 -- 172.21.15.27:0/1038283600 >> 172.21.15.27:0/1038283600 conn(0x7f3c0406ca30 msgr2=0x7f3c04071ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:07.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:07.625+0000 7f3c0c077640 1 -- 172.21.15.27:0/1038283600 shutdown_connections 2024-09-12T03:17:07.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:07.625+0000 7f3c0c077640 1 -- 172.21.15.27:0/1038283600 wait complete. 2024-09-12T03:17:07.636 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2024-09-12T03:17:07.740 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:07.740+0000 7f9bf1c27640 1 Processor -- start 2024-09-12T03:17:07.740 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:07.740+0000 7f9bf1c27640 1 -- start start 2024-09-12T03:17:07.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:07.740+0000 7f9bf1c27640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f9bec15eb90 0x7f9bec0d2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:07.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:07.740+0000 7f9bf1c27640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f9bec0d1720 0x7f9bec0b8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:07.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:07.740+0000 7f9bf1c27640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9bec0d3760 con 0x7f9bec0d5200 2024-09-12T03:17:07.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:07.740+0000 7f9bf1c27640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f9bec0b7d20 con 0x7f9bec0d1720 2024-09-12T03:17:07.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:07.740+0000 7f9bf1c27640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f9bec0d42e0 con 0x7f9bec15eb90 2024-09-12T03:17:07.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:07.741+0000 7f9bebfff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f9bec0d1720 0x7f9bec0b8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:07.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:07.741+0000 7f9bebfff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f9bec0d1720 0x7f9bec0b8190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58226/0 (socket says 172.21.15.27:58226) 2024-09-12T03:17:07.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:07.741+0000 7f9bebfff640 1 -- 172.21.15.27:0/704613046 learned_addr learned my addr 172.21.15.27:0/704613046 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:17:07.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:07.741+0000 7f9beb7fe640 1 --2- 172.21.15.27:0/704613046 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f9bec15eb90 0x7f9bec0d2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:07.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:07.741+0000 7f9bebfff640 1 -- 172.21.15.27:0/704613046 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f9bec15eb90 msgr2=0x7f9bec0d2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:07.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:07.741+0000 7f9bebfff640 1 --2- 172.21.15.27:0/704613046 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f9bec15eb90 0x7f9bec0d2df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:07.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:07.741+0000 7f9bebfff640 1 -- 172.21.15.27:0/704613046 >> v1:172.21.15.45:6789/0 conn(0x7f9bec0d5200 legacy=0x7f9bec0d1010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:07.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:07.741+0000 7f9bebfff640 1 -- 172.21.15.27:0/704613046 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bec0d3a40 con 0x7f9bec0d1720 2024-09-12T03:17:07.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:07.742+0000 7f9bebfff640 1 --2- 172.21.15.27:0/704613046 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9bec0d1720 0x7f9bec0b8190 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f9be4009e50 tx=0x7f9be402a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e93eaec27fa65e3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:07.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:07.742+0000 7f9bea7fc640 1 -- 172.21.15.27:0/704613046 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9be4036070 con 0x7f9bec0d1720 2024-09-12T03:17:07.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:07.742+0000 7f9bea7fc640 1 -- 172.21.15.27:0/704613046 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9be4031600 con 0x7f9bec0d1720 2024-09-12T03:17:07.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:07.742+0000 7f9bea7fc640 1 -- 172.21.15.27:0/704613046 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9be40319d0 con 0x7f9bec0d1720 2024-09-12T03:17:07.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:07.742+0000 7f9bf1c27640 1 -- 172.21.15.27:0/704613046 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9bec0d1720 msgr2=0x7f9bec0b8190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:07.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:07.742+0000 7f9bf1c27640 1 --2- 172.21.15.27:0/704613046 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9bec0d1720 0x7f9bec0b8190 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f9be4009e50 tx=0x7f9be402a710 comp rx=0 tx=0).stop 2024-09-12T03:17:07.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:07.743+0000 7f9bf1c27640 1 -- 172.21.15.27:0/704613046 shutdown_connections 2024-09-12T03:17:07.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:07.743+0000 7f9bf1c27640 1 --2- 172.21.15.27:0/704613046 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9bec0d1720 0x7f9bec0b8190 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:07.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:07.743+0000 7f9bf1c27640 1 --2- 172.21.15.27:0/704613046 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f9bec15eb90 0x7f9bec0d2df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:07.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:07.743+0000 7f9bf1c27640 1 -- 172.21.15.27:0/704613046 >> 172.21.15.27:0/704613046 conn(0x7f9bec06dce0 msgr2=0x7f9bec06e0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:07.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:07.743+0000 7f9bf1c27640 1 -- 172.21.15.27:0/704613046 shutdown_connections 2024-09-12T03:17:07.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:07.743+0000 7f9bf1c27640 1 -- 172.21.15.27:0/704613046 wait complete. 2024-09-12T03:17:07.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:07.744+0000 7f9bf1c27640 1 Processor -- start 2024-09-12T03:17:07.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:07.744+0000 7f9bf1c27640 1 -- start start 2024-09-12T03:17:07.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:07.745+0000 7f9bf1c27640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9bec0d1720 0x7f9bec14ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:07.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:07.745+0000 7f9bf1c27640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f9bec0d5200 0x7f9bec0ca1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:07.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:07.745+0000 7f9bf1c27640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9bec15eb90 0x7f9bec15c150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:07.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:07.745+0000 7f9bf1c27640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f9bec0d3330 con 0x7f9bec15eb90 2024-09-12T03:17:07.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:07.745+0000 7f9bf1c27640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f9bec0d4000 con 0x7f9bec0d1720 2024-09-12T03:17:07.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:07.745+0000 7f9bf1c27640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f9bec0d34b0 con 0x7f9bec0d5200 2024-09-12T03:17:07.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:07.745+0000 7f9bebfff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9bec15eb90 0x7f9bec15c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:07.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:07.745+0000 7f9bebfff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9bec15eb90 0x7f9bec15c150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58234/0 (socket says 172.21.15.27:58234) 2024-09-12T03:17:07.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:07.745+0000 7f9bebfff640 1 -- 172.21.15.27:0/765398636 learned_addr learned my addr 172.21.15.27:0/765398636 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:17:07.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:07.745+0000 7f9beb7fe640 1 --2- 172.21.15.27:0/765398636 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9bec0d1720 0x7f9bec14ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:07.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:07.745+0000 7f9beaffd640 1 --2- 172.21.15.27:0/765398636 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f9bec0d5200 0x7f9bec0ca1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:07.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:07.745+0000 7f9bebfff640 1 -- 172.21.15.27:0/765398636 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f9bec0d5200 msgr2=0x7f9bec0ca1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:07.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:07.745+0000 7f9bebfff640 1 --2- 172.21.15.27:0/765398636 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f9bec0d5200 0x7f9bec0ca1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:07.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:07.745+0000 7f9bebfff640 1 -- 172.21.15.27:0/765398636 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9bec0d1720 msgr2=0x7f9bec14ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:07.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:07.745+0000 7f9bebfff640 1 --2- 172.21.15.27:0/765398636 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9bec0d1720 0x7f9bec14ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:07.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:07.745+0000 7f9bebfff640 1 -- 172.21.15.27:0/765398636 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bec0c8590 con 0x7f9bec15eb90 2024-09-12T03:17:07.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:07.746+0000 7f9bebfff640 1 --2- 172.21.15.27:0/765398636 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9bec15eb90 0x7f9bec15c150 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f9be4034040 tx=0x7f9be4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:07.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:07.746+0000 7f9bd3fff640 1 -- 172.21.15.27:0/765398636 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9be4036050 con 0x7f9bec15eb90 2024-09-12T03:17:07.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:07.746+0000 7f9bd3fff640 1 -- 172.21.15.27:0/765398636 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9be4042070 con 0x7f9bec15eb90 2024-09-12T03:17:07.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:07.746+0000 7f9bf1c27640 1 -- 172.21.15.27:0/765398636 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9bec0c8870 con 0x7f9bec15eb90 2024-09-12T03:17:07.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:07.746+0000 7f9bd3fff640 1 -- 172.21.15.27:0/765398636 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9be4009510 con 0x7f9bec15eb90 2024-09-12T03:17:07.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:07.746+0000 7f9bf1c27640 1 -- 172.21.15.27:0/765398636 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9bec0c8e30 con 0x7f9bec15eb90 2024-09-12T03:17:07.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:07.747+0000 7f9bf1c27640 1 -- 172.21.15.27:0/765398636 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9bec15e6d0 con 0x7f9bec15eb90 2024-09-12T03:17:07.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:07.747+0000 7f9bd3fff640 1 -- 172.21.15.27:0/765398636 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9be4007bf0 con 0x7f9bec15eb90 2024-09-12T03:17:07.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:07.748+0000 7f9bd3fff640 1 --2- 172.21.15.27:0/765398636 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f9bd806be60 0x7f9bd806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:07.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:07.748+0000 7f9bd3fff640 1 -- 172.21.15.27:0/765398636 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 4857+0+0 (secure 0 0 0) 0x7f9be40ab780 con 0x7f9bec15eb90 2024-09-12T03:17:07.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:07.748+0000 7f9beb7fe640 1 --2- 172.21.15.27:0/765398636 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f9bd806be60 0x7f9bd806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:07.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:07.749+0000 7f9beb7fe640 1 --2- 172.21.15.27:0/765398636 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f9bd806be60 0x7f9bd806e320 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f9bdc0079a0 tx=0x7f9bdc008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:07.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:07.752+0000 7f9bd3fff640 1 -- 172.21.15.27:0/765398636 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f9be4079f50 con 0x7f9bec15eb90 2024-09-12T03:17:07.999 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:07.999+0000 7f9bf1c27640 1 -- 172.21.15.27:0/765398636 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) -- 0x7f9bec15f090 con 0x7f9bec15eb90 2024-09-12T03:17:08.001 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.001+0000 7f9bd3fff640 1 -- 172.21.15.27:0/765398636 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v9) ==== 97+0+0 (secure 0 0 0) 0x7f9be407e250 con 0x7f9bec15eb90 2024-09-12T03:17:08.003 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.003+0000 7f9bf1c27640 1 -- 172.21.15.27:0/765398636 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f9bd806be60 msgr2=0x7f9bd806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:08.004 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.003+0000 7f9bf1c27640 1 --2- 172.21.15.27:0/765398636 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f9bd806be60 0x7f9bd806e320 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f9bdc0079a0 tx=0x7f9bdc008040 comp rx=0 tx=0).stop 2024-09-12T03:17:08.004 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.003+0000 7f9bf1c27640 1 -- 172.21.15.27:0/765398636 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9bec15eb90 msgr2=0x7f9bec15c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:08.004 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.003+0000 7f9bf1c27640 1 --2- 172.21.15.27:0/765398636 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9bec15eb90 0x7f9bec15c150 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f9be4034040 tx=0x7f9be4004a40 comp rx=0 tx=0).stop 2024-09-12T03:17:08.004 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.003+0000 7f9bf1c27640 1 -- 172.21.15.27:0/765398636 shutdown_connections 2024-09-12T03:17:08.004 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.003+0000 7f9bf1c27640 1 --2- 172.21.15.27:0/765398636 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f9bd806be60 0x7f9bd806e320 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:08.004 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.003+0000 7f9bf1c27640 1 --2- 172.21.15.27:0/765398636 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9bec15eb90 0x7f9bec15c150 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:08.004 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.003+0000 7f9bf1c27640 1 --2- 172.21.15.27:0/765398636 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f9bec0d5200 0x7f9bec0ca1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:08.004 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.003+0000 7f9bf1c27640 1 --2- 172.21.15.27:0/765398636 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9bec0d1720 0x7f9bec14ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:08.004 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.003+0000 7f9bf1c27640 1 -- 172.21.15.27:0/765398636 >> 172.21.15.27:0/765398636 conn(0x7f9bec06dce0 msgr2=0x7f9bec0b9180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:08.004 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.003+0000 7f9bf1c27640 1 -- 172.21.15.27:0/765398636 shutdown_connections 2024-09-12T03:17:08.004 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.003+0000 7f9bf1c27640 1 -- 172.21.15.27:0/765398636 wait complete. 2024-09-12T03:17:08.014 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2024-09-12T03:17:08.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.117+0000 7f4600ad7640 1 Processor -- start 2024-09-12T03:17:08.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.117+0000 7f4600ad7640 1 -- start start 2024-09-12T03:17:08.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.118+0000 7f4600ad7640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f45fc146010 0x7f45fc0b4750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:08.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.118+0000 7f4600ad7640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f45fc0b4d70 con 0x7f45fc145c30 2024-09-12T03:17:08.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.118+0000 7f4600ad7640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f45fc06d140 con 0x7f45fc0cda40 2024-09-12T03:17:08.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.118+0000 7f4600ad7640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f45fc0b5050 con 0x7f45fc146010 2024-09-12T03:17:08.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.118+0000 7f45fa59c640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f45fc145c30 0x7f45fc0cd330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:44386/0 (socket says 172.21.15.27:44386) 2024-09-12T03:17:08.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.118+0000 7f45fa59c640 1 -- 172.21.15.27:0/1739786657 learned_addr learned my addr 172.21.15.27:0/1739786657 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:17:08.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.118+0000 7f45fad9d640 1 --2- 172.21.15.27:0/1739786657 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f45fc146010 0x7f45fc0b4750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:08.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.118+0000 7f45f959a640 1 -- 172.21.15.27:0/1739786657 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1675706078 0 0) 0x7f45fc0b4d70 con 0x7f45fc145c30 2024-09-12T03:17:08.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.118+0000 7f45f959a640 1 -- 172.21.15.27:0/1739786657 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f45e8003520 con 0x7f45fc145c30 2024-09-12T03:17:08.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.119+0000 7f45f959a640 1 -- 172.21.15.27:0/1739786657 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 652119692 0 0) 0x7f45e8003520 con 0x7f45fc145c30 2024-09-12T03:17:08.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.119+0000 7f45f959a640 1 -- 172.21.15.27:0/1739786657 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f45fc146010 msgr2=0x7f45fc0b4750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:08.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.119+0000 7f45f959a640 1 --2- 172.21.15.27:0/1739786657 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f45fc146010 0x7f45fc0b4750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:08.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.119+0000 7f45f959a640 1 -- 172.21.15.27:0/1739786657 >> v1:172.21.15.45:6789/0 conn(0x7f45fc0cda40 legacy=0x7f45fc0b4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:08.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.119+0000 7f45f959a640 1 -- 172.21.15.27:0/1739786657 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45fc06d420 con 0x7f45fc145c30 2024-09-12T03:17:08.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.119+0000 7f45f959a640 1 -- 172.21.15.27:0/1739786657 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 847755213 0 0) 0x7f45f4001670 con 0x7f45fc145c30 2024-09-12T03:17:08.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.119+0000 7f45f959a640 1 -- 172.21.15.27:0/1739786657 >> v1:172.21.15.27:6789/0 conn(0x7f45fc145c30 legacy=0x7f45fc0cd330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:08.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.119+0000 7f45f959a640 1 --2- 172.21.15.27:0/1739786657 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f45e80039e0 0x7f45e8003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:08.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.119+0000 7f45f959a640 1 --2- 172.21.15.27:0/1739786657 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f45fc146010 0x7f45e8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:08.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.119+0000 7f45f959a640 1 --2- 172.21.15.27:0/1739786657 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f45e8004d40 0x7f45e80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:08.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.119+0000 7f45f959a640 1 -- 172.21.15.27:0/1739786657 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f45fc0b5050 con 0x7f45e8004d40 2024-09-12T03:17:08.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.119+0000 7f45f959a640 1 -- 172.21.15.27:0/1739786657 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f45e800bb20 con 0x7f45e80039e0 2024-09-12T03:17:08.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.119+0000 7f45f959a640 1 -- 172.21.15.27:0/1739786657 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f45e800bd20 con 0x7f45fc146010 2024-09-12T03:17:08.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.119+0000 7f45fad9d640 1 --2- 172.21.15.27:0/1739786657 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f45e8004d40 0x7f45e80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:08.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.119+0000 7f45fa59c640 1 --2- 172.21.15.27:0/1739786657 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f45e80039e0 0x7f45e8003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:08.122 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.119+0000 7f45f9d9b640 1 --2- 172.21.15.27:0/1739786657 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f45fc146010 0x7f45e8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:08.122 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.119+0000 7f45fad9d640 1 -- 172.21.15.27:0/1739786657 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f45fc146010 msgr2=0x7f45e8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:08.122 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.119+0000 7f45fad9d640 1 --2- 172.21.15.27:0/1739786657 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f45fc146010 0x7f45e8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:08.122 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.119+0000 7f45fad9d640 1 -- 172.21.15.27:0/1739786657 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f45e80039e0 msgr2=0x7f45e8003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:08.122 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.119+0000 7f45fad9d640 1 --2- 172.21.15.27:0/1739786657 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f45e80039e0 0x7f45e8003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:08.122 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.119+0000 7f45fad9d640 1 -- 172.21.15.27:0/1739786657 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45e8003520 con 0x7f45e8004d40 2024-09-12T03:17:08.122 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.120+0000 7f45fad9d640 1 --2- 172.21.15.27:0/1739786657 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f45e8004d40 0x7f45e80051c0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f45ec009fc0 tx=0x7f45e8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c33b76dbdd1c562 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:08.125 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.120+0000 7f45f959a640 1 -- 172.21.15.27:0/1739786657 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45ec009c10 con 0x7f45e8004d40 2024-09-12T03:17:08.125 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.120+0000 7f45f959a640 1 -- 172.21.15.27:0/1739786657 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45ec036070 con 0x7f45e8004d40 2024-09-12T03:17:08.125 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.120+0000 7f45f959a640 1 -- 172.21.15.27:0/1739786657 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45ec031880 con 0x7f45e8004d40 2024-09-12T03:17:08.125 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.121+0000 7f4600ad7640 1 -- 172.21.15.27:0/1739786657 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f45e8004d40 msgr2=0x7f45e80051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:08.125 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.121+0000 7f4600ad7640 1 --2- 172.21.15.27:0/1739786657 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f45e8004d40 0x7f45e80051c0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f45ec009fc0 tx=0x7f45e8005830 comp rx=0 tx=0).stop 2024-09-12T03:17:08.125 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.121+0000 7f45fad9d640 1 -- 172.21.15.27:0/1739786657 reap_dead start 2024-09-12T03:17:08.125 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.121+0000 7f4600ad7640 1 -- 172.21.15.27:0/1739786657 shutdown_connections 2024-09-12T03:17:08.125 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.121+0000 7f4600ad7640 1 -- 172.21.15.27:0/1739786657 >> 172.21.15.27:0/1739786657 conn(0x7f45fc062c60 msgr2=0x7f45fc06bb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:08.125 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.121+0000 7f4600ad7640 1 -- 172.21.15.27:0/1739786657 shutdown_connections 2024-09-12T03:17:08.125 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.121+0000 7f4600ad7640 1 -- 172.21.15.27:0/1739786657 wait complete. 2024-09-12T03:17:08.125 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.122+0000 7f4600ad7640 1 Processor -- start 2024-09-12T03:17:08.126 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.122+0000 7f4600ad7640 1 -- start start 2024-09-12T03:17:08.126 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.123+0000 7f4600ad7640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f45e8004d40 0x7f45fc15bd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:08.126 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.123+0000 7f4600ad7640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f45fc072c10 0x7f45fc0b0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:08.126 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.123+0000 7f4600ad7640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f45fc0b0ba0 0x7f45fc15fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:08.126 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.123+0000 7f4600ad7640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f45fc0b48d0 con 0x7f45e8004d40 2024-09-12T03:17:08.126 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.123+0000 7f4600ad7640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f45fc0b4750 con 0x7f45fc072c10 2024-09-12T03:17:08.126 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.123+0000 7f4600ad7640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f45fc0b4a50 con 0x7f45fc0b0ba0 2024-09-12T03:17:08.126 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.123+0000 7f45fa59c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f45e8004d40 0x7f45fc15bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:08.126 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.123+0000 7f45fa59c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f45e8004d40 0x7f45fc15bd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58250/0 (socket says 172.21.15.27:58250) 2024-09-12T03:17:08.126 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.123+0000 7f45fa59c640 1 -- 172.21.15.27:0/4202736179 learned_addr learned my addr 172.21.15.27:0/4202736179 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:17:08.126 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.123+0000 7f45fad9d640 1 --2- 172.21.15.27:0/4202736179 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f45fc0b0ba0 0x7f45fc15fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:08.126 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.123+0000 7f45f9d9b640 1 --2- 172.21.15.27:0/4202736179 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f45fc072c10 0x7f45fc0b0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:08.126 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.123+0000 7f45fa59c640 1 -- 172.21.15.27:0/4202736179 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f45fc0b0ba0 msgr2=0x7f45fc15fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:08.127 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.123+0000 7f45fa59c640 1 --2- 172.21.15.27:0/4202736179 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f45fc0b0ba0 0x7f45fc15fdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:08.127 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.123+0000 7f45fa59c640 1 -- 172.21.15.27:0/4202736179 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f45fc072c10 msgr2=0x7f45fc0b0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:08.127 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.123+0000 7f45fa59c640 1 --2- 172.21.15.27:0/4202736179 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f45fc072c10 0x7f45fc0b0660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:08.127 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.123+0000 7f45fa59c640 1 -- 172.21.15.27:0/4202736179 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45fc0b1020 con 0x7f45e8004d40 2024-09-12T03:17:08.127 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.124+0000 7f45fa59c640 1 --2- 172.21.15.27:0/4202736179 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f45e8004d40 0x7f45fc15bd40 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f45f4007370 tx=0x7f45f4007830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:08.127 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.124+0000 7f45f2ffd640 1 -- 172.21.15.27:0/4202736179 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45f400f9c0 con 0x7f45e8004d40 2024-09-12T03:17:08.127 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.124+0000 7f45f2ffd640 1 -- 172.21.15.27:0/4202736179 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45f4018070 con 0x7f45e8004d40 2024-09-12T03:17:08.127 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.124+0000 7f45f2ffd640 1 -- 172.21.15.27:0/4202736179 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45f400fde0 con 0x7f45e8004d40 2024-09-12T03:17:08.127 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.124+0000 7f4600ad7640 1 -- 172.21.15.27:0/4202736179 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45fc1e5f20 con 0x7f45e8004d40 2024-09-12T03:17:08.127 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.124+0000 7f4600ad7640 1 -- 172.21.15.27:0/4202736179 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45fc1e64e0 con 0x7f45e8004d40 2024-09-12T03:17:08.127 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.125+0000 7f4600ad7640 1 -- 172.21.15.27:0/4202736179 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45fc15ca00 con 0x7f45e8004d40 2024-09-12T03:17:08.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.125+0000 7f45f2ffd640 1 -- 172.21.15.27:0/4202736179 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f45f401d460 con 0x7f45e8004d40 2024-09-12T03:17:08.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.126+0000 7f45f2ffd640 1 --2- 172.21.15.27:0/4202736179 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f45dc06be30 0x7f45dc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:08.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.126+0000 7f45f2ffd640 1 -- 172.21.15.27:0/4202736179 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 4857+0+0 (secure 0 0 0) 0x7f45f408ac80 con 0x7f45e8004d40 2024-09-12T03:17:08.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.126+0000 7f45f9d9b640 1 --2- 172.21.15.27:0/4202736179 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f45dc06be30 0x7f45dc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:08.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.127+0000 7f45f9d9b640 1 --2- 172.21.15.27:0/4202736179 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f45dc06be30 0x7f45dc06e2f0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f45fc073210 tx=0x7f45e4009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:08.130 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.130+0000 7f45f2ffd640 1 -- 172.21.15.27:0/4202736179 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f45f4059980 con 0x7f45e8004d40 2024-09-12T03:17:08.372 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.372+0000 7f4600ad7640 1 -- 172.21.15.27:0/4202736179 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) -- 0x7f45fc1453e0 con 0x7f45e8004d40 2024-09-12T03:17:08.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.374+0000 7f45f2ffd640 1 -- 172.21.15.27:0/4202736179 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v10) ==== 90+0+0 (secure 0 0 0) 0x7f45f4059280 con 0x7f45e8004d40 2024-09-12T03:17:08.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.377+0000 7f4600ad7640 1 -- 172.21.15.27:0/4202736179 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f45dc06be30 msgr2=0x7f45dc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:08.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.377+0000 7f4600ad7640 1 --2- 172.21.15.27:0/4202736179 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f45dc06be30 0x7f45dc06e2f0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f45fc073210 tx=0x7f45e4009040 comp rx=0 tx=0).stop 2024-09-12T03:17:08.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.377+0000 7f4600ad7640 1 -- 172.21.15.27:0/4202736179 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f45e8004d40 msgr2=0x7f45fc15bd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:08.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.377+0000 7f4600ad7640 1 --2- 172.21.15.27:0/4202736179 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f45e8004d40 0x7f45fc15bd40 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f45f4007370 tx=0x7f45f4007830 comp rx=0 tx=0).stop 2024-09-12T03:17:08.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.377+0000 7f4600ad7640 1 -- 172.21.15.27:0/4202736179 shutdown_connections 2024-09-12T03:17:08.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.377+0000 7f4600ad7640 1 --2- 172.21.15.27:0/4202736179 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f45dc06be30 0x7f45dc06e2f0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:08.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.377+0000 7f4600ad7640 1 --2- 172.21.15.27:0/4202736179 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f45fc0b0ba0 0x7f45fc15fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:08.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.377+0000 7f4600ad7640 1 --2- 172.21.15.27:0/4202736179 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f45fc072c10 0x7f45fc0b0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:08.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.377+0000 7f4600ad7640 1 --2- 172.21.15.27:0/4202736179 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f45e8004d40 0x7f45fc15bd40 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:08.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.377+0000 7f4600ad7640 1 -- 172.21.15.27:0/4202736179 >> 172.21.15.27:0/4202736179 conn(0x7f45fc062c60 msgr2=0x7f45fc141910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:08.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.377+0000 7f4600ad7640 1 -- 172.21.15.27:0/4202736179 shutdown_connections 2024-09-12T03:17:08.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.377+0000 7f4600ad7640 1 -- 172.21.15.27:0/4202736179 wait complete. 2024-09-12T03:17:08.388 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-12T03:17:08.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.490+0000 7f0626998640 1 Processor -- start 2024-09-12T03:17:08.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.491+0000 7f0626998640 1 -- start start 2024-09-12T03:17:08.493 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.491+0000 7f0626998640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f06200d5200 0x7f06200d1010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:08.494 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.491+0000 7f0626998640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f06200d1550 0x7f06200b8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:08.494 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.491+0000 7f0626998640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f06200d19c0 con 0x7f062015eb90 2024-09-12T03:17:08.494 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.491+0000 7f0626998640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f06200b7d20 con 0x7f06200d1550 2024-09-12T03:17:08.494 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.491+0000 7f0626998640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f06200d42e0 con 0x7f06200d5200 2024-09-12T03:17:08.494 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.491+0000 7f0624f35640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f06200d1550 0x7f06200b8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:08.494 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.492+0000 7f0624f35640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f06200d1550 0x7f06200b8190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58266/0 (socket says 172.21.15.27:58266) 2024-09-12T03:17:08.494 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.492+0000 7f0624f35640 1 -- 172.21.15.27:0/209367717 learned_addr learned my addr 172.21.15.27:0/209367717 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:17:08.494 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.492+0000 7f061f7fe640 1 --2- 172.21.15.27:0/209367717 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f06200d5200 0x7f06200d1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:08.494 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.492+0000 7f0624f35640 1 -- 172.21.15.27:0/209367717 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f06200d5200 msgr2=0x7f06200d1010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:08.495 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.492+0000 7f0624f35640 1 --2- 172.21.15.27:0/209367717 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f06200d5200 0x7f06200d1010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:08.495 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.492+0000 7f0624f35640 1 -- 172.21.15.27:0/209367717 >> v1:172.21.15.45:6789/0 conn(0x7f062015eb90 legacy=0x7f06200d2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:08.495 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.492+0000 7f0624f35640 1 -- 172.21.15.27:0/209367717 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06200d3760 con 0x7f06200d1550 2024-09-12T03:17:08.495 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.493+0000 7f0624f35640 1 --2- 172.21.15.27:0/209367717 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f06200d1550 0x7f06200b8190 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f0618009e50 tx=0x7f061802a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54213018cd360f1f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:08.495 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.493+0000 7f061effd640 1 -- 172.21.15.27:0/209367717 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0618036070 con 0x7f06200d1550 2024-09-12T03:17:08.495 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.493+0000 7f061effd640 1 -- 172.21.15.27:0/209367717 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0618031600 con 0x7f06200d1550 2024-09-12T03:17:08.495 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.493+0000 7f061effd640 1 -- 172.21.15.27:0/209367717 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06180319d0 con 0x7f06200d1550 2024-09-12T03:17:08.495 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.493+0000 7f0626998640 1 -- 172.21.15.27:0/209367717 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f06200d1550 msgr2=0x7f06200b8190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:08.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.493+0000 7f0626998640 1 --2- 172.21.15.27:0/209367717 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f06200d1550 0x7f06200b8190 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f0618009e50 tx=0x7f061802a710 comp rx=0 tx=0).stop 2024-09-12T03:17:08.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.493+0000 7f0626998640 1 -- 172.21.15.27:0/209367717 shutdown_connections 2024-09-12T03:17:08.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.493+0000 7f0626998640 1 --2- 172.21.15.27:0/209367717 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f06200d1550 0x7f06200b8190 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:08.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.493+0000 7f0626998640 1 --2- 172.21.15.27:0/209367717 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f06200d5200 0x7f06200d1010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:08.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.493+0000 7f0626998640 1 -- 172.21.15.27:0/209367717 >> 172.21.15.27:0/209367717 conn(0x7f062006dce0 msgr2=0x7f062006e0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:08.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.493+0000 7f0626998640 1 -- 172.21.15.27:0/209367717 shutdown_connections 2024-09-12T03:17:08.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.493+0000 7f0626998640 1 -- 172.21.15.27:0/209367717 wait complete. 2024-09-12T03:17:08.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.494+0000 7f0626998640 1 Processor -- start 2024-09-12T03:17:08.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.495+0000 7f0626998640 1 -- start start 2024-09-12T03:17:08.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.495+0000 7f0626998640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06200d1550 0x7f062015a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:08.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.495+0000 7f0626998640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f06200d5200 0x7f06200c9390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:08.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.495+0000 7f0626998640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f062015eb90 0x7f062015b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:08.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.495+0000 7f0626998640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f06200d3d20 con 0x7f06200d5200 2024-09-12T03:17:08.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.495+0000 7f0626998640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f06200d3500 con 0x7f06200d1550 2024-09-12T03:17:08.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.495+0000 7f0626998640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f06200d4000 con 0x7f062015eb90 2024-09-12T03:17:08.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.496+0000 7f061f7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f06200d5200 0x7f06200c9390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:08.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.496+0000 7f061f7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f06200d5200 0x7f06200c9390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58274/0 (socket says 172.21.15.27:58274) 2024-09-12T03:17:08.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.496+0000 7f061f7fe640 1 -- 172.21.15.27:0/1610210929 learned_addr learned my addr 172.21.15.27:0/1610210929 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:17:08.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.496+0000 7f0624f35640 1 --2- 172.21.15.27:0/1610210929 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f062015eb90 0x7f062015b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:08.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.496+0000 7f061ffff640 1 --2- 172.21.15.27:0/1610210929 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06200d1550 0x7f062015a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:08.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.496+0000 7f061f7fe640 1 -- 172.21.15.27:0/1610210929 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f062015eb90 msgr2=0x7f062015b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:08.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.496+0000 7f061f7fe640 1 --2- 172.21.15.27:0/1610210929 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f062015eb90 0x7f062015b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:08.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.496+0000 7f061f7fe640 1 -- 172.21.15.27:0/1610210929 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06200d1550 msgr2=0x7f062015a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:08.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.496+0000 7f061f7fe640 1 --2- 172.21.15.27:0/1610210929 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06200d1550 0x7f062015a880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:08.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.496+0000 7f061f7fe640 1 -- 172.21.15.27:0/1610210929 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06200c98d0 con 0x7f06200d5200 2024-09-12T03:17:08.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.496+0000 7f061ffff640 1 --2- 172.21.15.27:0/1610210929 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06200d1550 0x7f062015a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:17:08.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.496+0000 7f061f7fe640 1 --2- 172.21.15.27:0/1610210929 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f06200d5200 0x7f06200c9390 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f061400d7d0 tx=0x7f061400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:08.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.496+0000 7f061cff9640 1 -- 172.21.15.27:0/1610210929 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f061400b6c0 con 0x7f06200d5200 2024-09-12T03:17:08.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.496+0000 7f061cff9640 1 -- 172.21.15.27:0/1610210929 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0614019070 con 0x7f06200d5200 2024-09-12T03:17:08.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.496+0000 7f0626998640 1 -- 172.21.15.27:0/1610210929 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06200c9b70 con 0x7f06200d5200 2024-09-12T03:17:08.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.496+0000 7f061cff9640 1 -- 172.21.15.27:0/1610210929 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f061400bac0 con 0x7f06200d5200 2024-09-12T03:17:08.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.496+0000 7f0626998640 1 -- 172.21.15.27:0/1610210929 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06200ca130 con 0x7f06200d5200 2024-09-12T03:17:08.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.497+0000 7f0626998640 1 -- 172.21.15.27:0/1610210929 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f062015f0e0 con 0x7f06200d5200 2024-09-12T03:17:08.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.498+0000 7f061cff9640 1 -- 172.21.15.27:0/1610210929 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f061400bc60 con 0x7f06200d5200 2024-09-12T03:17:08.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.498+0000 7f061cff9640 1 --2- 172.21.15.27:0/1610210929 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f060c06be30 0x7f060c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:08.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.498+0000 7f061cff9640 1 -- 172.21.15.27:0/1610210929 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 4857+0+0 (secure 0 0 0) 0x7f061408af40 con 0x7f06200d5200 2024-09-12T03:17:08.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.499+0000 7f061ffff640 1 --2- 172.21.15.27:0/1610210929 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f060c06be30 0x7f060c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:08.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.499+0000 7f061ffff640 1 --2- 172.21.15.27:0/1610210929 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f060c06be30 0x7f060c06e2f0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f0620156a20 tx=0x7f0610009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:08.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.503+0000 7f061cff9640 1 -- 172.21.15.27:0/1610210929 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f061401c360 con 0x7f06200d5200 2024-09-12T03:17:08.678 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.679+0000 7f0626998640 1 -- 172.21.15.27:0/1610210929 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f06200cff70 con 0x7f06200d5200 2024-09-12T03:17:08.679 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.679+0000 7f061cff9640 1 -- 172.21.15.27:0/1610210929 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v28) ==== 84+0+65 (secure 0 0 0) 0x7f0614059cd0 con 0x7f06200d5200 2024-09-12T03:17:08.679 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:17:08.679 INFO:teuthology.orchestra.run.smithi027.stdout:[ 2024-09-12T03:17:08.679 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:17:08.679 INFO:teuthology.orchestra.run.smithi027.stdout: "poolnum": 1, 2024-09-12T03:17:08.680 INFO:teuthology.orchestra.run.smithi027.stdout: "poolname": ".mgr" 2024-09-12T03:17:08.680 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:17:08.680 INFO:teuthology.orchestra.run.smithi027.stdout:] 2024-09-12T03:17:08.681 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.681+0000 7f0626998640 1 -- 172.21.15.27:0/1610210929 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f060c06be30 msgr2=0x7f060c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:08.681 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.681+0000 7f0626998640 1 --2- 172.21.15.27:0/1610210929 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f060c06be30 0x7f060c06e2f0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f0620156a20 tx=0x7f0610009040 comp rx=0 tx=0).stop 2024-09-12T03:17:08.681 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.681+0000 7f0626998640 1 -- 172.21.15.27:0/1610210929 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f06200d5200 msgr2=0x7f06200c9390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:08.681 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.681+0000 7f0626998640 1 --2- 172.21.15.27:0/1610210929 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f06200d5200 0x7f06200c9390 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f061400d7d0 tx=0x7f061400dc90 comp rx=0 tx=0).stop 2024-09-12T03:17:08.682 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.681+0000 7f0626998640 1 -- 172.21.15.27:0/1610210929 shutdown_connections 2024-09-12T03:17:08.682 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.681+0000 7f0626998640 1 --2- 172.21.15.27:0/1610210929 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f060c06be30 0x7f060c06e2f0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:08.682 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.681+0000 7f0626998640 1 --2- 172.21.15.27:0/1610210929 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f062015eb90 0x7f062015b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:08.682 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.681+0000 7f0626998640 1 --2- 172.21.15.27:0/1610210929 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f06200d5200 0x7f06200c9390 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:08.682 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.681+0000 7f0626998640 1 --2- 172.21.15.27:0/1610210929 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06200d1550 0x7f062015a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:08.682 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.681+0000 7f0626998640 1 -- 172.21.15.27:0/1610210929 >> 172.21.15.27:0/1610210929 conn(0x7f062006dce0 msgr2=0x7f06200b8090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:08.682 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.681+0000 7f0626998640 1 -- 172.21.15.27:0/1610210929 shutdown_connections 2024-09-12T03:17:08.682 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.681+0000 7f0626998640 1 -- 172.21.15.27:0/1610210929 wait complete. 2024-09-12T03:17:08.692 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-12T03:17:08.692 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-12T03:17:08.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.799+0000 7faed66d0640 1 Processor -- start 2024-09-12T03:17:08.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.799+0000 7faed66d0640 1 -- start start 2024-09-12T03:17:08.803 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.800+0000 7faed66d0640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7faed00d3810 0x7faed0162f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:08.803 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.800+0000 7faed66d0640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faed00d4310 con 0x7faed00d3330 2024-09-12T03:17:08.803 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.800+0000 7faed66d0640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faed00d1e90 con 0x7faed00b6440 2024-09-12T03:17:08.803 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.800+0000 7faed66d0640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7faed00b7b30 con 0x7faed00d3810 2024-09-12T03:17:08.803 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.800+0000 7faed4c6d640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7faed00d3330 0x7faed00b7fc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:44388/0 (socket says 172.21.15.27:44388) 2024-09-12T03:17:08.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.800+0000 7faed4c6d640 1 -- 172.21.15.27:0/4109860295 learned_addr learned my addr 172.21.15.27:0/4109860295 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:17:08.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.800+0000 7faecf7fe640 1 --2- 172.21.15.27:0/4109860295 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7faed00d3810 0x7faed0162f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:08.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.800+0000 7faeceffd640 1 -- 172.21.15.27:0/4109860295 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2782405097 0 0) 0x7faed00d4310 con 0x7faed00d3330 2024-09-12T03:17:08.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.800+0000 7faeceffd640 1 -- 172.21.15.27:0/4109860295 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7faeb8003520 con 0x7faed00d3330 2024-09-12T03:17:08.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.800+0000 7faeceffd640 1 -- 172.21.15.27:0/4109860295 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3894374759 0 0) 0x7faeb8003520 con 0x7faed00d3330 2024-09-12T03:17:08.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.800+0000 7faeceffd640 1 -- 172.21.15.27:0/4109860295 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7faed00d3810 msgr2=0x7faed0162f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:08.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.800+0000 7faeceffd640 1 --2- 172.21.15.27:0/4109860295 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7faed00d3810 0x7faed0162f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:08.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.800+0000 7faeceffd640 1 -- 172.21.15.27:0/4109860295 >> v1:172.21.15.45:6789/0 conn(0x7faed00b6440 legacy=0x7faed0162840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:08.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.800+0000 7faeceffd640 1 -- 172.21.15.27:0/4109860295 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faed00b7750 con 0x7faed00d3330 2024-09-12T03:17:08.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.800+0000 7faeceffd640 1 -- 172.21.15.27:0/4109860295 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 847755213 0 0) 0x7faec80019f0 con 0x7faed00d3330 2024-09-12T03:17:08.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.801+0000 7faeceffd640 1 -- 172.21.15.27:0/4109860295 >> v1:172.21.15.27:6789/0 conn(0x7faed00d3330 legacy=0x7faed00b7fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:08.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.801+0000 7faeceffd640 1 --2- 172.21.15.27:0/4109860295 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faeb80039e0 0x7faeb8003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:08.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.801+0000 7faeceffd640 1 --2- 172.21.15.27:0/4109860295 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7faed00d3810 0x7faeb8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:08.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.801+0000 7faeceffd640 1 --2- 172.21.15.27:0/4109860295 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faeb8004d40 0x7faeb80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:08.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.801+0000 7faeceffd640 1 -- 172.21.15.27:0/4109860295 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7faed00b7b30 con 0x7faeb8004d40 2024-09-12T03:17:08.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.801+0000 7faeceffd640 1 -- 172.21.15.27:0/4109860295 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7faeb800ba70 con 0x7faeb80039e0 2024-09-12T03:17:08.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.801+0000 7faeceffd640 1 -- 172.21.15.27:0/4109860295 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7faeb800bc70 con 0x7faed00d3810 2024-09-12T03:17:08.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.801+0000 7faed4c6d640 1 --2- 172.21.15.27:0/4109860295 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faeb8004d40 0x7faeb80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:08.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.801+0000 7faecffff640 1 --2- 172.21.15.27:0/4109860295 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faeb80039e0 0x7faeb8003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:08.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.801+0000 7faecf7fe640 1 --2- 172.21.15.27:0/4109860295 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7faed00d3810 0x7faeb8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:08.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.801+0000 7faed4c6d640 1 -- 172.21.15.27:0/4109860295 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7faed00d3810 msgr2=0x7faeb8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:08.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.801+0000 7faed4c6d640 1 --2- 172.21.15.27:0/4109860295 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7faed00d3810 0x7faeb8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:08.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.801+0000 7faed4c6d640 1 -- 172.21.15.27:0/4109860295 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faeb80039e0 msgr2=0x7faeb8003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:08.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.801+0000 7faed4c6d640 1 --2- 172.21.15.27:0/4109860295 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faeb80039e0 0x7faeb8003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:08.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.801+0000 7faed4c6d640 1 -- 172.21.15.27:0/4109860295 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faeb8003520 con 0x7faeb8004d40 2024-09-12T03:17:08.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.802+0000 7faed4c6d640 1 --2- 172.21.15.27:0/4109860295 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faeb8004d40 0x7faeb80051c0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7faed00d3740 tx=0x7faeb8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc217715419fdf01 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:08.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.802+0000 7faeceffd640 1 -- 172.21.15.27:0/4109860295 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faec800a480 con 0x7faeb8004d40 2024-09-12T03:17:08.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.802+0000 7faeceffd640 1 -- 172.21.15.27:0/4109860295 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faec802f070 con 0x7faeb8004d40 2024-09-12T03:17:08.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.802+0000 7faeceffd640 1 -- 172.21.15.27:0/4109860295 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faec802bc90 con 0x7faeb8004d40 2024-09-12T03:17:08.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.802+0000 7faed66d0640 1 -- 172.21.15.27:0/4109860295 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faeb8004d40 msgr2=0x7faeb80051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:08.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.802+0000 7faed66d0640 1 --2- 172.21.15.27:0/4109860295 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faeb8004d40 0x7faeb80051c0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7faed00d3740 tx=0x7faeb8005830 comp rx=0 tx=0).stop 2024-09-12T03:17:08.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.802+0000 7faed4c6d640 1 -- 172.21.15.27:0/4109860295 reap_dead start 2024-09-12T03:17:08.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.802+0000 7faed66d0640 1 -- 172.21.15.27:0/4109860295 shutdown_connections 2024-09-12T03:17:08.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.802+0000 7faed66d0640 1 -- 172.21.15.27:0/4109860295 >> 172.21.15.27:0/4109860295 conn(0x7faed00d11d0 msgr2=0x7faed00d15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:08.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.803+0000 7faed66d0640 1 -- 172.21.15.27:0/4109860295 shutdown_connections 2024-09-12T03:17:08.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.803+0000 7faed66d0640 1 -- 172.21.15.27:0/4109860295 wait complete. 2024-09-12T03:17:08.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.804+0000 7faed66d0640 1 Processor -- start 2024-09-12T03:17:08.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.804+0000 7faed66d0640 1 -- start start 2024-09-12T03:17:08.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.805+0000 7faed66d0640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7faeb8004d40 0x7faed015ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:08.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.805+0000 7faed66d0640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faed01f5410 0x7faed00ce3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:08.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.805+0000 7faed66d0640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faed00ce920 0x7faed00ac280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:08.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.805+0000 7faed66d0640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7faed00b6820 con 0x7faed01f5410 2024-09-12T03:17:08.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.805+0000 7faed66d0640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7faed00d3f70 con 0x7faed00ce920 2024-09-12T03:17:08.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.805+0000 7faed66d0640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7faed00b69a0 con 0x7faeb8004d40 2024-09-12T03:17:08.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.805+0000 7faecf7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faed01f5410 0x7faed00ce3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:08.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.805+0000 7faecf7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faed01f5410 0x7faed00ce3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58298/0 (socket says 172.21.15.27:58298) 2024-09-12T03:17:08.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.805+0000 7faecf7fe640 1 -- 172.21.15.27:0/3917001772 learned_addr learned my addr 172.21.15.27:0/3917001772 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:17:08.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.805+0000 7faecffff640 1 --2- 172.21.15.27:0/3917001772 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7faeb8004d40 0x7faed015ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:08.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.805+0000 7faed4c6d640 1 --2- 172.21.15.27:0/3917001772 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faed00ce920 0x7faed00ac280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:08.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.805+0000 7faecf7fe640 1 -- 172.21.15.27:0/3917001772 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7faeb8004d40 msgr2=0x7faed015ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:08.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.805+0000 7faecf7fe640 1 --2- 172.21.15.27:0/3917001772 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7faeb8004d40 0x7faed015ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:08.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.805+0000 7faecf7fe640 1 -- 172.21.15.27:0/3917001772 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faed00ce920 msgr2=0x7faed00ac280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:08.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.805+0000 7faecf7fe640 1 --2- 172.21.15.27:0/3917001772 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faed00ce920 0x7faed00ac280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:08.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.805+0000 7faecf7fe640 1 -- 172.21.15.27:0/3917001772 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faed01f7e70 con 0x7faed01f5410 2024-09-12T03:17:08.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.805+0000 7faecf7fe640 1 --2- 172.21.15.27:0/3917001772 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faed01f5410 0x7faed00ce3e0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7faec4011910 tx=0x7faec4011dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:08.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.805+0000 7faeccff9640 1 -- 172.21.15.27:0/3917001772 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faec400faf0 con 0x7faed01f5410 2024-09-12T03:17:08.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.805+0000 7faeccff9640 1 -- 172.21.15.27:0/3917001772 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faec400fc90 con 0x7faed01f5410 2024-09-12T03:17:08.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.806+0000 7faed66d0640 1 -- 172.21.15.27:0/3917001772 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faed01f8150 con 0x7faed01f5410 2024-09-12T03:17:08.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.806+0000 7faeccff9640 1 -- 172.21.15.27:0/3917001772 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faec4014440 con 0x7faed01f5410 2024-09-12T03:17:08.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.806+0000 7faed66d0640 1 -- 172.21.15.27:0/3917001772 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faed01f8710 con 0x7faed01f5410 2024-09-12T03:17:08.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.807+0000 7faed66d0640 1 -- 172.21.15.27:0/3917001772 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faed0162840 con 0x7faed01f5410 2024-09-12T03:17:08.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.807+0000 7faeccff9640 1 -- 172.21.15.27:0/3917001772 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7faec4012040 con 0x7faed01f5410 2024-09-12T03:17:08.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.808+0000 7faeccff9640 1 --2- 172.21.15.27:0/3917001772 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7faebc06be60 0x7faebc06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:08.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.808+0000 7faeccff9640 1 -- 172.21.15.27:0/3917001772 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 4857+0+0 (secure 0 0 0) 0x7faec408b030 con 0x7faed01f5410 2024-09-12T03:17:08.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.808+0000 7faecffff640 1 --2- 172.21.15.27:0/3917001772 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7faebc06be60 0x7faebc06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:08.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.809+0000 7faecffff640 1 --2- 172.21.15.27:0/3917001772 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7faebc06be60 0x7faebc06e320 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7faec00017e0 tx=0x7faec0009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:08.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.812+0000 7faeccff9640 1 -- 172.21.15.27:0/3917001772 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7faec4019030 con 0x7faed01f5410 2024-09-12T03:17:08.988 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:08.988+0000 7faed66d0640 1 -- 172.21.15.27:0/3917001772 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7faed015f480 con 0x7faed01f5410 2024-09-12T03:17:09.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:09.749+0000 7faeccff9640 1 -- 172.21.15.27:0/3917001772 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v29) ==== 138+0+0 (secure 0 0 0) 0x7faec4059770 con 0x7faed01f5410 2024-09-12T03:17:09.750 INFO:teuthology.orchestra.run.smithi027.stderr:pool 'cephfs_metadata' created 2024-09-12T03:17:09.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:09.752+0000 7faed66d0640 1 -- 172.21.15.27:0/3917001772 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7faebc06be60 msgr2=0x7faebc06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:09.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:09.752+0000 7faed66d0640 1 --2- 172.21.15.27:0/3917001772 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7faebc06be60 0x7faebc06e320 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7faec00017e0 tx=0x7faec0009040 comp rx=0 tx=0).stop 2024-09-12T03:17:09.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:09.752+0000 7faed66d0640 1 -- 172.21.15.27:0/3917001772 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faed01f5410 msgr2=0x7faed00ce3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:09.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:09.752+0000 7faed66d0640 1 --2- 172.21.15.27:0/3917001772 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faed01f5410 0x7faed00ce3e0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7faec4011910 tx=0x7faec4011dd0 comp rx=0 tx=0).stop 2024-09-12T03:17:09.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:09.752+0000 7faed66d0640 1 -- 172.21.15.27:0/3917001772 shutdown_connections 2024-09-12T03:17:09.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:09.752+0000 7faed66d0640 1 --2- 172.21.15.27:0/3917001772 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7faebc06be60 0x7faebc06e320 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:09.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:09.752+0000 7faed66d0640 1 --2- 172.21.15.27:0/3917001772 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faed00ce920 0x7faed00ac280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:09.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:09.752+0000 7faed66d0640 1 --2- 172.21.15.27:0/3917001772 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faed01f5410 0x7faed00ce3e0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:09.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:09.752+0000 7faed66d0640 1 --2- 172.21.15.27:0/3917001772 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7faeb8004d40 0x7faed015ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:09.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:09.752+0000 7faed66d0640 1 -- 172.21.15.27:0/3917001772 >> 172.21.15.27:0/3917001772 conn(0x7faed00d11d0 msgr2=0x7faed00b4870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:09.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:09.752+0000 7faed66d0640 1 -- 172.21.15.27:0/3917001772 shutdown_connections 2024-09-12T03:17:09.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:09.752+0000 7faed66d0640 1 -- 172.21.15.27:0/3917001772 wait complete. 2024-09-12T03:17:09.770 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-12T03:17:09.886 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:09.886+0000 7f4a64166640 1 Processor -- start 2024-09-12T03:17:09.886 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:09.886+0000 7f4a64166640 1 -- start start 2024-09-12T03:17:09.889 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:09.887+0000 7f4a64166640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4a5c06f990 0x7f4a5c06e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:09.889 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:09.887+0000 7f4a64166640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f4a5c0701c0 0x7f4a5c0705e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:09.889 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:09.887+0000 7f4a64166640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4a5c06ed60 con 0x7f4a5c0725b0 2024-09-12T03:17:09.889 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:09.887+0000 7f4a64166640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f4a5c070b20 con 0x7f4a5c0701c0 2024-09-12T03:17:09.889 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:09.887+0000 7f4a64166640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f4a5c15ef90 con 0x7f4a5c06f990 2024-09-12T03:17:09.889 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:09.887+0000 7f4a62703640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f4a5c0725b0 0x7f4a5c162510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:42540/0 (socket says 172.21.15.27:42540) 2024-09-12T03:17:09.890 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:09.887+0000 7f4a62703640 1 -- 172.21.15.27:0/1536339076 learned_addr learned my addr 172.21.15.27:0/1536339076 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:17:09.890 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:09.888+0000 7f4a61f02640 1 --2- 172.21.15.27:0/1536339076 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4a5c06f990 0x7f4a5c06e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:09.890 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:09.888+0000 7f4a61701640 1 --2- 172.21.15.27:0/1536339076 >> v2:172.21.15.45:3300/0 conn(0x7f4a5c0701c0 0x7f4a5c0705e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:09.890 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:09.888+0000 7f4a60f00640 1 -- 172.21.15.27:0/1536339076 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1226637306 0 0) 0x7f4a5c06ed60 con 0x7f4a5c0725b0 2024-09-12T03:17:09.890 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:09.888+0000 7f4a60f00640 1 -- 172.21.15.27:0/1536339076 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4a48003520 con 0x7f4a5c0725b0 2024-09-12T03:17:09.890 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:09.888+0000 7f4a60f00640 1 -- 172.21.15.27:0/1536339076 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2311541035 0 0) 0x7f4a48003520 con 0x7f4a5c0725b0 2024-09-12T03:17:09.890 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:09.888+0000 7f4a60f00640 1 -- 172.21.15.27:0/1536339076 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4a5c06f990 msgr2=0x7f4a5c06e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:09.890 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:09.888+0000 7f4a60f00640 1 --2- 172.21.15.27:0/1536339076 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4a5c06f990 0x7f4a5c06e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:09.890 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:09.888+0000 7f4a60f00640 1 -- 172.21.15.27:0/1536339076 >> v2:172.21.15.45:3300/0 conn(0x7f4a5c0701c0 msgr2=0x7f4a5c0705e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:09.890 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:09.888+0000 7f4a60f00640 1 --2- 172.21.15.27:0/1536339076 >> v2:172.21.15.45:3300/0 conn(0x7f4a5c0701c0 0x7f4a5c0705e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:09.890 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:09.888+0000 7f4a60f00640 1 -- 172.21.15.27:0/1536339076 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a5c15f270 con 0x7f4a5c0725b0 2024-09-12T03:17:09.890 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:09.888+0000 7f4a60f00640 1 -- 172.21.15.27:0/1536339076 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 847755213 0 0) 0x7f4a540019f0 con 0x7f4a5c0725b0 2024-09-12T03:17:09.890 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:09.888+0000 7f4a60f00640 1 -- 172.21.15.27:0/1536339076 >> v1:172.21.15.27:6789/0 conn(0x7f4a5c0725b0 legacy=0x7f4a5c162510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:09.890 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:09.888+0000 7f4a60f00640 1 --2- 172.21.15.27:0/1536339076 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4a5c06f990 0x7f4a480039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:09.891 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:09.888+0000 7f4a60f00640 1 --2- 172.21.15.27:0/1536339076 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4a48003f30 0x7f4a48004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:09.891 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:09.888+0000 7f4a60f00640 1 --2- 172.21.15.27:0/1536339076 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4a48004d40 0x7f4a480051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:09.891 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:09.888+0000 7f4a60f00640 1 -- 172.21.15.27:0/1536339076 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f4a5c070b20 con 0x7f4a48004d40 2024-09-12T03:17:09.891 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:09.888+0000 7f4a60f00640 1 -- 172.21.15.27:0/1536339076 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f4a5c15ef90 con 0x7f4a48003f30 2024-09-12T03:17:09.891 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:09.888+0000 7f4a60f00640 1 -- 172.21.15.27:0/1536339076 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f4a4800ba10 con 0x7f4a5c06f990 2024-09-12T03:17:09.891 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:09.888+0000 7f4a62703640 1 --2- 172.21.15.27:0/1536339076 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4a48004d40 0x7f4a480051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:09.891 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:09.888+0000 7f4a61f02640 1 --2- 172.21.15.27:0/1536339076 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4a5c06f990 0x7f4a480039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:09.891 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:09.888+0000 7f4a61701640 1 --2- 172.21.15.27:0/1536339076 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4a48003f30 0x7f4a48004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:09.891 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:09.889+0000 7f4a62703640 1 -- 172.21.15.27:0/1536339076 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4a5c06f990 msgr2=0x7f4a480039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:09.891 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:09.889+0000 7f4a62703640 1 --2- 172.21.15.27:0/1536339076 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4a5c06f990 0x7f4a480039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:09.891 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:09.889+0000 7f4a62703640 1 -- 172.21.15.27:0/1536339076 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4a48003f30 msgr2=0x7f4a48004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:09.891 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:09.889+0000 7f4a62703640 1 --2- 172.21.15.27:0/1536339076 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4a48003f30 0x7f4a48004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:09.891 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:09.889+0000 7f4a62703640 1 -- 172.21.15.27:0/1536339076 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a48003520 con 0x7f4a48004d40 2024-09-12T03:17:09.892 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:09.889+0000 7f4a62703640 1 --2- 172.21.15.27:0/1536339076 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4a48004d40 0x7f4a480051c0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f4a5400a950 tx=0x7f4a48005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c4d10bdfdd965c9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:09.892 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:09.889+0000 7f4a60f00640 1 -- 172.21.15.27:0/1536339076 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a5400a4c0 con 0x7f4a48004d40 2024-09-12T03:17:09.892 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:09.889+0000 7f4a60f00640 1 -- 172.21.15.27:0/1536339076 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a5402f070 con 0x7f4a48004d40 2024-09-12T03:17:09.892 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:09.889+0000 7f4a60f00640 1 -- 172.21.15.27:0/1536339076 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a5402bd40 con 0x7f4a48004d40 2024-09-12T03:17:09.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:09.890+0000 7f4a64166640 1 -- 172.21.15.27:0/1536339076 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4a48004d40 msgr2=0x7f4a480051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:09.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:09.890+0000 7f4a64166640 1 --2- 172.21.15.27:0/1536339076 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4a48004d40 0x7f4a480051c0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f4a5400a950 tx=0x7f4a48005830 comp rx=0 tx=0).stop 2024-09-12T03:17:09.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:09.890+0000 7f4a62703640 1 -- 172.21.15.27:0/1536339076 reap_dead start 2024-09-12T03:17:09.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:09.890+0000 7f4a64166640 1 -- 172.21.15.27:0/1536339076 shutdown_connections 2024-09-12T03:17:09.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:09.890+0000 7f4a64166640 1 -- 172.21.15.27:0/1536339076 >> 172.21.15.27:0/1536339076 conn(0x7f4a5c06d060 msgr2=0x7f4a5c06d470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:09.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:09.890+0000 7f4a64166640 1 -- 172.21.15.27:0/1536339076 shutdown_connections 2024-09-12T03:17:09.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:09.890+0000 7f4a64166640 1 -- 172.21.15.27:0/1536339076 wait complete. 2024-09-12T03:17:09.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:09.891+0000 7f4a64166640 1 Processor -- start 2024-09-12T03:17:09.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:09.891+0000 7f4a64166640 1 -- start start 2024-09-12T03:17:09.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:09.892+0000 7f4a64166640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4a48004d40 0x7f4a5c14f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:09.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:09.892+0000 7f4a64166640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4a5c161320 0x7f4a5c1617a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:09.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:09.892+0000 7f4a64166640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4a5c0c8da0 0x7f4a5c0c9220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:09.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:09.892+0000 7f4a64166640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f4a5c15fb10 con 0x7f4a48004d40 2024-09-12T03:17:09.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:09.892+0000 7f4a64166640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f4a5c15f830 con 0x7f4a5c0c8da0 2024-09-12T03:17:09.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:09.892+0000 7f4a64166640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f4a5c15eb50 con 0x7f4a5c161320 2024-09-12T03:17:09.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:09.892+0000 7f4a61f02640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4a48004d40 0x7f4a5c14f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:09.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:09.892+0000 7f4a61f02640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4a48004d40 0x7f4a5c14f0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36846/0 (socket says 172.21.15.27:36846) 2024-09-12T03:17:09.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:09.892+0000 7f4a61f02640 1 -- 172.21.15.27:0/2646907567 learned_addr learned my addr 172.21.15.27:0/2646907567 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:17:09.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:09.892+0000 7f4a61f02640 1 -- 172.21.15.27:0/2646907567 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4a5c161320 msgr2=0x7f4a5c1617a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:09.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:09.892+0000 7f4a61f02640 1 --2- 172.21.15.27:0/2646907567 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4a5c161320 0x7f4a5c1617a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:09.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:09.892+0000 7f4a61f02640 1 -- 172.21.15.27:0/2646907567 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4a5c0c8da0 msgr2=0x7f4a5c0c9220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:09.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:09.892+0000 7f4a61f02640 1 --2- 172.21.15.27:0/2646907567 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4a5c0c8da0 0x7f4a5c0c9220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:09.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:09.892+0000 7f4a61f02640 1 -- 172.21.15.27:0/2646907567 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a5c1f41f0 con 0x7f4a48004d40 2024-09-12T03:17:09.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:09.893+0000 7f4a61f02640 1 --2- 172.21.15.27:0/2646907567 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4a48004d40 0x7f4a5c14f0a0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f4a58011a00 tx=0x7f4a58011ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:09.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:09.893+0000 7f4a527fc640 1 -- 172.21.15.27:0/2646907567 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a5800fa10 con 0x7f4a48004d40 2024-09-12T03:17:09.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:09.893+0000 7f4a527fc640 1 -- 172.21.15.27:0/2646907567 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a5800fbb0 con 0x7f4a48004d40 2024-09-12T03:17:09.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:09.893+0000 7f4a527fc640 1 -- 172.21.15.27:0/2646907567 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a58014440 con 0x7f4a48004d40 2024-09-12T03:17:09.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:09.893+0000 7f4a64166640 1 -- 172.21.15.27:0/2646907567 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a5c1f44d0 con 0x7f4a48004d40 2024-09-12T03:17:09.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:09.893+0000 7f4a64166640 1 -- 172.21.15.27:0/2646907567 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a5c0c9760 con 0x7f4a48004d40 2024-09-12T03:17:09.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:09.894+0000 7f4a64166640 1 -- 172.21.15.27:0/2646907567 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a5c161ce0 con 0x7f4a48004d40 2024-09-12T03:17:09.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:09.894+0000 7f4a527fc640 1 -- 172.21.15.27:0/2646907567 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4a58012040 con 0x7f4a48004d40 2024-09-12T03:17:09.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:09.895+0000 7f4a527fc640 1 --2- 172.21.15.27:0/2646907567 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f4a4006be30 0x7f4a4006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:09.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:09.895+0000 7f4a527fc640 1 -- 172.21.15.27:0/2646907567 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 5227+0+0 (secure 0 0 0) 0x7f4a5808b200 con 0x7f4a48004d40 2024-09-12T03:17:09.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:09.895+0000 7f4a61701640 1 --2- 172.21.15.27:0/2646907567 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f4a4006be30 0x7f4a4006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:09.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:09.896+0000 7f4a61701640 1 --2- 172.21.15.27:0/2646907567 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f4a4006be30 0x7f4a4006e2f0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f4a4c006e80 tx=0x7f4a4c006df0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:09.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:09.899+0000 7f4a527fc640 1 -- 172.21.15.27:0/2646907567 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f4a58019030 con 0x7f4a48004d40 2024-09-12T03:17:10.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:10.075+0000 7f4a64166640 1 -- 172.21.15.27:0/2646907567 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f4a5c0705e0 con 0x7f4a48004d40 2024-09-12T03:17:10.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:10.757+0000 7f4a527fc640 1 -- 172.21.15.27:0/2646907567 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v30) ==== 170+0+0 (secure 0 0 0) 0x7f4a58059d90 con 0x7f4a48004d40 2024-09-12T03:17:10.757 INFO:teuthology.orchestra.run.smithi027.stderr:pool 'cephfs_data' created 2024-09-12T03:17:10.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:10.759+0000 7f4a64166640 1 -- 172.21.15.27:0/2646907567 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f4a4006be30 msgr2=0x7f4a4006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:10.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:10.759+0000 7f4a64166640 1 --2- 172.21.15.27:0/2646907567 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f4a4006be30 0x7f4a4006e2f0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f4a4c006e80 tx=0x7f4a4c006df0 comp rx=0 tx=0).stop 2024-09-12T03:17:10.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:10.759+0000 7f4a64166640 1 -- 172.21.15.27:0/2646907567 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4a48004d40 msgr2=0x7f4a5c14f0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:10.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:10.759+0000 7f4a64166640 1 --2- 172.21.15.27:0/2646907567 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4a48004d40 0x7f4a5c14f0a0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f4a58011a00 tx=0x7f4a58011ec0 comp rx=0 tx=0).stop 2024-09-12T03:17:10.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:10.759+0000 7f4a64166640 1 -- 172.21.15.27:0/2646907567 shutdown_connections 2024-09-12T03:17:10.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:10.759+0000 7f4a64166640 1 --2- 172.21.15.27:0/2646907567 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f4a4006be30 0x7f4a4006e2f0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:10.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:10.759+0000 7f4a64166640 1 --2- 172.21.15.27:0/2646907567 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4a5c0c8da0 0x7f4a5c0c9220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:10.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:10.759+0000 7f4a64166640 1 --2- 172.21.15.27:0/2646907567 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4a5c161320 0x7f4a5c1617a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:10.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:10.759+0000 7f4a64166640 1 --2- 172.21.15.27:0/2646907567 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4a48004d40 0x7f4a5c14f0a0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:10.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:10.759+0000 7f4a64166640 1 -- 172.21.15.27:0/2646907567 >> 172.21.15.27:0/2646907567 conn(0x7f4a5c06d060 msgr2=0x7f4a5c0b0b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:10.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:10.759+0000 7f4a64166640 1 -- 172.21.15.27:0/2646907567 shutdown_connections 2024-09-12T03:17:10.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:10.759+0000 7f4a64166640 1 -- 172.21.15.27:0/2646907567 wait complete. 2024-09-12T03:17:10.785 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-12T03:17:10.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:10.894+0000 7f7d78274640 1 Processor -- start 2024-09-12T03:17:10.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:10.894+0000 7f7d78274640 1 -- start start 2024-09-12T03:17:10.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:10.895+0000 7f7d78274640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f7d7015a270 0x7f7d7006cff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:10.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:10.895+0000 7f7d78274640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f7d700cfb20 0x7f7d700726a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:10.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:10.895+0000 7f7d78274640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7d700ce650 con 0x7f7d7006d530 2024-09-12T03:17:10.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:10.895+0000 7f7d78274640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f7d700cef80 con 0x7f7d7015a270 2024-09-12T03:17:10.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:10.895+0000 7f7d78274640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f7d700ce930 con 0x7f7d700cfb20 2024-09-12T03:17:10.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:10.895+0000 7f7d76811640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f7d700cfb20 0x7f7d700726a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:10.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:10.895+0000 7f7d76010640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f7d7015a270 0x7f7d7006cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:10.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:10.895+0000 7f7d7580f640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f7d7006d530 0x7f7d700cf410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.27:34084/0 (socket says 172.21.15.27:34084) 2024-09-12T03:17:10.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:10.895+0000 7f7d76811640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f7d700cfb20 0x7f7d700726a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3301/0 says I am v2:172.21.15.27:39680/0 (socket says 172.21.15.27:39680) 2024-09-12T03:17:10.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:10.895+0000 7f7d76010640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f7d7015a270 0x7f7d7006cff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.27:43616/0 (socket says 172.21.15.27:43616) 2024-09-12T03:17:10.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:10.895+0000 7f7d76811640 1 -- 172.21.15.27:0/3327877557 learned_addr learned my addr 172.21.15.27:0/3327877557 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:17:10.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:10.896+0000 7f7d76811640 1 -- 172.21.15.27:0/3327877557 >> v2:172.21.15.45:3300/0 conn(0x7f7d7015a270 msgr2=0x7f7d7006cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:10.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:10.896+0000 7f7d76811640 1 --2- 172.21.15.27:0/3327877557 >> v2:172.21.15.45:3300/0 conn(0x7f7d7015a270 0x7f7d7006cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:10.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:10.896+0000 7f7d76811640 1 -- 172.21.15.27:0/3327877557 >> v1:172.21.15.45:6789/0 conn(0x7f7d7006d530 legacy=0x7f7d700cf410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:10.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:10.896+0000 7f7d76811640 1 -- 172.21.15.27:0/3327877557 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d700cec10 con 0x7f7d700cfb20 2024-09-12T03:17:10.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:10.896+0000 7f7d76010640 1 --2- 172.21.15.27:0/3327877557 >> v2:172.21.15.45:3300/0 conn(0x7f7d7015a270 0x7f7d7006cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:17:10.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:10.896+0000 7f7d76811640 1 --2- 172.21.15.27:0/3327877557 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f7d700cfb20 0x7f7d700726a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f7d68009e50 tx=0x7f7d6802a710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7543601c5faa1e44 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:10.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:10.897+0000 7f7d7500e640 1 -- 172.21.15.27:0/3327877557 <== mon.2 v2:172.21.15.45:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d68036070 con 0x7f7d700cfb20 2024-09-12T03:17:10.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:10.897+0000 7f7d7500e640 1 -- 172.21.15.27:0/3327877557 <== mon.2 v2:172.21.15.45:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d68031600 con 0x7f7d700cfb20 2024-09-12T03:17:10.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:10.897+0000 7f7d7500e640 1 -- 172.21.15.27:0/3327877557 <== mon.2 v2:172.21.15.45:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d680319d0 con 0x7f7d700cfb20 2024-09-12T03:17:10.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:10.897+0000 7f7d78274640 1 -- 172.21.15.27:0/3327877557 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f7d700cfb20 msgr2=0x7f7d700726a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:10.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:10.897+0000 7f7d78274640 1 --2- 172.21.15.27:0/3327877557 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f7d700cfb20 0x7f7d700726a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f7d68009e50 tx=0x7f7d6802a710 comp rx=0 tx=0).stop 2024-09-12T03:17:10.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:10.897+0000 7f7d78274640 1 -- 172.21.15.27:0/3327877557 shutdown_connections 2024-09-12T03:17:10.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:10.897+0000 7f7d78274640 1 --2- 172.21.15.27:0/3327877557 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f7d700cfb20 0x7f7d700726a0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:10.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:10.897+0000 7f7d78274640 1 --2- 172.21.15.27:0/3327877557 >> v2:172.21.15.45:3300/0 conn(0x7f7d7015a270 0x7f7d7006cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:10.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:10.897+0000 7f7d78274640 1 -- 172.21.15.27:0/3327877557 >> 172.21.15.27:0/3327877557 conn(0x7f7d7006bb30 msgr2=0x7f7d7006bf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:10.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:10.897+0000 7f7d78274640 1 -- 172.21.15.27:0/3327877557 shutdown_connections 2024-09-12T03:17:10.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:10.897+0000 7f7d78274640 1 -- 172.21.15.27:0/3327877557 wait complete. 2024-09-12T03:17:10.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:10.898+0000 7f7d78274640 1 Processor -- start 2024-09-12T03:17:10.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:10.898+0000 7f7d78274640 1 -- start start 2024-09-12T03:17:10.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:10.899+0000 7f7d78274640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7d7006d530 0x7f7d70158580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:10.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:10.899+0000 7f7d78274640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f7d700cfb20 0x7f7d701f5a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:10.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:10.899+0000 7f7d78274640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7d7015a270 0x7f7d701f5f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:10.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:10.899+0000 7f7d78274640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f7d7006c7d0 con 0x7f7d7015a270 2024-09-12T03:17:10.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:10.899+0000 7f7d78274640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f7d7006c650 con 0x7f7d7006d530 2024-09-12T03:17:10.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:10.899+0000 7f7d78274640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f7d7006c950 con 0x7f7d700cfb20 2024-09-12T03:17:10.902 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:10.899+0000 7f7d76811640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7d7015a270 0x7f7d701f5f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:10.902 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:10.899+0000 7f7d76811640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7d7015a270 0x7f7d701f5f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36858/0 (socket says 172.21.15.27:36858) 2024-09-12T03:17:10.902 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:10.899+0000 7f7d76811640 1 -- 172.21.15.27:0/223618647 learned_addr learned my addr 172.21.15.27:0/223618647 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:17:10.902 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:10.899+0000 7f7d7580f640 1 --2- 172.21.15.27:0/223618647 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f7d700cfb20 0x7f7d701f5a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:10.902 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:10.899+0000 7f7d76010640 1 --2- 172.21.15.27:0/223618647 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7d7006d530 0x7f7d70158580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:10.902 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:10.900+0000 7f7d76811640 1 -- 172.21.15.27:0/223618647 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f7d700cfb20 msgr2=0x7f7d701f5a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:10.902 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:10.900+0000 7f7d76811640 1 --2- 172.21.15.27:0/223618647 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f7d700cfb20 0x7f7d701f5a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:10.902 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:10.900+0000 7f7d76811640 1 -- 172.21.15.27:0/223618647 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7d7006d530 msgr2=0x7f7d70158580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:10.902 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:10.900+0000 7f7d76811640 1 --2- 172.21.15.27:0/223618647 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7d7006d530 0x7f7d70158580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:10.902 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:10.900+0000 7f7d76811640 1 -- 172.21.15.27:0/223618647 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d700ab740 con 0x7f7d7015a270 2024-09-12T03:17:10.902 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:10.900+0000 7f7d76811640 1 --2- 172.21.15.27:0/223618647 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7d7015a270 0x7f7d701f5f90 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f7d68009f70 tx=0x7f7d68004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:10.902 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:10.900+0000 7f7d66ffd640 1 -- 172.21.15.27:0/223618647 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d6803f020 con 0x7f7d7015a270 2024-09-12T03:17:10.902 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:10.900+0000 7f7d66ffd640 1 -- 172.21.15.27:0/223618647 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d68037040 con 0x7f7d7015a270 2024-09-12T03:17:10.902 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:10.900+0000 7f7d66ffd640 1 -- 172.21.15.27:0/223618647 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d68036070 con 0x7f7d7015a270 2024-09-12T03:17:10.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:10.900+0000 7f7d78274640 1 -- 172.21.15.27:0/223618647 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d700ab9e0 con 0x7f7d7015a270 2024-09-12T03:17:10.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:10.900+0000 7f7d78274640 1 -- 172.21.15.27:0/223618647 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d701f4410 con 0x7f7d7015a270 2024-09-12T03:17:10.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:10.901+0000 7f7d78274640 1 -- 172.21.15.27:0/223618647 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d7015d560 con 0x7f7d7015a270 2024-09-12T03:17:10.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:10.902+0000 7f7d66ffd640 1 -- 172.21.15.27:0/223618647 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7d68042020 con 0x7f7d7015a270 2024-09-12T03:17:10.904 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:10.902+0000 7f7d66ffd640 1 --2- 172.21.15.27:0/223618647 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f7d5406be30 0x7f7d5406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:10.904 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:10.902+0000 7f7d66ffd640 1 -- 172.21.15.27:0/223618647 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 5609+0+0 (secure 0 0 0) 0x7f7d680ab1a0 con 0x7f7d7015a270 2024-09-12T03:17:10.904 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:10.902+0000 7f7d76010640 1 --2- 172.21.15.27:0/223618647 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f7d5406be30 0x7f7d5406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:10.904 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:10.903+0000 7f7d76010640 1 --2- 172.21.15.27:0/223618647 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f7d5406be30 0x7f7d5406e2f0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f7d700acac0 tx=0x7f7d60009190 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:10.906 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:10.907+0000 7f7d66ffd640 1 -- 172.21.15.27:0/223618647 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f7d68049030 con 0x7f7d7015a270 2024-09-12T03:17:11.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:11.118+0000 7f7d78274640 1 -- 172.21.15.27:0/223618647 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f7d700ccdb0 con 0x7f7d7015a270 2024-09-12T03:17:11.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:11.758+0000 7f7d66ffd640 1 -- 172.21.15.27:0/223618647 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 4 and data pool 5 v11) ==== 172+0+0 (secure 0 0 0) 0x7f7d680794e0 con 0x7f7d7015a270 2024-09-12T03:17:11.758 INFO:teuthology.orchestra.run.smithi027.stderr:new fs with metadata pool 4 and data pool 5 2024-09-12T03:17:11.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:11.762+0000 7f7d78274640 1 -- 172.21.15.27:0/223618647 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f7d5406be30 msgr2=0x7f7d5406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:11.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:11.762+0000 7f7d78274640 1 --2- 172.21.15.27:0/223618647 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f7d5406be30 0x7f7d5406e2f0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f7d700acac0 tx=0x7f7d60009190 comp rx=0 tx=0).stop 2024-09-12T03:17:11.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:11.763+0000 7f7d78274640 1 -- 172.21.15.27:0/223618647 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7d7015a270 msgr2=0x7f7d701f5f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:11.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:11.763+0000 7f7d78274640 1 --2- 172.21.15.27:0/223618647 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7d7015a270 0x7f7d701f5f90 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f7d68009f70 tx=0x7f7d68004a40 comp rx=0 tx=0).stop 2024-09-12T03:17:11.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:11.763+0000 7f7d78274640 1 -- 172.21.15.27:0/223618647 shutdown_connections 2024-09-12T03:17:11.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:11.763+0000 7f7d78274640 1 --2- 172.21.15.27:0/223618647 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f7d5406be30 0x7f7d5406e2f0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:11.764 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:11.763+0000 7f7d78274640 1 --2- 172.21.15.27:0/223618647 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7d7015a270 0x7f7d701f5f90 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:11.764 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:11.763+0000 7f7d78274640 1 --2- 172.21.15.27:0/223618647 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f7d700cfb20 0x7f7d701f5a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:11.764 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:11.763+0000 7f7d78274640 1 --2- 172.21.15.27:0/223618647 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7d7006d530 0x7f7d70158580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:11.764 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:11.763+0000 7f7d78274640 1 -- 172.21.15.27:0/223618647 >> 172.21.15.27:0/223618647 conn(0x7f7d7006bb30 msgr2=0x7f7d7006bf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:11.764 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:11.763+0000 7f7d78274640 1 -- 172.21.15.27:0/223618647 shutdown_connections 2024-09-12T03:17:11.764 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:11.763+0000 7f7d78274640 1 -- 172.21.15.27:0/223618647 wait complete. 2024-09-12T03:17:11.798 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-12T03:17:11.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:11.901+0000 7f2408253640 1 Processor -- start 2024-09-12T03:17:11.902 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:11.902+0000 7f2408253640 1 -- start start 2024-09-12T03:17:11.905 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:11.902+0000 7f2408253640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f240015eb90 0x7f24000d2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:11.905 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:11.903+0000 7f2408253640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f24000d5200 0x7f24000d1010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:11.905 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:11.903+0000 7f2408253640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f24000d19c0 con 0x7f24000d1550 2024-09-12T03:17:11.905 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:11.903+0000 7f2408253640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f24000b7d20 con 0x7f240015eb90 2024-09-12T03:17:11.906 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:11.903+0000 7f2408253640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f24000d42e0 con 0x7f24000d5200 2024-09-12T03:17:11.906 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:11.903+0000 7f24067f0640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f24000d1550 0x7f24000cf200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:42542/0 (socket says 172.21.15.27:42542) 2024-09-12T03:17:11.906 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:11.903+0000 7f24067f0640 1 -- 172.21.15.27:0/2683230864 learned_addr learned my addr 172.21.15.27:0/2683230864 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:17:11.906 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:11.903+0000 7f2404fed640 1 -- 172.21.15.27:0/2683230864 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2662670570 0 0) 0x7f24000d19c0 con 0x7f24000d1550 2024-09-12T03:17:11.906 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:11.903+0000 7f2404fed640 1 -- 172.21.15.27:0/2683230864 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f23ec003520 con 0x7f24000d1550 2024-09-12T03:17:11.906 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:11.903+0000 7f2405fef640 1 --2- 172.21.15.27:0/2683230864 >> v2:172.21.15.45:3300/0 conn(0x7f240015eb90 0x7f24000d2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:11.906 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:11.903+0000 7f24057ee640 1 --2- 172.21.15.27:0/2683230864 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f24000d5200 0x7f24000d1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:11.906 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:11.903+0000 7f2404fed640 1 -- 172.21.15.27:0/2683230864 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1256815526 0 0) 0x7f23ec003520 con 0x7f24000d1550 2024-09-12T03:17:11.906 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:11.903+0000 7f2404fed640 1 -- 172.21.15.27:0/2683230864 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f24000d5200 msgr2=0x7f24000d1010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:11.906 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:11.903+0000 7f2404fed640 1 --2- 172.21.15.27:0/2683230864 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f24000d5200 0x7f24000d1010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:11.906 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:11.903+0000 7f2404fed640 1 -- 172.21.15.27:0/2683230864 >> v2:172.21.15.45:3300/0 conn(0x7f240015eb90 msgr2=0x7f24000d2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:11.906 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:11.903+0000 7f2404fed640 1 --2- 172.21.15.27:0/2683230864 >> v2:172.21.15.45:3300/0 conn(0x7f240015eb90 0x7f24000d2df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:11.906 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:11.903+0000 7f2404fed640 1 -- 172.21.15.27:0/2683230864 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24000d3760 con 0x7f24000d1550 2024-09-12T03:17:11.906 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:11.903+0000 7f2404fed640 1 -- 172.21.15.27:0/2683230864 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 847755213 0 0) 0x7f23f80019f0 con 0x7f24000d1550 2024-09-12T03:17:11.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:11.903+0000 7f2404fed640 1 -- 172.21.15.27:0/2683230864 >> v1:172.21.15.27:6789/0 conn(0x7f24000d1550 legacy=0x7f24000cf200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:11.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:11.903+0000 7f2404fed640 1 --2- 172.21.15.27:0/2683230864 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f23ec0039b0 0x7f23ec003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:11.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:11.903+0000 7f2404fed640 1 --2- 172.21.15.27:0/2683230864 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f23ec004800 0x7f23ec004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:11.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:11.903+0000 7f2404fed640 1 --2- 172.21.15.27:0/2683230864 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f24000d5200 0x7f23ec0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:11.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:11.903+0000 7f2404fed640 1 -- 172.21.15.27:0/2683230864 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f24000d42e0 con 0x7f23ec0039b0 2024-09-12T03:17:11.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:11.903+0000 7f2404fed640 1 -- 172.21.15.27:0/2683230864 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f23ec00b9c0 con 0x7f23ec004800 2024-09-12T03:17:11.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:11.903+0000 7f2404fed640 1 -- 172.21.15.27:0/2683230864 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f23ec00bb40 con 0x7f24000d5200 2024-09-12T03:17:11.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:11.903+0000 7f2405fef640 1 --2- 172.21.15.27:0/2683230864 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f23ec0039b0 0x7f23ec003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:11.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:11.904+0000 7f24057ee640 1 --2- 172.21.15.27:0/2683230864 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f23ec004800 0x7f23ec004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:11.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:11.904+0000 7f24067f0640 1 --2- 172.21.15.27:0/2683230864 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f24000d5200 0x7f23ec0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:11.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:11.904+0000 7f2405fef640 1 -- 172.21.15.27:0/2683230864 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f24000d5200 msgr2=0x7f23ec0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:11.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:11.904+0000 7f2405fef640 1 --2- 172.21.15.27:0/2683230864 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f24000d5200 0x7f23ec0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:11.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:11.904+0000 7f2405fef640 1 -- 172.21.15.27:0/2683230864 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f23ec004800 msgr2=0x7f23ec004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:11.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:11.904+0000 7f2405fef640 1 --2- 172.21.15.27:0/2683230864 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f23ec004800 0x7f23ec004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:11.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:11.904+0000 7f2405fef640 1 -- 172.21.15.27:0/2683230864 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23ec003520 con 0x7f23ec0039b0 2024-09-12T03:17:11.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:11.904+0000 7f2405fef640 1 --2- 172.21.15.27:0/2683230864 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f23ec0039b0 0x7f23ec003dd0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f23f4009ac0 tx=0x7f23ec005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddfcf4c51c6345c9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:11.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:11.905+0000 7f2404fed640 1 -- 172.21.15.27:0/2683230864 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23f4036070 con 0x7f23ec0039b0 2024-09-12T03:17:11.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:11.905+0000 7f2404fed640 1 -- 172.21.15.27:0/2683230864 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23f4031400 con 0x7f23ec0039b0 2024-09-12T03:17:11.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:11.905+0000 7f2404fed640 1 -- 172.21.15.27:0/2683230864 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23f4009690 con 0x7f23ec0039b0 2024-09-12T03:17:11.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:11.905+0000 7f2408253640 1 -- 172.21.15.27:0/2683230864 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f23ec0039b0 msgr2=0x7f23ec003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:11.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:11.905+0000 7f2408253640 1 --2- 172.21.15.27:0/2683230864 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f23ec0039b0 0x7f23ec003dd0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f23f4009ac0 tx=0x7f23ec005830 comp rx=0 tx=0).stop 2024-09-12T03:17:11.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:11.905+0000 7f24067f0640 1 -- 172.21.15.27:0/2683230864 reap_dead start 2024-09-12T03:17:11.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:11.905+0000 7f2408253640 1 -- 172.21.15.27:0/2683230864 shutdown_connections 2024-09-12T03:17:11.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:11.905+0000 7f2408253640 1 -- 172.21.15.27:0/2683230864 >> 172.21.15.27:0/2683230864 conn(0x7f240006dce0 msgr2=0x7f240006e0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:11.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:11.905+0000 7f2408253640 1 -- 172.21.15.27:0/2683230864 shutdown_connections 2024-09-12T03:17:11.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:11.905+0000 7f2408253640 1 -- 172.21.15.27:0/2683230864 wait complete. 2024-09-12T03:17:11.911 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:11.906+0000 7f2408253640 1 Processor -- start 2024-09-12T03:17:11.911 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:11.907+0000 7f2408253640 1 -- start start 2024-09-12T03:17:11.911 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:11.907+0000 7f2408253640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f23ec0039b0 0x7f240014cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:11.911 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:11.907+0000 7f2408253640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f24000c8480 0x7f24000ab210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:11.911 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:11.907+0000 7f2408253640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f24000c9fb0 0x7f240014d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:11.911 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:11.907+0000 7f2408253640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f24000d4000 con 0x7f23ec0039b0 2024-09-12T03:17:11.911 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:11.907+0000 7f2408253640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f24000d3d20 con 0x7f24000c9fb0 2024-09-12T03:17:11.911 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:11.907+0000 7f2408253640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f24000d3330 con 0x7f24000c8480 2024-09-12T03:17:11.912 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:11.907+0000 7f2405fef640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f23ec0039b0 0x7f240014cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:11.912 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:11.907+0000 7f2405fef640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f23ec0039b0 0x7f240014cdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36872/0 (socket says 172.21.15.27:36872) 2024-09-12T03:17:11.912 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:11.907+0000 7f2405fef640 1 -- 172.21.15.27:0/3208246289 learned_addr learned my addr 172.21.15.27:0/3208246289 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:17:11.912 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:11.907+0000 7f24057ee640 1 --2- 172.21.15.27:0/3208246289 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f24000c8480 0x7f24000ab210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:11.912 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:11.908+0000 7f24067f0640 1 --2- 172.21.15.27:0/3208246289 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f24000c9fb0 0x7f240014d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:11.912 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:11.908+0000 7f2405fef640 1 -- 172.21.15.27:0/3208246289 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f24000c8480 msgr2=0x7f24000ab210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:11.912 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:11.908+0000 7f2405fef640 1 --2- 172.21.15.27:0/3208246289 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f24000c8480 0x7f24000ab210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:11.912 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:11.908+0000 7f2405fef640 1 -- 172.21.15.27:0/3208246289 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f24000c9fb0 msgr2=0x7f240014d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:11.912 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:11.908+0000 7f2405fef640 1 --2- 172.21.15.27:0/3208246289 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f24000c9fb0 0x7f240014d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:11.912 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:11.908+0000 7f2405fef640 1 -- 172.21.15.27:0/3208246289 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24000ab750 con 0x7f23ec0039b0 2024-09-12T03:17:11.912 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:11.908+0000 7f2405fef640 1 --2- 172.21.15.27:0/3208246289 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f23ec0039b0 0x7f240014cdc0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f23f402abe0 tx=0x7f23f4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:11.912 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:11.908+0000 7f23f27fc640 1 -- 172.21.15.27:0/3208246289 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23f4040020 con 0x7f23ec0039b0 2024-09-12T03:17:11.912 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:11.908+0000 7f23f27fc640 1 -- 172.21.15.27:0/3208246289 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23f4003890 con 0x7f23ec0039b0 2024-09-12T03:17:11.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:11.908+0000 7f2408253640 1 -- 172.21.15.27:0/3208246289 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24000caf30 con 0x7f23ec0039b0 2024-09-12T03:17:11.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:11.908+0000 7f23f27fc640 1 -- 172.21.15.27:0/3208246289 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23f4036040 con 0x7f23ec0039b0 2024-09-12T03:17:11.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:11.908+0000 7f2408253640 1 -- 172.21.15.27:0/3208246289 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24000ca430 con 0x7f23ec0039b0 2024-09-12T03:17:11.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:11.909+0000 7f2408253640 1 -- 172.21.15.27:0/3208246289 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f240015ef70 con 0x7f23ec0039b0 2024-09-12T03:17:11.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:11.910+0000 7f23f27fc640 1 -- 172.21.15.27:0/3208246289 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f23f4043020 con 0x7f23ec0039b0 2024-09-12T03:17:11.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:11.910+0000 7f23f27fc640 1 --2- 172.21.15.27:0/3208246289 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f23e406be60 0x7f23e406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:11.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:11.910+0000 7f23f27fc640 1 -- 172.21.15.27:0/3208246289 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5709+0+0 (secure 0 0 0) 0x7f23f40ab370 con 0x7f23ec0039b0 2024-09-12T03:17:11.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:11.911+0000 7f24057ee640 1 --2- 172.21.15.27:0/3208246289 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f23e406be60 0x7f23e406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:11.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:11.911+0000 7f24057ee640 1 --2- 172.21.15.27:0/3208246289 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f23e406be60 0x7f23e406e320 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f24001608d0 tx=0x7f23fc009190 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:11.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:11.914+0000 7f23f27fc640 1 -- 172.21.15.27:0/3208246289 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f23f404a020 con 0x7f23ec0039b0 2024-09-12T03:17:12.090 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:12.090+0000 7f2408253640 1 -- 172.21.15.27:0/3208246289 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f24000cb580 con 0x7f23ec0039b0 2024-09-12T03:17:12.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:12.091+0000 7f23f27fc640 1 -- 172.21.15.27:0/3208246289 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v31) ==== 74+0+16694 (secure 0 0 0) 0x7f23f4079880 con 0x7f23ec0039b0 2024-09-12T03:17:12.091 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:17:12.092 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":31,"fsid":"d2df504e-2647-4385-a5db-9fdf69f487a0","created":"2024-09-12T03:16:07.778937+0000","modified":"2024-09-12T03:17:11.119638+0000","last_up_change":"2024-09-12T03:16:19.314778+0000","last_in_change":"2024-09-12T03:16:09.893086+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T03:16:20.097646+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-09-12T03:17:09.018856+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-09-12T03:17:10.105881+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1851116c-1fb3-4dc9-9772-f90e2ef4c355","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6817","nonce":3002321488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6819","nonce":3002321488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6823","nonce":3002321488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6821","nonce":3002321488}]},"public_addr":"172.21.15.27:6817/3002321488","cluster_addr":"172.21.15.27:6819/3002321488","heartbeat_back_addr":"172.21.15.27:6823/3002321488","heartbeat_front_addr":"172.21.15.27:6821/3002321488","state":["exists","up"]},{"osd":1,"uuid":"f38ae17d-73eb-4204-b99d-975fa806fb87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6809","nonce":1613102886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6811","nonce":1613102886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6815","nonce":1613102886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6813","nonce":1613102886}]},"public_addr":"172.21.15.27:6809/1613102886","cluster_addr":"172.21.15.27:6811/1613102886","heartbeat_back_addr":"172.21.15.27:6815/1613102886","heartbeat_front_addr":"172.21.15.27:6813/1613102886","state":["exists","up"]},{"osd":2,"uuid":"fb12dcc8-bdd0-42b8-b46d-8b10106224bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6825","nonce":3534723355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6827","nonce":3534723355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6831","nonce":3534723355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6829","nonce":3534723355}]},"public_addr":"172.21.15.27:6825/3534723355","cluster_addr":"172.21.15.27:6827/3534723355","heartbeat_back_addr":"172.21.15.27:6831/3534723355","heartbeat_front_addr":"172.21.15.27:6829/3534723355","state":["exists","up"]},{"osd":3,"uuid":"d74d3f49-9c24-4b9f-98e2-3ecd7b5ef3b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6801","nonce":80661598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6803","nonce":80661598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6807","nonce":80661598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6805","nonce":80661598}]},"public_addr":"172.21.15.27:6801/80661598","cluster_addr":"172.21.15.27:6803/80661598","heartbeat_back_addr":"172.21.15.27:6807/80661598","heartbeat_front_addr":"172.21.15.27:6805/80661598","state":["exists","up"]},{"osd":4,"uuid":"2bf123f2-1b0d-41c4-aa3b-805bab267e59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6817","nonce":3943189827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6819","nonce":3943189827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6823","nonce":3943189827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6821","nonce":3943189827}]},"public_addr":"172.21.15.45:6817/3943189827","cluster_addr":"172.21.15.45:6819/3943189827","heartbeat_back_addr":"172.21.15.45:6823/3943189827","heartbeat_front_addr":"172.21.15.45:6821/3943189827","state":["exists","up"]},{"osd":5,"uuid":"c49ba5da-91c6-4a86-a97b-4aba89c253ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6809","nonce":3620760330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6811","nonce":3620760330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6815","nonce":3620760330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6813","nonce":3620760330}]},"public_addr":"172.21.15.45:6809/3620760330","cluster_addr":"172.21.15.45:6811/3620760330","heartbeat_back_addr":"172.21.15.45:6815/3620760330","heartbeat_front_addr":"172.21.15.45:6813/3620760330","state":["exists","up"]},{"osd":6,"uuid":"b0123475-89f7-4a34-9e33-d797b99dd9dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6801","nonce":1989083510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6803","nonce":1989083510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6807","nonce":1989083510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6805","nonce":1989083510}]},"public_addr":"172.21.15.45:6801/1989083510","cluster_addr":"172.21.15.45:6803/1989083510","heartbeat_back_addr":"172.21.15.45:6807/1989083510","heartbeat_front_addr":"172.21.15.45:6805/1989083510","state":["exists","up"]},{"osd":7,"uuid":"4dad773d-b7d8-46fc-8a10-a972aad27b29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6825","nonce":1513300294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6827","nonce":1513300294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6831","nonce":1513300294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6829","nonce":1513300294}]},"public_addr":"172.21.15.45:6825/1513300294","cluster_addr":"172.21.15.45:6827/1513300294","heartbeat_back_addr":"172.21.15.45:6831/1513300294","heartbeat_front_addr":"172.21.15.45:6829/1513300294","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.881855+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:17.133252+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.735339+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.500742+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.477571+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:17.167667+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.470291+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.873743+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T03:17:12.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:12.093+0000 7f2408253640 1 -- 172.21.15.27:0/3208246289 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f23e406be60 msgr2=0x7f23e406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:12.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:12.093+0000 7f2408253640 1 --2- 172.21.15.27:0/3208246289 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f23e406be60 0x7f23e406e320 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f24001608d0 tx=0x7f23fc009190 comp rx=0 tx=0).stop 2024-09-12T03:17:12.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:12.093+0000 7f2408253640 1 -- 172.21.15.27:0/3208246289 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f23ec0039b0 msgr2=0x7f240014cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:12.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:12.093+0000 7f2408253640 1 --2- 172.21.15.27:0/3208246289 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f23ec0039b0 0x7f240014cdc0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f23f402abe0 tx=0x7f23f4004730 comp rx=0 tx=0).stop 2024-09-12T03:17:12.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:12.093+0000 7f2408253640 1 -- 172.21.15.27:0/3208246289 shutdown_connections 2024-09-12T03:17:12.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:12.093+0000 7f2408253640 1 --2- 172.21.15.27:0/3208246289 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f23e406be60 0x7f23e406e320 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:12.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:12.093+0000 7f2408253640 1 --2- 172.21.15.27:0/3208246289 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f24000c9fb0 0x7f240014d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:12.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:12.093+0000 7f2408253640 1 --2- 172.21.15.27:0/3208246289 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f24000c8480 0x7f24000ab210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:12.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:12.093+0000 7f2408253640 1 --2- 172.21.15.27:0/3208246289 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f23ec0039b0 0x7f240014cdc0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:12.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:12.093+0000 7f2408253640 1 -- 172.21.15.27:0/3208246289 >> 172.21.15.27:0/3208246289 conn(0x7f240006dce0 msgr2=0x7f24000cdfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:12.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:12.094+0000 7f2408253640 1 -- 172.21.15.27:0/3208246289 shutdown_connections 2024-09-12T03:17:12.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:12.094+0000 7f2408253640 1 -- 172.21.15.27:0/3208246289 wait complete. 2024-09-12T03:17:12.105 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-12T03:17:12.207 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:12.207+0000 7f46c359e640 1 Processor -- start 2024-09-12T03:17:12.207 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:12.207+0000 7f46c359e640 1 -- start start 2024-09-12T03:17:12.209 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:12.208+0000 7f46c359e640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f46c4147fe0 0x7f46c40b4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:12.210 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:12.208+0000 7f46c359e640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f46c4073f90 0x7f46c406bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:12.210 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:12.208+0000 7f46c359e640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f46c40b63b0 con 0x7f46c40d1e60 2024-09-12T03:17:12.210 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:12.208+0000 7f46c359e640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f46c41483c0 con 0x7f46c4147fe0 2024-09-12T03:17:12.210 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:12.208+0000 7f46c359e640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f46c40b4b10 con 0x7f46c4073f90 2024-09-12T03:17:12.210 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:12.208+0000 7f46c259c640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f46c4147fe0 0x7f46c40b4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:12.210 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:12.208+0000 7f46c2d9d640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f46c40d1e60 0x7f46c40ae8d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:42554/0 (socket says 172.21.15.27:42554) 2024-09-12T03:17:12.210 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:12.208+0000 7f46c2d9d640 1 -- 172.21.15.27:0/2334447238 learned_addr learned my addr 172.21.15.27:0/2334447238 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:17:12.210 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:12.209+0000 7f46c1d9b640 1 --2- 172.21.15.27:0/2334447238 >> v2:172.21.15.45:3300/0 conn(0x7f46c4073f90 0x7f46c406bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:12.210 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:12.209+0000 7f46c159a640 1 -- 172.21.15.27:0/2334447238 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 387103602 0 0) 0x7f46c40b63b0 con 0x7f46c40d1e60 2024-09-12T03:17:12.210 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:12.209+0000 7f46c159a640 1 -- 172.21.15.27:0/2334447238 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f46b0003520 con 0x7f46c40d1e60 2024-09-12T03:17:12.210 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:12.209+0000 7f46c259c640 1 -- 172.21.15.27:0/2334447238 >> v2:172.21.15.45:3300/0 conn(0x7f46c4073f90 msgr2=0x7f46c406bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:12.210 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:12.209+0000 7f46c259c640 1 --2- 172.21.15.27:0/2334447238 >> v2:172.21.15.45:3300/0 conn(0x7f46c4073f90 0x7f46c406bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:12.211 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:12.209+0000 7f46c259c640 1 -- 172.21.15.27:0/2334447238 >> v1:172.21.15.27:6789/0 conn(0x7f46c40d1e60 legacy=0x7f46c40ae8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:12.211 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:12.209+0000 7f46c259c640 1 -- 172.21.15.27:0/2334447238 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46c40b6690 con 0x7f46c4147fe0 2024-09-12T03:17:12.211 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:12.209+0000 7f46c259c640 1 --2- 172.21.15.27:0/2334447238 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f46c4147fe0 0x7f46c40b4fa0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f46c40715f0 tx=0x7f46b402a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=139823a88439ddf5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:12.211 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:12.210+0000 7f46c159a640 1 -- 172.21.15.27:0/2334447238 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46b4007740 con 0x7f46c4147fe0 2024-09-12T03:17:12.211 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:12.210+0000 7f46c159a640 1 -- 172.21.15.27:0/2334447238 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46b40078e0 con 0x7f46c4147fe0 2024-09-12T03:17:12.211 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:12.210+0000 7f46c159a640 1 -- 172.21.15.27:0/2334447238 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46b4007cb0 con 0x7f46c4147fe0 2024-09-12T03:17:12.213 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:12.210+0000 7f46c359e640 1 -- 172.21.15.27:0/2334447238 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f46c4147fe0 msgr2=0x7f46c40b4fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:12.213 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:12.210+0000 7f46c359e640 1 --2- 172.21.15.27:0/2334447238 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f46c4147fe0 0x7f46c40b4fa0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f46c40715f0 tx=0x7f46b402a680 comp rx=0 tx=0).stop 2024-09-12T03:17:12.213 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:12.210+0000 7f46c359e640 1 -- 172.21.15.27:0/2334447238 shutdown_connections 2024-09-12T03:17:12.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:12.210+0000 7f46c359e640 1 --2- 172.21.15.27:0/2334447238 >> v2:172.21.15.45:3300/0 conn(0x7f46c4073f90 0x7f46c406bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:12.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:12.210+0000 7f46c359e640 1 --2- 172.21.15.27:0/2334447238 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f46c4147fe0 0x7f46c40b4fa0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:12.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:12.210+0000 7f46c359e640 1 -- 172.21.15.27:0/2334447238 >> 172.21.15.27:0/2334447238 conn(0x7f46c40d1370 msgr2=0x7f46c40d1750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:12.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:12.210+0000 7f46c359e640 1 -- 172.21.15.27:0/2334447238 shutdown_connections 2024-09-12T03:17:12.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:12.210+0000 7f46c359e640 1 -- 172.21.15.27:0/2334447238 wait complete. 2024-09-12T03:17:12.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:12.212+0000 7f46c359e640 1 Processor -- start 2024-09-12T03:17:12.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:12.212+0000 7f46c359e640 1 -- start start 2024-09-12T03:17:12.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:12.212+0000 7f46c359e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f46c4073f90 0x7f46c4141060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:12.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:12.212+0000 7f46c359e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f46c40d1e60 0x7f46c4154cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:12.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:12.212+0000 7f46c359e640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f46c4147fe0 0x7f46c4143110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:12.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:12.212+0000 7f46c359e640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f46c40b5ff0 con 0x7f46c40d1e60 2024-09-12T03:17:12.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:12.212+0000 7f46c359e640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f46c40b47a0 con 0x7f46c4073f90 2024-09-12T03:17:12.215 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:12.212+0000 7f46c359e640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f46c40b6170 con 0x7f46c4147fe0 2024-09-12T03:17:12.215 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:12.212+0000 7f46c1d9b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f46c40d1e60 0x7f46c4154cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:12.215 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:12.213+0000 7f46c1d9b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f46c40d1e60 0x7f46c4154cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36894/0 (socket says 172.21.15.27:36894) 2024-09-12T03:17:12.215 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:12.213+0000 7f46c1d9b640 1 -- 172.21.15.27:0/272855325 learned_addr learned my addr 172.21.15.27:0/272855325 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:17:12.215 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:12.213+0000 7f46c259c640 1 --2- 172.21.15.27:0/272855325 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f46c4073f90 0x7f46c4141060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:12.215 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:12.213+0000 7f46c2d9d640 1 --2- 172.21.15.27:0/272855325 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f46c4147fe0 0x7f46c4143110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:12.215 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:12.213+0000 7f46c1d9b640 1 -- 172.21.15.27:0/272855325 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f46c4147fe0 msgr2=0x7f46c4143110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:12.215 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:12.213+0000 7f46c1d9b640 1 --2- 172.21.15.27:0/272855325 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f46c4147fe0 0x7f46c4143110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:12.215 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:12.213+0000 7f46c1d9b640 1 -- 172.21.15.27:0/272855325 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f46c4073f90 msgr2=0x7f46c4141060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:12.215 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:12.213+0000 7f46c1d9b640 1 --2- 172.21.15.27:0/272855325 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f46c4073f90 0x7f46c4141060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:12.215 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:12.213+0000 7f46c1d9b640 1 -- 172.21.15.27:0/272855325 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46c4141a90 con 0x7f46c40d1e60 2024-09-12T03:17:12.215 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:12.213+0000 7f46c259c640 1 --2- 172.21.15.27:0/272855325 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f46c4073f90 0x7f46c4141060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:17:12.216 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:12.213+0000 7f46c1d9b640 1 --2- 172.21.15.27:0/272855325 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f46c40d1e60 0x7f46c4154cc0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f46ac00d7d0 tx=0x7f46ac00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:12.216 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:12.213+0000 7f46baffd640 1 -- 172.21.15.27:0/272855325 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46ac00b6c0 con 0x7f46c40d1e60 2024-09-12T03:17:12.216 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:12.213+0000 7f46baffd640 1 -- 172.21.15.27:0/272855325 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46ac019070 con 0x7f46c40d1e60 2024-09-12T03:17:12.216 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:12.213+0000 7f46baffd640 1 -- 172.21.15.27:0/272855325 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46ac00ba80 con 0x7f46c40d1e60 2024-09-12T03:17:12.216 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:12.213+0000 7f46c359e640 1 -- 172.21.15.27:0/272855325 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46c4143f90 con 0x7f46c40d1e60 2024-09-12T03:17:12.216 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:12.213+0000 7f46c359e640 1 -- 172.21.15.27:0/272855325 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46c4155200 con 0x7f46c40d1e60 2024-09-12T03:17:12.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:12.214+0000 7f46c359e640 1 -- 172.21.15.27:0/272855325 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46c40b4f80 con 0x7f46c40d1e60 2024-09-12T03:17:12.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:12.215+0000 7f46baffd640 1 -- 172.21.15.27:0/272855325 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f46ac00bc20 con 0x7f46c40d1e60 2024-09-12T03:17:12.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:12.215+0000 7f46baffd640 1 --2- 172.21.15.27:0/272855325 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f46a406be30 0x7f46a406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:12.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:12.215+0000 7f46baffd640 1 -- 172.21.15.27:0/272855325 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5709+0+0 (secure 0 0 0) 0x7f46ac08b380 con 0x7f46c40d1e60 2024-09-12T03:17:12.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:12.216+0000 7f46c259c640 1 --2- 172.21.15.27:0/272855325 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f46a406be30 0x7f46a406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:12.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:12.216+0000 7f46c259c640 1 --2- 172.21.15.27:0/272855325 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f46a406be30 0x7f46a406e2f0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f46b4034000 tx=0x7f46b4033040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:12.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:12.220+0000 7f46baffd640 1 -- 172.21.15.27:0/272855325 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f46ac01c360 con 0x7f46c40d1e60 2024-09-12T03:17:12.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:12.395+0000 7f46c359e640 1 -- 172.21.15.27:0/272855325 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f46c4146380 con 0x7f46c40d1e60 2024-09-12T03:17:12.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:12.396+0000 7f46baffd640 1 -- 172.21.15.27:0/272855325 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v31) ==== 74+0+16694 (secure 0 0 0) 0x7f46ac059ca0 con 0x7f46c40d1e60 2024-09-12T03:17:12.397 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:17:12.397 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":31,"fsid":"d2df504e-2647-4385-a5db-9fdf69f487a0","created":"2024-09-12T03:16:07.778937+0000","modified":"2024-09-12T03:17:11.119638+0000","last_up_change":"2024-09-12T03:16:19.314778+0000","last_in_change":"2024-09-12T03:16:09.893086+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T03:16:20.097646+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-09-12T03:17:09.018856+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-09-12T03:17:10.105881+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1851116c-1fb3-4dc9-9772-f90e2ef4c355","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6817","nonce":3002321488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6819","nonce":3002321488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6823","nonce":3002321488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6821","nonce":3002321488}]},"public_addr":"172.21.15.27:6817/3002321488","cluster_addr":"172.21.15.27:6819/3002321488","heartbeat_back_addr":"172.21.15.27:6823/3002321488","heartbeat_front_addr":"172.21.15.27:6821/3002321488","state":["exists","up"]},{"osd":1,"uuid":"f38ae17d-73eb-4204-b99d-975fa806fb87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6809","nonce":1613102886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6811","nonce":1613102886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6815","nonce":1613102886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6813","nonce":1613102886}]},"public_addr":"172.21.15.27:6809/1613102886","cluster_addr":"172.21.15.27:6811/1613102886","heartbeat_back_addr":"172.21.15.27:6815/1613102886","heartbeat_front_addr":"172.21.15.27:6813/1613102886","state":["exists","up"]},{"osd":2,"uuid":"fb12dcc8-bdd0-42b8-b46d-8b10106224bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6825","nonce":3534723355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6827","nonce":3534723355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6831","nonce":3534723355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6829","nonce":3534723355}]},"public_addr":"172.21.15.27:6825/3534723355","cluster_addr":"172.21.15.27:6827/3534723355","heartbeat_back_addr":"172.21.15.27:6831/3534723355","heartbeat_front_addr":"172.21.15.27:6829/3534723355","state":["exists","up"]},{"osd":3,"uuid":"d74d3f49-9c24-4b9f-98e2-3ecd7b5ef3b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6801","nonce":80661598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6803","nonce":80661598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6807","nonce":80661598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6805","nonce":80661598}]},"public_addr":"172.21.15.27:6801/80661598","cluster_addr":"172.21.15.27:6803/80661598","heartbeat_back_addr":"172.21.15.27:6807/80661598","heartbeat_front_addr":"172.21.15.27:6805/80661598","state":["exists","up"]},{"osd":4,"uuid":"2bf123f2-1b0d-41c4-aa3b-805bab267e59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6817","nonce":3943189827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6819","nonce":3943189827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6823","nonce":3943189827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6821","nonce":3943189827}]},"public_addr":"172.21.15.45:6817/3943189827","cluster_addr":"172.21.15.45:6819/3943189827","heartbeat_back_addr":"172.21.15.45:6823/3943189827","heartbeat_front_addr":"172.21.15.45:6821/3943189827","state":["exists","up"]},{"osd":5,"uuid":"c49ba5da-91c6-4a86-a97b-4aba89c253ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6809","nonce":3620760330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6811","nonce":3620760330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6815","nonce":3620760330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6813","nonce":3620760330}]},"public_addr":"172.21.15.45:6809/3620760330","cluster_addr":"172.21.15.45:6811/3620760330","heartbeat_back_addr":"172.21.15.45:6815/3620760330","heartbeat_front_addr":"172.21.15.45:6813/3620760330","state":["exists","up"]},{"osd":6,"uuid":"b0123475-89f7-4a34-9e33-d797b99dd9dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6801","nonce":1989083510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6803","nonce":1989083510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6807","nonce":1989083510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6805","nonce":1989083510}]},"public_addr":"172.21.15.45:6801/1989083510","cluster_addr":"172.21.15.45:6803/1989083510","heartbeat_back_addr":"172.21.15.45:6807/1989083510","heartbeat_front_addr":"172.21.15.45:6805/1989083510","state":["exists","up"]},{"osd":7,"uuid":"4dad773d-b7d8-46fc-8a10-a972aad27b29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6825","nonce":1513300294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6827","nonce":1513300294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6831","nonce":1513300294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6829","nonce":1513300294}]},"public_addr":"172.21.15.45:6825/1513300294","cluster_addr":"172.21.15.45:6827/1513300294","heartbeat_back_addr":"172.21.15.45:6831/1513300294","heartbeat_front_addr":"172.21.15.45:6829/1513300294","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.881855+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:17.133252+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.735339+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.500742+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.477571+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:17.167667+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.470291+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.873743+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T03:17:12.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:12.398+0000 7f46c359e640 1 -- 172.21.15.27:0/272855325 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f46a406be30 msgr2=0x7f46a406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:12.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:12.398+0000 7f46c359e640 1 --2- 172.21.15.27:0/272855325 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f46a406be30 0x7f46a406e2f0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f46b4034000 tx=0x7f46b4033040 comp rx=0 tx=0).stop 2024-09-12T03:17:12.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:12.398+0000 7f46c359e640 1 -- 172.21.15.27:0/272855325 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f46c40d1e60 msgr2=0x7f46c4154cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:12.400 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:12.398+0000 7f46c359e640 1 --2- 172.21.15.27:0/272855325 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f46c40d1e60 0x7f46c4154cc0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f46ac00d7d0 tx=0x7f46ac00dc90 comp rx=0 tx=0).stop 2024-09-12T03:17:12.400 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:12.398+0000 7f46c359e640 1 -- 172.21.15.27:0/272855325 shutdown_connections 2024-09-12T03:17:12.400 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:12.398+0000 7f46c359e640 1 --2- 172.21.15.27:0/272855325 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f46a406be30 0x7f46a406e2f0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:12.400 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:12.398+0000 7f46c359e640 1 --2- 172.21.15.27:0/272855325 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f46c4147fe0 0x7f46c4143110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:12.400 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:12.398+0000 7f46c359e640 1 --2- 172.21.15.27:0/272855325 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f46c40d1e60 0x7f46c4154cc0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:12.400 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:12.398+0000 7f46c359e640 1 --2- 172.21.15.27:0/272855325 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f46c4073f90 0x7f46c4141060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:12.400 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:12.398+0000 7f46c359e640 1 -- 172.21.15.27:0/272855325 >> 172.21.15.27:0/272855325 conn(0x7f46c40d1370 msgr2=0x7f46c413d570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:12.400 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:12.398+0000 7f46c359e640 1 -- 172.21.15.27:0/272855325 shutdown_connections 2024-09-12T03:17:12.400 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:12.398+0000 7f46c359e640 1 -- 172.21.15.27:0/272855325 wait complete. 2024-09-12T03:17:12.410 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-12T03:17:12.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:12.514+0000 7f6bc84b7640 1 Processor -- start 2024-09-12T03:17:12.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:12.515+0000 7f6bc84b7640 1 -- start start 2024-09-12T03:17:12.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:12.515+0000 7f6bc84b7640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f6bc0070d30 0x7f6bc006cc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:12.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:12.515+0000 7f6bc84b7640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f6bc006e960 0x7f6bc00d0440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:12.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:12.516+0000 7f6bc84b7640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6bc00cf9f0 con 0x7f6bc00d0980 2024-09-12T03:17:12.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:12.516+0000 7f6bc84b7640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f6bc00d0df0 con 0x7f6bc006e960 2024-09-12T03:17:12.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:12.516+0000 7f6bc84b7640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f6bc00d1a40 con 0x7f6bc0070d30 2024-09-12T03:17:12.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:12.516+0000 7f6bc5a52640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f6bc006e960 0x7f6bc00d0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:12.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:12.516+0000 7f6bc5a52640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f6bc006e960 0x7f6bc00d0440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36904/0 (socket says 172.21.15.27:36904) 2024-09-12T03:17:12.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:12.516+0000 7f6bc5a52640 1 -- 172.21.15.27:0/2197873145 learned_addr learned my addr 172.21.15.27:0/2197873145 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:17:12.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:12.516+0000 7f6bc6253640 1 --2- 172.21.15.27:0/2197873145 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f6bc0070d30 0x7f6bc006cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:12.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:12.516+0000 7f6bc5a52640 1 -- 172.21.15.27:0/2197873145 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f6bc0070d30 msgr2=0x7f6bc006cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:12.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:12.516+0000 7f6bc5a52640 1 --2- 172.21.15.27:0/2197873145 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f6bc0070d30 0x7f6bc006cc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:12.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:12.516+0000 7f6bc5a52640 1 -- 172.21.15.27:0/2197873145 >> v1:172.21.15.45:6789/0 conn(0x7f6bc00d0980 legacy=0x7f6bc00d21b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:12.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:12.516+0000 7f6bc5a52640 1 -- 172.21.15.27:0/2197873145 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bc00cfc90 con 0x7f6bc006e960 2024-09-12T03:17:12.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:12.517+0000 7f6bc5a52640 1 --2- 172.21.15.27:0/2197873145 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6bc006e960 0x7f6bc00d0440 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f6bc0071d10 tx=0x7f6bb402a570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1de29d659a44344 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:12.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:12.517+0000 7f6bc5251640 1 -- 172.21.15.27:0/2197873145 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bb4036070 con 0x7f6bc006e960 2024-09-12T03:17:12.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:12.517+0000 7f6bc5251640 1 -- 172.21.15.27:0/2197873145 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6bb4031600 con 0x7f6bc006e960 2024-09-12T03:17:12.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:12.517+0000 7f6bc5251640 1 -- 172.21.15.27:0/2197873145 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bb40319d0 con 0x7f6bc006e960 2024-09-12T03:17:12.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:12.517+0000 7f6bc84b7640 1 -- 172.21.15.27:0/2197873145 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6bc006e960 msgr2=0x7f6bc00d0440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:12.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:12.517+0000 7f6bc84b7640 1 --2- 172.21.15.27:0/2197873145 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6bc006e960 0x7f6bc00d0440 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f6bc0071d10 tx=0x7f6bb402a570 comp rx=0 tx=0).stop 2024-09-12T03:17:12.519 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:12.518+0000 7f6bc84b7640 1 -- 172.21.15.27:0/2197873145 shutdown_connections 2024-09-12T03:17:12.519 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:12.518+0000 7f6bc84b7640 1 --2- 172.21.15.27:0/2197873145 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6bc006e960 0x7f6bc00d0440 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:12.519 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:12.518+0000 7f6bc84b7640 1 --2- 172.21.15.27:0/2197873145 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f6bc0070d30 0x7f6bc006cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:12.519 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:12.518+0000 7f6bc84b7640 1 -- 172.21.15.27:0/2197873145 >> 172.21.15.27:0/2197873145 conn(0x7f6bc00b7c10 msgr2=0x7f6bc009ba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:12.519 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:12.518+0000 7f6bc84b7640 1 -- 172.21.15.27:0/2197873145 shutdown_connections 2024-09-12T03:17:12.519 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:12.518+0000 7f6bc84b7640 1 -- 172.21.15.27:0/2197873145 wait complete. 2024-09-12T03:17:12.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:12.519+0000 7f6bc84b7640 1 Processor -- start 2024-09-12T03:17:12.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:12.519+0000 7f6bc84b7640 1 -- start start 2024-09-12T03:17:12.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:12.520+0000 7f6bc84b7640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6bc006e960 0x7f6bc015e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:12.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:12.520+0000 7f6bc84b7640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6bc0070d30 0x7f6bc01f7b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:12.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:12.520+0000 7f6bc84b7640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f6bc00d0980 0x7f6bc00b6ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:12.522 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:12.520+0000 7f6bc84b7640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f6bc006d180 con 0x7f6bc006e960 2024-09-12T03:17:12.522 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:12.520+0000 7f6bc84b7640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f6bc00d1c90 con 0x7f6bc0070d30 2024-09-12T03:17:12.522 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:12.520+0000 7f6bc84b7640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f6bc00d28c0 con 0x7f6bc00d0980 2024-09-12T03:17:12.522 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:12.520+0000 7f6bc6253640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6bc006e960 0x7f6bc015e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:12.522 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:12.520+0000 7f6bc6253640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6bc006e960 0x7f6bc015e9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36906/0 (socket says 172.21.15.27:36906) 2024-09-12T03:17:12.522 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:12.520+0000 7f6bc6253640 1 -- 172.21.15.27:0/1401313545 learned_addr learned my addr 172.21.15.27:0/1401313545 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:17:12.522 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:12.520+0000 7f6bc5a52640 1 --2- 172.21.15.27:0/1401313545 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6bc0070d30 0x7f6bc01f7b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:12.522 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:12.520+0000 7f6bc6a54640 1 --2- 172.21.15.27:0/1401313545 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f6bc00d0980 0x7f6bc00b6ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:12.522 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:12.520+0000 7f6bc6253640 1 -- 172.21.15.27:0/1401313545 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f6bc00d0980 msgr2=0x7f6bc00b6ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:12.522 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:12.520+0000 7f6bc6253640 1 --2- 172.21.15.27:0/1401313545 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f6bc00d0980 0x7f6bc00b6ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:12.522 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:12.520+0000 7f6bc6253640 1 -- 172.21.15.27:0/1401313545 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6bc0070d30 msgr2=0x7f6bc01f7b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:12.522 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:12.520+0000 7f6bc6253640 1 --2- 172.21.15.27:0/1401313545 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6bc0070d30 0x7f6bc01f7b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:12.522 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:12.520+0000 7f6bc6253640 1 -- 172.21.15.27:0/1401313545 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bc01f80a0 con 0x7f6bc006e960 2024-09-12T03:17:12.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:12.520+0000 7f6bc6a54640 1 --2- 172.21.15.27:0/1401313545 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f6bc00d0980 0x7f6bc00b6ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:17:12.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:12.520+0000 7f6bc6253640 1 --2- 172.21.15.27:0/1401313545 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6bc006e960 0x7f6bc015e9c0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f6bb000c7f0 tx=0x7f6bb000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:12.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:12.520+0000 7f6bbeffd640 1 -- 172.21.15.27:0/1401313545 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bb0015070 con 0x7f6bc006e960 2024-09-12T03:17:12.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:12.520+0000 7f6bbeffd640 1 -- 172.21.15.27:0/1401313545 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6bb0004240 con 0x7f6bc006e960 2024-09-12T03:17:12.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:12.520+0000 7f6bc84b7640 1 -- 172.21.15.27:0/1401313545 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6bc00cd8e0 con 0x7f6bc006e960 2024-09-12T03:17:12.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:12.520+0000 7f6bbeffd640 1 -- 172.21.15.27:0/1401313545 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bb0007620 con 0x7f6bc006e960 2024-09-12T03:17:12.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:12.520+0000 7f6bc84b7640 1 -- 172.21.15.27:0/1401313545 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6bc00cdea0 con 0x7f6bc006e960 2024-09-12T03:17:12.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:12.521+0000 7f6bc84b7640 1 -- 172.21.15.27:0/1401313545 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6bc006c660 con 0x7f6bc006e960 2024-09-12T03:17:12.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:12.522+0000 7f6bbeffd640 1 -- 172.21.15.27:0/1401313545 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6bb00079d0 con 0x7f6bc006e960 2024-09-12T03:17:12.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:12.522+0000 7f6bbeffd640 1 --2- 172.21.15.27:0/1401313545 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f6bac06be60 0x7f6bac06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:12.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:12.522+0000 7f6bbeffd640 1 -- 172.21.15.27:0/1401313545 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5709+0+0 (secure 0 0 0) 0x7f6bb008ae00 con 0x7f6bc006e960 2024-09-12T03:17:12.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:12.523+0000 7f6bc5a52640 1 --2- 172.21.15.27:0/1401313545 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f6bac06be60 0x7f6bac06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:12.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:12.523+0000 7f6bc5a52640 1 --2- 172.21.15.27:0/1401313545 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f6bac06be60 0x7f6bac06e320 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f6bb4033000 tx=0x7f6bb4009000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:12.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:12.527+0000 7f6bbeffd640 1 -- 172.21.15.27:0/1401313545 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f6bc006c660 con 0x7f6bc006e960 2024-09-12T03:17:12.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:12.744+0000 7f6bc84b7640 1 -- 172.21.15.27:0/1401313545 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f6bc00d1230 con 0x7f6bc006e960 2024-09-12T03:17:13.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:13.573+0000 7f6bbeffd640 1 -- 172.21.15.27:0/1401313545 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v13) ==== 118+0+0 (secure 0 0 0) 0x7f6bb0059160 con 0x7f6bc006e960 2024-09-12T03:17:13.577 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:13.576+0000 7f6bc84b7640 1 -- 172.21.15.27:0/1401313545 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f6bac06be60 msgr2=0x7f6bac06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:13.577 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:13.576+0000 7f6bc84b7640 1 --2- 172.21.15.27:0/1401313545 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f6bac06be60 0x7f6bac06e320 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f6bb4033000 tx=0x7f6bb4009000 comp rx=0 tx=0).stop 2024-09-12T03:17:13.577 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:13.576+0000 7f6bc84b7640 1 -- 172.21.15.27:0/1401313545 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6bc006e960 msgr2=0x7f6bc015e9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:13.577 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:13.576+0000 7f6bc84b7640 1 --2- 172.21.15.27:0/1401313545 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6bc006e960 0x7f6bc015e9c0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f6bb000c7f0 tx=0x7f6bb000ccb0 comp rx=0 tx=0).stop 2024-09-12T03:17:13.577 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:13.577+0000 7f6bc84b7640 1 -- 172.21.15.27:0/1401313545 shutdown_connections 2024-09-12T03:17:13.577 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:13.577+0000 7f6bc84b7640 1 --2- 172.21.15.27:0/1401313545 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f6bac06be60 0x7f6bac06e320 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:13.577 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:13.577+0000 7f6bc84b7640 1 --2- 172.21.15.27:0/1401313545 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f6bc00d0980 0x7f6bc00b6ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:13.577 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:13.577+0000 7f6bc84b7640 1 --2- 172.21.15.27:0/1401313545 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6bc0070d30 0x7f6bc01f7b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:13.577 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:13.577+0000 7f6bc84b7640 1 --2- 172.21.15.27:0/1401313545 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6bc006e960 0x7f6bc015e9c0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:13.577 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:13.577+0000 7f6bc84b7640 1 -- 172.21.15.27:0/1401313545 >> 172.21.15.27:0/1401313545 conn(0x7f6bc00b7c10 msgr2=0x7f6bc00af6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:13.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:13.577+0000 7f6bc84b7640 1 -- 172.21.15.27:0/1401313545 shutdown_connections 2024-09-12T03:17:13.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:13.577+0000 7f6bc84b7640 1 -- 172.21.15.27:0/1401313545 wait complete. 2024-09-12T03:17:13.590 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-12T03:17:13.590 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-12T03:17:13.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:13.703+0000 7f72586d0640 1 Processor -- start 2024-09-12T03:17:13.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:13.703+0000 7f72586d0640 1 -- start start 2024-09-12T03:17:13.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:13.704+0000 7f72586d0640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f725015e650 0x7f72500d1d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:13.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:13.704+0000 7f72586d0640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f725006db30 0x7f725006df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:13.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:13.704+0000 7f72586d0640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f72500d2280 con 0x7f725006cc40 2024-09-12T03:17:13.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:13.704+0000 7f72586d0640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f725006d850 con 0x7f725015e650 2024-09-12T03:17:13.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:13.704+0000 7f72586d0640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f72500d1260 con 0x7f725006db30 2024-09-12T03:17:13.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:13.704+0000 7f725646c640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f725015e650 0x7f72500d1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:13.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:13.704+0000 7f725646c640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f725015e650 0x7f72500d1d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36912/0 (socket says 172.21.15.27:36912) 2024-09-12T03:17:13.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:13.704+0000 7f725646c640 1 -- 172.21.15.27:0/495486824 learned_addr learned my addr 172.21.15.27:0/495486824 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:17:13.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:13.704+0000 7f7256c6d640 1 --2- 172.21.15.27:0/495486824 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f725006db30 0x7f725006df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:13.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:13.704+0000 7f725646c640 1 -- 172.21.15.27:0/495486824 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f725006db30 msgr2=0x7f725006df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:13.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:13.704+0000 7f725646c640 1 --2- 172.21.15.27:0/495486824 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f725006db30 0x7f725006df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:13.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:13.704+0000 7f725646c640 1 -- 172.21.15.27:0/495486824 >> v1:172.21.15.45:6789/0 conn(0x7f725006cc40 legacy=0x7f725006d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:13.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:13.704+0000 7f725646c640 1 -- 172.21.15.27:0/495486824 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72500d1540 con 0x7f725015e650 2024-09-12T03:17:13.707 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:13.705+0000 7f725646c640 1 --2- 172.21.15.27:0/495486824 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f725015e650 0x7f72500d1d40 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f724c009870 tx=0x7f724c02a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19bb53a24bf2b085 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:13.707 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:13.705+0000 7f725546a640 1 -- 172.21.15.27:0/495486824 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f724c037070 con 0x7f725015e650 2024-09-12T03:17:13.707 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:13.705+0000 7f725546a640 1 -- 172.21.15.27:0/495486824 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f724c032440 con 0x7f725015e650 2024-09-12T03:17:13.707 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:13.705+0000 7f725546a640 1 -- 172.21.15.27:0/495486824 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f724c032810 con 0x7f725015e650 2024-09-12T03:17:13.707 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:13.706+0000 7f72586d0640 1 -- 172.21.15.27:0/495486824 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f725015e650 msgr2=0x7f72500d1d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:13.707 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:13.706+0000 7f72586d0640 1 --2- 172.21.15.27:0/495486824 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f725015e650 0x7f72500d1d40 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f724c009870 tx=0x7f724c02a1c0 comp rx=0 tx=0).stop 2024-09-12T03:17:13.708 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:13.706+0000 7f72586d0640 1 -- 172.21.15.27:0/495486824 shutdown_connections 2024-09-12T03:17:13.708 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:13.706+0000 7f72586d0640 1 --2- 172.21.15.27:0/495486824 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f725006db30 0x7f725006df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:13.708 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:13.706+0000 7f72586d0640 1 --2- 172.21.15.27:0/495486824 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f725015e650 0x7f72500d1d40 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:13.708 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:13.706+0000 7f72586d0640 1 -- 172.21.15.27:0/495486824 >> 172.21.15.27:0/495486824 conn(0x7f725006f320 msgr2=0x7f725006f700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:13.708 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:13.706+0000 7f72586d0640 1 -- 172.21.15.27:0/495486824 shutdown_connections 2024-09-12T03:17:13.708 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:13.706+0000 7f72586d0640 1 -- 172.21.15.27:0/495486824 wait complete. 2024-09-12T03:17:13.708 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:13.707+0000 7f72586d0640 1 Processor -- start 2024-09-12T03:17:13.708 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:13.707+0000 7f72586d0640 1 -- start start 2024-09-12T03:17:13.711 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:13.708+0000 7f72586d0640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f725006cc40 0x7f7250156510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:13.711 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:13.708+0000 7f72586d0640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f725006db30 0x7f7250154540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:13.711 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:13.708+0000 7f72586d0640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f725015e650 0x7f72500b1a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:13.711 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:13.708+0000 7f72586d0640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f725006e5f0 con 0x7f725006db30 2024-09-12T03:17:13.711 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:13.708+0000 7f72586d0640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f725006e470 con 0x7f725015e650 2024-09-12T03:17:13.712 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:13.708+0000 7f72586d0640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f72500b55b0 con 0x7f725006cc40 2024-09-12T03:17:13.712 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:13.708+0000 7f7255c6b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f725006db30 0x7f7250154540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:13.712 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:13.708+0000 7f7255c6b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f725006db30 0x7f7250154540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36918/0 (socket says 172.21.15.27:36918) 2024-09-12T03:17:13.712 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:13.708+0000 7f7255c6b640 1 -- 172.21.15.27:0/1580248401 learned_addr learned my addr 172.21.15.27:0/1580248401 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:17:13.712 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:13.708+0000 7f7256c6d640 1 --2- 172.21.15.27:0/1580248401 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f725015e650 0x7f72500b1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:13.712 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:13.708+0000 7f725646c640 1 --2- 172.21.15.27:0/1580248401 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f725006cc40 0x7f7250156510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:13.712 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:13.708+0000 7f7255c6b640 1 -- 172.21.15.27:0/1580248401 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f725006cc40 msgr2=0x7f7250156510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:13.712 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:13.708+0000 7f7255c6b640 1 --2- 172.21.15.27:0/1580248401 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f725006cc40 0x7f7250156510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:13.712 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:13.708+0000 7f7255c6b640 1 -- 172.21.15.27:0/1580248401 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f725015e650 msgr2=0x7f72500b1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:13.712 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:13.708+0000 7f7255c6b640 1 --2- 172.21.15.27:0/1580248401 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f725015e650 0x7f72500b1a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:13.712 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:13.708+0000 7f7255c6b640 1 -- 172.21.15.27:0/1580248401 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72500b1fc0 con 0x7f725006db30 2024-09-12T03:17:13.713 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:13.708+0000 7f7255c6b640 1 --2- 172.21.15.27:0/1580248401 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f725006db30 0x7f7250154540 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f724000c400 tx=0x7f724000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:13.713 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:13.708+0000 7f7246ffd640 1 -- 172.21.15.27:0/1580248401 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72400052d0 con 0x7f725006db30 2024-09-12T03:17:13.713 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:13.709+0000 7f7246ffd640 1 -- 172.21.15.27:0/1580248401 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7240005490 con 0x7f725006db30 2024-09-12T03:17:13.713 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:13.709+0000 7f72586d0640 1 -- 172.21.15.27:0/1580248401 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7250156ba0 con 0x7f725006db30 2024-09-12T03:17:13.713 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:13.709+0000 7f7246ffd640 1 -- 172.21.15.27:0/1580248401 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7240007260 con 0x7f725006db30 2024-09-12T03:17:13.713 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:13.709+0000 7f72586d0640 1 -- 172.21.15.27:0/1580248401 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7250154e20 con 0x7f725006db30 2024-09-12T03:17:13.713 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:13.710+0000 7f72586d0640 1 -- 172.21.15.27:0/1580248401 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f725006df30 con 0x7f725006db30 2024-09-12T03:17:13.713 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:13.710+0000 7f7246ffd640 1 -- 172.21.15.27:0/1580248401 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7240007400 con 0x7f725006db30 2024-09-12T03:17:13.713 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:13.711+0000 7f7246ffd640 1 --2- 172.21.15.27:0/1580248401 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f723406be30 0x7f723406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:13.713 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:13.711+0000 7f7246ffd640 1 -- 172.21.15.27:0/1580248401 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f724008a310 con 0x7f725006db30 2024-09-12T03:17:13.713 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:13.711+0000 7f725646c640 1 --2- 172.21.15.27:0/1580248401 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f723406be30 0x7f723406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:13.713 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:13.711+0000 7f725646c640 1 --2- 172.21.15.27:0/1580248401 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f723406be30 0x7f723406e2f0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f724c034000 tx=0x7f724c02e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:13.715 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:13.715+0000 7f7246ffd640 1 -- 172.21.15.27:0/1580248401 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f7240058d50 con 0x7f725006db30 2024-09-12T03:17:13.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:13.928+0000 7f72586d0640 1 -- 172.21.15.27:0/1580248401 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f72500b3200 con 0x7f725006db30 2024-09-12T03:17:14.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:14.778+0000 7f7246ffd640 1 -- 172.21.15.27:0/1580248401 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v14) ==== 122+0+0 (secure 0 0 0) 0x7f7240058650 con 0x7f725006db30 2024-09-12T03:17:14.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:14.781+0000 7f72586d0640 1 -- 172.21.15.27:0/1580248401 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f723406be30 msgr2=0x7f723406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:14.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:14.781+0000 7f72586d0640 1 --2- 172.21.15.27:0/1580248401 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f723406be30 0x7f723406e2f0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f724c034000 tx=0x7f724c02e000 comp rx=0 tx=0).stop 2024-09-12T03:17:14.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:14.781+0000 7f72586d0640 1 -- 172.21.15.27:0/1580248401 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f725006db30 msgr2=0x7f7250154540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:14.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:14.781+0000 7f72586d0640 1 --2- 172.21.15.27:0/1580248401 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f725006db30 0x7f7250154540 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f724000c400 tx=0x7f724000c8c0 comp rx=0 tx=0).stop 2024-09-12T03:17:14.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:14.781+0000 7f72586d0640 1 -- 172.21.15.27:0/1580248401 shutdown_connections 2024-09-12T03:17:14.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:14.781+0000 7f72586d0640 1 --2- 172.21.15.27:0/1580248401 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f723406be30 0x7f723406e2f0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:14.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:14.781+0000 7f72586d0640 1 --2- 172.21.15.27:0/1580248401 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f725015e650 0x7f72500b1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:14.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:14.781+0000 7f72586d0640 1 --2- 172.21.15.27:0/1580248401 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f725006db30 0x7f7250154540 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:14.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:14.781+0000 7f72586d0640 1 --2- 172.21.15.27:0/1580248401 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f725006cc40 0x7f7250156510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:14.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:14.781+0000 7f72586d0640 1 -- 172.21.15.27:0/1580248401 >> 172.21.15.27:0/1580248401 conn(0x7f725006f320 msgr2=0x7f72500b3a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:14.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:14.781+0000 7f72586d0640 1 -- 172.21.15.27:0/1580248401 shutdown_connections 2024-09-12T03:17:14.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:14.781+0000 7f72586d0640 1 -- 172.21.15.27:0/1580248401 wait complete. 2024-09-12T03:17:14.795 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-12T03:17:14.795 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-12T03:17:14.905 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:14.905+0000 7f97f5311640 1 Processor -- start 2024-09-12T03:17:14.905 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:14.905+0000 7f97f5311640 1 -- start start 2024-09-12T03:17:14.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:14.906+0000 7f97f5311640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f97f015e5f0 0x7f97f0073250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:14.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:14.906+0000 7f97f5311640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f97f0073790 0x7f97f0162320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:14.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:14.906+0000 7f97f5311640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f97f0073bb0 con 0x7f97f0162860 2024-09-12T03:17:14.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:14.906+0000 7f97f5311640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f97f00d0da0 con 0x7f97f015e5f0 2024-09-12T03:17:14.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:14.906+0000 7f97f5311640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f97f0163090 con 0x7f97f0073790 2024-09-12T03:17:14.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:14.906+0000 7f97eeffd640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f97f015e5f0 0x7f97f0073250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:14.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:14.906+0000 7f97eeffd640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f97f015e5f0 0x7f97f0073250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36920/0 (socket says 172.21.15.27:36920) 2024-09-12T03:17:14.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:14.906+0000 7f97eeffd640 1 -- 172.21.15.27:0/3087594373 learned_addr learned my addr 172.21.15.27:0/3087594373 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:17:14.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:14.906+0000 7f97ee7fc640 1 --2- 172.21.15.27:0/3087594373 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f97f0073790 0x7f97f0162320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:14.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:14.906+0000 7f97eeffd640 1 -- 172.21.15.27:0/3087594373 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f97f0073790 msgr2=0x7f97f0162320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:14.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:14.906+0000 7f97eeffd640 1 --2- 172.21.15.27:0/3087594373 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f97f0073790 0x7f97f0162320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:14.909 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:14.906+0000 7f97eeffd640 1 -- 172.21.15.27:0/3087594373 >> v1:172.21.15.45:6789/0 conn(0x7f97f0162860 legacy=0x7f97f00d1230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:14.909 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:14.906+0000 7f97eeffd640 1 -- 172.21.15.27:0/3087594373 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97f00d0470 con 0x7f97f015e5f0 2024-09-12T03:17:14.909 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:14.906+0000 7f97ee7fc640 1 --2- 172.21.15.27:0/3087594373 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f97f0073790 0x7f97f0162320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:17:14.909 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:14.907+0000 7f97eeffd640 1 --2- 172.21.15.27:0/3087594373 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f97f015e5f0 0x7f97f0073250 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f97e0009870 tx=0x7f97e002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc6069e8ea143a7b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:14.909 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:14.907+0000 7f97edffb640 1 -- 172.21.15.27:0/3087594373 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97e0037070 con 0x7f97f015e5f0 2024-09-12T03:17:14.909 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:14.907+0000 7f97edffb640 1 -- 172.21.15.27:0/3087594373 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97e0032440 con 0x7f97f015e5f0 2024-09-12T03:17:14.909 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:14.907+0000 7f97edffb640 1 -- 172.21.15.27:0/3087594373 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97e0032810 con 0x7f97f015e5f0 2024-09-12T03:17:14.909 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:14.907+0000 7f97f5311640 1 -- 172.21.15.27:0/3087594373 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f97f015e5f0 msgr2=0x7f97f0073250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:14.909 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:14.907+0000 7f97f5311640 1 --2- 172.21.15.27:0/3087594373 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f97f015e5f0 0x7f97f0073250 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f97e0009870 tx=0x7f97e002a210 comp rx=0 tx=0).stop 2024-09-12T03:17:14.909 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:14.908+0000 7f97f5311640 1 -- 172.21.15.27:0/3087594373 shutdown_connections 2024-09-12T03:17:14.909 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:14.908+0000 7f97f5311640 1 --2- 172.21.15.27:0/3087594373 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f97f0073790 0x7f97f0162320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:14.910 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:14.908+0000 7f97f5311640 1 --2- 172.21.15.27:0/3087594373 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f97f015e5f0 0x7f97f0073250 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:14.910 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:14.908+0000 7f97f5311640 1 -- 172.21.15.27:0/3087594373 >> 172.21.15.27:0/3087594373 conn(0x7f97f006f300 msgr2=0x7f97f006f710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:14.910 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:14.908+0000 7f97f5311640 1 -- 172.21.15.27:0/3087594373 shutdown_connections 2024-09-12T03:17:14.910 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:14.908+0000 7f97f5311640 1 -- 172.21.15.27:0/3087594373 wait complete. 2024-09-12T03:17:14.912 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:14.909+0000 7f97f5311640 1 Processor -- start 2024-09-12T03:17:14.912 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:14.909+0000 7f97f5311640 1 -- start start 2024-09-12T03:17:14.912 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:14.910+0000 7f97f5311640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f97f0073790 0x7f97f015a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:14.912 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:14.910+0000 7f97f5311640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f97f015e5f0 0x7f97f00cb9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:14.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:14.910+0000 7f97f5311640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f97f0162860 0x7f97f00cbee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:14.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:14.910+0000 7f97f5311640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f97f00b5620 con 0x7f97f0162860 2024-09-12T03:17:14.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:14.910+0000 7f97f5311640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f97f00d0a30 con 0x7f97f015e5f0 2024-09-12T03:17:14.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:14.910+0000 7f97f5311640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f97f0162cf0 con 0x7f97f0073790 2024-09-12T03:17:14.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:14.910+0000 7f97ef7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f97f0162860 0x7f97f00cbee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:14.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:14.910+0000 7f97ef7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f97f0162860 0x7f97f00cbee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36932/0 (socket says 172.21.15.27:36932) 2024-09-12T03:17:14.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:14.910+0000 7f97ef7fe640 1 -- 172.21.15.27:0/1469215482 learned_addr learned my addr 172.21.15.27:0/1469215482 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:17:14.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:14.910+0000 7f97ee7fc640 1 --2- 172.21.15.27:0/1469215482 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f97f015e5f0 0x7f97f00cb9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:14.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:14.910+0000 7f97eeffd640 1 --2- 172.21.15.27:0/1469215482 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f97f0073790 0x7f97f015a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:14.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:14.910+0000 7f97ef7fe640 1 -- 172.21.15.27:0/1469215482 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f97f0073790 msgr2=0x7f97f015a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:14.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:14.910+0000 7f97ef7fe640 1 --2- 172.21.15.27:0/1469215482 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f97f0073790 0x7f97f015a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:14.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:14.910+0000 7f97ef7fe640 1 -- 172.21.15.27:0/1469215482 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f97f015e5f0 msgr2=0x7f97f00cb9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:14.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:14.910+0000 7f97ef7fe640 1 --2- 172.21.15.27:0/1469215482 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f97f015e5f0 0x7f97f00cb9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:14.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:14.910+0000 7f97ef7fe640 1 -- 172.21.15.27:0/1469215482 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97f0159450 con 0x7f97f0162860 2024-09-12T03:17:14.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:14.910+0000 7f97ef7fe640 1 --2- 172.21.15.27:0/1469215482 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f97f0162860 0x7f97f00cbee0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f97e800b970 tx=0x7f97e800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:14.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:14.910+0000 7f97d77fe640 1 -- 172.21.15.27:0/1469215482 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97e8019070 con 0x7f97f0162860 2024-09-12T03:17:14.915 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:14.911+0000 7f97d77fe640 1 -- 172.21.15.27:0/1469215482 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97e800cb40 con 0x7f97f0162860 2024-09-12T03:17:14.915 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:14.911+0000 7f97f5311640 1 -- 172.21.15.27:0/1469215482 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97f0159730 con 0x7f97f0162860 2024-09-12T03:17:14.915 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:14.911+0000 7f97d77fe640 1 -- 172.21.15.27:0/1469215482 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97e8014440 con 0x7f97f0162860 2024-09-12T03:17:14.915 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:14.911+0000 7f97f5311640 1 -- 172.21.15.27:0/1469215482 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97f0159cf0 con 0x7f97f0162860 2024-09-12T03:17:14.915 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:14.912+0000 7f97f5311640 1 -- 172.21.15.27:0/1469215482 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97f0073520 con 0x7f97f0162860 2024-09-12T03:17:14.915 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:14.912+0000 7f97d77fe640 1 -- 172.21.15.27:0/1469215482 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f97e801c070 con 0x7f97f0162860 2024-09-12T03:17:14.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:14.913+0000 7f97d77fe640 1 --2- 172.21.15.27:0/1469215482 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f97dc06be30 0x7f97dc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:14.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:14.913+0000 7f97d77fe640 1 -- 172.21.15.27:0/1469215482 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f97e808aab0 con 0x7f97f0162860 2024-09-12T03:17:14.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:14.913+0000 7f97eeffd640 1 --2- 172.21.15.27:0/1469215482 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f97dc06be30 0x7f97dc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:14.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:14.913+0000 7f97eeffd640 1 --2- 172.21.15.27:0/1469215482 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f97dc06be30 0x7f97dc06e2f0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f97e0034000 tx=0x7f97e002e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:14.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:14.917+0000 7f97d77fe640 1 -- 172.21.15.27:0/1469215482 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f97e8058e10 con 0x7f97f0162860 2024-09-12T03:17:15.126 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.126+0000 7f97f5311640 1 -- 172.21.15.27:0/1469215482 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f97f0069790 con 0x7f97f0162860 2024-09-12T03:17:15.126 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.126+0000 7f97d77fe640 1 -- 172.21.15.27:0/1469215482 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) ==== 94+0+5222 (secure 0 0 0) 0x7f97e805d110 con 0x7f97f0162860 2024-09-12T03:17:15.127 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:17:15.127 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":14,"btime":"2024-09-12T03:17:14:774103+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4592,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6836/3382368386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":3382368386},{"type":"v1","addr":"172.21.15.45:6836","nonce":3382368386}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":4595,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/2127097700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6835","nonce":2127097700},{"type":"v1","addr":"172.21.15.45:6837","nonce":2127097700}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":4815,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6834/2623734554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":2623734554},{"type":"v1","addr":"172.21.15.27:6834","nonce":2623734554}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-12T03:17:11.119994+0000","modified":"2024-09-12T03:17:13.929574+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4818},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4818":{"gid":4818,"name":"a","rank":0,"incarnation":11,"state":"up:active","state_seq":3,"addr":"172.21.15.27:6835/3533575930","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":3533575930},{"type":"v1","addr":"172.21.15.27:6835","nonce":3533575930}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4818,"qdb_cluster":[4818]},"id":2}]} 2024-09-12T03:17:15.127 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 14 2024-09-12T03:17:15.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.128+0000 7f97f5311640 1 -- 172.21.15.27:0/1469215482 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f97dc06be30 msgr2=0x7f97dc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:15.129 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.128+0000 7f97f5311640 1 --2- 172.21.15.27:0/1469215482 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f97dc06be30 0x7f97dc06e2f0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f97e0034000 tx=0x7f97e002e000 comp rx=0 tx=0).stop 2024-09-12T03:17:15.129 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.128+0000 7f97f5311640 1 -- 172.21.15.27:0/1469215482 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f97f0162860 msgr2=0x7f97f00cbee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:15.129 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.128+0000 7f97f5311640 1 --2- 172.21.15.27:0/1469215482 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f97f0162860 0x7f97f00cbee0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f97e800b970 tx=0x7f97e800be30 comp rx=0 tx=0).stop 2024-09-12T03:17:15.129 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.128+0000 7f97f5311640 1 -- 172.21.15.27:0/1469215482 shutdown_connections 2024-09-12T03:17:15.129 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.128+0000 7f97f5311640 1 --2- 172.21.15.27:0/1469215482 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f97dc06be30 0x7f97dc06e2f0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:15.129 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.128+0000 7f97f5311640 1 --2- 172.21.15.27:0/1469215482 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f97f0162860 0x7f97f00cbee0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:15.129 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.128+0000 7f97f5311640 1 --2- 172.21.15.27:0/1469215482 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f97f015e5f0 0x7f97f00cb9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:15.129 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.128+0000 7f97f5311640 1 --2- 172.21.15.27:0/1469215482 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f97f0073790 0x7f97f015a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:15.129 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.128+0000 7f97f5311640 1 -- 172.21.15.27:0/1469215482 >> 172.21.15.27:0/1469215482 conn(0x7f97f006f300 msgr2=0x7f97f015f6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:15.129 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.128+0000 7f97f5311640 1 -- 172.21.15.27:0/1469215482 shutdown_connections 2024-09-12T03:17:15.129 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.128+0000 7f97f5311640 1 -- 172.21.15.27:0/1469215482 wait complete. 2024-09-12T03:17:15.140 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-12T03:17:15.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.242+0000 7f5a16b66640 1 Processor -- start 2024-09-12T03:17:15.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.242+0000 7f5a16b66640 1 -- start start 2024-09-12T03:17:15.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.243+0000 7f5a16b66640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f5a100d5200 0x7f5a100d1010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:15.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.243+0000 7f5a16b66640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f5a100d1550 0x7f5a100b8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:15.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.243+0000 7f5a16b66640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5a100d3760 con 0x7f5a1015eb90 2024-09-12T03:17:15.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.243+0000 7f5a16b66640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f5a100b7d20 con 0x7f5a100d5200 2024-09-12T03:17:15.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.243+0000 7f5a16b66640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f5a100d42e0 con 0x7f5a100d1550 2024-09-12T03:17:15.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.243+0000 7f5a15103640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f5a100d1550 0x7f5a100b8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:15.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.243+0000 7f5a15103640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f5a100d1550 0x7f5a100b8190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3301/0 says I am v2:172.21.15.27:39792/0 (socket says 172.21.15.27:39792) 2024-09-12T03:17:15.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.243+0000 7f5a15103640 1 -- 172.21.15.27:0/4021758879 learned_addr learned my addr 172.21.15.27:0/4021758879 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:17:15.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.243+0000 7f5a07fff640 1 --2- 172.21.15.27:0/4021758879 >> v2:172.21.15.45:3300/0 conn(0x7f5a100d5200 0x7f5a100d1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:15.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.244+0000 7f5a15103640 1 -- 172.21.15.27:0/4021758879 >> v2:172.21.15.45:3300/0 conn(0x7f5a100d5200 msgr2=0x7f5a100d1010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:15.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.244+0000 7f5a15103640 1 --2- 172.21.15.27:0/4021758879 >> v2:172.21.15.45:3300/0 conn(0x7f5a100d5200 0x7f5a100d1010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:15.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.244+0000 7f5a15103640 1 -- 172.21.15.27:0/4021758879 >> v1:172.21.15.45:6789/0 conn(0x7f5a1015eb90 legacy=0x7f5a100d2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:15.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.244+0000 7f5a15103640 1 -- 172.21.15.27:0/4021758879 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a100d3a40 con 0x7f5a100d1550 2024-09-12T03:17:15.248 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.244+0000 7f5a15103640 1 --2- 172.21.15.27:0/4021758879 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f5a100d1550 0x7f5a100b8190 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f5a08009e50 tx=0x7f5a0802a710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=48fa99bcd51d3771 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:15.248 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.245+0000 7f5a077fe640 1 -- 172.21.15.27:0/4021758879 <== mon.2 v2:172.21.15.45:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a08036070 con 0x7f5a100d1550 2024-09-12T03:17:15.248 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.245+0000 7f5a077fe640 1 -- 172.21.15.27:0/4021758879 <== mon.2 v2:172.21.15.45:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a08031600 con 0x7f5a100d1550 2024-09-12T03:17:15.249 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.245+0000 7f5a077fe640 1 -- 172.21.15.27:0/4021758879 <== mon.2 v2:172.21.15.45:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a080319d0 con 0x7f5a100d1550 2024-09-12T03:17:15.249 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.245+0000 7f5a16b66640 1 -- 172.21.15.27:0/4021758879 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f5a100d1550 msgr2=0x7f5a100b8190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:15.249 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.245+0000 7f5a16b66640 1 --2- 172.21.15.27:0/4021758879 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f5a100d1550 0x7f5a100b8190 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f5a08009e50 tx=0x7f5a0802a710 comp rx=0 tx=0).stop 2024-09-12T03:17:15.249 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.245+0000 7f5a16b66640 1 -- 172.21.15.27:0/4021758879 shutdown_connections 2024-09-12T03:17:15.249 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.245+0000 7f5a16b66640 1 --2- 172.21.15.27:0/4021758879 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f5a100d1550 0x7f5a100b8190 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:15.249 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.245+0000 7f5a16b66640 1 --2- 172.21.15.27:0/4021758879 >> v2:172.21.15.45:3300/0 conn(0x7f5a100d5200 0x7f5a100d1010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:15.249 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.245+0000 7f5a16b66640 1 -- 172.21.15.27:0/4021758879 >> 172.21.15.27:0/4021758879 conn(0x7f5a1006dce0 msgr2=0x7f5a1006e0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:15.249 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.245+0000 7f5a16b66640 1 -- 172.21.15.27:0/4021758879 shutdown_connections 2024-09-12T03:17:15.249 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.245+0000 7f5a16b66640 1 -- 172.21.15.27:0/4021758879 wait complete. 2024-09-12T03:17:15.249 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.246+0000 7f5a16b66640 1 Processor -- start 2024-09-12T03:17:15.249 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.246+0000 7f5a16b66640 1 -- start start 2024-09-12T03:17:15.249 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.247+0000 7f5a16b66640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f5a100d1550 0x7f5a1015c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:15.249 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.247+0000 7f5a16b66640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5a100d5200 0x7f5a100aefd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:15.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.247+0000 7f5a16b66640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5a1015eb90 0x7f5a101f4b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:15.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.247+0000 7f5a16b66640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f5a100d4000 con 0x7f5a1015eb90 2024-09-12T03:17:15.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.247+0000 7f5a16b66640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f5a100d3500 con 0x7f5a100d5200 2024-09-12T03:17:15.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.247+0000 7f5a16b66640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f5a100d19a0 con 0x7f5a100d1550 2024-09-12T03:17:15.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.247+0000 7f5a15103640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5a1015eb90 0x7f5a101f4b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:15.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.247+0000 7f5a15103640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5a1015eb90 0x7f5a101f4b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36938/0 (socket says 172.21.15.27:36938) 2024-09-12T03:17:15.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.247+0000 7f5a15103640 1 -- 172.21.15.27:0/1942164539 learned_addr learned my addr 172.21.15.27:0/1942164539 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:17:15.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.248+0000 7f5a14902640 1 --2- 172.21.15.27:0/1942164539 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f5a100d1550 0x7f5a1015c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:15.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.248+0000 7f5a07fff640 1 --2- 172.21.15.27:0/1942164539 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5a100d5200 0x7f5a100aefd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:15.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.248+0000 7f5a14902640 1 -- 172.21.15.27:0/1942164539 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5a100d5200 msgr2=0x7f5a100aefd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:15.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.248+0000 7f5a14902640 1 --2- 172.21.15.27:0/1942164539 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5a100d5200 0x7f5a100aefd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:15.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.248+0000 7f5a14902640 1 -- 172.21.15.27:0/1942164539 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5a1015eb90 msgr2=0x7f5a101f4b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:15.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.248+0000 7f5a14902640 1 --2- 172.21.15.27:0/1942164539 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5a1015eb90 0x7f5a101f4b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:15.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.248+0000 7f5a14902640 1 -- 172.21.15.27:0/1942164539 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a100af9b0 con 0x7f5a100d1550 2024-09-12T03:17:15.251 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.248+0000 7f5a15103640 1 --2- 172.21.15.27:0/1942164539 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5a1015eb90 0x7f5a101f4b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:17:15.251 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.248+0000 7f5a07fff640 1 --2- 172.21.15.27:0/1942164539 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5a100d5200 0x7f5a100aefd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:17:15.251 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.248+0000 7f5a14902640 1 --2- 172.21.15.27:0/1942164539 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f5a100d1550 0x7f5a1015c440 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f5a0c00c3f0 tx=0x7f5a0c00c8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:15.251 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.248+0000 7f5a057fa640 1 -- 172.21.15.27:0/1942164539 <== mon.2 v2:172.21.15.45:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a0c0052d0 con 0x7f5a100d1550 2024-09-12T03:17:15.251 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.248+0000 7f5a057fa640 1 -- 172.21.15.27:0/1942164539 <== mon.2 v2:172.21.15.45:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a0c005490 con 0x7f5a100d1550 2024-09-12T03:17:15.251 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.248+0000 7f5a16b66640 1 -- 172.21.15.27:0/1942164539 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a100ac610 con 0x7f5a100d1550 2024-09-12T03:17:15.251 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.248+0000 7f5a057fa640 1 -- 172.21.15.27:0/1942164539 <== mon.2 v2:172.21.15.45:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a0c007260 con 0x7f5a100d1550 2024-09-12T03:17:15.251 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.248+0000 7f5a16b66640 1 -- 172.21.15.27:0/1942164539 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a100acbd0 con 0x7f5a100d1550 2024-09-12T03:17:15.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.249+0000 7f5a16b66640 1 -- 172.21.15.27:0/1942164539 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a1015f850 con 0x7f5a100d1550 2024-09-12T03:17:15.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.250+0000 7f5a057fa640 1 -- 172.21.15.27:0/1942164539 <== mon.2 v2:172.21.15.45:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5a0c007400 con 0x7f5a100d1550 2024-09-12T03:17:15.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.251+0000 7f5a057fa640 1 --2- 172.21.15.27:0/1942164539 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f59fc06be30 0x7f59fc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:15.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.251+0000 7f5a057fa640 1 -- 172.21.15.27:0/1942164539 <== mon.2 v2:172.21.15.45:3301/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f5a0c08a4d0 con 0x7f5a100d1550 2024-09-12T03:17:15.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.251+0000 7f5a07fff640 1 --2- 172.21.15.27:0/1942164539 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f59fc06be30 0x7f59fc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:15.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.251+0000 7f5a07fff640 1 --2- 172.21.15.27:0/1942164539 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f59fc06be30 0x7f59fc06e2f0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f5a100cad10 tx=0x7f5a00008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:15.254 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.255+0000 7f5a057fa640 1 -- 172.21.15.27:0/1942164539 <== mon.2 v2:172.21.15.45:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f5a0c058910 con 0x7f5a100d1550 2024-09-12T03:17:15.430 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.430+0000 7f5a16b66640 1 -- 172.21.15.27:0/1942164539 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5a100cb640 con 0x7f5a100d1550 2024-09-12T03:17:15.432 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.431+0000 7f5a057fa640 1 -- 172.21.15.27:0/1942164539 <== mon.2 v2:172.21.15.45:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) ==== 74+0+16681 (secure 0 0 0) 0x7f5a0c05cc10 con 0x7f5a100d1550 2024-09-12T03:17:15.432 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:17:15.432 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":32,"fsid":"d2df504e-2647-4385-a5db-9fdf69f487a0","created":"2024-09-12T03:16:07.778937+0000","modified":"2024-09-12T03:17:12.768450+0000","last_up_change":"2024-09-12T03:16:19.314778+0000","last_in_change":"2024-09-12T03:16:09.893086+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T03:16:20.097646+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-09-12T03:17:09.018856+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-09-12T03:17:10.105881+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1851116c-1fb3-4dc9-9772-f90e2ef4c355","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6817","nonce":3002321488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6819","nonce":3002321488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6823","nonce":3002321488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6821","nonce":3002321488}]},"public_addr":"172.21.15.27:6817/3002321488","cluster_addr":"172.21.15.27:6819/3002321488","heartbeat_back_addr":"172.21.15.27:6823/3002321488","heartbeat_front_addr":"172.21.15.27:6821/3002321488","state":["exists","up"]},{"osd":1,"uuid":"f38ae17d-73eb-4204-b99d-975fa806fb87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6809","nonce":1613102886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6811","nonce":1613102886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6815","nonce":1613102886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6813","nonce":1613102886}]},"public_addr":"172.21.15.27:6809/1613102886","cluster_addr":"172.21.15.27:6811/1613102886","heartbeat_back_addr":"172.21.15.27:6815/1613102886","heartbeat_front_addr":"172.21.15.27:6813/1613102886","state":["exists","up"]},{"osd":2,"uuid":"fb12dcc8-bdd0-42b8-b46d-8b10106224bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6825","nonce":3534723355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6827","nonce":3534723355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6831","nonce":3534723355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6829","nonce":3534723355}]},"public_addr":"172.21.15.27:6825/3534723355","cluster_addr":"172.21.15.27:6827/3534723355","heartbeat_back_addr":"172.21.15.27:6831/3534723355","heartbeat_front_addr":"172.21.15.27:6829/3534723355","state":["exists","up"]},{"osd":3,"uuid":"d74d3f49-9c24-4b9f-98e2-3ecd7b5ef3b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6801","nonce":80661598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6803","nonce":80661598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6807","nonce":80661598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6805","nonce":80661598}]},"public_addr":"172.21.15.27:6801/80661598","cluster_addr":"172.21.15.27:6803/80661598","heartbeat_back_addr":"172.21.15.27:6807/80661598","heartbeat_front_addr":"172.21.15.27:6805/80661598","state":["exists","up"]},{"osd":4,"uuid":"2bf123f2-1b0d-41c4-aa3b-805bab267e59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6817","nonce":3943189827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6819","nonce":3943189827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6823","nonce":3943189827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6821","nonce":3943189827}]},"public_addr":"172.21.15.45:6817/3943189827","cluster_addr":"172.21.15.45:6819/3943189827","heartbeat_back_addr":"172.21.15.45:6823/3943189827","heartbeat_front_addr":"172.21.15.45:6821/3943189827","state":["exists","up"]},{"osd":5,"uuid":"c49ba5da-91c6-4a86-a97b-4aba89c253ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6809","nonce":3620760330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6811","nonce":3620760330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6815","nonce":3620760330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6813","nonce":3620760330}]},"public_addr":"172.21.15.45:6809/3620760330","cluster_addr":"172.21.15.45:6811/3620760330","heartbeat_back_addr":"172.21.15.45:6815/3620760330","heartbeat_front_addr":"172.21.15.45:6813/3620760330","state":["exists","up"]},{"osd":6,"uuid":"b0123475-89f7-4a34-9e33-d797b99dd9dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6801","nonce":1989083510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6803","nonce":1989083510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6807","nonce":1989083510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6805","nonce":1989083510}]},"public_addr":"172.21.15.45:6801/1989083510","cluster_addr":"172.21.15.45:6803/1989083510","heartbeat_back_addr":"172.21.15.45:6807/1989083510","heartbeat_front_addr":"172.21.15.45:6805/1989083510","state":["exists","up"]},{"osd":7,"uuid":"4dad773d-b7d8-46fc-8a10-a972aad27b29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6825","nonce":1513300294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6827","nonce":1513300294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6831","nonce":1513300294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6829","nonce":1513300294}]},"public_addr":"172.21.15.45:6825/1513300294","cluster_addr":"172.21.15.45:6827/1513300294","heartbeat_back_addr":"172.21.15.45:6831/1513300294","heartbeat_front_addr":"172.21.15.45:6829/1513300294","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.881855+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:17.133252+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.735339+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.500742+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.477571+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:17.167667+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.470291+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.873743+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T03:17:15.434 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.433+0000 7f5a16b66640 1 -- 172.21.15.27:0/1942164539 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f59fc06be30 msgr2=0x7f59fc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:15.434 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.433+0000 7f5a16b66640 1 --2- 172.21.15.27:0/1942164539 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f59fc06be30 0x7f59fc06e2f0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f5a100cad10 tx=0x7f5a00008040 comp rx=0 tx=0).stop 2024-09-12T03:17:15.434 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.433+0000 7f5a16b66640 1 -- 172.21.15.27:0/1942164539 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f5a100d1550 msgr2=0x7f5a1015c440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:15.434 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.433+0000 7f5a16b66640 1 --2- 172.21.15.27:0/1942164539 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f5a100d1550 0x7f5a1015c440 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f5a0c00c3f0 tx=0x7f5a0c00c8b0 comp rx=0 tx=0).stop 2024-09-12T03:17:15.434 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.434+0000 7f5a16b66640 1 -- 172.21.15.27:0/1942164539 shutdown_connections 2024-09-12T03:17:15.434 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.434+0000 7f5a16b66640 1 --2- 172.21.15.27:0/1942164539 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f59fc06be30 0x7f59fc06e2f0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:15.434 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.434+0000 7f5a16b66640 1 --2- 172.21.15.27:0/1942164539 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5a1015eb90 0x7f5a101f4b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:15.434 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.434+0000 7f5a16b66640 1 --2- 172.21.15.27:0/1942164539 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5a100d5200 0x7f5a100aefd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:15.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.434+0000 7f5a16b66640 1 --2- 172.21.15.27:0/1942164539 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f5a100d1550 0x7f5a1015c440 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:15.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.434+0000 7f5a16b66640 1 -- 172.21.15.27:0/1942164539 >> 172.21.15.27:0/1942164539 conn(0x7f5a1006dce0 msgr2=0x7f5a1006e0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:15.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.434+0000 7f5a16b66640 1 -- 172.21.15.27:0/1942164539 shutdown_connections 2024-09-12T03:17:15.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.434+0000 7f5a16b66640 1 -- 172.21.15.27:0/1942164539 wait complete. 2024-09-12T03:17:15.445 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-12T03:17:15.445 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-12T03:17:15.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.547+0000 7fa01bbfd640 1 Processor -- start 2024-09-12T03:17:15.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.547+0000 7fa01bbfd640 1 -- start start 2024-09-12T03:17:15.551 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.548+0000 7fa01bbfd640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa01415e5f0 0x7fa014073250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:15.551 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.548+0000 7fa01bbfd640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa0140d2240 con 0x7fa014162a30 2024-09-12T03:17:15.551 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.548+0000 7fa01bbfd640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa014163090 con 0x7fa014073790 2024-09-12T03:17:15.551 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.548+0000 7fa01bbfd640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fa014162e60 con 0x7fa01415e5f0 2024-09-12T03:17:15.552 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.548+0000 7fa01a19a640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fa014162a30 0x7fa01415e9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:42556/0 (socket says 172.21.15.27:42556) 2024-09-12T03:17:15.552 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.548+0000 7fa01a19a640 1 -- 172.21.15.27:0/1240399084 learned_addr learned my addr 172.21.15.27:0/1240399084 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:17:15.552 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.548+0000 7fa019999640 1 --2- 172.21.15.27:0/1240399084 >> v2:172.21.15.45:3300/0 conn(0x7fa01415e5f0 0x7fa014073250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:15.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.548+0000 7fa018997640 1 -- 172.21.15.27:0/1240399084 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 820740492 0 0) 0x7fa0140d2240 con 0x7fa014162a30 2024-09-12T03:17:15.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.548+0000 7fa018997640 1 -- 172.21.15.27:0/1240399084 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa000003520 con 0x7fa014162a30 2024-09-12T03:17:15.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.549+0000 7fa018997640 1 -- 172.21.15.27:0/1240399084 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4026010733 0 0) 0x7fa000003520 con 0x7fa014162a30 2024-09-12T03:17:15.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.549+0000 7fa018997640 1 -- 172.21.15.27:0/1240399084 >> v2:172.21.15.45:3300/0 conn(0x7fa01415e5f0 msgr2=0x7fa014073250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:15.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.549+0000 7fa018997640 1 --2- 172.21.15.27:0/1240399084 >> v2:172.21.15.45:3300/0 conn(0x7fa01415e5f0 0x7fa014073250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:15.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.549+0000 7fa018997640 1 -- 172.21.15.27:0/1240399084 >> v1:172.21.15.45:6789/0 conn(0x7fa014073790 legacy=0x7fa014162320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:15.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.549+0000 7fa018997640 1 -- 172.21.15.27:0/1240399084 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0140d0470 con 0x7fa014162a30 2024-09-12T03:17:15.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.549+0000 7fa018997640 1 -- 172.21.15.27:0/1240399084 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 847755213 0 0) 0x7fa00c0019f0 con 0x7fa014162a30 2024-09-12T03:17:15.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.549+0000 7fa018997640 1 -- 172.21.15.27:0/1240399084 >> v1:172.21.15.27:6789/0 conn(0x7fa014162a30 legacy=0x7fa01415e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:15.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.549+0000 7fa018997640 1 --2- 172.21.15.27:0/1240399084 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa0000039b0 0x7fa000003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:15.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.549+0000 7fa018997640 1 --2- 172.21.15.27:0/1240399084 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa000004800 0x7fa000004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:15.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.549+0000 7fa018997640 1 --2- 172.21.15.27:0/1240399084 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa0000051c0 0x7fa00000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:15.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.549+0000 7fa018997640 1 -- 172.21.15.27:0/1240399084 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fa00000c030 con 0x7fa0000039b0 2024-09-12T03:17:15.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.549+0000 7fa018997640 1 -- 172.21.15.27:0/1240399084 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa00000c1e0 con 0x7fa000004800 2024-09-12T03:17:15.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.549+0000 7fa018997640 1 -- 172.21.15.27:0/1240399084 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fa00000c3e0 con 0x7fa0000051c0 2024-09-12T03:17:15.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.549+0000 7fa019999640 1 --2- 172.21.15.27:0/1240399084 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa0000039b0 0x7fa000003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:15.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.549+0000 7fa019198640 1 --2- 172.21.15.27:0/1240399084 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa000004800 0x7fa000004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:15.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.549+0000 7fa01a19a640 1 --2- 172.21.15.27:0/1240399084 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa0000051c0 0x7fa00000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:15.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.549+0000 7fa019999640 1 -- 172.21.15.27:0/1240399084 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa0000051c0 msgr2=0x7fa00000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:15.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.549+0000 7fa019999640 1 --2- 172.21.15.27:0/1240399084 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa0000051c0 0x7fa00000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:15.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.549+0000 7fa019999640 1 -- 172.21.15.27:0/1240399084 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa000004800 msgr2=0x7fa000004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:15.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.549+0000 7fa019999640 1 --2- 172.21.15.27:0/1240399084 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa000004800 0x7fa000004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:15.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.549+0000 7fa019999640 1 -- 172.21.15.27:0/1240399084 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa000003520 con 0x7fa0000039b0 2024-09-12T03:17:15.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.549+0000 7fa01a19a640 1 --2- 172.21.15.27:0/1240399084 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa0000051c0 0x7fa00000b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:17:15.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.549+0000 7fa01a19a640 1 -- 172.21.15.27:0/1240399084 reap_dead start 2024-09-12T03:17:15.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.550+0000 7fa019999640 1 --2- 172.21.15.27:0/1240399084 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa0000039b0 0x7fa000003dd0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fa010009ac0 tx=0x7fa000005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1800c005d27aee02 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:15.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.550+0000 7fa018997640 1 -- 172.21.15.27:0/1240399084 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa010036070 con 0x7fa0000039b0 2024-09-12T03:17:15.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.550+0000 7fa018997640 1 -- 172.21.15.27:0/1240399084 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa010031400 con 0x7fa0000039b0 2024-09-12T03:17:15.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.550+0000 7fa018997640 1 -- 172.21.15.27:0/1240399084 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa010009690 con 0x7fa0000039b0 2024-09-12T03:17:15.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.550+0000 7fa01bbfd640 1 -- 172.21.15.27:0/1240399084 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa0000039b0 msgr2=0x7fa000003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:15.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.550+0000 7fa01bbfd640 1 --2- 172.21.15.27:0/1240399084 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa0000039b0 0x7fa000003dd0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fa010009ac0 tx=0x7fa000005830 comp rx=0 tx=0).stop 2024-09-12T03:17:15.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.551+0000 7fa01bbfd640 1 -- 172.21.15.27:0/1240399084 shutdown_connections 2024-09-12T03:17:15.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.551+0000 7fa01bbfd640 1 --2- 172.21.15.27:0/1240399084 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa0000039b0 0x7fa000003dd0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:15.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.551+0000 7fa01bbfd640 1 -- 172.21.15.27:0/1240399084 >> 172.21.15.27:0/1240399084 conn(0x7fa01406f300 msgr2=0x7fa01406f710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:15.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.551+0000 7fa01bbfd640 1 -- 172.21.15.27:0/1240399084 shutdown_connections 2024-09-12T03:17:15.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.551+0000 7fa01bbfd640 1 -- 172.21.15.27:0/1240399084 wait complete. 2024-09-12T03:17:15.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.552+0000 7fa01bbfd640 1 Processor -- start 2024-09-12T03:17:15.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.552+0000 7fa01bbfd640 1 -- start start 2024-09-12T03:17:15.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.552+0000 7fa01bbfd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa0000039b0 0x7fa01415d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:15.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.553+0000 7fa01bbfd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa01415fb50 0x7fa014152ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:15.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.553+0000 7fa01bbfd640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa01415a9d0 0x7fa01415ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:15.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.553+0000 7fa01bbfd640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fa0140d0da0 con 0x7fa0000039b0 2024-09-12T03:17:15.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.553+0000 7fa01bbfd640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa014073be0 con 0x7fa01415fb50 2024-09-12T03:17:15.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.553+0000 7fa01bbfd640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fa0140d0a30 con 0x7fa01415a9d0 2024-09-12T03:17:15.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.553+0000 7fa019999640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa0000039b0 0x7fa01415d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:15.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.553+0000 7fa019999640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa0000039b0 0x7fa01415d970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36960/0 (socket says 172.21.15.27:36960) 2024-09-12T03:17:15.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.553+0000 7fa019999640 1 -- 172.21.15.27:0/2061994190 learned_addr learned my addr 172.21.15.27:0/2061994190 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:17:15.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.553+0000 7fa01a19a640 1 --2- 172.21.15.27:0/2061994190 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa01415a9d0 0x7fa01415ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:15.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.553+0000 7fa019198640 1 --2- 172.21.15.27:0/2061994190 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa01415fb50 0x7fa014152ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:15.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.553+0000 7fa019999640 1 -- 172.21.15.27:0/2061994190 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa01415a9d0 msgr2=0x7fa01415ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:15.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.553+0000 7fa019999640 1 --2- 172.21.15.27:0/2061994190 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa01415a9d0 0x7fa01415ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:15.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.553+0000 7fa019999640 1 -- 172.21.15.27:0/2061994190 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa01415fb50 msgr2=0x7fa014152ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:15.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.553+0000 7fa019999640 1 --2- 172.21.15.27:0/2061994190 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa01415fb50 0x7fa014152ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:15.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.553+0000 7fa019999640 1 -- 172.21.15.27:0/2061994190 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa01415ffb0 con 0x7fa0000039b0 2024-09-12T03:17:15.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.553+0000 7fa019999640 1 --2- 172.21.15.27:0/2061994190 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa0000039b0 0x7fa01415d970 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fa01002abb0 tx=0x7fa010004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:15.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.553+0000 7fa00a7fc640 1 -- 172.21.15.27:0/2061994190 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa01003f020 con 0x7fa0000039b0 2024-09-12T03:17:15.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.553+0000 7fa00a7fc640 1 -- 172.21.15.27:0/2061994190 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa010037040 con 0x7fa0000039b0 2024-09-12T03:17:15.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.553+0000 7fa01bbfd640 1 -- 172.21.15.27:0/2061994190 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa014160250 con 0x7fa0000039b0 2024-09-12T03:17:15.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.553+0000 7fa00a7fc640 1 -- 172.21.15.27:0/2061994190 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa010036070 con 0x7fa0000039b0 2024-09-12T03:17:15.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.553+0000 7fa01bbfd640 1 -- 172.21.15.27:0/2061994190 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa014160810 con 0x7fa0000039b0 2024-09-12T03:17:15.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.555+0000 7fa01bbfd640 1 -- 172.21.15.27:0/2061994190 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0140b5620 con 0x7fa0000039b0 2024-09-12T03:17:15.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.555+0000 7fa00a7fc640 1 -- 172.21.15.27:0/2061994190 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa010042020 con 0x7fa0000039b0 2024-09-12T03:17:15.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.555+0000 7fa00a7fc640 1 --2- 172.21.15.27:0/2061994190 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f9ff806be60 0x7f9ff806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:15.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.555+0000 7fa00a7fc640 1 -- 172.21.15.27:0/2061994190 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7fa0100aad20 con 0x7fa0000039b0 2024-09-12T03:17:15.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.555+0000 7fa019198640 1 --2- 172.21.15.27:0/2061994190 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f9ff806be60 0x7f9ff806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:15.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.556+0000 7fa019198640 1 --2- 172.21.15.27:0/2061994190 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f9ff806be60 0x7f9ff806e320 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fa004000c00 tx=0x7fa004009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:15.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.559+0000 7fa00a7fc640 1 -- 172.21.15.27:0/2061994190 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fa010049030 con 0x7fa0000039b0 2024-09-12T03:17:15.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.731+0000 7fa01bbfd640 1 -- 172.21.15.27:0/2061994190 --> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa014153010 con 0x7f9ff806be60 2024-09-12T03:17:15.738 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.738+0000 7fa00a7fc640 1 -- 172.21.15.27:0/2061994190 <== mgr.4102 v2:172.21.15.45:6832/3923821219 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+239855 (secure 0 0 0) 0x7fa014153010 con 0x7f9ff806be60 2024-09-12T03:17:15.740 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:17:15.742 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2024-09-12T03:17:15.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.741+0000 7fa01bbfd640 1 -- 172.21.15.27:0/2061994190 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f9ff806be60 msgr2=0x7f9ff806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:15.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.741+0000 7fa01bbfd640 1 --2- 172.21.15.27:0/2061994190 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f9ff806be60 0x7f9ff806e320 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fa004000c00 tx=0x7fa004009040 comp rx=0 tx=0).stop 2024-09-12T03:17:15.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.741+0000 7fa01bbfd640 1 -- 172.21.15.27:0/2061994190 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa0000039b0 msgr2=0x7fa01415d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:15.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.741+0000 7fa01bbfd640 1 --2- 172.21.15.27:0/2061994190 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa0000039b0 0x7fa01415d970 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fa01002abb0 tx=0x7fa010004730 comp rx=0 tx=0).stop 2024-09-12T03:17:15.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.741+0000 7fa01bbfd640 1 -- 172.21.15.27:0/2061994190 shutdown_connections 2024-09-12T03:17:15.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.741+0000 7fa01bbfd640 1 --2- 172.21.15.27:0/2061994190 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f9ff806be60 0x7f9ff806e320 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:15.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.741+0000 7fa01bbfd640 1 --2- 172.21.15.27:0/2061994190 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa01415a9d0 0x7fa01415ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:15.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.741+0000 7fa01bbfd640 1 --2- 172.21.15.27:0/2061994190 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa01415fb50 0x7fa014152ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:15.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.741+0000 7fa01bbfd640 1 --2- 172.21.15.27:0/2061994190 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa0000039b0 0x7fa01415d970 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:15.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.741+0000 7fa01bbfd640 1 -- 172.21.15.27:0/2061994190 >> 172.21.15.27:0/2061994190 conn(0x7fa01406f300 msgr2=0x7fa0140b2290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:15.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.741+0000 7fa01bbfd640 1 -- 172.21.15.27:0/2061994190 shutdown_connections 2024-09-12T03:17:15.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.741+0000 7fa01bbfd640 1 -- 172.21.15.27:0/2061994190 wait complete. 2024-09-12T03:17:15.752 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":52,"stamp":"2024-09-12T03:17:14.061342+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":15,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236500,"kb_used_data":3620,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738220,"statfs":{"total":773094113280,"available":772851937280,"internally_reserved":0,"allocated":3706880,"data_stored":2107389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.310390"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:17:10.750733+0000","last_change":"2024-09-12T03:17:10.750733+0000","last_active":"2024-09-12T03:17:10.750733+0000","last_peered":"2024-09-12T03:17:10.750733+0000","last_clean":"2024-09-12T03:17:10.750733+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:17:10.750733+0000","last_undegraded":"2024-09-12T03:17:10.750733+0000","last_fullsized":"2024-09-12T03:17:10.750733+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_clean_scrub_stamp":"2024-09-12T03:17:10.750733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:17:10.750733+0000","last_change":"2024-09-12T03:17:10.750733+0000","last_active":"2024-09-12T03:17:10.750733+0000","last_peered":"2024-09-12T03:17:10.750733+0000","last_clean":"2024-09-12T03:17:10.750733+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:17:10.750733+0000","last_undegraded":"2024-09-12T03:17:10.750733+0000","last_fullsized":"2024-09-12T03:17:10.750733+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_clean_scrub_stamp":"2024-09-12T03:17:10.750733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:17:10.750733+0000","last_change":"2024-09-12T03:17:10.750733+0000","last_active":"2024-09-12T03:17:10.750733+0000","last_peered":"2024-09-12T03:17:10.750733+0000","last_clean":"2024-09-12T03:17:10.750733+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:17:10.750733+0000","last_undegraded":"2024-09-12T03:17:10.750733+0000","last_fullsized":"2024-09-12T03:17:10.750733+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_clean_scrub_stamp":"2024-09-12T03:17:10.750733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:17:10.750733+0000","last_change":"2024-09-12T03:17:10.750733+0000","last_active":"2024-09-12T03:17:10.750733+0000","last_peered":"2024-09-12T03:17:10.750733+0000","last_clean":"2024-09-12T03:17:10.750733+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:17:10.750733+0000","last_undegraded":"2024-09-12T03:17:10.750733+0000","last_fullsized":"2024-09-12T03:17:10.750733+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_clean_scrub_stamp":"2024-09-12T03:17:10.750733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:17:10.750733+0000","last_change":"2024-09-12T03:17:10.750733+0000","last_active":"2024-09-12T03:17:10.750733+0000","last_peered":"2024-09-12T03:17:10.750733+0000","last_clean":"2024-09-12T03:17:10.750733+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:17:10.750733+0000","last_undegraded":"2024-09-12T03:17:10.750733+0000","last_fullsized":"2024-09-12T03:17:10.750733+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_clean_scrub_stamp":"2024-09-12T03:17:10.750733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:17:10.750733+0000","last_change":"2024-09-12T03:17:10.750733+0000","last_active":"2024-09-12T03:17:10.750733+0000","last_peered":"2024-09-12T03:17:10.750733+0000","last_clean":"2024-09-12T03:17:10.750733+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:17:10.750733+0000","last_undegraded":"2024-09-12T03:17:10.750733+0000","last_fullsized":"2024-09-12T03:17:10.750733+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_clean_scrub_stamp":"2024-09-12T03:17:10.750733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:17:10.750733+0000","last_change":"2024-09-12T03:17:10.750733+0000","last_active":"2024-09-12T03:17:10.750733+0000","last_peered":"2024-09-12T03:17:10.750733+0000","last_clean":"2024-09-12T03:17:10.750733+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:17:10.750733+0000","last_undegraded":"2024-09-12T03:17:10.750733+0000","last_fullsized":"2024-09-12T03:17:10.750733+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_clean_scrub_stamp":"2024-09-12T03:17:10.750733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:17:10.750733+0000","last_change":"2024-09-12T03:17:10.750733+0000","last_active":"2024-09-12T03:17:10.750733+0000","last_peered":"2024-09-12T03:17:10.750733+0000","last_clean":"2024-09-12T03:17:10.750733+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:17:10.750733+0000","last_undegraded":"2024-09-12T03:17:10.750733+0000","last_fullsized":"2024-09-12T03:17:10.750733+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_clean_scrub_stamp":"2024-09-12T03:17:10.750733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:17:10.750733+0000","last_change":"2024-09-12T03:17:10.750733+0000","last_active":"2024-09-12T03:17:10.750733+0000","last_peered":"2024-09-12T03:17:10.750733+0000","last_clean":"2024-09-12T03:17:10.750733+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:17:10.750733+0000","last_undegraded":"2024-09-12T03:17:10.750733+0000","last_fullsized":"2024-09-12T03:17:10.750733+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_clean_scrub_stamp":"2024-09-12T03:17:10.750733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:17:10.750733+0000","last_change":"2024-09-12T03:17:10.750733+0000","last_active":"2024-09-12T03:17:10.750733+0000","last_peered":"2024-09-12T03:17:10.750733+0000","last_clean":"2024-09-12T03:17:10.750733+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:17:10.750733+0000","last_undegraded":"2024-09-12T03:17:10.750733+0000","last_fullsized":"2024-09-12T03:17:10.750733+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_clean_scrub_stamp":"2024-09-12T03:17:10.750733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:17:10.750733+0000","last_change":"2024-09-12T03:17:10.750733+0000","last_active":"2024-09-12T03:17:10.750733+0000","last_peered":"2024-09-12T03:17:10.750733+0000","last_clean":"2024-09-12T03:17:10.750733+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:17:10.750733+0000","last_undegraded":"2024-09-12T03:17:10.750733+0000","last_fullsized":"2024-09-12T03:17:10.750733+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_clean_scrub_stamp":"2024-09-12T03:17:10.750733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:17:10.750733+0000","last_change":"2024-09-12T03:17:10.750733+0000","last_active":"2024-09-12T03:17:10.750733+0000","last_peered":"2024-09-12T03:17:10.750733+0000","last_clean":"2024-09-12T03:17:10.750733+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:17:10.750733+0000","last_undegraded":"2024-09-12T03:17:10.750733+0000","last_fullsized":"2024-09-12T03:17:10.750733+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_clean_scrub_stamp":"2024-09-12T03:17:10.750733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:17:10.750733+0000","last_change":"2024-09-12T03:17:10.750733+0000","last_active":"2024-09-12T03:17:10.750733+0000","last_peered":"2024-09-12T03:17:10.750733+0000","last_clean":"2024-09-12T03:17:10.750733+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:17:10.750733+0000","last_undegraded":"2024-09-12T03:17:10.750733+0000","last_fullsized":"2024-09-12T03:17:10.750733+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_clean_scrub_stamp":"2024-09-12T03:17:10.750733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:17:10.750733+0000","last_change":"2024-09-12T03:17:10.750733+0000","last_active":"2024-09-12T03:17:10.750733+0000","last_peered":"2024-09-12T03:17:10.750733+0000","last_clean":"2024-09-12T03:17:10.750733+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:17:10.750733+0000","last_undegraded":"2024-09-12T03:17:10.750733+0000","last_fullsized":"2024-09-12T03:17:10.750733+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_clean_scrub_stamp":"2024-09-12T03:17:10.750733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:17:10.750733+0000","last_change":"2024-09-12T03:17:10.750733+0000","last_active":"2024-09-12T03:17:10.750733+0000","last_peered":"2024-09-12T03:17:10.750733+0000","last_clean":"2024-09-12T03:17:10.750733+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:17:10.750733+0000","last_undegraded":"2024-09-12T03:17:10.750733+0000","last_fullsized":"2024-09-12T03:17:10.750733+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_clean_scrub_stamp":"2024-09-12T03:17:10.750733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:17:10.750733+0000","last_change":"2024-09-12T03:17:10.750733+0000","last_active":"2024-09-12T03:17:10.750733+0000","last_peered":"2024-09-12T03:17:10.750733+0000","last_clean":"2024-09-12T03:17:10.750733+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:17:10.750733+0000","last_undegraded":"2024-09-12T03:17:10.750733+0000","last_fullsized":"2024-09-12T03:17:10.750733+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_clean_scrub_stamp":"2024-09-12T03:17:10.750733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:17:10.750733+0000","last_change":"2024-09-12T03:17:10.750733+0000","last_active":"2024-09-12T03:17:10.750733+0000","last_peered":"2024-09-12T03:17:10.750733+0000","last_clean":"2024-09-12T03:17:10.750733+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:17:10.750733+0000","last_undegraded":"2024-09-12T03:17:10.750733+0000","last_fullsized":"2024-09-12T03:17:10.750733+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_clean_scrub_stamp":"2024-09-12T03:17:10.750733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:17:10.750733+0000","last_change":"2024-09-12T03:17:10.750733+0000","last_active":"2024-09-12T03:17:10.750733+0000","last_peered":"2024-09-12T03:17:10.750733+0000","last_clean":"2024-09-12T03:17:10.750733+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:17:10.750733+0000","last_undegraded":"2024-09-12T03:17:10.750733+0000","last_fullsized":"2024-09-12T03:17:10.750733+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_clean_scrub_stamp":"2024-09-12T03:17:10.750733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:17:10.750733+0000","last_change":"2024-09-12T03:17:10.750733+0000","last_active":"2024-09-12T03:17:10.750733+0000","last_peered":"2024-09-12T03:17:10.750733+0000","last_clean":"2024-09-12T03:17:10.750733+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:17:10.750733+0000","last_undegraded":"2024-09-12T03:17:10.750733+0000","last_fullsized":"2024-09-12T03:17:10.750733+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_clean_scrub_stamp":"2024-09-12T03:17:10.750733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:17:10.750733+0000","last_change":"2024-09-12T03:17:10.750733+0000","last_active":"2024-09-12T03:17:10.750733+0000","last_peered":"2024-09-12T03:17:10.750733+0000","last_clean":"2024-09-12T03:17:10.750733+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:17:10.750733+0000","last_undegraded":"2024-09-12T03:17:10.750733+0000","last_fullsized":"2024-09-12T03:17:10.750733+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_clean_scrub_stamp":"2024-09-12T03:17:10.750733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:17:10.750733+0000","last_change":"2024-09-12T03:17:10.750733+0000","last_active":"2024-09-12T03:17:10.750733+0000","last_peered":"2024-09-12T03:17:10.750733+0000","last_clean":"2024-09-12T03:17:10.750733+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:17:10.750733+0000","last_undegraded":"2024-09-12T03:17:10.750733+0000","last_fullsized":"2024-09-12T03:17:10.750733+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_clean_scrub_stamp":"2024-09-12T03:17:10.750733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:17:10.750733+0000","last_change":"2024-09-12T03:17:10.750733+0000","last_active":"2024-09-12T03:17:10.750733+0000","last_peered":"2024-09-12T03:17:10.750733+0000","last_clean":"2024-09-12T03:17:10.750733+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:17:10.750733+0000","last_undegraded":"2024-09-12T03:17:10.750733+0000","last_fullsized":"2024-09-12T03:17:10.750733+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_clean_scrub_stamp":"2024-09-12T03:17:10.750733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:17:10.750733+0000","last_change":"2024-09-12T03:17:10.750733+0000","last_active":"2024-09-12T03:17:10.750733+0000","last_peered":"2024-09-12T03:17:10.750733+0000","last_clean":"2024-09-12T03:17:10.750733+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:17:10.750733+0000","last_undegraded":"2024-09-12T03:17:10.750733+0000","last_fullsized":"2024-09-12T03:17:10.750733+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_clean_scrub_stamp":"2024-09-12T03:17:10.750733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:17:10.750733+0000","last_change":"2024-09-12T03:17:10.750733+0000","last_active":"2024-09-12T03:17:10.750733+0000","last_peered":"2024-09-12T03:17:10.750733+0000","last_clean":"2024-09-12T03:17:10.750733+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:17:10.750733+0000","last_undegraded":"2024-09-12T03:17:10.750733+0000","last_fullsized":"2024-09-12T03:17:10.750733+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_clean_scrub_stamp":"2024-09-12T03:17:10.750733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:17:10.750733+0000","last_change":"2024-09-12T03:17:10.750733+0000","last_active":"2024-09-12T03:17:10.750733+0000","last_peered":"2024-09-12T03:17:10.750733+0000","last_clean":"2024-09-12T03:17:10.750733+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:17:10.750733+0000","last_undegraded":"2024-09-12T03:17:10.750733+0000","last_fullsized":"2024-09-12T03:17:10.750733+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_clean_scrub_stamp":"2024-09-12T03:17:10.750733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:17:10.750733+0000","last_change":"2024-09-12T03:17:10.750733+0000","last_active":"2024-09-12T03:17:10.750733+0000","last_peered":"2024-09-12T03:17:10.750733+0000","last_clean":"2024-09-12T03:17:10.750733+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:17:10.750733+0000","last_undegraded":"2024-09-12T03:17:10.750733+0000","last_fullsized":"2024-09-12T03:17:10.750733+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_clean_scrub_stamp":"2024-09-12T03:17:10.750733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:17:10.750733+0000","last_change":"2024-09-12T03:17:10.750733+0000","last_active":"2024-09-12T03:17:10.750733+0000","last_peered":"2024-09-12T03:17:10.750733+0000","last_clean":"2024-09-12T03:17:10.750733+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:17:10.750733+0000","last_undegraded":"2024-09-12T03:17:10.750733+0000","last_fullsized":"2024-09-12T03:17:10.750733+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_clean_scrub_stamp":"2024-09-12T03:17:10.750733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:17:10.750733+0000","last_change":"2024-09-12T03:17:10.750733+0000","last_active":"2024-09-12T03:17:10.750733+0000","last_peered":"2024-09-12T03:17:10.750733+0000","last_clean":"2024-09-12T03:17:10.750733+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:17:10.750733+0000","last_undegraded":"2024-09-12T03:17:10.750733+0000","last_fullsized":"2024-09-12T03:17:10.750733+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_clean_scrub_stamp":"2024-09-12T03:17:10.750733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:17:10.750733+0000","last_change":"2024-09-12T03:17:10.750733+0000","last_active":"2024-09-12T03:17:10.750733+0000","last_peered":"2024-09-12T03:17:10.750733+0000","last_clean":"2024-09-12T03:17:10.750733+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:17:10.750733+0000","last_undegraded":"2024-09-12T03:17:10.750733+0000","last_fullsized":"2024-09-12T03:17:10.750733+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_clean_scrub_stamp":"2024-09-12T03:17:10.750733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:17:10.750733+0000","last_change":"2024-09-12T03:17:10.750733+0000","last_active":"2024-09-12T03:17:10.750733+0000","last_peered":"2024-09-12T03:17:10.750733+0000","last_clean":"2024-09-12T03:17:10.750733+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:17:10.750733+0000","last_undegraded":"2024-09-12T03:17:10.750733+0000","last_fullsized":"2024-09-12T03:17:10.750733+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_clean_scrub_stamp":"2024-09-12T03:17:10.750733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:17:10.750733+0000","last_change":"2024-09-12T03:17:10.750733+0000","last_active":"2024-09-12T03:17:10.750733+0000","last_peered":"2024-09-12T03:17:10.750733+0000","last_clean":"2024-09-12T03:17:10.750733+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:17:10.750733+0000","last_undegraded":"2024-09-12T03:17:10.750733+0000","last_fullsized":"2024-09-12T03:17:10.750733+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_clean_scrub_stamp":"2024-09-12T03:17:10.750733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:17:10.750733+0000","last_change":"2024-09-12T03:17:10.750733+0000","last_active":"2024-09-12T03:17:10.750733+0000","last_peered":"2024-09-12T03:17:10.750733+0000","last_clean":"2024-09-12T03:17:10.750733+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:17:10.750733+0000","last_undegraded":"2024-09-12T03:17:10.750733+0000","last_fullsized":"2024-09-12T03:17:10.750733+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_clean_scrub_stamp":"2024-09-12T03:17:10.750733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:17:10.750733+0000","last_change":"2024-09-12T03:17:10.750733+0000","last_active":"2024-09-12T03:17:10.750733+0000","last_peered":"2024-09-12T03:17:10.750733+0000","last_clean":"2024-09-12T03:17:10.750733+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:17:10.750733+0000","last_undegraded":"2024-09-12T03:17:10.750733+0000","last_fullsized":"2024-09-12T03:17:10.750733+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_clean_scrub_stamp":"2024-09-12T03:17:10.750733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-09-12T03:17:09.756701+0000","last_change":"2024-09-12T03:17:09.756701+0000","last_active":"2024-09-12T03:17:09.744809+0000","last_peered":"2024-09-12T03:17:09.744809+0000","last_clean":"2024-09-12T03:17:09.744809+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:17:09.756701+0000","last_undegraded":"2024-09-12T03:17:09.756701+0000","last_fullsized":"2024-09-12T03:17:09.756701+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:09.744809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:09.744809+0000","last_clean_scrub_stamp":"2024-09-12T03:17:09.744809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:17:10.750733+0000","last_change":"2024-09-12T03:17:10.750733+0000","last_active":"2024-09-12T03:17:10.750733+0000","last_peered":"2024-09-12T03:17:10.750733+0000","last_clean":"2024-09-12T03:17:10.750733+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:17:10.750733+0000","last_undegraded":"2024-09-12T03:17:10.750733+0000","last_fullsized":"2024-09-12T03:17:10.750733+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_clean_scrub_stamp":"2024-09-12T03:17:10.750733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-09-12T03:17:09.758104+0000","last_change":"2024-09-12T03:17:09.758104+0000","last_active":"2024-09-12T03:17:09.744809+0000","last_peered":"2024-09-12T03:17:09.744809+0000","last_clean":"2024-09-12T03:17:09.744809+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:17:09.758104+0000","last_undegraded":"2024-09-12T03:17:09.758104+0000","last_fullsized":"2024-09-12T03:17:09.758104+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:09.744809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:09.744809+0000","last_clean_scrub_stamp":"2024-09-12T03:17:09.744809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:17:10.750733+0000","last_change":"2024-09-12T03:17:10.750733+0000","last_active":"2024-09-12T03:17:10.750733+0000","last_peered":"2024-09-12T03:17:10.750733+0000","last_clean":"2024-09-12T03:17:10.750733+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:17:10.750733+0000","last_undegraded":"2024-09-12T03:17:10.750733+0000","last_fullsized":"2024-09-12T03:17:10.750733+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_clean_scrub_stamp":"2024-09-12T03:17:10.750733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-09-12T03:17:09.758440+0000","last_change":"2024-09-12T03:17:09.758440+0000","last_active":"2024-09-12T03:17:09.744809+0000","last_peered":"2024-09-12T03:17:09.744809+0000","last_clean":"2024-09-12T03:17:09.744809+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:17:09.758440+0000","last_undegraded":"2024-09-12T03:17:09.758440+0000","last_fullsized":"2024-09-12T03:17:09.758440+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:09.744809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:09.744809+0000","last_clean_scrub_stamp":"2024-09-12T03:17:09.744809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:17:10.750733+0000","last_change":"2024-09-12T03:17:10.750733+0000","last_active":"2024-09-12T03:17:10.750733+0000","last_peered":"2024-09-12T03:17:10.750733+0000","last_clean":"2024-09-12T03:17:10.750733+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:17:10.750733+0000","last_undegraded":"2024-09-12T03:17:10.750733+0000","last_fullsized":"2024-09-12T03:17:10.750733+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_clean_scrub_stamp":"2024-09-12T03:17:10.750733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-09-12T03:17:09.756873+0000","last_change":"2024-09-12T03:17:09.756873+0000","last_active":"2024-09-12T03:17:09.744809+0000","last_peered":"2024-09-12T03:17:09.744809+0000","last_clean":"2024-09-12T03:17:09.744809+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:17:09.756873+0000","last_undegraded":"2024-09-12T03:17:09.756873+0000","last_fullsized":"2024-09-12T03:17:09.756873+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:09.744809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:09.744809+0000","last_clean_scrub_stamp":"2024-09-12T03:17:09.744809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:17:10.750733+0000","last_change":"2024-09-12T03:17:10.750733+0000","last_active":"2024-09-12T03:17:10.750733+0000","last_peered":"2024-09-12T03:17:10.750733+0000","last_clean":"2024-09-12T03:17:10.750733+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:17:10.750733+0000","last_undegraded":"2024-09-12T03:17:10.750733+0000","last_fullsized":"2024-09-12T03:17:10.750733+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_clean_scrub_stamp":"2024-09-12T03:17:10.750733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-09-12T03:17:09.753785+0000","last_change":"2024-09-12T03:17:09.753785+0000","last_active":"2024-09-12T03:17:09.744809+0000","last_peered":"2024-09-12T03:17:09.744809+0000","last_clean":"2024-09-12T03:17:09.744809+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:17:09.753785+0000","last_undegraded":"2024-09-12T03:17:09.753785+0000","last_fullsized":"2024-09-12T03:17:09.753785+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:09.744809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:09.744809+0000","last_clean_scrub_stamp":"2024-09-12T03:17:09.744809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:17:10.750733+0000","last_change":"2024-09-12T03:17:10.750733+0000","last_active":"2024-09-12T03:17:10.750733+0000","last_peered":"2024-09-12T03:17:10.750733+0000","last_clean":"2024-09-12T03:17:10.750733+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:17:10.750733+0000","last_undegraded":"2024-09-12T03:17:10.750733+0000","last_fullsized":"2024-09-12T03:17:10.750733+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_clean_scrub_stamp":"2024-09-12T03:17:10.750733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-09-12T03:17:09.753092+0000","last_change":"2024-09-12T03:17:09.753092+0000","last_active":"2024-09-12T03:17:09.744809+0000","last_peered":"2024-09-12T03:17:09.744809+0000","last_clean":"2024-09-12T03:17:09.744809+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:17:09.753092+0000","last_undegraded":"2024-09-12T03:17:09.753092+0000","last_fullsized":"2024-09-12T03:17:09.753092+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:09.744809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:09.744809+0000","last_clean_scrub_stamp":"2024-09-12T03:17:09.744809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:17:10.750733+0000","last_change":"2024-09-12T03:17:10.750733+0000","last_active":"2024-09-12T03:17:10.750733+0000","last_peered":"2024-09-12T03:17:10.750733+0000","last_clean":"2024-09-12T03:17:10.750733+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:17:10.750733+0000","last_undegraded":"2024-09-12T03:17:10.750733+0000","last_fullsized":"2024-09-12T03:17:10.750733+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_clean_scrub_stamp":"2024-09-12T03:17:10.750733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-09-12T03:17:09.758388+0000","last_change":"2024-09-12T03:17:09.758388+0000","last_active":"2024-09-12T03:17:09.744809+0000","last_peered":"2024-09-12T03:17:09.744809+0000","last_clean":"2024-09-12T03:17:09.744809+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:17:09.758388+0000","last_undegraded":"2024-09-12T03:17:09.758388+0000","last_fullsized":"2024-09-12T03:17:09.758388+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:09.744809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:09.744809+0000","last_clean_scrub_stamp":"2024-09-12T03:17:09.744809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:17:10.750733+0000","last_change":"2024-09-12T03:17:10.750733+0000","last_active":"2024-09-12T03:17:10.750733+0000","last_peered":"2024-09-12T03:17:10.750733+0000","last_clean":"2024-09-12T03:17:10.750733+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:17:10.750733+0000","last_undegraded":"2024-09-12T03:17:10.750733+0000","last_fullsized":"2024-09-12T03:17:10.750733+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_clean_scrub_stamp":"2024-09-12T03:17:10.750733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-09-12T03:17:09.753397+0000","last_change":"2024-09-12T03:17:09.753397+0000","last_active":"2024-09-12T03:17:09.744809+0000","last_peered":"2024-09-12T03:17:09.744809+0000","last_clean":"2024-09-12T03:17:09.744809+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:17:09.753397+0000","last_undegraded":"2024-09-12T03:17:09.753397+0000","last_fullsized":"2024-09-12T03:17:09.753397+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:09.744809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:09.744809+0000","last_clean_scrub_stamp":"2024-09-12T03:17:09.744809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:17:10.750733+0000","last_change":"2024-09-12T03:17:10.750733+0000","last_active":"2024-09-12T03:17:10.750733+0000","last_peered":"2024-09-12T03:17:10.750733+0000","last_clean":"2024-09-12T03:17:10.750733+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:17:10.750733+0000","last_undegraded":"2024-09-12T03:17:10.750733+0000","last_fullsized":"2024-09-12T03:17:10.750733+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_clean_scrub_stamp":"2024-09-12T03:17:10.750733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-09-12T03:17:09.758301+0000","last_change":"2024-09-12T03:17:09.758301+0000","last_active":"2024-09-12T03:17:09.744809+0000","last_peered":"2024-09-12T03:17:09.744809+0000","last_clean":"2024-09-12T03:17:09.744809+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:17:09.758301+0000","last_undegraded":"2024-09-12T03:17:09.758301+0000","last_fullsized":"2024-09-12T03:17:09.758301+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:09.744809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:09.744809+0000","last_clean_scrub_stamp":"2024-09-12T03:17:09.744809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:17:10.750733+0000","last_change":"2024-09-12T03:17:10.750733+0000","last_active":"2024-09-12T03:17:10.750733+0000","last_peered":"2024-09-12T03:17:10.750733+0000","last_clean":"2024-09-12T03:17:10.750733+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:17:10.750733+0000","last_undegraded":"2024-09-12T03:17:10.750733+0000","last_fullsized":"2024-09-12T03:17:10.750733+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_clean_scrub_stamp":"2024-09-12T03:17:10.750733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-09-12T03:17:09.757500+0000","last_change":"2024-09-12T03:17:09.757500+0000","last_active":"2024-09-12T03:17:09.744809+0000","last_peered":"2024-09-12T03:17:09.744809+0000","last_clean":"2024-09-12T03:17:09.744809+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:17:09.757500+0000","last_undegraded":"2024-09-12T03:17:09.757500+0000","last_fullsized":"2024-09-12T03:17:09.757500+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:09.744809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:09.744809+0000","last_clean_scrub_stamp":"2024-09-12T03:17:09.744809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:17:10.750733+0000","last_change":"2024-09-12T03:17:10.750733+0000","last_active":"2024-09-12T03:17:10.750733+0000","last_peered":"2024-09-12T03:17:10.750733+0000","last_clean":"2024-09-12T03:17:10.750733+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:17:10.750733+0000","last_undegraded":"2024-09-12T03:17:10.750733+0000","last_fullsized":"2024-09-12T03:17:10.750733+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_clean_scrub_stamp":"2024-09-12T03:17:10.750733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-09-12T03:17:09.753513+0000","last_change":"2024-09-12T03:17:09.753513+0000","last_active":"2024-09-12T03:17:09.744809+0000","last_peered":"2024-09-12T03:17:09.744809+0000","last_clean":"2024-09-12T03:17:09.744809+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:17:09.753513+0000","last_undegraded":"2024-09-12T03:17:09.753513+0000","last_fullsized":"2024-09-12T03:17:09.753513+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:09.744809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:09.744809+0000","last_clean_scrub_stamp":"2024-09-12T03:17:09.744809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:17:10.750733+0000","last_change":"2024-09-12T03:17:10.750733+0000","last_active":"2024-09-12T03:17:10.750733+0000","last_peered":"2024-09-12T03:17:10.750733+0000","last_clean":"2024-09-12T03:17:10.750733+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:17:10.750733+0000","last_undegraded":"2024-09-12T03:17:10.750733+0000","last_fullsized":"2024-09-12T03:17:10.750733+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_clean_scrub_stamp":"2024-09-12T03:17:10.750733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-09-12T03:17:09.757875+0000","last_change":"2024-09-12T03:17:09.757875+0000","last_active":"2024-09-12T03:17:09.744809+0000","last_peered":"2024-09-12T03:17:09.744809+0000","last_clean":"2024-09-12T03:17:09.744809+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:17:09.757875+0000","last_undegraded":"2024-09-12T03:17:09.757875+0000","last_fullsized":"2024-09-12T03:17:09.757875+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:09.744809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:09.744809+0000","last_clean_scrub_stamp":"2024-09-12T03:17:09.744809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:17:10.750733+0000","last_change":"2024-09-12T03:17:10.750733+0000","last_active":"2024-09-12T03:17:10.750733+0000","last_peered":"2024-09-12T03:17:10.750733+0000","last_clean":"2024-09-12T03:17:10.750733+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:17:10.750733+0000","last_undegraded":"2024-09-12T03:17:10.750733+0000","last_fullsized":"2024-09-12T03:17:10.750733+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_clean_scrub_stamp":"2024-09-12T03:17:10.750733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-09-12T03:17:09.752946+0000","last_change":"2024-09-12T03:17:09.752946+0000","last_active":"2024-09-12T03:17:09.744809+0000","last_peered":"2024-09-12T03:17:09.744809+0000","last_clean":"2024-09-12T03:17:09.744809+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:17:09.752946+0000","last_undegraded":"2024-09-12T03:17:09.752946+0000","last_fullsized":"2024-09-12T03:17:09.752946+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:09.744809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:09.744809+0000","last_clean_scrub_stamp":"2024-09-12T03:17:09.744809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:17:10.750733+0000","last_change":"2024-09-12T03:17:10.750733+0000","last_active":"2024-09-12T03:17:10.750733+0000","last_peered":"2024-09-12T03:17:10.750733+0000","last_clean":"2024-09-12T03:17:10.750733+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:17:10.750733+0000","last_undegraded":"2024-09-12T03:17:10.750733+0000","last_fullsized":"2024-09-12T03:17:10.750733+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_clean_scrub_stamp":"2024-09-12T03:17:10.750733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-09-12T03:17:09.760241+0000","last_change":"2024-09-12T03:17:09.760241+0000","last_active":"2024-09-12T03:17:09.744809+0000","last_peered":"2024-09-12T03:17:09.744809+0000","last_clean":"2024-09-12T03:17:09.744809+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:17:09.760241+0000","last_undegraded":"2024-09-12T03:17:09.760241+0000","last_fullsized":"2024-09-12T03:17:09.760241+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:09.744809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:09.744809+0000","last_clean_scrub_stamp":"2024-09-12T03:17:09.744809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:17:10.750733+0000","last_change":"2024-09-12T03:17:10.750733+0000","last_active":"2024-09-12T03:17:10.750733+0000","last_peered":"2024-09-12T03:17:10.750733+0000","last_clean":"2024-09-12T03:17:10.750733+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:17:10.750733+0000","last_undegraded":"2024-09-12T03:17:10.750733+0000","last_fullsized":"2024-09-12T03:17:10.750733+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_clean_scrub_stamp":"2024-09-12T03:17:10.750733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-09-12T03:17:09.757189+0000","last_change":"2024-09-12T03:17:09.757189+0000","last_active":"2024-09-12T03:17:09.744809+0000","last_peered":"2024-09-12T03:17:09.744809+0000","last_clean":"2024-09-12T03:17:09.744809+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:17:09.757189+0000","last_undegraded":"2024-09-12T03:17:09.757189+0000","last_fullsized":"2024-09-12T03:17:09.757189+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:09.744809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:09.744809+0000","last_clean_scrub_stamp":"2024-09-12T03:17:09.744809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:17:10.750733+0000","last_change":"2024-09-12T03:17:10.750733+0000","last_active":"2024-09-12T03:17:10.750733+0000","last_peered":"2024-09-12T03:17:10.750733+0000","last_clean":"2024-09-12T03:17:10.750733+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:17:10.750733+0000","last_undegraded":"2024-09-12T03:17:10.750733+0000","last_fullsized":"2024-09-12T03:17:10.750733+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_clean_scrub_stamp":"2024-09-12T03:17:10.750733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-09-12T03:17:09.756898+0000","last_change":"2024-09-12T03:17:09.756898+0000","last_active":"2024-09-12T03:17:09.744809+0000","last_peered":"2024-09-12T03:17:09.744809+0000","last_clean":"2024-09-12T03:17:09.744809+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:17:09.756898+0000","last_undegraded":"2024-09-12T03:17:09.756898+0000","last_fullsized":"2024-09-12T03:17:09.756898+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:09.744809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:09.744809+0000","last_clean_scrub_stamp":"2024-09-12T03:17:09.744809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:17:10.750733+0000","last_change":"2024-09-12T03:17:10.750733+0000","last_active":"2024-09-12T03:17:10.750733+0000","last_peered":"2024-09-12T03:17:10.750733+0000","last_clean":"2024-09-12T03:17:10.750733+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:17:10.750733+0000","last_undegraded":"2024-09-12T03:17:10.750733+0000","last_fullsized":"2024-09-12T03:17:10.750733+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_clean_scrub_stamp":"2024-09-12T03:17:10.750733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-09-12T03:17:09.760175+0000","last_change":"2024-09-12T03:17:09.760175+0000","last_active":"2024-09-12T03:17:09.744809+0000","last_peered":"2024-09-12T03:17:09.744809+0000","last_clean":"2024-09-12T03:17:09.744809+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:17:09.760175+0000","last_undegraded":"2024-09-12T03:17:09.760175+0000","last_fullsized":"2024-09-12T03:17:09.760175+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:09.744809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:09.744809+0000","last_clean_scrub_stamp":"2024-09-12T03:17:09.744809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:17:10.750733+0000","last_change":"2024-09-12T03:17:10.750733+0000","last_active":"2024-09-12T03:17:10.750733+0000","last_peered":"2024-09-12T03:17:10.750733+0000","last_clean":"2024-09-12T03:17:10.750733+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:17:10.750733+0000","last_undegraded":"2024-09-12T03:17:10.750733+0000","last_fullsized":"2024-09-12T03:17:10.750733+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_clean_scrub_stamp":"2024-09-12T03:17:10.750733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-09-12T03:17:09.753909+0000","last_change":"2024-09-12T03:17:09.753909+0000","last_active":"2024-09-12T03:17:09.744809+0000","last_peered":"2024-09-12T03:17:09.744809+0000","last_clean":"2024-09-12T03:17:09.744809+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:17:09.753909+0000","last_undegraded":"2024-09-12T03:17:09.753909+0000","last_fullsized":"2024-09-12T03:17:09.753909+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:09.744809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:09.744809+0000","last_clean_scrub_stamp":"2024-09-12T03:17:09.744809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:17:10.750733+0000","last_change":"2024-09-12T03:17:10.750733+0000","last_active":"2024-09-12T03:17:10.750733+0000","last_peered":"2024-09-12T03:17:10.750733+0000","last_clean":"2024-09-12T03:17:10.750733+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:17:10.750733+0000","last_undegraded":"2024-09-12T03:17:10.750733+0000","last_fullsized":"2024-09-12T03:17:10.750733+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_clean_scrub_stamp":"2024-09-12T03:17:10.750733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-09-12T03:17:09.754355+0000","last_change":"2024-09-12T03:17:09.754355+0000","last_active":"2024-09-12T03:17:09.744809+0000","last_peered":"2024-09-12T03:17:09.744809+0000","last_clean":"2024-09-12T03:17:09.744809+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:17:09.754355+0000","last_undegraded":"2024-09-12T03:17:09.754355+0000","last_fullsized":"2024-09-12T03:17:09.754355+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:09.744809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:09.744809+0000","last_clean_scrub_stamp":"2024-09-12T03:17:09.744809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:17:10.750733+0000","last_change":"2024-09-12T03:17:10.750733+0000","last_active":"2024-09-12T03:17:10.750733+0000","last_peered":"2024-09-12T03:17:10.750733+0000","last_clean":"2024-09-12T03:17:10.750733+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:17:10.750733+0000","last_undegraded":"2024-09-12T03:17:10.750733+0000","last_fullsized":"2024-09-12T03:17:10.750733+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_clean_scrub_stamp":"2024-09-12T03:17:10.750733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-09-12T03:17:09.753124+0000","last_change":"2024-09-12T03:17:09.753124+0000","last_active":"2024-09-12T03:17:09.744809+0000","last_peered":"2024-09-12T03:17:09.744809+0000","last_clean":"2024-09-12T03:17:09.744809+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:17:09.753124+0000","last_undegraded":"2024-09-12T03:17:09.753124+0000","last_fullsized":"2024-09-12T03:17:09.753124+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:09.744809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:09.744809+0000","last_clean_scrub_stamp":"2024-09-12T03:17:09.744809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:17:10.750733+0000","last_change":"2024-09-12T03:17:10.750733+0000","last_active":"2024-09-12T03:17:10.750733+0000","last_peered":"2024-09-12T03:17:10.750733+0000","last_clean":"2024-09-12T03:17:10.750733+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:17:10.750733+0000","last_undegraded":"2024-09-12T03:17:10.750733+0000","last_fullsized":"2024-09-12T03:17:10.750733+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_clean_scrub_stamp":"2024-09-12T03:17:10.750733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-09-12T03:17:09.757025+0000","last_change":"2024-09-12T03:17:09.757025+0000","last_active":"2024-09-12T03:17:09.744809+0000","last_peered":"2024-09-12T03:17:09.744809+0000","last_clean":"2024-09-12T03:17:09.744809+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:17:09.757025+0000","last_undegraded":"2024-09-12T03:17:09.757025+0000","last_fullsized":"2024-09-12T03:17:09.757025+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:09.744809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:09.744809+0000","last_clean_scrub_stamp":"2024-09-12T03:17:09.744809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:17:10.750733+0000","last_change":"2024-09-12T03:17:10.750733+0000","last_active":"2024-09-12T03:17:10.750733+0000","last_peered":"2024-09-12T03:17:10.750733+0000","last_clean":"2024-09-12T03:17:10.750733+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:17:10.750733+0000","last_undegraded":"2024-09-12T03:17:10.750733+0000","last_fullsized":"2024-09-12T03:17:10.750733+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_clean_scrub_stamp":"2024-09-12T03:17:10.750733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-09-12T03:17:09.756766+0000","last_change":"2024-09-12T03:17:09.756766+0000","last_active":"2024-09-12T03:17:09.744809+0000","last_peered":"2024-09-12T03:17:09.744809+0000","last_clean":"2024-09-12T03:17:09.744809+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:17:09.756766+0000","last_undegraded":"2024-09-12T03:17:09.756766+0000","last_fullsized":"2024-09-12T03:17:09.756766+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:09.744809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:09.744809+0000","last_clean_scrub_stamp":"2024-09-12T03:17:09.744809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:17:10.750733+0000","last_change":"2024-09-12T03:17:10.750733+0000","last_active":"2024-09-12T03:17:10.750733+0000","last_peered":"2024-09-12T03:17:10.750733+0000","last_clean":"2024-09-12T03:17:10.750733+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:17:10.750733+0000","last_undegraded":"2024-09-12T03:17:10.750733+0000","last_fullsized":"2024-09-12T03:17:10.750733+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_clean_scrub_stamp":"2024-09-12T03:17:10.750733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-09-12T03:17:09.753586+0000","last_change":"2024-09-12T03:17:09.753586+0000","last_active":"2024-09-12T03:17:09.744809+0000","last_peered":"2024-09-12T03:17:09.744809+0000","last_clean":"2024-09-12T03:17:09.744809+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:17:09.753586+0000","last_undegraded":"2024-09-12T03:17:09.753586+0000","last_fullsized":"2024-09-12T03:17:09.753586+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:09.744809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:09.744809+0000","last_clean_scrub_stamp":"2024-09-12T03:17:09.744809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:17:10.750733+0000","last_change":"2024-09-12T03:17:10.750733+0000","last_active":"2024-09-12T03:17:10.750733+0000","last_peered":"2024-09-12T03:17:10.750733+0000","last_clean":"2024-09-12T03:17:10.750733+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:17:10.750733+0000","last_undegraded":"2024-09-12T03:17:10.750733+0000","last_fullsized":"2024-09-12T03:17:10.750733+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_clean_scrub_stamp":"2024-09-12T03:17:10.750733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-09-12T03:17:09.752944+0000","last_change":"2024-09-12T03:17:09.752944+0000","last_active":"2024-09-12T03:17:09.744809+0000","last_peered":"2024-09-12T03:17:09.744809+0000","last_clean":"2024-09-12T03:17:09.744809+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:17:09.752944+0000","last_undegraded":"2024-09-12T03:17:09.752944+0000","last_fullsized":"2024-09-12T03:17:09.752944+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:09.744809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:09.744809+0000","last_clean_scrub_stamp":"2024-09-12T03:17:09.744809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:17:10.750733+0000","last_change":"2024-09-12T03:17:10.750733+0000","last_active":"2024-09-12T03:17:10.750733+0000","last_peered":"2024-09-12T03:17:10.750733+0000","last_clean":"2024-09-12T03:17:10.750733+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:17:10.750733+0000","last_undegraded":"2024-09-12T03:17:10.750733+0000","last_fullsized":"2024-09-12T03:17:10.750733+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_clean_scrub_stamp":"2024-09-12T03:17:10.750733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-09-12T03:17:09.759683+0000","last_change":"2024-09-12T03:17:09.759683+0000","last_active":"2024-09-12T03:17:09.744809+0000","last_peered":"2024-09-12T03:17:09.744809+0000","last_clean":"2024-09-12T03:17:09.744809+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:17:09.759683+0000","last_undegraded":"2024-09-12T03:17:09.759683+0000","last_fullsized":"2024-09-12T03:17:09.759683+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:09.744809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:09.744809+0000","last_clean_scrub_stamp":"2024-09-12T03:17:09.744809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:17:10.750733+0000","last_change":"2024-09-12T03:17:10.750733+0000","last_active":"2024-09-12T03:17:10.750733+0000","last_peered":"2024-09-12T03:17:10.750733+0000","last_clean":"2024-09-12T03:17:10.750733+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:17:10.750733+0000","last_undegraded":"2024-09-12T03:17:10.750733+0000","last_fullsized":"2024-09-12T03:17:10.750733+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_clean_scrub_stamp":"2024-09-12T03:17:10.750733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-09-12T03:17:09.756522+0000","last_change":"2024-09-12T03:17:09.756522+0000","last_active":"2024-09-12T03:17:09.744809+0000","last_peered":"2024-09-12T03:17:09.744809+0000","last_clean":"2024-09-12T03:17:09.744809+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:17:09.756522+0000","last_undegraded":"2024-09-12T03:17:09.756522+0000","last_fullsized":"2024-09-12T03:17:09.756522+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:09.744809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:09.744809+0000","last_clean_scrub_stamp":"2024-09-12T03:17:09.744809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:17:10.750733+0000","last_change":"2024-09-12T03:17:10.750733+0000","last_active":"2024-09-12T03:17:10.750733+0000","last_peered":"2024-09-12T03:17:10.750733+0000","last_clean":"2024-09-12T03:17:10.750733+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:17:10.750733+0000","last_undegraded":"2024-09-12T03:17:10.750733+0000","last_fullsized":"2024-09-12T03:17:10.750733+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_clean_scrub_stamp":"2024-09-12T03:17:10.750733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-09-12T03:17:09.759700+0000","last_change":"2024-09-12T03:17:09.759700+0000","last_active":"2024-09-12T03:17:09.744809+0000","last_peered":"2024-09-12T03:17:09.744809+0000","last_clean":"2024-09-12T03:17:09.744809+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:17:09.759700+0000","last_undegraded":"2024-09-12T03:17:09.759700+0000","last_fullsized":"2024-09-12T03:17:09.759700+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:09.744809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:09.744809+0000","last_clean_scrub_stamp":"2024-09-12T03:17:09.744809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:17:10.750733+0000","last_change":"2024-09-12T03:17:10.750733+0000","last_active":"2024-09-12T03:17:10.750733+0000","last_peered":"2024-09-12T03:17:10.750733+0000","last_clean":"2024-09-12T03:17:10.750733+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:17:10.750733+0000","last_undegraded":"2024-09-12T03:17:10.750733+0000","last_fullsized":"2024-09-12T03:17:10.750733+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_clean_scrub_stamp":"2024-09-12T03:17:10.750733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-09-12T03:17:09.756417+0000","last_change":"2024-09-12T03:17:09.756417+0000","last_active":"2024-09-12T03:17:09.744809+0000","last_peered":"2024-09-12T03:17:09.744809+0000","last_clean":"2024-09-12T03:17:09.744809+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:17:09.756417+0000","last_undegraded":"2024-09-12T03:17:09.756417+0000","last_fullsized":"2024-09-12T03:17:09.756417+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:09.744809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:09.744809+0000","last_clean_scrub_stamp":"2024-09-12T03:17:09.744809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:17:10.750733+0000","last_change":"2024-09-12T03:17:10.750733+0000","last_active":"2024-09-12T03:17:10.750733+0000","last_peered":"2024-09-12T03:17:10.750733+0000","last_clean":"2024-09-12T03:17:10.750733+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:17:10.750733+0000","last_undegraded":"2024-09-12T03:17:10.750733+0000","last_fullsized":"2024-09-12T03:17:10.750733+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_clean_scrub_stamp":"2024-09-12T03:17:10.750733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-09-12T03:17:09.756529+0000","last_change":"2024-09-12T03:17:09.756529+0000","last_active":"2024-09-12T03:17:09.744809+0000","last_peered":"2024-09-12T03:17:09.744809+0000","last_clean":"2024-09-12T03:17:09.744809+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:17:09.756529+0000","last_undegraded":"2024-09-12T03:17:09.756529+0000","last_fullsized":"2024-09-12T03:17:09.756529+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:09.744809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:09.744809+0000","last_clean_scrub_stamp":"2024-09-12T03:17:09.744809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:17:10.750733+0000","last_change":"2024-09-12T03:17:10.750733+0000","last_active":"2024-09-12T03:17:10.750733+0000","last_peered":"2024-09-12T03:17:10.750733+0000","last_clean":"2024-09-12T03:17:10.750733+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:17:10.750733+0000","last_undegraded":"2024-09-12T03:17:10.750733+0000","last_fullsized":"2024-09-12T03:17:10.750733+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_clean_scrub_stamp":"2024-09-12T03:17:10.750733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-09-12T03:17:09.752990+0000","last_change":"2024-09-12T03:17:09.752990+0000","last_active":"2024-09-12T03:17:09.744809+0000","last_peered":"2024-09-12T03:17:09.744809+0000","last_clean":"2024-09-12T03:17:09.744809+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:17:09.752990+0000","last_undegraded":"2024-09-12T03:17:09.752990+0000","last_fullsized":"2024-09-12T03:17:09.752990+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:09.744809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:09.744809+0000","last_clean_scrub_stamp":"2024-09-12T03:17:09.744809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:17:10.750733+0000","last_change":"2024-09-12T03:17:10.750733+0000","last_active":"2024-09-12T03:17:10.750733+0000","last_peered":"2024-09-12T03:17:10.750733+0000","last_clean":"2024-09-12T03:17:10.750733+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:17:10.750733+0000","last_undegraded":"2024-09-12T03:17:10.750733+0000","last_fullsized":"2024-09-12T03:17:10.750733+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_clean_scrub_stamp":"2024-09-12T03:17:10.750733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-09-12T03:17:09.756339+0000","last_change":"2024-09-12T03:17:09.756339+0000","last_active":"2024-09-12T03:17:09.744809+0000","last_peered":"2024-09-12T03:17:09.744809+0000","last_clean":"2024-09-12T03:17:09.744809+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:17:09.756339+0000","last_undegraded":"2024-09-12T03:17:09.756339+0000","last_fullsized":"2024-09-12T03:17:09.756339+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:09.744809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:09.744809+0000","last_clean_scrub_stamp":"2024-09-12T03:17:09.744809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:17:10.750733+0000","last_change":"2024-09-12T03:17:10.750733+0000","last_active":"2024-09-12T03:17:10.750733+0000","last_peered":"2024-09-12T03:17:10.750733+0000","last_clean":"2024-09-12T03:17:10.750733+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:17:10.750733+0000","last_undegraded":"2024-09-12T03:17:10.750733+0000","last_fullsized":"2024-09-12T03:17:10.750733+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_clean_scrub_stamp":"2024-09-12T03:17:10.750733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-09-12T03:17:09.756711+0000","last_change":"2024-09-12T03:17:09.756711+0000","last_active":"2024-09-12T03:17:09.744809+0000","last_peered":"2024-09-12T03:17:09.744809+0000","last_clean":"2024-09-12T03:17:09.744809+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:17:09.756711+0000","last_undegraded":"2024-09-12T03:17:09.756711+0000","last_fullsized":"2024-09-12T03:17:09.756711+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:09.744809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:09.744809+0000","last_clean_scrub_stamp":"2024-09-12T03:17:09.744809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":182,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-12T03:17:09.755450+0000","last_change":"2024-09-12T03:16:21.883446+0000","last_active":"2024-09-12T03:17:09.755450+0000","last_peered":"2024-09-12T03:17:09.755450+0000","last_clean":"2024-09-12T03:17:09.755450+0000","last_became_active":"2024-09-12T03:16:21.882207+0000","last_became_peered":"2024-09-12T03:16:21.882207+0000","last_unstale":"2024-09-12T03:17:09.755450+0000","last_undegraded":"2024-09-12T03:17:09.755450+0000","last_fullsized":"2024-09-12T03:17:09.755450+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:20.316426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:20.316426+0000","last_clean_scrub_stamp":"2024-09-12T03:16:20.316426+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:12:15.424320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542156,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32080,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339760,"statfs":{"total":96636764160,"available":96603914240,"internally_reserved":0,"allocated":913408,"data_stored":708381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542158,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31488,"kb_used_data":300,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340352,"statfs":{"total":96636764160,"available":96604520448,"internally_reserved":0,"allocated":307200,"data_stored":112196,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542157,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31500,"kb_used_data":312,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340340,"statfs":{"total":96636764160,"available":96604508160,"internally_reserved":0,"allocated":319488,"data_stored":118013,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542156,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31500,"kb_used_data":312,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340340,"statfs":{"total":96636764160,"available":96604508160,"internally_reserved":0,"allocated":319488,"data_stored":118013,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542157,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27340,"kb_used_data":312,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344500,"statfs":{"total":96636764160,"available":96608768000,"internally_reserved":0,"allocated":319488,"data_stored":118013,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542157,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27340,"kb_used_data":300,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344500,"statfs":{"total":96636764160,"available":96608768000,"internally_reserved":0,"allocated":307200,"data_stored":112196,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542158,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27332,"kb_used_data":300,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344508,"statfs":{"total":96636764160,"available":96608776192,"internally_reserved":0,"allocated":307200,"data_stored":112196,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542158,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27920,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343920,"statfs":{"total":96636764160,"available":96608174080,"internally_reserved":0,"allocated":913408,"data_stored":708381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T03:17:15.756 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-12T03:17:15.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.894+0000 7f87d98af640 1 Processor -- start 2024-09-12T03:17:15.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.894+0000 7f87d98af640 1 -- start start 2024-09-12T03:17:15.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.895+0000 7f87d98af640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f87d415b2b0 0x7f87d40b3560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:15.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.895+0000 7f87d98af640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f87d4072d70 0x7f87d4072430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:15.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.895+0000 7f87d98af640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f87d40ce410 con 0x7f87d406e9b0 2024-09-12T03:17:15.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.895+0000 7f87d98af640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f87d40720a0 con 0x7f87d415b2b0 2024-09-12T03:17:15.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.895+0000 7f87d98af640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f87d4072970 con 0x7f87d4072d70 2024-09-12T03:17:15.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.895+0000 7f87d2ffd640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f87d415b2b0 0x7f87d40b3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:15.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.895+0000 7f87d37fe640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f87d406e9b0 0x7f87d40cebb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:42560/0 (socket says 172.21.15.27:42560) 2024-09-12T03:17:15.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.895+0000 7f87d37fe640 1 -- 172.21.15.27:0/1149329125 learned_addr learned my addr 172.21.15.27:0/1149329125 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:17:15.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.895+0000 7f87d27fc640 1 --2- 172.21.15.27:0/1149329125 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f87d4072d70 0x7f87d4072430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:15.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.895+0000 7f87d2ffd640 1 -- 172.21.15.27:0/1149329125 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f87d4072d70 msgr2=0x7f87d4072430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:15.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.895+0000 7f87d2ffd640 1 --2- 172.21.15.27:0/1149329125 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f87d4072d70 0x7f87d4072430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:15.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.895+0000 7f87d2ffd640 1 -- 172.21.15.27:0/1149329125 >> v1:172.21.15.27:6789/0 conn(0x7f87d406e9b0 legacy=0x7f87d40cebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:15.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.895+0000 7f87d1ffb640 1 -- 172.21.15.27:0/1149329125 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4027644987 0 0) 0x7f87cc001110 con 0x7f87d406e9b0 2024-09-12T03:17:15.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.895+0000 7f87d2ffd640 1 -- 172.21.15.27:0/1149329125 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87d40732c0 con 0x7f87d415b2b0 2024-09-12T03:17:15.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.896+0000 7f87d2ffd640 1 --2- 172.21.15.27:0/1149329125 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f87d415b2b0 0x7f87d40b3560 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f87c4009d90 tx=0x7f87c402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4119b6b6fa16369 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:15.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.896+0000 7f87d1ffb640 1 -- 172.21.15.27:0/1149329125 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87c4007740 con 0x7f87d415b2b0 2024-09-12T03:17:15.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.896+0000 7f87d1ffb640 1 -- 172.21.15.27:0/1149329125 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87c40078e0 con 0x7f87d415b2b0 2024-09-12T03:17:15.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.896+0000 7f87d1ffb640 1 -- 172.21.15.27:0/1149329125 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87c4007cb0 con 0x7f87d415b2b0 2024-09-12T03:17:15.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.897+0000 7f87d98af640 1 -- 172.21.15.27:0/1149329125 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f87d415b2b0 msgr2=0x7f87d40b3560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:15.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.897+0000 7f87d98af640 1 --2- 172.21.15.27:0/1149329125 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f87d415b2b0 0x7f87d40b3560 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f87c4009d90 tx=0x7f87c402a660 comp rx=0 tx=0).stop 2024-09-12T03:17:15.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.897+0000 7f87d98af640 1 -- 172.21.15.27:0/1149329125 shutdown_connections 2024-09-12T03:17:15.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.897+0000 7f87d98af640 1 --2- 172.21.15.27:0/1149329125 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f87d4072d70 0x7f87d4072430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:15.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.897+0000 7f87d98af640 1 --2- 172.21.15.27:0/1149329125 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f87d415b2b0 0x7f87d40b3560 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:15.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.897+0000 7f87d98af640 1 -- 172.21.15.27:0/1149329125 >> 172.21.15.27:0/1149329125 conn(0x7f87d4062c60 msgr2=0x7f87d406bb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:15.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.897+0000 7f87d98af640 1 -- 172.21.15.27:0/1149329125 shutdown_connections 2024-09-12T03:17:15.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.897+0000 7f87d98af640 1 -- 172.21.15.27:0/1149329125 wait complete. 2024-09-12T03:17:15.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.898+0000 7f87d98af640 1 Processor -- start 2024-09-12T03:17:15.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.898+0000 7f87d98af640 1 -- start start 2024-09-12T03:17:15.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.899+0000 7f87d98af640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f87d406e9b0 0x7f87d4141db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:15.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.899+0000 7f87d98af640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f87d4072d70 0x7f87d406d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:15.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.899+0000 7f87d98af640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f87d415b2b0 0x7f87d406dde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:15.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.899+0000 7f87d98af640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f87d40ce690 con 0x7f87d406e9b0 2024-09-12T03:17:15.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.899+0000 7f87d98af640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f87d4071dc0 con 0x7f87d415b2b0 2024-09-12T03:17:15.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.899+0000 7f87d98af640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f87d415b690 con 0x7f87d4072d70 2024-09-12T03:17:15.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.899+0000 7f87d2ffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f87d406e9b0 0x7f87d4141db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:15.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.899+0000 7f87d2ffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f87d406e9b0 0x7f87d4141db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36986/0 (socket says 172.21.15.27:36986) 2024-09-12T03:17:15.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.899+0000 7f87d2ffd640 1 -- 172.21.15.27:0/3204264807 learned_addr learned my addr 172.21.15.27:0/3204264807 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:17:15.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.899+0000 7f87d27fc640 1 --2- 172.21.15.27:0/3204264807 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f87d4072d70 0x7f87d406d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:15.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.899+0000 7f87d37fe640 1 --2- 172.21.15.27:0/3204264807 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f87d415b2b0 0x7f87d406dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:15.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.900+0000 7f87d2ffd640 1 -- 172.21.15.27:0/3204264807 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f87d4072d70 msgr2=0x7f87d406d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:15.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.900+0000 7f87d2ffd640 1 --2- 172.21.15.27:0/3204264807 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f87d4072d70 0x7f87d406d8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:15.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.900+0000 7f87d2ffd640 1 -- 172.21.15.27:0/3204264807 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f87d415b2b0 msgr2=0x7f87d406dde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:15.902 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.900+0000 7f87d2ffd640 1 --2- 172.21.15.27:0/3204264807 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f87d415b2b0 0x7f87d406dde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:15.902 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.900+0000 7f87d2ffd640 1 -- 172.21.15.27:0/3204264807 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87d41422f0 con 0x7f87d406e9b0 2024-09-12T03:17:15.902 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.900+0000 7f87d27fc640 1 --2- 172.21.15.27:0/3204264807 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f87d4072d70 0x7f87d406d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:17:15.902 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.900+0000 7f87d2ffd640 1 --2- 172.21.15.27:0/3204264807 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f87d406e9b0 0x7f87d4141db0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f87c4000f80 tx=0x7f87c4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:15.902 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.900+0000 7f87cbfff640 1 -- 172.21.15.27:0/3204264807 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87c403f070 con 0x7f87d406e9b0 2024-09-12T03:17:15.902 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.900+0000 7f87cbfff640 1 -- 172.21.15.27:0/3204264807 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87c4031c40 con 0x7f87d406e9b0 2024-09-12T03:17:15.902 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.900+0000 7f87d98af640 1 -- 172.21.15.27:0/3204264807 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87d41e5d00 con 0x7f87d406e9b0 2024-09-12T03:17:15.902 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.900+0000 7f87cbfff640 1 -- 172.21.15.27:0/3204264807 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87c4009450 con 0x7f87d406e9b0 2024-09-12T03:17:15.902 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.900+0000 7f87d98af640 1 -- 172.21.15.27:0/3204264807 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87d41e62c0 con 0x7f87d406e9b0 2024-09-12T03:17:15.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.901+0000 7f87d98af640 1 -- 172.21.15.27:0/3204264807 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87d4072430 con 0x7f87d406e9b0 2024-09-12T03:17:15.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.902+0000 7f87cbfff640 1 -- 172.21.15.27:0/3204264807 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f87c4003990 con 0x7f87d406e9b0 2024-09-12T03:17:15.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.902+0000 7f87cbfff640 1 --2- 172.21.15.27:0/3204264807 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f87b406be30 0x7f87b406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:15.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.902+0000 7f87cbfff640 1 -- 172.21.15.27:0/3204264807 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f87c40abfc0 con 0x7f87d406e9b0 2024-09-12T03:17:15.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.902+0000 7f87d27fc640 1 --2- 172.21.15.27:0/3204264807 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f87b406be30 0x7f87b406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:15.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.903+0000 7f87d27fc640 1 --2- 172.21.15.27:0/3204264807 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f87b406be30 0x7f87b406e2f0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f87d41f5b90 tx=0x7f87bc008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:15.906 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:15.906+0000 7f87cbfff640 1 -- 172.21.15.27:0/3204264807 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f87d41422f0 con 0x7f87d406e9b0 2024-09-12T03:17:16.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:16.080+0000 7f87d98af640 1 -- 172.21.15.27:0/3204264807 --> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f87d40cca90 con 0x7f87b406be30 2024-09-12T03:17:16.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:16.087+0000 7f87cbfff640 1 -- 172.21.15.27:0/3204264807 <== mgr.4102 v2:172.21.15.45:6832/3923821219 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251951 (secure 0 0 0) 0x7f87d40cca90 con 0x7f87b406be30 2024-09-12T03:17:16.089 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:17:16.090 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2024-09-12T03:17:16.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:16.090+0000 7f87d98af640 1 -- 172.21.15.27:0/3204264807 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f87b406be30 msgr2=0x7f87b406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:16.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:16.090+0000 7f87d98af640 1 --2- 172.21.15.27:0/3204264807 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f87b406be30 0x7f87b406e2f0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f87d41f5b90 tx=0x7f87bc008040 comp rx=0 tx=0).stop 2024-09-12T03:17:16.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:16.090+0000 7f87d98af640 1 -- 172.21.15.27:0/3204264807 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f87d406e9b0 msgr2=0x7f87d4141db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:16.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:16.090+0000 7f87d98af640 1 --2- 172.21.15.27:0/3204264807 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f87d406e9b0 0x7f87d4141db0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f87c4000f80 tx=0x7f87c4004a40 comp rx=0 tx=0).stop 2024-09-12T03:17:16.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:16.090+0000 7f87d98af640 1 -- 172.21.15.27:0/3204264807 shutdown_connections 2024-09-12T03:17:16.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:16.090+0000 7f87d98af640 1 --2- 172.21.15.27:0/3204264807 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f87b406be30 0x7f87b406e2f0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:16.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:16.090+0000 7f87d98af640 1 --2- 172.21.15.27:0/3204264807 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f87d415b2b0 0x7f87d406dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:16.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:16.090+0000 7f87d98af640 1 --2- 172.21.15.27:0/3204264807 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f87d4072d70 0x7f87d406d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:16.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:16.090+0000 7f87d98af640 1 --2- 172.21.15.27:0/3204264807 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f87d406e9b0 0x7f87d4141db0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:16.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:16.090+0000 7f87d98af640 1 -- 172.21.15.27:0/3204264807 >> 172.21.15.27:0/3204264807 conn(0x7f87d4062c60 msgr2=0x7f87d40b01d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:16.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:16.090+0000 7f87d98af640 1 -- 172.21.15.27:0/3204264807 shutdown_connections 2024-09-12T03:17:16.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:16.090+0000 7f87d98af640 1 -- 172.21.15.27:0/3204264807 wait complete. 2024-09-12T03:17:16.101 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":53,"stamp":"2024-09-12T03:17:16.061742+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":174,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236904,"kb_used_data":4008,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754737816,"statfs":{"total":773094113280,"available":772851523584,"internally_reserved":0,"allocated":4104192,"data_stored":2284396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001119"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T03:17:13.088858+0000","last_change":"2024-09-12T03:17:11.778285+0000","last_active":"2024-09-12T03:17:13.088858+0000","last_peered":"2024-09-12T03:17:13.088858+0000","last_clean":"2024-09-12T03:17:13.088858+0000","last_became_active":"2024-09-12T03:17:11.778008+0000","last_became_peered":"2024-09-12T03:17:11.778008+0000","last_unstale":"2024-09-12T03:17:13.088858+0000","last_undegraded":"2024-09-12T03:17:13.088858+0000","last_fullsized":"2024-09-12T03:17:13.088858+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_clean_scrub_stamp":"2024-09-12T03:17:10.750733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:50:49.293495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T03:17:13.089516+0000","last_change":"2024-09-12T03:17:11.777518+0000","last_active":"2024-09-12T03:17:13.089516+0000","last_peered":"2024-09-12T03:17:13.089516+0000","last_clean":"2024-09-12T03:17:13.089516+0000","last_became_active":"2024-09-12T03:17:11.777247+0000","last_became_peered":"2024-09-12T03:17:11.777247+0000","last_unstale":"2024-09-12T03:17:13.089516+0000","last_undegraded":"2024-09-12T03:17:13.089516+0000","last_fullsized":"2024-09-12T03:17:13.089516+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_clean_scrub_stamp":"2024-09-12T03:17:10.750733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:00:25.018692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T03:17:12.776522+0000","last_change":"2024-09-12T03:17:11.776795+0000","last_active":"2024-09-12T03:17:12.776522+0000","last_peered":"2024-09-12T03:17:12.776522+0000","last_clean":"2024-09-12T03:17:12.776522+0000","last_became_active":"2024-09-12T03:17:11.773524+0000","last_became_peered":"2024-09-12T03:17:11.773524+0000","last_unstale":"2024-09-12T03:17:12.776522+0000","last_undegraded":"2024-09-12T03:17:12.776522+0000","last_fullsized":"2024-09-12T03:17:12.776522+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_clean_scrub_stamp":"2024-09-12T03:17:10.750733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:09:58.180698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T03:17:13.089856+0000","last_change":"2024-09-12T03:17:11.780069+0000","last_active":"2024-09-12T03:17:13.089856+0000","last_peered":"2024-09-12T03:17:13.089856+0000","last_clean":"2024-09-12T03:17:13.089856+0000","last_became_active":"2024-09-12T03:17:11.779806+0000","last_became_peered":"2024-09-12T03:17:11.779806+0000","last_unstale":"2024-09-12T03:17:13.089856+0000","last_undegraded":"2024-09-12T03:17:13.089856+0000","last_fullsized":"2024-09-12T03:17:13.089856+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_clean_scrub_stamp":"2024-09-12T03:17:10.750733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:07:48.729423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T03:17:13.088550+0000","last_change":"2024-09-12T03:17:11.791197+0000","last_active":"2024-09-12T03:17:13.088550+0000","last_peered":"2024-09-12T03:17:13.088550+0000","last_clean":"2024-09-12T03:17:13.088550+0000","last_became_active":"2024-09-12T03:17:11.790841+0000","last_became_peered":"2024-09-12T03:17:11.790841+0000","last_unstale":"2024-09-12T03:17:13.088550+0000","last_undegraded":"2024-09-12T03:17:13.088550+0000","last_fullsized":"2024-09-12T03:17:13.088550+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_clean_scrub_stamp":"2024-09-12T03:17:10.750733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:40:29.773278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T03:17:13.088587+0000","last_change":"2024-09-12T03:17:11.784380+0000","last_active":"2024-09-12T03:17:13.088587+0000","last_peered":"2024-09-12T03:17:13.088587+0000","last_clean":"2024-09-12T03:17:13.088587+0000","last_became_active":"2024-09-12T03:17:11.783855+0000","last_became_peered":"2024-09-12T03:17:11.783855+0000","last_unstale":"2024-09-12T03:17:13.088587+0000","last_undegraded":"2024-09-12T03:17:13.088587+0000","last_fullsized":"2024-09-12T03:17:13.088587+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_clean_scrub_stamp":"2024-09-12T03:17:10.750733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:30:42.909149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T03:17:13.089441+0000","last_change":"2024-09-12T03:17:11.780916+0000","last_active":"2024-09-12T03:17:13.089441+0000","last_peered":"2024-09-12T03:17:13.089441+0000","last_clean":"2024-09-12T03:17:13.089441+0000","last_became_active":"2024-09-12T03:17:11.780330+0000","last_became_peered":"2024-09-12T03:17:11.780330+0000","last_unstale":"2024-09-12T03:17:13.089441+0000","last_undegraded":"2024-09-12T03:17:13.089441+0000","last_fullsized":"2024-09-12T03:17:13.089441+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_clean_scrub_stamp":"2024-09-12T03:17:10.750733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:59:35.623536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T03:17:13.088620+0000","last_change":"2024-09-12T03:17:11.779022+0000","last_active":"2024-09-12T03:17:13.088620+0000","last_peered":"2024-09-12T03:17:13.088620+0000","last_clean":"2024-09-12T03:17:13.088620+0000","last_became_active":"2024-09-12T03:17:11.778566+0000","last_became_peered":"2024-09-12T03:17:11.778566+0000","last_unstale":"2024-09-12T03:17:13.088620+0000","last_undegraded":"2024-09-12T03:17:13.088620+0000","last_fullsized":"2024-09-12T03:17:13.088620+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_clean_scrub_stamp":"2024-09-12T03:17:10.750733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:47:33.356750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T03:17:12.775640+0000","last_change":"2024-09-12T03:17:11.771243+0000","last_active":"2024-09-12T03:17:12.775640+0000","last_peered":"2024-09-12T03:17:12.775640+0000","last_clean":"2024-09-12T03:17:12.775640+0000","last_became_active":"2024-09-12T03:17:11.770965+0000","last_became_peered":"2024-09-12T03:17:11.770965+0000","last_unstale":"2024-09-12T03:17:12.775640+0000","last_undegraded":"2024-09-12T03:17:12.775640+0000","last_fullsized":"2024-09-12T03:17:12.775640+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_clean_scrub_stamp":"2024-09-12T03:17:10.750733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:41:03.837439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T03:17:13.090026+0000","last_change":"2024-09-12T03:17:11.790806+0000","last_active":"2024-09-12T03:17:13.090026+0000","last_peered":"2024-09-12T03:17:13.090026+0000","last_clean":"2024-09-12T03:17:13.090026+0000","last_became_active":"2024-09-12T03:17:11.790504+0000","last_became_peered":"2024-09-12T03:17:11.790504+0000","last_unstale":"2024-09-12T03:17:13.090026+0000","last_undegraded":"2024-09-12T03:17:13.090026+0000","last_fullsized":"2024-09-12T03:17:13.090026+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_clean_scrub_stamp":"2024-09-12T03:17:10.750733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:19:03.725717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T03:17:13.089209+0000","last_change":"2024-09-12T03:17:11.779033+0000","last_active":"2024-09-12T03:17:13.089209+0000","last_peered":"2024-09-12T03:17:13.089209+0000","last_clean":"2024-09-12T03:17:13.089209+0000","last_became_active":"2024-09-12T03:17:11.778681+0000","last_became_peered":"2024-09-12T03:17:11.778681+0000","last_unstale":"2024-09-12T03:17:13.089209+0000","last_undegraded":"2024-09-12T03:17:13.089209+0000","last_fullsized":"2024-09-12T03:17:13.089209+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_clean_scrub_stamp":"2024-09-12T03:17:10.750733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:23:01.003978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T03:17:12.776013+0000","last_change":"2024-09-12T03:17:11.778007+0000","last_active":"2024-09-12T03:17:12.776013+0000","last_peered":"2024-09-12T03:17:12.776013+0000","last_clean":"2024-09-12T03:17:12.776013+0000","last_became_active":"2024-09-12T03:17:11.777706+0000","last_became_peered":"2024-09-12T03:17:11.777706+0000","last_unstale":"2024-09-12T03:17:12.776013+0000","last_undegraded":"2024-09-12T03:17:12.776013+0000","last_fullsized":"2024-09-12T03:17:12.776013+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_clean_scrub_stamp":"2024-09-12T03:17:10.750733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:45:41.636309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T03:17:13.088725+0000","last_change":"2024-09-12T03:17:11.773368+0000","last_active":"2024-09-12T03:17:13.088725+0000","last_peered":"2024-09-12T03:17:13.088725+0000","last_clean":"2024-09-12T03:17:13.088725+0000","last_became_active":"2024-09-12T03:17:11.772803+0000","last_became_peered":"2024-09-12T03:17:11.772803+0000","last_unstale":"2024-09-12T03:17:13.088725+0000","last_undegraded":"2024-09-12T03:17:13.088725+0000","last_fullsized":"2024-09-12T03:17:13.088725+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_clean_scrub_stamp":"2024-09-12T03:17:10.750733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:08:10.397602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T03:17:13.088742+0000","last_change":"2024-09-12T03:17:11.785926+0000","last_active":"2024-09-12T03:17:13.088742+0000","last_peered":"2024-09-12T03:17:13.088742+0000","last_clean":"2024-09-12T03:17:13.088742+0000","last_became_active":"2024-09-12T03:17:11.785650+0000","last_became_peered":"2024-09-12T03:17:11.785650+0000","last_unstale":"2024-09-12T03:17:13.088742+0000","last_undegraded":"2024-09-12T03:17:13.088742+0000","last_fullsized":"2024-09-12T03:17:13.088742+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_clean_scrub_stamp":"2024-09-12T03:17:10.750733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:55:51.440990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T03:17:12.776173+0000","last_change":"2024-09-12T03:17:11.775428+0000","last_active":"2024-09-12T03:17:12.776173+0000","last_peered":"2024-09-12T03:17:12.776173+0000","last_clean":"2024-09-12T03:17:12.776173+0000","last_became_active":"2024-09-12T03:17:11.775076+0000","last_became_peered":"2024-09-12T03:17:11.775076+0000","last_unstale":"2024-09-12T03:17:12.776173+0000","last_undegraded":"2024-09-12T03:17:12.776173+0000","last_fullsized":"2024-09-12T03:17:12.776173+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_clean_scrub_stamp":"2024-09-12T03:17:10.750733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:04:32.887712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T03:17:13.089549+0000","last_change":"2024-09-12T03:17:11.791428+0000","last_active":"2024-09-12T03:17:13.089549+0000","last_peered":"2024-09-12T03:17:13.089549+0000","last_clean":"2024-09-12T03:17:13.089549+0000","last_became_active":"2024-09-12T03:17:11.791117+0000","last_became_peered":"2024-09-12T03:17:11.791117+0000","last_unstale":"2024-09-12T03:17:13.089549+0000","last_undegraded":"2024-09-12T03:17:13.089549+0000","last_fullsized":"2024-09-12T03:17:13.089549+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_clean_scrub_stamp":"2024-09-12T03:17:10.750733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:37:23.637407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T03:17:13.089249+0000","last_change":"2024-09-12T03:17:11.778544+0000","last_active":"2024-09-12T03:17:13.089249+0000","last_peered":"2024-09-12T03:17:13.089249+0000","last_clean":"2024-09-12T03:17:13.089249+0000","last_became_active":"2024-09-12T03:17:11.778233+0000","last_became_peered":"2024-09-12T03:17:11.778233+0000","last_unstale":"2024-09-12T03:17:13.089249+0000","last_undegraded":"2024-09-12T03:17:13.089249+0000","last_fullsized":"2024-09-12T03:17:13.089249+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_clean_scrub_stamp":"2024-09-12T03:17:10.750733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:01:52.026164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T03:17:13.089630+0000","last_change":"2024-09-12T03:17:11.786723+0000","last_active":"2024-09-12T03:17:13.089630+0000","last_peered":"2024-09-12T03:17:13.089630+0000","last_clean":"2024-09-12T03:17:13.089630+0000","last_became_active":"2024-09-12T03:17:11.786460+0000","last_became_peered":"2024-09-12T03:17:11.786460+0000","last_unstale":"2024-09-12T03:17:13.089630+0000","last_undegraded":"2024-09-12T03:17:13.089630+0000","last_fullsized":"2024-09-12T03:17:13.089630+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_clean_scrub_stamp":"2024-09-12T03:17:10.750733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:59:35.623536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T03:17:13.089442+0000","last_change":"2024-09-12T03:17:11.786082+0000","last_active":"2024-09-12T03:17:13.089442+0000","last_peered":"2024-09-12T03:17:13.089442+0000","last_clean":"2024-09-12T03:17:13.089442+0000","last_became_active":"2024-09-12T03:17:11.785774+0000","last_became_peered":"2024-09-12T03:17:11.785774+0000","last_unstale":"2024-09-12T03:17:13.089442+0000","last_undegraded":"2024-09-12T03:17:13.089442+0000","last_fullsized":"2024-09-12T03:17:13.089442+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_clean_scrub_stamp":"2024-09-12T03:17:10.750733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:45:01.240159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T03:17:13.088793+0000","last_change":"2024-09-12T03:17:11.782167+0000","last_active":"2024-09-12T03:17:13.088793+0000","last_peered":"2024-09-12T03:17:13.088793+0000","last_clean":"2024-09-12T03:17:13.088793+0000","last_became_active":"2024-09-12T03:17:11.781912+0000","last_became_peered":"2024-09-12T03:17:11.781912+0000","last_unstale":"2024-09-12T03:17:13.088793+0000","last_undegraded":"2024-09-12T03:17:13.088793+0000","last_fullsized":"2024-09-12T03:17:13.088793+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_clean_scrub_stamp":"2024-09-12T03:17:10.750733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:54:34.765447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T03:17:12.775612+0000","last_change":"2024-09-12T03:17:11.771671+0000","last_active":"2024-09-12T03:17:12.775612+0000","last_peered":"2024-09-12T03:17:12.775612+0000","last_clean":"2024-09-12T03:17:12.775612+0000","last_became_active":"2024-09-12T03:17:11.771368+0000","last_became_peered":"2024-09-12T03:17:11.771368+0000","last_unstale":"2024-09-12T03:17:12.775612+0000","last_undegraded":"2024-09-12T03:17:12.775612+0000","last_fullsized":"2024-09-12T03:17:12.775612+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_clean_scrub_stamp":"2024-09-12T03:17:10.750733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:39:32.513510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T03:17:12.775544+0000","last_change":"2024-09-12T03:17:11.781376+0000","last_active":"2024-09-12T03:17:12.775544+0000","last_peered":"2024-09-12T03:17:12.775544+0000","last_clean":"2024-09-12T03:17:12.775544+0000","last_became_active":"2024-09-12T03:17:11.781043+0000","last_became_peered":"2024-09-12T03:17:11.781043+0000","last_unstale":"2024-09-12T03:17:12.775544+0000","last_undegraded":"2024-09-12T03:17:12.775544+0000","last_fullsized":"2024-09-12T03:17:12.775544+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_clean_scrub_stamp":"2024-09-12T03:17:10.750733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:34:11.892901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T03:17:12.774516+0000","last_change":"2024-09-12T03:17:11.781592+0000","last_active":"2024-09-12T03:17:12.774516+0000","last_peered":"2024-09-12T03:17:12.774516+0000","last_clean":"2024-09-12T03:17:12.774516+0000","last_became_active":"2024-09-12T03:17:11.781287+0000","last_became_peered":"2024-09-12T03:17:11.781287+0000","last_unstale":"2024-09-12T03:17:12.774516+0000","last_undegraded":"2024-09-12T03:17:12.774516+0000","last_fullsized":"2024-09-12T03:17:12.774516+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_clean_scrub_stamp":"2024-09-12T03:17:10.750733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:56:03.873101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T03:17:12.775109+0000","last_change":"2024-09-12T03:17:11.775418+0000","last_active":"2024-09-12T03:17:12.775109+0000","last_peered":"2024-09-12T03:17:12.775109+0000","last_clean":"2024-09-12T03:17:12.775109+0000","last_became_active":"2024-09-12T03:17:11.774749+0000","last_became_peered":"2024-09-12T03:17:11.774749+0000","last_unstale":"2024-09-12T03:17:12.775109+0000","last_undegraded":"2024-09-12T03:17:12.775109+0000","last_fullsized":"2024-09-12T03:17:12.775109+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_clean_scrub_stamp":"2024-09-12T03:17:10.750733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:15:33.934285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T03:17:13.090238+0000","last_change":"2024-09-12T03:17:11.783257+0000","last_active":"2024-09-12T03:17:13.090238+0000","last_peered":"2024-09-12T03:17:13.090238+0000","last_clean":"2024-09-12T03:17:13.090238+0000","last_became_active":"2024-09-12T03:17:11.782998+0000","last_became_peered":"2024-09-12T03:17:11.782998+0000","last_unstale":"2024-09-12T03:17:13.090238+0000","last_undegraded":"2024-09-12T03:17:13.090238+0000","last_fullsized":"2024-09-12T03:17:13.090238+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_clean_scrub_stamp":"2024-09-12T03:17:10.750733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:15:33.934285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T03:17:13.089664+0000","last_change":"2024-09-12T03:17:11.785343+0000","last_active":"2024-09-12T03:17:13.089664+0000","last_peered":"2024-09-12T03:17:13.089664+0000","last_clean":"2024-09-12T03:17:13.089664+0000","last_became_active":"2024-09-12T03:17:11.785068+0000","last_became_peered":"2024-09-12T03:17:11.785068+0000","last_unstale":"2024-09-12T03:17:13.089664+0000","last_undegraded":"2024-09-12T03:17:13.089664+0000","last_fullsized":"2024-09-12T03:17:13.089664+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_clean_scrub_stamp":"2024-09-12T03:17:10.750733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:37:23.637407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T03:17:13.090079+0000","last_change":"2024-09-12T03:17:11.779518+0000","last_active":"2024-09-12T03:17:13.090079+0000","last_peered":"2024-09-12T03:17:13.090079+0000","last_clean":"2024-09-12T03:17:13.090079+0000","last_became_active":"2024-09-12T03:17:11.778395+0000","last_became_peered":"2024-09-12T03:17:11.778395+0000","last_unstale":"2024-09-12T03:17:13.090079+0000","last_undegraded":"2024-09-12T03:17:13.090079+0000","last_fullsized":"2024-09-12T03:17:13.090079+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_clean_scrub_stamp":"2024-09-12T03:17:10.750733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:13:19.607814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T03:17:12.776329+0000","last_change":"2024-09-12T03:17:11.772432+0000","last_active":"2024-09-12T03:17:12.776329+0000","last_peered":"2024-09-12T03:17:12.776329+0000","last_clean":"2024-09-12T03:17:12.776329+0000","last_became_active":"2024-09-12T03:17:11.772144+0000","last_became_peered":"2024-09-12T03:17:11.772144+0000","last_unstale":"2024-09-12T03:17:12.776329+0000","last_undegraded":"2024-09-12T03:17:12.776329+0000","last_fullsized":"2024-09-12T03:17:12.776329+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_clean_scrub_stamp":"2024-09-12T03:17:10.750733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:03:46.402225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T03:17:12.775609+0000","last_change":"2024-09-12T03:17:11.780265+0000","last_active":"2024-09-12T03:17:12.775609+0000","last_peered":"2024-09-12T03:17:12.775609+0000","last_clean":"2024-09-12T03:17:12.775609+0000","last_became_active":"2024-09-12T03:17:11.779842+0000","last_became_peered":"2024-09-12T03:17:11.779842+0000","last_unstale":"2024-09-12T03:17:12.775609+0000","last_undegraded":"2024-09-12T03:17:12.775609+0000","last_fullsized":"2024-09-12T03:17:12.775609+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_clean_scrub_stamp":"2024-09-12T03:17:10.750733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:47:53.209486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T03:17:13.088938+0000","last_change":"2024-09-12T03:17:11.779020+0000","last_active":"2024-09-12T03:17:13.088938+0000","last_peered":"2024-09-12T03:17:13.088938+0000","last_clean":"2024-09-12T03:17:13.088938+0000","last_became_active":"2024-09-12T03:17:11.778463+0000","last_became_peered":"2024-09-12T03:17:11.778463+0000","last_unstale":"2024-09-12T03:17:13.088938+0000","last_undegraded":"2024-09-12T03:17:13.088938+0000","last_fullsized":"2024-09-12T03:17:13.088938+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_clean_scrub_stamp":"2024-09-12T03:17:10.750733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:03:45.758992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T03:17:13.088662+0000","last_change":"2024-09-12T03:17:11.775960+0000","last_active":"2024-09-12T03:17:13.088662+0000","last_peered":"2024-09-12T03:17:13.088662+0000","last_clean":"2024-09-12T03:17:13.088662+0000","last_became_active":"2024-09-12T03:17:11.775687+0000","last_became_peered":"2024-09-12T03:17:11.775687+0000","last_unstale":"2024-09-12T03:17:13.088662+0000","last_undegraded":"2024-09-12T03:17:13.088662+0000","last_fullsized":"2024-09-12T03:17:13.088662+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_clean_scrub_stamp":"2024-09-12T03:17:10.750733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:40:11.027571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T03:17:13.089118+0000","last_change":"2024-09-12T03:17:11.779426+0000","last_active":"2024-09-12T03:17:13.089118+0000","last_peered":"2024-09-12T03:17:13.089118+0000","last_clean":"2024-09-12T03:17:13.089118+0000","last_became_active":"2024-09-12T03:17:11.778941+0000","last_became_peered":"2024-09-12T03:17:11.778941+0000","last_unstale":"2024-09-12T03:17:13.089118+0000","last_undegraded":"2024-09-12T03:17:13.089118+0000","last_fullsized":"2024-09-12T03:17:13.089118+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_clean_scrub_stamp":"2024-09-12T03:17:10.750733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:30:42.909149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T03:17:12.774705+0000","last_change":"2024-09-12T03:17:11.782473+0000","last_active":"2024-09-12T03:17:12.774705+0000","last_peered":"2024-09-12T03:17:12.774705+0000","last_clean":"2024-09-12T03:17:12.774705+0000","last_became_active":"2024-09-12T03:17:11.782170+0000","last_became_peered":"2024-09-12T03:17:11.782170+0000","last_unstale":"2024-09-12T03:17:12.774705+0000","last_undegraded":"2024-09-12T03:17:12.774705+0000","last_fullsized":"2024-09-12T03:17:12.774705+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_clean_scrub_stamp":"2024-09-12T03:17:10.750733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:36:23.226131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"31'4","reported_seq":18,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T03:17:13.088928+0000","last_change":"2024-09-12T03:17:10.773784+0000","last_active":"2024-09-12T03:17:13.088928+0000","last_peered":"2024-09-12T03:17:13.088928+0000","last_clean":"2024-09-12T03:17:13.088928+0000","last_became_active":"2024-09-12T03:17:10.773452+0000","last_became_peered":"2024-09-12T03:17:10.773452+0000","last_unstale":"2024-09-12T03:17:13.088928+0000","last_undegraded":"2024-09-12T03:17:13.088928+0000","last_fullsized":"2024-09-12T03:17:13.088928+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:09.744809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:09.744809+0000","last_clean_scrub_stamp":"2024-09-12T03:17:09.744809+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:14:53.342181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T03:17:13.088997+0000","last_change":"2024-09-12T03:17:11.778222+0000","last_active":"2024-09-12T03:17:13.088997+0000","last_peered":"2024-09-12T03:17:13.088997+0000","last_clean":"2024-09-12T03:17:13.088997+0000","last_became_active":"2024-09-12T03:17:11.777564+0000","last_became_peered":"2024-09-12T03:17:11.777564+0000","last_unstale":"2024-09-12T03:17:13.088997+0000","last_undegraded":"2024-09-12T03:17:13.088997+0000","last_fullsized":"2024-09-12T03:17:13.088997+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_clean_scrub_stamp":"2024-09-12T03:17:10.750733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:34:11.892901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"31'1","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T03:17:12.774608+0000","last_change":"2024-09-12T03:17:10.778806+0000","last_active":"2024-09-12T03:17:12.774608+0000","last_peered":"2024-09-12T03:17:12.774608+0000","last_clean":"2024-09-12T03:17:12.774608+0000","last_became_active":"2024-09-12T03:17:10.778263+0000","last_became_peered":"2024-09-12T03:17:10.778263+0000","last_unstale":"2024-09-12T03:17:12.774608+0000","last_undegraded":"2024-09-12T03:17:12.774608+0000","last_fullsized":"2024-09-12T03:17:12.774608+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:09.744809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:09.744809+0000","last_clean_scrub_stamp":"2024-09-12T03:17:09.744809+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:20:43.037133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T03:17:12.775696+0000","last_change":"2024-09-12T03:17:11.778688+0000","last_active":"2024-09-12T03:17:12.775696+0000","last_peered":"2024-09-12T03:17:12.775696+0000","last_clean":"2024-09-12T03:17:12.775696+0000","last_became_active":"2024-09-12T03:17:11.778401+0000","last_became_peered":"2024-09-12T03:17:11.778401+0000","last_unstale":"2024-09-12T03:17:12.775696+0000","last_undegraded":"2024-09-12T03:17:12.775696+0000","last_fullsized":"2024-09-12T03:17:12.775696+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_clean_scrub_stamp":"2024-09-12T03:17:10.750733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:17:31.404497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"31'2","reported_seq":16,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T03:17:13.089254+0000","last_change":"2024-09-12T03:17:10.779644+0000","last_active":"2024-09-12T03:17:13.089254+0000","last_peered":"2024-09-12T03:17:13.089254+0000","last_clean":"2024-09-12T03:17:13.089254+0000","last_became_active":"2024-09-12T03:17:10.779333+0000","last_became_peered":"2024-09-12T03:17:10.779333+0000","last_unstale":"2024-09-12T03:17:13.089254+0000","last_undegraded":"2024-09-12T03:17:13.089254+0000","last_fullsized":"2024-09-12T03:17:13.089254+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:09.744809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:09.744809+0000","last_clean_scrub_stamp":"2024-09-12T03:17:09.744809+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:50:48.287571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T03:17:13.090174+0000","last_change":"2024-09-12T03:17:11.778679+0000","last_active":"2024-09-12T03:17:13.090174+0000","last_peered":"2024-09-12T03:17:13.090174+0000","last_clean":"2024-09-12T03:17:13.090174+0000","last_became_active":"2024-09-12T03:17:11.778015+0000","last_became_peered":"2024-09-12T03:17:11.778015+0000","last_unstale":"2024-09-12T03:17:13.090174+0000","last_undegraded":"2024-09-12T03:17:13.090174+0000","last_fullsized":"2024-09-12T03:17:13.090174+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_clean_scrub_stamp":"2024-09-12T03:17:10.750733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:09:13.859903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"31'2","reported_seq":16,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T03:17:13.088810+0000","last_change":"2024-09-12T03:17:10.770885+0000","last_active":"2024-09-12T03:17:13.088810+0000","last_peered":"2024-09-12T03:17:13.088810+0000","last_clean":"2024-09-12T03:17:13.088810+0000","last_became_active":"2024-09-12T03:17:10.770069+0000","last_became_peered":"2024-09-12T03:17:10.770069+0000","last_unstale":"2024-09-12T03:17:13.088810+0000","last_undegraded":"2024-09-12T03:17:13.088810+0000","last_fullsized":"2024-09-12T03:17:13.088810+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:09.744809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:09.744809+0000","last_clean_scrub_stamp":"2024-09-12T03:17:09.744809+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:24:01.689326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T03:17:13.089195+0000","last_change":"2024-09-12T03:17:11.778767+0000","last_active":"2024-09-12T03:17:13.089195+0000","last_peered":"2024-09-12T03:17:13.089195+0000","last_clean":"2024-09-12T03:17:13.089195+0000","last_became_active":"2024-09-12T03:17:11.778134+0000","last_became_peered":"2024-09-12T03:17:11.778134+0000","last_unstale":"2024-09-12T03:17:13.089195+0000","last_undegraded":"2024-09-12T03:17:13.089195+0000","last_fullsized":"2024-09-12T03:17:13.089195+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_clean_scrub_stamp":"2024-09-12T03:17:10.750733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:55:51.440990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"31'1","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T03:17:12.775702+0000","last_change":"2024-09-12T03:17:10.769838+0000","last_active":"2024-09-12T03:17:12.775702+0000","last_peered":"2024-09-12T03:17:12.775702+0000","last_clean":"2024-09-12T03:17:12.775702+0000","last_became_active":"2024-09-12T03:17:10.769537+0000","last_became_peered":"2024-09-12T03:17:10.769537+0000","last_unstale":"2024-09-12T03:17:12.775702+0000","last_undegraded":"2024-09-12T03:17:12.775702+0000","last_fullsized":"2024-09-12T03:17:12.775702+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:09.744809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:09.744809+0000","last_clean_scrub_stamp":"2024-09-12T03:17:09.744809+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:14:53.342181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T03:17:12.776060+0000","last_change":"2024-09-12T03:17:11.770723+0000","last_active":"2024-09-12T03:17:12.776060+0000","last_peered":"2024-09-12T03:17:12.776060+0000","last_clean":"2024-09-12T03:17:12.776060+0000","last_became_active":"2024-09-12T03:17:11.770400+0000","last_became_peered":"2024-09-12T03:17:11.770400+0000","last_unstale":"2024-09-12T03:17:12.776060+0000","last_undegraded":"2024-09-12T03:17:12.776060+0000","last_fullsized":"2024-09-12T03:17:12.776060+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_clean_scrub_stamp":"2024-09-12T03:17:10.750733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:32:08.469199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"31'1","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T03:17:12.775634+0000","last_change":"2024-09-12T03:17:10.770703+0000","last_active":"2024-09-12T03:17:12.775634+0000","last_peered":"2024-09-12T03:17:12.775634+0000","last_clean":"2024-09-12T03:17:12.775634+0000","last_became_active":"2024-09-12T03:17:10.770292+0000","last_became_peered":"2024-09-12T03:17:10.770292+0000","last_unstale":"2024-09-12T03:17:12.775634+0000","last_undegraded":"2024-09-12T03:17:12.775634+0000","last_fullsized":"2024-09-12T03:17:12.775634+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:09.744809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:09.744809+0000","last_clean_scrub_stamp":"2024-09-12T03:17:09.744809+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:56:42.498299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T03:17:12.775091+0000","last_change":"2024-09-12T03:17:11.776472+0000","last_active":"2024-09-12T03:17:12.775091+0000","last_peered":"2024-09-12T03:17:12.775091+0000","last_clean":"2024-09-12T03:17:12.775091+0000","last_became_active":"2024-09-12T03:17:11.776194+0000","last_became_peered":"2024-09-12T03:17:11.776194+0000","last_unstale":"2024-09-12T03:17:12.775091+0000","last_undegraded":"2024-09-12T03:17:12.775091+0000","last_fullsized":"2024-09-12T03:17:12.775091+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_clean_scrub_stamp":"2024-09-12T03:17:10.750733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:11:36.377619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T03:17:13.088571+0000","last_change":"2024-09-12T03:17:10.769799+0000","last_active":"2024-09-12T03:17:13.088571+0000","last_peered":"2024-09-12T03:17:13.088571+0000","last_clean":"2024-09-12T03:17:13.088571+0000","last_became_active":"2024-09-12T03:17:10.769536+0000","last_became_peered":"2024-09-12T03:17:10.769536+0000","last_unstale":"2024-09-12T03:17:13.088571+0000","last_undegraded":"2024-09-12T03:17:13.088571+0000","last_fullsized":"2024-09-12T03:17:13.088571+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:09.744809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:09.744809+0000","last_clean_scrub_stamp":"2024-09-12T03:17:09.744809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:23:09.520304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T03:17:13.089115+0000","last_change":"2024-09-12T03:17:11.781294+0000","last_active":"2024-09-12T03:17:13.089115+0000","last_peered":"2024-09-12T03:17:13.089115+0000","last_clean":"2024-09-12T03:17:13.089115+0000","last_became_active":"2024-09-12T03:17:11.781007+0000","last_became_peered":"2024-09-12T03:17:11.781007+0000","last_unstale":"2024-09-12T03:17:13.089115+0000","last_undegraded":"2024-09-12T03:17:13.089115+0000","last_fullsized":"2024-09-12T03:17:13.089115+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_clean_scrub_stamp":"2024-09-12T03:17:10.750733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:15:59.115399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T03:17:12.775167+0000","last_change":"2024-09-12T03:17:10.770204+0000","last_active":"2024-09-12T03:17:12.775167+0000","last_peered":"2024-09-12T03:17:12.775167+0000","last_clean":"2024-09-12T03:17:12.775167+0000","last_became_active":"2024-09-12T03:17:10.769873+0000","last_became_peered":"2024-09-12T03:17:10.769873+0000","last_unstale":"2024-09-12T03:17:12.775167+0000","last_undegraded":"2024-09-12T03:17:12.775167+0000","last_fullsized":"2024-09-12T03:17:12.775167+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:09.744809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:09.744809+0000","last_clean_scrub_stamp":"2024-09-12T03:17:09.744809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:09:15.426787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T03:17:12.776638+0000","last_change":"2024-09-12T03:17:11.776120+0000","last_active":"2024-09-12T03:17:12.776638+0000","last_peered":"2024-09-12T03:17:12.776638+0000","last_clean":"2024-09-12T03:17:12.776638+0000","last_became_active":"2024-09-12T03:17:11.775864+0000","last_became_peered":"2024-09-12T03:17:11.775864+0000","last_unstale":"2024-09-12T03:17:12.776638+0000","last_undegraded":"2024-09-12T03:17:12.776638+0000","last_fullsized":"2024-09-12T03:17:12.776638+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_clean_scrub_stamp":"2024-09-12T03:17:10.750733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:24:28.798972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T03:17:13.089759+0000","last_change":"2024-09-12T03:17:10.781279+0000","last_active":"2024-09-12T03:17:13.089759+0000","last_peered":"2024-09-12T03:17:13.089759+0000","last_clean":"2024-09-12T03:17:13.089759+0000","last_became_active":"2024-09-12T03:17:10.780983+0000","last_became_peered":"2024-09-12T03:17:10.780983+0000","last_unstale":"2024-09-12T03:17:13.089759+0000","last_undegraded":"2024-09-12T03:17:13.089759+0000","last_fullsized":"2024-09-12T03:17:13.089759+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:09.744809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:09.744809+0000","last_clean_scrub_stamp":"2024-09-12T03:17:09.744809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:47:52.203562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T03:17:13.089737+0000","last_change":"2024-09-12T03:17:11.787249+0000","last_active":"2024-09-12T03:17:13.089737+0000","last_peered":"2024-09-12T03:17:13.089737+0000","last_clean":"2024-09-12T03:17:13.089737+0000","last_became_active":"2024-09-12T03:17:11.786387+0000","last_became_peered":"2024-09-12T03:17:11.786387+0000","last_unstale":"2024-09-12T03:17:13.089737+0000","last_undegraded":"2024-09-12T03:17:13.089737+0000","last_fullsized":"2024-09-12T03:17:13.089737+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_clean_scrub_stamp":"2024-09-12T03:17:10.750733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:15:36.085059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T03:17:12.774715+0000","last_change":"2024-09-12T03:17:10.780446+0000","last_active":"2024-09-12T03:17:12.774715+0000","last_peered":"2024-09-12T03:17:12.774715+0000","last_clean":"2024-09-12T03:17:12.774715+0000","last_became_active":"2024-09-12T03:17:10.780150+0000","last_became_peered":"2024-09-12T03:17:10.780150+0000","last_unstale":"2024-09-12T03:17:12.774715+0000","last_undegraded":"2024-09-12T03:17:12.774715+0000","last_fullsized":"2024-09-12T03:17:12.774715+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:09.744809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:09.744809+0000","last_clean_scrub_stamp":"2024-09-12T03:17:09.744809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:15:18.019049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T03:17:13.089245+0000","last_change":"2024-09-12T03:17:11.785146+0000","last_active":"2024-09-12T03:17:13.089245+0000","last_peered":"2024-09-12T03:17:13.089245+0000","last_clean":"2024-09-12T03:17:13.089245+0000","last_became_active":"2024-09-12T03:17:11.784870+0000","last_became_peered":"2024-09-12T03:17:11.784870+0000","last_unstale":"2024-09-12T03:17:13.089245+0000","last_undegraded":"2024-09-12T03:17:13.089245+0000","last_fullsized":"2024-09-12T03:17:13.089245+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_clean_scrub_stamp":"2024-09-12T03:17:10.750733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:01:52.026164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"31'3","reported_seq":17,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T03:17:12.776598+0000","last_change":"2024-09-12T03:17:10.779898+0000","last_active":"2024-09-12T03:17:12.776598+0000","last_peered":"2024-09-12T03:17:12.776598+0000","last_clean":"2024-09-12T03:17:12.776598+0000","last_became_active":"2024-09-12T03:17:10.779308+0000","last_became_peered":"2024-09-12T03:17:10.779308+0000","last_unstale":"2024-09-12T03:17:12.776598+0000","last_undegraded":"2024-09-12T03:17:12.776598+0000","last_fullsized":"2024-09-12T03:17:12.776598+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:09.744809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:09.744809+0000","last_clean_scrub_stamp":"2024-09-12T03:17:09.744809+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:04:40.775420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T03:17:12.776542+0000","last_change":"2024-09-12T03:17:11.777164+0000","last_active":"2024-09-12T03:17:12.776542+0000","last_peered":"2024-09-12T03:17:12.776542+0000","last_clean":"2024-09-12T03:17:12.776542+0000","last_became_active":"2024-09-12T03:17:11.776877+0000","last_became_peered":"2024-09-12T03:17:11.776877+0000","last_unstale":"2024-09-12T03:17:12.776542+0000","last_undegraded":"2024-09-12T03:17:12.776542+0000","last_fullsized":"2024-09-12T03:17:12.776542+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_clean_scrub_stamp":"2024-09-12T03:17:10.750733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:37:23.776329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"31'6","reported_seq":20,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T03:17:13.089342+0000","last_change":"2024-09-12T03:17:10.779835+0000","last_active":"2024-09-12T03:17:13.089342+0000","last_peered":"2024-09-12T03:17:13.089342+0000","last_clean":"2024-09-12T03:17:13.089342+0000","last_became_active":"2024-09-12T03:17:10.779475+0000","last_became_peered":"2024-09-12T03:17:10.779475+0000","last_unstale":"2024-09-12T03:17:13.089342+0000","last_undegraded":"2024-09-12T03:17:13.089342+0000","last_fullsized":"2024-09-12T03:17:13.089342+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:09.744809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:09.744809+0000","last_clean_scrub_stamp":"2024-09-12T03:17:09.744809+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:40:10.021647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T03:17:13.089288+0000","last_change":"2024-09-12T03:17:11.782513+0000","last_active":"2024-09-12T03:17:13.089288+0000","last_peered":"2024-09-12T03:17:13.089288+0000","last_clean":"2024-09-12T03:17:13.089288+0000","last_became_active":"2024-09-12T03:17:11.782240+0000","last_became_peered":"2024-09-12T03:17:11.782240+0000","last_unstale":"2024-09-12T03:17:13.089288+0000","last_undegraded":"2024-09-12T03:17:13.089288+0000","last_fullsized":"2024-09-12T03:17:13.089288+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_clean_scrub_stamp":"2024-09-12T03:17:10.750733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:29:38.693671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"31'2","reported_seq":16,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T03:17:12.775704+0000","last_change":"2024-09-12T03:17:10.769047+0000","last_active":"2024-09-12T03:17:12.775704+0000","last_peered":"2024-09-12T03:17:12.775704+0000","last_clean":"2024-09-12T03:17:12.775704+0000","last_became_active":"2024-09-12T03:17:10.768736+0000","last_became_peered":"2024-09-12T03:17:10.768736+0000","last_unstale":"2024-09-12T03:17:12.775704+0000","last_undegraded":"2024-09-12T03:17:12.775704+0000","last_fullsized":"2024-09-12T03:17:12.775704+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:09.744809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:09.744809+0000","last_clean_scrub_stamp":"2024-09-12T03:17:09.744809+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:06:19.543979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T03:17:13.088697+0000","last_change":"2024-09-12T03:17:11.778052+0000","last_active":"2024-09-12T03:17:13.088697+0000","last_peered":"2024-09-12T03:17:13.088697+0000","last_clean":"2024-09-12T03:17:13.088697+0000","last_became_active":"2024-09-12T03:17:11.777794+0000","last_became_peered":"2024-09-12T03:17:11.777794+0000","last_unstale":"2024-09-12T03:17:13.088697+0000","last_undegraded":"2024-09-12T03:17:13.088697+0000","last_fullsized":"2024-09-12T03:17:13.088697+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_clean_scrub_stamp":"2024-09-12T03:17:10.750733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:10:52.379296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T03:17:13.089414+0000","last_change":"2024-09-12T03:17:10.775903+0000","last_active":"2024-09-12T03:17:13.089414+0000","last_peered":"2024-09-12T03:17:13.089414+0000","last_clean":"2024-09-12T03:17:13.089414+0000","last_became_active":"2024-09-12T03:17:10.775488+0000","last_became_peered":"2024-09-12T03:17:10.775488+0000","last_unstale":"2024-09-12T03:17:13.089414+0000","last_undegraded":"2024-09-12T03:17:13.089414+0000","last_fullsized":"2024-09-12T03:17:13.089414+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:09.744809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:09.744809+0000","last_clean_scrub_stamp":"2024-09-12T03:17:09.744809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:16:25.907410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T03:17:13.089347+0000","last_change":"2024-09-12T03:17:11.787635+0000","last_active":"2024-09-12T03:17:13.089347+0000","last_peered":"2024-09-12T03:17:13.089347+0000","last_clean":"2024-09-12T03:17:13.089347+0000","last_became_active":"2024-09-12T03:17:11.787297+0000","last_became_peered":"2024-09-12T03:17:11.787297+0000","last_unstale":"2024-09-12T03:17:13.089347+0000","last_undegraded":"2024-09-12T03:17:13.089347+0000","last_fullsized":"2024-09-12T03:17:13.089347+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_clean_scrub_stamp":"2024-09-12T03:17:10.750733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:00:25.018692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T03:17:13.088827+0000","last_change":"2024-09-12T03:17:10.777953+0000","last_active":"2024-09-12T03:17:13.088827+0000","last_peered":"2024-09-12T03:17:13.088827+0000","last_clean":"2024-09-12T03:17:13.088827+0000","last_became_active":"2024-09-12T03:17:10.777683+0000","last_became_peered":"2024-09-12T03:17:10.777683+0000","last_unstale":"2024-09-12T03:17:13.088827+0000","last_undegraded":"2024-09-12T03:17:13.088827+0000","last_fullsized":"2024-09-12T03:17:13.088827+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:09.744809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:09.744809+0000","last_clean_scrub_stamp":"2024-09-12T03:17:09.744809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:11:35.371695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T03:17:13.088772+0000","last_change":"2024-09-12T03:17:11.778940+0000","last_active":"2024-09-12T03:17:13.088772+0000","last_peered":"2024-09-12T03:17:13.088772+0000","last_clean":"2024-09-12T03:17:13.088772+0000","last_became_active":"2024-09-12T03:17:11.778647+0000","last_became_peered":"2024-09-12T03:17:11.778647+0000","last_unstale":"2024-09-12T03:17:13.088772+0000","last_undegraded":"2024-09-12T03:17:13.088772+0000","last_fullsized":"2024-09-12T03:17:13.088772+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_clean_scrub_stamp":"2024-09-12T03:17:10.750733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:23:10.526228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"31'8","reported_seq":22,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T03:17:13.088875+0000","last_change":"2024-09-12T03:17:10.782332+0000","last_active":"2024-09-12T03:17:13.088875+0000","last_peered":"2024-09-12T03:17:13.088875+0000","last_clean":"2024-09-12T03:17:13.088875+0000","last_became_active":"2024-09-12T03:17:10.781975+0000","last_became_peered":"2024-09-12T03:17:10.781975+0000","last_unstale":"2024-09-12T03:17:13.088875+0000","last_undegraded":"2024-09-12T03:17:13.088875+0000","last_fullsized":"2024-09-12T03:17:13.088875+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:09.744809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:09.744809+0000","last_clean_scrub_stamp":"2024-09-12T03:17:09.744809+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:39:31.507587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T03:17:12.776442+0000","last_change":"2024-09-12T03:17:11.775385+0000","last_active":"2024-09-12T03:17:12.776442+0000","last_peered":"2024-09-12T03:17:12.776442+0000","last_clean":"2024-09-12T03:17:12.776442+0000","last_became_active":"2024-09-12T03:17:11.774907+0000","last_became_peered":"2024-09-12T03:17:11.774907+0000","last_unstale":"2024-09-12T03:17:12.776442+0000","last_undegraded":"2024-09-12T03:17:12.776442+0000","last_fullsized":"2024-09-12T03:17:12.776442+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_clean_scrub_stamp":"2024-09-12T03:17:10.750733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:56:03.873101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"31'1","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T03:17:13.090259+0000","last_change":"2024-09-12T03:17:10.782807+0000","last_active":"2024-09-12T03:17:13.090259+0000","last_peered":"2024-09-12T03:17:13.090259+0000","last_clean":"2024-09-12T03:17:13.090259+0000","last_became_active":"2024-09-12T03:17:10.782489+0000","last_became_peered":"2024-09-12T03:17:10.782489+0000","last_unstale":"2024-09-12T03:17:13.090259+0000","last_undegraded":"2024-09-12T03:17:13.090259+0000","last_fullsized":"2024-09-12T03:17:13.090259+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:09.744809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:09.744809+0000","last_clean_scrub_stamp":"2024-09-12T03:17:09.744809+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:20:56.967739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T03:17:12.776780+0000","last_change":"2024-09-12T03:17:11.767184+0000","last_active":"2024-09-12T03:17:12.776780+0000","last_peered":"2024-09-12T03:17:12.776780+0000","last_clean":"2024-09-12T03:17:12.776780+0000","last_became_active":"2024-09-12T03:17:11.766834+0000","last_became_peered":"2024-09-12T03:17:11.766834+0000","last_unstale":"2024-09-12T03:17:12.776780+0000","last_undegraded":"2024-09-12T03:17:12.776780+0000","last_fullsized":"2024-09-12T03:17:12.776780+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_clean_scrub_stamp":"2024-09-12T03:17:10.750733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:52:26.395994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"31'2","reported_seq":16,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T03:17:12.776195+0000","last_change":"2024-09-12T03:17:10.771697+0000","last_active":"2024-09-12T03:17:12.776195+0000","last_peered":"2024-09-12T03:17:12.776195+0000","last_clean":"2024-09-12T03:17:12.776195+0000","last_became_active":"2024-09-12T03:17:10.771344+0000","last_became_peered":"2024-09-12T03:17:10.771344+0000","last_unstale":"2024-09-12T03:17:12.776195+0000","last_undegraded":"2024-09-12T03:17:12.776195+0000","last_fullsized":"2024-09-12T03:17:12.776195+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:09.744809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:09.744809+0000","last_clean_scrub_stamp":"2024-09-12T03:17:09.744809+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:40:28.767355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T03:17:13.089968+0000","last_change":"2024-09-12T03:17:11.779894+0000","last_active":"2024-09-12T03:17:13.089968+0000","last_peered":"2024-09-12T03:17:13.089968+0000","last_clean":"2024-09-12T03:17:13.089968+0000","last_became_active":"2024-09-12T03:17:11.777544+0000","last_became_peered":"2024-09-12T03:17:11.777544+0000","last_unstale":"2024-09-12T03:17:13.089968+0000","last_undegraded":"2024-09-12T03:17:13.089968+0000","last_fullsized":"2024-09-12T03:17:13.089968+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_clean_scrub_stamp":"2024-09-12T03:17:10.750733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:21:01.438049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"31'1","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T03:17:12.776053+0000","last_change":"2024-09-12T03:17:10.772929+0000","last_active":"2024-09-12T03:17:12.776053+0000","last_peered":"2024-09-12T03:17:12.776053+0000","last_clean":"2024-09-12T03:17:12.776053+0000","last_became_active":"2024-09-12T03:17:10.771340+0000","last_became_peered":"2024-09-12T03:17:10.771340+0000","last_unstale":"2024-09-12T03:17:12.776053+0000","last_undegraded":"2024-09-12T03:17:12.776053+0000","last_fullsized":"2024-09-12T03:17:12.776053+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:09.744809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:09.744809+0000","last_clean_scrub_stamp":"2024-09-12T03:17:09.744809+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:05:56.976702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T03:17:13.088624+0000","last_change":"2024-09-12T03:17:11.785280+0000","last_active":"2024-09-12T03:17:13.088624+0000","last_peered":"2024-09-12T03:17:13.088624+0000","last_clean":"2024-09-12T03:17:13.088624+0000","last_became_active":"2024-09-12T03:17:11.785019+0000","last_became_peered":"2024-09-12T03:17:11.785019+0000","last_unstale":"2024-09-12T03:17:13.088624+0000","last_undegraded":"2024-09-12T03:17:13.088624+0000","last_fullsized":"2024-09-12T03:17:13.088624+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_clean_scrub_stamp":"2024-09-12T03:17:10.750733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:09:16.432711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T03:17:12.775703+0000","last_change":"2024-09-12T03:17:10.770842+0000","last_active":"2024-09-12T03:17:12.775703+0000","last_peered":"2024-09-12T03:17:12.775703+0000","last_clean":"2024-09-12T03:17:12.775703+0000","last_became_active":"2024-09-12T03:17:10.770474+0000","last_became_peered":"2024-09-12T03:17:10.770474+0000","last_unstale":"2024-09-12T03:17:12.775703+0000","last_undegraded":"2024-09-12T03:17:12.775703+0000","last_fullsized":"2024-09-12T03:17:12.775703+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:09.744809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:09.744809+0000","last_clean_scrub_stamp":"2024-09-12T03:17:09.744809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:09:27.839638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T03:17:13.088736+0000","last_change":"2024-09-12T03:17:11.784638+0000","last_active":"2024-09-12T03:17:13.088736+0000","last_peered":"2024-09-12T03:17:13.088736+0000","last_clean":"2024-09-12T03:17:13.088736+0000","last_became_active":"2024-09-12T03:17:11.784288+0000","last_became_peered":"2024-09-12T03:17:11.784288+0000","last_unstale":"2024-09-12T03:17:13.088736+0000","last_undegraded":"2024-09-12T03:17:13.088736+0000","last_fullsized":"2024-09-12T03:17:13.088736+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_clean_scrub_stamp":"2024-09-12T03:17:10.750733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:06:20.549902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"31'2","reported_seq":16,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T03:17:12.774595+0000","last_change":"2024-09-12T03:17:10.774974+0000","last_active":"2024-09-12T03:17:12.774595+0000","last_peered":"2024-09-12T03:17:12.774595+0000","last_clean":"2024-09-12T03:17:12.774595+0000","last_became_active":"2024-09-12T03:17:10.774704+0000","last_became_peered":"2024-09-12T03:17:10.774704+0000","last_unstale":"2024-09-12T03:17:12.774595+0000","last_undegraded":"2024-09-12T03:17:12.774595+0000","last_fullsized":"2024-09-12T03:17:12.774595+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:09.744809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:09.744809+0000","last_clean_scrub_stamp":"2024-09-12T03:17:09.744809+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:40:10.021647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T03:17:13.088883+0000","last_change":"2024-09-12T03:17:11.773198+0000","last_active":"2024-09-12T03:17:13.088883+0000","last_peered":"2024-09-12T03:17:13.088883+0000","last_clean":"2024-09-12T03:17:13.088883+0000","last_became_active":"2024-09-12T03:17:11.772753+0000","last_became_peered":"2024-09-12T03:17:11.772753+0000","last_unstale":"2024-09-12T03:17:13.088883+0000","last_undegraded":"2024-09-12T03:17:13.088883+0000","last_fullsized":"2024-09-12T03:17:13.088883+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_clean_scrub_stamp":"2024-09-12T03:17:10.750733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:14:54.348105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T03:17:13.088869+0000","last_change":"2024-09-12T03:17:10.778107+0000","last_active":"2024-09-12T03:17:13.088869+0000","last_peered":"2024-09-12T03:17:13.088869+0000","last_clean":"2024-09-12T03:17:13.088869+0000","last_became_active":"2024-09-12T03:17:10.777671+0000","last_became_peered":"2024-09-12T03:17:10.777671+0000","last_unstale":"2024-09-12T03:17:13.088869+0000","last_undegraded":"2024-09-12T03:17:13.088869+0000","last_fullsized":"2024-09-12T03:17:13.088869+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:09.744809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:09.744809+0000","last_clean_scrub_stamp":"2024-09-12T03:17:09.744809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:09:32.142057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T03:17:13.088440+0000","last_change":"2024-09-12T03:17:11.780736+0000","last_active":"2024-09-12T03:17:13.088440+0000","last_peered":"2024-09-12T03:17:13.088440+0000","last_clean":"2024-09-12T03:17:13.088440+0000","last_became_active":"2024-09-12T03:17:11.780381+0000","last_became_peered":"2024-09-12T03:17:11.780381+0000","last_unstale":"2024-09-12T03:17:13.088440+0000","last_undegraded":"2024-09-12T03:17:13.088440+0000","last_fullsized":"2024-09-12T03:17:13.088440+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_clean_scrub_stamp":"2024-09-12T03:17:10.750733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:56:43.504222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T03:17:12.775218+0000","last_change":"2024-09-12T03:17:10.773843+0000","last_active":"2024-09-12T03:17:12.775218+0000","last_peered":"2024-09-12T03:17:12.775218+0000","last_clean":"2024-09-12T03:17:12.775218+0000","last_became_active":"2024-09-12T03:17:10.773551+0000","last_became_peered":"2024-09-12T03:17:10.773551+0000","last_unstale":"2024-09-12T03:17:12.775218+0000","last_undegraded":"2024-09-12T03:17:12.775218+0000","last_fullsized":"2024-09-12T03:17:12.775218+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:09.744809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:09.744809+0000","last_clean_scrub_stamp":"2024-09-12T03:17:09.744809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:08:09.391678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T03:17:13.089330+0000","last_change":"2024-09-12T03:17:11.780655+0000","last_active":"2024-09-12T03:17:13.089330+0000","last_peered":"2024-09-12T03:17:13.089330+0000","last_clean":"2024-09-12T03:17:13.089330+0000","last_became_active":"2024-09-12T03:17:11.780022+0000","last_became_peered":"2024-09-12T03:17:11.780022+0000","last_unstale":"2024-09-12T03:17:13.089330+0000","last_undegraded":"2024-09-12T03:17:13.089330+0000","last_fullsized":"2024-09-12T03:17:13.089330+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_clean_scrub_stamp":"2024-09-12T03:17:10.750733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:45:01.240159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T03:17:12.775170+0000","last_change":"2024-09-12T03:17:10.779822+0000","last_active":"2024-09-12T03:17:12.775170+0000","last_peered":"2024-09-12T03:17:12.775170+0000","last_clean":"2024-09-12T03:17:12.775170+0000","last_became_active":"2024-09-12T03:17:10.779476+0000","last_became_peered":"2024-09-12T03:17:10.779476+0000","last_unstale":"2024-09-12T03:17:12.775170+0000","last_undegraded":"2024-09-12T03:17:12.775170+0000","last_fullsized":"2024-09-12T03:17:12.775170+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:09.744809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:09.744809+0000","last_clean_scrub_stamp":"2024-09-12T03:17:09.744809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:47:04.710984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T03:17:13.088634+0000","last_change":"2024-09-12T03:17:11.782814+0000","last_active":"2024-09-12T03:17:13.088634+0000","last_peered":"2024-09-12T03:17:13.088634+0000","last_clean":"2024-09-12T03:17:13.088634+0000","last_became_active":"2024-09-12T03:17:11.782537+0000","last_became_peered":"2024-09-12T03:17:11.782537+0000","last_unstale":"2024-09-12T03:17:13.088634+0000","last_undegraded":"2024-09-12T03:17:13.088634+0000","last_fullsized":"2024-09-12T03:17:13.088634+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_clean_scrub_stamp":"2024-09-12T03:17:10.750733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:09:28.845562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"31'2","reported_seq":16,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T03:17:13.090274+0000","last_change":"2024-09-12T03:17:10.776805+0000","last_active":"2024-09-12T03:17:13.090274+0000","last_peered":"2024-09-12T03:17:13.090274+0000","last_clean":"2024-09-12T03:17:13.090274+0000","last_became_active":"2024-09-12T03:17:10.776516+0000","last_became_peered":"2024-09-12T03:17:10.776516+0000","last_unstale":"2024-09-12T03:17:13.090274+0000","last_undegraded":"2024-09-12T03:17:13.090274+0000","last_fullsized":"2024-09-12T03:17:13.090274+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:09.744809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:09.744809+0000","last_clean_scrub_stamp":"2024-09-12T03:17:09.744809+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:03:14.255327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T03:17:12.776833+0000","last_change":"2024-09-12T03:17:11.772888+0000","last_active":"2024-09-12T03:17:12.776833+0000","last_peered":"2024-09-12T03:17:12.776833+0000","last_clean":"2024-09-12T03:17:12.776833+0000","last_became_active":"2024-09-12T03:17:11.772306+0000","last_became_peered":"2024-09-12T03:17:11.772306+0000","last_unstale":"2024-09-12T03:17:12.776833+0000","last_undegraded":"2024-09-12T03:17:12.776833+0000","last_fullsized":"2024-09-12T03:17:12.776833+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_clean_scrub_stamp":"2024-09-12T03:17:10.750733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:13:02.863333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"31'1","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T03:17:12.774762+0000","last_change":"2024-09-12T03:17:10.778922+0000","last_active":"2024-09-12T03:17:12.774762+0000","last_peered":"2024-09-12T03:17:12.774762+0000","last_clean":"2024-09-12T03:17:12.774762+0000","last_became_active":"2024-09-12T03:17:10.778644+0000","last_became_peered":"2024-09-12T03:17:10.778644+0000","last_unstale":"2024-09-12T03:17:12.774762+0000","last_undegraded":"2024-09-12T03:17:12.774762+0000","last_fullsized":"2024-09-12T03:17:12.774762+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:09.744809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:09.744809+0000","last_clean_scrub_stamp":"2024-09-12T03:17:09.744809+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:11:35.371695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T03:17:12.774571+0000","last_change":"2024-09-12T03:17:11.770253+0000","last_active":"2024-09-12T03:17:12.774571+0000","last_peered":"2024-09-12T03:17:12.774571+0000","last_clean":"2024-09-12T03:17:12.774571+0000","last_became_active":"2024-09-12T03:17:11.769980+0000","last_became_peered":"2024-09-12T03:17:11.769980+0000","last_unstale":"2024-09-12T03:17:12.774571+0000","last_undegraded":"2024-09-12T03:17:12.774571+0000","last_fullsized":"2024-09-12T03:17:12.774571+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_clean_scrub_stamp":"2024-09-12T03:17:10.750733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:08:36.119685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T03:17:13.090018+0000","last_change":"2024-09-12T03:17:10.773923+0000","last_active":"2024-09-12T03:17:13.090018+0000","last_peered":"2024-09-12T03:17:13.090018+0000","last_clean":"2024-09-12T03:17:13.090018+0000","last_became_active":"2024-09-12T03:17:10.773449+0000","last_became_peered":"2024-09-12T03:17:10.773449+0000","last_unstale":"2024-09-12T03:17:13.090018+0000","last_undegraded":"2024-09-12T03:17:13.090018+0000","last_fullsized":"2024-09-12T03:17:13.090018+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:09.744809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:09.744809+0000","last_clean_scrub_stamp":"2024-09-12T03:17:09.744809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:01:40.970760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T03:17:13.088799+0000","last_change":"2024-09-12T03:17:11.775892+0000","last_active":"2024-09-12T03:17:13.088799+0000","last_peered":"2024-09-12T03:17:13.088799+0000","last_clean":"2024-09-12T03:17:13.088799+0000","last_became_active":"2024-09-12T03:17:11.775609+0000","last_became_peered":"2024-09-12T03:17:11.775609+0000","last_unstale":"2024-09-12T03:17:13.088799+0000","last_undegraded":"2024-09-12T03:17:13.088799+0000","last_fullsized":"2024-09-12T03:17:13.088799+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_clean_scrub_stamp":"2024-09-12T03:17:10.750733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:11:36.377619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T03:17:12.774702+0000","last_change":"2024-09-12T03:17:10.774314+0000","last_active":"2024-09-12T03:17:12.774702+0000","last_peered":"2024-09-12T03:17:12.774702+0000","last_clean":"2024-09-12T03:17:12.774702+0000","last_became_active":"2024-09-12T03:17:10.774018+0000","last_became_peered":"2024-09-12T03:17:10.774018+0000","last_unstale":"2024-09-12T03:17:12.774702+0000","last_undegraded":"2024-09-12T03:17:12.774702+0000","last_fullsized":"2024-09-12T03:17:12.774702+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:09.744809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:09.744809+0000","last_clean_scrub_stamp":"2024-09-12T03:17:09.744809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:23:09.520304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T03:17:12.774706+0000","last_change":"2024-09-12T03:17:11.776193+0000","last_active":"2024-09-12T03:17:12.774706+0000","last_peered":"2024-09-12T03:17:12.774706+0000","last_clean":"2024-09-12T03:17:12.774706+0000","last_became_active":"2024-09-12T03:17:11.775900+0000","last_became_peered":"2024-09-12T03:17:11.775900+0000","last_unstale":"2024-09-12T03:17:12.774706+0000","last_undegraded":"2024-09-12T03:17:12.774706+0000","last_fullsized":"2024-09-12T03:17:12.774706+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_clean_scrub_stamp":"2024-09-12T03:17:10.750733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:19:57.013239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T03:17:13.089648+0000","last_change":"2024-09-12T03:17:10.776167+0000","last_active":"2024-09-12T03:17:13.089648+0000","last_peered":"2024-09-12T03:17:13.089648+0000","last_clean":"2024-09-12T03:17:13.089648+0000","last_became_active":"2024-09-12T03:17:10.775588+0000","last_became_peered":"2024-09-12T03:17:10.775588+0000","last_unstale":"2024-09-12T03:17:13.089648+0000","last_undegraded":"2024-09-12T03:17:13.089648+0000","last_fullsized":"2024-09-12T03:17:13.089648+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:09.744809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:09.744809+0000","last_clean_scrub_stamp":"2024-09-12T03:17:09.744809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:19:14.924865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T03:17:13.088820+0000","last_change":"2024-09-12T03:17:11.785308+0000","last_active":"2024-09-12T03:17:13.088820+0000","last_peered":"2024-09-12T03:17:13.088820+0000","last_clean":"2024-09-12T03:17:13.088820+0000","last_became_active":"2024-09-12T03:17:11.785002+0000","last_became_peered":"2024-09-12T03:17:11.785002+0000","last_unstale":"2024-09-12T03:17:13.088820+0000","last_undegraded":"2024-09-12T03:17:13.088820+0000","last_fullsized":"2024-09-12T03:17:13.088820+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_clean_scrub_stamp":"2024-09-12T03:17:10.750733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:23:01.003978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T03:17:12.775324+0000","last_change":"2024-09-12T03:17:10.771678+0000","last_active":"2024-09-12T03:17:12.775324+0000","last_peered":"2024-09-12T03:17:12.775324+0000","last_clean":"2024-09-12T03:17:12.775324+0000","last_became_active":"2024-09-12T03:17:10.771286+0000","last_became_peered":"2024-09-12T03:17:10.771286+0000","last_unstale":"2024-09-12T03:17:12.775324+0000","last_undegraded":"2024-09-12T03:17:12.775324+0000","last_fullsized":"2024-09-12T03:17:12.775324+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:09.744809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:09.744809+0000","last_clean_scrub_stamp":"2024-09-12T03:17:09.744809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:54:33.759523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T03:17:12.775160+0000","last_change":"2024-09-12T03:17:11.779416+0000","last_active":"2024-09-12T03:17:12.775160+0000","last_peered":"2024-09-12T03:17:12.775160+0000","last_clean":"2024-09-12T03:17:12.775160+0000","last_became_active":"2024-09-12T03:17:11.779155+0000","last_became_peered":"2024-09-12T03:17:11.779155+0000","last_unstale":"2024-09-12T03:17:12.775160+0000","last_undegraded":"2024-09-12T03:17:12.775160+0000","last_fullsized":"2024-09-12T03:17:12.775160+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_clean_scrub_stamp":"2024-09-12T03:17:10.750733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:50:49.293495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T03:17:13.088976+0000","last_change":"2024-09-12T03:17:10.781212+0000","last_active":"2024-09-12T03:17:13.088976+0000","last_peered":"2024-09-12T03:17:13.088976+0000","last_clean":"2024-09-12T03:17:13.088976+0000","last_became_active":"2024-09-12T03:17:10.780937+0000","last_became_peered":"2024-09-12T03:17:10.780937+0000","last_unstale":"2024-09-12T03:17:13.088976+0000","last_undegraded":"2024-09-12T03:17:13.088976+0000","last_fullsized":"2024-09-12T03:17:13.088976+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:09.744809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:09.744809+0000","last_clean_scrub_stamp":"2024-09-12T03:17:09.744809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:34:10.886978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T03:17:13.089132+0000","last_change":"2024-09-12T03:17:11.778410+0000","last_active":"2024-09-12T03:17:13.089132+0000","last_peered":"2024-09-12T03:17:13.089132+0000","last_clean":"2024-09-12T03:17:13.089132+0000","last_became_active":"2024-09-12T03:17:11.778088+0000","last_became_peered":"2024-09-12T03:17:11.778088+0000","last_unstale":"2024-09-12T03:17:13.089132+0000","last_undegraded":"2024-09-12T03:17:13.089132+0000","last_fullsized":"2024-09-12T03:17:13.089132+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:10.750733+0000","last_clean_scrub_stamp":"2024-09-12T03:17:10.750733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:10:52.379296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T03:17:13.088576+0000","last_change":"2024-09-12T03:17:10.774097+0000","last_active":"2024-09-12T03:17:13.088576+0000","last_peered":"2024-09-12T03:17:13.088576+0000","last_clean":"2024-09-12T03:17:13.088576+0000","last_became_active":"2024-09-12T03:17:10.773773+0000","last_became_peered":"2024-09-12T03:17:10.773773+0000","last_unstale":"2024-09-12T03:17:13.088576+0000","last_undegraded":"2024-09-12T03:17:13.088576+0000","last_fullsized":"2024-09-12T03:17:13.088576+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:09.744809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:09.744809+0000","last_clean_scrub_stamp":"2024-09-12T03:17:09.744809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:54:33.759523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":188,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T03:17:13.089457+0000","last_change":"2024-09-12T03:16:21.883446+0000","last_active":"2024-09-12T03:17:13.089457+0000","last_peered":"2024-09-12T03:17:13.089457+0000","last_clean":"2024-09-12T03:17:13.089457+0000","last_became_active":"2024-09-12T03:16:21.882207+0000","last_became_peered":"2024-09-12T03:16:21.882207+0000","last_unstale":"2024-09-12T03:17:13.089457+0000","last_undegraded":"2024-09-12T03:17:13.089457+0000","last_fullsized":"2024-09-12T03:17:13.089457+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:20.316426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:20.316426+0000","last_clean_scrub_stamp":"2024-09-12T03:16:20.316426+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:12:15.424320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542157,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32120,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339720,"statfs":{"total":96636764160,"available":96603873280,"internally_reserved":0,"allocated":954368,"data_stored":727813,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542159,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31548,"kb_used_data":356,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340292,"statfs":{"total":96636764160,"available":96604459008,"internally_reserved":0,"allocated":364544,"data_stored":137971,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542158,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31552,"kb_used_data":356,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340288,"statfs":{"total":96636764160,"available":96604454912,"internally_reserved":0,"allocated":364544,"data_stored":138027,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542157,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31548,"kb_used_data":356,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340292,"statfs":{"total":96636764160,"available":96604459008,"internally_reserved":0,"allocated":364544,"data_stored":137459,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542158,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27396,"kb_used_data":364,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344444,"statfs":{"total":96636764160,"available":96608710656,"internally_reserved":0,"allocated":372736,"data_stored":138989,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542158,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27380,"kb_used_data":352,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344460,"statfs":{"total":96636764160,"available":96608727040,"internally_reserved":0,"allocated":360448,"data_stored":137425,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542159,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27400,"kb_used_data":364,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344440,"statfs":{"total":96636764160,"available":96608706560,"internally_reserved":0,"allocated":372736,"data_stored":138989,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542159,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27960,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343880,"statfs":{"total":96636764160,"available":96608133120,"internally_reserved":0,"allocated":950272,"data_stored":727723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T03:17:16.105 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-12T03:17:16.105 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-12T03:17:16.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:16.250+0000 7fdec568d640 1 Processor -- start 2024-09-12T03:17:16.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:16.250+0000 7fdec568d640 1 -- start start 2024-09-12T03:17:16.253 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:16.251+0000 7fdec568d640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fdec015a270 0x7fdec006cff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:16.253 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:16.251+0000 7fdec568d640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdec00ce650 con 0x7fdec006d530 2024-09-12T03:17:16.253 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:16.251+0000 7fdec568d640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdec00ce930 con 0x7fdec00cfb20 2024-09-12T03:17:16.253 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:16.251+0000 7fdec568d640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fdec00cef80 con 0x7fdec015a270 2024-09-12T03:17:16.254 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:16.251+0000 7fdebe7fc640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fdec006d530 0x7fdec00cf410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:42574/0 (socket says 172.21.15.27:42574) 2024-09-12T03:17:16.254 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:16.251+0000 7fdebe7fc640 1 -- 172.21.15.27:0/475310656 learned_addr learned my addr 172.21.15.27:0/475310656 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:17:16.254 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:16.251+0000 7fdebeffd640 1 --2- 172.21.15.27:0/475310656 >> v2:172.21.15.45:3300/0 conn(0x7fdec015a270 0x7fdec006cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:16.254 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:16.251+0000 7fdebdffb640 1 -- 172.21.15.27:0/475310656 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3779433420 0 0) 0x7fdec00ce650 con 0x7fdec006d530 2024-09-12T03:17:16.254 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:16.251+0000 7fdebdffb640 1 -- 172.21.15.27:0/475310656 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdea8003520 con 0x7fdec006d530 2024-09-12T03:17:16.254 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:16.252+0000 7fdebdffb640 1 -- 172.21.15.27:0/475310656 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3097929379 0 0) 0x7fdea8003520 con 0x7fdec006d530 2024-09-12T03:17:16.254 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:16.252+0000 7fdebdffb640 1 -- 172.21.15.27:0/475310656 >> v2:172.21.15.45:3300/0 conn(0x7fdec015a270 msgr2=0x7fdec006cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:16.254 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:16.252+0000 7fdebdffb640 1 --2- 172.21.15.27:0/475310656 >> v2:172.21.15.45:3300/0 conn(0x7fdec015a270 0x7fdec006cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:16.254 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:16.252+0000 7fdebdffb640 1 -- 172.21.15.27:0/475310656 >> v1:172.21.15.45:6789/0 conn(0x7fdec00cfb20 legacy=0x7fdec00726a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:16.254 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:16.252+0000 7fdebdffb640 1 -- 172.21.15.27:0/475310656 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdec00cec10 con 0x7fdec006d530 2024-09-12T03:17:16.254 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:16.252+0000 7fdebdffb640 1 -- 172.21.15.27:0/475310656 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 847755213 0 0) 0x7fdeb4001670 con 0x7fdec006d530 2024-09-12T03:17:16.255 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:16.252+0000 7fdebeffd640 1 --2- 172.21.15.27:0/475310656 >> v2:172.21.15.45:3300/0 conn(0x7fdec015a270 0x7fdec006cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:17:16.255 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:16.252+0000 7fdebdffb640 1 -- 172.21.15.27:0/475310656 >> v1:172.21.15.27:6789/0 conn(0x7fdec006d530 legacy=0x7fdec00cf410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:16.255 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:16.252+0000 7fdebdffb640 1 --2- 172.21.15.27:0/475310656 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fdea80039b0 0x7fdea8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:16.255 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:16.252+0000 7fdebdffb640 1 --2- 172.21.15.27:0/475310656 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdea8004800 0x7fdea8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:16.255 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:16.252+0000 7fdebdffb640 1 --2- 172.21.15.27:0/475310656 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdea80051c0 0x7fdea800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:16.255 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:16.252+0000 7fdebdffb640 1 -- 172.21.15.27:0/475310656 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fdec00cef80 con 0x7fdea80051c0 2024-09-12T03:17:16.255 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:16.252+0000 7fdebdffb640 1 -- 172.21.15.27:0/475310656 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fdea800c060 con 0x7fdea8004800 2024-09-12T03:17:16.255 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:16.252+0000 7fdebdffb640 1 -- 172.21.15.27:0/475310656 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fdea800c260 con 0x7fdea80039b0 2024-09-12T03:17:16.255 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:16.252+0000 7fdebe7fc640 1 --2- 172.21.15.27:0/475310656 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdea8004800 0x7fdea8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:16.255 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:16.252+0000 7fdebf7fe640 1 --2- 172.21.15.27:0/475310656 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdea80051c0 0x7fdea800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:16.255 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:16.252+0000 7fdebeffd640 1 --2- 172.21.15.27:0/475310656 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fdea80039b0 0x7fdea8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:16.256 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:16.252+0000 7fdebf7fe640 1 -- 172.21.15.27:0/475310656 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fdea80039b0 msgr2=0x7fdea8003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:16.256 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:16.252+0000 7fdebf7fe640 1 --2- 172.21.15.27:0/475310656 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fdea80039b0 0x7fdea8003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:16.256 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:16.252+0000 7fdebf7fe640 1 -- 172.21.15.27:0/475310656 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdea8004800 msgr2=0x7fdea8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:16.256 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:16.252+0000 7fdebf7fe640 1 --2- 172.21.15.27:0/475310656 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdea8004800 0x7fdea8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:16.256 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:16.252+0000 7fdebf7fe640 1 -- 172.21.15.27:0/475310656 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdea8003520 con 0x7fdea80051c0 2024-09-12T03:17:16.256 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:16.253+0000 7fdebf7fe640 1 -- 172.21.15.27:0/475310656 reap_dead start 2024-09-12T03:17:16.256 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:16.253+0000 7fdebf7fe640 1 --2- 172.21.15.27:0/475310656 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdea80051c0 0x7fdea800b9c0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fdec00cff50 tx=0x7fdea8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e609386136a9452 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:16.256 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:16.253+0000 7fdebdffb640 1 -- 172.21.15.27:0/475310656 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdeb8001a70 con 0x7fdea80051c0 2024-09-12T03:17:16.256 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:16.253+0000 7fdebdffb640 1 -- 172.21.15.27:0/475310656 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdeb802ae10 con 0x7fdea80051c0 2024-09-12T03:17:16.256 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:16.253+0000 7fdebdffb640 1 -- 172.21.15.27:0/475310656 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdeb8003bb0 con 0x7fdea80051c0 2024-09-12T03:17:16.260 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:16.254+0000 7fdec568d640 1 -- 172.21.15.27:0/475310656 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdea80051c0 msgr2=0x7fdea800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:16.260 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:16.254+0000 7fdec568d640 1 --2- 172.21.15.27:0/475310656 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdea80051c0 0x7fdea800b9c0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fdec00cff50 tx=0x7fdea8005830 comp rx=0 tx=0).stop 2024-09-12T03:17:16.260 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:16.254+0000 7fdec568d640 1 -- 172.21.15.27:0/475310656 shutdown_connections 2024-09-12T03:17:16.260 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:16.254+0000 7fdec568d640 1 --2- 172.21.15.27:0/475310656 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdea80051c0 0x7fdea800b9c0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:16.260 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:16.254+0000 7fdec568d640 1 -- 172.21.15.27:0/475310656 >> 172.21.15.27:0/475310656 conn(0x7fdec006bb30 msgr2=0x7fdec006bf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:16.260 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:16.254+0000 7fdec568d640 1 -- 172.21.15.27:0/475310656 shutdown_connections 2024-09-12T03:17:16.260 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:16.254+0000 7fdec568d640 1 -- 172.21.15.27:0/475310656 wait complete. 2024-09-12T03:17:16.261 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:16.255+0000 7fdec568d640 1 Processor -- start 2024-09-12T03:17:16.261 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:16.255+0000 7fdec568d640 1 -- start start 2024-09-12T03:17:16.261 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:16.256+0000 7fdec568d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdea80051c0 0x7fdec00c6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:16.261 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:16.256+0000 7fdec568d640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fdec014ce40 0x7fdec014d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:16.261 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:16.256+0000 7fdec568d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdec00b4cc0 0x7fdec00b5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:16.261 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:16.256+0000 7fdec568d640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fdec006c7d0 con 0x7fdec00b4cc0 2024-09-12T03:17:16.261 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:16.256+0000 7fdec568d640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fdec006c650 con 0x7fdea80051c0 2024-09-12T03:17:16.261 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:16.256+0000 7fdec568d640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fdec006c950 con 0x7fdec014ce40 2024-09-12T03:17:16.261 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:16.256+0000 7fdebf7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdec00b4cc0 0x7fdec00b5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:16.261 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:16.256+0000 7fdebf7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdec00b4cc0 0x7fdec00b5140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37002/0 (socket says 172.21.15.27:37002) 2024-09-12T03:17:16.262 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:16.256+0000 7fdebf7fe640 1 -- 172.21.15.27:0/3894589117 learned_addr learned my addr 172.21.15.27:0/3894589117 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:17:16.262 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:16.256+0000 7fdebe7fc640 1 --2- 172.21.15.27:0/3894589117 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fdec014ce40 0x7fdec014d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:16.262 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:16.256+0000 7fdebeffd640 1 --2- 172.21.15.27:0/3894589117 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdea80051c0 0x7fdec00c6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:16.262 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:16.256+0000 7fdebf7fe640 1 -- 172.21.15.27:0/3894589117 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fdec014ce40 msgr2=0x7fdec014d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:16.262 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:16.256+0000 7fdebf7fe640 1 --2- 172.21.15.27:0/3894589117 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fdec014ce40 0x7fdec014d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:16.262 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:16.256+0000 7fdebf7fe640 1 -- 172.21.15.27:0/3894589117 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdea80051c0 msgr2=0x7fdec00c6cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:16.262 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:16.256+0000 7fdebf7fe640 1 --2- 172.21.15.27:0/3894589117 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdea80051c0 0x7fdec00c6cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:16.262 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:16.257+0000 7fdebf7fe640 1 -- 172.21.15.27:0/3894589117 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdec00c73c0 con 0x7fdec00b4cc0 2024-09-12T03:17:16.262 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:16.257+0000 7fdebf7fe640 1 --2- 172.21.15.27:0/3894589117 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdec00b4cc0 0x7fdec00b5140 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fdeb8001ee0 tx=0x7fdeb80014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:16.262 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:16.257+0000 7fdea77fe640 1 -- 172.21.15.27:0/3894589117 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdeb8007db0 con 0x7fdec00b4cc0 2024-09-12T03:17:16.262 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:16.257+0000 7fdea77fe640 1 -- 172.21.15.27:0/3894589117 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdeb802ac10 con 0x7fdec00b4cc0 2024-09-12T03:17:16.262 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:16.257+0000 7fdec568d640 1 -- 172.21.15.27:0/3894589117 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdec00c76a0 con 0x7fdec00b4cc0 2024-09-12T03:17:16.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:16.257+0000 7fdea77fe640 1 -- 172.21.15.27:0/3894589117 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdeb80053b0 con 0x7fdec00b4cc0 2024-09-12T03:17:16.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:16.257+0000 7fdec568d640 1 -- 172.21.15.27:0/3894589117 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdec00c5950 con 0x7fdec00b4cc0 2024-09-12T03:17:16.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:16.258+0000 7fdec568d640 1 -- 172.21.15.27:0/3894589117 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdec00b3100 con 0x7fdec00b4cc0 2024-09-12T03:17:16.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:16.258+0000 7fdea77fe640 1 -- 172.21.15.27:0/3894589117 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fdeb8005550 con 0x7fdec00b4cc0 2024-09-12T03:17:16.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:16.259+0000 7fdea77fe640 1 --2- 172.21.15.27:0/3894589117 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fdeac06be30 0x7fdeac06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:16.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:16.259+0000 7fdea77fe640 1 -- 172.21.15.27:0/3894589117 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7fdeb80aa300 con 0x7fdec00b4cc0 2024-09-12T03:17:16.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:16.259+0000 7fdebeffd640 1 --2- 172.21.15.27:0/3894589117 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fdeac06be30 0x7fdeac06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:16.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:16.260+0000 7fdebeffd640 1 --2- 172.21.15.27:0/3894589117 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fdeac06be30 0x7fdeac06e2f0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fdeb00093d0 tx=0x7fdeb0009190 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:16.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:16.263+0000 7fdea77fe640 1 -- 172.21.15.27:0/3894589117 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fdec00b3100 con 0x7fdec00b4cc0 2024-09-12T03:17:16.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:16.503+0000 7fdec568d640 1 -- 172.21.15.27:0/3894589117 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7fdec00c5b50 con 0x7fdec00b4cc0 2024-09-12T03:17:16.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:16.505+0000 7fdea77fe640 1 -- 172.21.15.27:0/3894589117 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v11) ==== 190+0+147 (secure 0 0 0) 0x7fdec00c5b50 con 0x7fdec00b4cc0 2024-09-12T03:17:16.506 INFO:teuthology.orchestra.run.smithi027.stdout:[client.0] 2024-09-12T03:17:16.506 INFO:teuthology.orchestra.run.smithi027.stdout: key = AQDVXOJmLm+5ExAAECM4CKfuVElmp/n0b02mXg== 2024-09-12T03:17:16.506 INFO:teuthology.orchestra.run.smithi027.stdout: caps mds = "allow" 2024-09-12T03:17:16.506 INFO:teuthology.orchestra.run.smithi027.stdout: caps mon = "allow r" 2024-09-12T03:17:16.506 INFO:teuthology.orchestra.run.smithi027.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-12T03:17:16.506 INFO:teuthology.orchestra.run.smithi027.stderr:updated caps for client.0 2024-09-12T03:17:16.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:16.507+0000 7fdec568d640 1 -- 172.21.15.27:0/3894589117 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fdeac06be30 msgr2=0x7fdeac06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:16.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:16.507+0000 7fdec568d640 1 --2- 172.21.15.27:0/3894589117 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fdeac06be30 0x7fdeac06e2f0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fdeb00093d0 tx=0x7fdeb0009190 comp rx=0 tx=0).stop 2024-09-12T03:17:16.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:16.507+0000 7fdec568d640 1 -- 172.21.15.27:0/3894589117 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdec00b4cc0 msgr2=0x7fdec00b5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:16.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:16.507+0000 7fdec568d640 1 --2- 172.21.15.27:0/3894589117 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdec00b4cc0 0x7fdec00b5140 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fdeb8001ee0 tx=0x7fdeb80014a0 comp rx=0 tx=0).stop 2024-09-12T03:17:16.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:16.508+0000 7fdec568d640 1 -- 172.21.15.27:0/3894589117 shutdown_connections 2024-09-12T03:17:16.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:16.508+0000 7fdec568d640 1 --2- 172.21.15.27:0/3894589117 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fdeac06be30 0x7fdeac06e2f0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:16.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:16.508+0000 7fdec568d640 1 --2- 172.21.15.27:0/3894589117 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdec00b4cc0 0x7fdec00b5140 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:16.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:16.508+0000 7fdec568d640 1 --2- 172.21.15.27:0/3894589117 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fdec014ce40 0x7fdec014d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:16.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:16.508+0000 7fdec568d640 1 --2- 172.21.15.27:0/3894589117 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdea80051c0 0x7fdec00c6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:16.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:16.508+0000 7fdec568d640 1 -- 172.21.15.27:0/3894589117 >> 172.21.15.27:0/3894589117 conn(0x7fdec006bb30 msgr2=0x7fdec006bf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:16.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:16.508+0000 7fdec568d640 1 -- 172.21.15.27:0/3894589117 shutdown_connections 2024-09-12T03:17:16.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:16.508+0000 7fdec568d640 1 -- 172.21.15.27:0/3894589117 wait complete. 2024-09-12T03:17:16.519 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-12T03:17:16.621 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:16.621+0000 7f639f78d640 1 Processor -- start 2024-09-12T03:17:16.621 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:16.621+0000 7f639f78d640 1 -- start start 2024-09-12T03:17:16.622 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:16.622+0000 7f639f78d640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f639815e780 0x7f63980d3e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:16.622 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:16.622+0000 7f639f78d640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f63980d1990 0x7f63980b3390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:16.622 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:16.622+0000 7f639f78d640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f63980b6690 con 0x7f63980d43b0 2024-09-12T03:17:16.622 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:16.622+0000 7f639f78d640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f63980b7950 con 0x7f63980d1990 2024-09-12T03:17:16.622 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:16.622+0000 7f639f78d640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f63980b7210 con 0x7f639815e780 2024-09-12T03:17:16.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:16.622+0000 7f639dd2a640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f63980d1990 0x7f63980b3390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:16.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:16.622+0000 7f639dd2a640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f63980d1990 0x7f63980b3390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37014/0 (socket says 172.21.15.27:37014) 2024-09-12T03:17:16.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:16.622+0000 7f639dd2a640 1 -- 172.21.15.27:0/3303169226 learned_addr learned my addr 172.21.15.27:0/3303169226 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:17:16.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:16.622+0000 7f639d529640 1 --2- 172.21.15.27:0/3303169226 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f639815e780 0x7f63980d3e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:16.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:16.622+0000 7f639dd2a640 1 -- 172.21.15.27:0/3303169226 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f639815e780 msgr2=0x7f63980d3e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:16.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:16.622+0000 7f639dd2a640 1 --2- 172.21.15.27:0/3303169226 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f639815e780 0x7f63980d3e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:16.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:16.622+0000 7f639dd2a640 1 -- 172.21.15.27:0/3303169226 >> v1:172.21.15.45:6789/0 conn(0x7f63980d43b0 legacy=0x7f63980d1280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:16.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:16.622+0000 7f639dd2a640 1 -- 172.21.15.27:0/3303169226 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63980b6970 con 0x7f63980d1990 2024-09-12T03:17:16.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:16.623+0000 7f639dd2a640 1 --2- 172.21.15.27:0/3303169226 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f63980d1990 0x7f63980b3390 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f6390009e50 tx=0x7f639002a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fce3ef073d37c435 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:16.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:16.623+0000 7f638ffff640 1 -- 172.21.15.27:0/3303169226 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6390036070 con 0x7f63980d1990 2024-09-12T03:17:16.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:16.623+0000 7f638ffff640 1 -- 172.21.15.27:0/3303169226 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6390031600 con 0x7f63980d1990 2024-09-12T03:17:16.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:16.623+0000 7f638ffff640 1 -- 172.21.15.27:0/3303169226 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63900319d0 con 0x7f63980d1990 2024-09-12T03:17:16.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:16.623+0000 7f639f78d640 1 -- 172.21.15.27:0/3303169226 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f63980d1990 msgr2=0x7f63980b3390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:16.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:16.623+0000 7f639f78d640 1 --2- 172.21.15.27:0/3303169226 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f63980d1990 0x7f63980b3390 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f6390009e50 tx=0x7f639002a710 comp rx=0 tx=0).stop 2024-09-12T03:17:16.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:16.624+0000 7f639f78d640 1 -- 172.21.15.27:0/3303169226 shutdown_connections 2024-09-12T03:17:16.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:16.624+0000 7f639f78d640 1 --2- 172.21.15.27:0/3303169226 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f63980d1990 0x7f63980b3390 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:16.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:16.624+0000 7f639f78d640 1 --2- 172.21.15.27:0/3303169226 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f639815e780 0x7f63980d3e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:16.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:16.624+0000 7f639f78d640 1 -- 172.21.15.27:0/3303169226 >> 172.21.15.27:0/3303169226 conn(0x7f639806da60 msgr2=0x7f639806de70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:16.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:16.624+0000 7f639f78d640 1 -- 172.21.15.27:0/3303169226 shutdown_connections 2024-09-12T03:17:16.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:16.624+0000 7f639f78d640 1 -- 172.21.15.27:0/3303169226 wait complete. 2024-09-12T03:17:16.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:16.625+0000 7f639f78d640 1 Processor -- start 2024-09-12T03:17:16.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:16.625+0000 7f639f78d640 1 -- start start 2024-09-12T03:17:16.628 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:16.626+0000 7f639f78d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f63980d1990 0x7f63981f8610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:16.628 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:16.626+0000 7f639f78d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f63980d43b0 0x7f639814d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:16.628 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:16.626+0000 7f639f78d640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f639815e780 0x7f639814fe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:16.628 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:16.626+0000 7f639f78d640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f63980b6340 con 0x7f63980d1990 2024-09-12T03:17:16.628 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:16.626+0000 7f639f78d640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f63980b6f30 con 0x7f63980d43b0 2024-09-12T03:17:16.628 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:16.626+0000 7f639f78d640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f639815eb60 con 0x7f639815e780 2024-09-12T03:17:16.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:16.626+0000 7f639d529640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f63980d1990 0x7f63981f8610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:16.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:16.626+0000 7f639d529640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f63980d1990 0x7f63981f8610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37020/0 (socket says 172.21.15.27:37020) 2024-09-12T03:17:16.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:16.626+0000 7f639d529640 1 -- 172.21.15.27:0/2071870716 learned_addr learned my addr 172.21.15.27:0/2071870716 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:17:16.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:16.626+0000 7f639dd2a640 1 --2- 172.21.15.27:0/2071870716 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f639815e780 0x7f639814fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:16.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:16.626+0000 7f639cd28640 1 --2- 172.21.15.27:0/2071870716 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f63980d43b0 0x7f639814d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:16.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:16.626+0000 7f639d529640 1 -- 172.21.15.27:0/2071870716 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f639815e780 msgr2=0x7f639814fe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:16.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:16.626+0000 7f639d529640 1 --2- 172.21.15.27:0/2071870716 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f639815e780 0x7f639814fe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:16.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:16.626+0000 7f639d529640 1 -- 172.21.15.27:0/2071870716 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f63980d43b0 msgr2=0x7f639814d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:16.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:16.626+0000 7f639d529640 1 --2- 172.21.15.27:0/2071870716 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f63980d43b0 0x7f639814d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:16.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:16.626+0000 7f639d529640 1 -- 172.21.15.27:0/2071870716 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63981503c0 con 0x7f63980d1990 2024-09-12T03:17:16.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:16.626+0000 7f639d529640 1 --2- 172.21.15.27:0/2071870716 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f63980d1990 0x7f63981f8610 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f639400e870 tx=0x7f639400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:16.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:16.626+0000 7f638dffb640 1 -- 172.21.15.27:0/2071870716 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6394017070 con 0x7f63980d1990 2024-09-12T03:17:16.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:16.626+0000 7f638dffb640 1 -- 172.21.15.27:0/2071870716 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f639400c940 con 0x7f63980d1990 2024-09-12T03:17:16.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:16.626+0000 7f639f78d640 1 -- 172.21.15.27:0/2071870716 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6398150690 con 0x7f63980d1990 2024-09-12T03:17:16.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:16.626+0000 7f638dffb640 1 -- 172.21.15.27:0/2071870716 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f639400cd30 con 0x7f63980d1990 2024-09-12T03:17:16.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:16.627+0000 7f639f78d640 1 -- 172.21.15.27:0/2071870716 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6398150c50 con 0x7f63980d1990 2024-09-12T03:17:16.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:16.627+0000 7f639f78d640 1 -- 172.21.15.27:0/2071870716 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63980d0cc0 con 0x7f63980d1990 2024-09-12T03:17:16.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:16.628+0000 7f638dffb640 1 -- 172.21.15.27:0/2071870716 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f639401a070 con 0x7f63980d1990 2024-09-12T03:17:16.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:16.628+0000 7f638dffb640 1 --2- 172.21.15.27:0/2071870716 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f637c06be60 0x7f637c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:16.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:16.629+0000 7f638dffb640 1 -- 172.21.15.27:0/2071870716 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f639408ac30 con 0x7f63980d1990 2024-09-12T03:17:16.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:16.629+0000 7f639cd28640 1 --2- 172.21.15.27:0/2071870716 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f637c06be60 0x7f637c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:16.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:16.629+0000 7f639cd28640 1 --2- 172.21.15.27:0/2071870716 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f637c06be60 0x7f637c06e320 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f639806e220 tx=0x7f6388009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:16.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:16.633+0000 7f638dffb640 1 -- 172.21.15.27:0/2071870716 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f63980d0cc0 con 0x7f63980d1990 2024-09-12T03:17:16.745 INFO:tasks.ceph.mds.a.smithi027.stderr:2024-09-12T03:17:16.744+0000 7f12f22e3640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-12T03:17:16.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:16.844+0000 7f639f78d640 1 -- 172.21.15.27:0/2071870716 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f63980d3e70 con 0x7f63980d1990 2024-09-12T03:17:16.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:16.844+0000 7f638dffb640 1 -- 172.21.15.27:0/2071870716 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) ==== 94+0+5222 (secure 0 0 0) 0x7f6394058f90 con 0x7f63980d1990 2024-09-12T03:17:16.845 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:17:16.845 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":14,"btime":"2024-09-12T03:17:14:774103+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4592,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6836/3382368386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":3382368386},{"type":"v1","addr":"172.21.15.45:6836","nonce":3382368386}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":4595,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/2127097700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6835","nonce":2127097700},{"type":"v1","addr":"172.21.15.45:6837","nonce":2127097700}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":4815,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6834/2623734554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":2623734554},{"type":"v1","addr":"172.21.15.27:6834","nonce":2623734554}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-12T03:17:11.119994+0000","modified":"2024-09-12T03:17:13.929574+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4818},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4818":{"gid":4818,"name":"a","rank":0,"incarnation":11,"state":"up:active","state_seq":3,"addr":"172.21.15.27:6835/3533575930","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":3533575930},{"type":"v1","addr":"172.21.15.27:6835","nonce":3533575930}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4818,"qdb_cluster":[4818]},"id":2}]} 2024-09-12T03:17:16.846 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 14 2024-09-12T03:17:16.847 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:16.846+0000 7f639f78d640 1 -- 172.21.15.27:0/2071870716 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f637c06be60 msgr2=0x7f637c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:16.847 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:16.846+0000 7f639f78d640 1 --2- 172.21.15.27:0/2071870716 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f637c06be60 0x7f637c06e320 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f639806e220 tx=0x7f6388009040 comp rx=0 tx=0).stop 2024-09-12T03:17:16.847 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:16.846+0000 7f639f78d640 1 -- 172.21.15.27:0/2071870716 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f63980d1990 msgr2=0x7f63981f8610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:16.847 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:16.846+0000 7f639f78d640 1 --2- 172.21.15.27:0/2071870716 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f63980d1990 0x7f63981f8610 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f639400e870 tx=0x7f639400ed30 comp rx=0 tx=0).stop 2024-09-12T03:17:16.847 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:16.847+0000 7f639f78d640 1 -- 172.21.15.27:0/2071870716 shutdown_connections 2024-09-12T03:17:16.847 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:16.847+0000 7f639f78d640 1 --2- 172.21.15.27:0/2071870716 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f637c06be60 0x7f637c06e320 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:16.847 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:16.847+0000 7f639f78d640 1 --2- 172.21.15.27:0/2071870716 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f639815e780 0x7f639814fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:16.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:16.847+0000 7f639f78d640 1 --2- 172.21.15.27:0/2071870716 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f63980d43b0 0x7f639814d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:16.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:16.847+0000 7f639f78d640 1 --2- 172.21.15.27:0/2071870716 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f63980d1990 0x7f63981f8610 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:16.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:16.847+0000 7f639f78d640 1 -- 172.21.15.27:0/2071870716 >> 172.21.15.27:0/2071870716 conn(0x7f639806da60 msgr2=0x7f639815fbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:16.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:16.847+0000 7f639f78d640 1 -- 172.21.15.27:0/2071870716 shutdown_connections 2024-09-12T03:17:16.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:16.847+0000 7f639f78d640 1 -- 172.21.15.27:0/2071870716 wait complete. 2024-09-12T03:17:17.260 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 14, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-12T03:17:11.119994+0000', 'modified': '2024-09-12T03:17:13.929574+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4818}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4818': {'gid': 4818, 'name': 'a', 'rank': 0, 'incarnation': 11, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.27:6835/3533575930', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6833', 'nonce': 3533575930}, {'type': 'v1', 'addr': '172.21.15.27:6835', 'nonce': 3533575930}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4818, 'qdb_cluster': [4818]} 2024-09-12T03:17:17.260 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-12T03:17:17.260 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-09-12T03:17:17.366 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.366+0000 7f9763b03640 1 Processor -- start 2024-09-12T03:17:17.366 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.366+0000 7f9763b03640 1 -- start start 2024-09-12T03:17:17.371 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.367+0000 7f9763b03640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f975c0cf660 0x7f975c0d2780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:17.371 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.367+0000 7f9763b03640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f975c06cc40 con 0x7f975c0b1ca0 2024-09-12T03:17:17.371 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.367+0000 7f9763b03640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f975c0d1ce0 con 0x7f975c06f360 2024-09-12T03:17:17.371 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.367+0000 7f9763b03640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f975c0d22f0 con 0x7f975c0cf660 2024-09-12T03:17:17.372 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.367+0000 7f976189f640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f975c0b1ca0 0x7f975c15f830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:42588/0 (socket says 172.21.15.27:42588) 2024-09-12T03:17:17.372 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.367+0000 7f976189f640 1 -- 172.21.15.27:0/4047789495 learned_addr learned my addr 172.21.15.27:0/4047789495 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:17:17.372 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.367+0000 7f976089d640 1 -- 172.21.15.27:0/4047789495 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3395000189 0 0) 0x7f975c06cc40 con 0x7f975c0b1ca0 2024-09-12T03:17:17.372 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.367+0000 7f976089d640 1 -- 172.21.15.27:0/4047789495 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9744003520 con 0x7f975c0b1ca0 2024-09-12T03:17:17.372 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.367+0000 7f97620a0640 1 --2- 172.21.15.27:0/4047789495 >> v2:172.21.15.45:3300/0 conn(0x7f975c0cf660 0x7f975c0d2780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:17.372 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.368+0000 7f976089d640 1 -- 172.21.15.27:0/4047789495 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 227626824 0 0) 0x7f9744003520 con 0x7f975c0b1ca0 2024-09-12T03:17:17.372 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.368+0000 7f976089d640 1 -- 172.21.15.27:0/4047789495 >> v2:172.21.15.45:3300/0 conn(0x7f975c0cf660 msgr2=0x7f975c0d2780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:17.372 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.368+0000 7f976089d640 1 --2- 172.21.15.27:0/4047789495 >> v2:172.21.15.45:3300/0 conn(0x7f975c0cf660 0x7f975c0d2780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:17.372 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.368+0000 7f976089d640 1 -- 172.21.15.27:0/4047789495 >> v1:172.21.15.45:6789/0 conn(0x7f975c06f360 legacy=0x7f975c0cef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:17.372 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.368+0000 7f976089d640 1 -- 172.21.15.27:0/4047789495 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f975c06cee0 con 0x7f975c0b1ca0 2024-09-12T03:17:17.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.368+0000 7f976089d640 1 -- 172.21.15.27:0/4047789495 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 847755213 0 0) 0x7f974c001670 con 0x7f975c0b1ca0 2024-09-12T03:17:17.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.368+0000 7f976089d640 1 -- 172.21.15.27:0/4047789495 >> v1:172.21.15.27:6789/0 conn(0x7f975c0b1ca0 legacy=0x7f975c15f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:17.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.368+0000 7f976089d640 1 --2- 172.21.15.27:0/4047789495 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f97440039b0 0x7f9744003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:17.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.368+0000 7f976089d640 1 --2- 172.21.15.27:0/4047789495 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f9744004800 0x7f9744004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:17.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.368+0000 7f976089d640 1 --2- 172.21.15.27:0/4047789495 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f97440051c0 0x7f974400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:17.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.368+0000 7f976089d640 1 -- 172.21.15.27:0/4047789495 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f975c0d22f0 con 0x7f97440051c0 2024-09-12T03:17:17.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.368+0000 7f976089d640 1 -- 172.21.15.27:0/4047789495 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f974400c110 con 0x7f97440039b0 2024-09-12T03:17:17.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.368+0000 7f976089d640 1 -- 172.21.15.27:0/4047789495 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f974400c310 con 0x7f9744004800 2024-09-12T03:17:17.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.368+0000 7f97620a0640 1 --2- 172.21.15.27:0/4047789495 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f97440051c0 0x7f974400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:17.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.368+0000 7f976189f640 1 --2- 172.21.15.27:0/4047789495 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f97440039b0 0x7f9744003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:17.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.368+0000 7f976109e640 1 --2- 172.21.15.27:0/4047789495 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f9744004800 0x7f9744004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:17.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.368+0000 7f97620a0640 1 -- 172.21.15.27:0/4047789495 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f9744004800 msgr2=0x7f9744004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:17.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.368+0000 7f97620a0640 1 --2- 172.21.15.27:0/4047789495 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f9744004800 0x7f9744004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:17.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.368+0000 7f97620a0640 1 -- 172.21.15.27:0/4047789495 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f97440039b0 msgr2=0x7f9744003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:17.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.368+0000 7f97620a0640 1 --2- 172.21.15.27:0/4047789495 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f97440039b0 0x7f9744003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:17.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.368+0000 7f97620a0640 1 -- 172.21.15.27:0/4047789495 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9744003520 con 0x7f97440051c0 2024-09-12T03:17:17.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.369+0000 7f97620a0640 1 -- 172.21.15.27:0/4047789495 reap_dead start 2024-09-12T03:17:17.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.369+0000 7f97620a0640 1 --2- 172.21.15.27:0/4047789495 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f97440051c0 0x7f974400b9c0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f975c069ba0 tx=0x7f9744005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efa1a1e5fa95466c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:17.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.369+0000 7f976089d640 1 -- 172.21.15.27:0/4047789495 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9754033070 con 0x7f97440051c0 2024-09-12T03:17:17.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.369+0000 7f976089d640 1 -- 172.21.15.27:0/4047789495 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f975402f650 con 0x7f97440051c0 2024-09-12T03:17:17.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.369+0000 7f976089d640 1 -- 172.21.15.27:0/4047789495 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9754009be0 con 0x7f97440051c0 2024-09-12T03:17:17.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.370+0000 7f9763b03640 1 -- 172.21.15.27:0/4047789495 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f97440051c0 msgr2=0x7f974400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:17.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.370+0000 7f9763b03640 1 --2- 172.21.15.27:0/4047789495 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f97440051c0 0x7f974400b9c0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f975c069ba0 tx=0x7f9744005830 comp rx=0 tx=0).stop 2024-09-12T03:17:17.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.370+0000 7f9763b03640 1 -- 172.21.15.27:0/4047789495 shutdown_connections 2024-09-12T03:17:17.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.370+0000 7f9763b03640 1 --2- 172.21.15.27:0/4047789495 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f97440051c0 0x7f974400b9c0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:17.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.370+0000 7f9763b03640 1 -- 172.21.15.27:0/4047789495 >> 172.21.15.27:0/4047789495 conn(0x7f975c073800 msgr2=0x7f975c09ba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:17.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.370+0000 7f9763b03640 1 -- 172.21.15.27:0/4047789495 shutdown_connections 2024-09-12T03:17:17.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.370+0000 7f9763b03640 1 -- 172.21.15.27:0/4047789495 wait complete. 2024-09-12T03:17:17.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.371+0000 7f9763b03640 1 Processor -- start 2024-09-12T03:17:17.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.371+0000 7f9763b03640 1 -- start start 2024-09-12T03:17:17.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.372+0000 7f9763b03640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f97440051c0 0x7f975c0b6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:17.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.372+0000 7f9763b03640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f975c0cbdb0 0x7f975c160c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:17.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.372+0000 7f9763b03640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f975c0cc230 0x7f975c1611a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:17.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.372+0000 7f9763b03640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f975c06d340 con 0x7f97440051c0 2024-09-12T03:17:17.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.372+0000 7f9763b03640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f975c06d1c0 con 0x7f975c0cc230 2024-09-12T03:17:17.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.372+0000 7f9763b03640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f975c0cfc40 con 0x7f975c0cbdb0 2024-09-12T03:17:17.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.372+0000 7f976189f640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f97440051c0 0x7f975c0b6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:17.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.372+0000 7f976189f640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f97440051c0 0x7f975c0b6070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37032/0 (socket says 172.21.15.27:37032) 2024-09-12T03:17:17.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.372+0000 7f976189f640 1 -- 172.21.15.27:0/3226171688 learned_addr learned my addr 172.21.15.27:0/3226171688 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:17:17.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.372+0000 7f97620a0640 1 --2- 172.21.15.27:0/3226171688 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f975c0cc230 0x7f975c1611a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:17.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.372+0000 7f976109e640 1 --2- 172.21.15.27:0/3226171688 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f975c0cbdb0 0x7f975c160c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:17.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.372+0000 7f976189f640 1 -- 172.21.15.27:0/3226171688 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f975c0cbdb0 msgr2=0x7f975c160c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:17.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.372+0000 7f976189f640 1 --2- 172.21.15.27:0/3226171688 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f975c0cbdb0 0x7f975c160c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:17.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.372+0000 7f976189f640 1 -- 172.21.15.27:0/3226171688 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f975c0cc230 msgr2=0x7f975c1611a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:17.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.372+0000 7f976189f640 1 --2- 172.21.15.27:0/3226171688 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f975c0cc230 0x7f975c1611a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:17.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.372+0000 7f976189f640 1 -- 172.21.15.27:0/3226171688 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f975c15fbc0 con 0x7f97440051c0 2024-09-12T03:17:17.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.372+0000 7f976189f640 1 --2- 172.21.15.27:0/3226171688 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f97440051c0 0x7f975c0b6070 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f974c004ed0 tx=0x7f974c007430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:17.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.372+0000 7f975a7fc640 1 -- 172.21.15.27:0/3226171688 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f974c00f7b0 con 0x7f97440051c0 2024-09-12T03:17:17.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.373+0000 7f975a7fc640 1 -- 172.21.15.27:0/3226171688 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f974c018070 con 0x7f97440051c0 2024-09-12T03:17:17.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.373+0000 7f9763b03640 1 -- 172.21.15.27:0/3226171688 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f975c15fea0 con 0x7f97440051c0 2024-09-12T03:17:17.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.373+0000 7f975a7fc640 1 -- 172.21.15.27:0/3226171688 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f974c00fbd0 con 0x7f97440051c0 2024-09-12T03:17:17.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.373+0000 7f9763b03640 1 -- 172.21.15.27:0/3226171688 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f975c160460 con 0x7f97440051c0 2024-09-12T03:17:17.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.374+0000 7f975a7fc640 1 -- 172.21.15.27:0/3226171688 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f974c01d460 con 0x7f97440051c0 2024-09-12T03:17:17.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.374+0000 7f975a7fc640 1 --2- 172.21.15.27:0/3226171688 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f974806be60 0x7f974806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:17.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.375+0000 7f975a7fc640 1 -- 172.21.15.27:0/3226171688 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f974c08ae10 con 0x7f97440051c0 2024-09-12T03:17:17.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.375+0000 7f976109e640 1 --2- 172.21.15.27:0/3226171688 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f974806be60 0x7f974806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:17.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.375+0000 7f976109e640 1 --2- 172.21.15.27:0/3226171688 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f974806be60 0x7f974806e320 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f975c15c050 tx=0x7f9750009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:17.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.376+0000 7f9766899740 1 Processor -- start 2024-09-12T03:17:17.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.376+0000 7f9766899740 1 -- start start 2024-09-12T03:17:17.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.376+0000 7f9766899740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x565045d702c0 0x565045db5a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:17.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.376+0000 7f9766899740 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x565045d6da80 0x565045d706a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:17.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.376+0000 7f9766899740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x565045d71a50 0x565045d6a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:17.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.376+0000 7f9766899740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x565045cd2520 con 0x565045d71a50 2024-09-12T03:17:17.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.376+0000 7f9766899740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x565045c7a3e0 con 0x565045d702c0 2024-09-12T03:17:17.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.376+0000 7f9766899740 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x565045cce4e0 con 0x565045d6da80 2024-09-12T03:17:17.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.377+0000 7f97620a0640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x565045d71a50 0x565045d6a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:17.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.377+0000 7f97620a0640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x565045d71a50 0x565045d6a8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37048/0 (socket says 172.21.15.27:37048) 2024-09-12T03:17:17.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.377+0000 7f97620a0640 1 -- 172.21.15.27:0/2680021061 learned_addr learned my addr 172.21.15.27:0/2680021061 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:17:17.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.377+0000 7f976189f640 1 --2- 172.21.15.27:0/2680021061 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x565045d702c0 0x565045db5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:17.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.377+0000 7f976109e640 1 --2- 172.21.15.27:0/2680021061 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x565045d6da80 0x565045d706a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:17.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.377+0000 7f97620a0640 1 -- 172.21.15.27:0/2680021061 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x565045d6da80 msgr2=0x565045d706a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:17.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.377+0000 7f97620a0640 1 --2- 172.21.15.27:0/2680021061 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x565045d6da80 0x565045d706a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:17.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.377+0000 7f97620a0640 1 -- 172.21.15.27:0/2680021061 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x565045d702c0 msgr2=0x565045db5a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:17.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.377+0000 7f97620a0640 1 --2- 172.21.15.27:0/2680021061 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x565045d702c0 0x565045db5a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:17.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.377+0000 7f97620a0640 1 -- 172.21.15.27:0/2680021061 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565045c952f0 con 0x565045d71a50 2024-09-12T03:17:17.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.377+0000 7f97620a0640 1 --2- 172.21.15.27:0/2680021061 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x565045d71a50 0x565045d6a8b0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f9754009e00 tx=0x7f975402f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c263b90338887cc server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:17.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.377+0000 7f97437fe640 1 -- 172.21.15.27:0/2680021061 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9754047070 con 0x565045d71a50 2024-09-12T03:17:17.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.377+0000 7f97437fe640 1 -- 172.21.15.27:0/2680021061 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97540387d0 con 0x565045d71a50 2024-09-12T03:17:17.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.377+0000 7f97437fe640 1 -- 172.21.15.27:0/2680021061 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9754033040 con 0x565045d71a50 2024-09-12T03:17:17.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.378+0000 7f9766899740 1 -- 172.21.15.27:0/2680021061 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x565045d71a50 msgr2=0x565045d6a8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:17.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.378+0000 7f9766899740 1 --2- 172.21.15.27:0/2680021061 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x565045d71a50 0x565045d6a8b0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f9754009e00 tx=0x7f975402f000 comp rx=0 tx=0).stop 2024-09-12T03:17:17.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.378+0000 7f9766899740 1 -- 172.21.15.27:0/2680021061 shutdown_connections 2024-09-12T03:17:17.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.378+0000 7f9766899740 1 --2- 172.21.15.27:0/2680021061 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x565045d71a50 0x565045d6a8b0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:17.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.378+0000 7f9766899740 1 --2- 172.21.15.27:0/2680021061 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x565045d6da80 0x565045d706a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:17.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.378+0000 7f9766899740 1 --2- 172.21.15.27:0/2680021061 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x565045d702c0 0x565045db5a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:17.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.378+0000 7f9766899740 1 -- 172.21.15.27:0/2680021061 >> 172.21.15.27:0/2680021061 conn(0x565045d6ca80 msgr2=0x565045db4f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:17.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.378+0000 7f9766899740 1 -- 172.21.15.27:0/2680021061 shutdown_connections 2024-09-12T03:17:17.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.378+0000 7f9766899740 1 -- 172.21.15.27:0/2680021061 wait complete. 2024-09-12T03:17:17.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.378+0000 7f9766899740 1 -- start start 2024-09-12T03:17:17.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.379+0000 7f9766899740 1 Processor -- start 2024-09-12T03:17:17.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.379+0000 7f9766899740 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x565045d702c0 0x565045d9a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:17.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.379+0000 7f9766899740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x565045d71a50 0x565045d8d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:17.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.379+0000 7f9766899740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x565045d91930 0x565045d6eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:17.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.379+0000 7f9766899740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x565045d6bf20 con 0x565045d71a50 2024-09-12T03:17:17.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.380+0000 7f9766899740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x565045c5b450 con 0x565045d91930 2024-09-12T03:17:17.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.380+0000 7f9766899740 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x565045db6290 con 0x565045d702c0 2024-09-12T03:17:17.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.380+0000 7f9766899740 10 client.0 _ll_register_callbacks cb 0x5650459e4ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:17:17.389 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.380+0000 7f976109e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x565045d71a50 0x565045d8d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:17.389 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.380+0000 7f976109e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x565045d71a50 0x565045d8d3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37054/0 (socket says 172.21.15.27:37054) 2024-09-12T03:17:17.389 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.380+0000 7f976109e640 1 -- 172.21.15.27:0/2189496671 learned_addr learned my addr 172.21.15.27:0/2189496671 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:17:17.389 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.380+0000 7f976189f640 1 --2- 172.21.15.27:0/2189496671 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x565045d702c0 0x565045d9a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:17.389 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.380+0000 7f97620a0640 1 --2- 172.21.15.27:0/2189496671 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x565045d91930 0x565045d6eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:17.389 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.380+0000 7f976109e640 1 -- 172.21.15.27:0/2189496671 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x565045d702c0 msgr2=0x565045d9a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:17.389 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.380+0000 7f976109e640 1 --2- 172.21.15.27:0/2189496671 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x565045d702c0 0x565045d9a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:17.389 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.380+0000 7f976109e640 1 -- 172.21.15.27:0/2189496671 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x565045d91930 msgr2=0x565045d6eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:17.389 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.380+0000 7f976109e640 1 --2- 172.21.15.27:0/2189496671 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x565045d91930 0x565045d6eaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:17.389 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.380+0000 7f976109e640 1 -- 172.21.15.27:0/2189496671 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5650459e52d0 con 0x565045d71a50 2024-09-12T03:17:17.389 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.380+0000 7f976109e640 1 --2- 172.21.15.27:0/2189496671 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x565045d71a50 0x565045d8d3c0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f9750001c30 tx=0x7f9750001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f445c16c4e283b90 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:17.389 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.380+0000 7f9740ff9640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-12T03:17:17.389 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.380+0000 7f9740ff9640 1 -- 172.21.15.27:0/2189496671 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9750007050 con 0x565045d71a50 2024-09-12T03:17:17.390 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.380+0000 7f9740ff9640 1 -- 172.21.15.27:0/2189496671 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f97500071f0 con 0x565045d71a50 2024-09-12T03:17:17.390 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.380+0000 7f9766899740 1 -- 172.21.15.27:0/2189496671 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f97540387d0 con 0x565045d71a50 2024-09-12T03:17:17.390 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.380+0000 7f9740ff9640 1 -- 172.21.15.27:0/2189496671 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=14) ==== 24+0+0 (secure 0 0 0) 0x7f97500073f0 con 0x565045d71a50 2024-09-12T03:17:17.390 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.380+0000 7f9766899740 10 client.4974 fetch_fsmap learned FSMap version 14 2024-09-12T03:17:17.390 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.380+0000 7f9766899740 1 -- 172.21.15.27:0/2189496671 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=14,monmap=2+}) -- 0x565045aaf6b0 con 0x565045d71a50 2024-09-12T03:17:17.390 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.381+0000 7f9740ff9640 1 -- 172.21.15.27:0/2189496671 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97540387d0 con 0x565045d71a50 2024-09-12T03:17:17.390 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.381+0000 7f9740ff9640 1 -- 172.21.15.27:0/2189496671 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 14) ==== 3502+0+0 (secure 0 0 0) 0x7f97500067d0 con 0x565045d71a50 2024-09-12T03:17:17.390 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.381+0000 7f9766899740 10 client.4974 fetch_fsmap finished waiting for FSMap version 14 2024-09-12T03:17:17.390 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.381+0000 7f9766899740 10 client.4974 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-12T03:17:17.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.381+0000 7f9766899740 20 client.4974 populate_metadata read hostname 'smithi027' 2024-09-12T03:17:17.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.381+0000 7f9766899740 1 --2- 172.21.15.27:0/2189496671 >> [v2:172.21.15.27:6833/3533575930,v1:172.21.15.27:6835/3533575930] conn(0x565045d97dc0 0x565045db0540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:17.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.381+0000 7f9766899740 4 client.4974 mds_command: new command op to 4818 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:17:17.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.381+0000 7f9766899740 1 -- 172.21.15.27:0/2189496671 --> [v2:172.21.15.27:6833/3533575930,v1:172.21.15.27:6835/3533575930] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x565045cdfc50 con 0x565045d97dc0 2024-09-12T03:17:17.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.381+0000 7f976189f640 1 --2- 172.21.15.27:0/2189496671 >> [v2:172.21.15.27:6833/3533575930,v1:172.21.15.27:6835/3533575930] conn(0x565045d97dc0 0x565045db0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:17.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.381+0000 7f976189f640 1 --2- 172.21.15.27:0/2189496671 >> [v2:172.21.15.27:6833/3533575930,v1:172.21.15.27:6835/3533575930] conn(0x565045d97dc0 0x565045db0540 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=90a72730d961c716 server_cookie=b52a9deaad23adf4 in_seq=0 out_seq=0 2024-09-12T03:17:17.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.381+0000 7f9740ff9640 10 client.4974 ms_handle_connect on v2:172.21.15.27:6833/3533575930 2024-09-12T03:17:17.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.382+0000 7f9740ff9640 1 -- 172.21.15.27:0/2189496671 <== mds.0 v2:172.21.15.27:6833/3533575930 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x565045cdfc50 con 0x565045d97dc0 2024-09-12T03:17:17.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.382+0000 7f9740ff9640 10 client.4974 handle_command_reply: tid=0 2024-09-12T03:17:17.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.382+0000 7f9766899740 1 client.4974 shutdown 2024-09-12T03:17:17.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.382+0000 7f9766899740 1 -- 172.21.15.27:0/2189496671 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x565045d71a50 msgr2=0x565045d8d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:17.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.382+0000 7f9766899740 1 --2- 172.21.15.27:0/2189496671 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x565045d71a50 0x565045d8d3c0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f9750001c30 tx=0x7f9750001120 comp rx=0 tx=0).stop 2024-09-12T03:17:17.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.382+0000 7f9766899740 1 -- 172.21.15.27:0/2189496671 shutdown_connections 2024-09-12T03:17:17.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.382+0000 7f9766899740 1 --2- 172.21.15.27:0/2189496671 >> [v2:172.21.15.27:6833/3533575930,v1:172.21.15.27:6835/3533575930] conn(0x565045d97dc0 0x565045db0540 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:17.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.382+0000 7f9766899740 1 --2- 172.21.15.27:0/2189496671 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x565045d91930 0x565045d6eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:17.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.382+0000 7f9740ff9640 0 client.4974 ms_handle_reset on v2:172.21.15.27:6833/3533575930 2024-09-12T03:17:17.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.382+0000 7f9766899740 1 --2- 172.21.15.27:0/2189496671 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x565045d71a50 0x565045d8d3c0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:17.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.382+0000 7f9766899740 1 --2- 172.21.15.27:0/2189496671 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x565045d702c0 0x565045d9a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:17.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.382+0000 7f9766899740 1 -- 172.21.15.27:0/2189496671 >> 172.21.15.27:0/2189496671 conn(0x565045d6da80 msgr2=0x565045db2ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:17.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.382+0000 7f9766899740 1 -- 172.21.15.27:0/2189496671 shutdown_connections 2024-09-12T03:17:17.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.382+0000 7f9766899740 1 -- 172.21.15.27:0/2189496671 wait complete. 2024-09-12T03:17:17.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.382+0000 7f9766899740 20 client.4974 trim_cache size 0 max 16384 2024-09-12T03:17:17.400 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.400+0000 7f9766899740 1 Processor -- start 2024-09-12T03:17:17.400 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.400+0000 7f9766899740 1 -- start start 2024-09-12T03:17:17.408 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.401+0000 7f9766899740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x565045d702c0 0x565045d8d390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:17.408 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.401+0000 7f9766899740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x565045d71a50 0x565045db0510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:17.408 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.401+0000 7f9766899740 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x565045d91930 0x565045db0a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:17.408 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.401+0000 7f9766899740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f975c2000a0 con 0x565045d71a50 2024-09-12T03:17:17.408 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.401+0000 7f9766899740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f975c1f88a0 con 0x565045d702c0 2024-09-12T03:17:17.408 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.401+0000 7f9766899740 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f975c150450 con 0x565045d91930 2024-09-12T03:17:17.408 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.401+0000 7f976109e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x565045d71a50 0x565045db0510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:17.408 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.401+0000 7f976109e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x565045d71a50 0x565045db0510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37066/0 (socket says 172.21.15.27:37066) 2024-09-12T03:17:17.408 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.401+0000 7f976109e640 1 -- 172.21.15.27:0/3737271394 learned_addr learned my addr 172.21.15.27:0/3737271394 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:17:17.409 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.401+0000 7f976109e640 1 -- 172.21.15.27:0/3737271394 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x565045d91930 msgr2=0x565045db0a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:17.409 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.401+0000 7f976109e640 1 --2- 172.21.15.27:0/3737271394 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x565045d91930 0x565045db0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:17.409 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.401+0000 7f976109e640 1 -- 172.21.15.27:0/3737271394 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x565045d702c0 msgr2=0x565045d8d390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:17.409 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.401+0000 7f976109e640 1 --2- 172.21.15.27:0/3737271394 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x565045d702c0 0x565045d8d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:17.409 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.401+0000 7f976109e640 1 -- 172.21.15.27:0/3737271394 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565045db3cb0 con 0x565045d71a50 2024-09-12T03:17:17.409 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.402+0000 7f976109e640 1 --2- 172.21.15.27:0/3737271394 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x565045d71a50 0x565045db0510 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f9750000fd0 tx=0x7f9750022000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd0e6f2e4916b11 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:17.409 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.402+0000 7f9742ffd640 1 -- 172.21.15.27:0/3737271394 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f975003c070 con 0x565045d71a50 2024-09-12T03:17:17.409 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.402+0000 7f9742ffd640 1 -- 172.21.15.27:0/3737271394 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9750027070 con 0x565045d71a50 2024-09-12T03:17:17.409 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.402+0000 7f9742ffd640 1 -- 172.21.15.27:0/3737271394 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9750014ba0 con 0x565045d71a50 2024-09-12T03:17:17.409 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.402+0000 7f9766899740 1 -- 172.21.15.27:0/3737271394 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x565045d71a50 msgr2=0x565045db0510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:17.409 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.402+0000 7f9766899740 1 --2- 172.21.15.27:0/3737271394 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x565045d71a50 0x565045db0510 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f9750000fd0 tx=0x7f9750022000 comp rx=0 tx=0).stop 2024-09-12T03:17:17.409 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.402+0000 7f9766899740 1 -- 172.21.15.27:0/3737271394 shutdown_connections 2024-09-12T03:17:17.409 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.402+0000 7f9766899740 1 --2- 172.21.15.27:0/3737271394 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x565045d91930 0x565045db0a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:17.409 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.402+0000 7f9766899740 1 --2- 172.21.15.27:0/3737271394 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x565045d71a50 0x565045db0510 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:17.410 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.402+0000 7f9766899740 1 --2- 172.21.15.27:0/3737271394 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x565045d702c0 0x565045d8d390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:17.410 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.402+0000 7f9766899740 1 -- 172.21.15.27:0/3737271394 >> 172.21.15.27:0/3737271394 conn(0x565045d6da80 msgr2=0x565045db2320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:17.410 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.402+0000 7f9766899740 1 -- 172.21.15.27:0/3737271394 shutdown_connections 2024-09-12T03:17:17.410 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.402+0000 7f9766899740 1 -- 172.21.15.27:0/3737271394 wait complete. 2024-09-12T03:17:17.410 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.403+0000 7f9766899740 1 -- start start 2024-09-12T03:17:17.410 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.403+0000 7f9766899740 1 Processor -- start 2024-09-12T03:17:17.410 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.404+0000 7f9766899740 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x565045d702c0 0x565045e334c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:17.410 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.404+0000 7f9766899740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x565045d71a50 0x565045db3ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:17.410 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.404+0000 7f9766899740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x565045d91930 0x565045db43e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:17.410 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.404+0000 7f9766899740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x565045d8da40 con 0x565045d71a50 2024-09-12T03:17:17.410 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.404+0000 7f9766899740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x565045db14d0 con 0x565045d91930 2024-09-12T03:17:17.410 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.404+0000 7f9766899740 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x565045d98210 con 0x565045d702c0 2024-09-12T03:17:17.410 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.404+0000 7f9766899740 10 client.0 _ll_register_callbacks cb 0x7f9750005240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:17:17.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.404+0000 7f976109e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x565045d71a50 0x565045db3ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:17.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.404+0000 7f976109e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x565045d71a50 0x565045db3ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37078/0 (socket says 172.21.15.27:37078) 2024-09-12T03:17:17.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.404+0000 7f976109e640 1 -- 172.21.15.27:0/986921505 learned_addr learned my addr 172.21.15.27:0/986921505 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:17:17.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.404+0000 7f976189f640 1 --2- 172.21.15.27:0/986921505 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x565045d702c0 0x565045e334c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:17.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.404+0000 7f97620a0640 1 --2- 172.21.15.27:0/986921505 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x565045d91930 0x565045db43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:17.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.404+0000 7f976109e640 1 -- 172.21.15.27:0/986921505 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x565045d702c0 msgr2=0x565045e334c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:17.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.404+0000 7f976109e640 1 --2- 172.21.15.27:0/986921505 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x565045d702c0 0x565045e334c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:17.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.404+0000 7f976109e640 1 -- 172.21.15.27:0/986921505 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x565045d91930 msgr2=0x565045db43e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:17.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.404+0000 7f976109e640 1 --2- 172.21.15.27:0/986921505 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x565045d91930 0x565045db43e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:17.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.404+0000 7f976109e640 1 -- 172.21.15.27:0/986921505 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565045d97000 con 0x565045d71a50 2024-09-12T03:17:17.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.405+0000 7f976109e640 1 --2- 172.21.15.27:0/986921505 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x565045d71a50 0x565045db3ea0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f975003a040 tx=0x7f9750014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e9b3927871ec115 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:17.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.405+0000 7f9740ff9640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-12T03:17:17.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.405+0000 7f9740ff9640 1 -- 172.21.15.27:0/986921505 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f975003c040 con 0x565045d71a50 2024-09-12T03:17:17.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.405+0000 7f9740ff9640 1 -- 172.21.15.27:0/986921505 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f9750006cd0 con 0x565045d71a50 2024-09-12T03:17:17.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.405+0000 7f9766899740 1 -- 172.21.15.27:0/986921505 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9750027070 con 0x565045d71a50 2024-09-12T03:17:17.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.405+0000 7f9740ff9640 1 -- 172.21.15.27:0/986921505 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=14) ==== 24+0+0 (secure 0 0 0) 0x7f975004f050 con 0x565045d71a50 2024-09-12T03:17:17.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.405+0000 7f9766899740 10 client.4980 fetch_fsmap learned FSMap version 14 2024-09-12T03:17:17.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.405+0000 7f9766899740 1 -- 172.21.15.27:0/986921505 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=14,monmap=2+}) -- 0x565045e33d80 con 0x565045d71a50 2024-09-12T03:17:17.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.405+0000 7f9740ff9640 1 -- 172.21.15.27:0/986921505 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9750027070 con 0x565045d71a50 2024-09-12T03:17:17.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.405+0000 7f9740ff9640 1 -- 172.21.15.27:0/986921505 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 14) ==== 3502+0+0 (secure 0 0 0) 0x7f9750052020 con 0x565045d71a50 2024-09-12T03:17:17.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.405+0000 7f9766899740 10 client.4980 fetch_fsmap finished waiting for FSMap version 14 2024-09-12T03:17:17.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.405+0000 7f9766899740 10 client.4980 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-12T03:17:17.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.405+0000 7f9766899740 20 client.4980 populate_metadata read hostname 'smithi027' 2024-09-12T03:17:17.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.405+0000 7f9766899740 1 --2- 172.21.15.27:0/986921505 >> [v2:172.21.15.27:6833/3533575930,v1:172.21.15.27:6835/3533575930] conn(0x565045d8c8f0 0x565045db0470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:17.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.405+0000 7f9766899740 4 client.4980 mds_command: new command op to 4818 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-12T03:17:17.413 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.405+0000 7f9766899740 1 -- 172.21.15.27:0/986921505 --> [v2:172.21.15.27:6833/3533575930,v1:172.21.15.27:6835/3533575930] -- command(tid 0: {"prefix": "status"}) -- 0x565045cd9fa0 con 0x565045d8c8f0 2024-09-12T03:17:17.413 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.405+0000 7f976189f640 1 --2- 172.21.15.27:0/986921505 >> [v2:172.21.15.27:6833/3533575930,v1:172.21.15.27:6835/3533575930] conn(0x565045d8c8f0 0x565045db0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:17.413 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.406+0000 7f9740ff9640 10 client.4980 ms_handle_connect on v2:172.21.15.27:6833/3533575930 2024-09-12T03:17:17.413 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.406+0000 7f976189f640 1 --2- 172.21.15.27:0/986921505 >> [v2:172.21.15.27:6833/3533575930,v1:172.21.15.27:6835/3533575930] conn(0x565045d8c8f0 0x565045db0470 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e2e0a5a509b70b8d server_cookie=1f9619d2f09d20f in_seq=0 out_seq=0 2024-09-12T03:17:17.413 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.406+0000 7f9740ff9640 1 -- 172.21.15.27:0/986921505 <== mds.0 v2:172.21.15.27:6833/3533575930 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x565045cd9fa0 con 0x565045d8c8f0 2024-09-12T03:17:17.413 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.406+0000 7f9740ff9640 10 client.4980 handle_command_reply: tid=0 2024-09-12T03:17:17.413 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.406+0000 7f9766899740 1 client.4980 shutdown 2024-09-12T03:17:17.413 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.406+0000 7f9766899740 1 -- 172.21.15.27:0/986921505 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x565045d71a50 msgr2=0x565045db3ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:17.413 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.406+0000 7f9766899740 1 --2- 172.21.15.27:0/986921505 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x565045d71a50 0x565045db3ea0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f975003a040 tx=0x7f9750014000 comp rx=0 tx=0).stop 2024-09-12T03:17:17.413 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.406+0000 7f9766899740 1 -- 172.21.15.27:0/986921505 shutdown_connections 2024-09-12T03:17:17.413 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.406+0000 7f9766899740 1 --2- 172.21.15.27:0/986921505 >> [v2:172.21.15.27:6833/3533575930,v1:172.21.15.27:6835/3533575930] conn(0x565045d8c8f0 0x565045db0470 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:17.413 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.406+0000 7f9766899740 1 --2- 172.21.15.27:0/986921505 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x565045d91930 0x565045db43e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:17.414 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.406+0000 7f9766899740 1 --2- 172.21.15.27:0/986921505 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x565045d71a50 0x565045db3ea0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:17.414 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.406+0000 7f9766899740 1 --2- 172.21.15.27:0/986921505 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x565045d702c0 0x565045e334c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:17.414 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.406+0000 7f9740ff9640 0 client.4980 ms_handle_reset on v2:172.21.15.27:6833/3533575930 2024-09-12T03:17:17.414 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.406+0000 7f9766899740 1 -- 172.21.15.27:0/986921505 >> 172.21.15.27:0/986921505 conn(0x565045d6da80 msgr2=0x565045d98d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:17.414 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.406+0000 7f9766899740 1 -- 172.21.15.27:0/986921505 shutdown_connections 2024-09-12T03:17:17.414 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.406+0000 7f9766899740 1 -- 172.21.15.27:0/986921505 wait complete. 2024-09-12T03:17:17.414 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.406+0000 7f9766899740 20 client.4980 trim_cache size 0 max 16384 2024-09-12T03:17:17.414 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-12T03:17:17.414 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_fsid": "d2df504e-2647-4385-a5db-9fdf69f487a0", 2024-09-12T03:17:17.414 INFO:teuthology.orchestra.run.smithi027.stdout: "whoami": 0, 2024-09-12T03:17:17.414 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 4818, 2024-09-12T03:17:17.415 INFO:teuthology.orchestra.run.smithi027.stdout: "want_state": "up:active", 2024-09-12T03:17:17.415 INFO:teuthology.orchestra.run.smithi027.stdout: "state": "up:active", 2024-09-12T03:17:17.415 INFO:teuthology.orchestra.run.smithi027.stdout: "fs_name": "cephfs", 2024-09-12T03:17:17.415 INFO:teuthology.orchestra.run.smithi027.stdout: "rank_uptime": 5.6470714160000002, 2024-09-12T03:17:17.415 INFO:teuthology.orchestra.run.smithi027.stdout: "mdsmap_epoch": 14, 2024-09-12T03:17:17.415 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch": 32, 2024-09-12T03:17:17.415 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch_barrier": 32, 2024-09-12T03:17:17.415 INFO:teuthology.orchestra.run.smithi027.stdout: "uptime": 12.519547671 2024-09-12T03:17:17.415 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-12T03:17:17.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.407+0000 7f9763b03640 1 -- 172.21.15.27:0/3226171688 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f974806be60 msgr2=0x7f974806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:17.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.407+0000 7f9763b03640 1 --2- 172.21.15.27:0/3226171688 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f974806be60 0x7f974806e320 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f975c15c050 tx=0x7f9750009040 comp rx=0 tx=0).stop 2024-09-12T03:17:17.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.407+0000 7f9763b03640 1 -- 172.21.15.27:0/3226171688 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f97440051c0 msgr2=0x7f975c0b6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:17.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.407+0000 7f9763b03640 1 --2- 172.21.15.27:0/3226171688 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f97440051c0 0x7f975c0b6070 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f974c004ed0 tx=0x7f974c007430 comp rx=0 tx=0).stop 2024-09-12T03:17:17.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.407+0000 7f9763b03640 1 -- 172.21.15.27:0/3226171688 shutdown_connections 2024-09-12T03:17:17.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.407+0000 7f9763b03640 1 --2- 172.21.15.27:0/3226171688 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f974806be60 0x7f974806e320 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:17.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.407+0000 7f9763b03640 1 --2- 172.21.15.27:0/3226171688 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f975c0cc230 0x7f975c1611a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:17.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.407+0000 7f9763b03640 1 --2- 172.21.15.27:0/3226171688 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f975c0cbdb0 0x7f975c160c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:17.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.407+0000 7f9763b03640 1 --2- 172.21.15.27:0/3226171688 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f97440051c0 0x7f975c0b6070 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:17.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.407+0000 7f9763b03640 1 -- 172.21.15.27:0/3226171688 >> 172.21.15.27:0/3226171688 conn(0x7f975c073800 msgr2=0x7f975c0b26d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:17.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.407+0000 7f9763b03640 1 -- 172.21.15.27:0/3226171688 shutdown_connections 2024-09-12T03:17:17.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.407+0000 7f9763b03640 1 -- 172.21.15.27:0/3226171688 wait complete. 2024-09-12T03:17:17.418 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-12T03:17:17.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.520+0000 7fa249f59640 1 Processor -- start 2024-09-12T03:17:17.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.520+0000 7fa249f59640 1 -- start start 2024-09-12T03:17:17.522 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.521+0000 7fa249f59640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa24415eb90 0x7fa2440d2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:17.522 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.521+0000 7fa249f59640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fa2440d5200 0x7fa2440d1010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:17.522 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.521+0000 7fa249f59640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa2440d19e0 con 0x7fa2440d1550 2024-09-12T03:17:17.522 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.521+0000 7fa249f59640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7fa2440b7d20 con 0x7fa2440d5200 2024-09-12T03:17:17.522 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.521+0000 7fa249f59640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fa2440d42e0 con 0x7fa24415eb90 2024-09-12T03:17:17.522 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.521+0000 7fa242ffd640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fa2440d5200 0x7fa2440d1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:17.522 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.521+0000 7fa243fff640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fa2440d1550 0x7fa2440cf200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:42604/0 (socket says 172.21.15.27:42604) 2024-09-12T03:17:17.522 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.521+0000 7fa243fff640 1 -- 172.21.15.27:0/4031698723 learned_addr learned my addr 172.21.15.27:0/4031698723 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:17:17.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.522+0000 7fa2437fe640 1 --2- 172.21.15.27:0/4031698723 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa24415eb90 0x7fa2440d2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:17.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.522+0000 7fa242ffd640 1 -- 172.21.15.27:0/4031698723 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa24415eb90 msgr2=0x7fa2440d2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:17.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.522+0000 7fa242ffd640 1 --2- 172.21.15.27:0/4031698723 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa24415eb90 0x7fa2440d2df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:17.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.522+0000 7fa242ffd640 1 -- 172.21.15.27:0/4031698723 >> v1:172.21.15.27:6789/0 conn(0x7fa2440d1550 legacy=0x7fa2440cf200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:17.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.522+0000 7fa242ffd640 1 -- 172.21.15.27:0/4031698723 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2440d3760 con 0x7fa2440d5200 2024-09-12T03:17:17.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.522+0000 7fa2427fc640 1 -- 172.21.15.27:0/4031698723 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 965354680 0 0) 0x7fa2440d19e0 con 0x7fa2440d1550 2024-09-12T03:17:17.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.523+0000 7fa242ffd640 1 --2- 172.21.15.27:0/4031698723 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa2440d5200 0x7fa2440d1010 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fa2380097a0 tx=0x7fa23802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fabf743473ebece server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:17.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.523+0000 7fa2427fc640 1 -- 172.21.15.27:0/4031698723 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa238037070 con 0x7fa2440d5200 2024-09-12T03:17:17.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.523+0000 7fa2427fc640 1 -- 172.21.15.27:0/4031698723 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa238032440 con 0x7fa2440d5200 2024-09-12T03:17:17.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.523+0000 7fa2427fc640 1 -- 172.21.15.27:0/4031698723 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa238032810 con 0x7fa2440d5200 2024-09-12T03:17:17.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.523+0000 7fa249f59640 1 -- 172.21.15.27:0/4031698723 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa2440d5200 msgr2=0x7fa2440d1010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:17.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.523+0000 7fa249f59640 1 --2- 172.21.15.27:0/4031698723 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa2440d5200 0x7fa2440d1010 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fa2380097a0 tx=0x7fa23802a180 comp rx=0 tx=0).stop 2024-09-12T03:17:17.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.523+0000 7fa249f59640 1 -- 172.21.15.27:0/4031698723 shutdown_connections 2024-09-12T03:17:17.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.523+0000 7fa249f59640 1 --2- 172.21.15.27:0/4031698723 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa2440d5200 0x7fa2440d1010 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:17.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.523+0000 7fa249f59640 1 --2- 172.21.15.27:0/4031698723 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa24415eb90 0x7fa2440d2df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:17.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.523+0000 7fa249f59640 1 -- 172.21.15.27:0/4031698723 >> 172.21.15.27:0/4031698723 conn(0x7fa24406dce0 msgr2=0x7fa24406e0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:17.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.523+0000 7fa249f59640 1 -- 172.21.15.27:0/4031698723 shutdown_connections 2024-09-12T03:17:17.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.523+0000 7fa249f59640 1 -- 172.21.15.27:0/4031698723 wait complete. 2024-09-12T03:17:17.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.525+0000 7fa249f59640 1 Processor -- start 2024-09-12T03:17:17.528 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.525+0000 7fa249f59640 1 -- start start 2024-09-12T03:17:17.529 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.525+0000 7fa249f59640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa2440d1550 0x7fa24415ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:17.529 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.526+0000 7fa249f59640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa2440d5200 0x7fa2440ac6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:17.529 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.526+0000 7fa249f59640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa24415eb90 0x7fa2440ab210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:17.529 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.526+0000 7fa249f59640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fa2440d4000 con 0x7fa24415eb90 2024-09-12T03:17:17.529 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.526+0000 7fa249f59640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa2440d3d20 con 0x7fa2440d5200 2024-09-12T03:17:17.529 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.526+0000 7fa249f59640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fa2440d3330 con 0x7fa2440d1550 2024-09-12T03:17:17.529 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.526+0000 7fa243fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa24415eb90 0x7fa2440ab210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:17.529 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.526+0000 7fa243fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa24415eb90 0x7fa2440ab210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37094/0 (socket says 172.21.15.27:37094) 2024-09-12T03:17:17.529 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.526+0000 7fa2437fe640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa2440d1550 0x7fa24415ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:17.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.526+0000 7fa243fff640 1 -- 172.21.15.27:0/3212708261 learned_addr learned my addr 172.21.15.27:0/3212708261 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:17:17.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.526+0000 7fa242ffd640 1 --2- 172.21.15.27:0/3212708261 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa2440d5200 0x7fa2440ac6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:17.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.526+0000 7fa243fff640 1 -- 172.21.15.27:0/3212708261 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa2440d1550 msgr2=0x7fa24415ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:17.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.526+0000 7fa243fff640 1 --2- 172.21.15.27:0/3212708261 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa2440d1550 0x7fa24415ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:17.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.526+0000 7fa243fff640 1 -- 172.21.15.27:0/3212708261 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa2440d5200 msgr2=0x7fa2440ac6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:17.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.526+0000 7fa243fff640 1 --2- 172.21.15.27:0/3212708261 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa2440d5200 0x7fa2440ac6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:17.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.526+0000 7fa243fff640 1 -- 172.21.15.27:0/3212708261 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2440acfe0 con 0x7fa24415eb90 2024-09-12T03:17:17.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.526+0000 7fa243fff640 1 --2- 172.21.15.27:0/3212708261 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa24415eb90 0x7fa2440ab210 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fa23c00c800 tx=0x7fa23c00ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:17.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.526+0000 7fa22bfff640 1 -- 172.21.15.27:0/3212708261 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa23c018070 con 0x7fa24415eb90 2024-09-12T03:17:17.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.526+0000 7fa22bfff640 1 -- 172.21.15.27:0/3212708261 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa23c004d80 con 0x7fa24415eb90 2024-09-12T03:17:17.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.526+0000 7fa249f59640 1 -- 172.21.15.27:0/3212708261 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2440ad2c0 con 0x7fa24415eb90 2024-09-12T03:17:17.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.526+0000 7fa22bfff640 1 -- 172.21.15.27:0/3212708261 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa23c00a880 con 0x7fa24415eb90 2024-09-12T03:17:17.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.526+0000 7fa249f59640 1 -- 172.21.15.27:0/3212708261 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2440ad880 con 0x7fa24415eb90 2024-09-12T03:17:17.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.527+0000 7fa249f59640 1 -- 172.21.15.27:0/3212708261 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa24415ef70 con 0x7fa24415eb90 2024-09-12T03:17:17.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.528+0000 7fa22bfff640 1 -- 172.21.15.27:0/3212708261 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa23c01b070 con 0x7fa24415eb90 2024-09-12T03:17:17.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.528+0000 7fa22bfff640 1 --2- 172.21.15.27:0/3212708261 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fa23006be30 0x7fa23006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:17.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.529+0000 7fa22bfff640 1 -- 172.21.15.27:0/3212708261 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7fa23c08a2d0 con 0x7fa24415eb90 2024-09-12T03:17:17.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.529+0000 7fa2437fe640 1 --2- 172.21.15.27:0/3212708261 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fa23006be30 0x7fa23006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:17.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.529+0000 7fa2437fe640 1 --2- 172.21.15.27:0/3212708261 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fa23006be30 0x7fa23006e2f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fa2440c8240 tx=0x7fa234009190 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:17.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.533+0000 7fa22bfff640 1 -- 172.21.15.27:0/3212708261 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fa23c058780 con 0x7fa24415eb90 2024-09-12T03:17:17.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.744+0000 7fa249f59640 1 -- 172.21.15.27:0/3212708261 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fa24415e340 con 0x7fa24415eb90 2024-09-12T03:17:17.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.745+0000 7fa22bfff640 1 -- 172.21.15.27:0/3212708261 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v14) ==== 53+0+73 (secure 0 0 0) 0x7fa23c05ca80 con 0x7fa24415eb90 2024-09-12T03:17:17.746 INFO:teuthology.orchestra.run.smithi027.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-12T03:17:17.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.747+0000 7fa249f59640 1 -- 172.21.15.27:0/3212708261 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fa23006be30 msgr2=0x7fa23006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:17.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.747+0000 7fa249f59640 1 --2- 172.21.15.27:0/3212708261 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fa23006be30 0x7fa23006e2f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fa2440c8240 tx=0x7fa234009190 comp rx=0 tx=0).stop 2024-09-12T03:17:17.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.747+0000 7fa249f59640 1 -- 172.21.15.27:0/3212708261 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa24415eb90 msgr2=0x7fa2440ab210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:17.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.747+0000 7fa249f59640 1 --2- 172.21.15.27:0/3212708261 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa24415eb90 0x7fa2440ab210 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fa23c00c800 tx=0x7fa23c00ccc0 comp rx=0 tx=0).stop 2024-09-12T03:17:17.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.747+0000 7fa249f59640 1 -- 172.21.15.27:0/3212708261 shutdown_connections 2024-09-12T03:17:17.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.747+0000 7fa249f59640 1 --2- 172.21.15.27:0/3212708261 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fa23006be30 0x7fa23006e2f0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:17.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.747+0000 7fa249f59640 1 --2- 172.21.15.27:0/3212708261 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa24415eb90 0x7fa2440ab210 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:17.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.747+0000 7fa249f59640 1 --2- 172.21.15.27:0/3212708261 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa2440d5200 0x7fa2440ac6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:17.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.747+0000 7fa249f59640 1 --2- 172.21.15.27:0/3212708261 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa2440d1550 0x7fa24415ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:17.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.747+0000 7fa249f59640 1 -- 172.21.15.27:0/3212708261 >> 172.21.15.27:0/3212708261 conn(0x7fa24406dce0 msgr2=0x7fa2441607a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:17.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.747+0000 7fa249f59640 1 -- 172.21.15.27:0/3212708261 shutdown_connections 2024-09-12T03:17:17.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17.747+0000 7fa249f59640 1 -- 172.21.15.27:0/3212708261 wait complete. 2024-09-12T03:17:17.757 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-12T03:17:17.757 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-12T03:17:17.758 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi027.front.sepia.ceph.com 2024-09-12T03:17:17.758 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-12T03:17:17.758 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-12T03:17:17.758 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-12T03:17:17.758 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-12T03:17:17.758 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-12T03:17:17.758 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-12T03:17:17.758 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:17:17.758 DEBUG:teuthology.orchestra.run.smithi027:> ip addr 2024-09-12T03:17:17.775 INFO:teuthology.orchestra.run.smithi027.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-12T03:17:17.775 INFO:teuthology.orchestra.run.smithi027.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-12T03:17:17.775 INFO:teuthology.orchestra.run.smithi027.stdout: inet 127.0.0.1/8 scope host lo 2024-09-12T03:17:17.775 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:17:17.775 INFO:teuthology.orchestra.run.smithi027.stdout: inet6 ::1/128 scope host 2024-09-12T03:17:17.775 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:17:17.775 INFO:teuthology.orchestra.run.smithi027.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-12T03:17:17.775 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether 0c:c4:7a:6b:fe:c8 brd ff:ff:ff:ff:ff:ff 2024-09-12T03:17:17.776 INFO:teuthology.orchestra.run.smithi027.stdout: altname enp7s0f0 2024-09-12T03:17:17.776 INFO:teuthology.orchestra.run.smithi027.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-12T03:17:17.776 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether 0c:c4:7a:6b:fe:c9 brd ff:ff:ff:ff:ff:ff 2024-09-12T03:17:17.776 INFO:teuthology.orchestra.run.smithi027.stdout: altname enp7s0f1 2024-09-12T03:17:17.776 INFO:teuthology.orchestra.run.smithi027.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-12T03:17:17.776 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether ac:1f:6b:f8:39:5a brd ff:ff:ff:ff:ff:ff 2024-09-12T03:17:17.776 INFO:teuthology.orchestra.run.smithi027.stdout: altname enp1s0f0 2024-09-12T03:17:17.776 INFO:teuthology.orchestra.run.smithi027.stdout: inet 172.21.15.27/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-12T03:17:17.776 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft 42751sec preferred_lft 42751sec 2024-09-12T03:17:17.776 INFO:teuthology.orchestra.run.smithi027.stdout: inet6 fe80::ae1f:6bff:fef8:395a/64 scope link 2024-09-12T03:17:17.776 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:17:17.777 INFO:teuthology.orchestra.run.smithi027.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-12T03:17:17.777 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether ac:1f:6b:f8:39:5b brd ff:ff:ff:ff:ff:ff 2024-09-12T03:17:17.777 INFO:teuthology.orchestra.run.smithi027.stdout: altname enp1s0f1 2024-09-12T03:17:17.777 INFO:teuthology.orchestra.run.smithi027.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-12T03:17:17.777 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether 2e:7f:89:6c:33:a0 brd ff:ff:ff:ff:ff:ff 2024-09-12T03:17:17.777 INFO:teuthology.orchestra.run.smithi027.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-12T03:17:17.777 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:17:17.777 INFO:teuthology.orchestra.run.smithi027.stdout: inet6 fe80::c84b:19ff:fef0:ca74/64 scope link 2024-09-12T03:17:17.777 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:17:17.777 INFO:teuthology.orchestra.run.smithi027.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-12T03:17:17.777 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether 42:40:d7:5c:7a:8e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-12T03:17:17.778 INFO:teuthology.orchestra.run.smithi027.stdout: inet6 fe80::4040:d7ff:fe5c:7a8e/64 scope link 2024-09-12T03:17:17.778 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:17:17.778 INFO:teuthology.orchestra.run.smithi027.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-12T03:17:17.778 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether 2e:7f:89:6c:33:a0 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-12T03:17:17.778 INFO:teuthology.orchestra.run.smithi027.stdout: inet6 fe80::2c7f:89ff:fe6c:33a0/64 scope link 2024-09-12T03:17:17.778 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:17:17.778 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:17:17.778 DEBUG:teuthology.orchestra.run.smithi027:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-12T03:17:17.839 INFO:teuthology.orchestra.run.smithi027.stdout:1 2024-09-12T03:17:17.841 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:17:17.841 DEBUG:teuthology.orchestra.run.smithi027:> route 2024-09-12T03:17:17.897 INFO:teuthology.orchestra.run.smithi027.stdout:Kernel IP routing table 2024-09-12T03:17:17.898 INFO:teuthology.orchestra.run.smithi027.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-12T03:17:17.898 INFO:teuthology.orchestra.run.smithi027.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-12T03:17:17.898 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-12T03:17:17.898 INFO:teuthology.orchestra.run.smithi027.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-12T03:17:17.898 DEBUG:teuthology.orchestra.run.smithi027:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-12T03:17:17.898 DEBUG:teuthology.orchestra.run.smithi027:> set -e 2024-09-12T03:17:17.898 DEBUG:teuthology.orchestra.run.smithi027:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-12T03:17:17.898 DEBUG:teuthology.orchestra.run.smithi027:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-12T03:17:17.899 DEBUG:teuthology.orchestra.run.smithi027:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-12T03:17:17.899 DEBUG:teuthology.orchestra.run.smithi027:> ') 2024-09-12T03:17:17.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:17 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T03:17:18.021 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T03:17:18.027 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:17:18.027 DEBUG:teuthology.orchestra.run.smithi027:> ip netns list 2024-09-12T03:17:18.043 INFO:teuthology.orchestra.run.smithi027.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-12T03:17:18.043 INFO:teuthology.orchestra.run.smithi027.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-12T03:17:18.043 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-12T03:17:18.044 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:17:18.044 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip link set brx.0 up 2024-09-12T03:17:18.110 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-12T03:17:18.110 DEBUG:teuthology.orchestra.run.smithi027:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-12T03:17:18.166 INFO:teuthology.orchestra.run.smithi027.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-12T03:17:18.166 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-12T03:17:18.166 DEBUG:teuthology.orchestra.run.smithi027:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-12T03:17:18.221 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-09-12T03:17:18.221 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:17:18.221 DEBUG:teuthology.orchestra.run.smithi027:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-12T03:17:18.306 INFO:teuthology.orchestra.run.smithi027.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-12T03:17:18.306 INFO:teuthology.orchestra.run.smithi027.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-12T03:17:18.315 INFO:teuthology.orchestra.run.smithi027.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-12T03:17:18.315 INFO:teuthology.orchestra.run.smithi027.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-12T03:17:18.315 INFO:teuthology.orchestra.run.smithi027.stdout:mount.ceph: resolved to: "172.21.15.27:3300,172.21.15.45:3300,172.21.15.45:3301" 2024-09-12T03:17:18.315 INFO:teuthology.orchestra.run.smithi027.stdout:mount.ceph: trying mount with new device syntax: 0@d2df504e-2647-4385-a5db-9fdf69f487a0.cephfs=/ 2024-09-12T03:17:18.316 INFO:teuthology.orchestra.run.smithi027.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.27:3300/172.21.15.45:3300/172.21.15.45:3301" will pass to kernel 2024-09-12T03:17:18.316 INFO:teuthology.orchestra.run.smithi027.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-12T03:17:18.316 INFO:teuthology.orchestra.run.smithi027.stdout: You just mounted a file system that supports labels which does not 2024-09-12T03:17:18.316 INFO:teuthology.orchestra.run.smithi027.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-12T03:17:18.316 INFO:teuthology.orchestra.run.smithi027.stdout: applications will generate AVC messages and not be allowed access to 2024-09-12T03:17:18.316 INFO:teuthology.orchestra.run.smithi027.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-12T03:17:18.318 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-12T03:17:18.318 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:17:18.318 DEBUG:teuthology.orchestra.run.smithi027:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-12T03:17:18.344 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-12T03:17:18.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:18 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-12T03:17:18.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T03:17:18.448 INFO:teuthology.orchestra.run.smithi027.stdout:client4706 2024-09-12T03:17:18.448 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-12T03:17:18.449 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T03:17:18.465 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-12T03:17:18.608 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:18.608+0000 7fbafd85a640 1 Processor -- start 2024-09-12T03:17:18.608 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:18.608+0000 7fbafd85a640 1 -- start start 2024-09-12T03:17:18.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:18.609+0000 7fbafd85a640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fbaf80ce6d0 0x7fbaf80ceab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:18.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:18.609+0000 7fbafd85a640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fbaf806c240 0x7fbaf806e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:18.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:18.609+0000 7fbafd85a640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbaf81463d0 con 0x7fbaf8160d80 2024-09-12T03:17:18.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:18.609+0000 7fbafd85a640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fbaf80ceff0 con 0x7fbaf80ce6d0 2024-09-12T03:17:18.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:18.609+0000 7fbafd85a640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fbaf806c7f0 con 0x7fbaf806c240 2024-09-12T03:17:18.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:18.609+0000 7fbaf77fe640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fbaf806c240 0x7fbaf806e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:18.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:18.609+0000 7fbaf6ffd640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fbaf80ce6d0 0x7fbaf80ceab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:18.610 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:18.609+0000 7fbaf67fc640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fbaf8160d80 0x7fbaf806bb30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.27:34164/0 (socket says 172.21.15.27:34164) 2024-09-12T03:17:18.610 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:18.609+0000 7fbaf67fc640 1 -- 172.21.15.27:0/3305825695 learned_addr learned my addr 172.21.15.27:0/3305825695 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:17:18.611 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:18.610+0000 7fbaf6ffd640 1 -- 172.21.15.27:0/3305825695 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fbaf806c240 msgr2=0x7fbaf806e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:18.611 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:18.610+0000 7fbaf6ffd640 1 --2- 172.21.15.27:0/3305825695 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fbaf806c240 0x7fbaf806e5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:18.611 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:18.610+0000 7fbaf6ffd640 1 -- 172.21.15.27:0/3305825695 >> v1:172.21.15.45:6789/0 conn(0x7fbaf8160d80 legacy=0x7fbaf806bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:18.612 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:18.610+0000 7fbaf6ffd640 1 -- 172.21.15.27:0/3305825695 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbaf81602a0 con 0x7fbaf80ce6d0 2024-09-12T03:17:18.612 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:18.610+0000 7fbaf5ffb640 1 -- 172.21.15.27:0/3305825695 <== mon.1 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1565745822 0 0) 0x7fbaf81463d0 con 0x7fbaf8160d80 2024-09-12T03:17:18.612 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:18.610+0000 7fbaf77fe640 1 --2- 172.21.15.27:0/3305825695 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fbaf806c240 0x7fbaf806e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:17:18.612 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:18.610+0000 7fbaf6ffd640 1 --2- 172.21.15.27:0/3305825695 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbaf80ce6d0 0x7fbaf80ceab0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fbae8009d90 tx=0x7fbae802a5d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1756602d722f6f3e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:18.612 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:18.611+0000 7fbaf5ffb640 1 -- 172.21.15.27:0/3305825695 <== mon.1 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbae8031600 con 0x7fbaf80ce6d0 2024-09-12T03:17:18.612 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:18.611+0000 7fbaf5ffb640 1 -- 172.21.15.27:0/3305825695 <== mon.1 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbae80317a0 con 0x7fbaf80ce6d0 2024-09-12T03:17:18.612 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:18.611+0000 7fbaf5ffb640 1 -- 172.21.15.27:0/3305825695 <== mon.1 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbae8031b70 con 0x7fbaf80ce6d0 2024-09-12T03:17:18.612 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:18.611+0000 7fbafd85a640 1 -- 172.21.15.27:0/3305825695 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbaf80ce6d0 msgr2=0x7fbaf80ceab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:18.612 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:18.611+0000 7fbafd85a640 1 --2- 172.21.15.27:0/3305825695 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbaf80ce6d0 0x7fbaf80ceab0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fbae8009d90 tx=0x7fbae802a5d0 comp rx=0 tx=0).stop 2024-09-12T03:17:18.612 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:18.611+0000 7fbafd85a640 1 -- 172.21.15.27:0/3305825695 shutdown_connections 2024-09-12T03:17:18.612 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:18.611+0000 7fbafd85a640 1 --2- 172.21.15.27:0/3305825695 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fbaf806c240 0x7fbaf806e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:18.612 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:18.611+0000 7fbafd85a640 1 --2- 172.21.15.27:0/3305825695 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbaf80ce6d0 0x7fbaf80ceab0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:18.613 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:18.611+0000 7fbafd85a640 1 -- 172.21.15.27:0/3305825695 >> 172.21.15.27:0/3305825695 conn(0x7fbaf806cc40 msgr2=0x7fbaf806d050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:18.613 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:18.611+0000 7fbafd85a640 1 -- 172.21.15.27:0/3305825695 shutdown_connections 2024-09-12T03:17:18.613 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:18.611+0000 7fbafd85a640 1 -- 172.21.15.27:0/3305825695 wait complete. 2024-09-12T03:17:18.615 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:18.612+0000 7fbafd85a640 1 Processor -- start 2024-09-12T03:17:18.615 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:18.613+0000 7fbafd85a640 1 -- start start 2024-09-12T03:17:18.615 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:18.613+0000 7fbafd85a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbaf806c240 0x7fbaf8141eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:18.615 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:18.613+0000 7fbafd85a640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fbaf80ce6d0 0x7fbaf80b4700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:18.615 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:18.613+0000 7fbafd85a640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbaf8160d80 0x7fbaf8144850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:18.615 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:18.613+0000 7fbafd85a640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fbaf806eaf0 con 0x7fbaf8160d80 2024-09-12T03:17:18.615 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:18.613+0000 7fbafd85a640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fbaf8160860 con 0x7fbaf806c240 2024-09-12T03:17:18.615 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:18.613+0000 7fbafd85a640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fbaf806ec70 con 0x7fbaf80ce6d0 2024-09-12T03:17:18.615 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:18.614+0000 7fbaf77fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbaf8160d80 0x7fbaf8144850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:18.615 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:18.614+0000 7fbaf77fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbaf8160d80 0x7fbaf8144850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37106/0 (socket says 172.21.15.27:37106) 2024-09-12T03:17:18.615 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:18.614+0000 7fbaf77fe640 1 -- 172.21.15.27:0/4037020151 learned_addr learned my addr 172.21.15.27:0/4037020151 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:17:18.616 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:18.614+0000 7fbaf67fc640 1 --2- 172.21.15.27:0/4037020151 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fbaf80ce6d0 0x7fbaf80b4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:18.616 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:18.614+0000 7fbaf6ffd640 1 --2- 172.21.15.27:0/4037020151 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbaf806c240 0x7fbaf8141eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:18.616 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:18.614+0000 7fbaf77fe640 1 -- 172.21.15.27:0/4037020151 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fbaf80ce6d0 msgr2=0x7fbaf80b4700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:18.616 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:18.614+0000 7fbaf77fe640 1 --2- 172.21.15.27:0/4037020151 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fbaf80ce6d0 0x7fbaf80b4700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:18.616 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:18.614+0000 7fbaf77fe640 1 -- 172.21.15.27:0/4037020151 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbaf806c240 msgr2=0x7fbaf8141eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:18.616 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:18.614+0000 7fbaf77fe640 1 --2- 172.21.15.27:0/4037020151 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbaf806c240 0x7fbaf8141eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:18.616 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:18.614+0000 7fbaf77fe640 1 -- 172.21.15.27:0/4037020151 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbaf80b5000 con 0x7fbaf8160d80 2024-09-12T03:17:18.616 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:18.614+0000 7fbaf77fe640 1 --2- 172.21.15.27:0/4037020151 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbaf8160d80 0x7fbaf8144850 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fbaf000e960 tx=0x7fbaf000ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:18.616 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:18.614+0000 7fbaeffff640 1 -- 172.21.15.27:0/4037020151 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaf0017070 con 0x7fbaf8160d80 2024-09-12T03:17:18.616 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:18.614+0000 7fbaeffff640 1 -- 172.21.15.27:0/4037020151 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbaf0004020 con 0x7fbaf8160d80 2024-09-12T03:17:18.616 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:18.614+0000 7fbafd85a640 1 -- 172.21.15.27:0/4037020151 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbaf80b52e0 con 0x7fbaf8160d80 2024-09-12T03:17:18.616 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:18.614+0000 7fbaeffff640 1 -- 172.21.15.27:0/4037020151 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaf00043f0 con 0x7fbaf8160d80 2024-09-12T03:17:18.616 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:18.614+0000 7fbafd85a640 1 -- 172.21.15.27:0/4037020151 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbaf80b58a0 con 0x7fbaf8160d80 2024-09-12T03:17:18.617 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:18.615+0000 7fbafd85a640 1 -- 172.21.15.27:0/4037020151 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbaf80b0f50 con 0x7fbaf8160d80 2024-09-12T03:17:18.617 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:18.616+0000 7fbaeffff640 1 -- 172.21.15.27:0/4037020151 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fbaf000c9a0 con 0x7fbaf8160d80 2024-09-12T03:17:18.617 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:18.616+0000 7fbaeffff640 1 --2- 172.21.15.27:0/4037020151 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fbad806be60 0x7fbad806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:18.618 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:18.617+0000 7fbaeffff640 1 -- 172.21.15.27:0/4037020151 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7fbaf008aed0 con 0x7fbaf8160d80 2024-09-12T03:17:18.618 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:18.617+0000 7fbaf6ffd640 1 --2- 172.21.15.27:0/4037020151 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fbad806be60 0x7fbad806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:18.618 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:18.617+0000 7fbaf6ffd640 1 --2- 172.21.15.27:0/4037020151 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fbad806be60 0x7fbad806e320 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fbae8034000 tx=0x7fbae802f000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:18.620 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:18.621+0000 7fbaeffff640 1 -- 172.21.15.27:0/4037020151 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fbaf00592c0 con 0x7fbaf8160d80 2024-09-12T03:17:18.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:18.833+0000 7fbafd85a640 1 -- 172.21.15.27:0/4037020151 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fbaf815e1c0 con 0x7fbaf8160d80 2024-09-12T03:17:18.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:18.834+0000 7fbaeffff640 1 -- 172.21.15.27:0/4037020151 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v14) ==== 53+0+73 (secure 0 0 0) 0x7fbaf005d5c0 con 0x7fbaf8160d80 2024-09-12T03:17:18.834 INFO:teuthology.orchestra.run.smithi027.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-12T03:17:18.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:18.835+0000 7fbafd85a640 1 -- 172.21.15.27:0/4037020151 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fbad806be60 msgr2=0x7fbad806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:18.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:18.835+0000 7fbafd85a640 1 --2- 172.21.15.27:0/4037020151 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fbad806be60 0x7fbad806e320 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fbae8034000 tx=0x7fbae802f000 comp rx=0 tx=0).stop 2024-09-12T03:17:18.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:18.835+0000 7fbafd85a640 1 -- 172.21.15.27:0/4037020151 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbaf8160d80 msgr2=0x7fbaf8144850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:18.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:18.835+0000 7fbafd85a640 1 --2- 172.21.15.27:0/4037020151 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbaf8160d80 0x7fbaf8144850 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fbaf000e960 tx=0x7fbaf000ee20 comp rx=0 tx=0).stop 2024-09-12T03:17:18.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:18.835+0000 7fbafd85a640 1 -- 172.21.15.27:0/4037020151 shutdown_connections 2024-09-12T03:17:18.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:18.835+0000 7fbafd85a640 1 --2- 172.21.15.27:0/4037020151 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fbad806be60 0x7fbad806e320 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:18.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:18.835+0000 7fbafd85a640 1 --2- 172.21.15.27:0/4037020151 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbaf8160d80 0x7fbaf8144850 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:18.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:18.835+0000 7fbafd85a640 1 --2- 172.21.15.27:0/4037020151 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fbaf80ce6d0 0x7fbaf80b4700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:18.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:18.835+0000 7fbafd85a640 1 --2- 172.21.15.27:0/4037020151 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbaf806c240 0x7fbaf8141eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:18.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:18.835+0000 7fbafd85a640 1 -- 172.21.15.27:0/4037020151 >> 172.21.15.27:0/4037020151 conn(0x7fbaf806cc40 msgr2=0x7fbaf80cf200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:18.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:18.836+0000 7fbafd85a640 1 -- 172.21.15.27:0/4037020151 shutdown_connections 2024-09-12T03:17:18.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:18.836+0000 7fbafd85a640 1 -- 172.21.15.27:0/4037020151 wait complete. 2024-09-12T03:17:18.846 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-12T03:17:18.847 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-12T03:17:18.847 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi045.front.sepia.ceph.com 2024-09-12T03:17:18.847 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-12T03:17:18.847 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-12T03:17:18.847 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-12T03:17:18.847 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-12T03:17:18.847 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-12T03:17:18.847 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-12T03:17:18.847 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:17:18.847 DEBUG:teuthology.orchestra.run.smithi045:> ip addr 2024-09-12T03:17:18.868 INFO:teuthology.orchestra.run.smithi045.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-12T03:17:18.868 INFO:teuthology.orchestra.run.smithi045.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-12T03:17:18.868 INFO:teuthology.orchestra.run.smithi045.stdout: inet 127.0.0.1/8 scope host lo 2024-09-12T03:17:18.868 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:17:18.868 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 ::1/128 scope host 2024-09-12T03:17:18.868 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:17:18.868 INFO:teuthology.orchestra.run.smithi045.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-12T03:17:18.868 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:6b:fd:a6 brd ff:ff:ff:ff:ff:ff 2024-09-12T03:17:18.868 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp7s0f0 2024-09-12T03:17:18.869 INFO:teuthology.orchestra.run.smithi045.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-12T03:17:18.869 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:6b:fd:a7 brd ff:ff:ff:ff:ff:ff 2024-09-12T03:17:18.869 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp7s0f1 2024-09-12T03:17:18.869 INFO:teuthology.orchestra.run.smithi045.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-12T03:17:18.869 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:bd:12:be brd ff:ff:ff:ff:ff:ff 2024-09-12T03:17:18.869 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp1s0f0 2024-09-12T03:17:18.869 INFO:teuthology.orchestra.run.smithi045.stdout: inet 172.21.15.45/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-12T03:17:18.869 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft 42695sec preferred_lft 42695sec 2024-09-12T03:17:18.869 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::ec4:7aff:febd:12be/64 scope link 2024-09-12T03:17:18.869 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:17:18.870 INFO:teuthology.orchestra.run.smithi045.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-12T03:17:18.870 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:bd:12:bf brd ff:ff:ff:ff:ff:ff 2024-09-12T03:17:18.870 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp1s0f1 2024-09-12T03:17:18.870 INFO:teuthology.orchestra.run.smithi045.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-12T03:17:18.870 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 52:97:33:5c:60:1b brd ff:ff:ff:ff:ff:ff 2024-09-12T03:17:18.870 INFO:teuthology.orchestra.run.smithi045.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-12T03:17:18.870 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:17:18.870 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::d8c2:5dff:fe26:78c8/64 scope link 2024-09-12T03:17:18.870 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:17:18.871 INFO:teuthology.orchestra.run.smithi045.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-12T03:17:18.871 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 5a:ec:d7:40:ef:75 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-12T03:17:18.871 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::58ec:d7ff:fe40:ef75/64 scope link 2024-09-12T03:17:18.871 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:17:18.871 INFO:teuthology.orchestra.run.smithi045.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-12T03:17:18.871 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 52:97:33:5c:60:1b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-12T03:17:18.871 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::5097:33ff:fe5c:601b/64 scope link 2024-09-12T03:17:18.871 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:17:18.871 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:17:18.871 DEBUG:teuthology.orchestra.run.smithi045:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-12T03:17:18.937 INFO:teuthology.orchestra.run.smithi045.stdout:1 2024-09-12T03:17:18.939 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:17:18.940 DEBUG:teuthology.orchestra.run.smithi045:> route 2024-09-12T03:17:18.999 INFO:teuthology.orchestra.run.smithi045.stdout:Kernel IP routing table 2024-09-12T03:17:18.999 INFO:teuthology.orchestra.run.smithi045.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-12T03:17:19.000 INFO:teuthology.orchestra.run.smithi045.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-12T03:17:19.000 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-12T03:17:19.000 INFO:teuthology.orchestra.run.smithi045.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-12T03:17:19.000 DEBUG:teuthology.orchestra.run.smithi045:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-12T03:17:19.000 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-09-12T03:17:19.000 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-12T03:17:19.001 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-12T03:17:19.001 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-12T03:17:19.001 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-09-12T03:17:19.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:17:19 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T03:17:19.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:17:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T03:17:19.137 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:17:19.138 DEBUG:teuthology.orchestra.run.smithi045:> ip netns list 2024-09-12T03:17:19.154 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-09-12T03:17:19.154 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-12T03:17:19.154 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-12T03:17:19.154 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:17:19.154 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link set brx.0 up 2024-09-12T03:17:19.222 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-12T03:17:19.222 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-12T03:17:19.278 INFO:teuthology.orchestra.run.smithi045.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-12T03:17:19.278 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-12T03:17:19.279 DEBUG:teuthology.orchestra.run.smithi045:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-12T03:17:19.335 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2024-09-12T03:17:19.335 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:17:19.335 DEBUG:teuthology.orchestra.run.smithi045:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-12T03:17:19.420 INFO:teuthology.orchestra.run.smithi045.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-12T03:17:19.421 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-12T03:17:19.428 INFO:teuthology.orchestra.run.smithi045.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-12T03:17:19.428 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-12T03:17:19.429 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: resolved to: "172.21.15.27:3300,172.21.15.45:3300,172.21.15.45:3301" 2024-09-12T03:17:19.429 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: trying mount with new device syntax: 1@d2df504e-2647-4385-a5db-9fdf69f487a0.cephfs=/ 2024-09-12T03:17:19.429 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.27:3300/172.21.15.45:3300/172.21.15.45:3301" will pass to kernel 2024-09-12T03:17:19.429 INFO:teuthology.orchestra.run.smithi045.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-09-12T03:17:19.429 INFO:teuthology.orchestra.run.smithi045.stdout: You just mounted a file system that supports labels which does not 2024-09-12T03:17:19.429 INFO:teuthology.orchestra.run.smithi045.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-12T03:17:19.429 INFO:teuthology.orchestra.run.smithi045.stdout: applications will generate AVC messages and not be allowed access to 2024-09-12T03:17:19.429 INFO:teuthology.orchestra.run.smithi045.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-12T03:17:19.431 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-12T03:17:19.431 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:17:19.431 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-12T03:17:19.457 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-12T03:17:19.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:17:19 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-12T03:17:19.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:17:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T03:17:19.555 INFO:teuthology.orchestra.run.smithi045.stdout:client4876 2024-09-12T03:17:19.555 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-12T03:17:19.556 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T03:17:19.572 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-09-12T03:17:19.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:19.685+0000 7f97fcb86640 1 Processor -- start 2024-09-12T03:17:19.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:19.685+0000 7f97fcb86640 1 -- start start 2024-09-12T03:17:19.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:19.686+0000 7f97fcb86640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f97f80d31c0 0x7f97f80d0440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:19.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:19.686+0000 7f97fcb86640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f97f806e960 0x7f97f80d0980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:19.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:19.686+0000 7f97fcb86640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f97f80cf9f0 con 0x7f97f8070d30 2024-09-12T03:17:19.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:19.686+0000 7f97fcb86640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f97f80d1a40 con 0x7f97f806e960 2024-09-12T03:17:19.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:19.686+0000 7f97fcb86640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f97f80cfc90 con 0x7f97f80d31c0 2024-09-12T03:17:19.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:19.686+0000 7f97f6d9d640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f97f806e960 0x7f97f80d0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:19.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:19.686+0000 7f97f659c640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f97f8070d30 0x7f97f806cc40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:51344/0 (socket says 172.21.15.27:51344) 2024-09-12T03:17:19.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:19.686+0000 7f97f659c640 1 -- 172.21.15.27:0/3809343202 learned_addr learned my addr 172.21.15.27:0/3809343202 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:17:19.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:19.686+0000 7f97f6d9d640 1 -- 172.21.15.27:0/3809343202 >> v2:172.21.15.45:3300/0 conn(0x7f97f80d31c0 msgr2=0x7f97f80d0440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:19.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:19.686+0000 7f97f5d9b640 1 --2- 172.21.15.27:0/3809343202 >> v2:172.21.15.45:3300/0 conn(0x7f97f80d31c0 0x7f97f80d0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:19.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:19.686+0000 7f97f559a640 1 -- 172.21.15.27:0/3809343202 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2163636148 0 0) 0x7f97f80cf9f0 con 0x7f97f8070d30 2024-09-12T03:17:19.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:19.686+0000 7f97f6d9d640 1 --2- 172.21.15.27:0/3809343202 >> v2:172.21.15.45:3300/0 conn(0x7f97f80d31c0 0x7f97f80d0440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:19.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:19.686+0000 7f97f6d9d640 1 -- 172.21.15.27:0/3809343202 >> v1:172.21.15.27:6789/0 conn(0x7f97f8070d30 legacy=0x7f97f806cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:19.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:19.686+0000 7f97f6d9d640 1 -- 172.21.15.27:0/3809343202 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97f80cff70 con 0x7f97f806e960 2024-09-12T03:17:19.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:19.686+0000 7f97f5d9b640 1 --2- 172.21.15.27:0/3809343202 >> v2:172.21.15.45:3300/0 conn(0x7f97f80d31c0 0x7f97f80d0440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:17:19.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:19.687+0000 7f97f6d9d640 1 --2- 172.21.15.27:0/3809343202 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f97f806e960 0x7f97f80d0980 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f97f80d1bc0 tx=0x7f97f002a540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=377ec46d69298c60 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:19.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:19.687+0000 7f97f559a640 1 -- 172.21.15.27:0/3809343202 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97f0007740 con 0x7f97f806e960 2024-09-12T03:17:19.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:19.687+0000 7f97f559a640 1 -- 172.21.15.27:0/3809343202 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97f00078e0 con 0x7f97f806e960 2024-09-12T03:17:19.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:19.687+0000 7f97f559a640 1 -- 172.21.15.27:0/3809343202 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97f0007cb0 con 0x7f97f806e960 2024-09-12T03:17:19.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:19.688+0000 7f97fcb86640 1 -- 172.21.15.27:0/3809343202 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f97f806e960 msgr2=0x7f97f80d0980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:19.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:19.688+0000 7f97fcb86640 1 --2- 172.21.15.27:0/3809343202 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f97f806e960 0x7f97f80d0980 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f97f80d1bc0 tx=0x7f97f002a540 comp rx=0 tx=0).stop 2024-09-12T03:17:19.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:19.688+0000 7f97fcb86640 1 -- 172.21.15.27:0/3809343202 shutdown_connections 2024-09-12T03:17:19.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:19.688+0000 7f97fcb86640 1 --2- 172.21.15.27:0/3809343202 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f97f806e960 0x7f97f80d0980 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:19.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:19.688+0000 7f97fcb86640 1 --2- 172.21.15.27:0/3809343202 >> v2:172.21.15.45:3300/0 conn(0x7f97f80d31c0 0x7f97f80d0440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:19.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:19.688+0000 7f97fcb86640 1 -- 172.21.15.27:0/3809343202 >> 172.21.15.27:0/3809343202 conn(0x7f97f80b7c10 msgr2=0x7f97f809ba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:19.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:19.688+0000 7f97fcb86640 1 -- 172.21.15.27:0/3809343202 shutdown_connections 2024-09-12T03:17:19.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:19.688+0000 7f97fcb86640 1 -- 172.21.15.27:0/3809343202 wait complete. 2024-09-12T03:17:19.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:19.689+0000 7f97fcb86640 1 Processor -- start 2024-09-12T03:17:19.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:19.689+0000 7f97fcb86640 1 -- start start 2024-09-12T03:17:19.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:19.690+0000 7f97fcb86640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f97f806e960 0x7f97f81513f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:19.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:19.690+0000 7f97fcb86640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f97f8070d30 0x7f97f8151c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:19.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:19.690+0000 7f97fcb86640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f97f80d31c0 0x7f97f80b25c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:19.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:19.690+0000 7f97fcb86640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f97f8168480 con 0x7f97f806e960 2024-09-12T03:17:19.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:19.690+0000 7f97fcb86640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f97f8168300 con 0x7f97f80d31c0 2024-09-12T03:17:19.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:19.690+0000 7f97fcb86640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f97f8168600 con 0x7f97f8070d30 2024-09-12T03:17:19.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:19.690+0000 7f97f659c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f97f806e960 0x7f97f81513f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:19.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:19.690+0000 7f97f659c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f97f806e960 0x7f97f81513f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59928/0 (socket says 172.21.15.27:59928) 2024-09-12T03:17:19.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:19.690+0000 7f97f659c640 1 -- 172.21.15.27:0/480880175 learned_addr learned my addr 172.21.15.27:0/480880175 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:17:19.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:19.690+0000 7f97f6d9d640 1 --2- 172.21.15.27:0/480880175 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f97f80d31c0 0x7f97f80b25c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:19.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:19.690+0000 7f97f5d9b640 1 --2- 172.21.15.27:0/480880175 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f97f8070d30 0x7f97f8151c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:19.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:19.690+0000 7f97f659c640 1 -- 172.21.15.27:0/480880175 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f97f8070d30 msgr2=0x7f97f8151c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:19.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:19.690+0000 7f97f659c640 1 --2- 172.21.15.27:0/480880175 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f97f8070d30 0x7f97f8151c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:19.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:19.690+0000 7f97f659c640 1 -- 172.21.15.27:0/480880175 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f97f80d31c0 msgr2=0x7f97f80b25c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:19.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:19.690+0000 7f97f659c640 1 --2- 172.21.15.27:0/480880175 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f97f80d31c0 0x7f97f80b25c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:19.693 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:19.690+0000 7f97f659c640 1 -- 172.21.15.27:0/480880175 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97f815a8d0 con 0x7f97f806e960 2024-09-12T03:17:19.693 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:19.690+0000 7f97f5d9b640 1 --2- 172.21.15.27:0/480880175 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f97f8070d30 0x7f97f8151c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:17:19.693 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:19.690+0000 7f97f659c640 1 --2- 172.21.15.27:0/480880175 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f97f806e960 0x7f97f81513f0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f97e800c4d0 tx=0x7f97e800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:19.693 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:19.691+0000 7f97e6ffd640 1 -- 172.21.15.27:0/480880175 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97e8018070 con 0x7f97f806e960 2024-09-12T03:17:19.693 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:19.691+0000 7f97e6ffd640 1 -- 172.21.15.27:0/480880175 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97e8007500 con 0x7f97f806e960 2024-09-12T03:17:19.693 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:19.691+0000 7f97fcb86640 1 -- 172.21.15.27:0/480880175 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97f815abb0 con 0x7f97f806e960 2024-09-12T03:17:19.693 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:19.691+0000 7f97e6ffd640 1 -- 172.21.15.27:0/480880175 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97e8004a20 con 0x7f97f806e960 2024-09-12T03:17:19.693 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:19.691+0000 7f97fcb86640 1 -- 172.21.15.27:0/480880175 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97f815a5c0 con 0x7f97f806e960 2024-09-12T03:17:19.693 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:19.692+0000 7f97fcb86640 1 -- 172.21.15.27:0/480880175 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97f806cc40 con 0x7f97f806e960 2024-09-12T03:17:19.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:19.692+0000 7f97e6ffd640 1 -- 172.21.15.27:0/480880175 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f97e801b070 con 0x7f97f806e960 2024-09-12T03:17:19.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:19.693+0000 7f97e6ffd640 1 --2- 172.21.15.27:0/480880175 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f97d806be30 0x7f97d806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:19.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:19.693+0000 7f97e6ffd640 1 -- 172.21.15.27:0/480880175 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f97e808a150 con 0x7f97f806e960 2024-09-12T03:17:19.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:19.693+0000 7f97f5d9b640 1 --2- 172.21.15.27:0/480880175 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f97d806be30 0x7f97d806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:19.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:19.693+0000 7f97f5d9b640 1 --2- 172.21.15.27:0/480880175 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f97d806be30 0x7f97d806e2f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f97ec006fd0 tx=0x7f97ec008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:19.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:19.697+0000 7f97e6ffd640 1 -- 172.21.15.27:0/480880175 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f97e8058b90 con 0x7f97f806e960 2024-09-12T03:17:19.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:19.913+0000 7f97fcb86640 1 -- 172.21.15.27:0/480880175 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7f97f80cbce0 con 0x7f97f806e960 2024-09-12T03:17:20.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:20.792+0000 7f97e6ffd640 1 -- 172.21.15.27:0/480880175 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v15) ==== 136+0+0 (secure 0 0 0) 0x7f97e8058490 con 0x7f97f806e960 2024-09-12T03:17:20.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:20.795+0000 7f97fcb86640 1 -- 172.21.15.27:0/480880175 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f97d806be30 msgr2=0x7f97d806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:20.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:20.795+0000 7f97fcb86640 1 --2- 172.21.15.27:0/480880175 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f97d806be30 0x7f97d806e2f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f97ec006fd0 tx=0x7f97ec008040 comp rx=0 tx=0).stop 2024-09-12T03:17:20.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:20.795+0000 7f97fcb86640 1 -- 172.21.15.27:0/480880175 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f97f806e960 msgr2=0x7f97f81513f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:20.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:20.795+0000 7f97fcb86640 1 --2- 172.21.15.27:0/480880175 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f97f806e960 0x7f97f81513f0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f97e800c4d0 tx=0x7f97e800c990 comp rx=0 tx=0).stop 2024-09-12T03:17:20.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:20.795+0000 7f97fcb86640 1 -- 172.21.15.27:0/480880175 shutdown_connections 2024-09-12T03:17:20.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:20.795+0000 7f97fcb86640 1 --2- 172.21.15.27:0/480880175 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f97d806be30 0x7f97d806e2f0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:20.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:20.795+0000 7f97fcb86640 1 --2- 172.21.15.27:0/480880175 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f97f80d31c0 0x7f97f80b25c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:20.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:20.795+0000 7f97fcb86640 1 --2- 172.21.15.27:0/480880175 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f97f8070d30 0x7f97f8151c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:20.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:20.795+0000 7f97fcb86640 1 --2- 172.21.15.27:0/480880175 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f97f806e960 0x7f97f81513f0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:20.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:20.795+0000 7f97fcb86640 1 -- 172.21.15.27:0/480880175 >> 172.21.15.27:0/480880175 conn(0x7f97f80b7c10 msgr2=0x7f97f80af4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:20.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:20.795+0000 7f97fcb86640 1 -- 172.21.15.27:0/480880175 shutdown_connections 2024-09-12T03:17:20.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:20.795+0000 7f97fcb86640 1 -- 172.21.15.27:0/480880175 wait complete. 2024-09-12T03:17:20.810 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-12T03:17:20.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:20.919+0000 7f229a752640 1 Processor -- start 2024-09-12T03:17:20.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:20.919+0000 7f229a752640 1 -- start start 2024-09-12T03:17:20.923 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:20.920+0000 7f229a752640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f229406f360 0x7f22940cef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:20.923 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:20.920+0000 7f229a752640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f229406cc40 con 0x7f22940b1ca0 2024-09-12T03:17:20.923 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:20.920+0000 7f229a752640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f22940d1ce0 con 0x7f22940cf490 2024-09-12T03:17:20.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:20.920+0000 7f229a752640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f22940d22f0 con 0x7f229406f360 2024-09-12T03:17:20.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:20.920+0000 7f2293fff640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f22940b1ca0 0x7f229415f830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:51354/0 (socket says 172.21.15.27:51354) 2024-09-12T03:17:20.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:20.920+0000 7f2293fff640 1 -- 172.21.15.27:0/199453851 learned_addr learned my addr 172.21.15.27:0/199453851 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:17:20.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:20.921+0000 7f22937fe640 1 --2- 172.21.15.27:0/199453851 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f229406f360 0x7f22940cef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:20.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:20.921+0000 7f2292ffd640 1 -- 172.21.15.27:0/199453851 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3889996655 0 0) 0x7f229406cc40 con 0x7f22940b1ca0 2024-09-12T03:17:20.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:20.921+0000 7f2292ffd640 1 -- 172.21.15.27:0/199453851 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f227c003520 con 0x7f22940b1ca0 2024-09-12T03:17:20.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:20.921+0000 7f2292ffd640 1 -- 172.21.15.27:0/199453851 <== mon.1 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3215708092 0 0) 0x7f22940d1ce0 con 0x7f22940cf490 2024-09-12T03:17:20.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:20.921+0000 7f2292ffd640 1 -- 172.21.15.27:0/199453851 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f229406cc40 con 0x7f22940cf490 2024-09-12T03:17:20.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:20.921+0000 7f2292ffd640 1 -- 172.21.15.27:0/199453851 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3475657880 0 0) 0x7f227c003520 con 0x7f22940b1ca0 2024-09-12T03:17:20.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:20.921+0000 7f2292ffd640 1 -- 172.21.15.27:0/199453851 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f229406f360 msgr2=0x7f22940cef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:20.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:20.921+0000 7f2292ffd640 1 --2- 172.21.15.27:0/199453851 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f229406f360 0x7f22940cef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:20.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:20.921+0000 7f2292ffd640 1 -- 172.21.15.27:0/199453851 >> v1:172.21.15.45:6789/0 conn(0x7f22940cf490 legacy=0x7f22940d2780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:20.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:20.921+0000 7f2292ffd640 1 -- 172.21.15.27:0/199453851 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f229406cee0 con 0x7f22940b1ca0 2024-09-12T03:17:20.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:20.921+0000 7f2292ffd640 1 -- 172.21.15.27:0/199453851 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 847755213 0 0) 0x7f2284001670 con 0x7f22940b1ca0 2024-09-12T03:17:20.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:20.921+0000 7f2292ffd640 1 -- 172.21.15.27:0/199453851 >> v1:172.21.15.27:6789/0 conn(0x7f22940b1ca0 legacy=0x7f229415f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:20.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:20.921+0000 7f2292ffd640 1 --2- 172.21.15.27:0/199453851 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f229406f360 0x7f227c003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:20.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:20.921+0000 7f2292ffd640 1 --2- 172.21.15.27:0/199453851 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f227c0040d0 0x7f227c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:20.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:20.921+0000 7f2292ffd640 1 --2- 172.21.15.27:0/199453851 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f227c004d40 0x7f227c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:20.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:20.921+0000 7f2292ffd640 1 -- 172.21.15.27:0/199453851 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f22940d22f0 con 0x7f227c004d40 2024-09-12T03:17:20.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:20.921+0000 7f2292ffd640 1 -- 172.21.15.27:0/199453851 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f227c00baa0 con 0x7f227c0040d0 2024-09-12T03:17:20.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:20.921+0000 7f2292ffd640 1 -- 172.21.15.27:0/199453851 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f227c00bca0 con 0x7f229406f360 2024-09-12T03:17:20.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:20.921+0000 7f2298cef640 1 --2- 172.21.15.27:0/199453851 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f227c004d40 0x7f227c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:20.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:20.921+0000 7f2293fff640 1 --2- 172.21.15.27:0/199453851 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f229406f360 0x7f227c003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:20.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:20.921+0000 7f22937fe640 1 --2- 172.21.15.27:0/199453851 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f227c0040d0 0x7f227c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:20.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:20.921+0000 7f2298cef640 1 -- 172.21.15.27:0/199453851 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f229406f360 msgr2=0x7f227c003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:20.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:20.921+0000 7f2298cef640 1 --2- 172.21.15.27:0/199453851 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f229406f360 0x7f227c003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:20.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:20.921+0000 7f2298cef640 1 -- 172.21.15.27:0/199453851 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f227c0040d0 msgr2=0x7f227c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:20.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:20.921+0000 7f2298cef640 1 --2- 172.21.15.27:0/199453851 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f227c0040d0 0x7f227c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:20.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:20.922+0000 7f2298cef640 1 -- 172.21.15.27:0/199453851 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f227c003520 con 0x7f227c004d40 2024-09-12T03:17:20.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:20.922+0000 7f2298cef640 1 --2- 172.21.15.27:0/199453851 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f227c004d40 0x7f227c0051c0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f228c001fd0 tx=0x7f227c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d10471fdb1ba75e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:20.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:20.922+0000 7f2292ffd640 1 -- 172.21.15.27:0/199453851 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f228c001bf0 con 0x7f227c004d40 2024-09-12T03:17:20.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:20.922+0000 7f2292ffd640 1 -- 172.21.15.27:0/199453851 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f228c003b70 con 0x7f227c004d40 2024-09-12T03:17:20.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:20.922+0000 7f2292ffd640 1 -- 172.21.15.27:0/199453851 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f228c0050d0 con 0x7f227c004d40 2024-09-12T03:17:20.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:20.923+0000 7f229a752640 1 -- 172.21.15.27:0/199453851 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f227c004d40 msgr2=0x7f227c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:20.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:20.923+0000 7f229a752640 1 --2- 172.21.15.27:0/199453851 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f227c004d40 0x7f227c0051c0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f228c001fd0 tx=0x7f227c005830 comp rx=0 tx=0).stop 2024-09-12T03:17:20.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:20.923+0000 7f2298cef640 1 -- 172.21.15.27:0/199453851 reap_dead start 2024-09-12T03:17:20.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:20.923+0000 7f229a752640 1 -- 172.21.15.27:0/199453851 shutdown_connections 2024-09-12T03:17:20.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:20.923+0000 7f229a752640 1 -- 172.21.15.27:0/199453851 >> 172.21.15.27:0/199453851 conn(0x7f2294073800 msgr2=0x7f229409ba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:20.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:20.923+0000 7f229a752640 1 -- 172.21.15.27:0/199453851 shutdown_connections 2024-09-12T03:17:20.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:20.923+0000 7f229a752640 1 -- 172.21.15.27:0/199453851 wait complete. 2024-09-12T03:17:20.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:20.924+0000 7f229a752640 1 Processor -- start 2024-09-12T03:17:20.930 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:20.924+0000 7f229a752640 1 -- start start 2024-09-12T03:17:20.930 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:20.925+0000 7f229a752640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f227c004d40 0x7f22941f3910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:20.930 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:20.925+0000 7f229a752640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f229414f490 0x7f2294160d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:20.930 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:20.925+0000 7f229a752640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f22941612d0 0x7f22940b2110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:20.930 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:20.925+0000 7f229a752640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f229406d340 con 0x7f22941612d0 2024-09-12T03:17:20.930 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:20.925+0000 7f229a752640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f229406d1c0 con 0x7f229414f490 2024-09-12T03:17:20.930 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:20.925+0000 7f229a752640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f22940cfc40 con 0x7f227c004d40 2024-09-12T03:17:20.930 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:20.925+0000 7f2298cef640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f22941612d0 0x7f22940b2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:20.930 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:20.925+0000 7f2298cef640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f22941612d0 0x7f22940b2110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59942/0 (socket says 172.21.15.27:59942) 2024-09-12T03:17:20.930 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:20.925+0000 7f2298cef640 1 -- 172.21.15.27:0/3944249944 learned_addr learned my addr 172.21.15.27:0/3944249944 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:17:20.930 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:20.926+0000 7f2293fff640 1 --2- 172.21.15.27:0/3944249944 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f227c004d40 0x7f22941f3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:20.930 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:20.926+0000 7f22937fe640 1 --2- 172.21.15.27:0/3944249944 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f229414f490 0x7f2294160d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:20.930 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:20.926+0000 7f2298cef640 1 -- 172.21.15.27:0/3944249944 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f227c004d40 msgr2=0x7f22941f3910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:20.930 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:20.926+0000 7f2298cef640 1 --2- 172.21.15.27:0/3944249944 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f227c004d40 0x7f22941f3910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:20.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:20.926+0000 7f2298cef640 1 -- 172.21.15.27:0/3944249944 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f229414f490 msgr2=0x7f2294160d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:20.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:20.926+0000 7f2298cef640 1 --2- 172.21.15.27:0/3944249944 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f229414f490 0x7f2294160d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:20.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:20.926+0000 7f2298cef640 1 -- 172.21.15.27:0/3944249944 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2294152a50 con 0x7f22941612d0 2024-09-12T03:17:20.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:20.926+0000 7f2298cef640 1 --2- 172.21.15.27:0/3944249944 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f22941612d0 0x7f22940b2110 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f228c033040 tx=0x7f228c001650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:20.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:20.926+0000 7f2290ff9640 1 -- 172.21.15.27:0/3944249944 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f228c001bf0 con 0x7f22941612d0 2024-09-12T03:17:20.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:20.926+0000 7f2290ff9640 1 -- 172.21.15.27:0/3944249944 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f228c043070 con 0x7f22941612d0 2024-09-12T03:17:20.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:20.926+0000 7f2290ff9640 1 -- 172.21.15.27:0/3944249944 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f228c005610 con 0x7f22941612d0 2024-09-12T03:17:20.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:20.926+0000 7f229a752640 1 -- 172.21.15.27:0/3944249944 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2294161900 con 0x7f22941612d0 2024-09-12T03:17:20.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:20.926+0000 7f229a752640 1 -- 172.21.15.27:0/3944249944 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f229414ec90 con 0x7f22941612d0 2024-09-12T03:17:20.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:20.927+0000 7f229a752640 1 -- 172.21.15.27:0/3944249944 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f229406f7a0 con 0x7f22941612d0 2024-09-12T03:17:20.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:20.928+0000 7f2290ff9640 1 -- 172.21.15.27:0/3944249944 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f228c03a040 con 0x7f22941612d0 2024-09-12T03:17:20.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:20.928+0000 7f2290ff9640 1 --2- 172.21.15.27:0/3944249944 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f228006be60 0x7f228006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:20.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:20.928+0000 7f2290ff9640 1 -- 172.21.15.27:0/3944249944 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f228c0ab390 con 0x7f22941612d0 2024-09-12T03:17:20.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:20.928+0000 7f2293fff640 1 --2- 172.21.15.27:0/3944249944 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f228006be60 0x7f228006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:20.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:20.929+0000 7f2293fff640 1 --2- 172.21.15.27:0/3944249944 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f228006be60 0x7f228006e320 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f2284007c20 tx=0x7f2284009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:20.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:20.932+0000 7f2290ff9640 1 -- 172.21.15.27:0/3944249944 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f228c079ef0 con 0x7f22941612d0 2024-09-12T03:17:21.130 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:21.129+0000 7f229a752640 1 -- 172.21.15.27:0/3944249944 --> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f229414e730 con 0x7f228006be60 2024-09-12T03:17:21.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:21.131+0000 7f2290ff9640 1 -- 172.21.15.27:0/3944249944 <== mgr.4102 v2:172.21.15.45:6832/3923821219 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f229414e730 con 0x7f228006be60 2024-09-12T03:17:21.132 INFO:teuthology.orchestra.run.smithi027.stdout:[ 2024-09-12T03:17:21.132 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:17:21.132 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "cephfs" 2024-09-12T03:17:21.132 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:17:21.133 INFO:teuthology.orchestra.run.smithi027.stdout:] 2024-09-12T03:17:21.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:21.133+0000 7f229a752640 1 -- 172.21.15.27:0/3944249944 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f228006be60 msgr2=0x7f228006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:21.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:21.133+0000 7f229a752640 1 --2- 172.21.15.27:0/3944249944 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f228006be60 0x7f228006e320 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f2284007c20 tx=0x7f2284009040 comp rx=0 tx=0).stop 2024-09-12T03:17:21.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:21.133+0000 7f229a752640 1 -- 172.21.15.27:0/3944249944 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f22941612d0 msgr2=0x7f22940b2110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:21.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:21.133+0000 7f229a752640 1 --2- 172.21.15.27:0/3944249944 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f22941612d0 0x7f22940b2110 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f228c033040 tx=0x7f228c001650 comp rx=0 tx=0).stop 2024-09-12T03:17:21.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:21.133+0000 7f229a752640 1 -- 172.21.15.27:0/3944249944 shutdown_connections 2024-09-12T03:17:21.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:21.133+0000 7f229a752640 1 --2- 172.21.15.27:0/3944249944 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f228006be60 0x7f228006e320 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:21.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:21.133+0000 7f229a752640 1 --2- 172.21.15.27:0/3944249944 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f22941612d0 0x7f22940b2110 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:21.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:21.133+0000 7f229a752640 1 --2- 172.21.15.27:0/3944249944 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f229414f490 0x7f2294160d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:21.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:21.133+0000 7f229a752640 1 --2- 172.21.15.27:0/3944249944 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f227c004d40 0x7f22941f3910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:21.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:21.133+0000 7f229a752640 1 -- 172.21.15.27:0/3944249944 >> 172.21.15.27:0/3944249944 conn(0x7f2294073800 msgr2=0x7f22940d2780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:21.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:21.133+0000 7f229a752640 1 -- 172.21.15.27:0/3944249944 shutdown_connections 2024-09-12T03:17:21.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:21.133+0000 7f229a752640 1 -- 172.21.15.27:0/3944249944 wait complete. 2024-09-12T03:17:21.145 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-12T03:17:21.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:21.249+0000 7f6305e1e640 1 Processor -- start 2024-09-12T03:17:21.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:21.249+0000 7f6305e1e640 1 -- start start 2024-09-12T03:17:21.251 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:21.250+0000 7f6305e1e640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f63001645c0 0x7f63001669b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:21.251 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:21.250+0000 7f6305e1e640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f63000b4720 con 0x7f630015ec30 2024-09-12T03:17:21.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:21.250+0000 7f6305e1e640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f63000ce5e0 con 0x7f63000af9d0 2024-09-12T03:17:21.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:21.250+0000 7f6305e1e640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f63000d02d0 con 0x7f63001645c0 2024-09-12T03:17:21.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:21.250+0000 7f62fffff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f63001645c0 0x7f63001669b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:21.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:21.250+0000 7f62ff7fe640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f630015ec30 0x7f63000b3900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:51368/0 (socket says 172.21.15.27:51368) 2024-09-12T03:17:21.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:21.250+0000 7f62ff7fe640 1 -- 172.21.15.27:0/1637557646 learned_addr learned my addr 172.21.15.27:0/1637557646 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:17:21.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:21.250+0000 7f62fffff640 1 -- 172.21.15.27:0/1637557646 >> v1:172.21.15.45:6789/0 conn(0x7f63000af9d0 legacy=0x7f63000b4010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:21.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:21.250+0000 7f62fffff640 1 -- 172.21.15.27:0/1637557646 >> v1:172.21.15.27:6789/0 conn(0x7f630015ec30 legacy=0x7f63000b3900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:21.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:21.250+0000 7f62fffff640 1 -- 172.21.15.27:0/1637557646 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63000ceac0 con 0x7f63001645c0 2024-09-12T03:17:21.253 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:21.251+0000 7f62fffff640 1 --2- 172.21.15.27:0/1637557646 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f63001645c0 0x7f63001669b0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f62f8009d90 tx=0x7f62f802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43e41f86dc143c9d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:21.253 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:21.251+0000 7f62fe7fc640 1 -- 172.21.15.27:0/1637557646 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62f8007740 con 0x7f63001645c0 2024-09-12T03:17:21.253 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:21.251+0000 7f62fe7fc640 1 -- 172.21.15.27:0/1637557646 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62f80078e0 con 0x7f63001645c0 2024-09-12T03:17:21.253 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:21.251+0000 7f62fe7fc640 1 -- 172.21.15.27:0/1637557646 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62f8007cb0 con 0x7f63001645c0 2024-09-12T03:17:21.253 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:21.252+0000 7f6305e1e640 1 -- 172.21.15.27:0/1637557646 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f63001645c0 msgr2=0x7f63001669b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:21.254 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:21.252+0000 7f6305e1e640 1 --2- 172.21.15.27:0/1637557646 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f63001645c0 0x7f63001669b0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f62f8009d90 tx=0x7f62f802a6a0 comp rx=0 tx=0).stop 2024-09-12T03:17:21.254 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:21.252+0000 7f6305e1e640 1 -- 172.21.15.27:0/1637557646 shutdown_connections 2024-09-12T03:17:21.254 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:21.252+0000 7f6305e1e640 1 --2- 172.21.15.27:0/1637557646 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f63001645c0 0x7f63001669b0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:21.254 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:21.252+0000 7f6305e1e640 1 -- 172.21.15.27:0/1637557646 >> 172.21.15.27:0/1637557646 conn(0x7f630006bb30 msgr2=0x7f630006bf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:21.254 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:21.252+0000 7f6305e1e640 1 -- 172.21.15.27:0/1637557646 shutdown_connections 2024-09-12T03:17:21.254 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:21.252+0000 7f6305e1e640 1 -- 172.21.15.27:0/1637557646 wait complete. 2024-09-12T03:17:21.254 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:21.253+0000 7f6305e1e640 1 Processor -- start 2024-09-12T03:17:21.254 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:21.253+0000 7f6305e1e640 1 -- start start 2024-09-12T03:17:21.256 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:21.254+0000 7f6305e1e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f63000af9d0 0x7f6300155720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:21.256 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:21.254+0000 7f6305e1e640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f630015ec30 0x7f630014caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:21.256 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:21.254+0000 7f6305e1e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f63001645c0 0x7f6300158ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:21.256 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:21.254+0000 7f6305e1e640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f630006c8a0 con 0x7f63000af9d0 2024-09-12T03:17:21.256 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:21.254+0000 7f6305e1e640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f630006c650 con 0x7f63001645c0 2024-09-12T03:17:21.257 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:21.254+0000 7f6305e1e640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f63000cfff0 con 0x7f630015ec30 2024-09-12T03:17:21.257 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:21.254+0000 7f62ff7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f63000af9d0 0x7f6300155720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:21.257 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:21.254+0000 7f62ff7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f63000af9d0 0x7f6300155720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59960/0 (socket says 172.21.15.27:59960) 2024-09-12T03:17:21.257 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:21.254+0000 7f62ff7fe640 1 -- 172.21.15.27:0/3118047906 learned_addr learned my addr 172.21.15.27:0/3118047906 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:17:21.257 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:21.254+0000 7f62fffff640 1 --2- 172.21.15.27:0/3118047906 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f63001645c0 0x7f6300158ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:21.257 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:21.254+0000 7f62feffd640 1 --2- 172.21.15.27:0/3118047906 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f630015ec30 0x7f630014caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:21.257 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:21.254+0000 7f62ff7fe640 1 -- 172.21.15.27:0/3118047906 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f630015ec30 msgr2=0x7f630014caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:21.257 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:21.254+0000 7f62ff7fe640 1 --2- 172.21.15.27:0/3118047906 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f630015ec30 0x7f630014caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:21.257 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:21.255+0000 7f62ff7fe640 1 -- 172.21.15.27:0/3118047906 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f63001645c0 msgr2=0x7f6300158ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:21.257 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:21.255+0000 7f62ff7fe640 1 --2- 172.21.15.27:0/3118047906 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f63001645c0 0x7f6300158ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:21.257 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:21.255+0000 7f62ff7fe640 1 -- 172.21.15.27:0/3118047906 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f630014cfe0 con 0x7f63000af9d0 2024-09-12T03:17:21.257 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:21.255+0000 7f62feffd640 1 --2- 172.21.15.27:0/3118047906 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f630015ec30 0x7f630014caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:17:21.258 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:21.255+0000 7f62fffff640 1 --2- 172.21.15.27:0/3118047906 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f63001645c0 0x7f6300158ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:17:21.258 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:21.255+0000 7f62ff7fe640 1 --2- 172.21.15.27:0/3118047906 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f63000af9d0 0x7f6300155720 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f62f000c400 tx=0x7f62f000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:21.258 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:21.255+0000 7f62e7fff640 1 -- 172.21.15.27:0/3118047906 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62f0017070 con 0x7f63000af9d0 2024-09-12T03:17:21.258 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:21.255+0000 7f62e7fff640 1 -- 172.21.15.27:0/3118047906 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62f0005ce0 con 0x7f63000af9d0 2024-09-12T03:17:21.258 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:21.255+0000 7f6305e1e640 1 -- 172.21.15.27:0/3118047906 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63001596f0 con 0x7f63000af9d0 2024-09-12T03:17:21.258 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:21.255+0000 7f62e7fff640 1 -- 172.21.15.27:0/3118047906 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62f00072f0 con 0x7f63000af9d0 2024-09-12T03:17:21.258 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:21.255+0000 7f6305e1e640 1 -- 172.21.15.27:0/3118047906 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f630015cdb0 con 0x7f63000af9d0 2024-09-12T03:17:21.259 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:21.256+0000 7f6305e1e640 1 -- 172.21.15.27:0/3118047906 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f630015c8f0 con 0x7f63000af9d0 2024-09-12T03:17:21.259 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:21.256+0000 7f62e7fff640 1 -- 172.21.15.27:0/3118047906 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f62f001a070 con 0x7f63000af9d0 2024-09-12T03:17:21.259 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:21.257+0000 7f62e7fff640 1 --2- 172.21.15.27:0/3118047906 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f62ec06be30 0x7f62ec06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:21.259 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:21.257+0000 7f62e7fff640 1 -- 172.21.15.27:0/3118047906 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f62f008a290 con 0x7f63000af9d0 2024-09-12T03:17:21.259 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:21.257+0000 7f62feffd640 1 --2- 172.21.15.27:0/3118047906 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f62ec06be30 0x7f62ec06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:21.259 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:21.258+0000 7f62feffd640 1 --2- 172.21.15.27:0/3118047906 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f62ec06be30 0x7f62ec06e2f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f62f4006fd0 tx=0x7f62f4008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:21.262 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:21.261+0000 7f62e7fff640 1 -- 172.21.15.27:0/3118047906 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f630015c8f0 con 0x7f63000af9d0 2024-09-12T03:17:21.438 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:21.437+0000 7f6305e1e640 1 -- 172.21.15.27:0/3118047906 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f63000715f0 con 0x7f63000af9d0 2024-09-12T03:17:21.438 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:21.437+0000 7f62e7fff640 1 -- 172.21.15.27:0/3118047906 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) ==== 122+0+0 (secure 0 0 0) 0x7f630014cfe0 con 0x7f63000af9d0 2024-09-12T03:17:21.440 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:21.439+0000 7f6305e1e640 1 -- 172.21.15.27:0/3118047906 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f62ec06be30 msgr2=0x7f62ec06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:21.440 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:21.439+0000 7f6305e1e640 1 --2- 172.21.15.27:0/3118047906 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f62ec06be30 0x7f62ec06e2f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f62f4006fd0 tx=0x7f62f4008040 comp rx=0 tx=0).stop 2024-09-12T03:17:21.440 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:21.439+0000 7f6305e1e640 1 -- 172.21.15.27:0/3118047906 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f63000af9d0 msgr2=0x7f6300155720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:21.440 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:21.439+0000 7f6305e1e640 1 --2- 172.21.15.27:0/3118047906 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f63000af9d0 0x7f6300155720 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f62f000c400 tx=0x7f62f000c8c0 comp rx=0 tx=0).stop 2024-09-12T03:17:21.440 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:21.439+0000 7f6305e1e640 1 -- 172.21.15.27:0/3118047906 shutdown_connections 2024-09-12T03:17:21.440 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:21.439+0000 7f6305e1e640 1 --2- 172.21.15.27:0/3118047906 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f62ec06be30 0x7f62ec06e2f0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:21.440 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:21.439+0000 7f6305e1e640 1 --2- 172.21.15.27:0/3118047906 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f63001645c0 0x7f6300158ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:21.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:21.439+0000 7f6305e1e640 1 --2- 172.21.15.27:0/3118047906 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f630015ec30 0x7f630014caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:21.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:21.439+0000 7f6305e1e640 1 --2- 172.21.15.27:0/3118047906 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f63000af9d0 0x7f6300155720 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:21.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:21.439+0000 7f6305e1e640 1 -- 172.21.15.27:0/3118047906 >> 172.21.15.27:0/3118047906 conn(0x7f630006bb30 msgr2=0x7f6300072c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:21.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:21.439+0000 7f6305e1e640 1 -- 172.21.15.27:0/3118047906 shutdown_connections 2024-09-12T03:17:21.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:21.439+0000 7f6305e1e640 1 -- 172.21.15.27:0/3118047906 wait complete. 2024-09-12T03:17:21.453 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:17:21.454 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p ./volumes/subvol_grp70490/subvol57290) 2024-09-12T03:17:21.485 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:17:21.485 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.layout.pool .) 2024-09-12T03:17:21.541 INFO:teuthology.orchestra.run.smithi027.stdout:cephfs_data 2024-09-12T03:17:21.541 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:17:21.542 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo setfattr -n ceph.dir.layout.pool -v cephfs_data ./volumes/subvol_grp70490/subvol57290) 2024-09-12T03:17:21.639 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-12T03:17:21.639 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd of=/home/ubuntu/cephtest/mnt.0/./volumes/subvol_grp70490/subvol57290/.meta 2024-09-12T03:17:21.668 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol57290 subvol_grp70490 2024-09-12T03:17:21.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:21.814+0000 7efc99e65640 1 Processor -- start 2024-09-12T03:17:21.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:21.814+0000 7efc99e65640 1 -- start start 2024-09-12T03:17:21.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:21.815+0000 7efc99e65640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7efc940d31c0 0x7efc940d0440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:21.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:21.815+0000 7efc99e65640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efc940cf9f0 con 0x7efc9406e960 2024-09-12T03:17:21.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:21.815+0000 7efc99e65640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efc940cfc90 con 0x7efc94070d30 2024-09-12T03:17:21.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:21.815+0000 7efc99e65640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7efc940d1a40 con 0x7efc940d31c0 2024-09-12T03:17:21.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:21.815+0000 7efc92ffd640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7efc940d31c0 0x7efc940d0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:21.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:21.815+0000 7efc93fff640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7efc9406e960 0x7efc94168300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:51374/0 (socket says 172.21.15.27:51374) 2024-09-12T03:17:21.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:21.815+0000 7efc93fff640 1 -- 172.21.15.27:0/3084730638 learned_addr learned my addr 172.21.15.27:0/3084730638 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:17:21.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:21.815+0000 7efc92ffd640 1 -- 172.21.15.27:0/3084730638 >> v1:172.21.15.45:6789/0 conn(0x7efc94070d30 legacy=0x7efc9406cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:21.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:21.815+0000 7efc92ffd640 1 -- 172.21.15.27:0/3084730638 >> v1:172.21.15.27:6789/0 conn(0x7efc9406e960 legacy=0x7efc94168300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:21.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:21.815+0000 7efc92ffd640 1 -- 172.21.15.27:0/3084730638 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc940cff70 con 0x7efc940d31c0 2024-09-12T03:17:21.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:21.815+0000 7efc927fc640 1 -- 172.21.15.27:0/3084730638 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1910662308 0 0) 0x7efc940cf9f0 con 0x7efc9406e960 2024-09-12T03:17:21.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:21.816+0000 7efc92ffd640 1 --2- 172.21.15.27:0/3084730638 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efc940d31c0 0x7efc940d0440 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7efc880097a0 tx=0x7efc8802a0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69060d5474944f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:21.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:21.816+0000 7efc927fc640 1 -- 172.21.15.27:0/3084730638 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc88037070 con 0x7efc940d31c0 2024-09-12T03:17:21.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:21.816+0000 7efc927fc640 1 -- 172.21.15.27:0/3084730638 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc8802adc0 con 0x7efc940d31c0 2024-09-12T03:17:21.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:21.816+0000 7efc927fc640 1 -- 172.21.15.27:0/3084730638 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc88032660 con 0x7efc940d31c0 2024-09-12T03:17:21.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:21.817+0000 7efc99e65640 1 -- 172.21.15.27:0/3084730638 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efc940d31c0 msgr2=0x7efc940d0440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:21.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:21.817+0000 7efc99e65640 1 --2- 172.21.15.27:0/3084730638 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efc940d31c0 0x7efc940d0440 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7efc880097a0 tx=0x7efc8802a0b0 comp rx=0 tx=0).stop 2024-09-12T03:17:21.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:21.817+0000 7efc99e65640 1 -- 172.21.15.27:0/3084730638 shutdown_connections 2024-09-12T03:17:21.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:21.817+0000 7efc99e65640 1 --2- 172.21.15.27:0/3084730638 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efc940d31c0 0x7efc940d0440 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:21.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:21.817+0000 7efc99e65640 1 -- 172.21.15.27:0/3084730638 >> 172.21.15.27:0/3084730638 conn(0x7efc940b7c10 msgr2=0x7efc9409ba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:21.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:21.817+0000 7efc99e65640 1 -- 172.21.15.27:0/3084730638 shutdown_connections 2024-09-12T03:17:21.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:21.817+0000 7efc99e65640 1 -- 172.21.15.27:0/3084730638 wait complete. 2024-09-12T03:17:21.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:21.818+0000 7efc99e65640 1 Processor -- start 2024-09-12T03:17:21.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:21.818+0000 7efc99e65640 1 -- start start 2024-09-12T03:17:21.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:21.819+0000 7efc99e65640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efc9406e960 0x7efc941f83b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:21.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:21.819+0000 7efc99e65640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7efc94070d30 0x7efc94158a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:21.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:21.819+0000 7efc99e65640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efc940d31c0 0x7efc9415e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:21.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:21.819+0000 7efc99e65640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7efc940d0b00 con 0x7efc940d31c0 2024-09-12T03:17:21.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:21.819+0000 7efc99e65640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7efc940d0980 con 0x7efc9406e960 2024-09-12T03:17:21.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:21.819+0000 7efc99e65640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7efc940d0c80 con 0x7efc94070d30 2024-09-12T03:17:21.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:21.819+0000 7efc93fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efc940d31c0 0x7efc9415e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:21.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:21.819+0000 7efc93fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efc940d31c0 0x7efc9415e9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59968/0 (socket says 172.21.15.27:59968) 2024-09-12T03:17:21.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:21.819+0000 7efc93fff640 1 -- 172.21.15.27:0/2217074790 learned_addr learned my addr 172.21.15.27:0/2217074790 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:17:21.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:21.819+0000 7efc937fe640 1 --2- 172.21.15.27:0/2217074790 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efc9406e960 0x7efc941f83b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:21.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:21.819+0000 7efc92ffd640 1 --2- 172.21.15.27:0/2217074790 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7efc94070d30 0x7efc94158a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:21.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:21.819+0000 7efc93fff640 1 -- 172.21.15.27:0/2217074790 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7efc94070d30 msgr2=0x7efc94158a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:21.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:21.819+0000 7efc93fff640 1 --2- 172.21.15.27:0/2217074790 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7efc94070d30 0x7efc94158a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:21.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:21.819+0000 7efc93fff640 1 -- 172.21.15.27:0/2217074790 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efc9406e960 msgr2=0x7efc941f83b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:21.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:21.819+0000 7efc93fff640 1 --2- 172.21.15.27:0/2217074790 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efc9406e960 0x7efc941f83b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:21.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:21.819+0000 7efc93fff640 1 -- 172.21.15.27:0/2217074790 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc941f8a90 con 0x7efc940d31c0 2024-09-12T03:17:21.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:21.820+0000 7efc93fff640 1 --2- 172.21.15.27:0/2217074790 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efc940d31c0 0x7efc9415e9c0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7efc8c00c7a0 tx=0x7efc8c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:21.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:21.820+0000 7efc7bfff640 1 -- 172.21.15.27:0/2217074790 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc8c018070 con 0x7efc940d31c0 2024-09-12T03:17:21.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:21.820+0000 7efc7bfff640 1 -- 172.21.15.27:0/2217074790 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc8c00a640 con 0x7efc940d31c0 2024-09-12T03:17:21.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:21.820+0000 7efc99e65640 1 -- 172.21.15.27:0/2217074790 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc941f8d70 con 0x7efc940d31c0 2024-09-12T03:17:21.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:21.820+0000 7efc7bfff640 1 -- 172.21.15.27:0/2217074790 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc8c00aab0 con 0x7efc940d31c0 2024-09-12T03:17:21.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:21.820+0000 7efc99e65640 1 -- 172.21.15.27:0/2217074790 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc940b2250 con 0x7efc940d31c0 2024-09-12T03:17:21.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:21.821+0000 7efc99e65640 1 -- 172.21.15.27:0/2217074790 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc940d0440 con 0x7efc940d31c0 2024-09-12T03:17:21.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:21.821+0000 7efc7bfff640 1 -- 172.21.15.27:0/2217074790 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7efc8c01b070 con 0x7efc940d31c0 2024-09-12T03:17:21.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:21.822+0000 7efc7bfff640 1 --2- 172.21.15.27:0/2217074790 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7efc8006be30 0x7efc8006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:21.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:21.822+0000 7efc7bfff640 1 -- 172.21.15.27:0/2217074790 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7efc8c08a410 con 0x7efc940d31c0 2024-09-12T03:17:21.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:21.822+0000 7efc937fe640 1 --2- 172.21.15.27:0/2217074790 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7efc8006be30 0x7efc8006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:21.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:21.823+0000 7efc937fe640 1 --2- 172.21.15.27:0/2217074790 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7efc8006be30 0x7efc8006e2f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7efc840015d0 tx=0x7efc84009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:21.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:21.826+0000 7efc7bfff640 1 -- 172.21.15.27:0/2217074790 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7efc8c058e50 con 0x7efc940d31c0 2024-09-12T03:17:22.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:22.030+0000 7efc99e65640 1 -- 172.21.15.27:0/2217074790 --> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol57290", "group_name": "subvol_grp70490", "target": ["mon-mgr", ""]}) -- 0x7efc940d1230 con 0x7efc8006be30 2024-09-12T03:17:22.036 INFO:tasks.ceph.mgr.x.smithi045.stderr:2024-09-12T03:17:22.036+0000 7fc16ffad640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:17:22.036 INFO:tasks.ceph.mgr.x.smithi045.stderr:2024-09-12T03:17:22.036+0000 7fc16ffad640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:17:22.037 INFO:tasks.ceph.mgr.x.smithi045.stderr:2024-09-12T03:17:22.036+0000 7fc16ffad640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:17:22.037 INFO:tasks.ceph.mgr.x.smithi045.stderr:2024-09-12T03:17:22.036+0000 7fc16ffad640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:17:22.037 INFO:tasks.ceph.mgr.x.smithi045.stderr:2024-09-12T03:17:22.036+0000 7fc16ffad640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:17:22.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:22.074+0000 7efc7bfff640 1 -- 172.21.15.27:0/2217074790 <== mgr.4102 v2:172.21.15.45:6832/3923821219 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7efc940d1230 con 0x7efc8006be30 2024-09-12T03:17:22.075 INFO:teuthology.orchestra.run.smithi027.stdout:/volumes/subvol_grp70490/subvol57290 2024-09-12T03:17:22.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:22.076+0000 7efc99e65640 1 -- 172.21.15.27:0/2217074790 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7efc8006be30 msgr2=0x7efc8006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:22.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:22.076+0000 7efc99e65640 1 --2- 172.21.15.27:0/2217074790 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7efc8006be30 0x7efc8006e2f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7efc840015d0 tx=0x7efc84009040 comp rx=0 tx=0).stop 2024-09-12T03:17:22.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:22.076+0000 7efc99e65640 1 -- 172.21.15.27:0/2217074790 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efc940d31c0 msgr2=0x7efc9415e9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:22.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:22.076+0000 7efc99e65640 1 --2- 172.21.15.27:0/2217074790 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efc940d31c0 0x7efc9415e9c0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7efc8c00c7a0 tx=0x7efc8c00cc60 comp rx=0 tx=0).stop 2024-09-12T03:17:22.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:22.076+0000 7efc99e65640 1 -- 172.21.15.27:0/2217074790 shutdown_connections 2024-09-12T03:17:22.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:22.076+0000 7efc99e65640 1 --2- 172.21.15.27:0/2217074790 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7efc8006be30 0x7efc8006e2f0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:22.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:22.076+0000 7efc99e65640 1 --2- 172.21.15.27:0/2217074790 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efc940d31c0 0x7efc9415e9c0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:22.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:22.076+0000 7efc99e65640 1 --2- 172.21.15.27:0/2217074790 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7efc94070d30 0x7efc94158a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:22.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:22.076+0000 7efc99e65640 1 --2- 172.21.15.27:0/2217074790 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efc9406e960 0x7efc941f83b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:22.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:22.076+0000 7efc99e65640 1 -- 172.21.15.27:0/2217074790 >> 172.21.15.27:0/2217074790 conn(0x7efc940b7c10 msgr2=0x7efc9409ba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:22.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:22.076+0000 7efc99e65640 1 -- 172.21.15.27:0/2217074790 shutdown_connections 2024-09-12T03:17:22.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:22.076+0000 7efc99e65640 1 -- 172.21.15.27:0/2217074790 wait complete. 2024-09-12T03:17:22.088 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol57290 subvol_grp70490 2024-09-12T03:17:22.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:22.193+0000 7f300c27b640 1 Processor -- start 2024-09-12T03:17:22.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:22.193+0000 7f300c27b640 1 -- start start 2024-09-12T03:17:22.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:22.194+0000 7f300c27b640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f300415e650 0x7f30040d1d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:22.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:22.194+0000 7f300c27b640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f300406cc40 0x7f300406d060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:22.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:22.194+0000 7f300c27b640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f300406db30 0x7f300406df60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:22.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:22.194+0000 7f300c27b640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f30040d2280 con 0x7f300406cc40 2024-09-12T03:17:22.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:22.194+0000 7f300c27b640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f300406d5a0 con 0x7f300406db30 2024-09-12T03:17:22.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:22.194+0000 7f300c27b640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f300406d850 con 0x7f300415e650 2024-09-12T03:17:22.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:22.194+0000 7f3009816640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f300406cc40 0x7f300406d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:22.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:22.194+0000 7f3009816640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f300406cc40 0x7f300406d060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59972/0 (socket says 172.21.15.27:59972) 2024-09-12T03:17:22.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:22.194+0000 7f3009816640 1 -- 172.21.15.27:0/4100881222 learned_addr learned my addr 172.21.15.27:0/4100881222 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:17:22.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:22.194+0000 7f300a017640 1 --2- 172.21.15.27:0/4100881222 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f300415e650 0x7f30040d1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:22.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:22.194+0000 7f300a818640 1 --2- 172.21.15.27:0/4100881222 >> v2:172.21.15.45:3300/0 conn(0x7f300406db30 0x7f300406df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:22.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:22.195+0000 7f3009816640 1 -- 172.21.15.27:0/4100881222 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f300415e650 msgr2=0x7f30040d1d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:22.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:22.195+0000 7f3009816640 1 --2- 172.21.15.27:0/4100881222 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f300415e650 0x7f30040d1d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:22.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:22.195+0000 7f3009816640 1 -- 172.21.15.27:0/4100881222 >> v2:172.21.15.45:3300/0 conn(0x7f300406db30 msgr2=0x7f300406df60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:22.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:22.195+0000 7f3009816640 1 --2- 172.21.15.27:0/4100881222 >> v2:172.21.15.45:3300/0 conn(0x7f300406db30 0x7f300406df60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:22.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:22.195+0000 7f3009816640 1 -- 172.21.15.27:0/4100881222 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30040d1260 con 0x7f300406cc40 2024-09-12T03:17:22.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:22.195+0000 7f3009816640 1 --2- 172.21.15.27:0/4100881222 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f300406cc40 0x7f300406d060 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f3004050f60 tx=0x7f2ff802a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d928aad695434d2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:22.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:22.195+0000 7f3009015640 1 -- 172.21.15.27:0/4100881222 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ff8037070 con 0x7f300406cc40 2024-09-12T03:17:22.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:22.196+0000 7f3009015640 1 -- 172.21.15.27:0/4100881222 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ff8032440 con 0x7f300406cc40 2024-09-12T03:17:22.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:22.196+0000 7f3009015640 1 -- 172.21.15.27:0/4100881222 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ff8032810 con 0x7f300406cc40 2024-09-12T03:17:22.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:22.196+0000 7f300c27b640 1 -- 172.21.15.27:0/4100881222 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f300406cc40 msgr2=0x7f300406d060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:22.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:22.196+0000 7f300c27b640 1 --2- 172.21.15.27:0/4100881222 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f300406cc40 0x7f300406d060 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f3004050f60 tx=0x7f2ff802a140 comp rx=0 tx=0).stop 2024-09-12T03:17:22.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:22.196+0000 7f300c27b640 1 -- 172.21.15.27:0/4100881222 shutdown_connections 2024-09-12T03:17:22.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:22.196+0000 7f300c27b640 1 --2- 172.21.15.27:0/4100881222 >> v2:172.21.15.45:3300/0 conn(0x7f300406db30 0x7f300406df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:22.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:22.196+0000 7f300c27b640 1 --2- 172.21.15.27:0/4100881222 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f300406cc40 0x7f300406d060 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:22.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:22.196+0000 7f300c27b640 1 --2- 172.21.15.27:0/4100881222 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f300415e650 0x7f30040d1d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:22.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:22.196+0000 7f300c27b640 1 -- 172.21.15.27:0/4100881222 >> 172.21.15.27:0/4100881222 conn(0x7f300406f320 msgr2=0x7f300406f700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:22.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:22.196+0000 7f300c27b640 1 -- 172.21.15.27:0/4100881222 shutdown_connections 2024-09-12T03:17:22.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:22.196+0000 7f300c27b640 1 -- 172.21.15.27:0/4100881222 wait complete. 2024-09-12T03:17:22.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:22.197+0000 7f300c27b640 1 Processor -- start 2024-09-12T03:17:22.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:22.197+0000 7f300c27b640 1 -- start start 2024-09-12T03:17:22.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:22.198+0000 7f300c27b640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f300406cc40 0x7f30041f75c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:22.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:22.198+0000 7f300c27b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f300406db30 0x7f300415aff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:22.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:22.198+0000 7f300c27b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f300415e650 0x7f3004158f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:22.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:22.198+0000 7f300c27b640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f300415ebb0 con 0x7f300415e650 2024-09-12T03:17:22.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:22.198+0000 7f300c27b640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f300415ea30 con 0x7f300406db30 2024-09-12T03:17:22.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:22.198+0000 7f300c27b640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f300406e4a0 con 0x7f300406cc40 2024-09-12T03:17:22.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:22.198+0000 7f300a818640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f300415e650 0x7f3004158f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:22.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:22.198+0000 7f300a818640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f300415e650 0x7f3004158f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59978/0 (socket says 172.21.15.27:59978) 2024-09-12T03:17:22.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:22.198+0000 7f300a818640 1 -- 172.21.15.27:0/2169925497 learned_addr learned my addr 172.21.15.27:0/2169925497 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:17:22.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:22.198+0000 7f300a017640 1 --2- 172.21.15.27:0/2169925497 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f300406cc40 0x7f30041f75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:22.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:22.198+0000 7f3009816640 1 --2- 172.21.15.27:0/2169925497 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f300406db30 0x7f300415aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:22.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:22.199+0000 7f300a818640 1 -- 172.21.15.27:0/2169925497 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f300406cc40 msgr2=0x7f30041f75c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:22.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:22.199+0000 7f300a818640 1 --2- 172.21.15.27:0/2169925497 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f300406cc40 0x7f30041f75c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:22.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:22.199+0000 7f300a818640 1 -- 172.21.15.27:0/2169925497 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f300406db30 msgr2=0x7f300415aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:22.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:22.199+0000 7f300a818640 1 --2- 172.21.15.27:0/2169925497 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f300406db30 0x7f300415aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:22.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:22.199+0000 7f300a818640 1 -- 172.21.15.27:0/2169925497 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3004159440 con 0x7f300415e650 2024-09-12T03:17:22.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:22.199+0000 7f300a818640 1 --2- 172.21.15.27:0/2169925497 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f300415e650 0x7f3004158f00 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f2ff4007d40 tx=0x7f2ff400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:22.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:22.199+0000 7f3002ffd640 1 -- 172.21.15.27:0/2169925497 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ff4014420 con 0x7f300415e650 2024-09-12T03:17:22.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:22.199+0000 7f3002ffd640 1 -- 172.21.15.27:0/2169925497 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ff4019070 con 0x7f300415e650 2024-09-12T03:17:22.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:22.199+0000 7f300c27b640 1 -- 172.21.15.27:0/2169925497 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30041596e0 con 0x7f300415e650 2024-09-12T03:17:22.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:22.199+0000 7f3002ffd640 1 -- 172.21.15.27:0/2169925497 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ff4014820 con 0x7f300415e650 2024-09-12T03:17:22.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:22.199+0000 7f300c27b640 1 -- 172.21.15.27:0/2169925497 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3004159ca0 con 0x7f300415e650 2024-09-12T03:17:22.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:22.200+0000 7f300c27b640 1 -- 172.21.15.27:0/2169925497 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f300406df60 con 0x7f300415e650 2024-09-12T03:17:22.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:22.201+0000 7f3002ffd640 1 -- 172.21.15.27:0/2169925497 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2ff40149c0 con 0x7f300415e650 2024-09-12T03:17:22.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:22.201+0000 7f3002ffd640 1 --2- 172.21.15.27:0/2169925497 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f2ff006be30 0x7f2ff006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:22.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:22.201+0000 7f3002ffd640 1 -- 172.21.15.27:0/2169925497 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f2ff408b3c0 con 0x7f300415e650 2024-09-12T03:17:22.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:22.201+0000 7f300a017640 1 --2- 172.21.15.27:0/2169925497 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f2ff006be30 0x7f2ff006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:22.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:22.202+0000 7f300a017640 1 --2- 172.21.15.27:0/2169925497 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f2ff006be30 0x7f2ff006e2f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f2ffc006fd0 tx=0x7f2ffc008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:22.206 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:22.205+0000 7f3002ffd640 1 -- 172.21.15.27:0/2169925497 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f2ff40597b0 con 0x7f300415e650 2024-09-12T03:17:22.410 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:22.409+0000 7f300c27b640 1 -- 172.21.15.27:0/2169925497 --> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol57290", "group_name": "subvol_grp70490", "target": ["mon-mgr", ""]}) -- 0x7f3004161bb0 con 0x7f2ff006be30 2024-09-12T03:17:22.415 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:22.414+0000 7f3002ffd640 1 -- 172.21.15.27:0/2169925497 <== mgr.4102 v2:172.21.15.45:6832/3923821219 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f3004161bb0 con 0x7f2ff006be30 2024-09-12T03:17:22.415 INFO:teuthology.orchestra.run.smithi027.stdout:/volumes/subvol_grp70490/subvol57290 2024-09-12T03:17:22.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:22.416+0000 7f300c27b640 1 -- 172.21.15.27:0/2169925497 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f2ff006be30 msgr2=0x7f2ff006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:22.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:22.416+0000 7f300c27b640 1 --2- 172.21.15.27:0/2169925497 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f2ff006be30 0x7f2ff006e2f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f2ffc006fd0 tx=0x7f2ffc008040 comp rx=0 tx=0).stop 2024-09-12T03:17:22.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:22.416+0000 7f300c27b640 1 -- 172.21.15.27:0/2169925497 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f300415e650 msgr2=0x7f3004158f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:22.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:22.416+0000 7f300c27b640 1 --2- 172.21.15.27:0/2169925497 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f300415e650 0x7f3004158f00 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f2ff4007d40 tx=0x7f2ff400e4b0 comp rx=0 tx=0).stop 2024-09-12T03:17:22.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:22.416+0000 7f300c27b640 1 -- 172.21.15.27:0/2169925497 shutdown_connections 2024-09-12T03:17:22.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:22.416+0000 7f300c27b640 1 --2- 172.21.15.27:0/2169925497 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f2ff006be30 0x7f2ff006e2f0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:22.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:22.416+0000 7f300c27b640 1 --2- 172.21.15.27:0/2169925497 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f300415e650 0x7f3004158f00 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:22.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:22.416+0000 7f300c27b640 1 --2- 172.21.15.27:0/2169925497 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f300406db30 0x7f300415aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:22.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:22.416+0000 7f300c27b640 1 --2- 172.21.15.27:0/2169925497 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f300406cc40 0x7f30041f75c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:22.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:22.416+0000 7f300c27b640 1 -- 172.21.15.27:0/2169925497 >> 172.21.15.27:0/2169925497 conn(0x7f300406f320 msgr2=0x7f300415f1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:22.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:22.416+0000 7f300c27b640 1 -- 172.21.15.27:0/2169925497 shutdown_connections 2024-09-12T03:17:22.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:22.416+0000 7f300c27b640 1 -- 172.21.15.27:0/2169925497 wait complete. 2024-09-12T03:17:22.428 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:17:22.428 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cat ./volumes/_legacy/dee90560513e9b0c970ed99e9069af97.meta) 2024-09-12T03:17:24.892 INFO:teuthology.orchestra.run.smithi027.stdout:[GLOBAL] 2024-09-12T03:17:24.892 INFO:teuthology.orchestra.run.smithi027.stdout:version = 1 2024-09-12T03:17:24.893 INFO:teuthology.orchestra.run.smithi027.stdout:type = subvolume 2024-09-12T03:17:24.893 INFO:teuthology.orchestra.run.smithi027.stdout:path = /volumes/subvol_grp70490/subvol57290 2024-09-12T03:17:24.893 INFO:teuthology.orchestra.run.smithi027.stdout:state = complete 2024-09-12T03:17:24.893 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:17:24.898 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol57290 subvol_grp70490 2024-09-12T03:17:25.015 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.014+0000 7f32add11640 1 Processor -- start 2024-09-12T03:17:25.015 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.014+0000 7f32add11640 1 -- start start 2024-09-12T03:17:25.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.015+0000 7f32add11640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f32a815ec30 0x7f32a80b3900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:25.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.015+0000 7f32add11640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f32a80af9d0 0x7f32a80b3e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:25.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.015+0000 7f32add11640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f32a80ce5e0 con 0x7f32a80b4380 2024-09-12T03:17:25.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.015+0000 7f32add11640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f32a80d02d0 con 0x7f32a80af9d0 2024-09-12T03:17:25.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.015+0000 7f32add11640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f32a806c650 con 0x7f32a815ec30 2024-09-12T03:17:25.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.015+0000 7f329effd640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f32a80af9d0 0x7f32a80b3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:25.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.015+0000 7f329effd640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f32a80af9d0 0x7f32a80b3e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59988/0 (socket says 172.21.15.27:59988) 2024-09-12T03:17:25.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.015+0000 7f329effd640 1 -- 172.21.15.27:0/420568733 learned_addr learned my addr 172.21.15.27:0/420568733 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:17:25.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.016+0000 7f32a77fe640 1 --2- 172.21.15.27:0/420568733 >> v2:172.21.15.45:3300/0 conn(0x7f32a815ec30 0x7f32a80b3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:25.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.016+0000 7f329effd640 1 -- 172.21.15.27:0/420568733 >> v2:172.21.15.45:3300/0 conn(0x7f32a815ec30 msgr2=0x7f32a80b3900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:25.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.016+0000 7f329effd640 1 --2- 172.21.15.27:0/420568733 >> v2:172.21.15.45:3300/0 conn(0x7f32a815ec30 0x7f32a80b3900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:25.018 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.016+0000 7f329effd640 1 -- 172.21.15.27:0/420568733 >> v1:172.21.15.45:6789/0 conn(0x7f32a80b4380 legacy=0x7f32a815f2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:25.018 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.016+0000 7f329effd640 1 -- 172.21.15.27:0/420568733 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32a80ceac0 con 0x7f32a80af9d0 2024-09-12T03:17:25.019 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.017+0000 7f329effd640 1 --2- 172.21.15.27:0/420568733 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f32a80af9d0 0x7f32a80b3e40 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f32a806fb80 tx=0x7f329002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d66c6d41776a3ed server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:25.019 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.017+0000 7f32a6ffd640 1 -- 172.21.15.27:0/420568733 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3290037070 con 0x7f32a80af9d0 2024-09-12T03:17:25.019 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.017+0000 7f32a6ffd640 1 -- 172.21.15.27:0/420568733 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3290032440 con 0x7f32a80af9d0 2024-09-12T03:17:25.019 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.017+0000 7f32a6ffd640 1 -- 172.21.15.27:0/420568733 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3290032810 con 0x7f32a80af9d0 2024-09-12T03:17:25.019 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.017+0000 7f32add11640 1 -- 172.21.15.27:0/420568733 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f32a80af9d0 msgr2=0x7f32a80b3e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:25.019 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.017+0000 7f32add11640 1 --2- 172.21.15.27:0/420568733 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f32a80af9d0 0x7f32a80b3e40 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f32a806fb80 tx=0x7f329002a270 comp rx=0 tx=0).stop 2024-09-12T03:17:25.019 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.017+0000 7f32add11640 1 -- 172.21.15.27:0/420568733 shutdown_connections 2024-09-12T03:17:25.019 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.017+0000 7f32add11640 1 --2- 172.21.15.27:0/420568733 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f32a80af9d0 0x7f32a80b3e40 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:25.019 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.017+0000 7f32add11640 1 --2- 172.21.15.27:0/420568733 >> v2:172.21.15.45:3300/0 conn(0x7f32a815ec30 0x7f32a80b3900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:25.020 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.017+0000 7f32add11640 1 -- 172.21.15.27:0/420568733 >> 172.21.15.27:0/420568733 conn(0x7f32a806bb30 msgr2=0x7f32a806bf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:25.020 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.017+0000 7f32add11640 1 -- 172.21.15.27:0/420568733 shutdown_connections 2024-09-12T03:17:25.020 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.017+0000 7f32add11640 1 -- 172.21.15.27:0/420568733 wait complete. 2024-09-12T03:17:25.022 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.019+0000 7f32add11640 1 Processor -- start 2024-09-12T03:17:25.022 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.019+0000 7f32add11640 1 -- start start 2024-09-12T03:17:25.022 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.019+0000 7f32add11640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f32a80af9d0 0x7f32a814e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:25.022 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.020+0000 7f32add11640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f32a80b4380 0x7f32a815db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:25.022 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.020+0000 7f32add11640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f32a815ec30 0x7f32a815b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:25.023 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.020+0000 7f32add11640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f32a80cfff0 con 0x7f32a80b4380 2024-09-12T03:17:25.023 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.020+0000 7f32add11640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f32a806c8a0 con 0x7f32a815ec30 2024-09-12T03:17:25.023 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.020+0000 7f32add11640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f32a80715f0 con 0x7f32a80af9d0 2024-09-12T03:17:25.023 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.020+0000 7f329effd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f32a80b4380 0x7f32a815db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:25.023 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.020+0000 7f329effd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f32a80b4380 0x7f32a815db60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60004/0 (socket says 172.21.15.27:60004) 2024-09-12T03:17:25.023 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.020+0000 7f329effd640 1 -- 172.21.15.27:0/3108630082 learned_addr learned my addr 172.21.15.27:0/3108630082 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:17:25.023 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.020+0000 7f32a77fe640 1 --2- 172.21.15.27:0/3108630082 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f32a80af9d0 0x7f32a814e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:25.023 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.020+0000 7f32a7fff640 1 --2- 172.21.15.27:0/3108630082 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f32a815ec30 0x7f32a815b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:25.023 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.020+0000 7f329effd640 1 -- 172.21.15.27:0/3108630082 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f32a80af9d0 msgr2=0x7f32a814e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:25.024 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.020+0000 7f329effd640 1 --2- 172.21.15.27:0/3108630082 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f32a80af9d0 0x7f32a814e130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:25.024 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.020+0000 7f329effd640 1 -- 172.21.15.27:0/3108630082 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f32a815ec30 msgr2=0x7f32a815b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:25.024 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.020+0000 7f329effd640 1 --2- 172.21.15.27:0/3108630082 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f32a815ec30 0x7f32a815b0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:25.024 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.020+0000 7f329effd640 1 -- 172.21.15.27:0/3108630082 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32a81f09a0 con 0x7f32a80b4380 2024-09-12T03:17:25.024 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.020+0000 7f32a7fff640 1 --2- 172.21.15.27:0/3108630082 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f32a815ec30 0x7f32a815b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:17:25.024 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.020+0000 7f329effd640 1 --2- 172.21.15.27:0/3108630082 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f32a80b4380 0x7f32a815db60 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f3290000c00 tx=0x7f32900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:25.024 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.020+0000 7f32a4ff9640 1 -- 172.21.15.27:0/3108630082 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3290041070 con 0x7f32a80b4380 2024-09-12T03:17:25.024 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.020+0000 7f32a4ff9640 1 -- 172.21.15.27:0/3108630082 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f329003c870 con 0x7f32a80b4380 2024-09-12T03:17:25.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.020+0000 7f32add11640 1 -- 172.21.15.27:0/3108630082 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32a81f0c80 con 0x7f32a80b4380 2024-09-12T03:17:25.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.020+0000 7f32a4ff9640 1 -- 172.21.15.27:0/3108630082 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3290037040 con 0x7f32a80b4380 2024-09-12T03:17:25.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.021+0000 7f32add11640 1 -- 172.21.15.27:0/3108630082 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32a81f1240 con 0x7f32a80b4380 2024-09-12T03:17:25.026 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.021+0000 7f32add11640 1 -- 172.21.15.27:0/3108630082 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32a80b3900 con 0x7f32a80b4380 2024-09-12T03:17:25.026 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.022+0000 7f32a4ff9640 1 -- 172.21.15.27:0/3108630082 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 6) ==== 90549+0+0 (secure 0 0 0) 0x7f3290044070 con 0x7f32a80b4380 2024-09-12T03:17:25.026 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.023+0000 7f32a4ff9640 1 --2- 172.21.15.27:0/3108630082 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f328806bec0 0x7f328806e380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:25.026 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.023+0000 7f32a4ff9640 1 -- 172.21.15.27:0/3108630082 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f32900abb90 con 0x7f32a80b4380 2024-09-12T03:17:25.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.023+0000 7f32a77fe640 1 --2- 172.21.15.27:0/3108630082 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f328806bec0 0x7f328806e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:25.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.023+0000 7f32a77fe640 1 --2- 172.21.15.27:0/3108630082 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f328806bec0 0x7f328806e380 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f3298006fd0 tx=0x7f3298008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:25.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.027+0000 7f32a4ff9640 1 -- 172.21.15.27:0/3108630082 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f329007a020 con 0x7f32a80b4380 2024-09-12T03:17:25.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.229+0000 7f32add11640 1 -- 172.21.15.27:0/3108630082 --> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol57290", "group_name": "subvol_grp70490", "target": ["mon-mgr", ""]}) -- 0x7f32a806f240 con 0x7f328806bec0 2024-09-12T03:17:25.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.245+0000 7f32a4ff9640 1 -- 172.21.15.27:0/3108630082 <== mgr.4102 v2:172.21.15.45:6832/3923821219 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f32a806f240 con 0x7f328806bec0 2024-09-12T03:17:25.248 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.247+0000 7f32add11640 1 -- 172.21.15.27:0/3108630082 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f328806bec0 msgr2=0x7f328806e380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:25.248 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.247+0000 7f32add11640 1 --2- 172.21.15.27:0/3108630082 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f328806bec0 0x7f328806e380 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f3298006fd0 tx=0x7f3298008040 comp rx=0 tx=0).stop 2024-09-12T03:17:25.248 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.247+0000 7f32add11640 1 -- 172.21.15.27:0/3108630082 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f32a80b4380 msgr2=0x7f32a815db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:25.248 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.247+0000 7f32add11640 1 --2- 172.21.15.27:0/3108630082 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f32a80b4380 0x7f32a815db60 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f3290000c00 tx=0x7f32900046c0 comp rx=0 tx=0).stop 2024-09-12T03:17:25.248 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.247+0000 7f32add11640 1 -- 172.21.15.27:0/3108630082 shutdown_connections 2024-09-12T03:17:25.249 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.247+0000 7f32add11640 1 --2- 172.21.15.27:0/3108630082 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f328806bec0 0x7f328806e380 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:25.249 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.247+0000 7f32add11640 1 --2- 172.21.15.27:0/3108630082 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f32a815ec30 0x7f32a815b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:25.249 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.247+0000 7f32add11640 1 --2- 172.21.15.27:0/3108630082 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f32a80b4380 0x7f32a815db60 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:25.249 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.247+0000 7f32add11640 1 --2- 172.21.15.27:0/3108630082 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f32a80af9d0 0x7f32a814e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:25.249 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.247+0000 7f32add11640 1 -- 172.21.15.27:0/3108630082 >> 172.21.15.27:0/3108630082 conn(0x7f32a806bb30 msgr2=0x7f32a80b18d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:25.249 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.247+0000 7f32add11640 1 -- 172.21.15.27:0/3108630082 shutdown_connections 2024-09-12T03:17:25.249 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.247+0000 7f32add11640 1 -- 172.21.15.27:0/3108630082 wait complete. 2024-09-12T03:17:25.250 INFO:tasks.ceph.mgr.x.smithi045.stderr:2024-09-12T03:17:25.249+0000 7fc1727b2640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:17:25.250 INFO:tasks.ceph.mgr.x.smithi045.stderr:2024-09-12T03:17:25.249+0000 7fc1727b2640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:17:25.250 INFO:tasks.ceph.mgr.x.smithi045.stderr:2024-09-12T03:17:25.249+0000 7fc1727b2640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:17:25.250 INFO:tasks.ceph.mgr.x.smithi045.stderr:2024-09-12T03:17:25.249+0000 7fc1727b2640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:17:25.250 INFO:tasks.ceph.mgr.x.smithi045.stderr:2024-09-12T03:17:25.249+0000 7fc1727b2640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:17:25.260 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-09-12T03:17:25.264 INFO:tasks.ceph.mgr.x.smithi045.stderr:2024-09-12T03:17:25.263+0000 7fc172fb3640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:17:25.264 INFO:tasks.ceph.mgr.x.smithi045.stderr:2024-09-12T03:17:25.263+0000 7fc172fb3640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:17:25.264 INFO:tasks.ceph.mgr.x.smithi045.stderr:2024-09-12T03:17:25.263+0000 7fc172fb3640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:17:25.264 INFO:tasks.ceph.mgr.x.smithi045.stderr:2024-09-12T03:17:25.263+0000 7fc172fb3640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:17:25.264 INFO:tasks.ceph.mgr.x.smithi045.stderr:2024-09-12T03:17:25.263+0000 7fc172fb3640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:17:25.310 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-12T03:17:25.312 INFO:teuthology.orchestra.run.smithi027.stdout:2 2024-09-12T03:17:25.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T03:17:25.318 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup rm cephfs subvol_grp70490 2024-09-12T03:17:25.422 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.421+0000 7fd646afa640 1 Processor -- start 2024-09-12T03:17:25.423 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.421+0000 7fd646afa640 1 -- start start 2024-09-12T03:17:25.427 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.422+0000 7fd646afa640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fd64006dfd0 0x7fd6400d1da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:25.427 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.422+0000 7fd646afa640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fd6400d3790 0x7fd6400b7a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:25.427 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.422+0000 7fd646afa640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd6400d2e30 con 0x7fd640162040 2024-09-12T03:17:25.427 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.422+0000 7fd646afa640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7fd6400b7f90 con 0x7fd64006dfd0 2024-09-12T03:17:25.427 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.422+0000 7fd646afa640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fd640162420 con 0x7fd6400d3790 2024-09-12T03:17:25.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.422+0000 7fd63ffff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fd64006dfd0 0x7fd6400d1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:25.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.422+0000 7fd63ffff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fd64006dfd0 0x7fd6400d1da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60016/0 (socket says 172.21.15.27:60016) 2024-09-12T03:17:25.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.423+0000 7fd63ffff640 1 -- 172.21.15.27:0/4265682558 learned_addr learned my addr 172.21.15.27:0/4265682558 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:17:25.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.423+0000 7fd645097640 1 --2- 172.21.15.27:0/4265682558 >> v2:172.21.15.45:3300/0 conn(0x7fd6400d3790 0x7fd6400b7a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:25.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.423+0000 7fd63ffff640 1 -- 172.21.15.27:0/4265682558 >> v2:172.21.15.45:3300/0 conn(0x7fd6400d3790 msgr2=0x7fd6400b7a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:25.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.423+0000 7fd63ffff640 1 --2- 172.21.15.27:0/4265682558 >> v2:172.21.15.45:3300/0 conn(0x7fd6400d3790 0x7fd6400b7a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:25.429 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.423+0000 7fd63ffff640 1 -- 172.21.15.27:0/4265682558 >> v1:172.21.15.45:6789/0 conn(0x7fd640162040 legacy=0x7fd64006d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:25.429 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.423+0000 7fd63ffff640 1 -- 172.21.15.27:0/4265682558 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6400d30d0 con 0x7fd64006dfd0 2024-09-12T03:17:25.429 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.423+0000 7fd645097640 1 --2- 172.21.15.27:0/4265682558 >> v2:172.21.15.45:3300/0 conn(0x7fd6400d3790 0x7fd6400b7a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:17:25.429 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.424+0000 7fd63ffff640 1 --2- 172.21.15.27:0/4265682558 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd64006dfd0 0x7fd6400d1da0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fd6400d1c90 tx=0x7fd62c02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d315947d0f45878 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:25.429 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.424+0000 7fd63f7fe640 1 -- 172.21.15.27:0/4265682558 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd62c037070 con 0x7fd64006dfd0 2024-09-12T03:17:25.430 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.424+0000 7fd63f7fe640 1 -- 172.21.15.27:0/4265682558 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd62c032440 con 0x7fd64006dfd0 2024-09-12T03:17:25.430 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.424+0000 7fd63f7fe640 1 -- 172.21.15.27:0/4265682558 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd62c032810 con 0x7fd64006dfd0 2024-09-12T03:17:25.430 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.424+0000 7fd646afa640 1 -- 172.21.15.27:0/4265682558 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd64006dfd0 msgr2=0x7fd6400d1da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:25.430 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.424+0000 7fd646afa640 1 --2- 172.21.15.27:0/4265682558 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd64006dfd0 0x7fd6400d1da0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fd6400d1c90 tx=0x7fd62c02a230 comp rx=0 tx=0).stop 2024-09-12T03:17:25.430 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.424+0000 7fd646afa640 1 -- 172.21.15.27:0/4265682558 shutdown_connections 2024-09-12T03:17:25.431 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.424+0000 7fd646afa640 1 --2- 172.21.15.27:0/4265682558 >> v2:172.21.15.45:3300/0 conn(0x7fd6400d3790 0x7fd6400b7a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:25.431 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.424+0000 7fd646afa640 1 --2- 172.21.15.27:0/4265682558 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd64006dfd0 0x7fd6400d1da0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:25.431 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.424+0000 7fd646afa640 1 -- 172.21.15.27:0/4265682558 >> 172.21.15.27:0/4265682558 conn(0x7fd6400b6700 msgr2=0x7fd6400b6b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:25.431 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.424+0000 7fd646afa640 1 -- 172.21.15.27:0/4265682558 shutdown_connections 2024-09-12T03:17:25.432 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.424+0000 7fd646afa640 1 -- 172.21.15.27:0/4265682558 wait complete. 2024-09-12T03:17:25.432 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.426+0000 7fd646afa640 1 Processor -- start 2024-09-12T03:17:25.432 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.426+0000 7fd646afa640 1 -- start start 2024-09-12T03:17:25.437 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.427+0000 7fd646afa640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fd64006dfd0 0x7fd64014dc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:25.437 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.427+0000 7fd646afa640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd6400d3790 0x7fd64015ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:25.437 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.427+0000 7fd646afa640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd640162040 0x7fd64015d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:25.438 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.427+0000 7fd646afa640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fd6400d4530 con 0x7fd6400d3790 2024-09-12T03:17:25.438 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.427+0000 7fd646afa640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fd6400cee00 con 0x7fd640162040 2024-09-12T03:17:25.438 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.427+0000 7fd646afa640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fd6400d4150 con 0x7fd64006dfd0 2024-09-12T03:17:25.438 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.427+0000 7fd63ffff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd6400d3790 0x7fd64015ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:25.438 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.427+0000 7fd63ffff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd6400d3790 0x7fd64015ce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60030/0 (socket says 172.21.15.27:60030) 2024-09-12T03:17:25.439 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.427+0000 7fd63ffff640 1 -- 172.21.15.27:0/1185539071 learned_addr learned my addr 172.21.15.27:0/1185539071 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:17:25.439 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.427+0000 7fd645097640 1 --2- 172.21.15.27:0/1185539071 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd640162040 0x7fd64015d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:25.439 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.427+0000 7fd644896640 1 --2- 172.21.15.27:0/1185539071 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fd64006dfd0 0x7fd64014dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:25.439 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.427+0000 7fd63ffff640 1 -- 172.21.15.27:0/1185539071 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fd64006dfd0 msgr2=0x7fd64014dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:25.439 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.427+0000 7fd63ffff640 1 --2- 172.21.15.27:0/1185539071 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fd64006dfd0 0x7fd64014dc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:25.439 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.427+0000 7fd63ffff640 1 -- 172.21.15.27:0/1185539071 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd640162040 msgr2=0x7fd64015d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:25.440 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.427+0000 7fd63ffff640 1 --2- 172.21.15.27:0/1185539071 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd640162040 0x7fd64015d370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:25.440 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.427+0000 7fd63ffff640 1 -- 172.21.15.27:0/1185539071 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd64014e1b0 con 0x7fd6400d3790 2024-09-12T03:17:25.440 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.427+0000 7fd63ffff640 1 --2- 172.21.15.27:0/1185539071 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd6400d3790 0x7fd64015ce30 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fd62c000c00 tx=0x7fd62c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:25.440 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.427+0000 7fd63d7fa640 1 -- 172.21.15.27:0/1185539071 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd62c041020 con 0x7fd6400d3790 2024-09-12T03:17:25.440 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.427+0000 7fd63d7fa640 1 -- 172.21.15.27:0/1185539071 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd62c030970 con 0x7fd6400d3790 2024-09-12T03:17:25.440 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.427+0000 7fd646afa640 1 -- 172.21.15.27:0/1185539071 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd64014e480 con 0x7fd6400d3790 2024-09-12T03:17:25.440 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.427+0000 7fd63d7fa640 1 -- 172.21.15.27:0/1185539071 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd62c037040 con 0x7fd6400d3790 2024-09-12T03:17:25.440 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.427+0000 7fd646afa640 1 -- 172.21.15.27:0/1185539071 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd64014ea40 con 0x7fd6400d3790 2024-09-12T03:17:25.440 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.428+0000 7fd646afa640 1 -- 172.21.15.27:0/1185539071 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd64006e6e0 con 0x7fd6400d3790 2024-09-12T03:17:25.440 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.429+0000 7fd63d7fa640 1 -- 172.21.15.27:0/1185539071 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 6) ==== 90549+0+0 (secure 0 0 0) 0x7fd62c044020 con 0x7fd6400d3790 2024-09-12T03:17:25.440 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.429+0000 7fd63d7fa640 1 --2- 172.21.15.27:0/1185539071 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fd62406bec0 0x7fd62406e380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:25.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.429+0000 7fd63d7fa640 1 -- 172.21.15.27:0/1185539071 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7fd62c0ab830 con 0x7fd6400d3790 2024-09-12T03:17:25.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.430+0000 7fd644896640 1 --2- 172.21.15.27:0/1185539071 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fd62406bec0 0x7fd62406e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:25.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.430+0000 7fd644896640 1 --2- 172.21.15.27:0/1185539071 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fd62406bec0 0x7fd62406e380 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fd634006fd0 tx=0x7fd634008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:25.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.434+0000 7fd63d7fa640 1 -- 172.21.15.27:0/1185539071 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fd62c04b030 con 0x7fd6400d3790 2024-09-12T03:17:25.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.631+0000 7fd646afa640 1 -- 172.21.15.27:0/1185539071 --> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup rm", "vol_name": "cephfs", "group_name": "subvol_grp70490", "target": ["mon-mgr", ""]}) -- 0x7fd6400d52c0 con 0x7fd62406bec0 2024-09-12T03:17:25.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.636+0000 7fd63d7fa640 1 -- 172.21.15.27:0/1185539071 <== mgr.4102 v2:172.21.15.45:6832/3923821219 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fd6400d52c0 con 0x7fd62406bec0 2024-09-12T03:17:25.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.638+0000 7fd646afa640 1 -- 172.21.15.27:0/1185539071 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fd62406bec0 msgr2=0x7fd62406e380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:25.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.638+0000 7fd646afa640 1 --2- 172.21.15.27:0/1185539071 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fd62406bec0 0x7fd62406e380 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fd634006fd0 tx=0x7fd634008040 comp rx=0 tx=0).stop 2024-09-12T03:17:25.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.638+0000 7fd646afa640 1 -- 172.21.15.27:0/1185539071 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd6400d3790 msgr2=0x7fd64015ce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:25.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.638+0000 7fd646afa640 1 --2- 172.21.15.27:0/1185539071 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd6400d3790 0x7fd64015ce30 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fd62c000c00 tx=0x7fd62c0046c0 comp rx=0 tx=0).stop 2024-09-12T03:17:25.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.638+0000 7fd646afa640 1 -- 172.21.15.27:0/1185539071 shutdown_connections 2024-09-12T03:17:25.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.638+0000 7fd646afa640 1 --2- 172.21.15.27:0/1185539071 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fd62406bec0 0x7fd62406e380 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:25.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.638+0000 7fd646afa640 1 --2- 172.21.15.27:0/1185539071 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd640162040 0x7fd64015d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:25.641 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.638+0000 7fd646afa640 1 --2- 172.21.15.27:0/1185539071 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd6400d3790 0x7fd64015ce30 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:25.641 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.638+0000 7fd646afa640 1 --2- 172.21.15.27:0/1185539071 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fd64006dfd0 0x7fd64014dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:25.641 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.638+0000 7fd646afa640 1 -- 172.21.15.27:0/1185539071 >> 172.21.15.27:0/1185539071 conn(0x7fd6400b6700 msgr2=0x7fd6400b5660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:25.641 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.638+0000 7fd646afa640 1 -- 172.21.15.27:0/1185539071 shutdown_connections 2024-09-12T03:17:25.641 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.638+0000 7fd646afa640 1 -- 172.21.15.27:0/1185539071 wait complete. 2024-09-12T03:17:25.650 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-12T03:17:25.650 DEBUG:teuthology.orchestra.run.smithi027:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-12T03:17:25.690 DEBUG:teuthology.orchestra.run.smithi045:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-12T03:17:25.738 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-12T03:17:25.738 INFO:tasks.cephfs.mount:Terminating background process 2024-09-12T03:17:25.739 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-12T03:17:25.739 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T03:17:25.755 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-12T03:17:25.755 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T03:17:25.812 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-12T03:17:25.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.954+0000 7fb721efa640 1 Processor -- start 2024-09-12T03:17:25.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.954+0000 7fb721efa640 1 -- start start 2024-09-12T03:17:25.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.955+0000 7fb721efa640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb71c0b55d0 0x7fb71c06bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:25.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.955+0000 7fb721efa640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb71c0b5a20 con 0x7fb71c1590d0 2024-09-12T03:17:25.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.955+0000 7fb721efa640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb71c15fd10 con 0x7fb71c158cf0 2024-09-12T03:17:25.959 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.955+0000 7fb721efa640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fb71c162c10 con 0x7fb71c0b55d0 2024-09-12T03:17:25.959 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.955+0000 7fb71affd640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fb71c1590d0 0x7fb71c15f600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:51382/0 (socket says 172.21.15.27:51382) 2024-09-12T03:17:25.959 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.955+0000 7fb71affd640 1 -- 172.21.15.27:0/3298746011 learned_addr learned my addr 172.21.15.27:0/3298746011 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:17:25.959 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.955+0000 7fb71a7fc640 1 -- 172.21.15.27:0/3298746011 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 816468058 0 0) 0x7fb71c0b5a20 con 0x7fb71c1590d0 2024-09-12T03:17:25.959 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.955+0000 7fb71a7fc640 1 -- 172.21.15.27:0/3298746011 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb708003520 con 0x7fb71c1590d0 2024-09-12T03:17:25.959 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.955+0000 7fb71bfff640 1 --2- 172.21.15.27:0/3298746011 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb71c0b55d0 0x7fb71c06bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:25.959 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.956+0000 7fb71a7fc640 1 -- 172.21.15.27:0/3298746011 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1740098241 0 0) 0x7fb708003520 con 0x7fb71c1590d0 2024-09-12T03:17:25.959 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.956+0000 7fb71a7fc640 1 -- 172.21.15.27:0/3298746011 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb71c0b55d0 msgr2=0x7fb71c06bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:25.959 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.956+0000 7fb71a7fc640 1 --2- 172.21.15.27:0/3298746011 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb71c0b55d0 0x7fb71c06bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:25.959 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.956+0000 7fb71a7fc640 1 -- 172.21.15.27:0/3298746011 >> v1:172.21.15.45:6789/0 conn(0x7fb71c158cf0 legacy=0x7fb71c162430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:25.959 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.956+0000 7fb71a7fc640 1 -- 172.21.15.27:0/3298746011 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb71c162ef0 con 0x7fb71c1590d0 2024-09-12T03:17:25.959 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.956+0000 7fb71a7fc640 1 -- 172.21.15.27:0/3298746011 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 847755213 0 0) 0x7fb704001690 con 0x7fb71c1590d0 2024-09-12T03:17:25.959 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.956+0000 7fb71a7fc640 1 -- 172.21.15.27:0/3298746011 >> v1:172.21.15.27:6789/0 conn(0x7fb71c1590d0 legacy=0x7fb71c15f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:25.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.956+0000 7fb71a7fc640 1 --2- 172.21.15.27:0/3298746011 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb71c0b55d0 0x7fb7080039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:25.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.956+0000 7fb71a7fc640 1 --2- 172.21.15.27:0/3298746011 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb708003f60 0x7fb708004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:25.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.956+0000 7fb71a7fc640 1 --2- 172.21.15.27:0/3298746011 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb708004d40 0x7fb7080051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:25.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.956+0000 7fb71a7fc640 1 -- 172.21.15.27:0/3298746011 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fb71c162c10 con 0x7fb708004d40 2024-09-12T03:17:25.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.956+0000 7fb71a7fc640 1 -- 172.21.15.27:0/3298746011 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb70800bc00 con 0x7fb708003f60 2024-09-12T03:17:25.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.956+0000 7fb71a7fc640 1 -- 172.21.15.27:0/3298746011 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fb70800be00 con 0x7fb71c0b55d0 2024-09-12T03:17:25.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.956+0000 7fb71bfff640 1 --2- 172.21.15.27:0/3298746011 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb708004d40 0x7fb7080051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:25.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.956+0000 7fb71affd640 1 --2- 172.21.15.27:0/3298746011 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb708003f60 0x7fb708004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:25.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.956+0000 7fb71b7fe640 1 --2- 172.21.15.27:0/3298746011 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb71c0b55d0 0x7fb7080039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:25.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.956+0000 7fb71bfff640 1 -- 172.21.15.27:0/3298746011 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb71c0b55d0 msgr2=0x7fb7080039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:25.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.956+0000 7fb71bfff640 1 --2- 172.21.15.27:0/3298746011 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb71c0b55d0 0x7fb7080039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:25.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.956+0000 7fb71bfff640 1 -- 172.21.15.27:0/3298746011 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb708003f60 msgr2=0x7fb708004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:25.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.956+0000 7fb71bfff640 1 --2- 172.21.15.27:0/3298746011 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb708003f60 0x7fb708004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:25.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.956+0000 7fb71bfff640 1 -- 172.21.15.27:0/3298746011 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb708003520 con 0x7fb708004d40 2024-09-12T03:17:25.961 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.956+0000 7fb71b7fe640 1 --2- 172.21.15.27:0/3298746011 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb71c0b55d0 0x7fb7080039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:17:25.961 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.957+0000 7fb71bfff640 1 --2- 172.21.15.27:0/3298746011 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb708004d40 0x7fb7080051c0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fb714009ac0 tx=0x7fb708005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbbe921de9def827 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:25.961 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.957+0000 7fb71a7fc640 1 -- 172.21.15.27:0/3298746011 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb714036070 con 0x7fb708004d40 2024-09-12T03:17:25.961 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.957+0000 7fb71a7fc640 1 -- 172.21.15.27:0/3298746011 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb714031400 con 0x7fb708004d40 2024-09-12T03:17:25.961 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.957+0000 7fb71a7fc640 1 -- 172.21.15.27:0/3298746011 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb714009690 con 0x7fb708004d40 2024-09-12T03:17:25.961 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.957+0000 7fb721efa640 1 -- 172.21.15.27:0/3298746011 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb708004d40 msgr2=0x7fb7080051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:25.961 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.957+0000 7fb721efa640 1 --2- 172.21.15.27:0/3298746011 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb708004d40 0x7fb7080051c0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fb714009ac0 tx=0x7fb708005830 comp rx=0 tx=0).stop 2024-09-12T03:17:25.961 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.957+0000 7fb71bfff640 1 -- 172.21.15.27:0/3298746011 reap_dead start 2024-09-12T03:17:25.961 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.958+0000 7fb721efa640 1 -- 172.21.15.27:0/3298746011 shutdown_connections 2024-09-12T03:17:25.961 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.958+0000 7fb721efa640 1 -- 172.21.15.27:0/3298746011 >> 172.21.15.27:0/3298746011 conn(0x7fb71c06cc40 msgr2=0x7fb71c06d050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:25.961 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.958+0000 7fb721efa640 1 -- 172.21.15.27:0/3298746011 shutdown_connections 2024-09-12T03:17:25.961 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.958+0000 7fb721efa640 1 -- 172.21.15.27:0/3298746011 wait complete. 2024-09-12T03:17:25.964 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.959+0000 7fb721efa640 1 Processor -- start 2024-09-12T03:17:25.964 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.959+0000 7fb721efa640 1 -- start start 2024-09-12T03:17:25.964 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.960+0000 7fb721efa640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb708004d40 0x7fb71c0b1b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:25.964 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.960+0000 7fb721efa640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb71c0b1380 0x7fb71c150d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:25.964 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.960+0000 7fb721efa640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb71c0b3730 0x7fb71c0b3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:25.964 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.960+0000 7fb721efa640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fb71c06bb30 con 0x7fb71c0b3730 2024-09-12T03:17:25.964 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.960+0000 7fb721efa640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb71c15edc0 con 0x7fb71c0b1380 2024-09-12T03:17:25.964 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.960+0000 7fb721efa640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fb71c06bcb0 con 0x7fb708004d40 2024-09-12T03:17:25.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.960+0000 7fb71bfff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb71c0b3730 0x7fb71c0b3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:25.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.960+0000 7fb71bfff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb71c0b3730 0x7fb71c0b3bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60058/0 (socket says 172.21.15.27:60058) 2024-09-12T03:17:25.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.960+0000 7fb71bfff640 1 -- 172.21.15.27:0/412313354 learned_addr learned my addr 172.21.15.27:0/412313354 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:17:25.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.960+0000 7fb71b7fe640 1 --2- 172.21.15.27:0/412313354 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb708004d40 0x7fb71c0b1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:25.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.960+0000 7fb71affd640 1 --2- 172.21.15.27:0/412313354 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb71c0b1380 0x7fb71c150d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:25.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.960+0000 7fb71bfff640 1 -- 172.21.15.27:0/412313354 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb708004d40 msgr2=0x7fb71c0b1b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:25.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.960+0000 7fb71bfff640 1 --2- 172.21.15.27:0/412313354 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb708004d40 0x7fb71c0b1b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:25.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.960+0000 7fb71bfff640 1 -- 172.21.15.27:0/412313354 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb71c0b1380 msgr2=0x7fb71c150d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:25.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.960+0000 7fb71bfff640 1 --2- 172.21.15.27:0/412313354 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb71c0b1380 0x7fb71c150d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:25.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.960+0000 7fb71bfff640 1 -- 172.21.15.27:0/412313354 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb71c0b1800 con 0x7fb71c0b3730 2024-09-12T03:17:25.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.960+0000 7fb71bfff640 1 --2- 172.21.15.27:0/412313354 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb71c0b3730 0x7fb71c0b3bb0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fb714009be0 tx=0x7fb714004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:25.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.960+0000 7fb713fff640 1 -- 172.21.15.27:0/412313354 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb714036040 con 0x7fb71c0b3730 2024-09-12T03:17:25.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.960+0000 7fb713fff640 1 -- 172.21.15.27:0/412313354 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb714041070 con 0x7fb71c0b3730 2024-09-12T03:17:25.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.960+0000 7fb721efa640 1 -- 172.21.15.27:0/412313354 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb71c0b2270 con 0x7fb71c0b3730 2024-09-12T03:17:25.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.960+0000 7fb713fff640 1 -- 172.21.15.27:0/412313354 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb714009690 con 0x7fb71c0b3730 2024-09-12T03:17:25.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.961+0000 7fb721efa640 1 -- 172.21.15.27:0/412313354 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb71c153fd0 con 0x7fb71c0b3730 2024-09-12T03:17:25.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.962+0000 7fb721efa640 1 -- 172.21.15.27:0/412313354 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb71c06c040 con 0x7fb71c0b3730 2024-09-12T03:17:25.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.962+0000 7fb713fff640 1 -- 172.21.15.27:0/412313354 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 6) ==== 90549+0+0 (secure 0 0 0) 0x7fb714039070 con 0x7fb71c0b3730 2024-09-12T03:17:25.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.962+0000 7fb713fff640 1 --2- 172.21.15.27:0/412313354 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fb6fc06bec0 0x7fb6fc06e380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:25.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.963+0000 7fb713fff640 1 -- 172.21.15.27:0/412313354 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7fb7140aa5f0 con 0x7fb71c0b3730 2024-09-12T03:17:25.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.963+0000 7fb71b7fe640 1 --2- 172.21.15.27:0/412313354 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fb6fc06bec0 0x7fb6fc06e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:25.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.963+0000 7fb71b7fe640 1 --2- 172.21.15.27:0/412313354 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fb6fc06bec0 0x7fb6fc06e380 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fb70c001e40 tx=0x7fb70c009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:25.967 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:25.967+0000 7fb713fff640 1 -- 172.21.15.27:0/412313354 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fb71404a020 con 0x7fb71c0b3730 2024-09-12T03:17:26.154 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:26.153+0000 7fb721efa640 1 -- 172.21.15.27:0/412313354 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fb71c075440 con 0x7fb71c0b3730 2024-09-12T03:17:26.155 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:26.154+0000 7fb713fff640 1 -- 172.21.15.27:0/412313354 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v32) ==== 80+0+0 (secure 0 0 0) 0x7fb7140788d0 con 0x7fb71c0b3730 2024-09-12T03:17:26.155 INFO:teuthology.orchestra.run.smithi027.stderr:listed 0 entries 2024-09-12T03:17:26.157 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:26.156+0000 7fb721efa640 1 -- 172.21.15.27:0/412313354 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fb6fc06bec0 msgr2=0x7fb6fc06e380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:26.157 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:26.156+0000 7fb721efa640 1 --2- 172.21.15.27:0/412313354 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fb6fc06bec0 0x7fb6fc06e380 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fb70c001e40 tx=0x7fb70c009040 comp rx=0 tx=0).stop 2024-09-12T03:17:26.158 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:26.156+0000 7fb721efa640 1 -- 172.21.15.27:0/412313354 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb71c0b3730 msgr2=0x7fb71c0b3bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:26.158 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:26.156+0000 7fb721efa640 1 --2- 172.21.15.27:0/412313354 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb71c0b3730 0x7fb71c0b3bb0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fb714009be0 tx=0x7fb714004730 comp rx=0 tx=0).stop 2024-09-12T03:17:26.158 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:26.156+0000 7fb721efa640 1 -- 172.21.15.27:0/412313354 shutdown_connections 2024-09-12T03:17:26.158 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:26.156+0000 7fb721efa640 1 --2- 172.21.15.27:0/412313354 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fb6fc06bec0 0x7fb6fc06e380 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:26.158 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:26.156+0000 7fb721efa640 1 --2- 172.21.15.27:0/412313354 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb71c0b3730 0x7fb71c0b3bb0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:26.158 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:26.156+0000 7fb721efa640 1 --2- 172.21.15.27:0/412313354 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb71c0b1380 0x7fb71c150d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:26.158 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:26.156+0000 7fb721efa640 1 --2- 172.21.15.27:0/412313354 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb708004d40 0x7fb71c0b1b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:26.158 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:26.156+0000 7fb721efa640 1 -- 172.21.15.27:0/412313354 >> 172.21.15.27:0/412313354 conn(0x7fb71c06cc40 msgr2=0x7fb71c15dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:26.158 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:26.156+0000 7fb721efa640 1 -- 172.21.15.27:0/412313354 shutdown_connections 2024-09-12T03:17:26.158 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:26.156+0000 7fb721efa640 1 -- 172.21.15.27:0/412313354 wait complete. 2024-09-12T03:17:26.168 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-12T03:17:26.168 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:17:26.168 DEBUG:teuthology.orchestra.run.smithi027:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-12T03:17:26.230 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi027.front.sepia.ceph.com 2024-09-12T03:17:26.230 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:17:26.230 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-12T03:17:26.252 INFO:tasks.cephfs.mount:Terminating background process 2024-09-12T03:17:26.252 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-12T03:17:26.252 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T03:17:26.271 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-12T03:17:26.271 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T03:17:26.330 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-12T03:17:26.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:26.432+0000 7fc39e02e640 1 Processor -- start 2024-09-12T03:17:26.434 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:26.432+0000 7fc39e02e640 1 -- start start 2024-09-12T03:17:26.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:26.433+0000 7fc39e02e640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fc3980b5a70 0x7fc39815f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:26.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:26.433+0000 7fc39e02e640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc3980ae8d0 0x7fc3980728b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:26.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:26.433+0000 7fc39e02e640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc39815fb40 con 0x7fc398072df0 2024-09-12T03:17:26.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:26.433+0000 7fc39e02e640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fc3980aee90 con 0x7fc3980ae8d0 2024-09-12T03:17:26.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:26.433+0000 7fc39e02e640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fc3980af170 con 0x7fc3980b5a70 2024-09-12T03:17:26.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:26.434+0000 7fc3977fe640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fc3980b5a70 0x7fc39815f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:26.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:26.434+0000 7fc3977fe640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fc3980b5a70 0x7fc39815f600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3301/0 says I am v2:172.21.15.27:45092/0 (socket says 172.21.15.27:45092) 2024-09-12T03:17:26.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:26.434+0000 7fc3977fe640 1 -- 172.21.15.27:0/38995422 learned_addr learned my addr 172.21.15.27:0/38995422 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:17:26.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:26.434+0000 7fc396ffd640 1 --2- 172.21.15.27:0/38995422 >> v2:172.21.15.45:3300/0 conn(0x7fc3980ae8d0 0x7fc3980728b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:26.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:26.434+0000 7fc3977fe640 1 -- 172.21.15.27:0/38995422 >> v2:172.21.15.45:3300/0 conn(0x7fc3980ae8d0 msgr2=0x7fc3980728b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:26.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:26.434+0000 7fc3977fe640 1 --2- 172.21.15.27:0/38995422 >> v2:172.21.15.45:3300/0 conn(0x7fc3980ae8d0 0x7fc3980728b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:26.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:26.434+0000 7fc3977fe640 1 -- 172.21.15.27:0/38995422 >> v1:172.21.15.45:6789/0 conn(0x7fc398072df0 legacy=0x7fc3981621c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:26.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:26.434+0000 7fc3977fe640 1 -- 172.21.15.27:0/38995422 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3980af450 con 0x7fc3980b5a70 2024-09-12T03:17:26.437 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:26.435+0000 7fc3977fe640 1 --2- 172.21.15.27:0/38995422 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fc3980b5a70 0x7fc39815f600 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fc388009870 tx=0x7fc38802a1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b3d6a9a2d4de37c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:26.437 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:26.435+0000 7fc3967fc640 1 -- 172.21.15.27:0/38995422 <== mon.2 v2:172.21.15.45:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc388037070 con 0x7fc3980b5a70 2024-09-12T03:17:26.437 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:26.435+0000 7fc3967fc640 1 -- 172.21.15.27:0/38995422 <== mon.2 v2:172.21.15.45:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc388032440 con 0x7fc3980b5a70 2024-09-12T03:17:26.437 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:26.435+0000 7fc3967fc640 1 -- 172.21.15.27:0/38995422 <== mon.2 v2:172.21.15.45:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc388032810 con 0x7fc3980b5a70 2024-09-12T03:17:26.437 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:26.435+0000 7fc39e02e640 1 -- 172.21.15.27:0/38995422 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fc3980b5a70 msgr2=0x7fc39815f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:26.437 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:26.435+0000 7fc39e02e640 1 --2- 172.21.15.27:0/38995422 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fc3980b5a70 0x7fc39815f600 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fc388009870 tx=0x7fc38802a1c0 comp rx=0 tx=0).stop 2024-09-12T03:17:26.437 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:26.436+0000 7fc39e02e640 1 -- 172.21.15.27:0/38995422 shutdown_connections 2024-09-12T03:17:26.437 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:26.436+0000 7fc39e02e640 1 --2- 172.21.15.27:0/38995422 >> v2:172.21.15.45:3300/0 conn(0x7fc3980ae8d0 0x7fc3980728b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:26.437 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:26.436+0000 7fc39e02e640 1 --2- 172.21.15.27:0/38995422 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fc3980b5a70 0x7fc39815f600 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:26.438 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:26.436+0000 7fc39e02e640 1 -- 172.21.15.27:0/38995422 >> 172.21.15.27:0/38995422 conn(0x7fc39806ca30 msgr2=0x7fc39806ce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:26.438 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:26.436+0000 7fc39e02e640 1 -- 172.21.15.27:0/38995422 shutdown_connections 2024-09-12T03:17:26.438 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:26.436+0000 7fc39e02e640 1 -- 172.21.15.27:0/38995422 wait complete. 2024-09-12T03:17:26.439 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:26.437+0000 7fc39e02e640 1 Processor -- start 2024-09-12T03:17:26.440 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:26.437+0000 7fc39e02e640 1 -- start start 2024-09-12T03:17:26.440 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:26.438+0000 7fc39e02e640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fc398072df0 0x7fc3980ce2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:26.440 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:26.438+0000 7fc39e02e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc3980ae8d0 0x7fc398159ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:26.440 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:26.438+0000 7fc39e02e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc3980b5a70 0x7fc39815f200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:26.440 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:26.438+0000 7fc39e02e640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fc39806fac0 con 0x7fc3980b5a70 2024-09-12T03:17:26.440 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:26.438+0000 7fc39e02e640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc39806f940 con 0x7fc3980ae8d0 2024-09-12T03:17:26.440 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:26.438+0000 7fc39e02e640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fc39806fc40 con 0x7fc398072df0 2024-09-12T03:17:26.440 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:26.438+0000 7fc397fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc3980b5a70 0x7fc39815f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:26.440 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:26.438+0000 7fc397fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc3980b5a70 0x7fc39815f200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60074/0 (socket says 172.21.15.27:60074) 2024-09-12T03:17:26.440 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:26.438+0000 7fc397fff640 1 -- 172.21.15.27:0/3540848282 learned_addr learned my addr 172.21.15.27:0/3540848282 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:17:26.440 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:26.438+0000 7fc3977fe640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fc398072df0 0x7fc3980ce2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:26.440 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:26.438+0000 7fc396ffd640 1 --2- 172.21.15.27:0/3540848282 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc3980ae8d0 0x7fc398159ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:26.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:26.438+0000 7fc397fff640 1 -- 172.21.15.27:0/3540848282 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fc398072df0 msgr2=0x7fc3980ce2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:26.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:26.438+0000 7fc397fff640 1 --2- 172.21.15.27:0/3540848282 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fc398072df0 0x7fc3980ce2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:26.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:26.438+0000 7fc397fff640 1 -- 172.21.15.27:0/3540848282 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc3980ae8d0 msgr2=0x7fc398159ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:26.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:26.438+0000 7fc397fff640 1 --2- 172.21.15.27:0/3540848282 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc3980ae8d0 0x7fc398159ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:26.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:26.438+0000 7fc397fff640 1 -- 172.21.15.27:0/3540848282 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc39815f740 con 0x7fc3980b5a70 2024-09-12T03:17:26.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:26.438+0000 7fc3977fe640 1 --2- 172.21.15.27:0/3540848282 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fc398072df0 0x7fc3980ce2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:17:26.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:26.438+0000 7fc397fff640 1 --2- 172.21.15.27:0/3540848282 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc3980b5a70 0x7fc39815f200 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fc39000c760 tx=0x7fc39000cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:26.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:26.438+0000 7fc37ffff640 1 -- 172.21.15.27:0/3540848282 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc390003b90 con 0x7fc3980b5a70 2024-09-12T03:17:26.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:26.439+0000 7fc37ffff640 1 -- 172.21.15.27:0/3540848282 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc390003d30 con 0x7fc3980b5a70 2024-09-12T03:17:26.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:26.439+0000 7fc37ffff640 1 -- 172.21.15.27:0/3540848282 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc39000aa30 con 0x7fc3980b5a70 2024-09-12T03:17:26.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:26.439+0000 7fc39e02e640 1 -- 172.21.15.27:0/3540848282 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3980ce8b0 con 0x7fc3980b5a70 2024-09-12T03:17:26.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:26.439+0000 7fc39e02e640 1 -- 172.21.15.27:0/3540848282 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc39815c890 con 0x7fc3980b5a70 2024-09-12T03:17:26.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:26.440+0000 7fc39e02e640 1 -- 172.21.15.27:0/3540848282 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3980728b0 con 0x7fc3980b5a70 2024-09-12T03:17:26.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:26.440+0000 7fc37ffff640 1 -- 172.21.15.27:0/3540848282 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 6) ==== 90549+0+0 (secure 0 0 0) 0x7fc39000abd0 con 0x7fc3980b5a70 2024-09-12T03:17:26.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:26.440+0000 7fc37ffff640 1 --2- 172.21.15.27:0/3540848282 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fc38406bec0 0x7fc38406e380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:26.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:26.441+0000 7fc37ffff640 1 -- 172.21.15.27:0/3540848282 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7fc39008ab30 con 0x7fc3980b5a70 2024-09-12T03:17:26.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:26.441+0000 7fc3977fe640 1 --2- 172.21.15.27:0/3540848282 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fc38406bec0 0x7fc38406e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:26.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:26.441+0000 7fc3977fe640 1 --2- 172.21.15.27:0/3540848282 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fc38406bec0 0x7fc38406e380 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fc388034000 tx=0x7fc38802e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:26.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:26.445+0000 7fc37ffff640 1 -- 172.21.15.27:0/3540848282 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fc390058f30 con 0x7fc3980b5a70 2024-09-12T03:17:26.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:26.631+0000 7fc39e02e640 1 -- 172.21.15.27:0/3540848282 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fc398157bb0 con 0x7fc3980b5a70 2024-09-12T03:17:26.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:26.631+0000 7fc37ffff640 1 -- 172.21.15.27:0/3540848282 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v32) ==== 80+0+0 (secure 0 0 0) 0x7fc39005d230 con 0x7fc3980b5a70 2024-09-12T03:17:26.632 INFO:teuthology.orchestra.run.smithi027.stderr:listed 0 entries 2024-09-12T03:17:26.634 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:26.633+0000 7fc39e02e640 1 -- 172.21.15.27:0/3540848282 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fc38406bec0 msgr2=0x7fc38406e380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:26.634 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:26.633+0000 7fc39e02e640 1 --2- 172.21.15.27:0/3540848282 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fc38406bec0 0x7fc38406e380 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fc388034000 tx=0x7fc38802e000 comp rx=0 tx=0).stop 2024-09-12T03:17:26.634 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:26.633+0000 7fc39e02e640 1 -- 172.21.15.27:0/3540848282 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc3980b5a70 msgr2=0x7fc39815f200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:26.634 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:26.633+0000 7fc39e02e640 1 --2- 172.21.15.27:0/3540848282 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc3980b5a70 0x7fc39815f200 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fc39000c760 tx=0x7fc39000cc20 comp rx=0 tx=0).stop 2024-09-12T03:17:26.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:26.633+0000 7fc39e02e640 1 -- 172.21.15.27:0/3540848282 shutdown_connections 2024-09-12T03:17:26.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:26.633+0000 7fc39e02e640 1 --2- 172.21.15.27:0/3540848282 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fc38406bec0 0x7fc38406e380 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:26.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:26.633+0000 7fc39e02e640 1 --2- 172.21.15.27:0/3540848282 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc3980b5a70 0x7fc39815f200 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:26.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:26.633+0000 7fc39e02e640 1 --2- 172.21.15.27:0/3540848282 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc3980ae8d0 0x7fc398159ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:26.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:26.633+0000 7fc39e02e640 1 --2- 172.21.15.27:0/3540848282 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fc398072df0 0x7fc3980ce2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:26.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:26.633+0000 7fc39e02e640 1 -- 172.21.15.27:0/3540848282 >> 172.21.15.27:0/3540848282 conn(0x7fc39806ca30 msgr2=0x7fc398160d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:26.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:26.633+0000 7fc39e02e640 1 -- 172.21.15.27:0/3540848282 shutdown_connections 2024-09-12T03:17:26.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:26.633+0000 7fc39e02e640 1 -- 172.21.15.27:0/3540848282 wait complete. 2024-09-12T03:17:26.645 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-09-12T03:17:26.645 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:17:26.645 DEBUG:teuthology.orchestra.run.smithi045:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-09-12T03:17:26.680 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi045.front.sepia.ceph.com 2024-09-12T03:17:26.680 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:17:26.680 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-12T03:17:26.739 INFO:tasks.cephfs.mount:Terminating background process 2024-09-12T03:17:26.739 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-12T03:17:26.739 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T03:17:26.755 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056004,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497432k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16324 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248716k,nr_inodes=812179,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497432k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-12T03:17:26.756 INFO:tasks.cephfs.mount:Terminating background process 2024-09-12T03:17:26.756 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-12T03:17:26.756 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T03:17:26.798 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056795,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16602 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249348k,nr_inodes=812337,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-12T03:17:26.798 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-12T03:17:26.902 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:26.901+0000 7fa5587f8640 1 Processor -- start 2024-09-12T03:17:26.902 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:26.901+0000 7fa5587f8640 1 -- start start 2024-09-12T03:17:26.905 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:26.902+0000 7fa5587f8640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa5500b1ca0 0x7fa55015f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:26.905 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:26.902+0000 7fa5587f8640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa55015fd70 con 0x7fa5500cf660 2024-09-12T03:17:26.905 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:26.902+0000 7fa5587f8640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa55006cc40 con 0x7fa55006f360 2024-09-12T03:17:26.905 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:26.902+0000 7fa5587f8640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fa5500d22f0 con 0x7fa5500b1ca0 2024-09-12T03:17:26.905 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:26.902+0000 7fa556d95640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fa5500cf660 0x7fa5500d2780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:51394/0 (socket says 172.21.15.27:51394) 2024-09-12T03:17:26.905 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:26.902+0000 7fa556d95640 1 -- 172.21.15.27:0/1629360996 learned_addr learned my addr 172.21.15.27:0/1629360996 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:17:26.905 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:26.902+0000 7fa556594640 1 --2- 172.21.15.27:0/1629360996 >> v2:172.21.15.45:3300/0 conn(0x7fa5500b1ca0 0x7fa55015f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:26.906 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:26.902+0000 7fa555592640 1 -- 172.21.15.27:0/1629360996 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3889936284 0 0) 0x7fa548001490 con 0x7fa5500cf660 2024-09-12T03:17:26.906 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:26.902+0000 7fa555592640 1 -- 172.21.15.27:0/1629360996 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa53c003520 con 0x7fa5500cf660 2024-09-12T03:17:26.906 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:26.903+0000 7fa555592640 1 -- 172.21.15.27:0/1629360996 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3491832258 0 0) 0x7fa53c003520 con 0x7fa5500cf660 2024-09-12T03:17:26.906 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:26.903+0000 7fa555592640 1 -- 172.21.15.27:0/1629360996 >> v2:172.21.15.45:3300/0 conn(0x7fa5500b1ca0 msgr2=0x7fa55015f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:26.906 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:26.903+0000 7fa555592640 1 --2- 172.21.15.27:0/1629360996 >> v2:172.21.15.45:3300/0 conn(0x7fa5500b1ca0 0x7fa55015f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:26.906 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:26.903+0000 7fa555592640 1 -- 172.21.15.27:0/1629360996 >> v1:172.21.15.45:6789/0 conn(0x7fa55006f360 legacy=0x7fa5500cef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:26.906 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:26.903+0000 7fa555592640 1 -- 172.21.15.27:0/1629360996 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5500d1ce0 con 0x7fa5500cf660 2024-09-12T03:17:26.906 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:26.903+0000 7fa555592640 1 -- 172.21.15.27:0/1629360996 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 847755213 0 0) 0x7fa548001bc0 con 0x7fa5500cf660 2024-09-12T03:17:26.906 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:26.903+0000 7fa555592640 1 -- 172.21.15.27:0/1629360996 >> v1:172.21.15.27:6789/0 conn(0x7fa5500cf660 legacy=0x7fa5500d2780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:26.906 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:26.903+0000 7fa555592640 1 --2- 172.21.15.27:0/1629360996 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa53c0039b0 0x7fa53c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:26.906 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:26.903+0000 7fa555592640 1 --2- 172.21.15.27:0/1629360996 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa53c004800 0x7fa53c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:26.906 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:26.903+0000 7fa555592640 1 --2- 172.21.15.27:0/1629360996 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa53c0051c0 0x7fa53c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:26.906 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:26.903+0000 7fa555592640 1 -- 172.21.15.27:0/1629360996 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fa5500d22f0 con 0x7fa53c004800 2024-09-12T03:17:26.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:26.903+0000 7fa555592640 1 -- 172.21.15.27:0/1629360996 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa53c00c060 con 0x7fa53c0039b0 2024-09-12T03:17:26.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:26.903+0000 7fa555592640 1 -- 172.21.15.27:0/1629360996 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fa53c00c260 con 0x7fa53c0051c0 2024-09-12T03:17:26.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:26.903+0000 7fa555d93640 1 --2- 172.21.15.27:0/1629360996 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa53c004800 0x7fa53c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:26.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:26.903+0000 7fa556594640 1 --2- 172.21.15.27:0/1629360996 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa53c0039b0 0x7fa53c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:26.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:26.903+0000 7fa556d95640 1 --2- 172.21.15.27:0/1629360996 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa53c0051c0 0x7fa53c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:26.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:26.903+0000 7fa555d93640 1 -- 172.21.15.27:0/1629360996 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa53c0051c0 msgr2=0x7fa53c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:26.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:26.903+0000 7fa555d93640 1 --2- 172.21.15.27:0/1629360996 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa53c0051c0 0x7fa53c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:26.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:26.903+0000 7fa555d93640 1 -- 172.21.15.27:0/1629360996 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa53c0039b0 msgr2=0x7fa53c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:26.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:26.903+0000 7fa555d93640 1 --2- 172.21.15.27:0/1629360996 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa53c0039b0 0x7fa53c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:26.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:26.903+0000 7fa556d95640 1 -- 172.21.15.27:0/1629360996 reap_dead start 2024-09-12T03:17:26.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:26.903+0000 7fa555d93640 1 -- 172.21.15.27:0/1629360996 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa53c003520 con 0x7fa53c004800 2024-09-12T03:17:26.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:26.904+0000 7fa555d93640 1 --2- 172.21.15.27:0/1629360996 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa53c004800 0x7fa53c004c80 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fa540001750 tx=0x7fa53c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7df475e380ff47b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:26.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:26.904+0000 7fa555592640 1 -- 172.21.15.27:0/1629360996 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa540001370 con 0x7fa53c004800 2024-09-12T03:17:26.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:26.904+0000 7fa555592640 1 -- 172.21.15.27:0/1629360996 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5400052d0 con 0x7fa53c004800 2024-09-12T03:17:26.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:26.904+0000 7fa555592640 1 -- 172.21.15.27:0/1629360996 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa54002abc0 con 0x7fa53c004800 2024-09-12T03:17:26.915 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:26.905+0000 7fa5587f8640 1 -- 172.21.15.27:0/1629360996 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa53c004800 msgr2=0x7fa53c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:26.915 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:26.905+0000 7fa5587f8640 1 --2- 172.21.15.27:0/1629360996 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa53c004800 0x7fa53c004c80 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fa540001750 tx=0x7fa53c005830 comp rx=0 tx=0).stop 2024-09-12T03:17:26.915 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:26.905+0000 7fa5587f8640 1 -- 172.21.15.27:0/1629360996 shutdown_connections 2024-09-12T03:17:26.915 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:26.905+0000 7fa5587f8640 1 --2- 172.21.15.27:0/1629360996 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa53c004800 0x7fa53c004c80 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:26.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:26.905+0000 7fa5587f8640 1 -- 172.21.15.27:0/1629360996 >> 172.21.15.27:0/1629360996 conn(0x7fa550073800 msgr2=0x7fa55009ba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:26.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:26.905+0000 7fa5587f8640 1 -- 172.21.15.27:0/1629360996 shutdown_connections 2024-09-12T03:17:26.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:26.905+0000 7fa5587f8640 1 -- 172.21.15.27:0/1629360996 wait complete. 2024-09-12T03:17:26.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:26.906+0000 7fa5587f8640 1 Processor -- start 2024-09-12T03:17:26.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:26.906+0000 7fa5587f8640 1 -- start start 2024-09-12T03:17:26.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:26.907+0000 7fa5587f8640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa53c004800 0x7fa550155500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:26.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:26.907+0000 7fa5587f8640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa5500ce4d0 0x7fa5500afe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:26.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:26.907+0000 7fa5587f8640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa55015e620 0x7fa5500d2680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:26.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:26.907+0000 7fa5587f8640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fa55006d1c0 con 0x7fa5500ce4d0 2024-09-12T03:17:26.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:26.907+0000 7fa5587f8640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa5500d1f80 con 0x7fa55015e620 2024-09-12T03:17:26.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:26.907+0000 7fa5587f8640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fa55006d340 con 0x7fa53c004800 2024-09-12T03:17:26.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:26.907+0000 7fa555d93640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa5500ce4d0 0x7fa5500afe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:26.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:26.907+0000 7fa555d93640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa5500ce4d0 0x7fa5500afe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60090/0 (socket says 172.21.15.27:60090) 2024-09-12T03:17:26.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:26.907+0000 7fa555d93640 1 -- 172.21.15.27:0/3910499679 learned_addr learned my addr 172.21.15.27:0/3910499679 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:17:26.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:26.907+0000 7fa556d95640 1 --2- 172.21.15.27:0/3910499679 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa55015e620 0x7fa5500d2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:26.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:26.907+0000 7fa556594640 1 --2- 172.21.15.27:0/3910499679 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa53c004800 0x7fa550155500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:26.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:26.907+0000 7fa555d93640 1 -- 172.21.15.27:0/3910499679 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa53c004800 msgr2=0x7fa550155500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:26.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:26.907+0000 7fa555d93640 1 --2- 172.21.15.27:0/3910499679 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa53c004800 0x7fa550155500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:26.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:26.907+0000 7fa555d93640 1 -- 172.21.15.27:0/3910499679 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa55015e620 msgr2=0x7fa5500d2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:26.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:26.907+0000 7fa555d93640 1 --2- 172.21.15.27:0/3910499679 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa55015e620 0x7fa5500d2680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:26.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:26.907+0000 7fa555d93640 1 -- 172.21.15.27:0/3910499679 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa55015d580 con 0x7fa5500ce4d0 2024-09-12T03:17:26.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:26.907+0000 7fa556d95640 1 --2- 172.21.15.27:0/3910499679 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa55015e620 0x7fa5500d2680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:17:26.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:26.907+0000 7fa555d93640 1 --2- 172.21.15.27:0/3910499679 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa5500ce4d0 0x7fa5500afe50 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fa540001870 tx=0x7fa540001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:26.918 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:26.907+0000 7fa546ffd640 1 -- 172.21.15.27:0/3910499679 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa54003f030 con 0x7fa5500ce4d0 2024-09-12T03:17:26.918 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:26.908+0000 7fa546ffd640 1 -- 172.21.15.27:0/3910499679 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa540005480 con 0x7fa5500ce4d0 2024-09-12T03:17:26.918 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:26.908+0000 7fa5587f8640 1 -- 172.21.15.27:0/3910499679 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa55015d860 con 0x7fa5500ce4d0 2024-09-12T03:17:26.918 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:26.908+0000 7fa546ffd640 1 -- 172.21.15.27:0/3910499679 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa540037040 con 0x7fa5500ce4d0 2024-09-12T03:17:26.918 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:26.908+0000 7fa5587f8640 1 -- 172.21.15.27:0/3910499679 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa55015de20 con 0x7fa5500ce4d0 2024-09-12T03:17:26.918 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:26.908+0000 7fa5587f8640 1 -- 172.21.15.27:0/3910499679 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5500cfc40 con 0x7fa5500ce4d0 2024-09-12T03:17:26.918 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:26.909+0000 7fa546ffd640 1 -- 172.21.15.27:0/3910499679 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 7) ==== 90651+0+0 (secure 0 0 0) 0x7fa540042070 con 0x7fa5500ce4d0 2024-09-12T03:17:26.918 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:26.909+0000 7fa546ffd640 1 --2- 172.21.15.27:0/3910499679 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fa53406c010 0x7fa53406e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:26.918 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:26.910+0000 7fa546ffd640 1 -- 172.21.15.27:0/3910499679 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7fa5400aa7b0 con 0x7fa5500ce4d0 2024-09-12T03:17:26.918 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:26.910+0000 7fa556594640 1 --2- 172.21.15.27:0/3910499679 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fa53406c010 0x7fa53406e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:26.918 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:26.910+0000 7fa556594640 1 --2- 172.21.15.27:0/3910499679 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fa53406c010 0x7fa53406e4d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fa550156530 tx=0x7fa54c008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:26.918 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:26.914+0000 7fa546ffd640 1 -- 172.21.15.27:0/3910499679 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fa540049070 con 0x7fa5500ce4d0 2024-09-12T03:17:27.124 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.123+0000 7fa5587f8640 1 -- 172.21.15.27:0/3910499679 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa5500d0c70 con 0x7fa5500ce4d0 2024-09-12T03:17:27.125 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.124+0000 7fa546ffd640 1 -- 172.21.15.27:0/3910499679 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) ==== 94+0+5222 (secure 0 0 0) 0x7fa540078f70 con 0x7fa5500ce4d0 2024-09-12T03:17:27.125 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:17:27.125 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":15,"btime":"2024-09-12T03:17:20:789023+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4592,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6836/3382368386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":3382368386},{"type":"v1","addr":"172.21.15.45:6836","nonce":3382368386}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":4595,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/2127097700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6835","nonce":2127097700},{"type":"v1","addr":"172.21.15.45:6837","nonce":2127097700}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":4815,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6834/2623734554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":2623734554},{"type":"v1","addr":"172.21.15.27:6834","nonce":2623734554}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-12T03:17:11.119994+0000","modified":"2024-09-12T03:17:13.929574+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4818},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4818":{"gid":4818,"name":"a","rank":0,"incarnation":11,"state":"up:active","state_seq":3,"addr":"172.21.15.27:6835/3533575930","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":3533575930},{"type":"v1","addr":"172.21.15.27:6835","nonce":3533575930}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4818,"qdb_cluster":[4818]},"id":2}]} 2024-09-12T03:17:27.125 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 15 2024-09-12T03:17:27.127 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.126+0000 7fa5587f8640 1 -- 172.21.15.27:0/3910499679 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fa53406c010 msgr2=0x7fa53406e4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:27.127 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.126+0000 7fa5587f8640 1 --2- 172.21.15.27:0/3910499679 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fa53406c010 0x7fa53406e4d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fa550156530 tx=0x7fa54c008040 comp rx=0 tx=0).stop 2024-09-12T03:17:27.127 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.126+0000 7fa5587f8640 1 -- 172.21.15.27:0/3910499679 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa5500ce4d0 msgr2=0x7fa5500afe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:27.127 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.126+0000 7fa5587f8640 1 --2- 172.21.15.27:0/3910499679 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa5500ce4d0 0x7fa5500afe50 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fa540001870 tx=0x7fa540001120 comp rx=0 tx=0).stop 2024-09-12T03:17:27.127 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.126+0000 7fa5587f8640 1 -- 172.21.15.27:0/3910499679 shutdown_connections 2024-09-12T03:17:27.127 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.126+0000 7fa5587f8640 1 --2- 172.21.15.27:0/3910499679 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fa53406c010 0x7fa53406e4d0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:27.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.126+0000 7fa5587f8640 1 --2- 172.21.15.27:0/3910499679 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa55015e620 0x7fa5500d2680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:27.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.126+0000 7fa5587f8640 1 --2- 172.21.15.27:0/3910499679 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa5500ce4d0 0x7fa5500afe50 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:27.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.126+0000 7fa5587f8640 1 --2- 172.21.15.27:0/3910499679 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa53c004800 0x7fa550155500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:27.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.126+0000 7fa5587f8640 1 -- 172.21.15.27:0/3910499679 >> 172.21.15.27:0/3910499679 conn(0x7fa550073800 msgr2=0x7fa550160f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:27.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.126+0000 7fa5587f8640 1 -- 172.21.15.27:0/3910499679 shutdown_connections 2024-09-12T03:17:27.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.126+0000 7fa5587f8640 1 -- 172.21.15.27:0/3910499679 wait complete. 2024-09-12T03:17:27.138 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-12T03:17:27.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.238+0000 7f343841e640 1 Processor -- start 2024-09-12T03:17:27.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.238+0000 7f343841e640 1 -- start start 2024-09-12T03:17:27.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.239+0000 7f343841e640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f343015b2b0 0x7f34300b3560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:27.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.239+0000 7f343841e640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f3430072d70 0x7f3430072430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:27.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.239+0000 7f343841e640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f34300ce410 con 0x7f343006e9b0 2024-09-12T03:17:27.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.239+0000 7f343841e640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f34300720a0 con 0x7f343015b2b0 2024-09-12T03:17:27.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.239+0000 7f343841e640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f3430072970 con 0x7f3430072d70 2024-09-12T03:17:27.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.239+0000 7f34359b9640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f3430072d70 0x7f3430072430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:27.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.239+0000 7f34359b9640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f3430072d70 0x7f3430072430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3301/0 says I am v2:172.21.15.27:45134/0 (socket says 172.21.15.27:45134) 2024-09-12T03:17:27.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.239+0000 7f34359b9640 1 -- 172.21.15.27:0/1235251492 learned_addr learned my addr 172.21.15.27:0/1235251492 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:17:27.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.239+0000 7f34361ba640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f343015b2b0 0x7f34300b3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:27.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.240+0000 7f34359b9640 1 -- 172.21.15.27:0/1235251492 >> v2:172.21.15.45:3300/0 conn(0x7f343015b2b0 msgr2=0x7f34300b3560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:27.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.240+0000 7f34359b9640 1 --2- 172.21.15.27:0/1235251492 >> v2:172.21.15.45:3300/0 conn(0x7f343015b2b0 0x7f34300b3560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:27.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.240+0000 7f34359b9640 1 -- 172.21.15.27:0/1235251492 >> v1:172.21.15.45:6789/0 conn(0x7f343006e9b0 legacy=0x7f34300cebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:27.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.240+0000 7f34359b9640 1 -- 172.21.15.27:0/1235251492 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34300732c0 con 0x7f3430072d70 2024-09-12T03:17:27.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.241+0000 7f34359b9640 1 --2- 172.21.15.27:0/1235251492 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f3430072d70 0x7f3430072430 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f3424009870 tx=0x7f342402a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=360b4498635c9851 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:27.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.241+0000 7f34351b8640 1 -- 172.21.15.27:0/1235251492 <== mon.2 v2:172.21.15.45:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3424037070 con 0x7f3430072d70 2024-09-12T03:17:27.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.241+0000 7f34351b8640 1 -- 172.21.15.27:0/1235251492 <== mon.2 v2:172.21.15.45:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3424032440 con 0x7f3430072d70 2024-09-12T03:17:27.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.241+0000 7f34351b8640 1 -- 172.21.15.27:0/1235251492 <== mon.2 v2:172.21.15.45:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3424032810 con 0x7f3430072d70 2024-09-12T03:17:27.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.241+0000 7f343841e640 1 -- 172.21.15.27:0/1235251492 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f3430072d70 msgr2=0x7f3430072430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:27.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.241+0000 7f343841e640 1 --2- 172.21.15.27:0/1235251492 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f3430072d70 0x7f3430072430 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f3424009870 tx=0x7f342402a270 comp rx=0 tx=0).stop 2024-09-12T03:17:27.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.241+0000 7f343841e640 1 -- 172.21.15.27:0/1235251492 shutdown_connections 2024-09-12T03:17:27.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.241+0000 7f343841e640 1 --2- 172.21.15.27:0/1235251492 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f3430072d70 0x7f3430072430 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:27.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.241+0000 7f343841e640 1 --2- 172.21.15.27:0/1235251492 >> v2:172.21.15.45:3300/0 conn(0x7f343015b2b0 0x7f34300b3560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:27.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.241+0000 7f343841e640 1 -- 172.21.15.27:0/1235251492 >> 172.21.15.27:0/1235251492 conn(0x7f3430062c60 msgr2=0x7f343006bb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:27.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.241+0000 7f343841e640 1 -- 172.21.15.27:0/1235251492 shutdown_connections 2024-09-12T03:17:27.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.241+0000 7f343841e640 1 -- 172.21.15.27:0/1235251492 wait complete. 2024-09-12T03:17:27.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.242+0000 7f343841e640 1 Processor -- start 2024-09-12T03:17:27.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.243+0000 7f343841e640 1 -- start start 2024-09-12T03:17:27.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.243+0000 7f343841e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f343006e9b0 0x7f3430155ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:27.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.243+0000 7f343841e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3430072d70 0x7f343006db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:27.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.243+0000 7f343841e640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f343015b2b0 0x7f343014d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:27.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.243+0000 7f343841e640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f34300ce690 con 0x7f343006e9b0 2024-09-12T03:17:27.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.243+0000 7f343841e640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f3430071dc0 con 0x7f3430072d70 2024-09-12T03:17:27.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.243+0000 7f343841e640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f343015b690 con 0x7f343015b2b0 2024-09-12T03:17:27.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.243+0000 7f34361ba640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f343006e9b0 0x7f3430155ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:27.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.243+0000 7f34361ba640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f343006e9b0 0x7f3430155ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60102/0 (socket says 172.21.15.27:60102) 2024-09-12T03:17:27.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.243+0000 7f34361ba640 1 -- 172.21.15.27:0/2203492171 learned_addr learned my addr 172.21.15.27:0/2203492171 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:17:27.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.244+0000 7f34359b9640 1 --2- 172.21.15.27:0/2203492171 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3430072d70 0x7f343006db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:27.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.244+0000 7f34369bb640 1 --2- 172.21.15.27:0/2203492171 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f343015b2b0 0x7f343014d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:27.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.244+0000 7f34361ba640 1 -- 172.21.15.27:0/2203492171 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f343015b2b0 msgr2=0x7f343014d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:27.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.244+0000 7f34361ba640 1 --2- 172.21.15.27:0/2203492171 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f343015b2b0 0x7f343014d960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:27.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.244+0000 7f34361ba640 1 -- 172.21.15.27:0/2203492171 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3430072d70 msgr2=0x7f343006db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:27.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.244+0000 7f34361ba640 1 --2- 172.21.15.27:0/2203492171 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3430072d70 0x7f343006db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:27.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.244+0000 7f34361ba640 1 -- 172.21.15.27:0/2203492171 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3430167e30 con 0x7f343006e9b0 2024-09-12T03:17:27.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.244+0000 7f34361ba640 1 --2- 172.21.15.27:0/2203492171 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f343006e9b0 0x7f3430155ea0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f3420009d10 tx=0x7f342000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:27.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.244+0000 7f342effd640 1 -- 172.21.15.27:0/2203492171 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3420018020 con 0x7f343006e9b0 2024-09-12T03:17:27.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.244+0000 7f342effd640 1 -- 172.21.15.27:0/2203492171 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3420014480 con 0x7f343006e9b0 2024-09-12T03:17:27.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.244+0000 7f343841e640 1 -- 172.21.15.27:0/2203492171 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3430168110 con 0x7f343006e9b0 2024-09-12T03:17:27.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.244+0000 7f342effd640 1 -- 172.21.15.27:0/2203492171 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3420014830 con 0x7f343006e9b0 2024-09-12T03:17:27.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.244+0000 7f343841e640 1 -- 172.21.15.27:0/2203492171 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f343006e050 con 0x7f343006e9b0 2024-09-12T03:17:27.248 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.245+0000 7f343841e640 1 -- 172.21.15.27:0/2203492171 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3430072430 con 0x7f343006e9b0 2024-09-12T03:17:27.248 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.246+0000 7f342effd640 1 -- 172.21.15.27:0/2203492171 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 7) ==== 90651+0+0 (secure 0 0 0) 0x7f3420005d10 con 0x7f343006e9b0 2024-09-12T03:17:27.248 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.246+0000 7f342effd640 1 --2- 172.21.15.27:0/2203492171 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f341c06bfe0 0x7f341c06e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:27.248 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.246+0000 7f342effd640 1 -- 172.21.15.27:0/2203492171 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f342008bb90 con 0x7f343006e9b0 2024-09-12T03:17:27.248 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.246+0000 7f34359b9640 1 --2- 172.21.15.27:0/2203492171 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f341c06bfe0 0x7f341c06e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:27.248 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.247+0000 7f34359b9640 1 --2- 172.21.15.27:0/2203492171 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f341c06bfe0 0x7f341c06e4a0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f3424034000 tx=0x7f342402e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:27.251 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.250+0000 7f342effd640 1 -- 172.21.15.27:0/2203492171 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f3420022070 con 0x7f343006e9b0 2024-09-12T03:17:27.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.462+0000 7f343841e640 1 -- 172.21.15.27:0/2203492171 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f343006e500 con 0x7f343006e9b0 2024-09-12T03:17:27.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.462+0000 7f342effd640 1 -- 172.21.15.27:0/2203492171 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) ==== 94+0+5222 (secure 0 0 0) 0x7f3420010040 con 0x7f343006e9b0 2024-09-12T03:17:27.464 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:17:27.464 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":15,"btime":"2024-09-12T03:17:20:789023+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4592,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6836/3382368386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":3382368386},{"type":"v1","addr":"172.21.15.45:6836","nonce":3382368386}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":4595,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/2127097700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6835","nonce":2127097700},{"type":"v1","addr":"172.21.15.45:6837","nonce":2127097700}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":4815,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6834/2623734554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":2623734554},{"type":"v1","addr":"172.21.15.27:6834","nonce":2623734554}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-12T03:17:11.119994+0000","modified":"2024-09-12T03:17:13.929574+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4818},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4818":{"gid":4818,"name":"a","rank":0,"incarnation":11,"state":"up:active","state_seq":3,"addr":"172.21.15.27:6835/3533575930","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":3533575930},{"type":"v1","addr":"172.21.15.27:6835","nonce":3533575930}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4818,"qdb_cluster":[4818]},"id":2}]} 2024-09-12T03:17:27.464 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 15 2024-09-12T03:17:27.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.465+0000 7f343841e640 1 -- 172.21.15.27:0/2203492171 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f341c06bfe0 msgr2=0x7f341c06e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:27.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.465+0000 7f343841e640 1 --2- 172.21.15.27:0/2203492171 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f341c06bfe0 0x7f341c06e4a0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f3424034000 tx=0x7f342402e000 comp rx=0 tx=0).stop 2024-09-12T03:17:27.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.465+0000 7f343841e640 1 -- 172.21.15.27:0/2203492171 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f343006e9b0 msgr2=0x7f3430155ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:27.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.465+0000 7f343841e640 1 --2- 172.21.15.27:0/2203492171 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f343006e9b0 0x7f3430155ea0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f3420009d10 tx=0x7f342000c490 comp rx=0 tx=0).stop 2024-09-12T03:17:27.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.465+0000 7f343841e640 1 -- 172.21.15.27:0/2203492171 shutdown_connections 2024-09-12T03:17:27.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.465+0000 7f343841e640 1 --2- 172.21.15.27:0/2203492171 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f341c06bfe0 0x7f341c06e4a0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:27.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.465+0000 7f343841e640 1 --2- 172.21.15.27:0/2203492171 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f343015b2b0 0x7f343014d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:27.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.465+0000 7f343841e640 1 --2- 172.21.15.27:0/2203492171 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3430072d70 0x7f343006db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:27.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.465+0000 7f343841e640 1 --2- 172.21.15.27:0/2203492171 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f343006e9b0 0x7f3430155ea0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:27.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.465+0000 7f343841e640 1 -- 172.21.15.27:0/2203492171 >> 172.21.15.27:0/2203492171 conn(0x7f3430062c60 msgr2=0x7f34300b02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:27.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.465+0000 7f343841e640 1 -- 172.21.15.27:0/2203492171 shutdown_connections 2024-09-12T03:17:27.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.465+0000 7f343841e640 1 -- 172.21.15.27:0/2203492171 wait complete. 2024-09-12T03:17:27.477 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-12T03:17:27.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.579+0000 7f22f70dd640 1 Processor -- start 2024-09-12T03:17:27.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.579+0000 7f22f70dd640 1 -- start start 2024-09-12T03:17:27.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.579+0000 7f22f70dd640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f22f015eb90 0x7f22f00d2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:27.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.579+0000 7f22f70dd640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f22f00d5200 0x7f22f00d1010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:27.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.579+0000 7f22f70dd640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f22f00d1550 0x7f22f00cf200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:27.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.579+0000 7f22f70dd640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f22f00b7d20 con 0x7f22f00d1550 2024-09-12T03:17:27.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.579+0000 7f22f70dd640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f22f00d42e0 con 0x7f22f00d5200 2024-09-12T03:17:27.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.579+0000 7f22f70dd640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f22f00d19e0 con 0x7f22f015eb90 2024-09-12T03:17:27.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.580+0000 7f22f567a640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f22f00d1550 0x7f22f00cf200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:27.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.580+0000 7f22f567a640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f22f00d1550 0x7f22f00cf200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60110/0 (socket says 172.21.15.27:60110) 2024-09-12T03:17:27.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.580+0000 7f22f567a640 1 -- 172.21.15.27:0/2187937213 learned_addr learned my addr 172.21.15.27:0/2187937213 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:17:27.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.580+0000 7f22effff640 1 --2- 172.21.15.27:0/2187937213 >> v2:172.21.15.45:3300/0 conn(0x7f22f00d5200 0x7f22f00d1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:27.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.580+0000 7f22f4e79640 1 --2- 172.21.15.27:0/2187937213 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f22f015eb90 0x7f22f00d2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:27.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.580+0000 7f22f567a640 1 -- 172.21.15.27:0/2187937213 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f22f015eb90 msgr2=0x7f22f00d2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:27.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.580+0000 7f22f567a640 1 --2- 172.21.15.27:0/2187937213 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f22f015eb90 0x7f22f00d2df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:27.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.580+0000 7f22f567a640 1 -- 172.21.15.27:0/2187937213 >> v2:172.21.15.45:3300/0 conn(0x7f22f00d5200 msgr2=0x7f22f00d1010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:27.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.580+0000 7f22f567a640 1 --2- 172.21.15.27:0/2187937213 >> v2:172.21.15.45:3300/0 conn(0x7f22f00d5200 0x7f22f00d1010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:27.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.580+0000 7f22f567a640 1 -- 172.21.15.27:0/2187937213 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22f00d3760 con 0x7f22f00d1550 2024-09-12T03:17:27.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.580+0000 7f22f4e79640 1 --2- 172.21.15.27:0/2187937213 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f22f015eb90 0x7f22f00d2df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:17:27.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.580+0000 7f22effff640 1 --2- 172.21.15.27:0/2187937213 >> v2:172.21.15.45:3300/0 conn(0x7f22f00d5200 0x7f22f00d1010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:17:27.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.581+0000 7f22f567a640 1 --2- 172.21.15.27:0/2187937213 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f22f00d1550 0x7f22f00cf200 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f22f0063620 tx=0x7f22e802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af767113d41cacf4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:27.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.581+0000 7f22ef7fe640 1 -- 172.21.15.27:0/2187937213 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22e8036070 con 0x7f22f00d1550 2024-09-12T03:17:27.584 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.581+0000 7f22ef7fe640 1 -- 172.21.15.27:0/2187937213 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22e8031600 con 0x7f22f00d1550 2024-09-12T03:17:27.584 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.581+0000 7f22ef7fe640 1 -- 172.21.15.27:0/2187937213 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22e80319d0 con 0x7f22f00d1550 2024-09-12T03:17:27.587 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.581+0000 7f22f70dd640 1 -- 172.21.15.27:0/2187937213 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f22f00d1550 msgr2=0x7f22f00cf200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:27.587 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.581+0000 7f22f70dd640 1 --2- 172.21.15.27:0/2187937213 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f22f00d1550 0x7f22f00cf200 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f22f0063620 tx=0x7f22e802a6a0 comp rx=0 tx=0).stop 2024-09-12T03:17:27.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.581+0000 7f22f70dd640 1 -- 172.21.15.27:0/2187937213 shutdown_connections 2024-09-12T03:17:27.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.581+0000 7f22f70dd640 1 --2- 172.21.15.27:0/2187937213 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f22f00d1550 0x7f22f00cf200 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:27.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.581+0000 7f22f70dd640 1 --2- 172.21.15.27:0/2187937213 >> v2:172.21.15.45:3300/0 conn(0x7f22f00d5200 0x7f22f00d1010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:27.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.581+0000 7f22f70dd640 1 --2- 172.21.15.27:0/2187937213 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f22f015eb90 0x7f22f00d2df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:27.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.581+0000 7f22f70dd640 1 -- 172.21.15.27:0/2187937213 >> 172.21.15.27:0/2187937213 conn(0x7f22f006dce0 msgr2=0x7f22f006e0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:27.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.581+0000 7f22f70dd640 1 -- 172.21.15.27:0/2187937213 shutdown_connections 2024-09-12T03:17:27.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.581+0000 7f22f70dd640 1 -- 172.21.15.27:0/2187937213 wait complete. 2024-09-12T03:17:27.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.583+0000 7f22f70dd640 1 Processor -- start 2024-09-12T03:17:27.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.583+0000 7f22f70dd640 1 -- start start 2024-09-12T03:17:27.589 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.583+0000 7f22f70dd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f22f00d1550 0x7f22f014dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:27.589 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.583+0000 7f22f70dd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f22f00d5200 0x7f22f01e8d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:27.589 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.583+0000 7f22f70dd640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f22f015eb90 0x7f22f01e9280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:27.589 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.584+0000 7f22f70dd640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f22f00d4000 con 0x7f22f00d1550 2024-09-12T03:17:27.589 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.584+0000 7f22f70dd640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f22f00d3d20 con 0x7f22f00d5200 2024-09-12T03:17:27.589 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.584+0000 7f22f70dd640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f22f00d3330 con 0x7f22f015eb90 2024-09-12T03:17:27.589 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.584+0000 7f22f4e79640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f22f00d1550 0x7f22f014dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:27.589 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.584+0000 7f22f4e79640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f22f00d1550 0x7f22f014dbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60124/0 (socket says 172.21.15.27:60124) 2024-09-12T03:17:27.589 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.584+0000 7f22f4e79640 1 -- 172.21.15.27:0/987218637 learned_addr learned my addr 172.21.15.27:0/987218637 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:17:27.589 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.584+0000 7f22f567a640 1 --2- 172.21.15.27:0/987218637 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f22f015eb90 0x7f22f01e9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:27.590 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.584+0000 7f22effff640 1 --2- 172.21.15.27:0/987218637 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f22f00d5200 0x7f22f01e8d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:27.590 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.584+0000 7f22f4e79640 1 -- 172.21.15.27:0/987218637 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f22f015eb90 msgr2=0x7f22f01e9280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:27.590 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.584+0000 7f22f4e79640 1 --2- 172.21.15.27:0/987218637 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f22f015eb90 0x7f22f01e9280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:27.590 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.584+0000 7f22f4e79640 1 -- 172.21.15.27:0/987218637 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f22f00d5200 msgr2=0x7f22f01e8d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:27.590 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.584+0000 7f22f4e79640 1 --2- 172.21.15.27:0/987218637 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f22f00d5200 0x7f22f01e8d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:27.590 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.584+0000 7f22f4e79640 1 -- 172.21.15.27:0/987218637 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22f00c9f70 con 0x7f22f00d1550 2024-09-12T03:17:27.590 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.584+0000 7f22effff640 1 --2- 172.21.15.27:0/987218637 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f22f00d5200 0x7f22f01e8d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:17:27.590 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.584+0000 7f22f4e79640 1 --2- 172.21.15.27:0/987218637 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f22f00d1550 0x7f22f014dbd0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f22e400c7f0 tx=0x7f22e400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:27.590 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.584+0000 7f22ed7fa640 1 -- 172.21.15.27:0/987218637 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22e4015070 con 0x7f22f00d1550 2024-09-12T03:17:27.590 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.584+0000 7f22ed7fa640 1 -- 172.21.15.27:0/987218637 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22e4007820 con 0x7f22f00d1550 2024-09-12T03:17:27.591 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.584+0000 7f22f70dd640 1 -- 172.21.15.27:0/987218637 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22f00c8ed0 con 0x7f22f00d1550 2024-09-12T03:17:27.591 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.584+0000 7f22ed7fa640 1 -- 172.21.15.27:0/987218637 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22e4007bd0 con 0x7f22f00d1550 2024-09-12T03:17:27.591 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.584+0000 7f22f70dd640 1 -- 172.21.15.27:0/987218637 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22f00c9490 con 0x7f22f00d1550 2024-09-12T03:17:27.591 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.585+0000 7f22f70dd640 1 -- 172.21.15.27:0/987218637 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22f015ef70 con 0x7f22f00d1550 2024-09-12T03:17:27.591 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.586+0000 7f22ed7fa640 1 -- 172.21.15.27:0/987218637 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 7) ==== 90651+0+0 (secure 0 0 0) 0x7f22e4018070 con 0x7f22f00d1550 2024-09-12T03:17:27.591 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.586+0000 7f22ed7fa640 1 --2- 172.21.15.27:0/987218637 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f22d406bfe0 0x7f22d406e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:27.591 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.587+0000 7f22ed7fa640 1 -- 172.21.15.27:0/987218637 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f22e408b0f0 con 0x7f22f00d1550 2024-09-12T03:17:27.591 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.587+0000 7f22effff640 1 --2- 172.21.15.27:0/987218637 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f22d406bfe0 0x7f22d406e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:27.592 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.587+0000 7f22effff640 1 --2- 172.21.15.27:0/987218637 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f22d406bfe0 0x7f22d406e4a0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f22f00ceaf0 tx=0x7f22dc008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:27.592 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.591+0000 7f22ed7fa640 1 -- 172.21.15.27:0/987218637 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f22e4059710 con 0x7f22f00d1550 2024-09-12T03:17:27.768 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.767+0000 7f22f70dd640 1 -- 172.21.15.27:0/987218637 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f22f015e340 con 0x7f22f00d1550 2024-09-12T03:17:27.769 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.768+0000 7f22ed7fa640 1 -- 172.21.15.27:0/987218637 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) ==== 74+0+16681 (secure 0 0 0) 0x7f22e4059010 con 0x7f22f00d1550 2024-09-12T03:17:27.770 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:17:27.770 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":32,"fsid":"d2df504e-2647-4385-a5db-9fdf69f487a0","created":"2024-09-12T03:16:07.778937+0000","modified":"2024-09-12T03:17:12.768450+0000","last_up_change":"2024-09-12T03:16:19.314778+0000","last_in_change":"2024-09-12T03:16:09.893086+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T03:16:20.097646+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-09-12T03:17:09.018856+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-09-12T03:17:10.105881+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1851116c-1fb3-4dc9-9772-f90e2ef4c355","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6817","nonce":3002321488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6819","nonce":3002321488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6823","nonce":3002321488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6821","nonce":3002321488}]},"public_addr":"172.21.15.27:6817/3002321488","cluster_addr":"172.21.15.27:6819/3002321488","heartbeat_back_addr":"172.21.15.27:6823/3002321488","heartbeat_front_addr":"172.21.15.27:6821/3002321488","state":["exists","up"]},{"osd":1,"uuid":"f38ae17d-73eb-4204-b99d-975fa806fb87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6809","nonce":1613102886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6811","nonce":1613102886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6815","nonce":1613102886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6813","nonce":1613102886}]},"public_addr":"172.21.15.27:6809/1613102886","cluster_addr":"172.21.15.27:6811/1613102886","heartbeat_back_addr":"172.21.15.27:6815/1613102886","heartbeat_front_addr":"172.21.15.27:6813/1613102886","state":["exists","up"]},{"osd":2,"uuid":"fb12dcc8-bdd0-42b8-b46d-8b10106224bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6825","nonce":3534723355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6827","nonce":3534723355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6831","nonce":3534723355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6829","nonce":3534723355}]},"public_addr":"172.21.15.27:6825/3534723355","cluster_addr":"172.21.15.27:6827/3534723355","heartbeat_back_addr":"172.21.15.27:6831/3534723355","heartbeat_front_addr":"172.21.15.27:6829/3534723355","state":["exists","up"]},{"osd":3,"uuid":"d74d3f49-9c24-4b9f-98e2-3ecd7b5ef3b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6801","nonce":80661598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6803","nonce":80661598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6807","nonce":80661598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6805","nonce":80661598}]},"public_addr":"172.21.15.27:6801/80661598","cluster_addr":"172.21.15.27:6803/80661598","heartbeat_back_addr":"172.21.15.27:6807/80661598","heartbeat_front_addr":"172.21.15.27:6805/80661598","state":["exists","up"]},{"osd":4,"uuid":"2bf123f2-1b0d-41c4-aa3b-805bab267e59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6817","nonce":3943189827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6819","nonce":3943189827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6823","nonce":3943189827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6821","nonce":3943189827}]},"public_addr":"172.21.15.45:6817/3943189827","cluster_addr":"172.21.15.45:6819/3943189827","heartbeat_back_addr":"172.21.15.45:6823/3943189827","heartbeat_front_addr":"172.21.15.45:6821/3943189827","state":["exists","up"]},{"osd":5,"uuid":"c49ba5da-91c6-4a86-a97b-4aba89c253ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6809","nonce":3620760330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6811","nonce":3620760330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6815","nonce":3620760330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6813","nonce":3620760330}]},"public_addr":"172.21.15.45:6809/3620760330","cluster_addr":"172.21.15.45:6811/3620760330","heartbeat_back_addr":"172.21.15.45:6815/3620760330","heartbeat_front_addr":"172.21.15.45:6813/3620760330","state":["exists","up"]},{"osd":6,"uuid":"b0123475-89f7-4a34-9e33-d797b99dd9dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6801","nonce":1989083510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6803","nonce":1989083510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6807","nonce":1989083510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6805","nonce":1989083510}]},"public_addr":"172.21.15.45:6801/1989083510","cluster_addr":"172.21.15.45:6803/1989083510","heartbeat_back_addr":"172.21.15.45:6807/1989083510","heartbeat_front_addr":"172.21.15.45:6805/1989083510","state":["exists","up"]},{"osd":7,"uuid":"4dad773d-b7d8-46fc-8a10-a972aad27b29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6825","nonce":1513300294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6827","nonce":1513300294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6831","nonce":1513300294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6829","nonce":1513300294}]},"public_addr":"172.21.15.45:6825/1513300294","cluster_addr":"172.21.15.45:6827/1513300294","heartbeat_back_addr":"172.21.15.45:6831/1513300294","heartbeat_front_addr":"172.21.15.45:6829/1513300294","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.881855+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:17.133252+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.735339+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.500742+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.477571+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:17.167667+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.470291+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.873743+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T03:17:27.773 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.771+0000 7f22f70dd640 1 -- 172.21.15.27:0/987218637 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f22d406bfe0 msgr2=0x7f22d406e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:27.773 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.771+0000 7f22f70dd640 1 --2- 172.21.15.27:0/987218637 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f22d406bfe0 0x7f22d406e4a0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f22f00ceaf0 tx=0x7f22dc008040 comp rx=0 tx=0).stop 2024-09-12T03:17:27.773 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.771+0000 7f22f70dd640 1 -- 172.21.15.27:0/987218637 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f22f00d1550 msgr2=0x7f22f014dbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:27.773 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.771+0000 7f22f70dd640 1 --2- 172.21.15.27:0/987218637 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f22f00d1550 0x7f22f014dbd0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f22e400c7f0 tx=0x7f22e400ccb0 comp rx=0 tx=0).stop 2024-09-12T03:17:27.773 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.771+0000 7f22f70dd640 1 -- 172.21.15.27:0/987218637 shutdown_connections 2024-09-12T03:17:27.773 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.771+0000 7f22f70dd640 1 --2- 172.21.15.27:0/987218637 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f22d406bfe0 0x7f22d406e4a0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:27.773 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.771+0000 7f22f70dd640 1 --2- 172.21.15.27:0/987218637 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f22f015eb90 0x7f22f01e9280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:27.773 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.771+0000 7f22f70dd640 1 --2- 172.21.15.27:0/987218637 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f22f00d5200 0x7f22f01e8d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:27.773 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.772+0000 7f22f70dd640 1 --2- 172.21.15.27:0/987218637 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f22f00d1550 0x7f22f014dbd0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:27.773 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.772+0000 7f22f70dd640 1 -- 172.21.15.27:0/987218637 >> 172.21.15.27:0/987218637 conn(0x7f22f006dce0 msgr2=0x7f22f0161060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:27.774 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.772+0000 7f22f70dd640 1 -- 172.21.15.27:0/987218637 shutdown_connections 2024-09-12T03:17:27.774 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.772+0000 7f22f70dd640 1 -- 172.21.15.27:0/987218637 wait complete. 2024-09-12T03:17:27.786 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-12T03:17:27.786 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-12T03:17:27.888 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.887+0000 7f251e8e6640 1 Processor -- start 2024-09-12T03:17:27.888 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.887+0000 7f251e8e6640 1 -- start start 2024-09-12T03:17:27.890 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.888+0000 7f251e8e6640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f25180b1ca0 0x7f251815f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:27.890 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.888+0000 7f251e8e6640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f251806f360 0x7f25180cef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:27.890 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.888+0000 7f251e8e6640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f251815fd70 con 0x7f25180cf490 2024-09-12T03:17:27.890 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.888+0000 7f251e8e6640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f25180cf900 con 0x7f251806f360 2024-09-12T03:17:27.890 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.888+0000 7f251e8e6640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f25180d22f0 con 0x7f25180b1ca0 2024-09-12T03:17:27.890 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.888+0000 7f25177fe640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f251806f360 0x7f25180cef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:27.890 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.888+0000 7f251ce83640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f25180cf490 0x7f25180d2780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:51406/0 (socket says 172.21.15.27:51406) 2024-09-12T03:17:27.890 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.888+0000 7f251ce83640 1 -- 172.21.15.27:0/2990956142 learned_addr learned my addr 172.21.15.27:0/2990956142 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:17:27.890 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.888+0000 7f2517fff640 1 --2- 172.21.15.27:0/2990956142 >> v2:172.21.15.45:3300/0 conn(0x7f25180b1ca0 0x7f251815f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:27.890 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.889+0000 7f25177fe640 1 -- 172.21.15.27:0/2990956142 >> v2:172.21.15.45:3300/0 conn(0x7f25180b1ca0 msgr2=0x7f251815f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:27.890 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.889+0000 7f25177fe640 1 --2- 172.21.15.27:0/2990956142 >> v2:172.21.15.45:3300/0 conn(0x7f25180b1ca0 0x7f251815f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:27.890 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.889+0000 7f25177fe640 1 -- 172.21.15.27:0/2990956142 >> v1:172.21.15.27:6789/0 conn(0x7f25180cf490 legacy=0x7f25180d2780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:27.890 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.889+0000 7f25177fe640 1 -- 172.21.15.27:0/2990956142 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f251806cc40 con 0x7f251806f360 2024-09-12T03:17:27.891 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.889+0000 7f25177fe640 1 --2- 172.21.15.27:0/2990956142 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f251806f360 0x7f25180cef50 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f2518073ed0 tx=0x7f250c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=850a9b63e2972945 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:27.892 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.889+0000 7f2516ffd640 1 -- 172.21.15.27:0/2990956142 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f250c037070 con 0x7f251806f360 2024-09-12T03:17:27.892 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.890+0000 7f2516ffd640 1 -- 172.21.15.27:0/2990956142 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f250c032440 con 0x7f251806f360 2024-09-12T03:17:27.892 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.890+0000 7f2516ffd640 1 -- 172.21.15.27:0/2990956142 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f250c032810 con 0x7f251806f360 2024-09-12T03:17:27.892 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.890+0000 7f251e8e6640 1 -- 172.21.15.27:0/2990956142 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f251806f360 msgr2=0x7f25180cef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:27.892 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.890+0000 7f251e8e6640 1 --2- 172.21.15.27:0/2990956142 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f251806f360 0x7f25180cef50 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f2518073ed0 tx=0x7f250c02a1a0 comp rx=0 tx=0).stop 2024-09-12T03:17:27.892 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.890+0000 7f251e8e6640 1 -- 172.21.15.27:0/2990956142 shutdown_connections 2024-09-12T03:17:27.892 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.890+0000 7f251e8e6640 1 --2- 172.21.15.27:0/2990956142 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f251806f360 0x7f25180cef50 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:27.892 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.890+0000 7f251e8e6640 1 --2- 172.21.15.27:0/2990956142 >> v2:172.21.15.45:3300/0 conn(0x7f25180b1ca0 0x7f251815f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:27.892 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.890+0000 7f251e8e6640 1 -- 172.21.15.27:0/2990956142 >> 172.21.15.27:0/2990956142 conn(0x7f2518073800 msgr2=0x7f251809ba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:27.892 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.890+0000 7f251e8e6640 1 -- 172.21.15.27:0/2990956142 shutdown_connections 2024-09-12T03:17:27.892 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.890+0000 7f251e8e6640 1 -- 172.21.15.27:0/2990956142 wait complete. 2024-09-12T03:17:27.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.891+0000 7f251e8e6640 1 Processor -- start 2024-09-12T03:17:27.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.892+0000 7f251e8e6640 1 -- start start 2024-09-12T03:17:27.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.892+0000 7f251e8e6640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f251806f360 0x7f25180cb850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:27.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.892+0000 7f251e8e6640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f25180b1ca0 0x7f2518161100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:27.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.892+0000 7f251e8e6640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f25180cf490 0x7f2518161640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:27.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.892+0000 7f251e8e6640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f25180d1f80 con 0x7f251806f360 2024-09-12T03:17:27.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.892+0000 7f251e8e6640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f25180d1ce0 con 0x7f25180cf490 2024-09-12T03:17:27.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.892+0000 7f251e8e6640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f251806d1c0 con 0x7f25180b1ca0 2024-09-12T03:17:27.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.892+0000 7f2517fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f251806f360 0x7f25180cb850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:27.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.892+0000 7f2517fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f251806f360 0x7f25180cb850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60138/0 (socket says 172.21.15.27:60138) 2024-09-12T03:17:27.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.892+0000 7f2517fff640 1 -- 172.21.15.27:0/2271793527 learned_addr learned my addr 172.21.15.27:0/2271793527 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:17:27.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.893+0000 7f25177fe640 1 --2- 172.21.15.27:0/2271793527 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f25180b1ca0 0x7f2518161100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:27.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.893+0000 7f251ce83640 1 --2- 172.21.15.27:0/2271793527 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f25180cf490 0x7f2518161640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:27.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.893+0000 7f2517fff640 1 -- 172.21.15.27:0/2271793527 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f25180b1ca0 msgr2=0x7f2518161100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:27.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.893+0000 7f2517fff640 1 --2- 172.21.15.27:0/2271793527 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f25180b1ca0 0x7f2518161100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:27.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.893+0000 7f2517fff640 1 -- 172.21.15.27:0/2271793527 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f25180cf490 msgr2=0x7f2518161640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:27.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.893+0000 7f2517fff640 1 --2- 172.21.15.27:0/2271793527 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f25180cf490 0x7f2518161640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:27.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.893+0000 7f2517fff640 1 -- 172.21.15.27:0/2271793527 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25180cc230 con 0x7f251806f360 2024-09-12T03:17:27.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.893+0000 7f251ce83640 1 --2- 172.21.15.27:0/2271793527 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f25180cf490 0x7f2518161640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:17:27.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.893+0000 7f25177fe640 1 --2- 172.21.15.27:0/2271793527 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f25180b1ca0 0x7f2518161100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:17:27.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.893+0000 7f2517fff640 1 --2- 172.21.15.27:0/2271793527 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f251806f360 0x7f25180cb850 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f250800d830 tx=0x7f250800dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:27.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.893+0000 7f2514ff9640 1 -- 172.21.15.27:0/2271793527 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f250800b6c0 con 0x7f251806f360 2024-09-12T03:17:27.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.893+0000 7f2514ff9640 1 -- 172.21.15.27:0/2271793527 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2508019070 con 0x7f251806f360 2024-09-12T03:17:27.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.893+0000 7f251e8e6640 1 -- 172.21.15.27:0/2271793527 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25180cc510 con 0x7f251806f360 2024-09-12T03:17:27.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.893+0000 7f2514ff9640 1 -- 172.21.15.27:0/2271793527 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f250800ba80 con 0x7f251806f360 2024-09-12T03:17:27.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.893+0000 7f251e8e6640 1 -- 172.21.15.27:0/2271793527 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25180ccad0 con 0x7f251806f360 2024-09-12T03:17:27.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.894+0000 7f251e8e6640 1 -- 172.21.15.27:0/2271793527 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25180cfc40 con 0x7f251806f360 2024-09-12T03:17:27.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.895+0000 7f2514ff9640 1 -- 172.21.15.27:0/2271793527 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 7) ==== 90651+0+0 (secure 0 0 0) 0x7f250800bc20 con 0x7f251806f360 2024-09-12T03:17:27.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.895+0000 7f2514ff9640 1 --2- 172.21.15.27:0/2271793527 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f250406bfe0 0x7f250406e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:27.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.895+0000 7f2514ff9640 1 -- 172.21.15.27:0/2271793527 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f250808b5b0 con 0x7f251806f360 2024-09-12T03:17:27.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.896+0000 7f25177fe640 1 --2- 172.21.15.27:0/2271793527 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f250406bfe0 0x7f250406e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:27.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.896+0000 7f25177fe640 1 --2- 172.21.15.27:0/2271793527 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f250406bfe0 0x7f250406e4a0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f250c034000 tx=0x7f250c02e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:27.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:27.900+0000 7f2514ff9640 1 -- 172.21.15.27:0/2271793527 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f2508059d70 con 0x7f251806f360 2024-09-12T03:17:28.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.111+0000 7f251e8e6640 1 -- 172.21.15.27:0/2271793527 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f251806e8c0 con 0x7f251806f360 2024-09-12T03:17:28.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.111+0000 7f2514ff9640 1 -- 172.21.15.27:0/2271793527 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) ==== 94+0+5222 (secure 0 0 0) 0x7f2508059670 con 0x7f251806f360 2024-09-12T03:17:28.113 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:17:28.113 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":15,"btime":"2024-09-12T03:17:20:789023+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4592,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6836/3382368386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":3382368386},{"type":"v1","addr":"172.21.15.45:6836","nonce":3382368386}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":4595,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/2127097700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6835","nonce":2127097700},{"type":"v1","addr":"172.21.15.45:6837","nonce":2127097700}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":4815,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6834/2623734554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":2623734554},{"type":"v1","addr":"172.21.15.27:6834","nonce":2623734554}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-12T03:17:11.119994+0000","modified":"2024-09-12T03:17:13.929574+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4818},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4818":{"gid":4818,"name":"a","rank":0,"incarnation":11,"state":"up:active","state_seq":3,"addr":"172.21.15.27:6835/3533575930","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":3533575930},{"type":"v1","addr":"172.21.15.27:6835","nonce":3533575930}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4818,"qdb_cluster":[4818]},"id":2}]} 2024-09-12T03:17:28.113 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 15 2024-09-12T03:17:28.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.113+0000 7f251e8e6640 1 -- 172.21.15.27:0/2271793527 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f250406bfe0 msgr2=0x7f250406e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:28.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.113+0000 7f251e8e6640 1 --2- 172.21.15.27:0/2271793527 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f250406bfe0 0x7f250406e4a0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f250c034000 tx=0x7f250c02e000 comp rx=0 tx=0).stop 2024-09-12T03:17:28.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.113+0000 7f251e8e6640 1 -- 172.21.15.27:0/2271793527 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f251806f360 msgr2=0x7f25180cb850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:28.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.113+0000 7f251e8e6640 1 --2- 172.21.15.27:0/2271793527 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f251806f360 0x7f25180cb850 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f250800d830 tx=0x7f250800dcf0 comp rx=0 tx=0).stop 2024-09-12T03:17:28.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.113+0000 7f251e8e6640 1 -- 172.21.15.27:0/2271793527 shutdown_connections 2024-09-12T03:17:28.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.113+0000 7f251e8e6640 1 --2- 172.21.15.27:0/2271793527 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f250406bfe0 0x7f250406e4a0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:28.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.113+0000 7f251e8e6640 1 --2- 172.21.15.27:0/2271793527 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f25180cf490 0x7f2518161640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:28.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.113+0000 7f251e8e6640 1 --2- 172.21.15.27:0/2271793527 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f25180b1ca0 0x7f2518161100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:28.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.113+0000 7f251e8e6640 1 --2- 172.21.15.27:0/2271793527 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f251806f360 0x7f25180cb850 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:28.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.113+0000 7f251e8e6640 1 -- 172.21.15.27:0/2271793527 >> 172.21.15.27:0/2271793527 conn(0x7f2518073800 msgr2=0x7f251815ffc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:28.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.113+0000 7f251e8e6640 1 -- 172.21.15.27:0/2271793527 shutdown_connections 2024-09-12T03:17:28.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.113+0000 7f251e8e6640 1 -- 172.21.15.27:0/2271793527 wait complete. 2024-09-12T03:17:28.126 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-12T03:17:28.228 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.227+0000 7f420b2e6640 1 Processor -- start 2024-09-12T03:17:28.228 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.227+0000 7f420b2e6640 1 -- start start 2024-09-12T03:17:28.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.228+0000 7f420b2e6640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f42040cda40 0x7f420415ca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:28.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.228+0000 7f420b2e6640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f4204146010 0x7f42040b4040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:28.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.228+0000 7f420b2e6640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f42040b4d70 con 0x7f4204145c30 2024-09-12T03:17:28.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.228+0000 7f420b2e6640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f42040b5050 con 0x7f4204146010 2024-09-12T03:17:28.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.228+0000 7f420b2e6640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f420406d140 con 0x7f42040cda40 2024-09-12T03:17:28.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.228+0000 7f4209082640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f4204145c30 0x7f42040cd330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:51416/0 (socket says 172.21.15.27:51416) 2024-09-12T03:17:28.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.228+0000 7f4209082640 1 -- 172.21.15.27:0/2136837705 learned_addr learned my addr 172.21.15.27:0/2136837705 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:17:28.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.228+0000 7f4209883640 1 --2- 172.21.15.27:0/2136837705 >> v2:172.21.15.27:3300/0 conn(0x7f4204146010 0x7f42040b4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:28.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.228+0000 7f41fbfff640 1 -- 172.21.15.27:0/2136837705 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2878597203 0 0) 0x7f42040b4d70 con 0x7f4204145c30 2024-09-12T03:17:28.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.228+0000 7f4208881640 1 --2- 172.21.15.27:0/2136837705 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f42040cda40 0x7f420415ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:28.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.228+0000 7f41fbfff640 1 -- 172.21.15.27:0/2136837705 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f41f0003520 con 0x7f4204145c30 2024-09-12T03:17:28.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.228+0000 7f4209883640 1 -- 172.21.15.27:0/2136837705 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f42040cda40 msgr2=0x7f420415ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:28.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.228+0000 7f4209883640 1 --2- 172.21.15.27:0/2136837705 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f42040cda40 0x7f420415ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:28.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.228+0000 7f4209883640 1 -- 172.21.15.27:0/2136837705 >> v1:172.21.15.27:6789/0 conn(0x7f4204145c30 legacy=0x7f42040cd330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:28.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.228+0000 7f4209883640 1 -- 172.21.15.27:0/2136837705 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f420406d420 con 0x7f4204146010 2024-09-12T03:17:28.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.229+0000 7f4209883640 1 --2- 172.21.15.27:0/2136837705 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4204146010 0x7f42040b4040 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f41fc009dc0 tx=0x7f41fc02a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27b68f5ae621ad57 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:28.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.229+0000 7f41fbfff640 1 -- 172.21.15.27:0/2136837705 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41fc007740 con 0x7f4204146010 2024-09-12T03:17:28.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.229+0000 7f41fbfff640 1 -- 172.21.15.27:0/2136837705 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41fc0078e0 con 0x7f4204146010 2024-09-12T03:17:28.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.229+0000 7f41fbfff640 1 -- 172.21.15.27:0/2136837705 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41fc007cb0 con 0x7f4204146010 2024-09-12T03:17:28.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.230+0000 7f420b2e6640 1 -- 172.21.15.27:0/2136837705 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4204146010 msgr2=0x7f42040b4040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:28.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.230+0000 7f420b2e6640 1 --2- 172.21.15.27:0/2136837705 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4204146010 0x7f42040b4040 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f41fc009dc0 tx=0x7f41fc02a5f0 comp rx=0 tx=0).stop 2024-09-12T03:17:28.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.230+0000 7f420b2e6640 1 -- 172.21.15.27:0/2136837705 shutdown_connections 2024-09-12T03:17:28.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.230+0000 7f420b2e6640 1 --2- 172.21.15.27:0/2136837705 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4204146010 0x7f42040b4040 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:28.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.230+0000 7f420b2e6640 1 --2- 172.21.15.27:0/2136837705 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f42040cda40 0x7f420415ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:28.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.230+0000 7f420b2e6640 1 -- 172.21.15.27:0/2136837705 >> 172.21.15.27:0/2136837705 conn(0x7f4204062c60 msgr2=0x7f420406bb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:28.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.230+0000 7f420b2e6640 1 -- 172.21.15.27:0/2136837705 shutdown_connections 2024-09-12T03:17:28.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.230+0000 7f420b2e6640 1 -- 172.21.15.27:0/2136837705 wait complete. 2024-09-12T03:17:28.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.231+0000 7f420b2e6640 1 Processor -- start 2024-09-12T03:17:28.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.231+0000 7f420b2e6640 1 -- start start 2024-09-12T03:17:28.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.232+0000 7f420b2e6640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f42040cda40 0x7f42040cb010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:28.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.232+0000 7f420b2e6640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4204145c30 0x7f4204140f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:28.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.232+0000 7f420b2e6640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4204146010 0x7f42040cd310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:28.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.232+0000 7f420b2e6640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f42040b4700 con 0x7f4204146010 2024-09-12T03:17:28.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.232+0000 7f420b2e6640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f42040b4580 con 0x7f4204145c30 2024-09-12T03:17:28.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.232+0000 7f420b2e6640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f42040b4880 con 0x7f42040cda40 2024-09-12T03:17:28.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.232+0000 7f4209883640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4204146010 0x7f42040cd310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:28.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.232+0000 7f4209883640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4204146010 0x7f42040cd310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60156/0 (socket says 172.21.15.27:60156) 2024-09-12T03:17:28.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.232+0000 7f4209883640 1 -- 172.21.15.27:0/2166740448 learned_addr learned my addr 172.21.15.27:0/2166740448 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:17:28.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.232+0000 7f4209082640 1 --2- 172.21.15.27:0/2166740448 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f42040cda40 0x7f42040cb010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:28.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.232+0000 7f4208881640 1 --2- 172.21.15.27:0/2166740448 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4204145c30 0x7f4204140f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:28.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.232+0000 7f4209883640 1 -- 172.21.15.27:0/2166740448 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f42040cda40 msgr2=0x7f42040cb010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:28.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.232+0000 7f4209883640 1 --2- 172.21.15.27:0/2166740448 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f42040cda40 0x7f42040cb010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:28.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.232+0000 7f4209883640 1 -- 172.21.15.27:0/2166740448 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4204145c30 msgr2=0x7f4204140f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:28.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.232+0000 7f4209883640 1 --2- 172.21.15.27:0/2166740448 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4204145c30 0x7f4204140f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:28.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.232+0000 7f4209883640 1 -- 172.21.15.27:0/2166740448 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42040cb710 con 0x7f4204146010 2024-09-12T03:17:28.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.232+0000 7f4208881640 1 --2- 172.21.15.27:0/2166740448 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4204145c30 0x7f4204140f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:17:28.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.232+0000 7f4209883640 1 --2- 172.21.15.27:0/2166740448 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4204146010 0x7f42040cd310 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f41fc009ee0 tx=0x7f41fc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:28.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.233+0000 7f41f9ffb640 1 -- 172.21.15.27:0/2166740448 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41fc03f070 con 0x7f4204146010 2024-09-12T03:17:28.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.233+0000 7f41f9ffb640 1 -- 172.21.15.27:0/2166740448 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41fc031bc0 con 0x7f4204146010 2024-09-12T03:17:28.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.233+0000 7f420b2e6640 1 -- 172.21.15.27:0/2166740448 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42040cb9f0 con 0x7f4204146010 2024-09-12T03:17:28.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.233+0000 7f41f9ffb640 1 -- 172.21.15.27:0/2166740448 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41fc009450 con 0x7f4204146010 2024-09-12T03:17:28.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.233+0000 7f420b2e6640 1 -- 172.21.15.27:0/2166740448 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42040c9bc0 con 0x7f4204146010 2024-09-12T03:17:28.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.234+0000 7f420b2e6640 1 -- 172.21.15.27:0/2166740448 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f420415ca00 con 0x7f4204146010 2024-09-12T03:17:28.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.234+0000 7f41f9ffb640 1 -- 172.21.15.27:0/2166740448 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 7) ==== 90651+0+0 (secure 0 0 0) 0x7f41fc044460 con 0x7f4204146010 2024-09-12T03:17:28.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.235+0000 7f41f9ffb640 1 --2- 172.21.15.27:0/2166740448 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f41e806bfe0 0x7f41e806e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:28.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.235+0000 7f41f9ffb640 1 -- 172.21.15.27:0/2166740448 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f41fc0ac8c0 con 0x7f4204146010 2024-09-12T03:17:28.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.235+0000 7f4209082640 1 --2- 172.21.15.27:0/2166740448 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f41e806bfe0 0x7f41e806e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:28.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.235+0000 7f4209082640 1 --2- 172.21.15.27:0/2166740448 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f41e806bfe0 0x7f41e806e4a0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f4200005880 tx=0x7f4200009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:28.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.239+0000 7f41f9ffb640 1 -- 172.21.15.27:0/2166740448 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f41fc07aee0 con 0x7f4204146010 2024-09-12T03:17:28.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.449+0000 7f420b2e6640 1 -- 172.21.15.27:0/2166740448 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4204144ab0 con 0x7f4204146010 2024-09-12T03:17:28.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.450+0000 7f41f9ffb640 1 -- 172.21.15.27:0/2166740448 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) ==== 94+0+5222 (secure 0 0 0) 0x7f41fc07a7e0 con 0x7f4204146010 2024-09-12T03:17:28.451 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:17:28.451 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":15,"btime":"2024-09-12T03:17:20:789023+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4592,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6836/3382368386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":3382368386},{"type":"v1","addr":"172.21.15.45:6836","nonce":3382368386}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":4595,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/2127097700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6835","nonce":2127097700},{"type":"v1","addr":"172.21.15.45:6837","nonce":2127097700}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":4815,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6834/2623734554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":2623734554},{"type":"v1","addr":"172.21.15.27:6834","nonce":2623734554}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-12T03:17:11.119994+0000","modified":"2024-09-12T03:17:13.929574+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4818},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4818":{"gid":4818,"name":"a","rank":0,"incarnation":11,"state":"up:active","state_seq":3,"addr":"172.21.15.27:6835/3533575930","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":3533575930},{"type":"v1","addr":"172.21.15.27:6835","nonce":3533575930}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4818,"qdb_cluster":[4818]},"id":2}]} 2024-09-12T03:17:28.451 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 15 2024-09-12T03:17:28.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.452+0000 7f420b2e6640 1 -- 172.21.15.27:0/2166740448 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f41e806bfe0 msgr2=0x7f41e806e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:28.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.452+0000 7f420b2e6640 1 --2- 172.21.15.27:0/2166740448 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f41e806bfe0 0x7f41e806e4a0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f4200005880 tx=0x7f4200009040 comp rx=0 tx=0).stop 2024-09-12T03:17:28.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.452+0000 7f420b2e6640 1 -- 172.21.15.27:0/2166740448 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4204146010 msgr2=0x7f42040cd310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:28.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.452+0000 7f420b2e6640 1 --2- 172.21.15.27:0/2166740448 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4204146010 0x7f42040cd310 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f41fc009ee0 tx=0x7f41fc004a40 comp rx=0 tx=0).stop 2024-09-12T03:17:28.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.452+0000 7f420b2e6640 1 -- 172.21.15.27:0/2166740448 shutdown_connections 2024-09-12T03:17:28.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.452+0000 7f420b2e6640 1 --2- 172.21.15.27:0/2166740448 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f41e806bfe0 0x7f41e806e4a0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:28.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.452+0000 7f420b2e6640 1 --2- 172.21.15.27:0/2166740448 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4204146010 0x7f42040cd310 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:28.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.452+0000 7f420b2e6640 1 --2- 172.21.15.27:0/2166740448 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4204145c30 0x7f4204140f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:28.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.452+0000 7f420b2e6640 1 --2- 172.21.15.27:0/2166740448 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f42040cda40 0x7f42040cb010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:28.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.452+0000 7f420b2e6640 1 -- 172.21.15.27:0/2166740448 >> 172.21.15.27:0/2166740448 conn(0x7f4204062c60 msgr2=0x7f4204141b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:28.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.452+0000 7f420b2e6640 1 -- 172.21.15.27:0/2166740448 shutdown_connections 2024-09-12T03:17:28.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.452+0000 7f420b2e6640 1 -- 172.21.15.27:0/2166740448 wait complete. 2024-09-12T03:17:28.464 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-12T03:17:28.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.565+0000 7ff13985a640 1 Processor -- start 2024-09-12T03:17:28.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.565+0000 7ff13985a640 1 -- start start 2024-09-12T03:17:28.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.566+0000 7ff13985a640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7ff1340cef50 0x7ff1340cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:28.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.566+0000 7ff13985a640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ff134070e80 0x7ff1340d0440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:28.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.566+0000 7ff13985a640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff13406fa00 con 0x7ff1340d0980 2024-09-12T03:17:28.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.566+0000 7ff13985a640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7ff13415ca90 con 0x7ff1340cef50 2024-09-12T03:17:28.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.566+0000 7ff13985a640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7ff1340cf870 con 0x7ff134070e80 2024-09-12T03:17:28.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.566+0000 7ff132ffd640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7ff1340cef50 0x7ff1340cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:28.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.566+0000 7ff132ffd640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7ff1340cef50 0x7ff1340cf330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60166/0 (socket says 172.21.15.27:60166) 2024-09-12T03:17:28.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.566+0000 7ff132ffd640 1 -- 172.21.15.27:0/937756721 learned_addr learned my addr 172.21.15.27:0/937756721 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:17:28.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.566+0000 7ff1327fc640 1 --2- 172.21.15.27:0/937756721 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ff134070e80 0x7ff1340d0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:28.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.566+0000 7ff132ffd640 1 -- 172.21.15.27:0/937756721 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ff134070e80 msgr2=0x7ff1340d0440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:28.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.566+0000 7ff132ffd640 1 --2- 172.21.15.27:0/937756721 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ff134070e80 0x7ff1340d0440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:28.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.566+0000 7ff132ffd640 1 -- 172.21.15.27:0/937756721 >> v1:172.21.15.45:6789/0 conn(0x7ff1340d0980 legacy=0x7ff1340d0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:28.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.566+0000 7ff132ffd640 1 -- 172.21.15.27:0/937756721 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff13406f720 con 0x7ff1340cef50 2024-09-12T03:17:28.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.567+0000 7ff132ffd640 1 --2- 172.21.15.27:0/937756721 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff1340cef50 0x7ff1340cf330 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7ff124009870 tx=0x7ff12402a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b1cf8ede9c27ebb server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:28.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.567+0000 7ff131ffb640 1 -- 172.21.15.27:0/937756721 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff124037070 con 0x7ff1340cef50 2024-09-12T03:17:28.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.567+0000 7ff131ffb640 1 -- 172.21.15.27:0/937756721 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff12402ae10 con 0x7ff1340cef50 2024-09-12T03:17:28.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.567+0000 7ff131ffb640 1 -- 172.21.15.27:0/937756721 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff124032660 con 0x7ff1340cef50 2024-09-12T03:17:28.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.568+0000 7ff13985a640 1 -- 172.21.15.27:0/937756721 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff1340cef50 msgr2=0x7ff1340cf330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:28.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.568+0000 7ff13985a640 1 --2- 172.21.15.27:0/937756721 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff1340cef50 0x7ff1340cf330 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7ff124009870 tx=0x7ff12402a100 comp rx=0 tx=0).stop 2024-09-12T03:17:28.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.568+0000 7ff13985a640 1 -- 172.21.15.27:0/937756721 shutdown_connections 2024-09-12T03:17:28.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.568+0000 7ff13985a640 1 --2- 172.21.15.27:0/937756721 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ff134070e80 0x7ff1340d0440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:28.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.568+0000 7ff13985a640 1 --2- 172.21.15.27:0/937756721 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff1340cef50 0x7ff1340cf330 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:28.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.568+0000 7ff13985a640 1 -- 172.21.15.27:0/937756721 >> 172.21.15.27:0/937756721 conn(0x7ff134073800 msgr2=0x7ff13409ba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:28.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.568+0000 7ff13985a640 1 -- 172.21.15.27:0/937756721 shutdown_connections 2024-09-12T03:17:28.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.568+0000 7ff13985a640 1 -- 172.21.15.27:0/937756721 wait complete. 2024-09-12T03:17:28.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.569+0000 7ff13985a640 1 Processor -- start 2024-09-12T03:17:28.574 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.569+0000 7ff13985a640 1 -- start start 2024-09-12T03:17:28.574 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.570+0000 7ff13985a640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ff134070e80 0x7ff13415f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:28.574 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.570+0000 7ff13985a640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff1340cef50 0x7ff1340cd940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:28.574 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.570+0000 7ff13985a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff1340d0980 0x7ff1340cbe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:28.574 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.570+0000 7ff13985a640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7ff13406f360 con 0x7ff1340cef50 2024-09-12T03:17:28.574 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.570+0000 7ff13985a640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ff1340d2d10 con 0x7ff1340d0980 2024-09-12T03:17:28.574 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.570+0000 7ff13985a640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7ff13406f4e0 con 0x7ff134070e80 2024-09-12T03:17:28.574 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.570+0000 7ff1327fc640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff1340cef50 0x7ff1340cd940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:28.574 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.570+0000 7ff1327fc640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff1340cef50 0x7ff1340cd940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60180/0 (socket says 172.21.15.27:60180) 2024-09-12T03:17:28.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.570+0000 7ff1327fc640 1 -- 172.21.15.27:0/1613672314 learned_addr learned my addr 172.21.15.27:0/1613672314 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:17:28.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.570+0000 7ff132ffd640 1 --2- 172.21.15.27:0/1613672314 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ff134070e80 0x7ff13415f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:28.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.570+0000 7ff1337fe640 1 --2- 172.21.15.27:0/1613672314 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff1340d0980 0x7ff1340cbe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:28.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.570+0000 7ff1327fc640 1 -- 172.21.15.27:0/1613672314 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ff134070e80 msgr2=0x7ff13415f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:28.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.570+0000 7ff1327fc640 1 --2- 172.21.15.27:0/1613672314 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ff134070e80 0x7ff13415f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:28.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.570+0000 7ff1327fc640 1 -- 172.21.15.27:0/1613672314 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff1340d0980 msgr2=0x7ff1340cbe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:28.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.570+0000 7ff1327fc640 1 --2- 172.21.15.27:0/1613672314 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff1340d0980 0x7ff1340cbe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:28.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.570+0000 7ff1327fc640 1 -- 172.21.15.27:0/1613672314 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1340ce690 con 0x7ff1340cef50 2024-09-12T03:17:28.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.570+0000 7ff132ffd640 1 --2- 172.21.15.27:0/1613672314 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ff134070e80 0x7ff13415f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:17:28.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.570+0000 7ff1337fe640 1 --2- 172.21.15.27:0/1613672314 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff1340d0980 0x7ff1340cbe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:17:28.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.571+0000 7ff1327fc640 1 --2- 172.21.15.27:0/1613672314 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff1340cef50 0x7ff1340cd940 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7ff128007e60 tx=0x7ff12800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:28.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.571+0000 7ff11bfff640 1 -- 172.21.15.27:0/1613672314 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff12801a070 con 0x7ff1340cef50 2024-09-12T03:17:28.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.571+0000 7ff11bfff640 1 -- 172.21.15.27:0/1613672314 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff128003b90 con 0x7ff1340cef50 2024-09-12T03:17:28.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.571+0000 7ff13985a640 1 -- 172.21.15.27:0/1613672314 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff13406cd70 con 0x7ff1340cef50 2024-09-12T03:17:28.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.571+0000 7ff11bfff640 1 -- 172.21.15.27:0/1613672314 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff128004030 con 0x7ff1340cef50 2024-09-12T03:17:28.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.571+0000 7ff13985a640 1 -- 172.21.15.27:0/1613672314 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1340ce3b0 con 0x7ff1340cef50 2024-09-12T03:17:28.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.572+0000 7ff13985a640 1 -- 172.21.15.27:0/1613672314 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1340712c0 con 0x7ff1340cef50 2024-09-12T03:17:28.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.572+0000 7ff11bfff640 1 -- 172.21.15.27:0/1613672314 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 7) ==== 90651+0+0 (secure 0 0 0) 0x7ff12801d070 con 0x7ff1340cef50 2024-09-12T03:17:28.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.573+0000 7ff11bfff640 1 --2- 172.21.15.27:0/1613672314 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7ff12006bfe0 0x7ff12006e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:28.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.573+0000 7ff11bfff640 1 -- 172.21.15.27:0/1613672314 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7ff12808b1f0 con 0x7ff1340cef50 2024-09-12T03:17:28.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.573+0000 7ff132ffd640 1 --2- 172.21.15.27:0/1613672314 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7ff12006bfe0 0x7ff12006e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:28.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.574+0000 7ff132ffd640 1 --2- 172.21.15.27:0/1613672314 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7ff12006bfe0 0x7ff12006e4a0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7ff124034000 tx=0x7ff12402e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:28.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.577+0000 7ff11bfff640 1 -- 172.21.15.27:0/1613672314 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7ff128059360 con 0x7ff1340cef50 2024-09-12T03:17:28.756 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.755+0000 7ff13985a640 1 -- 172.21.15.27:0/1613672314 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff1340af550 con 0x7ff1340cef50 2024-09-12T03:17:28.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.756+0000 7ff11bfff640 1 -- 172.21.15.27:0/1613672314 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) ==== 74+0+16681 (secure 0 0 0) 0x7ff12805d660 con 0x7ff1340cef50 2024-09-12T03:17:28.757 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:17:28.757 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":32,"fsid":"d2df504e-2647-4385-a5db-9fdf69f487a0","created":"2024-09-12T03:16:07.778937+0000","modified":"2024-09-12T03:17:12.768450+0000","last_up_change":"2024-09-12T03:16:19.314778+0000","last_in_change":"2024-09-12T03:16:09.893086+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T03:16:20.097646+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-09-12T03:17:09.018856+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-09-12T03:17:10.105881+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1851116c-1fb3-4dc9-9772-f90e2ef4c355","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6817","nonce":3002321488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6819","nonce":3002321488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6823","nonce":3002321488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6821","nonce":3002321488}]},"public_addr":"172.21.15.27:6817/3002321488","cluster_addr":"172.21.15.27:6819/3002321488","heartbeat_back_addr":"172.21.15.27:6823/3002321488","heartbeat_front_addr":"172.21.15.27:6821/3002321488","state":["exists","up"]},{"osd":1,"uuid":"f38ae17d-73eb-4204-b99d-975fa806fb87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6809","nonce":1613102886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6811","nonce":1613102886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6815","nonce":1613102886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6813","nonce":1613102886}]},"public_addr":"172.21.15.27:6809/1613102886","cluster_addr":"172.21.15.27:6811/1613102886","heartbeat_back_addr":"172.21.15.27:6815/1613102886","heartbeat_front_addr":"172.21.15.27:6813/1613102886","state":["exists","up"]},{"osd":2,"uuid":"fb12dcc8-bdd0-42b8-b46d-8b10106224bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6825","nonce":3534723355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6827","nonce":3534723355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6831","nonce":3534723355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6829","nonce":3534723355}]},"public_addr":"172.21.15.27:6825/3534723355","cluster_addr":"172.21.15.27:6827/3534723355","heartbeat_back_addr":"172.21.15.27:6831/3534723355","heartbeat_front_addr":"172.21.15.27:6829/3534723355","state":["exists","up"]},{"osd":3,"uuid":"d74d3f49-9c24-4b9f-98e2-3ecd7b5ef3b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6801","nonce":80661598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6803","nonce":80661598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6807","nonce":80661598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6805","nonce":80661598}]},"public_addr":"172.21.15.27:6801/80661598","cluster_addr":"172.21.15.27:6803/80661598","heartbeat_back_addr":"172.21.15.27:6807/80661598","heartbeat_front_addr":"172.21.15.27:6805/80661598","state":["exists","up"]},{"osd":4,"uuid":"2bf123f2-1b0d-41c4-aa3b-805bab267e59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6817","nonce":3943189827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6819","nonce":3943189827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6823","nonce":3943189827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6821","nonce":3943189827}]},"public_addr":"172.21.15.45:6817/3943189827","cluster_addr":"172.21.15.45:6819/3943189827","heartbeat_back_addr":"172.21.15.45:6823/3943189827","heartbeat_front_addr":"172.21.15.45:6821/3943189827","state":["exists","up"]},{"osd":5,"uuid":"c49ba5da-91c6-4a86-a97b-4aba89c253ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6809","nonce":3620760330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6811","nonce":3620760330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6815","nonce":3620760330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6813","nonce":3620760330}]},"public_addr":"172.21.15.45:6809/3620760330","cluster_addr":"172.21.15.45:6811/3620760330","heartbeat_back_addr":"172.21.15.45:6815/3620760330","heartbeat_front_addr":"172.21.15.45:6813/3620760330","state":["exists","up"]},{"osd":6,"uuid":"b0123475-89f7-4a34-9e33-d797b99dd9dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6801","nonce":1989083510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6803","nonce":1989083510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6807","nonce":1989083510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6805","nonce":1989083510}]},"public_addr":"172.21.15.45:6801/1989083510","cluster_addr":"172.21.15.45:6803/1989083510","heartbeat_back_addr":"172.21.15.45:6807/1989083510","heartbeat_front_addr":"172.21.15.45:6805/1989083510","state":["exists","up"]},{"osd":7,"uuid":"4dad773d-b7d8-46fc-8a10-a972aad27b29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6825","nonce":1513300294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6827","nonce":1513300294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6831","nonce":1513300294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6829","nonce":1513300294}]},"public_addr":"172.21.15.45:6825/1513300294","cluster_addr":"172.21.15.45:6827/1513300294","heartbeat_back_addr":"172.21.15.45:6831/1513300294","heartbeat_front_addr":"172.21.15.45:6829/1513300294","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.881855+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:17.133252+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.735339+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.500742+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.477571+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:17.167667+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.470291+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.873743+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T03:17:28.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.758+0000 7ff13985a640 1 -- 172.21.15.27:0/1613672314 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7ff12006bfe0 msgr2=0x7ff12006e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:28.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.758+0000 7ff13985a640 1 --2- 172.21.15.27:0/1613672314 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7ff12006bfe0 0x7ff12006e4a0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7ff124034000 tx=0x7ff12402e000 comp rx=0 tx=0).stop 2024-09-12T03:17:28.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.758+0000 7ff13985a640 1 -- 172.21.15.27:0/1613672314 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff1340cef50 msgr2=0x7ff1340cd940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:28.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.758+0000 7ff13985a640 1 --2- 172.21.15.27:0/1613672314 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff1340cef50 0x7ff1340cd940 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7ff128007e60 tx=0x7ff12800e4b0 comp rx=0 tx=0).stop 2024-09-12T03:17:28.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.758+0000 7ff13985a640 1 -- 172.21.15.27:0/1613672314 shutdown_connections 2024-09-12T03:17:28.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.758+0000 7ff13985a640 1 --2- 172.21.15.27:0/1613672314 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7ff12006bfe0 0x7ff12006e4a0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:28.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.758+0000 7ff13985a640 1 --2- 172.21.15.27:0/1613672314 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff1340d0980 0x7ff1340cbe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:28.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.758+0000 7ff13985a640 1 --2- 172.21.15.27:0/1613672314 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff1340cef50 0x7ff1340cd940 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:28.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.758+0000 7ff13985a640 1 --2- 172.21.15.27:0/1613672314 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ff134070e80 0x7ff13415f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:28.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.758+0000 7ff13985a640 1 -- 172.21.15.27:0/1613672314 >> 172.21.15.27:0/1613672314 conn(0x7ff134073800 msgr2=0x7ff1340b15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:28.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.758+0000 7ff13985a640 1 -- 172.21.15.27:0/1613672314 shutdown_connections 2024-09-12T03:17:28.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.758+0000 7ff13985a640 1 -- 172.21.15.27:0/1613672314 wait complete. 2024-09-12T03:17:28.770 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-12T03:17:28.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.873+0000 7f8b17bd8640 1 Processor -- start 2024-09-12T03:17:28.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.873+0000 7f8b17bd8640 1 -- start start 2024-09-12T03:17:28.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.874+0000 7f8b17bd8640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f8b1015e5f0 0x7f8b10073250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:28.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.874+0000 7f8b17bd8640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f8b10073790 0x7f8b10162320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:28.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.874+0000 7f8b17bd8640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8b10073bb0 con 0x7f8b10162860 2024-09-12T03:17:28.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.874+0000 7f8b17bd8640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f8b100d0da0 con 0x7f8b10073790 2024-09-12T03:17:28.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.874+0000 7f8b17bd8640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f8b10163090 con 0x7f8b1015e5f0 2024-09-12T03:17:28.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.874+0000 7f8b15173640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f8b10073790 0x7f8b10162320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:28.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.874+0000 7f8b16175640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f8b10162860 0x7f8b100d1230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:51436/0 (socket says 172.21.15.27:51436) 2024-09-12T03:17:28.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.874+0000 7f8b16175640 1 -- 172.21.15.27:0/427167377 learned_addr learned my addr 172.21.15.27:0/427167377 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:17:28.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.874+0000 7f8b15974640 1 --2- 172.21.15.27:0/427167377 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f8b1015e5f0 0x7f8b10073250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:28.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.874+0000 7f8b15173640 1 -- 172.21.15.27:0/427167377 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f8b1015e5f0 msgr2=0x7f8b10073250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:28.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.874+0000 7f8b15173640 1 --2- 172.21.15.27:0/427167377 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f8b1015e5f0 0x7f8b10073250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:28.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.874+0000 7f8b15173640 1 -- 172.21.15.27:0/427167377 >> v1:172.21.15.27:6789/0 conn(0x7f8b10162860 legacy=0x7f8b100d1230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:28.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.874+0000 7f8b15173640 1 -- 172.21.15.27:0/427167377 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b100d0470 con 0x7f8b10073790 2024-09-12T03:17:28.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.874+0000 7f8b14972640 1 -- 172.21.15.27:0/427167377 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 906387589 0 0) 0x7f8b10073bb0 con 0x7f8b10162860 2024-09-12T03:17:28.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.874+0000 7f8b15974640 1 --2- 172.21.15.27:0/427167377 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f8b1015e5f0 0x7f8b10073250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:17:28.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.875+0000 7f8b15173640 1 --2- 172.21.15.27:0/427167377 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8b10073790 0x7f8b10162320 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f8b000097a0 tx=0x7f8b0002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6f6e5a5dae56eea server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:28.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.875+0000 7f8b14972640 1 -- 172.21.15.27:0/427167377 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b00037070 con 0x7f8b10073790 2024-09-12T03:17:28.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.875+0000 7f8b14972640 1 -- 172.21.15.27:0/427167377 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b00032440 con 0x7f8b10073790 2024-09-12T03:17:28.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.875+0000 7f8b14972640 1 -- 172.21.15.27:0/427167377 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b00032810 con 0x7f8b10073790 2024-09-12T03:17:28.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.876+0000 7f8b17bd8640 1 -- 172.21.15.27:0/427167377 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8b10073790 msgr2=0x7f8b10162320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:28.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.876+0000 7f8b17bd8640 1 --2- 172.21.15.27:0/427167377 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8b10073790 0x7f8b10162320 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f8b000097a0 tx=0x7f8b0002a180 comp rx=0 tx=0).stop 2024-09-12T03:17:28.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.876+0000 7f8b17bd8640 1 -- 172.21.15.27:0/427167377 shutdown_connections 2024-09-12T03:17:28.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.876+0000 7f8b17bd8640 1 --2- 172.21.15.27:0/427167377 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8b10073790 0x7f8b10162320 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:28.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.876+0000 7f8b17bd8640 1 --2- 172.21.15.27:0/427167377 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f8b1015e5f0 0x7f8b10073250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:28.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.876+0000 7f8b17bd8640 1 -- 172.21.15.27:0/427167377 >> 172.21.15.27:0/427167377 conn(0x7f8b1006f300 msgr2=0x7f8b1006f710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:28.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.876+0000 7f8b17bd8640 1 -- 172.21.15.27:0/427167377 shutdown_connections 2024-09-12T03:17:28.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.876+0000 7f8b17bd8640 1 -- 172.21.15.27:0/427167377 wait complete. 2024-09-12T03:17:28.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.877+0000 7f8b17bd8640 1 Processor -- start 2024-09-12T03:17:28.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.877+0000 7f8b17bd8640 1 -- start start 2024-09-12T03:17:28.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.878+0000 7f8b17bd8640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f8b10073790 0x7f8b10157d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:28.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.878+0000 7f8b17bd8640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8b1015e5f0 0x7f8b101f78b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:28.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.878+0000 7f8b17bd8640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8b10162860 0x7f8b101f7df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:28.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.878+0000 7f8b17bd8640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f8b100b5620 con 0x7f8b10162860 2024-09-12T03:17:28.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.878+0000 7f8b17bd8640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f8b100d0a30 con 0x7f8b1015e5f0 2024-09-12T03:17:28.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.878+0000 7f8b17bd8640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f8b10162cf0 con 0x7f8b10073790 2024-09-12T03:17:28.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.878+0000 7f8b16175640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8b10162860 0x7f8b101f7df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:28.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.878+0000 7f8b16175640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8b10162860 0x7f8b101f7df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60192/0 (socket says 172.21.15.27:60192) 2024-09-12T03:17:28.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.878+0000 7f8b16175640 1 -- 172.21.15.27:0/625686797 learned_addr learned my addr 172.21.15.27:0/625686797 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:17:28.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.878+0000 7f8b15974640 1 --2- 172.21.15.27:0/625686797 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f8b10073790 0x7f8b10157d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:28.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.878+0000 7f8b15173640 1 --2- 172.21.15.27:0/625686797 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8b1015e5f0 0x7f8b101f78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:28.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.878+0000 7f8b16175640 1 -- 172.21.15.27:0/625686797 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f8b10073790 msgr2=0x7f8b10157d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:28.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.878+0000 7f8b16175640 1 --2- 172.21.15.27:0/625686797 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f8b10073790 0x7f8b10157d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:28.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.878+0000 7f8b16175640 1 -- 172.21.15.27:0/625686797 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8b1015e5f0 msgr2=0x7f8b101f78b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:28.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.878+0000 7f8b16175640 1 --2- 172.21.15.27:0/625686797 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8b1015e5f0 0x7f8b101f78b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:28.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.878+0000 7f8b16175640 1 -- 172.21.15.27:0/625686797 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b10158340 con 0x7f8b10162860 2024-09-12T03:17:28.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.878+0000 7f8b15173640 1 --2- 172.21.15.27:0/625686797 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8b1015e5f0 0x7f8b101f78b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:17:28.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.878+0000 7f8b16175640 1 --2- 172.21.15.27:0/625686797 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8b10162860 0x7f8b101f7df0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f8b0800c700 tx=0x7f8b0800cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:28.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.879+0000 7f8b067fc640 1 -- 172.21.15.27:0/625686797 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b08018070 con 0x7f8b10162860 2024-09-12T03:17:28.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.879+0000 7f8b067fc640 1 -- 172.21.15.27:0/625686797 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b0800ce10 con 0x7f8b10162860 2024-09-12T03:17:28.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.879+0000 7f8b067fc640 1 -- 172.21.15.27:0/625686797 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b0800a860 con 0x7f8b10162860 2024-09-12T03:17:28.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.879+0000 7f8b17bd8640 1 -- 172.21.15.27:0/625686797 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b10158620 con 0x7f8b10162860 2024-09-12T03:17:28.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.879+0000 7f8b17bd8640 1 -- 172.21.15.27:0/625686797 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b10158be0 con 0x7f8b10162860 2024-09-12T03:17:28.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.880+0000 7f8b17bd8640 1 -- 172.21.15.27:0/625686797 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b10160510 con 0x7f8b10162860 2024-09-12T03:17:28.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.880+0000 7f8b067fc640 1 -- 172.21.15.27:0/625686797 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 7) ==== 90651+0+0 (secure 0 0 0) 0x7f8b0800aa00 con 0x7f8b10162860 2024-09-12T03:17:28.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.881+0000 7f8b067fc640 1 --2- 172.21.15.27:0/625686797 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f8af406bfe0 0x7f8af406e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:28.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.881+0000 7f8b067fc640 1 -- 172.21.15.27:0/625686797 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f8b0808a6f0 con 0x7f8b10162860 2024-09-12T03:17:28.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.881+0000 7f8b15974640 1 --2- 172.21.15.27:0/625686797 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f8af406bfe0 0x7f8af406e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:28.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.881+0000 7f8b15974640 1 --2- 172.21.15.27:0/625686797 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f8af406bfe0 0x7f8af406e4a0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f8b0c00a990 tx=0x7f8b0c008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:28.886 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:28.885+0000 7f8b067fc640 1 -- 172.21.15.27:0/625686797 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f8b08058eb0 con 0x7f8b10162860 2024-09-12T03:17:29.098 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:29.097+0000 7f8b17bd8640 1 -- 172.21.15.27:0/625686797 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f8b10069790 con 0x7f8b10162860 2024-09-12T03:17:29.098 INFO:tasks.ceph.mon.a.smithi027.stderr:2024-09-12T03:17:29.098+0000 7f959bfd9640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-12T03:17:29.102 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:29.101+0000 7f8b067fc640 1 -- 172.21.15.27:0/625686797 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v16) ==== 161+0+0 (secure 0 0 0) 0x7f8b080587b0 con 0x7f8b10162860 2024-09-12T03:17:29.102 INFO:teuthology.orchestra.run.smithi027.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-12T03:17:29.105 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:29.103+0000 7f8b17bd8640 1 -- 172.21.15.27:0/625686797 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f8af406bfe0 msgr2=0x7f8af406e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:29.105 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:29.103+0000 7f8b17bd8640 1 --2- 172.21.15.27:0/625686797 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f8af406bfe0 0x7f8af406e4a0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f8b0c00a990 tx=0x7f8b0c008040 comp rx=0 tx=0).stop 2024-09-12T03:17:29.106 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:29.103+0000 7f8b17bd8640 1 -- 172.21.15.27:0/625686797 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8b10162860 msgr2=0x7f8b101f7df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:29.106 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:29.103+0000 7f8b17bd8640 1 --2- 172.21.15.27:0/625686797 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8b10162860 0x7f8b101f7df0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f8b0800c700 tx=0x7f8b0800cbc0 comp rx=0 tx=0).stop 2024-09-12T03:17:29.106 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:29.103+0000 7f8b17bd8640 1 -- 172.21.15.27:0/625686797 shutdown_connections 2024-09-12T03:17:29.106 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:29.103+0000 7f8b17bd8640 1 --2- 172.21.15.27:0/625686797 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f8af406bfe0 0x7f8af406e4a0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:29.106 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:29.103+0000 7f8b17bd8640 1 --2- 172.21.15.27:0/625686797 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8b10162860 0x7f8b101f7df0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:29.106 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:29.103+0000 7f8b17bd8640 1 --2- 172.21.15.27:0/625686797 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8b1015e5f0 0x7f8b101f78b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:29.106 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:29.103+0000 7f8b17bd8640 1 --2- 172.21.15.27:0/625686797 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f8b10073790 0x7f8b10157d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:29.106 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:29.103+0000 7f8b17bd8640 1 -- 172.21.15.27:0/625686797 >> 172.21.15.27:0/625686797 conn(0x7f8b1006f300 msgr2=0x7f8b100b5af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:29.106 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:29.103+0000 7f8b17bd8640 1 -- 172.21.15.27:0/625686797 shutdown_connections 2024-09-12T03:17:29.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:29.103+0000 7f8b17bd8640 1 -- 172.21.15.27:0/625686797 wait complete. 2024-09-12T03:17:29.117 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-12T03:17:29.143 INFO:tasks.ceph.mds.a.smithi027.stdout:starting mds.a at 2024-09-12T03:17:29.222 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:29.221+0000 7f124c874640 1 Processor -- start 2024-09-12T03:17:29.222 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:29.221+0000 7f124c874640 1 -- start start 2024-09-12T03:17:29.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:29.222+0000 7f124c874640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f12480b6240 0x7f12480b6620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:29.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:29.222+0000 7f124c874640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f12480d3c20 0x7f12480d4040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:29.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:29.222+0000 7f124c874640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f12480b6b60 con 0x7f124806d960 2024-09-12T03:17:29.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:29.222+0000 7f124c874640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f12480d2930 con 0x7f12480d3c20 2024-09-12T03:17:29.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:29.222+0000 7f124c874640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f12480d2c10 con 0x7f12480b6240 2024-09-12T03:17:29.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:29.222+0000 7f1245d9b640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f12480d3c20 0x7f12480d4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:29.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:29.222+0000 7f1246d9d640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f124806d960 0x7f124806de10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:51456/0 (socket says 172.21.15.27:51456) 2024-09-12T03:17:29.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:29.222+0000 7f1246d9d640 1 -- 172.21.15.27:0/3980991023 learned_addr learned my addr 172.21.15.27:0/3980991023 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:17:29.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:29.222+0000 7f124659c640 1 --2- 172.21.15.27:0/3980991023 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f12480b6240 0x7f12480b6620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:29.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:29.223+0000 7f1245d9b640 1 -- 172.21.15.27:0/3980991023 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f12480b6240 msgr2=0x7f12480b6620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:29.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:29.223+0000 7f1245d9b640 1 --2- 172.21.15.27:0/3980991023 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f12480b6240 0x7f12480b6620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:29.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:29.223+0000 7f1245d9b640 1 -- 172.21.15.27:0/3980991023 >> v1:172.21.15.27:6789/0 conn(0x7f124806d960 legacy=0x7f124806de10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:29.225 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:29.223+0000 7f1245d9b640 1 -- 172.21.15.27:0/3980991023 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12480d2ef0 con 0x7f12480d3c20 2024-09-12T03:17:29.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:29.223+0000 7f1245d9b640 1 --2- 172.21.15.27:0/3980991023 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f12480d3c20 0x7f12480d4040 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f123c0097a0 tx=0x7f123c02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65b6432bab105da6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:29.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:29.223+0000 7f124559a640 1 -- 172.21.15.27:0/3980991023 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f123c037070 con 0x7f12480d3c20 2024-09-12T03:17:29.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:29.224+0000 7f124559a640 1 -- 172.21.15.27:0/3980991023 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f123c032440 con 0x7f12480d3c20 2024-09-12T03:17:29.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:29.224+0000 7f124559a640 1 -- 172.21.15.27:0/3980991023 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f123c032810 con 0x7f12480d3c20 2024-09-12T03:17:29.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:29.224+0000 7f124c874640 1 -- 172.21.15.27:0/3980991023 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f12480d3c20 msgr2=0x7f12480d4040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:29.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:29.224+0000 7f124c874640 1 --2- 172.21.15.27:0/3980991023 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f12480d3c20 0x7f12480d4040 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f123c0097a0 tx=0x7f123c02a210 comp rx=0 tx=0).stop 2024-09-12T03:17:29.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:29.224+0000 7f124c874640 1 -- 172.21.15.27:0/3980991023 shutdown_connections 2024-09-12T03:17:29.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:29.224+0000 7f124c874640 1 --2- 172.21.15.27:0/3980991023 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f12480d3c20 0x7f12480d4040 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:29.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:29.224+0000 7f124c874640 1 --2- 172.21.15.27:0/3980991023 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f12480b6240 0x7f12480b6620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:29.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:29.224+0000 7f124c874640 1 -- 172.21.15.27:0/3980991023 >> 172.21.15.27:0/3980991023 conn(0x7f12480d1420 msgr2=0x7f12480d1830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:29.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:29.224+0000 7f124c874640 1 -- 172.21.15.27:0/3980991023 shutdown_connections 2024-09-12T03:17:29.227 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:29.224+0000 7f124c874640 1 -- 172.21.15.27:0/3980991023 wait complete. 2024-09-12T03:17:29.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:29.226+0000 7f124c874640 1 Processor -- start 2024-09-12T03:17:29.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:29.226+0000 7f124c874640 1 -- start start 2024-09-12T03:17:29.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:29.226+0000 7f124c874640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f124806d960 0x7f1248161b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:29.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:29.226+0000 7f124c874640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f12480b6240 0x7f124815ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:29.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:29.226+0000 7f124c874640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f12480d3c20 0x7f1248156d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:29.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:29.226+0000 7f124c874640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f124815ece0 con 0x7f12480d3c20 2024-09-12T03:17:29.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:29.226+0000 7f124c874640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f124815eb60 con 0x7f12480b6240 2024-09-12T03:17:29.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:29.226+0000 7f124c874640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f12480cfc70 con 0x7f124806d960 2024-09-12T03:17:29.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:29.226+0000 7f1246d9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f12480d3c20 0x7f1248156d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:29.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:29.227+0000 7f1246d9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f12480d3c20 0x7f1248156d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60240/0 (socket says 172.21.15.27:60240) 2024-09-12T03:17:29.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:29.227+0000 7f1246d9d640 1 -- 172.21.15.27:0/3268869695 learned_addr learned my addr 172.21.15.27:0/3268869695 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:17:29.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:29.227+0000 7f1245d9b640 1 --2- 172.21.15.27:0/3268869695 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f12480b6240 0x7f124815ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:29.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:29.227+0000 7f124659c640 1 --2- 172.21.15.27:0/3268869695 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f124806d960 0x7f1248161b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:29.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:29.227+0000 7f1246d9d640 1 -- 172.21.15.27:0/3268869695 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f124806d960 msgr2=0x7f1248161b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:29.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:29.227+0000 7f1246d9d640 1 --2- 172.21.15.27:0/3268869695 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f124806d960 0x7f1248161b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:29.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:29.227+0000 7f1246d9d640 1 -- 172.21.15.27:0/3268869695 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f12480b6240 msgr2=0x7f124815ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:29.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:29.227+0000 7f1246d9d640 1 --2- 172.21.15.27:0/3268869695 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f12480b6240 0x7f124815ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:29.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:29.227+0000 7f1246d9d640 1 -- 172.21.15.27:0/3268869695 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1248162070 con 0x7f12480d3c20 2024-09-12T03:17:29.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:29.227+0000 7f124659c640 1 --2- 172.21.15.27:0/3268869695 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f124806d960 0x7f1248161b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:17:29.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:29.227+0000 7f1246d9d640 1 --2- 172.21.15.27:0/3268869695 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f12480d3c20 0x7f1248156d80 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f124000cbc0 tx=0x7f124000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:29.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:29.227+0000 7f1236ffd640 1 -- 172.21.15.27:0/3268869695 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1240017070 con 0x7f12480d3c20 2024-09-12T03:17:29.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:29.227+0000 7f1236ffd640 1 -- 172.21.15.27:0/3268869695 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f124000ad40 con 0x7f12480d3c20 2024-09-12T03:17:29.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:29.227+0000 7f124c874640 1 -- 172.21.15.27:0/3268869695 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f124815f940 con 0x7f12480d3c20 2024-09-12T03:17:29.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:29.227+0000 7f1236ffd640 1 -- 172.21.15.27:0/3268869695 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1240012600 con 0x7f12480d3c20 2024-09-12T03:17:29.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:29.227+0000 7f124c874640 1 -- 172.21.15.27:0/3268869695 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f124815ff00 con 0x7f12480d3c20 2024-09-12T03:17:29.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:29.228+0000 7f124c874640 1 -- 172.21.15.27:0/3268869695 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12480b6620 con 0x7f12480d3c20 2024-09-12T03:17:29.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:29.229+0000 7f1236ffd640 1 -- 172.21.15.27:0/3268869695 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 7) ==== 90651+0+0 (secure 0 0 0) 0x7f124001a070 con 0x7f12480d3c20 2024-09-12T03:17:29.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:29.229+0000 7f1236ffd640 1 --2- 172.21.15.27:0/3268869695 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f122806bfe0 0x7f122806e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:29.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:29.229+0000 7f1236ffd640 1 -- 172.21.15.27:0/3268869695 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5795+0+0 (secure 0 0 0) 0x7f124008ae10 con 0x7f12480d3c20 2024-09-12T03:17:29.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:29.230+0000 7f124659c640 1 --2- 172.21.15.27:0/3268869695 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f122806bfe0 0x7f122806e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:29.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:29.230+0000 7f124659c640 1 --2- 172.21.15.27:0/3268869695 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f122806bfe0 0x7f122806e4a0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f1238006fd0 tx=0x7f1238008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:29.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:29.233+0000 7f1236ffd640 1 -- 172.21.15.27:0/3268869695 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f1248162070 con 0x7f12480d3c20 2024-09-12T03:17:29.449 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:29.448+0000 7f124c874640 1 -- 172.21.15.27:0/3268869695 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f12480b5be0 con 0x7f12480d3c20 2024-09-12T03:17:30.105 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:30.104+0000 7f1236ffd640 1 -- 172.21.15.27:0/3268869695 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v17) ==== 104+0+0 (secure 0 0 0) 0x7f1240058f40 con 0x7f12480d3c20 2024-09-12T03:17:30.109 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:30.108+0000 7f124c874640 1 -- 172.21.15.27:0/3268869695 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f122806bfe0 msgr2=0x7f122806e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:30.109 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:30.108+0000 7f124c874640 1 --2- 172.21.15.27:0/3268869695 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f122806bfe0 0x7f122806e4a0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f1238006fd0 tx=0x7f1238008040 comp rx=0 tx=0).stop 2024-09-12T03:17:30.109 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:30.108+0000 7f124c874640 1 -- 172.21.15.27:0/3268869695 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f12480d3c20 msgr2=0x7f1248156d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:30.109 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:30.108+0000 7f124c874640 1 --2- 172.21.15.27:0/3268869695 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f12480d3c20 0x7f1248156d80 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f124000cbc0 tx=0x7f124000a430 comp rx=0 tx=0).stop 2024-09-12T03:17:30.109 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:30.108+0000 7f124c874640 1 -- 172.21.15.27:0/3268869695 shutdown_connections 2024-09-12T03:17:30.109 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:30.108+0000 7f124c874640 1 --2- 172.21.15.27:0/3268869695 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f122806bfe0 0x7f122806e4a0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:30.109 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:30.108+0000 7f124c874640 1 --2- 172.21.15.27:0/3268869695 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f12480d3c20 0x7f1248156d80 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:30.110 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:30.108+0000 7f124c874640 1 --2- 172.21.15.27:0/3268869695 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f12480b6240 0x7f124815ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:30.110 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:30.108+0000 7f124c874640 1 --2- 172.21.15.27:0/3268869695 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f124806d960 0x7f1248161b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:30.110 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:30.108+0000 7f124c874640 1 -- 172.21.15.27:0/3268869695 >> 172.21.15.27:0/3268869695 conn(0x7f12480d1420 msgr2=0x7f12480b75f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:30.110 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:30.108+0000 7f124c874640 1 -- 172.21.15.27:0/3268869695 shutdown_connections 2024-09-12T03:17:30.110 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:30.108+0000 7f124c874640 1 -- 172.21.15.27:0/3268869695 wait complete. 2024-09-12T03:17:30.123 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-12T03:17:30.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:30.278+0000 7fe08021e640 1 Processor -- start 2024-09-12T03:17:30.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:30.278+0000 7fe08021e640 1 -- start start 2024-09-12T03:17:30.280 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:30.279+0000 7fe08021e640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fe07806cc40 0x7fe07806d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:30.280 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:30.279+0000 7fe08021e640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fe07806db30 0x7fe07806df60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:30.280 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:30.279+0000 7fe08021e640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe07806d5d0 con 0x7fe07815e650 2024-09-12T03:17:30.280 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:30.279+0000 7fe08021e640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7fe07806d850 con 0x7fe07806cc40 2024-09-12T03:17:30.281 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:30.279+0000 7fe08021e640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fe0780d1260 con 0x7fe07806db30 2024-09-12T03:17:30.281 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:30.279+0000 7fe07d7b9640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fe07806cc40 0x7fe07806d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:30.281 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:30.279+0000 7fe07dfba640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fe07815e650 0x7fe0780d1d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:46574/0 (socket says 172.21.15.27:46574) 2024-09-12T03:17:30.281 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:30.279+0000 7fe07dfba640 1 -- 172.21.15.27:0/3416112195 learned_addr learned my addr 172.21.15.27:0/3416112195 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:17:30.281 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:30.279+0000 7fe07e7bb640 1 --2- 172.21.15.27:0/3416112195 >> v2:172.21.15.45:3300/0 conn(0x7fe07806db30 0x7fe07806df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:30.281 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:30.279+0000 7fe07d7b9640 1 -- 172.21.15.27:0/3416112195 >> v2:172.21.15.45:3300/0 conn(0x7fe07806db30 msgr2=0x7fe07806df60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:30.281 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:30.279+0000 7fe07d7b9640 1 --2- 172.21.15.27:0/3416112195 >> v2:172.21.15.45:3300/0 conn(0x7fe07806db30 0x7fe07806df60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:30.281 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:30.279+0000 7fe07d7b9640 1 -- 172.21.15.27:0/3416112195 >> v1:172.21.15.27:6789/0 conn(0x7fe07815e650 legacy=0x7fe0780d1d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:30.281 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:30.279+0000 7fe07d7b9640 1 -- 172.21.15.27:0/3416112195 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0780d1540 con 0x7fe07806cc40 2024-09-12T03:17:30.282 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:30.280+0000 7fe07d7b9640 1 --2- 172.21.15.27:0/3416112195 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe07806cc40 0x7fe07806d090 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fe0780b21f0 tx=0x7fe06802a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d851a5004f0c6206 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:30.283 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:30.280+0000 7fe07cfb8640 1 -- 172.21.15.27:0/3416112195 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe068037070 con 0x7fe07806cc40 2024-09-12T03:17:30.283 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:30.280+0000 7fe07cfb8640 1 -- 172.21.15.27:0/3416112195 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe068032440 con 0x7fe07806cc40 2024-09-12T03:17:30.283 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:30.280+0000 7fe07cfb8640 1 -- 172.21.15.27:0/3416112195 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe06802ae60 con 0x7fe07806cc40 2024-09-12T03:17:30.283 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:30.281+0000 7fe08021e640 1 -- 172.21.15.27:0/3416112195 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe07806cc40 msgr2=0x7fe07806d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:30.283 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:30.281+0000 7fe08021e640 1 --2- 172.21.15.27:0/3416112195 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe07806cc40 0x7fe07806d090 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fe0780b21f0 tx=0x7fe06802a110 comp rx=0 tx=0).stop 2024-09-12T03:17:30.283 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:30.281+0000 7fe08021e640 1 -- 172.21.15.27:0/3416112195 shutdown_connections 2024-09-12T03:17:30.283 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:30.281+0000 7fe08021e640 1 --2- 172.21.15.27:0/3416112195 >> v2:172.21.15.45:3300/0 conn(0x7fe07806db30 0x7fe07806df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:30.283 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:30.281+0000 7fe08021e640 1 --2- 172.21.15.27:0/3416112195 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe07806cc40 0x7fe07806d090 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:30.283 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:30.281+0000 7fe08021e640 1 -- 172.21.15.27:0/3416112195 >> 172.21.15.27:0/3416112195 conn(0x7fe07806f320 msgr2=0x7fe07806f700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:30.283 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:30.281+0000 7fe08021e640 1 -- 172.21.15.27:0/3416112195 shutdown_connections 2024-09-12T03:17:30.283 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:30.281+0000 7fe08021e640 1 -- 172.21.15.27:0/3416112195 wait complete. 2024-09-12T03:17:30.285 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:30.282+0000 7fe08021e640 1 Processor -- start 2024-09-12T03:17:30.285 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:30.282+0000 7fe08021e640 1 -- start start 2024-09-12T03:17:30.285 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:30.283+0000 7fe08021e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe07806cc40 0x7fe07814b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:30.285 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:30.283+0000 7fe08021e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe07806db30 0x7fe0780ce510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:30.285 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:30.283+0000 7fe08021e640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe07815e650 0x7fe0780ad100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:30.286 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:30.283+0000 7fe08021e640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fe07806e620 con 0x7fe07806cc40 2024-09-12T03:17:30.286 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:30.283+0000 7fe08021e640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe07806e4a0 con 0x7fe07806db30 2024-09-12T03:17:30.286 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:30.283+0000 7fe08021e640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fe0780b55b0 con 0x7fe07815e650 2024-09-12T03:17:30.286 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:30.283+0000 7fe07dfba640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe07806cc40 0x7fe07814b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:30.286 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:30.283+0000 7fe07dfba640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe07806cc40 0x7fe07814b940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:48158/0 (socket says 172.21.15.27:48158) 2024-09-12T03:17:30.286 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:30.283+0000 7fe07dfba640 1 -- 172.21.15.27:0/1053429231 learned_addr learned my addr 172.21.15.27:0/1053429231 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:17:30.286 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:30.283+0000 7fe07dfba640 1 -- 172.21.15.27:0/1053429231 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe07815e650 msgr2=0x7fe0780ad100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:30.286 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:30.283+0000 7fe07dfba640 1 --2- 172.21.15.27:0/1053429231 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe07815e650 0x7fe0780ad100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:30.286 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:30.283+0000 7fe07d7b9640 1 --2- 172.21.15.27:0/1053429231 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe07806db30 0x7fe0780ce510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:30.286 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:30.283+0000 7fe07dfba640 1 -- 172.21.15.27:0/1053429231 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe07806db30 msgr2=0x7fe0780ce510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:30.286 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:30.283+0000 7fe07dfba640 1 --2- 172.21.15.27:0/1053429231 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe07806db30 0x7fe0780ce510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:30.286 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:30.283+0000 7fe07dfba640 1 -- 172.21.15.27:0/1053429231 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0780ceef0 con 0x7fe07806cc40 2024-09-12T03:17:30.286 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:30.284+0000 7fe07dfba640 1 --2- 172.21.15.27:0/1053429231 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe07806cc40 0x7fe07814b940 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fe07400c3f0 tx=0x7fe07400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:30.287 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:30.284+0000 7fe06e7fc640 1 -- 172.21.15.27:0/1053429231 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe074017070 con 0x7fe07806cc40 2024-09-12T03:17:30.287 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:30.284+0000 7fe06e7fc640 1 -- 172.21.15.27:0/1053429231 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe074005ce0 con 0x7fe07806cc40 2024-09-12T03:17:30.287 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:30.284+0000 7fe08021e640 1 -- 172.21.15.27:0/1053429231 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0780cf1d0 con 0x7fe07806cc40 2024-09-12T03:17:30.287 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:30.284+0000 7fe06e7fc640 1 -- 172.21.15.27:0/1053429231 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0740072f0 con 0x7fe07806cc40 2024-09-12T03:17:30.287 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:30.284+0000 7fe08021e640 1 -- 172.21.15.27:0/1053429231 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe07814c4f0 con 0x7fe07806cc40 2024-09-12T03:17:30.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:30.285+0000 7fe08021e640 1 -- 172.21.15.27:0/1053429231 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe07806d090 con 0x7fe07806cc40 2024-09-12T03:17:30.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:30.285+0000 7fe06e7fc640 1 -- 172.21.15.27:0/1053429231 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 7) ==== 90651+0+0 (secure 0 0 0) 0x7fe07401a070 con 0x7fe07806cc40 2024-09-12T03:17:30.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:30.286+0000 7fe06e7fc640 1 --2- 172.21.15.27:0/1053429231 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fe05c06bfe0 0x7fe05c06e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:30.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:30.286+0000 7fe06e7fc640 1 -- 172.21.15.27:0/1053429231 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5795+0+0 (secure 0 0 0) 0x7fe07408a4a0 con 0x7fe07806cc40 2024-09-12T03:17:30.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:30.286+0000 7fe07d7b9640 1 --2- 172.21.15.27:0/1053429231 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fe05c06bfe0 0x7fe05c06e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:30.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:30.287+0000 7fe07d7b9640 1 --2- 172.21.15.27:0/1053429231 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fe05c06bfe0 0x7fe05c06e4a0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fe068034000 tx=0x7fe06802e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:30.291 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:30.290+0000 7fe06e7fc640 1 -- 172.21.15.27:0/1053429231 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fe07806d5d0 con 0x7fe07806cc40 2024-09-12T03:17:30.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:30.466+0000 7fe08021e640 1 -- 172.21.15.27:0/1053429231 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fe078161930 con 0x7fe07806cc40 2024-09-12T03:17:30.540 DEBUG:teuthology.orchestra.run.smithi027:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-12T03:17:30.542 DEBUG:teuthology.orchestra.run.smithi045:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-12T03:17:31.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:31.106+0000 7fe06e7fc640 1 -- 172.21.15.27:0/1053429231 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v34) ==== 181+0+0 (secure 0 0 0) 0x7fe074058c00 con 0x7fe07806cc40 2024-09-12T03:17:31.107 INFO:teuthology.orchestra.run.smithi027.stderr:pool 'cephfs_metadata' removed 2024-09-12T03:17:31.110 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:31.109+0000 7fe08021e640 1 -- 172.21.15.27:0/1053429231 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fe05c06bfe0 msgr2=0x7fe05c06e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:31.110 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:31.109+0000 7fe08021e640 1 --2- 172.21.15.27:0/1053429231 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fe05c06bfe0 0x7fe05c06e4a0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fe068034000 tx=0x7fe06802e000 comp rx=0 tx=0).stop 2024-09-12T03:17:31.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:31.109+0000 7fe08021e640 1 -- 172.21.15.27:0/1053429231 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe07806cc40 msgr2=0x7fe07814b940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:31.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:31.109+0000 7fe08021e640 1 --2- 172.21.15.27:0/1053429231 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe07806cc40 0x7fe07814b940 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fe07400c3f0 tx=0x7fe07400c8b0 comp rx=0 tx=0).stop 2024-09-12T03:17:31.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:31.109+0000 7fe08021e640 1 -- 172.21.15.27:0/1053429231 shutdown_connections 2024-09-12T03:17:31.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:31.109+0000 7fe08021e640 1 --2- 172.21.15.27:0/1053429231 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fe05c06bfe0 0x7fe05c06e4a0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:31.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:31.109+0000 7fe08021e640 1 --2- 172.21.15.27:0/1053429231 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe07815e650 0x7fe0780ad100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:31.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:31.109+0000 7fe08021e640 1 --2- 172.21.15.27:0/1053429231 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe07806db30 0x7fe0780ce510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:31.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:31.109+0000 7fe08021e640 1 --2- 172.21.15.27:0/1053429231 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe07806cc40 0x7fe07814b940 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:31.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:31.109+0000 7fe08021e640 1 -- 172.21.15.27:0/1053429231 >> 172.21.15.27:0/1053429231 conn(0x7fe07806f320 msgr2=0x7fe0780b60a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:31.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:31.109+0000 7fe08021e640 1 -- 172.21.15.27:0/1053429231 shutdown_connections 2024-09-12T03:17:31.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:31.109+0000 7fe08021e640 1 -- 172.21.15.27:0/1053429231 wait complete. 2024-09-12T03:17:31.123 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-12T03:17:31.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:31.230+0000 7fd2e17e4640 1 Processor -- start 2024-09-12T03:17:31.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:31.230+0000 7fd2e17e4640 1 -- start start 2024-09-12T03:17:31.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:31.231+0000 7fd2e17e4640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fd2dc0d3c20 0x7fd2dc0d4040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:31.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:31.231+0000 7fd2e17e4640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fd2dc0b77c0 0x7fd2dc0b7c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:31.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:31.231+0000 7fd2e17e4640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd2dc0d2930 con 0x7fd2dc0b6240 2024-09-12T03:17:31.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:31.231+0000 7fd2e17e4640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7fd2dc0d2c10 con 0x7fd2dc0d3c20 2024-09-12T03:17:31.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:31.231+0000 7fd2e17e4640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fd2dc0d2ef0 con 0x7fd2dc0b77c0 2024-09-12T03:17:31.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:31.231+0000 7fd2da7fc640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fd2dc0d3c20 0x7fd2dc0d4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:31.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:31.231+0000 7fd2daffd640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fd2dc0b6240 0x7fd2dc0b6620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:46576/0 (socket says 172.21.15.27:46576) 2024-09-12T03:17:31.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:31.231+0000 7fd2daffd640 1 -- 172.21.15.27:0/4268963402 learned_addr learned my addr 172.21.15.27:0/4268963402 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:17:31.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:31.231+0000 7fd2da7fc640 1 -- 172.21.15.27:0/4268963402 >> v2:172.21.15.45:3300/0 conn(0x7fd2dc0b77c0 msgr2=0x7fd2dc0b7c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:31.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:31.231+0000 7fd2da7fc640 1 --2- 172.21.15.27:0/4268963402 >> v2:172.21.15.45:3300/0 conn(0x7fd2dc0b77c0 0x7fd2dc0b7c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:31.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:31.231+0000 7fd2da7fc640 1 -- 172.21.15.27:0/4268963402 >> v1:172.21.15.27:6789/0 conn(0x7fd2dc0b6240 legacy=0x7fd2dc0b6620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:31.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:31.232+0000 7fd2d9ffb640 1 -- 172.21.15.27:0/4268963402 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2838892298 0 0) 0x7fd2dc0d2930 con 0x7fd2dc0b6240 2024-09-12T03:17:31.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:31.232+0000 7fd2da7fc640 1 -- 172.21.15.27:0/4268963402 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2dc0d31d0 con 0x7fd2dc0d3c20 2024-09-12T03:17:31.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:31.232+0000 7fd2da7fc640 1 --2- 172.21.15.27:0/4268963402 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd2dc0d3c20 0x7fd2dc0d4040 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fd2dc08d8b0 tx=0x7fd2c402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8cb976379613b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:31.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:31.232+0000 7fd2d9ffb640 1 -- 172.21.15.27:0/4268963402 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2c4037070 con 0x7fd2dc0d3c20 2024-09-12T03:17:31.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:31.232+0000 7fd2d9ffb640 1 -- 172.21.15.27:0/4268963402 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2c4032440 con 0x7fd2dc0d3c20 2024-09-12T03:17:31.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:31.232+0000 7fd2d9ffb640 1 -- 172.21.15.27:0/4268963402 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2c4032810 con 0x7fd2dc0d3c20 2024-09-12T03:17:31.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:31.233+0000 7fd2e17e4640 1 -- 172.21.15.27:0/4268963402 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd2dc0d3c20 msgr2=0x7fd2dc0d4040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:31.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:31.233+0000 7fd2e17e4640 1 --2- 172.21.15.27:0/4268963402 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd2dc0d3c20 0x7fd2dc0d4040 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fd2dc08d8b0 tx=0x7fd2c402a230 comp rx=0 tx=0).stop 2024-09-12T03:17:31.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:31.233+0000 7fd2e17e4640 1 -- 172.21.15.27:0/4268963402 shutdown_connections 2024-09-12T03:17:31.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:31.233+0000 7fd2e17e4640 1 --2- 172.21.15.27:0/4268963402 >> v2:172.21.15.45:3300/0 conn(0x7fd2dc0b77c0 0x7fd2dc0b7c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:31.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:31.233+0000 7fd2e17e4640 1 --2- 172.21.15.27:0/4268963402 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd2dc0d3c20 0x7fd2dc0d4040 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:31.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:31.233+0000 7fd2e17e4640 1 -- 172.21.15.27:0/4268963402 >> 172.21.15.27:0/4268963402 conn(0x7fd2dc0d1420 msgr2=0x7fd2dc0d1830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:31.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:31.233+0000 7fd2e17e4640 1 -- 172.21.15.27:0/4268963402 shutdown_connections 2024-09-12T03:17:31.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:31.233+0000 7fd2e17e4640 1 -- 172.21.15.27:0/4268963402 wait complete. 2024-09-12T03:17:31.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:31.234+0000 7fd2e17e4640 1 Processor -- start 2024-09-12T03:17:31.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:31.234+0000 7fd2e17e4640 1 -- start start 2024-09-12T03:17:31.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:31.235+0000 7fd2e17e4640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fd2dc0b6240 0x7fd2dc1f52c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:31.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:31.235+0000 7fd2e17e4640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd2dc0b77c0 0x7fd2dc1615e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:31.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:31.235+0000 7fd2e17e4640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd2dc0d3c20 0x7fd2dc0ca8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:31.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:31.235+0000 7fd2e17e4640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fd2dc0cfdf0 con 0x7fd2dc0b77c0 2024-09-12T03:17:31.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:31.235+0000 7fd2e17e4640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fd2dc0cfc70 con 0x7fd2dc0d3c20 2024-09-12T03:17:31.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:31.235+0000 7fd2e17e4640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fd2dc0b8190 con 0x7fd2dc0b6240 2024-09-12T03:17:31.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:31.235+0000 7fd2da7fc640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd2dc0b77c0 0x7fd2dc1615e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:31.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:31.235+0000 7fd2da7fc640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd2dc0b77c0 0x7fd2dc1615e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:48178/0 (socket says 172.21.15.27:48178) 2024-09-12T03:17:31.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:31.235+0000 7fd2da7fc640 1 -- 172.21.15.27:0/3832386150 learned_addr learned my addr 172.21.15.27:0/3832386150 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:17:31.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:31.235+0000 7fd2db7fe640 1 --2- 172.21.15.27:0/3832386150 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd2dc0d3c20 0x7fd2dc0ca8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:31.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:31.235+0000 7fd2da7fc640 1 -- 172.21.15.27:0/3832386150 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fd2dc0b6240 msgr2=0x7fd2dc1f52c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:31.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:31.235+0000 7fd2da7fc640 1 --2- 172.21.15.27:0/3832386150 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fd2dc0b6240 0x7fd2dc1f52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:31.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:31.236+0000 7fd2da7fc640 1 -- 172.21.15.27:0/3832386150 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd2dc0d3c20 msgr2=0x7fd2dc0ca8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:31.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:31.236+0000 7fd2da7fc640 1 --2- 172.21.15.27:0/3832386150 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd2dc0d3c20 0x7fd2dc0ca8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:31.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:31.236+0000 7fd2da7fc640 1 -- 172.21.15.27:0/3832386150 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2dc0cb1d0 con 0x7fd2dc0b77c0 2024-09-12T03:17:31.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:31.236+0000 7fd2db7fe640 1 --2- 172.21.15.27:0/3832386150 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd2dc0d3c20 0x7fd2dc0ca8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:17:31.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:31.236+0000 7fd2da7fc640 1 --2- 172.21.15.27:0/3832386150 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd2dc0b77c0 0x7fd2dc1615e0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fd2c4035040 tx=0x7fd2c40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:31.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:31.236+0000 7fd2d37fe640 1 -- 172.21.15.27:0/3832386150 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2c4042020 con 0x7fd2dc0b77c0 2024-09-12T03:17:31.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:31.236+0000 7fd2d37fe640 1 -- 172.21.15.27:0/3832386150 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2c4030970 con 0x7fd2dc0b77c0 2024-09-12T03:17:31.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:31.236+0000 7fd2e17e4640 1 -- 172.21.15.27:0/3832386150 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2dc153a70 con 0x7fd2dc0b77c0 2024-09-12T03:17:31.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:31.236+0000 7fd2d37fe640 1 -- 172.21.15.27:0/3832386150 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2c4037040 con 0x7fd2dc0b77c0 2024-09-12T03:17:31.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:31.236+0000 7fd2e17e4640 1 -- 172.21.15.27:0/3832386150 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2dc154030 con 0x7fd2dc0b77c0 2024-09-12T03:17:31.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:31.237+0000 7fd2e17e4640 1 -- 172.21.15.27:0/3832386150 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2dc0b7c50 con 0x7fd2dc0b77c0 2024-09-12T03:17:31.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:31.237+0000 7fd2d37fe640 1 -- 172.21.15.27:0/3832386150 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 7) ==== 90651+0+0 (secure 0 0 0) 0x7fd2c4045020 con 0x7fd2dc0b77c0 2024-09-12T03:17:31.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:31.238+0000 7fd2d37fe640 1 --2- 172.21.15.27:0/3832386150 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fd2bc06bfe0 0x7fd2bc06e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:31.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:31.238+0000 7fd2d37fe640 1 -- 172.21.15.27:0/3832386150 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5357+0+0 (secure 0 0 0) 0x7fd2c40abd60 con 0x7fd2dc0b77c0 2024-09-12T03:17:31.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:31.238+0000 7fd2daffd640 1 --2- 172.21.15.27:0/3832386150 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fd2bc06bfe0 0x7fd2bc06e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:31.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:31.239+0000 7fd2daffd640 1 --2- 172.21.15.27:0/3832386150 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fd2bc06bfe0 0x7fd2bc06e4a0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fd2cc001560 tx=0x7fd2cc007ca0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:31.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:31.242+0000 7fd2d37fe640 1 -- 172.21.15.27:0/3832386150 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fd2c404c030 con 0x7fd2dc0b77c0 2024-09-12T03:17:31.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:31.416+0000 7fd2e17e4640 1 -- 172.21.15.27:0/3832386150 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fd2dc0d40a0 con 0x7fd2dc0b77c0 2024-09-12T03:17:32.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:32.110+0000 7fd2d37fe640 1 -- 172.21.15.27:0/3832386150 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v35) ==== 169+0+0 (secure 0 0 0) 0x7fd2c4032e20 con 0x7fd2dc0b77c0 2024-09-12T03:17:32.111 INFO:teuthology.orchestra.run.smithi027.stderr:pool 'cephfs_data' removed 2024-09-12T03:17:32.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:32.113+0000 7fd2e17e4640 1 -- 172.21.15.27:0/3832386150 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fd2bc06bfe0 msgr2=0x7fd2bc06e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:32.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:32.113+0000 7fd2e17e4640 1 --2- 172.21.15.27:0/3832386150 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fd2bc06bfe0 0x7fd2bc06e4a0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fd2cc001560 tx=0x7fd2cc007ca0 comp rx=0 tx=0).stop 2024-09-12T03:17:32.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:32.113+0000 7fd2e17e4640 1 -- 172.21.15.27:0/3832386150 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd2dc0b77c0 msgr2=0x7fd2dc1615e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:32.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:32.113+0000 7fd2e17e4640 1 --2- 172.21.15.27:0/3832386150 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd2dc0b77c0 0x7fd2dc1615e0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fd2c4035040 tx=0x7fd2c40046c0 comp rx=0 tx=0).stop 2024-09-12T03:17:32.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:32.113+0000 7fd2e17e4640 1 -- 172.21.15.27:0/3832386150 shutdown_connections 2024-09-12T03:17:32.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:32.113+0000 7fd2e17e4640 1 --2- 172.21.15.27:0/3832386150 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fd2bc06bfe0 0x7fd2bc06e4a0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:32.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:32.113+0000 7fd2e17e4640 1 --2- 172.21.15.27:0/3832386150 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd2dc0d3c20 0x7fd2dc0ca8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:32.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:32.113+0000 7fd2e17e4640 1 --2- 172.21.15.27:0/3832386150 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd2dc0b77c0 0x7fd2dc1615e0 secure :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fd2c4035040 tx=0x7fd2c40046c0 comp rx=0 tx=0).stop 2024-09-12T03:17:32.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:32.113+0000 7fd2e17e4640 1 --2- 172.21.15.27:0/3832386150 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fd2dc0b6240 0x7fd2dc1f52c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:32.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:32.113+0000 7fd2e17e4640 1 -- 172.21.15.27:0/3832386150 >> 172.21.15.27:0/3832386150 conn(0x7fd2dc0d1420 msgr2=0x7fd2dc0d1800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:32.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:32.114+0000 7fd2e17e4640 1 -- 172.21.15.27:0/3832386150 shutdown_connections 2024-09-12T03:17:32.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:32.115+0000 7fd2e17e4640 1 -- 172.21.15.27:0/3832386150 wait complete. 2024-09-12T03:17:32.130 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-09-12T03:17:32.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:32.236+0000 7ff5c88d0640 1 Processor -- start 2024-09-12T03:17:32.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:32.236+0000 7ff5c88d0640 1 -- start start 2024-09-12T03:17:32.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:32.237+0000 7ff5c88d0640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ff5c406cc40 0x7ff5c406d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:32.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:32.237+0000 7ff5c88d0640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ff5c406db30 0x7ff5c406df60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:32.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:32.237+0000 7ff5c88d0640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff5c406d5d0 con 0x7ff5c415e650 2024-09-12T03:17:32.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:32.237+0000 7ff5c88d0640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7ff5c406d850 con 0x7ff5c406cc40 2024-09-12T03:17:32.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:32.237+0000 7ff5c88d0640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7ff5c40d1260 con 0x7ff5c406db30 2024-09-12T03:17:32.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:32.237+0000 7ff5c1d9b640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ff5c406cc40 0x7ff5c406d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:32.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:32.237+0000 7ff5c2d9d640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ff5c406db30 0x7ff5c406df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:32.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:32.238+0000 7ff5c259c640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7ff5c415e650 0x7ff5c40d1d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.27:53892/0 (socket says 172.21.15.27:53892) 2024-09-12T03:17:32.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:32.238+0000 7ff5c259c640 1 -- 172.21.15.27:0/3159444576 learned_addr learned my addr 172.21.15.27:0/3159444576 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:17:32.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:32.238+0000 7ff5c2d9d640 1 -- 172.21.15.27:0/3159444576 >> v2:172.21.15.45:3300/0 conn(0x7ff5c406cc40 msgr2=0x7ff5c406d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:32.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:32.238+0000 7ff5c2d9d640 1 --2- 172.21.15.27:0/3159444576 >> v2:172.21.15.45:3300/0 conn(0x7ff5c406cc40 0x7ff5c406d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:32.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:32.238+0000 7ff5c2d9d640 1 -- 172.21.15.27:0/3159444576 >> v1:172.21.15.45:6789/0 conn(0x7ff5c415e650 legacy=0x7ff5c40d1d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:32.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:32.238+0000 7ff5c2d9d640 1 -- 172.21.15.27:0/3159444576 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5c40d1540 con 0x7ff5c406db30 2024-09-12T03:17:32.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:32.239+0000 7ff5c2d9d640 1 --2- 172.21.15.27:0/3159444576 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ff5c406db30 0x7ff5c406df60 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff5bc009d90 tx=0x7ff5bc02a5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=192de291c2348faf server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:32.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:32.239+0000 7ff5c159a640 1 -- 172.21.15.27:0/3159444576 <== mon.2 v2:172.21.15.45:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5bc031600 con 0x7ff5c406db30 2024-09-12T03:17:32.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:32.239+0000 7ff5c159a640 1 -- 172.21.15.27:0/3159444576 <== mon.2 v2:172.21.15.45:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5bc0317a0 con 0x7ff5c406db30 2024-09-12T03:17:32.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:32.239+0000 7ff5c159a640 1 -- 172.21.15.27:0/3159444576 <== mon.2 v2:172.21.15.45:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5bc031b70 con 0x7ff5c406db30 2024-09-12T03:17:32.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:32.239+0000 7ff5c88d0640 1 -- 172.21.15.27:0/3159444576 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ff5c406db30 msgr2=0x7ff5c406df60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:32.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:32.239+0000 7ff5c88d0640 1 --2- 172.21.15.27:0/3159444576 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ff5c406db30 0x7ff5c406df60 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff5bc009d90 tx=0x7ff5bc02a5d0 comp rx=0 tx=0).stop 2024-09-12T03:17:32.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:32.240+0000 7ff5c88d0640 1 -- 172.21.15.27:0/3159444576 shutdown_connections 2024-09-12T03:17:32.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:32.240+0000 7ff5c88d0640 1 --2- 172.21.15.27:0/3159444576 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ff5c406db30 0x7ff5c406df60 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:32.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:32.240+0000 7ff5c88d0640 1 --2- 172.21.15.27:0/3159444576 >> v2:172.21.15.45:3300/0 conn(0x7ff5c406cc40 0x7ff5c406d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:32.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:32.240+0000 7ff5c88d0640 1 -- 172.21.15.27:0/3159444576 >> 172.21.15.27:0/3159444576 conn(0x7ff5c406f320 msgr2=0x7ff5c406f700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:32.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:32.240+0000 7ff5c88d0640 1 -- 172.21.15.27:0/3159444576 shutdown_connections 2024-09-12T03:17:32.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:32.240+0000 7ff5c88d0640 1 -- 172.21.15.27:0/3159444576 wait complete. 2024-09-12T03:17:32.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:32.241+0000 7ff5c88d0640 1 Processor -- start 2024-09-12T03:17:32.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:32.241+0000 7ff5c88d0640 1 -- start start 2024-09-12T03:17:32.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:32.242+0000 7ff5c88d0640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ff5c406cc40 0x7ff5c41533a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:32.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:32.242+0000 7ff5c88d0640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff5c406db30 0x7ff5c40b6680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:32.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:32.242+0000 7ff5c88d0640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff5c415e650 0x7ff5c40b6bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:32.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:32.242+0000 7ff5c88d0640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7ff5c406e620 con 0x7ff5c406db30 2024-09-12T03:17:32.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:32.242+0000 7ff5c88d0640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ff5c406e4a0 con 0x7ff5c415e650 2024-09-12T03:17:32.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:32.242+0000 7ff5c88d0640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7ff5c40b55b0 con 0x7ff5c406cc40 2024-09-12T03:17:32.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:32.242+0000 7ff5c1d9b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff5c406db30 0x7ff5c40b6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:32.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:32.242+0000 7ff5c1d9b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff5c406db30 0x7ff5c40b6680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:48190/0 (socket says 172.21.15.27:48190) 2024-09-12T03:17:32.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:32.242+0000 7ff5c1d9b640 1 -- 172.21.15.27:0/2825613666 learned_addr learned my addr 172.21.15.27:0/2825613666 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:17:32.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:32.242+0000 7ff5c259c640 1 --2- 172.21.15.27:0/2825613666 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ff5c406cc40 0x7ff5c41533a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:32.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:32.242+0000 7ff5c2d9d640 1 --2- 172.21.15.27:0/2825613666 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff5c415e650 0x7ff5c40b6bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:32.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:32.242+0000 7ff5c1d9b640 1 -- 172.21.15.27:0/2825613666 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ff5c406cc40 msgr2=0x7ff5c41533a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:32.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:32.242+0000 7ff5c1d9b640 1 --2- 172.21.15.27:0/2825613666 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ff5c406cc40 0x7ff5c41533a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:32.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:32.242+0000 7ff5c1d9b640 1 -- 172.21.15.27:0/2825613666 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff5c415e650 msgr2=0x7ff5c40b6bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:32.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:32.242+0000 7ff5c1d9b640 1 --2- 172.21.15.27:0/2825613666 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff5c415e650 0x7ff5c40b6bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:32.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:32.242+0000 7ff5c1d9b640 1 -- 172.21.15.27:0/2825613666 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5c40ce760 con 0x7ff5c406db30 2024-09-12T03:17:32.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:32.242+0000 7ff5c1d9b640 1 --2- 172.21.15.27:0/2825613666 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff5c406db30 0x7ff5c40b6680 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7ff5b800e9b0 tx=0x7ff5b800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:32.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:32.243+0000 7ff5b2ffd640 1 -- 172.21.15.27:0/2825613666 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5b8017070 con 0x7ff5c406db30 2024-09-12T03:17:32.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:32.243+0000 7ff5b2ffd640 1 -- 172.21.15.27:0/2825613666 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5b800c9a0 con 0x7ff5c406db30 2024-09-12T03:17:32.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:32.243+0000 7ff5c88d0640 1 -- 172.21.15.27:0/2825613666 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5c40cea40 con 0x7ff5c406db30 2024-09-12T03:17:32.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:32.243+0000 7ff5b2ffd640 1 -- 172.21.15.27:0/2825613666 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5b800cd90 con 0x7ff5c406db30 2024-09-12T03:17:32.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:32.243+0000 7ff5c88d0640 1 -- 172.21.15.27:0/2825613666 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5c40cf000 con 0x7ff5c406db30 2024-09-12T03:17:32.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:32.244+0000 7ff5c88d0640 1 -- 172.21.15.27:0/2825613666 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5c406d090 con 0x7ff5c406db30 2024-09-12T03:17:32.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:32.244+0000 7ff5b2ffd640 1 -- 172.21.15.27:0/2825613666 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 7) ==== 90651+0+0 (secure 0 0 0) 0x7ff5b801a070 con 0x7ff5c406db30 2024-09-12T03:17:32.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:32.245+0000 7ff5b2ffd640 1 --2- 172.21.15.27:0/2825613666 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7ff5a406bfe0 0x7ff5a406e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:32.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:32.245+0000 7ff5b2ffd640 1 -- 172.21.15.27:0/2825613666 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 4943+0+0 (secure 0 0 0) 0x7ff5b808b0f0 con 0x7ff5c406db30 2024-09-12T03:17:32.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:32.245+0000 7ff5c259c640 1 --2- 172.21.15.27:0/2825613666 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7ff5a406bfe0 0x7ff5a406e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:32.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:32.246+0000 7ff5c259c640 1 --2- 172.21.15.27:0/2825613666 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7ff5a406bfe0 0x7ff5a406e4a0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ff5c415de50 tx=0x7ff5b4007e80 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:32.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:32.249+0000 7ff5b2ffd640 1 -- 172.21.15.27:0/2825613666 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7ff5b8059590 con 0x7ff5c406db30 2024-09-12T03:17:32.426 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:32.425+0000 7ff5c88d0640 1 -- 172.21.15.27:0/2825613666 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7ff5c40d3310 con 0x7ff5c406db30 2024-09-12T03:17:32.429 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:32.428+0000 7ff5b2ffd640 1 -- 172.21.15.27:0/2825613666 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) ==== 104+0+0 (secure 0 0 0) 0x7ff5b805d890 con 0x7ff5c406db30 2024-09-12T03:17:32.431 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:32.430+0000 7ff5c88d0640 1 -- 172.21.15.27:0/2825613666 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7ff5a406bfe0 msgr2=0x7ff5a406e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:32.431 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:32.430+0000 7ff5c88d0640 1 --2- 172.21.15.27:0/2825613666 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7ff5a406bfe0 0x7ff5a406e4a0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ff5c415de50 tx=0x7ff5b4007e80 comp rx=0 tx=0).stop 2024-09-12T03:17:32.431 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:32.430+0000 7ff5c88d0640 1 -- 172.21.15.27:0/2825613666 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff5c406db30 msgr2=0x7ff5c40b6680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:32.432 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:32.430+0000 7ff5c88d0640 1 --2- 172.21.15.27:0/2825613666 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff5c406db30 0x7ff5c40b6680 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7ff5b800e9b0 tx=0x7ff5b800ee70 comp rx=0 tx=0).stop 2024-09-12T03:17:32.432 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:32.430+0000 7ff5c88d0640 1 -- 172.21.15.27:0/2825613666 shutdown_connections 2024-09-12T03:17:32.432 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:32.430+0000 7ff5c88d0640 1 --2- 172.21.15.27:0/2825613666 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7ff5a406bfe0 0x7ff5a406e4a0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:32.432 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:32.430+0000 7ff5c88d0640 1 --2- 172.21.15.27:0/2825613666 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff5c415e650 0x7ff5c40b6bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:32.432 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:32.430+0000 7ff5c88d0640 1 --2- 172.21.15.27:0/2825613666 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff5c406db30 0x7ff5c40b6680 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:32.432 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:32.430+0000 7ff5c88d0640 1 --2- 172.21.15.27:0/2825613666 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ff5c406cc40 0x7ff5c41533a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:32.432 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:32.430+0000 7ff5c88d0640 1 -- 172.21.15.27:0/2825613666 >> 172.21.15.27:0/2825613666 conn(0x7ff5c406f320 msgr2=0x7ff5c4160c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:32.432 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:32.430+0000 7ff5c88d0640 1 -- 172.21.15.27:0/2825613666 shutdown_connections 2024-09-12T03:17:32.432 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:32.430+0000 7ff5c88d0640 1 -- 172.21.15.27:0/2825613666 wait complete. 2024-09-12T03:17:32.442 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestMisc.test_binary_metafile_on_legacy_to_v1_upgrade' 2024-09-12T03:17:32.552 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:32.551+0000 7f0a2732a640 1 Processor -- start 2024-09-12T03:17:32.552 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:32.551+0000 7f0a2732a640 1 -- start start 2024-09-12T03:17:32.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:32.552+0000 7f0a2732a640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f0a2006f2d0 0x7f0a2006f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:32.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:32.552+0000 7f0a2732a640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f0a200725b0 0x7f0a200cf250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:32.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:32.552+0000 7f0a2732a640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0a2006fbf0 con 0x7f0a2015c740 2024-09-12T03:17:32.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:32.552+0000 7f0a2732a640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f0a20072b90 con 0x7f0a200725b0 2024-09-12T03:17:32.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:32.552+0000 7f0a2732a640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f0a20072e70 con 0x7f0a2006f2d0 2024-09-12T03:17:32.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:32.552+0000 7f0a258c7640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f0a200725b0 0x7f0a200cf250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:32.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:32.552+0000 7f0a258c7640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f0a200725b0 0x7f0a200cf250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:48206/0 (socket says 172.21.15.27:48206) 2024-09-12T03:17:32.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:32.552+0000 7f0a258c7640 1 -- 172.21.15.27:0/1739560482 learned_addr learned my addr 172.21.15.27:0/1739560482 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:17:32.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:32.552+0000 7f0a250c6640 1 --2- 172.21.15.27:0/1739560482 >> v2:172.21.15.45:3300/0 conn(0x7f0a2006f2d0 0x7f0a2006f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:32.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:32.552+0000 7f0a258c7640 1 -- 172.21.15.27:0/1739560482 >> v2:172.21.15.45:3300/0 conn(0x7f0a2006f2d0 msgr2=0x7f0a2006f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:32.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:32.552+0000 7f0a258c7640 1 --2- 172.21.15.27:0/1739560482 >> v2:172.21.15.45:3300/0 conn(0x7f0a2006f2d0 0x7f0a2006f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:32.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:32.552+0000 7f0a258c7640 1 -- 172.21.15.27:0/1739560482 >> v1:172.21.15.45:6789/0 conn(0x7f0a2015c740 legacy=0x7f0a20071ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:32.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:32.552+0000 7f0a258c7640 1 -- 172.21.15.27:0/1739560482 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a200cea50 con 0x7f0a200725b0 2024-09-12T03:17:32.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:32.553+0000 7f0a258c7640 1 --2- 172.21.15.27:0/1739560482 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0a200725b0 0x7f0a200cf250 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f0a200b5ea0 tx=0x7f0a1802a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6c4e60c56a651e7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:32.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:32.553+0000 7f0a17fff640 1 -- 172.21.15.27:0/1739560482 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a18036070 con 0x7f0a200725b0 2024-09-12T03:17:32.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:32.553+0000 7f0a17fff640 1 -- 172.21.15.27:0/1739560482 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a18031600 con 0x7f0a200725b0 2024-09-12T03:17:32.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:32.553+0000 7f0a17fff640 1 -- 172.21.15.27:0/1739560482 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a180319d0 con 0x7f0a200725b0 2024-09-12T03:17:32.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:32.554+0000 7f0a2732a640 1 -- 172.21.15.27:0/1739560482 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0a200725b0 msgr2=0x7f0a200cf250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:32.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:32.554+0000 7f0a2732a640 1 --2- 172.21.15.27:0/1739560482 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0a200725b0 0x7f0a200cf250 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f0a200b5ea0 tx=0x7f0a1802a660 comp rx=0 tx=0).stop 2024-09-12T03:17:32.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:32.554+0000 7f0a2732a640 1 -- 172.21.15.27:0/1739560482 shutdown_connections 2024-09-12T03:17:32.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:32.554+0000 7f0a2732a640 1 --2- 172.21.15.27:0/1739560482 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0a200725b0 0x7f0a200cf250 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:32.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:32.554+0000 7f0a2732a640 1 --2- 172.21.15.27:0/1739560482 >> v2:172.21.15.45:3300/0 conn(0x7f0a2006f2d0 0x7f0a2006f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:32.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:32.554+0000 7f0a2732a640 1 -- 172.21.15.27:0/1739560482 >> 172.21.15.27:0/1739560482 conn(0x7f0a200b2500 msgr2=0x7f0a200b28e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:32.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:32.554+0000 7f0a2732a640 1 -- 172.21.15.27:0/1739560482 shutdown_connections 2024-09-12T03:17:32.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:32.554+0000 7f0a2732a640 1 -- 172.21.15.27:0/1739560482 wait complete. 2024-09-12T03:17:32.558 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:32.555+0000 7f0a2732a640 1 Processor -- start 2024-09-12T03:17:32.558 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:32.555+0000 7f0a2732a640 1 -- start start 2024-09-12T03:17:32.558 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:32.556+0000 7f0a2732a640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0a2006f2d0 0x7f0a2015f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:32.558 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:32.556+0000 7f0a2732a640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f0a200725b0 0x7f0a2015dc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:32.558 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:32.556+0000 7f0a2732a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0a2015c740 0x7f0a201f71f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:32.558 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:32.556+0000 7f0a2732a640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f0a200ce7f0 con 0x7f0a2006f2d0 2024-09-12T03:17:32.558 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:32.556+0000 7f0a2732a640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f0a200ce670 con 0x7f0a2015c740 2024-09-12T03:17:32.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:32.556+0000 7f0a2732a640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f0a200cf790 con 0x7f0a200725b0 2024-09-12T03:17:32.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:32.556+0000 7f0a250c6640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0a2006f2d0 0x7f0a2015f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:32.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:32.556+0000 7f0a250c6640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0a2006f2d0 0x7f0a2015f740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:48212/0 (socket says 172.21.15.27:48212) 2024-09-12T03:17:32.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:32.556+0000 7f0a250c6640 1 -- 172.21.15.27:0/4130264060 learned_addr learned my addr 172.21.15.27:0/4130264060 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:17:32.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:32.556+0000 7f0a258c7640 1 --2- 172.21.15.27:0/4130264060 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0a2015c740 0x7f0a201f71f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:32.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:32.556+0000 7f0a248c5640 1 --2- 172.21.15.27:0/4130264060 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f0a200725b0 0x7f0a2015dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:32.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:32.556+0000 7f0a250c6640 1 -- 172.21.15.27:0/4130264060 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f0a200725b0 msgr2=0x7f0a2015dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:32.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:32.556+0000 7f0a250c6640 1 --2- 172.21.15.27:0/4130264060 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f0a200725b0 0x7f0a2015dc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:32.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:32.556+0000 7f0a250c6640 1 -- 172.21.15.27:0/4130264060 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0a2015c740 msgr2=0x7f0a201f71f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:32.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:32.556+0000 7f0a250c6640 1 --2- 172.21.15.27:0/4130264060 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0a2015c740 0x7f0a201f71f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:32.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:32.556+0000 7f0a250c6640 1 -- 172.21.15.27:0/4130264060 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a2015e320 con 0x7f0a2006f2d0 2024-09-12T03:17:32.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:32.556+0000 7f0a258c7640 1 --2- 172.21.15.27:0/4130264060 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0a2015c740 0x7f0a201f71f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:17:32.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:32.556+0000 7f0a250c6640 1 --2- 172.21.15.27:0/4130264060 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0a2006f2d0 0x7f0a2015f740 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f0a1c00d800 tx=0x7f0a1c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:32.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:32.557+0000 7f0a15ffb640 1 -- 172.21.15.27:0/4130264060 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a1c00b6c0 con 0x7f0a2006f2d0 2024-09-12T03:17:32.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:32.557+0000 7f0a15ffb640 1 -- 172.21.15.27:0/4130264060 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a1c019070 con 0x7f0a2006f2d0 2024-09-12T03:17:32.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:32.557+0000 7f0a2732a640 1 -- 172.21.15.27:0/4130264060 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a2015e600 con 0x7f0a2006f2d0 2024-09-12T03:17:32.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:32.557+0000 7f0a15ffb640 1 -- 172.21.15.27:0/4130264060 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a1c00ba80 con 0x7f0a2006f2d0 2024-09-12T03:17:32.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:32.557+0000 7f0a2732a640 1 -- 172.21.15.27:0/4130264060 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a20152cd0 con 0x7f0a2006f2d0 2024-09-12T03:17:32.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:32.558+0000 7f0a2732a640 1 -- 172.21.15.27:0/4130264060 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a2006f6b0 con 0x7f0a2006f2d0 2024-09-12T03:17:32.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:32.558+0000 7f0a15ffb640 1 -- 172.21.15.27:0/4130264060 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 7) ==== 90651+0+0 (secure 0 0 0) 0x7f0a1c00bc20 con 0x7f0a2006f2d0 2024-09-12T03:17:32.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:32.559+0000 7f0a15ffb640 1 --2- 172.21.15.27:0/4130264060 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f0a0406bfe0 0x7f0a0406e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:32.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:32.559+0000 7f0a15ffb640 1 -- 172.21.15.27:0/4130264060 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 4943+0+0 (secure 0 0 0) 0x7f0a1c08b2d0 con 0x7f0a2006f2d0 2024-09-12T03:17:32.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:32.559+0000 7f0a248c5640 1 --2- 172.21.15.27:0/4130264060 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f0a0406bfe0 0x7f0a0406e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:32.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:32.559+0000 7f0a248c5640 1 --2- 172.21.15.27:0/4130264060 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f0a0406bfe0 0x7f0a0406e4a0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f0a10006fd0 tx=0x7f0a10008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:32.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:32.563+0000 7f0a15ffb640 1 -- 172.21.15.27:0/4130264060 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f0a1c059d90 con 0x7f0a2006f2d0 2024-09-12T03:17:32.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:32.807+0000 7f0a2732a640 1 -- 172.21.15.27:0/4130264060 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_binary_metafile_on_legacy_to_v1_upgrade"]} v 0) -- 0x7f0a200cf470 con 0x7f0a2006f2d0 2024-09-12T03:17:33.113 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.112+0000 7f0a15ffb640 1 -- 172.21.15.27:0/4130264060 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_binary_metafile_on_legacy_to_v1_upgrade"]}]=0 v79) ==== 158+0+0 (secure 0 0 0) 0x7f0a1c059690 con 0x7f0a2006f2d0 2024-09-12T03:17:33.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.114+0000 7f0a2732a640 1 -- 172.21.15.27:0/4130264060 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f0a0406bfe0 msgr2=0x7f0a0406e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:33.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.114+0000 7f0a2732a640 1 --2- 172.21.15.27:0/4130264060 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f0a0406bfe0 0x7f0a0406e4a0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f0a10006fd0 tx=0x7f0a10008040 comp rx=0 tx=0).stop 2024-09-12T03:17:33.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.114+0000 7f0a2732a640 1 -- 172.21.15.27:0/4130264060 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0a2006f2d0 msgr2=0x7f0a2015f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:33.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.114+0000 7f0a2732a640 1 --2- 172.21.15.27:0/4130264060 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0a2006f2d0 0x7f0a2015f740 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f0a1c00d800 tx=0x7f0a1c00dcc0 comp rx=0 tx=0).stop 2024-09-12T03:17:33.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.114+0000 7f0a2732a640 1 -- 172.21.15.27:0/4130264060 shutdown_connections 2024-09-12T03:17:33.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.114+0000 7f0a2732a640 1 --2- 172.21.15.27:0/4130264060 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f0a0406bfe0 0x7f0a0406e4a0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:33.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.114+0000 7f0a2732a640 1 --2- 172.21.15.27:0/4130264060 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0a2015c740 0x7f0a201f71f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:33.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.114+0000 7f0a2732a640 1 --2- 172.21.15.27:0/4130264060 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f0a200725b0 0x7f0a2015dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:33.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.114+0000 7f0a2732a640 1 --2- 172.21.15.27:0/4130264060 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0a2006f2d0 0x7f0a2015f740 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:33.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.114+0000 7f0a2732a640 1 -- 172.21.15.27:0/4130264060 >> 172.21.15.27:0/4130264060 conn(0x7f0a200b2500 msgr2=0x7f0a2006d580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:33.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.114+0000 7f0a2732a640 1 -- 172.21.15.27:0/4130264060 shutdown_connections 2024-09-12T03:17:33.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.114+0000 7f0a2732a640 1 -- 172.21.15.27:0/4130264060 wait complete. 2024-09-12T03:17:33.127 INFO:tasks.cephfs_test_runner:Validate binary .meta file on legacy subvol root doesn't break the system ... ok 2024-09-12T03:17:33.127 INFO:tasks.cephfs_test_runner:Starting test: test_connection_expiration (tasks.cephfs.test_volumes.TestMisc) 2024-09-12T03:17:33.128 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-12T03:17:33.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.229+0000 7f07f04d0640 1 Processor -- start 2024-09-12T03:17:33.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.229+0000 7f07f04d0640 1 -- start start 2024-09-12T03:17:33.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.230+0000 7f07f04d0640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f07e8073960 0x7f07e8162320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:33.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.230+0000 7f07f04d0640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f07e8162860 0x7f07e815e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:33.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.230+0000 7f07f04d0640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f07e80d2240 con 0x7f07e815e5f0 2024-09-12T03:17:33.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.230+0000 7f07f04d0640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f07e80d0da0 con 0x7f07e8162860 2024-09-12T03:17:33.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.230+0000 7f07f04d0640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f07e8163090 con 0x7f07e8073960 2024-09-12T03:17:33.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.230+0000 7f07eea6d640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f07e8162860 0x7f07e815e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:33.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.231+0000 7f07eea6d640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f07e8162860 0x7f07e815e9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:48216/0 (socket says 172.21.15.27:48216) 2024-09-12T03:17:33.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.231+0000 7f07eea6d640 1 -- 172.21.15.27:0/94015966 learned_addr learned my addr 172.21.15.27:0/94015966 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:17:33.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.231+0000 7f07eea6d640 1 -- 172.21.15.27:0/94015966 >> v2:172.21.15.45:3300/0 conn(0x7f07e8073960 msgr2=0x7f07e8162320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:33.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.231+0000 7f07eea6d640 1 --2- 172.21.15.27:0/94015966 >> v2:172.21.15.45:3300/0 conn(0x7f07e8073960 0x7f07e8162320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:33.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.231+0000 7f07eea6d640 1 -- 172.21.15.27:0/94015966 >> v1:172.21.15.45:6789/0 conn(0x7f07e815e5f0 legacy=0x7f07e8073250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:33.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.231+0000 7f07eea6d640 1 -- 172.21.15.27:0/94015966 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07e80d0470 con 0x7f07e8162860 2024-09-12T03:17:33.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.232+0000 7f07eea6d640 1 --2- 172.21.15.27:0/94015966 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f07e8162860 0x7f07e815e9d0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f07e8071520 tx=0x7f07e002a6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49494385acbc689 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:33.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.232+0000 7f07ed26a640 1 -- 172.21.15.27:0/94015966 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07e0036070 con 0x7f07e8162860 2024-09-12T03:17:33.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.232+0000 7f07ed26a640 1 -- 172.21.15.27:0/94015966 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07e0031600 con 0x7f07e8162860 2024-09-12T03:17:33.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.232+0000 7f07ed26a640 1 -- 172.21.15.27:0/94015966 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07e00319d0 con 0x7f07e8162860 2024-09-12T03:17:33.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.232+0000 7f07f04d0640 1 -- 172.21.15.27:0/94015966 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f07e8162860 msgr2=0x7f07e815e9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:33.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.232+0000 7f07f04d0640 1 --2- 172.21.15.27:0/94015966 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f07e8162860 0x7f07e815e9d0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f07e8071520 tx=0x7f07e002a6d0 comp rx=0 tx=0).stop 2024-09-12T03:17:33.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.232+0000 7f07f04d0640 1 -- 172.21.15.27:0/94015966 shutdown_connections 2024-09-12T03:17:33.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.232+0000 7f07f04d0640 1 --2- 172.21.15.27:0/94015966 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f07e8162860 0x7f07e815e9d0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:33.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.232+0000 7f07f04d0640 1 --2- 172.21.15.27:0/94015966 >> v2:172.21.15.45:3300/0 conn(0x7f07e8073960 0x7f07e8162320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:33.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.232+0000 7f07f04d0640 1 -- 172.21.15.27:0/94015966 >> 172.21.15.27:0/94015966 conn(0x7f07e806f300 msgr2=0x7f07e806f710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:33.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.232+0000 7f07f04d0640 1 -- 172.21.15.27:0/94015966 shutdown_connections 2024-09-12T03:17:33.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.232+0000 7f07f04d0640 1 -- 172.21.15.27:0/94015966 wait complete. 2024-09-12T03:17:33.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.233+0000 7f07f04d0640 1 Processor -- start 2024-09-12T03:17:33.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.234+0000 7f07f04d0640 1 -- start start 2024-09-12T03:17:33.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.234+0000 7f07f04d0640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f07e8073960 0x7f07e80ce860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:33.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.234+0000 7f07f04d0640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f07e815e5f0 0x7f07e81f4820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:33.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.234+0000 7f07f04d0640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f07e8162860 0x7f07e81f4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:33.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.234+0000 7f07f04d0640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f07e80b5620 con 0x7f07e815e5f0 2024-09-12T03:17:33.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.234+0000 7f07f04d0640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f07e80d0a30 con 0x7f07e8073960 2024-09-12T03:17:33.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.234+0000 7f07f04d0640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f07e8162c70 con 0x7f07e8162860 2024-09-12T03:17:33.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.234+0000 7f07eda6b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f07e815e5f0 0x7f07e81f4820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:33.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.234+0000 7f07eda6b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f07e815e5f0 0x7f07e81f4820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:48220/0 (socket says 172.21.15.27:48220) 2024-09-12T03:17:33.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.234+0000 7f07eda6b640 1 -- 172.21.15.27:0/1604761613 learned_addr learned my addr 172.21.15.27:0/1604761613 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:17:33.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.235+0000 7f07ee26c640 1 --2- 172.21.15.27:0/1604761613 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f07e8073960 0x7f07e80ce860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:33.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.235+0000 7f07eea6d640 1 --2- 172.21.15.27:0/1604761613 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f07e8162860 0x7f07e81f4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:33.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.235+0000 7f07eda6b640 1 -- 172.21.15.27:0/1604761613 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f07e8162860 msgr2=0x7f07e81f4d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:33.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.235+0000 7f07eda6b640 1 --2- 172.21.15.27:0/1604761613 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f07e8162860 0x7f07e81f4d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:33.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.235+0000 7f07eda6b640 1 -- 172.21.15.27:0/1604761613 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f07e8073960 msgr2=0x7f07e80ce860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:33.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.235+0000 7f07eda6b640 1 --2- 172.21.15.27:0/1604761613 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f07e8073960 0x7f07e80ce860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:33.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.235+0000 7f07eda6b640 1 -- 172.21.15.27:0/1604761613 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07e80cf0e0 con 0x7f07e815e5f0 2024-09-12T03:17:33.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.235+0000 7f07eea6d640 1 --2- 172.21.15.27:0/1604761613 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f07e8162860 0x7f07e81f4d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:17:33.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.235+0000 7f07eda6b640 1 --2- 172.21.15.27:0/1604761613 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f07e815e5f0 0x7f07e81f4820 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f07d800b4d0 tx=0x7f07d800b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:33.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.235+0000 7f07deffd640 1 -- 172.21.15.27:0/1604761613 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07d8014070 con 0x7f07e815e5f0 2024-09-12T03:17:33.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.235+0000 7f07deffd640 1 -- 172.21.15.27:0/1604761613 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07d8010610 con 0x7f07e815e5f0 2024-09-12T03:17:33.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.235+0000 7f07f04d0640 1 -- 172.21.15.27:0/1604761613 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07e80cf3c0 con 0x7f07e815e5f0 2024-09-12T03:17:33.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.235+0000 7f07deffd640 1 -- 172.21.15.27:0/1604761613 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07d8010a00 con 0x7f07e815e5f0 2024-09-12T03:17:33.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.235+0000 7f07f04d0640 1 -- 172.21.15.27:0/1604761613 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07e80ceda0 con 0x7f07e815e5f0 2024-09-12T03:17:33.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.236+0000 7f07f04d0640 1 -- 172.21.15.27:0/1604761613 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07e8162320 con 0x7f07e815e5f0 2024-09-12T03:17:33.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.237+0000 7f07deffd640 1 -- 172.21.15.27:0/1604761613 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 7) ==== 90651+0+0 (secure 0 0 0) 0x7f07d8017070 con 0x7f07e815e5f0 2024-09-12T03:17:33.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.237+0000 7f07deffd640 1 --2- 172.21.15.27:0/1604761613 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f07cc06bfe0 0x7f07cc06e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:33.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.237+0000 7f07deffd640 1 -- 172.21.15.27:0/1604761613 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 4943+0+0 (secure 0 0 0) 0x7f07d808a840 con 0x7f07e815e5f0 2024-09-12T03:17:33.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.237+0000 7f07ee26c640 1 --2- 172.21.15.27:0/1604761613 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f07cc06bfe0 0x7f07cc06e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:33.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.238+0000 7f07ee26c640 1 --2- 172.21.15.27:0/1604761613 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f07cc06bfe0 0x7f07cc06e4a0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f07e4007fd0 tx=0x7f07e4009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:33.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.242+0000 7f07deffd640 1 -- 172.21.15.27:0/1604761613 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f07d8059300 con 0x7f07e815e5f0 2024-09-12T03:17:33.419 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.418+0000 7f07f04d0640 1 -- 172.21.15.27:0/1604761613 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f07e80b6170 con 0x7f07e815e5f0 2024-09-12T03:17:33.420 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.419+0000 7f07deffd640 1 -- 172.21.15.27:0/1604761613 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) ==== 74+0+12984 (secure 0 0 0) 0x7f07d8058c00 con 0x7f07e815e5f0 2024-09-12T03:17:33.421 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:17:33.421 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":35,"fsid":"d2df504e-2647-4385-a5db-9fdf69f487a0","created":"2024-09-12T03:16:07.778937+0000","modified":"2024-09-12T03:17:32.108280+0000","last_up_change":"2024-09-12T03:16:19.314778+0000","last_in_change":"2024-09-12T03:16:09.893086+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T03:16:20.097646+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1851116c-1fb3-4dc9-9772-f90e2ef4c355","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6817","nonce":3002321488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6819","nonce":3002321488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6823","nonce":3002321488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6821","nonce":3002321488}]},"public_addr":"172.21.15.27:6817/3002321488","cluster_addr":"172.21.15.27:6819/3002321488","heartbeat_back_addr":"172.21.15.27:6823/3002321488","heartbeat_front_addr":"172.21.15.27:6821/3002321488","state":["exists","up"]},{"osd":1,"uuid":"f38ae17d-73eb-4204-b99d-975fa806fb87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6809","nonce":1613102886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6811","nonce":1613102886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6815","nonce":1613102886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6813","nonce":1613102886}]},"public_addr":"172.21.15.27:6809/1613102886","cluster_addr":"172.21.15.27:6811/1613102886","heartbeat_back_addr":"172.21.15.27:6815/1613102886","heartbeat_front_addr":"172.21.15.27:6813/1613102886","state":["exists","up"]},{"osd":2,"uuid":"fb12dcc8-bdd0-42b8-b46d-8b10106224bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6825","nonce":3534723355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6827","nonce":3534723355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6831","nonce":3534723355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6829","nonce":3534723355}]},"public_addr":"172.21.15.27:6825/3534723355","cluster_addr":"172.21.15.27:6827/3534723355","heartbeat_back_addr":"172.21.15.27:6831/3534723355","heartbeat_front_addr":"172.21.15.27:6829/3534723355","state":["exists","up"]},{"osd":3,"uuid":"d74d3f49-9c24-4b9f-98e2-3ecd7b5ef3b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6801","nonce":80661598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6803","nonce":80661598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6807","nonce":80661598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6805","nonce":80661598}]},"public_addr":"172.21.15.27:6801/80661598","cluster_addr":"172.21.15.27:6803/80661598","heartbeat_back_addr":"172.21.15.27:6807/80661598","heartbeat_front_addr":"172.21.15.27:6805/80661598","state":["exists","up"]},{"osd":4,"uuid":"2bf123f2-1b0d-41c4-aa3b-805bab267e59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6817","nonce":3943189827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6819","nonce":3943189827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6823","nonce":3943189827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6821","nonce":3943189827}]},"public_addr":"172.21.15.45:6817/3943189827","cluster_addr":"172.21.15.45:6819/3943189827","heartbeat_back_addr":"172.21.15.45:6823/3943189827","heartbeat_front_addr":"172.21.15.45:6821/3943189827","state":["exists","up"]},{"osd":5,"uuid":"c49ba5da-91c6-4a86-a97b-4aba89c253ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6809","nonce":3620760330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6811","nonce":3620760330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6815","nonce":3620760330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6813","nonce":3620760330}]},"public_addr":"172.21.15.45:6809/3620760330","cluster_addr":"172.21.15.45:6811/3620760330","heartbeat_back_addr":"172.21.15.45:6815/3620760330","heartbeat_front_addr":"172.21.15.45:6813/3620760330","state":["exists","up"]},{"osd":6,"uuid":"b0123475-89f7-4a34-9e33-d797b99dd9dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6801","nonce":1989083510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6803","nonce":1989083510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6807","nonce":1989083510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6805","nonce":1989083510}]},"public_addr":"172.21.15.45:6801/1989083510","cluster_addr":"172.21.15.45:6803/1989083510","heartbeat_back_addr":"172.21.15.45:6807/1989083510","heartbeat_front_addr":"172.21.15.45:6805/1989083510","state":["exists","up"]},{"osd":7,"uuid":"4dad773d-b7d8-46fc-8a10-a972aad27b29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6825","nonce":1513300294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6827","nonce":1513300294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6831","nonce":1513300294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6829","nonce":1513300294}]},"public_addr":"172.21.15.45:6825/1513300294","cluster_addr":"172.21.15.45:6827/1513300294","heartbeat_back_addr":"172.21.15.45:6831/1513300294","heartbeat_front_addr":"172.21.15.45:6829/1513300294","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.881855+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:17.133252+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.735339+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.500742+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.477571+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:17.167667+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.470291+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.873743+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.27:6835/3533575930":"2024-09-13T03:17:29.099454+0000","172.21.15.27:6833/3533575930":"2024-09-13T03:17:29.099454+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T03:17:33.422 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.421+0000 7f07f04d0640 1 -- 172.21.15.27:0/1604761613 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f07cc06bfe0 msgr2=0x7f07cc06e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:33.422 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.421+0000 7f07f04d0640 1 --2- 172.21.15.27:0/1604761613 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f07cc06bfe0 0x7f07cc06e4a0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f07e4007fd0 tx=0x7f07e4009040 comp rx=0 tx=0).stop 2024-09-12T03:17:33.423 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.421+0000 7f07f04d0640 1 -- 172.21.15.27:0/1604761613 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f07e815e5f0 msgr2=0x7f07e81f4820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:33.423 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.421+0000 7f07f04d0640 1 --2- 172.21.15.27:0/1604761613 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f07e815e5f0 0x7f07e81f4820 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f07d800b4d0 tx=0x7f07d800b990 comp rx=0 tx=0).stop 2024-09-12T03:17:33.423 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.421+0000 7f07f04d0640 1 -- 172.21.15.27:0/1604761613 shutdown_connections 2024-09-12T03:17:33.423 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.421+0000 7f07f04d0640 1 --2- 172.21.15.27:0/1604761613 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f07cc06bfe0 0x7f07cc06e4a0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:33.423 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.421+0000 7f07f04d0640 1 --2- 172.21.15.27:0/1604761613 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f07e8162860 0x7f07e81f4d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:33.423 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.421+0000 7f07f04d0640 1 --2- 172.21.15.27:0/1604761613 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f07e815e5f0 0x7f07e81f4820 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:33.423 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.421+0000 7f07f04d0640 1 --2- 172.21.15.27:0/1604761613 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f07e8073960 0x7f07e80ce860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:33.423 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.421+0000 7f07f04d0640 1 -- 172.21.15.27:0/1604761613 >> 172.21.15.27:0/1604761613 conn(0x7f07e806f300 msgr2=0x7f07e81613d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:33.423 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.421+0000 7f07f04d0640 1 -- 172.21.15.27:0/1604761613 shutdown_connections 2024-09-12T03:17:33.423 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.421+0000 7f07f04d0640 1 -- 172.21.15.27:0/1604761613 wait complete. 2024-09-12T03:17:33.433 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-12T03:16:20.097646+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-12T03:17:33.434 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-12T03:17:33.539 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.538+0000 7f92fc47b640 1 Processor -- start 2024-09-12T03:17:33.539 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.538+0000 7f92fc47b640 1 -- start start 2024-09-12T03:17:33.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.539+0000 7f92fc47b640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f92f415ec30 0x7f92f40b3900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:33.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.539+0000 7f92fc47b640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f92f40ce5e0 con 0x7f92f40af9d0 2024-09-12T03:17:33.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.539+0000 7f92fc47b640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f92f40ceac0 con 0x7f92f406da00 2024-09-12T03:17:33.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.539+0000 7f92fc47b640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f92f40d02d0 con 0x7f92f415ec30 2024-09-12T03:17:33.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.539+0000 7f92f9a16640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f92f40af9d0 0x7f92f40b3e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:46590/0 (socket says 172.21.15.27:46590) 2024-09-12T03:17:33.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.539+0000 7f92f9a16640 1 -- 172.21.15.27:0/3973500303 learned_addr learned my addr 172.21.15.27:0/3973500303 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:17:33.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.539+0000 7f92fa217640 1 --2- 172.21.15.27:0/3973500303 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f92f415ec30 0x7f92f40b3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:33.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.539+0000 7f92f9215640 1 -- 172.21.15.27:0/3973500303 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1751568275 0 0) 0x7f92f40ce5e0 con 0x7f92f40af9d0 2024-09-12T03:17:33.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.539+0000 7f92f9215640 1 -- 172.21.15.27:0/3973500303 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f92dc003520 con 0x7f92f40af9d0 2024-09-12T03:17:33.543 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.540+0000 7f92f9215640 1 -- 172.21.15.27:0/3973500303 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 184096509 0 0) 0x7f92dc003520 con 0x7f92f40af9d0 2024-09-12T03:17:33.543 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.540+0000 7f92f9215640 1 -- 172.21.15.27:0/3973500303 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f92f415ec30 msgr2=0x7f92f40b3900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:33.543 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.540+0000 7f92f9215640 1 --2- 172.21.15.27:0/3973500303 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f92f415ec30 0x7f92f40b3900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:33.543 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.540+0000 7f92f9215640 1 -- 172.21.15.27:0/3973500303 >> v1:172.21.15.45:6789/0 conn(0x7f92f406da00 legacy=0x7f92f406deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:33.543 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.540+0000 7f92f9215640 1 -- 172.21.15.27:0/3973500303 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92f415f010 con 0x7f92f40af9d0 2024-09-12T03:17:33.543 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.540+0000 7f92f9215640 1 -- 172.21.15.27:0/3973500303 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 847755213 0 0) 0x7f92e8001670 con 0x7f92f40af9d0 2024-09-12T03:17:33.543 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.540+0000 7f92f9215640 1 -- 172.21.15.27:0/3973500303 >> v1:172.21.15.27:6789/0 conn(0x7f92f40af9d0 legacy=0x7f92f40b3e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:33.543 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.540+0000 7f92f9215640 1 --2- 172.21.15.27:0/3973500303 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f92f415ec30 0x7f92dc0039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:33.543 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.540+0000 7f92f9215640 1 --2- 172.21.15.27:0/3973500303 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f92dc003f20 0x7f92dc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:33.543 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.540+0000 7f92f9215640 1 --2- 172.21.15.27:0/3973500303 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f92dc004d40 0x7f92dc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:33.543 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.540+0000 7f92f9215640 1 -- 172.21.15.27:0/3973500303 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f92f40d02d0 con 0x7f92dc004d40 2024-09-12T03:17:33.543 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.540+0000 7f92f9215640 1 -- 172.21.15.27:0/3973500303 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f92dc00ba20 con 0x7f92dc003f20 2024-09-12T03:17:33.543 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.540+0000 7f92f9215640 1 -- 172.21.15.27:0/3973500303 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f92dc00de30 con 0x7f92f415ec30 2024-09-12T03:17:33.543 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.540+0000 7f92faa18640 1 --2- 172.21.15.27:0/3973500303 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f92dc004d40 0x7f92dc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:33.544 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.540+0000 7f92fa217640 1 --2- 172.21.15.27:0/3973500303 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f92f415ec30 0x7f92dc0039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:33.544 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.540+0000 7f92f9a16640 1 --2- 172.21.15.27:0/3973500303 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f92dc003f20 0x7f92dc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:33.544 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.540+0000 7f92faa18640 1 -- 172.21.15.27:0/3973500303 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f92f415ec30 msgr2=0x7f92dc0039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:33.544 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.540+0000 7f92faa18640 1 --2- 172.21.15.27:0/3973500303 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f92f415ec30 0x7f92dc0039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:33.544 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.540+0000 7f92faa18640 1 -- 172.21.15.27:0/3973500303 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f92dc003f20 msgr2=0x7f92dc004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:33.544 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.540+0000 7f92faa18640 1 --2- 172.21.15.27:0/3973500303 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f92dc003f20 0x7f92dc004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:33.544 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.540+0000 7f92faa18640 1 -- 172.21.15.27:0/3973500303 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92dc003520 con 0x7f92dc004d40 2024-09-12T03:17:33.544 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.541+0000 7f92faa18640 1 --2- 172.21.15.27:0/3973500303 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f92dc004d40 0x7f92dc0051c0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f92e4001f60 tx=0x7f92dc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d85ef5b3a3319077 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:33.544 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.541+0000 7f92f9215640 1 -- 172.21.15.27:0/3973500303 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92e4001b30 con 0x7f92dc004d40 2024-09-12T03:17:33.544 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.541+0000 7f92f9215640 1 -- 172.21.15.27:0/3973500303 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92e4031620 con 0x7f92dc004d40 2024-09-12T03:17:33.544 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.541+0000 7f92f9215640 1 -- 172.21.15.27:0/3973500303 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92e4031a10 con 0x7f92dc004d40 2024-09-12T03:17:33.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.542+0000 7f92fc47b640 1 -- 172.21.15.27:0/3973500303 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f92dc004d40 msgr2=0x7f92dc0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:33.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.542+0000 7f92fc47b640 1 --2- 172.21.15.27:0/3973500303 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f92dc004d40 0x7f92dc0051c0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f92e4001f60 tx=0x7f92dc005830 comp rx=0 tx=0).stop 2024-09-12T03:17:33.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.542+0000 7f92faa18640 1 -- 172.21.15.27:0/3973500303 reap_dead start 2024-09-12T03:17:33.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.542+0000 7f92fc47b640 1 -- 172.21.15.27:0/3973500303 shutdown_connections 2024-09-12T03:17:33.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.542+0000 7f92fc47b640 1 -- 172.21.15.27:0/3973500303 >> 172.21.15.27:0/3973500303 conn(0x7f92f406bb30 msgr2=0x7f92f406bf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:33.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.542+0000 7f92fc47b640 1 -- 172.21.15.27:0/3973500303 shutdown_connections 2024-09-12T03:17:33.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.542+0000 7f92fc47b640 1 -- 172.21.15.27:0/3973500303 wait complete. 2024-09-12T03:17:33.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.543+0000 7f92fc47b640 1 Processor -- start 2024-09-12T03:17:33.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.543+0000 7f92fc47b640 1 -- start start 2024-09-12T03:17:33.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.544+0000 7f92fc47b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f92dc004d40 0x7f92f415ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:33.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.544+0000 7f92fc47b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f92f415ddf0 0x7f92f4151fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:33.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.544+0000 7f92fc47b640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f92f4071dc0 0x7f92f4072270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:33.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.544+0000 7f92fc47b640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f92f40cfff0 con 0x7f92dc004d40 2024-09-12T03:17:33.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.544+0000 7f92fc47b640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f92f406c650 con 0x7f92f415ddf0 2024-09-12T03:17:33.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.544+0000 7f92fc47b640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f92f40715f0 con 0x7f92f4071dc0 2024-09-12T03:17:33.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.544+0000 7f92fa217640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f92dc004d40 0x7f92f415ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:33.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.544+0000 7f92fa217640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f92dc004d40 0x7f92f415ade0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:48224/0 (socket says 172.21.15.27:48224) 2024-09-12T03:17:33.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.544+0000 7f92fa217640 1 -- 172.21.15.27:0/3020269121 learned_addr learned my addr 172.21.15.27:0/3020269121 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:17:33.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.544+0000 7f92f9a16640 1 --2- 172.21.15.27:0/3020269121 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f92f415ddf0 0x7f92f4151fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:33.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.544+0000 7f92faa18640 1 --2- 172.21.15.27:0/3020269121 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f92f4071dc0 0x7f92f4072270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:33.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.544+0000 7f92fa217640 1 -- 172.21.15.27:0/3020269121 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f92f4071dc0 msgr2=0x7f92f4072270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:33.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.544+0000 7f92fa217640 1 --2- 172.21.15.27:0/3020269121 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f92f4071dc0 0x7f92f4072270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:33.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.544+0000 7f92fa217640 1 -- 172.21.15.27:0/3020269121 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f92f415ddf0 msgr2=0x7f92f4151fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:33.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.544+0000 7f92fa217640 1 --2- 172.21.15.27:0/3020269121 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f92f415ddf0 0x7f92f4151fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:33.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.544+0000 7f92fa217640 1 -- 172.21.15.27:0/3020269121 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92f415e270 con 0x7f92dc004d40 2024-09-12T03:17:33.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.544+0000 7f92fa217640 1 --2- 172.21.15.27:0/3020269121 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f92dc004d40 0x7f92f415ade0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f92ec0119a0 tx=0x7f92ec011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:33.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.544+0000 7f92f2ffd640 1 -- 172.21.15.27:0/3020269121 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92ec00faf0 con 0x7f92dc004d40 2024-09-12T03:17:33.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.544+0000 7f92f2ffd640 1 -- 172.21.15.27:0/3020269121 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92ec00fc90 con 0x7f92dc004d40 2024-09-12T03:17:33.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.544+0000 7f92fc47b640 1 -- 172.21.15.27:0/3020269121 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92f415b890 con 0x7f92dc004d40 2024-09-12T03:17:33.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.544+0000 7f92f2ffd640 1 -- 172.21.15.27:0/3020269121 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92ec014440 con 0x7f92dc004d40 2024-09-12T03:17:33.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.545+0000 7f92fc47b640 1 -- 172.21.15.27:0/3020269121 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92f41538e0 con 0x7f92dc004d40 2024-09-12T03:17:33.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.545+0000 7f92fc47b640 1 -- 172.21.15.27:0/3020269121 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92f415bdb0 con 0x7f92dc004d40 2024-09-12T03:17:33.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.546+0000 7f92f2ffd640 1 -- 172.21.15.27:0/3020269121 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 7) ==== 90651+0+0 (secure 0 0 0) 0x7f92ec012040 con 0x7f92dc004d40 2024-09-12T03:17:33.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.546+0000 7f92f2ffd640 1 --2- 172.21.15.27:0/3020269121 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f92e006c010 0x7f92e006e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:33.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.547+0000 7f92f2ffd640 1 -- 172.21.15.27:0/3020269121 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 4943+0+0 (secure 0 0 0) 0x7f92ec08b430 con 0x7f92dc004d40 2024-09-12T03:17:33.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.547+0000 7f92f9a16640 1 --2- 172.21.15.27:0/3020269121 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f92e006c010 0x7f92e006e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:33.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.547+0000 7f92f9a16640 1 --2- 172.21.15.27:0/3020269121 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f92e006c010 0x7f92e006e4d0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f92f40ae9b0 tx=0x7f92e8009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:33.551 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.551+0000 7f92f2ffd640 1 -- 172.21.15.27:0/3020269121 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f92f415bdb0 con 0x7f92dc004d40 2024-09-12T03:17:33.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.723+0000 7f92fc47b640 1 -- 172.21.15.27:0/3020269121 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f92f406deb0 con 0x7f92dc004d40 2024-09-12T03:17:33.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.723+0000 7f92f2ffd640 1 -- 172.21.15.27:0/3020269121 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v35) ==== 93+0+10 (secure 0 0 0) 0x7f92ec019030 con 0x7f92dc004d40 2024-09-12T03:17:33.725 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 1 2024-09-12T03:17:33.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.726+0000 7f92fc47b640 1 -- 172.21.15.27:0/3020269121 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f92e006c010 msgr2=0x7f92e006e4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:33.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.726+0000 7f92fc47b640 1 --2- 172.21.15.27:0/3020269121 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f92e006c010 0x7f92e006e4d0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f92f40ae9b0 tx=0x7f92e8009040 comp rx=0 tx=0).stop 2024-09-12T03:17:33.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.726+0000 7f92fc47b640 1 -- 172.21.15.27:0/3020269121 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f92dc004d40 msgr2=0x7f92f415ade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:33.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.726+0000 7f92fc47b640 1 --2- 172.21.15.27:0/3020269121 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f92dc004d40 0x7f92f415ade0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f92ec0119a0 tx=0x7f92ec011e60 comp rx=0 tx=0).stop 2024-09-12T03:17:33.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.726+0000 7f92fc47b640 1 -- 172.21.15.27:0/3020269121 shutdown_connections 2024-09-12T03:17:33.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.726+0000 7f92fc47b640 1 --2- 172.21.15.27:0/3020269121 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f92e006c010 0x7f92e006e4d0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:33.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.726+0000 7f92fc47b640 1 --2- 172.21.15.27:0/3020269121 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f92f4071dc0 0x7f92f4072270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:33.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.726+0000 7f92fc47b640 1 --2- 172.21.15.27:0/3020269121 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f92f415ddf0 0x7f92f4151fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:33.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.726+0000 7f92fc47b640 1 --2- 172.21.15.27:0/3020269121 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f92dc004d40 0x7f92f415ade0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:33.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.726+0000 7f92fc47b640 1 -- 172.21.15.27:0/3020269121 >> 172.21.15.27:0/3020269121 conn(0x7f92f406bb30 msgr2=0x7f92f406bf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:33.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.726+0000 7f92fc47b640 1 -- 172.21.15.27:0/3020269121 shutdown_connections 2024-09-12T03:17:33.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.726+0000 7f92fc47b640 1 -- 172.21.15.27:0/3020269121 wait complete. 2024-09-12T03:17:33.738 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestMisc.test_connection_expiration' 2024-09-12T03:17:33.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.839+0000 7f89d02d7640 1 Processor -- start 2024-09-12T03:17:33.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.839+0000 7f89d02d7640 1 -- start start 2024-09-12T03:17:33.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.840+0000 7f89d02d7640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f89c815c460 0x7f89c80b55b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:33.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.840+0000 7f89d02d7640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f89c80b69b0 0x7f89c80d2360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:33.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.840+0000 7f89d02d7640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f89c80b5f00 con 0x7f89c80d28a0 2024-09-12T03:17:33.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.840+0000 7f89d02d7640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f89c80d2d10 con 0x7f89c815c460 2024-09-12T03:17:33.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.840+0000 7f89d02d7640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f89c80d00d0 con 0x7f89c80b69b0 2024-09-12T03:17:33.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.840+0000 7f89ce874640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f89c80d28a0 0x7f89c80731c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:46598/0 (socket says 172.21.15.27:46598) 2024-09-12T03:17:33.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.840+0000 7f89ce874640 1 -- 172.21.15.27:0/273621927 learned_addr learned my addr 172.21.15.27:0/273621927 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:17:33.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.841+0000 7f89ce073640 1 --2- 172.21.15.27:0/273621927 >> v2:172.21.15.45:3300/0 conn(0x7f89c815c460 0x7f89c80b55b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:33.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.841+0000 7f89cd872640 1 --2- 172.21.15.27:0/273621927 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f89c80b69b0 0x7f89c80d2360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:33.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.841+0000 7f89cd071640 1 -- 172.21.15.27:0/273621927 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1904767705 0 0) 0x7f89c80b5f00 con 0x7f89c80d28a0 2024-09-12T03:17:33.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.841+0000 7f89cd071640 1 -- 172.21.15.27:0/273621927 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f89b0003520 con 0x7f89c80d28a0 2024-09-12T03:17:33.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.841+0000 7f89ce073640 1 -- 172.21.15.27:0/273621927 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f89c80b69b0 msgr2=0x7f89c80d2360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:33.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.841+0000 7f89ce073640 1 --2- 172.21.15.27:0/273621927 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f89c80b69b0 0x7f89c80d2360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:33.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.841+0000 7f89ce073640 1 -- 172.21.15.27:0/273621927 >> v1:172.21.15.27:6789/0 conn(0x7f89c80d28a0 legacy=0x7f89c80731c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:33.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.841+0000 7f89ce073640 1 -- 172.21.15.27:0/273621927 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89c80b61e0 con 0x7f89c815c460 2024-09-12T03:17:33.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.841+0000 7f89cd872640 1 --2- 172.21.15.27:0/273621927 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f89c80b69b0 0x7f89c80d2360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:17:33.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.842+0000 7f89ce073640 1 --2- 172.21.15.27:0/273621927 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f89c815c460 0x7f89c80b55b0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f89c0009d90 tx=0x7f89c002a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a1df1cebf64dd38 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:33.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.842+0000 7f89cd071640 1 -- 172.21.15.27:0/273621927 <== mon.1 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89c0007740 con 0x7f89c815c460 2024-09-12T03:17:33.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.842+0000 7f89cd071640 1 -- 172.21.15.27:0/273621927 <== mon.1 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89c00078e0 con 0x7f89c815c460 2024-09-12T03:17:33.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.842+0000 7f89cd071640 1 -- 172.21.15.27:0/273621927 <== mon.1 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89c0007c90 con 0x7f89c815c460 2024-09-12T03:17:33.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.842+0000 7f89d02d7640 1 -- 172.21.15.27:0/273621927 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f89c815c460 msgr2=0x7f89c80b55b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:33.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.842+0000 7f89d02d7640 1 --2- 172.21.15.27:0/273621927 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f89c815c460 0x7f89c80b55b0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f89c0009d90 tx=0x7f89c002a660 comp rx=0 tx=0).stop 2024-09-12T03:17:33.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.843+0000 7f89d02d7640 1 -- 172.21.15.27:0/273621927 shutdown_connections 2024-09-12T03:17:33.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.843+0000 7f89d02d7640 1 --2- 172.21.15.27:0/273621927 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f89c80b69b0 0x7f89c80d2360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:33.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.843+0000 7f89d02d7640 1 --2- 172.21.15.27:0/273621927 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f89c815c460 0x7f89c80b55b0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:33.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.843+0000 7f89d02d7640 1 -- 172.21.15.27:0/273621927 >> 172.21.15.27:0/273621927 conn(0x7f89c806f320 msgr2=0x7f89c806f700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:33.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.843+0000 7f89d02d7640 1 -- 172.21.15.27:0/273621927 shutdown_connections 2024-09-12T03:17:33.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.843+0000 7f89d02d7640 1 -- 172.21.15.27:0/273621927 wait complete. 2024-09-12T03:17:33.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.844+0000 7f89d02d7640 1 Processor -- start 2024-09-12T03:17:33.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.844+0000 7f89d02d7640 1 -- start start 2024-09-12T03:17:33.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.845+0000 7f89d02d7640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f89c80b69b0 0x7f89c814cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:33.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.845+0000 7f89d02d7640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f89c80d28a0 0x7f89c80afbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:33.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.845+0000 7f89d02d7640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f89c815c460 0x7f89c80ae940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:33.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.845+0000 7f89d02d7640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f89c80d0670 con 0x7f89c815c460 2024-09-12T03:17:33.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.845+0000 7f89d02d7640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f89c80b64c0 con 0x7f89c80b69b0 2024-09-12T03:17:33.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.845+0000 7f89d02d7640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f89c80b5af0 con 0x7f89c80d28a0 2024-09-12T03:17:33.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.845+0000 7f89ce874640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f89c815c460 0x7f89c80ae940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:33.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.845+0000 7f89ce874640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f89c815c460 0x7f89c80ae940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:48236/0 (socket says 172.21.15.27:48236) 2024-09-12T03:17:33.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.845+0000 7f89ce874640 1 -- 172.21.15.27:0/2895407330 learned_addr learned my addr 172.21.15.27:0/2895407330 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:17:33.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.845+0000 7f89cd872640 1 --2- 172.21.15.27:0/2895407330 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f89c80d28a0 0x7f89c80afbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:33.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.845+0000 7f89ce073640 1 --2- 172.21.15.27:0/2895407330 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f89c80b69b0 0x7f89c814cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:33.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.845+0000 7f89ce874640 1 -- 172.21.15.27:0/2895407330 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f89c80d28a0 msgr2=0x7f89c80afbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:33.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.845+0000 7f89ce874640 1 --2- 172.21.15.27:0/2895407330 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f89c80d28a0 0x7f89c80afbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:33.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.845+0000 7f89ce874640 1 -- 172.21.15.27:0/2895407330 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f89c80b69b0 msgr2=0x7f89c814cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:33.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.845+0000 7f89ce874640 1 --2- 172.21.15.27:0/2895407330 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f89c80b69b0 0x7f89c814cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:33.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.845+0000 7f89ce874640 1 -- 172.21.15.27:0/2895407330 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89c814d160 con 0x7f89c815c460 2024-09-12T03:17:33.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.845+0000 7f89ce874640 1 --2- 172.21.15.27:0/2895407330 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f89c815c460 0x7f89c80ae940 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f89b800c430 tx=0x7f89b800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:33.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.845+0000 7f89c6ffd640 1 -- 172.21.15.27:0/2895407330 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89b8017070 con 0x7f89c815c460 2024-09-12T03:17:33.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.845+0000 7f89c6ffd640 1 -- 172.21.15.27:0/2895407330 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89b8005ce0 con 0x7f89c815c460 2024-09-12T03:17:33.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.845+0000 7f89c6ffd640 1 -- 172.21.15.27:0/2895407330 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89b8007310 con 0x7f89c815c460 2024-09-12T03:17:33.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.845+0000 7f89d02d7640 1 -- 172.21.15.27:0/2895407330 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89c80b19e0 con 0x7f89c815c460 2024-09-12T03:17:33.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.845+0000 7f89d02d7640 1 -- 172.21.15.27:0/2895407330 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89c806d270 con 0x7f89c815c460 2024-09-12T03:17:33.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.846+0000 7f89d02d7640 1 -- 172.21.15.27:0/2895407330 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89c815c8d0 con 0x7f89c815c460 2024-09-12T03:17:33.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.847+0000 7f89c6ffd640 1 -- 172.21.15.27:0/2895407330 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 7) ==== 90651+0+0 (secure 0 0 0) 0x7f89b801a070 con 0x7f89c815c460 2024-09-12T03:17:33.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.847+0000 7f89c6ffd640 1 --2- 172.21.15.27:0/2895407330 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f89b406bfe0 0x7f89b406e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:33.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.848+0000 7f89c6ffd640 1 -- 172.21.15.27:0/2895407330 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 4943+0+0 (secure 0 0 0) 0x7f89b808a220 con 0x7f89c815c460 2024-09-12T03:17:33.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.848+0000 7f89ce073640 1 --2- 172.21.15.27:0/2895407330 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f89b406bfe0 0x7f89b406e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:33.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.848+0000 7f89ce073640 1 --2- 172.21.15.27:0/2895407330 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f89b406bfe0 0x7f89b406e4a0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f89c0034000 tx=0x7f89c002f000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:33.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:33.852+0000 7f89c6ffd640 1 -- 172.21.15.27:0/2895407330 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f89b8058ce0 con 0x7f89c815c460 2024-09-12T03:17:34.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:34.094+0000 7f89d02d7640 1 -- 172.21.15.27:0/2895407330 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_connection_expiration"]} v 0) -- 0x7f89c80d23c0 con 0x7f89c815c460 2024-09-12T03:17:34.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:34.117+0000 7f89c6ffd640 1 -- 172.21.15.27:0/2895407330 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_connection_expiration"]}]=0 v80) ==== 143+0+0 (secure 0 0 0) 0x7f89b80585e0 con 0x7f89c815c460 2024-09-12T03:17:34.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:34.119+0000 7f89d02d7640 1 -- 172.21.15.27:0/2895407330 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f89b406bfe0 msgr2=0x7f89b406e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:34.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:34.119+0000 7f89d02d7640 1 --2- 172.21.15.27:0/2895407330 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f89b406bfe0 0x7f89b406e4a0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f89c0034000 tx=0x7f89c002f000 comp rx=0 tx=0).stop 2024-09-12T03:17:34.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:34.119+0000 7f89d02d7640 1 -- 172.21.15.27:0/2895407330 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f89c815c460 msgr2=0x7f89c80ae940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:34.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:34.119+0000 7f89d02d7640 1 --2- 172.21.15.27:0/2895407330 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f89c815c460 0x7f89c80ae940 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f89b800c430 tx=0x7f89b800c8f0 comp rx=0 tx=0).stop 2024-09-12T03:17:34.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:34.119+0000 7f89d02d7640 1 -- 172.21.15.27:0/2895407330 shutdown_connections 2024-09-12T03:17:34.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:34.119+0000 7f89d02d7640 1 --2- 172.21.15.27:0/2895407330 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f89b406bfe0 0x7f89b406e4a0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:34.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:34.119+0000 7f89d02d7640 1 --2- 172.21.15.27:0/2895407330 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f89c815c460 0x7f89c80ae940 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:34.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:34.119+0000 7f89d02d7640 1 --2- 172.21.15.27:0/2895407330 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f89c80d28a0 0x7f89c80afbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:34.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:34.119+0000 7f89d02d7640 1 --2- 172.21.15.27:0/2895407330 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f89c80b69b0 0x7f89c814cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:34.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:34.119+0000 7f89d02d7640 1 -- 172.21.15.27:0/2895407330 >> 172.21.15.27:0/2895407330 conn(0x7f89c806f320 msgr2=0x7f89c815eaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:34.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:34.119+0000 7f89d02d7640 1 -- 172.21.15.27:0/2895407330 shutdown_connections 2024-09-12T03:17:34.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:34.119+0000 7f89d02d7640 1 -- 172.21.15.27:0/2895407330 wait complete. 2024-09-12T03:17:34.131 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-12T03:17:34.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:34.235+0000 7f0b1f3cd640 1 Processor -- start 2024-09-12T03:17:34.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:34.235+0000 7f0b1f3cd640 1 -- start start 2024-09-12T03:17:34.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:34.236+0000 7f0b1f3cd640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f0b180d3c20 0x7f0b180d4040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:34.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:34.236+0000 7f0b1f3cd640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f0b180b77c0 0x7f0b180b7c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:34.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:34.236+0000 7f0b1f3cd640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0b180d2930 con 0x7f0b180b6240 2024-09-12T03:17:34.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:34.236+0000 7f0b1f3cd640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f0b180d2c10 con 0x7f0b180b77c0 2024-09-12T03:17:34.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:34.236+0000 7f0b1f3cd640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f0b180d2ef0 con 0x7f0b180d3c20 2024-09-12T03:17:34.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:34.236+0000 7f0b1d96a640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f0b180b77c0 0x7f0b180b7c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:34.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:34.236+0000 7f0b1d96a640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f0b180b77c0 0x7f0b180b7c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:48250/0 (socket says 172.21.15.27:48250) 2024-09-12T03:17:34.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:34.236+0000 7f0b1d96a640 1 -- 172.21.15.27:0/1335018139 learned_addr learned my addr 172.21.15.27:0/1335018139 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:17:34.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:34.237+0000 7f0b1c968640 1 --2- 172.21.15.27:0/1335018139 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f0b180d3c20 0x7f0b180d4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:34.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:34.237+0000 7f0b1d96a640 1 -- 172.21.15.27:0/1335018139 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f0b180d3c20 msgr2=0x7f0b180d4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:34.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:34.237+0000 7f0b1d96a640 1 --2- 172.21.15.27:0/1335018139 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f0b180d3c20 0x7f0b180d4040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:34.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:34.237+0000 7f0b1d96a640 1 -- 172.21.15.27:0/1335018139 >> v1:172.21.15.45:6789/0 conn(0x7f0b180b6240 legacy=0x7f0b180b6620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:34.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:34.237+0000 7f0b1d96a640 1 -- 172.21.15.27:0/1335018139 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b180d31d0 con 0x7f0b180b77c0 2024-09-12T03:17:34.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:34.237+0000 7f0b1c968640 1 --2- 172.21.15.27:0/1335018139 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f0b180d3c20 0x7f0b180d4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:17:34.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:34.237+0000 7f0b1d96a640 1 --2- 172.21.15.27:0/1335018139 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0b180b77c0 0x7f0b180b7c70 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f0b100098a0 tx=0x7f0b1002a2e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e1f91867c999d95 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:34.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:34.237+0000 7f0b17fff640 1 -- 172.21.15.27:0/1335018139 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b10037070 con 0x7f0b180b77c0 2024-09-12T03:17:34.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:34.238+0000 7f0b17fff640 1 -- 172.21.15.27:0/1335018139 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b10032440 con 0x7f0b180b77c0 2024-09-12T03:17:34.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:34.238+0000 7f0b17fff640 1 -- 172.21.15.27:0/1335018139 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b10032810 con 0x7f0b180b77c0 2024-09-12T03:17:34.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:34.238+0000 7f0b1f3cd640 1 -- 172.21.15.27:0/1335018139 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0b180b77c0 msgr2=0x7f0b180b7c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:34.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:34.238+0000 7f0b1f3cd640 1 --2- 172.21.15.27:0/1335018139 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0b180b77c0 0x7f0b180b7c70 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f0b100098a0 tx=0x7f0b1002a2e0 comp rx=0 tx=0).stop 2024-09-12T03:17:34.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:34.238+0000 7f0b1f3cd640 1 -- 172.21.15.27:0/1335018139 shutdown_connections 2024-09-12T03:17:34.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:34.238+0000 7f0b1f3cd640 1 --2- 172.21.15.27:0/1335018139 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0b180b77c0 0x7f0b180b7c70 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:34.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:34.238+0000 7f0b1f3cd640 1 --2- 172.21.15.27:0/1335018139 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f0b180d3c20 0x7f0b180d4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:34.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:34.238+0000 7f0b1f3cd640 1 -- 172.21.15.27:0/1335018139 >> 172.21.15.27:0/1335018139 conn(0x7f0b180d1420 msgr2=0x7f0b180d1830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:34.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:34.238+0000 7f0b1f3cd640 1 -- 172.21.15.27:0/1335018139 shutdown_connections 2024-09-12T03:17:34.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:34.238+0000 7f0b1f3cd640 1 -- 172.21.15.27:0/1335018139 wait complete. 2024-09-12T03:17:34.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:34.239+0000 7f0b1f3cd640 1 Processor -- start 2024-09-12T03:17:34.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:34.239+0000 7f0b1f3cd640 1 -- start start 2024-09-12T03:17:34.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:34.240+0000 7f0b1f3cd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0b180b6240 0x7f0b180c9900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:34.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:34.240+0000 7f0b1f3cd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0b180b77c0 0x7f0b18161730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:34.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:34.240+0000 7f0b1f3cd640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f0b180d3c20 0x7f0b1814ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:34.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:34.240+0000 7f0b1f3cd640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f0b180cfdf0 con 0x7f0b180b77c0 2024-09-12T03:17:34.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:34.240+0000 7f0b1f3cd640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f0b180cfc70 con 0x7f0b180b6240 2024-09-12T03:17:34.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:34.240+0000 7f0b1f3cd640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f0b180b81b0 con 0x7f0b180d3c20 2024-09-12T03:17:34.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:34.240+0000 7f0b1c968640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0b180b77c0 0x7f0b18161730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:34.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:34.240+0000 7f0b1c968640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0b180b77c0 0x7f0b18161730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:48264/0 (socket says 172.21.15.27:48264) 2024-09-12T03:17:34.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:34.240+0000 7f0b1c968640 1 -- 172.21.15.27:0/3267775877 learned_addr learned my addr 172.21.15.27:0/3267775877 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:17:34.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:34.240+0000 7f0b1d96a640 1 --2- 172.21.15.27:0/3267775877 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f0b180d3c20 0x7f0b1814ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:34.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:34.240+0000 7f0b1d169640 1 --2- 172.21.15.27:0/3267775877 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0b180b6240 0x7f0b180c9900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:34.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:34.241+0000 7f0b1c968640 1 -- 172.21.15.27:0/3267775877 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f0b180d3c20 msgr2=0x7f0b1814ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:34.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:34.241+0000 7f0b1c968640 1 --2- 172.21.15.27:0/3267775877 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f0b180d3c20 0x7f0b1814ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:34.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:34.241+0000 7f0b1c968640 1 -- 172.21.15.27:0/3267775877 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0b180b6240 msgr2=0x7f0b180c9900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:34.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:34.241+0000 7f0b1c968640 1 --2- 172.21.15.27:0/3267775877 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0b180b6240 0x7f0b180c9900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:34.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:34.241+0000 7f0b1c968640 1 -- 172.21.15.27:0/3267775877 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b18159520 con 0x7f0b180b77c0 2024-09-12T03:17:34.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:34.241+0000 7f0b1c968640 1 --2- 172.21.15.27:0/3267775877 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0b180b77c0 0x7f0b18161730 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f0b0c00c7f0 tx=0x7f0b0c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:34.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:34.241+0000 7f0b15ffb640 1 -- 172.21.15.27:0/3267775877 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b0c015070 con 0x7f0b180b77c0 2024-09-12T03:17:34.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:34.241+0000 7f0b15ffb640 1 -- 172.21.15.27:0/3267775877 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b0c007820 con 0x7f0b180b77c0 2024-09-12T03:17:34.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:34.241+0000 7f0b15ffb640 1 -- 172.21.15.27:0/3267775877 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b0c007c10 con 0x7f0b180b77c0 2024-09-12T03:17:34.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:34.241+0000 7f0b1f3cd640 1 -- 172.21.15.27:0/3267775877 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b180c8860 con 0x7f0b180b77c0 2024-09-12T03:17:34.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:34.241+0000 7f0b1f3cd640 1 -- 172.21.15.27:0/3267775877 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b180c8e20 con 0x7f0b180b77c0 2024-09-12T03:17:34.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:34.242+0000 7f0b1f3cd640 1 -- 172.21.15.27:0/3267775877 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b180d0420 con 0x7f0b180b77c0 2024-09-12T03:17:34.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:34.243+0000 7f0b15ffb640 1 -- 172.21.15.27:0/3267775877 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 7) ==== 90651+0+0 (secure 0 0 0) 0x7f0b0c018070 con 0x7f0b180b77c0 2024-09-12T03:17:34.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:34.243+0000 7f0b15ffb640 1 --2- 172.21.15.27:0/3267775877 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f0b0406bfe0 0x7f0b0406e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:34.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:34.243+0000 7f0b15ffb640 1 -- 172.21.15.27:0/3267775877 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 4943+0+0 (secure 0 0 0) 0x7f0b0c08ae80 con 0x7f0b180b77c0 2024-09-12T03:17:34.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:34.243+0000 7f0b1d169640 1 --2- 172.21.15.27:0/3267775877 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f0b0406bfe0 0x7f0b0406e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:34.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:34.244+0000 7f0b1d169640 1 --2- 172.21.15.27:0/3267775877 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f0b0406bfe0 0x7f0b0406e4a0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f0b08005660 tx=0x7f0b08003d90 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:34.248 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:34.247+0000 7f0b15ffb640 1 -- 172.21.15.27:0/3267775877 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f0b0c059200 con 0x7f0b180b77c0 2024-09-12T03:17:34.424 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:34.423+0000 7f0b1f3cd640 1 -- 172.21.15.27:0/3267775877 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f0b1814b090 con 0x7f0b180b77c0 2024-09-12T03:17:34.427 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:34.426+0000 7f0b15ffb640 1 -- 172.21.15.27:0/3267775877 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v12)=0 v12) ==== 122+0+0 (secure 0 0 0) 0x7f0b0c05d500 con 0x7f0b180b77c0 2024-09-12T03:17:34.430 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:34.428+0000 7f0b1f3cd640 1 -- 172.21.15.27:0/3267775877 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f0b0406bfe0 msgr2=0x7f0b0406e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:34.430 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:34.428+0000 7f0b1f3cd640 1 --2- 172.21.15.27:0/3267775877 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f0b0406bfe0 0x7f0b0406e4a0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f0b08005660 tx=0x7f0b08003d90 comp rx=0 tx=0).stop 2024-09-12T03:17:34.430 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:34.428+0000 7f0b1f3cd640 1 -- 172.21.15.27:0/3267775877 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0b180b77c0 msgr2=0x7f0b18161730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:34.430 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:34.428+0000 7f0b1f3cd640 1 --2- 172.21.15.27:0/3267775877 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0b180b77c0 0x7f0b18161730 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f0b0c00c7f0 tx=0x7f0b0c00ccb0 comp rx=0 tx=0).stop 2024-09-12T03:17:34.430 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:34.428+0000 7f0b1f3cd640 1 -- 172.21.15.27:0/3267775877 shutdown_connections 2024-09-12T03:17:34.430 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:34.428+0000 7f0b1f3cd640 1 --2- 172.21.15.27:0/3267775877 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f0b0406bfe0 0x7f0b0406e4a0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:34.430 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:34.428+0000 7f0b1f3cd640 1 --2- 172.21.15.27:0/3267775877 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f0b180d3c20 0x7f0b1814ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:34.430 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:34.428+0000 7f0b1f3cd640 1 --2- 172.21.15.27:0/3267775877 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0b180b77c0 0x7f0b18161730 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:34.430 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:34.428+0000 7f0b1f3cd640 1 --2- 172.21.15.27:0/3267775877 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0b180b6240 0x7f0b180c9900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:34.430 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:34.428+0000 7f0b1f3cd640 1 -- 172.21.15.27:0/3267775877 >> 172.21.15.27:0/3267775877 conn(0x7f0b180d1420 msgr2=0x7f0b180b6620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:34.430 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:34.429+0000 7f0b1f3cd640 1 -- 172.21.15.27:0/3267775877 shutdown_connections 2024-09-12T03:17:34.430 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:34.429+0000 7f0b1f3cd640 1 -- 172.21.15.27:0/3267775877 wait complete. 2024-09-12T03:17:34.441 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-12T03:17:34.441 DEBUG:teuthology.orchestra.run.smithi027:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-12T03:17:34.481 DEBUG:teuthology.orchestra.run.smithi045:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-12T03:17:34.528 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-12T03:17:34.529 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-12T03:17:34.529 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T03:17:34.545 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056004,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497432k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16324 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248716k,nr_inodes=812179,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497432k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-12T03:17:34.545 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-12T03:17:34.545 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T03:17:34.564 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056795,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16602 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249348k,nr_inodes=812337,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-12T03:17:34.564 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-12T03:17:34.564 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T03:17:34.600 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056004,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497432k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16324 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248716k,nr_inodes=812179,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497432k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-12T03:17:34.600 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-12T03:17:34.601 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T03:17:34.621 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056795,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16602 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249348k,nr_inodes=812337,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-12T03:17:34.622 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-12T03:17:34.622 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-12T03:17:34.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:34.742+0000 7ffa3da27640 1 Processor -- start 2024-09-12T03:17:34.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:34.742+0000 7ffa3da27640 1 -- start start 2024-09-12T03:17:34.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:34.743+0000 7ffa3da27640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ffa3815e780 0x7ffa380d3e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:34.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:34.743+0000 7ffa3da27640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffa380b6690 con 0x7ffa380d43b0 2024-09-12T03:17:34.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:34.743+0000 7ffa3da27640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffa380b6970 con 0x7ffa380d1990 2024-09-12T03:17:34.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:34.743+0000 7ffa3da27640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7ffa380b7950 con 0x7ffa3815e780 2024-09-12T03:17:34.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:34.743+0000 7ffa367fc640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7ffa380d43b0 0x7ffa380d1280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:46608/0 (socket says 172.21.15.27:46608) 2024-09-12T03:17:34.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:34.743+0000 7ffa367fc640 1 -- 172.21.15.27:0/635119072 learned_addr learned my addr 172.21.15.27:0/635119072 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:17:34.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:34.743+0000 7ffa36ffd640 1 --2- 172.21.15.27:0/635119072 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ffa3815e780 0x7ffa380d3e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:34.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:34.743+0000 7ffa35ffb640 1 -- 172.21.15.27:0/635119072 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1458880767 0 0) 0x7ffa380b6690 con 0x7ffa380d43b0 2024-09-12T03:17:34.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:34.743+0000 7ffa35ffb640 1 -- 172.21.15.27:0/635119072 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ffa20003520 con 0x7ffa380d43b0 2024-09-12T03:17:34.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:34.743+0000 7ffa35ffb640 1 -- 172.21.15.27:0/635119072 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1173566267 0 0) 0x7ffa20003520 con 0x7ffa380d43b0 2024-09-12T03:17:34.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:34.743+0000 7ffa35ffb640 1 -- 172.21.15.27:0/635119072 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ffa3815e780 msgr2=0x7ffa380d3e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:34.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:34.743+0000 7ffa35ffb640 1 --2- 172.21.15.27:0/635119072 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ffa3815e780 0x7ffa380d3e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:34.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:34.743+0000 7ffa35ffb640 1 -- 172.21.15.27:0/635119072 >> v1:172.21.15.45:6789/0 conn(0x7ffa380d1990 legacy=0x7ffa380b3390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:34.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:34.743+0000 7ffa35ffb640 1 -- 172.21.15.27:0/635119072 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa380b6c50 con 0x7ffa380d43b0 2024-09-12T03:17:34.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:34.743+0000 7ffa35ffb640 1 -- 172.21.15.27:0/635119072 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 847755213 0 0) 0x7ffa2c001670 con 0x7ffa380d43b0 2024-09-12T03:17:34.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:34.743+0000 7ffa35ffb640 1 -- 172.21.15.27:0/635119072 >> v1:172.21.15.27:6789/0 conn(0x7ffa380d43b0 legacy=0x7ffa380d1280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:34.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:34.743+0000 7ffa35ffb640 1 --2- 172.21.15.27:0/635119072 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffa200039e0 0x7ffa20003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:34.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:34.743+0000 7ffa35ffb640 1 --2- 172.21.15.27:0/635119072 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ffa3815e780 0x7ffa20004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:34.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:34.743+0000 7ffa35ffb640 1 --2- 172.21.15.27:0/635119072 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffa20004d40 0x7ffa200051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:34.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:34.743+0000 7ffa35ffb640 1 -- 172.21.15.27:0/635119072 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7ffa2000ba70 con 0x7ffa20004d40 2024-09-12T03:17:34.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:34.743+0000 7ffa35ffb640 1 -- 172.21.15.27:0/635119072 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ffa2000bbf0 con 0x7ffa200039e0 2024-09-12T03:17:34.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:34.743+0000 7ffa35ffb640 1 -- 172.21.15.27:0/635119072 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7ffa2000bdf0 con 0x7ffa3815e780 2024-09-12T03:17:34.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:34.743+0000 7ffa377fe640 1 --2- 172.21.15.27:0/635119072 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffa20004d40 0x7ffa200051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:34.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:34.744+0000 7ffa36ffd640 1 --2- 172.21.15.27:0/635119072 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffa200039e0 0x7ffa20003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:34.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:34.744+0000 7ffa367fc640 1 --2- 172.21.15.27:0/635119072 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ffa3815e780 0x7ffa20004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:34.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:34.744+0000 7ffa377fe640 1 -- 172.21.15.27:0/635119072 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ffa3815e780 msgr2=0x7ffa20004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:34.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:34.744+0000 7ffa377fe640 1 --2- 172.21.15.27:0/635119072 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ffa3815e780 0x7ffa20004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:34.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:34.744+0000 7ffa377fe640 1 -- 172.21.15.27:0/635119072 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffa200039e0 msgr2=0x7ffa20003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:34.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:34.744+0000 7ffa377fe640 1 --2- 172.21.15.27:0/635119072 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffa200039e0 0x7ffa20003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:34.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:34.744+0000 7ffa377fe640 1 -- 172.21.15.27:0/635119072 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa20003520 con 0x7ffa20004d40 2024-09-12T03:17:34.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:34.744+0000 7ffa377fe640 1 --2- 172.21.15.27:0/635119072 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffa20004d40 0x7ffa200051c0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7ffa3808d450 tx=0x7ffa20005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66c8a651e31d79bb server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:34.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:34.745+0000 7ffa35ffb640 1 -- 172.21.15.27:0/635119072 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa30001a70 con 0x7ffa20004d40 2024-09-12T03:17:34.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:34.745+0000 7ffa35ffb640 1 -- 172.21.15.27:0/635119072 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa3002ae10 con 0x7ffa20004d40 2024-09-12T03:17:34.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:34.745+0000 7ffa35ffb640 1 -- 172.21.15.27:0/635119072 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa30003bb0 con 0x7ffa20004d40 2024-09-12T03:17:34.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:34.745+0000 7ffa3da27640 1 -- 172.21.15.27:0/635119072 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffa20004d40 msgr2=0x7ffa200051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:34.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:34.745+0000 7ffa3da27640 1 --2- 172.21.15.27:0/635119072 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffa20004d40 0x7ffa200051c0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7ffa3808d450 tx=0x7ffa20005830 comp rx=0 tx=0).stop 2024-09-12T03:17:34.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:34.745+0000 7ffa377fe640 1 -- 172.21.15.27:0/635119072 reap_dead start 2024-09-12T03:17:34.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:34.745+0000 7ffa3da27640 1 -- 172.21.15.27:0/635119072 shutdown_connections 2024-09-12T03:17:34.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:34.745+0000 7ffa3da27640 1 -- 172.21.15.27:0/635119072 >> 172.21.15.27:0/635119072 conn(0x7ffa3806da60 msgr2=0x7ffa3806de70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:34.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:34.745+0000 7ffa3da27640 1 -- 172.21.15.27:0/635119072 shutdown_connections 2024-09-12T03:17:34.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:34.745+0000 7ffa3da27640 1 -- 172.21.15.27:0/635119072 wait complete. 2024-09-12T03:17:34.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:34.746+0000 7ffa3da27640 1 Processor -- start 2024-09-12T03:17:34.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:34.747+0000 7ffa3da27640 1 -- start start 2024-09-12T03:17:34.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:34.747+0000 7ffa3da27640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffa20004d40 0x7ffa38150bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:34.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:34.747+0000 7ffa3da27640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ffa380abd20 0x7ffa380b0900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:34.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:34.747+0000 7ffa3da27640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffa380ac1a0 0x7ffa380b0e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:34.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:34.747+0000 7ffa3da27640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7ffa380b6340 con 0x7ffa20004d40 2024-09-12T03:17:34.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:34.747+0000 7ffa3da27640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ffa380b7210 con 0x7ffa380ac1a0 2024-09-12T03:17:34.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:34.747+0000 7ffa3da27640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7ffa380b8dd0 con 0x7ffa380abd20 2024-09-12T03:17:34.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:34.747+0000 7ffa36ffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffa20004d40 0x7ffa38150bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:34.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:34.747+0000 7ffa36ffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffa20004d40 0x7ffa38150bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:48280/0 (socket says 172.21.15.27:48280) 2024-09-12T03:17:34.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:34.747+0000 7ffa36ffd640 1 -- 172.21.15.27:0/1019447610 learned_addr learned my addr 172.21.15.27:0/1019447610 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:17:34.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:34.748+0000 7ffa377fe640 1 --2- 172.21.15.27:0/1019447610 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffa380ac1a0 0x7ffa380b0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:34.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:34.748+0000 7ffa367fc640 1 --2- 172.21.15.27:0/1019447610 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ffa380abd20 0x7ffa380b0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:34.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:34.748+0000 7ffa36ffd640 1 -- 172.21.15.27:0/1019447610 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ffa380abd20 msgr2=0x7ffa380b0900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:34.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:34.748+0000 7ffa36ffd640 1 --2- 172.21.15.27:0/1019447610 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ffa380abd20 0x7ffa380b0900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:34.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:34.748+0000 7ffa36ffd640 1 -- 172.21.15.27:0/1019447610 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffa380ac1a0 msgr2=0x7ffa380b0e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:34.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:34.748+0000 7ffa36ffd640 1 --2- 172.21.15.27:0/1019447610 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffa380ac1a0 0x7ffa380b0e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:34.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:34.748+0000 7ffa36ffd640 1 -- 172.21.15.27:0/1019447610 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa380ac700 con 0x7ffa20004d40 2024-09-12T03:17:34.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:34.748+0000 7ffa36ffd640 1 --2- 172.21.15.27:0/1019447610 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffa20004d40 0x7ffa38150bf0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7ffa280119a0 tx=0x7ffa28011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:34.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:34.748+0000 7ffa1ffff640 1 -- 172.21.15.27:0/1019447610 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa2800fa90 con 0x7ffa20004d40 2024-09-12T03:17:34.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:34.748+0000 7ffa1ffff640 1 -- 172.21.15.27:0/1019447610 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa2800fc30 con 0x7ffa20004d40 2024-09-12T03:17:34.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:34.748+0000 7ffa1ffff640 1 -- 172.21.15.27:0/1019447610 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa28014440 con 0x7ffa20004d40 2024-09-12T03:17:34.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:34.748+0000 7ffa3da27640 1 -- 172.21.15.27:0/1019447610 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa380ac9e0 con 0x7ffa20004d40 2024-09-12T03:17:34.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:34.748+0000 7ffa3da27640 1 -- 172.21.15.27:0/1019447610 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa380ae4c0 con 0x7ffa20004d40 2024-09-12T03:17:34.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:34.749+0000 7ffa3da27640 1 -- 172.21.15.27:0/1019447610 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa3815eb60 con 0x7ffa20004d40 2024-09-12T03:17:34.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:34.750+0000 7ffa1ffff640 1 -- 172.21.15.27:0/1019447610 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 7) ==== 90651+0+0 (secure 0 0 0) 0x7ffa28012040 con 0x7ffa20004d40 2024-09-12T03:17:34.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:34.750+0000 7ffa1ffff640 1 --2- 172.21.15.27:0/1019447610 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7ffa2406bfe0 0x7ffa2406e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:34.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:34.750+0000 7ffa1ffff640 1 -- 172.21.15.27:0/1019447610 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 4943+0+0 (secure 0 0 0) 0x7ffa2808b3a0 con 0x7ffa20004d40 2024-09-12T03:17:34.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:34.750+0000 7ffa367fc640 1 --2- 172.21.15.27:0/1019447610 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7ffa2406bfe0 0x7ffa2406e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:34.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:34.751+0000 7ffa367fc640 1 --2- 172.21.15.27:0/1019447610 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7ffa2406bfe0 0x7ffa2406e4a0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7ffa380ad020 tx=0x7ffa2c007040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:34.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:34.754+0000 7ffa1ffff640 1 -- 172.21.15.27:0/1019447610 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7ffa28019030 con 0x7ffa20004d40 2024-09-12T03:17:34.969 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:34.968+0000 7ffa3da27640 1 -- 172.21.15.27:0/1019447610 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ffa380b9720 con 0x7ffa20004d40 2024-09-12T03:17:34.969 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:34.968+0000 7ffa1ffff640 1 -- 172.21.15.27:0/1019447610 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+3910 (secure 0 0 0) 0x7ffa28059780 con 0x7ffa20004d40 2024-09-12T03:17:34.970 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:17:34.970 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":17,"btime":"2024-09-12T03:17:30:102892+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4592,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6836/3382368386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":3382368386},{"type":"v1","addr":"172.21.15.45:6836","nonce":3382368386}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":4595,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/2127097700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6835","nonce":2127097700},{"type":"v1","addr":"172.21.15.45:6837","nonce":2127097700}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":4815,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6834/2623734554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":2623734554},{"type":"v1","addr":"172.21.15.27:6834","nonce":2623734554}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":5127,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/2816840901","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":2816840901},{"type":"v1","addr":"172.21.15.27:6835","nonce":2816840901}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":17}],"filesystems":[]} 2024-09-12T03:17:34.970 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 17 2024-09-12T03:17:34.971 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:34.970+0000 7ffa3da27640 1 -- 172.21.15.27:0/1019447610 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7ffa2406bfe0 msgr2=0x7ffa2406e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:34.971 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:34.970+0000 7ffa3da27640 1 --2- 172.21.15.27:0/1019447610 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7ffa2406bfe0 0x7ffa2406e4a0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7ffa380ad020 tx=0x7ffa2c007040 comp rx=0 tx=0).stop 2024-09-12T03:17:34.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:34.970+0000 7ffa3da27640 1 -- 172.21.15.27:0/1019447610 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffa20004d40 msgr2=0x7ffa38150bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:34.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:34.970+0000 7ffa3da27640 1 --2- 172.21.15.27:0/1019447610 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffa20004d40 0x7ffa38150bf0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7ffa280119a0 tx=0x7ffa28011e60 comp rx=0 tx=0).stop 2024-09-12T03:17:34.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:34.970+0000 7ffa3da27640 1 -- 172.21.15.27:0/1019447610 shutdown_connections 2024-09-12T03:17:34.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:34.970+0000 7ffa3da27640 1 --2- 172.21.15.27:0/1019447610 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7ffa2406bfe0 0x7ffa2406e4a0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:34.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:34.970+0000 7ffa3da27640 1 --2- 172.21.15.27:0/1019447610 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffa380ac1a0 0x7ffa380b0e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:34.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:34.970+0000 7ffa3da27640 1 --2- 172.21.15.27:0/1019447610 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ffa380abd20 0x7ffa380b0900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:34.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:34.970+0000 7ffa3da27640 1 --2- 172.21.15.27:0/1019447610 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffa20004d40 0x7ffa38150bf0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:34.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:34.970+0000 7ffa3da27640 1 -- 172.21.15.27:0/1019447610 >> 172.21.15.27:0/1019447610 conn(0x7ffa3806da60 msgr2=0x7ffa3815f820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:34.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:34.970+0000 7ffa3da27640 1 -- 172.21.15.27:0/1019447610 shutdown_connections 2024-09-12T03:17:34.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:34.970+0000 7ffa3da27640 1 -- 172.21.15.27:0/1019447610 wait complete. 2024-09-12T03:17:34.983 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-12T03:17:34.983 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-12T03:17:34.983 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-12T03:17:34.983 INFO:teuthology.orchestra.run:waiting for 300 2024-09-12T03:17:34.984 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-12T03:17:34.984 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-12T03:17:34.984 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-12T03:17:34.984 INFO:teuthology.orchestra.run:waiting for 300 2024-09-12T03:17:34.984 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-12T03:17:34.984 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-12T03:17:34.985 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-12T03:17:34.985 INFO:teuthology.orchestra.run:waiting for 300 2024-09-12T03:17:34.985 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-12T03:17:34.985 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-12T03:17:34.985 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-12T03:17:34.985 INFO:teuthology.orchestra.run:waiting for 300 2024-09-12T03:17:34.993 INFO:tasks.ceph.mds.a:Stopped 2024-09-12T03:17:34.993 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-12T03:17:34.994 INFO:tasks.ceph.mds.c:Stopped 2024-09-12T03:17:34.994 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-12T03:17:35.001 INFO:tasks.ceph.mds.d:Stopped 2024-09-12T03:17:35.001 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-12T03:17:35.002 INFO:tasks.ceph.mds.b:Stopped 2024-09-12T03:17:35.002 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-12T03:17:35.036 INFO:tasks.ceph.mds.a:Started 2024-09-12T03:17:35.037 DEBUG:teuthology.parallel:result is None 2024-09-12T03:17:35.038 INFO:tasks.ceph.mds.c:Started 2024-09-12T03:17:35.038 DEBUG:teuthology.parallel:result is None 2024-09-12T03:17:35.044 INFO:tasks.ceph.mds.d:Started 2024-09-12T03:17:35.044 DEBUG:teuthology.parallel:result is None 2024-09-12T03:17:35.045 INFO:tasks.ceph.mds.b:Started 2024-09-12T03:17:35.045 DEBUG:teuthology.parallel:result is None 2024-09-12T03:17:35.045 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-12T03:17:35.115 INFO:tasks.ceph.mds.c.smithi027.stdout:starting mds.c at 2024-09-12T03:17:35.115 INFO:tasks.ceph.mds.a.smithi027.stdout:starting mds.a at 2024-09-12T03:17:35.134 INFO:tasks.ceph.mds.d.smithi045.stdout:starting mds.d at 2024-09-12T03:17:35.134 INFO:tasks.ceph.mds.b.smithi045.stdout:starting mds.b at 2024-09-12T03:17:35.154 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:35.153+0000 7f45ce477640 1 Processor -- start 2024-09-12T03:17:35.154 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:35.153+0000 7f45ce477640 1 -- start start 2024-09-12T03:17:35.157 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:35.154+0000 7f45ce477640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f45c8145c30 0x7f45c815ca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:35.157 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:35.154+0000 7f45ce477640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f45c8146010 0x7f45c80cd330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:35.157 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:35.154+0000 7f45ce477640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f45c80b4d70 con 0x7f45c80cd870 2024-09-12T03:17:35.157 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:35.154+0000 7f45ce477640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f45c80b5050 con 0x7f45c8146010 2024-09-12T03:17:35.157 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:35.154+0000 7f45ce477640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f45c806d140 con 0x7f45c8145c30 2024-09-12T03:17:35.158 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:35.154+0000 7f45c7fff640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f45c8145c30 0x7f45c815ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:35.158 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:35.154+0000 7f45c77fe640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f45c8146010 0x7f45c80cd330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:35.158 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:35.154+0000 7f45cca14640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f45c80cd870 0x7f45c80b4040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.27:53950/0 (socket says 172.21.15.27:53950) 2024-09-12T03:17:35.158 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:35.154+0000 7f45cca14640 1 -- 172.21.15.27:0/239075125 learned_addr learned my addr 172.21.15.27:0/239075125 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:17:35.158 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:35.154+0000 7f45c77fe640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f45c8146010 0x7f45c80cd330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.27:38972/0 (socket says 172.21.15.27:38972) 2024-09-12T03:17:35.158 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:35.154+0000 7f45c6ffd640 1 -- 172.21.15.27:0/239075125 <== mon.1 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3099963774 0 0) 0x7f45c80b4d70 con 0x7f45c80cd870 2024-09-12T03:17:35.158 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:35.154+0000 7f45c6ffd640 1 -- 172.21.15.27:0/239075125 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f45b0003520 con 0x7f45c80cd870 2024-09-12T03:17:35.158 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:35.154+0000 7f45c7fff640 1 -- 172.21.15.27:0/239075125 >> v2:172.21.15.45:3300/0 conn(0x7f45c8146010 msgr2=0x7f45c80cd330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:35.158 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:35.154+0000 7f45c7fff640 1 --2- 172.21.15.27:0/239075125 >> v2:172.21.15.45:3300/0 conn(0x7f45c8146010 0x7f45c80cd330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:35.158 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:35.154+0000 7f45c7fff640 1 -- 172.21.15.27:0/239075125 >> v1:172.21.15.45:6789/0 conn(0x7f45c80cd870 legacy=0x7f45c80b4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:35.159 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:35.154+0000 7f45c7fff640 1 -- 172.21.15.27:0/239075125 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45c806d420 con 0x7f45c8145c30 2024-09-12T03:17:35.159 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:35.154+0000 7f45c77fe640 1 --2- 172.21.15.27:0/239075125 >> v2:172.21.15.45:3300/0 conn(0x7f45c8146010 0x7f45c80cd330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:17:35.159 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:35.155+0000 7f45c7fff640 1 --2- 172.21.15.27:0/239075125 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f45c8145c30 0x7f45c815ca00 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f45b80097a0 tx=0x7f45b802a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=578976332539e5c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:35.159 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:35.155+0000 7f45c6ffd640 1 -- 172.21.15.27:0/239075125 <== mon.2 v2:172.21.15.45:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45b8037070 con 0x7f45c8145c30 2024-09-12T03:17:35.159 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:35.155+0000 7f45c6ffd640 1 -- 172.21.15.27:0/239075125 <== mon.2 v2:172.21.15.45:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45b802ae50 con 0x7f45c8145c30 2024-09-12T03:17:35.159 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:35.155+0000 7f45c6ffd640 1 -- 172.21.15.27:0/239075125 <== mon.2 v2:172.21.15.45:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45b8032640 con 0x7f45c8145c30 2024-09-12T03:17:35.159 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:35.156+0000 7f45ce477640 1 -- 172.21.15.27:0/239075125 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f45c8145c30 msgr2=0x7f45c815ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:35.159 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:35.156+0000 7f45ce477640 1 --2- 172.21.15.27:0/239075125 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f45c8145c30 0x7f45c815ca00 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f45b80097a0 tx=0x7f45b802a100 comp rx=0 tx=0).stop 2024-09-12T03:17:35.159 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:35.156+0000 7f45ce477640 1 -- 172.21.15.27:0/239075125 shutdown_connections 2024-09-12T03:17:35.159 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:35.156+0000 7f45ce477640 1 --2- 172.21.15.27:0/239075125 >> v2:172.21.15.45:3300/0 conn(0x7f45c8146010 0x7f45c80cd330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:35.159 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:35.156+0000 7f45ce477640 1 --2- 172.21.15.27:0/239075125 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f45c8145c30 0x7f45c815ca00 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:35.159 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:35.156+0000 7f45ce477640 1 -- 172.21.15.27:0/239075125 >> 172.21.15.27:0/239075125 conn(0x7f45c8062c60 msgr2=0x7f45c806bb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:35.160 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:35.156+0000 7f45ce477640 1 -- 172.21.15.27:0/239075125 shutdown_connections 2024-09-12T03:17:35.160 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:35.156+0000 7f45ce477640 1 -- 172.21.15.27:0/239075125 wait complete. 2024-09-12T03:17:35.162 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:35.157+0000 7f45ce477640 1 Processor -- start 2024-09-12T03:17:35.162 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:35.157+0000 7f45ce477640 1 -- start start 2024-09-12T03:17:35.162 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:35.158+0000 7f45ce477640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f45c80cd870 0x7f45c81f3f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:35.162 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:35.158+0000 7f45ce477640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f45c8145c30 0x7f45c813e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:35.162 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:35.158+0000 7f45ce477640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f45c8146010 0x7f45c8140900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:35.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:35.158+0000 7f45ce477640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f45c80b48d0 con 0x7f45c8145c30 2024-09-12T03:17:35.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:35.158+0000 7f45ce477640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f45c80b4750 con 0x7f45c80cd870 2024-09-12T03:17:35.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:35.158+0000 7f45ce477640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f45c80b4a50 con 0x7f45c8146010 2024-09-12T03:17:35.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:35.158+0000 7f45c77fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f45c8145c30 0x7f45c813e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:35.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:35.158+0000 7f45c7fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f45c80cd870 0x7f45c81f3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:35.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:35.158+0000 7f45c77fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f45c8145c30 0x7f45c813e700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:48308/0 (socket says 172.21.15.27:48308) 2024-09-12T03:17:35.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:35.158+0000 7f45c77fe640 1 -- 172.21.15.27:0/115953271 learned_addr learned my addr 172.21.15.27:0/115953271 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:17:35.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:35.158+0000 7f45c7fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f45c80cd870 0x7f45c81f3f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.27:38974/0 (socket says 172.21.15.27:38974) 2024-09-12T03:17:35.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:35.158+0000 7f45cca14640 1 --2- 172.21.15.27:0/115953271 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f45c8146010 0x7f45c8140900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:35.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:35.159+0000 7f45c7fff640 1 -- 172.21.15.27:0/115953271 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f45c8146010 msgr2=0x7f45c8140900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:35.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:35.159+0000 7f45c7fff640 1 --2- 172.21.15.27:0/115953271 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f45c8146010 0x7f45c8140900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:35.164 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:35.159+0000 7f45c7fff640 1 -- 172.21.15.27:0/115953271 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f45c8145c30 msgr2=0x7f45c813e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:35.164 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:35.159+0000 7f45c7fff640 1 --2- 172.21.15.27:0/115953271 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f45c8145c30 0x7f45c813e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:35.164 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:35.159+0000 7f45c7fff640 1 -- 172.21.15.27:0/115953271 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45c80cca20 con 0x7f45c80cd870 2024-09-12T03:17:35.164 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:35.159+0000 7f45c77fe640 1 --2- 172.21.15.27:0/115953271 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f45c8145c30 0x7f45c813e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:17:35.164 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:35.159+0000 7f45c7fff640 1 --2- 172.21.15.27:0/115953271 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f45c80cd870 0x7f45c81f3f40 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f45b8031000 tx=0x7f45b80046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:35.164 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:35.159+0000 7f45c4ff9640 1 -- 172.21.15.27:0/115953271 <== mon.1 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45b8041070 con 0x7f45c80cd870 2024-09-12T03:17:35.164 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:35.159+0000 7f45c4ff9640 1 -- 172.21.15.27:0/115953271 <== mon.1 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45b803c460 con 0x7f45c80cd870 2024-09-12T03:17:35.164 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:35.159+0000 7f45ce477640 1 -- 172.21.15.27:0/115953271 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45c81f4480 con 0x7f45c80cd870 2024-09-12T03:17:35.164 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:35.159+0000 7f45c4ff9640 1 -- 172.21.15.27:0/115953271 <== mon.1 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45b8037040 con 0x7f45c80cd870 2024-09-12T03:17:35.164 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:35.159+0000 7f45ce477640 1 -- 172.21.15.27:0/115953271 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45c81f4a30 con 0x7f45c80cd870 2024-09-12T03:17:35.164 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:35.160+0000 7f45ce477640 1 -- 172.21.15.27:0/115953271 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45c815ca00 con 0x7f45c80cd870 2024-09-12T03:17:35.164 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:35.161+0000 7f45c4ff9640 1 -- 172.21.15.27:0/115953271 <== mon.1 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 7) ==== 90651+0+0 (secure 0 0 0) 0x7f45b803a070 con 0x7f45c80cd870 2024-09-12T03:17:35.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:35.161+0000 7f45c4ff9640 1 --2- 172.21.15.27:0/115953271 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f45b406bfe0 0x7f45b406e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:35.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:35.161+0000 7f45c4ff9640 1 -- 172.21.15.27:0/115953271 <== mon.1 v2:172.21.15.45:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 4943+0+0 (secure 0 0 0) 0x7f45b80aad80 con 0x7f45c80cd870 2024-09-12T03:17:35.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:35.161+0000 7f45c77fe640 1 --2- 172.21.15.27:0/115953271 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f45b406bfe0 0x7f45b406e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:35.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:35.162+0000 7f45c77fe640 1 --2- 172.21.15.27:0/115953271 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f45b406bfe0 0x7f45b406e4a0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f45bc009560 tx=0x7f45bc009190 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:35.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:35.166+0000 7f45c4ff9640 1 -- 172.21.15.27:0/115953271 <== mon.1 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f45b8079190 con 0x7f45c80cd870 2024-09-12T03:17:35.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:35.353+0000 7f45ce477640 1 -- 172.21.15.27:0/115953271 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f45c80aa840 con 0x7f45c80cd870 2024-09-12T03:17:36.131 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.130+0000 7f45c4ff9640 1 -- 172.21.15.27:0/115953271 <== mon.1 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v37) ==== 97+0+0 (secure 0 0 0) 0x7f45b807d490 con 0x7f45c80cd870 2024-09-12T03:17:36.131 INFO:teuthology.orchestra.run.smithi027.stderr: removed all blocklist entries 2024-09-12T03:17:36.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.133+0000 7f45ce477640 1 -- 172.21.15.27:0/115953271 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f45b406bfe0 msgr2=0x7f45b406e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:36.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.133+0000 7f45ce477640 1 --2- 172.21.15.27:0/115953271 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f45b406bfe0 0x7f45b406e4a0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f45bc009560 tx=0x7f45bc009190 comp rx=0 tx=0).stop 2024-09-12T03:17:36.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.133+0000 7f45ce477640 1 -- 172.21.15.27:0/115953271 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f45c80cd870 msgr2=0x7f45c81f3f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:36.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.133+0000 7f45ce477640 1 --2- 172.21.15.27:0/115953271 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f45c80cd870 0x7f45c81f3f40 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f45b8031000 tx=0x7f45b80046c0 comp rx=0 tx=0).stop 2024-09-12T03:17:36.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.133+0000 7f45ce477640 1 -- 172.21.15.27:0/115953271 shutdown_connections 2024-09-12T03:17:36.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.133+0000 7f45ce477640 1 --2- 172.21.15.27:0/115953271 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f45b406bfe0 0x7f45b406e4a0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:36.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.133+0000 7f45ce477640 1 --2- 172.21.15.27:0/115953271 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f45c8146010 0x7f45c8140900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:36.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.133+0000 7f45ce477640 1 --2- 172.21.15.27:0/115953271 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f45c8145c30 0x7f45c813e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:36.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.133+0000 7f45ce477640 1 --2- 172.21.15.27:0/115953271 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f45c80cd870 0x7f45c81f3f40 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:36.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.133+0000 7f45ce477640 1 -- 172.21.15.27:0/115953271 >> 172.21.15.27:0/115953271 conn(0x7f45c8062c60 msgr2=0x7f45c80aaf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:36.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.133+0000 7f45ce477640 1 -- 172.21.15.27:0/115953271 shutdown_connections 2024-09-12T03:17:36.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.133+0000 7f45ce477640 1 -- 172.21.15.27:0/115953271 wait complete. 2024-09-12T03:17:36.147 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-12T03:17:36.253 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.251+0000 7f9ffa406640 1 Processor -- start 2024-09-12T03:17:36.253 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.252+0000 7f9ffa406640 1 -- start start 2024-09-12T03:17:36.256 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.252+0000 7f9ffa406640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f9ff415eb90 0x7f9ff40d2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:36.256 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.252+0000 7f9ffa406640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9ff40d3760 con 0x7f9ff40d1720 2024-09-12T03:17:36.256 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.252+0000 7f9ffa406640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9ff40d3a40 con 0x7f9ff40d5200 2024-09-12T03:17:36.256 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.252+0000 7f9ffa406640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f9ff40b7d20 con 0x7f9ff415eb90 2024-09-12T03:17:36.256 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.252+0000 7f9ff89a3640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f9ff40d1720 0x7f9ff40b8190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:46644/0 (socket says 172.21.15.27:46644) 2024-09-12T03:17:36.257 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.253+0000 7f9ff89a3640 1 -- 172.21.15.27:0/2990492133 learned_addr learned my addr 172.21.15.27:0/2990492133 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:17:36.257 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.253+0000 7f9ff3fff640 1 --2- 172.21.15.27:0/2990492133 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f9ff415eb90 0x7f9ff40d2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:36.257 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.253+0000 7f9ff2ffd640 1 -- 172.21.15.27:0/2990492133 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4099971228 0 0) 0x7f9ff40d3760 con 0x7f9ff40d1720 2024-09-12T03:17:36.257 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.253+0000 7f9ff2ffd640 1 -- 172.21.15.27:0/2990492133 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9fe0003520 con 0x7f9ff40d1720 2024-09-12T03:17:36.257 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.253+0000 7f9ff2ffd640 1 -- 172.21.15.27:0/2990492133 <== mon.1 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1657331886 0 0) 0x7f9ff40d3a40 con 0x7f9ff40d5200 2024-09-12T03:17:36.257 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.253+0000 7f9ff2ffd640 1 -- 172.21.15.27:0/2990492133 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9ff40d3760 con 0x7f9ff40d5200 2024-09-12T03:17:36.257 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.253+0000 7f9ff2ffd640 1 -- 172.21.15.27:0/2990492133 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3355476478 0 0) 0x7f9fe0003520 con 0x7f9ff40d1720 2024-09-12T03:17:36.257 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.253+0000 7f9ff2ffd640 1 -- 172.21.15.27:0/2990492133 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f9ff415eb90 msgr2=0x7f9ff40d2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:36.257 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.253+0000 7f9ff2ffd640 1 --2- 172.21.15.27:0/2990492133 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f9ff415eb90 0x7f9ff40d2df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:36.257 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.253+0000 7f9ff2ffd640 1 -- 172.21.15.27:0/2990492133 >> v1:172.21.15.45:6789/0 conn(0x7f9ff40d5200 legacy=0x7f9ff40d1010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:36.257 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.253+0000 7f9ff2ffd640 1 -- 172.21.15.27:0/2990492133 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ff40d3d20 con 0x7f9ff40d1720 2024-09-12T03:17:36.257 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.253+0000 7f9ff2ffd640 1 -- 172.21.15.27:0/2990492133 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 847755213 0 0) 0x7f9fec0019f0 con 0x7f9ff40d1720 2024-09-12T03:17:36.257 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.253+0000 7f9ff2ffd640 1 -- 172.21.15.27:0/2990492133 >> v1:172.21.15.27:6789/0 conn(0x7f9ff40d1720 legacy=0x7f9ff40b8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:36.258 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.253+0000 7f9ff2ffd640 1 --2- 172.21.15.27:0/2990492133 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f9ff415eb90 0x7f9fe0003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:36.258 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.253+0000 7f9ff2ffd640 1 --2- 172.21.15.27:0/2990492133 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9fe00040d0 0x7f9fe0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:36.258 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.253+0000 7f9ff2ffd640 1 --2- 172.21.15.27:0/2990492133 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9fe0004d40 0x7f9fe00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:36.258 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.253+0000 7f9ff2ffd640 1 -- 172.21.15.27:0/2990492133 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f9fe000ba70 con 0x7f9fe0004d40 2024-09-12T03:17:36.258 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.253+0000 7f9ff2ffd640 1 -- 172.21.15.27:0/2990492133 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f9fe000bc20 con 0x7f9fe00040d0 2024-09-12T03:17:36.258 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.253+0000 7f9ff2ffd640 1 -- 172.21.15.27:0/2990492133 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f9fe000be20 con 0x7f9ff415eb90 2024-09-12T03:17:36.258 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.253+0000 7f9ff89a3640 1 --2- 172.21.15.27:0/2990492133 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9fe0004d40 0x7f9fe00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:36.258 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.253+0000 7f9ff3fff640 1 --2- 172.21.15.27:0/2990492133 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f9ff415eb90 0x7f9fe0003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:36.258 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.253+0000 7f9ff37fe640 1 --2- 172.21.15.27:0/2990492133 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9fe00040d0 0x7f9fe0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:36.258 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.254+0000 7f9ff89a3640 1 -- 172.21.15.27:0/2990492133 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f9ff415eb90 msgr2=0x7f9fe0003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:36.258 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.254+0000 7f9ff89a3640 1 --2- 172.21.15.27:0/2990492133 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f9ff415eb90 0x7f9fe0003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:36.258 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.254+0000 7f9ff89a3640 1 -- 172.21.15.27:0/2990492133 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9fe00040d0 msgr2=0x7f9fe0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:36.258 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.254+0000 7f9ff89a3640 1 --2- 172.21.15.27:0/2990492133 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9fe00040d0 0x7f9fe0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:36.258 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.254+0000 7f9ff89a3640 1 -- 172.21.15.27:0/2990492133 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fe0003520 con 0x7f9fe0004d40 2024-09-12T03:17:36.259 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.254+0000 7f9ff89a3640 1 --2- 172.21.15.27:0/2990492133 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9fe0004d40 0x7f9fe00051c0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f9fec00a7c0 tx=0x7f9fe0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b088dd4710d7cc7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:36.259 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.254+0000 7f9ff2ffd640 1 -- 172.21.15.27:0/2990492133 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fec00a410 con 0x7f9fe0004d40 2024-09-12T03:17:36.259 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.254+0000 7f9ff2ffd640 1 -- 172.21.15.27:0/2990492133 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9fec02f070 con 0x7f9fe0004d40 2024-09-12T03:17:36.259 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.255+0000 7f9ff2ffd640 1 -- 172.21.15.27:0/2990492133 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fec02bc40 con 0x7f9fe0004d40 2024-09-12T03:17:36.259 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.255+0000 7f9ffa406640 1 -- 172.21.15.27:0/2990492133 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9fe0004d40 msgr2=0x7f9fe00051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:36.259 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.255+0000 7f9ffa406640 1 --2- 172.21.15.27:0/2990492133 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9fe0004d40 0x7f9fe00051c0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f9fec00a7c0 tx=0x7f9fe0005830 comp rx=0 tx=0).stop 2024-09-12T03:17:36.259 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.255+0000 7f9ff89a3640 1 -- 172.21.15.27:0/2990492133 reap_dead start 2024-09-12T03:17:36.259 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.255+0000 7f9ffa406640 1 -- 172.21.15.27:0/2990492133 shutdown_connections 2024-09-12T03:17:36.259 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.255+0000 7f9ffa406640 1 -- 172.21.15.27:0/2990492133 >> 172.21.15.27:0/2990492133 conn(0x7f9ff406dce0 msgr2=0x7f9ff406e0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:36.259 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.255+0000 7f9ffa406640 1 -- 172.21.15.27:0/2990492133 shutdown_connections 2024-09-12T03:17:36.259 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.255+0000 7f9ffa406640 1 -- 172.21.15.27:0/2990492133 wait complete. 2024-09-12T03:17:36.262 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.256+0000 7f9ffa406640 1 Processor -- start 2024-09-12T03:17:36.262 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.256+0000 7f9ffa406640 1 -- start start 2024-09-12T03:17:36.262 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.257+0000 7f9ffa406640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9fe0004d40 0x7f9ff40c9e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:36.262 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.257+0000 7f9ffa406640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9ff40ca3a0 0x7f9ff40c8340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:36.262 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.257+0000 7f9ffa406640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f9ff40c8880 0x7f9ff41f4620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:36.262 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.257+0000 7f9ffa406640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f9ff415ef70 con 0x7f9ff40ca3a0 2024-09-12T03:17:36.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.257+0000 7f9ffa406640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f9ff40d42e0 con 0x7f9fe0004d40 2024-09-12T03:17:36.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.257+0000 7f9ffa406640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f9ff415f0f0 con 0x7f9ff40c8880 2024-09-12T03:17:36.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.257+0000 7f9ff37fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9ff40ca3a0 0x7f9ff40c8340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:36.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.257+0000 7f9ff37fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9ff40ca3a0 0x7f9ff40c8340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:48332/0 (socket says 172.21.15.27:48332) 2024-09-12T03:17:36.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.257+0000 7f9ff37fe640 1 -- 172.21.15.27:0/2503894026 learned_addr learned my addr 172.21.15.27:0/2503894026 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:17:36.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.257+0000 7f9ff37fe640 1 -- 172.21.15.27:0/2503894026 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f9ff40c8880 msgr2=0x7f9ff41f4620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:36.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.257+0000 7f9ff37fe640 1 --2- 172.21.15.27:0/2503894026 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f9ff40c8880 0x7f9ff41f4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:36.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.257+0000 7f9ff37fe640 1 -- 172.21.15.27:0/2503894026 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9fe0004d40 msgr2=0x7f9ff40c9e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:36.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.257+0000 7f9ff3fff640 1 --2- 172.21.15.27:0/2503894026 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9fe0004d40 0x7f9ff40c9e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:36.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.258+0000 7f9ff37fe640 1 --2- 172.21.15.27:0/2503894026 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9fe0004d40 0x7f9ff40c9e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:36.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.258+0000 7f9ff37fe640 1 -- 172.21.15.27:0/2503894026 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ff40c8dc0 con 0x7f9ff40ca3a0 2024-09-12T03:17:36.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.258+0000 7f9ff3fff640 1 --2- 172.21.15.27:0/2503894026 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9fe0004d40 0x7f9ff40c9e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:17:36.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.258+0000 7f9ff37fe640 1 --2- 172.21.15.27:0/2503894026 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9ff40ca3a0 0x7f9ff40c8340 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f9fdc00f6f0 tx=0x7f9fdc00fbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:36.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.258+0000 7f9ff0ff9640 1 -- 172.21.15.27:0/2503894026 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fdc0050d0 con 0x7f9ff40ca3a0 2024-09-12T03:17:36.264 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.258+0000 7f9ff0ff9640 1 -- 172.21.15.27:0/2503894026 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9fdc018070 con 0x7f9ff40ca3a0 2024-09-12T03:17:36.264 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.258+0000 7f9ffa406640 1 -- 172.21.15.27:0/2503894026 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ff40c90a0 con 0x7f9ff40ca3a0 2024-09-12T03:17:36.264 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.258+0000 7f9ff0ff9640 1 -- 172.21.15.27:0/2503894026 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fdc00d650 con 0x7f9ff40ca3a0 2024-09-12T03:17:36.264 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.258+0000 7f9ffa406640 1 -- 172.21.15.27:0/2503894026 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ff40c9660 con 0x7f9ff40ca3a0 2024-09-12T03:17:36.264 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.259+0000 7f9ffa406640 1 -- 172.21.15.27:0/2503894026 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ff40d1010 con 0x7f9ff40ca3a0 2024-09-12T03:17:36.264 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.259+0000 7f9ff0ff9640 1 -- 172.21.15.27:0/2503894026 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 7) ==== 90651+0+0 (secure 0 0 0) 0x7f9fdc00d7f0 con 0x7f9ff40ca3a0 2024-09-12T03:17:36.264 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.260+0000 7f9ff0ff9640 1 --2- 172.21.15.27:0/2503894026 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f9fd406c010 0x7f9fd406e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:36.264 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.260+0000 7f9ff0ff9640 1 -- 172.21.15.27:0/2503894026 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 4857+0+0 (secure 0 0 0) 0x7f9fdc08acf0 con 0x7f9ff40ca3a0 2024-09-12T03:17:36.264 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.260+0000 7f9ff3fff640 1 --2- 172.21.15.27:0/2503894026 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f9fd406c010 0x7f9fd406e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:36.264 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.261+0000 7f9ff3fff640 1 --2- 172.21.15.27:0/2503894026 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f9fd406c010 0x7f9fd406e4d0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f9fe4009ab0 tx=0x7f9fe4008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:36.265 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.265+0000 7f9ff0ff9640 1 -- 172.21.15.27:0/2503894026 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f9fdc08f070 con 0x7f9ff40ca3a0 2024-09-12T03:17:36.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.506+0000 7f9ffa406640 1 -- 172.21.15.27:0/2503894026 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f9ff40c7f50 con 0x7f9ff40ca3a0 2024-09-12T03:17:36.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.507+0000 7f9ff0ff9640 1 -- 172.21.15.27:0/2503894026 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v11) ==== 80+0+5304 (secure 0 0 0) 0x7f9fdc059770 con 0x7f9ff40ca3a0 2024-09-12T03:17:36.508 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:17:36.508 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-12T03:17:36.508 INFO:teuthology.orchestra.run.smithi027.stdout: "auth_dump": [ 2024-09-12T03:17:36.508 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:17:36.509 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "mds.a", 2024-09-12T03:17:36.509 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDUXOJmIhxjNxAAaGg8SwtW6NRe/lIgNCV20w==", 2024-09-12T03:17:36.509 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:17:36.509 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow", 2024-09-12T03:17:36.509 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow *", 2024-09-12T03:17:36.509 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow *", 2024-09-12T03:17:36.509 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2024-09-12T03:17:36.509 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:17:36.510 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:17:36.510 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:17:36.510 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "mds.b", 2024-09-12T03:17:36.510 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDVXOJm63kWCBAAzjfOsBsWUI+IAsuZ49yEkw==", 2024-09-12T03:17:36.510 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:17:36.510 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow", 2024-09-12T03:17:36.510 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow *", 2024-09-12T03:17:36.510 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow *", 2024-09-12T03:17:36.510 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2024-09-12T03:17:36.510 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:17:36.510 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:17:36.510 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:17:36.510 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "mds.c", 2024-09-12T03:17:36.511 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDVXOJm0sX7AhAAdynCcho2a/AgFF9Cw3+vAw==", 2024-09-12T03:17:36.511 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:17:36.511 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow", 2024-09-12T03:17:36.511 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow *", 2024-09-12T03:17:36.511 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow *", 2024-09-12T03:17:36.511 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2024-09-12T03:17:36.511 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:17:36.511 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:17:36.511 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:17:36.511 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "mds.d", 2024-09-12T03:17:36.511 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDVXOJm7Dd8DxAANHYmvKgpRZ6sVfXDSgPHWw==", 2024-09-12T03:17:36.511 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:17:36.511 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow", 2024-09-12T03:17:36.512 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow *", 2024-09-12T03:17:36.512 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow *", 2024-09-12T03:17:36.512 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2024-09-12T03:17:36.512 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:17:36.512 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:17:36.512 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:17:36.512 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "osd.0", 2024-09-12T03:17:36.512 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDYXOJmJTVYLxAAmo8kxRjaiwTT44dZLZlq4A==", 2024-09-12T03:17:36.512 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:17:36.512 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow profile osd", 2024-09-12T03:17:36.512 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile osd", 2024-09-12T03:17:36.512 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2024-09-12T03:17:36.512 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:17:36.513 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:17:36.513 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:17:36.513 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "osd.1", 2024-09-12T03:17:36.513 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDcXOJmYV0BFBAAmoGX9gf9yLoE/qkWFnv9tg==", 2024-09-12T03:17:36.513 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:17:36.513 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow profile osd", 2024-09-12T03:17:36.513 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile osd", 2024-09-12T03:17:36.513 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2024-09-12T03:17:36.513 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:17:36.513 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:17:36.513 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:17:36.513 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "osd.2", 2024-09-12T03:17:36.514 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDfXOJmEetiJRAAXk+AvfTfhjsB6T2UB4Z2/w==", 2024-09-12T03:17:36.514 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:17:36.514 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow profile osd", 2024-09-12T03:17:36.514 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile osd", 2024-09-12T03:17:36.514 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2024-09-12T03:17:36.514 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:17:36.514 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:17:36.514 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:17:36.514 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "osd.3", 2024-09-12T03:17:36.514 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDiXOJmrQbPJxAAQsi2B5a1LnPTPGZVdo7j7w==", 2024-09-12T03:17:36.514 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:17:36.514 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow profile osd", 2024-09-12T03:17:36.514 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile osd", 2024-09-12T03:17:36.514 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2024-09-12T03:17:36.515 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:17:36.515 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:17:36.515 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:17:36.515 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "osd.4", 2024-09-12T03:17:36.515 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDnXOJmoMr1NxAAv3u2q8hYt9w//9gz+N7UAQ==", 2024-09-12T03:17:36.515 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:17:36.515 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow profile osd", 2024-09-12T03:17:36.515 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile osd", 2024-09-12T03:17:36.515 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2024-09-12T03:17:36.515 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:17:36.515 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:17:36.515 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:17:36.515 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "osd.5", 2024-09-12T03:17:36.516 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDqXOJmxpUPOxAAxLH6RJsyLOZ2XMfzM0fQ1g==", 2024-09-12T03:17:36.516 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:17:36.516 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow profile osd", 2024-09-12T03:17:36.516 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile osd", 2024-09-12T03:17:36.516 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2024-09-12T03:17:36.516 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:17:36.516 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:17:36.516 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:17:36.516 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "osd.6", 2024-09-12T03:17:36.516 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDuXOJmuP0+LxAAMHoz5eSO81hPtkdBIS2cig==", 2024-09-12T03:17:36.516 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:17:36.516 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow profile osd", 2024-09-12T03:17:36.516 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile osd", 2024-09-12T03:17:36.517 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2024-09-12T03:17:36.517 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:17:36.517 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:17:36.517 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:17:36.517 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "osd.7", 2024-09-12T03:17:36.517 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDxXOJmuIC/MRAAPDM+lAHqdoQyGVODwjFp+Q==", 2024-09-12T03:17:36.517 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:17:36.517 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow profile osd", 2024-09-12T03:17:36.517 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile osd", 2024-09-12T03:17:36.517 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2024-09-12T03:17:36.517 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:17:36.517 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:17:36.517 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:17:36.517 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "client.0", 2024-09-12T03:17:36.518 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDVXOJmLm+5ExAAECM4CKfuVElmp/n0b02mXg==", 2024-09-12T03:17:36.518 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:17:36.518 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow", 2024-09-12T03:17:36.518 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow r", 2024-09-12T03:17:36.518 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-12T03:17:36.518 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:17:36.518 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:17:36.518 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:17:36.518 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "client.1", 2024-09-12T03:17:36.518 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDVXOJm61/FHBAALCQouXySfQeZ5DESGmqkzA==", 2024-09-12T03:17:36.518 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:17:36.518 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow", 2024-09-12T03:17:36.518 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow r", 2024-09-12T03:17:36.519 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow rw", 2024-09-12T03:17:36.519 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow rwx" 2024-09-12T03:17:36.519 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:17:36.519 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:17:36.519 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:17:36.519 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "client.2", 2024-09-12T03:17:36.519 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDVXOJmq/ZkGRAAOzmQzEIhb4b/YTxayc34Bw==", 2024-09-12T03:17:36.519 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:17:36.519 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow", 2024-09-12T03:17:36.519 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow r", 2024-09-12T03:17:36.519 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow rw", 2024-09-12T03:17:36.519 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow rwx" 2024-09-12T03:17:36.519 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:17:36.520 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:17:36.520 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:17:36.520 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "client.3", 2024-09-12T03:17:36.520 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDVXOJm+DlzIhAATQKZe+Zyk0MM8efTa6g9PA==", 2024-09-12T03:17:36.520 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:17:36.520 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow", 2024-09-12T03:17:36.520 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow r", 2024-09-12T03:17:36.520 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow rw", 2024-09-12T03:17:36.520 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow rwx" 2024-09-12T03:17:36.520 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:17:36.520 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:17:36.520 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:17:36.520 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "client.admin", 2024-09-12T03:17:36.520 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDUXOJm+JGWGhAApTQw9RtJWd/J5QzQaUFXvg==", 2024-09-12T03:17:36.521 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:17:36.521 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow *", 2024-09-12T03:17:36.521 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow *", 2024-09-12T03:17:36.521 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow *", 2024-09-12T03:17:36.521 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2024-09-12T03:17:36.521 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:17:36.521 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:17:36.521 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:17:36.521 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "mgr.x", 2024-09-12T03:17:36.521 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDUXOJmZcsONBAAMTY2KykuHaEkv1t6Vz5KjA==", 2024-09-12T03:17:36.521 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:17:36.521 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow *", 2024-09-12T03:17:36.521 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile mgr", 2024-09-12T03:17:36.522 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2024-09-12T03:17:36.522 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:17:36.522 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:17:36.522 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:17:36.522 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "mgr.y", 2024-09-12T03:17:36.522 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDUXOJmJOgZMBAAQ/JTxfhox7HgxBMtsvUCBg==", 2024-09-12T03:17:36.522 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:17:36.522 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow *", 2024-09-12T03:17:36.522 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile mgr", 2024-09-12T03:17:36.522 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2024-09-12T03:17:36.522 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:17:36.522 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:17:36.522 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-12T03:17:36.522 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-12T03:17:36.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.509+0000 7f9ffa406640 1 -- 172.21.15.27:0/2503894026 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f9fd406c010 msgr2=0x7f9fd406e4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:36.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.509+0000 7f9ffa406640 1 --2- 172.21.15.27:0/2503894026 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f9fd406c010 0x7f9fd406e4d0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f9fe4009ab0 tx=0x7f9fe4008040 comp rx=0 tx=0).stop 2024-09-12T03:17:36.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.509+0000 7f9ffa406640 1 -- 172.21.15.27:0/2503894026 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9ff40ca3a0 msgr2=0x7f9ff40c8340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:36.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.509+0000 7f9ffa406640 1 --2- 172.21.15.27:0/2503894026 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9ff40ca3a0 0x7f9ff40c8340 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f9fdc00f6f0 tx=0x7f9fdc00fbb0 comp rx=0 tx=0).stop 2024-09-12T03:17:36.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.509+0000 7f9ffa406640 1 -- 172.21.15.27:0/2503894026 shutdown_connections 2024-09-12T03:17:36.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.509+0000 7f9ffa406640 1 --2- 172.21.15.27:0/2503894026 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f9fd406c010 0x7f9fd406e4d0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:36.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.509+0000 7f9ffa406640 1 --2- 172.21.15.27:0/2503894026 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f9ff40c8880 0x7f9ff41f4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:36.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.509+0000 7f9ffa406640 1 --2- 172.21.15.27:0/2503894026 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9ff40ca3a0 0x7f9ff40c8340 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:36.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.509+0000 7f9ffa406640 1 --2- 172.21.15.27:0/2503894026 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9fe0004d40 0x7f9ff40c9e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:36.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.509+0000 7f9ffa406640 1 -- 172.21.15.27:0/2503894026 >> 172.21.15.27:0/2503894026 conn(0x7f9ff406dce0 msgr2=0x7f9ff4160ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:36.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.509+0000 7f9ffa406640 1 -- 172.21.15.27:0/2503894026 shutdown_connections 2024-09-12T03:17:36.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.509+0000 7f9ffa406640 1 -- 172.21.15.27:0/2503894026 wait complete. 2024-09-12T03:17:36.525 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-12T03:17:36.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.626+0000 7f2a8310a640 1 Processor -- start 2024-09-12T03:17:36.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.626+0000 7f2a8310a640 1 -- start start 2024-09-12T03:17:36.628 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.627+0000 7f2a8310a640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f2a7c15e780 0x7f2a7c0d3e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:36.628 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.627+0000 7f2a8310a640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2a7c0b6690 con 0x7f2a7c0d1990 2024-09-12T03:17:36.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.627+0000 7f2a8310a640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2a7c0b6970 con 0x7f2a7c0d43b0 2024-09-12T03:17:36.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.627+0000 7f2a8310a640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f2a7c0b7950 con 0x7f2a7c15e780 2024-09-12T03:17:36.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.627+0000 7f2a816a7640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f2a7c0d1990 0x7f2a7c0b3390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:46660/0 (socket says 172.21.15.27:46660) 2024-09-12T03:17:36.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.627+0000 7f2a816a7640 1 -- 172.21.15.27:0/2998454588 learned_addr learned my addr 172.21.15.27:0/2998454588 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:17:36.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.627+0000 7f2a80ea6640 1 --2- 172.21.15.27:0/2998454588 >> v2:172.21.15.45:3300/0 conn(0x7f2a7c15e780 0x7f2a7c0d3e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:36.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.627+0000 7f2a737fe640 1 -- 172.21.15.27:0/2998454588 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1784841666 0 0) 0x7f2a7c0b6690 con 0x7f2a7c0d1990 2024-09-12T03:17:36.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.627+0000 7f2a737fe640 1 -- 172.21.15.27:0/2998454588 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2a64003520 con 0x7f2a7c0d1990 2024-09-12T03:17:36.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.627+0000 7f2a737fe640 1 -- 172.21.15.27:0/2998454588 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1803485256 0 0) 0x7f2a64003520 con 0x7f2a7c0d1990 2024-09-12T03:17:36.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.627+0000 7f2a80ea6640 1 -- 172.21.15.27:0/2998454588 >> v1:172.21.15.45:6789/0 conn(0x7f2a7c0d43b0 legacy=0x7f2a7c0d1280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:36.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.627+0000 7f2a80ea6640 1 -- 172.21.15.27:0/2998454588 >> v1:172.21.15.27:6789/0 conn(0x7f2a7c0d1990 legacy=0x7f2a7c0b3390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:36.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.627+0000 7f2a80ea6640 1 -- 172.21.15.27:0/2998454588 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a7c0b6c50 con 0x7f2a7c15e780 2024-09-12T03:17:36.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.628+0000 7f2a80ea6640 1 --2- 172.21.15.27:0/2998454588 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2a7c15e780 0x7f2a7c0d3e70 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f2a78009d90 tx=0x7f2a7802a5f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d594fa5b79309c85 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:36.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.628+0000 7f2a737fe640 1 -- 172.21.15.27:0/2998454588 <== mon.1 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a74001670 con 0x7f2a7c15e780 2024-09-12T03:17:36.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.628+0000 7f2a737fe640 1 -- 172.21.15.27:0/2998454588 <== mon.1 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a78007540 con 0x7f2a7c15e780 2024-09-12T03:17:36.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.628+0000 7f2a737fe640 1 -- 172.21.15.27:0/2998454588 <== mon.1 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a780078f0 con 0x7f2a7c15e780 2024-09-12T03:17:36.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.629+0000 7f2a8310a640 1 -- 172.21.15.27:0/2998454588 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2a7c15e780 msgr2=0x7f2a7c0d3e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:36.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.629+0000 7f2a8310a640 1 --2- 172.21.15.27:0/2998454588 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2a7c15e780 0x7f2a7c0d3e70 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f2a78009d90 tx=0x7f2a7802a5f0 comp rx=0 tx=0).stop 2024-09-12T03:17:36.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.629+0000 7f2a8310a640 1 -- 172.21.15.27:0/2998454588 shutdown_connections 2024-09-12T03:17:36.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.629+0000 7f2a8310a640 1 --2- 172.21.15.27:0/2998454588 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2a7c15e780 0x7f2a7c0d3e70 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:36.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.629+0000 7f2a8310a640 1 -- 172.21.15.27:0/2998454588 >> 172.21.15.27:0/2998454588 conn(0x7f2a7c06da60 msgr2=0x7f2a7c06de70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:36.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.629+0000 7f2a8310a640 1 -- 172.21.15.27:0/2998454588 shutdown_connections 2024-09-12T03:17:36.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.629+0000 7f2a8310a640 1 -- 172.21.15.27:0/2998454588 wait complete. 2024-09-12T03:17:36.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.630+0000 7f2a8310a640 1 Processor -- start 2024-09-12T03:17:36.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.630+0000 7f2a8310a640 1 -- start start 2024-09-12T03:17:36.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.631+0000 7f2a8310a640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2a7c0d1990 0x7f2a7c0ca730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:36.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.631+0000 7f2a8310a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2a7c0d43b0 0x7f2a7c0c8e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:36.634 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.631+0000 7f2a8310a640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2a7c15e780 0x7f2a7c0c9340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:36.634 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.631+0000 7f2a8310a640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f2a7c0b6340 con 0x7f2a7c15e780 2024-09-12T03:17:36.634 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.631+0000 7f2a8310a640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f2a7c0b7210 con 0x7f2a7c0d43b0 2024-09-12T03:17:36.634 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.631+0000 7f2a8310a640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f2a7c0b8dd0 con 0x7f2a7c0d1990 2024-09-12T03:17:36.634 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.631+0000 7f2a816a7640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2a7c15e780 0x7f2a7c0c9340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:36.634 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.631+0000 7f2a816a7640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2a7c15e780 0x7f2a7c0c9340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:48340/0 (socket says 172.21.15.27:48340) 2024-09-12T03:17:36.634 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.631+0000 7f2a816a7640 1 -- 172.21.15.27:0/3890240371 learned_addr learned my addr 172.21.15.27:0/3890240371 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:17:36.634 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.631+0000 7f2a73fff640 1 --2- 172.21.15.27:0/3890240371 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2a7c0d43b0 0x7f2a7c0c8e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:36.634 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.631+0000 7f2a80ea6640 1 --2- 172.21.15.27:0/3890240371 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2a7c0d1990 0x7f2a7c0ca730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:36.634 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.631+0000 7f2a816a7640 1 -- 172.21.15.27:0/3890240371 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2a7c0d1990 msgr2=0x7f2a7c0ca730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:36.634 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.631+0000 7f2a816a7640 1 --2- 172.21.15.27:0/3890240371 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2a7c0d1990 0x7f2a7c0ca730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:36.634 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.631+0000 7f2a816a7640 1 -- 172.21.15.27:0/3890240371 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2a7c0d43b0 msgr2=0x7f2a7c0c8e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:36.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.631+0000 7f2a816a7640 1 --2- 172.21.15.27:0/3890240371 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2a7c0d43b0 0x7f2a7c0c8e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:36.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.632+0000 7f2a816a7640 1 -- 172.21.15.27:0/3890240371 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a7c0c99e0 con 0x7f2a7c15e780 2024-09-12T03:17:36.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.632+0000 7f2a80ea6640 1 --2- 172.21.15.27:0/3890240371 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2a7c0d1990 0x7f2a7c0ca730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:17:36.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.632+0000 7f2a816a7640 1 --2- 172.21.15.27:0/3890240371 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2a7c15e780 0x7f2a7c0c9340 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f2a7400c750 tx=0x7f2a7400cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:36.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.632+0000 7f2a717fa640 1 -- 172.21.15.27:0/3890240371 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a74017070 con 0x7f2a7c15e780 2024-09-12T03:17:36.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.632+0000 7f2a717fa640 1 -- 172.21.15.27:0/3890240371 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a74005ad0 con 0x7f2a7c15e780 2024-09-12T03:17:36.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.632+0000 7f2a8310a640 1 -- 172.21.15.27:0/3890240371 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a7c0c9cc0 con 0x7f2a7c15e780 2024-09-12T03:17:36.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.632+0000 7f2a717fa640 1 -- 172.21.15.27:0/3890240371 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a74007070 con 0x7f2a7c15e780 2024-09-12T03:17:36.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.632+0000 7f2a8310a640 1 -- 172.21.15.27:0/3890240371 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a7c0ab310 con 0x7f2a7c15e780 2024-09-12T03:17:36.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.633+0000 7f2a8310a640 1 -- 172.21.15.27:0/3890240371 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a7c15eb60 con 0x7f2a7c15e780 2024-09-12T03:17:36.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.633+0000 7f2a717fa640 1 -- 172.21.15.27:0/3890240371 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 7) ==== 90651+0+0 (secure 0 0 0) 0x7f2a7401a070 con 0x7f2a7c15e780 2024-09-12T03:17:36.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.634+0000 7f2a717fa640 1 --2- 172.21.15.27:0/3890240371 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f2a6806bfe0 0x7f2a6806e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:36.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.634+0000 7f2a717fa640 1 -- 172.21.15.27:0/3890240371 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 4857+0+0 (secure 0 0 0) 0x7f2a7408a2d0 con 0x7f2a7c15e780 2024-09-12T03:17:36.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.634+0000 7f2a80ea6640 1 --2- 172.21.15.27:0/3890240371 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f2a6806bfe0 0x7f2a6806e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:36.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.635+0000 7f2a80ea6640 1 --2- 172.21.15.27:0/3890240371 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f2a6806bfe0 0x7f2a6806e4a0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f2a78034000 tx=0x7f2a7802f000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:36.639 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.638+0000 7f2a717fa640 1 -- 172.21.15.27:0/3890240371 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f2a74058660 con 0x7f2a7c15e780 2024-09-12T03:17:36.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.816+0000 7f2a8310a640 1 -- 172.21.15.27:0/3890240371 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f2a7c069ff0 con 0x7f2a7c15e780 2024-09-12T03:17:36.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.817+0000 7f2a717fa640 1 -- 172.21.15.27:0/3890240371 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v39) ==== 84+0+65 (secure 0 0 0) 0x7f2a7408a0d0 con 0x7f2a7c15e780 2024-09-12T03:17:36.818 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:17:36.818 INFO:teuthology.orchestra.run.smithi027.stdout:[ 2024-09-12T03:17:36.818 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:17:36.818 INFO:teuthology.orchestra.run.smithi027.stdout: "poolnum": 1, 2024-09-12T03:17:36.818 INFO:teuthology.orchestra.run.smithi027.stdout: "poolname": ".mgr" 2024-09-12T03:17:36.818 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:17:36.818 INFO:teuthology.orchestra.run.smithi027.stdout:] 2024-09-12T03:17:36.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.818+0000 7f2a8310a640 1 -- 172.21.15.27:0/3890240371 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f2a6806bfe0 msgr2=0x7f2a6806e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:36.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.818+0000 7f2a8310a640 1 --2- 172.21.15.27:0/3890240371 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f2a6806bfe0 0x7f2a6806e4a0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f2a78034000 tx=0x7f2a7802f000 comp rx=0 tx=0).stop 2024-09-12T03:17:36.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.818+0000 7f2a8310a640 1 -- 172.21.15.27:0/3890240371 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2a7c15e780 msgr2=0x7f2a7c0c9340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:36.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.819+0000 7f2a8310a640 1 --2- 172.21.15.27:0/3890240371 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2a7c15e780 0x7f2a7c0c9340 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f2a7400c750 tx=0x7f2a7400cc10 comp rx=0 tx=0).stop 2024-09-12T03:17:36.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.819+0000 7f2a8310a640 1 -- 172.21.15.27:0/3890240371 shutdown_connections 2024-09-12T03:17:36.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.819+0000 7f2a8310a640 1 --2- 172.21.15.27:0/3890240371 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f2a6806bfe0 0x7f2a6806e4a0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:36.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.819+0000 7f2a8310a640 1 --2- 172.21.15.27:0/3890240371 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2a7c15e780 0x7f2a7c0c9340 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:36.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.819+0000 7f2a8310a640 1 --2- 172.21.15.27:0/3890240371 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2a7c0d43b0 0x7f2a7c0c8e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:36.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.819+0000 7f2a8310a640 1 --2- 172.21.15.27:0/3890240371 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2a7c0d1990 0x7f2a7c0ca730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:36.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.819+0000 7f2a8310a640 1 -- 172.21.15.27:0/3890240371 >> 172.21.15.27:0/3890240371 conn(0x7f2a7c06da60 msgr2=0x7f2a7c0cec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:36.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.819+0000 7f2a8310a640 1 -- 172.21.15.27:0/3890240371 shutdown_connections 2024-09-12T03:17:36.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.819+0000 7f2a8310a640 1 -- 172.21.15.27:0/3890240371 wait complete. 2024-09-12T03:17:36.830 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-12T03:17:36.830 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-12T03:17:36.936 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.935+0000 7f9fe3c1e640 1 Processor -- start 2024-09-12T03:17:36.936 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.935+0000 7f9fe3c1e640 1 -- start start 2024-09-12T03:17:36.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.936+0000 7f9fe3c1e640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f9fdc15e650 0x7f9fdc0d1d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:36.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.936+0000 7f9fe3c1e640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9fdc0d2280 con 0x7f9fdc06cc40 2024-09-12T03:17:36.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.936+0000 7f9fe3c1e640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9fdc06e670 con 0x7f9fdc06db30 2024-09-12T03:17:36.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.936+0000 7f9fe3c1e640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f9fdc06d850 con 0x7f9fdc15e650 2024-09-12T03:17:36.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.936+0000 7f9fe11b9640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f9fdc06cc40 0x7f9fdc06d090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:46672/0 (socket says 172.21.15.27:46672) 2024-09-12T03:17:36.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.936+0000 7f9fe11b9640 1 -- 172.21.15.27:0/3785605204 learned_addr learned my addr 172.21.15.27:0/3785605204 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:17:36.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.936+0000 7f9fe19ba640 1 --2- 172.21.15.27:0/3785605204 >> v2:172.21.15.45:3300/0 conn(0x7f9fdc15e650 0x7f9fdc0d1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:36.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.936+0000 7f9fe09b8640 1 -- 172.21.15.27:0/3785605204 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2569310684 0 0) 0x7f9fdc0d2280 con 0x7f9fdc06cc40 2024-09-12T03:17:36.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.937+0000 7f9fe09b8640 1 -- 172.21.15.27:0/3785605204 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9fc4003520 con 0x7f9fdc06cc40 2024-09-12T03:17:36.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.937+0000 7f9fe09b8640 1 -- 172.21.15.27:0/3785605204 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1808682985 0 0) 0x7f9fc4003520 con 0x7f9fdc06cc40 2024-09-12T03:17:36.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.937+0000 7f9fe19ba640 1 -- 172.21.15.27:0/3785605204 >> v1:172.21.15.45:6789/0 conn(0x7f9fdc06db30 legacy=0x7f9fdc06df60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:36.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.937+0000 7f9fe19ba640 1 -- 172.21.15.27:0/3785605204 >> v1:172.21.15.27:6789/0 conn(0x7f9fdc06cc40 legacy=0x7f9fdc06d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:36.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.937+0000 7f9fe19ba640 1 -- 172.21.15.27:0/3785605204 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fdc0d1260 con 0x7f9fdc15e650 2024-09-12T03:17:36.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.938+0000 7f9fe19ba640 1 --2- 172.21.15.27:0/3785605204 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9fdc15e650 0x7f9fdc0d1d40 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f9fcc009d90 tx=0x7f9fcc02a5f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5d741f8b6fb6e748 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:36.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.938+0000 7f9fe09b8640 1 -- 172.21.15.27:0/3785605204 <== mon.1 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fd0001670 con 0x7f9fdc15e650 2024-09-12T03:17:36.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.938+0000 7f9fe09b8640 1 -- 172.21.15.27:0/3785605204 <== mon.1 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9fcc007540 con 0x7f9fdc15e650 2024-09-12T03:17:36.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.938+0000 7f9fe09b8640 1 -- 172.21.15.27:0/3785605204 <== mon.1 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fcc0078f0 con 0x7f9fdc15e650 2024-09-12T03:17:36.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.938+0000 7f9fe3c1e640 1 -- 172.21.15.27:0/3785605204 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9fdc15e650 msgr2=0x7f9fdc0d1d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:36.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.938+0000 7f9fe3c1e640 1 --2- 172.21.15.27:0/3785605204 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9fdc15e650 0x7f9fdc0d1d40 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f9fcc009d90 tx=0x7f9fcc02a5f0 comp rx=0 tx=0).stop 2024-09-12T03:17:36.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.938+0000 7f9fe3c1e640 1 -- 172.21.15.27:0/3785605204 shutdown_connections 2024-09-12T03:17:36.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.938+0000 7f9fe3c1e640 1 --2- 172.21.15.27:0/3785605204 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9fdc15e650 0x7f9fdc0d1d40 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:36.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.938+0000 7f9fe3c1e640 1 -- 172.21.15.27:0/3785605204 >> 172.21.15.27:0/3785605204 conn(0x7f9fdc06f320 msgr2=0x7f9fdc06f700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:36.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.938+0000 7f9fe3c1e640 1 -- 172.21.15.27:0/3785605204 shutdown_connections 2024-09-12T03:17:36.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.938+0000 7f9fe3c1e640 1 -- 172.21.15.27:0/3785605204 wait complete. 2024-09-12T03:17:36.943 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.940+0000 7f9fe3c1e640 1 Processor -- start 2024-09-12T03:17:36.943 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.940+0000 7f9fe3c1e640 1 -- start start 2024-09-12T03:17:36.943 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.940+0000 7f9fe3c1e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9fdc06cc40 0x7f9fdc1f4850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:36.943 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.940+0000 7f9fe3c1e640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f9fdc06db30 0x7f9fdc1f4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:36.943 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.940+0000 7f9fe3c1e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9fdc15e650 0x7f9fdc1f52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:36.943 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.940+0000 7f9fe3c1e640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f9fdc15ebb0 con 0x7f9fdc15e650 2024-09-12T03:17:36.943 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.940+0000 7f9fe3c1e640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f9fdc15ea30 con 0x7f9fdc06cc40 2024-09-12T03:17:36.943 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.940+0000 7f9fe3c1e640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f9fdc0b55b0 con 0x7f9fdc06db30 2024-09-12T03:17:36.943 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.940+0000 7f9fe21bb640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9fdc15e650 0x7f9fdc1f52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:36.943 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.941+0000 7f9fe21bb640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9fdc15e650 0x7f9fdc1f52d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:48348/0 (socket says 172.21.15.27:48348) 2024-09-12T03:17:36.943 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.941+0000 7f9fe19ba640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9fdc06cc40 0x7f9fdc1f4850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:36.943 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.941+0000 7f9fe21bb640 1 -- 172.21.15.27:0/1823628522 learned_addr learned my addr 172.21.15.27:0/1823628522 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:17:36.944 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.941+0000 7f9fe11b9640 1 --2- 172.21.15.27:0/1823628522 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f9fdc06db30 0x7f9fdc1f4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:36.944 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.941+0000 7f9fe21bb640 1 -- 172.21.15.27:0/1823628522 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f9fdc06db30 msgr2=0x7f9fdc1f4d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:36.944 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.941+0000 7f9fe21bb640 1 --2- 172.21.15.27:0/1823628522 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f9fdc06db30 0x7f9fdc1f4d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:36.944 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.941+0000 7f9fe21bb640 1 -- 172.21.15.27:0/1823628522 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9fdc06cc40 msgr2=0x7f9fdc1f4850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:36.944 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.941+0000 7f9fe21bb640 1 --2- 172.21.15.27:0/1823628522 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9fdc06cc40 0x7f9fdc1f4850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:36.944 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.941+0000 7f9fe21bb640 1 -- 172.21.15.27:0/1823628522 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fdc158880 con 0x7f9fdc15e650 2024-09-12T03:17:36.944 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.941+0000 7f9fe21bb640 1 --2- 172.21.15.27:0/1823628522 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9fdc15e650 0x7f9fdc1f52d0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f9fd400c3f0 tx=0x7f9fd400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:36.944 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.941+0000 7f9fda7fc640 1 -- 172.21.15.27:0/1823628522 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fd4017070 con 0x7f9fdc15e650 2024-09-12T03:17:36.944 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.941+0000 7f9fda7fc640 1 -- 172.21.15.27:0/1823628522 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9fd40052d0 con 0x7f9fdc15e650 2024-09-12T03:17:36.944 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.941+0000 7f9fe3c1e640 1 -- 172.21.15.27:0/1823628522 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9fdc158b60 con 0x7f9fdc15e650 2024-09-12T03:17:36.944 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.941+0000 7f9fda7fc640 1 -- 172.21.15.27:0/1823628522 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fd4003830 con 0x7f9fdc15e650 2024-09-12T03:17:36.944 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.941+0000 7f9fe3c1e640 1 -- 172.21.15.27:0/1823628522 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9fdc159120 con 0x7f9fdc15e650 2024-09-12T03:17:36.945 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.942+0000 7f9fe3c1e640 1 -- 172.21.15.27:0/1823628522 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9fdc0d1d40 con 0x7f9fdc15e650 2024-09-12T03:17:36.945 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.943+0000 7f9fda7fc640 1 -- 172.21.15.27:0/1823628522 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 7) ==== 90651+0+0 (secure 0 0 0) 0x7f9fd401a070 con 0x7f9fdc15e650 2024-09-12T03:17:36.946 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.943+0000 7f9fda7fc640 1 --2- 172.21.15.27:0/1823628522 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f9fc806bfe0 0x7f9fc806e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:36.946 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.943+0000 7f9fda7fc640 1 -- 172.21.15.27:0/1823628522 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 4857+0+0 (secure 0 0 0) 0x7f9fd4089fe0 con 0x7f9fdc15e650 2024-09-12T03:17:36.946 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.943+0000 7f9fe19ba640 1 --2- 172.21.15.27:0/1823628522 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f9fc806bfe0 0x7f9fc806e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:36.946 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.944+0000 7f9fe19ba640 1 --2- 172.21.15.27:0/1823628522 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f9fc806bfe0 0x7f9fc806e4a0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f9fcc034000 tx=0x7f9fcc02f000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:36.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:36.947+0000 7f9fda7fc640 1 -- 172.21.15.27:0/1823628522 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f9fd40584d0 con 0x7f9fdc15e650 2024-09-12T03:17:37.124 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:37.123+0000 7f9fe3c1e640 1 -- 172.21.15.27:0/1823628522 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f9fdc0d09a0 con 0x7f9fdc15e650 2024-09-12T03:17:37.206 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:37.205+0000 7f9fda7fc640 1 -- 172.21.15.27:0/1823628522 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v40) ==== 138+0+0 (secure 0 0 0) 0x7f9fd405c7d0 con 0x7f9fdc15e650 2024-09-12T03:17:37.206 INFO:teuthology.orchestra.run.smithi027.stderr:pool 'cephfs_metadata' created 2024-09-12T03:17:37.209 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:37.207+0000 7f9fe3c1e640 1 -- 172.21.15.27:0/1823628522 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f9fc806bfe0 msgr2=0x7f9fc806e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:37.209 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:37.207+0000 7f9fe3c1e640 1 --2- 172.21.15.27:0/1823628522 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f9fc806bfe0 0x7f9fc806e4a0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f9fcc034000 tx=0x7f9fcc02f000 comp rx=0 tx=0).stop 2024-09-12T03:17:37.209 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:37.207+0000 7f9fe3c1e640 1 -- 172.21.15.27:0/1823628522 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9fdc15e650 msgr2=0x7f9fdc1f52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:37.209 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:37.207+0000 7f9fe3c1e640 1 --2- 172.21.15.27:0/1823628522 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9fdc15e650 0x7f9fdc1f52d0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f9fd400c3f0 tx=0x7f9fd400c8b0 comp rx=0 tx=0).stop 2024-09-12T03:17:37.209 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:37.208+0000 7f9fe3c1e640 1 -- 172.21.15.27:0/1823628522 shutdown_connections 2024-09-12T03:17:37.209 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:37.208+0000 7f9fe3c1e640 1 --2- 172.21.15.27:0/1823628522 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f9fc806bfe0 0x7f9fc806e4a0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:37.209 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:37.208+0000 7f9fe3c1e640 1 --2- 172.21.15.27:0/1823628522 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9fdc15e650 0x7f9fdc1f52d0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:37.210 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:37.208+0000 7f9fe3c1e640 1 --2- 172.21.15.27:0/1823628522 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f9fdc06db30 0x7f9fdc1f4d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:37.210 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:37.208+0000 7f9fe3c1e640 1 --2- 172.21.15.27:0/1823628522 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9fdc06cc40 0x7f9fdc1f4850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:37.210 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:37.208+0000 7f9fe3c1e640 1 -- 172.21.15.27:0/1823628522 >> 172.21.15.27:0/1823628522 conn(0x7f9fdc06f320 msgr2=0x7f9fdc06f700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:37.210 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:37.208+0000 7f9fe3c1e640 1 -- 172.21.15.27:0/1823628522 shutdown_connections 2024-09-12T03:17:37.210 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:37.209+0000 7f9fe3c1e640 1 -- 172.21.15.27:0/1823628522 wait complete. 2024-09-12T03:17:37.223 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-12T03:17:37.333 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:37.333+0000 7f38c2b98640 1 Processor -- start 2024-09-12T03:17:37.333 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:37.333+0000 7f38c2b98640 1 -- start start 2024-09-12T03:17:37.334 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:37.333+0000 7f38c2b98640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f38bc15e990 0x7f38bc0b7600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:37.334 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:37.333+0000 7f38c2b98640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f38bc0b6230 0x7f38bc0b66c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:37.334 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:37.333+0000 7f38c2b98640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f38bc15ed70 con 0x7f38bc0b3600 2024-09-12T03:17:37.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:37.333+0000 7f38c2b98640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f38bc0d2a30 con 0x7f38bc15e990 2024-09-12T03:17:37.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:37.333+0000 7f38c2b98640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f38bc0d2d10 con 0x7f38bc0b6230 2024-09-12T03:17:37.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:37.333+0000 7f38c0934640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f38bc15e990 0x7f38bc0b7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:37.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:37.333+0000 7f38c0934640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f38bc15e990 0x7f38bc0b7600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:48362/0 (socket says 172.21.15.27:48362) 2024-09-12T03:17:37.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:37.333+0000 7f38c0934640 1 -- 172.21.15.27:0/2826457104 learned_addr learned my addr 172.21.15.27:0/2826457104 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:17:37.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:37.333+0000 7f38c0934640 1 -- 172.21.15.27:0/2826457104 >> v2:172.21.15.45:3300/0 conn(0x7f38bc0b6230 msgr2=0x7f38bc0b66c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:37.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:37.333+0000 7f38bbfff640 1 --2- 172.21.15.27:0/2826457104 >> v2:172.21.15.45:3300/0 conn(0x7f38bc0b6230 0x7f38bc0b66c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:37.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:37.333+0000 7f38c0934640 1 --2- 172.21.15.27:0/2826457104 >> v2:172.21.15.45:3300/0 conn(0x7f38bc0b6230 0x7f38bc0b66c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:37.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:37.333+0000 7f38c0934640 1 -- 172.21.15.27:0/2826457104 >> v1:172.21.15.45:6789/0 conn(0x7f38bc0b3600 legacy=0x7f38bc0b3a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:37.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:37.333+0000 7f38c0934640 1 -- 172.21.15.27:0/2826457104 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38bc0d2ff0 con 0x7f38bc15e990 2024-09-12T03:17:37.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:37.333+0000 7f38bbfff640 1 --2- 172.21.15.27:0/2826457104 >> v2:172.21.15.45:3300/0 conn(0x7f38bc0b6230 0x7f38bc0b66c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:17:37.337 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:37.334+0000 7f38c0934640 1 --2- 172.21.15.27:0/2826457104 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f38bc15e990 0x7f38bc0b7600 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f38bc0d2310 tx=0x7f38b002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57c7a05b86c48d96 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:37.337 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:37.334+0000 7f38bb7fe640 1 -- 172.21.15.27:0/2826457104 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38b0037070 con 0x7f38bc15e990 2024-09-12T03:17:37.337 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:37.334+0000 7f38bb7fe640 1 -- 172.21.15.27:0/2826457104 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38b0032440 con 0x7f38bc15e990 2024-09-12T03:17:37.337 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:37.334+0000 7f38bb7fe640 1 -- 172.21.15.27:0/2826457104 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38b0032810 con 0x7f38bc15e990 2024-09-12T03:17:37.337 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:37.335+0000 7f38c2b98640 1 -- 172.21.15.27:0/2826457104 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f38bc15e990 msgr2=0x7f38bc0b7600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:37.337 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:37.335+0000 7f38c2b98640 1 --2- 172.21.15.27:0/2826457104 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f38bc15e990 0x7f38bc0b7600 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f38bc0d2310 tx=0x7f38b002a230 comp rx=0 tx=0).stop 2024-09-12T03:17:37.337 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:37.335+0000 7f38c2b98640 1 -- 172.21.15.27:0/2826457104 shutdown_connections 2024-09-12T03:17:37.337 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:37.335+0000 7f38c2b98640 1 --2- 172.21.15.27:0/2826457104 >> v2:172.21.15.45:3300/0 conn(0x7f38bc0b6230 0x7f38bc0b66c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:37.337 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:37.335+0000 7f38c2b98640 1 --2- 172.21.15.27:0/2826457104 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f38bc15e990 0x7f38bc0b7600 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:37.337 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:37.335+0000 7f38c2b98640 1 -- 172.21.15.27:0/2826457104 >> 172.21.15.27:0/2826457104 conn(0x7f38bc0d1230 msgr2=0x7f38bc0d1640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:37.337 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:37.335+0000 7f38c2b98640 1 -- 172.21.15.27:0/2826457104 shutdown_connections 2024-09-12T03:17:37.337 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:37.335+0000 7f38c2b98640 1 -- 172.21.15.27:0/2826457104 wait complete. 2024-09-12T03:17:37.338 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:37.336+0000 7f38c2b98640 1 Processor -- start 2024-09-12T03:17:37.338 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:37.336+0000 7f38c2b98640 1 -- start start 2024-09-12T03:17:37.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:37.337+0000 7f38c2b98640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f38bc0b3600 0x7f38bc1f3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:37.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:37.337+0000 7f38c2b98640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f38bc0b6230 0x7f38bc0ca6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:37.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:37.337+0000 7f38c2b98640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f38bc15e990 0x7f38bc0cac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:37.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:37.337+0000 7f38c2b98640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f38bc0b7b40 con 0x7f38bc0b6230 2024-09-12T03:17:37.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:37.337+0000 7f38c2b98640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f38bc0d35b0 con 0x7f38bc15e990 2024-09-12T03:17:37.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:37.337+0000 7f38c2b98640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f38bc0b7cc0 con 0x7f38bc0b3600 2024-09-12T03:17:37.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:37.337+0000 7f38bbfff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f38bc0b6230 0x7f38bc0ca6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:37.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:37.337+0000 7f38bbfff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f38bc0b6230 0x7f38bc0ca6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:48364/0 (socket says 172.21.15.27:48364) 2024-09-12T03:17:37.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:37.337+0000 7f38bbfff640 1 -- 172.21.15.27:0/2365755312 learned_addr learned my addr 172.21.15.27:0/2365755312 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:17:37.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:37.337+0000 7f38c1135640 1 --2- 172.21.15.27:0/2365755312 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f38bc15e990 0x7f38bc0cac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:37.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:37.337+0000 7f38c0934640 1 --2- 172.21.15.27:0/2365755312 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f38bc0b3600 0x7f38bc1f3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:37.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:37.337+0000 7f38bbfff640 1 -- 172.21.15.27:0/2365755312 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f38bc0b3600 msgr2=0x7f38bc1f3740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:37.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:37.337+0000 7f38bbfff640 1 --2- 172.21.15.27:0/2365755312 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f38bc0b3600 0x7f38bc1f3740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:37.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:37.337+0000 7f38bbfff640 1 -- 172.21.15.27:0/2365755312 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f38bc15e990 msgr2=0x7f38bc0cac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:37.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:37.337+0000 7f38bbfff640 1 --2- 172.21.15.27:0/2365755312 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f38bc15e990 0x7f38bc0cac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:37.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:37.337+0000 7f38bbfff640 1 -- 172.21.15.27:0/2365755312 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38bc0c8ef0 con 0x7f38bc0b6230 2024-09-12T03:17:37.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:37.338+0000 7f38bbfff640 1 --2- 172.21.15.27:0/2365755312 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f38bc0b6230 0x7f38bc0ca6f0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f38a800c7f0 tx=0x7f38a800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:37.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:37.338+0000 7f38b97fa640 1 -- 172.21.15.27:0/2365755312 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38a8015070 con 0x7f38bc0b6230 2024-09-12T03:17:37.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:37.338+0000 7f38b97fa640 1 -- 172.21.15.27:0/2365755312 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38a8007a30 con 0x7f38bc0b6230 2024-09-12T03:17:37.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:37.338+0000 7f38c2b98640 1 -- 172.21.15.27:0/2365755312 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38bc0c91d0 con 0x7f38bc0b6230 2024-09-12T03:17:37.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:37.338+0000 7f38b97fa640 1 -- 172.21.15.27:0/2365755312 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38a8007e00 con 0x7f38bc0b6230 2024-09-12T03:17:37.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:37.338+0000 7f38c2b98640 1 -- 172.21.15.27:0/2365755312 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38bc0c9790 con 0x7f38bc0b6230 2024-09-12T03:17:37.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:37.340+0000 7f38c2b98640 1 -- 172.21.15.27:0/2365755312 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38bc0b7600 con 0x7f38bc0b6230 2024-09-12T03:17:37.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:37.340+0000 7f38b97fa640 1 -- 172.21.15.27:0/2365755312 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 7) ==== 90651+0+0 (secure 0 0 0) 0x7f38a8018070 con 0x7f38bc0b6230 2024-09-12T03:17:37.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:37.341+0000 7f38b97fa640 1 --2- 172.21.15.27:0/2365755312 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f38a006bfe0 0x7f38a006e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:37.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:37.341+0000 7f38b97fa640 1 -- 172.21.15.27:0/2365755312 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 5227+0+0 (secure 0 0 0) 0x7f38a808aea0 con 0x7f38bc0b6230 2024-09-12T03:17:37.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:37.341+0000 7f38c0934640 1 --2- 172.21.15.27:0/2365755312 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f38a006bfe0 0x7f38a006e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:37.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:37.342+0000 7f38c0934640 1 --2- 172.21.15.27:0/2365755312 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f38a006bfe0 0x7f38a006e4a0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f38b0034000 tx=0x7f38b002e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:37.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:37.346+0000 7f38b97fa640 1 -- 172.21.15.27:0/2365755312 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f38a8059190 con 0x7f38bc0b6230 2024-09-12T03:17:37.522 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:37.522+0000 7f38c2b98640 1 -- 172.21.15.27:0/2365755312 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f38bc1f3c80 con 0x7f38bc0b6230 2024-09-12T03:17:38.212 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:38.210+0000 7f38b97fa640 1 -- 172.21.15.27:0/2365755312 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v41) ==== 170+0+0 (secure 0 0 0) 0x7f38a805d490 con 0x7f38bc0b6230 2024-09-12T03:17:38.212 INFO:teuthology.orchestra.run.smithi027.stderr:pool 'cephfs_data' created 2024-09-12T03:17:38.215 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:38.215+0000 7f38c2b98640 1 -- 172.21.15.27:0/2365755312 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f38a006bfe0 msgr2=0x7f38a006e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:38.216 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:38.215+0000 7f38c2b98640 1 --2- 172.21.15.27:0/2365755312 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f38a006bfe0 0x7f38a006e4a0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f38b0034000 tx=0x7f38b002e000 comp rx=0 tx=0).stop 2024-09-12T03:17:38.216 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:38.215+0000 7f38c2b98640 1 -- 172.21.15.27:0/2365755312 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f38bc0b6230 msgr2=0x7f38bc0ca6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:38.216 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:38.215+0000 7f38c2b98640 1 --2- 172.21.15.27:0/2365755312 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f38bc0b6230 0x7f38bc0ca6f0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f38a800c7f0 tx=0x7f38a800cbf0 comp rx=0 tx=0).stop 2024-09-12T03:17:38.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:38.216+0000 7f38c2b98640 1 -- 172.21.15.27:0/2365755312 shutdown_connections 2024-09-12T03:17:38.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:38.216+0000 7f38c2b98640 1 --2- 172.21.15.27:0/2365755312 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f38a006bfe0 0x7f38a006e4a0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:38.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:38.216+0000 7f38c2b98640 1 --2- 172.21.15.27:0/2365755312 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f38bc15e990 0x7f38bc0cac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:38.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:38.216+0000 7f38c2b98640 1 --2- 172.21.15.27:0/2365755312 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f38bc0b6230 0x7f38bc0ca6f0 secure :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f38a800c7f0 tx=0x7f38a800cbf0 comp rx=0 tx=0).stop 2024-09-12T03:17:38.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:38.216+0000 7f38c2b98640 1 --2- 172.21.15.27:0/2365755312 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f38bc0b3600 0x7f38bc1f3740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:38.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:38.216+0000 7f38c2b98640 1 -- 172.21.15.27:0/2365755312 >> 172.21.15.27:0/2365755312 conn(0x7f38bc0d1230 msgr2=0x7f38bc0d1610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:38.223 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:38.223+0000 7f38c2b98640 1 -- 172.21.15.27:0/2365755312 shutdown_connections 2024-09-12T03:17:38.223 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:38.224+0000 7f38c2b98640 1 -- 172.21.15.27:0/2365755312 wait complete. 2024-09-12T03:17:38.243 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-12T03:17:38.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:38.354+0000 7f0663aed640 1 Processor -- start 2024-09-12T03:17:38.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:38.354+0000 7f0663aed640 1 -- start start 2024-09-12T03:17:38.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:38.355+0000 7f0663aed640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f065c06cc40 0x7f065c06d060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:38.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:38.355+0000 7f0663aed640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f065c06db30 0x7f065c06df80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:38.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:38.355+0000 7f0663aed640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f065c06d5a0 con 0x7f065c15e650 2024-09-12T03:17:38.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:38.355+0000 7f0663aed640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f065c06d850 con 0x7f065c06db30 2024-09-12T03:17:38.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:38.355+0000 7f0663aed640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f065c0d1260 con 0x7f065c06cc40 2024-09-12T03:17:38.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:38.355+0000 7f0661889640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f065c15e650 0x7f065c0d1d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:46682/0 (socket says 172.21.15.27:46682) 2024-09-12T03:17:38.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:38.355+0000 7f0661889640 1 -- 172.21.15.27:0/3424450713 learned_addr learned my addr 172.21.15.27:0/3424450713 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:17:38.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:38.355+0000 7f0661088640 1 --2- 172.21.15.27:0/3424450713 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f065c06cc40 0x7f065c06d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:38.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:38.355+0000 7f066208a640 1 --2- 172.21.15.27:0/3424450713 >> v2:172.21.15.45:3300/0 conn(0x7f065c06db30 0x7f065c06df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:38.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:38.355+0000 7f0660887640 1 -- 172.21.15.27:0/3424450713 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4264628500 0 0) 0x7f065c06d5a0 con 0x7f065c15e650 2024-09-12T03:17:38.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:38.355+0000 7f0660887640 1 -- 172.21.15.27:0/3424450713 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0648003520 con 0x7f065c15e650 2024-09-12T03:17:38.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:38.355+0000 7f0660887640 1 -- 172.21.15.27:0/3424450713 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2399229483 0 0) 0x7f0648003520 con 0x7f065c15e650 2024-09-12T03:17:38.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:38.355+0000 7f0660887640 1 -- 172.21.15.27:0/3424450713 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f065c06cc40 msgr2=0x7f065c06d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:38.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:38.355+0000 7f0660887640 1 --2- 172.21.15.27:0/3424450713 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f065c06cc40 0x7f065c06d060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:38.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:38.355+0000 7f0660887640 1 -- 172.21.15.27:0/3424450713 >> v2:172.21.15.45:3300/0 conn(0x7f065c06db30 msgr2=0x7f065c06df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:38.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:38.355+0000 7f0660887640 1 --2- 172.21.15.27:0/3424450713 >> v2:172.21.15.45:3300/0 conn(0x7f065c06db30 0x7f065c06df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:38.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:38.355+0000 7f0660887640 1 -- 172.21.15.27:0/3424450713 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f065c0d1540 con 0x7f065c15e650 2024-09-12T03:17:38.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:38.355+0000 7f0660887640 1 -- 172.21.15.27:0/3424450713 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 847755213 0 0) 0x7f0658001670 con 0x7f065c15e650 2024-09-12T03:17:38.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:38.356+0000 7f0660887640 1 -- 172.21.15.27:0/3424450713 >> v1:172.21.15.27:6789/0 conn(0x7f065c15e650 legacy=0x7f065c0d1d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:38.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:38.356+0000 7f0660887640 1 --2- 172.21.15.27:0/3424450713 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f06480039b0 0x7f0648003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:38.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:38.356+0000 7f0660887640 1 --2- 172.21.15.27:0/3424450713 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0648004800 0x7f0648004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:38.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:38.356+0000 7f0660887640 1 --2- 172.21.15.27:0/3424450713 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f065c06cc40 0x7f06480051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:38.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:38.356+0000 7f0660887640 1 -- 172.21.15.27:0/3424450713 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f065c06d850 con 0x7f06480039b0 2024-09-12T03:17:38.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:38.356+0000 7f0660887640 1 -- 172.21.15.27:0/3424450713 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f065c0d1260 con 0x7f0648004800 2024-09-12T03:17:38.359 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:38.356+0000 7f0660887640 1 -- 172.21.15.27:0/3424450713 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f064800b9f0 con 0x7f065c06cc40 2024-09-12T03:17:38.359 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:38.356+0000 7f0661889640 1 --2- 172.21.15.27:0/3424450713 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f06480039b0 0x7f0648003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:38.359 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:38.356+0000 7f0661088640 1 --2- 172.21.15.27:0/3424450713 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0648004800 0x7f0648004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:38.359 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:38.356+0000 7f066208a640 1 --2- 172.21.15.27:0/3424450713 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f065c06cc40 0x7f06480051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:38.359 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:38.356+0000 7f0661889640 1 -- 172.21.15.27:0/3424450713 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f065c06cc40 msgr2=0x7f06480051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:38.359 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:38.356+0000 7f0661889640 1 --2- 172.21.15.27:0/3424450713 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f065c06cc40 0x7f06480051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:38.359 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:38.356+0000 7f0661889640 1 -- 172.21.15.27:0/3424450713 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0648004800 msgr2=0x7f0648004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:38.359 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:38.356+0000 7f0661889640 1 --2- 172.21.15.27:0/3424450713 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0648004800 0x7f0648004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:38.359 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:38.356+0000 7f0661889640 1 -- 172.21.15.27:0/3424450713 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0648003520 con 0x7f06480039b0 2024-09-12T03:17:38.359 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:38.356+0000 7f0661088640 1 --2- 172.21.15.27:0/3424450713 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0648004800 0x7f0648004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:17:38.359 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:38.357+0000 7f0661889640 1 --2- 172.21.15.27:0/3424450713 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f06480039b0 0x7f0648003dd0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f065800c290 tx=0x7f0648005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32b72605e166c70e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:38.359 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:38.357+0000 7f0660887640 1 -- 172.21.15.27:0/3424450713 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0658007d50 con 0x7f06480039b0 2024-09-12T03:17:38.359 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:38.357+0000 7f0660887640 1 -- 172.21.15.27:0/3424450713 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f065802e070 con 0x7f06480039b0 2024-09-12T03:17:38.360 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:38.357+0000 7f0660887640 1 -- 172.21.15.27:0/3424450713 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0658005d30 con 0x7f06480039b0 2024-09-12T03:17:38.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:38.357+0000 7f0663aed640 1 -- 172.21.15.27:0/3424450713 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f06480039b0 msgr2=0x7f0648003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:38.363 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:38.357+0000 7f0663aed640 1 --2- 172.21.15.27:0/3424450713 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f06480039b0 0x7f0648003dd0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f065800c290 tx=0x7f0648005830 comp rx=0 tx=0).stop 2024-09-12T03:17:38.363 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:38.357+0000 7f066208a640 1 -- 172.21.15.27:0/3424450713 reap_dead start 2024-09-12T03:17:38.363 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:38.357+0000 7f0663aed640 1 -- 172.21.15.27:0/3424450713 shutdown_connections 2024-09-12T03:17:38.363 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:38.357+0000 7f0663aed640 1 -- 172.21.15.27:0/3424450713 >> 172.21.15.27:0/3424450713 conn(0x7f065c06f320 msgr2=0x7f065c06f700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:38.363 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:38.358+0000 7f0663aed640 1 -- 172.21.15.27:0/3424450713 shutdown_connections 2024-09-12T03:17:38.363 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:38.358+0000 7f0663aed640 1 -- 172.21.15.27:0/3424450713 wait complete. 2024-09-12T03:17:38.363 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:38.359+0000 7f0663aed640 1 Processor -- start 2024-09-12T03:17:38.363 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:38.359+0000 7f0663aed640 1 -- start start 2024-09-12T03:17:38.363 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:38.359+0000 7f0663aed640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06480039b0 0x7f065c0cb930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:38.363 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:38.359+0000 7f0663aed640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f065c1577d0 0x7f065c159c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:38.363 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:38.359+0000 7f0663aed640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f065c15a9c0 0x7f065c1603c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:38.364 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:38.359+0000 7f0663aed640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f065c06e640 con 0x7f065c1577d0 2024-09-12T03:17:38.364 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:38.359+0000 7f0663aed640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f065c06e4c0 con 0x7f06480039b0 2024-09-12T03:17:38.364 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:38.359+0000 7f0663aed640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f065c06d060 con 0x7f065c15a9c0 2024-09-12T03:17:38.364 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:38.360+0000 7f0661088640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f065c1577d0 0x7f065c159c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:38.364 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:38.360+0000 7f0661088640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f065c1577d0 0x7f065c159c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:48386/0 (socket says 172.21.15.27:48386) 2024-09-12T03:17:38.364 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:38.360+0000 7f0661088640 1 -- 172.21.15.27:0/7079028 learned_addr learned my addr 172.21.15.27:0/7079028 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:17:38.364 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:38.360+0000 7f066208a640 1 --2- 172.21.15.27:0/7079028 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f065c15a9c0 0x7f065c1603c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:38.364 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:38.360+0000 7f0661889640 1 --2- 172.21.15.27:0/7079028 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06480039b0 0x7f065c0cb930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:38.364 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:38.360+0000 7f0661088640 1 -- 172.21.15.27:0/7079028 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f065c15a9c0 msgr2=0x7f065c1603c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:38.364 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:38.360+0000 7f0661088640 1 --2- 172.21.15.27:0/7079028 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f065c15a9c0 0x7f065c1603c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:38.364 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:38.360+0000 7f0661088640 1 -- 172.21.15.27:0/7079028 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06480039b0 msgr2=0x7f065c0cb930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:38.364 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:38.360+0000 7f0661088640 1 --2- 172.21.15.27:0/7079028 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06480039b0 0x7f065c0cb930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:38.364 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:38.360+0000 7f0661088640 1 -- 172.21.15.27:0/7079028 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f065c0cbe70 con 0x7f065c1577d0 2024-09-12T03:17:38.365 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:38.360+0000 7f0661889640 1 --2- 172.21.15.27:0/7079028 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06480039b0 0x7f065c0cb930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:17:38.365 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:38.360+0000 7f0661088640 1 --2- 172.21.15.27:0/7079028 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f065c1577d0 0x7f065c159c10 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f064c011540 tx=0x7f064c011a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:38.365 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:38.360+0000 7f06527fc640 1 -- 172.21.15.27:0/7079028 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f064c015870 con 0x7f065c1577d0 2024-09-12T03:17:38.365 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:38.360+0000 7f06527fc640 1 -- 172.21.15.27:0/7079028 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f064c015a10 con 0x7f065c1577d0 2024-09-12T03:17:38.365 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:38.360+0000 7f0663aed640 1 -- 172.21.15.27:0/7079028 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f065c0cc110 con 0x7f065c1577d0 2024-09-12T03:17:38.365 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:38.360+0000 7f06527fc640 1 -- 172.21.15.27:0/7079028 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f064c015dc0 con 0x7f065c1577d0 2024-09-12T03:17:38.365 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:38.360+0000 7f0663aed640 1 -- 172.21.15.27:0/7079028 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f065c0cc6d0 con 0x7f065c1577d0 2024-09-12T03:17:38.365 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:38.361+0000 7f0663aed640 1 -- 172.21.15.27:0/7079028 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f065c06d4a0 con 0x7f065c1577d0 2024-09-12T03:17:38.365 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:38.362+0000 7f06527fc640 1 -- 172.21.15.27:0/7079028 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 7) ==== 90651+0+0 (secure 0 0 0) 0x7f064c01d400 con 0x7f065c1577d0 2024-09-12T03:17:38.365 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:38.362+0000 7f06527fc640 1 --2- 172.21.15.27:0/7079028 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f064006bfe0 0x7f064006e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:38.365 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:38.362+0000 7f06527fc640 1 -- 172.21.15.27:0/7079028 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 5609+0+0 (secure 0 0 0) 0x7f064c08b8a0 con 0x7f065c1577d0 2024-09-12T03:17:38.365 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:38.362+0000 7f0661889640 1 --2- 172.21.15.27:0/7079028 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f064006bfe0 0x7f064006e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:38.365 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:38.363+0000 7f0661889640 1 --2- 172.21.15.27:0/7079028 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f064006bfe0 0x7f064006e4a0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f0658033000 tx=0x7f0658009030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:38.366 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:38.367+0000 7f06527fc640 1 -- 172.21.15.27:0/7079028 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f064c019030 con 0x7f065c1577d0 2024-09-12T03:17:38.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:38.579+0000 7f0663aed640 1 -- 172.21.15.27:0/7079028 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f065c0b2500 con 0x7f065c1577d0 2024-09-12T03:17:39.212 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.212+0000 7f06527fc640 1 -- 172.21.15.27:0/7079028 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 6 and data pool 7 v20) ==== 172+0+0 (secure 0 0 0) 0x7f064c05a0c0 con 0x7f065c1577d0 2024-09-12T03:17:39.212 INFO:teuthology.orchestra.run.smithi027.stderr:new fs with metadata pool 6 and data pool 7 2024-09-12T03:17:39.216 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.215+0000 7f0663aed640 1 -- 172.21.15.27:0/7079028 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f064006bfe0 msgr2=0x7f064006e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:39.216 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.215+0000 7f0663aed640 1 --2- 172.21.15.27:0/7079028 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f064006bfe0 0x7f064006e4a0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f0658033000 tx=0x7f0658009030 comp rx=0 tx=0).stop 2024-09-12T03:17:39.216 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.216+0000 7f0663aed640 1 -- 172.21.15.27:0/7079028 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f065c1577d0 msgr2=0x7f065c159c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:39.216 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.216+0000 7f0663aed640 1 --2- 172.21.15.27:0/7079028 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f065c1577d0 0x7f065c159c10 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f064c011540 tx=0x7f064c011a00 comp rx=0 tx=0).stop 2024-09-12T03:17:39.216 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.216+0000 7f0663aed640 1 -- 172.21.15.27:0/7079028 shutdown_connections 2024-09-12T03:17:39.216 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.216+0000 7f0663aed640 1 --2- 172.21.15.27:0/7079028 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f064006bfe0 0x7f064006e4a0 secure :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f0658033000 tx=0x7f0658009030 comp rx=0 tx=0).stop 2024-09-12T03:17:39.216 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.216+0000 7f0663aed640 1 --2- 172.21.15.27:0/7079028 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f065c15a9c0 0x7f065c1603c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:39.216 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.216+0000 7f0663aed640 1 --2- 172.21.15.27:0/7079028 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f065c1577d0 0x7f065c159c10 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:39.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.216+0000 7f0663aed640 1 --2- 172.21.15.27:0/7079028 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06480039b0 0x7f065c0cb930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:39.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.216+0000 7f0663aed640 1 -- 172.21.15.27:0/7079028 >> 172.21.15.27:0/7079028 conn(0x7f065c06f320 msgr2=0x7f065c0736c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:39.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.216+0000 7f0663aed640 1 -- 172.21.15.27:0/7079028 shutdown_connections 2024-09-12T03:17:39.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.216+0000 7f0663aed640 1 -- 172.21.15.27:0/7079028 wait complete. 2024-09-12T03:17:39.234 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-12T03:17:39.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.340+0000 7fe57bef1640 1 Processor -- start 2024-09-12T03:17:39.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.340+0000 7fe57bef1640 1 -- start start 2024-09-12T03:17:39.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.341+0000 7fe57bef1640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fe5740ae8d0 0x7fe5740728b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:39.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.341+0000 7fe57bef1640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe5740aee90 con 0x7fe5740b5a70 2024-09-12T03:17:39.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.341+0000 7fe57bef1640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe5740af170 con 0x7fe574072df0 2024-09-12T03:17:39.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.341+0000 7fe57bef1640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7fe574073240 con 0x7fe5740ae8d0 2024-09-12T03:17:39.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.341+0000 7fe579c8d640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fe5740b5a70 0x7fe57415f600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:40140/0 (socket says 172.21.15.27:40140) 2024-09-12T03:17:39.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.341+0000 7fe579c8d640 1 -- 172.21.15.27:0/1540980618 learned_addr learned my addr 172.21.15.27:0/1540980618 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:17:39.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.341+0000 7fe57948c640 1 --2- 172.21.15.27:0/1540980618 >> v2:172.21.15.27:3300/0 conn(0x7fe5740ae8d0 0x7fe5740728b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:39.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.342+0000 7fe57948c640 1 -- 172.21.15.27:0/1540980618 >> v1:172.21.15.45:6789/0 conn(0x7fe574072df0 legacy=0x7fe57406e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:39.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.342+0000 7fe57948c640 1 -- 172.21.15.27:0/1540980618 >> v1:172.21.15.27:6789/0 conn(0x7fe5740b5a70 legacy=0x7fe57415f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:39.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.342+0000 7fe578c8b640 1 -- 172.21.15.27:0/1540980618 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2091007384 0 0) 0x7fe5740aee90 con 0x7fe5740b5a70 2024-09-12T03:17:39.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.342+0000 7fe57948c640 1 -- 172.21.15.27:0/1540980618 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5740af450 con 0x7fe5740ae8d0 2024-09-12T03:17:39.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.342+0000 7fe57948c640 1 --2- 172.21.15.27:0/1540980618 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe5740ae8d0 0x7fe5740728b0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fe5680097a0 tx=0x7fe56802a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8381017c870f75ec server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:39.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.342+0000 7fe578c8b640 1 -- 172.21.15.27:0/1540980618 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe568037070 con 0x7fe5740ae8d0 2024-09-12T03:17:39.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.342+0000 7fe578c8b640 1 -- 172.21.15.27:0/1540980618 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe56802ae50 con 0x7fe5740ae8d0 2024-09-12T03:17:39.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.342+0000 7fe578c8b640 1 -- 172.21.15.27:0/1540980618 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe568032660 con 0x7fe5740ae8d0 2024-09-12T03:17:39.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.343+0000 7fe57bef1640 1 -- 172.21.15.27:0/1540980618 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe5740ae8d0 msgr2=0x7fe5740728b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:39.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.343+0000 7fe57bef1640 1 --2- 172.21.15.27:0/1540980618 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe5740ae8d0 0x7fe5740728b0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fe5680097a0 tx=0x7fe56802a140 comp rx=0 tx=0).stop 2024-09-12T03:17:39.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.343+0000 7fe57bef1640 1 -- 172.21.15.27:0/1540980618 shutdown_connections 2024-09-12T03:17:39.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.343+0000 7fe57bef1640 1 --2- 172.21.15.27:0/1540980618 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe5740ae8d0 0x7fe5740728b0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:39.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.343+0000 7fe57bef1640 1 -- 172.21.15.27:0/1540980618 >> 172.21.15.27:0/1540980618 conn(0x7fe57406ca30 msgr2=0x7fe57406ce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:39.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.343+0000 7fe57bef1640 1 -- 172.21.15.27:0/1540980618 shutdown_connections 2024-09-12T03:17:39.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.343+0000 7fe57bef1640 1 -- 172.21.15.27:0/1540980618 wait complete. 2024-09-12T03:17:39.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.344+0000 7fe57bef1640 1 Processor -- start 2024-09-12T03:17:39.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.344+0000 7fe57bef1640 1 -- start start 2024-09-12T03:17:39.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.345+0000 7fe57bef1640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe574072df0 0x7fe574152fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:39.348 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.345+0000 7fe57bef1640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe5740ae8d0 0x7fe57414d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:39.348 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.345+0000 7fe57bef1640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe5740b5a70 0x7fe5740b0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:39.348 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.345+0000 7fe57bef1640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fe57406f1c0 con 0x7fe5740ae8d0 2024-09-12T03:17:39.348 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.345+0000 7fe57bef1640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe57406f040 con 0x7fe574072df0 2024-09-12T03:17:39.348 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.345+0000 7fe57bef1640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fe57406f340 con 0x7fe5740b5a70 2024-09-12T03:17:39.348 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.345+0000 7fe57948c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe5740ae8d0 0x7fe57414d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:39.348 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.345+0000 7fe57948c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe5740ae8d0 0x7fe57414d7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60258/0 (socket says 172.21.15.27:60258) 2024-09-12T03:17:39.348 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.345+0000 7fe57948c640 1 -- 172.21.15.27:0/200522262 learned_addr learned my addr 172.21.15.27:0/200522262 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:17:39.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.345+0000 7fe579c8d640 1 --2- 172.21.15.27:0/200522262 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe574072df0 0x7fe574152fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:39.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.345+0000 7fe57a48e640 1 --2- 172.21.15.27:0/200522262 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe5740b5a70 0x7fe5740b0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:39.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.345+0000 7fe57948c640 1 -- 172.21.15.27:0/200522262 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe5740b5a70 msgr2=0x7fe5740b0dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:39.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.345+0000 7fe57948c640 1 --2- 172.21.15.27:0/200522262 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe5740b5a70 0x7fe5740b0dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:39.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.345+0000 7fe57948c640 1 -- 172.21.15.27:0/200522262 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe574072df0 msgr2=0x7fe574152fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:39.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.345+0000 7fe57948c640 1 --2- 172.21.15.27:0/200522262 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe574072df0 0x7fe574152fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:39.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.345+0000 7fe57948c640 1 -- 172.21.15.27:0/200522262 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5741f42c0 con 0x7fe5740ae8d0 2024-09-12T03:17:39.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.346+0000 7fe57948c640 1 --2- 172.21.15.27:0/200522262 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe5740ae8d0 0x7fe57414d7e0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fe5680098c0 tx=0x7fe5680046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:39.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.346+0000 7fe5727fc640 1 -- 172.21.15.27:0/200522262 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe568042070 con 0x7fe5740ae8d0 2024-09-12T03:17:39.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.346+0000 7fe5727fc640 1 -- 172.21.15.27:0/200522262 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe56803e670 con 0x7fe5740ae8d0 2024-09-12T03:17:39.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.346+0000 7fe57bef1640 1 -- 172.21.15.27:0/200522262 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe574159d20 con 0x7fe5740ae8d0 2024-09-12T03:17:39.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.346+0000 7fe5727fc640 1 -- 172.21.15.27:0/200522262 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe568037040 con 0x7fe5740ae8d0 2024-09-12T03:17:39.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.346+0000 7fe57bef1640 1 -- 172.21.15.27:0/200522262 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5741527d0 con 0x7fe5740ae8d0 2024-09-12T03:17:39.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.347+0000 7fe57bef1640 1 -- 172.21.15.27:0/200522262 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5740728b0 con 0x7fe5740ae8d0 2024-09-12T03:17:39.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.347+0000 7fe5727fc640 1 -- 172.21.15.27:0/200522262 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 7) ==== 90651+0+0 (secure 0 0 0) 0x7fe568045070 con 0x7fe5740ae8d0 2024-09-12T03:17:39.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.348+0000 7fe5727fc640 1 --2- 172.21.15.27:0/200522262 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fe56006bfe0 0x7fe56006e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:39.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.348+0000 7fe5727fc640 1 -- 172.21.15.27:0/200522262 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 5709+0+0 (secure 0 0 0) 0x7fe5680abbc0 con 0x7fe5740ae8d0 2024-09-12T03:17:39.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.348+0000 7fe579c8d640 1 --2- 172.21.15.27:0/200522262 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fe56006bfe0 0x7fe56006e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:39.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.349+0000 7fe579c8d640 1 --2- 172.21.15.27:0/200522262 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fe56006bfe0 0x7fe56006e4a0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fe56c000f60 tx=0x7fe56c007c00 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:39.352 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.352+0000 7fe5727fc640 1 -- 172.21.15.27:0/200522262 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fe5741f42c0 con 0x7fe5740ae8d0 2024-09-12T03:17:39.529 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.529+0000 7fe57bef1640 1 -- 172.21.15.27:0/200522262 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe57414dd20 con 0x7fe5740ae8d0 2024-09-12T03:17:39.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.531+0000 7fe5727fc640 1 -- 172.21.15.27:0/200522262 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v42) ==== 74+0+16638 (secure 0 0 0) 0x7fe56807a410 con 0x7fe5740ae8d0 2024-09-12T03:17:39.531 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:17:39.531 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":42,"fsid":"d2df504e-2647-4385-a5db-9fdf69f487a0","created":"2024-09-12T03:16:07.778937+0000","modified":"2024-09-12T03:17:38.580168+0000","last_up_change":"2024-09-12T03:16:19.314778+0000","last_in_change":"2024-09-12T03:16:09.893086+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T03:16:20.097646+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-09-12T03:17:37.155092+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-09-12T03:17:37.552917+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1851116c-1fb3-4dc9-9772-f90e2ef4c355","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6817","nonce":3002321488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6819","nonce":3002321488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6823","nonce":3002321488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6821","nonce":3002321488}]},"public_addr":"172.21.15.27:6817/3002321488","cluster_addr":"172.21.15.27:6819/3002321488","heartbeat_back_addr":"172.21.15.27:6823/3002321488","heartbeat_front_addr":"172.21.15.27:6821/3002321488","state":["exists","up"]},{"osd":1,"uuid":"f38ae17d-73eb-4204-b99d-975fa806fb87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6809","nonce":1613102886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6811","nonce":1613102886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6815","nonce":1613102886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6813","nonce":1613102886}]},"public_addr":"172.21.15.27:6809/1613102886","cluster_addr":"172.21.15.27:6811/1613102886","heartbeat_back_addr":"172.21.15.27:6815/1613102886","heartbeat_front_addr":"172.21.15.27:6813/1613102886","state":["exists","up"]},{"osd":2,"uuid":"fb12dcc8-bdd0-42b8-b46d-8b10106224bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6825","nonce":3534723355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6827","nonce":3534723355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6831","nonce":3534723355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6829","nonce":3534723355}]},"public_addr":"172.21.15.27:6825/3534723355","cluster_addr":"172.21.15.27:6827/3534723355","heartbeat_back_addr":"172.21.15.27:6831/3534723355","heartbeat_front_addr":"172.21.15.27:6829/3534723355","state":["exists","up"]},{"osd":3,"uuid":"d74d3f49-9c24-4b9f-98e2-3ecd7b5ef3b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6801","nonce":80661598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6803","nonce":80661598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6807","nonce":80661598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6805","nonce":80661598}]},"public_addr":"172.21.15.27:6801/80661598","cluster_addr":"172.21.15.27:6803/80661598","heartbeat_back_addr":"172.21.15.27:6807/80661598","heartbeat_front_addr":"172.21.15.27:6805/80661598","state":["exists","up"]},{"osd":4,"uuid":"2bf123f2-1b0d-41c4-aa3b-805bab267e59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6817","nonce":3943189827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6819","nonce":3943189827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6823","nonce":3943189827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6821","nonce":3943189827}]},"public_addr":"172.21.15.45:6817/3943189827","cluster_addr":"172.21.15.45:6819/3943189827","heartbeat_back_addr":"172.21.15.45:6823/3943189827","heartbeat_front_addr":"172.21.15.45:6821/3943189827","state":["exists","up"]},{"osd":5,"uuid":"c49ba5da-91c6-4a86-a97b-4aba89c253ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6809","nonce":3620760330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6811","nonce":3620760330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6815","nonce":3620760330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6813","nonce":3620760330}]},"public_addr":"172.21.15.45:6809/3620760330","cluster_addr":"172.21.15.45:6811/3620760330","heartbeat_back_addr":"172.21.15.45:6815/3620760330","heartbeat_front_addr":"172.21.15.45:6813/3620760330","state":["exists","up"]},{"osd":6,"uuid":"b0123475-89f7-4a34-9e33-d797b99dd9dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6801","nonce":1989083510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6803","nonce":1989083510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6807","nonce":1989083510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6805","nonce":1989083510}]},"public_addr":"172.21.15.45:6801/1989083510","cluster_addr":"172.21.15.45:6803/1989083510","heartbeat_back_addr":"172.21.15.45:6807/1989083510","heartbeat_front_addr":"172.21.15.45:6805/1989083510","state":["exists","up"]},{"osd":7,"uuid":"4dad773d-b7d8-46fc-8a10-a972aad27b29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6825","nonce":1513300294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6827","nonce":1513300294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6831","nonce":1513300294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6829","nonce":1513300294}]},"public_addr":"172.21.15.45:6825/1513300294","cluster_addr":"172.21.15.45:6827/1513300294","heartbeat_back_addr":"172.21.15.45:6831/1513300294","heartbeat_front_addr":"172.21.15.45:6829/1513300294","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.881855+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:17.133252+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.735339+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.500742+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.477571+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:17.167667+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.470291+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.873743+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T03:17:39.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.533+0000 7fe57bef1640 1 -- 172.21.15.27:0/200522262 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fe56006bfe0 msgr2=0x7fe56006e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:39.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.533+0000 7fe57bef1640 1 --2- 172.21.15.27:0/200522262 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fe56006bfe0 0x7fe56006e4a0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fe56c000f60 tx=0x7fe56c007c00 comp rx=0 tx=0).stop 2024-09-12T03:17:39.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.533+0000 7fe57bef1640 1 -- 172.21.15.27:0/200522262 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe5740ae8d0 msgr2=0x7fe57414d7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:39.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.533+0000 7fe57bef1640 1 --2- 172.21.15.27:0/200522262 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe5740ae8d0 0x7fe57414d7e0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fe5680098c0 tx=0x7fe5680046c0 comp rx=0 tx=0).stop 2024-09-12T03:17:39.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.533+0000 7fe57bef1640 1 -- 172.21.15.27:0/200522262 shutdown_connections 2024-09-12T03:17:39.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.533+0000 7fe57bef1640 1 --2- 172.21.15.27:0/200522262 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fe56006bfe0 0x7fe56006e4a0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:39.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.533+0000 7fe57bef1640 1 --2- 172.21.15.27:0/200522262 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe5740b5a70 0x7fe5740b0dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:39.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.533+0000 7fe57bef1640 1 --2- 172.21.15.27:0/200522262 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe5740ae8d0 0x7fe57414d7e0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:39.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.533+0000 7fe57bef1640 1 --2- 172.21.15.27:0/200522262 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe574072df0 0x7fe574152fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:39.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.533+0000 7fe57bef1640 1 -- 172.21.15.27:0/200522262 >> 172.21.15.27:0/200522262 conn(0x7fe57406ca30 msgr2=0x7fe57415fd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:39.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.533+0000 7fe57bef1640 1 -- 172.21.15.27:0/200522262 shutdown_connections 2024-09-12T03:17:39.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.533+0000 7fe57bef1640 1 -- 172.21.15.27:0/200522262 wait complete. 2024-09-12T03:17:39.544 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-12T03:17:39.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.647+0000 7f8519206640 1 Processor -- start 2024-09-12T03:17:39.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.647+0000 7f8519206640 1 -- start start 2024-09-12T03:17:39.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.648+0000 7f8519206640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f85140ce6d0 0x7f85140ceab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:39.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.648+0000 7f8519206640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f851406c240 0x7f851406e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:39.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.648+0000 7f8519206640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f85141463d0 con 0x7f8514160d80 2024-09-12T03:17:39.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.648+0000 7f8519206640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f85140ceff0 con 0x7f85140ce6d0 2024-09-12T03:17:39.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.648+0000 7f8519206640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f851406c7f0 con 0x7f851406c240 2024-09-12T03:17:39.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.648+0000 7f8512d9d640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f85140ce6d0 0x7f85140ceab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:39.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.648+0000 7f850a59c640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f8514160d80 0x7f851406bb30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:40144/0 (socket says 172.21.15.27:40144) 2024-09-12T03:17:39.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.648+0000 7f850a59c640 1 -- 172.21.15.27:0/2029551779 learned_addr learned my addr 172.21.15.27:0/2029551779 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:17:39.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.648+0000 7f851359e640 1 --2- 172.21.15.27:0/2029551779 >> v2:172.21.15.45:3300/0 conn(0x7f851406c240 0x7f851406e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:39.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.648+0000 7f8512d9d640 1 -- 172.21.15.27:0/2029551779 >> v2:172.21.15.45:3300/0 conn(0x7f851406c240 msgr2=0x7f851406e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:39.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.648+0000 7f8512d9d640 1 --2- 172.21.15.27:0/2029551779 >> v2:172.21.15.45:3300/0 conn(0x7f851406c240 0x7f851406e5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:39.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.648+0000 7f8512d9d640 1 -- 172.21.15.27:0/2029551779 >> v1:172.21.15.27:6789/0 conn(0x7f8514160d80 legacy=0x7f851406bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:39.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.648+0000 7f851259c640 1 -- 172.21.15.27:0/2029551779 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1771938071 0 0) 0x7f85141463d0 con 0x7f8514160d80 2024-09-12T03:17:39.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.648+0000 7f8512d9d640 1 -- 172.21.15.27:0/2029551779 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85141602a0 con 0x7f85140ce6d0 2024-09-12T03:17:39.650 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.649+0000 7f8512d9d640 1 --2- 172.21.15.27:0/2029551779 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f85140ce6d0 0x7f85140ceab0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f851415cb10 tx=0x7f850402a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=429878f945fbade2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:39.651 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.649+0000 7f851259c640 1 -- 172.21.15.27:0/2029551779 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8504037070 con 0x7f85140ce6d0 2024-09-12T03:17:39.651 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.649+0000 7f851259c640 1 -- 172.21.15.27:0/2029551779 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8504032440 con 0x7f85140ce6d0 2024-09-12T03:17:39.651 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.649+0000 7f851259c640 1 -- 172.21.15.27:0/2029551779 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f850402ae60 con 0x7f85140ce6d0 2024-09-12T03:17:39.651 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.649+0000 7f8519206640 1 -- 172.21.15.27:0/2029551779 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f85140ce6d0 msgr2=0x7f85140ceab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:39.651 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.649+0000 7f8519206640 1 --2- 172.21.15.27:0/2029551779 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f85140ce6d0 0x7f85140ceab0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f851415cb10 tx=0x7f850402a110 comp rx=0 tx=0).stop 2024-09-12T03:17:39.651 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.649+0000 7f8519206640 1 -- 172.21.15.27:0/2029551779 shutdown_connections 2024-09-12T03:17:39.651 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.649+0000 7f8519206640 1 --2- 172.21.15.27:0/2029551779 >> v2:172.21.15.45:3300/0 conn(0x7f851406c240 0x7f851406e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:39.651 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.649+0000 7f8519206640 1 --2- 172.21.15.27:0/2029551779 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f85140ce6d0 0x7f85140ceab0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:39.651 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.650+0000 7f8519206640 1 -- 172.21.15.27:0/2029551779 >> 172.21.15.27:0/2029551779 conn(0x7f851406cc40 msgr2=0x7f851406d050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:39.651 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.650+0000 7f8519206640 1 -- 172.21.15.27:0/2029551779 shutdown_connections 2024-09-12T03:17:39.651 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.650+0000 7f8519206640 1 -- 172.21.15.27:0/2029551779 wait complete. 2024-09-12T03:17:39.651 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.651+0000 7f8519206640 1 Processor -- start 2024-09-12T03:17:39.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.651+0000 7f8519206640 1 -- start start 2024-09-12T03:17:39.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.652+0000 7f8519206640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f851406c240 0x7f85140b0390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:39.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.652+0000 7f8519206640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f85140ce6d0 0x7f8514142b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:39.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.652+0000 7f8519206640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f8514160d80 0x7f8514140040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:39.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.652+0000 7f8519206640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f851406eaf0 con 0x7f85140ce6d0 2024-09-12T03:17:39.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.652+0000 7f8519206640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f8514160860 con 0x7f851406c240 2024-09-12T03:17:39.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.652+0000 7f8519206640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f851406ec70 con 0x7f8514160d80 2024-09-12T03:17:39.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.652+0000 7f850a59c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f85140ce6d0 0x7f8514142b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:39.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.652+0000 7f850a59c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f85140ce6d0 0x7f8514142b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60272/0 (socket says 172.21.15.27:60272) 2024-09-12T03:17:39.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.652+0000 7f850a59c640 1 -- 172.21.15.27:0/2897311801 learned_addr learned my addr 172.21.15.27:0/2897311801 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:17:39.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.652+0000 7f8512d9d640 1 --2- 172.21.15.27:0/2897311801 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f851406c240 0x7f85140b0390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:39.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.652+0000 7f851359e640 1 --2- 172.21.15.27:0/2897311801 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f8514160d80 0x7f8514140040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:39.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.652+0000 7f850a59c640 1 -- 172.21.15.27:0/2897311801 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f8514160d80 msgr2=0x7f8514140040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:39.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.652+0000 7f850a59c640 1 --2- 172.21.15.27:0/2897311801 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f8514160d80 0x7f8514140040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:39.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.652+0000 7f850a59c640 1 -- 172.21.15.27:0/2897311801 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f851406c240 msgr2=0x7f85140b0390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:39.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.652+0000 7f850a59c640 1 --2- 172.21.15.27:0/2897311801 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f851406c240 0x7f85140b0390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:39.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.652+0000 7f850a59c640 1 -- 172.21.15.27:0/2897311801 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85140ae8d0 con 0x7f85140ce6d0 2024-09-12T03:17:39.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.652+0000 7f850a59c640 1 --2- 172.21.15.27:0/2897311801 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f85140ce6d0 0x7f8514142b10 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f84fc00c430 tx=0x7f84fc00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:39.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.653+0000 7f850bfff640 1 -- 172.21.15.27:0/2897311801 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84fc0074e0 con 0x7f85140ce6d0 2024-09-12T03:17:39.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.653+0000 7f850bfff640 1 -- 172.21.15.27:0/2897311801 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84fc0047e0 con 0x7f85140ce6d0 2024-09-12T03:17:39.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.653+0000 7f8519206640 1 -- 172.21.15.27:0/2897311801 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85140aebb0 con 0x7f85140ce6d0 2024-09-12T03:17:39.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.653+0000 7f850bfff640 1 -- 172.21.15.27:0/2897311801 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84fc004bd0 con 0x7f85140ce6d0 2024-09-12T03:17:39.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.653+0000 7f8519206640 1 -- 172.21.15.27:0/2897311801 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85140af170 con 0x7f85140ce6d0 2024-09-12T03:17:39.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.654+0000 7f8519206640 1 -- 172.21.15.27:0/2897311801 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85140b0f50 con 0x7f85140ce6d0 2024-09-12T03:17:39.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.654+0000 7f850bfff640 1 -- 172.21.15.27:0/2897311801 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 7) ==== 90651+0+0 (secure 0 0 0) 0x7f84fc011070 con 0x7f85140ce6d0 2024-09-12T03:17:39.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.655+0000 7f850bfff640 1 --2- 172.21.15.27:0/2897311801 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f84f406bfe0 0x7f84f406e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:39.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.655+0000 7f850bfff640 1 -- 172.21.15.27:0/2897311801 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 5709+0+0 (secure 0 0 0) 0x7f84fc08ae00 con 0x7f85140ce6d0 2024-09-12T03:17:39.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.655+0000 7f8512d9d640 1 --2- 172.21.15.27:0/2897311801 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f84f406bfe0 0x7f84f406e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:39.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.656+0000 7f8512d9d640 1 --2- 172.21.15.27:0/2897311801 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f84f406bfe0 0x7f84f406e4a0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f8504034000 tx=0x7f850402e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:39.659 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.659+0000 7f850bfff640 1 -- 172.21.15.27:0/2897311801 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f84fc059530 con 0x7f85140ce6d0 2024-09-12T03:17:39.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.836+0000 7f8519206640 1 -- 172.21.15.27:0/2897311801 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f85140d1270 con 0x7f85140ce6d0 2024-09-12T03:17:39.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.837+0000 7f850bfff640 1 -- 172.21.15.27:0/2897311801 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v42) ==== 74+0+16638 (secure 0 0 0) 0x7f84fc058e30 con 0x7f85140ce6d0 2024-09-12T03:17:39.838 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:17:39.838 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":42,"fsid":"d2df504e-2647-4385-a5db-9fdf69f487a0","created":"2024-09-12T03:16:07.778937+0000","modified":"2024-09-12T03:17:38.580168+0000","last_up_change":"2024-09-12T03:16:19.314778+0000","last_in_change":"2024-09-12T03:16:09.893086+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T03:16:20.097646+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-09-12T03:17:37.155092+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-09-12T03:17:37.552917+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1851116c-1fb3-4dc9-9772-f90e2ef4c355","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6817","nonce":3002321488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6819","nonce":3002321488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6823","nonce":3002321488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6821","nonce":3002321488}]},"public_addr":"172.21.15.27:6817/3002321488","cluster_addr":"172.21.15.27:6819/3002321488","heartbeat_back_addr":"172.21.15.27:6823/3002321488","heartbeat_front_addr":"172.21.15.27:6821/3002321488","state":["exists","up"]},{"osd":1,"uuid":"f38ae17d-73eb-4204-b99d-975fa806fb87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6809","nonce":1613102886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6811","nonce":1613102886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6815","nonce":1613102886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6813","nonce":1613102886}]},"public_addr":"172.21.15.27:6809/1613102886","cluster_addr":"172.21.15.27:6811/1613102886","heartbeat_back_addr":"172.21.15.27:6815/1613102886","heartbeat_front_addr":"172.21.15.27:6813/1613102886","state":["exists","up"]},{"osd":2,"uuid":"fb12dcc8-bdd0-42b8-b46d-8b10106224bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6825","nonce":3534723355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6827","nonce":3534723355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6831","nonce":3534723355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6829","nonce":3534723355}]},"public_addr":"172.21.15.27:6825/3534723355","cluster_addr":"172.21.15.27:6827/3534723355","heartbeat_back_addr":"172.21.15.27:6831/3534723355","heartbeat_front_addr":"172.21.15.27:6829/3534723355","state":["exists","up"]},{"osd":3,"uuid":"d74d3f49-9c24-4b9f-98e2-3ecd7b5ef3b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6801","nonce":80661598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6803","nonce":80661598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6807","nonce":80661598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6805","nonce":80661598}]},"public_addr":"172.21.15.27:6801/80661598","cluster_addr":"172.21.15.27:6803/80661598","heartbeat_back_addr":"172.21.15.27:6807/80661598","heartbeat_front_addr":"172.21.15.27:6805/80661598","state":["exists","up"]},{"osd":4,"uuid":"2bf123f2-1b0d-41c4-aa3b-805bab267e59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6817","nonce":3943189827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6819","nonce":3943189827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6823","nonce":3943189827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6821","nonce":3943189827}]},"public_addr":"172.21.15.45:6817/3943189827","cluster_addr":"172.21.15.45:6819/3943189827","heartbeat_back_addr":"172.21.15.45:6823/3943189827","heartbeat_front_addr":"172.21.15.45:6821/3943189827","state":["exists","up"]},{"osd":5,"uuid":"c49ba5da-91c6-4a86-a97b-4aba89c253ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6809","nonce":3620760330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6811","nonce":3620760330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6815","nonce":3620760330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6813","nonce":3620760330}]},"public_addr":"172.21.15.45:6809/3620760330","cluster_addr":"172.21.15.45:6811/3620760330","heartbeat_back_addr":"172.21.15.45:6815/3620760330","heartbeat_front_addr":"172.21.15.45:6813/3620760330","state":["exists","up"]},{"osd":6,"uuid":"b0123475-89f7-4a34-9e33-d797b99dd9dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6801","nonce":1989083510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6803","nonce":1989083510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6807","nonce":1989083510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6805","nonce":1989083510}]},"public_addr":"172.21.15.45:6801/1989083510","cluster_addr":"172.21.15.45:6803/1989083510","heartbeat_back_addr":"172.21.15.45:6807/1989083510","heartbeat_front_addr":"172.21.15.45:6805/1989083510","state":["exists","up"]},{"osd":7,"uuid":"4dad773d-b7d8-46fc-8a10-a972aad27b29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6825","nonce":1513300294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6827","nonce":1513300294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6831","nonce":1513300294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6829","nonce":1513300294}]},"public_addr":"172.21.15.45:6825/1513300294","cluster_addr":"172.21.15.45:6827/1513300294","heartbeat_back_addr":"172.21.15.45:6831/1513300294","heartbeat_front_addr":"172.21.15.45:6829/1513300294","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.881855+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:17.133252+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.735339+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.500742+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.477571+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:17.167667+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.470291+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.873743+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T03:17:39.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.840+0000 7f8519206640 1 -- 172.21.15.27:0/2897311801 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f84f406bfe0 msgr2=0x7f84f406e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:39.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.840+0000 7f8519206640 1 --2- 172.21.15.27:0/2897311801 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f84f406bfe0 0x7f84f406e4a0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f8504034000 tx=0x7f850402e000 comp rx=0 tx=0).stop 2024-09-12T03:17:39.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.840+0000 7f8519206640 1 -- 172.21.15.27:0/2897311801 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f85140ce6d0 msgr2=0x7f8514142b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:39.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.840+0000 7f8519206640 1 --2- 172.21.15.27:0/2897311801 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f85140ce6d0 0x7f8514142b10 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f84fc00c430 tx=0x7f84fc00c8f0 comp rx=0 tx=0).stop 2024-09-12T03:17:39.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.840+0000 7f8519206640 1 -- 172.21.15.27:0/2897311801 shutdown_connections 2024-09-12T03:17:39.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.840+0000 7f8519206640 1 --2- 172.21.15.27:0/2897311801 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f84f406bfe0 0x7f84f406e4a0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:39.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.840+0000 7f8519206640 1 --2- 172.21.15.27:0/2897311801 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f8514160d80 0x7f8514140040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:39.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.840+0000 7f8519206640 1 --2- 172.21.15.27:0/2897311801 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f85140ce6d0 0x7f8514142b10 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:39.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.840+0000 7f8519206640 1 --2- 172.21.15.27:0/2897311801 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f851406c240 0x7f85140b0390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:39.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.840+0000 7f8519206640 1 -- 172.21.15.27:0/2897311801 >> 172.21.15.27:0/2897311801 conn(0x7f851406cc40 msgr2=0x7f851406bb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:39.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.840+0000 7f8519206640 1 -- 172.21.15.27:0/2897311801 shutdown_connections 2024-09-12T03:17:39.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.840+0000 7f8519206640 1 -- 172.21.15.27:0/2897311801 wait complete. 2024-09-12T03:17:39.851 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-12T03:17:39.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.956+0000 7f5bb665e640 1 Processor -- start 2024-09-12T03:17:39.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.956+0000 7f5bb665e640 1 -- start start 2024-09-12T03:17:39.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.957+0000 7f5bb665e640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f5bb00b3d10 0x7f5bb00b66c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:39.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.957+0000 7f5bb665e640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5bb015ed70 con 0x7f5bb00b6230 2024-09-12T03:17:39.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.957+0000 7f5bb665e640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5bb00b7d10 con 0x7f5bb015e990 2024-09-12T03:17:39.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.957+0000 7f5bb665e640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f5bb00d2a30 con 0x7f5bb00b3d10 2024-09-12T03:17:39.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.957+0000 7f5bb4bfb640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f5bb00b3d10 0x7f5bb00b66c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:39.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.957+0000 7f5baf7fe640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f5bb00b6230 0x7f5bb00b3600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:40148/0 (socket says 172.21.15.27:40148) 2024-09-12T03:17:39.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.957+0000 7f5baf7fe640 1 -- 172.21.15.27:0/220685292 learned_addr learned my addr 172.21.15.27:0/220685292 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:17:39.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.957+0000 7f5bb4bfb640 1 -- 172.21.15.27:0/220685292 >> v1:172.21.15.45:6789/0 conn(0x7f5bb015e990 legacy=0x7f5bb00b7600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:39.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.957+0000 7f5bb4bfb640 1 -- 172.21.15.27:0/220685292 >> v1:172.21.15.27:6789/0 conn(0x7f5bb00b6230 legacy=0x7f5bb00b3600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:39.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.957+0000 7f5bb4bfb640 1 -- 172.21.15.27:0/220685292 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bb00d2d10 con 0x7f5bb00b3d10 2024-09-12T03:17:39.959 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.958+0000 7f5bb4bfb640 1 --2- 172.21.15.27:0/220685292 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5bb00b3d10 0x7f5bb00b66c0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f5ba8009d90 tx=0x7f5ba802a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62705bdd737e5219 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:39.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.958+0000 7f5baeffd640 1 -- 172.21.15.27:0/220685292 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ba8007740 con 0x7f5bb00b3d10 2024-09-12T03:17:39.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.958+0000 7f5baeffd640 1 -- 172.21.15.27:0/220685292 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ba80078e0 con 0x7f5bb00b3d10 2024-09-12T03:17:39.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.958+0000 7f5baeffd640 1 -- 172.21.15.27:0/220685292 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ba8007cb0 con 0x7f5bb00b3d10 2024-09-12T03:17:39.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.959+0000 7f5bb665e640 1 -- 172.21.15.27:0/220685292 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5bb00b3d10 msgr2=0x7f5bb00b66c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:39.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.959+0000 7f5bb665e640 1 --2- 172.21.15.27:0/220685292 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5bb00b3d10 0x7f5bb00b66c0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f5ba8009d90 tx=0x7f5ba802a6f0 comp rx=0 tx=0).stop 2024-09-12T03:17:39.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.959+0000 7f5bb665e640 1 -- 172.21.15.27:0/220685292 shutdown_connections 2024-09-12T03:17:39.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.959+0000 7f5bb665e640 1 --2- 172.21.15.27:0/220685292 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5bb00b3d10 0x7f5bb00b66c0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:39.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.959+0000 7f5bb665e640 1 -- 172.21.15.27:0/220685292 >> 172.21.15.27:0/220685292 conn(0x7f5bb00d1230 msgr2=0x7f5bb00d1640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:39.961 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.959+0000 7f5bb665e640 1 -- 172.21.15.27:0/220685292 shutdown_connections 2024-09-12T03:17:39.961 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.959+0000 7f5bb665e640 1 -- 172.21.15.27:0/220685292 wait complete. 2024-09-12T03:17:39.963 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.960+0000 7f5bb665e640 1 Processor -- start 2024-09-12T03:17:39.963 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.960+0000 7f5bb665e640 1 -- start start 2024-09-12T03:17:39.963 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.961+0000 7f5bb665e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5bb00b3d10 0x7f5bb0151240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:39.964 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.961+0000 7f5bb665e640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f5bb00b6230 0x7f5bb00cb880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:39.964 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.961+0000 7f5bb665e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5bb015e990 0x7f5bb01588b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:39.964 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.961+0000 7f5bb665e640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f5bb00d35b0 con 0x7f5bb015e990 2024-09-12T03:17:39.964 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.961+0000 7f5bb665e640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f5bb00d32d0 con 0x7f5bb00b3d10 2024-09-12T03:17:39.964 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.961+0000 7f5bb665e640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f5bb00d3ad0 con 0x7f5bb00b6230 2024-09-12T03:17:39.964 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.961+0000 7f5bb4bfb640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5bb015e990 0x7f5bb01588b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:39.964 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.961+0000 7f5bb4bfb640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5bb015e990 0x7f5bb01588b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60290/0 (socket says 172.21.15.27:60290) 2024-09-12T03:17:39.964 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.961+0000 7f5bb4bfb640 1 -- 172.21.15.27:0/1881469983 learned_addr learned my addr 172.21.15.27:0/1881469983 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:17:39.964 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.961+0000 7f5baffff640 1 --2- 172.21.15.27:0/1881469983 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5bb00b3d10 0x7f5bb0151240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:39.964 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.961+0000 7f5baf7fe640 1 --2- 172.21.15.27:0/1881469983 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f5bb00b6230 0x7f5bb00cb880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:39.964 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.961+0000 7f5bb4bfb640 1 -- 172.21.15.27:0/1881469983 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f5bb00b6230 msgr2=0x7f5bb00cb880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:39.964 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.961+0000 7f5bb4bfb640 1 --2- 172.21.15.27:0/1881469983 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f5bb00b6230 0x7f5bb00cb880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:39.964 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.961+0000 7f5bb4bfb640 1 -- 172.21.15.27:0/1881469983 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5bb00b3d10 msgr2=0x7f5bb0151240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:39.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.961+0000 7f5bb4bfb640 1 --2- 172.21.15.27:0/1881469983 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5bb00b3d10 0x7f5bb0151240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:39.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.961+0000 7f5bb4bfb640 1 -- 172.21.15.27:0/1881469983 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bb00cd520 con 0x7f5bb015e990 2024-09-12T03:17:39.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.962+0000 7f5bb4bfb640 1 --2- 172.21.15.27:0/1881469983 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5bb015e990 0x7f5bb01588b0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f5ba8009eb0 tx=0x7f5ba8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:39.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.962+0000 7f5bacff9640 1 -- 172.21.15.27:0/1881469983 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ba8040070 con 0x7f5bb015e990 2024-09-12T03:17:39.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.962+0000 7f5bacff9640 1 -- 172.21.15.27:0/1881469983 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ba8031bc0 con 0x7f5bb015e990 2024-09-12T03:17:39.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.962+0000 7f5bb665e640 1 -- 172.21.15.27:0/1881469983 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5bb00cd7c0 con 0x7f5bb015e990 2024-09-12T03:17:39.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.962+0000 7f5bacff9640 1 -- 172.21.15.27:0/1881469983 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ba8045460 con 0x7f5bb015e990 2024-09-12T03:17:39.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.962+0000 7f5bb665e640 1 -- 172.21.15.27:0/1881469983 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5bb00cdd80 con 0x7f5bb015e990 2024-09-12T03:17:39.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.963+0000 7f5bb665e640 1 -- 172.21.15.27:0/1881469983 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5bb00b66c0 con 0x7f5bb015e990 2024-09-12T03:17:39.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.964+0000 7f5bacff9640 1 -- 172.21.15.27:0/1881469983 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 7) ==== 90651+0+0 (secure 0 0 0) 0x7f5ba8039040 con 0x7f5bb015e990 2024-09-12T03:17:39.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.964+0000 7f5bacff9640 1 --2- 172.21.15.27:0/1881469983 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f5b9c06c010 0x7f5b9c06e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:39.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.964+0000 7f5bacff9640 1 -- 172.21.15.27:0/1881469983 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 5709+0+0 (secure 0 0 0) 0x7f5ba80aca40 con 0x7f5bb015e990 2024-09-12T03:17:39.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.964+0000 7f5baffff640 1 --2- 172.21.15.27:0/1881469983 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f5b9c06c010 0x7f5b9c06e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:39.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.965+0000 7f5baffff640 1 --2- 172.21.15.27:0/1881469983 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f5b9c06c010 0x7f5b9c06e4d0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f5ba00016e0 tx=0x7f5ba0009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:39.968 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:39.968+0000 7f5bacff9640 1 -- 172.21.15.27:0/1881469983 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f5ba807aad0 con 0x7f5bb015e990 2024-09-12T03:17:40.181 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:40.181+0000 7f5bb665e640 1 -- 172.21.15.27:0/1881469983 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f5bb00b6a70 con 0x7f5bb015e990 2024-09-12T03:17:40.216 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:40.216+0000 7f5bacff9640 1 -- 172.21.15.27:0/1881469983 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v21) ==== 118+0+0 (secure 0 0 0) 0x7f5ba807edd0 con 0x7f5bb015e990 2024-09-12T03:17:40.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:40.218+0000 7f5bb665e640 1 -- 172.21.15.27:0/1881469983 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f5b9c06c010 msgr2=0x7f5b9c06e4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:40.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:40.218+0000 7f5bb665e640 1 --2- 172.21.15.27:0/1881469983 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f5b9c06c010 0x7f5b9c06e4d0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f5ba00016e0 tx=0x7f5ba0009040 comp rx=0 tx=0).stop 2024-09-12T03:17:40.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:40.218+0000 7f5bb665e640 1 -- 172.21.15.27:0/1881469983 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5bb015e990 msgr2=0x7f5bb01588b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:40.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:40.218+0000 7f5bb665e640 1 --2- 172.21.15.27:0/1881469983 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5bb015e990 0x7f5bb01588b0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f5ba8009eb0 tx=0x7f5ba8004a40 comp rx=0 tx=0).stop 2024-09-12T03:17:40.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:40.218+0000 7f5bb665e640 1 -- 172.21.15.27:0/1881469983 shutdown_connections 2024-09-12T03:17:40.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:40.218+0000 7f5bb665e640 1 --2- 172.21.15.27:0/1881469983 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f5b9c06c010 0x7f5b9c06e4d0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:40.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:40.218+0000 7f5bb665e640 1 --2- 172.21.15.27:0/1881469983 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5bb015e990 0x7f5bb01588b0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:40.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:40.218+0000 7f5bb665e640 1 --2- 172.21.15.27:0/1881469983 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f5bb00b6230 0x7f5bb00cb880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:40.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:40.218+0000 7f5bb665e640 1 --2- 172.21.15.27:0/1881469983 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5bb00b3d10 0x7f5bb0151240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:40.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:40.218+0000 7f5bb665e640 1 -- 172.21.15.27:0/1881469983 >> 172.21.15.27:0/1881469983 conn(0x7f5bb00d1230 msgr2=0x7f5bb00d1610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:40.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:40.218+0000 7f5bb665e640 1 -- 172.21.15.27:0/1881469983 shutdown_connections 2024-09-12T03:17:40.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:40.218+0000 7f5bb665e640 1 -- 172.21.15.27:0/1881469983 wait complete. 2024-09-12T03:17:40.229 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-12T03:17:40.229 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-12T03:17:40.334 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:40.334+0000 7f8bbb7cd640 1 Processor -- start 2024-09-12T03:17:40.334 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:40.335+0000 7f8bbb7cd640 1 -- start start 2024-09-12T03:17:40.338 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:40.335+0000 7f8bbb7cd640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f8bb40cef40 0x7f8bb40b4000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:40.338 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:40.335+0000 7f8bbb7cd640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8bb406cff0 con 0x7f8bb40cf320 2024-09-12T03:17:40.338 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:40.335+0000 7f8bbb7cd640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8bb406d2d0 con 0x7f8bb406c650 2024-09-12T03:17:40.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:40.335+0000 7f8bbb7cd640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f8bb406dee0 con 0x7f8bb40cef40 2024-09-12T03:17:40.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:40.336+0000 7f8bb9d6a640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f8bb40cf320 0x7f8bb415f0c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:40158/0 (socket says 172.21.15.27:40158) 2024-09-12T03:17:40.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:40.336+0000 7f8bb9d6a640 1 -- 172.21.15.27:0/3342808251 learned_addr learned my addr 172.21.15.27:0/3342808251 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:17:40.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:40.336+0000 7f8bb9569640 1 --2- 172.21.15.27:0/3342808251 >> v2:172.21.15.45:3300/0 conn(0x7f8bb40cef40 0x7f8bb40b4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:40.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:40.336+0000 7f8bb3fff640 1 -- 172.21.15.27:0/3342808251 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1696677516 0 0) 0x7f8bb406cff0 con 0x7f8bb40cf320 2024-09-12T03:17:40.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:40.336+0000 7f8bb3fff640 1 -- 172.21.15.27:0/3342808251 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8b9c003520 con 0x7f8bb40cf320 2024-09-12T03:17:40.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:40.336+0000 7f8bb3fff640 1 -- 172.21.15.27:0/3342808251 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2572047489 0 0) 0x7f8b9c003520 con 0x7f8bb40cf320 2024-09-12T03:17:40.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:40.336+0000 7f8bb3fff640 1 -- 172.21.15.27:0/3342808251 >> v2:172.21.15.45:3300/0 conn(0x7f8bb40cef40 msgr2=0x7f8bb40b4000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:40.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:40.336+0000 7f8bb3fff640 1 --2- 172.21.15.27:0/3342808251 >> v2:172.21.15.45:3300/0 conn(0x7f8bb40cef40 0x7f8bb40b4000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:40.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:40.336+0000 7f8bb3fff640 1 -- 172.21.15.27:0/3342808251 >> v1:172.21.15.45:6789/0 conn(0x7f8bb406c650 legacy=0x7f8bb40710c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:40.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:40.336+0000 7f8bb3fff640 1 -- 172.21.15.27:0/3342808251 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bb406d5b0 con 0x7f8bb40cf320 2024-09-12T03:17:40.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:40.336+0000 7f8bb3fff640 1 -- 172.21.15.27:0/3342808251 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 847755213 0 0) 0x7f8ba40019f0 con 0x7f8bb40cf320 2024-09-12T03:17:40.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:40.336+0000 7f8bb3fff640 1 -- 172.21.15.27:0/3342808251 >> v1:172.21.15.27:6789/0 conn(0x7f8bb40cf320 legacy=0x7f8bb415f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:40.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:40.336+0000 7f8bb3fff640 1 --2- 172.21.15.27:0/3342808251 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8b9c0039b0 0x7f8b9c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:40.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:40.336+0000 7f8bb3fff640 1 --2- 172.21.15.27:0/3342808251 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f8b9c004800 0x7f8b9c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:40.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:40.336+0000 7f8bb3fff640 1 --2- 172.21.15.27:0/3342808251 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8b9c0051c0 0x7f8b9c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:40.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:40.336+0000 7f8bb3fff640 1 -- 172.21.15.27:0/3342808251 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f8bb406dee0 con 0x7f8b9c0051c0 2024-09-12T03:17:40.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:40.336+0000 7f8bb3fff640 1 -- 172.21.15.27:0/3342808251 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f8b9c00c160 con 0x7f8b9c0039b0 2024-09-12T03:17:40.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:40.336+0000 7f8bb3fff640 1 -- 172.21.15.27:0/3342808251 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f8b9c00c360 con 0x7f8b9c004800 2024-09-12T03:17:40.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:40.336+0000 7f8bb9d6a640 1 --2- 172.21.15.27:0/3342808251 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8b9c0051c0 0x7f8b9c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:40.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:40.336+0000 7f8bb9569640 1 --2- 172.21.15.27:0/3342808251 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8b9c0039b0 0x7f8b9c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:40.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:40.336+0000 7f8bb8d68640 1 --2- 172.21.15.27:0/3342808251 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f8b9c004800 0x7f8b9c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:40.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:40.337+0000 7f8bb9d6a640 1 -- 172.21.15.27:0/3342808251 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f8b9c004800 msgr2=0x7f8b9c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:40.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:40.337+0000 7f8bb9d6a640 1 --2- 172.21.15.27:0/3342808251 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f8b9c004800 0x7f8b9c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:40.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:40.337+0000 7f8bb9d6a640 1 -- 172.21.15.27:0/3342808251 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8b9c0039b0 msgr2=0x7f8b9c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:40.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:40.337+0000 7f8bb9d6a640 1 --2- 172.21.15.27:0/3342808251 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8b9c0039b0 0x7f8b9c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:40.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:40.337+0000 7f8bb9d6a640 1 -- 172.21.15.27:0/3342808251 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b9c003520 con 0x7f8b9c0051c0 2024-09-12T03:17:40.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:40.337+0000 7f8bb9d6a640 1 -- 172.21.15.27:0/3342808251 reap_dead start 2024-09-12T03:17:40.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:40.337+0000 7f8bb9d6a640 1 --2- 172.21.15.27:0/3342808251 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8b9c0051c0 0x7f8b9c00b9c0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f8bb40cf7d0 tx=0x7f8b9c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98e6ca8c4c5eda7f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:40.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:40.338+0000 7f8bb3fff640 1 -- 172.21.15.27:0/3342808251 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ba400a4a0 con 0x7f8b9c0051c0 2024-09-12T03:17:40.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:40.338+0000 7f8bb3fff640 1 -- 172.21.15.27:0/3342808251 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ba402f070 con 0x7f8b9c0051c0 2024-09-12T03:17:40.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:40.338+0000 7f8bb3fff640 1 -- 172.21.15.27:0/3342808251 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ba402bcb0 con 0x7f8b9c0051c0 2024-09-12T03:17:40.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:40.338+0000 7f8bbb7cd640 1 -- 172.21.15.27:0/3342808251 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8b9c0051c0 msgr2=0x7f8b9c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:40.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:40.338+0000 7f8bbb7cd640 1 --2- 172.21.15.27:0/3342808251 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8b9c0051c0 0x7f8b9c00b9c0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f8bb40cf7d0 tx=0x7f8b9c005830 comp rx=0 tx=0).stop 2024-09-12T03:17:40.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:40.338+0000 7f8bbb7cd640 1 -- 172.21.15.27:0/3342808251 shutdown_connections 2024-09-12T03:17:40.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:40.338+0000 7f8bbb7cd640 1 --2- 172.21.15.27:0/3342808251 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8b9c0051c0 0x7f8b9c00b9c0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:40.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:40.338+0000 7f8bbb7cd640 1 -- 172.21.15.27:0/3342808251 >> 172.21.15.27:0/3342808251 conn(0x7f8bb406bb30 msgr2=0x7f8bb406bf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:40.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:40.338+0000 7f8bbb7cd640 1 -- 172.21.15.27:0/3342808251 shutdown_connections 2024-09-12T03:17:40.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:40.338+0000 7f8bbb7cd640 1 -- 172.21.15.27:0/3342808251 wait complete. 2024-09-12T03:17:40.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:40.339+0000 7f8bbb7cd640 1 Processor -- start 2024-09-12T03:17:40.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:40.339+0000 7f8bbb7cd640 1 -- start start 2024-09-12T03:17:40.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:40.340+0000 7f8bbb7cd640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f8b9c0051c0 0x7f8bb413a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:40.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:40.340+0000 7f8bbb7cd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8bb413be90 0x7f8bb413b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:40.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:40.340+0000 7f8bbb7cd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8bb413b9f0 0x7f8bb40b1620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:40.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:40.340+0000 7f8bbb7cd640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f8bb40b4000 con 0x7f8bb413b9f0 2024-09-12T03:17:40.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:40.340+0000 7f8bbb7cd640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f8bb406db70 con 0x7f8bb413be90 2024-09-12T03:17:40.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:40.340+0000 7f8bbb7cd640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f8bb40b4180 con 0x7f8b9c0051c0 2024-09-12T03:17:40.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:40.340+0000 7f8bb9d6a640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8bb413b9f0 0x7f8bb40b1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:40.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:40.340+0000 7f8bb9d6a640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8bb413b9f0 0x7f8bb40b1620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60304/0 (socket says 172.21.15.27:60304) 2024-09-12T03:17:40.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:40.340+0000 7f8bb9d6a640 1 -- 172.21.15.27:0/427329220 learned_addr learned my addr 172.21.15.27:0/427329220 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:17:40.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:40.340+0000 7f8bb9569640 1 --2- 172.21.15.27:0/427329220 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f8b9c0051c0 0x7f8bb413a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:40.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:40.340+0000 7f8bb8d68640 1 --2- 172.21.15.27:0/427329220 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8bb413be90 0x7f8bb413b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:40.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:40.341+0000 7f8bb9d6a640 1 -- 172.21.15.27:0/427329220 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f8b9c0051c0 msgr2=0x7f8bb413a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:40.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:40.341+0000 7f8bb9d6a640 1 --2- 172.21.15.27:0/427329220 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f8b9c0051c0 0x7f8bb413a170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:40.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:40.341+0000 7f8bb9d6a640 1 -- 172.21.15.27:0/427329220 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8bb413be90 msgr2=0x7f8bb413b4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:40.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:40.341+0000 7f8bb9d6a640 1 --2- 172.21.15.27:0/427329220 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8bb413be90 0x7f8bb413b4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:40.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:40.341+0000 7f8bb9d6a640 1 -- 172.21.15.27:0/427329220 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bb414d8f0 con 0x7f8bb413b9f0 2024-09-12T03:17:40.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:40.341+0000 7f8bb9d6a640 1 --2- 172.21.15.27:0/427329220 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8bb413b9f0 0x7f8bb40b1620 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f8ba4002230 tx=0x7f8ba402d070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:40.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:40.341+0000 7f8bb1ffb640 1 -- 172.21.15.27:0/427329220 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ba402b850 con 0x7f8bb413b9f0 2024-09-12T03:17:40.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:40.341+0000 7f8bb1ffb640 1 -- 172.21.15.27:0/427329220 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ba402b9f0 con 0x7f8bb413b9f0 2024-09-12T03:17:40.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:40.341+0000 7f8bbb7cd640 1 -- 172.21.15.27:0/427329220 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8bb414d670 con 0x7f8bb413b9f0 2024-09-12T03:17:40.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:40.341+0000 7f8bb1ffb640 1 -- 172.21.15.27:0/427329220 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ba402bde0 con 0x7f8bb413b9f0 2024-09-12T03:17:40.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:40.341+0000 7f8bbb7cd640 1 -- 172.21.15.27:0/427329220 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8bb413af90 con 0x7f8bb413b9f0 2024-09-12T03:17:40.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:40.342+0000 7f8bbb7cd640 1 -- 172.21.15.27:0/427329220 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8bb406cff0 con 0x7f8bb413b9f0 2024-09-12T03:17:40.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:40.342+0000 7f8bb1ffb640 1 -- 172.21.15.27:0/427329220 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 7) ==== 90651+0+0 (secure 0 0 0) 0x7f8ba4034470 con 0x7f8bb413b9f0 2024-09-12T03:17:40.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:40.343+0000 7f8bb1ffb640 1 --2- 172.21.15.27:0/427329220 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f8ba006bfe0 0x7f8ba006e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:40.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:40.343+0000 7f8bb1ffb640 1 -- 172.21.15.27:0/427329220 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5709+0+0 (secure 0 0 0) 0x7f8ba40ab190 con 0x7f8bb413b9f0 2024-09-12T03:17:40.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:40.343+0000 7f8bb9569640 1 --2- 172.21.15.27:0/427329220 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f8ba006bfe0 0x7f8ba006e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:40.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:40.344+0000 7f8bb9569640 1 --2- 172.21.15.27:0/427329220 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f8ba006bfe0 0x7f8ba006e4a0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f8bb413e950 tx=0x7f8bac008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:40.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:40.347+0000 7f8bb1ffb640 1 -- 172.21.15.27:0/427329220 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f8bb414d670 con 0x7f8bb413b9f0 2024-09-12T03:17:40.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:40.556+0000 7f8bbb7cd640 1 -- 172.21.15.27:0/427329220 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f8bb413b160 con 0x7f8bb413b9f0 2024-09-12T03:17:41.131 INFO:tasks.ceph.mds.a.smithi027.stderr:2024-09-12T03:17:41.131+0000 7fc665590640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-12T03:17:41.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.217+0000 7f8bb1ffb640 1 -- 172.21.15.27:0/427329220 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v22) ==== 122+0+0 (secure 0 0 0) 0x7f8ba4032e50 con 0x7f8bb413b9f0 2024-09-12T03:17:41.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.220+0000 7f8bbb7cd640 1 -- 172.21.15.27:0/427329220 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f8ba006bfe0 msgr2=0x7f8ba006e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:41.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.220+0000 7f8bbb7cd640 1 --2- 172.21.15.27:0/427329220 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f8ba006bfe0 0x7f8ba006e4a0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f8bb413e950 tx=0x7f8bac008040 comp rx=0 tx=0).stop 2024-09-12T03:17:41.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.220+0000 7f8bbb7cd640 1 -- 172.21.15.27:0/427329220 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8bb413b9f0 msgr2=0x7f8bb40b1620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:41.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.220+0000 7f8bbb7cd640 1 --2- 172.21.15.27:0/427329220 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8bb413b9f0 0x7f8bb40b1620 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f8ba4002230 tx=0x7f8ba402d070 comp rx=0 tx=0).stop 2024-09-12T03:17:41.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.220+0000 7f8bbb7cd640 1 -- 172.21.15.27:0/427329220 shutdown_connections 2024-09-12T03:17:41.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.220+0000 7f8bbb7cd640 1 --2- 172.21.15.27:0/427329220 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f8ba006bfe0 0x7f8ba006e4a0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:41.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.220+0000 7f8bbb7cd640 1 --2- 172.21.15.27:0/427329220 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8bb413b9f0 0x7f8bb40b1620 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:41.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.220+0000 7f8bbb7cd640 1 --2- 172.21.15.27:0/427329220 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8bb413be90 0x7f8bb413b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:41.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.220+0000 7f8bbb7cd640 1 --2- 172.21.15.27:0/427329220 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f8b9c0051c0 0x7f8bb413a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:41.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.220+0000 7f8bbb7cd640 1 -- 172.21.15.27:0/427329220 >> 172.21.15.27:0/427329220 conn(0x7f8bb406bb30 msgr2=0x7f8bb40cc860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:41.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.220+0000 7f8bbb7cd640 1 -- 172.21.15.27:0/427329220 shutdown_connections 2024-09-12T03:17:41.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.220+0000 7f8bbb7cd640 1 -- 172.21.15.27:0/427329220 wait complete. 2024-09-12T03:17:41.233 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-12T03:17:41.234 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-12T03:17:41.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.343+0000 7fe416253640 1 Processor -- start 2024-09-12T03:17:41.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.343+0000 7fe416253640 1 -- start start 2024-09-12T03:17:41.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.344+0000 7fe416253640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe4101460a0 0x7fe4100cedf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:41.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.344+0000 7fe416253640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fe41015df40 0x7fe41006e100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:41.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.344+0000 7fe416253640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe41015e380 con 0x7fe41015f6e0 2024-09-12T03:17:41.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.344+0000 7fe416253640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7fe41015e640 con 0x7fe41015df40 2024-09-12T03:17:41.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.344+0000 7fe416253640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fe41015e920 con 0x7fe4101460a0 2024-09-12T03:17:41.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.345+0000 7fe40effd640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fe41015df40 0x7fe41006e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:41.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.345+0000 7fe40ffff640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fe41015f6e0 0x7fe41015fb50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:40168/0 (socket says 172.21.15.27:40168) 2024-09-12T03:17:41.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.345+0000 7fe40ffff640 1 -- 172.21.15.27:0/992758033 learned_addr learned my addr 172.21.15.27:0/992758033 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:17:41.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.345+0000 7fe40f7fe640 1 --2- 172.21.15.27:0/992758033 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe4101460a0 0x7fe4100cedf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:41.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.345+0000 7fe40effd640 1 -- 172.21.15.27:0/992758033 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe4101460a0 msgr2=0x7fe4100cedf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:41.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.345+0000 7fe40effd640 1 --2- 172.21.15.27:0/992758033 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe4101460a0 0x7fe4100cedf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:41.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.345+0000 7fe40effd640 1 -- 172.21.15.27:0/992758033 >> v1:172.21.15.27:6789/0 conn(0x7fe41015f6e0 legacy=0x7fe41015fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:41.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.345+0000 7fe40effd640 1 -- 172.21.15.27:0/992758033 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe41015ec00 con 0x7fe41015df40 2024-09-12T03:17:41.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.345+0000 7fe40e7fc640 1 -- 172.21.15.27:0/992758033 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3090263884 0 0) 0x7fe41015e380 con 0x7fe41015f6e0 2024-09-12T03:17:41.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.346+0000 7fe40effd640 1 --2- 172.21.15.27:0/992758033 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe41015df40 0x7fe41006e100 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fe4040097a0 tx=0x7fe40402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77ccee6cff4075a3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:41.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.346+0000 7fe40e7fc640 1 -- 172.21.15.27:0/992758033 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe404037070 con 0x7fe41015df40 2024-09-12T03:17:41.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.346+0000 7fe40e7fc640 1 -- 172.21.15.27:0/992758033 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe404032440 con 0x7fe41015df40 2024-09-12T03:17:41.348 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.346+0000 7fe40e7fc640 1 -- 172.21.15.27:0/992758033 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe404032810 con 0x7fe41015df40 2024-09-12T03:17:41.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.346+0000 7fe416253640 1 -- 172.21.15.27:0/992758033 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe41015df40 msgr2=0x7fe41006e100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:41.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.346+0000 7fe416253640 1 --2- 172.21.15.27:0/992758033 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe41015df40 0x7fe41006e100 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fe4040097a0 tx=0x7fe40402a180 comp rx=0 tx=0).stop 2024-09-12T03:17:41.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.346+0000 7fe416253640 1 -- 172.21.15.27:0/992758033 shutdown_connections 2024-09-12T03:17:41.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.346+0000 7fe416253640 1 --2- 172.21.15.27:0/992758033 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe41015df40 0x7fe41006e100 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:41.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.346+0000 7fe416253640 1 --2- 172.21.15.27:0/992758033 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe4101460a0 0x7fe4100cedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:41.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.346+0000 7fe416253640 1 -- 172.21.15.27:0/992758033 >> 172.21.15.27:0/992758033 conn(0x7fe41006d270 msgr2=0x7fe4100b3270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:41.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.346+0000 7fe416253640 1 -- 172.21.15.27:0/992758033 shutdown_connections 2024-09-12T03:17:41.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.346+0000 7fe416253640 1 -- 172.21.15.27:0/992758033 wait complete. 2024-09-12T03:17:41.352 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.348+0000 7fe416253640 1 Processor -- start 2024-09-12T03:17:41.352 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.348+0000 7fe416253640 1 -- start start 2024-09-12T03:17:41.352 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.349+0000 7fe416253640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe4101460a0 0x7fe4101e78e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:41.352 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.349+0000 7fe416253640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe41015df40 0x7fe4101f2420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:41.352 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.349+0000 7fe416253640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe41015f6e0 0x7fe4101f2960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:41.352 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.349+0000 7fe416253640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fe410146480 con 0x7fe41015df40 2024-09-12T03:17:41.352 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.349+0000 7fe416253640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe41015f1c0 con 0x7fe4101460a0 2024-09-12T03:17:41.352 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.349+0000 7fe416253640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fe41006e640 con 0x7fe41015f6e0 2024-09-12T03:17:41.352 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.349+0000 7fe40effd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe41015df40 0x7fe4101f2420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:41.352 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.349+0000 7fe40effd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe41015df40 0x7fe4101f2420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60320/0 (socket says 172.21.15.27:60320) 2024-09-12T03:17:41.352 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.349+0000 7fe40effd640 1 -- 172.21.15.27:0/3979374170 learned_addr learned my addr 172.21.15.27:0/3979374170 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:17:41.353 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.349+0000 7fe40ffff640 1 --2- 172.21.15.27:0/3979374170 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe41015f6e0 0x7fe4101f2960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:41.353 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.349+0000 7fe40f7fe640 1 --2- 172.21.15.27:0/3979374170 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe4101460a0 0x7fe4101e78e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:41.353 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.349+0000 7fe40effd640 1 -- 172.21.15.27:0/3979374170 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe41015f6e0 msgr2=0x7fe4101f2960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:41.353 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.349+0000 7fe40effd640 1 --2- 172.21.15.27:0/3979374170 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe41015f6e0 0x7fe4101f2960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:41.353 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.349+0000 7fe40effd640 1 -- 172.21.15.27:0/3979374170 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe4101460a0 msgr2=0x7fe4101e78e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:41.353 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.349+0000 7fe40effd640 1 --2- 172.21.15.27:0/3979374170 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe4101460a0 0x7fe4101e78e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:41.353 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.349+0000 7fe40effd640 1 -- 172.21.15.27:0/3979374170 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4101e6840 con 0x7fe41015df40 2024-09-12T03:17:41.353 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.349+0000 7fe40effd640 1 --2- 172.21.15.27:0/3979374170 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe41015df40 0x7fe4101f2420 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fe4040097a0 tx=0x7fe4040046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:41.353 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.349+0000 7fe3f7fff640 1 -- 172.21.15.27:0/3979374170 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe404037050 con 0x7fe41015df40 2024-09-12T03:17:41.353 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.349+0000 7fe3f7fff640 1 -- 172.21.15.27:0/3979374170 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe404038040 con 0x7fe41015df40 2024-09-12T03:17:41.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.349+0000 7fe416253640 1 -- 172.21.15.27:0/3979374170 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4101e6b20 con 0x7fe41015df40 2024-09-12T03:17:41.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.349+0000 7fe3f7fff640 1 -- 172.21.15.27:0/3979374170 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe404004470 con 0x7fe41015df40 2024-09-12T03:17:41.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.350+0000 7fe416253640 1 -- 172.21.15.27:0/3979374170 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4101e70e0 con 0x7fe41015df40 2024-09-12T03:17:41.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.350+0000 7fe416253640 1 -- 172.21.15.27:0/3979374170 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe41006e100 con 0x7fe41015df40 2024-09-12T03:17:41.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.351+0000 7fe3f7fff640 1 -- 172.21.15.27:0/3979374170 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 7) ==== 90651+0+0 (secure 0 0 0) 0x7fe404030900 con 0x7fe41015df40 2024-09-12T03:17:41.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.351+0000 7fe3f7fff640 1 --2- 172.21.15.27:0/3979374170 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fe3fc06bfe0 0x7fe3fc06e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:41.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.352+0000 7fe3f7fff640 1 -- 172.21.15.27:0/3979374170 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5709+0+0 (secure 0 0 0) 0x7fe4040abad0 con 0x7fe41015df40 2024-09-12T03:17:41.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.352+0000 7fe40f7fe640 1 --2- 172.21.15.27:0/3979374170 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fe3fc06bfe0 0x7fe3fc06e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:41.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.352+0000 7fe40f7fe640 1 --2- 172.21.15.27:0/3979374170 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fe3fc06bfe0 0x7fe3fc06e4a0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fe400003b80 tx=0x7fe4000098d0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:41.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.356+0000 7fe3f7fff640 1 -- 172.21.15.27:0/3979374170 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fe404079cd0 con 0x7fe41015df40 2024-09-12T03:17:41.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.565+0000 7fe416253640 1 -- 172.21.15.27:0/3979374170 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe4100b3250 con 0x7fe41015df40 2024-09-12T03:17:41.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.565+0000 7fe3f7fff640 1 -- 172.21.15.27:0/3979374170 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) ==== 94+0+5223 (secure 0 0 0) 0x7fe40407dfd0 con 0x7fe41015df40 2024-09-12T03:17:41.566 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:17:41.566 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":22,"btime":"2024-09-12T03:17:41:216741+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4835,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6834/1457797481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":1457797481},{"type":"v1","addr":"172.21.15.27:6834","nonce":1457797481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5029,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6836/2842491281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":2842491281},{"type":"v1","addr":"172.21.15.45:6836","nonce":2842491281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5032,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/3339744340","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6835","nonce":3339744340},{"type":"v1","addr":"172.21.15.45:6837","nonce":3339744340}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-12T03:17:38.580473+0000","modified":"2024-09-12T03:17:40.557950+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5205},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5205":{"gid":5205,"name":"a","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.27:6835/2571101228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":2571101228},{"type":"v1","addr":"172.21.15.27:6835","nonce":2571101228}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5205,"qdb_cluster":[5205]},"id":3}]} 2024-09-12T03:17:41.566 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 22 2024-09-12T03:17:41.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.567+0000 7fe416253640 1 -- 172.21.15.27:0/3979374170 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fe3fc06bfe0 msgr2=0x7fe3fc06e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:41.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.567+0000 7fe416253640 1 --2- 172.21.15.27:0/3979374170 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fe3fc06bfe0 0x7fe3fc06e4a0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fe400003b80 tx=0x7fe4000098d0 comp rx=0 tx=0).stop 2024-09-12T03:17:41.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.567+0000 7fe416253640 1 -- 172.21.15.27:0/3979374170 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe41015df40 msgr2=0x7fe4101f2420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:41.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.567+0000 7fe416253640 1 --2- 172.21.15.27:0/3979374170 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe41015df40 0x7fe4101f2420 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fe4040097a0 tx=0x7fe4040046c0 comp rx=0 tx=0).stop 2024-09-12T03:17:41.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.567+0000 7fe416253640 1 -- 172.21.15.27:0/3979374170 shutdown_connections 2024-09-12T03:17:41.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.567+0000 7fe416253640 1 --2- 172.21.15.27:0/3979374170 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fe3fc06bfe0 0x7fe3fc06e4a0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:41.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.567+0000 7fe416253640 1 --2- 172.21.15.27:0/3979374170 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe41015f6e0 0x7fe4101f2960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:41.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.567+0000 7fe416253640 1 --2- 172.21.15.27:0/3979374170 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe41015df40 0x7fe4101f2420 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:41.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.567+0000 7fe416253640 1 --2- 172.21.15.27:0/3979374170 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe4101460a0 0x7fe4101e78e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:41.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.567+0000 7fe416253640 1 -- 172.21.15.27:0/3979374170 >> 172.21.15.27:0/3979374170 conn(0x7fe41006d270 msgr2=0x7fe4100d0790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:41.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.567+0000 7fe416253640 1 -- 172.21.15.27:0/3979374170 shutdown_connections 2024-09-12T03:17:41.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.568+0000 7fe416253640 1 -- 172.21.15.27:0/3979374170 wait complete. 2024-09-12T03:17:41.579 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-12T03:17:41.681 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.681+0000 7fd69f352640 1 Processor -- start 2024-09-12T03:17:41.681 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.681+0000 7fd69f352640 1 -- start start 2024-09-12T03:17:41.682 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.682+0000 7fd69f352640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fd6980d1050 0x7fd6980cf810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:41.682 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.682+0000 7fd69f352640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fd6980d52e0 0x7fd6980d1430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:41.682 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.682+0000 7fd69f352640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd6980d1e60 con 0x7fd698162520 2024-09-12T03:17:41.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.682+0000 7fd69f352640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7fd6980d2380 con 0x7fd6980d52e0 2024-09-12T03:17:41.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.682+0000 7fd69f352640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fd6980cfef0 con 0x7fd6980d1050 2024-09-12T03:17:41.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.682+0000 7fd69c8ed640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fd6980d52e0 0x7fd6980d1430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:41.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.682+0000 7fd69c8ed640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fd6980d52e0 0x7fd6980d1430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60336/0 (socket says 172.21.15.27:60336) 2024-09-12T03:17:41.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.682+0000 7fd69c8ed640 1 -- 172.21.15.27:0/2010691863 learned_addr learned my addr 172.21.15.27:0/2010691863 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:17:41.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.682+0000 7fd69d0ee640 1 --2- 172.21.15.27:0/2010691863 >> v2:172.21.15.45:3300/0 conn(0x7fd6980d1050 0x7fd6980cf810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:41.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.682+0000 7fd69c8ed640 1 -- 172.21.15.27:0/2010691863 >> v2:172.21.15.45:3300/0 conn(0x7fd6980d1050 msgr2=0x7fd6980cf810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:41.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.682+0000 7fd69c8ed640 1 --2- 172.21.15.27:0/2010691863 >> v2:172.21.15.45:3300/0 conn(0x7fd6980d1050 0x7fd6980cf810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:41.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.682+0000 7fd69c8ed640 1 -- 172.21.15.27:0/2010691863 >> v1:172.21.15.45:6789/0 conn(0x7fd698162520 legacy=0x7fd6981629b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:41.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.682+0000 7fd69c8ed640 1 -- 172.21.15.27:0/2010691863 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6980d2640 con 0x7fd6980d52e0 2024-09-12T03:17:41.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.683+0000 7fd69c8ed640 1 --2- 172.21.15.27:0/2010691863 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd6980d52e0 0x7fd6980d1430 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fd69808d180 tx=0x7fd68802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd8fe31c6cd20b5f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:41.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.683+0000 7fd68ffff640 1 -- 172.21.15.27:0/2010691863 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd688037070 con 0x7fd6980d52e0 2024-09-12T03:17:41.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.683+0000 7fd68ffff640 1 -- 172.21.15.27:0/2010691863 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd688032440 con 0x7fd6980d52e0 2024-09-12T03:17:41.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.683+0000 7fd68ffff640 1 -- 172.21.15.27:0/2010691863 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd688032810 con 0x7fd6980d52e0 2024-09-12T03:17:41.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.683+0000 7fd69f352640 1 -- 172.21.15.27:0/2010691863 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd6980d52e0 msgr2=0x7fd6980d1430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:41.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.683+0000 7fd69f352640 1 --2- 172.21.15.27:0/2010691863 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd6980d52e0 0x7fd6980d1430 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fd69808d180 tx=0x7fd68802a230 comp rx=0 tx=0).stop 2024-09-12T03:17:41.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.684+0000 7fd69f352640 1 -- 172.21.15.27:0/2010691863 shutdown_connections 2024-09-12T03:17:41.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.684+0000 7fd69f352640 1 --2- 172.21.15.27:0/2010691863 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd6980d52e0 0x7fd6980d1430 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:41.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.684+0000 7fd69f352640 1 --2- 172.21.15.27:0/2010691863 >> v2:172.21.15.45:3300/0 conn(0x7fd6980d1050 0x7fd6980cf810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:41.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.684+0000 7fd69f352640 1 -- 172.21.15.27:0/2010691863 >> 172.21.15.27:0/2010691863 conn(0x7fd69806ddf0 msgr2=0x7fd69806e200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:41.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.684+0000 7fd69f352640 1 -- 172.21.15.27:0/2010691863 shutdown_connections 2024-09-12T03:17:41.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.684+0000 7fd69f352640 1 -- 172.21.15.27:0/2010691863 wait complete. 2024-09-12T03:17:41.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.685+0000 7fd69f352640 1 Processor -- start 2024-09-12T03:17:41.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.685+0000 7fd69f352640 1 -- start start 2024-09-12T03:17:41.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.685+0000 7fd69f352640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd6980d1050 0x7fd698156e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:41.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.686+0000 7fd69f352640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fd6980d52e0 0x7fd6980cddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:41.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.686+0000 7fd69f352640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd698162520 0x7fd6981590c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:41.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.686+0000 7fd69f352640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fd6980d1970 con 0x7fd6980d1050 2024-09-12T03:17:41.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.686+0000 7fd69f352640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fd6980d2c00 con 0x7fd698162520 2024-09-12T03:17:41.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.686+0000 7fd69f352640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fd6980b7a50 con 0x7fd6980d52e0 2024-09-12T03:17:41.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.686+0000 7fd69d0ee640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd6980d1050 0x7fd698156e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:41.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.686+0000 7fd69d0ee640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd6980d1050 0x7fd698156e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60344/0 (socket says 172.21.15.27:60344) 2024-09-12T03:17:41.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.686+0000 7fd69d0ee640 1 -- 172.21.15.27:0/3024230719 learned_addr learned my addr 172.21.15.27:0/3024230719 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:17:41.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.686+0000 7fd69d8ef640 1 --2- 172.21.15.27:0/3024230719 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd698162520 0x7fd6981590c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:41.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.686+0000 7fd69c8ed640 1 --2- 172.21.15.27:0/3024230719 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fd6980d52e0 0x7fd6980cddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:41.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.686+0000 7fd69d0ee640 1 -- 172.21.15.27:0/3024230719 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fd6980d52e0 msgr2=0x7fd6980cddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:41.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.686+0000 7fd69d0ee640 1 --2- 172.21.15.27:0/3024230719 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fd6980d52e0 0x7fd6980cddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:41.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.686+0000 7fd69d0ee640 1 -- 172.21.15.27:0/3024230719 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd698162520 msgr2=0x7fd6981590c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:41.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.686+0000 7fd69d0ee640 1 --2- 172.21.15.27:0/3024230719 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd698162520 0x7fd6981590c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:41.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.686+0000 7fd69d0ee640 1 -- 172.21.15.27:0/3024230719 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6980ce9a0 con 0x7fd6980d1050 2024-09-12T03:17:41.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.686+0000 7fd69c8ed640 1 --2- 172.21.15.27:0/3024230719 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fd6980d52e0 0x7fd6980cddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:17:41.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.686+0000 7fd69d8ef640 1 --2- 172.21.15.27:0/3024230719 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd698162520 0x7fd6981590c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:17:41.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.686+0000 7fd69d0ee640 1 --2- 172.21.15.27:0/3024230719 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd6980d1050 0x7fd698156e90 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fd69400d5f0 tx=0x7fd69400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:41.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.686+0000 7fd68dffb640 1 -- 172.21.15.27:0/3024230719 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6940125f0 con 0x7fd6980d1050 2024-09-12T03:17:41.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.686+0000 7fd68dffb640 1 -- 172.21.15.27:0/3024230719 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd694019070 con 0x7fd6980d1050 2024-09-12T03:17:41.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.686+0000 7fd69f352640 1 -- 172.21.15.27:0/3024230719 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6980ce6c0 con 0x7fd6980d1050 2024-09-12T03:17:41.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.686+0000 7fd68dffb640 1 -- 172.21.15.27:0/3024230719 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6940129d0 con 0x7fd6980d1050 2024-09-12T03:17:41.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.687+0000 7fd69f352640 1 -- 172.21.15.27:0/3024230719 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6980cc010 con 0x7fd6980d1050 2024-09-12T03:17:41.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.688+0000 7fd69f352640 1 -- 172.21.15.27:0/3024230719 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd69815ec60 con 0x7fd6980d1050 2024-09-12T03:17:41.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.688+0000 7fd68dffb640 1 -- 172.21.15.27:0/3024230719 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 7) ==== 90651+0+0 (secure 0 0 0) 0x7fd694012b70 con 0x7fd6980d1050 2024-09-12T03:17:41.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.688+0000 7fd68dffb640 1 --2- 172.21.15.27:0/3024230719 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fd67c06bfe0 0x7fd67c06e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:41.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.689+0000 7fd68dffb640 1 -- 172.21.15.27:0/3024230719 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5709+0+0 (secure 0 0 0) 0x7fd69408b540 con 0x7fd6980d1050 2024-09-12T03:17:41.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.689+0000 7fd69c8ed640 1 --2- 172.21.15.27:0/3024230719 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fd67c06bfe0 0x7fd67c06e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:41.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.689+0000 7fd69c8ed640 1 --2- 172.21.15.27:0/3024230719 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fd67c06bfe0 0x7fd67c06e4a0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fd688034000 tx=0x7fd68802e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:41.693 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.693+0000 7fd68dffb640 1 -- 172.21.15.27:0/3024230719 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fd694059c80 con 0x7fd6980d1050 2024-09-12T03:17:41.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.873+0000 7fd69f352640 1 -- 172.21.15.27:0/3024230719 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd6980d20a0 con 0x7fd6980d1050 2024-09-12T03:17:41.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.874+0000 7fd68dffb640 1 -- 172.21.15.27:0/3024230719 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v43) ==== 74+0+16625 (secure 0 0 0) 0x7fd694059580 con 0x7fd6980d1050 2024-09-12T03:17:41.874 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:17:41.874 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":43,"fsid":"d2df504e-2647-4385-a5db-9fdf69f487a0","created":"2024-09-12T03:16:07.778937+0000","modified":"2024-09-12T03:17:40.163108+0000","last_up_change":"2024-09-12T03:16:19.314778+0000","last_in_change":"2024-09-12T03:16:09.893086+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T03:16:20.097646+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-09-12T03:17:37.155092+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-09-12T03:17:37.552917+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1851116c-1fb3-4dc9-9772-f90e2ef4c355","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6817","nonce":3002321488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6819","nonce":3002321488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6823","nonce":3002321488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6821","nonce":3002321488}]},"public_addr":"172.21.15.27:6817/3002321488","cluster_addr":"172.21.15.27:6819/3002321488","heartbeat_back_addr":"172.21.15.27:6823/3002321488","heartbeat_front_addr":"172.21.15.27:6821/3002321488","state":["exists","up"]},{"osd":1,"uuid":"f38ae17d-73eb-4204-b99d-975fa806fb87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6809","nonce":1613102886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6811","nonce":1613102886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6815","nonce":1613102886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6813","nonce":1613102886}]},"public_addr":"172.21.15.27:6809/1613102886","cluster_addr":"172.21.15.27:6811/1613102886","heartbeat_back_addr":"172.21.15.27:6815/1613102886","heartbeat_front_addr":"172.21.15.27:6813/1613102886","state":["exists","up"]},{"osd":2,"uuid":"fb12dcc8-bdd0-42b8-b46d-8b10106224bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6825","nonce":3534723355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6827","nonce":3534723355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6831","nonce":3534723355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6829","nonce":3534723355}]},"public_addr":"172.21.15.27:6825/3534723355","cluster_addr":"172.21.15.27:6827/3534723355","heartbeat_back_addr":"172.21.15.27:6831/3534723355","heartbeat_front_addr":"172.21.15.27:6829/3534723355","state":["exists","up"]},{"osd":3,"uuid":"d74d3f49-9c24-4b9f-98e2-3ecd7b5ef3b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6801","nonce":80661598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6803","nonce":80661598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6807","nonce":80661598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6805","nonce":80661598}]},"public_addr":"172.21.15.27:6801/80661598","cluster_addr":"172.21.15.27:6803/80661598","heartbeat_back_addr":"172.21.15.27:6807/80661598","heartbeat_front_addr":"172.21.15.27:6805/80661598","state":["exists","up"]},{"osd":4,"uuid":"2bf123f2-1b0d-41c4-aa3b-805bab267e59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6817","nonce":3943189827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6819","nonce":3943189827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6823","nonce":3943189827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6821","nonce":3943189827}]},"public_addr":"172.21.15.45:6817/3943189827","cluster_addr":"172.21.15.45:6819/3943189827","heartbeat_back_addr":"172.21.15.45:6823/3943189827","heartbeat_front_addr":"172.21.15.45:6821/3943189827","state":["exists","up"]},{"osd":5,"uuid":"c49ba5da-91c6-4a86-a97b-4aba89c253ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6809","nonce":3620760330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6811","nonce":3620760330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6815","nonce":3620760330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6813","nonce":3620760330}]},"public_addr":"172.21.15.45:6809/3620760330","cluster_addr":"172.21.15.45:6811/3620760330","heartbeat_back_addr":"172.21.15.45:6815/3620760330","heartbeat_front_addr":"172.21.15.45:6813/3620760330","state":["exists","up"]},{"osd":6,"uuid":"b0123475-89f7-4a34-9e33-d797b99dd9dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6801","nonce":1989083510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6803","nonce":1989083510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6807","nonce":1989083510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6805","nonce":1989083510}]},"public_addr":"172.21.15.45:6801/1989083510","cluster_addr":"172.21.15.45:6803/1989083510","heartbeat_back_addr":"172.21.15.45:6807/1989083510","heartbeat_front_addr":"172.21.15.45:6805/1989083510","state":["exists","up"]},{"osd":7,"uuid":"4dad773d-b7d8-46fc-8a10-a972aad27b29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6825","nonce":1513300294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6827","nonce":1513300294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6831","nonce":1513300294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6829","nonce":1513300294}]},"public_addr":"172.21.15.45:6825/1513300294","cluster_addr":"172.21.15.45:6827/1513300294","heartbeat_back_addr":"172.21.15.45:6831/1513300294","heartbeat_front_addr":"172.21.15.45:6829/1513300294","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.881855+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:17.133252+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.735339+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.500742+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.477571+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:17.167667+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.470291+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.873743+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T03:17:41.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.876+0000 7fd69f352640 1 -- 172.21.15.27:0/3024230719 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fd67c06bfe0 msgr2=0x7fd67c06e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:41.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.876+0000 7fd69f352640 1 --2- 172.21.15.27:0/3024230719 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fd67c06bfe0 0x7fd67c06e4a0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fd688034000 tx=0x7fd68802e000 comp rx=0 tx=0).stop 2024-09-12T03:17:41.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.876+0000 7fd69f352640 1 -- 172.21.15.27:0/3024230719 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd6980d1050 msgr2=0x7fd698156e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:41.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.876+0000 7fd69f352640 1 --2- 172.21.15.27:0/3024230719 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd6980d1050 0x7fd698156e90 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fd69400d5f0 tx=0x7fd69400dab0 comp rx=0 tx=0).stop 2024-09-12T03:17:41.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.876+0000 7fd69f352640 1 -- 172.21.15.27:0/3024230719 shutdown_connections 2024-09-12T03:17:41.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.876+0000 7fd69f352640 1 --2- 172.21.15.27:0/3024230719 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fd67c06bfe0 0x7fd67c06e4a0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:41.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.876+0000 7fd69f352640 1 --2- 172.21.15.27:0/3024230719 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd698162520 0x7fd6981590c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:41.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.876+0000 7fd69f352640 1 --2- 172.21.15.27:0/3024230719 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fd6980d52e0 0x7fd6980cddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:41.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.876+0000 7fd69f352640 1 --2- 172.21.15.27:0/3024230719 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd6980d1050 0x7fd698156e90 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:41.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.876+0000 7fd69f352640 1 -- 172.21.15.27:0/3024230719 >> 172.21.15.27:0/3024230719 conn(0x7fd69806ddf0 msgr2=0x7fd698162970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:41.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.876+0000 7fd69f352640 1 -- 172.21.15.27:0/3024230719 shutdown_connections 2024-09-12T03:17:41.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.876+0000 7fd69f352640 1 -- 172.21.15.27:0/3024230719 wait complete. 2024-09-12T03:17:41.889 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-12T03:17:41.889 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-12T03:17:41.991 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.991+0000 7fb448fe8640 1 Processor -- start 2024-09-12T03:17:41.991 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.991+0000 7fb448fe8640 1 -- start start 2024-09-12T03:17:41.992 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.992+0000 7fb448fe8640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fb444145c30 0x7fb44415ca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:41.993 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.992+0000 7fb448fe8640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb444146010 0x7fb4440cd330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:41.993 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.992+0000 7fb448fe8640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fb4440cd870 0x7fb4440b4040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:41.993 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.992+0000 7fb448fe8640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7fb4440b5050 con 0x7fb444145c30 2024-09-12T03:17:41.993 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.992+0000 7fb448fe8640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fb4440b4d70 con 0x7fb4440cd870 2024-09-12T03:17:41.993 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.992+0000 7fb448fe8640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fb44406d140 con 0x7fb444146010 2024-09-12T03:17:41.993 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.992+0000 7fb44259c640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fb444145c30 0x7fb44415ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:41.993 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.992+0000 7fb44259c640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fb444145c30 0x7fb44415ca00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60348/0 (socket says 172.21.15.27:60348) 2024-09-12T03:17:41.993 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.992+0000 7fb44259c640 1 -- 172.21.15.27:0/3679122643 learned_addr learned my addr 172.21.15.27:0/3679122643 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:17:41.993 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.992+0000 7fb442d9d640 1 --2- 172.21.15.27:0/3679122643 >> v2:172.21.15.45:3300/0 conn(0x7fb4440cd870 0x7fb4440b4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:41.993 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.992+0000 7fb44259c640 1 -- 172.21.15.27:0/3679122643 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb444146010 msgr2=0x7fb4440cd330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:41.993 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.992+0000 7fb44259c640 1 --2- 172.21.15.27:0/3679122643 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb444146010 0x7fb4440cd330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:41.993 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.992+0000 7fb44259c640 1 -- 172.21.15.27:0/3679122643 >> v2:172.21.15.45:3300/0 conn(0x7fb4440cd870 msgr2=0x7fb4440b4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:41.994 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.992+0000 7fb44259c640 1 --2- 172.21.15.27:0/3679122643 >> v2:172.21.15.45:3300/0 conn(0x7fb4440cd870 0x7fb4440b4040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:41.994 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.992+0000 7fb44259c640 1 -- 172.21.15.27:0/3679122643 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb44406d420 con 0x7fb444145c30 2024-09-12T03:17:41.995 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.993+0000 7fb44259c640 1 --2- 172.21.15.27:0/3679122643 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb444145c30 0x7fb44415ca00 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fb4440b3f20 tx=0x7fb43c02a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d20c5e830c3256 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:41.995 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.993+0000 7fb44159a640 1 -- 172.21.15.27:0/3679122643 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb43c037070 con 0x7fb444145c30 2024-09-12T03:17:41.995 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.993+0000 7fb44159a640 1 -- 172.21.15.27:0/3679122643 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb43c032440 con 0x7fb444145c30 2024-09-12T03:17:41.995 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.993+0000 7fb44159a640 1 -- 172.21.15.27:0/3679122643 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb43c032810 con 0x7fb444145c30 2024-09-12T03:17:41.995 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.994+0000 7fb448fe8640 1 -- 172.21.15.27:0/3679122643 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb444145c30 msgr2=0x7fb44415ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:41.995 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.994+0000 7fb448fe8640 1 --2- 172.21.15.27:0/3679122643 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb444145c30 0x7fb44415ca00 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fb4440b3f20 tx=0x7fb43c02a140 comp rx=0 tx=0).stop 2024-09-12T03:17:41.995 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.994+0000 7fb448fe8640 1 -- 172.21.15.27:0/3679122643 shutdown_connections 2024-09-12T03:17:41.996 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.994+0000 7fb448fe8640 1 --2- 172.21.15.27:0/3679122643 >> v2:172.21.15.45:3300/0 conn(0x7fb4440cd870 0x7fb4440b4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:41.996 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.994+0000 7fb448fe8640 1 --2- 172.21.15.27:0/3679122643 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb444146010 0x7fb4440cd330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:41.996 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.994+0000 7fb448fe8640 1 --2- 172.21.15.27:0/3679122643 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb444145c30 0x7fb44415ca00 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:41.996 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.994+0000 7fb448fe8640 1 -- 172.21.15.27:0/3679122643 >> 172.21.15.27:0/3679122643 conn(0x7fb444062c60 msgr2=0x7fb44406bb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:41.996 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.994+0000 7fb448fe8640 1 -- 172.21.15.27:0/3679122643 shutdown_connections 2024-09-12T03:17:41.996 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.994+0000 7fb448fe8640 1 -- 172.21.15.27:0/3679122643 wait complete. 2024-09-12T03:17:41.996 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.995+0000 7fb448fe8640 1 Processor -- start 2024-09-12T03:17:41.996 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.995+0000 7fb448fe8640 1 -- start start 2024-09-12T03:17:41.998 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.996+0000 7fb448fe8640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb4440cd870 0x7fb44415bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:41.998 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.996+0000 7fb448fe8640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb444145c30 0x7fb44413b530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:41.998 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.996+0000 7fb448fe8640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb444146010 0x7fb44413ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:41.998 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.996+0000 7fb448fe8640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fb4440b4700 con 0x7fb4440cd870 2024-09-12T03:17:41.998 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.996+0000 7fb448fe8640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb4440b4580 con 0x7fb444145c30 2024-09-12T03:17:41.999 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.996+0000 7fb448fe8640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fb4440b4880 con 0x7fb444146010 2024-09-12T03:17:41.999 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.996+0000 7fb44259c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb4440cd870 0x7fb44415bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:41.999 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.996+0000 7fb44259c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb4440cd870 0x7fb44415bfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60350/0 (socket says 172.21.15.27:60350) 2024-09-12T03:17:41.999 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.996+0000 7fb44259c640 1 -- 172.21.15.27:0/2081710642 learned_addr learned my addr 172.21.15.27:0/2081710642 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:17:41.999 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.996+0000 7fb442d9d640 1 --2- 172.21.15.27:0/2081710642 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb444146010 0x7fb44413ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:41.999 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.996+0000 7fb441d9b640 1 --2- 172.21.15.27:0/2081710642 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb444145c30 0x7fb44413b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:41.999 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.996+0000 7fb44259c640 1 -- 172.21.15.27:0/2081710642 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb444146010 msgr2=0x7fb44413ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:41.999 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.996+0000 7fb44259c640 1 --2- 172.21.15.27:0/2081710642 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb444146010 0x7fb44413ba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:41.999 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.996+0000 7fb44259c640 1 -- 172.21.15.27:0/2081710642 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb444145c30 msgr2=0x7fb44413b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:41.999 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.996+0000 7fb44259c640 1 --2- 172.21.15.27:0/2081710642 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb444145c30 0x7fb44413b530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:41.999 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.996+0000 7fb44259c640 1 -- 172.21.15.27:0/2081710642 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb44413b1f0 con 0x7fb4440cd870 2024-09-12T03:17:41.999 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.997+0000 7fb44259c640 1 --2- 172.21.15.27:0/2081710642 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb4440cd870 0x7fb44415bfe0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fb43c000c00 tx=0x7fb43c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:41.999 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.997+0000 7fb43affd640 1 -- 172.21.15.27:0/2081710642 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb43c041020 con 0x7fb4440cd870 2024-09-12T03:17:42.000 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.997+0000 7fb43affd640 1 -- 172.21.15.27:0/2081710642 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb43c030970 con 0x7fb4440cd870 2024-09-12T03:17:42.000 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.997+0000 7fb448fe8640 1 -- 172.21.15.27:0/2081710642 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb44413ac30 con 0x7fb4440cd870 2024-09-12T03:17:42.000 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.997+0000 7fb43affd640 1 -- 172.21.15.27:0/2081710642 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb43c037040 con 0x7fb4440cd870 2024-09-12T03:17:42.000 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.997+0000 7fb448fe8640 1 -- 172.21.15.27:0/2081710642 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb44413d470 con 0x7fb4440cd870 2024-09-12T03:17:42.000 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.998+0000 7fb448fe8640 1 -- 172.21.15.27:0/2081710642 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb44415ca00 con 0x7fb4440cd870 2024-09-12T03:17:42.000 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.998+0000 7fb43affd640 1 -- 172.21.15.27:0/2081710642 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 7) ==== 90651+0+0 (secure 0 0 0) 0x7fb43c044020 con 0x7fb4440cd870 2024-09-12T03:17:42.001 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.999+0000 7fb43affd640 1 --2- 172.21.15.27:0/2081710642 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fb42406bfe0 0x7fb42406e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:42.001 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.999+0000 7fb43affd640 1 -- 172.21.15.27:0/2081710642 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5709+0+0 (secure 0 0 0) 0x7fb43c0ab7b0 con 0x7fb4440cd870 2024-09-12T03:17:42.001 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:41.999+0000 7fb441d9b640 1 --2- 172.21.15.27:0/2081710642 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fb42406bfe0 0x7fb42406e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:42.001 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:42.000+0000 7fb441d9b640 1 --2- 172.21.15.27:0/2081710642 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fb42406bfe0 0x7fb42406e4a0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fb4441399f0 tx=0x7fb42c008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:42.003 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:42.003+0000 7fb43affd640 1 -- 172.21.15.27:0/2081710642 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fb43c04b070 con 0x7fb4440cd870 2024-09-12T03:17:42.175 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:42.175+0000 7fb448fe8640 1 -- 172.21.15.27:0/2081710642 --> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb4440af980 con 0x7fb42406bfe0 2024-09-12T03:17:42.182 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:42.183+0000 7fb43affd640 1 -- 172.21.15.27:0/2081710642 <== mgr.4102 v2:172.21.15.45:6832/3923821219 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251891 (secure 0 0 0) 0x7fb4440af980 con 0x7fb42406bfe0 2024-09-12T03:17:42.186 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:17:42.187 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2024-09-12T03:17:42.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:42.186+0000 7fb448fe8640 1 -- 172.21.15.27:0/2081710642 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fb42406bfe0 msgr2=0x7fb42406e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:42.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:42.186+0000 7fb448fe8640 1 --2- 172.21.15.27:0/2081710642 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fb42406bfe0 0x7fb42406e4a0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fb4441399f0 tx=0x7fb42c008040 comp rx=0 tx=0).stop 2024-09-12T03:17:42.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:42.186+0000 7fb448fe8640 1 -- 172.21.15.27:0/2081710642 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb4440cd870 msgr2=0x7fb44415bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:42.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:42.186+0000 7fb448fe8640 1 --2- 172.21.15.27:0/2081710642 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb4440cd870 0x7fb44415bfe0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fb43c000c00 tx=0x7fb43c0046c0 comp rx=0 tx=0).stop 2024-09-12T03:17:42.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:42.186+0000 7fb448fe8640 1 -- 172.21.15.27:0/2081710642 shutdown_connections 2024-09-12T03:17:42.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:42.186+0000 7fb448fe8640 1 --2- 172.21.15.27:0/2081710642 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fb42406bfe0 0x7fb42406e4a0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:42.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:42.186+0000 7fb448fe8640 1 --2- 172.21.15.27:0/2081710642 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb444146010 0x7fb44413ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:42.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:42.186+0000 7fb448fe8640 1 --2- 172.21.15.27:0/2081710642 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb444145c30 0x7fb44413b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:42.190 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:42.186+0000 7fb448fe8640 1 --2- 172.21.15.27:0/2081710642 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb4440cd870 0x7fb44415bfe0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:42.190 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:42.186+0000 7fb448fe8640 1 -- 172.21.15.27:0/2081710642 >> 172.21.15.27:0/2081710642 conn(0x7fb444062c60 msgr2=0x7fb444071110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:42.190 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:42.186+0000 7fb448fe8640 1 -- 172.21.15.27:0/2081710642 shutdown_connections 2024-09-12T03:17:42.190 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:42.186+0000 7fb448fe8640 1 -- 172.21.15.27:0/2081710642 wait complete. 2024-09-12T03:17:42.196 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":77,"stamp":"2024-09-12T03:17:42.066929+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":111,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":242064,"kb_used_data":5028,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754732656,"statfs":{"total":773094113280,"available":772846239744,"internally_reserved":0,"allocated":5148672,"data_stored":2739462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":242535984},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.859211"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T03:17:39.177700+0000","last_change":"2024-09-12T03:17:39.173270+0000","last_active":"2024-09-12T03:17:39.177700+0000","last_peered":"2024-09-12T03:17:39.177700+0000","last_clean":"2024-09-12T03:17:39.177700+0000","last_became_active":"2024-09-12T03:17:39.172969+0000","last_became_peered":"2024-09-12T03:17:39.172969+0000","last_unstale":"2024-09-12T03:17:39.177700+0000","last_undegraded":"2024-09-12T03:17:39.177700+0000","last_fullsized":"2024-09-12T03:17:39.177700+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_clean_scrub_stamp":"2024-09-12T03:17:38.207304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:00:46.565610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T03:17:39.187235+0000","last_change":"2024-09-12T03:17:39.184842+0000","last_active":"2024-09-12T03:17:39.187235+0000","last_peered":"2024-09-12T03:17:39.187235+0000","last_clean":"2024-09-12T03:17:39.187235+0000","last_became_active":"2024-09-12T03:17:39.184546+0000","last_became_peered":"2024-09-12T03:17:39.184546+0000","last_unstale":"2024-09-12T03:17:39.187235+0000","last_undegraded":"2024-09-12T03:17:39.187235+0000","last_fullsized":"2024-09-12T03:17:39.187235+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_clean_scrub_stamp":"2024-09-12T03:17:38.207304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:03:06.629215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T03:17:39.184307+0000","last_change":"2024-09-12T03:17:39.181688+0000","last_active":"2024-09-12T03:17:39.184307+0000","last_peered":"2024-09-12T03:17:39.184307+0000","last_clean":"2024-09-12T03:17:39.184307+0000","last_became_active":"2024-09-12T03:17:39.181260+0000","last_became_peered":"2024-09-12T03:17:39.181260+0000","last_unstale":"2024-09-12T03:17:39.184307+0000","last_undegraded":"2024-09-12T03:17:39.184307+0000","last_fullsized":"2024-09-12T03:17:39.184307+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_clean_scrub_stamp":"2024-09-12T03:17:38.207304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:22:59.414373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T03:17:39.195119+0000","last_change":"2024-09-12T03:17:39.191333+0000","last_active":"2024-09-12T03:17:39.195119+0000","last_peered":"2024-09-12T03:17:39.195119+0000","last_clean":"2024-09-12T03:17:39.195119+0000","last_became_active":"2024-09-12T03:17:39.190901+0000","last_became_peered":"2024-09-12T03:17:39.190901+0000","last_unstale":"2024-09-12T03:17:39.195119+0000","last_undegraded":"2024-09-12T03:17:39.195119+0000","last_fullsized":"2024-09-12T03:17:39.195119+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_clean_scrub_stamp":"2024-09-12T03:17:38.207304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:55:43.983680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T03:17:39.186605+0000","last_change":"2024-09-12T03:17:39.183448+0000","last_active":"2024-09-12T03:17:39.186605+0000","last_peered":"2024-09-12T03:17:39.186605+0000","last_clean":"2024-09-12T03:17:39.186605+0000","last_became_active":"2024-09-12T03:17:39.183167+0000","last_became_peered":"2024-09-12T03:17:39.183167+0000","last_unstale":"2024-09-12T03:17:39.186605+0000","last_undegraded":"2024-09-12T03:17:39.186605+0000","last_fullsized":"2024-09-12T03:17:39.186605+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_clean_scrub_stamp":"2024-09-12T03:17:38.207304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:48:08.910749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T03:17:39.175494+0000","last_change":"2024-09-12T03:17:39.172605+0000","last_active":"2024-09-12T03:17:39.175494+0000","last_peered":"2024-09-12T03:17:39.175494+0000","last_clean":"2024-09-12T03:17:39.175494+0000","last_became_active":"2024-09-12T03:17:39.172129+0000","last_became_peered":"2024-09-12T03:17:39.172129+0000","last_unstale":"2024-09-12T03:17:39.175494+0000","last_undegraded":"2024-09-12T03:17:39.175494+0000","last_fullsized":"2024-09-12T03:17:39.175494+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_clean_scrub_stamp":"2024-09-12T03:17:38.207304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:06:36.818875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T03:17:39.182104+0000","last_change":"2024-09-12T03:17:39.178938+0000","last_active":"2024-09-12T03:17:39.182104+0000","last_peered":"2024-09-12T03:17:39.182104+0000","last_clean":"2024-09-12T03:17:39.182104+0000","last_became_active":"2024-09-12T03:17:39.178657+0000","last_became_peered":"2024-09-12T03:17:39.178657+0000","last_unstale":"2024-09-12T03:17:39.182104+0000","last_undegraded":"2024-09-12T03:17:39.182104+0000","last_fullsized":"2024-09-12T03:17:39.182104+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_clean_scrub_stamp":"2024-09-12T03:17:38.207304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:53:01.961305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T03:17:39.195066+0000","last_change":"2024-09-12T03:17:39.191234+0000","last_active":"2024-09-12T03:17:39.195066+0000","last_peered":"2024-09-12T03:17:39.195066+0000","last_clean":"2024-09-12T03:17:39.195066+0000","last_became_active":"2024-09-12T03:17:39.190933+0000","last_became_peered":"2024-09-12T03:17:39.190933+0000","last_unstale":"2024-09-12T03:17:39.195066+0000","last_undegraded":"2024-09-12T03:17:39.195066+0000","last_fullsized":"2024-09-12T03:17:39.195066+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_clean_scrub_stamp":"2024-09-12T03:17:38.207304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:28:57.726725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T03:17:39.194972+0000","last_change":"2024-09-12T03:17:39.189763+0000","last_active":"2024-09-12T03:17:39.194972+0000","last_peered":"2024-09-12T03:17:39.194972+0000","last_clean":"2024-09-12T03:17:39.194972+0000","last_became_active":"2024-09-12T03:17:39.189331+0000","last_became_peered":"2024-09-12T03:17:39.189331+0000","last_unstale":"2024-09-12T03:17:39.194972+0000","last_undegraded":"2024-09-12T03:17:39.194972+0000","last_fullsized":"2024-09-12T03:17:39.194972+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_clean_scrub_stamp":"2024-09-12T03:17:38.207304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:57:56.207274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T03:17:39.195148+0000","last_change":"2024-09-12T03:17:39.191587+0000","last_active":"2024-09-12T03:17:39.195148+0000","last_peered":"2024-09-12T03:17:39.195148+0000","last_clean":"2024-09-12T03:17:39.195148+0000","last_became_active":"2024-09-12T03:17:39.191224+0000","last_became_peered":"2024-09-12T03:17:39.191224+0000","last_unstale":"2024-09-12T03:17:39.195148+0000","last_undegraded":"2024-09-12T03:17:39.195148+0000","last_fullsized":"2024-09-12T03:17:39.195148+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_clean_scrub_stamp":"2024-09-12T03:17:38.207304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:59:20.332763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T03:17:39.188634+0000","last_change":"2024-09-12T03:17:39.185246+0000","last_active":"2024-09-12T03:17:39.188634+0000","last_peered":"2024-09-12T03:17:39.188634+0000","last_clean":"2024-09-12T03:17:39.188634+0000","last_became_active":"2024-09-12T03:17:39.184976+0000","last_became_peered":"2024-09-12T03:17:39.184976+0000","last_unstale":"2024-09-12T03:17:39.188634+0000","last_undegraded":"2024-09-12T03:17:39.188634+0000","last_fullsized":"2024-09-12T03:17:39.188634+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_clean_scrub_stamp":"2024-09-12T03:17:38.207304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:53:41.873125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T03:17:39.186672+0000","last_change":"2024-09-12T03:17:39.183143+0000","last_active":"2024-09-12T03:17:39.186672+0000","last_peered":"2024-09-12T03:17:39.186672+0000","last_clean":"2024-09-12T03:17:39.186672+0000","last_became_active":"2024-09-12T03:17:39.182860+0000","last_became_peered":"2024-09-12T03:17:39.182860+0000","last_unstale":"2024-09-12T03:17:39.186672+0000","last_undegraded":"2024-09-12T03:17:39.186672+0000","last_fullsized":"2024-09-12T03:17:39.186672+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_clean_scrub_stamp":"2024-09-12T03:17:38.207304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:19:09.042280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T03:17:39.190558+0000","last_change":"2024-09-12T03:17:39.187326+0000","last_active":"2024-09-12T03:17:39.190558+0000","last_peered":"2024-09-12T03:17:39.190558+0000","last_clean":"2024-09-12T03:17:39.190558+0000","last_became_active":"2024-09-12T03:17:39.187040+0000","last_became_peered":"2024-09-12T03:17:39.187040+0000","last_unstale":"2024-09-12T03:17:39.190558+0000","last_undegraded":"2024-09-12T03:17:39.190558+0000","last_fullsized":"2024-09-12T03:17:39.190558+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_clean_scrub_stamp":"2024-09-12T03:17:38.207304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:09:03.576255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T03:17:39.194928+0000","last_change":"2024-09-12T03:17:39.187777+0000","last_active":"2024-09-12T03:17:39.194928+0000","last_peered":"2024-09-12T03:17:39.194928+0000","last_clean":"2024-09-12T03:17:39.194928+0000","last_became_active":"2024-09-12T03:17:39.187489+0000","last_became_peered":"2024-09-12T03:17:39.187489+0000","last_unstale":"2024-09-12T03:17:39.194928+0000","last_undegraded":"2024-09-12T03:17:39.194928+0000","last_fullsized":"2024-09-12T03:17:39.194928+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_clean_scrub_stamp":"2024-09-12T03:17:38.207304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:26:11.464219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T03:17:39.183039+0000","last_change":"2024-09-12T03:17:39.172527+0000","last_active":"2024-09-12T03:17:39.183039+0000","last_peered":"2024-09-12T03:17:39.183039+0000","last_clean":"2024-09-12T03:17:39.183039+0000","last_became_active":"2024-09-12T03:17:39.172224+0000","last_became_peered":"2024-09-12T03:17:39.172224+0000","last_unstale":"2024-09-12T03:17:39.183039+0000","last_undegraded":"2024-09-12T03:17:39.183039+0000","last_fullsized":"2024-09-12T03:17:39.183039+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_clean_scrub_stamp":"2024-09-12T03:17:38.207304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:20:59.682175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T03:17:39.190556+0000","last_change":"2024-09-12T03:17:39.189749+0000","last_active":"2024-09-12T03:17:39.190556+0000","last_peered":"2024-09-12T03:17:39.190556+0000","last_clean":"2024-09-12T03:17:39.190556+0000","last_became_active":"2024-09-12T03:17:39.189449+0000","last_became_peered":"2024-09-12T03:17:39.189449+0000","last_unstale":"2024-09-12T03:17:39.190556+0000","last_undegraded":"2024-09-12T03:17:39.190556+0000","last_fullsized":"2024-09-12T03:17:39.190556+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_clean_scrub_stamp":"2024-09-12T03:17:38.207304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:30:25.513267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T03:17:39.179973+0000","last_change":"2024-09-12T03:17:39.179077+0000","last_active":"2024-09-12T03:17:39.179973+0000","last_peered":"2024-09-12T03:17:39.179973+0000","last_clean":"2024-09-12T03:17:39.179973+0000","last_became_active":"2024-09-12T03:17:39.178771+0000","last_became_peered":"2024-09-12T03:17:39.178771+0000","last_unstale":"2024-09-12T03:17:39.179973+0000","last_undegraded":"2024-09-12T03:17:39.179973+0000","last_fullsized":"2024-09-12T03:17:39.179973+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_clean_scrub_stamp":"2024-09-12T03:17:38.207304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:51:18.036332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T03:17:39.187503+0000","last_change":"2024-09-12T03:17:39.186110+0000","last_active":"2024-09-12T03:17:39.187503+0000","last_peered":"2024-09-12T03:17:39.187503+0000","last_clean":"2024-09-12T03:17:39.187503+0000","last_became_active":"2024-09-12T03:17:39.185723+0000","last_became_peered":"2024-09-12T03:17:39.185723+0000","last_unstale":"2024-09-12T03:17:39.187503+0000","last_undegraded":"2024-09-12T03:17:39.187503+0000","last_fullsized":"2024-09-12T03:17:39.187503+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_clean_scrub_stamp":"2024-09-12T03:17:38.207304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:42:37.751637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T03:17:39.187837+0000","last_change":"2024-09-12T03:17:39.187049+0000","last_active":"2024-09-12T03:17:39.187837+0000","last_peered":"2024-09-12T03:17:39.187837+0000","last_clean":"2024-09-12T03:17:39.187837+0000","last_became_active":"2024-09-12T03:17:39.186743+0000","last_became_peered":"2024-09-12T03:17:39.186743+0000","last_unstale":"2024-09-12T03:17:39.187837+0000","last_undegraded":"2024-09-12T03:17:39.187837+0000","last_fullsized":"2024-09-12T03:17:39.187837+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_clean_scrub_stamp":"2024-09-12T03:17:38.207304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:00:52.475262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T03:17:39.190649+0000","last_change":"2024-09-12T03:17:39.189386+0000","last_active":"2024-09-12T03:17:39.190649+0000","last_peered":"2024-09-12T03:17:39.190649+0000","last_clean":"2024-09-12T03:17:39.190649+0000","last_became_active":"2024-09-12T03:17:39.189052+0000","last_became_peered":"2024-09-12T03:17:39.189052+0000","last_unstale":"2024-09-12T03:17:39.190649+0000","last_undegraded":"2024-09-12T03:17:39.190649+0000","last_fullsized":"2024-09-12T03:17:39.190649+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_clean_scrub_stamp":"2024-09-12T03:17:38.207304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:20:24.469810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T03:17:39.187224+0000","last_change":"2024-09-12T03:17:39.178220+0000","last_active":"2024-09-12T03:17:39.187224+0000","last_peered":"2024-09-12T03:17:39.187224+0000","last_clean":"2024-09-12T03:17:39.187224+0000","last_became_active":"2024-09-12T03:17:39.177874+0000","last_became_peered":"2024-09-12T03:17:39.177874+0000","last_unstale":"2024-09-12T03:17:39.187224+0000","last_undegraded":"2024-09-12T03:17:39.187224+0000","last_fullsized":"2024-09-12T03:17:39.187224+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_clean_scrub_stamp":"2024-09-12T03:17:38.207304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:37:43.236716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T03:17:39.195129+0000","last_change":"2024-09-12T03:17:39.191543+0000","last_active":"2024-09-12T03:17:39.195129+0000","last_peered":"2024-09-12T03:17:39.195129+0000","last_clean":"2024-09-12T03:17:39.195129+0000","last_became_active":"2024-09-12T03:17:39.191105+0000","last_became_peered":"2024-09-12T03:17:39.191105+0000","last_unstale":"2024-09-12T03:17:39.195129+0000","last_undegraded":"2024-09-12T03:17:39.195129+0000","last_fullsized":"2024-09-12T03:17:39.195129+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_clean_scrub_stamp":"2024-09-12T03:17:38.207304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:47:14.407194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T03:17:39.186339+0000","last_change":"2024-09-12T03:17:39.184418+0000","last_active":"2024-09-12T03:17:39.186339+0000","last_peered":"2024-09-12T03:17:39.186339+0000","last_clean":"2024-09-12T03:17:39.186339+0000","last_became_active":"2024-09-12T03:17:39.183640+0000","last_became_peered":"2024-09-12T03:17:39.183640+0000","last_unstale":"2024-09-12T03:17:39.186339+0000","last_undegraded":"2024-09-12T03:17:39.186339+0000","last_fullsized":"2024-09-12T03:17:39.186339+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_clean_scrub_stamp":"2024-09-12T03:17:38.207304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:02:19.482735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T03:17:39.186142+0000","last_change":"2024-09-12T03:17:39.184147+0000","last_active":"2024-09-12T03:17:39.186142+0000","last_peered":"2024-09-12T03:17:39.186142+0000","last_clean":"2024-09-12T03:17:39.186142+0000","last_became_active":"2024-09-12T03:17:39.183851+0000","last_became_peered":"2024-09-12T03:17:39.183851+0000","last_unstale":"2024-09-12T03:17:39.186142+0000","last_undegraded":"2024-09-12T03:17:39.186142+0000","last_fullsized":"2024-09-12T03:17:39.186142+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_clean_scrub_stamp":"2024-09-12T03:17:38.207304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:23:28.460549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T03:17:39.185505+0000","last_change":"2024-09-12T03:17:39.182136+0000","last_active":"2024-09-12T03:17:39.185505+0000","last_peered":"2024-09-12T03:17:39.185505+0000","last_clean":"2024-09-12T03:17:39.185505+0000","last_became_active":"2024-09-12T03:17:39.181819+0000","last_became_peered":"2024-09-12T03:17:39.181819+0000","last_unstale":"2024-09-12T03:17:39.185505+0000","last_undegraded":"2024-09-12T03:17:39.185505+0000","last_fullsized":"2024-09-12T03:17:39.185505+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_clean_scrub_stamp":"2024-09-12T03:17:38.207304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:56:18.897560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T03:17:39.190583+0000","last_change":"2024-09-12T03:17:39.188667+0000","last_active":"2024-09-12T03:17:39.190583+0000","last_peered":"2024-09-12T03:17:39.190583+0000","last_clean":"2024-09-12T03:17:39.190583+0000","last_became_active":"2024-09-12T03:17:39.188352+0000","last_became_peered":"2024-09-12T03:17:39.188352+0000","last_unstale":"2024-09-12T03:17:39.190583+0000","last_undegraded":"2024-09-12T03:17:39.190583+0000","last_fullsized":"2024-09-12T03:17:39.190583+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_clean_scrub_stamp":"2024-09-12T03:17:38.207304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:36:50.682702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T03:17:39.180193+0000","last_change":"2024-09-12T03:17:39.178027+0000","last_active":"2024-09-12T03:17:39.180193+0000","last_peered":"2024-09-12T03:17:39.180193+0000","last_clean":"2024-09-12T03:17:39.180193+0000","last_became_active":"2024-09-12T03:17:39.177682+0000","last_became_peered":"2024-09-12T03:17:39.177682+0000","last_unstale":"2024-09-12T03:17:39.180193+0000","last_undegraded":"2024-09-12T03:17:39.180193+0000","last_fullsized":"2024-09-12T03:17:39.180193+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_clean_scrub_stamp":"2024-09-12T03:17:38.207304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:28:33.787507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T03:17:39.186649+0000","last_change":"2024-09-12T03:17:39.183076+0000","last_active":"2024-09-12T03:17:39.186649+0000","last_peered":"2024-09-12T03:17:39.186649+0000","last_clean":"2024-09-12T03:17:39.186649+0000","last_became_active":"2024-09-12T03:17:39.182765+0000","last_became_peered":"2024-09-12T03:17:39.182765+0000","last_unstale":"2024-09-12T03:17:39.186649+0000","last_undegraded":"2024-09-12T03:17:39.186649+0000","last_fullsized":"2024-09-12T03:17:39.186649+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_clean_scrub_stamp":"2024-09-12T03:17:38.207304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:52:55.260126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T03:17:39.180047+0000","last_change":"2024-09-12T03:17:39.178484+0000","last_active":"2024-09-12T03:17:39.180047+0000","last_peered":"2024-09-12T03:17:39.180047+0000","last_clean":"2024-09-12T03:17:39.180047+0000","last_became_active":"2024-09-12T03:17:39.178149+0000","last_became_peered":"2024-09-12T03:17:39.178149+0000","last_unstale":"2024-09-12T03:17:39.180047+0000","last_undegraded":"2024-09-12T03:17:39.180047+0000","last_fullsized":"2024-09-12T03:17:39.180047+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_clean_scrub_stamp":"2024-09-12T03:17:38.207304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:11:19.835866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T03:17:39.171711+0000","last_change":"2024-09-12T03:17:39.170360+0000","last_active":"2024-09-12T03:17:39.171711+0000","last_peered":"2024-09-12T03:17:39.171711+0000","last_clean":"2024-09-12T03:17:39.171711+0000","last_became_active":"2024-09-12T03:17:39.169980+0000","last_became_peered":"2024-09-12T03:17:39.169980+0000","last_unstale":"2024-09-12T03:17:39.171711+0000","last_undegraded":"2024-09-12T03:17:39.171711+0000","last_fullsized":"2024-09-12T03:17:39.171711+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_clean_scrub_stamp":"2024-09-12T03:17:38.207304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:28:17.759494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T03:17:39.187808+0000","last_change":"2024-09-12T03:17:39.183165+0000","last_active":"2024-09-12T03:17:39.187808+0000","last_peered":"2024-09-12T03:17:39.187808+0000","last_clean":"2024-09-12T03:17:39.187808+0000","last_became_active":"2024-09-12T03:17:39.182828+0000","last_became_peered":"2024-09-12T03:17:39.182828+0000","last_unstale":"2024-09-12T03:17:39.187808+0000","last_undegraded":"2024-09-12T03:17:39.187808+0000","last_fullsized":"2024-09-12T03:17:39.187808+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_clean_scrub_stamp":"2024-09-12T03:17:38.207304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:48:16.421160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T03:17:39.182465+0000","last_change":"2024-09-12T03:17:39.180606+0000","last_active":"2024-09-12T03:17:39.182465+0000","last_peered":"2024-09-12T03:17:39.182465+0000","last_clean":"2024-09-12T03:17:39.182465+0000","last_became_active":"2024-09-12T03:17:39.180260+0000","last_became_peered":"2024-09-12T03:17:39.180260+0000","last_unstale":"2024-09-12T03:17:39.182465+0000","last_undegraded":"2024-09-12T03:17:39.182465+0000","last_fullsized":"2024-09-12T03:17:39.182465+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_clean_scrub_stamp":"2024-09-12T03:17:38.207304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:41:31.294009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T03:17:39.192599+0000","last_change":"2024-09-12T03:17:39.188782+0000","last_active":"2024-09-12T03:17:39.192599+0000","last_peered":"2024-09-12T03:17:39.192599+0000","last_clean":"2024-09-12T03:17:39.192599+0000","last_became_active":"2024-09-12T03:17:39.188477+0000","last_became_peered":"2024-09-12T03:17:39.188477+0000","last_unstale":"2024-09-12T03:17:39.192599+0000","last_undegraded":"2024-09-12T03:17:39.192599+0000","last_fullsized":"2024-09-12T03:17:39.192599+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_clean_scrub_stamp":"2024-09-12T03:17:38.207304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:51:02.079450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"42'4","reported_seq":16,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T03:17:39.223851+0000","last_change":"2024-09-12T03:17:38.226341+0000","last_active":"2024-09-12T03:17:39.223851+0000","last_peered":"2024-09-12T03:17:39.223851+0000","last_clean":"2024-09-12T03:17:39.223851+0000","last_became_active":"2024-09-12T03:17:38.226069+0000","last_became_peered":"2024-09-12T03:17:38.226069+0000","last_unstale":"2024-09-12T03:17:39.223851+0000","last_undegraded":"2024-09-12T03:17:39.223851+0000","last_fullsized":"2024-09-12T03:17:39.223851+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:37.205254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:37.205254+0000","last_clean_scrub_stamp":"2024-09-12T03:17:37.205254+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:55:42.981630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T03:17:39.181694+0000","last_change":"2024-09-12T03:17:39.178101+0000","last_active":"2024-09-12T03:17:39.181694+0000","last_peered":"2024-09-12T03:17:39.181694+0000","last_clean":"2024-09-12T03:17:39.181694+0000","last_became_active":"2024-09-12T03:17:39.177770+0000","last_became_peered":"2024-09-12T03:17:39.177770+0000","last_unstale":"2024-09-12T03:17:39.181694+0000","last_undegraded":"2024-09-12T03:17:39.181694+0000","last_fullsized":"2024-09-12T03:17:39.181694+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_clean_scrub_stamp":"2024-09-12T03:17:38.207304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:31:10.365720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"42'1","reported_seq":13,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T03:17:39.223607+0000","last_change":"2024-09-12T03:17:38.227887+0000","last_active":"2024-09-12T03:17:39.223607+0000","last_peered":"2024-09-12T03:17:39.223607+0000","last_clean":"2024-09-12T03:17:39.223607+0000","last_became_active":"2024-09-12T03:17:38.227609+0000","last_became_peered":"2024-09-12T03:17:38.227609+0000","last_unstale":"2024-09-12T03:17:39.223607+0000","last_undegraded":"2024-09-12T03:17:39.223607+0000","last_fullsized":"2024-09-12T03:17:39.223607+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:37.205254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:37.205254+0000","last_clean_scrub_stamp":"2024-09-12T03:17:37.205254+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:37:50.091928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T03:17:39.189160+0000","last_change":"2024-09-12T03:17:39.188518+0000","last_active":"2024-09-12T03:17:39.189160+0000","last_peered":"2024-09-12T03:17:39.189160+0000","last_clean":"2024-09-12T03:17:39.189160+0000","last_became_active":"2024-09-12T03:17:39.188158+0000","last_became_peered":"2024-09-12T03:17:39.188158+0000","last_unstale":"2024-09-12T03:17:39.189160+0000","last_undegraded":"2024-09-12T03:17:39.189160+0000","last_fullsized":"2024-09-12T03:17:39.189160+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_clean_scrub_stamp":"2024-09-12T03:17:38.207304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:40:17.397085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"42'2","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T03:17:39.222127+0000","last_change":"2024-09-12T03:17:38.231103+0000","last_active":"2024-09-12T03:17:39.222127+0000","last_peered":"2024-09-12T03:17:39.222127+0000","last_clean":"2024-09-12T03:17:39.222127+0000","last_became_active":"2024-09-12T03:17:38.230817+0000","last_became_peered":"2024-09-12T03:17:38.230817+0000","last_unstale":"2024-09-12T03:17:39.222127+0000","last_undegraded":"2024-09-12T03:17:39.222127+0000","last_fullsized":"2024-09-12T03:17:39.222127+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:37.205254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:37.205254+0000","last_clean_scrub_stamp":"2024-09-12T03:17:37.205254+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:41:02.541069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T03:17:39.192633+0000","last_change":"2024-09-12T03:17:39.190319+0000","last_active":"2024-09-12T03:17:39.192633+0000","last_peered":"2024-09-12T03:17:39.192633+0000","last_clean":"2024-09-12T03:17:39.192633+0000","last_became_active":"2024-09-12T03:17:39.189993+0000","last_became_peered":"2024-09-12T03:17:39.189993+0000","last_unstale":"2024-09-12T03:17:39.192633+0000","last_undegraded":"2024-09-12T03:17:39.192633+0000","last_fullsized":"2024-09-12T03:17:39.192633+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_clean_scrub_stamp":"2024-09-12T03:17:38.207304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:44:43.661143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"42'2","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T03:17:39.222537+0000","last_change":"2024-09-12T03:17:38.230293+0000","last_active":"2024-09-12T03:17:39.222537+0000","last_peered":"2024-09-12T03:17:39.222537+0000","last_clean":"2024-09-12T03:17:39.222537+0000","last_became_active":"2024-09-12T03:17:38.230030+0000","last_became_peered":"2024-09-12T03:17:38.230030+0000","last_unstale":"2024-09-12T03:17:39.222537+0000","last_undegraded":"2024-09-12T03:17:39.222537+0000","last_fullsized":"2024-09-12T03:17:39.222537+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:37.205254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:37.205254+0000","last_clean_scrub_stamp":"2024-09-12T03:17:37.205254+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:55:42.981630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T03:17:39.178483+0000","last_change":"2024-09-12T03:17:39.177269+0000","last_active":"2024-09-12T03:17:39.178483+0000","last_peered":"2024-09-12T03:17:39.178483+0000","last_clean":"2024-09-12T03:17:39.178483+0000","last_became_active":"2024-09-12T03:17:39.176965+0000","last_became_peered":"2024-09-12T03:17:39.176965+0000","last_unstale":"2024-09-12T03:17:39.178483+0000","last_undegraded":"2024-09-12T03:17:39.178483+0000","last_fullsized":"2024-09-12T03:17:39.178483+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_clean_scrub_stamp":"2024-09-12T03:17:38.207304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:19:36.833974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"42'1","reported_seq":13,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T03:17:39.222541+0000","last_change":"2024-09-12T03:17:38.228366+0000","last_active":"2024-09-12T03:17:39.222541+0000","last_peered":"2024-09-12T03:17:39.222541+0000","last_clean":"2024-09-12T03:17:39.222541+0000","last_became_active":"2024-09-12T03:17:38.228071+0000","last_became_peered":"2024-09-12T03:17:38.228071+0000","last_unstale":"2024-09-12T03:17:39.222541+0000","last_undegraded":"2024-09-12T03:17:39.222541+0000","last_fullsized":"2024-09-12T03:17:39.222541+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:37.205254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:37.205254+0000","last_clean_scrub_stamp":"2024-09-12T03:17:37.205254+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:59:19.330713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T03:17:39.188690+0000","last_change":"2024-09-12T03:17:39.185323+0000","last_active":"2024-09-12T03:17:39.188690+0000","last_peered":"2024-09-12T03:17:39.188690+0000","last_clean":"2024-09-12T03:17:39.188690+0000","last_became_active":"2024-09-12T03:17:39.184897+0000","last_became_peered":"2024-09-12T03:17:39.184897+0000","last_unstale":"2024-09-12T03:17:39.188690+0000","last_undegraded":"2024-09-12T03:17:39.188690+0000","last_fullsized":"2024-09-12T03:17:39.188690+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_clean_scrub_stamp":"2024-09-12T03:17:38.207304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:47:18.341852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"42'1","reported_seq":13,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T03:17:39.223762+0000","last_change":"2024-09-12T03:17:38.226857+0000","last_active":"2024-09-12T03:17:39.223762+0000","last_peered":"2024-09-12T03:17:39.223762+0000","last_clean":"2024-09-12T03:17:39.223762+0000","last_became_active":"2024-09-12T03:17:38.226565+0000","last_became_peered":"2024-09-12T03:17:38.226565+0000","last_unstale":"2024-09-12T03:17:39.223762+0000","last_undegraded":"2024-09-12T03:17:39.223762+0000","last_fullsized":"2024-09-12T03:17:39.223762+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:37.205254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:37.205254+0000","last_clean_scrub_stamp":"2024-09-12T03:17:37.205254+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:00:51.473212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T03:17:39.194994+0000","last_change":"2024-09-12T03:17:39.190485+0000","last_active":"2024-09-12T03:17:39.194994+0000","last_peered":"2024-09-12T03:17:39.194994+0000","last_clean":"2024-09-12T03:17:39.194994+0000","last_became_active":"2024-09-12T03:17:39.190198+0000","last_became_peered":"2024-09-12T03:17:39.190198+0000","last_unstale":"2024-09-12T03:17:39.194994+0000","last_undegraded":"2024-09-12T03:17:39.194994+0000","last_fullsized":"2024-09-12T03:17:39.194994+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_clean_scrub_stamp":"2024-09-12T03:17:38.207304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:13:15.002267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T03:17:39.174589+0000","last_change":"2024-09-12T03:17:38.226293+0000","last_active":"2024-09-12T03:17:39.174589+0000","last_peered":"2024-09-12T03:17:39.174589+0000","last_clean":"2024-09-12T03:17:39.174589+0000","last_became_active":"2024-09-12T03:17:38.226014+0000","last_became_peered":"2024-09-12T03:17:38.226014+0000","last_unstale":"2024-09-12T03:17:39.174589+0000","last_undegraded":"2024-09-12T03:17:39.174589+0000","last_fullsized":"2024-09-12T03:17:39.174589+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:37.205254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:37.205254+0000","last_clean_scrub_stamp":"2024-09-12T03:17:37.205254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:56:04.799549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T03:17:39.186358+0000","last_change":"2024-09-12T03:17:39.184689+0000","last_active":"2024-09-12T03:17:39.186358+0000","last_peered":"2024-09-12T03:17:39.186358+0000","last_clean":"2024-09-12T03:17:39.186358+0000","last_became_active":"2024-09-12T03:17:39.184051+0000","last_became_peered":"2024-09-12T03:17:39.184051+0000","last_unstale":"2024-09-12T03:17:39.186358+0000","last_undegraded":"2024-09-12T03:17:39.186358+0000","last_fullsized":"2024-09-12T03:17:39.186358+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_clean_scrub_stamp":"2024-09-12T03:17:38.207304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:45:28.696730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T03:17:39.173120+0000","last_change":"2024-09-12T03:17:38.228438+0000","last_active":"2024-09-12T03:17:39.173120+0000","last_peered":"2024-09-12T03:17:39.173120+0000","last_clean":"2024-09-12T03:17:39.173120+0000","last_became_active":"2024-09-12T03:17:38.228162+0000","last_became_peered":"2024-09-12T03:17:38.228162+0000","last_unstale":"2024-09-12T03:17:39.173120+0000","last_undegraded":"2024-09-12T03:17:39.173120+0000","last_fullsized":"2024-09-12T03:17:39.173120+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:37.205254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:37.205254+0000","last_clean_scrub_stamp":"2024-09-12T03:17:37.205254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:28:16.757444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T03:17:39.186771+0000","last_change":"2024-09-12T03:17:39.185650+0000","last_active":"2024-09-12T03:17:39.186771+0000","last_peered":"2024-09-12T03:17:39.186771+0000","last_clean":"2024-09-12T03:17:39.186771+0000","last_became_active":"2024-09-12T03:17:39.185277+0000","last_became_peered":"2024-09-12T03:17:39.185277+0000","last_unstale":"2024-09-12T03:17:39.186771+0000","last_undegraded":"2024-09-12T03:17:39.186771+0000","last_fullsized":"2024-09-12T03:17:39.186771+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_clean_scrub_stamp":"2024-09-12T03:17:38.207304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:24:56.579074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T03:17:39.169552+0000","last_change":"2024-09-12T03:17:38.223901+0000","last_active":"2024-09-12T03:17:39.169552+0000","last_peered":"2024-09-12T03:17:39.169552+0000","last_clean":"2024-09-12T03:17:39.169552+0000","last_became_active":"2024-09-12T03:17:38.223564+0000","last_became_peered":"2024-09-12T03:17:38.223564+0000","last_unstale":"2024-09-12T03:17:39.169552+0000","last_undegraded":"2024-09-12T03:17:39.169552+0000","last_fullsized":"2024-09-12T03:17:39.169552+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:37.205254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:37.205254+0000","last_clean_scrub_stamp":"2024-09-12T03:17:37.205254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:30:24.511217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T03:17:39.182153+0000","last_change":"2024-09-12T03:17:39.180327+0000","last_active":"2024-09-12T03:17:39.182153+0000","last_peered":"2024-09-12T03:17:39.182153+0000","last_clean":"2024-09-12T03:17:39.182153+0000","last_became_active":"2024-09-12T03:17:39.179952+0000","last_became_peered":"2024-09-12T03:17:39.179952+0000","last_unstale":"2024-09-12T03:17:39.182153+0000","last_undegraded":"2024-09-12T03:17:39.182153+0000","last_fullsized":"2024-09-12T03:17:39.182153+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_clean_scrub_stamp":"2024-09-12T03:17:38.207304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:20:59.682175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T03:17:39.176725+0000","last_change":"2024-09-12T03:17:38.230663+0000","last_active":"2024-09-12T03:17:39.176725+0000","last_peered":"2024-09-12T03:17:39.176725+0000","last_clean":"2024-09-12T03:17:39.176725+0000","last_became_active":"2024-09-12T03:17:38.230367+0000","last_became_peered":"2024-09-12T03:17:38.230367+0000","last_unstale":"2024-09-12T03:17:39.176725+0000","last_undegraded":"2024-09-12T03:17:39.176725+0000","last_fullsized":"2024-09-12T03:17:39.176725+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:37.205254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:37.205254+0000","last_clean_scrub_stamp":"2024-09-12T03:17:37.205254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:25:35.607418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T03:17:39.187589+0000","last_change":"2024-09-12T03:17:39.181753+0000","last_active":"2024-09-12T03:17:39.187589+0000","last_peered":"2024-09-12T03:17:39.187589+0000","last_clean":"2024-09-12T03:17:39.187589+0000","last_became_active":"2024-09-12T03:17:39.181440+0000","last_became_peered":"2024-09-12T03:17:39.181440+0000","last_unstale":"2024-09-12T03:17:39.187589+0000","last_undegraded":"2024-09-12T03:17:39.187589+0000","last_fullsized":"2024-09-12T03:17:39.187589+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_clean_scrub_stamp":"2024-09-12T03:17:38.207304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:20:08.109520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"42'3","reported_seq":15,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T03:17:39.223335+0000","last_change":"2024-09-12T03:17:38.228597+0000","last_active":"2024-09-12T03:17:39.223335+0000","last_peered":"2024-09-12T03:17:39.223335+0000","last_clean":"2024-09-12T03:17:39.223335+0000","last_became_active":"2024-09-12T03:17:38.227932+0000","last_became_peered":"2024-09-12T03:17:38.227932+0000","last_unstale":"2024-09-12T03:17:39.223335+0000","last_undegraded":"2024-09-12T03:17:39.223335+0000","last_fullsized":"2024-09-12T03:17:39.223335+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:37.205254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:37.205254+0000","last_clean_scrub_stamp":"2024-09-12T03:17:37.205254+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:59:19.330713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T03:17:39.188339+0000","last_change":"2024-09-12T03:17:39.187592+0000","last_active":"2024-09-12T03:17:39.188339+0000","last_peered":"2024-09-12T03:17:39.188339+0000","last_clean":"2024-09-12T03:17:39.188339+0000","last_became_active":"2024-09-12T03:17:39.187015+0000","last_became_peered":"2024-09-12T03:17:39.187015+0000","last_unstale":"2024-09-12T03:17:39.188339+0000","last_undegraded":"2024-09-12T03:17:39.188339+0000","last_fullsized":"2024-09-12T03:17:39.188339+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_clean_scrub_stamp":"2024-09-12T03:17:38.207304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:48:50.171581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"42'6","reported_seq":18,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T03:17:39.228571+0000","last_change":"2024-09-12T03:17:38.224132+0000","last_active":"2024-09-12T03:17:39.228571+0000","last_peered":"2024-09-12T03:17:39.228571+0000","last_clean":"2024-09-12T03:17:39.228571+0000","last_became_active":"2024-09-12T03:17:38.223865+0000","last_became_peered":"2024-09-12T03:17:38.223865+0000","last_unstale":"2024-09-12T03:17:39.228571+0000","last_undegraded":"2024-09-12T03:17:39.228571+0000","last_fullsized":"2024-09-12T03:17:39.228571+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:37.205254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:37.205254+0000","last_clean_scrub_stamp":"2024-09-12T03:17:37.205254+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:48:19.664006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T03:17:39.190125+0000","last_change":"2024-09-12T03:17:39.189275+0000","last_active":"2024-09-12T03:17:39.190125+0000","last_peered":"2024-09-12T03:17:39.190125+0000","last_clean":"2024-09-12T03:17:39.190125+0000","last_became_active":"2024-09-12T03:17:39.188902+0000","last_became_peered":"2024-09-12T03:17:39.188902+0000","last_unstale":"2024-09-12T03:17:39.190125+0000","last_undegraded":"2024-09-12T03:17:39.190125+0000","last_fullsized":"2024-09-12T03:17:39.190125+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_clean_scrub_stamp":"2024-09-12T03:17:38.207304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:16:03.541630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"42'2","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T03:17:39.228701+0000","last_change":"2024-09-12T03:17:38.227756+0000","last_active":"2024-09-12T03:17:39.228701+0000","last_peered":"2024-09-12T03:17:39.228701+0000","last_clean":"2024-09-12T03:17:39.228701+0000","last_became_active":"2024-09-12T03:17:38.227479+0000","last_became_peered":"2024-09-12T03:17:38.227479+0000","last_unstale":"2024-09-12T03:17:39.228701+0000","last_undegraded":"2024-09-12T03:17:39.228701+0000","last_fullsized":"2024-09-12T03:17:39.228701+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:37.205254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:37.205254+0000","last_clean_scrub_stamp":"2024-09-12T03:17:37.205254+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:42:17.989868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T03:17:39.192402+0000","last_change":"2024-09-12T03:17:39.187643+0000","last_active":"2024-09-12T03:17:39.192402+0000","last_peered":"2024-09-12T03:17:39.192402+0000","last_clean":"2024-09-12T03:17:39.192402+0000","last_became_active":"2024-09-12T03:17:39.187358+0000","last_became_peered":"2024-09-12T03:17:39.187358+0000","last_unstale":"2024-09-12T03:17:39.192402+0000","last_undegraded":"2024-09-12T03:17:39.192402+0000","last_fullsized":"2024-09-12T03:17:39.192402+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_clean_scrub_stamp":"2024-09-12T03:17:38.207304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:54:02.526889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T03:17:39.166986+0000","last_change":"2024-09-12T03:17:38.228351+0000","last_active":"2024-09-12T03:17:39.166986+0000","last_peered":"2024-09-12T03:17:39.166986+0000","last_clean":"2024-09-12T03:17:39.166986+0000","last_became_active":"2024-09-12T03:17:38.228013+0000","last_became_peered":"2024-09-12T03:17:38.228013+0000","last_unstale":"2024-09-12T03:17:39.166986+0000","last_undegraded":"2024-09-12T03:17:39.166986+0000","last_fullsized":"2024-09-12T03:17:39.166986+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:37.205254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:37.205254+0000","last_clean_scrub_stamp":"2024-09-12T03:17:37.205254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:28:56.724675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T03:17:39.187754+0000","last_change":"2024-09-12T03:17:39.181912+0000","last_active":"2024-09-12T03:17:39.187754+0000","last_peered":"2024-09-12T03:17:39.187754+0000","last_clean":"2024-09-12T03:17:39.187754+0000","last_became_active":"2024-09-12T03:17:39.181341+0000","last_became_peered":"2024-09-12T03:17:39.181341+0000","last_unstale":"2024-09-12T03:17:39.187754+0000","last_undegraded":"2024-09-12T03:17:39.187754+0000","last_fullsized":"2024-09-12T03:17:39.187754+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_clean_scrub_stamp":"2024-09-12T03:17:38.207304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:09:03.058255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T03:17:39.169262+0000","last_change":"2024-09-12T03:17:38.224498+0000","last_active":"2024-09-12T03:17:39.169262+0000","last_peered":"2024-09-12T03:17:39.169262+0000","last_clean":"2024-09-12T03:17:39.169262+0000","last_became_active":"2024-09-12T03:17:38.224211+0000","last_became_peered":"2024-09-12T03:17:38.224211+0000","last_unstale":"2024-09-12T03:17:39.169262+0000","last_undegraded":"2024-09-12T03:17:39.169262+0000","last_fullsized":"2024-09-12T03:17:39.169262+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:37.205254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:37.205254+0000","last_clean_scrub_stamp":"2024-09-12T03:17:37.205254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:47:13.405144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T03:17:39.182922+0000","last_change":"2024-09-12T03:17:39.181071+0000","last_active":"2024-09-12T03:17:39.182922+0000","last_peered":"2024-09-12T03:17:39.182922+0000","last_clean":"2024-09-12T03:17:39.182922+0000","last_became_active":"2024-09-12T03:17:39.180760+0000","last_became_peered":"2024-09-12T03:17:39.180760+0000","last_unstale":"2024-09-12T03:17:39.182922+0000","last_undegraded":"2024-09-12T03:17:39.182922+0000","last_fullsized":"2024-09-12T03:17:39.182922+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_clean_scrub_stamp":"2024-09-12T03:17:38.207304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:37:43.236716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"42'8","reported_seq":20,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T03:17:39.228770+0000","last_change":"2024-09-12T03:17:38.229628+0000","last_active":"2024-09-12T03:17:39.228770+0000","last_peered":"2024-09-12T03:17:39.228770+0000","last_clean":"2024-09-12T03:17:39.228770+0000","last_became_active":"2024-09-12T03:17:38.229289+0000","last_became_peered":"2024-09-12T03:17:38.229289+0000","last_unstale":"2024-09-12T03:17:39.228770+0000","last_undegraded":"2024-09-12T03:17:39.228770+0000","last_fullsized":"2024-09-12T03:17:39.228770+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:37.205254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:37.205254+0000","last_clean_scrub_stamp":"2024-09-12T03:17:37.205254+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:19:35.831924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T03:17:39.183827+0000","last_change":"2024-09-12T03:17:39.182414+0000","last_active":"2024-09-12T03:17:39.183827+0000","last_peered":"2024-09-12T03:17:39.183827+0000","last_clean":"2024-09-12T03:17:39.183827+0000","last_became_active":"2024-09-12T03:17:39.182130+0000","last_became_peered":"2024-09-12T03:17:39.182130+0000","last_unstale":"2024-09-12T03:17:39.183827+0000","last_undegraded":"2024-09-12T03:17:39.183827+0000","last_fullsized":"2024-09-12T03:17:39.183827+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_clean_scrub_stamp":"2024-09-12T03:17:38.207304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:17:58.861067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"42'1","reported_seq":13,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T03:17:39.221768+0000","last_change":"2024-09-12T03:17:38.226193+0000","last_active":"2024-09-12T03:17:39.221768+0000","last_peered":"2024-09-12T03:17:39.221768+0000","last_clean":"2024-09-12T03:17:39.221768+0000","last_became_active":"2024-09-12T03:17:38.225886+0000","last_became_peered":"2024-09-12T03:17:38.225886+0000","last_unstale":"2024-09-12T03:17:39.221768+0000","last_undegraded":"2024-09-12T03:17:39.221768+0000","last_fullsized":"2024-09-12T03:17:39.221768+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:37.205254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:37.205254+0000","last_clean_scrub_stamp":"2024-09-12T03:17:37.205254+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:39:58.968031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T03:17:39.182397+0000","last_change":"2024-09-12T03:17:39.179806+0000","last_active":"2024-09-12T03:17:39.182397+0000","last_peered":"2024-09-12T03:17:39.182397+0000","last_clean":"2024-09-12T03:17:39.182397+0000","last_became_active":"2024-09-12T03:17:39.178917+0000","last_became_peered":"2024-09-12T03:17:39.178917+0000","last_unstale":"2024-09-12T03:17:39.182397+0000","last_undegraded":"2024-09-12T03:17:39.182397+0000","last_fullsized":"2024-09-12T03:17:39.182397+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_clean_scrub_stamp":"2024-09-12T03:17:38.207304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:22:59.414373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"42'2","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T03:17:39.226101+0000","last_change":"2024-09-12T03:17:38.228137+0000","last_active":"2024-09-12T03:17:39.226101+0000","last_peered":"2024-09-12T03:17:39.226101+0000","last_clean":"2024-09-12T03:17:39.226101+0000","last_became_active":"2024-09-12T03:17:38.227234+0000","last_became_peered":"2024-09-12T03:17:38.227234+0000","last_unstale":"2024-09-12T03:17:39.226101+0000","last_undegraded":"2024-09-12T03:17:39.226101+0000","last_fullsized":"2024-09-12T03:17:39.226101+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:37.205254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:37.205254+0000","last_clean_scrub_stamp":"2024-09-12T03:17:37.205254+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:16:02.539580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T03:17:39.189281+0000","last_change":"2024-09-12T03:17:39.188494+0000","last_active":"2024-09-12T03:17:39.189281+0000","last_peered":"2024-09-12T03:17:39.189281+0000","last_clean":"2024-09-12T03:17:39.189281+0000","last_became_active":"2024-09-12T03:17:39.188174+0000","last_became_peered":"2024-09-12T03:17:39.188174+0000","last_unstale":"2024-09-12T03:17:39.189281+0000","last_undegraded":"2024-09-12T03:17:39.189281+0000","last_fullsized":"2024-09-12T03:17:39.189281+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_clean_scrub_stamp":"2024-09-12T03:17:38.207304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:00:03.080107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"42'1","reported_seq":13,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T03:17:39.223987+0000","last_change":"2024-09-12T03:17:38.228625+0000","last_active":"2024-09-12T03:17:39.223987+0000","last_peered":"2024-09-12T03:17:39.223987+0000","last_clean":"2024-09-12T03:17:39.223987+0000","last_became_active":"2024-09-12T03:17:38.228353+0000","last_became_peered":"2024-09-12T03:17:38.228353+0000","last_unstale":"2024-09-12T03:17:39.223987+0000","last_undegraded":"2024-09-12T03:17:39.223987+0000","last_fullsized":"2024-09-12T03:17:39.223987+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:37.205254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:37.205254+0000","last_clean_scrub_stamp":"2024-09-12T03:17:37.205254+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:54:28.297593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T03:17:39.190801+0000","last_change":"2024-09-12T03:17:39.189954+0000","last_active":"2024-09-12T03:17:39.190801+0000","last_peered":"2024-09-12T03:17:39.190801+0000","last_clean":"2024-09-12T03:17:39.190801+0000","last_became_active":"2024-09-12T03:17:39.189642+0000","last_became_peered":"2024-09-12T03:17:39.189642+0000","last_unstale":"2024-09-12T03:17:39.190801+0000","last_undegraded":"2024-09-12T03:17:39.190801+0000","last_fullsized":"2024-09-12T03:17:39.190801+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_clean_scrub_stamp":"2024-09-12T03:17:38.207304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:28:17.759494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T03:17:39.170373+0000","last_change":"2024-09-12T03:17:38.225655+0000","last_active":"2024-09-12T03:17:39.170373+0000","last_peered":"2024-09-12T03:17:39.170373+0000","last_clean":"2024-09-12T03:17:39.170373+0000","last_became_active":"2024-09-12T03:17:38.225350+0000","last_became_peered":"2024-09-12T03:17:38.225350+0000","last_unstale":"2024-09-12T03:17:39.170373+0000","last_undegraded":"2024-09-12T03:17:39.170373+0000","last_fullsized":"2024-09-12T03:17:39.170373+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:37.205254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:37.205254+0000","last_clean_scrub_stamp":"2024-09-12T03:17:37.205254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:53:00.959255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T03:17:39.187514+0000","last_change":"2024-09-12T03:17:39.180771+0000","last_active":"2024-09-12T03:17:39.187514+0000","last_peered":"2024-09-12T03:17:39.187514+0000","last_clean":"2024-09-12T03:17:39.187514+0000","last_became_active":"2024-09-12T03:17:39.180471+0000","last_became_peered":"2024-09-12T03:17:39.180471+0000","last_unstale":"2024-09-12T03:17:39.187514+0000","last_undegraded":"2024-09-12T03:17:39.187514+0000","last_fullsized":"2024-09-12T03:17:39.187514+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_clean_scrub_stamp":"2024-09-12T03:17:38.207304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:49:51.086834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"42'2","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T03:17:39.222876+0000","last_change":"2024-09-12T03:17:38.227044+0000","last_active":"2024-09-12T03:17:39.222876+0000","last_peered":"2024-09-12T03:17:39.222876+0000","last_clean":"2024-09-12T03:17:39.222876+0000","last_became_active":"2024-09-12T03:17:38.226612+0000","last_became_peered":"2024-09-12T03:17:38.226612+0000","last_unstale":"2024-09-12T03:17:39.222876+0000","last_undegraded":"2024-09-12T03:17:39.222876+0000","last_fullsized":"2024-09-12T03:17:39.222876+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:37.205254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:37.205254+0000","last_clean_scrub_stamp":"2024-09-12T03:17:37.205254+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:00:45.563560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T03:17:39.187769+0000","last_change":"2024-09-12T03:17:39.182674+0000","last_active":"2024-09-12T03:17:39.187769+0000","last_peered":"2024-09-12T03:17:39.187769+0000","last_clean":"2024-09-12T03:17:39.187769+0000","last_became_active":"2024-09-12T03:17:39.180953+0000","last_became_peered":"2024-09-12T03:17:39.180953+0000","last_unstale":"2024-09-12T03:17:39.187769+0000","last_undegraded":"2024-09-12T03:17:39.187769+0000","last_fullsized":"2024-09-12T03:17:39.187769+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_clean_scrub_stamp":"2024-09-12T03:17:38.207304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:07:34.120803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T03:17:39.174268+0000","last_change":"2024-09-12T03:17:38.225458+0000","last_active":"2024-09-12T03:17:39.174268+0000","last_peered":"2024-09-12T03:17:39.174268+0000","last_clean":"2024-09-12T03:17:39.174268+0000","last_became_active":"2024-09-12T03:17:38.225157+0000","last_became_peered":"2024-09-12T03:17:38.225157+0000","last_unstale":"2024-09-12T03:17:39.174268+0000","last_undegraded":"2024-09-12T03:17:39.174268+0000","last_fullsized":"2024-09-12T03:17:39.174268+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:37.205254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:37.205254+0000","last_clean_scrub_stamp":"2024-09-12T03:17:37.205254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:52:38.099540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T03:17:39.186530+0000","last_change":"2024-09-12T03:17:39.184851+0000","last_active":"2024-09-12T03:17:39.186530+0000","last_peered":"2024-09-12T03:17:39.186530+0000","last_clean":"2024-09-12T03:17:39.186530+0000","last_became_active":"2024-09-12T03:17:39.184542+0000","last_became_peered":"2024-09-12T03:17:39.184542+0000","last_unstale":"2024-09-12T03:17:39.186530+0000","last_undegraded":"2024-09-12T03:17:39.186530+0000","last_fullsized":"2024-09-12T03:17:39.186530+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_clean_scrub_stamp":"2024-09-12T03:17:38.207304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:56:31.329671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T03:17:39.174137+0000","last_change":"2024-09-12T03:17:38.231346+0000","last_active":"2024-09-12T03:17:39.174137+0000","last_peered":"2024-09-12T03:17:39.174137+0000","last_clean":"2024-09-12T03:17:39.174137+0000","last_became_active":"2024-09-12T03:17:38.231069+0000","last_became_peered":"2024-09-12T03:17:38.231069+0000","last_unstale":"2024-09-12T03:17:39.174137+0000","last_undegraded":"2024-09-12T03:17:39.174137+0000","last_fullsized":"2024-09-12T03:17:39.174137+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:37.205254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:37.205254+0000","last_clean_scrub_stamp":"2024-09-12T03:17:37.205254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:51:17.034282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T03:17:39.192349+0000","last_change":"2024-09-12T03:17:39.186987+0000","last_active":"2024-09-12T03:17:39.192349+0000","last_peered":"2024-09-12T03:17:39.192349+0000","last_clean":"2024-09-12T03:17:39.192349+0000","last_became_active":"2024-09-12T03:17:39.186693+0000","last_became_peered":"2024-09-12T03:17:39.186693+0000","last_unstale":"2024-09-12T03:17:39.192349+0000","last_undegraded":"2024-09-12T03:17:39.192349+0000","last_fullsized":"2024-09-12T03:17:39.192349+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_clean_scrub_stamp":"2024-09-12T03:17:38.207304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:06:30.838342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T03:17:39.166404+0000","last_change":"2024-09-12T03:17:38.229729+0000","last_active":"2024-09-12T03:17:39.166404+0000","last_peered":"2024-09-12T03:17:39.166404+0000","last_clean":"2024-09-12T03:17:39.166404+0000","last_became_active":"2024-09-12T03:17:38.229282+0000","last_became_peered":"2024-09-12T03:17:38.229282+0000","last_unstale":"2024-09-12T03:17:39.166404+0000","last_undegraded":"2024-09-12T03:17:39.166404+0000","last_fullsized":"2024-09-12T03:17:39.166404+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:37.205254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:37.205254+0000","last_clean_scrub_stamp":"2024-09-12T03:17:37.205254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:13:14.000217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T03:17:39.190124+0000","last_change":"2024-09-12T03:17:39.189150+0000","last_active":"2024-09-12T03:17:39.190124+0000","last_peered":"2024-09-12T03:17:39.190124+0000","last_clean":"2024-09-12T03:17:39.190124+0000","last_became_active":"2024-09-12T03:17:39.188853+0000","last_became_peered":"2024-09-12T03:17:39.188853+0000","last_unstale":"2024-09-12T03:17:39.190124+0000","last_undegraded":"2024-09-12T03:17:39.190124+0000","last_fullsized":"2024-09-12T03:17:39.190124+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_clean_scrub_stamp":"2024-09-12T03:17:38.207304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:37:51.093978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T03:17:39.168708+0000","last_change":"2024-09-12T03:17:38.225837+0000","last_active":"2024-09-12T03:17:39.168708+0000","last_peered":"2024-09-12T03:17:39.168708+0000","last_clean":"2024-09-12T03:17:39.168708+0000","last_became_active":"2024-09-12T03:17:38.225094+0000","last_became_peered":"2024-09-12T03:17:38.225094+0000","last_unstale":"2024-09-12T03:17:39.168708+0000","last_undegraded":"2024-09-12T03:17:39.168708+0000","last_fullsized":"2024-09-12T03:17:39.168708+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:37.205254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:37.205254+0000","last_clean_scrub_stamp":"2024-09-12T03:17:37.205254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:28:56.724675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T03:17:39.186432+0000","last_change":"2024-09-12T03:17:39.180265+0000","last_active":"2024-09-12T03:17:39.186432+0000","last_peered":"2024-09-12T03:17:39.186432+0000","last_clean":"2024-09-12T03:17:39.186432+0000","last_became_active":"2024-09-12T03:17:39.178568+0000","last_became_peered":"2024-09-12T03:17:39.178568+0000","last_unstale":"2024-09-12T03:17:39.186432+0000","last_undegraded":"2024-09-12T03:17:39.186432+0000","last_fullsized":"2024-09-12T03:17:39.186432+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_clean_scrub_stamp":"2024-09-12T03:17:38.207304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:24:56.579074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T03:17:39.168255+0000","last_change":"2024-09-12T03:17:38.228816+0000","last_active":"2024-09-12T03:17:39.168255+0000","last_peered":"2024-09-12T03:17:39.168255+0000","last_clean":"2024-09-12T03:17:39.168255+0000","last_became_active":"2024-09-12T03:17:38.228317+0000","last_became_peered":"2024-09-12T03:17:38.228317+0000","last_unstale":"2024-09-12T03:17:39.168255+0000","last_undegraded":"2024-09-12T03:17:39.168255+0000","last_fullsized":"2024-09-12T03:17:39.168255+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:37.205254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:37.205254+0000","last_clean_scrub_stamp":"2024-09-12T03:17:37.205254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:42:17.989868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T03:17:39.182259+0000","last_change":"2024-09-12T03:17:39.180919+0000","last_active":"2024-09-12T03:17:39.182259+0000","last_peered":"2024-09-12T03:17:39.182259+0000","last_clean":"2024-09-12T03:17:39.182259+0000","last_became_active":"2024-09-12T03:17:39.180625+0000","last_became_peered":"2024-09-12T03:17:39.180625+0000","last_unstale":"2024-09-12T03:17:39.182259+0000","last_undegraded":"2024-09-12T03:17:39.182259+0000","last_fullsized":"2024-09-12T03:17:39.182259+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_clean_scrub_stamp":"2024-09-12T03:17:38.207304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:06:36.818875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T03:17:39.176524+0000","last_change":"2024-09-12T03:17:38.230355+0000","last_active":"2024-09-12T03:17:39.176524+0000","last_peered":"2024-09-12T03:17:39.176524+0000","last_clean":"2024-09-12T03:17:39.176524+0000","last_became_active":"2024-09-12T03:17:38.230095+0000","last_became_peered":"2024-09-12T03:17:38.230095+0000","last_unstale":"2024-09-12T03:17:39.176524+0000","last_undegraded":"2024-09-12T03:17:39.176524+0000","last_fullsized":"2024-09-12T03:17:39.176524+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:37.205254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:37.205254+0000","last_clean_scrub_stamp":"2024-09-12T03:17:37.205254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:27:58.639003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T03:17:39.187682+0000","last_change":"2024-09-12T03:17:39.184187+0000","last_active":"2024-09-12T03:17:39.187682+0000","last_peered":"2024-09-12T03:17:39.187682+0000","last_clean":"2024-09-12T03:17:39.187682+0000","last_became_active":"2024-09-12T03:17:39.183923+0000","last_became_peered":"2024-09-12T03:17:39.183923+0000","last_unstale":"2024-09-12T03:17:39.187682+0000","last_undegraded":"2024-09-12T03:17:39.187682+0000","last_fullsized":"2024-09-12T03:17:39.187682+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_clean_scrub_stamp":"2024-09-12T03:17:38.207304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:32:55.415484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T03:17:39.167568+0000","last_change":"2024-09-12T03:17:38.228343+0000","last_active":"2024-09-12T03:17:39.167568+0000","last_peered":"2024-09-12T03:17:39.167568+0000","last_clean":"2024-09-12T03:17:39.167568+0000","last_became_active":"2024-09-12T03:17:38.228060+0000","last_became_peered":"2024-09-12T03:17:38.228060+0000","last_unstale":"2024-09-12T03:17:39.167568+0000","last_undegraded":"2024-09-12T03:17:39.167568+0000","last_fullsized":"2024-09-12T03:17:39.167568+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:37.205254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:37.205254+0000","last_clean_scrub_stamp":"2024-09-12T03:17:37.205254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:47:13.405144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T03:17:39.187245+0000","last_change":"2024-09-12T03:17:39.179998+0000","last_active":"2024-09-12T03:17:39.187245+0000","last_peered":"2024-09-12T03:17:39.187245+0000","last_clean":"2024-09-12T03:17:39.187245+0000","last_became_active":"2024-09-12T03:17:39.179175+0000","last_became_peered":"2024-09-12T03:17:39.179175+0000","last_unstale":"2024-09-12T03:17:39.187245+0000","last_undegraded":"2024-09-12T03:17:39.187245+0000","last_fullsized":"2024-09-12T03:17:39.187245+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_clean_scrub_stamp":"2024-09-12T03:17:38.207304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:48:50.171581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T03:17:39.167332+0000","last_change":"2024-09-12T03:17:38.223991+0000","last_active":"2024-09-12T03:17:39.167332+0000","last_peered":"2024-09-12T03:17:39.167332+0000","last_clean":"2024-09-12T03:17:39.167332+0000","last_became_active":"2024-09-12T03:17:38.223304+0000","last_became_peered":"2024-09-12T03:17:38.223304+0000","last_unstale":"2024-09-12T03:17:39.167332+0000","last_undegraded":"2024-09-12T03:17:39.167332+0000","last_fullsized":"2024-09-12T03:17:39.167332+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:37.205254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:37.205254+0000","last_clean_scrub_stamp":"2024-09-12T03:17:37.205254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:00:02.078057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T03:17:39.180867+0000","last_change":"2024-09-12T03:17:39.179669+0000","last_active":"2024-09-12T03:17:39.180867+0000","last_peered":"2024-09-12T03:17:39.180867+0000","last_clean":"2024-09-12T03:17:39.180867+0000","last_became_active":"2024-09-12T03:17:39.179360+0000","last_became_peered":"2024-09-12T03:17:39.179360+0000","last_unstale":"2024-09-12T03:17:39.180867+0000","last_undegraded":"2024-09-12T03:17:39.180867+0000","last_fullsized":"2024-09-12T03:17:39.180867+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_clean_scrub_stamp":"2024-09-12T03:17:38.207304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:50:38.156704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T03:17:39.175689+0000","last_change":"2024-09-12T03:17:38.228789+0000","last_active":"2024-09-12T03:17:39.175689+0000","last_peered":"2024-09-12T03:17:39.175689+0000","last_clean":"2024-09-12T03:17:39.175689+0000","last_became_active":"2024-09-12T03:17:38.228402+0000","last_became_peered":"2024-09-12T03:17:38.228402+0000","last_unstale":"2024-09-12T03:17:39.175689+0000","last_undegraded":"2024-09-12T03:17:39.175689+0000","last_fullsized":"2024-09-12T03:17:39.175689+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:37.205254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:37.205254+0000","last_clean_scrub_stamp":"2024-09-12T03:17:37.205254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:58:00.543783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T03:17:39.183628+0000","last_change":"2024-09-12T03:17:39.180384+0000","last_active":"2024-09-12T03:17:39.183628+0000","last_peered":"2024-09-12T03:17:39.183628+0000","last_clean":"2024-09-12T03:17:39.183628+0000","last_became_active":"2024-09-12T03:17:39.179269+0000","last_became_peered":"2024-09-12T03:17:39.179269+0000","last_unstale":"2024-09-12T03:17:39.183628+0000","last_undegraded":"2024-09-12T03:17:39.183628+0000","last_fullsized":"2024-09-12T03:17:39.183628+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_clean_scrub_stamp":"2024-09-12T03:17:38.207304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:25:36.609468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T03:17:39.173475+0000","last_change":"2024-09-12T03:17:38.232402+0000","last_active":"2024-09-12T03:17:39.173475+0000","last_peered":"2024-09-12T03:17:39.173475+0000","last_clean":"2024-09-12T03:17:39.173475+0000","last_became_active":"2024-09-12T03:17:38.232034+0000","last_became_peered":"2024-09-12T03:17:38.232034+0000","last_unstale":"2024-09-12T03:17:39.173475+0000","last_undegraded":"2024-09-12T03:17:39.173475+0000","last_fullsized":"2024-09-12T03:17:39.173475+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:37.205254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:37.205254+0000","last_clean_scrub_stamp":"2024-09-12T03:17:37.205254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:28:32.785457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T03:17:39.187856+0000","last_change":"2024-09-12T03:17:39.183225+0000","last_active":"2024-09-12T03:17:39.187856+0000","last_peered":"2024-09-12T03:17:39.187856+0000","last_clean":"2024-09-12T03:17:39.187856+0000","last_became_active":"2024-09-12T03:17:39.182734+0000","last_became_peered":"2024-09-12T03:17:39.182734+0000","last_unstale":"2024-09-12T03:17:39.187856+0000","last_undegraded":"2024-09-12T03:17:39.187856+0000","last_fullsized":"2024-09-12T03:17:39.187856+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_clean_scrub_stamp":"2024-09-12T03:17:38.207304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:42:37.751637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"42'2","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T03:17:39.223043+0000","last_change":"2024-09-12T03:17:38.231155+0000","last_active":"2024-09-12T03:17:39.223043+0000","last_peered":"2024-09-12T03:17:39.223043+0000","last_clean":"2024-09-12T03:17:39.223043+0000","last_became_active":"2024-09-12T03:17:38.230869+0000","last_became_peered":"2024-09-12T03:17:38.230869+0000","last_unstale":"2024-09-12T03:17:39.223043+0000","last_undegraded":"2024-09-12T03:17:39.223043+0000","last_fullsized":"2024-09-12T03:17:39.223043+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:37.205254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:37.205254+0000","last_clean_scrub_stamp":"2024-09-12T03:17:37.205254+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:54:31.074525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":206,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T03:17:39.166279+0000","last_change":"2024-09-12T03:16:21.883446+0000","last_active":"2024-09-12T03:17:39.166279+0000","last_peered":"2024-09-12T03:17:39.166279+0000","last_clean":"2024-09-12T03:17:39.166279+0000","last_became_active":"2024-09-12T03:16:21.882207+0000","last_became_peered":"2024-09-12T03:16:21.882207+0000","last_unstale":"2024-09-12T03:17:39.166279+0000","last_undegraded":"2024-09-12T03:17:39.166279+0000","last_fullsized":"2024-09-12T03:17:39.166279+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:20.316426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:20.316426+0000","last_clean_scrub_stamp":"2024-09-12T03:16:20.316426+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:12:15.424320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542162,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32248,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339592,"statfs":{"total":96636764160,"available":96603742208,"internally_reserved":0,"allocated":1085440,"data_stored":784705,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542164,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31672,"kb_used_data":484,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340168,"statfs":{"total":96636764160,"available":96604332032,"internally_reserved":0,"allocated":495616,"data_stored":194919,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542163,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35832,"kb_used_data":480,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336008,"statfs":{"total":96636764160,"available":96600072192,"internally_reserved":0,"allocated":491520,"data_stored":194317,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542162,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31676,"kb_used_data":488,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340164,"statfs":{"total":96636764160,"available":96604327936,"internally_reserved":0,"allocated":499712,"data_stored":195859,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542163,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27516,"kb_used_data":488,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344324,"statfs":{"total":96636764160,"available":96608587776,"internally_reserved":0,"allocated":499712,"data_stored":194885,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542163,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27520,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344320,"statfs":{"total":96636764160,"available":96608583680,"internally_reserved":0,"allocated":503808,"data_stored":195881,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542164,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27508,"kb_used_data":476,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344332,"statfs":{"total":96636764160,"available":96608595968,"internally_reserved":0,"allocated":487424,"data_stored":194247,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542164,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28092,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343748,"statfs":{"total":96636764160,"available":96607997952,"internally_reserved":0,"allocated":1085440,"data_stored":784649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T03:17:42.201 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-12T03:17:42.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:42.339+0000 7fc627f6d640 1 Processor -- start 2024-09-12T03:17:42.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:42.339+0000 7fc627f6d640 1 -- start start 2024-09-12T03:17:42.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:42.340+0000 7fc627f6d640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc6201590d0 0x7fc6200b55d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:42.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:42.340+0000 7fc627f6d640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc620162c10 con 0x7fc620158cf0 2024-09-12T03:17:42.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:42.340+0000 7fc627f6d640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc620162ef0 con 0x7fc62015f600 2024-09-12T03:17:42.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:42.340+0000 7fc627f6d640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fc6201631d0 con 0x7fc6201590d0 2024-09-12T03:17:42.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:42.340+0000 7fc625d09640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fc620158cf0 0x7fc620162430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:40176/0 (socket says 172.21.15.27:40176) 2024-09-12T03:17:42.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:42.340+0000 7fc625d09640 1 -- 172.21.15.27:0/423813354 learned_addr learned my addr 172.21.15.27:0/423813354 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:17:42.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:42.340+0000 7fc625508640 1 --2- 172.21.15.27:0/423813354 >> v2:172.21.15.45:3300/0 conn(0x7fc6201590d0 0x7fc6200b55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:42.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:42.340+0000 7fc624d07640 1 -- 172.21.15.27:0/423813354 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 175012269 0 0) 0x7fc620162c10 con 0x7fc620158cf0 2024-09-12T03:17:42.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:42.340+0000 7fc624d07640 1 -- 172.21.15.27:0/423813354 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc608003520 con 0x7fc620158cf0 2024-09-12T03:17:42.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:42.341+0000 7fc624d07640 1 -- 172.21.15.27:0/423813354 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 463258225 0 0) 0x7fc608003520 con 0x7fc620158cf0 2024-09-12T03:17:42.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:42.341+0000 7fc624d07640 1 -- 172.21.15.27:0/423813354 >> v2:172.21.15.45:3300/0 conn(0x7fc6201590d0 msgr2=0x7fc6200b55d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:42.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:42.341+0000 7fc624d07640 1 --2- 172.21.15.27:0/423813354 >> v2:172.21.15.45:3300/0 conn(0x7fc6201590d0 0x7fc6200b55d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:42.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:42.341+0000 7fc624d07640 1 -- 172.21.15.27:0/423813354 >> v1:172.21.15.45:6789/0 conn(0x7fc62015f600 legacy=0x7fc62006bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:42.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:42.341+0000 7fc624d07640 1 -- 172.21.15.27:0/423813354 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc62015edc0 con 0x7fc620158cf0 2024-09-12T03:17:42.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:42.341+0000 7fc624d07640 1 -- 172.21.15.27:0/423813354 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 847755213 0 0) 0x7fc618001670 con 0x7fc620158cf0 2024-09-12T03:17:42.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:42.341+0000 7fc624d07640 1 -- 172.21.15.27:0/423813354 >> v1:172.21.15.27:6789/0 conn(0x7fc620158cf0 legacy=0x7fc620162430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:42.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:42.341+0000 7fc624d07640 1 --2- 172.21.15.27:0/423813354 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc6080039b0 0x7fc608003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:42.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:42.341+0000 7fc624d07640 1 --2- 172.21.15.27:0/423813354 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc608004800 0x7fc608004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:42.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:42.341+0000 7fc624d07640 1 --2- 172.21.15.27:0/423813354 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fc6080052e0 0x7fc60800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:42.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:42.341+0000 7fc624d07640 1 -- 172.21.15.27:0/423813354 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fc6201631d0 con 0x7fc608004800 2024-09-12T03:17:42.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:42.341+0000 7fc624d07640 1 -- 172.21.15.27:0/423813354 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc60800c1b0 con 0x7fc6080039b0 2024-09-12T03:17:42.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:42.341+0000 7fc624d07640 1 -- 172.21.15.27:0/423813354 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fc60800c3b0 con 0x7fc6080052e0 2024-09-12T03:17:42.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:42.341+0000 7fc625508640 1 --2- 172.21.15.27:0/423813354 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc608004800 0x7fc608004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:42.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:42.341+0000 7fc625d09640 1 --2- 172.21.15.27:0/423813354 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc6080039b0 0x7fc608003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:42.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:42.341+0000 7fc62650a640 1 --2- 172.21.15.27:0/423813354 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fc6080052e0 0x7fc60800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:42.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:42.341+0000 7fc625508640 1 -- 172.21.15.27:0/423813354 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fc6080052e0 msgr2=0x7fc60800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:42.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:42.341+0000 7fc625508640 1 --2- 172.21.15.27:0/423813354 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fc6080052e0 0x7fc60800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:42.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:42.341+0000 7fc625508640 1 -- 172.21.15.27:0/423813354 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc6080039b0 msgr2=0x7fc608003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:42.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:42.341+0000 7fc625508640 1 --2- 172.21.15.27:0/423813354 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc6080039b0 0x7fc608003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:42.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:42.341+0000 7fc625508640 1 -- 172.21.15.27:0/423813354 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc608003520 con 0x7fc608004800 2024-09-12T03:17:42.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:42.341+0000 7fc62650a640 1 -- 172.21.15.27:0/423813354 reap_dead start 2024-09-12T03:17:42.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:42.342+0000 7fc625508640 1 --2- 172.21.15.27:0/423813354 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc608004800 0x7fc608004c80 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fc620063400 tx=0x7fc608005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6e588ad534e1a0b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:42.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:42.342+0000 7fc624d07640 1 -- 172.21.15.27:0/423813354 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc614036070 con 0x7fc608004800 2024-09-12T03:17:42.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:42.342+0000 7fc624d07640 1 -- 172.21.15.27:0/423813354 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc614031400 con 0x7fc608004800 2024-09-12T03:17:42.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:42.342+0000 7fc624d07640 1 -- 172.21.15.27:0/423813354 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc614009690 con 0x7fc608004800 2024-09-12T03:17:42.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:42.343+0000 7fc627f6d640 1 -- 172.21.15.27:0/423813354 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc608004800 msgr2=0x7fc608004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:42.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:42.343+0000 7fc627f6d640 1 --2- 172.21.15.27:0/423813354 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc608004800 0x7fc608004c80 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fc620063400 tx=0x7fc608005830 comp rx=0 tx=0).stop 2024-09-12T03:17:42.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:42.343+0000 7fc627f6d640 1 -- 172.21.15.27:0/423813354 shutdown_connections 2024-09-12T03:17:42.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:42.343+0000 7fc627f6d640 1 --2- 172.21.15.27:0/423813354 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc608004800 0x7fc608004c80 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:42.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:42.343+0000 7fc627f6d640 1 -- 172.21.15.27:0/423813354 >> 172.21.15.27:0/423813354 conn(0x7fc62006cc40 msgr2=0x7fc62006d050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:42.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:42.343+0000 7fc627f6d640 1 -- 172.21.15.27:0/423813354 shutdown_connections 2024-09-12T03:17:42.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:42.343+0000 7fc627f6d640 1 -- 172.21.15.27:0/423813354 wait complete. 2024-09-12T03:17:42.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:42.344+0000 7fc627f6d640 1 Processor -- start 2024-09-12T03:17:42.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:42.344+0000 7fc627f6d640 1 -- start start 2024-09-12T03:17:42.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:42.345+0000 7fc627f6d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc608004800 0x7fc62015ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:42.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:42.345+0000 7fc627f6d640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fc62015b040 0x7fc62015bf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:42.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:42.345+0000 7fc627f6d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc62015c470 0x7fc6201f43a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:42.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:42.345+0000 7fc627f6d640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fc6200b55d0 con 0x7fc608004800 2024-09-12T03:17:42.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:42.345+0000 7fc627f6d640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc62015fc60 con 0x7fc62015c470 2024-09-12T03:17:42.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:42.345+0000 7fc627f6d640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fc6200b5750 con 0x7fc62015b040 2024-09-12T03:17:42.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:42.345+0000 7fc625d09640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc608004800 0x7fc62015ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:42.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:42.345+0000 7fc625d09640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc608004800 0x7fc62015ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60366/0 (socket says 172.21.15.27:60366) 2024-09-12T03:17:42.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:42.345+0000 7fc625d09640 1 -- 172.21.15.27:0/2408834696 learned_addr learned my addr 172.21.15.27:0/2408834696 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:17:42.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:42.345+0000 7fc625508640 1 --2- 172.21.15.27:0/2408834696 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fc62015b040 0x7fc62015bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:42.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:42.345+0000 7fc62650a640 1 --2- 172.21.15.27:0/2408834696 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc62015c470 0x7fc6201f43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:42.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:42.345+0000 7fc625d09640 1 -- 172.21.15.27:0/2408834696 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fc62015b040 msgr2=0x7fc62015bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:42.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:42.345+0000 7fc625d09640 1 --2- 172.21.15.27:0/2408834696 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fc62015b040 0x7fc62015bf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:42.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:42.345+0000 7fc625d09640 1 -- 172.21.15.27:0/2408834696 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc62015c470 msgr2=0x7fc6201f43a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:42.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:42.345+0000 7fc625d09640 1 --2- 172.21.15.27:0/2408834696 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc62015c470 0x7fc6201f43a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:42.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:42.345+0000 7fc625d09640 1 -- 172.21.15.27:0/2408834696 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc620152ee0 con 0x7fc608004800 2024-09-12T03:17:42.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:42.345+0000 7fc62650a640 1 --2- 172.21.15.27:0/2408834696 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc62015c470 0x7fc6201f43a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:17:42.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:42.345+0000 7fc625508640 1 --2- 172.21.15.27:0/2408834696 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fc62015b040 0x7fc62015bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:17:42.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:42.345+0000 7fc625d09640 1 --2- 172.21.15.27:0/2408834696 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc608004800 0x7fc62015ab00 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fc618007290 tx=0x7fc618007750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:42.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:42.345+0000 7fc61e7fc640 1 -- 172.21.15.27:0/2408834696 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc61800f960 con 0x7fc608004800 2024-09-12T03:17:42.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:42.345+0000 7fc61e7fc640 1 -- 172.21.15.27:0/2408834696 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc618018070 con 0x7fc608004800 2024-09-12T03:17:42.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:42.345+0000 7fc61e7fc640 1 -- 172.21.15.27:0/2408834696 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc61800fd60 con 0x7fc608004800 2024-09-12T03:17:42.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:42.345+0000 7fc627f6d640 1 -- 172.21.15.27:0/2408834696 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc620159a60 con 0x7fc608004800 2024-09-12T03:17:42.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:42.345+0000 7fc627f6d640 1 -- 172.21.15.27:0/2408834696 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc62015a020 con 0x7fc608004800 2024-09-12T03:17:42.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:42.346+0000 7fc627f6d640 1 -- 172.21.15.27:0/2408834696 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc620162430 con 0x7fc608004800 2024-09-12T03:17:42.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:42.347+0000 7fc61e7fc640 1 -- 172.21.15.27:0/2408834696 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 7) ==== 90651+0+0 (secure 0 0 0) 0x7fc61801d460 con 0x7fc608004800 2024-09-12T03:17:42.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:42.348+0000 7fc61e7fc640 1 --2- 172.21.15.27:0/2408834696 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fc60c06c010 0x7fc60c06e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:42.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:42.348+0000 7fc61e7fc640 1 -- 172.21.15.27:0/2408834696 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5709+0+0 (secure 0 0 0) 0x7fc61808b1f0 con 0x7fc608004800 2024-09-12T03:17:42.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:42.348+0000 7fc625508640 1 --2- 172.21.15.27:0/2408834696 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fc60c06c010 0x7fc60c06e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:42.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:42.348+0000 7fc625508640 1 --2- 172.21.15.27:0/2408834696 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fc60c06c010 0x7fc60c06e4d0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fc614033000 tx=0x7fc614008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:42.352 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:42.352+0000 7fc61e7fc640 1 -- 172.21.15.27:0/2408834696 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fc620152ee0 con 0x7fc608004800 2024-09-12T03:17:42.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:42.525+0000 7fc627f6d640 1 -- 172.21.15.27:0/2408834696 --> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc6200b4ce0 con 0x7fc60c06c010 2024-09-12T03:17:42.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:42.532+0000 7fc61e7fc640 1 -- 172.21.15.27:0/2408834696 <== mgr.4102 v2:172.21.15.45:6832/3923821219 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251891 (secure 0 0 0) 0x7fc6200b4ce0 con 0x7fc60c06c010 2024-09-12T03:17:42.534 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:17:42.536 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2024-09-12T03:17:42.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:42.535+0000 7fc627f6d640 1 -- 172.21.15.27:0/2408834696 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fc60c06c010 msgr2=0x7fc60c06e4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:42.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:42.535+0000 7fc627f6d640 1 --2- 172.21.15.27:0/2408834696 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fc60c06c010 0x7fc60c06e4d0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fc614033000 tx=0x7fc614008030 comp rx=0 tx=0).stop 2024-09-12T03:17:42.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:42.535+0000 7fc627f6d640 1 -- 172.21.15.27:0/2408834696 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc608004800 msgr2=0x7fc62015ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:42.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:42.535+0000 7fc627f6d640 1 --2- 172.21.15.27:0/2408834696 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc608004800 0x7fc62015ab00 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fc618007290 tx=0x7fc618007750 comp rx=0 tx=0).stop 2024-09-12T03:17:42.538 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:42.535+0000 7fc627f6d640 1 -- 172.21.15.27:0/2408834696 shutdown_connections 2024-09-12T03:17:42.538 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:42.535+0000 7fc627f6d640 1 --2- 172.21.15.27:0/2408834696 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fc60c06c010 0x7fc60c06e4d0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:42.538 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:42.535+0000 7fc627f6d640 1 --2- 172.21.15.27:0/2408834696 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc62015c470 0x7fc6201f43a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:42.538 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:42.535+0000 7fc627f6d640 1 --2- 172.21.15.27:0/2408834696 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fc62015b040 0x7fc62015bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:42.538 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:42.535+0000 7fc627f6d640 1 --2- 172.21.15.27:0/2408834696 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc608004800 0x7fc62015ab00 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:42.538 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:42.535+0000 7fc627f6d640 1 -- 172.21.15.27:0/2408834696 >> 172.21.15.27:0/2408834696 conn(0x7fc62006cc40 msgr2=0x7fc62006f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:42.538 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:42.535+0000 7fc627f6d640 1 -- 172.21.15.27:0/2408834696 shutdown_connections 2024-09-12T03:17:42.538 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:42.535+0000 7fc627f6d640 1 -- 172.21.15.27:0/2408834696 wait complete. 2024-09-12T03:17:42.545 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":77,"stamp":"2024-09-12T03:17:42.066929+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":111,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":242064,"kb_used_data":5028,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754732656,"statfs":{"total":773094113280,"available":772846239744,"internally_reserved":0,"allocated":5148672,"data_stored":2739462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":242535984},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.859211"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T03:17:39.177700+0000","last_change":"2024-09-12T03:17:39.173270+0000","last_active":"2024-09-12T03:17:39.177700+0000","last_peered":"2024-09-12T03:17:39.177700+0000","last_clean":"2024-09-12T03:17:39.177700+0000","last_became_active":"2024-09-12T03:17:39.172969+0000","last_became_peered":"2024-09-12T03:17:39.172969+0000","last_unstale":"2024-09-12T03:17:39.177700+0000","last_undegraded":"2024-09-12T03:17:39.177700+0000","last_fullsized":"2024-09-12T03:17:39.177700+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_clean_scrub_stamp":"2024-09-12T03:17:38.207304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:00:46.565610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T03:17:39.187235+0000","last_change":"2024-09-12T03:17:39.184842+0000","last_active":"2024-09-12T03:17:39.187235+0000","last_peered":"2024-09-12T03:17:39.187235+0000","last_clean":"2024-09-12T03:17:39.187235+0000","last_became_active":"2024-09-12T03:17:39.184546+0000","last_became_peered":"2024-09-12T03:17:39.184546+0000","last_unstale":"2024-09-12T03:17:39.187235+0000","last_undegraded":"2024-09-12T03:17:39.187235+0000","last_fullsized":"2024-09-12T03:17:39.187235+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_clean_scrub_stamp":"2024-09-12T03:17:38.207304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:03:06.629215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T03:17:39.184307+0000","last_change":"2024-09-12T03:17:39.181688+0000","last_active":"2024-09-12T03:17:39.184307+0000","last_peered":"2024-09-12T03:17:39.184307+0000","last_clean":"2024-09-12T03:17:39.184307+0000","last_became_active":"2024-09-12T03:17:39.181260+0000","last_became_peered":"2024-09-12T03:17:39.181260+0000","last_unstale":"2024-09-12T03:17:39.184307+0000","last_undegraded":"2024-09-12T03:17:39.184307+0000","last_fullsized":"2024-09-12T03:17:39.184307+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_clean_scrub_stamp":"2024-09-12T03:17:38.207304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:22:59.414373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T03:17:39.195119+0000","last_change":"2024-09-12T03:17:39.191333+0000","last_active":"2024-09-12T03:17:39.195119+0000","last_peered":"2024-09-12T03:17:39.195119+0000","last_clean":"2024-09-12T03:17:39.195119+0000","last_became_active":"2024-09-12T03:17:39.190901+0000","last_became_peered":"2024-09-12T03:17:39.190901+0000","last_unstale":"2024-09-12T03:17:39.195119+0000","last_undegraded":"2024-09-12T03:17:39.195119+0000","last_fullsized":"2024-09-12T03:17:39.195119+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_clean_scrub_stamp":"2024-09-12T03:17:38.207304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:55:43.983680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T03:17:39.186605+0000","last_change":"2024-09-12T03:17:39.183448+0000","last_active":"2024-09-12T03:17:39.186605+0000","last_peered":"2024-09-12T03:17:39.186605+0000","last_clean":"2024-09-12T03:17:39.186605+0000","last_became_active":"2024-09-12T03:17:39.183167+0000","last_became_peered":"2024-09-12T03:17:39.183167+0000","last_unstale":"2024-09-12T03:17:39.186605+0000","last_undegraded":"2024-09-12T03:17:39.186605+0000","last_fullsized":"2024-09-12T03:17:39.186605+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_clean_scrub_stamp":"2024-09-12T03:17:38.207304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:48:08.910749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T03:17:39.175494+0000","last_change":"2024-09-12T03:17:39.172605+0000","last_active":"2024-09-12T03:17:39.175494+0000","last_peered":"2024-09-12T03:17:39.175494+0000","last_clean":"2024-09-12T03:17:39.175494+0000","last_became_active":"2024-09-12T03:17:39.172129+0000","last_became_peered":"2024-09-12T03:17:39.172129+0000","last_unstale":"2024-09-12T03:17:39.175494+0000","last_undegraded":"2024-09-12T03:17:39.175494+0000","last_fullsized":"2024-09-12T03:17:39.175494+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_clean_scrub_stamp":"2024-09-12T03:17:38.207304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:06:36.818875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T03:17:39.182104+0000","last_change":"2024-09-12T03:17:39.178938+0000","last_active":"2024-09-12T03:17:39.182104+0000","last_peered":"2024-09-12T03:17:39.182104+0000","last_clean":"2024-09-12T03:17:39.182104+0000","last_became_active":"2024-09-12T03:17:39.178657+0000","last_became_peered":"2024-09-12T03:17:39.178657+0000","last_unstale":"2024-09-12T03:17:39.182104+0000","last_undegraded":"2024-09-12T03:17:39.182104+0000","last_fullsized":"2024-09-12T03:17:39.182104+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_clean_scrub_stamp":"2024-09-12T03:17:38.207304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:53:01.961305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T03:17:39.195066+0000","last_change":"2024-09-12T03:17:39.191234+0000","last_active":"2024-09-12T03:17:39.195066+0000","last_peered":"2024-09-12T03:17:39.195066+0000","last_clean":"2024-09-12T03:17:39.195066+0000","last_became_active":"2024-09-12T03:17:39.190933+0000","last_became_peered":"2024-09-12T03:17:39.190933+0000","last_unstale":"2024-09-12T03:17:39.195066+0000","last_undegraded":"2024-09-12T03:17:39.195066+0000","last_fullsized":"2024-09-12T03:17:39.195066+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_clean_scrub_stamp":"2024-09-12T03:17:38.207304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:28:57.726725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T03:17:39.194972+0000","last_change":"2024-09-12T03:17:39.189763+0000","last_active":"2024-09-12T03:17:39.194972+0000","last_peered":"2024-09-12T03:17:39.194972+0000","last_clean":"2024-09-12T03:17:39.194972+0000","last_became_active":"2024-09-12T03:17:39.189331+0000","last_became_peered":"2024-09-12T03:17:39.189331+0000","last_unstale":"2024-09-12T03:17:39.194972+0000","last_undegraded":"2024-09-12T03:17:39.194972+0000","last_fullsized":"2024-09-12T03:17:39.194972+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_clean_scrub_stamp":"2024-09-12T03:17:38.207304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:57:56.207274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T03:17:39.195148+0000","last_change":"2024-09-12T03:17:39.191587+0000","last_active":"2024-09-12T03:17:39.195148+0000","last_peered":"2024-09-12T03:17:39.195148+0000","last_clean":"2024-09-12T03:17:39.195148+0000","last_became_active":"2024-09-12T03:17:39.191224+0000","last_became_peered":"2024-09-12T03:17:39.191224+0000","last_unstale":"2024-09-12T03:17:39.195148+0000","last_undegraded":"2024-09-12T03:17:39.195148+0000","last_fullsized":"2024-09-12T03:17:39.195148+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_clean_scrub_stamp":"2024-09-12T03:17:38.207304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:59:20.332763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T03:17:39.188634+0000","last_change":"2024-09-12T03:17:39.185246+0000","last_active":"2024-09-12T03:17:39.188634+0000","last_peered":"2024-09-12T03:17:39.188634+0000","last_clean":"2024-09-12T03:17:39.188634+0000","last_became_active":"2024-09-12T03:17:39.184976+0000","last_became_peered":"2024-09-12T03:17:39.184976+0000","last_unstale":"2024-09-12T03:17:39.188634+0000","last_undegraded":"2024-09-12T03:17:39.188634+0000","last_fullsized":"2024-09-12T03:17:39.188634+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_clean_scrub_stamp":"2024-09-12T03:17:38.207304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:53:41.873125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T03:17:39.186672+0000","last_change":"2024-09-12T03:17:39.183143+0000","last_active":"2024-09-12T03:17:39.186672+0000","last_peered":"2024-09-12T03:17:39.186672+0000","last_clean":"2024-09-12T03:17:39.186672+0000","last_became_active":"2024-09-12T03:17:39.182860+0000","last_became_peered":"2024-09-12T03:17:39.182860+0000","last_unstale":"2024-09-12T03:17:39.186672+0000","last_undegraded":"2024-09-12T03:17:39.186672+0000","last_fullsized":"2024-09-12T03:17:39.186672+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_clean_scrub_stamp":"2024-09-12T03:17:38.207304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:19:09.042280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T03:17:39.190558+0000","last_change":"2024-09-12T03:17:39.187326+0000","last_active":"2024-09-12T03:17:39.190558+0000","last_peered":"2024-09-12T03:17:39.190558+0000","last_clean":"2024-09-12T03:17:39.190558+0000","last_became_active":"2024-09-12T03:17:39.187040+0000","last_became_peered":"2024-09-12T03:17:39.187040+0000","last_unstale":"2024-09-12T03:17:39.190558+0000","last_undegraded":"2024-09-12T03:17:39.190558+0000","last_fullsized":"2024-09-12T03:17:39.190558+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_clean_scrub_stamp":"2024-09-12T03:17:38.207304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:09:03.576255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T03:17:39.194928+0000","last_change":"2024-09-12T03:17:39.187777+0000","last_active":"2024-09-12T03:17:39.194928+0000","last_peered":"2024-09-12T03:17:39.194928+0000","last_clean":"2024-09-12T03:17:39.194928+0000","last_became_active":"2024-09-12T03:17:39.187489+0000","last_became_peered":"2024-09-12T03:17:39.187489+0000","last_unstale":"2024-09-12T03:17:39.194928+0000","last_undegraded":"2024-09-12T03:17:39.194928+0000","last_fullsized":"2024-09-12T03:17:39.194928+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_clean_scrub_stamp":"2024-09-12T03:17:38.207304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:26:11.464219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T03:17:39.183039+0000","last_change":"2024-09-12T03:17:39.172527+0000","last_active":"2024-09-12T03:17:39.183039+0000","last_peered":"2024-09-12T03:17:39.183039+0000","last_clean":"2024-09-12T03:17:39.183039+0000","last_became_active":"2024-09-12T03:17:39.172224+0000","last_became_peered":"2024-09-12T03:17:39.172224+0000","last_unstale":"2024-09-12T03:17:39.183039+0000","last_undegraded":"2024-09-12T03:17:39.183039+0000","last_fullsized":"2024-09-12T03:17:39.183039+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_clean_scrub_stamp":"2024-09-12T03:17:38.207304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:20:59.682175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T03:17:39.190556+0000","last_change":"2024-09-12T03:17:39.189749+0000","last_active":"2024-09-12T03:17:39.190556+0000","last_peered":"2024-09-12T03:17:39.190556+0000","last_clean":"2024-09-12T03:17:39.190556+0000","last_became_active":"2024-09-12T03:17:39.189449+0000","last_became_peered":"2024-09-12T03:17:39.189449+0000","last_unstale":"2024-09-12T03:17:39.190556+0000","last_undegraded":"2024-09-12T03:17:39.190556+0000","last_fullsized":"2024-09-12T03:17:39.190556+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_clean_scrub_stamp":"2024-09-12T03:17:38.207304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:30:25.513267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T03:17:39.179973+0000","last_change":"2024-09-12T03:17:39.179077+0000","last_active":"2024-09-12T03:17:39.179973+0000","last_peered":"2024-09-12T03:17:39.179973+0000","last_clean":"2024-09-12T03:17:39.179973+0000","last_became_active":"2024-09-12T03:17:39.178771+0000","last_became_peered":"2024-09-12T03:17:39.178771+0000","last_unstale":"2024-09-12T03:17:39.179973+0000","last_undegraded":"2024-09-12T03:17:39.179973+0000","last_fullsized":"2024-09-12T03:17:39.179973+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_clean_scrub_stamp":"2024-09-12T03:17:38.207304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:51:18.036332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T03:17:39.187503+0000","last_change":"2024-09-12T03:17:39.186110+0000","last_active":"2024-09-12T03:17:39.187503+0000","last_peered":"2024-09-12T03:17:39.187503+0000","last_clean":"2024-09-12T03:17:39.187503+0000","last_became_active":"2024-09-12T03:17:39.185723+0000","last_became_peered":"2024-09-12T03:17:39.185723+0000","last_unstale":"2024-09-12T03:17:39.187503+0000","last_undegraded":"2024-09-12T03:17:39.187503+0000","last_fullsized":"2024-09-12T03:17:39.187503+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_clean_scrub_stamp":"2024-09-12T03:17:38.207304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:42:37.751637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T03:17:39.187837+0000","last_change":"2024-09-12T03:17:39.187049+0000","last_active":"2024-09-12T03:17:39.187837+0000","last_peered":"2024-09-12T03:17:39.187837+0000","last_clean":"2024-09-12T03:17:39.187837+0000","last_became_active":"2024-09-12T03:17:39.186743+0000","last_became_peered":"2024-09-12T03:17:39.186743+0000","last_unstale":"2024-09-12T03:17:39.187837+0000","last_undegraded":"2024-09-12T03:17:39.187837+0000","last_fullsized":"2024-09-12T03:17:39.187837+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_clean_scrub_stamp":"2024-09-12T03:17:38.207304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:00:52.475262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T03:17:39.190649+0000","last_change":"2024-09-12T03:17:39.189386+0000","last_active":"2024-09-12T03:17:39.190649+0000","last_peered":"2024-09-12T03:17:39.190649+0000","last_clean":"2024-09-12T03:17:39.190649+0000","last_became_active":"2024-09-12T03:17:39.189052+0000","last_became_peered":"2024-09-12T03:17:39.189052+0000","last_unstale":"2024-09-12T03:17:39.190649+0000","last_undegraded":"2024-09-12T03:17:39.190649+0000","last_fullsized":"2024-09-12T03:17:39.190649+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_clean_scrub_stamp":"2024-09-12T03:17:38.207304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:20:24.469810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T03:17:39.187224+0000","last_change":"2024-09-12T03:17:39.178220+0000","last_active":"2024-09-12T03:17:39.187224+0000","last_peered":"2024-09-12T03:17:39.187224+0000","last_clean":"2024-09-12T03:17:39.187224+0000","last_became_active":"2024-09-12T03:17:39.177874+0000","last_became_peered":"2024-09-12T03:17:39.177874+0000","last_unstale":"2024-09-12T03:17:39.187224+0000","last_undegraded":"2024-09-12T03:17:39.187224+0000","last_fullsized":"2024-09-12T03:17:39.187224+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_clean_scrub_stamp":"2024-09-12T03:17:38.207304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:37:43.236716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T03:17:39.195129+0000","last_change":"2024-09-12T03:17:39.191543+0000","last_active":"2024-09-12T03:17:39.195129+0000","last_peered":"2024-09-12T03:17:39.195129+0000","last_clean":"2024-09-12T03:17:39.195129+0000","last_became_active":"2024-09-12T03:17:39.191105+0000","last_became_peered":"2024-09-12T03:17:39.191105+0000","last_unstale":"2024-09-12T03:17:39.195129+0000","last_undegraded":"2024-09-12T03:17:39.195129+0000","last_fullsized":"2024-09-12T03:17:39.195129+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_clean_scrub_stamp":"2024-09-12T03:17:38.207304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:47:14.407194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T03:17:39.186339+0000","last_change":"2024-09-12T03:17:39.184418+0000","last_active":"2024-09-12T03:17:39.186339+0000","last_peered":"2024-09-12T03:17:39.186339+0000","last_clean":"2024-09-12T03:17:39.186339+0000","last_became_active":"2024-09-12T03:17:39.183640+0000","last_became_peered":"2024-09-12T03:17:39.183640+0000","last_unstale":"2024-09-12T03:17:39.186339+0000","last_undegraded":"2024-09-12T03:17:39.186339+0000","last_fullsized":"2024-09-12T03:17:39.186339+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_clean_scrub_stamp":"2024-09-12T03:17:38.207304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:02:19.482735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T03:17:39.186142+0000","last_change":"2024-09-12T03:17:39.184147+0000","last_active":"2024-09-12T03:17:39.186142+0000","last_peered":"2024-09-12T03:17:39.186142+0000","last_clean":"2024-09-12T03:17:39.186142+0000","last_became_active":"2024-09-12T03:17:39.183851+0000","last_became_peered":"2024-09-12T03:17:39.183851+0000","last_unstale":"2024-09-12T03:17:39.186142+0000","last_undegraded":"2024-09-12T03:17:39.186142+0000","last_fullsized":"2024-09-12T03:17:39.186142+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_clean_scrub_stamp":"2024-09-12T03:17:38.207304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:23:28.460549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T03:17:39.185505+0000","last_change":"2024-09-12T03:17:39.182136+0000","last_active":"2024-09-12T03:17:39.185505+0000","last_peered":"2024-09-12T03:17:39.185505+0000","last_clean":"2024-09-12T03:17:39.185505+0000","last_became_active":"2024-09-12T03:17:39.181819+0000","last_became_peered":"2024-09-12T03:17:39.181819+0000","last_unstale":"2024-09-12T03:17:39.185505+0000","last_undegraded":"2024-09-12T03:17:39.185505+0000","last_fullsized":"2024-09-12T03:17:39.185505+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_clean_scrub_stamp":"2024-09-12T03:17:38.207304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:56:18.897560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T03:17:39.190583+0000","last_change":"2024-09-12T03:17:39.188667+0000","last_active":"2024-09-12T03:17:39.190583+0000","last_peered":"2024-09-12T03:17:39.190583+0000","last_clean":"2024-09-12T03:17:39.190583+0000","last_became_active":"2024-09-12T03:17:39.188352+0000","last_became_peered":"2024-09-12T03:17:39.188352+0000","last_unstale":"2024-09-12T03:17:39.190583+0000","last_undegraded":"2024-09-12T03:17:39.190583+0000","last_fullsized":"2024-09-12T03:17:39.190583+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_clean_scrub_stamp":"2024-09-12T03:17:38.207304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:36:50.682702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T03:17:39.180193+0000","last_change":"2024-09-12T03:17:39.178027+0000","last_active":"2024-09-12T03:17:39.180193+0000","last_peered":"2024-09-12T03:17:39.180193+0000","last_clean":"2024-09-12T03:17:39.180193+0000","last_became_active":"2024-09-12T03:17:39.177682+0000","last_became_peered":"2024-09-12T03:17:39.177682+0000","last_unstale":"2024-09-12T03:17:39.180193+0000","last_undegraded":"2024-09-12T03:17:39.180193+0000","last_fullsized":"2024-09-12T03:17:39.180193+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_clean_scrub_stamp":"2024-09-12T03:17:38.207304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:28:33.787507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T03:17:39.186649+0000","last_change":"2024-09-12T03:17:39.183076+0000","last_active":"2024-09-12T03:17:39.186649+0000","last_peered":"2024-09-12T03:17:39.186649+0000","last_clean":"2024-09-12T03:17:39.186649+0000","last_became_active":"2024-09-12T03:17:39.182765+0000","last_became_peered":"2024-09-12T03:17:39.182765+0000","last_unstale":"2024-09-12T03:17:39.186649+0000","last_undegraded":"2024-09-12T03:17:39.186649+0000","last_fullsized":"2024-09-12T03:17:39.186649+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_clean_scrub_stamp":"2024-09-12T03:17:38.207304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:52:55.260126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T03:17:39.180047+0000","last_change":"2024-09-12T03:17:39.178484+0000","last_active":"2024-09-12T03:17:39.180047+0000","last_peered":"2024-09-12T03:17:39.180047+0000","last_clean":"2024-09-12T03:17:39.180047+0000","last_became_active":"2024-09-12T03:17:39.178149+0000","last_became_peered":"2024-09-12T03:17:39.178149+0000","last_unstale":"2024-09-12T03:17:39.180047+0000","last_undegraded":"2024-09-12T03:17:39.180047+0000","last_fullsized":"2024-09-12T03:17:39.180047+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_clean_scrub_stamp":"2024-09-12T03:17:38.207304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:11:19.835866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T03:17:39.171711+0000","last_change":"2024-09-12T03:17:39.170360+0000","last_active":"2024-09-12T03:17:39.171711+0000","last_peered":"2024-09-12T03:17:39.171711+0000","last_clean":"2024-09-12T03:17:39.171711+0000","last_became_active":"2024-09-12T03:17:39.169980+0000","last_became_peered":"2024-09-12T03:17:39.169980+0000","last_unstale":"2024-09-12T03:17:39.171711+0000","last_undegraded":"2024-09-12T03:17:39.171711+0000","last_fullsized":"2024-09-12T03:17:39.171711+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_clean_scrub_stamp":"2024-09-12T03:17:38.207304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:28:17.759494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T03:17:39.187808+0000","last_change":"2024-09-12T03:17:39.183165+0000","last_active":"2024-09-12T03:17:39.187808+0000","last_peered":"2024-09-12T03:17:39.187808+0000","last_clean":"2024-09-12T03:17:39.187808+0000","last_became_active":"2024-09-12T03:17:39.182828+0000","last_became_peered":"2024-09-12T03:17:39.182828+0000","last_unstale":"2024-09-12T03:17:39.187808+0000","last_undegraded":"2024-09-12T03:17:39.187808+0000","last_fullsized":"2024-09-12T03:17:39.187808+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_clean_scrub_stamp":"2024-09-12T03:17:38.207304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:48:16.421160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T03:17:39.182465+0000","last_change":"2024-09-12T03:17:39.180606+0000","last_active":"2024-09-12T03:17:39.182465+0000","last_peered":"2024-09-12T03:17:39.182465+0000","last_clean":"2024-09-12T03:17:39.182465+0000","last_became_active":"2024-09-12T03:17:39.180260+0000","last_became_peered":"2024-09-12T03:17:39.180260+0000","last_unstale":"2024-09-12T03:17:39.182465+0000","last_undegraded":"2024-09-12T03:17:39.182465+0000","last_fullsized":"2024-09-12T03:17:39.182465+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_clean_scrub_stamp":"2024-09-12T03:17:38.207304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:41:31.294009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T03:17:39.192599+0000","last_change":"2024-09-12T03:17:39.188782+0000","last_active":"2024-09-12T03:17:39.192599+0000","last_peered":"2024-09-12T03:17:39.192599+0000","last_clean":"2024-09-12T03:17:39.192599+0000","last_became_active":"2024-09-12T03:17:39.188477+0000","last_became_peered":"2024-09-12T03:17:39.188477+0000","last_unstale":"2024-09-12T03:17:39.192599+0000","last_undegraded":"2024-09-12T03:17:39.192599+0000","last_fullsized":"2024-09-12T03:17:39.192599+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_clean_scrub_stamp":"2024-09-12T03:17:38.207304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:51:02.079450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"42'4","reported_seq":16,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T03:17:39.223851+0000","last_change":"2024-09-12T03:17:38.226341+0000","last_active":"2024-09-12T03:17:39.223851+0000","last_peered":"2024-09-12T03:17:39.223851+0000","last_clean":"2024-09-12T03:17:39.223851+0000","last_became_active":"2024-09-12T03:17:38.226069+0000","last_became_peered":"2024-09-12T03:17:38.226069+0000","last_unstale":"2024-09-12T03:17:39.223851+0000","last_undegraded":"2024-09-12T03:17:39.223851+0000","last_fullsized":"2024-09-12T03:17:39.223851+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:37.205254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:37.205254+0000","last_clean_scrub_stamp":"2024-09-12T03:17:37.205254+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:55:42.981630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T03:17:39.181694+0000","last_change":"2024-09-12T03:17:39.178101+0000","last_active":"2024-09-12T03:17:39.181694+0000","last_peered":"2024-09-12T03:17:39.181694+0000","last_clean":"2024-09-12T03:17:39.181694+0000","last_became_active":"2024-09-12T03:17:39.177770+0000","last_became_peered":"2024-09-12T03:17:39.177770+0000","last_unstale":"2024-09-12T03:17:39.181694+0000","last_undegraded":"2024-09-12T03:17:39.181694+0000","last_fullsized":"2024-09-12T03:17:39.181694+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_clean_scrub_stamp":"2024-09-12T03:17:38.207304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:31:10.365720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"42'1","reported_seq":13,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T03:17:39.223607+0000","last_change":"2024-09-12T03:17:38.227887+0000","last_active":"2024-09-12T03:17:39.223607+0000","last_peered":"2024-09-12T03:17:39.223607+0000","last_clean":"2024-09-12T03:17:39.223607+0000","last_became_active":"2024-09-12T03:17:38.227609+0000","last_became_peered":"2024-09-12T03:17:38.227609+0000","last_unstale":"2024-09-12T03:17:39.223607+0000","last_undegraded":"2024-09-12T03:17:39.223607+0000","last_fullsized":"2024-09-12T03:17:39.223607+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:37.205254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:37.205254+0000","last_clean_scrub_stamp":"2024-09-12T03:17:37.205254+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:37:50.091928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T03:17:39.189160+0000","last_change":"2024-09-12T03:17:39.188518+0000","last_active":"2024-09-12T03:17:39.189160+0000","last_peered":"2024-09-12T03:17:39.189160+0000","last_clean":"2024-09-12T03:17:39.189160+0000","last_became_active":"2024-09-12T03:17:39.188158+0000","last_became_peered":"2024-09-12T03:17:39.188158+0000","last_unstale":"2024-09-12T03:17:39.189160+0000","last_undegraded":"2024-09-12T03:17:39.189160+0000","last_fullsized":"2024-09-12T03:17:39.189160+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_clean_scrub_stamp":"2024-09-12T03:17:38.207304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:40:17.397085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"42'2","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T03:17:39.222127+0000","last_change":"2024-09-12T03:17:38.231103+0000","last_active":"2024-09-12T03:17:39.222127+0000","last_peered":"2024-09-12T03:17:39.222127+0000","last_clean":"2024-09-12T03:17:39.222127+0000","last_became_active":"2024-09-12T03:17:38.230817+0000","last_became_peered":"2024-09-12T03:17:38.230817+0000","last_unstale":"2024-09-12T03:17:39.222127+0000","last_undegraded":"2024-09-12T03:17:39.222127+0000","last_fullsized":"2024-09-12T03:17:39.222127+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:37.205254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:37.205254+0000","last_clean_scrub_stamp":"2024-09-12T03:17:37.205254+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:41:02.541069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T03:17:39.192633+0000","last_change":"2024-09-12T03:17:39.190319+0000","last_active":"2024-09-12T03:17:39.192633+0000","last_peered":"2024-09-12T03:17:39.192633+0000","last_clean":"2024-09-12T03:17:39.192633+0000","last_became_active":"2024-09-12T03:17:39.189993+0000","last_became_peered":"2024-09-12T03:17:39.189993+0000","last_unstale":"2024-09-12T03:17:39.192633+0000","last_undegraded":"2024-09-12T03:17:39.192633+0000","last_fullsized":"2024-09-12T03:17:39.192633+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_clean_scrub_stamp":"2024-09-12T03:17:38.207304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:44:43.661143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"42'2","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T03:17:39.222537+0000","last_change":"2024-09-12T03:17:38.230293+0000","last_active":"2024-09-12T03:17:39.222537+0000","last_peered":"2024-09-12T03:17:39.222537+0000","last_clean":"2024-09-12T03:17:39.222537+0000","last_became_active":"2024-09-12T03:17:38.230030+0000","last_became_peered":"2024-09-12T03:17:38.230030+0000","last_unstale":"2024-09-12T03:17:39.222537+0000","last_undegraded":"2024-09-12T03:17:39.222537+0000","last_fullsized":"2024-09-12T03:17:39.222537+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:37.205254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:37.205254+0000","last_clean_scrub_stamp":"2024-09-12T03:17:37.205254+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:55:42.981630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T03:17:39.178483+0000","last_change":"2024-09-12T03:17:39.177269+0000","last_active":"2024-09-12T03:17:39.178483+0000","last_peered":"2024-09-12T03:17:39.178483+0000","last_clean":"2024-09-12T03:17:39.178483+0000","last_became_active":"2024-09-12T03:17:39.176965+0000","last_became_peered":"2024-09-12T03:17:39.176965+0000","last_unstale":"2024-09-12T03:17:39.178483+0000","last_undegraded":"2024-09-12T03:17:39.178483+0000","last_fullsized":"2024-09-12T03:17:39.178483+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_clean_scrub_stamp":"2024-09-12T03:17:38.207304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:19:36.833974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"42'1","reported_seq":13,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T03:17:39.222541+0000","last_change":"2024-09-12T03:17:38.228366+0000","last_active":"2024-09-12T03:17:39.222541+0000","last_peered":"2024-09-12T03:17:39.222541+0000","last_clean":"2024-09-12T03:17:39.222541+0000","last_became_active":"2024-09-12T03:17:38.228071+0000","last_became_peered":"2024-09-12T03:17:38.228071+0000","last_unstale":"2024-09-12T03:17:39.222541+0000","last_undegraded":"2024-09-12T03:17:39.222541+0000","last_fullsized":"2024-09-12T03:17:39.222541+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:37.205254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:37.205254+0000","last_clean_scrub_stamp":"2024-09-12T03:17:37.205254+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:59:19.330713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T03:17:39.188690+0000","last_change":"2024-09-12T03:17:39.185323+0000","last_active":"2024-09-12T03:17:39.188690+0000","last_peered":"2024-09-12T03:17:39.188690+0000","last_clean":"2024-09-12T03:17:39.188690+0000","last_became_active":"2024-09-12T03:17:39.184897+0000","last_became_peered":"2024-09-12T03:17:39.184897+0000","last_unstale":"2024-09-12T03:17:39.188690+0000","last_undegraded":"2024-09-12T03:17:39.188690+0000","last_fullsized":"2024-09-12T03:17:39.188690+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_clean_scrub_stamp":"2024-09-12T03:17:38.207304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:47:18.341852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"42'1","reported_seq":13,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T03:17:39.223762+0000","last_change":"2024-09-12T03:17:38.226857+0000","last_active":"2024-09-12T03:17:39.223762+0000","last_peered":"2024-09-12T03:17:39.223762+0000","last_clean":"2024-09-12T03:17:39.223762+0000","last_became_active":"2024-09-12T03:17:38.226565+0000","last_became_peered":"2024-09-12T03:17:38.226565+0000","last_unstale":"2024-09-12T03:17:39.223762+0000","last_undegraded":"2024-09-12T03:17:39.223762+0000","last_fullsized":"2024-09-12T03:17:39.223762+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:37.205254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:37.205254+0000","last_clean_scrub_stamp":"2024-09-12T03:17:37.205254+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:00:51.473212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T03:17:39.194994+0000","last_change":"2024-09-12T03:17:39.190485+0000","last_active":"2024-09-12T03:17:39.194994+0000","last_peered":"2024-09-12T03:17:39.194994+0000","last_clean":"2024-09-12T03:17:39.194994+0000","last_became_active":"2024-09-12T03:17:39.190198+0000","last_became_peered":"2024-09-12T03:17:39.190198+0000","last_unstale":"2024-09-12T03:17:39.194994+0000","last_undegraded":"2024-09-12T03:17:39.194994+0000","last_fullsized":"2024-09-12T03:17:39.194994+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_clean_scrub_stamp":"2024-09-12T03:17:38.207304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:13:15.002267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T03:17:39.174589+0000","last_change":"2024-09-12T03:17:38.226293+0000","last_active":"2024-09-12T03:17:39.174589+0000","last_peered":"2024-09-12T03:17:39.174589+0000","last_clean":"2024-09-12T03:17:39.174589+0000","last_became_active":"2024-09-12T03:17:38.226014+0000","last_became_peered":"2024-09-12T03:17:38.226014+0000","last_unstale":"2024-09-12T03:17:39.174589+0000","last_undegraded":"2024-09-12T03:17:39.174589+0000","last_fullsized":"2024-09-12T03:17:39.174589+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:37.205254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:37.205254+0000","last_clean_scrub_stamp":"2024-09-12T03:17:37.205254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:56:04.799549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T03:17:39.186358+0000","last_change":"2024-09-12T03:17:39.184689+0000","last_active":"2024-09-12T03:17:39.186358+0000","last_peered":"2024-09-12T03:17:39.186358+0000","last_clean":"2024-09-12T03:17:39.186358+0000","last_became_active":"2024-09-12T03:17:39.184051+0000","last_became_peered":"2024-09-12T03:17:39.184051+0000","last_unstale":"2024-09-12T03:17:39.186358+0000","last_undegraded":"2024-09-12T03:17:39.186358+0000","last_fullsized":"2024-09-12T03:17:39.186358+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_clean_scrub_stamp":"2024-09-12T03:17:38.207304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:45:28.696730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T03:17:39.173120+0000","last_change":"2024-09-12T03:17:38.228438+0000","last_active":"2024-09-12T03:17:39.173120+0000","last_peered":"2024-09-12T03:17:39.173120+0000","last_clean":"2024-09-12T03:17:39.173120+0000","last_became_active":"2024-09-12T03:17:38.228162+0000","last_became_peered":"2024-09-12T03:17:38.228162+0000","last_unstale":"2024-09-12T03:17:39.173120+0000","last_undegraded":"2024-09-12T03:17:39.173120+0000","last_fullsized":"2024-09-12T03:17:39.173120+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:37.205254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:37.205254+0000","last_clean_scrub_stamp":"2024-09-12T03:17:37.205254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:28:16.757444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T03:17:39.186771+0000","last_change":"2024-09-12T03:17:39.185650+0000","last_active":"2024-09-12T03:17:39.186771+0000","last_peered":"2024-09-12T03:17:39.186771+0000","last_clean":"2024-09-12T03:17:39.186771+0000","last_became_active":"2024-09-12T03:17:39.185277+0000","last_became_peered":"2024-09-12T03:17:39.185277+0000","last_unstale":"2024-09-12T03:17:39.186771+0000","last_undegraded":"2024-09-12T03:17:39.186771+0000","last_fullsized":"2024-09-12T03:17:39.186771+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_clean_scrub_stamp":"2024-09-12T03:17:38.207304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:24:56.579074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T03:17:39.169552+0000","last_change":"2024-09-12T03:17:38.223901+0000","last_active":"2024-09-12T03:17:39.169552+0000","last_peered":"2024-09-12T03:17:39.169552+0000","last_clean":"2024-09-12T03:17:39.169552+0000","last_became_active":"2024-09-12T03:17:38.223564+0000","last_became_peered":"2024-09-12T03:17:38.223564+0000","last_unstale":"2024-09-12T03:17:39.169552+0000","last_undegraded":"2024-09-12T03:17:39.169552+0000","last_fullsized":"2024-09-12T03:17:39.169552+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:37.205254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:37.205254+0000","last_clean_scrub_stamp":"2024-09-12T03:17:37.205254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:30:24.511217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T03:17:39.182153+0000","last_change":"2024-09-12T03:17:39.180327+0000","last_active":"2024-09-12T03:17:39.182153+0000","last_peered":"2024-09-12T03:17:39.182153+0000","last_clean":"2024-09-12T03:17:39.182153+0000","last_became_active":"2024-09-12T03:17:39.179952+0000","last_became_peered":"2024-09-12T03:17:39.179952+0000","last_unstale":"2024-09-12T03:17:39.182153+0000","last_undegraded":"2024-09-12T03:17:39.182153+0000","last_fullsized":"2024-09-12T03:17:39.182153+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_clean_scrub_stamp":"2024-09-12T03:17:38.207304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:20:59.682175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T03:17:39.176725+0000","last_change":"2024-09-12T03:17:38.230663+0000","last_active":"2024-09-12T03:17:39.176725+0000","last_peered":"2024-09-12T03:17:39.176725+0000","last_clean":"2024-09-12T03:17:39.176725+0000","last_became_active":"2024-09-12T03:17:38.230367+0000","last_became_peered":"2024-09-12T03:17:38.230367+0000","last_unstale":"2024-09-12T03:17:39.176725+0000","last_undegraded":"2024-09-12T03:17:39.176725+0000","last_fullsized":"2024-09-12T03:17:39.176725+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:37.205254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:37.205254+0000","last_clean_scrub_stamp":"2024-09-12T03:17:37.205254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:25:35.607418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T03:17:39.187589+0000","last_change":"2024-09-12T03:17:39.181753+0000","last_active":"2024-09-12T03:17:39.187589+0000","last_peered":"2024-09-12T03:17:39.187589+0000","last_clean":"2024-09-12T03:17:39.187589+0000","last_became_active":"2024-09-12T03:17:39.181440+0000","last_became_peered":"2024-09-12T03:17:39.181440+0000","last_unstale":"2024-09-12T03:17:39.187589+0000","last_undegraded":"2024-09-12T03:17:39.187589+0000","last_fullsized":"2024-09-12T03:17:39.187589+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_clean_scrub_stamp":"2024-09-12T03:17:38.207304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:20:08.109520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"42'3","reported_seq":15,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T03:17:39.223335+0000","last_change":"2024-09-12T03:17:38.228597+0000","last_active":"2024-09-12T03:17:39.223335+0000","last_peered":"2024-09-12T03:17:39.223335+0000","last_clean":"2024-09-12T03:17:39.223335+0000","last_became_active":"2024-09-12T03:17:38.227932+0000","last_became_peered":"2024-09-12T03:17:38.227932+0000","last_unstale":"2024-09-12T03:17:39.223335+0000","last_undegraded":"2024-09-12T03:17:39.223335+0000","last_fullsized":"2024-09-12T03:17:39.223335+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:37.205254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:37.205254+0000","last_clean_scrub_stamp":"2024-09-12T03:17:37.205254+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:59:19.330713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T03:17:39.188339+0000","last_change":"2024-09-12T03:17:39.187592+0000","last_active":"2024-09-12T03:17:39.188339+0000","last_peered":"2024-09-12T03:17:39.188339+0000","last_clean":"2024-09-12T03:17:39.188339+0000","last_became_active":"2024-09-12T03:17:39.187015+0000","last_became_peered":"2024-09-12T03:17:39.187015+0000","last_unstale":"2024-09-12T03:17:39.188339+0000","last_undegraded":"2024-09-12T03:17:39.188339+0000","last_fullsized":"2024-09-12T03:17:39.188339+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_clean_scrub_stamp":"2024-09-12T03:17:38.207304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:48:50.171581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"42'6","reported_seq":18,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T03:17:39.228571+0000","last_change":"2024-09-12T03:17:38.224132+0000","last_active":"2024-09-12T03:17:39.228571+0000","last_peered":"2024-09-12T03:17:39.228571+0000","last_clean":"2024-09-12T03:17:39.228571+0000","last_became_active":"2024-09-12T03:17:38.223865+0000","last_became_peered":"2024-09-12T03:17:38.223865+0000","last_unstale":"2024-09-12T03:17:39.228571+0000","last_undegraded":"2024-09-12T03:17:39.228571+0000","last_fullsized":"2024-09-12T03:17:39.228571+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:37.205254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:37.205254+0000","last_clean_scrub_stamp":"2024-09-12T03:17:37.205254+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:48:19.664006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T03:17:39.190125+0000","last_change":"2024-09-12T03:17:39.189275+0000","last_active":"2024-09-12T03:17:39.190125+0000","last_peered":"2024-09-12T03:17:39.190125+0000","last_clean":"2024-09-12T03:17:39.190125+0000","last_became_active":"2024-09-12T03:17:39.188902+0000","last_became_peered":"2024-09-12T03:17:39.188902+0000","last_unstale":"2024-09-12T03:17:39.190125+0000","last_undegraded":"2024-09-12T03:17:39.190125+0000","last_fullsized":"2024-09-12T03:17:39.190125+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_clean_scrub_stamp":"2024-09-12T03:17:38.207304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:16:03.541630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"42'2","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T03:17:39.228701+0000","last_change":"2024-09-12T03:17:38.227756+0000","last_active":"2024-09-12T03:17:39.228701+0000","last_peered":"2024-09-12T03:17:39.228701+0000","last_clean":"2024-09-12T03:17:39.228701+0000","last_became_active":"2024-09-12T03:17:38.227479+0000","last_became_peered":"2024-09-12T03:17:38.227479+0000","last_unstale":"2024-09-12T03:17:39.228701+0000","last_undegraded":"2024-09-12T03:17:39.228701+0000","last_fullsized":"2024-09-12T03:17:39.228701+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:37.205254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:37.205254+0000","last_clean_scrub_stamp":"2024-09-12T03:17:37.205254+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:42:17.989868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T03:17:39.192402+0000","last_change":"2024-09-12T03:17:39.187643+0000","last_active":"2024-09-12T03:17:39.192402+0000","last_peered":"2024-09-12T03:17:39.192402+0000","last_clean":"2024-09-12T03:17:39.192402+0000","last_became_active":"2024-09-12T03:17:39.187358+0000","last_became_peered":"2024-09-12T03:17:39.187358+0000","last_unstale":"2024-09-12T03:17:39.192402+0000","last_undegraded":"2024-09-12T03:17:39.192402+0000","last_fullsized":"2024-09-12T03:17:39.192402+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_clean_scrub_stamp":"2024-09-12T03:17:38.207304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:54:02.526889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T03:17:39.166986+0000","last_change":"2024-09-12T03:17:38.228351+0000","last_active":"2024-09-12T03:17:39.166986+0000","last_peered":"2024-09-12T03:17:39.166986+0000","last_clean":"2024-09-12T03:17:39.166986+0000","last_became_active":"2024-09-12T03:17:38.228013+0000","last_became_peered":"2024-09-12T03:17:38.228013+0000","last_unstale":"2024-09-12T03:17:39.166986+0000","last_undegraded":"2024-09-12T03:17:39.166986+0000","last_fullsized":"2024-09-12T03:17:39.166986+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:37.205254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:37.205254+0000","last_clean_scrub_stamp":"2024-09-12T03:17:37.205254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:28:56.724675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T03:17:39.187754+0000","last_change":"2024-09-12T03:17:39.181912+0000","last_active":"2024-09-12T03:17:39.187754+0000","last_peered":"2024-09-12T03:17:39.187754+0000","last_clean":"2024-09-12T03:17:39.187754+0000","last_became_active":"2024-09-12T03:17:39.181341+0000","last_became_peered":"2024-09-12T03:17:39.181341+0000","last_unstale":"2024-09-12T03:17:39.187754+0000","last_undegraded":"2024-09-12T03:17:39.187754+0000","last_fullsized":"2024-09-12T03:17:39.187754+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_clean_scrub_stamp":"2024-09-12T03:17:38.207304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:09:03.058255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T03:17:39.169262+0000","last_change":"2024-09-12T03:17:38.224498+0000","last_active":"2024-09-12T03:17:39.169262+0000","last_peered":"2024-09-12T03:17:39.169262+0000","last_clean":"2024-09-12T03:17:39.169262+0000","last_became_active":"2024-09-12T03:17:38.224211+0000","last_became_peered":"2024-09-12T03:17:38.224211+0000","last_unstale":"2024-09-12T03:17:39.169262+0000","last_undegraded":"2024-09-12T03:17:39.169262+0000","last_fullsized":"2024-09-12T03:17:39.169262+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:37.205254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:37.205254+0000","last_clean_scrub_stamp":"2024-09-12T03:17:37.205254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:47:13.405144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T03:17:39.182922+0000","last_change":"2024-09-12T03:17:39.181071+0000","last_active":"2024-09-12T03:17:39.182922+0000","last_peered":"2024-09-12T03:17:39.182922+0000","last_clean":"2024-09-12T03:17:39.182922+0000","last_became_active":"2024-09-12T03:17:39.180760+0000","last_became_peered":"2024-09-12T03:17:39.180760+0000","last_unstale":"2024-09-12T03:17:39.182922+0000","last_undegraded":"2024-09-12T03:17:39.182922+0000","last_fullsized":"2024-09-12T03:17:39.182922+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_clean_scrub_stamp":"2024-09-12T03:17:38.207304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:37:43.236716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"42'8","reported_seq":20,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T03:17:39.228770+0000","last_change":"2024-09-12T03:17:38.229628+0000","last_active":"2024-09-12T03:17:39.228770+0000","last_peered":"2024-09-12T03:17:39.228770+0000","last_clean":"2024-09-12T03:17:39.228770+0000","last_became_active":"2024-09-12T03:17:38.229289+0000","last_became_peered":"2024-09-12T03:17:38.229289+0000","last_unstale":"2024-09-12T03:17:39.228770+0000","last_undegraded":"2024-09-12T03:17:39.228770+0000","last_fullsized":"2024-09-12T03:17:39.228770+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:37.205254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:37.205254+0000","last_clean_scrub_stamp":"2024-09-12T03:17:37.205254+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:19:35.831924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T03:17:39.183827+0000","last_change":"2024-09-12T03:17:39.182414+0000","last_active":"2024-09-12T03:17:39.183827+0000","last_peered":"2024-09-12T03:17:39.183827+0000","last_clean":"2024-09-12T03:17:39.183827+0000","last_became_active":"2024-09-12T03:17:39.182130+0000","last_became_peered":"2024-09-12T03:17:39.182130+0000","last_unstale":"2024-09-12T03:17:39.183827+0000","last_undegraded":"2024-09-12T03:17:39.183827+0000","last_fullsized":"2024-09-12T03:17:39.183827+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_clean_scrub_stamp":"2024-09-12T03:17:38.207304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:17:58.861067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"42'1","reported_seq":13,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T03:17:39.221768+0000","last_change":"2024-09-12T03:17:38.226193+0000","last_active":"2024-09-12T03:17:39.221768+0000","last_peered":"2024-09-12T03:17:39.221768+0000","last_clean":"2024-09-12T03:17:39.221768+0000","last_became_active":"2024-09-12T03:17:38.225886+0000","last_became_peered":"2024-09-12T03:17:38.225886+0000","last_unstale":"2024-09-12T03:17:39.221768+0000","last_undegraded":"2024-09-12T03:17:39.221768+0000","last_fullsized":"2024-09-12T03:17:39.221768+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:37.205254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:37.205254+0000","last_clean_scrub_stamp":"2024-09-12T03:17:37.205254+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:39:58.968031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T03:17:39.182397+0000","last_change":"2024-09-12T03:17:39.179806+0000","last_active":"2024-09-12T03:17:39.182397+0000","last_peered":"2024-09-12T03:17:39.182397+0000","last_clean":"2024-09-12T03:17:39.182397+0000","last_became_active":"2024-09-12T03:17:39.178917+0000","last_became_peered":"2024-09-12T03:17:39.178917+0000","last_unstale":"2024-09-12T03:17:39.182397+0000","last_undegraded":"2024-09-12T03:17:39.182397+0000","last_fullsized":"2024-09-12T03:17:39.182397+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_clean_scrub_stamp":"2024-09-12T03:17:38.207304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:22:59.414373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"42'2","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T03:17:39.226101+0000","last_change":"2024-09-12T03:17:38.228137+0000","last_active":"2024-09-12T03:17:39.226101+0000","last_peered":"2024-09-12T03:17:39.226101+0000","last_clean":"2024-09-12T03:17:39.226101+0000","last_became_active":"2024-09-12T03:17:38.227234+0000","last_became_peered":"2024-09-12T03:17:38.227234+0000","last_unstale":"2024-09-12T03:17:39.226101+0000","last_undegraded":"2024-09-12T03:17:39.226101+0000","last_fullsized":"2024-09-12T03:17:39.226101+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:37.205254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:37.205254+0000","last_clean_scrub_stamp":"2024-09-12T03:17:37.205254+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:16:02.539580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T03:17:39.189281+0000","last_change":"2024-09-12T03:17:39.188494+0000","last_active":"2024-09-12T03:17:39.189281+0000","last_peered":"2024-09-12T03:17:39.189281+0000","last_clean":"2024-09-12T03:17:39.189281+0000","last_became_active":"2024-09-12T03:17:39.188174+0000","last_became_peered":"2024-09-12T03:17:39.188174+0000","last_unstale":"2024-09-12T03:17:39.189281+0000","last_undegraded":"2024-09-12T03:17:39.189281+0000","last_fullsized":"2024-09-12T03:17:39.189281+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_clean_scrub_stamp":"2024-09-12T03:17:38.207304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:00:03.080107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"42'1","reported_seq":13,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T03:17:39.223987+0000","last_change":"2024-09-12T03:17:38.228625+0000","last_active":"2024-09-12T03:17:39.223987+0000","last_peered":"2024-09-12T03:17:39.223987+0000","last_clean":"2024-09-12T03:17:39.223987+0000","last_became_active":"2024-09-12T03:17:38.228353+0000","last_became_peered":"2024-09-12T03:17:38.228353+0000","last_unstale":"2024-09-12T03:17:39.223987+0000","last_undegraded":"2024-09-12T03:17:39.223987+0000","last_fullsized":"2024-09-12T03:17:39.223987+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:37.205254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:37.205254+0000","last_clean_scrub_stamp":"2024-09-12T03:17:37.205254+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:54:28.297593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T03:17:39.190801+0000","last_change":"2024-09-12T03:17:39.189954+0000","last_active":"2024-09-12T03:17:39.190801+0000","last_peered":"2024-09-12T03:17:39.190801+0000","last_clean":"2024-09-12T03:17:39.190801+0000","last_became_active":"2024-09-12T03:17:39.189642+0000","last_became_peered":"2024-09-12T03:17:39.189642+0000","last_unstale":"2024-09-12T03:17:39.190801+0000","last_undegraded":"2024-09-12T03:17:39.190801+0000","last_fullsized":"2024-09-12T03:17:39.190801+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_clean_scrub_stamp":"2024-09-12T03:17:38.207304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:28:17.759494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T03:17:39.170373+0000","last_change":"2024-09-12T03:17:38.225655+0000","last_active":"2024-09-12T03:17:39.170373+0000","last_peered":"2024-09-12T03:17:39.170373+0000","last_clean":"2024-09-12T03:17:39.170373+0000","last_became_active":"2024-09-12T03:17:38.225350+0000","last_became_peered":"2024-09-12T03:17:38.225350+0000","last_unstale":"2024-09-12T03:17:39.170373+0000","last_undegraded":"2024-09-12T03:17:39.170373+0000","last_fullsized":"2024-09-12T03:17:39.170373+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:37.205254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:37.205254+0000","last_clean_scrub_stamp":"2024-09-12T03:17:37.205254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:53:00.959255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T03:17:39.187514+0000","last_change":"2024-09-12T03:17:39.180771+0000","last_active":"2024-09-12T03:17:39.187514+0000","last_peered":"2024-09-12T03:17:39.187514+0000","last_clean":"2024-09-12T03:17:39.187514+0000","last_became_active":"2024-09-12T03:17:39.180471+0000","last_became_peered":"2024-09-12T03:17:39.180471+0000","last_unstale":"2024-09-12T03:17:39.187514+0000","last_undegraded":"2024-09-12T03:17:39.187514+0000","last_fullsized":"2024-09-12T03:17:39.187514+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_clean_scrub_stamp":"2024-09-12T03:17:38.207304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:49:51.086834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"42'2","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T03:17:39.222876+0000","last_change":"2024-09-12T03:17:38.227044+0000","last_active":"2024-09-12T03:17:39.222876+0000","last_peered":"2024-09-12T03:17:39.222876+0000","last_clean":"2024-09-12T03:17:39.222876+0000","last_became_active":"2024-09-12T03:17:38.226612+0000","last_became_peered":"2024-09-12T03:17:38.226612+0000","last_unstale":"2024-09-12T03:17:39.222876+0000","last_undegraded":"2024-09-12T03:17:39.222876+0000","last_fullsized":"2024-09-12T03:17:39.222876+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:37.205254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:37.205254+0000","last_clean_scrub_stamp":"2024-09-12T03:17:37.205254+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:00:45.563560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T03:17:39.187769+0000","last_change":"2024-09-12T03:17:39.182674+0000","last_active":"2024-09-12T03:17:39.187769+0000","last_peered":"2024-09-12T03:17:39.187769+0000","last_clean":"2024-09-12T03:17:39.187769+0000","last_became_active":"2024-09-12T03:17:39.180953+0000","last_became_peered":"2024-09-12T03:17:39.180953+0000","last_unstale":"2024-09-12T03:17:39.187769+0000","last_undegraded":"2024-09-12T03:17:39.187769+0000","last_fullsized":"2024-09-12T03:17:39.187769+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_clean_scrub_stamp":"2024-09-12T03:17:38.207304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:07:34.120803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T03:17:39.174268+0000","last_change":"2024-09-12T03:17:38.225458+0000","last_active":"2024-09-12T03:17:39.174268+0000","last_peered":"2024-09-12T03:17:39.174268+0000","last_clean":"2024-09-12T03:17:39.174268+0000","last_became_active":"2024-09-12T03:17:38.225157+0000","last_became_peered":"2024-09-12T03:17:38.225157+0000","last_unstale":"2024-09-12T03:17:39.174268+0000","last_undegraded":"2024-09-12T03:17:39.174268+0000","last_fullsized":"2024-09-12T03:17:39.174268+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:37.205254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:37.205254+0000","last_clean_scrub_stamp":"2024-09-12T03:17:37.205254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:52:38.099540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T03:17:39.186530+0000","last_change":"2024-09-12T03:17:39.184851+0000","last_active":"2024-09-12T03:17:39.186530+0000","last_peered":"2024-09-12T03:17:39.186530+0000","last_clean":"2024-09-12T03:17:39.186530+0000","last_became_active":"2024-09-12T03:17:39.184542+0000","last_became_peered":"2024-09-12T03:17:39.184542+0000","last_unstale":"2024-09-12T03:17:39.186530+0000","last_undegraded":"2024-09-12T03:17:39.186530+0000","last_fullsized":"2024-09-12T03:17:39.186530+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_clean_scrub_stamp":"2024-09-12T03:17:38.207304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:56:31.329671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T03:17:39.174137+0000","last_change":"2024-09-12T03:17:38.231346+0000","last_active":"2024-09-12T03:17:39.174137+0000","last_peered":"2024-09-12T03:17:39.174137+0000","last_clean":"2024-09-12T03:17:39.174137+0000","last_became_active":"2024-09-12T03:17:38.231069+0000","last_became_peered":"2024-09-12T03:17:38.231069+0000","last_unstale":"2024-09-12T03:17:39.174137+0000","last_undegraded":"2024-09-12T03:17:39.174137+0000","last_fullsized":"2024-09-12T03:17:39.174137+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:37.205254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:37.205254+0000","last_clean_scrub_stamp":"2024-09-12T03:17:37.205254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:51:17.034282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T03:17:39.192349+0000","last_change":"2024-09-12T03:17:39.186987+0000","last_active":"2024-09-12T03:17:39.192349+0000","last_peered":"2024-09-12T03:17:39.192349+0000","last_clean":"2024-09-12T03:17:39.192349+0000","last_became_active":"2024-09-12T03:17:39.186693+0000","last_became_peered":"2024-09-12T03:17:39.186693+0000","last_unstale":"2024-09-12T03:17:39.192349+0000","last_undegraded":"2024-09-12T03:17:39.192349+0000","last_fullsized":"2024-09-12T03:17:39.192349+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_clean_scrub_stamp":"2024-09-12T03:17:38.207304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:06:30.838342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T03:17:39.166404+0000","last_change":"2024-09-12T03:17:38.229729+0000","last_active":"2024-09-12T03:17:39.166404+0000","last_peered":"2024-09-12T03:17:39.166404+0000","last_clean":"2024-09-12T03:17:39.166404+0000","last_became_active":"2024-09-12T03:17:38.229282+0000","last_became_peered":"2024-09-12T03:17:38.229282+0000","last_unstale":"2024-09-12T03:17:39.166404+0000","last_undegraded":"2024-09-12T03:17:39.166404+0000","last_fullsized":"2024-09-12T03:17:39.166404+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:37.205254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:37.205254+0000","last_clean_scrub_stamp":"2024-09-12T03:17:37.205254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:13:14.000217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T03:17:39.190124+0000","last_change":"2024-09-12T03:17:39.189150+0000","last_active":"2024-09-12T03:17:39.190124+0000","last_peered":"2024-09-12T03:17:39.190124+0000","last_clean":"2024-09-12T03:17:39.190124+0000","last_became_active":"2024-09-12T03:17:39.188853+0000","last_became_peered":"2024-09-12T03:17:39.188853+0000","last_unstale":"2024-09-12T03:17:39.190124+0000","last_undegraded":"2024-09-12T03:17:39.190124+0000","last_fullsized":"2024-09-12T03:17:39.190124+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_clean_scrub_stamp":"2024-09-12T03:17:38.207304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:37:51.093978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T03:17:39.168708+0000","last_change":"2024-09-12T03:17:38.225837+0000","last_active":"2024-09-12T03:17:39.168708+0000","last_peered":"2024-09-12T03:17:39.168708+0000","last_clean":"2024-09-12T03:17:39.168708+0000","last_became_active":"2024-09-12T03:17:38.225094+0000","last_became_peered":"2024-09-12T03:17:38.225094+0000","last_unstale":"2024-09-12T03:17:39.168708+0000","last_undegraded":"2024-09-12T03:17:39.168708+0000","last_fullsized":"2024-09-12T03:17:39.168708+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:37.205254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:37.205254+0000","last_clean_scrub_stamp":"2024-09-12T03:17:37.205254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:28:56.724675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T03:17:39.186432+0000","last_change":"2024-09-12T03:17:39.180265+0000","last_active":"2024-09-12T03:17:39.186432+0000","last_peered":"2024-09-12T03:17:39.186432+0000","last_clean":"2024-09-12T03:17:39.186432+0000","last_became_active":"2024-09-12T03:17:39.178568+0000","last_became_peered":"2024-09-12T03:17:39.178568+0000","last_unstale":"2024-09-12T03:17:39.186432+0000","last_undegraded":"2024-09-12T03:17:39.186432+0000","last_fullsized":"2024-09-12T03:17:39.186432+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_clean_scrub_stamp":"2024-09-12T03:17:38.207304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:24:56.579074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T03:17:39.168255+0000","last_change":"2024-09-12T03:17:38.228816+0000","last_active":"2024-09-12T03:17:39.168255+0000","last_peered":"2024-09-12T03:17:39.168255+0000","last_clean":"2024-09-12T03:17:39.168255+0000","last_became_active":"2024-09-12T03:17:38.228317+0000","last_became_peered":"2024-09-12T03:17:38.228317+0000","last_unstale":"2024-09-12T03:17:39.168255+0000","last_undegraded":"2024-09-12T03:17:39.168255+0000","last_fullsized":"2024-09-12T03:17:39.168255+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:37.205254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:37.205254+0000","last_clean_scrub_stamp":"2024-09-12T03:17:37.205254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:42:17.989868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T03:17:39.182259+0000","last_change":"2024-09-12T03:17:39.180919+0000","last_active":"2024-09-12T03:17:39.182259+0000","last_peered":"2024-09-12T03:17:39.182259+0000","last_clean":"2024-09-12T03:17:39.182259+0000","last_became_active":"2024-09-12T03:17:39.180625+0000","last_became_peered":"2024-09-12T03:17:39.180625+0000","last_unstale":"2024-09-12T03:17:39.182259+0000","last_undegraded":"2024-09-12T03:17:39.182259+0000","last_fullsized":"2024-09-12T03:17:39.182259+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_clean_scrub_stamp":"2024-09-12T03:17:38.207304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:06:36.818875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T03:17:39.176524+0000","last_change":"2024-09-12T03:17:38.230355+0000","last_active":"2024-09-12T03:17:39.176524+0000","last_peered":"2024-09-12T03:17:39.176524+0000","last_clean":"2024-09-12T03:17:39.176524+0000","last_became_active":"2024-09-12T03:17:38.230095+0000","last_became_peered":"2024-09-12T03:17:38.230095+0000","last_unstale":"2024-09-12T03:17:39.176524+0000","last_undegraded":"2024-09-12T03:17:39.176524+0000","last_fullsized":"2024-09-12T03:17:39.176524+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:37.205254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:37.205254+0000","last_clean_scrub_stamp":"2024-09-12T03:17:37.205254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:27:58.639003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T03:17:39.187682+0000","last_change":"2024-09-12T03:17:39.184187+0000","last_active":"2024-09-12T03:17:39.187682+0000","last_peered":"2024-09-12T03:17:39.187682+0000","last_clean":"2024-09-12T03:17:39.187682+0000","last_became_active":"2024-09-12T03:17:39.183923+0000","last_became_peered":"2024-09-12T03:17:39.183923+0000","last_unstale":"2024-09-12T03:17:39.187682+0000","last_undegraded":"2024-09-12T03:17:39.187682+0000","last_fullsized":"2024-09-12T03:17:39.187682+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_clean_scrub_stamp":"2024-09-12T03:17:38.207304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:32:55.415484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T03:17:39.167568+0000","last_change":"2024-09-12T03:17:38.228343+0000","last_active":"2024-09-12T03:17:39.167568+0000","last_peered":"2024-09-12T03:17:39.167568+0000","last_clean":"2024-09-12T03:17:39.167568+0000","last_became_active":"2024-09-12T03:17:38.228060+0000","last_became_peered":"2024-09-12T03:17:38.228060+0000","last_unstale":"2024-09-12T03:17:39.167568+0000","last_undegraded":"2024-09-12T03:17:39.167568+0000","last_fullsized":"2024-09-12T03:17:39.167568+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:37.205254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:37.205254+0000","last_clean_scrub_stamp":"2024-09-12T03:17:37.205254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:47:13.405144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T03:17:39.187245+0000","last_change":"2024-09-12T03:17:39.179998+0000","last_active":"2024-09-12T03:17:39.187245+0000","last_peered":"2024-09-12T03:17:39.187245+0000","last_clean":"2024-09-12T03:17:39.187245+0000","last_became_active":"2024-09-12T03:17:39.179175+0000","last_became_peered":"2024-09-12T03:17:39.179175+0000","last_unstale":"2024-09-12T03:17:39.187245+0000","last_undegraded":"2024-09-12T03:17:39.187245+0000","last_fullsized":"2024-09-12T03:17:39.187245+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_clean_scrub_stamp":"2024-09-12T03:17:38.207304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:48:50.171581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T03:17:39.167332+0000","last_change":"2024-09-12T03:17:38.223991+0000","last_active":"2024-09-12T03:17:39.167332+0000","last_peered":"2024-09-12T03:17:39.167332+0000","last_clean":"2024-09-12T03:17:39.167332+0000","last_became_active":"2024-09-12T03:17:38.223304+0000","last_became_peered":"2024-09-12T03:17:38.223304+0000","last_unstale":"2024-09-12T03:17:39.167332+0000","last_undegraded":"2024-09-12T03:17:39.167332+0000","last_fullsized":"2024-09-12T03:17:39.167332+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:37.205254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:37.205254+0000","last_clean_scrub_stamp":"2024-09-12T03:17:37.205254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:00:02.078057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T03:17:39.180867+0000","last_change":"2024-09-12T03:17:39.179669+0000","last_active":"2024-09-12T03:17:39.180867+0000","last_peered":"2024-09-12T03:17:39.180867+0000","last_clean":"2024-09-12T03:17:39.180867+0000","last_became_active":"2024-09-12T03:17:39.179360+0000","last_became_peered":"2024-09-12T03:17:39.179360+0000","last_unstale":"2024-09-12T03:17:39.180867+0000","last_undegraded":"2024-09-12T03:17:39.180867+0000","last_fullsized":"2024-09-12T03:17:39.180867+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_clean_scrub_stamp":"2024-09-12T03:17:38.207304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:50:38.156704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T03:17:39.175689+0000","last_change":"2024-09-12T03:17:38.228789+0000","last_active":"2024-09-12T03:17:39.175689+0000","last_peered":"2024-09-12T03:17:39.175689+0000","last_clean":"2024-09-12T03:17:39.175689+0000","last_became_active":"2024-09-12T03:17:38.228402+0000","last_became_peered":"2024-09-12T03:17:38.228402+0000","last_unstale":"2024-09-12T03:17:39.175689+0000","last_undegraded":"2024-09-12T03:17:39.175689+0000","last_fullsized":"2024-09-12T03:17:39.175689+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:37.205254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:37.205254+0000","last_clean_scrub_stamp":"2024-09-12T03:17:37.205254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:58:00.543783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T03:17:39.183628+0000","last_change":"2024-09-12T03:17:39.180384+0000","last_active":"2024-09-12T03:17:39.183628+0000","last_peered":"2024-09-12T03:17:39.183628+0000","last_clean":"2024-09-12T03:17:39.183628+0000","last_became_active":"2024-09-12T03:17:39.179269+0000","last_became_peered":"2024-09-12T03:17:39.179269+0000","last_unstale":"2024-09-12T03:17:39.183628+0000","last_undegraded":"2024-09-12T03:17:39.183628+0000","last_fullsized":"2024-09-12T03:17:39.183628+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_clean_scrub_stamp":"2024-09-12T03:17:38.207304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:25:36.609468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T03:17:39.173475+0000","last_change":"2024-09-12T03:17:38.232402+0000","last_active":"2024-09-12T03:17:39.173475+0000","last_peered":"2024-09-12T03:17:39.173475+0000","last_clean":"2024-09-12T03:17:39.173475+0000","last_became_active":"2024-09-12T03:17:38.232034+0000","last_became_peered":"2024-09-12T03:17:38.232034+0000","last_unstale":"2024-09-12T03:17:39.173475+0000","last_undegraded":"2024-09-12T03:17:39.173475+0000","last_fullsized":"2024-09-12T03:17:39.173475+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:37.205254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:37.205254+0000","last_clean_scrub_stamp":"2024-09-12T03:17:37.205254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:28:32.785457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T03:17:39.187856+0000","last_change":"2024-09-12T03:17:39.183225+0000","last_active":"2024-09-12T03:17:39.187856+0000","last_peered":"2024-09-12T03:17:39.187856+0000","last_clean":"2024-09-12T03:17:39.187856+0000","last_became_active":"2024-09-12T03:17:39.182734+0000","last_became_peered":"2024-09-12T03:17:39.182734+0000","last_unstale":"2024-09-12T03:17:39.187856+0000","last_undegraded":"2024-09-12T03:17:39.187856+0000","last_fullsized":"2024-09-12T03:17:39.187856+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:38.207304+0000","last_clean_scrub_stamp":"2024-09-12T03:17:38.207304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:42:37.751637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"42'2","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T03:17:39.223043+0000","last_change":"2024-09-12T03:17:38.231155+0000","last_active":"2024-09-12T03:17:39.223043+0000","last_peered":"2024-09-12T03:17:39.223043+0000","last_clean":"2024-09-12T03:17:39.223043+0000","last_became_active":"2024-09-12T03:17:38.230869+0000","last_became_peered":"2024-09-12T03:17:38.230869+0000","last_unstale":"2024-09-12T03:17:39.223043+0000","last_undegraded":"2024-09-12T03:17:39.223043+0000","last_fullsized":"2024-09-12T03:17:39.223043+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:37.205254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:37.205254+0000","last_clean_scrub_stamp":"2024-09-12T03:17:37.205254+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:54:31.074525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":206,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T03:17:39.166279+0000","last_change":"2024-09-12T03:16:21.883446+0000","last_active":"2024-09-12T03:17:39.166279+0000","last_peered":"2024-09-12T03:17:39.166279+0000","last_clean":"2024-09-12T03:17:39.166279+0000","last_became_active":"2024-09-12T03:16:21.882207+0000","last_became_peered":"2024-09-12T03:16:21.882207+0000","last_unstale":"2024-09-12T03:17:39.166279+0000","last_undegraded":"2024-09-12T03:17:39.166279+0000","last_fullsized":"2024-09-12T03:17:39.166279+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:20.316426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:20.316426+0000","last_clean_scrub_stamp":"2024-09-12T03:16:20.316426+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:12:15.424320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542162,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32248,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339592,"statfs":{"total":96636764160,"available":96603742208,"internally_reserved":0,"allocated":1085440,"data_stored":784705,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542164,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31672,"kb_used_data":484,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340168,"statfs":{"total":96636764160,"available":96604332032,"internally_reserved":0,"allocated":495616,"data_stored":194919,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542163,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35832,"kb_used_data":480,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336008,"statfs":{"total":96636764160,"available":96600072192,"internally_reserved":0,"allocated":491520,"data_stored":194317,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542162,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31676,"kb_used_data":488,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340164,"statfs":{"total":96636764160,"available":96604327936,"internally_reserved":0,"allocated":499712,"data_stored":195859,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542163,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27516,"kb_used_data":488,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344324,"statfs":{"total":96636764160,"available":96608587776,"internally_reserved":0,"allocated":499712,"data_stored":194885,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542163,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27520,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344320,"statfs":{"total":96636764160,"available":96608583680,"internally_reserved":0,"allocated":503808,"data_stored":195881,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542164,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27508,"kb_used_data":476,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344332,"statfs":{"total":96636764160,"available":96608595968,"internally_reserved":0,"allocated":487424,"data_stored":194247,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542164,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28092,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343748,"statfs":{"total":96636764160,"available":96607997952,"internally_reserved":0,"allocated":1085440,"data_stored":784649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T03:17:42.549 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-12T03:17:42.549 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-12T03:17:42.695 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:42.695+0000 7fb49470c640 1 Processor -- start 2024-09-12T03:17:42.695 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:42.695+0000 7fb49470c640 1 -- start start 2024-09-12T03:17:42.696 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:42.696+0000 7fb49470c640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fb48c0af9d0 0x7fb48c0b4010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:42.696 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:42.696+0000 7fb49470c640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb48c0ce5e0 con 0x7fb48c06da00 2024-09-12T03:17:42.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:42.696+0000 7fb49470c640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb48c0ceac0 con 0x7fb48c15ec30 2024-09-12T03:17:42.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:42.696+0000 7fb49470c640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fb48c0d02d0 con 0x7fb48c0af9d0 2024-09-12T03:17:42.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:42.696+0000 7fb492ca9640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fb48c06da00 0x7fb48c06de90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:40182/0 (socket says 172.21.15.27:40182) 2024-09-12T03:17:42.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:42.696+0000 7fb492ca9640 1 -- 172.21.15.27:0/1611488655 learned_addr learned my addr 172.21.15.27:0/1611488655 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:17:42.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:42.696+0000 7fb491ca7640 1 --2- 172.21.15.27:0/1611488655 >> v2:172.21.15.45:3300/0 conn(0x7fb48c0af9d0 0x7fb48c0b4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:42.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:42.696+0000 7fb4914a6640 1 -- 172.21.15.27:0/1611488655 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1701978403 0 0) 0x7fb48c0ce5e0 con 0x7fb48c06da00 2024-09-12T03:17:42.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:42.696+0000 7fb4914a6640 1 -- 172.21.15.27:0/1611488655 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb478005730 con 0x7fb48c06da00 2024-09-12T03:17:42.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:42.697+0000 7fb491ca7640 1 -- 172.21.15.27:0/1611488655 >> v1:172.21.15.45:6789/0 conn(0x7fb48c15ec30 legacy=0x7fb48c0b3900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:42.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:42.697+0000 7fb491ca7640 1 -- 172.21.15.27:0/1611488655 >> v1:172.21.15.27:6789/0 conn(0x7fb48c06da00 legacy=0x7fb48c06de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:42.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:42.697+0000 7fb491ca7640 1 -- 172.21.15.27:0/1611488655 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb48c15f010 con 0x7fb48c0af9d0 2024-09-12T03:17:42.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:42.697+0000 7fb491ca7640 1 --2- 172.21.15.27:0/1611488655 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb48c0af9d0 0x7fb48c0b4010 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fb47c0097a0 tx=0x7fb47c02a230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d06cc79be9996a2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:42.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:42.698+0000 7fb4914a6640 1 -- 172.21.15.27:0/1611488655 <== mon.1 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb47c037070 con 0x7fb48c0af9d0 2024-09-12T03:17:42.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:42.698+0000 7fb4914a6640 1 -- 172.21.15.27:0/1611488655 <== mon.1 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb47c032440 con 0x7fb48c0af9d0 2024-09-12T03:17:42.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:42.698+0000 7fb4914a6640 1 -- 172.21.15.27:0/1611488655 <== mon.1 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb47c0327f0 con 0x7fb48c0af9d0 2024-09-12T03:17:42.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:42.698+0000 7fb49470c640 1 -- 172.21.15.27:0/1611488655 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb48c0af9d0 msgr2=0x7fb48c0b4010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:42.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:42.698+0000 7fb49470c640 1 --2- 172.21.15.27:0/1611488655 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb48c0af9d0 0x7fb48c0b4010 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fb47c0097a0 tx=0x7fb47c02a230 comp rx=0 tx=0).stop 2024-09-12T03:17:42.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:42.698+0000 7fb49470c640 1 -- 172.21.15.27:0/1611488655 shutdown_connections 2024-09-12T03:17:42.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:42.698+0000 7fb49470c640 1 --2- 172.21.15.27:0/1611488655 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb48c0af9d0 0x7fb48c0b4010 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:42.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:42.698+0000 7fb49470c640 1 -- 172.21.15.27:0/1611488655 >> 172.21.15.27:0/1611488655 conn(0x7fb48c06bb30 msgr2=0x7fb48c06bf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:42.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:42.698+0000 7fb49470c640 1 -- 172.21.15.27:0/1611488655 shutdown_connections 2024-09-12T03:17:42.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:42.698+0000 7fb49470c640 1 -- 172.21.15.27:0/1611488655 wait complete. 2024-09-12T03:17:42.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:42.700+0000 7fb49470c640 1 Processor -- start 2024-09-12T03:17:42.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:42.700+0000 7fb49470c640 1 -- start start 2024-09-12T03:17:42.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:42.700+0000 7fb49470c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb48c06da00 0x7fb48c1417a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:42.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:42.700+0000 7fb49470c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb48c0af9d0 0x7fb48c0cbeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:42.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:42.700+0000 7fb49470c640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb48c15ec30 0x7fb48c13fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:42.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:42.700+0000 7fb49470c640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fb48c0cfff0 con 0x7fb48c0af9d0 2024-09-12T03:17:42.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:42.700+0000 7fb49470c640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb48c06c650 con 0x7fb48c06da00 2024-09-12T03:17:42.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:42.700+0000 7fb49470c640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fb48c0715f0 con 0x7fb48c15ec30 2024-09-12T03:17:42.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:42.700+0000 7fb491ca7640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb48c0af9d0 0x7fb48c0cbeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:42.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:42.700+0000 7fb491ca7640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb48c0af9d0 0x7fb48c0cbeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60382/0 (socket says 172.21.15.27:60382) 2024-09-12T03:17:42.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:42.700+0000 7fb491ca7640 1 -- 172.21.15.27:0/135843813 learned_addr learned my addr 172.21.15.27:0/135843813 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:17:42.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:42.701+0000 7fb4924a8640 1 --2- 172.21.15.27:0/135843813 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb48c06da00 0x7fb48c1417a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:42.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:42.701+0000 7fb492ca9640 1 --2- 172.21.15.27:0/135843813 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb48c15ec30 0x7fb48c13fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:42.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:42.701+0000 7fb491ca7640 1 -- 172.21.15.27:0/135843813 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb48c15ec30 msgr2=0x7fb48c13fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:42.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:42.701+0000 7fb491ca7640 1 --2- 172.21.15.27:0/135843813 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb48c15ec30 0x7fb48c13fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:42.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:42.701+0000 7fb491ca7640 1 -- 172.21.15.27:0/135843813 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb48c06da00 msgr2=0x7fb48c1417a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:42.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:42.701+0000 7fb491ca7640 1 --2- 172.21.15.27:0/135843813 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb48c06da00 0x7fb48c1417a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:42.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:42.701+0000 7fb491ca7640 1 -- 172.21.15.27:0/135843813 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb48c140700 con 0x7fb48c0af9d0 2024-09-12T03:17:42.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:42.701+0000 7fb491ca7640 1 --2- 172.21.15.27:0/135843813 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb48c0af9d0 0x7fb48c0cbeb0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fb47c035040 tx=0x7fb47c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:42.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:42.701+0000 7fb482ffd640 1 -- 172.21.15.27:0/135843813 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb47c042020 con 0x7fb48c0af9d0 2024-09-12T03:17:42.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:42.701+0000 7fb482ffd640 1 -- 172.21.15.27:0/135843813 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb47c003c40 con 0x7fb48c0af9d0 2024-09-12T03:17:42.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:42.701+0000 7fb482ffd640 1 -- 172.21.15.27:0/135843813 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb47c037040 con 0x7fb48c0af9d0 2024-09-12T03:17:42.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:42.701+0000 7fb49470c640 1 -- 172.21.15.27:0/135843813 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb48c1409e0 con 0x7fb48c0af9d0 2024-09-12T03:17:42.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:42.701+0000 7fb49470c640 1 -- 172.21.15.27:0/135843813 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb48c140fa0 con 0x7fb48c0af9d0 2024-09-12T03:17:42.707 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:42.702+0000 7fb49470c640 1 -- 172.21.15.27:0/135843813 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb48c06de90 con 0x7fb48c0af9d0 2024-09-12T03:17:42.707 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:42.703+0000 7fb482ffd640 1 -- 172.21.15.27:0/135843813 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 7) ==== 90651+0+0 (secure 0 0 0) 0x7fb47c045020 con 0x7fb48c0af9d0 2024-09-12T03:17:42.707 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:42.703+0000 7fb482ffd640 1 --2- 172.21.15.27:0/135843813 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fb47006bfe0 0x7fb47006e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:42.707 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:42.703+0000 7fb482ffd640 1 -- 172.21.15.27:0/135843813 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5709+0+0 (secure 0 0 0) 0x7fb47c0ab990 con 0x7fb48c0af9d0 2024-09-12T03:17:42.707 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:42.704+0000 7fb4924a8640 1 --2- 172.21.15.27:0/135843813 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fb47006bfe0 0x7fb47006e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:42.707 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:42.704+0000 7fb4924a8640 1 --2- 172.21.15.27:0/135843813 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fb47006bfe0 0x7fb47006e4a0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fb488000f60 tx=0x7fb488009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:42.708 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:42.708+0000 7fb482ffd640 1 -- 172.21.15.27:0/135843813 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fb47c04c030 con 0x7fb48c0af9d0 2024-09-12T03:17:42.950 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:42.950+0000 7fb49470c640 1 -- 172.21.15.27:0/135843813 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7fb48c06f240 con 0x7fb48c0af9d0 2024-09-12T03:17:42.951 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:42.952+0000 7fb482ffd640 1 -- 172.21.15.27:0/135843813 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v12) ==== 190+0+147 (secure 0 0 0) 0x7fb47c079a70 con 0x7fb48c0af9d0 2024-09-12T03:17:42.952 INFO:teuthology.orchestra.run.smithi027.stdout:[client.0] 2024-09-12T03:17:42.952 INFO:teuthology.orchestra.run.smithi027.stdout: key = AQDVXOJmLm+5ExAAECM4CKfuVElmp/n0b02mXg== 2024-09-12T03:17:42.952 INFO:teuthology.orchestra.run.smithi027.stdout: caps mds = "allow" 2024-09-12T03:17:42.952 INFO:teuthology.orchestra.run.smithi027.stdout: caps mon = "allow r" 2024-09-12T03:17:42.952 INFO:teuthology.orchestra.run.smithi027.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-12T03:17:42.952 INFO:teuthology.orchestra.run.smithi027.stderr:updated caps for client.0 2024-09-12T03:17:42.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:42.954+0000 7fb49470c640 1 -- 172.21.15.27:0/135843813 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fb47006bfe0 msgr2=0x7fb47006e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:42.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:42.954+0000 7fb49470c640 1 --2- 172.21.15.27:0/135843813 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fb47006bfe0 0x7fb47006e4a0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fb488000f60 tx=0x7fb488009040 comp rx=0 tx=0).stop 2024-09-12T03:17:42.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:42.954+0000 7fb49470c640 1 -- 172.21.15.27:0/135843813 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb48c0af9d0 msgr2=0x7fb48c0cbeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:42.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:42.954+0000 7fb49470c640 1 --2- 172.21.15.27:0/135843813 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb48c0af9d0 0x7fb48c0cbeb0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fb47c035040 tx=0x7fb47c0046c0 comp rx=0 tx=0).stop 2024-09-12T03:17:42.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:42.954+0000 7fb49470c640 1 -- 172.21.15.27:0/135843813 shutdown_connections 2024-09-12T03:17:42.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:42.954+0000 7fb49470c640 1 --2- 172.21.15.27:0/135843813 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fb47006bfe0 0x7fb47006e4a0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:42.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:42.954+0000 7fb49470c640 1 --2- 172.21.15.27:0/135843813 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb48c15ec30 0x7fb48c13fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:42.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:42.954+0000 7fb49470c640 1 --2- 172.21.15.27:0/135843813 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb48c0af9d0 0x7fb48c0cbeb0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:42.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:42.954+0000 7fb49470c640 1 --2- 172.21.15.27:0/135843813 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb48c06da00 0x7fb48c1417a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:42.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:42.954+0000 7fb49470c640 1 -- 172.21.15.27:0/135843813 >> 172.21.15.27:0/135843813 conn(0x7fb48c06bb30 msgr2=0x7fb48c0b35f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:42.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:42.954+0000 7fb49470c640 1 -- 172.21.15.27:0/135843813 shutdown_connections 2024-09-12T03:17:42.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:42.954+0000 7fb49470c640 1 -- 172.21.15.27:0/135843813 wait complete. 2024-09-12T03:17:42.965 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-12T03:17:43.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.067+0000 7f468fd18640 1 Processor -- start 2024-09-12T03:17:43.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.067+0000 7f468fd18640 1 -- start start 2024-09-12T03:17:43.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.068+0000 7f468fd18640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f468815eb60 0x7f46880d1280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:43.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.068+0000 7f468fd18640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f46880d17c0 0x7f46880b3390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:43.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.068+0000 7f468fd18640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f46880b6690 con 0x7f468815e780 2024-09-12T03:17:43.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.068+0000 7f468fd18640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f46880d1c30 con 0x7f468815eb60 2024-09-12T03:17:43.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.068+0000 7f468fd18640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f46880b7950 con 0x7f46880d17c0 2024-09-12T03:17:43.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.068+0000 7f468dab4640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f468815e780 0x7f46880d3e70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:40194/0 (socket says 172.21.15.27:40194) 2024-09-12T03:17:43.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.068+0000 7f468dab4640 1 -- 172.21.15.27:0/2192753668 learned_addr learned my addr 172.21.15.27:0/2192753668 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:17:43.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.068+0000 7f468d2b3640 1 --2- 172.21.15.27:0/2192753668 >> v2:172.21.15.45:3300/0 conn(0x7f468815eb60 0x7f46880d1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:43.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.068+0000 7f468e2b5640 1 --2- 172.21.15.27:0/2192753668 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f46880d17c0 0x7f46880b3390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:43.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.068+0000 7f468cab2640 1 -- 172.21.15.27:0/2192753668 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1777269941 0 0) 0x7f46880b6690 con 0x7f468815e780 2024-09-12T03:17:43.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.068+0000 7f468cab2640 1 -- 172.21.15.27:0/2192753668 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4674003520 con 0x7f468815e780 2024-09-12T03:17:43.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.068+0000 7f468cab2640 1 -- 172.21.15.27:0/2192753668 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1036815080 0 0) 0x7f4674003520 con 0x7f468815e780 2024-09-12T03:17:43.072 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.068+0000 7f468cab2640 1 -- 172.21.15.27:0/2192753668 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f46880d17c0 msgr2=0x7f46880b3390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:43.072 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.068+0000 7f468cab2640 1 --2- 172.21.15.27:0/2192753668 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f46880d17c0 0x7f46880b3390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:43.072 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.068+0000 7f468cab2640 1 -- 172.21.15.27:0/2192753668 >> v2:172.21.15.45:3300/0 conn(0x7f468815eb60 msgr2=0x7f46880d1280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:43.072 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.068+0000 7f468cab2640 1 --2- 172.21.15.27:0/2192753668 >> v2:172.21.15.45:3300/0 conn(0x7f468815eb60 0x7f46880d1280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:43.072 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.069+0000 7f468cab2640 1 -- 172.21.15.27:0/2192753668 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46880b6970 con 0x7f468815e780 2024-09-12T03:17:43.072 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.069+0000 7f468cab2640 1 -- 172.21.15.27:0/2192753668 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 847755213 0 0) 0x7f4684001670 con 0x7f468815e780 2024-09-12T03:17:43.072 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.069+0000 7f468d2b3640 1 --2- 172.21.15.27:0/2192753668 >> v2:172.21.15.45:3300/0 conn(0x7f468815eb60 0x7f46880d1280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:17:43.072 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.069+0000 7f468cab2640 1 -- 172.21.15.27:0/2192753668 >> v1:172.21.15.27:6789/0 conn(0x7f468815e780 legacy=0x7f46880d3e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:43.072 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.069+0000 7f468cab2640 1 --2- 172.21.15.27:0/2192753668 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f46740039e0 0x7f4674003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:43.072 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.069+0000 7f468cab2640 1 --2- 172.21.15.27:0/2192753668 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f46880d17c0 0x7f4674004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:43.072 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.069+0000 7f468cab2640 1 --2- 172.21.15.27:0/2192753668 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4674004d40 0x7f46740051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:43.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.069+0000 7f468cab2640 1 -- 172.21.15.27:0/2192753668 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f46880d1c30 con 0x7f4674004d40 2024-09-12T03:17:43.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.069+0000 7f468cab2640 1 -- 172.21.15.27:0/2192753668 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f467400b9c0 con 0x7f46740039e0 2024-09-12T03:17:43.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.069+0000 7f468cab2640 1 -- 172.21.15.27:0/2192753668 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f467400bb40 con 0x7f46880d17c0 2024-09-12T03:17:43.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.069+0000 7f468e2b5640 1 --2- 172.21.15.27:0/2192753668 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4674004d40 0x7f46740051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:43.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.069+0000 7f468dab4640 1 --2- 172.21.15.27:0/2192753668 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f46740039e0 0x7f4674003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:43.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.069+0000 7f468d2b3640 1 --2- 172.21.15.27:0/2192753668 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f46880d17c0 0x7f4674004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:43.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.069+0000 7f468e2b5640 1 -- 172.21.15.27:0/2192753668 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f46880d17c0 msgr2=0x7f4674004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:43.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.069+0000 7f468e2b5640 1 --2- 172.21.15.27:0/2192753668 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f46880d17c0 0x7f4674004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:43.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.069+0000 7f468e2b5640 1 -- 172.21.15.27:0/2192753668 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f46740039e0 msgr2=0x7f4674003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:43.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.069+0000 7f468e2b5640 1 --2- 172.21.15.27:0/2192753668 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f46740039e0 0x7f4674003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:43.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.069+0000 7f468e2b5640 1 -- 172.21.15.27:0/2192753668 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4674003520 con 0x7f4674004d40 2024-09-12T03:17:43.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.070+0000 7f468e2b5640 1 --2- 172.21.15.27:0/2192753668 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4674004d40 0x7f46740051c0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f468000e170 tx=0x7f4674005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6131b65888b23b9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:43.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.070+0000 7f468cab2640 1 -- 172.21.15.27:0/2192753668 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4680034020 con 0x7f4674004d40 2024-09-12T03:17:43.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.070+0000 7f468cab2640 1 -- 172.21.15.27:0/2192753668 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46800305f0 con 0x7f4674004d40 2024-09-12T03:17:43.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.070+0000 7f468cab2640 1 -- 172.21.15.27:0/2192753668 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4680009c10 con 0x7f4674004d40 2024-09-12T03:17:43.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.070+0000 7f468fd18640 1 -- 172.21.15.27:0/2192753668 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4674004d40 msgr2=0x7f46740051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:43.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.070+0000 7f468fd18640 1 --2- 172.21.15.27:0/2192753668 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4674004d40 0x7f46740051c0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f468000e170 tx=0x7f4674005830 comp rx=0 tx=0).stop 2024-09-12T03:17:43.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.070+0000 7f468e2b5640 1 -- 172.21.15.27:0/2192753668 reap_dead start 2024-09-12T03:17:43.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.071+0000 7f468fd18640 1 -- 172.21.15.27:0/2192753668 shutdown_connections 2024-09-12T03:17:43.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.071+0000 7f468fd18640 1 -- 172.21.15.27:0/2192753668 >> 172.21.15.27:0/2192753668 conn(0x7f468806da60 msgr2=0x7f468806de70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:43.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.071+0000 7f468fd18640 1 -- 172.21.15.27:0/2192753668 shutdown_connections 2024-09-12T03:17:43.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.071+0000 7f468fd18640 1 -- 172.21.15.27:0/2192753668 wait complete. 2024-09-12T03:17:43.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.072+0000 7f468fd18640 1 Processor -- start 2024-09-12T03:17:43.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.072+0000 7f468fd18640 1 -- start start 2024-09-12T03:17:43.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.073+0000 7f468fd18640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f468814dcf0 0x7f46881e9a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:43.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.073+0000 7f468fd18640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f46881e9fb0 0x7f468815bc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:43.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.073+0000 7f468fd18640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f468815c140 0x7f4688160cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:43.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.073+0000 7f468fd18640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f46880b6f30 con 0x7f468814dcf0 2024-09-12T03:17:43.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.073+0000 7f468fd18640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f46880b7210 con 0x7f46881e9fb0 2024-09-12T03:17:43.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.073+0000 7f468fd18640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f46880b6340 con 0x7f468815c140 2024-09-12T03:17:43.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.073+0000 7f468dab4640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f468814dcf0 0x7f46881e9a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:43.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.073+0000 7f468dab4640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f468814dcf0 0x7f46881e9a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60392/0 (socket says 172.21.15.27:60392) 2024-09-12T03:17:43.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.073+0000 7f468dab4640 1 -- 172.21.15.27:0/1217665243 learned_addr learned my addr 172.21.15.27:0/1217665243 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:17:43.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.073+0000 7f468d2b3640 1 --2- 172.21.15.27:0/1217665243 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f46881e9fb0 0x7f468815bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:43.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.073+0000 7f468e2b5640 1 --2- 172.21.15.27:0/1217665243 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f468815c140 0x7f4688160cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:43.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.073+0000 7f468dab4640 1 -- 172.21.15.27:0/1217665243 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f468815c140 msgr2=0x7f4688160cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:43.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.073+0000 7f468dab4640 1 --2- 172.21.15.27:0/1217665243 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f468815c140 0x7f4688160cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:43.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.073+0000 7f468dab4640 1 -- 172.21.15.27:0/1217665243 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f46881e9fb0 msgr2=0x7f468815bc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:43.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.073+0000 7f468dab4640 1 --2- 172.21.15.27:0/1217665243 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f46881e9fb0 0x7f468815bc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:43.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.073+0000 7f468dab4640 1 -- 172.21.15.27:0/1217665243 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46881e76d0 con 0x7f468814dcf0 2024-09-12T03:17:43.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.073+0000 7f468dab4640 1 --2- 172.21.15.27:0/1217665243 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f468814dcf0 0x7f46881e9a70 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f46840070c0 tx=0x7f4684007580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:43.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.073+0000 7f467e7fc640 1 -- 172.21.15.27:0/1217665243 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f468401a070 con 0x7f468814dcf0 2024-09-12T03:17:43.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.073+0000 7f467e7fc640 1 -- 172.21.15.27:0/1217665243 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f468400f9c0 con 0x7f468814dcf0 2024-09-12T03:17:43.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.073+0000 7f468fd18640 1 -- 172.21.15.27:0/1217665243 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46881e7110 con 0x7f468814dcf0 2024-09-12T03:17:43.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.073+0000 7f467e7fc640 1 -- 172.21.15.27:0/1217665243 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f468400fdb0 con 0x7f468814dcf0 2024-09-12T03:17:43.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.073+0000 7f468fd18640 1 -- 172.21.15.27:0/1217665243 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46880cd2b0 con 0x7f468814dcf0 2024-09-12T03:17:43.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.074+0000 7f468fd18640 1 -- 172.21.15.27:0/1217665243 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46880d1280 con 0x7f468814dcf0 2024-09-12T03:17:43.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.075+0000 7f467e7fc640 1 -- 172.21.15.27:0/1217665243 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 7) ==== 90651+0+0 (secure 0 0 0) 0x7f468401d070 con 0x7f468814dcf0 2024-09-12T03:17:43.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.075+0000 7f467e7fc640 1 --2- 172.21.15.27:0/1217665243 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f466c06c010 0x7f466c06e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:43.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.076+0000 7f467e7fc640 1 -- 172.21.15.27:0/1217665243 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5709+0+0 (secure 0 0 0) 0x7f468408ab50 con 0x7f468814dcf0 2024-09-12T03:17:43.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.076+0000 7f468d2b3640 1 --2- 172.21.15.27:0/1217665243 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f466c06c010 0x7f466c06e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:43.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.076+0000 7f468d2b3640 1 --2- 172.21.15.27:0/1217665243 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f466c06c010 0x7f466c06e4d0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f46881f7890 tx=0x7f4678008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:43.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.080+0000 7f467e7fc640 1 -- 172.21.15.27:0/1217665243 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f4684059280 con 0x7f468814dcf0 2024-09-12T03:17:43.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.292+0000 7f468fd18640 1 -- 172.21.15.27:0/1217665243 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4688068cc0 con 0x7f468814dcf0 2024-09-12T03:17:43.293 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.293+0000 7f467e7fc640 1 -- 172.21.15.27:0/1217665243 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) ==== 94+0+5223 (secure 0 0 0) 0x7f4684058b80 con 0x7f468814dcf0 2024-09-12T03:17:43.293 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:17:43.293 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":22,"btime":"2024-09-12T03:17:41:216741+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4835,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6834/1457797481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":1457797481},{"type":"v1","addr":"172.21.15.27:6834","nonce":1457797481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5029,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6836/2842491281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":2842491281},{"type":"v1","addr":"172.21.15.45:6836","nonce":2842491281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5032,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/3339744340","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6835","nonce":3339744340},{"type":"v1","addr":"172.21.15.45:6837","nonce":3339744340}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-12T03:17:38.580473+0000","modified":"2024-09-12T03:17:40.557950+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5205},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5205":{"gid":5205,"name":"a","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.27:6835/2571101228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":2571101228},{"type":"v1","addr":"172.21.15.27:6835","nonce":2571101228}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5205,"qdb_cluster":[5205]},"id":3}]} 2024-09-12T03:17:43.293 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 22 2024-09-12T03:17:43.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.295+0000 7f468fd18640 1 -- 172.21.15.27:0/1217665243 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f466c06c010 msgr2=0x7f466c06e4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:43.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.295+0000 7f468fd18640 1 --2- 172.21.15.27:0/1217665243 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f466c06c010 0x7f466c06e4d0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f46881f7890 tx=0x7f4678008040 comp rx=0 tx=0).stop 2024-09-12T03:17:43.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.295+0000 7f468fd18640 1 -- 172.21.15.27:0/1217665243 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f468814dcf0 msgr2=0x7f46881e9a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:43.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.295+0000 7f468fd18640 1 --2- 172.21.15.27:0/1217665243 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f468814dcf0 0x7f46881e9a70 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f46840070c0 tx=0x7f4684007580 comp rx=0 tx=0).stop 2024-09-12T03:17:43.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.295+0000 7f468fd18640 1 -- 172.21.15.27:0/1217665243 shutdown_connections 2024-09-12T03:17:43.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.295+0000 7f468fd18640 1 --2- 172.21.15.27:0/1217665243 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f466c06c010 0x7f466c06e4d0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:43.296 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.295+0000 7f468fd18640 1 --2- 172.21.15.27:0/1217665243 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f468815c140 0x7f4688160cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:43.296 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.295+0000 7f468fd18640 1 --2- 172.21.15.27:0/1217665243 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f46881e9fb0 0x7f468815bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:43.296 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.295+0000 7f468fd18640 1 --2- 172.21.15.27:0/1217665243 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f468814dcf0 0x7f46881e9a70 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:43.296 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.295+0000 7f468fd18640 1 -- 172.21.15.27:0/1217665243 >> 172.21.15.27:0/1217665243 conn(0x7f468806da60 msgr2=0x7f46880d4410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:43.296 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.295+0000 7f468fd18640 1 -- 172.21.15.27:0/1217665243 shutdown_connections 2024-09-12T03:17:43.296 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.295+0000 7f468fd18640 1 -- 172.21.15.27:0/1217665243 wait complete. 2024-09-12T03:17:43.708 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 22, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-12T03:17:38.580473+0000', 'modified': '2024-09-12T03:17:40.557950+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5205}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5205': {'gid': 5205, 'name': 'a', 'rank': 0, 'incarnation': 20, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.27:6835/2571101228', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6833', 'nonce': 2571101228}, {'type': 'v1', 'addr': '172.21.15.27:6835', 'nonce': 2571101228}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [7], 'metadata_pool': 6, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5205, 'qdb_cluster': [5205]} 2024-09-12T03:17:43.708 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-12T03:17:43.709 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-09-12T03:17:43.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.816+0000 7f64a7682640 1 Processor -- start 2024-09-12T03:17:43.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.816+0000 7f64a7682640 1 -- start start 2024-09-12T03:17:43.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.817+0000 7f64a7682640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f64a0158cf0 0x7f64a00b55d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:43.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.817+0000 7f64a7682640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f64a01590d0 0x7f64a0162430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:43.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.817+0000 7f64a7682640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f64a015f600 0x7f64a006bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:43.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.817+0000 7f64a7682640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f64a0162970 con 0x7f64a015f600 2024-09-12T03:17:43.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.817+0000 7f64a7682640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f64a0162c10 con 0x7f64a0158cf0 2024-09-12T03:17:43.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.817+0000 7f64a7682640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f64a0162ef0 con 0x7f64a01590d0 2024-09-12T03:17:43.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.817+0000 7f64a5c1f640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f64a015f600 0x7f64a006bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:43.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.817+0000 7f64a5c1f640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f64a015f600 0x7f64a006bb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60404/0 (socket says 172.21.15.27:60404) 2024-09-12T03:17:43.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.817+0000 7f64a5c1f640 1 -- 172.21.15.27:0/699243111 learned_addr learned my addr 172.21.15.27:0/699243111 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:17:43.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.817+0000 7f64a541e640 1 --2- 172.21.15.27:0/699243111 >> v2:172.21.15.45:3300/0 conn(0x7f64a0158cf0 0x7f64a00b55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:43.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.817+0000 7f64a4c1d640 1 --2- 172.21.15.27:0/699243111 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f64a01590d0 0x7f64a0162430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:43.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.817+0000 7f64a5c1f640 1 -- 172.21.15.27:0/699243111 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f64a01590d0 msgr2=0x7f64a0162430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:43.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.817+0000 7f64a5c1f640 1 --2- 172.21.15.27:0/699243111 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f64a01590d0 0x7f64a0162430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:43.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.817+0000 7f64a5c1f640 1 -- 172.21.15.27:0/699243111 >> v2:172.21.15.45:3300/0 conn(0x7f64a0158cf0 msgr2=0x7f64a00b55d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:43.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.817+0000 7f64a5c1f640 1 --2- 172.21.15.27:0/699243111 >> v2:172.21.15.45:3300/0 conn(0x7f64a0158cf0 0x7f64a00b55d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:43.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.817+0000 7f64a5c1f640 1 -- 172.21.15.27:0/699243111 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64a01631d0 con 0x7f64a015f600 2024-09-12T03:17:43.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.818+0000 7f64a5c1f640 1 --2- 172.21.15.27:0/699243111 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f64a015f600 0x7f64a006bb30 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f64a00733c0 tx=0x7f649002a570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1149c3e13e8f5712 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:43.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.818+0000 7f649ffff640 1 -- 172.21.15.27:0/699243111 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6490036070 con 0x7f64a015f600 2024-09-12T03:17:43.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.818+0000 7f649ffff640 1 -- 172.21.15.27:0/699243111 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6490031600 con 0x7f64a015f600 2024-09-12T03:17:43.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.818+0000 7f649ffff640 1 -- 172.21.15.27:0/699243111 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64900319d0 con 0x7f64a015f600 2024-09-12T03:17:43.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.819+0000 7f64a7682640 1 -- 172.21.15.27:0/699243111 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f64a015f600 msgr2=0x7f64a006bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:43.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.819+0000 7f64a7682640 1 --2- 172.21.15.27:0/699243111 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f64a015f600 0x7f64a006bb30 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f64a00733c0 tx=0x7f649002a570 comp rx=0 tx=0).stop 2024-09-12T03:17:43.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.819+0000 7f64a7682640 1 -- 172.21.15.27:0/699243111 shutdown_connections 2024-09-12T03:17:43.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.819+0000 7f64a7682640 1 --2- 172.21.15.27:0/699243111 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f64a015f600 0x7f64a006bb30 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:43.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.819+0000 7f64a7682640 1 --2- 172.21.15.27:0/699243111 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f64a01590d0 0x7f64a0162430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:43.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.819+0000 7f64a7682640 1 --2- 172.21.15.27:0/699243111 >> v2:172.21.15.45:3300/0 conn(0x7f64a0158cf0 0x7f64a00b55d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:43.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.819+0000 7f64a7682640 1 -- 172.21.15.27:0/699243111 >> 172.21.15.27:0/699243111 conn(0x7f64a006cc40 msgr2=0x7f64a006d050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:43.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.819+0000 7f64a7682640 1 -- 172.21.15.27:0/699243111 shutdown_connections 2024-09-12T03:17:43.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.819+0000 7f64a7682640 1 -- 172.21.15.27:0/699243111 wait complete. 2024-09-12T03:17:43.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.820+0000 7f64a7682640 1 Processor -- start 2024-09-12T03:17:43.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.820+0000 7f64a7682640 1 -- start start 2024-09-12T03:17:43.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.821+0000 7f64a7682640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f64a0158cf0 0x7f64a0145800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:43.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.821+0000 7f64a7682640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f64a01590d0 0x7f64a0146150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:43.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.821+0000 7f64a7682640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f64a015f600 0x7f64a0146690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:43.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.821+0000 7f64a7682640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f64a015fbd0 con 0x7f64a0158cf0 2024-09-12T03:17:43.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.821+0000 7f64a7682640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f64a015fa50 con 0x7f64a01590d0 2024-09-12T03:17:43.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.821+0000 7f64a7682640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f64a015fd50 con 0x7f64a015f600 2024-09-12T03:17:43.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.821+0000 7f64a541e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f64a0158cf0 0x7f64a0145800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:43.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.821+0000 7f64a541e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f64a0158cf0 0x7f64a0145800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60414/0 (socket says 172.21.15.27:60414) 2024-09-12T03:17:43.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.821+0000 7f64a541e640 1 -- 172.21.15.27:0/3058461387 learned_addr learned my addr 172.21.15.27:0/3058461387 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:17:43.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.821+0000 7f64a4c1d640 1 --2- 172.21.15.27:0/3058461387 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f64a01590d0 0x7f64a0146150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:43.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.821+0000 7f64a5c1f640 1 --2- 172.21.15.27:0/3058461387 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f64a015f600 0x7f64a0146690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:43.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.821+0000 7f64a541e640 1 -- 172.21.15.27:0/3058461387 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f64a015f600 msgr2=0x7f64a0146690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:43.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.821+0000 7f64a541e640 1 --2- 172.21.15.27:0/3058461387 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f64a015f600 0x7f64a0146690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:43.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.821+0000 7f64a541e640 1 -- 172.21.15.27:0/3058461387 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f64a01590d0 msgr2=0x7f64a0146150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:43.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.821+0000 7f64a541e640 1 --2- 172.21.15.27:0/3058461387 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f64a01590d0 0x7f64a0146150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:43.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.821+0000 7f64a541e640 1 -- 172.21.15.27:0/3058461387 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64a015d020 con 0x7f64a0158cf0 2024-09-12T03:17:43.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.821+0000 7f64a5c1f640 1 --2- 172.21.15.27:0/3058461387 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f64a015f600 0x7f64a0146690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:17:43.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.822+0000 7f64a541e640 1 --2- 172.21.15.27:0/3058461387 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f64a0158cf0 0x7f64a0145800 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f649800e870 tx=0x7f649800ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:43.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.822+0000 7f649dffb640 1 -- 172.21.15.27:0/3058461387 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6498017070 con 0x7f64a0158cf0 2024-09-12T03:17:43.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.822+0000 7f649dffb640 1 -- 172.21.15.27:0/3058461387 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f649800ee60 con 0x7f64a0158cf0 2024-09-12T03:17:43.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.822+0000 7f64a7682640 1 -- 172.21.15.27:0/3058461387 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64a015d300 con 0x7f64a0158cf0 2024-09-12T03:17:43.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.822+0000 7f649dffb640 1 -- 172.21.15.27:0/3058461387 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f649800cb80 con 0x7f64a0158cf0 2024-09-12T03:17:43.826 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.822+0000 7f64a7682640 1 -- 172.21.15.27:0/3058461387 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64a015d8c0 con 0x7f64a0158cf0 2024-09-12T03:17:43.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.824+0000 7f649dffb640 1 -- 172.21.15.27:0/3058461387 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 7) ==== 90651+0+0 (secure 0 0 0) 0x7f649801a070 con 0x7f64a0158cf0 2024-09-12T03:17:43.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.824+0000 7f649dffb640 1 --2- 172.21.15.27:0/3058461387 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f648c06bfe0 0x7f648c06e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:43.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.824+0000 7f649dffb640 1 -- 172.21.15.27:0/3058461387 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5709+0+0 (secure 0 0 0) 0x7f649808b2d0 con 0x7f64a0158cf0 2024-09-12T03:17:43.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.824+0000 7f64a4c1d640 1 --2- 172.21.15.27:0/3058461387 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f648c06bfe0 0x7f648c06e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:43.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.825+0000 7f64aa422740 1 Processor -- start 2024-09-12T03:17:43.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.825+0000 7f64aa422740 1 -- start start 2024-09-12T03:17:43.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.825+0000 7f64a4c1d640 1 --2- 172.21.15.27:0/3058461387 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f648c06bfe0 0x7f648c06e4a0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f64a006fdc0 tx=0x7f6494009190 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:43.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.825+0000 7f64aa422740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562a9ad202c0 0x562a9ad65a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:43.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.825+0000 7f64aa422740 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x562a9ad1da80 0x562a9ad206a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:43.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.825+0000 7f64aa422740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562a9ad21a50 0x562a9ad1a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:43.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.825+0000 7f64aa422740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x562a9ac82520 con 0x562a9ad21a50 2024-09-12T03:17:43.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.825+0000 7f64aa422740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x562a9ac2a3e0 con 0x562a9ad202c0 2024-09-12T03:17:43.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.825+0000 7f64aa422740 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x562a9ac7e4e0 con 0x562a9ad1da80 2024-09-12T03:17:43.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.826+0000 7f64a5c1f640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562a9ad21a50 0x562a9ad1a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:43.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.826+0000 7f64a541e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562a9ad202c0 0x562a9ad65a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:43.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.826+0000 7f64a5c1f640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562a9ad21a50 0x562a9ad1a8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60416/0 (socket says 172.21.15.27:60416) 2024-09-12T03:17:43.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.826+0000 7f64a5c1f640 1 -- 172.21.15.27:0/2634081225 learned_addr learned my addr 172.21.15.27:0/2634081225 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:17:43.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.826+0000 7f64a4c1d640 1 --2- 172.21.15.27:0/2634081225 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x562a9ad1da80 0x562a9ad206a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:43.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.826+0000 7f64a541e640 1 -- 172.21.15.27:0/2634081225 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x562a9ad1da80 msgr2=0x562a9ad206a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:43.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.826+0000 7f64a541e640 1 --2- 172.21.15.27:0/2634081225 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x562a9ad1da80 0x562a9ad206a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:43.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.826+0000 7f64a541e640 1 -- 172.21.15.27:0/2634081225 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562a9ad21a50 msgr2=0x562a9ad1a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:43.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.826+0000 7f64a541e640 1 --2- 172.21.15.27:0/2634081225 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562a9ad21a50 0x562a9ad1a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:43.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.826+0000 7f64a541e640 1 -- 172.21.15.27:0/2634081225 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562a9ac452f0 con 0x562a9ad202c0 2024-09-12T03:17:43.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.826+0000 7f64a5c1f640 1 --2- 172.21.15.27:0/2634081225 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562a9ad21a50 0x562a9ad1a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:17:43.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.826+0000 7f64a541e640 1 --2- 172.21.15.27:0/2634081225 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562a9ad202c0 0x562a9ad65a60 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f6498089590 tx=0x7f64980594f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=da6078e07dc142e2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:43.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.826+0000 7f6486ffd640 1 -- 172.21.15.27:0/2634081225 <== mon.1 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6498021040 con 0x562a9ad202c0 2024-09-12T03:17:43.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.826+0000 7f6486ffd640 1 -- 172.21.15.27:0/2634081225 <== mon.1 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f649800cd20 con 0x562a9ad202c0 2024-09-12T03:17:43.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.826+0000 7f6486ffd640 1 -- 172.21.15.27:0/2634081225 <== mon.1 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6498089760 con 0x562a9ad202c0 2024-09-12T03:17:43.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.827+0000 7f64aa422740 1 -- 172.21.15.27:0/2634081225 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562a9ad202c0 msgr2=0x562a9ad65a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:43.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.827+0000 7f64aa422740 1 --2- 172.21.15.27:0/2634081225 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562a9ad202c0 0x562a9ad65a60 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f6498089590 tx=0x7f64980594f0 comp rx=0 tx=0).stop 2024-09-12T03:17:43.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.827+0000 7f64aa422740 1 -- 172.21.15.27:0/2634081225 shutdown_connections 2024-09-12T03:17:43.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.827+0000 7f64aa422740 1 --2- 172.21.15.27:0/2634081225 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562a9ad21a50 0x562a9ad1a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:43.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.827+0000 7f64aa422740 1 --2- 172.21.15.27:0/2634081225 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x562a9ad1da80 0x562a9ad206a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:43.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.827+0000 7f64aa422740 1 --2- 172.21.15.27:0/2634081225 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562a9ad202c0 0x562a9ad65a60 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:43.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.827+0000 7f64aa422740 1 -- 172.21.15.27:0/2634081225 >> 172.21.15.27:0/2634081225 conn(0x562a9ad1ca80 msgr2=0x562a9ad64f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:43.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.827+0000 7f64aa422740 1 -- 172.21.15.27:0/2634081225 shutdown_connections 2024-09-12T03:17:43.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.827+0000 7f64aa422740 1 -- 172.21.15.27:0/2634081225 wait complete. 2024-09-12T03:17:43.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.828+0000 7f64aa422740 1 -- start start 2024-09-12T03:17:43.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.828+0000 7f64aa422740 1 Processor -- start 2024-09-12T03:17:43.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.829+0000 7f64aa422740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562a9ad202c0 0x562a9ad3f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:43.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.829+0000 7f64aa422740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562a9ad21a50 0x562a9ad49690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:43.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.829+0000 7f64aa422740 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x562a9ad3f760 0x562a9ad48ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:43.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.829+0000 7f64aa422740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x562a9ad1bf20 con 0x562a9ad21a50 2024-09-12T03:17:43.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.829+0000 7f64aa422740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x562a9ac0b450 con 0x562a9ad202c0 2024-09-12T03:17:43.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.829+0000 7f64aa422740 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x562a9ad66290 con 0x562a9ad3f760 2024-09-12T03:17:43.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.829+0000 7f64aa422740 10 client.0 _ll_register_callbacks cb 0x562a9a994ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:17:43.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.829+0000 7f64a4c1d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562a9ad21a50 0x562a9ad49690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:43.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.829+0000 7f64a4c1d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562a9ad21a50 0x562a9ad49690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60426/0 (socket says 172.21.15.27:60426) 2024-09-12T03:17:43.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.829+0000 7f64a4c1d640 1 -- 172.21.15.27:0/3611190308 learned_addr learned my addr 172.21.15.27:0/3611190308 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:17:43.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.829+0000 7f64a541e640 1 --2- 172.21.15.27:0/3611190308 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562a9ad202c0 0x562a9ad3f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:43.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.829+0000 7f64a5c1f640 1 --2- 172.21.15.27:0/3611190308 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x562a9ad3f760 0x562a9ad48ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:43.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.829+0000 7f64a4c1d640 1 -- 172.21.15.27:0/3611190308 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x562a9ad3f760 msgr2=0x562a9ad48ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:43.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.829+0000 7f64a4c1d640 1 --2- 172.21.15.27:0/3611190308 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x562a9ad3f760 0x562a9ad48ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:43.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.829+0000 7f64a4c1d640 1 -- 172.21.15.27:0/3611190308 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562a9ad202c0 msgr2=0x562a9ad3f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:43.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.829+0000 7f64a4c1d640 1 --2- 172.21.15.27:0/3611190308 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562a9ad202c0 0x562a9ad3f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:43.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.829+0000 7f64a4c1d640 1 -- 172.21.15.27:0/3611190308 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562a9a9952d0 con 0x562a9ad21a50 2024-09-12T03:17:43.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.829+0000 7f64a5c1f640 1 --2- 172.21.15.27:0/3611190308 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x562a9ad3f760 0x562a9ad48ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:17:43.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.829+0000 7f64a4c1d640 1 --2- 172.21.15.27:0/3611190308 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562a9ad21a50 0x562a9ad49690 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f6494011fb0 tx=0x7f64940046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94d5498daa4d0f9c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:43.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.829+0000 7f647ffff640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-12T03:17:43.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.829+0000 7f647ffff640 1 -- 172.21.15.27:0/3611190308 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6494027070 con 0x562a9ad21a50 2024-09-12T03:17:43.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.829+0000 7f647ffff640 1 -- 172.21.15.27:0/3611190308 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(43..43 src has 1..43) ==== 5709+0+0 (secure 0 0 0) 0x7f6494014440 con 0x562a9ad21a50 2024-09-12T03:17:43.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.829+0000 7f64aa422740 1 -- 172.21.15.27:0/3611190308 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f649800cd20 con 0x562a9ad21a50 2024-09-12T03:17:43.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.829+0000 7f647ffff640 1 -- 172.21.15.27:0/3611190308 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f6494032050 con 0x562a9ad21a50 2024-09-12T03:17:43.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.829+0000 7f64aa422740 10 client.5337 fetch_fsmap learned FSMap version 22 2024-09-12T03:17:43.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.829+0000 7f64aa422740 1 -- 172.21.15.27:0/3611190308 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=2+}) -- 0x562a9aa5f6b0 con 0x562a9ad21a50 2024-09-12T03:17:43.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.830+0000 7f647ffff640 1 -- 172.21.15.27:0/3611190308 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f649800cd20 con 0x562a9ad21a50 2024-09-12T03:17:43.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.830+0000 7f647ffff640 1 -- 172.21.15.27:0/3611190308 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 22) ==== 3502+0+0 (secure 0 0 0) 0x7f6494035020 con 0x562a9ad21a50 2024-09-12T03:17:43.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.830+0000 7f64aa422740 10 client.5337 fetch_fsmap finished waiting for FSMap version 22 2024-09-12T03:17:43.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.830+0000 7f64aa422740 10 client.5337 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-12T03:17:43.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.830+0000 7f64aa422740 20 client.5337 populate_metadata read hostname 'smithi027' 2024-09-12T03:17:43.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.830+0000 7f64aa422740 1 --2- 172.21.15.27:0/3611190308 >> [v2:172.21.15.27:6833/2571101228,v1:172.21.15.27:6835/2571101228] conn(0x562a9ad1a360 0x562a9ad82e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:43.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.830+0000 7f64aa422740 4 client.5337 mds_command: new command op to 5205 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:17:43.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.830+0000 7f64aa422740 1 -- 172.21.15.27:0/3611190308 --> [v2:172.21.15.27:6833/2571101228,v1:172.21.15.27:6835/2571101228] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562a9ac8fc50 con 0x562a9ad1a360 2024-09-12T03:17:43.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.830+0000 7f64a541e640 1 --2- 172.21.15.27:0/3611190308 >> [v2:172.21.15.27:6833/2571101228,v1:172.21.15.27:6835/2571101228] conn(0x562a9ad1a360 0x562a9ad82e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:43.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.830+0000 7f64a541e640 1 --2- 172.21.15.27:0/3611190308 >> [v2:172.21.15.27:6833/2571101228,v1:172.21.15.27:6835/2571101228] conn(0x562a9ad1a360 0x562a9ad82e20 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1e8b765eb1e33572 server_cookie=c18fafd57a866b40 in_seq=0 out_seq=0 2024-09-12T03:17:43.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.830+0000 7f647ffff640 10 client.5337 ms_handle_connect on v2:172.21.15.27:6833/2571101228 2024-09-12T03:17:43.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.831+0000 7f647ffff640 1 -- 172.21.15.27:0/3611190308 <== mds.0 v2:172.21.15.27:6833/2571101228 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562a9ac8fc50 con 0x562a9ad1a360 2024-09-12T03:17:43.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.831+0000 7f647ffff640 10 client.5337 handle_command_reply: tid=0 2024-09-12T03:17:43.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.831+0000 7f64aa422740 1 client.5337 shutdown 2024-09-12T03:17:43.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.831+0000 7f64aa422740 1 -- 172.21.15.27:0/3611190308 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562a9ad21a50 msgr2=0x562a9ad49690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:43.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.831+0000 7f64aa422740 1 --2- 172.21.15.27:0/3611190308 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562a9ad21a50 0x562a9ad49690 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f6494011fb0 tx=0x7f64940046c0 comp rx=0 tx=0).stop 2024-09-12T03:17:43.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.831+0000 7f64aa422740 1 -- 172.21.15.27:0/3611190308 shutdown_connections 2024-09-12T03:17:43.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.831+0000 7f64aa422740 1 --2- 172.21.15.27:0/3611190308 >> [v2:172.21.15.27:6833/2571101228,v1:172.21.15.27:6835/2571101228] conn(0x562a9ad1a360 0x562a9ad82e20 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:43.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.831+0000 7f64aa422740 1 --2- 172.21.15.27:0/3611190308 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x562a9ad3f760 0x562a9ad48ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:43.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.831+0000 7f64aa422740 1 --2- 172.21.15.27:0/3611190308 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562a9ad21a50 0x562a9ad49690 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:43.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.831+0000 7f64aa422740 1 --2- 172.21.15.27:0/3611190308 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562a9ad202c0 0x562a9ad3f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:43.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.831+0000 7f647ffff640 0 client.5337 ms_handle_reset on v2:172.21.15.27:6833/2571101228 2024-09-12T03:17:43.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.831+0000 7f64aa422740 1 -- 172.21.15.27:0/3611190308 >> 172.21.15.27:0/3611190308 conn(0x562a9ad1da80 msgr2=0x562a9ad63e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:43.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.831+0000 7f64aa422740 1 -- 172.21.15.27:0/3611190308 shutdown_connections 2024-09-12T03:17:43.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.831+0000 7f64aa422740 1 -- 172.21.15.27:0/3611190308 wait complete. 2024-09-12T03:17:43.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.831+0000 7f64aa422740 20 client.5337 trim_cache size 0 max 16384 2024-09-12T03:17:43.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.849+0000 7f64aa422740 1 Processor -- start 2024-09-12T03:17:43.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.849+0000 7f64aa422740 1 -- start start 2024-09-12T03:17:43.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.850+0000 7f64aa422740 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x562a9ad202c0 0x562a9ad44e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:43.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.850+0000 7f64aa422740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562a9ad21a50 0x562a9ad3c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:43.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.850+0000 7f64aa422740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562a9ad3f760 0x562a9ad3c700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:43.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.850+0000 7f64aa422740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f64a0148fa0 con 0x562a9ad3f760 2024-09-12T03:17:43.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.850+0000 7f64aa422740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f64a0148300 con 0x562a9ad21a50 2024-09-12T03:17:43.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.850+0000 7f64aa422740 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f64a0148950 con 0x562a9ad202c0 2024-09-12T03:17:43.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.850+0000 7f64a5c1f640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562a9ad3f760 0x562a9ad3c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:43.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.850+0000 7f64a5c1f640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562a9ad3f760 0x562a9ad3c700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60442/0 (socket says 172.21.15.27:60442) 2024-09-12T03:17:43.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.850+0000 7f64a5c1f640 1 -- 172.21.15.27:0/4020653615 learned_addr learned my addr 172.21.15.27:0/4020653615 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:17:43.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.850+0000 7f64a4c1d640 1 --2- 172.21.15.27:0/4020653615 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562a9ad21a50 0x562a9ad3c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:43.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.850+0000 7f64a541e640 1 --2- 172.21.15.27:0/4020653615 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x562a9ad202c0 0x562a9ad44e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:43.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.850+0000 7f64a5c1f640 1 -- 172.21.15.27:0/4020653615 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x562a9ad202c0 msgr2=0x562a9ad44e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:43.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.850+0000 7f64a5c1f640 1 --2- 172.21.15.27:0/4020653615 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x562a9ad202c0 0x562a9ad44e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:43.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.850+0000 7f64a5c1f640 1 -- 172.21.15.27:0/4020653615 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562a9ad21a50 msgr2=0x562a9ad3c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:43.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.850+0000 7f64a5c1f640 1 --2- 172.21.15.27:0/4020653615 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562a9ad21a50 0x562a9ad3c1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:43.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.850+0000 7f64a5c1f640 1 -- 172.21.15.27:0/4020653615 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562a9ad48e20 con 0x562a9ad3f760 2024-09-12T03:17:43.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.850+0000 7f64a541e640 1 --2- 172.21.15.27:0/4020653615 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x562a9ad202c0 0x562a9ad44e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:17:43.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.851+0000 7f64a5c1f640 1 --2- 172.21.15.27:0/4020653615 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562a9ad3f760 0x562a9ad3c700 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f6490009f10 tx=0x7f649002e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8892a26883bb3b3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:43.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.851+0000 7f64867fc640 1 -- 172.21.15.27:0/4020653615 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6490042070 con 0x562a9ad3f760 2024-09-12T03:17:43.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.851+0000 7f64867fc640 1 -- 172.21.15.27:0/4020653615 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64900317c0 con 0x562a9ad3f760 2024-09-12T03:17:43.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.851+0000 7f64867fc640 1 -- 172.21.15.27:0/4020653615 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6490036040 con 0x562a9ad3f760 2024-09-12T03:17:43.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.851+0000 7f64aa422740 1 -- 172.21.15.27:0/4020653615 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562a9ad3f760 msgr2=0x562a9ad3c700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:43.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.851+0000 7f64aa422740 1 --2- 172.21.15.27:0/4020653615 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562a9ad3f760 0x562a9ad3c700 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f6490009f10 tx=0x7f649002e000 comp rx=0 tx=0).stop 2024-09-12T03:17:43.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.851+0000 7f64aa422740 1 -- 172.21.15.27:0/4020653615 shutdown_connections 2024-09-12T03:17:43.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.851+0000 7f64aa422740 1 --2- 172.21.15.27:0/4020653615 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562a9ad3f760 0x562a9ad3c700 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:43.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.851+0000 7f64aa422740 1 --2- 172.21.15.27:0/4020653615 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562a9ad21a50 0x562a9ad3c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:43.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.851+0000 7f64aa422740 1 --2- 172.21.15.27:0/4020653615 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x562a9ad202c0 0x562a9ad44e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:43.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.851+0000 7f64aa422740 1 -- 172.21.15.27:0/4020653615 >> 172.21.15.27:0/4020653615 conn(0x562a9ad1da80 msgr2=0x562a9ad66a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:43.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.851+0000 7f64aa422740 1 -- 172.21.15.27:0/4020653615 shutdown_connections 2024-09-12T03:17:43.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.851+0000 7f64aa422740 1 -- 172.21.15.27:0/4020653615 wait complete. 2024-09-12T03:17:43.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.852+0000 7f64aa422740 1 -- start start 2024-09-12T03:17:43.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.852+0000 7f64aa422740 1 Processor -- start 2024-09-12T03:17:43.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.853+0000 7f64aa422740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562a9ad202c0 0x562a9adebb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:43.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.853+0000 7f64aa422740 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x562a9ad21a50 0x562a9ad3c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:43.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.853+0000 7f64aa422740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562a9ad3f760 0x562a9ad3c700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:43.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.853+0000 7f64aa422740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x562a9ad1a1e0 con 0x562a9ad3f760 2024-09-12T03:17:43.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.853+0000 7f64aa422740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x562a9ad4aa10 con 0x562a9ad202c0 2024-09-12T03:17:43.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.853+0000 7f64aa422740 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x562a9ad1a830 con 0x562a9ad21a50 2024-09-12T03:17:43.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.853+0000 7f64aa422740 10 client.0 _ll_register_callbacks cb 0x7f6494004040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:17:43.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.853+0000 7f64a5c1f640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562a9ad3f760 0x562a9ad3c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:43.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.853+0000 7f64a5c1f640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562a9ad3f760 0x562a9ad3c700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60458/0 (socket says 172.21.15.27:60458) 2024-09-12T03:17:43.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.853+0000 7f64a5c1f640 1 -- 172.21.15.27:0/2377296463 learned_addr learned my addr 172.21.15.27:0/2377296463 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:17:43.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.853+0000 7f64a541e640 1 --2- 172.21.15.27:0/2377296463 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562a9ad202c0 0x562a9adebb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:43.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.853+0000 7f64a4c1d640 1 --2- 172.21.15.27:0/2377296463 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x562a9ad21a50 0x562a9ad3c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:43.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.853+0000 7f64a5c1f640 1 -- 172.21.15.27:0/2377296463 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x562a9ad21a50 msgr2=0x562a9ad3c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:43.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.853+0000 7f64a5c1f640 1 --2- 172.21.15.27:0/2377296463 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x562a9ad21a50 0x562a9ad3c1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:43.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.853+0000 7f64a5c1f640 1 -- 172.21.15.27:0/2377296463 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562a9ad202c0 msgr2=0x562a9adebb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:43.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.853+0000 7f64a5c1f640 1 --2- 172.21.15.27:0/2377296463 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562a9ad202c0 0x562a9adebb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:43.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.853+0000 7f64a5c1f640 1 -- 172.21.15.27:0/2377296463 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562a9ad45100 con 0x562a9ad3f760 2024-09-12T03:17:43.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.853+0000 7f64a5c1f640 1 --2- 172.21.15.27:0/2377296463 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562a9ad3f760 0x562a9ad3c700 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f6490009f40 tx=0x7f6490004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c85bad46d43804e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:43.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.853+0000 7f647ffff640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-12T03:17:43.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.854+0000 7f647ffff640 1 -- 172.21.15.27:0/2377296463 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6490036070 con 0x562a9ad3f760 2024-09-12T03:17:43.860 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.854+0000 7f647ffff640 1 -- 172.21.15.27:0/2377296463 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(43..43 src has 1..43) ==== 5709+0+0 (secure 0 0 0) 0x7f6490003ba0 con 0x562a9ad3f760 2024-09-12T03:17:43.860 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.854+0000 7f64aa422740 1 -- 172.21.15.27:0/2377296463 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f64900317c0 con 0x562a9ad3f760 2024-09-12T03:17:43.860 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.854+0000 7f647ffff640 1 -- 172.21.15.27:0/2377296463 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f6490058050 con 0x562a9ad3f760 2024-09-12T03:17:43.860 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.854+0000 7f64aa422740 10 client.5343 fetch_fsmap learned FSMap version 22 2024-09-12T03:17:43.860 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.854+0000 7f64aa422740 1 -- 172.21.15.27:0/2377296463 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=2+}) -- 0x562a9ad496c0 con 0x562a9ad3f760 2024-09-12T03:17:43.860 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.854+0000 7f647ffff640 1 -- 172.21.15.27:0/2377296463 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64900317c0 con 0x562a9ad3f760 2024-09-12T03:17:43.860 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.854+0000 7f647ffff640 1 -- 172.21.15.27:0/2377296463 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 22) ==== 3502+0+0 (secure 0 0 0) 0x7f649005b020 con 0x562a9ad3f760 2024-09-12T03:17:43.860 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.854+0000 7f64aa422740 10 client.5343 fetch_fsmap finished waiting for FSMap version 22 2024-09-12T03:17:43.860 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.854+0000 7f64aa422740 10 client.5343 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-12T03:17:43.860 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.854+0000 7f64aa422740 20 client.5343 populate_metadata read hostname 'smithi027' 2024-09-12T03:17:43.860 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.854+0000 7f64aa422740 1 --2- 172.21.15.27:0/2377296463 >> [v2:172.21.15.27:6833/2571101228,v1:172.21.15.27:6835/2571101228] conn(0x562a9ad74460 0x562a9ad77480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:43.860 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.854+0000 7f64aa422740 4 client.5343 mds_command: new command op to 5205 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-12T03:17:43.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.854+0000 7f64aa422740 1 -- 172.21.15.27:0/2377296463 --> [v2:172.21.15.27:6833/2571101228,v1:172.21.15.27:6835/2571101228] -- command(tid 0: {"prefix": "status"}) -- 0x562a9ac89fa0 con 0x562a9ad74460 2024-09-12T03:17:43.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.854+0000 7f64a541e640 1 --2- 172.21.15.27:0/2377296463 >> [v2:172.21.15.27:6833/2571101228,v1:172.21.15.27:6835/2571101228] conn(0x562a9ad74460 0x562a9ad77480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:43.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.855+0000 7f64a541e640 1 --2- 172.21.15.27:0/2377296463 >> [v2:172.21.15.27:6833/2571101228,v1:172.21.15.27:6835/2571101228] conn(0x562a9ad74460 0x562a9ad77480 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1eaef23ee488a456 server_cookie=1465e2e988916a92 in_seq=0 out_seq=0 2024-09-12T03:17:43.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.855+0000 7f647ffff640 10 client.5343 ms_handle_connect on v2:172.21.15.27:6833/2571101228 2024-09-12T03:17:43.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.855+0000 7f647ffff640 1 -- 172.21.15.27:0/2377296463 <== mds.0 v2:172.21.15.27:6833/2571101228 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x562a9ac89fa0 con 0x562a9ad74460 2024-09-12T03:17:43.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.855+0000 7f647ffff640 10 client.5343 handle_command_reply: tid=0 2024-09-12T03:17:43.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.855+0000 7f64aa422740 1 client.5343 shutdown 2024-09-12T03:17:43.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.855+0000 7f64aa422740 1 -- 172.21.15.27:0/2377296463 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562a9ad3f760 msgr2=0x562a9ad3c700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:43.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.855+0000 7f64aa422740 1 --2- 172.21.15.27:0/2377296463 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562a9ad3f760 0x562a9ad3c700 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f6490009f40 tx=0x7f6490004ab0 comp rx=0 tx=0).stop 2024-09-12T03:17:43.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.855+0000 7f64aa422740 1 -- 172.21.15.27:0/2377296463 shutdown_connections 2024-09-12T03:17:43.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.855+0000 7f64aa422740 1 --2- 172.21.15.27:0/2377296463 >> [v2:172.21.15.27:6833/2571101228,v1:172.21.15.27:6835/2571101228] conn(0x562a9ad74460 0x562a9ad77480 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:43.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.855+0000 7f64aa422740 1 --2- 172.21.15.27:0/2377296463 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562a9ad3f760 0x562a9ad3c700 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:43.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.855+0000 7f64aa422740 1 --2- 172.21.15.27:0/2377296463 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x562a9ad21a50 0x562a9ad3c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:43.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.855+0000 7f647ffff640 0 client.5343 ms_handle_reset on v2:172.21.15.27:6833/2571101228 2024-09-12T03:17:43.862 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.855+0000 7f64aa422740 1 --2- 172.21.15.27:0/2377296463 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562a9ad202c0 0x562a9adebb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:43.862 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.855+0000 7f64aa422740 1 -- 172.21.15.27:0/2377296463 >> 172.21.15.27:0/2377296463 conn(0x562a9ad1da80 msgr2=0x562a9ad62fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:43.862 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.855+0000 7f64aa422740 1 -- 172.21.15.27:0/2377296463 shutdown_connections 2024-09-12T03:17:43.862 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.855+0000 7f64aa422740 1 -- 172.21.15.27:0/2377296463 wait complete. 2024-09-12T03:17:43.862 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.855+0000 7f64aa422740 20 client.5343 trim_cache size 0 max 16384 2024-09-12T03:17:43.862 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-12T03:17:43.862 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_fsid": "d2df504e-2647-4385-a5db-9fdf69f487a0", 2024-09-12T03:17:43.862 INFO:teuthology.orchestra.run.smithi027.stdout: "whoami": 0, 2024-09-12T03:17:43.862 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 5205, 2024-09-12T03:17:43.862 INFO:teuthology.orchestra.run.smithi027.stdout: "want_state": "up:active", 2024-09-12T03:17:43.862 INFO:teuthology.orchestra.run.smithi027.stdout: "state": "up:active", 2024-09-12T03:17:43.863 INFO:teuthology.orchestra.run.smithi027.stdout: "fs_name": "cephfs", 2024-09-12T03:17:43.863 INFO:teuthology.orchestra.run.smithi027.stdout: "rank_uptime": 4.6421961700000001, 2024-09-12T03:17:43.863 INFO:teuthology.orchestra.run.smithi027.stdout: "mdsmap_epoch": 22, 2024-09-12T03:17:43.863 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch": 43, 2024-09-12T03:17:43.863 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch_barrier": 43, 2024-09-12T03:17:43.863 INFO:teuthology.orchestra.run.smithi027.stdout: "uptime": 8.7376827590000001 2024-09-12T03:17:43.863 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-12T03:17:43.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.856+0000 7f64a7682640 1 -- 172.21.15.27:0/3058461387 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f648c06bfe0 msgr2=0x7f648c06e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:43.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.856+0000 7f64a7682640 1 --2- 172.21.15.27:0/3058461387 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f648c06bfe0 0x7f648c06e4a0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f64a006fdc0 tx=0x7f6494009190 comp rx=0 tx=0).stop 2024-09-12T03:17:43.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.856+0000 7f64a7682640 1 -- 172.21.15.27:0/3058461387 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f64a0158cf0 msgr2=0x7f64a0145800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:43.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.856+0000 7f64a7682640 1 --2- 172.21.15.27:0/3058461387 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f64a0158cf0 0x7f64a0145800 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f649800e870 tx=0x7f649800ed30 comp rx=0 tx=0).stop 2024-09-12T03:17:43.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.856+0000 7f64a7682640 1 -- 172.21.15.27:0/3058461387 shutdown_connections 2024-09-12T03:17:43.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.856+0000 7f64a7682640 1 --2- 172.21.15.27:0/3058461387 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f648c06bfe0 0x7f648c06e4a0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:43.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.856+0000 7f64a7682640 1 --2- 172.21.15.27:0/3058461387 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f64a015f600 0x7f64a0146690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:43.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.856+0000 7f64a7682640 1 --2- 172.21.15.27:0/3058461387 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f64a01590d0 0x7f64a0146150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:43.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.856+0000 7f64a7682640 1 --2- 172.21.15.27:0/3058461387 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f64a0158cf0 0x7f64a0145800 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:43.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.856+0000 7f64a7682640 1 -- 172.21.15.27:0/3058461387 >> 172.21.15.27:0/3058461387 conn(0x7f64a006cc40 msgr2=0x7f64a00af980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:43.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.856+0000 7f64a7682640 1 -- 172.21.15.27:0/3058461387 shutdown_connections 2024-09-12T03:17:43.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.856+0000 7f64a7682640 1 -- 172.21.15.27:0/3058461387 wait complete. 2024-09-12T03:17:43.867 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-12T03:17:43.969 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.968+0000 7f15cf949640 1 Processor -- start 2024-09-12T03:17:43.969 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.969+0000 7f15cf949640 1 -- start start 2024-09-12T03:17:43.969 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.969+0000 7f15cf949640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f15c815eb90 0x7f15c80d2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:43.969 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.969+0000 7f15cf949640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f15c80d5200 0x7f15c80d1010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:43.970 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.970+0000 7f15cf949640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f15c80d19e0 con 0x7f15c80d1550 2024-09-12T03:17:43.970 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.970+0000 7f15cf949640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f15c80b7d20 con 0x7f15c80d5200 2024-09-12T03:17:43.970 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.970+0000 7f15cf949640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f15c80d42e0 con 0x7f15c815eb90 2024-09-12T03:17:43.971 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.970+0000 7f15cd6e5640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f15c815eb90 0x7f15c80d2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:43.971 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.970+0000 7f15ccee4640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f15c80d5200 0x7f15c80d1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:43.971 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.970+0000 7f15cdee6640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f15c80d1550 0x7f15c80cf200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.27:41646/0 (socket says 172.21.15.27:41646) 2024-09-12T03:17:43.971 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.970+0000 7f15cdee6640 1 -- 172.21.15.27:0/775409162 learned_addr learned my addr 172.21.15.27:0/775409162 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:17:43.971 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.970+0000 7f15bffff640 1 -- 172.21.15.27:0/775409162 <== mon.1 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2080191470 0 0) 0x7f15c80d19e0 con 0x7f15c80d1550 2024-09-12T03:17:43.971 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.970+0000 7f15ccee4640 1 -- 172.21.15.27:0/775409162 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f15c815eb90 msgr2=0x7f15c80d2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:43.971 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.970+0000 7f15ccee4640 1 --2- 172.21.15.27:0/775409162 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f15c815eb90 0x7f15c80d2df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:43.971 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.970+0000 7f15ccee4640 1 -- 172.21.15.27:0/775409162 >> v1:172.21.15.45:6789/0 conn(0x7f15c80d1550 legacy=0x7f15c80cf200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:43.971 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.970+0000 7f15ccee4640 1 -- 172.21.15.27:0/775409162 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15c80d3760 con 0x7f15c80d5200 2024-09-12T03:17:43.973 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.971+0000 7f15ccee4640 1 --2- 172.21.15.27:0/775409162 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f15c80d5200 0x7f15c80d1010 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f15b80097a0 tx=0x7f15b802a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b7fbbfc901ec1503 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:43.973 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.971+0000 7f15bffff640 1 -- 172.21.15.27:0/775409162 <== mon.1 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15b8037070 con 0x7f15c80d5200 2024-09-12T03:17:43.973 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.971+0000 7f15bffff640 1 -- 172.21.15.27:0/775409162 <== mon.1 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f15b8032440 con 0x7f15c80d5200 2024-09-12T03:17:43.973 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.971+0000 7f15bffff640 1 -- 172.21.15.27:0/775409162 <== mon.1 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15b8032810 con 0x7f15c80d5200 2024-09-12T03:17:43.973 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.972+0000 7f15cf949640 1 -- 172.21.15.27:0/775409162 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f15c80d5200 msgr2=0x7f15c80d1010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:43.973 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.972+0000 7f15cf949640 1 --2- 172.21.15.27:0/775409162 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f15c80d5200 0x7f15c80d1010 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f15b80097a0 tx=0x7f15b802a180 comp rx=0 tx=0).stop 2024-09-12T03:17:43.973 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.972+0000 7f15cf949640 1 -- 172.21.15.27:0/775409162 shutdown_connections 2024-09-12T03:17:43.973 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.972+0000 7f15cf949640 1 --2- 172.21.15.27:0/775409162 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f15c80d5200 0x7f15c80d1010 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:43.973 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.972+0000 7f15cf949640 1 --2- 172.21.15.27:0/775409162 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f15c815eb90 0x7f15c80d2df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:43.973 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.972+0000 7f15cf949640 1 -- 172.21.15.27:0/775409162 >> 172.21.15.27:0/775409162 conn(0x7f15c806dce0 msgr2=0x7f15c806e0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:43.973 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.972+0000 7f15cf949640 1 -- 172.21.15.27:0/775409162 shutdown_connections 2024-09-12T03:17:43.974 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.972+0000 7f15cf949640 1 -- 172.21.15.27:0/775409162 wait complete. 2024-09-12T03:17:43.976 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.973+0000 7f15cf949640 1 Processor -- start 2024-09-12T03:17:43.976 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.973+0000 7f15cf949640 1 -- start start 2024-09-12T03:17:43.976 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.974+0000 7f15cf949640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f15c80d1550 0x7f15c81528d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:43.976 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.974+0000 7f15cf949640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f15c80d5200 0x7f15c8155ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:43.976 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.974+0000 7f15cf949640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f15c815eb90 0x7f15c80ca790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:43.976 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.974+0000 7f15cf949640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f15c80d4000 con 0x7f15c80d1550 2024-09-12T03:17:43.976 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.974+0000 7f15cf949640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f15c80d3d20 con 0x7f15c80d5200 2024-09-12T03:17:43.976 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.974+0000 7f15cf949640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f15c80d3330 con 0x7f15c815eb90 2024-09-12T03:17:43.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.974+0000 7f15cd6e5640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f15c80d1550 0x7f15c81528d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:43.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.974+0000 7f15cd6e5640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f15c80d1550 0x7f15c81528d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60474/0 (socket says 172.21.15.27:60474) 2024-09-12T03:17:43.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.974+0000 7f15cd6e5640 1 -- 172.21.15.27:0/1901688717 learned_addr learned my addr 172.21.15.27:0/1901688717 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:17:43.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.974+0000 7f15ccee4640 1 --2- 172.21.15.27:0/1901688717 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f15c80d5200 0x7f15c8155ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:43.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.974+0000 7f15cdee6640 1 --2- 172.21.15.27:0/1901688717 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f15c815eb90 0x7f15c80ca790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:43.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.974+0000 7f15cd6e5640 1 -- 172.21.15.27:0/1901688717 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f15c815eb90 msgr2=0x7f15c80ca790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:43.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.974+0000 7f15cd6e5640 1 --2- 172.21.15.27:0/1901688717 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f15c815eb90 0x7f15c80ca790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:43.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.974+0000 7f15cd6e5640 1 -- 172.21.15.27:0/1901688717 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f15c80d5200 msgr2=0x7f15c8155ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:43.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.974+0000 7f15cd6e5640 1 --2- 172.21.15.27:0/1901688717 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f15c80d5200 0x7f15c8155ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:43.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.974+0000 7f15cd6e5640 1 -- 172.21.15.27:0/1901688717 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15c8152e10 con 0x7f15c80d1550 2024-09-12T03:17:43.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.975+0000 7f15cd6e5640 1 --2- 172.21.15.27:0/1901688717 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f15c80d1550 0x7f15c81528d0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f15c400e870 tx=0x7f15c400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:43.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.975+0000 7f15bdffb640 1 -- 172.21.15.27:0/1901688717 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15c4017070 con 0x7f15c80d1550 2024-09-12T03:17:43.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.975+0000 7f15bdffb640 1 -- 172.21.15.27:0/1901688717 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f15c400c940 con 0x7f15c80d1550 2024-09-12T03:17:43.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.975+0000 7f15cf949640 1 -- 172.21.15.27:0/1901688717 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15c81568d0 con 0x7f15c80d1550 2024-09-12T03:17:43.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.975+0000 7f15bdffb640 1 -- 172.21.15.27:0/1901688717 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15c400cd30 con 0x7f15c80d1550 2024-09-12T03:17:43.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.975+0000 7f15cf949640 1 -- 172.21.15.27:0/1901688717 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15c8156e90 con 0x7f15c80d1550 2024-09-12T03:17:43.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.976+0000 7f15cf949640 1 -- 172.21.15.27:0/1901688717 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15c81600f0 con 0x7f15c80d1550 2024-09-12T03:17:43.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.977+0000 7f15bdffb640 1 -- 172.21.15.27:0/1901688717 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 7) ==== 90651+0+0 (secure 0 0 0) 0x7f15c401a070 con 0x7f15c80d1550 2024-09-12T03:17:43.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.977+0000 7f15bdffb640 1 --2- 172.21.15.27:0/1901688717 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f15ac06c010 0x7f15ac06e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:43.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.977+0000 7f15bdffb640 1 -- 172.21.15.27:0/1901688717 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5709+0+0 (secure 0 0 0) 0x7f15c408aec0 con 0x7f15c80d1550 2024-09-12T03:17:43.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.977+0000 7f15ccee4640 1 --2- 172.21.15.27:0/1901688717 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f15ac06c010 0x7f15ac06e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:43.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.978+0000 7f15ccee4640 1 --2- 172.21.15.27:0/1901688717 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f15ac06c010 0x7f15ac06e4d0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f15b8034000 tx=0x7f15b802e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:43.981 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:43.981+0000 7f15bdffb640 1 -- 172.21.15.27:0/1901688717 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f15c4059050 con 0x7f15c80d1550 2024-09-12T03:17:44.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:44.191+0000 7f15cf949640 1 -- 172.21.15.27:0/1901688717 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f15c8161310 con 0x7f15c80d1550 2024-09-12T03:17:44.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:44.192+0000 7f15bdffb640 1 -- 172.21.15.27:0/1901688717 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v22) ==== 53+0+73 (secure 0 0 0) 0x7f15c405d350 con 0x7f15c80d1550 2024-09-12T03:17:44.192 INFO:teuthology.orchestra.run.smithi027.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-12T03:17:44.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:44.194+0000 7f15cf949640 1 -- 172.21.15.27:0/1901688717 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f15ac06c010 msgr2=0x7f15ac06e4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:44.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:44.194+0000 7f15cf949640 1 --2- 172.21.15.27:0/1901688717 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f15ac06c010 0x7f15ac06e4d0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f15b8034000 tx=0x7f15b802e000 comp rx=0 tx=0).stop 2024-09-12T03:17:44.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:44.194+0000 7f15cf949640 1 -- 172.21.15.27:0/1901688717 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f15c80d1550 msgr2=0x7f15c81528d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:44.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:44.194+0000 7f15cf949640 1 --2- 172.21.15.27:0/1901688717 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f15c80d1550 0x7f15c81528d0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f15c400e870 tx=0x7f15c400ed30 comp rx=0 tx=0).stop 2024-09-12T03:17:44.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:44.194+0000 7f15cf949640 1 -- 172.21.15.27:0/1901688717 shutdown_connections 2024-09-12T03:17:44.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:44.194+0000 7f15cf949640 1 --2- 172.21.15.27:0/1901688717 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f15ac06c010 0x7f15ac06e4d0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:44.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:44.194+0000 7f15cf949640 1 --2- 172.21.15.27:0/1901688717 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f15c815eb90 0x7f15c80ca790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:44.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:44.194+0000 7f15cf949640 1 --2- 172.21.15.27:0/1901688717 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f15c80d5200 0x7f15c8155ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:44.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:44.194+0000 7f15cf949640 1 --2- 172.21.15.27:0/1901688717 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f15c80d1550 0x7f15c81528d0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:44.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:44.194+0000 7f15cf949640 1 -- 172.21.15.27:0/1901688717 >> 172.21.15.27:0/1901688717 conn(0x7f15c806dce0 msgr2=0x7f15c8160b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:44.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:44.194+0000 7f15cf949640 1 -- 172.21.15.27:0/1901688717 shutdown_connections 2024-09-12T03:17:44.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:44.194+0000 7f15cf949640 1 -- 172.21.15.27:0/1901688717 wait complete. 2024-09-12T03:17:44.205 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-12T03:17:44.205 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-12T03:17:44.205 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi027.front.sepia.ceph.com 2024-09-12T03:17:44.205 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-12T03:17:44.205 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-12T03:17:44.205 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-12T03:17:44.205 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-12T03:17:44.205 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-12T03:17:44.206 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-12T03:17:44.206 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:17:44.206 DEBUG:teuthology.orchestra.run.smithi027:> ip addr 2024-09-12T03:17:44.222 INFO:teuthology.orchestra.run.smithi027.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-12T03:17:44.222 INFO:teuthology.orchestra.run.smithi027.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-12T03:17:44.223 INFO:teuthology.orchestra.run.smithi027.stdout: inet 127.0.0.1/8 scope host lo 2024-09-12T03:17:44.223 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:17:44.223 INFO:teuthology.orchestra.run.smithi027.stdout: inet6 ::1/128 scope host 2024-09-12T03:17:44.223 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:17:44.223 INFO:teuthology.orchestra.run.smithi027.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-12T03:17:44.223 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether 0c:c4:7a:6b:fe:c8 brd ff:ff:ff:ff:ff:ff 2024-09-12T03:17:44.223 INFO:teuthology.orchestra.run.smithi027.stdout: altname enp7s0f0 2024-09-12T03:17:44.223 INFO:teuthology.orchestra.run.smithi027.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-12T03:17:44.223 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether 0c:c4:7a:6b:fe:c9 brd ff:ff:ff:ff:ff:ff 2024-09-12T03:17:44.223 INFO:teuthology.orchestra.run.smithi027.stdout: altname enp7s0f1 2024-09-12T03:17:44.223 INFO:teuthology.orchestra.run.smithi027.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-12T03:17:44.223 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether ac:1f:6b:f8:39:5a brd ff:ff:ff:ff:ff:ff 2024-09-12T03:17:44.224 INFO:teuthology.orchestra.run.smithi027.stdout: altname enp1s0f0 2024-09-12T03:17:44.224 INFO:teuthology.orchestra.run.smithi027.stdout: inet 172.21.15.27/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-12T03:17:44.224 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft 42725sec preferred_lft 42725sec 2024-09-12T03:17:44.224 INFO:teuthology.orchestra.run.smithi027.stdout: inet6 fe80::ae1f:6bff:fef8:395a/64 scope link 2024-09-12T03:17:44.224 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:17:44.224 INFO:teuthology.orchestra.run.smithi027.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-12T03:17:44.224 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether ac:1f:6b:f8:39:5b brd ff:ff:ff:ff:ff:ff 2024-09-12T03:17:44.224 INFO:teuthology.orchestra.run.smithi027.stdout: altname enp1s0f1 2024-09-12T03:17:44.224 INFO:teuthology.orchestra.run.smithi027.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-12T03:17:44.224 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether 2e:7f:89:6c:33:a0 brd ff:ff:ff:ff:ff:ff 2024-09-12T03:17:44.224 INFO:teuthology.orchestra.run.smithi027.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-12T03:17:44.224 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:17:44.224 INFO:teuthology.orchestra.run.smithi027.stdout: inet6 fe80::c84b:19ff:fef0:ca74/64 scope link 2024-09-12T03:17:44.224 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:17:44.225 INFO:teuthology.orchestra.run.smithi027.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-12T03:17:44.225 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether 42:40:d7:5c:7a:8e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-12T03:17:44.225 INFO:teuthology.orchestra.run.smithi027.stdout: inet6 fe80::4040:d7ff:fe5c:7a8e/64 scope link 2024-09-12T03:17:44.225 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:17:44.225 INFO:teuthology.orchestra.run.smithi027.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-12T03:17:44.225 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether 2e:7f:89:6c:33:a0 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-12T03:17:44.225 INFO:teuthology.orchestra.run.smithi027.stdout: inet6 fe80::2c7f:89ff:fe6c:33a0/64 scope link 2024-09-12T03:17:44.225 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:17:44.225 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:17:44.225 DEBUG:teuthology.orchestra.run.smithi027:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-12T03:17:44.287 INFO:teuthology.orchestra.run.smithi027.stdout:1 2024-09-12T03:17:44.289 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:17:44.289 DEBUG:teuthology.orchestra.run.smithi027:> route 2024-09-12T03:17:44.345 INFO:teuthology.orchestra.run.smithi027.stdout:Kernel IP routing table 2024-09-12T03:17:44.345 INFO:teuthology.orchestra.run.smithi027.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-12T03:17:44.345 INFO:teuthology.orchestra.run.smithi027.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-12T03:17:44.346 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-12T03:17:44.346 INFO:teuthology.orchestra.run.smithi027.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-12T03:17:44.346 DEBUG:teuthology.orchestra.run.smithi027:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-12T03:17:44.346 DEBUG:teuthology.orchestra.run.smithi027:> set -e 2024-09-12T03:17:44.346 DEBUG:teuthology.orchestra.run.smithi027:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-12T03:17:44.346 DEBUG:teuthology.orchestra.run.smithi027:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-12T03:17:44.346 DEBUG:teuthology.orchestra.run.smithi027:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-12T03:17:44.346 DEBUG:teuthology.orchestra.run.smithi027:> ') 2024-09-12T03:17:44.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:44 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T03:17:44.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T03:17:44.475 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:17:44.475 DEBUG:teuthology.orchestra.run.smithi027:> ip netns list 2024-09-12T03:17:44.491 INFO:teuthology.orchestra.run.smithi027.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-12T03:17:44.491 INFO:teuthology.orchestra.run.smithi027.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-12T03:17:44.491 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-12T03:17:44.491 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:17:44.491 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip link set brx.0 up 2024-09-12T03:17:44.557 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-12T03:17:44.557 DEBUG:teuthology.orchestra.run.smithi027:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-12T03:17:44.613 INFO:teuthology.orchestra.run.smithi027.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-12T03:17:44.613 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-12T03:17:44.613 DEBUG:teuthology.orchestra.run.smithi027:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-12T03:17:44.668 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-09-12T03:17:44.669 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:17:44.669 DEBUG:teuthology.orchestra.run.smithi027:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-12T03:17:44.753 INFO:teuthology.orchestra.run.smithi027.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-12T03:17:44.753 INFO:teuthology.orchestra.run.smithi027.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-12T03:17:44.761 INFO:teuthology.orchestra.run.smithi027.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-12T03:17:44.761 INFO:teuthology.orchestra.run.smithi027.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-12T03:17:44.761 INFO:teuthology.orchestra.run.smithi027.stdout:mount.ceph: resolved to: "172.21.15.27:3300,172.21.15.45:3300,172.21.15.45:3301" 2024-09-12T03:17:44.761 INFO:teuthology.orchestra.run.smithi027.stdout:mount.ceph: trying mount with new device syntax: 0@d2df504e-2647-4385-a5db-9fdf69f487a0.cephfs=/ 2024-09-12T03:17:44.761 INFO:teuthology.orchestra.run.smithi027.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.27:3300/172.21.15.45:3300/172.21.15.45:3301" will pass to kernel 2024-09-12T03:17:44.762 INFO:teuthology.orchestra.run.smithi027.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-12T03:17:44.762 INFO:teuthology.orchestra.run.smithi027.stdout: You just mounted a file system that supports labels which does not 2024-09-12T03:17:44.762 INFO:teuthology.orchestra.run.smithi027.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-12T03:17:44.762 INFO:teuthology.orchestra.run.smithi027.stdout: applications will generate AVC messages and not be allowed access to 2024-09-12T03:17:44.762 INFO:teuthology.orchestra.run.smithi027.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-12T03:17:44.764 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-12T03:17:44.764 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:17:44.764 DEBUG:teuthology.orchestra.run.smithi027:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-12T03:17:44.789 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-12T03:17:44.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:44 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-12T03:17:44.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T03:17:44.891 INFO:teuthology.orchestra.run.smithi027.stdout:client5349 2024-09-12T03:17:44.891 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-12T03:17:44.891 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T03:17:44.908 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-12T03:17:45.050 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:45.050+0000 7f49dec65640 1 Processor -- start 2024-09-12T03:17:45.050 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:45.050+0000 7f49dec65640 1 -- start start 2024-09-12T03:17:45.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:45.051+0000 7f49dec65640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f49d815eb00 0x7f49d80cede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:45.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:45.051+0000 7f49dec65640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f49d80d3dd0 0x7f49d80b75e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:45.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:45.051+0000 7f49dec65640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f49d80b7b20 0x7f49d80b3500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:45.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:45.051+0000 7f49dec65640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f49d80cf320 con 0x7f49d815eb00 2024-09-12T03:17:45.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:45.051+0000 7f49dec65640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f49d80cfb80 con 0x7f49d80d3dd0 2024-09-12T03:17:45.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:45.051+0000 7f49dec65640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f49d80cf5c0 con 0x7f49d80b7b20 2024-09-12T03:17:45.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:45.051+0000 7f49dca01640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f49d815eb00 0x7f49d80cede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:45.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:45.051+0000 7f49dca01640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f49d815eb00 0x7f49d80cede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60482/0 (socket says 172.21.15.27:60482) 2024-09-12T03:17:45.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:45.051+0000 7f49dca01640 1 -- 172.21.15.27:0/1124133552 learned_addr learned my addr 172.21.15.27:0/1124133552 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:17:45.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:45.051+0000 7f49d7fff640 1 --2- 172.21.15.27:0/1124133552 >> v2:172.21.15.45:3300/0 conn(0x7f49d80d3dd0 0x7f49d80b75e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:45.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:45.051+0000 7f49dd202640 1 --2- 172.21.15.27:0/1124133552 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f49d80b7b20 0x7f49d80b3500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:45.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:45.052+0000 7f49dca01640 1 -- 172.21.15.27:0/1124133552 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f49d80b7b20 msgr2=0x7f49d80b3500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:45.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:45.052+0000 7f49dca01640 1 --2- 172.21.15.27:0/1124133552 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f49d80b7b20 0x7f49d80b3500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:45.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:45.052+0000 7f49dca01640 1 -- 172.21.15.27:0/1124133552 >> v2:172.21.15.45:3300/0 conn(0x7f49d80d3dd0 msgr2=0x7f49d80b75e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:45.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:45.052+0000 7f49dca01640 1 --2- 172.21.15.27:0/1124133552 >> v2:172.21.15.45:3300/0 conn(0x7f49d80d3dd0 0x7f49d80b75e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:45.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:45.052+0000 7f49dca01640 1 -- 172.21.15.27:0/1124133552 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49d80cf8a0 con 0x7f49d815eb00 2024-09-12T03:17:45.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:45.052+0000 7f49dd202640 1 --2- 172.21.15.27:0/1124133552 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f49d80b7b20 0x7f49d80b3500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:17:45.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:45.052+0000 7f49dca01640 1 --2- 172.21.15.27:0/1124133552 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f49d815eb00 0x7f49d80cede0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f49d80b7360 tx=0x7f49c002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5628b68bbd2fc772 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:45.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:45.053+0000 7f49d77fe640 1 -- 172.21.15.27:0/1124133552 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49c0037070 con 0x7f49d815eb00 2024-09-12T03:17:45.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:45.053+0000 7f49d77fe640 1 -- 172.21.15.27:0/1124133552 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49c0032440 con 0x7f49d815eb00 2024-09-12T03:17:45.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:45.053+0000 7f49d77fe640 1 -- 172.21.15.27:0/1124133552 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49c0032810 con 0x7f49d815eb00 2024-09-12T03:17:45.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:45.053+0000 7f49dec65640 1 -- 172.21.15.27:0/1124133552 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f49d815eb00 msgr2=0x7f49d80cede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:45.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:45.053+0000 7f49dec65640 1 --2- 172.21.15.27:0/1124133552 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f49d815eb00 0x7f49d80cede0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f49d80b7360 tx=0x7f49c002a1e0 comp rx=0 tx=0).stop 2024-09-12T03:17:45.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:45.053+0000 7f49dec65640 1 -- 172.21.15.27:0/1124133552 shutdown_connections 2024-09-12T03:17:45.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:45.053+0000 7f49dec65640 1 --2- 172.21.15.27:0/1124133552 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f49d80b7b20 0x7f49d80b3500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:45.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:45.053+0000 7f49dec65640 1 --2- 172.21.15.27:0/1124133552 >> v2:172.21.15.45:3300/0 conn(0x7f49d80d3dd0 0x7f49d80b75e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:45.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:45.053+0000 7f49dec65640 1 --2- 172.21.15.27:0/1124133552 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f49d815eb00 0x7f49d80cede0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:45.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:45.053+0000 7f49dec65640 1 -- 172.21.15.27:0/1124133552 >> 172.21.15.27:0/1124133552 conn(0x7f49d80d4790 msgr2=0x7f49d80d4ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:45.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:45.053+0000 7f49dec65640 1 -- 172.21.15.27:0/1124133552 shutdown_connections 2024-09-12T03:17:45.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:45.053+0000 7f49dec65640 1 -- 172.21.15.27:0/1124133552 wait complete. 2024-09-12T03:17:45.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:45.055+0000 7f49dec65640 1 Processor -- start 2024-09-12T03:17:45.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:45.055+0000 7f49dec65640 1 -- start start 2024-09-12T03:17:45.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:45.055+0000 7f49dec65640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f49d80d3dd0 0x7f49d81530b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:45.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:45.055+0000 7f49dec65640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f49d815eb00 0x7f49d80b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:45.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:45.055+0000 7f49dec65640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f49d80b6790 0x7f49d815fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:45.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:45.056+0000 7f49dec65640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f49d80b3a40 con 0x7f49d80b6790 2024-09-12T03:17:45.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:45.056+0000 7f49dec65640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f49d80d2cf0 con 0x7f49d815eb00 2024-09-12T03:17:45.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:45.056+0000 7f49dec65640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f49d80b3bc0 con 0x7f49d80d3dd0 2024-09-12T03:17:45.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:45.056+0000 7f49dd202640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f49d80b6790 0x7f49d815fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:45.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:45.056+0000 7f49dd202640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f49d80b6790 0x7f49d815fb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60498/0 (socket says 172.21.15.27:60498) 2024-09-12T03:17:45.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:45.056+0000 7f49dd202640 1 -- 172.21.15.27:0/2514129187 learned_addr learned my addr 172.21.15.27:0/2514129187 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:17:45.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:45.056+0000 7f49d7fff640 1 --2- 172.21.15.27:0/2514129187 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f49d815eb00 0x7f49d80b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:45.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:45.056+0000 7f49dca01640 1 --2- 172.21.15.27:0/2514129187 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f49d80d3dd0 0x7f49d81530b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:45.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:45.056+0000 7f49dd202640 1 -- 172.21.15.27:0/2514129187 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f49d80d3dd0 msgr2=0x7f49d81530b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:45.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:45.056+0000 7f49dd202640 1 --2- 172.21.15.27:0/2514129187 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f49d80d3dd0 0x7f49d81530b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:45.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:45.056+0000 7f49dd202640 1 -- 172.21.15.27:0/2514129187 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f49d815eb00 msgr2=0x7f49d80b6250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:45.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:45.056+0000 7f49dd202640 1 --2- 172.21.15.27:0/2514129187 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f49d815eb00 0x7f49d80b6250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:45.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:45.056+0000 7f49dd202640 1 -- 172.21.15.27:0/2514129187 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49d81f8800 con 0x7f49d80b6790 2024-09-12T03:17:45.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:45.056+0000 7f49dd202640 1 --2- 172.21.15.27:0/2514129187 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f49d80b6790 0x7f49d815fb20 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f49d000e530 tx=0x7f49d000e9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:45.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:45.056+0000 7f49d57fa640 1 -- 172.21.15.27:0/2514129187 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49d000f040 con 0x7f49d80b6790 2024-09-12T03:17:45.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:45.056+0000 7f49d57fa640 1 -- 172.21.15.27:0/2514129187 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49d0004230 con 0x7f49d80b6790 2024-09-12T03:17:45.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:45.056+0000 7f49dec65640 1 -- 172.21.15.27:0/2514129187 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49d81f7a40 con 0x7f49d80b6790 2024-09-12T03:17:45.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:45.056+0000 7f49d57fa640 1 -- 172.21.15.27:0/2514129187 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49d0003610 con 0x7f49d80b6790 2024-09-12T03:17:45.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:45.056+0000 7f49dec65640 1 -- 172.21.15.27:0/2514129187 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49d81f8000 con 0x7f49d80b6790 2024-09-12T03:17:45.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:45.057+0000 7f49dec65640 1 -- 172.21.15.27:0/2514129187 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49d80b75e0 con 0x7f49d80b6790 2024-09-12T03:17:45.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:45.058+0000 7f49d57fa640 1 -- 172.21.15.27:0/2514129187 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 7) ==== 90651+0+0 (secure 0 0 0) 0x7f49d00037b0 con 0x7f49d80b6790 2024-09-12T03:17:45.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:45.058+0000 7f49d57fa640 1 --2- 172.21.15.27:0/2514129187 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f49b806bfe0 0x7f49b806e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:45.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:45.058+0000 7f49d57fa640 1 -- 172.21.15.27:0/2514129187 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5709+0+0 (secure 0 0 0) 0x7f49d008acc0 con 0x7f49d80b6790 2024-09-12T03:17:45.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:45.058+0000 7f49dca01640 1 --2- 172.21.15.27:0/2514129187 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f49b806bfe0 0x7f49b806e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:45.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:45.059+0000 7f49dca01640 1 --2- 172.21.15.27:0/2514129187 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f49b806bfe0 0x7f49b806e4a0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f49c0034000 tx=0x7f49c002e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:45.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:45.063+0000 7f49d57fa640 1 -- 172.21.15.27:0/2514129187 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f49d0017020 con 0x7f49d80b6790 2024-09-12T03:17:45.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:45.273+0000 7f49dec65640 1 -- 172.21.15.27:0/2514129187 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f49d80b9d40 con 0x7f49d80b6790 2024-09-12T03:17:45.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:45.273+0000 7f49d57fa640 1 -- 172.21.15.27:0/2514129187 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v22) ==== 53+0+73 (secure 0 0 0) 0x7f49d0058dd0 con 0x7f49d80b6790 2024-09-12T03:17:45.273 INFO:teuthology.orchestra.run.smithi027.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-12T03:17:45.276 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:45.275+0000 7f49dec65640 1 -- 172.21.15.27:0/2514129187 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f49b806bfe0 msgr2=0x7f49b806e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:45.276 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:45.275+0000 7f49dec65640 1 --2- 172.21.15.27:0/2514129187 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f49b806bfe0 0x7f49b806e4a0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f49c0034000 tx=0x7f49c002e000 comp rx=0 tx=0).stop 2024-09-12T03:17:45.276 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:45.275+0000 7f49dec65640 1 -- 172.21.15.27:0/2514129187 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f49d80b6790 msgr2=0x7f49d815fb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:45.276 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:45.275+0000 7f49dec65640 1 --2- 172.21.15.27:0/2514129187 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f49d80b6790 0x7f49d815fb20 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f49d000e530 tx=0x7f49d000e9f0 comp rx=0 tx=0).stop 2024-09-12T03:17:45.276 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:45.275+0000 7f49dec65640 1 -- 172.21.15.27:0/2514129187 shutdown_connections 2024-09-12T03:17:45.276 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:45.275+0000 7f49dec65640 1 --2- 172.21.15.27:0/2514129187 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f49b806bfe0 0x7f49b806e4a0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:45.276 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:45.275+0000 7f49dec65640 1 --2- 172.21.15.27:0/2514129187 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f49d80b6790 0x7f49d815fb20 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:45.276 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:45.275+0000 7f49dec65640 1 --2- 172.21.15.27:0/2514129187 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f49d815eb00 0x7f49d80b6250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:45.276 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:45.275+0000 7f49dec65640 1 --2- 172.21.15.27:0/2514129187 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f49d80d3dd0 0x7f49d81530b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:45.276 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:45.275+0000 7f49dec65640 1 -- 172.21.15.27:0/2514129187 >> 172.21.15.27:0/2514129187 conn(0x7f49d80d4790 msgr2=0x7f49d8073260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:45.276 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:45.275+0000 7f49dec65640 1 -- 172.21.15.27:0/2514129187 shutdown_connections 2024-09-12T03:17:45.276 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:45.275+0000 7f49dec65640 1 -- 172.21.15.27:0/2514129187 wait complete. 2024-09-12T03:17:45.287 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-12T03:17:45.287 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-12T03:17:45.287 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi045.front.sepia.ceph.com 2024-09-12T03:17:45.287 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-12T03:17:45.287 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-12T03:17:45.287 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-12T03:17:45.287 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-12T03:17:45.288 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-12T03:17:45.288 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-12T03:17:45.288 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:17:45.288 DEBUG:teuthology.orchestra.run.smithi045:> ip addr 2024-09-12T03:17:45.308 INFO:teuthology.orchestra.run.smithi045.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-12T03:17:45.309 INFO:teuthology.orchestra.run.smithi045.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-12T03:17:45.309 INFO:teuthology.orchestra.run.smithi045.stdout: inet 127.0.0.1/8 scope host lo 2024-09-12T03:17:45.309 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:17:45.309 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 ::1/128 scope host 2024-09-12T03:17:45.309 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:17:45.309 INFO:teuthology.orchestra.run.smithi045.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-12T03:17:45.309 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:6b:fd:a6 brd ff:ff:ff:ff:ff:ff 2024-09-12T03:17:45.309 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp7s0f0 2024-09-12T03:17:45.309 INFO:teuthology.orchestra.run.smithi045.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-12T03:17:45.309 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:6b:fd:a7 brd ff:ff:ff:ff:ff:ff 2024-09-12T03:17:45.309 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp7s0f1 2024-09-12T03:17:45.309 INFO:teuthology.orchestra.run.smithi045.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-12T03:17:45.309 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:bd:12:be brd ff:ff:ff:ff:ff:ff 2024-09-12T03:17:45.310 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp1s0f0 2024-09-12T03:17:45.310 INFO:teuthology.orchestra.run.smithi045.stdout: inet 172.21.15.45/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-12T03:17:45.310 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft 42668sec preferred_lft 42668sec 2024-09-12T03:17:45.310 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::ec4:7aff:febd:12be/64 scope link 2024-09-12T03:17:45.310 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:17:45.310 INFO:teuthology.orchestra.run.smithi045.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-12T03:17:45.310 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:bd:12:bf brd ff:ff:ff:ff:ff:ff 2024-09-12T03:17:45.310 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp1s0f1 2024-09-12T03:17:45.310 INFO:teuthology.orchestra.run.smithi045.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-12T03:17:45.310 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 52:97:33:5c:60:1b brd ff:ff:ff:ff:ff:ff 2024-09-12T03:17:45.310 INFO:teuthology.orchestra.run.smithi045.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-12T03:17:45.310 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:17:45.310 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::d8c2:5dff:fe26:78c8/64 scope link 2024-09-12T03:17:45.310 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:17:45.310 INFO:teuthology.orchestra.run.smithi045.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-12T03:17:45.311 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 5a:ec:d7:40:ef:75 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-12T03:17:45.311 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::58ec:d7ff:fe40:ef75/64 scope link 2024-09-12T03:17:45.311 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:17:45.311 INFO:teuthology.orchestra.run.smithi045.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-12T03:17:45.311 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 52:97:33:5c:60:1b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-12T03:17:45.311 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::5097:33ff:fe5c:601b/64 scope link 2024-09-12T03:17:45.311 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:17:45.311 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:17:45.311 DEBUG:teuthology.orchestra.run.smithi045:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-12T03:17:45.378 INFO:teuthology.orchestra.run.smithi045.stdout:1 2024-09-12T03:17:45.380 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:17:45.380 DEBUG:teuthology.orchestra.run.smithi045:> route 2024-09-12T03:17:45.440 INFO:teuthology.orchestra.run.smithi045.stdout:Kernel IP routing table 2024-09-12T03:17:45.440 INFO:teuthology.orchestra.run.smithi045.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-12T03:17:45.440 INFO:teuthology.orchestra.run.smithi045.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-12T03:17:45.440 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-12T03:17:45.440 INFO:teuthology.orchestra.run.smithi045.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-12T03:17:45.441 DEBUG:teuthology.orchestra.run.smithi045:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-12T03:17:45.441 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-09-12T03:17:45.441 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-12T03:17:45.441 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-12T03:17:45.441 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-12T03:17:45.441 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-09-12T03:17:45.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:17:45 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T03:17:45.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:17:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T03:17:45.580 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:17:45.580 DEBUG:teuthology.orchestra.run.smithi045:> ip netns list 2024-09-12T03:17:45.597 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-09-12T03:17:45.597 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-12T03:17:45.597 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-12T03:17:45.597 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:17:45.597 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link set brx.0 up 2024-09-12T03:17:45.664 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-12T03:17:45.664 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-12T03:17:45.721 INFO:teuthology.orchestra.run.smithi045.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-12T03:17:45.721 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-12T03:17:45.721 DEBUG:teuthology.orchestra.run.smithi045:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-12T03:17:45.780 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2024-09-12T03:17:45.780 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:17:45.780 DEBUG:teuthology.orchestra.run.smithi045:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-12T03:17:45.865 INFO:teuthology.orchestra.run.smithi045.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-12T03:17:45.865 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-12T03:17:45.872 INFO:teuthology.orchestra.run.smithi045.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-12T03:17:45.872 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-12T03:17:45.872 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: resolved to: "172.21.15.27:3300,172.21.15.45:3300,172.21.15.45:3301" 2024-09-12T03:17:45.872 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: trying mount with new device syntax: 1@d2df504e-2647-4385-a5db-9fdf69f487a0.cephfs=/ 2024-09-12T03:17:45.872 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.27:3300/172.21.15.45:3300/172.21.15.45:3301" will pass to kernel 2024-09-12T03:17:45.873 INFO:teuthology.orchestra.run.smithi045.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-09-12T03:17:45.873 INFO:teuthology.orchestra.run.smithi045.stdout: You just mounted a file system that supports labels which does not 2024-09-12T03:17:45.873 INFO:teuthology.orchestra.run.smithi045.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-12T03:17:45.873 INFO:teuthology.orchestra.run.smithi045.stdout: applications will generate AVC messages and not be allowed access to 2024-09-12T03:17:45.873 INFO:teuthology.orchestra.run.smithi045.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-12T03:17:45.875 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-12T03:17:45.875 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:17:45.875 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-12T03:17:45.900 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-12T03:17:45.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:17:45 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-12T03:17:45.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:17:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T03:17:45.997 INFO:teuthology.orchestra.run.smithi045.stdout:client5358 2024-09-12T03:17:45.998 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-12T03:17:45.998 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T03:17:46.014 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-09-12T03:17:46.127 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:46.127+0000 7fef86586640 1 Processor -- start 2024-09-12T03:17:46.127 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:46.127+0000 7fef86586640 1 -- start start 2024-09-12T03:17:46.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:46.127+0000 7fef86586640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fef800b7f80 0x7fef80167170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:46.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:46.128+0000 7fef86586640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fef800d07b0 con 0x7fef800d1040 2024-09-12T03:17:46.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:46.128+0000 7fef86586640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fef800d0a80 con 0x7fef8015c6c0 2024-09-12T03:17:46.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:46.128+0000 7fef86586640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7fef8015caa0 con 0x7fef800b7f80 2024-09-12T03:17:46.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:46.128+0000 7fef84b23640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fef800b7f80 0x7fef80167170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:46.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:46.128+0000 7fef77fff640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fef800d1040 0x7fef800d14b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:40206/0 (socket says 172.21.15.27:40206) 2024-09-12T03:17:46.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:46.128+0000 7fef84b23640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fef800b7f80 0x7fef80167170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60510/0 (socket says 172.21.15.27:60510) 2024-09-12T03:17:46.129 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:46.128+0000 7fef77fff640 1 -- 172.21.15.27:0/532180451 learned_addr learned my addr 172.21.15.27:0/532180451 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:17:46.129 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:46.128+0000 7fef84b23640 1 -- 172.21.15.27:0/532180451 >> v1:172.21.15.45:6789/0 conn(0x7fef8015c6c0 legacy=0x7fef800d00a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:46.129 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:46.128+0000 7fef7f7fe640 1 -- 172.21.15.27:0/532180451 <== mon.1 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1603280759 0 0) 0x7fef800d0a80 con 0x7fef8015c6c0 2024-09-12T03:17:46.129 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:46.128+0000 7fef84b23640 1 -- 172.21.15.27:0/532180451 >> v1:172.21.15.27:6789/0 conn(0x7fef800d1040 legacy=0x7fef800d14b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:46.129 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:46.128+0000 7fef84b23640 1 -- 172.21.15.27:0/532180451 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef800d0d60 con 0x7fef800b7f80 2024-09-12T03:17:46.130 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:46.129+0000 7fef84b23640 1 --2- 172.21.15.27:0/532180451 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fef800b7f80 0x7fef80167170 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fef78009d90 tx=0x7fef7802a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd284dbd6d71d435 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:46.130 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:46.129+0000 7fef7f7fe640 1 -- 172.21.15.27:0/532180451 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef78007740 con 0x7fef800b7f80 2024-09-12T03:17:46.130 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:46.129+0000 7fef7f7fe640 1 -- 172.21.15.27:0/532180451 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef780078e0 con 0x7fef800b7f80 2024-09-12T03:17:46.130 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:46.129+0000 7fef7f7fe640 1 -- 172.21.15.27:0/532180451 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef78007cb0 con 0x7fef800b7f80 2024-09-12T03:17:46.130 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:46.129+0000 7fef86586640 1 -- 172.21.15.27:0/532180451 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fef800b7f80 msgr2=0x7fef80167170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:46.130 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:46.129+0000 7fef86586640 1 --2- 172.21.15.27:0/532180451 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fef800b7f80 0x7fef80167170 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fef78009d90 tx=0x7fef7802a660 comp rx=0 tx=0).stop 2024-09-12T03:17:46.130 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:46.130+0000 7fef86586640 1 -- 172.21.15.27:0/532180451 shutdown_connections 2024-09-12T03:17:46.131 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:46.130+0000 7fef86586640 1 --2- 172.21.15.27:0/532180451 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fef800b7f80 0x7fef80167170 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:46.131 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:46.130+0000 7fef86586640 1 -- 172.21.15.27:0/532180451 >> 172.21.15.27:0/532180451 conn(0x7fef8006f320 msgr2=0x7fef8006f700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:46.131 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:46.130+0000 7fef86586640 1 -- 172.21.15.27:0/532180451 shutdown_connections 2024-09-12T03:17:46.131 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:46.130+0000 7fef86586640 1 -- 172.21.15.27:0/532180451 wait complete. 2024-09-12T03:17:46.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:46.131+0000 7fef86586640 1 Processor -- start 2024-09-12T03:17:46.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:46.131+0000 7fef86586640 1 -- start start 2024-09-12T03:17:46.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:46.132+0000 7fef86586640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fef800b7f80 0x7fef801f2e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:46.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:46.132+0000 7fef86586640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fef800d1040 0x7fef801f5310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:46.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:46.132+0000 7fef86586640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef8015c6c0 0x7fef801f5850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:46.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:46.132+0000 7fef86586640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fef800b6a50 con 0x7fef800d1040 2024-09-12T03:17:46.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:46.132+0000 7fef86586640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fef800b6510 con 0x7fef8015c6c0 2024-09-12T03:17:46.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:46.132+0000 7fef86586640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fef800b4400 con 0x7fef800b7f80 2024-09-12T03:17:46.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:46.132+0000 7fef77fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fef800d1040 0x7fef801f5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:46.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:46.132+0000 7fef77fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fef800d1040 0x7fef801f5310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60518/0 (socket says 172.21.15.27:60518) 2024-09-12T03:17:46.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:46.132+0000 7fef77fff640 1 -- 172.21.15.27:0/3982542441 learned_addr learned my addr 172.21.15.27:0/3982542441 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:17:46.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:46.132+0000 7fef7ffff640 1 --2- 172.21.15.27:0/3982542441 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fef800b7f80 0x7fef801f2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:46.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:46.132+0000 7fef84b23640 1 --2- 172.21.15.27:0/3982542441 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef8015c6c0 0x7fef801f5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:46.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:46.132+0000 7fef77fff640 1 -- 172.21.15.27:0/3982542441 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fef800b7f80 msgr2=0x7fef801f2e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:46.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:46.132+0000 7fef77fff640 1 --2- 172.21.15.27:0/3982542441 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fef800b7f80 0x7fef801f2e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:46.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:46.132+0000 7fef77fff640 1 -- 172.21.15.27:0/3982542441 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef8015c6c0 msgr2=0x7fef801f5850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:46.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:46.132+0000 7fef77fff640 1 --2- 172.21.15.27:0/3982542441 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef8015c6c0 0x7fef801f5850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:46.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:46.132+0000 7fef77fff640 1 -- 172.21.15.27:0/3982542441 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef800cc560 con 0x7fef800d1040 2024-09-12T03:17:46.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:46.132+0000 7fef7ffff640 1 --2- 172.21.15.27:0/3982542441 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fef800b7f80 0x7fef801f2e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:17:46.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:46.132+0000 7fef77fff640 1 --2- 172.21.15.27:0/3982542441 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fef800d1040 0x7fef801f5310 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fef6800c430 tx=0x7fef6800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:46.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:46.132+0000 7fef7d7fa640 1 -- 172.21.15.27:0/3982542441 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef68017070 con 0x7fef800d1040 2024-09-12T03:17:46.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:46.132+0000 7fef7d7fa640 1 -- 172.21.15.27:0/3982542441 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef68005ce0 con 0x7fef800d1040 2024-09-12T03:17:46.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:46.132+0000 7fef86586640 1 -- 172.21.15.27:0/3982542441 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef800cc840 con 0x7fef800d1040 2024-09-12T03:17:46.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:46.132+0000 7fef7d7fa640 1 -- 172.21.15.27:0/3982542441 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef680072f0 con 0x7fef800d1040 2024-09-12T03:17:46.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:46.132+0000 7fef86586640 1 -- 172.21.15.27:0/3982542441 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef800cce00 con 0x7fef800d1040 2024-09-12T03:17:46.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:46.133+0000 7fef86586640 1 -- 172.21.15.27:0/3982542441 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef800d0080 con 0x7fef800d1040 2024-09-12T03:17:46.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:46.134+0000 7fef7d7fa640 1 -- 172.21.15.27:0/3982542441 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 7) ==== 90651+0+0 (secure 0 0 0) 0x7fef6801a070 con 0x7fef800d1040 2024-09-12T03:17:46.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:46.134+0000 7fef7d7fa640 1 --2- 172.21.15.27:0/3982542441 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fef6006c010 0x7fef6006e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:46.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:46.134+0000 7fef7d7fa640 1 -- 172.21.15.27:0/3982542441 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5709+0+0 (secure 0 0 0) 0x7fef6808a510 con 0x7fef800d1040 2024-09-12T03:17:46.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:46.135+0000 7fef7ffff640 1 --2- 172.21.15.27:0/3982542441 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fef6006c010 0x7fef6006e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:46.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:46.135+0000 7fef7ffff640 1 --2- 172.21.15.27:0/3982542441 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fef6006c010 0x7fef6006e4d0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fef70000f60 tx=0x7fef70009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:46.138 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:46.139+0000 7fef7d7fa640 1 -- 172.21.15.27:0/3982542441 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fef68058cd0 con 0x7fef800d1040 2024-09-12T03:17:46.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:46.351+0000 7fef86586640 1 -- 172.21.15.27:0/3982542441 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7fef80075040 con 0x7fef800d1040 2024-09-12T03:17:47.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:47.243+0000 7fef7d7fa640 1 -- 172.21.15.27:0/3982542441 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v23) ==== 136+0+0 (secure 0 0 0) 0x7fef680585d0 con 0x7fef800d1040 2024-09-12T03:17:47.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:47.245+0000 7fef86586640 1 -- 172.21.15.27:0/3982542441 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fef6006c010 msgr2=0x7fef6006e4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:47.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:47.245+0000 7fef86586640 1 --2- 172.21.15.27:0/3982542441 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fef6006c010 0x7fef6006e4d0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fef70000f60 tx=0x7fef70009040 comp rx=0 tx=0).stop 2024-09-12T03:17:47.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:47.245+0000 7fef86586640 1 -- 172.21.15.27:0/3982542441 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fef800d1040 msgr2=0x7fef801f5310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:47.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:47.245+0000 7fef86586640 1 --2- 172.21.15.27:0/3982542441 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fef800d1040 0x7fef801f5310 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fef6800c430 tx=0x7fef6800c8f0 comp rx=0 tx=0).stop 2024-09-12T03:17:47.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:47.245+0000 7fef86586640 1 -- 172.21.15.27:0/3982542441 shutdown_connections 2024-09-12T03:17:47.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:47.245+0000 7fef86586640 1 --2- 172.21.15.27:0/3982542441 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fef6006c010 0x7fef6006e4d0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:47.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:47.245+0000 7fef86586640 1 --2- 172.21.15.27:0/3982542441 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef8015c6c0 0x7fef801f5850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:47.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:47.245+0000 7fef86586640 1 --2- 172.21.15.27:0/3982542441 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fef800d1040 0x7fef801f5310 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:47.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:47.245+0000 7fef86586640 1 --2- 172.21.15.27:0/3982542441 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fef800b7f80 0x7fef801f2e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:47.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:47.245+0000 7fef86586640 1 -- 172.21.15.27:0/3982542441 >> 172.21.15.27:0/3982542441 conn(0x7fef8006f320 msgr2=0x7fef800cf730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:47.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:47.245+0000 7fef86586640 1 -- 172.21.15.27:0/3982542441 shutdown_connections 2024-09-12T03:17:47.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:47.245+0000 7fef86586640 1 -- 172.21.15.27:0/3982542441 wait complete. 2024-09-12T03:17:47.259 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-12T03:17:47.367 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:47.367+0000 7f979dd33640 1 Processor -- start 2024-09-12T03:17:47.367 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:47.367+0000 7f979dd33640 1 -- start start 2024-09-12T03:17:47.369 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:47.368+0000 7f979dd33640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f97980b1ca0 0x7f979815f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:47.370 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:47.368+0000 7f979dd33640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f97980cf660 0x7f97980d2780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:47.370 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:47.368+0000 7f979dd33640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f979815fd70 con 0x7f979806f360 2024-09-12T03:17:47.370 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:47.368+0000 7f979dd33640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f97980d22f0 con 0x7f97980cf660 2024-09-12T03:17:47.370 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:47.368+0000 7f979dd33640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f979806cc40 con 0x7f97980b1ca0 2024-09-12T03:17:47.370 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:47.368+0000 7f9797fff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f97980cf660 0x7f97980d2780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:47.370 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:47.368+0000 7f9797fff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f97980cf660 0x7f97980d2780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60522/0 (socket says 172.21.15.27:60522) 2024-09-12T03:17:47.370 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:47.368+0000 7f9797fff640 1 -- 172.21.15.27:0/1502186282 learned_addr learned my addr 172.21.15.27:0/1502186282 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:17:47.370 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:47.369+0000 7f9797fff640 1 -- 172.21.15.27:0/1502186282 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f97980b1ca0 msgr2=0x7f979815f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:47.370 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:47.369+0000 7f9797fff640 1 --2- 172.21.15.27:0/1502186282 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f97980b1ca0 0x7f979815f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:47.370 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:47.369+0000 7f9797fff640 1 -- 172.21.15.27:0/1502186282 >> v1:172.21.15.45:6789/0 conn(0x7f979806f360 legacy=0x7f97980cef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:47.370 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:47.369+0000 7f9797fff640 1 -- 172.21.15.27:0/1502186282 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97980d1ce0 con 0x7f97980cf660 2024-09-12T03:17:47.370 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:47.369+0000 7f9797fff640 1 --2- 172.21.15.27:0/1502186282 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f97980cf660 0x7f97980d2780 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f9790009e80 tx=0x7f979002a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fcf5aba2d5ec03a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:47.371 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:47.369+0000 7f9796ffd640 1 -- 172.21.15.27:0/1502186282 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9790036070 con 0x7f97980cf660 2024-09-12T03:17:47.371 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:47.370+0000 7f9796ffd640 1 -- 172.21.15.27:0/1502186282 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9790031600 con 0x7f97980cf660 2024-09-12T03:17:47.371 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:47.370+0000 7f9796ffd640 1 -- 172.21.15.27:0/1502186282 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97900319d0 con 0x7f97980cf660 2024-09-12T03:17:47.372 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:47.370+0000 7f979dd33640 1 -- 172.21.15.27:0/1502186282 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f97980cf660 msgr2=0x7f97980d2780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:47.372 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:47.370+0000 7f979dd33640 1 --2- 172.21.15.27:0/1502186282 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f97980cf660 0x7f97980d2780 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f9790009e80 tx=0x7f979002a6f0 comp rx=0 tx=0).stop 2024-09-12T03:17:47.372 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:47.370+0000 7f979dd33640 1 -- 172.21.15.27:0/1502186282 shutdown_connections 2024-09-12T03:17:47.372 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:47.370+0000 7f979dd33640 1 --2- 172.21.15.27:0/1502186282 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f97980cf660 0x7f97980d2780 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:47.372 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:47.370+0000 7f979dd33640 1 --2- 172.21.15.27:0/1502186282 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f97980b1ca0 0x7f979815f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:47.372 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:47.370+0000 7f979dd33640 1 -- 172.21.15.27:0/1502186282 >> 172.21.15.27:0/1502186282 conn(0x7f9798073800 msgr2=0x7f979809ba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:47.372 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:47.370+0000 7f979dd33640 1 -- 172.21.15.27:0/1502186282 shutdown_connections 2024-09-12T03:17:47.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:47.370+0000 7f979dd33640 1 -- 172.21.15.27:0/1502186282 wait complete. 2024-09-12T03:17:47.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:47.371+0000 7f979dd33640 1 Processor -- start 2024-09-12T03:17:47.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:47.372+0000 7f979dd33640 1 -- start start 2024-09-12T03:17:47.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:47.372+0000 7f979dd33640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f979806f360 0x7f97980cb850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:47.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:47.372+0000 7f979dd33640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f97980b1ca0 0x7f979814bff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:47.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:47.372+0000 7f979dd33640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f97980cf660 0x7f97980b6660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:47.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:47.372+0000 7f979dd33640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f979806d1c0 con 0x7f97980b1ca0 2024-09-12T03:17:47.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:47.372+0000 7f979dd33640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f97980d1f80 con 0x7f97980cf660 2024-09-12T03:17:47.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:47.372+0000 7f979dd33640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f979806d340 con 0x7f979806f360 2024-09-12T03:17:47.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:47.372+0000 7f978effd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f97980b1ca0 0x7f979814bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:47.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:47.372+0000 7f978effd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f97980b1ca0 0x7f979814bff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60538/0 (socket says 172.21.15.27:60538) 2024-09-12T03:17:47.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:47.372+0000 7f978effd640 1 -- 172.21.15.27:0/3977061819 learned_addr learned my addr 172.21.15.27:0/3977061819 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:17:47.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:47.373+0000 7f97977fe640 1 --2- 172.21.15.27:0/3977061819 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f979806f360 0x7f97980cb850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:47.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:47.373+0000 7f9797fff640 1 --2- 172.21.15.27:0/3977061819 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f97980cf660 0x7f97980b6660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:47.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:47.373+0000 7f978effd640 1 -- 172.21.15.27:0/3977061819 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f979806f360 msgr2=0x7f97980cb850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:47.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:47.373+0000 7f978effd640 1 --2- 172.21.15.27:0/3977061819 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f979806f360 0x7f97980cb850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:47.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:47.373+0000 7f978effd640 1 -- 172.21.15.27:0/3977061819 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f97980cf660 msgr2=0x7f97980b6660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:47.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:47.373+0000 7f978effd640 1 --2- 172.21.15.27:0/3977061819 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f97980cf660 0x7f97980b6660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:47.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:47.373+0000 7f978effd640 1 -- 172.21.15.27:0/3977061819 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f979814c530 con 0x7f97980b1ca0 2024-09-12T03:17:47.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:47.373+0000 7f97977fe640 1 --2- 172.21.15.27:0/3977061819 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f979806f360 0x7f97980cb850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:17:47.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:47.373+0000 7f9797fff640 1 --2- 172.21.15.27:0/3977061819 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f97980cf660 0x7f97980b6660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:17:47.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:47.373+0000 7f978effd640 1 --2- 172.21.15.27:0/3977061819 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f97980b1ca0 0x7f979814bff0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f978000c430 tx=0x7f978000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:47.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:47.373+0000 7f9794ff9640 1 -- 172.21.15.27:0/3977061819 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9780011790 con 0x7f97980b1ca0 2024-09-12T03:17:47.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:47.373+0000 7f9794ff9640 1 -- 172.21.15.27:0/3977061819 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9780017070 con 0x7f97980b1ca0 2024-09-12T03:17:47.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:47.373+0000 7f979dd33640 1 -- 172.21.15.27:0/3977061819 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97980b6ba0 con 0x7f97980b1ca0 2024-09-12T03:17:47.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:47.373+0000 7f9794ff9640 1 -- 172.21.15.27:0/3977061819 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9780011b70 con 0x7f97980b1ca0 2024-09-12T03:17:47.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:47.373+0000 7f979dd33640 1 -- 172.21.15.27:0/3977061819 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97980cc180 con 0x7f97980b1ca0 2024-09-12T03:17:47.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:47.374+0000 7f979dd33640 1 -- 172.21.15.27:0/3977061819 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97980cfc40 con 0x7f97980b1ca0 2024-09-12T03:17:47.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:47.375+0000 7f9794ff9640 1 -- 172.21.15.27:0/3977061819 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 7) ==== 90651+0+0 (secure 0 0 0) 0x7f9780011d10 con 0x7f97980b1ca0 2024-09-12T03:17:47.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:47.375+0000 7f9794ff9640 1 --2- 172.21.15.27:0/3977061819 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f977806bfe0 0x7f977806e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:47.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:47.375+0000 7f9794ff9640 1 -- 172.21.15.27:0/3977061819 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5709+0+0 (secure 0 0 0) 0x7f978008afd0 con 0x7f97980b1ca0 2024-09-12T03:17:47.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:47.375+0000 7f97977fe640 1 --2- 172.21.15.27:0/3977061819 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f977806bfe0 0x7f977806e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:47.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:47.376+0000 7f97977fe640 1 --2- 172.21.15.27:0/3977061819 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f977806bfe0 0x7f977806e4a0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f9788006fd0 tx=0x7f9788008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:47.379 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:47.379+0000 7f9794ff9640 1 -- 172.21.15.27:0/3977061819 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f979814c530 con 0x7f97980b1ca0 2024-09-12T03:17:47.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:47.579+0000 7f979dd33640 1 -- 172.21.15.27:0/3977061819 --> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f9798161d50 con 0x7f977806bfe0 2024-09-12T03:17:47.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:47.581+0000 7f9794ff9640 1 -- 172.21.15.27:0/3977061819 <== mgr.4102 v2:172.21.15.45:6832/3923821219 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f9798161d50 con 0x7f977806bfe0 2024-09-12T03:17:47.581 INFO:teuthology.orchestra.run.smithi027.stdout:[ 2024-09-12T03:17:47.581 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:17:47.581 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "cephfs" 2024-09-12T03:17:47.581 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:17:47.581 INFO:teuthology.orchestra.run.smithi027.stdout:] 2024-09-12T03:17:47.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:47.583+0000 7f979dd33640 1 -- 172.21.15.27:0/3977061819 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f977806bfe0 msgr2=0x7f977806e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:47.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:47.583+0000 7f979dd33640 1 --2- 172.21.15.27:0/3977061819 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f977806bfe0 0x7f977806e4a0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f9788006fd0 tx=0x7f9788008040 comp rx=0 tx=0).stop 2024-09-12T03:17:47.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:47.583+0000 7f979dd33640 1 -- 172.21.15.27:0/3977061819 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f97980b1ca0 msgr2=0x7f979814bff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:47.584 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:47.583+0000 7f979dd33640 1 --2- 172.21.15.27:0/3977061819 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f97980b1ca0 0x7f979814bff0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f978000c430 tx=0x7f978000c8f0 comp rx=0 tx=0).stop 2024-09-12T03:17:47.584 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:47.583+0000 7f979dd33640 1 -- 172.21.15.27:0/3977061819 shutdown_connections 2024-09-12T03:17:47.584 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:47.583+0000 7f979dd33640 1 --2- 172.21.15.27:0/3977061819 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f977806bfe0 0x7f977806e4a0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:47.584 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:47.583+0000 7f979dd33640 1 --2- 172.21.15.27:0/3977061819 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f97980cf660 0x7f97980b6660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:47.584 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:47.583+0000 7f979dd33640 1 --2- 172.21.15.27:0/3977061819 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f97980b1ca0 0x7f979814bff0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:47.584 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:47.583+0000 7f979dd33640 1 --2- 172.21.15.27:0/3977061819 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f979806f360 0x7f97980cb850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:47.584 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:47.583+0000 7f979dd33640 1 -- 172.21.15.27:0/3977061819 >> 172.21.15.27:0/3977061819 conn(0x7f9798073800 msgr2=0x7f97980cef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:47.584 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:47.583+0000 7f979dd33640 1 -- 172.21.15.27:0/3977061819 shutdown_connections 2024-09-12T03:17:47.584 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:47.583+0000 7f979dd33640 1 -- 172.21.15.27:0/3977061819 wait complete. 2024-09-12T03:17:47.595 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-12T03:17:47.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:47.701+0000 7fc357d31640 1 Processor -- start 2024-09-12T03:17:47.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:47.701+0000 7fc357d31640 1 -- start start 2024-09-12T03:17:47.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:47.702+0000 7fc357d31640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc35006cc40 0x7fc35006d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:47.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:47.702+0000 7fc357d31640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fc35006db30 0x7fc35006df60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:47.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:47.702+0000 7fc357d31640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc35006d5d0 con 0x7fc35015e650 2024-09-12T03:17:47.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:47.702+0000 7fc357d31640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fc35006d850 con 0x7fc35006cc40 2024-09-12T03:17:47.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:47.702+0000 7fc357d31640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fc3500d1260 con 0x7fc35006db30 2024-09-12T03:17:47.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:47.702+0000 7fc3552cc640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc35006cc40 0x7fc35006d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:47.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:47.702+0000 7fc3562ce640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fc35006db30 0x7fc35006df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:47.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:47.702+0000 7fc355acd640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fc35015e650 0x7fc3500d1d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.27:41672/0 (socket says 172.21.15.27:41672) 2024-09-12T03:17:47.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:47.702+0000 7fc355acd640 1 -- 172.21.15.27:0/3985340128 learned_addr learned my addr 172.21.15.27:0/3985340128 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:17:47.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:47.703+0000 7fc3552cc640 1 -- 172.21.15.27:0/3985340128 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fc35006db30 msgr2=0x7fc35006df60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:47.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:47.703+0000 7fc3552cc640 1 --2- 172.21.15.27:0/3985340128 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fc35006db30 0x7fc35006df60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:47.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:47.703+0000 7fc3552cc640 1 -- 172.21.15.27:0/3985340128 >> v1:172.21.15.45:6789/0 conn(0x7fc35015e650 legacy=0x7fc3500d1d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:47.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:47.703+0000 7fc3552cc640 1 -- 172.21.15.27:0/3985340128 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3500d1540 con 0x7fc35006cc40 2024-09-12T03:17:47.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:47.703+0000 7fc3552cc640 1 --2- 172.21.15.27:0/3985340128 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc35006cc40 0x7fc35006d090 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fc3400097a0 tx=0x7fc34002a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1803a66a4f36f492 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:47.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:47.704+0000 7fc354acb640 1 -- 172.21.15.27:0/3985340128 <== mon.1 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc340037070 con 0x7fc35006cc40 2024-09-12T03:17:47.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:47.704+0000 7fc354acb640 1 -- 172.21.15.27:0/3985340128 <== mon.1 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc34002ae50 con 0x7fc35006cc40 2024-09-12T03:17:47.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:47.704+0000 7fc354acb640 1 -- 172.21.15.27:0/3985340128 <== mon.1 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc340032660 con 0x7fc35006cc40 2024-09-12T03:17:47.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:47.704+0000 7fc357d31640 1 -- 172.21.15.27:0/3985340128 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc35006cc40 msgr2=0x7fc35006d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:47.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:47.704+0000 7fc357d31640 1 --2- 172.21.15.27:0/3985340128 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc35006cc40 0x7fc35006d090 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fc3400097a0 tx=0x7fc34002a100 comp rx=0 tx=0).stop 2024-09-12T03:17:47.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:47.704+0000 7fc357d31640 1 -- 172.21.15.27:0/3985340128 shutdown_connections 2024-09-12T03:17:47.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:47.704+0000 7fc357d31640 1 --2- 172.21.15.27:0/3985340128 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fc35006db30 0x7fc35006df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:47.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:47.704+0000 7fc357d31640 1 --2- 172.21.15.27:0/3985340128 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc35006cc40 0x7fc35006d090 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:47.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:47.704+0000 7fc357d31640 1 -- 172.21.15.27:0/3985340128 >> 172.21.15.27:0/3985340128 conn(0x7fc35006f320 msgr2=0x7fc35006f700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:47.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:47.704+0000 7fc357d31640 1 -- 172.21.15.27:0/3985340128 shutdown_connections 2024-09-12T03:17:47.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:47.704+0000 7fc357d31640 1 -- 172.21.15.27:0/3985340128 wait complete. 2024-09-12T03:17:47.707 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:47.705+0000 7fc357d31640 1 Processor -- start 2024-09-12T03:17:47.707 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:47.705+0000 7fc357d31640 1 -- start start 2024-09-12T03:17:47.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:47.706+0000 7fc357d31640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fc35006cc40 0x7fc3500cfaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:47.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:47.706+0000 7fc357d31640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc35006db30 0x7fc350150e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:47.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:47.706+0000 7fc357d31640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc35015e650 0x7fc3500cdd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:47.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:47.706+0000 7fc357d31640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fc35006e620 con 0x7fc35006db30 2024-09-12T03:17:47.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:47.706+0000 7fc357d31640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc35006e4a0 con 0x7fc35015e650 2024-09-12T03:17:47.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:47.706+0000 7fc357d31640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fc3500b55b0 con 0x7fc35006cc40 2024-09-12T03:17:47.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:47.706+0000 7fc3552cc640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc35006db30 0x7fc350150e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:47.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:47.706+0000 7fc3552cc640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc35006db30 0x7fc350150e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60554/0 (socket says 172.21.15.27:60554) 2024-09-12T03:17:47.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:47.706+0000 7fc3552cc640 1 -- 172.21.15.27:0/2592912551 learned_addr learned my addr 172.21.15.27:0/2592912551 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:17:47.710 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:47.706+0000 7fc355acd640 1 --2- 172.21.15.27:0/2592912551 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fc35006cc40 0x7fc3500cfaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:47.710 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:47.706+0000 7fc3562ce640 1 --2- 172.21.15.27:0/2592912551 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc35015e650 0x7fc3500cdd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:47.710 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:47.707+0000 7fc3552cc640 1 -- 172.21.15.27:0/2592912551 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fc35006cc40 msgr2=0x7fc3500cfaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:47.710 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:47.707+0000 7fc3552cc640 1 --2- 172.21.15.27:0/2592912551 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fc35006cc40 0x7fc3500cfaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:47.710 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:47.707+0000 7fc3552cc640 1 -- 172.21.15.27:0/2592912551 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc35015e650 msgr2=0x7fc3500cdd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:47.710 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:47.707+0000 7fc3552cc640 1 --2- 172.21.15.27:0/2592912551 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc35015e650 0x7fc3500cdd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:47.710 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:47.707+0000 7fc3552cc640 1 -- 172.21.15.27:0/2592912551 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3500cc980 con 0x7fc35006db30 2024-09-12T03:17:47.710 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:47.707+0000 7fc3552cc640 1 --2- 172.21.15.27:0/2592912551 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc35006db30 0x7fc350150e30 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fc340035040 tx=0x7fc3400046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:47.710 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:47.707+0000 7fc3467fc640 1 -- 172.21.15.27:0/2592912551 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc340041070 con 0x7fc35006db30 2024-09-12T03:17:47.710 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:47.707+0000 7fc3467fc640 1 -- 172.21.15.27:0/2592912551 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc34003c460 con 0x7fc35006db30 2024-09-12T03:17:47.711 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:47.707+0000 7fc3467fc640 1 -- 172.21.15.27:0/2592912551 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc340037040 con 0x7fc35006db30 2024-09-12T03:17:47.711 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:47.707+0000 7fc357d31640 1 -- 172.21.15.27:0/2592912551 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3500ccc60 con 0x7fc35006db30 2024-09-12T03:17:47.711 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:47.707+0000 7fc357d31640 1 -- 172.21.15.27:0/2592912551 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3500cd220 con 0x7fc35006db30 2024-09-12T03:17:47.711 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:47.708+0000 7fc357d31640 1 -- 172.21.15.27:0/2592912551 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc35006d090 con 0x7fc35006db30 2024-09-12T03:17:47.711 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:47.709+0000 7fc3467fc640 1 -- 172.21.15.27:0/2592912551 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 7) ==== 90651+0+0 (secure 0 0 0) 0x7fc34003a070 con 0x7fc35006db30 2024-09-12T03:17:47.712 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:47.709+0000 7fc3467fc640 1 --2- 172.21.15.27:0/2592912551 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fc33406bfe0 0x7fc33406e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:47.712 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:47.709+0000 7fc3467fc640 1 -- 172.21.15.27:0/2592912551 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5709+0+0 (secure 0 0 0) 0x7fc3400ab550 con 0x7fc35006db30 2024-09-12T03:17:47.712 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:47.709+0000 7fc355acd640 1 --2- 172.21.15.27:0/2592912551 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fc33406bfe0 0x7fc33406e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:47.712 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:47.710+0000 7fc355acd640 1 --2- 172.21.15.27:0/2592912551 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fc33406bfe0 0x7fc33406e4a0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fc34c007fd0 tx=0x7fc34c007e80 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:47.714 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:47.714+0000 7fc3467fc640 1 -- 172.21.15.27:0/2592912551 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fc340079770 con 0x7fc35006db30 2024-09-12T03:17:47.890 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:47.890+0000 7fc357d31640 1 -- 172.21.15.27:0/2592912551 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fc3500b67a0 con 0x7fc35006db30 2024-09-12T03:17:47.890 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:47.890+0000 7fc3467fc640 1 -- 172.21.15.27:0/2592912551 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v12)=0 v12) ==== 122+0+0 (secure 0 0 0) 0x7fc34007da70 con 0x7fc35006db30 2024-09-12T03:17:47.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:47.892+0000 7fc357d31640 1 -- 172.21.15.27:0/2592912551 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fc33406bfe0 msgr2=0x7fc33406e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:47.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:47.892+0000 7fc357d31640 1 --2- 172.21.15.27:0/2592912551 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fc33406bfe0 0x7fc33406e4a0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fc34c007fd0 tx=0x7fc34c007e80 comp rx=0 tx=0).stop 2024-09-12T03:17:47.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:47.892+0000 7fc357d31640 1 -- 172.21.15.27:0/2592912551 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc35006db30 msgr2=0x7fc350150e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:47.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:47.892+0000 7fc357d31640 1 --2- 172.21.15.27:0/2592912551 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc35006db30 0x7fc350150e30 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fc340035040 tx=0x7fc3400046c0 comp rx=0 tx=0).stop 2024-09-12T03:17:47.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:47.892+0000 7fc357d31640 1 -- 172.21.15.27:0/2592912551 shutdown_connections 2024-09-12T03:17:47.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:47.892+0000 7fc357d31640 1 --2- 172.21.15.27:0/2592912551 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fc33406bfe0 0x7fc33406e4a0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:47.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:47.892+0000 7fc357d31640 1 --2- 172.21.15.27:0/2592912551 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc35015e650 0x7fc3500cdd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:47.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:47.892+0000 7fc357d31640 1 --2- 172.21.15.27:0/2592912551 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc35006db30 0x7fc350150e30 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:47.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:47.892+0000 7fc357d31640 1 --2- 172.21.15.27:0/2592912551 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fc35006cc40 0x7fc3500cfaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:47.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:47.892+0000 7fc357d31640 1 -- 172.21.15.27:0/2592912551 >> 172.21.15.27:0/2592912551 conn(0x7fc35006f320 msgr2=0x7fc3501619a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:47.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:47.892+0000 7fc357d31640 1 -- 172.21.15.27:0/2592912551 shutdown_connections 2024-09-12T03:17:47.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:47.892+0000 7fc357d31640 1 -- 172.21.15.27:0/2592912551 wait complete. 2024-09-12T03:17:47.904 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-12T03:17:47.904 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T03:17:47.920 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-12T03:17:47.920 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T03:17:47.976 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-12T03:17:48.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:48.119+0000 7f2c3a15f640 1 Processor -- start 2024-09-12T03:17:48.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:48.119+0000 7f2c3a15f640 1 -- start start 2024-09-12T03:17:48.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:48.120+0000 7f2c3a15f640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f2c34158cf0 0x7f2c340b55d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:48.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:48.120+0000 7f2c3a15f640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f2c341590d0 0x7f2c34162430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:48.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:48.120+0000 7f2c3a15f640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2c3415f600 0x7f2c3406bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:48.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:48.120+0000 7f2c3a15f640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f2c34162970 con 0x7f2c341590d0 2024-09-12T03:17:48.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:48.120+0000 7f2c3a15f640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f2c34162c10 con 0x7f2c34158cf0 2024-09-12T03:17:48.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:48.120+0000 7f2c3a15f640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f2c34162ef0 con 0x7f2c3415f600 2024-09-12T03:17:48.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:48.120+0000 7f2c32ffd640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f2c341590d0 0x7f2c34162430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:48.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:48.120+0000 7f2c32ffd640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f2c341590d0 0x7f2c34162430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60570/0 (socket says 172.21.15.27:60570) 2024-09-12T03:17:48.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:48.120+0000 7f2c32ffd640 1 -- 172.21.15.27:0/1769238778 learned_addr learned my addr 172.21.15.27:0/1769238778 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:17:48.122 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:48.121+0000 7f2c337fe640 1 --2- 172.21.15.27:0/1769238778 >> v2:172.21.15.45:3300/0 conn(0x7f2c34158cf0 0x7f2c340b55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:48.122 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:48.121+0000 7f2c33fff640 1 --2- 172.21.15.27:0/1769238778 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2c3415f600 0x7f2c3406bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:48.122 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:48.121+0000 7f2c32ffd640 1 -- 172.21.15.27:0/1769238778 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2c3415f600 msgr2=0x7f2c3406bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:48.122 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:48.121+0000 7f2c32ffd640 1 --2- 172.21.15.27:0/1769238778 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2c3415f600 0x7f2c3406bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:48.122 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:48.121+0000 7f2c32ffd640 1 -- 172.21.15.27:0/1769238778 >> v2:172.21.15.45:3300/0 conn(0x7f2c34158cf0 msgr2=0x7f2c340b55d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:48.122 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:48.121+0000 7f2c32ffd640 1 --2- 172.21.15.27:0/1769238778 >> v2:172.21.15.45:3300/0 conn(0x7f2c34158cf0 0x7f2c340b55d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:48.122 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:48.121+0000 7f2c32ffd640 1 -- 172.21.15.27:0/1769238778 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c341631d0 con 0x7f2c341590d0 2024-09-12T03:17:48.123 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:48.121+0000 7f2c32ffd640 1 --2- 172.21.15.27:0/1769238778 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2c341590d0 0x7f2c34162430 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f2c340733c0 tx=0x7f2c2802a0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d942f51d10cc20ba server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:48.123 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:48.122+0000 7f2c327fc640 1 -- 172.21.15.27:0/1769238778 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c28037070 con 0x7f2c341590d0 2024-09-12T03:17:48.124 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:48.122+0000 7f2c327fc640 1 -- 172.21.15.27:0/1769238778 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c2802ae00 con 0x7f2c341590d0 2024-09-12T03:17:48.124 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:48.122+0000 7f2c327fc640 1 -- 172.21.15.27:0/1769238778 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c28032660 con 0x7f2c341590d0 2024-09-12T03:17:48.124 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:48.122+0000 7f2c3a15f640 1 -- 172.21.15.27:0/1769238778 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2c341590d0 msgr2=0x7f2c34162430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:48.124 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:48.122+0000 7f2c3a15f640 1 --2- 172.21.15.27:0/1769238778 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2c341590d0 0x7f2c34162430 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f2c340733c0 tx=0x7f2c2802a0b0 comp rx=0 tx=0).stop 2024-09-12T03:17:48.124 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:48.122+0000 7f2c3a15f640 1 -- 172.21.15.27:0/1769238778 shutdown_connections 2024-09-12T03:17:48.124 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:48.122+0000 7f2c3a15f640 1 --2- 172.21.15.27:0/1769238778 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2c3415f600 0x7f2c3406bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:48.124 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:48.122+0000 7f2c3a15f640 1 --2- 172.21.15.27:0/1769238778 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2c341590d0 0x7f2c34162430 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:48.124 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:48.122+0000 7f2c3a15f640 1 --2- 172.21.15.27:0/1769238778 >> v2:172.21.15.45:3300/0 conn(0x7f2c34158cf0 0x7f2c340b55d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:48.125 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:48.122+0000 7f2c3a15f640 1 -- 172.21.15.27:0/1769238778 >> 172.21.15.27:0/1769238778 conn(0x7f2c3406cc40 msgr2=0x7f2c3406d050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:48.125 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:48.122+0000 7f2c3a15f640 1 -- 172.21.15.27:0/1769238778 shutdown_connections 2024-09-12T03:17:48.125 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:48.122+0000 7f2c3a15f640 1 -- 172.21.15.27:0/1769238778 wait complete. 2024-09-12T03:17:48.125 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:48.123+0000 7f2c3a15f640 1 Processor -- start 2024-09-12T03:17:48.125 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:48.124+0000 7f2c3a15f640 1 -- start start 2024-09-12T03:17:48.127 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:48.124+0000 7f2c3a15f640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2c34158cf0 0x7f2c341f44e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:48.127 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:48.124+0000 7f2c3a15f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2c341590d0 0x7f2c34152520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:48.127 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:48.124+0000 7f2c3a15f640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2c3415f600 0x7f2c3415dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:48.127 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:48.124+0000 7f2c3a15f640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f2c3415fbf0 con 0x7f2c34158cf0 2024-09-12T03:17:48.127 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:48.124+0000 7f2c3a15f640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f2c3415fa70 con 0x7f2c341590d0 2024-09-12T03:17:48.127 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:48.124+0000 7f2c3a15f640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f2c3415fd70 con 0x7f2c3415f600 2024-09-12T03:17:48.127 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:48.125+0000 7f2c337fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2c34158cf0 0x7f2c341f44e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:48.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:48.125+0000 7f2c337fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2c34158cf0 0x7f2c341f44e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60584/0 (socket says 172.21.15.27:60584) 2024-09-12T03:17:48.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:48.125+0000 7f2c337fe640 1 -- 172.21.15.27:0/3757816752 learned_addr learned my addr 172.21.15.27:0/3757816752 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:17:48.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:48.125+0000 7f2c33fff640 1 --2- 172.21.15.27:0/3757816752 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2c3415f600 0x7f2c3415dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:48.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:48.125+0000 7f2c32ffd640 1 --2- 172.21.15.27:0/3757816752 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2c341590d0 0x7f2c34152520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:48.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:48.125+0000 7f2c337fe640 1 -- 172.21.15.27:0/3757816752 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2c3415f600 msgr2=0x7f2c3415dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:48.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:48.125+0000 7f2c337fe640 1 --2- 172.21.15.27:0/3757816752 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2c3415f600 0x7f2c3415dd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:48.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:48.125+0000 7f2c337fe640 1 -- 172.21.15.27:0/3757816752 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2c341590d0 msgr2=0x7f2c34152520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:48.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:48.125+0000 7f2c337fe640 1 --2- 172.21.15.27:0/3757816752 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2c341590d0 0x7f2c34152520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:48.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:48.125+0000 7f2c337fe640 1 -- 172.21.15.27:0/3757816752 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c34152a60 con 0x7f2c34158cf0 2024-09-12T03:17:48.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:48.125+0000 7f2c337fe640 1 --2- 172.21.15.27:0/3757816752 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2c34158cf0 0x7f2c341f44e0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f2c2400e870 tx=0x7f2c2400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:48.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:48.125+0000 7f2c1bfff640 1 -- 172.21.15.27:0/3757816752 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c24017070 con 0x7f2c34158cf0 2024-09-12T03:17:48.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:48.125+0000 7f2c1bfff640 1 -- 172.21.15.27:0/3757816752 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c2400ee60 con 0x7f2c34158cf0 2024-09-12T03:17:48.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:48.125+0000 7f2c1bfff640 1 -- 172.21.15.27:0/3757816752 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c2400cb80 con 0x7f2c34158cf0 2024-09-12T03:17:48.129 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:48.125+0000 7f2c3a15f640 1 -- 172.21.15.27:0/3757816752 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c341f4c50 con 0x7f2c34158cf0 2024-09-12T03:17:48.129 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:48.125+0000 7f2c3a15f640 1 -- 172.21.15.27:0/3757816752 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c341f5210 con 0x7f2c34158cf0 2024-09-12T03:17:48.129 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:48.126+0000 7f2c3a15f640 1 -- 172.21.15.27:0/3757816752 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c340b55d0 con 0x7f2c34158cf0 2024-09-12T03:17:48.129 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:48.127+0000 7f2c1bfff640 1 -- 172.21.15.27:0/3757816752 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 7) ==== 90651+0+0 (secure 0 0 0) 0x7f2c2401a070 con 0x7f2c34158cf0 2024-09-12T03:17:48.129 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:48.127+0000 7f2c1bfff640 1 --2- 172.21.15.27:0/3757816752 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f2c2006bfe0 0x7f2c2006e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:48.129 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:48.127+0000 7f2c1bfff640 1 -- 172.21.15.27:0/3757816752 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5709+0+0 (secure 0 0 0) 0x7f2c2408af60 con 0x7f2c34158cf0 2024-09-12T03:17:48.129 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:48.127+0000 7f2c32ffd640 1 --2- 172.21.15.27:0/3757816752 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f2c2006bfe0 0x7f2c2006e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:48.130 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:48.128+0000 7f2c32ffd640 1 --2- 172.21.15.27:0/3757816752 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f2c2006bfe0 0x7f2c2006e4a0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f2c28034000 tx=0x7f2c2802e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:48.131 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:48.131+0000 7f2c1bfff640 1 -- 172.21.15.27:0/3757816752 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f2c24059720 con 0x7f2c34158cf0 2024-09-12T03:17:48.321 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:48.321+0000 7f2c3a15f640 1 -- 172.21.15.27:0/3757816752 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f2c3406eb90 con 0x7f2c34158cf0 2024-09-12T03:17:48.321 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:48.321+0000 7f2c1bfff640 1 -- 172.21.15.27:0/3757816752 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v43) ==== 80+0+0 (secure 0 0 0) 0x7f2c24059020 con 0x7f2c34158cf0 2024-09-12T03:17:48.321 INFO:teuthology.orchestra.run.smithi027.stderr:listed 0 entries 2024-09-12T03:17:48.323 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:48.323+0000 7f2c3a15f640 1 -- 172.21.15.27:0/3757816752 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f2c2006bfe0 msgr2=0x7f2c2006e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:48.323 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:48.323+0000 7f2c3a15f640 1 --2- 172.21.15.27:0/3757816752 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f2c2006bfe0 0x7f2c2006e4a0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f2c28034000 tx=0x7f2c2802e000 comp rx=0 tx=0).stop 2024-09-12T03:17:48.323 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:48.323+0000 7f2c3a15f640 1 -- 172.21.15.27:0/3757816752 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2c34158cf0 msgr2=0x7f2c341f44e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:48.324 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:48.323+0000 7f2c3a15f640 1 --2- 172.21.15.27:0/3757816752 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2c34158cf0 0x7f2c341f44e0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f2c2400e870 tx=0x7f2c2400ed30 comp rx=0 tx=0).stop 2024-09-12T03:17:48.324 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:48.323+0000 7f2c3a15f640 1 -- 172.21.15.27:0/3757816752 shutdown_connections 2024-09-12T03:17:48.324 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:48.323+0000 7f2c3a15f640 1 --2- 172.21.15.27:0/3757816752 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f2c2006bfe0 0x7f2c2006e4a0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:48.324 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:48.323+0000 7f2c3a15f640 1 --2- 172.21.15.27:0/3757816752 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2c3415f600 0x7f2c3415dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:48.324 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:48.323+0000 7f2c3a15f640 1 --2- 172.21.15.27:0/3757816752 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2c341590d0 0x7f2c34152520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:48.324 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:48.323+0000 7f2c3a15f640 1 --2- 172.21.15.27:0/3757816752 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2c34158cf0 0x7f2c341f44e0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:48.324 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:48.323+0000 7f2c3a15f640 1 -- 172.21.15.27:0/3757816752 >> 172.21.15.27:0/3757816752 conn(0x7f2c3406cc40 msgr2=0x7f2c340af9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:48.324 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:48.323+0000 7f2c3a15f640 1 -- 172.21.15.27:0/3757816752 shutdown_connections 2024-09-12T03:17:48.324 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:48.323+0000 7f2c3a15f640 1 -- 172.21.15.27:0/3757816752 wait complete. 2024-09-12T03:17:48.334 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-12T03:17:48.334 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:17:48.334 DEBUG:teuthology.orchestra.run.smithi027:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-12T03:17:48.363 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi027.front.sepia.ceph.com 2024-09-12T03:17:48.364 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:17:48.364 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-12T03:17:48.420 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-12T03:17:48.420 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T03:17:48.440 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-12T03:17:48.440 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T03:17:48.500 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-12T03:17:48.604 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:48.604+0000 7fb590b6d640 1 Processor -- start 2024-09-12T03:17:48.604 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:48.604+0000 7fb590b6d640 1 -- start start 2024-09-12T03:17:48.607 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:48.605+0000 7fb590b6d640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb58c0d1450 0x7fb58c070e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:48.607 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:48.605+0000 7fb590b6d640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fb58c164140 0x7fb58c166530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:48.607 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:48.605+0000 7fb590b6d640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb58c06fa00 con 0x7fb58c0cef50 2024-09-12T03:17:48.607 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:48.605+0000 7fb590b6d640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fb58c15ca90 con 0x7fb58c164140 2024-09-12T03:17:48.607 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:48.605+0000 7fb590b6d640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fb58c0713c0 con 0x7fb58c0d1450 2024-09-12T03:17:48.607 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:48.605+0000 7fb58a59c640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fb58c0cef50 0x7fb58c0cf330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:40218/0 (socket says 172.21.15.27:40218) 2024-09-12T03:17:48.608 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:48.605+0000 7fb58a59c640 1 -- 172.21.15.27:0/1036383492 learned_addr learned my addr 172.21.15.27:0/1036383492 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:17:48.608 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:48.605+0000 7fb58ad9d640 1 --2- 172.21.15.27:0/1036383492 >> v2:172.21.15.45:3300/0 conn(0x7fb58c164140 0x7fb58c166530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:48.608 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:48.605+0000 7fb58959a640 1 -- 172.21.15.27:0/1036383492 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4244461602 0 0) 0x7fb58c06fa00 con 0x7fb58c0cef50 2024-09-12T03:17:48.608 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:48.605+0000 7fb58959a640 1 -- 172.21.15.27:0/1036383492 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb574003520 con 0x7fb58c0cef50 2024-09-12T03:17:48.608 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:48.605+0000 7fb589d9b640 1 --2- 172.21.15.27:0/1036383492 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb58c0d1450 0x7fb58c070e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:48.608 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:48.606+0000 7fb58959a640 1 -- 172.21.15.27:0/1036383492 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 393595274 0 0) 0x7fb574003520 con 0x7fb58c0cef50 2024-09-12T03:17:48.608 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:48.606+0000 7fb58959a640 1 -- 172.21.15.27:0/1036383492 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb58c0d1450 msgr2=0x7fb58c070e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:48.608 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:48.606+0000 7fb58959a640 1 --2- 172.21.15.27:0/1036383492 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb58c0d1450 0x7fb58c070e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:48.608 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:48.606+0000 7fb58959a640 1 -- 172.21.15.27:0/1036383492 >> v2:172.21.15.45:3300/0 conn(0x7fb58c164140 msgr2=0x7fb58c166530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:48.608 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:48.606+0000 7fb58959a640 1 --2- 172.21.15.27:0/1036383492 >> v2:172.21.15.45:3300/0 conn(0x7fb58c164140 0x7fb58c166530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:48.608 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:48.606+0000 7fb58959a640 1 -- 172.21.15.27:0/1036383492 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb58c06f720 con 0x7fb58c0cef50 2024-09-12T03:17:48.608 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:48.606+0000 7fb58959a640 1 -- 172.21.15.27:0/1036383492 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 847755213 0 0) 0x7fb57c001690 con 0x7fb58c0cef50 2024-09-12T03:17:48.608 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:48.606+0000 7fb58959a640 1 -- 172.21.15.27:0/1036383492 >> v1:172.21.15.27:6789/0 conn(0x7fb58c0cef50 legacy=0x7fb58c0cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:48.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:48.606+0000 7fb58959a640 1 --2- 172.21.15.27:0/1036383492 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb5740039b0 0x7fb574003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:48.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:48.606+0000 7fb58959a640 1 --2- 172.21.15.27:0/1036383492 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb58c0d1450 0x7fb574004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:48.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:48.606+0000 7fb58959a640 1 --2- 172.21.15.27:0/1036383492 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb574004d40 0x7fb5740051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:48.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:48.606+0000 7fb58959a640 1 -- 172.21.15.27:0/1036383492 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fb58c0713c0 con 0x7fb5740039b0 2024-09-12T03:17:48.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:48.606+0000 7fb58959a640 1 -- 172.21.15.27:0/1036383492 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb57400b9c0 con 0x7fb574004d40 2024-09-12T03:17:48.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:48.606+0000 7fb58959a640 1 -- 172.21.15.27:0/1036383492 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fb57400bbc0 con 0x7fb58c0d1450 2024-09-12T03:17:48.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:48.606+0000 7fb58a59c640 1 --2- 172.21.15.27:0/1036383492 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb5740039b0 0x7fb574003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:48.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:48.606+0000 7fb589d9b640 1 --2- 172.21.15.27:0/1036383492 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb58c0d1450 0x7fb574004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:48.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:48.606+0000 7fb58ad9d640 1 --2- 172.21.15.27:0/1036383492 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb574004d40 0x7fb5740051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:48.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:48.606+0000 7fb58a59c640 1 -- 172.21.15.27:0/1036383492 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb58c0d1450 msgr2=0x7fb574004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:48.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:48.606+0000 7fb58a59c640 1 --2- 172.21.15.27:0/1036383492 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb58c0d1450 0x7fb574004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:48.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:48.606+0000 7fb58a59c640 1 -- 172.21.15.27:0/1036383492 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb574004d40 msgr2=0x7fb5740051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:48.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:48.606+0000 7fb58a59c640 1 --2- 172.21.15.27:0/1036383492 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb574004d40 0x7fb5740051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:48.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:48.606+0000 7fb58a59c640 1 -- 172.21.15.27:0/1036383492 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb574003520 con 0x7fb5740039b0 2024-09-12T03:17:48.610 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:48.607+0000 7fb58a59c640 1 --2- 172.21.15.27:0/1036383492 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb5740039b0 0x7fb574003dd0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fb57c00c210 tx=0x7fb574005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b493dd13dce1ea33 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:48.610 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:48.607+0000 7fb58959a640 1 -- 172.21.15.27:0/1036383492 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb57c007d60 con 0x7fb5740039b0 2024-09-12T03:17:48.610 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:48.607+0000 7fb58959a640 1 -- 172.21.15.27:0/1036383492 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb57c005af0 con 0x7fb5740039b0 2024-09-12T03:17:48.610 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:48.607+0000 7fb58959a640 1 -- 172.21.15.27:0/1036383492 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb57c007070 con 0x7fb5740039b0 2024-09-12T03:17:48.613 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:48.608+0000 7fb590b6d640 1 -- 172.21.15.27:0/1036383492 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb5740039b0 msgr2=0x7fb574003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:48.613 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:48.608+0000 7fb590b6d640 1 --2- 172.21.15.27:0/1036383492 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb5740039b0 0x7fb574003dd0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fb57c00c210 tx=0x7fb574005830 comp rx=0 tx=0).stop 2024-09-12T03:17:48.613 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:48.608+0000 7fb58ad9d640 1 -- 172.21.15.27:0/1036383492 reap_dead start 2024-09-12T03:17:48.613 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:48.608+0000 7fb590b6d640 1 -- 172.21.15.27:0/1036383492 shutdown_connections 2024-09-12T03:17:48.613 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:48.608+0000 7fb590b6d640 1 -- 172.21.15.27:0/1036383492 >> 172.21.15.27:0/1036383492 conn(0x7fb58c073800 msgr2=0x7fb58c09ba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:48.613 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:48.608+0000 7fb590b6d640 1 -- 172.21.15.27:0/1036383492 shutdown_connections 2024-09-12T03:17:48.613 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:48.608+0000 7fb590b6d640 1 -- 172.21.15.27:0/1036383492 wait complete. 2024-09-12T03:17:48.613 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:48.609+0000 7fb590b6d640 1 Processor -- start 2024-09-12T03:17:48.613 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:48.609+0000 7fb590b6d640 1 -- start start 2024-09-12T03:17:48.613 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:48.610+0000 7fb590b6d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb5740039b0 0x7fb58c155e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:48.613 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:48.610+0000 7fb590b6d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb58c14f1a0 0x7fb58c14fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:48.614 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:48.610+0000 7fb590b6d640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb58c153470 0x7fb58c153900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:48.614 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:48.610+0000 7fb590b6d640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fb58c06f360 con 0x7fb58c14f1a0 2024-09-12T03:17:48.614 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:48.610+0000 7fb590b6d640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb58c0d2d10 con 0x7fb5740039b0 2024-09-12T03:17:48.614 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:48.610+0000 7fb590b6d640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fb58c06f4e0 con 0x7fb58c153470 2024-09-12T03:17:48.614 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:48.610+0000 7fb589d9b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb58c14f1a0 0x7fb58c14fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:48.614 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:48.610+0000 7fb589d9b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb58c14f1a0 0x7fb58c14fc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60604/0 (socket says 172.21.15.27:60604) 2024-09-12T03:17:48.614 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:48.610+0000 7fb589d9b640 1 -- 172.21.15.27:0/2316308288 learned_addr learned my addr 172.21.15.27:0/2316308288 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:17:48.614 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:48.610+0000 7fb58a59c640 1 --2- 172.21.15.27:0/2316308288 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb5740039b0 0x7fb58c155e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:48.614 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:48.610+0000 7fb58ad9d640 1 --2- 172.21.15.27:0/2316308288 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb58c153470 0x7fb58c153900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:48.614 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:48.610+0000 7fb589d9b640 1 -- 172.21.15.27:0/2316308288 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb58c153470 msgr2=0x7fb58c153900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:48.614 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:48.610+0000 7fb589d9b640 1 --2- 172.21.15.27:0/2316308288 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb58c153470 0x7fb58c153900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:48.614 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:48.610+0000 7fb589d9b640 1 -- 172.21.15.27:0/2316308288 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb5740039b0 msgr2=0x7fb58c155e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:48.614 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:48.610+0000 7fb589d9b640 1 --2- 172.21.15.27:0/2316308288 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb5740039b0 0x7fb58c155e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:48.615 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:48.610+0000 7fb589d9b640 1 -- 172.21.15.27:0/2316308288 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb58c150570 con 0x7fb58c14f1a0 2024-09-12T03:17:48.615 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:48.611+0000 7fb589d9b640 1 --2- 172.21.15.27:0/2316308288 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb58c14f1a0 0x7fb58c14fc00 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fb580011960 tx=0x7fb580011e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:48.615 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:48.611+0000 7fb57affd640 1 -- 172.21.15.27:0/2316308288 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb58000faf0 con 0x7fb58c14f1a0 2024-09-12T03:17:48.615 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:48.611+0000 7fb57affd640 1 -- 172.21.15.27:0/2316308288 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb58000fc90 con 0x7fb58c14f1a0 2024-09-12T03:17:48.615 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:48.611+0000 7fb590b6d640 1 -- 172.21.15.27:0/2316308288 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb58c150850 con 0x7fb58c14f1a0 2024-09-12T03:17:48.615 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:48.611+0000 7fb57affd640 1 -- 172.21.15.27:0/2316308288 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb580014440 con 0x7fb58c14f1a0 2024-09-12T03:17:48.615 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:48.611+0000 7fb590b6d640 1 -- 172.21.15.27:0/2316308288 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb58c1510f0 con 0x7fb58c14f1a0 2024-09-12T03:17:48.615 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:48.612+0000 7fb590b6d640 1 -- 172.21.15.27:0/2316308288 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb58c150c40 con 0x7fb58c14f1a0 2024-09-12T03:17:48.615 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:48.612+0000 7fb57affd640 1 -- 172.21.15.27:0/2316308288 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 7) ==== 90651+0+0 (secure 0 0 0) 0x7fb580012040 con 0x7fb58c14f1a0 2024-09-12T03:17:48.615 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:48.613+0000 7fb57affd640 1 --2- 172.21.15.27:0/2316308288 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fb56c06c010 0x7fb56c06e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:48.615 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:48.613+0000 7fb57affd640 1 -- 172.21.15.27:0/2316308288 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5709+0+0 (secure 0 0 0) 0x7fb58008b650 con 0x7fb58c14f1a0 2024-09-12T03:17:48.615 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:48.613+0000 7fb58a59c640 1 --2- 172.21.15.27:0/2316308288 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fb56c06c010 0x7fb56c06e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:48.616 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:48.613+0000 7fb58a59c640 1 --2- 172.21.15.27:0/2316308288 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fb56c06c010 0x7fb56c06e4d0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fb57c033000 tx=0x7fb57c009030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:48.617 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:48.617+0000 7fb57affd640 1 -- 172.21.15.27:0/2316308288 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fb580019030 con 0x7fb58c14f1a0 2024-09-12T03:17:48.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:48.804+0000 7fb590b6d640 1 -- 172.21.15.27:0/2316308288 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fb58c0cfbd0 con 0x7fb58c14f1a0 2024-09-12T03:17:48.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:48.804+0000 7fb57affd640 1 -- 172.21.15.27:0/2316308288 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v43) ==== 80+0+0 (secure 0 0 0) 0x7fb580059730 con 0x7fb58c14f1a0 2024-09-12T03:17:48.804 INFO:teuthology.orchestra.run.smithi027.stderr:listed 0 entries 2024-09-12T03:17:48.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:48.806+0000 7fb590b6d640 1 -- 172.21.15.27:0/2316308288 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fb56c06c010 msgr2=0x7fb56c06e4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:48.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:48.806+0000 7fb590b6d640 1 --2- 172.21.15.27:0/2316308288 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fb56c06c010 0x7fb56c06e4d0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fb57c033000 tx=0x7fb57c009030 comp rx=0 tx=0).stop 2024-09-12T03:17:48.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:48.806+0000 7fb590b6d640 1 -- 172.21.15.27:0/2316308288 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb58c14f1a0 msgr2=0x7fb58c14fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:48.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:48.806+0000 7fb590b6d640 1 --2- 172.21.15.27:0/2316308288 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb58c14f1a0 0x7fb58c14fc00 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fb580011960 tx=0x7fb580011e20 comp rx=0 tx=0).stop 2024-09-12T03:17:48.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:48.806+0000 7fb590b6d640 1 -- 172.21.15.27:0/2316308288 shutdown_connections 2024-09-12T03:17:48.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:48.806+0000 7fb590b6d640 1 --2- 172.21.15.27:0/2316308288 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fb56c06c010 0x7fb56c06e4d0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:48.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:48.806+0000 7fb590b6d640 1 --2- 172.21.15.27:0/2316308288 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb58c153470 0x7fb58c153900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:48.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:48.806+0000 7fb590b6d640 1 --2- 172.21.15.27:0/2316308288 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb58c14f1a0 0x7fb58c14fc00 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:48.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:48.806+0000 7fb590b6d640 1 --2- 172.21.15.27:0/2316308288 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb5740039b0 0x7fb58c155e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:48.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:48.806+0000 7fb590b6d640 1 -- 172.21.15.27:0/2316308288 >> 172.21.15.27:0/2316308288 conn(0x7fb58c073800 msgr2=0x7fb58c156b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:48.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:48.806+0000 7fb590b6d640 1 -- 172.21.15.27:0/2316308288 shutdown_connections 2024-09-12T03:17:48.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:48.806+0000 7fb590b6d640 1 -- 172.21.15.27:0/2316308288 wait complete. 2024-09-12T03:17:48.817 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-09-12T03:17:48.817 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:17:48.817 DEBUG:teuthology.orchestra.run.smithi045:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-09-12T03:17:48.852 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi045.front.sepia.ceph.com 2024-09-12T03:17:48.852 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:17:48.852 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-12T03:17:48.910 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-12T03:17:49.013 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.013+0000 7fb18661e640 1 Processor -- start 2024-09-12T03:17:49.013 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.013+0000 7fb18661e640 1 -- start start 2024-09-12T03:17:49.014 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.013+0000 7fb18661e640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fb18015eb90 0x7fb1800d2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:49.014 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.013+0000 7fb18661e640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb1800d5200 0x7fb1800d1010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:49.014 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.013+0000 7fb18661e640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fb1800d1550 0x7fb1800cf200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:49.014 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.013+0000 7fb18661e640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7fb1800b7d20 con 0x7fb1800d1550 2024-09-12T03:17:49.014 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.013+0000 7fb18661e640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fb1800d42e0 con 0x7fb18015eb90 2024-09-12T03:17:49.014 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.013+0000 7fb18661e640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fb1800d19c0 con 0x7fb1800d5200 2024-09-12T03:17:49.015 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.014+0000 7fb184bbb640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fb1800d1550 0x7fb1800cf200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:49.015 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.014+0000 7fb184bbb640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fb1800d1550 0x7fb1800cf200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60610/0 (socket says 172.21.15.27:60610) 2024-09-12T03:17:49.015 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.014+0000 7fb184bbb640 1 -- 172.21.15.27:0/952118 learned_addr learned my addr 172.21.15.27:0/952118 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:17:49.015 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.014+0000 7fb17ffff640 1 --2- 172.21.15.27:0/952118 >> v2:172.21.15.45:3300/0 conn(0x7fb18015eb90 0x7fb1800d2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:49.015 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.014+0000 7fb177fff640 1 --2- 172.21.15.27:0/952118 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb1800d5200 0x7fb1800d1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:49.015 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.014+0000 7fb184bbb640 1 -- 172.21.15.27:0/952118 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb1800d5200 msgr2=0x7fb1800d1010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:49.015 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.014+0000 7fb184bbb640 1 --2- 172.21.15.27:0/952118 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb1800d5200 0x7fb1800d1010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:49.015 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.014+0000 7fb184bbb640 1 -- 172.21.15.27:0/952118 >> v2:172.21.15.45:3300/0 conn(0x7fb18015eb90 msgr2=0x7fb1800d2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:49.015 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.014+0000 7fb184bbb640 1 --2- 172.21.15.27:0/952118 >> v2:172.21.15.45:3300/0 conn(0x7fb18015eb90 0x7fb1800d2df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:49.015 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.014+0000 7fb184bbb640 1 -- 172.21.15.27:0/952118 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1800d3760 con 0x7fb1800d1550 2024-09-12T03:17:49.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.015+0000 7fb184bbb640 1 --2- 172.21.15.27:0/952118 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb1800d1550 0x7fb1800cf200 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fb180075560 tx=0x7fb17802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3dea43f0c67fcd5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:49.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.015+0000 7fb17f7fe640 1 -- 172.21.15.27:0/952118 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb178036070 con 0x7fb1800d1550 2024-09-12T03:17:49.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.015+0000 7fb17f7fe640 1 -- 172.21.15.27:0/952118 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb178031600 con 0x7fb1800d1550 2024-09-12T03:17:49.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.015+0000 7fb17f7fe640 1 -- 172.21.15.27:0/952118 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1780319d0 con 0x7fb1800d1550 2024-09-12T03:17:49.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.015+0000 7fb18661e640 1 -- 172.21.15.27:0/952118 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb1800d1550 msgr2=0x7fb1800cf200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:49.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.015+0000 7fb18661e640 1 --2- 172.21.15.27:0/952118 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb1800d1550 0x7fb1800cf200 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fb180075560 tx=0x7fb17802a6a0 comp rx=0 tx=0).stop 2024-09-12T03:17:49.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.015+0000 7fb18661e640 1 -- 172.21.15.27:0/952118 shutdown_connections 2024-09-12T03:17:49.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.015+0000 7fb18661e640 1 --2- 172.21.15.27:0/952118 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb1800d1550 0x7fb1800cf200 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:49.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.015+0000 7fb18661e640 1 --2- 172.21.15.27:0/952118 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb1800d5200 0x7fb1800d1010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:49.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.015+0000 7fb18661e640 1 --2- 172.21.15.27:0/952118 >> v2:172.21.15.45:3300/0 conn(0x7fb18015eb90 0x7fb1800d2df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:49.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.015+0000 7fb18661e640 1 -- 172.21.15.27:0/952118 >> 172.21.15.27:0/952118 conn(0x7fb18006dce0 msgr2=0x7fb18006e0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:49.018 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.016+0000 7fb18661e640 1 -- 172.21.15.27:0/952118 shutdown_connections 2024-09-12T03:17:49.018 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.016+0000 7fb18661e640 1 -- 172.21.15.27:0/952118 wait complete. 2024-09-12T03:17:49.018 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.017+0000 7fb18661e640 1 Processor -- start 2024-09-12T03:17:49.018 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.017+0000 7fb18661e640 1 -- start start 2024-09-12T03:17:49.020 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.017+0000 7fb18661e640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb1800d1550 0x7fb18015acf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:49.020 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.017+0000 7fb18661e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb1800d5200 0x7fb1801e7290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:49.020 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.017+0000 7fb18661e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb18015eb90 0x7fb1800cdab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:49.020 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.017+0000 7fb18661e640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fb1800d4000 con 0x7fb18015eb90 2024-09-12T03:17:49.020 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.017+0000 7fb18661e640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb1800d3d20 con 0x7fb1800d5200 2024-09-12T03:17:49.020 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.017+0000 7fb18661e640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fb1800d3330 con 0x7fb1800d1550 2024-09-12T03:17:49.020 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.018+0000 7fb184bbb640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb18015eb90 0x7fb1800cdab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:49.020 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.018+0000 7fb184bbb640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb18015eb90 0x7fb1800cdab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60620/0 (socket says 172.21.15.27:60620) 2024-09-12T03:17:49.020 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.018+0000 7fb184bbb640 1 -- 172.21.15.27:0/4068463097 learned_addr learned my addr 172.21.15.27:0/4068463097 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:17:49.020 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.018+0000 7fb177fff640 1 --2- 172.21.15.27:0/4068463097 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb1800d5200 0x7fb1801e7290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:49.020 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.018+0000 7fb17ffff640 1 --2- 172.21.15.27:0/4068463097 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb1800d1550 0x7fb18015acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:49.021 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.018+0000 7fb184bbb640 1 -- 172.21.15.27:0/4068463097 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb1800d1550 msgr2=0x7fb18015acf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:49.021 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.018+0000 7fb184bbb640 1 --2- 172.21.15.27:0/4068463097 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb1800d1550 0x7fb18015acf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:49.021 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.018+0000 7fb184bbb640 1 -- 172.21.15.27:0/4068463097 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb1800d5200 msgr2=0x7fb1801e7290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:49.021 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.018+0000 7fb184bbb640 1 --2- 172.21.15.27:0/4068463097 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb1800d5200 0x7fb1801e7290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:49.021 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.018+0000 7fb184bbb640 1 -- 172.21.15.27:0/4068463097 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1801f76c0 con 0x7fb18015eb90 2024-09-12T03:17:49.021 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.018+0000 7fb184bbb640 1 --2- 172.21.15.27:0/4068463097 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb18015eb90 0x7fb1800cdab0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fb17802ac90 tx=0x7fb178004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:49.021 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.018+0000 7fb17d7fa640 1 -- 172.21.15.27:0/4068463097 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb17803f020 con 0x7fb18015eb90 2024-09-12T03:17:49.021 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.018+0000 7fb17d7fa640 1 -- 172.21.15.27:0/4068463097 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb178037040 con 0x7fb18015eb90 2024-09-12T03:17:49.021 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.018+0000 7fb18661e640 1 -- 172.21.15.27:0/4068463097 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1801f79a0 con 0x7fb18015eb90 2024-09-12T03:17:49.021 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.018+0000 7fb17d7fa640 1 -- 172.21.15.27:0/4068463097 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb178036070 con 0x7fb18015eb90 2024-09-12T03:17:49.021 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.018+0000 7fb18661e640 1 -- 172.21.15.27:0/4068463097 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1801f7f60 con 0x7fb18015eb90 2024-09-12T03:17:49.021 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.019+0000 7fb18661e640 1 -- 172.21.15.27:0/4068463097 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb18015ef70 con 0x7fb18015eb90 2024-09-12T03:17:49.021 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.020+0000 7fb17d7fa640 1 -- 172.21.15.27:0/4068463097 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 7) ==== 90651+0+0 (secure 0 0 0) 0x7fb178042020 con 0x7fb18015eb90 2024-09-12T03:17:49.022 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.020+0000 7fb17d7fa640 1 --2- 172.21.15.27:0/4068463097 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fb16006bfe0 0x7fb16006e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:49.022 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.020+0000 7fb17d7fa640 1 -- 172.21.15.27:0/4068463097 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5709+0+0 (secure 0 0 0) 0x7fb1780ab4c0 con 0x7fb18015eb90 2024-09-12T03:17:49.022 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.021+0000 7fb17ffff640 1 --2- 172.21.15.27:0/4068463097 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fb16006bfe0 0x7fb16006e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:49.022 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.021+0000 7fb17ffff640 1 --2- 172.21.15.27:0/4068463097 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fb16006bfe0 0x7fb16006e4a0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fb1700098f0 tx=0x7fb170008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:49.024 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.025+0000 7fb17d7fa640 1 -- 172.21.15.27:0/4068463097 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fb178049030 con 0x7fb18015eb90 2024-09-12T03:17:49.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.234+0000 7fb18661e640 1 -- 172.21.15.27:0/4068463097 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb180069ea0 con 0x7fb18015eb90 2024-09-12T03:17:49.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.235+0000 7fb17d7fa640 1 -- 172.21.15.27:0/4068463097 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+5223 (secure 0 0 0) 0x7fb178079b70 con 0x7fb18015eb90 2024-09-12T03:17:49.235 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:17:49.235 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":23,"btime":"2024-09-12T03:17:47:237558+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4835,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6834/1457797481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":1457797481},{"type":"v1","addr":"172.21.15.27:6834","nonce":1457797481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5029,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6836/2842491281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":2842491281},{"type":"v1","addr":"172.21.15.45:6836","nonce":2842491281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5032,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/3339744340","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6835","nonce":3339744340},{"type":"v1","addr":"172.21.15.45:6837","nonce":3339744340}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-12T03:17:38.580473+0000","modified":"2024-09-12T03:17:40.557950+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5205},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5205":{"gid":5205,"name":"a","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.27:6835/2571101228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":2571101228},{"type":"v1","addr":"172.21.15.27:6835","nonce":2571101228}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5205,"qdb_cluster":[5205]},"id":3}]} 2024-09-12T03:17:49.235 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 23 2024-09-12T03:17:49.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.237+0000 7fb18661e640 1 -- 172.21.15.27:0/4068463097 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fb16006bfe0 msgr2=0x7fb16006e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:49.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.237+0000 7fb18661e640 1 --2- 172.21.15.27:0/4068463097 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fb16006bfe0 0x7fb16006e4a0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fb1700098f0 tx=0x7fb170008040 comp rx=0 tx=0).stop 2024-09-12T03:17:49.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.237+0000 7fb18661e640 1 -- 172.21.15.27:0/4068463097 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb18015eb90 msgr2=0x7fb1800cdab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:49.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.237+0000 7fb18661e640 1 --2- 172.21.15.27:0/4068463097 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb18015eb90 0x7fb1800cdab0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fb17802ac90 tx=0x7fb178004a40 comp rx=0 tx=0).stop 2024-09-12T03:17:49.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.237+0000 7fb18661e640 1 -- 172.21.15.27:0/4068463097 shutdown_connections 2024-09-12T03:17:49.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.237+0000 7fb18661e640 1 --2- 172.21.15.27:0/4068463097 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fb16006bfe0 0x7fb16006e4a0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:49.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.237+0000 7fb18661e640 1 --2- 172.21.15.27:0/4068463097 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb18015eb90 0x7fb1800cdab0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:49.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.237+0000 7fb18661e640 1 --2- 172.21.15.27:0/4068463097 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb1800d5200 0x7fb1801e7290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:49.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.237+0000 7fb18661e640 1 --2- 172.21.15.27:0/4068463097 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb1800d1550 0x7fb18015acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:49.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.237+0000 7fb18661e640 1 -- 172.21.15.27:0/4068463097 >> 172.21.15.27:0/4068463097 conn(0x7fb18006dce0 msgr2=0x7fb180161310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:49.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.237+0000 7fb18661e640 1 -- 172.21.15.27:0/4068463097 shutdown_connections 2024-09-12T03:17:49.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.237+0000 7fb18661e640 1 -- 172.21.15.27:0/4068463097 wait complete. 2024-09-12T03:17:49.248 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json session ls 2024-09-12T03:17:49.368 INFO:teuthology.orchestra.run.smithi027.stdout:[] 2024-09-12T03:17:49.369 DEBUG:tasks.cephfs.filesystem:_json_asok output [] 2024-09-12T03:17:49.369 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs subvol40690 2024-09-12T03:17:49.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.474+0000 7f62a63fd640 1 Processor -- start 2024-09-12T03:17:49.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.475+0000 7f62a63fd640 1 -- start start 2024-09-12T03:17:49.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.475+0000 7f62a63fd640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f62a0162040 0x7f62a00d1da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:49.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.475+0000 7f62a63fd640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f62a00d3790 0x7f62a00b7a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:49.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.475+0000 7f62a63fd640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f62a00d2e30 con 0x7f62a006d8c0 2024-09-12T03:17:49.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.475+0000 7f62a63fd640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f62a00b7f90 con 0x7f62a00d3790 2024-09-12T03:17:49.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.475+0000 7f62a63fd640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f62a0162420 con 0x7f62a0162040 2024-09-12T03:17:49.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.476+0000 7f629f7fe640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f62a00d3790 0x7f62a00b7a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:49.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.476+0000 7f62a499a640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f62a006d8c0 0x7f62a00cf320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:56638/0 (socket says 172.21.15.27:56638) 2024-09-12T03:17:49.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.476+0000 7f62a499a640 1 -- 172.21.15.27:0/2753846922 learned_addr learned my addr 172.21.15.27:0/2753846922 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:17:49.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.476+0000 7f629ffff640 1 --2- 172.21.15.27:0/2753846922 >> v2:172.21.15.45:3300/0 conn(0x7f62a0162040 0x7f62a00d1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:49.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.476+0000 7f629f7fe640 1 -- 172.21.15.27:0/2753846922 >> v2:172.21.15.45:3300/0 conn(0x7f62a0162040 msgr2=0x7f62a00d1da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:49.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.476+0000 7f629f7fe640 1 --2- 172.21.15.27:0/2753846922 >> v2:172.21.15.45:3300/0 conn(0x7f62a0162040 0x7f62a00d1da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:49.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.476+0000 7f629f7fe640 1 -- 172.21.15.27:0/2753846922 >> v1:172.21.15.27:6789/0 conn(0x7f62a006d8c0 legacy=0x7f62a00cf320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:49.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.476+0000 7f629f7fe640 1 -- 172.21.15.27:0/2753846922 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62a00d30d0 con 0x7f62a00d3790 2024-09-12T03:17:49.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.476+0000 7f629effd640 1 -- 172.21.15.27:0/2753846922 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1778660143 0 0) 0x7f62a00d2e30 con 0x7f62a006d8c0 2024-09-12T03:17:49.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.476+0000 7f629ffff640 1 --2- 172.21.15.27:0/2753846922 >> v2:172.21.15.45:3300/0 conn(0x7f62a0162040 0x7f62a00d1da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:17:49.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.477+0000 7f629f7fe640 1 --2- 172.21.15.27:0/2753846922 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f62a00d3790 0x7f62a00b7a50 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f62a00d17b0 tx=0x7f629402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35ab8b404bd8fc7f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:49.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.477+0000 7f629effd640 1 -- 172.21.15.27:0/2753846922 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6294037070 con 0x7f62a00d3790 2024-09-12T03:17:49.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.477+0000 7f629effd640 1 -- 172.21.15.27:0/2753846922 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6294032440 con 0x7f62a00d3790 2024-09-12T03:17:49.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.477+0000 7f629effd640 1 -- 172.21.15.27:0/2753846922 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6294032810 con 0x7f62a00d3790 2024-09-12T03:17:49.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.477+0000 7f62a63fd640 1 -- 172.21.15.27:0/2753846922 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f62a00d3790 msgr2=0x7f62a00b7a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:49.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.477+0000 7f62a63fd640 1 --2- 172.21.15.27:0/2753846922 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f62a00d3790 0x7f62a00b7a50 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f62a00d17b0 tx=0x7f629402a1a0 comp rx=0 tx=0).stop 2024-09-12T03:17:49.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.477+0000 7f62a63fd640 1 -- 172.21.15.27:0/2753846922 shutdown_connections 2024-09-12T03:17:49.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.477+0000 7f62a63fd640 1 --2- 172.21.15.27:0/2753846922 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f62a00d3790 0x7f62a00b7a50 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:49.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.477+0000 7f62a63fd640 1 --2- 172.21.15.27:0/2753846922 >> v2:172.21.15.45:3300/0 conn(0x7f62a0162040 0x7f62a00d1da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:49.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.477+0000 7f62a63fd640 1 -- 172.21.15.27:0/2753846922 >> 172.21.15.27:0/2753846922 conn(0x7f62a00b6700 msgr2=0x7f62a00b6b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:49.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.477+0000 7f62a63fd640 1 -- 172.21.15.27:0/2753846922 shutdown_connections 2024-09-12T03:17:49.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.477+0000 7f62a63fd640 1 -- 172.21.15.27:0/2753846922 wait complete. 2024-09-12T03:17:49.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.479+0000 7f62a63fd640 1 Processor -- start 2024-09-12T03:17:49.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.479+0000 7f62a63fd640 1 -- start start 2024-09-12T03:17:49.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.480+0000 7f62a63fd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f62a006d8c0 0x7f62a015c130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:49.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.480+0000 7f62a63fd640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f62a00d3790 0x7f62a015c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:49.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.480+0000 7f62a63fd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f62a0162040 0x7f62a01f8690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:49.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.480+0000 7f62a63fd640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f62a00d4530 con 0x7f62a006d8c0 2024-09-12T03:17:49.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.480+0000 7f62a63fd640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f62a00cee00 con 0x7f62a0162040 2024-09-12T03:17:49.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.480+0000 7f62a63fd640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f62a00d4150 con 0x7f62a00d3790 2024-09-12T03:17:49.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.480+0000 7f629ffff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f62a006d8c0 0x7f62a015c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:49.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.480+0000 7f629ffff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f62a006d8c0 0x7f62a015c130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41070/0 (socket says 172.21.15.27:41070) 2024-09-12T03:17:49.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.480+0000 7f629ffff640 1 -- 172.21.15.27:0/563499492 learned_addr learned my addr 172.21.15.27:0/563499492 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:17:49.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.480+0000 7f62a499a640 1 --2- 172.21.15.27:0/563499492 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f62a0162040 0x7f62a01f8690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:49.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.480+0000 7f629f7fe640 1 --2- 172.21.15.27:0/563499492 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f62a00d3790 0x7f62a015c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:49.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.480+0000 7f629ffff640 1 -- 172.21.15.27:0/563499492 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f62a00d3790 msgr2=0x7f62a015c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:49.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.480+0000 7f629ffff640 1 --2- 172.21.15.27:0/563499492 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f62a00d3790 0x7f62a015c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:49.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.480+0000 7f629ffff640 1 -- 172.21.15.27:0/563499492 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f62a0162040 msgr2=0x7f62a01f8690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:49.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.480+0000 7f629ffff640 1 --2- 172.21.15.27:0/563499492 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f62a0162040 0x7f62a01f8690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:49.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.480+0000 7f629ffff640 1 -- 172.21.15.27:0/563499492 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62a015d160 con 0x7f62a006d8c0 2024-09-12T03:17:49.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.480+0000 7f629f7fe640 1 --2- 172.21.15.27:0/563499492 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f62a00d3790 0x7f62a015c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:17:49.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.480+0000 7f629ffff640 1 --2- 172.21.15.27:0/563499492 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f62a006d8c0 0x7f62a015c130 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f629000e7e0 tx=0x7f629000eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:49.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.480+0000 7f629cff9640 1 -- 172.21.15.27:0/563499492 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f629000f040 con 0x7f62a006d8c0 2024-09-12T03:17:49.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.481+0000 7f629cff9640 1 -- 172.21.15.27:0/563499492 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6290004030 con 0x7f62a006d8c0 2024-09-12T03:17:49.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.481+0000 7f62a63fd640 1 -- 172.21.15.27:0/563499492 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62a00ac4e0 con 0x7f62a006d8c0 2024-09-12T03:17:49.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.481+0000 7f629cff9640 1 -- 172.21.15.27:0/563499492 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6290004400 con 0x7f62a006d8c0 2024-09-12T03:17:49.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.481+0000 7f62a63fd640 1 -- 172.21.15.27:0/563499492 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62a00acaa0 con 0x7f62a006d8c0 2024-09-12T03:17:49.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.482+0000 7f62a63fd640 1 -- 172.21.15.27:0/563499492 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62a00b7a50 con 0x7f62a006d8c0 2024-09-12T03:17:49.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.482+0000 7f629cff9640 1 -- 172.21.15.27:0/563499492 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 7) ==== 90651+0+0 (secure 0 0 0) 0x7f629000c6d0 con 0x7f62a006d8c0 2024-09-12T03:17:49.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.483+0000 7f629cff9640 1 --2- 172.21.15.27:0/563499492 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f628c06bfe0 0x7f628c06e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:49.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.483+0000 7f629cff9640 1 -- 172.21.15.27:0/563499492 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5709+0+0 (secure 0 0 0) 0x7f629008b020 con 0x7f62a006d8c0 2024-09-12T03:17:49.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.483+0000 7f629f7fe640 1 --2- 172.21.15.27:0/563499492 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f628c06bfe0 0x7f628c06e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:49.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.483+0000 7f629f7fe640 1 --2- 172.21.15.27:0/563499492 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f628c06bfe0 0x7f628c06e4a0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f6294034000 tx=0x7f629402e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:49.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.487+0000 7f629cff9640 1 -- 172.21.15.27:0/563499492 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f6290017020 con 0x7f62a006d8c0 2024-09-12T03:17:49.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.687+0000 7f62a63fd640 1 -- 172.21.15.27:0/563499492 --> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvol40690", "target": ["mon-mgr", ""]}) -- 0x7f62a00d20c0 con 0x7f628c06bfe0 2024-09-12T03:17:49.693 INFO:tasks.ceph.mgr.x.smithi045.stderr:2024-09-12T03:17:49.692+0000 7fc16ffad640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:17:49.693 INFO:tasks.ceph.mgr.x.smithi045.stderr:2024-09-12T03:17:49.692+0000 7fc16ffad640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:17:49.693 INFO:tasks.ceph.mgr.x.smithi045.stderr:2024-09-12T03:17:49.692+0000 7fc16ffad640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:17:49.693 INFO:tasks.ceph.mgr.x.smithi045.stderr:2024-09-12T03:17:49.692+0000 7fc16ffad640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:17:49.693 INFO:tasks.ceph.mgr.x.smithi045.stderr:2024-09-12T03:17:49.692+0000 7fc16ffad640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:17:49.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.724+0000 7f629cff9640 1 -- 172.21.15.27:0/563499492 <== mgr.4102 v2:172.21.15.45:6832/3923821219 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f62a00d20c0 con 0x7f628c06bfe0 2024-09-12T03:17:49.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.727+0000 7f62a63fd640 1 -- 172.21.15.27:0/563499492 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f628c06bfe0 msgr2=0x7f628c06e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:49.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.727+0000 7f62a63fd640 1 --2- 172.21.15.27:0/563499492 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f628c06bfe0 0x7f628c06e4a0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f6294034000 tx=0x7f629402e000 comp rx=0 tx=0).stop 2024-09-12T03:17:49.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.727+0000 7f62a63fd640 1 -- 172.21.15.27:0/563499492 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f62a006d8c0 msgr2=0x7f62a015c130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:49.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.727+0000 7f62a63fd640 1 --2- 172.21.15.27:0/563499492 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f62a006d8c0 0x7f62a015c130 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f629000e7e0 tx=0x7f629000eca0 comp rx=0 tx=0).stop 2024-09-12T03:17:49.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.727+0000 7f62a63fd640 1 -- 172.21.15.27:0/563499492 shutdown_connections 2024-09-12T03:17:49.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.727+0000 7f62a63fd640 1 --2- 172.21.15.27:0/563499492 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f628c06bfe0 0x7f628c06e4a0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:49.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.727+0000 7f62a63fd640 1 --2- 172.21.15.27:0/563499492 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f62a0162040 0x7f62a01f8690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:49.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.727+0000 7f62a63fd640 1 --2- 172.21.15.27:0/563499492 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f62a00d3790 0x7f62a015c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:49.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.727+0000 7f62a63fd640 1 --2- 172.21.15.27:0/563499492 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f62a006d8c0 0x7f62a015c130 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:49.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.727+0000 7f62a63fd640 1 -- 172.21.15.27:0/563499492 >> 172.21.15.27:0/563499492 conn(0x7f62a00b6700 msgr2=0x7f62a015f630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:49.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.727+0000 7f62a63fd640 1 -- 172.21.15.27:0/563499492 shutdown_connections 2024-09-12T03:17:49.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.727+0000 7f62a63fd640 1 -- 172.21.15.27:0/563499492 wait complete. 2024-09-12T03:17:49.738 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-12T03:17:49.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.840+0000 7f8461a5e640 1 Processor -- start 2024-09-12T03:17:49.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.840+0000 7f8461a5e640 1 -- start start 2024-09-12T03:17:49.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.841+0000 7f8461a5e640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f845c0cf560 0x7f845c0cf940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:49.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.841+0000 7f8461a5e640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f845c0b7930 0x7f845c0b7d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:49.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.841+0000 7f8461a5e640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f845c0d2170 con 0x7f845c0d3eb0 2024-09-12T03:17:49.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.841+0000 7f8461a5e640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f845c0d3a20 con 0x7f845c0cf560 2024-09-12T03:17:49.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.841+0000 7f8461a5e640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f845c0cfe80 con 0x7f845c0b7930 2024-09-12T03:17:49.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.841+0000 7f845affd640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f845c0cf560 0x7f845c0cf940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:49.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.841+0000 7f845a7fc640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f845c0d3eb0 0x7f845c0ba120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:56640/0 (socket says 172.21.15.27:56640) 2024-09-12T03:17:49.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.841+0000 7f845affd640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f845c0cf560 0x7f845c0cf940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41072/0 (socket says 172.21.15.27:41072) 2024-09-12T03:17:49.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.841+0000 7f845a7fc640 1 -- 172.21.15.27:0/2182185285 learned_addr learned my addr 172.21.15.27:0/2182185285 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:17:49.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.842+0000 7f845affd640 1 -- 172.21.15.27:0/2182185285 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f845c0b7930 msgr2=0x7f845c0b7d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:49.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.842+0000 7f845b7fe640 1 --2- 172.21.15.27:0/2182185285 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f845c0b7930 0x7f845c0b7d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:49.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.842+0000 7f845affd640 1 --2- 172.21.15.27:0/2182185285 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f845c0b7930 0x7f845c0b7d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:49.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.842+0000 7f845affd640 1 -- 172.21.15.27:0/2182185285 >> v1:172.21.15.27:6789/0 conn(0x7f845c0d3eb0 legacy=0x7f845c0ba120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:49.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.842+0000 7f8459ffb640 1 -- 172.21.15.27:0/2182185285 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1514899816 0 0) 0x7f845c0d2170 con 0x7f845c0d3eb0 2024-09-12T03:17:49.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.842+0000 7f845affd640 1 -- 172.21.15.27:0/2182185285 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f845c0d2e10 con 0x7f845c0cf560 2024-09-12T03:17:49.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.843+0000 7f845affd640 1 --2- 172.21.15.27:0/2182185285 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f845c0cf560 0x7f845c0cf940 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f844c009d90 tx=0x7f844c02a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6857d6731c228984 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:49.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.843+0000 7f8459ffb640 1 -- 172.21.15.27:0/2182185285 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f844c007740 con 0x7f845c0cf560 2024-09-12T03:17:49.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.843+0000 7f8459ffb640 1 -- 172.21.15.27:0/2182185285 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f844c0078e0 con 0x7f845c0cf560 2024-09-12T03:17:49.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.843+0000 7f8459ffb640 1 -- 172.21.15.27:0/2182185285 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f844c007cb0 con 0x7f845c0cf560 2024-09-12T03:17:49.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.843+0000 7f8461a5e640 1 -- 172.21.15.27:0/2182185285 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f845c0cf560 msgr2=0x7f845c0cf940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:49.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.843+0000 7f8461a5e640 1 --2- 172.21.15.27:0/2182185285 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f845c0cf560 0x7f845c0cf940 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f844c009d90 tx=0x7f844c02a6f0 comp rx=0 tx=0).stop 2024-09-12T03:17:49.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.843+0000 7f8461a5e640 1 -- 172.21.15.27:0/2182185285 shutdown_connections 2024-09-12T03:17:49.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.843+0000 7f8461a5e640 1 --2- 172.21.15.27:0/2182185285 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f845c0b7930 0x7f845c0b7d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:49.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.843+0000 7f8461a5e640 1 --2- 172.21.15.27:0/2182185285 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f845c0cf560 0x7f845c0cf940 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:49.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.843+0000 7f8461a5e640 1 -- 172.21.15.27:0/2182185285 >> 172.21.15.27:0/2182185285 conn(0x7f845c0d1210 msgr2=0x7f845c0d1620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:49.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.843+0000 7f8461a5e640 1 -- 172.21.15.27:0/2182185285 shutdown_connections 2024-09-12T03:17:49.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.843+0000 7f8461a5e640 1 -- 172.21.15.27:0/2182185285 wait complete. 2024-09-12T03:17:49.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.844+0000 7f8461a5e640 1 Processor -- start 2024-09-12T03:17:49.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.845+0000 7f8461a5e640 1 -- start start 2024-09-12T03:17:49.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.845+0000 7f8461a5e640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f845c0b7930 0x7f845c1e8110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:49.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.845+0000 7f8461a5e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f845c0cf560 0x7f845c0ab190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:49.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.845+0000 7f8461a5e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f845c0d3eb0 0x7f845c1f6fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:49.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.845+0000 7f8461a5e640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f845c0d36b0 con 0x7f845c0d3eb0 2024-09-12T03:17:49.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.845+0000 7f8461a5e640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f845c0d33d0 con 0x7f845c0cf560 2024-09-12T03:17:49.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.845+0000 7f8461a5e640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f845c0cb520 con 0x7f845c0b7930 2024-09-12T03:17:49.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.846+0000 7f845b7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f845c0d3eb0 0x7f845c1f6fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:49.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.846+0000 7f845b7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f845c0d3eb0 0x7f845c1f6fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41082/0 (socket says 172.21.15.27:41082) 2024-09-12T03:17:49.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.846+0000 7f845b7fe640 1 -- 172.21.15.27:0/3877565609 learned_addr learned my addr 172.21.15.27:0/3877565609 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:17:49.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.846+0000 7f845affd640 1 --2- 172.21.15.27:0/3877565609 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f845c0b7930 0x7f845c1e8110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:49.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.846+0000 7f845a7fc640 1 --2- 172.21.15.27:0/3877565609 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f845c0cf560 0x7f845c0ab190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:49.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.846+0000 7f845b7fe640 1 -- 172.21.15.27:0/3877565609 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f845c0b7930 msgr2=0x7f845c1e8110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:49.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.846+0000 7f845b7fe640 1 --2- 172.21.15.27:0/3877565609 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f845c0b7930 0x7f845c1e8110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:49.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.846+0000 7f845b7fe640 1 -- 172.21.15.27:0/3877565609 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f845c0cf560 msgr2=0x7f845c0ab190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:49.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.846+0000 7f845b7fe640 1 --2- 172.21.15.27:0/3877565609 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f845c0cf560 0x7f845c0ab190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:49.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.846+0000 7f845b7fe640 1 -- 172.21.15.27:0/3877565609 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f845c14a880 con 0x7f845c0d3eb0 2024-09-12T03:17:49.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.846+0000 7f845b7fe640 1 --2- 172.21.15.27:0/3877565609 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f845c0d3eb0 0x7f845c1f6fb0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f845400c7f0 tx=0x7f845400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:49.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.846+0000 7f8453fff640 1 -- 172.21.15.27:0/3877565609 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8454015070 con 0x7f845c0d3eb0 2024-09-12T03:17:49.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.846+0000 7f8453fff640 1 -- 172.21.15.27:0/3877565609 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8454004220 con 0x7f845c0d3eb0 2024-09-12T03:17:49.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.846+0000 7f8461a5e640 1 -- 172.21.15.27:0/3877565609 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f845c14ab40 con 0x7f845c0d3eb0 2024-09-12T03:17:49.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.846+0000 7f8453fff640 1 -- 172.21.15.27:0/3877565609 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8454007620 con 0x7f845c0d3eb0 2024-09-12T03:17:49.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.846+0000 7f8461a5e640 1 -- 172.21.15.27:0/3877565609 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f845c14b100 con 0x7f845c0d3eb0 2024-09-12T03:17:49.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.847+0000 7f8461a5e640 1 -- 172.21.15.27:0/3877565609 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f845c15edd0 con 0x7f845c0d3eb0 2024-09-12T03:17:49.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.848+0000 7f8453fff640 1 -- 172.21.15.27:0/3877565609 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 7) ==== 90651+0+0 (secure 0 0 0) 0x7f84540079d0 con 0x7f845c0d3eb0 2024-09-12T03:17:49.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.848+0000 7f8453fff640 1 --2- 172.21.15.27:0/3877565609 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f843c06bfe0 0x7f843c06e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:49.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.848+0000 7f8453fff640 1 -- 172.21.15.27:0/3877565609 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5709+0+0 (secure 0 0 0) 0x7f845408b360 con 0x7f845c0d3eb0 2024-09-12T03:17:49.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.849+0000 7f845affd640 1 --2- 172.21.15.27:0/3877565609 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f843c06bfe0 0x7f843c06e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:49.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.849+0000 7f845affd640 1 --2- 172.21.15.27:0/3877565609 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f843c06bfe0 0x7f843c06e4a0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f844c034000 tx=0x7f844c02f000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:49.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:49.853+0000 7f8453fff640 1 -- 172.21.15.27:0/3877565609 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f8454059980 con 0x7f845c0d3eb0 2024-09-12T03:17:50.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:50.061+0000 7f8461a5e640 1 -- 172.21.15.27:0/3877565609 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f845c0d0cf0 con 0x7f845c0d3eb0 2024-09-12T03:17:50.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:50.061+0000 7f8453fff640 1 -- 172.21.15.27:0/3877565609 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+5223 (secure 0 0 0) 0x7f8454059280 con 0x7f845c0d3eb0 2024-09-12T03:17:50.062 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:17:50.062 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":23,"btime":"2024-09-12T03:17:47:237558+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4835,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6834/1457797481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":1457797481},{"type":"v1","addr":"172.21.15.27:6834","nonce":1457797481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5029,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6836/2842491281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":2842491281},{"type":"v1","addr":"172.21.15.45:6836","nonce":2842491281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5032,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/3339744340","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6835","nonce":3339744340},{"type":"v1","addr":"172.21.15.45:6837","nonce":3339744340}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-12T03:17:38.580473+0000","modified":"2024-09-12T03:17:40.557950+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5205},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5205":{"gid":5205,"name":"a","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.27:6835/2571101228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":2571101228},{"type":"v1","addr":"172.21.15.27:6835","nonce":2571101228}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5205,"qdb_cluster":[5205]},"id":3}]} 2024-09-12T03:17:50.062 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 23 2024-09-12T03:17:50.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:50.063+0000 7f8461a5e640 1 -- 172.21.15.27:0/3877565609 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f843c06bfe0 msgr2=0x7f843c06e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:50.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:50.063+0000 7f8461a5e640 1 --2- 172.21.15.27:0/3877565609 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f843c06bfe0 0x7f843c06e4a0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f844c034000 tx=0x7f844c02f000 comp rx=0 tx=0).stop 2024-09-12T03:17:50.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:50.063+0000 7f8461a5e640 1 -- 172.21.15.27:0/3877565609 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f845c0d3eb0 msgr2=0x7f845c1f6fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:50.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:50.063+0000 7f8461a5e640 1 --2- 172.21.15.27:0/3877565609 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f845c0d3eb0 0x7f845c1f6fb0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f845400c7f0 tx=0x7f845400ccb0 comp rx=0 tx=0).stop 2024-09-12T03:17:50.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:50.063+0000 7f8461a5e640 1 -- 172.21.15.27:0/3877565609 shutdown_connections 2024-09-12T03:17:50.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:50.063+0000 7f8461a5e640 1 --2- 172.21.15.27:0/3877565609 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f843c06bfe0 0x7f843c06e4a0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:50.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:50.063+0000 7f8461a5e640 1 --2- 172.21.15.27:0/3877565609 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f845c0d3eb0 0x7f845c1f6fb0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:50.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:50.063+0000 7f8461a5e640 1 --2- 172.21.15.27:0/3877565609 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f845c0cf560 0x7f845c0ab190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:50.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:50.063+0000 7f8461a5e640 1 --2- 172.21.15.27:0/3877565609 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f845c0b7930 0x7f845c1e8110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:50.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:50.063+0000 7f8461a5e640 1 -- 172.21.15.27:0/3877565609 >> 172.21.15.27:0/3877565609 conn(0x7f845c0d1210 msgr2=0x7f845c0b5420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:50.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:50.064+0000 7f8461a5e640 1 -- 172.21.15.27:0/3877565609 shutdown_connections 2024-09-12T03:17:50.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:50.064+0000 7f8461a5e640 1 -- 172.21.15.27:0/3877565609 wait complete. 2024-09-12T03:17:50.074 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json session ls 2024-09-12T03:17:50.196 INFO:teuthology.orchestra.run.smithi027.stdout:[{"id":4955,"entity":{"name":{"type":"client","num":4955},"addr":{"type":"any","addr":"172.21.15.45:0","nonce":3904873653}},"state":"open","num_leases":1,"num_caps":7,"request_load_avg":14,"uptime":0.49098743,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":21.975643064965961,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.4955 172.21.15.45:0/3904873653","completed_requests":[{"tid":14,"created_ino":"0x0"},{"tid":15,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc","ceph_version":"ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)","entity_id":"x","hostname":"smithi045","pid":"36189","root":"/"}}] 2024-09-12T03:17:50.196 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc", "ceph_version": "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "x", "hostname": "smithi045", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "36189", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 14 }, { "created_ino": "0x100000001f9", "tid": 15 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.45:0", "nonce": 3904873653, "type": "any" }, "name": { "num": 4955, "type": "client" } }, "id": 4955, "inst": "client.4955 172.21.15.45:0/3904873653", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 1, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 14, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 21.97564306496596 }, "state": "open", "uptime": 0.49098743 } ] 2024-09-12T03:17:50.196 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-12T03:17:50.301 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:50.301+0000 7fc618bc4640 1 Processor -- start 2024-09-12T03:17:50.301 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:50.301+0000 7fc618bc4640 1 -- start start 2024-09-12T03:17:50.303 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:50.302+0000 7fc618bc4640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc61415eb90 0x7fc6140d2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:50.303 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:50.302+0000 7fc618bc4640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fc6140d1720 0x7fc6140b8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:50.303 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:50.302+0000 7fc618bc4640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc6140d3760 con 0x7fc6140d5200 2024-09-12T03:17:50.303 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:50.302+0000 7fc618bc4640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7fc6140b7d20 con 0x7fc6140d1720 2024-09-12T03:17:50.303 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:50.302+0000 7fc618bc4640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fc6140d42e0 con 0x7fc61415eb90 2024-09-12T03:17:50.303 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:50.303+0000 7fc612d9d640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fc6140d1720 0x7fc6140b8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:50.303 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:50.303+0000 7fc612d9d640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fc6140d1720 0x7fc6140b8190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41088/0 (socket says 172.21.15.27:41088) 2024-09-12T03:17:50.303 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:50.303+0000 7fc612d9d640 1 -- 172.21.15.27:0/617674711 learned_addr learned my addr 172.21.15.27:0/617674711 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:17:50.303 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:50.303+0000 7fc61259c640 1 --2- 172.21.15.27:0/617674711 >> v2:172.21.15.45:3300/0 conn(0x7fc61415eb90 0x7fc6140d2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:50.304 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:50.303+0000 7fc612d9d640 1 -- 172.21.15.27:0/617674711 >> v2:172.21.15.45:3300/0 conn(0x7fc61415eb90 msgr2=0x7fc6140d2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:50.304 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:50.303+0000 7fc612d9d640 1 --2- 172.21.15.27:0/617674711 >> v2:172.21.15.45:3300/0 conn(0x7fc61415eb90 0x7fc6140d2df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:50.304 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:50.303+0000 7fc612d9d640 1 -- 172.21.15.27:0/617674711 >> v1:172.21.15.45:6789/0 conn(0x7fc6140d5200 legacy=0x7fc6140d1010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:50.304 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:50.303+0000 7fc612d9d640 1 -- 172.21.15.27:0/617674711 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6140d3a40 con 0x7fc6140d1720 2024-09-12T03:17:50.305 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:50.304+0000 7fc612d9d640 1 --2- 172.21.15.27:0/617674711 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc6140d1720 0x7fc6140b8190 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fc614075560 tx=0x7fc60c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c1062c91f4b2031 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:50.305 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:50.304+0000 7fc61159a640 1 -- 172.21.15.27:0/617674711 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc60c037070 con 0x7fc6140d1720 2024-09-12T03:17:50.305 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:50.304+0000 7fc61159a640 1 -- 172.21.15.27:0/617674711 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc60c032440 con 0x7fc6140d1720 2024-09-12T03:17:50.305 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:50.304+0000 7fc61159a640 1 -- 172.21.15.27:0/617674711 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc60c032810 con 0x7fc6140d1720 2024-09-12T03:17:50.305 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:50.304+0000 7fc618bc4640 1 -- 172.21.15.27:0/617674711 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc6140d1720 msgr2=0x7fc6140b8190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:50.305 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:50.304+0000 7fc618bc4640 1 --2- 172.21.15.27:0/617674711 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc6140d1720 0x7fc6140b8190 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fc614075560 tx=0x7fc60c02a270 comp rx=0 tx=0).stop 2024-09-12T03:17:50.305 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:50.304+0000 7fc618bc4640 1 -- 172.21.15.27:0/617674711 shutdown_connections 2024-09-12T03:17:50.305 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:50.304+0000 7fc618bc4640 1 --2- 172.21.15.27:0/617674711 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc6140d1720 0x7fc6140b8190 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:50.305 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:50.304+0000 7fc618bc4640 1 --2- 172.21.15.27:0/617674711 >> v2:172.21.15.45:3300/0 conn(0x7fc61415eb90 0x7fc6140d2df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:50.305 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:50.304+0000 7fc618bc4640 1 -- 172.21.15.27:0/617674711 >> 172.21.15.27:0/617674711 conn(0x7fc61406dce0 msgr2=0x7fc61406e0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:50.306 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:50.304+0000 7fc618bc4640 1 -- 172.21.15.27:0/617674711 shutdown_connections 2024-09-12T03:17:50.306 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:50.304+0000 7fc618bc4640 1 -- 172.21.15.27:0/617674711 wait complete. 2024-09-12T03:17:50.308 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:50.306+0000 7fc618bc4640 1 Processor -- start 2024-09-12T03:17:50.308 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:50.306+0000 7fc618bc4640 1 -- start start 2024-09-12T03:17:50.308 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:50.307+0000 7fc618bc4640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc6140d1720 0x7fc614154fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:50.308 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:50.307+0000 7fc618bc4640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fc6140d5200 0x7fc6140cdb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:50.308 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:50.307+0000 7fc618bc4640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc61415eb90 0x7fc6140ce0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:50.308 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:50.307+0000 7fc618bc4640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fc6140d3330 con 0x7fc61415eb90 2024-09-12T03:17:50.308 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:50.307+0000 7fc618bc4640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc6140d4000 con 0x7fc6140d1720 2024-09-12T03:17:50.308 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:50.307+0000 7fc618bc4640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fc6140d34b0 con 0x7fc6140d5200 2024-09-12T03:17:50.308 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:50.307+0000 7fc612d9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc61415eb90 0x7fc6140ce0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:50.308 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:50.307+0000 7fc612d9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc61415eb90 0x7fc6140ce0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41094/0 (socket says 172.21.15.27:41094) 2024-09-12T03:17:50.309 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:50.307+0000 7fc612d9d640 1 -- 172.21.15.27:0/4007339294 learned_addr learned my addr 172.21.15.27:0/4007339294 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:17:50.309 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:50.307+0000 7fc61259c640 1 --2- 172.21.15.27:0/4007339294 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc6140d1720 0x7fc614154fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:50.309 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:50.307+0000 7fc611d9b640 1 --2- 172.21.15.27:0/4007339294 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fc6140d5200 0x7fc6140cdb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:50.309 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:50.307+0000 7fc612d9d640 1 -- 172.21.15.27:0/4007339294 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fc6140d5200 msgr2=0x7fc6140cdb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:50.309 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:50.307+0000 7fc612d9d640 1 --2- 172.21.15.27:0/4007339294 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fc6140d5200 0x7fc6140cdb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:50.309 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:50.307+0000 7fc612d9d640 1 -- 172.21.15.27:0/4007339294 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc6140d1720 msgr2=0x7fc614154fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:50.309 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:50.307+0000 7fc612d9d640 1 --2- 172.21.15.27:0/4007339294 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc6140d1720 0x7fc614154fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:50.309 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:50.307+0000 7fc612d9d640 1 -- 172.21.15.27:0/4007339294 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc614155510 con 0x7fc61415eb90 2024-09-12T03:17:50.309 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:50.307+0000 7fc612d9d640 1 --2- 172.21.15.27:0/4007339294 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc61415eb90 0x7fc6140ce0b0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fc60c0090b0 tx=0x7fc60c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:50.309 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:50.307+0000 7fc60affd640 1 -- 172.21.15.27:0/4007339294 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc60c041020 con 0x7fc61415eb90 2024-09-12T03:17:50.309 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:50.307+0000 7fc60affd640 1 -- 172.21.15.27:0/4007339294 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc60c03d640 con 0x7fc61415eb90 2024-09-12T03:17:50.309 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:50.307+0000 7fc618bc4640 1 -- 172.21.15.27:0/4007339294 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc61415b9d0 con 0x7fc61415eb90 2024-09-12T03:17:50.309 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:50.307+0000 7fc60affd640 1 -- 172.21.15.27:0/4007339294 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc60c037040 con 0x7fc61415eb90 2024-09-12T03:17:50.309 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:50.308+0000 7fc618bc4640 1 -- 172.21.15.27:0/4007339294 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc61415bf90 con 0x7fc61415eb90 2024-09-12T03:17:50.310 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:50.309+0000 7fc618bc4640 1 -- 172.21.15.27:0/4007339294 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc61415f090 con 0x7fc61415eb90 2024-09-12T03:17:50.310 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:50.309+0000 7fc60affd640 1 -- 172.21.15.27:0/4007339294 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 8) ==== 90702+0+0 (secure 0 0 0) 0x7fc60c044020 con 0x7fc61415eb90 2024-09-12T03:17:50.310 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:50.309+0000 7fc60affd640 1 --2- 172.21.15.27:0/4007339294 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fc5f406c070 0x7fc5f406e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:50.311 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:50.310+0000 7fc60affd640 1 -- 172.21.15.27:0/4007339294 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5709+0+0 (secure 0 0 0) 0x7fc60c0ab3b0 con 0x7fc61415eb90 2024-09-12T03:17:50.311 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:50.310+0000 7fc61259c640 1 --2- 172.21.15.27:0/4007339294 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fc5f406c070 0x7fc5f406e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:50.311 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:50.310+0000 7fc61259c640 1 --2- 172.21.15.27:0/4007339294 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fc5f406c070 0x7fc5f406e530 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fc6140ce880 tx=0x7fc60400b040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:50.314 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:50.314+0000 7fc60affd640 1 -- 172.21.15.27:0/4007339294 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fc60c04b070 con 0x7fc61415eb90 2024-09-12T03:17:50.528 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:50.528+0000 7fc618bc4640 1 -- 172.21.15.27:0/4007339294 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc61415f900 con 0x7fc61415eb90 2024-09-12T03:17:50.528 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:50.528+0000 7fc60affd640 1 -- 172.21.15.27:0/4007339294 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+5223 (secure 0 0 0) 0x7fc60c079440 con 0x7fc61415eb90 2024-09-12T03:17:50.529 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:17:50.529 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":23,"btime":"2024-09-12T03:17:47:237558+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4835,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6834/1457797481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":1457797481},{"type":"v1","addr":"172.21.15.27:6834","nonce":1457797481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5029,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6836/2842491281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":2842491281},{"type":"v1","addr":"172.21.15.45:6836","nonce":2842491281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5032,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/3339744340","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6835","nonce":3339744340},{"type":"v1","addr":"172.21.15.45:6837","nonce":3339744340}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-12T03:17:38.580473+0000","modified":"2024-09-12T03:17:40.557950+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5205},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5205":{"gid":5205,"name":"a","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.27:6835/2571101228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":2571101228},{"type":"v1","addr":"172.21.15.27:6835","nonce":2571101228}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5205,"qdb_cluster":[5205]},"id":3}]} 2024-09-12T03:17:50.530 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 23 2024-09-12T03:17:50.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:50.530+0000 7fc618bc4640 1 -- 172.21.15.27:0/4007339294 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fc5f406c070 msgr2=0x7fc5f406e530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:50.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:50.530+0000 7fc618bc4640 1 --2- 172.21.15.27:0/4007339294 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fc5f406c070 0x7fc5f406e530 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fc6140ce880 tx=0x7fc60400b040 comp rx=0 tx=0).stop 2024-09-12T03:17:50.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:50.530+0000 7fc618bc4640 1 -- 172.21.15.27:0/4007339294 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc61415eb90 msgr2=0x7fc6140ce0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:50.532 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:50.530+0000 7fc618bc4640 1 --2- 172.21.15.27:0/4007339294 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc61415eb90 0x7fc6140ce0b0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fc60c0090b0 tx=0x7fc60c0046c0 comp rx=0 tx=0).stop 2024-09-12T03:17:50.532 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:50.531+0000 7fc618bc4640 1 -- 172.21.15.27:0/4007339294 shutdown_connections 2024-09-12T03:17:50.532 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:50.531+0000 7fc618bc4640 1 --2- 172.21.15.27:0/4007339294 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fc5f406c070 0x7fc5f406e530 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:50.532 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:50.531+0000 7fc618bc4640 1 --2- 172.21.15.27:0/4007339294 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc61415eb90 0x7fc6140ce0b0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:50.532 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:50.531+0000 7fc618bc4640 1 --2- 172.21.15.27:0/4007339294 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fc6140d5200 0x7fc6140cdb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:50.532 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:50.531+0000 7fc618bc4640 1 --2- 172.21.15.27:0/4007339294 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc6140d1720 0x7fc614154fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:50.532 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:50.531+0000 7fc618bc4640 1 -- 172.21.15.27:0/4007339294 >> 172.21.15.27:0/4007339294 conn(0x7fc61406dce0 msgr2=0x7fc6140b92a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:50.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:50.531+0000 7fc618bc4640 1 -- 172.21.15.27:0/4007339294 shutdown_connections 2024-09-12T03:17:50.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:50.531+0000 7fc618bc4640 1 -- 172.21.15.27:0/4007339294 wait complete. 2024-09-12T03:17:50.542 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json session ls 2024-09-12T03:17:50.663 INFO:teuthology.orchestra.run.smithi027.stdout:[{"id":4955,"entity":{"name":{"type":"client","num":4955},"addr":{"type":"any","addr":"172.21.15.45:0","nonce":3904873653}},"state":"open","num_leases":1,"num_caps":7,"request_load_avg":14,"uptime":0.95797547400000005,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":21.95194475989058,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.4955 172.21.15.45:0/3904873653","completed_requests":[{"tid":14,"created_ino":"0x0"},{"tid":15,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc","ceph_version":"ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)","entity_id":"x","hostname":"smithi045","pid":"36189","root":"/"}}] 2024-09-12T03:17:50.664 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc", "ceph_version": "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "x", "hostname": "smithi045", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "36189", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 14 }, { "created_ino": "0x100000001f9", "tid": 15 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.45:0", "nonce": 3904873653, "type": "any" }, "name": { "num": 4955, "type": "client" } }, "id": 4955, "inst": "client.4955 172.21.15.45:0/3904873653", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 1, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 14, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 21.95194475989058 }, "state": "open", "uptime": 0.957975474 } ] 2024-09-12T03:17:50.664 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2024-09-12T03:17:55.665 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-12T03:17:55.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:55.780+0000 7fd93cee6640 1 Processor -- start 2024-09-12T03:17:55.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:55.780+0000 7fd93cee6640 1 -- start start 2024-09-12T03:17:55.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:55.781+0000 7fd93cee6640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fd938145c30 0x7fd93815ca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:55.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:55.781+0000 7fd93cee6640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fd9380cda40 0x7fd9380b4040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:55.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:55.781+0000 7fd93cee6640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd9380b4d70 con 0x7fd938146010 2024-09-12T03:17:55.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:55.781+0000 7fd93cee6640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7fd9380b5050 con 0x7fd9380cda40 2024-09-12T03:17:55.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:55.781+0000 7fd93cee6640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fd93806d140 con 0x7fd938145c30 2024-09-12T03:17:55.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:55.782+0000 7fd936d9d640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fd9380cda40 0x7fd9380b4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:55.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:55.782+0000 7fd936d9d640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fd9380cda40 0x7fd9380b4040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41104/0 (socket says 172.21.15.27:41104) 2024-09-12T03:17:55.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:55.782+0000 7fd936d9d640 1 -- 172.21.15.27:0/1393751024 learned_addr learned my addr 172.21.15.27:0/1393751024 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:17:55.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:55.782+0000 7fd93659c640 1 --2- 172.21.15.27:0/1393751024 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fd938145c30 0x7fd93815ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:55.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:55.782+0000 7fd936d9d640 1 -- 172.21.15.27:0/1393751024 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fd938145c30 msgr2=0x7fd93815ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:55.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:55.782+0000 7fd936d9d640 1 --2- 172.21.15.27:0/1393751024 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fd938145c30 0x7fd93815ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:55.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:55.782+0000 7fd936d9d640 1 -- 172.21.15.27:0/1393751024 >> v1:172.21.15.45:6789/0 conn(0x7fd938146010 legacy=0x7fd9380cd330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:55.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:55.782+0000 7fd936d9d640 1 -- 172.21.15.27:0/1393751024 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd93806d420 con 0x7fd9380cda40 2024-09-12T03:17:55.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:55.783+0000 7fd936d9d640 1 --2- 172.21.15.27:0/1393751024 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd9380cda40 0x7fd9380b4040 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fd930009870 tx=0x7fd93002a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9456276c77164e90 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:55.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:55.783+0000 7fd93559a640 1 -- 172.21.15.27:0/1393751024 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd930037070 con 0x7fd9380cda40 2024-09-12T03:17:55.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:55.783+0000 7fd93559a640 1 -- 172.21.15.27:0/1393751024 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd930032440 con 0x7fd9380cda40 2024-09-12T03:17:55.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:55.783+0000 7fd93559a640 1 -- 172.21.15.27:0/1393751024 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd930032810 con 0x7fd9380cda40 2024-09-12T03:17:55.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:55.783+0000 7fd93cee6640 1 -- 172.21.15.27:0/1393751024 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd9380cda40 msgr2=0x7fd9380b4040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:55.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:55.783+0000 7fd93cee6640 1 --2- 172.21.15.27:0/1393751024 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd9380cda40 0x7fd9380b4040 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fd930009870 tx=0x7fd93002a1c0 comp rx=0 tx=0).stop 2024-09-12T03:17:55.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:55.784+0000 7fd93cee6640 1 -- 172.21.15.27:0/1393751024 shutdown_connections 2024-09-12T03:17:55.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:55.784+0000 7fd93cee6640 1 --2- 172.21.15.27:0/1393751024 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd9380cda40 0x7fd9380b4040 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:55.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:55.784+0000 7fd93cee6640 1 --2- 172.21.15.27:0/1393751024 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fd938145c30 0x7fd93815ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:55.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:55.784+0000 7fd93cee6640 1 -- 172.21.15.27:0/1393751024 >> 172.21.15.27:0/1393751024 conn(0x7fd938062c60 msgr2=0x7fd93806bb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:55.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:55.784+0000 7fd93cee6640 1 -- 172.21.15.27:0/1393751024 shutdown_connections 2024-09-12T03:17:55.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:55.784+0000 7fd93cee6640 1 -- 172.21.15.27:0/1393751024 wait complete. 2024-09-12T03:17:55.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:55.785+0000 7fd93cee6640 1 Processor -- start 2024-09-12T03:17:55.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:55.785+0000 7fd93cee6640 1 -- start start 2024-09-12T03:17:55.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:55.786+0000 7fd93cee6640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fd9380cda40 0x7fd938141850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:55.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:55.786+0000 7fd93cee6640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd938145c30 0x7fd9380b1490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:55.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:55.786+0000 7fd93cee6640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd938146010 0x7fd938140ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:55.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:55.786+0000 7fd93cee6640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fd9380b4700 con 0x7fd938145c30 2024-09-12T03:17:55.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:55.786+0000 7fd93cee6640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fd9380b4580 con 0x7fd938146010 2024-09-12T03:17:55.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:55.786+0000 7fd93cee6640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fd9380b4880 con 0x7fd9380cda40 2024-09-12T03:17:55.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:55.786+0000 7fd935d9b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd938145c30 0x7fd9380b1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:55.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:55.786+0000 7fd935d9b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd938145c30 0x7fd9380b1490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41116/0 (socket says 172.21.15.27:41116) 2024-09-12T03:17:55.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:55.786+0000 7fd93659c640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fd9380cda40 0x7fd938141850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:55.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:55.786+0000 7fd935d9b640 1 -- 172.21.15.27:0/4141503420 learned_addr learned my addr 172.21.15.27:0/4141503420 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:17:55.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:55.786+0000 7fd936d9d640 1 --2- 172.21.15.27:0/4141503420 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd938146010 0x7fd938140ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:55.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:55.786+0000 7fd935d9b640 1 -- 172.21.15.27:0/4141503420 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fd9380cda40 msgr2=0x7fd938141850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:55.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:55.786+0000 7fd935d9b640 1 --2- 172.21.15.27:0/4141503420 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fd9380cda40 0x7fd938141850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:55.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:55.786+0000 7fd935d9b640 1 -- 172.21.15.27:0/4141503420 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd938146010 msgr2=0x7fd938140ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:55.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:55.786+0000 7fd935d9b640 1 --2- 172.21.15.27:0/4141503420 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd938146010 0x7fd938140ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:55.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:55.786+0000 7fd935d9b640 1 -- 172.21.15.27:0/4141503420 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9381e2070 con 0x7fd938145c30 2024-09-12T03:17:55.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:55.787+0000 7fd935d9b640 1 --2- 172.21.15.27:0/4141503420 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd938145c30 0x7fd9380b1490 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fd92000c3c0 tx=0x7fd92000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:55.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:55.787+0000 7fd92effd640 1 -- 172.21.15.27:0/4141503420 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd920017070 con 0x7fd938145c30 2024-09-12T03:17:55.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:55.787+0000 7fd92effd640 1 -- 172.21.15.27:0/4141503420 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9200052f0 con 0x7fd938145c30 2024-09-12T03:17:55.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:55.787+0000 7fd93cee6640 1 -- 172.21.15.27:0/4141503420 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9381e2350 con 0x7fd938145c30 2024-09-12T03:17:55.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:55.787+0000 7fd92effd640 1 -- 172.21.15.27:0/4141503420 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd920003830 con 0x7fd938145c30 2024-09-12T03:17:55.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:55.787+0000 7fd93cee6640 1 -- 172.21.15.27:0/4141503420 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9381e2910 con 0x7fd938145c30 2024-09-12T03:17:55.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:55.788+0000 7fd93cee6640 1 -- 172.21.15.27:0/4141503420 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9380b4040 con 0x7fd938145c30 2024-09-12T03:17:55.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:55.788+0000 7fd92effd640 1 -- 172.21.15.27:0/4141503420 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 8) ==== 90702+0+0 (secure 0 0 0) 0x7fd92001a070 con 0x7fd938145c30 2024-09-12T03:17:55.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:55.789+0000 7fd92effd640 1 --2- 172.21.15.27:0/4141503420 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fd91806c070 0x7fd91806e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:55.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:55.789+0000 7fd92effd640 1 -- 172.21.15.27:0/4141503420 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5709+0+0 (secure 0 0 0) 0x7fd92008a400 con 0x7fd938145c30 2024-09-12T03:17:55.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:55.789+0000 7fd93659c640 1 --2- 172.21.15.27:0/4141503420 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fd91806c070 0x7fd91806e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:55.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:55.790+0000 7fd93659c640 1 --2- 172.21.15.27:0/4141503420 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fd91806c070 0x7fd91806e530 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fd9280113e0 tx=0x7fd92800b040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:55.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:55.793+0000 7fd92effd640 1 -- 172.21.15.27:0/4141503420 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fd920058560 con 0x7fd938145c30 2024-09-12T03:17:56.004 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:56.004+0000 7fd93cee6640 1 -- 172.21.15.27:0/4141503420 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd9380b1fb0 con 0x7fd938145c30 2024-09-12T03:17:56.005 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:56.005+0000 7fd92effd640 1 -- 172.21.15.27:0/4141503420 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+5223 (secure 0 0 0) 0x7fd92005c860 con 0x7fd938145c30 2024-09-12T03:17:56.005 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:17:56.005 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":23,"btime":"2024-09-12T03:17:47:237558+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4835,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6834/1457797481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":1457797481},{"type":"v1","addr":"172.21.15.27:6834","nonce":1457797481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5029,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6836/2842491281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":2842491281},{"type":"v1","addr":"172.21.15.45:6836","nonce":2842491281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5032,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/3339744340","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6835","nonce":3339744340},{"type":"v1","addr":"172.21.15.45:6837","nonce":3339744340}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-12T03:17:38.580473+0000","modified":"2024-09-12T03:17:40.557950+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5205},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5205":{"gid":5205,"name":"a","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.27:6835/2571101228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":2571101228},{"type":"v1","addr":"172.21.15.27:6835","nonce":2571101228}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5205,"qdb_cluster":[5205]},"id":3}]} 2024-09-12T03:17:56.005 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 23 2024-09-12T03:17:56.007 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:56.007+0000 7fd93cee6640 1 -- 172.21.15.27:0/4141503420 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fd91806c070 msgr2=0x7fd91806e530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:56.007 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:56.007+0000 7fd93cee6640 1 --2- 172.21.15.27:0/4141503420 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fd91806c070 0x7fd91806e530 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fd9280113e0 tx=0x7fd92800b040 comp rx=0 tx=0).stop 2024-09-12T03:17:56.007 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:56.007+0000 7fd93cee6640 1 -- 172.21.15.27:0/4141503420 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd938145c30 msgr2=0x7fd9380b1490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:56.007 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:56.007+0000 7fd93cee6640 1 --2- 172.21.15.27:0/4141503420 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd938145c30 0x7fd9380b1490 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fd92000c3c0 tx=0x7fd92000c880 comp rx=0 tx=0).stop 2024-09-12T03:17:56.007 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:56.007+0000 7fd93cee6640 1 -- 172.21.15.27:0/4141503420 shutdown_connections 2024-09-12T03:17:56.007 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:56.007+0000 7fd93cee6640 1 --2- 172.21.15.27:0/4141503420 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fd91806c070 0x7fd91806e530 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:56.008 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:56.007+0000 7fd93cee6640 1 --2- 172.21.15.27:0/4141503420 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd938146010 0x7fd938140ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:56.008 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:56.007+0000 7fd93cee6640 1 --2- 172.21.15.27:0/4141503420 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd938145c30 0x7fd9380b1490 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:56.008 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:56.007+0000 7fd93cee6640 1 --2- 172.21.15.27:0/4141503420 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fd9380cda40 0x7fd938141850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:56.008 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:56.007+0000 7fd93cee6640 1 -- 172.21.15.27:0/4141503420 >> 172.21.15.27:0/4141503420 conn(0x7fd938062c60 msgr2=0x7fd9380b2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:56.008 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:56.007+0000 7fd93cee6640 1 -- 172.21.15.27:0/4141503420 shutdown_connections 2024-09-12T03:17:56.008 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:17:56.007+0000 7fd93cee6640 1 -- 172.21.15.27:0/4141503420 wait complete. 2024-09-12T03:17:56.018 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json session ls 2024-09-12T03:17:56.139 INFO:teuthology.orchestra.run.smithi027.stdout:[{"id":4955,"entity":{"name":{"type":"client","num":4955},"addr":{"type":"any","addr":"172.21.15.45:0","nonce":3904873653}},"state":"open","num_leases":1,"num_caps":7,"request_load_avg":13,"uptime":6.4348352569999996,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":21.675909922874347,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.4955 172.21.15.45:0/3904873653","completed_requests":[{"tid":14,"created_ino":"0x0"},{"tid":15,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc","ceph_version":"ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)","entity_id":"x","hostname":"smithi045","pid":"36189","root":"/"}}] 2024-09-12T03:17:56.140 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc", "ceph_version": "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "x", "hostname": "smithi045", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "36189", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 14 }, { "created_ino": "0x100000001f9", "tid": 15 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.45:0", "nonce": 3904873653, "type": "any" }, "name": { "num": 4955, "type": "client" } }, "id": 4955, "inst": "client.4955 172.21.15.45:0/3904873653", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 1, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 13, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 21.675909922874347 }, "state": "open", "uptime": 6.434835257 } ] 2024-09-12T03:17:56.140 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2024-09-12T03:18:00.599 DEBUG:teuthology.orchestra.run.smithi027:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-12T03:18:00.602 DEBUG:teuthology.orchestra.run.smithi045:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-12T03:18:01.141 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-12T03:18:01.257 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:01.255+0000 7f5f9728d640 1 Processor -- start 2024-09-12T03:18:01.257 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:01.256+0000 7f5f9728d640 1 -- start start 2024-09-12T03:18:01.265 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:01.256+0000 7f5f9728d640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f5f9015eb90 0x7f5f900d2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:01.266 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:01.256+0000 7f5f9728d640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5f900d3760 con 0x7f5f900d5200 2024-09-12T03:18:01.266 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:01.256+0000 7f5f9728d640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5f900d3a40 con 0x7f5f900d1720 2024-09-12T03:18:01.266 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:01.256+0000 7f5f9728d640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f5f900b7d20 con 0x7f5f9015eb90 2024-09-12T03:18:01.266 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:01.257+0000 7f5f94828640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f5f900d5200 0x7f5f900d1010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:55920/0 (socket says 172.21.15.27:55920) 2024-09-12T03:18:01.266 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:01.257+0000 7f5f94828640 1 -- 172.21.15.27:0/3294961027 learned_addr learned my addr 172.21.15.27:0/3294961027 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:18:01.266 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:01.257+0000 7f5f95029640 1 --2- 172.21.15.27:0/3294961027 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f5f9015eb90 0x7f5f900d2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:01.267 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:01.257+0000 7f5f8ffff640 1 -- 172.21.15.27:0/3294961027 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1828853208 0 0) 0x7f5f900d3760 con 0x7f5f900d5200 2024-09-12T03:18:01.267 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:01.257+0000 7f5f8ffff640 1 -- 172.21.15.27:0/3294961027 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5f78003520 con 0x7f5f900d5200 2024-09-12T03:18:01.267 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:01.257+0000 7f5f8ffff640 1 -- 172.21.15.27:0/3294961027 <== mon.1 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4116279025 0 0) 0x7f5f900d3a40 con 0x7f5f900d1720 2024-09-12T03:18:01.267 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:01.257+0000 7f5f8ffff640 1 -- 172.21.15.27:0/3294961027 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5f900d3760 con 0x7f5f900d1720 2024-09-12T03:18:01.267 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:01.257+0000 7f5f95029640 1 -- 172.21.15.27:0/3294961027 >> v1:172.21.15.45:6789/0 conn(0x7f5f900d1720 legacy=0x7f5f900b8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:01.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:01.257+0000 7f5f95029640 1 -- 172.21.15.27:0/3294961027 >> v1:172.21.15.27:6789/0 conn(0x7f5f900d5200 legacy=0x7f5f900d1010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:01.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:01.257+0000 7f5f8ffff640 1 -- 172.21.15.27:0/3294961027 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1631603342 0 0) 0x7f5f78003520 con 0x7f5f900d5200 2024-09-12T03:18:01.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:01.257+0000 7f5f95029640 1 -- 172.21.15.27:0/3294961027 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f900d3d20 con 0x7f5f9015eb90 2024-09-12T03:18:01.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:01.258+0000 7f5f95029640 1 --2- 172.21.15.27:0/3294961027 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f5f9015eb90 0x7f5f900d2df0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f5f800097d0 tx=0x7f5f8002a1a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=277744edf0c06294 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:01.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:01.258+0000 7f5f8ffff640 1 -- 172.21.15.27:0/3294961027 <== mon.2 v2:172.21.15.45:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f80037070 con 0x7f5f9015eb90 2024-09-12T03:18:01.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:01.258+0000 7f5f8ffff640 1 -- 172.21.15.27:0/3294961027 <== mon.2 v2:172.21.15.45:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f8002ac30 con 0x7f5f9015eb90 2024-09-12T03:18:01.269 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:01.258+0000 7f5f8ffff640 1 -- 172.21.15.27:0/3294961027 <== mon.2 v2:172.21.15.45:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f840019f0 con 0x7f5f9015eb90 2024-09-12T03:18:01.269 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:01.259+0000 7f5f9728d640 1 -- 172.21.15.27:0/3294961027 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f5f9015eb90 msgr2=0x7f5f900d2df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:01.269 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:01.259+0000 7f5f9728d640 1 --2- 172.21.15.27:0/3294961027 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f5f9015eb90 0x7f5f900d2df0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f5f800097d0 tx=0x7f5f8002a1a0 comp rx=0 tx=0).stop 2024-09-12T03:18:01.269 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:01.259+0000 7f5f9728d640 1 -- 172.21.15.27:0/3294961027 shutdown_connections 2024-09-12T03:18:01.269 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:01.259+0000 7f5f9728d640 1 --2- 172.21.15.27:0/3294961027 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f5f9015eb90 0x7f5f900d2df0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:01.269 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:01.259+0000 7f5f9728d640 1 -- 172.21.15.27:0/3294961027 >> 172.21.15.27:0/3294961027 conn(0x7f5f9006dce0 msgr2=0x7f5f9006e0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:18:01.270 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:01.259+0000 7f5f9728d640 1 -- 172.21.15.27:0/3294961027 shutdown_connections 2024-09-12T03:18:01.270 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:01.259+0000 7f5f9728d640 1 -- 172.21.15.27:0/3294961027 wait complete. 2024-09-12T03:18:01.270 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:01.260+0000 7f5f9728d640 1 Processor -- start 2024-09-12T03:18:01.270 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:01.260+0000 7f5f9728d640 1 -- start start 2024-09-12T03:18:01.270 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:01.261+0000 7f5f9728d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5f900d1720 0x7f5f9015fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:01.270 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:01.261+0000 7f5f9728d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5f900d5200 0x7f5f901601c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:01.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:01.261+0000 7f5f9728d640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f5f9015eb90 0x7f5f901f7920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:01.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:01.261+0000 7f5f9728d640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f5f900d3330 con 0x7f5f900d5200 2024-09-12T03:18:01.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:01.261+0000 7f5f9728d640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f5f900d42e0 con 0x7f5f900d1720 2024-09-12T03:18:01.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:01.261+0000 7f5f9728d640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f5f900d34b0 con 0x7f5f9015eb90 2024-09-12T03:18:01.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:01.261+0000 7f5f94828640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5f900d5200 0x7f5f901601c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:01.272 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:01.261+0000 7f5f94828640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5f900d5200 0x7f5f901601c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:50826/0 (socket says 172.21.15.27:50826) 2024-09-12T03:18:01.272 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:01.261+0000 7f5f94828640 1 -- 172.21.15.27:0/2442065417 learned_addr learned my addr 172.21.15.27:0/2442065417 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:18:01.272 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:01.261+0000 7f5f94828640 1 -- 172.21.15.27:0/2442065417 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f5f9015eb90 msgr2=0x7f5f901f7920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:01.272 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:01.261+0000 7f5f94828640 1 --2- 172.21.15.27:0/2442065417 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f5f9015eb90 0x7f5f901f7920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:01.272 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:01.261+0000 7f5f95029640 1 --2- 172.21.15.27:0/2442065417 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5f900d1720 0x7f5f9015fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:01.272 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:01.261+0000 7f5f94828640 1 -- 172.21.15.27:0/2442065417 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5f900d1720 msgr2=0x7f5f9015fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:01.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:01.261+0000 7f5f94828640 1 --2- 172.21.15.27:0/2442065417 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5f900d1720 0x7f5f9015fc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:01.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:01.261+0000 7f5f94828640 1 -- 172.21.15.27:0/2442065417 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f901f8650 con 0x7f5f900d5200 2024-09-12T03:18:01.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:01.262+0000 7f5f94828640 1 --2- 172.21.15.27:0/2442065417 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5f900d5200 0x7f5f901601c0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f5f84001fb0 tx=0x7f5f8400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:01.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:01.262+0000 7f5f8dffb640 1 -- 172.21.15.27:0/2442065417 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f84017070 con 0x7f5f900d5200 2024-09-12T03:18:01.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:01.262+0000 7f5f8dffb640 1 -- 172.21.15.27:0/2442065417 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f84012400 con 0x7f5f900d5200 2024-09-12T03:18:01.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:01.262+0000 7f5f8dffb640 1 -- 172.21.15.27:0/2442065417 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f840127f0 con 0x7f5f900d5200 2024-09-12T03:18:01.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:01.262+0000 7f5f9728d640 1 -- 172.21.15.27:0/2442065417 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f9014da20 con 0x7f5f900d5200 2024-09-12T03:18:01.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:01.262+0000 7f5f9728d640 1 -- 172.21.15.27:0/2442065417 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f9014dfe0 con 0x7f5f900d5200 2024-09-12T03:18:01.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:01.263+0000 7f5f9728d640 1 -- 172.21.15.27:0/2442065417 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f900d2df0 con 0x7f5f900d5200 2024-09-12T03:18:01.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:01.263+0000 7f5f8dffb640 1 -- 172.21.15.27:0/2442065417 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 8) ==== 90702+0+0 (secure 0 0 0) 0x7f5f8401a070 con 0x7f5f900d5200 2024-09-12T03:18:01.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:01.264+0000 7f5f8dffb640 1 --2- 172.21.15.27:0/2442065417 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f5f7c06c070 0x7f5f7c06e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:01.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:01.264+0000 7f5f8dffb640 1 -- 172.21.15.27:0/2442065417 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5709+0+0 (secure 0 0 0) 0x7f5f8408b1b0 con 0x7f5f900d5200 2024-09-12T03:18:01.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:01.264+0000 7f5f95029640 1 --2- 172.21.15.27:0/2442065417 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f5f7c06c070 0x7f5f7c06e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:01.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:01.265+0000 7f5f95029640 1 --2- 172.21.15.27:0/2442065417 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f5f7c06c070 0x7f5f7c06e530 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f5f80034000 tx=0x7f5f8002e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:01.276 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:01.268+0000 7f5f8dffb640 1 -- 172.21.15.27:0/2442065417 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f5f840591b0 con 0x7f5f900d5200 2024-09-12T03:18:01.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:01.480+0000 7f5f9728d640 1 -- 172.21.15.27:0/2442065417 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5f9015f940 con 0x7f5f900d5200 2024-09-12T03:18:01.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:01.480+0000 7f5f8dffb640 1 -- 172.21.15.27:0/2442065417 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+5223 (secure 0 0 0) 0x7f5f8405d4b0 con 0x7f5f900d5200 2024-09-12T03:18:01.482 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:18:01.482 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":23,"btime":"2024-09-12T03:17:47:237558+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4835,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6834/1457797481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":1457797481},{"type":"v1","addr":"172.21.15.27:6834","nonce":1457797481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5029,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6836/2842491281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":2842491281},{"type":"v1","addr":"172.21.15.45:6836","nonce":2842491281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5032,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/3339744340","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6835","nonce":3339744340},{"type":"v1","addr":"172.21.15.45:6837","nonce":3339744340}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-12T03:17:38.580473+0000","modified":"2024-09-12T03:17:40.557950+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5205},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5205":{"gid":5205,"name":"a","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.27:6835/2571101228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":2571101228},{"type":"v1","addr":"172.21.15.27:6835","nonce":2571101228}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5205,"qdb_cluster":[5205]},"id":3}]} 2024-09-12T03:18:01.482 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 23 2024-09-12T03:18:01.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:01.482+0000 7f5f9728d640 1 -- 172.21.15.27:0/2442065417 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f5f7c06c070 msgr2=0x7f5f7c06e530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:01.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:01.482+0000 7f5f9728d640 1 --2- 172.21.15.27:0/2442065417 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f5f7c06c070 0x7f5f7c06e530 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f5f80034000 tx=0x7f5f8002e000 comp rx=0 tx=0).stop 2024-09-12T03:18:01.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:01.482+0000 7f5f9728d640 1 -- 172.21.15.27:0/2442065417 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5f900d5200 msgr2=0x7f5f901601c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:01.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:01.482+0000 7f5f9728d640 1 --2- 172.21.15.27:0/2442065417 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5f900d5200 0x7f5f901601c0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f5f84001fb0 tx=0x7f5f8400a430 comp rx=0 tx=0).stop 2024-09-12T03:18:01.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:01.482+0000 7f5f9728d640 1 -- 172.21.15.27:0/2442065417 shutdown_connections 2024-09-12T03:18:01.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:01.482+0000 7f5f9728d640 1 --2- 172.21.15.27:0/2442065417 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f5f7c06c070 0x7f5f7c06e530 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:01.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:01.482+0000 7f5f9728d640 1 --2- 172.21.15.27:0/2442065417 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f5f9015eb90 0x7f5f901f7920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:01.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:01.482+0000 7f5f9728d640 1 --2- 172.21.15.27:0/2442065417 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5f900d5200 0x7f5f901601c0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:01.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:01.482+0000 7f5f9728d640 1 --2- 172.21.15.27:0/2442065417 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5f900d1720 0x7f5f9015fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:01.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:01.482+0000 7f5f9728d640 1 -- 172.21.15.27:0/2442065417 >> 172.21.15.27:0/2442065417 conn(0x7f5f9006dce0 msgr2=0x7f5f900ce180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:18:01.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:01.483+0000 7f5f9728d640 1 -- 172.21.15.27:0/2442065417 shutdown_connections 2024-09-12T03:18:01.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:01.483+0000 7f5f9728d640 1 -- 172.21.15.27:0/2442065417 wait complete. 2024-09-12T03:18:01.495 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json session ls 2024-09-12T03:18:01.617 INFO:teuthology.orchestra.run.smithi027.stdout:[{"id":4955,"entity":{"name":{"type":"client","num":4955},"addr":{"type":"any","addr":"172.21.15.45:0","nonce":3904873653}},"state":"open","num_leases":1,"num_caps":7,"request_load_avg":13,"uptime":11.911695038,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":21.403346087368512,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.4955 172.21.15.45:0/3904873653","completed_requests":[{"tid":14,"created_ino":"0x0"},{"tid":15,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc","ceph_version":"ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)","entity_id":"x","hostname":"smithi045","pid":"36189","root":"/"}}] 2024-09-12T03:18:01.617 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc", "ceph_version": "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "x", "hostname": "smithi045", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "36189", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 14 }, { "created_ino": "0x100000001f9", "tid": 15 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.45:0", "nonce": 3904873653, "type": "any" }, "name": { "num": 4955, "type": "client" } }, "id": 4955, "inst": "client.4955 172.21.15.45:0/3904873653", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 1, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 13, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 21.40334608736851 }, "state": "open", "uptime": 11.911695038 } ] 2024-09-12T03:18:01.618 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2024-09-12T03:18:06.618 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-12T03:18:06.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:06.733+0000 7f40e0703640 1 Processor -- start 2024-09-12T03:18:06.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:06.733+0000 7f40e0703640 1 -- start start 2024-09-12T03:18:06.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:06.734+0000 7f40e0703640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f40d80cef50 0x7f40d80cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:06.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:06.734+0000 7f40e0703640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f40d8070e80 0x7f40d80d0440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:06.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:06.734+0000 7f40e0703640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f40d806fa00 con 0x7f40d80d0980 2024-09-12T03:18:06.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:06.734+0000 7f40e0703640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f40d815ca90 con 0x7f40d8070e80 2024-09-12T03:18:06.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:06.734+0000 7f40e0703640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f40d80cf870 con 0x7f40d80cef50 2024-09-12T03:18:06.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:06.735+0000 7f40de49f640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f40d80cef50 0x7f40d80cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:06.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:06.735+0000 7f40ddc9e640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f40d8070e80 0x7f40d80d0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:06.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:06.735+0000 7f40deca0640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f40d80d0980 0x7f40d80d0e10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.27:44460/0 (socket says 172.21.15.27:44460) 2024-09-12T03:18:06.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:06.735+0000 7f40deca0640 1 -- 172.21.15.27:0/2663067389 learned_addr learned my addr 172.21.15.27:0/2663067389 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:18:06.738 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:06.735+0000 7f40de49f640 1 -- 172.21.15.27:0/2663067389 >> v2:172.21.15.45:3300/0 conn(0x7f40d8070e80 msgr2=0x7f40d80d0440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:18:06.738 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:06.735+0000 7f40de49f640 1 --2- 172.21.15.27:0/2663067389 >> v2:172.21.15.45:3300/0 conn(0x7f40d8070e80 0x7f40d80d0440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:06.738 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:06.735+0000 7f40de49f640 1 -- 172.21.15.27:0/2663067389 >> v1:172.21.15.45:6789/0 conn(0x7f40d80d0980 legacy=0x7f40d80d0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:06.738 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:06.735+0000 7f40de49f640 1 -- 172.21.15.27:0/2663067389 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40d806f720 con 0x7f40d80cef50 2024-09-12T03:18:06.738 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:06.735+0000 7f40ddc9e640 1 --2- 172.21.15.27:0/2663067389 >> v2:172.21.15.45:3300/0 conn(0x7f40d8070e80 0x7f40d80d0440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:18:06.738 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:06.736+0000 7f40de49f640 1 --2- 172.21.15.27:0/2663067389 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f40d80cef50 0x7f40d80cf330 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f40d40097a0 tx=0x7f40d402a060 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6743c03284f11844 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:06.738 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:06.736+0000 7f40dd49d640 1 -- 172.21.15.27:0/2663067389 <== mon.2 v2:172.21.15.45:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40d4037070 con 0x7f40d80cef50 2024-09-12T03:18:06.738 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:06.736+0000 7f40dd49d640 1 -- 172.21.15.27:0/2663067389 <== mon.2 v2:172.21.15.45:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40d402adb0 con 0x7f40d80cef50 2024-09-12T03:18:06.738 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:06.736+0000 7f40dd49d640 1 -- 172.21.15.27:0/2663067389 <== mon.2 v2:172.21.15.45:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40d4032660 con 0x7f40d80cef50 2024-09-12T03:18:06.738 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:06.736+0000 7f40e0703640 1 -- 172.21.15.27:0/2663067389 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f40d80cef50 msgr2=0x7f40d80cf330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:06.738 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:06.736+0000 7f40e0703640 1 --2- 172.21.15.27:0/2663067389 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f40d80cef50 0x7f40d80cf330 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f40d40097a0 tx=0x7f40d402a060 comp rx=0 tx=0).stop 2024-09-12T03:18:06.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:06.737+0000 7f40e0703640 1 -- 172.21.15.27:0/2663067389 shutdown_connections 2024-09-12T03:18:06.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:06.737+0000 7f40e0703640 1 --2- 172.21.15.27:0/2663067389 >> v2:172.21.15.45:3300/0 conn(0x7f40d8070e80 0x7f40d80d0440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:06.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:06.737+0000 7f40e0703640 1 --2- 172.21.15.27:0/2663067389 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f40d80cef50 0x7f40d80cf330 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:06.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:06.737+0000 7f40e0703640 1 -- 172.21.15.27:0/2663067389 >> 172.21.15.27:0/2663067389 conn(0x7f40d8073800 msgr2=0x7f40d809ba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:18:06.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:06.737+0000 7f40e0703640 1 -- 172.21.15.27:0/2663067389 shutdown_connections 2024-09-12T03:18:06.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:06.737+0000 7f40e0703640 1 -- 172.21.15.27:0/2663067389 wait complete. 2024-09-12T03:18:06.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:06.738+0000 7f40e0703640 1 Processor -- start 2024-09-12T03:18:06.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:06.738+0000 7f40e0703640 1 -- start start 2024-09-12T03:18:06.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:06.739+0000 7f40e0703640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f40d8070e80 0x7f40d81562f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:06.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:06.739+0000 7f40e0703640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f40d80cef50 0x7f40d8150660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:06.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:06.739+0000 7f40e0703640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f40d80d0980 0x7f40d8150ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:06.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:06.739+0000 7f40e0703640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f40d806f360 con 0x7f40d80cef50 2024-09-12T03:18:06.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:06.739+0000 7f40e0703640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f40d80d2d10 con 0x7f40d8070e80 2024-09-12T03:18:06.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:06.739+0000 7f40e0703640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f40d806f4e0 con 0x7f40d80d0980 2024-09-12T03:18:06.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:06.739+0000 7f40ddc9e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f40d80cef50 0x7f40d8150660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:06.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:06.739+0000 7f40ddc9e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f40d80cef50 0x7f40d8150660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:50840/0 (socket says 172.21.15.27:50840) 2024-09-12T03:18:06.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:06.739+0000 7f40ddc9e640 1 -- 172.21.15.27:0/4158339381 learned_addr learned my addr 172.21.15.27:0/4158339381 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:18:06.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:06.739+0000 7f40deca0640 1 --2- 172.21.15.27:0/4158339381 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f40d80d0980 0x7f40d8150ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:06.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:06.739+0000 7f40de49f640 1 --2- 172.21.15.27:0/4158339381 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f40d8070e80 0x7f40d81562f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:06.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:06.739+0000 7f40ddc9e640 1 -- 172.21.15.27:0/4158339381 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f40d80d0980 msgr2=0x7f40d8150ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:06.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:06.739+0000 7f40ddc9e640 1 --2- 172.21.15.27:0/4158339381 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f40d80d0980 0x7f40d8150ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:06.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:06.739+0000 7f40ddc9e640 1 -- 172.21.15.27:0/4158339381 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f40d8070e80 msgr2=0x7f40d81562f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:06.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:06.739+0000 7f40ddc9e640 1 --2- 172.21.15.27:0/4158339381 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f40d8070e80 0x7f40d81562f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:06.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:06.739+0000 7f40ddc9e640 1 -- 172.21.15.27:0/4158339381 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40d8156830 con 0x7f40d80cef50 2024-09-12T03:18:06.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:06.740+0000 7f40ddc9e640 1 --2- 172.21.15.27:0/4158339381 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f40d80cef50 0x7f40d8150660 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f40c800e910 tx=0x7f40c800edd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:06.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:06.740+0000 7f40ceffd640 1 -- 172.21.15.27:0/4158339381 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40c8017070 con 0x7f40d80cef50 2024-09-12T03:18:06.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:06.740+0000 7f40ceffd640 1 -- 172.21.15.27:0/4158339381 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40c800ca00 con 0x7f40d80cef50 2024-09-12T03:18:06.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:06.740+0000 7f40e0703640 1 -- 172.21.15.27:0/4158339381 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40d8153120 con 0x7f40d80cef50 2024-09-12T03:18:06.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:06.740+0000 7f40ceffd640 1 -- 172.21.15.27:0/4158339381 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40c800cdb0 con 0x7f40d80cef50 2024-09-12T03:18:06.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:06.740+0000 7f40e0703640 1 -- 172.21.15.27:0/4158339381 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40d806dab0 con 0x7f40d80cef50 2024-09-12T03:18:06.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:06.741+0000 7f40e0703640 1 -- 172.21.15.27:0/4158339381 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40d80712c0 con 0x7f40d80cef50 2024-09-12T03:18:06.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:06.741+0000 7f40ceffd640 1 -- 172.21.15.27:0/4158339381 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 8) ==== 90702+0+0 (secure 0 0 0) 0x7f40c801a070 con 0x7f40d80cef50 2024-09-12T03:18:06.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:06.742+0000 7f40ceffd640 1 --2- 172.21.15.27:0/4158339381 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f40bc06c070 0x7f40bc06e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:06.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:06.742+0000 7f40ceffd640 1 -- 172.21.15.27:0/4158339381 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5709+0+0 (secure 0 0 0) 0x7f40c808b070 con 0x7f40d80cef50 2024-09-12T03:18:06.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:06.742+0000 7f40de49f640 1 --2- 172.21.15.27:0/4158339381 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f40bc06c070 0x7f40bc06e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:06.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:06.743+0000 7f40de49f640 1 --2- 172.21.15.27:0/4158339381 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f40bc06c070 0x7f40bc06e530 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f40d4034000 tx=0x7f40d402e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:06.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:06.746+0000 7f40ceffd640 1 -- 172.21.15.27:0/4158339381 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f40c8059070 con 0x7f40d80cef50 2024-09-12T03:18:06.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:06.957+0000 7f40e0703640 1 -- 172.21.15.27:0/4158339381 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f40d80b5f10 con 0x7f40d80cef50 2024-09-12T03:18:06.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:06.957+0000 7f40ceffd640 1 -- 172.21.15.27:0/4158339381 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+5223 (secure 0 0 0) 0x7f40c805d370 con 0x7f40d80cef50 2024-09-12T03:18:06.958 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:18:06.959 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":23,"btime":"2024-09-12T03:17:47:237558+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4835,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6834/1457797481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":1457797481},{"type":"v1","addr":"172.21.15.27:6834","nonce":1457797481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5029,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6836/2842491281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":2842491281},{"type":"v1","addr":"172.21.15.45:6836","nonce":2842491281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5032,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/3339744340","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6835","nonce":3339744340},{"type":"v1","addr":"172.21.15.45:6837","nonce":3339744340}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-12T03:17:38.580473+0000","modified":"2024-09-12T03:17:40.557950+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5205},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5205":{"gid":5205,"name":"a","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.27:6835/2571101228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":2571101228},{"type":"v1","addr":"172.21.15.27:6835","nonce":2571101228}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5205,"qdb_cluster":[5205]},"id":3}]} 2024-09-12T03:18:06.959 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 23 2024-09-12T03:18:06.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:06.959+0000 7f40e0703640 1 -- 172.21.15.27:0/4158339381 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f40bc06c070 msgr2=0x7f40bc06e530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:06.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:06.959+0000 7f40e0703640 1 --2- 172.21.15.27:0/4158339381 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f40bc06c070 0x7f40bc06e530 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f40d4034000 tx=0x7f40d402e000 comp rx=0 tx=0).stop 2024-09-12T03:18:06.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:06.959+0000 7f40e0703640 1 -- 172.21.15.27:0/4158339381 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f40d80cef50 msgr2=0x7f40d8150660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:06.961 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:06.959+0000 7f40e0703640 1 --2- 172.21.15.27:0/4158339381 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f40d80cef50 0x7f40d8150660 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f40c800e910 tx=0x7f40c800edd0 comp rx=0 tx=0).stop 2024-09-12T03:18:06.961 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:06.959+0000 7f40e0703640 1 -- 172.21.15.27:0/4158339381 shutdown_connections 2024-09-12T03:18:06.961 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:06.959+0000 7f40e0703640 1 --2- 172.21.15.27:0/4158339381 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f40bc06c070 0x7f40bc06e530 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:06.961 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:06.959+0000 7f40e0703640 1 --2- 172.21.15.27:0/4158339381 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f40d80d0980 0x7f40d8150ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:06.961 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:06.959+0000 7f40e0703640 1 --2- 172.21.15.27:0/4158339381 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f40d80cef50 0x7f40d8150660 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:06.961 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:06.959+0000 7f40e0703640 1 --2- 172.21.15.27:0/4158339381 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f40d8070e80 0x7f40d81562f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:06.961 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:06.959+0000 7f40e0703640 1 -- 172.21.15.27:0/4158339381 >> 172.21.15.27:0/4158339381 conn(0x7f40d8073800 msgr2=0x7f40d80d1630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:18:06.961 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:06.959+0000 7f40e0703640 1 -- 172.21.15.27:0/4158339381 shutdown_connections 2024-09-12T03:18:06.961 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:06.959+0000 7f40e0703640 1 -- 172.21.15.27:0/4158339381 wait complete. 2024-09-12T03:18:06.971 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json session ls 2024-09-12T03:18:07.093 INFO:teuthology.orchestra.run.smithi027.stdout:[{"id":4955,"entity":{"name":{"type":"client","num":4955},"addr":{"type":"any","addr":"172.21.15.45:0","nonce":3904873653}},"state":"open","num_leases":1,"num_caps":7,"request_load_avg":12,"uptime":17.387554844,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":21.134258436375738,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.4955 172.21.15.45:0/3904873653","completed_requests":[{"tid":14,"created_ino":"0x0"},{"tid":15,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc","ceph_version":"ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)","entity_id":"x","hostname":"smithi045","pid":"36189","root":"/"}}] 2024-09-12T03:18:07.093 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc", "ceph_version": "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "x", "hostname": "smithi045", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "36189", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 14 }, { "created_ino": "0x100000001f9", "tid": 15 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.45:0", "nonce": 3904873653, "type": "any" }, "name": { "num": 4955, "type": "client" } }, "id": 4955, "inst": "client.4955 172.21.15.45:0/3904873653", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 1, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 12, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 21.134258436375738 }, "state": "open", "uptime": 17.387554844 } ] 2024-09-12T03:18:07.094 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2024-09-12T03:18:12.095 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-12T03:18:12.211 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:12.210+0000 7fb7eeaa5640 1 Processor -- start 2024-09-12T03:18:12.211 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:12.210+0000 7fb7eeaa5640 1 -- start start 2024-09-12T03:18:12.212 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:12.211+0000 7fb7eeaa5640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fb7e806c240 0x7fb7e806e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:12.212 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:12.211+0000 7fb7eeaa5640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb7e81463d0 con 0x7fb7e8160d80 2024-09-12T03:18:12.213 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:12.211+0000 7fb7eeaa5640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb7e806c7f0 con 0x7fb7e80ce6d0 2024-09-12T03:18:12.213 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:12.211+0000 7fb7eeaa5640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fb7e81602a0 con 0x7fb7e806c240 2024-09-12T03:18:12.213 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:12.211+0000 7fb7e7fff640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fb7e8160d80 0x7fb7e806bb30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:53684/0 (socket says 172.21.15.27:53684) 2024-09-12T03:18:12.213 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:12.211+0000 7fb7e7fff640 1 -- 172.21.15.27:0/20297434 learned_addr learned my addr 172.21.15.27:0/20297434 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:18:12.213 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:12.211+0000 7fb7ed042640 1 --2- 172.21.15.27:0/20297434 >> v2:172.21.15.45:3300/0 conn(0x7fb7e806c240 0x7fb7e806e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:12.213 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:12.211+0000 7fb7e77fe640 1 -- 172.21.15.27:0/20297434 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3672518318 0 0) 0x7fb7e81463d0 con 0x7fb7e8160d80 2024-09-12T03:18:12.213 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:12.211+0000 7fb7e77fe640 1 -- 172.21.15.27:0/20297434 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb7d8003520 con 0x7fb7e8160d80 2024-09-12T03:18:12.213 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:12.212+0000 7fb7e77fe640 1 -- 172.21.15.27:0/20297434 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1497734380 0 0) 0x7fb7d8003520 con 0x7fb7e8160d80 2024-09-12T03:18:12.213 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:12.212+0000 7fb7ed042640 1 -- 172.21.15.27:0/20297434 >> v1:172.21.15.45:6789/0 conn(0x7fb7e80ce6d0 legacy=0x7fb7e80ceab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:12.213 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:12.212+0000 7fb7ed042640 1 -- 172.21.15.27:0/20297434 >> v1:172.21.15.27:6789/0 conn(0x7fb7e8160d80 legacy=0x7fb7e806bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:12.213 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:12.212+0000 7fb7ed042640 1 -- 172.21.15.27:0/20297434 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7e8160580 con 0x7fb7e806c240 2024-09-12T03:18:12.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:12.212+0000 7fb7ed042640 1 --2- 172.21.15.27:0/20297434 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb7e806c240 0x7fb7e806e5b0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fb7e0009d90 tx=0x7fb7e002a5f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e231761ab1bf6d50 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:12.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:12.213+0000 7fb7e77fe640 1 -- 172.21.15.27:0/20297434 <== mon.1 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7d4001670 con 0x7fb7e806c240 2024-09-12T03:18:12.215 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:12.213+0000 7fb7e77fe640 1 -- 172.21.15.27:0/20297434 <== mon.1 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7e0007540 con 0x7fb7e806c240 2024-09-12T03:18:12.215 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:12.213+0000 7fb7e77fe640 1 -- 172.21.15.27:0/20297434 <== mon.1 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7e00078f0 con 0x7fb7e806c240 2024-09-12T03:18:12.215 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:12.213+0000 7fb7eeaa5640 1 -- 172.21.15.27:0/20297434 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb7e806c240 msgr2=0x7fb7e806e5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:12.215 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:12.213+0000 7fb7eeaa5640 1 --2- 172.21.15.27:0/20297434 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb7e806c240 0x7fb7e806e5b0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fb7e0009d90 tx=0x7fb7e002a5f0 comp rx=0 tx=0).stop 2024-09-12T03:18:12.215 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:12.213+0000 7fb7eeaa5640 1 -- 172.21.15.27:0/20297434 shutdown_connections 2024-09-12T03:18:12.215 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:12.213+0000 7fb7eeaa5640 1 --2- 172.21.15.27:0/20297434 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb7e806c240 0x7fb7e806e5b0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:12.215 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:12.213+0000 7fb7eeaa5640 1 -- 172.21.15.27:0/20297434 >> 172.21.15.27:0/20297434 conn(0x7fb7e806cc40 msgr2=0x7fb7e806d050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:18:12.215 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:12.213+0000 7fb7eeaa5640 1 -- 172.21.15.27:0/20297434 shutdown_connections 2024-09-12T03:18:12.215 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:12.213+0000 7fb7eeaa5640 1 -- 172.21.15.27:0/20297434 wait complete. 2024-09-12T03:18:12.216 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:12.214+0000 7fb7eeaa5640 1 Processor -- start 2024-09-12T03:18:12.216 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:12.214+0000 7fb7eeaa5640 1 -- start start 2024-09-12T03:18:12.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:12.215+0000 7fb7eeaa5640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb7e806c240 0x7fb7e81f36a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:12.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:12.215+0000 7fb7eeaa5640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb7e80ce6d0 0x7fb7e80aea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:12.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:12.215+0000 7fb7eeaa5640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb7e8160d80 0x7fb7e80b3420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:12.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:12.215+0000 7fb7eeaa5640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fb7e806ec70 con 0x7fb7e8160d80 2024-09-12T03:18:12.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:12.215+0000 7fb7eeaa5640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb7e806eaf0 con 0x7fb7e80ce6d0 2024-09-12T03:18:12.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:12.215+0000 7fb7eeaa5640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fb7e806edf0 con 0x7fb7e806c240 2024-09-12T03:18:12.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:12.215+0000 7fb7ed042640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb7e8160d80 0x7fb7e80b3420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:12.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:12.216+0000 7fb7ed042640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb7e8160d80 0x7fb7e80b3420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42702/0 (socket says 172.21.15.27:42702) 2024-09-12T03:18:12.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:12.216+0000 7fb7ed042640 1 -- 172.21.15.27:0/3534955180 learned_addr learned my addr 172.21.15.27:0/3534955180 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:18:12.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:12.216+0000 7fb7ec841640 1 --2- 172.21.15.27:0/3534955180 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb7e806c240 0x7fb7e81f36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:12.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:12.216+0000 7fb7e7fff640 1 --2- 172.21.15.27:0/3534955180 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb7e80ce6d0 0x7fb7e80aea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:12.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:12.216+0000 7fb7ed042640 1 -- 172.21.15.27:0/3534955180 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb7e806c240 msgr2=0x7fb7e81f36a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:12.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:12.216+0000 7fb7ed042640 1 --2- 172.21.15.27:0/3534955180 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb7e806c240 0x7fb7e81f36a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:12.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:12.216+0000 7fb7ed042640 1 -- 172.21.15.27:0/3534955180 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb7e80ce6d0 msgr2=0x7fb7e80aea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:12.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:12.216+0000 7fb7ed042640 1 --2- 172.21.15.27:0/3534955180 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb7e80ce6d0 0x7fb7e80aea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:12.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:12.216+0000 7fb7ed042640 1 -- 172.21.15.27:0/3534955180 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7e80af500 con 0x7fb7e8160d80 2024-09-12T03:18:12.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:12.216+0000 7fb7e7fff640 1 --2- 172.21.15.27:0/3534955180 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb7e80ce6d0 0x7fb7e80aea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:18:12.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:12.216+0000 7fb7ed042640 1 --2- 172.21.15.27:0/3534955180 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb7e8160d80 0x7fb7e80b3420 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fb7e002abe0 tx=0x7fb7e0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:12.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:12.216+0000 7fb7e57fa640 1 -- 172.21.15.27:0/3534955180 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7e0040070 con 0x7fb7e8160d80 2024-09-12T03:18:12.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:12.216+0000 7fb7e57fa640 1 -- 172.21.15.27:0/3534955180 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7e0031c40 con 0x7fb7e8160d80 2024-09-12T03:18:12.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:12.216+0000 7fb7eeaa5640 1 -- 172.21.15.27:0/3534955180 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7e80ac170 con 0x7fb7e8160d80 2024-09-12T03:18:12.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:12.216+0000 7fb7e57fa640 1 -- 172.21.15.27:0/3534955180 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7e0009450 con 0x7fb7e8160d80 2024-09-12T03:18:12.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:12.216+0000 7fb7eeaa5640 1 -- 172.21.15.27:0/3534955180 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7e80ac730 con 0x7fb7e8160d80 2024-09-12T03:18:12.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:12.217+0000 7fb7eeaa5640 1 -- 172.21.15.27:0/3534955180 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7e806f180 con 0x7fb7e8160d80 2024-09-12T03:18:12.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:12.218+0000 7fb7e57fa640 1 -- 172.21.15.27:0/3534955180 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 8) ==== 90702+0+0 (secure 0 0 0) 0x7fb7e0039040 con 0x7fb7e8160d80 2024-09-12T03:18:12.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:12.218+0000 7fb7e57fa640 1 --2- 172.21.15.27:0/3534955180 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fb7cc06c070 0x7fb7cc06e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:12.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:12.218+0000 7fb7e57fa640 1 -- 172.21.15.27:0/3534955180 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5709+0+0 (secure 0 0 0) 0x7fb7e00ac730 con 0x7fb7e8160d80 2024-09-12T03:18:12.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:12.219+0000 7fb7ec841640 1 --2- 172.21.15.27:0/3534955180 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fb7cc06c070 0x7fb7cc06e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:12.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:12.219+0000 7fb7ec841640 1 --2- 172.21.15.27:0/3534955180 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fb7cc06c070 0x7fb7cc06e530 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fb7dc000f60 tx=0x7fb7dc009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:12.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:12.223+0000 7fb7e57fa640 1 -- 172.21.15.27:0/3534955180 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fb7e007ad40 con 0x7fb7e8160d80 2024-09-12T03:18:12.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:12.435+0000 7fb7eeaa5640 1 -- 172.21.15.27:0/3534955180 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb7e80b0f50 con 0x7fb7e8160d80 2024-09-12T03:18:12.437 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:12.436+0000 7fb7e57fa640 1 -- 172.21.15.27:0/3534955180 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+5223 (secure 0 0 0) 0x7fb7e007a640 con 0x7fb7e8160d80 2024-09-12T03:18:12.437 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:18:12.437 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":23,"btime":"2024-09-12T03:17:47:237558+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4835,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6834/1457797481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":1457797481},{"type":"v1","addr":"172.21.15.27:6834","nonce":1457797481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5029,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6836/2842491281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":2842491281},{"type":"v1","addr":"172.21.15.45:6836","nonce":2842491281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5032,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/3339744340","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6835","nonce":3339744340},{"type":"v1","addr":"172.21.15.45:6837","nonce":3339744340}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-12T03:17:38.580473+0000","modified":"2024-09-12T03:17:40.557950+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5205},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5205":{"gid":5205,"name":"a","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.27:6835/2571101228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":2571101228},{"type":"v1","addr":"172.21.15.27:6835","nonce":2571101228}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5205,"qdb_cluster":[5205]},"id":3}]} 2024-09-12T03:18:12.437 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 23 2024-09-12T03:18:12.439 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:12.438+0000 7fb7eeaa5640 1 -- 172.21.15.27:0/3534955180 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fb7cc06c070 msgr2=0x7fb7cc06e530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:12.439 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:12.438+0000 7fb7eeaa5640 1 --2- 172.21.15.27:0/3534955180 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fb7cc06c070 0x7fb7cc06e530 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fb7dc000f60 tx=0x7fb7dc009040 comp rx=0 tx=0).stop 2024-09-12T03:18:12.439 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:12.438+0000 7fb7eeaa5640 1 -- 172.21.15.27:0/3534955180 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb7e8160d80 msgr2=0x7fb7e80b3420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:12.439 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:12.438+0000 7fb7eeaa5640 1 --2- 172.21.15.27:0/3534955180 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb7e8160d80 0x7fb7e80b3420 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fb7e002abe0 tx=0x7fb7e0004a40 comp rx=0 tx=0).stop 2024-09-12T03:18:12.439 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:12.438+0000 7fb7eeaa5640 1 -- 172.21.15.27:0/3534955180 shutdown_connections 2024-09-12T03:18:12.439 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:12.438+0000 7fb7eeaa5640 1 --2- 172.21.15.27:0/3534955180 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fb7cc06c070 0x7fb7cc06e530 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:12.439 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:12.438+0000 7fb7eeaa5640 1 --2- 172.21.15.27:0/3534955180 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb7e8160d80 0x7fb7e80b3420 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:12.439 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:12.438+0000 7fb7eeaa5640 1 --2- 172.21.15.27:0/3534955180 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb7e80ce6d0 0x7fb7e80aea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:12.440 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:12.438+0000 7fb7eeaa5640 1 --2- 172.21.15.27:0/3534955180 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb7e806c240 0x7fb7e81f36a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:12.440 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:12.438+0000 7fb7eeaa5640 1 -- 172.21.15.27:0/3534955180 >> 172.21.15.27:0/3534955180 conn(0x7fb7e806cc40 msgr2=0x7fb7e8145130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:18:12.440 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:12.438+0000 7fb7eeaa5640 1 -- 172.21.15.27:0/3534955180 shutdown_connections 2024-09-12T03:18:12.440 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:12.438+0000 7fb7eeaa5640 1 -- 172.21.15.27:0/3534955180 wait complete. 2024-09-12T03:18:12.450 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json session ls 2024-09-12T03:18:12.571 INFO:teuthology.orchestra.run.smithi027.stdout:[{"id":4955,"entity":{"name":{"type":"client","num":4955},"addr":{"type":"any","addr":"172.21.15.45:0","nonce":3904873653}},"state":"open","num_leases":1,"num_caps":7,"request_load_avg":11,"uptime":22.865414598000001,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":20.868457385011666,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.4955 172.21.15.45:0/3904873653","completed_requests":[{"tid":14,"created_ino":"0x0"},{"tid":15,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc","ceph_version":"ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)","entity_id":"x","hostname":"smithi045","pid":"36189","root":"/"}}] 2024-09-12T03:18:12.572 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc", "ceph_version": "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "x", "hostname": "smithi045", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "36189", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 14 }, { "created_ino": "0x100000001f9", "tid": 15 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.45:0", "nonce": 3904873653, "type": "any" }, "name": { "num": 4955, "type": "client" } }, "id": 4955, "inst": "client.4955 172.21.15.45:0/3904873653", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 1, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 11, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 20.868457385011666 }, "state": "open", "uptime": 22.865414598 } ] 2024-09-12T03:18:12.572 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2024-09-12T03:18:17.573 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-12T03:18:17.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:17.688+0000 7f6d291c4640 1 Processor -- start 2024-09-12T03:18:17.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:17.688+0000 7f6d291c4640 1 -- start start 2024-09-12T03:18:17.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:17.689+0000 7f6d291c4640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f6d24145c30 0x7f6d2415ca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:17.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:17.689+0000 7f6d291c4640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f6d240cda40 0x7f6d240b4040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:17.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:17.689+0000 7f6d291c4640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6d240b4d70 con 0x7f6d24146010 2024-09-12T03:18:17.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:17.689+0000 7f6d291c4640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f6d240b5050 con 0x7f6d240cda40 2024-09-12T03:18:17.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:17.689+0000 7f6d291c4640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f6d2406d140 con 0x7f6d24145c30 2024-09-12T03:18:17.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:17.689+0000 7f6d2359e640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f6d240cda40 0x7f6d240b4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:17.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:17.689+0000 7f6d2359e640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f6d240cda40 0x7f6d240b4040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42714/0 (socket says 172.21.15.27:42714) 2024-09-12T03:18:17.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:17.689+0000 7f6d2359e640 1 -- 172.21.15.27:0/1386717942 learned_addr learned my addr 172.21.15.27:0/1386717942 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:18:17.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:17.689+0000 7f6d22d9d640 1 --2- 172.21.15.27:0/1386717942 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f6d24145c30 0x7f6d2415ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:17.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:17.690+0000 7f6d2359e640 1 -- 172.21.15.27:0/1386717942 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f6d24145c30 msgr2=0x7f6d2415ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:18:17.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:17.690+0000 7f6d2359e640 1 --2- 172.21.15.27:0/1386717942 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f6d24145c30 0x7f6d2415ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:17.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:17.690+0000 7f6d2359e640 1 -- 172.21.15.27:0/1386717942 >> v1:172.21.15.45:6789/0 conn(0x7f6d24146010 legacy=0x7f6d240cd330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:17.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:17.690+0000 7f6d2359e640 1 -- 172.21.15.27:0/1386717942 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d2406d420 con 0x7f6d240cda40 2024-09-12T03:18:17.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:17.690+0000 7f6d2359e640 1 --2- 172.21.15.27:0/1386717942 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6d240cda40 0x7f6d240b4040 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f6d1c009e50 tx=0x7f6d1c02a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfcd106980b020a3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:17.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:17.691+0000 7f6d21d9b640 1 -- 172.21.15.27:0/1386717942 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d1c036070 con 0x7f6d240cda40 2024-09-12T03:18:17.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:17.691+0000 7f6d21d9b640 1 -- 172.21.15.27:0/1386717942 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d1c031600 con 0x7f6d240cda40 2024-09-12T03:18:17.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:17.691+0000 7f6d21d9b640 1 -- 172.21.15.27:0/1386717942 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d1c0319d0 con 0x7f6d240cda40 2024-09-12T03:18:17.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:17.691+0000 7f6d291c4640 1 -- 172.21.15.27:0/1386717942 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6d240cda40 msgr2=0x7f6d240b4040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:17.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:17.691+0000 7f6d291c4640 1 --2- 172.21.15.27:0/1386717942 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6d240cda40 0x7f6d240b4040 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f6d1c009e50 tx=0x7f6d1c02a680 comp rx=0 tx=0).stop 2024-09-12T03:18:17.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:17.691+0000 7f6d291c4640 1 -- 172.21.15.27:0/1386717942 shutdown_connections 2024-09-12T03:18:17.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:17.691+0000 7f6d291c4640 1 --2- 172.21.15.27:0/1386717942 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6d240cda40 0x7f6d240b4040 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:17.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:17.691+0000 7f6d291c4640 1 --2- 172.21.15.27:0/1386717942 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f6d24145c30 0x7f6d2415ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:17.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:17.691+0000 7f6d291c4640 1 -- 172.21.15.27:0/1386717942 >> 172.21.15.27:0/1386717942 conn(0x7f6d24062c60 msgr2=0x7f6d2406bb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:18:17.693 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:17.691+0000 7f6d291c4640 1 -- 172.21.15.27:0/1386717942 shutdown_connections 2024-09-12T03:18:17.693 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:17.691+0000 7f6d291c4640 1 -- 172.21.15.27:0/1386717942 wait complete. 2024-09-12T03:18:17.695 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:17.693+0000 7f6d291c4640 1 Processor -- start 2024-09-12T03:18:17.695 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:17.693+0000 7f6d291c4640 1 -- start start 2024-09-12T03:18:17.695 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:17.693+0000 7f6d291c4640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6d240cda40 0x7f6d241f3f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:17.695 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:17.694+0000 7f6d291c4640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f6d24145c30 0x7f6d2413f940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:17.695 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:17.694+0000 7f6d291c4640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6d24146010 0x7f6d240ad0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:17.695 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:17.694+0000 7f6d291c4640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f6d240b4700 con 0x7f6d240cda40 2024-09-12T03:18:17.695 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:17.694+0000 7f6d291c4640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f6d240b4580 con 0x7f6d24146010 2024-09-12T03:18:17.695 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:17.694+0000 7f6d291c4640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f6d240b4880 con 0x7f6d24145c30 2024-09-12T03:18:17.695 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:17.694+0000 7f6d22d9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6d240cda40 0x7f6d241f3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:17.695 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:17.694+0000 7f6d22d9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6d240cda40 0x7f6d241f3f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42726/0 (socket says 172.21.15.27:42726) 2024-09-12T03:18:17.695 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:17.694+0000 7f6d22d9d640 1 -- 172.21.15.27:0/3918967546 learned_addr learned my addr 172.21.15.27:0/3918967546 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:18:17.696 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:17.694+0000 7f6d2359e640 1 --2- 172.21.15.27:0/3918967546 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6d24146010 0x7f6d240ad0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:17.696 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:17.694+0000 7f6d2259c640 1 --2- 172.21.15.27:0/3918967546 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f6d24145c30 0x7f6d2413f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:17.696 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:17.694+0000 7f6d22d9d640 1 -- 172.21.15.27:0/3918967546 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f6d24145c30 msgr2=0x7f6d2413f940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:17.696 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:17.694+0000 7f6d22d9d640 1 --2- 172.21.15.27:0/3918967546 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f6d24145c30 0x7f6d2413f940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:17.696 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:17.694+0000 7f6d22d9d640 1 -- 172.21.15.27:0/3918967546 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6d24146010 msgr2=0x7f6d240ad0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:17.696 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:17.694+0000 7f6d22d9d640 1 --2- 172.21.15.27:0/3918967546 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6d24146010 0x7f6d240ad0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:17.696 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:17.694+0000 7f6d22d9d640 1 -- 172.21.15.27:0/3918967546 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d240ad610 con 0x7f6d240cda40 2024-09-12T03:18:17.696 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:17.694+0000 7f6d22d9d640 1 --2- 172.21.15.27:0/3918967546 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6d240cda40 0x7f6d241f3f30 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f6d1400d800 tx=0x7f6d1400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:17.696 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:17.694+0000 7f6d0b7fe640 1 -- 172.21.15.27:0/3918967546 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d1400b6c0 con 0x7f6d240cda40 2024-09-12T03:18:17.696 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:17.694+0000 7f6d0b7fe640 1 -- 172.21.15.27:0/3918967546 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d14019070 con 0x7f6d240cda40 2024-09-12T03:18:17.696 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:17.694+0000 7f6d291c4640 1 -- 172.21.15.27:0/3918967546 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d240afbd0 con 0x7f6d240cda40 2024-09-12T03:18:17.696 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:17.694+0000 7f6d0b7fe640 1 -- 172.21.15.27:0/3918967546 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d1400bac0 con 0x7f6d240cda40 2024-09-12T03:18:17.696 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:17.695+0000 7f6d291c4640 1 -- 172.21.15.27:0/3918967546 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d241e64b0 con 0x7f6d240cda40 2024-09-12T03:18:17.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:17.696+0000 7f6d291c4640 1 -- 172.21.15.27:0/3918967546 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d240b4040 con 0x7f6d240cda40 2024-09-12T03:18:17.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:17.696+0000 7f6d0b7fe640 1 -- 172.21.15.27:0/3918967546 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 8) ==== 90702+0+0 (secure 0 0 0) 0x7f6d1400bc60 con 0x7f6d240cda40 2024-09-12T03:18:17.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:17.697+0000 7f6d0b7fe640 1 --2- 172.21.15.27:0/3918967546 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f6d1006c070 0x7f6d1006e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:17.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:17.697+0000 7f6d0b7fe640 1 -- 172.21.15.27:0/3918967546 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5767+0+0 (secure 0 0 0) 0x7f6d1408b740 con 0x7f6d240cda40 2024-09-12T03:18:17.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:17.697+0000 7f6d2259c640 1 --2- 172.21.15.27:0/3918967546 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f6d1006c070 0x7f6d1006e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:17.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:17.698+0000 7f6d2259c640 1 --2- 172.21.15.27:0/3918967546 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f6d1006c070 0x7f6d1006e530 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f6d2406e920 tx=0x7f6d18009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:17.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:17.701+0000 7f6d0b7fe640 1 -- 172.21.15.27:0/3918967546 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f6d14059de0 con 0x7f6d240cda40 2024-09-12T03:18:17.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:17.913+0000 7f6d291c4640 1 -- 172.21.15.27:0/3918967546 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6d2415cbc0 con 0x7f6d240cda40 2024-09-12T03:18:17.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:17.913+0000 7f6d0b7fe640 1 -- 172.21.15.27:0/3918967546 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+5223 (secure 0 0 0) 0x7f6d140596e0 con 0x7f6d240cda40 2024-09-12T03:18:17.913 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:18:17.914 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":23,"btime":"2024-09-12T03:17:47:237558+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4835,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6834/1457797481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":1457797481},{"type":"v1","addr":"172.21.15.27:6834","nonce":1457797481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5029,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6836/2842491281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":2842491281},{"type":"v1","addr":"172.21.15.45:6836","nonce":2842491281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5032,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/3339744340","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6835","nonce":3339744340},{"type":"v1","addr":"172.21.15.45:6837","nonce":3339744340}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-12T03:17:38.580473+0000","modified":"2024-09-12T03:17:40.557950+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5205},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5205":{"gid":5205,"name":"a","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.27:6835/2571101228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":2571101228},{"type":"v1","addr":"172.21.15.27:6835","nonce":2571101228}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5205,"qdb_cluster":[5205]},"id":3}]} 2024-09-12T03:18:17.914 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 23 2024-09-12T03:18:17.915 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:17.915+0000 7f6d291c4640 1 -- 172.21.15.27:0/3918967546 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f6d1006c070 msgr2=0x7f6d1006e530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:17.915 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:17.915+0000 7f6d291c4640 1 --2- 172.21.15.27:0/3918967546 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f6d1006c070 0x7f6d1006e530 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f6d2406e920 tx=0x7f6d18009040 comp rx=0 tx=0).stop 2024-09-12T03:18:17.915 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:17.915+0000 7f6d291c4640 1 -- 172.21.15.27:0/3918967546 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6d240cda40 msgr2=0x7f6d241f3f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:17.915 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:17.915+0000 7f6d291c4640 1 --2- 172.21.15.27:0/3918967546 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6d240cda40 0x7f6d241f3f30 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f6d1400d800 tx=0x7f6d1400dcc0 comp rx=0 tx=0).stop 2024-09-12T03:18:17.915 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:17.915+0000 7f6d291c4640 1 -- 172.21.15.27:0/3918967546 shutdown_connections 2024-09-12T03:18:17.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:17.915+0000 7f6d291c4640 1 --2- 172.21.15.27:0/3918967546 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f6d1006c070 0x7f6d1006e530 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:17.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:17.915+0000 7f6d291c4640 1 --2- 172.21.15.27:0/3918967546 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6d24146010 0x7f6d240ad0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:17.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:17.915+0000 7f6d291c4640 1 --2- 172.21.15.27:0/3918967546 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f6d24145c30 0x7f6d2413f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:17.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:17.915+0000 7f6d291c4640 1 --2- 172.21.15.27:0/3918967546 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6d240cda40 0x7f6d241f3f30 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:17.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:17.915+0000 7f6d291c4640 1 -- 172.21.15.27:0/3918967546 >> 172.21.15.27:0/3918967546 conn(0x7f6d24062c60 msgr2=0x7f6d2406bb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:18:17.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:17.915+0000 7f6d291c4640 1 -- 172.21.15.27:0/3918967546 shutdown_connections 2024-09-12T03:18:17.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:17.915+0000 7f6d291c4640 1 -- 172.21.15.27:0/3918967546 wait complete. 2024-09-12T03:18:17.927 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json session ls 2024-09-12T03:18:18.047 INFO:teuthology.orchestra.run.smithi027.stdout:[{"id":4955,"entity":{"name":{"type":"client","num":4955},"addr":{"type":"any","addr":"172.21.15.45:0","nonce":3904873653}},"state":"open","num_leases":1,"num_caps":7,"request_load_avg":10,"uptime":28.342274786000001,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":20.606046846021236,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.4955 172.21.15.45:0/3904873653","completed_requests":[{"tid":14,"created_ino":"0x0"},{"tid":15,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc","ceph_version":"ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)","entity_id":"x","hostname":"smithi045","pid":"36189","root":"/"}}] 2024-09-12T03:18:18.048 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc", "ceph_version": "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "x", "hostname": "smithi045", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "36189", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 14 }, { "created_ino": "0x100000001f9", "tid": 15 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.45:0", "nonce": 3904873653, "type": "any" }, "name": { "num": 4955, "type": "client" } }, "id": 4955, "inst": "client.4955 172.21.15.45:0/3904873653", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 1, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 10, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 20.606046846021236 }, "state": "open", "uptime": 28.342274786 } ] 2024-09-12T03:18:18.048 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2024-09-12T03:18:23.049 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-12T03:18:23.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:23.166+0000 7f273e1c4640 1 Processor -- start 2024-09-12T03:18:23.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:23.166+0000 7f273e1c4640 1 -- start start 2024-09-12T03:18:23.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:23.167+0000 7f273e1c4640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f27380ce6d0 0x7f27380ceab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:23.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:23.167+0000 7f273e1c4640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f273806c240 0x7f273806e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:23.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:23.167+0000 7f273e1c4640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f27381463d0 con 0x7f2738160d80 2024-09-12T03:18:23.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:23.167+0000 7f273e1c4640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f27380ceff0 con 0x7f273806c240 2024-09-12T03:18:23.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:23.167+0000 7f273e1c4640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f273806c7f0 con 0x7f27380ce6d0 2024-09-12T03:18:23.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:23.167+0000 7f2737fff640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f273806c240 0x7f273806e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:23.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:23.167+0000 7f27377fe640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f27380ce6d0 0x7f27380ceab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:23.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:23.167+0000 7f2736ffd640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f2738160d80 0x7f273806bb30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.27:50316/0 (socket says 172.21.15.27:50316) 2024-09-12T03:18:23.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:23.167+0000 7f2736ffd640 1 -- 172.21.15.27:0/4244746137 learned_addr learned my addr 172.21.15.27:0/4244746137 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:18:23.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:23.168+0000 7f2737fff640 1 -- 172.21.15.27:0/4244746137 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f27380ce6d0 msgr2=0x7f27380ceab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:18:23.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:23.168+0000 7f2737fff640 1 --2- 172.21.15.27:0/4244746137 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f27380ce6d0 0x7f27380ceab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:23.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:23.168+0000 7f2737fff640 1 -- 172.21.15.27:0/4244746137 >> v1:172.21.15.45:6789/0 conn(0x7f2738160d80 legacy=0x7f273806bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:23.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:23.168+0000 7f2737fff640 1 -- 172.21.15.27:0/4244746137 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27381602a0 con 0x7f273806c240 2024-09-12T03:18:23.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:23.168+0000 7f27367fc640 1 -- 172.21.15.27:0/4244746137 <== mon.1 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 99449023 0 0) 0x7f27381463d0 con 0x7f2738160d80 2024-09-12T03:18:23.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:23.168+0000 7f27377fe640 1 --2- 172.21.15.27:0/4244746137 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f27380ce6d0 0x7f27380ceab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:18:23.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:23.168+0000 7f2737fff640 1 --2- 172.21.15.27:0/4244746137 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f273806c240 0x7f273806e5b0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f2730009d90 tx=0x7f273002a5d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=31dd7bcb405fa1c6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:23.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:23.169+0000 7f27367fc640 1 -- 172.21.15.27:0/4244746137 <== mon.1 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2730031600 con 0x7f273806c240 2024-09-12T03:18:23.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:23.169+0000 7f27367fc640 1 -- 172.21.15.27:0/4244746137 <== mon.1 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f27300317a0 con 0x7f273806c240 2024-09-12T03:18:23.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:23.169+0000 7f27367fc640 1 -- 172.21.15.27:0/4244746137 <== mon.1 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2730031b70 con 0x7f273806c240 2024-09-12T03:18:23.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:23.169+0000 7f273e1c4640 1 -- 172.21.15.27:0/4244746137 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f273806c240 msgr2=0x7f273806e5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:23.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:23.169+0000 7f273e1c4640 1 --2- 172.21.15.27:0/4244746137 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f273806c240 0x7f273806e5b0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f2730009d90 tx=0x7f273002a5d0 comp rx=0 tx=0).stop 2024-09-12T03:18:23.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:23.169+0000 7f273e1c4640 1 -- 172.21.15.27:0/4244746137 shutdown_connections 2024-09-12T03:18:23.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:23.169+0000 7f273e1c4640 1 --2- 172.21.15.27:0/4244746137 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f273806c240 0x7f273806e5b0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:23.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:23.169+0000 7f273e1c4640 1 --2- 172.21.15.27:0/4244746137 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f27380ce6d0 0x7f27380ceab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:23.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:23.169+0000 7f273e1c4640 1 -- 172.21.15.27:0/4244746137 >> 172.21.15.27:0/4244746137 conn(0x7f273806cc40 msgr2=0x7f273806d050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:18:23.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:23.169+0000 7f273e1c4640 1 -- 172.21.15.27:0/4244746137 shutdown_connections 2024-09-12T03:18:23.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:23.169+0000 7f273e1c4640 1 -- 172.21.15.27:0/4244746137 wait complete. 2024-09-12T03:18:23.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:23.170+0000 7f273e1c4640 1 Processor -- start 2024-09-12T03:18:23.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:23.171+0000 7f273e1c4640 1 -- start start 2024-09-12T03:18:23.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:23.171+0000 7f273e1c4640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f273806c240 0x7f27380b3950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:23.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:23.171+0000 7f273e1c4640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f27380ce6d0 0x7f27381f96e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:23.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:23.172+0000 7f273e1c4640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2738160d80 0x7f27380aa4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:23.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:23.172+0000 7f273e1c4640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f273806eaf0 con 0x7f273806c240 2024-09-12T03:18:23.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:23.172+0000 7f273e1c4640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f2738160860 con 0x7f27380ce6d0 2024-09-12T03:18:23.176 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:23.172+0000 7f273e1c4640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f273806ec70 con 0x7f2738160d80 2024-09-12T03:18:23.177 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:23.172+0000 7f27377fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f273806c240 0x7f27380b3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:23.177 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:23.172+0000 7f27377fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f273806c240 0x7f27380b3950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37024/0 (socket says 172.21.15.27:37024) 2024-09-12T03:18:23.177 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:23.172+0000 7f27377fe640 1 -- 172.21.15.27:0/991249181 learned_addr learned my addr 172.21.15.27:0/991249181 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:18:23.177 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:23.172+0000 7f2736ffd640 1 --2- 172.21.15.27:0/991249181 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f27380ce6d0 0x7f27381f96e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:23.177 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:23.172+0000 7f2737fff640 1 --2- 172.21.15.27:0/991249181 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2738160d80 0x7f27380aa4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:23.177 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:23.172+0000 7f27377fe640 1 -- 172.21.15.27:0/991249181 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2738160d80 msgr2=0x7f27380aa4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:23.177 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:23.172+0000 7f27377fe640 1 --2- 172.21.15.27:0/991249181 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2738160d80 0x7f27380aa4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:23.177 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:23.172+0000 7f27377fe640 1 -- 172.21.15.27:0/991249181 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f27380ce6d0 msgr2=0x7f27381f96e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:23.177 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:23.172+0000 7f27377fe640 1 --2- 172.21.15.27:0/991249181 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f27380ce6d0 0x7f27381f96e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:23.177 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:23.172+0000 7f27377fe640 1 -- 172.21.15.27:0/991249181 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27381f9c20 con 0x7f273806c240 2024-09-12T03:18:23.177 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:23.172+0000 7f27377fe640 1 --2- 172.21.15.27:0/991249181 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f273806c240 0x7f27380b3950 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f272800ebe0 tx=0x7f272800c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:23.177 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:23.172+0000 7f271ffff640 1 -- 172.21.15.27:0/991249181 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2728017070 con 0x7f273806c240 2024-09-12T03:18:23.178 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:23.172+0000 7f271ffff640 1 -- 172.21.15.27:0/991249181 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f272800cbd0 con 0x7f273806c240 2024-09-12T03:18:23.178 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:23.172+0000 7f273e1c4640 1 -- 172.21.15.27:0/991249181 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f273815c760 con 0x7f273806c240 2024-09-12T03:18:23.178 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:23.172+0000 7f271ffff640 1 -- 172.21.15.27:0/991249181 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2728013470 con 0x7f273806c240 2024-09-12T03:18:23.178 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:23.173+0000 7f273e1c4640 1 -- 172.21.15.27:0/991249181 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27380b47a0 con 0x7f273806c240 2024-09-12T03:18:23.178 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:23.174+0000 7f273e1c4640 1 -- 172.21.15.27:0/991249181 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27380b0f50 con 0x7f273806c240 2024-09-12T03:18:23.179 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:23.174+0000 7f271ffff640 1 -- 172.21.15.27:0/991249181 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 8) ==== 90702+0+0 (secure 0 0 0) 0x7f272801a070 con 0x7f273806c240 2024-09-12T03:18:23.179 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:23.175+0000 7f271ffff640 1 --2- 172.21.15.27:0/991249181 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f272406c070 0x7f272406e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:23.179 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:23.175+0000 7f271ffff640 1 -- 172.21.15.27:0/991249181 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5767+0+0 (secure 0 0 0) 0x7f272808b080 con 0x7f273806c240 2024-09-12T03:18:23.179 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:23.175+0000 7f2736ffd640 1 --2- 172.21.15.27:0/991249181 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f272406c070 0x7f272406e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:23.179 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:23.175+0000 7f2736ffd640 1 --2- 172.21.15.27:0/991249181 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f272406c070 0x7f272406e530 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f27381fd310 tx=0x7f272c007de0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:23.179 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:23.179+0000 7f271ffff640 1 -- 172.21.15.27:0/991249181 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f27381f9c20 con 0x7f273806c240 2024-09-12T03:18:23.390 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:23.390+0000 7f273e1c4640 1 -- 172.21.15.27:0/991249181 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f273815d700 con 0x7f273806c240 2024-09-12T03:18:23.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:23.391+0000 7f271ffff640 1 -- 172.21.15.27:0/991249181 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+5223 (secure 0 0 0) 0x7f27280590d0 con 0x7f273806c240 2024-09-12T03:18:23.391 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:18:23.391 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":23,"btime":"2024-09-12T03:17:47:237558+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4835,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6834/1457797481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":1457797481},{"type":"v1","addr":"172.21.15.27:6834","nonce":1457797481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5029,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6836/2842491281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":2842491281},{"type":"v1","addr":"172.21.15.45:6836","nonce":2842491281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5032,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/3339744340","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6835","nonce":3339744340},{"type":"v1","addr":"172.21.15.45:6837","nonce":3339744340}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-12T03:17:38.580473+0000","modified":"2024-09-12T03:17:40.557950+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5205},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5205":{"gid":5205,"name":"a","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.27:6835/2571101228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":2571101228},{"type":"v1","addr":"172.21.15.27:6835","nonce":2571101228}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5205,"qdb_cluster":[5205]},"id":3}]} 2024-09-12T03:18:23.392 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 23 2024-09-12T03:18:23.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:23.393+0000 7f273e1c4640 1 -- 172.21.15.27:0/991249181 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f272406c070 msgr2=0x7f272406e530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:23.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:23.393+0000 7f273e1c4640 1 --2- 172.21.15.27:0/991249181 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f272406c070 0x7f272406e530 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f27381fd310 tx=0x7f272c007de0 comp rx=0 tx=0).stop 2024-09-12T03:18:23.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:23.393+0000 7f273e1c4640 1 -- 172.21.15.27:0/991249181 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f273806c240 msgr2=0x7f27380b3950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:23.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:23.393+0000 7f273e1c4640 1 --2- 172.21.15.27:0/991249181 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f273806c240 0x7f27380b3950 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f272800ebe0 tx=0x7f272800c520 comp rx=0 tx=0).stop 2024-09-12T03:18:23.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:23.393+0000 7f273e1c4640 1 -- 172.21.15.27:0/991249181 shutdown_connections 2024-09-12T03:18:23.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:23.393+0000 7f273e1c4640 1 --2- 172.21.15.27:0/991249181 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f272406c070 0x7f272406e530 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:23.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:23.393+0000 7f273e1c4640 1 --2- 172.21.15.27:0/991249181 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2738160d80 0x7f27380aa4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:23.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:23.393+0000 7f273e1c4640 1 --2- 172.21.15.27:0/991249181 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f27380ce6d0 0x7f27381f96e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:23.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:23.393+0000 7f273e1c4640 1 --2- 172.21.15.27:0/991249181 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f273806c240 0x7f27380b3950 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:23.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:23.393+0000 7f273e1c4640 1 -- 172.21.15.27:0/991249181 >> 172.21.15.27:0/991249181 conn(0x7f273806cc40 msgr2=0x7f2738145190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:18:23.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:23.393+0000 7f273e1c4640 1 -- 172.21.15.27:0/991249181 shutdown_connections 2024-09-12T03:18:23.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:23.393+0000 7f273e1c4640 1 -- 172.21.15.27:0/991249181 wait complete. 2024-09-12T03:18:23.405 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json session ls 2024-09-12T03:18:23.525 INFO:teuthology.orchestra.run.smithi027.stdout:[{"id":4955,"entity":{"name":{"type":"client","num":4955},"addr":{"type":"any","addr":"172.21.15.45:0","nonce":3904873653}},"state":"open","num_leases":0,"num_caps":7,"request_load_avg":10,"uptime":33.820136067,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":20.346888927149013,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.4955 172.21.15.45:0/3904873653","completed_requests":[{"tid":14,"created_ino":"0x0"},{"tid":15,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc","ceph_version":"ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)","entity_id":"x","hostname":"smithi045","pid":"36189","root":"/"}}] 2024-09-12T03:18:23.526 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc", "ceph_version": "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "x", "hostname": "smithi045", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "36189", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 14 }, { "created_ino": "0x100000001f9", "tid": 15 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.45:0", "nonce": 3904873653, "type": "any" }, "name": { "num": 4955, "type": "client" } }, "id": 4955, "inst": "client.4955 172.21.15.45:0/3904873653", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 0, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 10, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 20.346888927149013 }, "state": "open", "uptime": 33.820136067 } ] 2024-09-12T03:18:23.526 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2024-09-12T03:18:28.527 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-12T03:18:28.643 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:28.643+0000 7fd346277640 1 Processor -- start 2024-09-12T03:18:28.643 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:28.643+0000 7fd346277640 1 -- start start 2024-09-12T03:18:28.644 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:28.643+0000 7fd346277640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fd34015eb90 0x7fd3400d2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:28.644 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:28.643+0000 7fd346277640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd3400d3760 con 0x7fd3400d1720 2024-09-12T03:18:28.644 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:28.643+0000 7fd346277640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd3400d3a40 con 0x7fd3400d5200 2024-09-12T03:18:28.644 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:28.643+0000 7fd346277640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7fd3400b7d20 con 0x7fd34015eb90 2024-09-12T03:18:28.644 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:28.644+0000 7fd33ffff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fd34015eb90 0x7fd3400d2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:28.644 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:28.644+0000 7fd344814640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fd3400d1720 0x7fd3400b8190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:41106/0 (socket says 172.21.15.27:41106) 2024-09-12T03:18:28.644 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:28.644+0000 7fd344814640 1 -- 172.21.15.27:0/484650203 learned_addr learned my addr 172.21.15.27:0/484650203 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:18:28.644 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:28.644+0000 7fd33ffff640 1 -- 172.21.15.27:0/484650203 >> v1:172.21.15.45:6789/0 conn(0x7fd3400d5200 legacy=0x7fd3400d1010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:18:28.644 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:28.644+0000 7fd33ffff640 1 -- 172.21.15.27:0/484650203 >> v1:172.21.15.27:6789/0 conn(0x7fd3400d1720 legacy=0x7fd3400b8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:28.645 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:28.644+0000 7fd33ffff640 1 -- 172.21.15.27:0/484650203 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3400d3d20 con 0x7fd34015eb90 2024-09-12T03:18:28.645 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:28.644+0000 7fd33effd640 1 -- 172.21.15.27:0/484650203 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3645452757 0 0) 0x7fd3400d3760 con 0x7fd3400d1720 2024-09-12T03:18:28.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:28.645+0000 7fd33ffff640 1 --2- 172.21.15.27:0/484650203 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd34015eb90 0x7fd3400d2df0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fd3300097a0 tx=0x7fd33002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c184834c3f812ef5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:28.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:28.645+0000 7fd33effd640 1 -- 172.21.15.27:0/484650203 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd330037070 con 0x7fd34015eb90 2024-09-12T03:18:28.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:28.645+0000 7fd33effd640 1 -- 172.21.15.27:0/484650203 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd330032440 con 0x7fd34015eb90 2024-09-12T03:18:28.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:28.645+0000 7fd33effd640 1 -- 172.21.15.27:0/484650203 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd330032810 con 0x7fd34015eb90 2024-09-12T03:18:28.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:28.645+0000 7fd346277640 1 -- 172.21.15.27:0/484650203 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd34015eb90 msgr2=0x7fd3400d2df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:28.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:28.645+0000 7fd346277640 1 --2- 172.21.15.27:0/484650203 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd34015eb90 0x7fd3400d2df0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fd3300097a0 tx=0x7fd33002a1e0 comp rx=0 tx=0).stop 2024-09-12T03:18:28.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:28.645+0000 7fd346277640 1 -- 172.21.15.27:0/484650203 shutdown_connections 2024-09-12T03:18:28.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:28.646+0000 7fd346277640 1 --2- 172.21.15.27:0/484650203 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd34015eb90 0x7fd3400d2df0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:28.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:28.646+0000 7fd346277640 1 -- 172.21.15.27:0/484650203 >> 172.21.15.27:0/484650203 conn(0x7fd34006dce0 msgr2=0x7fd34006e0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:18:28.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:28.646+0000 7fd346277640 1 -- 172.21.15.27:0/484650203 shutdown_connections 2024-09-12T03:18:28.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:28.646+0000 7fd346277640 1 -- 172.21.15.27:0/484650203 wait complete. 2024-09-12T03:18:28.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:28.647+0000 7fd346277640 1 Processor -- start 2024-09-12T03:18:28.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:28.647+0000 7fd346277640 1 -- start start 2024-09-12T03:18:28.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:28.648+0000 7fd346277640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd3400d1720 0x7fd3400cc570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:28.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:28.648+0000 7fd346277640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd3400d5200 0x7fd34015b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:28.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:28.648+0000 7fd346277640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fd34015eb90 0x7fd340154e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:28.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:28.648+0000 7fd346277640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fd3400d3330 con 0x7fd3400d5200 2024-09-12T03:18:28.650 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:28.648+0000 7fd346277640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fd3400d42e0 con 0x7fd3400d1720 2024-09-12T03:18:28.650 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:28.648+0000 7fd346277640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fd3400d34b0 con 0x7fd34015eb90 2024-09-12T03:18:28.650 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:28.648+0000 7fd33f7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd3400d5200 0x7fd34015b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:28.650 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:28.648+0000 7fd33f7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd3400d5200 0x7fd34015b310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37034/0 (socket says 172.21.15.27:37034) 2024-09-12T03:18:28.650 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:28.648+0000 7fd33f7fe640 1 -- 172.21.15.27:0/549908998 learned_addr learned my addr 172.21.15.27:0/549908998 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:18:28.650 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:28.648+0000 7fd33ffff640 1 --2- 172.21.15.27:0/549908998 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd3400d1720 0x7fd3400cc570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:28.650 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:28.648+0000 7fd344814640 1 --2- 172.21.15.27:0/549908998 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fd34015eb90 0x7fd340154e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:28.650 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:28.648+0000 7fd33f7fe640 1 -- 172.21.15.27:0/549908998 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fd34015eb90 msgr2=0x7fd340154e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:28.650 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:28.648+0000 7fd33f7fe640 1 --2- 172.21.15.27:0/549908998 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fd34015eb90 0x7fd340154e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:28.650 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:28.648+0000 7fd33f7fe640 1 -- 172.21.15.27:0/549908998 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd3400d1720 msgr2=0x7fd3400cc570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:28.650 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:28.648+0000 7fd33f7fe640 1 --2- 172.21.15.27:0/549908998 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd3400d1720 0x7fd3400cc570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:28.650 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:28.648+0000 7fd33f7fe640 1 -- 172.21.15.27:0/549908998 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3400ccab0 con 0x7fd3400d5200 2024-09-12T03:18:28.650 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:28.648+0000 7fd33f7fe640 1 --2- 172.21.15.27:0/549908998 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd3400d5200 0x7fd34015b310 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fd33400c430 tx=0x7fd33400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:28.651 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:28.649+0000 7fd33cff9640 1 -- 172.21.15.27:0/549908998 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd334017070 con 0x7fd3400d5200 2024-09-12T03:18:28.651 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:28.649+0000 7fd33cff9640 1 -- 172.21.15.27:0/549908998 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3340035f0 con 0x7fd3400d5200 2024-09-12T03:18:28.651 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:28.649+0000 7fd346277640 1 -- 172.21.15.27:0/549908998 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3400ce490 con 0x7fd3400d5200 2024-09-12T03:18:28.651 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:28.649+0000 7fd33cff9640 1 -- 172.21.15.27:0/549908998 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3340039e0 con 0x7fd3400d5200 2024-09-12T03:18:28.651 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:28.649+0000 7fd346277640 1 -- 172.21.15.27:0/549908998 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3400cea50 con 0x7fd3400d5200 2024-09-12T03:18:28.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:28.650+0000 7fd346277640 1 -- 172.21.15.27:0/549908998 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3400693d0 con 0x7fd3400d5200 2024-09-12T03:18:28.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:28.650+0000 7fd33cff9640 1 -- 172.21.15.27:0/549908998 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 8) ==== 90702+0+0 (secure 0 0 0) 0x7fd33401a070 con 0x7fd3400d5200 2024-09-12T03:18:28.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:28.651+0000 7fd33cff9640 1 --2- 172.21.15.27:0/549908998 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fd32c06c070 0x7fd32c06e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:28.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:28.651+0000 7fd33cff9640 1 -- 172.21.15.27:0/549908998 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5767+0+0 (secure 0 0 0) 0x7fd33408a560 con 0x7fd3400d5200 2024-09-12T03:18:28.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:28.651+0000 7fd33ffff640 1 --2- 172.21.15.27:0/549908998 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fd32c06c070 0x7fd32c06e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:28.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:28.651+0000 7fd33ffff640 1 --2- 172.21.15.27:0/549908998 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fd32c06c070 0x7fd32c06e530 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fd330034000 tx=0x7fd33002e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:28.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:28.655+0000 7fd33cff9640 1 -- 172.21.15.27:0/549908998 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fd3340585b0 con 0x7fd3400d5200 2024-09-12T03:18:28.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:28.867+0000 7fd346277640 1 -- 172.21.15.27:0/549908998 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd3401620e0 con 0x7fd3400d5200 2024-09-12T03:18:28.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:28.868+0000 7fd33cff9640 1 -- 172.21.15.27:0/549908998 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+5223 (secure 0 0 0) 0x7fd33405c8b0 con 0x7fd3400d5200 2024-09-12T03:18:28.868 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:18:28.868 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":23,"btime":"2024-09-12T03:17:47:237558+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4835,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6834/1457797481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":1457797481},{"type":"v1","addr":"172.21.15.27:6834","nonce":1457797481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5029,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6836/2842491281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":2842491281},{"type":"v1","addr":"172.21.15.45:6836","nonce":2842491281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5032,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/3339744340","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6835","nonce":3339744340},{"type":"v1","addr":"172.21.15.45:6837","nonce":3339744340}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-12T03:17:38.580473+0000","modified":"2024-09-12T03:17:40.557950+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5205},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5205":{"gid":5205,"name":"a","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.27:6835/2571101228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":2571101228},{"type":"v1","addr":"172.21.15.27:6835","nonce":2571101228}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5205,"qdb_cluster":[5205]},"id":3}]} 2024-09-12T03:18:28.869 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 23 2024-09-12T03:18:28.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:28.870+0000 7fd346277640 1 -- 172.21.15.27:0/549908998 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fd32c06c070 msgr2=0x7fd32c06e530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:28.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:28.870+0000 7fd346277640 1 --2- 172.21.15.27:0/549908998 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fd32c06c070 0x7fd32c06e530 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fd330034000 tx=0x7fd33002e000 comp rx=0 tx=0).stop 2024-09-12T03:18:28.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:28.870+0000 7fd346277640 1 -- 172.21.15.27:0/549908998 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd3400d5200 msgr2=0x7fd34015b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:28.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:28.870+0000 7fd346277640 1 --2- 172.21.15.27:0/549908998 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd3400d5200 0x7fd34015b310 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fd33400c430 tx=0x7fd33400c8f0 comp rx=0 tx=0).stop 2024-09-12T03:18:28.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:28.870+0000 7fd346277640 1 -- 172.21.15.27:0/549908998 shutdown_connections 2024-09-12T03:18:28.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:28.870+0000 7fd346277640 1 --2- 172.21.15.27:0/549908998 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fd32c06c070 0x7fd32c06e530 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:28.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:28.870+0000 7fd346277640 1 --2- 172.21.15.27:0/549908998 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fd34015eb90 0x7fd340154e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:28.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:28.870+0000 7fd346277640 1 --2- 172.21.15.27:0/549908998 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd3400d5200 0x7fd34015b310 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:28.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:28.870+0000 7fd346277640 1 --2- 172.21.15.27:0/549908998 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd3400d1720 0x7fd3400cc570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:28.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:28.870+0000 7fd346277640 1 -- 172.21.15.27:0/549908998 >> 172.21.15.27:0/549908998 conn(0x7fd34006dce0 msgr2=0x7fd3400b8190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:18:28.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:28.870+0000 7fd346277640 1 -- 172.21.15.27:0/549908998 shutdown_connections 2024-09-12T03:18:28.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:28.870+0000 7fd346277640 1 -- 172.21.15.27:0/549908998 wait complete. 2024-09-12T03:18:28.882 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json session ls 2024-09-12T03:18:29.003 INFO:teuthology.orchestra.run.smithi027.stdout:[{"id":4955,"entity":{"name":{"type":"client","num":4955},"addr":{"type":"any","addr":"172.21.15.45:0","nonce":3904873653}},"state":"open","num_leases":0,"num_caps":7,"request_load_avg":9,"uptime":39.298997321999998,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":20.090943964151972,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.4955 172.21.15.45:0/3904873653","completed_requests":[{"tid":14,"created_ino":"0x0"},{"tid":15,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc","ceph_version":"ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)","entity_id":"x","hostname":"smithi045","pid":"36189","root":"/"}}] 2024-09-12T03:18:29.004 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc", "ceph_version": "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "x", "hostname": "smithi045", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "36189", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 14 }, { "created_ino": "0x100000001f9", "tid": 15 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.45:0", "nonce": 3904873653, "type": "any" }, "name": { "num": 4955, "type": "client" } }, "id": 4955, "inst": "client.4955 172.21.15.45:0/3904873653", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 0, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 9, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 20.090943964151972 }, "state": "open", "uptime": 39.298997322 } ] 2024-09-12T03:18:29.004 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2024-09-12T03:18:30.661 DEBUG:teuthology.orchestra.run.smithi027:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-12T03:18:30.664 DEBUG:teuthology.orchestra.run.smithi045:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-12T03:18:34.006 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-12T03:18:34.122 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:34.122+0000 7f4b2a72c640 1 Processor -- start 2024-09-12T03:18:34.122 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:34.122+0000 7f4b2a72c640 1 -- start start 2024-09-12T03:18:34.124 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:34.123+0000 7f4b2a72c640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f4b2406f360 0x7f4b240cef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:34.124 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:34.123+0000 7f4b2a72c640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f4b240cf490 0x7f4b240d2780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:34.124 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:34.123+0000 7f4b2a72c640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4b2406cc40 con 0x7f4b240b1ca0 2024-09-12T03:18:34.124 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:34.123+0000 7f4b2a72c640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f4b240cf900 con 0x7f4b2406f360 2024-09-12T03:18:34.124 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:34.123+0000 7f4b2a72c640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f4b240d22f0 con 0x7f4b240cf490 2024-09-12T03:18:34.124 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:34.123+0000 7f4b237fe640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f4b2406f360 0x7f4b240cef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:34.124 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:34.124+0000 7f4b237fe640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f4b2406f360 0x7f4b240cef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44886/0 (socket says 172.21.15.27:44886) 2024-09-12T03:18:34.124 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:34.124+0000 7f4b237fe640 1 -- 172.21.15.27:0/1456821395 learned_addr learned my addr 172.21.15.27:0/1456821395 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:18:34.124 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:34.124+0000 7f4b28cc9640 1 --2- 172.21.15.27:0/1456821395 >> v2:172.21.15.45:3300/0 conn(0x7f4b240cf490 0x7f4b240d2780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:34.124 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:34.124+0000 7f4b237fe640 1 -- 172.21.15.27:0/1456821395 >> v2:172.21.15.45:3300/0 conn(0x7f4b240cf490 msgr2=0x7f4b240d2780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:18:34.124 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:34.124+0000 7f4b237fe640 1 --2- 172.21.15.27:0/1456821395 >> v2:172.21.15.45:3300/0 conn(0x7f4b240cf490 0x7f4b240d2780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:34.124 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:34.124+0000 7f4b237fe640 1 -- 172.21.15.27:0/1456821395 >> v1:172.21.15.45:6789/0 conn(0x7f4b240b1ca0 legacy=0x7f4b2415f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:34.125 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:34.124+0000 7f4b237fe640 1 -- 172.21.15.27:0/1456821395 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b240d1ce0 con 0x7f4b2406f360 2024-09-12T03:18:34.126 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:34.125+0000 7f4b237fe640 1 --2- 172.21.15.27:0/1456821395 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4b2406f360 0x7f4b240cef50 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f4b24069cf0 tx=0x7f4b1802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c35d2c8926de2d2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:34.126 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:34.125+0000 7f4b22ffd640 1 -- 172.21.15.27:0/1456821395 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b18037070 con 0x7f4b2406f360 2024-09-12T03:18:34.126 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:34.125+0000 7f4b22ffd640 1 -- 172.21.15.27:0/1456821395 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b18032440 con 0x7f4b2406f360 2024-09-12T03:18:34.126 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:34.125+0000 7f4b22ffd640 1 -- 172.21.15.27:0/1456821395 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b18032810 con 0x7f4b2406f360 2024-09-12T03:18:34.126 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:34.125+0000 7f4b2a72c640 1 -- 172.21.15.27:0/1456821395 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4b2406f360 msgr2=0x7f4b240cef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:34.126 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:34.125+0000 7f4b2a72c640 1 --2- 172.21.15.27:0/1456821395 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4b2406f360 0x7f4b240cef50 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f4b24069cf0 tx=0x7f4b1802a270 comp rx=0 tx=0).stop 2024-09-12T03:18:34.127 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:34.125+0000 7f4b2a72c640 1 -- 172.21.15.27:0/1456821395 shutdown_connections 2024-09-12T03:18:34.127 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:34.125+0000 7f4b2a72c640 1 --2- 172.21.15.27:0/1456821395 >> v2:172.21.15.45:3300/0 conn(0x7f4b240cf490 0x7f4b240d2780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:34.127 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:34.125+0000 7f4b2a72c640 1 --2- 172.21.15.27:0/1456821395 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4b2406f360 0x7f4b240cef50 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:34.127 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:34.125+0000 7f4b2a72c640 1 -- 172.21.15.27:0/1456821395 >> 172.21.15.27:0/1456821395 conn(0x7f4b24073800 msgr2=0x7f4b2409ba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:18:34.127 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:34.125+0000 7f4b2a72c640 1 -- 172.21.15.27:0/1456821395 shutdown_connections 2024-09-12T03:18:34.127 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:34.125+0000 7f4b2a72c640 1 -- 172.21.15.27:0/1456821395 wait complete. 2024-09-12T03:18:34.129 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:34.127+0000 7f4b2a72c640 1 Processor -- start 2024-09-12T03:18:34.129 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:34.127+0000 7f4b2a72c640 1 -- start start 2024-09-12T03:18:34.129 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:34.128+0000 7f4b2a72c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4b2406f360 0x7f4b241f4110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:34.129 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:34.128+0000 7f4b2a72c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b240b1ca0 0x7f4b240b0060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:34.129 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:34.128+0000 7f4b2a72c640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4b240cf490 0x7f4b241f4b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:34.129 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:34.128+0000 7f4b2a72c640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f4b2406d1c0 con 0x7f4b2406f360 2024-09-12T03:18:34.129 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:34.128+0000 7f4b2a72c640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f4b240d1f80 con 0x7f4b240b1ca0 2024-09-12T03:18:34.129 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:34.128+0000 7f4b2a72c640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f4b2406d340 con 0x7f4b240cf490 2024-09-12T03:18:34.130 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:34.128+0000 7f4b23fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4b2406f360 0x7f4b241f4110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:34.130 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:34.128+0000 7f4b23fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4b2406f360 0x7f4b241f4110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44900/0 (socket says 172.21.15.27:44900) 2024-09-12T03:18:34.130 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:34.128+0000 7f4b23fff640 1 -- 172.21.15.27:0/2229908644 learned_addr learned my addr 172.21.15.27:0/2229908644 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:18:34.130 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:34.128+0000 7f4b237fe640 1 --2- 172.21.15.27:0/2229908644 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b240b1ca0 0x7f4b240b0060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:34.130 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:34.128+0000 7f4b28cc9640 1 --2- 172.21.15.27:0/2229908644 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4b240cf490 0x7f4b241f4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:34.130 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:34.128+0000 7f4b23fff640 1 -- 172.21.15.27:0/2229908644 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4b240cf490 msgr2=0x7f4b241f4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:34.130 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:34.128+0000 7f4b23fff640 1 --2- 172.21.15.27:0/2229908644 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4b240cf490 0x7f4b241f4b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:34.130 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:34.128+0000 7f4b23fff640 1 -- 172.21.15.27:0/2229908644 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b240b1ca0 msgr2=0x7f4b240b0060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:34.130 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:34.128+0000 7f4b23fff640 1 --2- 172.21.15.27:0/2229908644 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b240b1ca0 0x7f4b240b0060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:34.130 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:34.128+0000 7f4b23fff640 1 -- 172.21.15.27:0/2229908644 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b241f5040 con 0x7f4b2406f360 2024-09-12T03:18:34.130 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:34.128+0000 7f4b237fe640 1 --2- 172.21.15.27:0/2229908644 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b240b1ca0 0x7f4b240b0060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:18:34.130 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:34.128+0000 7f4b23fff640 1 --2- 172.21.15.27:0/2229908644 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4b2406f360 0x7f4b241f4110 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f4b1400c400 tx=0x7f4b1400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:34.130 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:34.128+0000 7f4b20ff9640 1 -- 172.21.15.27:0/2229908644 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b140052d0 con 0x7f4b2406f360 2024-09-12T03:18:34.131 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:34.128+0000 7f4b20ff9640 1 -- 172.21.15.27:0/2229908644 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b14005490 con 0x7f4b2406f360 2024-09-12T03:18:34.131 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:34.128+0000 7f4b20ff9640 1 -- 172.21.15.27:0/2229908644 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b14007260 con 0x7f4b2406f360 2024-09-12T03:18:34.131 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:34.128+0000 7f4b2a72c640 1 -- 172.21.15.27:0/2229908644 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b240b1800 con 0x7f4b2406f360 2024-09-12T03:18:34.131 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:34.129+0000 7f4b2a72c640 1 -- 172.21.15.27:0/2229908644 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b240b0a40 con 0x7f4b2406f360 2024-09-12T03:18:34.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:34.130+0000 7f4b2a72c640 1 -- 172.21.15.27:0/2229908644 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b240cfc40 con 0x7f4b2406f360 2024-09-12T03:18:34.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:34.130+0000 7f4b20ff9640 1 -- 172.21.15.27:0/2229908644 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 8) ==== 90702+0+0 (secure 0 0 0) 0x7f4b14007400 con 0x7f4b2406f360 2024-09-12T03:18:34.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:34.131+0000 7f4b20ff9640 1 --2- 172.21.15.27:0/2229908644 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f4b1006c070 0x7f4b1006e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:34.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:34.131+0000 7f4b20ff9640 1 -- 172.21.15.27:0/2229908644 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5767+0+0 (secure 0 0 0) 0x7f4b1408a730 con 0x7f4b2406f360 2024-09-12T03:18:34.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:34.131+0000 7f4b237fe640 1 --2- 172.21.15.27:0/2229908644 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f4b1006c070 0x7f4b1006e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:34.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:34.132+0000 7f4b237fe640 1 --2- 172.21.15.27:0/2229908644 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f4b1006c070 0x7f4b1006e530 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f4b18034000 tx=0x7f4b1802e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:34.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:34.135+0000 7f4b20ff9640 1 -- 172.21.15.27:0/2229908644 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f4b14058e10 con 0x7f4b2406f360 2024-09-12T03:18:34.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:34.346+0000 7f4b2a72c640 1 -- 172.21.15.27:0/2229908644 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4b240b3c30 con 0x7f4b2406f360 2024-09-12T03:18:34.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:34.346+0000 7f4b20ff9640 1 -- 172.21.15.27:0/2229908644 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+5223 (secure 0 0 0) 0x7f4b14058710 con 0x7f4b2406f360 2024-09-12T03:18:34.346 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:18:34.346 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":23,"btime":"2024-09-12T03:17:47:237558+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4835,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6834/1457797481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":1457797481},{"type":"v1","addr":"172.21.15.27:6834","nonce":1457797481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5029,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6836/2842491281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":2842491281},{"type":"v1","addr":"172.21.15.45:6836","nonce":2842491281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5032,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/3339744340","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6835","nonce":3339744340},{"type":"v1","addr":"172.21.15.45:6837","nonce":3339744340}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-12T03:17:38.580473+0000","modified":"2024-09-12T03:17:40.557950+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5205},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5205":{"gid":5205,"name":"a","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.27:6835/2571101228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":2571101228},{"type":"v1","addr":"172.21.15.27:6835","nonce":2571101228}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5205,"qdb_cluster":[5205]},"id":3}]} 2024-09-12T03:18:34.347 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 23 2024-09-12T03:18:34.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:34.348+0000 7f4b2a72c640 1 -- 172.21.15.27:0/2229908644 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f4b1006c070 msgr2=0x7f4b1006e530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:34.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:34.349+0000 7f4b2a72c640 1 --2- 172.21.15.27:0/2229908644 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f4b1006c070 0x7f4b1006e530 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f4b18034000 tx=0x7f4b1802e000 comp rx=0 tx=0).stop 2024-09-12T03:18:34.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:34.349+0000 7f4b2a72c640 1 -- 172.21.15.27:0/2229908644 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4b2406f360 msgr2=0x7f4b241f4110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:34.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:34.349+0000 7f4b2a72c640 1 --2- 172.21.15.27:0/2229908644 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4b2406f360 0x7f4b241f4110 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f4b1400c400 tx=0x7f4b1400c8c0 comp rx=0 tx=0).stop 2024-09-12T03:18:34.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:34.349+0000 7f4b2a72c640 1 -- 172.21.15.27:0/2229908644 shutdown_connections 2024-09-12T03:18:34.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:34.349+0000 7f4b2a72c640 1 --2- 172.21.15.27:0/2229908644 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f4b1006c070 0x7f4b1006e530 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:34.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:34.349+0000 7f4b2a72c640 1 --2- 172.21.15.27:0/2229908644 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4b240cf490 0x7f4b241f4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:34.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:34.349+0000 7f4b2a72c640 1 --2- 172.21.15.27:0/2229908644 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b240b1ca0 0x7f4b240b0060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:34.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:34.349+0000 7f4b2a72c640 1 --2- 172.21.15.27:0/2229908644 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4b2406f360 0x7f4b241f4110 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:34.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:34.349+0000 7f4b2a72c640 1 -- 172.21.15.27:0/2229908644 >> 172.21.15.27:0/2229908644 conn(0x7f4b24073800 msgr2=0x7f4b240b2520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:18:34.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:34.349+0000 7f4b2a72c640 1 -- 172.21.15.27:0/2229908644 shutdown_connections 2024-09-12T03:18:34.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:34.349+0000 7f4b2a72c640 1 -- 172.21.15.27:0/2229908644 wait complete. 2024-09-12T03:18:34.360 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json session ls 2024-09-12T03:18:34.481 INFO:teuthology.orchestra.run.smithi027.stdout:[{"id":4955,"entity":{"name":{"type":"client","num":4955},"addr":{"type":"any","addr":"172.21.15.45:0","nonce":3904873653}},"state":"open","num_leases":0,"num_caps":7,"request_load_avg":8,"uptime":44.775858628999998,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":19.838310220912046,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.4955 172.21.15.45:0/3904873653","completed_requests":[{"tid":14,"created_ino":"0x0"},{"tid":15,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc","ceph_version":"ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)","entity_id":"x","hostname":"smithi045","pid":"36189","root":"/"}}] 2024-09-12T03:18:34.481 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc", "ceph_version": "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "x", "hostname": "smithi045", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "36189", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 14 }, { "created_ino": "0x100000001f9", "tid": 15 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.45:0", "nonce": 3904873653, "type": "any" }, "name": { "num": 4955, "type": "client" } }, "id": 4955, "inst": "client.4955 172.21.15.45:0/3904873653", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 0, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 8, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 19.838310220912046 }, "state": "open", "uptime": 44.775858629 } ] 2024-09-12T03:18:34.481 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2024-09-12T03:18:39.482 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-12T03:18:39.599 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:39.598+0000 7f875fba8640 1 Processor -- start 2024-09-12T03:18:39.599 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:39.598+0000 7f875fba8640 1 -- start start 2024-09-12T03:18:39.600 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:39.599+0000 7f875fba8640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f8758158cf0 0x7f87580b55d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:39.600 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:39.599+0000 7f875fba8640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8758162c10 con 0x7f875815f600 2024-09-12T03:18:39.600 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:39.599+0000 7f875fba8640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8758162ef0 con 0x7f87581590d0 2024-09-12T03:18:39.601 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:39.599+0000 7f875fba8640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f87581631d0 con 0x7f8758158cf0 2024-09-12T03:18:39.601 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:39.599+0000 7f875d944640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f8758158cf0 0x7f87580b55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:39.601 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:39.599+0000 7f875d944640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f8758158cf0 0x7f87580b55d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:51550/0 (socket says 172.21.15.27:51550) 2024-09-12T03:18:39.601 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:39.599+0000 7f875d944640 1 -- 172.21.15.27:0/3722901523 learned_addr learned my addr 172.21.15.27:0/3722901523 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:18:39.601 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:39.599+0000 7f875e145640 1 --1- 172.21.15.27:0/3722901523 >> v1:172.21.15.27:6789/0 conn(0x7f875815f600 0x7f875806bb30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:47060/0 (socket says 172.21.15.27:47060) 2024-09-12T03:18:39.601 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:39.599+0000 7f875d944640 1 -- 172.21.15.27:0/3722901523 >> v1:172.21.15.45:6789/0 conn(0x7f87581590d0 legacy=0x7f8758162430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:18:39.601 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:39.599+0000 7f875d944640 1 -- 172.21.15.27:0/3722901523 >> v1:172.21.15.27:6789/0 conn(0x7f875815f600 legacy=0x7f875806bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:39.601 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:39.599+0000 7f875d944640 1 -- 172.21.15.27:0/3722901523 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f875815edc0 con 0x7f8758158cf0 2024-09-12T03:18:39.601 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:39.599+0000 7f875c942640 1 -- 172.21.15.27:0/3722901523 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1192978825 0 0) 0x7f8758162c10 con 0x7f875815f600 2024-09-12T03:18:39.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:39.600+0000 7f875d944640 1 --2- 172.21.15.27:0/3722901523 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8758158cf0 0x7f87580b55d0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f8754009870 tx=0x7f875402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ddd853a9d372b59 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:39.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:39.600+0000 7f875c942640 1 -- 172.21.15.27:0/3722901523 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8754037070 con 0x7f8758158cf0 2024-09-12T03:18:39.603 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:39.600+0000 7f875c942640 1 -- 172.21.15.27:0/3722901523 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8754032440 con 0x7f8758158cf0 2024-09-12T03:18:39.603 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:39.600+0000 7f875c942640 1 -- 172.21.15.27:0/3722901523 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8754032810 con 0x7f8758158cf0 2024-09-12T03:18:39.603 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:39.601+0000 7f875fba8640 1 -- 172.21.15.27:0/3722901523 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8758158cf0 msgr2=0x7f87580b55d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:39.603 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:39.601+0000 7f875fba8640 1 --2- 172.21.15.27:0/3722901523 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8758158cf0 0x7f87580b55d0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f8754009870 tx=0x7f875402a180 comp rx=0 tx=0).stop 2024-09-12T03:18:39.603 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:39.601+0000 7f875fba8640 1 -- 172.21.15.27:0/3722901523 shutdown_connections 2024-09-12T03:18:39.603 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:39.601+0000 7f875fba8640 1 --2- 172.21.15.27:0/3722901523 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8758158cf0 0x7f87580b55d0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:39.603 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:39.601+0000 7f875fba8640 1 -- 172.21.15.27:0/3722901523 >> 172.21.15.27:0/3722901523 conn(0x7f875806cc40 msgr2=0x7f875806d050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:18:39.603 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:39.601+0000 7f875fba8640 1 -- 172.21.15.27:0/3722901523 shutdown_connections 2024-09-12T03:18:39.603 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:39.601+0000 7f875fba8640 1 -- 172.21.15.27:0/3722901523 wait complete. 2024-09-12T03:18:39.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:39.602+0000 7f875fba8640 1 Processor -- start 2024-09-12T03:18:39.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:39.602+0000 7f875fba8640 1 -- start start 2024-09-12T03:18:39.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:39.603+0000 7f875fba8640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8758158cf0 0x7f8758148af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:39.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:39.603+0000 7f875fba8640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f87581590d0 0x7f8758149030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:39.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:39.603+0000 7f875fba8640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f875815f600 0x7f87580aec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:39.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:39.603+0000 7f875fba8640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f875815fa50 con 0x7f87581590d0 2024-09-12T03:18:39.607 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:39.603+0000 7f875fba8640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f875806c450 con 0x7f8758158cf0 2024-09-12T03:18:39.607 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:39.603+0000 7f875fba8640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f875815fbd0 con 0x7f875815f600 2024-09-12T03:18:39.607 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:39.603+0000 7f875d143640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f87581590d0 0x7f8758149030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:39.607 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:39.603+0000 7f875d143640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f87581590d0 0x7f8758149030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:51566/0 (socket says 172.21.15.27:51566) 2024-09-12T03:18:39.607 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:39.603+0000 7f875d143640 1 -- 172.21.15.27:0/3692664795 learned_addr learned my addr 172.21.15.27:0/3692664795 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:18:39.607 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:39.603+0000 7f875d944640 1 --2- 172.21.15.27:0/3692664795 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8758158cf0 0x7f8758148af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:39.607 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:39.603+0000 7f875e145640 1 --2- 172.21.15.27:0/3692664795 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f875815f600 0x7f87580aec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:39.607 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:39.603+0000 7f875d143640 1 -- 172.21.15.27:0/3692664795 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f875815f600 msgr2=0x7f87580aec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:39.607 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:39.603+0000 7f875d143640 1 --2- 172.21.15.27:0/3692664795 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f875815f600 0x7f87580aec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:39.607 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:39.603+0000 7f875d143640 1 -- 172.21.15.27:0/3692664795 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8758158cf0 msgr2=0x7f8758148af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:39.607 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:39.603+0000 7f875d143640 1 --2- 172.21.15.27:0/3692664795 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8758158cf0 0x7f8758148af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:39.607 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:39.603+0000 7f875d143640 1 -- 172.21.15.27:0/3692664795 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f875815c3b0 con 0x7f87581590d0 2024-09-12T03:18:39.607 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:39.603+0000 7f875e145640 1 --2- 172.21.15.27:0/3692664795 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f875815f600 0x7f87580aec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:18:39.608 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:39.604+0000 7f875d143640 1 --2- 172.21.15.27:0/3692664795 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f87581590d0 0x7f8758149030 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f874800c3f0 tx=0x7f874800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:39.608 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:39.604+0000 7f874e7fc640 1 -- 172.21.15.27:0/3692664795 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8748007020 con 0x7f87581590d0 2024-09-12T03:18:39.608 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:39.604+0000 7f874e7fc640 1 -- 172.21.15.27:0/3692664795 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87480071c0 con 0x7f87581590d0 2024-09-12T03:18:39.608 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:39.604+0000 7f874e7fc640 1 -- 172.21.15.27:0/3692664795 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87480075b0 con 0x7f87581590d0 2024-09-12T03:18:39.608 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:39.604+0000 7f875fba8640 1 -- 172.21.15.27:0/3692664795 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87581e82d0 con 0x7f87581590d0 2024-09-12T03:18:39.608 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:39.604+0000 7f875fba8640 1 -- 172.21.15.27:0/3692664795 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87581e8890 con 0x7f87581590d0 2024-09-12T03:18:39.608 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:39.605+0000 7f875fba8640 1 -- 172.21.15.27:0/3692664795 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87580b55d0 con 0x7f87581590d0 2024-09-12T03:18:39.608 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:39.605+0000 7f874e7fc640 1 -- 172.21.15.27:0/3692664795 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 8) ==== 90702+0+0 (secure 0 0 0) 0x7f8748019020 con 0x7f87581590d0 2024-09-12T03:18:39.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:39.606+0000 7f874e7fc640 1 --2- 172.21.15.27:0/3692664795 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f873c06c070 0x7f873c06e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:39.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:39.606+0000 7f874e7fc640 1 -- 172.21.15.27:0/3692664795 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5767+0+0 (secure 0 0 0) 0x7f874808a8c0 con 0x7f87581590d0 2024-09-12T03:18:39.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:39.606+0000 7f875d944640 1 --2- 172.21.15.27:0/3692664795 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f873c06c070 0x7f873c06e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:39.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:39.607+0000 7f875d944640 1 --2- 172.21.15.27:0/3692664795 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f873c06c070 0x7f873c06e530 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f8754034000 tx=0x7f875402e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:39.611 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:39.611+0000 7f874e7fc640 1 -- 172.21.15.27:0/3692664795 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f8748059030 con 0x7f87581590d0 2024-09-12T03:18:39.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:39.820+0000 7f875fba8640 1 -- 172.21.15.27:0/3692664795 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f87580b4b50 con 0x7f87581590d0 2024-09-12T03:18:39.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:39.821+0000 7f874e7fc640 1 -- 172.21.15.27:0/3692664795 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+5223 (secure 0 0 0) 0x7f8748058930 con 0x7f87581590d0 2024-09-12T03:18:39.822 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:18:39.822 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":23,"btime":"2024-09-12T03:17:47:237558+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4835,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6834/1457797481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":1457797481},{"type":"v1","addr":"172.21.15.27:6834","nonce":1457797481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5029,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6836/2842491281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":2842491281},{"type":"v1","addr":"172.21.15.45:6836","nonce":2842491281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5032,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/3339744340","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6835","nonce":3339744340},{"type":"v1","addr":"172.21.15.45:6837","nonce":3339744340}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-12T03:17:38.580473+0000","modified":"2024-09-12T03:17:40.557950+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5205},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5205":{"gid":5205,"name":"a","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.27:6835/2571101228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":2571101228},{"type":"v1","addr":"172.21.15.27:6835","nonce":2571101228}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5205,"qdb_cluster":[5205]},"id":3}]} 2024-09-12T03:18:39.822 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 23 2024-09-12T03:18:39.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:39.823+0000 7f875fba8640 1 -- 172.21.15.27:0/3692664795 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f873c06c070 msgr2=0x7f873c06e530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:39.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:39.823+0000 7f875fba8640 1 --2- 172.21.15.27:0/3692664795 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f873c06c070 0x7f873c06e530 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f8754034000 tx=0x7f875402e000 comp rx=0 tx=0).stop 2024-09-12T03:18:39.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:39.823+0000 7f875fba8640 1 -- 172.21.15.27:0/3692664795 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f87581590d0 msgr2=0x7f8758149030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:39.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:39.823+0000 7f875fba8640 1 --2- 172.21.15.27:0/3692664795 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f87581590d0 0x7f8758149030 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f874800c3f0 tx=0x7f874800c8b0 comp rx=0 tx=0).stop 2024-09-12T03:18:39.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:39.823+0000 7f875fba8640 1 -- 172.21.15.27:0/3692664795 shutdown_connections 2024-09-12T03:18:39.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:39.823+0000 7f875fba8640 1 --2- 172.21.15.27:0/3692664795 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f873c06c070 0x7f873c06e530 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:39.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:39.823+0000 7f875fba8640 1 --2- 172.21.15.27:0/3692664795 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f875815f600 0x7f87580aec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:39.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:39.823+0000 7f875fba8640 1 --2- 172.21.15.27:0/3692664795 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f87581590d0 0x7f8758149030 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:39.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:39.823+0000 7f875fba8640 1 --2- 172.21.15.27:0/3692664795 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8758158cf0 0x7f8758148af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:39.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:39.823+0000 7f875fba8640 1 -- 172.21.15.27:0/3692664795 >> 172.21.15.27:0/3692664795 conn(0x7f875806cc40 msgr2=0x7f87580b32e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:18:39.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:39.823+0000 7f875fba8640 1 -- 172.21.15.27:0/3692664795 shutdown_connections 2024-09-12T03:18:39.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:39.823+0000 7f875fba8640 1 -- 172.21.15.27:0/3692664795 wait complete. 2024-09-12T03:18:39.835 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json session ls 2024-09-12T03:18:39.956 INFO:teuthology.orchestra.run.smithi027.stdout:[{"id":4955,"entity":{"name":{"type":"client","num":4955},"addr":{"type":"any","addr":"172.21.15.45:0","nonce":3904873653}},"state":"open","num_leases":0,"num_caps":7,"request_load_avg":8,"uptime":50.250719986,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":19.588943740462259,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.4955 172.21.15.45:0/3904873653","completed_requests":[{"tid":14,"created_ino":"0x0"},{"tid":15,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc","ceph_version":"ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)","entity_id":"x","hostname":"smithi045","pid":"36189","root":"/"}}] 2024-09-12T03:18:39.956 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc", "ceph_version": "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "x", "hostname": "smithi045", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "36189", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 14 }, { "created_ino": "0x100000001f9", "tid": 15 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.45:0", "nonce": 3904873653, "type": "any" }, "name": { "num": 4955, "type": "client" } }, "id": 4955, "inst": "client.4955 172.21.15.45:0/3904873653", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 0, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 8, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 19.58894374046226 }, "state": "open", "uptime": 50.250719986 } ] 2024-09-12T03:18:39.956 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2024-09-12T03:18:44.958 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-12T03:18:45.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:45.072+0000 7f1518665640 1 Processor -- start 2024-09-12T03:18:45.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:45.072+0000 7f1518665640 1 -- start start 2024-09-12T03:18:45.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:45.073+0000 7f1518665640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f151006f360 0x7f15100cef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:45.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:45.073+0000 7f1518665640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f15100cf490 0x7f15100d2780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:45.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:45.073+0000 7f1518665640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f151006cc40 con 0x7f15100b1ca0 2024-09-12T03:18:45.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:45.073+0000 7f1518665640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f15100cf900 con 0x7f15100cf490 2024-09-12T03:18:45.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:45.073+0000 7f1518665640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f15100d22f0 con 0x7f151006f360 2024-09-12T03:18:45.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:45.074+0000 7f1516401640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f15100b1ca0 0x7f151015f830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:47074/0 (socket says 172.21.15.27:47074) 2024-09-12T03:18:45.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:45.074+0000 7f1516401640 1 -- 172.21.15.27:0/453734694 learned_addr learned my addr 172.21.15.27:0/453734694 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:18:45.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:45.074+0000 7f1516c02640 1 --2- 172.21.15.27:0/453734694 >> v2:172.21.15.27:3300/0 conn(0x7f15100cf490 0x7f15100d2780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:45.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:45.074+0000 7f1515c00640 1 --2- 172.21.15.27:0/453734694 >> v2:172.21.15.45:3300/0 conn(0x7f151006f360 0x7f15100cef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:45.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:45.074+0000 7f1516c02640 1 -- 172.21.15.27:0/453734694 >> v2:172.21.15.45:3300/0 conn(0x7f151006f360 msgr2=0x7f15100cef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:18:45.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:45.074+0000 7f1516c02640 1 --2- 172.21.15.27:0/453734694 >> v2:172.21.15.45:3300/0 conn(0x7f151006f360 0x7f15100cef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:45.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:45.074+0000 7f1516c02640 1 -- 172.21.15.27:0/453734694 >> v1:172.21.15.27:6789/0 conn(0x7f15100b1ca0 legacy=0x7f151015f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:45.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:45.074+0000 7f1516c02640 1 -- 172.21.15.27:0/453734694 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15100d1ce0 con 0x7f15100cf490 2024-09-12T03:18:45.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:45.075+0000 7f1516c02640 1 --2- 172.21.15.27:0/453734694 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f15100cf490 0x7f15100d2780 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f15100b5220 tx=0x7f150802a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f78639ccbc646c4b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:45.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:45.075+0000 7f15153ff640 1 -- 172.21.15.27:0/453734694 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1508007740 con 0x7f15100cf490 2024-09-12T03:18:45.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:45.075+0000 7f15153ff640 1 -- 172.21.15.27:0/453734694 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f15080078e0 con 0x7f15100cf490 2024-09-12T03:18:45.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:45.075+0000 7f15153ff640 1 -- 172.21.15.27:0/453734694 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1508007cb0 con 0x7f15100cf490 2024-09-12T03:18:45.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:45.076+0000 7f1518665640 1 -- 172.21.15.27:0/453734694 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f15100cf490 msgr2=0x7f15100d2780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:45.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:45.076+0000 7f1518665640 1 --2- 172.21.15.27:0/453734694 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f15100cf490 0x7f15100d2780 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f15100b5220 tx=0x7f150802a680 comp rx=0 tx=0).stop 2024-09-12T03:18:45.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:45.076+0000 7f1518665640 1 -- 172.21.15.27:0/453734694 shutdown_connections 2024-09-12T03:18:45.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:45.076+0000 7f1518665640 1 --2- 172.21.15.27:0/453734694 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f15100cf490 0x7f15100d2780 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:45.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:45.076+0000 7f1518665640 1 --2- 172.21.15.27:0/453734694 >> v2:172.21.15.45:3300/0 conn(0x7f151006f360 0x7f15100cef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:45.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:45.076+0000 7f1518665640 1 -- 172.21.15.27:0/453734694 >> 172.21.15.27:0/453734694 conn(0x7f1510073800 msgr2=0x7f151009ba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:18:45.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:45.076+0000 7f1518665640 1 -- 172.21.15.27:0/453734694 shutdown_connections 2024-09-12T03:18:45.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:45.076+0000 7f1518665640 1 -- 172.21.15.27:0/453734694 wait complete. 2024-09-12T03:18:45.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:45.077+0000 7f1518665640 1 Processor -- start 2024-09-12T03:18:45.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:45.077+0000 7f1518665640 1 -- start start 2024-09-12T03:18:45.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:45.078+0000 7f1518665640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f151006f360 0x7f15100b75c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:45.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:45.078+0000 7f1518665640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f15100b1ca0 0x7f151014e080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:45.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:45.078+0000 7f1518665640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f15100cf490 0x7f15100cd450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:45.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:45.078+0000 7f1518665640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f151006d1c0 con 0x7f15100cf490 2024-09-12T03:18:45.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:45.078+0000 7f1518665640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f15100d1f80 con 0x7f15100b1ca0 2024-09-12T03:18:45.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:45.078+0000 7f1518665640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f151006d340 con 0x7f151006f360 2024-09-12T03:18:45.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:45.078+0000 7f1516c02640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f15100cf490 0x7f15100cd450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:45.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:45.078+0000 7f1516c02640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f15100cf490 0x7f15100cd450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:51584/0 (socket says 172.21.15.27:51584) 2024-09-12T03:18:45.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:45.078+0000 7f1516c02640 1 -- 172.21.15.27:0/3973890393 learned_addr learned my addr 172.21.15.27:0/3973890393 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:18:45.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:45.078+0000 7f1516401640 1 --2- 172.21.15.27:0/3973890393 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f151006f360 0x7f15100b75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:45.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:45.078+0000 7f1515c00640 1 --2- 172.21.15.27:0/3973890393 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f15100b1ca0 0x7f151014e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:45.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:45.078+0000 7f1516c02640 1 -- 172.21.15.27:0/3973890393 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f151006f360 msgr2=0x7f15100b75c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:45.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:45.078+0000 7f1516c02640 1 --2- 172.21.15.27:0/3973890393 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f151006f360 0x7f15100b75c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:45.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:45.078+0000 7f1516c02640 1 -- 172.21.15.27:0/3973890393 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f15100b1ca0 msgr2=0x7f151014e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:45.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:45.078+0000 7f1516c02640 1 --2- 172.21.15.27:0/3973890393 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f15100b1ca0 0x7f151014e080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:45.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:45.078+0000 7f1516c02640 1 -- 172.21.15.27:0/3973890393 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1510160850 con 0x7f15100cf490 2024-09-12T03:18:45.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:45.079+0000 7f1516c02640 1 --2- 172.21.15.27:0/3973890393 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f15100cf490 0x7f15100cd450 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f150802ac70 tx=0x7f1508004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:45.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:45.079+0000 7f1506ffd640 1 -- 172.21.15.27:0/3973890393 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f150803f070 con 0x7f15100cf490 2024-09-12T03:18:45.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:45.079+0000 7f1506ffd640 1 -- 172.21.15.27:0/3973890393 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1508031bc0 con 0x7f15100cf490 2024-09-12T03:18:45.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:45.079+0000 7f1518665640 1 -- 172.21.15.27:0/3973890393 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15100cbbf0 con 0x7f15100cf490 2024-09-12T03:18:45.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:45.079+0000 7f1506ffd640 1 -- 172.21.15.27:0/3973890393 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1508044460 con 0x7f15100cf490 2024-09-12T03:18:45.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:45.079+0000 7f1518665640 1 -- 172.21.15.27:0/3973890393 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15100cb850 con 0x7f15100cf490 2024-09-12T03:18:45.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:45.080+0000 7f1518665640 1 -- 172.21.15.27:0/3973890393 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15100cfc40 con 0x7f15100cf490 2024-09-12T03:18:45.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:45.081+0000 7f1506ffd640 1 -- 172.21.15.27:0/3973890393 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 8) ==== 90702+0+0 (secure 0 0 0) 0x7f1508007790 con 0x7f15100cf490 2024-09-12T03:18:45.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:45.081+0000 7f1506ffd640 1 --2- 172.21.15.27:0/3973890393 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f14f406c0a0 0x7f14f406e560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:45.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:45.081+0000 7f1506ffd640 1 -- 172.21.15.27:0/3973890393 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5767+0+0 (secure 0 0 0) 0x7f15080aca80 con 0x7f15100cf490 2024-09-12T03:18:45.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:45.081+0000 7f1516401640 1 --2- 172.21.15.27:0/3973890393 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f14f406c0a0 0x7f14f406e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:45.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:45.082+0000 7f1516401640 1 --2- 172.21.15.27:0/3973890393 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f14f406c0a0 0x7f14f406e560 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f150c000f60 tx=0x7f150c009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:45.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:45.086+0000 7f1506ffd640 1 -- 172.21.15.27:0/3973890393 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f150807b0e0 con 0x7f15100cf490 2024-09-12T03:18:45.296 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:45.295+0000 7f1518665640 1 -- 172.21.15.27:0/3973890393 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f15100d1130 con 0x7f15100cf490 2024-09-12T03:18:45.296 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:45.295+0000 7f1506ffd640 1 -- 172.21.15.27:0/3973890393 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+5223 (secure 0 0 0) 0x7f150807a9e0 con 0x7f15100cf490 2024-09-12T03:18:45.296 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:18:45.297 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":23,"btime":"2024-09-12T03:17:47:237558+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4835,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6834/1457797481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":1457797481},{"type":"v1","addr":"172.21.15.27:6834","nonce":1457797481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5029,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6836/2842491281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":2842491281},{"type":"v1","addr":"172.21.15.45:6836","nonce":2842491281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5032,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/3339744340","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6835","nonce":3339744340},{"type":"v1","addr":"172.21.15.45:6837","nonce":3339744340}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-12T03:17:38.580473+0000","modified":"2024-09-12T03:17:40.557950+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5205},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5205":{"gid":5205,"name":"a","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.27:6835/2571101228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":2571101228},{"type":"v1","addr":"172.21.15.27:6835","nonce":2571101228}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5205,"qdb_cluster":[5205]},"id":3}]} 2024-09-12T03:18:45.297 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 23 2024-09-12T03:18:45.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:45.298+0000 7f1518665640 1 -- 172.21.15.27:0/3973890393 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f14f406c0a0 msgr2=0x7f14f406e560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:45.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:45.298+0000 7f1518665640 1 --2- 172.21.15.27:0/3973890393 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f14f406c0a0 0x7f14f406e560 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f150c000f60 tx=0x7f150c009040 comp rx=0 tx=0).stop 2024-09-12T03:18:45.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:45.298+0000 7f1518665640 1 -- 172.21.15.27:0/3973890393 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f15100cf490 msgr2=0x7f15100cd450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:45.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:45.298+0000 7f1518665640 1 --2- 172.21.15.27:0/3973890393 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f15100cf490 0x7f15100cd450 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f150802ac70 tx=0x7f1508004a40 comp rx=0 tx=0).stop 2024-09-12T03:18:45.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:45.298+0000 7f1518665640 1 -- 172.21.15.27:0/3973890393 shutdown_connections 2024-09-12T03:18:45.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:45.298+0000 7f1518665640 1 --2- 172.21.15.27:0/3973890393 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f14f406c0a0 0x7f14f406e560 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:45.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:45.298+0000 7f1518665640 1 --2- 172.21.15.27:0/3973890393 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f15100cf490 0x7f15100cd450 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:45.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:45.298+0000 7f1518665640 1 --2- 172.21.15.27:0/3973890393 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f15100b1ca0 0x7f151014e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:45.300 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:45.298+0000 7f1518665640 1 --2- 172.21.15.27:0/3973890393 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f151006f360 0x7f15100b75c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:45.300 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:45.298+0000 7f1518665640 1 -- 172.21.15.27:0/3973890393 >> 172.21.15.27:0/3973890393 conn(0x7f1510073800 msgr2=0x7f15100d2780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:18:45.300 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:45.298+0000 7f1518665640 1 -- 172.21.15.27:0/3973890393 shutdown_connections 2024-09-12T03:18:45.300 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:45.298+0000 7f1518665640 1 -- 172.21.15.27:0/3973890393 wait complete. 2024-09-12T03:18:45.310 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json session ls 2024-09-12T03:18:45.432 INFO:teuthology.orchestra.run.smithi027.stdout:[{"id":4955,"entity":{"name":{"type":"client","num":4955},"addr":{"type":"any","addr":"172.21.15.45:0","nonce":3904873653}},"state":"open","num_leases":0,"num_caps":7,"request_load_avg":7,"uptime":55.725581343000002,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":19.34271178310841,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.4955 172.21.15.45:0/3904873653","completed_requests":[{"tid":14,"created_ino":"0x0"},{"tid":15,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc","ceph_version":"ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)","entity_id":"x","hostname":"smithi045","pid":"36189","root":"/"}}] 2024-09-12T03:18:45.433 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc", "ceph_version": "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "x", "hostname": "smithi045", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "36189", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 14 }, { "created_ino": "0x100000001f9", "tid": 15 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.45:0", "nonce": 3904873653, "type": "any" }, "name": { "num": 4955, "type": "client" } }, "id": 4955, "inst": "client.4955 172.21.15.45:0/3904873653", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 0, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 7, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 19.34271178310841 }, "state": "open", "uptime": 55.725581343 } ] 2024-09-12T03:18:45.433 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2024-09-12T03:18:50.434 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-12T03:18:50.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:50.549+0000 7f4122fc4640 1 Processor -- start 2024-09-12T03:18:50.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:50.549+0000 7f4122fc4640 1 -- start start 2024-09-12T03:18:50.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:50.550+0000 7f4122fc4640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f411c15b2b0 0x7f411c0b3560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:50.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:50.550+0000 7f4122fc4640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f411c0ce410 con 0x7f411c06e9b0 2024-09-12T03:18:50.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:50.550+0000 7f4122fc4640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f411c0732c0 con 0x7f411c072d70 2024-09-12T03:18:50.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:50.550+0000 7f4122fc4640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f411c0720a0 con 0x7f411c15b2b0 2024-09-12T03:18:50.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:50.550+0000 7f4121561640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f411c06e9b0 0x7f411c0cebb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:51600/0 (socket says 172.21.15.27:51600) 2024-09-12T03:18:50.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:50.550+0000 7f4121561640 1 -- 172.21.15.27:0/1557566285 learned_addr learned my addr 172.21.15.27:0/1557566285 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:18:50.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:50.550+0000 7f4120d60640 1 --2- 172.21.15.27:0/1557566285 >> v2:172.21.15.45:3300/0 conn(0x7f411c15b2b0 0x7f411c0b3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:50.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:50.550+0000 7f411b7fe640 1 -- 172.21.15.27:0/1557566285 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3813739835 0 0) 0x7f4114001490 con 0x7f411c06e9b0 2024-09-12T03:18:50.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:50.551+0000 7f411b7fe640 1 -- 172.21.15.27:0/1557566285 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4104003520 con 0x7f411c06e9b0 2024-09-12T03:18:50.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:50.551+0000 7f411b7fe640 1 -- 172.21.15.27:0/1557566285 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1912020620 0 0) 0x7f4104003520 con 0x7f411c06e9b0 2024-09-12T03:18:50.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:50.551+0000 7f411b7fe640 1 -- 172.21.15.27:0/1557566285 >> v2:172.21.15.45:3300/0 conn(0x7f411c15b2b0 msgr2=0x7f411c0b3560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:18:50.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:50.551+0000 7f411b7fe640 1 --2- 172.21.15.27:0/1557566285 >> v2:172.21.15.45:3300/0 conn(0x7f411c15b2b0 0x7f411c0b3560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:50.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:50.551+0000 7f411b7fe640 1 -- 172.21.15.27:0/1557566285 >> v1:172.21.15.45:6789/0 conn(0x7f411c072d70 legacy=0x7f411c072430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:50.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:50.551+0000 7f411b7fe640 1 -- 172.21.15.27:0/1557566285 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f411c071ae0 con 0x7f411c06e9b0 2024-09-12T03:18:50.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:50.551+0000 7f411b7fe640 1 -- 172.21.15.27:0/1557566285 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 847755213 0 0) 0x7f4114001bc0 con 0x7f411c06e9b0 2024-09-12T03:18:50.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:50.551+0000 7f411b7fe640 1 -- 172.21.15.27:0/1557566285 >> v1:172.21.15.27:6789/0 conn(0x7f411c06e9b0 legacy=0x7f411c0cebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:50.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:50.551+0000 7f411b7fe640 1 --2- 172.21.15.27:0/1557566285 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f41040039b0 0x7f4104003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:50.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:50.551+0000 7f411b7fe640 1 --2- 172.21.15.27:0/1557566285 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4104004800 0x7f4104004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:50.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:50.551+0000 7f411b7fe640 1 --2- 172.21.15.27:0/1557566285 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f41040051c0 0x7f410400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:50.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:50.551+0000 7f411b7fe640 1 -- 172.21.15.27:0/1557566285 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f411c0720a0 con 0x7f41040039b0 2024-09-12T03:18:50.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:50.551+0000 7f411b7fe640 1 -- 172.21.15.27:0/1557566285 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f410400c060 con 0x7f41040051c0 2024-09-12T03:18:50.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:50.551+0000 7f411b7fe640 1 -- 172.21.15.27:0/1557566285 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f410400c260 con 0x7f4104004800 2024-09-12T03:18:50.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:50.551+0000 7f4120d60640 1 --2- 172.21.15.27:0/1557566285 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f41040039b0 0x7f4104003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:50.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:50.551+0000 7f4121561640 1 --2- 172.21.15.27:0/1557566285 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f41040051c0 0x7f410400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:50.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:50.551+0000 7f411bfff640 1 --2- 172.21.15.27:0/1557566285 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4104004800 0x7f4104004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:50.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:50.551+0000 7f4120d60640 1 -- 172.21.15.27:0/1557566285 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4104004800 msgr2=0x7f4104004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:18:50.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:50.551+0000 7f4120d60640 1 --2- 172.21.15.27:0/1557566285 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4104004800 0x7f4104004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:50.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:50.551+0000 7f4120d60640 1 -- 172.21.15.27:0/1557566285 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f41040051c0 msgr2=0x7f410400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:18:50.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:50.551+0000 7f4120d60640 1 --2- 172.21.15.27:0/1557566285 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f41040051c0 0x7f410400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:50.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:50.551+0000 7f4120d60640 1 -- 172.21.15.27:0/1557566285 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4104003520 con 0x7f41040039b0 2024-09-12T03:18:50.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:50.551+0000 7f4121561640 1 -- 172.21.15.27:0/1557566285 reap_dead start 2024-09-12T03:18:50.558 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:50.552+0000 7f4120d60640 1 --2- 172.21.15.27:0/1557566285 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f41040039b0 0x7f4104003dd0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f410c009ac0 tx=0x7f4104005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9df4aaac5c1208ee server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:50.558 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:50.552+0000 7f411b7fe640 1 -- 172.21.15.27:0/1557566285 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f410c036070 con 0x7f41040039b0 2024-09-12T03:18:50.558 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:50.552+0000 7f411b7fe640 1 -- 172.21.15.27:0/1557566285 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f410c031400 con 0x7f41040039b0 2024-09-12T03:18:50.558 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:50.552+0000 7f411b7fe640 1 -- 172.21.15.27:0/1557566285 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f410c009690 con 0x7f41040039b0 2024-09-12T03:18:50.558 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:50.553+0000 7f4122fc4640 1 -- 172.21.15.27:0/1557566285 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f41040039b0 msgr2=0x7f4104003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:50.558 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:50.553+0000 7f4122fc4640 1 --2- 172.21.15.27:0/1557566285 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f41040039b0 0x7f4104003dd0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f410c009ac0 tx=0x7f4104005830 comp rx=0 tx=0).stop 2024-09-12T03:18:50.558 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:50.553+0000 7f4122fc4640 1 -- 172.21.15.27:0/1557566285 shutdown_connections 2024-09-12T03:18:50.558 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:50.553+0000 7f4122fc4640 1 --2- 172.21.15.27:0/1557566285 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f41040039b0 0x7f4104003dd0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:50.558 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:50.553+0000 7f4122fc4640 1 -- 172.21.15.27:0/1557566285 >> 172.21.15.27:0/1557566285 conn(0x7f411c062c60 msgr2=0x7f411c06bb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:18:50.558 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:50.553+0000 7f4122fc4640 1 -- 172.21.15.27:0/1557566285 shutdown_connections 2024-09-12T03:18:50.558 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:50.553+0000 7f4122fc4640 1 -- 172.21.15.27:0/1557566285 wait complete. 2024-09-12T03:18:50.558 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:50.554+0000 7f4122fc4640 1 Processor -- start 2024-09-12T03:18:50.558 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:50.554+0000 7f4122fc4640 1 -- start start 2024-09-12T03:18:50.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:50.555+0000 7f4122fc4640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f41040039b0 0x7f411c14fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:50.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:50.555+0000 7f4122fc4640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f411c1506d0 0x7f411c15a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:50.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:50.555+0000 7f4122fc4640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f411c15a730 0x7f411c1e6f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:50.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:50.555+0000 7f4122fc4640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f411c15b690 con 0x7f411c1506d0 2024-09-12T03:18:50.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:50.555+0000 7f4122fc4640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f411c0ce690 con 0x7f411c15a730 2024-09-12T03:18:50.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:50.555+0000 7f4122fc4640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f411c15b810 con 0x7f41040039b0 2024-09-12T03:18:50.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:50.555+0000 7f411bfff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f411c1506d0 0x7f411c15a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:50.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:50.555+0000 7f411bfff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f411c1506d0 0x7f411c15a1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:53450/0 (socket says 172.21.15.27:53450) 2024-09-12T03:18:50.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:50.555+0000 7f411bfff640 1 -- 172.21.15.27:0/457453081 learned_addr learned my addr 172.21.15.27:0/457453081 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:18:50.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:50.555+0000 7f4120d60640 1 --2- 172.21.15.27:0/457453081 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f41040039b0 0x7f411c14fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:50.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:50.555+0000 7f4121561640 1 --2- 172.21.15.27:0/457453081 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f411c15a730 0x7f411c1e6f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:50.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:50.556+0000 7f411bfff640 1 -- 172.21.15.27:0/457453081 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f41040039b0 msgr2=0x7f411c14fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:50.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:50.556+0000 7f411bfff640 1 --2- 172.21.15.27:0/457453081 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f41040039b0 0x7f411c14fbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:50.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:50.556+0000 7f411bfff640 1 -- 172.21.15.27:0/457453081 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f411c15a730 msgr2=0x7f411c1e6f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:50.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:50.556+0000 7f411bfff640 1 --2- 172.21.15.27:0/457453081 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f411c15a730 0x7f411c1e6f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:50.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:50.556+0000 7f411bfff640 1 -- 172.21.15.27:0/457453081 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f411c0b46f0 con 0x7f411c1506d0 2024-09-12T03:18:50.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:50.556+0000 7f4121561640 1 --2- 172.21.15.27:0/457453081 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f411c15a730 0x7f411c1e6f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:18:50.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:50.556+0000 7f411bfff640 1 --2- 172.21.15.27:0/457453081 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f411c1506d0 0x7f411c15a1f0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f411000f880 tx=0x7f411000fd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:50.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:50.556+0000 7f41197fa640 1 -- 172.21.15.27:0/457453081 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4110001d20 con 0x7f411c1506d0 2024-09-12T03:18:50.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:50.556+0000 7f41197fa640 1 -- 172.21.15.27:0/457453081 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4110018070 con 0x7f411c1506d0 2024-09-12T03:18:50.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:50.556+0000 7f4122fc4640 1 -- 172.21.15.27:0/457453081 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f411c0b49d0 con 0x7f411c1506d0 2024-09-12T03:18:50.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:50.556+0000 7f41197fa640 1 -- 172.21.15.27:0/457453081 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f411000d730 con 0x7f411c1506d0 2024-09-12T03:18:50.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:50.556+0000 7f4122fc4640 1 -- 172.21.15.27:0/457453081 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f411c0b4f90 con 0x7f411c1506d0 2024-09-12T03:18:50.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:50.557+0000 7f4122fc4640 1 -- 172.21.15.27:0/457453081 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f411c0b3560 con 0x7f411c1506d0 2024-09-12T03:18:50.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:50.558+0000 7f41197fa640 1 -- 172.21.15.27:0/457453081 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 8) ==== 90702+0+0 (secure 0 0 0) 0x7f411000d8d0 con 0x7f411c1506d0 2024-09-12T03:18:50.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:50.558+0000 7f41197fa640 1 --2- 172.21.15.27:0/457453081 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f410806c0a0 0x7f410806e560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:50.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:50.558+0000 7f41197fa640 1 -- 172.21.15.27:0/457453081 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5767+0+0 (secure 0 0 0) 0x7f411008af10 con 0x7f411c1506d0 2024-09-12T03:18:50.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:50.558+0000 7f4120d60640 1 --2- 172.21.15.27:0/457453081 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f410806c0a0 0x7f410806e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:50.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:50.559+0000 7f4120d60640 1 --2- 172.21.15.27:0/457453081 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f410806c0a0 0x7f410806e560 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f410c033000 tx=0x7f410c008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:50.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:50.562+0000 7f41197fa640 1 -- 172.21.15.27:0/457453081 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f41100595b0 con 0x7f411c1506d0 2024-09-12T03:18:50.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:50.778+0000 7f4122fc4640 1 -- 172.21.15.27:0/457453081 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f411c15fdf0 con 0x7f411c1506d0 2024-09-12T03:18:50.780 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:50.779+0000 7f41197fa640 1 -- 172.21.15.27:0/457453081 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+5223 (secure 0 0 0) 0x7f4110058eb0 con 0x7f411c1506d0 2024-09-12T03:18:50.780 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:18:50.780 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":23,"btime":"2024-09-12T03:17:47:237558+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4835,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6834/1457797481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":1457797481},{"type":"v1","addr":"172.21.15.27:6834","nonce":1457797481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5029,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6836/2842491281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":2842491281},{"type":"v1","addr":"172.21.15.45:6836","nonce":2842491281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5032,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/3339744340","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6835","nonce":3339744340},{"type":"v1","addr":"172.21.15.45:6837","nonce":3339744340}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-12T03:17:38.580473+0000","modified":"2024-09-12T03:17:40.557950+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5205},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5205":{"gid":5205,"name":"a","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.27:6835/2571101228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":2571101228},{"type":"v1","addr":"172.21.15.27:6835","nonce":2571101228}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5205,"qdb_cluster":[5205]},"id":3}]} 2024-09-12T03:18:50.780 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 23 2024-09-12T03:18:50.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:50.781+0000 7f4122fc4640 1 -- 172.21.15.27:0/457453081 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f410806c0a0 msgr2=0x7f410806e560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:50.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:50.781+0000 7f4122fc4640 1 --2- 172.21.15.27:0/457453081 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f410806c0a0 0x7f410806e560 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f410c033000 tx=0x7f410c008030 comp rx=0 tx=0).stop 2024-09-12T03:18:50.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:50.781+0000 7f4122fc4640 1 -- 172.21.15.27:0/457453081 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f411c1506d0 msgr2=0x7f411c15a1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:50.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:50.781+0000 7f4122fc4640 1 --2- 172.21.15.27:0/457453081 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f411c1506d0 0x7f411c15a1f0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f411000f880 tx=0x7f411000fd40 comp rx=0 tx=0).stop 2024-09-12T03:18:50.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:50.781+0000 7f4122fc4640 1 -- 172.21.15.27:0/457453081 shutdown_connections 2024-09-12T03:18:50.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:50.781+0000 7f4122fc4640 1 --2- 172.21.15.27:0/457453081 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f410806c0a0 0x7f410806e560 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:50.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:50.781+0000 7f4122fc4640 1 --2- 172.21.15.27:0/457453081 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f411c15a730 0x7f411c1e6f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:50.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:50.781+0000 7f4122fc4640 1 --2- 172.21.15.27:0/457453081 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f411c1506d0 0x7f411c15a1f0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:50.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:50.781+0000 7f4122fc4640 1 --2- 172.21.15.27:0/457453081 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f41040039b0 0x7f411c14fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:50.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:50.781+0000 7f4122fc4640 1 -- 172.21.15.27:0/457453081 >> 172.21.15.27:0/457453081 conn(0x7f411c062c60 msgr2=0x7f411c06bb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:18:50.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:50.781+0000 7f4122fc4640 1 -- 172.21.15.27:0/457453081 shutdown_connections 2024-09-12T03:18:50.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:50.781+0000 7f4122fc4640 1 -- 172.21.15.27:0/457453081 wait complete. 2024-09-12T03:18:50.794 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json session ls 2024-09-12T03:18:50.914 INFO:teuthology.orchestra.run.smithi027.stdout:[{"id":4955,"entity":{"name":{"type":"client","num":4955},"addr":{"type":"any","addr":"172.21.15.45:0","nonce":3904873653}},"state":"open","num_leases":0,"num_caps":7,"request_load_avg":7,"uptime":61.208442499,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":19.099221925090902,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.4955 172.21.15.45:0/3904873653","completed_requests":[{"tid":14,"created_ino":"0x0"},{"tid":15,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc","ceph_version":"ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)","entity_id":"x","hostname":"smithi045","pid":"36189","root":"/"}}] 2024-09-12T03:18:50.915 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc", "ceph_version": "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "x", "hostname": "smithi045", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "36189", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 14 }, { "created_ino": "0x100000001f9", "tid": 15 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.45:0", "nonce": 3904873653, "type": "any" }, "name": { "num": 4955, "type": "client" } }, "id": 4955, "inst": "client.4955 172.21.15.45:0/3904873653", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 0, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 7, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 19.0992219250909 }, "state": "open", "uptime": 61.208442499 } ] 2024-09-12T03:18:50.915 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2024-09-12T03:18:55.917 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-12T03:18:56.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:56.032+0000 7efdbf311640 1 Processor -- start 2024-09-12T03:18:56.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:56.032+0000 7efdbf311640 1 -- start start 2024-09-12T03:18:56.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:56.033+0000 7efdbf311640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7efdb806f360 0x7efdb80cef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:56.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:56.033+0000 7efdbf311640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7efdb80cf490 0x7efdb80d2780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:56.034 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:56.033+0000 7efdbf311640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efdb806cc40 con 0x7efdb80b1ca0 2024-09-12T03:18:56.034 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:56.033+0000 7efdbf311640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7efdb80d22f0 con 0x7efdb80cf490 2024-09-12T03:18:56.034 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:56.033+0000 7efdbf311640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7efdb80d1ce0 con 0x7efdb806f360 2024-09-12T03:18:56.034 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:56.033+0000 7efdbd0ad640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7efdb80b1ca0 0x7efdb815f830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:51602/0 (socket says 172.21.15.27:51602) 2024-09-12T03:18:56.034 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:56.033+0000 7efdbd0ad640 1 -- 172.21.15.27:0/1477281073 learned_addr learned my addr 172.21.15.27:0/1477281073 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:18:56.034 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:56.034+0000 7efdbd8ae640 1 --2- 172.21.15.27:0/1477281073 >> v2:172.21.15.45:3300/0 conn(0x7efdb80cf490 0x7efdb80d2780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:56.034 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:56.034+0000 7efdbc8ac640 1 --2- 172.21.15.27:0/1477281073 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7efdb806f360 0x7efdb80cef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:56.034 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:56.034+0000 7efdaffff640 1 -- 172.21.15.27:0/1477281073 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 977044023 0 0) 0x7efdb806cc40 con 0x7efdb80b1ca0 2024-09-12T03:18:56.034 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:56.034+0000 7efdaffff640 1 -- 172.21.15.27:0/1477281073 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efda4003520 con 0x7efdb80b1ca0 2024-09-12T03:18:56.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:56.034+0000 7efdbc8ac640 1 -- 172.21.15.27:0/1477281073 >> v2:172.21.15.45:3300/0 conn(0x7efdb80cf490 msgr2=0x7efdb80d2780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:18:56.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:56.034+0000 7efdaffff640 1 -- 172.21.15.27:0/1477281073 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2703992045 0 0) 0x7efda4003520 con 0x7efdb80b1ca0 2024-09-12T03:18:56.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:56.034+0000 7efdbc8ac640 1 --2- 172.21.15.27:0/1477281073 >> v2:172.21.15.45:3300/0 conn(0x7efdb80cf490 0x7efdb80d2780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:56.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:56.034+0000 7efdbc8ac640 1 -- 172.21.15.27:0/1477281073 >> v1:172.21.15.27:6789/0 conn(0x7efdb80b1ca0 legacy=0x7efdb815f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:56.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:56.034+0000 7efdbc8ac640 1 -- 172.21.15.27:0/1477281073 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdb806cee0 con 0x7efdb806f360 2024-09-12T03:18:56.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:56.035+0000 7efdbc8ac640 1 --2- 172.21.15.27:0/1477281073 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7efdb806f360 0x7efdb80cef50 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7efda80097a0 tx=0x7efda802a110 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f637679d860d5ab8 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:56.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:56.035+0000 7efdaffff640 1 -- 172.21.15.27:0/1477281073 <== mon.2 v2:172.21.15.45:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efda8037070 con 0x7efdb806f360 2024-09-12T03:18:56.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:56.035+0000 7efdaffff640 1 -- 172.21.15.27:0/1477281073 <== mon.2 v2:172.21.15.45:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efda802aba0 con 0x7efdb806f360 2024-09-12T03:18:56.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:56.035+0000 7efdaffff640 1 -- 172.21.15.27:0/1477281073 <== mon.2 v2:172.21.15.45:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdb4001670 con 0x7efdb806f360 2024-09-12T03:18:56.037 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:56.036+0000 7efdbf311640 1 -- 172.21.15.27:0/1477281073 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7efdb806f360 msgr2=0x7efdb80cef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:56.037 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:56.036+0000 7efdbf311640 1 --2- 172.21.15.27:0/1477281073 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7efdb806f360 0x7efdb80cef50 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7efda80097a0 tx=0x7efda802a110 comp rx=0 tx=0).stop 2024-09-12T03:18:56.037 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:56.036+0000 7efdbf311640 1 -- 172.21.15.27:0/1477281073 shutdown_connections 2024-09-12T03:18:56.037 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:56.036+0000 7efdbf311640 1 --2- 172.21.15.27:0/1477281073 >> v2:172.21.15.45:3300/0 conn(0x7efdb80cf490 0x7efdb80d2780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:56.037 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:56.036+0000 7efdbf311640 1 --2- 172.21.15.27:0/1477281073 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7efdb806f360 0x7efdb80cef50 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:56.037 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:56.036+0000 7efdbf311640 1 -- 172.21.15.27:0/1477281073 >> 172.21.15.27:0/1477281073 conn(0x7efdb8073800 msgr2=0x7efdb809ba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:18:56.037 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:56.036+0000 7efdbf311640 1 -- 172.21.15.27:0/1477281073 shutdown_connections 2024-09-12T03:18:56.037 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:56.036+0000 7efdbf311640 1 -- 172.21.15.27:0/1477281073 wait complete. 2024-09-12T03:18:56.037 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:56.037+0000 7efdbf311640 1 Processor -- start 2024-09-12T03:18:56.037 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:56.037+0000 7efdbf311640 1 -- start start 2024-09-12T03:18:56.039 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:56.038+0000 7efdbf311640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efdb806f360 0x7efdb815cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:56.039 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:56.038+0000 7efdbf311640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7efdb80b1ca0 0x7efdb8147e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:56.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:56.038+0000 7efdbf311640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efdb80cf490 0x7efdb8160ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:56.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:56.038+0000 7efdbf311640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7efdb806d340 con 0x7efdb806f360 2024-09-12T03:18:56.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:56.038+0000 7efdbf311640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7efdb806d1c0 con 0x7efdb80cf490 2024-09-12T03:18:56.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:56.038+0000 7efdbf311640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7efdb80cfc40 con 0x7efdb80b1ca0 2024-09-12T03:18:56.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:56.038+0000 7efdbd0ad640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efdb806f360 0x7efdb815cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:56.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:56.038+0000 7efdbd0ad640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efdb806f360 0x7efdb815cf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:53466/0 (socket says 172.21.15.27:53466) 2024-09-12T03:18:56.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:56.038+0000 7efdbd0ad640 1 -- 172.21.15.27:0/1198972100 learned_addr learned my addr 172.21.15.27:0/1198972100 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:18:56.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:56.038+0000 7efdbc8ac640 1 --2- 172.21.15.27:0/1198972100 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7efdb80b1ca0 0x7efdb8147e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:56.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:56.038+0000 7efdbd8ae640 1 --2- 172.21.15.27:0/1198972100 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efdb80cf490 0x7efdb8160ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:56.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:56.038+0000 7efdbd0ad640 1 -- 172.21.15.27:0/1198972100 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7efdb80b1ca0 msgr2=0x7efdb8147e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:56.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:56.038+0000 7efdbd0ad640 1 --2- 172.21.15.27:0/1198972100 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7efdb80b1ca0 0x7efdb8147e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:56.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:56.038+0000 7efdbd0ad640 1 -- 172.21.15.27:0/1198972100 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efdb80cf490 msgr2=0x7efdb8160ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:56.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:56.038+0000 7efdbd0ad640 1 --2- 172.21.15.27:0/1198972100 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efdb80cf490 0x7efdb8160ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:56.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:56.038+0000 7efdbd0ad640 1 -- 172.21.15.27:0/1198972100 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdb8155640 con 0x7efdb806f360 2024-09-12T03:18:56.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:56.038+0000 7efdbd0ad640 1 --2- 172.21.15.27:0/1198972100 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efdb806f360 0x7efdb815cf70 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7efdb400c700 tx=0x7efdb400cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:56.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:56.039+0000 7efdadffb640 1 -- 172.21.15.27:0/1198972100 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdb4017070 con 0x7efdb806f360 2024-09-12T03:18:56.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:56.039+0000 7efdadffb640 1 -- 172.21.15.27:0/1198972100 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efdb4005ad0 con 0x7efdb806f360 2024-09-12T03:18:56.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:56.039+0000 7efdadffb640 1 -- 172.21.15.27:0/1198972100 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdb4007070 con 0x7efdb806f360 2024-09-12T03:18:56.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:56.039+0000 7efdbf311640 1 -- 172.21.15.27:0/1198972100 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efdb81f7c80 con 0x7efdb806f360 2024-09-12T03:18:56.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:56.039+0000 7efdbf311640 1 -- 172.21.15.27:0/1198972100 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efdb81f8200 con 0x7efdb806f360 2024-09-12T03:18:56.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:56.040+0000 7efdbf311640 1 -- 172.21.15.27:0/1198972100 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efdb806f7a0 con 0x7efdb806f360 2024-09-12T03:18:56.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:56.040+0000 7efdadffb640 1 -- 172.21.15.27:0/1198972100 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 8) ==== 90702+0+0 (secure 0 0 0) 0x7efdb401a070 con 0x7efdb806f360 2024-09-12T03:18:56.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:56.041+0000 7efdadffb640 1 --2- 172.21.15.27:0/1198972100 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7efd9c06c070 0x7efd9c06e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:56.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:56.041+0000 7efdadffb640 1 -- 172.21.15.27:0/1198972100 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5767+0+0 (secure 0 0 0) 0x7efdb408a6b0 con 0x7efdb806f360 2024-09-12T03:18:56.043 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:56.041+0000 7efdbc8ac640 1 --2- 172.21.15.27:0/1198972100 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7efd9c06c070 0x7efd9c06e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:56.043 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:56.042+0000 7efdbc8ac640 1 --2- 172.21.15.27:0/1198972100 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7efd9c06c070 0x7efd9c06e530 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7efda8034000 tx=0x7efda802e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:56.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:56.046+0000 7efdadffb640 1 -- 172.21.15.27:0/1198972100 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7efdb4058670 con 0x7efdb806f360 2024-09-12T03:18:56.259 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:56.259+0000 7efdbf311640 1 -- 172.21.15.27:0/1198972100 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efdb815bf80 con 0x7efdb806f360 2024-09-12T03:18:56.259 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:56.259+0000 7efdadffb640 1 -- 172.21.15.27:0/1198972100 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+5223 (secure 0 0 0) 0x7efdb405c970 con 0x7efdb806f360 2024-09-12T03:18:56.260 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:18:56.260 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":23,"btime":"2024-09-12T03:17:47:237558+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4835,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6834/1457797481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":1457797481},{"type":"v1","addr":"172.21.15.27:6834","nonce":1457797481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5029,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6836/2842491281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":2842491281},{"type":"v1","addr":"172.21.15.45:6836","nonce":2842491281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5032,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/3339744340","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6835","nonce":3339744340},{"type":"v1","addr":"172.21.15.45:6837","nonce":3339744340}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-12T03:17:38.580473+0000","modified":"2024-09-12T03:17:40.557950+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5205},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5205":{"gid":5205,"name":"a","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.27:6835/2571101228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":2571101228},{"type":"v1","addr":"172.21.15.27:6835","nonce":2571101228}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5205,"qdb_cluster":[5205]},"id":3}]} 2024-09-12T03:18:56.260 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 23 2024-09-12T03:18:56.262 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:56.261+0000 7efdbf311640 1 -- 172.21.15.27:0/1198972100 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7efd9c06c070 msgr2=0x7efd9c06e530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:56.262 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:56.261+0000 7efdbf311640 1 --2- 172.21.15.27:0/1198972100 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7efd9c06c070 0x7efd9c06e530 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7efda8034000 tx=0x7efda802e000 comp rx=0 tx=0).stop 2024-09-12T03:18:56.262 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:56.261+0000 7efdbf311640 1 -- 172.21.15.27:0/1198972100 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efdb806f360 msgr2=0x7efdb815cf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:56.262 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:56.261+0000 7efdbf311640 1 --2- 172.21.15.27:0/1198972100 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efdb806f360 0x7efdb815cf70 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7efdb400c700 tx=0x7efdb400cbc0 comp rx=0 tx=0).stop 2024-09-12T03:18:56.262 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:56.262+0000 7efdbf311640 1 -- 172.21.15.27:0/1198972100 shutdown_connections 2024-09-12T03:18:56.262 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:56.262+0000 7efdbf311640 1 --2- 172.21.15.27:0/1198972100 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7efd9c06c070 0x7efd9c06e530 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:56.262 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:56.262+0000 7efdbf311640 1 --2- 172.21.15.27:0/1198972100 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efdb80cf490 0x7efdb8160ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:56.262 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:56.262+0000 7efdbf311640 1 --2- 172.21.15.27:0/1198972100 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7efdb80b1ca0 0x7efdb8147e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:56.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:56.262+0000 7efdbf311640 1 --2- 172.21.15.27:0/1198972100 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efdb806f360 0x7efdb815cf70 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:56.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:56.262+0000 7efdbf311640 1 -- 172.21.15.27:0/1198972100 >> 172.21.15.27:0/1198972100 conn(0x7efdb8073800 msgr2=0x7efdb8161a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:18:56.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:56.262+0000 7efdbf311640 1 -- 172.21.15.27:0/1198972100 shutdown_connections 2024-09-12T03:18:56.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:18:56.262+0000 7efdbf311640 1 -- 172.21.15.27:0/1198972100 wait complete. 2024-09-12T03:18:56.273 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json session ls 2024-09-12T03:18:56.393 INFO:teuthology.orchestra.run.smithi027.stdout:[{"id":4955,"entity":{"name":{"type":"client","num":4955},"addr":{"type":"any","addr":"172.21.15.45:0","nonce":3904873653}},"state":"open","num_leases":0,"num_caps":7,"request_load_avg":6,"uptime":66.688303728999998,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":18.85892788171704,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.4955 172.21.15.45:0/3904873653","completed_requests":[{"tid":14,"created_ino":"0x0"},{"tid":15,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc","ceph_version":"ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)","entity_id":"x","hostname":"smithi045","pid":"36189","root":"/"}}] 2024-09-12T03:18:56.394 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc", "ceph_version": "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "x", "hostname": "smithi045", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "36189", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 14 }, { "created_ino": "0x100000001f9", "tid": 15 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.45:0", "nonce": 3904873653, "type": "any" }, "name": { "num": 4955, "type": "client" } }, "id": 4955, "inst": "client.4955 172.21.15.45:0/3904873653", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 0, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 6, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 18.85892788171704 }, "state": "open", "uptime": 66.688303729 } ] 2024-09-12T03:18:56.394 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2024-09-12T03:19:00.742 DEBUG:teuthology.orchestra.run.smithi027:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-12T03:19:00.746 DEBUG:teuthology.orchestra.run.smithi045:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-12T03:19:01.395 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-12T03:19:01.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:01.511+0000 7f3b15c77640 1 Processor -- start 2024-09-12T03:19:01.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:01.511+0000 7f3b15c77640 1 -- start start 2024-09-12T03:19:01.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:01.512+0000 7f3b15c77640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f3b100d5200 0x7f3b100d1010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:01.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:01.512+0000 7f3b15c77640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3b100d3760 con 0x7f3b100d1550 2024-09-12T03:19:01.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:01.512+0000 7f3b15c77640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3b100d3a40 con 0x7f3b1015eb90 2024-09-12T03:19:01.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:01.512+0000 7f3b15c77640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f3b100b7d20 con 0x7f3b100d5200 2024-09-12T03:19:01.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:01.512+0000 7f3b0effd640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f3b100d5200 0x7f3b100d1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:01.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:01.512+0000 7f3b0ffff640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f3b100d1550 0x7f3b100b8190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:52586/0 (socket says 172.21.15.27:52586) 2024-09-12T03:19:01.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:01.512+0000 7f3b0ffff640 1 -- 172.21.15.27:0/1887947661 learned_addr learned my addr 172.21.15.27:0/1887947661 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:19:01.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:01.512+0000 7f3b0effd640 1 -- 172.21.15.27:0/1887947661 >> v1:172.21.15.45:6789/0 conn(0x7f3b1015eb90 legacy=0x7f3b100d2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:01.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:01.512+0000 7f3b0effd640 1 -- 172.21.15.27:0/1887947661 >> v1:172.21.15.27:6789/0 conn(0x7f3b100d1550 legacy=0x7f3b100b8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:01.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:01.512+0000 7f3b0e7fc640 1 -- 172.21.15.27:0/1887947661 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2368517824 0 0) 0x7f3b100d3760 con 0x7f3b100d1550 2024-09-12T03:19:01.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:01.512+0000 7f3b0effd640 1 -- 172.21.15.27:0/1887947661 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b100d3d20 con 0x7f3b100d5200 2024-09-12T03:19:01.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:01.513+0000 7f3b0effd640 1 --2- 172.21.15.27:0/1887947661 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3b100d5200 0x7f3b100d1010 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f3b040097a0 tx=0x7f3b0402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3dc462ac59438e05 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:01.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:01.513+0000 7f3b0e7fc640 1 -- 172.21.15.27:0/1887947661 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b04037070 con 0x7f3b100d5200 2024-09-12T03:19:01.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:01.513+0000 7f3b0e7fc640 1 -- 172.21.15.27:0/1887947661 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b04032440 con 0x7f3b100d5200 2024-09-12T03:19:01.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:01.513+0000 7f3b0e7fc640 1 -- 172.21.15.27:0/1887947661 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b04032810 con 0x7f3b100d5200 2024-09-12T03:19:01.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:01.514+0000 7f3b15c77640 1 -- 172.21.15.27:0/1887947661 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3b100d5200 msgr2=0x7f3b100d1010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:01.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:01.514+0000 7f3b15c77640 1 --2- 172.21.15.27:0/1887947661 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3b100d5200 0x7f3b100d1010 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f3b040097a0 tx=0x7f3b0402a230 comp rx=0 tx=0).stop 2024-09-12T03:19:01.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:01.514+0000 7f3b15c77640 1 -- 172.21.15.27:0/1887947661 shutdown_connections 2024-09-12T03:19:01.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:01.514+0000 7f3b15c77640 1 --2- 172.21.15.27:0/1887947661 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3b100d5200 0x7f3b100d1010 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:01.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:01.514+0000 7f3b15c77640 1 -- 172.21.15.27:0/1887947661 >> 172.21.15.27:0/1887947661 conn(0x7f3b1006dce0 msgr2=0x7f3b1006e0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:01.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:01.514+0000 7f3b15c77640 1 -- 172.21.15.27:0/1887947661 shutdown_connections 2024-09-12T03:19:01.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:01.514+0000 7f3b15c77640 1 -- 172.21.15.27:0/1887947661 wait complete. 2024-09-12T03:19:01.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:01.515+0000 7f3b15c77640 1 Processor -- start 2024-09-12T03:19:01.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:01.515+0000 7f3b15c77640 1 -- start start 2024-09-12T03:19:01.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:01.516+0000 7f3b15c77640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3b100d1550 0x7f3b1014daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:01.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:01.516+0000 7f3b15c77640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f3b100d5200 0x7f3b100ce0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:01.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:01.516+0000 7f3b15c77640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3b1015eb90 0x7f3b1015afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:01.519 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:01.516+0000 7f3b15c77640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f3b100d3500 con 0x7f3b100d1550 2024-09-12T03:19:01.519 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:01.516+0000 7f3b15c77640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f3b100d42e0 con 0x7f3b1015eb90 2024-09-12T03:19:01.519 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:01.516+0000 7f3b15c77640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f3b100d1960 con 0x7f3b100d5200 2024-09-12T03:19:01.519 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:01.516+0000 7f3b0f7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3b100d1550 0x7f3b1014daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:01.519 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:01.516+0000 7f3b0f7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3b100d1550 0x7f3b1014daf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45638/0 (socket says 172.21.15.27:45638) 2024-09-12T03:19:01.519 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:01.516+0000 7f3b0f7fe640 1 -- 172.21.15.27:0/1426363145 learned_addr learned my addr 172.21.15.27:0/1426363145 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:19:01.519 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:01.516+0000 7f3b0effd640 1 --2- 172.21.15.27:0/1426363145 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f3b100d5200 0x7f3b100ce0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:01.519 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:01.516+0000 7f3b0ffff640 1 --2- 172.21.15.27:0/1426363145 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3b1015eb90 0x7f3b1015afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:01.519 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:01.517+0000 7f3b0f7fe640 1 -- 172.21.15.27:0/1426363145 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f3b100d5200 msgr2=0x7f3b100ce0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:01.519 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:01.517+0000 7f3b0f7fe640 1 --2- 172.21.15.27:0/1426363145 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f3b100d5200 0x7f3b100ce0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:01.519 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:01.517+0000 7f3b0f7fe640 1 -- 172.21.15.27:0/1426363145 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3b1015eb90 msgr2=0x7f3b1015afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:01.519 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:01.517+0000 7f3b0f7fe640 1 --2- 172.21.15.27:0/1426363145 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3b1015eb90 0x7f3b1015afa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:01.519 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:01.517+0000 7f3b0f7fe640 1 -- 172.21.15.27:0/1426363145 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b101f59d0 con 0x7f3b100d1550 2024-09-12T03:19:01.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:01.517+0000 7f3b0f7fe640 1 --2- 172.21.15.27:0/1426363145 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3b100d1550 0x7f3b1014daf0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f3b0000c450 tx=0x7f3b0000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:01.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:01.517+0000 7f3af7fff640 1 -- 172.21.15.27:0/1426363145 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b000052d0 con 0x7f3b100d1550 2024-09-12T03:19:01.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:01.517+0000 7f3af7fff640 1 -- 172.21.15.27:0/1426363145 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b00005470 con 0x7f3b100d1550 2024-09-12T03:19:01.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:01.517+0000 7f3b15c77640 1 -- 172.21.15.27:0/1426363145 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b101f5cb0 con 0x7f3b100d1550 2024-09-12T03:19:01.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:01.517+0000 7f3af7fff640 1 -- 172.21.15.27:0/1426363145 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b00012870 con 0x7f3b100d1550 2024-09-12T03:19:01.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:01.517+0000 7f3b15c77640 1 -- 172.21.15.27:0/1426363145 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b1014e030 con 0x7f3b100d1550 2024-09-12T03:19:01.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:01.518+0000 7f3b15c77640 1 -- 172.21.15.27:0/1426363145 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b100693d0 con 0x7f3b100d1550 2024-09-12T03:19:01.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:01.519+0000 7f3af7fff640 1 -- 172.21.15.27:0/1426363145 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 8) ==== 90702+0+0 (secure 0 0 0) 0x7f3b00012a10 con 0x7f3b100d1550 2024-09-12T03:19:01.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:01.519+0000 7f3af7fff640 1 --2- 172.21.15.27:0/1426363145 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f3afc06c070 0x7f3afc06e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:01.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:01.519+0000 7f3af7fff640 1 -- 172.21.15.27:0/1426363145 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5767+0+0 (secure 0 0 0) 0x7f3b0008ad90 con 0x7f3b100d1550 2024-09-12T03:19:01.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:01.520+0000 7f3b0effd640 1 --2- 172.21.15.27:0/1426363145 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f3afc06c070 0x7f3afc06e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:01.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:01.520+0000 7f3b0effd640 1 --2- 172.21.15.27:0/1426363145 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f3afc06c070 0x7f3afc06e530 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f3b04034000 tx=0x7f3b0402e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:01.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:01.524+0000 7f3af7fff640 1 -- 172.21.15.27:0/1426363145 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f3b00058d50 con 0x7f3b100d1550 2024-09-12T03:19:01.738 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:01.738+0000 7f3b15c77640 1 -- 172.21.15.27:0/1426363145 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3b1015f120 con 0x7f3b100d1550 2024-09-12T03:19:01.738 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:01.738+0000 7f3af7fff640 1 -- 172.21.15.27:0/1426363145 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+5223 (secure 0 0 0) 0x7f3b0005d050 con 0x7f3b100d1550 2024-09-12T03:19:01.739 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:19:01.739 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":23,"btime":"2024-09-12T03:17:47:237558+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4835,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6834/1457797481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":1457797481},{"type":"v1","addr":"172.21.15.27:6834","nonce":1457797481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5029,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6836/2842491281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":2842491281},{"type":"v1","addr":"172.21.15.45:6836","nonce":2842491281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5032,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/3339744340","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6835","nonce":3339744340},{"type":"v1","addr":"172.21.15.45:6837","nonce":3339744340}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-12T03:17:38.580473+0000","modified":"2024-09-12T03:17:40.557950+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5205},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5205":{"gid":5205,"name":"a","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.27:6835/2571101228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":2571101228},{"type":"v1","addr":"172.21.15.27:6835","nonce":2571101228}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5205,"qdb_cluster":[5205]},"id":3}]} 2024-09-12T03:19:01.739 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 23 2024-09-12T03:19:01.740 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:01.740+0000 7f3b15c77640 1 -- 172.21.15.27:0/1426363145 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f3afc06c070 msgr2=0x7f3afc06e530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:01.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:01.740+0000 7f3b15c77640 1 --2- 172.21.15.27:0/1426363145 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f3afc06c070 0x7f3afc06e530 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f3b04034000 tx=0x7f3b0402e000 comp rx=0 tx=0).stop 2024-09-12T03:19:01.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:01.740+0000 7f3b15c77640 1 -- 172.21.15.27:0/1426363145 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3b100d1550 msgr2=0x7f3b1014daf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:01.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:01.740+0000 7f3b15c77640 1 --2- 172.21.15.27:0/1426363145 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3b100d1550 0x7f3b1014daf0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f3b0000c450 tx=0x7f3b0000c910 comp rx=0 tx=0).stop 2024-09-12T03:19:01.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:01.740+0000 7f3b15c77640 1 -- 172.21.15.27:0/1426363145 shutdown_connections 2024-09-12T03:19:01.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:01.740+0000 7f3b15c77640 1 --2- 172.21.15.27:0/1426363145 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f3afc06c070 0x7f3afc06e530 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:01.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:01.740+0000 7f3b15c77640 1 --2- 172.21.15.27:0/1426363145 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3b1015eb90 0x7f3b1015afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:01.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:01.740+0000 7f3b15c77640 1 --2- 172.21.15.27:0/1426363145 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f3b100d5200 0x7f3b100ce0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:01.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:01.740+0000 7f3b15c77640 1 --2- 172.21.15.27:0/1426363145 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3b100d1550 0x7f3b1014daf0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:01.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:01.740+0000 7f3b15c77640 1 -- 172.21.15.27:0/1426363145 >> 172.21.15.27:0/1426363145 conn(0x7f3b1006dce0 msgr2=0x7f3b10157940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:01.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:01.740+0000 7f3b15c77640 1 -- 172.21.15.27:0/1426363145 shutdown_connections 2024-09-12T03:19:01.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:01.740+0000 7f3b15c77640 1 -- 172.21.15.27:0/1426363145 wait complete. 2024-09-12T03:19:01.752 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json session ls 2024-09-12T03:19:01.873 INFO:teuthology.orchestra.run.smithi027.stdout:[{"id":4955,"entity":{"name":{"type":"client","num":4955},"addr":{"type":"any","addr":"172.21.15.45:0","nonce":3904873653}},"state":"open","num_leases":0,"num_caps":7,"request_load_avg":6,"uptime":72.168164958999995,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":18.621657062404743,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.4955 172.21.15.45:0/3904873653","completed_requests":[{"tid":14,"created_ino":"0x0"},{"tid":15,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc","ceph_version":"ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)","entity_id":"x","hostname":"smithi045","pid":"36189","root":"/"}}] 2024-09-12T03:19:01.873 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc", "ceph_version": "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "x", "hostname": "smithi045", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "36189", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 14 }, { "created_ino": "0x100000001f9", "tid": 15 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.45:0", "nonce": 3904873653, "type": "any" }, "name": { "num": 4955, "type": "client" } }, "id": 4955, "inst": "client.4955 172.21.15.45:0/3904873653", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 0, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 6, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 18.621657062404743 }, "state": "open", "uptime": 72.168164959 } ] 2024-09-12T03:19:01.874 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2024-09-12T03:19:06.875 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-12T03:19:06.991 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:06.991+0000 7f4c4812c640 1 Processor -- start 2024-09-12T03:19:06.991 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:06.991+0000 7f4c4812c640 1 -- start start 2024-09-12T03:19:06.992 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:06.992+0000 7f4c4812c640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f4c4015eb90 0x7f4c400d2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:06.992 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:06.992+0000 7f4c4812c640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4c400d5200 0x7f4c400d1010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:06.992 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:06.992+0000 7f4c4812c640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f4c400d1550 0x7f4c400cf200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:06.992 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:06.992+0000 7f4c4812c640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f4c400b7d20 con 0x7f4c4015eb90 2024-09-12T03:19:06.992 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:06.992+0000 7f4c4812c640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f4c400d42e0 con 0x7f4c400d1550 2024-09-12T03:19:06.993 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:06.992+0000 7f4c4812c640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f4c400d19c0 con 0x7f4c400d5200 2024-09-12T03:19:06.993 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:06.992+0000 7f4c45ec8640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f4c4015eb90 0x7f4c400d2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:06.993 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:06.992+0000 7f4c45ec8640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f4c4015eb90 0x7f4c400d2df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45646/0 (socket says 172.21.15.27:45646) 2024-09-12T03:19:06.993 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:06.992+0000 7f4c45ec8640 1 -- 172.21.15.27:0/1105848352 learned_addr learned my addr 172.21.15.27:0/1105848352 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:19:06.993 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:06.992+0000 7f4c45ec8640 1 -- 172.21.15.27:0/1105848352 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4c400d5200 msgr2=0x7f4c400d1010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:06.993 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:06.992+0000 7f4c466c9640 1 --2- 172.21.15.27:0/1105848352 >> v2:172.21.15.45:3300/0 conn(0x7f4c400d1550 0x7f4c400cf200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:06.993 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:06.992+0000 7f4c456c7640 1 --2- 172.21.15.27:0/1105848352 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4c400d5200 0x7f4c400d1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:06.993 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:06.992+0000 7f4c45ec8640 1 --2- 172.21.15.27:0/1105848352 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4c400d5200 0x7f4c400d1010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:06.993 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:06.992+0000 7f4c45ec8640 1 -- 172.21.15.27:0/1105848352 >> v2:172.21.15.45:3300/0 conn(0x7f4c400d1550 msgr2=0x7f4c400cf200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:06.993 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:06.992+0000 7f4c45ec8640 1 --2- 172.21.15.27:0/1105848352 >> v2:172.21.15.45:3300/0 conn(0x7f4c400d1550 0x7f4c400cf200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:06.993 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:06.992+0000 7f4c45ec8640 1 -- 172.21.15.27:0/1105848352 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c400d3760 con 0x7f4c4015eb90 2024-09-12T03:19:06.994 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:06.993+0000 7f4c45ec8640 1 --2- 172.21.15.27:0/1105848352 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4c4015eb90 0x7f4c400d2df0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f4c4008d510 tx=0x7f4c3c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77f6bfa9f1692870 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:06.994 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:06.993+0000 7f4c44ec6640 1 -- 172.21.15.27:0/1105848352 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c3c037070 con 0x7f4c4015eb90 2024-09-12T03:19:06.994 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:06.993+0000 7f4c44ec6640 1 -- 172.21.15.27:0/1105848352 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c3c032440 con 0x7f4c4015eb90 2024-09-12T03:19:06.995 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:06.993+0000 7f4c44ec6640 1 -- 172.21.15.27:0/1105848352 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c3c032810 con 0x7f4c4015eb90 2024-09-12T03:19:06.995 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:06.994+0000 7f4c4812c640 1 -- 172.21.15.27:0/1105848352 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4c4015eb90 msgr2=0x7f4c400d2df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:06.995 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:06.994+0000 7f4c4812c640 1 --2- 172.21.15.27:0/1105848352 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4c4015eb90 0x7f4c400d2df0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f4c4008d510 tx=0x7f4c3c02a1e0 comp rx=0 tx=0).stop 2024-09-12T03:19:06.995 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:06.994+0000 7f4c4812c640 1 -- 172.21.15.27:0/1105848352 shutdown_connections 2024-09-12T03:19:06.995 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:06.994+0000 7f4c4812c640 1 --2- 172.21.15.27:0/1105848352 >> v2:172.21.15.45:3300/0 conn(0x7f4c400d1550 0x7f4c400cf200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:06.995 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:06.994+0000 7f4c4812c640 1 --2- 172.21.15.27:0/1105848352 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4c400d5200 0x7f4c400d1010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:06.995 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:06.994+0000 7f4c4812c640 1 --2- 172.21.15.27:0/1105848352 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4c4015eb90 0x7f4c400d2df0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:06.995 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:06.994+0000 7f4c4812c640 1 -- 172.21.15.27:0/1105848352 >> 172.21.15.27:0/1105848352 conn(0x7f4c4006dce0 msgr2=0x7f4c4006e0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:06.995 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:06.994+0000 7f4c4812c640 1 -- 172.21.15.27:0/1105848352 shutdown_connections 2024-09-12T03:19:06.995 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:06.994+0000 7f4c4812c640 1 -- 172.21.15.27:0/1105848352 wait complete. 2024-09-12T03:19:06.997 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:06.995+0000 7f4c4812c640 1 Processor -- start 2024-09-12T03:19:06.997 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:06.995+0000 7f4c4812c640 1 -- start start 2024-09-12T03:19:06.998 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:06.996+0000 7f4c4812c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4c400d1550 0x7f4c401f4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:06.998 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:06.996+0000 7f4c4812c640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4c400d5200 0x7f4c40155600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:06.998 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:06.996+0000 7f4c4812c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4c4015eb90 0x7f4c401f17d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:06.998 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:06.996+0000 7f4c4812c640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f4c400d4000 con 0x7f4c400d1550 2024-09-12T03:19:06.998 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:06.996+0000 7f4c4812c640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f4c400d3d20 con 0x7f4c4015eb90 2024-09-12T03:19:06.998 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:06.996+0000 7f4c4812c640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f4c400d3330 con 0x7f4c400d5200 2024-09-12T03:19:06.998 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:06.996+0000 7f4c45ec8640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4c400d1550 0x7f4c401f4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:06.998 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:06.996+0000 7f4c45ec8640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4c400d1550 0x7f4c401f4ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45656/0 (socket says 172.21.15.27:45656) 2024-09-12T03:19:06.998 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:06.996+0000 7f4c45ec8640 1 -- 172.21.15.27:0/1194432971 learned_addr learned my addr 172.21.15.27:0/1194432971 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:19:06.998 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:06.996+0000 7f4c456c7640 1 --2- 172.21.15.27:0/1194432971 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4c400d5200 0x7f4c40155600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:06.998 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:06.996+0000 7f4c466c9640 1 --2- 172.21.15.27:0/1194432971 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4c4015eb90 0x7f4c401f17d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:06.998 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:06.996+0000 7f4c45ec8640 1 -- 172.21.15.27:0/1194432971 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4c400d5200 msgr2=0x7f4c40155600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:06.998 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:06.996+0000 7f4c45ec8640 1 --2- 172.21.15.27:0/1194432971 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4c400d5200 0x7f4c40155600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:06.999 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:06.996+0000 7f4c45ec8640 1 -- 172.21.15.27:0/1194432971 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4c4015eb90 msgr2=0x7f4c401f17d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:06.999 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:06.996+0000 7f4c45ec8640 1 --2- 172.21.15.27:0/1194432971 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4c4015eb90 0x7f4c401f17d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:06.999 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:06.996+0000 7f4c45ec8640 1 -- 172.21.15.27:0/1194432971 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c40155f10 con 0x7f4c400d1550 2024-09-12T03:19:06.999 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:06.996+0000 7f4c45ec8640 1 --2- 172.21.15.27:0/1194432971 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4c400d1550 0x7f4c401f4ff0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f4c3c02a220 tx=0x7f4c3c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:06.999 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:06.997+0000 7f4c327fc640 1 -- 172.21.15.27:0/1194432971 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c3c037040 con 0x7f4c400d1550 2024-09-12T03:19:06.999 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:06.997+0000 7f4c327fc640 1 -- 172.21.15.27:0/1194432971 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c3c042070 con 0x7f4c400d1550 2024-09-12T03:19:06.999 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:06.997+0000 7f4c4812c640 1 -- 172.21.15.27:0/1194432971 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c401564d0 con 0x7f4c400d1550 2024-09-12T03:19:06.999 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:06.997+0000 7f4c327fc640 1 -- 172.21.15.27:0/1194432971 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c3c030bb0 con 0x7f4c400d1550 2024-09-12T03:19:06.999 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:06.997+0000 7f4c4812c640 1 -- 172.21.15.27:0/1194432971 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c40156a90 con 0x7f4c400d1550 2024-09-12T03:19:06.999 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:06.998+0000 7f4c4812c640 1 -- 172.21.15.27:0/1194432971 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c401594e0 con 0x7f4c400d1550 2024-09-12T03:19:07.000 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:06.998+0000 7f4c327fc640 1 -- 172.21.15.27:0/1194432971 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 8) ==== 90702+0+0 (secure 0 0 0) 0x7f4c3c047460 con 0x7f4c400d1550 2024-09-12T03:19:07.000 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:06.999+0000 7f4c327fc640 1 --2- 172.21.15.27:0/1194432971 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f4c2406c070 0x7f4c2406e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:07.000 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:06.999+0000 7f4c327fc640 1 -- 172.21.15.27:0/1194432971 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5767+0+0 (secure 0 0 0) 0x7f4c3c0abde0 con 0x7f4c400d1550 2024-09-12T03:19:07.000 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:06.999+0000 7f4c456c7640 1 --2- 172.21.15.27:0/1194432971 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f4c2406c070 0x7f4c2406e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:07.000 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:06.999+0000 7f4c456c7640 1 --2- 172.21.15.27:0/1194432971 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f4c2406c070 0x7f4c2406e530 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f4c40161690 tx=0x7f4c34008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:07.003 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:07.003+0000 7f4c327fc640 1 -- 172.21.15.27:0/1194432971 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f4c3c079da0 con 0x7f4c400d1550 2024-09-12T03:19:07.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:07.217+0000 7f4c4812c640 1 -- 172.21.15.27:0/1194432971 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4c401600f0 con 0x7f4c400d1550 2024-09-12T03:19:07.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:07.217+0000 7f4c327fc640 1 -- 172.21.15.27:0/1194432971 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+5223 (secure 0 0 0) 0x7f4c3c07e0a0 con 0x7f4c400d1550 2024-09-12T03:19:07.217 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:19:07.218 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":23,"btime":"2024-09-12T03:17:47:237558+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4835,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6834/1457797481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":1457797481},{"type":"v1","addr":"172.21.15.27:6834","nonce":1457797481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5029,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6836/2842491281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":2842491281},{"type":"v1","addr":"172.21.15.45:6836","nonce":2842491281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5032,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/3339744340","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6835","nonce":3339744340},{"type":"v1","addr":"172.21.15.45:6837","nonce":3339744340}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-12T03:17:38.580473+0000","modified":"2024-09-12T03:17:40.557950+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5205},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5205":{"gid":5205,"name":"a","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.27:6835/2571101228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":2571101228},{"type":"v1","addr":"172.21.15.27:6835","nonce":2571101228}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5205,"qdb_cluster":[5205]},"id":3}]} 2024-09-12T03:19:07.218 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 23 2024-09-12T03:19:07.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:07.219+0000 7f4c4812c640 1 -- 172.21.15.27:0/1194432971 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f4c2406c070 msgr2=0x7f4c2406e530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:07.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:07.219+0000 7f4c4812c640 1 --2- 172.21.15.27:0/1194432971 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f4c2406c070 0x7f4c2406e530 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f4c40161690 tx=0x7f4c34008040 comp rx=0 tx=0).stop 2024-09-12T03:19:07.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:07.219+0000 7f4c4812c640 1 -- 172.21.15.27:0/1194432971 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4c400d1550 msgr2=0x7f4c401f4ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:07.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:07.219+0000 7f4c4812c640 1 --2- 172.21.15.27:0/1194432971 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4c400d1550 0x7f4c401f4ff0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f4c3c02a220 tx=0x7f4c3c0046c0 comp rx=0 tx=0).stop 2024-09-12T03:19:07.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:07.219+0000 7f4c4812c640 1 -- 172.21.15.27:0/1194432971 shutdown_connections 2024-09-12T03:19:07.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:07.219+0000 7f4c4812c640 1 --2- 172.21.15.27:0/1194432971 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f4c2406c070 0x7f4c2406e530 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:07.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:07.219+0000 7f4c4812c640 1 --2- 172.21.15.27:0/1194432971 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4c4015eb90 0x7f4c401f17d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:07.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:07.219+0000 7f4c4812c640 1 --2- 172.21.15.27:0/1194432971 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4c400d5200 0x7f4c40155600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:07.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:07.219+0000 7f4c4812c640 1 --2- 172.21.15.27:0/1194432971 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4c400d1550 0x7f4c401f4ff0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:07.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:07.219+0000 7f4c4812c640 1 -- 172.21.15.27:0/1194432971 >> 172.21.15.27:0/1194432971 conn(0x7f4c4006dce0 msgr2=0x7f4c400cfcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:07.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:07.219+0000 7f4c4812c640 1 -- 172.21.15.27:0/1194432971 shutdown_connections 2024-09-12T03:19:07.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:07.219+0000 7f4c4812c640 1 -- 172.21.15.27:0/1194432971 wait complete. 2024-09-12T03:19:07.230 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json session ls 2024-09-12T03:19:07.351 INFO:teuthology.orchestra.run.smithi027.stdout:[{"id":4955,"entity":{"name":{"type":"client","num":4955},"addr":{"type":"any","addr":"172.21.15.45:0","nonce":3904873653}},"state":"open","num_leases":0,"num_caps":7,"request_load_avg":6,"uptime":77.646026238999994,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":18.387456396676303,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.4955 172.21.15.45:0/3904873653","completed_requests":[{"tid":14,"created_ino":"0x0"},{"tid":15,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc","ceph_version":"ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)","entity_id":"x","hostname":"smithi045","pid":"36189","root":"/"}}] 2024-09-12T03:19:07.351 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc", "ceph_version": "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "x", "hostname": "smithi045", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "36189", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 14 }, { "created_ino": "0x100000001f9", "tid": 15 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.45:0", "nonce": 3904873653, "type": "any" }, "name": { "num": 4955, "type": "client" } }, "id": 4955, "inst": "client.4955 172.21.15.45:0/3904873653", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 0, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 6, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 18.387456396676303 }, "state": "open", "uptime": 77.646026239 } ] 2024-09-12T03:19:07.351 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2024-09-12T03:19:12.353 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-12T03:19:12.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:12.464+0000 7efe214af640 1 Processor -- start 2024-09-12T03:19:12.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:12.464+0000 7efe214af640 1 -- start start 2024-09-12T03:19:12.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:12.465+0000 7efe214af640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7efe1c0d5200 0x7efe1c0d1010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:12.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:12.465+0000 7efe214af640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efe1c0d19c0 con 0x7efe1c15eb90 2024-09-12T03:19:12.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:12.465+0000 7efe214af640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efe1c0d3760 con 0x7efe1c0d1550 2024-09-12T03:19:12.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:12.465+0000 7efe214af640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7efe1c0b7d20 con 0x7efe1c0d5200 2024-09-12T03:19:12.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:12.465+0000 7efe1affd640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7efe1c15eb90 0x7efe1c0d2df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:33896/0 (socket says 172.21.15.27:33896) 2024-09-12T03:19:12.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:12.465+0000 7efe1affd640 1 -- 172.21.15.27:0/3364680923 learned_addr learned my addr 172.21.15.27:0/3364680923 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:19:12.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:12.465+0000 7efe1a7fc640 1 --2- 172.21.15.27:0/3364680923 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7efe1c0d5200 0x7efe1c0d1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:12.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:12.466+0000 7efe1a7fc640 1 -- 172.21.15.27:0/3364680923 >> v1:172.21.15.45:6789/0 conn(0x7efe1c0d1550 legacy=0x7efe1c0b8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:12.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:12.466+0000 7efe1a7fc640 1 -- 172.21.15.27:0/3364680923 >> v1:172.21.15.27:6789/0 conn(0x7efe1c15eb90 legacy=0x7efe1c0d2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:12.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:12.466+0000 7efe1a7fc640 1 -- 172.21.15.27:0/3364680923 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe1c0d3a40 con 0x7efe1c0d5200 2024-09-12T03:19:12.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:12.466+0000 7efe1a7fc640 1 --2- 172.21.15.27:0/3364680923 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7efe1c0d5200 0x7efe1c0d1010 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7efe100097d0 tx=0x7efe1002a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6883657ff2ed8a73 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:12.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:12.467+0000 7efe19ffb640 1 -- 172.21.15.27:0/3364680923 <== mon.2 v2:172.21.15.45:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe10037070 con 0x7efe1c0d5200 2024-09-12T03:19:12.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:12.467+0000 7efe19ffb640 1 -- 172.21.15.27:0/3364680923 <== mon.2 v2:172.21.15.45:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe10032440 con 0x7efe1c0d5200 2024-09-12T03:19:12.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:12.467+0000 7efe19ffb640 1 -- 172.21.15.27:0/3364680923 <== mon.2 v2:172.21.15.45:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe10032810 con 0x7efe1c0d5200 2024-09-12T03:19:12.468 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:12.467+0000 7efe214af640 1 -- 172.21.15.27:0/3364680923 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7efe1c0d5200 msgr2=0x7efe1c0d1010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:12.468 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:12.467+0000 7efe214af640 1 --2- 172.21.15.27:0/3364680923 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7efe1c0d5200 0x7efe1c0d1010 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7efe100097d0 tx=0x7efe1002a210 comp rx=0 tx=0).stop 2024-09-12T03:19:12.468 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:12.467+0000 7efe214af640 1 -- 172.21.15.27:0/3364680923 shutdown_connections 2024-09-12T03:19:12.468 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:12.467+0000 7efe214af640 1 --2- 172.21.15.27:0/3364680923 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7efe1c0d5200 0x7efe1c0d1010 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:12.468 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:12.467+0000 7efe214af640 1 -- 172.21.15.27:0/3364680923 >> 172.21.15.27:0/3364680923 conn(0x7efe1c06dce0 msgr2=0x7efe1c06e0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:12.468 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:12.468+0000 7efe214af640 1 -- 172.21.15.27:0/3364680923 shutdown_connections 2024-09-12T03:19:12.468 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:12.468+0000 7efe214af640 1 -- 172.21.15.27:0/3364680923 wait complete. 2024-09-12T03:19:12.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:12.469+0000 7efe214af640 1 Processor -- start 2024-09-12T03:19:12.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:12.469+0000 7efe214af640 1 -- start start 2024-09-12T03:19:12.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:12.470+0000 7efe214af640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efe1c0d1550 0x7efe1c15ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:12.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:12.470+0000 7efe214af640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efe1c0d5200 0x7efe1c1560f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:12.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:12.470+0000 7efe214af640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7efe1c15eb90 0x7efe1c0c8b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:12.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:12.470+0000 7efe214af640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7efe1c0d3500 con 0x7efe1c0d1550 2024-09-12T03:19:12.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:12.470+0000 7efe214af640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7efe1c0d42e0 con 0x7efe1c0d5200 2024-09-12T03:19:12.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:12.470+0000 7efe214af640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7efe1c0d4000 con 0x7efe1c15eb90 2024-09-12T03:19:12.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:12.470+0000 7efe1affd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efe1c0d1550 0x7efe1c15ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:12.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:12.470+0000 7efe1affd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efe1c0d1550 0x7efe1c15ac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55250/0 (socket says 172.21.15.27:55250) 2024-09-12T03:19:12.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:12.470+0000 7efe1affd640 1 -- 172.21.15.27:0/849729517 learned_addr learned my addr 172.21.15.27:0/849729517 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:19:12.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:12.470+0000 7efe1a7fc640 1 --2- 172.21.15.27:0/849729517 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efe1c0d5200 0x7efe1c1560f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:12.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:12.470+0000 7efe1b7fe640 1 --2- 172.21.15.27:0/849729517 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7efe1c15eb90 0x7efe1c0c8b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:12.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:12.470+0000 7efe1affd640 1 -- 172.21.15.27:0/849729517 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7efe1c15eb90 msgr2=0x7efe1c0c8b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:12.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:12.470+0000 7efe1affd640 1 --2- 172.21.15.27:0/849729517 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7efe1c15eb90 0x7efe1c0c8b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:12.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:12.470+0000 7efe1affd640 1 -- 172.21.15.27:0/849729517 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efe1c0d5200 msgr2=0x7efe1c1560f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:12.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:12.470+0000 7efe1affd640 1 --2- 172.21.15.27:0/849729517 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efe1c0d5200 0x7efe1c1560f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:12.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:12.470+0000 7efe1affd640 1 -- 172.21.15.27:0/849729517 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe1c0c83e0 con 0x7efe1c0d1550 2024-09-12T03:19:12.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:12.470+0000 7efe1b7fe640 1 --2- 172.21.15.27:0/849729517 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7efe1c15eb90 0x7efe1c0c8b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:19:12.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:12.470+0000 7efe1affd640 1 --2- 172.21.15.27:0/849729517 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efe1c0d1550 0x7efe1c15ac30 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7efe0c00c4b0 tx=0x7efe0c00c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:12.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:12.470+0000 7efe037fe640 1 -- 172.21.15.27:0/849729517 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe0c017070 con 0x7efe1c0d1550 2024-09-12T03:19:12.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:12.471+0000 7efe037fe640 1 -- 172.21.15.27:0/849729517 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe0c005ce0 con 0x7efe1c0d1550 2024-09-12T03:19:12.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:12.471+0000 7efe214af640 1 -- 172.21.15.27:0/849729517 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe1c15b2a0 con 0x7efe1c0d1550 2024-09-12T03:19:12.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:12.471+0000 7efe037fe640 1 -- 172.21.15.27:0/849729517 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe0c007270 con 0x7efe1c0d1550 2024-09-12T03:19:12.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:12.471+0000 7efe214af640 1 -- 172.21.15.27:0/849729517 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe1c15b860 con 0x7efe1c0d1550 2024-09-12T03:19:12.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:12.472+0000 7efe214af640 1 -- 172.21.15.27:0/849729517 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe1c0d1010 con 0x7efe1c0d1550 2024-09-12T03:19:12.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:12.472+0000 7efe037fe640 1 -- 172.21.15.27:0/849729517 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 8) ==== 90702+0+0 (secure 0 0 0) 0x7efe0c01a070 con 0x7efe1c0d1550 2024-09-12T03:19:12.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:12.473+0000 7efe037fe640 1 --2- 172.21.15.27:0/849729517 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7efe0806c070 0x7efe0806e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:12.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:12.473+0000 7efe037fe640 1 -- 172.21.15.27:0/849729517 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5767+0+0 (secure 0 0 0) 0x7efe0c08a490 con 0x7efe1c0d1550 2024-09-12T03:19:12.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:12.473+0000 7efe1a7fc640 1 --2- 172.21.15.27:0/849729517 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7efe0806c070 0x7efe0806e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:12.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:12.473+0000 7efe1a7fc640 1 --2- 172.21.15.27:0/849729517 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7efe0806c070 0x7efe0806e530 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7efe10034000 tx=0x7efe1002e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:12.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:12.477+0000 7efe037fe640 1 -- 172.21.15.27:0/849729517 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7efe0c0584e0 con 0x7efe1c0d1550 2024-09-12T03:19:12.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:12.687+0000 7efe214af640 1 -- 172.21.15.27:0/849729517 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efe1c0c8790 con 0x7efe1c0d1550 2024-09-12T03:19:12.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:12.687+0000 7efe037fe640 1 -- 172.21.15.27:0/849729517 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+5223 (secure 0 0 0) 0x7efe0c05c7e0 con 0x7efe1c0d1550 2024-09-12T03:19:12.687 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:19:12.688 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":23,"btime":"2024-09-12T03:17:47:237558+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4835,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6834/1457797481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":1457797481},{"type":"v1","addr":"172.21.15.27:6834","nonce":1457797481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5029,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6836/2842491281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":2842491281},{"type":"v1","addr":"172.21.15.45:6836","nonce":2842491281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5032,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/3339744340","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6835","nonce":3339744340},{"type":"v1","addr":"172.21.15.45:6837","nonce":3339744340}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-12T03:17:38.580473+0000","modified":"2024-09-12T03:17:40.557950+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5205},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5205":{"gid":5205,"name":"a","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.27:6835/2571101228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":2571101228},{"type":"v1","addr":"172.21.15.27:6835","nonce":2571101228}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5205,"qdb_cluster":[5205]},"id":3}]} 2024-09-12T03:19:12.688 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 23 2024-09-12T03:19:12.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:12.689+0000 7efe214af640 1 -- 172.21.15.27:0/849729517 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7efe0806c070 msgr2=0x7efe0806e530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:12.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:12.689+0000 7efe214af640 1 --2- 172.21.15.27:0/849729517 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7efe0806c070 0x7efe0806e530 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7efe10034000 tx=0x7efe1002e000 comp rx=0 tx=0).stop 2024-09-12T03:19:12.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:12.689+0000 7efe214af640 1 -- 172.21.15.27:0/849729517 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efe1c0d1550 msgr2=0x7efe1c15ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:12.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:12.689+0000 7efe214af640 1 --2- 172.21.15.27:0/849729517 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efe1c0d1550 0x7efe1c15ac30 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7efe0c00c4b0 tx=0x7efe0c00c970 comp rx=0 tx=0).stop 2024-09-12T03:19:12.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:12.689+0000 7efe214af640 1 -- 172.21.15.27:0/849729517 shutdown_connections 2024-09-12T03:19:12.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:12.689+0000 7efe214af640 1 --2- 172.21.15.27:0/849729517 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7efe0806c070 0x7efe0806e530 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:12.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:12.689+0000 7efe214af640 1 --2- 172.21.15.27:0/849729517 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7efe1c15eb90 0x7efe1c0c8b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:12.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:12.689+0000 7efe214af640 1 --2- 172.21.15.27:0/849729517 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efe1c0d5200 0x7efe1c1560f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:12.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:12.689+0000 7efe214af640 1 --2- 172.21.15.27:0/849729517 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efe1c0d1550 0x7efe1c15ac30 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:12.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:12.689+0000 7efe214af640 1 -- 172.21.15.27:0/849729517 >> 172.21.15.27:0/849729517 conn(0x7efe1c06dce0 msgr2=0x7efe1c15fbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:12.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:12.689+0000 7efe214af640 1 -- 172.21.15.27:0/849729517 shutdown_connections 2024-09-12T03:19:12.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:12.689+0000 7efe214af640 1 -- 172.21.15.27:0/849729517 wait complete. 2024-09-12T03:19:12.701 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json session ls 2024-09-12T03:19:12.822 INFO:teuthology.orchestra.run.smithi027.stdout:[] 2024-09-12T03:19:12.822 DEBUG:tasks.cephfs.filesystem:_json_asok output [] 2024-09-12T03:19:12.823 DEBUG:tasks.ceph_test_case:wait_until_true: success in 75s and 0 retries 2024-09-12T03:19:12.823 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-12T03:19:12.823 DEBUG:teuthology.orchestra.run.smithi027:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-12T03:19:12.864 DEBUG:teuthology.orchestra.run.smithi045:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-12T03:19:12.917 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-12T03:19:12.918 INFO:tasks.cephfs.mount:Terminating background process 2024-09-12T03:19:12.918 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-12T03:19:12.918 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T03:19:12.934 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056004,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497432k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16324 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248716k,nr_inodes=812179,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497432k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-12T03:19:12.935 INFO:tasks.cephfs.mount:Terminating background process 2024-09-12T03:19:12.935 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-12T03:19:12.935 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T03:19:12.954 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056795,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16602 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249348k,nr_inodes=812337,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-12T03:19:12.954 INFO:tasks.cephfs.mount:Terminating background process 2024-09-12T03:19:12.954 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-12T03:19:12.954 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T03:19:12.990 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056004,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497432k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16324 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248716k,nr_inodes=812179,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497432k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-12T03:19:12.991 INFO:tasks.cephfs.mount:Terminating background process 2024-09-12T03:19:12.991 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-12T03:19:12.991 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T03:19:13.011 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056795,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16602 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249348k,nr_inodes=812337,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-12T03:19:13.011 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-12T03:19:13.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.133+0000 7fba1cf9f640 1 Processor -- start 2024-09-12T03:19:13.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.133+0000 7fba1cf9f640 1 -- start start 2024-09-12T03:19:13.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.134+0000 7fba1cf9f640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fba180d5200 0x7fba180d1010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:13.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.134+0000 7fba1cf9f640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fba180d1550 0x7fba180b8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:13.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.134+0000 7fba1cf9f640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fba180d3760 con 0x7fba1815eb90 2024-09-12T03:19:13.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.134+0000 7fba1cf9f640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7fba180b7d20 con 0x7fba180d5200 2024-09-12T03:19:13.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.134+0000 7fba1cf9f640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fba180d42e0 con 0x7fba180d1550 2024-09-12T03:19:13.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.134+0000 7fba15d9b640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fba180d5200 0x7fba180d1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:13.138 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.135+0000 7fba15d9b640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fba180d5200 0x7fba180d1010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55252/0 (socket says 172.21.15.27:55252) 2024-09-12T03:19:13.138 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.135+0000 7fba15d9b640 1 -- 172.21.15.27:0/1870191274 learned_addr learned my addr 172.21.15.27:0/1870191274 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:19:13.138 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.135+0000 7fba16d9d640 1 --2- 172.21.15.27:0/1870191274 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fba180d1550 0x7fba180b8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:13.138 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.135+0000 7fba15d9b640 1 -- 172.21.15.27:0/1870191274 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fba180d1550 msgr2=0x7fba180b8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:13.138 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.135+0000 7fba15d9b640 1 --2- 172.21.15.27:0/1870191274 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fba180d1550 0x7fba180b8190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:13.138 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.135+0000 7fba15d9b640 1 -- 172.21.15.27:0/1870191274 >> v1:172.21.15.45:6789/0 conn(0x7fba1815eb90 legacy=0x7fba180d2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:13.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.135+0000 7fba15d9b640 1 -- 172.21.15.27:0/1870191274 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba180d3a40 con 0x7fba180d5200 2024-09-12T03:19:13.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.135+0000 7fba16d9d640 1 --2- 172.21.15.27:0/1870191274 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fba180d1550 0x7fba180b8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:19:13.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.136+0000 7fba15d9b640 1 --2- 172.21.15.27:0/1870191274 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fba180d5200 0x7fba180d1010 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fba00009870 tx=0x7fba0002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=861975574a69c53e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:13.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.136+0000 7fba1559a640 1 -- 172.21.15.27:0/1870191274 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba00037070 con 0x7fba180d5200 2024-09-12T03:19:13.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.136+0000 7fba1559a640 1 -- 172.21.15.27:0/1870191274 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba00032440 con 0x7fba180d5200 2024-09-12T03:19:13.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.136+0000 7fba1559a640 1 -- 172.21.15.27:0/1870191274 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba00032810 con 0x7fba180d5200 2024-09-12T03:19:13.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.136+0000 7fba1cf9f640 1 -- 172.21.15.27:0/1870191274 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fba180d5200 msgr2=0x7fba180d1010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:13.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.136+0000 7fba1cf9f640 1 --2- 172.21.15.27:0/1870191274 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fba180d5200 0x7fba180d1010 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fba00009870 tx=0x7fba0002a210 comp rx=0 tx=0).stop 2024-09-12T03:19:13.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.136+0000 7fba1cf9f640 1 -- 172.21.15.27:0/1870191274 shutdown_connections 2024-09-12T03:19:13.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.136+0000 7fba1cf9f640 1 --2- 172.21.15.27:0/1870191274 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fba180d1550 0x7fba180b8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:13.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.136+0000 7fba1cf9f640 1 --2- 172.21.15.27:0/1870191274 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fba180d5200 0x7fba180d1010 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:13.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.136+0000 7fba1cf9f640 1 -- 172.21.15.27:0/1870191274 >> 172.21.15.27:0/1870191274 conn(0x7fba1806dce0 msgr2=0x7fba1806e0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:13.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.136+0000 7fba1cf9f640 1 -- 172.21.15.27:0/1870191274 shutdown_connections 2024-09-12T03:19:13.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.136+0000 7fba1cf9f640 1 -- 172.21.15.27:0/1870191274 wait complete. 2024-09-12T03:19:13.144 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.138+0000 7fba1cf9f640 1 Processor -- start 2024-09-12T03:19:13.144 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.138+0000 7fba1cf9f640 1 -- start start 2024-09-12T03:19:13.144 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.138+0000 7fba1cf9f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fba180d1550 0x7fba18155ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:13.144 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.139+0000 7fba1cf9f640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fba180d5200 0x7fba181538d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:13.144 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.139+0000 7fba1cf9f640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fba1815eb90 0x7fba181f7920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:13.144 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.139+0000 7fba1cf9f640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fba180d4000 con 0x7fba1815eb90 2024-09-12T03:19:13.145 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.139+0000 7fba1cf9f640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fba180d3500 con 0x7fba180d1550 2024-09-12T03:19:13.145 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.139+0000 7fba1cf9f640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fba180d19a0 con 0x7fba180d5200 2024-09-12T03:19:13.145 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.139+0000 7fba16d9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fba1815eb90 0x7fba181f7920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:13.145 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.139+0000 7fba16d9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fba1815eb90 0x7fba181f7920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55256/0 (socket says 172.21.15.27:55256) 2024-09-12T03:19:13.145 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.139+0000 7fba16d9d640 1 -- 172.21.15.27:0/1863741819 learned_addr learned my addr 172.21.15.27:0/1863741819 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:19:13.145 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.139+0000 7fba15d9b640 1 --2- 172.21.15.27:0/1863741819 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fba180d5200 0x7fba181538d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:13.145 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.139+0000 7fba1659c640 1 --2- 172.21.15.27:0/1863741819 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fba180d1550 0x7fba18155ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:13.145 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.139+0000 7fba16d9d640 1 -- 172.21.15.27:0/1863741819 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fba180d5200 msgr2=0x7fba181538d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:13.145 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.139+0000 7fba16d9d640 1 --2- 172.21.15.27:0/1863741819 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fba180d5200 0x7fba181538d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:13.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.139+0000 7fba16d9d640 1 -- 172.21.15.27:0/1863741819 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fba180d1550 msgr2=0x7fba18155ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:13.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.139+0000 7fba16d9d640 1 --2- 172.21.15.27:0/1863741819 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fba180d1550 0x7fba18155ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:13.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.139+0000 7fba16d9d640 1 -- 172.21.15.27:0/1863741819 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba18156a60 con 0x7fba1815eb90 2024-09-12T03:19:13.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.139+0000 7fba1659c640 1 --2- 172.21.15.27:0/1863741819 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fba180d1550 0x7fba18155ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:19:13.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.139+0000 7fba15d9b640 1 --2- 172.21.15.27:0/1863741819 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fba180d5200 0x7fba181538d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:19:13.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.139+0000 7fba16d9d640 1 --2- 172.21.15.27:0/1863741819 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fba1815eb90 0x7fba181f7920 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fba1000c610 tx=0x7fba1000cad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:13.147 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.139+0000 7fba0effd640 1 -- 172.21.15.27:0/1863741819 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba10015070 con 0x7fba1815eb90 2024-09-12T03:19:13.147 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.139+0000 7fba0effd640 1 -- 172.21.15.27:0/1863741819 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba10011800 con 0x7fba1815eb90 2024-09-12T03:19:13.147 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.139+0000 7fba1cf9f640 1 -- 172.21.15.27:0/1863741819 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba181564a0 con 0x7fba1815eb90 2024-09-12T03:19:13.147 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.139+0000 7fba0effd640 1 -- 172.21.15.27:0/1863741819 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba10011bf0 con 0x7fba1815eb90 2024-09-12T03:19:13.147 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.139+0000 7fba1cf9f640 1 -- 172.21.15.27:0/1863741819 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba1815c300 con 0x7fba1815eb90 2024-09-12T03:19:13.147 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.140+0000 7fba1cf9f640 1 -- 172.21.15.27:0/1863741819 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba1815f050 con 0x7fba1815eb90 2024-09-12T03:19:13.147 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.141+0000 7fba0effd640 1 -- 172.21.15.27:0/1863741819 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 8) ==== 90702+0+0 (secure 0 0 0) 0x7fba10018070 con 0x7fba1815eb90 2024-09-12T03:19:13.148 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.141+0000 7fba0effd640 1 --2- 172.21.15.27:0/1863741819 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fb9f806c070 0x7fb9f806e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:13.148 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.141+0000 7fba0effd640 1 -- 172.21.15.27:0/1863741819 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5767+0+0 (secure 0 0 0) 0x7fba1008b290 con 0x7fba1815eb90 2024-09-12T03:19:13.148 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.142+0000 7fba1659c640 1 --2- 172.21.15.27:0/1863741819 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fb9f806c070 0x7fb9f806e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:13.148 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.142+0000 7fba1659c640 1 --2- 172.21.15.27:0/1863741819 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fb9f806c070 0x7fb9f806e530 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fba08007e90 tx=0x7fba080073a0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:13.148 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.146+0000 7fba0effd640 1 -- 172.21.15.27:0/1863741819 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fba10059790 con 0x7fba1815eb90 2024-09-12T03:19:13.360 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.361+0000 7fba1cf9f640 1 -- 172.21.15.27:0/1863741819 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fba18143920 con 0x7fba1815eb90 2024-09-12T03:19:13.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.361+0000 7fba0effd640 1 -- 172.21.15.27:0/1863741819 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+5223 (secure 0 0 0) 0x7fba10059090 con 0x7fba1815eb90 2024-09-12T03:19:13.361 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:19:13.361 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":23,"btime":"2024-09-12T03:17:47:237558+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4835,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6834/1457797481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":1457797481},{"type":"v1","addr":"172.21.15.27:6834","nonce":1457797481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5029,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6836/2842491281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":2842491281},{"type":"v1","addr":"172.21.15.45:6836","nonce":2842491281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5032,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/3339744340","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6835","nonce":3339744340},{"type":"v1","addr":"172.21.15.45:6837","nonce":3339744340}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-12T03:17:38.580473+0000","modified":"2024-09-12T03:17:40.557950+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5205},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5205":{"gid":5205,"name":"a","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.27:6835/2571101228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":2571101228},{"type":"v1","addr":"172.21.15.27:6835","nonce":2571101228}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5205,"qdb_cluster":[5205]},"id":3}]} 2024-09-12T03:19:13.362 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 23 2024-09-12T03:19:13.363 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.363+0000 7fba1cf9f640 1 -- 172.21.15.27:0/1863741819 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fb9f806c070 msgr2=0x7fb9f806e530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:13.363 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.363+0000 7fba1cf9f640 1 --2- 172.21.15.27:0/1863741819 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fb9f806c070 0x7fb9f806e530 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fba08007e90 tx=0x7fba080073a0 comp rx=0 tx=0).stop 2024-09-12T03:19:13.364 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.363+0000 7fba1cf9f640 1 -- 172.21.15.27:0/1863741819 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fba1815eb90 msgr2=0x7fba181f7920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:13.364 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.363+0000 7fba1cf9f640 1 --2- 172.21.15.27:0/1863741819 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fba1815eb90 0x7fba181f7920 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fba1000c610 tx=0x7fba1000cad0 comp rx=0 tx=0).stop 2024-09-12T03:19:13.364 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.363+0000 7fba1cf9f640 1 -- 172.21.15.27:0/1863741819 shutdown_connections 2024-09-12T03:19:13.364 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.363+0000 7fba1cf9f640 1 --2- 172.21.15.27:0/1863741819 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fb9f806c070 0x7fb9f806e530 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:13.364 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.363+0000 7fba1cf9f640 1 --2- 172.21.15.27:0/1863741819 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fba1815eb90 0x7fba181f7920 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:13.364 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.363+0000 7fba1cf9f640 1 --2- 172.21.15.27:0/1863741819 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fba180d5200 0x7fba181538d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:13.364 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.363+0000 7fba1cf9f640 1 --2- 172.21.15.27:0/1863741819 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fba180d1550 0x7fba18155ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:13.364 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.363+0000 7fba1cf9f640 1 -- 172.21.15.27:0/1863741819 >> 172.21.15.27:0/1863741819 conn(0x7fba1806dce0 msgr2=0x7fba1815ba10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:13.364 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.363+0000 7fba1cf9f640 1 -- 172.21.15.27:0/1863741819 shutdown_connections 2024-09-12T03:19:13.364 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.363+0000 7fba1cf9f640 1 -- 172.21.15.27:0/1863741819 wait complete. 2024-09-12T03:19:13.374 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-12T03:19:13.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.477+0000 7f44130e4640 1 Processor -- start 2024-09-12T03:19:13.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.477+0000 7f44130e4640 1 -- start start 2024-09-12T03:19:13.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.478+0000 7f44130e4640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f440c15eb90 0x7f440c0d2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:13.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.478+0000 7f44130e4640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f440c0d1720 0x7f440c0b8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:13.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.478+0000 7f44130e4640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f440c0d3760 con 0x7f440c0d5200 2024-09-12T03:19:13.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.478+0000 7f44130e4640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f440c0b7d20 con 0x7f440c0d1720 2024-09-12T03:19:13.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.478+0000 7f44130e4640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f440c0d42e0 con 0x7f440c15eb90 2024-09-12T03:19:13.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.478+0000 7f4410e80640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f440c15eb90 0x7f440c0d2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:13.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.478+0000 7f4411681640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f440c0d1720 0x7f440c0b8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:13.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.478+0000 7f440bfff640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f440c0d5200 0x7f440c0d1010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.27:36618/0 (socket says 172.21.15.27:36618) 2024-09-12T03:19:13.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.478+0000 7f440bfff640 1 -- 172.21.15.27:0/3030353971 learned_addr learned my addr 172.21.15.27:0/3030353971 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:19:13.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.479+0000 7f4410e80640 1 -- 172.21.15.27:0/3030353971 >> v2:172.21.15.45:3300/0 conn(0x7f440c0d1720 msgr2=0x7f440c0b8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:13.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.479+0000 7f4410e80640 1 --2- 172.21.15.27:0/3030353971 >> v2:172.21.15.45:3300/0 conn(0x7f440c0d1720 0x7f440c0b8190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:13.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.479+0000 7f4410e80640 1 -- 172.21.15.27:0/3030353971 >> v1:172.21.15.45:6789/0 conn(0x7f440c0d5200 legacy=0x7f440c0d1010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:13.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.479+0000 7f4410e80640 1 -- 172.21.15.27:0/3030353971 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f440c0d3a40 con 0x7f440c15eb90 2024-09-12T03:19:13.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.479+0000 7f4411681640 1 --2- 172.21.15.27:0/3030353971 >> v2:172.21.15.45:3300/0 conn(0x7f440c0d1720 0x7f440c0b8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:19:13.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.479+0000 7f4410e80640 1 --2- 172.21.15.27:0/3030353971 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f440c15eb90 0x7f440c0d2df0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f44000097a0 tx=0x7f440002a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=402a68d2bfe74cc3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:13.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.480+0000 7f440b7fe640 1 -- 172.21.15.27:0/3030353971 <== mon.2 v2:172.21.15.45:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4400037070 con 0x7f440c15eb90 2024-09-12T03:19:13.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.480+0000 7f440b7fe640 1 -- 172.21.15.27:0/3030353971 <== mon.2 v2:172.21.15.45:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4400032440 con 0x7f440c15eb90 2024-09-12T03:19:13.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.480+0000 7f440b7fe640 1 -- 172.21.15.27:0/3030353971 <== mon.2 v2:172.21.15.45:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4400032810 con 0x7f440c15eb90 2024-09-12T03:19:13.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.480+0000 7f44130e4640 1 -- 172.21.15.27:0/3030353971 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f440c15eb90 msgr2=0x7f440c0d2df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:13.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.480+0000 7f44130e4640 1 --2- 172.21.15.27:0/3030353971 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f440c15eb90 0x7f440c0d2df0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f44000097a0 tx=0x7f440002a180 comp rx=0 tx=0).stop 2024-09-12T03:19:13.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.480+0000 7f44130e4640 1 -- 172.21.15.27:0/3030353971 shutdown_connections 2024-09-12T03:19:13.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.480+0000 7f44130e4640 1 --2- 172.21.15.27:0/3030353971 >> v2:172.21.15.45:3300/0 conn(0x7f440c0d1720 0x7f440c0b8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:13.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.480+0000 7f44130e4640 1 --2- 172.21.15.27:0/3030353971 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f440c15eb90 0x7f440c0d2df0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:13.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.480+0000 7f44130e4640 1 -- 172.21.15.27:0/3030353971 >> 172.21.15.27:0/3030353971 conn(0x7f440c06dce0 msgr2=0x7f440c06e0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:13.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.480+0000 7f44130e4640 1 -- 172.21.15.27:0/3030353971 shutdown_connections 2024-09-12T03:19:13.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.480+0000 7f44130e4640 1 -- 172.21.15.27:0/3030353971 wait complete. 2024-09-12T03:19:13.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.481+0000 7f44130e4640 1 Processor -- start 2024-09-12T03:19:13.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.482+0000 7f44130e4640 1 -- start start 2024-09-12T03:19:13.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.482+0000 7f44130e4640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f440c0d1720 0x7f440c15d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:13.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.482+0000 7f44130e4640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f440c0d5200 0x7f440c149c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:13.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.482+0000 7f44130e4640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f440c15eb90 0x7f440c15a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:13.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.482+0000 7f44130e4640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f440c0d3330 con 0x7f440c0d1720 2024-09-12T03:19:13.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.482+0000 7f44130e4640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f440c0d4000 con 0x7f440c15eb90 2024-09-12T03:19:13.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.482+0000 7f44130e4640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f440c0d34b0 con 0x7f440c0d5200 2024-09-12T03:19:13.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.483+0000 7f4410e80640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f440c0d1720 0x7f440c15d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:13.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.483+0000 7f4410e80640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f440c0d1720 0x7f440c15d990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55272/0 (socket says 172.21.15.27:55272) 2024-09-12T03:19:13.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.483+0000 7f4410e80640 1 -- 172.21.15.27:0/3387893396 learned_addr learned my addr 172.21.15.27:0/3387893396 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:19:13.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.483+0000 7f4411681640 1 --2- 172.21.15.27:0/3387893396 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f440c15eb90 0x7f440c15a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:13.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.483+0000 7f440bfff640 1 --2- 172.21.15.27:0/3387893396 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f440c0d5200 0x7f440c149c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:13.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.483+0000 7f4410e80640 1 -- 172.21.15.27:0/3387893396 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f440c0d5200 msgr2=0x7f440c149c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:13.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.483+0000 7f4410e80640 1 --2- 172.21.15.27:0/3387893396 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f440c0d5200 0x7f440c149c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:13.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.483+0000 7f4410e80640 1 -- 172.21.15.27:0/3387893396 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f440c15eb90 msgr2=0x7f440c15a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:13.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.483+0000 7f4410e80640 1 --2- 172.21.15.27:0/3387893396 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f440c15eb90 0x7f440c15a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:13.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.483+0000 7f4410e80640 1 -- 172.21.15.27:0/3387893396 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f440c14a140 con 0x7f440c0d1720 2024-09-12T03:19:13.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.483+0000 7f4410e80640 1 --2- 172.21.15.27:0/3387893396 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f440c0d1720 0x7f440c15d990 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f44000098c0 tx=0x7f44000046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:13.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.483+0000 7f44097fa640 1 -- 172.21.15.27:0/3387893396 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4400037040 con 0x7f440c0d1720 2024-09-12T03:19:13.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.483+0000 7f44097fa640 1 -- 172.21.15.27:0/3387893396 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4400042070 con 0x7f440c0d1720 2024-09-12T03:19:13.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.483+0000 7f44097fa640 1 -- 172.21.15.27:0/3387893396 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4400030920 con 0x7f440c0d1720 2024-09-12T03:19:13.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.483+0000 7f44130e4640 1 -- 172.21.15.27:0/3387893396 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f440c15aea0 con 0x7f440c0d1720 2024-09-12T03:19:13.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.483+0000 7f44130e4640 1 -- 172.21.15.27:0/3387893396 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f440c15b460 con 0x7f440c0d1720 2024-09-12T03:19:13.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.484+0000 7f44130e4640 1 -- 172.21.15.27:0/3387893396 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f440c15f090 con 0x7f440c0d1720 2024-09-12T03:19:13.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.485+0000 7f44097fa640 1 -- 172.21.15.27:0/3387893396 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 8) ==== 90702+0+0 (secure 0 0 0) 0x7f4400030ac0 con 0x7f440c0d1720 2024-09-12T03:19:13.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.485+0000 7f44097fa640 1 --2- 172.21.15.27:0/3387893396 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f43f006c070 0x7f43f006e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:13.490 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.485+0000 7f44097fa640 1 -- 172.21.15.27:0/3387893396 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5767+0+0 (secure 0 0 0) 0x7f44000aba50 con 0x7f440c0d1720 2024-09-12T03:19:13.490 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.485+0000 7f440bfff640 1 --2- 172.21.15.27:0/3387893396 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f43f006c070 0x7f43f006e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:13.490 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.486+0000 7f440bfff640 1 --2- 172.21.15.27:0/3387893396 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f43f006c070 0x7f43f006e530 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f440c161bc0 tx=0x7f43f8007e80 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:13.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.490+0000 7f44097fa640 1 -- 172.21.15.27:0/3387893396 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f440c14a140 con 0x7f440c0d1720 2024-09-12T03:19:13.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.700+0000 7f44130e4640 1 -- 172.21.15.27:0/3387893396 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f440c15e340 con 0x7f440c0d1720 2024-09-12T03:19:13.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.700+0000 7f44097fa640 1 -- 172.21.15.27:0/3387893396 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+5223 (secure 0 0 0) 0x7f4400079a10 con 0x7f440c0d1720 2024-09-12T03:19:13.885 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:19:13.885 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":23,"btime":"2024-09-12T03:17:47:237558+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4835,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6834/1457797481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":1457797481},{"type":"v1","addr":"172.21.15.27:6834","nonce":1457797481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5029,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6836/2842491281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":2842491281},{"type":"v1","addr":"172.21.15.45:6836","nonce":2842491281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5032,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/3339744340","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6835","nonce":3339744340},{"type":"v1","addr":"172.21.15.45:6837","nonce":3339744340}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-12T03:17:38.580473+0000","modified":"2024-09-12T03:17:40.557950+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5205},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5205":{"gid":5205,"name":"a","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.27:6835/2571101228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":2571101228},{"type":"v1","addr":"172.21.15.27:6835","nonce":2571101228}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5205,"qdb_cluster":[5205]},"id":3}]} 2024-09-12T03:19:13.887 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 23 2024-09-12T03:19:13.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.702+0000 7f44130e4640 1 -- 172.21.15.27:0/3387893396 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f43f006c070 msgr2=0x7f43f006e530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:13.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.702+0000 7f44130e4640 1 --2- 172.21.15.27:0/3387893396 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f43f006c070 0x7f43f006e530 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f440c161bc0 tx=0x7f43f8007e80 comp rx=0 tx=0).stop 2024-09-12T03:19:13.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.702+0000 7f44130e4640 1 -- 172.21.15.27:0/3387893396 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f440c0d1720 msgr2=0x7f440c15d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:13.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.702+0000 7f44130e4640 1 --2- 172.21.15.27:0/3387893396 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f440c0d1720 0x7f440c15d990 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f44000098c0 tx=0x7f44000046c0 comp rx=0 tx=0).stop 2024-09-12T03:19:13.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.702+0000 7f44130e4640 1 -- 172.21.15.27:0/3387893396 shutdown_connections 2024-09-12T03:19:13.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.702+0000 7f44130e4640 1 --2- 172.21.15.27:0/3387893396 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f43f006c070 0x7f43f006e530 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:13.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.702+0000 7f44130e4640 1 --2- 172.21.15.27:0/3387893396 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f440c15eb90 0x7f440c15a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:13.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.702+0000 7f44130e4640 1 --2- 172.21.15.27:0/3387893396 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f440c0d5200 0x7f440c149c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:13.888 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.702+0000 7f44130e4640 1 --2- 172.21.15.27:0/3387893396 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f440c0d1720 0x7f440c15d990 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:13.888 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.702+0000 7f44130e4640 1 -- 172.21.15.27:0/3387893396 >> 172.21.15.27:0/3387893396 conn(0x7f440c06dce0 msgr2=0x7f440c15f2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:13.888 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.703+0000 7f44130e4640 1 -- 172.21.15.27:0/3387893396 shutdown_connections 2024-09-12T03:19:13.888 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.703+0000 7f44130e4640 1 -- 172.21.15.27:0/3387893396 wait complete. 2024-09-12T03:19:13.888 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-12T03:19:13.991 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.991+0000 7f9f18853640 1 Processor -- start 2024-09-12T03:19:13.991 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.991+0000 7f9f18853640 1 -- start start 2024-09-12T03:19:13.995 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.992+0000 7f9f18853640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f9f1406f360 0x7f9f140cef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:13.995 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.992+0000 7f9f18853640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9f1406cc40 con 0x7f9f140b1ca0 2024-09-12T03:19:13.995 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.992+0000 7f9f18853640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9f140d1ce0 con 0x7f9f140cf490 2024-09-12T03:19:13.995 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.992+0000 7f9f18853640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f9f140cf900 con 0x7f9f1406f360 2024-09-12T03:19:13.995 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.992+0000 7f9f1259c640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f9f140b1ca0 0x7f9f1415f830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:33900/0 (socket says 172.21.15.27:33900) 2024-09-12T03:19:13.995 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.992+0000 7f9f1259c640 1 -- 172.21.15.27:0/1975306697 learned_addr learned my addr 172.21.15.27:0/1975306697 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:19:13.995 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.993+0000 7f9f11d9b640 1 --2- 172.21.15.27:0/1975306697 >> v2:172.21.15.45:3300/0 conn(0x7f9f1406f360 0x7f9f140cef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:13.995 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.993+0000 7f9f1159a640 1 -- 172.21.15.27:0/1975306697 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1755845899 0 0) 0x7f9f1406cc40 con 0x7f9f140b1ca0 2024-09-12T03:19:13.995 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.993+0000 7f9f1159a640 1 -- 172.21.15.27:0/1975306697 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9efc003520 con 0x7f9f140b1ca0 2024-09-12T03:19:13.996 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.993+0000 7f9f1159a640 1 -- 172.21.15.27:0/1975306697 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 975072635 0 0) 0x7f9efc003520 con 0x7f9f140b1ca0 2024-09-12T03:19:13.996 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.993+0000 7f9f1159a640 1 -- 172.21.15.27:0/1975306697 >> v2:172.21.15.45:3300/0 conn(0x7f9f1406f360 msgr2=0x7f9f140cef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:13.996 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.993+0000 7f9f1159a640 1 --2- 172.21.15.27:0/1975306697 >> v2:172.21.15.45:3300/0 conn(0x7f9f1406f360 0x7f9f140cef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:13.996 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.993+0000 7f9f1159a640 1 -- 172.21.15.27:0/1975306697 >> v1:172.21.15.45:6789/0 conn(0x7f9f140cf490 legacy=0x7f9f140d2780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:13.996 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.993+0000 7f9f1159a640 1 -- 172.21.15.27:0/1975306697 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f1406cee0 con 0x7f9f140b1ca0 2024-09-12T03:19:13.996 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.993+0000 7f9f1159a640 1 -- 172.21.15.27:0/1975306697 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 847755213 0 0) 0x7f9f04001670 con 0x7f9f140b1ca0 2024-09-12T03:19:13.996 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.993+0000 7f9f1159a640 1 -- 172.21.15.27:0/1975306697 >> v1:172.21.15.27:6789/0 conn(0x7f9f140b1ca0 legacy=0x7f9f1415f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:13.996 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.993+0000 7f9f1159a640 1 --2- 172.21.15.27:0/1975306697 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f9efc0039b0 0x7f9efc003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:13.996 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.993+0000 7f9f1159a640 1 --2- 172.21.15.27:0/1975306697 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9efc004800 0x7f9efc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:13.996 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.993+0000 7f9f1159a640 1 --2- 172.21.15.27:0/1975306697 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9efc0051c0 0x7f9efc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:13.996 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.993+0000 7f9f1159a640 1 -- 172.21.15.27:0/1975306697 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f9f140cf900 con 0x7f9efc004800 2024-09-12T03:19:13.996 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.993+0000 7f9f1159a640 1 -- 172.21.15.27:0/1975306697 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f9efc00c090 con 0x7f9efc0051c0 2024-09-12T03:19:13.996 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.993+0000 7f9f1159a640 1 -- 172.21.15.27:0/1975306697 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f9efc00c290 con 0x7f9efc0039b0 2024-09-12T03:19:13.996 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.993+0000 7f9f11d9b640 1 --2- 172.21.15.27:0/1975306697 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9efc004800 0x7f9efc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:13.997 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.993+0000 7f9f1259c640 1 --2- 172.21.15.27:0/1975306697 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f9efc0039b0 0x7f9efc003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:13.997 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.993+0000 7f9f12d9d640 1 --2- 172.21.15.27:0/1975306697 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9efc0051c0 0x7f9efc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:13.997 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.993+0000 7f9f11d9b640 1 -- 172.21.15.27:0/1975306697 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f9efc0039b0 msgr2=0x7f9efc003df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:13.997 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.993+0000 7f9f11d9b640 1 --2- 172.21.15.27:0/1975306697 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f9efc0039b0 0x7f9efc003df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:13.997 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.993+0000 7f9f11d9b640 1 -- 172.21.15.27:0/1975306697 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9efc0051c0 msgr2=0x7f9efc00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:13.997 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.994+0000 7f9f11d9b640 1 --2- 172.21.15.27:0/1975306697 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9efc0051c0 0x7f9efc00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:13.997 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.994+0000 7f9f11d9b640 1 -- 172.21.15.27:0/1975306697 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9efc003520 con 0x7f9efc004800 2024-09-12T03:19:13.997 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.994+0000 7f9f12d9d640 1 --2- 172.21.15.27:0/1975306697 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9efc0051c0 0x7f9efc00b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:19:13.997 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.994+0000 7f9f12d9d640 1 -- 172.21.15.27:0/1975306697 reap_dead start 2024-09-12T03:19:13.998 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.994+0000 7f9f11d9b640 1 --2- 172.21.15.27:0/1975306697 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9efc004800 0x7f9efc004c80 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f9f08009ac0 tx=0x7f9efc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90a1d0919c831e75 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:13.998 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.994+0000 7f9f1159a640 1 -- 172.21.15.27:0/1975306697 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f08036070 con 0x7f9efc004800 2024-09-12T03:19:13.998 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.994+0000 7f9f1159a640 1 -- 172.21.15.27:0/1975306697 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f08031400 con 0x7f9efc004800 2024-09-12T03:19:13.998 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.994+0000 7f9f1159a640 1 -- 172.21.15.27:0/1975306697 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f08009690 con 0x7f9efc004800 2024-09-12T03:19:13.998 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.995+0000 7f9f18853640 1 -- 172.21.15.27:0/1975306697 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9efc004800 msgr2=0x7f9efc004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:13.998 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.995+0000 7f9f18853640 1 --2- 172.21.15.27:0/1975306697 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9efc004800 0x7f9efc004c80 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f9f08009ac0 tx=0x7f9efc005830 comp rx=0 tx=0).stop 2024-09-12T03:19:13.998 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.995+0000 7f9f18853640 1 -- 172.21.15.27:0/1975306697 shutdown_connections 2024-09-12T03:19:13.998 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.995+0000 7f9f18853640 1 --2- 172.21.15.27:0/1975306697 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9efc004800 0x7f9efc004c80 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:13.999 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.995+0000 7f9f18853640 1 -- 172.21.15.27:0/1975306697 >> 172.21.15.27:0/1975306697 conn(0x7f9f14073800 msgr2=0x7f9f1409ba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:13.999 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.995+0000 7f9f18853640 1 -- 172.21.15.27:0/1975306697 shutdown_connections 2024-09-12T03:19:13.999 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.995+0000 7f9f18853640 1 -- 172.21.15.27:0/1975306697 wait complete. 2024-09-12T03:19:14.002 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.996+0000 7f9f18853640 1 Processor -- start 2024-09-12T03:19:14.002 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.996+0000 7f9f18853640 1 -- start start 2024-09-12T03:19:14.002 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.997+0000 7f9f18853640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9efc004800 0x7f9f140b54d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:14.002 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.997+0000 7f9f18853640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9f14160a60 0x7f9f141f4470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:14.002 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.997+0000 7f9f18853640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f9f141f49b0 0x7f9f141f4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:14.002 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.997+0000 7f9f18853640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f9f1406d1c0 con 0x7f9efc004800 2024-09-12T03:19:14.002 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.997+0000 7f9f18853640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f9f140d22f0 con 0x7f9f14160a60 2024-09-12T03:19:14.002 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.997+0000 7f9f18853640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f9f1406d340 con 0x7f9f141f49b0 2024-09-12T03:19:14.002 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.997+0000 7f9f1259c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9efc004800 0x7f9f140b54d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:14.002 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.997+0000 7f9f1259c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9efc004800 0x7f9f140b54d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55282/0 (socket says 172.21.15.27:55282) 2024-09-12T03:19:14.002 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.997+0000 7f9f1259c640 1 -- 172.21.15.27:0/3671082992 learned_addr learned my addr 172.21.15.27:0/3671082992 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:19:14.002 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.997+0000 7f9f11d9b640 1 --2- 172.21.15.27:0/3671082992 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9f14160a60 0x7f9f141f4470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:14.003 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.997+0000 7f9f12d9d640 1 --2- 172.21.15.27:0/3671082992 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f9f141f49b0 0x7f9f141f4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:14.003 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.998+0000 7f9f1259c640 1 -- 172.21.15.27:0/3671082992 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f9f141f49b0 msgr2=0x7f9f141f4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:14.003 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.998+0000 7f9f1259c640 1 --2- 172.21.15.27:0/3671082992 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f9f141f49b0 0x7f9f141f4e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:14.003 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.998+0000 7f9f1259c640 1 -- 172.21.15.27:0/3671082992 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9f14160a60 msgr2=0x7f9f141f4470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:14.003 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.998+0000 7f9f1259c640 1 --2- 172.21.15.27:0/3671082992 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9f14160a60 0x7f9f141f4470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:14.003 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.998+0000 7f9f1259c640 1 -- 172.21.15.27:0/3671082992 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f140b5b60 con 0x7f9efc004800 2024-09-12T03:19:14.003 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.998+0000 7f9f12d9d640 1 --2- 172.21.15.27:0/3671082992 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f9f141f49b0 0x7f9f141f4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:19:14.003 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.998+0000 7f9f1259c640 1 --2- 172.21.15.27:0/3671082992 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9efc004800 0x7f9f140b54d0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f9f04001b80 tx=0x7f9f04007720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:14.003 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.998+0000 7f9f02ffd640 1 -- 172.21.15.27:0/3671082992 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f0400f960 con 0x7f9efc004800 2024-09-12T03:19:14.003 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.998+0000 7f9f02ffd640 1 -- 172.21.15.27:0/3671082992 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f04018070 con 0x7f9efc004800 2024-09-12T03:19:14.003 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.998+0000 7f9f18853640 1 -- 172.21.15.27:0/3671082992 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f140b5e40 con 0x7f9efc004800 2024-09-12T03:19:14.003 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.998+0000 7f9f02ffd640 1 -- 172.21.15.27:0/3671082992 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f0400fd20 con 0x7f9efc004800 2024-09-12T03:19:14.004 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.998+0000 7f9f18853640 1 -- 172.21.15.27:0/3671082992 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f140b6400 con 0x7f9efc004800 2024-09-12T03:19:14.004 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:13.999+0000 7f9f18853640 1 -- 172.21.15.27:0/3671082992 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f140cfc40 con 0x7f9efc004800 2024-09-12T03:19:14.004 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.000+0000 7f9f02ffd640 1 -- 172.21.15.27:0/3671082992 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 8) ==== 90702+0+0 (secure 0 0 0) 0x7f9f0401d460 con 0x7f9efc004800 2024-09-12T03:19:14.004 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.000+0000 7f9f02ffd640 1 --2- 172.21.15.27:0/3671082992 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f9ef406c070 0x7f9ef406e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:14.004 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.000+0000 7f9f02ffd640 1 -- 172.21.15.27:0/3671082992 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5767+0+0 (secure 0 0 0) 0x7f9f0408b310 con 0x7f9efc004800 2024-09-12T03:19:14.004 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.001+0000 7f9f11d9b640 1 --2- 172.21.15.27:0/3671082992 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f9ef406c070 0x7f9ef406e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:14.004 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.001+0000 7f9f11d9b640 1 --2- 172.21.15.27:0/3671082992 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f9ef406c070 0x7f9ef406e530 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f9f08009ac0 tx=0x7f9f08008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:14.004 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.005+0000 7f9f02ffd640 1 -- 172.21.15.27:0/3671082992 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f9f040599b0 con 0x7f9efc004800 2024-09-12T03:19:14.180 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.180+0000 7f9f18853640 1 -- 172.21.15.27:0/3671082992 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9f140af5e0 con 0x7f9efc004800 2024-09-12T03:19:14.181 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.181+0000 7f9f02ffd640 1 -- 172.21.15.27:0/3671082992 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v45) ==== 74+0+16777 (secure 0 0 0) 0x7f9f040592b0 con 0x7f9efc004800 2024-09-12T03:19:14.181 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:19:14.182 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":45,"fsid":"d2df504e-2647-4385-a5db-9fdf69f487a0","created":"2024-09-12T03:16:07.778937+0000","modified":"2024-09-12T03:18:13.300695+0000","last_up_change":"2024-09-12T03:16:19.314778+0000","last_in_change":"2024-09-12T03:16:09.893086+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T03:16:20.097646+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-09-12T03:17:37.155092+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-09-12T03:17:37.552917+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1851116c-1fb3-4dc9-9772-f90e2ef4c355","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6817","nonce":3002321488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6819","nonce":3002321488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6823","nonce":3002321488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6821","nonce":3002321488}]},"public_addr":"172.21.15.27:6817/3002321488","cluster_addr":"172.21.15.27:6819/3002321488","heartbeat_back_addr":"172.21.15.27:6823/3002321488","heartbeat_front_addr":"172.21.15.27:6821/3002321488","state":["exists","up"]},{"osd":1,"uuid":"f38ae17d-73eb-4204-b99d-975fa806fb87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6809","nonce":1613102886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6811","nonce":1613102886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6815","nonce":1613102886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6813","nonce":1613102886}]},"public_addr":"172.21.15.27:6809/1613102886","cluster_addr":"172.21.15.27:6811/1613102886","heartbeat_back_addr":"172.21.15.27:6815/1613102886","heartbeat_front_addr":"172.21.15.27:6813/1613102886","state":["exists","up"]},{"osd":2,"uuid":"fb12dcc8-bdd0-42b8-b46d-8b10106224bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6825","nonce":3534723355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6827","nonce":3534723355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6831","nonce":3534723355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6829","nonce":3534723355}]},"public_addr":"172.21.15.27:6825/3534723355","cluster_addr":"172.21.15.27:6827/3534723355","heartbeat_back_addr":"172.21.15.27:6831/3534723355","heartbeat_front_addr":"172.21.15.27:6829/3534723355","state":["exists","up"]},{"osd":3,"uuid":"d74d3f49-9c24-4b9f-98e2-3ecd7b5ef3b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6801","nonce":80661598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6803","nonce":80661598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6807","nonce":80661598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6805","nonce":80661598}]},"public_addr":"172.21.15.27:6801/80661598","cluster_addr":"172.21.15.27:6803/80661598","heartbeat_back_addr":"172.21.15.27:6807/80661598","heartbeat_front_addr":"172.21.15.27:6805/80661598","state":["exists","up"]},{"osd":4,"uuid":"2bf123f2-1b0d-41c4-aa3b-805bab267e59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6817","nonce":3943189827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6819","nonce":3943189827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6823","nonce":3943189827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6821","nonce":3943189827}]},"public_addr":"172.21.15.45:6817/3943189827","cluster_addr":"172.21.15.45:6819/3943189827","heartbeat_back_addr":"172.21.15.45:6823/3943189827","heartbeat_front_addr":"172.21.15.45:6821/3943189827","state":["exists","up"]},{"osd":5,"uuid":"c49ba5da-91c6-4a86-a97b-4aba89c253ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6809","nonce":3620760330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6811","nonce":3620760330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6815","nonce":3620760330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6813","nonce":3620760330}]},"public_addr":"172.21.15.45:6809/3620760330","cluster_addr":"172.21.15.45:6811/3620760330","heartbeat_back_addr":"172.21.15.45:6815/3620760330","heartbeat_front_addr":"172.21.15.45:6813/3620760330","state":["exists","up"]},{"osd":6,"uuid":"b0123475-89f7-4a34-9e33-d797b99dd9dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6801","nonce":1989083510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6803","nonce":1989083510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6807","nonce":1989083510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6805","nonce":1989083510}]},"public_addr":"172.21.15.45:6801/1989083510","cluster_addr":"172.21.15.45:6803/1989083510","heartbeat_back_addr":"172.21.15.45:6807/1989083510","heartbeat_front_addr":"172.21.15.45:6805/1989083510","state":["exists","up"]},{"osd":7,"uuid":"4dad773d-b7d8-46fc-8a10-a972aad27b29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6825","nonce":1513300294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6827","nonce":1513300294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6831","nonce":1513300294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6829","nonce":1513300294}]},"public_addr":"172.21.15.45:6825/1513300294","cluster_addr":"172.21.15.45:6827/1513300294","heartbeat_back_addr":"172.21.15.45:6831/1513300294","heartbeat_front_addr":"172.21.15.45:6829/1513300294","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.881855+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:17.133252+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.735339+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.500742+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.477571+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:17.167667+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.470291+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.873743+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.38","mappings":[{"from":0,"to":4}]},{"pgid":"7.3b","mappings":[{"from":0,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T03:19:14.183 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.183+0000 7f9f18853640 1 -- 172.21.15.27:0/3671082992 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f9ef406c070 msgr2=0x7f9ef406e530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:14.183 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.183+0000 7f9f18853640 1 --2- 172.21.15.27:0/3671082992 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f9ef406c070 0x7f9ef406e530 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f9f08009ac0 tx=0x7f9f08008030 comp rx=0 tx=0).stop 2024-09-12T03:19:14.183 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.183+0000 7f9f18853640 1 -- 172.21.15.27:0/3671082992 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9efc004800 msgr2=0x7f9f140b54d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:14.183 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.183+0000 7f9f18853640 1 --2- 172.21.15.27:0/3671082992 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9efc004800 0x7f9f140b54d0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f9f04001b80 tx=0x7f9f04007720 comp rx=0 tx=0).stop 2024-09-12T03:19:14.183 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.183+0000 7f9f18853640 1 -- 172.21.15.27:0/3671082992 shutdown_connections 2024-09-12T03:19:14.184 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.183+0000 7f9f18853640 1 --2- 172.21.15.27:0/3671082992 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f9ef406c070 0x7f9ef406e530 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:14.184 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.183+0000 7f9f18853640 1 --2- 172.21.15.27:0/3671082992 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f9f141f49b0 0x7f9f141f4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:14.184 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.183+0000 7f9f18853640 1 --2- 172.21.15.27:0/3671082992 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9f14160a60 0x7f9f141f4470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:14.184 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.183+0000 7f9f18853640 1 --2- 172.21.15.27:0/3671082992 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9efc004800 0x7f9f140b54d0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:14.184 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.183+0000 7f9f18853640 1 -- 172.21.15.27:0/3671082992 >> 172.21.15.27:0/3671082992 conn(0x7f9f14073800 msgr2=0x7f9f140d2780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:14.184 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.183+0000 7f9f18853640 1 -- 172.21.15.27:0/3671082992 shutdown_connections 2024-09-12T03:19:14.184 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.183+0000 7f9f18853640 1 -- 172.21.15.27:0/3671082992 wait complete. 2024-09-12T03:19:14.194 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-12T03:19:14.194 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-12T03:19:14.297 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.297+0000 7f546550a640 1 Processor -- start 2024-09-12T03:19:14.297 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.297+0000 7f546550a640 1 -- start start 2024-09-12T03:19:14.298 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.298+0000 7f546550a640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f5460145090 0x7f54600affb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:14.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.298+0000 7f546550a640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f546006fe30 0x7f546006bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:14.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.298+0000 7f546550a640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f546015f260 con 0x7f546006bb30 2024-09-12T03:19:14.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.298+0000 7f546550a640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f546006c4e0 con 0x7f5460145090 2024-09-12T03:19:14.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.298+0000 7f546550a640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f546015f540 con 0x7f546006fe30 2024-09-12T03:19:14.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.298+0000 7f545effd640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f5460145090 0x7f54600affb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:14.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.298+0000 7f545e7fc640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f546006bb30 0x7f54600ae8d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:33914/0 (socket says 172.21.15.27:33914) 2024-09-12T03:19:14.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.298+0000 7f545effd640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f5460145090 0x7f54600affb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55288/0 (socket says 172.21.15.27:55288) 2024-09-12T03:19:14.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.298+0000 7f545e7fc640 1 -- 172.21.15.27:0/201318587 learned_addr learned my addr 172.21.15.27:0/201318587 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:19:14.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.299+0000 7f545f7fe640 1 --2- 172.21.15.27:0/201318587 >> v2:172.21.15.45:3300/0 conn(0x7f546006fe30 0x7f546006bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:14.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.299+0000 7f545effd640 1 -- 172.21.15.27:0/201318587 >> v2:172.21.15.45:3300/0 conn(0x7f546006fe30 msgr2=0x7f546006bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:14.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.299+0000 7f545effd640 1 --2- 172.21.15.27:0/201318587 >> v2:172.21.15.45:3300/0 conn(0x7f546006fe30 0x7f546006bfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:14.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.299+0000 7f545effd640 1 -- 172.21.15.27:0/201318587 >> v1:172.21.15.27:6789/0 conn(0x7f546006bb30 legacy=0x7f54600ae8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:14.300 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.299+0000 7f545dffb640 1 -- 172.21.15.27:0/201318587 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 166572620 0 0) 0x7f546015f260 con 0x7f546006bb30 2024-09-12T03:19:14.300 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.299+0000 7f545effd640 1 -- 172.21.15.27:0/201318587 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f546015f820 con 0x7f5460145090 2024-09-12T03:19:14.301 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.299+0000 7f545effd640 1 --2- 172.21.15.27:0/201318587 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5460145090 0x7f54600affb0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f546006cbe0 tx=0x7f545802a560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a8fe4e36fb94338 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:14.301 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.299+0000 7f545dffb640 1 -- 172.21.15.27:0/201318587 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5458007740 con 0x7f5460145090 2024-09-12T03:19:14.301 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.300+0000 7f545dffb640 1 -- 172.21.15.27:0/201318587 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54580078e0 con 0x7f5460145090 2024-09-12T03:19:14.301 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.300+0000 7f545dffb640 1 -- 172.21.15.27:0/201318587 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5458007cb0 con 0x7f5460145090 2024-09-12T03:19:14.301 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.300+0000 7f546550a640 1 -- 172.21.15.27:0/201318587 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5460145090 msgr2=0x7f54600affb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:14.301 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.300+0000 7f546550a640 1 --2- 172.21.15.27:0/201318587 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5460145090 0x7f54600affb0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f546006cbe0 tx=0x7f545802a560 comp rx=0 tx=0).stop 2024-09-12T03:19:14.301 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.300+0000 7f546550a640 1 -- 172.21.15.27:0/201318587 shutdown_connections 2024-09-12T03:19:14.301 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.300+0000 7f546550a640 1 --2- 172.21.15.27:0/201318587 >> v2:172.21.15.45:3300/0 conn(0x7f546006fe30 0x7f546006bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:14.301 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.300+0000 7f546550a640 1 --2- 172.21.15.27:0/201318587 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5460145090 0x7f54600affb0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:14.301 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.300+0000 7f546550a640 1 -- 172.21.15.27:0/201318587 >> 172.21.15.27:0/201318587 conn(0x7f546006d270 msgr2=0x7f54600b3270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:14.301 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.300+0000 7f546550a640 1 -- 172.21.15.27:0/201318587 shutdown_connections 2024-09-12T03:19:14.302 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.300+0000 7f546550a640 1 -- 172.21.15.27:0/201318587 wait complete. 2024-09-12T03:19:14.304 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.301+0000 7f546550a640 1 Processor -- start 2024-09-12T03:19:14.304 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.302+0000 7f546550a640 1 -- start start 2024-09-12T03:19:14.304 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.302+0000 7f546550a640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f546006bb30 0x7f5460144410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:14.304 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.302+0000 7f546550a640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f546006fe30 0x7f546013ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:14.304 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.302+0000 7f546550a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5460145090 0x7f54600b28c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:14.305 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.302+0000 7f546550a640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f546015fde0 con 0x7f546006fe30 2024-09-12T03:19:14.305 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.302+0000 7f546550a640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f546015ef50 con 0x7f5460145090 2024-09-12T03:19:14.305 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.302+0000 7f546550a640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f54600b04f0 con 0x7f546006bb30 2024-09-12T03:19:14.305 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.302+0000 7f545e7fc640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f546006fe30 0x7f546013ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:14.305 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.302+0000 7f545e7fc640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f546006fe30 0x7f546013ce80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55290/0 (socket says 172.21.15.27:55290) 2024-09-12T03:19:14.305 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.302+0000 7f545e7fc640 1 -- 172.21.15.27:0/2754734449 learned_addr learned my addr 172.21.15.27:0/2754734449 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:19:14.305 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.302+0000 7f545effd640 1 --2- 172.21.15.27:0/2754734449 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f546006bb30 0x7f5460144410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:14.305 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.302+0000 7f545f7fe640 1 --2- 172.21.15.27:0/2754734449 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5460145090 0x7f54600b28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:14.305 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.303+0000 7f545e7fc640 1 -- 172.21.15.27:0/2754734449 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f546006bb30 msgr2=0x7f5460144410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:14.305 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.303+0000 7f545e7fc640 1 --2- 172.21.15.27:0/2754734449 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f546006bb30 0x7f5460144410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:14.305 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.303+0000 7f545e7fc640 1 -- 172.21.15.27:0/2754734449 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5460145090 msgr2=0x7f54600b28c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:14.305 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.303+0000 7f545e7fc640 1 --2- 172.21.15.27:0/2754734449 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5460145090 0x7f54600b28c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:14.306 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.303+0000 7f545e7fc640 1 -- 172.21.15.27:0/2754734449 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54601f48c0 con 0x7f546006fe30 2024-09-12T03:19:14.306 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.303+0000 7f545e7fc640 1 --2- 172.21.15.27:0/2754734449 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f546006fe30 0x7f546013ce80 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f544800c460 tx=0x7f544800c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:14.306 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.303+0000 7f54577fe640 1 -- 172.21.15.27:0/2754734449 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54480074e0 con 0x7f546006fe30 2024-09-12T03:19:14.306 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.303+0000 7f54577fe640 1 -- 172.21.15.27:0/2754734449 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54480047e0 con 0x7f546006fe30 2024-09-12T03:19:14.306 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.303+0000 7f54577fe640 1 -- 172.21.15.27:0/2754734449 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5448004bd0 con 0x7f546006fe30 2024-09-12T03:19:14.306 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.303+0000 7f546550a640 1 -- 172.21.15.27:0/2754734449 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54601f4ba0 con 0x7f546006fe30 2024-09-12T03:19:14.306 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.303+0000 7f546550a640 1 -- 172.21.15.27:0/2754734449 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54601f5160 con 0x7f546006fe30 2024-09-12T03:19:14.306 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.304+0000 7f546550a640 1 -- 172.21.15.27:0/2754734449 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54600afad0 con 0x7f546006fe30 2024-09-12T03:19:14.307 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.305+0000 7f54577fe640 1 -- 172.21.15.27:0/2754734449 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 8) ==== 90702+0+0 (secure 0 0 0) 0x7f5448011070 con 0x7f546006fe30 2024-09-12T03:19:14.307 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.305+0000 7f54577fe640 1 --2- 172.21.15.27:0/2754734449 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f544006c070 0x7f544006e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:14.307 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.305+0000 7f54577fe640 1 -- 172.21.15.27:0/2754734449 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5767+0+0 (secure 0 0 0) 0x7f544808ad80 con 0x7f546006fe30 2024-09-12T03:19:14.307 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.305+0000 7f545effd640 1 --2- 172.21.15.27:0/2754734449 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f544006c070 0x7f544006e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:14.307 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.306+0000 7f545effd640 1 --2- 172.21.15.27:0/2754734449 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f544006c070 0x7f544006e530 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f5458034000 tx=0x7f545802f000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:14.309 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.309+0000 7f54577fe640 1 -- 172.21.15.27:0/2754734449 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f5448058d70 con 0x7f546006fe30 2024-09-12T03:19:14.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.477+0000 7f54577fe640 1 -- 172.21.15.27:0/2754734449 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mgrmap(e 9) ==== 90651+0+0 (secure 0 0 0) 0x7f54480553e0 con 0x7f546006fe30 2024-09-12T03:19:14.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.520+0000 7f546550a640 1 -- 172.21.15.27:0/2754734449 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f54600b02d0 con 0x7f546006fe30 2024-09-12T03:19:14.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.521+0000 7f54577fe640 1 -- 172.21.15.27:0/2754734449 <== mon.0 v2:172.21.15.27:3300/0 8 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+5223 (secure 0 0 0) 0x7f54480133c0 con 0x7f546006fe30 2024-09-12T03:19:14.521 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:19:14.521 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":23,"btime":"2024-09-12T03:17:47:237558+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4835,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6834/1457797481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":1457797481},{"type":"v1","addr":"172.21.15.27:6834","nonce":1457797481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5029,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6836/2842491281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":2842491281},{"type":"v1","addr":"172.21.15.45:6836","nonce":2842491281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5032,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/3339744340","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6835","nonce":3339744340},{"type":"v1","addr":"172.21.15.45:6837","nonce":3339744340}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-12T03:17:38.580473+0000","modified":"2024-09-12T03:17:40.557950+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5205},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5205":{"gid":5205,"name":"a","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.27:6835/2571101228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":2571101228},{"type":"v1","addr":"172.21.15.27:6835","nonce":2571101228}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5205,"qdb_cluster":[5205]},"id":3}]} 2024-09-12T03:19:14.521 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 23 2024-09-12T03:19:14.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.523+0000 7f546550a640 1 -- 172.21.15.27:0/2754734449 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f544006c070 msgr2=0x7f544006e530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:14.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.523+0000 7f546550a640 1 --2- 172.21.15.27:0/2754734449 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f544006c070 0x7f544006e530 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f5458034000 tx=0x7f545802f000 comp rx=0 tx=0).stop 2024-09-12T03:19:14.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.523+0000 7f546550a640 1 -- 172.21.15.27:0/2754734449 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f546006fe30 msgr2=0x7f546013ce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:14.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.523+0000 7f546550a640 1 --2- 172.21.15.27:0/2754734449 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f546006fe30 0x7f546013ce80 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f544800c460 tx=0x7f544800c920 comp rx=0 tx=0).stop 2024-09-12T03:19:14.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.523+0000 7f546550a640 1 -- 172.21.15.27:0/2754734449 shutdown_connections 2024-09-12T03:19:14.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.523+0000 7f546550a640 1 --2- 172.21.15.27:0/2754734449 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f544006c070 0x7f544006e530 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:14.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.523+0000 7f546550a640 1 --2- 172.21.15.27:0/2754734449 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5460145090 0x7f54600b28c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:14.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.523+0000 7f546550a640 1 --2- 172.21.15.27:0/2754734449 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f546006fe30 0x7f546013ce80 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:14.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.523+0000 7f546550a640 1 --2- 172.21.15.27:0/2754734449 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f546006bb30 0x7f5460144410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:14.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.523+0000 7f546550a640 1 -- 172.21.15.27:0/2754734449 >> 172.21.15.27:0/2754734449 conn(0x7f546006d270 msgr2=0x7f546015ff60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:14.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.523+0000 7f546550a640 1 -- 172.21.15.27:0/2754734449 shutdown_connections 2024-09-12T03:19:14.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.523+0000 7f546550a640 1 -- 172.21.15.27:0/2754734449 wait complete. 2024-09-12T03:19:14.534 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-12T03:19:14.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.636+0000 7f82fdcb7640 1 Processor -- start 2024-09-12T03:19:14.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.636+0000 7f82fdcb7640 1 -- start start 2024-09-12T03:19:14.638 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.637+0000 7f82fdcb7640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f82f815b2b0 0x7f82f80b3560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:14.638 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.637+0000 7f82fdcb7640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f82f8072d70 0x7f82f8072430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:14.638 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.637+0000 7f82fdcb7640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f82f80ce410 con 0x7f82f806e9b0 2024-09-12T03:19:14.638 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.637+0000 7f82fdcb7640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f82f80720a0 con 0x7f82f8072d70 2024-09-12T03:19:14.638 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.637+0000 7f82fdcb7640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f82f8072970 con 0x7f82f815b2b0 2024-09-12T03:19:14.638 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.637+0000 7f82effff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f82f8072d70 0x7f82f8072430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:14.638 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.637+0000 7f82effff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f82f8072d70 0x7f82f8072430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55302/0 (socket says 172.21.15.27:55302) 2024-09-12T03:19:14.638 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.637+0000 7f82effff640 1 -- 172.21.15.27:0/2827024841 learned_addr learned my addr 172.21.15.27:0/2827024841 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:19:14.638 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.638+0000 7f82f77fe640 1 --2- 172.21.15.27:0/2827024841 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f82f815b2b0 0x7f82f80b3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:14.638 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.638+0000 7f82effff640 1 -- 172.21.15.27:0/2827024841 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f82f815b2b0 msgr2=0x7f82f80b3560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:14.639 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.638+0000 7f82effff640 1 --2- 172.21.15.27:0/2827024841 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f82f815b2b0 0x7f82f80b3560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:14.639 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.638+0000 7f82effff640 1 -- 172.21.15.27:0/2827024841 >> v1:172.21.15.45:6789/0 conn(0x7f82f806e9b0 legacy=0x7f82f80cebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:14.639 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.638+0000 7f82effff640 1 -- 172.21.15.27:0/2827024841 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82f80732c0 con 0x7f82f8072d70 2024-09-12T03:19:14.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.638+0000 7f82effff640 1 --2- 172.21.15.27:0/2827024841 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f82f8072d70 0x7f82f8072430 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f82e0009870 tx=0x7f82e002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d58e8cdbad4cb724 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:14.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.639+0000 7f82f6ffd640 1 -- 172.21.15.27:0/2827024841 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82e0037070 con 0x7f82f8072d70 2024-09-12T03:19:14.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.639+0000 7f82f6ffd640 1 -- 172.21.15.27:0/2827024841 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82e0032440 con 0x7f82f8072d70 2024-09-12T03:19:14.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.639+0000 7f82f6ffd640 1 -- 172.21.15.27:0/2827024841 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82e0032810 con 0x7f82f8072d70 2024-09-12T03:19:14.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.639+0000 7f82fdcb7640 1 -- 172.21.15.27:0/2827024841 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f82f8072d70 msgr2=0x7f82f8072430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:14.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.639+0000 7f82fdcb7640 1 --2- 172.21.15.27:0/2827024841 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f82f8072d70 0x7f82f8072430 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f82e0009870 tx=0x7f82e002a210 comp rx=0 tx=0).stop 2024-09-12T03:19:14.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.639+0000 7f82fdcb7640 1 -- 172.21.15.27:0/2827024841 shutdown_connections 2024-09-12T03:19:14.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.639+0000 7f82fdcb7640 1 --2- 172.21.15.27:0/2827024841 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f82f8072d70 0x7f82f8072430 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:14.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.639+0000 7f82fdcb7640 1 --2- 172.21.15.27:0/2827024841 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f82f815b2b0 0x7f82f80b3560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:14.641 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.639+0000 7f82fdcb7640 1 -- 172.21.15.27:0/2827024841 >> 172.21.15.27:0/2827024841 conn(0x7f82f8062c60 msgr2=0x7f82f806bb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:14.641 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.639+0000 7f82fdcb7640 1 -- 172.21.15.27:0/2827024841 shutdown_connections 2024-09-12T03:19:14.641 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.639+0000 7f82fdcb7640 1 -- 172.21.15.27:0/2827024841 wait complete. 2024-09-12T03:19:14.643 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.641+0000 7f82fdcb7640 1 Processor -- start 2024-09-12T03:19:14.643 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.641+0000 7f82fdcb7640 1 -- start start 2024-09-12T03:19:14.644 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.641+0000 7f82fdcb7640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f82f806e9b0 0x7f82f814da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:14.644 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.641+0000 7f82fdcb7640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f82f8072d70 0x7f82f815a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:14.644 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.641+0000 7f82fdcb7640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f82f815b2b0 0x7f82f815d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:14.644 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.641+0000 7f82fdcb7640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f82f80ce690 con 0x7f82f815b2b0 2024-09-12T03:19:14.644 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.641+0000 7f82fdcb7640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f82f8071dc0 con 0x7f82f806e9b0 2024-09-12T03:19:14.644 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.641+0000 7f82fdcb7640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f82f815b690 con 0x7f82f8072d70 2024-09-12T03:19:14.644 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.641+0000 7f82f7fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f82f815b2b0 0x7f82f815d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:14.644 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.641+0000 7f82f7fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f82f815b2b0 0x7f82f815d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55316/0 (socket says 172.21.15.27:55316) 2024-09-12T03:19:14.644 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.641+0000 7f82f7fff640 1 -- 172.21.15.27:0/2799321499 learned_addr learned my addr 172.21.15.27:0/2799321499 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:19:14.644 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.641+0000 7f82f77fe640 1 --2- 172.21.15.27:0/2799321499 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f82f806e9b0 0x7f82f814da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:14.644 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.641+0000 7f82effff640 1 --2- 172.21.15.27:0/2799321499 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f82f8072d70 0x7f82f815a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:14.644 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.642+0000 7f82f7fff640 1 -- 172.21.15.27:0/2799321499 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f82f8072d70 msgr2=0x7f82f815a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:14.644 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.642+0000 7f82f7fff640 1 --2- 172.21.15.27:0/2799321499 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f82f8072d70 0x7f82f815a2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:14.645 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.642+0000 7f82f7fff640 1 -- 172.21.15.27:0/2799321499 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f82f806e9b0 msgr2=0x7f82f814da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:14.645 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.642+0000 7f82f7fff640 1 --2- 172.21.15.27:0/2799321499 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f82f806e9b0 0x7f82f814da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:14.645 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.642+0000 7f82f7fff640 1 -- 172.21.15.27:0/2799321499 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82f8159260 con 0x7f82f815b2b0 2024-09-12T03:19:14.645 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.642+0000 7f82f7fff640 1 --2- 172.21.15.27:0/2799321499 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f82f815b2b0 0x7f82f815d5d0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f82f000c780 tx=0x7f82f000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:14.645 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.642+0000 7f82f4ff9640 1 -- 172.21.15.27:0/2799321499 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82f0004b60 con 0x7f82f815b2b0 2024-09-12T03:19:14.645 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.642+0000 7f82f4ff9640 1 -- 172.21.15.27:0/2799321499 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82f0004d00 con 0x7f82f815b2b0 2024-09-12T03:19:14.645 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.642+0000 7f82fdcb7640 1 -- 172.21.15.27:0/2799321499 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82f8159500 con 0x7f82f815b2b0 2024-09-12T03:19:14.645 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.642+0000 7f82f4ff9640 1 -- 172.21.15.27:0/2799321499 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82f000aa90 con 0x7f82f815b2b0 2024-09-12T03:19:14.645 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.642+0000 7f82fdcb7640 1 -- 172.21.15.27:0/2799321499 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82f8159ac0 con 0x7f82f815b2b0 2024-09-12T03:19:14.645 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.643+0000 7f82fdcb7640 1 -- 172.21.15.27:0/2799321499 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82f8072430 con 0x7f82f815b2b0 2024-09-12T03:19:14.645 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.644+0000 7f82f4ff9640 1 -- 172.21.15.27:0/2799321499 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 9) ==== 90651+0+0 (secure 0 0 0) 0x7f82f000ac30 con 0x7f82f815b2b0 2024-09-12T03:19:14.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.644+0000 7f82f4ff9640 1 --2- 172.21.15.27:0/2799321499 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f82d806bfe0 0x7f82d806e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:14.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.644+0000 7f82f4ff9640 1 -- 172.21.15.27:0/2799321499 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5767+0+0 (secure 0 0 0) 0x7f82f008b590 con 0x7f82f815b2b0 2024-09-12T03:19:14.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.644+0000 7f82f77fe640 1 --2- 172.21.15.27:0/2799321499 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f82d806bfe0 0x7f82d806e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:14.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.645+0000 7f82f77fe640 1 --2- 172.21.15.27:0/2799321499 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f82d806bfe0 0x7f82d806e4a0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f82e80079a0 tx=0x7f82e8008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:14.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.649+0000 7f82f4ff9640 1 -- 172.21.15.27:0/2799321499 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f82f0059630 con 0x7f82f815b2b0 2024-09-12T03:19:14.860 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.860+0000 7f82fdcb7640 1 -- 172.21.15.27:0/2799321499 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f82f815a800 con 0x7f82f815b2b0 2024-09-12T03:19:14.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.861+0000 7f82f4ff9640 1 -- 172.21.15.27:0/2799321499 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+5223 (secure 0 0 0) 0x7f82f005d930 con 0x7f82f815b2b0 2024-09-12T03:19:14.861 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:19:14.861 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":23,"btime":"2024-09-12T03:17:47:237558+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4835,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6834/1457797481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":1457797481},{"type":"v1","addr":"172.21.15.27:6834","nonce":1457797481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5029,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6836/2842491281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":2842491281},{"type":"v1","addr":"172.21.15.45:6836","nonce":2842491281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5032,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/3339744340","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6835","nonce":3339744340},{"type":"v1","addr":"172.21.15.45:6837","nonce":3339744340}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-12T03:17:38.580473+0000","modified":"2024-09-12T03:17:40.557950+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5205},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5205":{"gid":5205,"name":"a","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.27:6835/2571101228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":2571101228},{"type":"v1","addr":"172.21.15.27:6835","nonce":2571101228}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5205,"qdb_cluster":[5205]},"id":3}]} 2024-09-12T03:19:14.863 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 23 2024-09-12T03:19:14.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.862+0000 7f82fdcb7640 1 -- 172.21.15.27:0/2799321499 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f82d806bfe0 msgr2=0x7f82d806e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:14.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.862+0000 7f82fdcb7640 1 --2- 172.21.15.27:0/2799321499 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f82d806bfe0 0x7f82d806e4a0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f82e80079a0 tx=0x7f82e8008040 comp rx=0 tx=0).stop 2024-09-12T03:19:14.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.862+0000 7f82fdcb7640 1 -- 172.21.15.27:0/2799321499 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f82f815b2b0 msgr2=0x7f82f815d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:14.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.862+0000 7f82fdcb7640 1 --2- 172.21.15.27:0/2799321499 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f82f815b2b0 0x7f82f815d5d0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f82f000c780 tx=0x7f82f000cc40 comp rx=0 tx=0).stop 2024-09-12T03:19:14.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.863+0000 7f82fdcb7640 1 -- 172.21.15.27:0/2799321499 shutdown_connections 2024-09-12T03:19:14.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.863+0000 7f82fdcb7640 1 --2- 172.21.15.27:0/2799321499 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f82d806bfe0 0x7f82d806e4a0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:14.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.863+0000 7f82fdcb7640 1 --2- 172.21.15.27:0/2799321499 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f82f815b2b0 0x7f82f815d5d0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:14.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.863+0000 7f82fdcb7640 1 --2- 172.21.15.27:0/2799321499 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f82f8072d70 0x7f82f815a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:14.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.863+0000 7f82fdcb7640 1 --2- 172.21.15.27:0/2799321499 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f82f806e9b0 0x7f82f814da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:14.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.863+0000 7f82fdcb7640 1 -- 172.21.15.27:0/2799321499 >> 172.21.15.27:0/2799321499 conn(0x7f82f8062c60 msgr2=0x7f82f80b1a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:14.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.863+0000 7f82fdcb7640 1 -- 172.21.15.27:0/2799321499 shutdown_connections 2024-09-12T03:19:14.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.863+0000 7f82fdcb7640 1 -- 172.21.15.27:0/2799321499 wait complete. 2024-09-12T03:19:14.874 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-12T03:19:14.975 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.975+0000 7eff8a542640 1 Processor -- start 2024-09-12T03:19:14.976 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.976+0000 7eff8a542640 1 -- start start 2024-09-12T03:19:14.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.976+0000 7eff8a542640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7eff8415eb90 0x7eff840d2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:14.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.976+0000 7eff8a542640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7eff840d1720 0x7eff840b8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:14.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.976+0000 7eff8a542640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7eff840d3760 con 0x7eff840d5200 2024-09-12T03:19:14.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.976+0000 7eff8a542640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7eff840b7d20 con 0x7eff840d1720 2024-09-12T03:19:14.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.976+0000 7eff8a542640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7eff840d42e0 con 0x7eff8415eb90 2024-09-12T03:19:14.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.976+0000 7eff88adf640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7eff840d1720 0x7eff840b8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:14.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.976+0000 7eff837fe640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7eff840d5200 0x7eff840d1010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:33930/0 (socket says 172.21.15.27:33930) 2024-09-12T03:19:14.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.976+0000 7eff837fe640 1 -- 172.21.15.27:0/116672803 learned_addr learned my addr 172.21.15.27:0/116672803 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:19:14.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.977+0000 7eff83fff640 1 --2- 172.21.15.27:0/116672803 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7eff8415eb90 0x7eff840d2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:14.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.977+0000 7eff88adf640 1 -- 172.21.15.27:0/116672803 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7eff8415eb90 msgr2=0x7eff840d2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:14.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.977+0000 7eff88adf640 1 --2- 172.21.15.27:0/116672803 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7eff8415eb90 0x7eff840d2df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:14.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.977+0000 7eff88adf640 1 -- 172.21.15.27:0/116672803 >> v1:172.21.15.27:6789/0 conn(0x7eff840d5200 legacy=0x7eff840d1010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:14.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.977+0000 7eff88adf640 1 -- 172.21.15.27:0/116672803 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff840d3a40 con 0x7eff840d1720 2024-09-12T03:19:14.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.977+0000 7eff82ffd640 1 -- 172.21.15.27:0/116672803 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1527871302 0 0) 0x7eff840d3760 con 0x7eff840d5200 2024-09-12T03:19:14.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.978+0000 7eff88adf640 1 --2- 172.21.15.27:0/116672803 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7eff840d1720 0x7eff840b8190 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7eff7c009d90 tx=0x7eff7c02a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fce8cbc0bb927e59 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:14.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.978+0000 7eff82ffd640 1 -- 172.21.15.27:0/116672803 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff7c007740 con 0x7eff840d1720 2024-09-12T03:19:14.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.978+0000 7eff82ffd640 1 -- 172.21.15.27:0/116672803 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff7c0078e0 con 0x7eff840d1720 2024-09-12T03:19:14.980 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.978+0000 7eff82ffd640 1 -- 172.21.15.27:0/116672803 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff7c007cb0 con 0x7eff840d1720 2024-09-12T03:19:14.980 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.978+0000 7eff8a542640 1 -- 172.21.15.27:0/116672803 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7eff840d1720 msgr2=0x7eff840b8190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:14.980 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.978+0000 7eff8a542640 1 --2- 172.21.15.27:0/116672803 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7eff840d1720 0x7eff840b8190 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7eff7c009d90 tx=0x7eff7c02a660 comp rx=0 tx=0).stop 2024-09-12T03:19:14.980 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.978+0000 7eff8a542640 1 -- 172.21.15.27:0/116672803 shutdown_connections 2024-09-12T03:19:14.980 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.978+0000 7eff8a542640 1 --2- 172.21.15.27:0/116672803 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7eff840d1720 0x7eff840b8190 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:14.980 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.978+0000 7eff8a542640 1 --2- 172.21.15.27:0/116672803 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7eff8415eb90 0x7eff840d2df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:14.980 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.978+0000 7eff8a542640 1 -- 172.21.15.27:0/116672803 >> 172.21.15.27:0/116672803 conn(0x7eff8406dce0 msgr2=0x7eff8406e0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:14.980 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.978+0000 7eff8a542640 1 -- 172.21.15.27:0/116672803 shutdown_connections 2024-09-12T03:19:14.980 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.978+0000 7eff8a542640 1 -- 172.21.15.27:0/116672803 wait complete. 2024-09-12T03:19:14.980 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.979+0000 7eff8a542640 1 Processor -- start 2024-09-12T03:19:14.981 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.980+0000 7eff8a542640 1 -- start start 2024-09-12T03:19:14.984 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.980+0000 7eff8a542640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7eff840d1720 0x7eff840ce7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:14.984 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.980+0000 7eff8a542640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7eff840d5200 0x7eff84158ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:14.984 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.980+0000 7eff8a542640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7eff8415eb90 0x7eff84159420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:14.984 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.980+0000 7eff8a542640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7eff840d3330 con 0x7eff840d1720 2024-09-12T03:19:14.984 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.980+0000 7eff8a542640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7eff840d4000 con 0x7eff8415eb90 2024-09-12T03:19:14.984 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.980+0000 7eff8a542640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7eff840d34b0 con 0x7eff840d5200 2024-09-12T03:19:14.984 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.980+0000 7eff83fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7eff840d1720 0x7eff840ce7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:14.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.981+0000 7eff83fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7eff840d1720 0x7eff840ce7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55330/0 (socket says 172.21.15.27:55330) 2024-09-12T03:19:14.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.981+0000 7eff83fff640 1 -- 172.21.15.27:0/743371694 learned_addr learned my addr 172.21.15.27:0/743371694 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:19:14.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.981+0000 7eff88adf640 1 --2- 172.21.15.27:0/743371694 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7eff8415eb90 0x7eff84159420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:14.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.981+0000 7eff837fe640 1 --2- 172.21.15.27:0/743371694 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7eff840d5200 0x7eff84158ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:14.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.981+0000 7eff83fff640 1 -- 172.21.15.27:0/743371694 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7eff840d5200 msgr2=0x7eff84158ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:14.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.981+0000 7eff83fff640 1 --2- 172.21.15.27:0/743371694 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7eff840d5200 0x7eff84158ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:14.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.981+0000 7eff83fff640 1 -- 172.21.15.27:0/743371694 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7eff8415eb90 msgr2=0x7eff84159420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:14.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.981+0000 7eff83fff640 1 --2- 172.21.15.27:0/743371694 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7eff8415eb90 0x7eff84159420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:14.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.981+0000 7eff83fff640 1 -- 172.21.15.27:0/743371694 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff84158180 con 0x7eff840d1720 2024-09-12T03:19:14.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.981+0000 7eff88adf640 1 --2- 172.21.15.27:0/743371694 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7eff8415eb90 0x7eff84159420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:19:14.986 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.981+0000 7eff837fe640 1 --2- 172.21.15.27:0/743371694 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7eff840d5200 0x7eff84158ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:19:14.986 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.981+0000 7eff83fff640 1 --2- 172.21.15.27:0/743371694 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7eff840d1720 0x7eff840ce7d0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7eff7400d5f0 tx=0x7eff7400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:14.986 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.981+0000 7eff80ff9640 1 -- 172.21.15.27:0/743371694 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff740125f0 con 0x7eff840d1720 2024-09-12T03:19:14.986 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.981+0000 7eff80ff9640 1 -- 172.21.15.27:0/743371694 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff74019070 con 0x7eff840d1720 2024-09-12T03:19:14.986 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.981+0000 7eff8a542640 1 -- 172.21.15.27:0/743371694 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff840cbf80 con 0x7eff840d1720 2024-09-12T03:19:14.986 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.981+0000 7eff80ff9640 1 -- 172.21.15.27:0/743371694 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff740129d0 con 0x7eff840d1720 2024-09-12T03:19:14.986 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.981+0000 7eff8a542640 1 -- 172.21.15.27:0/743371694 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff84158400 con 0x7eff840d1720 2024-09-12T03:19:14.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.982+0000 7eff8a542640 1 -- 172.21.15.27:0/743371694 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff8415f850 con 0x7eff840d1720 2024-09-12T03:19:14.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.983+0000 7eff80ff9640 1 -- 172.21.15.27:0/743371694 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 9) ==== 90651+0+0 (secure 0 0 0) 0x7eff74012b70 con 0x7eff840d1720 2024-09-12T03:19:14.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.983+0000 7eff80ff9640 1 --2- 172.21.15.27:0/743371694 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7eff7006bfe0 0x7eff7006e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:14.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.983+0000 7eff80ff9640 1 -- 172.21.15.27:0/743371694 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5767+0+0 (secure 0 0 0) 0x7eff7408b530 con 0x7eff840d1720 2024-09-12T03:19:14.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.983+0000 7eff837fe640 1 --2- 172.21.15.27:0/743371694 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7eff7006bfe0 0x7eff7006e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:14.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.984+0000 7eff837fe640 1 --2- 172.21.15.27:0/743371694 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7eff7006bfe0 0x7eff7006e4a0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7eff84161320 tx=0x7eff78007c10 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:14.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:14.987+0000 7eff80ff9640 1 -- 172.21.15.27:0/743371694 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7eff84158180 con 0x7eff840d1720 2024-09-12T03:19:15.162 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:15.162+0000 7eff8a542640 1 -- 172.21.15.27:0/743371694 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7eff840ba440 con 0x7eff840d1720 2024-09-12T03:19:15.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:15.163+0000 7eff80ff9640 1 -- 172.21.15.27:0/743371694 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v45) ==== 74+0+16777 (secure 0 0 0) 0x7eff74059cb0 con 0x7eff840d1720 2024-09-12T03:19:15.163 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:19:15.163 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":45,"fsid":"d2df504e-2647-4385-a5db-9fdf69f487a0","created":"2024-09-12T03:16:07.778937+0000","modified":"2024-09-12T03:18:13.300695+0000","last_up_change":"2024-09-12T03:16:19.314778+0000","last_in_change":"2024-09-12T03:16:09.893086+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T03:16:20.097646+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-09-12T03:17:37.155092+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-09-12T03:17:37.552917+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1851116c-1fb3-4dc9-9772-f90e2ef4c355","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6817","nonce":3002321488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6819","nonce":3002321488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6823","nonce":3002321488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6821","nonce":3002321488}]},"public_addr":"172.21.15.27:6817/3002321488","cluster_addr":"172.21.15.27:6819/3002321488","heartbeat_back_addr":"172.21.15.27:6823/3002321488","heartbeat_front_addr":"172.21.15.27:6821/3002321488","state":["exists","up"]},{"osd":1,"uuid":"f38ae17d-73eb-4204-b99d-975fa806fb87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6809","nonce":1613102886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6811","nonce":1613102886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6815","nonce":1613102886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6813","nonce":1613102886}]},"public_addr":"172.21.15.27:6809/1613102886","cluster_addr":"172.21.15.27:6811/1613102886","heartbeat_back_addr":"172.21.15.27:6815/1613102886","heartbeat_front_addr":"172.21.15.27:6813/1613102886","state":["exists","up"]},{"osd":2,"uuid":"fb12dcc8-bdd0-42b8-b46d-8b10106224bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6825","nonce":3534723355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6827","nonce":3534723355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6831","nonce":3534723355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6829","nonce":3534723355}]},"public_addr":"172.21.15.27:6825/3534723355","cluster_addr":"172.21.15.27:6827/3534723355","heartbeat_back_addr":"172.21.15.27:6831/3534723355","heartbeat_front_addr":"172.21.15.27:6829/3534723355","state":["exists","up"]},{"osd":3,"uuid":"d74d3f49-9c24-4b9f-98e2-3ecd7b5ef3b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6801","nonce":80661598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6803","nonce":80661598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6807","nonce":80661598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6805","nonce":80661598}]},"public_addr":"172.21.15.27:6801/80661598","cluster_addr":"172.21.15.27:6803/80661598","heartbeat_back_addr":"172.21.15.27:6807/80661598","heartbeat_front_addr":"172.21.15.27:6805/80661598","state":["exists","up"]},{"osd":4,"uuid":"2bf123f2-1b0d-41c4-aa3b-805bab267e59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6817","nonce":3943189827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6819","nonce":3943189827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6823","nonce":3943189827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6821","nonce":3943189827}]},"public_addr":"172.21.15.45:6817/3943189827","cluster_addr":"172.21.15.45:6819/3943189827","heartbeat_back_addr":"172.21.15.45:6823/3943189827","heartbeat_front_addr":"172.21.15.45:6821/3943189827","state":["exists","up"]},{"osd":5,"uuid":"c49ba5da-91c6-4a86-a97b-4aba89c253ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6809","nonce":3620760330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6811","nonce":3620760330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6815","nonce":3620760330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6813","nonce":3620760330}]},"public_addr":"172.21.15.45:6809/3620760330","cluster_addr":"172.21.15.45:6811/3620760330","heartbeat_back_addr":"172.21.15.45:6815/3620760330","heartbeat_front_addr":"172.21.15.45:6813/3620760330","state":["exists","up"]},{"osd":6,"uuid":"b0123475-89f7-4a34-9e33-d797b99dd9dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6801","nonce":1989083510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6803","nonce":1989083510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6807","nonce":1989083510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6805","nonce":1989083510}]},"public_addr":"172.21.15.45:6801/1989083510","cluster_addr":"172.21.15.45:6803/1989083510","heartbeat_back_addr":"172.21.15.45:6807/1989083510","heartbeat_front_addr":"172.21.15.45:6805/1989083510","state":["exists","up"]},{"osd":7,"uuid":"4dad773d-b7d8-46fc-8a10-a972aad27b29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6825","nonce":1513300294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6827","nonce":1513300294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6831","nonce":1513300294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6829","nonce":1513300294}]},"public_addr":"172.21.15.45:6825/1513300294","cluster_addr":"172.21.15.45:6827/1513300294","heartbeat_back_addr":"172.21.15.45:6831/1513300294","heartbeat_front_addr":"172.21.15.45:6829/1513300294","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.881855+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:17.133252+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.735339+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.500742+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.477571+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:17.167667+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.470291+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.873743+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.38","mappings":[{"from":0,"to":4}]},{"pgid":"7.3b","mappings":[{"from":0,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T03:19:15.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:15.165+0000 7eff8a542640 1 -- 172.21.15.27:0/743371694 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7eff7006bfe0 msgr2=0x7eff7006e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:15.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:15.165+0000 7eff8a542640 1 --2- 172.21.15.27:0/743371694 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7eff7006bfe0 0x7eff7006e4a0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7eff84161320 tx=0x7eff78007c10 comp rx=0 tx=0).stop 2024-09-12T03:19:15.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:15.165+0000 7eff8a542640 1 -- 172.21.15.27:0/743371694 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7eff840d1720 msgr2=0x7eff840ce7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:15.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:15.165+0000 7eff8a542640 1 --2- 172.21.15.27:0/743371694 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7eff840d1720 0x7eff840ce7d0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7eff7400d5f0 tx=0x7eff7400dab0 comp rx=0 tx=0).stop 2024-09-12T03:19:15.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:15.165+0000 7eff8a542640 1 -- 172.21.15.27:0/743371694 shutdown_connections 2024-09-12T03:19:15.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:15.165+0000 7eff8a542640 1 --2- 172.21.15.27:0/743371694 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7eff7006bfe0 0x7eff7006e4a0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:15.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:15.165+0000 7eff8a542640 1 --2- 172.21.15.27:0/743371694 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7eff8415eb90 0x7eff84159420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:15.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:15.165+0000 7eff8a542640 1 --2- 172.21.15.27:0/743371694 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7eff840d5200 0x7eff84158ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:15.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:15.165+0000 7eff8a542640 1 --2- 172.21.15.27:0/743371694 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7eff840d1720 0x7eff840ce7d0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:15.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:15.165+0000 7eff8a542640 1 -- 172.21.15.27:0/743371694 >> 172.21.15.27:0/743371694 conn(0x7eff8406dce0 msgr2=0x7eff84161b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:15.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:15.165+0000 7eff8a542640 1 -- 172.21.15.27:0/743371694 shutdown_connections 2024-09-12T03:19:15.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:15.165+0000 7eff8a542640 1 -- 172.21.15.27:0/743371694 wait complete. 2024-09-12T03:19:15.176 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-12T03:19:15.280 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:15.280+0000 7fecfed0a640 1 Processor -- start 2024-09-12T03:19:15.281 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:15.280+0000 7fecfed0a640 1 -- start start 2024-09-12T03:19:15.282 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:15.281+0000 7fecfed0a640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fecf80d5200 0x7fecf80d1010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:15.282 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:15.281+0000 7fecfed0a640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fecf80d1550 0x7fecf80b8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:15.282 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:15.281+0000 7fecfed0a640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fecf80d3760 con 0x7fecf815eb90 2024-09-12T03:19:15.282 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:15.281+0000 7fecfed0a640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7fecf80b7d20 con 0x7fecf80d5200 2024-09-12T03:19:15.282 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:15.281+0000 7fecfed0a640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fecf80d42e0 con 0x7fecf80d1550 2024-09-12T03:19:15.282 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:15.281+0000 7fecf7fff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fecf80d5200 0x7fecf80d1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:15.282 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:15.281+0000 7fecf7fff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fecf80d5200 0x7fecf80d1010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55344/0 (socket says 172.21.15.27:55344) 2024-09-12T03:19:15.282 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:15.281+0000 7fecf7fff640 1 -- 172.21.15.27:0/3283638945 learned_addr learned my addr 172.21.15.27:0/3283638945 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:19:15.282 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:15.282+0000 7fecfd2a7640 1 --2- 172.21.15.27:0/3283638945 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fecf80d1550 0x7fecf80b8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:15.282 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:15.282+0000 7fecf7fff640 1 -- 172.21.15.27:0/3283638945 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fecf80d1550 msgr2=0x7fecf80b8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:15.282 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:15.282+0000 7fecf7fff640 1 --2- 172.21.15.27:0/3283638945 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fecf80d1550 0x7fecf80b8190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:15.283 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:15.282+0000 7fecf7fff640 1 -- 172.21.15.27:0/3283638945 >> v1:172.21.15.45:6789/0 conn(0x7fecf815eb90 legacy=0x7fecf80d2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:15.283 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:15.282+0000 7fecf7fff640 1 -- 172.21.15.27:0/3283638945 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecf80d3a40 con 0x7fecf80d5200 2024-09-12T03:19:15.284 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:15.283+0000 7fecf7fff640 1 --2- 172.21.15.27:0/3283638945 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fecf80d5200 0x7fecf80d1010 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fece0009870 tx=0x7fece002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fa0ca900c23f85f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:15.284 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:15.283+0000 7fecf77fe640 1 -- 172.21.15.27:0/3283638945 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fece0037070 con 0x7fecf80d5200 2024-09-12T03:19:15.284 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:15.283+0000 7fecf77fe640 1 -- 172.21.15.27:0/3283638945 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fece0032440 con 0x7fecf80d5200 2024-09-12T03:19:15.285 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:15.283+0000 7fecf77fe640 1 -- 172.21.15.27:0/3283638945 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fece0032810 con 0x7fecf80d5200 2024-09-12T03:19:15.285 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:15.283+0000 7fecfed0a640 1 -- 172.21.15.27:0/3283638945 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fecf80d5200 msgr2=0x7fecf80d1010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:15.285 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:15.283+0000 7fecfed0a640 1 --2- 172.21.15.27:0/3283638945 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fecf80d5200 0x7fecf80d1010 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fece0009870 tx=0x7fece002a210 comp rx=0 tx=0).stop 2024-09-12T03:19:15.285 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:15.283+0000 7fecfed0a640 1 -- 172.21.15.27:0/3283638945 shutdown_connections 2024-09-12T03:19:15.285 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:15.283+0000 7fecfed0a640 1 --2- 172.21.15.27:0/3283638945 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fecf80d1550 0x7fecf80b8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:15.285 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:15.283+0000 7fecfed0a640 1 --2- 172.21.15.27:0/3283638945 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fecf80d5200 0x7fecf80d1010 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:15.285 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:15.283+0000 7fecfed0a640 1 -- 172.21.15.27:0/3283638945 >> 172.21.15.27:0/3283638945 conn(0x7fecf806dce0 msgr2=0x7fecf806e0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:15.285 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:15.283+0000 7fecfed0a640 1 -- 172.21.15.27:0/3283638945 shutdown_connections 2024-09-12T03:19:15.285 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:15.283+0000 7fecfed0a640 1 -- 172.21.15.27:0/3283638945 wait complete. 2024-09-12T03:19:15.285 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:15.285+0000 7fecfed0a640 1 Processor -- start 2024-09-12T03:19:15.285 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:15.285+0000 7fecfed0a640 1 -- start start 2024-09-12T03:19:15.287 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:15.285+0000 7fecfed0a640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fecf80d1550 0x7fecf814cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:15.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:15.285+0000 7fecfed0a640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fecf80d5200 0x7fecf8158520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:15.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:15.286+0000 7fecfed0a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fecf815eb90 0x7fecf8159ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:15.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:15.286+0000 7fecfed0a640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fecf80d4000 con 0x7fecf80d1550 2024-09-12T03:19:15.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:15.286+0000 7fecfed0a640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fecf80d3500 con 0x7fecf815eb90 2024-09-12T03:19:15.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:15.286+0000 7fecfed0a640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fecf80d19a0 con 0x7fecf80d5200 2024-09-12T03:19:15.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:15.286+0000 7fecfcaa6640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fecf80d1550 0x7fecf814cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:15.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:15.286+0000 7fecfcaa6640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fecf80d1550 0x7fecf814cdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55354/0 (socket says 172.21.15.27:55354) 2024-09-12T03:19:15.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:15.286+0000 7fecfcaa6640 1 -- 172.21.15.27:0/1593578953 learned_addr learned my addr 172.21.15.27:0/1593578953 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:19:15.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:15.286+0000 7fecf7fff640 1 --2- 172.21.15.27:0/1593578953 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fecf80d5200 0x7fecf8158520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:15.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:15.286+0000 7fecfd2a7640 1 --2- 172.21.15.27:0/1593578953 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fecf815eb90 0x7fecf8159ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:15.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:15.286+0000 7fecfcaa6640 1 -- 172.21.15.27:0/1593578953 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fecf80d5200 msgr2=0x7fecf8158520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:15.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:15.286+0000 7fecfcaa6640 1 --2- 172.21.15.27:0/1593578953 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fecf80d5200 0x7fecf8158520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:15.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:15.286+0000 7fecfcaa6640 1 -- 172.21.15.27:0/1593578953 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fecf815eb90 msgr2=0x7fecf8159ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:15.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:15.286+0000 7fecfcaa6640 1 --2- 172.21.15.27:0/1593578953 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fecf815eb90 0x7fecf8159ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:15.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:15.286+0000 7fecfcaa6640 1 -- 172.21.15.27:0/1593578953 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecf8158f00 con 0x7fecf80d1550 2024-09-12T03:19:15.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:15.286+0000 7fecfcaa6640 1 --2- 172.21.15.27:0/1593578953 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fecf80d1550 0x7fecf814cdc0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fecf000ba50 tx=0x7fecf000bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:15.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:15.286+0000 7fecf57fa640 1 -- 172.21.15.27:0/1593578953 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecf0019070 con 0x7fecf80d1550 2024-09-12T03:19:15.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:15.286+0000 7fecf57fa640 1 -- 172.21.15.27:0/1593578953 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fecf000cc20 con 0x7fecf80d1550 2024-09-12T03:19:15.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:15.286+0000 7fecfed0a640 1 -- 172.21.15.27:0/1593578953 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fecf81591e0 con 0x7fecf80d1550 2024-09-12T03:19:15.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:15.286+0000 7fecf57fa640 1 -- 172.21.15.27:0/1593578953 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecf0014680 con 0x7fecf80d1550 2024-09-12T03:19:15.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:15.286+0000 7fecfed0a640 1 -- 172.21.15.27:0/1593578953 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fecf80cba80 con 0x7fecf80d1550 2024-09-12T03:19:15.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:15.287+0000 7fecfed0a640 1 -- 172.21.15.27:0/1593578953 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fecf815f090 con 0x7fecf80d1550 2024-09-12T03:19:15.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:15.288+0000 7fecf57fa640 1 -- 172.21.15.27:0/1593578953 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 9) ==== 90651+0+0 (secure 0 0 0) 0x7fecf001c070 con 0x7fecf80d1550 2024-09-12T03:19:15.290 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:15.288+0000 7fecf57fa640 1 --2- 172.21.15.27:0/1593578953 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fecd806bfe0 0x7fecd806e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:15.290 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:15.288+0000 7fecf57fa640 1 -- 172.21.15.27:0/1593578953 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5767+0+0 (secure 0 0 0) 0x7fecf008adb0 con 0x7fecf80d1550 2024-09-12T03:19:15.290 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:15.289+0000 7fecf7fff640 1 --2- 172.21.15.27:0/1593578953 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fecd806bfe0 0x7fecd806e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:15.290 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:15.289+0000 7fecf7fff640 1 --2- 172.21.15.27:0/1593578953 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fecd806bfe0 0x7fecd806e4a0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fece0034000 tx=0x7fece002e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:15.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:15.292+0000 7fecf57fa640 1 -- 172.21.15.27:0/1593578953 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fecf0058f90 con 0x7fecf80d1550 2024-09-12T03:19:15.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:15.505+0000 7fecfed0a640 1 -- 172.21.15.27:0/1593578953 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fecf8159700 con 0x7fecf80d1550 2024-09-12T03:19:15.505 INFO:tasks.ceph.mon.a.smithi027.stderr:2024-09-12T03:19:15.505+0000 7f959bfd9640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-12T03:19:15.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:15.507+0000 7fecf57fa640 1 -- 172.21.15.27:0/1593578953 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v24) ==== 161+0+0 (secure 0 0 0) 0x7fecf005d290 con 0x7fecf80d1550 2024-09-12T03:19:15.506 INFO:teuthology.orchestra.run.smithi027.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-12T03:19:15.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:15.509+0000 7fecfed0a640 1 -- 172.21.15.27:0/1593578953 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fecd806bfe0 msgr2=0x7fecd806e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:15.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:15.509+0000 7fecfed0a640 1 --2- 172.21.15.27:0/1593578953 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fecd806bfe0 0x7fecd806e4a0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fece0034000 tx=0x7fece002e000 comp rx=0 tx=0).stop 2024-09-12T03:19:15.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:15.509+0000 7fecfed0a640 1 -- 172.21.15.27:0/1593578953 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fecf80d1550 msgr2=0x7fecf814cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:15.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:15.509+0000 7fecfed0a640 1 --2- 172.21.15.27:0/1593578953 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fecf80d1550 0x7fecf814cdc0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fecf000ba50 tx=0x7fecf000bf10 comp rx=0 tx=0).stop 2024-09-12T03:19:15.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:15.509+0000 7fecfed0a640 1 -- 172.21.15.27:0/1593578953 shutdown_connections 2024-09-12T03:19:15.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:15.509+0000 7fecfed0a640 1 --2- 172.21.15.27:0/1593578953 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fecd806bfe0 0x7fecd806e4a0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:15.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:15.509+0000 7fecfed0a640 1 --2- 172.21.15.27:0/1593578953 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fecf815eb90 0x7fecf8159ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:15.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:15.509+0000 7fecfed0a640 1 --2- 172.21.15.27:0/1593578953 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fecf80d5200 0x7fecf8158520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:15.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:15.509+0000 7fecfed0a640 1 --2- 172.21.15.27:0/1593578953 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fecf80d1550 0x7fecf814cdc0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:15.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:15.509+0000 7fecfed0a640 1 -- 172.21.15.27:0/1593578953 >> 172.21.15.27:0/1593578953 conn(0x7fecf806dce0 msgr2=0x7fecf815f2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:15.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:15.509+0000 7fecfed0a640 1 -- 172.21.15.27:0/1593578953 shutdown_connections 2024-09-12T03:19:15.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:15.509+0000 7fecfed0a640 1 -- 172.21.15.27:0/1593578953 wait complete. 2024-09-12T03:19:15.521 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-12T03:19:15.549 INFO:tasks.ceph.mds.a.smithi027.stdout:starting mds.a at 2024-09-12T03:19:15.628 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:15.628+0000 7fa1f10ae640 1 Processor -- start 2024-09-12T03:19:15.628 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:15.628+0000 7fa1f10ae640 1 -- start start 2024-09-12T03:19:15.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:15.629+0000 7fa1f10ae640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa1ec15e900 0x7fa1ec0af370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:15.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:15.629+0000 7fa1f10ae640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa1ec06d730 0x7fa1ec0b5e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:15.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:15.629+0000 7fa1f10ae640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fa1ec0b6370 0x7fa1ec0b4600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:15.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:15.629+0000 7fa1f10ae640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7fa1ec15ece0 con 0x7fa1ec0b6370 2024-09-12T03:19:15.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:15.629+0000 7fa1f10ae640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fa1ec0b5940 con 0x7fa1ec15e900 2024-09-12T03:19:15.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:15.629+0000 7fa1f10ae640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fa1ec0af950 con 0x7fa1ec06d730 2024-09-12T03:19:15.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:15.629+0000 7fa1eb59e640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fa1ec0b6370 0x7fa1ec0b4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:15.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:15.629+0000 7fa1eb59e640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fa1ec0b6370 0x7fa1ec0b4600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55378/0 (socket says 172.21.15.27:55378) 2024-09-12T03:19:15.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:15.629+0000 7fa1eb59e640 1 -- 172.21.15.27:0/4061216428 learned_addr learned my addr 172.21.15.27:0/4061216428 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:19:15.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:15.629+0000 7fa1ead9d640 1 --2- 172.21.15.27:0/4061216428 >> v2:172.21.15.45:3300/0 conn(0x7fa1ec15e900 0x7fa1ec0af370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:15.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:15.629+0000 7fa1ea59c640 1 --2- 172.21.15.27:0/4061216428 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa1ec06d730 0x7fa1ec0b5e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:15.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:15.629+0000 7fa1eb59e640 1 -- 172.21.15.27:0/4061216428 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa1ec06d730 msgr2=0x7fa1ec0b5e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:15.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:15.629+0000 7fa1eb59e640 1 --2- 172.21.15.27:0/4061216428 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa1ec06d730 0x7fa1ec0b5e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:15.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:15.629+0000 7fa1eb59e640 1 -- 172.21.15.27:0/4061216428 >> v2:172.21.15.45:3300/0 conn(0x7fa1ec15e900 msgr2=0x7fa1ec0af370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:15.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:15.629+0000 7fa1eb59e640 1 --2- 172.21.15.27:0/4061216428 >> v2:172.21.15.45:3300/0 conn(0x7fa1ec15e900 0x7fa1ec0af370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:15.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:15.629+0000 7fa1eb59e640 1 -- 172.21.15.27:0/4061216428 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1ec0afc30 con 0x7fa1ec0b6370 2024-09-12T03:19:15.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:15.630+0000 7fa1eb59e640 1 --2- 172.21.15.27:0/4061216428 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa1ec0b6370 0x7fa1ec0b4600 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fa1ec051020 tx=0x7fa1e402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a784eac370a66c3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:15.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:15.630+0000 7fa1e9d9b640 1 -- 172.21.15.27:0/4061216428 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1e4036070 con 0x7fa1ec0b6370 2024-09-12T03:19:15.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:15.630+0000 7fa1e9d9b640 1 -- 172.21.15.27:0/4061216428 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1e4031600 con 0x7fa1ec0b6370 2024-09-12T03:19:15.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:15.630+0000 7fa1e9d9b640 1 -- 172.21.15.27:0/4061216428 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1e40319d0 con 0x7fa1ec0b6370 2024-09-12T03:19:15.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:15.631+0000 7fa1f10ae640 1 -- 172.21.15.27:0/4061216428 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa1ec0b6370 msgr2=0x7fa1ec0b4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:15.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:15.631+0000 7fa1f10ae640 1 --2- 172.21.15.27:0/4061216428 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa1ec0b6370 0x7fa1ec0b4600 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fa1ec051020 tx=0x7fa1e402a6a0 comp rx=0 tx=0).stop 2024-09-12T03:19:15.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:15.631+0000 7fa1f10ae640 1 -- 172.21.15.27:0/4061216428 shutdown_connections 2024-09-12T03:19:15.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:15.631+0000 7fa1f10ae640 1 --2- 172.21.15.27:0/4061216428 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa1ec0b6370 0x7fa1ec0b4600 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:15.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:15.631+0000 7fa1f10ae640 1 --2- 172.21.15.27:0/4061216428 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa1ec06d730 0x7fa1ec0b5e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:15.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:15.631+0000 7fa1f10ae640 1 --2- 172.21.15.27:0/4061216428 >> v2:172.21.15.45:3300/0 conn(0x7fa1ec15e900 0x7fa1ec0af370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:15.634 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:15.631+0000 7fa1f10ae640 1 -- 172.21.15.27:0/4061216428 >> 172.21.15.27:0/4061216428 conn(0x7fa1ec06cc40 msgr2=0x7fa1ec06d020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:15.634 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:15.631+0000 7fa1f10ae640 1 -- 172.21.15.27:0/4061216428 shutdown_connections 2024-09-12T03:19:15.634 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:15.631+0000 7fa1f10ae640 1 -- 172.21.15.27:0/4061216428 wait complete. 2024-09-12T03:19:15.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:15.632+0000 7fa1f10ae640 1 Processor -- start 2024-09-12T03:19:15.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:15.632+0000 7fa1f10ae640 1 -- start start 2024-09-12T03:19:15.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:15.633+0000 7fa1f10ae640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa1ec06d730 0x7fa1ec152970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:15.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:15.633+0000 7fa1f10ae640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa1ec0b6370 0x7fa1ec1589b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:15.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:15.633+0000 7fa1f10ae640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa1ec15e900 0x7fa1ec1f77b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:15.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:15.633+0000 7fa1f10ae640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fa1ec06e880 con 0x7fa1ec0b6370 2024-09-12T03:19:15.638 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:15.633+0000 7fa1f10ae640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa1ec0b01f0 con 0x7fa1ec06d730 2024-09-12T03:19:15.638 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:15.633+0000 7fa1f10ae640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fa1ec06ea00 con 0x7fa1ec15e900 2024-09-12T03:19:15.638 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:15.633+0000 7fa1ea59c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa1ec0b6370 0x7fa1ec1589b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:15.638 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:15.633+0000 7fa1ea59c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa1ec0b6370 0x7fa1ec1589b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55392/0 (socket says 172.21.15.27:55392) 2024-09-12T03:19:15.638 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:15.633+0000 7fa1ea59c640 1 -- 172.21.15.27:0/2729544971 learned_addr learned my addr 172.21.15.27:0/2729544971 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:19:15.638 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:15.633+0000 7fa1ead9d640 1 --2- 172.21.15.27:0/2729544971 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa1ec06d730 0x7fa1ec152970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:15.638 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:15.633+0000 7fa1eb59e640 1 --2- 172.21.15.27:0/2729544971 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa1ec15e900 0x7fa1ec1f77b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:15.638 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:15.633+0000 7fa1ea59c640 1 -- 172.21.15.27:0/2729544971 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa1ec15e900 msgr2=0x7fa1ec1f77b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:15.639 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:15.633+0000 7fa1ea59c640 1 --2- 172.21.15.27:0/2729544971 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa1ec15e900 0x7fa1ec1f77b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:15.639 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:15.633+0000 7fa1ea59c640 1 -- 172.21.15.27:0/2729544971 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa1ec06d730 msgr2=0x7fa1ec152970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:15.639 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:15.633+0000 7fa1ea59c640 1 --2- 172.21.15.27:0/2729544971 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa1ec06d730 0x7fa1ec152970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:15.639 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:15.633+0000 7fa1ea59c640 1 -- 172.21.15.27:0/2729544971 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1ec154180 con 0x7fa1ec0b6370 2024-09-12T03:19:15.639 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:15.633+0000 7fa1eb59e640 1 --2- 172.21.15.27:0/2729544971 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa1ec15e900 0x7fa1ec1f77b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:19:15.639 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:15.633+0000 7fa1ead9d640 1 --2- 172.21.15.27:0/2729544971 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa1ec06d730 0x7fa1ec152970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:19:15.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:15.634+0000 7fa1ea59c640 1 --2- 172.21.15.27:0/2729544971 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa1ec0b6370 0x7fa1ec1589b0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fa1e000d800 tx=0x7fa1e000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:15.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:15.634+0000 7fa1d37fe640 1 -- 172.21.15.27:0/2729544971 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1e0018020 con 0x7fa1ec0b6370 2024-09-12T03:19:15.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:15.634+0000 7fa1d37fe640 1 -- 172.21.15.27:0/2729544971 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1e00096a0 con 0x7fa1ec0b6370 2024-09-12T03:19:15.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:15.634+0000 7fa1f10ae640 1 -- 172.21.15.27:0/2729544971 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1ec1530e0 con 0x7fa1ec0b6370 2024-09-12T03:19:15.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:15.634+0000 7fa1d37fe640 1 -- 172.21.15.27:0/2729544971 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1e0009a70 con 0x7fa1ec0b6370 2024-09-12T03:19:15.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:15.634+0000 7fa1f10ae640 1 -- 172.21.15.27:0/2729544971 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1ec1536a0 con 0x7fa1ec0b6370 2024-09-12T03:19:15.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:15.635+0000 7fa1f10ae640 1 -- 172.21.15.27:0/2729544971 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1ec153dc0 con 0x7fa1ec0b6370 2024-09-12T03:19:15.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:15.635+0000 7fa1d37fe640 1 -- 172.21.15.27:0/2729544971 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 9) ==== 90651+0+0 (secure 0 0 0) 0x7fa1e001b020 con 0x7fa1ec0b6370 2024-09-12T03:19:15.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:15.636+0000 7fa1d37fe640 1 --2- 172.21.15.27:0/2729544971 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fa1d806bfe0 0x7fa1d806e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:15.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:15.636+0000 7fa1d37fe640 1 -- 172.21.15.27:0/2729544971 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5853+0+0 (secure 0 0 0) 0x7fa1e008b440 con 0x7fa1ec0b6370 2024-09-12T03:19:15.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:15.636+0000 7fa1ead9d640 1 --2- 172.21.15.27:0/2729544971 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fa1d806bfe0 0x7fa1d806e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:15.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:15.637+0000 7fa1ead9d640 1 --2- 172.21.15.27:0/2729544971 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fa1d806bfe0 0x7fa1d806e4a0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fa1ec1526d0 tx=0x7fa1dc008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:15.641 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:15.640+0000 7fa1d37fe640 1 -- 172.21.15.27:0/2729544971 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fa1e0022030 con 0x7fa1ec0b6370 2024-09-12T03:19:15.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:15.852+0000 7fa1f10ae640 1 -- 172.21.15.27:0/2729544971 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fa1ec0b5e30 con 0x7fa1ec0b6370 2024-09-12T03:19:16.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:16.510+0000 7fa1d37fe640 1 -- 172.21.15.27:0/2729544971 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v25) ==== 104+0+0 (secure 0 0 0) 0x7fa1e0059af0 con 0x7fa1ec0b6370 2024-09-12T03:19:16.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:16.513+0000 7fa1f10ae640 1 -- 172.21.15.27:0/2729544971 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fa1d806bfe0 msgr2=0x7fa1d806e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:16.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:16.513+0000 7fa1f10ae640 1 --2- 172.21.15.27:0/2729544971 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fa1d806bfe0 0x7fa1d806e4a0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fa1ec1526d0 tx=0x7fa1dc008040 comp rx=0 tx=0).stop 2024-09-12T03:19:16.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:16.513+0000 7fa1f10ae640 1 -- 172.21.15.27:0/2729544971 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa1ec0b6370 msgr2=0x7fa1ec1589b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:16.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:16.513+0000 7fa1f10ae640 1 --2- 172.21.15.27:0/2729544971 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa1ec0b6370 0x7fa1ec1589b0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fa1e000d800 tx=0x7fa1e000dcc0 comp rx=0 tx=0).stop 2024-09-12T03:19:16.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:16.513+0000 7fa1f10ae640 1 -- 172.21.15.27:0/2729544971 shutdown_connections 2024-09-12T03:19:16.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:16.513+0000 7fa1f10ae640 1 --2- 172.21.15.27:0/2729544971 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fa1d806bfe0 0x7fa1d806e4a0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:16.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:16.513+0000 7fa1f10ae640 1 --2- 172.21.15.27:0/2729544971 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa1ec15e900 0x7fa1ec1f77b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:16.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:16.513+0000 7fa1f10ae640 1 --2- 172.21.15.27:0/2729544971 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa1ec0b6370 0x7fa1ec1589b0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:16.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:16.513+0000 7fa1f10ae640 1 --2- 172.21.15.27:0/2729544971 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa1ec06d730 0x7fa1ec152970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:16.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:16.513+0000 7fa1f10ae640 1 -- 172.21.15.27:0/2729544971 >> 172.21.15.27:0/2729544971 conn(0x7fa1ec06cc40 msgr2=0x7fa1ec0d0730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:16.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:16.513+0000 7fa1f10ae640 1 -- 172.21.15.27:0/2729544971 shutdown_connections 2024-09-12T03:19:16.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:16.513+0000 7fa1f10ae640 1 -- 172.21.15.27:0/2729544971 wait complete. 2024-09-12T03:19:16.527 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-12T03:19:16.677 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:16.676+0000 7ff0e7d9f640 1 Processor -- start 2024-09-12T03:19:16.677 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:16.677+0000 7ff0e7d9f640 1 -- start start 2024-09-12T03:19:16.678 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:16.677+0000 7ff0e7d9f640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ff0e0162040 0x7ff0e00d1da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:16.678 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:16.677+0000 7ff0e7d9f640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ff0e00d3790 0x7ff0e00b7a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:16.679 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:16.677+0000 7ff0e7d9f640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff0e00d2e30 con 0x7ff0e006d8c0 2024-09-12T03:19:16.679 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:16.677+0000 7ff0e7d9f640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7ff0e00b7f90 con 0x7ff0e00d3790 2024-09-12T03:19:16.679 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:16.677+0000 7ff0e7d9f640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7ff0e0162420 con 0x7ff0e0162040 2024-09-12T03:19:16.679 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:16.678+0000 7ff0e633c640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7ff0e006d8c0 0x7ff0e00cf320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.27:36650/0 (socket says 172.21.15.27:36650) 2024-09-12T03:19:16.679 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:16.678+0000 7ff0e633c640 1 -- 172.21.15.27:0/4269486395 learned_addr learned my addr 172.21.15.27:0/4269486395 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:19:16.679 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:16.678+0000 7ff0e533a640 1 --2- 172.21.15.27:0/4269486395 >> v2:172.21.15.45:3300/0 conn(0x7ff0e00d3790 0x7ff0e00b7a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:16.679 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:16.678+0000 7ff0e5b3b640 1 --2- 172.21.15.27:0/4269486395 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ff0e0162040 0x7ff0e00d1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:16.680 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:16.678+0000 7ff0e4b39640 1 -- 172.21.15.27:0/4269486395 <== mon.1 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4015529281 0 0) 0x7ff0e00d2e30 con 0x7ff0e006d8c0 2024-09-12T03:19:16.680 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:16.678+0000 7ff0e4b39640 1 -- 172.21.15.27:0/4269486395 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff0cc003520 con 0x7ff0e006d8c0 2024-09-12T03:19:16.680 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:16.678+0000 7ff0e533a640 1 -- 172.21.15.27:0/4269486395 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ff0e0162040 msgr2=0x7ff0e00d1da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:16.680 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:16.678+0000 7ff0e533a640 1 --2- 172.21.15.27:0/4269486395 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ff0e0162040 0x7ff0e00d1da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:16.680 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:16.678+0000 7ff0e533a640 1 -- 172.21.15.27:0/4269486395 >> v1:172.21.15.45:6789/0 conn(0x7ff0e006d8c0 legacy=0x7ff0e00cf320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:16.680 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:16.678+0000 7ff0e533a640 1 -- 172.21.15.27:0/4269486395 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0e00d30d0 con 0x7ff0e00d3790 2024-09-12T03:19:16.680 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:16.678+0000 7ff0e5b3b640 1 --2- 172.21.15.27:0/4269486395 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ff0e0162040 0x7ff0e00d1da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:19:16.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:16.679+0000 7ff0e533a640 1 --2- 172.21.15.27:0/4269486395 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff0e00d3790 0x7ff0e00b7a50 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7ff0d00097a0 tx=0x7ff0d002a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bda505954391ed6f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:16.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:16.679+0000 7ff0e4b39640 1 -- 172.21.15.27:0/4269486395 <== mon.1 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0d0037070 con 0x7ff0e00d3790 2024-09-12T03:19:16.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:16.679+0000 7ff0e4b39640 1 -- 172.21.15.27:0/4269486395 <== mon.1 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0d0032440 con 0x7ff0e00d3790 2024-09-12T03:19:16.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:16.679+0000 7ff0e4b39640 1 -- 172.21.15.27:0/4269486395 <== mon.1 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0d00327f0 con 0x7ff0e00d3790 2024-09-12T03:19:16.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:16.680+0000 7ff0e7d9f640 1 -- 172.21.15.27:0/4269486395 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff0e00d3790 msgr2=0x7ff0e00b7a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:16.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:16.680+0000 7ff0e7d9f640 1 --2- 172.21.15.27:0/4269486395 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff0e00d3790 0x7ff0e00b7a50 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7ff0d00097a0 tx=0x7ff0d002a180 comp rx=0 tx=0).stop 2024-09-12T03:19:16.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:16.680+0000 7ff0e7d9f640 1 -- 172.21.15.27:0/4269486395 shutdown_connections 2024-09-12T03:19:16.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:16.680+0000 7ff0e7d9f640 1 --2- 172.21.15.27:0/4269486395 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff0e00d3790 0x7ff0e00b7a50 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:16.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:16.680+0000 7ff0e7d9f640 1 --2- 172.21.15.27:0/4269486395 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ff0e0162040 0x7ff0e00d1da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:16.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:16.680+0000 7ff0e7d9f640 1 -- 172.21.15.27:0/4269486395 >> 172.21.15.27:0/4269486395 conn(0x7ff0e00b6700 msgr2=0x7ff0e00b6b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:16.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:16.680+0000 7ff0e7d9f640 1 -- 172.21.15.27:0/4269486395 shutdown_connections 2024-09-12T03:19:16.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:16.680+0000 7ff0e7d9f640 1 -- 172.21.15.27:0/4269486395 wait complete. 2024-09-12T03:19:16.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:16.681+0000 7ff0e7d9f640 1 Processor -- start 2024-09-12T03:19:16.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:16.681+0000 7ff0e7d9f640 1 -- start start 2024-09-12T03:19:16.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:16.682+0000 7ff0e7d9f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff0e006d8c0 0x7ff0e015b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:16.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:16.682+0000 7ff0e7d9f640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ff0e00d3790 0x7ff0e0155de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:16.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:16.682+0000 7ff0e7d9f640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff0e0162040 0x7ff0e0153b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:16.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:16.682+0000 7ff0e7d9f640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7ff0e00d4530 con 0x7ff0e0162040 2024-09-12T03:19:16.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:16.682+0000 7ff0e7d9f640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ff0e00cee00 con 0x7ff0e006d8c0 2024-09-12T03:19:16.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:16.682+0000 7ff0e7d9f640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7ff0e00d4150 con 0x7ff0e00d3790 2024-09-12T03:19:16.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:16.682+0000 7ff0e633c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff0e0162040 0x7ff0e0153b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:16.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:16.682+0000 7ff0e633c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff0e0162040 0x7ff0e0153b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55404/0 (socket says 172.21.15.27:55404) 2024-09-12T03:19:16.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:16.682+0000 7ff0e633c640 1 -- 172.21.15.27:0/503992475 learned_addr learned my addr 172.21.15.27:0/503992475 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:19:16.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:16.682+0000 7ff0e5b3b640 1 --2- 172.21.15.27:0/503992475 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff0e006d8c0 0x7ff0e015b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:16.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:16.682+0000 7ff0e533a640 1 --2- 172.21.15.27:0/503992475 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ff0e00d3790 0x7ff0e0155de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:16.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:16.682+0000 7ff0e633c640 1 -- 172.21.15.27:0/503992475 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ff0e00d3790 msgr2=0x7ff0e0155de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:16.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:16.682+0000 7ff0e633c640 1 --2- 172.21.15.27:0/503992475 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ff0e00d3790 0x7ff0e0155de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:16.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:16.682+0000 7ff0e633c640 1 -- 172.21.15.27:0/503992475 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff0e006d8c0 msgr2=0x7ff0e015b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:16.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:16.682+0000 7ff0e633c640 1 --2- 172.21.15.27:0/503992475 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff0e006d8c0 0x7ff0e015b2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:16.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:16.682+0000 7ff0e633c640 1 -- 172.21.15.27:0/503992475 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0e0156320 con 0x7ff0e0162040 2024-09-12T03:19:16.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:16.682+0000 7ff0e633c640 1 --2- 172.21.15.27:0/503992475 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff0e0162040 0x7ff0e0153b40 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7ff0d800caa0 tx=0x7ff0d800cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:16.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:16.683+0000 7ff0d67fc640 1 -- 172.21.15.27:0/503992475 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0d8017070 con 0x7ff0e0162040 2024-09-12T03:19:16.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:16.683+0000 7ff0d67fc640 1 -- 172.21.15.27:0/503992475 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0d800ac00 con 0x7ff0e0162040 2024-09-12T03:19:16.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:16.683+0000 7ff0e7d9f640 1 -- 172.21.15.27:0/503992475 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0e015fcf0 con 0x7ff0e0162040 2024-09-12T03:19:16.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:16.683+0000 7ff0d67fc640 1 -- 172.21.15.27:0/503992475 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0d8012400 con 0x7ff0e0162040 2024-09-12T03:19:16.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:16.683+0000 7ff0e7d9f640 1 -- 172.21.15.27:0/503992475 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0e01565c0 con 0x7ff0e0162040 2024-09-12T03:19:16.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:16.684+0000 7ff0e7d9f640 1 -- 172.21.15.27:0/503992475 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0e00b7a50 con 0x7ff0e0162040 2024-09-12T03:19:16.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:16.684+0000 7ff0d67fc640 1 -- 172.21.15.27:0/503992475 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 9) ==== 90651+0+0 (secure 0 0 0) 0x7ff0d801a070 con 0x7ff0e0162040 2024-09-12T03:19:16.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:16.685+0000 7ff0d67fc640 1 --2- 172.21.15.27:0/503992475 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7ff0c406bfe0 0x7ff0c406e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:16.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:16.685+0000 7ff0d67fc640 1 -- 172.21.15.27:0/503992475 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5853+0+0 (secure 0 0 0) 0x7ff0d808af90 con 0x7ff0e0162040 2024-09-12T03:19:16.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:16.685+0000 7ff0e5b3b640 1 --2- 172.21.15.27:0/503992475 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7ff0c406bfe0 0x7ff0c406e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:16.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:16.685+0000 7ff0e5b3b640 1 --2- 172.21.15.27:0/503992475 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7ff0c406bfe0 0x7ff0c406e4a0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7ff0dc009560 tx=0x7ff0dc008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:16.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:16.689+0000 7ff0d67fc640 1 -- 172.21.15.27:0/503992475 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7ff0d8059630 con 0x7ff0e0162040 2024-09-12T03:19:16.862 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:16.862+0000 7ff0e7d9f640 1 -- 172.21.15.27:0/503992475 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7ff0e0161860 con 0x7ff0e0162040 2024-09-12T03:19:17.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:17.512+0000 7ff0d67fc640 1 -- 172.21.15.27:0/503992475 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v47) ==== 181+0+0 (secure 0 0 0) 0x7ff0d8058f30 con 0x7ff0e0162040 2024-09-12T03:19:17.512 INFO:teuthology.orchestra.run.smithi027.stderr:pool 'cephfs_metadata' removed 2024-09-12T03:19:17.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:17.516+0000 7ff0e7d9f640 1 -- 172.21.15.27:0/503992475 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7ff0c406bfe0 msgr2=0x7ff0c406e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:17.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:17.516+0000 7ff0e7d9f640 1 --2- 172.21.15.27:0/503992475 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7ff0c406bfe0 0x7ff0c406e4a0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7ff0dc009560 tx=0x7ff0dc008040 comp rx=0 tx=0).stop 2024-09-12T03:19:17.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:17.516+0000 7ff0e7d9f640 1 -- 172.21.15.27:0/503992475 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff0e0162040 msgr2=0x7ff0e0153b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:17.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:17.516+0000 7ff0e7d9f640 1 --2- 172.21.15.27:0/503992475 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff0e0162040 0x7ff0e0153b40 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7ff0d800caa0 tx=0x7ff0d800cf60 comp rx=0 tx=0).stop 2024-09-12T03:19:17.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:17.516+0000 7ff0e7d9f640 1 -- 172.21.15.27:0/503992475 shutdown_connections 2024-09-12T03:19:17.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:17.516+0000 7ff0e7d9f640 1 --2- 172.21.15.27:0/503992475 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7ff0c406bfe0 0x7ff0c406e4a0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:17.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:17.516+0000 7ff0e7d9f640 1 --2- 172.21.15.27:0/503992475 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff0e0162040 0x7ff0e0153b40 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:17.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:17.516+0000 7ff0e7d9f640 1 --2- 172.21.15.27:0/503992475 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ff0e00d3790 0x7ff0e0155de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:17.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:17.516+0000 7ff0e7d9f640 1 --2- 172.21.15.27:0/503992475 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff0e006d8c0 0x7ff0e015b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:17.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:17.516+0000 7ff0e7d9f640 1 -- 172.21.15.27:0/503992475 >> 172.21.15.27:0/503992475 conn(0x7ff0e00b6700 msgr2=0x7ff0e00cf320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:17.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:17.516+0000 7ff0e7d9f640 1 -- 172.21.15.27:0/503992475 shutdown_connections 2024-09-12T03:19:17.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:17.516+0000 7ff0e7d9f640 1 -- 172.21.15.27:0/503992475 wait complete. 2024-09-12T03:19:17.530 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-12T03:19:17.643 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:17.643+0000 7fe58233b640 1 Processor -- start 2024-09-12T03:19:17.644 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:17.643+0000 7fe58233b640 1 -- start start 2024-09-12T03:19:17.645 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:17.644+0000 7fe58233b640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fe57c15e990 0x7fe57c0b7600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:17.645 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:17.644+0000 7fe58233b640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fe57c0b3d10 0x7fe57c0b66c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:17.645 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:17.644+0000 7fe58233b640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe57c15ed70 con 0x7fe57c0b6230 2024-09-12T03:19:17.645 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:17.644+0000 7fe58233b640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7fe57c0d2a30 con 0x7fe57c0b3d10 2024-09-12T03:19:17.645 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:17.644+0000 7fe58233b640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fe57c0d2d10 con 0x7fe57c15e990 2024-09-12T03:19:17.645 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:17.644+0000 7fe5808d8640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fe57c0b3d10 0x7fe57c0b66c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:17.645 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:17.644+0000 7fe57b7fe640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fe57c0b6230 0x7fe57c0b3600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:33934/0 (socket says 172.21.15.27:33934) 2024-09-12T03:19:17.645 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:17.644+0000 7fe5808d8640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fe57c0b3d10 0x7fe57c0b66c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55420/0 (socket says 172.21.15.27:55420) 2024-09-12T03:19:17.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:17.644+0000 7fe57b7fe640 1 -- 172.21.15.27:0/2079767141 learned_addr learned my addr 172.21.15.27:0/2079767141 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:19:17.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:17.645+0000 7fe57bfff640 1 --2- 172.21.15.27:0/2079767141 >> v2:172.21.15.45:3300/0 conn(0x7fe57c15e990 0x7fe57c0b7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:17.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:17.645+0000 7fe5808d8640 1 -- 172.21.15.27:0/2079767141 >> v2:172.21.15.45:3300/0 conn(0x7fe57c15e990 msgr2=0x7fe57c0b7600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:17.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:17.645+0000 7fe5808d8640 1 --2- 172.21.15.27:0/2079767141 >> v2:172.21.15.45:3300/0 conn(0x7fe57c15e990 0x7fe57c0b7600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:17.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:17.645+0000 7fe5808d8640 1 -- 172.21.15.27:0/2079767141 >> v1:172.21.15.27:6789/0 conn(0x7fe57c0b6230 legacy=0x7fe57c0b3600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:17.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:17.645+0000 7fe5808d8640 1 -- 172.21.15.27:0/2079767141 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe57c0d2ff0 con 0x7fe57c0b3d10 2024-09-12T03:19:17.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:17.645+0000 7fe5808d8640 1 --2- 172.21.15.27:0/2079767141 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe57c0b3d10 0x7fe57c0b66c0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fe57c0d2270 tx=0x7fe57402a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fa2308663b79ef3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:17.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:17.646+0000 7fe57affd640 1 -- 172.21.15.27:0/2079767141 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe574007740 con 0x7fe57c0b3d10 2024-09-12T03:19:17.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:17.646+0000 7fe57affd640 1 -- 172.21.15.27:0/2079767141 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5740078e0 con 0x7fe57c0b3d10 2024-09-12T03:19:17.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:17.646+0000 7fe57affd640 1 -- 172.21.15.27:0/2079767141 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe574007cb0 con 0x7fe57c0b3d10 2024-09-12T03:19:17.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:17.646+0000 7fe58233b640 1 -- 172.21.15.27:0/2079767141 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe57c0b3d10 msgr2=0x7fe57c0b66c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:17.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:17.646+0000 7fe58233b640 1 --2- 172.21.15.27:0/2079767141 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe57c0b3d10 0x7fe57c0b66c0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fe57c0d2270 tx=0x7fe57402a680 comp rx=0 tx=0).stop 2024-09-12T03:19:17.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:17.646+0000 7fe58233b640 1 -- 172.21.15.27:0/2079767141 shutdown_connections 2024-09-12T03:19:17.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:17.646+0000 7fe58233b640 1 --2- 172.21.15.27:0/2079767141 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe57c0b3d10 0x7fe57c0b66c0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:17.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:17.646+0000 7fe58233b640 1 --2- 172.21.15.27:0/2079767141 >> v2:172.21.15.45:3300/0 conn(0x7fe57c15e990 0x7fe57c0b7600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:17.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:17.646+0000 7fe58233b640 1 -- 172.21.15.27:0/2079767141 >> 172.21.15.27:0/2079767141 conn(0x7fe57c0d1230 msgr2=0x7fe57c0d1640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:17.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:17.646+0000 7fe58233b640 1 -- 172.21.15.27:0/2079767141 shutdown_connections 2024-09-12T03:19:17.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:17.646+0000 7fe58233b640 1 -- 172.21.15.27:0/2079767141 wait complete. 2024-09-12T03:19:17.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:17.647+0000 7fe58233b640 1 Processor -- start 2024-09-12T03:19:17.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:17.648+0000 7fe58233b640 1 -- start start 2024-09-12T03:19:17.650 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:17.648+0000 7fe58233b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe57c0b3d10 0x7fe57c0c8cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:17.650 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:17.648+0000 7fe58233b640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe57c0b6230 0x7fe57c0cacf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:17.651 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:17.648+0000 7fe58233b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe57c15e990 0x7fe57c1f7d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:17.651 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:17.648+0000 7fe58233b640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fe57c0b7b40 con 0x7fe57c15e990 2024-09-12T03:19:17.651 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:17.648+0000 7fe58233b640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe57c0d35b0 con 0x7fe57c0b3d10 2024-09-12T03:19:17.651 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:17.648+0000 7fe58233b640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fe57c0b7cc0 con 0x7fe57c0b6230 2024-09-12T03:19:17.651 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:17.649+0000 7fe5808d8640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe57c15e990 0x7fe57c1f7d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:17.651 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:17.649+0000 7fe5808d8640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe57c15e990 0x7fe57c1f7d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55436/0 (socket says 172.21.15.27:55436) 2024-09-12T03:19:17.651 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:17.649+0000 7fe5808d8640 1 -- 172.21.15.27:0/2335331759 learned_addr learned my addr 172.21.15.27:0/2335331759 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:19:17.651 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:17.649+0000 7fe57b7fe640 1 --2- 172.21.15.27:0/2335331759 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe57c0b6230 0x7fe57c0cacf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:17.651 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:17.649+0000 7fe57bfff640 1 --2- 172.21.15.27:0/2335331759 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe57c0b3d10 0x7fe57c0c8cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:17.651 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:17.649+0000 7fe5808d8640 1 -- 172.21.15.27:0/2335331759 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe57c0b6230 msgr2=0x7fe57c0cacf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:17.651 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:17.649+0000 7fe5808d8640 1 --2- 172.21.15.27:0/2335331759 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe57c0b6230 0x7fe57c0cacf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:17.651 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:17.649+0000 7fe5808d8640 1 -- 172.21.15.27:0/2335331759 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe57c0b3d10 msgr2=0x7fe57c0c8cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:17.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:17.649+0000 7fe5808d8640 1 --2- 172.21.15.27:0/2335331759 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe57c0b3d10 0x7fe57c0c8cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:17.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:17.649+0000 7fe5808d8640 1 -- 172.21.15.27:0/2335331759 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe57c0c9be0 con 0x7fe57c15e990 2024-09-12T03:19:17.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:17.649+0000 7fe57bfff640 1 --2- 172.21.15.27:0/2335331759 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe57c0b3d10 0x7fe57c0c8cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:19:17.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:17.649+0000 7fe5808d8640 1 --2- 172.21.15.27:0/2335331759 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe57c15e990 0x7fe57c1f7d30 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fe57402ac70 tx=0x7fe574004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:17.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:17.649+0000 7fe578ff9640 1 -- 172.21.15.27:0/2335331759 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe574040070 con 0x7fe57c15e990 2024-09-12T03:19:17.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:17.649+0000 7fe578ff9640 1 -- 172.21.15.27:0/2335331759 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe574031bc0 con 0x7fe57c15e990 2024-09-12T03:19:17.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:17.649+0000 7fe58233b640 1 -- 172.21.15.27:0/2335331759 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe57c0c9340 con 0x7fe57c15e990 2024-09-12T03:19:17.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:17.649+0000 7fe578ff9640 1 -- 172.21.15.27:0/2335331759 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe574009450 con 0x7fe57c15e990 2024-09-12T03:19:17.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:17.649+0000 7fe58233b640 1 -- 172.21.15.27:0/2335331759 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe57c0c9900 con 0x7fe57c15e990 2024-09-12T03:19:17.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:17.651+0000 7fe58233b640 1 -- 172.21.15.27:0/2335331759 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe57c0b7600 con 0x7fe57c15e990 2024-09-12T03:19:17.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:17.651+0000 7fe578ff9640 1 -- 172.21.15.27:0/2335331759 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 9) ==== 90651+0+0 (secure 0 0 0) 0x7fe574039040 con 0x7fe57c15e990 2024-09-12T03:19:17.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:17.651+0000 7fe578ff9640 1 --2- 172.21.15.27:0/2335331759 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fe56806bfe0 0x7fe56806e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:17.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:17.652+0000 7fe578ff9640 1 -- 172.21.15.27:0/2335331759 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5415+0+0 (secure 0 0 0) 0x7fe5740ac850 con 0x7fe57c15e990 2024-09-12T03:19:17.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:17.652+0000 7fe57bfff640 1 --2- 172.21.15.27:0/2335331759 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fe56806bfe0 0x7fe56806e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:17.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:17.652+0000 7fe57bfff640 1 --2- 172.21.15.27:0/2335331759 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fe56806bfe0 0x7fe56806e4a0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fe56c006fd0 tx=0x7fe56c008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:17.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:17.656+0000 7fe578ff9640 1 -- 172.21.15.27:0/2335331759 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fe57407b020 con 0x7fe57c15e990 2024-09-12T03:19:17.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:17.830+0000 7fe58233b640 1 -- 172.21.15.27:0/2335331759 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fe57c0b5040 con 0x7fe57c15e990 2024-09-12T03:19:18.519 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:18.519+0000 7fe578ff9640 1 -- 172.21.15.27:0/2335331759 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v48) ==== 169+0+0 (secure 0 0 0) 0x7fe57407a920 con 0x7fe57c15e990 2024-09-12T03:19:18.520 INFO:teuthology.orchestra.run.smithi027.stderr:pool 'cephfs_data' removed 2024-09-12T03:19:18.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:18.523+0000 7fe58233b640 1 -- 172.21.15.27:0/2335331759 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fe56806bfe0 msgr2=0x7fe56806e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:18.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:18.523+0000 7fe58233b640 1 --2- 172.21.15.27:0/2335331759 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fe56806bfe0 0x7fe56806e4a0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fe56c006fd0 tx=0x7fe56c008040 comp rx=0 tx=0).stop 2024-09-12T03:19:18.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:18.523+0000 7fe58233b640 1 -- 172.21.15.27:0/2335331759 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe57c15e990 msgr2=0x7fe57c1f7d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:18.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:18.523+0000 7fe58233b640 1 --2- 172.21.15.27:0/2335331759 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe57c15e990 0x7fe57c1f7d30 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fe57402ac70 tx=0x7fe574004a40 comp rx=0 tx=0).stop 2024-09-12T03:19:18.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:18.524+0000 7fe58233b640 1 -- 172.21.15.27:0/2335331759 shutdown_connections 2024-09-12T03:19:18.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:18.524+0000 7fe58233b640 1 --2- 172.21.15.27:0/2335331759 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fe56806bfe0 0x7fe56806e4a0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:18.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:18.524+0000 7fe58233b640 1 --2- 172.21.15.27:0/2335331759 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe57c15e990 0x7fe57c1f7d30 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:18.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:18.524+0000 7fe58233b640 1 --2- 172.21.15.27:0/2335331759 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe57c0b6230 0x7fe57c0cacf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:18.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:18.524+0000 7fe58233b640 1 --2- 172.21.15.27:0/2335331759 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe57c0b3d10 0x7fe57c0c8cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:18.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:18.524+0000 7fe58233b640 1 -- 172.21.15.27:0/2335331759 >> 172.21.15.27:0/2335331759 conn(0x7fe57c0d1230 msgr2=0x7fe57c0cf710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:18.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:18.524+0000 7fe58233b640 1 -- 172.21.15.27:0/2335331759 shutdown_connections 2024-09-12T03:19:18.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:18.524+0000 7fe58233b640 1 -- 172.21.15.27:0/2335331759 wait complete. 2024-09-12T03:19:18.540 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-09-12T03:19:18.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:18.649+0000 7fe4d1453640 1 Processor -- start 2024-09-12T03:19:18.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:18.649+0000 7fe4d1453640 1 -- start start 2024-09-12T03:19:18.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:18.650+0000 7fe4d1453640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe4cc0b64a0 0x7fe4cc0d0830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:18.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:18.650+0000 7fe4d1453640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fe4cc0d40e0 0x7fe4cc161ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:18.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:18.650+0000 7fe4d1453640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe4cc0d3640 con 0x7fe4cc0b6880 2024-09-12T03:19:18.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:18.650+0000 7fe4d1453640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7fe4cc0cff80 con 0x7fe4cc0d40e0 2024-09-12T03:19:18.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:18.650+0000 7fe4d1453640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fe4cc0cf810 con 0x7fe4cc0b64a0 2024-09-12T03:19:18.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:18.650+0000 7fe4ca7fc640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fe4cc0d40e0 0x7fe4cc161ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:18.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:18.651+0000 7fe4ca7fc640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fe4cc0d40e0 0x7fe4cc161ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55438/0 (socket says 172.21.15.27:55438) 2024-09-12T03:19:18.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:18.651+0000 7fe4ca7fc640 1 -- 172.21.15.27:0/3144998080 learned_addr learned my addr 172.21.15.27:0/3144998080 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:19:18.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:18.651+0000 7fe4caffd640 1 --2- 172.21.15.27:0/3144998080 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe4cc0b64a0 0x7fe4cc0d0830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:18.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:18.651+0000 7fe4ca7fc640 1 -- 172.21.15.27:0/3144998080 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe4cc0b64a0 msgr2=0x7fe4cc0d0830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:18.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:18.651+0000 7fe4ca7fc640 1 --2- 172.21.15.27:0/3144998080 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe4cc0b64a0 0x7fe4cc0d0830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:18.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:18.651+0000 7fe4ca7fc640 1 -- 172.21.15.27:0/3144998080 >> v1:172.21.15.45:6789/0 conn(0x7fe4cc0b6880 legacy=0x7fe4cc0d2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:18.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:18.651+0000 7fe4ca7fc640 1 -- 172.21.15.27:0/3144998080 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4cc0d3920 con 0x7fe4cc0d40e0 2024-09-12T03:19:18.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:18.652+0000 7fe4ca7fc640 1 --2- 172.21.15.27:0/3144998080 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe4cc0d40e0 0x7fe4cc161ab0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fe4c0009870 tx=0x7fe4c002a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d521b1665988263d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:18.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:18.652+0000 7fe4c9ffb640 1 -- 172.21.15.27:0/3144998080 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4c0037070 con 0x7fe4cc0d40e0 2024-09-12T03:19:18.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:18.652+0000 7fe4c9ffb640 1 -- 172.21.15.27:0/3144998080 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4c0032440 con 0x7fe4cc0d40e0 2024-09-12T03:19:18.657 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:18.652+0000 7fe4c9ffb640 1 -- 172.21.15.27:0/3144998080 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4c0032810 con 0x7fe4cc0d40e0 2024-09-12T03:19:18.657 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:18.652+0000 7fe4d1453640 1 -- 172.21.15.27:0/3144998080 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe4cc0d40e0 msgr2=0x7fe4cc161ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:18.657 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:18.652+0000 7fe4d1453640 1 --2- 172.21.15.27:0/3144998080 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe4cc0d40e0 0x7fe4cc161ab0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fe4c0009870 tx=0x7fe4c002a1c0 comp rx=0 tx=0).stop 2024-09-12T03:19:18.657 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:18.652+0000 7fe4d1453640 1 -- 172.21.15.27:0/3144998080 shutdown_connections 2024-09-12T03:19:18.657 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:18.652+0000 7fe4d1453640 1 --2- 172.21.15.27:0/3144998080 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe4cc0d40e0 0x7fe4cc161ab0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:18.657 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:18.652+0000 7fe4d1453640 1 --2- 172.21.15.27:0/3144998080 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe4cc0b64a0 0x7fe4cc0d0830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:18.657 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:18.652+0000 7fe4d1453640 1 -- 172.21.15.27:0/3144998080 >> 172.21.15.27:0/3144998080 conn(0x7fe4cc0d1230 msgr2=0x7fe4cc0d1640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:18.657 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:18.652+0000 7fe4d1453640 1 -- 172.21.15.27:0/3144998080 shutdown_connections 2024-09-12T03:19:18.657 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:18.652+0000 7fe4d1453640 1 -- 172.21.15.27:0/3144998080 wait complete. 2024-09-12T03:19:18.657 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:18.654+0000 7fe4d1453640 1 Processor -- start 2024-09-12T03:19:18.657 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:18.654+0000 7fe4d1453640 1 -- start start 2024-09-12T03:19:18.659 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:18.655+0000 7fe4d1453640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe4cc0b64a0 0x7fe4cc15dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:18.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:18.655+0000 7fe4d1453640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe4cc0b6880 0x7fe4cc1f4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:18.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:18.655+0000 7fe4d1453640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe4cc0d40e0 0x7fe4cc0b3050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:18.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:18.655+0000 7fe4d1453640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fe4cc0d0d70 con 0x7fe4cc0d40e0 2024-09-12T03:19:18.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:18.655+0000 7fe4d1453640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe4cc0cfaf0 con 0x7fe4cc0b64a0 2024-09-12T03:19:18.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:18.655+0000 7fe4d1453640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fe4cc0b89f0 con 0x7fe4cc0b6880 2024-09-12T03:19:18.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:18.655+0000 7fe4cb7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe4cc0d40e0 0x7fe4cc0b3050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:18.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:18.655+0000 7fe4cb7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe4cc0d40e0 0x7fe4cc0b3050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55454/0 (socket says 172.21.15.27:55454) 2024-09-12T03:19:18.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:18.655+0000 7fe4cb7fe640 1 -- 172.21.15.27:0/64317653 learned_addr learned my addr 172.21.15.27:0/64317653 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:19:18.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:18.655+0000 7fe4caffd640 1 --2- 172.21.15.27:0/64317653 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe4cc0b64a0 0x7fe4cc15dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:18.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:18.655+0000 7fe4ca7fc640 1 --2- 172.21.15.27:0/64317653 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe4cc0b6880 0x7fe4cc1f4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:18.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:18.655+0000 7fe4cb7fe640 1 -- 172.21.15.27:0/64317653 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe4cc0b6880 msgr2=0x7fe4cc1f4d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:18.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:18.655+0000 7fe4cb7fe640 1 --2- 172.21.15.27:0/64317653 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe4cc0b6880 0x7fe4cc1f4d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:18.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:18.655+0000 7fe4cb7fe640 1 -- 172.21.15.27:0/64317653 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe4cc0b64a0 msgr2=0x7fe4cc15dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:18.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:18.655+0000 7fe4cb7fe640 1 --2- 172.21.15.27:0/64317653 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe4cc0b64a0 0x7fe4cc15dfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:18.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:18.655+0000 7fe4cb7fe640 1 -- 172.21.15.27:0/64317653 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4cc1f52b0 con 0x7fe4cc0d40e0 2024-09-12T03:19:18.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:18.655+0000 7fe4cb7fe640 1 --2- 172.21.15.27:0/64317653 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe4cc0d40e0 0x7fe4cc0b3050 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fe4c400c760 tx=0x7fe4c400cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:18.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:18.655+0000 7fe4b37fe640 1 -- 172.21.15.27:0/64317653 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4c4003b70 con 0x7fe4cc0d40e0 2024-09-12T03:19:18.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:18.655+0000 7fe4b37fe640 1 -- 172.21.15.27:0/64317653 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4c4003d10 con 0x7fe4cc0d40e0 2024-09-12T03:19:18.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:18.655+0000 7fe4b37fe640 1 -- 172.21.15.27:0/64317653 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4c400aa90 con 0x7fe4cc0d40e0 2024-09-12T03:19:18.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:18.655+0000 7fe4d1453640 1 -- 172.21.15.27:0/64317653 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4cc15ce70 con 0x7fe4cc0d40e0 2024-09-12T03:19:18.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:18.655+0000 7fe4d1453640 1 -- 172.21.15.27:0/64317653 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4cc15d7c0 con 0x7fe4cc0d40e0 2024-09-12T03:19:18.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:18.656+0000 7fe4d1453640 1 -- 172.21.15.27:0/64317653 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4cc0d0830 con 0x7fe4cc0d40e0 2024-09-12T03:19:18.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:18.657+0000 7fe4b37fe640 1 -- 172.21.15.27:0/64317653 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 9) ==== 90651+0+0 (secure 0 0 0) 0x7fe4c400ac30 con 0x7fe4cc0d40e0 2024-09-12T03:19:18.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:18.657+0000 7fe4b37fe640 1 --2- 172.21.15.27:0/64317653 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fe4b806c010 0x7fe4b806e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:18.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:18.657+0000 7fe4b37fe640 1 -- 172.21.15.27:0/64317653 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 4943+0+0 (secure 0 0 0) 0x7fe4c408ac40 con 0x7fe4cc0d40e0 2024-09-12T03:19:18.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:18.658+0000 7fe4caffd640 1 --2- 172.21.15.27:0/64317653 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fe4b806c010 0x7fe4b806e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:18.662 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:18.658+0000 7fe4caffd640 1 --2- 172.21.15.27:0/64317653 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fe4b806c010 0x7fe4b806e4d0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fe4bc0079a0 tx=0x7fe4bc008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:18.662 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:18.662+0000 7fe4b37fe640 1 -- 172.21.15.27:0/64317653 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fe4cc1f52b0 con 0x7fe4cc0d40e0 2024-09-12T03:19:18.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:18.841+0000 7fe4d1453640 1 -- 172.21.15.27:0/64317653 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7fe4cc0ba410 con 0x7fe4cc0d40e0 2024-09-12T03:19:18.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:18.844+0000 7fe4b37fe640 1 -- 172.21.15.27:0/64317653 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v13) ==== 104+0+0 (secure 0 0 0) 0x7fe4c4059020 con 0x7fe4cc0d40e0 2024-09-12T03:19:18.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:18.846+0000 7fe4d1453640 1 -- 172.21.15.27:0/64317653 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fe4b806c010 msgr2=0x7fe4b806e4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:18.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:18.846+0000 7fe4d1453640 1 --2- 172.21.15.27:0/64317653 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fe4b806c010 0x7fe4b806e4d0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fe4bc0079a0 tx=0x7fe4bc008040 comp rx=0 tx=0).stop 2024-09-12T03:19:18.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:18.846+0000 7fe4d1453640 1 -- 172.21.15.27:0/64317653 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe4cc0d40e0 msgr2=0x7fe4cc0b3050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:18.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:18.846+0000 7fe4d1453640 1 --2- 172.21.15.27:0/64317653 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe4cc0d40e0 0x7fe4cc0b3050 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fe4c400c760 tx=0x7fe4c400cc20 comp rx=0 tx=0).stop 2024-09-12T03:19:18.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:18.846+0000 7fe4d1453640 1 -- 172.21.15.27:0/64317653 shutdown_connections 2024-09-12T03:19:18.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:18.846+0000 7fe4d1453640 1 --2- 172.21.15.27:0/64317653 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fe4b806c010 0x7fe4b806e4d0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:18.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:18.846+0000 7fe4d1453640 1 --2- 172.21.15.27:0/64317653 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe4cc0d40e0 0x7fe4cc0b3050 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:18.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:18.846+0000 7fe4d1453640 1 --2- 172.21.15.27:0/64317653 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe4cc0b6880 0x7fe4cc1f4d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:18.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:18.846+0000 7fe4d1453640 1 --2- 172.21.15.27:0/64317653 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe4cc0b64a0 0x7fe4cc15dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:18.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:18.846+0000 7fe4d1453640 1 -- 172.21.15.27:0/64317653 >> 172.21.15.27:0/64317653 conn(0x7fe4cc0d1230 msgr2=0x7fe4cc0cde50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:18.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:18.846+0000 7fe4d1453640 1 -- 172.21.15.27:0/64317653 shutdown_connections 2024-09-12T03:19:18.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:18.846+0000 7fe4d1453640 1 -- 172.21.15.27:0/64317653 wait complete. 2024-09-12T03:19:18.859 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestMisc.test_connection_expiration' 2024-09-12T03:19:18.961 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:18.960+0000 7f820d5ed640 1 Processor -- start 2024-09-12T03:19:18.962 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:18.960+0000 7f820d5ed640 1 -- start start 2024-09-12T03:19:18.963 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:18.961+0000 7f820d5ed640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f82080d5200 0x7f82080d1010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:18.963 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:18.961+0000 7f820d5ed640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f82080d3760 con 0x7f82080d1550 2024-09-12T03:19:18.963 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:18.961+0000 7f820d5ed640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f82080d3a40 con 0x7f820815eb90 2024-09-12T03:19:18.963 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:18.961+0000 7f820d5ed640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f82080b7d20 con 0x7f82080d5200 2024-09-12T03:19:18.963 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:18.961+0000 7f82067fc640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f82080d5200 0x7f82080d1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:18.963 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:18.961+0000 7f82067fc640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f82080d5200 0x7f82080d1010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55462/0 (socket says 172.21.15.27:55462) 2024-09-12T03:19:18.963 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:18.961+0000 7f82067fc640 1 -- 172.21.15.27:0/2424046125 learned_addr learned my addr 172.21.15.27:0/2424046125 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:19:18.963 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:18.962+0000 7f82067fc640 1 -- 172.21.15.27:0/2424046125 >> v1:172.21.15.45:6789/0 conn(0x7f820815eb90 legacy=0x7f82080d2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:18.963 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:18.962+0000 7f82067fc640 1 -- 172.21.15.27:0/2424046125 >> v1:172.21.15.27:6789/0 conn(0x7f82080d1550 legacy=0x7f82080b8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:18.963 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:18.962+0000 7f82067fc640 1 -- 172.21.15.27:0/2424046125 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82080d3d20 con 0x7f82080d5200 2024-09-12T03:19:18.963 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:18.962+0000 7f8205ffb640 1 -- 172.21.15.27:0/2424046125 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3160113230 0 0) 0x7f82080d3760 con 0x7f82080d1550 2024-09-12T03:19:18.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:18.963+0000 7f82067fc640 1 --2- 172.21.15.27:0/2424046125 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f82080d5200 0x7f82080d1010 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f81f0009870 tx=0x7f81f002a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e0c3585eef0cde server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:18.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:18.963+0000 7f8205ffb640 1 -- 172.21.15.27:0/2424046125 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81f0037070 con 0x7f82080d5200 2024-09-12T03:19:18.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:18.963+0000 7f8205ffb640 1 -- 172.21.15.27:0/2424046125 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81f002ae50 con 0x7f82080d5200 2024-09-12T03:19:18.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:18.963+0000 7f8205ffb640 1 -- 172.21.15.27:0/2424046125 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81f0032860 con 0x7f82080d5200 2024-09-12T03:19:18.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:18.963+0000 7f820d5ed640 1 -- 172.21.15.27:0/2424046125 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f82080d5200 msgr2=0x7f82080d1010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:18.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:18.963+0000 7f820d5ed640 1 --2- 172.21.15.27:0/2424046125 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f82080d5200 0x7f82080d1010 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f81f0009870 tx=0x7f81f002a300 comp rx=0 tx=0).stop 2024-09-12T03:19:18.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:18.963+0000 7f820d5ed640 1 -- 172.21.15.27:0/2424046125 shutdown_connections 2024-09-12T03:19:18.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:18.963+0000 7f820d5ed640 1 --2- 172.21.15.27:0/2424046125 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f82080d5200 0x7f82080d1010 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:18.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:18.963+0000 7f820d5ed640 1 -- 172.21.15.27:0/2424046125 >> 172.21.15.27:0/2424046125 conn(0x7f820806dce0 msgr2=0x7f820806e0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:18.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:18.963+0000 7f820d5ed640 1 -- 172.21.15.27:0/2424046125 shutdown_connections 2024-09-12T03:19:18.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:18.963+0000 7f820d5ed640 1 -- 172.21.15.27:0/2424046125 wait complete. 2024-09-12T03:19:18.967 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:18.965+0000 7f820d5ed640 1 Processor -- start 2024-09-12T03:19:18.967 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:18.965+0000 7f820d5ed640 1 -- start start 2024-09-12T03:19:18.967 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:18.965+0000 7f820d5ed640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f82080d1550 0x7f8208149c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:18.967 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:18.965+0000 7f820d5ed640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f82080d5200 0x7f82081586b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:18.967 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:18.965+0000 7f820d5ed640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f820815eb90 0x7f820814a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:18.967 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:18.965+0000 7f820d5ed640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f82080d3500 con 0x7f82080d5200 2024-09-12T03:19:18.968 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:18.965+0000 7f820d5ed640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f82080d42e0 con 0x7f820815eb90 2024-09-12T03:19:18.968 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:18.965+0000 7f820d5ed640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f82080d1960 con 0x7f82080d1550 2024-09-12T03:19:18.968 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:18.965+0000 7f82067fc640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f82080d5200 0x7f82081586b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:18.968 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:18.966+0000 7f82067fc640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f82080d5200 0x7f82081586b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55466/0 (socket says 172.21.15.27:55466) 2024-09-12T03:19:18.968 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:18.966+0000 7f82067fc640 1 -- 172.21.15.27:0/15052863 learned_addr learned my addr 172.21.15.27:0/15052863 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:19:18.968 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:18.966+0000 7f8206ffd640 1 --2- 172.21.15.27:0/15052863 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f82080d1550 0x7f8208149c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:18.968 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:18.966+0000 7f82077fe640 1 --2- 172.21.15.27:0/15052863 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f820815eb90 0x7f820814a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:18.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:18.966+0000 7f82067fc640 1 -- 172.21.15.27:0/15052863 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f82080d1550 msgr2=0x7f8208149c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:18.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:18.966+0000 7f82067fc640 1 --2- 172.21.15.27:0/15052863 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f82080d1550 0x7f8208149c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:18.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:18.966+0000 7f82067fc640 1 -- 172.21.15.27:0/15052863 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f820815eb90 msgr2=0x7f820814a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:18.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:18.966+0000 7f82067fc640 1 --2- 172.21.15.27:0/15052863 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f820815eb90 0x7f820814a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:18.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:18.966+0000 7f82067fc640 1 -- 172.21.15.27:0/15052863 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8208158bf0 con 0x7f82080d5200 2024-09-12T03:19:18.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:18.966+0000 7f82067fc640 1 --2- 172.21.15.27:0/15052863 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f82080d5200 0x7f82081586b0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f81f0009870 tx=0x7f81f00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:18.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:18.966+0000 7f81ff7fe640 1 -- 172.21.15.27:0/15052863 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81f0037040 con 0x7f82080d5200 2024-09-12T03:19:18.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:18.966+0000 7f81ff7fe640 1 -- 172.21.15.27:0/15052863 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81f0004230 con 0x7f82080d5200 2024-09-12T03:19:18.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:18.966+0000 7f820d5ed640 1 -- 172.21.15.27:0/15052863 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8208158e90 con 0x7f82080d5200 2024-09-12T03:19:18.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:18.966+0000 7f81ff7fe640 1 -- 172.21.15.27:0/15052863 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81f0032e40 con 0x7f82080d5200 2024-09-12T03:19:18.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:18.966+0000 7f820d5ed640 1 -- 172.21.15.27:0/15052863 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8208159450 con 0x7f82080d5200 2024-09-12T03:19:18.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:18.967+0000 7f820d5ed640 1 -- 172.21.15.27:0/15052863 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82080693d0 con 0x7f82080d5200 2024-09-12T03:19:18.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:18.968+0000 7f81ff7fe640 1 -- 172.21.15.27:0/15052863 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 9) ==== 90651+0+0 (secure 0 0 0) 0x7f81f003a070 con 0x7f82080d5200 2024-09-12T03:19:18.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:18.968+0000 7f81ff7fe640 1 --2- 172.21.15.27:0/15052863 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f81e806c010 0x7f81e806e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:18.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:18.968+0000 7f81ff7fe640 1 -- 172.21.15.27:0/15052863 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 4943+0+0 (secure 0 0 0) 0x7f81f00ab4e0 con 0x7f82080d5200 2024-09-12T03:19:18.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:18.969+0000 7f8206ffd640 1 --2- 172.21.15.27:0/15052863 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f81e806c010 0x7f81e806e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:18.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:18.969+0000 7f8206ffd640 1 --2- 172.21.15.27:0/15052863 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f81e806c010 0x7f81e806e4d0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f81f80012e0 tx=0x7f81f8009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:18.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:18.973+0000 7f81ff7fe640 1 -- 172.21.15.27:0/15052863 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f81f0041020 con 0x7f82080d5200 2024-09-12T03:19:19.216 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:19.215+0000 7f820d5ed640 1 -- 172.21.15.27:0/15052863 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_connection_expiration"]} v 0) -- 0x7f820815f120 con 0x7f82080d5200 2024-09-12T03:19:19.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:19.520+0000 7f81ff7fe640 1 -- 172.21.15.27:0/15052863 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_connection_expiration"]}]=0 v154) ==== 140+0+0 (secure 0 0 0) 0x7f81f00798e0 con 0x7f82080d5200 2024-09-12T03:19:19.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:19.522+0000 7f820d5ed640 1 -- 172.21.15.27:0/15052863 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f81e806c010 msgr2=0x7f81e806e4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:19.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:19.522+0000 7f820d5ed640 1 --2- 172.21.15.27:0/15052863 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f81e806c010 0x7f81e806e4d0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f81f80012e0 tx=0x7f81f8009040 comp rx=0 tx=0).stop 2024-09-12T03:19:19.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:19.522+0000 7f820d5ed640 1 -- 172.21.15.27:0/15052863 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f82080d5200 msgr2=0x7f82081586b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:19.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:19.522+0000 7f820d5ed640 1 --2- 172.21.15.27:0/15052863 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f82080d5200 0x7f82081586b0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f81f0009870 tx=0x7f81f00046c0 comp rx=0 tx=0).stop 2024-09-12T03:19:19.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:19.522+0000 7f820d5ed640 1 -- 172.21.15.27:0/15052863 shutdown_connections 2024-09-12T03:19:19.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:19.522+0000 7f820d5ed640 1 --2- 172.21.15.27:0/15052863 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f81e806c010 0x7f81e806e4d0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:19.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:19.522+0000 7f820d5ed640 1 --2- 172.21.15.27:0/15052863 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f820815eb90 0x7f820814a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:19.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:19.522+0000 7f820d5ed640 1 --2- 172.21.15.27:0/15052863 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f82080d5200 0x7f82081586b0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:19.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:19.522+0000 7f820d5ed640 1 --2- 172.21.15.27:0/15052863 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f82080d1550 0x7f8208149c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:19.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:19.522+0000 7f820d5ed640 1 -- 172.21.15.27:0/15052863 >> 172.21.15.27:0/15052863 conn(0x7f820806dce0 msgr2=0x7f82081612b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:19.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:19.522+0000 7f820d5ed640 1 -- 172.21.15.27:0/15052863 shutdown_connections 2024-09-12T03:19:19.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:19.522+0000 7f820d5ed640 1 -- 172.21.15.27:0/15052863 wait complete. 2024-09-12T03:19:19.535 INFO:tasks.cephfs_test_runner:test_connection_expiration (tasks.cephfs.test_volumes.TestMisc) ... ok 2024-09-12T03:19:19.536 INFO:tasks.cephfs_test_runner:Starting test: test_malicious_metafile_on_legacy_to_v1_upgrade (tasks.cephfs.test_volumes.TestMisc) Validate handcrafted .meta file on legacy subvol root doesn't break the system 2024-09-12T03:19:19.536 INFO:tasks.cephfs_test_runner:test_malicious_metafile_on_legacy_to_v1_upgrade (tasks.cephfs.test_volumes.TestMisc) 2024-09-12T03:19:19.537 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-12T03:19:19.641 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:19.640+0000 7fe9b881e640 1 Processor -- start 2024-09-12T03:19:19.642 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:19.640+0000 7fe9b881e640 1 -- start start 2024-09-12T03:19:19.645 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:19.641+0000 7fe9b881e640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe9b415e780 0x7fe9b40d3e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:19.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:19.641+0000 7fe9b881e640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe9b40b6690 con 0x7fe9b40d1990 2024-09-12T03:19:19.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:19.641+0000 7fe9b881e640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe9b40b6970 con 0x7fe9b40d43b0 2024-09-12T03:19:19.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:19.641+0000 7fe9b881e640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fe9b40b7950 con 0x7fe9b415e780 2024-09-12T03:19:19.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:19.641+0000 7fe9b2d9d640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fe9b40d1990 0x7fe9b40b3390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:49062/0 (socket says 172.21.15.27:49062) 2024-09-12T03:19:19.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:19.641+0000 7fe9b2d9d640 1 -- 172.21.15.27:0/1768933949 learned_addr learned my addr 172.21.15.27:0/1768933949 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:19:19.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:19.641+0000 7fe9b259c640 1 --2- 172.21.15.27:0/1768933949 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe9b415e780 0x7fe9b40d3e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:19.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:19.641+0000 7fe9b159a640 1 -- 172.21.15.27:0/1768933949 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1465991617 0 0) 0x7fe9b40b6690 con 0x7fe9b40d1990 2024-09-12T03:19:19.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:19.641+0000 7fe9b159a640 1 -- 172.21.15.27:0/1768933949 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe99c003520 con 0x7fe9b40d1990 2024-09-12T03:19:19.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:19.641+0000 7fe9b159a640 1 -- 172.21.15.27:0/1768933949 <== mon.1 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1977274556 0 0) 0x7fe9b40b6970 con 0x7fe9b40d43b0 2024-09-12T03:19:19.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:19.641+0000 7fe9b159a640 1 -- 172.21.15.27:0/1768933949 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe9b40b6690 con 0x7fe9b40d43b0 2024-09-12T03:19:19.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:19.641+0000 7fe9b259c640 1 -- 172.21.15.27:0/1768933949 >> v1:172.21.15.45:6789/0 conn(0x7fe9b40d43b0 legacy=0x7fe9b40d1280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:19.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:19.641+0000 7fe9b259c640 1 -- 172.21.15.27:0/1768933949 >> v1:172.21.15.27:6789/0 conn(0x7fe9b40d1990 legacy=0x7fe9b40b3390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:19.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:19.641+0000 7fe9b259c640 1 -- 172.21.15.27:0/1768933949 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9b40b6c50 con 0x7fe9b415e780 2024-09-12T03:19:19.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:19.642+0000 7fe9b259c640 1 --2- 172.21.15.27:0/1768933949 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe9b415e780 0x7fe9b40d3e70 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fe9a40097d0 tx=0x7fe9a402a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2b1485def4707ef3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:19.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:19.642+0000 7fe9b159a640 1 -- 172.21.15.27:0/1768933949 <== mon.2 v2:172.21.15.45:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9a4037070 con 0x7fe9b415e780 2024-09-12T03:19:19.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:19.642+0000 7fe9b159a640 1 -- 172.21.15.27:0/1768933949 <== mon.2 v2:172.21.15.45:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9a4032440 con 0x7fe9b415e780 2024-09-12T03:19:19.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:19.642+0000 7fe9b159a640 1 -- 172.21.15.27:0/1768933949 <== mon.2 v2:172.21.15.45:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9a4032810 con 0x7fe9b415e780 2024-09-12T03:19:19.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:19.643+0000 7fe9b881e640 1 -- 172.21.15.27:0/1768933949 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe9b415e780 msgr2=0x7fe9b40d3e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:19.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:19.643+0000 7fe9b881e640 1 --2- 172.21.15.27:0/1768933949 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe9b415e780 0x7fe9b40d3e70 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fe9a40097d0 tx=0x7fe9a402a210 comp rx=0 tx=0).stop 2024-09-12T03:19:19.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:19.643+0000 7fe9b881e640 1 -- 172.21.15.27:0/1768933949 shutdown_connections 2024-09-12T03:19:19.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:19.643+0000 7fe9b881e640 1 --2- 172.21.15.27:0/1768933949 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe9b415e780 0x7fe9b40d3e70 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:19.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:19.643+0000 7fe9b881e640 1 -- 172.21.15.27:0/1768933949 >> 172.21.15.27:0/1768933949 conn(0x7fe9b406da60 msgr2=0x7fe9b406de70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:19.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:19.643+0000 7fe9b881e640 1 -- 172.21.15.27:0/1768933949 shutdown_connections 2024-09-12T03:19:19.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:19.643+0000 7fe9b881e640 1 -- 172.21.15.27:0/1768933949 wait complete. 2024-09-12T03:19:19.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:19.644+0000 7fe9b881e640 1 Processor -- start 2024-09-12T03:19:19.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:19.644+0000 7fe9b881e640 1 -- start start 2024-09-12T03:19:19.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:19.645+0000 7fe9b881e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe9b40d1990 0x7fe9b41f53f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:19.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:19.645+0000 7fe9b881e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe9b40d43b0 0x7fe9b4158ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:19.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:19.645+0000 7fe9b881e640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe9b415e780 0x7fe9b4161110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:19.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:19.645+0000 7fe9b881e640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fe9b40b6340 con 0x7fe9b40d43b0 2024-09-12T03:19:19.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:19.645+0000 7fe9b881e640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe9b40b7210 con 0x7fe9b40d1990 2024-09-12T03:19:19.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:19.645+0000 7fe9b881e640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fe9b40b8dd0 con 0x7fe9b415e780 2024-09-12T03:19:19.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:19.645+0000 7fe9b1d9b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe9b40d43b0 0x7fe9b4158ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:19.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:19.645+0000 7fe9b1d9b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe9b40d43b0 0x7fe9b4158ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42180/0 (socket says 172.21.15.27:42180) 2024-09-12T03:19:19.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:19.645+0000 7fe9b1d9b640 1 -- 172.21.15.27:0/1234861786 learned_addr learned my addr 172.21.15.27:0/1234861786 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:19:19.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:19.646+0000 7fe9b259c640 1 --2- 172.21.15.27:0/1234861786 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe9b40d1990 0x7fe9b41f53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:19.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:19.646+0000 7fe9b1d9b640 1 -- 172.21.15.27:0/1234861786 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe9b415e780 msgr2=0x7fe9b4161110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:19.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:19.646+0000 7fe9b2d9d640 1 --2- 172.21.15.27:0/1234861786 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe9b415e780 0x7fe9b4161110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:19.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:19.646+0000 7fe9b1d9b640 1 --2- 172.21.15.27:0/1234861786 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe9b415e780 0x7fe9b4161110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:19.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:19.646+0000 7fe9b1d9b640 1 -- 172.21.15.27:0/1234861786 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe9b40d1990 msgr2=0x7fe9b41f53f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:19.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:19.646+0000 7fe9b1d9b640 1 --2- 172.21.15.27:0/1234861786 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe9b40d1990 0x7fe9b41f53f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:19.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:19.646+0000 7fe9b1d9b640 1 -- 172.21.15.27:0/1234861786 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9b40cd260 con 0x7fe9b40d43b0 2024-09-12T03:19:19.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:19.646+0000 7fe9b259c640 1 --2- 172.21.15.27:0/1234861786 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe9b40d1990 0x7fe9b41f53f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:19:19.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:19.646+0000 7fe9b2d9d640 1 --2- 172.21.15.27:0/1234861786 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe9b415e780 0x7fe9b4161110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:19:19.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:19.646+0000 7fe9b1d9b640 1 --2- 172.21.15.27:0/1234861786 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe9b40d43b0 0x7fe9b4158ea0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fe9a800c450 tx=0x7fe9a800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:19.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:19.646+0000 7fe9a2ffd640 1 -- 172.21.15.27:0/1234861786 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9a80052d0 con 0x7fe9b40d43b0 2024-09-12T03:19:19.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:19.646+0000 7fe9a2ffd640 1 -- 172.21.15.27:0/1234861786 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9a8005470 con 0x7fe9b40d43b0 2024-09-12T03:19:19.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:19.646+0000 7fe9b881e640 1 -- 172.21.15.27:0/1234861786 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9b40cd540 con 0x7fe9b40d43b0 2024-09-12T03:19:19.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:19.646+0000 7fe9a2ffd640 1 -- 172.21.15.27:0/1234861786 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9a8012870 con 0x7fe9b40d43b0 2024-09-12T03:19:19.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:19.646+0000 7fe9b881e640 1 -- 172.21.15.27:0/1234861786 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9b40cdb00 con 0x7fe9b40d43b0 2024-09-12T03:19:19.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:19.647+0000 7fe9b881e640 1 -- 172.21.15.27:0/1234861786 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9b415eb60 con 0x7fe9b40d43b0 2024-09-12T03:19:19.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:19.648+0000 7fe9a2ffd640 1 -- 172.21.15.27:0/1234861786 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 9) ==== 90651+0+0 (secure 0 0 0) 0x7fe9a8012a10 con 0x7fe9b40d43b0 2024-09-12T03:19:19.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:19.648+0000 7fe9a2ffd640 1 --2- 172.21.15.27:0/1234861786 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fe99406bfe0 0x7fe99406e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:19.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:19.648+0000 7fe9a2ffd640 1 -- 172.21.15.27:0/1234861786 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 4943+0+0 (secure 0 0 0) 0x7fe9a808aaf0 con 0x7fe9b40d43b0 2024-09-12T03:19:19.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:19.648+0000 7fe9b259c640 1 --2- 172.21.15.27:0/1234861786 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fe99406bfe0 0x7fe99406e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:19.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:19.649+0000 7fe9b259c640 1 --2- 172.21.15.27:0/1234861786 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fe99406bfe0 0x7fe99406e4a0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fe9a4034000 tx=0x7fe9a402e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:19.657 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:19.652+0000 7fe9a2ffd640 1 -- 172.21.15.27:0/1234861786 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fe9a80595e0 con 0x7fe9b40d43b0 2024-09-12T03:19:19.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:19.830+0000 7fe9b881e640 1 -- 172.21.15.27:0/1234861786 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe9b40d2440 con 0x7fe9b40d43b0 2024-09-12T03:19:19.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:19.831+0000 7fe9a2ffd640 1 -- 172.21.15.27:0/1234861786 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v48) ==== 74+0+12984 (secure 0 0 0) 0x7fe9a8058ee0 con 0x7fe9b40d43b0 2024-09-12T03:19:19.832 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:19:19.832 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":48,"fsid":"d2df504e-2647-4385-a5db-9fdf69f487a0","created":"2024-09-12T03:16:07.778937+0000","modified":"2024-09-12T03:19:18.513427+0000","last_up_change":"2024-09-12T03:16:19.314778+0000","last_in_change":"2024-09-12T03:16:09.893086+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T03:16:20.097646+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1851116c-1fb3-4dc9-9772-f90e2ef4c355","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6817","nonce":3002321488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6819","nonce":3002321488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6823","nonce":3002321488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6821","nonce":3002321488}]},"public_addr":"172.21.15.27:6817/3002321488","cluster_addr":"172.21.15.27:6819/3002321488","heartbeat_back_addr":"172.21.15.27:6823/3002321488","heartbeat_front_addr":"172.21.15.27:6821/3002321488","state":["exists","up"]},{"osd":1,"uuid":"f38ae17d-73eb-4204-b99d-975fa806fb87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6809","nonce":1613102886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6811","nonce":1613102886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6815","nonce":1613102886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6813","nonce":1613102886}]},"public_addr":"172.21.15.27:6809/1613102886","cluster_addr":"172.21.15.27:6811/1613102886","heartbeat_back_addr":"172.21.15.27:6815/1613102886","heartbeat_front_addr":"172.21.15.27:6813/1613102886","state":["exists","up"]},{"osd":2,"uuid":"fb12dcc8-bdd0-42b8-b46d-8b10106224bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6825","nonce":3534723355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6827","nonce":3534723355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6831","nonce":3534723355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6829","nonce":3534723355}]},"public_addr":"172.21.15.27:6825/3534723355","cluster_addr":"172.21.15.27:6827/3534723355","heartbeat_back_addr":"172.21.15.27:6831/3534723355","heartbeat_front_addr":"172.21.15.27:6829/3534723355","state":["exists","up"]},{"osd":3,"uuid":"d74d3f49-9c24-4b9f-98e2-3ecd7b5ef3b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6801","nonce":80661598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6803","nonce":80661598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6807","nonce":80661598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6805","nonce":80661598}]},"public_addr":"172.21.15.27:6801/80661598","cluster_addr":"172.21.15.27:6803/80661598","heartbeat_back_addr":"172.21.15.27:6807/80661598","heartbeat_front_addr":"172.21.15.27:6805/80661598","state":["exists","up"]},{"osd":4,"uuid":"2bf123f2-1b0d-41c4-aa3b-805bab267e59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6817","nonce":3943189827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6819","nonce":3943189827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6823","nonce":3943189827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6821","nonce":3943189827}]},"public_addr":"172.21.15.45:6817/3943189827","cluster_addr":"172.21.15.45:6819/3943189827","heartbeat_back_addr":"172.21.15.45:6823/3943189827","heartbeat_front_addr":"172.21.15.45:6821/3943189827","state":["exists","up"]},{"osd":5,"uuid":"c49ba5da-91c6-4a86-a97b-4aba89c253ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6809","nonce":3620760330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6811","nonce":3620760330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6815","nonce":3620760330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6813","nonce":3620760330}]},"public_addr":"172.21.15.45:6809/3620760330","cluster_addr":"172.21.15.45:6811/3620760330","heartbeat_back_addr":"172.21.15.45:6815/3620760330","heartbeat_front_addr":"172.21.15.45:6813/3620760330","state":["exists","up"]},{"osd":6,"uuid":"b0123475-89f7-4a34-9e33-d797b99dd9dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6801","nonce":1989083510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6803","nonce":1989083510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6807","nonce":1989083510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6805","nonce":1989083510}]},"public_addr":"172.21.15.45:6801/1989083510","cluster_addr":"172.21.15.45:6803/1989083510","heartbeat_back_addr":"172.21.15.45:6807/1989083510","heartbeat_front_addr":"172.21.15.45:6805/1989083510","state":["exists","up"]},{"osd":7,"uuid":"4dad773d-b7d8-46fc-8a10-a972aad27b29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6825","nonce":1513300294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6827","nonce":1513300294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6831","nonce":1513300294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6829","nonce":1513300294}]},"public_addr":"172.21.15.45:6825/1513300294","cluster_addr":"172.21.15.45:6827/1513300294","heartbeat_back_addr":"172.21.15.45:6831/1513300294","heartbeat_front_addr":"172.21.15.45:6829/1513300294","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.881855+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:17.133252+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.735339+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.500742+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.477571+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:17.167667+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.470291+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.873743+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.27:6835/2571101228":"2024-09-13T03:19:15.506179+0000","172.21.15.27:6833/2571101228":"2024-09-13T03:19:15.506179+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T03:19:19.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:19.833+0000 7fe9b881e640 1 -- 172.21.15.27:0/1234861786 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fe99406bfe0 msgr2=0x7fe99406e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:19.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:19.833+0000 7fe9b881e640 1 --2- 172.21.15.27:0/1234861786 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fe99406bfe0 0x7fe99406e4a0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fe9a4034000 tx=0x7fe9a402e000 comp rx=0 tx=0).stop 2024-09-12T03:19:19.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:19.833+0000 7fe9b881e640 1 -- 172.21.15.27:0/1234861786 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe9b40d43b0 msgr2=0x7fe9b4158ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:19.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:19.833+0000 7fe9b881e640 1 --2- 172.21.15.27:0/1234861786 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe9b40d43b0 0x7fe9b4158ea0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fe9a800c450 tx=0x7fe9a800c910 comp rx=0 tx=0).stop 2024-09-12T03:19:19.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:19.833+0000 7fe9b881e640 1 -- 172.21.15.27:0/1234861786 shutdown_connections 2024-09-12T03:19:19.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:19.833+0000 7fe9b881e640 1 --2- 172.21.15.27:0/1234861786 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fe99406bfe0 0x7fe99406e4a0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:19.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:19.833+0000 7fe9b881e640 1 --2- 172.21.15.27:0/1234861786 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe9b415e780 0x7fe9b4161110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:19.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:19.833+0000 7fe9b881e640 1 --2- 172.21.15.27:0/1234861786 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe9b40d43b0 0x7fe9b4158ea0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:19.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:19.833+0000 7fe9b881e640 1 --2- 172.21.15.27:0/1234861786 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe9b40d1990 0x7fe9b41f53f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:19.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:19.833+0000 7fe9b881e640 1 -- 172.21.15.27:0/1234861786 >> 172.21.15.27:0/1234861786 conn(0x7fe9b406da60 msgr2=0x7fe9b40d1280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:19.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:19.833+0000 7fe9b881e640 1 -- 172.21.15.27:0/1234861786 shutdown_connections 2024-09-12T03:19:19.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:19.833+0000 7fe9b881e640 1 -- 172.21.15.27:0/1234861786 wait complete. 2024-09-12T03:19:19.845 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-12T03:16:20.097646+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-12T03:19:19.846 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-12T03:19:19.951 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:19.950+0000 7f4c81ca5640 1 Processor -- start 2024-09-12T03:19:19.951 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:19.950+0000 7f4c81ca5640 1 -- start start 2024-09-12T03:19:19.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:19.951+0000 7f4c81ca5640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f4c7c070d30 0x7f4c7c06cc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:19.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:19.951+0000 7f4c81ca5640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4c7c0cf9f0 con 0x7f4c7c0d0b50 2024-09-12T03:19:19.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:19.951+0000 7f4c81ca5640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4c7c0cfc90 con 0x7f4c7c06e960 2024-09-12T03:19:19.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:19.951+0000 7f4c81ca5640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f4c7c0d1a40 con 0x7f4c7c070d30 2024-09-12T03:19:19.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:19.951+0000 7f4c7bfff640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f4c7c0d0b50 0x7f4c7c168300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:49068/0 (socket says 172.21.15.27:49068) 2024-09-12T03:19:19.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:19.951+0000 7f4c7bfff640 1 -- 172.21.15.27:0/24779651 learned_addr learned my addr 172.21.15.27:0/24779651 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:19:19.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:19.952+0000 7f4c7b7fe640 1 --2- 172.21.15.27:0/24779651 >> v2:172.21.15.45:3300/0 conn(0x7f4c7c070d30 0x7f4c7c06cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:19.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:19.952+0000 7f4c7a7fc640 1 -- 172.21.15.27:0/24779651 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4175467048 0 0) 0x7f4c7c0cf9f0 con 0x7f4c7c0d0b50 2024-09-12T03:19:19.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:19.952+0000 7f4c7a7fc640 1 -- 172.21.15.27:0/24779651 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4c64003520 con 0x7f4c7c0d0b50 2024-09-12T03:19:19.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:19.952+0000 7f4c7a7fc640 1 -- 172.21.15.27:0/24779651 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2653059586 0 0) 0x7f4c64003520 con 0x7f4c7c0d0b50 2024-09-12T03:19:19.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:19.952+0000 7f4c7a7fc640 1 -- 172.21.15.27:0/24779651 >> v2:172.21.15.45:3300/0 conn(0x7f4c7c070d30 msgr2=0x7f4c7c06cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:19.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:19.952+0000 7f4c7a7fc640 1 --2- 172.21.15.27:0/24779651 >> v2:172.21.15.45:3300/0 conn(0x7f4c7c070d30 0x7f4c7c06cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:19.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:19.952+0000 7f4c7a7fc640 1 -- 172.21.15.27:0/24779651 >> v1:172.21.15.45:6789/0 conn(0x7f4c7c06e960 legacy=0x7f4c7c0d0440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:19.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:19.952+0000 7f4c7a7fc640 1 -- 172.21.15.27:0/24779651 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c7c0cff70 con 0x7f4c7c0d0b50 2024-09-12T03:19:19.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:19.952+0000 7f4c7a7fc640 1 -- 172.21.15.27:0/24779651 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 847755213 0 0) 0x7f4c74001a10 con 0x7f4c7c0d0b50 2024-09-12T03:19:19.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:19.952+0000 7f4c7a7fc640 1 -- 172.21.15.27:0/24779651 >> v1:172.21.15.27:6789/0 conn(0x7f4c7c0d0b50 legacy=0x7f4c7c168300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:19.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:19.952+0000 7f4c7a7fc640 1 --2- 172.21.15.27:0/24779651 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4c640039b0 0x7f4c64003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:19.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:19.952+0000 7f4c7a7fc640 1 --2- 172.21.15.27:0/24779651 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4c64004800 0x7f4c64004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:19.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:19.952+0000 7f4c7a7fc640 1 --2- 172.21.15.27:0/24779651 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4c640052e0 0x7f4c6400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:19.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:19.952+0000 7f4c7a7fc640 1 -- 172.21.15.27:0/24779651 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f4c7c0d1a40 con 0x7f4c64004800 2024-09-12T03:19:19.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:19.952+0000 7f4c7a7fc640 1 -- 172.21.15.27:0/24779651 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f4c6400c1b0 con 0x7f4c640052e0 2024-09-12T03:19:19.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:19.952+0000 7f4c7a7fc640 1 -- 172.21.15.27:0/24779651 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f4c6400c3b0 con 0x7f4c640039b0 2024-09-12T03:19:19.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:19.952+0000 7f4c7affd640 1 --2- 172.21.15.27:0/24779651 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4c64004800 0x7f4c64004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:19.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:19.952+0000 7f4c7b7fe640 1 --2- 172.21.15.27:0/24779651 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4c640039b0 0x7f4c64003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:19.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:19.952+0000 7f4c7bfff640 1 --2- 172.21.15.27:0/24779651 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4c640052e0 0x7f4c6400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:19.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:19.952+0000 7f4c7affd640 1 -- 172.21.15.27:0/24779651 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4c640039b0 msgr2=0x7f4c64003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:19.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:19.952+0000 7f4c7affd640 1 --2- 172.21.15.27:0/24779651 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4c640039b0 0x7f4c64003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:19.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:19.952+0000 7f4c7affd640 1 -- 172.21.15.27:0/24779651 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4c640052e0 msgr2=0x7f4c6400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:19.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:19.952+0000 7f4c7affd640 1 --2- 172.21.15.27:0/24779651 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4c640052e0 0x7f4c6400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:19.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:19.952+0000 7f4c7affd640 1 -- 172.21.15.27:0/24779651 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c64003520 con 0x7f4c64004800 2024-09-12T03:19:19.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:19.952+0000 7f4c7bfff640 1 -- 172.21.15.27:0/24779651 reap_dead start 2024-09-12T03:19:19.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:19.953+0000 7f4c7affd640 1 --2- 172.21.15.27:0/24779651 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4c64004800 0x7f4c64004c80 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f4c70001720 tx=0x7f4c64005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cac81eb68593a4bf server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:19.959 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:19.953+0000 7f4c7a7fc640 1 -- 172.21.15.27:0/24779651 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c70001370 con 0x7f4c64004800 2024-09-12T03:19:19.959 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:19.953+0000 7f4c7a7fc640 1 -- 172.21.15.27:0/24779651 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c700052d0 con 0x7f4c64004800 2024-09-12T03:19:19.959 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:19.953+0000 7f4c7a7fc640 1 -- 172.21.15.27:0/24779651 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c7002ab90 con 0x7f4c64004800 2024-09-12T03:19:19.959 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:19.954+0000 7f4c81ca5640 1 -- 172.21.15.27:0/24779651 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4c64004800 msgr2=0x7f4c64004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:19.959 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:19.954+0000 7f4c81ca5640 1 --2- 172.21.15.27:0/24779651 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4c64004800 0x7f4c64004c80 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f4c70001720 tx=0x7f4c64005830 comp rx=0 tx=0).stop 2024-09-12T03:19:19.959 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:19.954+0000 7f4c81ca5640 1 -- 172.21.15.27:0/24779651 shutdown_connections 2024-09-12T03:19:19.959 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:19.954+0000 7f4c81ca5640 1 --2- 172.21.15.27:0/24779651 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4c64004800 0x7f4c64004c80 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:19.959 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:19.954+0000 7f4c81ca5640 1 -- 172.21.15.27:0/24779651 >> 172.21.15.27:0/24779651 conn(0x7f4c7c0b7c10 msgr2=0x7f4c7c09ba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:19.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:19.954+0000 7f4c81ca5640 1 -- 172.21.15.27:0/24779651 shutdown_connections 2024-09-12T03:19:19.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:19.954+0000 7f4c81ca5640 1 -- 172.21.15.27:0/24779651 wait complete. 2024-09-12T03:19:19.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:19.955+0000 7f4c81ca5640 1 Processor -- start 2024-09-12T03:19:19.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:19.955+0000 7f4c81ca5640 1 -- start start 2024-09-12T03:19:19.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:19.956+0000 7f4c81ca5640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4c64004800 0x7f4c7c158190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:19.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:19.956+0000 7f4c81ca5640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4c7c15ecf0 0x7f4c7c1f5610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:19.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:19.956+0000 7f4c81ca5640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4c7c0cd860 0x7f4c7c0cdd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:19.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:19.956+0000 7f4c81ca5640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f4c7c0d05c0 con 0x7f4c7c0cd860 2024-09-12T03:19:19.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:19.956+0000 7f4c81ca5640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f4c7c0d0440 con 0x7f4c7c15ecf0 2024-09-12T03:19:19.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:19.956+0000 7f4c81ca5640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f4c7c0d0740 con 0x7f4c64004800 2024-09-12T03:19:19.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:19.956+0000 7f4c7bfff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4c7c0cd860 0x7f4c7c0cdd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:19.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:19.956+0000 7f4c7bfff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4c7c0cd860 0x7f4c7c0cdd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42192/0 (socket says 172.21.15.27:42192) 2024-09-12T03:19:19.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:19.956+0000 7f4c7bfff640 1 -- 172.21.15.27:0/1504908757 learned_addr learned my addr 172.21.15.27:0/1504908757 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:19:19.961 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:19.956+0000 7f4c7affd640 1 --2- 172.21.15.27:0/1504908757 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4c7c15ecf0 0x7f4c7c1f5610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:19.961 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:19.956+0000 7f4c7b7fe640 1 --2- 172.21.15.27:0/1504908757 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4c64004800 0x7f4c7c158190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:19.961 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:19.956+0000 7f4c7bfff640 1 -- 172.21.15.27:0/1504908757 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4c64004800 msgr2=0x7f4c7c158190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:19.961 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:19.956+0000 7f4c7bfff640 1 --2- 172.21.15.27:0/1504908757 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4c64004800 0x7f4c7c158190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:19.961 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:19.956+0000 7f4c7bfff640 1 -- 172.21.15.27:0/1504908757 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4c7c15ecf0 msgr2=0x7f4c7c1f5610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:19.961 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:19.956+0000 7f4c7bfff640 1 --2- 172.21.15.27:0/1504908757 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4c7c15ecf0 0x7f4c7c1f5610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:19.961 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:19.956+0000 7f4c7bfff640 1 -- 172.21.15.27:0/1504908757 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c7c1f5b50 con 0x7f4c7c0cd860 2024-09-12T03:19:19.961 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:19.957+0000 7f4c7bfff640 1 --2- 172.21.15.27:0/1504908757 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4c7c0cd860 0x7f4c7c0cdd10 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f4c7c06df30 tx=0x7f4c740079d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:19.961 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:19.957+0000 7f4c63fff640 1 -- 172.21.15.27:0/1504908757 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c7400aa40 con 0x7f4c7c0cd860 2024-09-12T03:19:19.961 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:19.957+0000 7f4c63fff640 1 -- 172.21.15.27:0/1504908757 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c74019070 con 0x7f4c7c0cd860 2024-09-12T03:19:19.961 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:19.957+0000 7f4c81ca5640 1 -- 172.21.15.27:0/1504908757 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c7c161840 con 0x7f4c7c0cd860 2024-09-12T03:19:19.961 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:19.957+0000 7f4c63fff640 1 -- 172.21.15.27:0/1504908757 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c7400ae60 con 0x7f4c7c0cd860 2024-09-12T03:19:19.961 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:19.957+0000 7f4c81ca5640 1 -- 172.21.15.27:0/1504908757 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c7c15f490 con 0x7f4c7c0cd860 2024-09-12T03:19:19.961 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:19.958+0000 7f4c81ca5640 1 -- 172.21.15.27:0/1504908757 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c7c06cc40 con 0x7f4c7c0cd860 2024-09-12T03:19:19.962 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:19.958+0000 7f4c63fff640 1 -- 172.21.15.27:0/1504908757 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 9) ==== 90651+0+0 (secure 0 0 0) 0x7f4c7401e460 con 0x7f4c7c0cd860 2024-09-12T03:19:19.962 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:19.959+0000 7f4c63fff640 1 --2- 172.21.15.27:0/1504908757 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f4c6806bfe0 0x7f4c6806e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:19.962 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:19.959+0000 7f4c63fff640 1 -- 172.21.15.27:0/1504908757 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 4943+0+0 (secure 0 0 0) 0x7f4c7408b240 con 0x7f4c7c0cd860 2024-09-12T03:19:19.962 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:19.959+0000 7f4c7b7fe640 1 --2- 172.21.15.27:0/1504908757 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f4c6806bfe0 0x7f4c6806e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:19.962 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:19.959+0000 7f4c7b7fe640 1 --2- 172.21.15.27:0/1504908757 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f4c6806bfe0 0x7f4c6806e4a0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f4c6c009bc0 tx=0x7f4c6c009190 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:19.964 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:19.963+0000 7f4c63fff640 1 -- 172.21.15.27:0/1504908757 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f4c7c1f5b50 con 0x7f4c7c0cd860 2024-09-12T03:19:20.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:20.138+0000 7f4c81ca5640 1 -- 172.21.15.27:0/1504908757 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f4c7c0d1230 con 0x7f4c7c0cd860 2024-09-12T03:19:20.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:20.138+0000 7f4c63fff640 1 -- 172.21.15.27:0/1504908757 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v48) ==== 93+0+10 (secure 0 0 0) 0x7f4c74059c80 con 0x7f4c7c0cd860 2024-09-12T03:19:20.139 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 1 2024-09-12T03:19:20.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:20.140+0000 7f4c81ca5640 1 -- 172.21.15.27:0/1504908757 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f4c6806bfe0 msgr2=0x7f4c6806e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:20.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:20.140+0000 7f4c81ca5640 1 --2- 172.21.15.27:0/1504908757 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f4c6806bfe0 0x7f4c6806e4a0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f4c6c009bc0 tx=0x7f4c6c009190 comp rx=0 tx=0).stop 2024-09-12T03:19:20.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:20.140+0000 7f4c81ca5640 1 -- 172.21.15.27:0/1504908757 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4c7c0cd860 msgr2=0x7f4c7c0cdd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:20.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:20.140+0000 7f4c81ca5640 1 --2- 172.21.15.27:0/1504908757 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4c7c0cd860 0x7f4c7c0cdd10 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f4c7c06df30 tx=0x7f4c740079d0 comp rx=0 tx=0).stop 2024-09-12T03:19:20.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:20.140+0000 7f4c81ca5640 1 -- 172.21.15.27:0/1504908757 shutdown_connections 2024-09-12T03:19:20.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:20.140+0000 7f4c81ca5640 1 --2- 172.21.15.27:0/1504908757 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f4c6806bfe0 0x7f4c6806e4a0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:20.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:20.140+0000 7f4c81ca5640 1 --2- 172.21.15.27:0/1504908757 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4c7c0cd860 0x7f4c7c0cdd10 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:20.142 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:20.140+0000 7f4c81ca5640 1 --2- 172.21.15.27:0/1504908757 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4c7c15ecf0 0x7f4c7c1f5610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:20.142 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:20.140+0000 7f4c81ca5640 1 --2- 172.21.15.27:0/1504908757 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4c64004800 0x7f4c7c158190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:20.142 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:20.140+0000 7f4c81ca5640 1 -- 172.21.15.27:0/1504908757 >> 172.21.15.27:0/1504908757 conn(0x7f4c7c0b7c10 msgr2=0x7f4c7c09ba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:20.142 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:20.140+0000 7f4c81ca5640 1 -- 172.21.15.27:0/1504908757 shutdown_connections 2024-09-12T03:19:20.142 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:20.140+0000 7f4c81ca5640 1 -- 172.21.15.27:0/1504908757 wait complete. 2024-09-12T03:19:20.153 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestMisc.test_malicious_metafile_on_legacy_to_v1_upgrade' 2024-09-12T03:19:20.256 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:20.255+0000 7fc703a4c640 1 Processor -- start 2024-09-12T03:19:20.256 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:20.255+0000 7fc703a4c640 1 -- start start 2024-09-12T03:19:20.256 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:20.256+0000 7fc703a4c640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc6fc15eb90 0x7fc6fc0d2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:20.257 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:20.256+0000 7fc703a4c640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fc6fc0d1720 0x7fc6fc0b8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:20.257 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:20.256+0000 7fc703a4c640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc6fc0d3760 con 0x7fc6fc0d5200 2024-09-12T03:19:20.257 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:20.256+0000 7fc703a4c640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fc6fc0b7d20 con 0x7fc6fc15eb90 2024-09-12T03:19:20.257 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:20.256+0000 7fc703a4c640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fc6fc0d42e0 con 0x7fc6fc0d1720 2024-09-12T03:19:20.258 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:20.256+0000 7fc7017e8640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc6fc15eb90 0x7fc6fc0d2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:20.258 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:20.256+0000 7fc701fe9640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fc6fc0d1720 0x7fc6fc0b8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:20.258 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:20.256+0000 7fc700fe7640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fc6fc0d5200 0x7fc6fc0d1010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.27:51990/0 (socket says 172.21.15.27:51990) 2024-09-12T03:19:20.258 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:20.256+0000 7fc700fe7640 1 -- 172.21.15.27:0/3156201773 learned_addr learned my addr 172.21.15.27:0/3156201773 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:19:20.258 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:20.256+0000 7fc6f3fff640 1 -- 172.21.15.27:0/3156201773 <== mon.1 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 559337201 0 0) 0x7fc6fc0d3760 con 0x7fc6fc0d5200 2024-09-12T03:19:20.258 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:20.257+0000 7fc701fe9640 1 -- 172.21.15.27:0/3156201773 >> v2:172.21.15.45:3300/0 conn(0x7fc6fc15eb90 msgr2=0x7fc6fc0d2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:20.258 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:20.257+0000 7fc701fe9640 1 --2- 172.21.15.27:0/3156201773 >> v2:172.21.15.45:3300/0 conn(0x7fc6fc15eb90 0x7fc6fc0d2df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:20.258 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:20.257+0000 7fc701fe9640 1 -- 172.21.15.27:0/3156201773 >> v1:172.21.15.45:6789/0 conn(0x7fc6fc0d5200 legacy=0x7fc6fc0d1010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:20.258 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:20.257+0000 7fc701fe9640 1 -- 172.21.15.27:0/3156201773 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6fc0d3a40 con 0x7fc6fc0d1720 2024-09-12T03:19:20.260 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:20.257+0000 7fc701fe9640 1 --2- 172.21.15.27:0/3156201773 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fc6fc0d1720 0x7fc6fc0b8190 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fc6f4009d90 tx=0x7fc6f402a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5d1d71eb7bcfd6af server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:20.260 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:20.258+0000 7fc6f3fff640 1 -- 172.21.15.27:0/3156201773 <== mon.2 v2:172.21.15.45:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6f4007740 con 0x7fc6fc0d1720 2024-09-12T03:19:20.260 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:20.258+0000 7fc6f3fff640 1 -- 172.21.15.27:0/3156201773 <== mon.2 v2:172.21.15.45:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc6f40078e0 con 0x7fc6fc0d1720 2024-09-12T03:19:20.260 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:20.258+0000 7fc6f3fff640 1 -- 172.21.15.27:0/3156201773 <== mon.2 v2:172.21.15.45:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6f4007cb0 con 0x7fc6fc0d1720 2024-09-12T03:19:20.260 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:20.258+0000 7fc703a4c640 1 -- 172.21.15.27:0/3156201773 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fc6fc0d1720 msgr2=0x7fc6fc0b8190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:20.260 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:20.258+0000 7fc703a4c640 1 --2- 172.21.15.27:0/3156201773 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fc6fc0d1720 0x7fc6fc0b8190 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fc6f4009d90 tx=0x7fc6f402a660 comp rx=0 tx=0).stop 2024-09-12T03:19:20.260 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:20.258+0000 7fc703a4c640 1 -- 172.21.15.27:0/3156201773 shutdown_connections 2024-09-12T03:19:20.260 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:20.258+0000 7fc703a4c640 1 --2- 172.21.15.27:0/3156201773 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fc6fc0d1720 0x7fc6fc0b8190 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:20.260 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:20.258+0000 7fc703a4c640 1 --2- 172.21.15.27:0/3156201773 >> v2:172.21.15.45:3300/0 conn(0x7fc6fc15eb90 0x7fc6fc0d2df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:20.261 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:20.258+0000 7fc703a4c640 1 -- 172.21.15.27:0/3156201773 >> 172.21.15.27:0/3156201773 conn(0x7fc6fc06dce0 msgr2=0x7fc6fc06e0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:20.261 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:20.258+0000 7fc703a4c640 1 -- 172.21.15.27:0/3156201773 shutdown_connections 2024-09-12T03:19:20.261 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:20.258+0000 7fc703a4c640 1 -- 172.21.15.27:0/3156201773 wait complete. 2024-09-12T03:19:20.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:20.259+0000 7fc703a4c640 1 Processor -- start 2024-09-12T03:19:20.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:20.259+0000 7fc703a4c640 1 -- start start 2024-09-12T03:19:20.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:20.260+0000 7fc703a4c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc6fc0d1720 0x7fc6fc0cb0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:20.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:20.260+0000 7fc703a4c640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fc6fc0d5200 0x7fc6fc0ce1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:20.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:20.260+0000 7fc703a4c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc6fc15eb90 0x7fc6fc159510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:20.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:20.260+0000 7fc703a4c640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fc6fc0d3330 con 0x7fc6fc15eb90 2024-09-12T03:19:20.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:20.260+0000 7fc703a4c640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc6fc0d4000 con 0x7fc6fc0d1720 2024-09-12T03:19:20.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:20.260+0000 7fc703a4c640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fc6fc0d34b0 con 0x7fc6fc0d5200 2024-09-12T03:19:20.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:20.260+0000 7fc701fe9640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc6fc15eb90 0x7fc6fc159510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:20.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:20.260+0000 7fc701fe9640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc6fc15eb90 0x7fc6fc159510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42200/0 (socket says 172.21.15.27:42200) 2024-09-12T03:19:20.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:20.260+0000 7fc701fe9640 1 -- 172.21.15.27:0/2767121977 learned_addr learned my addr 172.21.15.27:0/2767121977 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:19:20.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:20.260+0000 7fc7017e8640 1 --2- 172.21.15.27:0/2767121977 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc6fc0d1720 0x7fc6fc0cb0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:20.264 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:20.260+0000 7fc700fe7640 1 --2- 172.21.15.27:0/2767121977 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fc6fc0d5200 0x7fc6fc0ce1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:20.264 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:20.261+0000 7fc701fe9640 1 -- 172.21.15.27:0/2767121977 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fc6fc0d5200 msgr2=0x7fc6fc0ce1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:20.264 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:20.261+0000 7fc701fe9640 1 --2- 172.21.15.27:0/2767121977 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fc6fc0d5200 0x7fc6fc0ce1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:20.264 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:20.261+0000 7fc701fe9640 1 -- 172.21.15.27:0/2767121977 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc6fc0d1720 msgr2=0x7fc6fc0cb0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:20.264 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:20.261+0000 7fc701fe9640 1 --2- 172.21.15.27:0/2767121977 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc6fc0d1720 0x7fc6fc0cb0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:20.264 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:20.261+0000 7fc701fe9640 1 -- 172.21.15.27:0/2767121977 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6fc0cce40 con 0x7fc6fc15eb90 2024-09-12T03:19:20.264 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:20.261+0000 7fc7017e8640 1 --2- 172.21.15.27:0/2767121977 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc6fc0d1720 0x7fc6fc0cb0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:19:20.264 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:20.261+0000 7fc700fe7640 1 --2- 172.21.15.27:0/2767121977 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fc6fc0d5200 0x7fc6fc0ce1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:19:20.264 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:20.261+0000 7fc701fe9640 1 --2- 172.21.15.27:0/2767121977 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc6fc15eb90 0x7fc6fc159510 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fc6f4009d90 tx=0x7fc6f4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:20.264 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:20.261+0000 7fc6f1ffb640 1 -- 172.21.15.27:0/2767121977 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6f4040070 con 0x7fc6fc15eb90 2024-09-12T03:19:20.264 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:20.261+0000 7fc6f1ffb640 1 -- 172.21.15.27:0/2767121977 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc6f4031c40 con 0x7fc6fc15eb90 2024-09-12T03:19:20.264 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:20.261+0000 7fc703a4c640 1 -- 172.21.15.27:0/2767121977 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6fc0cd120 con 0x7fc6fc15eb90 2024-09-12T03:19:20.264 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:20.261+0000 7fc6f1ffb640 1 -- 172.21.15.27:0/2767121977 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6f4009450 con 0x7fc6fc15eb90 2024-09-12T03:19:20.265 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:20.261+0000 7fc703a4c640 1 -- 172.21.15.27:0/2767121977 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6fc0cd6e0 con 0x7fc6fc15eb90 2024-09-12T03:19:20.265 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:20.262+0000 7fc703a4c640 1 -- 172.21.15.27:0/2767121977 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6fc15f090 con 0x7fc6fc15eb90 2024-09-12T03:19:20.265 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:20.263+0000 7fc6f1ffb640 1 -- 172.21.15.27:0/2767121977 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 9) ==== 90651+0+0 (secure 0 0 0) 0x7fc6f4039040 con 0x7fc6fc15eb90 2024-09-12T03:19:20.266 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:20.263+0000 7fc6f1ffb640 1 --2- 172.21.15.27:0/2767121977 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fc6e006bfe0 0x7fc6e006e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:20.266 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:20.263+0000 7fc6f1ffb640 1 -- 172.21.15.27:0/2767121977 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 4943+0+0 (secure 0 0 0) 0x7fc6f40ac070 con 0x7fc6fc15eb90 2024-09-12T03:19:20.266 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:20.264+0000 7fc7017e8640 1 --2- 172.21.15.27:0/2767121977 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fc6e006bfe0 0x7fc6e006e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:20.266 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:20.264+0000 7fc7017e8640 1 --2- 172.21.15.27:0/2767121977 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fc6e006bfe0 0x7fc6e006e4a0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fc6f8009af0 tx=0x7fc6f8008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:20.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:20.268+0000 7fc6f1ffb640 1 -- 172.21.15.27:0/2767121977 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fc6f407a990 con 0x7fc6fc15eb90 2024-09-12T03:19:20.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:20.510+0000 7fc703a4c640 1 -- 172.21.15.27:0/2767121977 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_malicious_metafile_on_legacy_to_v1_upgrade"]} v 0) -- 0x7fc6fc0d00e0 con 0x7fc6fc15eb90 2024-09-12T03:19:20.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:20.524+0000 7fc6f1ffb640 1 -- 172.21.15.27:0/2767121977 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_malicious_metafile_on_legacy_to_v1_upgrade"]}]=0 v155) ==== 164+0+0 (secure 0 0 0) 0x7fc6f407a290 con 0x7fc6fc15eb90 2024-09-12T03:19:20.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:20.526+0000 7fc703a4c640 1 -- 172.21.15.27:0/2767121977 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fc6e006bfe0 msgr2=0x7fc6e006e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:20.528 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:20.526+0000 7fc703a4c640 1 --2- 172.21.15.27:0/2767121977 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fc6e006bfe0 0x7fc6e006e4a0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fc6f8009af0 tx=0x7fc6f8008040 comp rx=0 tx=0).stop 2024-09-12T03:19:20.528 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:20.526+0000 7fc703a4c640 1 -- 172.21.15.27:0/2767121977 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc6fc15eb90 msgr2=0x7fc6fc159510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:20.528 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:20.526+0000 7fc703a4c640 1 --2- 172.21.15.27:0/2767121977 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc6fc15eb90 0x7fc6fc159510 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fc6f4009d90 tx=0x7fc6f4004a40 comp rx=0 tx=0).stop 2024-09-12T03:19:20.528 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:20.526+0000 7fc703a4c640 1 -- 172.21.15.27:0/2767121977 shutdown_connections 2024-09-12T03:19:20.528 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:20.526+0000 7fc703a4c640 1 --2- 172.21.15.27:0/2767121977 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fc6e006bfe0 0x7fc6e006e4a0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:20.528 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:20.526+0000 7fc703a4c640 1 --2- 172.21.15.27:0/2767121977 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc6fc15eb90 0x7fc6fc159510 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:20.528 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:20.526+0000 7fc703a4c640 1 --2- 172.21.15.27:0/2767121977 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fc6fc0d5200 0x7fc6fc0ce1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:20.529 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:20.526+0000 7fc703a4c640 1 --2- 172.21.15.27:0/2767121977 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc6fc0d1720 0x7fc6fc0cb0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:20.529 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:20.526+0000 7fc703a4c640 1 -- 172.21.15.27:0/2767121977 >> 172.21.15.27:0/2767121977 conn(0x7fc6fc06dce0 msgr2=0x7fc6fc0d27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:20.529 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:20.526+0000 7fc703a4c640 1 -- 172.21.15.27:0/2767121977 shutdown_connections 2024-09-12T03:19:20.529 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:20.526+0000 7fc703a4c640 1 -- 172.21.15.27:0/2767121977 wait complete. 2024-09-12T03:19:20.538 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-12T03:19:20.644 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:20.643+0000 7f3dbedcd640 1 Processor -- start 2024-09-12T03:19:20.644 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:20.643+0000 7f3dbedcd640 1 -- start start 2024-09-12T03:19:20.645 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:20.644+0000 7f3dbedcd640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f3db8162040 0x7f3db80d1da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:20.645 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:20.644+0000 7f3dbedcd640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f3db80d3790 0x7f3db80b7a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:20.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:20.644+0000 7f3dbedcd640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f3db806d8c0 0x7f3db806dd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:20.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:20.644+0000 7f3dbedcd640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f3db80b7f90 con 0x7f3db8162040 2024-09-12T03:19:20.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:20.644+0000 7f3dbedcd640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f3db80d2e30 con 0x7f3db806d8c0 2024-09-12T03:19:20.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:20.644+0000 7f3dbedcd640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f3db8162420 con 0x7f3db80d3790 2024-09-12T03:19:20.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:20.644+0000 7f3dbcb69640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f3db8162040 0x7f3db80d1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:20.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:20.644+0000 7f3dbcb69640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f3db8162040 0x7f3db80d1da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42216/0 (socket says 172.21.15.27:42216) 2024-09-12T03:19:20.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:20.644+0000 7f3dbcb69640 1 -- 172.21.15.27:0/2690973868 learned_addr learned my addr 172.21.15.27:0/2690973868 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:19:20.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:20.644+0000 7f3dbcb69640 1 -- 172.21.15.27:0/2690973868 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f3db80d3790 msgr2=0x7f3db80b7a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:20.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:20.644+0000 7f3dbcb69640 1 --2- 172.21.15.27:0/2690973868 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f3db80d3790 0x7f3db80b7a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:20.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:20.644+0000 7f3dbcb69640 1 -- 172.21.15.27:0/2690973868 >> v2:172.21.15.45:3300/0 conn(0x7f3db806d8c0 msgr2=0x7f3db806dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:20.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:20.644+0000 7f3dbcb69640 1 --2- 172.21.15.27:0/2690973868 >> v2:172.21.15.45:3300/0 conn(0x7f3db806d8c0 0x7f3db806dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:20.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:20.644+0000 7f3dbcb69640 1 -- 172.21.15.27:0/2690973868 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3db80d30d0 con 0x7f3db8162040 2024-09-12T03:19:20.651 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:20.645+0000 7f3dbcb69640 1 --2- 172.21.15.27:0/2690973868 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3db8162040 0x7f3db80d1da0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f3db80b7820 tx=0x7f3dac02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40a9a84d822b97d4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:20.651 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:20.645+0000 7f3db77fe640 1 -- 172.21.15.27:0/2690973868 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dac037070 con 0x7f3db8162040 2024-09-12T03:19:20.651 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:20.645+0000 7f3db77fe640 1 -- 172.21.15.27:0/2690973868 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3dac032440 con 0x7f3db8162040 2024-09-12T03:19:20.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:20.645+0000 7f3db77fe640 1 -- 172.21.15.27:0/2690973868 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dac032810 con 0x7f3db8162040 2024-09-12T03:19:20.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:20.646+0000 7f3dbedcd640 1 -- 172.21.15.27:0/2690973868 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3db8162040 msgr2=0x7f3db80d1da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:20.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:20.646+0000 7f3dbedcd640 1 --2- 172.21.15.27:0/2690973868 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3db8162040 0x7f3db80d1da0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f3db80b7820 tx=0x7f3dac02a1e0 comp rx=0 tx=0).stop 2024-09-12T03:19:20.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:20.646+0000 7f3dbedcd640 1 -- 172.21.15.27:0/2690973868 shutdown_connections 2024-09-12T03:19:20.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:20.646+0000 7f3dbedcd640 1 --2- 172.21.15.27:0/2690973868 >> v2:172.21.15.45:3300/0 conn(0x7f3db806d8c0 0x7f3db806dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:20.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:20.646+0000 7f3dbedcd640 1 --2- 172.21.15.27:0/2690973868 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f3db80d3790 0x7f3db80b7a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:20.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:20.646+0000 7f3dbedcd640 1 --2- 172.21.15.27:0/2690973868 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3db8162040 0x7f3db80d1da0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:20.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:20.646+0000 7f3dbedcd640 1 -- 172.21.15.27:0/2690973868 >> 172.21.15.27:0/2690973868 conn(0x7f3db80b6700 msgr2=0x7f3db80b6b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:20.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:20.646+0000 7f3dbedcd640 1 -- 172.21.15.27:0/2690973868 shutdown_connections 2024-09-12T03:19:20.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:20.646+0000 7f3dbedcd640 1 -- 172.21.15.27:0/2690973868 wait complete. 2024-09-12T03:19:20.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:20.647+0000 7f3dbedcd640 1 Processor -- start 2024-09-12T03:19:20.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:20.647+0000 7f3dbedcd640 1 -- start start 2024-09-12T03:19:20.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:20.648+0000 7f3dbedcd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3db806d8c0 0x7f3db814fb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:20.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:20.648+0000 7f3dbedcd640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f3db80d3790 0x7f3db80cc250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:20.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:20.648+0000 7f3dbedcd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3db8162040 0x7f3db80cc790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:20.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:20.648+0000 7f3dbedcd640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f3db80d4530 con 0x7f3db8162040 2024-09-12T03:19:20.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:20.648+0000 7f3dbedcd640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f3db80cee00 con 0x7f3db806d8c0 2024-09-12T03:19:20.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:20.648+0000 7f3dbedcd640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f3db80d4150 con 0x7f3db80d3790 2024-09-12T03:19:20.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:20.648+0000 7f3dbd36a640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3db8162040 0x7f3db80cc790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:20.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:20.648+0000 7f3dbd36a640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3db8162040 0x7f3db80cc790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42220/0 (socket says 172.21.15.27:42220) 2024-09-12T03:19:20.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:20.648+0000 7f3dbd36a640 1 -- 172.21.15.27:0/2789014009 learned_addr learned my addr 172.21.15.27:0/2789014009 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:19:20.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:20.648+0000 7f3db7fff640 1 --2- 172.21.15.27:0/2789014009 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f3db80d3790 0x7f3db80cc250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:20.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:20.648+0000 7f3dbcb69640 1 --2- 172.21.15.27:0/2789014009 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3db806d8c0 0x7f3db814fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:20.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:20.648+0000 7f3dbd36a640 1 -- 172.21.15.27:0/2789014009 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f3db80d3790 msgr2=0x7f3db80cc250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:20.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:20.648+0000 7f3dbd36a640 1 --2- 172.21.15.27:0/2789014009 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f3db80d3790 0x7f3db80cc250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:20.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:20.648+0000 7f3dbd36a640 1 -- 172.21.15.27:0/2789014009 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3db806d8c0 msgr2=0x7f3db814fb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:20.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:20.648+0000 7f3dbd36a640 1 --2- 172.21.15.27:0/2789014009 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3db806d8c0 0x7f3db814fb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:20.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:20.648+0000 7f3dbd36a640 1 -- 172.21.15.27:0/2789014009 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3db80ccd90 con 0x7f3db8162040 2024-09-12T03:19:20.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:20.649+0000 7f3dbd36a640 1 --2- 172.21.15.27:0/2789014009 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3db8162040 0x7f3db80cc790 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f3db000b970 tx=0x7f3db000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:20.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:20.649+0000 7f3db57fa640 1 -- 172.21.15.27:0/2789014009 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3db0019070 con 0x7f3db8162040 2024-09-12T03:19:20.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:20.649+0000 7f3db57fa640 1 -- 172.21.15.27:0/2789014009 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3db000cac0 con 0x7f3db8162040 2024-09-12T03:19:20.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:20.649+0000 7f3dbedcd640 1 -- 172.21.15.27:0/2789014009 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3db80cd070 con 0x7f3db8162040 2024-09-12T03:19:20.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:20.649+0000 7f3db57fa640 1 -- 172.21.15.27:0/2789014009 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3db0014440 con 0x7f3db8162040 2024-09-12T03:19:20.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:20.649+0000 7f3dbedcd640 1 -- 172.21.15.27:0/2789014009 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3db815e930 con 0x7f3db8162040 2024-09-12T03:19:20.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:20.650+0000 7f3dbedcd640 1 -- 172.21.15.27:0/2789014009 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3db80cd240 con 0x7f3db8162040 2024-09-12T03:19:20.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:20.650+0000 7f3db57fa640 1 -- 172.21.15.27:0/2789014009 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 9) ==== 90651+0+0 (secure 0 0 0) 0x7f3db001c070 con 0x7f3db8162040 2024-09-12T03:19:20.657 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:20.651+0000 7f3db57fa640 1 --2- 172.21.15.27:0/2789014009 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f3d9c06bfe0 0x7f3d9c06e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:20.657 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:20.651+0000 7f3db57fa640 1 -- 172.21.15.27:0/2789014009 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 4943+0+0 (secure 0 0 0) 0x7f3db008aa60 con 0x7f3db8162040 2024-09-12T03:19:20.657 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:20.651+0000 7f3dbcb69640 1 --2- 172.21.15.27:0/2789014009 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f3d9c06bfe0 0x7f3d9c06e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:20.657 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:20.652+0000 7f3dbcb69640 1 --2- 172.21.15.27:0/2789014009 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f3d9c06bfe0 0x7f3d9c06e4a0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f3dac034000 tx=0x7f3dac02e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:20.657 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:20.655+0000 7f3db57fa640 1 -- 172.21.15.27:0/2789014009 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f3db0058e40 con 0x7f3db8162040 2024-09-12T03:19:20.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:20.830+0000 7f3dbedcd640 1 -- 172.21.15.27:0/2789014009 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f3db80b6090 con 0x7f3db8162040 2024-09-12T03:19:20.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:20.833+0000 7f3db57fa640 1 -- 172.21.15.27:0/2789014009 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v14)=0 v14) ==== 122+0+0 (secure 0 0 0) 0x7f3db80b6090 con 0x7f3db8162040 2024-09-12T03:19:20.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:20.835+0000 7f3dbedcd640 1 -- 172.21.15.27:0/2789014009 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f3d9c06bfe0 msgr2=0x7f3d9c06e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:20.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:20.835+0000 7f3dbedcd640 1 --2- 172.21.15.27:0/2789014009 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f3d9c06bfe0 0x7f3d9c06e4a0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f3dac034000 tx=0x7f3dac02e000 comp rx=0 tx=0).stop 2024-09-12T03:19:20.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:20.835+0000 7f3dbedcd640 1 -- 172.21.15.27:0/2789014009 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3db8162040 msgr2=0x7f3db80cc790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:20.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:20.835+0000 7f3dbedcd640 1 --2- 172.21.15.27:0/2789014009 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3db8162040 0x7f3db80cc790 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f3db000b970 tx=0x7f3db000bd70 comp rx=0 tx=0).stop 2024-09-12T03:19:20.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:20.835+0000 7f3dbedcd640 1 -- 172.21.15.27:0/2789014009 shutdown_connections 2024-09-12T03:19:20.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:20.835+0000 7f3dbedcd640 1 --2- 172.21.15.27:0/2789014009 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f3d9c06bfe0 0x7f3d9c06e4a0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:20.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:20.835+0000 7f3dbedcd640 1 --2- 172.21.15.27:0/2789014009 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3db8162040 0x7f3db80cc790 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:20.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:20.835+0000 7f3dbedcd640 1 --2- 172.21.15.27:0/2789014009 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f3db80d3790 0x7f3db80cc250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:20.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:20.835+0000 7f3dbedcd640 1 --2- 172.21.15.27:0/2789014009 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3db806d8c0 0x7f3db814fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:20.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:20.835+0000 7f3dbedcd640 1 -- 172.21.15.27:0/2789014009 >> 172.21.15.27:0/2789014009 conn(0x7f3db80b6700 msgr2=0x7f3db80b5980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:20.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:20.835+0000 7f3dbedcd640 1 -- 172.21.15.27:0/2789014009 shutdown_connections 2024-09-12T03:19:20.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:20.835+0000 7f3dbedcd640 1 -- 172.21.15.27:0/2789014009 wait complete. 2024-09-12T03:19:20.847 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-12T03:19:20.847 DEBUG:teuthology.orchestra.run.smithi027:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-12T03:19:20.892 DEBUG:teuthology.orchestra.run.smithi045:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-12T03:19:20.937 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-12T03:19:20.938 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-12T03:19:20.938 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T03:19:20.954 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056004,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497432k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16324 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248716k,nr_inodes=812179,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497432k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-12T03:19:20.954 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-12T03:19:20.954 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T03:19:20.973 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056795,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16602 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249348k,nr_inodes=812337,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-12T03:19:20.973 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-12T03:19:20.973 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T03:19:21.010 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056004,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497432k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16324 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248716k,nr_inodes=812179,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497432k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-12T03:19:21.010 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-12T03:19:21.010 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T03:19:21.031 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056795,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16602 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249348k,nr_inodes=812337,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-12T03:19:21.031 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-12T03:19:21.031 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-12T03:19:21.152 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:21.151+0000 7f5f1e0af640 1 Processor -- start 2024-09-12T03:19:21.152 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:21.151+0000 7f5f1e0af640 1 -- start start 2024-09-12T03:19:21.154 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:21.152+0000 7f5f1e0af640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f5f180cda40 0x7f5f1815ca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:21.154 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:21.152+0000 7f5f1e0af640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f5f18146010 0x7f5f180b4040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:21.154 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:21.152+0000 7f5f1e0af640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5f180b4d70 con 0x7f5f18145c30 2024-09-12T03:19:21.154 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:21.152+0000 7f5f1e0af640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f5f180b5050 con 0x7f5f180cda40 2024-09-12T03:19:21.154 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:21.152+0000 7f5f1e0af640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f5f1806d140 con 0x7f5f18146010 2024-09-12T03:19:21.154 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:21.152+0000 7f5f16ffd640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f5f180cda40 0x7f5f1815ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:21.154 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:21.152+0000 7f5f17fff640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f5f18146010 0x7f5f180b4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:21.154 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:21.152+0000 7f5f177fe640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f5f18145c30 0x7f5f180cd330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.27:52006/0 (socket says 172.21.15.27:52006) 2024-09-12T03:19:21.154 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:21.152+0000 7f5f177fe640 1 -- 172.21.15.27:0/1871525823 learned_addr learned my addr 172.21.15.27:0/1871525823 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:19:21.154 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:21.152+0000 7f5f167fc640 1 -- 172.21.15.27:0/1871525823 <== mon.1 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3736942834 0 0) 0x7f5f180b4d70 con 0x7f5f18145c30 2024-09-12T03:19:21.154 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:21.152+0000 7f5f167fc640 1 -- 172.21.15.27:0/1871525823 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5f04003520 con 0x7f5f18145c30 2024-09-12T03:19:21.155 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:21.153+0000 7f5f16ffd640 1 -- 172.21.15.27:0/1871525823 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f5f18146010 msgr2=0x7f5f180b4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:21.155 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:21.153+0000 7f5f16ffd640 1 --2- 172.21.15.27:0/1871525823 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f5f18146010 0x7f5f180b4040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:21.155 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:21.153+0000 7f5f16ffd640 1 -- 172.21.15.27:0/1871525823 >> v1:172.21.15.45:6789/0 conn(0x7f5f18145c30 legacy=0x7f5f180cd330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:21.155 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:21.153+0000 7f5f16ffd640 1 -- 172.21.15.27:0/1871525823 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f1806d420 con 0x7f5f180cda40 2024-09-12T03:19:21.155 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:21.153+0000 7f5f17fff640 1 --2- 172.21.15.27:0/1871525823 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f5f18146010 0x7f5f180b4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:19:21.156 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:21.153+0000 7f5f16ffd640 1 --2- 172.21.15.27:0/1871525823 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5f180cda40 0x7f5f1815ca00 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f5f000097a0 tx=0x7f5f0002a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=116f7c5516d9c31b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:21.156 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:21.154+0000 7f5f167fc640 1 -- 172.21.15.27:0/1871525823 <== mon.1 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f00037070 con 0x7f5f180cda40 2024-09-12T03:19:21.156 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:21.154+0000 7f5f167fc640 1 -- 172.21.15.27:0/1871525823 <== mon.1 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f0002ae50 con 0x7f5f180cda40 2024-09-12T03:19:21.156 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:21.154+0000 7f5f167fc640 1 -- 172.21.15.27:0/1871525823 <== mon.1 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f00032640 con 0x7f5f180cda40 2024-09-12T03:19:21.157 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:21.154+0000 7f5f1e0af640 1 -- 172.21.15.27:0/1871525823 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5f180cda40 msgr2=0x7f5f1815ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:21.157 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:21.154+0000 7f5f1e0af640 1 --2- 172.21.15.27:0/1871525823 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5f180cda40 0x7f5f1815ca00 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f5f000097a0 tx=0x7f5f0002a100 comp rx=0 tx=0).stop 2024-09-12T03:19:21.157 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:21.154+0000 7f5f1e0af640 1 -- 172.21.15.27:0/1871525823 shutdown_connections 2024-09-12T03:19:21.157 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:21.154+0000 7f5f1e0af640 1 --2- 172.21.15.27:0/1871525823 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f5f18146010 0x7f5f180b4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:21.157 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:21.154+0000 7f5f1e0af640 1 --2- 172.21.15.27:0/1871525823 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5f180cda40 0x7f5f1815ca00 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:21.157 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:21.154+0000 7f5f1e0af640 1 -- 172.21.15.27:0/1871525823 >> 172.21.15.27:0/1871525823 conn(0x7f5f18062c60 msgr2=0x7f5f1806bb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:21.157 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:21.154+0000 7f5f1e0af640 1 -- 172.21.15.27:0/1871525823 shutdown_connections 2024-09-12T03:19:21.157 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:21.154+0000 7f5f1e0af640 1 -- 172.21.15.27:0/1871525823 wait complete. 2024-09-12T03:19:21.157 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:21.155+0000 7f5f1e0af640 1 Processor -- start 2024-09-12T03:19:21.157 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:21.156+0000 7f5f1e0af640 1 -- start start 2024-09-12T03:19:21.159 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:21.156+0000 7f5f1e0af640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5f180cda40 0x7f5f181f2770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:21.159 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:21.156+0000 7f5f1e0af640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5f18145c30 0x7f5f18141300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:21.160 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:21.156+0000 7f5f1e0af640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f5f18146010 0x7f5f181f1190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:21.160 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:21.156+0000 7f5f1e0af640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f5f180b4700 con 0x7f5f180cda40 2024-09-12T03:19:21.160 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:21.156+0000 7f5f1e0af640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f5f180b4580 con 0x7f5f18145c30 2024-09-12T03:19:21.160 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:21.156+0000 7f5f1e0af640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f5f180b4880 con 0x7f5f18146010 2024-09-12T03:19:21.160 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:21.156+0000 7f5f177fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5f180cda40 0x7f5f181f2770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:21.160 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:21.156+0000 7f5f177fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5f180cda40 0x7f5f181f2770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42234/0 (socket says 172.21.15.27:42234) 2024-09-12T03:19:21.160 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:21.156+0000 7f5f177fe640 1 -- 172.21.15.27:0/418764628 learned_addr learned my addr 172.21.15.27:0/418764628 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:19:21.160 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:21.157+0000 7f5f16ffd640 1 --2- 172.21.15.27:0/418764628 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5f18145c30 0x7f5f18141300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:21.160 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:21.157+0000 7f5f17fff640 1 --2- 172.21.15.27:0/418764628 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f5f18146010 0x7f5f181f1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:21.160 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:21.157+0000 7f5f177fe640 1 -- 172.21.15.27:0/418764628 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f5f18146010 msgr2=0x7f5f181f1190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:21.160 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:21.157+0000 7f5f177fe640 1 --2- 172.21.15.27:0/418764628 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f5f18146010 0x7f5f181f1190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:21.160 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:21.157+0000 7f5f177fe640 1 -- 172.21.15.27:0/418764628 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5f18145c30 msgr2=0x7f5f18141300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:21.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:21.157+0000 7f5f177fe640 1 --2- 172.21.15.27:0/418764628 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5f18145c30 0x7f5f18141300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:21.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:21.157+0000 7f5f177fe640 1 -- 172.21.15.27:0/418764628 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f181f5510 con 0x7f5f180cda40 2024-09-12T03:19:21.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:21.157+0000 7f5f177fe640 1 --2- 172.21.15.27:0/418764628 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5f180cda40 0x7f5f181f2770 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f5f0800c500 tx=0x7f5f0800c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:21.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:21.157+0000 7f5f0ffff640 1 -- 172.21.15.27:0/418764628 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f08017070 con 0x7f5f180cda40 2024-09-12T03:19:21.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:21.157+0000 7f5f0ffff640 1 -- 172.21.15.27:0/418764628 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f08005ce0 con 0x7f5f180cda40 2024-09-12T03:19:21.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:21.157+0000 7f5f1e0af640 1 -- 172.21.15.27:0/418764628 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f181f57f0 con 0x7f5f180cda40 2024-09-12T03:19:21.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:21.157+0000 7f5f0ffff640 1 -- 172.21.15.27:0/418764628 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f080072b0 con 0x7f5f180cda40 2024-09-12T03:19:21.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:21.157+0000 7f5f1e0af640 1 -- 172.21.15.27:0/418764628 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f181f5db0 con 0x7f5f180cda40 2024-09-12T03:19:21.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:21.158+0000 7f5f1e0af640 1 -- 172.21.15.27:0/418764628 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f1815ca00 con 0x7f5f180cda40 2024-09-12T03:19:21.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:21.159+0000 7f5f0ffff640 1 -- 172.21.15.27:0/418764628 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 9) ==== 90651+0+0 (secure 0 0 0) 0x7f5f0801a070 con 0x7f5f180cda40 2024-09-12T03:19:21.162 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:21.159+0000 7f5f0ffff640 1 --2- 172.21.15.27:0/418764628 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f5ef806bfe0 0x7f5ef806e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:21.162 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:21.159+0000 7f5f0ffff640 1 -- 172.21.15.27:0/418764628 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 4943+0+0 (secure 0 0 0) 0x7f5f0808a180 con 0x7f5f180cda40 2024-09-12T03:19:21.162 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:21.159+0000 7f5f16ffd640 1 --2- 172.21.15.27:0/418764628 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f5ef806bfe0 0x7f5ef806e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:21.162 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:21.160+0000 7f5f16ffd640 1 --2- 172.21.15.27:0/418764628 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f5ef806bfe0 0x7f5ef806e4a0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f5f00034000 tx=0x7f5f0002e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:21.164 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:21.163+0000 7f5f0ffff640 1 -- 172.21.15.27:0/418764628 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f5f080585a0 con 0x7f5f180cda40 2024-09-12T03:19:21.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:21.372+0000 7f5f1e0af640 1 -- 172.21.15.27:0/418764628 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5f1806e4e0 con 0x7f5f180cda40 2024-09-12T03:19:21.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:21.373+0000 7f5f0ffff640 1 -- 172.21.15.27:0/418764628 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+3911 (secure 0 0 0) 0x7f5f0805c8a0 con 0x7f5f180cda40 2024-09-12T03:19:21.374 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:19:21.374 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":25,"btime":"2024-09-12T03:19:16:508053+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4835,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6834/1457797481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":1457797481},{"type":"v1","addr":"172.21.15.27:6834","nonce":1457797481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5029,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6836/2842491281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":2842491281},{"type":"v1","addr":"172.21.15.45:6836","nonce":2842491281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5032,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/3339744340","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6835","nonce":3339744340},{"type":"v1","addr":"172.21.15.45:6837","nonce":3339744340}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5568,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/4253571220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":4253571220},{"type":"v1","addr":"172.21.15.27:6835","nonce":4253571220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":25}],"filesystems":[]} 2024-09-12T03:19:21.375 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 25 2024-09-12T03:19:21.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:21.375+0000 7f5f1e0af640 1 -- 172.21.15.27:0/418764628 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f5ef806bfe0 msgr2=0x7f5ef806e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:21.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:21.375+0000 7f5f1e0af640 1 --2- 172.21.15.27:0/418764628 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f5ef806bfe0 0x7f5ef806e4a0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f5f00034000 tx=0x7f5f0002e000 comp rx=0 tx=0).stop 2024-09-12T03:19:21.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:21.375+0000 7f5f1e0af640 1 -- 172.21.15.27:0/418764628 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5f180cda40 msgr2=0x7f5f181f2770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:21.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:21.375+0000 7f5f1e0af640 1 --2- 172.21.15.27:0/418764628 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5f180cda40 0x7f5f181f2770 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f5f0800c500 tx=0x7f5f0800c900 comp rx=0 tx=0).stop 2024-09-12T03:19:21.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:21.375+0000 7f5f1e0af640 1 -- 172.21.15.27:0/418764628 shutdown_connections 2024-09-12T03:19:21.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:21.375+0000 7f5f1e0af640 1 --2- 172.21.15.27:0/418764628 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f5ef806bfe0 0x7f5ef806e4a0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:21.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:21.375+0000 7f5f1e0af640 1 --2- 172.21.15.27:0/418764628 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f5f18146010 0x7f5f181f1190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:21.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:21.375+0000 7f5f1e0af640 1 --2- 172.21.15.27:0/418764628 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5f18145c30 0x7f5f18141300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:21.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:21.375+0000 7f5f1e0af640 1 --2- 172.21.15.27:0/418764628 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5f180cda40 0x7f5f181f2770 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:21.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:21.375+0000 7f5f1e0af640 1 -- 172.21.15.27:0/418764628 >> 172.21.15.27:0/418764628 conn(0x7f5f18062c60 msgr2=0x7f5f1806bb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:21.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:21.375+0000 7f5f1e0af640 1 -- 172.21.15.27:0/418764628 shutdown_connections 2024-09-12T03:19:21.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:21.375+0000 7f5f1e0af640 1 -- 172.21.15.27:0/418764628 wait complete. 2024-09-12T03:19:21.387 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-12T03:19:21.387 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-12T03:19:21.388 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-12T03:19:21.388 INFO:teuthology.orchestra.run:waiting for 300 2024-09-12T03:19:21.388 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-12T03:19:21.388 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-12T03:19:21.389 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-12T03:19:21.389 INFO:teuthology.orchestra.run:waiting for 300 2024-09-12T03:19:21.389 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-12T03:19:21.389 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-12T03:19:21.389 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-12T03:19:21.389 INFO:teuthology.orchestra.run:waiting for 300 2024-09-12T03:19:21.390 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-12T03:19:21.390 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-12T03:19:21.390 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-12T03:19:21.390 INFO:teuthology.orchestra.run:waiting for 300 2024-09-12T03:19:21.397 INFO:tasks.ceph.mds.a:Stopped 2024-09-12T03:19:21.398 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-12T03:19:21.399 INFO:tasks.ceph.mds.c:Stopped 2024-09-12T03:19:21.399 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-12T03:19:21.401 INFO:tasks.ceph.mds.b:Stopped 2024-09-12T03:19:21.401 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-12T03:19:21.402 INFO:tasks.ceph.mds.d:Stopped 2024-09-12T03:19:21.402 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-12T03:19:21.440 INFO:tasks.ceph.mds.a:Started 2024-09-12T03:19:21.441 DEBUG:teuthology.parallel:result is None 2024-09-12T03:19:21.441 INFO:tasks.ceph.mds.c:Started 2024-09-12T03:19:21.441 DEBUG:teuthology.parallel:result is None 2024-09-12T03:19:21.444 INFO:tasks.ceph.mds.b:Started 2024-09-12T03:19:21.444 DEBUG:teuthology.parallel:result is None 2024-09-12T03:19:21.445 INFO:tasks.ceph.mds.d:Started 2024-09-12T03:19:21.445 DEBUG:teuthology.parallel:result is None 2024-09-12T03:19:21.445 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-12T03:19:21.519 INFO:tasks.ceph.mds.a.smithi027.stdout:starting mds.a at 2024-09-12T03:19:21.520 INFO:tasks.ceph.mds.c.smithi027.stdout:starting mds.c at 2024-09-12T03:19:21.531 INFO:tasks.ceph.mds.b.smithi045.stdout:starting mds.b at 2024-09-12T03:19:21.531 INFO:tasks.ceph.mds.d.smithi045.stdout:starting mds.d at 2024-09-12T03:19:21.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:21.555+0000 7f0d91fb7640 1 Processor -- start 2024-09-12T03:19:21.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:21.555+0000 7f0d91fb7640 1 -- start start 2024-09-12T03:19:21.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:21.555+0000 7f0d91fb7640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f0d8c15eb90 0x7f0d8c0d2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:21.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:21.556+0000 7f0d91fb7640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0d8c0d3760 con 0x7f0d8c0d1720 2024-09-12T03:19:21.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:21.556+0000 7f0d91fb7640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0d8c0d3a40 con 0x7f0d8c0d5200 2024-09-12T03:19:21.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:21.556+0000 7f0d91fb7640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f0d8c0b7d20 con 0x7f0d8c15eb90 2024-09-12T03:19:21.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:21.556+0000 7f0d8bfff640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f0d8c0d1720 0x7f0d8c0b8190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:49092/0 (socket says 172.21.15.27:49092) 2024-09-12T03:19:21.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:21.556+0000 7f0d8bfff640 1 -- 172.21.15.27:0/3276503792 learned_addr learned my addr 172.21.15.27:0/3276503792 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:19:21.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:21.556+0000 7f0d8b7fe640 1 --2- 172.21.15.27:0/3276503792 >> v2:172.21.15.45:3300/0 conn(0x7f0d8c15eb90 0x7f0d8c0d2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:21.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:21.556+0000 7f0d8a7fc640 1 -- 172.21.15.27:0/3276503792 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1866187690 0 0) 0x7f0d8c0d3760 con 0x7f0d8c0d1720 2024-09-12T03:19:21.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:21.556+0000 7f0d8a7fc640 1 -- 172.21.15.27:0/3276503792 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0d78003520 con 0x7f0d8c0d1720 2024-09-12T03:19:21.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:21.556+0000 7f0d8a7fc640 1 -- 172.21.15.27:0/3276503792 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1419093802 0 0) 0x7f0d78003520 con 0x7f0d8c0d1720 2024-09-12T03:19:21.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:21.556+0000 7f0d8a7fc640 1 -- 172.21.15.27:0/3276503792 >> v2:172.21.15.45:3300/0 conn(0x7f0d8c15eb90 msgr2=0x7f0d8c0d2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:21.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:21.556+0000 7f0d8a7fc640 1 --2- 172.21.15.27:0/3276503792 >> v2:172.21.15.45:3300/0 conn(0x7f0d8c15eb90 0x7f0d8c0d2df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:21.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:21.556+0000 7f0d8a7fc640 1 -- 172.21.15.27:0/3276503792 >> v1:172.21.15.45:6789/0 conn(0x7f0d8c0d5200 legacy=0x7f0d8c0d1010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:21.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:21.556+0000 7f0d8a7fc640 1 -- 172.21.15.27:0/3276503792 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d8c0d3d20 con 0x7f0d8c0d1720 2024-09-12T03:19:21.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:21.556+0000 7f0d8a7fc640 1 -- 172.21.15.27:0/3276503792 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 847755213 0 0) 0x7f0d84001670 con 0x7f0d8c0d1720 2024-09-12T03:19:21.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:21.556+0000 7f0d8a7fc640 1 -- 172.21.15.27:0/3276503792 >> v1:172.21.15.27:6789/0 conn(0x7f0d8c0d1720 legacy=0x7f0d8c0b8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:21.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:21.556+0000 7f0d8a7fc640 1 --2- 172.21.15.27:0/3276503792 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0d780039b0 0x7f0d78003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:21.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:21.556+0000 7f0d8a7fc640 1 --2- 172.21.15.27:0/3276503792 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0d78004800 0x7f0d78004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:21.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:21.556+0000 7f0d8a7fc640 1 --2- 172.21.15.27:0/3276503792 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f0d780051c0 0x7f0d7800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:21.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:21.556+0000 7f0d8a7fc640 1 -- 172.21.15.27:0/3276503792 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f0d7800c030 con 0x7f0d78004800 2024-09-12T03:19:21.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:21.556+0000 7f0d8a7fc640 1 -- 172.21.15.27:0/3276503792 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f0d7800c1e0 con 0x7f0d780039b0 2024-09-12T03:19:21.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:21.556+0000 7f0d8a7fc640 1 -- 172.21.15.27:0/3276503792 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f0d7800c3e0 con 0x7f0d780051c0 2024-09-12T03:19:21.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:21.556+0000 7f0d8b7fe640 1 --2- 172.21.15.27:0/3276503792 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0d780039b0 0x7f0d78003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:21.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:21.556+0000 7f0d8affd640 1 --2- 172.21.15.27:0/3276503792 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0d78004800 0x7f0d78004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:21.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:21.557+0000 7f0d8bfff640 1 --2- 172.21.15.27:0/3276503792 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f0d780051c0 0x7f0d7800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:21.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:21.557+0000 7f0d8b7fe640 1 -- 172.21.15.27:0/3276503792 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f0d780051c0 msgr2=0x7f0d7800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:21.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:21.557+0000 7f0d8b7fe640 1 --2- 172.21.15.27:0/3276503792 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f0d780051c0 0x7f0d7800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:21.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:21.557+0000 7f0d8b7fe640 1 -- 172.21.15.27:0/3276503792 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0d78004800 msgr2=0x7f0d78004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:21.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:21.557+0000 7f0d8b7fe640 1 --2- 172.21.15.27:0/3276503792 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0d78004800 0x7f0d78004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:21.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:21.557+0000 7f0d8b7fe640 1 -- 172.21.15.27:0/3276503792 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d78003520 con 0x7f0d780039b0 2024-09-12T03:19:21.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:21.557+0000 7f0d8bfff640 1 -- 172.21.15.27:0/3276503792 reap_dead start 2024-09-12T03:19:21.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:21.558+0000 7f0d8b7fe640 1 --2- 172.21.15.27:0/3276503792 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0d780039b0 0x7f0d78003dd0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f0d7c009fc0 tx=0x7f0d78005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=91566529b8bfdb1e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:21.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:21.558+0000 7f0d8a7fc640 1 -- 172.21.15.27:0/3276503792 <== mon.1 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d7c034020 con 0x7f0d780039b0 2024-09-12T03:19:21.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:21.558+0000 7f0d8a7fc640 1 -- 172.21.15.27:0/3276503792 <== mon.1 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d7c0305f0 con 0x7f0d780039b0 2024-09-12T03:19:21.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:21.558+0000 7f0d8a7fc640 1 -- 172.21.15.27:0/3276503792 <== mon.1 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d7c009c10 con 0x7f0d780039b0 2024-09-12T03:19:21.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:21.558+0000 7f0d91fb7640 1 -- 172.21.15.27:0/3276503792 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0d780039b0 msgr2=0x7f0d78003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:21.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:21.558+0000 7f0d91fb7640 1 --2- 172.21.15.27:0/3276503792 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0d780039b0 0x7f0d78003dd0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f0d7c009fc0 tx=0x7f0d78005830 comp rx=0 tx=0).stop 2024-09-12T03:19:21.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:21.558+0000 7f0d91fb7640 1 -- 172.21.15.27:0/3276503792 shutdown_connections 2024-09-12T03:19:21.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:21.558+0000 7f0d91fb7640 1 --2- 172.21.15.27:0/3276503792 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0d780039b0 0x7f0d78003dd0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:21.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:21.558+0000 7f0d91fb7640 1 -- 172.21.15.27:0/3276503792 >> 172.21.15.27:0/3276503792 conn(0x7f0d8c06dce0 msgr2=0x7f0d8c06e0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:21.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:21.558+0000 7f0d91fb7640 1 -- 172.21.15.27:0/3276503792 shutdown_connections 2024-09-12T03:19:21.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:21.558+0000 7f0d91fb7640 1 -- 172.21.15.27:0/3276503792 wait complete. 2024-09-12T03:19:21.567 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:21.559+0000 7f0d91fb7640 1 Processor -- start 2024-09-12T03:19:21.567 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:21.560+0000 7f0d91fb7640 1 -- start start 2024-09-12T03:19:21.567 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:21.560+0000 7f0d91fb7640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0d780039b0 0x7f0d8c1f4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:21.567 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:21.560+0000 7f0d91fb7640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0d8c1f5130 0x7f0d8c0b8b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:21.567 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:21.560+0000 7f0d91fb7640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f0d8c0b90a0 0x7f0d8c0c9e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:21.567 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:21.560+0000 7f0d91fb7640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f0d8c15ef70 con 0x7f0d8c1f5130 2024-09-12T03:19:21.567 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:21.560+0000 7f0d91fb7640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f0d8c0d42e0 con 0x7f0d780039b0 2024-09-12T03:19:21.567 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:21.560+0000 7f0d91fb7640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f0d8c15f0f0 con 0x7f0d8c0b90a0 2024-09-12T03:19:21.567 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:21.560+0000 7f0d8affd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0d8c1f5130 0x7f0d8c0b8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:21.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:21.561+0000 7f0d8affd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0d8c1f5130 0x7f0d8c0b8b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42294/0 (socket says 172.21.15.27:42294) 2024-09-12T03:19:21.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:21.561+0000 7f0d8affd640 1 -- 172.21.15.27:0/821553278 learned_addr learned my addr 172.21.15.27:0/821553278 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:19:21.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:21.561+0000 7f0d8bfff640 1 --2- 172.21.15.27:0/821553278 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f0d8c0b90a0 0x7f0d8c0c9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:21.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:21.561+0000 7f0d8b7fe640 1 --2- 172.21.15.27:0/821553278 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0d780039b0 0x7f0d8c1f4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:21.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:21.561+0000 7f0d8affd640 1 -- 172.21.15.27:0/821553278 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f0d8c0b90a0 msgr2=0x7f0d8c0c9e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:21.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:21.561+0000 7f0d8affd640 1 --2- 172.21.15.27:0/821553278 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f0d8c0b90a0 0x7f0d8c0c9e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:21.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:21.561+0000 7f0d8affd640 1 -- 172.21.15.27:0/821553278 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0d780039b0 msgr2=0x7f0d8c1f4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:21.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:21.561+0000 7f0d8affd640 1 --2- 172.21.15.27:0/821553278 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0d780039b0 0x7f0d8c1f4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:21.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:21.561+0000 7f0d8affd640 1 -- 172.21.15.27:0/821553278 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d8c0c8340 con 0x7f0d8c1f5130 2024-09-12T03:19:21.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:21.561+0000 7f0d8b7fe640 1 --2- 172.21.15.27:0/821553278 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0d780039b0 0x7f0d8c1f4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:19:21.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:21.561+0000 7f0d8bfff640 1 --2- 172.21.15.27:0/821553278 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f0d8c0b90a0 0x7f0d8c0c9e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:19:21.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:21.561+0000 7f0d8affd640 1 --2- 172.21.15.27:0/821553278 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0d8c1f5130 0x7f0d8c0b8b60 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f0d7400f590 tx=0x7f0d7400fa50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:21.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:21.561+0000 7f0d83fff640 1 -- 172.21.15.27:0/821553278 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d740052e0 con 0x7f0d8c1f5130 2024-09-12T03:19:21.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:21.561+0000 7f0d83fff640 1 -- 172.21.15.27:0/821553278 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d74018070 con 0x7f0d8c1f5130 2024-09-12T03:19:21.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:21.561+0000 7f0d91fb7640 1 -- 172.21.15.27:0/821553278 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d8c0c8600 con 0x7f0d8c1f5130 2024-09-12T03:19:21.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:21.561+0000 7f0d83fff640 1 -- 172.21.15.27:0/821553278 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d74003830 con 0x7f0d8c1f5130 2024-09-12T03:19:21.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:21.561+0000 7f0d91fb7640 1 -- 172.21.15.27:0/821553278 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d8c0c8bc0 con 0x7f0d8c1f5130 2024-09-12T03:19:21.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:21.562+0000 7f0d91fb7640 1 -- 172.21.15.27:0/821553278 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d8c0d10a0 con 0x7f0d8c1f5130 2024-09-12T03:19:21.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:21.563+0000 7f0d83fff640 1 -- 172.21.15.27:0/821553278 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 9) ==== 90651+0+0 (secure 0 0 0) 0x7f0d740039d0 con 0x7f0d8c1f5130 2024-09-12T03:19:21.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:21.563+0000 7f0d83fff640 1 --2- 172.21.15.27:0/821553278 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f0d6c06c010 0x7f0d6c06e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:21.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:21.563+0000 7f0d83fff640 1 -- 172.21.15.27:0/821553278 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 4943+0+0 (secure 0 0 0) 0x7f0d7408aad0 con 0x7f0d8c1f5130 2024-09-12T03:19:21.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:21.563+0000 7f0d8b7fe640 1 --2- 172.21.15.27:0/821553278 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f0d6c06c010 0x7f0d6c06e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:21.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:21.564+0000 7f0d8b7fe640 1 --2- 172.21.15.27:0/821553278 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f0d6c06c010 0x7f0d6c06e4d0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f0d7c035000 tx=0x7f0d7c02f000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:21.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:21.567+0000 7f0d83fff640 1 -- 172.21.15.27:0/821553278 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f0d74059590 con 0x7f0d8c1f5130 2024-09-12T03:19:21.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:21.754+0000 7f0d91fb7640 1 -- 172.21.15.27:0/821553278 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f0d8c161bd0 con 0x7f0d8c1f5130 2024-09-12T03:19:22.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:22.530+0000 7f0d83fff640 1 -- 172.21.15.27:0/821553278 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v51) ==== 97+0+0 (secure 0 0 0) 0x7f0d74058e90 con 0x7f0d8c1f5130 2024-09-12T03:19:22.532 INFO:teuthology.orchestra.run.smithi027.stderr: removed all blocklist entries 2024-09-12T03:19:22.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:22.534+0000 7f0d91fb7640 1 -- 172.21.15.27:0/821553278 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f0d6c06c010 msgr2=0x7f0d6c06e4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:22.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:22.534+0000 7f0d91fb7640 1 --2- 172.21.15.27:0/821553278 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f0d6c06c010 0x7f0d6c06e4d0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f0d7c035000 tx=0x7f0d7c02f000 comp rx=0 tx=0).stop 2024-09-12T03:19:22.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:22.534+0000 7f0d91fb7640 1 -- 172.21.15.27:0/821553278 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0d8c1f5130 msgr2=0x7f0d8c0b8b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:22.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:22.534+0000 7f0d91fb7640 1 --2- 172.21.15.27:0/821553278 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0d8c1f5130 0x7f0d8c0b8b60 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f0d7400f590 tx=0x7f0d7400fa50 comp rx=0 tx=0).stop 2024-09-12T03:19:22.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:22.534+0000 7f0d91fb7640 1 -- 172.21.15.27:0/821553278 shutdown_connections 2024-09-12T03:19:22.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:22.534+0000 7f0d91fb7640 1 --2- 172.21.15.27:0/821553278 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f0d6c06c010 0x7f0d6c06e4d0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:22.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:22.534+0000 7f0d91fb7640 1 --2- 172.21.15.27:0/821553278 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f0d8c0b90a0 0x7f0d8c0c9e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:22.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:22.534+0000 7f0d91fb7640 1 --2- 172.21.15.27:0/821553278 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0d8c1f5130 0x7f0d8c0b8b60 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:22.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:22.534+0000 7f0d91fb7640 1 --2- 172.21.15.27:0/821553278 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0d780039b0 0x7f0d8c1f4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:22.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:22.534+0000 7f0d91fb7640 1 -- 172.21.15.27:0/821553278 >> 172.21.15.27:0/821553278 conn(0x7f0d8c06dce0 msgr2=0x7f0d8c0ce170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:22.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:22.534+0000 7f0d91fb7640 1 -- 172.21.15.27:0/821553278 shutdown_connections 2024-09-12T03:19:22.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:22.534+0000 7f0d91fb7640 1 -- 172.21.15.27:0/821553278 wait complete. 2024-09-12T03:19:22.548 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-12T03:19:22.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:22.651+0000 7f1ce072c640 1 Processor -- start 2024-09-12T03:19:22.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:22.651+0000 7f1ce072c640 1 -- start start 2024-09-12T03:19:22.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:22.652+0000 7f1ce072c640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1cd815eb60 0x7f1cd80d1280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:22.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:22.652+0000 7f1ce072c640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f1cd80d17c0 0x7f1cd80b3390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:22.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:22.652+0000 7f1ce072c640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1cd80b6690 con 0x7f1cd815e780 2024-09-12T03:19:22.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:22.652+0000 7f1ce072c640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f1cd80d1c30 con 0x7f1cd80d17c0 2024-09-12T03:19:22.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:22.652+0000 7f1ce072c640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f1cd80b7950 con 0x7f1cd815eb60 2024-09-12T03:19:22.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:22.652+0000 7f1cdecc9640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f1cd80d17c0 0x7f1cd80b3390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:22.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:22.652+0000 7f1cdecc9640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f1cd80d17c0 0x7f1cd80b3390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42296/0 (socket says 172.21.15.27:42296) 2024-09-12T03:19:22.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:22.652+0000 7f1cdecc9640 1 -- 172.21.15.27:0/4014770881 learned_addr learned my addr 172.21.15.27:0/4014770881 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:19:22.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:22.652+0000 7f1cde4c8640 1 --1- 172.21.15.27:0/4014770881 >> v1:172.21.15.27:6789/0 conn(0x7f1cd815e780 0x7f1cd80d3e70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:49106/0 (socket says 172.21.15.27:49106) 2024-09-12T03:19:22.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:22.652+0000 7f1cddcc7640 1 --2- 172.21.15.27:0/4014770881 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1cd815eb60 0x7f1cd80d1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:22.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:22.652+0000 7f1cdecc9640 1 -- 172.21.15.27:0/4014770881 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1cd815eb60 msgr2=0x7f1cd80d1280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:22.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:22.652+0000 7f1cdecc9640 1 --2- 172.21.15.27:0/4014770881 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1cd815eb60 0x7f1cd80d1280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:22.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:22.652+0000 7f1cdecc9640 1 -- 172.21.15.27:0/4014770881 >> v1:172.21.15.27:6789/0 conn(0x7f1cd815e780 legacy=0x7f1cd80d3e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:22.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:22.652+0000 7f1cdd4c6640 1 -- 172.21.15.27:0/4014770881 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2647399911 0 0) 0x7f1cd80b6690 con 0x7f1cd815e780 2024-09-12T03:19:22.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:22.652+0000 7f1cdecc9640 1 -- 172.21.15.27:0/4014770881 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cd80b6970 con 0x7f1cd80d17c0 2024-09-12T03:19:22.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:22.652+0000 7f1cddcc7640 1 --2- 172.21.15.27:0/4014770881 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1cd815eb60 0x7f1cd80d1280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:19:22.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:22.653+0000 7f1cdecc9640 1 --2- 172.21.15.27:0/4014770881 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1cd80d17c0 0x7f1cd80b3390 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f1cd0009e50 tx=0x7f1cd002a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcc216c5e1c7c9e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:22.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:22.653+0000 7f1cdd4c6640 1 -- 172.21.15.27:0/4014770881 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cd0036070 con 0x7f1cd80d17c0 2024-09-12T03:19:22.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:22.653+0000 7f1cdd4c6640 1 -- 172.21.15.27:0/4014770881 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1cd0031600 con 0x7f1cd80d17c0 2024-09-12T03:19:22.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:22.653+0000 7f1cdd4c6640 1 -- 172.21.15.27:0/4014770881 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cd00319d0 con 0x7f1cd80d17c0 2024-09-12T03:19:22.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:22.654+0000 7f1ce072c640 1 -- 172.21.15.27:0/4014770881 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1cd80d17c0 msgr2=0x7f1cd80b3390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:22.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:22.654+0000 7f1ce072c640 1 --2- 172.21.15.27:0/4014770881 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1cd80d17c0 0x7f1cd80b3390 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f1cd0009e50 tx=0x7f1cd002a710 comp rx=0 tx=0).stop 2024-09-12T03:19:22.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:22.654+0000 7f1ce072c640 1 -- 172.21.15.27:0/4014770881 shutdown_connections 2024-09-12T03:19:22.657 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:22.654+0000 7f1ce072c640 1 --2- 172.21.15.27:0/4014770881 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1cd80d17c0 0x7f1cd80b3390 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:22.657 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:22.654+0000 7f1ce072c640 1 --2- 172.21.15.27:0/4014770881 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1cd815eb60 0x7f1cd80d1280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:22.657 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:22.654+0000 7f1ce072c640 1 -- 172.21.15.27:0/4014770881 >> 172.21.15.27:0/4014770881 conn(0x7f1cd806da60 msgr2=0x7f1cd806de70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:22.657 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:22.654+0000 7f1ce072c640 1 -- 172.21.15.27:0/4014770881 shutdown_connections 2024-09-12T03:19:22.657 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:22.654+0000 7f1ce072c640 1 -- 172.21.15.27:0/4014770881 wait complete. 2024-09-12T03:19:22.657 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:22.655+0000 7f1ce072c640 1 Processor -- start 2024-09-12T03:19:22.657 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:22.655+0000 7f1ce072c640 1 -- start start 2024-09-12T03:19:22.659 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:22.656+0000 7f1ce072c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1cd80d17c0 0x7f1cd80cb000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:22.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:22.656+0000 7f1ce072c640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1cd815e780 0x7f1cd814bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:22.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:22.656+0000 7f1ce072c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1cd815eb60 0x7f1cd815b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:22.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:22.656+0000 7f1ce072c640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f1cd80b6f30 con 0x7f1cd815eb60 2024-09-12T03:19:22.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:22.656+0000 7f1ce072c640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f1cd80b7210 con 0x7f1cd80d17c0 2024-09-12T03:19:22.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:22.656+0000 7f1ce072c640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f1cd80b6340 con 0x7f1cd815e780 2024-09-12T03:19:22.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:22.656+0000 7f1cdecc9640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1cd815eb60 0x7f1cd815b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:22.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:22.656+0000 7f1cdecc9640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1cd815eb60 0x7f1cd815b1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42310/0 (socket says 172.21.15.27:42310) 2024-09-12T03:19:22.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:22.656+0000 7f1cdecc9640 1 -- 172.21.15.27:0/1649801253 learned_addr learned my addr 172.21.15.27:0/1649801253 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:19:22.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:22.656+0000 7f1cde4c8640 1 --2- 172.21.15.27:0/1649801253 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1cd80d17c0 0x7f1cd80cb000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:22.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:22.656+0000 7f1cddcc7640 1 --2- 172.21.15.27:0/1649801253 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1cd815e780 0x7f1cd814bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:22.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:22.656+0000 7f1cdecc9640 1 -- 172.21.15.27:0/1649801253 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1cd815e780 msgr2=0x7f1cd814bc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:22.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:22.656+0000 7f1cdecc9640 1 --2- 172.21.15.27:0/1649801253 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1cd815e780 0x7f1cd814bc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:22.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:22.656+0000 7f1cdecc9640 1 -- 172.21.15.27:0/1649801253 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1cd80d17c0 msgr2=0x7f1cd80cb000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:22.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:22.656+0000 7f1cdecc9640 1 --2- 172.21.15.27:0/1649801253 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1cd80d17c0 0x7f1cd80cb000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:22.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:22.656+0000 7f1cdecc9640 1 -- 172.21.15.27:0/1649801253 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cd815cbe0 con 0x7f1cd815eb60 2024-09-12T03:19:22.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:22.656+0000 7f1cddcc7640 1 --2- 172.21.15.27:0/1649801253 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1cd815e780 0x7f1cd814bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:19:22.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:22.656+0000 7f1cdecc9640 1 --2- 172.21.15.27:0/1649801253 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1cd815eb60 0x7f1cd815b1a0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f1cd0034040 tx=0x7f1cd0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:22.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:22.657+0000 7f1cceffd640 1 -- 172.21.15.27:0/1649801253 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cd0036050 con 0x7f1cd815eb60 2024-09-12T03:19:22.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:22.657+0000 7f1cceffd640 1 -- 172.21.15.27:0/1649801253 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1cd0042070 con 0x7f1cd815eb60 2024-09-12T03:19:22.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:22.657+0000 7f1cceffd640 1 -- 172.21.15.27:0/1649801253 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cd0009510 con 0x7f1cd815eb60 2024-09-12T03:19:22.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:22.657+0000 7f1ce072c640 1 -- 172.21.15.27:0/1649801253 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1cd815a3e0 con 0x7f1cd815eb60 2024-09-12T03:19:22.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:22.657+0000 7f1ce072c640 1 -- 172.21.15.27:0/1649801253 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1cd815a9a0 con 0x7f1cd815eb60 2024-09-12T03:19:22.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:22.658+0000 7f1ce072c640 1 -- 172.21.15.27:0/1649801253 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1cd80d1280 con 0x7f1cd815eb60 2024-09-12T03:19:22.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:22.658+0000 7f1cceffd640 1 -- 172.21.15.27:0/1649801253 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 9) ==== 90651+0+0 (secure 0 0 0) 0x7f1cd0007bd0 con 0x7f1cd815eb60 2024-09-12T03:19:22.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:22.659+0000 7f1cceffd640 1 --2- 172.21.15.27:0/1649801253 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f1cbc06bfe0 0x7f1cbc06e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:22.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:22.659+0000 7f1cceffd640 1 -- 172.21.15.27:0/1649801253 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 4857+0+0 (secure 0 0 0) 0x7f1cd00abcc0 con 0x7f1cd815eb60 2024-09-12T03:19:22.662 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:22.659+0000 7f1cde4c8640 1 --2- 172.21.15.27:0/1649801253 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f1cbc06bfe0 0x7f1cbc06e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:22.662 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:22.659+0000 7f1cde4c8640 1 --2- 172.21.15.27:0/1649801253 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f1cbc06bfe0 0x7f1cbc06e4a0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f1cd4000f60 tx=0x7f1cd4007d50 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:22.664 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:22.663+0000 7f1cceffd640 1 -- 172.21.15.27:0/1649801253 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f1cd007a6c0 con 0x7f1cd815eb60 2024-09-12T03:19:22.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:22.902+0000 7f1ce072c640 1 -- 172.21.15.27:0/1649801253 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f1cd80c8190 con 0x7f1cd815eb60 2024-09-12T03:19:22.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:22.902+0000 7f1cceffd640 1 -- 172.21.15.27:0/1649801253 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v12) ==== 80+0+5304 (secure 0 0 0) 0x7f1cd0079fc0 con 0x7f1cd815eb60 2024-09-12T03:19:22.904 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:19:22.904 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-12T03:19:22.904 INFO:teuthology.orchestra.run.smithi027.stdout: "auth_dump": [ 2024-09-12T03:19:22.904 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:19:22.904 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "mds.a", 2024-09-12T03:19:22.904 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDUXOJmIhxjNxAAaGg8SwtW6NRe/lIgNCV20w==", 2024-09-12T03:19:22.904 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:19:22.904 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow", 2024-09-12T03:19:22.904 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow *", 2024-09-12T03:19:22.904 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow *", 2024-09-12T03:19:22.905 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2024-09-12T03:19:22.905 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:19:22.905 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:19:22.905 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:19:22.905 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "mds.b", 2024-09-12T03:19:22.905 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDVXOJm63kWCBAAzjfOsBsWUI+IAsuZ49yEkw==", 2024-09-12T03:19:22.905 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:19:22.905 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow", 2024-09-12T03:19:22.905 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow *", 2024-09-12T03:19:22.905 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow *", 2024-09-12T03:19:22.905 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2024-09-12T03:19:22.905 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:19:22.906 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:19:22.906 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:19:22.906 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "mds.c", 2024-09-12T03:19:22.906 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDVXOJm0sX7AhAAdynCcho2a/AgFF9Cw3+vAw==", 2024-09-12T03:19:22.906 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:19:22.906 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow", 2024-09-12T03:19:22.906 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow *", 2024-09-12T03:19:22.906 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow *", 2024-09-12T03:19:22.906 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2024-09-12T03:19:22.906 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:19:22.907 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:19:22.907 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:19:22.907 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "mds.d", 2024-09-12T03:19:22.907 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDVXOJm7Dd8DxAANHYmvKgpRZ6sVfXDSgPHWw==", 2024-09-12T03:19:22.907 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:19:22.907 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow", 2024-09-12T03:19:22.907 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow *", 2024-09-12T03:19:22.907 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow *", 2024-09-12T03:19:22.907 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2024-09-12T03:19:22.907 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:19:22.907 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:19:22.907 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:19:22.907 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "osd.0", 2024-09-12T03:19:22.907 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDYXOJmJTVYLxAAmo8kxRjaiwTT44dZLZlq4A==", 2024-09-12T03:19:22.908 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:19:22.908 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow profile osd", 2024-09-12T03:19:22.908 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile osd", 2024-09-12T03:19:22.908 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2024-09-12T03:19:22.908 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:19:22.908 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:19:22.908 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:19:22.908 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "osd.1", 2024-09-12T03:19:22.908 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDcXOJmYV0BFBAAmoGX9gf9yLoE/qkWFnv9tg==", 2024-09-12T03:19:22.908 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:19:22.908 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow profile osd", 2024-09-12T03:19:22.909 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile osd", 2024-09-12T03:19:22.909 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2024-09-12T03:19:22.909 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:19:22.909 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:19:22.909 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:19:22.909 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "osd.2", 2024-09-12T03:19:22.909 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDfXOJmEetiJRAAXk+AvfTfhjsB6T2UB4Z2/w==", 2024-09-12T03:19:22.909 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:19:22.909 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow profile osd", 2024-09-12T03:19:22.909 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile osd", 2024-09-12T03:19:22.909 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2024-09-12T03:19:22.909 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:19:22.909 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:19:22.910 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:19:22.910 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "osd.3", 2024-09-12T03:19:22.910 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDiXOJmrQbPJxAAQsi2B5a1LnPTPGZVdo7j7w==", 2024-09-12T03:19:22.910 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:19:22.910 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow profile osd", 2024-09-12T03:19:22.910 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile osd", 2024-09-12T03:19:22.910 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2024-09-12T03:19:22.910 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:19:22.910 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:19:22.910 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:19:22.910 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "osd.4", 2024-09-12T03:19:22.911 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDnXOJmoMr1NxAAv3u2q8hYt9w//9gz+N7UAQ==", 2024-09-12T03:19:22.911 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:19:22.911 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow profile osd", 2024-09-12T03:19:22.911 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile osd", 2024-09-12T03:19:22.911 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2024-09-12T03:19:22.911 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:19:22.911 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:19:22.911 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:19:22.911 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "osd.5", 2024-09-12T03:19:22.912 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDqXOJmxpUPOxAAxLH6RJsyLOZ2XMfzM0fQ1g==", 2024-09-12T03:19:22.912 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:19:22.912 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow profile osd", 2024-09-12T03:19:22.912 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile osd", 2024-09-12T03:19:22.912 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2024-09-12T03:19:22.912 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:19:22.912 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:19:22.912 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:19:22.912 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "osd.6", 2024-09-12T03:19:22.912 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDuXOJmuP0+LxAAMHoz5eSO81hPtkdBIS2cig==", 2024-09-12T03:19:22.913 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:19:22.913 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow profile osd", 2024-09-12T03:19:22.913 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile osd", 2024-09-12T03:19:22.913 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2024-09-12T03:19:22.913 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:19:22.913 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:19:22.913 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:19:22.913 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "osd.7", 2024-09-12T03:19:22.913 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDxXOJmuIC/MRAAPDM+lAHqdoQyGVODwjFp+Q==", 2024-09-12T03:19:22.913 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:19:22.914 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow profile osd", 2024-09-12T03:19:22.914 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile osd", 2024-09-12T03:19:22.914 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2024-09-12T03:19:22.914 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:19:22.914 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:19:22.914 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:19:22.914 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "client.0", 2024-09-12T03:19:22.914 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDVXOJmLm+5ExAAECM4CKfuVElmp/n0b02mXg==", 2024-09-12T03:19:22.914 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:19:22.914 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow", 2024-09-12T03:19:22.914 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow r", 2024-09-12T03:19:22.914 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-12T03:19:22.914 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:19:22.915 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:19:22.915 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:19:22.915 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "client.1", 2024-09-12T03:19:22.915 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDVXOJm61/FHBAALCQouXySfQeZ5DESGmqkzA==", 2024-09-12T03:19:22.915 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:19:22.915 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow", 2024-09-12T03:19:22.915 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow r", 2024-09-12T03:19:22.915 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow rw", 2024-09-12T03:19:22.915 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow rwx" 2024-09-12T03:19:22.915 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:19:22.915 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:19:22.915 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:19:22.915 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "client.2", 2024-09-12T03:19:22.916 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDVXOJmq/ZkGRAAOzmQzEIhb4b/YTxayc34Bw==", 2024-09-12T03:19:22.916 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:19:22.916 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow", 2024-09-12T03:19:22.916 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow r", 2024-09-12T03:19:22.916 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow rw", 2024-09-12T03:19:22.916 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow rwx" 2024-09-12T03:19:22.917 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:19:22.917 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:19:22.917 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:19:22.917 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "client.3", 2024-09-12T03:19:22.917 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDVXOJm+DlzIhAATQKZe+Zyk0MM8efTa6g9PA==", 2024-09-12T03:19:22.917 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:19:22.917 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow", 2024-09-12T03:19:22.917 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow r", 2024-09-12T03:19:22.917 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow rw", 2024-09-12T03:19:22.917 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow rwx" 2024-09-12T03:19:22.917 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:19:22.918 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:19:22.918 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:19:22.918 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "client.admin", 2024-09-12T03:19:22.918 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDUXOJm+JGWGhAApTQw9RtJWd/J5QzQaUFXvg==", 2024-09-12T03:19:22.918 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:19:22.918 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow *", 2024-09-12T03:19:22.918 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow *", 2024-09-12T03:19:22.918 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow *", 2024-09-12T03:19:22.918 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2024-09-12T03:19:22.918 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:19:22.918 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:19:22.918 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:19:22.918 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "mgr.x", 2024-09-12T03:19:22.918 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDUXOJmZcsONBAAMTY2KykuHaEkv1t6Vz5KjA==", 2024-09-12T03:19:22.918 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:19:22.919 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow *", 2024-09-12T03:19:22.919 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile mgr", 2024-09-12T03:19:22.919 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2024-09-12T03:19:22.919 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:19:22.919 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:19:22.919 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:19:22.919 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "mgr.y", 2024-09-12T03:19:22.919 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDUXOJmJOgZMBAAQ/JTxfhox7HgxBMtsvUCBg==", 2024-09-12T03:19:22.919 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:19:22.919 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow *", 2024-09-12T03:19:22.919 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile mgr", 2024-09-12T03:19:22.920 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2024-09-12T03:19:22.920 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:19:22.920 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:19:22.920 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-12T03:19:22.920 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-12T03:19:22.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:22.905+0000 7f1ce072c640 1 -- 172.21.15.27:0/1649801253 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f1cbc06bfe0 msgr2=0x7f1cbc06e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:22.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:22.905+0000 7f1ce072c640 1 --2- 172.21.15.27:0/1649801253 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f1cbc06bfe0 0x7f1cbc06e4a0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f1cd4000f60 tx=0x7f1cd4007d50 comp rx=0 tx=0).stop 2024-09-12T03:19:22.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:22.905+0000 7f1ce072c640 1 -- 172.21.15.27:0/1649801253 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1cd815eb60 msgr2=0x7f1cd815b1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:22.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:22.905+0000 7f1ce072c640 1 --2- 172.21.15.27:0/1649801253 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1cd815eb60 0x7f1cd815b1a0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f1cd0034040 tx=0x7f1cd0004a40 comp rx=0 tx=0).stop 2024-09-12T03:19:22.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:22.905+0000 7f1ce072c640 1 -- 172.21.15.27:0/1649801253 shutdown_connections 2024-09-12T03:19:22.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:22.905+0000 7f1ce072c640 1 --2- 172.21.15.27:0/1649801253 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f1cbc06bfe0 0x7f1cbc06e4a0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:22.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:22.905+0000 7f1ce072c640 1 --2- 172.21.15.27:0/1649801253 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1cd815eb60 0x7f1cd815b1a0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:22.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:22.905+0000 7f1ce072c640 1 --2- 172.21.15.27:0/1649801253 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1cd815e780 0x7f1cd814bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:22.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:22.905+0000 7f1ce072c640 1 --2- 172.21.15.27:0/1649801253 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1cd80d17c0 0x7f1cd80cb000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:22.923 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:22.905+0000 7f1ce072c640 1 -- 172.21.15.27:0/1649801253 >> 172.21.15.27:0/1649801253 conn(0x7f1cd806da60 msgr2=0x7f1cd806de40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:22.923 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:22.905+0000 7f1ce072c640 1 -- 172.21.15.27:0/1649801253 shutdown_connections 2024-09-12T03:19:22.923 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:22.905+0000 7f1ce072c640 1 -- 172.21.15.27:0/1649801253 wait complete. 2024-09-12T03:19:22.923 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-12T03:19:23.026 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.025+0000 7fc9f23e4640 1 Processor -- start 2024-09-12T03:19:23.026 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.025+0000 7fc9f23e4640 1 -- start start 2024-09-12T03:19:23.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.026+0000 7fc9f23e4640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fc9ec0ae8d0 0x7fc9ec0728b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:23.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.026+0000 7fc9f23e4640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc9ec072df0 0x7fc9ec06e930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:23.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.026+0000 7fc9f23e4640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc9ec0aee90 con 0x7fc9ec0b5a70 2024-09-12T03:19:23.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.026+0000 7fc9f23e4640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7fc9ec073240 con 0x7fc9ec0ae8d0 2024-09-12T03:19:23.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.026+0000 7fc9f23e4640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fc9ec0af170 con 0x7fc9ec072df0 2024-09-12T03:19:23.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.026+0000 7fc9eb7fe640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fc9ec0ae8d0 0x7fc9ec0728b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:23.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.026+0000 7fc9eb7fe640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fc9ec0ae8d0 0x7fc9ec0728b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42322/0 (socket says 172.21.15.27:42322) 2024-09-12T03:19:23.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.026+0000 7fc9eb7fe640 1 -- 172.21.15.27:0/287663986 learned_addr learned my addr 172.21.15.27:0/287663986 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:19:23.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.026+0000 7fc9f0981640 1 --2- 172.21.15.27:0/287663986 >> v2:172.21.15.45:3300/0 conn(0x7fc9ec072df0 0x7fc9ec06e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:23.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.026+0000 7fc9eb7fe640 1 -- 172.21.15.27:0/287663986 >> v2:172.21.15.45:3300/0 conn(0x7fc9ec072df0 msgr2=0x7fc9ec06e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:23.029 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.026+0000 7fc9eb7fe640 1 --2- 172.21.15.27:0/287663986 >> v2:172.21.15.45:3300/0 conn(0x7fc9ec072df0 0x7fc9ec06e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:23.029 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.026+0000 7fc9eb7fe640 1 -- 172.21.15.27:0/287663986 >> v1:172.21.15.45:6789/0 conn(0x7fc9ec0b5a70 legacy=0x7fc9ec15f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:23.029 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.026+0000 7fc9eb7fe640 1 -- 172.21.15.27:0/287663986 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9ec0af450 con 0x7fc9ec0ae8d0 2024-09-12T03:19:23.029 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.026+0000 7fc9f0981640 1 --2- 172.21.15.27:0/287663986 >> v2:172.21.15.45:3300/0 conn(0x7fc9ec072df0 0x7fc9ec06e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:19:23.029 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.027+0000 7fc9eb7fe640 1 --2- 172.21.15.27:0/287663986 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc9ec0ae8d0 0x7fc9ec0728b0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fc9ec073ac0 tx=0x7fc9e002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5b0bcc475b1e5dc server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:23.029 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.027+0000 7fc9eaffd640 1 -- 172.21.15.27:0/287663986 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9e0037070 con 0x7fc9ec0ae8d0 2024-09-12T03:19:23.029 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.027+0000 7fc9eaffd640 1 -- 172.21.15.27:0/287663986 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc9e0032440 con 0x7fc9ec0ae8d0 2024-09-12T03:19:23.029 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.027+0000 7fc9eaffd640 1 -- 172.21.15.27:0/287663986 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9e0032810 con 0x7fc9ec0ae8d0 2024-09-12T03:19:23.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.027+0000 7fc9f23e4640 1 -- 172.21.15.27:0/287663986 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc9ec0ae8d0 msgr2=0x7fc9ec0728b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:23.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.027+0000 7fc9f23e4640 1 --2- 172.21.15.27:0/287663986 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc9ec0ae8d0 0x7fc9ec0728b0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fc9ec073ac0 tx=0x7fc9e002a1a0 comp rx=0 tx=0).stop 2024-09-12T03:19:23.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.028+0000 7fc9f23e4640 1 -- 172.21.15.27:0/287663986 shutdown_connections 2024-09-12T03:19:23.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.028+0000 7fc9f23e4640 1 --2- 172.21.15.27:0/287663986 >> v2:172.21.15.45:3300/0 conn(0x7fc9ec072df0 0x7fc9ec06e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:23.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.028+0000 7fc9f23e4640 1 --2- 172.21.15.27:0/287663986 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc9ec0ae8d0 0x7fc9ec0728b0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:23.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.028+0000 7fc9f23e4640 1 -- 172.21.15.27:0/287663986 >> 172.21.15.27:0/287663986 conn(0x7fc9ec06ca30 msgr2=0x7fc9ec06ce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:23.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.028+0000 7fc9f23e4640 1 -- 172.21.15.27:0/287663986 shutdown_connections 2024-09-12T03:19:23.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.028+0000 7fc9f23e4640 1 -- 172.21.15.27:0/287663986 wait complete. 2024-09-12T03:19:23.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.029+0000 7fc9f23e4640 1 Processor -- start 2024-09-12T03:19:23.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.029+0000 7fc9f23e4640 1 -- start start 2024-09-12T03:19:23.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.030+0000 7fc9f23e4640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fc9ec072df0 0x7fc9ec1f8190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:23.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.030+0000 7fc9f23e4640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc9ec0ae8d0 0x7fc9ec14da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:23.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.030+0000 7fc9f23e4640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc9ec0b5a70 0x7fc9ec0b15d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:23.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.030+0000 7fc9f23e4640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fc9ec06eff0 con 0x7fc9ec0ae8d0 2024-09-12T03:19:23.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.030+0000 7fc9f23e4640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc9ec06ee70 con 0x7fc9ec0b5a70 2024-09-12T03:19:23.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.030+0000 7fc9f23e4640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fc9ec06f170 con 0x7fc9ec072df0 2024-09-12T03:19:23.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.030+0000 7fc9eb7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc9ec0ae8d0 0x7fc9ec14da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:23.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.030+0000 7fc9eb7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc9ec0ae8d0 0x7fc9ec14da00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42332/0 (socket says 172.21.15.27:42332) 2024-09-12T03:19:23.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.030+0000 7fc9eb7fe640 1 -- 172.21.15.27:0/1270077003 learned_addr learned my addr 172.21.15.27:0/1270077003 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:19:23.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.030+0000 7fc9f0981640 1 --2- 172.21.15.27:0/1270077003 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc9ec0b5a70 0x7fc9ec0b15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:23.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.030+0000 7fc9ebfff640 1 --2- 172.21.15.27:0/1270077003 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fc9ec072df0 0x7fc9ec1f8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:23.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.030+0000 7fc9eb7fe640 1 -- 172.21.15.27:0/1270077003 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fc9ec072df0 msgr2=0x7fc9ec1f8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:23.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.030+0000 7fc9eb7fe640 1 --2- 172.21.15.27:0/1270077003 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fc9ec072df0 0x7fc9ec1f8190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:23.034 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.030+0000 7fc9eb7fe640 1 -- 172.21.15.27:0/1270077003 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc9ec0b5a70 msgr2=0x7fc9ec0b15d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:23.034 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.030+0000 7fc9eb7fe640 1 --2- 172.21.15.27:0/1270077003 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc9ec0b5a70 0x7fc9ec0b15d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:23.034 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.030+0000 7fc9eb7fe640 1 -- 172.21.15.27:0/1270077003 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9ec1f86d0 con 0x7fc9ec0ae8d0 2024-09-12T03:19:23.034 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.030+0000 7fc9ebfff640 1 --2- 172.21.15.27:0/1270077003 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fc9ec072df0 0x7fc9ec1f8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:19:23.034 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.030+0000 7fc9eb7fe640 1 --2- 172.21.15.27:0/1270077003 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc9ec0ae8d0 0x7fc9ec14da00 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fc9e0000c00 tx=0x7fc9e00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:23.034 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.030+0000 7fc9e8ff9640 1 -- 172.21.15.27:0/1270077003 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9e0041020 con 0x7fc9ec0ae8d0 2024-09-12T03:19:23.034 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.030+0000 7fc9e8ff9640 1 -- 172.21.15.27:0/1270077003 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc9e0030970 con 0x7fc9ec0ae8d0 2024-09-12T03:19:23.034 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.031+0000 7fc9f23e4640 1 -- 172.21.15.27:0/1270077003 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9ec159700 con 0x7fc9ec0ae8d0 2024-09-12T03:19:23.034 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.031+0000 7fc9e8ff9640 1 -- 172.21.15.27:0/1270077003 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9e0037040 con 0x7fc9ec0ae8d0 2024-09-12T03:19:23.034 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.031+0000 7fc9f23e4640 1 -- 172.21.15.27:0/1270077003 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9ec159cc0 con 0x7fc9ec0ae8d0 2024-09-12T03:19:23.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.031+0000 7fc9f23e4640 1 -- 172.21.15.27:0/1270077003 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9ec06e910 con 0x7fc9ec0ae8d0 2024-09-12T03:19:23.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.032+0000 7fc9e8ff9640 1 -- 172.21.15.27:0/1270077003 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 9) ==== 90651+0+0 (secure 0 0 0) 0x7fc9e0044020 con 0x7fc9ec0ae8d0 2024-09-12T03:19:23.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.033+0000 7fc9e8ff9640 1 --2- 172.21.15.27:0/1270077003 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fc9d806bfe0 0x7fc9d806e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:23.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.033+0000 7fc9e8ff9640 1 -- 172.21.15.27:0/1270077003 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 4857+0+0 (secure 0 0 0) 0x7fc9e00ab510 con 0x7fc9ec0ae8d0 2024-09-12T03:19:23.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.033+0000 7fc9ebfff640 1 --2- 172.21.15.27:0/1270077003 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fc9d806bfe0 0x7fc9d806e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:23.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.033+0000 7fc9ebfff640 1 --2- 172.21.15.27:0/1270077003 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fc9d806bfe0 0x7fc9d806e4a0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fc9dc006fd0 tx=0x7fc9dc008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:23.037 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.037+0000 7fc9e8ff9640 1 -- 172.21.15.27:0/1270077003 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fc9e004b070 con 0x7fc9ec0ae8d0 2024-09-12T03:19:23.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.215+0000 7fc9f23e4640 1 -- 172.21.15.27:0/1270077003 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fc9ec160ae0 con 0x7fc9ec0ae8d0 2024-09-12T03:19:23.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.216+0000 7fc9e8ff9640 1 -- 172.21.15.27:0/1270077003 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v52) ==== 84+0+65 (secure 0 0 0) 0x7fc9e00ab020 con 0x7fc9ec0ae8d0 2024-09-12T03:19:23.217 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:19:23.217 INFO:teuthology.orchestra.run.smithi027.stdout:[ 2024-09-12T03:19:23.217 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:19:23.217 INFO:teuthology.orchestra.run.smithi027.stdout: "poolnum": 1, 2024-09-12T03:19:23.217 INFO:teuthology.orchestra.run.smithi027.stdout: "poolname": ".mgr" 2024-09-12T03:19:23.217 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:19:23.217 INFO:teuthology.orchestra.run.smithi027.stdout:] 2024-09-12T03:19:23.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.218+0000 7fc9f23e4640 1 -- 172.21.15.27:0/1270077003 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fc9d806bfe0 msgr2=0x7fc9d806e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:23.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.218+0000 7fc9f23e4640 1 --2- 172.21.15.27:0/1270077003 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fc9d806bfe0 0x7fc9d806e4a0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fc9dc006fd0 tx=0x7fc9dc008040 comp rx=0 tx=0).stop 2024-09-12T03:19:23.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.218+0000 7fc9f23e4640 1 -- 172.21.15.27:0/1270077003 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc9ec0ae8d0 msgr2=0x7fc9ec14da00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:23.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.218+0000 7fc9f23e4640 1 --2- 172.21.15.27:0/1270077003 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc9ec0ae8d0 0x7fc9ec14da00 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fc9e0000c00 tx=0x7fc9e00046c0 comp rx=0 tx=0).stop 2024-09-12T03:19:23.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.218+0000 7fc9f23e4640 1 -- 172.21.15.27:0/1270077003 shutdown_connections 2024-09-12T03:19:23.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.218+0000 7fc9f23e4640 1 --2- 172.21.15.27:0/1270077003 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fc9d806bfe0 0x7fc9d806e4a0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:23.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.218+0000 7fc9f23e4640 1 --2- 172.21.15.27:0/1270077003 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc9ec0b5a70 0x7fc9ec0b15d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:23.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.218+0000 7fc9f23e4640 1 --2- 172.21.15.27:0/1270077003 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc9ec0ae8d0 0x7fc9ec14da00 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:23.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.218+0000 7fc9f23e4640 1 --2- 172.21.15.27:0/1270077003 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fc9ec072df0 0x7fc9ec1f8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:23.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.218+0000 7fc9f23e4640 1 -- 172.21.15.27:0/1270077003 >> 172.21.15.27:0/1270077003 conn(0x7fc9ec06ca30 msgr2=0x7fc9ec1613a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:23.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.218+0000 7fc9f23e4640 1 -- 172.21.15.27:0/1270077003 shutdown_connections 2024-09-12T03:19:23.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.218+0000 7fc9f23e4640 1 -- 172.21.15.27:0/1270077003 wait complete. 2024-09-12T03:19:23.230 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-12T03:19:23.230 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-12T03:19:23.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.334+0000 7f19f9981640 1 Processor -- start 2024-09-12T03:19:23.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.335+0000 7f19f9981640 1 -- start start 2024-09-12T03:19:23.337 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.335+0000 7f19f9981640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f19f40cef40 0x7f19f40b4000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:23.337 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.335+0000 7f19f9981640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f19f406c650 0x7f19f40710c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:23.337 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.335+0000 7f19f9981640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f19f4071600 con 0x7f19f40cf320 2024-09-12T03:19:23.337 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.335+0000 7f19f9981640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f19f406dee0 con 0x7f19f406c650 2024-09-12T03:19:23.337 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.335+0000 7f19f9981640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f19f406cff0 con 0x7f19f40cef40 2024-09-12T03:19:23.337 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.336+0000 7f19f2ffd640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f19f40cef40 0x7f19f40b4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:23.337 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.336+0000 7f19f27fc640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f19f406c650 0x7f19f40710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:23.338 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.336+0000 7f19f27fc640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f19f406c650 0x7f19f40710c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.27:37792/0 (socket says 172.21.15.27:37792) 2024-09-12T03:19:23.338 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.336+0000 7f19f2ffd640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f19f40cef40 0x7f19f40b4000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3301/0 says I am v2:172.21.15.27:34572/0 (socket says 172.21.15.27:34572) 2024-09-12T03:19:23.338 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.336+0000 7f19f37fe640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f19f40cf320 0x7f19f406e370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.27:52050/0 (socket says 172.21.15.27:52050) 2024-09-12T03:19:23.338 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.336+0000 7f19f2ffd640 1 -- 172.21.15.27:0/543976040 learned_addr learned my addr 172.21.15.27:0/543976040 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:19:23.338 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.336+0000 7f19f27fc640 1 -- 172.21.15.27:0/543976040 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f19f40cef40 msgr2=0x7f19f40b4000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:23.338 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.336+0000 7f19f27fc640 1 --2- 172.21.15.27:0/543976040 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f19f40cef40 0x7f19f40b4000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:23.338 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.336+0000 7f19f27fc640 1 -- 172.21.15.27:0/543976040 >> v1:172.21.15.45:6789/0 conn(0x7f19f40cf320 legacy=0x7f19f406e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:23.338 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.336+0000 7f19f27fc640 1 -- 172.21.15.27:0/543976040 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19f406d2d0 con 0x7f19f406c650 2024-09-12T03:19:23.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.337+0000 7f19f27fc640 1 --2- 172.21.15.27:0/543976040 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f19f406c650 0x7f19f40710c0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f19e80097a0 tx=0x7f19e802a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c1ab9e93cdc4a848 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:23.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.337+0000 7f19f1ffb640 1 -- 172.21.15.27:0/543976040 <== mon.1 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19e8037070 con 0x7f19f406c650 2024-09-12T03:19:23.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.337+0000 7f19f1ffb640 1 -- 172.21.15.27:0/543976040 <== mon.1 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19e802ae50 con 0x7f19f406c650 2024-09-12T03:19:23.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.337+0000 7f19f1ffb640 1 -- 172.21.15.27:0/543976040 <== mon.1 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19e8032660 con 0x7f19f406c650 2024-09-12T03:19:23.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.338+0000 7f19f9981640 1 -- 172.21.15.27:0/543976040 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f19f406c650 msgr2=0x7f19f40710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:23.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.338+0000 7f19f9981640 1 --2- 172.21.15.27:0/543976040 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f19f406c650 0x7f19f40710c0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f19e80097a0 tx=0x7f19e802a100 comp rx=0 tx=0).stop 2024-09-12T03:19:23.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.338+0000 7f19f9981640 1 -- 172.21.15.27:0/543976040 shutdown_connections 2024-09-12T03:19:23.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.338+0000 7f19f9981640 1 --2- 172.21.15.27:0/543976040 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f19f406c650 0x7f19f40710c0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:23.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.338+0000 7f19f9981640 1 --2- 172.21.15.27:0/543976040 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f19f40cef40 0x7f19f40b4000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:23.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.338+0000 7f19f9981640 1 -- 172.21.15.27:0/543976040 >> 172.21.15.27:0/543976040 conn(0x7f19f406bb30 msgr2=0x7f19f406bf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:23.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.338+0000 7f19f9981640 1 -- 172.21.15.27:0/543976040 shutdown_connections 2024-09-12T03:19:23.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.338+0000 7f19f9981640 1 -- 172.21.15.27:0/543976040 wait complete. 2024-09-12T03:19:23.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.339+0000 7f19f9981640 1 Processor -- start 2024-09-12T03:19:23.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.339+0000 7f19f9981640 1 -- start start 2024-09-12T03:19:23.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.340+0000 7f19f9981640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f19f406c650 0x7f19f40ad290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:23.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.340+0000 7f19f9981640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f19f40cef40 0x7f19f414e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:23.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.340+0000 7f19f9981640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f19f40cf320 0x7f19f40aecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:23.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.340+0000 7f19f9981640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f19f406db70 con 0x7f19f40cf320 2024-09-12T03:19:23.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.340+0000 7f19f9981640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f19f406d890 con 0x7f19f406c650 2024-09-12T03:19:23.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.340+0000 7f19f9981640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f19f406ea80 con 0x7f19f40cef40 2024-09-12T03:19:23.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.340+0000 7f19f37fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f19f40cf320 0x7f19f40aecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:23.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.340+0000 7f19f37fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f19f40cf320 0x7f19f40aecf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42344/0 (socket says 172.21.15.27:42344) 2024-09-12T03:19:23.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.340+0000 7f19f37fe640 1 -- 172.21.15.27:0/2989520088 learned_addr learned my addr 172.21.15.27:0/2989520088 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:19:23.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.340+0000 7f19f27fc640 1 --2- 172.21.15.27:0/2989520088 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f19f40cef40 0x7f19f414e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:23.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.340+0000 7f19f2ffd640 1 --2- 172.21.15.27:0/2989520088 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f19f406c650 0x7f19f40ad290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:23.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.340+0000 7f19f37fe640 1 -- 172.21.15.27:0/2989520088 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f19f40cef40 msgr2=0x7f19f414e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:23.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.340+0000 7f19f37fe640 1 --2- 172.21.15.27:0/2989520088 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f19f40cef40 0x7f19f414e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:23.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.340+0000 7f19f37fe640 1 -- 172.21.15.27:0/2989520088 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f19f406c650 msgr2=0x7f19f40ad290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:23.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.340+0000 7f19f37fe640 1 --2- 172.21.15.27:0/2989520088 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f19f406c650 0x7f19f40ad290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:23.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.340+0000 7f19f37fe640 1 -- 172.21.15.27:0/2989520088 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19f415be10 con 0x7f19f40cf320 2024-09-12T03:19:23.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.341+0000 7f19f37fe640 1 --2- 172.21.15.27:0/2989520088 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f19f40cf320 0x7f19f40aecf0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f19ec00c730 tx=0x7f19ec00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:23.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.341+0000 7f19dbfff640 1 -- 172.21.15.27:0/2989520088 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19ec017070 con 0x7f19f40cf320 2024-09-12T03:19:23.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.341+0000 7f19dbfff640 1 -- 172.21.15.27:0/2989520088 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19ec00ce40 con 0x7f19f40cf320 2024-09-12T03:19:23.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.341+0000 7f19f9981640 1 -- 172.21.15.27:0/2989520088 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19f40ac4d0 con 0x7f19f40cf320 2024-09-12T03:19:23.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.341+0000 7f19dbfff640 1 -- 172.21.15.27:0/2989520088 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19ec00a840 con 0x7f19f40cf320 2024-09-12T03:19:23.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.341+0000 7f19f9981640 1 -- 172.21.15.27:0/2989520088 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19f40aca90 con 0x7f19f40cf320 2024-09-12T03:19:23.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.342+0000 7f19f9981640 1 -- 172.21.15.27:0/2989520088 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19f415a570 con 0x7f19f40cf320 2024-09-12T03:19:23.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.342+0000 7f19dbfff640 1 -- 172.21.15.27:0/2989520088 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 9) ==== 90651+0+0 (secure 0 0 0) 0x7f19ec01a070 con 0x7f19f40cf320 2024-09-12T03:19:23.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.343+0000 7f19dbfff640 1 --2- 172.21.15.27:0/2989520088 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f19e006bfe0 0x7f19e006e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:23.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.343+0000 7f19dbfff640 1 -- 172.21.15.27:0/2989520088 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 4857+0+0 (secure 0 0 0) 0x7f19ec08a250 con 0x7f19f40cf320 2024-09-12T03:19:23.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.343+0000 7f19f2ffd640 1 --2- 172.21.15.27:0/2989520088 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f19e006bfe0 0x7f19e006e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:23.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.343+0000 7f19f2ffd640 1 --2- 172.21.15.27:0/2989520088 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f19e006bfe0 0x7f19e006e4a0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f19e4009b60 tx=0x7f19e4008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:23.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.347+0000 7f19dbfff640 1 -- 172.21.15.27:0/2989520088 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f19f415a570 con 0x7f19f40cf320 2024-09-12T03:19:23.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.524+0000 7f19f9981640 1 -- 172.21.15.27:0/2989520088 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f19f406ede0 con 0x7f19f40cf320 2024-09-12T03:19:23.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.836+0000 7f19dbfff640 1 -- 172.21.15.27:0/2989520088 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v53) ==== 138+0+0 (secure 0 0 0) 0x7f19ec058710 con 0x7f19f40cf320 2024-09-12T03:19:23.841 INFO:teuthology.orchestra.run.smithi027.stderr:pool 'cephfs_metadata' created 2024-09-12T03:19:23.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.838+0000 7f19f9981640 1 -- 172.21.15.27:0/2989520088 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f19e006bfe0 msgr2=0x7f19e006e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:23.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.838+0000 7f19f9981640 1 --2- 172.21.15.27:0/2989520088 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f19e006bfe0 0x7f19e006e4a0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f19e4009b60 tx=0x7f19e4008040 comp rx=0 tx=0).stop 2024-09-12T03:19:23.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.838+0000 7f19f9981640 1 -- 172.21.15.27:0/2989520088 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f19f40cf320 msgr2=0x7f19f40aecf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:23.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.838+0000 7f19f9981640 1 --2- 172.21.15.27:0/2989520088 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f19f40cf320 0x7f19f40aecf0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f19ec00c730 tx=0x7f19ec00cbf0 comp rx=0 tx=0).stop 2024-09-12T03:19:23.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.838+0000 7f19f9981640 1 -- 172.21.15.27:0/2989520088 shutdown_connections 2024-09-12T03:19:23.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.838+0000 7f19f9981640 1 --2- 172.21.15.27:0/2989520088 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f19e006bfe0 0x7f19e006e4a0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:23.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.838+0000 7f19f9981640 1 --2- 172.21.15.27:0/2989520088 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f19f40cf320 0x7f19f40aecf0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:23.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.838+0000 7f19f9981640 1 --2- 172.21.15.27:0/2989520088 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f19f40cef40 0x7f19f414e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:23.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.838+0000 7f19f9981640 1 --2- 172.21.15.27:0/2989520088 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f19f406c650 0x7f19f40ad290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:23.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.838+0000 7f19f9981640 1 -- 172.21.15.27:0/2989520088 >> 172.21.15.27:0/2989520088 conn(0x7f19f406bb30 msgr2=0x7f19f40ce440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:23.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.838+0000 7f19f9981640 1 -- 172.21.15.27:0/2989520088 shutdown_connections 2024-09-12T03:19:23.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.838+0000 7f19f9981640 1 -- 172.21.15.27:0/2989520088 wait complete. 2024-09-12T03:19:23.854 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-12T03:19:23.963 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.962+0000 7f68cc477640 1 Processor -- start 2024-09-12T03:19:23.963 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.962+0000 7f68cc477640 1 -- start start 2024-09-12T03:19:23.964 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.963+0000 7f68cc477640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f68c415e900 0x7f68c40af370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:23.964 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.963+0000 7f68cc477640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f68c40b6540 0x7f68c406ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:23.964 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.963+0000 7f68cc477640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f68c40b5940 con 0x7f68c406d730 2024-09-12T03:19:23.964 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.963+0000 7f68cc477640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f68c415ece0 con 0x7f68c40b6540 2024-09-12T03:19:23.964 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.963+0000 7f68cc477640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f68c40af950 con 0x7f68c415e900 2024-09-12T03:19:23.964 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.963+0000 7f68caa14640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f68c40b6540 0x7f68c406ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:23.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.963+0000 7f68c9a12640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f68c406d730 0x7f68c40b5e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:49108/0 (socket says 172.21.15.27:49108) 2024-09-12T03:19:23.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.963+0000 7f68c9a12640 1 -- 172.21.15.27:0/2569676786 learned_addr learned my addr 172.21.15.27:0/2569676786 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:19:23.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.963+0000 7f68caa14640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f68c40b6540 0x7f68c406ee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42356/0 (socket says 172.21.15.27:42356) 2024-09-12T03:19:23.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.963+0000 7f68ca213640 1 --2- 172.21.15.27:0/2569676786 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f68c415e900 0x7f68c40af370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:23.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.963+0000 7f68caa14640 1 -- 172.21.15.27:0/2569676786 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f68c415e900 msgr2=0x7f68c40af370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:23.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.963+0000 7f68caa14640 1 --2- 172.21.15.27:0/2569676786 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f68c415e900 0x7f68c40af370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:23.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.963+0000 7f68caa14640 1 -- 172.21.15.27:0/2569676786 >> v1:172.21.15.27:6789/0 conn(0x7f68c406d730 legacy=0x7f68c40b5e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:23.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.963+0000 7f68caa14640 1 -- 172.21.15.27:0/2569676786 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68c40afc30 con 0x7f68c40b6540 2024-09-12T03:19:23.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.964+0000 7f68caa14640 1 --2- 172.21.15.27:0/2569676786 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f68c40b6540 0x7f68c406ee80 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f68bc009d90 tx=0x7f68bc02a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=480cc3115b83666a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:23.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.964+0000 7f68c9211640 1 -- 172.21.15.27:0/2569676786 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68bc007740 con 0x7f68c40b6540 2024-09-12T03:19:23.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.964+0000 7f68c9211640 1 -- 172.21.15.27:0/2569676786 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68bc0078e0 con 0x7f68c40b6540 2024-09-12T03:19:23.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.964+0000 7f68c9211640 1 -- 172.21.15.27:0/2569676786 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68bc007cb0 con 0x7f68c40b6540 2024-09-12T03:19:23.967 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.965+0000 7f68cc477640 1 -- 172.21.15.27:0/2569676786 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f68c40b6540 msgr2=0x7f68c406ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:23.967 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.965+0000 7f68cc477640 1 --2- 172.21.15.27:0/2569676786 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f68c40b6540 0x7f68c406ee80 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f68bc009d90 tx=0x7f68bc02a660 comp rx=0 tx=0).stop 2024-09-12T03:19:23.967 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.965+0000 7f68cc477640 1 -- 172.21.15.27:0/2569676786 shutdown_connections 2024-09-12T03:19:23.967 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.965+0000 7f68cc477640 1 --2- 172.21.15.27:0/2569676786 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f68c40b6540 0x7f68c406ee80 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:23.967 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.965+0000 7f68cc477640 1 --2- 172.21.15.27:0/2569676786 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f68c415e900 0x7f68c40af370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:23.967 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.965+0000 7f68cc477640 1 -- 172.21.15.27:0/2569676786 >> 172.21.15.27:0/2569676786 conn(0x7f68c406cc40 msgr2=0x7f68c406d020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:23.967 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.965+0000 7f68cc477640 1 -- 172.21.15.27:0/2569676786 shutdown_connections 2024-09-12T03:19:23.967 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.965+0000 7f68cc477640 1 -- 172.21.15.27:0/2569676786 wait complete. 2024-09-12T03:19:23.969 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.966+0000 7f68cc477640 1 Processor -- start 2024-09-12T03:19:23.969 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.966+0000 7f68cc477640 1 -- start start 2024-09-12T03:19:23.969 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.967+0000 7f68cc477640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f68c406d730 0x7f68c41f51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:23.969 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.967+0000 7f68cc477640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f68c40b6540 0x7f68c406ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:23.969 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.967+0000 7f68cc477640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f68c415e900 0x7f68c4150490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:23.970 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.967+0000 7f68cc477640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f68c406e880 con 0x7f68c40b6540 2024-09-12T03:19:23.970 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.967+0000 7f68cc477640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f68c40b01f0 con 0x7f68c415e900 2024-09-12T03:19:23.970 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.967+0000 7f68cc477640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f68c406ea00 con 0x7f68c406d730 2024-09-12T03:19:23.970 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.967+0000 7f68c9a12640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f68c40b6540 0x7f68c406ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:23.970 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.967+0000 7f68c9a12640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f68c40b6540 0x7f68c406ef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42370/0 (socket says 172.21.15.27:42370) 2024-09-12T03:19:23.970 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.967+0000 7f68c9a12640 1 -- 172.21.15.27:0/1548550993 learned_addr learned my addr 172.21.15.27:0/1548550993 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:19:23.970 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.967+0000 7f68ca213640 1 --2- 172.21.15.27:0/1548550993 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f68c406d730 0x7f68c41f51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:23.970 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.967+0000 7f68caa14640 1 --2- 172.21.15.27:0/1548550993 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f68c415e900 0x7f68c4150490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:23.970 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.967+0000 7f68c9a12640 1 -- 172.21.15.27:0/1548550993 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f68c406d730 msgr2=0x7f68c41f51c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:23.970 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.967+0000 7f68c9a12640 1 --2- 172.21.15.27:0/1548550993 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f68c406d730 0x7f68c41f51c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:23.970 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.967+0000 7f68c9a12640 1 -- 172.21.15.27:0/1548550993 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f68c415e900 msgr2=0x7f68c4150490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:23.970 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.967+0000 7f68c9a12640 1 --2- 172.21.15.27:0/1548550993 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f68c415e900 0x7f68c4150490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:23.970 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.967+0000 7f68c9a12640 1 -- 172.21.15.27:0/1548550993 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68c4160630 con 0x7f68c40b6540 2024-09-12T03:19:23.971 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.968+0000 7f68c9a12640 1 --2- 172.21.15.27:0/1548550993 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f68c40b6540 0x7f68c406ef40 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f68b400c440 tx=0x7f68b4001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:23.971 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.968+0000 7f68baffd640 1 -- 172.21.15.27:0/1548550993 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68b4017070 con 0x7f68c40b6540 2024-09-12T03:19:23.971 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.968+0000 7f68baffd640 1 -- 172.21.15.27:0/1548550993 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68b4005ce0 con 0x7f68c40b6540 2024-09-12T03:19:23.971 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.968+0000 7f68cc477640 1 -- 172.21.15.27:0/1548550993 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68c41f7d60 con 0x7f68c40b6540 2024-09-12T03:19:23.971 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.968+0000 7f68baffd640 1 -- 172.21.15.27:0/1548550993 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68b40072f0 con 0x7f68c40b6540 2024-09-12T03:19:23.971 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.968+0000 7f68cc477640 1 -- 172.21.15.27:0/1548550993 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68c41f8320 con 0x7f68c40b6540 2024-09-12T03:19:23.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.969+0000 7f68cc477640 1 -- 172.21.15.27:0/1548550993 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68c40af370 con 0x7f68c40b6540 2024-09-12T03:19:23.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.969+0000 7f68baffd640 1 -- 172.21.15.27:0/1548550993 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 9) ==== 90651+0+0 (secure 0 0 0) 0x7f68b401a070 con 0x7f68c40b6540 2024-09-12T03:19:23.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.970+0000 7f68baffd640 1 --2- 172.21.15.27:0/1548550993 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f68a806bfe0 0x7f68a806e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:23.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.970+0000 7f68baffd640 1 -- 172.21.15.27:0/1548550993 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 5227+0+0 (secure 0 0 0) 0x7f68b408a1d0 con 0x7f68c40b6540 2024-09-12T03:19:23.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.970+0000 7f68ca213640 1 --2- 172.21.15.27:0/1548550993 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f68a806bfe0 0x7f68a806e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:23.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.971+0000 7f68ca213640 1 --2- 172.21.15.27:0/1548550993 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f68a806bfe0 0x7f68a806e4a0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f68c4069170 tx=0x7f68c0008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:23.975 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:23.975+0000 7f68baffd640 1 -- 172.21.15.27:0/1548550993 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f68c40b5940 con 0x7f68c40b6540 2024-09-12T03:19:24.153 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:24.152+0000 7f68cc477640 1 -- 172.21.15.27:0/1548550993 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f68c40b1490 con 0x7f68c40b6540 2024-09-12T03:19:24.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:24.840+0000 7f68baffd640 1 -- 172.21.15.27:0/1548550993 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v54) ==== 170+0+0 (secure 0 0 0) 0x7f68b40585d0 con 0x7f68c40b6540 2024-09-12T03:19:24.842 INFO:teuthology.orchestra.run.smithi027.stderr:pool 'cephfs_data' created 2024-09-12T03:19:24.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:24.844+0000 7f68cc477640 1 -- 172.21.15.27:0/1548550993 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f68a806bfe0 msgr2=0x7f68a806e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:24.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:24.844+0000 7f68cc477640 1 --2- 172.21.15.27:0/1548550993 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f68a806bfe0 0x7f68a806e4a0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f68c4069170 tx=0x7f68c0008040 comp rx=0 tx=0).stop 2024-09-12T03:19:24.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:24.844+0000 7f68cc477640 1 -- 172.21.15.27:0/1548550993 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f68c40b6540 msgr2=0x7f68c406ef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:24.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:24.844+0000 7f68cc477640 1 --2- 172.21.15.27:0/1548550993 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f68c40b6540 0x7f68c406ef40 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f68b400c440 tx=0x7f68b4001e70 comp rx=0 tx=0).stop 2024-09-12T03:19:24.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:24.844+0000 7f68cc477640 1 -- 172.21.15.27:0/1548550993 shutdown_connections 2024-09-12T03:19:24.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:24.844+0000 7f68cc477640 1 --2- 172.21.15.27:0/1548550993 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f68a806bfe0 0x7f68a806e4a0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:24.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:24.844+0000 7f68cc477640 1 --2- 172.21.15.27:0/1548550993 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f68c415e900 0x7f68c4150490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:24.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:24.844+0000 7f68cc477640 1 --2- 172.21.15.27:0/1548550993 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f68c40b6540 0x7f68c406ef40 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:24.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:24.844+0000 7f68cc477640 1 --2- 172.21.15.27:0/1548550993 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f68c406d730 0x7f68c41f51c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:24.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:24.844+0000 7f68cc477640 1 -- 172.21.15.27:0/1548550993 >> 172.21.15.27:0/1548550993 conn(0x7f68c406cc40 msgr2=0x7f68c415d910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:24.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:24.845+0000 7f68cc477640 1 -- 172.21.15.27:0/1548550993 shutdown_connections 2024-09-12T03:19:24.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:24.845+0000 7f68cc477640 1 -- 172.21.15.27:0/1548550993 wait complete. 2024-09-12T03:19:24.867 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-12T03:19:24.976 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:24.975+0000 7f6f7c2e6640 1 Processor -- start 2024-09-12T03:19:24.976 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:24.975+0000 7f6f7c2e6640 1 -- start start 2024-09-12T03:19:24.984 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:24.976+0000 7f6f7c2e6640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f6f740b6230 0x7f6f740b6690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:24.984 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:24.976+0000 7f6f7c2e6640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f6f740b3600 0x7f6f740b3a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:24.984 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:24.976+0000 7f6f7c2e6640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6f7415ed70 con 0x7f6f7415e990 2024-09-12T03:19:24.984 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:24.976+0000 7f6f7c2e6640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f6f740b6bd0 con 0x7f6f740b3600 2024-09-12T03:19:24.984 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:24.976+0000 7f6f7c2e6640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f6f740b7d10 con 0x7f6f740b6230 2024-09-12T03:19:24.984 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:24.976+0000 7f6f7a883640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f6f740b3600 0x7f6f740b3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:24.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:24.976+0000 7f6f7a883640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f6f740b3600 0x7f6f740b3a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42382/0 (socket says 172.21.15.27:42382) 2024-09-12T03:19:24.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:24.976+0000 7f6f7a883640 1 -- 172.21.15.27:0/1641513633 learned_addr learned my addr 172.21.15.27:0/1641513633 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:19:24.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:24.977+0000 7f6f79881640 1 --2- 172.21.15.27:0/1641513633 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f6f740b6230 0x7f6f740b6690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:24.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:24.977+0000 7f6f7a883640 1 -- 172.21.15.27:0/1641513633 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f6f740b6230 msgr2=0x7f6f740b6690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:24.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:24.977+0000 7f6f7a883640 1 --2- 172.21.15.27:0/1641513633 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f6f740b6230 0x7f6f740b6690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:24.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:24.977+0000 7f6f7a883640 1 -- 172.21.15.27:0/1641513633 >> v1:172.21.15.27:6789/0 conn(0x7f6f7415e990 legacy=0x7f6f740b7600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:24.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:24.977+0000 7f6f79080640 1 -- 172.21.15.27:0/1641513633 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1100503800 0 0) 0x7f6f7415ed70 con 0x7f6f7415e990 2024-09-12T03:19:24.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:24.977+0000 7f6f7a883640 1 -- 172.21.15.27:0/1641513633 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f740b3fb0 con 0x7f6f740b3600 2024-09-12T03:19:24.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:24.977+0000 7f6f7a883640 1 --2- 172.21.15.27:0/1641513633 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6f740b3600 0x7f6f740b3a70 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f6f6c009870 tx=0x7f6f6c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a43cffa1b0743ed7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:24.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:24.978+0000 7f6f79080640 1 -- 172.21.15.27:0/1641513633 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f6c037070 con 0x7f6f740b3600 2024-09-12T03:19:24.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:24.978+0000 7f6f79080640 1 -- 172.21.15.27:0/1641513633 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f6c032440 con 0x7f6f740b3600 2024-09-12T03:19:24.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:24.978+0000 7f6f79080640 1 -- 172.21.15.27:0/1641513633 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f6c032810 con 0x7f6f740b3600 2024-09-12T03:19:24.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:24.978+0000 7f6f7c2e6640 1 -- 172.21.15.27:0/1641513633 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6f740b3600 msgr2=0x7f6f740b3a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:24.986 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:24.978+0000 7f6f7c2e6640 1 --2- 172.21.15.27:0/1641513633 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6f740b3600 0x7f6f740b3a70 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f6f6c009870 tx=0x7f6f6c02a270 comp rx=0 tx=0).stop 2024-09-12T03:19:24.986 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:24.978+0000 7f6f7c2e6640 1 -- 172.21.15.27:0/1641513633 shutdown_connections 2024-09-12T03:19:24.986 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:24.978+0000 7f6f7c2e6640 1 --2- 172.21.15.27:0/1641513633 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6f740b3600 0x7f6f740b3a70 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:24.986 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:24.978+0000 7f6f7c2e6640 1 --2- 172.21.15.27:0/1641513633 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f6f740b6230 0x7f6f740b6690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:24.986 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:24.978+0000 7f6f7c2e6640 1 -- 172.21.15.27:0/1641513633 >> 172.21.15.27:0/1641513633 conn(0x7f6f740d1230 msgr2=0x7f6f740d1640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:24.986 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:24.978+0000 7f6f7c2e6640 1 -- 172.21.15.27:0/1641513633 shutdown_connections 2024-09-12T03:19:24.986 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:24.978+0000 7f6f7c2e6640 1 -- 172.21.15.27:0/1641513633 wait complete. 2024-09-12T03:19:24.986 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:24.979+0000 7f6f7c2e6640 1 Processor -- start 2024-09-12T03:19:24.986 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:24.979+0000 7f6f7c2e6640 1 -- start start 2024-09-12T03:19:24.986 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:24.980+0000 7f6f7c2e6640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6f740b3600 0x7f6f74158fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:24.986 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:24.980+0000 7f6f7c2e6640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f6f740b6230 0x7f6f74160690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:24.986 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:24.980+0000 7f6f7c2e6640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6f7415e990 0x7f6f74160bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:24.986 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:24.980+0000 7f6f7c2e6640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f6f740d2ff0 con 0x7f6f740b3600 2024-09-12T03:19:24.986 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:24.980+0000 7f6f7c2e6640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f6f740d2d10 con 0x7f6f7415e990 2024-09-12T03:19:24.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:24.980+0000 7f6f7c2e6640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f6f740d32d0 con 0x7f6f740b6230 2024-09-12T03:19:24.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:24.980+0000 7f6f7a082640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6f740b3600 0x7f6f74158fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:24.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:24.980+0000 7f6f7a082640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6f740b3600 0x7f6f74158fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42392/0 (socket says 172.21.15.27:42392) 2024-09-12T03:19:24.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:24.980+0000 7f6f7a082640 1 -- 172.21.15.27:0/561067204 learned_addr learned my addr 172.21.15.27:0/561067204 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:19:24.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:24.980+0000 7f6f7a082640 1 -- 172.21.15.27:0/561067204 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f6f740b6230 msgr2=0x7f6f74160690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:24.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:24.980+0000 7f6f79881640 1 --2- 172.21.15.27:0/561067204 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f6f740b6230 0x7f6f74160690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:24.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:24.980+0000 7f6f7a883640 1 --2- 172.21.15.27:0/561067204 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6f7415e990 0x7f6f74160bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:24.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:24.981+0000 7f6f7a082640 1 --2- 172.21.15.27:0/561067204 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f6f740b6230 0x7f6f74160690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:24.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:24.981+0000 7f6f7a082640 1 -- 172.21.15.27:0/561067204 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6f7415e990 msgr2=0x7f6f74160bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:24.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:24.981+0000 7f6f7a082640 1 --2- 172.21.15.27:0/561067204 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6f7415e990 0x7f6f74160bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:24.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:24.981+0000 7f6f7a082640 1 -- 172.21.15.27:0/561067204 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f740cba00 con 0x7f6f740b3600 2024-09-12T03:19:24.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:24.981+0000 7f6f79881640 1 --2- 172.21.15.27:0/561067204 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f6f740b6230 0x7f6f74160690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:19:24.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:24.981+0000 7f6f7a883640 1 --2- 172.21.15.27:0/561067204 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6f7415e990 0x7f6f74160bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:19:24.988 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:24.981+0000 7f6f7a082640 1 --2- 172.21.15.27:0/561067204 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6f740b3600 0x7f6f74158fa0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f6f6400c7f0 tx=0x7f6f6400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:24.988 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:24.981+0000 7f6f72ffd640 1 -- 172.21.15.27:0/561067204 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f64017070 con 0x7f6f740b3600 2024-09-12T03:19:24.988 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:24.981+0000 7f6f72ffd640 1 -- 172.21.15.27:0/561067204 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f64003da0 con 0x7f6f740b3600 2024-09-12T03:19:24.988 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:24.981+0000 7f6f72ffd640 1 -- 172.21.15.27:0/561067204 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f64005290 con 0x7f6f740b3600 2024-09-12T03:19:24.988 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:24.981+0000 7f6f7c2e6640 1 -- 172.21.15.27:0/561067204 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f741f5020 con 0x7f6f740b3600 2024-09-12T03:19:24.988 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:24.981+0000 7f6f7c2e6640 1 -- 172.21.15.27:0/561067204 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f741f5440 con 0x7f6f740b3600 2024-09-12T03:19:24.988 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:24.982+0000 7f6f7c2e6640 1 -- 172.21.15.27:0/561067204 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f740d35b0 con 0x7f6f740b3600 2024-09-12T03:19:24.988 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:24.983+0000 7f6f72ffd640 1 -- 172.21.15.27:0/561067204 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 9) ==== 90651+0+0 (secure 0 0 0) 0x7f6f6401a070 con 0x7f6f740b3600 2024-09-12T03:19:24.988 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:24.983+0000 7f6f72ffd640 1 --2- 172.21.15.27:0/561067204 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f6f6006bfe0 0x7f6f6006e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:24.988 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:24.983+0000 7f6f72ffd640 1 -- 172.21.15.27:0/561067204 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 5609+0+0 (secure 0 0 0) 0x7f6f6408a740 con 0x7f6f740b3600 2024-09-12T03:19:24.988 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:24.983+0000 7f6f79881640 1 --2- 172.21.15.27:0/561067204 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f6f6006bfe0 0x7f6f6006e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:24.989 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:24.984+0000 7f6f79881640 1 --2- 172.21.15.27:0/561067204 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f6f6006bfe0 0x7f6f6006e4a0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f6f740cb640 tx=0x7f6f68008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:24.989 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:24.988+0000 7f6f72ffd640 1 -- 172.21.15.27:0/561067204 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f6f64058f60 con 0x7f6f740b3600 2024-09-12T03:19:25.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:25.198+0000 7f6f7c2e6640 1 -- 172.21.15.27:0/561067204 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f6f741f5700 con 0x7f6f740b3600 2024-09-12T03:19:25.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:25.842+0000 7f6f72ffd640 1 -- 172.21.15.27:0/561067204 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 8 and data pool 9 v28) ==== 172+0+0 (secure 0 0 0) 0x7f6f64058860 con 0x7f6f740b3600 2024-09-12T03:19:25.843 INFO:teuthology.orchestra.run.smithi027.stderr:new fs with metadata pool 8 and data pool 9 2024-09-12T03:19:25.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:25.844+0000 7f6f7c2e6640 1 -- 172.21.15.27:0/561067204 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f6f6006bfe0 msgr2=0x7f6f6006e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:25.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:25.844+0000 7f6f7c2e6640 1 --2- 172.21.15.27:0/561067204 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f6f6006bfe0 0x7f6f6006e4a0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f6f740cb640 tx=0x7f6f68008040 comp rx=0 tx=0).stop 2024-09-12T03:19:25.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:25.844+0000 7f6f7c2e6640 1 -- 172.21.15.27:0/561067204 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6f740b3600 msgr2=0x7f6f74158fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:25.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:25.844+0000 7f6f7c2e6640 1 --2- 172.21.15.27:0/561067204 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6f740b3600 0x7f6f74158fa0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f6f6400c7f0 tx=0x7f6f6400ccb0 comp rx=0 tx=0).stop 2024-09-12T03:19:25.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:25.844+0000 7f6f7c2e6640 1 -- 172.21.15.27:0/561067204 shutdown_connections 2024-09-12T03:19:25.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:25.844+0000 7f6f7c2e6640 1 --2- 172.21.15.27:0/561067204 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f6f6006bfe0 0x7f6f6006e4a0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:25.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:25.844+0000 7f6f7c2e6640 1 --2- 172.21.15.27:0/561067204 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6f7415e990 0x7f6f74160bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:25.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:25.844+0000 7f6f7c2e6640 1 --2- 172.21.15.27:0/561067204 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f6f740b6230 0x7f6f74160690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:25.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:25.844+0000 7f6f7c2e6640 1 --2- 172.21.15.27:0/561067204 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6f740b3600 0x7f6f74158fa0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:25.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:25.844+0000 7f6f7c2e6640 1 -- 172.21.15.27:0/561067204 >> 172.21.15.27:0/561067204 conn(0x7f6f740d1230 msgr2=0x7f6f740d1610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:25.847 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:25.845+0000 7f6f7c2e6640 1 -- 172.21.15.27:0/561067204 shutdown_connections 2024-09-12T03:19:25.847 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:25.845+0000 7f6f7c2e6640 1 -- 172.21.15.27:0/561067204 wait complete. 2024-09-12T03:19:25.862 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-12T03:19:25.968 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:25.967+0000 7fb439a77640 1 Processor -- start 2024-09-12T03:19:25.968 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:25.967+0000 7fb439a77640 1 -- start start 2024-09-12T03:19:25.971 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:25.968+0000 7fb439a77640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb4340b1ca0 0x7fb43415f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:25.971 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:25.968+0000 7fb439a77640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb43415fd70 con 0x7fb4340cf660 2024-09-12T03:19:25.971 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:25.968+0000 7fb439a77640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb43406cc40 con 0x7fb43406f360 2024-09-12T03:19:25.971 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:25.968+0000 7fb439a77640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fb4340d22f0 con 0x7fb4340b1ca0 2024-09-12T03:19:25.971 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:25.968+0000 7fb433fff640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fb4340cf660 0x7fb4340d2780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:49128/0 (socket says 172.21.15.27:49128) 2024-09-12T03:19:25.971 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:25.968+0000 7fb433fff640 1 -- 172.21.15.27:0/3412877557 learned_addr learned my addr 172.21.15.27:0/3412877557 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:19:25.971 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:25.968+0000 7fb4327fc640 1 -- 172.21.15.27:0/3412877557 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1787216411 0 0) 0x7fb42c001490 con 0x7fb4340cf660 2024-09-12T03:19:25.971 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:25.968+0000 7fb4327fc640 1 -- 172.21.15.27:0/3412877557 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb420003520 con 0x7fb4340cf660 2024-09-12T03:19:25.971 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:25.968+0000 7fb4337fe640 1 --2- 172.21.15.27:0/3412877557 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb4340b1ca0 0x7fb43415f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:25.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:25.969+0000 7fb4327fc640 1 -- 172.21.15.27:0/3412877557 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 809339367 0 0) 0x7fb420003520 con 0x7fb4340cf660 2024-09-12T03:19:25.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:25.969+0000 7fb4327fc640 1 -- 172.21.15.27:0/3412877557 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb4340b1ca0 msgr2=0x7fb43415f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:25.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:25.969+0000 7fb4327fc640 1 --2- 172.21.15.27:0/3412877557 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb4340b1ca0 0x7fb43415f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:25.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:25.969+0000 7fb4327fc640 1 -- 172.21.15.27:0/3412877557 >> v1:172.21.15.45:6789/0 conn(0x7fb43406f360 legacy=0x7fb4340cef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:25.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:25.969+0000 7fb4327fc640 1 -- 172.21.15.27:0/3412877557 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4340d1ce0 con 0x7fb4340cf660 2024-09-12T03:19:25.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:25.969+0000 7fb4327fc640 1 -- 172.21.15.27:0/3412877557 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 847755213 0 0) 0x7fb42c001bc0 con 0x7fb4340cf660 2024-09-12T03:19:25.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:25.969+0000 7fb4327fc640 1 -- 172.21.15.27:0/3412877557 >> v1:172.21.15.27:6789/0 conn(0x7fb4340cf660 legacy=0x7fb4340d2780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:25.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:25.969+0000 7fb4327fc640 1 --2- 172.21.15.27:0/3412877557 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb4200039e0 0x7fb420003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:25.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:25.969+0000 7fb4327fc640 1 --2- 172.21.15.27:0/3412877557 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb4340b1ca0 0x7fb420004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:25.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:25.969+0000 7fb4327fc640 1 --2- 172.21.15.27:0/3412877557 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb420004d40 0x7fb4200051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:25.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:25.969+0000 7fb4327fc640 1 -- 172.21.15.27:0/3412877557 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fb4340d22f0 con 0x7fb420004d40 2024-09-12T03:19:25.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:25.969+0000 7fb4327fc640 1 -- 172.21.15.27:0/3412877557 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb42000ba70 con 0x7fb4200039e0 2024-09-12T03:19:25.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:25.969+0000 7fb4327fc640 1 -- 172.21.15.27:0/3412877557 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fb42000bc70 con 0x7fb4340b1ca0 2024-09-12T03:19:25.973 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:25.969+0000 7fb433fff640 1 --2- 172.21.15.27:0/3412877557 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb420004d40 0x7fb4200051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:25.973 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:25.969+0000 7fb4337fe640 1 --2- 172.21.15.27:0/3412877557 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb4200039e0 0x7fb420003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:25.973 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:25.969+0000 7fb432ffd640 1 --2- 172.21.15.27:0/3412877557 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb4340b1ca0 0x7fb420004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:25.973 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:25.969+0000 7fb433fff640 1 -- 172.21.15.27:0/3412877557 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb4340b1ca0 msgr2=0x7fb420004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:25.973 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:25.969+0000 7fb433fff640 1 --2- 172.21.15.27:0/3412877557 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb4340b1ca0 0x7fb420004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:25.973 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:25.969+0000 7fb433fff640 1 -- 172.21.15.27:0/3412877557 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb4200039e0 msgr2=0x7fb420003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:25.973 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:25.969+0000 7fb433fff640 1 --2- 172.21.15.27:0/3412877557 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb4200039e0 0x7fb420003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:25.973 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:25.969+0000 7fb433fff640 1 -- 172.21.15.27:0/3412877557 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb420003520 con 0x7fb420004d40 2024-09-12T03:19:25.973 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:25.970+0000 7fb433fff640 1 --2- 172.21.15.27:0/3412877557 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb420004d40 0x7fb4200051c0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fb434069d10 tx=0x7fb420005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc8d8963ee3bb819 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:25.973 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:25.970+0000 7fb4327fc640 1 -- 172.21.15.27:0/3412877557 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb42c00a670 con 0x7fb420004d40 2024-09-12T03:19:25.973 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:25.970+0000 7fb4327fc640 1 -- 172.21.15.27:0/3412877557 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb42c02f070 con 0x7fb420004d40 2024-09-12T03:19:25.973 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:25.970+0000 7fb4327fc640 1 -- 172.21.15.27:0/3412877557 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb42c02be40 con 0x7fb420004d40 2024-09-12T03:19:25.976 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:25.971+0000 7fb439a77640 1 -- 172.21.15.27:0/3412877557 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb420004d40 msgr2=0x7fb4200051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:25.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:25.971+0000 7fb439a77640 1 --2- 172.21.15.27:0/3412877557 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb420004d40 0x7fb4200051c0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fb434069d10 tx=0x7fb420005830 comp rx=0 tx=0).stop 2024-09-12T03:19:25.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:25.971+0000 7fb433fff640 1 -- 172.21.15.27:0/3412877557 reap_dead start 2024-09-12T03:19:25.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:25.971+0000 7fb439a77640 1 -- 172.21.15.27:0/3412877557 shutdown_connections 2024-09-12T03:19:25.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:25.971+0000 7fb439a77640 1 -- 172.21.15.27:0/3412877557 >> 172.21.15.27:0/3412877557 conn(0x7fb434073800 msgr2=0x7fb43409ba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:25.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:25.971+0000 7fb439a77640 1 -- 172.21.15.27:0/3412877557 shutdown_connections 2024-09-12T03:19:25.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:25.971+0000 7fb439a77640 1 -- 172.21.15.27:0/3412877557 wait complete. 2024-09-12T03:19:25.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:25.972+0000 7fb439a77640 1 Processor -- start 2024-09-12T03:19:25.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:25.972+0000 7fb439a77640 1 -- start start 2024-09-12T03:19:25.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:25.973+0000 7fb439a77640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb420004d40 0x7fb4340b4940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:25.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:25.973+0000 7fb439a77640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb4340ce3e0 0x7fb434151140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:25.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:25.973+0000 7fb439a77640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb4340cbf60 0x7fb4340cc3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:25.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:25.973+0000 7fb439a77640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fb43406d1c0 con 0x7fb420004d40 2024-09-12T03:19:25.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:25.973+0000 7fb439a77640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb4340d1f80 con 0x7fb4340cbf60 2024-09-12T03:19:25.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:25.973+0000 7fb439a77640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fb43406d340 con 0x7fb4340ce3e0 2024-09-12T03:19:25.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:25.973+0000 7fb4337fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb420004d40 0x7fb4340b4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:25.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:25.973+0000 7fb4337fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb420004d40 0x7fb4340b4940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42412/0 (socket says 172.21.15.27:42412) 2024-09-12T03:19:25.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:25.973+0000 7fb4337fe640 1 -- 172.21.15.27:0/3726105422 learned_addr learned my addr 172.21.15.27:0/3726105422 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:19:25.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:25.973+0000 7fb4337fe640 1 -- 172.21.15.27:0/3726105422 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb4340ce3e0 msgr2=0x7fb434151140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:25.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:25.973+0000 7fb4337fe640 1 --2- 172.21.15.27:0/3726105422 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb4340ce3e0 0x7fb434151140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:25.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:25.973+0000 7fb4337fe640 1 -- 172.21.15.27:0/3726105422 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb4340cbf60 msgr2=0x7fb4340cc3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:25.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:25.973+0000 7fb4337fe640 1 --2- 172.21.15.27:0/3726105422 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb4340cbf60 0x7fb4340cc3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:25.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:25.973+0000 7fb4337fe640 1 -- 172.21.15.27:0/3726105422 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb434151680 con 0x7fb420004d40 2024-09-12T03:19:25.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:25.973+0000 7fb4337fe640 1 --2- 172.21.15.27:0/3726105422 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb420004d40 0x7fb4340b4940 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fb4240119a0 tx=0x7fb424011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:25.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:25.973+0000 7fb42bfff640 1 -- 172.21.15.27:0/3726105422 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb42400fa70 con 0x7fb420004d40 2024-09-12T03:19:25.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:25.973+0000 7fb42bfff640 1 -- 172.21.15.27:0/3726105422 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb42400fc10 con 0x7fb420004d40 2024-09-12T03:19:25.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:25.973+0000 7fb439a77640 1 -- 172.21.15.27:0/3726105422 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4340cc920 con 0x7fb420004d40 2024-09-12T03:19:25.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:25.973+0000 7fb42bfff640 1 -- 172.21.15.27:0/3726105422 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb424014440 con 0x7fb420004d40 2024-09-12T03:19:25.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:25.974+0000 7fb439a77640 1 -- 172.21.15.27:0/3726105422 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4340b3d30 con 0x7fb420004d40 2024-09-12T03:19:25.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:25.974+0000 7fb439a77640 1 -- 172.21.15.27:0/3726105422 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4340cfc40 con 0x7fb420004d40 2024-09-12T03:19:25.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:25.975+0000 7fb42bfff640 1 -- 172.21.15.27:0/3726105422 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 9) ==== 90651+0+0 (secure 0 0 0) 0x7fb424012040 con 0x7fb420004d40 2024-09-12T03:19:25.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:25.976+0000 7fb42bfff640 1 --2- 172.21.15.27:0/3726105422 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fb41406c010 0x7fb41406e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:25.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:25.976+0000 7fb42bfff640 1 -- 172.21.15.27:0/3726105422 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 5709+0+0 (secure 0 0 0) 0x7fb42408b740 con 0x7fb420004d40 2024-09-12T03:19:25.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:25.976+0000 7fb432ffd640 1 --2- 172.21.15.27:0/3726105422 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fb41406c010 0x7fb41406e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:25.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:25.976+0000 7fb432ffd640 1 --2- 172.21.15.27:0/3726105422 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fb41406c010 0x7fb41406e4d0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fb4340b3210 tx=0x7fb41c009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:25.981 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:25.980+0000 7fb42bfff640 1 -- 172.21.15.27:0/3726105422 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fb4340cc920 con 0x7fb420004d40 2024-09-12T03:19:26.157 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.156+0000 7fb439a77640 1 -- 172.21.15.27:0/3726105422 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb4340b1810 con 0x7fb420004d40 2024-09-12T03:19:26.158 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.157+0000 7fb42bfff640 1 -- 172.21.15.27:0/3726105422 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v55) ==== 74+0+16690 (secure 0 0 0) 0x7fb434151680 con 0x7fb420004d40 2024-09-12T03:19:26.158 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:19:26.159 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":55,"fsid":"d2df504e-2647-4385-a5db-9fdf69f487a0","created":"2024-09-12T03:16:07.778937+0000","modified":"2024-09-12T03:19:25.200876+0000","last_up_change":"2024-09-12T03:16:19.314778+0000","last_in_change":"2024-09-12T03:16:09.893086+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T03:16:20.097646+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-09-12T03:19:23.556103+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-09-12T03:19:24.184337+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1851116c-1fb3-4dc9-9772-f90e2ef4c355","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6817","nonce":3002321488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6819","nonce":3002321488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6823","nonce":3002321488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6821","nonce":3002321488}]},"public_addr":"172.21.15.27:6817/3002321488","cluster_addr":"172.21.15.27:6819/3002321488","heartbeat_back_addr":"172.21.15.27:6823/3002321488","heartbeat_front_addr":"172.21.15.27:6821/3002321488","state":["exists","up"]},{"osd":1,"uuid":"f38ae17d-73eb-4204-b99d-975fa806fb87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6809","nonce":1613102886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6811","nonce":1613102886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6815","nonce":1613102886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6813","nonce":1613102886}]},"public_addr":"172.21.15.27:6809/1613102886","cluster_addr":"172.21.15.27:6811/1613102886","heartbeat_back_addr":"172.21.15.27:6815/1613102886","heartbeat_front_addr":"172.21.15.27:6813/1613102886","state":["exists","up"]},{"osd":2,"uuid":"fb12dcc8-bdd0-42b8-b46d-8b10106224bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6825","nonce":3534723355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6827","nonce":3534723355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6831","nonce":3534723355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6829","nonce":3534723355}]},"public_addr":"172.21.15.27:6825/3534723355","cluster_addr":"172.21.15.27:6827/3534723355","heartbeat_back_addr":"172.21.15.27:6831/3534723355","heartbeat_front_addr":"172.21.15.27:6829/3534723355","state":["exists","up"]},{"osd":3,"uuid":"d74d3f49-9c24-4b9f-98e2-3ecd7b5ef3b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6801","nonce":80661598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6803","nonce":80661598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6807","nonce":80661598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6805","nonce":80661598}]},"public_addr":"172.21.15.27:6801/80661598","cluster_addr":"172.21.15.27:6803/80661598","heartbeat_back_addr":"172.21.15.27:6807/80661598","heartbeat_front_addr":"172.21.15.27:6805/80661598","state":["exists","up"]},{"osd":4,"uuid":"2bf123f2-1b0d-41c4-aa3b-805bab267e59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6817","nonce":3943189827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6819","nonce":3943189827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6823","nonce":3943189827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6821","nonce":3943189827}]},"public_addr":"172.21.15.45:6817/3943189827","cluster_addr":"172.21.15.45:6819/3943189827","heartbeat_back_addr":"172.21.15.45:6823/3943189827","heartbeat_front_addr":"172.21.15.45:6821/3943189827","state":["exists","up"]},{"osd":5,"uuid":"c49ba5da-91c6-4a86-a97b-4aba89c253ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6809","nonce":3620760330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6811","nonce":3620760330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6815","nonce":3620760330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6813","nonce":3620760330}]},"public_addr":"172.21.15.45:6809/3620760330","cluster_addr":"172.21.15.45:6811/3620760330","heartbeat_back_addr":"172.21.15.45:6815/3620760330","heartbeat_front_addr":"172.21.15.45:6813/3620760330","state":["exists","up"]},{"osd":6,"uuid":"b0123475-89f7-4a34-9e33-d797b99dd9dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6801","nonce":1989083510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6803","nonce":1989083510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6807","nonce":1989083510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6805","nonce":1989083510}]},"public_addr":"172.21.15.45:6801/1989083510","cluster_addr":"172.21.15.45:6803/1989083510","heartbeat_back_addr":"172.21.15.45:6807/1989083510","heartbeat_front_addr":"172.21.15.45:6805/1989083510","state":["exists","up"]},{"osd":7,"uuid":"4dad773d-b7d8-46fc-8a10-a972aad27b29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6825","nonce":1513300294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6827","nonce":1513300294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6831","nonce":1513300294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6829","nonce":1513300294}]},"public_addr":"172.21.15.45:6825/1513300294","cluster_addr":"172.21.15.45:6827/1513300294","heartbeat_back_addr":"172.21.15.45:6831/1513300294","heartbeat_front_addr":"172.21.15.45:6829/1513300294","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.881855+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:17.133252+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.735339+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.500742+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.477571+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:17.167667+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.470291+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.873743+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T03:19:26.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.159+0000 7fb439a77640 1 -- 172.21.15.27:0/3726105422 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fb41406c010 msgr2=0x7fb41406e4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:26.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.159+0000 7fb439a77640 1 --2- 172.21.15.27:0/3726105422 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fb41406c010 0x7fb41406e4d0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fb4340b3210 tx=0x7fb41c009040 comp rx=0 tx=0).stop 2024-09-12T03:19:26.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.159+0000 7fb439a77640 1 -- 172.21.15.27:0/3726105422 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb420004d40 msgr2=0x7fb4340b4940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:26.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.159+0000 7fb439a77640 1 --2- 172.21.15.27:0/3726105422 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb420004d40 0x7fb4340b4940 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fb4240119a0 tx=0x7fb424011e60 comp rx=0 tx=0).stop 2024-09-12T03:19:26.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.159+0000 7fb439a77640 1 -- 172.21.15.27:0/3726105422 shutdown_connections 2024-09-12T03:19:26.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.159+0000 7fb439a77640 1 --2- 172.21.15.27:0/3726105422 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fb41406c010 0x7fb41406e4d0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:26.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.159+0000 7fb439a77640 1 --2- 172.21.15.27:0/3726105422 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb4340cbf60 0x7fb4340cc3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:26.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.159+0000 7fb439a77640 1 --2- 172.21.15.27:0/3726105422 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb4340ce3e0 0x7fb434151140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:26.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.159+0000 7fb439a77640 1 --2- 172.21.15.27:0/3726105422 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb420004d40 0x7fb4340b4940 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:26.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.159+0000 7fb439a77640 1 -- 172.21.15.27:0/3726105422 >> 172.21.15.27:0/3726105422 conn(0x7fb434073800 msgr2=0x7fb43415be50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:26.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.159+0000 7fb439a77640 1 -- 172.21.15.27:0/3726105422 shutdown_connections 2024-09-12T03:19:26.162 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.159+0000 7fb439a77640 1 -- 172.21.15.27:0/3726105422 wait complete. 2024-09-12T03:19:26.171 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-12T03:19:26.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.272+0000 7fa985dcf640 1 Processor -- start 2024-09-12T03:19:26.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.272+0000 7fa985dcf640 1 -- start start 2024-09-12T03:19:26.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.273+0000 7fa985dcf640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fa980145c30 0x7fa98015ca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:26.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.273+0000 7fa985dcf640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa9800cda40 0x7fa9800b4040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:26.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.273+0000 7fa985dcf640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa9800b4d70 con 0x7fa980146010 2024-09-12T03:19:26.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.273+0000 7fa985dcf640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7fa9800b5050 con 0x7fa980145c30 2024-09-12T03:19:26.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.273+0000 7fa985dcf640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fa98006d140 con 0x7fa9800cda40 2024-09-12T03:19:26.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.273+0000 7fa97f7fe640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fa980145c30 0x7fa98015ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:26.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.273+0000 7fa97f7fe640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fa980145c30 0x7fa98015ca00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42420/0 (socket says 172.21.15.27:42420) 2024-09-12T03:19:26.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.273+0000 7fa97f7fe640 1 -- 172.21.15.27:0/2824751223 learned_addr learned my addr 172.21.15.27:0/2824751223 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:19:26.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.273+0000 7fa97f7fe640 1 -- 172.21.15.27:0/2824751223 >> v2:172.21.15.45:3300/0 conn(0x7fa9800cda40 msgr2=0x7fa9800b4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:26.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.273+0000 7fa97f7fe640 1 --2- 172.21.15.27:0/2824751223 >> v2:172.21.15.45:3300/0 conn(0x7fa9800cda40 0x7fa9800b4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:26.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.273+0000 7fa97f7fe640 1 -- 172.21.15.27:0/2824751223 >> v1:172.21.15.45:6789/0 conn(0x7fa980146010 legacy=0x7fa9800cd330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:26.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.273+0000 7fa97f7fe640 1 -- 172.21.15.27:0/2824751223 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa98006d420 con 0x7fa980145c30 2024-09-12T03:19:26.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.274+0000 7fa97f7fe640 1 --2- 172.21.15.27:0/2824751223 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa980145c30 0x7fa98015ca00 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fa9800b3ef0 tx=0x7fa97002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c245f5bdf515bc4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:26.276 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.274+0000 7fa97e7fc640 1 -- 172.21.15.27:0/2824751223 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa970037070 con 0x7fa980145c30 2024-09-12T03:19:26.277 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.274+0000 7fa97e7fc640 1 -- 172.21.15.27:0/2824751223 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa970032440 con 0x7fa980145c30 2024-09-12T03:19:26.277 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.274+0000 7fa97e7fc640 1 -- 172.21.15.27:0/2824751223 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa970032810 con 0x7fa980145c30 2024-09-12T03:19:26.277 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.275+0000 7fa985dcf640 1 -- 172.21.15.27:0/2824751223 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa980145c30 msgr2=0x7fa98015ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:26.277 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.275+0000 7fa985dcf640 1 --2- 172.21.15.27:0/2824751223 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa980145c30 0x7fa98015ca00 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fa9800b3ef0 tx=0x7fa97002a1a0 comp rx=0 tx=0).stop 2024-09-12T03:19:26.277 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.275+0000 7fa985dcf640 1 -- 172.21.15.27:0/2824751223 shutdown_connections 2024-09-12T03:19:26.277 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.275+0000 7fa985dcf640 1 --2- 172.21.15.27:0/2824751223 >> v2:172.21.15.45:3300/0 conn(0x7fa9800cda40 0x7fa9800b4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:26.277 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.275+0000 7fa985dcf640 1 --2- 172.21.15.27:0/2824751223 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa980145c30 0x7fa98015ca00 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:26.277 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.275+0000 7fa985dcf640 1 -- 172.21.15.27:0/2824751223 >> 172.21.15.27:0/2824751223 conn(0x7fa980062c60 msgr2=0x7fa98006bb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:26.277 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.275+0000 7fa985dcf640 1 -- 172.21.15.27:0/2824751223 shutdown_connections 2024-09-12T03:19:26.277 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.275+0000 7fa985dcf640 1 -- 172.21.15.27:0/2824751223 wait complete. 2024-09-12T03:19:26.277 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.276+0000 7fa985dcf640 1 Processor -- start 2024-09-12T03:19:26.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.276+0000 7fa985dcf640 1 -- start start 2024-09-12T03:19:26.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.277+0000 7fa985dcf640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa9800cda40 0x7fa9801e7390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:26.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.277+0000 7fa985dcf640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa980145c30 0x7fa9800ac750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:26.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.277+0000 7fa985dcf640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa980146010 0x7fa98013df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:26.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.277+0000 7fa985dcf640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fa9800b4700 con 0x7fa9800cda40 2024-09-12T03:19:26.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.277+0000 7fa985dcf640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa9800b4580 con 0x7fa980145c30 2024-09-12T03:19:26.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.277+0000 7fa985dcf640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fa9800b4880 con 0x7fa980146010 2024-09-12T03:19:26.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.277+0000 7fa97f7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa9800cda40 0x7fa9801e7390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:26.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.277+0000 7fa97f7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa9800cda40 0x7fa9801e7390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42436/0 (socket says 172.21.15.27:42436) 2024-09-12T03:19:26.280 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.277+0000 7fa97f7fe640 1 -- 172.21.15.27:0/901247209 learned_addr learned my addr 172.21.15.27:0/901247209 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:19:26.280 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.277+0000 7fa97f7fe640 1 -- 172.21.15.27:0/901247209 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa980146010 msgr2=0x7fa98013df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:26.280 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.277+0000 7fa97f7fe640 1 --2- 172.21.15.27:0/901247209 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa980146010 0x7fa98013df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:26.280 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.277+0000 7fa97f7fe640 1 -- 172.21.15.27:0/901247209 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa980145c30 msgr2=0x7fa9800ac750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:26.280 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.277+0000 7fa97f7fe640 1 --2- 172.21.15.27:0/901247209 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa980145c30 0x7fa9800ac750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:26.280 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.277+0000 7fa97f7fe640 1 -- 172.21.15.27:0/901247209 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9800acc90 con 0x7fa9800cda40 2024-09-12T03:19:26.280 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.277+0000 7fa97f7fe640 1 --2- 172.21.15.27:0/901247209 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa9800cda40 0x7fa9801e7390 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fa970000c00 tx=0x7fa9700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:26.280 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.278+0000 7fa967fff640 1 -- 172.21.15.27:0/901247209 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa970041070 con 0x7fa9800cda40 2024-09-12T03:19:26.280 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.278+0000 7fa967fff640 1 -- 172.21.15.27:0/901247209 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa97003c870 con 0x7fa9800cda40 2024-09-12T03:19:26.280 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.278+0000 7fa985dcf640 1 -- 172.21.15.27:0/901247209 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9800acf30 con 0x7fa9800cda40 2024-09-12T03:19:26.280 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.278+0000 7fa967fff640 1 -- 172.21.15.27:0/901247209 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa970037040 con 0x7fa9800cda40 2024-09-12T03:19:26.280 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.278+0000 7fa985dcf640 1 -- 172.21.15.27:0/901247209 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9800ad4f0 con 0x7fa9800cda40 2024-09-12T03:19:26.281 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.279+0000 7fa985dcf640 1 -- 172.21.15.27:0/901247209 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9800b4040 con 0x7fa9800cda40 2024-09-12T03:19:26.281 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.279+0000 7fa967fff640 1 -- 172.21.15.27:0/901247209 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 9) ==== 90651+0+0 (secure 0 0 0) 0x7fa970044070 con 0x7fa9800cda40 2024-09-12T03:19:26.281 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.279+0000 7fa967fff640 1 --2- 172.21.15.27:0/901247209 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fa96c06c010 0x7fa96c06e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:26.281 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.280+0000 7fa967fff640 1 -- 172.21.15.27:0/901247209 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 5709+0+0 (secure 0 0 0) 0x7fa9700abe60 con 0x7fa9800cda40 2024-09-12T03:19:26.281 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.280+0000 7fa97effd640 1 --2- 172.21.15.27:0/901247209 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fa96c06c010 0x7fa96c06e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:26.282 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.280+0000 7fa97effd640 1 --2- 172.21.15.27:0/901247209 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fa96c06c010 0x7fa96c06e4d0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fa98013f5e0 tx=0x7fa974006ba0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:26.285 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.284+0000 7fa967fff640 1 -- 172.21.15.27:0/901247209 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fa97007a740 con 0x7fa9800cda40 2024-09-12T03:19:26.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.460+0000 7fa985dcf640 1 -- 172.21.15.27:0/901247209 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa980142e50 con 0x7fa9800cda40 2024-09-12T03:19:26.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.461+0000 7fa967fff640 1 -- 172.21.15.27:0/901247209 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v55) ==== 74+0+16690 (secure 0 0 0) 0x7fa97007a040 con 0x7fa9800cda40 2024-09-12T03:19:26.462 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:19:26.463 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":55,"fsid":"d2df504e-2647-4385-a5db-9fdf69f487a0","created":"2024-09-12T03:16:07.778937+0000","modified":"2024-09-12T03:19:25.200876+0000","last_up_change":"2024-09-12T03:16:19.314778+0000","last_in_change":"2024-09-12T03:16:09.893086+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T03:16:20.097646+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-09-12T03:19:23.556103+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-09-12T03:19:24.184337+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1851116c-1fb3-4dc9-9772-f90e2ef4c355","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6817","nonce":3002321488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6819","nonce":3002321488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6823","nonce":3002321488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6821","nonce":3002321488}]},"public_addr":"172.21.15.27:6817/3002321488","cluster_addr":"172.21.15.27:6819/3002321488","heartbeat_back_addr":"172.21.15.27:6823/3002321488","heartbeat_front_addr":"172.21.15.27:6821/3002321488","state":["exists","up"]},{"osd":1,"uuid":"f38ae17d-73eb-4204-b99d-975fa806fb87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6809","nonce":1613102886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6811","nonce":1613102886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6815","nonce":1613102886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6813","nonce":1613102886}]},"public_addr":"172.21.15.27:6809/1613102886","cluster_addr":"172.21.15.27:6811/1613102886","heartbeat_back_addr":"172.21.15.27:6815/1613102886","heartbeat_front_addr":"172.21.15.27:6813/1613102886","state":["exists","up"]},{"osd":2,"uuid":"fb12dcc8-bdd0-42b8-b46d-8b10106224bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6825","nonce":3534723355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6827","nonce":3534723355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6831","nonce":3534723355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6829","nonce":3534723355}]},"public_addr":"172.21.15.27:6825/3534723355","cluster_addr":"172.21.15.27:6827/3534723355","heartbeat_back_addr":"172.21.15.27:6831/3534723355","heartbeat_front_addr":"172.21.15.27:6829/3534723355","state":["exists","up"]},{"osd":3,"uuid":"d74d3f49-9c24-4b9f-98e2-3ecd7b5ef3b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6801","nonce":80661598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6803","nonce":80661598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6807","nonce":80661598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6805","nonce":80661598}]},"public_addr":"172.21.15.27:6801/80661598","cluster_addr":"172.21.15.27:6803/80661598","heartbeat_back_addr":"172.21.15.27:6807/80661598","heartbeat_front_addr":"172.21.15.27:6805/80661598","state":["exists","up"]},{"osd":4,"uuid":"2bf123f2-1b0d-41c4-aa3b-805bab267e59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6817","nonce":3943189827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6819","nonce":3943189827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6823","nonce":3943189827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6821","nonce":3943189827}]},"public_addr":"172.21.15.45:6817/3943189827","cluster_addr":"172.21.15.45:6819/3943189827","heartbeat_back_addr":"172.21.15.45:6823/3943189827","heartbeat_front_addr":"172.21.15.45:6821/3943189827","state":["exists","up"]},{"osd":5,"uuid":"c49ba5da-91c6-4a86-a97b-4aba89c253ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6809","nonce":3620760330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6811","nonce":3620760330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6815","nonce":3620760330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6813","nonce":3620760330}]},"public_addr":"172.21.15.45:6809/3620760330","cluster_addr":"172.21.15.45:6811/3620760330","heartbeat_back_addr":"172.21.15.45:6815/3620760330","heartbeat_front_addr":"172.21.15.45:6813/3620760330","state":["exists","up"]},{"osd":6,"uuid":"b0123475-89f7-4a34-9e33-d797b99dd9dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6801","nonce":1989083510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6803","nonce":1989083510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6807","nonce":1989083510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6805","nonce":1989083510}]},"public_addr":"172.21.15.45:6801/1989083510","cluster_addr":"172.21.15.45:6803/1989083510","heartbeat_back_addr":"172.21.15.45:6807/1989083510","heartbeat_front_addr":"172.21.15.45:6805/1989083510","state":["exists","up"]},{"osd":7,"uuid":"4dad773d-b7d8-46fc-8a10-a972aad27b29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6825","nonce":1513300294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6827","nonce":1513300294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6831","nonce":1513300294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6829","nonce":1513300294}]},"public_addr":"172.21.15.45:6825/1513300294","cluster_addr":"172.21.15.45:6827/1513300294","heartbeat_back_addr":"172.21.15.45:6831/1513300294","heartbeat_front_addr":"172.21.15.45:6829/1513300294","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.881855+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:17.133252+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.735339+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.500742+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.477571+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:17.167667+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.470291+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.873743+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T03:19:26.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.463+0000 7fa985dcf640 1 -- 172.21.15.27:0/901247209 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fa96c06c010 msgr2=0x7fa96c06e4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:26.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.463+0000 7fa985dcf640 1 --2- 172.21.15.27:0/901247209 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fa96c06c010 0x7fa96c06e4d0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fa98013f5e0 tx=0x7fa974006ba0 comp rx=0 tx=0).stop 2024-09-12T03:19:26.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.463+0000 7fa985dcf640 1 -- 172.21.15.27:0/901247209 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa9800cda40 msgr2=0x7fa9801e7390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:26.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.463+0000 7fa985dcf640 1 --2- 172.21.15.27:0/901247209 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa9800cda40 0x7fa9801e7390 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fa970000c00 tx=0x7fa9700046c0 comp rx=0 tx=0).stop 2024-09-12T03:19:26.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.463+0000 7fa985dcf640 1 -- 172.21.15.27:0/901247209 shutdown_connections 2024-09-12T03:19:26.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.463+0000 7fa985dcf640 1 --2- 172.21.15.27:0/901247209 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fa96c06c010 0x7fa96c06e4d0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:26.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.463+0000 7fa985dcf640 1 --2- 172.21.15.27:0/901247209 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa980146010 0x7fa98013df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:26.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.463+0000 7fa985dcf640 1 --2- 172.21.15.27:0/901247209 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa980145c30 0x7fa9800ac750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:26.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.463+0000 7fa985dcf640 1 --2- 172.21.15.27:0/901247209 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa9800cda40 0x7fa9801e7390 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:26.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.463+0000 7fa985dcf640 1 -- 172.21.15.27:0/901247209 >> 172.21.15.27:0/901247209 conn(0x7fa980062c60 msgr2=0x7fa9800b51d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:26.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.463+0000 7fa985dcf640 1 -- 172.21.15.27:0/901247209 shutdown_connections 2024-09-12T03:19:26.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.463+0000 7fa985dcf640 1 -- 172.21.15.27:0/901247209 wait complete. 2024-09-12T03:19:26.475 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-12T03:19:26.581 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.580+0000 7fb498b28640 1 Processor -- start 2024-09-12T03:19:26.581 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.580+0000 7fb498b28640 1 -- start start 2024-09-12T03:19:26.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.581+0000 7fb498b28640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fb49406dfd0 0x7fb4940d1da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:26.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.581+0000 7fb498b28640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fb4940d3790 0x7fb4940b7a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:26.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.581+0000 7fb498b28640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb4940d2e30 con 0x7fb494162040 2024-09-12T03:19:26.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.581+0000 7fb498b28640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7fb4940b7f90 con 0x7fb49406dfd0 2024-09-12T03:19:26.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.581+0000 7fb498b28640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fb494162420 con 0x7fb4940d3790 2024-09-12T03:19:26.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.581+0000 7fb491d9b640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fb49406dfd0 0x7fb4940d1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:26.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.581+0000 7fb49259c640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fb494162040 0x7fb49406d8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:49132/0 (socket says 172.21.15.27:49132) 2024-09-12T03:19:26.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.582+0000 7fb49259c640 1 -- 172.21.15.27:0/1989793624 learned_addr learned my addr 172.21.15.27:0/1989793624 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:19:26.584 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.582+0000 7fb492d9d640 1 --2- 172.21.15.27:0/1989793624 >> v2:172.21.15.45:3300/0 conn(0x7fb4940d3790 0x7fb4940b7a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:26.584 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.582+0000 7fb491d9b640 1 -- 172.21.15.27:0/1989793624 >> v2:172.21.15.45:3300/0 conn(0x7fb4940d3790 msgr2=0x7fb4940b7a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:26.584 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.582+0000 7fb491d9b640 1 --2- 172.21.15.27:0/1989793624 >> v2:172.21.15.45:3300/0 conn(0x7fb4940d3790 0x7fb4940b7a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:26.584 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.582+0000 7fb491d9b640 1 -- 172.21.15.27:0/1989793624 >> v1:172.21.15.27:6789/0 conn(0x7fb494162040 legacy=0x7fb49406d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:26.584 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.582+0000 7fb491d9b640 1 -- 172.21.15.27:0/1989793624 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4940d30d0 con 0x7fb49406dfd0 2024-09-12T03:19:26.584 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.582+0000 7fb49159a640 1 -- 172.21.15.27:0/1989793624 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2239335589 0 0) 0x7fb4940d2e30 con 0x7fb494162040 2024-09-12T03:19:26.585 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.583+0000 7fb491d9b640 1 --2- 172.21.15.27:0/1989793624 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb49406dfd0 0x7fb4940d1da0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fb4940d1c90 tx=0x7fb47c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86007bb27b726e18 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:26.586 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.583+0000 7fb49159a640 1 -- 172.21.15.27:0/1989793624 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb47c037070 con 0x7fb49406dfd0 2024-09-12T03:19:26.586 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.583+0000 7fb49159a640 1 -- 172.21.15.27:0/1989793624 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb47c032440 con 0x7fb49406dfd0 2024-09-12T03:19:26.586 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.583+0000 7fb49159a640 1 -- 172.21.15.27:0/1989793624 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb47c032810 con 0x7fb49406dfd0 2024-09-12T03:19:26.586 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.583+0000 7fb498b28640 1 -- 172.21.15.27:0/1989793624 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb49406dfd0 msgr2=0x7fb4940d1da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:26.586 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.583+0000 7fb498b28640 1 --2- 172.21.15.27:0/1989793624 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb49406dfd0 0x7fb4940d1da0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fb4940d1c90 tx=0x7fb47c02a1a0 comp rx=0 tx=0).stop 2024-09-12T03:19:26.586 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.583+0000 7fb498b28640 1 -- 172.21.15.27:0/1989793624 shutdown_connections 2024-09-12T03:19:26.586 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.583+0000 7fb498b28640 1 --2- 172.21.15.27:0/1989793624 >> v2:172.21.15.45:3300/0 conn(0x7fb4940d3790 0x7fb4940b7a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:26.586 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.583+0000 7fb498b28640 1 --2- 172.21.15.27:0/1989793624 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb49406dfd0 0x7fb4940d1da0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:26.586 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.583+0000 7fb498b28640 1 -- 172.21.15.27:0/1989793624 >> 172.21.15.27:0/1989793624 conn(0x7fb4940b6700 msgr2=0x7fb4940b6b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:26.586 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.583+0000 7fb498b28640 1 -- 172.21.15.27:0/1989793624 shutdown_connections 2024-09-12T03:19:26.586 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.583+0000 7fb498b28640 1 -- 172.21.15.27:0/1989793624 wait complete. 2024-09-12T03:19:26.586 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.585+0000 7fb498b28640 1 Processor -- start 2024-09-12T03:19:26.587 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.585+0000 7fb498b28640 1 -- start start 2024-09-12T03:19:26.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.586+0000 7fb498b28640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb49406dfd0 0x7fb4941f4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:26.589 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.586+0000 7fb498b28640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb4940d3790 0x7fb4940b9190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:26.589 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.586+0000 7fb498b28640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb494162040 0x7fb4940b96d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:26.589 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.586+0000 7fb498b28640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fb4940d4530 con 0x7fb4940d3790 2024-09-12T03:19:26.589 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.586+0000 7fb498b28640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb4940cee00 con 0x7fb494162040 2024-09-12T03:19:26.589 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.586+0000 7fb498b28640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fb4940d4150 con 0x7fb49406dfd0 2024-09-12T03:19:26.589 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.586+0000 7fb491d9b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb4940d3790 0x7fb4940b9190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:26.589 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.586+0000 7fb491d9b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb4940d3790 0x7fb4940b9190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42456/0 (socket says 172.21.15.27:42456) 2024-09-12T03:19:26.589 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.586+0000 7fb491d9b640 1 -- 172.21.15.27:0/1735359168 learned_addr learned my addr 172.21.15.27:0/1735359168 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:19:26.589 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.586+0000 7fb49259c640 1 --2- 172.21.15.27:0/1735359168 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb49406dfd0 0x7fb4941f4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:26.589 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.586+0000 7fb492d9d640 1 --2- 172.21.15.27:0/1735359168 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb494162040 0x7fb4940b96d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:26.589 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.586+0000 7fb491d9b640 1 -- 172.21.15.27:0/1735359168 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb49406dfd0 msgr2=0x7fb4941f4d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:26.590 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.586+0000 7fb491d9b640 1 --2- 172.21.15.27:0/1735359168 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb49406dfd0 0x7fb4941f4d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:26.590 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.586+0000 7fb491d9b640 1 -- 172.21.15.27:0/1735359168 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb494162040 msgr2=0x7fb4940b96d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:26.590 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.586+0000 7fb491d9b640 1 --2- 172.21.15.27:0/1735359168 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb494162040 0x7fb4940b96d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:26.590 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.586+0000 7fb491d9b640 1 -- 172.21.15.27:0/1735359168 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4940cb880 con 0x7fb4940d3790 2024-09-12T03:19:26.590 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.586+0000 7fb492d9d640 1 --2- 172.21.15.27:0/1735359168 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb494162040 0x7fb4940b96d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:19:26.590 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.586+0000 7fb491d9b640 1 --2- 172.21.15.27:0/1735359168 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb4940d3790 0x7fb4940b9190 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fb47c035040 tx=0x7fb47c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:26.590 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.586+0000 7fb48affd640 1 -- 172.21.15.27:0/1735359168 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb47c037050 con 0x7fb4940d3790 2024-09-12T03:19:26.590 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.586+0000 7fb48affd640 1 -- 172.21.15.27:0/1735359168 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb47c038040 con 0x7fb4940d3790 2024-09-12T03:19:26.590 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.586+0000 7fb498b28640 1 -- 172.21.15.27:0/1735359168 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4940cbb40 con 0x7fb4940d3790 2024-09-12T03:19:26.590 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.586+0000 7fb48affd640 1 -- 172.21.15.27:0/1735359168 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb47c006e10 con 0x7fb4940d3790 2024-09-12T03:19:26.590 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.587+0000 7fb498b28640 1 -- 172.21.15.27:0/1735359168 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4940cc100 con 0x7fb4940d3790 2024-09-12T03:19:26.590 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.587+0000 7fb498b28640 1 -- 172.21.15.27:0/1735359168 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb49406e6e0 con 0x7fb4940d3790 2024-09-12T03:19:26.591 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.588+0000 7fb48affd640 1 -- 172.21.15.27:0/1735359168 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 9) ==== 90651+0+0 (secure 0 0 0) 0x7fb47c030980 con 0x7fb4940d3790 2024-09-12T03:19:26.591 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.588+0000 7fb48affd640 1 --2- 172.21.15.27:0/1735359168 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fb47406bfe0 0x7fb47406e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:26.591 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.589+0000 7fb48affd640 1 -- 172.21.15.27:0/1735359168 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5709+0+0 (secure 0 0 0) 0x7fb47c0abbd0 con 0x7fb4940d3790 2024-09-12T03:19:26.591 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.589+0000 7fb49259c640 1 --2- 172.21.15.27:0/1735359168 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fb47406bfe0 0x7fb47406e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:26.592 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.589+0000 7fb49259c640 1 --2- 172.21.15.27:0/1735359168 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fb47406bfe0 0x7fb47406e4a0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fb48c0012e0 tx=0x7fb48c001630 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:26.593 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.593+0000 7fb48affd640 1 -- 172.21.15.27:0/1735359168 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fb47c07a420 con 0x7fb4940d3790 2024-09-12T03:19:26.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.803+0000 7fb498b28640 1 -- 172.21.15.27:0/1735359168 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fb4940cfe40 con 0x7fb4940d3790 2024-09-12T03:19:26.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.847+0000 7fb48affd640 1 -- 172.21.15.27:0/1735359168 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v29) ==== 118+0+0 (secure 0 0 0) 0x7fb47c079d20 con 0x7fb4940d3790 2024-09-12T03:19:26.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.849+0000 7fb498b28640 1 -- 172.21.15.27:0/1735359168 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fb47406bfe0 msgr2=0x7fb47406e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:26.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.849+0000 7fb498b28640 1 --2- 172.21.15.27:0/1735359168 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fb47406bfe0 0x7fb47406e4a0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fb48c0012e0 tx=0x7fb48c001630 comp rx=0 tx=0).stop 2024-09-12T03:19:26.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.849+0000 7fb498b28640 1 -- 172.21.15.27:0/1735359168 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb4940d3790 msgr2=0x7fb4940b9190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:26.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.849+0000 7fb498b28640 1 --2- 172.21.15.27:0/1735359168 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb4940d3790 0x7fb4940b9190 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fb47c035040 tx=0x7fb47c0046c0 comp rx=0 tx=0).stop 2024-09-12T03:19:26.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.849+0000 7fb498b28640 1 -- 172.21.15.27:0/1735359168 shutdown_connections 2024-09-12T03:19:26.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.849+0000 7fb498b28640 1 --2- 172.21.15.27:0/1735359168 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fb47406bfe0 0x7fb47406e4a0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:26.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.849+0000 7fb498b28640 1 --2- 172.21.15.27:0/1735359168 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb494162040 0x7fb4940b96d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:26.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.849+0000 7fb498b28640 1 --2- 172.21.15.27:0/1735359168 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb4940d3790 0x7fb4940b9190 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:26.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.849+0000 7fb498b28640 1 --2- 172.21.15.27:0/1735359168 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb49406dfd0 0x7fb4941f4d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:26.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.849+0000 7fb498b28640 1 -- 172.21.15.27:0/1735359168 >> 172.21.15.27:0/1735359168 conn(0x7fb4940b6700 msgr2=0x7fb4940cf170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:26.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.849+0000 7fb498b28640 1 -- 172.21.15.27:0/1735359168 shutdown_connections 2024-09-12T03:19:26.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.849+0000 7fb498b28640 1 -- 172.21.15.27:0/1735359168 wait complete. 2024-09-12T03:19:26.861 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-12T03:19:26.861 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-12T03:19:26.967 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.966+0000 7f9de25e4640 1 Processor -- start 2024-09-12T03:19:26.967 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.966+0000 7f9de25e4640 1 -- start start 2024-09-12T03:19:26.968 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.967+0000 7f9de25e4640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f9ddc15e900 0x7f9ddc0af370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:26.968 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.967+0000 7f9de25e4640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f9ddc06d730 0x7f9ddc0b5e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:26.968 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.967+0000 7f9de25e4640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f9ddc0b6370 0x7f9ddc0b4600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:26.968 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.967+0000 7f9de25e4640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f9ddc15ece0 con 0x7f9ddc06d730 2024-09-12T03:19:26.968 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.967+0000 7f9de25e4640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f9ddc0b5940 con 0x7f9ddc0b6370 2024-09-12T03:19:26.969 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.967+0000 7f9de25e4640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f9ddc0af950 con 0x7f9ddc15e900 2024-09-12T03:19:26.969 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.967+0000 7f9ddb7fe640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f9ddc06d730 0x7f9ddc0b5e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:26.969 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.967+0000 7f9ddb7fe640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f9ddc06d730 0x7f9ddc0b5e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42466/0 (socket says 172.21.15.27:42466) 2024-09-12T03:19:26.969 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.967+0000 7f9ddb7fe640 1 -- 172.21.15.27:0/339209947 learned_addr learned my addr 172.21.15.27:0/339209947 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:19:26.969 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.967+0000 7f9ddbfff640 1 --2- 172.21.15.27:0/339209947 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f9ddc15e900 0x7f9ddc0af370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:26.969 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.967+0000 7f9de0b81640 1 --2- 172.21.15.27:0/339209947 >> v2:172.21.15.45:3300/0 conn(0x7f9ddc0b6370 0x7f9ddc0b4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:26.969 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.967+0000 7f9ddb7fe640 1 -- 172.21.15.27:0/339209947 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f9ddc15e900 msgr2=0x7f9ddc0af370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:26.969 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.967+0000 7f9ddb7fe640 1 --2- 172.21.15.27:0/339209947 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f9ddc15e900 0x7f9ddc0af370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:26.969 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.967+0000 7f9ddb7fe640 1 -- 172.21.15.27:0/339209947 >> v2:172.21.15.45:3300/0 conn(0x7f9ddc0b6370 msgr2=0x7f9ddc0b4600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:26.969 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.967+0000 7f9ddb7fe640 1 --2- 172.21.15.27:0/339209947 >> v2:172.21.15.45:3300/0 conn(0x7f9ddc0b6370 0x7f9ddc0b4600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:26.969 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.967+0000 7f9ddb7fe640 1 -- 172.21.15.27:0/339209947 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ddc0afc30 con 0x7f9ddc06d730 2024-09-12T03:19:26.971 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.968+0000 7f9ddb7fe640 1 --2- 172.21.15.27:0/339209947 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9ddc06d730 0x7f9ddc0b5e30 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f9ddc0aeff0 tx=0x7f9dd002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38442dfaa73d7892 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:26.971 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.968+0000 7f9ddaffd640 1 -- 172.21.15.27:0/339209947 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dd0037070 con 0x7f9ddc06d730 2024-09-12T03:19:26.971 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.968+0000 7f9ddaffd640 1 -- 172.21.15.27:0/339209947 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9dd0032440 con 0x7f9ddc06d730 2024-09-12T03:19:26.971 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.968+0000 7f9ddaffd640 1 -- 172.21.15.27:0/339209947 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dd0032810 con 0x7f9ddc06d730 2024-09-12T03:19:26.971 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.969+0000 7f9de25e4640 1 -- 172.21.15.27:0/339209947 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9ddc06d730 msgr2=0x7f9ddc0b5e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:26.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.969+0000 7f9de25e4640 1 --2- 172.21.15.27:0/339209947 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9ddc06d730 0x7f9ddc0b5e30 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f9ddc0aeff0 tx=0x7f9dd002a1e0 comp rx=0 tx=0).stop 2024-09-12T03:19:26.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.969+0000 7f9de25e4640 1 -- 172.21.15.27:0/339209947 shutdown_connections 2024-09-12T03:19:26.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.969+0000 7f9de25e4640 1 --2- 172.21.15.27:0/339209947 >> v2:172.21.15.45:3300/0 conn(0x7f9ddc0b6370 0x7f9ddc0b4600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:26.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.969+0000 7f9de25e4640 1 --2- 172.21.15.27:0/339209947 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9ddc06d730 0x7f9ddc0b5e30 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:26.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.969+0000 7f9de25e4640 1 --2- 172.21.15.27:0/339209947 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f9ddc15e900 0x7f9ddc0af370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:26.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.969+0000 7f9de25e4640 1 -- 172.21.15.27:0/339209947 >> 172.21.15.27:0/339209947 conn(0x7f9ddc06cc40 msgr2=0x7f9ddc06d020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:26.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.969+0000 7f9de25e4640 1 -- 172.21.15.27:0/339209947 shutdown_connections 2024-09-12T03:19:26.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.969+0000 7f9de25e4640 1 -- 172.21.15.27:0/339209947 wait complete. 2024-09-12T03:19:26.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.970+0000 7f9de25e4640 1 Processor -- start 2024-09-12T03:19:26.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.970+0000 7f9de25e4640 1 -- start start 2024-09-12T03:19:26.976 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.971+0000 7f9de25e4640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f9ddc06d730 0x7f9ddc14cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:26.976 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.971+0000 7f9de25e4640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9ddc0b6370 0x7f9ddc152d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:26.976 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.971+0000 7f9de25e4640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9ddc15e900 0x7f9ddc161240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:26.976 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.971+0000 7f9de25e4640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f9ddc06e880 con 0x7f9ddc15e900 2024-09-12T03:19:26.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.971+0000 7f9de25e4640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f9ddc0b01f0 con 0x7f9ddc0b6370 2024-09-12T03:19:26.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.971+0000 7f9de25e4640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f9ddc06ea00 con 0x7f9ddc06d730 2024-09-12T03:19:26.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.971+0000 7f9de0b81640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9ddc15e900 0x7f9ddc161240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:26.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.971+0000 7f9de0b81640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9ddc15e900 0x7f9ddc161240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42474/0 (socket says 172.21.15.27:42474) 2024-09-12T03:19:26.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.971+0000 7f9de0b81640 1 -- 172.21.15.27:0/649195091 learned_addr learned my addr 172.21.15.27:0/649195091 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:19:26.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.971+0000 7f9ddbfff640 1 --2- 172.21.15.27:0/649195091 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f9ddc06d730 0x7f9ddc14cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:26.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.971+0000 7f9ddb7fe640 1 --2- 172.21.15.27:0/649195091 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9ddc0b6370 0x7f9ddc152d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:26.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.971+0000 7f9de0b81640 1 -- 172.21.15.27:0/649195091 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f9ddc06d730 msgr2=0x7f9ddc14cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:26.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.971+0000 7f9de0b81640 1 --2- 172.21.15.27:0/649195091 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f9ddc06d730 0x7f9ddc14cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:26.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.971+0000 7f9de0b81640 1 -- 172.21.15.27:0/649195091 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9ddc0b6370 msgr2=0x7f9ddc152d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:26.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.971+0000 7f9de0b81640 1 --2- 172.21.15.27:0/649195091 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9ddc0b6370 0x7f9ddc152d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:26.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.971+0000 7f9de0b81640 1 -- 172.21.15.27:0/649195091 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ddc156850 con 0x7f9ddc15e900 2024-09-12T03:19:26.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.971+0000 7f9ddb7fe640 1 --2- 172.21.15.27:0/649195091 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9ddc0b6370 0x7f9ddc152d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:19:26.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.971+0000 7f9ddbfff640 1 --2- 172.21.15.27:0/649195091 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f9ddc06d730 0x7f9ddc14cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:19:26.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.971+0000 7f9de0b81640 1 --2- 172.21.15.27:0/649195091 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9ddc15e900 0x7f9ddc161240 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f9dd4007e60 tx=0x7f9dd400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:26.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.972+0000 7f9dd8ff9640 1 -- 172.21.15.27:0/649195091 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dd4014620 con 0x7f9ddc15e900 2024-09-12T03:19:26.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.972+0000 7f9dd8ff9640 1 -- 172.21.15.27:0/649195091 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9dd4019070 con 0x7f9ddc15e900 2024-09-12T03:19:26.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.972+0000 7f9de25e4640 1 -- 172.21.15.27:0/649195091 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ddc156b30 con 0x7f9ddc15e900 2024-09-12T03:19:26.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.972+0000 7f9dd8ff9640 1 -- 172.21.15.27:0/649195091 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dd4014a20 con 0x7f9ddc15e900 2024-09-12T03:19:26.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.972+0000 7f9de25e4640 1 -- 172.21.15.27:0/649195091 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ddc1570f0 con 0x7f9ddc15e900 2024-09-12T03:19:26.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.973+0000 7f9de25e4640 1 -- 172.21.15.27:0/649195091 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ddc0b5e30 con 0x7f9ddc15e900 2024-09-12T03:19:26.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.973+0000 7f9dd8ff9640 1 -- 172.21.15.27:0/649195091 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 9) ==== 90651+0+0 (secure 0 0 0) 0x7f9dd4014bc0 con 0x7f9ddc15e900 2024-09-12T03:19:26.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.973+0000 7f9dd8ff9640 1 --2- 172.21.15.27:0/649195091 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f9dc806c010 0x7f9dc806e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:26.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.974+0000 7f9dd8ff9640 1 -- 172.21.15.27:0/649195091 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5709+0+0 (secure 0 0 0) 0x7f9dd408b9e0 con 0x7f9ddc15e900 2024-09-12T03:19:26.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.974+0000 7f9ddbfff640 1 --2- 172.21.15.27:0/649195091 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f9dc806c010 0x7f9dc806e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:26.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.974+0000 7f9ddbfff640 1 --2- 172.21.15.27:0/649195091 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f9dc806c010 0x7f9dc806e4d0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f9dcc006fd0 tx=0x7f9dcc008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:26.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:26.978+0000 7f9dd8ff9640 1 -- 172.21.15.27:0/649195091 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f9dd405a110 con 0x7f9ddc15e900 2024-09-12T03:19:27.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:27.188+0000 7f9de25e4640 1 -- 172.21.15.27:0/649195091 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f9ddc06fca0 con 0x7f9ddc15e900 2024-09-12T03:19:27.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:27.848+0000 7f9dd8ff9640 1 -- 172.21.15.27:0/649195091 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v30) ==== 122+0+0 (secure 0 0 0) 0x7f9dd4059a10 con 0x7f9ddc15e900 2024-09-12T03:19:27.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:27.852+0000 7f9de25e4640 1 -- 172.21.15.27:0/649195091 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f9dc806c010 msgr2=0x7f9dc806e4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:27.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:27.852+0000 7f9de25e4640 1 --2- 172.21.15.27:0/649195091 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f9dc806c010 0x7f9dc806e4d0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f9dcc006fd0 tx=0x7f9dcc008040 comp rx=0 tx=0).stop 2024-09-12T03:19:27.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:27.852+0000 7f9de25e4640 1 -- 172.21.15.27:0/649195091 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9ddc15e900 msgr2=0x7f9ddc161240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:27.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:27.852+0000 7f9de25e4640 1 --2- 172.21.15.27:0/649195091 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9ddc15e900 0x7f9ddc161240 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f9dd4007e60 tx=0x7f9dd400e4b0 comp rx=0 tx=0).stop 2024-09-12T03:19:27.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:27.852+0000 7f9de25e4640 1 -- 172.21.15.27:0/649195091 shutdown_connections 2024-09-12T03:19:27.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:27.852+0000 7f9de25e4640 1 --2- 172.21.15.27:0/649195091 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f9dc806c010 0x7f9dc806e4d0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:27.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:27.852+0000 7f9de25e4640 1 --2- 172.21.15.27:0/649195091 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9ddc15e900 0x7f9ddc161240 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:27.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:27.852+0000 7f9de25e4640 1 --2- 172.21.15.27:0/649195091 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9ddc0b6370 0x7f9ddc152d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:27.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:27.852+0000 7f9de25e4640 1 --2- 172.21.15.27:0/649195091 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f9ddc06d730 0x7f9ddc14cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:27.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:27.852+0000 7f9de25e4640 1 -- 172.21.15.27:0/649195091 >> 172.21.15.27:0/649195091 conn(0x7f9ddc06cc40 msgr2=0x7f9ddc0b2300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:27.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:27.852+0000 7f9de25e4640 1 -- 172.21.15.27:0/649195091 shutdown_connections 2024-09-12T03:19:27.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:27.852+0000 7f9de25e4640 1 -- 172.21.15.27:0/649195091 wait complete. 2024-09-12T03:19:27.866 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-12T03:19:27.867 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-12T03:19:27.974 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:27.973+0000 7ffb5c0c9640 1 Processor -- start 2024-09-12T03:19:27.974 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:27.973+0000 7ffb5c0c9640 1 -- start start 2024-09-12T03:19:27.975 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:27.974+0000 7ffb5c0c9640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ffb5415a8c0 0x7ffb540cdb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:27.975 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:27.974+0000 7ffb5c0c9640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ffb5415aca0 0x7ffb540b4710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:27.975 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:27.974+0000 7ffb5c0c9640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffb5406daf0 con 0x7ffb54071be0 2024-09-12T03:19:27.976 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:27.974+0000 7ffb5c0c9640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7ffb5406d8a0 con 0x7ffb5415a8c0 2024-09-12T03:19:27.976 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:27.974+0000 7ffb5c0c9640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7ffb5406ddd0 con 0x7ffb5415aca0 2024-09-12T03:19:27.976 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:27.974+0000 7ffb5a666640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ffb5415aca0 0x7ffb540b4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:27.976 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:27.974+0000 7ffb59e65640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ffb5415a8c0 0x7ffb540cdb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:27.976 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:27.975+0000 7ffb5a666640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ffb5415aca0 0x7ffb540b4710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3301/0 says I am v2:172.21.15.27:34670/0 (socket says 172.21.15.27:34670) 2024-09-12T03:19:27.976 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:27.975+0000 7ffb59664640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7ffb54071be0 0x7ffb5406e8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.27:52086/0 (socket says 172.21.15.27:52086) 2024-09-12T03:19:27.976 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:27.975+0000 7ffb5a666640 1 -- 172.21.15.27:0/620477669 learned_addr learned my addr 172.21.15.27:0/620477669 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:19:27.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:27.975+0000 7ffb58e63640 1 -- 172.21.15.27:0/620477669 <== mon.1 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3452891077 0 0) 0x7ffb5406daf0 con 0x7ffb54071be0 2024-09-12T03:19:27.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:27.975+0000 7ffb58e63640 1 -- 172.21.15.27:0/620477669 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ffb3c003520 con 0x7ffb54071be0 2024-09-12T03:19:27.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:27.975+0000 7ffb5a666640 1 -- 172.21.15.27:0/620477669 >> v2:172.21.15.45:3300/0 conn(0x7ffb5415a8c0 msgr2=0x7ffb540cdb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:27.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:27.975+0000 7ffb5a666640 1 --2- 172.21.15.27:0/620477669 >> v2:172.21.15.45:3300/0 conn(0x7ffb5415a8c0 0x7ffb540cdb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:27.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:27.975+0000 7ffb5a666640 1 -- 172.21.15.27:0/620477669 >> v1:172.21.15.45:6789/0 conn(0x7ffb54071be0 legacy=0x7ffb5406e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:27.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:27.975+0000 7ffb5a666640 1 -- 172.21.15.27:0/620477669 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb5406e0b0 con 0x7ffb5415aca0 2024-09-12T03:19:27.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:27.975+0000 7ffb59e65640 1 --2- 172.21.15.27:0/620477669 >> v2:172.21.15.45:3300/0 conn(0x7ffb5415a8c0 0x7ffb540cdb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:19:27.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:27.976+0000 7ffb5a666640 1 --2- 172.21.15.27:0/620477669 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ffb5415aca0 0x7ffb540b4710 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ffb5406da20 tx=0x7ffb4c02a250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8f0a9689441a3e84 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:27.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:27.976+0000 7ffb58e63640 1 -- 172.21.15.27:0/620477669 <== mon.2 v2:172.21.15.45:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb4c037070 con 0x7ffb5415aca0 2024-09-12T03:19:27.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:27.976+0000 7ffb58e63640 1 -- 172.21.15.27:0/620477669 <== mon.2 v2:172.21.15.45:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb4c032440 con 0x7ffb5415aca0 2024-09-12T03:19:27.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:27.976+0000 7ffb58e63640 1 -- 172.21.15.27:0/620477669 <== mon.2 v2:172.21.15.45:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb4c0327f0 con 0x7ffb5415aca0 2024-09-12T03:19:27.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:27.976+0000 7ffb5c0c9640 1 -- 172.21.15.27:0/620477669 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ffb5415aca0 msgr2=0x7ffb540b4710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:27.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:27.976+0000 7ffb5c0c9640 1 --2- 172.21.15.27:0/620477669 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ffb5415aca0 0x7ffb540b4710 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ffb5406da20 tx=0x7ffb4c02a250 comp rx=0 tx=0).stop 2024-09-12T03:19:27.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:27.977+0000 7ffb5c0c9640 1 -- 172.21.15.27:0/620477669 shutdown_connections 2024-09-12T03:19:27.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:27.977+0000 7ffb5c0c9640 1 --2- 172.21.15.27:0/620477669 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ffb5415aca0 0x7ffb540b4710 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:27.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:27.977+0000 7ffb5c0c9640 1 --2- 172.21.15.27:0/620477669 >> v2:172.21.15.45:3300/0 conn(0x7ffb5415a8c0 0x7ffb540cdb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:27.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:27.977+0000 7ffb5c0c9640 1 -- 172.21.15.27:0/620477669 >> 172.21.15.27:0/620477669 conn(0x7ffb54062c60 msgr2=0x7ffb5406bb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:27.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:27.977+0000 7ffb5c0c9640 1 -- 172.21.15.27:0/620477669 shutdown_connections 2024-09-12T03:19:27.980 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:27.977+0000 7ffb5c0c9640 1 -- 172.21.15.27:0/620477669 wait complete. 2024-09-12T03:19:27.982 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:27.978+0000 7ffb5c0c9640 1 Processor -- start 2024-09-12T03:19:27.983 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:27.978+0000 7ffb5c0c9640 1 -- start start 2024-09-12T03:19:27.983 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:27.979+0000 7ffb5c0c9640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ffb54071be0 0x7ffb541f1d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:27.983 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:27.979+0000 7ffb5c0c9640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffb5415a8c0 0x7ffb541f4b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:27.983 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:27.979+0000 7ffb5c0c9640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffb5415aca0 0x7ffb540c5ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:27.983 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:27.979+0000 7ffb5c0c9640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7ffb5406f140 con 0x7ffb5415a8c0 2024-09-12T03:19:27.983 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:27.979+0000 7ffb5c0c9640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ffb5406efc0 con 0x7ffb5415aca0 2024-09-12T03:19:27.983 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:27.979+0000 7ffb5c0c9640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7ffb540b3560 con 0x7ffb54071be0 2024-09-12T03:19:27.984 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:27.979+0000 7ffb59664640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffb5415a8c0 0x7ffb541f4b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:27.984 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:27.979+0000 7ffb59664640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffb5415a8c0 0x7ffb541f4b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42482/0 (socket says 172.21.15.27:42482) 2024-09-12T03:19:27.984 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:27.979+0000 7ffb59664640 1 -- 172.21.15.27:0/3576609001 learned_addr learned my addr 172.21.15.27:0/3576609001 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:19:27.984 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:27.979+0000 7ffb59e65640 1 --2- 172.21.15.27:0/3576609001 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ffb54071be0 0x7ffb541f1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:27.984 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:27.979+0000 7ffb5a666640 1 --2- 172.21.15.27:0/3576609001 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffb5415aca0 0x7ffb540c5ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:27.984 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:27.979+0000 7ffb59664640 1 -- 172.21.15.27:0/3576609001 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ffb54071be0 msgr2=0x7ffb541f1d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:27.984 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:27.979+0000 7ffb59664640 1 --2- 172.21.15.27:0/3576609001 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ffb54071be0 0x7ffb541f1d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:27.984 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:27.979+0000 7ffb59664640 1 -- 172.21.15.27:0/3576609001 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffb5415aca0 msgr2=0x7ffb540c5ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:27.984 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:27.979+0000 7ffb59664640 1 --2- 172.21.15.27:0/3576609001 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffb5415aca0 0x7ffb540c5ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:27.984 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:27.979+0000 7ffb59664640 1 -- 172.21.15.27:0/3576609001 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb541f5060 con 0x7ffb5415a8c0 2024-09-12T03:19:27.984 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:27.979+0000 7ffb59664640 1 --2- 172.21.15.27:0/3576609001 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffb5415a8c0 0x7ffb541f4b20 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7ffb4800c390 tx=0x7ffb4800c790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:27.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:27.979+0000 7ffb527fc640 1 -- 172.21.15.27:0/3576609001 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb480074e0 con 0x7ffb5415a8c0 2024-09-12T03:19:27.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:27.979+0000 7ffb527fc640 1 -- 172.21.15.27:0/3576609001 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb480047e0 con 0x7ffb5415a8c0 2024-09-12T03:19:27.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:27.979+0000 7ffb5c0c9640 1 -- 172.21.15.27:0/3576609001 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb540ad5c0 con 0x7ffb5415a8c0 2024-09-12T03:19:27.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:27.979+0000 7ffb527fc640 1 -- 172.21.15.27:0/3576609001 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb48004bd0 con 0x7ffb5415a8c0 2024-09-12T03:19:27.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:27.980+0000 7ffb5c0c9640 1 -- 172.21.15.27:0/3576609001 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb540adb80 con 0x7ffb5415a8c0 2024-09-12T03:19:27.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:27.980+0000 7ffb5c0c9640 1 -- 172.21.15.27:0/3576609001 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb54072ba0 con 0x7ffb5415a8c0 2024-09-12T03:19:27.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:27.981+0000 7ffb527fc640 1 -- 172.21.15.27:0/3576609001 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 9) ==== 90651+0+0 (secure 0 0 0) 0x7ffb4801a020 con 0x7ffb5415a8c0 2024-09-12T03:19:27.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:27.981+0000 7ffb527fc640 1 --2- 172.21.15.27:0/3576609001 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7ffb4006bfe0 0x7ffb4006e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:27.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:27.982+0000 7ffb527fc640 1 -- 172.21.15.27:0/3576609001 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5709+0+0 (secure 0 0 0) 0x7ffb4808a860 con 0x7ffb5415a8c0 2024-09-12T03:19:27.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:27.982+0000 7ffb59e65640 1 --2- 172.21.15.27:0/3576609001 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7ffb4006bfe0 0x7ffb4006e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:27.986 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:27.982+0000 7ffb59e65640 1 --2- 172.21.15.27:0/3576609001 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7ffb4006bfe0 0x7ffb4006e4a0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7ffb4400ace0 tx=0x7ffb4400a860 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:27.986 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:27.986+0000 7ffb527fc640 1 -- 172.21.15.27:0/3576609001 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7ffb48059050 con 0x7ffb5415a8c0 2024-09-12T03:19:28.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.195+0000 7ffb5c0c9640 1 -- 172.21.15.27:0/3576609001 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ffb54075440 con 0x7ffb5415a8c0 2024-09-12T03:19:28.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.196+0000 7ffb527fc640 1 -- 172.21.15.27:0/3576609001 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+5220 (secure 0 0 0) 0x7ffb4808a090 con 0x7ffb5415a8c0 2024-09-12T03:19:28.197 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:19:28.197 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":30,"btime":"2024-09-12T03:19:27:848915+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5102,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/565788762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":565788762},{"type":"v1","addr":"172.21.15.27:6835","nonce":565788762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26},{"gid":5413,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6836/2689309305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":2689309305},{"type":"v1","addr":"172.21.15.45:6836","nonce":2689309305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":5640,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6834/3694936463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":3694936463},{"type":"v1","addr":"172.21.15.27:6834","nonce":3694936463}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":30,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-12T03:19:25.201174+0000","modified":"2024-09-12T03:19:27.190661+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5649},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5649":{"gid":5649,"name":"b","rank":0,"incarnation":28,"state":"up:active","state_seq":3,"addr":"172.21.15.45:6837/3720118075","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6835","nonce":3720118075},{"type":"v1","addr":"172.21.15.45:6837","nonce":3720118075}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5649,"qdb_cluster":[5649]},"id":4}]} 2024-09-12T03:19:28.197 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 30 2024-09-12T03:19:28.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.197+0000 7ffb5c0c9640 1 -- 172.21.15.27:0/3576609001 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7ffb4006bfe0 msgr2=0x7ffb4006e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:28.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.197+0000 7ffb5c0c9640 1 --2- 172.21.15.27:0/3576609001 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7ffb4006bfe0 0x7ffb4006e4a0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7ffb4400ace0 tx=0x7ffb4400a860 comp rx=0 tx=0).stop 2024-09-12T03:19:28.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.198+0000 7ffb5c0c9640 1 -- 172.21.15.27:0/3576609001 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffb5415a8c0 msgr2=0x7ffb541f4b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:28.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.198+0000 7ffb5c0c9640 1 --2- 172.21.15.27:0/3576609001 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffb5415a8c0 0x7ffb541f4b20 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7ffb4800c390 tx=0x7ffb4800c790 comp rx=0 tx=0).stop 2024-09-12T03:19:28.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.198+0000 7ffb5c0c9640 1 -- 172.21.15.27:0/3576609001 shutdown_connections 2024-09-12T03:19:28.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.198+0000 7ffb5c0c9640 1 --2- 172.21.15.27:0/3576609001 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7ffb4006bfe0 0x7ffb4006e4a0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:28.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.198+0000 7ffb5c0c9640 1 --2- 172.21.15.27:0/3576609001 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffb5415aca0 0x7ffb540c5ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:28.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.198+0000 7ffb5c0c9640 1 --2- 172.21.15.27:0/3576609001 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffb5415a8c0 0x7ffb541f4b20 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:28.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.198+0000 7ffb5c0c9640 1 --2- 172.21.15.27:0/3576609001 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ffb54071be0 0x7ffb541f1d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:28.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.198+0000 7ffb5c0c9640 1 -- 172.21.15.27:0/3576609001 >> 172.21.15.27:0/3576609001 conn(0x7ffb54062c60 msgr2=0x7ffb5406d010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:28.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.198+0000 7ffb5c0c9640 1 -- 172.21.15.27:0/3576609001 shutdown_connections 2024-09-12T03:19:28.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.198+0000 7ffb5c0c9640 1 -- 172.21.15.27:0/3576609001 wait complete. 2024-09-12T03:19:28.211 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-12T03:19:28.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.312+0000 7f7db15d8640 1 Processor -- start 2024-09-12T03:19:28.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.312+0000 7f7db15d8640 1 -- start start 2024-09-12T03:19:28.314 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.313+0000 7f7db15d8640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f7dac1590d0 0x7f7dac0b55d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:28.314 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.313+0000 7f7db15d8640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f7dac15f600 0x7f7dac06bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:28.314 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.313+0000 7f7db15d8640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7dac162c10 con 0x7f7dac158cf0 2024-09-12T03:19:28.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.313+0000 7f7db15d8640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f7dac162ef0 con 0x7f7dac1590d0 2024-09-12T03:19:28.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.313+0000 7f7db15d8640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f7dac1631d0 con 0x7f7dac15f600 2024-09-12T03:19:28.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.313+0000 7f7daa7fc640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f7dac1590d0 0x7f7dac0b55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:28.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.313+0000 7f7daaffd640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f7dac158cf0 0x7f7dac162430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:49134/0 (socket says 172.21.15.27:49134) 2024-09-12T03:19:28.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.313+0000 7f7daa7fc640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f7dac1590d0 0x7f7dac0b55d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42488/0 (socket says 172.21.15.27:42488) 2024-09-12T03:19:28.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.313+0000 7f7daaffd640 1 -- 172.21.15.27:0/1543598377 learned_addr learned my addr 172.21.15.27:0/1543598377 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:19:28.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.313+0000 7f7dab7fe640 1 --2- 172.21.15.27:0/1543598377 >> v2:172.21.15.45:3300/0 conn(0x7f7dac15f600 0x7f7dac06bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:28.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.313+0000 7f7daa7fc640 1 -- 172.21.15.27:0/1543598377 >> v2:172.21.15.45:3300/0 conn(0x7f7dac15f600 msgr2=0x7f7dac06bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:28.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.313+0000 7f7daa7fc640 1 --2- 172.21.15.27:0/1543598377 >> v2:172.21.15.45:3300/0 conn(0x7f7dac15f600 0x7f7dac06bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:28.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.313+0000 7f7daa7fc640 1 -- 172.21.15.27:0/1543598377 >> v1:172.21.15.27:6789/0 conn(0x7f7dac158cf0 legacy=0x7f7dac162430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:28.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.313+0000 7f7daa7fc640 1 -- 172.21.15.27:0/1543598377 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7dac15edc0 con 0x7f7dac1590d0 2024-09-12T03:19:28.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.313+0000 7f7dab7fe640 1 --2- 172.21.15.27:0/1543598377 >> v2:172.21.15.45:3300/0 conn(0x7f7dac15f600 0x7f7dac06bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:19:28.317 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.314+0000 7f7daa7fc640 1 --2- 172.21.15.27:0/1543598377 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7dac1590d0 0x7f7dac0b55d0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f7dac0b1240 tx=0x7f7da002a080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7613ec58d3b59e17 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:28.317 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.314+0000 7f7da9ffb640 1 -- 172.21.15.27:0/1543598377 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7da0037070 con 0x7f7dac1590d0 2024-09-12T03:19:28.317 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.314+0000 7f7da9ffb640 1 -- 172.21.15.27:0/1543598377 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7da002add0 con 0x7f7dac1590d0 2024-09-12T03:19:28.317 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.314+0000 7f7da9ffb640 1 -- 172.21.15.27:0/1543598377 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7da0032660 con 0x7f7dac1590d0 2024-09-12T03:19:28.317 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.315+0000 7f7db15d8640 1 -- 172.21.15.27:0/1543598377 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7dac1590d0 msgr2=0x7f7dac0b55d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:28.317 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.315+0000 7f7db15d8640 1 --2- 172.21.15.27:0/1543598377 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7dac1590d0 0x7f7dac0b55d0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f7dac0b1240 tx=0x7f7da002a080 comp rx=0 tx=0).stop 2024-09-12T03:19:28.318 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.315+0000 7f7db15d8640 1 -- 172.21.15.27:0/1543598377 shutdown_connections 2024-09-12T03:19:28.318 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.315+0000 7f7db15d8640 1 --2- 172.21.15.27:0/1543598377 >> v2:172.21.15.45:3300/0 conn(0x7f7dac15f600 0x7f7dac06bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:28.318 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.315+0000 7f7db15d8640 1 --2- 172.21.15.27:0/1543598377 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7dac1590d0 0x7f7dac0b55d0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:28.318 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.315+0000 7f7db15d8640 1 -- 172.21.15.27:0/1543598377 >> 172.21.15.27:0/1543598377 conn(0x7f7dac06cc40 msgr2=0x7f7dac06d050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:28.318 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.315+0000 7f7db15d8640 1 -- 172.21.15.27:0/1543598377 shutdown_connections 2024-09-12T03:19:28.318 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.315+0000 7f7db15d8640 1 -- 172.21.15.27:0/1543598377 wait complete. 2024-09-12T03:19:28.318 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.316+0000 7f7db15d8640 1 Processor -- start 2024-09-12T03:19:28.318 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.316+0000 7f7db15d8640 1 -- start start 2024-09-12T03:19:28.321 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.317+0000 7f7db15d8640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7dac158cf0 0x7f7dac06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:28.321 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.317+0000 7f7db15d8640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7dac1590d0 0x7f7dac156910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:28.321 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.317+0000 7f7db15d8640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f7dac15f600 0x7f7dac145950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:28.321 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.317+0000 7f7db15d8640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f7dac06c070 con 0x7f7dac158cf0 2024-09-12T03:19:28.321 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.317+0000 7f7db15d8640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f7dac15fca0 con 0x7f7dac1590d0 2024-09-12T03:19:28.321 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.317+0000 7f7db15d8640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f7dac06c1f0 con 0x7f7dac15f600 2024-09-12T03:19:28.321 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.317+0000 7f7daaffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7dac158cf0 0x7f7dac06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:28.322 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.317+0000 7f7daaffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7dac158cf0 0x7f7dac06e9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42498/0 (socket says 172.21.15.27:42498) 2024-09-12T03:19:28.322 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.317+0000 7f7daaffd640 1 -- 172.21.15.27:0/1414623789 learned_addr learned my addr 172.21.15.27:0/1414623789 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:19:28.322 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.317+0000 7f7daa7fc640 1 --2- 172.21.15.27:0/1414623789 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7dac1590d0 0x7f7dac156910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:28.322 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.317+0000 7f7dab7fe640 1 --2- 172.21.15.27:0/1414623789 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f7dac15f600 0x7f7dac145950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:28.322 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.317+0000 7f7daaffd640 1 -- 172.21.15.27:0/1414623789 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f7dac15f600 msgr2=0x7f7dac145950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:28.322 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.317+0000 7f7daaffd640 1 --2- 172.21.15.27:0/1414623789 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f7dac15f600 0x7f7dac145950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:28.322 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.317+0000 7f7daaffd640 1 -- 172.21.15.27:0/1414623789 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7dac1590d0 msgr2=0x7f7dac156910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:28.322 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.317+0000 7f7daaffd640 1 --2- 172.21.15.27:0/1414623789 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7dac1590d0 0x7f7dac156910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:28.322 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.317+0000 7f7daaffd640 1 -- 172.21.15.27:0/1414623789 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7dac155020 con 0x7f7dac158cf0 2024-09-12T03:19:28.322 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.317+0000 7f7daaffd640 1 --2- 172.21.15.27:0/1414623789 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7dac158cf0 0x7f7dac06e9f0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f7d9c00c430 tx=0x7f7d9c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:28.322 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.317+0000 7f7d937fe640 1 -- 172.21.15.27:0/1414623789 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d9c017070 con 0x7f7dac158cf0 2024-09-12T03:19:28.322 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.318+0000 7f7d937fe640 1 -- 172.21.15.27:0/1414623789 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d9c005ce0 con 0x7f7dac158cf0 2024-09-12T03:19:28.322 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.318+0000 7f7db15d8640 1 -- 172.21.15.27:0/1414623789 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7dac155300 con 0x7f7dac158cf0 2024-09-12T03:19:28.322 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.318+0000 7f7d937fe640 1 -- 172.21.15.27:0/1414623789 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d9c0072f0 con 0x7f7dac158cf0 2024-09-12T03:19:28.323 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.318+0000 7f7db15d8640 1 -- 172.21.15.27:0/1414623789 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7dac1558c0 con 0x7f7dac158cf0 2024-09-12T03:19:28.323 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.319+0000 7f7db15d8640 1 -- 172.21.15.27:0/1414623789 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7dac06bb30 con 0x7f7dac158cf0 2024-09-12T03:19:28.323 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.319+0000 7f7d937fe640 1 -- 172.21.15.27:0/1414623789 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 9) ==== 90651+0+0 (secure 0 0 0) 0x7f7d9c01a070 con 0x7f7dac158cf0 2024-09-12T03:19:28.323 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.320+0000 7f7d937fe640 1 --2- 172.21.15.27:0/1414623789 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f7d9806bfe0 0x7f7d9806e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:28.323 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.320+0000 7f7d937fe640 1 -- 172.21.15.27:0/1414623789 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5709+0+0 (secure 0 0 0) 0x7f7d9c08a4e0 con 0x7f7dac158cf0 2024-09-12T03:19:28.323 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.320+0000 7f7daa7fc640 1 --2- 172.21.15.27:0/1414623789 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f7d9806bfe0 0x7f7d9806e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:28.323 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.320+0000 7f7daa7fc640 1 --2- 172.21.15.27:0/1414623789 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f7d9806bfe0 0x7f7d9806e4a0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f7da0034000 tx=0x7f7da002e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:28.325 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.324+0000 7f7d937fe640 1 -- 172.21.15.27:0/1414623789 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f7d9c058ca0 con 0x7f7dac158cf0 2024-09-12T03:19:28.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.504+0000 7f7db15d8640 1 -- 172.21.15.27:0/1414623789 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7dac1607b0 con 0x7f7dac158cf0 2024-09-12T03:19:28.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.505+0000 7f7d937fe640 1 -- 172.21.15.27:0/1414623789 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v56) ==== 74+0+16677 (secure 0 0 0) 0x7f7d9c0585a0 con 0x7f7dac158cf0 2024-09-12T03:19:28.507 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:19:28.507 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":56,"fsid":"d2df504e-2647-4385-a5db-9fdf69f487a0","created":"2024-09-12T03:16:07.778937+0000","modified":"2024-09-12T03:19:26.561081+0000","last_up_change":"2024-09-12T03:16:19.314778+0000","last_in_change":"2024-09-12T03:16:09.893086+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T03:16:20.097646+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-09-12T03:19:23.556103+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-09-12T03:19:24.184337+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1851116c-1fb3-4dc9-9772-f90e2ef4c355","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6817","nonce":3002321488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6819","nonce":3002321488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6823","nonce":3002321488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6821","nonce":3002321488}]},"public_addr":"172.21.15.27:6817/3002321488","cluster_addr":"172.21.15.27:6819/3002321488","heartbeat_back_addr":"172.21.15.27:6823/3002321488","heartbeat_front_addr":"172.21.15.27:6821/3002321488","state":["exists","up"]},{"osd":1,"uuid":"f38ae17d-73eb-4204-b99d-975fa806fb87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6809","nonce":1613102886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6811","nonce":1613102886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6815","nonce":1613102886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6813","nonce":1613102886}]},"public_addr":"172.21.15.27:6809/1613102886","cluster_addr":"172.21.15.27:6811/1613102886","heartbeat_back_addr":"172.21.15.27:6815/1613102886","heartbeat_front_addr":"172.21.15.27:6813/1613102886","state":["exists","up"]},{"osd":2,"uuid":"fb12dcc8-bdd0-42b8-b46d-8b10106224bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6825","nonce":3534723355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6827","nonce":3534723355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6831","nonce":3534723355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6829","nonce":3534723355}]},"public_addr":"172.21.15.27:6825/3534723355","cluster_addr":"172.21.15.27:6827/3534723355","heartbeat_back_addr":"172.21.15.27:6831/3534723355","heartbeat_front_addr":"172.21.15.27:6829/3534723355","state":["exists","up"]},{"osd":3,"uuid":"d74d3f49-9c24-4b9f-98e2-3ecd7b5ef3b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6801","nonce":80661598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6803","nonce":80661598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6807","nonce":80661598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6805","nonce":80661598}]},"public_addr":"172.21.15.27:6801/80661598","cluster_addr":"172.21.15.27:6803/80661598","heartbeat_back_addr":"172.21.15.27:6807/80661598","heartbeat_front_addr":"172.21.15.27:6805/80661598","state":["exists","up"]},{"osd":4,"uuid":"2bf123f2-1b0d-41c4-aa3b-805bab267e59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6817","nonce":3943189827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6819","nonce":3943189827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6823","nonce":3943189827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6821","nonce":3943189827}]},"public_addr":"172.21.15.45:6817/3943189827","cluster_addr":"172.21.15.45:6819/3943189827","heartbeat_back_addr":"172.21.15.45:6823/3943189827","heartbeat_front_addr":"172.21.15.45:6821/3943189827","state":["exists","up"]},{"osd":5,"uuid":"c49ba5da-91c6-4a86-a97b-4aba89c253ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6809","nonce":3620760330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6811","nonce":3620760330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6815","nonce":3620760330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6813","nonce":3620760330}]},"public_addr":"172.21.15.45:6809/3620760330","cluster_addr":"172.21.15.45:6811/3620760330","heartbeat_back_addr":"172.21.15.45:6815/3620760330","heartbeat_front_addr":"172.21.15.45:6813/3620760330","state":["exists","up"]},{"osd":6,"uuid":"b0123475-89f7-4a34-9e33-d797b99dd9dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6801","nonce":1989083510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6803","nonce":1989083510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6807","nonce":1989083510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6805","nonce":1989083510}]},"public_addr":"172.21.15.45:6801/1989083510","cluster_addr":"172.21.15.45:6803/1989083510","heartbeat_back_addr":"172.21.15.45:6807/1989083510","heartbeat_front_addr":"172.21.15.45:6805/1989083510","state":["exists","up"]},{"osd":7,"uuid":"4dad773d-b7d8-46fc-8a10-a972aad27b29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6825","nonce":1513300294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6827","nonce":1513300294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6831","nonce":1513300294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6829","nonce":1513300294}]},"public_addr":"172.21.15.45:6825/1513300294","cluster_addr":"172.21.15.45:6827/1513300294","heartbeat_back_addr":"172.21.15.45:6831/1513300294","heartbeat_front_addr":"172.21.15.45:6829/1513300294","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.881855+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:17.133252+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.735339+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.500742+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.477571+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:17.167667+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.470291+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.873743+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T03:19:28.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.508+0000 7f7db15d8640 1 -- 172.21.15.27:0/1414623789 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f7d9806bfe0 msgr2=0x7f7d9806e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:28.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.508+0000 7f7db15d8640 1 --2- 172.21.15.27:0/1414623789 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f7d9806bfe0 0x7f7d9806e4a0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f7da0034000 tx=0x7f7da002e000 comp rx=0 tx=0).stop 2024-09-12T03:19:28.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.508+0000 7f7db15d8640 1 -- 172.21.15.27:0/1414623789 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7dac158cf0 msgr2=0x7f7dac06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:28.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.508+0000 7f7db15d8640 1 --2- 172.21.15.27:0/1414623789 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7dac158cf0 0x7f7dac06e9f0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f7d9c00c430 tx=0x7f7d9c00c8f0 comp rx=0 tx=0).stop 2024-09-12T03:19:28.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.508+0000 7f7db15d8640 1 -- 172.21.15.27:0/1414623789 shutdown_connections 2024-09-12T03:19:28.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.508+0000 7f7db15d8640 1 --2- 172.21.15.27:0/1414623789 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f7d9806bfe0 0x7f7d9806e4a0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:28.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.508+0000 7f7db15d8640 1 --2- 172.21.15.27:0/1414623789 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f7dac15f600 0x7f7dac145950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:28.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.508+0000 7f7db15d8640 1 --2- 172.21.15.27:0/1414623789 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7dac1590d0 0x7f7dac156910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:28.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.508+0000 7f7db15d8640 1 --2- 172.21.15.27:0/1414623789 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7dac158cf0 0x7f7dac06e9f0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:28.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.508+0000 7f7db15d8640 1 -- 172.21.15.27:0/1414623789 >> 172.21.15.27:0/1414623789 conn(0x7f7dac06cc40 msgr2=0x7f7dac0af480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:28.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.508+0000 7f7db15d8640 1 -- 172.21.15.27:0/1414623789 shutdown_connections 2024-09-12T03:19:28.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.508+0000 7f7db15d8640 1 -- 172.21.15.27:0/1414623789 wait complete. 2024-09-12T03:19:28.520 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-12T03:19:28.521 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-12T03:19:28.566 INFO:tasks.ceph.mds.b.smithi045.stderr:2024-09-12T03:19:28.566+0000 7f49f31f8640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-12T03:19:28.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.622+0000 7efd2968d640 1 Processor -- start 2024-09-12T03:19:28.624 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.622+0000 7efd2968d640 1 -- start start 2024-09-12T03:19:28.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.623+0000 7efd2968d640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7efd24144de0 0x7efd240d0e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:28.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.623+0000 7efd2968d640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efd240b4610 con 0x7efd2415c450 2024-09-12T03:19:28.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.623+0000 7efd2968d640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efd240d0710 con 0x7efd241451c0 2024-09-12T03:19:28.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.623+0000 7efd2968d640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7efd2415cbc0 con 0x7efd24144de0 2024-09-12T03:19:28.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.623+0000 7efd237fe640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7efd2415c450 0x7efd240b50b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:49136/0 (socket says 172.21.15.27:49136) 2024-09-12T03:19:28.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.623+0000 7efd237fe640 1 -- 172.21.15.27:0/1873541419 learned_addr learned my addr 172.21.15.27:0/1873541419 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:19:28.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.624+0000 7efd22ffd640 1 --2- 172.21.15.27:0/1873541419 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7efd24144de0 0x7efd240d0e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:28.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.624+0000 7efd21ffb640 1 -- 172.21.15.27:0/1873541419 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1006282725 0 0) 0x7efd240b4610 con 0x7efd2415c450 2024-09-12T03:19:28.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.624+0000 7efd21ffb640 1 -- 172.21.15.27:0/1873541419 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efd10003520 con 0x7efd2415c450 2024-09-12T03:19:28.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.624+0000 7efd21ffb640 1 -- 172.21.15.27:0/1873541419 <== mon.1 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1867918480 0 0) 0x7efd240d0710 con 0x7efd241451c0 2024-09-12T03:19:28.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.624+0000 7efd22ffd640 1 -- 172.21.15.27:0/1873541419 >> v1:172.21.15.45:6789/0 conn(0x7efd241451c0 legacy=0x7efd2415bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:28.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.624+0000 7efd22ffd640 1 -- 172.21.15.27:0/1873541419 >> v1:172.21.15.27:6789/0 conn(0x7efd2415c450 legacy=0x7efd240b50b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:28.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.624+0000 7efd22ffd640 1 -- 172.21.15.27:0/1873541419 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd240b48b0 con 0x7efd24144de0 2024-09-12T03:19:28.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.625+0000 7efd22ffd640 1 --2- 172.21.15.27:0/1873541419 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7efd24144de0 0x7efd240d0e60 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7efd14009dc0 tx=0x7efd1402a6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4436a340ad567eb6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:28.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.625+0000 7efd21ffb640 1 -- 172.21.15.27:0/1873541419 <== mon.2 v2:172.21.15.45:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd14007740 con 0x7efd24144de0 2024-09-12T03:19:28.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.625+0000 7efd21ffb640 1 -- 172.21.15.27:0/1873541419 <== mon.2 v2:172.21.15.45:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd140078e0 con 0x7efd24144de0 2024-09-12T03:19:28.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.625+0000 7efd21ffb640 1 -- 172.21.15.27:0/1873541419 <== mon.2 v2:172.21.15.45:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd14007c90 con 0x7efd24144de0 2024-09-12T03:19:28.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.625+0000 7efd2968d640 1 -- 172.21.15.27:0/1873541419 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7efd24144de0 msgr2=0x7efd240d0e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:28.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.625+0000 7efd2968d640 1 --2- 172.21.15.27:0/1873541419 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7efd24144de0 0x7efd240d0e60 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7efd14009dc0 tx=0x7efd1402a6d0 comp rx=0 tx=0).stop 2024-09-12T03:19:28.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.625+0000 7efd2968d640 1 -- 172.21.15.27:0/1873541419 shutdown_connections 2024-09-12T03:19:28.628 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.625+0000 7efd2968d640 1 --2- 172.21.15.27:0/1873541419 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7efd24144de0 0x7efd240d0e60 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:28.628 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.625+0000 7efd2968d640 1 -- 172.21.15.27:0/1873541419 >> 172.21.15.27:0/1873541419 conn(0x7efd2406d270 msgr2=0x7efd240b3270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:28.628 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.626+0000 7efd2968d640 1 -- 172.21.15.27:0/1873541419 shutdown_connections 2024-09-12T03:19:28.628 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.626+0000 7efd2968d640 1 -- 172.21.15.27:0/1873541419 wait complete. 2024-09-12T03:19:28.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.627+0000 7efd2968d640 1 Processor -- start 2024-09-12T03:19:28.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.627+0000 7efd2968d640 1 -- start start 2024-09-12T03:19:28.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.628+0000 7efd2968d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efd24144de0 0x7efd240af2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:28.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.628+0000 7efd2968d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efd241451c0 0x7efd241f4a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:28.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.628+0000 7efd2968d640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7efd2415c450 0x7efd241f4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:28.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.628+0000 7efd2968d640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7efd2415cdc0 con 0x7efd24144de0 2024-09-12T03:19:28.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.628+0000 7efd2968d640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7efd240b4c20 con 0x7efd241451c0 2024-09-12T03:19:28.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.628+0000 7efd2968d640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7efd2415cf40 con 0x7efd2415c450 2024-09-12T03:19:28.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.628+0000 7efd22ffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efd24144de0 0x7efd240af2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:28.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.628+0000 7efd22ffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efd24144de0 0x7efd240af2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42504/0 (socket says 172.21.15.27:42504) 2024-09-12T03:19:28.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.628+0000 7efd22ffd640 1 -- 172.21.15.27:0/4232175396 learned_addr learned my addr 172.21.15.27:0/4232175396 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:19:28.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.628+0000 7efd227fc640 1 --2- 172.21.15.27:0/4232175396 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efd241451c0 0x7efd241f4a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:28.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.628+0000 7efd237fe640 1 --2- 172.21.15.27:0/4232175396 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7efd2415c450 0x7efd241f4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:28.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.628+0000 7efd22ffd640 1 -- 172.21.15.27:0/4232175396 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7efd2415c450 msgr2=0x7efd241f4fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:28.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.628+0000 7efd22ffd640 1 --2- 172.21.15.27:0/4232175396 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7efd2415c450 0x7efd241f4fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:28.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.628+0000 7efd22ffd640 1 -- 172.21.15.27:0/4232175396 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efd241451c0 msgr2=0x7efd241f4a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:28.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.628+0000 7efd22ffd640 1 --2- 172.21.15.27:0/4232175396 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efd241451c0 0x7efd241f4a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:28.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.628+0000 7efd22ffd640 1 -- 172.21.15.27:0/4232175396 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd2413e7c0 con 0x7efd24144de0 2024-09-12T03:19:28.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.628+0000 7efd237fe640 1 --2- 172.21.15.27:0/4232175396 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7efd2415c450 0x7efd241f4fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:19:28.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.628+0000 7efd22ffd640 1 --2- 172.21.15.27:0/4232175396 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efd24144de0 0x7efd240af2e0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7efd14009eb0 tx=0x7efd14004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:28.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.628+0000 7efd1b7fe640 1 -- 172.21.15.27:0/4232175396 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd14009450 con 0x7efd24144de0 2024-09-12T03:19:28.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.629+0000 7efd1b7fe640 1 -- 172.21.15.27:0/4232175396 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd14004240 con 0x7efd24144de0 2024-09-12T03:19:28.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.629+0000 7efd2968d640 1 -- 172.21.15.27:0/4232175396 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd240afbc0 con 0x7efd24144de0 2024-09-12T03:19:28.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.629+0000 7efd1b7fe640 1 -- 172.21.15.27:0/4232175396 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd140045f0 con 0x7efd24144de0 2024-09-12T03:19:28.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.629+0000 7efd2968d640 1 -- 172.21.15.27:0/4232175396 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd240af820 con 0x7efd24144de0 2024-09-12T03:19:28.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.630+0000 7efd2968d640 1 -- 172.21.15.27:0/4232175396 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd240affb0 con 0x7efd24144de0 2024-09-12T03:19:28.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.630+0000 7efd1b7fe640 1 -- 172.21.15.27:0/4232175396 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 9) ==== 90651+0+0 (secure 0 0 0) 0x7efd14039040 con 0x7efd24144de0 2024-09-12T03:19:28.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.630+0000 7efd1b7fe640 1 --2- 172.21.15.27:0/4232175396 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7efd0406bfe0 0x7efd0406e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:28.634 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.631+0000 7efd1b7fe640 1 -- 172.21.15.27:0/4232175396 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5709+0+0 (secure 0 0 0) 0x7efd140ac790 con 0x7efd24144de0 2024-09-12T03:19:28.634 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.631+0000 7efd227fc640 1 --2- 172.21.15.27:0/4232175396 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7efd0406bfe0 0x7efd0406e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:28.634 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.631+0000 7efd227fc640 1 --2- 172.21.15.27:0/4232175396 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7efd0406bfe0 0x7efd0406e4a0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7efd2415f7a0 tx=0x7efd0c009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:28.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.635+0000 7efd1b7fe640 1 -- 172.21.15.27:0/4232175396 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7efd14040030 con 0x7efd24144de0 2024-09-12T03:19:28.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.806+0000 7efd2968d640 1 -- 172.21.15.27:0/4232175396 --> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7efd240d0170 con 0x7efd0406bfe0 2024-09-12T03:19:28.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.813+0000 7efd1b7fe640 1 -- 172.21.15.27:0/4232175396 <== mgr.4102 v2:172.21.15.45:6832/3923821219 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+246638 (secure 0 0 0) 0x7efd240d0170 con 0x7efd0406bfe0 2024-09-12T03:19:28.818 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:19:28.819 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2024-09-12T03:19:28.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.816+0000 7efd2968d640 1 -- 172.21.15.27:0/4232175396 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7efd0406bfe0 msgr2=0x7efd0406e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:28.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.816+0000 7efd2968d640 1 --2- 172.21.15.27:0/4232175396 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7efd0406bfe0 0x7efd0406e4a0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7efd2415f7a0 tx=0x7efd0c009040 comp rx=0 tx=0).stop 2024-09-12T03:19:28.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.816+0000 7efd2968d640 1 -- 172.21.15.27:0/4232175396 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efd24144de0 msgr2=0x7efd240af2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:28.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.816+0000 7efd2968d640 1 --2- 172.21.15.27:0/4232175396 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efd24144de0 0x7efd240af2e0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7efd14009eb0 tx=0x7efd14004a40 comp rx=0 tx=0).stop 2024-09-12T03:19:28.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.816+0000 7efd2968d640 1 -- 172.21.15.27:0/4232175396 shutdown_connections 2024-09-12T03:19:28.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.816+0000 7efd2968d640 1 --2- 172.21.15.27:0/4232175396 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7efd0406bfe0 0x7efd0406e4a0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:28.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.816+0000 7efd2968d640 1 --2- 172.21.15.27:0/4232175396 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7efd2415c450 0x7efd241f4fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:28.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.816+0000 7efd2968d640 1 --2- 172.21.15.27:0/4232175396 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efd241451c0 0x7efd241f4a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:28.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.816+0000 7efd2968d640 1 --2- 172.21.15.27:0/4232175396 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efd24144de0 0x7efd240af2e0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:28.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.816+0000 7efd2968d640 1 -- 172.21.15.27:0/4232175396 >> 172.21.15.27:0/4232175396 conn(0x7efd2406d270 msgr2=0x7efd2406c310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:28.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.816+0000 7efd2968d640 1 -- 172.21.15.27:0/4232175396 shutdown_connections 2024-09-12T03:19:28.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.816+0000 7efd2968d640 1 -- 172.21.15.27:0/4232175396 wait complete. 2024-09-12T03:19:28.828 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":143,"stamp":"2024-09-12T03:19:28.095434+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":29,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":247332,"kb_used_data":6100,"kb_used_omap":12,"kb_used_meta":241011,"kb_avail":754727388,"statfs":{"total":773094113280,"available":772840845312,"internally_reserved":0,"allocated":6246400,"data_stored":3240131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":246795824},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":20,"apply_latency_ms":20,"commit_latency_ns":20000000,"apply_latency_ns":20000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.256497"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.851741+0000","last_change":"2024-09-12T03:19:24.851741+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.851741+0000","last_undegraded":"2024-09-12T03:19:24.851741+0000","last_fullsized":"2024-09-12T03:19:24.851741+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.853689+0000","last_change":"2024-09-12T03:19:24.853689+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.853689+0000","last_undegraded":"2024-09-12T03:19:24.853689+0000","last_fullsized":"2024-09-12T03:19:24.853689+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.853107+0000","last_change":"2024-09-12T03:19:24.853107+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.853107+0000","last_undegraded":"2024-09-12T03:19:24.853107+0000","last_fullsized":"2024-09-12T03:19:24.853107+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.849111+0000","last_change":"2024-09-12T03:19:24.849111+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.849111+0000","last_undegraded":"2024-09-12T03:19:24.849111+0000","last_fullsized":"2024-09-12T03:19:24.849111+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.852648+0000","last_change":"2024-09-12T03:19:24.852648+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.852648+0000","last_undegraded":"2024-09-12T03:19:24.852648+0000","last_fullsized":"2024-09-12T03:19:24.852648+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.854574+0000","last_change":"2024-09-12T03:19:24.854574+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.854574+0000","last_undegraded":"2024-09-12T03:19:24.854574+0000","last_fullsized":"2024-09-12T03:19:24.854574+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.846611+0000","last_change":"2024-09-12T03:19:24.846611+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.846611+0000","last_undegraded":"2024-09-12T03:19:24.846611+0000","last_fullsized":"2024-09-12T03:19:24.846611+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.855228+0000","last_change":"2024-09-12T03:19:24.855228+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.855228+0000","last_undegraded":"2024-09-12T03:19:24.855228+0000","last_fullsized":"2024-09-12T03:19:24.855228+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.845883+0000","last_change":"2024-09-12T03:19:24.845883+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.845883+0000","last_undegraded":"2024-09-12T03:19:24.845883+0000","last_fullsized":"2024-09-12T03:19:24.845883+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.848151+0000","last_change":"2024-09-12T03:19:24.848151+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.848151+0000","last_undegraded":"2024-09-12T03:19:24.848151+0000","last_fullsized":"2024-09-12T03:19:24.848151+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.848151+0000","last_change":"2024-09-12T03:19:24.848151+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.848151+0000","last_undegraded":"2024-09-12T03:19:24.848151+0000","last_fullsized":"2024-09-12T03:19:24.848151+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.848357+0000","last_change":"2024-09-12T03:19:24.848357+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.848357+0000","last_undegraded":"2024-09-12T03:19:24.848357+0000","last_fullsized":"2024-09-12T03:19:24.848357+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.851899+0000","last_change":"2024-09-12T03:19:24.851899+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.851899+0000","last_undegraded":"2024-09-12T03:19:24.851899+0000","last_fullsized":"2024-09-12T03:19:24.851899+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.847361+0000","last_change":"2024-09-12T03:19:24.847361+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.847361+0000","last_undegraded":"2024-09-12T03:19:24.847361+0000","last_fullsized":"2024-09-12T03:19:24.847361+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.845866+0000","last_change":"2024-09-12T03:19:24.845866+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.845866+0000","last_undegraded":"2024-09-12T03:19:24.845866+0000","last_fullsized":"2024-09-12T03:19:24.845866+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.852548+0000","last_change":"2024-09-12T03:19:24.852548+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.852548+0000","last_undegraded":"2024-09-12T03:19:24.852548+0000","last_fullsized":"2024-09-12T03:19:24.852548+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.850960+0000","last_change":"2024-09-12T03:19:24.850960+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.850960+0000","last_undegraded":"2024-09-12T03:19:24.850960+0000","last_fullsized":"2024-09-12T03:19:24.850960+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.852064+0000","last_change":"2024-09-12T03:19:24.852064+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.852064+0000","last_undegraded":"2024-09-12T03:19:24.852064+0000","last_fullsized":"2024-09-12T03:19:24.852064+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.850808+0000","last_change":"2024-09-12T03:19:24.850808+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.850808+0000","last_undegraded":"2024-09-12T03:19:24.850808+0000","last_fullsized":"2024-09-12T03:19:24.850808+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.850087+0000","last_change":"2024-09-12T03:19:24.850087+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.850087+0000","last_undegraded":"2024-09-12T03:19:24.850087+0000","last_fullsized":"2024-09-12T03:19:24.850087+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.851577+0000","last_change":"2024-09-12T03:19:24.851577+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.851577+0000","last_undegraded":"2024-09-12T03:19:24.851577+0000","last_fullsized":"2024-09-12T03:19:24.851577+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.856451+0000","last_change":"2024-09-12T03:19:24.856451+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.856451+0000","last_undegraded":"2024-09-12T03:19:24.856451+0000","last_fullsized":"2024-09-12T03:19:24.856451+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.852682+0000","last_change":"2024-09-12T03:19:24.852682+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.852682+0000","last_undegraded":"2024-09-12T03:19:24.852682+0000","last_fullsized":"2024-09-12T03:19:24.852682+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.852444+0000","last_change":"2024-09-12T03:19:24.852444+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.852444+0000","last_undegraded":"2024-09-12T03:19:24.852444+0000","last_fullsized":"2024-09-12T03:19:24.852444+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.851312+0000","last_change":"2024-09-12T03:19:24.851312+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.851312+0000","last_undegraded":"2024-09-12T03:19:24.851312+0000","last_fullsized":"2024-09-12T03:19:24.851312+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.848902+0000","last_change":"2024-09-12T03:19:24.848902+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.848902+0000","last_undegraded":"2024-09-12T03:19:24.848902+0000","last_fullsized":"2024-09-12T03:19:24.848902+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.855319+0000","last_change":"2024-09-12T03:19:24.855319+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.855319+0000","last_undegraded":"2024-09-12T03:19:24.855319+0000","last_fullsized":"2024-09-12T03:19:24.855319+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.851621+0000","last_change":"2024-09-12T03:19:24.851621+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.851621+0000","last_undegraded":"2024-09-12T03:19:24.851621+0000","last_fullsized":"2024-09-12T03:19:24.851621+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.850354+0000","last_change":"2024-09-12T03:19:24.850354+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.850354+0000","last_undegraded":"2024-09-12T03:19:24.850354+0000","last_fullsized":"2024-09-12T03:19:24.850354+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.850218+0000","last_change":"2024-09-12T03:19:24.850218+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.850218+0000","last_undegraded":"2024-09-12T03:19:24.850218+0000","last_fullsized":"2024-09-12T03:19:24.850218+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.850269+0000","last_change":"2024-09-12T03:19:24.850269+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.850269+0000","last_undegraded":"2024-09-12T03:19:24.850269+0000","last_fullsized":"2024-09-12T03:19:24.850269+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.852116+0000","last_change":"2024-09-12T03:19:24.852116+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.852116+0000","last_undegraded":"2024-09-12T03:19:24.852116+0000","last_fullsized":"2024-09-12T03:19:24.852116+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.856894+0000","last_change":"2024-09-12T03:19:24.856894+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.856894+0000","last_undegraded":"2024-09-12T03:19:24.856894+0000","last_fullsized":"2024-09-12T03:19:24.856894+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T03:19:24.869900+0000","last_change":"2024-09-12T03:19:24.860694+0000","last_active":"2024-09-12T03:19:24.869900+0000","last_peered":"2024-09-12T03:19:24.869900+0000","last_clean":"2024-09-12T03:19:24.869900+0000","last_became_active":"2024-09-12T03:19:24.860380+0000","last_became_peered":"2024-09-12T03:19:24.860380+0000","last_unstale":"2024-09-12T03:19:24.869900+0000","last_undegraded":"2024-09-12T03:19:24.869900+0000","last_fullsized":"2024-09-12T03:19:24.869900+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_clean_scrub_stamp":"2024-09-12T03:19:23.835989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:22:23.745090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.848080+0000","last_change":"2024-09-12T03:19:24.848080+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.848080+0000","last_undegraded":"2024-09-12T03:19:24.848080+0000","last_fullsized":"2024-09-12T03:19:24.848080+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T03:19:24.866702+0000","last_change":"2024-09-12T03:19:24.863015+0000","last_active":"2024-09-12T03:19:24.866702+0000","last_peered":"2024-09-12T03:19:24.866702+0000","last_clean":"2024-09-12T03:19:24.866702+0000","last_became_active":"2024-09-12T03:19:24.862457+0000","last_became_peered":"2024-09-12T03:19:24.862457+0000","last_unstale":"2024-09-12T03:19:24.866702+0000","last_undegraded":"2024-09-12T03:19:24.866702+0000","last_fullsized":"2024-09-12T03:19:24.866702+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_clean_scrub_stamp":"2024-09-12T03:19:23.835989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:25:17.547373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.854180+0000","last_change":"2024-09-12T03:19:24.854180+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.854180+0000","last_undegraded":"2024-09-12T03:19:24.854180+0000","last_fullsized":"2024-09-12T03:19:24.854180+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T03:19:24.869928+0000","last_change":"2024-09-12T03:19:24.862226+0000","last_active":"2024-09-12T03:19:24.869928+0000","last_peered":"2024-09-12T03:19:24.869928+0000","last_clean":"2024-09-12T03:19:24.869928+0000","last_became_active":"2024-09-12T03:19:24.861956+0000","last_became_peered":"2024-09-12T03:19:24.861956+0000","last_unstale":"2024-09-12T03:19:24.869928+0000","last_undegraded":"2024-09-12T03:19:24.869928+0000","last_fullsized":"2024-09-12T03:19:24.869928+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_clean_scrub_stamp":"2024-09-12T03:19:23.835989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:56:28.082698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.851109+0000","last_change":"2024-09-12T03:19:24.851109+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.851109+0000","last_undegraded":"2024-09-12T03:19:24.851109+0000","last_fullsized":"2024-09-12T03:19:24.851109+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T03:19:24.861878+0000","last_change":"2024-09-12T03:19:24.859901+0000","last_active":"2024-09-12T03:19:24.861878+0000","last_peered":"2024-09-12T03:19:24.861878+0000","last_clean":"2024-09-12T03:19:24.861878+0000","last_became_active":"2024-09-12T03:19:24.859637+0000","last_became_peered":"2024-09-12T03:19:24.859637+0000","last_unstale":"2024-09-12T03:19:24.861878+0000","last_undegraded":"2024-09-12T03:19:24.861878+0000","last_fullsized":"2024-09-12T03:19:24.861878+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_clean_scrub_stamp":"2024-09-12T03:19:23.835989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:48:14.537533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.853953+0000","last_change":"2024-09-12T03:19:24.853953+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.853953+0000","last_undegraded":"2024-09-12T03:19:24.853953+0000","last_fullsized":"2024-09-12T03:19:24.853953+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T03:19:24.858328+0000","last_change":"2024-09-12T03:19:24.856981+0000","last_active":"2024-09-12T03:19:24.858328+0000","last_peered":"2024-09-12T03:19:24.858328+0000","last_clean":"2024-09-12T03:19:24.858328+0000","last_became_active":"2024-09-12T03:19:24.856607+0000","last_became_peered":"2024-09-12T03:19:24.856607+0000","last_unstale":"2024-09-12T03:19:24.858328+0000","last_undegraded":"2024-09-12T03:19:24.858328+0000","last_fullsized":"2024-09-12T03:19:24.858328+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_clean_scrub_stamp":"2024-09-12T03:19:23.835989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:56:55.814953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.851504+0000","last_change":"2024-09-12T03:19:24.851504+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.851504+0000","last_undegraded":"2024-09-12T03:19:24.851504+0000","last_fullsized":"2024-09-12T03:19:24.851504+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T03:19:24.869885+0000","last_change":"2024-09-12T03:19:24.861305+0000","last_active":"2024-09-12T03:19:24.869885+0000","last_peered":"2024-09-12T03:19:24.869885+0000","last_clean":"2024-09-12T03:19:24.869885+0000","last_became_active":"2024-09-12T03:19:24.861021+0000","last_became_peered":"2024-09-12T03:19:24.861021+0000","last_unstale":"2024-09-12T03:19:24.869885+0000","last_undegraded":"2024-09-12T03:19:24.869885+0000","last_fullsized":"2024-09-12T03:19:24.869885+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_clean_scrub_stamp":"2024-09-12T03:19:23.835989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:52:21.815077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.851467+0000","last_change":"2024-09-12T03:19:24.851467+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.851467+0000","last_undegraded":"2024-09-12T03:19:24.851467+0000","last_fullsized":"2024-09-12T03:19:24.851467+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T03:19:24.854193+0000","last_change":"2024-09-12T03:19:24.850360+0000","last_active":"2024-09-12T03:19:24.854193+0000","last_peered":"2024-09-12T03:19:24.854193+0000","last_clean":"2024-09-12T03:19:24.854193+0000","last_became_active":"2024-09-12T03:19:24.850027+0000","last_became_peered":"2024-09-12T03:19:24.850027+0000","last_unstale":"2024-09-12T03:19:24.854193+0000","last_undegraded":"2024-09-12T03:19:24.854193+0000","last_fullsized":"2024-09-12T03:19:24.854193+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_clean_scrub_stamp":"2024-09-12T03:19:23.835989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:35:50.136024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.853049+0000","last_change":"2024-09-12T03:19:24.853049+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.853049+0000","last_undegraded":"2024-09-12T03:19:24.853049+0000","last_fullsized":"2024-09-12T03:19:24.853049+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T03:19:24.864569+0000","last_change":"2024-09-12T03:19:24.863770+0000","last_active":"2024-09-12T03:19:24.864569+0000","last_peered":"2024-09-12T03:19:24.864569+0000","last_clean":"2024-09-12T03:19:24.864569+0000","last_became_active":"2024-09-12T03:19:24.863471+0000","last_became_peered":"2024-09-12T03:19:24.863471+0000","last_unstale":"2024-09-12T03:19:24.864569+0000","last_undegraded":"2024-09-12T03:19:24.864569+0000","last_fullsized":"2024-09-12T03:19:24.864569+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_clean_scrub_stamp":"2024-09-12T03:19:23.835989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:00:17.289844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.846718+0000","last_change":"2024-09-12T03:19:24.846718+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.846718+0000","last_undegraded":"2024-09-12T03:19:24.846718+0000","last_fullsized":"2024-09-12T03:19:24.846718+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T03:19:24.870959+0000","last_change":"2024-09-12T03:19:24.870526+0000","last_active":"2024-09-12T03:19:24.870959+0000","last_peered":"2024-09-12T03:19:24.870959+0000","last_clean":"2024-09-12T03:19:24.870959+0000","last_became_active":"2024-09-12T03:19:24.870226+0000","last_became_peered":"2024-09-12T03:19:24.870226+0000","last_unstale":"2024-09-12T03:19:24.870959+0000","last_undegraded":"2024-09-12T03:19:24.870959+0000","last_fullsized":"2024-09-12T03:19:24.870959+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_clean_scrub_stamp":"2024-09-12T03:19:23.835989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:43:45.344248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.849991+0000","last_change":"2024-09-12T03:19:24.849991+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.849991+0000","last_undegraded":"2024-09-12T03:19:24.849991+0000","last_fullsized":"2024-09-12T03:19:24.849991+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T03:19:24.866683+0000","last_change":"2024-09-12T03:19:24.862733+0000","last_active":"2024-09-12T03:19:24.866683+0000","last_peered":"2024-09-12T03:19:24.866683+0000","last_clean":"2024-09-12T03:19:24.866683+0000","last_became_active":"2024-09-12T03:19:24.862164+0000","last_became_peered":"2024-09-12T03:19:24.862164+0000","last_unstale":"2024-09-12T03:19:24.866683+0000","last_undegraded":"2024-09-12T03:19:24.866683+0000","last_fullsized":"2024-09-12T03:19:24.866683+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_clean_scrub_stamp":"2024-09-12T03:19:23.835989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:50:19.592853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.848059+0000","last_change":"2024-09-12T03:19:24.848059+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.848059+0000","last_undegraded":"2024-09-12T03:19:24.848059+0000","last_fullsized":"2024-09-12T03:19:24.848059+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T03:19:24.860582+0000","last_change":"2024-09-12T03:19:24.859732+0000","last_active":"2024-09-12T03:19:24.860582+0000","last_peered":"2024-09-12T03:19:24.860582+0000","last_clean":"2024-09-12T03:19:24.860582+0000","last_became_active":"2024-09-12T03:19:24.859452+0000","last_became_peered":"2024-09-12T03:19:24.859452+0000","last_unstale":"2024-09-12T03:19:24.860582+0000","last_undegraded":"2024-09-12T03:19:24.860582+0000","last_fullsized":"2024-09-12T03:19:24.860582+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_clean_scrub_stamp":"2024-09-12T03:19:23.835989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:16:48.958106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.852512+0000","last_change":"2024-09-12T03:19:24.852512+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.852512+0000","last_undegraded":"2024-09-12T03:19:24.852512+0000","last_fullsized":"2024-09-12T03:19:24.852512+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T03:19:24.862745+0000","last_change":"2024-09-12T03:19:24.861967+0000","last_active":"2024-09-12T03:19:24.862745+0000","last_peered":"2024-09-12T03:19:24.862745+0000","last_clean":"2024-09-12T03:19:24.862745+0000","last_became_active":"2024-09-12T03:19:24.861645+0000","last_became_peered":"2024-09-12T03:19:24.861645+0000","last_unstale":"2024-09-12T03:19:24.862745+0000","last_undegraded":"2024-09-12T03:19:24.862745+0000","last_fullsized":"2024-09-12T03:19:24.862745+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_clean_scrub_stamp":"2024-09-12T03:19:23.835989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:11:47.170009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.847408+0000","last_change":"2024-09-12T03:19:24.847408+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.847408+0000","last_undegraded":"2024-09-12T03:19:24.847408+0000","last_fullsized":"2024-09-12T03:19:24.847408+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T03:19:24.866725+0000","last_change":"2024-09-12T03:19:24.863067+0000","last_active":"2024-09-12T03:19:24.866725+0000","last_peered":"2024-09-12T03:19:24.866725+0000","last_clean":"2024-09-12T03:19:24.866725+0000","last_became_active":"2024-09-12T03:19:24.862471+0000","last_became_peered":"2024-09-12T03:19:24.862471+0000","last_unstale":"2024-09-12T03:19:24.866725+0000","last_undegraded":"2024-09-12T03:19:24.866725+0000","last_fullsized":"2024-09-12T03:19:24.866725+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_clean_scrub_stamp":"2024-09-12T03:19:23.835989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:04:37.233100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.851140+0000","last_change":"2024-09-12T03:19:24.851140+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.851140+0000","last_undegraded":"2024-09-12T03:19:24.851140+0000","last_fullsized":"2024-09-12T03:19:24.851140+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T03:19:24.862050+0000","last_change":"2024-09-12T03:19:24.859981+0000","last_active":"2024-09-12T03:19:24.862050+0000","last_peered":"2024-09-12T03:19:24.862050+0000","last_clean":"2024-09-12T03:19:24.862050+0000","last_became_active":"2024-09-12T03:19:24.859679+0000","last_became_peered":"2024-09-12T03:19:24.859679+0000","last_unstale":"2024-09-12T03:19:24.862050+0000","last_undegraded":"2024-09-12T03:19:24.862050+0000","last_fullsized":"2024-09-12T03:19:24.862050+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_clean_scrub_stamp":"2024-09-12T03:19:23.835989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:51:59.350797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.846337+0000","last_change":"2024-09-12T03:19:24.846337+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.846337+0000","last_undegraded":"2024-09-12T03:19:24.846337+0000","last_fullsized":"2024-09-12T03:19:24.846337+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T03:19:24.857304+0000","last_change":"2024-09-12T03:19:24.855795+0000","last_active":"2024-09-12T03:19:24.857304+0000","last_peered":"2024-09-12T03:19:24.857304+0000","last_clean":"2024-09-12T03:19:24.857304+0000","last_became_active":"2024-09-12T03:19:24.855377+0000","last_became_peered":"2024-09-12T03:19:24.855377+0000","last_unstale":"2024-09-12T03:19:24.857304+0000","last_undegraded":"2024-09-12T03:19:24.857304+0000","last_fullsized":"2024-09-12T03:19:24.857304+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_clean_scrub_stamp":"2024-09-12T03:19:23.835989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:50:07.794274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.852242+0000","last_change":"2024-09-12T03:19:24.852242+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.852242+0000","last_undegraded":"2024-09-12T03:19:24.852242+0000","last_fullsized":"2024-09-12T03:19:24.852242+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T03:19:24.860954+0000","last_change":"2024-09-12T03:19:24.859883+0000","last_active":"2024-09-12T03:19:24.860954+0000","last_peered":"2024-09-12T03:19:24.860954+0000","last_clean":"2024-09-12T03:19:24.860954+0000","last_became_active":"2024-09-12T03:19:24.859566+0000","last_became_peered":"2024-09-12T03:19:24.859566+0000","last_unstale":"2024-09-12T03:19:24.860954+0000","last_undegraded":"2024-09-12T03:19:24.860954+0000","last_fullsized":"2024-09-12T03:19:24.860954+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_clean_scrub_stamp":"2024-09-12T03:19:23.835989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:44:44.153823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.851328+0000","last_change":"2024-09-12T03:19:24.851328+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.851328+0000","last_undegraded":"2024-09-12T03:19:24.851328+0000","last_fullsized":"2024-09-12T03:19:24.851328+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T03:19:24.866633+0000","last_change":"2024-09-12T03:19:24.861891+0000","last_active":"2024-09-12T03:19:24.866633+0000","last_peered":"2024-09-12T03:19:24.866633+0000","last_clean":"2024-09-12T03:19:24.866633+0000","last_became_active":"2024-09-12T03:19:24.861524+0000","last_became_peered":"2024-09-12T03:19:24.861524+0000","last_unstale":"2024-09-12T03:19:24.866633+0000","last_undegraded":"2024-09-12T03:19:24.866633+0000","last_fullsized":"2024-09-12T03:19:24.866633+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_clean_scrub_stamp":"2024-09-12T03:19:23.835989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:16:02.571700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.848416+0000","last_change":"2024-09-12T03:19:24.848416+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.848416+0000","last_undegraded":"2024-09-12T03:19:24.848416+0000","last_fullsized":"2024-09-12T03:19:24.848416+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T03:19:24.872537+0000","last_change":"2024-09-12T03:19:24.865925+0000","last_active":"2024-09-12T03:19:24.872537+0000","last_peered":"2024-09-12T03:19:24.872537+0000","last_clean":"2024-09-12T03:19:24.872537+0000","last_became_active":"2024-09-12T03:19:24.865689+0000","last_became_peered":"2024-09-12T03:19:24.865689+0000","last_unstale":"2024-09-12T03:19:24.872537+0000","last_undegraded":"2024-09-12T03:19:24.872537+0000","last_fullsized":"2024-09-12T03:19:24.872537+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_clean_scrub_stamp":"2024-09-12T03:19:23.835989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:17:44.078064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.849465+0000","last_change":"2024-09-12T03:19:24.849465+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.849465+0000","last_undegraded":"2024-09-12T03:19:24.849465+0000","last_fullsized":"2024-09-12T03:19:24.849465+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T03:19:24.857771+0000","last_change":"2024-09-12T03:19:24.855690+0000","last_active":"2024-09-12T03:19:24.857771+0000","last_peered":"2024-09-12T03:19:24.857771+0000","last_clean":"2024-09-12T03:19:24.857771+0000","last_became_active":"2024-09-12T03:19:24.855284+0000","last_became_peered":"2024-09-12T03:19:24.855284+0000","last_unstale":"2024-09-12T03:19:24.857771+0000","last_undegraded":"2024-09-12T03:19:24.857771+0000","last_fullsized":"2024-09-12T03:19:24.857771+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_clean_scrub_stamp":"2024-09-12T03:19:23.835989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:38:35.821379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.851636+0000","last_change":"2024-09-12T03:19:24.851636+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.851636+0000","last_undegraded":"2024-09-12T03:19:24.851636+0000","last_fullsized":"2024-09-12T03:19:24.851636+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T03:19:24.872475+0000","last_change":"2024-09-12T03:19:24.864872+0000","last_active":"2024-09-12T03:19:24.872475+0000","last_peered":"2024-09-12T03:19:24.872475+0000","last_clean":"2024-09-12T03:19:24.872475+0000","last_became_active":"2024-09-12T03:19:24.864506+0000","last_became_peered":"2024-09-12T03:19:24.864506+0000","last_unstale":"2024-09-12T03:19:24.872475+0000","last_undegraded":"2024-09-12T03:19:24.872475+0000","last_fullsized":"2024-09-12T03:19:24.872475+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_clean_scrub_stamp":"2024-09-12T03:19:23.835989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:16:51.163113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.846915+0000","last_change":"2024-09-12T03:19:24.846915+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.846915+0000","last_undegraded":"2024-09-12T03:19:24.846915+0000","last_fullsized":"2024-09-12T03:19:24.846915+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T03:19:24.866700+0000","last_change":"2024-09-12T03:19:24.862743+0000","last_active":"2024-09-12T03:19:24.866700+0000","last_peered":"2024-09-12T03:19:24.866700+0000","last_clean":"2024-09-12T03:19:24.866700+0000","last_became_active":"2024-09-12T03:19:24.862186+0000","last_became_peered":"2024-09-12T03:19:24.862186+0000","last_unstale":"2024-09-12T03:19:24.866700+0000","last_undegraded":"2024-09-12T03:19:24.866700+0000","last_fullsized":"2024-09-12T03:19:24.866700+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_clean_scrub_stamp":"2024-09-12T03:19:23.835989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:04:27.775435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.855497+0000","last_change":"2024-09-12T03:19:24.855497+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.855497+0000","last_undegraded":"2024-09-12T03:19:24.855497+0000","last_fullsized":"2024-09-12T03:19:24.855497+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T03:19:24.860844+0000","last_change":"2024-09-12T03:19:24.857052+0000","last_active":"2024-09-12T03:19:24.860844+0000","last_peered":"2024-09-12T03:19:24.860844+0000","last_clean":"2024-09-12T03:19:24.860844+0000","last_became_active":"2024-09-12T03:19:24.856771+0000","last_became_peered":"2024-09-12T03:19:24.856771+0000","last_unstale":"2024-09-12T03:19:24.860844+0000","last_undegraded":"2024-09-12T03:19:24.860844+0000","last_fullsized":"2024-09-12T03:19:24.860844+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_clean_scrub_stamp":"2024-09-12T03:19:23.835989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:35:47.557902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.852418+0000","last_change":"2024-09-12T03:19:24.852418+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.852418+0000","last_undegraded":"2024-09-12T03:19:24.852418+0000","last_fullsized":"2024-09-12T03:19:24.852418+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T03:19:24.864938+0000","last_change":"2024-09-12T03:19:24.864514+0000","last_active":"2024-09-12T03:19:24.864938+0000","last_peered":"2024-09-12T03:19:24.864938+0000","last_clean":"2024-09-12T03:19:24.864938+0000","last_became_active":"2024-09-12T03:19:24.864206+0000","last_became_peered":"2024-09-12T03:19:24.864206+0000","last_unstale":"2024-09-12T03:19:24.864938+0000","last_undegraded":"2024-09-12T03:19:24.864938+0000","last_fullsized":"2024-09-12T03:19:24.864938+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_clean_scrub_stamp":"2024-09-12T03:19:23.835989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:24:19.872805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.847841+0000","last_change":"2024-09-12T03:19:24.847841+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.847841+0000","last_undegraded":"2024-09-12T03:19:24.847841+0000","last_fullsized":"2024-09-12T03:19:24.847841+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T03:19:24.866519+0000","last_change":"2024-09-12T03:19:24.863777+0000","last_active":"2024-09-12T03:19:24.866519+0000","last_peered":"2024-09-12T03:19:24.866519+0000","last_clean":"2024-09-12T03:19:24.866519+0000","last_became_active":"2024-09-12T03:19:24.863506+0000","last_became_peered":"2024-09-12T03:19:24.863506+0000","last_unstale":"2024-09-12T03:19:24.866519+0000","last_undegraded":"2024-09-12T03:19:24.866519+0000","last_fullsized":"2024-09-12T03:19:24.866519+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_clean_scrub_stamp":"2024-09-12T03:19:23.835989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:53:10.464555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.847949+0000","last_change":"2024-09-12T03:19:24.847949+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.847949+0000","last_undegraded":"2024-09-12T03:19:24.847949+0000","last_fullsized":"2024-09-12T03:19:24.847949+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T03:19:24.872512+0000","last_change":"2024-09-12T03:19:24.865383+0000","last_active":"2024-09-12T03:19:24.872512+0000","last_peered":"2024-09-12T03:19:24.872512+0000","last_clean":"2024-09-12T03:19:24.872512+0000","last_became_active":"2024-09-12T03:19:24.865031+0000","last_became_peered":"2024-09-12T03:19:24.865031+0000","last_unstale":"2024-09-12T03:19:24.872512+0000","last_undegraded":"2024-09-12T03:19:24.872512+0000","last_fullsized":"2024-09-12T03:19:24.872512+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_clean_scrub_stamp":"2024-09-12T03:19:23.835989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:36:23.795897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.847544+0000","last_change":"2024-09-12T03:19:24.847544+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.847544+0000","last_undegraded":"2024-09-12T03:19:24.847544+0000","last_fullsized":"2024-09-12T03:19:24.847544+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T03:19:24.861103+0000","last_change":"2024-09-12T03:19:24.857501+0000","last_active":"2024-09-12T03:19:24.861103+0000","last_peered":"2024-09-12T03:19:24.861103+0000","last_clean":"2024-09-12T03:19:24.861103+0000","last_became_active":"2024-09-12T03:19:24.857129+0000","last_became_peered":"2024-09-12T03:19:24.857129+0000","last_unstale":"2024-09-12T03:19:24.861103+0000","last_undegraded":"2024-09-12T03:19:24.861103+0000","last_fullsized":"2024-09-12T03:19:24.861103+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_clean_scrub_stamp":"2024-09-12T03:19:23.835989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:54:09.253982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.850621+0000","last_change":"2024-09-12T03:19:24.850621+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.850621+0000","last_undegraded":"2024-09-12T03:19:24.850621+0000","last_fullsized":"2024-09-12T03:19:24.850621+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T03:19:24.856019+0000","last_change":"2024-09-12T03:19:24.852831+0000","last_active":"2024-09-12T03:19:24.856019+0000","last_peered":"2024-09-12T03:19:24.856019+0000","last_clean":"2024-09-12T03:19:24.856019+0000","last_became_active":"2024-09-12T03:19:24.851427+0000","last_became_peered":"2024-09-12T03:19:24.851427+0000","last_unstale":"2024-09-12T03:19:24.856019+0000","last_undegraded":"2024-09-12T03:19:24.856019+0000","last_fullsized":"2024-09-12T03:19:24.856019+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_clean_scrub_stamp":"2024-09-12T03:19:23.835989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:39:10.957546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.853481+0000","last_change":"2024-09-12T03:19:24.853481+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.853481+0000","last_undegraded":"2024-09-12T03:19:24.853481+0000","last_fullsized":"2024-09-12T03:19:24.853481+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T03:19:24.855106+0000","last_change":"2024-09-12T03:19:24.849318+0000","last_active":"2024-09-12T03:19:24.855106+0000","last_peered":"2024-09-12T03:19:24.855106+0000","last_clean":"2024-09-12T03:19:24.855106+0000","last_became_active":"2024-09-12T03:19:24.848981+0000","last_became_peered":"2024-09-12T03:19:24.848981+0000","last_unstale":"2024-09-12T03:19:24.855106+0000","last_undegraded":"2024-09-12T03:19:24.855106+0000","last_fullsized":"2024-09-12T03:19:24.855106+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_clean_scrub_stamp":"2024-09-12T03:19:23.835989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:07:42.042664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.850646+0000","last_change":"2024-09-12T03:19:24.850646+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.850646+0000","last_undegraded":"2024-09-12T03:19:24.850646+0000","last_fullsized":"2024-09-12T03:19:24.850646+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T03:19:24.862763+0000","last_change":"2024-09-12T03:19:24.861998+0000","last_active":"2024-09-12T03:19:24.862763+0000","last_peered":"2024-09-12T03:19:24.862763+0000","last_clean":"2024-09-12T03:19:24.862763+0000","last_became_active":"2024-09-12T03:19:24.861694+0000","last_became_peered":"2024-09-12T03:19:24.861694+0000","last_unstale":"2024-09-12T03:19:24.862763+0000","last_undegraded":"2024-09-12T03:19:24.862763+0000","last_fullsized":"2024-09-12T03:19:24.862763+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_clean_scrub_stamp":"2024-09-12T03:19:23.835989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:09:05.230238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.846974+0000","last_change":"2024-09-12T03:19:24.846974+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.846974+0000","last_undegraded":"2024-09-12T03:19:24.846974+0000","last_fullsized":"2024-09-12T03:19:24.846974+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T03:19:24.863848+0000","last_change":"2024-09-12T03:19:24.860398+0000","last_active":"2024-09-12T03:19:24.863848+0000","last_peered":"2024-09-12T03:19:24.863848+0000","last_clean":"2024-09-12T03:19:24.863848+0000","last_became_active":"2024-09-12T03:19:24.860093+0000","last_became_peered":"2024-09-12T03:19:24.860093+0000","last_unstale":"2024-09-12T03:19:24.863848+0000","last_undegraded":"2024-09-12T03:19:24.863848+0000","last_fullsized":"2024-09-12T03:19:24.863848+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_clean_scrub_stamp":"2024-09-12T03:19:23.835989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:40:45.875415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.852543+0000","last_change":"2024-09-12T03:19:24.852543+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.852543+0000","last_undegraded":"2024-09-12T03:19:24.852543+0000","last_fullsized":"2024-09-12T03:19:24.852543+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T03:19:24.864154+0000","last_change":"2024-09-12T03:19:24.858756+0000","last_active":"2024-09-12T03:19:24.864154+0000","last_peered":"2024-09-12T03:19:24.864154+0000","last_clean":"2024-09-12T03:19:24.864154+0000","last_became_active":"2024-09-12T03:19:24.858474+0000","last_became_peered":"2024-09-12T03:19:24.858474+0000","last_unstale":"2024-09-12T03:19:24.864154+0000","last_undegraded":"2024-09-12T03:19:24.864154+0000","last_fullsized":"2024-09-12T03:19:24.864154+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_clean_scrub_stamp":"2024-09-12T03:19:23.835989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:51:36.581086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.851171+0000","last_change":"2024-09-12T03:19:24.851171+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.851171+0000","last_undegraded":"2024-09-12T03:19:24.851171+0000","last_fullsized":"2024-09-12T03:19:24.851171+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T03:19:24.861322+0000","last_change":"2024-09-12T03:19:24.858024+0000","last_active":"2024-09-12T03:19:24.861322+0000","last_peered":"2024-09-12T03:19:24.861322+0000","last_clean":"2024-09-12T03:19:24.861322+0000","last_became_active":"2024-09-12T03:19:24.857142+0000","last_became_peered":"2024-09-12T03:19:24.857142+0000","last_unstale":"2024-09-12T03:19:24.861322+0000","last_undegraded":"2024-09-12T03:19:24.861322+0000","last_fullsized":"2024-09-12T03:19:24.861322+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_clean_scrub_stamp":"2024-09-12T03:19:23.835989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:59:02.611792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":228,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T03:19:24.853689+0000","last_change":"2024-09-12T03:16:21.883446+0000","last_active":"2024-09-12T03:19:24.853689+0000","last_peered":"2024-09-12T03:19:24.853689+0000","last_clean":"2024-09-12T03:19:24.853689+0000","last_became_active":"2024-09-12T03:16:21.882207+0000","last_became_peered":"2024-09-12T03:16:21.882207+0000","last_unstale":"2024-09-12T03:19:24.853689+0000","last_undegraded":"2024-09-12T03:19:24.853689+0000","last_fullsized":"2024-09-12T03:19:24.853689+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:20.316426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:20.316426+0000","last_clean_scrub_stamp":"2024-09-12T03:16:20.316426+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:12:15.424320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542183,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32396,"kb_used_data":1208,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339444,"statfs":{"total":96636764160,"available":96603590656,"internally_reserved":0,"allocated":1236992,"data_stored":853276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542185,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31816,"kb_used_data":616,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340024,"statfs":{"total":96636764160,"available":96604184576,"internally_reserved":0,"allocated":630784,"data_stored":256641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542184,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35964,"kb_used_data":616,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335876,"statfs":{"total":96636764160,"available":96599937024,"internally_reserved":0,"allocated":630784,"data_stored":256641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542183,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31820,"kb_used_data":616,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340020,"statfs":{"total":96636764160,"available":96604180480,"internally_reserved":0,"allocated":630784,"data_stored":256641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542184,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27656,"kb_used_data":616,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344184,"statfs":{"total":96636764160,"available":96608444416,"internally_reserved":0,"allocated":630784,"data_stored":256641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542184,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27648,"kb_used_data":616,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344192,"statfs":{"total":96636764160,"available":96608452608,"internally_reserved":0,"allocated":630784,"data_stored":256641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542185,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31808,"kb_used_data":616,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340032,"statfs":{"total":96636764160,"available":96604192768,"internally_reserved":0,"allocated":630784,"data_stored":256641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542185,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28224,"kb_used_data":1196,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343616,"statfs":{"total":96636764160,"available":96607862784,"internally_reserved":0,"allocated":1224704,"data_stored":847009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T03:19:28.833 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-12T03:19:28.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.971+0000 7f8d61578640 1 Processor -- start 2024-09-12T03:19:28.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.971+0000 7f8d61578640 1 -- start start 2024-09-12T03:19:28.973 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.972+0000 7f8d61578640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f8d5c06f360 0x7f8d5c0cef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:28.973 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.972+0000 7f8d61578640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f8d5c0cf490 0x7f8d5c0d2780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:28.974 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.972+0000 7f8d61578640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8d5c06cc40 con 0x7f8d5c0b1ca0 2024-09-12T03:19:28.974 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.972+0000 7f8d61578640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f8d5c0cf900 con 0x7f8d5c0cf490 2024-09-12T03:19:28.974 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.972+0000 7f8d61578640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f8d5c0d22f0 con 0x7f8d5c06f360 2024-09-12T03:19:28.974 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.972+0000 7f8d5b7fe640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f8d5c0cf490 0x7f8d5c0d2780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:28.974 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.972+0000 7f8d5b7fe640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f8d5c0cf490 0x7f8d5c0d2780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42508/0 (socket says 172.21.15.27:42508) 2024-09-12T03:19:28.974 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.972+0000 7f8d5b7fe640 1 -- 172.21.15.27:0/14107687 learned_addr learned my addr 172.21.15.27:0/14107687 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:19:28.974 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.972+0000 7f8d5a7fc640 1 --2- 172.21.15.27:0/14107687 >> v2:172.21.15.45:3300/0 conn(0x7f8d5c06f360 0x7f8d5c0cef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:28.974 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.972+0000 7f8d5b7fe640 1 -- 172.21.15.27:0/14107687 >> v2:172.21.15.45:3300/0 conn(0x7f8d5c06f360 msgr2=0x7f8d5c0cef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:28.974 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.972+0000 7f8d5b7fe640 1 --2- 172.21.15.27:0/14107687 >> v2:172.21.15.45:3300/0 conn(0x7f8d5c06f360 0x7f8d5c0cef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:28.974 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.972+0000 7f8d5b7fe640 1 -- 172.21.15.27:0/14107687 >> v1:172.21.15.45:6789/0 conn(0x7f8d5c0b1ca0 legacy=0x7f8d5c15f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:28.974 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.972+0000 7f8d5b7fe640 1 -- 172.21.15.27:0/14107687 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d5c0d1ce0 con 0x7f8d5c0cf490 2024-09-12T03:19:28.975 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.973+0000 7f8d5b7fe640 1 --2- 172.21.15.27:0/14107687 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8d5c0cf490 0x7f8d5c0d2780 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f8d5c0b5220 tx=0x7f8d5402a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bed859ade08b72b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:28.976 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.973+0000 7f8d59ffb640 1 -- 172.21.15.27:0/14107687 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d54036070 con 0x7f8d5c0cf490 2024-09-12T03:19:28.976 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.973+0000 7f8d59ffb640 1 -- 172.21.15.27:0/14107687 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d54031600 con 0x7f8d5c0cf490 2024-09-12T03:19:28.976 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.973+0000 7f8d59ffb640 1 -- 172.21.15.27:0/14107687 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d540319d0 con 0x7f8d5c0cf490 2024-09-12T03:19:28.976 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.974+0000 7f8d61578640 1 -- 172.21.15.27:0/14107687 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8d5c0cf490 msgr2=0x7f8d5c0d2780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:28.976 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.974+0000 7f8d61578640 1 --2- 172.21.15.27:0/14107687 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8d5c0cf490 0x7f8d5c0d2780 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f8d5c0b5220 tx=0x7f8d5402a730 comp rx=0 tx=0).stop 2024-09-12T03:19:28.976 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.974+0000 7f8d61578640 1 -- 172.21.15.27:0/14107687 shutdown_connections 2024-09-12T03:19:28.976 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.974+0000 7f8d61578640 1 --2- 172.21.15.27:0/14107687 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8d5c0cf490 0x7f8d5c0d2780 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:28.976 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.974+0000 7f8d61578640 1 --2- 172.21.15.27:0/14107687 >> v2:172.21.15.45:3300/0 conn(0x7f8d5c06f360 0x7f8d5c0cef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:28.976 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.974+0000 7f8d61578640 1 -- 172.21.15.27:0/14107687 >> 172.21.15.27:0/14107687 conn(0x7f8d5c073800 msgr2=0x7f8d5c09ba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:28.976 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.974+0000 7f8d61578640 1 -- 172.21.15.27:0/14107687 shutdown_connections 2024-09-12T03:19:28.976 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.974+0000 7f8d61578640 1 -- 172.21.15.27:0/14107687 wait complete. 2024-09-12T03:19:28.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.975+0000 7f8d61578640 1 Processor -- start 2024-09-12T03:19:28.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.975+0000 7f8d61578640 1 -- start start 2024-09-12T03:19:28.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.976+0000 7f8d61578640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f8d5c06f360 0x7f8d5c0a0130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:28.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.976+0000 7f8d61578640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8d5c0b1ca0 0x7f8d5c15f250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:28.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.976+0000 7f8d61578640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8d5c0cf490 0x7f8d5c0cde90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:28.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.976+0000 7f8d61578640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f8d5c06d1c0 con 0x7f8d5c0cf490 2024-09-12T03:19:28.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.976+0000 7f8d61578640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f8d5c0d1f80 con 0x7f8d5c0b1ca0 2024-09-12T03:19:28.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.976+0000 7f8d61578640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f8d5c06d340 con 0x7f8d5c06f360 2024-09-12T03:19:28.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.976+0000 7f8d5b7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8d5c0cf490 0x7f8d5c0cde90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:28.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.976+0000 7f8d5b7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8d5c0cf490 0x7f8d5c0cde90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42516/0 (socket says 172.21.15.27:42516) 2024-09-12T03:19:28.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.976+0000 7f8d5b7fe640 1 -- 172.21.15.27:0/1771692179 learned_addr learned my addr 172.21.15.27:0/1771692179 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:19:28.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.976+0000 7f8d5affd640 1 --2- 172.21.15.27:0/1771692179 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f8d5c06f360 0x7f8d5c0a0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:28.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.976+0000 7f8d5a7fc640 1 --2- 172.21.15.27:0/1771692179 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8d5c0b1ca0 0x7f8d5c15f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:28.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.976+0000 7f8d5b7fe640 1 -- 172.21.15.27:0/1771692179 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f8d5c06f360 msgr2=0x7f8d5c0a0130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:28.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.976+0000 7f8d5b7fe640 1 --2- 172.21.15.27:0/1771692179 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f8d5c06f360 0x7f8d5c0a0130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:28.980 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.976+0000 7f8d5b7fe640 1 -- 172.21.15.27:0/1771692179 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8d5c0b1ca0 msgr2=0x7f8d5c15f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:28.980 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.976+0000 7f8d5b7fe640 1 --2- 172.21.15.27:0/1771692179 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8d5c0b1ca0 0x7f8d5c15f250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:28.980 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.976+0000 7f8d5b7fe640 1 -- 172.21.15.27:0/1771692179 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d5c0a0670 con 0x7f8d5c0cf490 2024-09-12T03:19:28.980 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.977+0000 7f8d5b7fe640 1 --2- 172.21.15.27:0/1771692179 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8d5c0cf490 0x7f8d5c0cde90 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f8d54009f40 tx=0x7f8d54004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:28.980 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.977+0000 7f8d437fe640 1 -- 172.21.15.27:0/1771692179 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d54036050 con 0x7f8d5c0cf490 2024-09-12T03:19:28.980 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.977+0000 7f8d437fe640 1 -- 172.21.15.27:0/1771692179 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d54042070 con 0x7f8d5c0cf490 2024-09-12T03:19:28.980 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.977+0000 7f8d61578640 1 -- 172.21.15.27:0/1771692179 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d5c15fcd0 con 0x7f8d5c0cf490 2024-09-12T03:19:28.980 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.977+0000 7f8d437fe640 1 -- 172.21.15.27:0/1771692179 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d54009510 con 0x7f8d5c0cf490 2024-09-12T03:19:28.980 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.977+0000 7f8d61578640 1 -- 172.21.15.27:0/1771692179 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d5c160290 con 0x7f8d5c0cf490 2024-09-12T03:19:28.981 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.978+0000 7f8d61578640 1 -- 172.21.15.27:0/1771692179 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d5c0cfc40 con 0x7f8d5c0cf490 2024-09-12T03:19:28.981 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.978+0000 7f8d437fe640 1 -- 172.21.15.27:0/1771692179 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 9) ==== 90651+0+0 (secure 0 0 0) 0x7f8d54007bf0 con 0x7f8d5c0cf490 2024-09-12T03:19:28.981 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.979+0000 7f8d437fe640 1 --2- 172.21.15.27:0/1771692179 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f8d4806bfe0 0x7f8d4806e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:28.981 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.979+0000 7f8d437fe640 1 -- 172.21.15.27:0/1771692179 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5709+0+0 (secure 0 0 0) 0x7f8d540ac030 con 0x7f8d5c0cf490 2024-09-12T03:19:28.982 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.979+0000 7f8d5affd640 1 --2- 172.21.15.27:0/1771692179 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f8d4806bfe0 0x7f8d4806e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:28.982 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.979+0000 7f8d5affd640 1 --2- 172.21.15.27:0/1771692179 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f8d4806bfe0 0x7f8d4806e4a0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f8d4c0015a0 tx=0x7f8d4c009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:28.984 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:28.983+0000 7f8d437fe640 1 -- 172.21.15.27:0/1771692179 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f8d5407a6e0 con 0x7f8d5c0cf490 2024-09-12T03:19:29.156 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:29.155+0000 7f8d61578640 1 -- 172.21.15.27:0/1771692179 --> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8d5c1f4390 con 0x7f8d4806bfe0 2024-09-12T03:19:29.162 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:29.161+0000 7f8d437fe640 1 -- 172.21.15.27:0/1771692179 <== mgr.4102 v2:172.21.15.45:6832/3923821219 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+246638 (secure 0 0 0) 0x7f8d5c1f4390 con 0x7f8d4806bfe0 2024-09-12T03:19:29.165 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:19:29.166 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2024-09-12T03:19:29.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:29.164+0000 7f8d61578640 1 -- 172.21.15.27:0/1771692179 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f8d4806bfe0 msgr2=0x7f8d4806e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:29.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:29.165+0000 7f8d61578640 1 --2- 172.21.15.27:0/1771692179 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f8d4806bfe0 0x7f8d4806e4a0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f8d4c0015a0 tx=0x7f8d4c009040 comp rx=0 tx=0).stop 2024-09-12T03:19:29.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:29.165+0000 7f8d61578640 1 -- 172.21.15.27:0/1771692179 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8d5c0cf490 msgr2=0x7f8d5c0cde90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:29.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:29.165+0000 7f8d61578640 1 --2- 172.21.15.27:0/1771692179 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8d5c0cf490 0x7f8d5c0cde90 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f8d54009f40 tx=0x7f8d54004a40 comp rx=0 tx=0).stop 2024-09-12T03:19:29.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:29.165+0000 7f8d61578640 1 -- 172.21.15.27:0/1771692179 shutdown_connections 2024-09-12T03:19:29.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:29.165+0000 7f8d61578640 1 --2- 172.21.15.27:0/1771692179 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f8d4806bfe0 0x7f8d4806e4a0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:29.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:29.165+0000 7f8d61578640 1 --2- 172.21.15.27:0/1771692179 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8d5c0cf490 0x7f8d5c0cde90 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:29.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:29.165+0000 7f8d61578640 1 --2- 172.21.15.27:0/1771692179 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8d5c0b1ca0 0x7f8d5c15f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:29.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:29.165+0000 7f8d61578640 1 --2- 172.21.15.27:0/1771692179 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f8d5c06f360 0x7f8d5c0a0130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:29.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:29.165+0000 7f8d61578640 1 -- 172.21.15.27:0/1771692179 >> 172.21.15.27:0/1771692179 conn(0x7f8d5c073800 msgr2=0x7f8d5c06f7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:29.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:29.165+0000 7f8d61578640 1 -- 172.21.15.27:0/1771692179 shutdown_connections 2024-09-12T03:19:29.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:29.165+0000 7f8d61578640 1 -- 172.21.15.27:0/1771692179 wait complete. 2024-09-12T03:19:29.176 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":143,"stamp":"2024-09-12T03:19:28.095434+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":29,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":247332,"kb_used_data":6100,"kb_used_omap":12,"kb_used_meta":241011,"kb_avail":754727388,"statfs":{"total":773094113280,"available":772840845312,"internally_reserved":0,"allocated":6246400,"data_stored":3240131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":246795824},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":20,"apply_latency_ms":20,"commit_latency_ns":20000000,"apply_latency_ns":20000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.256497"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.851741+0000","last_change":"2024-09-12T03:19:24.851741+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.851741+0000","last_undegraded":"2024-09-12T03:19:24.851741+0000","last_fullsized":"2024-09-12T03:19:24.851741+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.853689+0000","last_change":"2024-09-12T03:19:24.853689+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.853689+0000","last_undegraded":"2024-09-12T03:19:24.853689+0000","last_fullsized":"2024-09-12T03:19:24.853689+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.853107+0000","last_change":"2024-09-12T03:19:24.853107+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.853107+0000","last_undegraded":"2024-09-12T03:19:24.853107+0000","last_fullsized":"2024-09-12T03:19:24.853107+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.849111+0000","last_change":"2024-09-12T03:19:24.849111+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.849111+0000","last_undegraded":"2024-09-12T03:19:24.849111+0000","last_fullsized":"2024-09-12T03:19:24.849111+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.852648+0000","last_change":"2024-09-12T03:19:24.852648+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.852648+0000","last_undegraded":"2024-09-12T03:19:24.852648+0000","last_fullsized":"2024-09-12T03:19:24.852648+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.854574+0000","last_change":"2024-09-12T03:19:24.854574+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.854574+0000","last_undegraded":"2024-09-12T03:19:24.854574+0000","last_fullsized":"2024-09-12T03:19:24.854574+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.846611+0000","last_change":"2024-09-12T03:19:24.846611+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.846611+0000","last_undegraded":"2024-09-12T03:19:24.846611+0000","last_fullsized":"2024-09-12T03:19:24.846611+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.855228+0000","last_change":"2024-09-12T03:19:24.855228+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.855228+0000","last_undegraded":"2024-09-12T03:19:24.855228+0000","last_fullsized":"2024-09-12T03:19:24.855228+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.845883+0000","last_change":"2024-09-12T03:19:24.845883+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.845883+0000","last_undegraded":"2024-09-12T03:19:24.845883+0000","last_fullsized":"2024-09-12T03:19:24.845883+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.848151+0000","last_change":"2024-09-12T03:19:24.848151+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.848151+0000","last_undegraded":"2024-09-12T03:19:24.848151+0000","last_fullsized":"2024-09-12T03:19:24.848151+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.848151+0000","last_change":"2024-09-12T03:19:24.848151+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.848151+0000","last_undegraded":"2024-09-12T03:19:24.848151+0000","last_fullsized":"2024-09-12T03:19:24.848151+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.848357+0000","last_change":"2024-09-12T03:19:24.848357+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.848357+0000","last_undegraded":"2024-09-12T03:19:24.848357+0000","last_fullsized":"2024-09-12T03:19:24.848357+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.851899+0000","last_change":"2024-09-12T03:19:24.851899+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.851899+0000","last_undegraded":"2024-09-12T03:19:24.851899+0000","last_fullsized":"2024-09-12T03:19:24.851899+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.847361+0000","last_change":"2024-09-12T03:19:24.847361+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.847361+0000","last_undegraded":"2024-09-12T03:19:24.847361+0000","last_fullsized":"2024-09-12T03:19:24.847361+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.845866+0000","last_change":"2024-09-12T03:19:24.845866+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.845866+0000","last_undegraded":"2024-09-12T03:19:24.845866+0000","last_fullsized":"2024-09-12T03:19:24.845866+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.852548+0000","last_change":"2024-09-12T03:19:24.852548+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.852548+0000","last_undegraded":"2024-09-12T03:19:24.852548+0000","last_fullsized":"2024-09-12T03:19:24.852548+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.850960+0000","last_change":"2024-09-12T03:19:24.850960+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.850960+0000","last_undegraded":"2024-09-12T03:19:24.850960+0000","last_fullsized":"2024-09-12T03:19:24.850960+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.852064+0000","last_change":"2024-09-12T03:19:24.852064+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.852064+0000","last_undegraded":"2024-09-12T03:19:24.852064+0000","last_fullsized":"2024-09-12T03:19:24.852064+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.850808+0000","last_change":"2024-09-12T03:19:24.850808+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.850808+0000","last_undegraded":"2024-09-12T03:19:24.850808+0000","last_fullsized":"2024-09-12T03:19:24.850808+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.850087+0000","last_change":"2024-09-12T03:19:24.850087+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.850087+0000","last_undegraded":"2024-09-12T03:19:24.850087+0000","last_fullsized":"2024-09-12T03:19:24.850087+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.851577+0000","last_change":"2024-09-12T03:19:24.851577+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.851577+0000","last_undegraded":"2024-09-12T03:19:24.851577+0000","last_fullsized":"2024-09-12T03:19:24.851577+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.856451+0000","last_change":"2024-09-12T03:19:24.856451+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.856451+0000","last_undegraded":"2024-09-12T03:19:24.856451+0000","last_fullsized":"2024-09-12T03:19:24.856451+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.852682+0000","last_change":"2024-09-12T03:19:24.852682+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.852682+0000","last_undegraded":"2024-09-12T03:19:24.852682+0000","last_fullsized":"2024-09-12T03:19:24.852682+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.852444+0000","last_change":"2024-09-12T03:19:24.852444+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.852444+0000","last_undegraded":"2024-09-12T03:19:24.852444+0000","last_fullsized":"2024-09-12T03:19:24.852444+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.851312+0000","last_change":"2024-09-12T03:19:24.851312+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.851312+0000","last_undegraded":"2024-09-12T03:19:24.851312+0000","last_fullsized":"2024-09-12T03:19:24.851312+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.848902+0000","last_change":"2024-09-12T03:19:24.848902+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.848902+0000","last_undegraded":"2024-09-12T03:19:24.848902+0000","last_fullsized":"2024-09-12T03:19:24.848902+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.855319+0000","last_change":"2024-09-12T03:19:24.855319+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.855319+0000","last_undegraded":"2024-09-12T03:19:24.855319+0000","last_fullsized":"2024-09-12T03:19:24.855319+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.851621+0000","last_change":"2024-09-12T03:19:24.851621+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.851621+0000","last_undegraded":"2024-09-12T03:19:24.851621+0000","last_fullsized":"2024-09-12T03:19:24.851621+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.850354+0000","last_change":"2024-09-12T03:19:24.850354+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.850354+0000","last_undegraded":"2024-09-12T03:19:24.850354+0000","last_fullsized":"2024-09-12T03:19:24.850354+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.850218+0000","last_change":"2024-09-12T03:19:24.850218+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.850218+0000","last_undegraded":"2024-09-12T03:19:24.850218+0000","last_fullsized":"2024-09-12T03:19:24.850218+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.850269+0000","last_change":"2024-09-12T03:19:24.850269+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.850269+0000","last_undegraded":"2024-09-12T03:19:24.850269+0000","last_fullsized":"2024-09-12T03:19:24.850269+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.852116+0000","last_change":"2024-09-12T03:19:24.852116+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.852116+0000","last_undegraded":"2024-09-12T03:19:24.852116+0000","last_fullsized":"2024-09-12T03:19:24.852116+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.856894+0000","last_change":"2024-09-12T03:19:24.856894+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.856894+0000","last_undegraded":"2024-09-12T03:19:24.856894+0000","last_fullsized":"2024-09-12T03:19:24.856894+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T03:19:24.869900+0000","last_change":"2024-09-12T03:19:24.860694+0000","last_active":"2024-09-12T03:19:24.869900+0000","last_peered":"2024-09-12T03:19:24.869900+0000","last_clean":"2024-09-12T03:19:24.869900+0000","last_became_active":"2024-09-12T03:19:24.860380+0000","last_became_peered":"2024-09-12T03:19:24.860380+0000","last_unstale":"2024-09-12T03:19:24.869900+0000","last_undegraded":"2024-09-12T03:19:24.869900+0000","last_fullsized":"2024-09-12T03:19:24.869900+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_clean_scrub_stamp":"2024-09-12T03:19:23.835989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:22:23.745090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.848080+0000","last_change":"2024-09-12T03:19:24.848080+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.848080+0000","last_undegraded":"2024-09-12T03:19:24.848080+0000","last_fullsized":"2024-09-12T03:19:24.848080+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T03:19:24.866702+0000","last_change":"2024-09-12T03:19:24.863015+0000","last_active":"2024-09-12T03:19:24.866702+0000","last_peered":"2024-09-12T03:19:24.866702+0000","last_clean":"2024-09-12T03:19:24.866702+0000","last_became_active":"2024-09-12T03:19:24.862457+0000","last_became_peered":"2024-09-12T03:19:24.862457+0000","last_unstale":"2024-09-12T03:19:24.866702+0000","last_undegraded":"2024-09-12T03:19:24.866702+0000","last_fullsized":"2024-09-12T03:19:24.866702+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_clean_scrub_stamp":"2024-09-12T03:19:23.835989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:25:17.547373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.854180+0000","last_change":"2024-09-12T03:19:24.854180+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.854180+0000","last_undegraded":"2024-09-12T03:19:24.854180+0000","last_fullsized":"2024-09-12T03:19:24.854180+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T03:19:24.869928+0000","last_change":"2024-09-12T03:19:24.862226+0000","last_active":"2024-09-12T03:19:24.869928+0000","last_peered":"2024-09-12T03:19:24.869928+0000","last_clean":"2024-09-12T03:19:24.869928+0000","last_became_active":"2024-09-12T03:19:24.861956+0000","last_became_peered":"2024-09-12T03:19:24.861956+0000","last_unstale":"2024-09-12T03:19:24.869928+0000","last_undegraded":"2024-09-12T03:19:24.869928+0000","last_fullsized":"2024-09-12T03:19:24.869928+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_clean_scrub_stamp":"2024-09-12T03:19:23.835989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:56:28.082698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.851109+0000","last_change":"2024-09-12T03:19:24.851109+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.851109+0000","last_undegraded":"2024-09-12T03:19:24.851109+0000","last_fullsized":"2024-09-12T03:19:24.851109+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T03:19:24.861878+0000","last_change":"2024-09-12T03:19:24.859901+0000","last_active":"2024-09-12T03:19:24.861878+0000","last_peered":"2024-09-12T03:19:24.861878+0000","last_clean":"2024-09-12T03:19:24.861878+0000","last_became_active":"2024-09-12T03:19:24.859637+0000","last_became_peered":"2024-09-12T03:19:24.859637+0000","last_unstale":"2024-09-12T03:19:24.861878+0000","last_undegraded":"2024-09-12T03:19:24.861878+0000","last_fullsized":"2024-09-12T03:19:24.861878+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_clean_scrub_stamp":"2024-09-12T03:19:23.835989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:48:14.537533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.853953+0000","last_change":"2024-09-12T03:19:24.853953+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.853953+0000","last_undegraded":"2024-09-12T03:19:24.853953+0000","last_fullsized":"2024-09-12T03:19:24.853953+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T03:19:24.858328+0000","last_change":"2024-09-12T03:19:24.856981+0000","last_active":"2024-09-12T03:19:24.858328+0000","last_peered":"2024-09-12T03:19:24.858328+0000","last_clean":"2024-09-12T03:19:24.858328+0000","last_became_active":"2024-09-12T03:19:24.856607+0000","last_became_peered":"2024-09-12T03:19:24.856607+0000","last_unstale":"2024-09-12T03:19:24.858328+0000","last_undegraded":"2024-09-12T03:19:24.858328+0000","last_fullsized":"2024-09-12T03:19:24.858328+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_clean_scrub_stamp":"2024-09-12T03:19:23.835989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:56:55.814953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.851504+0000","last_change":"2024-09-12T03:19:24.851504+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.851504+0000","last_undegraded":"2024-09-12T03:19:24.851504+0000","last_fullsized":"2024-09-12T03:19:24.851504+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T03:19:24.869885+0000","last_change":"2024-09-12T03:19:24.861305+0000","last_active":"2024-09-12T03:19:24.869885+0000","last_peered":"2024-09-12T03:19:24.869885+0000","last_clean":"2024-09-12T03:19:24.869885+0000","last_became_active":"2024-09-12T03:19:24.861021+0000","last_became_peered":"2024-09-12T03:19:24.861021+0000","last_unstale":"2024-09-12T03:19:24.869885+0000","last_undegraded":"2024-09-12T03:19:24.869885+0000","last_fullsized":"2024-09-12T03:19:24.869885+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_clean_scrub_stamp":"2024-09-12T03:19:23.835989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:52:21.815077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.851467+0000","last_change":"2024-09-12T03:19:24.851467+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.851467+0000","last_undegraded":"2024-09-12T03:19:24.851467+0000","last_fullsized":"2024-09-12T03:19:24.851467+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T03:19:24.854193+0000","last_change":"2024-09-12T03:19:24.850360+0000","last_active":"2024-09-12T03:19:24.854193+0000","last_peered":"2024-09-12T03:19:24.854193+0000","last_clean":"2024-09-12T03:19:24.854193+0000","last_became_active":"2024-09-12T03:19:24.850027+0000","last_became_peered":"2024-09-12T03:19:24.850027+0000","last_unstale":"2024-09-12T03:19:24.854193+0000","last_undegraded":"2024-09-12T03:19:24.854193+0000","last_fullsized":"2024-09-12T03:19:24.854193+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_clean_scrub_stamp":"2024-09-12T03:19:23.835989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:35:50.136024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.853049+0000","last_change":"2024-09-12T03:19:24.853049+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.853049+0000","last_undegraded":"2024-09-12T03:19:24.853049+0000","last_fullsized":"2024-09-12T03:19:24.853049+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T03:19:24.864569+0000","last_change":"2024-09-12T03:19:24.863770+0000","last_active":"2024-09-12T03:19:24.864569+0000","last_peered":"2024-09-12T03:19:24.864569+0000","last_clean":"2024-09-12T03:19:24.864569+0000","last_became_active":"2024-09-12T03:19:24.863471+0000","last_became_peered":"2024-09-12T03:19:24.863471+0000","last_unstale":"2024-09-12T03:19:24.864569+0000","last_undegraded":"2024-09-12T03:19:24.864569+0000","last_fullsized":"2024-09-12T03:19:24.864569+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_clean_scrub_stamp":"2024-09-12T03:19:23.835989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:00:17.289844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.846718+0000","last_change":"2024-09-12T03:19:24.846718+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.846718+0000","last_undegraded":"2024-09-12T03:19:24.846718+0000","last_fullsized":"2024-09-12T03:19:24.846718+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T03:19:24.870959+0000","last_change":"2024-09-12T03:19:24.870526+0000","last_active":"2024-09-12T03:19:24.870959+0000","last_peered":"2024-09-12T03:19:24.870959+0000","last_clean":"2024-09-12T03:19:24.870959+0000","last_became_active":"2024-09-12T03:19:24.870226+0000","last_became_peered":"2024-09-12T03:19:24.870226+0000","last_unstale":"2024-09-12T03:19:24.870959+0000","last_undegraded":"2024-09-12T03:19:24.870959+0000","last_fullsized":"2024-09-12T03:19:24.870959+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_clean_scrub_stamp":"2024-09-12T03:19:23.835989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:43:45.344248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.849991+0000","last_change":"2024-09-12T03:19:24.849991+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.849991+0000","last_undegraded":"2024-09-12T03:19:24.849991+0000","last_fullsized":"2024-09-12T03:19:24.849991+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T03:19:24.866683+0000","last_change":"2024-09-12T03:19:24.862733+0000","last_active":"2024-09-12T03:19:24.866683+0000","last_peered":"2024-09-12T03:19:24.866683+0000","last_clean":"2024-09-12T03:19:24.866683+0000","last_became_active":"2024-09-12T03:19:24.862164+0000","last_became_peered":"2024-09-12T03:19:24.862164+0000","last_unstale":"2024-09-12T03:19:24.866683+0000","last_undegraded":"2024-09-12T03:19:24.866683+0000","last_fullsized":"2024-09-12T03:19:24.866683+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_clean_scrub_stamp":"2024-09-12T03:19:23.835989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:50:19.592853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.848059+0000","last_change":"2024-09-12T03:19:24.848059+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.848059+0000","last_undegraded":"2024-09-12T03:19:24.848059+0000","last_fullsized":"2024-09-12T03:19:24.848059+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T03:19:24.860582+0000","last_change":"2024-09-12T03:19:24.859732+0000","last_active":"2024-09-12T03:19:24.860582+0000","last_peered":"2024-09-12T03:19:24.860582+0000","last_clean":"2024-09-12T03:19:24.860582+0000","last_became_active":"2024-09-12T03:19:24.859452+0000","last_became_peered":"2024-09-12T03:19:24.859452+0000","last_unstale":"2024-09-12T03:19:24.860582+0000","last_undegraded":"2024-09-12T03:19:24.860582+0000","last_fullsized":"2024-09-12T03:19:24.860582+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_clean_scrub_stamp":"2024-09-12T03:19:23.835989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:16:48.958106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.852512+0000","last_change":"2024-09-12T03:19:24.852512+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.852512+0000","last_undegraded":"2024-09-12T03:19:24.852512+0000","last_fullsized":"2024-09-12T03:19:24.852512+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T03:19:24.862745+0000","last_change":"2024-09-12T03:19:24.861967+0000","last_active":"2024-09-12T03:19:24.862745+0000","last_peered":"2024-09-12T03:19:24.862745+0000","last_clean":"2024-09-12T03:19:24.862745+0000","last_became_active":"2024-09-12T03:19:24.861645+0000","last_became_peered":"2024-09-12T03:19:24.861645+0000","last_unstale":"2024-09-12T03:19:24.862745+0000","last_undegraded":"2024-09-12T03:19:24.862745+0000","last_fullsized":"2024-09-12T03:19:24.862745+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_clean_scrub_stamp":"2024-09-12T03:19:23.835989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:11:47.170009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.847408+0000","last_change":"2024-09-12T03:19:24.847408+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.847408+0000","last_undegraded":"2024-09-12T03:19:24.847408+0000","last_fullsized":"2024-09-12T03:19:24.847408+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T03:19:24.866725+0000","last_change":"2024-09-12T03:19:24.863067+0000","last_active":"2024-09-12T03:19:24.866725+0000","last_peered":"2024-09-12T03:19:24.866725+0000","last_clean":"2024-09-12T03:19:24.866725+0000","last_became_active":"2024-09-12T03:19:24.862471+0000","last_became_peered":"2024-09-12T03:19:24.862471+0000","last_unstale":"2024-09-12T03:19:24.866725+0000","last_undegraded":"2024-09-12T03:19:24.866725+0000","last_fullsized":"2024-09-12T03:19:24.866725+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_clean_scrub_stamp":"2024-09-12T03:19:23.835989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:04:37.233100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.851140+0000","last_change":"2024-09-12T03:19:24.851140+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.851140+0000","last_undegraded":"2024-09-12T03:19:24.851140+0000","last_fullsized":"2024-09-12T03:19:24.851140+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T03:19:24.862050+0000","last_change":"2024-09-12T03:19:24.859981+0000","last_active":"2024-09-12T03:19:24.862050+0000","last_peered":"2024-09-12T03:19:24.862050+0000","last_clean":"2024-09-12T03:19:24.862050+0000","last_became_active":"2024-09-12T03:19:24.859679+0000","last_became_peered":"2024-09-12T03:19:24.859679+0000","last_unstale":"2024-09-12T03:19:24.862050+0000","last_undegraded":"2024-09-12T03:19:24.862050+0000","last_fullsized":"2024-09-12T03:19:24.862050+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_clean_scrub_stamp":"2024-09-12T03:19:23.835989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:51:59.350797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.846337+0000","last_change":"2024-09-12T03:19:24.846337+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.846337+0000","last_undegraded":"2024-09-12T03:19:24.846337+0000","last_fullsized":"2024-09-12T03:19:24.846337+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T03:19:24.857304+0000","last_change":"2024-09-12T03:19:24.855795+0000","last_active":"2024-09-12T03:19:24.857304+0000","last_peered":"2024-09-12T03:19:24.857304+0000","last_clean":"2024-09-12T03:19:24.857304+0000","last_became_active":"2024-09-12T03:19:24.855377+0000","last_became_peered":"2024-09-12T03:19:24.855377+0000","last_unstale":"2024-09-12T03:19:24.857304+0000","last_undegraded":"2024-09-12T03:19:24.857304+0000","last_fullsized":"2024-09-12T03:19:24.857304+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_clean_scrub_stamp":"2024-09-12T03:19:23.835989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:50:07.794274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.852242+0000","last_change":"2024-09-12T03:19:24.852242+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.852242+0000","last_undegraded":"2024-09-12T03:19:24.852242+0000","last_fullsized":"2024-09-12T03:19:24.852242+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T03:19:24.860954+0000","last_change":"2024-09-12T03:19:24.859883+0000","last_active":"2024-09-12T03:19:24.860954+0000","last_peered":"2024-09-12T03:19:24.860954+0000","last_clean":"2024-09-12T03:19:24.860954+0000","last_became_active":"2024-09-12T03:19:24.859566+0000","last_became_peered":"2024-09-12T03:19:24.859566+0000","last_unstale":"2024-09-12T03:19:24.860954+0000","last_undegraded":"2024-09-12T03:19:24.860954+0000","last_fullsized":"2024-09-12T03:19:24.860954+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_clean_scrub_stamp":"2024-09-12T03:19:23.835989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:44:44.153823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.851328+0000","last_change":"2024-09-12T03:19:24.851328+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.851328+0000","last_undegraded":"2024-09-12T03:19:24.851328+0000","last_fullsized":"2024-09-12T03:19:24.851328+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T03:19:24.866633+0000","last_change":"2024-09-12T03:19:24.861891+0000","last_active":"2024-09-12T03:19:24.866633+0000","last_peered":"2024-09-12T03:19:24.866633+0000","last_clean":"2024-09-12T03:19:24.866633+0000","last_became_active":"2024-09-12T03:19:24.861524+0000","last_became_peered":"2024-09-12T03:19:24.861524+0000","last_unstale":"2024-09-12T03:19:24.866633+0000","last_undegraded":"2024-09-12T03:19:24.866633+0000","last_fullsized":"2024-09-12T03:19:24.866633+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_clean_scrub_stamp":"2024-09-12T03:19:23.835989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:16:02.571700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.848416+0000","last_change":"2024-09-12T03:19:24.848416+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.848416+0000","last_undegraded":"2024-09-12T03:19:24.848416+0000","last_fullsized":"2024-09-12T03:19:24.848416+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T03:19:24.872537+0000","last_change":"2024-09-12T03:19:24.865925+0000","last_active":"2024-09-12T03:19:24.872537+0000","last_peered":"2024-09-12T03:19:24.872537+0000","last_clean":"2024-09-12T03:19:24.872537+0000","last_became_active":"2024-09-12T03:19:24.865689+0000","last_became_peered":"2024-09-12T03:19:24.865689+0000","last_unstale":"2024-09-12T03:19:24.872537+0000","last_undegraded":"2024-09-12T03:19:24.872537+0000","last_fullsized":"2024-09-12T03:19:24.872537+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_clean_scrub_stamp":"2024-09-12T03:19:23.835989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:17:44.078064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.849465+0000","last_change":"2024-09-12T03:19:24.849465+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.849465+0000","last_undegraded":"2024-09-12T03:19:24.849465+0000","last_fullsized":"2024-09-12T03:19:24.849465+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T03:19:24.857771+0000","last_change":"2024-09-12T03:19:24.855690+0000","last_active":"2024-09-12T03:19:24.857771+0000","last_peered":"2024-09-12T03:19:24.857771+0000","last_clean":"2024-09-12T03:19:24.857771+0000","last_became_active":"2024-09-12T03:19:24.855284+0000","last_became_peered":"2024-09-12T03:19:24.855284+0000","last_unstale":"2024-09-12T03:19:24.857771+0000","last_undegraded":"2024-09-12T03:19:24.857771+0000","last_fullsized":"2024-09-12T03:19:24.857771+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_clean_scrub_stamp":"2024-09-12T03:19:23.835989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:38:35.821379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.851636+0000","last_change":"2024-09-12T03:19:24.851636+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.851636+0000","last_undegraded":"2024-09-12T03:19:24.851636+0000","last_fullsized":"2024-09-12T03:19:24.851636+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T03:19:24.872475+0000","last_change":"2024-09-12T03:19:24.864872+0000","last_active":"2024-09-12T03:19:24.872475+0000","last_peered":"2024-09-12T03:19:24.872475+0000","last_clean":"2024-09-12T03:19:24.872475+0000","last_became_active":"2024-09-12T03:19:24.864506+0000","last_became_peered":"2024-09-12T03:19:24.864506+0000","last_unstale":"2024-09-12T03:19:24.872475+0000","last_undegraded":"2024-09-12T03:19:24.872475+0000","last_fullsized":"2024-09-12T03:19:24.872475+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_clean_scrub_stamp":"2024-09-12T03:19:23.835989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:16:51.163113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.846915+0000","last_change":"2024-09-12T03:19:24.846915+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.846915+0000","last_undegraded":"2024-09-12T03:19:24.846915+0000","last_fullsized":"2024-09-12T03:19:24.846915+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T03:19:24.866700+0000","last_change":"2024-09-12T03:19:24.862743+0000","last_active":"2024-09-12T03:19:24.866700+0000","last_peered":"2024-09-12T03:19:24.866700+0000","last_clean":"2024-09-12T03:19:24.866700+0000","last_became_active":"2024-09-12T03:19:24.862186+0000","last_became_peered":"2024-09-12T03:19:24.862186+0000","last_unstale":"2024-09-12T03:19:24.866700+0000","last_undegraded":"2024-09-12T03:19:24.866700+0000","last_fullsized":"2024-09-12T03:19:24.866700+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_clean_scrub_stamp":"2024-09-12T03:19:23.835989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:04:27.775435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.855497+0000","last_change":"2024-09-12T03:19:24.855497+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.855497+0000","last_undegraded":"2024-09-12T03:19:24.855497+0000","last_fullsized":"2024-09-12T03:19:24.855497+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T03:19:24.860844+0000","last_change":"2024-09-12T03:19:24.857052+0000","last_active":"2024-09-12T03:19:24.860844+0000","last_peered":"2024-09-12T03:19:24.860844+0000","last_clean":"2024-09-12T03:19:24.860844+0000","last_became_active":"2024-09-12T03:19:24.856771+0000","last_became_peered":"2024-09-12T03:19:24.856771+0000","last_unstale":"2024-09-12T03:19:24.860844+0000","last_undegraded":"2024-09-12T03:19:24.860844+0000","last_fullsized":"2024-09-12T03:19:24.860844+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_clean_scrub_stamp":"2024-09-12T03:19:23.835989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:35:47.557902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.852418+0000","last_change":"2024-09-12T03:19:24.852418+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.852418+0000","last_undegraded":"2024-09-12T03:19:24.852418+0000","last_fullsized":"2024-09-12T03:19:24.852418+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T03:19:24.864938+0000","last_change":"2024-09-12T03:19:24.864514+0000","last_active":"2024-09-12T03:19:24.864938+0000","last_peered":"2024-09-12T03:19:24.864938+0000","last_clean":"2024-09-12T03:19:24.864938+0000","last_became_active":"2024-09-12T03:19:24.864206+0000","last_became_peered":"2024-09-12T03:19:24.864206+0000","last_unstale":"2024-09-12T03:19:24.864938+0000","last_undegraded":"2024-09-12T03:19:24.864938+0000","last_fullsized":"2024-09-12T03:19:24.864938+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_clean_scrub_stamp":"2024-09-12T03:19:23.835989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:24:19.872805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.847841+0000","last_change":"2024-09-12T03:19:24.847841+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.847841+0000","last_undegraded":"2024-09-12T03:19:24.847841+0000","last_fullsized":"2024-09-12T03:19:24.847841+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T03:19:24.866519+0000","last_change":"2024-09-12T03:19:24.863777+0000","last_active":"2024-09-12T03:19:24.866519+0000","last_peered":"2024-09-12T03:19:24.866519+0000","last_clean":"2024-09-12T03:19:24.866519+0000","last_became_active":"2024-09-12T03:19:24.863506+0000","last_became_peered":"2024-09-12T03:19:24.863506+0000","last_unstale":"2024-09-12T03:19:24.866519+0000","last_undegraded":"2024-09-12T03:19:24.866519+0000","last_fullsized":"2024-09-12T03:19:24.866519+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_clean_scrub_stamp":"2024-09-12T03:19:23.835989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:53:10.464555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.847949+0000","last_change":"2024-09-12T03:19:24.847949+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.847949+0000","last_undegraded":"2024-09-12T03:19:24.847949+0000","last_fullsized":"2024-09-12T03:19:24.847949+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T03:19:24.872512+0000","last_change":"2024-09-12T03:19:24.865383+0000","last_active":"2024-09-12T03:19:24.872512+0000","last_peered":"2024-09-12T03:19:24.872512+0000","last_clean":"2024-09-12T03:19:24.872512+0000","last_became_active":"2024-09-12T03:19:24.865031+0000","last_became_peered":"2024-09-12T03:19:24.865031+0000","last_unstale":"2024-09-12T03:19:24.872512+0000","last_undegraded":"2024-09-12T03:19:24.872512+0000","last_fullsized":"2024-09-12T03:19:24.872512+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_clean_scrub_stamp":"2024-09-12T03:19:23.835989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:36:23.795897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.847544+0000","last_change":"2024-09-12T03:19:24.847544+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.847544+0000","last_undegraded":"2024-09-12T03:19:24.847544+0000","last_fullsized":"2024-09-12T03:19:24.847544+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T03:19:24.861103+0000","last_change":"2024-09-12T03:19:24.857501+0000","last_active":"2024-09-12T03:19:24.861103+0000","last_peered":"2024-09-12T03:19:24.861103+0000","last_clean":"2024-09-12T03:19:24.861103+0000","last_became_active":"2024-09-12T03:19:24.857129+0000","last_became_peered":"2024-09-12T03:19:24.857129+0000","last_unstale":"2024-09-12T03:19:24.861103+0000","last_undegraded":"2024-09-12T03:19:24.861103+0000","last_fullsized":"2024-09-12T03:19:24.861103+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_clean_scrub_stamp":"2024-09-12T03:19:23.835989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:54:09.253982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.850621+0000","last_change":"2024-09-12T03:19:24.850621+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.850621+0000","last_undegraded":"2024-09-12T03:19:24.850621+0000","last_fullsized":"2024-09-12T03:19:24.850621+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T03:19:24.856019+0000","last_change":"2024-09-12T03:19:24.852831+0000","last_active":"2024-09-12T03:19:24.856019+0000","last_peered":"2024-09-12T03:19:24.856019+0000","last_clean":"2024-09-12T03:19:24.856019+0000","last_became_active":"2024-09-12T03:19:24.851427+0000","last_became_peered":"2024-09-12T03:19:24.851427+0000","last_unstale":"2024-09-12T03:19:24.856019+0000","last_undegraded":"2024-09-12T03:19:24.856019+0000","last_fullsized":"2024-09-12T03:19:24.856019+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_clean_scrub_stamp":"2024-09-12T03:19:23.835989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:39:10.957546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.853481+0000","last_change":"2024-09-12T03:19:24.853481+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.853481+0000","last_undegraded":"2024-09-12T03:19:24.853481+0000","last_fullsized":"2024-09-12T03:19:24.853481+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T03:19:24.855106+0000","last_change":"2024-09-12T03:19:24.849318+0000","last_active":"2024-09-12T03:19:24.855106+0000","last_peered":"2024-09-12T03:19:24.855106+0000","last_clean":"2024-09-12T03:19:24.855106+0000","last_became_active":"2024-09-12T03:19:24.848981+0000","last_became_peered":"2024-09-12T03:19:24.848981+0000","last_unstale":"2024-09-12T03:19:24.855106+0000","last_undegraded":"2024-09-12T03:19:24.855106+0000","last_fullsized":"2024-09-12T03:19:24.855106+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_clean_scrub_stamp":"2024-09-12T03:19:23.835989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:07:42.042664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.850646+0000","last_change":"2024-09-12T03:19:24.850646+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.850646+0000","last_undegraded":"2024-09-12T03:19:24.850646+0000","last_fullsized":"2024-09-12T03:19:24.850646+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T03:19:24.862763+0000","last_change":"2024-09-12T03:19:24.861998+0000","last_active":"2024-09-12T03:19:24.862763+0000","last_peered":"2024-09-12T03:19:24.862763+0000","last_clean":"2024-09-12T03:19:24.862763+0000","last_became_active":"2024-09-12T03:19:24.861694+0000","last_became_peered":"2024-09-12T03:19:24.861694+0000","last_unstale":"2024-09-12T03:19:24.862763+0000","last_undegraded":"2024-09-12T03:19:24.862763+0000","last_fullsized":"2024-09-12T03:19:24.862763+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_clean_scrub_stamp":"2024-09-12T03:19:23.835989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:09:05.230238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.846974+0000","last_change":"2024-09-12T03:19:24.846974+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.846974+0000","last_undegraded":"2024-09-12T03:19:24.846974+0000","last_fullsized":"2024-09-12T03:19:24.846974+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T03:19:24.863848+0000","last_change":"2024-09-12T03:19:24.860398+0000","last_active":"2024-09-12T03:19:24.863848+0000","last_peered":"2024-09-12T03:19:24.863848+0000","last_clean":"2024-09-12T03:19:24.863848+0000","last_became_active":"2024-09-12T03:19:24.860093+0000","last_became_peered":"2024-09-12T03:19:24.860093+0000","last_unstale":"2024-09-12T03:19:24.863848+0000","last_undegraded":"2024-09-12T03:19:24.863848+0000","last_fullsized":"2024-09-12T03:19:24.863848+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_clean_scrub_stamp":"2024-09-12T03:19:23.835989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:40:45.875415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.852543+0000","last_change":"2024-09-12T03:19:24.852543+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.852543+0000","last_undegraded":"2024-09-12T03:19:24.852543+0000","last_fullsized":"2024-09-12T03:19:24.852543+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T03:19:24.864154+0000","last_change":"2024-09-12T03:19:24.858756+0000","last_active":"2024-09-12T03:19:24.864154+0000","last_peered":"2024-09-12T03:19:24.864154+0000","last_clean":"2024-09-12T03:19:24.864154+0000","last_became_active":"2024-09-12T03:19:24.858474+0000","last_became_peered":"2024-09-12T03:19:24.858474+0000","last_unstale":"2024-09-12T03:19:24.864154+0000","last_undegraded":"2024-09-12T03:19:24.864154+0000","last_fullsized":"2024-09-12T03:19:24.864154+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_clean_scrub_stamp":"2024-09-12T03:19:23.835989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:51:36.581086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.851171+0000","last_change":"2024-09-12T03:19:24.851171+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.851171+0000","last_undegraded":"2024-09-12T03:19:24.851171+0000","last_fullsized":"2024-09-12T03:19:24.851171+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T03:19:24.861322+0000","last_change":"2024-09-12T03:19:24.858024+0000","last_active":"2024-09-12T03:19:24.861322+0000","last_peered":"2024-09-12T03:19:24.861322+0000","last_clean":"2024-09-12T03:19:24.861322+0000","last_became_active":"2024-09-12T03:19:24.857142+0000","last_became_peered":"2024-09-12T03:19:24.857142+0000","last_unstale":"2024-09-12T03:19:24.861322+0000","last_undegraded":"2024-09-12T03:19:24.861322+0000","last_fullsized":"2024-09-12T03:19:24.861322+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_clean_scrub_stamp":"2024-09-12T03:19:23.835989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:59:02.611792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":228,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T03:19:24.853689+0000","last_change":"2024-09-12T03:16:21.883446+0000","last_active":"2024-09-12T03:19:24.853689+0000","last_peered":"2024-09-12T03:19:24.853689+0000","last_clean":"2024-09-12T03:19:24.853689+0000","last_became_active":"2024-09-12T03:16:21.882207+0000","last_became_peered":"2024-09-12T03:16:21.882207+0000","last_unstale":"2024-09-12T03:19:24.853689+0000","last_undegraded":"2024-09-12T03:19:24.853689+0000","last_fullsized":"2024-09-12T03:19:24.853689+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:20.316426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:20.316426+0000","last_clean_scrub_stamp":"2024-09-12T03:16:20.316426+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:12:15.424320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542183,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32396,"kb_used_data":1208,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339444,"statfs":{"total":96636764160,"available":96603590656,"internally_reserved":0,"allocated":1236992,"data_stored":853276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542185,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31816,"kb_used_data":616,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340024,"statfs":{"total":96636764160,"available":96604184576,"internally_reserved":0,"allocated":630784,"data_stored":256641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542184,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35964,"kb_used_data":616,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335876,"statfs":{"total":96636764160,"available":96599937024,"internally_reserved":0,"allocated":630784,"data_stored":256641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542183,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31820,"kb_used_data":616,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340020,"statfs":{"total":96636764160,"available":96604180480,"internally_reserved":0,"allocated":630784,"data_stored":256641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542184,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27656,"kb_used_data":616,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344184,"statfs":{"total":96636764160,"available":96608444416,"internally_reserved":0,"allocated":630784,"data_stored":256641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542184,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27648,"kb_used_data":616,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344192,"statfs":{"total":96636764160,"available":96608452608,"internally_reserved":0,"allocated":630784,"data_stored":256641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542185,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31808,"kb_used_data":616,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340032,"statfs":{"total":96636764160,"available":96604192768,"internally_reserved":0,"allocated":630784,"data_stored":256641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542185,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28224,"kb_used_data":1196,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343616,"statfs":{"total":96636764160,"available":96607862784,"internally_reserved":0,"allocated":1224704,"data_stored":847009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T03:19:29.180 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-12T03:19:29.319 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:29.318+0000 7f0e0ded7640 1 Processor -- start 2024-09-12T03:19:29.320 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:29.318+0000 7f0e0ded7640 1 -- start start 2024-09-12T03:19:29.321 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:29.319+0000 7f0e0ded7640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f0e080d1050 0x7f0e080cf810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:29.321 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:29.319+0000 7f0e0ded7640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0e080d1e60 con 0x7f0e080d52e0 2024-09-12T03:19:29.321 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:29.319+0000 7f0e0ded7640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0e080d2380 con 0x7f0e080d3120 2024-09-12T03:19:29.321 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:29.319+0000 7f0e0ded7640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f0e080cfef0 con 0x7f0e080d1050 2024-09-12T03:19:29.321 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:29.319+0000 7f0e077fe640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f0e080d1050 0x7f0e080cf810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:29.321 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:29.319+0000 7f0e06ffd640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f0e080d52e0 0x7f0e080d1430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:42364/0 (socket says 172.21.15.27:42364) 2024-09-12T03:19:29.321 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:29.319+0000 7f0e077fe640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f0e080d1050 0x7f0e080cf810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38338/0 (socket says 172.21.15.27:38338) 2024-09-12T03:19:29.321 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:29.319+0000 7f0e06ffd640 1 -- 172.21.15.27:0/3103434996 learned_addr learned my addr 172.21.15.27:0/3103434996 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:19:29.321 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:29.320+0000 7f0e077fe640 1 -- 172.21.15.27:0/3103434996 >> v1:172.21.15.45:6789/0 conn(0x7f0e080d3120 legacy=0x7f0e080d35b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:29.322 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:29.320+0000 7f0e077fe640 1 -- 172.21.15.27:0/3103434996 >> v1:172.21.15.27:6789/0 conn(0x7f0e080d52e0 legacy=0x7f0e080d1430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:29.322 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:29.320+0000 7f0e077fe640 1 -- 172.21.15.27:0/3103434996 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e080d2640 con 0x7f0e080d1050 2024-09-12T03:19:29.322 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:29.320+0000 7f0e067fc640 1 -- 172.21.15.27:0/3103434996 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1753460370 0 0) 0x7f0e080d1e60 con 0x7f0e080d52e0 2024-09-12T03:19:29.323 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:29.320+0000 7f0e077fe640 1 --2- 172.21.15.27:0/3103434996 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e080d1050 0x7f0e080cf810 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f0df80097a0 tx=0x7f0df802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db1df8882cdb2235 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:29.323 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:29.321+0000 7f0e067fc640 1 -- 172.21.15.27:0/3103434996 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0df8037070 con 0x7f0e080d1050 2024-09-12T03:19:29.323 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:29.321+0000 7f0e067fc640 1 -- 172.21.15.27:0/3103434996 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0df8032440 con 0x7f0e080d1050 2024-09-12T03:19:29.324 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:29.321+0000 7f0e067fc640 1 -- 172.21.15.27:0/3103434996 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0df8032810 con 0x7f0e080d1050 2024-09-12T03:19:29.324 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:29.321+0000 7f0e0ded7640 1 -- 172.21.15.27:0/3103434996 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e080d1050 msgr2=0x7f0e080cf810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:29.324 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:29.321+0000 7f0e0ded7640 1 --2- 172.21.15.27:0/3103434996 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e080d1050 0x7f0e080cf810 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f0df80097a0 tx=0x7f0df802a230 comp rx=0 tx=0).stop 2024-09-12T03:19:29.324 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:29.321+0000 7f0e0ded7640 1 -- 172.21.15.27:0/3103434996 shutdown_connections 2024-09-12T03:19:29.324 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:29.321+0000 7f0e0ded7640 1 --2- 172.21.15.27:0/3103434996 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e080d1050 0x7f0e080cf810 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:29.324 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:29.321+0000 7f0e0ded7640 1 -- 172.21.15.27:0/3103434996 >> 172.21.15.27:0/3103434996 conn(0x7f0e0806ddf0 msgr2=0x7f0e0806e200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:29.324 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:29.321+0000 7f0e0ded7640 1 -- 172.21.15.27:0/3103434996 shutdown_connections 2024-09-12T03:19:29.324 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:29.321+0000 7f0e0ded7640 1 -- 172.21.15.27:0/3103434996 wait complete. 2024-09-12T03:19:29.324 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:29.322+0000 7f0e0ded7640 1 Processor -- start 2024-09-12T03:19:29.326 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:29.323+0000 7f0e0ded7640 1 -- start start 2024-09-12T03:19:29.327 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:29.323+0000 7f0e0ded7640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f0e080d1050 0x7f0e080cafe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:29.327 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:29.323+0000 7f0e0ded7640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0e080d3120 0x7f0e080cb880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:29.327 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:29.323+0000 7f0e0ded7640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e080d52e0 0x7f0e0814cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:29.327 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:29.323+0000 7f0e0ded7640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f0e080b7a50 con 0x7f0e080d52e0 2024-09-12T03:19:29.327 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:29.323+0000 7f0e0ded7640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f0e080d2c00 con 0x7f0e080d3120 2024-09-12T03:19:29.327 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:29.323+0000 7f0e0ded7640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f0e080b7bd0 con 0x7f0e080d1050 2024-09-12T03:19:29.327 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:29.323+0000 7f0e07fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e080d52e0 0x7f0e0814cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:29.327 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:29.323+0000 7f0e07fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e080d52e0 0x7f0e0814cab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38350/0 (socket says 172.21.15.27:38350) 2024-09-12T03:19:29.327 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:29.323+0000 7f0e07fff640 1 -- 172.21.15.27:0/1663622090 learned_addr learned my addr 172.21.15.27:0/1663622090 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:19:29.327 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:29.324+0000 7f0e077fe640 1 --2- 172.21.15.27:0/1663622090 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f0e080d1050 0x7f0e080cafe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:29.327 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:29.324+0000 7f0e06ffd640 1 --2- 172.21.15.27:0/1663622090 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0e080d3120 0x7f0e080cb880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:29.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:29.324+0000 7f0e07fff640 1 -- 172.21.15.27:0/1663622090 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f0e080d1050 msgr2=0x7f0e080cafe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:29.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:29.324+0000 7f0e07fff640 1 --2- 172.21.15.27:0/1663622090 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f0e080d1050 0x7f0e080cafe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:29.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:29.324+0000 7f0e07fff640 1 -- 172.21.15.27:0/1663622090 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0e080d3120 msgr2=0x7f0e080cb880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:29.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:29.324+0000 7f0e07fff640 1 --2- 172.21.15.27:0/1663622090 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0e080d3120 0x7f0e080cb880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:29.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:29.324+0000 7f0e07fff640 1 -- 172.21.15.27:0/1663622090 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e080ce040 con 0x7f0e080d52e0 2024-09-12T03:19:29.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:29.324+0000 7f0e07fff640 1 --2- 172.21.15.27:0/1663622090 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e080d52e0 0x7f0e0814cab0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f0e0000c7a0 tx=0x7f0e0000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:29.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:29.324+0000 7f0deffff640 1 -- 172.21.15.27:0/1663622090 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e00003b70 con 0x7f0e080d52e0 2024-09-12T03:19:29.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:29.324+0000 7f0deffff640 1 -- 172.21.15.27:0/1663622090 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e00003d30 con 0x7f0e080d52e0 2024-09-12T03:19:29.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:29.324+0000 7f0deffff640 1 -- 172.21.15.27:0/1663622090 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e0000aa90 con 0x7f0e080d52e0 2024-09-12T03:19:29.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:29.324+0000 7f0e0ded7640 1 -- 172.21.15.27:0/1663622090 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e080ce320 con 0x7f0e080d52e0 2024-09-12T03:19:29.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:29.324+0000 7f0e0ded7640 1 -- 172.21.15.27:0/1663622090 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e080cdc50 con 0x7f0e080d52e0 2024-09-12T03:19:29.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:29.325+0000 7f0e0ded7640 1 -- 172.21.15.27:0/1663622090 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e0815fde0 con 0x7f0e080d52e0 2024-09-12T03:19:29.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:29.326+0000 7f0deffff640 1 -- 172.21.15.27:0/1663622090 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 9) ==== 90651+0+0 (secure 0 0 0) 0x7f0e0000ac30 con 0x7f0e080d52e0 2024-09-12T03:19:29.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:29.326+0000 7f0deffff640 1 --2- 172.21.15.27:0/1663622090 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f0df406bfe0 0x7f0df406e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:29.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:29.326+0000 7f0deffff640 1 -- 172.21.15.27:0/1663622090 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5709+0+0 (secure 0 0 0) 0x7f0e0008af80 con 0x7f0e080d52e0 2024-09-12T03:19:29.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:29.326+0000 7f0e077fe640 1 --2- 172.21.15.27:0/1663622090 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f0df406bfe0 0x7f0df406e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:29.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:29.327+0000 7f0e077fe640 1 --2- 172.21.15.27:0/1663622090 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f0df406bfe0 0x7f0df406e4a0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f0df8034000 tx=0x7f0df802e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:29.331 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:29.330+0000 7f0deffff640 1 -- 172.21.15.27:0/1663622090 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f0e00059060 con 0x7f0e080d52e0 2024-09-12T03:19:29.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:29.501+0000 7f0e0ded7640 1 -- 172.21.15.27:0/1663622090 --> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f0e0814cff0 con 0x7f0df406bfe0 2024-09-12T03:19:29.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:29.508+0000 7f0deffff640 1 -- 172.21.15.27:0/1663622090 <== mgr.4102 v2:172.21.15.45:6832/3923821219 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+246638 (secure 0 0 0) 0x7f0e0814cff0 con 0x7f0df406bfe0 2024-09-12T03:19:29.511 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:19:29.512 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2024-09-12T03:19:29.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:29.511+0000 7f0e0ded7640 1 -- 172.21.15.27:0/1663622090 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f0df406bfe0 msgr2=0x7f0df406e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:29.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:29.511+0000 7f0e0ded7640 1 --2- 172.21.15.27:0/1663622090 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f0df406bfe0 0x7f0df406e4a0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f0df8034000 tx=0x7f0df802e000 comp rx=0 tx=0).stop 2024-09-12T03:19:29.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:29.511+0000 7f0e0ded7640 1 -- 172.21.15.27:0/1663622090 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e080d52e0 msgr2=0x7f0e0814cab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:29.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:29.511+0000 7f0e0ded7640 1 --2- 172.21.15.27:0/1663622090 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e080d52e0 0x7f0e0814cab0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f0e0000c7a0 tx=0x7f0e0000cc60 comp rx=0 tx=0).stop 2024-09-12T03:19:29.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:29.511+0000 7f0e0ded7640 1 -- 172.21.15.27:0/1663622090 shutdown_connections 2024-09-12T03:19:29.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:29.511+0000 7f0e0ded7640 1 --2- 172.21.15.27:0/1663622090 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f0df406bfe0 0x7f0df406e4a0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:29.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:29.511+0000 7f0e0ded7640 1 --2- 172.21.15.27:0/1663622090 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e080d52e0 0x7f0e0814cab0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:29.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:29.511+0000 7f0e0ded7640 1 --2- 172.21.15.27:0/1663622090 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0e080d3120 0x7f0e080cb880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:29.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:29.511+0000 7f0e0ded7640 1 --2- 172.21.15.27:0/1663622090 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f0e080d1050 0x7f0e080cafe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:29.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:29.511+0000 7f0e0ded7640 1 -- 172.21.15.27:0/1663622090 >> 172.21.15.27:0/1663622090 conn(0x7f0e0806ddf0 msgr2=0x7f0e080b4880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:29.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:29.511+0000 7f0e0ded7640 1 -- 172.21.15.27:0/1663622090 shutdown_connections 2024-09-12T03:19:29.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:29.511+0000 7f0e0ded7640 1 -- 172.21.15.27:0/1663622090 wait complete. 2024-09-12T03:19:29.523 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":143,"stamp":"2024-09-12T03:19:28.095434+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":29,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":247332,"kb_used_data":6100,"kb_used_omap":12,"kb_used_meta":241011,"kb_avail":754727388,"statfs":{"total":773094113280,"available":772840845312,"internally_reserved":0,"allocated":6246400,"data_stored":3240131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":246795824},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":20,"apply_latency_ms":20,"commit_latency_ns":20000000,"apply_latency_ns":20000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.256497"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.851741+0000","last_change":"2024-09-12T03:19:24.851741+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.851741+0000","last_undegraded":"2024-09-12T03:19:24.851741+0000","last_fullsized":"2024-09-12T03:19:24.851741+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.853689+0000","last_change":"2024-09-12T03:19:24.853689+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.853689+0000","last_undegraded":"2024-09-12T03:19:24.853689+0000","last_fullsized":"2024-09-12T03:19:24.853689+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.853107+0000","last_change":"2024-09-12T03:19:24.853107+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.853107+0000","last_undegraded":"2024-09-12T03:19:24.853107+0000","last_fullsized":"2024-09-12T03:19:24.853107+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.849111+0000","last_change":"2024-09-12T03:19:24.849111+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.849111+0000","last_undegraded":"2024-09-12T03:19:24.849111+0000","last_fullsized":"2024-09-12T03:19:24.849111+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.852648+0000","last_change":"2024-09-12T03:19:24.852648+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.852648+0000","last_undegraded":"2024-09-12T03:19:24.852648+0000","last_fullsized":"2024-09-12T03:19:24.852648+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.854574+0000","last_change":"2024-09-12T03:19:24.854574+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.854574+0000","last_undegraded":"2024-09-12T03:19:24.854574+0000","last_fullsized":"2024-09-12T03:19:24.854574+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.846611+0000","last_change":"2024-09-12T03:19:24.846611+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.846611+0000","last_undegraded":"2024-09-12T03:19:24.846611+0000","last_fullsized":"2024-09-12T03:19:24.846611+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.855228+0000","last_change":"2024-09-12T03:19:24.855228+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.855228+0000","last_undegraded":"2024-09-12T03:19:24.855228+0000","last_fullsized":"2024-09-12T03:19:24.855228+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.845883+0000","last_change":"2024-09-12T03:19:24.845883+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.845883+0000","last_undegraded":"2024-09-12T03:19:24.845883+0000","last_fullsized":"2024-09-12T03:19:24.845883+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.848151+0000","last_change":"2024-09-12T03:19:24.848151+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.848151+0000","last_undegraded":"2024-09-12T03:19:24.848151+0000","last_fullsized":"2024-09-12T03:19:24.848151+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.848151+0000","last_change":"2024-09-12T03:19:24.848151+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.848151+0000","last_undegraded":"2024-09-12T03:19:24.848151+0000","last_fullsized":"2024-09-12T03:19:24.848151+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.848357+0000","last_change":"2024-09-12T03:19:24.848357+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.848357+0000","last_undegraded":"2024-09-12T03:19:24.848357+0000","last_fullsized":"2024-09-12T03:19:24.848357+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.851899+0000","last_change":"2024-09-12T03:19:24.851899+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.851899+0000","last_undegraded":"2024-09-12T03:19:24.851899+0000","last_fullsized":"2024-09-12T03:19:24.851899+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.847361+0000","last_change":"2024-09-12T03:19:24.847361+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.847361+0000","last_undegraded":"2024-09-12T03:19:24.847361+0000","last_fullsized":"2024-09-12T03:19:24.847361+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.845866+0000","last_change":"2024-09-12T03:19:24.845866+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.845866+0000","last_undegraded":"2024-09-12T03:19:24.845866+0000","last_fullsized":"2024-09-12T03:19:24.845866+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.852548+0000","last_change":"2024-09-12T03:19:24.852548+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.852548+0000","last_undegraded":"2024-09-12T03:19:24.852548+0000","last_fullsized":"2024-09-12T03:19:24.852548+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.850960+0000","last_change":"2024-09-12T03:19:24.850960+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.850960+0000","last_undegraded":"2024-09-12T03:19:24.850960+0000","last_fullsized":"2024-09-12T03:19:24.850960+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.852064+0000","last_change":"2024-09-12T03:19:24.852064+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.852064+0000","last_undegraded":"2024-09-12T03:19:24.852064+0000","last_fullsized":"2024-09-12T03:19:24.852064+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.850808+0000","last_change":"2024-09-12T03:19:24.850808+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.850808+0000","last_undegraded":"2024-09-12T03:19:24.850808+0000","last_fullsized":"2024-09-12T03:19:24.850808+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.850087+0000","last_change":"2024-09-12T03:19:24.850087+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.850087+0000","last_undegraded":"2024-09-12T03:19:24.850087+0000","last_fullsized":"2024-09-12T03:19:24.850087+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.851577+0000","last_change":"2024-09-12T03:19:24.851577+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.851577+0000","last_undegraded":"2024-09-12T03:19:24.851577+0000","last_fullsized":"2024-09-12T03:19:24.851577+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.856451+0000","last_change":"2024-09-12T03:19:24.856451+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.856451+0000","last_undegraded":"2024-09-12T03:19:24.856451+0000","last_fullsized":"2024-09-12T03:19:24.856451+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.852682+0000","last_change":"2024-09-12T03:19:24.852682+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.852682+0000","last_undegraded":"2024-09-12T03:19:24.852682+0000","last_fullsized":"2024-09-12T03:19:24.852682+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.852444+0000","last_change":"2024-09-12T03:19:24.852444+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.852444+0000","last_undegraded":"2024-09-12T03:19:24.852444+0000","last_fullsized":"2024-09-12T03:19:24.852444+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.851312+0000","last_change":"2024-09-12T03:19:24.851312+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.851312+0000","last_undegraded":"2024-09-12T03:19:24.851312+0000","last_fullsized":"2024-09-12T03:19:24.851312+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.848902+0000","last_change":"2024-09-12T03:19:24.848902+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.848902+0000","last_undegraded":"2024-09-12T03:19:24.848902+0000","last_fullsized":"2024-09-12T03:19:24.848902+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.855319+0000","last_change":"2024-09-12T03:19:24.855319+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.855319+0000","last_undegraded":"2024-09-12T03:19:24.855319+0000","last_fullsized":"2024-09-12T03:19:24.855319+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.851621+0000","last_change":"2024-09-12T03:19:24.851621+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.851621+0000","last_undegraded":"2024-09-12T03:19:24.851621+0000","last_fullsized":"2024-09-12T03:19:24.851621+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.850354+0000","last_change":"2024-09-12T03:19:24.850354+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.850354+0000","last_undegraded":"2024-09-12T03:19:24.850354+0000","last_fullsized":"2024-09-12T03:19:24.850354+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.850218+0000","last_change":"2024-09-12T03:19:24.850218+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.850218+0000","last_undegraded":"2024-09-12T03:19:24.850218+0000","last_fullsized":"2024-09-12T03:19:24.850218+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.850269+0000","last_change":"2024-09-12T03:19:24.850269+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.850269+0000","last_undegraded":"2024-09-12T03:19:24.850269+0000","last_fullsized":"2024-09-12T03:19:24.850269+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.852116+0000","last_change":"2024-09-12T03:19:24.852116+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.852116+0000","last_undegraded":"2024-09-12T03:19:24.852116+0000","last_fullsized":"2024-09-12T03:19:24.852116+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.856894+0000","last_change":"2024-09-12T03:19:24.856894+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.856894+0000","last_undegraded":"2024-09-12T03:19:24.856894+0000","last_fullsized":"2024-09-12T03:19:24.856894+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T03:19:24.869900+0000","last_change":"2024-09-12T03:19:24.860694+0000","last_active":"2024-09-12T03:19:24.869900+0000","last_peered":"2024-09-12T03:19:24.869900+0000","last_clean":"2024-09-12T03:19:24.869900+0000","last_became_active":"2024-09-12T03:19:24.860380+0000","last_became_peered":"2024-09-12T03:19:24.860380+0000","last_unstale":"2024-09-12T03:19:24.869900+0000","last_undegraded":"2024-09-12T03:19:24.869900+0000","last_fullsized":"2024-09-12T03:19:24.869900+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_clean_scrub_stamp":"2024-09-12T03:19:23.835989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:22:23.745090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.848080+0000","last_change":"2024-09-12T03:19:24.848080+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.848080+0000","last_undegraded":"2024-09-12T03:19:24.848080+0000","last_fullsized":"2024-09-12T03:19:24.848080+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T03:19:24.866702+0000","last_change":"2024-09-12T03:19:24.863015+0000","last_active":"2024-09-12T03:19:24.866702+0000","last_peered":"2024-09-12T03:19:24.866702+0000","last_clean":"2024-09-12T03:19:24.866702+0000","last_became_active":"2024-09-12T03:19:24.862457+0000","last_became_peered":"2024-09-12T03:19:24.862457+0000","last_unstale":"2024-09-12T03:19:24.866702+0000","last_undegraded":"2024-09-12T03:19:24.866702+0000","last_fullsized":"2024-09-12T03:19:24.866702+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_clean_scrub_stamp":"2024-09-12T03:19:23.835989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:25:17.547373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.854180+0000","last_change":"2024-09-12T03:19:24.854180+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.854180+0000","last_undegraded":"2024-09-12T03:19:24.854180+0000","last_fullsized":"2024-09-12T03:19:24.854180+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T03:19:24.869928+0000","last_change":"2024-09-12T03:19:24.862226+0000","last_active":"2024-09-12T03:19:24.869928+0000","last_peered":"2024-09-12T03:19:24.869928+0000","last_clean":"2024-09-12T03:19:24.869928+0000","last_became_active":"2024-09-12T03:19:24.861956+0000","last_became_peered":"2024-09-12T03:19:24.861956+0000","last_unstale":"2024-09-12T03:19:24.869928+0000","last_undegraded":"2024-09-12T03:19:24.869928+0000","last_fullsized":"2024-09-12T03:19:24.869928+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_clean_scrub_stamp":"2024-09-12T03:19:23.835989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:56:28.082698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.851109+0000","last_change":"2024-09-12T03:19:24.851109+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.851109+0000","last_undegraded":"2024-09-12T03:19:24.851109+0000","last_fullsized":"2024-09-12T03:19:24.851109+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T03:19:24.861878+0000","last_change":"2024-09-12T03:19:24.859901+0000","last_active":"2024-09-12T03:19:24.861878+0000","last_peered":"2024-09-12T03:19:24.861878+0000","last_clean":"2024-09-12T03:19:24.861878+0000","last_became_active":"2024-09-12T03:19:24.859637+0000","last_became_peered":"2024-09-12T03:19:24.859637+0000","last_unstale":"2024-09-12T03:19:24.861878+0000","last_undegraded":"2024-09-12T03:19:24.861878+0000","last_fullsized":"2024-09-12T03:19:24.861878+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_clean_scrub_stamp":"2024-09-12T03:19:23.835989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:48:14.537533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.853953+0000","last_change":"2024-09-12T03:19:24.853953+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.853953+0000","last_undegraded":"2024-09-12T03:19:24.853953+0000","last_fullsized":"2024-09-12T03:19:24.853953+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T03:19:24.858328+0000","last_change":"2024-09-12T03:19:24.856981+0000","last_active":"2024-09-12T03:19:24.858328+0000","last_peered":"2024-09-12T03:19:24.858328+0000","last_clean":"2024-09-12T03:19:24.858328+0000","last_became_active":"2024-09-12T03:19:24.856607+0000","last_became_peered":"2024-09-12T03:19:24.856607+0000","last_unstale":"2024-09-12T03:19:24.858328+0000","last_undegraded":"2024-09-12T03:19:24.858328+0000","last_fullsized":"2024-09-12T03:19:24.858328+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_clean_scrub_stamp":"2024-09-12T03:19:23.835989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:56:55.814953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.851504+0000","last_change":"2024-09-12T03:19:24.851504+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.851504+0000","last_undegraded":"2024-09-12T03:19:24.851504+0000","last_fullsized":"2024-09-12T03:19:24.851504+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T03:19:24.869885+0000","last_change":"2024-09-12T03:19:24.861305+0000","last_active":"2024-09-12T03:19:24.869885+0000","last_peered":"2024-09-12T03:19:24.869885+0000","last_clean":"2024-09-12T03:19:24.869885+0000","last_became_active":"2024-09-12T03:19:24.861021+0000","last_became_peered":"2024-09-12T03:19:24.861021+0000","last_unstale":"2024-09-12T03:19:24.869885+0000","last_undegraded":"2024-09-12T03:19:24.869885+0000","last_fullsized":"2024-09-12T03:19:24.869885+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_clean_scrub_stamp":"2024-09-12T03:19:23.835989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:52:21.815077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.851467+0000","last_change":"2024-09-12T03:19:24.851467+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.851467+0000","last_undegraded":"2024-09-12T03:19:24.851467+0000","last_fullsized":"2024-09-12T03:19:24.851467+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T03:19:24.854193+0000","last_change":"2024-09-12T03:19:24.850360+0000","last_active":"2024-09-12T03:19:24.854193+0000","last_peered":"2024-09-12T03:19:24.854193+0000","last_clean":"2024-09-12T03:19:24.854193+0000","last_became_active":"2024-09-12T03:19:24.850027+0000","last_became_peered":"2024-09-12T03:19:24.850027+0000","last_unstale":"2024-09-12T03:19:24.854193+0000","last_undegraded":"2024-09-12T03:19:24.854193+0000","last_fullsized":"2024-09-12T03:19:24.854193+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_clean_scrub_stamp":"2024-09-12T03:19:23.835989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:35:50.136024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.853049+0000","last_change":"2024-09-12T03:19:24.853049+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.853049+0000","last_undegraded":"2024-09-12T03:19:24.853049+0000","last_fullsized":"2024-09-12T03:19:24.853049+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T03:19:24.864569+0000","last_change":"2024-09-12T03:19:24.863770+0000","last_active":"2024-09-12T03:19:24.864569+0000","last_peered":"2024-09-12T03:19:24.864569+0000","last_clean":"2024-09-12T03:19:24.864569+0000","last_became_active":"2024-09-12T03:19:24.863471+0000","last_became_peered":"2024-09-12T03:19:24.863471+0000","last_unstale":"2024-09-12T03:19:24.864569+0000","last_undegraded":"2024-09-12T03:19:24.864569+0000","last_fullsized":"2024-09-12T03:19:24.864569+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_clean_scrub_stamp":"2024-09-12T03:19:23.835989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:00:17.289844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.846718+0000","last_change":"2024-09-12T03:19:24.846718+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.846718+0000","last_undegraded":"2024-09-12T03:19:24.846718+0000","last_fullsized":"2024-09-12T03:19:24.846718+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T03:19:24.870959+0000","last_change":"2024-09-12T03:19:24.870526+0000","last_active":"2024-09-12T03:19:24.870959+0000","last_peered":"2024-09-12T03:19:24.870959+0000","last_clean":"2024-09-12T03:19:24.870959+0000","last_became_active":"2024-09-12T03:19:24.870226+0000","last_became_peered":"2024-09-12T03:19:24.870226+0000","last_unstale":"2024-09-12T03:19:24.870959+0000","last_undegraded":"2024-09-12T03:19:24.870959+0000","last_fullsized":"2024-09-12T03:19:24.870959+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_clean_scrub_stamp":"2024-09-12T03:19:23.835989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:43:45.344248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.849991+0000","last_change":"2024-09-12T03:19:24.849991+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.849991+0000","last_undegraded":"2024-09-12T03:19:24.849991+0000","last_fullsized":"2024-09-12T03:19:24.849991+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T03:19:24.866683+0000","last_change":"2024-09-12T03:19:24.862733+0000","last_active":"2024-09-12T03:19:24.866683+0000","last_peered":"2024-09-12T03:19:24.866683+0000","last_clean":"2024-09-12T03:19:24.866683+0000","last_became_active":"2024-09-12T03:19:24.862164+0000","last_became_peered":"2024-09-12T03:19:24.862164+0000","last_unstale":"2024-09-12T03:19:24.866683+0000","last_undegraded":"2024-09-12T03:19:24.866683+0000","last_fullsized":"2024-09-12T03:19:24.866683+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_clean_scrub_stamp":"2024-09-12T03:19:23.835989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:50:19.592853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.848059+0000","last_change":"2024-09-12T03:19:24.848059+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.848059+0000","last_undegraded":"2024-09-12T03:19:24.848059+0000","last_fullsized":"2024-09-12T03:19:24.848059+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T03:19:24.860582+0000","last_change":"2024-09-12T03:19:24.859732+0000","last_active":"2024-09-12T03:19:24.860582+0000","last_peered":"2024-09-12T03:19:24.860582+0000","last_clean":"2024-09-12T03:19:24.860582+0000","last_became_active":"2024-09-12T03:19:24.859452+0000","last_became_peered":"2024-09-12T03:19:24.859452+0000","last_unstale":"2024-09-12T03:19:24.860582+0000","last_undegraded":"2024-09-12T03:19:24.860582+0000","last_fullsized":"2024-09-12T03:19:24.860582+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_clean_scrub_stamp":"2024-09-12T03:19:23.835989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:16:48.958106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.852512+0000","last_change":"2024-09-12T03:19:24.852512+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.852512+0000","last_undegraded":"2024-09-12T03:19:24.852512+0000","last_fullsized":"2024-09-12T03:19:24.852512+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T03:19:24.862745+0000","last_change":"2024-09-12T03:19:24.861967+0000","last_active":"2024-09-12T03:19:24.862745+0000","last_peered":"2024-09-12T03:19:24.862745+0000","last_clean":"2024-09-12T03:19:24.862745+0000","last_became_active":"2024-09-12T03:19:24.861645+0000","last_became_peered":"2024-09-12T03:19:24.861645+0000","last_unstale":"2024-09-12T03:19:24.862745+0000","last_undegraded":"2024-09-12T03:19:24.862745+0000","last_fullsized":"2024-09-12T03:19:24.862745+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_clean_scrub_stamp":"2024-09-12T03:19:23.835989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:11:47.170009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.847408+0000","last_change":"2024-09-12T03:19:24.847408+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.847408+0000","last_undegraded":"2024-09-12T03:19:24.847408+0000","last_fullsized":"2024-09-12T03:19:24.847408+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T03:19:24.866725+0000","last_change":"2024-09-12T03:19:24.863067+0000","last_active":"2024-09-12T03:19:24.866725+0000","last_peered":"2024-09-12T03:19:24.866725+0000","last_clean":"2024-09-12T03:19:24.866725+0000","last_became_active":"2024-09-12T03:19:24.862471+0000","last_became_peered":"2024-09-12T03:19:24.862471+0000","last_unstale":"2024-09-12T03:19:24.866725+0000","last_undegraded":"2024-09-12T03:19:24.866725+0000","last_fullsized":"2024-09-12T03:19:24.866725+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_clean_scrub_stamp":"2024-09-12T03:19:23.835989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:04:37.233100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.851140+0000","last_change":"2024-09-12T03:19:24.851140+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.851140+0000","last_undegraded":"2024-09-12T03:19:24.851140+0000","last_fullsized":"2024-09-12T03:19:24.851140+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T03:19:24.862050+0000","last_change":"2024-09-12T03:19:24.859981+0000","last_active":"2024-09-12T03:19:24.862050+0000","last_peered":"2024-09-12T03:19:24.862050+0000","last_clean":"2024-09-12T03:19:24.862050+0000","last_became_active":"2024-09-12T03:19:24.859679+0000","last_became_peered":"2024-09-12T03:19:24.859679+0000","last_unstale":"2024-09-12T03:19:24.862050+0000","last_undegraded":"2024-09-12T03:19:24.862050+0000","last_fullsized":"2024-09-12T03:19:24.862050+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_clean_scrub_stamp":"2024-09-12T03:19:23.835989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:51:59.350797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.846337+0000","last_change":"2024-09-12T03:19:24.846337+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.846337+0000","last_undegraded":"2024-09-12T03:19:24.846337+0000","last_fullsized":"2024-09-12T03:19:24.846337+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T03:19:24.857304+0000","last_change":"2024-09-12T03:19:24.855795+0000","last_active":"2024-09-12T03:19:24.857304+0000","last_peered":"2024-09-12T03:19:24.857304+0000","last_clean":"2024-09-12T03:19:24.857304+0000","last_became_active":"2024-09-12T03:19:24.855377+0000","last_became_peered":"2024-09-12T03:19:24.855377+0000","last_unstale":"2024-09-12T03:19:24.857304+0000","last_undegraded":"2024-09-12T03:19:24.857304+0000","last_fullsized":"2024-09-12T03:19:24.857304+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_clean_scrub_stamp":"2024-09-12T03:19:23.835989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:50:07.794274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.852242+0000","last_change":"2024-09-12T03:19:24.852242+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.852242+0000","last_undegraded":"2024-09-12T03:19:24.852242+0000","last_fullsized":"2024-09-12T03:19:24.852242+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T03:19:24.860954+0000","last_change":"2024-09-12T03:19:24.859883+0000","last_active":"2024-09-12T03:19:24.860954+0000","last_peered":"2024-09-12T03:19:24.860954+0000","last_clean":"2024-09-12T03:19:24.860954+0000","last_became_active":"2024-09-12T03:19:24.859566+0000","last_became_peered":"2024-09-12T03:19:24.859566+0000","last_unstale":"2024-09-12T03:19:24.860954+0000","last_undegraded":"2024-09-12T03:19:24.860954+0000","last_fullsized":"2024-09-12T03:19:24.860954+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_clean_scrub_stamp":"2024-09-12T03:19:23.835989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:44:44.153823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.851328+0000","last_change":"2024-09-12T03:19:24.851328+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.851328+0000","last_undegraded":"2024-09-12T03:19:24.851328+0000","last_fullsized":"2024-09-12T03:19:24.851328+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T03:19:24.866633+0000","last_change":"2024-09-12T03:19:24.861891+0000","last_active":"2024-09-12T03:19:24.866633+0000","last_peered":"2024-09-12T03:19:24.866633+0000","last_clean":"2024-09-12T03:19:24.866633+0000","last_became_active":"2024-09-12T03:19:24.861524+0000","last_became_peered":"2024-09-12T03:19:24.861524+0000","last_unstale":"2024-09-12T03:19:24.866633+0000","last_undegraded":"2024-09-12T03:19:24.866633+0000","last_fullsized":"2024-09-12T03:19:24.866633+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_clean_scrub_stamp":"2024-09-12T03:19:23.835989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:16:02.571700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.848416+0000","last_change":"2024-09-12T03:19:24.848416+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.848416+0000","last_undegraded":"2024-09-12T03:19:24.848416+0000","last_fullsized":"2024-09-12T03:19:24.848416+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T03:19:24.872537+0000","last_change":"2024-09-12T03:19:24.865925+0000","last_active":"2024-09-12T03:19:24.872537+0000","last_peered":"2024-09-12T03:19:24.872537+0000","last_clean":"2024-09-12T03:19:24.872537+0000","last_became_active":"2024-09-12T03:19:24.865689+0000","last_became_peered":"2024-09-12T03:19:24.865689+0000","last_unstale":"2024-09-12T03:19:24.872537+0000","last_undegraded":"2024-09-12T03:19:24.872537+0000","last_fullsized":"2024-09-12T03:19:24.872537+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_clean_scrub_stamp":"2024-09-12T03:19:23.835989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:17:44.078064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.849465+0000","last_change":"2024-09-12T03:19:24.849465+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.849465+0000","last_undegraded":"2024-09-12T03:19:24.849465+0000","last_fullsized":"2024-09-12T03:19:24.849465+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T03:19:24.857771+0000","last_change":"2024-09-12T03:19:24.855690+0000","last_active":"2024-09-12T03:19:24.857771+0000","last_peered":"2024-09-12T03:19:24.857771+0000","last_clean":"2024-09-12T03:19:24.857771+0000","last_became_active":"2024-09-12T03:19:24.855284+0000","last_became_peered":"2024-09-12T03:19:24.855284+0000","last_unstale":"2024-09-12T03:19:24.857771+0000","last_undegraded":"2024-09-12T03:19:24.857771+0000","last_fullsized":"2024-09-12T03:19:24.857771+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_clean_scrub_stamp":"2024-09-12T03:19:23.835989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:38:35.821379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.851636+0000","last_change":"2024-09-12T03:19:24.851636+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.851636+0000","last_undegraded":"2024-09-12T03:19:24.851636+0000","last_fullsized":"2024-09-12T03:19:24.851636+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T03:19:24.872475+0000","last_change":"2024-09-12T03:19:24.864872+0000","last_active":"2024-09-12T03:19:24.872475+0000","last_peered":"2024-09-12T03:19:24.872475+0000","last_clean":"2024-09-12T03:19:24.872475+0000","last_became_active":"2024-09-12T03:19:24.864506+0000","last_became_peered":"2024-09-12T03:19:24.864506+0000","last_unstale":"2024-09-12T03:19:24.872475+0000","last_undegraded":"2024-09-12T03:19:24.872475+0000","last_fullsized":"2024-09-12T03:19:24.872475+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_clean_scrub_stamp":"2024-09-12T03:19:23.835989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:16:51.163113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.846915+0000","last_change":"2024-09-12T03:19:24.846915+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.846915+0000","last_undegraded":"2024-09-12T03:19:24.846915+0000","last_fullsized":"2024-09-12T03:19:24.846915+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T03:19:24.866700+0000","last_change":"2024-09-12T03:19:24.862743+0000","last_active":"2024-09-12T03:19:24.866700+0000","last_peered":"2024-09-12T03:19:24.866700+0000","last_clean":"2024-09-12T03:19:24.866700+0000","last_became_active":"2024-09-12T03:19:24.862186+0000","last_became_peered":"2024-09-12T03:19:24.862186+0000","last_unstale":"2024-09-12T03:19:24.866700+0000","last_undegraded":"2024-09-12T03:19:24.866700+0000","last_fullsized":"2024-09-12T03:19:24.866700+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_clean_scrub_stamp":"2024-09-12T03:19:23.835989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:04:27.775435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.855497+0000","last_change":"2024-09-12T03:19:24.855497+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.855497+0000","last_undegraded":"2024-09-12T03:19:24.855497+0000","last_fullsized":"2024-09-12T03:19:24.855497+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T03:19:24.860844+0000","last_change":"2024-09-12T03:19:24.857052+0000","last_active":"2024-09-12T03:19:24.860844+0000","last_peered":"2024-09-12T03:19:24.860844+0000","last_clean":"2024-09-12T03:19:24.860844+0000","last_became_active":"2024-09-12T03:19:24.856771+0000","last_became_peered":"2024-09-12T03:19:24.856771+0000","last_unstale":"2024-09-12T03:19:24.860844+0000","last_undegraded":"2024-09-12T03:19:24.860844+0000","last_fullsized":"2024-09-12T03:19:24.860844+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_clean_scrub_stamp":"2024-09-12T03:19:23.835989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:35:47.557902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.852418+0000","last_change":"2024-09-12T03:19:24.852418+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.852418+0000","last_undegraded":"2024-09-12T03:19:24.852418+0000","last_fullsized":"2024-09-12T03:19:24.852418+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T03:19:24.864938+0000","last_change":"2024-09-12T03:19:24.864514+0000","last_active":"2024-09-12T03:19:24.864938+0000","last_peered":"2024-09-12T03:19:24.864938+0000","last_clean":"2024-09-12T03:19:24.864938+0000","last_became_active":"2024-09-12T03:19:24.864206+0000","last_became_peered":"2024-09-12T03:19:24.864206+0000","last_unstale":"2024-09-12T03:19:24.864938+0000","last_undegraded":"2024-09-12T03:19:24.864938+0000","last_fullsized":"2024-09-12T03:19:24.864938+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_clean_scrub_stamp":"2024-09-12T03:19:23.835989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:24:19.872805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.847841+0000","last_change":"2024-09-12T03:19:24.847841+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.847841+0000","last_undegraded":"2024-09-12T03:19:24.847841+0000","last_fullsized":"2024-09-12T03:19:24.847841+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T03:19:24.866519+0000","last_change":"2024-09-12T03:19:24.863777+0000","last_active":"2024-09-12T03:19:24.866519+0000","last_peered":"2024-09-12T03:19:24.866519+0000","last_clean":"2024-09-12T03:19:24.866519+0000","last_became_active":"2024-09-12T03:19:24.863506+0000","last_became_peered":"2024-09-12T03:19:24.863506+0000","last_unstale":"2024-09-12T03:19:24.866519+0000","last_undegraded":"2024-09-12T03:19:24.866519+0000","last_fullsized":"2024-09-12T03:19:24.866519+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_clean_scrub_stamp":"2024-09-12T03:19:23.835989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:53:10.464555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.847949+0000","last_change":"2024-09-12T03:19:24.847949+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.847949+0000","last_undegraded":"2024-09-12T03:19:24.847949+0000","last_fullsized":"2024-09-12T03:19:24.847949+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T03:19:24.872512+0000","last_change":"2024-09-12T03:19:24.865383+0000","last_active":"2024-09-12T03:19:24.872512+0000","last_peered":"2024-09-12T03:19:24.872512+0000","last_clean":"2024-09-12T03:19:24.872512+0000","last_became_active":"2024-09-12T03:19:24.865031+0000","last_became_peered":"2024-09-12T03:19:24.865031+0000","last_unstale":"2024-09-12T03:19:24.872512+0000","last_undegraded":"2024-09-12T03:19:24.872512+0000","last_fullsized":"2024-09-12T03:19:24.872512+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_clean_scrub_stamp":"2024-09-12T03:19:23.835989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:36:23.795897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.847544+0000","last_change":"2024-09-12T03:19:24.847544+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.847544+0000","last_undegraded":"2024-09-12T03:19:24.847544+0000","last_fullsized":"2024-09-12T03:19:24.847544+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T03:19:24.861103+0000","last_change":"2024-09-12T03:19:24.857501+0000","last_active":"2024-09-12T03:19:24.861103+0000","last_peered":"2024-09-12T03:19:24.861103+0000","last_clean":"2024-09-12T03:19:24.861103+0000","last_became_active":"2024-09-12T03:19:24.857129+0000","last_became_peered":"2024-09-12T03:19:24.857129+0000","last_unstale":"2024-09-12T03:19:24.861103+0000","last_undegraded":"2024-09-12T03:19:24.861103+0000","last_fullsized":"2024-09-12T03:19:24.861103+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_clean_scrub_stamp":"2024-09-12T03:19:23.835989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:54:09.253982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.850621+0000","last_change":"2024-09-12T03:19:24.850621+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.850621+0000","last_undegraded":"2024-09-12T03:19:24.850621+0000","last_fullsized":"2024-09-12T03:19:24.850621+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T03:19:24.856019+0000","last_change":"2024-09-12T03:19:24.852831+0000","last_active":"2024-09-12T03:19:24.856019+0000","last_peered":"2024-09-12T03:19:24.856019+0000","last_clean":"2024-09-12T03:19:24.856019+0000","last_became_active":"2024-09-12T03:19:24.851427+0000","last_became_peered":"2024-09-12T03:19:24.851427+0000","last_unstale":"2024-09-12T03:19:24.856019+0000","last_undegraded":"2024-09-12T03:19:24.856019+0000","last_fullsized":"2024-09-12T03:19:24.856019+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_clean_scrub_stamp":"2024-09-12T03:19:23.835989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:39:10.957546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.853481+0000","last_change":"2024-09-12T03:19:24.853481+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.853481+0000","last_undegraded":"2024-09-12T03:19:24.853481+0000","last_fullsized":"2024-09-12T03:19:24.853481+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T03:19:24.855106+0000","last_change":"2024-09-12T03:19:24.849318+0000","last_active":"2024-09-12T03:19:24.855106+0000","last_peered":"2024-09-12T03:19:24.855106+0000","last_clean":"2024-09-12T03:19:24.855106+0000","last_became_active":"2024-09-12T03:19:24.848981+0000","last_became_peered":"2024-09-12T03:19:24.848981+0000","last_unstale":"2024-09-12T03:19:24.855106+0000","last_undegraded":"2024-09-12T03:19:24.855106+0000","last_fullsized":"2024-09-12T03:19:24.855106+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_clean_scrub_stamp":"2024-09-12T03:19:23.835989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:07:42.042664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.850646+0000","last_change":"2024-09-12T03:19:24.850646+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.850646+0000","last_undegraded":"2024-09-12T03:19:24.850646+0000","last_fullsized":"2024-09-12T03:19:24.850646+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T03:19:24.862763+0000","last_change":"2024-09-12T03:19:24.861998+0000","last_active":"2024-09-12T03:19:24.862763+0000","last_peered":"2024-09-12T03:19:24.862763+0000","last_clean":"2024-09-12T03:19:24.862763+0000","last_became_active":"2024-09-12T03:19:24.861694+0000","last_became_peered":"2024-09-12T03:19:24.861694+0000","last_unstale":"2024-09-12T03:19:24.862763+0000","last_undegraded":"2024-09-12T03:19:24.862763+0000","last_fullsized":"2024-09-12T03:19:24.862763+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_clean_scrub_stamp":"2024-09-12T03:19:23.835989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:09:05.230238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.846974+0000","last_change":"2024-09-12T03:19:24.846974+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.846974+0000","last_undegraded":"2024-09-12T03:19:24.846974+0000","last_fullsized":"2024-09-12T03:19:24.846974+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T03:19:24.863848+0000","last_change":"2024-09-12T03:19:24.860398+0000","last_active":"2024-09-12T03:19:24.863848+0000","last_peered":"2024-09-12T03:19:24.863848+0000","last_clean":"2024-09-12T03:19:24.863848+0000","last_became_active":"2024-09-12T03:19:24.860093+0000","last_became_peered":"2024-09-12T03:19:24.860093+0000","last_unstale":"2024-09-12T03:19:24.863848+0000","last_undegraded":"2024-09-12T03:19:24.863848+0000","last_fullsized":"2024-09-12T03:19:24.863848+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_clean_scrub_stamp":"2024-09-12T03:19:23.835989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:40:45.875415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.852543+0000","last_change":"2024-09-12T03:19:24.852543+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.852543+0000","last_undegraded":"2024-09-12T03:19:24.852543+0000","last_fullsized":"2024-09-12T03:19:24.852543+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T03:19:24.864154+0000","last_change":"2024-09-12T03:19:24.858756+0000","last_active":"2024-09-12T03:19:24.864154+0000","last_peered":"2024-09-12T03:19:24.864154+0000","last_clean":"2024-09-12T03:19:24.864154+0000","last_became_active":"2024-09-12T03:19:24.858474+0000","last_became_peered":"2024-09-12T03:19:24.858474+0000","last_unstale":"2024-09-12T03:19:24.864154+0000","last_undegraded":"2024-09-12T03:19:24.864154+0000","last_fullsized":"2024-09-12T03:19:24.864154+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_clean_scrub_stamp":"2024-09-12T03:19:23.835989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:51:36.581086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.851171+0000","last_change":"2024-09-12T03:19:24.851171+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.851171+0000","last_undegraded":"2024-09-12T03:19:24.851171+0000","last_fullsized":"2024-09-12T03:19:24.851171+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T03:19:24.861322+0000","last_change":"2024-09-12T03:19:24.858024+0000","last_active":"2024-09-12T03:19:24.861322+0000","last_peered":"2024-09-12T03:19:24.861322+0000","last_clean":"2024-09-12T03:19:24.861322+0000","last_became_active":"2024-09-12T03:19:24.857142+0000","last_became_peered":"2024-09-12T03:19:24.857142+0000","last_unstale":"2024-09-12T03:19:24.861322+0000","last_undegraded":"2024-09-12T03:19:24.861322+0000","last_fullsized":"2024-09-12T03:19:24.861322+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_clean_scrub_stamp":"2024-09-12T03:19:23.835989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:59:02.611792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":228,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T03:19:24.853689+0000","last_change":"2024-09-12T03:16:21.883446+0000","last_active":"2024-09-12T03:19:24.853689+0000","last_peered":"2024-09-12T03:19:24.853689+0000","last_clean":"2024-09-12T03:19:24.853689+0000","last_became_active":"2024-09-12T03:16:21.882207+0000","last_became_peered":"2024-09-12T03:16:21.882207+0000","last_unstale":"2024-09-12T03:19:24.853689+0000","last_undegraded":"2024-09-12T03:19:24.853689+0000","last_fullsized":"2024-09-12T03:19:24.853689+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:20.316426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:20.316426+0000","last_clean_scrub_stamp":"2024-09-12T03:16:20.316426+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:12:15.424320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542183,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32396,"kb_used_data":1208,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339444,"statfs":{"total":96636764160,"available":96603590656,"internally_reserved":0,"allocated":1236992,"data_stored":853276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542185,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31816,"kb_used_data":616,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340024,"statfs":{"total":96636764160,"available":96604184576,"internally_reserved":0,"allocated":630784,"data_stored":256641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542184,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35964,"kb_used_data":616,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335876,"statfs":{"total":96636764160,"available":96599937024,"internally_reserved":0,"allocated":630784,"data_stored":256641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542183,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31820,"kb_used_data":616,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340020,"statfs":{"total":96636764160,"available":96604180480,"internally_reserved":0,"allocated":630784,"data_stored":256641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542184,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27656,"kb_used_data":616,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344184,"statfs":{"total":96636764160,"available":96608444416,"internally_reserved":0,"allocated":630784,"data_stored":256641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542184,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27648,"kb_used_data":616,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344192,"statfs":{"total":96636764160,"available":96608452608,"internally_reserved":0,"allocated":630784,"data_stored":256641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542185,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31808,"kb_used_data":616,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340032,"statfs":{"total":96636764160,"available":96604192768,"internally_reserved":0,"allocated":630784,"data_stored":256641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542185,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28224,"kb_used_data":1196,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343616,"statfs":{"total":96636764160,"available":96607862784,"internally_reserved":0,"allocated":1224704,"data_stored":847009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T03:19:29.526 INFO:tasks.ceph.ceph_manager.ceph:PG 9.3f is not active+clean 2024-09-12T03:19:29.526 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 54, 'state': 'creating+peering', 'last_fresh': '2024-09-12T03:19:24.851741+0000', 'last_change': '2024-09-12T03:19:24.851741+0000', 'last_active': '2024-09-12T03:19:24.838022+0000', 'last_peered': '2024-09-12T03:19:24.838022+0000', 'last_clean': '2024-09-12T03:19:24.838022+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:19:24.851741+0000', 'last_undegraded': '2024-09-12T03:19:24.851741+0000', 'last_fullsized': '2024-09-12T03:19:24.851741+0000', 'mapping_epoch': 54, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 54, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:19:24.838022+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:19:24.838022+0000', 'last_clean_scrub_stamp': '2024-09-12T03:19:24.838022+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-12T03:19:29.527 INFO:tasks.ceph.ceph_manager.ceph:PG 9.3e is not active+clean 2024-09-12T03:19:29.527 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 54, 'state': 'creating+peering', 'last_fresh': '2024-09-12T03:19:24.853689+0000', 'last_change': '2024-09-12T03:19:24.853689+0000', 'last_active': '2024-09-12T03:19:24.838022+0000', 'last_peered': '2024-09-12T03:19:24.838022+0000', 'last_clean': '2024-09-12T03:19:24.838022+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:19:24.853689+0000', 'last_undegraded': '2024-09-12T03:19:24.853689+0000', 'last_fullsized': '2024-09-12T03:19:24.853689+0000', 'mapping_epoch': 54, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 54, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:19:24.838022+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:19:24.838022+0000', 'last_clean_scrub_stamp': '2024-09-12T03:19:24.838022+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-12T03:19:29.527 INFO:tasks.ceph.ceph_manager.ceph:PG 9.3d is not active+clean 2024-09-12T03:19:29.527 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 54, 'state': 'creating+peering', 'last_fresh': '2024-09-12T03:19:24.853107+0000', 'last_change': '2024-09-12T03:19:24.853107+0000', 'last_active': '2024-09-12T03:19:24.838022+0000', 'last_peered': '2024-09-12T03:19:24.838022+0000', 'last_clean': '2024-09-12T03:19:24.838022+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:19:24.853107+0000', 'last_undegraded': '2024-09-12T03:19:24.853107+0000', 'last_fullsized': '2024-09-12T03:19:24.853107+0000', 'mapping_epoch': 54, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 54, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:19:24.838022+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:19:24.838022+0000', 'last_clean_scrub_stamp': '2024-09-12T03:19:24.838022+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-12T03:19:29.527 INFO:tasks.ceph.ceph_manager.ceph:PG 9.3c is not active+clean 2024-09-12T03:19:29.527 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 54, 'state': 'creating+peering', 'last_fresh': '2024-09-12T03:19:24.849111+0000', 'last_change': '2024-09-12T03:19:24.849111+0000', 'last_active': '2024-09-12T03:19:24.838022+0000', 'last_peered': '2024-09-12T03:19:24.838022+0000', 'last_clean': '2024-09-12T03:19:24.838022+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:19:24.849111+0000', 'last_undegraded': '2024-09-12T03:19:24.849111+0000', 'last_fullsized': '2024-09-12T03:19:24.849111+0000', 'mapping_epoch': 54, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 54, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:19:24.838022+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:19:24.838022+0000', 'last_clean_scrub_stamp': '2024-09-12T03:19:24.838022+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-12T03:19:29.527 INFO:tasks.ceph.ceph_manager.ceph:PG 9.3b is not active+clean 2024-09-12T03:19:29.527 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 54, 'state': 'creating+peering', 'last_fresh': '2024-09-12T03:19:24.852648+0000', 'last_change': '2024-09-12T03:19:24.852648+0000', 'last_active': '2024-09-12T03:19:24.838022+0000', 'last_peered': '2024-09-12T03:19:24.838022+0000', 'last_clean': '2024-09-12T03:19:24.838022+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:19:24.852648+0000', 'last_undegraded': '2024-09-12T03:19:24.852648+0000', 'last_fullsized': '2024-09-12T03:19:24.852648+0000', 'mapping_epoch': 54, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 54, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:19:24.838022+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:19:24.838022+0000', 'last_clean_scrub_stamp': '2024-09-12T03:19:24.838022+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-12T03:19:29.527 INFO:tasks.ceph.ceph_manager.ceph:PG 9.3a is not active+clean 2024-09-12T03:19:29.527 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 54, 'state': 'creating+peering', 'last_fresh': '2024-09-12T03:19:24.854574+0000', 'last_change': '2024-09-12T03:19:24.854574+0000', 'last_active': '2024-09-12T03:19:24.838022+0000', 'last_peered': '2024-09-12T03:19:24.838022+0000', 'last_clean': '2024-09-12T03:19:24.838022+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:19:24.854574+0000', 'last_undegraded': '2024-09-12T03:19:24.854574+0000', 'last_fullsized': '2024-09-12T03:19:24.854574+0000', 'mapping_epoch': 54, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 54, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:19:24.838022+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:19:24.838022+0000', 'last_clean_scrub_stamp': '2024-09-12T03:19:24.838022+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-12T03:19:29.527 INFO:tasks.ceph.ceph_manager.ceph:PG 9.39 is not active+clean 2024-09-12T03:19:29.528 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 54, 'state': 'creating+peering', 'last_fresh': '2024-09-12T03:19:24.846611+0000', 'last_change': '2024-09-12T03:19:24.846611+0000', 'last_active': '2024-09-12T03:19:24.838022+0000', 'last_peered': '2024-09-12T03:19:24.838022+0000', 'last_clean': '2024-09-12T03:19:24.838022+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:19:24.846611+0000', 'last_undegraded': '2024-09-12T03:19:24.846611+0000', 'last_fullsized': '2024-09-12T03:19:24.846611+0000', 'mapping_epoch': 54, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 54, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:19:24.838022+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:19:24.838022+0000', 'last_clean_scrub_stamp': '2024-09-12T03:19:24.838022+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-12T03:19:29.528 INFO:tasks.ceph.ceph_manager.ceph:PG 9.38 is not active+clean 2024-09-12T03:19:29.528 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 54, 'state': 'creating+peering', 'last_fresh': '2024-09-12T03:19:24.855228+0000', 'last_change': '2024-09-12T03:19:24.855228+0000', 'last_active': '2024-09-12T03:19:24.838022+0000', 'last_peered': '2024-09-12T03:19:24.838022+0000', 'last_clean': '2024-09-12T03:19:24.838022+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:19:24.855228+0000', 'last_undegraded': '2024-09-12T03:19:24.855228+0000', 'last_fullsized': '2024-09-12T03:19:24.855228+0000', 'mapping_epoch': 54, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 54, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:19:24.838022+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:19:24.838022+0000', 'last_clean_scrub_stamp': '2024-09-12T03:19:24.838022+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-12T03:19:29.528 INFO:tasks.ceph.ceph_manager.ceph:PG 9.37 is not active+clean 2024-09-12T03:19:29.528 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 54, 'state': 'creating+peering', 'last_fresh': '2024-09-12T03:19:24.845883+0000', 'last_change': '2024-09-12T03:19:24.845883+0000', 'last_active': '2024-09-12T03:19:24.838022+0000', 'last_peered': '2024-09-12T03:19:24.838022+0000', 'last_clean': '2024-09-12T03:19:24.838022+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:19:24.845883+0000', 'last_undegraded': '2024-09-12T03:19:24.845883+0000', 'last_fullsized': '2024-09-12T03:19:24.845883+0000', 'mapping_epoch': 54, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 54, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:19:24.838022+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:19:24.838022+0000', 'last_clean_scrub_stamp': '2024-09-12T03:19:24.838022+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-12T03:19:29.528 INFO:tasks.ceph.ceph_manager.ceph:PG 9.36 is not active+clean 2024-09-12T03:19:29.528 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 54, 'state': 'creating+peering', 'last_fresh': '2024-09-12T03:19:24.848151+0000', 'last_change': '2024-09-12T03:19:24.848151+0000', 'last_active': '2024-09-12T03:19:24.838022+0000', 'last_peered': '2024-09-12T03:19:24.838022+0000', 'last_clean': '2024-09-12T03:19:24.838022+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:19:24.848151+0000', 'last_undegraded': '2024-09-12T03:19:24.848151+0000', 'last_fullsized': '2024-09-12T03:19:24.848151+0000', 'mapping_epoch': 54, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 54, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:19:24.838022+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:19:24.838022+0000', 'last_clean_scrub_stamp': '2024-09-12T03:19:24.838022+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-12T03:19:29.528 INFO:tasks.ceph.ceph_manager.ceph:PG 9.35 is not active+clean 2024-09-12T03:19:29.528 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 54, 'state': 'creating+peering', 'last_fresh': '2024-09-12T03:19:24.848151+0000', 'last_change': '2024-09-12T03:19:24.848151+0000', 'last_active': '2024-09-12T03:19:24.838022+0000', 'last_peered': '2024-09-12T03:19:24.838022+0000', 'last_clean': '2024-09-12T03:19:24.838022+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:19:24.848151+0000', 'last_undegraded': '2024-09-12T03:19:24.848151+0000', 'last_fullsized': '2024-09-12T03:19:24.848151+0000', 'mapping_epoch': 54, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 54, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:19:24.838022+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:19:24.838022+0000', 'last_clean_scrub_stamp': '2024-09-12T03:19:24.838022+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-12T03:19:29.528 INFO:tasks.ceph.ceph_manager.ceph:PG 9.34 is not active+clean 2024-09-12T03:19:29.528 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 54, 'state': 'creating+peering', 'last_fresh': '2024-09-12T03:19:24.848357+0000', 'last_change': '2024-09-12T03:19:24.848357+0000', 'last_active': '2024-09-12T03:19:24.838022+0000', 'last_peered': '2024-09-12T03:19:24.838022+0000', 'last_clean': '2024-09-12T03:19:24.838022+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:19:24.848357+0000', 'last_undegraded': '2024-09-12T03:19:24.848357+0000', 'last_fullsized': '2024-09-12T03:19:24.848357+0000', 'mapping_epoch': 54, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 54, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:19:24.838022+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:19:24.838022+0000', 'last_clean_scrub_stamp': '2024-09-12T03:19:24.838022+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-12T03:19:29.529 INFO:tasks.ceph.ceph_manager.ceph:PG 9.33 is not active+clean 2024-09-12T03:19:29.529 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 54, 'state': 'creating+peering', 'last_fresh': '2024-09-12T03:19:24.851899+0000', 'last_change': '2024-09-12T03:19:24.851899+0000', 'last_active': '2024-09-12T03:19:24.838022+0000', 'last_peered': '2024-09-12T03:19:24.838022+0000', 'last_clean': '2024-09-12T03:19:24.838022+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:19:24.851899+0000', 'last_undegraded': '2024-09-12T03:19:24.851899+0000', 'last_fullsized': '2024-09-12T03:19:24.851899+0000', 'mapping_epoch': 54, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 54, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:19:24.838022+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:19:24.838022+0000', 'last_clean_scrub_stamp': '2024-09-12T03:19:24.838022+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-12T03:19:29.529 INFO:tasks.ceph.ceph_manager.ceph:PG 9.32 is not active+clean 2024-09-12T03:19:29.529 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 54, 'state': 'creating+peering', 'last_fresh': '2024-09-12T03:19:24.847361+0000', 'last_change': '2024-09-12T03:19:24.847361+0000', 'last_active': '2024-09-12T03:19:24.838022+0000', 'last_peered': '2024-09-12T03:19:24.838022+0000', 'last_clean': '2024-09-12T03:19:24.838022+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:19:24.847361+0000', 'last_undegraded': '2024-09-12T03:19:24.847361+0000', 'last_fullsized': '2024-09-12T03:19:24.847361+0000', 'mapping_epoch': 54, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 54, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:19:24.838022+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:19:24.838022+0000', 'last_clean_scrub_stamp': '2024-09-12T03:19:24.838022+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-12T03:19:29.529 INFO:tasks.ceph.ceph_manager.ceph:PG 9.31 is not active+clean 2024-09-12T03:19:29.529 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 54, 'state': 'creating+peering', 'last_fresh': '2024-09-12T03:19:24.845866+0000', 'last_change': '2024-09-12T03:19:24.845866+0000', 'last_active': '2024-09-12T03:19:24.838022+0000', 'last_peered': '2024-09-12T03:19:24.838022+0000', 'last_clean': '2024-09-12T03:19:24.838022+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:19:24.845866+0000', 'last_undegraded': '2024-09-12T03:19:24.845866+0000', 'last_fullsized': '2024-09-12T03:19:24.845866+0000', 'mapping_epoch': 54, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 54, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:19:24.838022+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:19:24.838022+0000', 'last_clean_scrub_stamp': '2024-09-12T03:19:24.838022+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-12T03:19:29.529 INFO:tasks.ceph.ceph_manager.ceph:PG 9.30 is not active+clean 2024-09-12T03:19:29.529 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 54, 'state': 'creating+peering', 'last_fresh': '2024-09-12T03:19:24.852548+0000', 'last_change': '2024-09-12T03:19:24.852548+0000', 'last_active': '2024-09-12T03:19:24.838022+0000', 'last_peered': '2024-09-12T03:19:24.838022+0000', 'last_clean': '2024-09-12T03:19:24.838022+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:19:24.852548+0000', 'last_undegraded': '2024-09-12T03:19:24.852548+0000', 'last_fullsized': '2024-09-12T03:19:24.852548+0000', 'mapping_epoch': 54, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 54, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:19:24.838022+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:19:24.838022+0000', 'last_clean_scrub_stamp': '2024-09-12T03:19:24.838022+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-12T03:19:29.529 INFO:tasks.ceph.ceph_manager.ceph:PG 9.2f is not active+clean 2024-09-12T03:19:29.529 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 54, 'state': 'creating+peering', 'last_fresh': '2024-09-12T03:19:24.850960+0000', 'last_change': '2024-09-12T03:19:24.850960+0000', 'last_active': '2024-09-12T03:19:24.838022+0000', 'last_peered': '2024-09-12T03:19:24.838022+0000', 'last_clean': '2024-09-12T03:19:24.838022+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:19:24.850960+0000', 'last_undegraded': '2024-09-12T03:19:24.850960+0000', 'last_fullsized': '2024-09-12T03:19:24.850960+0000', 'mapping_epoch': 54, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 54, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:19:24.838022+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:19:24.838022+0000', 'last_clean_scrub_stamp': '2024-09-12T03:19:24.838022+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-12T03:19:29.529 INFO:tasks.ceph.ceph_manager.ceph:PG 9.2e is not active+clean 2024-09-12T03:19:29.530 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 54, 'state': 'creating+peering', 'last_fresh': '2024-09-12T03:19:24.852064+0000', 'last_change': '2024-09-12T03:19:24.852064+0000', 'last_active': '2024-09-12T03:19:24.838022+0000', 'last_peered': '2024-09-12T03:19:24.838022+0000', 'last_clean': '2024-09-12T03:19:24.838022+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:19:24.852064+0000', 'last_undegraded': '2024-09-12T03:19:24.852064+0000', 'last_fullsized': '2024-09-12T03:19:24.852064+0000', 'mapping_epoch': 54, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 54, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:19:24.838022+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:19:24.838022+0000', 'last_clean_scrub_stamp': '2024-09-12T03:19:24.838022+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-12T03:19:29.530 INFO:tasks.ceph.ceph_manager.ceph:PG 9.2d is not active+clean 2024-09-12T03:19:29.530 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 54, 'state': 'creating+peering', 'last_fresh': '2024-09-12T03:19:24.850808+0000', 'last_change': '2024-09-12T03:19:24.850808+0000', 'last_active': '2024-09-12T03:19:24.838022+0000', 'last_peered': '2024-09-12T03:19:24.838022+0000', 'last_clean': '2024-09-12T03:19:24.838022+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:19:24.850808+0000', 'last_undegraded': '2024-09-12T03:19:24.850808+0000', 'last_fullsized': '2024-09-12T03:19:24.850808+0000', 'mapping_epoch': 54, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 54, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:19:24.838022+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:19:24.838022+0000', 'last_clean_scrub_stamp': '2024-09-12T03:19:24.838022+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-12T03:19:29.530 INFO:tasks.ceph.ceph_manager.ceph:PG 9.2c is not active+clean 2024-09-12T03:19:29.530 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 54, 'state': 'creating+peering', 'last_fresh': '2024-09-12T03:19:24.850087+0000', 'last_change': '2024-09-12T03:19:24.850087+0000', 'last_active': '2024-09-12T03:19:24.838022+0000', 'last_peered': '2024-09-12T03:19:24.838022+0000', 'last_clean': '2024-09-12T03:19:24.838022+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:19:24.850087+0000', 'last_undegraded': '2024-09-12T03:19:24.850087+0000', 'last_fullsized': '2024-09-12T03:19:24.850087+0000', 'mapping_epoch': 54, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 54, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:19:24.838022+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:19:24.838022+0000', 'last_clean_scrub_stamp': '2024-09-12T03:19:24.838022+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-12T03:19:29.530 INFO:tasks.ceph.ceph_manager.ceph:PG 9.2b is not active+clean 2024-09-12T03:19:29.530 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 54, 'state': 'creating+peering', 'last_fresh': '2024-09-12T03:19:24.851577+0000', 'last_change': '2024-09-12T03:19:24.851577+0000', 'last_active': '2024-09-12T03:19:24.838022+0000', 'last_peered': '2024-09-12T03:19:24.838022+0000', 'last_clean': '2024-09-12T03:19:24.838022+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:19:24.851577+0000', 'last_undegraded': '2024-09-12T03:19:24.851577+0000', 'last_fullsized': '2024-09-12T03:19:24.851577+0000', 'mapping_epoch': 54, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 54, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:19:24.838022+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:19:24.838022+0000', 'last_clean_scrub_stamp': '2024-09-12T03:19:24.838022+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-12T03:19:29.530 INFO:tasks.ceph.ceph_manager.ceph:PG 9.2a is not active+clean 2024-09-12T03:19:29.530 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 54, 'state': 'creating+peering', 'last_fresh': '2024-09-12T03:19:24.856451+0000', 'last_change': '2024-09-12T03:19:24.856451+0000', 'last_active': '2024-09-12T03:19:24.838022+0000', 'last_peered': '2024-09-12T03:19:24.838022+0000', 'last_clean': '2024-09-12T03:19:24.838022+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:19:24.856451+0000', 'last_undegraded': '2024-09-12T03:19:24.856451+0000', 'last_fullsized': '2024-09-12T03:19:24.856451+0000', 'mapping_epoch': 54, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 54, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:19:24.838022+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:19:24.838022+0000', 'last_clean_scrub_stamp': '2024-09-12T03:19:24.838022+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-12T03:19:29.530 INFO:tasks.ceph.ceph_manager.ceph:PG 9.29 is not active+clean 2024-09-12T03:19:29.530 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 54, 'state': 'creating+peering', 'last_fresh': '2024-09-12T03:19:24.852682+0000', 'last_change': '2024-09-12T03:19:24.852682+0000', 'last_active': '2024-09-12T03:19:24.838022+0000', 'last_peered': '2024-09-12T03:19:24.838022+0000', 'last_clean': '2024-09-12T03:19:24.838022+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:19:24.852682+0000', 'last_undegraded': '2024-09-12T03:19:24.852682+0000', 'last_fullsized': '2024-09-12T03:19:24.852682+0000', 'mapping_epoch': 54, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 54, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:19:24.838022+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:19:24.838022+0000', 'last_clean_scrub_stamp': '2024-09-12T03:19:24.838022+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-12T03:19:29.531 INFO:tasks.ceph.ceph_manager.ceph:PG 9.28 is not active+clean 2024-09-12T03:19:29.531 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 54, 'state': 'creating+peering', 'last_fresh': '2024-09-12T03:19:24.852444+0000', 'last_change': '2024-09-12T03:19:24.852444+0000', 'last_active': '2024-09-12T03:19:24.838022+0000', 'last_peered': '2024-09-12T03:19:24.838022+0000', 'last_clean': '2024-09-12T03:19:24.838022+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:19:24.852444+0000', 'last_undegraded': '2024-09-12T03:19:24.852444+0000', 'last_fullsized': '2024-09-12T03:19:24.852444+0000', 'mapping_epoch': 54, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 54, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:19:24.838022+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:19:24.838022+0000', 'last_clean_scrub_stamp': '2024-09-12T03:19:24.838022+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-12T03:19:29.531 INFO:tasks.ceph.ceph_manager.ceph:PG 9.27 is not active+clean 2024-09-12T03:19:29.531 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 54, 'state': 'creating+peering', 'last_fresh': '2024-09-12T03:19:24.851312+0000', 'last_change': '2024-09-12T03:19:24.851312+0000', 'last_active': '2024-09-12T03:19:24.838022+0000', 'last_peered': '2024-09-12T03:19:24.838022+0000', 'last_clean': '2024-09-12T03:19:24.838022+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:19:24.851312+0000', 'last_undegraded': '2024-09-12T03:19:24.851312+0000', 'last_fullsized': '2024-09-12T03:19:24.851312+0000', 'mapping_epoch': 54, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 54, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:19:24.838022+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:19:24.838022+0000', 'last_clean_scrub_stamp': '2024-09-12T03:19:24.838022+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-12T03:19:29.531 INFO:tasks.ceph.ceph_manager.ceph:PG 9.26 is not active+clean 2024-09-12T03:19:29.531 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 54, 'state': 'creating+peering', 'last_fresh': '2024-09-12T03:19:24.848902+0000', 'last_change': '2024-09-12T03:19:24.848902+0000', 'last_active': '2024-09-12T03:19:24.838022+0000', 'last_peered': '2024-09-12T03:19:24.838022+0000', 'last_clean': '2024-09-12T03:19:24.838022+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:19:24.848902+0000', 'last_undegraded': '2024-09-12T03:19:24.848902+0000', 'last_fullsized': '2024-09-12T03:19:24.848902+0000', 'mapping_epoch': 54, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 54, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:19:24.838022+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:19:24.838022+0000', 'last_clean_scrub_stamp': '2024-09-12T03:19:24.838022+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-12T03:19:29.531 INFO:tasks.ceph.ceph_manager.ceph:PG 9.25 is not active+clean 2024-09-12T03:19:29.531 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 54, 'state': 'creating+peering', 'last_fresh': '2024-09-12T03:19:24.855319+0000', 'last_change': '2024-09-12T03:19:24.855319+0000', 'last_active': '2024-09-12T03:19:24.838022+0000', 'last_peered': '2024-09-12T03:19:24.838022+0000', 'last_clean': '2024-09-12T03:19:24.838022+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:19:24.855319+0000', 'last_undegraded': '2024-09-12T03:19:24.855319+0000', 'last_fullsized': '2024-09-12T03:19:24.855319+0000', 'mapping_epoch': 54, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 54, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:19:24.838022+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:19:24.838022+0000', 'last_clean_scrub_stamp': '2024-09-12T03:19:24.838022+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-12T03:19:29.531 INFO:tasks.ceph.ceph_manager.ceph:PG 9.24 is not active+clean 2024-09-12T03:19:29.531 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 54, 'state': 'creating+peering', 'last_fresh': '2024-09-12T03:19:24.851621+0000', 'last_change': '2024-09-12T03:19:24.851621+0000', 'last_active': '2024-09-12T03:19:24.838022+0000', 'last_peered': '2024-09-12T03:19:24.838022+0000', 'last_clean': '2024-09-12T03:19:24.838022+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:19:24.851621+0000', 'last_undegraded': '2024-09-12T03:19:24.851621+0000', 'last_fullsized': '2024-09-12T03:19:24.851621+0000', 'mapping_epoch': 54, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 54, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:19:24.838022+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:19:24.838022+0000', 'last_clean_scrub_stamp': '2024-09-12T03:19:24.838022+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-12T03:19:29.531 INFO:tasks.ceph.ceph_manager.ceph:PG 9.23 is not active+clean 2024-09-12T03:19:29.531 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 54, 'state': 'creating+peering', 'last_fresh': '2024-09-12T03:19:24.850354+0000', 'last_change': '2024-09-12T03:19:24.850354+0000', 'last_active': '2024-09-12T03:19:24.838022+0000', 'last_peered': '2024-09-12T03:19:24.838022+0000', 'last_clean': '2024-09-12T03:19:24.838022+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:19:24.850354+0000', 'last_undegraded': '2024-09-12T03:19:24.850354+0000', 'last_fullsized': '2024-09-12T03:19:24.850354+0000', 'mapping_epoch': 54, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 54, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:19:24.838022+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:19:24.838022+0000', 'last_clean_scrub_stamp': '2024-09-12T03:19:24.838022+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-12T03:19:29.532 INFO:tasks.ceph.ceph_manager.ceph:PG 9.22 is not active+clean 2024-09-12T03:19:29.532 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 54, 'state': 'creating+peering', 'last_fresh': '2024-09-12T03:19:24.850218+0000', 'last_change': '2024-09-12T03:19:24.850218+0000', 'last_active': '2024-09-12T03:19:24.838022+0000', 'last_peered': '2024-09-12T03:19:24.838022+0000', 'last_clean': '2024-09-12T03:19:24.838022+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:19:24.850218+0000', 'last_undegraded': '2024-09-12T03:19:24.850218+0000', 'last_fullsized': '2024-09-12T03:19:24.850218+0000', 'mapping_epoch': 54, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 54, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:19:24.838022+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:19:24.838022+0000', 'last_clean_scrub_stamp': '2024-09-12T03:19:24.838022+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-12T03:19:29.532 INFO:tasks.ceph.ceph_manager.ceph:PG 9.21 is not active+clean 2024-09-12T03:19:29.532 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 54, 'state': 'creating+peering', 'last_fresh': '2024-09-12T03:19:24.850269+0000', 'last_change': '2024-09-12T03:19:24.850269+0000', 'last_active': '2024-09-12T03:19:24.838022+0000', 'last_peered': '2024-09-12T03:19:24.838022+0000', 'last_clean': '2024-09-12T03:19:24.838022+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:19:24.850269+0000', 'last_undegraded': '2024-09-12T03:19:24.850269+0000', 'last_fullsized': '2024-09-12T03:19:24.850269+0000', 'mapping_epoch': 54, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 54, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:19:24.838022+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:19:24.838022+0000', 'last_clean_scrub_stamp': '2024-09-12T03:19:24.838022+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-12T03:19:29.532 INFO:tasks.ceph.ceph_manager.ceph:PG 9.20 is not active+clean 2024-09-12T03:19:29.532 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 54, 'state': 'creating+peering', 'last_fresh': '2024-09-12T03:19:24.852116+0000', 'last_change': '2024-09-12T03:19:24.852116+0000', 'last_active': '2024-09-12T03:19:24.838022+0000', 'last_peered': '2024-09-12T03:19:24.838022+0000', 'last_clean': '2024-09-12T03:19:24.838022+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:19:24.852116+0000', 'last_undegraded': '2024-09-12T03:19:24.852116+0000', 'last_fullsized': '2024-09-12T03:19:24.852116+0000', 'mapping_epoch': 54, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 54, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:19:24.838022+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:19:24.838022+0000', 'last_clean_scrub_stamp': '2024-09-12T03:19:24.838022+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-12T03:19:29.532 INFO:tasks.ceph.ceph_manager.ceph:PG 9.1e is not active+clean 2024-09-12T03:19:29.532 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 54, 'state': 'creating+peering', 'last_fresh': '2024-09-12T03:19:24.856894+0000', 'last_change': '2024-09-12T03:19:24.856894+0000', 'last_active': '2024-09-12T03:19:24.838022+0000', 'last_peered': '2024-09-12T03:19:24.838022+0000', 'last_clean': '2024-09-12T03:19:24.838022+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:19:24.856894+0000', 'last_undegraded': '2024-09-12T03:19:24.856894+0000', 'last_fullsized': '2024-09-12T03:19:24.856894+0000', 'mapping_epoch': 54, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 54, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:19:24.838022+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:19:24.838022+0000', 'last_clean_scrub_stamp': '2024-09-12T03:19:24.838022+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-12T03:19:29.532 INFO:tasks.ceph.ceph_manager.ceph:PG 9.1f is not active+clean 2024-09-12T03:19:29.532 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 54, 'state': 'creating+peering', 'last_fresh': '2024-09-12T03:19:24.848080+0000', 'last_change': '2024-09-12T03:19:24.848080+0000', 'last_active': '2024-09-12T03:19:24.838022+0000', 'last_peered': '2024-09-12T03:19:24.838022+0000', 'last_clean': '2024-09-12T03:19:24.838022+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:19:24.848080+0000', 'last_undegraded': '2024-09-12T03:19:24.848080+0000', 'last_fullsized': '2024-09-12T03:19:24.848080+0000', 'mapping_epoch': 54, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 54, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:19:24.838022+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:19:24.838022+0000', 'last_clean_scrub_stamp': '2024-09-12T03:19:24.838022+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-12T03:19:29.533 INFO:tasks.ceph.ceph_manager.ceph:PG 9.1c is not active+clean 2024-09-12T03:19:29.533 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 54, 'state': 'creating+peering', 'last_fresh': '2024-09-12T03:19:24.854180+0000', 'last_change': '2024-09-12T03:19:24.854180+0000', 'last_active': '2024-09-12T03:19:24.838022+0000', 'last_peered': '2024-09-12T03:19:24.838022+0000', 'last_clean': '2024-09-12T03:19:24.838022+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:19:24.854180+0000', 'last_undegraded': '2024-09-12T03:19:24.854180+0000', 'last_fullsized': '2024-09-12T03:19:24.854180+0000', 'mapping_epoch': 54, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 54, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:19:24.838022+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:19:24.838022+0000', 'last_clean_scrub_stamp': '2024-09-12T03:19:24.838022+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-12T03:19:29.533 INFO:tasks.ceph.ceph_manager.ceph:PG 9.1d is not active+clean 2024-09-12T03:19:29.533 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 54, 'state': 'creating+peering', 'last_fresh': '2024-09-12T03:19:24.851109+0000', 'last_change': '2024-09-12T03:19:24.851109+0000', 'last_active': '2024-09-12T03:19:24.838022+0000', 'last_peered': '2024-09-12T03:19:24.838022+0000', 'last_clean': '2024-09-12T03:19:24.838022+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:19:24.851109+0000', 'last_undegraded': '2024-09-12T03:19:24.851109+0000', 'last_fullsized': '2024-09-12T03:19:24.851109+0000', 'mapping_epoch': 54, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 54, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:19:24.838022+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:19:24.838022+0000', 'last_clean_scrub_stamp': '2024-09-12T03:19:24.838022+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-12T03:19:29.533 INFO:tasks.ceph.ceph_manager.ceph:PG 9.1a is not active+clean 2024-09-12T03:19:29.533 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 54, 'state': 'creating+peering', 'last_fresh': '2024-09-12T03:19:24.853953+0000', 'last_change': '2024-09-12T03:19:24.853953+0000', 'last_active': '2024-09-12T03:19:24.838022+0000', 'last_peered': '2024-09-12T03:19:24.838022+0000', 'last_clean': '2024-09-12T03:19:24.838022+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:19:24.853953+0000', 'last_undegraded': '2024-09-12T03:19:24.853953+0000', 'last_fullsized': '2024-09-12T03:19:24.853953+0000', 'mapping_epoch': 54, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 54, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:19:24.838022+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:19:24.838022+0000', 'last_clean_scrub_stamp': '2024-09-12T03:19:24.838022+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-12T03:19:29.533 INFO:tasks.ceph.ceph_manager.ceph:PG 9.1b is not active+clean 2024-09-12T03:19:29.533 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 54, 'state': 'creating+peering', 'last_fresh': '2024-09-12T03:19:24.851504+0000', 'last_change': '2024-09-12T03:19:24.851504+0000', 'last_active': '2024-09-12T03:19:24.838022+0000', 'last_peered': '2024-09-12T03:19:24.838022+0000', 'last_clean': '2024-09-12T03:19:24.838022+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:19:24.851504+0000', 'last_undegraded': '2024-09-12T03:19:24.851504+0000', 'last_fullsized': '2024-09-12T03:19:24.851504+0000', 'mapping_epoch': 54, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 54, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:19:24.838022+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:19:24.838022+0000', 'last_clean_scrub_stamp': '2024-09-12T03:19:24.838022+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-12T03:19:29.533 INFO:tasks.ceph.ceph_manager.ceph:PG 9.18 is not active+clean 2024-09-12T03:19:29.533 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 54, 'state': 'creating+peering', 'last_fresh': '2024-09-12T03:19:24.851467+0000', 'last_change': '2024-09-12T03:19:24.851467+0000', 'last_active': '2024-09-12T03:19:24.838022+0000', 'last_peered': '2024-09-12T03:19:24.838022+0000', 'last_clean': '2024-09-12T03:19:24.838022+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:19:24.851467+0000', 'last_undegraded': '2024-09-12T03:19:24.851467+0000', 'last_fullsized': '2024-09-12T03:19:24.851467+0000', 'mapping_epoch': 54, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 54, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:19:24.838022+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:19:24.838022+0000', 'last_clean_scrub_stamp': '2024-09-12T03:19:24.838022+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-12T03:19:29.533 INFO:tasks.ceph.ceph_manager.ceph:PG 9.19 is not active+clean 2024-09-12T03:19:29.534 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 54, 'state': 'creating+peering', 'last_fresh': '2024-09-12T03:19:24.853049+0000', 'last_change': '2024-09-12T03:19:24.853049+0000', 'last_active': '2024-09-12T03:19:24.838022+0000', 'last_peered': '2024-09-12T03:19:24.838022+0000', 'last_clean': '2024-09-12T03:19:24.838022+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:19:24.853049+0000', 'last_undegraded': '2024-09-12T03:19:24.853049+0000', 'last_fullsized': '2024-09-12T03:19:24.853049+0000', 'mapping_epoch': 54, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 54, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:19:24.838022+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:19:24.838022+0000', 'last_clean_scrub_stamp': '2024-09-12T03:19:24.838022+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-12T03:19:29.534 INFO:tasks.ceph.ceph_manager.ceph:PG 9.16 is not active+clean 2024-09-12T03:19:29.534 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 54, 'state': 'creating+peering', 'last_fresh': '2024-09-12T03:19:24.846718+0000', 'last_change': '2024-09-12T03:19:24.846718+0000', 'last_active': '2024-09-12T03:19:24.838022+0000', 'last_peered': '2024-09-12T03:19:24.838022+0000', 'last_clean': '2024-09-12T03:19:24.838022+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:19:24.846718+0000', 'last_undegraded': '2024-09-12T03:19:24.846718+0000', 'last_fullsized': '2024-09-12T03:19:24.846718+0000', 'mapping_epoch': 54, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 54, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:19:24.838022+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:19:24.838022+0000', 'last_clean_scrub_stamp': '2024-09-12T03:19:24.838022+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-12T03:19:29.534 INFO:tasks.ceph.ceph_manager.ceph:PG 9.17 is not active+clean 2024-09-12T03:19:29.534 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 54, 'state': 'creating+peering', 'last_fresh': '2024-09-12T03:19:24.849991+0000', 'last_change': '2024-09-12T03:19:24.849991+0000', 'last_active': '2024-09-12T03:19:24.838022+0000', 'last_peered': '2024-09-12T03:19:24.838022+0000', 'last_clean': '2024-09-12T03:19:24.838022+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:19:24.849991+0000', 'last_undegraded': '2024-09-12T03:19:24.849991+0000', 'last_fullsized': '2024-09-12T03:19:24.849991+0000', 'mapping_epoch': 54, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 54, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:19:24.838022+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:19:24.838022+0000', 'last_clean_scrub_stamp': '2024-09-12T03:19:24.838022+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-12T03:19:29.534 INFO:tasks.ceph.ceph_manager.ceph:PG 9.14 is not active+clean 2024-09-12T03:19:29.534 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 54, 'state': 'creating+peering', 'last_fresh': '2024-09-12T03:19:24.848059+0000', 'last_change': '2024-09-12T03:19:24.848059+0000', 'last_active': '2024-09-12T03:19:24.838022+0000', 'last_peered': '2024-09-12T03:19:24.838022+0000', 'last_clean': '2024-09-12T03:19:24.838022+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:19:24.848059+0000', 'last_undegraded': '2024-09-12T03:19:24.848059+0000', 'last_fullsized': '2024-09-12T03:19:24.848059+0000', 'mapping_epoch': 54, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 54, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:19:24.838022+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:19:24.838022+0000', 'last_clean_scrub_stamp': '2024-09-12T03:19:24.838022+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-12T03:19:29.534 INFO:tasks.ceph.ceph_manager.ceph:PG 9.15 is not active+clean 2024-09-12T03:19:29.534 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 54, 'state': 'creating+peering', 'last_fresh': '2024-09-12T03:19:24.852512+0000', 'last_change': '2024-09-12T03:19:24.852512+0000', 'last_active': '2024-09-12T03:19:24.838022+0000', 'last_peered': '2024-09-12T03:19:24.838022+0000', 'last_clean': '2024-09-12T03:19:24.838022+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:19:24.852512+0000', 'last_undegraded': '2024-09-12T03:19:24.852512+0000', 'last_fullsized': '2024-09-12T03:19:24.852512+0000', 'mapping_epoch': 54, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 54, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:19:24.838022+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:19:24.838022+0000', 'last_clean_scrub_stamp': '2024-09-12T03:19:24.838022+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-12T03:19:29.534 INFO:tasks.ceph.ceph_manager.ceph:PG 9.12 is not active+clean 2024-09-12T03:19:29.534 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 54, 'state': 'creating+peering', 'last_fresh': '2024-09-12T03:19:24.847408+0000', 'last_change': '2024-09-12T03:19:24.847408+0000', 'last_active': '2024-09-12T03:19:24.838022+0000', 'last_peered': '2024-09-12T03:19:24.838022+0000', 'last_clean': '2024-09-12T03:19:24.838022+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:19:24.847408+0000', 'last_undegraded': '2024-09-12T03:19:24.847408+0000', 'last_fullsized': '2024-09-12T03:19:24.847408+0000', 'mapping_epoch': 54, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 54, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:19:24.838022+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:19:24.838022+0000', 'last_clean_scrub_stamp': '2024-09-12T03:19:24.838022+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-12T03:19:29.535 INFO:tasks.ceph.ceph_manager.ceph:PG 9.13 is not active+clean 2024-09-12T03:19:29.535 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 54, 'state': 'creating+peering', 'last_fresh': '2024-09-12T03:19:24.851140+0000', 'last_change': '2024-09-12T03:19:24.851140+0000', 'last_active': '2024-09-12T03:19:24.838022+0000', 'last_peered': '2024-09-12T03:19:24.838022+0000', 'last_clean': '2024-09-12T03:19:24.838022+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:19:24.851140+0000', 'last_undegraded': '2024-09-12T03:19:24.851140+0000', 'last_fullsized': '2024-09-12T03:19:24.851140+0000', 'mapping_epoch': 54, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 54, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:19:24.838022+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:19:24.838022+0000', 'last_clean_scrub_stamp': '2024-09-12T03:19:24.838022+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-12T03:19:29.535 INFO:tasks.ceph.ceph_manager.ceph:PG 9.10 is not active+clean 2024-09-12T03:19:29.535 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 54, 'state': 'creating+peering', 'last_fresh': '2024-09-12T03:19:24.846337+0000', 'last_change': '2024-09-12T03:19:24.846337+0000', 'last_active': '2024-09-12T03:19:24.838022+0000', 'last_peered': '2024-09-12T03:19:24.838022+0000', 'last_clean': '2024-09-12T03:19:24.838022+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:19:24.846337+0000', 'last_undegraded': '2024-09-12T03:19:24.846337+0000', 'last_fullsized': '2024-09-12T03:19:24.846337+0000', 'mapping_epoch': 54, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 54, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:19:24.838022+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:19:24.838022+0000', 'last_clean_scrub_stamp': '2024-09-12T03:19:24.838022+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-12T03:19:29.535 INFO:tasks.ceph.ceph_manager.ceph:PG 9.11 is not active+clean 2024-09-12T03:19:29.535 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 54, 'state': 'creating+peering', 'last_fresh': '2024-09-12T03:19:24.852242+0000', 'last_change': '2024-09-12T03:19:24.852242+0000', 'last_active': '2024-09-12T03:19:24.838022+0000', 'last_peered': '2024-09-12T03:19:24.838022+0000', 'last_clean': '2024-09-12T03:19:24.838022+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:19:24.852242+0000', 'last_undegraded': '2024-09-12T03:19:24.852242+0000', 'last_fullsized': '2024-09-12T03:19:24.852242+0000', 'mapping_epoch': 54, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 54, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:19:24.838022+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:19:24.838022+0000', 'last_clean_scrub_stamp': '2024-09-12T03:19:24.838022+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-12T03:19:29.535 INFO:tasks.ceph.ceph_manager.ceph:PG 9.e is not active+clean 2024-09-12T03:19:29.535 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 54, 'state': 'creating+peering', 'last_fresh': '2024-09-12T03:19:24.851328+0000', 'last_change': '2024-09-12T03:19:24.851328+0000', 'last_active': '2024-09-12T03:19:24.838022+0000', 'last_peered': '2024-09-12T03:19:24.838022+0000', 'last_clean': '2024-09-12T03:19:24.838022+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:19:24.851328+0000', 'last_undegraded': '2024-09-12T03:19:24.851328+0000', 'last_fullsized': '2024-09-12T03:19:24.851328+0000', 'mapping_epoch': 54, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 54, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:19:24.838022+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:19:24.838022+0000', 'last_clean_scrub_stamp': '2024-09-12T03:19:24.838022+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-12T03:19:29.535 INFO:tasks.ceph.ceph_manager.ceph:PG 9.f is not active+clean 2024-09-12T03:19:29.535 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 54, 'state': 'creating+peering', 'last_fresh': '2024-09-12T03:19:24.848416+0000', 'last_change': '2024-09-12T03:19:24.848416+0000', 'last_active': '2024-09-12T03:19:24.838022+0000', 'last_peered': '2024-09-12T03:19:24.838022+0000', 'last_clean': '2024-09-12T03:19:24.838022+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:19:24.848416+0000', 'last_undegraded': '2024-09-12T03:19:24.848416+0000', 'last_fullsized': '2024-09-12T03:19:24.848416+0000', 'mapping_epoch': 54, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 54, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:19:24.838022+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:19:24.838022+0000', 'last_clean_scrub_stamp': '2024-09-12T03:19:24.838022+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-12T03:19:29.535 INFO:tasks.ceph.ceph_manager.ceph:PG 9.c is not active+clean 2024-09-12T03:19:29.535 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 54, 'state': 'creating+peering', 'last_fresh': '2024-09-12T03:19:24.849465+0000', 'last_change': '2024-09-12T03:19:24.849465+0000', 'last_active': '2024-09-12T03:19:24.838022+0000', 'last_peered': '2024-09-12T03:19:24.838022+0000', 'last_clean': '2024-09-12T03:19:24.838022+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:19:24.849465+0000', 'last_undegraded': '2024-09-12T03:19:24.849465+0000', 'last_fullsized': '2024-09-12T03:19:24.849465+0000', 'mapping_epoch': 54, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 54, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:19:24.838022+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:19:24.838022+0000', 'last_clean_scrub_stamp': '2024-09-12T03:19:24.838022+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-12T03:19:29.536 INFO:tasks.ceph.ceph_manager.ceph:PG 9.d is not active+clean 2024-09-12T03:19:29.536 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 54, 'state': 'creating+peering', 'last_fresh': '2024-09-12T03:19:24.851636+0000', 'last_change': '2024-09-12T03:19:24.851636+0000', 'last_active': '2024-09-12T03:19:24.838022+0000', 'last_peered': '2024-09-12T03:19:24.838022+0000', 'last_clean': '2024-09-12T03:19:24.838022+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:19:24.851636+0000', 'last_undegraded': '2024-09-12T03:19:24.851636+0000', 'last_fullsized': '2024-09-12T03:19:24.851636+0000', 'mapping_epoch': 54, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 54, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:19:24.838022+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:19:24.838022+0000', 'last_clean_scrub_stamp': '2024-09-12T03:19:24.838022+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-12T03:19:29.536 INFO:tasks.ceph.ceph_manager.ceph:PG 9.a is not active+clean 2024-09-12T03:19:29.536 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 54, 'state': 'creating+peering', 'last_fresh': '2024-09-12T03:19:24.846915+0000', 'last_change': '2024-09-12T03:19:24.846915+0000', 'last_active': '2024-09-12T03:19:24.838022+0000', 'last_peered': '2024-09-12T03:19:24.838022+0000', 'last_clean': '2024-09-12T03:19:24.838022+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:19:24.846915+0000', 'last_undegraded': '2024-09-12T03:19:24.846915+0000', 'last_fullsized': '2024-09-12T03:19:24.846915+0000', 'mapping_epoch': 54, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 54, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:19:24.838022+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:19:24.838022+0000', 'last_clean_scrub_stamp': '2024-09-12T03:19:24.838022+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-12T03:19:29.536 INFO:tasks.ceph.ceph_manager.ceph:PG 9.b is not active+clean 2024-09-12T03:19:29.536 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 54, 'state': 'creating+peering', 'last_fresh': '2024-09-12T03:19:24.855497+0000', 'last_change': '2024-09-12T03:19:24.855497+0000', 'last_active': '2024-09-12T03:19:24.838022+0000', 'last_peered': '2024-09-12T03:19:24.838022+0000', 'last_clean': '2024-09-12T03:19:24.838022+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:19:24.855497+0000', 'last_undegraded': '2024-09-12T03:19:24.855497+0000', 'last_fullsized': '2024-09-12T03:19:24.855497+0000', 'mapping_epoch': 54, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 54, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:19:24.838022+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:19:24.838022+0000', 'last_clean_scrub_stamp': '2024-09-12T03:19:24.838022+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-12T03:19:29.536 INFO:tasks.ceph.ceph_manager.ceph:PG 9.9 is not active+clean 2024-09-12T03:19:29.536 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 54, 'state': 'creating+peering', 'last_fresh': '2024-09-12T03:19:24.852418+0000', 'last_change': '2024-09-12T03:19:24.852418+0000', 'last_active': '2024-09-12T03:19:24.838022+0000', 'last_peered': '2024-09-12T03:19:24.838022+0000', 'last_clean': '2024-09-12T03:19:24.838022+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:19:24.852418+0000', 'last_undegraded': '2024-09-12T03:19:24.852418+0000', 'last_fullsized': '2024-09-12T03:19:24.852418+0000', 'mapping_epoch': 54, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 54, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:19:24.838022+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:19:24.838022+0000', 'last_clean_scrub_stamp': '2024-09-12T03:19:24.838022+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-12T03:19:29.536 INFO:tasks.ceph.ceph_manager.ceph:PG 9.6 is not active+clean 2024-09-12T03:19:29.536 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 54, 'state': 'creating+peering', 'last_fresh': '2024-09-12T03:19:24.847841+0000', 'last_change': '2024-09-12T03:19:24.847841+0000', 'last_active': '2024-09-12T03:19:24.838022+0000', 'last_peered': '2024-09-12T03:19:24.838022+0000', 'last_clean': '2024-09-12T03:19:24.838022+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:19:24.847841+0000', 'last_undegraded': '2024-09-12T03:19:24.847841+0000', 'last_fullsized': '2024-09-12T03:19:24.847841+0000', 'mapping_epoch': 54, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 54, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:19:24.838022+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:19:24.838022+0000', 'last_clean_scrub_stamp': '2024-09-12T03:19:24.838022+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-12T03:19:29.537 INFO:tasks.ceph.ceph_manager.ceph:PG 9.7 is not active+clean 2024-09-12T03:19:29.537 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 54, 'state': 'creating+peering', 'last_fresh': '2024-09-12T03:19:24.847949+0000', 'last_change': '2024-09-12T03:19:24.847949+0000', 'last_active': '2024-09-12T03:19:24.838022+0000', 'last_peered': '2024-09-12T03:19:24.838022+0000', 'last_clean': '2024-09-12T03:19:24.838022+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:19:24.847949+0000', 'last_undegraded': '2024-09-12T03:19:24.847949+0000', 'last_fullsized': '2024-09-12T03:19:24.847949+0000', 'mapping_epoch': 54, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 54, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:19:24.838022+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:19:24.838022+0000', 'last_clean_scrub_stamp': '2024-09-12T03:19:24.838022+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-12T03:19:29.537 INFO:tasks.ceph.ceph_manager.ceph:PG 9.4 is not active+clean 2024-09-12T03:19:29.537 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 54, 'state': 'creating+peering', 'last_fresh': '2024-09-12T03:19:24.847544+0000', 'last_change': '2024-09-12T03:19:24.847544+0000', 'last_active': '2024-09-12T03:19:24.838022+0000', 'last_peered': '2024-09-12T03:19:24.838022+0000', 'last_clean': '2024-09-12T03:19:24.838022+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:19:24.847544+0000', 'last_undegraded': '2024-09-12T03:19:24.847544+0000', 'last_fullsized': '2024-09-12T03:19:24.847544+0000', 'mapping_epoch': 54, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 54, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:19:24.838022+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:19:24.838022+0000', 'last_clean_scrub_stamp': '2024-09-12T03:19:24.838022+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-12T03:19:29.537 INFO:tasks.ceph.ceph_manager.ceph:PG 9.5 is not active+clean 2024-09-12T03:19:29.537 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 54, 'state': 'creating+peering', 'last_fresh': '2024-09-12T03:19:24.850621+0000', 'last_change': '2024-09-12T03:19:24.850621+0000', 'last_active': '2024-09-12T03:19:24.838022+0000', 'last_peered': '2024-09-12T03:19:24.838022+0000', 'last_clean': '2024-09-12T03:19:24.838022+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:19:24.850621+0000', 'last_undegraded': '2024-09-12T03:19:24.850621+0000', 'last_fullsized': '2024-09-12T03:19:24.850621+0000', 'mapping_epoch': 54, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 54, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:19:24.838022+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:19:24.838022+0000', 'last_clean_scrub_stamp': '2024-09-12T03:19:24.838022+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-12T03:19:29.537 INFO:tasks.ceph.ceph_manager.ceph:PG 9.2 is not active+clean 2024-09-12T03:19:29.537 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 54, 'state': 'creating+peering', 'last_fresh': '2024-09-12T03:19:24.853481+0000', 'last_change': '2024-09-12T03:19:24.853481+0000', 'last_active': '2024-09-12T03:19:24.838022+0000', 'last_peered': '2024-09-12T03:19:24.838022+0000', 'last_clean': '2024-09-12T03:19:24.838022+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:19:24.853481+0000', 'last_undegraded': '2024-09-12T03:19:24.853481+0000', 'last_fullsized': '2024-09-12T03:19:24.853481+0000', 'mapping_epoch': 54, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 54, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:19:24.838022+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:19:24.838022+0000', 'last_clean_scrub_stamp': '2024-09-12T03:19:24.838022+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-12T03:19:29.538 INFO:tasks.ceph.ceph_manager.ceph:PG 9.3 is not active+clean 2024-09-12T03:19:29.538 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 54, 'state': 'creating+peering', 'last_fresh': '2024-09-12T03:19:24.850646+0000', 'last_change': '2024-09-12T03:19:24.850646+0000', 'last_active': '2024-09-12T03:19:24.838022+0000', 'last_peered': '2024-09-12T03:19:24.838022+0000', 'last_clean': '2024-09-12T03:19:24.838022+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:19:24.850646+0000', 'last_undegraded': '2024-09-12T03:19:24.850646+0000', 'last_fullsized': '2024-09-12T03:19:24.850646+0000', 'mapping_epoch': 54, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 54, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:19:24.838022+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:19:24.838022+0000', 'last_clean_scrub_stamp': '2024-09-12T03:19:24.838022+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-12T03:19:29.538 INFO:tasks.ceph.ceph_manager.ceph:PG 9.0 is not active+clean 2024-09-12T03:19:29.538 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 54, 'state': 'creating+peering', 'last_fresh': '2024-09-12T03:19:24.846974+0000', 'last_change': '2024-09-12T03:19:24.846974+0000', 'last_active': '2024-09-12T03:19:24.838022+0000', 'last_peered': '2024-09-12T03:19:24.838022+0000', 'last_clean': '2024-09-12T03:19:24.838022+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:19:24.846974+0000', 'last_undegraded': '2024-09-12T03:19:24.846974+0000', 'last_fullsized': '2024-09-12T03:19:24.846974+0000', 'mapping_epoch': 54, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 54, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:19:24.838022+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:19:24.838022+0000', 'last_clean_scrub_stamp': '2024-09-12T03:19:24.838022+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-12T03:19:29.538 INFO:tasks.ceph.ceph_manager.ceph:PG 9.1 is not active+clean 2024-09-12T03:19:29.538 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 54, 'state': 'creating+peering', 'last_fresh': '2024-09-12T03:19:24.852543+0000', 'last_change': '2024-09-12T03:19:24.852543+0000', 'last_active': '2024-09-12T03:19:24.838022+0000', 'last_peered': '2024-09-12T03:19:24.838022+0000', 'last_clean': '2024-09-12T03:19:24.838022+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:19:24.852543+0000', 'last_undegraded': '2024-09-12T03:19:24.852543+0000', 'last_fullsized': '2024-09-12T03:19:24.852543+0000', 'mapping_epoch': 54, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 54, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:19:24.838022+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:19:24.838022+0000', 'last_clean_scrub_stamp': '2024-09-12T03:19:24.838022+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-12T03:19:29.538 INFO:tasks.ceph.ceph_manager.ceph:PG 9.8 is not active+clean 2024-09-12T03:19:29.538 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 54, 'state': 'creating+peering', 'last_fresh': '2024-09-12T03:19:24.851171+0000', 'last_change': '2024-09-12T03:19:24.851171+0000', 'last_active': '2024-09-12T03:19:24.838022+0000', 'last_peered': '2024-09-12T03:19:24.838022+0000', 'last_clean': '2024-09-12T03:19:24.838022+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:19:24.851171+0000', 'last_undegraded': '2024-09-12T03:19:24.851171+0000', 'last_fullsized': '2024-09-12T03:19:24.851171+0000', 'mapping_epoch': 54, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 54, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:19:24.838022+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:19:24.838022+0000', 'last_clean_scrub_stamp': '2024-09-12T03:19:24.838022+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-12T03:19:29.539 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-12T03:19:29.664 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:29.663+0000 7fecc2d42640 1 Processor -- start 2024-09-12T03:19:29.664 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:29.663+0000 7fecc2d42640 1 -- start start 2024-09-12T03:19:29.665 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:29.664+0000 7fecc2d42640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fecbc15e780 0x7fecbc0d3e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:29.665 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:29.664+0000 7fecc2d42640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fecbc0d43b0 0x7fecbc0d1280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:29.665 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:29.664+0000 7fecc2d42640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fecbc0b6690 con 0x7fecbc15eb60 2024-09-12T03:19:29.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:29.664+0000 7fecc2d42640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fecbc0b7950 con 0x7fecbc15e780 2024-09-12T03:19:29.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:29.664+0000 7fecc2d42640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fecbc0b7210 con 0x7fecbc0d43b0 2024-09-12T03:19:29.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:29.664+0000 7fecc12df640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fecbc15eb60 0x7fecbc0b3390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:42380/0 (socket says 172.21.15.27:42380) 2024-09-12T03:19:29.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:29.664+0000 7fecc12df640 1 -- 172.21.15.27:0/3303110516 learned_addr learned my addr 172.21.15.27:0/3303110516 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:19:29.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:29.664+0000 7fecc0ade640 1 --2- 172.21.15.27:0/3303110516 >> v2:172.21.15.45:3300/0 conn(0x7fecbc15e780 0x7fecbc0d3e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:29.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:29.664+0000 7fecbbfff640 1 --2- 172.21.15.27:0/3303110516 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fecbc0d43b0 0x7fecbc0d1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:29.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:29.664+0000 7fecbb7fe640 1 -- 172.21.15.27:0/3303110516 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3971776396 0 0) 0x7fecbc0b6690 con 0x7fecbc15eb60 2024-09-12T03:19:29.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:29.664+0000 7fecbb7fe640 1 -- 172.21.15.27:0/3303110516 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fecac003520 con 0x7fecbc15eb60 2024-09-12T03:19:29.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:29.664+0000 7fecc0ade640 1 -- 172.21.15.27:0/3303110516 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fecbc0d43b0 msgr2=0x7fecbc0d1280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:29.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:29.664+0000 7fecc0ade640 1 --2- 172.21.15.27:0/3303110516 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fecbc0d43b0 0x7fecbc0d1280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:29.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:29.664+0000 7fecc0ade640 1 -- 172.21.15.27:0/3303110516 >> v1:172.21.15.27:6789/0 conn(0x7fecbc15eb60 legacy=0x7fecbc0b3390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:29.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:29.664+0000 7fecc0ade640 1 -- 172.21.15.27:0/3303110516 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecbc0b6970 con 0x7fecbc15e780 2024-09-12T03:19:29.667 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:29.665+0000 7fecc0ade640 1 --2- 172.21.15.27:0/3303110516 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fecbc15e780 0x7fecbc0d3e70 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fecb00097a0 tx=0x7fecb002a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4248cad368d5e494 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:29.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:29.665+0000 7fecbb7fe640 1 -- 172.21.15.27:0/3303110516 <== mon.1 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecb0037070 con 0x7fecbc15e780 2024-09-12T03:19:29.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:29.666+0000 7fecbb7fe640 1 -- 172.21.15.27:0/3303110516 <== mon.1 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fecb0032440 con 0x7fecbc15e780 2024-09-12T03:19:29.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:29.666+0000 7fecbb7fe640 1 -- 172.21.15.27:0/3303110516 <== mon.1 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecb00327f0 con 0x7fecbc15e780 2024-09-12T03:19:29.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:29.666+0000 7fecc2d42640 1 -- 172.21.15.27:0/3303110516 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fecbc15e780 msgr2=0x7fecbc0d3e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:29.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:29.666+0000 7fecc2d42640 1 --2- 172.21.15.27:0/3303110516 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fecbc15e780 0x7fecbc0d3e70 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fecb00097a0 tx=0x7fecb002a180 comp rx=0 tx=0).stop 2024-09-12T03:19:29.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:29.666+0000 7fecc2d42640 1 -- 172.21.15.27:0/3303110516 shutdown_connections 2024-09-12T03:19:29.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:29.666+0000 7fecc2d42640 1 --2- 172.21.15.27:0/3303110516 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fecbc0d43b0 0x7fecbc0d1280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:29.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:29.666+0000 7fecc2d42640 1 --2- 172.21.15.27:0/3303110516 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fecbc15e780 0x7fecbc0d3e70 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:29.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:29.666+0000 7fecc2d42640 1 -- 172.21.15.27:0/3303110516 >> 172.21.15.27:0/3303110516 conn(0x7fecbc06da60 msgr2=0x7fecbc06de70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:29.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:29.666+0000 7fecc2d42640 1 -- 172.21.15.27:0/3303110516 shutdown_connections 2024-09-12T03:19:29.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:29.666+0000 7fecc2d42640 1 -- 172.21.15.27:0/3303110516 wait complete. 2024-09-12T03:19:29.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:29.667+0000 7fecc2d42640 1 Processor -- start 2024-09-12T03:19:29.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:29.667+0000 7fecc2d42640 1 -- start start 2024-09-12T03:19:29.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:29.668+0000 7fecc2d42640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fecbc0d43b0 0x7fecbc0cad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:29.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:29.668+0000 7fecc2d42640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fecbc15e780 0x7fecbc0c83d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:29.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:29.668+0000 7fecc2d42640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fecbc15eb60 0x7fecbc0ab310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:29.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:29.668+0000 7fecc2d42640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fecbc0b6340 con 0x7fecbc15e780 2024-09-12T03:19:29.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:29.668+0000 7fecc2d42640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fecbc0b6f30 con 0x7fecbc15eb60 2024-09-12T03:19:29.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:29.668+0000 7fecc2d42640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fecbc0d17c0 con 0x7fecbc0d43b0 2024-09-12T03:19:29.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:29.668+0000 7fecbbfff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fecbc15e780 0x7fecbc0c83d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:29.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:29.668+0000 7fecbbfff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fecbc15e780 0x7fecbc0c83d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38358/0 (socket says 172.21.15.27:38358) 2024-09-12T03:19:29.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:29.668+0000 7fecbbfff640 1 -- 172.21.15.27:0/2478948878 learned_addr learned my addr 172.21.15.27:0/2478948878 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:19:29.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:29.668+0000 7fecc12df640 1 --2- 172.21.15.27:0/2478948878 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fecbc15eb60 0x7fecbc0ab310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:29.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:29.668+0000 7fecc0ade640 1 --2- 172.21.15.27:0/2478948878 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fecbc0d43b0 0x7fecbc0cad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:29.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:29.669+0000 7fecbbfff640 1 -- 172.21.15.27:0/2478948878 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fecbc0d43b0 msgr2=0x7fecbc0cad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:29.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:29.669+0000 7fecbbfff640 1 --2- 172.21.15.27:0/2478948878 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fecbc0d43b0 0x7fecbc0cad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:29.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:29.669+0000 7fecbbfff640 1 -- 172.21.15.27:0/2478948878 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fecbc15eb60 msgr2=0x7fecbc0ab310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:29.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:29.669+0000 7fecbbfff640 1 --2- 172.21.15.27:0/2478948878 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fecbc15eb60 0x7fecbc0ab310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:29.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:29.669+0000 7fecbbfff640 1 -- 172.21.15.27:0/2478948878 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecbc0c8db0 con 0x7fecbc15e780 2024-09-12T03:19:29.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:29.669+0000 7fecc12df640 1 --2- 172.21.15.27:0/2478948878 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fecbc15eb60 0x7fecbc0ab310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:19:29.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:29.669+0000 7fecbbfff640 1 --2- 172.21.15.27:0/2478948878 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fecbc15e780 0x7fecbc0c83d0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7feca800e9e0 tx=0x7feca800eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:29.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:29.669+0000 7fecb97fa640 1 -- 172.21.15.27:0/2478948878 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feca8017070 con 0x7fecbc15e780 2024-09-12T03:19:29.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:29.669+0000 7fecb97fa640 1 -- 172.21.15.27:0/2478948878 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feca800c970 con 0x7fecbc15e780 2024-09-12T03:19:29.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:29.669+0000 7fecc2d42640 1 -- 172.21.15.27:0/2478948878 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fecbc0c9090 con 0x7fecbc15e780 2024-09-12T03:19:29.674 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:29.669+0000 7fecb97fa640 1 -- 172.21.15.27:0/2478948878 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feca800cd60 con 0x7fecbc15e780 2024-09-12T03:19:29.674 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:29.669+0000 7fecc2d42640 1 -- 172.21.15.27:0/2478948878 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fecbc0c9650 con 0x7fecbc15e780 2024-09-12T03:19:29.674 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:29.670+0000 7fecc2d42640 1 -- 172.21.15.27:0/2478948878 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fecbc0d1b50 con 0x7fecbc15e780 2024-09-12T03:19:29.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:29.670+0000 7fecb97fa640 1 -- 172.21.15.27:0/2478948878 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 9) ==== 90651+0+0 (secure 0 0 0) 0x7feca801a070 con 0x7fecbc15e780 2024-09-12T03:19:29.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:29.671+0000 7fecb97fa640 1 --2- 172.21.15.27:0/2478948878 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7feca006bfe0 0x7feca006e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:29.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:29.671+0000 7fecb97fa640 1 -- 172.21.15.27:0/2478948878 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5709+0+0 (secure 0 0 0) 0x7feca808b540 con 0x7fecbc15e780 2024-09-12T03:19:29.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:29.671+0000 7fecc0ade640 1 --2- 172.21.15.27:0/2478948878 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7feca006bfe0 0x7feca006e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:29.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:29.672+0000 7fecc0ade640 1 --2- 172.21.15.27:0/2478948878 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7feca006bfe0 0x7feca006e4a0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fecb0034000 tx=0x7fecb002e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:29.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:29.675+0000 7fecb97fa640 1 -- 172.21.15.27:0/2478948878 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7feca8059bf0 con 0x7fecbc15e780 2024-09-12T03:19:29.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:29.916+0000 7fecc2d42640 1 -- 172.21.15.27:0/2478948878 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7fecbc0cf3b0 con 0x7fecbc15e780 2024-09-12T03:19:29.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:29.916+0000 7fecb97fa640 1 -- 172.21.15.27:0/2478948878 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1076 (secure 0 0 0) 0x7feca80594f0 con 0x7fecbc15e780 2024-09-12T03:19:29.918 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:19:29.918 INFO:teuthology.orchestra.run.smithi027.stdout:{"fsid":"d2df504e-2647-4385-a5db-9fdf69f487a0","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":202,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":56,"num_osds":8,"num_up_osds":8,"osd_up_since":1726110979,"num_in_osds":8,"osd_in_since":1726110969,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"creating+peering","count":64},{"state_name":"active+clean","count":33}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":253267968,"bytes_avail":772840845312,"bytes_total":773094113280,"inactive_pgs_ratio":0.65979379415512085},"fsmap":{"epoch":30,"btime":"2024-09-12T03:19:27:848915+0000","id":4,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":4,"rank":0,"name":"b","status":"up:active","gid":5649}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-12T03:18:48.084437+0000","services":{}},"progress_events":{}} 2024-09-12T03:19:29.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:29.918+0000 7fecc2d42640 1 -- 172.21.15.27:0/2478948878 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7feca006bfe0 msgr2=0x7feca006e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:29.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:29.918+0000 7fecc2d42640 1 --2- 172.21.15.27:0/2478948878 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7feca006bfe0 0x7feca006e4a0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fecb0034000 tx=0x7fecb002e000 comp rx=0 tx=0).stop 2024-09-12T03:19:29.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:29.918+0000 7fecc2d42640 1 -- 172.21.15.27:0/2478948878 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fecbc15e780 msgr2=0x7fecbc0c83d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:29.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:29.918+0000 7fecc2d42640 1 --2- 172.21.15.27:0/2478948878 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fecbc15e780 0x7fecbc0c83d0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7feca800e9e0 tx=0x7feca800eea0 comp rx=0 tx=0).stop 2024-09-12T03:19:29.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:29.918+0000 7fecc2d42640 1 -- 172.21.15.27:0/2478948878 shutdown_connections 2024-09-12T03:19:29.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:29.918+0000 7fecc2d42640 1 --2- 172.21.15.27:0/2478948878 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7feca006bfe0 0x7feca006e4a0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:29.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:29.918+0000 7fecc2d42640 1 --2- 172.21.15.27:0/2478948878 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fecbc15eb60 0x7fecbc0ab310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:29.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:29.918+0000 7fecc2d42640 1 --2- 172.21.15.27:0/2478948878 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fecbc15e780 0x7fecbc0c83d0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:29.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:29.918+0000 7fecc2d42640 1 --2- 172.21.15.27:0/2478948878 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fecbc0d43b0 0x7fecbc0cad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:29.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:29.918+0000 7fecc2d42640 1 -- 172.21.15.27:0/2478948878 >> 172.21.15.27:0/2478948878 conn(0x7fecbc06da60 msgr2=0x7fecbc0b86d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:29.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:29.919+0000 7fecc2d42640 1 -- 172.21.15.27:0/2478948878 shutdown_connections 2024-09-12T03:19:29.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:29.919+0000 7fecc2d42640 1 -- 172.21.15.27:0/2478948878 wait complete. 2024-09-12T03:19:29.930 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-09-12T03:19:29.931 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-12T03:19:30.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:30.031+0000 7faf70f6d640 1 Processor -- start 2024-09-12T03:19:30.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:30.031+0000 7faf70f6d640 1 -- start start 2024-09-12T03:19:30.034 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:30.032+0000 7faf70f6d640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7faf6c15eb90 0x7faf6c0d2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:30.034 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:30.032+0000 7faf70f6d640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7faf6c0d1720 0x7faf6c0b8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:30.034 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:30.032+0000 7faf70f6d640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faf6c0d3760 con 0x7faf6c0d5200 2024-09-12T03:19:30.034 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:30.032+0000 7faf70f6d640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7faf6c0b7d20 con 0x7faf6c0d1720 2024-09-12T03:19:30.034 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:30.032+0000 7faf70f6d640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7faf6c0d42e0 con 0x7faf6c15eb90 2024-09-12T03:19:30.034 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:30.032+0000 7faf6ad9d640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7faf6c0d1720 0x7faf6c0b8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:30.034 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:30.033+0000 7faf69d9b640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7faf6c0d5200 0x7faf6c0d1010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:42384/0 (socket says 172.21.15.27:42384) 2024-09-12T03:19:30.034 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:30.033+0000 7faf69d9b640 1 -- 172.21.15.27:0/4192625923 learned_addr learned my addr 172.21.15.27:0/4192625923 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:19:30.034 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:30.033+0000 7faf6ad9d640 1 --2- 172.21.15.27:0/4192625923 >> v2:172.21.15.27:3300/0 conn(0x7faf6c0d1720 0x7faf6c0b8190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38364/0 (socket says 172.21.15.27:38364) 2024-09-12T03:19:30.034 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:30.033+0000 7faf6a59c640 1 --2- 172.21.15.27:0/4192625923 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7faf6c15eb90 0x7faf6c0d2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:30.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:30.033+0000 7faf6ad9d640 1 -- 172.21.15.27:0/4192625923 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7faf6c15eb90 msgr2=0x7faf6c0d2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:30.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:30.033+0000 7faf6ad9d640 1 --2- 172.21.15.27:0/4192625923 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7faf6c15eb90 0x7faf6c0d2df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:30.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:30.033+0000 7faf6ad9d640 1 -- 172.21.15.27:0/4192625923 >> v1:172.21.15.27:6789/0 conn(0x7faf6c0d5200 legacy=0x7faf6c0d1010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:30.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:30.033+0000 7faf6959a640 1 -- 172.21.15.27:0/4192625923 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1758513130 0 0) 0x7faf6c0d3760 con 0x7faf6c0d5200 2024-09-12T03:19:30.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:30.033+0000 7faf6ad9d640 1 -- 172.21.15.27:0/4192625923 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf6c0d3a40 con 0x7faf6c0d1720 2024-09-12T03:19:30.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:30.034+0000 7faf6ad9d640 1 --2- 172.21.15.27:0/4192625923 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faf6c0d1720 0x7faf6c0b8190 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7faf640097a0 tx=0x7faf6402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bc090430f7d6114 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:30.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:30.034+0000 7faf6959a640 1 -- 172.21.15.27:0/4192625923 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf64037070 con 0x7faf6c0d1720 2024-09-12T03:19:30.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:30.034+0000 7faf6959a640 1 -- 172.21.15.27:0/4192625923 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf64032440 con 0x7faf6c0d1720 2024-09-12T03:19:30.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:30.034+0000 7faf6959a640 1 -- 172.21.15.27:0/4192625923 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf64032810 con 0x7faf6c0d1720 2024-09-12T03:19:30.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:30.034+0000 7faf70f6d640 1 -- 172.21.15.27:0/4192625923 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faf6c0d1720 msgr2=0x7faf6c0b8190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:30.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:30.034+0000 7faf70f6d640 1 --2- 172.21.15.27:0/4192625923 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faf6c0d1720 0x7faf6c0b8190 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7faf640097a0 tx=0x7faf6402a180 comp rx=0 tx=0).stop 2024-09-12T03:19:30.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:30.034+0000 7faf70f6d640 1 -- 172.21.15.27:0/4192625923 shutdown_connections 2024-09-12T03:19:30.037 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:30.034+0000 7faf70f6d640 1 --2- 172.21.15.27:0/4192625923 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faf6c0d1720 0x7faf6c0b8190 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:30.037 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:30.034+0000 7faf70f6d640 1 --2- 172.21.15.27:0/4192625923 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7faf6c15eb90 0x7faf6c0d2df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:30.037 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:30.034+0000 7faf70f6d640 1 -- 172.21.15.27:0/4192625923 >> 172.21.15.27:0/4192625923 conn(0x7faf6c06dce0 msgr2=0x7faf6c06e0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:30.037 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:30.034+0000 7faf70f6d640 1 -- 172.21.15.27:0/4192625923 shutdown_connections 2024-09-12T03:19:30.037 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:30.034+0000 7faf70f6d640 1 -- 172.21.15.27:0/4192625923 wait complete. 2024-09-12T03:19:30.039 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:30.036+0000 7faf70f6d640 1 Processor -- start 2024-09-12T03:19:30.039 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:30.036+0000 7faf70f6d640 1 -- start start 2024-09-12T03:19:30.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:30.036+0000 7faf70f6d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faf6c0d1720 0x7faf6c0c9fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:30.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:30.037+0000 7faf70f6d640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7faf6c0d5200 0x7faf6c149c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:30.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:30.037+0000 7faf70f6d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faf6c15eb90 0x7faf6c0c8340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:30.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:30.037+0000 7faf70f6d640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7faf6c0d3330 con 0x7faf6c0d1720 2024-09-12T03:19:30.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:30.037+0000 7faf70f6d640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7faf6c0d4000 con 0x7faf6c15eb90 2024-09-12T03:19:30.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:30.037+0000 7faf70f6d640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7faf6c0d34b0 con 0x7faf6c0d5200 2024-09-12T03:19:30.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:30.037+0000 7faf6a59c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faf6c0d1720 0x7faf6c0c9fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:30.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:30.037+0000 7faf6a59c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faf6c0d1720 0x7faf6c0c9fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38376/0 (socket says 172.21.15.27:38376) 2024-09-12T03:19:30.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:30.037+0000 7faf6a59c640 1 -- 172.21.15.27:0/2367442489 learned_addr learned my addr 172.21.15.27:0/2367442489 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:19:30.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:30.037+0000 7faf69d9b640 1 --2- 172.21.15.27:0/2367442489 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7faf6c0d5200 0x7faf6c149c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:30.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:30.037+0000 7faf6a59c640 1 -- 172.21.15.27:0/2367442489 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7faf6c0d5200 msgr2=0x7faf6c149c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:30.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:30.037+0000 7faf6ad9d640 1 --2- 172.21.15.27:0/2367442489 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faf6c15eb90 0x7faf6c0c8340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:30.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:30.037+0000 7faf6a59c640 1 --2- 172.21.15.27:0/2367442489 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7faf6c0d5200 0x7faf6c149c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:30.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:30.037+0000 7faf6a59c640 1 -- 172.21.15.27:0/2367442489 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faf6c15eb90 msgr2=0x7faf6c0c8340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:30.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:30.037+0000 7faf6a59c640 1 --2- 172.21.15.27:0/2367442489 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faf6c15eb90 0x7faf6c0c8340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:30.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:30.037+0000 7faf6a59c640 1 -- 172.21.15.27:0/2367442489 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf6c15a8b0 con 0x7faf6c0d1720 2024-09-12T03:19:30.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:30.037+0000 7faf69d9b640 1 --2- 172.21.15.27:0/2367442489 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7faf6c0d5200 0x7faf6c149c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:19:30.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:30.037+0000 7faf6a59c640 1 --2- 172.21.15.27:0/2367442489 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faf6c0d1720 0x7faf6c0c9fa0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7faf5c007e40 tx=0x7faf5c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:30.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:30.037+0000 7faf5affd640 1 -- 172.21.15.27:0/2367442489 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf5c014620 con 0x7faf6c0d1720 2024-09-12T03:19:30.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:30.037+0000 7faf5affd640 1 -- 172.21.15.27:0/2367442489 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf5c019070 con 0x7faf6c0d1720 2024-09-12T03:19:30.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:30.037+0000 7faf5affd640 1 -- 172.21.15.27:0/2367442489 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf5c014a00 con 0x7faf6c0d1720 2024-09-12T03:19:30.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:30.037+0000 7faf70f6d640 1 -- 172.21.15.27:0/2367442489 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf6c14a140 con 0x7faf6c0d1720 2024-09-12T03:19:30.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:30.037+0000 7faf70f6d640 1 -- 172.21.15.27:0/2367442489 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf6c0c91e0 con 0x7faf6c0d1720 2024-09-12T03:19:30.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:30.038+0000 7faf70f6d640 1 -- 172.21.15.27:0/2367442489 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf6c15f090 con 0x7faf6c0d1720 2024-09-12T03:19:30.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:30.039+0000 7faf5affd640 1 -- 172.21.15.27:0/2367442489 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 9) ==== 90651+0+0 (secure 0 0 0) 0x7faf5c014ba0 con 0x7faf6c0d1720 2024-09-12T03:19:30.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:30.040+0000 7faf5affd640 1 --2- 172.21.15.27:0/2367442489 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7faf4c06bfe0 0x7faf4c06e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:30.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:30.040+0000 7faf5affd640 1 -- 172.21.15.27:0/2367442489 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5709+0+0 (secure 0 0 0) 0x7faf5c08b9d0 con 0x7faf6c0d1720 2024-09-12T03:19:30.043 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:30.040+0000 7faf69d9b640 1 --2- 172.21.15.27:0/2367442489 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7faf4c06bfe0 0x7faf4c06e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:30.043 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:30.040+0000 7faf69d9b640 1 --2- 172.21.15.27:0/2367442489 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7faf4c06bfe0 0x7faf4c06e4a0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7faf6c1f7840 tx=0x7faf60009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:30.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:30.044+0000 7faf5affd640 1 -- 172.21.15.27:0/2367442489 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7faf6c14a140 con 0x7faf6c0d1720 2024-09-12T03:19:30.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:30.217+0000 7faf70f6d640 1 -- 172.21.15.27:0/2367442489 --> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7faf6c0d2850 con 0x7faf4c06bfe0 2024-09-12T03:19:30.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:30.224+0000 7faf5affd640 1 -- 172.21.15.27:0/2367442489 <== mgr.4102 v2:172.21.15.45:6832/3923821219 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+250796 (secure 0 0 0) 0x7faf6c0d2850 con 0x7faf4c06bfe0 2024-09-12T03:19:30.227 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:19:30.228 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2024-09-12T03:19:30.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:30.227+0000 7faf70f6d640 1 -- 172.21.15.27:0/2367442489 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7faf4c06bfe0 msgr2=0x7faf4c06e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:30.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:30.227+0000 7faf70f6d640 1 --2- 172.21.15.27:0/2367442489 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7faf4c06bfe0 0x7faf4c06e4a0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7faf6c1f7840 tx=0x7faf60009040 comp rx=0 tx=0).stop 2024-09-12T03:19:30.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:30.227+0000 7faf70f6d640 1 -- 172.21.15.27:0/2367442489 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faf6c0d1720 msgr2=0x7faf6c0c9fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:30.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:30.227+0000 7faf70f6d640 1 --2- 172.21.15.27:0/2367442489 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faf6c0d1720 0x7faf6c0c9fa0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7faf5c007e40 tx=0x7faf5c00e4b0 comp rx=0 tx=0).stop 2024-09-12T03:19:30.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:30.227+0000 7faf70f6d640 1 -- 172.21.15.27:0/2367442489 shutdown_connections 2024-09-12T03:19:30.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:30.227+0000 7faf70f6d640 1 --2- 172.21.15.27:0/2367442489 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7faf4c06bfe0 0x7faf4c06e4a0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:30.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:30.227+0000 7faf70f6d640 1 --2- 172.21.15.27:0/2367442489 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faf6c15eb90 0x7faf6c0c8340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:30.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:30.227+0000 7faf70f6d640 1 --2- 172.21.15.27:0/2367442489 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7faf6c0d5200 0x7faf6c149c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:30.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:30.227+0000 7faf70f6d640 1 --2- 172.21.15.27:0/2367442489 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faf6c0d1720 0x7faf6c0c9fa0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:30.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:30.227+0000 7faf70f6d640 1 -- 172.21.15.27:0/2367442489 >> 172.21.15.27:0/2367442489 conn(0x7faf6c06dce0 msgr2=0x7faf6c0d1010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:30.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:30.227+0000 7faf70f6d640 1 -- 172.21.15.27:0/2367442489 shutdown_connections 2024-09-12T03:19:30.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:30.227+0000 7faf70f6d640 1 -- 172.21.15.27:0/2367442489 wait complete. 2024-09-12T03:19:30.238 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":144,"stamp":"2024-09-12T03:19:30.095842+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592708,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":197,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":151,"ondisk_log_size":151,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":147,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":247568,"kb_used_data":6356,"kb_used_omap":12,"kb_used_meta":241011,"kb_avail":754727152,"statfs":{"total":773094113280,"available":772840603648,"internally_reserved":0,"allocated":6508544,"data_stored":3353994,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":246795824},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2340,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":38,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001169"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-12T03:19:28.206981+0000","last_change":"2024-09-12T03:19:25.579321+0000","last_active":"2024-09-12T03:19:28.206981+0000","last_peered":"2024-09-12T03:19:28.206981+0000","last_clean":"2024-09-12T03:19:28.206981+0000","last_became_active":"2024-09-12T03:19:25.579050+0000","last_became_peered":"2024-09-12T03:19:25.579050+0000","last_unstale":"2024-09-12T03:19:28.206981+0000","last_undegraded":"2024-09-12T03:19:28.206981+0000","last_fullsized":"2024-09-12T03:19:28.206981+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:00:00.080693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-12T03:19:26.568533+0000","last_change":"2024-09-12T03:19:25.581758+0000","last_active":"2024-09-12T03:19:26.568533+0000","last_peered":"2024-09-12T03:19:26.568533+0000","last_clean":"2024-09-12T03:19:26.568533+0000","last_became_active":"2024-09-12T03:19:25.581483+0000","last_became_peered":"2024-09-12T03:19:25.581483+0000","last_unstale":"2024-09-12T03:19:26.568533+0000","last_undegraded":"2024-09-12T03:19:26.568533+0000","last_fullsized":"2024-09-12T03:19:26.568533+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:43:46.675840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-12T03:19:26.569352+0000","last_change":"2024-09-12T03:19:25.584735+0000","last_active":"2024-09-12T03:19:26.569352+0000","last_peered":"2024-09-12T03:19:26.569352+0000","last_clean":"2024-09-12T03:19:26.569352+0000","last_became_active":"2024-09-12T03:19:25.584357+0000","last_became_peered":"2024-09-12T03:19:25.584357+0000","last_unstale":"2024-09-12T03:19:26.569352+0000","last_undegraded":"2024-09-12T03:19:26.569352+0000","last_fullsized":"2024-09-12T03:19:26.569352+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:59:03.828976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.849111+0000","last_change":"2024-09-12T03:19:24.849111+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.849111+0000","last_undegraded":"2024-09-12T03:19:24.849111+0000","last_fullsized":"2024-09-12T03:19:24.849111+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-12T03:19:28.206806+0000","last_change":"2024-09-12T03:19:25.573294+0000","last_active":"2024-09-12T03:19:28.206806+0000","last_peered":"2024-09-12T03:19:28.206806+0000","last_clean":"2024-09-12T03:19:28.206806+0000","last_became_active":"2024-09-12T03:19:25.572975+0000","last_became_peered":"2024-09-12T03:19:25.572975+0000","last_unstale":"2024-09-12T03:19:28.206806+0000","last_undegraded":"2024-09-12T03:19:28.206806+0000","last_fullsized":"2024-09-12T03:19:28.206806+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:45:37.626405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-12T03:19:27.850378+0000","last_change":"2024-09-12T03:19:25.581045+0000","last_active":"2024-09-12T03:19:27.850378+0000","last_peered":"2024-09-12T03:19:27.850378+0000","last_clean":"2024-09-12T03:19:27.850378+0000","last_became_active":"2024-09-12T03:19:25.580431+0000","last_became_peered":"2024-09-12T03:19:25.580431+0000","last_unstale":"2024-09-12T03:19:27.850378+0000","last_undegraded":"2024-09-12T03:19:27.850378+0000","last_fullsized":"2024-09-12T03:19:27.850378+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:44:04.569606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-12T03:19:26.569886+0000","last_change":"2024-09-12T03:19:25.568621+0000","last_active":"2024-09-12T03:19:26.569886+0000","last_peered":"2024-09-12T03:19:26.569886+0000","last_clean":"2024-09-12T03:19:26.569886+0000","last_became_active":"2024-09-12T03:19:25.568346+0000","last_became_peered":"2024-09-12T03:19:25.568346+0000","last_unstale":"2024-09-12T03:19:26.569886+0000","last_undegraded":"2024-09-12T03:19:26.569886+0000","last_fullsized":"2024-09-12T03:19:26.569886+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:43:46.346281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-12T03:19:26.568190+0000","last_change":"2024-09-12T03:19:25.575264+0000","last_active":"2024-09-12T03:19:26.568190+0000","last_peered":"2024-09-12T03:19:26.568190+0000","last_clean":"2024-09-12T03:19:26.568190+0000","last_became_active":"2024-09-12T03:19:25.574998+0000","last_became_peered":"2024-09-12T03:19:25.574998+0000","last_unstale":"2024-09-12T03:19:26.568190+0000","last_undegraded":"2024-09-12T03:19:26.568190+0000","last_fullsized":"2024-09-12T03:19:26.568190+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:14:06.603640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-12T03:19:26.570405+0000","last_change":"2024-09-12T03:19:25.582104+0000","last_active":"2024-09-12T03:19:26.570405+0000","last_peered":"2024-09-12T03:19:26.570405+0000","last_clean":"2024-09-12T03:19:26.570405+0000","last_became_active":"2024-09-12T03:19:25.581789+0000","last_became_peered":"2024-09-12T03:19:25.581789+0000","last_unstale":"2024-09-12T03:19:26.570405+0000","last_undegraded":"2024-09-12T03:19:26.570405+0000","last_fullsized":"2024-09-12T03:19:26.570405+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:22:24.747123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.848151+0000","last_change":"2024-09-12T03:19:24.848151+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.848151+0000","last_undegraded":"2024-09-12T03:19:24.848151+0000","last_fullsized":"2024-09-12T03:19:24.848151+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.848151+0000","last_change":"2024-09-12T03:19:24.848151+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.848151+0000","last_undegraded":"2024-09-12T03:19:24.848151+0000","last_fullsized":"2024-09-12T03:19:24.848151+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.848357+0000","last_change":"2024-09-12T03:19:24.848357+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.848357+0000","last_undegraded":"2024-09-12T03:19:24.848357+0000","last_fullsized":"2024-09-12T03:19:24.848357+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-12T03:19:27.854297+0000","last_change":"2024-09-12T03:19:25.578981+0000","last_active":"2024-09-12T03:19:27.854297+0000","last_peered":"2024-09-12T03:19:27.854297+0000","last_clean":"2024-09-12T03:19:27.854297+0000","last_became_active":"2024-09-12T03:19:25.578701+0000","last_became_peered":"2024-09-12T03:19:25.578701+0000","last_unstale":"2024-09-12T03:19:27.854297+0000","last_undegraded":"2024-09-12T03:19:27.854297+0000","last_fullsized":"2024-09-12T03:19:27.854297+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:29:11.261684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-12T03:19:26.569608+0000","last_change":"2024-09-12T03:19:25.577974+0000","last_active":"2024-09-12T03:19:26.569608+0000","last_peered":"2024-09-12T03:19:26.569608+0000","last_clean":"2024-09-12T03:19:26.569608+0000","last_became_active":"2024-09-12T03:19:25.577683+0000","last_became_peered":"2024-09-12T03:19:25.577683+0000","last_unstale":"2024-09-12T03:19:26.569608+0000","last_undegraded":"2024-09-12T03:19:26.569608+0000","last_fullsized":"2024-09-12T03:19:26.569608+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:44:45.155856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-12T03:19:26.569328+0000","last_change":"2024-09-12T03:19:25.573261+0000","last_active":"2024-09-12T03:19:26.569328+0000","last_peered":"2024-09-12T03:19:26.569328+0000","last_clean":"2024-09-12T03:19:26.569328+0000","last_became_active":"2024-09-12T03:19:25.572969+0000","last_became_peered":"2024-09-12T03:19:25.572969+0000","last_unstale":"2024-09-12T03:19:26.569328+0000","last_undegraded":"2024-09-12T03:19:26.569328+0000","last_fullsized":"2024-09-12T03:19:26.569328+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:16:49.960138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-12T03:19:27.850337+0000","last_change":"2024-09-12T03:19:25.574807+0000","last_active":"2024-09-12T03:19:27.850337+0000","last_peered":"2024-09-12T03:19:27.850337+0000","last_clean":"2024-09-12T03:19:27.850337+0000","last_became_active":"2024-09-12T03:19:25.574550+0000","last_became_peered":"2024-09-12T03:19:25.574550+0000","last_unstale":"2024-09-12T03:19:27.850337+0000","last_undegraded":"2024-09-12T03:19:27.850337+0000","last_fullsized":"2024-09-12T03:19:27.850337+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:56:51.692997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-12T03:19:26.568652+0000","last_change":"2024-09-12T03:19:25.577565+0000","last_active":"2024-09-12T03:19:26.568652+0000","last_peered":"2024-09-12T03:19:26.568652+0000","last_clean":"2024-09-12T03:19:26.568652+0000","last_became_active":"2024-09-12T03:19:25.577294+0000","last_became_peered":"2024-09-12T03:19:25.577294+0000","last_unstale":"2024-09-12T03:19:26.568652+0000","last_undegraded":"2024-09-12T03:19:26.568652+0000","last_fullsized":"2024-09-12T03:19:26.568652+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:56:05.227809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-12T03:19:26.570511+0000","last_change":"2024-09-12T03:19:25.584433+0000","last_active":"2024-09-12T03:19:26.570511+0000","last_peered":"2024-09-12T03:19:26.570511+0000","last_clean":"2024-09-12T03:19:26.570511+0000","last_became_active":"2024-09-12T03:19:25.583951+0000","last_became_peered":"2024-09-12T03:19:25.583951+0000","last_unstale":"2024-09-12T03:19:26.570511+0000","last_undegraded":"2024-09-12T03:19:26.570511+0000","last_fullsized":"2024-09-12T03:19:26.570511+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:43:55.912919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-12T03:19:28.206763+0000","last_change":"2024-09-12T03:19:25.574096+0000","last_active":"2024-09-12T03:19:28.206763+0000","last_peered":"2024-09-12T03:19:28.206763+0000","last_clean":"2024-09-12T03:19:28.206763+0000","last_became_active":"2024-09-12T03:19:25.573759+0000","last_became_peered":"2024-09-12T03:19:25.573759+0000","last_unstale":"2024-09-12T03:19:28.206763+0000","last_undegraded":"2024-09-12T03:19:28.206763+0000","last_fullsized":"2024-09-12T03:19:28.206763+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:52:01.678549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-12T03:19:26.568866+0000","last_change":"2024-09-12T03:19:25.577212+0000","last_active":"2024-09-12T03:19:26.568866+0000","last_peered":"2024-09-12T03:19:26.568866+0000","last_clean":"2024-09-12T03:19:26.568866+0000","last_became_active":"2024-09-12T03:19:25.576935+0000","last_became_peered":"2024-09-12T03:19:25.576935+0000","last_unstale":"2024-09-12T03:19:26.568866+0000","last_undegraded":"2024-09-12T03:19:26.568866+0000","last_fullsized":"2024-09-12T03:19:26.568866+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:55:32.939225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-12T03:19:26.570425+0000","last_change":"2024-09-12T03:19:25.580816+0000","last_active":"2024-09-12T03:19:26.570425+0000","last_peered":"2024-09-12T03:19:26.570425+0000","last_clean":"2024-09-12T03:19:26.570425+0000","last_became_active":"2024-09-12T03:19:25.580500+0000","last_became_peered":"2024-09-12T03:19:25.580500+0000","last_unstale":"2024-09-12T03:19:26.570425+0000","last_undegraded":"2024-09-12T03:19:26.570425+0000","last_fullsized":"2024-09-12T03:19:26.570425+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:56:51.692997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-12T03:19:26.568892+0000","last_change":"2024-09-12T03:19:25.582734+0000","last_active":"2024-09-12T03:19:26.568892+0000","last_peered":"2024-09-12T03:19:26.568892+0000","last_clean":"2024-09-12T03:19:26.568892+0000","last_became_active":"2024-09-12T03:19:25.582364+0000","last_became_peered":"2024-09-12T03:19:25.582364+0000","last_unstale":"2024-09-12T03:19:26.568892+0000","last_undegraded":"2024-09-12T03:19:26.568892+0000","last_fullsized":"2024-09-12T03:19:26.568892+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:41:18.624900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-12T03:19:26.569424+0000","last_change":"2024-09-12T03:19:25.582851+0000","last_active":"2024-09-12T03:19:26.569424+0000","last_peered":"2024-09-12T03:19:26.569424+0000","last_clean":"2024-09-12T03:19:26.569424+0000","last_became_active":"2024-09-12T03:19:25.582538+0000","last_became_peered":"2024-09-12T03:19:25.582538+0000","last_unstale":"2024-09-12T03:19:26.569424+0000","last_undegraded":"2024-09-12T03:19:26.569424+0000","last_fullsized":"2024-09-12T03:19:26.569424+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:04:06.053023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-12T03:19:27.854182+0000","last_change":"2024-09-12T03:19:25.579664+0000","last_active":"2024-09-12T03:19:27.854182+0000","last_peered":"2024-09-12T03:19:27.854182+0000","last_clean":"2024-09-12T03:19:27.854182+0000","last_became_active":"2024-09-12T03:19:25.579368+0000","last_became_peered":"2024-09-12T03:19:25.579368+0000","last_unstale":"2024-09-12T03:19:27.854182+0000","last_undegraded":"2024-09-12T03:19:27.854182+0000","last_fullsized":"2024-09-12T03:19:27.854182+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:00:18.291876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-12T03:19:28.207086+0000","last_change":"2024-09-12T03:19:25.572942+0000","last_active":"2024-09-12T03:19:28.207086+0000","last_peered":"2024-09-12T03:19:28.207086+0000","last_clean":"2024-09-12T03:19:28.207086+0000","last_became_active":"2024-09-12T03:19:25.572615+0000","last_became_peered":"2024-09-12T03:19:25.572615+0000","last_unstale":"2024-09-12T03:19:28.207086+0000","last_undegraded":"2024-09-12T03:19:28.207086+0000","last_fullsized":"2024-09-12T03:19:28.207086+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:50:58.833474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.848902+0000","last_change":"2024-09-12T03:19:24.848902+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.848902+0000","last_undegraded":"2024-09-12T03:19:24.848902+0000","last_fullsized":"2024-09-12T03:19:24.848902+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-12T03:19:27.850882+0000","last_change":"2024-09-12T03:19:25.576399+0000","last_active":"2024-09-12T03:19:27.850882+0000","last_peered":"2024-09-12T03:19:27.850882+0000","last_clean":"2024-09-12T03:19:27.850882+0000","last_became_active":"2024-09-12T03:19:25.575434+0000","last_became_peered":"2024-09-12T03:19:25.575434+0000","last_unstale":"2024-09-12T03:19:27.850882+0000","last_undegraded":"2024-09-12T03:19:27.850882+0000","last_fullsized":"2024-09-12T03:19:27.850882+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:11:06.694559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-12T03:19:28.206843+0000","last_change":"2024-09-12T03:19:25.574672+0000","last_active":"2024-09-12T03:19:28.206843+0000","last_peered":"2024-09-12T03:19:28.206843+0000","last_clean":"2024-09-12T03:19:28.206843+0000","last_became_active":"2024-09-12T03:19:25.574069+0000","last_became_peered":"2024-09-12T03:19:25.574069+0000","last_unstale":"2024-09-12T03:19:28.206843+0000","last_undegraded":"2024-09-12T03:19:28.206843+0000","last_fullsized":"2024-09-12T03:19:28.206843+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:55:09.070173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-12T03:19:26.568772+0000","last_change":"2024-09-12T03:19:25.572562+0000","last_active":"2024-09-12T03:19:26.568772+0000","last_peered":"2024-09-12T03:19:26.568772+0000","last_clean":"2024-09-12T03:19:26.568772+0000","last_became_active":"2024-09-12T03:19:25.572293+0000","last_became_peered":"2024-09-12T03:19:25.572293+0000","last_unstale":"2024-09-12T03:19:26.568772+0000","last_undegraded":"2024-09-12T03:19:26.568772+0000","last_fullsized":"2024-09-12T03:19:26.568772+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:28:50.615286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-12T03:19:26.569889+0000","last_change":"2024-09-12T03:19:25.581520+0000","last_active":"2024-09-12T03:19:26.569889+0000","last_peered":"2024-09-12T03:19:26.569889+0000","last_clean":"2024-09-12T03:19:26.569889+0000","last_became_active":"2024-09-12T03:19:25.581240+0000","last_became_peered":"2024-09-12T03:19:25.581240+0000","last_unstale":"2024-09-12T03:19:26.569889+0000","last_undegraded":"2024-09-12T03:19:26.569889+0000","last_fullsized":"2024-09-12T03:19:26.569889+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:12:38.654058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-12T03:19:28.206432+0000","last_change":"2024-09-12T03:19:25.572846+0000","last_active":"2024-09-12T03:19:28.206432+0000","last_peered":"2024-09-12T03:19:28.206432+0000","last_clean":"2024-09-12T03:19:28.206432+0000","last_became_active":"2024-09-12T03:19:25.572563+0000","last_became_peered":"2024-09-12T03:19:25.572563+0000","last_unstale":"2024-09-12T03:19:28.206432+0000","last_undegraded":"2024-09-12T03:19:28.206432+0000","last_fullsized":"2024-09-12T03:19:28.206432+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:21:29.241164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-12T03:19:27.850402+0000","last_change":"2024-09-12T03:19:25.581204+0000","last_active":"2024-09-12T03:19:27.850402+0000","last_peered":"2024-09-12T03:19:27.850402+0000","last_clean":"2024-09-12T03:19:27.850402+0000","last_became_active":"2024-09-12T03:19:25.580777+0000","last_became_peered":"2024-09-12T03:19:25.580777+0000","last_unstale":"2024-09-12T03:19:27.850402+0000","last_undegraded":"2024-09-12T03:19:27.850402+0000","last_fullsized":"2024-09-12T03:19:27.850402+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:13:41.572442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-12T03:19:27.851263+0000","last_change":"2024-09-12T03:19:25.580037+0000","last_active":"2024-09-12T03:19:27.851263+0000","last_peered":"2024-09-12T03:19:27.851263+0000","last_clean":"2024-09-12T03:19:27.851263+0000","last_became_active":"2024-09-12T03:19:25.579695+0000","last_became_peered":"2024-09-12T03:19:25.579695+0000","last_unstale":"2024-09-12T03:19:27.851263+0000","last_undegraded":"2024-09-12T03:19:27.851263+0000","last_fullsized":"2024-09-12T03:19:27.851263+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:14:06.603640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"55'4","reported_seq":18,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-12T03:19:27.851275+0000","last_change":"2024-09-12T03:19:24.860694+0000","last_active":"2024-09-12T03:19:27.851275+0000","last_peered":"2024-09-12T03:19:27.851275+0000","last_clean":"2024-09-12T03:19:27.851275+0000","last_became_active":"2024-09-12T03:19:24.860380+0000","last_became_peered":"2024-09-12T03:19:24.860380+0000","last_unstale":"2024-09-12T03:19:27.851275+0000","last_undegraded":"2024-09-12T03:19:27.851275+0000","last_fullsized":"2024-09-12T03:19:27.851275+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_clean_scrub_stamp":"2024-09-12T03:19:23.835989+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:51:36.581086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.848080+0000","last_change":"2024-09-12T03:19:24.848080+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.848080+0000","last_undegraded":"2024-09-12T03:19:24.848080+0000","last_fullsized":"2024-09-12T03:19:24.848080+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"55'1","reported_seq":15,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-12T03:19:26.568647+0000","last_change":"2024-09-12T03:19:24.863015+0000","last_active":"2024-09-12T03:19:26.568647+0000","last_peered":"2024-09-12T03:19:26.568647+0000","last_clean":"2024-09-12T03:19:26.568647+0000","last_became_active":"2024-09-12T03:19:24.862457+0000","last_became_peered":"2024-09-12T03:19:24.862457+0000","last_unstale":"2024-09-12T03:19:26.568647+0000","last_undegraded":"2024-09-12T03:19:26.568647+0000","last_fullsized":"2024-09-12T03:19:26.568647+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_clean_scrub_stamp":"2024-09-12T03:19:23.835989+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:17:40.881373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-12T03:19:27.850653+0000","last_change":"2024-09-12T03:19:25.581589+0000","last_active":"2024-09-12T03:19:27.850653+0000","last_peered":"2024-09-12T03:19:27.850653+0000","last_clean":"2024-09-12T03:19:27.850653+0000","last_became_active":"2024-09-12T03:19:25.581294+0000","last_became_peered":"2024-09-12T03:19:25.581294+0000","last_unstale":"2024-09-12T03:19:27.850653+0000","last_undegraded":"2024-09-12T03:19:27.850653+0000","last_fullsized":"2024-09-12T03:19:27.850653+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:37:42.235320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"55'2","reported_seq":16,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-12T03:19:27.851078+0000","last_change":"2024-09-12T03:19:24.862226+0000","last_active":"2024-09-12T03:19:27.851078+0000","last_peered":"2024-09-12T03:19:27.851078+0000","last_clean":"2024-09-12T03:19:27.851078+0000","last_became_active":"2024-09-12T03:19:24.861956+0000","last_became_peered":"2024-09-12T03:19:24.861956+0000","last_unstale":"2024-09-12T03:19:27.851078+0000","last_undegraded":"2024-09-12T03:19:27.851078+0000","last_fullsized":"2024-09-12T03:19:27.851078+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_clean_scrub_stamp":"2024-09-12T03:19:23.835989+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:16:02.571700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-12T03:19:27.850978+0000","last_change":"2024-09-12T03:19:25.576177+0000","last_active":"2024-09-12T03:19:27.850978+0000","last_peered":"2024-09-12T03:19:27.850978+0000","last_clean":"2024-09-12T03:19:27.850978+0000","last_became_active":"2024-09-12T03:19:25.575854+0000","last_became_peered":"2024-09-12T03:19:25.575854+0000","last_unstale":"2024-09-12T03:19:27.850978+0000","last_undegraded":"2024-09-12T03:19:27.850978+0000","last_fullsized":"2024-09-12T03:19:27.850978+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:03:02.553134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T03:19:24.861878+0000","last_change":"2024-09-12T03:19:24.859901+0000","last_active":"2024-09-12T03:19:24.861878+0000","last_peered":"2024-09-12T03:19:24.861878+0000","last_clean":"2024-09-12T03:19:24.861878+0000","last_became_active":"2024-09-12T03:19:24.859637+0000","last_became_peered":"2024-09-12T03:19:24.859637+0000","last_unstale":"2024-09-12T03:19:24.861878+0000","last_undegraded":"2024-09-12T03:19:24.861878+0000","last_fullsized":"2024-09-12T03:19:24.861878+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_clean_scrub_stamp":"2024-09-12T03:19:23.835989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:48:14.537533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-12T03:19:26.568261+0000","last_change":"2024-09-12T03:19:25.576779+0000","last_active":"2024-09-12T03:19:26.568261+0000","last_peered":"2024-09-12T03:19:26.568261+0000","last_clean":"2024-09-12T03:19:26.568261+0000","last_became_active":"2024-09-12T03:19:25.576482+0000","last_became_peered":"2024-09-12T03:19:25.576482+0000","last_unstale":"2024-09-12T03:19:26.568261+0000","last_undegraded":"2024-09-12T03:19:26.568261+0000","last_fullsized":"2024-09-12T03:19:26.568261+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:36:00.532243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"55'1","reported_seq":15,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-12T03:19:26.570081+0000","last_change":"2024-09-12T03:19:24.856981+0000","last_active":"2024-09-12T03:19:26.570081+0000","last_peered":"2024-09-12T03:19:26.570081+0000","last_clean":"2024-09-12T03:19:26.570081+0000","last_became_active":"2024-09-12T03:19:24.856607+0000","last_became_peered":"2024-09-12T03:19:24.856607+0000","last_unstale":"2024-09-12T03:19:26.570081+0000","last_undegraded":"2024-09-12T03:19:26.570081+0000","last_fullsized":"2024-09-12T03:19:26.570081+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_clean_scrub_stamp":"2024-09-12T03:19:23.835989+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:32:58.661071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-12T03:19:26.568522+0000","last_change":"2024-09-12T03:19:25.577980+0000","last_active":"2024-09-12T03:19:26.568522+0000","last_peered":"2024-09-12T03:19:26.568522+0000","last_clean":"2024-09-12T03:19:26.568522+0000","last_became_active":"2024-09-12T03:19:25.577717+0000","last_became_peered":"2024-09-12T03:19:25.577717+0000","last_unstale":"2024-09-12T03:19:26.568522+0000","last_undegraded":"2024-09-12T03:19:26.568522+0000","last_fullsized":"2024-09-12T03:19:26.568522+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:25:09.470946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"55'1","reported_seq":15,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-12T03:19:27.850627+0000","last_change":"2024-09-12T03:19:24.861305+0000","last_active":"2024-09-12T03:19:27.850627+0000","last_peered":"2024-09-12T03:19:27.850627+0000","last_clean":"2024-09-12T03:19:27.850627+0000","last_became_active":"2024-09-12T03:19:24.861021+0000","last_became_peered":"2024-09-12T03:19:24.861021+0000","last_unstale":"2024-09-12T03:19:27.850627+0000","last_undegraded":"2024-09-12T03:19:27.850627+0000","last_fullsized":"2024-09-12T03:19:27.850627+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_clean_scrub_stamp":"2024-09-12T03:19:23.835989+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:04:27.775435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-12T03:19:27.854225+0000","last_change":"2024-09-12T03:19:25.577969+0000","last_active":"2024-09-12T03:19:27.854225+0000","last_peered":"2024-09-12T03:19:27.854225+0000","last_clean":"2024-09-12T03:19:27.854225+0000","last_became_active":"2024-09-12T03:19:25.577648+0000","last_became_peered":"2024-09-12T03:19:25.577648+0000","last_unstale":"2024-09-12T03:19:27.854225+0000","last_undegraded":"2024-09-12T03:19:27.854225+0000","last_fullsized":"2024-09-12T03:19:27.854225+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:00:46.783593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T03:19:24.854193+0000","last_change":"2024-09-12T03:19:24.850360+0000","last_active":"2024-09-12T03:19:24.854193+0000","last_peered":"2024-09-12T03:19:24.854193+0000","last_clean":"2024-09-12T03:19:24.854193+0000","last_became_active":"2024-09-12T03:19:24.850027+0000","last_became_peered":"2024-09-12T03:19:24.850027+0000","last_unstale":"2024-09-12T03:19:24.854193+0000","last_undegraded":"2024-09-12T03:19:24.854193+0000","last_fullsized":"2024-09-12T03:19:24.854193+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_clean_scrub_stamp":"2024-09-12T03:19:23.835989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:35:50.136024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-12T03:19:26.568402+0000","last_change":"2024-09-12T03:19:25.578243+0000","last_active":"2024-09-12T03:19:26.568402+0000","last_peered":"2024-09-12T03:19:26.568402+0000","last_clean":"2024-09-12T03:19:26.568402+0000","last_became_active":"2024-09-12T03:19:25.577962+0000","last_became_peered":"2024-09-12T03:19:25.577962+0000","last_unstale":"2024-09-12T03:19:26.568402+0000","last_undegraded":"2024-09-12T03:19:26.568402+0000","last_fullsized":"2024-09-12T03:19:26.568402+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:11:33.930744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":14,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-12T03:19:26.569127+0000","last_change":"2024-09-12T03:19:24.863770+0000","last_active":"2024-09-12T03:19:26.569127+0000","last_peered":"2024-09-12T03:19:26.569127+0000","last_clean":"2024-09-12T03:19:26.569127+0000","last_became_active":"2024-09-12T03:19:24.863471+0000","last_became_peered":"2024-09-12T03:19:24.863471+0000","last_unstale":"2024-09-12T03:19:26.569127+0000","last_undegraded":"2024-09-12T03:19:26.569127+0000","last_fullsized":"2024-09-12T03:19:26.569127+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_clean_scrub_stamp":"2024-09-12T03:19:23.835989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:31:29.254101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-12T03:19:26.570427+0000","last_change":"2024-09-12T03:19:25.574294+0000","last_active":"2024-09-12T03:19:26.570427+0000","last_peered":"2024-09-12T03:19:26.570427+0000","last_clean":"2024-09-12T03:19:26.570427+0000","last_became_active":"2024-09-12T03:19:25.573875+0000","last_became_peered":"2024-09-12T03:19:25.573875+0000","last_unstale":"2024-09-12T03:19:26.570427+0000","last_undegraded":"2024-09-12T03:19:26.570427+0000","last_fullsized":"2024-09-12T03:19:26.570427+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:52:22.817110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":14,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-12T03:19:26.570255+0000","last_change":"2024-09-12T03:19:24.870526+0000","last_active":"2024-09-12T03:19:26.570255+0000","last_peered":"2024-09-12T03:19:26.570255+0000","last_clean":"2024-09-12T03:19:26.570255+0000","last_became_active":"2024-09-12T03:19:24.870226+0000","last_became_peered":"2024-09-12T03:19:24.870226+0000","last_unstale":"2024-09-12T03:19:26.570255+0000","last_undegraded":"2024-09-12T03:19:26.570255+0000","last_fullsized":"2024-09-12T03:19:26.570255+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_clean_scrub_stamp":"2024-09-12T03:19:23.835989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:52:21.815077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-12T03:19:26.570279+0000","last_change":"2024-09-12T03:19:25.586558+0000","last_active":"2024-09-12T03:19:26.570279+0000","last_peered":"2024-09-12T03:19:26.570279+0000","last_clean":"2024-09-12T03:19:26.570279+0000","last_became_active":"2024-09-12T03:19:25.586154+0000","last_became_peered":"2024-09-12T03:19:25.586154+0000","last_unstale":"2024-09-12T03:19:26.570279+0000","last_undegraded":"2024-09-12T03:19:26.570279+0000","last_fullsized":"2024-09-12T03:19:26.570279+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:38:29.897420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":14,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-12T03:19:26.568760+0000","last_change":"2024-09-12T03:19:24.862733+0000","last_active":"2024-09-12T03:19:26.568760+0000","last_peered":"2024-09-12T03:19:26.568760+0000","last_clean":"2024-09-12T03:19:26.568760+0000","last_became_active":"2024-09-12T03:19:24.862164+0000","last_became_peered":"2024-09-12T03:19:24.862164+0000","last_unstale":"2024-09-12T03:19:26.568760+0000","last_undegraded":"2024-09-12T03:19:26.568760+0000","last_fullsized":"2024-09-12T03:19:26.568760+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_clean_scrub_stamp":"2024-09-12T03:19:23.835989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:54:18.627329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.848059+0000","last_change":"2024-09-12T03:19:24.848059+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.848059+0000","last_undegraded":"2024-09-12T03:19:24.848059+0000","last_fullsized":"2024-09-12T03:19:24.848059+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"55'3","reported_seq":17,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-12T03:19:26.570389+0000","last_change":"2024-09-12T03:19:24.859732+0000","last_active":"2024-09-12T03:19:26.570389+0000","last_peered":"2024-09-12T03:19:26.570389+0000","last_clean":"2024-09-12T03:19:26.570389+0000","last_became_active":"2024-09-12T03:19:24.859452+0000","last_became_peered":"2024-09-12T03:19:24.859452+0000","last_unstale":"2024-09-12T03:19:26.570389+0000","last_undegraded":"2024-09-12T03:19:26.570389+0000","last_fullsized":"2024-09-12T03:19:26.570389+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_clean_scrub_stamp":"2024-09-12T03:19:23.835989+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:56:28.082698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-12T03:19:26.569344+0000","last_change":"2024-09-12T03:19:25.586524+0000","last_active":"2024-09-12T03:19:26.569344+0000","last_peered":"2024-09-12T03:19:26.569344+0000","last_clean":"2024-09-12T03:19:26.569344+0000","last_became_active":"2024-09-12T03:19:25.586168+0000","last_became_peered":"2024-09-12T03:19:25.586168+0000","last_unstale":"2024-09-12T03:19:26.569344+0000","last_undegraded":"2024-09-12T03:19:26.569344+0000","last_fullsized":"2024-09-12T03:19:26.569344+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:22:54.210265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"55'6","reported_seq":20,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-12T03:19:26.570462+0000","last_change":"2024-09-12T03:19:24.861967+0000","last_active":"2024-09-12T03:19:26.570462+0000","last_peered":"2024-09-12T03:19:26.570462+0000","last_clean":"2024-09-12T03:19:26.570462+0000","last_became_active":"2024-09-12T03:19:24.861645+0000","last_became_peered":"2024-09-12T03:19:24.861645+0000","last_unstale":"2024-09-12T03:19:26.570462+0000","last_undegraded":"2024-09-12T03:19:26.570462+0000","last_fullsized":"2024-09-12T03:19:26.570462+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_clean_scrub_stamp":"2024-09-12T03:19:23.835989+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:29:10.259651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.847408+0000","last_change":"2024-09-12T03:19:24.847408+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.847408+0000","last_undegraded":"2024-09-12T03:19:24.847408+0000","last_fullsized":"2024-09-12T03:19:24.847408+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"55'2","reported_seq":16,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-12T03:19:26.568637+0000","last_change":"2024-09-12T03:19:24.863067+0000","last_active":"2024-09-12T03:19:26.568637+0000","last_peered":"2024-09-12T03:19:26.568637+0000","last_clean":"2024-09-12T03:19:26.568637+0000","last_became_active":"2024-09-12T03:19:24.862471+0000","last_became_peered":"2024-09-12T03:19:24.862471+0000","last_unstale":"2024-09-12T03:19:26.568637+0000","last_undegraded":"2024-09-12T03:19:26.568637+0000","last_fullsized":"2024-09-12T03:19:26.568637+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_clean_scrub_stamp":"2024-09-12T03:19:23.835989+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:22:53.208233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-12T03:19:26.570597+0000","last_change":"2024-09-12T03:19:25.576709+0000","last_active":"2024-09-12T03:19:26.570597+0000","last_peered":"2024-09-12T03:19:26.570597+0000","last_clean":"2024-09-12T03:19:26.570597+0000","last_became_active":"2024-09-12T03:19:25.576410+0000","last_became_peered":"2024-09-12T03:19:25.576410+0000","last_unstale":"2024-09-12T03:19:26.570597+0000","last_undegraded":"2024-09-12T03:19:26.570597+0000","last_fullsized":"2024-09-12T03:19:26.570597+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:44:04.569606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":14,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-12T03:19:27.853992+0000","last_change":"2024-09-12T03:19:24.859981+0000","last_active":"2024-09-12T03:19:27.853992+0000","last_peered":"2024-09-12T03:19:27.853992+0000","last_clean":"2024-09-12T03:19:27.853992+0000","last_became_active":"2024-09-12T03:19:24.859679+0000","last_became_peered":"2024-09-12T03:19:24.859679+0000","last_unstale":"2024-09-12T03:19:27.853992+0000","last_undegraded":"2024-09-12T03:19:27.853992+0000","last_fullsized":"2024-09-12T03:19:27.853992+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_clean_scrub_stamp":"2024-09-12T03:19:23.835989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:32:15.030338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-12T03:19:26.569091+0000","last_change":"2024-09-12T03:19:25.579614+0000","last_active":"2024-09-12T03:19:26.569091+0000","last_peered":"2024-09-12T03:19:26.569091+0000","last_clean":"2024-09-12T03:19:26.569091+0000","last_became_active":"2024-09-12T03:19:25.579280+0000","last_became_peered":"2024-09-12T03:19:25.579280+0000","last_unstale":"2024-09-12T03:19:26.569091+0000","last_undegraded":"2024-09-12T03:19:26.569091+0000","last_fullsized":"2024-09-12T03:19:26.569091+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:00:43.742359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T03:19:24.857304+0000","last_change":"2024-09-12T03:19:24.855795+0000","last_active":"2024-09-12T03:19:24.857304+0000","last_peered":"2024-09-12T03:19:24.857304+0000","last_clean":"2024-09-12T03:19:24.857304+0000","last_became_active":"2024-09-12T03:19:24.855377+0000","last_became_peered":"2024-09-12T03:19:24.855377+0000","last_unstale":"2024-09-12T03:19:24.857304+0000","last_undegraded":"2024-09-12T03:19:24.857304+0000","last_fullsized":"2024-09-12T03:19:24.857304+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_clean_scrub_stamp":"2024-09-12T03:19:23.835989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:50:07.794274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-12T03:19:26.569606+0000","last_change":"2024-09-12T03:19:25.583346+0000","last_active":"2024-09-12T03:19:26.569606+0000","last_peered":"2024-09-12T03:19:26.569606+0000","last_clean":"2024-09-12T03:19:26.569606+0000","last_became_active":"2024-09-12T03:19:25.583046+0000","last_became_peered":"2024-09-12T03:19:25.583046+0000","last_unstale":"2024-09-12T03:19:26.569606+0000","last_undegraded":"2024-09-12T03:19:26.569606+0000","last_fullsized":"2024-09-12T03:19:26.569606+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:34:59.628397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"55'8","reported_seq":22,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-12T03:19:26.570683+0000","last_change":"2024-09-12T03:19:24.859883+0000","last_active":"2024-09-12T03:19:26.570683+0000","last_peered":"2024-09-12T03:19:26.570683+0000","last_clean":"2024-09-12T03:19:26.570683+0000","last_became_active":"2024-09-12T03:19:24.859566+0000","last_became_peered":"2024-09-12T03:19:24.859566+0000","last_unstale":"2024-09-12T03:19:26.570683+0000","last_undegraded":"2024-09-12T03:19:26.570683+0000","last_fullsized":"2024-09-12T03:19:26.570683+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_clean_scrub_stamp":"2024-09-12T03:19:23.835989+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:22:23.745090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-12T03:19:26.570675+0000","last_change":"2024-09-12T03:19:25.584759+0000","last_active":"2024-09-12T03:19:26.570675+0000","last_peered":"2024-09-12T03:19:26.570675+0000","last_clean":"2024-09-12T03:19:26.570675+0000","last_became_active":"2024-09-12T03:19:25.584378+0000","last_became_peered":"2024-09-12T03:19:25.584378+0000","last_unstale":"2024-09-12T03:19:26.570675+0000","last_undegraded":"2024-09-12T03:19:26.570675+0000","last_fullsized":"2024-09-12T03:19:26.570675+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:13:41.572442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"55'1","reported_seq":15,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-12T03:19:26.568792+0000","last_change":"2024-09-12T03:19:24.861891+0000","last_active":"2024-09-12T03:19:26.568792+0000","last_peered":"2024-09-12T03:19:26.568792+0000","last_clean":"2024-09-12T03:19:26.568792+0000","last_became_active":"2024-09-12T03:19:24.861524+0000","last_became_peered":"2024-09-12T03:19:24.861524+0000","last_unstale":"2024-09-12T03:19:26.568792+0000","last_undegraded":"2024-09-12T03:19:26.568792+0000","last_fullsized":"2024-09-12T03:19:26.568792+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_clean_scrub_stamp":"2024-09-12T03:19:23.835989+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:41:59.754398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.848416+0000","last_change":"2024-09-12T03:19:24.848416+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.848416+0000","last_undegraded":"2024-09-12T03:19:24.848416+0000","last_fullsized":"2024-09-12T03:19:24.848416+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"55'2","reported_seq":16,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-12T03:19:27.851190+0000","last_change":"2024-09-12T03:19:24.865925+0000","last_active":"2024-09-12T03:19:27.851190+0000","last_peered":"2024-09-12T03:19:27.851190+0000","last_clean":"2024-09-12T03:19:27.851190+0000","last_became_active":"2024-09-12T03:19:24.865689+0000","last_became_peered":"2024-09-12T03:19:24.865689+0000","last_unstale":"2024-09-12T03:19:27.851190+0000","last_undegraded":"2024-09-12T03:19:27.851190+0000","last_fullsized":"2024-09-12T03:19:27.851190+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_clean_scrub_stamp":"2024-09-12T03:19:23.835989+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:04:37.233100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-12T03:19:26.568939+0000","last_change":"2024-09-12T03:19:25.571563+0000","last_active":"2024-09-12T03:19:26.568939+0000","last_peered":"2024-09-12T03:19:26.568939+0000","last_clean":"2024-09-12T03:19:26.568939+0000","last_became_active":"2024-09-12T03:19:25.571285+0000","last_became_peered":"2024-09-12T03:19:25.571285+0000","last_unstale":"2024-09-12T03:19:26.568939+0000","last_undegraded":"2024-09-12T03:19:26.568939+0000","last_fullsized":"2024-09-12T03:19:26.568939+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:51:26.533695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T03:19:24.857771+0000","last_change":"2024-09-12T03:19:24.855690+0000","last_active":"2024-09-12T03:19:24.857771+0000","last_peered":"2024-09-12T03:19:24.857771+0000","last_clean":"2024-09-12T03:19:24.857771+0000","last_became_active":"2024-09-12T03:19:24.855284+0000","last_became_peered":"2024-09-12T03:19:24.855284+0000","last_unstale":"2024-09-12T03:19:24.857771+0000","last_undegraded":"2024-09-12T03:19:24.857771+0000","last_fullsized":"2024-09-12T03:19:24.857771+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_clean_scrub_stamp":"2024-09-12T03:19:23.835989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:38:35.821379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-12T03:19:26.569989+0000","last_change":"2024-09-12T03:19:25.586655+0000","last_active":"2024-09-12T03:19:26.569989+0000","last_peered":"2024-09-12T03:19:26.569989+0000","last_clean":"2024-09-12T03:19:26.569989+0000","last_became_active":"2024-09-12T03:19:25.586242+0000","last_became_peered":"2024-09-12T03:19:25.586242+0000","last_unstale":"2024-09-12T03:19:26.569989+0000","last_undegraded":"2024-09-12T03:19:26.569989+0000","last_fullsized":"2024-09-12T03:19:26.569989+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:16:22.378881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":14,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-12T03:19:27.850794+0000","last_change":"2024-09-12T03:19:24.864872+0000","last_active":"2024-09-12T03:19:27.850794+0000","last_peered":"2024-09-12T03:19:27.850794+0000","last_clean":"2024-09-12T03:19:27.850794+0000","last_became_active":"2024-09-12T03:19:24.864506+0000","last_became_peered":"2024-09-12T03:19:24.864506+0000","last_unstale":"2024-09-12T03:19:27.850794+0000","last_undegraded":"2024-09-12T03:19:27.850794+0000","last_fullsized":"2024-09-12T03:19:27.850794+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_clean_scrub_stamp":"2024-09-12T03:19:23.835989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:50:19.592853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.846915+0000","last_change":"2024-09-12T03:19:24.846915+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.846915+0000","last_undegraded":"2024-09-12T03:19:24.846915+0000","last_fullsized":"2024-09-12T03:19:24.846915+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"55'2","reported_seq":16,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-12T03:19:26.568792+0000","last_change":"2024-09-12T03:19:24.862743+0000","last_active":"2024-09-12T03:19:26.568792+0000","last_peered":"2024-09-12T03:19:26.568792+0000","last_clean":"2024-09-12T03:19:26.568792+0000","last_became_active":"2024-09-12T03:19:24.862186+0000","last_became_peered":"2024-09-12T03:19:24.862186+0000","last_unstale":"2024-09-12T03:19:26.568792+0000","last_undegraded":"2024-09-12T03:19:26.568792+0000","last_fullsized":"2024-09-12T03:19:26.568792+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_clean_scrub_stamp":"2024-09-12T03:19:23.835989+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:59:02.826944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-12T03:19:27.850728+0000","last_change":"2024-09-12T03:19:25.575390+0000","last_active":"2024-09-12T03:19:27.850728+0000","last_peered":"2024-09-12T03:19:27.850728+0000","last_clean":"2024-09-12T03:19:27.850728+0000","last_became_active":"2024-09-12T03:19:25.574143+0000","last_became_peered":"2024-09-12T03:19:25.574143+0000","last_unstale":"2024-09-12T03:19:27.850728+0000","last_undegraded":"2024-09-12T03:19:27.850728+0000","last_fullsized":"2024-09-12T03:19:27.850728+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:25:23.473943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T03:19:24.860844+0000","last_change":"2024-09-12T03:19:24.857052+0000","last_active":"2024-09-12T03:19:24.860844+0000","last_peered":"2024-09-12T03:19:24.860844+0000","last_clean":"2024-09-12T03:19:24.860844+0000","last_became_active":"2024-09-12T03:19:24.856771+0000","last_became_peered":"2024-09-12T03:19:24.856771+0000","last_unstale":"2024-09-12T03:19:24.860844+0000","last_undegraded":"2024-09-12T03:19:24.860844+0000","last_fullsized":"2024-09-12T03:19:24.860844+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_clean_scrub_stamp":"2024-09-12T03:19:23.835989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:35:47.557902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-12T03:19:27.850360+0000","last_change":"2024-09-12T03:19:25.578776+0000","last_active":"2024-09-12T03:19:27.850360+0000","last_peered":"2024-09-12T03:19:27.850360+0000","last_clean":"2024-09-12T03:19:27.850360+0000","last_became_active":"2024-09-12T03:19:25.578465+0000","last_became_peered":"2024-09-12T03:19:25.578465+0000","last_unstale":"2024-09-12T03:19:27.850360+0000","last_undegraded":"2024-09-12T03:19:27.850360+0000","last_fullsized":"2024-09-12T03:19:27.850360+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:43:55.912919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T03:19:24.864938+0000","last_change":"2024-09-12T03:19:24.864514+0000","last_active":"2024-09-12T03:19:24.864938+0000","last_peered":"2024-09-12T03:19:24.864938+0000","last_clean":"2024-09-12T03:19:24.864938+0000","last_became_active":"2024-09-12T03:19:24.864206+0000","last_became_peered":"2024-09-12T03:19:24.864206+0000","last_unstale":"2024-09-12T03:19:24.864938+0000","last_undegraded":"2024-09-12T03:19:24.864938+0000","last_fullsized":"2024-09-12T03:19:24.864938+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_clean_scrub_stamp":"2024-09-12T03:19:23.835989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:24:19.872805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.847841+0000","last_change":"2024-09-12T03:19:24.847841+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.847841+0000","last_undegraded":"2024-09-12T03:19:24.847841+0000","last_fullsized":"2024-09-12T03:19:24.847841+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T03:19:24.866519+0000","last_change":"2024-09-12T03:19:24.863777+0000","last_active":"2024-09-12T03:19:24.866519+0000","last_peered":"2024-09-12T03:19:24.866519+0000","last_clean":"2024-09-12T03:19:24.866519+0000","last_became_active":"2024-09-12T03:19:24.863506+0000","last_became_peered":"2024-09-12T03:19:24.863506+0000","last_unstale":"2024-09-12T03:19:24.866519+0000","last_undegraded":"2024-09-12T03:19:24.866519+0000","last_fullsized":"2024-09-12T03:19:24.866519+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_clean_scrub_stamp":"2024-09-12T03:19:23.835989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:53:10.464555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.847949+0000","last_change":"2024-09-12T03:19:24.847949+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.847949+0000","last_undegraded":"2024-09-12T03:19:24.847949+0000","last_fullsized":"2024-09-12T03:19:24.847949+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"56'1","reported_seq":15,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-12T03:19:27.851354+0000","last_change":"2024-09-12T03:19:24.865383+0000","last_active":"2024-09-12T03:19:27.851354+0000","last_peered":"2024-09-12T03:19:27.851354+0000","last_clean":"2024-09-12T03:19:27.851354+0000","last_became_active":"2024-09-12T03:19:24.865031+0000","last_became_peered":"2024-09-12T03:19:24.865031+0000","last_unstale":"2024-09-12T03:19:27.851354+0000","last_undegraded":"2024-09-12T03:19:27.851354+0000","last_fullsized":"2024-09-12T03:19:27.851354+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_clean_scrub_stamp":"2024-09-12T03:19:23.835989+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:25:17.547373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.847544+0000","last_change":"2024-09-12T03:19:24.847544+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.847544+0000","last_undegraded":"2024-09-12T03:19:24.847544+0000","last_fullsized":"2024-09-12T03:19:24.847544+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":14,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-12T03:19:27.854065+0000","last_change":"2024-09-12T03:19:24.857501+0000","last_active":"2024-09-12T03:19:27.854065+0000","last_peered":"2024-09-12T03:19:27.854065+0000","last_clean":"2024-09-12T03:19:27.854065+0000","last_became_active":"2024-09-12T03:19:24.857129+0000","last_became_peered":"2024-09-12T03:19:24.857129+0000","last_unstale":"2024-09-12T03:19:27.854065+0000","last_undegraded":"2024-09-12T03:19:27.854065+0000","last_fullsized":"2024-09-12T03:19:27.854065+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_clean_scrub_stamp":"2024-09-12T03:19:23.835989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:33:41.476236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-12T03:19:27.851032+0000","last_change":"2024-09-12T03:19:25.572553+0000","last_active":"2024-09-12T03:19:27.851032+0000","last_peered":"2024-09-12T03:19:27.851032+0000","last_clean":"2024-09-12T03:19:27.851032+0000","last_became_active":"2024-09-12T03:19:25.572213+0000","last_became_peered":"2024-09-12T03:19:25.572213+0000","last_unstale":"2024-09-12T03:19:27.851032+0000","last_undegraded":"2024-09-12T03:19:27.851032+0000","last_fullsized":"2024-09-12T03:19:27.851032+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:49:44.982182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T03:19:24.856019+0000","last_change":"2024-09-12T03:19:24.852831+0000","last_active":"2024-09-12T03:19:24.856019+0000","last_peered":"2024-09-12T03:19:24.856019+0000","last_clean":"2024-09-12T03:19:24.856019+0000","last_became_active":"2024-09-12T03:19:24.851427+0000","last_became_peered":"2024-09-12T03:19:24.851427+0000","last_unstale":"2024-09-12T03:19:24.856019+0000","last_undegraded":"2024-09-12T03:19:24.856019+0000","last_fullsized":"2024-09-12T03:19:24.856019+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_clean_scrub_stamp":"2024-09-12T03:19:23.835989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:39:10.957546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-12T03:19:26.568449+0000","last_change":"2024-09-12T03:19:25.576513+0000","last_active":"2024-09-12T03:19:26.568449+0000","last_peered":"2024-09-12T03:19:26.568449+0000","last_clean":"2024-09-12T03:19:26.568449+0000","last_became_active":"2024-09-12T03:19:25.576249+0000","last_became_peered":"2024-09-12T03:19:25.576249+0000","last_unstale":"2024-09-12T03:19:26.568449+0000","last_undegraded":"2024-09-12T03:19:26.568449+0000","last_fullsized":"2024-09-12T03:19:26.568449+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:53:00.492150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T03:19:24.855106+0000","last_change":"2024-09-12T03:19:24.849318+0000","last_active":"2024-09-12T03:19:24.855106+0000","last_peered":"2024-09-12T03:19:24.855106+0000","last_clean":"2024-09-12T03:19:24.855106+0000","last_became_active":"2024-09-12T03:19:24.848981+0000","last_became_peered":"2024-09-12T03:19:24.848981+0000","last_unstale":"2024-09-12T03:19:24.855106+0000","last_undegraded":"2024-09-12T03:19:24.855106+0000","last_fullsized":"2024-09-12T03:19:24.855106+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_clean_scrub_stamp":"2024-09-12T03:19:23.835989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:07:42.042664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-12T03:19:26.570846+0000","last_change":"2024-09-12T03:19:25.583374+0000","last_active":"2024-09-12T03:19:26.570846+0000","last_peered":"2024-09-12T03:19:26.570846+0000","last_clean":"2024-09-12T03:19:26.570846+0000","last_became_active":"2024-09-12T03:19:25.583066+0000","last_became_peered":"2024-09-12T03:19:25.583066+0000","last_unstale":"2024-09-12T03:19:26.570846+0000","last_undegraded":"2024-09-12T03:19:26.570846+0000","last_fullsized":"2024-09-12T03:19:26.570846+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:37:42.235320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":14,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-12T03:19:26.569729+0000","last_change":"2024-09-12T03:19:24.861998+0000","last_active":"2024-09-12T03:19:26.569729+0000","last_peered":"2024-09-12T03:19:26.569729+0000","last_clean":"2024-09-12T03:19:26.569729+0000","last_became_active":"2024-09-12T03:19:24.861694+0000","last_became_peered":"2024-09-12T03:19:24.861694+0000","last_unstale":"2024-09-12T03:19:26.569729+0000","last_undegraded":"2024-09-12T03:19:26.569729+0000","last_fullsized":"2024-09-12T03:19:26.569729+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_clean_scrub_stamp":"2024-09-12T03:19:23.835989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:31:40.161867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-09-12T03:19:24.846974+0000","last_change":"2024-09-12T03:19:24.846974+0000","last_active":"2024-09-12T03:19:24.838022+0000","last_peered":"2024-09-12T03:19:24.838022+0000","last_clean":"2024-09-12T03:19:24.838022+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:24.846974+0000","last_undegraded":"2024-09-12T03:19:24.846974+0000","last_fullsized":"2024-09-12T03:19:24.846974+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T03:19:24.863848+0000","last_change":"2024-09-12T03:19:24.860398+0000","last_active":"2024-09-12T03:19:24.863848+0000","last_peered":"2024-09-12T03:19:24.863848+0000","last_clean":"2024-09-12T03:19:24.863848+0000","last_became_active":"2024-09-12T03:19:24.860093+0000","last_became_peered":"2024-09-12T03:19:24.860093+0000","last_unstale":"2024-09-12T03:19:24.863848+0000","last_undegraded":"2024-09-12T03:19:24.863848+0000","last_fullsized":"2024-09-12T03:19:24.863848+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_clean_scrub_stamp":"2024-09-12T03:19:23.835989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:40:45.875415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-12T03:19:26.568565+0000","last_change":"2024-09-12T03:19:25.577033+0000","last_active":"2024-09-12T03:19:26.568565+0000","last_peered":"2024-09-12T03:19:26.568565+0000","last_clean":"2024-09-12T03:19:26.568565+0000","last_became_active":"2024-09-12T03:19:25.575361+0000","last_became_peered":"2024-09-12T03:19:25.575361+0000","last_unstale":"2024-09-12T03:19:26.568565+0000","last_undegraded":"2024-09-12T03:19:26.568565+0000","last_fullsized":"2024-09-12T03:19:26.568565+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:49:39.227841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":14,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-12T03:19:26.568443+0000","last_change":"2024-09-12T03:19:24.858756+0000","last_active":"2024-09-12T03:19:26.568443+0000","last_peered":"2024-09-12T03:19:26.568443+0000","last_clean":"2024-09-12T03:19:26.568443+0000","last_became_active":"2024-09-12T03:19:24.858474+0000","last_became_peered":"2024-09-12T03:19:24.858474+0000","last_unstale":"2024-09-12T03:19:26.568443+0000","last_undegraded":"2024-09-12T03:19:26.568443+0000","last_fullsized":"2024-09-12T03:19:26.568443+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_clean_scrub_stamp":"2024-09-12T03:19:23.835989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:04:05.050990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-12T03:19:26.568261+0000","last_change":"2024-09-12T03:19:25.578617+0000","last_active":"2024-09-12T03:19:26.568261+0000","last_peered":"2024-09-12T03:19:26.568261+0000","last_clean":"2024-09-12T03:19:26.568261+0000","last_became_active":"2024-09-12T03:19:25.578346+0000","last_became_peered":"2024-09-12T03:19:25.578346+0000","last_unstale":"2024-09-12T03:19:26.568261+0000","last_undegraded":"2024-09-12T03:19:26.568261+0000","last_fullsized":"2024-09-12T03:19:26.568261+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:26:49.228871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":14,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-12T03:19:27.854125+0000","last_change":"2024-09-12T03:19:24.858024+0000","last_active":"2024-09-12T03:19:27.854125+0000","last_peered":"2024-09-12T03:19:27.854125+0000","last_clean":"2024-09-12T03:19:27.854125+0000","last_became_active":"2024-09-12T03:19:24.857142+0000","last_became_peered":"2024-09-12T03:19:24.857142+0000","last_unstale":"2024-09-12T03:19:27.854125+0000","last_undegraded":"2024-09-12T03:19:27.854125+0000","last_fullsized":"2024-09-12T03:19:27.854125+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_clean_scrub_stamp":"2024-09-12T03:19:23.835989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:28:38.811682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":232,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-12T03:19:26.570756+0000","last_change":"2024-09-12T03:16:21.883446+0000","last_active":"2024-09-12T03:19:26.570756+0000","last_peered":"2024-09-12T03:19:26.570756+0000","last_clean":"2024-09-12T03:19:26.570756+0000","last_became_active":"2024-09-12T03:16:21.882207+0000","last_became_peered":"2024-09-12T03:16:21.882207+0000","last_unstale":"2024-09-12T03:19:26.570756+0000","last_undegraded":"2024-09-12T03:19:26.570756+0000","last_fullsized":"2024-09-12T03:19:26.570756+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:20.316426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:20.316426+0000","last_clean_scrub_stamp":"2024-09-12T03:16:20.316426+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:12:15.424320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2340,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":38,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":4078,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":34,"ondisk_log_size":34,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542184,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32436,"kb_used_data":1240,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339404,"statfs":{"total":96636764160,"available":96603549696,"internally_reserved":0,"allocated":1269760,"data_stored":866511,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542186,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31840,"kb_used_data":652,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340000,"statfs":{"total":96636764160,"available":96604160000,"internally_reserved":0,"allocated":667648,"data_stored":275983,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542184,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35964,"kb_used_data":616,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335876,"statfs":{"total":96636764160,"available":96599937024,"internally_reserved":0,"allocated":630784,"data_stored":256641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542184,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31860,"kb_used_data":664,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339980,"statfs":{"total":96636764160,"available":96604139520,"internally_reserved":0,"allocated":679936,"data_stored":277595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542185,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27696,"kb_used_data":660,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344144,"statfs":{"total":96636764160,"available":96608403456,"internally_reserved":0,"allocated":675840,"data_stored":276039,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542185,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27688,"kb_used_data":660,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344152,"statfs":{"total":96636764160,"available":96608411648,"internally_reserved":0,"allocated":675840,"data_stored":276039,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542185,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31808,"kb_used_data":616,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340032,"statfs":{"total":96636764160,"available":96604192768,"internally_reserved":0,"allocated":630784,"data_stored":256641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542186,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28276,"kb_used_data":1248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343564,"statfs":{"total":96636764160,"available":96607809536,"internally_reserved":0,"allocated":1277952,"data_stored":868545,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T03:19:30.806 DEBUG:teuthology.orchestra.run.smithi027:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-12T03:19:30.809 DEBUG:teuthology.orchestra.run.smithi045:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-12T03:19:33.244 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-12T03:19:33.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:33.360+0000 7f67762d0640 1 Processor -- start 2024-09-12T03:19:33.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:33.360+0000 7f67762d0640 1 -- start start 2024-09-12T03:19:33.363 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:33.360+0000 7f67762d0640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f67700cf3c0 0x7f67701691f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:33.363 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:33.360+0000 7f67762d0640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f677015ea40 con 0x7f677015e660 2024-09-12T03:19:33.363 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:33.360+0000 7f67762d0640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f677006d240 con 0x7f6770073920 2024-09-12T03:19:33.363 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:33.360+0000 7f67762d0640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f67700b5a20 con 0x7f67700cf3c0 2024-09-12T03:19:33.363 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:33.361+0000 7f677486d640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f67700cf3c0 0x7f67701691f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:33.363 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:33.361+0000 7f676ffff640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f677015e660 0x7f6770073210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:42400/0 (socket says 172.21.15.27:42400) 2024-09-12T03:19:33.364 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:33.361+0000 7f676ffff640 1 -- 172.21.15.27:0/1976160799 learned_addr learned my addr 172.21.15.27:0/1976160799 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:19:33.364 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:33.361+0000 7f677486d640 1 -- 172.21.15.27:0/1976160799 >> v1:172.21.15.45:6789/0 conn(0x7f6770073920 legacy=0x7f6770070a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:33.364 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:33.361+0000 7f677486d640 1 -- 172.21.15.27:0/1976160799 >> v1:172.21.15.27:6789/0 conn(0x7f677015e660 legacy=0x7f6770073210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:33.364 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:33.361+0000 7f677486d640 1 -- 172.21.15.27:0/1976160799 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6770071970 con 0x7f67700cf3c0 2024-09-12T03:19:33.364 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:33.362+0000 7f677486d640 1 --2- 172.21.15.27:0/1976160799 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f67700cf3c0 0x7f67701691f0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f6768009d90 tx=0x7f676802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4c22ba7fc7a4cc1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:33.364 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:33.362+0000 7f676effd640 1 -- 172.21.15.27:0/1976160799 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6768007740 con 0x7f67700cf3c0 2024-09-12T03:19:33.364 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:33.362+0000 7f676effd640 1 -- 172.21.15.27:0/1976160799 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f67680078e0 con 0x7f67700cf3c0 2024-09-12T03:19:33.364 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:33.362+0000 7f676effd640 1 -- 172.21.15.27:0/1976160799 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6768007cb0 con 0x7f67700cf3c0 2024-09-12T03:19:33.368 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:33.362+0000 7f67762d0640 1 -- 172.21.15.27:0/1976160799 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f67700cf3c0 msgr2=0x7f67701691f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:33.368 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:33.362+0000 7f67762d0640 1 --2- 172.21.15.27:0/1976160799 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f67700cf3c0 0x7f67701691f0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f6768009d90 tx=0x7f676802a6a0 comp rx=0 tx=0).stop 2024-09-12T03:19:33.368 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:33.363+0000 7f67762d0640 1 -- 172.21.15.27:0/1976160799 shutdown_connections 2024-09-12T03:19:33.368 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:33.363+0000 7f67762d0640 1 --2- 172.21.15.27:0/1976160799 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f67700cf3c0 0x7f67701691f0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:33.368 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:33.363+0000 7f67762d0640 1 -- 172.21.15.27:0/1976160799 >> 172.21.15.27:0/1976160799 conn(0x7f677006f2c0 msgr2=0x7f677006f6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:33.368 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:33.363+0000 7f67762d0640 1 -- 172.21.15.27:0/1976160799 shutdown_connections 2024-09-12T03:19:33.368 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:33.363+0000 7f67762d0640 1 -- 172.21.15.27:0/1976160799 wait complete. 2024-09-12T03:19:33.368 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:33.364+0000 7f67762d0640 1 Processor -- start 2024-09-12T03:19:33.368 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:33.364+0000 7f67762d0640 1 -- start start 2024-09-12T03:19:33.369 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:33.365+0000 7f67762d0640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6770073920 0x7f6770149df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:33.369 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:33.365+0000 7f67762d0640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f67700cf3c0 0x7f6770146ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:33.369 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:33.365+0000 7f67762d0640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f677015e660 0x7f67700ceb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:33.369 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:33.365+0000 7f67762d0640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f677006d7c0 con 0x7f677015e660 2024-09-12T03:19:33.369 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:33.365+0000 7f67762d0640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f677006d4e0 con 0x7f6770073920 2024-09-12T03:19:33.369 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:33.365+0000 7f67762d0640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f6770071150 con 0x7f67700cf3c0 2024-09-12T03:19:33.369 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:33.365+0000 7f677486d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f677015e660 0x7f67700ceb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:33.369 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:33.365+0000 7f677486d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f677015e660 0x7f67700ceb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38382/0 (socket says 172.21.15.27:38382) 2024-09-12T03:19:33.369 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:33.365+0000 7f677486d640 1 -- 172.21.15.27:0/1521368971 learned_addr learned my addr 172.21.15.27:0/1521368971 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:19:33.369 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:33.365+0000 7f676f7fe640 1 --2- 172.21.15.27:0/1521368971 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f67700cf3c0 0x7f6770146ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:33.369 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:33.365+0000 7f676ffff640 1 --2- 172.21.15.27:0/1521368971 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6770073920 0x7f6770149df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:33.370 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:33.365+0000 7f677486d640 1 -- 172.21.15.27:0/1521368971 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f67700cf3c0 msgr2=0x7f6770146ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:33.370 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:33.365+0000 7f677486d640 1 --2- 172.21.15.27:0/1521368971 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f67700cf3c0 0x7f6770146ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:33.370 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:33.365+0000 7f677486d640 1 -- 172.21.15.27:0/1521368971 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6770073920 msgr2=0x7f6770149df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:33.370 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:33.365+0000 7f677486d640 1 --2- 172.21.15.27:0/1521368971 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6770073920 0x7f6770149df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:33.370 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:33.365+0000 7f677486d640 1 -- 172.21.15.27:0/1521368971 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67700cd810 con 0x7f677015e660 2024-09-12T03:19:33.370 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:33.365+0000 7f676f7fe640 1 --2- 172.21.15.27:0/1521368971 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f67700cf3c0 0x7f6770146ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:19:33.370 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:33.365+0000 7f676ffff640 1 --2- 172.21.15.27:0/1521368971 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6770073920 0x7f6770149df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:19:33.370 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:33.366+0000 7f677486d640 1 --2- 172.21.15.27:0/1521368971 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f677015e660 0x7f67700ceb50 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f6768009eb0 tx=0x7f6768004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:33.370 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:33.366+0000 7f676cff9640 1 -- 172.21.15.27:0/1521368971 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6768040070 con 0x7f677015e660 2024-09-12T03:19:33.370 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:33.366+0000 7f676cff9640 1 -- 172.21.15.27:0/1521368971 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6768031c90 con 0x7f677015e660 2024-09-12T03:19:33.370 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:33.366+0000 7f67762d0640 1 -- 172.21.15.27:0/1521368971 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67700cdab0 con 0x7f677015e660 2024-09-12T03:19:33.370 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:33.366+0000 7f676cff9640 1 -- 172.21.15.27:0/1521368971 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f676803c640 con 0x7f677015e660 2024-09-12T03:19:33.370 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:33.366+0000 7f67762d0640 1 -- 172.21.15.27:0/1521368971 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67700ce070 con 0x7f677015e660 2024-09-12T03:19:33.371 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:33.367+0000 7f67762d0640 1 -- 172.21.15.27:0/1521368971 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6770070a40 con 0x7f677015e660 2024-09-12T03:19:33.371 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:33.368+0000 7f676cff9640 1 -- 172.21.15.27:0/1521368971 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 9) ==== 90651+0+0 (secure 0 0 0) 0x7f6768039040 con 0x7f677015e660 2024-09-12T03:19:33.371 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:33.368+0000 7f676cff9640 1 --2- 172.21.15.27:0/1521368971 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f675c06c010 0x7f675c06e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:33.371 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:33.368+0000 7f676cff9640 1 -- 172.21.15.27:0/1521368971 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5709+0+0 (secure 0 0 0) 0x7f67680ac8c0 con 0x7f677015e660 2024-09-12T03:19:33.371 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:33.368+0000 7f676ffff640 1 --2- 172.21.15.27:0/1521368971 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f675c06c010 0x7f675c06e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:33.372 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:33.369+0000 7f676ffff640 1 --2- 172.21.15.27:0/1521368971 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f675c06c010 0x7f675c06e4d0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f6760000f60 tx=0x7f6760009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:33.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:33.372+0000 7f676cff9640 1 -- 172.21.15.27:0/1521368971 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f676804b340 con 0x7f677015e660 2024-09-12T03:19:33.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:33.545+0000 7f67762d0640 1 -- 172.21.15.27:0/1521368971 --> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f677015d5f0 con 0x7f675c06c010 2024-09-12T03:19:33.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:33.552+0000 7f676cff9640 1 -- 172.21.15.27:0/1521368971 <== mgr.4102 v2:172.21.15.45:6832/3923821219 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251954 (secure 0 0 0) 0x7f677015d5f0 con 0x7f675c06c010 2024-09-12T03:19:33.555 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:19:33.556 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2024-09-12T03:19:33.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:33.555+0000 7f67762d0640 1 -- 172.21.15.27:0/1521368971 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f675c06c010 msgr2=0x7f675c06e4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:33.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:33.555+0000 7f67762d0640 1 --2- 172.21.15.27:0/1521368971 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f675c06c010 0x7f675c06e4d0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f6760000f60 tx=0x7f6760009040 comp rx=0 tx=0).stop 2024-09-12T03:19:33.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:33.555+0000 7f67762d0640 1 -- 172.21.15.27:0/1521368971 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f677015e660 msgr2=0x7f67700ceb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:33.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:33.555+0000 7f67762d0640 1 --2- 172.21.15.27:0/1521368971 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f677015e660 0x7f67700ceb50 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f6768009eb0 tx=0x7f6768004a40 comp rx=0 tx=0).stop 2024-09-12T03:19:33.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:33.555+0000 7f67762d0640 1 -- 172.21.15.27:0/1521368971 shutdown_connections 2024-09-12T03:19:33.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:33.555+0000 7f67762d0640 1 --2- 172.21.15.27:0/1521368971 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f675c06c010 0x7f675c06e4d0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:33.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:33.555+0000 7f67762d0640 1 --2- 172.21.15.27:0/1521368971 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f677015e660 0x7f67700ceb50 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:33.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:33.555+0000 7f67762d0640 1 --2- 172.21.15.27:0/1521368971 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f67700cf3c0 0x7f6770146ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:33.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:33.555+0000 7f67762d0640 1 --2- 172.21.15.27:0/1521368971 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6770073920 0x7f6770149df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:33.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:33.555+0000 7f67762d0640 1 -- 172.21.15.27:0/1521368971 >> 172.21.15.27:0/1521368971 conn(0x7f677006f2c0 msgr2=0x7f677015c860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:33.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:33.555+0000 7f67762d0640 1 -- 172.21.15.27:0/1521368971 shutdown_connections 2024-09-12T03:19:33.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:33.555+0000 7f67762d0640 1 -- 172.21.15.27:0/1521368971 wait complete. 2024-09-12T03:19:33.567 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":145,"stamp":"2024-09-12T03:19:32.096361+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":173,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":247648,"kb_used_data":6440,"kb_used_omap":12,"kb_used_meta":241011,"kb_avail":754727072,"statfs":{"total":773094113280,"available":772840521728,"internally_reserved":0,"allocated":6594560,"data_stored":3393420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":246795824},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":20,"apply_latency_ms":20,"commit_latency_ns":20000000,"apply_latency_ns":20000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.253011"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-12T03:19:28.206981+0000","last_change":"2024-09-12T03:19:25.579321+0000","last_active":"2024-09-12T03:19:28.206981+0000","last_peered":"2024-09-12T03:19:28.206981+0000","last_clean":"2024-09-12T03:19:28.206981+0000","last_became_active":"2024-09-12T03:19:25.579050+0000","last_became_peered":"2024-09-12T03:19:25.579050+0000","last_unstale":"2024-09-12T03:19:28.206981+0000","last_undegraded":"2024-09-12T03:19:28.206981+0000","last_fullsized":"2024-09-12T03:19:28.206981+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:00:00.080693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-12T03:19:26.568533+0000","last_change":"2024-09-12T03:19:25.581758+0000","last_active":"2024-09-12T03:19:26.568533+0000","last_peered":"2024-09-12T03:19:26.568533+0000","last_clean":"2024-09-12T03:19:26.568533+0000","last_became_active":"2024-09-12T03:19:25.581483+0000","last_became_peered":"2024-09-12T03:19:25.581483+0000","last_unstale":"2024-09-12T03:19:26.568533+0000","last_undegraded":"2024-09-12T03:19:26.568533+0000","last_fullsized":"2024-09-12T03:19:26.568533+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:43:46.675840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-12T03:19:26.569352+0000","last_change":"2024-09-12T03:19:25.584735+0000","last_active":"2024-09-12T03:19:26.569352+0000","last_peered":"2024-09-12T03:19:26.569352+0000","last_clean":"2024-09-12T03:19:26.569352+0000","last_became_active":"2024-09-12T03:19:25.584357+0000","last_became_peered":"2024-09-12T03:19:25.584357+0000","last_unstale":"2024-09-12T03:19:26.569352+0000","last_undegraded":"2024-09-12T03:19:26.569352+0000","last_fullsized":"2024-09-12T03:19:26.569352+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:59:03.828976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-12T03:19:28.006939+0000","last_change":"2024-09-12T03:19:25.570901+0000","last_active":"2024-09-12T03:19:28.006939+0000","last_peered":"2024-09-12T03:19:28.006939+0000","last_clean":"2024-09-12T03:19:28.006939+0000","last_became_active":"2024-09-12T03:19:25.570622+0000","last_became_peered":"2024-09-12T03:19:25.570622+0000","last_unstale":"2024-09-12T03:19:28.006939+0000","last_undegraded":"2024-09-12T03:19:28.006939+0000","last_fullsized":"2024-09-12T03:19:28.006939+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:42:52.144923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-12T03:19:28.206806+0000","last_change":"2024-09-12T03:19:25.573294+0000","last_active":"2024-09-12T03:19:28.206806+0000","last_peered":"2024-09-12T03:19:28.206806+0000","last_clean":"2024-09-12T03:19:28.206806+0000","last_became_active":"2024-09-12T03:19:25.572975+0000","last_became_peered":"2024-09-12T03:19:25.572975+0000","last_unstale":"2024-09-12T03:19:28.206806+0000","last_undegraded":"2024-09-12T03:19:28.206806+0000","last_fullsized":"2024-09-12T03:19:28.206806+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:45:37.626405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-12T03:19:27.850378+0000","last_change":"2024-09-12T03:19:25.581045+0000","last_active":"2024-09-12T03:19:27.850378+0000","last_peered":"2024-09-12T03:19:27.850378+0000","last_clean":"2024-09-12T03:19:27.850378+0000","last_became_active":"2024-09-12T03:19:25.580431+0000","last_became_peered":"2024-09-12T03:19:25.580431+0000","last_unstale":"2024-09-12T03:19:27.850378+0000","last_undegraded":"2024-09-12T03:19:27.850378+0000","last_fullsized":"2024-09-12T03:19:27.850378+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:44:04.569606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-12T03:19:26.569886+0000","last_change":"2024-09-12T03:19:25.568621+0000","last_active":"2024-09-12T03:19:26.569886+0000","last_peered":"2024-09-12T03:19:26.569886+0000","last_clean":"2024-09-12T03:19:26.569886+0000","last_became_active":"2024-09-12T03:19:25.568346+0000","last_became_peered":"2024-09-12T03:19:25.568346+0000","last_unstale":"2024-09-12T03:19:26.569886+0000","last_undegraded":"2024-09-12T03:19:26.569886+0000","last_fullsized":"2024-09-12T03:19:26.569886+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:43:46.346281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-12T03:19:26.568190+0000","last_change":"2024-09-12T03:19:25.575264+0000","last_active":"2024-09-12T03:19:26.568190+0000","last_peered":"2024-09-12T03:19:26.568190+0000","last_clean":"2024-09-12T03:19:26.568190+0000","last_became_active":"2024-09-12T03:19:25.574998+0000","last_became_peered":"2024-09-12T03:19:25.574998+0000","last_unstale":"2024-09-12T03:19:26.568190+0000","last_undegraded":"2024-09-12T03:19:26.568190+0000","last_fullsized":"2024-09-12T03:19:26.568190+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:14:06.603640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-12T03:19:26.570405+0000","last_change":"2024-09-12T03:19:25.582104+0000","last_active":"2024-09-12T03:19:26.570405+0000","last_peered":"2024-09-12T03:19:26.570405+0000","last_clean":"2024-09-12T03:19:26.570405+0000","last_became_active":"2024-09-12T03:19:25.581789+0000","last_became_peered":"2024-09-12T03:19:25.581789+0000","last_unstale":"2024-09-12T03:19:26.570405+0000","last_undegraded":"2024-09-12T03:19:26.570405+0000","last_fullsized":"2024-09-12T03:19:26.570405+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:22:24.747123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-12T03:19:28.206939+0000","last_change":"2024-09-12T03:19:25.575892+0000","last_active":"2024-09-12T03:19:28.206939+0000","last_peered":"2024-09-12T03:19:28.206939+0000","last_clean":"2024-09-12T03:19:28.206939+0000","last_became_active":"2024-09-12T03:19:25.575644+0000","last_became_peered":"2024-09-12T03:19:25.575644+0000","last_unstale":"2024-09-12T03:19:28.206939+0000","last_undegraded":"2024-09-12T03:19:28.206939+0000","last_fullsized":"2024-09-12T03:19:28.206939+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:24:20.874837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-12T03:19:28.207355+0000","last_change":"2024-09-12T03:19:25.574080+0000","last_active":"2024-09-12T03:19:28.207355+0000","last_peered":"2024-09-12T03:19:28.207355+0000","last_clean":"2024-09-12T03:19:28.207355+0000","last_became_active":"2024-09-12T03:19:25.573802+0000","last_became_peered":"2024-09-12T03:19:25.573802+0000","last_unstale":"2024-09-12T03:19:28.207355+0000","last_undegraded":"2024-09-12T03:19:28.207355+0000","last_fullsized":"2024-09-12T03:19:28.207355+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:17:09.999225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-12T03:19:28.007036+0000","last_change":"2024-09-12T03:19:25.576622+0000","last_active":"2024-09-12T03:19:28.007036+0000","last_peered":"2024-09-12T03:19:28.007036+0000","last_clean":"2024-09-12T03:19:28.007036+0000","last_became_active":"2024-09-12T03:19:25.576317+0000","last_became_peered":"2024-09-12T03:19:25.576317+0000","last_unstale":"2024-09-12T03:19:28.007036+0000","last_undegraded":"2024-09-12T03:19:28.007036+0000","last_fullsized":"2024-09-12T03:19:28.007036+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:53:07.729005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-12T03:19:27.854297+0000","last_change":"2024-09-12T03:19:25.578981+0000","last_active":"2024-09-12T03:19:27.854297+0000","last_peered":"2024-09-12T03:19:27.854297+0000","last_clean":"2024-09-12T03:19:27.854297+0000","last_became_active":"2024-09-12T03:19:25.578701+0000","last_became_peered":"2024-09-12T03:19:25.578701+0000","last_unstale":"2024-09-12T03:19:27.854297+0000","last_undegraded":"2024-09-12T03:19:27.854297+0000","last_fullsized":"2024-09-12T03:19:27.854297+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:29:11.261684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-12T03:19:26.569608+0000","last_change":"2024-09-12T03:19:25.577974+0000","last_active":"2024-09-12T03:19:26.569608+0000","last_peered":"2024-09-12T03:19:26.569608+0000","last_clean":"2024-09-12T03:19:26.569608+0000","last_became_active":"2024-09-12T03:19:25.577683+0000","last_became_peered":"2024-09-12T03:19:25.577683+0000","last_unstale":"2024-09-12T03:19:26.569608+0000","last_undegraded":"2024-09-12T03:19:26.569608+0000","last_fullsized":"2024-09-12T03:19:26.569608+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:44:45.155856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-12T03:19:26.569328+0000","last_change":"2024-09-12T03:19:25.573261+0000","last_active":"2024-09-12T03:19:26.569328+0000","last_peered":"2024-09-12T03:19:26.569328+0000","last_clean":"2024-09-12T03:19:26.569328+0000","last_became_active":"2024-09-12T03:19:25.572969+0000","last_became_peered":"2024-09-12T03:19:25.572969+0000","last_unstale":"2024-09-12T03:19:26.569328+0000","last_undegraded":"2024-09-12T03:19:26.569328+0000","last_fullsized":"2024-09-12T03:19:26.569328+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:16:49.960138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-12T03:19:27.850337+0000","last_change":"2024-09-12T03:19:25.574807+0000","last_active":"2024-09-12T03:19:27.850337+0000","last_peered":"2024-09-12T03:19:27.850337+0000","last_clean":"2024-09-12T03:19:27.850337+0000","last_became_active":"2024-09-12T03:19:25.574550+0000","last_became_peered":"2024-09-12T03:19:25.574550+0000","last_unstale":"2024-09-12T03:19:27.850337+0000","last_undegraded":"2024-09-12T03:19:27.850337+0000","last_fullsized":"2024-09-12T03:19:27.850337+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:56:51.692997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-12T03:19:26.568652+0000","last_change":"2024-09-12T03:19:25.577565+0000","last_active":"2024-09-12T03:19:26.568652+0000","last_peered":"2024-09-12T03:19:26.568652+0000","last_clean":"2024-09-12T03:19:26.568652+0000","last_became_active":"2024-09-12T03:19:25.577294+0000","last_became_peered":"2024-09-12T03:19:25.577294+0000","last_unstale":"2024-09-12T03:19:26.568652+0000","last_undegraded":"2024-09-12T03:19:26.568652+0000","last_fullsized":"2024-09-12T03:19:26.568652+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:56:05.227809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-12T03:19:26.570511+0000","last_change":"2024-09-12T03:19:25.584433+0000","last_active":"2024-09-12T03:19:26.570511+0000","last_peered":"2024-09-12T03:19:26.570511+0000","last_clean":"2024-09-12T03:19:26.570511+0000","last_became_active":"2024-09-12T03:19:25.583951+0000","last_became_peered":"2024-09-12T03:19:25.583951+0000","last_unstale":"2024-09-12T03:19:26.570511+0000","last_undegraded":"2024-09-12T03:19:26.570511+0000","last_fullsized":"2024-09-12T03:19:26.570511+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:43:55.912919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-12T03:19:28.206763+0000","last_change":"2024-09-12T03:19:25.574096+0000","last_active":"2024-09-12T03:19:28.206763+0000","last_peered":"2024-09-12T03:19:28.206763+0000","last_clean":"2024-09-12T03:19:28.206763+0000","last_became_active":"2024-09-12T03:19:25.573759+0000","last_became_peered":"2024-09-12T03:19:25.573759+0000","last_unstale":"2024-09-12T03:19:28.206763+0000","last_undegraded":"2024-09-12T03:19:28.206763+0000","last_fullsized":"2024-09-12T03:19:28.206763+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:52:01.678549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-12T03:19:26.568866+0000","last_change":"2024-09-12T03:19:25.577212+0000","last_active":"2024-09-12T03:19:26.568866+0000","last_peered":"2024-09-12T03:19:26.568866+0000","last_clean":"2024-09-12T03:19:26.568866+0000","last_became_active":"2024-09-12T03:19:25.576935+0000","last_became_peered":"2024-09-12T03:19:25.576935+0000","last_unstale":"2024-09-12T03:19:26.568866+0000","last_undegraded":"2024-09-12T03:19:26.568866+0000","last_fullsized":"2024-09-12T03:19:26.568866+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:55:32.939225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-12T03:19:26.570425+0000","last_change":"2024-09-12T03:19:25.580816+0000","last_active":"2024-09-12T03:19:26.570425+0000","last_peered":"2024-09-12T03:19:26.570425+0000","last_clean":"2024-09-12T03:19:26.570425+0000","last_became_active":"2024-09-12T03:19:25.580500+0000","last_became_peered":"2024-09-12T03:19:25.580500+0000","last_unstale":"2024-09-12T03:19:26.570425+0000","last_undegraded":"2024-09-12T03:19:26.570425+0000","last_fullsized":"2024-09-12T03:19:26.570425+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:56:51.692997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-12T03:19:26.568892+0000","last_change":"2024-09-12T03:19:25.582734+0000","last_active":"2024-09-12T03:19:26.568892+0000","last_peered":"2024-09-12T03:19:26.568892+0000","last_clean":"2024-09-12T03:19:26.568892+0000","last_became_active":"2024-09-12T03:19:25.582364+0000","last_became_peered":"2024-09-12T03:19:25.582364+0000","last_unstale":"2024-09-12T03:19:26.568892+0000","last_undegraded":"2024-09-12T03:19:26.568892+0000","last_fullsized":"2024-09-12T03:19:26.568892+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:41:18.624900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-12T03:19:26.569424+0000","last_change":"2024-09-12T03:19:25.582851+0000","last_active":"2024-09-12T03:19:26.569424+0000","last_peered":"2024-09-12T03:19:26.569424+0000","last_clean":"2024-09-12T03:19:26.569424+0000","last_became_active":"2024-09-12T03:19:25.582538+0000","last_became_peered":"2024-09-12T03:19:25.582538+0000","last_unstale":"2024-09-12T03:19:26.569424+0000","last_undegraded":"2024-09-12T03:19:26.569424+0000","last_fullsized":"2024-09-12T03:19:26.569424+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:04:06.053023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-12T03:19:27.854182+0000","last_change":"2024-09-12T03:19:25.579664+0000","last_active":"2024-09-12T03:19:27.854182+0000","last_peered":"2024-09-12T03:19:27.854182+0000","last_clean":"2024-09-12T03:19:27.854182+0000","last_became_active":"2024-09-12T03:19:25.579368+0000","last_became_peered":"2024-09-12T03:19:25.579368+0000","last_unstale":"2024-09-12T03:19:27.854182+0000","last_undegraded":"2024-09-12T03:19:27.854182+0000","last_fullsized":"2024-09-12T03:19:27.854182+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:00:18.291876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-12T03:19:28.207086+0000","last_change":"2024-09-12T03:19:25.572942+0000","last_active":"2024-09-12T03:19:28.207086+0000","last_peered":"2024-09-12T03:19:28.207086+0000","last_clean":"2024-09-12T03:19:28.207086+0000","last_became_active":"2024-09-12T03:19:25.572615+0000","last_became_peered":"2024-09-12T03:19:25.572615+0000","last_unstale":"2024-09-12T03:19:28.207086+0000","last_undegraded":"2024-09-12T03:19:28.207086+0000","last_fullsized":"2024-09-12T03:19:28.207086+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:50:58.833474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-12T03:19:28.007050+0000","last_change":"2024-09-12T03:19:25.571685+0000","last_active":"2024-09-12T03:19:28.007050+0000","last_peered":"2024-09-12T03:19:28.007050+0000","last_clean":"2024-09-12T03:19:28.007050+0000","last_became_active":"2024-09-12T03:19:25.571304+0000","last_became_peered":"2024-09-12T03:19:25.571304+0000","last_unstale":"2024-09-12T03:19:28.007050+0000","last_undegraded":"2024-09-12T03:19:28.007050+0000","last_fullsized":"2024-09-12T03:19:28.007050+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:02:52.964182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-12T03:19:27.850882+0000","last_change":"2024-09-12T03:19:25.576399+0000","last_active":"2024-09-12T03:19:27.850882+0000","last_peered":"2024-09-12T03:19:27.850882+0000","last_clean":"2024-09-12T03:19:27.850882+0000","last_became_active":"2024-09-12T03:19:25.575434+0000","last_became_peered":"2024-09-12T03:19:25.575434+0000","last_unstale":"2024-09-12T03:19:27.850882+0000","last_undegraded":"2024-09-12T03:19:27.850882+0000","last_fullsized":"2024-09-12T03:19:27.850882+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:11:06.694559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-12T03:19:28.206843+0000","last_change":"2024-09-12T03:19:25.574672+0000","last_active":"2024-09-12T03:19:28.206843+0000","last_peered":"2024-09-12T03:19:28.206843+0000","last_clean":"2024-09-12T03:19:28.206843+0000","last_became_active":"2024-09-12T03:19:25.574069+0000","last_became_peered":"2024-09-12T03:19:25.574069+0000","last_unstale":"2024-09-12T03:19:28.206843+0000","last_undegraded":"2024-09-12T03:19:28.206843+0000","last_fullsized":"2024-09-12T03:19:28.206843+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:55:09.070173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-12T03:19:26.568772+0000","last_change":"2024-09-12T03:19:25.572562+0000","last_active":"2024-09-12T03:19:26.568772+0000","last_peered":"2024-09-12T03:19:26.568772+0000","last_clean":"2024-09-12T03:19:26.568772+0000","last_became_active":"2024-09-12T03:19:25.572293+0000","last_became_peered":"2024-09-12T03:19:25.572293+0000","last_unstale":"2024-09-12T03:19:26.568772+0000","last_undegraded":"2024-09-12T03:19:26.568772+0000","last_fullsized":"2024-09-12T03:19:26.568772+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:28:50.615286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-12T03:19:26.569889+0000","last_change":"2024-09-12T03:19:25.581520+0000","last_active":"2024-09-12T03:19:26.569889+0000","last_peered":"2024-09-12T03:19:26.569889+0000","last_clean":"2024-09-12T03:19:26.569889+0000","last_became_active":"2024-09-12T03:19:25.581240+0000","last_became_peered":"2024-09-12T03:19:25.581240+0000","last_unstale":"2024-09-12T03:19:26.569889+0000","last_undegraded":"2024-09-12T03:19:26.569889+0000","last_fullsized":"2024-09-12T03:19:26.569889+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:12:38.654058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-12T03:19:28.206432+0000","last_change":"2024-09-12T03:19:25.572846+0000","last_active":"2024-09-12T03:19:28.206432+0000","last_peered":"2024-09-12T03:19:28.206432+0000","last_clean":"2024-09-12T03:19:28.206432+0000","last_became_active":"2024-09-12T03:19:25.572563+0000","last_became_peered":"2024-09-12T03:19:25.572563+0000","last_unstale":"2024-09-12T03:19:28.206432+0000","last_undegraded":"2024-09-12T03:19:28.206432+0000","last_fullsized":"2024-09-12T03:19:28.206432+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:21:29.241164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-12T03:19:27.850402+0000","last_change":"2024-09-12T03:19:25.581204+0000","last_active":"2024-09-12T03:19:27.850402+0000","last_peered":"2024-09-12T03:19:27.850402+0000","last_clean":"2024-09-12T03:19:27.850402+0000","last_became_active":"2024-09-12T03:19:25.580777+0000","last_became_peered":"2024-09-12T03:19:25.580777+0000","last_unstale":"2024-09-12T03:19:27.850402+0000","last_undegraded":"2024-09-12T03:19:27.850402+0000","last_fullsized":"2024-09-12T03:19:27.850402+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:13:41.572442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-12T03:19:27.851263+0000","last_change":"2024-09-12T03:19:25.580037+0000","last_active":"2024-09-12T03:19:27.851263+0000","last_peered":"2024-09-12T03:19:27.851263+0000","last_clean":"2024-09-12T03:19:27.851263+0000","last_became_active":"2024-09-12T03:19:25.579695+0000","last_became_peered":"2024-09-12T03:19:25.579695+0000","last_unstale":"2024-09-12T03:19:27.851263+0000","last_undegraded":"2024-09-12T03:19:27.851263+0000","last_fullsized":"2024-09-12T03:19:27.851263+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:14:06.603640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"55'4","reported_seq":18,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-12T03:19:27.851275+0000","last_change":"2024-09-12T03:19:24.860694+0000","last_active":"2024-09-12T03:19:27.851275+0000","last_peered":"2024-09-12T03:19:27.851275+0000","last_clean":"2024-09-12T03:19:27.851275+0000","last_became_active":"2024-09-12T03:19:24.860380+0000","last_became_peered":"2024-09-12T03:19:24.860380+0000","last_unstale":"2024-09-12T03:19:27.851275+0000","last_undegraded":"2024-09-12T03:19:27.851275+0000","last_fullsized":"2024-09-12T03:19:27.851275+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_clean_scrub_stamp":"2024-09-12T03:19:23.835989+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:51:36.581086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-12T03:19:28.207098+0000","last_change":"2024-09-12T03:19:25.578131+0000","last_active":"2024-09-12T03:19:28.207098+0000","last_peered":"2024-09-12T03:19:28.207098+0000","last_clean":"2024-09-12T03:19:28.207098+0000","last_became_active":"2024-09-12T03:19:25.577855+0000","last_became_peered":"2024-09-12T03:19:25.577855+0000","last_unstale":"2024-09-12T03:19:28.207098+0000","last_undegraded":"2024-09-12T03:19:28.207098+0000","last_fullsized":"2024-09-12T03:19:28.207098+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:31:31.908376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"55'1","reported_seq":15,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-12T03:19:26.568647+0000","last_change":"2024-09-12T03:19:24.863015+0000","last_active":"2024-09-12T03:19:26.568647+0000","last_peered":"2024-09-12T03:19:26.568647+0000","last_clean":"2024-09-12T03:19:26.568647+0000","last_became_active":"2024-09-12T03:19:24.862457+0000","last_became_peered":"2024-09-12T03:19:24.862457+0000","last_unstale":"2024-09-12T03:19:26.568647+0000","last_undegraded":"2024-09-12T03:19:26.568647+0000","last_fullsized":"2024-09-12T03:19:26.568647+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_clean_scrub_stamp":"2024-09-12T03:19:23.835989+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:17:40.881373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-12T03:19:27.850653+0000","last_change":"2024-09-12T03:19:25.581589+0000","last_active":"2024-09-12T03:19:27.850653+0000","last_peered":"2024-09-12T03:19:27.850653+0000","last_clean":"2024-09-12T03:19:27.850653+0000","last_became_active":"2024-09-12T03:19:25.581294+0000","last_became_peered":"2024-09-12T03:19:25.581294+0000","last_unstale":"2024-09-12T03:19:27.850653+0000","last_undegraded":"2024-09-12T03:19:27.850653+0000","last_fullsized":"2024-09-12T03:19:27.850653+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:37:42.235320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"55'2","reported_seq":16,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-12T03:19:27.851078+0000","last_change":"2024-09-12T03:19:24.862226+0000","last_active":"2024-09-12T03:19:27.851078+0000","last_peered":"2024-09-12T03:19:27.851078+0000","last_clean":"2024-09-12T03:19:27.851078+0000","last_became_active":"2024-09-12T03:19:24.861956+0000","last_became_peered":"2024-09-12T03:19:24.861956+0000","last_unstale":"2024-09-12T03:19:27.851078+0000","last_undegraded":"2024-09-12T03:19:27.851078+0000","last_fullsized":"2024-09-12T03:19:27.851078+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_clean_scrub_stamp":"2024-09-12T03:19:23.835989+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:16:02.571700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-12T03:19:27.850978+0000","last_change":"2024-09-12T03:19:25.576177+0000","last_active":"2024-09-12T03:19:27.850978+0000","last_peered":"2024-09-12T03:19:27.850978+0000","last_clean":"2024-09-12T03:19:27.850978+0000","last_became_active":"2024-09-12T03:19:25.575854+0000","last_became_peered":"2024-09-12T03:19:25.575854+0000","last_unstale":"2024-09-12T03:19:27.850978+0000","last_undegraded":"2024-09-12T03:19:27.850978+0000","last_fullsized":"2024-09-12T03:19:27.850978+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:03:02.553134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"55'2","reported_seq":16,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-12T03:19:28.007074+0000","last_change":"2024-09-12T03:19:24.859901+0000","last_active":"2024-09-12T03:19:28.007074+0000","last_peered":"2024-09-12T03:19:28.007074+0000","last_clean":"2024-09-12T03:19:28.007074+0000","last_became_active":"2024-09-12T03:19:24.859637+0000","last_became_peered":"2024-09-12T03:19:24.859637+0000","last_unstale":"2024-09-12T03:19:28.007074+0000","last_undegraded":"2024-09-12T03:19:28.007074+0000","last_fullsized":"2024-09-12T03:19:28.007074+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_clean_scrub_stamp":"2024-09-12T03:19:23.835989+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:55:17.955383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-12T03:19:26.568261+0000","last_change":"2024-09-12T03:19:25.576779+0000","last_active":"2024-09-12T03:19:26.568261+0000","last_peered":"2024-09-12T03:19:26.568261+0000","last_clean":"2024-09-12T03:19:26.568261+0000","last_became_active":"2024-09-12T03:19:25.576482+0000","last_became_peered":"2024-09-12T03:19:25.576482+0000","last_unstale":"2024-09-12T03:19:26.568261+0000","last_undegraded":"2024-09-12T03:19:26.568261+0000","last_fullsized":"2024-09-12T03:19:26.568261+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:36:00.532243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"55'1","reported_seq":15,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-12T03:19:26.570081+0000","last_change":"2024-09-12T03:19:24.856981+0000","last_active":"2024-09-12T03:19:26.570081+0000","last_peered":"2024-09-12T03:19:26.570081+0000","last_clean":"2024-09-12T03:19:26.570081+0000","last_became_active":"2024-09-12T03:19:24.856607+0000","last_became_peered":"2024-09-12T03:19:24.856607+0000","last_unstale":"2024-09-12T03:19:26.570081+0000","last_undegraded":"2024-09-12T03:19:26.570081+0000","last_fullsized":"2024-09-12T03:19:26.570081+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_clean_scrub_stamp":"2024-09-12T03:19:23.835989+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:32:58.661071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-12T03:19:26.568522+0000","last_change":"2024-09-12T03:19:25.577980+0000","last_active":"2024-09-12T03:19:26.568522+0000","last_peered":"2024-09-12T03:19:26.568522+0000","last_clean":"2024-09-12T03:19:26.568522+0000","last_became_active":"2024-09-12T03:19:25.577717+0000","last_became_peered":"2024-09-12T03:19:25.577717+0000","last_unstale":"2024-09-12T03:19:26.568522+0000","last_undegraded":"2024-09-12T03:19:26.568522+0000","last_fullsized":"2024-09-12T03:19:26.568522+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:25:09.470946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"55'1","reported_seq":15,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-12T03:19:27.850627+0000","last_change":"2024-09-12T03:19:24.861305+0000","last_active":"2024-09-12T03:19:27.850627+0000","last_peered":"2024-09-12T03:19:27.850627+0000","last_clean":"2024-09-12T03:19:27.850627+0000","last_became_active":"2024-09-12T03:19:24.861021+0000","last_became_peered":"2024-09-12T03:19:24.861021+0000","last_unstale":"2024-09-12T03:19:27.850627+0000","last_undegraded":"2024-09-12T03:19:27.850627+0000","last_fullsized":"2024-09-12T03:19:27.850627+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_clean_scrub_stamp":"2024-09-12T03:19:23.835989+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:04:27.775435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-12T03:19:27.854225+0000","last_change":"2024-09-12T03:19:25.577969+0000","last_active":"2024-09-12T03:19:27.854225+0000","last_peered":"2024-09-12T03:19:27.854225+0000","last_clean":"2024-09-12T03:19:27.854225+0000","last_became_active":"2024-09-12T03:19:25.577648+0000","last_became_peered":"2024-09-12T03:19:25.577648+0000","last_unstale":"2024-09-12T03:19:27.854225+0000","last_undegraded":"2024-09-12T03:19:27.854225+0000","last_fullsized":"2024-09-12T03:19:27.854225+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:00:46.783593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":14,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-12T03:19:28.006610+0000","last_change":"2024-09-12T03:19:24.850360+0000","last_active":"2024-09-12T03:19:28.006610+0000","last_peered":"2024-09-12T03:19:28.006610+0000","last_clean":"2024-09-12T03:19:28.006610+0000","last_became_active":"2024-09-12T03:19:24.850027+0000","last_became_peered":"2024-09-12T03:19:24.850027+0000","last_unstale":"2024-09-12T03:19:28.006610+0000","last_undegraded":"2024-09-12T03:19:28.006610+0000","last_fullsized":"2024-09-12T03:19:28.006610+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_clean_scrub_stamp":"2024-09-12T03:19:23.835989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:31:30.906343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-12T03:19:26.568402+0000","last_change":"2024-09-12T03:19:25.578243+0000","last_active":"2024-09-12T03:19:26.568402+0000","last_peered":"2024-09-12T03:19:26.568402+0000","last_clean":"2024-09-12T03:19:26.568402+0000","last_became_active":"2024-09-12T03:19:25.577962+0000","last_became_peered":"2024-09-12T03:19:25.577962+0000","last_unstale":"2024-09-12T03:19:26.568402+0000","last_undegraded":"2024-09-12T03:19:26.568402+0000","last_fullsized":"2024-09-12T03:19:26.568402+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:11:33.930744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":14,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-12T03:19:26.569127+0000","last_change":"2024-09-12T03:19:24.863770+0000","last_active":"2024-09-12T03:19:26.569127+0000","last_peered":"2024-09-12T03:19:26.569127+0000","last_clean":"2024-09-12T03:19:26.569127+0000","last_became_active":"2024-09-12T03:19:24.863471+0000","last_became_peered":"2024-09-12T03:19:24.863471+0000","last_unstale":"2024-09-12T03:19:26.569127+0000","last_undegraded":"2024-09-12T03:19:26.569127+0000","last_fullsized":"2024-09-12T03:19:26.569127+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_clean_scrub_stamp":"2024-09-12T03:19:23.835989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:31:29.254101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-12T03:19:26.570427+0000","last_change":"2024-09-12T03:19:25.574294+0000","last_active":"2024-09-12T03:19:26.570427+0000","last_peered":"2024-09-12T03:19:26.570427+0000","last_clean":"2024-09-12T03:19:26.570427+0000","last_became_active":"2024-09-12T03:19:25.573875+0000","last_became_peered":"2024-09-12T03:19:25.573875+0000","last_unstale":"2024-09-12T03:19:26.570427+0000","last_undegraded":"2024-09-12T03:19:26.570427+0000","last_fullsized":"2024-09-12T03:19:26.570427+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:52:22.817110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":14,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-12T03:19:26.570255+0000","last_change":"2024-09-12T03:19:24.870526+0000","last_active":"2024-09-12T03:19:26.570255+0000","last_peered":"2024-09-12T03:19:26.570255+0000","last_clean":"2024-09-12T03:19:26.570255+0000","last_became_active":"2024-09-12T03:19:24.870226+0000","last_became_peered":"2024-09-12T03:19:24.870226+0000","last_unstale":"2024-09-12T03:19:26.570255+0000","last_undegraded":"2024-09-12T03:19:26.570255+0000","last_fullsized":"2024-09-12T03:19:26.570255+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_clean_scrub_stamp":"2024-09-12T03:19:23.835989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:52:21.815077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-12T03:19:26.570279+0000","last_change":"2024-09-12T03:19:25.586558+0000","last_active":"2024-09-12T03:19:26.570279+0000","last_peered":"2024-09-12T03:19:26.570279+0000","last_clean":"2024-09-12T03:19:26.570279+0000","last_became_active":"2024-09-12T03:19:25.586154+0000","last_became_peered":"2024-09-12T03:19:25.586154+0000","last_unstale":"2024-09-12T03:19:26.570279+0000","last_undegraded":"2024-09-12T03:19:26.570279+0000","last_fullsized":"2024-09-12T03:19:26.570279+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:38:29.897420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":14,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-12T03:19:26.568760+0000","last_change":"2024-09-12T03:19:24.862733+0000","last_active":"2024-09-12T03:19:26.568760+0000","last_peered":"2024-09-12T03:19:26.568760+0000","last_clean":"2024-09-12T03:19:26.568760+0000","last_became_active":"2024-09-12T03:19:24.862164+0000","last_became_peered":"2024-09-12T03:19:24.862164+0000","last_unstale":"2024-09-12T03:19:26.568760+0000","last_undegraded":"2024-09-12T03:19:26.568760+0000","last_fullsized":"2024-09-12T03:19:26.568760+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_clean_scrub_stamp":"2024-09-12T03:19:23.835989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:54:18.627329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-12T03:19:28.206694+0000","last_change":"2024-09-12T03:19:25.569342+0000","last_active":"2024-09-12T03:19:28.206694+0000","last_peered":"2024-09-12T03:19:28.206694+0000","last_clean":"2024-09-12T03:19:28.206694+0000","last_became_active":"2024-09-12T03:19:25.568498+0000","last_became_peered":"2024-09-12T03:19:25.568498+0000","last_unstale":"2024-09-12T03:19:28.206694+0000","last_undegraded":"2024-09-12T03:19:28.206694+0000","last_fullsized":"2024-09-12T03:19:28.206694+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:50:08.796307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"55'3","reported_seq":17,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-12T03:19:26.570389+0000","last_change":"2024-09-12T03:19:24.859732+0000","last_active":"2024-09-12T03:19:26.570389+0000","last_peered":"2024-09-12T03:19:26.570389+0000","last_clean":"2024-09-12T03:19:26.570389+0000","last_became_active":"2024-09-12T03:19:24.859452+0000","last_became_peered":"2024-09-12T03:19:24.859452+0000","last_unstale":"2024-09-12T03:19:26.570389+0000","last_undegraded":"2024-09-12T03:19:26.570389+0000","last_fullsized":"2024-09-12T03:19:26.570389+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_clean_scrub_stamp":"2024-09-12T03:19:23.835989+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:56:28.082698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-12T03:19:26.569344+0000","last_change":"2024-09-12T03:19:25.586524+0000","last_active":"2024-09-12T03:19:26.569344+0000","last_peered":"2024-09-12T03:19:26.569344+0000","last_clean":"2024-09-12T03:19:26.569344+0000","last_became_active":"2024-09-12T03:19:25.586168+0000","last_became_peered":"2024-09-12T03:19:25.586168+0000","last_unstale":"2024-09-12T03:19:26.569344+0000","last_undegraded":"2024-09-12T03:19:26.569344+0000","last_fullsized":"2024-09-12T03:19:26.569344+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:22:54.210265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"55'6","reported_seq":20,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-12T03:19:26.570462+0000","last_change":"2024-09-12T03:19:24.861967+0000","last_active":"2024-09-12T03:19:26.570462+0000","last_peered":"2024-09-12T03:19:26.570462+0000","last_clean":"2024-09-12T03:19:26.570462+0000","last_became_active":"2024-09-12T03:19:24.861645+0000","last_became_peered":"2024-09-12T03:19:24.861645+0000","last_unstale":"2024-09-12T03:19:26.570462+0000","last_undegraded":"2024-09-12T03:19:26.570462+0000","last_fullsized":"2024-09-12T03:19:26.570462+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_clean_scrub_stamp":"2024-09-12T03:19:23.835989+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:29:10.259651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-12T03:19:28.006655+0000","last_change":"2024-09-12T03:19:25.576722+0000","last_active":"2024-09-12T03:19:28.006655+0000","last_peered":"2024-09-12T03:19:28.006655+0000","last_clean":"2024-09-12T03:19:28.006655+0000","last_became_active":"2024-09-12T03:19:25.576437+0000","last_became_peered":"2024-09-12T03:19:25.576437+0000","last_unstale":"2024-09-12T03:19:28.006655+0000","last_undegraded":"2024-09-12T03:19:28.006655+0000","last_fullsized":"2024-09-12T03:19:28.006655+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:41:05.248563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"55'2","reported_seq":16,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-12T03:19:26.568637+0000","last_change":"2024-09-12T03:19:24.863067+0000","last_active":"2024-09-12T03:19:26.568637+0000","last_peered":"2024-09-12T03:19:26.568637+0000","last_clean":"2024-09-12T03:19:26.568637+0000","last_became_active":"2024-09-12T03:19:24.862471+0000","last_became_peered":"2024-09-12T03:19:24.862471+0000","last_unstale":"2024-09-12T03:19:26.568637+0000","last_undegraded":"2024-09-12T03:19:26.568637+0000","last_fullsized":"2024-09-12T03:19:26.568637+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_clean_scrub_stamp":"2024-09-12T03:19:23.835989+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:22:53.208233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-12T03:19:26.570597+0000","last_change":"2024-09-12T03:19:25.576709+0000","last_active":"2024-09-12T03:19:26.570597+0000","last_peered":"2024-09-12T03:19:26.570597+0000","last_clean":"2024-09-12T03:19:26.570597+0000","last_became_active":"2024-09-12T03:19:25.576410+0000","last_became_peered":"2024-09-12T03:19:25.576410+0000","last_unstale":"2024-09-12T03:19:26.570597+0000","last_undegraded":"2024-09-12T03:19:26.570597+0000","last_fullsized":"2024-09-12T03:19:26.570597+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:44:04.569606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":14,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-12T03:19:27.853992+0000","last_change":"2024-09-12T03:19:24.859981+0000","last_active":"2024-09-12T03:19:27.853992+0000","last_peered":"2024-09-12T03:19:27.853992+0000","last_clean":"2024-09-12T03:19:27.853992+0000","last_became_active":"2024-09-12T03:19:24.859679+0000","last_became_peered":"2024-09-12T03:19:24.859679+0000","last_unstale":"2024-09-12T03:19:27.853992+0000","last_undegraded":"2024-09-12T03:19:27.853992+0000","last_fullsized":"2024-09-12T03:19:27.853992+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_clean_scrub_stamp":"2024-09-12T03:19:23.835989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:32:15.030338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-12T03:19:26.569091+0000","last_change":"2024-09-12T03:19:25.579614+0000","last_active":"2024-09-12T03:19:26.569091+0000","last_peered":"2024-09-12T03:19:26.569091+0000","last_clean":"2024-09-12T03:19:26.569091+0000","last_became_active":"2024-09-12T03:19:25.579280+0000","last_became_peered":"2024-09-12T03:19:25.579280+0000","last_unstale":"2024-09-12T03:19:26.569091+0000","last_undegraded":"2024-09-12T03:19:26.569091+0000","last_fullsized":"2024-09-12T03:19:26.569091+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:00:43.742359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":14,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-12T03:19:28.006644+0000","last_change":"2024-09-12T03:19:24.855795+0000","last_active":"2024-09-12T03:19:28.006644+0000","last_peered":"2024-09-12T03:19:28.006644+0000","last_clean":"2024-09-12T03:19:28.006644+0000","last_became_active":"2024-09-12T03:19:24.855377+0000","last_became_peered":"2024-09-12T03:19:24.855377+0000","last_unstale":"2024-09-12T03:19:28.006644+0000","last_undegraded":"2024-09-12T03:19:28.006644+0000","last_fullsized":"2024-09-12T03:19:28.006644+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_clean_scrub_stamp":"2024-09-12T03:19:23.835989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:17:08.997193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-12T03:19:26.569606+0000","last_change":"2024-09-12T03:19:25.583346+0000","last_active":"2024-09-12T03:19:26.569606+0000","last_peered":"2024-09-12T03:19:26.569606+0000","last_clean":"2024-09-12T03:19:26.569606+0000","last_became_active":"2024-09-12T03:19:25.583046+0000","last_became_peered":"2024-09-12T03:19:25.583046+0000","last_unstale":"2024-09-12T03:19:26.569606+0000","last_undegraded":"2024-09-12T03:19:26.569606+0000","last_fullsized":"2024-09-12T03:19:26.569606+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:34:59.628397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"55'8","reported_seq":22,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-12T03:19:26.570683+0000","last_change":"2024-09-12T03:19:24.859883+0000","last_active":"2024-09-12T03:19:26.570683+0000","last_peered":"2024-09-12T03:19:26.570683+0000","last_clean":"2024-09-12T03:19:26.570683+0000","last_became_active":"2024-09-12T03:19:24.859566+0000","last_became_peered":"2024-09-12T03:19:24.859566+0000","last_unstale":"2024-09-12T03:19:26.570683+0000","last_undegraded":"2024-09-12T03:19:26.570683+0000","last_fullsized":"2024-09-12T03:19:26.570683+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_clean_scrub_stamp":"2024-09-12T03:19:23.835989+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:22:23.745090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-12T03:19:26.570675+0000","last_change":"2024-09-12T03:19:25.584759+0000","last_active":"2024-09-12T03:19:26.570675+0000","last_peered":"2024-09-12T03:19:26.570675+0000","last_clean":"2024-09-12T03:19:26.570675+0000","last_became_active":"2024-09-12T03:19:25.584378+0000","last_became_peered":"2024-09-12T03:19:25.584378+0000","last_unstale":"2024-09-12T03:19:26.570675+0000","last_undegraded":"2024-09-12T03:19:26.570675+0000","last_fullsized":"2024-09-12T03:19:26.570675+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:13:41.572442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"55'1","reported_seq":15,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-12T03:19:26.568792+0000","last_change":"2024-09-12T03:19:24.861891+0000","last_active":"2024-09-12T03:19:26.568792+0000","last_peered":"2024-09-12T03:19:26.568792+0000","last_clean":"2024-09-12T03:19:26.568792+0000","last_became_active":"2024-09-12T03:19:24.861524+0000","last_became_peered":"2024-09-12T03:19:24.861524+0000","last_unstale":"2024-09-12T03:19:26.568792+0000","last_undegraded":"2024-09-12T03:19:26.568792+0000","last_fullsized":"2024-09-12T03:19:26.568792+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_clean_scrub_stamp":"2024-09-12T03:19:23.835989+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:41:59.754398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-12T03:19:28.007210+0000","last_change":"2024-09-12T03:19:25.584053+0000","last_active":"2024-09-12T03:19:28.007210+0000","last_peered":"2024-09-12T03:19:28.007210+0000","last_clean":"2024-09-12T03:19:28.007210+0000","last_became_active":"2024-09-12T03:19:25.583789+0000","last_became_peered":"2024-09-12T03:19:25.583789+0000","last_unstale":"2024-09-12T03:19:28.007210+0000","last_undegraded":"2024-09-12T03:19:28.007210+0000","last_fullsized":"2024-09-12T03:19:28.007210+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:18:41.706585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"55'2","reported_seq":16,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-12T03:19:27.851190+0000","last_change":"2024-09-12T03:19:24.865925+0000","last_active":"2024-09-12T03:19:27.851190+0000","last_peered":"2024-09-12T03:19:27.851190+0000","last_clean":"2024-09-12T03:19:27.851190+0000","last_became_active":"2024-09-12T03:19:24.865689+0000","last_became_peered":"2024-09-12T03:19:24.865689+0000","last_unstale":"2024-09-12T03:19:27.851190+0000","last_undegraded":"2024-09-12T03:19:27.851190+0000","last_fullsized":"2024-09-12T03:19:27.851190+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_clean_scrub_stamp":"2024-09-12T03:19:23.835989+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:04:37.233100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-12T03:19:26.568939+0000","last_change":"2024-09-12T03:19:25.571563+0000","last_active":"2024-09-12T03:19:26.568939+0000","last_peered":"2024-09-12T03:19:26.568939+0000","last_clean":"2024-09-12T03:19:26.568939+0000","last_became_active":"2024-09-12T03:19:25.571285+0000","last_became_peered":"2024-09-12T03:19:25.571285+0000","last_unstale":"2024-09-12T03:19:26.568939+0000","last_undegraded":"2024-09-12T03:19:26.568939+0000","last_fullsized":"2024-09-12T03:19:26.568939+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:51:26.533695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"55'1","reported_seq":15,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-12T03:19:28.007058+0000","last_change":"2024-09-12T03:19:24.855690+0000","last_active":"2024-09-12T03:19:28.007058+0000","last_peered":"2024-09-12T03:19:28.007058+0000","last_clean":"2024-09-12T03:19:28.007058+0000","last_became_active":"2024-09-12T03:19:24.855284+0000","last_became_peered":"2024-09-12T03:19:24.855284+0000","last_unstale":"2024-09-12T03:19:28.007058+0000","last_undegraded":"2024-09-12T03:19:28.007058+0000","last_fullsized":"2024-09-12T03:19:28.007058+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_clean_scrub_stamp":"2024-09-12T03:19:23.835989+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:00:10.732520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-12T03:19:26.569989+0000","last_change":"2024-09-12T03:19:25.586655+0000","last_active":"2024-09-12T03:19:26.569989+0000","last_peered":"2024-09-12T03:19:26.569989+0000","last_clean":"2024-09-12T03:19:26.569989+0000","last_became_active":"2024-09-12T03:19:25.586242+0000","last_became_peered":"2024-09-12T03:19:25.586242+0000","last_unstale":"2024-09-12T03:19:26.569989+0000","last_undegraded":"2024-09-12T03:19:26.569989+0000","last_fullsized":"2024-09-12T03:19:26.569989+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:16:22.378881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":14,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-12T03:19:27.850794+0000","last_change":"2024-09-12T03:19:24.864872+0000","last_active":"2024-09-12T03:19:27.850794+0000","last_peered":"2024-09-12T03:19:27.850794+0000","last_clean":"2024-09-12T03:19:27.850794+0000","last_became_active":"2024-09-12T03:19:24.864506+0000","last_became_peered":"2024-09-12T03:19:24.864506+0000","last_unstale":"2024-09-12T03:19:27.850794+0000","last_undegraded":"2024-09-12T03:19:27.850794+0000","last_fullsized":"2024-09-12T03:19:27.850794+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_clean_scrub_stamp":"2024-09-12T03:19:23.835989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:50:19.592853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-12T03:19:28.006689+0000","last_change":"2024-09-12T03:19:25.582508+0000","last_active":"2024-09-12T03:19:28.006689+0000","last_peered":"2024-09-12T03:19:28.006689+0000","last_clean":"2024-09-12T03:19:28.006689+0000","last_became_active":"2024-09-12T03:19:25.582163+0000","last_became_peered":"2024-09-12T03:19:25.582163+0000","last_unstale":"2024-09-12T03:19:28.006689+0000","last_undegraded":"2024-09-12T03:19:28.006689+0000","last_fullsized":"2024-09-12T03:19:28.006689+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:40:57.837089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"55'2","reported_seq":16,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-12T03:19:26.568792+0000","last_change":"2024-09-12T03:19:24.862743+0000","last_active":"2024-09-12T03:19:26.568792+0000","last_peered":"2024-09-12T03:19:26.568792+0000","last_clean":"2024-09-12T03:19:26.568792+0000","last_became_active":"2024-09-12T03:19:24.862186+0000","last_became_peered":"2024-09-12T03:19:24.862186+0000","last_unstale":"2024-09-12T03:19:26.568792+0000","last_undegraded":"2024-09-12T03:19:26.568792+0000","last_fullsized":"2024-09-12T03:19:26.568792+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_clean_scrub_stamp":"2024-09-12T03:19:23.835989+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:59:02.826944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-12T03:19:27.850728+0000","last_change":"2024-09-12T03:19:25.575390+0000","last_active":"2024-09-12T03:19:27.850728+0000","last_peered":"2024-09-12T03:19:27.850728+0000","last_clean":"2024-09-12T03:19:27.850728+0000","last_became_active":"2024-09-12T03:19:25.574143+0000","last_became_peered":"2024-09-12T03:19:25.574143+0000","last_unstale":"2024-09-12T03:19:27.850728+0000","last_undegraded":"2024-09-12T03:19:27.850728+0000","last_fullsized":"2024-09-12T03:19:27.850728+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:25:23.473943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":14,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-12T03:19:28.206776+0000","last_change":"2024-09-12T03:19:24.857052+0000","last_active":"2024-09-12T03:19:28.206776+0000","last_peered":"2024-09-12T03:19:28.206776+0000","last_clean":"2024-09-12T03:19:28.206776+0000","last_became_active":"2024-09-12T03:19:24.856771+0000","last_became_peered":"2024-09-12T03:19:24.856771+0000","last_unstale":"2024-09-12T03:19:28.206776+0000","last_undegraded":"2024-09-12T03:19:28.206776+0000","last_fullsized":"2024-09-12T03:19:28.206776+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_clean_scrub_stamp":"2024-09-12T03:19:23.835989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:41:18.963082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-12T03:19:27.850360+0000","last_change":"2024-09-12T03:19:25.578776+0000","last_active":"2024-09-12T03:19:27.850360+0000","last_peered":"2024-09-12T03:19:27.850360+0000","last_clean":"2024-09-12T03:19:27.850360+0000","last_became_active":"2024-09-12T03:19:25.578465+0000","last_became_peered":"2024-09-12T03:19:25.578465+0000","last_unstale":"2024-09-12T03:19:27.850360+0000","last_undegraded":"2024-09-12T03:19:27.850360+0000","last_fullsized":"2024-09-12T03:19:27.850360+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:43:55.912919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":14,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-12T03:19:28.006619+0000","last_change":"2024-09-12T03:19:24.864514+0000","last_active":"2024-09-12T03:19:28.006619+0000","last_peered":"2024-09-12T03:19:28.006619+0000","last_clean":"2024-09-12T03:19:28.006619+0000","last_became_active":"2024-09-12T03:19:24.864206+0000","last_became_peered":"2024-09-12T03:19:24.864206+0000","last_unstale":"2024-09-12T03:19:28.006619+0000","last_undegraded":"2024-09-12T03:19:28.006619+0000","last_fullsized":"2024-09-12T03:19:28.006619+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_clean_scrub_stamp":"2024-09-12T03:19:23.835989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:30:58.826988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-12T03:19:28.007097+0000","last_change":"2024-09-12T03:19:25.577966+0000","last_active":"2024-09-12T03:19:28.007097+0000","last_peered":"2024-09-12T03:19:28.007097+0000","last_clean":"2024-09-12T03:19:28.007097+0000","last_became_active":"2024-09-12T03:19:25.577677+0000","last_became_peered":"2024-09-12T03:19:25.577677+0000","last_unstale":"2024-09-12T03:19:28.007097+0000","last_undegraded":"2024-09-12T03:19:28.007097+0000","last_fullsized":"2024-09-12T03:19:28.007097+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:47:45.205506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"55'2","reported_seq":16,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-12T03:19:28.207157+0000","last_change":"2024-09-12T03:19:24.863777+0000","last_active":"2024-09-12T03:19:28.207157+0000","last_peered":"2024-09-12T03:19:28.207157+0000","last_clean":"2024-09-12T03:19:28.207157+0000","last_became_active":"2024-09-12T03:19:24.863506+0000","last_became_peered":"2024-09-12T03:19:24.863506+0000","last_unstale":"2024-09-12T03:19:28.207157+0000","last_undegraded":"2024-09-12T03:19:28.207157+0000","last_fullsized":"2024-09-12T03:19:28.207157+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_clean_scrub_stamp":"2024-09-12T03:19:23.835989+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:58:26.747684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-12T03:19:28.007283+0000","last_change":"2024-09-12T03:19:25.579830+0000","last_active":"2024-09-12T03:19:28.007283+0000","last_peered":"2024-09-12T03:19:28.007283+0000","last_clean":"2024-09-12T03:19:28.007283+0000","last_became_active":"2024-09-12T03:19:25.579551+0000","last_became_peered":"2024-09-12T03:19:25.579551+0000","last_unstale":"2024-09-12T03:19:28.007283+0000","last_undegraded":"2024-09-12T03:19:28.007283+0000","last_fullsized":"2024-09-12T03:19:28.007283+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:03:32.419232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"56'1","reported_seq":15,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-12T03:19:27.851354+0000","last_change":"2024-09-12T03:19:24.865383+0000","last_active":"2024-09-12T03:19:27.851354+0000","last_peered":"2024-09-12T03:19:27.851354+0000","last_clean":"2024-09-12T03:19:27.851354+0000","last_became_active":"2024-09-12T03:19:24.865031+0000","last_became_peered":"2024-09-12T03:19:24.865031+0000","last_unstale":"2024-09-12T03:19:27.851354+0000","last_undegraded":"2024-09-12T03:19:27.851354+0000","last_fullsized":"2024-09-12T03:19:27.851354+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_clean_scrub_stamp":"2024-09-12T03:19:23.835989+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:25:17.547373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-12T03:19:28.206841+0000","last_change":"2024-09-12T03:19:25.569171+0000","last_active":"2024-09-12T03:19:28.206841+0000","last_peered":"2024-09-12T03:19:28.206841+0000","last_clean":"2024-09-12T03:19:28.206841+0000","last_became_active":"2024-09-12T03:19:25.568528+0000","last_became_peered":"2024-09-12T03:19:25.568528+0000","last_unstale":"2024-09-12T03:19:28.206841+0000","last_undegraded":"2024-09-12T03:19:28.206841+0000","last_fullsized":"2024-09-12T03:19:28.206841+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:48:15.539565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":14,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-12T03:19:27.854065+0000","last_change":"2024-09-12T03:19:24.857501+0000","last_active":"2024-09-12T03:19:27.854065+0000","last_peered":"2024-09-12T03:19:27.854065+0000","last_clean":"2024-09-12T03:19:27.854065+0000","last_became_active":"2024-09-12T03:19:24.857129+0000","last_became_peered":"2024-09-12T03:19:24.857129+0000","last_unstale":"2024-09-12T03:19:27.854065+0000","last_undegraded":"2024-09-12T03:19:27.854065+0000","last_fullsized":"2024-09-12T03:19:27.854065+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_clean_scrub_stamp":"2024-09-12T03:19:23.835989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:33:41.476236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-12T03:19:27.851032+0000","last_change":"2024-09-12T03:19:25.572553+0000","last_active":"2024-09-12T03:19:27.851032+0000","last_peered":"2024-09-12T03:19:27.851032+0000","last_clean":"2024-09-12T03:19:27.851032+0000","last_became_active":"2024-09-12T03:19:25.572213+0000","last_became_peered":"2024-09-12T03:19:25.572213+0000","last_unstale":"2024-09-12T03:19:27.851032+0000","last_undegraded":"2024-09-12T03:19:27.851032+0000","last_fullsized":"2024-09-12T03:19:27.851032+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:49:44.982182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":14,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-12T03:19:28.007197+0000","last_change":"2024-09-12T03:19:24.852831+0000","last_active":"2024-09-12T03:19:28.007197+0000","last_peered":"2024-09-12T03:19:28.007197+0000","last_clean":"2024-09-12T03:19:28.007197+0000","last_became_active":"2024-09-12T03:19:24.851427+0000","last_became_peered":"2024-09-12T03:19:24.851427+0000","last_unstale":"2024-09-12T03:19:28.007197+0000","last_undegraded":"2024-09-12T03:19:28.007197+0000","last_fullsized":"2024-09-12T03:19:28.007197+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_clean_scrub_stamp":"2024-09-12T03:19:23.835989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:21:51.999147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-12T03:19:26.568449+0000","last_change":"2024-09-12T03:19:25.576513+0000","last_active":"2024-09-12T03:19:26.568449+0000","last_peered":"2024-09-12T03:19:26.568449+0000","last_clean":"2024-09-12T03:19:26.568449+0000","last_became_active":"2024-09-12T03:19:25.576249+0000","last_became_peered":"2024-09-12T03:19:25.576249+0000","last_unstale":"2024-09-12T03:19:26.568449+0000","last_undegraded":"2024-09-12T03:19:26.568449+0000","last_fullsized":"2024-09-12T03:19:26.568449+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:53:00.492150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":14,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-12T03:19:28.206547+0000","last_change":"2024-09-12T03:19:24.849318+0000","last_active":"2024-09-12T03:19:28.206547+0000","last_peered":"2024-09-12T03:19:28.206547+0000","last_clean":"2024-09-12T03:19:28.206547+0000","last_became_active":"2024-09-12T03:19:24.848981+0000","last_became_peered":"2024-09-12T03:19:24.848981+0000","last_unstale":"2024-09-12T03:19:28.206547+0000","last_undegraded":"2024-09-12T03:19:28.206547+0000","last_fullsized":"2024-09-12T03:19:28.206547+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_clean_scrub_stamp":"2024-09-12T03:19:23.835989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:28:52.361884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-12T03:19:26.570846+0000","last_change":"2024-09-12T03:19:25.583374+0000","last_active":"2024-09-12T03:19:26.570846+0000","last_peered":"2024-09-12T03:19:26.570846+0000","last_clean":"2024-09-12T03:19:26.570846+0000","last_became_active":"2024-09-12T03:19:25.583066+0000","last_became_peered":"2024-09-12T03:19:25.583066+0000","last_unstale":"2024-09-12T03:19:26.570846+0000","last_undegraded":"2024-09-12T03:19:26.570846+0000","last_fullsized":"2024-09-12T03:19:26.570846+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:37:42.235320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":14,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-12T03:19:26.569729+0000","last_change":"2024-09-12T03:19:24.861998+0000","last_active":"2024-09-12T03:19:26.569729+0000","last_peered":"2024-09-12T03:19:26.569729+0000","last_clean":"2024-09-12T03:19:26.569729+0000","last_became_active":"2024-09-12T03:19:24.861694+0000","last_became_peered":"2024-09-12T03:19:24.861694+0000","last_unstale":"2024-09-12T03:19:26.569729+0000","last_undegraded":"2024-09-12T03:19:26.569729+0000","last_fullsized":"2024-09-12T03:19:26.569729+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_clean_scrub_stamp":"2024-09-12T03:19:23.835989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:31:40.161867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-12T03:19:28.206554+0000","last_change":"2024-09-12T03:19:25.575272+0000","last_active":"2024-09-12T03:19:28.206554+0000","last_peered":"2024-09-12T03:19:28.206554+0000","last_clean":"2024-09-12T03:19:28.206554+0000","last_became_active":"2024-09-12T03:19:25.575018+0000","last_became_peered":"2024-09-12T03:19:25.575018+0000","last_unstale":"2024-09-12T03:19:28.206554+0000","last_undegraded":"2024-09-12T03:19:28.206554+0000","last_fullsized":"2024-09-12T03:19:28.206554+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:38:36.823412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":14,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-12T03:19:28.207049+0000","last_change":"2024-09-12T03:19:24.860398+0000","last_active":"2024-09-12T03:19:28.207049+0000","last_peered":"2024-09-12T03:19:28.207049+0000","last_clean":"2024-09-12T03:19:28.207049+0000","last_became_active":"2024-09-12T03:19:24.860093+0000","last_became_peered":"2024-09-12T03:19:24.860093+0000","last_unstale":"2024-09-12T03:19:28.207049+0000","last_undegraded":"2024-09-12T03:19:28.207049+0000","last_fullsized":"2024-09-12T03:19:28.207049+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_clean_scrub_stamp":"2024-09-12T03:19:23.835989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:31:26.316452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-12T03:19:26.568565+0000","last_change":"2024-09-12T03:19:25.577033+0000","last_active":"2024-09-12T03:19:26.568565+0000","last_peered":"2024-09-12T03:19:26.568565+0000","last_clean":"2024-09-12T03:19:26.568565+0000","last_became_active":"2024-09-12T03:19:25.575361+0000","last_became_peered":"2024-09-12T03:19:25.575361+0000","last_unstale":"2024-09-12T03:19:26.568565+0000","last_undegraded":"2024-09-12T03:19:26.568565+0000","last_fullsized":"2024-09-12T03:19:26.568565+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:49:39.227841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":14,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-12T03:19:26.568443+0000","last_change":"2024-09-12T03:19:24.858756+0000","last_active":"2024-09-12T03:19:26.568443+0000","last_peered":"2024-09-12T03:19:26.568443+0000","last_clean":"2024-09-12T03:19:26.568443+0000","last_became_active":"2024-09-12T03:19:24.858474+0000","last_became_peered":"2024-09-12T03:19:24.858474+0000","last_unstale":"2024-09-12T03:19:26.568443+0000","last_undegraded":"2024-09-12T03:19:26.568443+0000","last_fullsized":"2024-09-12T03:19:26.568443+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_clean_scrub_stamp":"2024-09-12T03:19:23.835989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:04:05.050990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-12T03:19:26.568261+0000","last_change":"2024-09-12T03:19:25.578617+0000","last_active":"2024-09-12T03:19:26.568261+0000","last_peered":"2024-09-12T03:19:26.568261+0000","last_clean":"2024-09-12T03:19:26.568261+0000","last_became_active":"2024-09-12T03:19:25.578346+0000","last_became_peered":"2024-09-12T03:19:25.578346+0000","last_unstale":"2024-09-12T03:19:26.568261+0000","last_undegraded":"2024-09-12T03:19:26.568261+0000","last_fullsized":"2024-09-12T03:19:26.568261+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:24.838022+0000","last_clean_scrub_stamp":"2024-09-12T03:19:24.838022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:26:49.228871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":14,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-12T03:19:27.854125+0000","last_change":"2024-09-12T03:19:24.858024+0000","last_active":"2024-09-12T03:19:27.854125+0000","last_peered":"2024-09-12T03:19:27.854125+0000","last_clean":"2024-09-12T03:19:27.854125+0000","last_became_active":"2024-09-12T03:19:24.857142+0000","last_became_peered":"2024-09-12T03:19:24.857142+0000","last_unstale":"2024-09-12T03:19:27.854125+0000","last_undegraded":"2024-09-12T03:19:27.854125+0000","last_fullsized":"2024-09-12T03:19:27.854125+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:23.835989+0000","last_clean_scrub_stamp":"2024-09-12T03:19:23.835989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:28:38.811682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":232,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-12T03:19:26.570756+0000","last_change":"2024-09-12T03:16:21.883446+0000","last_active":"2024-09-12T03:19:26.570756+0000","last_peered":"2024-09-12T03:19:26.570756+0000","last_clean":"2024-09-12T03:19:26.570756+0000","last_became_active":"2024-09-12T03:16:21.882207+0000","last_became_peered":"2024-09-12T03:16:21.882207+0000","last_unstale":"2024-09-12T03:19:26.570756+0000","last_undegraded":"2024-09-12T03:19:26.570756+0000","last_fullsized":"2024-09-12T03:19:26.570756+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:20.316426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:20.316426+0000","last_clean_scrub_stamp":"2024-09-12T03:16:20.316426+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:12:15.424320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542184,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32436,"kb_used_data":1240,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339404,"statfs":{"total":96636764160,"available":96603549696,"internally_reserved":0,"allocated":1269760,"data_stored":866511,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542186,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31840,"kb_used_data":652,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340000,"statfs":{"total":96636764160,"available":96604160000,"internally_reserved":0,"allocated":667648,"data_stored":275983,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542185,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36008,"kb_used_data":660,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335832,"statfs":{"total":96636764160,"available":96599891968,"internally_reserved":0,"allocated":675840,"data_stored":276635,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542184,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31860,"kb_used_data":664,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339980,"statfs":{"total":96636764160,"available":96604139520,"internally_reserved":0,"allocated":679936,"data_stored":277595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542185,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27696,"kb_used_data":660,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344144,"statfs":{"total":96636764160,"available":96608403456,"internally_reserved":0,"allocated":675840,"data_stored":276039,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542185,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27688,"kb_used_data":660,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344152,"statfs":{"total":96636764160,"available":96608411648,"internally_reserved":0,"allocated":675840,"data_stored":276039,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542186,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31844,"kb_used_data":656,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339996,"statfs":{"total":96636764160,"available":96604155904,"internally_reserved":0,"allocated":671744,"data_stored":276073,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542186,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28276,"kb_used_data":1248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343564,"statfs":{"total":96636764160,"available":96607809536,"internally_reserved":0,"allocated":1277952,"data_stored":868545,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T03:19:33.570 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-12T03:19:33.571 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-12T03:19:33.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:33.715+0000 7f18ff41e640 1 Processor -- start 2024-09-12T03:19:33.717 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:33.716+0000 7f18ff41e640 1 -- start start 2024-09-12T03:19:33.718 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:33.716+0000 7f18ff41e640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f18f8162040 0x7f18f80d1da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:33.718 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:33.716+0000 7f18ff41e640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f18f806dfd0 0x7f18f80b7a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:33.718 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:33.716+0000 7f18ff41e640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f18f80d2e30 con 0x7f18f80d3790 2024-09-12T03:19:33.718 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:33.716+0000 7f18ff41e640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f18f80b7f90 con 0x7f18f806dfd0 2024-09-12T03:19:33.718 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:33.716+0000 7f18ff41e640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f18f8162420 con 0x7f18f8162040 2024-09-12T03:19:33.718 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:33.717+0000 7f18fd9bb640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f18f806dfd0 0x7f18f80b7a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:33.718 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:33.717+0000 7f18fc9b9640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f18f80d3790 0x7f18f806d8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:42404/0 (socket says 172.21.15.27:42404) 2024-09-12T03:19:33.718 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:33.717+0000 7f18fd9bb640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f18f806dfd0 0x7f18f80b7a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38394/0 (socket says 172.21.15.27:38394) 2024-09-12T03:19:33.718 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:33.717+0000 7f18fc9b9640 1 -- 172.21.15.27:0/1714116456 learned_addr learned my addr 172.21.15.27:0/1714116456 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:19:33.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:33.717+0000 7f18fd1ba640 1 --2- 172.21.15.27:0/1714116456 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f18f8162040 0x7f18f80d1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:33.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:33.717+0000 7f18fd9bb640 1 -- 172.21.15.27:0/1714116456 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f18f8162040 msgr2=0x7f18f80d1da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:33.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:33.717+0000 7f18fd9bb640 1 --2- 172.21.15.27:0/1714116456 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f18f8162040 0x7f18f80d1da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:33.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:33.717+0000 7f18fd9bb640 1 -- 172.21.15.27:0/1714116456 >> v1:172.21.15.27:6789/0 conn(0x7f18f80d3790 legacy=0x7f18f806d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:33.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:33.717+0000 7f18effff640 1 -- 172.21.15.27:0/1714116456 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4211976316 0 0) 0x7f18f80d2e30 con 0x7f18f80d3790 2024-09-12T03:19:33.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:33.717+0000 7f18fd9bb640 1 -- 172.21.15.27:0/1714116456 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18f80d30d0 con 0x7f18f806dfd0 2024-09-12T03:19:33.720 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:33.718+0000 7f18fd9bb640 1 --2- 172.21.15.27:0/1714116456 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f18f806dfd0 0x7f18f80b7a50 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f18f0009d90 tx=0x7f18f002a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=846a2ee79cb458c9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:33.720 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:33.718+0000 7f18effff640 1 -- 172.21.15.27:0/1714116456 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18f0007740 con 0x7f18f806dfd0 2024-09-12T03:19:33.720 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:33.718+0000 7f18effff640 1 -- 172.21.15.27:0/1714116456 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f18f00078e0 con 0x7f18f806dfd0 2024-09-12T03:19:33.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:33.718+0000 7f18effff640 1 -- 172.21.15.27:0/1714116456 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18f0007cb0 con 0x7f18f806dfd0 2024-09-12T03:19:33.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:33.718+0000 7f18ff41e640 1 -- 172.21.15.27:0/1714116456 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f18f806dfd0 msgr2=0x7f18f80b7a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:33.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:33.718+0000 7f18ff41e640 1 --2- 172.21.15.27:0/1714116456 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f18f806dfd0 0x7f18f80b7a50 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f18f0009d90 tx=0x7f18f002a660 comp rx=0 tx=0).stop 2024-09-12T03:19:33.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:33.718+0000 7f18ff41e640 1 -- 172.21.15.27:0/1714116456 shutdown_connections 2024-09-12T03:19:33.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:33.718+0000 7f18ff41e640 1 --2- 172.21.15.27:0/1714116456 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f18f806dfd0 0x7f18f80b7a50 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:33.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:33.718+0000 7f18ff41e640 1 --2- 172.21.15.27:0/1714116456 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f18f8162040 0x7f18f80d1da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:33.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:33.718+0000 7f18ff41e640 1 -- 172.21.15.27:0/1714116456 >> 172.21.15.27:0/1714116456 conn(0x7f18f80b6700 msgr2=0x7f18f80b6b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:33.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:33.718+0000 7f18ff41e640 1 -- 172.21.15.27:0/1714116456 shutdown_connections 2024-09-12T03:19:33.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:33.718+0000 7f18ff41e640 1 -- 172.21.15.27:0/1714116456 wait complete. 2024-09-12T03:19:33.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:33.720+0000 7f18ff41e640 1 Processor -- start 2024-09-12T03:19:33.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:33.720+0000 7f18ff41e640 1 -- start start 2024-09-12T03:19:33.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:33.720+0000 7f18ff41e640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f18f806dfd0 0x7f18f81f7560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:33.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:33.720+0000 7f18ff41e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f18f80d3790 0x7f18f815ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:33.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:33.720+0000 7f18ff41e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f18f8162040 0x7f18f815b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:33.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:33.720+0000 7f18ff41e640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f18f80d4530 con 0x7f18f80d3790 2024-09-12T03:19:33.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:33.720+0000 7f18ff41e640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f18f80cee00 con 0x7f18f8162040 2024-09-12T03:19:33.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:33.720+0000 7f18ff41e640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f18f80d4150 con 0x7f18f806dfd0 2024-09-12T03:19:33.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:33.720+0000 7f18fc9b9640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f18f80d3790 0x7f18f815ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:33.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:33.721+0000 7f18fc9b9640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f18f80d3790 0x7f18f815ac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38408/0 (socket says 172.21.15.27:38408) 2024-09-12T03:19:33.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:33.721+0000 7f18fc9b9640 1 -- 172.21.15.27:0/1560042170 learned_addr learned my addr 172.21.15.27:0/1560042170 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:19:33.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:33.721+0000 7f18fd1ba640 1 --2- 172.21.15.27:0/1560042170 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f18f806dfd0 0x7f18f81f7560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:33.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:33.721+0000 7f18fd9bb640 1 --2- 172.21.15.27:0/1560042170 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f18f8162040 0x7f18f815b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:33.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:33.721+0000 7f18fc9b9640 1 -- 172.21.15.27:0/1560042170 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f18f806dfd0 msgr2=0x7f18f81f7560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:33.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:33.721+0000 7f18fc9b9640 1 --2- 172.21.15.27:0/1560042170 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f18f806dfd0 0x7f18f81f7560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:33.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:33.721+0000 7f18fc9b9640 1 -- 172.21.15.27:0/1560042170 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f18f8162040 msgr2=0x7f18f815b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:33.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:33.721+0000 7f18fc9b9640 1 --2- 172.21.15.27:0/1560042170 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f18f8162040 0x7f18f815b180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:33.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:33.721+0000 7f18fc9b9640 1 -- 172.21.15.27:0/1560042170 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18f81f54f0 con 0x7f18f80d3790 2024-09-12T03:19:33.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:33.721+0000 7f18fd9bb640 1 --2- 172.21.15.27:0/1560042170 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f18f8162040 0x7f18f815b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:19:33.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:33.721+0000 7f18fc9b9640 1 --2- 172.21.15.27:0/1560042170 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f18f80d3790 0x7f18f815ac40 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f18e800c470 tx=0x7f18e800c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:33.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:33.721+0000 7f18edffb640 1 -- 172.21.15.27:0/1560042170 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18e8018070 con 0x7f18f80d3790 2024-09-12T03:19:33.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:33.721+0000 7f18edffb640 1 -- 172.21.15.27:0/1560042170 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f18e80074e0 con 0x7f18f80d3790 2024-09-12T03:19:33.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:33.721+0000 7f18edffb640 1 -- 172.21.15.27:0/1560042170 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18e8004a20 con 0x7f18f80d3790 2024-09-12T03:19:33.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:33.721+0000 7f18ff41e640 1 -- 172.21.15.27:0/1560042170 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18f80cba90 con 0x7f18f80d3790 2024-09-12T03:19:33.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:33.721+0000 7f18ff41e640 1 -- 172.21.15.27:0/1560042170 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18f80cc050 con 0x7f18f80d3790 2024-09-12T03:19:33.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:33.723+0000 7f18ff41e640 1 -- 172.21.15.27:0/1560042170 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18f80cc250 con 0x7f18f80d3790 2024-09-12T03:19:33.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:33.723+0000 7f18edffb640 1 -- 172.21.15.27:0/1560042170 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 9) ==== 90651+0+0 (secure 0 0 0) 0x7f18e8004bc0 con 0x7f18f80d3790 2024-09-12T03:19:33.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:33.723+0000 7f18edffb640 1 --2- 172.21.15.27:0/1560042170 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f18dc06bfe0 0x7f18dc06e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:33.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:33.724+0000 7f18edffb640 1 -- 172.21.15.27:0/1560042170 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5709+0+0 (secure 0 0 0) 0x7f18e808a440 con 0x7f18f80d3790 2024-09-12T03:19:33.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:33.724+0000 7f18fd1ba640 1 --2- 172.21.15.27:0/1560042170 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f18dc06bfe0 0x7f18dc06e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:33.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:33.724+0000 7f18fd1ba640 1 --2- 172.21.15.27:0/1560042170 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f18dc06bfe0 0x7f18dc06e4a0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f18f80cdc00 tx=0x7f18f4008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:33.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:33.728+0000 7f18edffb640 1 -- 172.21.15.27:0/1560042170 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f18f80cc250 con 0x7f18f80d3790 2024-09-12T03:19:33.968 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:33.967+0000 7f18ff41e640 1 -- 172.21.15.27:0/1560042170 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f18f80b7a50 con 0x7f18f80d3790 2024-09-12T03:19:33.970 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:33.969+0000 7f18edffb640 1 -- 172.21.15.27:0/1560042170 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v13) ==== 190+0+147 (secure 0 0 0) 0x7f18e8058520 con 0x7f18f80d3790 2024-09-12T03:19:33.970 INFO:teuthology.orchestra.run.smithi027.stdout:[client.0] 2024-09-12T03:19:33.971 INFO:teuthology.orchestra.run.smithi027.stdout: key = AQDVXOJmLm+5ExAAECM4CKfuVElmp/n0b02mXg== 2024-09-12T03:19:33.971 INFO:teuthology.orchestra.run.smithi027.stdout: caps mds = "allow" 2024-09-12T03:19:33.971 INFO:teuthology.orchestra.run.smithi027.stdout: caps mon = "allow r" 2024-09-12T03:19:33.971 INFO:teuthology.orchestra.run.smithi027.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-12T03:19:33.971 INFO:teuthology.orchestra.run.smithi027.stderr:updated caps for client.0 2024-09-12T03:19:33.973 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:33.971+0000 7f18ff41e640 1 -- 172.21.15.27:0/1560042170 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f18dc06bfe0 msgr2=0x7f18dc06e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:33.973 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:33.971+0000 7f18ff41e640 1 --2- 172.21.15.27:0/1560042170 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f18dc06bfe0 0x7f18dc06e4a0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f18f80cdc00 tx=0x7f18f4008040 comp rx=0 tx=0).stop 2024-09-12T03:19:33.973 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:33.971+0000 7f18ff41e640 1 -- 172.21.15.27:0/1560042170 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f18f80d3790 msgr2=0x7f18f815ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:33.973 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:33.971+0000 7f18ff41e640 1 --2- 172.21.15.27:0/1560042170 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f18f80d3790 0x7f18f815ac40 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f18e800c470 tx=0x7f18e800c930 comp rx=0 tx=0).stop 2024-09-12T03:19:33.973 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:33.972+0000 7f18ff41e640 1 -- 172.21.15.27:0/1560042170 shutdown_connections 2024-09-12T03:19:33.973 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:33.972+0000 7f18ff41e640 1 --2- 172.21.15.27:0/1560042170 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f18dc06bfe0 0x7f18dc06e4a0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:33.973 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:33.972+0000 7f18ff41e640 1 --2- 172.21.15.27:0/1560042170 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f18f8162040 0x7f18f815b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:33.973 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:33.972+0000 7f18ff41e640 1 --2- 172.21.15.27:0/1560042170 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f18f80d3790 0x7f18f815ac40 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:33.973 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:33.972+0000 7f18ff41e640 1 --2- 172.21.15.27:0/1560042170 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f18f806dfd0 0x7f18f81f7560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:33.973 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:33.972+0000 7f18ff41e640 1 -- 172.21.15.27:0/1560042170 >> 172.21.15.27:0/1560042170 conn(0x7f18f80b6700 msgr2=0x7f18f80cfd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:33.973 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:33.972+0000 7f18ff41e640 1 -- 172.21.15.27:0/1560042170 shutdown_connections 2024-09-12T03:19:33.973 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:33.972+0000 7f18ff41e640 1 -- 172.21.15.27:0/1560042170 wait complete. 2024-09-12T03:19:33.983 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-12T03:19:34.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.084+0000 7f8c94f98640 1 Processor -- start 2024-09-12T03:19:34.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.084+0000 7f8c94f98640 1 -- start start 2024-09-12T03:19:34.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.085+0000 7f8c94f98640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f8c9015eb90 0x7f8c900d2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:34.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.085+0000 7f8c94f98640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f8c900d5200 0x7f8c900d1010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:34.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.085+0000 7f8c94f98640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8c900d3760 con 0x7f8c900d1550 2024-09-12T03:19:34.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.085+0000 7f8c94f98640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f8c900b7d20 con 0x7f8c9015eb90 2024-09-12T03:19:34.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.085+0000 7f8c94f98640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f8c900d42e0 con 0x7f8c900d5200 2024-09-12T03:19:34.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.086+0000 7f8c8e59c640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f8c9015eb90 0x7f8c900d2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:34.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.086+0000 7f8c8e59c640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f8c9015eb90 0x7f8c900d2df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38420/0 (socket says 172.21.15.27:38420) 2024-09-12T03:19:34.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.086+0000 7f8c8e59c640 1 -- 172.21.15.27:0/2123668150 learned_addr learned my addr 172.21.15.27:0/2123668150 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:19:34.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.086+0000 7f8c8dd9b640 1 --2- 172.21.15.27:0/2123668150 >> v2:172.21.15.45:3300/0 conn(0x7f8c900d5200 0x7f8c900d1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:34.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.086+0000 7f8c8e59c640 1 -- 172.21.15.27:0/2123668150 >> v2:172.21.15.45:3300/0 conn(0x7f8c900d5200 msgr2=0x7f8c900d1010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:34.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.086+0000 7f8c8e59c640 1 --2- 172.21.15.27:0/2123668150 >> v2:172.21.15.45:3300/0 conn(0x7f8c900d5200 0x7f8c900d1010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:34.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.086+0000 7f8c8e59c640 1 -- 172.21.15.27:0/2123668150 >> v1:172.21.15.45:6789/0 conn(0x7f8c900d1550 legacy=0x7f8c900cf200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:34.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.086+0000 7f8c8e59c640 1 -- 172.21.15.27:0/2123668150 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c900d3a40 con 0x7f8c9015eb90 2024-09-12T03:19:34.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.087+0000 7f8c8e59c640 1 --2- 172.21.15.27:0/2123668150 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8c9015eb90 0x7f8c900d2df0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f8c900d4750 tx=0x7f8c8002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8394a277181c2de9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:34.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.087+0000 7f8c8d59a640 1 -- 172.21.15.27:0/2123668150 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c80037070 con 0x7f8c9015eb90 2024-09-12T03:19:34.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.087+0000 7f8c8d59a640 1 -- 172.21.15.27:0/2123668150 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c80032440 con 0x7f8c9015eb90 2024-09-12T03:19:34.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.087+0000 7f8c8d59a640 1 -- 172.21.15.27:0/2123668150 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c80032810 con 0x7f8c9015eb90 2024-09-12T03:19:34.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.087+0000 7f8c94f98640 1 -- 172.21.15.27:0/2123668150 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8c9015eb90 msgr2=0x7f8c900d2df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:34.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.087+0000 7f8c94f98640 1 --2- 172.21.15.27:0/2123668150 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8c9015eb90 0x7f8c900d2df0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f8c900d4750 tx=0x7f8c8002a230 comp rx=0 tx=0).stop 2024-09-12T03:19:34.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.087+0000 7f8c94f98640 1 -- 172.21.15.27:0/2123668150 shutdown_connections 2024-09-12T03:19:34.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.087+0000 7f8c94f98640 1 --2- 172.21.15.27:0/2123668150 >> v2:172.21.15.45:3300/0 conn(0x7f8c900d5200 0x7f8c900d1010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:34.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.087+0000 7f8c94f98640 1 --2- 172.21.15.27:0/2123668150 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8c9015eb90 0x7f8c900d2df0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:34.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.087+0000 7f8c94f98640 1 -- 172.21.15.27:0/2123668150 >> 172.21.15.27:0/2123668150 conn(0x7f8c9006dce0 msgr2=0x7f8c9006e0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:34.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.087+0000 7f8c94f98640 1 -- 172.21.15.27:0/2123668150 shutdown_connections 2024-09-12T03:19:34.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.088+0000 7f8c94f98640 1 -- 172.21.15.27:0/2123668150 wait complete. 2024-09-12T03:19:34.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.089+0000 7f8c94f98640 1 Processor -- start 2024-09-12T03:19:34.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.089+0000 7f8c94f98640 1 -- start start 2024-09-12T03:19:34.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.089+0000 7f8c94f98640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8c900d1550 0x7f8c90157e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:34.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.090+0000 7f8c94f98640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8c900d5200 0x7f8c9014cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:34.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.090+0000 7f8c94f98640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f8c9015eb90 0x7f8c900c8340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:34.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.090+0000 7f8c94f98640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f8c900d1960 con 0x7f8c900d1550 2024-09-12T03:19:34.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.090+0000 7f8c94f98640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f8c900d4000 con 0x7f8c900d5200 2024-09-12T03:19:34.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.090+0000 7f8c94f98640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f8c900d3330 con 0x7f8c9015eb90 2024-09-12T03:19:34.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.090+0000 7f8c8e59c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8c900d1550 0x7f8c90157e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:34.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.090+0000 7f8c8e59c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8c900d1550 0x7f8c90157e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38426/0 (socket says 172.21.15.27:38426) 2024-09-12T03:19:34.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.090+0000 7f8c8e59c640 1 -- 172.21.15.27:0/2587437573 learned_addr learned my addr 172.21.15.27:0/2587437573 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:19:34.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.090+0000 7f8c8dd9b640 1 --2- 172.21.15.27:0/2587437573 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8c900d5200 0x7f8c9014cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:34.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.090+0000 7f8c8ed9d640 1 --2- 172.21.15.27:0/2587437573 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f8c9015eb90 0x7f8c900c8340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:34.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.090+0000 7f8c8e59c640 1 -- 172.21.15.27:0/2587437573 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f8c9015eb90 msgr2=0x7f8c900c8340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:34.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.090+0000 7f8c8e59c640 1 --2- 172.21.15.27:0/2587437573 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f8c9015eb90 0x7f8c900c8340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:34.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.090+0000 7f8c8e59c640 1 -- 172.21.15.27:0/2587437573 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8c900d5200 msgr2=0x7f8c9014cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:34.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.090+0000 7f8c8e59c640 1 --2- 172.21.15.27:0/2587437573 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8c900d5200 0x7f8c9014cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:34.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.090+0000 7f8c8e59c640 1 -- 172.21.15.27:0/2587437573 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c900c8b70 con 0x7f8c900d1550 2024-09-12T03:19:34.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.090+0000 7f8c8dd9b640 1 --2- 172.21.15.27:0/2587437573 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8c900d5200 0x7f8c9014cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:19:34.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.090+0000 7f8c8e59c640 1 --2- 172.21.15.27:0/2587437573 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8c900d1550 0x7f8c90157e10 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f8c80000c00 tx=0x7f8c800046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:34.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.090+0000 7f8c7effd640 1 -- 172.21.15.27:0/2587437573 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c80041020 con 0x7f8c900d1550 2024-09-12T03:19:34.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.090+0000 7f8c7effd640 1 -- 172.21.15.27:0/2587437573 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c80030970 con 0x7f8c900d1550 2024-09-12T03:19:34.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.090+0000 7f8c94f98640 1 -- 172.21.15.27:0/2587437573 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c900c8e50 con 0x7f8c900d1550 2024-09-12T03:19:34.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.090+0000 7f8c7effd640 1 -- 172.21.15.27:0/2587437573 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c80037040 con 0x7f8c900d1550 2024-09-12T03:19:34.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.090+0000 7f8c94f98640 1 -- 172.21.15.27:0/2587437573 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c900c9410 con 0x7f8c900d1550 2024-09-12T03:19:34.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.091+0000 7f8c94f98640 1 -- 172.21.15.27:0/2587437573 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c9015ef70 con 0x7f8c900d1550 2024-09-12T03:19:34.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.092+0000 7f8c7effd640 1 -- 172.21.15.27:0/2587437573 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 9) ==== 90651+0+0 (secure 0 0 0) 0x7f8c80044020 con 0x7f8c900d1550 2024-09-12T03:19:34.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.092+0000 7f8c7effd640 1 --2- 172.21.15.27:0/2587437573 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f8c7006bfe0 0x7f8c7006e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:34.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.092+0000 7f8c7effd640 1 -- 172.21.15.27:0/2587437573 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5709+0+0 (secure 0 0 0) 0x7f8c800abbf0 con 0x7f8c900d1550 2024-09-12T03:19:34.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.092+0000 7f8c8dd9b640 1 --2- 172.21.15.27:0/2587437573 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f8c7006bfe0 0x7f8c7006e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:34.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.093+0000 7f8c8dd9b640 1 --2- 172.21.15.27:0/2587437573 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f8c7006bfe0 0x7f8c7006e4a0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f8c90160c60 tx=0x7f8c84008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:34.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.097+0000 7f8c7effd640 1 -- 172.21.15.27:0/2587437573 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f8c8004b030 con 0x7f8c900d1550 2024-09-12T03:19:34.308 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.307+0000 7f8c94f98640 1 -- 172.21.15.27:0/2587437573 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8c900b8f20 con 0x7f8c900d1550 2024-09-12T03:19:34.309 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.308+0000 7f8c7effd640 1 -- 172.21.15.27:0/2587437573 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+5220 (secure 0 0 0) 0x7f8c8007a320 con 0x7f8c900d1550 2024-09-12T03:19:34.309 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:19:34.309 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":30,"btime":"2024-09-12T03:19:27:848915+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5102,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/565788762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":565788762},{"type":"v1","addr":"172.21.15.27:6835","nonce":565788762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26},{"gid":5413,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6836/2689309305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":2689309305},{"type":"v1","addr":"172.21.15.45:6836","nonce":2689309305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":5640,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6834/3694936463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":3694936463},{"type":"v1","addr":"172.21.15.27:6834","nonce":3694936463}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":30,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-12T03:19:25.201174+0000","modified":"2024-09-12T03:19:27.190661+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5649},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5649":{"gid":5649,"name":"b","rank":0,"incarnation":28,"state":"up:active","state_seq":3,"addr":"172.21.15.45:6837/3720118075","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6835","nonce":3720118075},{"type":"v1","addr":"172.21.15.45:6837","nonce":3720118075}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5649,"qdb_cluster":[5649]},"id":4}]} 2024-09-12T03:19:34.309 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 30 2024-09-12T03:19:34.311 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.310+0000 7f8c94f98640 1 -- 172.21.15.27:0/2587437573 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f8c7006bfe0 msgr2=0x7f8c7006e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:34.311 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.310+0000 7f8c94f98640 1 --2- 172.21.15.27:0/2587437573 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f8c7006bfe0 0x7f8c7006e4a0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f8c90160c60 tx=0x7f8c84008040 comp rx=0 tx=0).stop 2024-09-12T03:19:34.311 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.310+0000 7f8c94f98640 1 -- 172.21.15.27:0/2587437573 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8c900d1550 msgr2=0x7f8c90157e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:34.311 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.310+0000 7f8c94f98640 1 --2- 172.21.15.27:0/2587437573 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8c900d1550 0x7f8c90157e10 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f8c80000c00 tx=0x7f8c800046c0 comp rx=0 tx=0).stop 2024-09-12T03:19:34.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.310+0000 7f8c94f98640 1 -- 172.21.15.27:0/2587437573 shutdown_connections 2024-09-12T03:19:34.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.310+0000 7f8c94f98640 1 --2- 172.21.15.27:0/2587437573 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f8c7006bfe0 0x7f8c7006e4a0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:34.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.310+0000 7f8c94f98640 1 --2- 172.21.15.27:0/2587437573 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f8c9015eb90 0x7f8c900c8340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:34.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.310+0000 7f8c94f98640 1 --2- 172.21.15.27:0/2587437573 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8c900d5200 0x7f8c9014cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:34.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.310+0000 7f8c94f98640 1 --2- 172.21.15.27:0/2587437573 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8c900d1550 0x7f8c90157e10 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:34.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.310+0000 7f8c94f98640 1 -- 172.21.15.27:0/2587437573 >> 172.21.15.27:0/2587437573 conn(0x7f8c9006dce0 msgr2=0x7f8c900cfdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:34.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.310+0000 7f8c94f98640 1 -- 172.21.15.27:0/2587437573 shutdown_connections 2024-09-12T03:19:34.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.310+0000 7f8c94f98640 1 -- 172.21.15.27:0/2587437573 wait complete. 2024-09-12T03:19:34.726 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 30, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-12T03:19:25.201174+0000', 'modified': '2024-09-12T03:19:27.190661+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5649}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5649': {'gid': 5649, 'name': 'b', 'rank': 0, 'incarnation': 28, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.45:6837/3720118075', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.45:6835', 'nonce': 3720118075}, {'type': 'v1', 'addr': '172.21.15.45:6837', 'nonce': 3720118075}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [9], 'metadata_pool': 8, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5649, 'qdb_cluster': [5649]} 2024-09-12T03:19:34.726 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-12T03:19:34.726 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-09-12T03:19:34.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.831+0000 7faab69a2640 1 Processor -- start 2024-09-12T03:19:34.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.831+0000 7faab69a2640 1 -- start start 2024-09-12T03:19:34.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.832+0000 7faab69a2640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7faab015e770 0x7faab00d3a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:34.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.832+0000 7faab69a2640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7faab00d17a0 0x7faab00b75f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:34.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.832+0000 7faab69a2640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faab00d2790 con 0x7faab00d3fc0 2024-09-12T03:19:34.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.832+0000 7faab69a2640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7faab00d1c10 con 0x7faab00d17a0 2024-09-12T03:19:34.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.832+0000 7faab69a2640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7faab00d4400 con 0x7faab015e770 2024-09-12T03:19:34.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.832+0000 7faab61a1640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7faab00d17a0 0x7faab00b75f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:34.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.832+0000 7faab61a1640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7faab00d17a0 0x7faab00b75f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38440/0 (socket says 172.21.15.27:38440) 2024-09-12T03:19:34.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.832+0000 7faab61a1640 1 -- 172.21.15.27:0/1524944517 learned_addr learned my addr 172.21.15.27:0/1524944517 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:19:34.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.832+0000 7faab59a0640 1 --2- 172.21.15.27:0/1524944517 >> v2:172.21.15.45:3300/0 conn(0x7faab015e770 0x7faab00d3a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:34.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.832+0000 7faab61a1640 1 -- 172.21.15.27:0/1524944517 >> v2:172.21.15.45:3300/0 conn(0x7faab015e770 msgr2=0x7faab00d3a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:34.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.832+0000 7faab61a1640 1 --2- 172.21.15.27:0/1524944517 >> v2:172.21.15.45:3300/0 conn(0x7faab015e770 0x7faab00d3a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:34.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.832+0000 7faab61a1640 1 -- 172.21.15.27:0/1524944517 >> v1:172.21.15.45:6789/0 conn(0x7faab00d3fc0 legacy=0x7faab00d1090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:34.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.832+0000 7faab61a1640 1 -- 172.21.15.27:0/1524944517 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faab00d2a70 con 0x7faab00d17a0 2024-09-12T03:19:34.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.833+0000 7faab61a1640 1 --2- 172.21.15.27:0/1524944517 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faab00d17a0 0x7faab00b75f0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7faab00d5600 tx=0x7faaa802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=998ba53c57ee2208 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:34.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.833+0000 7faab499e640 1 -- 172.21.15.27:0/1524944517 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaa8037070 con 0x7faab00d17a0 2024-09-12T03:19:34.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.833+0000 7faab499e640 1 -- 172.21.15.27:0/1524944517 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faaa8032440 con 0x7faab00d17a0 2024-09-12T03:19:34.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.833+0000 7faab499e640 1 -- 172.21.15.27:0/1524944517 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaa8032810 con 0x7faab00d17a0 2024-09-12T03:19:34.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.833+0000 7faab69a2640 1 -- 172.21.15.27:0/1524944517 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faab00d17a0 msgr2=0x7faab00b75f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:34.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.833+0000 7faab69a2640 1 --2- 172.21.15.27:0/1524944517 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faab00d17a0 0x7faab00b75f0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7faab00d5600 tx=0x7faaa802a270 comp rx=0 tx=0).stop 2024-09-12T03:19:34.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.834+0000 7faab69a2640 1 -- 172.21.15.27:0/1524944517 shutdown_connections 2024-09-12T03:19:34.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.834+0000 7faab69a2640 1 --2- 172.21.15.27:0/1524944517 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faab00d17a0 0x7faab00b75f0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:34.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.834+0000 7faab69a2640 1 --2- 172.21.15.27:0/1524944517 >> v2:172.21.15.45:3300/0 conn(0x7faab015e770 0x7faab00d3a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:34.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.834+0000 7faab69a2640 1 -- 172.21.15.27:0/1524944517 >> 172.21.15.27:0/1524944517 conn(0x7faab00b6100 msgr2=0x7faab00b6510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:34.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.834+0000 7faab69a2640 1 -- 172.21.15.27:0/1524944517 shutdown_connections 2024-09-12T03:19:34.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.834+0000 7faab69a2640 1 -- 172.21.15.27:0/1524944517 wait complete. 2024-09-12T03:19:34.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.835+0000 7faab69a2640 1 Processor -- start 2024-09-12T03:19:34.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.835+0000 7faab69a2640 1 -- start start 2024-09-12T03:19:34.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.835+0000 7faab69a2640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faab00d17a0 0x7faab00cd160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:34.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.835+0000 7faab69a2640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faab00d3fc0 0x7faab0160b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:34.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.835+0000 7faab69a2640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7faab015e770 0x7faab0161090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:34.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.835+0000 7faab69a2640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7faab00d3310 con 0x7faab00d3fc0 2024-09-12T03:19:34.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.835+0000 7faab69a2640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7faab00d3030 con 0x7faab00d17a0 2024-09-12T03:19:34.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.835+0000 7faab69a2640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7faab00b7b30 con 0x7faab015e770 2024-09-12T03:19:34.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.836+0000 7faab519f640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faab00d3fc0 0x7faab0160b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:34.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.836+0000 7faab519f640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faab00d3fc0 0x7faab0160b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38446/0 (socket says 172.21.15.27:38446) 2024-09-12T03:19:34.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.836+0000 7faab519f640 1 -- 172.21.15.27:0/1639076550 learned_addr learned my addr 172.21.15.27:0/1639076550 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:19:34.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.836+0000 7faab59a0640 1 --2- 172.21.15.27:0/1639076550 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faab00d17a0 0x7faab00cd160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:34.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.836+0000 7faab61a1640 1 --2- 172.21.15.27:0/1639076550 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7faab015e770 0x7faab0161090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:34.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.836+0000 7faab519f640 1 -- 172.21.15.27:0/1639076550 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7faab015e770 msgr2=0x7faab0161090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:34.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.836+0000 7faab519f640 1 --2- 172.21.15.27:0/1639076550 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7faab015e770 0x7faab0161090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:34.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.836+0000 7faab519f640 1 -- 172.21.15.27:0/1639076550 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faab00d17a0 msgr2=0x7faab00cd160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:34.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.836+0000 7faab519f640 1 --2- 172.21.15.27:0/1639076550 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faab00d17a0 0x7faab00cd160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:34.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.836+0000 7faab519f640 1 -- 172.21.15.27:0/1639076550 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faab00cd6a0 con 0x7faab00d3fc0 2024-09-12T03:19:34.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.836+0000 7faab519f640 1 --2- 172.21.15.27:0/1639076550 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faab00d3fc0 0x7faab0160b50 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7faaa400c400 tx=0x7faaa400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:34.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.836+0000 7faaae7fc640 1 -- 172.21.15.27:0/1639076550 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaa40052d0 con 0x7faab00d3fc0 2024-09-12T03:19:34.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.836+0000 7faaae7fc640 1 -- 172.21.15.27:0/1639076550 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faaa4005490 con 0x7faab00d3fc0 2024-09-12T03:19:34.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.836+0000 7faaae7fc640 1 -- 172.21.15.27:0/1639076550 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaa4007260 con 0x7faab00d3fc0 2024-09-12T03:19:34.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.836+0000 7faab69a2640 1 -- 172.21.15.27:0/1639076550 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faab00cd940 con 0x7faab00d3fc0 2024-09-12T03:19:34.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.836+0000 7faab69a2640 1 -- 172.21.15.27:0/1639076550 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faab00cdf00 con 0x7faab00d3fc0 2024-09-12T03:19:34.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.838+0000 7faaae7fc640 1 -- 172.21.15.27:0/1639076550 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 9) ==== 90651+0+0 (secure 0 0 0) 0x7faaa4007400 con 0x7faab00d3fc0 2024-09-12T03:19:34.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.838+0000 7faaae7fc640 1 --2- 172.21.15.27:0/1639076550 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7faa9c06bfe0 0x7faa9c06e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:34.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.838+0000 7faaae7fc640 1 -- 172.21.15.27:0/1639076550 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5709+0+0 (secure 0 0 0) 0x7faaa408a4c0 con 0x7faab00d3fc0 2024-09-12T03:19:34.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.838+0000 7faab59a0640 1 --2- 172.21.15.27:0/1639076550 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7faa9c06bfe0 0x7faa9c06e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:34.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.839+0000 7faab59a0640 1 --2- 172.21.15.27:0/1639076550 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7faa9c06bfe0 0x7faa9c06e4a0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7faab0161850 tx=0x7faaa0009830 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:34.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.839+0000 7faabaa79740 1 Processor -- start 2024-09-12T03:19:34.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.839+0000 7faabaa79740 1 -- start start 2024-09-12T03:19:34.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.840+0000 7faabaa79740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5605e34e14b0 0x5605e3526c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:34.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.840+0000 7faabaa79740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5605e34dec70 0x5605e34e1890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:34.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.840+0000 7faabaa79740 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x5605e34e2c40 0x5605e34dbaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:34.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.840+0000 7faabaa79740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5605e3443520 con 0x5605e34dec70 2024-09-12T03:19:34.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.840+0000 7faabaa79740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5605e33eb3e0 con 0x5605e34e14b0 2024-09-12T03:19:34.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.840+0000 7faabaa79740 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x5605e343f4e0 con 0x5605e34e2c40 2024-09-12T03:19:34.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.840+0000 7faab519f640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5605e34dec70 0x5605e34e1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:34.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.840+0000 7faab519f640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5605e34dec70 0x5605e34e1890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38454/0 (socket says 172.21.15.27:38454) 2024-09-12T03:19:34.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.840+0000 7faab519f640 1 -- 172.21.15.27:0/2167368751 learned_addr learned my addr 172.21.15.27:0/2167368751 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:19:34.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.840+0000 7faab59a0640 1 --2- 172.21.15.27:0/2167368751 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5605e34e14b0 0x5605e3526c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:34.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.840+0000 7faab61a1640 1 --2- 172.21.15.27:0/2167368751 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x5605e34e2c40 0x5605e34dbaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:34.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.840+0000 7faab519f640 1 -- 172.21.15.27:0/2167368751 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x5605e34e2c40 msgr2=0x5605e34dbaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:34.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.841+0000 7faab519f640 1 --2- 172.21.15.27:0/2167368751 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x5605e34e2c40 0x5605e34dbaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:34.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.841+0000 7faab519f640 1 -- 172.21.15.27:0/2167368751 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5605e34e14b0 msgr2=0x5605e3526c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:34.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.841+0000 7faab519f640 1 --2- 172.21.15.27:0/2167368751 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5605e34e14b0 0x5605e3526c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:34.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.841+0000 7faab519f640 1 -- 172.21.15.27:0/2167368751 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5605e34062f0 con 0x5605e34dec70 2024-09-12T03:19:34.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.841+0000 7faab519f640 1 --2- 172.21.15.27:0/2167368751 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5605e34dec70 0x5605e34e1890 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7faaa4000fb0 tx=0x7faaa405c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f2f97221e191b96 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:34.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.841+0000 7faa977fe640 1 -- 172.21.15.27:0/2167368751 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaa408a0c0 con 0x5605e34dec70 2024-09-12T03:19:34.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.841+0000 7faa977fe640 1 -- 172.21.15.27:0/2167368751 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faaa408a260 con 0x5605e34dec70 2024-09-12T03:19:34.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.841+0000 7faa977fe640 1 -- 172.21.15.27:0/2167368751 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaa4088ad0 con 0x5605e34dec70 2024-09-12T03:19:34.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.841+0000 7faabaa79740 1 -- 172.21.15.27:0/2167368751 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5605e34dec70 msgr2=0x5605e34e1890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:34.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.841+0000 7faabaa79740 1 --2- 172.21.15.27:0/2167368751 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5605e34dec70 0x5605e34e1890 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7faaa4000fb0 tx=0x7faaa405c9e0 comp rx=0 tx=0).stop 2024-09-12T03:19:34.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.842+0000 7faabaa79740 1 -- 172.21.15.27:0/2167368751 shutdown_connections 2024-09-12T03:19:34.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.842+0000 7faabaa79740 1 --2- 172.21.15.27:0/2167368751 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x5605e34e2c40 0x5605e34dbaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:34.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.842+0000 7faabaa79740 1 --2- 172.21.15.27:0/2167368751 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5605e34dec70 0x5605e34e1890 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:34.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.842+0000 7faabaa79740 1 --2- 172.21.15.27:0/2167368751 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5605e34e14b0 0x5605e3526c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:34.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.842+0000 7faabaa79740 1 -- 172.21.15.27:0/2167368751 >> 172.21.15.27:0/2167368751 conn(0x5605e34ddc70 msgr2=0x5605e3526120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:34.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.842+0000 7faabaa79740 1 -- 172.21.15.27:0/2167368751 shutdown_connections 2024-09-12T03:19:34.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.842+0000 7faabaa79740 1 -- 172.21.15.27:0/2167368751 wait complete. 2024-09-12T03:19:34.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.842+0000 7faabaa79740 1 -- start start 2024-09-12T03:19:34.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.842+0000 7faabaa79740 1 Processor -- start 2024-09-12T03:19:34.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.843+0000 7faabaa79740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5605e34e14b0 0x5605e350acd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:34.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.843+0000 7faabaa79740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5605e34e2c40 0x5605e34fcca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:34.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.843+0000 7faabaa79740 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x5605e350c7e0 0x5605e350cc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:34.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.843+0000 7faabaa79740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5605e34dd110 con 0x5605e34e2c40 2024-09-12T03:19:34.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.843+0000 7faabaa79740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5605e33cc450 con 0x5605e34e14b0 2024-09-12T03:19:34.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.843+0000 7faabaa79740 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x5605e3527480 con 0x5605e350c7e0 2024-09-12T03:19:34.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.843+0000 7faabaa79740 10 client.0 _ll_register_callbacks cb 0x5605e3155ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:19:34.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.843+0000 7faab519f640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5605e34e2c40 0x5605e34fcca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:34.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.843+0000 7faab519f640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5605e34e2c40 0x5605e34fcca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38460/0 (socket says 172.21.15.27:38460) 2024-09-12T03:19:34.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.843+0000 7faab519f640 1 -- 172.21.15.27:0/1892297762 learned_addr learned my addr 172.21.15.27:0/1892297762 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:19:34.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.843+0000 7faab59a0640 1 --2- 172.21.15.27:0/1892297762 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5605e34e14b0 0x5605e350acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:34.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.843+0000 7faab61a1640 1 --2- 172.21.15.27:0/1892297762 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x5605e350c7e0 0x5605e350cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:34.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.844+0000 7faab519f640 1 -- 172.21.15.27:0/1892297762 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x5605e350c7e0 msgr2=0x5605e350cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:34.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.844+0000 7faab519f640 1 --2- 172.21.15.27:0/1892297762 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x5605e350c7e0 0x5605e350cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:34.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.844+0000 7faab519f640 1 -- 172.21.15.27:0/1892297762 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5605e34e14b0 msgr2=0x5605e350acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:34.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.844+0000 7faab519f640 1 --2- 172.21.15.27:0/1892297762 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5605e34e14b0 0x5605e350acd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:34.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.844+0000 7faab519f640 1 -- 172.21.15.27:0/1892297762 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5605e31562d0 con 0x5605e34e2c40 2024-09-12T03:19:34.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.844+0000 7faab519f640 1 --2- 172.21.15.27:0/1892297762 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5605e34e2c40 0x5605e34fcca0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7faaa400cfd0 tx=0x7faaa4000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e8d1a03891af8be server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:34.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.844+0000 7faa94ff9640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-12T03:19:34.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.844+0000 7faa94ff9640 1 -- 172.21.15.27:0/1892297762 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaa4075060 con 0x5605e34e2c40 2024-09-12T03:19:34.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.844+0000 7faa94ff9640 1 -- 172.21.15.27:0/1892297762 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(56..56 src has 1..56) ==== 5709+0+0 (secure 0 0 0) 0x7faaa405dd70 con 0x5605e34e2c40 2024-09-12T03:19:34.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.844+0000 7faabaa79740 1 -- 172.21.15.27:0/1892297762 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faaa408a260 con 0x5605e34e2c40 2024-09-12T03:19:34.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.844+0000 7faa94ff9640 1 -- 172.21.15.27:0/1892297762 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7faaa408a0c0 con 0x5605e34e2c40 2024-09-12T03:19:34.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.844+0000 7faabaa79740 10 client.5805 fetch_fsmap learned FSMap version 30 2024-09-12T03:19:34.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.844+0000 7faabaa79740 1 -- 172.21.15.27:0/1892297762 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=2+}) -- 0x5605e32206b0 con 0x5605e34e2c40 2024-09-12T03:19:34.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.844+0000 7faa94ff9640 1 -- 172.21.15.27:0/1892297762 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faaa408a260 con 0x5605e34e2c40 2024-09-12T03:19:34.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.844+0000 7faa94ff9640 1 -- 172.21.15.27:0/1892297762 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 30) ==== 3502+0+0 (secure 0 0 0) 0x7faaa4085070 con 0x5605e34e2c40 2024-09-12T03:19:34.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.844+0000 7faabaa79740 10 client.5805 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T03:19:34.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.844+0000 7faabaa79740 10 client.5805 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-12T03:19:34.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.844+0000 7faabaa79740 20 client.5805 populate_metadata read hostname 'smithi027' 2024-09-12T03:19:34.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.844+0000 7faabaa79740 1 --2- 172.21.15.27:0/1892297762 >> [v2:172.21.15.45:6835/3720118075,v1:172.21.15.45:6837/3720118075] conn(0x5605e35220a0 0x5605e3522480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:34.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.844+0000 7faabaa79740 4 client.5805 mds_command: new command op to 5649 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:19:34.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.844+0000 7faabaa79740 1 -- 172.21.15.27:0/1892297762 --> [v2:172.21.15.45:6835/3720118075,v1:172.21.15.45:6837/3720118075] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5605e3450c50 con 0x5605e35220a0 2024-09-12T03:19:34.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.845+0000 7faab59a0640 1 --2- 172.21.15.27:0/1892297762 >> [v2:172.21.15.45:6835/3720118075,v1:172.21.15.45:6837/3720118075] conn(0x5605e35220a0 0x5605e3522480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:34.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.845+0000 7faa94ff9640 10 client.5805 ms_handle_connect on v2:172.21.15.45:6835/3720118075 2024-09-12T03:19:34.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.845+0000 7faab59a0640 1 --2- 172.21.15.27:0/1892297762 >> [v2:172.21.15.45:6835/3720118075,v1:172.21.15.45:6837/3720118075] conn(0x5605e35220a0 0x5605e3522480 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=70959b7819cf146a server_cookie=80e20fc8ef6e7933 in_seq=0 out_seq=0 2024-09-12T03:19:34.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.846+0000 7faa94ff9640 1 -- 172.21.15.27:0/1892297762 <== mds.0 v2:172.21.15.45:6835/3720118075 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5605e3450c50 con 0x5605e35220a0 2024-09-12T03:19:34.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.846+0000 7faa94ff9640 10 client.5805 handle_command_reply: tid=0 2024-09-12T03:19:34.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.846+0000 7faabaa79740 1 client.5805 shutdown 2024-09-12T03:19:34.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.846+0000 7faabaa79740 1 -- 172.21.15.27:0/1892297762 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5605e34e2c40 msgr2=0x5605e34fcca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:34.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.846+0000 7faabaa79740 1 --2- 172.21.15.27:0/1892297762 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5605e34e2c40 0x5605e34fcca0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7faaa400cfd0 tx=0x7faaa4000ed0 comp rx=0 tx=0).stop 2024-09-12T03:19:34.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.846+0000 7faabaa79740 1 -- 172.21.15.27:0/1892297762 shutdown_connections 2024-09-12T03:19:34.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.846+0000 7faabaa79740 1 --2- 172.21.15.27:0/1892297762 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x5605e350c7e0 0x5605e350cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:34.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.846+0000 7faabaa79740 1 --2- 172.21.15.27:0/1892297762 >> [v2:172.21.15.45:6835/3720118075,v1:172.21.15.45:6837/3720118075] conn(0x5605e35220a0 0x5605e3522480 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:34.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.846+0000 7faabaa79740 1 --2- 172.21.15.27:0/1892297762 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5605e34e2c40 0x5605e34fcca0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:34.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.846+0000 7faabaa79740 1 --2- 172.21.15.27:0/1892297762 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5605e34e14b0 0x5605e350acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:34.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.846+0000 7faa94ff9640 0 client.5805 ms_handle_reset on v2:172.21.15.45:6835/3720118075 2024-09-12T03:19:34.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.846+0000 7faabaa79740 1 -- 172.21.15.27:0/1892297762 >> 172.21.15.27:0/1892297762 conn(0x5605e34dec70 msgr2=0x5605e3527c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:34.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.846+0000 7faabaa79740 1 -- 172.21.15.27:0/1892297762 shutdown_connections 2024-09-12T03:19:34.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.846+0000 7faabaa79740 1 -- 172.21.15.27:0/1892297762 wait complete. 2024-09-12T03:19:34.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.846+0000 7faabaa79740 20 client.5805 trim_cache size 0 max 16384 2024-09-12T03:19:34.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.865+0000 7faabaa79740 1 Processor -- start 2024-09-12T03:19:34.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.865+0000 7faabaa79740 1 -- start start 2024-09-12T03:19:34.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.865+0000 7faabaa79740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5605e34e14b0 0x5605e35246f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:34.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.865+0000 7faabaa79740 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x5605e34e2c40 0x5605e3523850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:34.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.865+0000 7faabaa79740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5605e350c7e0 0x5605e350cf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:34.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.865+0000 7faabaa79740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7faab01ffb00 con 0x5605e34e14b0 2024-09-12T03:19:34.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.865+0000 7faabaa79740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7faab00cc470 con 0x5605e350c7e0 2024-09-12T03:19:34.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.865+0000 7faabaa79740 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7faab01f4ff0 con 0x5605e34e2c40 2024-09-12T03:19:34.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.865+0000 7faab59a0640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5605e34e14b0 0x5605e35246f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:34.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.865+0000 7faab59a0640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5605e34e14b0 0x5605e35246f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38466/0 (socket says 172.21.15.27:38466) 2024-09-12T03:19:34.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.865+0000 7faab59a0640 1 -- 172.21.15.27:0/3942108172 learned_addr learned my addr 172.21.15.27:0/3942108172 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:19:34.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.865+0000 7faab59a0640 1 -- 172.21.15.27:0/3942108172 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x5605e34e2c40 msgr2=0x5605e3523850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:34.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.865+0000 7faab61a1640 1 --2- 172.21.15.27:0/3942108172 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5605e350c7e0 0x5605e350cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:34.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.865+0000 7faab59a0640 1 --2- 172.21.15.27:0/3942108172 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x5605e34e2c40 0x5605e3523850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:34.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.865+0000 7faab59a0640 1 -- 172.21.15.27:0/3942108172 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5605e350c7e0 msgr2=0x5605e350cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:34.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.865+0000 7faab59a0640 1 --2- 172.21.15.27:0/3942108172 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5605e350c7e0 0x5605e350cf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:34.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.865+0000 7faab59a0640 1 -- 172.21.15.27:0/3942108172 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5605e3520ed0 con 0x5605e34e14b0 2024-09-12T03:19:34.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.866+0000 7faab59a0640 1 --2- 172.21.15.27:0/3942108172 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5605e34e14b0 0x5605e35246f0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7faaa0009f90 tx=0x7faaa0016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fecd8a21551ce9af server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:34.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.866+0000 7faa96ffd640 1 -- 172.21.15.27:0/3942108172 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaa0017af0 con 0x5605e34e14b0 2024-09-12T03:19:34.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.866+0000 7faa96ffd640 1 -- 172.21.15.27:0/3942108172 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faaa0017c90 con 0x5605e34e14b0 2024-09-12T03:19:34.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.866+0000 7faa96ffd640 1 -- 172.21.15.27:0/3942108172 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaa0017e40 con 0x5605e34e14b0 2024-09-12T03:19:34.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.866+0000 7faabaa79740 1 -- 172.21.15.27:0/3942108172 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5605e34e14b0 msgr2=0x5605e35246f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:34.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.866+0000 7faabaa79740 1 --2- 172.21.15.27:0/3942108172 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5605e34e14b0 0x5605e35246f0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7faaa0009f90 tx=0x7faaa0016000 comp rx=0 tx=0).stop 2024-09-12T03:19:34.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.866+0000 7faabaa79740 1 -- 172.21.15.27:0/3942108172 shutdown_connections 2024-09-12T03:19:34.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.866+0000 7faabaa79740 1 --2- 172.21.15.27:0/3942108172 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5605e350c7e0 0x5605e350cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:34.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.866+0000 7faabaa79740 1 --2- 172.21.15.27:0/3942108172 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x5605e34e2c40 0x5605e3523850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:34.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.866+0000 7faabaa79740 1 --2- 172.21.15.27:0/3942108172 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5605e34e14b0 0x5605e35246f0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:34.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.866+0000 7faabaa79740 1 -- 172.21.15.27:0/3942108172 >> 172.21.15.27:0/3942108172 conn(0x5605e34dec70 msgr2=0x5605e34d7270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:34.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.866+0000 7faabaa79740 1 -- 172.21.15.27:0/3942108172 shutdown_connections 2024-09-12T03:19:34.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.866+0000 7faabaa79740 1 -- 172.21.15.27:0/3942108172 wait complete. 2024-09-12T03:19:34.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.867+0000 7faabaa79740 1 -- start start 2024-09-12T03:19:34.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.867+0000 7faabaa79740 1 Processor -- start 2024-09-12T03:19:34.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.868+0000 7faabaa79740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5605e34e14b0 0x5605e351f370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:34.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.868+0000 7faabaa79740 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x5605e34e2c40 0x5605e358d360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:34.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.868+0000 7faabaa79740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5605e350c7e0 0x5605e3522480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:34.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.868+0000 7faabaa79740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5605e34dd110 con 0x5605e350c7e0 2024-09-12T03:19:34.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.868+0000 7faabaa79740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5605e33cc450 con 0x5605e34e14b0 2024-09-12T03:19:34.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.868+0000 7faabaa79740 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x5605e34fd300 con 0x5605e34e2c40 2024-09-12T03:19:34.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.868+0000 7faabaa79740 10 client.0 _ll_register_callbacks cb 0x7faaa405f030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:19:34.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.868+0000 7faab61a1640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5605e350c7e0 0x5605e3522480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:34.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.868+0000 7faab61a1640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5605e350c7e0 0x5605e3522480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38472/0 (socket says 172.21.15.27:38472) 2024-09-12T03:19:34.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.868+0000 7faab61a1640 1 -- 172.21.15.27:0/734543160 learned_addr learned my addr 172.21.15.27:0/734543160 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:19:34.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.868+0000 7faab59a0640 1 --2- 172.21.15.27:0/734543160 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5605e34e14b0 0x5605e351f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:34.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.868+0000 7faab519f640 1 --2- 172.21.15.27:0/734543160 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x5605e34e2c40 0x5605e358d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:34.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.868+0000 7faab61a1640 1 -- 172.21.15.27:0/734543160 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x5605e34e2c40 msgr2=0x5605e358d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:34.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.868+0000 7faab61a1640 1 --2- 172.21.15.27:0/734543160 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x5605e34e2c40 0x5605e358d360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:34.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.868+0000 7faab61a1640 1 -- 172.21.15.27:0/734543160 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5605e34e14b0 msgr2=0x5605e351f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:34.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.868+0000 7faab61a1640 1 --2- 172.21.15.27:0/734543160 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5605e34e14b0 0x5605e351f370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:34.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.868+0000 7faab61a1640 1 -- 172.21.15.27:0/734543160 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5605e3528400 con 0x5605e350c7e0 2024-09-12T03:19:34.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.868+0000 7faab61a1640 1 --2- 172.21.15.27:0/734543160 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5605e350c7e0 0x5605e3522480 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7faaa8009930 tx=0x7faaa802c070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4869089c2dc5a1f7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:34.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.868+0000 7faa94ff9640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-12T03:19:34.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.869+0000 7faa94ff9640 1 -- 172.21.15.27:0/734543160 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaa804a060 con 0x5605e350c7e0 2024-09-12T03:19:34.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.869+0000 7faa94ff9640 1 -- 172.21.15.27:0/734543160 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(56..56 src has 1..56) ==== 5709+0+0 (secure 0 0 0) 0x7faaa8045460 con 0x5605e350c7e0 2024-09-12T03:19:34.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.869+0000 7faabaa79740 1 -- 172.21.15.27:0/734543160 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faaa0017c90 con 0x5605e350c7e0 2024-09-12T03:19:34.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.869+0000 7faa94ff9640 1 -- 172.21.15.27:0/734543160 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7faaa8050050 con 0x5605e350c7e0 2024-09-12T03:19:34.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.869+0000 7faabaa79740 10 client.5811 fetch_fsmap learned FSMap version 30 2024-09-12T03:19:34.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.869+0000 7faabaa79740 1 -- 172.21.15.27:0/734543160 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=2+}) -- 0x5605e351eb70 con 0x5605e350c7e0 2024-09-12T03:19:34.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.869+0000 7faa94ff9640 1 -- 172.21.15.27:0/734543160 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faaa0017c90 con 0x5605e350c7e0 2024-09-12T03:19:34.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.869+0000 7faa94ff9640 1 -- 172.21.15.27:0/734543160 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 30) ==== 3502+0+0 (secure 0 0 0) 0x7faaa8053020 con 0x5605e350c7e0 2024-09-12T03:19:34.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.869+0000 7faabaa79740 10 client.5811 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T03:19:34.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.869+0000 7faabaa79740 10 client.5811 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-12T03:19:34.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.869+0000 7faabaa79740 20 client.5811 populate_metadata read hostname 'smithi027' 2024-09-12T03:19:34.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.869+0000 7faabaa79740 1 --2- 172.21.15.27:0/734543160 >> [v2:172.21.15.45:6835/3720118075,v1:172.21.15.45:6837/3720118075] conn(0x5605e350d1f0 0x5605e35a7770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:34.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.869+0000 7faabaa79740 4 client.5811 mds_command: new command op to 5649 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-12T03:19:34.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.869+0000 7faabaa79740 1 -- 172.21.15.27:0/734543160 --> [v2:172.21.15.45:6835/3720118075,v1:172.21.15.45:6837/3720118075] -- command(tid 0: {"prefix": "status"}) -- 0x5605e34dccb0 con 0x5605e350d1f0 2024-09-12T03:19:34.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.869+0000 7faab59a0640 1 --2- 172.21.15.27:0/734543160 >> [v2:172.21.15.45:6835/3720118075,v1:172.21.15.45:6837/3720118075] conn(0x5605e350d1f0 0x5605e35a7770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:34.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.870+0000 7faab59a0640 1 --2- 172.21.15.27:0/734543160 >> [v2:172.21.15.45:6835/3720118075,v1:172.21.15.45:6837/3720118075] conn(0x5605e350d1f0 0x5605e35a7770 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e5a77a1b95e91c27 server_cookie=4e72f0fffe8e0111 in_seq=0 out_seq=0 2024-09-12T03:19:34.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.870+0000 7faa94ff9640 10 client.5811 ms_handle_connect on v2:172.21.15.45:6835/3720118075 2024-09-12T03:19:34.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.870+0000 7faa94ff9640 1 -- 172.21.15.27:0/734543160 <== mds.0 v2:172.21.15.45:6835/3720118075 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x5605e34dccb0 con 0x5605e350d1f0 2024-09-12T03:19:34.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.870+0000 7faa94ff9640 10 client.5811 handle_command_reply: tid=0 2024-09-12T03:19:34.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.870+0000 7faabaa79740 1 client.5811 shutdown 2024-09-12T03:19:34.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.870+0000 7faabaa79740 1 -- 172.21.15.27:0/734543160 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5605e350c7e0 msgr2=0x5605e3522480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:34.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.870+0000 7faabaa79740 1 --2- 172.21.15.27:0/734543160 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5605e350c7e0 0x5605e3522480 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7faaa8009930 tx=0x7faaa802c070 comp rx=0 tx=0).stop 2024-09-12T03:19:34.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.870+0000 7faabaa79740 1 -- 172.21.15.27:0/734543160 shutdown_connections 2024-09-12T03:19:34.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.870+0000 7faabaa79740 1 --2- 172.21.15.27:0/734543160 >> [v2:172.21.15.45:6835/3720118075,v1:172.21.15.45:6837/3720118075] conn(0x5605e350d1f0 0x5605e35a7770 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:34.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.870+0000 7faabaa79740 1 --2- 172.21.15.27:0/734543160 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5605e350c7e0 0x5605e3522480 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:34.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.870+0000 7faa94ff9640 0 client.5811 ms_handle_reset on v2:172.21.15.45:6835/3720118075 2024-09-12T03:19:34.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.870+0000 7faabaa79740 1 --2- 172.21.15.27:0/734543160 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x5605e34e2c40 0x5605e358d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:34.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.870+0000 7faabaa79740 1 --2- 172.21.15.27:0/734543160 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5605e34e14b0 0x5605e351f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:34.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.870+0000 7faabaa79740 1 -- 172.21.15.27:0/734543160 >> 172.21.15.27:0/734543160 conn(0x5605e34dec70 msgr2=0x5605e35200c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:34.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.870+0000 7faabaa79740 1 -- 172.21.15.27:0/734543160 shutdown_connections 2024-09-12T03:19:34.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.870+0000 7faabaa79740 1 -- 172.21.15.27:0/734543160 wait complete. 2024-09-12T03:19:34.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.870+0000 7faabaa79740 20 client.5811 trim_cache size 0 max 16384 2024-09-12T03:19:34.878 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-12T03:19:34.878 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_fsid": "d2df504e-2647-4385-a5db-9fdf69f487a0", 2024-09-12T03:19:34.878 INFO:teuthology.orchestra.run.smithi027.stdout: "whoami": 0, 2024-09-12T03:19:34.878 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 5649, 2024-09-12T03:19:34.878 INFO:teuthology.orchestra.run.smithi027.stdout: "want_state": "up:active", 2024-09-12T03:19:34.878 INFO:teuthology.orchestra.run.smithi027.stdout: "state": "up:active", 2024-09-12T03:19:34.878 INFO:teuthology.orchestra.run.smithi027.stdout: "fs_name": "cephfs", 2024-09-12T03:19:34.878 INFO:teuthology.orchestra.run.smithi027.stdout: "rank_uptime": 9.0275686660000005, 2024-09-12T03:19:34.878 INFO:teuthology.orchestra.run.smithi027.stdout: "mdsmap_epoch": 30, 2024-09-12T03:19:34.878 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch": 56, 2024-09-12T03:19:34.879 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch_barrier": 56, 2024-09-12T03:19:34.879 INFO:teuthology.orchestra.run.smithi027.stdout: "uptime": 13.338057513000001 2024-09-12T03:19:34.879 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-12T03:19:34.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.871+0000 7faab69a2640 1 -- 172.21.15.27:0/1639076550 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7faa9c06bfe0 msgr2=0x7faa9c06e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:34.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.871+0000 7faab69a2640 1 --2- 172.21.15.27:0/1639076550 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7faa9c06bfe0 0x7faa9c06e4a0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7faab0161850 tx=0x7faaa0009830 comp rx=0 tx=0).stop 2024-09-12T03:19:34.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.871+0000 7faab69a2640 1 -- 172.21.15.27:0/1639076550 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faab00d3fc0 msgr2=0x7faab0160b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:34.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.871+0000 7faab69a2640 1 --2- 172.21.15.27:0/1639076550 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faab00d3fc0 0x7faab0160b50 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7faaa400c400 tx=0x7faaa400c8c0 comp rx=0 tx=0).stop 2024-09-12T03:19:34.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.871+0000 7faab69a2640 1 -- 172.21.15.27:0/1639076550 shutdown_connections 2024-09-12T03:19:34.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.871+0000 7faab69a2640 1 --2- 172.21.15.27:0/1639076550 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7faa9c06bfe0 0x7faa9c06e4a0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:34.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.871+0000 7faab69a2640 1 --2- 172.21.15.27:0/1639076550 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7faab015e770 0x7faab0161090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:34.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.871+0000 7faab69a2640 1 --2- 172.21.15.27:0/1639076550 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faab00d3fc0 0x7faab0160b50 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:34.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.871+0000 7faab69a2640 1 --2- 172.21.15.27:0/1639076550 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faab00d17a0 0x7faab00cd160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:34.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.871+0000 7faab69a2640 1 -- 172.21.15.27:0/1639076550 >> 172.21.15.27:0/1639076550 conn(0x7faab00b6100 msgr2=0x7faab01e8c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:34.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.871+0000 7faab69a2640 1 -- 172.21.15.27:0/1639076550 shutdown_connections 2024-09-12T03:19:34.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.871+0000 7faab69a2640 1 -- 172.21.15.27:0/1639076550 wait complete. 2024-09-12T03:19:34.883 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-12T03:19:34.986 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.985+0000 7f4be534b640 1 Processor -- start 2024-09-12T03:19:34.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.985+0000 7f4be534b640 1 -- start start 2024-09-12T03:19:34.990 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.987+0000 7f4be534b640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f4be015e780 0x7f4be00d3e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:34.990 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.987+0000 7f4be534b640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4be00d43b0 0x7f4be00d1280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:34.990 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.987+0000 7f4be534b640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4be00b6690 con 0x7f4be015eb60 2024-09-12T03:19:34.990 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.987+0000 7f4be534b640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f4be00b7950 con 0x7f4be015e780 2024-09-12T03:19:34.990 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.987+0000 7f4be534b640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f4be00b7210 con 0x7f4be00d43b0 2024-09-12T03:19:34.990 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.987+0000 7f4bd7fff640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4be00d43b0 0x7f4be00d1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:34.990 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.987+0000 7f4bdeffd640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f4be015e780 0x7f4be00d3e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:34.991 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.987+0000 7f4bd7fff640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4be00d43b0 0x7f4be00d1280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3301/0 says I am v2:172.21.15.27:47906/0 (socket says 172.21.15.27:47906) 2024-09-12T03:19:34.991 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.987+0000 7f4bd7fff640 1 -- 172.21.15.27:0/4801606 learned_addr learned my addr 172.21.15.27:0/4801606 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:19:34.991 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.987+0000 7f4bdeffd640 1 -- 172.21.15.27:0/4801606 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4be00d43b0 msgr2=0x7f4be00d1280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:34.991 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.987+0000 7f4bdeffd640 1 --2- 172.21.15.27:0/4801606 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4be00d43b0 0x7f4be00d1280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:34.991 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.987+0000 7f4bdeffd640 1 -- 172.21.15.27:0/4801606 >> v1:172.21.15.45:6789/0 conn(0x7f4be015eb60 legacy=0x7f4be00b3390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:34.991 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.987+0000 7f4bde7fc640 1 -- 172.21.15.27:0/4801606 <== mon.1 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2480622612 0 0) 0x7f4be00b6690 con 0x7f4be015eb60 2024-09-12T03:19:34.992 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.987+0000 7f4bdeffd640 1 -- 172.21.15.27:0/4801606 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4be00b6970 con 0x7f4be015e780 2024-09-12T03:19:34.992 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.988+0000 7f4bdeffd640 1 --2- 172.21.15.27:0/4801606 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4be015e780 0x7f4be00d3e70 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f4bd00097a0 tx=0x7f4bd002a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=12dc3d3f27014635 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:34.992 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.988+0000 7f4bde7fc640 1 -- 172.21.15.27:0/4801606 <== mon.1 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bd0037070 con 0x7f4be015e780 2024-09-12T03:19:34.992 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.988+0000 7f4bde7fc640 1 -- 172.21.15.27:0/4801606 <== mon.1 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4bd0032440 con 0x7f4be015e780 2024-09-12T03:19:34.992 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.988+0000 7f4bde7fc640 1 -- 172.21.15.27:0/4801606 <== mon.1 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bd0032810 con 0x7f4be015e780 2024-09-12T03:19:34.992 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.988+0000 7f4be534b640 1 -- 172.21.15.27:0/4801606 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4be015e780 msgr2=0x7f4be00d3e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:34.992 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.988+0000 7f4be534b640 1 --2- 172.21.15.27:0/4801606 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4be015e780 0x7f4be00d3e70 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f4bd00097a0 tx=0x7f4bd002a180 comp rx=0 tx=0).stop 2024-09-12T03:19:34.992 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.988+0000 7f4be534b640 1 -- 172.21.15.27:0/4801606 shutdown_connections 2024-09-12T03:19:34.992 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.988+0000 7f4be534b640 1 --2- 172.21.15.27:0/4801606 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4be00d43b0 0x7f4be00d1280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:34.992 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.988+0000 7f4be534b640 1 --2- 172.21.15.27:0/4801606 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4be015e780 0x7f4be00d3e70 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:34.993 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.988+0000 7f4be534b640 1 -- 172.21.15.27:0/4801606 >> 172.21.15.27:0/4801606 conn(0x7f4be006da60 msgr2=0x7f4be006de70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:34.993 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.988+0000 7f4be534b640 1 -- 172.21.15.27:0/4801606 shutdown_connections 2024-09-12T03:19:34.993 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.988+0000 7f4be534b640 1 -- 172.21.15.27:0/4801606 wait complete. 2024-09-12T03:19:34.998 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.990+0000 7f4be534b640 1 Processor -- start 2024-09-12T03:19:34.999 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.990+0000 7f4be534b640 1 -- start start 2024-09-12T03:19:34.999 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.990+0000 7f4be534b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4be00d43b0 0x7f4be00ae0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:34.999 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.990+0000 7f4be534b640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4be015e780 0x7f4be015c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:34.999 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.990+0000 7f4be534b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4be015eb60 0x7f4be00abeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:34.999 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.990+0000 7f4be534b640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f4be00b6340 con 0x7f4be015eb60 2024-09-12T03:19:34.999 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.990+0000 7f4be534b640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f4be00b6f30 con 0x7f4be00d43b0 2024-09-12T03:19:34.999 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.990+0000 7f4be534b640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f4be00d17c0 con 0x7f4be015e780 2024-09-12T03:19:34.999 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.990+0000 7f4bdf7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4be015eb60 0x7f4be00abeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:35.000 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.990+0000 7f4bdf7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4be015eb60 0x7f4be00abeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38486/0 (socket says 172.21.15.27:38486) 2024-09-12T03:19:35.000 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.990+0000 7f4bdf7fe640 1 -- 172.21.15.27:0/2680565778 learned_addr learned my addr 172.21.15.27:0/2680565778 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:19:35.000 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.990+0000 7f4bdeffd640 1 --2- 172.21.15.27:0/2680565778 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4be00d43b0 0x7f4be00ae0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:35.000 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.991+0000 7f4bd7fff640 1 --2- 172.21.15.27:0/2680565778 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4be015e780 0x7f4be015c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:35.000 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.991+0000 7f4bdf7fe640 1 -- 172.21.15.27:0/2680565778 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4be015e780 msgr2=0x7f4be015c0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:35.000 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.991+0000 7f4bdf7fe640 1 --2- 172.21.15.27:0/2680565778 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4be015e780 0x7f4be015c0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:35.000 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.991+0000 7f4bdf7fe640 1 -- 172.21.15.27:0/2680565778 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4be00d43b0 msgr2=0x7f4be00ae0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:35.000 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.991+0000 7f4bdf7fe640 1 --2- 172.21.15.27:0/2680565778 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4be00d43b0 0x7f4be00ae0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:35.000 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.991+0000 7f4bdf7fe640 1 -- 172.21.15.27:0/2680565778 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4be00cd3c0 con 0x7f4be015eb60 2024-09-12T03:19:35.000 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.991+0000 7f4bdf7fe640 1 --2- 172.21.15.27:0/2680565778 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4be015eb60 0x7f4be00abeb0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f4bd800c7b0 tx=0x7f4bd800cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:35.000 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.991+0000 7f4bd77fe640 1 -- 172.21.15.27:0/2680565778 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bd8003ba0 con 0x7f4be015eb60 2024-09-12T03:19:35.000 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.991+0000 7f4bd77fe640 1 -- 172.21.15.27:0/2680565778 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4bd8003d40 con 0x7f4be015eb60 2024-09-12T03:19:35.000 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.991+0000 7f4be534b640 1 -- 172.21.15.27:0/2680565778 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4be00cd660 con 0x7f4be015eb60 2024-09-12T03:19:35.001 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.991+0000 7f4bd77fe640 1 -- 172.21.15.27:0/2680565778 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bd8005270 con 0x7f4be015eb60 2024-09-12T03:19:35.001 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.991+0000 7f4be534b640 1 -- 172.21.15.27:0/2680565778 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4be00cdc20 con 0x7f4be015eb60 2024-09-12T03:19:35.001 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.992+0000 7f4be534b640 1 -- 172.21.15.27:0/2680565778 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4be00d1ba0 con 0x7f4be015eb60 2024-09-12T03:19:35.001 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.993+0000 7f4bd77fe640 1 -- 172.21.15.27:0/2680565778 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 9) ==== 90651+0+0 (secure 0 0 0) 0x7f4bd8005410 con 0x7f4be015eb60 2024-09-12T03:19:35.001 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.993+0000 7f4bd77fe640 1 --2- 172.21.15.27:0/2680565778 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f4bc006bfe0 0x7f4bc006e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:35.001 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.993+0000 7f4bd77fe640 1 -- 172.21.15.27:0/2680565778 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5709+0+0 (secure 0 0 0) 0x7f4bd808ae20 con 0x7f4be015eb60 2024-09-12T03:19:35.001 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.993+0000 7f4bdeffd640 1 --2- 172.21.15.27:0/2680565778 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f4bc006bfe0 0x7f4bc006e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:35.001 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.994+0000 7f4bdeffd640 1 --2- 172.21.15.27:0/2680565778 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f4bc006bfe0 0x7f4bc006e4a0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f4bd0034000 tx=0x7f4bd002e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:35.001 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:34.997+0000 7f4bd77fe640 1 -- 172.21.15.27:0/2680565778 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f4bd8058f90 con 0x7f4be015eb60 2024-09-12T03:19:35.211 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:35.210+0000 7f4be534b640 1 -- 172.21.15.27:0/2680565778 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f4be006de40 con 0x7f4be015eb60 2024-09-12T03:19:35.211 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:35.210+0000 7f4bd77fe640 1 -- 172.21.15.27:0/2680565778 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v30) ==== 53+0+73 (secure 0 0 0) 0x7f4bd805d290 con 0x7f4be015eb60 2024-09-12T03:19:35.212 INFO:teuthology.orchestra.run.smithi027.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-12T03:19:35.213 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:35.212+0000 7f4be534b640 1 -- 172.21.15.27:0/2680565778 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f4bc006bfe0 msgr2=0x7f4bc006e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:35.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:35.212+0000 7f4be534b640 1 --2- 172.21.15.27:0/2680565778 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f4bc006bfe0 0x7f4bc006e4a0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f4bd0034000 tx=0x7f4bd002e000 comp rx=0 tx=0).stop 2024-09-12T03:19:35.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:35.212+0000 7f4be534b640 1 -- 172.21.15.27:0/2680565778 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4be015eb60 msgr2=0x7f4be00abeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:35.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:35.212+0000 7f4be534b640 1 --2- 172.21.15.27:0/2680565778 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4be015eb60 0x7f4be00abeb0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f4bd800c7b0 tx=0x7f4bd800cbb0 comp rx=0 tx=0).stop 2024-09-12T03:19:35.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:35.212+0000 7f4be534b640 1 -- 172.21.15.27:0/2680565778 shutdown_connections 2024-09-12T03:19:35.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:35.212+0000 7f4be534b640 1 --2- 172.21.15.27:0/2680565778 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f4bc006bfe0 0x7f4bc006e4a0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:35.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:35.212+0000 7f4be534b640 1 --2- 172.21.15.27:0/2680565778 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4be015eb60 0x7f4be00abeb0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:35.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:35.212+0000 7f4be534b640 1 --2- 172.21.15.27:0/2680565778 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4be015e780 0x7f4be015c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:35.215 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:35.212+0000 7f4be534b640 1 --2- 172.21.15.27:0/2680565778 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4be00d43b0 0x7f4be00ae0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:35.215 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:35.212+0000 7f4be534b640 1 -- 172.21.15.27:0/2680565778 >> 172.21.15.27:0/2680565778 conn(0x7f4be006da60 msgr2=0x7f4be00cf270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:35.215 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:35.212+0000 7f4be534b640 1 -- 172.21.15.27:0/2680565778 shutdown_connections 2024-09-12T03:19:35.215 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:35.212+0000 7f4be534b640 1 -- 172.21.15.27:0/2680565778 wait complete. 2024-09-12T03:19:35.225 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-12T03:19:35.225 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-12T03:19:35.225 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi027.front.sepia.ceph.com 2024-09-12T03:19:35.225 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-12T03:19:35.225 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-12T03:19:35.225 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-12T03:19:35.225 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-12T03:19:35.225 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-12T03:19:35.225 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-12T03:19:35.225 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:19:35.226 DEBUG:teuthology.orchestra.run.smithi027:> ip addr 2024-09-12T03:19:35.242 INFO:teuthology.orchestra.run.smithi027.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-12T03:19:35.242 INFO:teuthology.orchestra.run.smithi027.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-12T03:19:35.242 INFO:teuthology.orchestra.run.smithi027.stdout: inet 127.0.0.1/8 scope host lo 2024-09-12T03:19:35.242 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:19:35.242 INFO:teuthology.orchestra.run.smithi027.stdout: inet6 ::1/128 scope host 2024-09-12T03:19:35.242 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:19:35.242 INFO:teuthology.orchestra.run.smithi027.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-12T03:19:35.242 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether 0c:c4:7a:6b:fe:c8 brd ff:ff:ff:ff:ff:ff 2024-09-12T03:19:35.242 INFO:teuthology.orchestra.run.smithi027.stdout: altname enp7s0f0 2024-09-12T03:19:35.242 INFO:teuthology.orchestra.run.smithi027.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-12T03:19:35.242 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether 0c:c4:7a:6b:fe:c9 brd ff:ff:ff:ff:ff:ff 2024-09-12T03:19:35.242 INFO:teuthology.orchestra.run.smithi027.stdout: altname enp7s0f1 2024-09-12T03:19:35.243 INFO:teuthology.orchestra.run.smithi027.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-12T03:19:35.243 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether ac:1f:6b:f8:39:5a brd ff:ff:ff:ff:ff:ff 2024-09-12T03:19:35.243 INFO:teuthology.orchestra.run.smithi027.stdout: altname enp1s0f0 2024-09-12T03:19:35.243 INFO:teuthology.orchestra.run.smithi027.stdout: inet 172.21.15.27/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-12T03:19:35.243 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft 42613sec preferred_lft 42613sec 2024-09-12T03:19:35.243 INFO:teuthology.orchestra.run.smithi027.stdout: inet6 fe80::ae1f:6bff:fef8:395a/64 scope link 2024-09-12T03:19:35.243 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:19:35.243 INFO:teuthology.orchestra.run.smithi027.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-12T03:19:35.243 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether ac:1f:6b:f8:39:5b brd ff:ff:ff:ff:ff:ff 2024-09-12T03:19:35.243 INFO:teuthology.orchestra.run.smithi027.stdout: altname enp1s0f1 2024-09-12T03:19:35.243 INFO:teuthology.orchestra.run.smithi027.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-12T03:19:35.243 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether 2e:7f:89:6c:33:a0 brd ff:ff:ff:ff:ff:ff 2024-09-12T03:19:35.243 INFO:teuthology.orchestra.run.smithi027.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-12T03:19:35.244 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:19:35.244 INFO:teuthology.orchestra.run.smithi027.stdout: inet6 fe80::c84b:19ff:fef0:ca74/64 scope link 2024-09-12T03:19:35.244 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:19:35.244 INFO:teuthology.orchestra.run.smithi027.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-12T03:19:35.244 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether 42:40:d7:5c:7a:8e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-12T03:19:35.244 INFO:teuthology.orchestra.run.smithi027.stdout: inet6 fe80::4040:d7ff:fe5c:7a8e/64 scope link 2024-09-12T03:19:35.244 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:19:35.244 INFO:teuthology.orchestra.run.smithi027.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-12T03:19:35.244 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether 2e:7f:89:6c:33:a0 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-12T03:19:35.244 INFO:teuthology.orchestra.run.smithi027.stdout: inet6 fe80::2c7f:89ff:fe6c:33a0/64 scope link 2024-09-12T03:19:35.244 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:19:35.245 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:19:35.245 DEBUG:teuthology.orchestra.run.smithi027:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-12T03:19:35.306 INFO:teuthology.orchestra.run.smithi027.stdout:1 2024-09-12T03:19:35.308 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:19:35.308 DEBUG:teuthology.orchestra.run.smithi027:> route 2024-09-12T03:19:35.364 INFO:teuthology.orchestra.run.smithi027.stdout:Kernel IP routing table 2024-09-12T03:19:35.364 INFO:teuthology.orchestra.run.smithi027.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-12T03:19:35.365 INFO:teuthology.orchestra.run.smithi027.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-12T03:19:35.365 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-12T03:19:35.365 INFO:teuthology.orchestra.run.smithi027.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-12T03:19:35.365 DEBUG:teuthology.orchestra.run.smithi027:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-12T03:19:35.365 DEBUG:teuthology.orchestra.run.smithi027:> set -e 2024-09-12T03:19:35.366 DEBUG:teuthology.orchestra.run.smithi027:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-12T03:19:35.366 DEBUG:teuthology.orchestra.run.smithi027:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-12T03:19:35.366 DEBUG:teuthology.orchestra.run.smithi027:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-12T03:19:35.366 DEBUG:teuthology.orchestra.run.smithi027:> ') 2024-09-12T03:19:35.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:35 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T03:19:35.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T03:19:35.494 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:19:35.494 DEBUG:teuthology.orchestra.run.smithi027:> ip netns list 2024-09-12T03:19:35.510 INFO:teuthology.orchestra.run.smithi027.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-12T03:19:35.510 INFO:teuthology.orchestra.run.smithi027.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-12T03:19:35.510 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-12T03:19:35.510 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:19:35.510 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip link set brx.0 up 2024-09-12T03:19:35.576 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-12T03:19:35.576 DEBUG:teuthology.orchestra.run.smithi027:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-12T03:19:35.632 INFO:teuthology.orchestra.run.smithi027.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-12T03:19:35.632 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-12T03:19:35.632 DEBUG:teuthology.orchestra.run.smithi027:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-12T03:19:35.687 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-09-12T03:19:35.688 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:19:35.688 DEBUG:teuthology.orchestra.run.smithi027:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-12T03:19:35.772 INFO:teuthology.orchestra.run.smithi027.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-12T03:19:35.773 INFO:teuthology.orchestra.run.smithi027.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-12T03:19:35.787 INFO:teuthology.orchestra.run.smithi027.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-12T03:19:35.787 INFO:teuthology.orchestra.run.smithi027.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-12T03:19:35.787 INFO:teuthology.orchestra.run.smithi027.stdout:mount.ceph: resolved to: "172.21.15.27:3300,172.21.15.45:3300,172.21.15.45:3301" 2024-09-12T03:19:35.787 INFO:teuthology.orchestra.run.smithi027.stdout:mount.ceph: trying mount with new device syntax: 0@d2df504e-2647-4385-a5db-9fdf69f487a0.cephfs=/ 2024-09-12T03:19:35.787 INFO:teuthology.orchestra.run.smithi027.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.27:3300/172.21.15.45:3300/172.21.15.45:3301" will pass to kernel 2024-09-12T03:19:35.788 INFO:teuthology.orchestra.run.smithi027.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-12T03:19:35.788 INFO:teuthology.orchestra.run.smithi027.stdout: You just mounted a file system that supports labels which does not 2024-09-12T03:19:35.788 INFO:teuthology.orchestra.run.smithi027.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-12T03:19:35.788 INFO:teuthology.orchestra.run.smithi027.stdout: applications will generate AVC messages and not be allowed access to 2024-09-12T03:19:35.788 INFO:teuthology.orchestra.run.smithi027.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-12T03:19:35.789 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-12T03:19:35.789 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:19:35.789 DEBUG:teuthology.orchestra.run.smithi027:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-12T03:19:35.815 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-12T03:19:35.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:35 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-12T03:19:35.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T03:19:35.920 INFO:teuthology.orchestra.run.smithi027.stdout:client5524 2024-09-12T03:19:35.920 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-12T03:19:35.920 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T03:19:35.938 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-12T03:19:36.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:36.082+0000 7fbb1ac4c640 1 Processor -- start 2024-09-12T03:19:36.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:36.082+0000 7fbb1ac4c640 1 -- start start 2024-09-12T03:19:36.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:36.083+0000 7fbb1ac4c640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fbb140ce6d0 0x7fbb140ceab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:36.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:36.083+0000 7fbb1ac4c640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fbb14160d80 0x7fbb1406bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:36.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:36.083+0000 7fbb1ac4c640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbb141463d0 con 0x7fbb1406c070 2024-09-12T03:19:36.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:36.083+0000 7fbb1ac4c640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7fbb140ceff0 con 0x7fbb14160d80 2024-09-12T03:19:36.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:36.083+0000 7fbb1ac4c640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fbb1406c520 con 0x7fbb140ce6d0 2024-09-12T03:19:36.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:36.083+0000 7fbb13fff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fbb14160d80 0x7fbb1406bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:36.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:36.083+0000 7fbb13fff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fbb14160d80 0x7fbb1406bb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38492/0 (socket says 172.21.15.27:38492) 2024-09-12T03:19:36.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:36.083+0000 7fbb13fff640 1 -- 172.21.15.27:0/801780807 learned_addr learned my addr 172.21.15.27:0/801780807 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:19:36.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:36.083+0000 7fbb189e8640 1 --2- 172.21.15.27:0/801780807 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fbb140ce6d0 0x7fbb140ceab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:36.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:36.083+0000 7fbb13fff640 1 -- 172.21.15.27:0/801780807 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fbb140ce6d0 msgr2=0x7fbb140ceab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:36.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:36.083+0000 7fbb13fff640 1 --2- 172.21.15.27:0/801780807 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fbb140ce6d0 0x7fbb140ceab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:36.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:36.083+0000 7fbb13fff640 1 -- 172.21.15.27:0/801780807 >> v1:172.21.15.45:6789/0 conn(0x7fbb1406c070 legacy=0x7fbb1406e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:36.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:36.083+0000 7fbb13fff640 1 -- 172.21.15.27:0/801780807 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb1406c7f0 con 0x7fbb14160d80 2024-09-12T03:19:36.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:36.083+0000 7fbb189e8640 1 --2- 172.21.15.27:0/801780807 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fbb140ce6d0 0x7fbb140ceab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:19:36.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:36.084+0000 7fbb13fff640 1 --2- 172.21.15.27:0/801780807 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbb14160d80 0x7fbb1406bb30 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fbafc009e50 tx=0x7fbafc02a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d43769dbb33e5d0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:36.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:36.084+0000 7fbb137fe640 1 -- 172.21.15.27:0/801780807 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbafc036070 con 0x7fbb14160d80 2024-09-12T03:19:36.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:36.084+0000 7fbb137fe640 1 -- 172.21.15.27:0/801780807 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbafc031600 con 0x7fbb14160d80 2024-09-12T03:19:36.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:36.084+0000 7fbb137fe640 1 -- 172.21.15.27:0/801780807 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbafc0319d0 con 0x7fbb14160d80 2024-09-12T03:19:36.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:36.085+0000 7fbb1ac4c640 1 -- 172.21.15.27:0/801780807 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbb14160d80 msgr2=0x7fbb1406bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:36.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:36.085+0000 7fbb1ac4c640 1 --2- 172.21.15.27:0/801780807 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbb14160d80 0x7fbb1406bb30 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fbafc009e50 tx=0x7fbafc02a680 comp rx=0 tx=0).stop 2024-09-12T03:19:36.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:36.085+0000 7fbb1ac4c640 1 -- 172.21.15.27:0/801780807 shutdown_connections 2024-09-12T03:19:36.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:36.085+0000 7fbb1ac4c640 1 --2- 172.21.15.27:0/801780807 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbb14160d80 0x7fbb1406bb30 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:36.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:36.085+0000 7fbb1ac4c640 1 --2- 172.21.15.27:0/801780807 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fbb140ce6d0 0x7fbb140ceab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:36.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:36.085+0000 7fbb1ac4c640 1 -- 172.21.15.27:0/801780807 >> 172.21.15.27:0/801780807 conn(0x7fbb1406cc40 msgr2=0x7fbb1406d050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:36.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:36.085+0000 7fbb1ac4c640 1 -- 172.21.15.27:0/801780807 shutdown_connections 2024-09-12T03:19:36.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:36.085+0000 7fbb1ac4c640 1 -- 172.21.15.27:0/801780807 wait complete. 2024-09-12T03:19:36.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:36.086+0000 7fbb1ac4c640 1 Processor -- start 2024-09-12T03:19:36.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:36.086+0000 7fbb1ac4c640 1 -- start start 2024-09-12T03:19:36.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:36.087+0000 7fbb1ac4c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbb1406c070 0x7fbb141f23e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:36.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:36.087+0000 7fbb1ac4c640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fbb140ce6d0 0x7fbb14132f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:36.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:36.087+0000 7fbb1ac4c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbb14160d80 0x7fbb141f5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:36.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:36.087+0000 7fbb1ac4c640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fbb14160860 con 0x7fbb14160d80 2024-09-12T03:19:36.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:36.087+0000 7fbb1ac4c640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fbb14160580 con 0x7fbb1406c070 2024-09-12T03:19:36.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:36.087+0000 7fbb1ac4c640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fbb1406ecc0 con 0x7fbb140ce6d0 2024-09-12T03:19:36.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:36.087+0000 7fbb191e9640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbb14160d80 0x7fbb141f5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:36.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:36.087+0000 7fbb191e9640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbb14160d80 0x7fbb141f5250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38500/0 (socket says 172.21.15.27:38500) 2024-09-12T03:19:36.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:36.087+0000 7fbb191e9640 1 -- 172.21.15.27:0/2678821419 learned_addr learned my addr 172.21.15.27:0/2678821419 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:19:36.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:36.087+0000 7fbb13fff640 1 --2- 172.21.15.27:0/2678821419 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fbb140ce6d0 0x7fbb14132f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:36.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:36.087+0000 7fbb189e8640 1 --2- 172.21.15.27:0/2678821419 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbb1406c070 0x7fbb141f23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:36.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:36.087+0000 7fbb191e9640 1 -- 172.21.15.27:0/2678821419 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fbb140ce6d0 msgr2=0x7fbb14132f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:36.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:36.087+0000 7fbb191e9640 1 --2- 172.21.15.27:0/2678821419 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fbb140ce6d0 0x7fbb14132f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:36.090 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:36.087+0000 7fbb191e9640 1 -- 172.21.15.27:0/2678821419 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbb1406c070 msgr2=0x7fbb141f23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:36.090 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:36.087+0000 7fbb191e9640 1 --2- 172.21.15.27:0/2678821419 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbb1406c070 0x7fbb141f23e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:36.090 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:36.087+0000 7fbb191e9640 1 -- 172.21.15.27:0/2678821419 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb14133670 con 0x7fbb14160d80 2024-09-12T03:19:36.090 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:36.088+0000 7fbb191e9640 1 --2- 172.21.15.27:0/2678821419 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbb14160d80 0x7fbb141f5250 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c001eb0 tx=0x7fbb0c00c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:36.090 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:36.088+0000 7fbb117fa640 1 -- 172.21.15.27:0/2678821419 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb0c017070 con 0x7fbb14160d80 2024-09-12T03:19:36.090 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:36.088+0000 7fbb117fa640 1 -- 172.21.15.27:0/2678821419 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb0c0052d0 con 0x7fbb14160d80 2024-09-12T03:19:36.090 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:36.088+0000 7fbb117fa640 1 -- 172.21.15.27:0/2678821419 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb0c003810 con 0x7fbb14160d80 2024-09-12T03:19:36.090 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:36.088+0000 7fbb1ac4c640 1 -- 172.21.15.27:0/2678821419 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb14133950 con 0x7fbb14160d80 2024-09-12T03:19:36.090 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:36.088+0000 7fbb1ac4c640 1 -- 172.21.15.27:0/2678821419 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb14133f10 con 0x7fbb14160d80 2024-09-12T03:19:36.090 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:36.089+0000 7fbb1ac4c640 1 -- 172.21.15.27:0/2678821419 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb1406f0f0 con 0x7fbb14160d80 2024-09-12T03:19:36.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:36.089+0000 7fbb117fa640 1 -- 172.21.15.27:0/2678821419 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 9) ==== 90651+0+0 (secure 0 0 0) 0x7fbb0c01a070 con 0x7fbb14160d80 2024-09-12T03:19:36.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:36.090+0000 7fbb117fa640 1 --2- 172.21.15.27:0/2678821419 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fbaf406bfe0 0x7fbaf406e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:36.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:36.090+0000 7fbb117fa640 1 -- 172.21.15.27:0/2678821419 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5709+0+0 (secure 0 0 0) 0x7fbb0c08a340 con 0x7fbb14160d80 2024-09-12T03:19:36.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:36.090+0000 7fbb189e8640 1 --2- 172.21.15.27:0/2678821419 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fbaf406bfe0 0x7fbaf406e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:36.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:36.091+0000 7fbb189e8640 1 --2- 172.21.15.27:0/2678821419 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fbaf406bfe0 0x7fbaf406e4a0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fbb04006fd0 tx=0x7fbb04008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:36.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:36.094+0000 7fbb117fa640 1 -- 172.21.15.27:0/2678821419 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fbb0c058560 con 0x7fbb14160d80 2024-09-12T03:19:36.307 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:36.307+0000 7fbb1ac4c640 1 -- 172.21.15.27:0/2678821419 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fbb1406e390 con 0x7fbb14160d80 2024-09-12T03:19:36.307 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:36.307+0000 7fbb117fa640 1 -- 172.21.15.27:0/2678821419 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v30) ==== 53+0+73 (secure 0 0 0) 0x7fbb0c05c860 con 0x7fbb14160d80 2024-09-12T03:19:36.308 INFO:teuthology.orchestra.run.smithi027.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-12T03:19:36.310 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:36.310+0000 7fbb1ac4c640 1 -- 172.21.15.27:0/2678821419 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fbaf406bfe0 msgr2=0x7fbaf406e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:36.310 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:36.310+0000 7fbb1ac4c640 1 --2- 172.21.15.27:0/2678821419 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fbaf406bfe0 0x7fbaf406e4a0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fbb04006fd0 tx=0x7fbb04008040 comp rx=0 tx=0).stop 2024-09-12T03:19:36.310 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:36.310+0000 7fbb1ac4c640 1 -- 172.21.15.27:0/2678821419 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbb14160d80 msgr2=0x7fbb141f5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:36.310 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:36.310+0000 7fbb1ac4c640 1 --2- 172.21.15.27:0/2678821419 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbb14160d80 0x7fbb141f5250 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c001eb0 tx=0x7fbb0c00c890 comp rx=0 tx=0).stop 2024-09-12T03:19:36.310 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:36.310+0000 7fbb1ac4c640 1 -- 172.21.15.27:0/2678821419 shutdown_connections 2024-09-12T03:19:36.310 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:36.310+0000 7fbb1ac4c640 1 --2- 172.21.15.27:0/2678821419 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fbaf406bfe0 0x7fbaf406e4a0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:36.310 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:36.310+0000 7fbb1ac4c640 1 --2- 172.21.15.27:0/2678821419 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbb14160d80 0x7fbb141f5250 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:36.310 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:36.310+0000 7fbb1ac4c640 1 --2- 172.21.15.27:0/2678821419 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fbb140ce6d0 0x7fbb14132f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:36.311 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:36.310+0000 7fbb1ac4c640 1 --2- 172.21.15.27:0/2678821419 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbb1406c070 0x7fbb141f23e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:36.311 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:36.310+0000 7fbb1ac4c640 1 -- 172.21.15.27:0/2678821419 >> 172.21.15.27:0/2678821419 conn(0x7fbb1406cc40 msgr2=0x7fbb140d0620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:36.311 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:36.310+0000 7fbb1ac4c640 1 -- 172.21.15.27:0/2678821419 shutdown_connections 2024-09-12T03:19:36.311 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:36.310+0000 7fbb1ac4c640 1 -- 172.21.15.27:0/2678821419 wait complete. 2024-09-12T03:19:36.320 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-12T03:19:36.321 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-12T03:19:36.321 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi045.front.sepia.ceph.com 2024-09-12T03:19:36.321 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-12T03:19:36.321 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-12T03:19:36.321 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-12T03:19:36.321 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-12T03:19:36.321 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-12T03:19:36.321 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-12T03:19:36.321 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:19:36.321 DEBUG:teuthology.orchestra.run.smithi045:> ip addr 2024-09-12T03:19:36.341 INFO:teuthology.orchestra.run.smithi045.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-12T03:19:36.341 INFO:teuthology.orchestra.run.smithi045.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-12T03:19:36.341 INFO:teuthology.orchestra.run.smithi045.stdout: inet 127.0.0.1/8 scope host lo 2024-09-12T03:19:36.342 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:19:36.342 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 ::1/128 scope host 2024-09-12T03:19:36.342 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:19:36.342 INFO:teuthology.orchestra.run.smithi045.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-12T03:19:36.342 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:6b:fd:a6 brd ff:ff:ff:ff:ff:ff 2024-09-12T03:19:36.342 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp7s0f0 2024-09-12T03:19:36.342 INFO:teuthology.orchestra.run.smithi045.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-12T03:19:36.342 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:6b:fd:a7 brd ff:ff:ff:ff:ff:ff 2024-09-12T03:19:36.342 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp7s0f1 2024-09-12T03:19:36.342 INFO:teuthology.orchestra.run.smithi045.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-12T03:19:36.342 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:bd:12:be brd ff:ff:ff:ff:ff:ff 2024-09-12T03:19:36.342 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp1s0f0 2024-09-12T03:19:36.342 INFO:teuthology.orchestra.run.smithi045.stdout: inet 172.21.15.45/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-12T03:19:36.343 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft 42557sec preferred_lft 42557sec 2024-09-12T03:19:36.343 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::ec4:7aff:febd:12be/64 scope link 2024-09-12T03:19:36.343 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:19:36.343 INFO:teuthology.orchestra.run.smithi045.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-12T03:19:36.343 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:bd:12:bf brd ff:ff:ff:ff:ff:ff 2024-09-12T03:19:36.343 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp1s0f1 2024-09-12T03:19:36.343 INFO:teuthology.orchestra.run.smithi045.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-12T03:19:36.343 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 52:97:33:5c:60:1b brd ff:ff:ff:ff:ff:ff 2024-09-12T03:19:36.343 INFO:teuthology.orchestra.run.smithi045.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-12T03:19:36.343 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:19:36.343 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::d8c2:5dff:fe26:78c8/64 scope link 2024-09-12T03:19:36.343 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:19:36.343 INFO:teuthology.orchestra.run.smithi045.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-12T03:19:36.343 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 5a:ec:d7:40:ef:75 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-12T03:19:36.344 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::58ec:d7ff:fe40:ef75/64 scope link 2024-09-12T03:19:36.344 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:19:36.344 INFO:teuthology.orchestra.run.smithi045.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-12T03:19:36.344 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 52:97:33:5c:60:1b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-12T03:19:36.344 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::5097:33ff:fe5c:601b/64 scope link 2024-09-12T03:19:36.344 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:19:36.344 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:19:36.344 DEBUG:teuthology.orchestra.run.smithi045:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-12T03:19:36.410 INFO:teuthology.orchestra.run.smithi045.stdout:1 2024-09-12T03:19:36.412 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:19:36.412 DEBUG:teuthology.orchestra.run.smithi045:> route 2024-09-12T03:19:36.472 INFO:teuthology.orchestra.run.smithi045.stdout:Kernel IP routing table 2024-09-12T03:19:36.473 INFO:teuthology.orchestra.run.smithi045.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-12T03:19:36.473 INFO:teuthology.orchestra.run.smithi045.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-12T03:19:36.473 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-12T03:19:36.473 INFO:teuthology.orchestra.run.smithi045.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-12T03:19:36.473 DEBUG:teuthology.orchestra.run.smithi045:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-12T03:19:36.473 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-09-12T03:19:36.473 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-12T03:19:36.473 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-12T03:19:36.473 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-12T03:19:36.473 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-09-12T03:19:36.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:19:36 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T03:19:36.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:19:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T03:19:36.613 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:19:36.613 DEBUG:teuthology.orchestra.run.smithi045:> ip netns list 2024-09-12T03:19:36.629 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-09-12T03:19:36.630 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-12T03:19:36.630 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-12T03:19:36.630 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:19:36.630 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link set brx.0 up 2024-09-12T03:19:36.696 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-12T03:19:36.696 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-12T03:19:36.753 INFO:teuthology.orchestra.run.smithi045.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-12T03:19:36.753 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-12T03:19:36.753 DEBUG:teuthology.orchestra.run.smithi045:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-12T03:19:36.809 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2024-09-12T03:19:36.809 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:19:36.810 DEBUG:teuthology.orchestra.run.smithi045:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-12T03:19:36.895 INFO:teuthology.orchestra.run.smithi045.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-12T03:19:36.895 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-12T03:19:36.902 INFO:teuthology.orchestra.run.smithi045.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-12T03:19:36.903 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-12T03:19:36.903 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: resolved to: "172.21.15.27:3300,172.21.15.45:3300,172.21.15.45:3301" 2024-09-12T03:19:36.903 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: trying mount with new device syntax: 1@d2df504e-2647-4385-a5db-9fdf69f487a0.cephfs=/ 2024-09-12T03:19:36.903 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.27:3300/172.21.15.45:3300/172.21.15.45:3301" will pass to kernel 2024-09-12T03:19:36.903 INFO:teuthology.orchestra.run.smithi045.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-09-12T03:19:36.903 INFO:teuthology.orchestra.run.smithi045.stdout: You just mounted a file system that supports labels which does not 2024-09-12T03:19:36.903 INFO:teuthology.orchestra.run.smithi045.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-12T03:19:36.903 INFO:teuthology.orchestra.run.smithi045.stdout: applications will generate AVC messages and not be allowed access to 2024-09-12T03:19:36.904 INFO:teuthology.orchestra.run.smithi045.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-12T03:19:36.905 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-12T03:19:36.905 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:19:36.905 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-12T03:19:36.930 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-12T03:19:37.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:19:37 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-12T03:19:37.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:19:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T03:19:37.028 INFO:teuthology.orchestra.run.smithi045.stdout:client5533 2024-09-12T03:19:37.028 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-12T03:19:37.028 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T03:19:37.044 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-09-12T03:19:37.159 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:37.159+0000 7f0a01227640 1 Processor -- start 2024-09-12T03:19:37.159 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:37.159+0000 7f0a01227640 1 -- start start 2024-09-12T03:19:37.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:37.160+0000 7f0a01227640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f09fc0cef40 0x7f09fc0b4000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:37.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:37.160+0000 7f0a01227640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f09fc0cf320 0x7f09fc15f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:37.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:37.160+0000 7f0a01227640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f09fc15f600 con 0x7f09fc06c650 2024-09-12T03:19:37.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:37.160+0000 7f0a01227640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f09fc06dee0 con 0x7f09fc0cef40 2024-09-12T03:19:37.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:37.160+0000 7f0a01227640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f09fc06cff0 con 0x7f09fc0cf320 2024-09-12T03:19:37.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:37.160+0000 7f09fb59e640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f09fc0cf320 0x7f09fc15f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:37.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:37.160+0000 7f09fad9d640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f09fc0cef40 0x7f09fc0b4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:37.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:37.160+0000 7f09fb59e640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f09fc0cf320 0x7f09fc15f0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3301/0 says I am v2:172.21.15.27:47948/0 (socket says 172.21.15.27:47948) 2024-09-12T03:19:37.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:37.160+0000 7f09fb59e640 1 -- 172.21.15.27:0/845708832 learned_addr learned my addr 172.21.15.27:0/845708832 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:19:37.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:37.160+0000 7f09fa59c640 1 --1- 172.21.15.27:0/845708832 >> v1:172.21.15.45:6789/0 conn(0x7f09fc06c650 0x7f09fc0710c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.27:34142/0 (socket says 172.21.15.27:34142) 2024-09-12T03:19:37.162 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:37.161+0000 7f09fb59e640 1 -- 172.21.15.27:0/845708832 >> v2:172.21.15.45:3300/0 conn(0x7f09fc0cef40 msgr2=0x7f09fc0b4000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:37.162 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:37.161+0000 7f09fb59e640 1 --2- 172.21.15.27:0/845708832 >> v2:172.21.15.45:3300/0 conn(0x7f09fc0cef40 0x7f09fc0b4000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:37.162 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:37.161+0000 7f09fb59e640 1 -- 172.21.15.27:0/845708832 >> v1:172.21.15.45:6789/0 conn(0x7f09fc06c650 legacy=0x7f09fc0710c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:37.162 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:37.161+0000 7f09fb59e640 1 -- 172.21.15.27:0/845708832 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09fc06d2d0 con 0x7f09fc0cf320 2024-09-12T03:19:37.162 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:37.161+0000 7f09fad9d640 1 --2- 172.21.15.27:0/845708832 >> v2:172.21.15.45:3300/0 conn(0x7f09fc0cef40 0x7f09fc0b4000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:19:37.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:37.161+0000 7f09fb59e640 1 --2- 172.21.15.27:0/845708832 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f09fc0cf320 0x7f09fc15f0c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f09f4009e50 tx=0x7f09f402a680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=23bb12e10ebd3ffd server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:37.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:37.162+0000 7f09f9d9b640 1 -- 172.21.15.27:0/845708832 <== mon.2 v2:172.21.15.45:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09f4036070 con 0x7f09fc0cf320 2024-09-12T03:19:37.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:37.162+0000 7f09f9d9b640 1 -- 172.21.15.27:0/845708832 <== mon.2 v2:172.21.15.45:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09f4031600 con 0x7f09fc0cf320 2024-09-12T03:19:37.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:37.162+0000 7f09f9d9b640 1 -- 172.21.15.27:0/845708832 <== mon.2 v2:172.21.15.45:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09f40319d0 con 0x7f09fc0cf320 2024-09-12T03:19:37.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:37.162+0000 7f0a01227640 1 -- 172.21.15.27:0/845708832 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f09fc0cf320 msgr2=0x7f09fc15f0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:37.164 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:37.162+0000 7f0a01227640 1 --2- 172.21.15.27:0/845708832 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f09fc0cf320 0x7f09fc15f0c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f09f4009e50 tx=0x7f09f402a680 comp rx=0 tx=0).stop 2024-09-12T03:19:37.164 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:37.162+0000 7f0a01227640 1 -- 172.21.15.27:0/845708832 shutdown_connections 2024-09-12T03:19:37.164 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:37.162+0000 7f0a01227640 1 --2- 172.21.15.27:0/845708832 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f09fc0cf320 0x7f09fc15f0c0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:37.164 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:37.162+0000 7f0a01227640 1 --2- 172.21.15.27:0/845708832 >> v2:172.21.15.45:3300/0 conn(0x7f09fc0cef40 0x7f09fc0b4000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:37.164 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:37.162+0000 7f0a01227640 1 -- 172.21.15.27:0/845708832 >> 172.21.15.27:0/845708832 conn(0x7f09fc06bb30 msgr2=0x7f09fc06bf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:37.164 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:37.162+0000 7f0a01227640 1 -- 172.21.15.27:0/845708832 shutdown_connections 2024-09-12T03:19:37.164 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:37.162+0000 7f0a01227640 1 -- 172.21.15.27:0/845708832 wait complete. 2024-09-12T03:19:37.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:37.164+0000 7f0a01227640 1 Processor -- start 2024-09-12T03:19:37.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:37.164+0000 7f0a01227640 1 -- start start 2024-09-12T03:19:37.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:37.165+0000 7f0a01227640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f09fc06c650 0x7f09fc148e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:37.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:37.165+0000 7f0a01227640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f09fc0cef40 0x7f09fc0c5560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:37.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:37.165+0000 7f0a01227640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f09fc0cf320 0x7f09fc0ae8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:37.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:37.165+0000 7f0a01227640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f09fc06db70 con 0x7f09fc0cef40 2024-09-12T03:19:37.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:37.165+0000 7f0a01227640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f09fc06d890 con 0x7f09fc0cf320 2024-09-12T03:19:37.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:37.165+0000 7f0a01227640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f09fc0ce440 con 0x7f09fc06c650 2024-09-12T03:19:37.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:37.165+0000 7f09fa59c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f09fc0cef40 0x7f09fc0c5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:37.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:37.165+0000 7f09fad9d640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f09fc06c650 0x7f09fc148e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:37.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:37.165+0000 7f09fb59e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f09fc0cf320 0x7f09fc0ae8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:37.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:37.165+0000 7f09fa59c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f09fc0cef40 0x7f09fc0c5560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38502/0 (socket says 172.21.15.27:38502) 2024-09-12T03:19:37.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:37.165+0000 7f09fa59c640 1 -- 172.21.15.27:0/3620866444 learned_addr learned my addr 172.21.15.27:0/3620866444 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:19:37.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:37.165+0000 7f09fad9d640 1 -- 172.21.15.27:0/3620866444 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f09fc0cf320 msgr2=0x7f09fc0ae8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:37.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:37.165+0000 7f09fad9d640 1 --2- 172.21.15.27:0/3620866444 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f09fc0cf320 0x7f09fc0ae8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:37.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:37.165+0000 7f09fad9d640 1 -- 172.21.15.27:0/3620866444 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f09fc0cef40 msgr2=0x7f09fc0c5560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:37.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:37.165+0000 7f09fad9d640 1 --2- 172.21.15.27:0/3620866444 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f09fc0cef40 0x7f09fc0c5560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:37.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:37.165+0000 7f09fad9d640 1 -- 172.21.15.27:0/3620866444 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09fc0c5e60 con 0x7f09fc06c650 2024-09-12T03:19:37.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:37.165+0000 7f09fad9d640 1 --2- 172.21.15.27:0/3620866444 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f09fc06c650 0x7f09fc148e40 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f09ec00e8e0 tx=0x7f09ec00eda0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:37.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:37.165+0000 7f09f37fe640 1 -- 172.21.15.27:0/3620866444 <== mon.2 v2:172.21.15.45:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09ec00b040 con 0x7f09fc06c650 2024-09-12T03:19:37.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:37.165+0000 7f09f37fe640 1 -- 172.21.15.27:0/3620866444 <== mon.2 v2:172.21.15.45:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09ec00cc40 con 0x7f09fc06c650 2024-09-12T03:19:37.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:37.166+0000 7f0a01227640 1 -- 172.21.15.27:0/3620866444 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09fc0c6140 con 0x7f09fc06c650 2024-09-12T03:19:37.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:37.166+0000 7f09f37fe640 1 -- 172.21.15.27:0/3620866444 <== mon.2 v2:172.21.15.45:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09ec004030 con 0x7f09fc06c650 2024-09-12T03:19:37.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:37.166+0000 7f0a01227640 1 -- 172.21.15.27:0/3620866444 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f09fc0c6700 con 0x7f09fc06c650 2024-09-12T03:19:37.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:37.167+0000 7f0a01227640 1 -- 172.21.15.27:0/3620866444 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09fc0ce7c0 con 0x7f09fc06c650 2024-09-12T03:19:37.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:37.167+0000 7f09f37fe640 1 -- 172.21.15.27:0/3620866444 <== mon.2 v2:172.21.15.45:3301/0 4 ==== mgrmap(e 9) ==== 90651+0+0 (secure 0 0 0) 0x7f09ec0041d0 con 0x7f09fc06c650 2024-09-12T03:19:37.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:37.167+0000 7f09f37fe640 1 --2- 172.21.15.27:0/3620866444 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f09dc06c010 0x7f09dc06e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:37.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:37.167+0000 7f09f37fe640 1 -- 172.21.15.27:0/3620866444 <== mon.2 v2:172.21.15.45:3301/0 5 ==== osd_map(56..56 src has 1..56) ==== 5709+0+0 (secure 0 0 0) 0x7f09ec08af90 con 0x7f09fc06c650 2024-09-12T03:19:37.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:37.167+0000 7f09fa59c640 1 --2- 172.21.15.27:0/3620866444 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f09dc06c010 0x7f09dc06e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:37.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:37.168+0000 7f09fa59c640 1 --2- 172.21.15.27:0/3620866444 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f09dc06c010 0x7f09dc06e4d0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f09fc0722a0 tx=0x7f09e4007ee0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:37.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:37.172+0000 7f09f37fe640 1 -- 172.21.15.27:0/3620866444 <== mon.2 v2:172.21.15.45:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f09ec016020 con 0x7f09fc06c650 2024-09-12T03:19:37.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:37.384+0000 7f0a01227640 1 -- 172.21.15.27:0/3620866444 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7f09fc06f240 con 0x7f09fc06c650 2024-09-12T03:19:38.162 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:38.162+0000 7f09f37fe640 1 -- 172.21.15.27:0/3620866444 <== mon.2 v2:172.21.15.45:3301/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v31) ==== 136+0+0 (secure 0 0 0) 0x7f09ec00a550 con 0x7f09fc06c650 2024-09-12T03:19:38.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:38.166+0000 7f0a01227640 1 -- 172.21.15.27:0/3620866444 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f09dc06c010 msgr2=0x7f09dc06e4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:38.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:38.166+0000 7f0a01227640 1 --2- 172.21.15.27:0/3620866444 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f09dc06c010 0x7f09dc06e4d0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f09fc0722a0 tx=0x7f09e4007ee0 comp rx=0 tx=0).stop 2024-09-12T03:19:38.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:38.166+0000 7f0a01227640 1 -- 172.21.15.27:0/3620866444 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f09fc06c650 msgr2=0x7f09fc148e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:38.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:38.166+0000 7f0a01227640 1 --2- 172.21.15.27:0/3620866444 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f09fc06c650 0x7f09fc148e40 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f09ec00e8e0 tx=0x7f09ec00eda0 comp rx=0 tx=0).stop 2024-09-12T03:19:38.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:38.166+0000 7f0a01227640 1 -- 172.21.15.27:0/3620866444 shutdown_connections 2024-09-12T03:19:38.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:38.166+0000 7f0a01227640 1 --2- 172.21.15.27:0/3620866444 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f09dc06c010 0x7f09dc06e4d0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:38.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:38.166+0000 7f0a01227640 1 --2- 172.21.15.27:0/3620866444 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f09fc0cf320 0x7f09fc0ae8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:38.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:38.166+0000 7f0a01227640 1 --2- 172.21.15.27:0/3620866444 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f09fc0cef40 0x7f09fc0c5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:38.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:38.166+0000 7f0a01227640 1 --2- 172.21.15.27:0/3620866444 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f09fc06c650 0x7f09fc148e40 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:38.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:38.166+0000 7f0a01227640 1 -- 172.21.15.27:0/3620866444 >> 172.21.15.27:0/3620866444 conn(0x7f09fc06bb30 msgr2=0x7f09fc0726c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:38.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:38.166+0000 7f0a01227640 1 -- 172.21.15.27:0/3620866444 shutdown_connections 2024-09-12T03:19:38.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:38.166+0000 7f0a01227640 1 -- 172.21.15.27:0/3620866444 wait complete. 2024-09-12T03:19:38.179 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-12T03:19:38.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:38.289+0000 7fd52a042640 1 Processor -- start 2024-09-12T03:19:38.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:38.289+0000 7fd52a042640 1 -- start start 2024-09-12T03:19:38.291 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:38.290+0000 7fd52a042640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fd524158cf0 0x7fd5240b55d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:38.291 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:38.290+0000 7fd52a042640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fd5241590d0 0x7fd524162430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:38.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:38.290+0000 7fd52a042640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fd52415f600 0x7fd52406bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:38.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:38.290+0000 7fd52a042640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7fd524162970 con 0x7fd52415f600 2024-09-12T03:19:38.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:38.290+0000 7fd52a042640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fd524162c10 con 0x7fd5241590d0 2024-09-12T03:19:38.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:38.290+0000 7fd52a042640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fd524162ef0 con 0x7fd524158cf0 2024-09-12T03:19:38.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:38.290+0000 7fd523fff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fd52415f600 0x7fd52406bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:38.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:38.290+0000 7fd523fff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fd52415f600 0x7fd52406bb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38514/0 (socket says 172.21.15.27:38514) 2024-09-12T03:19:38.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:38.290+0000 7fd523fff640 1 -- 172.21.15.27:0/3183777201 learned_addr learned my addr 172.21.15.27:0/3183777201 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:19:38.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:38.290+0000 7fd522ffd640 1 --2- 172.21.15.27:0/3183777201 >> v2:172.21.15.45:3300/0 conn(0x7fd5241590d0 0x7fd524162430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:38.293 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:38.290+0000 7fd5237fe640 1 --2- 172.21.15.27:0/3183777201 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fd524158cf0 0x7fd5240b55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:38.293 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:38.290+0000 7fd523fff640 1 -- 172.21.15.27:0/3183777201 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fd524158cf0 msgr2=0x7fd5240b55d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:38.293 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:38.290+0000 7fd523fff640 1 --2- 172.21.15.27:0/3183777201 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fd524158cf0 0x7fd5240b55d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:38.293 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:38.290+0000 7fd523fff640 1 -- 172.21.15.27:0/3183777201 >> v2:172.21.15.45:3300/0 conn(0x7fd5241590d0 msgr2=0x7fd524162430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:38.293 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:38.290+0000 7fd523fff640 1 --2- 172.21.15.27:0/3183777201 >> v2:172.21.15.45:3300/0 conn(0x7fd5241590d0 0x7fd524162430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:38.293 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:38.290+0000 7fd523fff640 1 -- 172.21.15.27:0/3183777201 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5241631d0 con 0x7fd52415f600 2024-09-12T03:19:38.293 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:38.291+0000 7fd523fff640 1 --2- 172.21.15.27:0/3183777201 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd52415f600 0x7fd52406bb30 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fd5240730d0 tx=0x7fd51c02a570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc845f977b64a4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:38.293 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:38.291+0000 7fd5227fc640 1 -- 172.21.15.27:0/3183777201 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd51c036070 con 0x7fd52415f600 2024-09-12T03:19:38.293 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:38.291+0000 7fd5227fc640 1 -- 172.21.15.27:0/3183777201 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd51c031600 con 0x7fd52415f600 2024-09-12T03:19:38.293 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:38.291+0000 7fd5227fc640 1 -- 172.21.15.27:0/3183777201 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd51c0319d0 con 0x7fd52415f600 2024-09-12T03:19:38.298 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:38.292+0000 7fd52a042640 1 -- 172.21.15.27:0/3183777201 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd52415f600 msgr2=0x7fd52406bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:38.298 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:38.292+0000 7fd52a042640 1 --2- 172.21.15.27:0/3183777201 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd52415f600 0x7fd52406bb30 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fd5240730d0 tx=0x7fd51c02a570 comp rx=0 tx=0).stop 2024-09-12T03:19:38.298 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:38.292+0000 7fd52a042640 1 -- 172.21.15.27:0/3183777201 shutdown_connections 2024-09-12T03:19:38.298 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:38.292+0000 7fd52a042640 1 --2- 172.21.15.27:0/3183777201 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd52415f600 0x7fd52406bb30 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:38.298 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:38.292+0000 7fd52a042640 1 --2- 172.21.15.27:0/3183777201 >> v2:172.21.15.45:3300/0 conn(0x7fd5241590d0 0x7fd524162430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:38.298 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:38.292+0000 7fd52a042640 1 --2- 172.21.15.27:0/3183777201 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fd524158cf0 0x7fd5240b55d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:38.298 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:38.292+0000 7fd52a042640 1 -- 172.21.15.27:0/3183777201 >> 172.21.15.27:0/3183777201 conn(0x7fd52406cc40 msgr2=0x7fd52406d050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:38.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:38.292+0000 7fd52a042640 1 -- 172.21.15.27:0/3183777201 shutdown_connections 2024-09-12T03:19:38.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:38.292+0000 7fd52a042640 1 -- 172.21.15.27:0/3183777201 wait complete. 2024-09-12T03:19:38.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:38.293+0000 7fd52a042640 1 Processor -- start 2024-09-12T03:19:38.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:38.293+0000 7fd52a042640 1 -- start start 2024-09-12T03:19:38.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:38.294+0000 7fd52a042640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd524158cf0 0x7fd5241e8bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:38.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:38.294+0000 7fd52a042640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd5241590d0 0x7fd5241e9110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:38.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:38.294+0000 7fd52a042640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fd52415f600 0x7fd524153750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:38.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:38.294+0000 7fd52a042640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fd52415fb90 con 0x7fd5241590d0 2024-09-12T03:19:38.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:38.294+0000 7fd52a042640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fd52415fa10 con 0x7fd524158cf0 2024-09-12T03:19:38.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:38.294+0000 7fd52a042640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fd52415fd10 con 0x7fd52415f600 2024-09-12T03:19:38.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:38.294+0000 7fd522ffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd5241590d0 0x7fd5241e9110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:38.300 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:38.294+0000 7fd522ffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd5241590d0 0x7fd5241e9110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38524/0 (socket says 172.21.15.27:38524) 2024-09-12T03:19:38.300 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:38.294+0000 7fd522ffd640 1 -- 172.21.15.27:0/974694039 learned_addr learned my addr 172.21.15.27:0/974694039 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:19:38.300 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:38.294+0000 7fd5237fe640 1 --2- 172.21.15.27:0/974694039 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd524158cf0 0x7fd5241e8bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:38.300 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:38.294+0000 7fd523fff640 1 --2- 172.21.15.27:0/974694039 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fd52415f600 0x7fd524153750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:38.300 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:38.295+0000 7fd522ffd640 1 -- 172.21.15.27:0/974694039 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fd52415f600 msgr2=0x7fd524153750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:38.300 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:38.295+0000 7fd522ffd640 1 --2- 172.21.15.27:0/974694039 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fd52415f600 0x7fd524153750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:38.300 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:38.295+0000 7fd522ffd640 1 -- 172.21.15.27:0/974694039 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd524158cf0 msgr2=0x7fd5241e8bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:38.300 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:38.295+0000 7fd522ffd640 1 --2- 172.21.15.27:0/974694039 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd524158cf0 0x7fd5241e8bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:38.300 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:38.295+0000 7fd522ffd640 1 -- 172.21.15.27:0/974694039 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5240b1c10 con 0x7fd5241590d0 2024-09-12T03:19:38.301 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:38.295+0000 7fd523fff640 1 --2- 172.21.15.27:0/974694039 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fd52415f600 0x7fd524153750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:19:38.301 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:38.295+0000 7fd5237fe640 1 --2- 172.21.15.27:0/974694039 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd524158cf0 0x7fd5241e8bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:19:38.301 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:38.295+0000 7fd522ffd640 1 --2- 172.21.15.27:0/974694039 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd5241590d0 0x7fd5241e9110 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fd51800e840 tx=0x7fd51800ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:38.301 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:38.295+0000 7fd50bfff640 1 -- 172.21.15.27:0/974694039 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd518017070 con 0x7fd5241590d0 2024-09-12T03:19:38.301 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:38.295+0000 7fd50bfff640 1 -- 172.21.15.27:0/974694039 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd51800c940 con 0x7fd5241590d0 2024-09-12T03:19:38.301 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:38.295+0000 7fd52a042640 1 -- 172.21.15.27:0/974694039 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5241e86c0 con 0x7fd5241590d0 2024-09-12T03:19:38.301 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:38.295+0000 7fd50bfff640 1 -- 172.21.15.27:0/974694039 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd51800cd10 con 0x7fd5241590d0 2024-09-12T03:19:38.301 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:38.295+0000 7fd52a042640 1 -- 172.21.15.27:0/974694039 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5240b2170 con 0x7fd5241590d0 2024-09-12T03:19:38.301 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:38.296+0000 7fd52a042640 1 -- 172.21.15.27:0/974694039 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd524162430 con 0x7fd5241590d0 2024-09-12T03:19:38.302 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:38.296+0000 7fd50bfff640 1 -- 172.21.15.27:0/974694039 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 9) ==== 90651+0+0 (secure 0 0 0) 0x7fd51801a070 con 0x7fd5241590d0 2024-09-12T03:19:38.302 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:38.297+0000 7fd50bfff640 1 --2- 172.21.15.27:0/974694039 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fd51006bfe0 0x7fd51006e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:38.302 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:38.297+0000 7fd50bfff640 1 -- 172.21.15.27:0/974694039 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5709+0+0 (secure 0 0 0) 0x7fd51808afb0 con 0x7fd5241590d0 2024-09-12T03:19:38.302 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:38.298+0000 7fd5237fe640 1 --2- 172.21.15.27:0/974694039 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fd51006bfe0 0x7fd51006e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:38.302 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:38.298+0000 7fd5237fe640 1 --2- 172.21.15.27:0/974694039 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fd51006bfe0 0x7fd51006e4a0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fd514009a70 tx=0x7fd514008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:38.302 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:38.302+0000 7fd50bfff640 1 -- 172.21.15.27:0/974694039 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fd5240b1c10 con 0x7fd5241590d0 2024-09-12T03:19:38.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:38.499+0000 7fd52a042640 1 -- 172.21.15.27:0/974694039 --> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7fd524157030 con 0x7fd51006bfe0 2024-09-12T03:19:38.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:38.501+0000 7fd50bfff640 1 -- 172.21.15.27:0/974694039 <== mgr.4102 v2:172.21.15.45:6832/3923821219 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7fd524157030 con 0x7fd51006bfe0 2024-09-12T03:19:38.501 INFO:teuthology.orchestra.run.smithi027.stdout:[ 2024-09-12T03:19:38.501 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:19:38.502 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "cephfs" 2024-09-12T03:19:38.502 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:19:38.502 INFO:teuthology.orchestra.run.smithi027.stdout:] 2024-09-12T03:19:38.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:38.503+0000 7fd52a042640 1 -- 172.21.15.27:0/974694039 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fd51006bfe0 msgr2=0x7fd51006e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:38.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:38.503+0000 7fd52a042640 1 --2- 172.21.15.27:0/974694039 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fd51006bfe0 0x7fd51006e4a0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fd514009a70 tx=0x7fd514008040 comp rx=0 tx=0).stop 2024-09-12T03:19:38.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:38.503+0000 7fd52a042640 1 -- 172.21.15.27:0/974694039 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd5241590d0 msgr2=0x7fd5241e9110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:38.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:38.503+0000 7fd52a042640 1 --2- 172.21.15.27:0/974694039 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd5241590d0 0x7fd5241e9110 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fd51800e840 tx=0x7fd51800ed00 comp rx=0 tx=0).stop 2024-09-12T03:19:38.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:38.503+0000 7fd52a042640 1 -- 172.21.15.27:0/974694039 shutdown_connections 2024-09-12T03:19:38.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:38.503+0000 7fd52a042640 1 --2- 172.21.15.27:0/974694039 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fd51006bfe0 0x7fd51006e4a0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:38.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:38.503+0000 7fd52a042640 1 --2- 172.21.15.27:0/974694039 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fd52415f600 0x7fd524153750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:38.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:38.503+0000 7fd52a042640 1 --2- 172.21.15.27:0/974694039 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd5241590d0 0x7fd5241e9110 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:38.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:38.503+0000 7fd52a042640 1 --2- 172.21.15.27:0/974694039 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd524158cf0 0x7fd5241e8bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:38.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:38.503+0000 7fd52a042640 1 -- 172.21.15.27:0/974694039 >> 172.21.15.27:0/974694039 conn(0x7fd52406cc40 msgr2=0x7fd524157550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:38.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:38.503+0000 7fd52a042640 1 -- 172.21.15.27:0/974694039 shutdown_connections 2024-09-12T03:19:38.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:38.503+0000 7fd52a042640 1 -- 172.21.15.27:0/974694039 wait complete. 2024-09-12T03:19:38.515 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-12T03:19:38.620 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:38.620+0000 7fc4a1e5e640 1 Processor -- start 2024-09-12T03:19:38.621 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:38.620+0000 7fc4a1e5e640 1 -- start start 2024-09-12T03:19:38.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:38.621+0000 7fc4a1e5e640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc49c0b6230 0x7fc49c0b66c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:38.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:38.621+0000 7fc4a1e5e640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc49c15ed70 con 0x7fc49c0b3600 2024-09-12T03:19:38.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:38.621+0000 7fc4a1e5e640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc49c0b7d10 con 0x7fc49c15e990 2024-09-12T03:19:38.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:38.621+0000 7fc4a1e5e640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fc49c0d2a30 con 0x7fc49c0b6230 2024-09-12T03:19:38.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:38.621+0000 7fc49bfff640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fc49c0b3600 0x7fc49c0b3a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:42420/0 (socket says 172.21.15.27:42420) 2024-09-12T03:19:38.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:38.621+0000 7fc49bfff640 1 -- 172.21.15.27:0/126425037 learned_addr learned my addr 172.21.15.27:0/126425037 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:19:38.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:38.622+0000 7fc492ffd640 1 --2- 172.21.15.27:0/126425037 >> v2:172.21.15.45:3300/0 conn(0x7fc49c0b6230 0x7fc49c0b66c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:38.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:38.622+0000 7fc49affd640 1 -- 172.21.15.27:0/126425037 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2049392630 0 0) 0x7fc49c15ed70 con 0x7fc49c0b3600 2024-09-12T03:19:38.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:38.622+0000 7fc49affd640 1 -- 172.21.15.27:0/126425037 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc488003520 con 0x7fc49c0b3600 2024-09-12T03:19:38.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:38.622+0000 7fc49affd640 1 -- 172.21.15.27:0/126425037 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3964332690 0 0) 0x7fc488003520 con 0x7fc49c0b3600 2024-09-12T03:19:38.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:38.622+0000 7fc49affd640 1 -- 172.21.15.27:0/126425037 >> v2:172.21.15.45:3300/0 conn(0x7fc49c0b6230 msgr2=0x7fc49c0b66c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:38.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:38.622+0000 7fc49affd640 1 --2- 172.21.15.27:0/126425037 >> v2:172.21.15.45:3300/0 conn(0x7fc49c0b6230 0x7fc49c0b66c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:38.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:38.622+0000 7fc49affd640 1 -- 172.21.15.27:0/126425037 >> v1:172.21.15.45:6789/0 conn(0x7fc49c15e990 legacy=0x7fc49c0b7600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:38.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:38.622+0000 7fc49affd640 1 -- 172.21.15.27:0/126425037 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc49c0d2d10 con 0x7fc49c0b3600 2024-09-12T03:19:38.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:38.622+0000 7fc49affd640 1 -- 172.21.15.27:0/126425037 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 847755213 0 0) 0x7fc4940019f0 con 0x7fc49c0b3600 2024-09-12T03:19:38.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:38.622+0000 7fc49affd640 1 -- 172.21.15.27:0/126425037 >> v1:172.21.15.27:6789/0 conn(0x7fc49c0b3600 legacy=0x7fc49c0b3a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:38.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:38.622+0000 7fc49affd640 1 --2- 172.21.15.27:0/126425037 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc4880039b0 0x7fc488003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:38.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:38.622+0000 7fc49affd640 1 --2- 172.21.15.27:0/126425037 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc488004800 0x7fc488004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:38.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:38.622+0000 7fc49affd640 1 --2- 172.21.15.27:0/126425037 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fc4880051c0 0x7fc48800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:38.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:38.622+0000 7fc49affd640 1 -- 172.21.15.27:0/126425037 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fc49c0d2a30 con 0x7fc488004800 2024-09-12T03:19:38.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:38.622+0000 7fc49affd640 1 -- 172.21.15.27:0/126425037 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc48800c060 con 0x7fc4880039b0 2024-09-12T03:19:38.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:38.622+0000 7fc49affd640 1 -- 172.21.15.27:0/126425037 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fc48800c260 con 0x7fc4880051c0 2024-09-12T03:19:38.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:38.622+0000 7fc492ffd640 1 --2- 172.21.15.27:0/126425037 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc488004800 0x7fc488004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:38.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:38.622+0000 7fc49b7fe640 1 --2- 172.21.15.27:0/126425037 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc4880039b0 0x7fc488003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:38.628 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:38.622+0000 7fc49bfff640 1 --2- 172.21.15.27:0/126425037 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fc4880051c0 0x7fc48800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:38.628 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:38.622+0000 7fc492ffd640 1 -- 172.21.15.27:0/126425037 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fc4880051c0 msgr2=0x7fc48800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:38.628 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:38.622+0000 7fc492ffd640 1 --2- 172.21.15.27:0/126425037 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fc4880051c0 0x7fc48800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:38.628 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:38.622+0000 7fc492ffd640 1 -- 172.21.15.27:0/126425037 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc4880039b0 msgr2=0x7fc488003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:38.628 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:38.622+0000 7fc492ffd640 1 --2- 172.21.15.27:0/126425037 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc4880039b0 0x7fc488003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:38.628 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:38.622+0000 7fc492ffd640 1 -- 172.21.15.27:0/126425037 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc488003520 con 0x7fc488004800 2024-09-12T03:19:38.628 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:38.623+0000 7fc49bfff640 1 --2- 172.21.15.27:0/126425037 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fc4880051c0 0x7fc48800b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:19:38.628 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:38.623+0000 7fc49bfff640 1 -- 172.21.15.27:0/126425037 reap_dead start 2024-09-12T03:19:38.628 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:38.623+0000 7fc492ffd640 1 --2- 172.21.15.27:0/126425037 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc488004800 0x7fc488004c80 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fc484009ac0 tx=0x7fc488005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f1d69e20bb1f1bb server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:38.628 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:38.623+0000 7fc49affd640 1 -- 172.21.15.27:0/126425037 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc484036070 con 0x7fc488004800 2024-09-12T03:19:38.628 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:38.623+0000 7fc49affd640 1 -- 172.21.15.27:0/126425037 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc484031400 con 0x7fc488004800 2024-09-12T03:19:38.628 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:38.623+0000 7fc49affd640 1 -- 172.21.15.27:0/126425037 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc484009690 con 0x7fc488004800 2024-09-12T03:19:38.628 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:38.624+0000 7fc4a1e5e640 1 -- 172.21.15.27:0/126425037 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc488004800 msgr2=0x7fc488004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:38.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:38.624+0000 7fc4a1e5e640 1 --2- 172.21.15.27:0/126425037 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc488004800 0x7fc488004c80 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fc484009ac0 tx=0x7fc488005830 comp rx=0 tx=0).stop 2024-09-12T03:19:38.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:38.624+0000 7fc4a1e5e640 1 -- 172.21.15.27:0/126425037 shutdown_connections 2024-09-12T03:19:38.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:38.624+0000 7fc4a1e5e640 1 --2- 172.21.15.27:0/126425037 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc488004800 0x7fc488004c80 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:38.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:38.624+0000 7fc4a1e5e640 1 -- 172.21.15.27:0/126425037 >> 172.21.15.27:0/126425037 conn(0x7fc49c0d1230 msgr2=0x7fc49c0d1640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:38.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:38.624+0000 7fc4a1e5e640 1 -- 172.21.15.27:0/126425037 shutdown_connections 2024-09-12T03:19:38.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:38.624+0000 7fc4a1e5e640 1 -- 172.21.15.27:0/126425037 wait complete. 2024-09-12T03:19:38.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:38.625+0000 7fc4a1e5e640 1 Processor -- start 2024-09-12T03:19:38.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:38.625+0000 7fc4a1e5e640 1 -- start start 2024-09-12T03:19:38.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:38.626+0000 7fc4a1e5e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc488004800 0x7fc49c0ce150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:38.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:38.626+0000 7fc4a1e5e640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fc49c1f4e80 0x7fc49c0b8a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:38.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:38.626+0000 7fc4a1e5e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc49c160780 0x7fc49c160c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:38.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:38.626+0000 7fc4a1e5e640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fc49c0d35b0 con 0x7fc49c160780 2024-09-12T03:19:38.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:38.626+0000 7fc4a1e5e640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc49c0d32d0 con 0x7fc488004800 2024-09-12T03:19:38.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:38.626+0000 7fc4a1e5e640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fc49c0b66c0 con 0x7fc49c1f4e80 2024-09-12T03:19:38.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:38.626+0000 7fc49bfff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc49c160780 0x7fc49c160c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:38.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:38.626+0000 7fc49bfff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc49c160780 0x7fc49c160c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38538/0 (socket says 172.21.15.27:38538) 2024-09-12T03:19:38.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:38.626+0000 7fc49bfff640 1 -- 172.21.15.27:0/2180675374 learned_addr learned my addr 172.21.15.27:0/2180675374 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:19:38.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:38.626+0000 7fc49b7fe640 1 --2- 172.21.15.27:0/2180675374 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc488004800 0x7fc49c0ce150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:38.634 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:38.626+0000 7fc492ffd640 1 --2- 172.21.15.27:0/2180675374 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fc49c1f4e80 0x7fc49c0b8a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:38.634 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:38.626+0000 7fc49bfff640 1 -- 172.21.15.27:0/2180675374 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fc49c1f4e80 msgr2=0x7fc49c0b8a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:38.634 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:38.626+0000 7fc49bfff640 1 --2- 172.21.15.27:0/2180675374 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fc49c1f4e80 0x7fc49c0b8a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:38.634 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:38.626+0000 7fc49bfff640 1 -- 172.21.15.27:0/2180675374 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc488004800 msgr2=0x7fc49c0ce150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:38.634 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:38.626+0000 7fc49bfff640 1 --2- 172.21.15.27:0/2180675374 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc488004800 0x7fc49c0ce150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:38.634 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:38.626+0000 7fc49bfff640 1 -- 172.21.15.27:0/2180675374 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc49c1f7920 con 0x7fc49c160780 2024-09-12T03:19:38.634 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:38.626+0000 7fc492ffd640 1 --2- 172.21.15.27:0/2180675374 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fc49c1f4e80 0x7fc49c0b8a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:19:38.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:38.627+0000 7fc49bfff640 1 --2- 172.21.15.27:0/2180675374 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc49c160780 0x7fc49c160c00 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fc49c06cf70 tx=0x7fc494007af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:38.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:38.627+0000 7fc498ff9640 1 -- 172.21.15.27:0/2180675374 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc49400e9c0 con 0x7fc49c160780 2024-09-12T03:19:38.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:38.627+0000 7fc498ff9640 1 -- 172.21.15.27:0/2180675374 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc494018070 con 0x7fc49c160780 2024-09-12T03:19:38.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:38.627+0000 7fc498ff9640 1 -- 172.21.15.27:0/2180675374 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc49400edc0 con 0x7fc49c160780 2024-09-12T03:19:38.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:38.627+0000 7fc4a1e5e640 1 -- 172.21.15.27:0/2180675374 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc49c1f7c00 con 0x7fc49c160780 2024-09-12T03:19:38.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:38.627+0000 7fc4a1e5e640 1 -- 172.21.15.27:0/2180675374 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc49c0ce690 con 0x7fc49c160780 2024-09-12T03:19:38.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:38.628+0000 7fc4a1e5e640 1 -- 172.21.15.27:0/2180675374 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc49c0b6a50 con 0x7fc49c160780 2024-09-12T03:19:38.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:38.628+0000 7fc498ff9640 1 -- 172.21.15.27:0/2180675374 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 9) ==== 90651+0+0 (secure 0 0 0) 0x7fc49401d460 con 0x7fc49c160780 2024-09-12T03:19:38.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:38.629+0000 7fc498ff9640 1 --2- 172.21.15.27:0/2180675374 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fc47c06bfe0 0x7fc47c06e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:38.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:38.629+0000 7fc498ff9640 1 -- 172.21.15.27:0/2180675374 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5709+0+0 (secure 0 0 0) 0x7fc49408b250 con 0x7fc49c160780 2024-09-12T03:19:38.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:38.629+0000 7fc49b7fe640 1 --2- 172.21.15.27:0/2180675374 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fc47c06bfe0 0x7fc47c06e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:38.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:38.630+0000 7fc49b7fe640 1 --2- 172.21.15.27:0/2180675374 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fc47c06bfe0 0x7fc47c06e4a0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fc49c161eb0 tx=0x7fc48c009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:38.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:38.633+0000 7fc498ff9640 1 -- 172.21.15.27:0/2180675374 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fc494059980 con 0x7fc49c160780 2024-09-12T03:19:38.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:38.807+0000 7fc4a1e5e640 1 -- 172.21.15.27:0/2180675374 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fc49c160560 con 0x7fc49c160780 2024-09-12T03:19:38.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:38.808+0000 7fc498ff9640 1 -- 172.21.15.27:0/2180675374 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v14)=0 v14) ==== 122+0+0 (secure 0 0 0) 0x7fc494059280 con 0x7fc49c160780 2024-09-12T03:19:38.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:38.809+0000 7fc4a1e5e640 1 -- 172.21.15.27:0/2180675374 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fc47c06bfe0 msgr2=0x7fc47c06e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:38.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:38.809+0000 7fc4a1e5e640 1 --2- 172.21.15.27:0/2180675374 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fc47c06bfe0 0x7fc47c06e4a0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fc49c161eb0 tx=0x7fc48c009040 comp rx=0 tx=0).stop 2024-09-12T03:19:38.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:38.809+0000 7fc4a1e5e640 1 -- 172.21.15.27:0/2180675374 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc49c160780 msgr2=0x7fc49c160c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:38.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:38.810+0000 7fc4a1e5e640 1 --2- 172.21.15.27:0/2180675374 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc49c160780 0x7fc49c160c00 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fc49c06cf70 tx=0x7fc494007af0 comp rx=0 tx=0).stop 2024-09-12T03:19:38.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:38.810+0000 7fc4a1e5e640 1 -- 172.21.15.27:0/2180675374 shutdown_connections 2024-09-12T03:19:38.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:38.810+0000 7fc4a1e5e640 1 --2- 172.21.15.27:0/2180675374 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fc47c06bfe0 0x7fc47c06e4a0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:38.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:38.810+0000 7fc4a1e5e640 1 --2- 172.21.15.27:0/2180675374 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc49c160780 0x7fc49c160c00 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:38.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:38.810+0000 7fc4a1e5e640 1 --2- 172.21.15.27:0/2180675374 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fc49c1f4e80 0x7fc49c0b8a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:38.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:38.810+0000 7fc4a1e5e640 1 --2- 172.21.15.27:0/2180675374 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc488004800 0x7fc49c0ce150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:38.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:38.810+0000 7fc4a1e5e640 1 -- 172.21.15.27:0/2180675374 >> 172.21.15.27:0/2180675374 conn(0x7fc49c0d1230 msgr2=0x7fc49c0b5a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:38.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:38.810+0000 7fc4a1e5e640 1 -- 172.21.15.27:0/2180675374 shutdown_connections 2024-09-12T03:19:38.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:38.810+0000 7fc4a1e5e640 1 -- 172.21.15.27:0/2180675374 wait complete. 2024-09-12T03:19:38.821 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:19:38.821 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p ./volumes/_nogroup/subvol82580) 2024-09-12T03:19:38.853 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:19:38.853 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.layout.pool .) 2024-09-12T03:19:38.911 INFO:teuthology.orchestra.run.smithi027.stdout:cephfs_data 2024-09-12T03:19:38.911 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:19:38.912 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo setfattr -n ceph.dir.layout.pool -v cephfs_data ./volumes/_nogroup/subvol82580) 2024-09-12T03:19:38.984 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs subvol65231 2024-09-12T03:19:39.129 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:39.129+0000 7fe544349640 1 Processor -- start 2024-09-12T03:19:39.129 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:39.129+0000 7fe544349640 1 -- start start 2024-09-12T03:19:39.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:39.130+0000 7fe544349640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe53c15e900 0x7fe53c0af370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:39.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:39.130+0000 7fe544349640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe53c0b5940 con 0x7fe53c0b6540 2024-09-12T03:19:39.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:39.130+0000 7fe544349640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe53c0af950 con 0x7fe53c06d730 2024-09-12T03:19:39.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:39.130+0000 7fe544349640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fe53c15ece0 con 0x7fe53c15e900 2024-09-12T03:19:39.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:39.130+0000 7fe5428e6640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fe53c0b6540 0x7fe53c06ee80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:42436/0 (socket says 172.21.15.27:42436) 2024-09-12T03:19:39.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:39.130+0000 7fe5428e6640 1 -- 172.21.15.27:0/4010171444 learned_addr learned my addr 172.21.15.27:0/4010171444 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:19:39.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:39.131+0000 7fe5420e5640 1 --2- 172.21.15.27:0/4010171444 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe53c15e900 0x7fe53c0af370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:39.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:39.131+0000 7fe5410e3640 1 -- 172.21.15.27:0/4010171444 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4139250689 0 0) 0x7fe534001110 con 0x7fe53c0b6540 2024-09-12T03:19:39.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:39.131+0000 7fe5410e3640 1 -- 172.21.15.27:0/4010171444 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe524003520 con 0x7fe53c0b6540 2024-09-12T03:19:39.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:39.131+0000 7fe5410e3640 1 -- 172.21.15.27:0/4010171444 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 828677828 0 0) 0x7fe524003520 con 0x7fe53c0b6540 2024-09-12T03:19:39.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:39.131+0000 7fe5410e3640 1 -- 172.21.15.27:0/4010171444 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe53c15e900 msgr2=0x7fe53c0af370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:39.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:39.131+0000 7fe5410e3640 1 --2- 172.21.15.27:0/4010171444 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe53c15e900 0x7fe53c0af370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:39.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:39.131+0000 7fe5410e3640 1 -- 172.21.15.27:0/4010171444 >> v1:172.21.15.45:6789/0 conn(0x7fe53c06d730 legacy=0x7fe53c0b5e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:39.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:39.131+0000 7fe5410e3640 1 -- 172.21.15.27:0/4010171444 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe53c0afc30 con 0x7fe53c0b6540 2024-09-12T03:19:39.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:39.131+0000 7fe5410e3640 1 -- 172.21.15.27:0/4010171444 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 847755213 0 0) 0x7fe534001840 con 0x7fe53c0b6540 2024-09-12T03:19:39.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:39.131+0000 7fe5410e3640 1 -- 172.21.15.27:0/4010171444 >> v1:172.21.15.27:6789/0 conn(0x7fe53c0b6540 legacy=0x7fe53c06ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:39.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:39.131+0000 7fe5410e3640 1 --2- 172.21.15.27:0/4010171444 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe5240039e0 0x7fe524003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:39.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:39.131+0000 7fe5410e3640 1 --2- 172.21.15.27:0/4010171444 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe524004800 0x7fe524004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:39.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:39.131+0000 7fe5410e3640 1 --2- 172.21.15.27:0/4010171444 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe53c15e900 0x7fe5240051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:39.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:39.131+0000 7fe5410e3640 1 -- 172.21.15.27:0/4010171444 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fe52400ba70 con 0x7fe524004800 2024-09-12T03:19:39.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:39.131+0000 7fe5410e3640 1 -- 172.21.15.27:0/4010171444 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe52400bc20 con 0x7fe5240039e0 2024-09-12T03:19:39.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:39.131+0000 7fe5410e3640 1 -- 172.21.15.27:0/4010171444 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fe52400be20 con 0x7fe53c15e900 2024-09-12T03:19:39.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:39.131+0000 7fe5418e4640 1 --2- 172.21.15.27:0/4010171444 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe524004800 0x7fe524004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:39.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:39.131+0000 7fe5420e5640 1 --2- 172.21.15.27:0/4010171444 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe5240039e0 0x7fe524003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:39.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:39.131+0000 7fe5428e6640 1 --2- 172.21.15.27:0/4010171444 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe53c15e900 0x7fe5240051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:39.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:39.131+0000 7fe5418e4640 1 -- 172.21.15.27:0/4010171444 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe53c15e900 msgr2=0x7fe5240051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:39.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:39.131+0000 7fe5418e4640 1 --2- 172.21.15.27:0/4010171444 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe53c15e900 0x7fe5240051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:39.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:39.131+0000 7fe5418e4640 1 -- 172.21.15.27:0/4010171444 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe5240039e0 msgr2=0x7fe524003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:39.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:39.131+0000 7fe5418e4640 1 --2- 172.21.15.27:0/4010171444 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe5240039e0 0x7fe524003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:39.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:39.131+0000 7fe5418e4640 1 -- 172.21.15.27:0/4010171444 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe524003520 con 0x7fe524004800 2024-09-12T03:19:39.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:39.132+0000 7fe5418e4640 1 --2- 172.21.15.27:0/4010171444 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe524004800 0x7fe524004c80 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fe53c0aed80 tx=0x7fe524005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99e8265348ba380d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:39.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:39.132+0000 7fe5410e3640 1 -- 172.21.15.27:0/4010171444 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe530001370 con 0x7fe524004800 2024-09-12T03:19:39.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:39.132+0000 7fe5410e3640 1 -- 172.21.15.27:0/4010171444 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5300052d0 con 0x7fe524004800 2024-09-12T03:19:39.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:39.132+0000 7fe5410e3640 1 -- 172.21.15.27:0/4010171444 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe53002ab70 con 0x7fe524004800 2024-09-12T03:19:39.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:39.133+0000 7fe544349640 1 -- 172.21.15.27:0/4010171444 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe524004800 msgr2=0x7fe524004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:39.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:39.133+0000 7fe544349640 1 --2- 172.21.15.27:0/4010171444 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe524004800 0x7fe524004c80 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fe53c0aed80 tx=0x7fe524005830 comp rx=0 tx=0).stop 2024-09-12T03:19:39.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:39.133+0000 7fe5428e6640 1 -- 172.21.15.27:0/4010171444 reap_dead start 2024-09-12T03:19:39.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:39.133+0000 7fe544349640 1 -- 172.21.15.27:0/4010171444 shutdown_connections 2024-09-12T03:19:39.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:39.133+0000 7fe544349640 1 -- 172.21.15.27:0/4010171444 >> 172.21.15.27:0/4010171444 conn(0x7fe53c06cc40 msgr2=0x7fe53c06d020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:39.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:39.133+0000 7fe544349640 1 -- 172.21.15.27:0/4010171444 shutdown_connections 2024-09-12T03:19:39.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:39.133+0000 7fe544349640 1 -- 172.21.15.27:0/4010171444 wait complete. 2024-09-12T03:19:39.138 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:39.134+0000 7fe544349640 1 Processor -- start 2024-09-12T03:19:39.138 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:39.134+0000 7fe544349640 1 -- start start 2024-09-12T03:19:39.138 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:39.135+0000 7fe544349640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe524004800 0x7fe53c14b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:39.138 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:39.135+0000 7fe544349640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe53c160740 0x7fe53c144ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:39.138 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:39.135+0000 7fe544349640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe53c147470 0x7fe53c1478f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:39.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:39.135+0000 7fe544349640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fe53c06e880 con 0x7fe524004800 2024-09-12T03:19:39.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:39.135+0000 7fe544349640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe53c0b01f0 con 0x7fe53c160740 2024-09-12T03:19:39.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:39.135+0000 7fe544349640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fe53c06ea00 con 0x7fe53c147470 2024-09-12T03:19:39.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:39.135+0000 7fe5420e5640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe524004800 0x7fe53c14b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:39.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:39.135+0000 7fe5420e5640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe524004800 0x7fe53c14b410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38564/0 (socket says 172.21.15.27:38564) 2024-09-12T03:19:39.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:39.135+0000 7fe5420e5640 1 -- 172.21.15.27:0/2101356472 learned_addr learned my addr 172.21.15.27:0/2101356472 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:19:39.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:39.135+0000 7fe5428e6640 1 --2- 172.21.15.27:0/2101356472 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe53c147470 0x7fe53c1478f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:39.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:39.135+0000 7fe5418e4640 1 --2- 172.21.15.27:0/2101356472 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe53c160740 0x7fe53c144ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:39.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:39.135+0000 7fe5420e5640 1 -- 172.21.15.27:0/2101356472 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe53c147470 msgr2=0x7fe53c1478f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:39.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:39.135+0000 7fe5420e5640 1 --2- 172.21.15.27:0/2101356472 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe53c147470 0x7fe53c1478f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:39.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:39.135+0000 7fe5420e5640 1 -- 172.21.15.27:0/2101356472 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe53c160740 msgr2=0x7fe53c144ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:39.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:39.135+0000 7fe5420e5640 1 --2- 172.21.15.27:0/2101356472 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe53c160740 0x7fe53c144ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:39.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:39.135+0000 7fe5420e5640 1 -- 172.21.15.27:0/2101356472 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe53c160f90 con 0x7fe524004800 2024-09-12T03:19:39.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:39.135+0000 7fe5428e6640 1 --2- 172.21.15.27:0/2101356472 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe53c147470 0x7fe53c1478f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:19:39.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:39.135+0000 7fe5418e4640 1 --2- 172.21.15.27:0/2101356472 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe53c160740 0x7fe53c144ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:19:39.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:39.135+0000 7fe5420e5640 1 --2- 172.21.15.27:0/2101356472 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe524004800 0x7fe53c14b410 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fe52c011e90 tx=0x7fe52c00f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:39.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:39.136+0000 7fe53affd640 1 -- 172.21.15.27:0/2101356472 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe52c015630 con 0x7fe524004800 2024-09-12T03:19:39.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:39.136+0000 7fe53affd640 1 -- 172.21.15.27:0/2101356472 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe52c0157d0 con 0x7fe524004800 2024-09-12T03:19:39.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:39.136+0000 7fe53affd640 1 -- 172.21.15.27:0/2101356472 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe52c015ba0 con 0x7fe524004800 2024-09-12T03:19:39.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:39.136+0000 7fe544349640 1 -- 172.21.15.27:0/2101356472 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe53c161270 con 0x7fe524004800 2024-09-12T03:19:39.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:39.136+0000 7fe544349640 1 -- 172.21.15.27:0/2101356472 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe53c144ff0 con 0x7fe524004800 2024-09-12T03:19:39.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:39.137+0000 7fe544349640 1 -- 172.21.15.27:0/2101356472 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe53c0af370 con 0x7fe524004800 2024-09-12T03:19:39.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:39.137+0000 7fe53affd640 1 -- 172.21.15.27:0/2101356472 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 9) ==== 90651+0+0 (secure 0 0 0) 0x7fe52c01d400 con 0x7fe524004800 2024-09-12T03:19:39.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:39.138+0000 7fe53affd640 1 --2- 172.21.15.27:0/2101356472 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fe52806c010 0x7fe52806e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:39.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:39.138+0000 7fe53affd640 1 -- 172.21.15.27:0/2101356472 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5709+0+0 (secure 0 0 0) 0x7fe52c08c400 con 0x7fe524004800 2024-09-12T03:19:39.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:39.138+0000 7fe5418e4640 1 --2- 172.21.15.27:0/2101356472 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fe52806c010 0x7fe52806e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:39.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:39.139+0000 7fe5418e4640 1 --2- 172.21.15.27:0/2101356472 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fe52806c010 0x7fe52806e4d0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fe530032000 tx=0x7fe53002e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:39.142 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:39.142+0000 7fe53affd640 1 -- 172.21.15.27:0/2101356472 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fe52c019030 con 0x7fe524004800 2024-09-12T03:19:39.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:39.342+0000 7fe544349640 1 -- 172.21.15.27:0/2101356472 --> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvol65231", "target": ["mon-mgr", ""]}) -- 0x7fe53c06d0c0 con 0x7fe52806c010 2024-09-12T03:19:39.658 INFO:tasks.ceph.mgr.x.smithi045.stderr:2024-09-12T03:19:39.346+0000 7fc16ffad640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:19:39.658 INFO:tasks.ceph.mgr.x.smithi045.stderr:2024-09-12T03:19:39.346+0000 7fc16ffad640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:19:39.659 INFO:tasks.ceph.mgr.x.smithi045.stderr:2024-09-12T03:19:39.346+0000 7fc16ffad640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:19:39.659 INFO:tasks.ceph.mgr.x.smithi045.stderr:2024-09-12T03:19:39.346+0000 7fc16ffad640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:19:39.659 INFO:tasks.ceph.mgr.x.smithi045.stderr:2024-09-12T03:19:39.346+0000 7fc16ffad640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:19:39.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:39.380+0000 7fe53affd640 1 -- 172.21.15.27:0/2101356472 <== mgr.4102 v2:172.21.15.45:6832/3923821219 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fe53c06d0c0 con 0x7fe52806c010 2024-09-12T03:19:39.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:39.381+0000 7fe544349640 1 -- 172.21.15.27:0/2101356472 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fe52806c010 msgr2=0x7fe52806e4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:39.662 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:39.381+0000 7fe544349640 1 --2- 172.21.15.27:0/2101356472 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fe52806c010 0x7fe52806e4d0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fe530032000 tx=0x7fe53002e000 comp rx=0 tx=0).stop 2024-09-12T03:19:39.662 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:39.382+0000 7fe544349640 1 -- 172.21.15.27:0/2101356472 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe524004800 msgr2=0x7fe53c14b410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:39.662 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:39.382+0000 7fe544349640 1 --2- 172.21.15.27:0/2101356472 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe524004800 0x7fe53c14b410 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fe52c011e90 tx=0x7fe52c00f580 comp rx=0 tx=0).stop 2024-09-12T03:19:39.662 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:39.382+0000 7fe544349640 1 -- 172.21.15.27:0/2101356472 shutdown_connections 2024-09-12T03:19:39.662 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:39.382+0000 7fe544349640 1 --2- 172.21.15.27:0/2101356472 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fe52806c010 0x7fe52806e4d0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:39.663 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:39.382+0000 7fe544349640 1 --2- 172.21.15.27:0/2101356472 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe53c147470 0x7fe53c1478f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:39.663 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:39.382+0000 7fe544349640 1 --2- 172.21.15.27:0/2101356472 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe53c160740 0x7fe53c144ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:39.663 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:39.382+0000 7fe544349640 1 --2- 172.21.15.27:0/2101356472 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe524004800 0x7fe53c14b410 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:39.663 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:39.382+0000 7fe544349640 1 -- 172.21.15.27:0/2101356472 >> 172.21.15.27:0/2101356472 conn(0x7fe53c06cc40 msgr2=0x7fe53c0d0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:39.663 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:39.382+0000 7fe544349640 1 -- 172.21.15.27:0/2101356472 shutdown_connections 2024-09-12T03:19:39.664 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:39.382+0000 7fe544349640 1 -- 172.21.15.27:0/2101356472 wait complete. 2024-09-12T03:19:39.664 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:19:39.664 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cp ./volumes/_nogroup/subvol65231/.meta ./volumes/_nogroup/subvol82580) 2024-09-12T03:19:41.545 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol82580 2024-09-12T03:19:41.664 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:41.664+0000 7f0a9f8e4640 1 Processor -- start 2024-09-12T03:19:41.665 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:41.665+0000 7f0a9f8e4640 1 -- start start 2024-09-12T03:19:41.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:41.666+0000 7f0a9f8e4640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f0a9815e900 0x7f0a980af370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:41.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:41.666+0000 7f0a9f8e4640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0a980b5940 con 0x7f0a980b6540 2024-09-12T03:19:41.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:41.666+0000 7f0a9f8e4640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0a980af950 con 0x7f0a9806d730 2024-09-12T03:19:41.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:41.666+0000 7f0a9f8e4640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f0a9815ece0 con 0x7f0a9815e900 2024-09-12T03:19:41.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:41.666+0000 7f0a9de81640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f0a980b6540 0x7f0a9806ee80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:45988/0 (socket says 172.21.15.27:45988) 2024-09-12T03:19:41.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:41.666+0000 7f0a9de81640 1 -- 172.21.15.27:0/3284501743 learned_addr learned my addr 172.21.15.27:0/3284501743 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:19:41.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:41.666+0000 7f0a9d680640 1 --2- 172.21.15.27:0/3284501743 >> v2:172.21.15.45:3300/0 conn(0x7f0a9815e900 0x7f0a980af370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:41.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:41.666+0000 7f0a8ffff640 1 -- 172.21.15.27:0/3284501743 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2548417326 0 0) 0x7f0a90001490 con 0x7f0a980b6540 2024-09-12T03:19:41.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:41.666+0000 7f0a8ffff640 1 -- 172.21.15.27:0/3284501743 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0a84003520 con 0x7f0a980b6540 2024-09-12T03:19:41.667 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:41.666+0000 7f0a9d680640 1 -- 172.21.15.27:0/3284501743 >> v1:172.21.15.45:6789/0 conn(0x7f0a9806d730 legacy=0x7f0a980b5e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:41.667 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:41.666+0000 7f0a8ffff640 1 -- 172.21.15.27:0/3284501743 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2153483089 0 0) 0x7f0a84003520 con 0x7f0a980b6540 2024-09-12T03:19:41.667 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:41.666+0000 7f0a9d680640 1 -- 172.21.15.27:0/3284501743 >> v1:172.21.15.27:6789/0 conn(0x7f0a980b6540 legacy=0x7f0a9806ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:41.667 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:41.666+0000 7f0a9d680640 1 -- 172.21.15.27:0/3284501743 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a980afc30 con 0x7f0a9815e900 2024-09-12T03:19:41.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:41.667+0000 7f0a9d680640 1 --2- 172.21.15.27:0/3284501743 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0a9815e900 0x7f0a980af370 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f0a940097a0 tx=0x7f0a9402a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=29b0a846732d65f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:41.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:41.667+0000 7f0a8ffff640 1 -- 172.21.15.27:0/3284501743 <== mon.1 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a94037070 con 0x7f0a9815e900 2024-09-12T03:19:41.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:41.667+0000 7f0a8ffff640 1 -- 172.21.15.27:0/3284501743 <== mon.1 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a9402ac50 con 0x7f0a9815e900 2024-09-12T03:19:41.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:41.667+0000 7f0a8ffff640 1 -- 172.21.15.27:0/3284501743 <== mon.1 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a90001bc0 con 0x7f0a9815e900 2024-09-12T03:19:41.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:41.668+0000 7f0a9f8e4640 1 -- 172.21.15.27:0/3284501743 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0a9815e900 msgr2=0x7f0a980af370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:41.669 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:41.668+0000 7f0a9f8e4640 1 --2- 172.21.15.27:0/3284501743 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0a9815e900 0x7f0a980af370 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f0a940097a0 tx=0x7f0a9402a1c0 comp rx=0 tx=0).stop 2024-09-12T03:19:41.669 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:41.668+0000 7f0a9f8e4640 1 -- 172.21.15.27:0/3284501743 shutdown_connections 2024-09-12T03:19:41.669 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:41.668+0000 7f0a9f8e4640 1 --2- 172.21.15.27:0/3284501743 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0a9815e900 0x7f0a980af370 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:41.669 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:41.668+0000 7f0a9f8e4640 1 -- 172.21.15.27:0/3284501743 >> 172.21.15.27:0/3284501743 conn(0x7f0a9806cc40 msgr2=0x7f0a9806d020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:41.669 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:41.668+0000 7f0a9f8e4640 1 -- 172.21.15.27:0/3284501743 shutdown_connections 2024-09-12T03:19:41.669 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:41.668+0000 7f0a9f8e4640 1 -- 172.21.15.27:0/3284501743 wait complete. 2024-09-12T03:19:41.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:41.669+0000 7f0a9f8e4640 1 Processor -- start 2024-09-12T03:19:41.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:41.669+0000 7f0a9f8e4640 1 -- start start 2024-09-12T03:19:41.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:41.670+0000 7f0a9f8e4640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0a9806d730 0x7f0a98153fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:41.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:41.670+0000 7f0a9f8e4640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f0a980b6540 0x7f0a9815cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:41.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:41.670+0000 7f0a9f8e4640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0a9815e900 0x7f0a98149bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:41.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:41.670+0000 7f0a9f8e4640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f0a9806e880 con 0x7f0a9806d730 2024-09-12T03:19:41.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:41.670+0000 7f0a9f8e4640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f0a980b01f0 con 0x7f0a9815e900 2024-09-12T03:19:41.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:41.670+0000 7f0a9f8e4640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f0a9806ea00 con 0x7f0a980b6540 2024-09-12T03:19:41.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:41.670+0000 7f0a9d680640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0a9806d730 0x7f0a98153fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:41.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:41.670+0000 7f0a9d680640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0a9806d730 0x7f0a98153fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58156/0 (socket says 172.21.15.27:58156) 2024-09-12T03:19:41.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:41.670+0000 7f0a9d680640 1 -- 172.21.15.27:0/1460241783 learned_addr learned my addr 172.21.15.27:0/1460241783 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:19:41.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:41.670+0000 7f0a9ce7f640 1 --2- 172.21.15.27:0/1460241783 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f0a980b6540 0x7f0a9815cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:41.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:41.670+0000 7f0a9de81640 1 --2- 172.21.15.27:0/1460241783 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0a9815e900 0x7f0a98149bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:41.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:41.671+0000 7f0a9d680640 1 -- 172.21.15.27:0/1460241783 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f0a980b6540 msgr2=0x7f0a9815cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:41.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:41.671+0000 7f0a9d680640 1 --2- 172.21.15.27:0/1460241783 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f0a980b6540 0x7f0a9815cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:41.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:41.671+0000 7f0a9d680640 1 -- 172.21.15.27:0/1460241783 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0a9815e900 msgr2=0x7f0a98149bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:41.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:41.671+0000 7f0a9d680640 1 --2- 172.21.15.27:0/1460241783 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0a9815e900 0x7f0a98149bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:41.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:41.671+0000 7f0a9d680640 1 -- 172.21.15.27:0/1460241783 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a98160320 con 0x7f0a9806d730 2024-09-12T03:19:41.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:41.671+0000 7f0a9d680640 1 --2- 172.21.15.27:0/1460241783 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0a9806d730 0x7f0a98153fd0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f0a94035040 tx=0x7f0a940046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:41.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:41.671+0000 7f0a8dffb640 1 -- 172.21.15.27:0/1460241783 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a94042020 con 0x7f0a9806d730 2024-09-12T03:19:41.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:41.671+0000 7f0a8dffb640 1 -- 172.21.15.27:0/1460241783 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a9403d470 con 0x7f0a9806d730 2024-09-12T03:19:41.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:41.671+0000 7f0a9f8e4640 1 -- 172.21.15.27:0/1460241783 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a981548d0 con 0x7f0a9806d730 2024-09-12T03:19:41.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:41.671+0000 7f0a8dffb640 1 -- 172.21.15.27:0/1460241783 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a94037040 con 0x7f0a9806d730 2024-09-12T03:19:41.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:41.671+0000 7f0a9f8e4640 1 -- 172.21.15.27:0/1460241783 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a98154e90 con 0x7f0a9806d730 2024-09-12T03:19:41.674 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:41.672+0000 7f0a9f8e4640 1 -- 172.21.15.27:0/1460241783 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a980af370 con 0x7f0a9806d730 2024-09-12T03:19:41.674 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:41.673+0000 7f0a8dffb640 1 -- 172.21.15.27:0/1460241783 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 10) ==== 90702+0+0 (secure 0 0 0) 0x7f0a94045020 con 0x7f0a9806d730 2024-09-12T03:19:41.674 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:41.673+0000 7f0a8dffb640 1 --2- 172.21.15.27:0/1460241783 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f0a7c06c070 0x7f0a7c06e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:41.674 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:41.673+0000 7f0a8dffb640 1 -- 172.21.15.27:0/1460241783 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5709+0+0 (secure 0 0 0) 0x7f0a940aba70 con 0x7f0a9806d730 2024-09-12T03:19:41.674 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:41.673+0000 7f0a9ce7f640 1 --2- 172.21.15.27:0/1460241783 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f0a7c06c070 0x7f0a7c06e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:41.674 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:41.674+0000 7f0a9ce7f640 1 --2- 172.21.15.27:0/1460241783 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f0a7c06c070 0x7f0a7c06e530 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f0a9815e310 tx=0x7f0a88009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:41.677 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:41.678+0000 7f0a8dffb640 1 -- 172.21.15.27:0/1460241783 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f0a9404c030 con 0x7f0a9806d730 2024-09-12T03:19:41.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:41.880+0000 7f0a9f8e4640 1 -- 172.21.15.27:0/1460241783 --> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol82580", "target": ["mon-mgr", ""]}) -- 0x7f0a980b1800 con 0x7f0a7c06c070 2024-09-12T03:19:41.911 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:41.911+0000 7f0a8dffb640 1 -- 172.21.15.27:0/1460241783 <== mgr.4102 v2:172.21.15.45:6832/3923821219 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+30 (secure 0 0 0) 0x7f0a980b1800 con 0x7f0a7c06c070 2024-09-12T03:19:41.911 INFO:teuthology.orchestra.run.smithi027.stdout:/volumes/_nogroup/subvol82580 2024-09-12T03:19:41.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:41.913+0000 7f0a9f8e4640 1 -- 172.21.15.27:0/1460241783 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f0a7c06c070 msgr2=0x7f0a7c06e530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:41.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:41.913+0000 7f0a9f8e4640 1 --2- 172.21.15.27:0/1460241783 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f0a7c06c070 0x7f0a7c06e530 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f0a9815e310 tx=0x7f0a88009040 comp rx=0 tx=0).stop 2024-09-12T03:19:41.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:41.913+0000 7f0a9f8e4640 1 -- 172.21.15.27:0/1460241783 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0a9806d730 msgr2=0x7f0a98153fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:41.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:41.913+0000 7f0a9f8e4640 1 --2- 172.21.15.27:0/1460241783 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0a9806d730 0x7f0a98153fd0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f0a94035040 tx=0x7f0a940046c0 comp rx=0 tx=0).stop 2024-09-12T03:19:41.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:41.913+0000 7f0a9f8e4640 1 -- 172.21.15.27:0/1460241783 shutdown_connections 2024-09-12T03:19:41.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:41.913+0000 7f0a9f8e4640 1 --2- 172.21.15.27:0/1460241783 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f0a7c06c070 0x7f0a7c06e530 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:41.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:41.913+0000 7f0a9f8e4640 1 --2- 172.21.15.27:0/1460241783 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0a9815e900 0x7f0a98149bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:41.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:41.913+0000 7f0a9f8e4640 1 --2- 172.21.15.27:0/1460241783 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f0a980b6540 0x7f0a9815cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:41.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:41.913+0000 7f0a9f8e4640 1 --2- 172.21.15.27:0/1460241783 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0a9806d730 0x7f0a98153fd0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:41.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:41.913+0000 7f0a9f8e4640 1 -- 172.21.15.27:0/1460241783 >> 172.21.15.27:0/1460241783 conn(0x7f0a9806cc40 msgr2=0x7f0a980b2270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:41.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:41.913+0000 7f0a9f8e4640 1 -- 172.21.15.27:0/1460241783 shutdown_connections 2024-09-12T03:19:41.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:41.913+0000 7f0a9f8e4640 1 -- 172.21.15.27:0/1460241783 wait complete. 2024-09-12T03:19:41.924 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol82580 2024-09-12T03:19:42.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.031+0000 7fe15d7c4640 1 Processor -- start 2024-09-12T03:19:42.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.031+0000 7fe15d7c4640 1 -- start start 2024-09-12T03:19:42.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.031+0000 7fe15d7c4640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fe15815e650 0x7fe1580d1d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:42.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.031+0000 7fe15d7c4640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe15806cc40 0x7fe15806d060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:42.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.031+0000 7fe15d7c4640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fe15806db30 0x7fe15806df60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:42.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.031+0000 7fe15d7c4640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7fe1580d2280 con 0x7fe15815e650 2024-09-12T03:19:42.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.031+0000 7fe15d7c4640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fe15806d5a0 con 0x7fe15806db30 2024-09-12T03:19:42.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.031+0000 7fe15d7c4640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fe15806d850 con 0x7fe15806cc40 2024-09-12T03:19:42.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.032+0000 7fe156ffd640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fe15815e650 0x7fe1580d1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:42.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.032+0000 7fe156ffd640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fe15815e650 0x7fe1580d1d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58158/0 (socket says 172.21.15.27:58158) 2024-09-12T03:19:42.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.032+0000 7fe156ffd640 1 -- 172.21.15.27:0/178929797 learned_addr learned my addr 172.21.15.27:0/178929797 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:19:42.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.032+0000 7fe1567fc640 1 --2- 172.21.15.27:0/178929797 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe15806cc40 0x7fe15806d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:42.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.032+0000 7fe1577fe640 1 --2- 172.21.15.27:0/178929797 >> v2:172.21.15.45:3300/0 conn(0x7fe15806db30 0x7fe15806df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:42.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.032+0000 7fe156ffd640 1 -- 172.21.15.27:0/178929797 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe15806cc40 msgr2=0x7fe15806d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:42.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.032+0000 7fe156ffd640 1 --2- 172.21.15.27:0/178929797 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe15806cc40 0x7fe15806d060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:42.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.032+0000 7fe156ffd640 1 -- 172.21.15.27:0/178929797 >> v2:172.21.15.45:3300/0 conn(0x7fe15806db30 msgr2=0x7fe15806df60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:42.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.032+0000 7fe156ffd640 1 --2- 172.21.15.27:0/178929797 >> v2:172.21.15.45:3300/0 conn(0x7fe15806db30 0x7fe15806df60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:42.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.032+0000 7fe156ffd640 1 -- 172.21.15.27:0/178929797 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1580d1260 con 0x7fe15815e650 2024-09-12T03:19:42.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.032+0000 7fe1577fe640 1 --2- 172.21.15.27:0/178929797 >> v2:172.21.15.45:3300/0 conn(0x7fe15806db30 0x7fe15806df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:19:42.034 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.033+0000 7fe156ffd640 1 --2- 172.21.15.27:0/178929797 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe15815e650 0x7fe1580d1d40 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fe158050f60 tx=0x7fe14802a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2037b0cecf936b89 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:42.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.033+0000 7fe155ffb640 1 -- 172.21.15.27:0/178929797 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe148037070 con 0x7fe15815e650 2024-09-12T03:19:42.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.033+0000 7fe155ffb640 1 -- 172.21.15.27:0/178929797 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe148032440 con 0x7fe15815e650 2024-09-12T03:19:42.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.033+0000 7fe155ffb640 1 -- 172.21.15.27:0/178929797 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe148032810 con 0x7fe15815e650 2024-09-12T03:19:42.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.033+0000 7fe15d7c4640 1 -- 172.21.15.27:0/178929797 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe15815e650 msgr2=0x7fe1580d1d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:42.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.033+0000 7fe15d7c4640 1 --2- 172.21.15.27:0/178929797 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe15815e650 0x7fe1580d1d40 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fe158050f60 tx=0x7fe14802a140 comp rx=0 tx=0).stop 2024-09-12T03:19:42.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.033+0000 7fe15d7c4640 1 -- 172.21.15.27:0/178929797 shutdown_connections 2024-09-12T03:19:42.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.033+0000 7fe15d7c4640 1 --2- 172.21.15.27:0/178929797 >> v2:172.21.15.45:3300/0 conn(0x7fe15806db30 0x7fe15806df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:42.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.033+0000 7fe15d7c4640 1 --2- 172.21.15.27:0/178929797 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe15806cc40 0x7fe15806d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:42.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.033+0000 7fe15d7c4640 1 --2- 172.21.15.27:0/178929797 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe15815e650 0x7fe1580d1d40 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:42.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.033+0000 7fe15d7c4640 1 -- 172.21.15.27:0/178929797 >> 172.21.15.27:0/178929797 conn(0x7fe15806f320 msgr2=0x7fe15806f700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:42.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.033+0000 7fe15d7c4640 1 -- 172.21.15.27:0/178929797 shutdown_connections 2024-09-12T03:19:42.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.034+0000 7fe15d7c4640 1 -- 172.21.15.27:0/178929797 wait complete. 2024-09-12T03:19:42.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.035+0000 7fe15d7c4640 1 Processor -- start 2024-09-12T03:19:42.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.035+0000 7fe15d7c4640 1 -- start start 2024-09-12T03:19:42.038 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.036+0000 7fe15d7c4640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe15806cc40 0x7fe158155710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:42.039 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.036+0000 7fe15d7c4640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe15806db30 0x7fe1580b5820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:42.039 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.036+0000 7fe15d7c4640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe15815e650 0x7fe158156360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:42.039 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.036+0000 7fe15d7c4640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fe15815ebb0 con 0x7fe15806cc40 2024-09-12T03:19:42.039 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.036+0000 7fe15d7c4640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe15815ea30 con 0x7fe15815e650 2024-09-12T03:19:42.039 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.036+0000 7fe15d7c4640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fe15806e4a0 con 0x7fe15806db30 2024-09-12T03:19:42.039 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.036+0000 7fe156ffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe15806cc40 0x7fe158155710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:42.039 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.036+0000 7fe156ffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe15806cc40 0x7fe158155710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58166/0 (socket says 172.21.15.27:58166) 2024-09-12T03:19:42.039 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.036+0000 7fe156ffd640 1 -- 172.21.15.27:0/2339914722 learned_addr learned my addr 172.21.15.27:0/2339914722 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:19:42.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.036+0000 7fe1567fc640 1 --2- 172.21.15.27:0/2339914722 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe15806db30 0x7fe1580b5820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:42.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.036+0000 7fe1577fe640 1 --2- 172.21.15.27:0/2339914722 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe15815e650 0x7fe158156360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:42.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.036+0000 7fe156ffd640 1 -- 172.21.15.27:0/2339914722 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe15806db30 msgr2=0x7fe1580b5820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:42.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.036+0000 7fe156ffd640 1 --2- 172.21.15.27:0/2339914722 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe15806db30 0x7fe1580b5820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:42.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.036+0000 7fe156ffd640 1 -- 172.21.15.27:0/2339914722 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe15815e650 msgr2=0x7fe158156360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:42.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.036+0000 7fe156ffd640 1 --2- 172.21.15.27:0/2339914722 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe15815e650 0x7fe158156360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:42.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.036+0000 7fe156ffd640 1 -- 172.21.15.27:0/2339914722 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe158154c90 con 0x7fe15806cc40 2024-09-12T03:19:42.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.036+0000 7fe1577fe640 1 --2- 172.21.15.27:0/2339914722 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe15815e650 0x7fe158156360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:19:42.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.036+0000 7fe156ffd640 1 --2- 172.21.15.27:0/2339914722 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe15806cc40 0x7fe158155710 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fe14802a180 tx=0x7fe1480046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:42.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.036+0000 7fe13f7fe640 1 -- 172.21.15.27:0/2339914722 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe148037040 con 0x7fe15806cc40 2024-09-12T03:19:42.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.036+0000 7fe13f7fe640 1 -- 172.21.15.27:0/2339914722 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe148042070 con 0x7fe15806cc40 2024-09-12T03:19:42.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.036+0000 7fe15d7c4640 1 -- 172.21.15.27:0/2339914722 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe158156e20 con 0x7fe15806cc40 2024-09-12T03:19:42.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.036+0000 7fe13f7fe640 1 -- 172.21.15.27:0/2339914722 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe148030880 con 0x7fe15806cc40 2024-09-12T03:19:42.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.037+0000 7fe15d7c4640 1 -- 172.21.15.27:0/2339914722 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe158155210 con 0x7fe15806cc40 2024-09-12T03:19:42.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.037+0000 7fe15d7c4640 1 -- 172.21.15.27:0/2339914722 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe15806df60 con 0x7fe15806cc40 2024-09-12T03:19:42.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.038+0000 7fe13f7fe640 1 -- 172.21.15.27:0/2339914722 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 10) ==== 90702+0+0 (secure 0 0 0) 0x7fe148030a20 con 0x7fe15806cc40 2024-09-12T03:19:42.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.039+0000 7fe13f7fe640 1 --2- 172.21.15.27:0/2339914722 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fe14406c070 0x7fe14406e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:42.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.039+0000 7fe13f7fe640 1 -- 172.21.15.27:0/2339914722 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5709+0+0 (secure 0 0 0) 0x7fe1480abad0 con 0x7fe15806cc40 2024-09-12T03:19:42.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.039+0000 7fe1567fc640 1 --2- 172.21.15.27:0/2339914722 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fe14406c070 0x7fe14406e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:42.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.040+0000 7fe1567fc640 1 --2- 172.21.15.27:0/2339914722 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fe14406c070 0x7fe14406e530 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fe14c009a20 tx=0x7fe14c009190 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:42.043 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.043+0000 7fe13f7fe640 1 -- 172.21.15.27:0/2339914722 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fe14807a120 con 0x7fe15806cc40 2024-09-12T03:19:42.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.243+0000 7fe15d7c4640 1 -- 172.21.15.27:0/2339914722 --> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol82580", "target": ["mon-mgr", ""]}) -- 0x7fe1580b3290 con 0x7fe14406c070 2024-09-12T03:19:42.249 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.248+0000 7fe13f7fe640 1 -- 172.21.15.27:0/2339914722 <== mgr.4102 v2:172.21.15.45:6832/3923821219 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+30 (secure 0 0 0) 0x7fe1580b3290 con 0x7fe14406c070 2024-09-12T03:19:42.249 INFO:teuthology.orchestra.run.smithi027.stdout:/volumes/_nogroup/subvol82580 2024-09-12T03:19:42.251 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.250+0000 7fe15d7c4640 1 -- 172.21.15.27:0/2339914722 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fe14406c070 msgr2=0x7fe14406e530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:42.251 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.250+0000 7fe15d7c4640 1 --2- 172.21.15.27:0/2339914722 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fe14406c070 0x7fe14406e530 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fe14c009a20 tx=0x7fe14c009190 comp rx=0 tx=0).stop 2024-09-12T03:19:42.251 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.250+0000 7fe15d7c4640 1 -- 172.21.15.27:0/2339914722 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe15806cc40 msgr2=0x7fe158155710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:42.251 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.250+0000 7fe15d7c4640 1 --2- 172.21.15.27:0/2339914722 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe15806cc40 0x7fe158155710 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fe14802a180 tx=0x7fe1480046c0 comp rx=0 tx=0).stop 2024-09-12T03:19:42.251 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.250+0000 7fe15d7c4640 1 -- 172.21.15.27:0/2339914722 shutdown_connections 2024-09-12T03:19:42.251 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.250+0000 7fe15d7c4640 1 --2- 172.21.15.27:0/2339914722 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fe14406c070 0x7fe14406e530 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:42.251 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.251+0000 7fe15d7c4640 1 --2- 172.21.15.27:0/2339914722 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe15815e650 0x7fe158156360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:42.251 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.251+0000 7fe15d7c4640 1 --2- 172.21.15.27:0/2339914722 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe15806db30 0x7fe1580b5820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:42.251 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.251+0000 7fe15d7c4640 1 --2- 172.21.15.27:0/2339914722 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe15806cc40 0x7fe158155710 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:42.251 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.251+0000 7fe15d7c4640 1 -- 172.21.15.27:0/2339914722 >> 172.21.15.27:0/2339914722 conn(0x7fe15806f320 msgr2=0x7fe1580b6b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:42.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.251+0000 7fe15d7c4640 1 -- 172.21.15.27:0/2339914722 shutdown_connections 2024-09-12T03:19:42.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.251+0000 7fe15d7c4640 1 -- 172.21.15.27:0/2339914722 wait complete. 2024-09-12T03:19:42.262 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:19:42.262 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cat ./volumes/_legacy/855830149da5a31d24c5a77a4c2cef3d.meta) 2024-09-12T03:19:42.291 INFO:teuthology.orchestra.run.smithi027.stdout:[GLOBAL] 2024-09-12T03:19:42.292 INFO:teuthology.orchestra.run.smithi027.stdout:version = 1 2024-09-12T03:19:42.292 INFO:teuthology.orchestra.run.smithi027.stdout:type = subvolume 2024-09-12T03:19:42.292 INFO:teuthology.orchestra.run.smithi027.stdout:path = /volumes/_nogroup/subvol82580 2024-09-12T03:19:42.292 INFO:teuthology.orchestra.run.smithi027.stdout:state = complete 2024-09-12T03:19:42.292 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:19:42.293 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume authorize cephfs subvol82580 alice 2024-09-12T03:19:42.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.441+0000 7f4add465640 1 Processor -- start 2024-09-12T03:19:42.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.441+0000 7f4add465640 1 -- start start 2024-09-12T03:19:42.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.441+0000 7f4add465640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f4ad80b4000 0x7f4ad815f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:42.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.442+0000 7f4add465640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4ad806cff0 con 0x7f4ad806c650 2024-09-12T03:19:42.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.442+0000 7f4add465640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4ad806d2d0 con 0x7f4ad80ce440 2024-09-12T03:19:42.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.442+0000 7f4add465640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f4ad806dee0 con 0x7f4ad80b4000 2024-09-12T03:19:42.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.442+0000 7f4ad67fc640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f4ad80b4000 0x7f4ad815f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:42.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.442+0000 7f4ad6ffd640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f4ad806c650 0x7f4ad80710c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:45998/0 (socket says 172.21.15.27:45998) 2024-09-12T03:19:42.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.442+0000 7f4ad67fc640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f4ad80b4000 0x7f4ad815f1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58168/0 (socket says 172.21.15.27:58168) 2024-09-12T03:19:42.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.442+0000 7f4ad6ffd640 1 -- 172.21.15.27:0/1507853923 learned_addr learned my addr 172.21.15.27:0/1507853923 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:19:42.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.442+0000 7f4ad67fc640 1 -- 172.21.15.27:0/1507853923 >> v1:172.21.15.45:6789/0 conn(0x7f4ad80ce440 legacy=0x7f4ad80cef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:42.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.442+0000 7f4ad67fc640 1 -- 172.21.15.27:0/1507853923 >> v1:172.21.15.27:6789/0 conn(0x7f4ad806c650 legacy=0x7f4ad80710c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:42.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.442+0000 7f4ad5ffb640 1 -- 172.21.15.27:0/1507853923 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3511356062 0 0) 0x7f4ad806cff0 con 0x7f4ad806c650 2024-09-12T03:19:42.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.442+0000 7f4ad67fc640 1 -- 172.21.15.27:0/1507853923 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ad806d5b0 con 0x7f4ad80b4000 2024-09-12T03:19:42.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.443+0000 7f4ad67fc640 1 --2- 172.21.15.27:0/1507853923 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4ad80b4000 0x7f4ad815f1c0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f4acc0097a0 tx=0x7f4acc02a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbaaeb9235f7e422 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:42.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.443+0000 7f4ad5ffb640 1 -- 172.21.15.27:0/1507853923 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4acc037070 con 0x7f4ad80b4000 2024-09-12T03:19:42.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.443+0000 7f4ad5ffb640 1 -- 172.21.15.27:0/1507853923 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4acc02ae50 con 0x7f4ad80b4000 2024-09-12T03:19:42.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.443+0000 7f4ad5ffb640 1 -- 172.21.15.27:0/1507853923 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4acc032660 con 0x7f4ad80b4000 2024-09-12T03:19:42.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.443+0000 7f4add465640 1 -- 172.21.15.27:0/1507853923 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4ad80b4000 msgr2=0x7f4ad815f1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:42.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.443+0000 7f4add465640 1 --2- 172.21.15.27:0/1507853923 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4ad80b4000 0x7f4ad815f1c0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f4acc0097a0 tx=0x7f4acc02a140 comp rx=0 tx=0).stop 2024-09-12T03:19:42.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.443+0000 7f4add465640 1 -- 172.21.15.27:0/1507853923 shutdown_connections 2024-09-12T03:19:42.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.443+0000 7f4add465640 1 --2- 172.21.15.27:0/1507853923 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4ad80b4000 0x7f4ad815f1c0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:42.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.443+0000 7f4add465640 1 -- 172.21.15.27:0/1507853923 >> 172.21.15.27:0/1507853923 conn(0x7f4ad806bb30 msgr2=0x7f4ad806bf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:42.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.444+0000 7f4add465640 1 -- 172.21.15.27:0/1507853923 shutdown_connections 2024-09-12T03:19:42.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.444+0000 7f4add465640 1 -- 172.21.15.27:0/1507853923 wait complete. 2024-09-12T03:19:42.447 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.445+0000 7f4add465640 1 Processor -- start 2024-09-12T03:19:42.447 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.445+0000 7f4add465640 1 -- start start 2024-09-12T03:19:42.447 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.446+0000 7f4add465640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4ad806c650 0x7f4ad814de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:42.447 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.446+0000 7f4add465640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4ad80b4000 0x7f4ad81f8340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:42.447 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.446+0000 7f4add465640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4ad80ce440 0x7f4ad814a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:42.447 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.446+0000 7f4add465640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f4ad806db70 con 0x7f4ad80b4000 2024-09-12T03:19:42.447 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.446+0000 7f4add465640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f4ad80cf650 con 0x7f4ad806c650 2024-09-12T03:19:42.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.446+0000 7f4add465640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f4ad80ce8d0 con 0x7f4ad80ce440 2024-09-12T03:19:42.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.446+0000 7f4ad67fc640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4ad80b4000 0x7f4ad81f8340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:42.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.446+0000 7f4ad67fc640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4ad80b4000 0x7f4ad81f8340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58174/0 (socket says 172.21.15.27:58174) 2024-09-12T03:19:42.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.446+0000 7f4ad67fc640 1 -- 172.21.15.27:0/2118069922 learned_addr learned my addr 172.21.15.27:0/2118069922 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:19:42.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.446+0000 7f4ad6ffd640 1 --2- 172.21.15.27:0/2118069922 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4ad806c650 0x7f4ad814de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:42.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.446+0000 7f4ad77fe640 1 --2- 172.21.15.27:0/2118069922 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4ad80ce440 0x7f4ad814a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:42.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.446+0000 7f4ad67fc640 1 -- 172.21.15.27:0/2118069922 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4ad80ce440 msgr2=0x7f4ad814a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:42.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.446+0000 7f4ad67fc640 1 --2- 172.21.15.27:0/2118069922 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4ad80ce440 0x7f4ad814a050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:42.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.446+0000 7f4ad67fc640 1 -- 172.21.15.27:0/2118069922 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4ad806c650 msgr2=0x7f4ad814de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:42.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.446+0000 7f4ad67fc640 1 --2- 172.21.15.27:0/2118069922 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4ad806c650 0x7f4ad814de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:42.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.446+0000 7f4ad67fc640 1 -- 172.21.15.27:0/2118069922 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ad81f8880 con 0x7f4ad80b4000 2024-09-12T03:19:42.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.446+0000 7f4ad6ffd640 1 --2- 172.21.15.27:0/2118069922 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4ad806c650 0x7f4ad814de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:19:42.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.446+0000 7f4ad77fe640 1 --2- 172.21.15.27:0/2118069922 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4ad80ce440 0x7f4ad814a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:19:42.449 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.446+0000 7f4ad67fc640 1 --2- 172.21.15.27:0/2118069922 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4ad80b4000 0x7f4ad81f8340 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f4acc0097a0 tx=0x7f4acc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:42.449 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.447+0000 7f4abf7fe640 1 -- 172.21.15.27:0/2118069922 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4acc037040 con 0x7f4ad80b4000 2024-09-12T03:19:42.449 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.447+0000 7f4abf7fe640 1 -- 172.21.15.27:0/2118069922 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4acc004030 con 0x7f4ad80b4000 2024-09-12T03:19:42.449 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.447+0000 7f4add465640 1 -- 172.21.15.27:0/2118069922 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ad814aa20 con 0x7f4ad80b4000 2024-09-12T03:19:42.449 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.447+0000 7f4abf7fe640 1 -- 172.21.15.27:0/2118069922 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4acc006e10 con 0x7f4ad80b4000 2024-09-12T03:19:42.449 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.447+0000 7f4add465640 1 -- 172.21.15.27:0/2118069922 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ad814afe0 con 0x7f4ad80b4000 2024-09-12T03:19:42.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.448+0000 7f4add465640 1 -- 172.21.15.27:0/2118069922 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ad815f1c0 con 0x7f4ad80b4000 2024-09-12T03:19:42.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.448+0000 7f4abf7fe640 1 -- 172.21.15.27:0/2118069922 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 10) ==== 90702+0+0 (secure 0 0 0) 0x7f4acc03d650 con 0x7f4ad80b4000 2024-09-12T03:19:42.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.449+0000 7f4abf7fe640 1 --2- 172.21.15.27:0/2118069922 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f4ac406c0a0 0x7f4ac406e560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:42.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.449+0000 7f4abf7fe640 1 -- 172.21.15.27:0/2118069922 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5709+0+0 (secure 0 0 0) 0x7f4acc0abba0 con 0x7f4ad80b4000 2024-09-12T03:19:42.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.449+0000 7f4ad6ffd640 1 --2- 172.21.15.27:0/2118069922 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f4ac406c0a0 0x7f4ac406e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:42.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.449+0000 7f4ad6ffd640 1 --2- 172.21.15.27:0/2118069922 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f4ac406c0a0 0x7f4ac406e560 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f4ac8000f60 tx=0x7f4ac8009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:42.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.453+0000 7f4abf7fe640 1 -- 172.21.15.27:0/2118069922 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f4ad81f8880 con 0x7f4ad80b4000 2024-09-12T03:19:42.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.653+0000 7f4add465640 1 -- 172.21.15.27:0/2118069922 --> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] -- mgr_command(tid 0: {"prefix": "fs subvolume authorize", "vol_name": "cephfs", "sub_name": "subvol82580", "auth_id": "alice", "target": ["mon-mgr", ""]}) -- 0x7f4ad806f240 con 0x7f4ac406c0a0 2024-09-12T03:19:42.663 INFO:tasks.ceph.mgr.x.smithi045.stderr:Creating meta for ID alice with tenant None 2024-09-12T03:19:42.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.698+0000 7f4abf7fe640 1 -- 172.21.15.27:0/2118069922 <== mgr.4102 v2:172.21.15.45:6832/3923821219 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f4ad806f240 con 0x7f4ac406c0a0 2024-09-12T03:19:42.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.700+0000 7f4add465640 1 -- 172.21.15.27:0/2118069922 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f4ac406c0a0 msgr2=0x7f4ac406e560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:42.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.700+0000 7f4add465640 1 --2- 172.21.15.27:0/2118069922 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f4ac406c0a0 0x7f4ac406e560 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f4ac8000f60 tx=0x7f4ac8009040 comp rx=0 tx=0).stop 2024-09-12T03:19:42.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.700+0000 7f4add465640 1 -- 172.21.15.27:0/2118069922 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4ad80b4000 msgr2=0x7f4ad81f8340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:42.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.700+0000 7f4add465640 1 --2- 172.21.15.27:0/2118069922 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4ad80b4000 0x7f4ad81f8340 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f4acc0097a0 tx=0x7f4acc0046c0 comp rx=0 tx=0).stop 2024-09-12T03:19:42.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.700+0000 7f4add465640 1 -- 172.21.15.27:0/2118069922 shutdown_connections 2024-09-12T03:19:42.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.700+0000 7f4add465640 1 --2- 172.21.15.27:0/2118069922 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f4ac406c0a0 0x7f4ac406e560 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:42.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.700+0000 7f4add465640 1 --2- 172.21.15.27:0/2118069922 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4ad80ce440 0x7f4ad814a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:42.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.700+0000 7f4add465640 1 --2- 172.21.15.27:0/2118069922 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4ad80b4000 0x7f4ad81f8340 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:42.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.700+0000 7f4add465640 1 --2- 172.21.15.27:0/2118069922 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4ad806c650 0x7f4ad814de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:42.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.700+0000 7f4add465640 1 -- 172.21.15.27:0/2118069922 >> 172.21.15.27:0/2118069922 conn(0x7f4ad806bb30 msgr2=0x7f4ad8071a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:42.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.701+0000 7f4add465640 1 -- 172.21.15.27:0/2118069922 shutdown_connections 2024-09-12T03:19:42.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.701+0000 7f4add465640 1 -- 172.21.15.27:0/2118069922 wait complete. 2024-09-12T03:19:42.712 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth get client.alice --format=json-pretty 2024-09-12T03:19:42.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.815+0000 7f287348d640 1 Processor -- start 2024-09-12T03:19:42.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.815+0000 7f287348d640 1 -- start start 2024-09-12T03:19:42.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.816+0000 7f287348d640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f286c0b1ca0 0x7f286c15f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:42.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.816+0000 7f287348d640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f286c0cf660 0x7f286c0d2780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:42.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.816+0000 7f287348d640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f286c15fd70 con 0x7f286c06f360 2024-09-12T03:19:42.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.816+0000 7f287348d640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f286c0d22f0 con 0x7f286c0b1ca0 2024-09-12T03:19:42.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.816+0000 7f287348d640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f286c06cc40 con 0x7f286c0cf660 2024-09-12T03:19:42.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.816+0000 7f2871229640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f286c0b1ca0 0x7f286c15f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:42.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.816+0000 7f2871229640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f286c0b1ca0 0x7f286c15f830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58186/0 (socket says 172.21.15.27:58186) 2024-09-12T03:19:42.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.816+0000 7f2871229640 1 -- 172.21.15.27:0/1764048733 learned_addr learned my addr 172.21.15.27:0/1764048733 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:19:42.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.816+0000 7f2871229640 1 -- 172.21.15.27:0/1764048733 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f286c0cf660 msgr2=0x7f286c0d2780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:42.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.816+0000 7f2871229640 1 --2- 172.21.15.27:0/1764048733 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f286c0cf660 0x7f286c0d2780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:42.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.816+0000 7f2871229640 1 -- 172.21.15.27:0/1764048733 >> v1:172.21.15.45:6789/0 conn(0x7f286c06f360 legacy=0x7f286c0cef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:42.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.816+0000 7f2871229640 1 -- 172.21.15.27:0/1764048733 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f286c0d1ce0 con 0x7f286c0b1ca0 2024-09-12T03:19:42.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.817+0000 7f2871229640 1 --2- 172.21.15.27:0/1764048733 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f286c0b1ca0 0x7f286c15f830 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f2868009870 tx=0x7f286802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffc1e2c9251c833a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:42.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.817+0000 7f2863fff640 1 -- 172.21.15.27:0/1764048733 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2868037070 con 0x7f286c0b1ca0 2024-09-12T03:19:42.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.817+0000 7f2863fff640 1 -- 172.21.15.27:0/1764048733 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2868032440 con 0x7f286c0b1ca0 2024-09-12T03:19:42.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.817+0000 7f2863fff640 1 -- 172.21.15.27:0/1764048733 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2868032810 con 0x7f286c0b1ca0 2024-09-12T03:19:42.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.818+0000 7f287348d640 1 -- 172.21.15.27:0/1764048733 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f286c0b1ca0 msgr2=0x7f286c15f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:42.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.818+0000 7f287348d640 1 --2- 172.21.15.27:0/1764048733 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f286c0b1ca0 0x7f286c15f830 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f2868009870 tx=0x7f286802a210 comp rx=0 tx=0).stop 2024-09-12T03:19:42.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.818+0000 7f287348d640 1 -- 172.21.15.27:0/1764048733 shutdown_connections 2024-09-12T03:19:42.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.818+0000 7f287348d640 1 --2- 172.21.15.27:0/1764048733 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f286c0cf660 0x7f286c0d2780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:42.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.818+0000 7f287348d640 1 --2- 172.21.15.27:0/1764048733 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f286c0b1ca0 0x7f286c15f830 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:42.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.818+0000 7f287348d640 1 -- 172.21.15.27:0/1764048733 >> 172.21.15.27:0/1764048733 conn(0x7f286c073800 msgr2=0x7f286c09ba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:42.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.818+0000 7f287348d640 1 -- 172.21.15.27:0/1764048733 shutdown_connections 2024-09-12T03:19:42.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.818+0000 7f287348d640 1 -- 172.21.15.27:0/1764048733 wait complete. 2024-09-12T03:19:42.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.819+0000 7f287348d640 1 Processor -- start 2024-09-12T03:19:42.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.819+0000 7f287348d640 1 -- start start 2024-09-12T03:19:42.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.820+0000 7f287348d640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f286c06f360 0x7f286c0cc020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:42.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.820+0000 7f287348d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f286c0b1ca0 0x7f286c0cda00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:42.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.820+0000 7f287348d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f286c0cf660 0x7f286c15e570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:42.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.820+0000 7f287348d640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f286c06d1c0 con 0x7f286c0cf660 2024-09-12T03:19:42.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.820+0000 7f287348d640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f286c0d1f80 con 0x7f286c0b1ca0 2024-09-12T03:19:42.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.820+0000 7f287348d640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f286c06d340 con 0x7f286c06f360 2024-09-12T03:19:42.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.820+0000 7f2871a2a640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f286c0cf660 0x7f286c15e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:42.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.820+0000 7f2871a2a640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f286c0cf660 0x7f286c15e570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58196/0 (socket says 172.21.15.27:58196) 2024-09-12T03:19:42.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.820+0000 7f2871a2a640 1 -- 172.21.15.27:0/4293655572 learned_addr learned my addr 172.21.15.27:0/4293655572 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:19:42.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.820+0000 7f2870a28640 1 --2- 172.21.15.27:0/4293655572 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f286c0b1ca0 0x7f286c0cda00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:42.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.820+0000 7f2871229640 1 --2- 172.21.15.27:0/4293655572 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f286c06f360 0x7f286c0cc020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:42.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.820+0000 7f2871a2a640 1 -- 172.21.15.27:0/4293655572 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f286c06f360 msgr2=0x7f286c0cc020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:42.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.820+0000 7f2871a2a640 1 --2- 172.21.15.27:0/4293655572 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f286c06f360 0x7f286c0cc020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:42.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.820+0000 7f2871a2a640 1 -- 172.21.15.27:0/4293655572 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f286c0b1ca0 msgr2=0x7f286c0cda00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:42.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.820+0000 7f2871a2a640 1 --2- 172.21.15.27:0/4293655572 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f286c0b1ca0 0x7f286c0cda00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:42.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.820+0000 7f2871a2a640 1 -- 172.21.15.27:0/4293655572 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f286c0cc560 con 0x7f286c0cf660 2024-09-12T03:19:42.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.821+0000 7f2871a2a640 1 --2- 172.21.15.27:0/4293655572 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f286c0cf660 0x7f286c15e570 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f286400b9e0 tx=0x7f286400bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:42.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.821+0000 7f2861ffb640 1 -- 172.21.15.27:0/4293655572 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2864019070 con 0x7f286c0cf660 2024-09-12T03:19:42.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.821+0000 7f2861ffb640 1 -- 172.21.15.27:0/4293655572 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f286400cb40 con 0x7f286c0cf660 2024-09-12T03:19:42.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.821+0000 7f287348d640 1 -- 172.21.15.27:0/4293655572 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f286c0ce350 con 0x7f286c0cf660 2024-09-12T03:19:42.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.821+0000 7f2861ffb640 1 -- 172.21.15.27:0/4293655572 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2864014440 con 0x7f286c0cf660 2024-09-12T03:19:42.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.821+0000 7f287348d640 1 -- 172.21.15.27:0/4293655572 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f286c0ce910 con 0x7f286c0cf660 2024-09-12T03:19:42.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.822+0000 7f287348d640 1 -- 172.21.15.27:0/4293655572 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f286c0cfc40 con 0x7f286c0cf660 2024-09-12T03:19:42.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.822+0000 7f2861ffb640 1 -- 172.21.15.27:0/4293655572 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 10) ==== 90702+0+0 (secure 0 0 0) 0x7f286401c070 con 0x7f286c0cf660 2024-09-12T03:19:42.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.823+0000 7f2861ffb640 1 --2- 172.21.15.27:0/4293655572 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f285006c070 0x7f285006e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:42.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.823+0000 7f2861ffb640 1 -- 172.21.15.27:0/4293655572 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5709+0+0 (secure 0 0 0) 0x7f286408aee0 con 0x7f286c0cf660 2024-09-12T03:19:42.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.823+0000 7f2871229640 1 --2- 172.21.15.27:0/4293655572 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f285006c070 0x7f285006e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:42.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.824+0000 7f2871229640 1 --2- 172.21.15.27:0/4293655572 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f285006c070 0x7f285006e530 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f2868034000 tx=0x7f286802e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:42.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:42.827+0000 7f2861ffb640 1 -- 172.21.15.27:0/4293655572 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f2864058ee0 con 0x7f286c0cf660 2024-09-12T03:19:43.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:43.074+0000 7f287348d640 1 -- 172.21.15.27:0/4293655572 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "auth get", "entity": "client.alice", "format": "json-pretty"} v 0) -- 0x7f286c070fb0 con 0x7f286c0cf660 2024-09-12T03:19:43.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:43.075+0000 7f2861ffb640 1 -- 172.21.15.27:0/4293655572 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "auth get", "entity": "client.alice", "format": "json-pretty"}]=0 v14) ==== 107+0+280 (secure 0 0 0) 0x7f286405d1e0 con 0x7f286c0cf660 2024-09-12T03:19:43.075 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:19:43.075 INFO:teuthology.orchestra.run.smithi027.stdout:[ 2024-09-12T03:19:43.075 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:19:43.075 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "client.alice", 2024-09-12T03:19:43.075 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDOXeJmirdoKBAAdMkSM4D68jo2uQVNOwfw9Q==", 2024-09-12T03:19:43.075 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:19:43.075 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow rw path=/volumes/_nogroup/subvol82580", 2024-09-12T03:19:43.075 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow r", 2024-09-12T03:19:43.075 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow rw pool=cephfs_data" 2024-09-12T03:19:43.075 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:19:43.075 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:19:43.076 INFO:teuthology.orchestra.run.smithi027.stdout:] 2024-09-12T03:19:43.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:43.077+0000 7f287348d640 1 -- 172.21.15.27:0/4293655572 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f285006c070 msgr2=0x7f285006e530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:43.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:43.077+0000 7f287348d640 1 --2- 172.21.15.27:0/4293655572 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f285006c070 0x7f285006e530 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f2868034000 tx=0x7f286802e000 comp rx=0 tx=0).stop 2024-09-12T03:19:43.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:43.077+0000 7f287348d640 1 -- 172.21.15.27:0/4293655572 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f286c0cf660 msgr2=0x7f286c15e570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:43.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:43.077+0000 7f287348d640 1 --2- 172.21.15.27:0/4293655572 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f286c0cf660 0x7f286c15e570 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f286400b9e0 tx=0x7f286400bea0 comp rx=0 tx=0).stop 2024-09-12T03:19:43.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:43.077+0000 7f287348d640 1 -- 172.21.15.27:0/4293655572 shutdown_connections 2024-09-12T03:19:43.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:43.077+0000 7f287348d640 1 --2- 172.21.15.27:0/4293655572 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f285006c070 0x7f285006e530 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:43.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:43.077+0000 7f287348d640 1 --2- 172.21.15.27:0/4293655572 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f286c0cf660 0x7f286c15e570 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:43.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:43.077+0000 7f287348d640 1 --2- 172.21.15.27:0/4293655572 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f286c0b1ca0 0x7f286c0cda00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:43.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:43.077+0000 7f287348d640 1 --2- 172.21.15.27:0/4293655572 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f286c06f360 0x7f286c0cc020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:43.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:43.077+0000 7f287348d640 1 -- 172.21.15.27:0/4293655572 >> 172.21.15.27:0/4293655572 conn(0x7f286c073800 msgr2=0x7f286c0b2a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:43.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:43.077+0000 7f287348d640 1 -- 172.21.15.27:0/4293655572 shutdown_connections 2024-09-12T03:19:43.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:43.077+0000 7f287348d640 1 -- 172.21.15.27:0/4293655572 wait complete. 2024-09-12T03:19:43.087 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol82580 2024-09-12T03:19:43.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:43.193+0000 7ff2685c4640 1 Processor -- start 2024-09-12T03:19:43.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:43.193+0000 7ff2685c4640 1 -- start start 2024-09-12T03:19:43.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:43.194+0000 7ff2685c4640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ff2600d4040 0x7ff2600b7ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:43.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:43.194+0000 7ff2685c4640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff2600d2930 con 0x7ff2600b6240 2024-09-12T03:19:43.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:43.194+0000 7ff2685c4640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff2600d2c10 con 0x7ff2600d3c20 2024-09-12T03:19:43.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:43.194+0000 7ff2685c4640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7ff2600d2ef0 con 0x7ff2600d4040 2024-09-12T03:19:43.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:43.194+0000 7ff266360640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7ff2600b6240 0x7ff2600b6620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:46014/0 (socket says 172.21.15.27:46014) 2024-09-12T03:19:43.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:43.194+0000 7ff266360640 1 -- 172.21.15.27:0/1710594669 learned_addr learned my addr 172.21.15.27:0/1710594669 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:19:43.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:43.194+0000 7ff266b61640 1 --2- 172.21.15.27:0/1710594669 >> v2:172.21.15.45:3300/0 conn(0x7ff2600d4040 0x7ff2600b7ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:43.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:43.194+0000 7ff26535e640 1 -- 172.21.15.27:0/1710594669 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1652398954 0 0) 0x7ff2600d2930 con 0x7ff2600b6240 2024-09-12T03:19:43.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:43.195+0000 7ff26535e640 1 -- 172.21.15.27:0/1710594669 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff24c003520 con 0x7ff2600b6240 2024-09-12T03:19:43.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:43.195+0000 7ff26535e640 1 -- 172.21.15.27:0/1710594669 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2496237683 0 0) 0x7ff24c003520 con 0x7ff2600b6240 2024-09-12T03:19:43.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:43.195+0000 7ff26535e640 1 -- 172.21.15.27:0/1710594669 >> v2:172.21.15.45:3300/0 conn(0x7ff2600d4040 msgr2=0x7ff2600b7ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:43.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:43.195+0000 7ff26535e640 1 --2- 172.21.15.27:0/1710594669 >> v2:172.21.15.45:3300/0 conn(0x7ff2600d4040 0x7ff2600b7ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:43.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:43.195+0000 7ff26535e640 1 -- 172.21.15.27:0/1710594669 >> v1:172.21.15.45:6789/0 conn(0x7ff2600d3c20 legacy=0x7ff2600b77c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:43.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:43.195+0000 7ff26535e640 1 -- 172.21.15.27:0/1710594669 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2600d31d0 con 0x7ff2600b6240 2024-09-12T03:19:43.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:43.195+0000 7ff26535e640 1 -- 172.21.15.27:0/1710594669 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 847755213 0 0) 0x7ff25c001670 con 0x7ff2600b6240 2024-09-12T03:19:43.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:43.195+0000 7ff26535e640 1 -- 172.21.15.27:0/1710594669 >> v1:172.21.15.27:6789/0 conn(0x7ff2600b6240 legacy=0x7ff2600b6620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:43.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:43.195+0000 7ff26535e640 1 --2- 172.21.15.27:0/1710594669 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff24c0039b0 0x7ff24c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:43.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:43.195+0000 7ff26535e640 1 --2- 172.21.15.27:0/1710594669 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ff24c004800 0x7ff24c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:43.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:43.195+0000 7ff26535e640 1 --2- 172.21.15.27:0/1710594669 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff24c0051c0 0x7ff24c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:43.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:43.195+0000 7ff26535e640 1 -- 172.21.15.27:0/1710594669 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7ff2600d2ef0 con 0x7ff24c0039b0 2024-09-12T03:19:43.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:43.195+0000 7ff26535e640 1 -- 172.21.15.27:0/1710594669 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ff24c00c070 con 0x7ff24c0051c0 2024-09-12T03:19:43.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:43.195+0000 7ff26535e640 1 -- 172.21.15.27:0/1710594669 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7ff24c00c270 con 0x7ff24c004800 2024-09-12T03:19:43.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:43.195+0000 7ff266360640 1 --2- 172.21.15.27:0/1710594669 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff24c0039b0 0x7ff24c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:43.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:43.195+0000 7ff266b61640 1 --2- 172.21.15.27:0/1710594669 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff24c0051c0 0x7ff24c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:43.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:43.195+0000 7ff265b5f640 1 --2- 172.21.15.27:0/1710594669 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ff24c004800 0x7ff24c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:43.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:43.195+0000 7ff266360640 1 -- 172.21.15.27:0/1710594669 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ff24c004800 msgr2=0x7ff24c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:43.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:43.195+0000 7ff266360640 1 --2- 172.21.15.27:0/1710594669 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ff24c004800 0x7ff24c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:43.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:43.195+0000 7ff266360640 1 -- 172.21.15.27:0/1710594669 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff24c0051c0 msgr2=0x7ff24c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:43.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:43.195+0000 7ff266360640 1 --2- 172.21.15.27:0/1710594669 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff24c0051c0 0x7ff24c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:43.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:43.195+0000 7ff266360640 1 -- 172.21.15.27:0/1710594669 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff24c003520 con 0x7ff24c0039b0 2024-09-12T03:19:43.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:43.195+0000 7ff266b61640 1 -- 172.21.15.27:0/1710594669 reap_dead start 2024-09-12T03:19:43.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:43.196+0000 7ff266360640 1 --2- 172.21.15.27:0/1710594669 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff24c0039b0 0x7ff24c003dd0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7ff25c00c210 tx=0x7ff24c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2d8876a1fe5bafe server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:43.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:43.196+0000 7ff26535e640 1 -- 172.21.15.27:0/1710594669 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff25c007d30 con 0x7ff24c0039b0 2024-09-12T03:19:43.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:43.196+0000 7ff26535e640 1 -- 172.21.15.27:0/1710594669 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff25c0058d0 con 0x7ff24c0039b0 2024-09-12T03:19:43.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:43.196+0000 7ff26535e640 1 -- 172.21.15.27:0/1710594669 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff25c005d20 con 0x7ff24c0039b0 2024-09-12T03:19:43.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:43.197+0000 7ff2685c4640 1 -- 172.21.15.27:0/1710594669 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff24c0039b0 msgr2=0x7ff24c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:43.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:43.197+0000 7ff2685c4640 1 --2- 172.21.15.27:0/1710594669 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff24c0039b0 0x7ff24c003dd0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7ff25c00c210 tx=0x7ff24c005830 comp rx=0 tx=0).stop 2024-09-12T03:19:43.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:43.197+0000 7ff2685c4640 1 -- 172.21.15.27:0/1710594669 shutdown_connections 2024-09-12T03:19:43.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:43.197+0000 7ff2685c4640 1 --2- 172.21.15.27:0/1710594669 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff24c0039b0 0x7ff24c003dd0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:43.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:43.197+0000 7ff2685c4640 1 -- 172.21.15.27:0/1710594669 >> 172.21.15.27:0/1710594669 conn(0x7ff2600d1420 msgr2=0x7ff2600d1830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:43.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:43.197+0000 7ff2685c4640 1 -- 172.21.15.27:0/1710594669 shutdown_connections 2024-09-12T03:19:43.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:43.197+0000 7ff2685c4640 1 -- 172.21.15.27:0/1710594669 wait complete. 2024-09-12T03:19:43.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:43.198+0000 7ff2685c4640 1 Processor -- start 2024-09-12T03:19:43.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:43.198+0000 7ff2685c4640 1 -- start start 2024-09-12T03:19:43.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:43.199+0000 7ff2685c4640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff24c0039b0 0x7ff2600c9610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:43.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:43.199+0000 7ff2685c4640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ff2601f4460 0x7ff2600cc9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:43.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:43.199+0000 7ff2685c4640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff260161290 0x7ff26015a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:43.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:43.199+0000 7ff2685c4640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7ff2600cfdf0 con 0x7ff24c0039b0 2024-09-12T03:19:43.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:43.199+0000 7ff2685c4640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ff2600cfc70 con 0x7ff260161290 2024-09-12T03:19:43.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:43.199+0000 7ff2685c4640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7ff2600b6620 con 0x7ff2601f4460 2024-09-12T03:19:43.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:43.199+0000 7ff266360640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff24c0039b0 0x7ff2600c9610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:43.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:43.199+0000 7ff266360640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff24c0039b0 0x7ff2600c9610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58210/0 (socket says 172.21.15.27:58210) 2024-09-12T03:19:43.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:43.199+0000 7ff266360640 1 -- 172.21.15.27:0/1510094768 learned_addr learned my addr 172.21.15.27:0/1510094768 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:19:43.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:43.199+0000 7ff265b5f640 1 --2- 172.21.15.27:0/1510094768 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ff2601f4460 0x7ff2600cc9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:43.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:43.199+0000 7ff266b61640 1 --2- 172.21.15.27:0/1510094768 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff260161290 0x7ff26015a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:43.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:43.199+0000 7ff266360640 1 -- 172.21.15.27:0/1510094768 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ff2601f4460 msgr2=0x7ff2600cc9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:43.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:43.199+0000 7ff266360640 1 --2- 172.21.15.27:0/1510094768 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ff2601f4460 0x7ff2600cc9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:43.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:43.199+0000 7ff266360640 1 -- 172.21.15.27:0/1510094768 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff260161290 msgr2=0x7ff26015a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:43.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:43.199+0000 7ff266360640 1 --2- 172.21.15.27:0/1510094768 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff260161290 0x7ff26015a1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:43.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:43.199+0000 7ff266360640 1 -- 172.21.15.27:0/1510094768 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2600caf70 con 0x7ff24c0039b0 2024-09-12T03:19:43.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:43.199+0000 7ff265b5f640 1 --2- 172.21.15.27:0/1510094768 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ff2601f4460 0x7ff2600cc9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:19:43.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:43.199+0000 7ff266360640 1 --2- 172.21.15.27:0/1510094768 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff24c0039b0 0x7ff2600c9610 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7ff25c00c210 tx=0x7ff25c001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:43.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:43.199+0000 7ff256ffd640 1 -- 172.21.15.27:0/1510094768 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff25c038040 con 0x7ff24c0039b0 2024-09-12T03:19:43.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:43.199+0000 7ff256ffd640 1 -- 172.21.15.27:0/1510094768 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff25c042070 con 0x7ff24c0039b0 2024-09-12T03:19:43.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:43.199+0000 7ff2685c4640 1 -- 172.21.15.27:0/1510094768 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2600cb210 con 0x7ff24c0039b0 2024-09-12T03:19:43.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:43.200+0000 7ff256ffd640 1 -- 172.21.15.27:0/1510094768 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff25c007270 con 0x7ff24c0039b0 2024-09-12T03:19:43.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:43.200+0000 7ff2685c4640 1 -- 172.21.15.27:0/1510094768 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2600cd630 con 0x7ff24c0039b0 2024-09-12T03:19:43.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:43.200+0000 7ff2685c4640 1 -- 172.21.15.27:0/1510094768 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2600cb3e0 con 0x7ff24c0039b0 2024-09-12T03:19:43.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:43.201+0000 7ff256ffd640 1 -- 172.21.15.27:0/1510094768 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 10) ==== 90702+0+0 (secure 0 0 0) 0x7ff25c007410 con 0x7ff24c0039b0 2024-09-12T03:19:43.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:43.201+0000 7ff256ffd640 1 --2- 172.21.15.27:0/1510094768 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7ff24406c0a0 0x7ff24406e560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:43.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:43.201+0000 7ff256ffd640 1 -- 172.21.15.27:0/1510094768 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5709+0+0 (secure 0 0 0) 0x7ff25c0ab050 con 0x7ff24c0039b0 2024-09-12T03:19:43.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:43.202+0000 7ff265b5f640 1 --2- 172.21.15.27:0/1510094768 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7ff24406c0a0 0x7ff24406e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:43.206 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:43.202+0000 7ff265b5f640 1 --2- 172.21.15.27:0/1510094768 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7ff24406c0a0 0x7ff24406e560 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7ff2600d1df0 tx=0x7ff250009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:43.206 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:43.206+0000 7ff256ffd640 1 -- 172.21.15.27:0/1510094768 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7ff25c0796a0 con 0x7ff24c0039b0 2024-09-12T03:19:43.406 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:43.406+0000 7ff2685c4640 1 -- 172.21.15.27:0/1510094768 --> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol82580", "target": ["mon-mgr", ""]}) -- 0x7ff2600b6950 con 0x7ff24406c0a0 2024-09-12T03:19:43.422 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:43.422+0000 7ff256ffd640 1 -- 172.21.15.27:0/1510094768 <== mgr.4102 v2:172.21.15.45:6832/3923821219 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7ff2600b6950 con 0x7ff24406c0a0 2024-09-12T03:19:43.425 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:43.424+0000 7ff2685c4640 1 -- 172.21.15.27:0/1510094768 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7ff24406c0a0 msgr2=0x7ff24406e560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:43.425 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:43.424+0000 7ff2685c4640 1 --2- 172.21.15.27:0/1510094768 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7ff24406c0a0 0x7ff24406e560 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7ff2600d1df0 tx=0x7ff250009040 comp rx=0 tx=0).stop 2024-09-12T03:19:43.425 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:43.424+0000 7ff2685c4640 1 -- 172.21.15.27:0/1510094768 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff24c0039b0 msgr2=0x7ff2600c9610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:43.425 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:43.424+0000 7ff2685c4640 1 --2- 172.21.15.27:0/1510094768 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff24c0039b0 0x7ff2600c9610 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7ff25c00c210 tx=0x7ff25c001a40 comp rx=0 tx=0).stop 2024-09-12T03:19:43.425 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:43.424+0000 7ff2685c4640 1 -- 172.21.15.27:0/1510094768 shutdown_connections 2024-09-12T03:19:43.425 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:43.424+0000 7ff2685c4640 1 --2- 172.21.15.27:0/1510094768 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7ff24406c0a0 0x7ff24406e560 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:43.425 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:43.424+0000 7ff2685c4640 1 --2- 172.21.15.27:0/1510094768 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff260161290 0x7ff26015a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:43.425 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:43.424+0000 7ff2685c4640 1 --2- 172.21.15.27:0/1510094768 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ff2601f4460 0x7ff2600cc9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:43.425 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:43.424+0000 7ff2685c4640 1 --2- 172.21.15.27:0/1510094768 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff24c0039b0 0x7ff2600c9610 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:43.426 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:43.425+0000 7ff2685c4640 1 -- 172.21.15.27:0/1510094768 >> 172.21.15.27:0/1510094768 conn(0x7ff2600d1420 msgr2=0x7ff2600d47a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:43.426 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:43.425+0000 7ff2685c4640 1 -- 172.21.15.27:0/1510094768 shutdown_connections 2024-09-12T03:19:43.426 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:43.425+0000 7ff2685c4640 1 -- 172.21.15.27:0/1510094768 wait complete. 2024-09-12T03:19:43.426 INFO:tasks.ceph.mgr.x.smithi045.stderr:2024-09-12T03:19:43.425+0000 7fc172fb3640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:19:43.426 INFO:tasks.ceph.mgr.x.smithi045.stderr:2024-09-12T03:19:43.425+0000 7fc172fb3640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:19:43.426 INFO:tasks.ceph.mgr.x.smithi045.stderr:2024-09-12T03:19:43.425+0000 7fc172fb3640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:19:43.426 INFO:tasks.ceph.mgr.x.smithi045.stderr:2024-09-12T03:19:43.425+0000 7fc172fb3640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:19:43.426 INFO:tasks.ceph.mgr.x.smithi045.stderr:2024-09-12T03:19:43.425+0000 7fc172fb3640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:19:43.436 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol65231 2024-09-12T03:19:43.440 INFO:tasks.ceph.mgr.x.smithi045.stderr:2024-09-12T03:19:43.440+0000 7fc171fb1640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:19:43.441 INFO:tasks.ceph.mgr.x.smithi045.stderr:2024-09-12T03:19:43.440+0000 7fc171fb1640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:19:43.441 INFO:tasks.ceph.mgr.x.smithi045.stderr:2024-09-12T03:19:43.440+0000 7fc171fb1640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:19:43.441 INFO:tasks.ceph.mgr.x.smithi045.stderr:2024-09-12T03:19:43.440+0000 7fc171fb1640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:19:43.441 INFO:tasks.ceph.mgr.x.smithi045.stderr:2024-09-12T03:19:43.440+0000 7fc171fb1640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:19:43.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:43.541+0000 7f775b703640 1 Processor -- start 2024-09-12T03:19:43.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:43.541+0000 7f775b703640 1 -- start start 2024-09-12T03:19:43.543 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:43.542+0000 7f775b703640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f7754070d30 0x7f775406cc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:43.543 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:43.542+0000 7f775b703640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f775406e960 0x7f77540d0440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:43.543 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:43.542+0000 7f775b703640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f77540cf9f0 con 0x7f77540d0980 2024-09-12T03:19:43.543 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:43.542+0000 7f775b703640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f77540d1a40 con 0x7f7754070d30 2024-09-12T03:19:43.543 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:43.542+0000 7f775b703640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f77540cfc90 con 0x7f775406e960 2024-09-12T03:19:43.543 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:43.542+0000 7f775949f640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f7754070d30 0x7f775406cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:43.543 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:43.542+0000 7f775949f640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f7754070d30 0x7f775406cc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58216/0 (socket says 172.21.15.27:58216) 2024-09-12T03:19:43.543 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:43.542+0000 7f7759ca0640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f77540d0980 0x7f77540d21b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:46028/0 (socket says 172.21.15.27:46028) 2024-09-12T03:19:43.543 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:43.542+0000 7f775949f640 1 -- 172.21.15.27:0/637767841 learned_addr learned my addr 172.21.15.27:0/637767841 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:19:43.543 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:43.542+0000 7f7758c9e640 1 --2- 172.21.15.27:0/637767841 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f775406e960 0x7f77540d0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:43.543 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:43.543+0000 7f775949f640 1 -- 172.21.15.27:0/637767841 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f775406e960 msgr2=0x7f77540d0440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:43.544 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:43.543+0000 7f775949f640 1 --2- 172.21.15.27:0/637767841 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f775406e960 0x7f77540d0440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:43.544 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:43.543+0000 7f775949f640 1 -- 172.21.15.27:0/637767841 >> v1:172.21.15.27:6789/0 conn(0x7f77540d0980 legacy=0x7f77540d21b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:43.544 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:43.543+0000 7f775949f640 1 -- 172.21.15.27:0/637767841 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77540cff70 con 0x7f7754070d30 2024-09-12T03:19:43.544 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:43.543+0000 7f774bfff640 1 -- 172.21.15.27:0/637767841 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3590521352 0 0) 0x7f77540cf9f0 con 0x7f77540d0980 2024-09-12T03:19:43.545 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:43.543+0000 7f775949f640 1 --2- 172.21.15.27:0/637767841 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7754070d30 0x7f775406cc40 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f7750009870 tx=0x7f775002a130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee5311fce63b70bb server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:43.545 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:43.543+0000 7f774bfff640 1 -- 172.21.15.27:0/637767841 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7750037070 con 0x7f7754070d30 2024-09-12T03:19:43.545 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:43.544+0000 7f774bfff640 1 -- 172.21.15.27:0/637767841 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7750032440 con 0x7f7754070d30 2024-09-12T03:19:43.545 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:43.544+0000 7f774bfff640 1 -- 172.21.15.27:0/637767841 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7750032810 con 0x7f7754070d30 2024-09-12T03:19:43.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:43.544+0000 7f775b703640 1 -- 172.21.15.27:0/637767841 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7754070d30 msgr2=0x7f775406cc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:43.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:43.544+0000 7f775b703640 1 --2- 172.21.15.27:0/637767841 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7754070d30 0x7f775406cc40 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f7750009870 tx=0x7f775002a130 comp rx=0 tx=0).stop 2024-09-12T03:19:43.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:43.544+0000 7f775b703640 1 -- 172.21.15.27:0/637767841 shutdown_connections 2024-09-12T03:19:43.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:43.544+0000 7f775b703640 1 --2- 172.21.15.27:0/637767841 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f775406e960 0x7f77540d0440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:43.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:43.544+0000 7f775b703640 1 --2- 172.21.15.27:0/637767841 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7754070d30 0x7f775406cc40 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:43.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:43.544+0000 7f775b703640 1 -- 172.21.15.27:0/637767841 >> 172.21.15.27:0/637767841 conn(0x7f77540b7c10 msgr2=0x7f775409ba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:43.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:43.544+0000 7f775b703640 1 -- 172.21.15.27:0/637767841 shutdown_connections 2024-09-12T03:19:43.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:43.544+0000 7f775b703640 1 -- 172.21.15.27:0/637767841 wait complete. 2024-09-12T03:19:43.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:43.545+0000 7f775b703640 1 Processor -- start 2024-09-12T03:19:43.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:43.545+0000 7f775b703640 1 -- start start 2024-09-12T03:19:43.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:43.546+0000 7f775b703640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f775406e960 0x7f7754154870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:43.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:43.546+0000 7f775b703640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f7754070d30 0x7f77540ccbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:43.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:43.546+0000 7f775b703640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f77540d0980 0x7f77540b3970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:43.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:43.546+0000 7f775b703640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f77540d28c0 con 0x7f77540d0980 2024-09-12T03:19:43.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:43.546+0000 7f775b703640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f775406d180 con 0x7f775406e960 2024-09-12T03:19:43.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:43.546+0000 7f775b703640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f77540d2a40 con 0x7f7754070d30 2024-09-12T03:19:43.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:43.546+0000 7f7759ca0640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f77540d0980 0x7f77540b3970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:43.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:43.546+0000 7f7759ca0640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f77540d0980 0x7f77540b3970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58218/0 (socket says 172.21.15.27:58218) 2024-09-12T03:19:43.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:43.546+0000 7f7759ca0640 1 -- 172.21.15.27:0/871795065 learned_addr learned my addr 172.21.15.27:0/871795065 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:19:43.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:43.546+0000 7f7758c9e640 1 --2- 172.21.15.27:0/871795065 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f7754070d30 0x7f77540ccbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:43.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:43.546+0000 7f775949f640 1 --2- 172.21.15.27:0/871795065 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f775406e960 0x7f7754154870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:43.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:43.547+0000 7f7759ca0640 1 -- 172.21.15.27:0/871795065 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f7754070d30 msgr2=0x7f77540ccbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:43.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:43.547+0000 7f7759ca0640 1 --2- 172.21.15.27:0/871795065 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f7754070d30 0x7f77540ccbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:43.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:43.547+0000 7f7759ca0640 1 -- 172.21.15.27:0/871795065 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f775406e960 msgr2=0x7f7754154870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:43.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:43.547+0000 7f7759ca0640 1 --2- 172.21.15.27:0/871795065 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f775406e960 0x7f7754154870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:43.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:43.547+0000 7f7759ca0640 1 -- 172.21.15.27:0/871795065 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7754154db0 con 0x7f77540d0980 2024-09-12T03:19:43.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:43.547+0000 7f775949f640 1 --2- 172.21.15.27:0/871795065 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f775406e960 0x7f7754154870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:19:43.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:43.547+0000 7f7759ca0640 1 --2- 172.21.15.27:0/871795065 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f77540d0980 0x7f77540b3970 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f774c00c700 tx=0x7f774c00cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:43.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:43.547+0000 7f7749ffb640 1 -- 172.21.15.27:0/871795065 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f774c018070 con 0x7f77540d0980 2024-09-12T03:19:43.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:43.547+0000 7f7749ffb640 1 -- 172.21.15.27:0/871795065 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f774c00ce10 con 0x7f77540d0980 2024-09-12T03:19:43.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:43.547+0000 7f7749ffb640 1 -- 172.21.15.27:0/871795065 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f774c00a860 con 0x7f77540d0980 2024-09-12T03:19:43.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:43.547+0000 7f775b703640 1 -- 172.21.15.27:0/871795065 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77540cd4e0 con 0x7f77540d0980 2024-09-12T03:19:43.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:43.547+0000 7f775b703640 1 -- 172.21.15.27:0/871795065 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77540cdaa0 con 0x7f77540d0980 2024-09-12T03:19:43.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:43.548+0000 7f775b703640 1 -- 172.21.15.27:0/871795065 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77540d0440 con 0x7f77540d0980 2024-09-12T03:19:43.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:43.548+0000 7f7749ffb640 1 -- 172.21.15.27:0/871795065 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 10) ==== 90702+0+0 (secure 0 0 0) 0x7f774c01b070 con 0x7f77540d0980 2024-09-12T03:19:43.551 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:43.549+0000 7f7749ffb640 1 --2- 172.21.15.27:0/871795065 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f773806c070 0x7f773806e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:43.551 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:43.549+0000 7f7749ffb640 1 -- 172.21.15.27:0/871795065 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5709+0+0 (secure 0 0 0) 0x7f774c08a800 con 0x7f77540d0980 2024-09-12T03:19:43.551 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:43.549+0000 7f775949f640 1 --2- 172.21.15.27:0/871795065 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f773806c070 0x7f773806e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:43.551 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:43.550+0000 7f775949f640 1 --2- 172.21.15.27:0/871795065 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f773806c070 0x7f773806e530 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f7750034000 tx=0x7f775002e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:43.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:43.553+0000 7f7749ffb640 1 -- 172.21.15.27:0/871795065 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f774c058e50 con 0x7f77540d0980 2024-09-12T03:19:43.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:43.752+0000 7f775b703640 1 -- 172.21.15.27:0/871795065 --> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol65231", "target": ["mon-mgr", ""]}) -- 0x7f77540b79d0 con 0x7f773806c070 2024-09-12T03:19:43.772 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:43.772+0000 7f7749ffb640 1 -- 172.21.15.27:0/871795065 <== mgr.4102 v2:172.21.15.45:6832/3923821219 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f77540b79d0 con 0x7f773806c070 2024-09-12T03:19:43.774 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:43.774+0000 7f775b703640 1 -- 172.21.15.27:0/871795065 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f773806c070 msgr2=0x7f773806e530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:43.774 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:43.774+0000 7f775b703640 1 --2- 172.21.15.27:0/871795065 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f773806c070 0x7f773806e530 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f7750034000 tx=0x7f775002e000 comp rx=0 tx=0).stop 2024-09-12T03:19:43.774 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:43.774+0000 7f775b703640 1 -- 172.21.15.27:0/871795065 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f77540d0980 msgr2=0x7f77540b3970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:43.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:43.774+0000 7f775b703640 1 --2- 172.21.15.27:0/871795065 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f77540d0980 0x7f77540b3970 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f774c00c700 tx=0x7f774c00cbc0 comp rx=0 tx=0).stop 2024-09-12T03:19:43.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:43.774+0000 7f775b703640 1 -- 172.21.15.27:0/871795065 shutdown_connections 2024-09-12T03:19:43.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:43.774+0000 7f775b703640 1 --2- 172.21.15.27:0/871795065 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f773806c070 0x7f773806e530 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:43.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:43.774+0000 7f775b703640 1 --2- 172.21.15.27:0/871795065 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f77540d0980 0x7f77540b3970 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:43.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:43.774+0000 7f775b703640 1 --2- 172.21.15.27:0/871795065 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f7754070d30 0x7f77540ccbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:43.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:43.774+0000 7f775b703640 1 --2- 172.21.15.27:0/871795065 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f775406e960 0x7f7754154870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:43.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:43.774+0000 7f775b703640 1 -- 172.21.15.27:0/871795065 >> 172.21.15.27:0/871795065 conn(0x7f77540b7c10 msgr2=0x7f77540b54d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:43.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:43.774+0000 7f775b703640 1 -- 172.21.15.27:0/871795065 shutdown_connections 2024-09-12T03:19:43.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:43.774+0000 7f775b703640 1 -- 172.21.15.27:0/871795065 wait complete. 2024-09-12T03:19:43.785 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-09-12T03:19:43.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:43 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-12T03:19:43.840 INFO:teuthology.orchestra.run.smithi027.stdout:2 2024-09-12T03:19:43.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T03:19:43.846 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-12T03:19:43.847 DEBUG:teuthology.orchestra.run.smithi027:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-12T03:19:43.887 DEBUG:teuthology.orchestra.run.smithi045:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-12T03:19:43.927 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-12T03:19:43.927 INFO:tasks.cephfs.mount:Terminating background process 2024-09-12T03:19:43.928 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-12T03:19:43.928 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T03:19:43.944 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-12T03:19:43.944 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T03:19:44.004 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-12T03:19:44.145 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:44.145+0000 7f7fdd1b7640 1 Processor -- start 2024-09-12T03:19:44.145 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:44.145+0000 7f7fdd1b7640 1 -- start start 2024-09-12T03:19:44.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:44.145+0000 7f7fdd1b7640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f7fd806f360 0x7f7fd80cef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:44.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:44.145+0000 7f7fdd1b7640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7fd806cc40 con 0x7f7fd80b1ca0 2024-09-12T03:19:44.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:44.145+0000 7f7fdd1b7640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7fd80d1ce0 con 0x7f7fd80cf490 2024-09-12T03:19:44.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:44.145+0000 7f7fdd1b7640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f7fd80cf900 con 0x7f7fd806f360 2024-09-12T03:19:44.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:44.146+0000 7f7fd659c640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f7fd806f360 0x7f7fd80cef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:44.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:44.146+0000 7f7fd6d9d640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f7fd80b1ca0 0x7f7fd815f830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:46032/0 (socket says 172.21.15.27:46032) 2024-09-12T03:19:44.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:44.146+0000 7f7fd659c640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f7fd806f360 0x7f7fd80cef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58224/0 (socket says 172.21.15.27:58224) 2024-09-12T03:19:44.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:44.146+0000 7f7fd6d9d640 1 -- 172.21.15.27:0/4062484213 learned_addr learned my addr 172.21.15.27:0/4062484213 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:19:44.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:44.146+0000 7f7fd659c640 1 -- 172.21.15.27:0/4062484213 >> v1:172.21.15.45:6789/0 conn(0x7f7fd80cf490 legacy=0x7f7fd80d2780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:44.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:44.146+0000 7f7fd659c640 1 -- 172.21.15.27:0/4062484213 >> v1:172.21.15.27:6789/0 conn(0x7f7fd80b1ca0 legacy=0x7f7fd815f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:44.147 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:44.146+0000 7f7fd659c640 1 -- 172.21.15.27:0/4062484213 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fd806cee0 con 0x7f7fd806f360 2024-09-12T03:19:44.147 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:44.146+0000 7f7fd5d9b640 1 -- 172.21.15.27:0/4062484213 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1966439494 0 0) 0x7f7fd806cc40 con 0x7f7fd80b1ca0 2024-09-12T03:19:44.148 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:44.147+0000 7f7fd659c640 1 --2- 172.21.15.27:0/4062484213 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7fd806f360 0x7f7fd80cef50 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f7fd80d1eb0 tx=0x7f7fc802a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5cb77a28b0b73ba server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:44.148 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:44.147+0000 7f7fd5d9b640 1 -- 172.21.15.27:0/4062484213 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fc8037070 con 0x7f7fd806f360 2024-09-12T03:19:44.148 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:44.147+0000 7f7fd5d9b640 1 -- 172.21.15.27:0/4062484213 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7fc8032440 con 0x7f7fd806f360 2024-09-12T03:19:44.149 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:44.147+0000 7f7fd5d9b640 1 -- 172.21.15.27:0/4062484213 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fc8032810 con 0x7f7fd806f360 2024-09-12T03:19:44.149 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:44.147+0000 7f7fdd1b7640 1 -- 172.21.15.27:0/4062484213 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7fd806f360 msgr2=0x7f7fd80cef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:44.149 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:44.147+0000 7f7fdd1b7640 1 --2- 172.21.15.27:0/4062484213 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7fd806f360 0x7f7fd80cef50 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f7fd80d1eb0 tx=0x7f7fc802a1c0 comp rx=0 tx=0).stop 2024-09-12T03:19:44.149 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:44.147+0000 7f7fdd1b7640 1 -- 172.21.15.27:0/4062484213 shutdown_connections 2024-09-12T03:19:44.149 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:44.147+0000 7f7fdd1b7640 1 --2- 172.21.15.27:0/4062484213 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7fd806f360 0x7f7fd80cef50 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:44.149 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:44.147+0000 7f7fdd1b7640 1 -- 172.21.15.27:0/4062484213 >> 172.21.15.27:0/4062484213 conn(0x7f7fd8073800 msgr2=0x7f7fd809ba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:44.149 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:44.147+0000 7f7fdd1b7640 1 -- 172.21.15.27:0/4062484213 shutdown_connections 2024-09-12T03:19:44.149 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:44.147+0000 7f7fdd1b7640 1 -- 172.21.15.27:0/4062484213 wait complete. 2024-09-12T03:19:44.152 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:44.149+0000 7f7fdd1b7640 1 Processor -- start 2024-09-12T03:19:44.152 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:44.149+0000 7f7fdd1b7640 1 -- start start 2024-09-12T03:19:44.153 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:44.150+0000 7f7fdd1b7640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7fd806f360 0x7f7fd814e080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:44.153 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:44.150+0000 7f7fdd1b7640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f7fd80b1ca0 0x7f7fd80cb850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:44.153 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:44.150+0000 7f7fdd1b7640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7fd80cf490 0x7f7fd80cc620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:44.153 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:44.150+0000 7f7fdd1b7640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f7fd806d1c0 con 0x7f7fd806f360 2024-09-12T03:19:44.153 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:44.150+0000 7f7fdd1b7640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f7fd80d22f0 con 0x7f7fd80cf490 2024-09-12T03:19:44.153 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:44.150+0000 7f7fdd1b7640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f7fd806d340 con 0x7f7fd80b1ca0 2024-09-12T03:19:44.153 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:44.150+0000 7f7fd6d9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7fd806f360 0x7f7fd814e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:44.154 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:44.150+0000 7f7fd6d9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7fd806f360 0x7f7fd814e080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58226/0 (socket says 172.21.15.27:58226) 2024-09-12T03:19:44.154 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:44.150+0000 7f7fd6d9d640 1 -- 172.21.15.27:0/3503100776 learned_addr learned my addr 172.21.15.27:0/3503100776 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:19:44.154 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:44.150+0000 7f7fd659c640 1 --2- 172.21.15.27:0/3503100776 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f7fd80b1ca0 0x7f7fd80cb850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:44.154 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:44.150+0000 7f7fd759e640 1 --2- 172.21.15.27:0/3503100776 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7fd80cf490 0x7f7fd80cc620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:44.154 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:44.150+0000 7f7fd6d9d640 1 -- 172.21.15.27:0/3503100776 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f7fd80b1ca0 msgr2=0x7f7fd80cb850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:44.154 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:44.150+0000 7f7fd6d9d640 1 --2- 172.21.15.27:0/3503100776 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f7fd80b1ca0 0x7f7fd80cb850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:44.154 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:44.150+0000 7f7fd6d9d640 1 -- 172.21.15.27:0/3503100776 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7fd80cf490 msgr2=0x7f7fd80cc620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:44.154 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:44.150+0000 7f7fd6d9d640 1 --2- 172.21.15.27:0/3503100776 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7fd80cf490 0x7f7fd80cc620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:44.154 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:44.150+0000 7f7fd6d9d640 1 -- 172.21.15.27:0/3503100776 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fd8161110 con 0x7f7fd806f360 2024-09-12T03:19:44.155 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:44.150+0000 7f7fd6d9d640 1 --2- 172.21.15.27:0/3503100776 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7fd806f360 0x7f7fd814e080 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f7fc000c4d0 tx=0x7f7fc000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:44.155 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:44.150+0000 7f7fcf7fe640 1 -- 172.21.15.27:0/3503100776 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fc0018070 con 0x7f7fd806f360 2024-09-12T03:19:44.155 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:44.150+0000 7f7fcf7fe640 1 -- 172.21.15.27:0/3503100776 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7fc0007500 con 0x7f7fd806f360 2024-09-12T03:19:44.155 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:44.150+0000 7f7fdd1b7640 1 -- 172.21.15.27:0/3503100776 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7fd81613f0 con 0x7f7fd806f360 2024-09-12T03:19:44.155 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:44.150+0000 7f7fcf7fe640 1 -- 172.21.15.27:0/3503100776 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fc0004a20 con 0x7f7fd806f360 2024-09-12T03:19:44.155 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:44.150+0000 7f7fdd1b7640 1 -- 172.21.15.27:0/3503100776 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7fd81619b0 con 0x7f7fd806f360 2024-09-12T03:19:44.155 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:44.151+0000 7f7fdd1b7640 1 -- 172.21.15.27:0/3503100776 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7fd80cfc40 con 0x7f7fd806f360 2024-09-12T03:19:44.155 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:44.152+0000 7f7fcf7fe640 1 -- 172.21.15.27:0/3503100776 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 10) ==== 90702+0+0 (secure 0 0 0) 0x7f7fc001b070 con 0x7f7fd806f360 2024-09-12T03:19:44.155 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:44.152+0000 7f7fcf7fe640 1 --2- 172.21.15.27:0/3503100776 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f7fb806c070 0x7f7fb806e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:44.155 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:44.153+0000 7f7fcf7fe640 1 -- 172.21.15.27:0/3503100776 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5709+0+0 (secure 0 0 0) 0x7f7fc008a600 con 0x7f7fd806f360 2024-09-12T03:19:44.155 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:44.153+0000 7f7fd659c640 1 --2- 172.21.15.27:0/3503100776 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f7fb806c070 0x7f7fb806e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:44.156 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:44.153+0000 7f7fd659c640 1 --2- 172.21.15.27:0/3503100776 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f7fb806c070 0x7f7fb806e530 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f7fc8034000 tx=0x7f7fc802e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:44.156 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:44.157+0000 7f7fcf7fe640 1 -- 172.21.15.27:0/3503100776 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f7fc0058c50 con 0x7f7fd806f360 2024-09-12T03:19:44.181 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:44.181+0000 7f7fcf7fe640 1 -- 172.21.15.27:0/3503100776 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mgrmap(e 11) ==== 90804+0+0 (secure 0 0 0) 0x7f7fc0054bc0 con 0x7f7fd806f360 2024-09-12T03:19:44.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:44.345+0000 7f7fdd1b7640 1 -- 172.21.15.27:0/3503100776 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f7fd80b3b20 con 0x7f7fd806f360 2024-09-12T03:19:44.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:44.345+0000 7f7fcf7fe640 1 -- 172.21.15.27:0/3503100776 <== mon.0 v2:172.21.15.27:3300/0 8 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v56) ==== 80+0+0 (secure 0 0 0) 0x7f7fc00787b0 con 0x7f7fd806f360 2024-09-12T03:19:44.345 INFO:teuthology.orchestra.run.smithi027.stderr:listed 0 entries 2024-09-12T03:19:44.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:44.347+0000 7f7fdd1b7640 1 -- 172.21.15.27:0/3503100776 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f7fb806c070 msgr2=0x7f7fb806e530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:44.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:44.347+0000 7f7fdd1b7640 1 --2- 172.21.15.27:0/3503100776 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f7fb806c070 0x7f7fb806e530 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f7fc8034000 tx=0x7f7fc802e000 comp rx=0 tx=0).stop 2024-09-12T03:19:44.348 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:44.347+0000 7f7fdd1b7640 1 -- 172.21.15.27:0/3503100776 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7fd806f360 msgr2=0x7f7fd814e080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:44.348 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:44.347+0000 7f7fdd1b7640 1 --2- 172.21.15.27:0/3503100776 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7fd806f360 0x7f7fd814e080 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f7fc000c4d0 tx=0x7f7fc000c990 comp rx=0 tx=0).stop 2024-09-12T03:19:44.348 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:44.347+0000 7f7fdd1b7640 1 -- 172.21.15.27:0/3503100776 shutdown_connections 2024-09-12T03:19:44.348 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:44.347+0000 7f7fdd1b7640 1 --2- 172.21.15.27:0/3503100776 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f7fb806c070 0x7f7fb806e530 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:44.348 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:44.347+0000 7f7fdd1b7640 1 --2- 172.21.15.27:0/3503100776 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7fd80cf490 0x7f7fd80cc620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:44.348 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:44.347+0000 7f7fdd1b7640 1 --2- 172.21.15.27:0/3503100776 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f7fd80b1ca0 0x7f7fd80cb850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:44.348 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:44.347+0000 7f7fdd1b7640 1 --2- 172.21.15.27:0/3503100776 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7fd806f360 0x7f7fd814e080 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:44.348 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:44.347+0000 7f7fdd1b7640 1 -- 172.21.15.27:0/3503100776 >> 172.21.15.27:0/3503100776 conn(0x7f7fd8073800 msgr2=0x7f7fd80b2570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:44.348 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:44.347+0000 7f7fdd1b7640 1 -- 172.21.15.27:0/3503100776 shutdown_connections 2024-09-12T03:19:44.348 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:44.347+0000 7f7fdd1b7640 1 -- 172.21.15.27:0/3503100776 wait complete. 2024-09-12T03:19:44.358 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-12T03:19:44.358 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:19:44.358 DEBUG:teuthology.orchestra.run.smithi027:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-12T03:19:44.429 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi027.front.sepia.ceph.com 2024-09-12T03:19:44.429 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:19:44.430 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-12T03:19:44.447 INFO:tasks.cephfs.mount:Terminating background process 2024-09-12T03:19:44.447 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-12T03:19:44.447 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T03:19:44.466 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-12T03:19:44.467 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T03:19:44.529 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-12T03:19:44.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:44.633+0000 7fba09dc4640 1 Processor -- start 2024-09-12T03:19:44.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:44.633+0000 7fba09dc4640 1 -- start start 2024-09-12T03:19:44.634 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:44.634+0000 7fba09dc4640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fba040ae8d0 0x7fba040728b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:44.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:44.634+0000 7fba09dc4640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fba04072df0 0x7fba0406e930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:44.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:44.634+0000 7fba09dc4640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fba040aee90 con 0x7fba040b5a70 2024-09-12T03:19:44.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:44.634+0000 7fba09dc4640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7fba040af170 con 0x7fba04072df0 2024-09-12T03:19:44.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:44.634+0000 7fba09dc4640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fba040af450 con 0x7fba040ae8d0 2024-09-12T03:19:44.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:44.634+0000 7fba03fff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fba04072df0 0x7fba0406e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:44.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:44.634+0000 7fba03fff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fba04072df0 0x7fba0406e930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58232/0 (socket says 172.21.15.27:58232) 2024-09-12T03:19:44.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:44.634+0000 7fba03fff640 1 -- 172.21.15.27:0/2407728311 learned_addr learned my addr 172.21.15.27:0/2407728311 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:19:44.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:44.634+0000 7fba02ffd640 1 --2- 172.21.15.27:0/2407728311 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fba040ae8d0 0x7fba040728b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:44.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:44.634+0000 7fba03fff640 1 -- 172.21.15.27:0/2407728311 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fba040ae8d0 msgr2=0x7fba040728b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:44.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:44.634+0000 7fba03fff640 1 --2- 172.21.15.27:0/2407728311 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fba040ae8d0 0x7fba040728b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:44.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:44.634+0000 7fba03fff640 1 -- 172.21.15.27:0/2407728311 >> v1:172.21.15.45:6789/0 conn(0x7fba040b5a70 legacy=0x7fba0415f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:44.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:44.634+0000 7fba03fff640 1 -- 172.21.15.27:0/2407728311 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba040af730 con 0x7fba04072df0 2024-09-12T03:19:44.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:44.635+0000 7fba03fff640 1 --2- 172.21.15.27:0/2407728311 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fba04072df0 0x7fba0406e930 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fba040cd580 tx=0x7fb9fc02a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93322445350febe server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:44.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:44.635+0000 7fba027fc640 1 -- 172.21.15.27:0/2407728311 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9fc036070 con 0x7fba04072df0 2024-09-12T03:19:44.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:44.635+0000 7fba027fc640 1 -- 172.21.15.27:0/2407728311 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9fc031400 con 0x7fba04072df0 2024-09-12T03:19:44.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:44.635+0000 7fba027fc640 1 -- 172.21.15.27:0/2407728311 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9fc0317d0 con 0x7fba04072df0 2024-09-12T03:19:44.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:44.636+0000 7fba09dc4640 1 -- 172.21.15.27:0/2407728311 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fba04072df0 msgr2=0x7fba0406e930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:44.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:44.636+0000 7fba09dc4640 1 --2- 172.21.15.27:0/2407728311 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fba04072df0 0x7fba0406e930 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fba040cd580 tx=0x7fb9fc02a610 comp rx=0 tx=0).stop 2024-09-12T03:19:44.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:44.636+0000 7fba09dc4640 1 -- 172.21.15.27:0/2407728311 shutdown_connections 2024-09-12T03:19:44.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:44.636+0000 7fba09dc4640 1 --2- 172.21.15.27:0/2407728311 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fba04072df0 0x7fba0406e930 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:44.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:44.636+0000 7fba09dc4640 1 --2- 172.21.15.27:0/2407728311 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fba040ae8d0 0x7fba040728b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:44.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:44.636+0000 7fba09dc4640 1 -- 172.21.15.27:0/2407728311 >> 172.21.15.27:0/2407728311 conn(0x7fba0406ca30 msgr2=0x7fba0406ce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:44.638 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:44.636+0000 7fba09dc4640 1 -- 172.21.15.27:0/2407728311 shutdown_connections 2024-09-12T03:19:44.638 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:44.636+0000 7fba09dc4640 1 -- 172.21.15.27:0/2407728311 wait complete. 2024-09-12T03:19:44.638 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:44.637+0000 7fba09dc4640 1 Processor -- start 2024-09-12T03:19:44.639 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:44.637+0000 7fba09dc4640 1 -- start start 2024-09-12T03:19:44.639 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:44.638+0000 7fba09dc4640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fba04072df0 0x7fba04156b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:44.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:44.638+0000 7fba09dc4640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fba040ae8d0 0x7fba041f3e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:44.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:44.638+0000 7fba09dc4640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fba040b5a70 0x7fba040d06d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:44.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:44.638+0000 7fba09dc4640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fba0406f200 con 0x7fba040b5a70 2024-09-12T03:19:44.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:44.638+0000 7fba09dc4640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fba0406f080 con 0x7fba04072df0 2024-09-12T03:19:44.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:44.638+0000 7fba09dc4640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fba0406f380 con 0x7fba040ae8d0 2024-09-12T03:19:44.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:44.638+0000 7fba03fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fba040b5a70 0x7fba040d06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:44.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:44.638+0000 7fba03fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fba040b5a70 0x7fba040d06d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58234/0 (socket says 172.21.15.27:58234) 2024-09-12T03:19:44.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:44.638+0000 7fba03fff640 1 -- 172.21.15.27:0/4211575246 learned_addr learned my addr 172.21.15.27:0/4211575246 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:19:44.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:44.638+0000 7fba02ffd640 1 --2- 172.21.15.27:0/4211575246 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fba040ae8d0 0x7fba041f3e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:44.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:44.638+0000 7fba037fe640 1 --2- 172.21.15.27:0/4211575246 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fba04072df0 0x7fba04156b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:44.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:44.639+0000 7fba03fff640 1 -- 172.21.15.27:0/4211575246 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fba040ae8d0 msgr2=0x7fba041f3e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:44.641 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:44.639+0000 7fba03fff640 1 --2- 172.21.15.27:0/4211575246 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fba040ae8d0 0x7fba041f3e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:44.641 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:44.639+0000 7fba03fff640 1 -- 172.21.15.27:0/4211575246 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fba04072df0 msgr2=0x7fba04156b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:44.641 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:44.639+0000 7fba03fff640 1 --2- 172.21.15.27:0/4211575246 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fba04072df0 0x7fba04156b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:44.641 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:44.639+0000 7fba03fff640 1 -- 172.21.15.27:0/4211575246 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba040ce730 con 0x7fba040b5a70 2024-09-12T03:19:44.641 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:44.639+0000 7fba03fff640 1 --2- 172.21.15.27:0/4211575246 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fba040b5a70 0x7fba040d06d0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fb9fc009f40 tx=0x7fb9fc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:44.641 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:44.639+0000 7fb9ebfff640 1 -- 172.21.15.27:0/4211575246 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9fc036050 con 0x7fba040b5a70 2024-09-12T03:19:44.641 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:44.639+0000 7fb9ebfff640 1 -- 172.21.15.27:0/4211575246 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9fc042070 con 0x7fba040b5a70 2024-09-12T03:19:44.641 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:44.639+0000 7fb9ebfff640 1 -- 172.21.15.27:0/4211575246 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9fc03d8b0 con 0x7fba040b5a70 2024-09-12T03:19:44.641 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:44.639+0000 7fba09dc4640 1 -- 172.21.15.27:0/4211575246 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba041f4340 con 0x7fba040b5a70 2024-09-12T03:19:44.641 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:44.639+0000 7fba09dc4640 1 -- 172.21.15.27:0/4211575246 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba040ce170 con 0x7fba040b5a70 2024-09-12T03:19:44.642 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:44.640+0000 7fba09dc4640 1 -- 172.21.15.27:0/4211575246 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba0406e910 con 0x7fba040b5a70 2024-09-12T03:19:44.642 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:44.640+0000 7fb9ebfff640 1 -- 172.21.15.27:0/4211575246 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 11) ==== 90804+0+0 (secure 0 0 0) 0x7fb9fc03da50 con 0x7fba040b5a70 2024-09-12T03:19:44.642 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:44.641+0000 7fb9ebfff640 1 --2- 172.21.15.27:0/4211575246 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fb9f006c1c0 0x7fb9f006e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:44.642 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:44.641+0000 7fb9ebfff640 1 -- 172.21.15.27:0/4211575246 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5709+0+0 (secure 0 0 0) 0x7fb9fc0ac0a0 con 0x7fba040b5a70 2024-09-12T03:19:44.642 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:44.641+0000 7fba037fe640 1 --2- 172.21.15.27:0/4211575246 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fb9f006c1c0 0x7fb9f006e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:44.643 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:44.642+0000 7fba037fe640 1 --2- 172.21.15.27:0/4211575246 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fb9f006c1c0 0x7fb9f006e680 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fb9f4006fd0 tx=0x7fb9f4009060 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:44.645 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:44.645+0000 7fb9ebfff640 1 -- 172.21.15.27:0/4211575246 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fb9fc079f70 con 0x7fba040b5a70 2024-09-12T03:19:44.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:44.833+0000 7fba09dc4640 1 -- 172.21.15.27:0/4211575246 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fba040ce450 con 0x7fba040b5a70 2024-09-12T03:19:44.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:44.833+0000 7fb9ebfff640 1 -- 172.21.15.27:0/4211575246 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v56) ==== 80+0+0 (secure 0 0 0) 0x7fb9fc07e270 con 0x7fba040b5a70 2024-09-12T03:19:44.833 INFO:teuthology.orchestra.run.smithi027.stderr:listed 0 entries 2024-09-12T03:19:44.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:44.835+0000 7fba09dc4640 1 -- 172.21.15.27:0/4211575246 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fb9f006c1c0 msgr2=0x7fb9f006e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:44.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:44.835+0000 7fba09dc4640 1 --2- 172.21.15.27:0/4211575246 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fb9f006c1c0 0x7fb9f006e680 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fb9f4006fd0 tx=0x7fb9f4009060 comp rx=0 tx=0).stop 2024-09-12T03:19:44.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:44.835+0000 7fba09dc4640 1 -- 172.21.15.27:0/4211575246 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fba040b5a70 msgr2=0x7fba040d06d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:44.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:44.835+0000 7fba09dc4640 1 --2- 172.21.15.27:0/4211575246 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fba040b5a70 0x7fba040d06d0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fb9fc009f40 tx=0x7fb9fc004a40 comp rx=0 tx=0).stop 2024-09-12T03:19:44.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:44.835+0000 7fba09dc4640 1 -- 172.21.15.27:0/4211575246 shutdown_connections 2024-09-12T03:19:44.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:44.835+0000 7fba09dc4640 1 --2- 172.21.15.27:0/4211575246 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fb9f006c1c0 0x7fb9f006e680 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:44.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:44.835+0000 7fba09dc4640 1 --2- 172.21.15.27:0/4211575246 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fba040b5a70 0x7fba040d06d0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:44.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:44.835+0000 7fba09dc4640 1 --2- 172.21.15.27:0/4211575246 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fba040ae8d0 0x7fba041f3e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:44.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:44.835+0000 7fba09dc4640 1 --2- 172.21.15.27:0/4211575246 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fba04072df0 0x7fba04156b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:44.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:44.835+0000 7fba09dc4640 1 -- 172.21.15.27:0/4211575246 >> 172.21.15.27:0/4211575246 conn(0x7fba0406ca30 msgr2=0x7fba04160c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:44.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:44.835+0000 7fba09dc4640 1 -- 172.21.15.27:0/4211575246 shutdown_connections 2024-09-12T03:19:44.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:44.835+0000 7fba09dc4640 1 -- 172.21.15.27:0/4211575246 wait complete. 2024-09-12T03:19:44.846 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-09-12T03:19:44.847 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:19:44.847 DEBUG:teuthology.orchestra.run.smithi045:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-09-12T03:19:44.882 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi045.front.sepia.ceph.com 2024-09-12T03:19:44.883 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:19:44.883 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-12T03:19:44.949 INFO:tasks.cephfs.mount:Terminating background process 2024-09-12T03:19:44.950 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-12T03:19:44.950 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T03:19:44.969 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056004,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497432k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16324 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248716k,nr_inodes=812179,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497432k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-12T03:19:44.969 INFO:tasks.cephfs.mount:Terminating background process 2024-09-12T03:19:44.969 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-12T03:19:44.969 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T03:19:45.019 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056795,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16602 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249348k,nr_inodes=812337,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-12T03:19:45.020 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-12T03:19:45.130 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.130+0000 7fa60e7b9640 1 Processor -- start 2024-09-12T03:19:45.130 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.130+0000 7fa60e7b9640 1 -- start start 2024-09-12T03:19:45.131 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.131+0000 7fa60e7b9640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fa6080d5200 0x7fa6080d1010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:45.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.131+0000 7fa60e7b9640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa6080d1550 0x7fa6080b8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:45.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.131+0000 7fa60e7b9640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa6080d3760 con 0x7fa60815eb90 2024-09-12T03:19:45.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.131+0000 7fa60e7b9640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7fa6080b7d20 con 0x7fa6080d5200 2024-09-12T03:19:45.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.131+0000 7fa60e7b9640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fa6080d42e0 con 0x7fa6080d1550 2024-09-12T03:19:45.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.131+0000 7fa6077fe640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fa6080d5200 0x7fa6080d1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:45.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.131+0000 7fa607fff640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fa60815eb90 0x7fa6080d2df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:46042/0 (socket says 172.21.15.27:46042) 2024-09-12T03:19:45.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.131+0000 7fa607fff640 1 -- 172.21.15.27:0/435652864 learned_addr learned my addr 172.21.15.27:0/435652864 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:19:45.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.131+0000 7fa6077fe640 1 -- 172.21.15.27:0/435652864 >> v2:172.21.15.45:3300/0 conn(0x7fa6080d1550 msgr2=0x7fa6080b8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:45.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.131+0000 7fa60cd56640 1 --2- 172.21.15.27:0/435652864 >> v2:172.21.15.45:3300/0 conn(0x7fa6080d1550 0x7fa6080b8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:45.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.131+0000 7fa606ffd640 1 -- 172.21.15.27:0/435652864 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2236615160 0 0) 0x7fa6080d3760 con 0x7fa60815eb90 2024-09-12T03:19:45.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.131+0000 7fa6077fe640 1 --2- 172.21.15.27:0/435652864 >> v2:172.21.15.45:3300/0 conn(0x7fa6080d1550 0x7fa6080b8190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:45.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.131+0000 7fa6077fe640 1 -- 172.21.15.27:0/435652864 >> v1:172.21.15.27:6789/0 conn(0x7fa60815eb90 legacy=0x7fa6080d2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:45.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.131+0000 7fa6077fe640 1 -- 172.21.15.27:0/435652864 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6080d3a40 con 0x7fa6080d5200 2024-09-12T03:19:45.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.131+0000 7fa60cd56640 1 --2- 172.21.15.27:0/435652864 >> v2:172.21.15.45:3300/0 conn(0x7fa6080d1550 0x7fa6080b8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:19:45.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.132+0000 7fa6077fe640 1 --2- 172.21.15.27:0/435652864 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa6080d5200 0x7fa6080d1010 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fa6080c8270 tx=0x7fa5fc02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d64e7b859be87167 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:45.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.132+0000 7fa606ffd640 1 -- 172.21.15.27:0/435652864 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5fc037070 con 0x7fa6080d5200 2024-09-12T03:19:45.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.132+0000 7fa606ffd640 1 -- 172.21.15.27:0/435652864 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5fc032440 con 0x7fa6080d5200 2024-09-12T03:19:45.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.132+0000 7fa606ffd640 1 -- 172.21.15.27:0/435652864 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5fc032810 con 0x7fa6080d5200 2024-09-12T03:19:45.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.133+0000 7fa60e7b9640 1 -- 172.21.15.27:0/435652864 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa6080d5200 msgr2=0x7fa6080d1010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:45.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.133+0000 7fa60e7b9640 1 --2- 172.21.15.27:0/435652864 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa6080d5200 0x7fa6080d1010 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fa6080c8270 tx=0x7fa5fc02a1a0 comp rx=0 tx=0).stop 2024-09-12T03:19:45.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.133+0000 7fa60e7b9640 1 -- 172.21.15.27:0/435652864 shutdown_connections 2024-09-12T03:19:45.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.133+0000 7fa60e7b9640 1 --2- 172.21.15.27:0/435652864 >> v2:172.21.15.45:3300/0 conn(0x7fa6080d1550 0x7fa6080b8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:45.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.133+0000 7fa60e7b9640 1 --2- 172.21.15.27:0/435652864 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa6080d5200 0x7fa6080d1010 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:45.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.133+0000 7fa60e7b9640 1 -- 172.21.15.27:0/435652864 >> 172.21.15.27:0/435652864 conn(0x7fa60806dce0 msgr2=0x7fa60806e0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:45.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.133+0000 7fa60e7b9640 1 -- 172.21.15.27:0/435652864 shutdown_connections 2024-09-12T03:19:45.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.133+0000 7fa60e7b9640 1 -- 172.21.15.27:0/435652864 wait complete. 2024-09-12T03:19:45.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.134+0000 7fa60e7b9640 1 Processor -- start 2024-09-12T03:19:45.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.134+0000 7fa60e7b9640 1 -- start start 2024-09-12T03:19:45.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.135+0000 7fa60e7b9640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa6080d1550 0x7fa608156070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:45.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.135+0000 7fa60e7b9640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa6080d5200 0x7fa6080cba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:45.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.135+0000 7fa60e7b9640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa60815eb90 0x7fa6080ce3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:45.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.135+0000 7fa60e7b9640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fa6080d4000 con 0x7fa6080d5200 2024-09-12T03:19:45.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.135+0000 7fa60e7b9640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa6080d3500 con 0x7fa6080d1550 2024-09-12T03:19:45.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.135+0000 7fa60e7b9640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fa6080d1960 con 0x7fa60815eb90 2024-09-12T03:19:45.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.135+0000 7fa6077fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa6080d5200 0x7fa6080cba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:45.138 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.135+0000 7fa6077fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa6080d5200 0x7fa6080cba70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58250/0 (socket says 172.21.15.27:58250) 2024-09-12T03:19:45.138 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.135+0000 7fa6077fe640 1 -- 172.21.15.27:0/1814709169 learned_addr learned my addr 172.21.15.27:0/1814709169 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:19:45.138 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.135+0000 7fa607fff640 1 --2- 172.21.15.27:0/1814709169 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa6080d1550 0x7fa608156070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:45.138 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.135+0000 7fa60cd56640 1 --2- 172.21.15.27:0/1814709169 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa60815eb90 0x7fa6080ce3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:45.138 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.135+0000 7fa6077fe640 1 -- 172.21.15.27:0/1814709169 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa60815eb90 msgr2=0x7fa6080ce3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:45.138 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.135+0000 7fa6077fe640 1 --2- 172.21.15.27:0/1814709169 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa60815eb90 0x7fa6080ce3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:45.138 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.135+0000 7fa6077fe640 1 -- 172.21.15.27:0/1814709169 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa6080d1550 msgr2=0x7fa608156070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:45.138 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.135+0000 7fa6077fe640 1 --2- 172.21.15.27:0/1814709169 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa6080d1550 0x7fa608156070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:45.138 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.135+0000 7fa6077fe640 1 -- 172.21.15.27:0/1814709169 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6081f85a0 con 0x7fa6080d5200 2024-09-12T03:19:45.138 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.135+0000 7fa607fff640 1 --2- 172.21.15.27:0/1814709169 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa6080d1550 0x7fa608156070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:19:45.138 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.136+0000 7fa6077fe640 1 --2- 172.21.15.27:0/1814709169 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa6080d5200 0x7fa6080cba70 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fa5fc009890 tx=0x7fa5fc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:45.138 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.136+0000 7fa604ff9640 1 -- 172.21.15.27:0/1814709169 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5fc037040 con 0x7fa6080d5200 2024-09-12T03:19:45.138 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.136+0000 7fa604ff9640 1 -- 172.21.15.27:0/1814709169 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5fc042070 con 0x7fa6080d5200 2024-09-12T03:19:45.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.136+0000 7fa604ff9640 1 -- 172.21.15.27:0/1814709169 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5fc030a50 con 0x7fa6080d5200 2024-09-12T03:19:45.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.136+0000 7fa60e7b9640 1 -- 172.21.15.27:0/1814709169 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa608159e10 con 0x7fa6080d5200 2024-09-12T03:19:45.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.136+0000 7fa60e7b9640 1 -- 172.21.15.27:0/1814709169 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa60815a3d0 con 0x7fa6080d5200 2024-09-12T03:19:45.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.137+0000 7fa60e7b9640 1 -- 172.21.15.27:0/1814709169 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa60815f090 con 0x7fa6080d5200 2024-09-12T03:19:45.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.137+0000 7fa604ff9640 1 -- 172.21.15.27:0/1814709169 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 11) ==== 90804+0+0 (secure 0 0 0) 0x7fa5fc030bf0 con 0x7fa6080d5200 2024-09-12T03:19:45.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.138+0000 7fa604ff9640 1 --2- 172.21.15.27:0/1814709169 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fa5f406c190 0x7fa5f406e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:45.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.138+0000 7fa604ff9640 1 -- 172.21.15.27:0/1814709169 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5709+0+0 (secure 0 0 0) 0x7fa5fc0abcb0 con 0x7fa6080d5200 2024-09-12T03:19:45.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.138+0000 7fa607fff640 1 --2- 172.21.15.27:0/1814709169 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fa5f406c190 0x7fa5f406e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:45.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.139+0000 7fa607fff640 1 --2- 172.21.15.27:0/1814709169 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fa5f406c190 0x7fa5f406e650 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fa5f8000f60 tx=0x7fa5f8007d50 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:45.142 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.142+0000 7fa604ff9640 1 -- 172.21.15.27:0/1814709169 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fa5fc079c00 con 0x7fa6080d5200 2024-09-12T03:19:45.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.354+0000 7fa60e7b9640 1 -- 172.21.15.27:0/1814709169 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa60815ab30 con 0x7fa6080d5200 2024-09-12T03:19:45.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.355+0000 7fa604ff9640 1 -- 172.21.15.27:0/1814709169 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+5220 (secure 0 0 0) 0x7fa5fc07df00 con 0x7fa6080d5200 2024-09-12T03:19:45.355 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:19:45.355 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":31,"btime":"2024-09-12T03:19:38:158456+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5102,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/565788762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":565788762},{"type":"v1","addr":"172.21.15.27:6835","nonce":565788762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26},{"gid":5413,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6836/2689309305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":2689309305},{"type":"v1","addr":"172.21.15.45:6836","nonce":2689309305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":5640,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6834/3694936463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":3694936463},{"type":"v1","addr":"172.21.15.27:6834","nonce":3694936463}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":30,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-12T03:19:25.201174+0000","modified":"2024-09-12T03:19:27.190661+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5649},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5649":{"gid":5649,"name":"b","rank":0,"incarnation":28,"state":"up:active","state_seq":3,"addr":"172.21.15.45:6837/3720118075","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6835","nonce":3720118075},{"type":"v1","addr":"172.21.15.45:6837","nonce":3720118075}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5649,"qdb_cluster":[5649]},"id":4}]} 2024-09-12T03:19:45.356 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 31 2024-09-12T03:19:45.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.357+0000 7fa60e7b9640 1 -- 172.21.15.27:0/1814709169 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fa5f406c190 msgr2=0x7fa5f406e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:45.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.357+0000 7fa60e7b9640 1 --2- 172.21.15.27:0/1814709169 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fa5f406c190 0x7fa5f406e650 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fa5f8000f60 tx=0x7fa5f8007d50 comp rx=0 tx=0).stop 2024-09-12T03:19:45.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.357+0000 7fa60e7b9640 1 -- 172.21.15.27:0/1814709169 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa6080d5200 msgr2=0x7fa6080cba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:45.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.357+0000 7fa60e7b9640 1 --2- 172.21.15.27:0/1814709169 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa6080d5200 0x7fa6080cba70 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fa5fc009890 tx=0x7fa5fc0046c0 comp rx=0 tx=0).stop 2024-09-12T03:19:45.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.357+0000 7fa60e7b9640 1 -- 172.21.15.27:0/1814709169 shutdown_connections 2024-09-12T03:19:45.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.357+0000 7fa60e7b9640 1 --2- 172.21.15.27:0/1814709169 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fa5f406c190 0x7fa5f406e650 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:45.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.357+0000 7fa60e7b9640 1 --2- 172.21.15.27:0/1814709169 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa60815eb90 0x7fa6080ce3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:45.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.357+0000 7fa60e7b9640 1 --2- 172.21.15.27:0/1814709169 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa6080d5200 0x7fa6080cba70 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:45.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.357+0000 7fa60e7b9640 1 --2- 172.21.15.27:0/1814709169 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa6080d1550 0x7fa608156070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:45.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.357+0000 7fa60e7b9640 1 -- 172.21.15.27:0/1814709169 >> 172.21.15.27:0/1814709169 conn(0x7fa60806dce0 msgr2=0x7fa608156830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:45.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.357+0000 7fa60e7b9640 1 -- 172.21.15.27:0/1814709169 shutdown_connections 2024-09-12T03:19:45.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.357+0000 7fa60e7b9640 1 -- 172.21.15.27:0/1814709169 wait complete. 2024-09-12T03:19:45.368 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-12T03:19:45.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.470+0000 7f6815dfd640 1 Processor -- start 2024-09-12T03:19:45.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.470+0000 7f6815dfd640 1 -- start start 2024-09-12T03:19:45.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.471+0000 7f6815dfd640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f681015eb60 0x7f68100d1280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:45.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.471+0000 7f6815dfd640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f68100b6690 con 0x7f68100d17c0 2024-09-12T03:19:45.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.471+0000 7f6815dfd640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f68100b6970 con 0x7f681015e780 2024-09-12T03:19:45.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.471+0000 7f6815dfd640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f68100b7950 con 0x7f681015eb60 2024-09-12T03:19:45.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.471+0000 7f680effd640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f681015eb60 0x7f68100d1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:45.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.471+0000 7f680ffff640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f68100d17c0 0x7f68100b3390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:46050/0 (socket says 172.21.15.27:46050) 2024-09-12T03:19:45.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.471+0000 7f680ffff640 1 -- 172.21.15.27:0/174227601 learned_addr learned my addr 172.21.15.27:0/174227601 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:19:45.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.472+0000 7f680effd640 1 -- 172.21.15.27:0/174227601 >> v1:172.21.15.45:6789/0 conn(0x7f681015e780 legacy=0x7f68100d3e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:45.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.472+0000 7f680effd640 1 -- 172.21.15.27:0/174227601 >> v1:172.21.15.27:6789/0 conn(0x7f68100d17c0 legacy=0x7f68100b3390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:45.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.472+0000 7f680e7fc640 1 -- 172.21.15.27:0/174227601 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1302679242 0 0) 0x7f68100b6690 con 0x7f68100d17c0 2024-09-12T03:19:45.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.472+0000 7f680effd640 1 -- 172.21.15.27:0/174227601 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68100b6c50 con 0x7f681015eb60 2024-09-12T03:19:45.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.472+0000 7f680effd640 1 --2- 172.21.15.27:0/174227601 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f681015eb60 0x7f68100d1280 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f68040097a0 tx=0x7f680402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f40a6dfbe14f8b4e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:45.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.473+0000 7f680e7fc640 1 -- 172.21.15.27:0/174227601 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6804037070 con 0x7f681015eb60 2024-09-12T03:19:45.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.473+0000 7f680e7fc640 1 -- 172.21.15.27:0/174227601 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6804032440 con 0x7f681015eb60 2024-09-12T03:19:45.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.473+0000 7f680e7fc640 1 -- 172.21.15.27:0/174227601 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6804032810 con 0x7f681015eb60 2024-09-12T03:19:45.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.473+0000 7f6815dfd640 1 -- 172.21.15.27:0/174227601 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f681015eb60 msgr2=0x7f68100d1280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:45.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.473+0000 7f6815dfd640 1 --2- 172.21.15.27:0/174227601 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f681015eb60 0x7f68100d1280 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f68040097a0 tx=0x7f680402a230 comp rx=0 tx=0).stop 2024-09-12T03:19:45.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.473+0000 7f6815dfd640 1 -- 172.21.15.27:0/174227601 shutdown_connections 2024-09-12T03:19:45.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.473+0000 7f6815dfd640 1 --2- 172.21.15.27:0/174227601 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f681015eb60 0x7f68100d1280 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:45.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.473+0000 7f6815dfd640 1 -- 172.21.15.27:0/174227601 >> 172.21.15.27:0/174227601 conn(0x7f681006da60 msgr2=0x7f681006de70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:45.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.473+0000 7f6815dfd640 1 -- 172.21.15.27:0/174227601 shutdown_connections 2024-09-12T03:19:45.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.473+0000 7f6815dfd640 1 -- 172.21.15.27:0/174227601 wait complete. 2024-09-12T03:19:45.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.474+0000 7f6815dfd640 1 Processor -- start 2024-09-12T03:19:45.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.475+0000 7f6815dfd640 1 -- start start 2024-09-12T03:19:45.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.475+0000 7f6815dfd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f68100d17c0 0x7f681015d770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:45.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.475+0000 7f6815dfd640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f681015e780 0x7f6810161450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:45.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.475+0000 7f6815dfd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f681015eb60 0x7f68100b4a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:45.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.475+0000 7f6815dfd640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f68100d1bf0 con 0x7f68100d17c0 2024-09-12T03:19:45.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.475+0000 7f6815dfd640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f68100b7210 con 0x7f681015eb60 2024-09-12T03:19:45.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.475+0000 7f6815dfd640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f68100b6340 con 0x7f681015e780 2024-09-12T03:19:45.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.476+0000 7f680f7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f68100d17c0 0x7f681015d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:45.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.476+0000 7f680f7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f68100d17c0 0x7f681015d770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58264/0 (socket says 172.21.15.27:58264) 2024-09-12T03:19:45.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.476+0000 7f680f7fe640 1 -- 172.21.15.27:0/166777986 learned_addr learned my addr 172.21.15.27:0/166777986 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:19:45.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.476+0000 7f680effd640 1 --2- 172.21.15.27:0/166777986 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f681015e780 0x7f6810161450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:45.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.476+0000 7f680ffff640 1 --2- 172.21.15.27:0/166777986 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f681015eb60 0x7f68100b4a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:45.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.476+0000 7f680f7fe640 1 -- 172.21.15.27:0/166777986 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f681015e780 msgr2=0x7f6810161450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:45.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.476+0000 7f680f7fe640 1 --2- 172.21.15.27:0/166777986 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f681015e780 0x7f6810161450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:45.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.476+0000 7f680f7fe640 1 -- 172.21.15.27:0/166777986 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f681015eb60 msgr2=0x7f68100b4a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:45.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.476+0000 7f680f7fe640 1 --2- 172.21.15.27:0/166777986 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f681015eb60 0x7f68100b4a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:45.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.476+0000 7f680f7fe640 1 -- 172.21.15.27:0/166777986 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f681015dcb0 con 0x7f68100d17c0 2024-09-12T03:19:45.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.476+0000 7f680ffff640 1 --2- 172.21.15.27:0/166777986 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f681015eb60 0x7f68100b4a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:19:45.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.476+0000 7f680f7fe640 1 --2- 172.21.15.27:0/166777986 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f68100d17c0 0x7f681015d770 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f680000c450 tx=0x7f680000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:45.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.476+0000 7f67f7fff640 1 -- 172.21.15.27:0/166777986 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68000052d0 con 0x7f68100d17c0 2024-09-12T03:19:45.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.476+0000 7f67f7fff640 1 -- 172.21.15.27:0/166777986 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6800005470 con 0x7f68100d17c0 2024-09-12T03:19:45.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.476+0000 7f67f7fff640 1 -- 172.21.15.27:0/166777986 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6800012870 con 0x7f68100d17c0 2024-09-12T03:19:45.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.476+0000 7f6815dfd640 1 -- 172.21.15.27:0/166777986 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6810161990 con 0x7f68100d17c0 2024-09-12T03:19:45.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.476+0000 7f6815dfd640 1 -- 172.21.15.27:0/166777986 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68100abb90 con 0x7f68100d17c0 2024-09-12T03:19:45.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.477+0000 7f6815dfd640 1 -- 172.21.15.27:0/166777986 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68100d3ca0 con 0x7f68100d17c0 2024-09-12T03:19:45.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.478+0000 7f67f7fff640 1 -- 172.21.15.27:0/166777986 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 11) ==== 90804+0+0 (secure 0 0 0) 0x7f6800012a10 con 0x7f68100d17c0 2024-09-12T03:19:45.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.478+0000 7f67f7fff640 1 --2- 172.21.15.27:0/166777986 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f67fc06c190 0x7f67fc06e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:45.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.478+0000 7f67f7fff640 1 -- 172.21.15.27:0/166777986 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5709+0+0 (secure 0 0 0) 0x7f680008afc0 con 0x7f68100d17c0 2024-09-12T03:19:45.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.478+0000 7f680effd640 1 --2- 172.21.15.27:0/166777986 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f67fc06c190 0x7f67fc06e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:45.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.479+0000 7f680effd640 1 --2- 172.21.15.27:0/166777986 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f67fc06c190 0x7f67fc06e650 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f6804034000 tx=0x7f680402e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:45.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.482+0000 7f67f7fff640 1 -- 172.21.15.27:0/166777986 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f68000594f0 con 0x7f68100d17c0 2024-09-12T03:19:45.695 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.695+0000 7f6815dfd640 1 -- 172.21.15.27:0/166777986 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f68100d4610 con 0x7f68100d17c0 2024-09-12T03:19:45.695 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.695+0000 7f67f7fff640 1 -- 172.21.15.27:0/166777986 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+5220 (secure 0 0 0) 0x7f6800058df0 con 0x7f68100d17c0 2024-09-12T03:19:45.695 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:19:45.696 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":31,"btime":"2024-09-12T03:19:38:158456+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5102,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/565788762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":565788762},{"type":"v1","addr":"172.21.15.27:6835","nonce":565788762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26},{"gid":5413,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6836/2689309305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":2689309305},{"type":"v1","addr":"172.21.15.45:6836","nonce":2689309305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":5640,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6834/3694936463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":3694936463},{"type":"v1","addr":"172.21.15.27:6834","nonce":3694936463}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":30,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-12T03:19:25.201174+0000","modified":"2024-09-12T03:19:27.190661+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5649},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5649":{"gid":5649,"name":"b","rank":0,"incarnation":28,"state":"up:active","state_seq":3,"addr":"172.21.15.45:6837/3720118075","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6835","nonce":3720118075},{"type":"v1","addr":"172.21.15.45:6837","nonce":3720118075}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5649,"qdb_cluster":[5649]},"id":4}]} 2024-09-12T03:19:45.696 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 31 2024-09-12T03:19:45.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.697+0000 7f6815dfd640 1 -- 172.21.15.27:0/166777986 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f67fc06c190 msgr2=0x7f67fc06e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:45.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.697+0000 7f6815dfd640 1 --2- 172.21.15.27:0/166777986 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f67fc06c190 0x7f67fc06e650 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f6804034000 tx=0x7f680402e000 comp rx=0 tx=0).stop 2024-09-12T03:19:45.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.697+0000 7f6815dfd640 1 -- 172.21.15.27:0/166777986 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f68100d17c0 msgr2=0x7f681015d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:45.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.697+0000 7f6815dfd640 1 --2- 172.21.15.27:0/166777986 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f68100d17c0 0x7f681015d770 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f680000c450 tx=0x7f680000c910 comp rx=0 tx=0).stop 2024-09-12T03:19:45.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.697+0000 7f6815dfd640 1 -- 172.21.15.27:0/166777986 shutdown_connections 2024-09-12T03:19:45.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.697+0000 7f6815dfd640 1 --2- 172.21.15.27:0/166777986 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f67fc06c190 0x7f67fc06e650 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:45.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.697+0000 7f6815dfd640 1 --2- 172.21.15.27:0/166777986 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f681015eb60 0x7f68100b4a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:45.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.697+0000 7f6815dfd640 1 --2- 172.21.15.27:0/166777986 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f681015e780 0x7f6810161450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:45.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.697+0000 7f6815dfd640 1 --2- 172.21.15.27:0/166777986 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f68100d17c0 0x7f681015d770 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:45.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.697+0000 7f6815dfd640 1 -- 172.21.15.27:0/166777986 >> 172.21.15.27:0/166777986 conn(0x7f681006da60 msgr2=0x7f681006de40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:45.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.697+0000 7f6815dfd640 1 -- 172.21.15.27:0/166777986 shutdown_connections 2024-09-12T03:19:45.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.697+0000 7f6815dfd640 1 -- 172.21.15.27:0/166777986 wait complete. 2024-09-12T03:19:45.709 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-12T03:19:45.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.811+0000 7fd12f9c4640 1 Processor -- start 2024-09-12T03:19:45.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.811+0000 7fd12f9c4640 1 -- start start 2024-09-12T03:19:45.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.811+0000 7fd12f9c4640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fd1280d5200 0x7fd1280d1010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:45.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.812+0000 7fd12f9c4640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fd1280d1550 0x7fd1280b8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:45.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.812+0000 7fd12f9c4640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd1280d19c0 con 0x7fd12815eb90 2024-09-12T03:19:45.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.812+0000 7fd12f9c4640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7fd1280b7d20 con 0x7fd1280d1550 2024-09-12T03:19:45.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.812+0000 7fd12f9c4640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fd1280d42e0 con 0x7fd1280d5200 2024-09-12T03:19:45.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.812+0000 7fd12df61640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fd1280d1550 0x7fd1280b8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:45.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.812+0000 7fd12df61640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fd1280d1550 0x7fd1280b8190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58278/0 (socket says 172.21.15.27:58278) 2024-09-12T03:19:45.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.812+0000 7fd12df61640 1 -- 172.21.15.27:0/1303896549 learned_addr learned my addr 172.21.15.27:0/1303896549 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:19:45.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.812+0000 7fd12cf5f640 1 --2- 172.21.15.27:0/1303896549 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fd1280d5200 0x7fd1280d1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:45.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.812+0000 7fd12df61640 1 -- 172.21.15.27:0/1303896549 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fd1280d5200 msgr2=0x7fd1280d1010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:45.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.812+0000 7fd12df61640 1 --2- 172.21.15.27:0/1303896549 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fd1280d5200 0x7fd1280d1010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:45.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.812+0000 7fd12df61640 1 -- 172.21.15.27:0/1303896549 >> v1:172.21.15.45:6789/0 conn(0x7fd12815eb90 legacy=0x7fd1280d2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:45.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.812+0000 7fd12df61640 1 -- 172.21.15.27:0/1303896549 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1280d3760 con 0x7fd1280d1550 2024-09-12T03:19:45.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.813+0000 7fd12df61640 1 --2- 172.21.15.27:0/1303896549 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd1280d1550 0x7fd1280b8190 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fd120009e50 tx=0x7fd12002a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d30d7a30e76fce24 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:45.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.813+0000 7fd11ffff640 1 -- 172.21.15.27:0/1303896549 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd120036070 con 0x7fd1280d1550 2024-09-12T03:19:45.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.813+0000 7fd11ffff640 1 -- 172.21.15.27:0/1303896549 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd120031600 con 0x7fd1280d1550 2024-09-12T03:19:45.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.813+0000 7fd11ffff640 1 -- 172.21.15.27:0/1303896549 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1200319d0 con 0x7fd1280d1550 2024-09-12T03:19:45.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.813+0000 7fd12f9c4640 1 -- 172.21.15.27:0/1303896549 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd1280d1550 msgr2=0x7fd1280b8190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:45.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.813+0000 7fd12f9c4640 1 --2- 172.21.15.27:0/1303896549 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd1280d1550 0x7fd1280b8190 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fd120009e50 tx=0x7fd12002a710 comp rx=0 tx=0).stop 2024-09-12T03:19:45.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.814+0000 7fd12f9c4640 1 -- 172.21.15.27:0/1303896549 shutdown_connections 2024-09-12T03:19:45.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.814+0000 7fd12f9c4640 1 --2- 172.21.15.27:0/1303896549 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd1280d1550 0x7fd1280b8190 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:45.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.814+0000 7fd12f9c4640 1 --2- 172.21.15.27:0/1303896549 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fd1280d5200 0x7fd1280d1010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:45.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.814+0000 7fd12f9c4640 1 -- 172.21.15.27:0/1303896549 >> 172.21.15.27:0/1303896549 conn(0x7fd12806dce0 msgr2=0x7fd12806e0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:45.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.814+0000 7fd12f9c4640 1 -- 172.21.15.27:0/1303896549 shutdown_connections 2024-09-12T03:19:45.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.814+0000 7fd12f9c4640 1 -- 172.21.15.27:0/1303896549 wait complete. 2024-09-12T03:19:45.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.815+0000 7fd12f9c4640 1 Processor -- start 2024-09-12T03:19:45.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.815+0000 7fd12f9c4640 1 -- start start 2024-09-12T03:19:45.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.816+0000 7fd12f9c4640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fd1280d1550 0x7fd1280ad980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:45.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.816+0000 7fd12f9c4640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd1280d5200 0x7fd1280c9920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:45.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.816+0000 7fd12f9c4640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd12815eb90 0x7fd1280ae6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:45.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.816+0000 7fd12f9c4640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fd1280d3d20 con 0x7fd12815eb90 2024-09-12T03:19:45.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.816+0000 7fd12f9c4640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fd1280d3500 con 0x7fd1280d5200 2024-09-12T03:19:45.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.816+0000 7fd12f9c4640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fd1280d4000 con 0x7fd1280d1550 2024-09-12T03:19:45.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.816+0000 7fd12df61640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd12815eb90 0x7fd1280ae6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:45.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.816+0000 7fd12df61640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd12815eb90 0x7fd1280ae6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58286/0 (socket says 172.21.15.27:58286) 2024-09-12T03:19:45.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.816+0000 7fd12df61640 1 -- 172.21.15.27:0/3135505888 learned_addr learned my addr 172.21.15.27:0/3135505888 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:19:45.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.816+0000 7fd12d760640 1 --2- 172.21.15.27:0/3135505888 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fd1280d1550 0x7fd1280ad980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:45.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.816+0000 7fd12cf5f640 1 --2- 172.21.15.27:0/3135505888 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd1280d5200 0x7fd1280c9920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:45.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.816+0000 7fd12df61640 1 -- 172.21.15.27:0/3135505888 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fd1280d1550 msgr2=0x7fd1280ad980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:45.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.816+0000 7fd12df61640 1 --2- 172.21.15.27:0/3135505888 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fd1280d1550 0x7fd1280ad980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:45.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.816+0000 7fd12df61640 1 -- 172.21.15.27:0/3135505888 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd1280d5200 msgr2=0x7fd1280c9920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:45.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.816+0000 7fd12df61640 1 --2- 172.21.15.27:0/3135505888 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd1280d5200 0x7fd1280c9920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:45.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.816+0000 7fd12df61640 1 -- 172.21.15.27:0/3135505888 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1280ca3e0 con 0x7fd12815eb90 2024-09-12T03:19:45.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.816+0000 7fd12df61640 1 --2- 172.21.15.27:0/3135505888 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd12815eb90 0x7fd1280ae6d0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fd120034040 tx=0x7fd120004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:45.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.816+0000 7fd11dffb640 1 -- 172.21.15.27:0/3135505888 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd120036050 con 0x7fd12815eb90 2024-09-12T03:19:45.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.816+0000 7fd11dffb640 1 -- 172.21.15.27:0/3135505888 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd120042070 con 0x7fd12815eb90 2024-09-12T03:19:45.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.816+0000 7fd11dffb640 1 -- 172.21.15.27:0/3135505888 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd120009510 con 0x7fd12815eb90 2024-09-12T03:19:45.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.816+0000 7fd12f9c4640 1 -- 172.21.15.27:0/3135505888 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1280ca6c0 con 0x7fd12815eb90 2024-09-12T03:19:45.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.817+0000 7fd12f9c4640 1 -- 172.21.15.27:0/3135505888 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1280cac80 con 0x7fd12815eb90 2024-09-12T03:19:45.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.818+0000 7fd12f9c4640 1 -- 172.21.15.27:0/3135505888 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1280cb380 con 0x7fd12815eb90 2024-09-12T03:19:45.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.818+0000 7fd11dffb640 1 -- 172.21.15.27:0/3135505888 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 11) ==== 90804+0+0 (secure 0 0 0) 0x7fd120007c70 con 0x7fd12815eb90 2024-09-12T03:19:45.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.819+0000 7fd11dffb640 1 --2- 172.21.15.27:0/3135505888 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fd10c06c190 0x7fd10c06e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:45.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.819+0000 7fd11dffb640 1 -- 172.21.15.27:0/3135505888 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5709+0+0 (secure 0 0 0) 0x7fd1200ac210 con 0x7fd12815eb90 2024-09-12T03:19:45.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.819+0000 7fd12d760640 1 --2- 172.21.15.27:0/3135505888 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fd10c06c190 0x7fd10c06e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:45.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.820+0000 7fd12d760640 1 --2- 172.21.15.27:0/3135505888 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fd10c06c190 0x7fd10c06e650 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fd12815d750 tx=0x7fd118009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:45.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:45.823+0000 7fd11dffb640 1 -- 172.21.15.27:0/3135505888 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fd12007a170 con 0x7fd12815eb90 2024-09-12T03:19:46.000 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.000+0000 7fd12f9c4640 1 -- 172.21.15.27:0/3135505888 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd1280b90a0 con 0x7fd12815eb90 2024-09-12T03:19:46.001 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.001+0000 7fd11dffb640 1 -- 172.21.15.27:0/3135505888 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v56) ==== 74+0+16677 (secure 0 0 0) 0x7fd12007e470 con 0x7fd12815eb90 2024-09-12T03:19:46.001 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:19:46.001 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":56,"fsid":"d2df504e-2647-4385-a5db-9fdf69f487a0","created":"2024-09-12T03:16:07.778937+0000","modified":"2024-09-12T03:19:26.561081+0000","last_up_change":"2024-09-12T03:16:19.314778+0000","last_in_change":"2024-09-12T03:16:09.893086+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T03:16:20.097646+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-09-12T03:19:23.556103+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-09-12T03:19:24.184337+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1851116c-1fb3-4dc9-9772-f90e2ef4c355","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6817","nonce":3002321488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6819","nonce":3002321488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6823","nonce":3002321488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6821","nonce":3002321488}]},"public_addr":"172.21.15.27:6817/3002321488","cluster_addr":"172.21.15.27:6819/3002321488","heartbeat_back_addr":"172.21.15.27:6823/3002321488","heartbeat_front_addr":"172.21.15.27:6821/3002321488","state":["exists","up"]},{"osd":1,"uuid":"f38ae17d-73eb-4204-b99d-975fa806fb87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6809","nonce":1613102886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6811","nonce":1613102886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6815","nonce":1613102886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6813","nonce":1613102886}]},"public_addr":"172.21.15.27:6809/1613102886","cluster_addr":"172.21.15.27:6811/1613102886","heartbeat_back_addr":"172.21.15.27:6815/1613102886","heartbeat_front_addr":"172.21.15.27:6813/1613102886","state":["exists","up"]},{"osd":2,"uuid":"fb12dcc8-bdd0-42b8-b46d-8b10106224bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6825","nonce":3534723355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6827","nonce":3534723355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6831","nonce":3534723355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6829","nonce":3534723355}]},"public_addr":"172.21.15.27:6825/3534723355","cluster_addr":"172.21.15.27:6827/3534723355","heartbeat_back_addr":"172.21.15.27:6831/3534723355","heartbeat_front_addr":"172.21.15.27:6829/3534723355","state":["exists","up"]},{"osd":3,"uuid":"d74d3f49-9c24-4b9f-98e2-3ecd7b5ef3b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6801","nonce":80661598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6803","nonce":80661598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6807","nonce":80661598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6805","nonce":80661598}]},"public_addr":"172.21.15.27:6801/80661598","cluster_addr":"172.21.15.27:6803/80661598","heartbeat_back_addr":"172.21.15.27:6807/80661598","heartbeat_front_addr":"172.21.15.27:6805/80661598","state":["exists","up"]},{"osd":4,"uuid":"2bf123f2-1b0d-41c4-aa3b-805bab267e59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6817","nonce":3943189827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6819","nonce":3943189827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6823","nonce":3943189827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6821","nonce":3943189827}]},"public_addr":"172.21.15.45:6817/3943189827","cluster_addr":"172.21.15.45:6819/3943189827","heartbeat_back_addr":"172.21.15.45:6823/3943189827","heartbeat_front_addr":"172.21.15.45:6821/3943189827","state":["exists","up"]},{"osd":5,"uuid":"c49ba5da-91c6-4a86-a97b-4aba89c253ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6809","nonce":3620760330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6811","nonce":3620760330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6815","nonce":3620760330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6813","nonce":3620760330}]},"public_addr":"172.21.15.45:6809/3620760330","cluster_addr":"172.21.15.45:6811/3620760330","heartbeat_back_addr":"172.21.15.45:6815/3620760330","heartbeat_front_addr":"172.21.15.45:6813/3620760330","state":["exists","up"]},{"osd":6,"uuid":"b0123475-89f7-4a34-9e33-d797b99dd9dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6801","nonce":1989083510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6803","nonce":1989083510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6807","nonce":1989083510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6805","nonce":1989083510}]},"public_addr":"172.21.15.45:6801/1989083510","cluster_addr":"172.21.15.45:6803/1989083510","heartbeat_back_addr":"172.21.15.45:6807/1989083510","heartbeat_front_addr":"172.21.15.45:6805/1989083510","state":["exists","up"]},{"osd":7,"uuid":"4dad773d-b7d8-46fc-8a10-a972aad27b29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6825","nonce":1513300294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6827","nonce":1513300294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6831","nonce":1513300294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6829","nonce":1513300294}]},"public_addr":"172.21.15.45:6825/1513300294","cluster_addr":"172.21.15.45:6827/1513300294","heartbeat_back_addr":"172.21.15.45:6831/1513300294","heartbeat_front_addr":"172.21.15.45:6829/1513300294","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.881855+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:17.133252+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.735339+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.500742+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.477571+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:17.167667+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.470291+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.873743+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T03:19:46.003 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.003+0000 7fd12f9c4640 1 -- 172.21.15.27:0/3135505888 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fd10c06c190 msgr2=0x7fd10c06e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:46.003 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.003+0000 7fd12f9c4640 1 --2- 172.21.15.27:0/3135505888 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fd10c06c190 0x7fd10c06e650 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fd12815d750 tx=0x7fd118009040 comp rx=0 tx=0).stop 2024-09-12T03:19:46.003 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.003+0000 7fd12f9c4640 1 -- 172.21.15.27:0/3135505888 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd12815eb90 msgr2=0x7fd1280ae6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:46.003 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.003+0000 7fd12f9c4640 1 --2- 172.21.15.27:0/3135505888 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd12815eb90 0x7fd1280ae6d0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fd120034040 tx=0x7fd120004a40 comp rx=0 tx=0).stop 2024-09-12T03:19:46.004 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.003+0000 7fd12f9c4640 1 -- 172.21.15.27:0/3135505888 shutdown_connections 2024-09-12T03:19:46.004 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.003+0000 7fd12f9c4640 1 --2- 172.21.15.27:0/3135505888 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fd10c06c190 0x7fd10c06e650 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:46.004 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.003+0000 7fd12f9c4640 1 --2- 172.21.15.27:0/3135505888 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd12815eb90 0x7fd1280ae6d0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:46.004 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.003+0000 7fd12f9c4640 1 --2- 172.21.15.27:0/3135505888 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd1280d5200 0x7fd1280c9920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:46.004 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.003+0000 7fd12f9c4640 1 --2- 172.21.15.27:0/3135505888 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fd1280d1550 0x7fd1280ad980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:46.004 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.003+0000 7fd12f9c4640 1 -- 172.21.15.27:0/3135505888 >> 172.21.15.27:0/3135505888 conn(0x7fd12806dce0 msgr2=0x7fd128160ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:46.004 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.003+0000 7fd12f9c4640 1 -- 172.21.15.27:0/3135505888 shutdown_connections 2024-09-12T03:19:46.004 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.003+0000 7fd12f9c4640 1 -- 172.21.15.27:0/3135505888 wait complete. 2024-09-12T03:19:46.017 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-12T03:19:46.018 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-12T03:19:46.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.120+0000 7fe7fc931640 1 Processor -- start 2024-09-12T03:19:46.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.120+0000 7fe7fc931640 1 -- start start 2024-09-12T03:19:46.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.121+0000 7fe7fc931640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fe7f80b5a70 0x7fe7f815f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:46.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.121+0000 7fe7fc931640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fe7f80ae8d0 0x7fe7f80728b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:46.122 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.121+0000 7fe7fc931640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe7f815fb40 con 0x7fe7f8072df0 2024-09-12T03:19:46.122 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.121+0000 7fe7fc931640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7fe7f8073240 con 0x7fe7f80b5a70 2024-09-12T03:19:46.122 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.121+0000 7fe7fc931640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fe7f80aee90 con 0x7fe7f80ae8d0 2024-09-12T03:19:46.122 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.121+0000 7fe7f659c640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fe7f80b5a70 0x7fe7f815f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:46.122 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.121+0000 7fe7f6d9d640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fe7f8072df0 0x7fe7f81621c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:46056/0 (socket says 172.21.15.27:46056) 2024-09-12T03:19:46.122 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.121+0000 7fe7f659c640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fe7f80b5a70 0x7fe7f815f600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58294/0 (socket says 172.21.15.27:58294) 2024-09-12T03:19:46.122 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.121+0000 7fe7f6d9d640 1 -- 172.21.15.27:0/3414660377 learned_addr learned my addr 172.21.15.27:0/3414660377 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:19:46.122 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.122+0000 7fe7f659c640 1 -- 172.21.15.27:0/3414660377 >> v2:172.21.15.45:3300/0 conn(0x7fe7f80ae8d0 msgr2=0x7fe7f80728b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:46.122 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.122+0000 7fe7f659c640 1 --2- 172.21.15.27:0/3414660377 >> v2:172.21.15.45:3300/0 conn(0x7fe7f80ae8d0 0x7fe7f80728b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:46.122 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.122+0000 7fe7f659c640 1 -- 172.21.15.27:0/3414660377 >> v1:172.21.15.27:6789/0 conn(0x7fe7f8072df0 legacy=0x7fe7f81621c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:46.122 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.122+0000 7fe7f659c640 1 -- 172.21.15.27:0/3414660377 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7f80af170 con 0x7fe7f80b5a70 2024-09-12T03:19:46.122 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.122+0000 7fe7f559a640 1 -- 172.21.15.27:0/3414660377 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3083678100 0 0) 0x7fe7f815fb40 con 0x7fe7f8072df0 2024-09-12T03:19:46.124 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.122+0000 7fe7f659c640 1 --2- 172.21.15.27:0/3414660377 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe7f80b5a70 0x7fe7f815f600 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fe7f8062840 tx=0x7fe7e802a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97d287d09e5d5b0b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:46.124 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.122+0000 7fe7f559a640 1 -- 172.21.15.27:0/3414660377 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7e8007740 con 0x7fe7f80b5a70 2024-09-12T03:19:46.124 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.122+0000 7fe7f559a640 1 -- 172.21.15.27:0/3414660377 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7e80078e0 con 0x7fe7f80b5a70 2024-09-12T03:19:46.124 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.122+0000 7fe7f559a640 1 -- 172.21.15.27:0/3414660377 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7e8007cb0 con 0x7fe7f80b5a70 2024-09-12T03:19:46.124 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.123+0000 7fe7fc931640 1 -- 172.21.15.27:0/3414660377 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe7f80b5a70 msgr2=0x7fe7f815f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:46.124 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.123+0000 7fe7fc931640 1 --2- 172.21.15.27:0/3414660377 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe7f80b5a70 0x7fe7f815f600 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fe7f8062840 tx=0x7fe7e802a5f0 comp rx=0 tx=0).stop 2024-09-12T03:19:46.124 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.123+0000 7fe7fc931640 1 -- 172.21.15.27:0/3414660377 shutdown_connections 2024-09-12T03:19:46.124 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.123+0000 7fe7fc931640 1 --2- 172.21.15.27:0/3414660377 >> v2:172.21.15.45:3300/0 conn(0x7fe7f80ae8d0 0x7fe7f80728b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:46.124 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.123+0000 7fe7fc931640 1 --2- 172.21.15.27:0/3414660377 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe7f80b5a70 0x7fe7f815f600 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:46.124 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.123+0000 7fe7fc931640 1 -- 172.21.15.27:0/3414660377 >> 172.21.15.27:0/3414660377 conn(0x7fe7f806ca30 msgr2=0x7fe7f806ce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:46.124 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.123+0000 7fe7fc931640 1 -- 172.21.15.27:0/3414660377 shutdown_connections 2024-09-12T03:19:46.124 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.123+0000 7fe7fc931640 1 -- 172.21.15.27:0/3414660377 wait complete. 2024-09-12T03:19:46.127 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.124+0000 7fe7fc931640 1 Processor -- start 2024-09-12T03:19:46.127 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.124+0000 7fe7fc931640 1 -- start start 2024-09-12T03:19:46.127 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.125+0000 7fe7fc931640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe7f8072df0 0x7fe7f8146900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:46.127 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.125+0000 7fe7fc931640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe7f80ae8d0 0x7fe7f80b1230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:46.127 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.125+0000 7fe7fc931640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe7f80b5a70 0x7fe7f80b1770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:46.127 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.125+0000 7fe7fc931640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fe7f806f940 con 0x7fe7f8072df0 2024-09-12T03:19:46.127 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.125+0000 7fe7fc931640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe7f80af730 con 0x7fe7f80b5a70 2024-09-12T03:19:46.127 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.125+0000 7fe7fc931640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fe7f806fac0 con 0x7fe7f80ae8d0 2024-09-12T03:19:46.127 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.125+0000 7fe7f659c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe7f8072df0 0x7fe7f8146900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:46.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.125+0000 7fe7f659c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe7f8072df0 0x7fe7f8146900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58306/0 (socket says 172.21.15.27:58306) 2024-09-12T03:19:46.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.125+0000 7fe7f659c640 1 -- 172.21.15.27:0/1704340543 learned_addr learned my addr 172.21.15.27:0/1704340543 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:19:46.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.125+0000 7fe7f6d9d640 1 --2- 172.21.15.27:0/1704340543 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe7f80b5a70 0x7fe7f80b1770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:46.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.125+0000 7fe7f5d9b640 1 --2- 172.21.15.27:0/1704340543 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe7f80ae8d0 0x7fe7f80b1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:46.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.126+0000 7fe7f659c640 1 -- 172.21.15.27:0/1704340543 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe7f80ae8d0 msgr2=0x7fe7f80b1230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:46.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.126+0000 7fe7f659c640 1 --2- 172.21.15.27:0/1704340543 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe7f80ae8d0 0x7fe7f80b1230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:46.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.126+0000 7fe7f659c640 1 -- 172.21.15.27:0/1704340543 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe7f80b5a70 msgr2=0x7fe7f80b1770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:46.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.126+0000 7fe7f659c640 1 --2- 172.21.15.27:0/1704340543 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe7f80b5a70 0x7fe7f80b1770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:46.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.126+0000 7fe7f659c640 1 -- 172.21.15.27:0/1704340543 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7f815d750 con 0x7fe7f8072df0 2024-09-12T03:19:46.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.126+0000 7fe7f6d9d640 1 --2- 172.21.15.27:0/1704340543 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe7f80b5a70 0x7fe7f80b1770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:19:46.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.126+0000 7fe7f5d9b640 1 --2- 172.21.15.27:0/1704340543 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe7f80ae8d0 0x7fe7f80b1230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:19:46.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.126+0000 7fe7f659c640 1 --2- 172.21.15.27:0/1704340543 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe7f8072df0 0x7fe7f8146900 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fe7e802abe0 tx=0x7fe7e8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:46.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.126+0000 7fe7eeffd640 1 -- 172.21.15.27:0/1704340543 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7e8040070 con 0x7fe7f8072df0 2024-09-12T03:19:46.129 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.126+0000 7fe7eeffd640 1 -- 172.21.15.27:0/1704340543 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7e8031c40 con 0x7fe7f8072df0 2024-09-12T03:19:46.129 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.126+0000 7fe7fc931640 1 -- 172.21.15.27:0/1704340543 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7f81591e0 con 0x7fe7f8072df0 2024-09-12T03:19:46.129 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.126+0000 7fe7eeffd640 1 -- 172.21.15.27:0/1704340543 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7e803c420 con 0x7fe7f8072df0 2024-09-12T03:19:46.129 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.126+0000 7fe7fc931640 1 -- 172.21.15.27:0/1704340543 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7f81597a0 con 0x7fe7f8072df0 2024-09-12T03:19:46.129 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.127+0000 7fe7fc931640 1 -- 172.21.15.27:0/1704340543 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7f80728b0 con 0x7fe7f8072df0 2024-09-12T03:19:46.130 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.127+0000 7fe7eeffd640 1 -- 172.21.15.27:0/1704340543 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 11) ==== 90804+0+0 (secure 0 0 0) 0x7fe7e8039040 con 0x7fe7f8072df0 2024-09-12T03:19:46.130 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.128+0000 7fe7eeffd640 1 --2- 172.21.15.27:0/1704340543 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fe7d806c190 0x7fe7d806e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:46.130 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.128+0000 7fe7eeffd640 1 -- 172.21.15.27:0/1704340543 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5709+0+0 (secure 0 0 0) 0x7fe7e80acad0 con 0x7fe7f8072df0 2024-09-12T03:19:46.130 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.128+0000 7fe7f5d9b640 1 --2- 172.21.15.27:0/1704340543 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fe7d806c190 0x7fe7d806e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:46.130 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.129+0000 7fe7f5d9b640 1 --2- 172.21.15.27:0/1704340543 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fe7d806c190 0x7fe7d806e650 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fe7f8158550 tx=0x7fe7e0008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:46.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.132+0000 7fe7eeffd640 1 -- 172.21.15.27:0/1704340543 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fe7e807aff0 con 0x7fe7f8072df0 2024-09-12T03:19:46.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.344+0000 7fe7fc931640 1 -- 172.21.15.27:0/1704340543 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe7f8157fc0 con 0x7fe7f8072df0 2024-09-12T03:19:46.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.344+0000 7fe7eeffd640 1 -- 172.21.15.27:0/1704340543 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+5220 (secure 0 0 0) 0x7fe7e807a8f0 con 0x7fe7f8072df0 2024-09-12T03:19:46.344 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:19:46.345 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":31,"btime":"2024-09-12T03:19:38:158456+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5102,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/565788762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":565788762},{"type":"v1","addr":"172.21.15.27:6835","nonce":565788762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26},{"gid":5413,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6836/2689309305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":2689309305},{"type":"v1","addr":"172.21.15.45:6836","nonce":2689309305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":5640,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6834/3694936463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":3694936463},{"type":"v1","addr":"172.21.15.27:6834","nonce":3694936463}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":30,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-12T03:19:25.201174+0000","modified":"2024-09-12T03:19:27.190661+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5649},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5649":{"gid":5649,"name":"b","rank":0,"incarnation":28,"state":"up:active","state_seq":3,"addr":"172.21.15.45:6837/3720118075","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6835","nonce":3720118075},{"type":"v1","addr":"172.21.15.45:6837","nonce":3720118075}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5649,"qdb_cluster":[5649]},"id":4}]} 2024-09-12T03:19:46.345 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 31 2024-09-12T03:19:46.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.346+0000 7fe7fc931640 1 -- 172.21.15.27:0/1704340543 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fe7d806c190 msgr2=0x7fe7d806e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:46.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.346+0000 7fe7fc931640 1 --2- 172.21.15.27:0/1704340543 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fe7d806c190 0x7fe7d806e650 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fe7f8158550 tx=0x7fe7e0008040 comp rx=0 tx=0).stop 2024-09-12T03:19:46.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.346+0000 7fe7fc931640 1 -- 172.21.15.27:0/1704340543 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe7f8072df0 msgr2=0x7fe7f8146900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:46.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.346+0000 7fe7fc931640 1 --2- 172.21.15.27:0/1704340543 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe7f8072df0 0x7fe7f8146900 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fe7e802abe0 tx=0x7fe7e8004a40 comp rx=0 tx=0).stop 2024-09-12T03:19:46.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.346+0000 7fe7fc931640 1 -- 172.21.15.27:0/1704340543 shutdown_connections 2024-09-12T03:19:46.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.346+0000 7fe7fc931640 1 --2- 172.21.15.27:0/1704340543 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fe7d806c190 0x7fe7d806e650 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:46.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.346+0000 7fe7fc931640 1 --2- 172.21.15.27:0/1704340543 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe7f80b5a70 0x7fe7f80b1770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:46.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.346+0000 7fe7fc931640 1 --2- 172.21.15.27:0/1704340543 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe7f80ae8d0 0x7fe7f80b1230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:46.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.346+0000 7fe7fc931640 1 --2- 172.21.15.27:0/1704340543 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe7f8072df0 0x7fe7f8146900 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:46.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.347+0000 7fe7fc931640 1 -- 172.21.15.27:0/1704340543 >> 172.21.15.27:0/1704340543 conn(0x7fe7f806ca30 msgr2=0x7fe7f806e930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:46.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.347+0000 7fe7fc931640 1 -- 172.21.15.27:0/1704340543 shutdown_connections 2024-09-12T03:19:46.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.347+0000 7fe7fc931640 1 -- 172.21.15.27:0/1704340543 wait complete. 2024-09-12T03:19:46.357 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-12T03:19:46.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.460+0000 7f58ef26e640 1 Processor -- start 2024-09-12T03:19:46.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.460+0000 7f58ef26e640 1 -- start start 2024-09-12T03:19:46.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.461+0000 7f58ef26e640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f58e80736c0 0x7f58e815d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:46.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.461+0000 7f58ef26e640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f58e80d1e30 con 0x7f58e8073250 2024-09-12T03:19:46.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.461+0000 7f58ef26e640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f58e80d2100 con 0x7f58e8146020 2024-09-12T03:19:46.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.461+0000 7f58ef26e640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f58e8146400 con 0x7f58e80736c0 2024-09-12T03:19:46.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.461+0000 7f58ec809640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f58e8073250 0x7f58e815cbe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:46058/0 (socket says 172.21.15.27:46058) 2024-09-12T03:19:46.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.461+0000 7f58ec809640 1 -- 172.21.15.27:0/1261050980 learned_addr learned my addr 172.21.15.27:0/1261050980 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:19:46.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.461+0000 7f58ed80b640 1 --2- 172.21.15.27:0/1261050980 >> v2:172.21.15.45:3300/0 conn(0x7f58e80736c0 0x7f58e815d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:46.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.461+0000 7f58dffff640 1 -- 172.21.15.27:0/1261050980 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3642512997 0 0) 0x7f58e80d1e30 con 0x7f58e8073250 2024-09-12T03:19:46.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.461+0000 7f58dffff640 1 -- 172.21.15.27:0/1261050980 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f58d4003520 con 0x7f58e8073250 2024-09-12T03:19:46.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.462+0000 7f58ed80b640 1 -- 172.21.15.27:0/1261050980 >> v1:172.21.15.45:6789/0 conn(0x7f58e8146020 legacy=0x7f58e80d1720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:46.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.462+0000 7f58dffff640 1 -- 172.21.15.27:0/1261050980 <== mon.1 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1169226208 0 0) 0x7f58e80d2100 con 0x7f58e8146020 2024-09-12T03:19:46.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.462+0000 7f58ed80b640 1 -- 172.21.15.27:0/1261050980 >> v1:172.21.15.27:6789/0 conn(0x7f58e8073250 legacy=0x7f58e815cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:46.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.462+0000 7f58ed80b640 1 -- 172.21.15.27:0/1261050980 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58e80b6fb0 con 0x7f58e80736c0 2024-09-12T03:19:46.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.462+0000 7f58ed80b640 1 --2- 172.21.15.27:0/1261050980 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f58e80736c0 0x7f58e815d2f0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f58e0009d90 tx=0x7f58e002a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e11e540917c89ea7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:46.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.463+0000 7f58dffff640 1 -- 172.21.15.27:0/1261050980 <== mon.1 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58e0035020 con 0x7f58e80736c0 2024-09-12T03:19:46.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.463+0000 7f58dffff640 1 -- 172.21.15.27:0/1261050980 <== mon.1 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58e0007540 con 0x7f58e80736c0 2024-09-12T03:19:46.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.463+0000 7f58dffff640 1 -- 172.21.15.27:0/1261050980 <== mon.1 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58d80015f0 con 0x7f58e80736c0 2024-09-12T03:19:46.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.463+0000 7f58ef26e640 1 -- 172.21.15.27:0/1261050980 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f58e80736c0 msgr2=0x7f58e815d2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:46.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.463+0000 7f58ef26e640 1 --2- 172.21.15.27:0/1261050980 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f58e80736c0 0x7f58e815d2f0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f58e0009d90 tx=0x7f58e002a660 comp rx=0 tx=0).stop 2024-09-12T03:19:46.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.463+0000 7f58ef26e640 1 -- 172.21.15.27:0/1261050980 shutdown_connections 2024-09-12T03:19:46.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.463+0000 7f58ef26e640 1 --2- 172.21.15.27:0/1261050980 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f58e80736c0 0x7f58e815d2f0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:46.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.463+0000 7f58ef26e640 1 -- 172.21.15.27:0/1261050980 >> 172.21.15.27:0/1261050980 conn(0x7f58e806f300 msgr2=0x7f58e806f710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:46.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.463+0000 7f58ef26e640 1 -- 172.21.15.27:0/1261050980 shutdown_connections 2024-09-12T03:19:46.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.463+0000 7f58ef26e640 1 -- 172.21.15.27:0/1261050980 wait complete. 2024-09-12T03:19:46.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.464+0000 7f58ef26e640 1 Processor -- start 2024-09-12T03:19:46.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.464+0000 7f58ef26e640 1 -- start start 2024-09-12T03:19:46.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.465+0000 7f58ef26e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f58e8073250 0x7f58e815ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:46.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.465+0000 7f58ef26e640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f58e80736c0 0x7f58e813fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:46.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.465+0000 7f58ef26e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f58e8146020 0x7f58e80cf6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:46.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.465+0000 7f58ef26e640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f58e80b6c60 con 0x7f58e8146020 2024-09-12T03:19:46.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.465+0000 7f58ef26e640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f58e80b7570 con 0x7f58e8073250 2024-09-12T03:19:46.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.465+0000 7f58ef26e640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f58e8073ad0 con 0x7f58e80736c0 2024-09-12T03:19:46.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.465+0000 7f58ed80b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f58e8146020 0x7f58e80cf6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:46.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.465+0000 7f58ed80b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f58e8146020 0x7f58e80cf6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58322/0 (socket says 172.21.15.27:58322) 2024-09-12T03:19:46.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.465+0000 7f58ed80b640 1 -- 172.21.15.27:0/481880763 learned_addr learned my addr 172.21.15.27:0/481880763 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:19:46.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.466+0000 7f58ec809640 1 --2- 172.21.15.27:0/481880763 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f58e80736c0 0x7f58e813fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:46.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.466+0000 7f58ed00a640 1 --2- 172.21.15.27:0/481880763 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f58e8073250 0x7f58e815ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:46.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.466+0000 7f58ed80b640 1 -- 172.21.15.27:0/481880763 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f58e80736c0 msgr2=0x7f58e813fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:46.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.466+0000 7f58ed80b640 1 --2- 172.21.15.27:0/481880763 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f58e80736c0 0x7f58e813fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:46.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.466+0000 7f58ed80b640 1 -- 172.21.15.27:0/481880763 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f58e8073250 msgr2=0x7f58e815ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:46.468 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.466+0000 7f58ed80b640 1 --2- 172.21.15.27:0/481880763 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f58e8073250 0x7f58e815ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:46.468 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.466+0000 7f58ed80b640 1 -- 172.21.15.27:0/481880763 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58e815da20 con 0x7f58e8146020 2024-09-12T03:19:46.468 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.466+0000 7f58ed00a640 1 --2- 172.21.15.27:0/481880763 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f58e8073250 0x7f58e815ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:19:46.468 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.466+0000 7f58ed80b640 1 --2- 172.21.15.27:0/481880763 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f58e8146020 0x7f58e80cf6b0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f58e002ac50 tx=0x7f58e0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:46.468 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.466+0000 7f58ddffb640 1 -- 172.21.15.27:0/481880763 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58e0009450 con 0x7f58e8146020 2024-09-12T03:19:46.468 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.466+0000 7f58ddffb640 1 -- 172.21.15.27:0/481880763 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58e0031c40 con 0x7f58e8146020 2024-09-12T03:19:46.468 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.466+0000 7f58ef26e640 1 -- 172.21.15.27:0/481880763 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58e815d460 con 0x7f58e8146020 2024-09-12T03:19:46.468 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.466+0000 7f58ddffb640 1 -- 172.21.15.27:0/481880763 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58e0003990 con 0x7f58e8146020 2024-09-12T03:19:46.468 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.466+0000 7f58ef26e640 1 -- 172.21.15.27:0/481880763 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58e81f5b10 con 0x7f58e8146020 2024-09-12T03:19:46.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.467+0000 7f58ef26e640 1 -- 172.21.15.27:0/481880763 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58e806cf20 con 0x7f58e8146020 2024-09-12T03:19:46.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.468+0000 7f58ddffb640 1 -- 172.21.15.27:0/481880763 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 11) ==== 90804+0+0 (secure 0 0 0) 0x7f58e0003b30 con 0x7f58e8146020 2024-09-12T03:19:46.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.468+0000 7f58ddffb640 1 --2- 172.21.15.27:0/481880763 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f58cc06c190 0x7f58cc06e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:46.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.468+0000 7f58ddffb640 1 -- 172.21.15.27:0/481880763 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5709+0+0 (secure 0 0 0) 0x7f58e00ac650 con 0x7f58e8146020 2024-09-12T03:19:46.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.468+0000 7f58ed00a640 1 --2- 172.21.15.27:0/481880763 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f58cc06c190 0x7f58cc06e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:46.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.469+0000 7f58ed00a640 1 --2- 172.21.15.27:0/481880763 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f58cc06c190 0x7f58cc06e650 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f58e8142960 tx=0x7f58e4009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:46.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.473+0000 7f58ddffb640 1 -- 172.21.15.27:0/481880763 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f58e0041020 con 0x7f58e8146020 2024-09-12T03:19:46.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.685+0000 7f58ef26e640 1 -- 172.21.15.27:0/481880763 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f58e806eb10 con 0x7f58e8146020 2024-09-12T03:19:46.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.686+0000 7f58ddffb640 1 -- 172.21.15.27:0/481880763 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+5220 (secure 0 0 0) 0x7f58e007ab70 con 0x7f58e8146020 2024-09-12T03:19:46.686 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:19:46.687 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":31,"btime":"2024-09-12T03:19:38:158456+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5102,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/565788762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":565788762},{"type":"v1","addr":"172.21.15.27:6835","nonce":565788762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26},{"gid":5413,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6836/2689309305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":2689309305},{"type":"v1","addr":"172.21.15.45:6836","nonce":2689309305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":5640,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6834/3694936463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":3694936463},{"type":"v1","addr":"172.21.15.27:6834","nonce":3694936463}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":30,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-12T03:19:25.201174+0000","modified":"2024-09-12T03:19:27.190661+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5649},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5649":{"gid":5649,"name":"b","rank":0,"incarnation":28,"state":"up:active","state_seq":3,"addr":"172.21.15.45:6837/3720118075","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6835","nonce":3720118075},{"type":"v1","addr":"172.21.15.45:6837","nonce":3720118075}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5649,"qdb_cluster":[5649]},"id":4}]} 2024-09-12T03:19:46.687 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 31 2024-09-12T03:19:46.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.688+0000 7f58ef26e640 1 -- 172.21.15.27:0/481880763 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f58cc06c190 msgr2=0x7f58cc06e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:46.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.688+0000 7f58ef26e640 1 --2- 172.21.15.27:0/481880763 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f58cc06c190 0x7f58cc06e650 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f58e8142960 tx=0x7f58e4009040 comp rx=0 tx=0).stop 2024-09-12T03:19:46.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.688+0000 7f58ef26e640 1 -- 172.21.15.27:0/481880763 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f58e8146020 msgr2=0x7f58e80cf6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:46.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.688+0000 7f58ef26e640 1 --2- 172.21.15.27:0/481880763 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f58e8146020 0x7f58e80cf6b0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f58e002ac50 tx=0x7f58e0004a40 comp rx=0 tx=0).stop 2024-09-12T03:19:46.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.688+0000 7f58ef26e640 1 -- 172.21.15.27:0/481880763 shutdown_connections 2024-09-12T03:19:46.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.688+0000 7f58ef26e640 1 --2- 172.21.15.27:0/481880763 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f58cc06c190 0x7f58cc06e650 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:46.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.688+0000 7f58ef26e640 1 --2- 172.21.15.27:0/481880763 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f58e8146020 0x7f58e80cf6b0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:46.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.688+0000 7f58ef26e640 1 --2- 172.21.15.27:0/481880763 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f58e80736c0 0x7f58e813fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:46.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.688+0000 7f58ef26e640 1 --2- 172.21.15.27:0/481880763 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f58e8073250 0x7f58e815ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:46.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.688+0000 7f58ef26e640 1 -- 172.21.15.27:0/481880763 >> 172.21.15.27:0/481880763 conn(0x7f58e806f300 msgr2=0x7f58e815e8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:46.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.688+0000 7f58ef26e640 1 -- 172.21.15.27:0/481880763 shutdown_connections 2024-09-12T03:19:46.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.688+0000 7f58ef26e640 1 -- 172.21.15.27:0/481880763 wait complete. 2024-09-12T03:19:46.699 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-12T03:19:46.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.802+0000 7f6d1973b640 1 Processor -- start 2024-09-12T03:19:46.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.802+0000 7f6d1973b640 1 -- start start 2024-09-12T03:19:46.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.803+0000 7f6d1973b640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f6d14162a30 0x7f6d141693f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:46.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.803+0000 7f6d1973b640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6d140d2240 con 0x7f6d14073960 2024-09-12T03:19:46.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.803+0000 7f6d1973b640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6d14163090 con 0x7f6d1415e5f0 2024-09-12T03:19:46.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.803+0000 7f6d1973b640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f6d140d0da0 con 0x7f6d14162a30 2024-09-12T03:19:46.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.803+0000 7f6d0bfff640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f6d14073960 0x7f6d14162320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:46064/0 (socket says 172.21.15.27:46064) 2024-09-12T03:19:46.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.803+0000 7f6d0bfff640 1 -- 172.21.15.27:0/3685332573 learned_addr learned my addr 172.21.15.27:0/3685332573 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:19:46.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.803+0000 7f6d137fe640 1 --2- 172.21.15.27:0/3685332573 >> v2:172.21.15.45:3300/0 conn(0x7f6d14162a30 0x7f6d141693f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:46.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.803+0000 7f6d127fc640 1 -- 172.21.15.27:0/3685332573 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1692571059 0 0) 0x7f6d140d2240 con 0x7f6d14073960 2024-09-12T03:19:46.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.803+0000 7f6d127fc640 1 -- 172.21.15.27:0/3685332573 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6d00003520 con 0x7f6d14073960 2024-09-12T03:19:46.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.803+0000 7f6d127fc640 1 -- 172.21.15.27:0/3685332573 <== mon.1 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1326700295 0 0) 0x7f6d14163090 con 0x7f6d1415e5f0 2024-09-12T03:19:46.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.803+0000 7f6d127fc640 1 -- 172.21.15.27:0/3685332573 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6d140d2240 con 0x7f6d1415e5f0 2024-09-12T03:19:46.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.803+0000 7f6d127fc640 1 -- 172.21.15.27:0/3685332573 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2834576968 0 0) 0x7f6d00003520 con 0x7f6d14073960 2024-09-12T03:19:46.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.803+0000 7f6d127fc640 1 -- 172.21.15.27:0/3685332573 >> v2:172.21.15.45:3300/0 conn(0x7f6d14162a30 msgr2=0x7f6d141693f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:46.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.803+0000 7f6d127fc640 1 --2- 172.21.15.27:0/3685332573 >> v2:172.21.15.45:3300/0 conn(0x7f6d14162a30 0x7f6d141693f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:46.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.804+0000 7f6d127fc640 1 -- 172.21.15.27:0/3685332573 >> v1:172.21.15.45:6789/0 conn(0x7f6d1415e5f0 legacy=0x7f6d14073250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:46.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.804+0000 7f6d127fc640 1 -- 172.21.15.27:0/3685332573 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d140d0470 con 0x7f6d14073960 2024-09-12T03:19:46.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.804+0000 7f6d127fc640 1 -- 172.21.15.27:0/3685332573 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 847755213 0 0) 0x7f6cfc001690 con 0x7f6d14073960 2024-09-12T03:19:46.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.804+0000 7f6d137fe640 1 --2- 172.21.15.27:0/3685332573 >> v2:172.21.15.45:3300/0 conn(0x7f6d14162a30 0x7f6d141693f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:19:46.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.804+0000 7f6d127fc640 1 -- 172.21.15.27:0/3685332573 >> v1:172.21.15.27:6789/0 conn(0x7f6d14073960 legacy=0x7f6d14162320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:46.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.804+0000 7f6d127fc640 1 --2- 172.21.15.27:0/3685332573 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6d00003b40 0x7f6d00003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:46.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.804+0000 7f6d127fc640 1 --2- 172.21.15.27:0/3685332573 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f6d00004800 0x7f6d00004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:46.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.804+0000 7f6d127fc640 1 --2- 172.21.15.27:0/3685332573 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6d000051c0 0x7f6d0000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:46.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.804+0000 7f6d127fc640 1 -- 172.21.15.27:0/3685332573 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f6d140d0da0 con 0x7f6d000051c0 2024-09-12T03:19:46.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.804+0000 7f6d127fc640 1 -- 172.21.15.27:0/3685332573 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f6d0000c040 con 0x7f6d00003b40 2024-09-12T03:19:46.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.804+0000 7f6d127fc640 1 -- 172.21.15.27:0/3685332573 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f6d0000c240 con 0x7f6d00004800 2024-09-12T03:19:46.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.804+0000 7f6d137fe640 1 --2- 172.21.15.27:0/3685332573 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6d000051c0 0x7f6d0000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:46.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.804+0000 7f6d12ffd640 1 --2- 172.21.15.27:0/3685332573 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6d00003b40 0x7f6d00003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:46.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.804+0000 7f6d0bfff640 1 --2- 172.21.15.27:0/3685332573 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f6d00004800 0x7f6d00004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:46.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.804+0000 7f6d137fe640 1 -- 172.21.15.27:0/3685332573 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f6d00004800 msgr2=0x7f6d00004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:46.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.804+0000 7f6d137fe640 1 --2- 172.21.15.27:0/3685332573 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f6d00004800 0x7f6d00004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:46.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.804+0000 7f6d137fe640 1 -- 172.21.15.27:0/3685332573 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6d00003b40 msgr2=0x7f6d00003f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:46.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.804+0000 7f6d137fe640 1 --2- 172.21.15.27:0/3685332573 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6d00003b40 0x7f6d00003f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:46.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.804+0000 7f6d137fe640 1 -- 172.21.15.27:0/3685332573 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d00003520 con 0x7f6d000051c0 2024-09-12T03:19:46.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.804+0000 7f6d12ffd640 1 --2- 172.21.15.27:0/3685332573 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6d00003b40 0x7f6d00003f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:19:46.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.805+0000 7f6d137fe640 1 -- 172.21.15.27:0/3685332573 reap_dead start 2024-09-12T03:19:46.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.805+0000 7f6d137fe640 1 --2- 172.21.15.27:0/3685332573 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6d000051c0 0x7f6d0000b9c0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f6d0c00e350 tx=0x7f6d00005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95c92ca2be00b54e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:46.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.805+0000 7f6d127fc640 1 -- 172.21.15.27:0/3685332573 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d0c009de0 con 0x7f6d000051c0 2024-09-12T03:19:46.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.805+0000 7f6d127fc640 1 -- 172.21.15.27:0/3685332573 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d0c02e680 con 0x7f6d000051c0 2024-09-12T03:19:46.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.805+0000 7f6d127fc640 1 -- 172.21.15.27:0/3685332573 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d0c02ea70 con 0x7f6d000051c0 2024-09-12T03:19:46.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.805+0000 7f6d1973b640 1 -- 172.21.15.27:0/3685332573 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6d000051c0 msgr2=0x7f6d0000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:46.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.805+0000 7f6d1973b640 1 --2- 172.21.15.27:0/3685332573 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6d000051c0 0x7f6d0000b9c0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f6d0c00e350 tx=0x7f6d00005830 comp rx=0 tx=0).stop 2024-09-12T03:19:46.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.806+0000 7f6d1973b640 1 -- 172.21.15.27:0/3685332573 shutdown_connections 2024-09-12T03:19:46.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.806+0000 7f6d1973b640 1 --2- 172.21.15.27:0/3685332573 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6d000051c0 0x7f6d0000b9c0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:46.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.806+0000 7f6d1973b640 1 -- 172.21.15.27:0/3685332573 >> 172.21.15.27:0/3685332573 conn(0x7f6d1406f300 msgr2=0x7f6d1406f710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:46.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.806+0000 7f6d1973b640 1 -- 172.21.15.27:0/3685332573 shutdown_connections 2024-09-12T03:19:46.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.806+0000 7f6d1973b640 1 -- 172.21.15.27:0/3685332573 wait complete. 2024-09-12T03:19:46.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.807+0000 7f6d1973b640 1 Processor -- start 2024-09-12T03:19:46.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.807+0000 7f6d1973b640 1 -- start start 2024-09-12T03:19:46.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.808+0000 7f6d1973b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6d000051c0 0x7f6d14156830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:46.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.808+0000 7f6d1973b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6d14154e20 0x7f6d14148d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:46.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.808+0000 7f6d1973b640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f6d1415c2f0 0x7f6d1415c770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:46.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.808+0000 7f6d1973b640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f6d140d0a30 con 0x7f6d000051c0 2024-09-12T03:19:46.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.808+0000 7f6d1973b640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f6d14162e40 con 0x7f6d14154e20 2024-09-12T03:19:46.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.808+0000 7f6d1973b640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f6d140b5620 con 0x7f6d1415c2f0 2024-09-12T03:19:46.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.808+0000 7f6d12ffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6d000051c0 0x7f6d14156830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:46.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.808+0000 7f6d12ffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6d000051c0 0x7f6d14156830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58332/0 (socket says 172.21.15.27:58332) 2024-09-12T03:19:46.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.808+0000 7f6d12ffd640 1 -- 172.21.15.27:0/2714397306 learned_addr learned my addr 172.21.15.27:0/2714397306 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:19:46.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.808+0000 7f6d137fe640 1 --2- 172.21.15.27:0/2714397306 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f6d1415c2f0 0x7f6d1415c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:46.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.808+0000 7f6d0bfff640 1 --2- 172.21.15.27:0/2714397306 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6d14154e20 0x7f6d14148d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:46.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.808+0000 7f6d12ffd640 1 -- 172.21.15.27:0/2714397306 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f6d1415c2f0 msgr2=0x7f6d1415c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:46.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.808+0000 7f6d12ffd640 1 --2- 172.21.15.27:0/2714397306 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f6d1415c2f0 0x7f6d1415c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:46.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.808+0000 7f6d12ffd640 1 -- 172.21.15.27:0/2714397306 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6d14154e20 msgr2=0x7f6d14148d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:46.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.808+0000 7f6d12ffd640 1 --2- 172.21.15.27:0/2714397306 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6d14154e20 0x7f6d14148d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:46.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.808+0000 7f6d12ffd640 1 -- 172.21.15.27:0/2714397306 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d141492d0 con 0x7f6d000051c0 2024-09-12T03:19:46.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.808+0000 7f6d0bfff640 1 --2- 172.21.15.27:0/2714397306 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6d14154e20 0x7f6d14148d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:19:46.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.808+0000 7f6d12ffd640 1 --2- 172.21.15.27:0/2714397306 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6d000051c0 0x7f6d14156830 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f6d0400e580 tx=0x7f6d0400ea40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:46.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.808+0000 7f6d0b7fe640 1 -- 172.21.15.27:0/2714397306 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d040052e0 con 0x7f6d000051c0 2024-09-12T03:19:46.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.808+0000 7f6d0b7fe640 1 -- 172.21.15.27:0/2714397306 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d04007020 con 0x7f6d000051c0 2024-09-12T03:19:46.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.808+0000 7f6d1973b640 1 -- 172.21.15.27:0/2714397306 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d14153d80 con 0x7f6d000051c0 2024-09-12T03:19:46.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.808+0000 7f6d0b7fe640 1 -- 172.21.15.27:0/2714397306 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d040071f0 con 0x7f6d000051c0 2024-09-12T03:19:46.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.808+0000 7f6d1973b640 1 -- 172.21.15.27:0/2714397306 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d14154340 con 0x7f6d000051c0 2024-09-12T03:19:46.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.809+0000 7f6d1973b640 1 -- 172.21.15.27:0/2714397306 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d14162320 con 0x7f6d000051c0 2024-09-12T03:19:46.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.810+0000 7f6d0b7fe640 1 -- 172.21.15.27:0/2714397306 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 11) ==== 90804+0+0 (secure 0 0 0) 0x7f6d0401a020 con 0x7f6d000051c0 2024-09-12T03:19:46.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.810+0000 7f6d0b7fe640 1 --2- 172.21.15.27:0/2714397306 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f6cf406c1c0 0x7f6cf406e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:46.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.810+0000 7f6d0b7fe640 1 -- 172.21.15.27:0/2714397306 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5709+0+0 (secure 0 0 0) 0x7f6d0408a9d0 con 0x7f6d000051c0 2024-09-12T03:19:46.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.810+0000 7f6d0bfff640 1 --2- 172.21.15.27:0/2714397306 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f6cf406c1c0 0x7f6cf406e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:46.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.811+0000 7f6d0bfff640 1 --2- 172.21.15.27:0/2714397306 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f6cf406c1c0 0x7f6cf406e680 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f6d141f7b30 tx=0x7f6cfc009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:46.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.814+0000 7f6d0b7fe640 1 -- 172.21.15.27:0/2714397306 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f6d04058f80 con 0x7f6d000051c0 2024-09-12T03:19:46.992 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.992+0000 7f6d1973b640 1 -- 172.21.15.27:0/2714397306 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6d1415e9d0 con 0x7f6d000051c0 2024-09-12T03:19:46.993 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.994+0000 7f6d0b7fe640 1 -- 172.21.15.27:0/2714397306 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v56) ==== 74+0+16677 (secure 0 0 0) 0x7f6d04058880 con 0x7f6d000051c0 2024-09-12T03:19:46.994 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:19:46.994 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":56,"fsid":"d2df504e-2647-4385-a5db-9fdf69f487a0","created":"2024-09-12T03:16:07.778937+0000","modified":"2024-09-12T03:19:26.561081+0000","last_up_change":"2024-09-12T03:16:19.314778+0000","last_in_change":"2024-09-12T03:16:09.893086+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T03:16:20.097646+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-09-12T03:19:23.556103+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-09-12T03:19:24.184337+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1851116c-1fb3-4dc9-9772-f90e2ef4c355","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6817","nonce":3002321488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6819","nonce":3002321488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6823","nonce":3002321488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6821","nonce":3002321488}]},"public_addr":"172.21.15.27:6817/3002321488","cluster_addr":"172.21.15.27:6819/3002321488","heartbeat_back_addr":"172.21.15.27:6823/3002321488","heartbeat_front_addr":"172.21.15.27:6821/3002321488","state":["exists","up"]},{"osd":1,"uuid":"f38ae17d-73eb-4204-b99d-975fa806fb87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6809","nonce":1613102886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6811","nonce":1613102886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6815","nonce":1613102886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6813","nonce":1613102886}]},"public_addr":"172.21.15.27:6809/1613102886","cluster_addr":"172.21.15.27:6811/1613102886","heartbeat_back_addr":"172.21.15.27:6815/1613102886","heartbeat_front_addr":"172.21.15.27:6813/1613102886","state":["exists","up"]},{"osd":2,"uuid":"fb12dcc8-bdd0-42b8-b46d-8b10106224bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6825","nonce":3534723355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6827","nonce":3534723355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6831","nonce":3534723355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6829","nonce":3534723355}]},"public_addr":"172.21.15.27:6825/3534723355","cluster_addr":"172.21.15.27:6827/3534723355","heartbeat_back_addr":"172.21.15.27:6831/3534723355","heartbeat_front_addr":"172.21.15.27:6829/3534723355","state":["exists","up"]},{"osd":3,"uuid":"d74d3f49-9c24-4b9f-98e2-3ecd7b5ef3b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6801","nonce":80661598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6803","nonce":80661598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6807","nonce":80661598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6805","nonce":80661598}]},"public_addr":"172.21.15.27:6801/80661598","cluster_addr":"172.21.15.27:6803/80661598","heartbeat_back_addr":"172.21.15.27:6807/80661598","heartbeat_front_addr":"172.21.15.27:6805/80661598","state":["exists","up"]},{"osd":4,"uuid":"2bf123f2-1b0d-41c4-aa3b-805bab267e59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6817","nonce":3943189827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6819","nonce":3943189827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6823","nonce":3943189827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6821","nonce":3943189827}]},"public_addr":"172.21.15.45:6817/3943189827","cluster_addr":"172.21.15.45:6819/3943189827","heartbeat_back_addr":"172.21.15.45:6823/3943189827","heartbeat_front_addr":"172.21.15.45:6821/3943189827","state":["exists","up"]},{"osd":5,"uuid":"c49ba5da-91c6-4a86-a97b-4aba89c253ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6809","nonce":3620760330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6811","nonce":3620760330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6815","nonce":3620760330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6813","nonce":3620760330}]},"public_addr":"172.21.15.45:6809/3620760330","cluster_addr":"172.21.15.45:6811/3620760330","heartbeat_back_addr":"172.21.15.45:6815/3620760330","heartbeat_front_addr":"172.21.15.45:6813/3620760330","state":["exists","up"]},{"osd":6,"uuid":"b0123475-89f7-4a34-9e33-d797b99dd9dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6801","nonce":1989083510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6803","nonce":1989083510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6807","nonce":1989083510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6805","nonce":1989083510}]},"public_addr":"172.21.15.45:6801/1989083510","cluster_addr":"172.21.15.45:6803/1989083510","heartbeat_back_addr":"172.21.15.45:6807/1989083510","heartbeat_front_addr":"172.21.15.45:6805/1989083510","state":["exists","up"]},{"osd":7,"uuid":"4dad773d-b7d8-46fc-8a10-a972aad27b29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6825","nonce":1513300294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6827","nonce":1513300294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6831","nonce":1513300294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6829","nonce":1513300294}]},"public_addr":"172.21.15.45:6825/1513300294","cluster_addr":"172.21.15.45:6827/1513300294","heartbeat_back_addr":"172.21.15.45:6831/1513300294","heartbeat_front_addr":"172.21.15.45:6829/1513300294","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.881855+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:17.133252+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.735339+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.500742+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.477571+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:17.167667+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.470291+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.873743+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T03:19:46.996 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.996+0000 7f6d1973b640 1 -- 172.21.15.27:0/2714397306 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f6cf406c1c0 msgr2=0x7f6cf406e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:46.996 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.996+0000 7f6d1973b640 1 --2- 172.21.15.27:0/2714397306 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f6cf406c1c0 0x7f6cf406e680 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f6d141f7b30 tx=0x7f6cfc009040 comp rx=0 tx=0).stop 2024-09-12T03:19:46.996 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.996+0000 7f6d1973b640 1 -- 172.21.15.27:0/2714397306 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6d000051c0 msgr2=0x7f6d14156830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:46.996 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.996+0000 7f6d1973b640 1 --2- 172.21.15.27:0/2714397306 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6d000051c0 0x7f6d14156830 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f6d0400e580 tx=0x7f6d0400ea40 comp rx=0 tx=0).stop 2024-09-12T03:19:46.997 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.996+0000 7f6d1973b640 1 -- 172.21.15.27:0/2714397306 shutdown_connections 2024-09-12T03:19:46.997 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.996+0000 7f6d1973b640 1 --2- 172.21.15.27:0/2714397306 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f6cf406c1c0 0x7f6cf406e680 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:46.997 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.996+0000 7f6d1973b640 1 --2- 172.21.15.27:0/2714397306 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f6d1415c2f0 0x7f6d1415c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:46.997 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.996+0000 7f6d1973b640 1 --2- 172.21.15.27:0/2714397306 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6d14154e20 0x7f6d14148d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:46.997 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.996+0000 7f6d1973b640 1 --2- 172.21.15.27:0/2714397306 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6d000051c0 0x7f6d14156830 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:46.997 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.996+0000 7f6d1973b640 1 -- 172.21.15.27:0/2714397306 >> 172.21.15.27:0/2714397306 conn(0x7f6d1406f300 msgr2=0x7f6d14161460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:46.997 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.996+0000 7f6d1973b640 1 -- 172.21.15.27:0/2714397306 shutdown_connections 2024-09-12T03:19:46.997 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:46.996+0000 7f6d1973b640 1 -- 172.21.15.27:0/2714397306 wait complete. 2024-09-12T03:19:47.007 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-12T03:19:47.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:47.110+0000 7f816b978640 1 Processor -- start 2024-09-12T03:19:47.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:47.111+0000 7f816b978640 1 -- start start 2024-09-12T03:19:47.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:47.111+0000 7f816b978640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f816415eb60 0x7f81640d1280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:47.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:47.111+0000 7f816b978640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f81640b6690 con 0x7f81640d17c0 2024-09-12T03:19:47.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:47.112+0000 7f816b978640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f81640b6970 con 0x7f816415e780 2024-09-12T03:19:47.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:47.112+0000 7f816b978640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f81640b7950 con 0x7f816415eb60 2024-09-12T03:19:47.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:47.112+0000 7f8169f15640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f81640d17c0 0x7f81640b3390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:46072/0 (socket says 172.21.15.27:46072) 2024-09-12T03:19:47.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:47.112+0000 7f8169f15640 1 -- 172.21.15.27:0/3472127975 learned_addr learned my addr 172.21.15.27:0/3472127975 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:19:47.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:47.112+0000 7f815bfff640 1 -- 172.21.15.27:0/3472127975 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1296385267 0 0) 0x7f81640b6690 con 0x7f81640d17c0 2024-09-12T03:19:47.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:47.112+0000 7f815bfff640 1 -- 172.21.15.27:0/3472127975 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8150003520 con 0x7f81640d17c0 2024-09-12T03:19:47.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:47.112+0000 7f8168f13640 1 --2- 172.21.15.27:0/3472127975 >> v2:172.21.15.45:3300/0 conn(0x7f816415eb60 0x7f81640d1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:47.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:47.112+0000 7f815bfff640 1 -- 172.21.15.27:0/3472127975 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1854614959 0 0) 0x7f8150003520 con 0x7f81640d17c0 2024-09-12T03:19:47.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:47.112+0000 7f815bfff640 1 -- 172.21.15.27:0/3472127975 >> v2:172.21.15.45:3300/0 conn(0x7f816415eb60 msgr2=0x7f81640d1280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:47.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:47.112+0000 7f815bfff640 1 --2- 172.21.15.27:0/3472127975 >> v2:172.21.15.45:3300/0 conn(0x7f816415eb60 0x7f81640d1280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:47.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:47.112+0000 7f815bfff640 1 -- 172.21.15.27:0/3472127975 >> v1:172.21.15.45:6789/0 conn(0x7f816415e780 legacy=0x7f81640d3e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:47.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:47.112+0000 7f815bfff640 1 -- 172.21.15.27:0/3472127975 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81640b6c50 con 0x7f81640d17c0 2024-09-12T03:19:47.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:47.112+0000 7f815bfff640 1 -- 172.21.15.27:0/3472127975 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 847755213 0 0) 0x7f815c0019f0 con 0x7f81640d17c0 2024-09-12T03:19:47.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:47.112+0000 7f815bfff640 1 -- 172.21.15.27:0/3472127975 >> v1:172.21.15.27:6789/0 conn(0x7f81640d17c0 legacy=0x7f81640b3390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:47.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:47.112+0000 7f815bfff640 1 --2- 172.21.15.27:0/3472127975 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f81500039b0 0x7f8150003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:47.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:47.112+0000 7f815bfff640 1 --2- 172.21.15.27:0/3472127975 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8150004800 0x7f8150004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:47.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:47.112+0000 7f815bfff640 1 --2- 172.21.15.27:0/3472127975 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f81500051c0 0x7f815000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:47.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:47.112+0000 7f815bfff640 1 -- 172.21.15.27:0/3472127975 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f815000c030 con 0x7f81500039b0 2024-09-12T03:19:47.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:47.112+0000 7f815bfff640 1 -- 172.21.15.27:0/3472127975 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f815000c1e0 con 0x7f8150004800 2024-09-12T03:19:47.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:47.112+0000 7f815bfff640 1 -- 172.21.15.27:0/3472127975 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f815000c3e0 con 0x7f81500051c0 2024-09-12T03:19:47.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:47.112+0000 7f8169714640 1 --2- 172.21.15.27:0/3472127975 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f81500039b0 0x7f8150003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:47.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:47.112+0000 7f8168f13640 1 --2- 172.21.15.27:0/3472127975 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8150004800 0x7f8150004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:47.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:47.113+0000 7f8169f15640 1 --2- 172.21.15.27:0/3472127975 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f81500051c0 0x7f815000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:47.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:47.113+0000 7f8169714640 1 -- 172.21.15.27:0/3472127975 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f81500051c0 msgr2=0x7f815000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:47.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:47.113+0000 7f8169714640 1 --2- 172.21.15.27:0/3472127975 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f81500051c0 0x7f815000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:47.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:47.113+0000 7f8169714640 1 -- 172.21.15.27:0/3472127975 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8150004800 msgr2=0x7f8150004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:47.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:47.113+0000 7f8169714640 1 --2- 172.21.15.27:0/3472127975 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8150004800 0x7f8150004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:47.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:47.113+0000 7f8169f15640 1 -- 172.21.15.27:0/3472127975 reap_dead start 2024-09-12T03:19:47.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:47.113+0000 7f8169714640 1 -- 172.21.15.27:0/3472127975 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8150003520 con 0x7f81500039b0 2024-09-12T03:19:47.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:47.113+0000 7f8169714640 1 --2- 172.21.15.27:0/3472127975 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f81500039b0 0x7f8150003dd0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f8160001750 tx=0x7f8150005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87d7299dde7ff6e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:47.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:47.114+0000 7f815bfff640 1 -- 172.21.15.27:0/3472127975 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8160001370 con 0x7f81500039b0 2024-09-12T03:19:47.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:47.114+0000 7f815bfff640 1 -- 172.21.15.27:0/3472127975 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81600052d0 con 0x7f81500039b0 2024-09-12T03:19:47.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:47.114+0000 7f815bfff640 1 -- 172.21.15.27:0/3472127975 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f816002abc0 con 0x7f81500039b0 2024-09-12T03:19:47.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:47.114+0000 7f816b978640 1 -- 172.21.15.27:0/3472127975 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f81500039b0 msgr2=0x7f8150003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:47.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:47.114+0000 7f816b978640 1 --2- 172.21.15.27:0/3472127975 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f81500039b0 0x7f8150003dd0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f8160001750 tx=0x7f8150005830 comp rx=0 tx=0).stop 2024-09-12T03:19:47.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:47.114+0000 7f816b978640 1 -- 172.21.15.27:0/3472127975 shutdown_connections 2024-09-12T03:19:47.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:47.114+0000 7f816b978640 1 --2- 172.21.15.27:0/3472127975 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f81500039b0 0x7f8150003dd0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:47.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:47.114+0000 7f816b978640 1 -- 172.21.15.27:0/3472127975 >> 172.21.15.27:0/3472127975 conn(0x7f816406da60 msgr2=0x7f816406de70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:47.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:47.114+0000 7f816b978640 1 -- 172.21.15.27:0/3472127975 shutdown_connections 2024-09-12T03:19:47.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:47.114+0000 7f816b978640 1 -- 172.21.15.27:0/3472127975 wait complete. 2024-09-12T03:19:47.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:47.115+0000 7f816b978640 1 Processor -- start 2024-09-12T03:19:47.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:47.116+0000 7f816b978640 1 -- start start 2024-09-12T03:19:47.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:47.116+0000 7f816b978640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f81500039b0 0x7f816415c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:47.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:47.116+0000 7f816b978640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f816415cee0 0x7f816415d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:47.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:47.116+0000 7f816b978640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f81640abb00 0x7f81640abfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:47.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:47.116+0000 7f816b978640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f81640d1bf0 con 0x7f81500039b0 2024-09-12T03:19:47.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:47.116+0000 7f816b978640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f81640b7210 con 0x7f816415cee0 2024-09-12T03:19:47.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:47.116+0000 7f816b978640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f81640b6340 con 0x7f81640abb00 2024-09-12T03:19:47.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:47.116+0000 7f8169714640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f81500039b0 0x7f816415c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:47.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:47.116+0000 7f8169714640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f81500039b0 0x7f816415c0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58350/0 (socket says 172.21.15.27:58350) 2024-09-12T03:19:47.122 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:47.116+0000 7f8169714640 1 -- 172.21.15.27:0/493581910 learned_addr learned my addr 172.21.15.27:0/493581910 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:19:47.122 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:47.117+0000 7f8169f15640 1 --2- 172.21.15.27:0/493581910 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f81640abb00 0x7f81640abfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:47.122 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:47.117+0000 7f8168f13640 1 --2- 172.21.15.27:0/493581910 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f816415cee0 0x7f816415d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:47.122 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:47.117+0000 7f8169714640 1 -- 172.21.15.27:0/493581910 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f81640abb00 msgr2=0x7f81640abfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:47.122 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:47.117+0000 7f8169714640 1 --2- 172.21.15.27:0/493581910 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f81640abb00 0x7f81640abfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:47.122 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:47.117+0000 7f8169714640 1 -- 172.21.15.27:0/493581910 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f816415cee0 msgr2=0x7f816415d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:47.122 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:47.117+0000 7f8169714640 1 --2- 172.21.15.27:0/493581910 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f816415cee0 0x7f816415d360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:47.122 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:47.117+0000 7f8169714640 1 -- 172.21.15.27:0/493581910 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81640cbba0 con 0x7f81500039b0 2024-09-12T03:19:47.122 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:47.117+0000 7f8168f13640 1 --2- 172.21.15.27:0/493581910 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f816415cee0 0x7f816415d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:19:47.123 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:47.117+0000 7f8169f15640 1 --2- 172.21.15.27:0/493581910 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f81640abb00 0x7f81640abfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:19:47.123 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:47.117+0000 7f8169714640 1 --2- 172.21.15.27:0/493581910 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f81500039b0 0x7f816415c0f0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f8160000c00 tx=0x7f8160001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:47.123 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:47.117+0000 7f8159ffb640 1 -- 172.21.15.27:0/493581910 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f816003f030 con 0x7f81500039b0 2024-09-12T03:19:47.123 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:47.117+0000 7f8159ffb640 1 -- 172.21.15.27:0/493581910 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8160005480 con 0x7f81500039b0 2024-09-12T03:19:47.123 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:47.117+0000 7f816b978640 1 -- 172.21.15.27:0/493581910 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81640cbe80 con 0x7f81500039b0 2024-09-12T03:19:47.123 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:47.117+0000 7f8159ffb640 1 -- 172.21.15.27:0/493581910 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8160037040 con 0x7f81500039b0 2024-09-12T03:19:47.123 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:47.117+0000 7f816b978640 1 -- 172.21.15.27:0/493581910 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81640cc440 con 0x7f81500039b0 2024-09-12T03:19:47.123 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:47.118+0000 7f816b978640 1 -- 172.21.15.27:0/493581910 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81640d1280 con 0x7f81500039b0 2024-09-12T03:19:47.123 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:47.119+0000 7f8159ffb640 1 -- 172.21.15.27:0/493581910 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 11) ==== 90804+0+0 (secure 0 0 0) 0x7f8160042070 con 0x7f81500039b0 2024-09-12T03:19:47.123 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:47.119+0000 7f8159ffb640 1 --2- 172.21.15.27:0/493581910 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f814806c190 0x7f814806e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:47.123 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:47.119+0000 7f8159ffb640 1 -- 172.21.15.27:0/493581910 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5709+0+0 (secure 0 0 0) 0x7f81600aa950 con 0x7f81500039b0 2024-09-12T03:19:47.124 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:47.120+0000 7f8168f13640 1 --2- 172.21.15.27:0/493581910 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f814806c190 0x7f814806e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:47.124 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:47.120+0000 7f8168f13640 1 --2- 172.21.15.27:0/493581910 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f814806c190 0x7f814806e650 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f8164068eb0 tx=0x7f8154008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:47.124 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:47.124+0000 7f8159ffb640 1 -- 172.21.15.27:0/493581910 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f8160078e60 con 0x7f81500039b0 2024-09-12T03:19:47.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:47.335+0000 7f816b978640 1 -- 172.21.15.27:0/493581910 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f81640d3a60 con 0x7f81500039b0 2024-09-12T03:19:47.335 INFO:tasks.ceph.mon.a.smithi027.stderr:2024-09-12T03:19:47.335+0000 7f959bfd9640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-12T03:19:47.337 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:47.337+0000 7f8159ffb640 1 -- 172.21.15.27:0/493581910 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v32) ==== 161+0+0 (secure 0 0 0) 0x7f8160078760 con 0x7f81500039b0 2024-09-12T03:19:47.337 INFO:teuthology.orchestra.run.smithi027.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-12T03:19:47.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:47.340+0000 7f816b978640 1 -- 172.21.15.27:0/493581910 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f814806c190 msgr2=0x7f814806e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:47.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:47.340+0000 7f816b978640 1 --2- 172.21.15.27:0/493581910 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f814806c190 0x7f814806e650 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f8164068eb0 tx=0x7f8154008040 comp rx=0 tx=0).stop 2024-09-12T03:19:47.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:47.340+0000 7f816b978640 1 -- 172.21.15.27:0/493581910 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f81500039b0 msgr2=0x7f816415c0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:47.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:47.340+0000 7f816b978640 1 --2- 172.21.15.27:0/493581910 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f81500039b0 0x7f816415c0f0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f8160000c00 tx=0x7f8160001120 comp rx=0 tx=0).stop 2024-09-12T03:19:47.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:47.340+0000 7f816b978640 1 -- 172.21.15.27:0/493581910 shutdown_connections 2024-09-12T03:19:47.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:47.340+0000 7f816b978640 1 --2- 172.21.15.27:0/493581910 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f814806c190 0x7f814806e650 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:47.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:47.340+0000 7f816b978640 1 --2- 172.21.15.27:0/493581910 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f81640abb00 0x7f81640abfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:47.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:47.340+0000 7f816b978640 1 --2- 172.21.15.27:0/493581910 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f816415cee0 0x7f816415d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:47.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:47.340+0000 7f816b978640 1 --2- 172.21.15.27:0/493581910 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f81500039b0 0x7f816415c0f0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:47.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:47.340+0000 7f816b978640 1 -- 172.21.15.27:0/493581910 >> 172.21.15.27:0/493581910 conn(0x7f816406da60 msgr2=0x7f8164161190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:47.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:47.340+0000 7f816b978640 1 -- 172.21.15.27:0/493581910 shutdown_connections 2024-09-12T03:19:47.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:47.340+0000 7f816b978640 1 -- 172.21.15.27:0/493581910 wait complete. 2024-09-12T03:19:47.351 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-12T03:19:47.382 INFO:tasks.ceph.mds.b.smithi045.stdout:starting mds.b at 2024-09-12T03:19:47.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:47.456+0000 7f322100e640 1 Processor -- start 2024-09-12T03:19:47.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:47.456+0000 7f322100e640 1 -- start start 2024-09-12T03:19:47.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:47.457+0000 7f322100e640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f321c15ec30 0x7f321c0b3900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:47.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:47.457+0000 7f322100e640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f321c0af9d0 0x7f321c0b3e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:47.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:47.457+0000 7f322100e640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f321c0ce5e0 con 0x7f321c0b4380 2024-09-12T03:19:47.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:47.457+0000 7f322100e640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f321c0d02d0 con 0x7f321c0af9d0 2024-09-12T03:19:47.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:47.457+0000 7f322100e640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f321c06c650 con 0x7f321c15ec30 2024-09-12T03:19:47.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:47.457+0000 7f321ad9d640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f321c15ec30 0x7f321c0b3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:47.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:47.457+0000 7f3213fff640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f321c0af9d0 0x7f321c0b3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:47.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:47.457+0000 7f321ad9d640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f321c15ec30 0x7f321c0b3900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3301/0 says I am v2:172.21.15.27:37532/0 (socket says 172.21.15.27:37532) 2024-09-12T03:19:47.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:47.457+0000 7f321ad9d640 1 -- 172.21.15.27:0/574157259 learned_addr learned my addr 172.21.15.27:0/574157259 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:19:47.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:47.458+0000 7f321ad9d640 1 -- 172.21.15.27:0/574157259 >> v2:172.21.15.45:3300/0 conn(0x7f321c0af9d0 msgr2=0x7f321c0b3e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:47.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:47.458+0000 7f321ad9d640 1 --2- 172.21.15.27:0/574157259 >> v2:172.21.15.45:3300/0 conn(0x7f321c0af9d0 0x7f321c0b3e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:47.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:47.458+0000 7f321ad9d640 1 -- 172.21.15.27:0/574157259 >> v1:172.21.15.45:6789/0 conn(0x7f321c0b4380 legacy=0x7f321c15f2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:47.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:47.458+0000 7f321ad9d640 1 -- 172.21.15.27:0/574157259 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f321c0ceac0 con 0x7f321c15ec30 2024-09-12T03:19:47.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:47.458+0000 7f321ad9d640 1 --2- 172.21.15.27:0/574157259 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f321c15ec30 0x7f321c0b3900 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f320c009870 tx=0x7f320c02a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=74519846e0841915 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:47.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:47.458+0000 7f321a59c640 1 -- 172.21.15.27:0/574157259 <== mon.2 v2:172.21.15.45:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f320c037070 con 0x7f321c15ec30 2024-09-12T03:19:47.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:47.459+0000 7f321a59c640 1 -- 172.21.15.27:0/574157259 <== mon.2 v2:172.21.15.45:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f320c032440 con 0x7f321c15ec30 2024-09-12T03:19:47.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:47.459+0000 7f321a59c640 1 -- 172.21.15.27:0/574157259 <== mon.2 v2:172.21.15.45:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f320c032810 con 0x7f321c15ec30 2024-09-12T03:19:47.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:47.459+0000 7f322100e640 1 -- 172.21.15.27:0/574157259 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f321c15ec30 msgr2=0x7f321c0b3900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:47.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:47.459+0000 7f322100e640 1 --2- 172.21.15.27:0/574157259 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f321c15ec30 0x7f321c0b3900 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f320c009870 tx=0x7f320c02a270 comp rx=0 tx=0).stop 2024-09-12T03:19:47.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:47.459+0000 7f322100e640 1 -- 172.21.15.27:0/574157259 shutdown_connections 2024-09-12T03:19:47.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:47.459+0000 7f322100e640 1 --2- 172.21.15.27:0/574157259 >> v2:172.21.15.45:3300/0 conn(0x7f321c0af9d0 0x7f321c0b3e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:47.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:47.459+0000 7f322100e640 1 --2- 172.21.15.27:0/574157259 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f321c15ec30 0x7f321c0b3900 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:47.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:47.459+0000 7f322100e640 1 -- 172.21.15.27:0/574157259 >> 172.21.15.27:0/574157259 conn(0x7f321c06bb30 msgr2=0x7f321c06bf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:47.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:47.459+0000 7f322100e640 1 -- 172.21.15.27:0/574157259 shutdown_connections 2024-09-12T03:19:47.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:47.459+0000 7f322100e640 1 -- 172.21.15.27:0/574157259 wait complete. 2024-09-12T03:19:47.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:47.460+0000 7f322100e640 1 Processor -- start 2024-09-12T03:19:47.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:47.461+0000 7f322100e640 1 -- start start 2024-09-12T03:19:47.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:47.461+0000 7f322100e640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f321c0af9d0 0x7f321c15dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:47.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:47.461+0000 7f322100e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f321c0b4380 0x7f321c153d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:47.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:47.461+0000 7f322100e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f321c15ec30 0x7f321c1542a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:47.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:47.461+0000 7f322100e640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f321c0cf140 con 0x7f321c0b4380 2024-09-12T03:19:47.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:47.461+0000 7f322100e640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f321c0cefc0 con 0x7f321c15ec30 2024-09-12T03:19:47.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:47.461+0000 7f322100e640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f321c0cf2c0 con 0x7f321c0af9d0 2024-09-12T03:19:47.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:47.461+0000 7f3213fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f321c0b4380 0x7f321c153d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:47.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:47.461+0000 7f3213fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f321c0b4380 0x7f321c153d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58364/0 (socket says 172.21.15.27:58364) 2024-09-12T03:19:47.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:47.461+0000 7f3213fff640 1 -- 172.21.15.27:0/2504183906 learned_addr learned my addr 172.21.15.27:0/2504183906 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:19:47.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:47.462+0000 7f321ad9d640 1 --2- 172.21.15.27:0/2504183906 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f321c0af9d0 0x7f321c15dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:47.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:47.462+0000 7f321b59e640 1 --2- 172.21.15.27:0/2504183906 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f321c15ec30 0x7f321c1542a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:47.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:47.462+0000 7f3213fff640 1 -- 172.21.15.27:0/2504183906 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f321c0af9d0 msgr2=0x7f321c15dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:47.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:47.462+0000 7f3213fff640 1 --2- 172.21.15.27:0/2504183906 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f321c0af9d0 0x7f321c15dee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:47.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:47.462+0000 7f3213fff640 1 -- 172.21.15.27:0/2504183906 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f321c15ec30 msgr2=0x7f321c1542a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:47.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:47.462+0000 7f3213fff640 1 --2- 172.21.15.27:0/2504183906 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f321c15ec30 0x7f321c1542a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:47.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:47.462+0000 7f3213fff640 1 -- 172.21.15.27:0/2504183906 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f321c15c950 con 0x7f321c0b4380 2024-09-12T03:19:47.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:47.462+0000 7f321b59e640 1 --2- 172.21.15.27:0/2504183906 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f321c15ec30 0x7f321c1542a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:19:47.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:47.462+0000 7f3213fff640 1 --2- 172.21.15.27:0/2504183906 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f321c0b4380 0x7f321c153d60 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f320400e980 tx=0x7f320400ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:47.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:47.462+0000 7f32137fe640 1 -- 172.21.15.27:0/2504183906 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3204017070 con 0x7f321c0b4380 2024-09-12T03:19:47.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:47.462+0000 7f32137fe640 1 -- 172.21.15.27:0/2504183906 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f320400ca00 con 0x7f321c0b4380 2024-09-12T03:19:47.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:47.462+0000 7f322100e640 1 -- 172.21.15.27:0/2504183906 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f321c152cc0 con 0x7f321c0b4380 2024-09-12T03:19:47.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:47.462+0000 7f32137fe640 1 -- 172.21.15.27:0/2504183906 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f320400cdd0 con 0x7f321c0b4380 2024-09-12T03:19:47.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:47.462+0000 7f322100e640 1 -- 172.21.15.27:0/2504183906 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f321c153280 con 0x7f321c0b4380 2024-09-12T03:19:47.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:47.463+0000 7f322100e640 1 -- 172.21.15.27:0/2504183906 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f321c0695a0 con 0x7f321c0b4380 2024-09-12T03:19:47.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:47.464+0000 7f32137fe640 1 -- 172.21.15.27:0/2504183906 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 11) ==== 90804+0+0 (secure 0 0 0) 0x7f320401a070 con 0x7f321c0b4380 2024-09-12T03:19:47.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:47.464+0000 7f32137fe640 1 --2- 172.21.15.27:0/2504183906 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f31fc06c190 0x7f31fc06e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:47.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:47.464+0000 7f32137fe640 1 -- 172.21.15.27:0/2504183906 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5795+0+0 (secure 0 0 0) 0x7f320408b850 con 0x7f321c0b4380 2024-09-12T03:19:47.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:47.464+0000 7f321ad9d640 1 --2- 172.21.15.27:0/2504183906 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f31fc06c190 0x7f31fc06e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:47.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:47.465+0000 7f321ad9d640 1 --2- 172.21.15.27:0/2504183906 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f31fc06c190 0x7f31fc06e650 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f320c034000 tx=0x7f320c02e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:47.468 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:47.469+0000 7f32137fe640 1 -- 172.21.15.27:0/2504183906 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f3204059d10 con 0x7f321c0b4380 2024-09-12T03:19:47.681 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:47.681+0000 7f322100e640 1 -- 172.21.15.27:0/2504183906 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f321c15f010 con 0x7f321c0b4380 2024-09-12T03:19:48.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:48.341+0000 7f32137fe640 1 -- 172.21.15.27:0/2504183906 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v33) ==== 104+0+0 (secure 0 0 0) 0x7f3204059610 con 0x7f321c0b4380 2024-09-12T03:19:48.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:48.343+0000 7f322100e640 1 -- 172.21.15.27:0/2504183906 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f31fc06c190 msgr2=0x7f31fc06e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:48.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:48.343+0000 7f322100e640 1 --2- 172.21.15.27:0/2504183906 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f31fc06c190 0x7f31fc06e650 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f320c034000 tx=0x7f320c02e000 comp rx=0 tx=0).stop 2024-09-12T03:19:48.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:48.343+0000 7f322100e640 1 -- 172.21.15.27:0/2504183906 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f321c0b4380 msgr2=0x7f321c153d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:48.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:48.343+0000 7f322100e640 1 --2- 172.21.15.27:0/2504183906 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f321c0b4380 0x7f321c153d60 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f320400e980 tx=0x7f320400ee40 comp rx=0 tx=0).stop 2024-09-12T03:19:48.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:48.344+0000 7f322100e640 1 -- 172.21.15.27:0/2504183906 shutdown_connections 2024-09-12T03:19:48.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:48.344+0000 7f322100e640 1 --2- 172.21.15.27:0/2504183906 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f31fc06c190 0x7f31fc06e650 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:48.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:48.344+0000 7f322100e640 1 --2- 172.21.15.27:0/2504183906 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f321c15ec30 0x7f321c1542a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:48.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:48.344+0000 7f322100e640 1 --2- 172.21.15.27:0/2504183906 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f321c0b4380 0x7f321c153d60 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:48.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:48.344+0000 7f322100e640 1 --2- 172.21.15.27:0/2504183906 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f321c0af9d0 0x7f321c15dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:48.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:48.344+0000 7f322100e640 1 -- 172.21.15.27:0/2504183906 >> 172.21.15.27:0/2504183906 conn(0x7f321c06bb30 msgr2=0x7f321c15f230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:48.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:48.344+0000 7f322100e640 1 -- 172.21.15.27:0/2504183906 shutdown_connections 2024-09-12T03:19:48.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:48.344+0000 7f322100e640 1 -- 172.21.15.27:0/2504183906 wait complete. 2024-09-12T03:19:48.357 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-12T03:19:48.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:48.467+0000 7f7e76653640 1 Processor -- start 2024-09-12T03:19:48.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:48.467+0000 7f7e76653640 1 -- start start 2024-09-12T03:19:48.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:48.468+0000 7f7e76653640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f7e700b3d10 0x7f7e700b66c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:48.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:48.468+0000 7f7e76653640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7e7015ed70 con 0x7f7e700b6230 2024-09-12T03:19:48.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:48.468+0000 7f7e76653640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7e700b7d10 con 0x7f7e7015e990 2024-09-12T03:19:48.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:48.468+0000 7f7e76653640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f7e700d2a30 con 0x7f7e700b3d10 2024-09-12T03:19:48.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:48.468+0000 7f7e74bf0640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f7e700b3d10 0x7f7e700b66c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:48.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:48.468+0000 7f7e74bf0640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f7e700b3d10 0x7f7e700b66c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58368/0 (socket says 172.21.15.27:58368) 2024-09-12T03:19:48.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:48.468+0000 7f7e74bf0640 1 -- 172.21.15.27:0/660331188 learned_addr learned my addr 172.21.15.27:0/660331188 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:19:48.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:48.468+0000 7f7e6f7fe640 1 --1- 172.21.15.27:0/660331188 >> v1:172.21.15.27:6789/0 conn(0x7f7e700b6230 0x7f7e700b3600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:46082/0 (socket says 172.21.15.27:46082) 2024-09-12T03:19:48.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:48.469+0000 7f7e74bf0640 1 -- 172.21.15.27:0/660331188 >> v1:172.21.15.45:6789/0 conn(0x7f7e7015e990 legacy=0x7f7e700b7600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:48.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:48.469+0000 7f7e74bf0640 1 -- 172.21.15.27:0/660331188 >> v1:172.21.15.27:6789/0 conn(0x7f7e700b6230 legacy=0x7f7e700b3600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:48.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:48.469+0000 7f7e6effd640 1 -- 172.21.15.27:0/660331188 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2974938114 0 0) 0x7f7e7015ed70 con 0x7f7e700b6230 2024-09-12T03:19:48.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:48.469+0000 7f7e74bf0640 1 -- 172.21.15.27:0/660331188 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e700d2d10 con 0x7f7e700b3d10 2024-09-12T03:19:48.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:48.469+0000 7f7e74bf0640 1 --2- 172.21.15.27:0/660331188 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7e700b3d10 0x7f7e700b66c0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f7e68009e50 tx=0x7f7e6802a760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1ece93769d6385d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:48.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:48.469+0000 7f7e6effd640 1 -- 172.21.15.27:0/660331188 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e68036070 con 0x7f7e700b3d10 2024-09-12T03:19:48.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:48.470+0000 7f7e6effd640 1 -- 172.21.15.27:0/660331188 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e68031600 con 0x7f7e700b3d10 2024-09-12T03:19:48.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:48.470+0000 7f7e6effd640 1 -- 172.21.15.27:0/660331188 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e680319d0 con 0x7f7e700b3d10 2024-09-12T03:19:48.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:48.470+0000 7f7e76653640 1 -- 172.21.15.27:0/660331188 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7e700b3d10 msgr2=0x7f7e700b66c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:48.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:48.470+0000 7f7e76653640 1 --2- 172.21.15.27:0/660331188 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7e700b3d10 0x7f7e700b66c0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f7e68009e50 tx=0x7f7e6802a760 comp rx=0 tx=0).stop 2024-09-12T03:19:48.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:48.470+0000 7f7e76653640 1 -- 172.21.15.27:0/660331188 shutdown_connections 2024-09-12T03:19:48.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:48.470+0000 7f7e76653640 1 --2- 172.21.15.27:0/660331188 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7e700b3d10 0x7f7e700b66c0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:48.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:48.470+0000 7f7e76653640 1 -- 172.21.15.27:0/660331188 >> 172.21.15.27:0/660331188 conn(0x7f7e700d1230 msgr2=0x7f7e700d1640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:48.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:48.470+0000 7f7e76653640 1 -- 172.21.15.27:0/660331188 shutdown_connections 2024-09-12T03:19:48.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:48.470+0000 7f7e76653640 1 -- 172.21.15.27:0/660331188 wait complete. 2024-09-12T03:19:48.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:48.471+0000 7f7e76653640 1 Processor -- start 2024-09-12T03:19:48.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:48.472+0000 7f7e76653640 1 -- start start 2024-09-12T03:19:48.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:48.472+0000 7f7e76653640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7e700b3d10 0x7f7e70159380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:48.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:48.472+0000 7f7e76653640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f7e700b6230 0x7f7e700cd550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:48.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:48.472+0000 7f7e76653640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7e7015e990 0x7f7e700c8640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:48.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:48.472+0000 7f7e76653640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f7e700d35b0 con 0x7f7e700b3d10 2024-09-12T03:19:48.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:48.472+0000 7f7e76653640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f7e700d32d0 con 0x7f7e7015e990 2024-09-12T03:19:48.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:48.472+0000 7f7e76653640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f7e700d3ad0 con 0x7f7e700b6230 2024-09-12T03:19:48.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:48.472+0000 7f7e6ffff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7e700b3d10 0x7f7e70159380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:48.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:48.472+0000 7f7e6ffff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7e700b3d10 0x7f7e70159380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58380/0 (socket says 172.21.15.27:58380) 2024-09-12T03:19:48.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:48.472+0000 7f7e6ffff640 1 -- 172.21.15.27:0/3286803451 learned_addr learned my addr 172.21.15.27:0/3286803451 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:19:48.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:48.473+0000 7f7e74bf0640 1 --2- 172.21.15.27:0/3286803451 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7e7015e990 0x7f7e700c8640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:48.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:48.473+0000 7f7e6f7fe640 1 --2- 172.21.15.27:0/3286803451 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f7e700b6230 0x7f7e700cd550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:48.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:48.473+0000 7f7e6ffff640 1 -- 172.21.15.27:0/3286803451 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f7e700b6230 msgr2=0x7f7e700cd550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:48.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:48.473+0000 7f7e6ffff640 1 --2- 172.21.15.27:0/3286803451 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f7e700b6230 0x7f7e700cd550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:48.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:48.473+0000 7f7e6ffff640 1 -- 172.21.15.27:0/3286803451 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7e7015e990 msgr2=0x7f7e700c8640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:48.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:48.473+0000 7f7e6ffff640 1 --2- 172.21.15.27:0/3286803451 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7e7015e990 0x7f7e700c8640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:48.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:48.473+0000 7f7e6ffff640 1 -- 172.21.15.27:0/3286803451 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e700ce950 con 0x7f7e700b3d10 2024-09-12T03:19:48.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:48.473+0000 7f7e6ffff640 1 --2- 172.21.15.27:0/3286803451 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7e700b3d10 0x7f7e70159380 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f7e6000b3f0 tx=0x7f7e6000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:48.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:48.473+0000 7f7e6cff9640 1 -- 172.21.15.27:0/3286803451 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e60015020 con 0x7f7e700b3d10 2024-09-12T03:19:48.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:48.473+0000 7f7e6cff9640 1 -- 172.21.15.27:0/3286803451 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e60011830 con 0x7f7e700b3d10 2024-09-12T03:19:48.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:48.473+0000 7f7e76653640 1 -- 172.21.15.27:0/3286803451 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e700cdc10 con 0x7f7e700b3d10 2024-09-12T03:19:48.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:48.473+0000 7f7e6cff9640 1 -- 172.21.15.27:0/3286803451 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e60011c20 con 0x7f7e700b3d10 2024-09-12T03:19:48.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:48.473+0000 7f7e76653640 1 -- 172.21.15.27:0/3286803451 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e7015f1f0 con 0x7f7e700b3d10 2024-09-12T03:19:48.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:48.474+0000 7f7e76653640 1 -- 172.21.15.27:0/3286803451 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e700b66c0 con 0x7f7e700b3d10 2024-09-12T03:19:48.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:48.474+0000 7f7e6cff9640 1 -- 172.21.15.27:0/3286803451 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 11) ==== 90804+0+0 (secure 0 0 0) 0x7f7e60018020 con 0x7f7e700b3d10 2024-09-12T03:19:48.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:48.475+0000 7f7e6cff9640 1 --2- 172.21.15.27:0/3286803451 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f7e5c06c1c0 0x7f7e5c06e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:48.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:48.475+0000 7f7e6cff9640 1 -- 172.21.15.27:0/3286803451 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5795+0+0 (secure 0 0 0) 0x7f7e6008af70 con 0x7f7e700b3d10 2024-09-12T03:19:48.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:48.475+0000 7f7e6f7fe640 1 --2- 172.21.15.27:0/3286803451 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f7e5c06c1c0 0x7f7e5c06e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:48.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:48.476+0000 7f7e6f7fe640 1 --2- 172.21.15.27:0/3286803451 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f7e5c06c1c0 0x7f7e5c06e680 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f7e700cb4d0 tx=0x7f7e64007b50 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:48.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:48.479+0000 7f7e6cff9640 1 -- 172.21.15.27:0/3286803451 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f7e6001f030 con 0x7f7e700b3d10 2024-09-12T03:19:48.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:48.653+0000 7f7e76653640 1 -- 172.21.15.27:0/3286803451 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f7e70069f10 con 0x7f7e700b3d10 2024-09-12T03:19:49.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:49.343+0000 7f7e6cff9640 1 -- 172.21.15.27:0/3286803451 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v58) ==== 181+0+0 (secure 0 0 0) 0x7f7e600172c0 con 0x7f7e700b3d10 2024-09-12T03:19:49.343 INFO:teuthology.orchestra.run.smithi027.stderr:pool 'cephfs_metadata' removed 2024-09-12T03:19:49.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:49.346+0000 7f7e76653640 1 -- 172.21.15.27:0/3286803451 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f7e5c06c1c0 msgr2=0x7f7e5c06e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:49.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:49.346+0000 7f7e76653640 1 --2- 172.21.15.27:0/3286803451 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f7e5c06c1c0 0x7f7e5c06e680 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f7e700cb4d0 tx=0x7f7e64007b50 comp rx=0 tx=0).stop 2024-09-12T03:19:49.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:49.346+0000 7f7e76653640 1 -- 172.21.15.27:0/3286803451 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7e700b3d10 msgr2=0x7f7e70159380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:49.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:49.346+0000 7f7e76653640 1 --2- 172.21.15.27:0/3286803451 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7e700b3d10 0x7f7e70159380 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f7e6000b3f0 tx=0x7f7e6000b8b0 comp rx=0 tx=0).stop 2024-09-12T03:19:49.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:49.346+0000 7f7e76653640 1 -- 172.21.15.27:0/3286803451 shutdown_connections 2024-09-12T03:19:49.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:49.346+0000 7f7e76653640 1 --2- 172.21.15.27:0/3286803451 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f7e5c06c1c0 0x7f7e5c06e680 secure :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f7e700cb4d0 tx=0x7f7e64007b50 comp rx=0 tx=0).stop 2024-09-12T03:19:49.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:49.346+0000 7f7e76653640 1 --2- 172.21.15.27:0/3286803451 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7e7015e990 0x7f7e700c8640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:49.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:49.346+0000 7f7e76653640 1 --2- 172.21.15.27:0/3286803451 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f7e700b6230 0x7f7e700cd550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:49.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:49.346+0000 7f7e76653640 1 --2- 172.21.15.27:0/3286803451 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7e700b3d10 0x7f7e70159380 secure :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f7e6000b3f0 tx=0x7f7e6000b8b0 comp rx=0 tx=0).stop 2024-09-12T03:19:49.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:49.346+0000 7f7e76653640 1 -- 172.21.15.27:0/3286803451 >> 172.21.15.27:0/3286803451 conn(0x7f7e700d1230 msgr2=0x7f7e700d1610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:49.348 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:49.347+0000 7f7e76653640 1 -- 172.21.15.27:0/3286803451 shutdown_connections 2024-09-12T03:19:49.348 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:49.347+0000 7f7e76653640 1 -- 172.21.15.27:0/3286803451 wait complete. 2024-09-12T03:19:49.361 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-12T03:19:49.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:49.472+0000 7fd95e34b640 1 Processor -- start 2024-09-12T03:19:49.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:49.472+0000 7fd95e34b640 1 -- start start 2024-09-12T03:19:49.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:49.473+0000 7fd95e34b640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fd9580cf320 0x7fd9580b4000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:49.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:49.473+0000 7fd95e34b640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd95806cff0 con 0x7fd95806c650 2024-09-12T03:19:49.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:49.473+0000 7fd95e34b640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd95806d2d0 con 0x7fd9580cef40 2024-09-12T03:19:49.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:49.473+0000 7fd95e34b640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fd95806dee0 con 0x7fd9580cf320 2024-09-12T03:19:49.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:49.473+0000 7fd9577fe640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fd95806c650 0x7fd95815f0c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:53056/0 (socket says 172.21.15.27:53056) 2024-09-12T03:19:49.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:49.473+0000 7fd9577fe640 1 -- 172.21.15.27:0/857616167 learned_addr learned my addr 172.21.15.27:0/857616167 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:19:49.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:49.473+0000 7fd95c8e8640 1 --2- 172.21.15.27:0/857616167 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fd9580cf320 0x7fd9580b4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:49.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:49.473+0000 7fd956ffd640 1 -- 172.21.15.27:0/857616167 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2540416815 0 0) 0x7fd95806cff0 con 0x7fd95806c650 2024-09-12T03:19:49.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:49.473+0000 7fd956ffd640 1 -- 172.21.15.27:0/857616167 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd940003420 con 0x7fd95806c650 2024-09-12T03:19:49.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:49.473+0000 7fd956ffd640 1 -- 172.21.15.27:0/857616167 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 321419800 0 0) 0x7fd940003420 con 0x7fd95806c650 2024-09-12T03:19:49.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:49.473+0000 7fd956ffd640 1 -- 172.21.15.27:0/857616167 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fd9580cf320 msgr2=0x7fd9580b4000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:49.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:49.473+0000 7fd956ffd640 1 --2- 172.21.15.27:0/857616167 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fd9580cf320 0x7fd9580b4000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:49.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:49.473+0000 7fd956ffd640 1 -- 172.21.15.27:0/857616167 >> v1:172.21.15.45:6789/0 conn(0x7fd9580cef40 legacy=0x7fd9580710c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:49.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:49.473+0000 7fd956ffd640 1 -- 172.21.15.27:0/857616167 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd95806d5b0 con 0x7fd95806c650 2024-09-12T03:19:49.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:49.473+0000 7fd956ffd640 1 -- 172.21.15.27:0/857616167 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 847755213 0 0) 0x7fd94c001670 con 0x7fd95806c650 2024-09-12T03:19:49.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:49.473+0000 7fd956ffd640 1 -- 172.21.15.27:0/857616167 >> v1:172.21.15.27:6789/0 conn(0x7fd95806c650 legacy=0x7fd95815f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:49.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:49.473+0000 7fd956ffd640 1 --2- 172.21.15.27:0/857616167 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd940004ac0 0x7fd940004ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:49.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:49.473+0000 7fd956ffd640 1 --2- 172.21.15.27:0/857616167 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fd9580cf320 0x7fd9400036a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:49.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:49.473+0000 7fd956ffd640 1 --2- 172.21.15.27:0/857616167 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd940003d00 0x7fd94000fb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:49.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:49.473+0000 7fd956ffd640 1 -- 172.21.15.27:0/857616167 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fd95806dee0 con 0x7fd940004ac0 2024-09-12T03:19:49.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:49.473+0000 7fd956ffd640 1 -- 172.21.15.27:0/857616167 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fd9400044f0 con 0x7fd940003d00 2024-09-12T03:19:49.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:49.473+0000 7fd956ffd640 1 -- 172.21.15.27:0/857616167 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fd9400100b0 con 0x7fd9580cf320 2024-09-12T03:19:49.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:49.474+0000 7fd957fff640 1 --2- 172.21.15.27:0/857616167 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd940004ac0 0x7fd940004ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:49.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:49.474+0000 7fd9577fe640 1 --2- 172.21.15.27:0/857616167 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fd9580cf320 0x7fd9400036a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:49.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:49.474+0000 7fd95c8e8640 1 --2- 172.21.15.27:0/857616167 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd940003d00 0x7fd94000fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:49.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:49.474+0000 7fd957fff640 1 -- 172.21.15.27:0/857616167 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fd9580cf320 msgr2=0x7fd9400036a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:49.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:49.474+0000 7fd957fff640 1 --2- 172.21.15.27:0/857616167 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fd9580cf320 0x7fd9400036a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:49.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:49.474+0000 7fd957fff640 1 -- 172.21.15.27:0/857616167 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd940003d00 msgr2=0x7fd94000fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:49.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:49.474+0000 7fd957fff640 1 --2- 172.21.15.27:0/857616167 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd940003d00 0x7fd94000fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:49.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:49.474+0000 7fd957fff640 1 -- 172.21.15.27:0/857616167 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd940003420 con 0x7fd940004ac0 2024-09-12T03:19:49.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:49.474+0000 7fd95c8e8640 1 --2- 172.21.15.27:0/857616167 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd940003d00 0x7fd94000fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:19:49.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:49.475+0000 7fd957fff640 1 --2- 172.21.15.27:0/857616167 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd940004ac0 0x7fd940004ea0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fd948001890 tx=0x7fd940004720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f680a9cc181847f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:49.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:49.475+0000 7fd956ffd640 1 -- 172.21.15.27:0/857616167 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd948037070 con 0x7fd940004ac0 2024-09-12T03:19:49.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:49.475+0000 7fd956ffd640 1 -- 172.21.15.27:0/857616167 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd94802ac00 con 0x7fd940004ac0 2024-09-12T03:19:49.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:49.475+0000 7fd956ffd640 1 -- 172.21.15.27:0/857616167 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd948001430 con 0x7fd940004ac0 2024-09-12T03:19:49.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:49.475+0000 7fd95e34b640 1 -- 172.21.15.27:0/857616167 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd940004ac0 msgr2=0x7fd940004ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:49.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:49.475+0000 7fd95e34b640 1 --2- 172.21.15.27:0/857616167 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd940004ac0 0x7fd940004ea0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fd948001890 tx=0x7fd940004720 comp rx=0 tx=0).stop 2024-09-12T03:19:49.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:49.475+0000 7fd95c8e8640 1 -- 172.21.15.27:0/857616167 reap_dead start 2024-09-12T03:19:49.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:49.475+0000 7fd95e34b640 1 -- 172.21.15.27:0/857616167 shutdown_connections 2024-09-12T03:19:49.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:49.475+0000 7fd95e34b640 1 -- 172.21.15.27:0/857616167 >> 172.21.15.27:0/857616167 conn(0x7fd95806bb30 msgr2=0x7fd95806bf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:49.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:49.475+0000 7fd95e34b640 1 -- 172.21.15.27:0/857616167 shutdown_connections 2024-09-12T03:19:49.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:49.475+0000 7fd95e34b640 1 -- 172.21.15.27:0/857616167 wait complete. 2024-09-12T03:19:49.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:49.477+0000 7fd95e34b640 1 Processor -- start 2024-09-12T03:19:49.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:49.477+0000 7fd95e34b640 1 -- start start 2024-09-12T03:19:49.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:49.477+0000 7fd95e34b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd940004ac0 0x7fd958149720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:49.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:49.477+0000 7fd95e34b640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fd9580b0520 0x7fd958146010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:49.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:49.477+0000 7fd95e34b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd9580b09a0 0x7fd958146550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:49.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:49.478+0000 7fd95e34b640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fd9580b4000 con 0x7fd9580b09a0 2024-09-12T03:19:49.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:49.478+0000 7fd95e34b640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fd95806db70 con 0x7fd940004ac0 2024-09-12T03:19:49.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:49.478+0000 7fd95e34b640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fd9580b4180 con 0x7fd9580b0520 2024-09-12T03:19:49.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:49.478+0000 7fd95c8e8640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd9580b09a0 0x7fd958146550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:49.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:49.478+0000 7fd95c8e8640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd9580b09a0 0x7fd958146550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35360/0 (socket says 172.21.15.27:35360) 2024-09-12T03:19:49.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:49.478+0000 7fd95c8e8640 1 -- 172.21.15.27:0/779182741 learned_addr learned my addr 172.21.15.27:0/779182741 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:19:49.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:49.478+0000 7fd9577fe640 1 --2- 172.21.15.27:0/779182741 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fd9580b0520 0x7fd958146010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:49.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:49.478+0000 7fd957fff640 1 --2- 172.21.15.27:0/779182741 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd940004ac0 0x7fd958149720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:49.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:49.478+0000 7fd95c8e8640 1 -- 172.21.15.27:0/779182741 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fd9580b0520 msgr2=0x7fd958146010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:49.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:49.478+0000 7fd95c8e8640 1 --2- 172.21.15.27:0/779182741 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fd9580b0520 0x7fd958146010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:49.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:49.478+0000 7fd95c8e8640 1 -- 172.21.15.27:0/779182741 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd940004ac0 msgr2=0x7fd958149720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:49.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:49.478+0000 7fd95c8e8640 1 --2- 172.21.15.27:0/779182741 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd940004ac0 0x7fd958149720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:49.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:49.478+0000 7fd95c8e8640 1 -- 172.21.15.27:0/779182741 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd958149c60 con 0x7fd9580b09a0 2024-09-12T03:19:49.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:49.478+0000 7fd95c8e8640 1 --2- 172.21.15.27:0/779182741 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd9580b09a0 0x7fd958146550 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fd9580ce290 tx=0x7fd950008e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:49.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:49.478+0000 7fd954ff9640 1 -- 172.21.15.27:0/779182741 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd950007cb0 con 0x7fd9580b09a0 2024-09-12T03:19:49.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:49.478+0000 7fd954ff9640 1 -- 172.21.15.27:0/779182741 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd95001a070 con 0x7fd9580b09a0 2024-09-12T03:19:49.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:49.478+0000 7fd95e34b640 1 -- 172.21.15.27:0/779182741 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9580aa8a0 con 0x7fd9580b09a0 2024-09-12T03:19:49.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:49.478+0000 7fd954ff9640 1 -- 172.21.15.27:0/779182741 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9500156b0 con 0x7fd9580b09a0 2024-09-12T03:19:49.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:49.478+0000 7fd95e34b640 1 -- 172.21.15.27:0/779182741 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9580aae60 con 0x7fd9580b09a0 2024-09-12T03:19:49.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:49.479+0000 7fd95e34b640 1 -- 172.21.15.27:0/779182741 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd95806d2d0 con 0x7fd9580b09a0 2024-09-12T03:19:49.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:49.480+0000 7fd954ff9640 1 -- 172.21.15.27:0/779182741 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 11) ==== 90804+0+0 (secure 0 0 0) 0x7fd950005d10 con 0x7fd9580b09a0 2024-09-12T03:19:49.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:49.480+0000 7fd954ff9640 1 --2- 172.21.15.27:0/779182741 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fd94406c1c0 0x7fd94406e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:49.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:49.480+0000 7fd954ff9640 1 -- 172.21.15.27:0/779182741 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5357+0+0 (secure 0 0 0) 0x7fd95008b690 con 0x7fd9580b09a0 2024-09-12T03:19:49.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:49.481+0000 7fd957fff640 1 --2- 172.21.15.27:0/779182741 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fd94406c1c0 0x7fd94406e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:49.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:49.481+0000 7fd957fff640 1 --2- 172.21.15.27:0/779182741 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fd94406c1c0 0x7fd94406e680 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fd948034000 tx=0x7fd948009030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:49.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:49.484+0000 7fd954ff9640 1 -- 172.21.15.27:0/779182741 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fd958149c60 con 0x7fd9580b09a0 2024-09-12T03:19:49.665 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:49.665+0000 7fd95e34b640 1 -- 172.21.15.27:0/779182741 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fd958146c80 con 0x7fd9580b09a0 2024-09-12T03:19:50.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:50.350+0000 7fd954ff9640 1 -- 172.21.15.27:0/779182741 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v59) ==== 169+0+0 (secure 0 0 0) 0x7fd950011040 con 0x7fd9580b09a0 2024-09-12T03:19:50.351 INFO:teuthology.orchestra.run.smithi027.stderr:pool 'cephfs_data' removed 2024-09-12T03:19:50.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:50.353+0000 7fd95e34b640 1 -- 172.21.15.27:0/779182741 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fd94406c1c0 msgr2=0x7fd94406e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:50.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:50.353+0000 7fd95e34b640 1 --2- 172.21.15.27:0/779182741 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fd94406c1c0 0x7fd94406e680 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fd948034000 tx=0x7fd948009030 comp rx=0 tx=0).stop 2024-09-12T03:19:50.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:50.353+0000 7fd95e34b640 1 -- 172.21.15.27:0/779182741 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd9580b09a0 msgr2=0x7fd958146550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:50.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:50.353+0000 7fd95e34b640 1 --2- 172.21.15.27:0/779182741 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd9580b09a0 0x7fd958146550 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fd9580ce290 tx=0x7fd950008e70 comp rx=0 tx=0).stop 2024-09-12T03:19:50.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:50.353+0000 7fd95e34b640 1 -- 172.21.15.27:0/779182741 shutdown_connections 2024-09-12T03:19:50.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:50.353+0000 7fd95e34b640 1 --2- 172.21.15.27:0/779182741 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fd94406c1c0 0x7fd94406e680 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:50.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:50.353+0000 7fd95e34b640 1 --2- 172.21.15.27:0/779182741 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd9580b09a0 0x7fd958146550 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:50.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:50.353+0000 7fd95e34b640 1 --2- 172.21.15.27:0/779182741 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fd9580b0520 0x7fd958146010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:50.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:50.353+0000 7fd95e34b640 1 --2- 172.21.15.27:0/779182741 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd940004ac0 0x7fd958149720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:50.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:50.353+0000 7fd95e34b640 1 -- 172.21.15.27:0/779182741 >> 172.21.15.27:0/779182741 conn(0x7fd95806bb30 msgr2=0x7fd958157ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:50.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:50.353+0000 7fd95e34b640 1 -- 172.21.15.27:0/779182741 shutdown_connections 2024-09-12T03:19:50.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:50.353+0000 7fd95e34b640 1 -- 172.21.15.27:0/779182741 wait complete. 2024-09-12T03:19:50.368 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-09-12T03:19:50.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:50.478+0000 7fb78d5af640 1 Processor -- start 2024-09-12T03:19:50.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:50.478+0000 7fb78d5af640 1 -- start start 2024-09-12T03:19:50.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:50.479+0000 7fb78d5af640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fb788162040 0x7fb7880d1da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:50.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:50.479+0000 7fb78d5af640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb7880d3790 0x7fb7880b7a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:50.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:50.479+0000 7fb78d5af640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb7880d2e30 con 0x7fb78806d8c0 2024-09-12T03:19:50.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:50.479+0000 7fb78d5af640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fb7880b7f90 con 0x7fb788162040 2024-09-12T03:19:50.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:50.479+0000 7fb78d5af640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fb788162420 con 0x7fb7880d3790 2024-09-12T03:19:50.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:50.479+0000 7fb7877fe640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fb78806d8c0 0x7fb7880cf320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:53062/0 (socket says 172.21.15.27:53062) 2024-09-12T03:19:50.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:50.479+0000 7fb7877fe640 1 -- 172.21.15.27:0/4068604884 learned_addr learned my addr 172.21.15.27:0/4068604884 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:19:50.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:50.480+0000 7fb7867fc640 1 --2- 172.21.15.27:0/4068604884 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb7880d3790 0x7fb7880b7a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:50.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:50.480+0000 7fb786ffd640 1 --2- 172.21.15.27:0/4068604884 >> v2:172.21.15.45:3300/0 conn(0x7fb788162040 0x7fb7880d1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:50.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:50.480+0000 7fb785ffb640 1 -- 172.21.15.27:0/4068604884 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3648615529 0 0) 0x7fb7880d2e30 con 0x7fb78806d8c0 2024-09-12T03:19:50.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:50.480+0000 7fb785ffb640 1 -- 172.21.15.27:0/4068604884 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb774003520 con 0x7fb78806d8c0 2024-09-12T03:19:50.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:50.480+0000 7fb785ffb640 1 -- 172.21.15.27:0/4068604884 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 458321028 0 0) 0x7fb774003520 con 0x7fb78806d8c0 2024-09-12T03:19:50.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:50.480+0000 7fb785ffb640 1 -- 172.21.15.27:0/4068604884 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb7880d3790 msgr2=0x7fb7880b7a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:50.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:50.480+0000 7fb785ffb640 1 --2- 172.21.15.27:0/4068604884 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb7880d3790 0x7fb7880b7a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:50.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:50.480+0000 7fb785ffb640 1 -- 172.21.15.27:0/4068604884 >> v2:172.21.15.45:3300/0 conn(0x7fb788162040 msgr2=0x7fb7880d1da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:50.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:50.480+0000 7fb785ffb640 1 --2- 172.21.15.27:0/4068604884 >> v2:172.21.15.45:3300/0 conn(0x7fb788162040 0x7fb7880d1da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:50.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:50.480+0000 7fb785ffb640 1 -- 172.21.15.27:0/4068604884 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7880d30d0 con 0x7fb78806d8c0 2024-09-12T03:19:50.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:50.480+0000 7fb785ffb640 1 -- 172.21.15.27:0/4068604884 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 847755213 0 0) 0x7fb780001670 con 0x7fb78806d8c0 2024-09-12T03:19:50.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:50.480+0000 7fb785ffb640 1 -- 172.21.15.27:0/4068604884 >> v1:172.21.15.27:6789/0 conn(0x7fb78806d8c0 legacy=0x7fb7880cf320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:50.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:50.480+0000 7fb785ffb640 1 --2- 172.21.15.27:0/4068604884 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb7740039b0 0x7fb774003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:50.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:50.480+0000 7fb785ffb640 1 --2- 172.21.15.27:0/4068604884 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb7880d3790 0x7fb774004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:50.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:50.480+0000 7fb785ffb640 1 --2- 172.21.15.27:0/4068604884 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb774004d40 0x7fb7740051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:50.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:50.480+0000 7fb785ffb640 1 -- 172.21.15.27:0/4068604884 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fb7880b7f90 con 0x7fb774004d40 2024-09-12T03:19:50.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:50.480+0000 7fb785ffb640 1 -- 172.21.15.27:0/4068604884 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb788162420 con 0x7fb7740039b0 2024-09-12T03:19:50.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:50.480+0000 7fb785ffb640 1 -- 172.21.15.27:0/4068604884 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fb77400b9c0 con 0x7fb7880d3790 2024-09-12T03:19:50.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:50.480+0000 7fb7877fe640 1 --2- 172.21.15.27:0/4068604884 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb774004d40 0x7fb7740051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:50.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:50.480+0000 7fb786ffd640 1 --2- 172.21.15.27:0/4068604884 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb7740039b0 0x7fb774003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:50.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:50.480+0000 7fb7867fc640 1 --2- 172.21.15.27:0/4068604884 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb7880d3790 0x7fb774004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:50.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:50.480+0000 7fb7877fe640 1 -- 172.21.15.27:0/4068604884 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb7880d3790 msgr2=0x7fb774004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:50.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:50.480+0000 7fb7877fe640 1 --2- 172.21.15.27:0/4068604884 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb7880d3790 0x7fb774004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:50.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:50.480+0000 7fb7877fe640 1 -- 172.21.15.27:0/4068604884 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb7740039b0 msgr2=0x7fb774003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:50.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:50.480+0000 7fb7877fe640 1 --2- 172.21.15.27:0/4068604884 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb7740039b0 0x7fb774003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:50.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:50.481+0000 7fb7877fe640 1 -- 172.21.15.27:0/4068604884 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb774003520 con 0x7fb774004d40 2024-09-12T03:19:50.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:50.481+0000 7fb7867fc640 1 --2- 172.21.15.27:0/4068604884 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb7880d3790 0x7fb774004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:19:50.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:50.481+0000 7fb7877fe640 1 --2- 172.21.15.27:0/4068604884 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb774004d40 0x7fb7740051c0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fb780001050 tx=0x7fb774005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3fd6e8796d3ff12 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:50.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:50.481+0000 7fb785ffb640 1 -- 172.21.15.27:0/4068604884 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb78002e070 con 0x7fb774004d40 2024-09-12T03:19:50.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:50.481+0000 7fb785ffb640 1 -- 172.21.15.27:0/4068604884 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb780005af0 con 0x7fb774004d40 2024-09-12T03:19:50.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:50.481+0000 7fb785ffb640 1 -- 172.21.15.27:0/4068604884 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb780007d70 con 0x7fb774004d40 2024-09-12T03:19:50.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:50.482+0000 7fb78d5af640 1 -- 172.21.15.27:0/4068604884 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb774004d40 msgr2=0x7fb7740051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:50.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:50.482+0000 7fb78d5af640 1 --2- 172.21.15.27:0/4068604884 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb774004d40 0x7fb7740051c0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fb780001050 tx=0x7fb774005830 comp rx=0 tx=0).stop 2024-09-12T03:19:50.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:50.482+0000 7fb7877fe640 1 -- 172.21.15.27:0/4068604884 reap_dead start 2024-09-12T03:19:50.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:50.482+0000 7fb78d5af640 1 -- 172.21.15.27:0/4068604884 shutdown_connections 2024-09-12T03:19:50.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:50.482+0000 7fb78d5af640 1 -- 172.21.15.27:0/4068604884 >> 172.21.15.27:0/4068604884 conn(0x7fb7880b6700 msgr2=0x7fb7880b6b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:50.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:50.482+0000 7fb78d5af640 1 -- 172.21.15.27:0/4068604884 shutdown_connections 2024-09-12T03:19:50.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:50.482+0000 7fb78d5af640 1 -- 172.21.15.27:0/4068604884 wait complete. 2024-09-12T03:19:50.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:50.483+0000 7fb78d5af640 1 Processor -- start 2024-09-12T03:19:50.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:50.483+0000 7fb78d5af640 1 -- start start 2024-09-12T03:19:50.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:50.484+0000 7fb78d5af640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb774004d40 0x7fb78815b590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:50.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:50.484+0000 7fb78d5af640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb78815bd90 0x7fb7881e7360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:50.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:50.484+0000 7fb78d5af640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb7881e9860 0x7fb7881e9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:50.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:50.484+0000 7fb78d5af640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fb7880d4530 con 0x7fb7881e9860 2024-09-12T03:19:50.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:50.484+0000 7fb78d5af640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb7880cee00 con 0x7fb78815bd90 2024-09-12T03:19:50.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:50.484+0000 7fb78d5af640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fb7880d4150 con 0x7fb774004d40 2024-09-12T03:19:50.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:50.484+0000 7fb7877fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb7881e9860 0x7fb7881e9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:50.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:50.484+0000 7fb7877fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb7881e9860 0x7fb7881e9cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35382/0 (socket says 172.21.15.27:35382) 2024-09-12T03:19:50.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:50.484+0000 7fb7877fe640 1 -- 172.21.15.27:0/411977401 learned_addr learned my addr 172.21.15.27:0/411977401 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:19:50.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:50.484+0000 7fb786ffd640 1 --2- 172.21.15.27:0/411977401 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb774004d40 0x7fb78815b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:50.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:50.484+0000 7fb7867fc640 1 --2- 172.21.15.27:0/411977401 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb78815bd90 0x7fb7881e7360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:50.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:50.485+0000 7fb7877fe640 1 -- 172.21.15.27:0/411977401 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb774004d40 msgr2=0x7fb78815b590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:50.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:50.485+0000 7fb7877fe640 1 --2- 172.21.15.27:0/411977401 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb774004d40 0x7fb78815b590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:50.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:50.485+0000 7fb7877fe640 1 -- 172.21.15.27:0/411977401 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb78815bd90 msgr2=0x7fb7881e7360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:50.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:50.485+0000 7fb7877fe640 1 --2- 172.21.15.27:0/411977401 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb78815bd90 0x7fb7881e7360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:50.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:50.485+0000 7fb7877fe640 1 -- 172.21.15.27:0/411977401 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7881e7c60 con 0x7fb7881e9860 2024-09-12T03:19:50.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:50.485+0000 7fb786ffd640 1 --2- 172.21.15.27:0/411977401 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb774004d40 0x7fb78815b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:19:50.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:50.485+0000 7fb7877fe640 1 --2- 172.21.15.27:0/411977401 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb7881e9860 0x7fb7881e9cf0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fb780000c00 tx=0x7fb780000fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:50.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:50.485+0000 7fb77f7fe640 1 -- 172.21.15.27:0/411977401 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb780007d50 con 0x7fb7881e9860 2024-09-12T03:19:50.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:50.485+0000 7fb77f7fe640 1 -- 172.21.15.27:0/411977401 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb780043070 con 0x7fb7881e9860 2024-09-12T03:19:50.490 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:50.485+0000 7fb78d5af640 1 -- 172.21.15.27:0/411977401 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7881e7f40 con 0x7fb7881e9860 2024-09-12T03:19:50.490 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:50.485+0000 7fb77f7fe640 1 -- 172.21.15.27:0/411977401 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb780007070 con 0x7fb7881e9860 2024-09-12T03:19:50.490 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:50.485+0000 7fb78d5af640 1 -- 172.21.15.27:0/411977401 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7881e8500 con 0x7fb7881e9860 2024-09-12T03:19:50.490 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:50.486+0000 7fb78d5af640 1 -- 172.21.15.27:0/411977401 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb788161860 con 0x7fb7881e9860 2024-09-12T03:19:50.490 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:50.486+0000 7fb77f7fe640 1 -- 172.21.15.27:0/411977401 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 11) ==== 90804+0+0 (secure 0 0 0) 0x7fb780007210 con 0x7fb7881e9860 2024-09-12T03:19:50.490 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:50.487+0000 7fb77f7fe640 1 --2- 172.21.15.27:0/411977401 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fb76806c190 0x7fb76806e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:50.490 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:50.487+0000 7fb77f7fe640 1 -- 172.21.15.27:0/411977401 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 4943+0+0 (secure 0 0 0) 0x7fb7800ab600 con 0x7fb7881e9860 2024-09-12T03:19:50.490 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:50.487+0000 7fb786ffd640 1 --2- 172.21.15.27:0/411977401 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fb76806c190 0x7fb76806e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:50.490 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:50.488+0000 7fb786ffd640 1 --2- 172.21.15.27:0/411977401 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fb76806c190 0x7fb76806e650 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fb788161830 tx=0x7fb77800b040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:50.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:50.491+0000 7fb77f7fe640 1 -- 172.21.15.27:0/411977401 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fb780079f30 con 0x7fb7881e9860 2024-09-12T03:19:50.664 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:50.664+0000 7fb78d5af640 1 -- 172.21.15.27:0/411977401 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7fb78806e6e0 con 0x7fb7881e9860 2024-09-12T03:19:50.669 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:50.669+0000 7fb77f7fe640 1 -- 172.21.15.27:0/411977401 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v15) ==== 104+0+0 (secure 0 0 0) 0x7fb780079830 con 0x7fb7881e9860 2024-09-12T03:19:50.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:50.671+0000 7fb78d5af640 1 -- 172.21.15.27:0/411977401 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fb76806c190 msgr2=0x7fb76806e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:50.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:50.671+0000 7fb78d5af640 1 --2- 172.21.15.27:0/411977401 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fb76806c190 0x7fb76806e650 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fb788161830 tx=0x7fb77800b040 comp rx=0 tx=0).stop 2024-09-12T03:19:50.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:50.671+0000 7fb78d5af640 1 -- 172.21.15.27:0/411977401 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb7881e9860 msgr2=0x7fb7881e9cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:50.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:50.671+0000 7fb78d5af640 1 --2- 172.21.15.27:0/411977401 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb7881e9860 0x7fb7881e9cf0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fb780000c00 tx=0x7fb780000fe0 comp rx=0 tx=0).stop 2024-09-12T03:19:50.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:50.671+0000 7fb78d5af640 1 -- 172.21.15.27:0/411977401 shutdown_connections 2024-09-12T03:19:50.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:50.671+0000 7fb78d5af640 1 --2- 172.21.15.27:0/411977401 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fb76806c190 0x7fb76806e650 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:50.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:50.671+0000 7fb78d5af640 1 --2- 172.21.15.27:0/411977401 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb7881e9860 0x7fb7881e9cf0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:50.674 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:50.671+0000 7fb78d5af640 1 --2- 172.21.15.27:0/411977401 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb78815bd90 0x7fb7881e7360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:50.674 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:50.671+0000 7fb78d5af640 1 --2- 172.21.15.27:0/411977401 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb774004d40 0x7fb78815b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:50.674 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:50.672+0000 7fb78d5af640 1 -- 172.21.15.27:0/411977401 >> 172.21.15.27:0/411977401 conn(0x7fb7880b6700 msgr2=0x7fb7880cba90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:50.674 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:50.672+0000 7fb78d5af640 1 -- 172.21.15.27:0/411977401 shutdown_connections 2024-09-12T03:19:50.674 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:50.672+0000 7fb78d5af640 1 -- 172.21.15.27:0/411977401 wait complete. 2024-09-12T03:19:50.683 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestMisc.test_malicious_metafile_on_legacy_to_v1_upgrade' 2024-09-12T03:19:50.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:50.785+0000 7fea35b11640 1 Processor -- start 2024-09-12T03:19:50.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:50.785+0000 7fea35b11640 1 -- start start 2024-09-12T03:19:50.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:50.786+0000 7fea35b11640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fea300cf660 0x7fea300d2780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:50.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:50.786+0000 7fea35b11640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fea3006cc40 con 0x7fea3006f360 2024-09-12T03:19:50.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:50.786+0000 7fea35b11640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fea300d1ce0 con 0x7fea300b1ca0 2024-09-12T03:19:50.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:50.786+0000 7fea35b11640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fea300d22f0 con 0x7fea300cf660 2024-09-12T03:19:50.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:50.786+0000 7fea2effd640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fea3006f360 0x7fea300cef50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:53070/0 (socket says 172.21.15.27:53070) 2024-09-12T03:19:50.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:50.786+0000 7fea2effd640 1 -- 172.21.15.27:0/2381004589 learned_addr learned my addr 172.21.15.27:0/2381004589 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:19:50.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:50.786+0000 7fea2ffff640 1 --2- 172.21.15.27:0/2381004589 >> v2:172.21.15.45:3300/0 conn(0x7fea300cf660 0x7fea300d2780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:50.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:50.787+0000 7fea2e7fc640 1 -- 172.21.15.27:0/2381004589 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1402842803 0 0) 0x7fea3006cc40 con 0x7fea3006f360 2024-09-12T03:19:50.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:50.787+0000 7fea2e7fc640 1 -- 172.21.15.27:0/2381004589 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fea18003520 con 0x7fea3006f360 2024-09-12T03:19:50.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:50.787+0000 7fea2e7fc640 1 -- 172.21.15.27:0/2381004589 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2851338945 0 0) 0x7fea18003520 con 0x7fea3006f360 2024-09-12T03:19:50.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:50.787+0000 7fea2e7fc640 1 -- 172.21.15.27:0/2381004589 >> v2:172.21.15.45:3300/0 conn(0x7fea300cf660 msgr2=0x7fea300d2780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:50.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:50.787+0000 7fea2e7fc640 1 --2- 172.21.15.27:0/2381004589 >> v2:172.21.15.45:3300/0 conn(0x7fea300cf660 0x7fea300d2780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:50.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:50.787+0000 7fea2e7fc640 1 -- 172.21.15.27:0/2381004589 >> v1:172.21.15.45:6789/0 conn(0x7fea300b1ca0 legacy=0x7fea3015f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:50.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:50.787+0000 7fea2e7fc640 1 -- 172.21.15.27:0/2381004589 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea3006cee0 con 0x7fea3006f360 2024-09-12T03:19:50.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:50.787+0000 7fea2e7fc640 1 -- 172.21.15.27:0/2381004589 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 847755213 0 0) 0x7fea24001670 con 0x7fea3006f360 2024-09-12T03:19:50.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:50.787+0000 7fea2ffff640 1 --2- 172.21.15.27:0/2381004589 >> v2:172.21.15.45:3300/0 conn(0x7fea300cf660 0x7fea300d2780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:19:50.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:50.787+0000 7fea2e7fc640 1 -- 172.21.15.27:0/2381004589 >> v1:172.21.15.27:6789/0 conn(0x7fea3006f360 legacy=0x7fea300cef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:50.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:50.787+0000 7fea2e7fc640 1 --2- 172.21.15.27:0/2381004589 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fea180039b0 0x7fea18003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:50.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:50.787+0000 7fea2e7fc640 1 --2- 172.21.15.27:0/2381004589 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fea18004800 0x7fea18004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:50.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:50.787+0000 7fea2e7fc640 1 --2- 172.21.15.27:0/2381004589 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fea180051c0 0x7fea1800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:50.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:50.787+0000 7fea2e7fc640 1 -- 172.21.15.27:0/2381004589 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fea300d22f0 con 0x7fea180039b0 2024-09-12T03:19:50.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:50.787+0000 7fea2e7fc640 1 -- 172.21.15.27:0/2381004589 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fea1800c060 con 0x7fea18004800 2024-09-12T03:19:50.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:50.787+0000 7fea2e7fc640 1 -- 172.21.15.27:0/2381004589 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fea1800c260 con 0x7fea180051c0 2024-09-12T03:19:50.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:50.787+0000 7fea2f7fe640 1 --2- 172.21.15.27:0/2381004589 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fea180039b0 0x7fea18003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:50.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:50.787+0000 7fea2effd640 1 --2- 172.21.15.27:0/2381004589 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fea18004800 0x7fea18004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:50.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:50.787+0000 7fea2ffff640 1 --2- 172.21.15.27:0/2381004589 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fea180051c0 0x7fea1800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:50.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:50.787+0000 7fea2f7fe640 1 -- 172.21.15.27:0/2381004589 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fea180051c0 msgr2=0x7fea1800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:50.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:50.787+0000 7fea2f7fe640 1 --2- 172.21.15.27:0/2381004589 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fea180051c0 0x7fea1800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:50.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:50.787+0000 7fea2f7fe640 1 -- 172.21.15.27:0/2381004589 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fea18004800 msgr2=0x7fea18004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:50.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:50.787+0000 7fea2f7fe640 1 --2- 172.21.15.27:0/2381004589 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fea18004800 0x7fea18004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:50.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:50.787+0000 7fea2f7fe640 1 -- 172.21.15.27:0/2381004589 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea18003520 con 0x7fea180039b0 2024-09-12T03:19:50.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:50.787+0000 7fea2ffff640 1 -- 172.21.15.27:0/2381004589 reap_dead start 2024-09-12T03:19:50.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:50.788+0000 7fea2f7fe640 1 --2- 172.21.15.27:0/2381004589 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fea180039b0 0x7fea18003dd0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fea20001720 tx=0x7fea18005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c92b33a141f7fd8 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:50.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:50.788+0000 7fea2e7fc640 1 -- 172.21.15.27:0/2381004589 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea20001370 con 0x7fea180039b0 2024-09-12T03:19:50.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:50.788+0000 7fea2e7fc640 1 -- 172.21.15.27:0/2381004589 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea200052d0 con 0x7fea180039b0 2024-09-12T03:19:50.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:50.788+0000 7fea2e7fc640 1 -- 172.21.15.27:0/2381004589 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea2002ab90 con 0x7fea180039b0 2024-09-12T03:19:50.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:50.789+0000 7fea35b11640 1 -- 172.21.15.27:0/2381004589 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fea180039b0 msgr2=0x7fea18003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:50.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:50.789+0000 7fea35b11640 1 --2- 172.21.15.27:0/2381004589 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fea180039b0 0x7fea18003dd0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fea20001720 tx=0x7fea18005830 comp rx=0 tx=0).stop 2024-09-12T03:19:50.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:50.789+0000 7fea35b11640 1 -- 172.21.15.27:0/2381004589 shutdown_connections 2024-09-12T03:19:50.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:50.789+0000 7fea35b11640 1 --2- 172.21.15.27:0/2381004589 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fea180039b0 0x7fea18003dd0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:50.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:50.789+0000 7fea35b11640 1 -- 172.21.15.27:0/2381004589 >> 172.21.15.27:0/2381004589 conn(0x7fea30073800 msgr2=0x7fea3009ba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:50.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:50.789+0000 7fea35b11640 1 -- 172.21.15.27:0/2381004589 shutdown_connections 2024-09-12T03:19:50.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:50.789+0000 7fea35b11640 1 -- 172.21.15.27:0/2381004589 wait complete. 2024-09-12T03:19:50.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:50.790+0000 7fea35b11640 1 Processor -- start 2024-09-12T03:19:50.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:50.790+0000 7fea35b11640 1 -- start start 2024-09-12T03:19:50.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:50.791+0000 7fea35b11640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fea180039b0 0x7fea30150c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:50.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:50.791+0000 7fea35b11640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fea300a0170 0x7fea301500c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:50.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:50.791+0000 7fea35b11640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fea300a05d0 0x7fea30150600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:50.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:50.791+0000 7fea35b11640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fea3006d340 con 0x7fea300a0170 2024-09-12T03:19:50.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:50.791+0000 7fea35b11640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fea3006d1c0 con 0x7fea180039b0 2024-09-12T03:19:50.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:50.791+0000 7fea35b11640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fea300cfc40 con 0x7fea300a05d0 2024-09-12T03:19:50.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:50.791+0000 7fea2effd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fea300a0170 0x7fea301500c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:50.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:50.791+0000 7fea2effd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fea300a0170 0x7fea301500c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35396/0 (socket says 172.21.15.27:35396) 2024-09-12T03:19:50.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:50.791+0000 7fea2effd640 1 -- 172.21.15.27:0/2379384014 learned_addr learned my addr 172.21.15.27:0/2379384014 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:19:50.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:50.791+0000 7fea2f7fe640 1 --2- 172.21.15.27:0/2379384014 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fea180039b0 0x7fea30150c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:50.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:50.791+0000 7fea2ffff640 1 --2- 172.21.15.27:0/2379384014 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fea300a05d0 0x7fea30150600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:50.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:50.791+0000 7fea2effd640 1 -- 172.21.15.27:0/2379384014 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fea300a05d0 msgr2=0x7fea30150600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:50.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:50.791+0000 7fea2effd640 1 --2- 172.21.15.27:0/2379384014 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fea300a05d0 0x7fea30150600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:50.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:50.791+0000 7fea2effd640 1 -- 172.21.15.27:0/2379384014 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fea180039b0 msgr2=0x7fea30150c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:50.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:50.791+0000 7fea2effd640 1 --2- 172.21.15.27:0/2379384014 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fea180039b0 0x7fea30150c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:50.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:50.791+0000 7fea2effd640 1 -- 172.21.15.27:0/2379384014 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea30160d00 con 0x7fea300a0170 2024-09-12T03:19:50.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:50.791+0000 7fea2effd640 1 --2- 172.21.15.27:0/2379384014 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fea300a0170 0x7fea301500c0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fea240072c0 tx=0x7fea24007780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:50.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:50.792+0000 7fea17fff640 1 -- 172.21.15.27:0/2379384014 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea2400f960 con 0x7fea300a0170 2024-09-12T03:19:50.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:50.792+0000 7fea17fff640 1 -- 172.21.15.27:0/2379384014 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea24018070 con 0x7fea300a0170 2024-09-12T03:19:50.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:50.792+0000 7fea35b11640 1 -- 172.21.15.27:0/2379384014 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea30160a20 con 0x7fea300a0170 2024-09-12T03:19:50.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:50.792+0000 7fea17fff640 1 -- 172.21.15.27:0/2379384014 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea2400fd60 con 0x7fea300a0170 2024-09-12T03:19:50.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:50.792+0000 7fea35b11640 1 -- 172.21.15.27:0/2379384014 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea3014e3b0 con 0x7fea300a0170 2024-09-12T03:19:50.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:50.793+0000 7fea35b11640 1 -- 172.21.15.27:0/2379384014 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea3006f7d0 con 0x7fea300a0170 2024-09-12T03:19:50.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:50.793+0000 7fea17fff640 1 -- 172.21.15.27:0/2379384014 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 11) ==== 90804+0+0 (secure 0 0 0) 0x7fea2401d460 con 0x7fea300a0170 2024-09-12T03:19:50.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:50.794+0000 7fea17fff640 1 --2- 172.21.15.27:0/2379384014 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fea1c06c1c0 0x7fea1c06e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:50.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:50.794+0000 7fea17fff640 1 -- 172.21.15.27:0/2379384014 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 4943+0+0 (secure 0 0 0) 0x7fea2408aff0 con 0x7fea300a0170 2024-09-12T03:19:50.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:50.794+0000 7fea2f7fe640 1 --2- 172.21.15.27:0/2379384014 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fea1c06c1c0 0x7fea1c06e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:50.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:50.795+0000 7fea2f7fe640 1 --2- 172.21.15.27:0/2379384014 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fea1c06c1c0 0x7fea1c06e680 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fea20032000 tx=0x7fea2002e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:50.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:50.798+0000 7fea17fff640 1 -- 172.21.15.27:0/2379384014 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fea240591b0 con 0x7fea300a0170 2024-09-12T03:19:51.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:51.041+0000 7fea35b11640 1 -- 172.21.15.27:0/2379384014 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_malicious_metafile_on_legacy_to_v1_upgrade"]} v 0) -- 0x7fea300b74e0 con 0x7fea300a0170 2024-09-12T03:19:51.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:51.351+0000 7fea17fff640 1 -- 172.21.15.27:0/2379384014 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_malicious_metafile_on_legacy_to_v1_upgrade"]}]=0 v183) ==== 161+0+0 (secure 0 0 0) 0x7fea2405d4b0 con 0x7fea300a0170 2024-09-12T03:19:51.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:51.353+0000 7fea35b11640 1 -- 172.21.15.27:0/2379384014 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fea1c06c1c0 msgr2=0x7fea1c06e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:51.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:51.353+0000 7fea35b11640 1 --2- 172.21.15.27:0/2379384014 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fea1c06c1c0 0x7fea1c06e680 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fea20032000 tx=0x7fea2002e000 comp rx=0 tx=0).stop 2024-09-12T03:19:51.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:51.353+0000 7fea35b11640 1 -- 172.21.15.27:0/2379384014 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fea300a0170 msgr2=0x7fea301500c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:51.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:51.353+0000 7fea35b11640 1 --2- 172.21.15.27:0/2379384014 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fea300a0170 0x7fea301500c0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fea240072c0 tx=0x7fea24007780 comp rx=0 tx=0).stop 2024-09-12T03:19:51.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:51.353+0000 7fea35b11640 1 -- 172.21.15.27:0/2379384014 shutdown_connections 2024-09-12T03:19:51.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:51.353+0000 7fea35b11640 1 --2- 172.21.15.27:0/2379384014 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fea1c06c1c0 0x7fea1c06e680 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:51.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:51.353+0000 7fea35b11640 1 --2- 172.21.15.27:0/2379384014 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fea300a05d0 0x7fea30150600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:51.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:51.353+0000 7fea35b11640 1 --2- 172.21.15.27:0/2379384014 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fea300a0170 0x7fea301500c0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:51.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:51.353+0000 7fea35b11640 1 --2- 172.21.15.27:0/2379384014 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fea180039b0 0x7fea30150c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:51.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:51.353+0000 7fea35b11640 1 -- 172.21.15.27:0/2379384014 >> 172.21.15.27:0/2379384014 conn(0x7fea30073800 msgr2=0x7fea300708f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:51.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:51.354+0000 7fea35b11640 1 -- 172.21.15.27:0/2379384014 shutdown_connections 2024-09-12T03:19:51.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:51.354+0000 7fea35b11640 1 -- 172.21.15.27:0/2379384014 wait complete. 2024-09-12T03:19:51.365 INFO:tasks.cephfs_test_runner:Validate handcrafted .meta file on legacy subvol root doesn't break the system ... ok 2024-09-12T03:19:51.365 INFO:tasks.cephfs_test_runner:Starting test: test_mgr_eviction (tasks.cephfs.test_volumes.TestMisc) 2024-09-12T03:19:51.366 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-12T03:19:51.468 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:51.468+0000 7fdc32131640 1 Processor -- start 2024-09-12T03:19:51.468 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:51.468+0000 7fdc32131640 1 -- start start 2024-09-12T03:19:51.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:51.469+0000 7fdc32131640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fdc2c15eb60 0x7fdc2c0d1280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:51.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:51.469+0000 7fdc32131640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fdc2c0d17c0 0x7fdc2c0b3390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:51.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:51.469+0000 7fdc32131640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdc2c0b6690 con 0x7fdc2c15e780 2024-09-12T03:19:51.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:51.469+0000 7fdc32131640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fdc2c0d1c30 con 0x7fdc2c0d17c0 2024-09-12T03:19:51.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:51.469+0000 7fdc32131640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fdc2c0b7950 con 0x7fdc2c15eb60 2024-09-12T03:19:51.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:51.469+0000 7fdc2b7fe640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fdc2c15e780 0x7fdc2c0d3e70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:53082/0 (socket says 172.21.15.27:53082) 2024-09-12T03:19:51.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:51.469+0000 7fdc2b7fe640 1 -- 172.21.15.27:0/234165879 learned_addr learned my addr 172.21.15.27:0/234165879 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:19:51.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:51.469+0000 7fdc2affd640 1 --2- 172.21.15.27:0/234165879 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fdc2c15eb60 0x7fdc2c0d1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:51.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:51.469+0000 7fdc2bfff640 1 --2- 172.21.15.27:0/234165879 >> v2:172.21.15.45:3300/0 conn(0x7fdc2c0d17c0 0x7fdc2c0b3390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:51.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:51.470+0000 7fdc2a7fc640 1 -- 172.21.15.27:0/234165879 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1626597784 0 0) 0x7fdc2c0b6690 con 0x7fdc2c15e780 2024-09-12T03:19:51.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:51.470+0000 7fdc2a7fc640 1 -- 172.21.15.27:0/234165879 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdc14003520 con 0x7fdc2c15e780 2024-09-12T03:19:51.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:51.470+0000 7fdc2a7fc640 1 -- 172.21.15.27:0/234165879 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 160801541 0 0) 0x7fdc14003520 con 0x7fdc2c15e780 2024-09-12T03:19:51.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:51.470+0000 7fdc2a7fc640 1 -- 172.21.15.27:0/234165879 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fdc2c15eb60 msgr2=0x7fdc2c0d1280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:51.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:51.470+0000 7fdc2a7fc640 1 --2- 172.21.15.27:0/234165879 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fdc2c15eb60 0x7fdc2c0d1280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:51.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:51.470+0000 7fdc2a7fc640 1 -- 172.21.15.27:0/234165879 >> v2:172.21.15.45:3300/0 conn(0x7fdc2c0d17c0 msgr2=0x7fdc2c0b3390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:51.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:51.470+0000 7fdc2a7fc640 1 --2- 172.21.15.27:0/234165879 >> v2:172.21.15.45:3300/0 conn(0x7fdc2c0d17c0 0x7fdc2c0b3390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:51.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:51.470+0000 7fdc2a7fc640 1 -- 172.21.15.27:0/234165879 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc2c0b6970 con 0x7fdc2c15e780 2024-09-12T03:19:51.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:51.470+0000 7fdc2a7fc640 1 -- 172.21.15.27:0/234165879 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 847755213 0 0) 0x7fdc1c0019f0 con 0x7fdc2c15e780 2024-09-12T03:19:51.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:51.470+0000 7fdc2a7fc640 1 -- 172.21.15.27:0/234165879 >> v1:172.21.15.27:6789/0 conn(0x7fdc2c15e780 legacy=0x7fdc2c0d3e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:51.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:51.470+0000 7fdc2a7fc640 1 --2- 172.21.15.27:0/234165879 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fdc2c15eb60 0x7fdc140039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:51.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:51.470+0000 7fdc2a7fc640 1 --2- 172.21.15.27:0/234165879 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdc14003f30 0x7fdc14004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:51.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:51.470+0000 7fdc2a7fc640 1 --2- 172.21.15.27:0/234165879 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdc14004d40 0x7fdc140051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:51.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:51.470+0000 7fdc2a7fc640 1 -- 172.21.15.27:0/234165879 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fdc2c0d1c30 con 0x7fdc14003f30 2024-09-12T03:19:51.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:51.470+0000 7fdc2a7fc640 1 -- 172.21.15.27:0/234165879 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fdc1400b9c0 con 0x7fdc14004d40 2024-09-12T03:19:51.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:51.470+0000 7fdc2a7fc640 1 -- 172.21.15.27:0/234165879 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fdc1400bb40 con 0x7fdc2c15eb60 2024-09-12T03:19:51.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:51.470+0000 7fdc2affd640 1 --2- 172.21.15.27:0/234165879 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdc14003f30 0x7fdc14004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:51.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:51.470+0000 7fdc2b7fe640 1 --2- 172.21.15.27:0/234165879 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fdc2c15eb60 0x7fdc140039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:51.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:51.470+0000 7fdc2bfff640 1 --2- 172.21.15.27:0/234165879 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdc14004d40 0x7fdc140051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:51.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:51.470+0000 7fdc2affd640 1 -- 172.21.15.27:0/234165879 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fdc2c15eb60 msgr2=0x7fdc140039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:51.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:51.470+0000 7fdc2affd640 1 --2- 172.21.15.27:0/234165879 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fdc2c15eb60 0x7fdc140039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:51.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:51.470+0000 7fdc2affd640 1 -- 172.21.15.27:0/234165879 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdc14004d40 msgr2=0x7fdc140051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:51.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:51.470+0000 7fdc2affd640 1 --2- 172.21.15.27:0/234165879 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdc14004d40 0x7fdc140051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:51.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:51.470+0000 7fdc2affd640 1 -- 172.21.15.27:0/234165879 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc14003520 con 0x7fdc14003f30 2024-09-12T03:19:51.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:51.470+0000 7fdc2bfff640 1 --2- 172.21.15.27:0/234165879 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdc14004d40 0x7fdc140051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:19:51.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:51.471+0000 7fdc2affd640 1 --2- 172.21.15.27:0/234165879 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdc14003f30 0x7fdc14004800 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fdc20009ac0 tx=0x7fdc14005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4e1ca83b3e9916c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:51.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:51.471+0000 7fdc2a7fc640 1 -- 172.21.15.27:0/234165879 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc20036070 con 0x7fdc14003f30 2024-09-12T03:19:51.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:51.471+0000 7fdc2a7fc640 1 -- 172.21.15.27:0/234165879 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc20031400 con 0x7fdc14003f30 2024-09-12T03:19:51.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:51.471+0000 7fdc2a7fc640 1 -- 172.21.15.27:0/234165879 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc20009690 con 0x7fdc14003f30 2024-09-12T03:19:51.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:51.472+0000 7fdc32131640 1 -- 172.21.15.27:0/234165879 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdc14003f30 msgr2=0x7fdc14004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:51.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:51.472+0000 7fdc32131640 1 --2- 172.21.15.27:0/234165879 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdc14003f30 0x7fdc14004800 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fdc20009ac0 tx=0x7fdc14005830 comp rx=0 tx=0).stop 2024-09-12T03:19:51.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:51.472+0000 7fdc2bfff640 1 -- 172.21.15.27:0/234165879 reap_dead start 2024-09-12T03:19:51.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:51.472+0000 7fdc32131640 1 -- 172.21.15.27:0/234165879 shutdown_connections 2024-09-12T03:19:51.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:51.472+0000 7fdc32131640 1 -- 172.21.15.27:0/234165879 >> 172.21.15.27:0/234165879 conn(0x7fdc2c06da60 msgr2=0x7fdc2c06de70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:51.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:51.472+0000 7fdc32131640 1 -- 172.21.15.27:0/234165879 shutdown_connections 2024-09-12T03:19:51.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:51.472+0000 7fdc32131640 1 -- 172.21.15.27:0/234165879 wait complete. 2024-09-12T03:19:51.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:51.473+0000 7fdc32131640 1 Processor -- start 2024-09-12T03:19:51.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:51.473+0000 7fdc32131640 1 -- start start 2024-09-12T03:19:51.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:51.474+0000 7fdc32131640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdc14003f30 0x7fdc2c1f5370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:51.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:51.474+0000 7fdc32131640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdc2c1f4ee0 0x7fdc2c1f7140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:51.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:51.474+0000 7fdc32131640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fdc2c0abb00 0x7fdc2c0ca3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:51.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:51.474+0000 7fdc32131640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fdc2c0b6f30 con 0x7fdc2c1f4ee0 2024-09-12T03:19:51.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:51.474+0000 7fdc32131640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fdc2c0b7210 con 0x7fdc14003f30 2024-09-12T03:19:51.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:51.474+0000 7fdc32131640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fdc2c0b6340 con 0x7fdc2c0abb00 2024-09-12T03:19:51.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:51.474+0000 7fdc2affd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdc2c1f4ee0 0x7fdc2c1f7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:51.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:51.474+0000 7fdc2affd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdc2c1f4ee0 0x7fdc2c1f7140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35426/0 (socket says 172.21.15.27:35426) 2024-09-12T03:19:51.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:51.474+0000 7fdc2affd640 1 -- 172.21.15.27:0/2693844013 learned_addr learned my addr 172.21.15.27:0/2693844013 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:19:51.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:51.474+0000 7fdc2bfff640 1 --2- 172.21.15.27:0/2693844013 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fdc2c0abb00 0x7fdc2c0ca3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:51.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:51.474+0000 7fdc2b7fe640 1 --2- 172.21.15.27:0/2693844013 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdc14003f30 0x7fdc2c1f5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:51.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:51.474+0000 7fdc2affd640 1 -- 172.21.15.27:0/2693844013 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fdc2c0abb00 msgr2=0x7fdc2c0ca3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:51.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:51.474+0000 7fdc2affd640 1 --2- 172.21.15.27:0/2693844013 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fdc2c0abb00 0x7fdc2c0ca3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:51.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:51.474+0000 7fdc2affd640 1 -- 172.21.15.27:0/2693844013 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdc14003f30 msgr2=0x7fdc2c1f5370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:51.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:51.474+0000 7fdc2affd640 1 --2- 172.21.15.27:0/2693844013 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdc14003f30 0x7fdc2c1f5370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:51.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:51.474+0000 7fdc2affd640 1 -- 172.21.15.27:0/2693844013 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc2c0ccd30 con 0x7fdc2c1f4ee0 2024-09-12T03:19:51.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:51.474+0000 7fdc2b7fe640 1 --2- 172.21.15.27:0/2693844013 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdc14003f30 0x7fdc2c1f5370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:19:51.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:51.475+0000 7fdc2affd640 1 --2- 172.21.15.27:0/2693844013 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdc2c1f4ee0 0x7fdc2c1f7140 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fdc20034040 tx=0x7fdc20004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:51.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:51.475+0000 7fdc13fff640 1 -- 172.21.15.27:0/2693844013 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc20040020 con 0x7fdc2c1f4ee0 2024-09-12T03:19:51.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:51.475+0000 7fdc13fff640 1 -- 172.21.15.27:0/2693844013 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc20003c60 con 0x7fdc2c1f4ee0 2024-09-12T03:19:51.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:51.475+0000 7fdc32131640 1 -- 172.21.15.27:0/2693844013 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc2c0cd010 con 0x7fdc2c1f4ee0 2024-09-12T03:19:51.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:51.475+0000 7fdc13fff640 1 -- 172.21.15.27:0/2693844013 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc20036040 con 0x7fdc2c1f4ee0 2024-09-12T03:19:51.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:51.475+0000 7fdc32131640 1 -- 172.21.15.27:0/2693844013 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc2c0cd5d0 con 0x7fdc2c1f4ee0 2024-09-12T03:19:51.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:51.476+0000 7fdc32131640 1 -- 172.21.15.27:0/2693844013 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc2c0d1280 con 0x7fdc2c1f4ee0 2024-09-12T03:19:51.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:51.476+0000 7fdc13fff640 1 -- 172.21.15.27:0/2693844013 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 11) ==== 90804+0+0 (secure 0 0 0) 0x7fdc20043020 con 0x7fdc2c1f4ee0 2024-09-12T03:19:51.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:51.477+0000 7fdc13fff640 1 --2- 172.21.15.27:0/2693844013 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fdc1806c190 0x7fdc1806e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:51.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:51.477+0000 7fdc13fff640 1 -- 172.21.15.27:0/2693844013 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 4943+0+0 (secure 0 0 0) 0x7fdc200ab040 con 0x7fdc2c1f4ee0 2024-09-12T03:19:51.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:51.477+0000 7fdc2b7fe640 1 --2- 172.21.15.27:0/2693844013 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fdc1806c190 0x7fdc1806e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:51.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:51.478+0000 7fdc2b7fe640 1 --2- 172.21.15.27:0/2693844013 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fdc1806c190 0x7fdc1806e650 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fdc1c004da0 tx=0x7fdc1c006040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:51.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:51.481+0000 7fdc13fff640 1 -- 172.21.15.27:0/2693844013 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fdc2004a070 con 0x7fdc2c1f4ee0 2024-09-12T03:19:51.662 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:51.662+0000 7fdc32131640 1 -- 172.21.15.27:0/2693844013 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdc2c0cffd0 con 0x7fdc2c1f4ee0 2024-09-12T03:19:51.662 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:51.663+0000 7fdc13fff640 1 -- 172.21.15.27:0/2693844013 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v59) ==== 74+0+12984 (secure 0 0 0) 0x7fdc200044e0 con 0x7fdc2c1f4ee0 2024-09-12T03:19:51.663 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:19:51.663 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":59,"fsid":"d2df504e-2647-4385-a5db-9fdf69f487a0","created":"2024-09-12T03:16:07.778937+0000","modified":"2024-09-12T03:19:50.344091+0000","last_up_change":"2024-09-12T03:16:19.314778+0000","last_in_change":"2024-09-12T03:16:09.893086+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T03:16:20.097646+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1851116c-1fb3-4dc9-9772-f90e2ef4c355","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6817","nonce":3002321488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6819","nonce":3002321488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6823","nonce":3002321488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6821","nonce":3002321488}]},"public_addr":"172.21.15.27:6817/3002321488","cluster_addr":"172.21.15.27:6819/3002321488","heartbeat_back_addr":"172.21.15.27:6823/3002321488","heartbeat_front_addr":"172.21.15.27:6821/3002321488","state":["exists","up"]},{"osd":1,"uuid":"f38ae17d-73eb-4204-b99d-975fa806fb87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6809","nonce":1613102886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6811","nonce":1613102886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6815","nonce":1613102886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6813","nonce":1613102886}]},"public_addr":"172.21.15.27:6809/1613102886","cluster_addr":"172.21.15.27:6811/1613102886","heartbeat_back_addr":"172.21.15.27:6815/1613102886","heartbeat_front_addr":"172.21.15.27:6813/1613102886","state":["exists","up"]},{"osd":2,"uuid":"fb12dcc8-bdd0-42b8-b46d-8b10106224bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6825","nonce":3534723355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6827","nonce":3534723355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6831","nonce":3534723355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6829","nonce":3534723355}]},"public_addr":"172.21.15.27:6825/3534723355","cluster_addr":"172.21.15.27:6827/3534723355","heartbeat_back_addr":"172.21.15.27:6831/3534723355","heartbeat_front_addr":"172.21.15.27:6829/3534723355","state":["exists","up"]},{"osd":3,"uuid":"d74d3f49-9c24-4b9f-98e2-3ecd7b5ef3b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6801","nonce":80661598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6803","nonce":80661598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6807","nonce":80661598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6805","nonce":80661598}]},"public_addr":"172.21.15.27:6801/80661598","cluster_addr":"172.21.15.27:6803/80661598","heartbeat_back_addr":"172.21.15.27:6807/80661598","heartbeat_front_addr":"172.21.15.27:6805/80661598","state":["exists","up"]},{"osd":4,"uuid":"2bf123f2-1b0d-41c4-aa3b-805bab267e59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6817","nonce":3943189827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6819","nonce":3943189827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6823","nonce":3943189827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6821","nonce":3943189827}]},"public_addr":"172.21.15.45:6817/3943189827","cluster_addr":"172.21.15.45:6819/3943189827","heartbeat_back_addr":"172.21.15.45:6823/3943189827","heartbeat_front_addr":"172.21.15.45:6821/3943189827","state":["exists","up"]},{"osd":5,"uuid":"c49ba5da-91c6-4a86-a97b-4aba89c253ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6809","nonce":3620760330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6811","nonce":3620760330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6815","nonce":3620760330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6813","nonce":3620760330}]},"public_addr":"172.21.15.45:6809/3620760330","cluster_addr":"172.21.15.45:6811/3620760330","heartbeat_back_addr":"172.21.15.45:6815/3620760330","heartbeat_front_addr":"172.21.15.45:6813/3620760330","state":["exists","up"]},{"osd":6,"uuid":"b0123475-89f7-4a34-9e33-d797b99dd9dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6801","nonce":1989083510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6803","nonce":1989083510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6807","nonce":1989083510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6805","nonce":1989083510}]},"public_addr":"172.21.15.45:6801/1989083510","cluster_addr":"172.21.15.45:6803/1989083510","heartbeat_back_addr":"172.21.15.45:6807/1989083510","heartbeat_front_addr":"172.21.15.45:6805/1989083510","state":["exists","up"]},{"osd":7,"uuid":"4dad773d-b7d8-46fc-8a10-a972aad27b29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6825","nonce":1513300294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6827","nonce":1513300294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6831","nonce":1513300294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6829","nonce":1513300294}]},"public_addr":"172.21.15.45:6825/1513300294","cluster_addr":"172.21.15.45:6827/1513300294","heartbeat_back_addr":"172.21.15.45:6831/1513300294","heartbeat_front_addr":"172.21.15.45:6829/1513300294","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.881855+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:17.133252+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.735339+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.500742+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.477571+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:17.167667+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.470291+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.873743+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:6837/3720118075":"2024-09-13T03:19:47.335980+0000","172.21.15.45:6835/3720118075":"2024-09-13T03:19:47.335980+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T03:19:51.665 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:51.665+0000 7fdc32131640 1 -- 172.21.15.27:0/2693844013 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fdc1806c190 msgr2=0x7fdc1806e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:51.665 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:51.665+0000 7fdc32131640 1 --2- 172.21.15.27:0/2693844013 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fdc1806c190 0x7fdc1806e650 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fdc1c004da0 tx=0x7fdc1c006040 comp rx=0 tx=0).stop 2024-09-12T03:19:51.665 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:51.665+0000 7fdc32131640 1 -- 172.21.15.27:0/2693844013 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdc2c1f4ee0 msgr2=0x7fdc2c1f7140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:51.665 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:51.665+0000 7fdc32131640 1 --2- 172.21.15.27:0/2693844013 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdc2c1f4ee0 0x7fdc2c1f7140 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fdc20034040 tx=0x7fdc20004730 comp rx=0 tx=0).stop 2024-09-12T03:19:51.665 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:51.665+0000 7fdc32131640 1 -- 172.21.15.27:0/2693844013 shutdown_connections 2024-09-12T03:19:51.665 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:51.665+0000 7fdc32131640 1 --2- 172.21.15.27:0/2693844013 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fdc1806c190 0x7fdc1806e650 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:51.665 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:51.665+0000 7fdc32131640 1 --2- 172.21.15.27:0/2693844013 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fdc2c0abb00 0x7fdc2c0ca3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:51.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:51.665+0000 7fdc32131640 1 --2- 172.21.15.27:0/2693844013 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdc2c1f4ee0 0x7fdc2c1f7140 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:51.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:51.665+0000 7fdc32131640 1 --2- 172.21.15.27:0/2693844013 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdc14003f30 0x7fdc2c1f5370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:51.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:51.665+0000 7fdc32131640 1 -- 172.21.15.27:0/2693844013 >> 172.21.15.27:0/2693844013 conn(0x7fdc2c06da60 msgr2=0x7fdc2c0b9390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:51.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:51.665+0000 7fdc32131640 1 -- 172.21.15.27:0/2693844013 shutdown_connections 2024-09-12T03:19:51.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:51.665+0000 7fdc32131640 1 -- 172.21.15.27:0/2693844013 wait complete. 2024-09-12T03:19:51.676 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-12T03:16:20.097646+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-12T03:19:51.677 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-12T03:19:51.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:51.782+0000 7f42108e6640 1 Processor -- start 2024-09-12T03:19:51.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:51.782+0000 7f42108e6640 1 -- start start 2024-09-12T03:19:51.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:51.783+0000 7f42108e6640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f420c15e650 0x7f420c0d1d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:51.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:51.783+0000 7f42108e6640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f420c06db30 0x7f420c06df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:51.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:51.783+0000 7f42108e6640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f420c0d2280 con 0x7f420c06cc40 2024-09-12T03:19:51.784 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:51.783+0000 7f42108e6640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f420c06d850 con 0x7f420c15e650 2024-09-12T03:19:51.784 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:51.783+0000 7f42108e6640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f420c0d1260 con 0x7f420c06db30 2024-09-12T03:19:51.784 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:51.783+0000 7f420a59c640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f420c15e650 0x7f420c0d1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:51.784 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:51.783+0000 7f420a59c640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f420c15e650 0x7f420c0d1d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35430/0 (socket says 172.21.15.27:35430) 2024-09-12T03:19:51.784 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:51.783+0000 7f420a59c640 1 -- 172.21.15.27:0/2902829213 learned_addr learned my addr 172.21.15.27:0/2902829213 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:19:51.784 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:51.783+0000 7f420ad9d640 1 --2- 172.21.15.27:0/2902829213 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f420c06db30 0x7f420c06df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:51.784 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:51.783+0000 7f420a59c640 1 -- 172.21.15.27:0/2902829213 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f420c06db30 msgr2=0x7f420c06df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:51.784 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:51.783+0000 7f420a59c640 1 --2- 172.21.15.27:0/2902829213 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f420c06db30 0x7f420c06df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:51.784 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:51.783+0000 7f420a59c640 1 -- 172.21.15.27:0/2902829213 >> v1:172.21.15.45:6789/0 conn(0x7f420c06cc40 legacy=0x7f420c06d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:51.784 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:51.783+0000 7f420a59c640 1 -- 172.21.15.27:0/2902829213 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f420c0d1540 con 0x7f420c15e650 2024-09-12T03:19:51.784 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:51.783+0000 7f420ad9d640 1 --2- 172.21.15.27:0/2902829213 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f420c06db30 0x7f420c06df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:19:51.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:51.784+0000 7f420a59c640 1 --2- 172.21.15.27:0/2902829213 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f420c15e650 0x7f420c0d1d40 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f41fc009e50 tx=0x7f41fc02a630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fda8fc0e4bd81e05 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:51.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:51.784+0000 7f420959a640 1 -- 172.21.15.27:0/2902829213 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41fc036070 con 0x7f420c15e650 2024-09-12T03:19:51.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:51.784+0000 7f420959a640 1 -- 172.21.15.27:0/2902829213 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41fc031600 con 0x7f420c15e650 2024-09-12T03:19:51.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:51.784+0000 7f420959a640 1 -- 172.21.15.27:0/2902829213 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41fc0319d0 con 0x7f420c15e650 2024-09-12T03:19:51.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:51.785+0000 7f42108e6640 1 -- 172.21.15.27:0/2902829213 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f420c15e650 msgr2=0x7f420c0d1d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:51.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:51.785+0000 7f42108e6640 1 --2- 172.21.15.27:0/2902829213 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f420c15e650 0x7f420c0d1d40 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f41fc009e50 tx=0x7f41fc02a630 comp rx=0 tx=0).stop 2024-09-12T03:19:51.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:51.785+0000 7f42108e6640 1 -- 172.21.15.27:0/2902829213 shutdown_connections 2024-09-12T03:19:51.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:51.785+0000 7f42108e6640 1 --2- 172.21.15.27:0/2902829213 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f420c06db30 0x7f420c06df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:51.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:51.785+0000 7f42108e6640 1 --2- 172.21.15.27:0/2902829213 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f420c15e650 0x7f420c0d1d40 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:51.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:51.785+0000 7f42108e6640 1 -- 172.21.15.27:0/2902829213 >> 172.21.15.27:0/2902829213 conn(0x7f420c06f320 msgr2=0x7f420c06f700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:51.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:51.785+0000 7f42108e6640 1 -- 172.21.15.27:0/2902829213 shutdown_connections 2024-09-12T03:19:51.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:51.785+0000 7f42108e6640 1 -- 172.21.15.27:0/2902829213 wait complete. 2024-09-12T03:19:51.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:51.786+0000 7f42108e6640 1 Processor -- start 2024-09-12T03:19:51.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:51.786+0000 7f42108e6640 1 -- start start 2024-09-12T03:19:51.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:51.787+0000 7f42108e6640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f420c06cc40 0x7f420c153d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:51.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:51.787+0000 7f42108e6640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f420c06db30 0x7f420c0b1740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:51.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:51.787+0000 7f42108e6640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f420c15e650 0x7f420c1572f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:51.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:51.787+0000 7f42108e6640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f420c06e5f0 con 0x7f420c06db30 2024-09-12T03:19:51.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:51.787+0000 7f42108e6640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f420c06e470 con 0x7f420c15e650 2024-09-12T03:19:51.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:51.787+0000 7f42108e6640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f420c0b55b0 con 0x7f420c06cc40 2024-09-12T03:19:51.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:51.787+0000 7f4209d9b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f420c06db30 0x7f420c0b1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:51.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:51.787+0000 7f4209d9b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f420c06db30 0x7f420c0b1740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35442/0 (socket says 172.21.15.27:35442) 2024-09-12T03:19:51.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:51.787+0000 7f4209d9b640 1 -- 172.21.15.27:0/3744349155 learned_addr learned my addr 172.21.15.27:0/3744349155 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:19:51.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:51.787+0000 7f420a59c640 1 --2- 172.21.15.27:0/3744349155 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f420c06cc40 0x7f420c153d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:51.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:51.787+0000 7f420ad9d640 1 --2- 172.21.15.27:0/3744349155 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f420c15e650 0x7f420c1572f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:51.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:51.787+0000 7f4209d9b640 1 -- 172.21.15.27:0/3744349155 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f420c06cc40 msgr2=0x7f420c153d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:51.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:51.787+0000 7f4209d9b640 1 --2- 172.21.15.27:0/3744349155 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f420c06cc40 0x7f420c153d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:51.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:51.787+0000 7f4209d9b640 1 -- 172.21.15.27:0/3744349155 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f420c15e650 msgr2=0x7f420c1572f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:51.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:51.787+0000 7f4209d9b640 1 --2- 172.21.15.27:0/3744349155 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f420c15e650 0x7f420c1572f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:51.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:51.787+0000 7f4209d9b640 1 -- 172.21.15.27:0/3744349155 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f420c158310 con 0x7f420c06db30 2024-09-12T03:19:51.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:51.787+0000 7f4209d9b640 1 --2- 172.21.15.27:0/3744349155 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f420c06db30 0x7f420c0b1740 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f41f400c410 tx=0x7f41f400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:51.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:51.788+0000 7f4202ffd640 1 -- 172.21.15.27:0/3744349155 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41f4017070 con 0x7f420c06db30 2024-09-12T03:19:51.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:51.788+0000 7f4202ffd640 1 -- 172.21.15.27:0/3744349155 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41f40035f0 con 0x7f420c06db30 2024-09-12T03:19:51.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:51.788+0000 7f42108e6640 1 -- 172.21.15.27:0/3744349155 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f420c1585f0 con 0x7f420c06db30 2024-09-12T03:19:51.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:51.788+0000 7f4202ffd640 1 -- 172.21.15.27:0/3744349155 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41f4003a20 con 0x7f420c06db30 2024-09-12T03:19:51.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:51.788+0000 7f42108e6640 1 -- 172.21.15.27:0/3744349155 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f420c158e90 con 0x7f420c06db30 2024-09-12T03:19:51.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:51.789+0000 7f42108e6640 1 -- 172.21.15.27:0/3744349155 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f420c06df30 con 0x7f420c06db30 2024-09-12T03:19:51.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:51.789+0000 7f4202ffd640 1 -- 172.21.15.27:0/3744349155 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 11) ==== 90804+0+0 (secure 0 0 0) 0x7f41f401a070 con 0x7f420c06db30 2024-09-12T03:19:51.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:51.790+0000 7f4202ffd640 1 --2- 172.21.15.27:0/3744349155 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f41ec06c190 0x7f41ec06e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:51.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:51.790+0000 7f4202ffd640 1 -- 172.21.15.27:0/3744349155 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 4943+0+0 (secure 0 0 0) 0x7f41f408a400 con 0x7f420c06db30 2024-09-12T03:19:51.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:51.790+0000 7f420a59c640 1 --2- 172.21.15.27:0/3744349155 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f41ec06c190 0x7f41ec06e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:51.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:51.790+0000 7f420a59c640 1 --2- 172.21.15.27:0/3744349155 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f41ec06c190 0x7f41ec06e650 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f41fc033000 tx=0x7f41fc009000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:51.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:51.794+0000 7f4202ffd640 1 -- 172.21.15.27:0/3744349155 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f41f40586d0 con 0x7f420c06db30 2024-09-12T03:19:51.971 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:51.971+0000 7f42108e6640 1 -- 172.21.15.27:0/3744349155 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f420c1587c0 con 0x7f420c06db30 2024-09-12T03:19:51.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:51.972+0000 7f4202ffd640 1 -- 172.21.15.27:0/3744349155 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v59) ==== 93+0+10 (secure 0 0 0) 0x7f41f405c9d0 con 0x7f420c06db30 2024-09-12T03:19:51.972 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 1 2024-09-12T03:19:51.974 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:51.974+0000 7f42108e6640 1 -- 172.21.15.27:0/3744349155 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f41ec06c190 msgr2=0x7f41ec06e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:51.974 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:51.974+0000 7f42108e6640 1 --2- 172.21.15.27:0/3744349155 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f41ec06c190 0x7f41ec06e650 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f41fc033000 tx=0x7f41fc009000 comp rx=0 tx=0).stop 2024-09-12T03:19:51.974 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:51.974+0000 7f42108e6640 1 -- 172.21.15.27:0/3744349155 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f420c06db30 msgr2=0x7f420c0b1740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:51.974 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:51.974+0000 7f42108e6640 1 --2- 172.21.15.27:0/3744349155 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f420c06db30 0x7f420c0b1740 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f41f400c410 tx=0x7f41f400c8d0 comp rx=0 tx=0).stop 2024-09-12T03:19:51.974 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:51.974+0000 7f42108e6640 1 -- 172.21.15.27:0/3744349155 shutdown_connections 2024-09-12T03:19:51.974 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:51.974+0000 7f42108e6640 1 --2- 172.21.15.27:0/3744349155 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f41ec06c190 0x7f41ec06e650 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:51.974 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:51.974+0000 7f42108e6640 1 --2- 172.21.15.27:0/3744349155 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f420c15e650 0x7f420c1572f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:51.974 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:51.974+0000 7f42108e6640 1 --2- 172.21.15.27:0/3744349155 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f420c06db30 0x7f420c0b1740 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:51.975 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:51.974+0000 7f42108e6640 1 --2- 172.21.15.27:0/3744349155 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f420c06cc40 0x7f420c153d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:51.975 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:51.974+0000 7f42108e6640 1 -- 172.21.15.27:0/3744349155 >> 172.21.15.27:0/3744349155 conn(0x7f420c06f320 msgr2=0x7f420c0b3bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:51.975 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:51.974+0000 7f42108e6640 1 -- 172.21.15.27:0/3744349155 shutdown_connections 2024-09-12T03:19:51.975 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:51.974+0000 7f42108e6640 1 -- 172.21.15.27:0/3744349155 wait complete. 2024-09-12T03:19:51.985 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestMisc.test_mgr_eviction' 2024-09-12T03:19:52.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.088+0000 7f2878186640 1 Processor -- start 2024-09-12T03:19:52.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.088+0000 7f2878186640 1 -- start start 2024-09-12T03:19:52.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.088+0000 7f2878186640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f287015eb90 0x7f28700d2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:52.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.089+0000 7f2878186640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f28700d5200 0x7f28700d1010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:52.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.089+0000 7f2878186640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f28700d3760 con 0x7f28700d1550 2024-09-12T03:19:52.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.089+0000 7f2878186640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f28700b7d20 con 0x7f287015eb90 2024-09-12T03:19:52.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.089+0000 7f2878186640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f28700d42e0 con 0x7f28700d5200 2024-09-12T03:19:52.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.089+0000 7f2876723640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f28700d1550 0x7f28700cf200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:53086/0 (socket says 172.21.15.27:53086) 2024-09-12T03:19:52.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.089+0000 7f2876723640 1 -- 172.21.15.27:0/3867578351 learned_addr learned my addr 172.21.15.27:0/3867578351 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:19:52.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.089+0000 7f2875f22640 1 --2- 172.21.15.27:0/3867578351 >> v2:172.21.15.27:3300/0 conn(0x7f287015eb90 0x7f28700d2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:52.090 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.089+0000 7f2875f22640 1 -- 172.21.15.27:0/3867578351 >> v2:172.21.15.45:3300/0 conn(0x7f28700d5200 msgr2=0x7f28700d1010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:52.090 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.089+0000 7f2875721640 1 --2- 172.21.15.27:0/3867578351 >> v2:172.21.15.45:3300/0 conn(0x7f28700d5200 0x7f28700d1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:52.090 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.089+0000 7f2875f22640 1 --2- 172.21.15.27:0/3867578351 >> v2:172.21.15.45:3300/0 conn(0x7f28700d5200 0x7f28700d1010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:52.090 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.089+0000 7f2875f22640 1 -- 172.21.15.27:0/3867578351 >> v1:172.21.15.27:6789/0 conn(0x7f28700d1550 legacy=0x7f28700cf200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:52.090 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.089+0000 7f2874f20640 1 -- 172.21.15.27:0/3867578351 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 899438522 0 0) 0x7f28700d3760 con 0x7f28700d1550 2024-09-12T03:19:52.090 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.089+0000 7f2875f22640 1 -- 172.21.15.27:0/3867578351 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28700d3a40 con 0x7f287015eb90 2024-09-12T03:19:52.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.090+0000 7f2875f22640 1 --2- 172.21.15.27:0/3867578351 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f287015eb90 0x7f28700d2df0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f28700d4750 tx=0x7f286002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a40ddd9a846ad3a1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:52.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.090+0000 7f2874f20640 1 -- 172.21.15.27:0/3867578351 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2860037070 con 0x7f287015eb90 2024-09-12T03:19:52.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.090+0000 7f2874f20640 1 -- 172.21.15.27:0/3867578351 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2860032440 con 0x7f287015eb90 2024-09-12T03:19:52.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.090+0000 7f2874f20640 1 -- 172.21.15.27:0/3867578351 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2860032810 con 0x7f287015eb90 2024-09-12T03:19:52.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.090+0000 7f2878186640 1 -- 172.21.15.27:0/3867578351 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f287015eb90 msgr2=0x7f28700d2df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:52.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.090+0000 7f2878186640 1 --2- 172.21.15.27:0/3867578351 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f287015eb90 0x7f28700d2df0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f28700d4750 tx=0x7f286002a1a0 comp rx=0 tx=0).stop 2024-09-12T03:19:52.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.091+0000 7f2878186640 1 -- 172.21.15.27:0/3867578351 shutdown_connections 2024-09-12T03:19:52.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.091+0000 7f2878186640 1 --2- 172.21.15.27:0/3867578351 >> v2:172.21.15.45:3300/0 conn(0x7f28700d5200 0x7f28700d1010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:52.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.091+0000 7f2878186640 1 --2- 172.21.15.27:0/3867578351 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f287015eb90 0x7f28700d2df0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:52.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.091+0000 7f2878186640 1 -- 172.21.15.27:0/3867578351 >> 172.21.15.27:0/3867578351 conn(0x7f287006dce0 msgr2=0x7f287006e0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:52.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.091+0000 7f2878186640 1 -- 172.21.15.27:0/3867578351 shutdown_connections 2024-09-12T03:19:52.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.091+0000 7f2878186640 1 -- 172.21.15.27:0/3867578351 wait complete. 2024-09-12T03:19:52.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.092+0000 7f2878186640 1 Processor -- start 2024-09-12T03:19:52.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.092+0000 7f2878186640 1 -- start start 2024-09-12T03:19:52.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.093+0000 7f2878186640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f28700d1550 0x7f28701f1b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:52.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.093+0000 7f2878186640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f28700d5200 0x7f28701f5020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:52.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.093+0000 7f2878186640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f287015eb90 0x7f287015ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:52.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.093+0000 7f2878186640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f28700d1960 con 0x7f28700d1550 2024-09-12T03:19:52.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.093+0000 7f2878186640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f28700d4000 con 0x7f28700d5200 2024-09-12T03:19:52.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.093+0000 7f2878186640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f28700d3330 con 0x7f287015eb90 2024-09-12T03:19:52.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.093+0000 7f2875f22640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f28700d1550 0x7f28701f1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:52.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.093+0000 7f2875f22640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f28700d1550 0x7f28701f1b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35458/0 (socket says 172.21.15.27:35458) 2024-09-12T03:19:52.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.093+0000 7f2875f22640 1 -- 172.21.15.27:0/4197528338 learned_addr learned my addr 172.21.15.27:0/4197528338 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:19:52.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.093+0000 7f2875721640 1 --2- 172.21.15.27:0/4197528338 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f28700d5200 0x7f28701f5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:52.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.093+0000 7f2876723640 1 --2- 172.21.15.27:0/4197528338 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f287015eb90 0x7f287015ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:52.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.093+0000 7f2875f22640 1 -- 172.21.15.27:0/4197528338 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f287015eb90 msgr2=0x7f287015ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:52.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.093+0000 7f2875f22640 1 --2- 172.21.15.27:0/4197528338 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f287015eb90 0x7f287015ad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:52.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.093+0000 7f2875f22640 1 -- 172.21.15.27:0/4197528338 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f28700d5200 msgr2=0x7f28701f5020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:52.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.093+0000 7f2875f22640 1 --2- 172.21.15.27:0/4197528338 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f28700d5200 0x7f28701f5020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:52.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.093+0000 7f2875f22640 1 -- 172.21.15.27:0/4197528338 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28700c8470 con 0x7f28700d1550 2024-09-12T03:19:52.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.093+0000 7f2875721640 1 --2- 172.21.15.27:0/4197528338 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f28700d5200 0x7f28701f5020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:19:52.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.093+0000 7f2875f22640 1 --2- 172.21.15.27:0/4197528338 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f28700d1550 0x7f28701f1b20 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f2860000c00 tx=0x7f28600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:52.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.093+0000 7f28667fc640 1 -- 172.21.15.27:0/4197528338 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2860037040 con 0x7f28700d1550 2024-09-12T03:19:52.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.093+0000 7f28667fc640 1 -- 172.21.15.27:0/4197528338 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2860042070 con 0x7f28700d1550 2024-09-12T03:19:52.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.093+0000 7f28667fc640 1 -- 172.21.15.27:0/4197528338 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2860030bd0 con 0x7f28700d1550 2024-09-12T03:19:52.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.093+0000 7f2878186640 1 -- 172.21.15.27:0/4197528338 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28700c8750 con 0x7f28700d1550 2024-09-12T03:19:52.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.093+0000 7f2878186640 1 -- 172.21.15.27:0/4197528338 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28700c9230 con 0x7f28700d1550 2024-09-12T03:19:52.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.094+0000 7f2878186640 1 -- 172.21.15.27:0/4197528338 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f287015ef70 con 0x7f28700d1550 2024-09-12T03:19:52.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.095+0000 7f28667fc640 1 -- 172.21.15.27:0/4197528338 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 11) ==== 90804+0+0 (secure 0 0 0) 0x7f2860047460 con 0x7f28700d1550 2024-09-12T03:19:52.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.096+0000 7f28667fc640 1 --2- 172.21.15.27:0/4197528338 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f285406c190 0x7f285406e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:52.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.096+0000 7f28667fc640 1 -- 172.21.15.27:0/4197528338 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 4943+0+0 (secure 0 0 0) 0x7f28600abcd0 con 0x7f28700d1550 2024-09-12T03:19:52.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.096+0000 7f2875721640 1 --2- 172.21.15.27:0/4197528338 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f285406c190 0x7f285406e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:52.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.096+0000 7f2875721640 1 --2- 172.21.15.27:0/4197528338 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f285406c190 0x7f285406e650 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f28700c8140 tx=0x7f286c008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:52.100 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.100+0000 7f28667fc640 1 -- 172.21.15.27:0/4197528338 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f28600b1020 con 0x7f28700d1550 2024-09-12T03:19:52.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.345+0000 7f2878186640 1 -- 172.21.15.27:0/4197528338 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_mgr_eviction"]} v 0) -- 0x7f28700b9170 con 0x7f28700d1550 2024-09-12T03:19:52.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.358+0000 7f28667fc640 1 -- 172.21.15.27:0/4197528338 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_mgr_eviction"]}]=0 v184) ==== 134+0+0 (secure 0 0 0) 0x7f2860032e20 con 0x7f28700d1550 2024-09-12T03:19:52.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.360+0000 7f2878186640 1 -- 172.21.15.27:0/4197528338 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f285406c190 msgr2=0x7f285406e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:52.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.360+0000 7f2878186640 1 --2- 172.21.15.27:0/4197528338 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f285406c190 0x7f285406e650 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f28700c8140 tx=0x7f286c008040 comp rx=0 tx=0).stop 2024-09-12T03:19:52.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.360+0000 7f2878186640 1 -- 172.21.15.27:0/4197528338 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f28700d1550 msgr2=0x7f28701f1b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:52.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.360+0000 7f2878186640 1 --2- 172.21.15.27:0/4197528338 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f28700d1550 0x7f28701f1b20 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f2860000c00 tx=0x7f28600046c0 comp rx=0 tx=0).stop 2024-09-12T03:19:52.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.360+0000 7f2878186640 1 -- 172.21.15.27:0/4197528338 shutdown_connections 2024-09-12T03:19:52.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.360+0000 7f2878186640 1 --2- 172.21.15.27:0/4197528338 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f285406c190 0x7f285406e650 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:52.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.360+0000 7f2878186640 1 --2- 172.21.15.27:0/4197528338 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f287015eb90 0x7f287015ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:52.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.360+0000 7f2878186640 1 --2- 172.21.15.27:0/4197528338 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f28700d5200 0x7f28701f5020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:52.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.360+0000 7f2878186640 1 --2- 172.21.15.27:0/4197528338 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f28700d1550 0x7f28701f1b20 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:52.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.360+0000 7f2878186640 1 -- 172.21.15.27:0/4197528338 >> 172.21.15.27:0/4197528338 conn(0x7f287006dce0 msgr2=0x7f2870160f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:52.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.361+0000 7f2878186640 1 -- 172.21.15.27:0/4197528338 shutdown_connections 2024-09-12T03:19:52.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.361+0000 7f2878186640 1 -- 172.21.15.27:0/4197528338 wait complete. 2024-09-12T03:19:52.371 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-12T03:19:52.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.477+0000 7f5a8642e640 1 Processor -- start 2024-09-12T03:19:52.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.477+0000 7f5a8642e640 1 -- start start 2024-09-12T03:19:52.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.477+0000 7f5a8642e640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f5a8015ec30 0x7f5a800b3900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:52.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.477+0000 7f5a8642e640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f5a800af9d0 0x7f5a800b3e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:52.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.478+0000 7f5a8642e640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5a800ce5e0 con 0x7f5a800b4380 2024-09-12T03:19:52.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.478+0000 7f5a8642e640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f5a800d02d0 con 0x7f5a8015ec30 2024-09-12T03:19:52.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.478+0000 7f5a8642e640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f5a8006c650 con 0x7f5a800af9d0 2024-09-12T03:19:52.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.478+0000 7f5a7ffff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f5a8015ec30 0x7f5a800b3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:52.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.478+0000 7f5a7ffff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f5a8015ec30 0x7f5a800b3900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35470/0 (socket says 172.21.15.27:35470) 2024-09-12T03:19:52.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.478+0000 7f5a7ffff640 1 -- 172.21.15.27:0/2383306469 learned_addr learned my addr 172.21.15.27:0/2383306469 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:19:52.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.478+0000 7f5a7f7fe640 1 --2- 172.21.15.27:0/2383306469 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f5a800af9d0 0x7f5a800b3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:52.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.478+0000 7f5a7ffff640 1 -- 172.21.15.27:0/2383306469 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f5a800af9d0 msgr2=0x7f5a800b3e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:52.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.478+0000 7f5a7ffff640 1 --2- 172.21.15.27:0/2383306469 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f5a800af9d0 0x7f5a800b3e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:52.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.478+0000 7f5a7ffff640 1 -- 172.21.15.27:0/2383306469 >> v1:172.21.15.45:6789/0 conn(0x7f5a800b4380 legacy=0x7f5a8015f2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:52.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.478+0000 7f5a7ffff640 1 -- 172.21.15.27:0/2383306469 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a8015f010 con 0x7f5a8015ec30 2024-09-12T03:19:52.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.479+0000 7f5a7ffff640 1 --2- 172.21.15.27:0/2383306469 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5a8015ec30 0x7f5a800b3900 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f5a70009870 tx=0x7f5a7002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9eb4813bbe517dae server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:52.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.479+0000 7f5a7effd640 1 -- 172.21.15.27:0/2383306469 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a70037070 con 0x7f5a8015ec30 2024-09-12T03:19:52.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.479+0000 7f5a7effd640 1 -- 172.21.15.27:0/2383306469 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a70032440 con 0x7f5a8015ec30 2024-09-12T03:19:52.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.479+0000 7f5a7effd640 1 -- 172.21.15.27:0/2383306469 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a70032810 con 0x7f5a8015ec30 2024-09-12T03:19:52.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.479+0000 7f5a8642e640 1 -- 172.21.15.27:0/2383306469 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5a8015ec30 msgr2=0x7f5a800b3900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:52.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.479+0000 7f5a8642e640 1 --2- 172.21.15.27:0/2383306469 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5a8015ec30 0x7f5a800b3900 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f5a70009870 tx=0x7f5a7002a270 comp rx=0 tx=0).stop 2024-09-12T03:19:52.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.479+0000 7f5a8642e640 1 -- 172.21.15.27:0/2383306469 shutdown_connections 2024-09-12T03:19:52.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.480+0000 7f5a8642e640 1 --2- 172.21.15.27:0/2383306469 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f5a800af9d0 0x7f5a800b3e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:52.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.480+0000 7f5a8642e640 1 --2- 172.21.15.27:0/2383306469 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5a8015ec30 0x7f5a800b3900 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:52.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.480+0000 7f5a8642e640 1 -- 172.21.15.27:0/2383306469 >> 172.21.15.27:0/2383306469 conn(0x7f5a8006bb30 msgr2=0x7f5a8006bf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:52.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.480+0000 7f5a8642e640 1 -- 172.21.15.27:0/2383306469 shutdown_connections 2024-09-12T03:19:52.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.480+0000 7f5a8642e640 1 -- 172.21.15.27:0/2383306469 wait complete. 2024-09-12T03:19:52.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.481+0000 7f5a8642e640 1 Processor -- start 2024-09-12T03:19:52.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.481+0000 7f5a8642e640 1 -- start start 2024-09-12T03:19:52.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.482+0000 7f5a8642e640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f5a800af9d0 0x7f5a800b08c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:52.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.482+0000 7f5a8642e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5a800b4380 0x7f5a800b0e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:52.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.482+0000 7f5a8642e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5a8015ec30 0x7f5a8014a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:52.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.482+0000 7f5a8642e640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f5a800715f0 con 0x7f5a800b4380 2024-09-12T03:19:52.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.482+0000 7f5a8642e640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f5a800cfff0 con 0x7f5a8015ec30 2024-09-12T03:19:52.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.482+0000 7f5a8642e640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f5a8015f9b0 con 0x7f5a800af9d0 2024-09-12T03:19:52.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.482+0000 7f5a7f7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5a800b4380 0x7f5a800b0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:52.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.482+0000 7f5a7f7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5a800b4380 0x7f5a800b0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35480/0 (socket says 172.21.15.27:35480) 2024-09-12T03:19:52.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.482+0000 7f5a7f7fe640 1 -- 172.21.15.27:0/282256669 learned_addr learned my addr 172.21.15.27:0/282256669 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:19:52.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.482+0000 7f5a7ffff640 1 --2- 172.21.15.27:0/282256669 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f5a800af9d0 0x7f5a800b08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:52.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.482+0000 7f5a849cb640 1 --2- 172.21.15.27:0/282256669 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5a8015ec30 0x7f5a8014a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:52.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.482+0000 7f5a7f7fe640 1 -- 172.21.15.27:0/282256669 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f5a800af9d0 msgr2=0x7f5a800b08c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:52.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.482+0000 7f5a7f7fe640 1 --2- 172.21.15.27:0/282256669 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f5a800af9d0 0x7f5a800b08c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:52.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.482+0000 7f5a7f7fe640 1 -- 172.21.15.27:0/282256669 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5a8015ec30 msgr2=0x7f5a8014a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:52.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.482+0000 7f5a7f7fe640 1 --2- 172.21.15.27:0/282256669 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5a8015ec30 0x7f5a8014a3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:52.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.482+0000 7f5a7f7fe640 1 -- 172.21.15.27:0/282256669 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a8014b480 con 0x7f5a800b4380 2024-09-12T03:19:52.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.482+0000 7f5a849cb640 1 --2- 172.21.15.27:0/282256669 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5a8015ec30 0x7f5a8014a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:19:52.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.482+0000 7f5a7f7fe640 1 --2- 172.21.15.27:0/282256669 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5a800b4380 0x7f5a800b0e00 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f5a6800e980 tx=0x7f5a6800ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:52.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.482+0000 7f5a7cff9640 1 -- 172.21.15.27:0/282256669 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a68017070 con 0x7f5a800b4380 2024-09-12T03:19:52.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.483+0000 7f5a7cff9640 1 -- 172.21.15.27:0/282256669 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a6800ca00 con 0x7f5a800b4380 2024-09-12T03:19:52.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.483+0000 7f5a8642e640 1 -- 172.21.15.27:0/282256669 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a8014b760 con 0x7f5a800b4380 2024-09-12T03:19:52.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.483+0000 7f5a7cff9640 1 -- 172.21.15.27:0/282256669 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a6800cdd0 con 0x7f5a800b4380 2024-09-12T03:19:52.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.483+0000 7f5a8642e640 1 -- 172.21.15.27:0/282256669 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a8014bd20 con 0x7f5a800b4380 2024-09-12T03:19:52.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.484+0000 7f5a8642e640 1 -- 172.21.15.27:0/282256669 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a8015fd70 con 0x7f5a800b4380 2024-09-12T03:19:52.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.484+0000 7f5a7cff9640 1 -- 172.21.15.27:0/282256669 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 11) ==== 90804+0+0 (secure 0 0 0) 0x7f5a6801a070 con 0x7f5a800b4380 2024-09-12T03:19:52.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.485+0000 7f5a7cff9640 1 --2- 172.21.15.27:0/282256669 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f5a6006c190 0x7f5a6006e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:52.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.485+0000 7f5a7cff9640 1 -- 172.21.15.27:0/282256669 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 4943+0+0 (secure 0 0 0) 0x7f5a6808b3f0 con 0x7f5a800b4380 2024-09-12T03:19:52.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.485+0000 7f5a7ffff640 1 --2- 172.21.15.27:0/282256669 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f5a6006c190 0x7f5a6006e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:52.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.486+0000 7f5a7ffff640 1 --2- 172.21.15.27:0/282256669 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f5a6006c190 0x7f5a6006e650 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f5a70034000 tx=0x7f5a7002e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:52.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.489+0000 7f5a7cff9640 1 -- 172.21.15.27:0/282256669 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f5a68059600 con 0x7f5a800b4380 2024-09-12T03:19:52.665 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.664+0000 7f5a8642e640 1 -- 172.21.15.27:0/282256669 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f5a8006f240 con 0x7f5a800b4380 2024-09-12T03:19:52.670 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.669+0000 7f5a7cff9640 1 -- 172.21.15.27:0/282256669 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v16)=0 v16) ==== 122+0+0 (secure 0 0 0) 0x7f5a6805d900 con 0x7f5a800b4380 2024-09-12T03:19:52.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.672+0000 7f5a8642e640 1 -- 172.21.15.27:0/282256669 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f5a6006c190 msgr2=0x7f5a6006e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:52.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.672+0000 7f5a8642e640 1 --2- 172.21.15.27:0/282256669 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f5a6006c190 0x7f5a6006e650 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f5a70034000 tx=0x7f5a7002e000 comp rx=0 tx=0).stop 2024-09-12T03:19:52.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.672+0000 7f5a8642e640 1 -- 172.21.15.27:0/282256669 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5a800b4380 msgr2=0x7f5a800b0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:52.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.672+0000 7f5a8642e640 1 --2- 172.21.15.27:0/282256669 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5a800b4380 0x7f5a800b0e00 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f5a6800e980 tx=0x7f5a6800ee40 comp rx=0 tx=0).stop 2024-09-12T03:19:52.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.672+0000 7f5a8642e640 1 -- 172.21.15.27:0/282256669 shutdown_connections 2024-09-12T03:19:52.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.672+0000 7f5a8642e640 1 --2- 172.21.15.27:0/282256669 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f5a6006c190 0x7f5a6006e650 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:52.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.672+0000 7f5a8642e640 1 --2- 172.21.15.27:0/282256669 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5a8015ec30 0x7f5a8014a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:52.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.672+0000 7f5a8642e640 1 --2- 172.21.15.27:0/282256669 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5a800b4380 0x7f5a800b0e00 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:52.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.672+0000 7f5a8642e640 1 --2- 172.21.15.27:0/282256669 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f5a800af9d0 0x7f5a800b08c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:52.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.672+0000 7f5a8642e640 1 -- 172.21.15.27:0/282256669 >> 172.21.15.27:0/282256669 conn(0x7f5a8006bb30 msgr2=0x7f5a8006bf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:52.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.672+0000 7f5a8642e640 1 -- 172.21.15.27:0/282256669 shutdown_connections 2024-09-12T03:19:52.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.672+0000 7f5a8642e640 1 -- 172.21.15.27:0/282256669 wait complete. 2024-09-12T03:19:52.682 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-12T03:19:52.683 DEBUG:teuthology.orchestra.run.smithi027:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-12T03:19:52.720 DEBUG:teuthology.orchestra.run.smithi045:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-12T03:19:52.764 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-12T03:19:52.765 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-12T03:19:52.765 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T03:19:52.783 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056004,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497432k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16324 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248716k,nr_inodes=812179,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497432k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-12T03:19:52.784 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-12T03:19:52.784 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T03:19:52.824 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056795,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16602 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249348k,nr_inodes=812337,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-12T03:19:52.824 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-12T03:19:52.824 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T03:19:52.840 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056004,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497432k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16324 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248716k,nr_inodes=812179,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497432k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-12T03:19:52.841 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-12T03:19:52.841 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T03:19:52.883 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056795,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16602 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249348k,nr_inodes=812337,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-12T03:19:52.883 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-12T03:19:52.884 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-12T03:19:52.986 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.986+0000 7fc80a0b7640 1 Processor -- start 2024-09-12T03:19:52.986 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.986+0000 7fc80a0b7640 1 -- start start 2024-09-12T03:19:52.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.986+0000 7fc80a0b7640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fc8040d5200 0x7fc8040d1010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:52.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.986+0000 7fc80a0b7640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fc8040d1550 0x7fc8040b8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:52.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.987+0000 7fc80a0b7640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc8040d3760 con 0x7fc80415eb90 2024-09-12T03:19:52.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.987+0000 7fc80a0b7640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7fc8040b7d20 con 0x7fc8040d5200 2024-09-12T03:19:52.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.987+0000 7fc80a0b7640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fc8040d42e0 con 0x7fc8040d1550 2024-09-12T03:19:52.988 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.987+0000 7fc802ffd640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fc8040d5200 0x7fc8040d1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:52.988 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.987+0000 7fc8037fe640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fc80415eb90 0x7fc8040d2df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:53100/0 (socket says 172.21.15.27:53100) 2024-09-12T03:19:52.988 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.987+0000 7fc802ffd640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fc8040d5200 0x7fc8040d1010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35486/0 (socket says 172.21.15.27:35486) 2024-09-12T03:19:52.988 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.987+0000 7fc8037fe640 1 -- 172.21.15.27:0/3005543687 learned_addr learned my addr 172.21.15.27:0/3005543687 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:19:52.988 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.987+0000 7fc803fff640 1 --2- 172.21.15.27:0/3005543687 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fc8040d1550 0x7fc8040b8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:52.988 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.987+0000 7fc802ffd640 1 -- 172.21.15.27:0/3005543687 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fc8040d1550 msgr2=0x7fc8040b8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:52.988 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.987+0000 7fc802ffd640 1 --2- 172.21.15.27:0/3005543687 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fc8040d1550 0x7fc8040b8190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:52.988 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.987+0000 7fc802ffd640 1 -- 172.21.15.27:0/3005543687 >> v1:172.21.15.27:6789/0 conn(0x7fc80415eb90 legacy=0x7fc8040d2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:52.988 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.987+0000 7fc8027fc640 1 -- 172.21.15.27:0/3005543687 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2910248672 0 0) 0x7fc8040d3760 con 0x7fc80415eb90 2024-09-12T03:19:52.988 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.987+0000 7fc802ffd640 1 -- 172.21.15.27:0/3005543687 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8040d3a40 con 0x7fc8040d5200 2024-09-12T03:19:52.988 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.987+0000 7fc803fff640 1 --2- 172.21.15.27:0/3005543687 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fc8040d1550 0x7fc8040b8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:19:52.990 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.988+0000 7fc802ffd640 1 --2- 172.21.15.27:0/3005543687 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc8040d5200 0x7fc8040d1010 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fc7f80097a0 tx=0x7fc7f802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a661062bbe3704a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:52.990 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.988+0000 7fc8027fc640 1 -- 172.21.15.27:0/3005543687 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7f8037070 con 0x7fc8040d5200 2024-09-12T03:19:52.990 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.988+0000 7fc8027fc640 1 -- 172.21.15.27:0/3005543687 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7f8032440 con 0x7fc8040d5200 2024-09-12T03:19:52.990 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.988+0000 7fc8027fc640 1 -- 172.21.15.27:0/3005543687 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7f8032810 con 0x7fc8040d5200 2024-09-12T03:19:52.990 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.988+0000 7fc80a0b7640 1 -- 172.21.15.27:0/3005543687 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc8040d5200 msgr2=0x7fc8040d1010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:52.990 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.988+0000 7fc80a0b7640 1 --2- 172.21.15.27:0/3005543687 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc8040d5200 0x7fc8040d1010 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fc7f80097a0 tx=0x7fc7f802a180 comp rx=0 tx=0).stop 2024-09-12T03:19:52.990 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.988+0000 7fc80a0b7640 1 -- 172.21.15.27:0/3005543687 shutdown_connections 2024-09-12T03:19:52.990 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.988+0000 7fc80a0b7640 1 --2- 172.21.15.27:0/3005543687 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fc8040d1550 0x7fc8040b8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:52.991 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.988+0000 7fc80a0b7640 1 --2- 172.21.15.27:0/3005543687 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc8040d5200 0x7fc8040d1010 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:52.991 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.988+0000 7fc80a0b7640 1 -- 172.21.15.27:0/3005543687 >> 172.21.15.27:0/3005543687 conn(0x7fc80406dce0 msgr2=0x7fc80406e0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:52.991 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.988+0000 7fc80a0b7640 1 -- 172.21.15.27:0/3005543687 shutdown_connections 2024-09-12T03:19:52.991 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.989+0000 7fc80a0b7640 1 -- 172.21.15.27:0/3005543687 wait complete. 2024-09-12T03:19:52.991 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.990+0000 7fc80a0b7640 1 Processor -- start 2024-09-12T03:19:52.991 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.990+0000 7fc80a0b7640 1 -- start start 2024-09-12T03:19:52.993 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.991+0000 7fc80a0b7640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc8040d1550 0x7fc8040abbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:52.993 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.991+0000 7fc80a0b7640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc8040d5200 0x7fc8040ca7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:52.993 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.991+0000 7fc80a0b7640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fc80415eb90 0x7fc8040aca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:52.993 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.991+0000 7fc80a0b7640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fc8040d4000 con 0x7fc8040d1550 2024-09-12T03:19:52.993 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.991+0000 7fc80a0b7640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc8040d3500 con 0x7fc8040d5200 2024-09-12T03:19:52.993 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.991+0000 7fc80a0b7640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fc8040d19a0 con 0x7fc80415eb90 2024-09-12T03:19:52.993 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.991+0000 7fc8037fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc8040d1550 0x7fc8040abbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:52.994 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.991+0000 7fc8037fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc8040d1550 0x7fc8040abbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35500/0 (socket says 172.21.15.27:35500) 2024-09-12T03:19:52.994 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.991+0000 7fc8037fe640 1 -- 172.21.15.27:0/1498183053 learned_addr learned my addr 172.21.15.27:0/1498183053 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:19:52.994 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.991+0000 7fc803fff640 1 --2- 172.21.15.27:0/1498183053 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fc80415eb90 0x7fc8040aca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:52.994 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.991+0000 7fc802ffd640 1 --2- 172.21.15.27:0/1498183053 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc8040d5200 0x7fc8040ca7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:52.994 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.991+0000 7fc8037fe640 1 -- 172.21.15.27:0/1498183053 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fc80415eb90 msgr2=0x7fc8040aca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:52.994 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.991+0000 7fc8037fe640 1 --2- 172.21.15.27:0/1498183053 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fc80415eb90 0x7fc8040aca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:52.994 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.991+0000 7fc8037fe640 1 -- 172.21.15.27:0/1498183053 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc8040d5200 msgr2=0x7fc8040ca7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:52.994 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.991+0000 7fc8037fe640 1 --2- 172.21.15.27:0/1498183053 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc8040d5200 0x7fc8040ca7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:52.994 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.991+0000 7fc8037fe640 1 -- 172.21.15.27:0/1498183053 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8040ac4c0 con 0x7fc8040d1550 2024-09-12T03:19:52.994 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.991+0000 7fc8037fe640 1 --2- 172.21.15.27:0/1498183053 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc8040d1550 0x7fc8040abbf0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fc7f400c430 tx=0x7fc7f400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:52.994 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.991+0000 7fc7ebfff640 1 -- 172.21.15.27:0/1498183053 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7f40074e0 con 0x7fc8040d1550 2024-09-12T03:19:52.994 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.991+0000 7fc7ebfff640 1 -- 172.21.15.27:0/1498183053 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7f40047e0 con 0x7fc8040d1550 2024-09-12T03:19:52.995 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.991+0000 7fc80a0b7640 1 -- 172.21.15.27:0/1498183053 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8040cad00 con 0x7fc8040d1550 2024-09-12T03:19:52.995 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.991+0000 7fc7ebfff640 1 -- 172.21.15.27:0/1498183053 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7f4004bd0 con 0x7fc8040d1550 2024-09-12T03:19:52.995 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.992+0000 7fc80a0b7640 1 -- 172.21.15.27:0/1498183053 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8040c8450 con 0x7fc8040d1550 2024-09-12T03:19:52.995 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.993+0000 7fc80a0b7640 1 -- 172.21.15.27:0/1498183053 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc80415f090 con 0x7fc8040d1550 2024-09-12T03:19:52.995 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.993+0000 7fc7ebfff640 1 -- 172.21.15.27:0/1498183053 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 11) ==== 90804+0+0 (secure 0 0 0) 0x7fc7f4011070 con 0x7fc8040d1550 2024-09-12T03:19:52.995 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.993+0000 7fc7ebfff640 1 --2- 172.21.15.27:0/1498183053 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fc7f006c190 0x7fc7f006e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:52.995 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.993+0000 7fc7ebfff640 1 -- 172.21.15.27:0/1498183053 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 4943+0+0 (secure 0 0 0) 0x7fc7f408ac80 con 0x7fc8040d1550 2024-09-12T03:19:52.995 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.994+0000 7fc802ffd640 1 --2- 172.21.15.27:0/1498183053 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fc7f006c190 0x7fc7f006e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:52.996 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.994+0000 7fc802ffd640 1 --2- 172.21.15.27:0/1498183053 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fc7f006c190 0x7fc7f006e650 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fc7f8034000 tx=0x7fc7f802e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:52.997 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:52.998+0000 7fc7ebfff640 1 -- 172.21.15.27:0/1498183053 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fc7f4091080 con 0x7fc8040d1550 2024-09-12T03:19:53.208 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:53.208+0000 7fc80a0b7640 1 -- 172.21.15.27:0/1498183053 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc804161ba0 con 0x7fc8040d1550 2024-09-12T03:19:53.209 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:53.209+0000 7fc7ebfff640 1 -- 172.21.15.27:0/1498183053 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+3908 (secure 0 0 0) 0x7fc7f4058dd0 con 0x7fc8040d1550 2024-09-12T03:19:53.209 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:19:53.209 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":33,"btime":"2024-09-12T03:19:48:338265+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5102,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/565788762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":565788762},{"type":"v1","addr":"172.21.15.27:6835","nonce":565788762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26},{"gid":5413,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6836/2689309305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":2689309305},{"type":"v1","addr":"172.21.15.45:6836","nonce":2689309305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":5640,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6834/3694936463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":3694936463},{"type":"v1","addr":"172.21.15.27:6834","nonce":3694936463}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26},{"gid":5665,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/2189955603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6835","nonce":2189955603},{"type":"v1","addr":"172.21.15.45:6837","nonce":2189955603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":33}],"filesystems":[]} 2024-09-12T03:19:53.210 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 33 2024-09-12T03:19:53.211 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:53.211+0000 7fc80a0b7640 1 -- 172.21.15.27:0/1498183053 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fc7f006c190 msgr2=0x7fc7f006e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:53.211 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:53.211+0000 7fc80a0b7640 1 --2- 172.21.15.27:0/1498183053 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fc7f006c190 0x7fc7f006e650 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fc7f8034000 tx=0x7fc7f802e000 comp rx=0 tx=0).stop 2024-09-12T03:19:53.211 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:53.211+0000 7fc80a0b7640 1 -- 172.21.15.27:0/1498183053 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc8040d1550 msgr2=0x7fc8040abbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:53.211 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:53.211+0000 7fc80a0b7640 1 --2- 172.21.15.27:0/1498183053 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc8040d1550 0x7fc8040abbf0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fc7f400c430 tx=0x7fc7f400c8f0 comp rx=0 tx=0).stop 2024-09-12T03:19:53.211 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:53.211+0000 7fc80a0b7640 1 -- 172.21.15.27:0/1498183053 shutdown_connections 2024-09-12T03:19:53.211 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:53.211+0000 7fc80a0b7640 1 --2- 172.21.15.27:0/1498183053 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fc7f006c190 0x7fc7f006e650 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:53.212 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:53.211+0000 7fc80a0b7640 1 --2- 172.21.15.27:0/1498183053 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fc80415eb90 0x7fc8040aca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:53.212 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:53.211+0000 7fc80a0b7640 1 --2- 172.21.15.27:0/1498183053 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc8040d5200 0x7fc8040ca7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:53.212 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:53.211+0000 7fc80a0b7640 1 --2- 172.21.15.27:0/1498183053 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc8040d1550 0x7fc8040abbf0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:53.212 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:53.211+0000 7fc80a0b7640 1 -- 172.21.15.27:0/1498183053 >> 172.21.15.27:0/1498183053 conn(0x7fc80406dce0 msgr2=0x7fc804160bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:53.212 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:53.211+0000 7fc80a0b7640 1 -- 172.21.15.27:0/1498183053 shutdown_connections 2024-09-12T03:19:53.212 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:53.211+0000 7fc80a0b7640 1 -- 172.21.15.27:0/1498183053 wait complete. 2024-09-12T03:19:53.222 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-12T03:19:53.222 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-12T03:19:53.222 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-12T03:19:53.222 INFO:teuthology.orchestra.run:waiting for 300 2024-09-12T03:19:53.222 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-12T03:19:53.223 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-12T03:19:53.223 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-12T03:19:53.223 INFO:teuthology.orchestra.run:waiting for 300 2024-09-12T03:19:53.223 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-12T03:19:53.223 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-12T03:19:53.223 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-12T03:19:53.223 INFO:teuthology.orchestra.run:waiting for 300 2024-09-12T03:19:53.224 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-12T03:19:53.224 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-12T03:19:53.224 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-12T03:19:53.225 INFO:teuthology.orchestra.run:waiting for 300 2024-09-12T03:19:53.232 INFO:tasks.ceph.mds.a:Stopped 2024-09-12T03:19:53.232 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-12T03:19:53.233 INFO:tasks.ceph.mds.c:Stopped 2024-09-12T03:19:53.233 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-12T03:19:53.236 INFO:tasks.ceph.mds.b:Stopped 2024-09-12T03:19:53.237 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-12T03:19:53.237 INFO:tasks.ceph.mds.d:Stopped 2024-09-12T03:19:53.238 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-12T03:19:53.275 INFO:tasks.ceph.mds.a:Started 2024-09-12T03:19:53.276 DEBUG:teuthology.parallel:result is None 2024-09-12T03:19:53.276 INFO:tasks.ceph.mds.c:Started 2024-09-12T03:19:53.276 DEBUG:teuthology.parallel:result is None 2024-09-12T03:19:53.279 INFO:tasks.ceph.mds.b:Started 2024-09-12T03:19:53.279 DEBUG:teuthology.parallel:result is None 2024-09-12T03:19:53.280 INFO:tasks.ceph.mds.d:Started 2024-09-12T03:19:53.280 DEBUG:teuthology.parallel:result is None 2024-09-12T03:19:53.280 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-12T03:19:53.353 INFO:tasks.ceph.mds.a.smithi027.stdout:starting mds.a at 2024-09-12T03:19:53.354 INFO:tasks.ceph.mds.c.smithi027.stdout:starting mds.c at 2024-09-12T03:19:53.370 INFO:tasks.ceph.mds.b.smithi045.stdout:starting mds.b at 2024-09-12T03:19:53.370 INFO:tasks.ceph.mds.d.smithi045.stdout:starting mds.d at 2024-09-12T03:19:53.390 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:53.390+0000 7f9260a2e640 1 Processor -- start 2024-09-12T03:19:53.390 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:53.390+0000 7f9260a2e640 1 -- start start 2024-09-12T03:19:53.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:53.391+0000 7f9260a2e640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f925c0d5200 0x7f925c0d1010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:53.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:53.391+0000 7f9260a2e640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f925c0d1550 0x7f925c0b8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:53.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:53.391+0000 7f9260a2e640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f925c0d3760 con 0x7f925c15eb90 2024-09-12T03:19:53.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:53.391+0000 7f9260a2e640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f925c0b7d20 con 0x7f925c0d1550 2024-09-12T03:19:53.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:53.391+0000 7f9260a2e640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f925c0d42e0 con 0x7f925c0d5200 2024-09-12T03:19:53.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:53.392+0000 7f925ad9d640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f925c0d1550 0x7f925c0b8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:53.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:53.392+0000 7f925ad9d640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f925c0d1550 0x7f925c0b8190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35546/0 (socket says 172.21.15.27:35546) 2024-09-12T03:19:53.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:53.392+0000 7f925ad9d640 1 -- 172.21.15.27:0/4036460811 learned_addr learned my addr 172.21.15.27:0/4036460811 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:19:53.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:53.392+0000 7f9259d9b640 1 --2- 172.21.15.27:0/4036460811 >> v2:172.21.15.45:3300/0 conn(0x7f925c0d5200 0x7f925c0d1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:53.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:53.392+0000 7f925ad9d640 1 -- 172.21.15.27:0/4036460811 >> v2:172.21.15.45:3300/0 conn(0x7f925c0d5200 msgr2=0x7f925c0d1010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:53.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:53.392+0000 7f925ad9d640 1 --2- 172.21.15.27:0/4036460811 >> v2:172.21.15.45:3300/0 conn(0x7f925c0d5200 0x7f925c0d1010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:53.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:53.392+0000 7f925ad9d640 1 -- 172.21.15.27:0/4036460811 >> v1:172.21.15.45:6789/0 conn(0x7f925c15eb90 legacy=0x7f925c0d2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:53.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:53.392+0000 7f925ad9d640 1 -- 172.21.15.27:0/4036460811 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f925c0d3a40 con 0x7f925c0d1550 2024-09-12T03:19:53.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:53.393+0000 7f925ad9d640 1 --2- 172.21.15.27:0/4036460811 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f925c0d1550 0x7f925c0b8190 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f925c0d4750 tx=0x7f925402a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3082e24effd0a87 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:53.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:53.393+0000 7f925959a640 1 -- 172.21.15.27:0/4036460811 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9254036070 con 0x7f925c0d1550 2024-09-12T03:19:53.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:53.393+0000 7f925959a640 1 -- 172.21.15.27:0/4036460811 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9254031600 con 0x7f925c0d1550 2024-09-12T03:19:53.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:53.393+0000 7f925959a640 1 -- 172.21.15.27:0/4036460811 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92540319d0 con 0x7f925c0d1550 2024-09-12T03:19:53.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:53.393+0000 7f9260a2e640 1 -- 172.21.15.27:0/4036460811 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f925c0d1550 msgr2=0x7f925c0b8190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:53.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:53.393+0000 7f9260a2e640 1 --2- 172.21.15.27:0/4036460811 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f925c0d1550 0x7f925c0b8190 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f925c0d4750 tx=0x7f925402a730 comp rx=0 tx=0).stop 2024-09-12T03:19:53.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:53.394+0000 7f9260a2e640 1 -- 172.21.15.27:0/4036460811 shutdown_connections 2024-09-12T03:19:53.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:53.394+0000 7f9260a2e640 1 --2- 172.21.15.27:0/4036460811 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f925c0d1550 0x7f925c0b8190 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:53.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:53.394+0000 7f9260a2e640 1 --2- 172.21.15.27:0/4036460811 >> v2:172.21.15.45:3300/0 conn(0x7f925c0d5200 0x7f925c0d1010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:53.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:53.394+0000 7f9260a2e640 1 -- 172.21.15.27:0/4036460811 >> 172.21.15.27:0/4036460811 conn(0x7f925c06dce0 msgr2=0x7f925c06e0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:53.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:53.394+0000 7f9260a2e640 1 -- 172.21.15.27:0/4036460811 shutdown_connections 2024-09-12T03:19:53.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:53.394+0000 7f9260a2e640 1 -- 172.21.15.27:0/4036460811 wait complete. 2024-09-12T03:19:53.397 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:53.395+0000 7f9260a2e640 1 Processor -- start 2024-09-12T03:19:53.397 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:53.395+0000 7f9260a2e640 1 -- start start 2024-09-12T03:19:53.397 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:53.396+0000 7f9260a2e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f925c0d1550 0x7f925c14fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:53.397 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:53.396+0000 7f9260a2e640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f925c0d5200 0x7f925c0c97e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:53.397 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:53.396+0000 7f9260a2e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f925c15eb90 0x7f925c0ad850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:53.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:53.396+0000 7f9260a2e640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f925c0d4000 con 0x7f925c0d1550 2024-09-12T03:19:53.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:53.396+0000 7f9260a2e640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f925c0d3500 con 0x7f925c15eb90 2024-09-12T03:19:53.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:53.396+0000 7f9260a2e640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f925c0d1960 con 0x7f925c0d5200 2024-09-12T03:19:53.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:53.396+0000 7f925a59c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f925c0d1550 0x7f925c14fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:53.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:53.396+0000 7f925a59c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f925c0d1550 0x7f925c14fbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35560/0 (socket says 172.21.15.27:35560) 2024-09-12T03:19:53.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:53.396+0000 7f925a59c640 1 -- 172.21.15.27:0/3877812729 learned_addr learned my addr 172.21.15.27:0/3877812729 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:19:53.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:53.396+0000 7f9259d9b640 1 --2- 172.21.15.27:0/3877812729 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f925c0d5200 0x7f925c0c97e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:53.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:53.396+0000 7f925ad9d640 1 --2- 172.21.15.27:0/3877812729 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f925c15eb90 0x7f925c0ad850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:53.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:53.396+0000 7f925a59c640 1 -- 172.21.15.27:0/3877812729 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f925c0d5200 msgr2=0x7f925c0c97e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:53.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:53.396+0000 7f925a59c640 1 --2- 172.21.15.27:0/3877812729 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f925c0d5200 0x7f925c0c97e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:53.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:53.396+0000 7f925a59c640 1 -- 172.21.15.27:0/3877812729 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f925c15eb90 msgr2=0x7f925c0ad850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:53.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:53.396+0000 7f925a59c640 1 --2- 172.21.15.27:0/3877812729 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f925c15eb90 0x7f925c0ad850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:53.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:53.396+0000 7f925a59c640 1 -- 172.21.15.27:0/3877812729 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f925c0ac0e0 con 0x7f925c0d1550 2024-09-12T03:19:53.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:53.396+0000 7f925ad9d640 1 --2- 172.21.15.27:0/3877812729 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f925c15eb90 0x7f925c0ad850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:19:53.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:53.396+0000 7f925a59c640 1 --2- 172.21.15.27:0/3877812729 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f925c0d1550 0x7f925c14fbf0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f924400c400 tx=0x7f924400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:53.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:53.396+0000 7f9252ffd640 1 -- 172.21.15.27:0/3877812729 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92440052d0 con 0x7f925c0d1550 2024-09-12T03:19:53.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:53.396+0000 7f9252ffd640 1 -- 172.21.15.27:0/3877812729 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9244005490 con 0x7f925c0d1550 2024-09-12T03:19:53.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:53.396+0000 7f9252ffd640 1 -- 172.21.15.27:0/3877812729 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9244007260 con 0x7f925c0d1550 2024-09-12T03:19:53.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:53.397+0000 7f9260a2e640 1 -- 172.21.15.27:0/3877812729 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f925c0ac3c0 con 0x7f925c0d1550 2024-09-12T03:19:53.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:53.397+0000 7f9260a2e640 1 -- 172.21.15.27:0/3877812729 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f925c0ac980 con 0x7f925c0d1550 2024-09-12T03:19:53.400 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:53.398+0000 7f9260a2e640 1 -- 172.21.15.27:0/3877812729 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f925c15f090 con 0x7f925c0d1550 2024-09-12T03:19:53.401 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:53.398+0000 7f9252ffd640 1 -- 172.21.15.27:0/3877812729 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 11) ==== 90804+0+0 (secure 0 0 0) 0x7f9244007400 con 0x7f925c0d1550 2024-09-12T03:19:53.401 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:53.399+0000 7f9252ffd640 1 --2- 172.21.15.27:0/3877812729 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f923c06c190 0x7f923c06e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:53.401 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:53.399+0000 7f9252ffd640 1 -- 172.21.15.27:0/3877812729 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 4943+0+0 (secure 0 0 0) 0x7f924408a530 con 0x7f925c0d1550 2024-09-12T03:19:53.401 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:53.399+0000 7f9259d9b640 1 --2- 172.21.15.27:0/3877812729 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f923c06c190 0x7f923c06e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:53.401 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:53.400+0000 7f9259d9b640 1 --2- 172.21.15.27:0/3877812729 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f923c06c190 0x7f923c06e650 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f925c0ca940 tx=0x7f924c008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:53.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:53.403+0000 7f9252ffd640 1 -- 172.21.15.27:0/3877812729 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f9244058780 con 0x7f925c0d1550 2024-09-12T03:19:53.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:53.589+0000 7f9260a2e640 1 -- 172.21.15.27:0/3877812729 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f925c0b93e0 con 0x7f925c0d1550 2024-09-12T03:19:54.365 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:54.365+0000 7f9252ffd640 1 -- 172.21.15.27:0/3877812729 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v63) ==== 97+0+0 (secure 0 0 0) 0x7f924405ca80 con 0x7f925c0d1550 2024-09-12T03:19:54.365 INFO:teuthology.orchestra.run.smithi027.stderr: removed all blocklist entries 2024-09-12T03:19:54.368 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:54.368+0000 7f9260a2e640 1 -- 172.21.15.27:0/3877812729 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f923c06c190 msgr2=0x7f923c06e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:54.369 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:54.368+0000 7f9260a2e640 1 --2- 172.21.15.27:0/3877812729 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f923c06c190 0x7f923c06e650 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f925c0ca940 tx=0x7f924c008040 comp rx=0 tx=0).stop 2024-09-12T03:19:54.369 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:54.368+0000 7f9260a2e640 1 -- 172.21.15.27:0/3877812729 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f925c0d1550 msgr2=0x7f925c14fbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:54.369 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:54.368+0000 7f9260a2e640 1 --2- 172.21.15.27:0/3877812729 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f925c0d1550 0x7f925c14fbf0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f924400c400 tx=0x7f924400c8c0 comp rx=0 tx=0).stop 2024-09-12T03:19:54.369 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:54.368+0000 7f9260a2e640 1 -- 172.21.15.27:0/3877812729 shutdown_connections 2024-09-12T03:19:54.369 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:54.368+0000 7f9260a2e640 1 --2- 172.21.15.27:0/3877812729 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f923c06c190 0x7f923c06e650 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:54.369 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:54.368+0000 7f9260a2e640 1 --2- 172.21.15.27:0/3877812729 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f925c15eb90 0x7f925c0ad850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:54.369 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:54.368+0000 7f9260a2e640 1 --2- 172.21.15.27:0/3877812729 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f925c0d5200 0x7f925c0c97e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:54.369 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:54.368+0000 7f9260a2e640 1 --2- 172.21.15.27:0/3877812729 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f925c0d1550 0x7f925c14fbf0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:54.369 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:54.368+0000 7f9260a2e640 1 -- 172.21.15.27:0/3877812729 >> 172.21.15.27:0/3877812729 conn(0x7f925c06dce0 msgr2=0x7f925c06e0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:54.369 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:54.369+0000 7f9260a2e640 1 -- 172.21.15.27:0/3877812729 shutdown_connections 2024-09-12T03:19:54.369 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:54.369+0000 7f9260a2e640 1 -- 172.21.15.27:0/3877812729 wait complete. 2024-09-12T03:19:54.382 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-12T03:19:54.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:54.492+0000 7f298774b640 1 Processor -- start 2024-09-12T03:19:54.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:54.492+0000 7f298774b640 1 -- start start 2024-09-12T03:19:54.494 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:54.493+0000 7f298774b640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f298015e780 0x7f29800d3e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:54.494 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:54.493+0000 7f298774b640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f29800d1990 0x7f29800b3390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:54.494 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:54.493+0000 7f298774b640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f29800b6690 con 0x7f29800d43b0 2024-09-12T03:19:54.494 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:54.493+0000 7f298774b640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f29800b7950 con 0x7f298015e780 2024-09-12T03:19:54.494 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:54.493+0000 7f298774b640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f29800b7210 con 0x7f29800d1990 2024-09-12T03:19:54.494 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:54.493+0000 7f29854e7640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f298015e780 0x7f29800d3e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:54.494 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:54.493+0000 7f2984ce6640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f29800d43b0 0x7f29800d1280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:53116/0 (socket says 172.21.15.27:53116) 2024-09-12T03:19:54.495 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:54.493+0000 7f2984ce6640 1 -- 172.21.15.27:0/1725638351 learned_addr learned my addr 172.21.15.27:0/1725638351 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:19:54.495 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:54.493+0000 7f2985ce8640 1 --2- 172.21.15.27:0/1725638351 >> v2:172.21.15.45:3300/0 conn(0x7f29800d1990 0x7f29800b3390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:54.495 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:54.493+0000 7f29854e7640 1 -- 172.21.15.27:0/1725638351 >> v2:172.21.15.45:3300/0 conn(0x7f29800d1990 msgr2=0x7f29800b3390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:54.495 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:54.493+0000 7f29854e7640 1 --2- 172.21.15.27:0/1725638351 >> v2:172.21.15.45:3300/0 conn(0x7f29800d1990 0x7f29800b3390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:54.495 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:54.493+0000 7f29854e7640 1 -- 172.21.15.27:0/1725638351 >> v1:172.21.15.27:6789/0 conn(0x7f29800d43b0 legacy=0x7f29800d1280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:54.495 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:54.493+0000 7f29854e7640 1 -- 172.21.15.27:0/1725638351 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29800b6970 con 0x7f298015e780 2024-09-12T03:19:54.495 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:54.494+0000 7f29854e7640 1 --2- 172.21.15.27:0/1725638351 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f298015e780 0x7f29800d3e70 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f29800d4820 tx=0x7f297c02a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea29b0d8058f68c4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:54.495 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:54.494+0000 7f2977fff640 1 -- 172.21.15.27:0/1725638351 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f297c037070 con 0x7f298015e780 2024-09-12T03:19:54.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:54.494+0000 7f2977fff640 1 -- 172.21.15.27:0/1725638351 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f297c032440 con 0x7f298015e780 2024-09-12T03:19:54.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:54.494+0000 7f2977fff640 1 -- 172.21.15.27:0/1725638351 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f297c02ae60 con 0x7f298015e780 2024-09-12T03:19:54.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:54.495+0000 7f298774b640 1 -- 172.21.15.27:0/1725638351 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f298015e780 msgr2=0x7f29800d3e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:54.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:54.495+0000 7f298774b640 1 --2- 172.21.15.27:0/1725638351 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f298015e780 0x7f29800d3e70 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f29800d4820 tx=0x7f297c02a110 comp rx=0 tx=0).stop 2024-09-12T03:19:54.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:54.495+0000 7f298774b640 1 -- 172.21.15.27:0/1725638351 shutdown_connections 2024-09-12T03:19:54.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:54.495+0000 7f298774b640 1 --2- 172.21.15.27:0/1725638351 >> v2:172.21.15.45:3300/0 conn(0x7f29800d1990 0x7f29800b3390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:54.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:54.495+0000 7f298774b640 1 --2- 172.21.15.27:0/1725638351 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f298015e780 0x7f29800d3e70 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:54.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:54.495+0000 7f298774b640 1 -- 172.21.15.27:0/1725638351 >> 172.21.15.27:0/1725638351 conn(0x7f298006da60 msgr2=0x7f298006de70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:54.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:54.495+0000 7f298774b640 1 -- 172.21.15.27:0/1725638351 shutdown_connections 2024-09-12T03:19:54.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:54.495+0000 7f298774b640 1 -- 172.21.15.27:0/1725638351 wait complete. 2024-09-12T03:19:54.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:54.496+0000 7f298774b640 1 Processor -- start 2024-09-12T03:19:54.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:54.496+0000 7f298774b640 1 -- start start 2024-09-12T03:19:54.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:54.497+0000 7f298774b640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f29800d1990 0x7f29800ce670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:54.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:54.497+0000 7f298774b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f29800d43b0 0x7f29801f4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:54.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:54.497+0000 7f298774b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f298015e780 0x7f29800cc230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:54.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:54.497+0000 7f298774b640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f29800b6340 con 0x7f298015e780 2024-09-12T03:19:54.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:54.497+0000 7f298774b640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f29800b6f30 con 0x7f29800d43b0 2024-09-12T03:19:54.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:54.497+0000 7f298774b640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f298015eb60 con 0x7f29800d1990 2024-09-12T03:19:54.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:54.497+0000 7f2985ce8640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f298015e780 0x7f29800cc230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:54.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:54.497+0000 7f2985ce8640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f298015e780 0x7f29800cc230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35576/0 (socket says 172.21.15.27:35576) 2024-09-12T03:19:54.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:54.497+0000 7f2985ce8640 1 -- 172.21.15.27:0/1381707338 learned_addr learned my addr 172.21.15.27:0/1381707338 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:19:54.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:54.497+0000 7f2984ce6640 1 --2- 172.21.15.27:0/1381707338 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f29800d43b0 0x7f29801f4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:54.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:54.497+0000 7f29854e7640 1 --2- 172.21.15.27:0/1381707338 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f29800d1990 0x7f29800ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:54.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:54.497+0000 7f2985ce8640 1 -- 172.21.15.27:0/1381707338 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f29800d1990 msgr2=0x7f29800ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:54.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:54.497+0000 7f2985ce8640 1 --2- 172.21.15.27:0/1381707338 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f29800d1990 0x7f29800ce670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:54.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:54.497+0000 7f2985ce8640 1 -- 172.21.15.27:0/1381707338 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f29800d43b0 msgr2=0x7f29801f4240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:54.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:54.497+0000 7f2985ce8640 1 --2- 172.21.15.27:0/1381707338 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f29800d43b0 0x7f29801f4240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:54.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:54.497+0000 7f2985ce8640 1 -- 172.21.15.27:0/1381707338 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29800cebb0 con 0x7f298015e780 2024-09-12T03:19:54.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:54.497+0000 7f29854e7640 1 --2- 172.21.15.27:0/1381707338 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f29800d1990 0x7f29800ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:19:54.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:54.497+0000 7f2985ce8640 1 --2- 172.21.15.27:0/1381707338 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f298015e780 0x7f29800cc230 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f2978007e70 tx=0x7f297800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:54.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:54.498+0000 7f2975ffb640 1 -- 172.21.15.27:0/1381707338 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2978014620 con 0x7f298015e780 2024-09-12T03:19:54.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:54.498+0000 7f2975ffb640 1 -- 172.21.15.27:0/1381707338 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2978019070 con 0x7f298015e780 2024-09-12T03:19:54.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:54.498+0000 7f298774b640 1 -- 172.21.15.27:0/1381707338 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29801f4bb0 con 0x7f298015e780 2024-09-12T03:19:54.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:54.498+0000 7f2975ffb640 1 -- 172.21.15.27:0/1381707338 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2978014a20 con 0x7f298015e780 2024-09-12T03:19:54.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:54.498+0000 7f298774b640 1 -- 172.21.15.27:0/1381707338 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29801f5170 con 0x7f298015e780 2024-09-12T03:19:54.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:54.499+0000 7f298774b640 1 -- 172.21.15.27:0/1381707338 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f298006e070 con 0x7f298015e780 2024-09-12T03:19:54.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:54.499+0000 7f2975ffb640 1 -- 172.21.15.27:0/1381707338 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 11) ==== 90804+0+0 (secure 0 0 0) 0x7f2978014bc0 con 0x7f298015e780 2024-09-12T03:19:54.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:54.500+0000 7f2975ffb640 1 --2- 172.21.15.27:0/1381707338 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f296406c190 0x7f296406e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:54.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:54.500+0000 7f2975ffb640 1 -- 172.21.15.27:0/1381707338 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 4857+0+0 (secure 0 0 0) 0x7f297808b8f0 con 0x7f298015e780 2024-09-12T03:19:54.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:54.500+0000 7f29854e7640 1 --2- 172.21.15.27:0/1381707338 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f296406c190 0x7f296406e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:54.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:54.501+0000 7f29854e7640 1 --2- 172.21.15.27:0/1381707338 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f296406c190 0x7f296406e650 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f297c034000 tx=0x7f297c02e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:54.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:54.504+0000 7f2975ffb640 1 -- 172.21.15.27:0/1381707338 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f297805a1e0 con 0x7f298015e780 2024-09-12T03:19:54.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:54.751+0000 7f298774b640 1 -- 172.21.15.27:0/1381707338 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f29800d3e20 con 0x7f298015e780 2024-09-12T03:19:54.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:54.752+0000 7f2975ffb640 1 -- 172.21.15.27:0/1381707338 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v14) ==== 80+0+5617 (secure 0 0 0) 0x7f2978059ae0 con 0x7f298015e780 2024-09-12T03:19:54.752 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:19:54.752 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-12T03:19:54.752 INFO:teuthology.orchestra.run.smithi027.stdout: "auth_dump": [ 2024-09-12T03:19:54.752 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:19:54.753 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "mds.a", 2024-09-12T03:19:54.753 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDUXOJmIhxjNxAAaGg8SwtW6NRe/lIgNCV20w==", 2024-09-12T03:19:54.753 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:19:54.753 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow", 2024-09-12T03:19:54.753 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow *", 2024-09-12T03:19:54.753 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow *", 2024-09-12T03:19:54.753 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2024-09-12T03:19:54.753 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:19:54.753 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:19:54.753 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:19:54.753 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "mds.b", 2024-09-12T03:19:54.753 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDVXOJm63kWCBAAzjfOsBsWUI+IAsuZ49yEkw==", 2024-09-12T03:19:54.753 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:19:54.754 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow", 2024-09-12T03:19:54.754 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow *", 2024-09-12T03:19:54.754 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow *", 2024-09-12T03:19:54.754 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2024-09-12T03:19:54.754 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:19:54.754 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:19:54.754 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:19:54.754 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "mds.c", 2024-09-12T03:19:54.754 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDVXOJm0sX7AhAAdynCcho2a/AgFF9Cw3+vAw==", 2024-09-12T03:19:54.754 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:19:54.754 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow", 2024-09-12T03:19:54.754 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow *", 2024-09-12T03:19:54.754 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow *", 2024-09-12T03:19:54.754 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2024-09-12T03:19:54.755 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:19:54.755 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:19:54.755 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:19:54.755 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "mds.d", 2024-09-12T03:19:54.755 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDVXOJm7Dd8DxAANHYmvKgpRZ6sVfXDSgPHWw==", 2024-09-12T03:19:54.755 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:19:54.755 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow", 2024-09-12T03:19:54.755 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow *", 2024-09-12T03:19:54.755 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow *", 2024-09-12T03:19:54.755 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2024-09-12T03:19:54.755 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:19:54.755 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:19:54.755 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:19:54.755 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "osd.0", 2024-09-12T03:19:54.756 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDYXOJmJTVYLxAAmo8kxRjaiwTT44dZLZlq4A==", 2024-09-12T03:19:54.756 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:19:54.756 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow profile osd", 2024-09-12T03:19:54.756 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile osd", 2024-09-12T03:19:54.756 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2024-09-12T03:19:54.756 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:19:54.756 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:19:54.756 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:19:54.756 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "osd.1", 2024-09-12T03:19:54.756 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDcXOJmYV0BFBAAmoGX9gf9yLoE/qkWFnv9tg==", 2024-09-12T03:19:54.756 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:19:54.756 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow profile osd", 2024-09-12T03:19:54.756 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile osd", 2024-09-12T03:19:54.757 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2024-09-12T03:19:54.757 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:19:54.757 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:19:54.757 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:19:54.757 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "osd.2", 2024-09-12T03:19:54.757 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDfXOJmEetiJRAAXk+AvfTfhjsB6T2UB4Z2/w==", 2024-09-12T03:19:54.757 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:19:54.757 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow profile osd", 2024-09-12T03:19:54.757 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile osd", 2024-09-12T03:19:54.757 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2024-09-12T03:19:54.757 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:19:54.757 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:19:54.757 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:19:54.758 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "osd.3", 2024-09-12T03:19:54.758 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDiXOJmrQbPJxAAQsi2B5a1LnPTPGZVdo7j7w==", 2024-09-12T03:19:54.758 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:19:54.758 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow profile osd", 2024-09-12T03:19:54.758 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile osd", 2024-09-12T03:19:54.758 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2024-09-12T03:19:54.758 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:19:54.758 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:19:54.758 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:19:54.758 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "osd.4", 2024-09-12T03:19:54.758 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDnXOJmoMr1NxAAv3u2q8hYt9w//9gz+N7UAQ==", 2024-09-12T03:19:54.758 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:19:54.758 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow profile osd", 2024-09-12T03:19:54.758 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile osd", 2024-09-12T03:19:54.759 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2024-09-12T03:19:54.759 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:19:54.759 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:19:54.759 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:19:54.759 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "osd.5", 2024-09-12T03:19:54.759 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDqXOJmxpUPOxAAxLH6RJsyLOZ2XMfzM0fQ1g==", 2024-09-12T03:19:54.759 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:19:54.759 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow profile osd", 2024-09-12T03:19:54.759 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile osd", 2024-09-12T03:19:54.759 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2024-09-12T03:19:54.760 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:19:54.760 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:19:54.760 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:19:54.760 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "osd.6", 2024-09-12T03:19:54.760 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDuXOJmuP0+LxAAMHoz5eSO81hPtkdBIS2cig==", 2024-09-12T03:19:54.760 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:19:54.760 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow profile osd", 2024-09-12T03:19:54.760 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile osd", 2024-09-12T03:19:54.760 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2024-09-12T03:19:54.760 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:19:54.760 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:19:54.760 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:19:54.760 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "osd.7", 2024-09-12T03:19:54.761 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDxXOJmuIC/MRAAPDM+lAHqdoQyGVODwjFp+Q==", 2024-09-12T03:19:54.761 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:19:54.761 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow profile osd", 2024-09-12T03:19:54.761 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile osd", 2024-09-12T03:19:54.761 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2024-09-12T03:19:54.761 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:19:54.761 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:19:54.761 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:19:54.761 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "client.0", 2024-09-12T03:19:54.761 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDVXOJmLm+5ExAAECM4CKfuVElmp/n0b02mXg==", 2024-09-12T03:19:54.761 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:19:54.761 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow", 2024-09-12T03:19:54.761 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow r", 2024-09-12T03:19:54.761 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-12T03:19:54.762 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:19:54.762 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:19:54.762 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:19:54.762 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "client.1", 2024-09-12T03:19:54.762 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDVXOJm61/FHBAALCQouXySfQeZ5DESGmqkzA==", 2024-09-12T03:19:54.762 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:19:54.762 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow", 2024-09-12T03:19:54.762 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow r", 2024-09-12T03:19:54.762 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow rw", 2024-09-12T03:19:54.762 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow rwx" 2024-09-12T03:19:54.762 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:19:54.762 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:19:54.763 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:19:54.763 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "client.2", 2024-09-12T03:19:54.763 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDVXOJmq/ZkGRAAOzmQzEIhb4b/YTxayc34Bw==", 2024-09-12T03:19:54.763 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:19:54.763 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow", 2024-09-12T03:19:54.763 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow r", 2024-09-12T03:19:54.763 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow rw", 2024-09-12T03:19:54.763 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow rwx" 2024-09-12T03:19:54.763 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:19:54.763 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:19:54.763 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:19:54.763 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "client.3", 2024-09-12T03:19:54.764 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDVXOJm+DlzIhAATQKZe+Zyk0MM8efTa6g9PA==", 2024-09-12T03:19:54.764 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:19:54.764 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow", 2024-09-12T03:19:54.764 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow r", 2024-09-12T03:19:54.764 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow rw", 2024-09-12T03:19:54.764 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow rwx" 2024-09-12T03:19:54.764 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:19:54.764 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:19:54.764 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:19:54.764 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "client.admin", 2024-09-12T03:19:54.764 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDUXOJm+JGWGhAApTQw9RtJWd/J5QzQaUFXvg==", 2024-09-12T03:19:54.765 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:19:54.765 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow *", 2024-09-12T03:19:54.765 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow *", 2024-09-12T03:19:54.765 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow *", 2024-09-12T03:19:54.765 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2024-09-12T03:19:54.765 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:19:54.765 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:19:54.765 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:19:54.765 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "client.alice", 2024-09-12T03:19:54.765 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDOXeJmirdoKBAAdMkSM4D68jo2uQVNOwfw9Q==", 2024-09-12T03:19:54.765 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:19:54.765 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow rw path=/volumes/_nogroup/subvol82580", 2024-09-12T03:19:54.765 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow r", 2024-09-12T03:19:54.765 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow rw pool=cephfs_data" 2024-09-12T03:19:54.766 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:19:54.766 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:19:54.766 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:19:54.766 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "mgr.x", 2024-09-12T03:19:54.766 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDUXOJmZcsONBAAMTY2KykuHaEkv1t6Vz5KjA==", 2024-09-12T03:19:54.766 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:19:54.766 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow *", 2024-09-12T03:19:54.766 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile mgr", 2024-09-12T03:19:54.766 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2024-09-12T03:19:54.766 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:19:54.766 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:19:54.766 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:19:54.766 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "mgr.y", 2024-09-12T03:19:54.766 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDUXOJmJOgZMBAAQ/JTxfhox7HgxBMtsvUCBg==", 2024-09-12T03:19:54.767 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:19:54.767 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow *", 2024-09-12T03:19:54.767 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile mgr", 2024-09-12T03:19:54.767 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2024-09-12T03:19:54.767 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:19:54.767 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:19:54.767 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-12T03:19:54.767 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-12T03:19:54.768 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:54.754+0000 7f298774b640 1 -- 172.21.15.27:0/1381707338 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f296406c190 msgr2=0x7f296406e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:54.769 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:54.754+0000 7f298774b640 1 --2- 172.21.15.27:0/1381707338 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f296406c190 0x7f296406e650 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f297c034000 tx=0x7f297c02e000 comp rx=0 tx=0).stop 2024-09-12T03:19:54.769 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:54.754+0000 7f298774b640 1 -- 172.21.15.27:0/1381707338 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f298015e780 msgr2=0x7f29800cc230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:54.769 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:54.754+0000 7f298774b640 1 --2- 172.21.15.27:0/1381707338 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f298015e780 0x7f29800cc230 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f2978007e70 tx=0x7f297800e4b0 comp rx=0 tx=0).stop 2024-09-12T03:19:54.769 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:54.754+0000 7f298774b640 1 -- 172.21.15.27:0/1381707338 shutdown_connections 2024-09-12T03:19:54.769 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:54.754+0000 7f298774b640 1 --2- 172.21.15.27:0/1381707338 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f296406c190 0x7f296406e650 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:54.769 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:54.754+0000 7f298774b640 1 --2- 172.21.15.27:0/1381707338 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f298015e780 0x7f29800cc230 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:54.769 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:54.754+0000 7f298774b640 1 --2- 172.21.15.27:0/1381707338 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f29800d43b0 0x7f29801f4240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:54.769 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:54.754+0000 7f298774b640 1 --2- 172.21.15.27:0/1381707338 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f29800d1990 0x7f29800ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:54.769 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:54.754+0000 7f298774b640 1 -- 172.21.15.27:0/1381707338 >> 172.21.15.27:0/1381707338 conn(0x7f298006da60 msgr2=0x7f29800d5020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:54.769 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:54.754+0000 7f298774b640 1 -- 172.21.15.27:0/1381707338 shutdown_connections 2024-09-12T03:19:54.769 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:54.754+0000 7f298774b640 1 -- 172.21.15.27:0/1381707338 wait complete. 2024-09-12T03:19:54.770 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.alice 2024-09-12T03:19:54.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:54.873+0000 7f0ed4ee6640 1 Processor -- start 2024-09-12T03:19:54.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:54.873+0000 7f0ed4ee6640 1 -- start start 2024-09-12T03:19:54.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:54.874+0000 7f0ed4ee6640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f0ed015eb90 0x7f0ed00d2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:54.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:54.874+0000 7f0ed4ee6640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f0ed00d1720 0x7f0ed00b8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:54.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:54.874+0000 7f0ed4ee6640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0ed00d3760 con 0x7f0ed00d5200 2024-09-12T03:19:54.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:54.874+0000 7f0ed4ee6640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f0ed00b7d20 con 0x7f0ed00d1720 2024-09-12T03:19:54.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:54.874+0000 7f0ed4ee6640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f0ed00d42e0 con 0x7f0ed015eb90 2024-09-12T03:19:54.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:54.875+0000 7f0eced9d640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f0ed00d1720 0x7f0ed00b8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:54.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:54.875+0000 7f0eced9d640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f0ed00d1720 0x7f0ed00b8190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35580/0 (socket says 172.21.15.27:35580) 2024-09-12T03:19:54.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:54.875+0000 7f0eced9d640 1 -- 172.21.15.27:0/4062594011 learned_addr learned my addr 172.21.15.27:0/4062594011 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:19:54.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:54.875+0000 7f0ece59c640 1 --2- 172.21.15.27:0/4062594011 >> v2:172.21.15.45:3300/0 conn(0x7f0ed015eb90 0x7f0ed00d2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:54.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:54.875+0000 7f0eced9d640 1 -- 172.21.15.27:0/4062594011 >> v2:172.21.15.45:3300/0 conn(0x7f0ed015eb90 msgr2=0x7f0ed00d2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:54.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:54.875+0000 7f0eced9d640 1 --2- 172.21.15.27:0/4062594011 >> v2:172.21.15.45:3300/0 conn(0x7f0ed015eb90 0x7f0ed00d2df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:54.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:54.875+0000 7f0eced9d640 1 -- 172.21.15.27:0/4062594011 >> v1:172.21.15.45:6789/0 conn(0x7f0ed00d5200 legacy=0x7f0ed00d1010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:54.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:54.875+0000 7f0eced9d640 1 -- 172.21.15.27:0/4062594011 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ed00d3a40 con 0x7f0ed00d1720 2024-09-12T03:19:54.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:54.876+0000 7f0eced9d640 1 --2- 172.21.15.27:0/4062594011 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0ed00d1720 0x7f0ed00b8190 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f0ed0075560 tx=0x7f0ec802a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8337b3cd6f1e3265 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:54.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:54.876+0000 7f0ecd59a640 1 -- 172.21.15.27:0/4062594011 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ec8036070 con 0x7f0ed00d1720 2024-09-12T03:19:54.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:54.876+0000 7f0ecd59a640 1 -- 172.21.15.27:0/4062594011 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ec8031600 con 0x7f0ed00d1720 2024-09-12T03:19:54.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:54.876+0000 7f0ecd59a640 1 -- 172.21.15.27:0/4062594011 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ec80319d0 con 0x7f0ed00d1720 2024-09-12T03:19:54.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:54.876+0000 7f0ed4ee6640 1 -- 172.21.15.27:0/4062594011 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0ed00d1720 msgr2=0x7f0ed00b8190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:54.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:54.876+0000 7f0ed4ee6640 1 --2- 172.21.15.27:0/4062594011 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0ed00d1720 0x7f0ed00b8190 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f0ed0075560 tx=0x7f0ec802a730 comp rx=0 tx=0).stop 2024-09-12T03:19:54.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:54.876+0000 7f0ed4ee6640 1 -- 172.21.15.27:0/4062594011 shutdown_connections 2024-09-12T03:19:54.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:54.876+0000 7f0ed4ee6640 1 --2- 172.21.15.27:0/4062594011 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0ed00d1720 0x7f0ed00b8190 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:54.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:54.876+0000 7f0ed4ee6640 1 --2- 172.21.15.27:0/4062594011 >> v2:172.21.15.45:3300/0 conn(0x7f0ed015eb90 0x7f0ed00d2df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:54.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:54.876+0000 7f0ed4ee6640 1 -- 172.21.15.27:0/4062594011 >> 172.21.15.27:0/4062594011 conn(0x7f0ed006dce0 msgr2=0x7f0ed006e0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:54.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:54.876+0000 7f0ed4ee6640 1 -- 172.21.15.27:0/4062594011 shutdown_connections 2024-09-12T03:19:54.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:54.876+0000 7f0ed4ee6640 1 -- 172.21.15.27:0/4062594011 wait complete. 2024-09-12T03:19:54.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:54.878+0000 7f0ed4ee6640 1 Processor -- start 2024-09-12T03:19:54.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:54.878+0000 7f0ed4ee6640 1 -- start start 2024-09-12T03:19:54.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:54.879+0000 7f0ed4ee6640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0ed00d1720 0x7f0ed0154fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:54.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:54.879+0000 7f0ed4ee6640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f0ed00d5200 0x7f0ed00cdb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:54.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:54.879+0000 7f0ed4ee6640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0ed015eb90 0x7f0ed00ce0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:54.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:54.879+0000 7f0ed4ee6640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f0ed00d3330 con 0x7f0ed015eb90 2024-09-12T03:19:54.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:54.879+0000 7f0ed4ee6640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f0ed00d4000 con 0x7f0ed00d1720 2024-09-12T03:19:54.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:54.879+0000 7f0ed4ee6640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f0ed00d34b0 con 0x7f0ed00d5200 2024-09-12T03:19:54.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:54.879+0000 7f0eced9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0ed015eb90 0x7f0ed00ce0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:54.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:54.879+0000 7f0eced9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0ed015eb90 0x7f0ed00ce0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35590/0 (socket says 172.21.15.27:35590) 2024-09-12T03:19:54.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:54.879+0000 7f0eced9d640 1 -- 172.21.15.27:0/2781394048 learned_addr learned my addr 172.21.15.27:0/2781394048 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:19:54.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:54.879+0000 7f0ece59c640 1 --2- 172.21.15.27:0/2781394048 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0ed00d1720 0x7f0ed0154fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:54.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:54.879+0000 7f0ecdd9b640 1 --2- 172.21.15.27:0/2781394048 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f0ed00d5200 0x7f0ed00cdb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:54.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:54.879+0000 7f0eced9d640 1 -- 172.21.15.27:0/2781394048 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f0ed00d5200 msgr2=0x7f0ed00cdb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:54.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:54.879+0000 7f0eced9d640 1 --2- 172.21.15.27:0/2781394048 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f0ed00d5200 0x7f0ed00cdb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:54.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:54.879+0000 7f0eced9d640 1 -- 172.21.15.27:0/2781394048 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0ed00d1720 msgr2=0x7f0ed0154fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:54.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:54.879+0000 7f0eced9d640 1 --2- 172.21.15.27:0/2781394048 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0ed00d1720 0x7f0ed0154fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:54.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:54.879+0000 7f0eced9d640 1 -- 172.21.15.27:0/2781394048 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ed0155510 con 0x7f0ed015eb90 2024-09-12T03:19:54.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:54.879+0000 7f0eced9d640 1 --2- 172.21.15.27:0/2781394048 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0ed015eb90 0x7f0ed00ce0b0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f0ec8009f40 tx=0x7f0ec8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:54.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:54.879+0000 7f0ebeffd640 1 -- 172.21.15.27:0/2781394048 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ec8036050 con 0x7f0ed015eb90 2024-09-12T03:19:54.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:54.879+0000 7f0ebeffd640 1 -- 172.21.15.27:0/2781394048 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ec8042070 con 0x7f0ed015eb90 2024-09-12T03:19:54.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:54.879+0000 7f0ed4ee6640 1 -- 172.21.15.27:0/2781394048 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ed015b9d0 con 0x7f0ed015eb90 2024-09-12T03:19:54.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:54.880+0000 7f0ebeffd640 1 -- 172.21.15.27:0/2781394048 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ec8009510 con 0x7f0ed015eb90 2024-09-12T03:19:54.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:54.880+0000 7f0ed4ee6640 1 -- 172.21.15.27:0/2781394048 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ed015bf90 con 0x7f0ed015eb90 2024-09-12T03:19:54.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:54.881+0000 7f0ed4ee6640 1 -- 172.21.15.27:0/2781394048 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ed015f090 con 0x7f0ed015eb90 2024-09-12T03:19:54.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:54.881+0000 7f0ebeffd640 1 -- 172.21.15.27:0/2781394048 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 11) ==== 90804+0+0 (secure 0 0 0) 0x7f0ec8007bf0 con 0x7f0ed015eb90 2024-09-12T03:19:54.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:54.881+0000 7f0ebeffd640 1 --2- 172.21.15.27:0/2781394048 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f0eb006c190 0x7f0eb006e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:54.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:54.882+0000 7f0ebeffd640 1 -- 172.21.15.27:0/2781394048 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 4857+0+0 (secure 0 0 0) 0x7f0ec80abd20 con 0x7f0ed015eb90 2024-09-12T03:19:54.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:54.882+0000 7f0ece59c640 1 --2- 172.21.15.27:0/2781394048 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f0eb006c190 0x7f0eb006e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:54.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:54.882+0000 7f0ece59c640 1 --2- 172.21.15.27:0/2781394048 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f0eb006c190 0x7f0eb006e650 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f0ed00ce880 tx=0x7f0ec0009190 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:54.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:54.886+0000 7f0ebeffd640 1 -- 172.21.15.27:0/2781394048 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f0ec8079f40 con 0x7f0ed015eb90 2024-09-12T03:19:55.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:55.132+0000 7f0ed4ee6640 1 -- 172.21.15.27:0/2781394048 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.alice"} v 0) -- 0x7f0ed015f900 con 0x7f0ed015eb90 2024-09-12T03:19:55.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:55.134+0000 7f0ebeffd640 1 -- 172.21.15.27:0/2781394048 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.alice"}]=0 v15) ==== 82+0+0 (secure 0 0 0) 0x7f0ec807e240 con 0x7f0ed015eb90 2024-09-12T03:19:55.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:55.136+0000 7f0ed4ee6640 1 -- 172.21.15.27:0/2781394048 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f0eb006c190 msgr2=0x7f0eb006e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:55.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:55.136+0000 7f0ed4ee6640 1 --2- 172.21.15.27:0/2781394048 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f0eb006c190 0x7f0eb006e650 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f0ed00ce880 tx=0x7f0ec0009190 comp rx=0 tx=0).stop 2024-09-12T03:19:55.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:55.136+0000 7f0ed4ee6640 1 -- 172.21.15.27:0/2781394048 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0ed015eb90 msgr2=0x7f0ed00ce0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:55.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:55.136+0000 7f0ed4ee6640 1 --2- 172.21.15.27:0/2781394048 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0ed015eb90 0x7f0ed00ce0b0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f0ec8009f40 tx=0x7f0ec8004a40 comp rx=0 tx=0).stop 2024-09-12T03:19:55.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:55.136+0000 7f0ed4ee6640 1 -- 172.21.15.27:0/2781394048 shutdown_connections 2024-09-12T03:19:55.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:55.136+0000 7f0ed4ee6640 1 --2- 172.21.15.27:0/2781394048 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f0eb006c190 0x7f0eb006e650 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:55.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:55.136+0000 7f0ed4ee6640 1 --2- 172.21.15.27:0/2781394048 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0ed015eb90 0x7f0ed00ce0b0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:55.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:55.136+0000 7f0ed4ee6640 1 --2- 172.21.15.27:0/2781394048 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f0ed00d5200 0x7f0ed00cdb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:55.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:55.136+0000 7f0ed4ee6640 1 --2- 172.21.15.27:0/2781394048 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0ed00d1720 0x7f0ed0154fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:55.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:55.136+0000 7f0ed4ee6640 1 -- 172.21.15.27:0/2781394048 >> 172.21.15.27:0/2781394048 conn(0x7f0ed006dce0 msgr2=0x7f0ed00b92a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:55.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:55.136+0000 7f0ed4ee6640 1 -- 172.21.15.27:0/2781394048 shutdown_connections 2024-09-12T03:19:55.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:55.136+0000 7f0ed4ee6640 1 -- 172.21.15.27:0/2781394048 wait complete. 2024-09-12T03:19:55.147 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-12T03:19:55.249 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:55.250+0000 7f4f285fd640 1 Processor -- start 2024-09-12T03:19:55.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:55.250+0000 7f4f285fd640 1 -- start start 2024-09-12T03:19:55.251 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:55.250+0000 7f4f285fd640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f4f2015eb90 0x7f4f200d2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:55.251 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:55.250+0000 7f4f285fd640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4f200d5200 0x7f4f200d1010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:55.251 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:55.251+0000 7f4f285fd640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f4f200d1550 0x7f4f200cf200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:55.251 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:55.251+0000 7f4f285fd640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f4f200b7d20 con 0x7f4f2015eb90 2024-09-12T03:19:55.251 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:55.251+0000 7f4f285fd640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f4f200d42e0 con 0x7f4f200d1550 2024-09-12T03:19:55.251 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:55.251+0000 7f4f285fd640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f4f200d19c0 con 0x7f4f200d5200 2024-09-12T03:19:55.251 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:55.251+0000 7f4f26399640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f4f2015eb90 0x7f4f200d2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:55.251 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:55.251+0000 7f4f26399640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f4f2015eb90 0x7f4f200d2df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35604/0 (socket says 172.21.15.27:35604) 2024-09-12T03:19:55.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:55.251+0000 7f4f26399640 1 -- 172.21.15.27:0/1977940409 learned_addr learned my addr 172.21.15.27:0/1977940409 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:19:55.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:55.251+0000 7f4f25b98640 1 --2- 172.21.15.27:0/1977940409 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4f200d5200 0x7f4f200d1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:55.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:55.251+0000 7f4f26b9a640 1 --2- 172.21.15.27:0/1977940409 >> v2:172.21.15.45:3300/0 conn(0x7f4f200d1550 0x7f4f200cf200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:55.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:55.251+0000 7f4f26399640 1 -- 172.21.15.27:0/1977940409 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4f200d5200 msgr2=0x7f4f200d1010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:55.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:55.251+0000 7f4f26399640 1 --2- 172.21.15.27:0/1977940409 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4f200d5200 0x7f4f200d1010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:55.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:55.251+0000 7f4f26399640 1 -- 172.21.15.27:0/1977940409 >> v2:172.21.15.45:3300/0 conn(0x7f4f200d1550 msgr2=0x7f4f200cf200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:55.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:55.251+0000 7f4f26399640 1 --2- 172.21.15.27:0/1977940409 >> v2:172.21.15.45:3300/0 conn(0x7f4f200d1550 0x7f4f200cf200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:55.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:55.251+0000 7f4f26399640 1 -- 172.21.15.27:0/1977940409 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f200d3760 con 0x7f4f2015eb90 2024-09-12T03:19:55.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:55.251+0000 7f4f26b9a640 1 --2- 172.21.15.27:0/1977940409 >> v2:172.21.15.45:3300/0 conn(0x7f4f200d1550 0x7f4f200cf200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:19:55.253 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:55.252+0000 7f4f26399640 1 --2- 172.21.15.27:0/1977940409 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4f2015eb90 0x7f4f200d2df0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f4f2008d510 tx=0x7f4f1002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39b9a0f4ad06a455 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:55.253 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:55.252+0000 7f4f25397640 1 -- 172.21.15.27:0/1977940409 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f10036070 con 0x7f4f2015eb90 2024-09-12T03:19:55.253 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:55.252+0000 7f4f25397640 1 -- 172.21.15.27:0/1977940409 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f10031600 con 0x7f4f2015eb90 2024-09-12T03:19:55.254 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:55.252+0000 7f4f25397640 1 -- 172.21.15.27:0/1977940409 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f100319d0 con 0x7f4f2015eb90 2024-09-12T03:19:55.254 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:55.252+0000 7f4f285fd640 1 -- 172.21.15.27:0/1977940409 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4f2015eb90 msgr2=0x7f4f200d2df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:55.254 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:55.252+0000 7f4f285fd640 1 --2- 172.21.15.27:0/1977940409 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4f2015eb90 0x7f4f200d2df0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f4f2008d510 tx=0x7f4f1002a6a0 comp rx=0 tx=0).stop 2024-09-12T03:19:55.254 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:55.253+0000 7f4f285fd640 1 -- 172.21.15.27:0/1977940409 shutdown_connections 2024-09-12T03:19:55.254 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:55.253+0000 7f4f285fd640 1 --2- 172.21.15.27:0/1977940409 >> v2:172.21.15.45:3300/0 conn(0x7f4f200d1550 0x7f4f200cf200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:55.254 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:55.253+0000 7f4f285fd640 1 --2- 172.21.15.27:0/1977940409 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4f200d5200 0x7f4f200d1010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:55.254 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:55.253+0000 7f4f285fd640 1 --2- 172.21.15.27:0/1977940409 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4f2015eb90 0x7f4f200d2df0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:55.254 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:55.253+0000 7f4f285fd640 1 -- 172.21.15.27:0/1977940409 >> 172.21.15.27:0/1977940409 conn(0x7f4f2006dce0 msgr2=0x7f4f2006e0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:55.254 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:55.253+0000 7f4f285fd640 1 -- 172.21.15.27:0/1977940409 shutdown_connections 2024-09-12T03:19:55.254 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:55.253+0000 7f4f285fd640 1 -- 172.21.15.27:0/1977940409 wait complete. 2024-09-12T03:19:55.254 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:55.254+0000 7f4f285fd640 1 Processor -- start 2024-09-12T03:19:55.254 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:55.254+0000 7f4f285fd640 1 -- start start 2024-09-12T03:19:55.256 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:55.254+0000 7f4f285fd640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4f200d1550 0x7f4f20157900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:55.256 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:55.255+0000 7f4f285fd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4f200d5200 0x7f4f20159460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:55.256 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:55.255+0000 7f4f285fd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4f2015eb90 0x7f4f20159d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:55.256 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:55.255+0000 7f4f285fd640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f4f200d4000 con 0x7f4f2015eb90 2024-09-12T03:19:55.256 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:55.255+0000 7f4f285fd640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f4f200d3d20 con 0x7f4f200d5200 2024-09-12T03:19:55.257 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:55.255+0000 7f4f285fd640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f4f200d3330 con 0x7f4f200d1550 2024-09-12T03:19:55.257 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:55.255+0000 7f4f26b9a640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4f2015eb90 0x7f4f20159d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:55.257 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:55.255+0000 7f4f26b9a640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4f2015eb90 0x7f4f20159d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35618/0 (socket says 172.21.15.27:35618) 2024-09-12T03:19:55.257 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:55.255+0000 7f4f26399640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4f200d1550 0x7f4f20157900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:55.257 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:55.255+0000 7f4f26b9a640 1 -- 172.21.15.27:0/1628232481 learned_addr learned my addr 172.21.15.27:0/1628232481 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:19:55.257 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:55.255+0000 7f4f25b98640 1 --2- 172.21.15.27:0/1628232481 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4f200d5200 0x7f4f20159460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:55.257 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:55.255+0000 7f4f26b9a640 1 -- 172.21.15.27:0/1628232481 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4f200d1550 msgr2=0x7f4f20157900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:55.257 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:55.255+0000 7f4f26b9a640 1 --2- 172.21.15.27:0/1628232481 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4f200d1550 0x7f4f20157900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:55.257 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:55.255+0000 7f4f26b9a640 1 -- 172.21.15.27:0/1628232481 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4f200d5200 msgr2=0x7f4f20159460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:55.257 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:55.255+0000 7f4f26b9a640 1 --2- 172.21.15.27:0/1628232481 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4f200d5200 0x7f4f20159460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:55.257 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:55.255+0000 7f4f26b9a640 1 -- 172.21.15.27:0/1628232481 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f201586a0 con 0x7f4f2015eb90 2024-09-12T03:19:55.257 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:55.255+0000 7f4f26b9a640 1 --2- 172.21.15.27:0/1628232481 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4f2015eb90 0x7f4f20159d50 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f4f1800c5e0 tx=0x7f4f1800caa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:55.257 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:55.255+0000 7f4f1effd640 1 -- 172.21.15.27:0/1628232481 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f18015070 con 0x7f4f2015eb90 2024-09-12T03:19:55.258 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:55.255+0000 7f4f1effd640 1 -- 172.21.15.27:0/1628232481 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f18011840 con 0x7f4f2015eb90 2024-09-12T03:19:55.258 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:55.255+0000 7f4f1effd640 1 -- 172.21.15.27:0/1628232481 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f18011c30 con 0x7f4f2015eb90 2024-09-12T03:19:55.258 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:55.255+0000 7f4f285fd640 1 -- 172.21.15.27:0/1628232481 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f20158980 con 0x7f4f2015eb90 2024-09-12T03:19:55.258 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:55.255+0000 7f4f285fd640 1 -- 172.21.15.27:0/1628232481 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f20158f40 con 0x7f4f2015eb90 2024-09-12T03:19:55.258 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:55.256+0000 7f4f285fd640 1 -- 172.21.15.27:0/1628232481 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f2015ef70 con 0x7f4f2015eb90 2024-09-12T03:19:55.259 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:55.257+0000 7f4f1effd640 1 -- 172.21.15.27:0/1628232481 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 11) ==== 90804+0+0 (secure 0 0 0) 0x7f4f18018070 con 0x7f4f2015eb90 2024-09-12T03:19:55.259 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:55.257+0000 7f4f1effd640 1 --2- 172.21.15.27:0/1628232481 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f4f0c06c190 0x7f4f0c06e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:55.259 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:55.257+0000 7f4f1effd640 1 -- 172.21.15.27:0/1628232481 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 4857+0+0 (secure 0 0 0) 0x7f4f1808af00 con 0x7f4f2015eb90 2024-09-12T03:19:55.259 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:55.258+0000 7f4f26399640 1 --2- 172.21.15.27:0/1628232481 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f4f0c06c190 0x7f4f0c06e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:55.259 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:55.258+0000 7f4f26399640 1 --2- 172.21.15.27:0/1628232481 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f4f0c06c190 0x7f4f0c06e650 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f4f10033000 tx=0x7f4f10009000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:55.261 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:55.262+0000 7f4f1effd640 1 -- 172.21.15.27:0/1628232481 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f4f18059120 con 0x7f4f2015eb90 2024-09-12T03:19:55.440 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:55.440+0000 7f4f285fd640 1 -- 172.21.15.27:0/1628232481 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f4f200c8050 con 0x7f4f2015eb90 2024-09-12T03:19:55.440 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:55.440+0000 7f4f1effd640 1 -- 172.21.15.27:0/1628232481 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v63) ==== 84+0+65 (secure 0 0 0) 0x7f4f1805d420 con 0x7f4f2015eb90 2024-09-12T03:19:55.441 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:19:55.441 INFO:teuthology.orchestra.run.smithi027.stdout:[ 2024-09-12T03:19:55.441 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:19:55.441 INFO:teuthology.orchestra.run.smithi027.stdout: "poolnum": 1, 2024-09-12T03:19:55.441 INFO:teuthology.orchestra.run.smithi027.stdout: "poolname": ".mgr" 2024-09-12T03:19:55.441 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:19:55.441 INFO:teuthology.orchestra.run.smithi027.stdout:] 2024-09-12T03:19:55.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:55.442+0000 7f4f285fd640 1 -- 172.21.15.27:0/1628232481 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f4f0c06c190 msgr2=0x7f4f0c06e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:55.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:55.442+0000 7f4f285fd640 1 --2- 172.21.15.27:0/1628232481 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f4f0c06c190 0x7f4f0c06e650 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f4f10033000 tx=0x7f4f10009000 comp rx=0 tx=0).stop 2024-09-12T03:19:55.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:55.442+0000 7f4f285fd640 1 -- 172.21.15.27:0/1628232481 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4f2015eb90 msgr2=0x7f4f20159d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:55.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:55.442+0000 7f4f285fd640 1 --2- 172.21.15.27:0/1628232481 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4f2015eb90 0x7f4f20159d50 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f4f1800c5e0 tx=0x7f4f1800caa0 comp rx=0 tx=0).stop 2024-09-12T03:19:55.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:55.442+0000 7f4f285fd640 1 -- 172.21.15.27:0/1628232481 shutdown_connections 2024-09-12T03:19:55.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:55.442+0000 7f4f285fd640 1 --2- 172.21.15.27:0/1628232481 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f4f0c06c190 0x7f4f0c06e650 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:55.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:55.442+0000 7f4f285fd640 1 --2- 172.21.15.27:0/1628232481 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4f2015eb90 0x7f4f20159d50 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:55.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:55.442+0000 7f4f285fd640 1 --2- 172.21.15.27:0/1628232481 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4f200d5200 0x7f4f20159460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:55.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:55.442+0000 7f4f285fd640 1 --2- 172.21.15.27:0/1628232481 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4f200d1550 0x7f4f20157900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:55.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:55.442+0000 7f4f285fd640 1 -- 172.21.15.27:0/1628232481 >> 172.21.15.27:0/1628232481 conn(0x7f4f2006dce0 msgr2=0x7f4f200b9320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:55.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:55.442+0000 7f4f285fd640 1 -- 172.21.15.27:0/1628232481 shutdown_connections 2024-09-12T03:19:55.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:55.442+0000 7f4f285fd640 1 -- 172.21.15.27:0/1628232481 wait complete. 2024-09-12T03:19:55.453 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-12T03:19:55.454 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-12T03:19:55.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:55.560+0000 7f2bbb949640 1 Processor -- start 2024-09-12T03:19:55.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:55.560+0000 7f2bbb949640 1 -- start start 2024-09-12T03:19:55.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:55.561+0000 7f2bbb949640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f2bb406cc40 0x7f2bb406d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:55.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:55.561+0000 7f2bbb949640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2bb406d5d0 con 0x7f2bb415e650 2024-09-12T03:19:55.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:55.561+0000 7f2bbb949640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2bb406e670 con 0x7f2bb406db30 2024-09-12T03:19:55.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:55.561+0000 7f2bbb949640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f2bb406d850 con 0x7f2bb406cc40 2024-09-12T03:19:55.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:55.561+0000 7f2bb8ee4640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f2bb406cc40 0x7f2bb406d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:55.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:55.561+0000 7f2bb96e5640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f2bb415e650 0x7f2bb40d1d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:53118/0 (socket says 172.21.15.27:53118) 2024-09-12T03:19:55.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:55.561+0000 7f2bb96e5640 1 -- 172.21.15.27:0/1967440356 learned_addr learned my addr 172.21.15.27:0/1967440356 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:19:55.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:55.561+0000 7f2bb8ee4640 1 -- 172.21.15.27:0/1967440356 >> v1:172.21.15.45:6789/0 conn(0x7f2bb406db30 legacy=0x7f2bb406df60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:55.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:55.561+0000 7f2bb8ee4640 1 -- 172.21.15.27:0/1967440356 >> v1:172.21.15.27:6789/0 conn(0x7f2bb415e650 legacy=0x7f2bb40d1d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:55.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:55.561+0000 7f2bb8ee4640 1 -- 172.21.15.27:0/1967440356 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bb40d1260 con 0x7f2bb406cc40 2024-09-12T03:19:55.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:55.562+0000 7f2bb8ee4640 1 --2- 172.21.15.27:0/1967440356 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2bb406cc40 0x7f2bb406d090 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f2ba40097a0 tx=0x7f2ba402a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55974945283adb56 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:55.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:55.562+0000 7f2babfff640 1 -- 172.21.15.27:0/1967440356 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ba4037070 con 0x7f2bb406cc40 2024-09-12T03:19:55.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:55.562+0000 7f2babfff640 1 -- 172.21.15.27:0/1967440356 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ba402ae50 con 0x7f2bb406cc40 2024-09-12T03:19:55.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:55.562+0000 7f2babfff640 1 -- 172.21.15.27:0/1967440356 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ba4032660 con 0x7f2bb406cc40 2024-09-12T03:19:55.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:55.563+0000 7f2bbb949640 1 -- 172.21.15.27:0/1967440356 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2bb406cc40 msgr2=0x7f2bb406d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:55.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:55.563+0000 7f2bbb949640 1 --2- 172.21.15.27:0/1967440356 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2bb406cc40 0x7f2bb406d090 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f2ba40097a0 tx=0x7f2ba402a140 comp rx=0 tx=0).stop 2024-09-12T03:19:55.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:55.563+0000 7f2bbb949640 1 -- 172.21.15.27:0/1967440356 shutdown_connections 2024-09-12T03:19:55.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:55.563+0000 7f2bbb949640 1 --2- 172.21.15.27:0/1967440356 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2bb406cc40 0x7f2bb406d090 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:55.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:55.563+0000 7f2bbb949640 1 -- 172.21.15.27:0/1967440356 >> 172.21.15.27:0/1967440356 conn(0x7f2bb406f320 msgr2=0x7f2bb406f700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:55.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:55.563+0000 7f2bbb949640 1 -- 172.21.15.27:0/1967440356 shutdown_connections 2024-09-12T03:19:55.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:55.563+0000 7f2bbb949640 1 -- 172.21.15.27:0/1967440356 wait complete. 2024-09-12T03:19:55.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:55.564+0000 7f2bbb949640 1 Processor -- start 2024-09-12T03:19:55.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:55.564+0000 7f2bbb949640 1 -- start start 2024-09-12T03:19:55.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:55.565+0000 7f2bbb949640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2bb406cc40 0x7f2bb414c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:55.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:55.565+0000 7f2bbb949640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2bb406db30 0x7f2bb4161a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:55.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:55.565+0000 7f2bbb949640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2bb415e650 0x7f2bb40b25d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:55.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:55.565+0000 7f2bbb949640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f2bb415ebb0 con 0x7f2bb406cc40 2024-09-12T03:19:55.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:55.565+0000 7f2bbb949640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f2bb415ea30 con 0x7f2bb406db30 2024-09-12T03:19:55.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:55.565+0000 7f2bbb949640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f2bb40b55b0 con 0x7f2bb415e650 2024-09-12T03:19:55.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:55.565+0000 7f2bb96e5640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2bb406cc40 0x7f2bb414c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:55.567 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:55.565+0000 7f2bb96e5640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2bb406cc40 0x7f2bb414c330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35626/0 (socket says 172.21.15.27:35626) 2024-09-12T03:19:55.567 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:55.565+0000 7f2bb96e5640 1 -- 172.21.15.27:0/1205294403 learned_addr learned my addr 172.21.15.27:0/1205294403 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:19:55.567 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:55.565+0000 7f2bb96e5640 1 -- 172.21.15.27:0/1205294403 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2bb415e650 msgr2=0x7f2bb40b25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:55.567 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:55.565+0000 7f2bb96e5640 1 --2- 172.21.15.27:0/1205294403 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2bb415e650 0x7f2bb40b25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:55.567 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:55.565+0000 7f2bb96e5640 1 -- 172.21.15.27:0/1205294403 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2bb406db30 msgr2=0x7f2bb4161a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:55.567 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:55.565+0000 7f2bb96e5640 1 --2- 172.21.15.27:0/1205294403 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2bb406db30 0x7f2bb4161a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:55.567 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:55.565+0000 7f2bb96e5640 1 -- 172.21.15.27:0/1205294403 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bb41f8990 con 0x7f2bb406cc40 2024-09-12T03:19:55.567 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:55.565+0000 7f2bb96e5640 1 --2- 172.21.15.27:0/1205294403 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2bb406cc40 0x7f2bb414c330 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f2bb000c3c0 tx=0x7f2bb000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:55.567 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:55.565+0000 7f2ba9ffb640 1 -- 172.21.15.27:0/1205294403 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bb0017070 con 0x7f2bb406cc40 2024-09-12T03:19:55.567 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:55.565+0000 7f2ba9ffb640 1 -- 172.21.15.27:0/1205294403 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2bb0005ce0 con 0x7f2bb406cc40 2024-09-12T03:19:55.567 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:55.566+0000 7f2bbb949640 1 -- 172.21.15.27:0/1205294403 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2bb41f7e10 con 0x7f2bb406cc40 2024-09-12T03:19:55.567 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:55.566+0000 7f2ba9ffb640 1 -- 172.21.15.27:0/1205294403 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bb00072f0 con 0x7f2bb406cc40 2024-09-12T03:19:55.567 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:55.566+0000 7f2bbb949640 1 -- 172.21.15.27:0/1205294403 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2bb41f83d0 con 0x7f2bb406cc40 2024-09-12T03:19:55.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:55.567+0000 7f2bbb949640 1 -- 172.21.15.27:0/1205294403 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2bb406d090 con 0x7f2bb406cc40 2024-09-12T03:19:55.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:55.567+0000 7f2ba9ffb640 1 -- 172.21.15.27:0/1205294403 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 11) ==== 90804+0+0 (secure 0 0 0) 0x7f2bb001a070 con 0x7f2bb406cc40 2024-09-12T03:19:55.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:55.568+0000 7f2ba9ffb640 1 --2- 172.21.15.27:0/1205294403 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f2b9806c190 0x7f2b9806e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:55.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:55.568+0000 7f2ba9ffb640 1 -- 172.21.15.27:0/1205294403 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 4857+0+0 (secure 0 0 0) 0x7f2bb008a320 con 0x7f2bb406cc40 2024-09-12T03:19:55.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:55.568+0000 7f2bb8ee4640 1 --2- 172.21.15.27:0/1205294403 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f2b9806c190 0x7f2b9806e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:55.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:55.568+0000 7f2bb8ee4640 1 --2- 172.21.15.27:0/1205294403 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f2b9806c190 0x7f2b9806e650 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f2ba4034000 tx=0x7f2ba402c000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:55.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:55.572+0000 7f2ba9ffb640 1 -- 172.21.15.27:0/1205294403 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f2bb406d5d0 con 0x7f2bb406cc40 2024-09-12T03:19:55.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:55.747+0000 7f2bbb949640 1 -- 172.21.15.27:0/1205294403 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f2bb40b4f90 con 0x7f2bb406cc40 2024-09-12T03:19:56.370 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:56.370+0000 7f2ba9ffb640 1 -- 172.21.15.27:0/1205294403 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v64) ==== 138+0+0 (secure 0 0 0) 0x7f2bb0058ca0 con 0x7f2bb406cc40 2024-09-12T03:19:56.371 INFO:teuthology.orchestra.run.smithi027.stderr:pool 'cephfs_metadata' created 2024-09-12T03:19:56.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:56.373+0000 7f2bbb949640 1 -- 172.21.15.27:0/1205294403 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f2b9806c190 msgr2=0x7f2b9806e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:56.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:56.373+0000 7f2bbb949640 1 --2- 172.21.15.27:0/1205294403 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f2b9806c190 0x7f2b9806e650 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f2ba4034000 tx=0x7f2ba402c000 comp rx=0 tx=0).stop 2024-09-12T03:19:56.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:56.373+0000 7f2bbb949640 1 -- 172.21.15.27:0/1205294403 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2bb406cc40 msgr2=0x7f2bb414c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:56.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:56.373+0000 7f2bbb949640 1 --2- 172.21.15.27:0/1205294403 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2bb406cc40 0x7f2bb414c330 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f2bb000c3c0 tx=0x7f2bb000c880 comp rx=0 tx=0).stop 2024-09-12T03:19:56.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:56.373+0000 7f2bbb949640 1 -- 172.21.15.27:0/1205294403 shutdown_connections 2024-09-12T03:19:56.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:56.373+0000 7f2bbb949640 1 --2- 172.21.15.27:0/1205294403 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f2b9806c190 0x7f2b9806e650 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:56.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:56.373+0000 7f2bbb949640 1 --2- 172.21.15.27:0/1205294403 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2bb415e650 0x7f2bb40b25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:56.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:56.373+0000 7f2bbb949640 1 --2- 172.21.15.27:0/1205294403 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2bb406db30 0x7f2bb4161a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:56.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:56.373+0000 7f2bbb949640 1 --2- 172.21.15.27:0/1205294403 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2bb406cc40 0x7f2bb414c330 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:56.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:56.373+0000 7f2bbb949640 1 -- 172.21.15.27:0/1205294403 >> 172.21.15.27:0/1205294403 conn(0x7f2bb406f320 msgr2=0x7f2bb40b6330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:56.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:56.373+0000 7f2bbb949640 1 -- 172.21.15.27:0/1205294403 shutdown_connections 2024-09-12T03:19:56.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:56.373+0000 7f2bbb949640 1 -- 172.21.15.27:0/1205294403 wait complete. 2024-09-12T03:19:56.387 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-12T03:19:56.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:56.499+0000 7fa6c5053640 1 Processor -- start 2024-09-12T03:19:56.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:56.499+0000 7fa6c5053640 1 -- start start 2024-09-12T03:19:56.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:56.500+0000 7fa6c5053640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa6c0070d30 0x7fa6c006cc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:56.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:56.500+0000 7fa6c5053640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa6c006e960 0x7fa6c00d0440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:56.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:56.500+0000 7fa6c5053640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fa6c00d0980 0x7fa6c00d21b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:56.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:56.500+0000 7fa6c5053640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7fa6c00d0df0 con 0x7fa6c00d0980 2024-09-12T03:19:56.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:56.500+0000 7fa6c5053640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fa6c00d1a40 con 0x7fa6c006e960 2024-09-12T03:19:56.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:56.500+0000 7fa6c5053640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fa6c00cf9f0 con 0x7fa6c0070d30 2024-09-12T03:19:56.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:56.500+0000 7fa6bf59e640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fa6c00d0980 0x7fa6c00d21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:56.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:56.500+0000 7fa6bf59e640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fa6c00d0980 0x7fa6c00d21b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35642/0 (socket says 172.21.15.27:35642) 2024-09-12T03:19:56.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:56.500+0000 7fa6bf59e640 1 -- 172.21.15.27:0/2397737142 learned_addr learned my addr 172.21.15.27:0/2397737142 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:19:56.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:56.500+0000 7fa6be59c640 1 --2- 172.21.15.27:0/2397737142 >> v2:172.21.15.45:3300/0 conn(0x7fa6c006e960 0x7fa6c00d0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:56.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:56.500+0000 7fa6bed9d640 1 --2- 172.21.15.27:0/2397737142 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa6c0070d30 0x7fa6c006cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:56.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:56.500+0000 7fa6bf59e640 1 -- 172.21.15.27:0/2397737142 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa6c0070d30 msgr2=0x7fa6c006cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:56.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:56.500+0000 7fa6bf59e640 1 --2- 172.21.15.27:0/2397737142 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa6c0070d30 0x7fa6c006cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:56.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:56.500+0000 7fa6bf59e640 1 -- 172.21.15.27:0/2397737142 >> v2:172.21.15.45:3300/0 conn(0x7fa6c006e960 msgr2=0x7fa6c00d0440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:56.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:56.500+0000 7fa6bf59e640 1 --2- 172.21.15.27:0/2397737142 >> v2:172.21.15.45:3300/0 conn(0x7fa6c006e960 0x7fa6c00d0440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:56.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:56.500+0000 7fa6bf59e640 1 -- 172.21.15.27:0/2397737142 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6c00cfc90 con 0x7fa6c00d0980 2024-09-12T03:19:56.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:56.500+0000 7fa6be59c640 1 --2- 172.21.15.27:0/2397737142 >> v2:172.21.15.45:3300/0 conn(0x7fa6c006e960 0x7fa6c00d0440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:19:56.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:56.501+0000 7fa6bf59e640 1 --2- 172.21.15.27:0/2397737142 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa6c00d0980 0x7fa6c00d21b0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fa6c0071d10 tx=0x7fa6b802a570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb2cc9f9064f9a29 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:56.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:56.501+0000 7fa6bdd9b640 1 -- 172.21.15.27:0/2397737142 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6b8036070 con 0x7fa6c00d0980 2024-09-12T03:19:56.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:56.501+0000 7fa6bdd9b640 1 -- 172.21.15.27:0/2397737142 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6b8031600 con 0x7fa6c00d0980 2024-09-12T03:19:56.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:56.501+0000 7fa6bdd9b640 1 -- 172.21.15.27:0/2397737142 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6b80319d0 con 0x7fa6c00d0980 2024-09-12T03:19:56.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:56.502+0000 7fa6c5053640 1 -- 172.21.15.27:0/2397737142 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa6c00d0980 msgr2=0x7fa6c00d21b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:56.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:56.502+0000 7fa6c5053640 1 --2- 172.21.15.27:0/2397737142 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa6c00d0980 0x7fa6c00d21b0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fa6c0071d10 tx=0x7fa6b802a570 comp rx=0 tx=0).stop 2024-09-12T03:19:56.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:56.502+0000 7fa6c5053640 1 -- 172.21.15.27:0/2397737142 shutdown_connections 2024-09-12T03:19:56.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:56.502+0000 7fa6c5053640 1 --2- 172.21.15.27:0/2397737142 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa6c00d0980 0x7fa6c00d21b0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:56.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:56.502+0000 7fa6c5053640 1 --2- 172.21.15.27:0/2397737142 >> v2:172.21.15.45:3300/0 conn(0x7fa6c006e960 0x7fa6c00d0440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:56.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:56.502+0000 7fa6c5053640 1 --2- 172.21.15.27:0/2397737142 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa6c0070d30 0x7fa6c006cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:56.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:56.502+0000 7fa6c5053640 1 -- 172.21.15.27:0/2397737142 >> 172.21.15.27:0/2397737142 conn(0x7fa6c00b7c10 msgr2=0x7fa6c009ba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:56.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:56.502+0000 7fa6c5053640 1 -- 172.21.15.27:0/2397737142 shutdown_connections 2024-09-12T03:19:56.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:56.502+0000 7fa6c5053640 1 -- 172.21.15.27:0/2397737142 wait complete. 2024-09-12T03:19:56.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:56.503+0000 7fa6c5053640 1 Processor -- start 2024-09-12T03:19:56.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:56.504+0000 7fa6c5053640 1 -- start start 2024-09-12T03:19:56.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:56.504+0000 7fa6c5053640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa6c006e960 0x7fa6c01f59d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:56.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:56.504+0000 7fa6c5053640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa6c0070d30 0x7fa6c0160040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:56.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:56.504+0000 7fa6c5053640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa6c00d0980 0x7fa6c00cda20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:56.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:56.504+0000 7fa6c5053640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fa6c006d180 con 0x7fa6c0070d30 2024-09-12T03:19:56.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:56.504+0000 7fa6c5053640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa6c00d1c90 con 0x7fa6c006e960 2024-09-12T03:19:56.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:56.504+0000 7fa6c5053640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fa6c00d26f0 con 0x7fa6c00d0980 2024-09-12T03:19:56.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:56.504+0000 7fa6be59c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa6c0070d30 0x7fa6c0160040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:56.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:56.504+0000 7fa6be59c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa6c0070d30 0x7fa6c0160040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35652/0 (socket says 172.21.15.27:35652) 2024-09-12T03:19:56.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:56.504+0000 7fa6be59c640 1 -- 172.21.15.27:0/448174462 learned_addr learned my addr 172.21.15.27:0/448174462 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:19:56.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:56.504+0000 7fa6bed9d640 1 --2- 172.21.15.27:0/448174462 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa6c006e960 0x7fa6c01f59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:56.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:56.505+0000 7fa6bf59e640 1 --2- 172.21.15.27:0/448174462 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa6c00d0980 0x7fa6c00cda20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:56.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:56.505+0000 7fa6be59c640 1 -- 172.21.15.27:0/448174462 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa6c00d0980 msgr2=0x7fa6c00cda20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:56.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:56.505+0000 7fa6be59c640 1 --2- 172.21.15.27:0/448174462 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa6c00d0980 0x7fa6c00cda20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:56.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:56.505+0000 7fa6be59c640 1 -- 172.21.15.27:0/448174462 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa6c006e960 msgr2=0x7fa6c01f59d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:56.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:56.505+0000 7fa6be59c640 1 --2- 172.21.15.27:0/448174462 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa6c006e960 0x7fa6c01f59d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:56.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:56.505+0000 7fa6be59c640 1 -- 172.21.15.27:0/448174462 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6c00b2120 con 0x7fa6c0070d30 2024-09-12T03:19:56.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:56.505+0000 7fa6be59c640 1 --2- 172.21.15.27:0/448174462 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa6c0070d30 0x7fa6c0160040 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fa6b400e7e0 tx=0x7fa6b400eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:56.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:56.505+0000 7fa6a77fe640 1 -- 172.21.15.27:0/448174462 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6b400f040 con 0x7fa6c0070d30 2024-09-12T03:19:56.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:56.505+0000 7fa6a77fe640 1 -- 172.21.15.27:0/448174462 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6b4004030 con 0x7fa6c0070d30 2024-09-12T03:19:56.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:56.505+0000 7fa6c5053640 1 -- 172.21.15.27:0/448174462 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6c01f5f10 con 0x7fa6c0070d30 2024-09-12T03:19:56.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:56.505+0000 7fa6a77fe640 1 -- 172.21.15.27:0/448174462 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6b4004400 con 0x7fa6c0070d30 2024-09-12T03:19:56.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:56.505+0000 7fa6c5053640 1 -- 172.21.15.27:0/448174462 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6c00b26a0 con 0x7fa6c0070d30 2024-09-12T03:19:56.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:56.506+0000 7fa6c5053640 1 -- 172.21.15.27:0/448174462 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6c006cc40 con 0x7fa6c0070d30 2024-09-12T03:19:56.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:56.507+0000 7fa6a77fe640 1 -- 172.21.15.27:0/448174462 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 11) ==== 90804+0+0 (secure 0 0 0) 0x7fa6b400c6d0 con 0x7fa6c0070d30 2024-09-12T03:19:56.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:56.507+0000 7fa6a77fe640 1 --2- 172.21.15.27:0/448174462 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fa6ac06c190 0x7fa6ac06e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:56.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:56.507+0000 7fa6a77fe640 1 -- 172.21.15.27:0/448174462 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 5227+0+0 (secure 0 0 0) 0x7fa6b408af70 con 0x7fa6c0070d30 2024-09-12T03:19:56.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:56.508+0000 7fa6bed9d640 1 --2- 172.21.15.27:0/448174462 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fa6ac06c190 0x7fa6ac06e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:56.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:56.508+0000 7fa6bed9d640 1 --2- 172.21.15.27:0/448174462 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fa6ac06c190 0x7fa6ac06e650 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fa6c0154b00 tx=0x7fa6b0009190 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:56.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:56.512+0000 7fa6a77fe640 1 -- 172.21.15.27:0/448174462 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fa6b4017020 con 0x7fa6c0070d30 2024-09-12T03:19:56.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:56.687+0000 7fa6c5053640 1 -- 172.21.15.27:0/448174462 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fa6c00b79d0 con 0x7fa6c0070d30 2024-09-12T03:19:57.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:57.373+0000 7fa6a77fe640 1 -- 172.21.15.27:0/448174462 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v65) ==== 170+0+0 (secure 0 0 0) 0x7fa6b4059020 con 0x7fa6c0070d30 2024-09-12T03:19:57.373 INFO:teuthology.orchestra.run.smithi027.stderr:pool 'cephfs_data' created 2024-09-12T03:19:57.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:57.377+0000 7fa6c5053640 1 -- 172.21.15.27:0/448174462 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fa6ac06c190 msgr2=0x7fa6ac06e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:57.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:57.377+0000 7fa6c5053640 1 --2- 172.21.15.27:0/448174462 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fa6ac06c190 0x7fa6ac06e650 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fa6c0154b00 tx=0x7fa6b0009190 comp rx=0 tx=0).stop 2024-09-12T03:19:57.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:57.377+0000 7fa6c5053640 1 -- 172.21.15.27:0/448174462 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa6c0070d30 msgr2=0x7fa6c0160040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:57.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:57.377+0000 7fa6c5053640 1 --2- 172.21.15.27:0/448174462 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa6c0070d30 0x7fa6c0160040 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fa6b400e7e0 tx=0x7fa6b400eca0 comp rx=0 tx=0).stop 2024-09-12T03:19:57.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:57.377+0000 7fa6c5053640 1 -- 172.21.15.27:0/448174462 shutdown_connections 2024-09-12T03:19:57.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:57.377+0000 7fa6c5053640 1 --2- 172.21.15.27:0/448174462 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fa6ac06c190 0x7fa6ac06e650 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:57.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:57.377+0000 7fa6c5053640 1 --2- 172.21.15.27:0/448174462 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa6c00d0980 0x7fa6c00cda20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:57.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:57.377+0000 7fa6c5053640 1 --2- 172.21.15.27:0/448174462 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa6c0070d30 0x7fa6c0160040 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:57.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:57.377+0000 7fa6c5053640 1 --2- 172.21.15.27:0/448174462 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa6c006e960 0x7fa6c01f59d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:57.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:57.377+0000 7fa6c5053640 1 -- 172.21.15.27:0/448174462 >> 172.21.15.27:0/448174462 conn(0x7fa6c00b7c10 msgr2=0x7fa6c00d2dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:57.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:57.377+0000 7fa6c5053640 1 -- 172.21.15.27:0/448174462 shutdown_connections 2024-09-12T03:19:57.379 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:57.378+0000 7fa6c5053640 1 -- 172.21.15.27:0/448174462 wait complete. 2024-09-12T03:19:57.404 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-12T03:19:57.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:57.514+0000 7f6e308d0640 1 Processor -- start 2024-09-12T03:19:57.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:57.514+0000 7f6e308d0640 1 -- start start 2024-09-12T03:19:57.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:57.515+0000 7f6e308d0640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f6e2c15a0a0 0x7f6e2c160c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:57.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:57.515+0000 7f6e308d0640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f6e2c0d04d0 0x7f6e2c15cae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:57.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:57.515+0000 7f6e308d0640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6e2c0b5b80 con 0x7f6e2c0ce280 2024-09-12T03:19:57.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:57.515+0000 7f6e308d0640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f6e2c06da80 con 0x7f6e2c0d04d0 2024-09-12T03:19:57.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:57.515+0000 7f6e308d0640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f6e2c0b02b0 con 0x7f6e2c15a0a0 2024-09-12T03:19:57.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:57.516+0000 7f6e2ad9d640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f6e2c0d04d0 0x7f6e2c15cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:57.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:57.516+0000 7f6e2ad9d640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f6e2c0d04d0 0x7f6e2c15cae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35656/0 (socket says 172.21.15.27:35656) 2024-09-12T03:19:57.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:57.516+0000 7f6e2ad9d640 1 -- 172.21.15.27:0/171166290 learned_addr learned my addr 172.21.15.27:0/171166290 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:19:57.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:57.516+0000 7f6e2a59c640 1 --2- 172.21.15.27:0/171166290 >> v2:172.21.15.45:3300/0 conn(0x7f6e2c15a0a0 0x7f6e2c160c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:57.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:57.516+0000 7f6e2ad9d640 1 -- 172.21.15.27:0/171166290 >> v2:172.21.15.45:3300/0 conn(0x7f6e2c15a0a0 msgr2=0x7f6e2c160c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:57.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:57.516+0000 7f6e2ad9d640 1 --2- 172.21.15.27:0/171166290 >> v2:172.21.15.45:3300/0 conn(0x7f6e2c15a0a0 0x7f6e2c160c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:57.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:57.516+0000 7f6e2ad9d640 1 -- 172.21.15.27:0/171166290 >> v1:172.21.15.45:6789/0 conn(0x7f6e2c0ce280 legacy=0x7f6e2c0cfdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:57.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:57.516+0000 7f6e2ad9d640 1 -- 172.21.15.27:0/171166290 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e2c0b6340 con 0x7f6e2c0d04d0 2024-09-12T03:19:57.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:57.517+0000 7f6e2ad9d640 1 --2- 172.21.15.27:0/171166290 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6e2c0d04d0 0x7f6e2c15cae0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f6e2c160060 tx=0x7f6e1c02a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b14de2269b3fd5b0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:57.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:57.517+0000 7f6e2959a640 1 -- 172.21.15.27:0/171166290 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e1c036070 con 0x7f6e2c0d04d0 2024-09-12T03:19:57.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:57.517+0000 7f6e2959a640 1 -- 172.21.15.27:0/171166290 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e1c031600 con 0x7f6e2c0d04d0 2024-09-12T03:19:57.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:57.517+0000 7f6e2959a640 1 -- 172.21.15.27:0/171166290 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e1c0319d0 con 0x7f6e2c0d04d0 2024-09-12T03:19:57.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:57.517+0000 7f6e308d0640 1 -- 172.21.15.27:0/171166290 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6e2c0d04d0 msgr2=0x7f6e2c15cae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:57.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:57.517+0000 7f6e308d0640 1 --2- 172.21.15.27:0/171166290 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6e2c0d04d0 0x7f6e2c15cae0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f6e2c160060 tx=0x7f6e1c02a6c0 comp rx=0 tx=0).stop 2024-09-12T03:19:57.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:57.517+0000 7f6e308d0640 1 -- 172.21.15.27:0/171166290 shutdown_connections 2024-09-12T03:19:57.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:57.517+0000 7f6e308d0640 1 --2- 172.21.15.27:0/171166290 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6e2c0d04d0 0x7f6e2c15cae0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:57.519 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:57.517+0000 7f6e308d0640 1 --2- 172.21.15.27:0/171166290 >> v2:172.21.15.45:3300/0 conn(0x7f6e2c15a0a0 0x7f6e2c160c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:57.519 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:57.517+0000 7f6e308d0640 1 -- 172.21.15.27:0/171166290 >> 172.21.15.27:0/171166290 conn(0x7f6e2c06bb30 msgr2=0x7f6e2c06bf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:57.519 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:57.518+0000 7f6e308d0640 1 -- 172.21.15.27:0/171166290 shutdown_connections 2024-09-12T03:19:57.519 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:57.518+0000 7f6e308d0640 1 -- 172.21.15.27:0/171166290 wait complete. 2024-09-12T03:19:57.522 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:57.519+0000 7f6e308d0640 1 Processor -- start 2024-09-12T03:19:57.522 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:57.519+0000 7f6e308d0640 1 -- start start 2024-09-12T03:19:57.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:57.520+0000 7f6e308d0640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6e2c0ce280 0x7f6e2c157040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:57.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:57.520+0000 7f6e308d0640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6e2c0d04d0 0x7f6e2c154b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:57.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:57.520+0000 7f6e308d0640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f6e2c15a0a0 0x7f6e2c14c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:57.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:57.520+0000 7f6e308d0640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f6e2c0b5e20 con 0x7f6e2c0ce280 2024-09-12T03:19:57.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:57.520+0000 7f6e308d0640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f6e2c0b07c0 con 0x7f6e2c0d04d0 2024-09-12T03:19:57.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:57.520+0000 7f6e308d0640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f6e2c0ce710 con 0x7f6e2c15a0a0 2024-09-12T03:19:57.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:57.520+0000 7f6e2a59c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6e2c0ce280 0x7f6e2c157040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:57.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:57.520+0000 7f6e2a59c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6e2c0ce280 0x7f6e2c157040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35658/0 (socket says 172.21.15.27:35658) 2024-09-12T03:19:57.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:57.520+0000 7f6e2a59c640 1 -- 172.21.15.27:0/1941176179 learned_addr learned my addr 172.21.15.27:0/1941176179 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:19:57.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:57.520+0000 7f6e2ad9d640 1 --2- 172.21.15.27:0/1941176179 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f6e2c15a0a0 0x7f6e2c14c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:57.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:57.520+0000 7f6e29d9b640 1 --2- 172.21.15.27:0/1941176179 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6e2c0d04d0 0x7f6e2c154b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:57.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:57.520+0000 7f6e2a59c640 1 -- 172.21.15.27:0/1941176179 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f6e2c15a0a0 msgr2=0x7f6e2c14c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:57.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:57.520+0000 7f6e2a59c640 1 --2- 172.21.15.27:0/1941176179 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f6e2c15a0a0 0x7f6e2c14c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:57.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:57.520+0000 7f6e2a59c640 1 -- 172.21.15.27:0/1941176179 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6e2c0d04d0 msgr2=0x7f6e2c154b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:57.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:57.520+0000 7f6e2a59c640 1 --2- 172.21.15.27:0/1941176179 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6e2c0d04d0 0x7f6e2c154b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:57.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:57.520+0000 7f6e2a59c640 1 -- 172.21.15.27:0/1941176179 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e2c158130 con 0x7f6e2c0ce280 2024-09-12T03:19:57.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:57.520+0000 7f6e29d9b640 1 --2- 172.21.15.27:0/1941176179 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6e2c0d04d0 0x7f6e2c154b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:19:57.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:57.520+0000 7f6e2a59c640 1 --2- 172.21.15.27:0/1941176179 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6e2c0ce280 0x7f6e2c157040 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f6e2400d5f0 tx=0x7f6e2400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:57.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:57.520+0000 7f6e22ffd640 1 -- 172.21.15.27:0/1941176179 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e240125f0 con 0x7f6e2c0ce280 2024-09-12T03:19:57.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:57.520+0000 7f6e22ffd640 1 -- 172.21.15.27:0/1941176179 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e24019070 con 0x7f6e2c0ce280 2024-09-12T03:19:57.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:57.520+0000 7f6e22ffd640 1 -- 172.21.15.27:0/1941176179 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e240129d0 con 0x7f6e2c0ce280 2024-09-12T03:19:57.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:57.520+0000 7f6e308d0640 1 -- 172.21.15.27:0/1941176179 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e2c158410 con 0x7f6e2c0ce280 2024-09-12T03:19:57.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:57.521+0000 7f6e308d0640 1 -- 172.21.15.27:0/1941176179 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e2c1f4780 con 0x7f6e2c0ce280 2024-09-12T03:19:57.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:57.521+0000 7f6e308d0640 1 -- 172.21.15.27:0/1941176179 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e2c15b600 con 0x7f6e2c0ce280 2024-09-12T03:19:57.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:57.522+0000 7f6e22ffd640 1 -- 172.21.15.27:0/1941176179 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 11) ==== 90804+0+0 (secure 0 0 0) 0x7f6e24012b70 con 0x7f6e2c0ce280 2024-09-12T03:19:57.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:57.523+0000 7f6e22ffd640 1 --2- 172.21.15.27:0/1941176179 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f6e0c06c190 0x7f6e0c06e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:57.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:57.523+0000 7f6e22ffd640 1 -- 172.21.15.27:0/1941176179 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 5609+0+0 (secure 0 0 0) 0x7f6e2408c100 con 0x7f6e2c0ce280 2024-09-12T03:19:57.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:57.523+0000 7f6e29d9b640 1 --2- 172.21.15.27:0/1941176179 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f6e0c06c190 0x7f6e0c06e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:57.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:57.524+0000 7f6e29d9b640 1 --2- 172.21.15.27:0/1941176179 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f6e0c06c190 0x7f6e0c06e650 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f6e2c073770 tx=0x7f6e14009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:57.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:57.527+0000 7f6e22ffd640 1 -- 172.21.15.27:0/1941176179 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f6e2405a790 con 0x7f6e2c0ce280 2024-09-12T03:19:57.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:57.737+0000 7f6e308d0640 1 -- 172.21.15.27:0/1941176179 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f6e2c0751f0 con 0x7f6e2c0ce280 2024-09-12T03:19:58.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:58.394+0000 7f6e22ffd640 1 -- 172.21.15.27:0/1941176179 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 10 and data pool 11 v36) ==== 174+0+0 (secure 0 0 0) 0x7f6e2405a090 con 0x7f6e2c0ce280 2024-09-12T03:19:58.396 INFO:teuthology.orchestra.run.smithi027.stderr:new fs with metadata pool 10 and data pool 11 2024-09-12T03:19:58.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:58.402+0000 7f6e308d0640 1 -- 172.21.15.27:0/1941176179 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f6e0c06c190 msgr2=0x7f6e0c06e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:58.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:58.402+0000 7f6e308d0640 1 --2- 172.21.15.27:0/1941176179 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f6e0c06c190 0x7f6e0c06e650 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f6e2c073770 tx=0x7f6e14009040 comp rx=0 tx=0).stop 2024-09-12T03:19:58.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:58.403+0000 7f6e308d0640 1 -- 172.21.15.27:0/1941176179 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6e2c0ce280 msgr2=0x7f6e2c157040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:58.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:58.403+0000 7f6e308d0640 1 --2- 172.21.15.27:0/1941176179 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6e2c0ce280 0x7f6e2c157040 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f6e2400d5f0 tx=0x7f6e2400dab0 comp rx=0 tx=0).stop 2024-09-12T03:19:58.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:58.403+0000 7f6e308d0640 1 -- 172.21.15.27:0/1941176179 shutdown_connections 2024-09-12T03:19:58.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:58.403+0000 7f6e308d0640 1 --2- 172.21.15.27:0/1941176179 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f6e0c06c190 0x7f6e0c06e650 secure :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f6e2c073770 tx=0x7f6e14009040 comp rx=0 tx=0).stop 2024-09-12T03:19:58.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:58.403+0000 7f6e308d0640 1 --2- 172.21.15.27:0/1941176179 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f6e2c15a0a0 0x7f6e2c14c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:58.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:58.403+0000 7f6e308d0640 1 --2- 172.21.15.27:0/1941176179 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6e2c0d04d0 0x7f6e2c154b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:58.405 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:58.403+0000 7f6e308d0640 1 --2- 172.21.15.27:0/1941176179 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6e2c0ce280 0x7f6e2c157040 secure :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f6e2400d5f0 tx=0x7f6e2400dab0 comp rx=0 tx=0).stop 2024-09-12T03:19:58.405 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:58.403+0000 7f6e308d0640 1 -- 172.21.15.27:0/1941176179 >> 172.21.15.27:0/1941176179 conn(0x7f6e2c06bb30 msgr2=0x7f6e2c0b0ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:58.405 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:58.404+0000 7f6e308d0640 1 -- 172.21.15.27:0/1941176179 shutdown_connections 2024-09-12T03:19:58.405 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:58.404+0000 7f6e308d0640 1 -- 172.21.15.27:0/1941176179 wait complete. 2024-09-12T03:19:58.426 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-12T03:19:58.532 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:58.531+0000 7f601dcac640 1 Processor -- start 2024-09-12T03:19:58.532 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:58.531+0000 7f601dcac640 1 -- start start 2024-09-12T03:19:58.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:58.532+0000 7f601dcac640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f601815e5f0 0x7f6018073250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:58.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:58.532+0000 7f601dcac640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f6018073790 0x7f6018162320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:58.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:58.532+0000 7f601dcac640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6018073bb0 con 0x7f6018162860 2024-09-12T03:19:58.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:58.532+0000 7f601dcac640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f60180d0da0 con 0x7f6018073790 2024-09-12T03:19:58.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:58.532+0000 7f601dcac640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f6018163090 con 0x7f601815e5f0 2024-09-12T03:19:58.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:58.532+0000 7f6016ffd640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f6018073790 0x7f6018162320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:58.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:58.532+0000 7f6017fff640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f6018162860 0x7f60180d1230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:53120/0 (socket says 172.21.15.27:53120) 2024-09-12T03:19:58.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:58.532+0000 7f6016ffd640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f6018073790 0x7f6018162320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35672/0 (socket says 172.21.15.27:35672) 2024-09-12T03:19:58.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:58.532+0000 7f6017fff640 1 -- 172.21.15.27:0/3914607735 learned_addr learned my addr 172.21.15.27:0/3914607735 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:19:58.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:58.532+0000 7f60177fe640 1 --2- 172.21.15.27:0/3914607735 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f601815e5f0 0x7f6018073250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:58.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:58.532+0000 7f6016ffd640 1 -- 172.21.15.27:0/3914607735 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f601815e5f0 msgr2=0x7f6018073250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:58.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:58.532+0000 7f6016ffd640 1 --2- 172.21.15.27:0/3914607735 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f601815e5f0 0x7f6018073250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:58.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:58.532+0000 7f6016ffd640 1 -- 172.21.15.27:0/3914607735 >> v1:172.21.15.27:6789/0 conn(0x7f6018162860 legacy=0x7f60180d1230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:58.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:58.532+0000 7f6016ffd640 1 -- 172.21.15.27:0/3914607735 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60180d0470 con 0x7f6018073790 2024-09-12T03:19:58.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:58.532+0000 7f60167fc640 1 -- 172.21.15.27:0/3914607735 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1742371874 0 0) 0x7f6018073bb0 con 0x7f6018162860 2024-09-12T03:19:58.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:58.533+0000 7f6016ffd640 1 --2- 172.21.15.27:0/3914607735 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6018073790 0x7f6018162320 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f600c0097a0 tx=0x7f600c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b8c00bcdad9ae03 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:58.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:58.533+0000 7f60167fc640 1 -- 172.21.15.27:0/3914607735 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f600c037070 con 0x7f6018073790 2024-09-12T03:19:58.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:58.533+0000 7f60167fc640 1 -- 172.21.15.27:0/3914607735 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f600c032440 con 0x7f6018073790 2024-09-12T03:19:58.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:58.533+0000 7f60167fc640 1 -- 172.21.15.27:0/3914607735 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f600c032810 con 0x7f6018073790 2024-09-12T03:19:58.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:58.534+0000 7f601dcac640 1 -- 172.21.15.27:0/3914607735 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6018073790 msgr2=0x7f6018162320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:58.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:58.534+0000 7f601dcac640 1 --2- 172.21.15.27:0/3914607735 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6018073790 0x7f6018162320 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f600c0097a0 tx=0x7f600c02a180 comp rx=0 tx=0).stop 2024-09-12T03:19:58.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:58.534+0000 7f601dcac640 1 -- 172.21.15.27:0/3914607735 shutdown_connections 2024-09-12T03:19:58.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:58.534+0000 7f601dcac640 1 --2- 172.21.15.27:0/3914607735 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6018073790 0x7f6018162320 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:58.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:58.534+0000 7f601dcac640 1 --2- 172.21.15.27:0/3914607735 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f601815e5f0 0x7f6018073250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:58.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:58.534+0000 7f601dcac640 1 -- 172.21.15.27:0/3914607735 >> 172.21.15.27:0/3914607735 conn(0x7f601806f300 msgr2=0x7f601806f710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:58.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:58.534+0000 7f601dcac640 1 -- 172.21.15.27:0/3914607735 shutdown_connections 2024-09-12T03:19:58.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:58.534+0000 7f601dcac640 1 -- 172.21.15.27:0/3914607735 wait complete. 2024-09-12T03:19:58.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:58.535+0000 7f601dcac640 1 Processor -- start 2024-09-12T03:19:58.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:58.535+0000 7f601dcac640 1 -- start start 2024-09-12T03:19:58.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:58.536+0000 7f601dcac640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6018073790 0x7f601815c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:58.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:58.536+0000 7f601dcac640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f601815e5f0 0x7f6018156040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:58.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:58.536+0000 7f601dcac640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6018162860 0x7f6018156580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:58.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:58.536+0000 7f601dcac640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f60180b5620 con 0x7f6018162860 2024-09-12T03:19:58.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:58.536+0000 7f601dcac640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f60180d0a30 con 0x7f6018073790 2024-09-12T03:19:58.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:58.536+0000 7f601dcac640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f6018162cf0 con 0x7f601815e5f0 2024-09-12T03:19:58.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:58.536+0000 7f6017fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6018162860 0x7f6018156580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:58.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:58.536+0000 7f6017fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6018162860 0x7f6018156580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35676/0 (socket says 172.21.15.27:35676) 2024-09-12T03:19:58.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:58.536+0000 7f6017fff640 1 -- 172.21.15.27:0/1299288349 learned_addr learned my addr 172.21.15.27:0/1299288349 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:19:58.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:58.536+0000 7f6016ffd640 1 --2- 172.21.15.27:0/1299288349 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f601815e5f0 0x7f6018156040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:58.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:58.536+0000 7f60177fe640 1 --2- 172.21.15.27:0/1299288349 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6018073790 0x7f601815c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:58.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:58.536+0000 7f6017fff640 1 -- 172.21.15.27:0/1299288349 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f601815e5f0 msgr2=0x7f6018156040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:58.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:58.536+0000 7f6017fff640 1 --2- 172.21.15.27:0/1299288349 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f601815e5f0 0x7f6018156040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:58.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:58.536+0000 7f6017fff640 1 -- 172.21.15.27:0/1299288349 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6018073790 msgr2=0x7f601815c0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:58.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:58.536+0000 7f6017fff640 1 --2- 172.21.15.27:0/1299288349 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6018073790 0x7f601815c0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:58.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:58.537+0000 7f6017fff640 1 -- 172.21.15.27:0/1299288349 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6018151480 con 0x7f6018162860 2024-09-12T03:19:58.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:58.537+0000 7f60177fe640 1 --2- 172.21.15.27:0/1299288349 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6018073790 0x7f601815c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:19:58.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:58.537+0000 7f6016ffd640 1 --2- 172.21.15.27:0/1299288349 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f601815e5f0 0x7f6018156040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:19:58.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:58.537+0000 7f6017fff640 1 --2- 172.21.15.27:0/1299288349 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6018162860 0x7f6018156580 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f601000c730 tx=0x7f601000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:58.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:58.537+0000 7f5ffffff640 1 -- 172.21.15.27:0/1299288349 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6010018070 con 0x7f6018162860 2024-09-12T03:19:58.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:58.537+0000 7f5ffffff640 1 -- 172.21.15.27:0/1299288349 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f601000ce40 con 0x7f6018162860 2024-09-12T03:19:58.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:58.537+0000 7f601dcac640 1 -- 172.21.15.27:0/1299288349 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6018151720 con 0x7f6018162860 2024-09-12T03:19:58.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:58.537+0000 7f5ffffff640 1 -- 172.21.15.27:0/1299288349 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f601000a860 con 0x7f6018162860 2024-09-12T03:19:58.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:58.537+0000 7f601dcac640 1 -- 172.21.15.27:0/1299288349 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6018151ce0 con 0x7f6018162860 2024-09-12T03:19:58.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:58.538+0000 7f601dcac640 1 -- 172.21.15.27:0/1299288349 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f601815f4c0 con 0x7f6018162860 2024-09-12T03:19:58.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:58.538+0000 7f5ffffff640 1 -- 172.21.15.27:0/1299288349 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 11) ==== 90804+0+0 (secure 0 0 0) 0x7f601001b070 con 0x7f6018162860 2024-09-12T03:19:58.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:58.539+0000 7f5ffffff640 1 --2- 172.21.15.27:0/1299288349 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f600406c190 0x7f600406e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:58.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:58.539+0000 7f5ffffff640 1 -- 172.21.15.27:0/1299288349 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 5709+0+0 (secure 0 0 0) 0x7f601008a990 con 0x7f6018162860 2024-09-12T03:19:58.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:58.539+0000 7f60177fe640 1 --2- 172.21.15.27:0/1299288349 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f600406c190 0x7f600406e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:58.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:58.540+0000 7f60177fe640 1 --2- 172.21.15.27:0/1299288349 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f600406c190 0x7f600406e650 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f6008006fd0 tx=0x7f6008008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:58.544 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:58.543+0000 7f5ffffff640 1 -- 172.21.15.27:0/1299288349 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f6010058ec0 con 0x7f6018162860 2024-09-12T03:19:58.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:58.718+0000 7f601dcac640 1 -- 172.21.15.27:0/1299288349 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f601815ef00 con 0x7f6018162860 2024-09-12T03:19:58.720 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:58.720+0000 7f5ffffff640 1 -- 172.21.15.27:0/1299288349 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v66) ==== 74+0+16693 (secure 0 0 0) 0x7f60100587c0 con 0x7f6018162860 2024-09-12T03:19:58.721 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:19:58.721 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":66,"fsid":"d2df504e-2647-4385-a5db-9fdf69f487a0","created":"2024-09-12T03:16:07.778937+0000","modified":"2024-09-12T03:19:57.738546+0000","last_up_change":"2024-09-12T03:16:19.314778+0000","last_in_change":"2024-09-12T03:16:09.893086+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T03:16:20.097646+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-09-12T03:19:55.777868+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-09-12T03:19:56.718653+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1851116c-1fb3-4dc9-9772-f90e2ef4c355","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6817","nonce":3002321488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6819","nonce":3002321488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6823","nonce":3002321488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6821","nonce":3002321488}]},"public_addr":"172.21.15.27:6817/3002321488","cluster_addr":"172.21.15.27:6819/3002321488","heartbeat_back_addr":"172.21.15.27:6823/3002321488","heartbeat_front_addr":"172.21.15.27:6821/3002321488","state":["exists","up"]},{"osd":1,"uuid":"f38ae17d-73eb-4204-b99d-975fa806fb87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6809","nonce":1613102886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6811","nonce":1613102886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6815","nonce":1613102886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6813","nonce":1613102886}]},"public_addr":"172.21.15.27:6809/1613102886","cluster_addr":"172.21.15.27:6811/1613102886","heartbeat_back_addr":"172.21.15.27:6815/1613102886","heartbeat_front_addr":"172.21.15.27:6813/1613102886","state":["exists","up"]},{"osd":2,"uuid":"fb12dcc8-bdd0-42b8-b46d-8b10106224bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6825","nonce":3534723355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6827","nonce":3534723355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6831","nonce":3534723355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6829","nonce":3534723355}]},"public_addr":"172.21.15.27:6825/3534723355","cluster_addr":"172.21.15.27:6827/3534723355","heartbeat_back_addr":"172.21.15.27:6831/3534723355","heartbeat_front_addr":"172.21.15.27:6829/3534723355","state":["exists","up"]},{"osd":3,"uuid":"d74d3f49-9c24-4b9f-98e2-3ecd7b5ef3b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6801","nonce":80661598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6803","nonce":80661598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6807","nonce":80661598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6805","nonce":80661598}]},"public_addr":"172.21.15.27:6801/80661598","cluster_addr":"172.21.15.27:6803/80661598","heartbeat_back_addr":"172.21.15.27:6807/80661598","heartbeat_front_addr":"172.21.15.27:6805/80661598","state":["exists","up"]},{"osd":4,"uuid":"2bf123f2-1b0d-41c4-aa3b-805bab267e59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6817","nonce":3943189827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6819","nonce":3943189827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6823","nonce":3943189827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6821","nonce":3943189827}]},"public_addr":"172.21.15.45:6817/3943189827","cluster_addr":"172.21.15.45:6819/3943189827","heartbeat_back_addr":"172.21.15.45:6823/3943189827","heartbeat_front_addr":"172.21.15.45:6821/3943189827","state":["exists","up"]},{"osd":5,"uuid":"c49ba5da-91c6-4a86-a97b-4aba89c253ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6809","nonce":3620760330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6811","nonce":3620760330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6815","nonce":3620760330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6813","nonce":3620760330}]},"public_addr":"172.21.15.45:6809/3620760330","cluster_addr":"172.21.15.45:6811/3620760330","heartbeat_back_addr":"172.21.15.45:6815/3620760330","heartbeat_front_addr":"172.21.15.45:6813/3620760330","state":["exists","up"]},{"osd":6,"uuid":"b0123475-89f7-4a34-9e33-d797b99dd9dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6801","nonce":1989083510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6803","nonce":1989083510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6807","nonce":1989083510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6805","nonce":1989083510}]},"public_addr":"172.21.15.45:6801/1989083510","cluster_addr":"172.21.15.45:6803/1989083510","heartbeat_back_addr":"172.21.15.45:6807/1989083510","heartbeat_front_addr":"172.21.15.45:6805/1989083510","state":["exists","up"]},{"osd":7,"uuid":"4dad773d-b7d8-46fc-8a10-a972aad27b29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6825","nonce":1513300294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6827","nonce":1513300294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6831","nonce":1513300294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6829","nonce":1513300294}]},"public_addr":"172.21.15.45:6825/1513300294","cluster_addr":"172.21.15.45:6827/1513300294","heartbeat_back_addr":"172.21.15.45:6831/1513300294","heartbeat_front_addr":"172.21.15.45:6829/1513300294","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.881855+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:17.133252+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.735339+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.500742+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.477571+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:17.167667+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.470291+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.873743+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T03:19:58.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:58.722+0000 7f601dcac640 1 -- 172.21.15.27:0/1299288349 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f600406c190 msgr2=0x7f600406e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:58.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:58.722+0000 7f601dcac640 1 --2- 172.21.15.27:0/1299288349 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f600406c190 0x7f600406e650 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f6008006fd0 tx=0x7f6008008040 comp rx=0 tx=0).stop 2024-09-12T03:19:58.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:58.722+0000 7f601dcac640 1 -- 172.21.15.27:0/1299288349 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6018162860 msgr2=0x7f6018156580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:58.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:58.722+0000 7f601dcac640 1 --2- 172.21.15.27:0/1299288349 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6018162860 0x7f6018156580 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f601000c730 tx=0x7f601000cbf0 comp rx=0 tx=0).stop 2024-09-12T03:19:58.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:58.722+0000 7f601dcac640 1 -- 172.21.15.27:0/1299288349 shutdown_connections 2024-09-12T03:19:58.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:58.722+0000 7f601dcac640 1 --2- 172.21.15.27:0/1299288349 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f600406c190 0x7f600406e650 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:58.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:58.722+0000 7f601dcac640 1 --2- 172.21.15.27:0/1299288349 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6018162860 0x7f6018156580 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:58.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:58.722+0000 7f601dcac640 1 --2- 172.21.15.27:0/1299288349 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f601815e5f0 0x7f6018156040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:58.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:58.722+0000 7f601dcac640 1 --2- 172.21.15.27:0/1299288349 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6018073790 0x7f601815c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:58.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:58.722+0000 7f601dcac640 1 -- 172.21.15.27:0/1299288349 >> 172.21.15.27:0/1299288349 conn(0x7f601806f300 msgr2=0x7f60180b5910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:58.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:58.722+0000 7f601dcac640 1 -- 172.21.15.27:0/1299288349 shutdown_connections 2024-09-12T03:19:58.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:58.722+0000 7f601dcac640 1 -- 172.21.15.27:0/1299288349 wait complete. 2024-09-12T03:19:58.734 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-12T03:19:58.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:58.835+0000 7fe8c7ed0640 1 Processor -- start 2024-09-12T03:19:58.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:58.835+0000 7fe8c7ed0640 1 -- start start 2024-09-12T03:19:58.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:58.836+0000 7fe8c7ed0640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe8c00b1ca0 0x7fe8c015f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:58.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:58.836+0000 7fe8c7ed0640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fe8c00cf660 0x7fe8c00d2780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:58.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:58.836+0000 7fe8c7ed0640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe8c015fd70 con 0x7fe8c006f360 2024-09-12T03:19:58.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:58.836+0000 7fe8c7ed0640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7fe8c00d22f0 con 0x7fe8c00cf660 2024-09-12T03:19:58.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:58.836+0000 7fe8c7ed0640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fe8c006cc40 con 0x7fe8c00b1ca0 2024-09-12T03:19:58.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:58.836+0000 7fe8c646d640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fe8c00cf660 0x7fe8c00d2780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:58.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:58.836+0000 7fe8c646d640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fe8c00cf660 0x7fe8c00d2780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35690/0 (socket says 172.21.15.27:35690) 2024-09-12T03:19:58.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:58.836+0000 7fe8c646d640 1 -- 172.21.15.27:0/1953171270 learned_addr learned my addr 172.21.15.27:0/1953171270 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:19:58.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:58.836+0000 7fe8c5c6c640 1 --2- 172.21.15.27:0/1953171270 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe8c00b1ca0 0x7fe8c015f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:58.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:58.837+0000 7fe8c646d640 1 -- 172.21.15.27:0/1953171270 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe8c00b1ca0 msgr2=0x7fe8c015f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:58.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:58.837+0000 7fe8c646d640 1 --2- 172.21.15.27:0/1953171270 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe8c00b1ca0 0x7fe8c015f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:58.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:58.837+0000 7fe8c646d640 1 -- 172.21.15.27:0/1953171270 >> v1:172.21.15.45:6789/0 conn(0x7fe8c006f360 legacy=0x7fe8c00cef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:58.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:58.837+0000 7fe8c646d640 1 -- 172.21.15.27:0/1953171270 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8c00d1ce0 con 0x7fe8c00cf660 2024-09-12T03:19:58.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:58.837+0000 7fe8c646d640 1 --2- 172.21.15.27:0/1953171270 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe8c00cf660 0x7fe8c00d2780 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fe8b8009e80 tx=0x7fe8b802a750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61058061dc4cb9b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:58.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:58.837+0000 7fe8c4c6a640 1 -- 172.21.15.27:0/1953171270 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8b8036070 con 0x7fe8c00cf660 2024-09-12T03:19:58.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:58.838+0000 7fe8c4c6a640 1 -- 172.21.15.27:0/1953171270 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8b8031600 con 0x7fe8c00cf660 2024-09-12T03:19:58.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:58.838+0000 7fe8c4c6a640 1 -- 172.21.15.27:0/1953171270 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8b80319d0 con 0x7fe8c00cf660 2024-09-12T03:19:58.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:58.838+0000 7fe8c7ed0640 1 -- 172.21.15.27:0/1953171270 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe8c00cf660 msgr2=0x7fe8c00d2780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:58.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:58.838+0000 7fe8c7ed0640 1 --2- 172.21.15.27:0/1953171270 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe8c00cf660 0x7fe8c00d2780 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fe8b8009e80 tx=0x7fe8b802a750 comp rx=0 tx=0).stop 2024-09-12T03:19:58.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:58.838+0000 7fe8c7ed0640 1 -- 172.21.15.27:0/1953171270 shutdown_connections 2024-09-12T03:19:58.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:58.838+0000 7fe8c7ed0640 1 --2- 172.21.15.27:0/1953171270 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe8c00cf660 0x7fe8c00d2780 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:58.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:58.838+0000 7fe8c7ed0640 1 --2- 172.21.15.27:0/1953171270 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe8c00b1ca0 0x7fe8c015f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:58.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:58.838+0000 7fe8c7ed0640 1 -- 172.21.15.27:0/1953171270 >> 172.21.15.27:0/1953171270 conn(0x7fe8c0073800 msgr2=0x7fe8c009ba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:58.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:58.838+0000 7fe8c7ed0640 1 -- 172.21.15.27:0/1953171270 shutdown_connections 2024-09-12T03:19:58.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:58.838+0000 7fe8c7ed0640 1 -- 172.21.15.27:0/1953171270 wait complete. 2024-09-12T03:19:58.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:58.839+0000 7fe8c7ed0640 1 Processor -- start 2024-09-12T03:19:58.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:58.839+0000 7fe8c7ed0640 1 -- start start 2024-09-12T03:19:58.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:58.840+0000 7fe8c7ed0640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe8c006f360 0x7fe8c00cb850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:58.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:58.840+0000 7fe8c7ed0640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe8c00b1ca0 0x7fe8c014f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:58.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:58.840+0000 7fe8c7ed0640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe8c00cf660 0x7fe8c00ce530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:58.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:58.840+0000 7fe8c7ed0640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fe8c006d1c0 con 0x7fe8c00cf660 2024-09-12T03:19:58.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:58.840+0000 7fe8c7ed0640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe8c00d1f80 con 0x7fe8c00b1ca0 2024-09-12T03:19:58.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:58.840+0000 7fe8c7ed0640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fe8c006d340 con 0x7fe8c006f360 2024-09-12T03:19:58.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:58.840+0000 7fe8c646d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe8c00cf660 0x7fe8c00ce530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:58.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:58.840+0000 7fe8c646d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe8c00cf660 0x7fe8c00ce530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35706/0 (socket says 172.21.15.27:35706) 2024-09-12T03:19:58.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:58.840+0000 7fe8c646d640 1 -- 172.21.15.27:0/460375437 learned_addr learned my addr 172.21.15.27:0/460375437 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:19:58.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:58.840+0000 7fe8c5c6c640 1 --2- 172.21.15.27:0/460375437 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe8c006f360 0x7fe8c00cb850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:58.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:58.840+0000 7fe8c546b640 1 --2- 172.21.15.27:0/460375437 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe8c00b1ca0 0x7fe8c014f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:58.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:58.841+0000 7fe8c646d640 1 -- 172.21.15.27:0/460375437 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe8c006f360 msgr2=0x7fe8c00cb850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:58.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:58.841+0000 7fe8c646d640 1 --2- 172.21.15.27:0/460375437 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe8c006f360 0x7fe8c00cb850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:58.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:58.841+0000 7fe8c646d640 1 -- 172.21.15.27:0/460375437 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe8c00b1ca0 msgr2=0x7fe8c014f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:58.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:58.841+0000 7fe8c646d640 1 --2- 172.21.15.27:0/460375437 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe8c00b1ca0 0x7fe8c014f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:58.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:58.841+0000 7fe8c646d640 1 -- 172.21.15.27:0/460375437 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8c00b5670 con 0x7fe8c00cf660 2024-09-12T03:19:58.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:58.841+0000 7fe8c546b640 1 --2- 172.21.15.27:0/460375437 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe8c00b1ca0 0x7fe8c014f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:19:58.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:58.841+0000 7fe8c646d640 1 --2- 172.21.15.27:0/460375437 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe8c00cf660 0x7fe8c00ce530 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fe8b8034040 tx=0x7fe8b8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:58.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:58.841+0000 7fe8b67fc640 1 -- 172.21.15.27:0/460375437 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8b8041020 con 0x7fe8c00cf660 2024-09-12T03:19:58.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:58.841+0000 7fe8b67fc640 1 -- 172.21.15.27:0/460375437 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8b8007720 con 0x7fe8c00cf660 2024-09-12T03:19:58.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:58.841+0000 7fe8c7ed0640 1 -- 172.21.15.27:0/460375437 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8c00b5950 con 0x7fe8c00cf660 2024-09-12T03:19:58.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:58.841+0000 7fe8b67fc640 1 -- 172.21.15.27:0/460375437 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8b8036040 con 0x7fe8c00cf660 2024-09-12T03:19:58.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:58.841+0000 7fe8c7ed0640 1 -- 172.21.15.27:0/460375437 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8c00b5f10 con 0x7fe8c00cf660 2024-09-12T03:19:58.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:58.842+0000 7fe8c7ed0640 1 -- 172.21.15.27:0/460375437 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8c006c660 con 0x7fe8c00cf660 2024-09-12T03:19:58.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:58.843+0000 7fe8b67fc640 1 -- 172.21.15.27:0/460375437 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 11) ==== 90804+0+0 (secure 0 0 0) 0x7fe8b8044020 con 0x7fe8c00cf660 2024-09-12T03:19:58.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:58.843+0000 7fe8b67fc640 1 --2- 172.21.15.27:0/460375437 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fe8a406c190 0x7fe8a406e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:58.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:58.843+0000 7fe8b67fc640 1 -- 172.21.15.27:0/460375437 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 5709+0+0 (secure 0 0 0) 0x7fe8b80ac030 con 0x7fe8c00cf660 2024-09-12T03:19:58.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:58.843+0000 7fe8c5c6c640 1 --2- 172.21.15.27:0/460375437 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fe8a406c190 0x7fe8a406e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:58.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:58.844+0000 7fe8c5c6c640 1 --2- 172.21.15.27:0/460375437 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fe8a406c190 0x7fe8a406e650 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fe8c01f7ce0 tx=0x7fe8bc009190 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:58.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:58.847+0000 7fe8b67fc640 1 -- 172.21.15.27:0/460375437 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fe8c006c660 con 0x7fe8c00cf660 2024-09-12T03:19:59.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:59.025+0000 7fe8c7ed0640 1 -- 172.21.15.27:0/460375437 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe8c009c1b0 con 0x7fe8c00cf660 2024-09-12T03:19:59.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:59.026+0000 7fe8b67fc640 1 -- 172.21.15.27:0/460375437 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v66) ==== 74+0+16693 (secure 0 0 0) 0x7fe8c009c1b0 con 0x7fe8c00cf660 2024-09-12T03:19:59.027 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:19:59.027 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":66,"fsid":"d2df504e-2647-4385-a5db-9fdf69f487a0","created":"2024-09-12T03:16:07.778937+0000","modified":"2024-09-12T03:19:57.738546+0000","last_up_change":"2024-09-12T03:16:19.314778+0000","last_in_change":"2024-09-12T03:16:09.893086+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T03:16:20.097646+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-09-12T03:19:55.777868+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-09-12T03:19:56.718653+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1851116c-1fb3-4dc9-9772-f90e2ef4c355","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6817","nonce":3002321488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6819","nonce":3002321488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6823","nonce":3002321488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6821","nonce":3002321488}]},"public_addr":"172.21.15.27:6817/3002321488","cluster_addr":"172.21.15.27:6819/3002321488","heartbeat_back_addr":"172.21.15.27:6823/3002321488","heartbeat_front_addr":"172.21.15.27:6821/3002321488","state":["exists","up"]},{"osd":1,"uuid":"f38ae17d-73eb-4204-b99d-975fa806fb87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6809","nonce":1613102886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6811","nonce":1613102886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6815","nonce":1613102886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6813","nonce":1613102886}]},"public_addr":"172.21.15.27:6809/1613102886","cluster_addr":"172.21.15.27:6811/1613102886","heartbeat_back_addr":"172.21.15.27:6815/1613102886","heartbeat_front_addr":"172.21.15.27:6813/1613102886","state":["exists","up"]},{"osd":2,"uuid":"fb12dcc8-bdd0-42b8-b46d-8b10106224bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6825","nonce":3534723355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6827","nonce":3534723355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6831","nonce":3534723355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6829","nonce":3534723355}]},"public_addr":"172.21.15.27:6825/3534723355","cluster_addr":"172.21.15.27:6827/3534723355","heartbeat_back_addr":"172.21.15.27:6831/3534723355","heartbeat_front_addr":"172.21.15.27:6829/3534723355","state":["exists","up"]},{"osd":3,"uuid":"d74d3f49-9c24-4b9f-98e2-3ecd7b5ef3b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6801","nonce":80661598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6803","nonce":80661598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6807","nonce":80661598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6805","nonce":80661598}]},"public_addr":"172.21.15.27:6801/80661598","cluster_addr":"172.21.15.27:6803/80661598","heartbeat_back_addr":"172.21.15.27:6807/80661598","heartbeat_front_addr":"172.21.15.27:6805/80661598","state":["exists","up"]},{"osd":4,"uuid":"2bf123f2-1b0d-41c4-aa3b-805bab267e59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6817","nonce":3943189827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6819","nonce":3943189827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6823","nonce":3943189827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6821","nonce":3943189827}]},"public_addr":"172.21.15.45:6817/3943189827","cluster_addr":"172.21.15.45:6819/3943189827","heartbeat_back_addr":"172.21.15.45:6823/3943189827","heartbeat_front_addr":"172.21.15.45:6821/3943189827","state":["exists","up"]},{"osd":5,"uuid":"c49ba5da-91c6-4a86-a97b-4aba89c253ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6809","nonce":3620760330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6811","nonce":3620760330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6815","nonce":3620760330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6813","nonce":3620760330}]},"public_addr":"172.21.15.45:6809/3620760330","cluster_addr":"172.21.15.45:6811/3620760330","heartbeat_back_addr":"172.21.15.45:6815/3620760330","heartbeat_front_addr":"172.21.15.45:6813/3620760330","state":["exists","up"]},{"osd":6,"uuid":"b0123475-89f7-4a34-9e33-d797b99dd9dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6801","nonce":1989083510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6803","nonce":1989083510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6807","nonce":1989083510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6805","nonce":1989083510}]},"public_addr":"172.21.15.45:6801/1989083510","cluster_addr":"172.21.15.45:6803/1989083510","heartbeat_back_addr":"172.21.15.45:6807/1989083510","heartbeat_front_addr":"172.21.15.45:6805/1989083510","state":["exists","up"]},{"osd":7,"uuid":"4dad773d-b7d8-46fc-8a10-a972aad27b29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6825","nonce":1513300294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6827","nonce":1513300294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6831","nonce":1513300294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6829","nonce":1513300294}]},"public_addr":"172.21.15.45:6825/1513300294","cluster_addr":"172.21.15.45:6827/1513300294","heartbeat_back_addr":"172.21.15.45:6831/1513300294","heartbeat_front_addr":"172.21.15.45:6829/1513300294","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.881855+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:17.133252+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.735339+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.500742+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.477571+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:17.167667+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.470291+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.873743+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T03:19:59.029 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:59.028+0000 7fe8c7ed0640 1 -- 172.21.15.27:0/460375437 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fe8a406c190 msgr2=0x7fe8a406e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:59.029 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:59.028+0000 7fe8c7ed0640 1 --2- 172.21.15.27:0/460375437 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fe8a406c190 0x7fe8a406e650 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fe8c01f7ce0 tx=0x7fe8bc009190 comp rx=0 tx=0).stop 2024-09-12T03:19:59.029 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:59.028+0000 7fe8c7ed0640 1 -- 172.21.15.27:0/460375437 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe8c00cf660 msgr2=0x7fe8c00ce530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:59.029 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:59.028+0000 7fe8c7ed0640 1 --2- 172.21.15.27:0/460375437 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe8c00cf660 0x7fe8c00ce530 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fe8b8034040 tx=0x7fe8b8004a40 comp rx=0 tx=0).stop 2024-09-12T03:19:59.029 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:59.028+0000 7fe8c7ed0640 1 -- 172.21.15.27:0/460375437 shutdown_connections 2024-09-12T03:19:59.029 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:59.028+0000 7fe8c7ed0640 1 --2- 172.21.15.27:0/460375437 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fe8a406c190 0x7fe8a406e650 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:59.029 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:59.028+0000 7fe8c7ed0640 1 --2- 172.21.15.27:0/460375437 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe8c00cf660 0x7fe8c00ce530 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:59.029 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:59.028+0000 7fe8c7ed0640 1 --2- 172.21.15.27:0/460375437 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe8c00b1ca0 0x7fe8c014f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:59.029 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:59.028+0000 7fe8c7ed0640 1 --2- 172.21.15.27:0/460375437 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe8c006f360 0x7fe8c00cb850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:59.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:59.028+0000 7fe8c7ed0640 1 -- 172.21.15.27:0/460375437 >> 172.21.15.27:0/460375437 conn(0x7fe8c0073800 msgr2=0x7fe8c00d0440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:59.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:59.028+0000 7fe8c7ed0640 1 -- 172.21.15.27:0/460375437 shutdown_connections 2024-09-12T03:19:59.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:59.028+0000 7fe8c7ed0640 1 -- 172.21.15.27:0/460375437 wait complete. 2024-09-12T03:19:59.040 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-12T03:19:59.145 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:59.144+0000 7f1a87e30640 1 Processor -- start 2024-09-12T03:19:59.145 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:59.144+0000 7f1a87e30640 1 -- start start 2024-09-12T03:19:59.148 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:59.145+0000 7f1a87e30640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f1a800d1040 0x7f1a800d14b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:59.148 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:59.145+0000 7f1a87e30640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1a800b6f70 0x7f1a800b7400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:59.149 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:59.145+0000 7f1a87e30640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1a800d07b0 con 0x7f1a8015c6c0 2024-09-12T03:19:59.149 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:59.145+0000 7f1a87e30640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f1a8015caa0 con 0x7f1a800d1040 2024-09-12T03:19:59.149 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:59.145+0000 7f1a87e30640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f1a800b6510 con 0x7f1a800b6f70 2024-09-12T03:19:59.149 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:59.145+0000 7f1a853cb640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f1a800d1040 0x7f1a800d14b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:59.149 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:59.145+0000 7f1a85bcc640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f1a8015c6c0 0x7f1a800d00a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:53122/0 (socket says 172.21.15.27:53122) 2024-09-12T03:19:59.149 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:59.145+0000 7f1a853cb640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f1a800d1040 0x7f1a800d14b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35720/0 (socket says 172.21.15.27:35720) 2024-09-12T03:19:59.149 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:59.145+0000 7f1a85bcc640 1 -- 172.21.15.27:0/778940764 learned_addr learned my addr 172.21.15.27:0/778940764 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:19:59.149 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:59.145+0000 7f1a853cb640 1 -- 172.21.15.27:0/778940764 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1a800b6f70 msgr2=0x7f1a800b7400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:59.149 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:59.145+0000 7f1a863cd640 1 --2- 172.21.15.27:0/778940764 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1a800b6f70 0x7f1a800b7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:59.150 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:59.145+0000 7f1a853cb640 1 --2- 172.21.15.27:0/778940764 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1a800b6f70 0x7f1a800b7400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:59.150 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:59.145+0000 7f1a84bca640 1 -- 172.21.15.27:0/778940764 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4053236093 0 0) 0x7f1a800d07b0 con 0x7f1a8015c6c0 2024-09-12T03:19:59.150 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:59.145+0000 7f1a853cb640 1 -- 172.21.15.27:0/778940764 >> v1:172.21.15.27:6789/0 conn(0x7f1a8015c6c0 legacy=0x7f1a800d00a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:59.150 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:59.145+0000 7f1a853cb640 1 -- 172.21.15.27:0/778940764 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a800d0a80 con 0x7f1a800d1040 2024-09-12T03:19:59.150 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:59.145+0000 7f1a863cd640 1 --2- 172.21.15.27:0/778940764 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1a800b6f70 0x7f1a800b7400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:19:59.150 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:59.146+0000 7f1a853cb640 1 --2- 172.21.15.27:0/778940764 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1a800d1040 0x7f1a800d14b0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f1a700097a0 tx=0x7f1a7002a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94402d87b217e6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:59.151 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:59.146+0000 7f1a84bca640 1 -- 172.21.15.27:0/778940764 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a70037070 con 0x7f1a800d1040 2024-09-12T03:19:59.151 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:59.146+0000 7f1a84bca640 1 -- 172.21.15.27:0/778940764 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a7002ae50 con 0x7f1a800d1040 2024-09-12T03:19:59.151 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:59.146+0000 7f1a84bca640 1 -- 172.21.15.27:0/778940764 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a70032660 con 0x7f1a800d1040 2024-09-12T03:19:59.151 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:59.147+0000 7f1a87e30640 1 -- 172.21.15.27:0/778940764 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1a800d1040 msgr2=0x7f1a800d14b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:59.151 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:59.147+0000 7f1a87e30640 1 --2- 172.21.15.27:0/778940764 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1a800d1040 0x7f1a800d14b0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f1a700097a0 tx=0x7f1a7002a100 comp rx=0 tx=0).stop 2024-09-12T03:19:59.151 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:59.147+0000 7f1a87e30640 1 -- 172.21.15.27:0/778940764 shutdown_connections 2024-09-12T03:19:59.151 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:59.147+0000 7f1a87e30640 1 --2- 172.21.15.27:0/778940764 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1a800b6f70 0x7f1a800b7400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:59.151 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:59.147+0000 7f1a87e30640 1 --2- 172.21.15.27:0/778940764 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1a800d1040 0x7f1a800d14b0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:59.152 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:59.147+0000 7f1a87e30640 1 -- 172.21.15.27:0/778940764 >> 172.21.15.27:0/778940764 conn(0x7f1a8006f320 msgr2=0x7f1a8006f700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:59.152 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:59.147+0000 7f1a87e30640 1 -- 172.21.15.27:0/778940764 shutdown_connections 2024-09-12T03:19:59.152 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:59.147+0000 7f1a87e30640 1 -- 172.21.15.27:0/778940764 wait complete. 2024-09-12T03:19:59.154 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:59.148+0000 7f1a87e30640 1 Processor -- start 2024-09-12T03:19:59.154 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:59.148+0000 7f1a87e30640 1 -- start start 2024-09-12T03:19:59.154 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:59.149+0000 7f1a87e30640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1a800b6f70 0x7f1a80147470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:59.154 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:59.149+0000 7f1a87e30640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1a800d1040 0x7f1a801f5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:59.154 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:59.149+0000 7f1a87e30640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1a8015c6c0 0x7f1a801f56a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:59.154 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:59.149+0000 7f1a87e30640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f1a800b6a50 con 0x7f1a800b6f70 2024-09-12T03:19:59.155 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:59.149+0000 7f1a87e30640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f1a800b6770 con 0x7f1a8015c6c0 2024-09-12T03:19:59.155 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:59.149+0000 7f1a87e30640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f1a800d19f0 con 0x7f1a800d1040 2024-09-12T03:19:59.155 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:59.149+0000 7f1a85bcc640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1a800b6f70 0x7f1a80147470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:59.155 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:59.149+0000 7f1a85bcc640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1a800b6f70 0x7f1a80147470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35726/0 (socket says 172.21.15.27:35726) 2024-09-12T03:19:59.155 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:59.149+0000 7f1a85bcc640 1 -- 172.21.15.27:0/900726395 learned_addr learned my addr 172.21.15.27:0/900726395 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:19:59.155 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:59.149+0000 7f1a853cb640 1 --2- 172.21.15.27:0/900726395 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1a800d1040 0x7f1a801f5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:59.155 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:59.149+0000 7f1a863cd640 1 --2- 172.21.15.27:0/900726395 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1a8015c6c0 0x7f1a801f56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:59.155 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:59.149+0000 7f1a85bcc640 1 -- 172.21.15.27:0/900726395 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1a800d1040 msgr2=0x7f1a801f5160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:59.155 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:59.149+0000 7f1a85bcc640 1 --2- 172.21.15.27:0/900726395 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1a800d1040 0x7f1a801f5160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:59.155 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:59.149+0000 7f1a85bcc640 1 -- 172.21.15.27:0/900726395 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1a8015c6c0 msgr2=0x7f1a801f56a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:59.155 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:59.149+0000 7f1a85bcc640 1 --2- 172.21.15.27:0/900726395 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1a8015c6c0 0x7f1a801f56a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:59.155 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:59.149+0000 7f1a85bcc640 1 -- 172.21.15.27:0/900726395 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a801479b0 con 0x7f1a800b6f70 2024-09-12T03:19:59.156 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:59.149+0000 7f1a863cd640 1 --2- 172.21.15.27:0/900726395 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1a8015c6c0 0x7f1a801f56a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:19:59.156 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:59.150+0000 7f1a85bcc640 1 --2- 172.21.15.27:0/900726395 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1a800b6f70 0x7f1a80147470 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f1a7c00c430 tx=0x7f1a7c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:59.156 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:59.150+0000 7f1a767fc640 1 -- 172.21.15.27:0/900726395 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a7c0074e0 con 0x7f1a800b6f70 2024-09-12T03:19:59.156 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:59.150+0000 7f1a767fc640 1 -- 172.21.15.27:0/900726395 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a7c0047e0 con 0x7f1a800b6f70 2024-09-12T03:19:59.156 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:59.150+0000 7f1a767fc640 1 -- 172.21.15.27:0/900726395 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a7c004bd0 con 0x7f1a800b6f70 2024-09-12T03:19:59.156 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:59.150+0000 7f1a87e30640 1 -- 172.21.15.27:0/900726395 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a800c91c0 con 0x7f1a800b6f70 2024-09-12T03:19:59.156 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:59.150+0000 7f1a87e30640 1 -- 172.21.15.27:0/900726395 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a800c9780 con 0x7f1a800b6f70 2024-09-12T03:19:59.156 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:59.151+0000 7f1a87e30640 1 -- 172.21.15.27:0/900726395 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a800b4400 con 0x7f1a800b6f70 2024-09-12T03:19:59.156 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:59.151+0000 7f1a767fc640 1 -- 172.21.15.27:0/900726395 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 11) ==== 90804+0+0 (secure 0 0 0) 0x7f1a7c011070 con 0x7f1a800b6f70 2024-09-12T03:19:59.156 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:59.152+0000 7f1a767fc640 1 --2- 172.21.15.27:0/900726395 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f1a6406c190 0x7f1a6406e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:59.156 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:59.152+0000 7f1a767fc640 1 -- 172.21.15.27:0/900726395 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 5709+0+0 (secure 0 0 0) 0x7f1a7c08b110 con 0x7f1a800b6f70 2024-09-12T03:19:59.156 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:59.152+0000 7f1a853cb640 1 --2- 172.21.15.27:0/900726395 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f1a6406c190 0x7f1a6406e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:59.156 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:59.152+0000 7f1a853cb640 1 --2- 172.21.15.27:0/900726395 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f1a6406c190 0x7f1a6406e650 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f1a70034000 tx=0x7f1a7002e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:59.157 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:59.156+0000 7f1a767fc640 1 -- 172.21.15.27:0/900726395 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f1a7c0595b0 con 0x7f1a800b6f70 2024-09-12T03:19:59.368 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:59.367+0000 7f1a87e30640 1 -- 172.21.15.27:0/900726395 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f1a800c99c0 con 0x7f1a800b6f70 2024-09-12T03:19:59.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:59.391+0000 7f1a767fc640 1 -- 172.21.15.27:0/900726395 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v37) ==== 118+0+0 (secure 0 0 0) 0x7f1a7c058eb0 con 0x7f1a800b6f70 2024-09-12T03:19:59.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:59.393+0000 7f1a87e30640 1 -- 172.21.15.27:0/900726395 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f1a6406c190 msgr2=0x7f1a6406e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:59.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:59.393+0000 7f1a87e30640 1 --2- 172.21.15.27:0/900726395 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f1a6406c190 0x7f1a6406e650 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f1a70034000 tx=0x7f1a7002e000 comp rx=0 tx=0).stop 2024-09-12T03:19:59.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:59.393+0000 7f1a87e30640 1 -- 172.21.15.27:0/900726395 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1a800b6f70 msgr2=0x7f1a80147470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:59.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:59.393+0000 7f1a87e30640 1 --2- 172.21.15.27:0/900726395 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1a800b6f70 0x7f1a80147470 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f1a7c00c430 tx=0x7f1a7c00c8f0 comp rx=0 tx=0).stop 2024-09-12T03:19:59.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:59.393+0000 7f1a87e30640 1 -- 172.21.15.27:0/900726395 shutdown_connections 2024-09-12T03:19:59.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:59.393+0000 7f1a87e30640 1 --2- 172.21.15.27:0/900726395 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f1a6406c190 0x7f1a6406e650 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:59.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:59.393+0000 7f1a87e30640 1 --2- 172.21.15.27:0/900726395 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1a8015c6c0 0x7f1a801f56a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:59.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:59.393+0000 7f1a87e30640 1 --2- 172.21.15.27:0/900726395 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1a800d1040 0x7f1a801f5160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:59.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:59.393+0000 7f1a87e30640 1 --2- 172.21.15.27:0/900726395 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1a800b6f70 0x7f1a80147470 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:59.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:59.393+0000 7f1a87e30640 1 -- 172.21.15.27:0/900726395 >> 172.21.15.27:0/900726395 conn(0x7f1a8006f320 msgr2=0x7f1a8006f700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:59.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:59.393+0000 7f1a87e30640 1 -- 172.21.15.27:0/900726395 shutdown_connections 2024-09-12T03:19:59.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:59.393+0000 7f1a87e30640 1 -- 172.21.15.27:0/900726395 wait complete. 2024-09-12T03:19:59.406 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-12T03:19:59.406 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-12T03:19:59.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:59.511+0000 7f1b02217640 1 Processor -- start 2024-09-12T03:19:59.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:59.511+0000 7f1b02217640 1 -- start start 2024-09-12T03:19:59.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:59.512+0000 7f1b02217640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f1afc1645c0 0x7f1afc1669b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:59.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:59.512+0000 7f1b02217640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1afc0b4720 con 0x7f1afc0af9d0 2024-09-12T03:19:59.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:59.512+0000 7f1b02217640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1afc0ce5e0 con 0x7f1afc15ec30 2024-09-12T03:19:59.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:59.512+0000 7f1b02217640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f1afc0d02d0 con 0x7f1afc1645c0 2024-09-12T03:19:59.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:59.512+0000 7f1afaffd640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f1afc0af9d0 0x7f1afc0b4010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:34818/0 (socket says 172.21.15.27:34818) 2024-09-12T03:19:59.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:59.512+0000 7f1afaffd640 1 -- 172.21.15.27:0/1748604313 learned_addr learned my addr 172.21.15.27:0/1748604313 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:19:59.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:59.512+0000 7f1afbfff640 1 --2- 172.21.15.27:0/1748604313 >> v2:172.21.15.45:3300/0 conn(0x7f1afc1645c0 0x7f1afc1669b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:59.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:59.512+0000 7f1afa7fc640 1 -- 172.21.15.27:0/1748604313 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 234459795 0 0) 0x7f1afc0b4720 con 0x7f1afc0af9d0 2024-09-12T03:19:59.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:59.512+0000 7f1afa7fc640 1 -- 172.21.15.27:0/1748604313 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1ae4005730 con 0x7f1afc0af9d0 2024-09-12T03:19:59.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:59.513+0000 7f1afa7fc640 1 -- 172.21.15.27:0/1748604313 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3504338048 0 0) 0x7f1ae4005730 con 0x7f1afc0af9d0 2024-09-12T03:19:59.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:59.513+0000 7f1afbfff640 1 -- 172.21.15.27:0/1748604313 >> v1:172.21.15.45:6789/0 conn(0x7f1afc15ec30 legacy=0x7f1afc0b3900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:59.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:59.513+0000 7f1afbfff640 1 -- 172.21.15.27:0/1748604313 >> v1:172.21.15.27:6789/0 conn(0x7f1afc0af9d0 legacy=0x7f1afc0b4010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:59.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:59.513+0000 7f1afbfff640 1 -- 172.21.15.27:0/1748604313 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1afc0ceac0 con 0x7f1afc1645c0 2024-09-12T03:19:59.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:59.513+0000 7f1afbfff640 1 --2- 172.21.15.27:0/1748604313 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1afc1645c0 0x7f1afc1669b0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f1af4009d90 tx=0x7f1af402a680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c5971bf6710093ec server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:59.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:59.514+0000 7f1afa7fc640 1 -- 172.21.15.27:0/1748604313 <== mon.1 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1af0001670 con 0x7f1afc1645c0 2024-09-12T03:19:59.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:59.514+0000 7f1afa7fc640 1 -- 172.21.15.27:0/1748604313 <== mon.1 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1af4007540 con 0x7f1afc1645c0 2024-09-12T03:19:59.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:59.514+0000 7f1afa7fc640 1 -- 172.21.15.27:0/1748604313 <== mon.1 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1af40078f0 con 0x7f1afc1645c0 2024-09-12T03:19:59.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:59.514+0000 7f1b02217640 1 -- 172.21.15.27:0/1748604313 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1afc1645c0 msgr2=0x7f1afc1669b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:59.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:59.514+0000 7f1b02217640 1 --2- 172.21.15.27:0/1748604313 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1afc1645c0 0x7f1afc1669b0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f1af4009d90 tx=0x7f1af402a680 comp rx=0 tx=0).stop 2024-09-12T03:19:59.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:59.514+0000 7f1b02217640 1 -- 172.21.15.27:0/1748604313 shutdown_connections 2024-09-12T03:19:59.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:59.514+0000 7f1b02217640 1 --2- 172.21.15.27:0/1748604313 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1afc1645c0 0x7f1afc1669b0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:59.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:59.514+0000 7f1b02217640 1 -- 172.21.15.27:0/1748604313 >> 172.21.15.27:0/1748604313 conn(0x7f1afc06bb30 msgr2=0x7f1afc06bf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:59.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:59.514+0000 7f1b02217640 1 -- 172.21.15.27:0/1748604313 shutdown_connections 2024-09-12T03:19:59.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:59.514+0000 7f1b02217640 1 -- 172.21.15.27:0/1748604313 wait complete. 2024-09-12T03:19:59.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:59.515+0000 7f1b02217640 1 Processor -- start 2024-09-12T03:19:59.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:59.516+0000 7f1b02217640 1 -- start start 2024-09-12T03:19:59.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:59.516+0000 7f1b02217640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1afc0af9d0 0x7f1afc153ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:59.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:59.516+0000 7f1b02217640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1afc15ec30 0x7f1afc158ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:59.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:59.516+0000 7f1b02217640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1afc1645c0 0x7f1afc14a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:59.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:59.516+0000 7f1b02217640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f1afc0cf140 con 0x7f1afc0af9d0 2024-09-12T03:19:59.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:59.516+0000 7f1b02217640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f1afc0cefc0 con 0x7f1afc15ec30 2024-09-12T03:19:59.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:59.516+0000 7f1b02217640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f1afc0cf2c0 con 0x7f1afc1645c0 2024-09-12T03:19:59.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:59.516+0000 7f1afb7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1afc0af9d0 0x7f1afc153ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:59.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:59.517+0000 7f1afb7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1afc0af9d0 0x7f1afc153ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56124/0 (socket says 172.21.15.27:56124) 2024-09-12T03:19:59.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:59.517+0000 7f1afb7fe640 1 -- 172.21.15.27:0/57450499 learned_addr learned my addr 172.21.15.27:0/57450499 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:19:59.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:59.517+0000 7f1afaffd640 1 --2- 172.21.15.27:0/57450499 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1afc15ec30 0x7f1afc158ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:59.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:59.517+0000 7f1afb7fe640 1 -- 172.21.15.27:0/57450499 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1afc1645c0 msgr2=0x7f1afc14a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:59.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:59.517+0000 7f1afbfff640 1 --2- 172.21.15.27:0/57450499 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1afc1645c0 0x7f1afc14a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:59.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:59.517+0000 7f1afb7fe640 1 --2- 172.21.15.27:0/57450499 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1afc1645c0 0x7f1afc14a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:59.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:59.517+0000 7f1afb7fe640 1 -- 172.21.15.27:0/57450499 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1afc15ec30 msgr2=0x7f1afc158ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:59.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:59.517+0000 7f1afb7fe640 1 --2- 172.21.15.27:0/57450499 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1afc15ec30 0x7f1afc158ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:59.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:59.517+0000 7f1afb7fe640 1 -- 172.21.15.27:0/57450499 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1afc15c720 con 0x7f1afc0af9d0 2024-09-12T03:19:59.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:59.517+0000 7f1afb7fe640 1 --2- 172.21.15.27:0/57450499 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1afc0af9d0 0x7f1afc153ca0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f1aec00c450 tx=0x7f1aec00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:59.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:59.517+0000 7f1ae3fff640 1 -- 172.21.15.27:0/57450499 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aec0052d0 con 0x7f1afc0af9d0 2024-09-12T03:19:59.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:59.517+0000 7f1ae3fff640 1 -- 172.21.15.27:0/57450499 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1aec005470 con 0x7f1afc0af9d0 2024-09-12T03:19:59.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:59.517+0000 7f1b02217640 1 -- 172.21.15.27:0/57450499 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1afc15ca00 con 0x7f1afc0af9d0 2024-09-12T03:19:59.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:59.517+0000 7f1ae3fff640 1 -- 172.21.15.27:0/57450499 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aec012870 con 0x7f1afc0af9d0 2024-09-12T03:19:59.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:59.517+0000 7f1b02217640 1 -- 172.21.15.27:0/57450499 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1afc15cfc0 con 0x7f1afc0af9d0 2024-09-12T03:19:59.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:59.518+0000 7f1b02217640 1 -- 172.21.15.27:0/57450499 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1afc0695a0 con 0x7f1afc0af9d0 2024-09-12T03:19:59.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:59.519+0000 7f1ae3fff640 1 -- 172.21.15.27:0/57450499 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 11) ==== 90804+0+0 (secure 0 0 0) 0x7f1aec012a10 con 0x7f1afc0af9d0 2024-09-12T03:19:59.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:59.519+0000 7f1ae3fff640 1 --2- 172.21.15.27:0/57450499 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f1ae806c190 0x7f1ae806e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:59.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:59.520+0000 7f1ae3fff640 1 -- 172.21.15.27:0/57450499 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 5709+0+0 (secure 0 0 0) 0x7f1aec08b0d0 con 0x7f1afc0af9d0 2024-09-12T03:19:59.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:59.520+0000 7f1afaffd640 1 --2- 172.21.15.27:0/57450499 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f1ae806c190 0x7f1ae806e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:59.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:59.520+0000 7f1afaffd640 1 --2- 172.21.15.27:0/57450499 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f1ae806c190 0x7f1ae806e650 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f1afc068e40 tx=0x7f1af0009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:59.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:59.524+0000 7f1ae3fff640 1 -- 172.21.15.27:0/57450499 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f1aec059570 con 0x7f1afc0af9d0 2024-09-12T03:19:59.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:19:59.738+0000 7f1b02217640 1 -- 172.21.15.27:0/57450499 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f1afc15f010 con 0x7f1afc0af9d0 2024-09-12T03:20:00.369 INFO:tasks.ceph.mds.d.smithi045.stderr:2024-09-12T03:20:00.369+0000 7fba813ea640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-12T03:20:00.401 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:00.400+0000 7f1ae3fff640 1 -- 172.21.15.27:0/57450499 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v38) ==== 122+0+0 (secure 0 0 0) 0x7f1aec058e70 con 0x7f1afc0af9d0 2024-09-12T03:20:00.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:00.402+0000 7f1b02217640 1 -- 172.21.15.27:0/57450499 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f1ae806c190 msgr2=0x7f1ae806e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:00.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:00.402+0000 7f1b02217640 1 --2- 172.21.15.27:0/57450499 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f1ae806c190 0x7f1ae806e650 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f1afc068e40 tx=0x7f1af0009040 comp rx=0 tx=0).stop 2024-09-12T03:20:00.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:00.402+0000 7f1b02217640 1 -- 172.21.15.27:0/57450499 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1afc0af9d0 msgr2=0x7f1afc153ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:00.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:00.402+0000 7f1b02217640 1 --2- 172.21.15.27:0/57450499 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1afc0af9d0 0x7f1afc153ca0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f1aec00c450 tx=0x7f1aec00c910 comp rx=0 tx=0).stop 2024-09-12T03:20:00.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:00.402+0000 7f1b02217640 1 -- 172.21.15.27:0/57450499 shutdown_connections 2024-09-12T03:20:00.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:00.402+0000 7f1b02217640 1 --2- 172.21.15.27:0/57450499 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f1ae806c190 0x7f1ae806e650 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:00.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:00.402+0000 7f1b02217640 1 --2- 172.21.15.27:0/57450499 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1afc1645c0 0x7f1afc14a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:00.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:00.402+0000 7f1b02217640 1 --2- 172.21.15.27:0/57450499 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1afc15ec30 0x7f1afc158ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:00.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:00.402+0000 7f1b02217640 1 --2- 172.21.15.27:0/57450499 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1afc0af9d0 0x7f1afc153ca0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:00.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:00.402+0000 7f1b02217640 1 -- 172.21.15.27:0/57450499 >> 172.21.15.27:0/57450499 conn(0x7f1afc06bb30 msgr2=0x7f1afc152400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:00.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:00.402+0000 7f1b02217640 1 -- 172.21.15.27:0/57450499 shutdown_connections 2024-09-12T03:20:00.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:00.402+0000 7f1b02217640 1 -- 172.21.15.27:0/57450499 wait complete. 2024-09-12T03:20:00.416 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-12T03:20:00.416 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-12T03:20:00.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:00.524+0000 7fecc53cd640 1 Processor -- start 2024-09-12T03:20:00.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:00.524+0000 7fecc53cd640 1 -- start start 2024-09-12T03:20:00.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:00.525+0000 7fecc53cd640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fecc015a8c0 0x7fecc00cdb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:00.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:00.525+0000 7fecc53cd640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fecc015aca0 0x7fecc00b4710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:00.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:00.525+0000 7fecc53cd640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fecc006daf0 con 0x7fecc0071be0 2024-09-12T03:20:00.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:00.525+0000 7fecc53cd640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7fecc006d8a0 con 0x7fecc015aca0 2024-09-12T03:20:00.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:00.525+0000 7fecc53cd640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fecc006ddd0 con 0x7fecc015a8c0 2024-09-12T03:20:00.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:00.526+0000 7fecbf7fe640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fecc015aca0 0x7fecc00b4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:00.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:00.526+0000 7fecbe7fc640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fecc0071be0 0x7fecc006e8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:34832/0 (socket says 172.21.15.27:34832) 2024-09-12T03:20:00.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:00.526+0000 7fecbe7fc640 1 -- 172.21.15.27:0/1919096563 learned_addr learned my addr 172.21.15.27:0/1919096563 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:20:00.528 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:00.526+0000 7fecbeffd640 1 --2- 172.21.15.27:0/1919096563 >> v2:172.21.15.45:3300/0 conn(0x7fecc015a8c0 0x7fecc00cdb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:00.528 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:00.526+0000 7fecbf7fe640 1 -- 172.21.15.27:0/1919096563 >> v2:172.21.15.45:3300/0 conn(0x7fecc015a8c0 msgr2=0x7fecc00cdb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:00.528 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:00.526+0000 7fecbf7fe640 1 --2- 172.21.15.27:0/1919096563 >> v2:172.21.15.45:3300/0 conn(0x7fecc015a8c0 0x7fecc00cdb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:00.528 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:00.526+0000 7fecbf7fe640 1 -- 172.21.15.27:0/1919096563 >> v1:172.21.15.27:6789/0 conn(0x7fecc0071be0 legacy=0x7fecc006e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:00.528 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:00.526+0000 7fecbdffb640 1 -- 172.21.15.27:0/1919096563 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2383824515 0 0) 0x7fecc006daf0 con 0x7fecc0071be0 2024-09-12T03:20:00.528 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:00.526+0000 7fecbf7fe640 1 -- 172.21.15.27:0/1919096563 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecc006e0b0 con 0x7fecc015aca0 2024-09-12T03:20:00.529 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:00.527+0000 7fecbf7fe640 1 --2- 172.21.15.27:0/1919096563 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fecc015aca0 0x7fecc00b4710 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fecc006f470 tx=0x7fecb802a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e13f90aba36b4d42 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:00.529 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:00.527+0000 7fecbdffb640 1 -- 172.21.15.27:0/1919096563 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecb8007740 con 0x7fecc015aca0 2024-09-12T03:20:00.529 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:00.527+0000 7fecbdffb640 1 -- 172.21.15.27:0/1919096563 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fecb80078e0 con 0x7fecc015aca0 2024-09-12T03:20:00.529 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:00.527+0000 7fecbdffb640 1 -- 172.21.15.27:0/1919096563 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecb8007cb0 con 0x7fecc015aca0 2024-09-12T03:20:00.529 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:00.528+0000 7fecc53cd640 1 -- 172.21.15.27:0/1919096563 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fecc015aca0 msgr2=0x7fecc00b4710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:00.529 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:00.528+0000 7fecc53cd640 1 --2- 172.21.15.27:0/1919096563 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fecc015aca0 0x7fecc00b4710 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fecc006f470 tx=0x7fecb802a680 comp rx=0 tx=0).stop 2024-09-12T03:20:00.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:00.528+0000 7fecc53cd640 1 -- 172.21.15.27:0/1919096563 shutdown_connections 2024-09-12T03:20:00.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:00.528+0000 7fecc53cd640 1 --2- 172.21.15.27:0/1919096563 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fecc015aca0 0x7fecc00b4710 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:00.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:00.528+0000 7fecc53cd640 1 --2- 172.21.15.27:0/1919096563 >> v2:172.21.15.45:3300/0 conn(0x7fecc015a8c0 0x7fecc00cdb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:00.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:00.528+0000 7fecc53cd640 1 -- 172.21.15.27:0/1919096563 >> 172.21.15.27:0/1919096563 conn(0x7fecc0062c60 msgr2=0x7fecc006bb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:00.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:00.528+0000 7fecc53cd640 1 -- 172.21.15.27:0/1919096563 shutdown_connections 2024-09-12T03:20:00.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:00.528+0000 7fecc53cd640 1 -- 172.21.15.27:0/1919096563 wait complete. 2024-09-12T03:20:00.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:00.529+0000 7fecc53cd640 1 Processor -- start 2024-09-12T03:20:00.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:00.529+0000 7fecc53cd640 1 -- start start 2024-09-12T03:20:00.532 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:00.530+0000 7fecc53cd640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fecc0071be0 0x7fecc00c5d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:00.532 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:00.530+0000 7fecc53cd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fecc015a8c0 0x7fecc015e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:00.532 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:00.530+0000 7fecc53cd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fecc015aca0 0x7fecc015ea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:00.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:00.530+0000 7fecc53cd640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fecc006f140 con 0x7fecc015a8c0 2024-09-12T03:20:00.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:00.530+0000 7fecc53cd640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fecc006efc0 con 0x7fecc015aca0 2024-09-12T03:20:00.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:00.530+0000 7fecc53cd640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fecc00b3560 con 0x7fecc0071be0 2024-09-12T03:20:00.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:00.530+0000 7fecbe7fc640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fecc015a8c0 0x7fecc015e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:00.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:00.530+0000 7fecbe7fc640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fecc015a8c0 0x7fecc015e4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56152/0 (socket says 172.21.15.27:56152) 2024-09-12T03:20:00.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:00.530+0000 7fecbe7fc640 1 -- 172.21.15.27:0/543190231 learned_addr learned my addr 172.21.15.27:0/543190231 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:20:00.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:00.531+0000 7fecbeffd640 1 --2- 172.21.15.27:0/543190231 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fecc0071be0 0x7fecc00c5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:00.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:00.531+0000 7fecbf7fe640 1 --2- 172.21.15.27:0/543190231 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fecc015aca0 0x7fecc015ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:00.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:00.531+0000 7fecbe7fc640 1 -- 172.21.15.27:0/543190231 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fecc0071be0 msgr2=0x7fecc00c5d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:00.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:00.531+0000 7fecbe7fc640 1 --2- 172.21.15.27:0/543190231 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fecc0071be0 0x7fecc00c5d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:00.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:00.531+0000 7fecbe7fc640 1 -- 172.21.15.27:0/543190231 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fecc015aca0 msgr2=0x7fecc015ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:00.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:00.531+0000 7fecbe7fc640 1 --2- 172.21.15.27:0/543190231 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fecc015aca0 0x7fecc015ea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:00.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:00.531+0000 7fecbe7fc640 1 -- 172.21.15.27:0/543190231 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecc014bb90 con 0x7fecc015a8c0 2024-09-12T03:20:00.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:00.531+0000 7fecbe7fc640 1 --2- 172.21.15.27:0/543190231 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fecc015a8c0 0x7fecc015e4c0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7feca800c500 tx=0x7feca800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:00.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:00.531+0000 7fecb77fe640 1 -- 172.21.15.27:0/543190231 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feca8018070 con 0x7fecc015a8c0 2024-09-12T03:20:00.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:00.531+0000 7fecb77fe640 1 -- 172.21.15.27:0/543190231 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feca8007580 con 0x7fecc015a8c0 2024-09-12T03:20:00.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:00.531+0000 7fecc53cd640 1 -- 172.21.15.27:0/543190231 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fecc014b8b0 con 0x7fecc015a8c0 2024-09-12T03:20:00.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:00.531+0000 7fecb77fe640 1 -- 172.21.15.27:0/543190231 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feca8004a20 con 0x7fecc015a8c0 2024-09-12T03:20:00.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:00.531+0000 7fecc53cd640 1 -- 172.21.15.27:0/543190231 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fecc0148f90 con 0x7fecc015a8c0 2024-09-12T03:20:00.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:00.532+0000 7fecc53cd640 1 -- 172.21.15.27:0/543190231 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fecc0072ba0 con 0x7fecc015a8c0 2024-09-12T03:20:00.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:00.533+0000 7fecb77fe640 1 -- 172.21.15.27:0/543190231 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 11) ==== 90804+0+0 (secure 0 0 0) 0x7feca801b070 con 0x7fecc015a8c0 2024-09-12T03:20:00.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:00.533+0000 7fecb77fe640 1 --2- 172.21.15.27:0/543190231 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7feca006c100 0x7feca006e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:00.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:00.533+0000 7fecb77fe640 1 -- 172.21.15.27:0/543190231 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 5709+0+0 (secure 0 0 0) 0x7feca808a630 con 0x7fecc015a8c0 2024-09-12T03:20:00.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:00.534+0000 7fecbeffd640 1 --2- 172.21.15.27:0/543190231 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7feca006c100 0x7feca006e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:00.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:00.534+0000 7fecbeffd640 1 --2- 172.21.15.27:0/543190231 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7feca006c100 0x7feca006e5c0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fecb0009ad0 tx=0x7fecb0008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:00.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:00.538+0000 7fecb77fe640 1 -- 172.21.15.27:0/543190231 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7feca8058c40 con 0x7fecc015a8c0 2024-09-12T03:20:00.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:00.751+0000 7fecc53cd640 1 -- 172.21.15.27:0/543190231 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fecc0072850 con 0x7fecc015a8c0 2024-09-12T03:20:00.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:00.752+0000 7fecb77fe640 1 -- 172.21.15.27:0/543190231 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) ==== 94+0+5225 (secure 0 0 0) 0x7feca8058540 con 0x7fecc015a8c0 2024-09-12T03:20:00.753 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:20:00.753 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":38,"btime":"2024-09-12T03:20:00:397329+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6045,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6834/3058537826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":3058537826},{"type":"v1","addr":"172.21.15.27:6834","nonce":3058537826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":34},{"gid":6048,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/1585628536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":1585628536},{"type":"v1","addr":"172.21.15.27:6835","nonce":1585628536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":34},{"gid":6063,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6836/1421588365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":1421588365},{"type":"v1","addr":"172.21.15.45:6836","nonce":1421588365}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-12T03:19:57.738864+0000","modified":"2024-09-12T03:19:59.740115+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6066},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6066":{"gid":6066,"name":"d","rank":0,"incarnation":36,"state":"up:active","state_seq":3,"addr":"172.21.15.45:6837/2890007436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6835","nonce":2890007436},{"type":"v1","addr":"172.21.15.45:6837","nonce":2890007436}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6066,"qdb_cluster":[6066]},"id":5}]} 2024-09-12T03:20:00.753 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 38 2024-09-12T03:20:00.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:00.754+0000 7fecc53cd640 1 -- 172.21.15.27:0/543190231 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7feca006c100 msgr2=0x7feca006e5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:00.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:00.754+0000 7fecc53cd640 1 --2- 172.21.15.27:0/543190231 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7feca006c100 0x7feca006e5c0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fecb0009ad0 tx=0x7fecb0008040 comp rx=0 tx=0).stop 2024-09-12T03:20:00.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:00.754+0000 7fecc53cd640 1 -- 172.21.15.27:0/543190231 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fecc015a8c0 msgr2=0x7fecc015e4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:00.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:00.754+0000 7fecc53cd640 1 --2- 172.21.15.27:0/543190231 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fecc015a8c0 0x7fecc015e4c0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7feca800c500 tx=0x7feca800c9c0 comp rx=0 tx=0).stop 2024-09-12T03:20:00.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:00.754+0000 7fecc53cd640 1 -- 172.21.15.27:0/543190231 shutdown_connections 2024-09-12T03:20:00.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:00.754+0000 7fecc53cd640 1 --2- 172.21.15.27:0/543190231 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7feca006c100 0x7feca006e5c0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:00.756 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:00.754+0000 7fecc53cd640 1 --2- 172.21.15.27:0/543190231 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fecc015aca0 0x7fecc015ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:00.756 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:00.754+0000 7fecc53cd640 1 --2- 172.21.15.27:0/543190231 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fecc015a8c0 0x7fecc015e4c0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:00.756 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:00.754+0000 7fecc53cd640 1 --2- 172.21.15.27:0/543190231 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fecc0071be0 0x7fecc00c5d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:00.756 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:00.754+0000 7fecc53cd640 1 -- 172.21.15.27:0/543190231 >> 172.21.15.27:0/543190231 conn(0x7fecc0062c60 msgr2=0x7fecc00b4710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:00.756 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:00.754+0000 7fecc53cd640 1 -- 172.21.15.27:0/543190231 shutdown_connections 2024-09-12T03:20:00.756 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:00.754+0000 7fecc53cd640 1 -- 172.21.15.27:0/543190231 wait complete. 2024-09-12T03:20:00.766 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-12T03:20:00.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:00.869+0000 7f4b5daeb640 1 Processor -- start 2024-09-12T03:20:00.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:00.869+0000 7f4b5daeb640 1 -- start start 2024-09-12T03:20:00.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:00.870+0000 7f4b5daeb640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4b58071be0 0x7f4b5806e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:00.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:00.870+0000 7f4b5daeb640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f4b5815aca0 0x7f4b5806edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:00.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:00.870+0000 7f4b5daeb640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4b5806daf0 con 0x7f4b5815a8c0 2024-09-12T03:20:00.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:00.870+0000 7f4b5daeb640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f4b5806d8a0 con 0x7f4b5815aca0 2024-09-12T03:20:00.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:00.870+0000 7f4b5daeb640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f4b5806ddd0 con 0x7f4b58071be0 2024-09-12T03:20:00.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:00.870+0000 7f4b57fff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f4b5815aca0 0x7f4b5806edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:00.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:00.870+0000 7f4b57fff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f4b5815aca0 0x7f4b5806edf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56164/0 (socket says 172.21.15.27:56164) 2024-09-12T03:20:00.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:00.870+0000 7f4b57fff640 1 -- 172.21.15.27:0/1703729776 learned_addr learned my addr 172.21.15.27:0/1703729776 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:20:00.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:00.870+0000 7f4b56ffd640 1 --2- 172.21.15.27:0/1703729776 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4b58071be0 0x7f4b5806e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:00.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:00.870+0000 7f4b57fff640 1 -- 172.21.15.27:0/1703729776 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4b58071be0 msgr2=0x7f4b5806e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:00.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:00.870+0000 7f4b57fff640 1 --2- 172.21.15.27:0/1703729776 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4b58071be0 0x7f4b5806e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:00.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:00.870+0000 7f4b57fff640 1 -- 172.21.15.27:0/1703729776 >> v1:172.21.15.45:6789/0 conn(0x7f4b5815a8c0 legacy=0x7f4b580cdb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:00.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:00.870+0000 7f4b57fff640 1 -- 172.21.15.27:0/1703729776 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b5806e0b0 con 0x7f4b5815aca0 2024-09-12T03:20:00.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:00.870+0000 7f4b56ffd640 1 --2- 172.21.15.27:0/1703729776 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4b58071be0 0x7f4b5806e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:20:00.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:00.871+0000 7f4b57fff640 1 --2- 172.21.15.27:0/1703729776 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4b5815aca0 0x7f4b5806edf0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f4b50009e50 tx=0x7f4b5002a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9504b323fb0998ce server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:00.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:00.871+0000 7f4b567fc640 1 -- 172.21.15.27:0/1703729776 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b50036070 con 0x7f4b5815aca0 2024-09-12T03:20:00.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:00.871+0000 7f4b567fc640 1 -- 172.21.15.27:0/1703729776 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b50031600 con 0x7f4b5815aca0 2024-09-12T03:20:00.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:00.871+0000 7f4b567fc640 1 -- 172.21.15.27:0/1703729776 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b500319d0 con 0x7f4b5815aca0 2024-09-12T03:20:00.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:00.872+0000 7f4b5daeb640 1 -- 172.21.15.27:0/1703729776 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4b5815aca0 msgr2=0x7f4b5806edf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:00.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:00.872+0000 7f4b5daeb640 1 --2- 172.21.15.27:0/1703729776 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4b5815aca0 0x7f4b5806edf0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f4b50009e50 tx=0x7f4b5002a710 comp rx=0 tx=0).stop 2024-09-12T03:20:00.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:00.872+0000 7f4b5daeb640 1 -- 172.21.15.27:0/1703729776 shutdown_connections 2024-09-12T03:20:00.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:00.872+0000 7f4b5daeb640 1 --2- 172.21.15.27:0/1703729776 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4b5815aca0 0x7f4b5806edf0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:00.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:00.872+0000 7f4b5daeb640 1 --2- 172.21.15.27:0/1703729776 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4b58071be0 0x7f4b5806e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:00.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:00.872+0000 7f4b5daeb640 1 -- 172.21.15.27:0/1703729776 >> 172.21.15.27:0/1703729776 conn(0x7f4b58062c60 msgr2=0x7f4b5806bb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:00.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:00.872+0000 7f4b5daeb640 1 -- 172.21.15.27:0/1703729776 shutdown_connections 2024-09-12T03:20:00.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:00.872+0000 7f4b5daeb640 1 -- 172.21.15.27:0/1703729776 wait complete. 2024-09-12T03:20:00.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:00.873+0000 7f4b5daeb640 1 Processor -- start 2024-09-12T03:20:00.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:00.873+0000 7f4b5daeb640 1 -- start start 2024-09-12T03:20:00.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:00.874+0000 7f4b5daeb640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b58071be0 0x7f4b58147ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:00.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:00.874+0000 7f4b5daeb640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4b5815a8c0 0x7f4b5814b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:00.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:00.874+0000 7f4b5daeb640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4b5815aca0 0x7f4b58146060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:00.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:00.874+0000 7f4b5daeb640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f4b580ce3c0 con 0x7f4b5815aca0 2024-09-12T03:20:00.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:00.874+0000 7f4b5daeb640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f4b580ce240 con 0x7f4b58071be0 2024-09-12T03:20:00.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:00.874+0000 7f4b5daeb640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f4b580b3560 con 0x7f4b5815a8c0 2024-09-12T03:20:00.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:00.874+0000 7f4b57fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4b5815aca0 0x7f4b58146060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:00.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:00.874+0000 7f4b57fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4b5815aca0 0x7f4b58146060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56174/0 (socket says 172.21.15.27:56174) 2024-09-12T03:20:00.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:00.874+0000 7f4b57fff640 1 -- 172.21.15.27:0/3756544355 learned_addr learned my addr 172.21.15.27:0/3756544355 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:20:00.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:00.874+0000 7f4b577fe640 1 --2- 172.21.15.27:0/3756544355 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b58071be0 0x7f4b58147ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:00.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:00.874+0000 7f4b56ffd640 1 --2- 172.21.15.27:0/3756544355 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4b5815a8c0 0x7f4b5814b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:00.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:00.874+0000 7f4b57fff640 1 -- 172.21.15.27:0/3756544355 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4b5815a8c0 msgr2=0x7f4b5814b4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:00.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:00.874+0000 7f4b57fff640 1 --2- 172.21.15.27:0/3756544355 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4b5815a8c0 0x7f4b5814b4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:00.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:00.874+0000 7f4b57fff640 1 -- 172.21.15.27:0/3756544355 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b58071be0 msgr2=0x7f4b58147ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:00.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:00.874+0000 7f4b57fff640 1 --2- 172.21.15.27:0/3756544355 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b58071be0 0x7f4b58147ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:00.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:00.874+0000 7f4b57fff640 1 -- 172.21.15.27:0/3756544355 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b5814a730 con 0x7f4b5815aca0 2024-09-12T03:20:00.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:00.874+0000 7f4b57fff640 1 --2- 172.21.15.27:0/3756544355 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4b5815aca0 0x7f4b58146060 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f4b50009f70 tx=0x7f4b50004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:00.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:00.875+0000 7f4b3ffff640 1 -- 172.21.15.27:0/3756544355 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b50041020 con 0x7f4b5815aca0 2024-09-12T03:20:00.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:00.875+0000 7f4b3ffff640 1 -- 172.21.15.27:0/3756544355 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b50007780 con 0x7f4b5815aca0 2024-09-12T03:20:00.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:00.875+0000 7f4b5daeb640 1 -- 172.21.15.27:0/3756544355 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b5814aa10 con 0x7f4b5815aca0 2024-09-12T03:20:00.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:00.875+0000 7f4b3ffff640 1 -- 172.21.15.27:0/3756544355 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b50036040 con 0x7f4b5815aca0 2024-09-12T03:20:00.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:00.875+0000 7f4b5daeb640 1 -- 172.21.15.27:0/3756544355 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b58145860 con 0x7f4b5815aca0 2024-09-12T03:20:00.879 DEBUG:teuthology.orchestra.run.smithi027:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-12T03:20:00.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:00.876+0000 7f4b5daeb640 1 -- 172.21.15.27:0/3756544355 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b58072ba0 con 0x7f4b5815aca0 2024-09-12T03:20:00.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:00.876+0000 7f4b3ffff640 1 -- 172.21.15.27:0/3756544355 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 11) ==== 90804+0+0 (secure 0 0 0) 0x7f4b50044020 con 0x7f4b5815aca0 2024-09-12T03:20:00.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:00.877+0000 7f4b3ffff640 1 --2- 172.21.15.27:0/3756544355 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f4b4406c190 0x7f4b4406e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:00.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:00.877+0000 7f4b3ffff640 1 -- 172.21.15.27:0/3756544355 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 5709+0+0 (secure 0 0 0) 0x7f4b500abf90 con 0x7f4b5815aca0 2024-09-12T03:20:00.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:00.877+0000 7f4b577fe640 1 --2- 172.21.15.27:0/3756544355 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f4b4406c190 0x7f4b4406e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:00.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:00.878+0000 7f4b577fe640 1 --2- 172.21.15.27:0/3756544355 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f4b4406c190 0x7f4b4406e650 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f4b580ce0a0 tx=0x7f4b48009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:00.882 DEBUG:teuthology.orchestra.run.smithi045:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-12T03:20:00.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:00.881+0000 7f4b3ffff640 1 -- 172.21.15.27:0/3756544355 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f4b5004b030 con 0x7f4b5815aca0 2024-09-12T03:20:01.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.060+0000 7f4b5daeb640 1 -- 172.21.15.27:0/3756544355 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4b58075440 con 0x7f4b5815aca0 2024-09-12T03:20:01.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.061+0000 7f4b3ffff640 1 -- 172.21.15.27:0/3756544355 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v67) ==== 74+0+16680 (secure 0 0 0) 0x7f4b50079e60 con 0x7f4b5815aca0 2024-09-12T03:20:01.063 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:20:01.063 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":67,"fsid":"d2df504e-2647-4385-a5db-9fdf69f487a0","created":"2024-09-12T03:16:07.778937+0000","modified":"2024-09-12T03:19:59.392062+0000","last_up_change":"2024-09-12T03:16:19.314778+0000","last_in_change":"2024-09-12T03:16:09.893086+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T03:16:20.097646+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-09-12T03:19:55.777868+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-09-12T03:19:56.718653+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1851116c-1fb3-4dc9-9772-f90e2ef4c355","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6817","nonce":3002321488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6819","nonce":3002321488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6823","nonce":3002321488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6821","nonce":3002321488}]},"public_addr":"172.21.15.27:6817/3002321488","cluster_addr":"172.21.15.27:6819/3002321488","heartbeat_back_addr":"172.21.15.27:6823/3002321488","heartbeat_front_addr":"172.21.15.27:6821/3002321488","state":["exists","up"]},{"osd":1,"uuid":"f38ae17d-73eb-4204-b99d-975fa806fb87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6809","nonce":1613102886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6811","nonce":1613102886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6815","nonce":1613102886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6813","nonce":1613102886}]},"public_addr":"172.21.15.27:6809/1613102886","cluster_addr":"172.21.15.27:6811/1613102886","heartbeat_back_addr":"172.21.15.27:6815/1613102886","heartbeat_front_addr":"172.21.15.27:6813/1613102886","state":["exists","up"]},{"osd":2,"uuid":"fb12dcc8-bdd0-42b8-b46d-8b10106224bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6825","nonce":3534723355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6827","nonce":3534723355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6831","nonce":3534723355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6829","nonce":3534723355}]},"public_addr":"172.21.15.27:6825/3534723355","cluster_addr":"172.21.15.27:6827/3534723355","heartbeat_back_addr":"172.21.15.27:6831/3534723355","heartbeat_front_addr":"172.21.15.27:6829/3534723355","state":["exists","up"]},{"osd":3,"uuid":"d74d3f49-9c24-4b9f-98e2-3ecd7b5ef3b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6801","nonce":80661598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6803","nonce":80661598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6807","nonce":80661598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6805","nonce":80661598}]},"public_addr":"172.21.15.27:6801/80661598","cluster_addr":"172.21.15.27:6803/80661598","heartbeat_back_addr":"172.21.15.27:6807/80661598","heartbeat_front_addr":"172.21.15.27:6805/80661598","state":["exists","up"]},{"osd":4,"uuid":"2bf123f2-1b0d-41c4-aa3b-805bab267e59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6817","nonce":3943189827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6819","nonce":3943189827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6823","nonce":3943189827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6821","nonce":3943189827}]},"public_addr":"172.21.15.45:6817/3943189827","cluster_addr":"172.21.15.45:6819/3943189827","heartbeat_back_addr":"172.21.15.45:6823/3943189827","heartbeat_front_addr":"172.21.15.45:6821/3943189827","state":["exists","up"]},{"osd":5,"uuid":"c49ba5da-91c6-4a86-a97b-4aba89c253ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6809","nonce":3620760330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6811","nonce":3620760330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6815","nonce":3620760330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6813","nonce":3620760330}]},"public_addr":"172.21.15.45:6809/3620760330","cluster_addr":"172.21.15.45:6811/3620760330","heartbeat_back_addr":"172.21.15.45:6815/3620760330","heartbeat_front_addr":"172.21.15.45:6813/3620760330","state":["exists","up"]},{"osd":6,"uuid":"b0123475-89f7-4a34-9e33-d797b99dd9dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6801","nonce":1989083510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6803","nonce":1989083510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6807","nonce":1989083510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6805","nonce":1989083510}]},"public_addr":"172.21.15.45:6801/1989083510","cluster_addr":"172.21.15.45:6803/1989083510","heartbeat_back_addr":"172.21.15.45:6807/1989083510","heartbeat_front_addr":"172.21.15.45:6805/1989083510","state":["exists","up"]},{"osd":7,"uuid":"4dad773d-b7d8-46fc-8a10-a972aad27b29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6825","nonce":1513300294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6827","nonce":1513300294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6831","nonce":1513300294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6829","nonce":1513300294}]},"public_addr":"172.21.15.45:6825/1513300294","cluster_addr":"172.21.15.45:6827/1513300294","heartbeat_back_addr":"172.21.15.45:6831/1513300294","heartbeat_front_addr":"172.21.15.45:6829/1513300294","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.881855+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:17.133252+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.735339+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.500742+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.477571+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:17.167667+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.470291+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.873743+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T03:20:01.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.064+0000 7f4b5daeb640 1 -- 172.21.15.27:0/3756544355 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f4b4406c190 msgr2=0x7f4b4406e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:01.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.064+0000 7f4b5daeb640 1 --2- 172.21.15.27:0/3756544355 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f4b4406c190 0x7f4b4406e650 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f4b580ce0a0 tx=0x7f4b48009040 comp rx=0 tx=0).stop 2024-09-12T03:20:01.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.064+0000 7f4b5daeb640 1 -- 172.21.15.27:0/3756544355 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4b5815aca0 msgr2=0x7f4b58146060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:01.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.064+0000 7f4b5daeb640 1 --2- 172.21.15.27:0/3756544355 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4b5815aca0 0x7f4b58146060 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f4b50009f70 tx=0x7f4b50004a40 comp rx=0 tx=0).stop 2024-09-12T03:20:01.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.064+0000 7f4b5daeb640 1 -- 172.21.15.27:0/3756544355 shutdown_connections 2024-09-12T03:20:01.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.064+0000 7f4b5daeb640 1 --2- 172.21.15.27:0/3756544355 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f4b4406c190 0x7f4b4406e650 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:01.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.064+0000 7f4b5daeb640 1 --2- 172.21.15.27:0/3756544355 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4b5815aca0 0x7f4b58146060 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:01.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.064+0000 7f4b5daeb640 1 --2- 172.21.15.27:0/3756544355 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4b5815a8c0 0x7f4b5814b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:01.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.064+0000 7f4b5daeb640 1 --2- 172.21.15.27:0/3756544355 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b58071be0 0x7f4b58147ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:01.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.064+0000 7f4b5daeb640 1 -- 172.21.15.27:0/3756544355 >> 172.21.15.27:0/3756544355 conn(0x7f4b58062c60 msgr2=0x7f4b58153af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:01.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.064+0000 7f4b5daeb640 1 -- 172.21.15.27:0/3756544355 shutdown_connections 2024-09-12T03:20:01.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.064+0000 7f4b5daeb640 1 -- 172.21.15.27:0/3756544355 wait complete. 2024-09-12T03:20:01.076 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-12T03:20:01.076 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-12T03:20:01.179 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.178+0000 7f5555dc4640 1 Processor -- start 2024-09-12T03:20:01.179 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.178+0000 7f5555dc4640 1 -- start start 2024-09-12T03:20:01.180 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.179+0000 7f5555dc4640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f5550146010 0x7f55500b4750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:01.181 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.179+0000 7f5555dc4640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f55500b4d70 con 0x7f55500cda40 2024-09-12T03:20:01.181 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.179+0000 7f5555dc4640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f555006d140 con 0x7f5550145c30 2024-09-12T03:20:01.181 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.179+0000 7f5555dc4640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f55500b5050 con 0x7f5550146010 2024-09-12T03:20:01.181 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.179+0000 7f5547fff640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f55500cda40 0x7f55500b4040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:34840/0 (socket says 172.21.15.27:34840) 2024-09-12T03:20:01.181 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.179+0000 7f5547fff640 1 -- 172.21.15.27:0/3762251092 learned_addr learned my addr 172.21.15.27:0/3762251092 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:20:01.181 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.179+0000 7f554ffff640 1 --2- 172.21.15.27:0/3762251092 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f5550146010 0x7f55500b4750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:01.181 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.179+0000 7f554effd640 1 -- 172.21.15.27:0/3762251092 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 949974857 0 0) 0x7f55500b4d70 con 0x7f55500cda40 2024-09-12T03:20:01.181 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.179+0000 7f554effd640 1 -- 172.21.15.27:0/3762251092 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f553c003520 con 0x7f55500cda40 2024-09-12T03:20:01.181 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.179+0000 7f554effd640 1 -- 172.21.15.27:0/3762251092 <== mon.1 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1311896394 0 0) 0x7f555006d140 con 0x7f5550145c30 2024-09-12T03:20:01.181 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.179+0000 7f554effd640 1 -- 172.21.15.27:0/3762251092 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f55500b4d70 con 0x7f5550145c30 2024-09-12T03:20:01.181 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.179+0000 7f554ffff640 1 -- 172.21.15.27:0/3762251092 >> v1:172.21.15.45:6789/0 conn(0x7f5550145c30 legacy=0x7f55500cd330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:01.181 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.179+0000 7f554effd640 1 -- 172.21.15.27:0/3762251092 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2208264809 0 0) 0x7f553c003520 con 0x7f55500cda40 2024-09-12T03:20:01.181 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.179+0000 7f554ffff640 1 -- 172.21.15.27:0/3762251092 >> v1:172.21.15.27:6789/0 conn(0x7f55500cda40 legacy=0x7f55500b4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:01.182 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.179+0000 7f554ffff640 1 -- 172.21.15.27:0/3762251092 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f555006d420 con 0x7f5550146010 2024-09-12T03:20:01.183 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.180+0000 7f554ffff640 1 --2- 172.21.15.27:0/3762251092 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f5550146010 0x7f55500b4750 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f5548009dc0 tx=0x7f554802a5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=beac463ac5bf4ed4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:01.183 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.180+0000 7f554effd640 1 -- 172.21.15.27:0/3762251092 <== mon.2 v2:172.21.15.45:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5538001670 con 0x7f5550146010 2024-09-12T03:20:01.183 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.180+0000 7f554effd640 1 -- 172.21.15.27:0/3762251092 <== mon.2 v2:172.21.15.45:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5548007540 con 0x7f5550146010 2024-09-12T03:20:01.183 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.180+0000 7f554effd640 1 -- 172.21.15.27:0/3762251092 <== mon.2 v2:172.21.15.45:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55480078f0 con 0x7f5550146010 2024-09-12T03:20:01.183 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.181+0000 7f5555dc4640 1 -- 172.21.15.27:0/3762251092 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f5550146010 msgr2=0x7f55500b4750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:01.183 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.181+0000 7f5555dc4640 1 --2- 172.21.15.27:0/3762251092 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f5550146010 0x7f55500b4750 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f5548009dc0 tx=0x7f554802a5d0 comp rx=0 tx=0).stop 2024-09-12T03:20:01.183 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.181+0000 7f5555dc4640 1 -- 172.21.15.27:0/3762251092 shutdown_connections 2024-09-12T03:20:01.184 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.181+0000 7f5555dc4640 1 --2- 172.21.15.27:0/3762251092 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f5550146010 0x7f55500b4750 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:01.184 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.181+0000 7f5555dc4640 1 -- 172.21.15.27:0/3762251092 >> 172.21.15.27:0/3762251092 conn(0x7f5550062c60 msgr2=0x7f555006bb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:01.184 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.181+0000 7f5555dc4640 1 -- 172.21.15.27:0/3762251092 shutdown_connections 2024-09-12T03:20:01.184 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.181+0000 7f5555dc4640 1 -- 172.21.15.27:0/3762251092 wait complete. 2024-09-12T03:20:01.184 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.182+0000 7f5555dc4640 1 Processor -- start 2024-09-12T03:20:01.184 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.182+0000 7f5555dc4640 1 -- start start 2024-09-12T03:20:01.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.183+0000 7f5555dc4640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f55500cda40 0x7f555015bd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:01.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.183+0000 7f5555dc4640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5550145c30 0x7f5550137550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:01.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.183+0000 7f5555dc4640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f5550146010 0x7f555013fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:01.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.183+0000 7f5555dc4640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f555015cb80 con 0x7f5550145c30 2024-09-12T03:20:01.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.183+0000 7f5555dc4640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f555015ca00 con 0x7f55500cda40 2024-09-12T03:20:01.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.183+0000 7f5555dc4640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f555015cd00 con 0x7f5550146010 2024-09-12T03:20:01.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.183+0000 7f5547fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5550145c30 0x7f5550137550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:01.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.183+0000 7f5547fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5550145c30 0x7f5550137550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56184/0 (socket says 172.21.15.27:56184) 2024-09-12T03:20:01.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.183+0000 7f5547fff640 1 -- 172.21.15.27:0/1131411024 learned_addr learned my addr 172.21.15.27:0/1131411024 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:20:01.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.183+0000 7f554f7fe640 1 --2- 172.21.15.27:0/1131411024 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f55500cda40 0x7f555015bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:01.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.183+0000 7f554ffff640 1 --2- 172.21.15.27:0/1131411024 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f5550146010 0x7f555013fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:01.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.183+0000 7f5547fff640 1 -- 172.21.15.27:0/1131411024 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f5550146010 msgr2=0x7f555013fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:01.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.183+0000 7f5547fff640 1 --2- 172.21.15.27:0/1131411024 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f5550146010 0x7f555013fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:01.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.183+0000 7f5547fff640 1 -- 172.21.15.27:0/1131411024 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f55500cda40 msgr2=0x7f555015bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:01.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.183+0000 7f5547fff640 1 --2- 172.21.15.27:0/1131411024 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f55500cda40 0x7f555015bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:01.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.183+0000 7f5547fff640 1 -- 172.21.15.27:0/1131411024 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f555013b140 con 0x7f5550145c30 2024-09-12T03:20:01.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.183+0000 7f554ffff640 1 --2- 172.21.15.27:0/1131411024 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f5550146010 0x7f555013fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:20:01.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.184+0000 7f5547fff640 1 --2- 172.21.15.27:0/1131411024 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5550145c30 0x7f5550137550 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f553800c520 tx=0x7f553800c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:01.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.184+0000 7f554cff9640 1 -- 172.21.15.27:0/1131411024 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5538017070 con 0x7f5550145c30 2024-09-12T03:20:01.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.184+0000 7f554cff9640 1 -- 172.21.15.27:0/1131411024 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5538005ce0 con 0x7f5550145c30 2024-09-12T03:20:01.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.184+0000 7f5555dc4640 1 -- 172.21.15.27:0/1131411024 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f555015c490 con 0x7f5550145c30 2024-09-12T03:20:01.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.184+0000 7f554cff9640 1 -- 172.21.15.27:0/1131411024 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55380072b0 con 0x7f5550145c30 2024-09-12T03:20:01.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.184+0000 7f5555dc4640 1 -- 172.21.15.27:0/1131411024 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55501598c0 con 0x7f5550145c30 2024-09-12T03:20:01.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.185+0000 7f5555dc4640 1 -- 172.21.15.27:0/1131411024 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55500cd330 con 0x7f5550145c30 2024-09-12T03:20:01.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.185+0000 7f554cff9640 1 -- 172.21.15.27:0/1131411024 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 11) ==== 90804+0+0 (secure 0 0 0) 0x7f553801a070 con 0x7f5550145c30 2024-09-12T03:20:01.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.186+0000 7f554cff9640 1 --2- 172.21.15.27:0/1131411024 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f553006c190 0x7f553006e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:01.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.186+0000 7f554cff9640 1 -- 172.21.15.27:0/1131411024 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 5709+0+0 (secure 0 0 0) 0x7f553808a7b0 con 0x7f5550145c30 2024-09-12T03:20:01.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.186+0000 7f554f7fe640 1 --2- 172.21.15.27:0/1131411024 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f553006c190 0x7f553006e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:01.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.187+0000 7f554f7fe640 1 --2- 172.21.15.27:0/1131411024 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f553006c190 0x7f553006e650 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f55400011e0 tx=0x7f5540009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:01.191 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.190+0000 7f554cff9640 1 -- 172.21.15.27:0/1131411024 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f5538058de0 con 0x7f5550145c30 2024-09-12T03:20:01.363 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.362+0000 7f5555dc4640 1 -- 172.21.15.27:0/1131411024 --> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f555006e4e0 con 0x7f553006c190 2024-09-12T03:20:01.370 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.369+0000 7f554cff9640 1 -- 172.21.15.27:0/1131411024 <== mgr.4102 v2:172.21.15.45:6832/3923821219 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+248831 (secure 0 0 0) 0x7f555006e4e0 con 0x7f553006c190 2024-09-12T03:20:01.375 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:20:01.377 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2024-09-12T03:20:01.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.372+0000 7f5555dc4640 1 -- 172.21.15.27:0/1131411024 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f553006c190 msgr2=0x7f553006e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:01.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.372+0000 7f5555dc4640 1 --2- 172.21.15.27:0/1131411024 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f553006c190 0x7f553006e650 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f55400011e0 tx=0x7f5540009040 comp rx=0 tx=0).stop 2024-09-12T03:20:01.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.372+0000 7f5555dc4640 1 -- 172.21.15.27:0/1131411024 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5550145c30 msgr2=0x7f5550137550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:01.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.372+0000 7f5555dc4640 1 --2- 172.21.15.27:0/1131411024 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5550145c30 0x7f5550137550 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f553800c520 tx=0x7f553800c9e0 comp rx=0 tx=0).stop 2024-09-12T03:20:01.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.372+0000 7f5555dc4640 1 -- 172.21.15.27:0/1131411024 shutdown_connections 2024-09-12T03:20:01.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.372+0000 7f5555dc4640 1 --2- 172.21.15.27:0/1131411024 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f553006c190 0x7f553006e650 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:01.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.372+0000 7f5555dc4640 1 --2- 172.21.15.27:0/1131411024 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f5550146010 0x7f555013fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:01.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.372+0000 7f5555dc4640 1 --2- 172.21.15.27:0/1131411024 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5550145c30 0x7f5550137550 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:01.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.372+0000 7f5555dc4640 1 --2- 172.21.15.27:0/1131411024 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f55500cda40 0x7f555015bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:01.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.372+0000 7f5555dc4640 1 -- 172.21.15.27:0/1131411024 >> 172.21.15.27:0/1131411024 conn(0x7f5550062c60 msgr2=0x7f55500aa550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:01.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.372+0000 7f5555dc4640 1 -- 172.21.15.27:0/1131411024 shutdown_connections 2024-09-12T03:20:01.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.372+0000 7f5555dc4640 1 -- 172.21.15.27:0/1131411024 wait complete. 2024-09-12T03:20:01.384 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":170,"stamp":"2024-09-12T03:20:00.103531+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":591132,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":194,"num_write_kb":2420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":147,"ondisk_log_size":147,"up":154,"acting":154,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":100,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":6,"kb":754974720,"kb_used":252804,"kb_used_data":7404,"kb_used_omap":12,"kb_used_meta":245171,"kb_avail":754721916,"statfs":{"total":773094113280,"available":772835241984,"internally_reserved":0,"allocated":7581696,"data_stored":3828844,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":251055664},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":764,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":35,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001083"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.412295+0000","last_change":"2024-09-12T03:19:58.405738+0000","last_active":"2024-09-12T03:19:59.412295+0000","last_peered":"2024-09-12T03:19:59.412295+0000","last_clean":"2024-09-12T03:19:59.412295+0000","last_became_active":"2024-09-12T03:19:58.405044+0000","last_became_peered":"2024-09-12T03:19:58.405044+0000","last_unstale":"2024-09-12T03:19:59.412295+0000","last_undegraded":"2024-09-12T03:19:59.412295+0000","last_fullsized":"2024-09-12T03:19:59.412295+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:24:50.892593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.412848+0000","last_change":"2024-09-12T03:19:58.407057+0000","last_active":"2024-09-12T03:19:59.412848+0000","last_peered":"2024-09-12T03:19:59.412848+0000","last_clean":"2024-09-12T03:19:59.412848+0000","last_became_active":"2024-09-12T03:19:58.406455+0000","last_became_peered":"2024-09-12T03:19:58.406455+0000","last_unstale":"2024-09-12T03:19:59.412848+0000","last_undegraded":"2024-09-12T03:19:59.412848+0000","last_fullsized":"2024-09-12T03:19:59.412848+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:12:06.463949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.411816+0000","last_change":"2024-09-12T03:19:58.402689+0000","last_active":"2024-09-12T03:19:59.411816+0000","last_peered":"2024-09-12T03:19:59.411816+0000","last_clean":"2024-09-12T03:19:59.411816+0000","last_became_active":"2024-09-12T03:19:58.402424+0000","last_became_peered":"2024-09-12T03:19:58.402424+0000","last_unstale":"2024-09-12T03:19:59.411816+0000","last_undegraded":"2024-09-12T03:19:59.411816+0000","last_fullsized":"2024-09-12T03:19:59.411816+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:01:53.466856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:19:57.371227+0000","last_change":"2024-09-12T03:19:57.371227+0000","last_active":"2024-09-12T03:19:57.371227+0000","last_peered":"2024-09-12T03:19:57.371227+0000","last_clean":"2024-09-12T03:19:57.371227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:57.371227+0000","last_undegraded":"2024-09-12T03:19:57.371227+0000","last_fullsized":"2024-09-12T03:19:57.371227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.411395+0000","last_change":"2024-09-12T03:19:58.405012+0000","last_active":"2024-09-12T03:19:59.411395+0000","last_peered":"2024-09-12T03:19:59.411395+0000","last_clean":"2024-09-12T03:19:59.411395+0000","last_became_active":"2024-09-12T03:19:58.404706+0000","last_became_peered":"2024-09-12T03:19:58.404706+0000","last_unstale":"2024-09-12T03:19:59.411395+0000","last_undegraded":"2024-09-12T03:19:59.411395+0000","last_fullsized":"2024-09-12T03:19:59.411395+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:13:43.693628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.412119+0000","last_change":"2024-09-12T03:19:58.404152+0000","last_active":"2024-09-12T03:19:59.412119+0000","last_peered":"2024-09-12T03:19:59.412119+0000","last_clean":"2024-09-12T03:19:59.412119+0000","last_became_active":"2024-09-12T03:19:58.403865+0000","last_became_peered":"2024-09-12T03:19:58.403865+0000","last_unstale":"2024-09-12T03:19:59.412119+0000","last_undegraded":"2024-09-12T03:19:59.412119+0000","last_fullsized":"2024-09-12T03:19:59.412119+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:03:35.086339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:19:57.371227+0000","last_change":"2024-09-12T03:19:57.371227+0000","last_active":"2024-09-12T03:19:57.371227+0000","last_peered":"2024-09-12T03:19:57.371227+0000","last_clean":"2024-09-12T03:19:57.371227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:57.371227+0000","last_undegraded":"2024-09-12T03:19:57.371227+0000","last_fullsized":"2024-09-12T03:19:57.371227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.412001+0000","last_change":"2024-09-12T03:19:58.408692+0000","last_active":"2024-09-12T03:19:59.412001+0000","last_peered":"2024-09-12T03:19:59.412001+0000","last_clean":"2024-09-12T03:19:59.412001+0000","last_became_active":"2024-09-12T03:19:58.408182+0000","last_became_peered":"2024-09-12T03:19:58.408182+0000","last_unstale":"2024-09-12T03:19:59.412001+0000","last_undegraded":"2024-09-12T03:19:59.412001+0000","last_fullsized":"2024-09-12T03:19:59.412001+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:25:56.007148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.412647+0000","last_change":"2024-09-12T03:19:58.408054+0000","last_active":"2024-09-12T03:19:59.412647+0000","last_peered":"2024-09-12T03:19:59.412647+0000","last_clean":"2024-09-12T03:19:59.412647+0000","last_became_active":"2024-09-12T03:19:58.407439+0000","last_became_peered":"2024-09-12T03:19:58.407439+0000","last_unstale":"2024-09-12T03:19:59.412647+0000","last_undegraded":"2024-09-12T03:19:59.412647+0000","last_fullsized":"2024-09-12T03:19:59.412647+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:52:55.350315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.399858+0000","last_change":"2024-09-12T03:19:58.400878+0000","last_active":"2024-09-12T03:19:59.399858+0000","last_peered":"2024-09-12T03:19:59.399858+0000","last_clean":"2024-09-12T03:19:59.399858+0000","last_became_active":"2024-09-12T03:19:58.400601+0000","last_became_peered":"2024-09-12T03:19:58.400601+0000","last_unstale":"2024-09-12T03:19:59.399858+0000","last_undegraded":"2024-09-12T03:19:59.399858+0000","last_fullsized":"2024-09-12T03:19:59.399858+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:53:33.025355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.412388+0000","last_change":"2024-09-12T03:19:58.404349+0000","last_active":"2024-09-12T03:19:59.412388+0000","last_peered":"2024-09-12T03:19:59.412388+0000","last_clean":"2024-09-12T03:19:59.412388+0000","last_became_active":"2024-09-12T03:19:58.404067+0000","last_became_peered":"2024-09-12T03:19:58.404067+0000","last_unstale":"2024-09-12T03:19:59.412388+0000","last_undegraded":"2024-09-12T03:19:59.412388+0000","last_fullsized":"2024-09-12T03:19:59.412388+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:31:06.800562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:19:57.371227+0000","last_change":"2024-09-12T03:19:57.371227+0000","last_active":"2024-09-12T03:19:57.371227+0000","last_peered":"2024-09-12T03:19:57.371227+0000","last_clean":"2024-09-12T03:19:57.371227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:57.371227+0000","last_undegraded":"2024-09-12T03:19:57.371227+0000","last_fullsized":"2024-09-12T03:19:57.371227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.398467+0000","last_change":"2024-09-12T03:19:58.408147+0000","last_active":"2024-09-12T03:19:59.398467+0000","last_peered":"2024-09-12T03:19:59.398467+0000","last_clean":"2024-09-12T03:19:59.398467+0000","last_became_active":"2024-09-12T03:19:58.407855+0000","last_became_peered":"2024-09-12T03:19:58.407855+0000","last_unstale":"2024-09-12T03:19:59.398467+0000","last_undegraded":"2024-09-12T03:19:59.398467+0000","last_fullsized":"2024-09-12T03:19:59.398467+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:50:40.770697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.412570+0000","last_change":"2024-09-12T03:19:58.401979+0000","last_active":"2024-09-12T03:19:59.412570+0000","last_peered":"2024-09-12T03:19:59.412570+0000","last_clean":"2024-09-12T03:19:59.412570+0000","last_became_active":"2024-09-12T03:19:58.401689+0000","last_became_peered":"2024-09-12T03:19:58.401689+0000","last_unstale":"2024-09-12T03:19:59.412570+0000","last_undegraded":"2024-09-12T03:19:59.412570+0000","last_fullsized":"2024-09-12T03:19:59.412570+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:22:57.280328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.398306+0000","last_change":"2024-09-12T03:19:58.399312+0000","last_active":"2024-09-12T03:19:59.398306+0000","last_peered":"2024-09-12T03:19:59.398306+0000","last_clean":"2024-09-12T03:19:59.398306+0000","last_became_active":"2024-09-12T03:19:58.399050+0000","last_became_peered":"2024-09-12T03:19:58.399050+0000","last_unstale":"2024-09-12T03:19:59.398306+0000","last_undegraded":"2024-09-12T03:19:59.398306+0000","last_fullsized":"2024-09-12T03:19:59.398306+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:41:13.177460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.400875+0000","last_change":"2024-09-12T03:19:58.404580+0000","last_active":"2024-09-12T03:19:59.400875+0000","last_peered":"2024-09-12T03:19:59.400875+0000","last_clean":"2024-09-12T03:19:59.400875+0000","last_became_active":"2024-09-12T03:19:58.404320+0000","last_became_peered":"2024-09-12T03:19:58.404320+0000","last_unstale":"2024-09-12T03:19:59.400875+0000","last_undegraded":"2024-09-12T03:19:59.400875+0000","last_fullsized":"2024-09-12T03:19:59.400875+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:41:51.158105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.412705+0000","last_change":"2024-09-12T03:19:58.408389+0000","last_active":"2024-09-12T03:19:59.412705+0000","last_peered":"2024-09-12T03:19:59.412705+0000","last_clean":"2024-09-12T03:19:59.412705+0000","last_became_active":"2024-09-12T03:19:58.407797+0000","last_became_peered":"2024-09-12T03:19:58.407797+0000","last_unstale":"2024-09-12T03:19:59.412705+0000","last_undegraded":"2024-09-12T03:19:59.412705+0000","last_fullsized":"2024-09-12T03:19:59.412705+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:17:24.698351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:19:57.371227+0000","last_change":"2024-09-12T03:19:57.371227+0000","last_active":"2024-09-12T03:19:57.371227+0000","last_peered":"2024-09-12T03:19:57.371227+0000","last_clean":"2024-09-12T03:19:57.371227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:57.371227+0000","last_undegraded":"2024-09-12T03:19:57.371227+0000","last_fullsized":"2024-09-12T03:19:57.371227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.412173+0000","last_change":"2024-09-12T03:19:58.408115+0000","last_active":"2024-09-12T03:19:59.412173+0000","last_peered":"2024-09-12T03:19:59.412173+0000","last_clean":"2024-09-12T03:19:59.412173+0000","last_became_active":"2024-09-12T03:19:58.407745+0000","last_became_peered":"2024-09-12T03:19:58.407745+0000","last_unstale":"2024-09-12T03:19:59.412173+0000","last_undegraded":"2024-09-12T03:19:59.412173+0000","last_fullsized":"2024-09-12T03:19:59.412173+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:50:17.515387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.398554+0000","last_change":"2024-09-12T03:19:58.398467+0000","last_active":"2024-09-12T03:19:59.398554+0000","last_peered":"2024-09-12T03:19:59.398554+0000","last_clean":"2024-09-12T03:19:59.398554+0000","last_became_active":"2024-09-12T03:19:58.398152+0000","last_became_peered":"2024-09-12T03:19:58.398152+0000","last_unstale":"2024-09-12T03:19:59.398554+0000","last_undegraded":"2024-09-12T03:19:59.398554+0000","last_fullsized":"2024-09-12T03:19:59.398554+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:24:50.892593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.400458+0000","last_change":"2024-09-12T03:19:58.400541+0000","last_active":"2024-09-12T03:19:59.400458+0000","last_peered":"2024-09-12T03:19:59.400458+0000","last_clean":"2024-09-12T03:19:59.400458+0000","last_became_active":"2024-09-12T03:19:58.399602+0000","last_became_peered":"2024-09-12T03:19:58.399602+0000","last_unstale":"2024-09-12T03:19:59.400458+0000","last_undegraded":"2024-09-12T03:19:59.400458+0000","last_fullsized":"2024-09-12T03:19:59.400458+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:56:37.761014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.400370+0000","last_change":"2024-09-12T03:19:58.399427+0000","last_active":"2024-09-12T03:19:59.400370+0000","last_peered":"2024-09-12T03:19:59.400370+0000","last_clean":"2024-09-12T03:19:59.400370+0000","last_became_active":"2024-09-12T03:19:58.398464+0000","last_became_peered":"2024-09-12T03:19:58.398464+0000","last_unstale":"2024-09-12T03:19:59.400370+0000","last_undegraded":"2024-09-12T03:19:59.400370+0000","last_fullsized":"2024-09-12T03:19:59.400370+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:44:19.209045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.411588+0000","last_change":"2024-09-12T03:19:58.405456+0000","last_active":"2024-09-12T03:19:59.411588+0000","last_peered":"2024-09-12T03:19:59.411588+0000","last_clean":"2024-09-12T03:19:59.411588+0000","last_became_active":"2024-09-12T03:19:58.404974+0000","last_became_peered":"2024-09-12T03:19:58.404974+0000","last_unstale":"2024-09-12T03:19:59.411588+0000","last_undegraded":"2024-09-12T03:19:59.411588+0000","last_fullsized":"2024-09-12T03:19:59.411588+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:58:58.914180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.411477+0000","last_change":"2024-09-12T03:19:58.408893+0000","last_active":"2024-09-12T03:19:59.411477+0000","last_peered":"2024-09-12T03:19:59.411477+0000","last_clean":"2024-09-12T03:19:59.411477+0000","last_became_active":"2024-09-12T03:19:58.408589+0000","last_became_peered":"2024-09-12T03:19:58.408589+0000","last_unstale":"2024-09-12T03:19:59.411477+0000","last_undegraded":"2024-09-12T03:19:59.411477+0000","last_fullsized":"2024-09-12T03:19:59.411477+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:54:48.020501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.400037+0000","last_change":"2024-09-12T03:19:58.411515+0000","last_active":"2024-09-12T03:19:59.400037+0000","last_peered":"2024-09-12T03:19:59.400037+0000","last_clean":"2024-09-12T03:19:59.400037+0000","last_became_active":"2024-09-12T03:19:58.409091+0000","last_became_peered":"2024-09-12T03:19:58.409091+0000","last_unstale":"2024-09-12T03:19:59.400037+0000","last_undegraded":"2024-09-12T03:19:59.400037+0000","last_fullsized":"2024-09-12T03:19:59.400037+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:25:42.004151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:19:57.371227+0000","last_change":"2024-09-12T03:19:57.371227+0000","last_active":"2024-09-12T03:19:57.371227+0000","last_peered":"2024-09-12T03:19:57.371227+0000","last_clean":"2024-09-12T03:19:57.371227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:57.371227+0000","last_undegraded":"2024-09-12T03:19:57.371227+0000","last_fullsized":"2024-09-12T03:19:57.371227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.411966+0000","last_change":"2024-09-12T03:19:58.406104+0000","last_active":"2024-09-12T03:19:59.411966+0000","last_peered":"2024-09-12T03:19:59.411966+0000","last_clean":"2024-09-12T03:19:59.411966+0000","last_became_active":"2024-09-12T03:19:58.405546+0000","last_became_peered":"2024-09-12T03:19:58.405546+0000","last_unstale":"2024-09-12T03:19:59.411966+0000","last_undegraded":"2024-09-12T03:19:59.411966+0000","last_fullsized":"2024-09-12T03:19:59.411966+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:45:16.630721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.400401+0000","last_change":"2024-09-12T03:19:58.401764+0000","last_active":"2024-09-12T03:19:59.400401+0000","last_peered":"2024-09-12T03:19:59.400401+0000","last_clean":"2024-09-12T03:19:59.400401+0000","last_became_active":"2024-09-12T03:19:58.397972+0000","last_became_peered":"2024-09-12T03:19:58.397972+0000","last_unstale":"2024-09-12T03:19:59.400401+0000","last_undegraded":"2024-09-12T03:19:59.400401+0000","last_fullsized":"2024-09-12T03:19:59.400401+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:50:11.761046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.412120+0000","last_change":"2024-09-12T03:19:58.403549+0000","last_active":"2024-09-12T03:19:59.412120+0000","last_peered":"2024-09-12T03:19:59.412120+0000","last_clean":"2024-09-12T03:19:59.412120+0000","last_became_active":"2024-09-12T03:19:58.403294+0000","last_became_peered":"2024-09-12T03:19:58.403294+0000","last_unstale":"2024-09-12T03:19:59.412120+0000","last_undegraded":"2024-09-12T03:19:59.412120+0000","last_fullsized":"2024-09-12T03:19:59.412120+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:20:56.214083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:19:57.371227+0000","last_change":"2024-09-12T03:19:57.371227+0000","last_active":"2024-09-12T03:19:57.371227+0000","last_peered":"2024-09-12T03:19:57.371227+0000","last_clean":"2024-09-12T03:19:57.371227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:57.371227+0000","last_undegraded":"2024-09-12T03:19:57.371227+0000","last_fullsized":"2024-09-12T03:19:57.371227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:19:57.371227+0000","last_change":"2024-09-12T03:19:57.371227+0000","last_active":"2024-09-12T03:19:57.371227+0000","last_peered":"2024-09-12T03:19:57.371227+0000","last_clean":"2024-09-12T03:19:57.371227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:57.371227+0000","last_undegraded":"2024-09-12T03:19:57.371227+0000","last_fullsized":"2024-09-12T03:19:57.371227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.412055+0000","last_change":"2024-09-12T03:19:58.407063+0000","last_active":"2024-09-12T03:19:59.412055+0000","last_peered":"2024-09-12T03:19:59.412055+0000","last_clean":"2024-09-12T03:19:59.412055+0000","last_became_active":"2024-09-12T03:19:58.406116+0000","last_became_peered":"2024-09-12T03:19:58.406116+0000","last_unstale":"2024-09-12T03:19:59.412055+0000","last_undegraded":"2024-09-12T03:19:59.412055+0000","last_fullsized":"2024-09-12T03:19:59.412055+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:11:39.227764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.411730+0000","last_change":"2024-09-12T03:19:58.409587+0000","last_active":"2024-09-12T03:19:59.411730+0000","last_peered":"2024-09-12T03:19:59.411730+0000","last_clean":"2024-09-12T03:19:59.411730+0000","last_became_active":"2024-09-12T03:19:58.409275+0000","last_became_peered":"2024-09-12T03:19:58.409275+0000","last_unstale":"2024-09-12T03:19:59.411730+0000","last_undegraded":"2024-09-12T03:19:59.411730+0000","last_fullsized":"2024-09-12T03:19:59.411730+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:00:46.656402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"66'4","reported_seq":18,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.412596+0000","last_change":"2024-09-12T03:19:57.394595+0000","last_active":"2024-09-12T03:19:59.412596+0000","last_peered":"2024-09-12T03:19:59.412596+0000","last_clean":"2024-09-12T03:19:59.412596+0000","last_became_active":"2024-09-12T03:19:57.394294+0000","last_became_peered":"2024-09-12T03:19:57.394294+0000","last_unstale":"2024-09-12T03:19:59.412596+0000","last_undegraded":"2024-09-12T03:19:59.412596+0000","last_fullsized":"2024-09-12T03:19:59.412596+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_clean_scrub_stamp":"2024-09-12T03:19:56.366974+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:51:58.062647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.412771+0000","last_change":"2024-09-12T03:19:58.407115+0000","last_active":"2024-09-12T03:19:59.412771+0000","last_peered":"2024-09-12T03:19:59.412771+0000","last_clean":"2024-09-12T03:19:59.412771+0000","last_became_active":"2024-09-12T03:19:58.406683+0000","last_became_peered":"2024-09-12T03:19:58.406683+0000","last_unstale":"2024-09-12T03:19:59.412771+0000","last_undegraded":"2024-09-12T03:19:59.412771+0000","last_fullsized":"2024-09-12T03:19:59.412771+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:36:57.331135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"66'1","reported_seq":15,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.398595+0000","last_change":"2024-09-12T03:19:57.389982+0000","last_active":"2024-09-12T03:19:59.398595+0000","last_peered":"2024-09-12T03:19:59.398595+0000","last_clean":"2024-09-12T03:19:59.398595+0000","last_became_active":"2024-09-12T03:19:57.389694+0000","last_became_peered":"2024-09-12T03:19:57.389694+0000","last_unstale":"2024-09-12T03:19:59.398595+0000","last_undegraded":"2024-09-12T03:19:59.398595+0000","last_fullsized":"2024-09-12T03:19:59.398595+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_clean_scrub_stamp":"2024-09-12T03:19:56.366974+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:01:26.160193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.412792+0000","last_change":"2024-09-12T03:19:58.415175+0000","last_active":"2024-09-12T03:19:59.412792+0000","last_peered":"2024-09-12T03:19:59.412792+0000","last_clean":"2024-09-12T03:19:59.412792+0000","last_became_active":"2024-09-12T03:19:58.414899+0000","last_became_peered":"2024-09-12T03:19:58.414899+0000","last_unstale":"2024-09-12T03:19:59.412792+0000","last_undegraded":"2024-09-12T03:19:59.412792+0000","last_fullsized":"2024-09-12T03:19:59.412792+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:36:33.065448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"66'2","reported_seq":16,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.400437+0000","last_change":"2024-09-12T03:19:57.393946+0000","last_active":"2024-09-12T03:19:59.400437+0000","last_peered":"2024-09-12T03:19:59.400437+0000","last_clean":"2024-09-12T03:19:59.400437+0000","last_became_active":"2024-09-12T03:19:57.393672+0000","last_became_peered":"2024-09-12T03:19:57.393672+0000","last_unstale":"2024-09-12T03:19:59.400437+0000","last_undegraded":"2024-09-12T03:19:59.400437+0000","last_fullsized":"2024-09-12T03:19:59.400437+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_clean_scrub_stamp":"2024-09-12T03:19:56.366974+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:57:23.221949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.412495+0000","last_change":"2024-09-12T03:19:58.403124+0000","last_active":"2024-09-12T03:19:59.412495+0000","last_peered":"2024-09-12T03:19:59.412495+0000","last_clean":"2024-09-12T03:19:59.412495+0000","last_became_active":"2024-09-12T03:19:58.402859+0000","last_became_peered":"2024-09-12T03:19:58.402859+0000","last_unstale":"2024-09-12T03:19:59.412495+0000","last_undegraded":"2024-09-12T03:19:59.412495+0000","last_fullsized":"2024-09-12T03:19:59.412495+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:41:26.586794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"66'2","reported_seq":16,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.412896+0000","last_change":"2024-09-12T03:19:57.396585+0000","last_active":"2024-09-12T03:19:59.412896+0000","last_peered":"2024-09-12T03:19:59.412896+0000","last_clean":"2024-09-12T03:19:59.412896+0000","last_became_active":"2024-09-12T03:19:57.396109+0000","last_became_peered":"2024-09-12T03:19:57.396109+0000","last_unstale":"2024-09-12T03:19:59.412896+0000","last_undegraded":"2024-09-12T03:19:59.412896+0000","last_fullsized":"2024-09-12T03:19:59.412896+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_clean_scrub_stamp":"2024-09-12T03:19:56.366974+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:25:50.078358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:19:57.371227+0000","last_change":"2024-09-12T03:19:57.371227+0000","last_active":"2024-09-12T03:19:57.371227+0000","last_peered":"2024-09-12T03:19:57.371227+0000","last_clean":"2024-09-12T03:19:57.371227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:57.371227+0000","last_undegraded":"2024-09-12T03:19:57.371227+0000","last_fullsized":"2024-09-12T03:19:57.371227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1b","version":"66'1","reported_seq":15,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.412362+0000","last_change":"2024-09-12T03:19:57.391056+0000","last_active":"2024-09-12T03:19:59.412362+0000","last_peered":"2024-09-12T03:19:59.412362+0000","last_clean":"2024-09-12T03:19:59.412362+0000","last_became_active":"2024-09-12T03:19:57.390769+0000","last_became_peered":"2024-09-12T03:19:57.390769+0000","last_unstale":"2024-09-12T03:19:59.412362+0000","last_undegraded":"2024-09-12T03:19:59.412362+0000","last_fullsized":"2024-09-12T03:19:59.412362+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_clean_scrub_stamp":"2024-09-12T03:19:56.366974+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:41:12.173208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.400570+0000","last_change":"2024-09-12T03:19:58.415098+0000","last_active":"2024-09-12T03:19:59.400570+0000","last_peered":"2024-09-12T03:19:59.400570+0000","last_clean":"2024-09-12T03:19:59.400570+0000","last_became_active":"2024-09-12T03:19:58.411028+0000","last_became_peered":"2024-09-12T03:19:58.411028+0000","last_unstale":"2024-09-12T03:19:59.400570+0000","last_undegraded":"2024-09-12T03:19:59.400570+0000","last_fullsized":"2024-09-12T03:19:59.400570+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:29:23.148491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"66'1","reported_seq":15,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.412238+0000","last_change":"2024-09-12T03:19:57.395730+0000","last_active":"2024-09-12T03:19:59.412238+0000","last_peered":"2024-09-12T03:19:59.412238+0000","last_clean":"2024-09-12T03:19:59.412238+0000","last_became_active":"2024-09-12T03:19:57.395253+0000","last_became_peered":"2024-09-12T03:19:57.395253+0000","last_unstale":"2024-09-12T03:19:59.412238+0000","last_undegraded":"2024-09-12T03:19:59.412238+0000","last_fullsized":"2024-09-12T03:19:59.412238+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_clean_scrub_stamp":"2024-09-12T03:19:56.366974+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:54:51.158313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.411683+0000","last_change":"2024-09-12T03:19:58.400222+0000","last_active":"2024-09-12T03:19:59.411683+0000","last_peered":"2024-09-12T03:19:59.411683+0000","last_clean":"2024-09-12T03:19:59.411683+0000","last_became_active":"2024-09-12T03:19:58.399906+0000","last_became_peered":"2024-09-12T03:19:58.399906+0000","last_unstale":"2024-09-12T03:19:59.411683+0000","last_undegraded":"2024-09-12T03:19:59.411683+0000","last_fullsized":"2024-09-12T03:19:59.411683+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:17:22.493344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:19:56.366974+0000","last_change":"2024-09-12T03:19:56.366974+0000","last_active":"2024-09-12T03:19:56.366974+0000","last_peered":"2024-09-12T03:19:56.366974+0000","last_clean":"2024-09-12T03:19:56.366974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:56.366974+0000","last_undegraded":"2024-09-12T03:19:56.366974+0000","last_fullsized":"2024-09-12T03:19:56.366974+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_clean_scrub_stamp":"2024-09-12T03:19:56.366974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.398361+0000","last_change":"2024-09-12T03:19:58.405943+0000","last_active":"2024-09-12T03:19:59.398361+0000","last_peered":"2024-09-12T03:19:59.398361+0000","last_clean":"2024-09-12T03:19:59.398361+0000","last_became_active":"2024-09-12T03:19:58.405630+0000","last_became_peered":"2024-09-12T03:19:58.405630+0000","last_unstale":"2024-09-12T03:19:59.398361+0000","last_undegraded":"2024-09-12T03:19:59.398361+0000","last_fullsized":"2024-09-12T03:19:59.398361+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:00:46.656402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.411776+0000","last_change":"2024-09-12T03:19:57.394447+0000","last_active":"2024-09-12T03:19:59.411776+0000","last_peered":"2024-09-12T03:19:59.411776+0000","last_clean":"2024-09-12T03:19:59.411776+0000","last_became_active":"2024-09-12T03:19:57.394153+0000","last_became_peered":"2024-09-12T03:19:57.394153+0000","last_unstale":"2024-09-12T03:19:59.411776+0000","last_undegraded":"2024-09-12T03:19:59.411776+0000","last_fullsized":"2024-09-12T03:19:59.411776+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_clean_scrub_stamp":"2024-09-12T03:19:56.366974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:12:19.700994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:19:57.371227+0000","last_change":"2024-09-12T03:19:57.371227+0000","last_active":"2024-09-12T03:19:57.371227+0000","last_peered":"2024-09-12T03:19:57.371227+0000","last_clean":"2024-09-12T03:19:57.371227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:57.371227+0000","last_undegraded":"2024-09-12T03:19:57.371227+0000","last_fullsized":"2024-09-12T03:19:57.371227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.400155+0000","last_change":"2024-09-12T03:19:57.393730+0000","last_active":"2024-09-12T03:19:59.400155+0000","last_peered":"2024-09-12T03:19:59.400155+0000","last_clean":"2024-09-12T03:19:59.400155+0000","last_became_active":"2024-09-12T03:19:57.393425+0000","last_became_peered":"2024-09-12T03:19:57.393425+0000","last_unstale":"2024-09-12T03:19:59.400155+0000","last_undegraded":"2024-09-12T03:19:59.400155+0000","last_fullsized":"2024-09-12T03:19:59.400155+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_clean_scrub_stamp":"2024-09-12T03:19:56.366974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:39:01.426373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.412571+0000","last_change":"2024-09-12T03:19:58.409130+0000","last_active":"2024-09-12T03:19:59.412571+0000","last_peered":"2024-09-12T03:19:59.412571+0000","last_clean":"2024-09-12T03:19:59.412571+0000","last_became_active":"2024-09-12T03:19:58.408838+0000","last_became_peered":"2024-09-12T03:19:58.408838+0000","last_unstale":"2024-09-12T03:19:59.412571+0000","last_undegraded":"2024-09-12T03:19:59.412571+0000","last_fullsized":"2024-09-12T03:19:59.412571+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:38:31.191391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.412967+0000","last_change":"2024-09-12T03:19:57.391703+0000","last_active":"2024-09-12T03:19:59.412967+0000","last_peered":"2024-09-12T03:19:59.412967+0000","last_clean":"2024-09-12T03:19:59.412967+0000","last_became_active":"2024-09-12T03:19:57.391345+0000","last_became_peered":"2024-09-12T03:19:57.391345+0000","last_unstale":"2024-09-12T03:19:59.412967+0000","last_undegraded":"2024-09-12T03:19:59.412967+0000","last_fullsized":"2024-09-12T03:19:59.412967+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_clean_scrub_stamp":"2024-09-12T03:19:56.366974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:05:09.764085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.412786+0000","last_change":"2024-09-12T03:19:58.410704+0000","last_active":"2024-09-12T03:19:59.412786+0000","last_peered":"2024-09-12T03:19:59.412786+0000","last_clean":"2024-09-12T03:19:59.412786+0000","last_became_active":"2024-09-12T03:19:58.410390+0000","last_became_peered":"2024-09-12T03:19:58.410390+0000","last_unstale":"2024-09-12T03:19:59.412786+0000","last_undegraded":"2024-09-12T03:19:59.412786+0000","last_fullsized":"2024-09-12T03:19:59.412786+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:27:21.762076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"66'3","reported_seq":17,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.412074+0000","last_change":"2024-09-12T03:19:57.383973+0000","last_active":"2024-09-12T03:19:59.412074+0000","last_peered":"2024-09-12T03:19:59.412074+0000","last_clean":"2024-09-12T03:19:59.412074+0000","last_became_active":"2024-09-12T03:19:57.383481+0000","last_became_peered":"2024-09-12T03:19:57.383481+0000","last_unstale":"2024-09-12T03:19:59.412074+0000","last_undegraded":"2024-09-12T03:19:59.412074+0000","last_fullsized":"2024-09-12T03:19:59.412074+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_clean_scrub_stamp":"2024-09-12T03:19:56.366974+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:00:45.652150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.411674+0000","last_change":"2024-09-12T03:19:58.400811+0000","last_active":"2024-09-12T03:19:59.411674+0000","last_peered":"2024-09-12T03:19:59.411674+0000","last_clean":"2024-09-12T03:19:59.411674+0000","last_became_active":"2024-09-12T03:19:58.400542+0000","last_became_peered":"2024-09-12T03:19:58.400542+0000","last_unstale":"2024-09-12T03:19:59.411674+0000","last_undegraded":"2024-09-12T03:19:59.411674+0000","last_fullsized":"2024-09-12T03:19:59.411674+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:01:16.275564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:19:56.366974+0000","last_change":"2024-09-12T03:19:56.366974+0000","last_active":"2024-09-12T03:19:56.366974+0000","last_peered":"2024-09-12T03:19:56.366974+0000","last_clean":"2024-09-12T03:19:56.366974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:56.366974+0000","last_undegraded":"2024-09-12T03:19:56.366974+0000","last_fullsized":"2024-09-12T03:19:56.366974+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_clean_scrub_stamp":"2024-09-12T03:19:56.366974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.411683+0000","last_change":"2024-09-12T03:19:58.402659+0000","last_active":"2024-09-12T03:19:59.411683+0000","last_peered":"2024-09-12T03:19:59.411683+0000","last_clean":"2024-09-12T03:19:59.411683+0000","last_became_active":"2024-09-12T03:19:58.402358+0000","last_became_peered":"2024-09-12T03:19:58.402358+0000","last_unstale":"2024-09-12T03:19:59.411683+0000","last_undegraded":"2024-09-12T03:19:59.411683+0000","last_fullsized":"2024-09-12T03:19:59.411683+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:41:13.177460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:19:56.366974+0000","last_change":"2024-09-12T03:19:56.366974+0000","last_active":"2024-09-12T03:19:56.366974+0000","last_peered":"2024-09-12T03:19:56.366974+0000","last_clean":"2024-09-12T03:19:56.366974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:56.366974+0000","last_undegraded":"2024-09-12T03:19:56.366974+0000","last_fullsized":"2024-09-12T03:19:56.366974+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_clean_scrub_stamp":"2024-09-12T03:19:56.366974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:19:57.371227+0000","last_change":"2024-09-12T03:19:57.371227+0000","last_active":"2024-09-12T03:19:57.371227+0000","last_peered":"2024-09-12T03:19:57.371227+0000","last_clean":"2024-09-12T03:19:57.371227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:57.371227+0000","last_undegraded":"2024-09-12T03:19:57.371227+0000","last_fullsized":"2024-09-12T03:19:57.371227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.412349+0000","last_change":"2024-09-12T03:19:57.395906+0000","last_active":"2024-09-12T03:19:59.412349+0000","last_peered":"2024-09-12T03:19:59.412349+0000","last_clean":"2024-09-12T03:19:59.412349+0000","last_became_active":"2024-09-12T03:19:57.395558+0000","last_became_peered":"2024-09-12T03:19:57.395558+0000","last_unstale":"2024-09-12T03:19:59.412349+0000","last_undegraded":"2024-09-12T03:19:59.412349+0000","last_fullsized":"2024-09-12T03:19:59.412349+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_clean_scrub_stamp":"2024-09-12T03:19:56.366974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:23:25.739217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.398297+0000","last_change":"2024-09-12T03:19:58.407844+0000","last_active":"2024-09-12T03:19:59.398297+0000","last_peered":"2024-09-12T03:19:59.398297+0000","last_clean":"2024-09-12T03:19:59.398297+0000","last_became_active":"2024-09-12T03:19:58.407513+0000","last_became_peered":"2024-09-12T03:19:58.407513+0000","last_unstale":"2024-09-12T03:19:59.398297+0000","last_undegraded":"2024-09-12T03:19:59.398297+0000","last_fullsized":"2024-09-12T03:19:59.398297+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:58:58.914180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:19:56.366974+0000","last_change":"2024-09-12T03:19:56.366974+0000","last_active":"2024-09-12T03:19:56.366974+0000","last_peered":"2024-09-12T03:19:56.366974+0000","last_clean":"2024-09-12T03:19:56.366974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:56.366974+0000","last_undegraded":"2024-09-12T03:19:56.366974+0000","last_fullsized":"2024-09-12T03:19:56.366974+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_clean_scrub_stamp":"2024-09-12T03:19:56.366974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.412013+0000","last_change":"2024-09-12T03:19:58.406880+0000","last_active":"2024-09-12T03:19:59.412013+0000","last_peered":"2024-09-12T03:19:59.412013+0000","last_clean":"2024-09-12T03:19:59.412013+0000","last_became_active":"2024-09-12T03:19:58.406585+0000","last_became_peered":"2024-09-12T03:19:58.406585+0000","last_unstale":"2024-09-12T03:19:59.412013+0000","last_undegraded":"2024-09-12T03:19:59.412013+0000","last_fullsized":"2024-09-12T03:19:59.412013+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:44:18.879486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"66'8","reported_seq":22,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.400987+0000","last_change":"2024-09-12T03:19:57.397454+0000","last_active":"2024-09-12T03:19:59.400987+0000","last_peered":"2024-09-12T03:19:59.400987+0000","last_clean":"2024-09-12T03:19:59.400987+0000","last_became_active":"2024-09-12T03:19:57.397126+0000","last_became_peered":"2024-09-12T03:19:57.397126+0000","last_unstale":"2024-09-12T03:19:59.400987+0000","last_undegraded":"2024-09-12T03:19:59.400987+0000","last_fullsized":"2024-09-12T03:19:59.400987+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_clean_scrub_stamp":"2024-09-12T03:19:56.366974+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:14:13.101395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.400608+0000","last_change":"2024-09-12T03:19:58.402681+0000","last_active":"2024-09-12T03:19:59.400608+0000","last_peered":"2024-09-12T03:19:59.400608+0000","last_clean":"2024-09-12T03:19:59.400608+0000","last_became_active":"2024-09-12T03:19:58.402409+0000","last_became_peered":"2024-09-12T03:19:58.402409+0000","last_unstale":"2024-09-12T03:19:59.400608+0000","last_undegraded":"2024-09-12T03:19:59.400608+0000","last_fullsized":"2024-09-12T03:19:59.400608+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:56:05.472430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"66'1","reported_seq":15,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.412766+0000","last_change":"2024-09-12T03:19:57.394555+0000","last_active":"2024-09-12T03:19:59.412766+0000","last_peered":"2024-09-12T03:19:59.412766+0000","last_clean":"2024-09-12T03:19:59.412766+0000","last_became_active":"2024-09-12T03:19:57.394277+0000","last_became_peered":"2024-09-12T03:19:57.394277+0000","last_unstale":"2024-09-12T03:19:59.412766+0000","last_undegraded":"2024-09-12T03:19:59.412766+0000","last_fullsized":"2024-09-12T03:19:59.412766+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_clean_scrub_stamp":"2024-09-12T03:19:56.366974+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:50:52.123837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.412469+0000","last_change":"2024-09-12T03:19:58.411847+0000","last_active":"2024-09-12T03:19:59.412469+0000","last_peered":"2024-09-12T03:19:59.412469+0000","last_clean":"2024-09-12T03:19:59.412469+0000","last_became_active":"2024-09-12T03:19:58.409965+0000","last_became_peered":"2024-09-12T03:19:58.409965+0000","last_unstale":"2024-09-12T03:19:59.412469+0000","last_undegraded":"2024-09-12T03:19:59.412469+0000","last_fullsized":"2024-09-12T03:19:59.412469+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:45:22.536315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"66'2","reported_seq":16,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.400471+0000","last_change":"2024-09-12T03:19:57.394701+0000","last_active":"2024-09-12T03:19:59.400471+0000","last_peered":"2024-09-12T03:19:59.400471+0000","last_clean":"2024-09-12T03:19:59.400471+0000","last_became_active":"2024-09-12T03:19:57.394348+0000","last_became_peered":"2024-09-12T03:19:57.394348+0000","last_unstale":"2024-09-12T03:19:59.400471+0000","last_undegraded":"2024-09-12T03:19:59.400471+0000","last_fullsized":"2024-09-12T03:19:59.400471+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_clean_scrub_stamp":"2024-09-12T03:19:56.366974+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:44:36.098558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:19:57.371227+0000","last_change":"2024-09-12T03:19:57.371227+0000","last_active":"2024-09-12T03:19:57.371227+0000","last_peered":"2024-09-12T03:19:57.371227+0000","last_clean":"2024-09-12T03:19:57.371227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:57.371227+0000","last_undegraded":"2024-09-12T03:19:57.371227+0000","last_fullsized":"2024-09-12T03:19:57.371227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.d","version":"66'1","reported_seq":15,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.412373+0000","last_change":"2024-09-12T03:19:57.397975+0000","last_active":"2024-09-12T03:19:59.412373+0000","last_peered":"2024-09-12T03:19:59.412373+0000","last_clean":"2024-09-12T03:19:59.412373+0000","last_became_active":"2024-09-12T03:19:57.397637+0000","last_became_peered":"2024-09-12T03:19:57.397637+0000","last_unstale":"2024-09-12T03:19:59.412373+0000","last_undegraded":"2024-09-12T03:19:59.412373+0000","last_fullsized":"2024-09-12T03:19:59.412373+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_clean_scrub_stamp":"2024-09-12T03:19:56.366974+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:16:35.102685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.411820+0000","last_change":"2024-09-12T03:19:58.399501+0000","last_active":"2024-09-12T03:19:59.411820+0000","last_peered":"2024-09-12T03:19:59.411820+0000","last_clean":"2024-09-12T03:19:59.411820+0000","last_became_active":"2024-09-12T03:19:58.399203+0000","last_became_peered":"2024-09-12T03:19:58.399203+0000","last_unstale":"2024-09-12T03:19:59.411820+0000","last_undegraded":"2024-09-12T03:19:59.411820+0000","last_fullsized":"2024-09-12T03:19:59.411820+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:45:17.689061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.412050+0000","last_change":"2024-09-12T03:19:57.394026+0000","last_active":"2024-09-12T03:19:59.412050+0000","last_peered":"2024-09-12T03:19:59.412050+0000","last_clean":"2024-09-12T03:19:59.412050+0000","last_became_active":"2024-09-12T03:19:57.393679+0000","last_became_peered":"2024-09-12T03:19:57.393679+0000","last_unstale":"2024-09-12T03:19:59.412050+0000","last_undegraded":"2024-09-12T03:19:59.412050+0000","last_fullsized":"2024-09-12T03:19:59.412050+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_clean_scrub_stamp":"2024-09-12T03:19:56.366974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:41:50.153852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.412261+0000","last_change":"2024-09-12T03:19:58.407764+0000","last_active":"2024-09-12T03:19:59.412261+0000","last_peered":"2024-09-12T03:19:59.412261+0000","last_clean":"2024-09-12T03:19:59.412261+0000","last_became_active":"2024-09-12T03:19:58.407483+0000","last_became_peered":"2024-09-12T03:19:58.407483+0000","last_unstale":"2024-09-12T03:19:59.412261+0000","last_undegraded":"2024-09-12T03:19:59.412261+0000","last_fullsized":"2024-09-12T03:19:59.412261+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:11:19.244120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.412443+0000","last_change":"2024-09-12T03:19:57.394831+0000","last_active":"2024-09-12T03:19:59.412443+0000","last_peered":"2024-09-12T03:19:59.412443+0000","last_clean":"2024-09-12T03:19:59.412443+0000","last_became_active":"2024-09-12T03:19:57.394505+0000","last_became_peered":"2024-09-12T03:19:57.394505+0000","last_unstale":"2024-09-12T03:19:59.412443+0000","last_undegraded":"2024-09-12T03:19:59.412443+0000","last_fullsized":"2024-09-12T03:19:59.412443+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_clean_scrub_stamp":"2024-09-12T03:19:56.366974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:59:35.357929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.411253+0000","last_change":"2024-09-12T03:19:58.408696+0000","last_active":"2024-09-12T03:19:59.411253+0000","last_peered":"2024-09-12T03:19:59.411253+0000","last_clean":"2024-09-12T03:19:59.411253+0000","last_became_active":"2024-09-12T03:19:58.408393+0000","last_became_peered":"2024-09-12T03:19:58.408393+0000","last_unstale":"2024-09-12T03:19:59.411253+0000","last_undegraded":"2024-09-12T03:19:59.411253+0000","last_fullsized":"2024-09-12T03:19:59.411253+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:15:02.240922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:19:56.366974+0000","last_change":"2024-09-12T03:19:56.366974+0000","last_active":"2024-09-12T03:19:56.366974+0000","last_peered":"2024-09-12T03:19:56.366974+0000","last_clean":"2024-09-12T03:19:56.366974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:56.366974+0000","last_undegraded":"2024-09-12T03:19:56.366974+0000","last_fullsized":"2024-09-12T03:19:56.366974+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_clean_scrub_stamp":"2024-09-12T03:19:56.366974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.411564+0000","last_change":"2024-09-12T03:19:58.409403+0000","last_active":"2024-09-12T03:19:59.411564+0000","last_peered":"2024-09-12T03:19:59.411564+0000","last_clean":"2024-09-12T03:19:59.411564+0000","last_became_active":"2024-09-12T03:19:58.409087+0000","last_became_peered":"2024-09-12T03:19:58.409087+0000","last_unstale":"2024-09-12T03:19:59.411564+0000","last_undegraded":"2024-09-12T03:19:59.411564+0000","last_fullsized":"2024-09-12T03:19:59.411564+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:44:15.824316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.398237+0000","last_change":"2024-09-12T03:19:57.394055+0000","last_active":"2024-09-12T03:19:59.398237+0000","last_peered":"2024-09-12T03:19:59.398237+0000","last_clean":"2024-09-12T03:19:59.398237+0000","last_became_active":"2024-09-12T03:19:57.393767+0000","last_became_peered":"2024-09-12T03:19:57.393767+0000","last_unstale":"2024-09-12T03:19:59.398237+0000","last_undegraded":"2024-09-12T03:19:59.398237+0000","last_fullsized":"2024-09-12T03:19:59.398237+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_clean_scrub_stamp":"2024-09-12T03:19:56.366974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:34:00.504515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.398615+0000","last_change":"2024-09-12T03:19:58.400064+0000","last_active":"2024-09-12T03:19:59.398615+0000","last_peered":"2024-09-12T03:19:59.398615+0000","last_clean":"2024-09-12T03:19:59.398615+0000","last_became_active":"2024-09-12T03:19:58.399777+0000","last_became_peered":"2024-09-12T03:19:58.399777+0000","last_unstale":"2024-09-12T03:19:59.398615+0000","last_undegraded":"2024-09-12T03:19:59.398615+0000","last_fullsized":"2024-09-12T03:19:59.398615+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:31:06.800562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"66'2","reported_seq":16,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.412883+0000","last_change":"2024-09-12T03:19:57.393966+0000","last_active":"2024-09-12T03:19:59.412883+0000","last_peered":"2024-09-12T03:19:59.412883+0000","last_clean":"2024-09-12T03:19:59.412883+0000","last_became_active":"2024-09-12T03:19:57.393680+0000","last_became_peered":"2024-09-12T03:19:57.393680+0000","last_unstale":"2024-09-12T03:19:59.412883+0000","last_undegraded":"2024-09-12T03:19:59.412883+0000","last_fullsized":"2024-09-12T03:19:59.412883+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_clean_scrub_stamp":"2024-09-12T03:19:56.366974+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:05:00.306420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:19:57.371227+0000","last_change":"2024-09-12T03:19:57.371227+0000","last_active":"2024-09-12T03:19:57.371227+0000","last_peered":"2024-09-12T03:19:57.371227+0000","last_clean":"2024-09-12T03:19:57.371227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:57.371227+0000","last_undegraded":"2024-09-12T03:19:57.371227+0000","last_fullsized":"2024-09-12T03:19:57.371227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:19:56.366974+0000","last_change":"2024-09-12T03:19:56.366974+0000","last_active":"2024-09-12T03:19:56.366974+0000","last_peered":"2024-09-12T03:19:56.366974+0000","last_clean":"2024-09-12T03:19:56.366974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:56.366974+0000","last_undegraded":"2024-09-12T03:19:56.366974+0000","last_fullsized":"2024-09-12T03:19:56.366974+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_clean_scrub_stamp":"2024-09-12T03:19:56.366974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.411946+0000","last_change":"2024-09-12T03:19:58.408958+0000","last_active":"2024-09-12T03:19:59.411946+0000","last_peered":"2024-09-12T03:19:59.411946+0000","last_clean":"2024-09-12T03:19:59.411946+0000","last_became_active":"2024-09-12T03:19:58.408665+0000","last_became_peered":"2024-09-12T03:19:58.408665+0000","last_unstale":"2024-09-12T03:19:59.411946+0000","last_undegraded":"2024-09-12T03:19:59.411946+0000","last_fullsized":"2024-09-12T03:19:59.411946+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:50:40.770697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.411911+0000","last_change":"2024-09-12T03:19:57.393827+0000","last_active":"2024-09-12T03:19:59.411911+0000","last_peered":"2024-09-12T03:19:59.411911+0000","last_clean":"2024-09-12T03:19:59.411911+0000","last_became_active":"2024-09-12T03:19:57.393501+0000","last_became_peered":"2024-09-12T03:19:57.393501+0000","last_unstale":"2024-09-12T03:19:59.411911+0000","last_undegraded":"2024-09-12T03:19:59.411911+0000","last_fullsized":"2024-09-12T03:19:59.411911+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_clean_scrub_stamp":"2024-09-12T03:19:56.366974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:00:49.820829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.412577+0000","last_change":"2024-09-12T03:19:58.416479+0000","last_active":"2024-09-12T03:19:59.412577+0000","last_peered":"2024-09-12T03:19:59.412577+0000","last_clean":"2024-09-12T03:19:59.412577+0000","last_became_active":"2024-09-12T03:19:58.416145+0000","last_became_peered":"2024-09-12T03:19:58.416145+0000","last_unstale":"2024-09-12T03:19:59.412577+0000","last_undegraded":"2024-09-12T03:19:59.412577+0000","last_fullsized":"2024-09-12T03:19:59.412577+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:37:31.629037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.398503+0000","last_change":"2024-09-12T03:19:57.392451+0000","last_active":"2024-09-12T03:19:59.398503+0000","last_peered":"2024-09-12T03:19:59.398503+0000","last_clean":"2024-09-12T03:19:59.398503+0000","last_became_active":"2024-09-12T03:19:57.392127+0000","last_became_peered":"2024-09-12T03:19:57.392127+0000","last_unstale":"2024-09-12T03:19:59.398503+0000","last_undegraded":"2024-09-12T03:19:59.398503+0000","last_fullsized":"2024-09-12T03:19:59.398503+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_clean_scrub_stamp":"2024-09-12T03:19:56.366974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:11:02.225246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.412704+0000","last_change":"2024-09-12T03:19:58.407515+0000","last_active":"2024-09-12T03:19:59.412704+0000","last_peered":"2024-09-12T03:19:59.412704+0000","last_clean":"2024-09-12T03:19:59.412704+0000","last_became_active":"2024-09-12T03:19:58.407189+0000","last_became_peered":"2024-09-12T03:19:58.407189+0000","last_unstale":"2024-09-12T03:19:59.412704+0000","last_undegraded":"2024-09-12T03:19:59.412704+0000","last_fullsized":"2024-09-12T03:19:59.412704+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:57:01.617936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.411502+0000","last_change":"2024-09-12T03:19:57.391167+0000","last_active":"2024-09-12T03:19:59.411502+0000","last_peered":"2024-09-12T03:19:59.411502+0000","last_clean":"2024-09-12T03:19:59.411502+0000","last_became_active":"2024-09-12T03:19:57.390901+0000","last_became_peered":"2024-09-12T03:19:57.390901+0000","last_unstale":"2024-09-12T03:19:59.411502+0000","last_undegraded":"2024-09-12T03:19:59.411502+0000","last_fullsized":"2024-09-12T03:19:59.411502+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_clean_scrub_stamp":"2024-09-12T03:19:56.366974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:09:37.761223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.412332+0000","last_change":"2024-09-12T03:19:58.420294+0000","last_active":"2024-09-12T03:19:59.412332+0000","last_peered":"2024-09-12T03:19:59.412332+0000","last_clean":"2024-09-12T03:19:59.412332+0000","last_became_active":"2024-09-12T03:19:58.418912+0000","last_became_peered":"2024-09-12T03:19:58.418912+0000","last_unstale":"2024-09-12T03:19:59.412332+0000","last_undegraded":"2024-09-12T03:19:59.412332+0000","last_fullsized":"2024-09-12T03:19:59.412332+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:14:39.136845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:19:56.366974+0000","last_change":"2024-09-12T03:19:56.366974+0000","last_active":"2024-09-12T03:19:56.366974+0000","last_peered":"2024-09-12T03:19:56.366974+0000","last_clean":"2024-09-12T03:19:56.366974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:56.366974+0000","last_undegraded":"2024-09-12T03:19:56.366974+0000","last_fullsized":"2024-09-12T03:19:56.366974+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_clean_scrub_stamp":"2024-09-12T03:19:56.366974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:19:57.371227+0000","last_change":"2024-09-12T03:19:57.371227+0000","last_active":"2024-09-12T03:19:57.371227+0000","last_peered":"2024-09-12T03:19:57.371227+0000","last_clean":"2024-09-12T03:19:57.371227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:57.371227+0000","last_undegraded":"2024-09-12T03:19:57.371227+0000","last_fullsized":"2024-09-12T03:19:57.371227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.399794+0000","last_change":"2024-09-12T03:19:57.393150+0000","last_active":"2024-09-12T03:19:59.399794+0000","last_peered":"2024-09-12T03:19:59.399794+0000","last_clean":"2024-09-12T03:19:59.399794+0000","last_became_active":"2024-09-12T03:19:57.392692+0000","last_became_peered":"2024-09-12T03:19:57.392692+0000","last_unstale":"2024-09-12T03:19:59.399794+0000","last_undegraded":"2024-09-12T03:19:59.399794+0000","last_fullsized":"2024-09-12T03:19:59.399794+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_clean_scrub_stamp":"2024-09-12T03:19:56.366974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:44:27.441871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.411918+0000","last_change":"2024-09-12T03:19:58.407787+0000","last_active":"2024-09-12T03:19:59.411918+0000","last_peered":"2024-09-12T03:19:59.411918+0000","last_clean":"2024-09-12T03:19:59.411918+0000","last_became_active":"2024-09-12T03:19:58.407407+0000","last_became_peered":"2024-09-12T03:19:58.407407+0000","last_unstale":"2024-09-12T03:19:59.411918+0000","last_undegraded":"2024-09-12T03:19:59.411918+0000","last_fullsized":"2024-09-12T03:19:59.411918+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:38:14.768525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.412289+0000","last_change":"2024-09-12T03:19:57.391520+0000","last_active":"2024-09-12T03:19:59.412289+0000","last_peered":"2024-09-12T03:19:59.412289+0000","last_clean":"2024-09-12T03:19:59.412289+0000","last_became_active":"2024-09-12T03:19:57.391140+0000","last_became_peered":"2024-09-12T03:19:57.391140+0000","last_unstale":"2024-09-12T03:19:59.412289+0000","last_undegraded":"2024-09-12T03:19:59.412289+0000","last_fullsized":"2024-09-12T03:19:59.412289+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_clean_scrub_stamp":"2024-09-12T03:19:56.366974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:42:32.285383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.412286+0000","last_change":"2024-09-12T03:19:58.409818+0000","last_active":"2024-09-12T03:19:59.412286+0000","last_peered":"2024-09-12T03:19:59.412286+0000","last_clean":"2024-09-12T03:19:59.412286+0000","last_became_active":"2024-09-12T03:19:58.409507+0000","last_became_peered":"2024-09-12T03:19:58.409507+0000","last_unstale":"2024-09-12T03:19:59.412286+0000","last_undegraded":"2024-09-12T03:19:59.412286+0000","last_fullsized":"2024-09-12T03:19:59.412286+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:21:24.731904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"66'2","reported_seq":16,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.412444+0000","last_change":"2024-09-12T03:19:57.395758+0000","last_active":"2024-09-12T03:19:59.412444+0000","last_peered":"2024-09-12T03:19:59.412444+0000","last_clean":"2024-09-12T03:19:59.412444+0000","last_became_active":"2024-09-12T03:19:57.395422+0000","last_became_peered":"2024-09-12T03:19:57.395422+0000","last_unstale":"2024-09-12T03:19:59.412444+0000","last_undegraded":"2024-09-12T03:19:59.412444+0000","last_fullsized":"2024-09-12T03:19:59.412444+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_clean_scrub_stamp":"2024-09-12T03:19:56.366974+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:18:13.412357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":252,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.411416+0000","last_change":"2024-09-12T03:16:21.883446+0000","last_active":"2024-09-12T03:19:59.411416+0000","last_peered":"2024-09-12T03:19:59.411416+0000","last_clean":"2024-09-12T03:19:59.411416+0000","last_became_active":"2024-09-12T03:16:21.882207+0000","last_became_peered":"2024-09-12T03:16:21.882207+0000","last_unstale":"2024-09-12T03:19:59.411416+0000","last_undegraded":"2024-09-12T03:19:59.411416+0000","last_fullsized":"2024-09-12T03:19:59.411416+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:20.316426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:20.316426+0000","last_clean_scrub_stamp":"2024-09-12T03:16:20.316426+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:12:15.424320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":102,"acting":102,"num_store_stats":6},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":764,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":35,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":30,"ondisk_log_size":30,"up":50,"acting":50,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542190,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36716,"kb_used_data":1368,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335124,"statfs":{"total":96636764160,"available":96599166976,"internally_reserved":0,"allocated":1400832,"data_stored":924039,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542192,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31996,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339844,"statfs":{"total":96636764160,"available":96604000256,"internally_reserved":0,"allocated":819200,"data_stored":339447,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542190,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":36104,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335736,"statfs":{"total":96636764160,"available":96599793664,"internally_reserved":0,"allocated":761856,"data_stored":314188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542190,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31980,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339860,"statfs":{"total":96636764160,"available":96604016640,"internally_reserved":0,"allocated":802816,"data_stored":333125,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542191,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27836,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344004,"statfs":{"total":96636764160,"available":96608260096,"internally_reserved":0,"allocated":819200,"data_stored":339467,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542191,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27840,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344000,"statfs":{"total":96636764160,"available":96608256000,"internally_reserved":0,"allocated":823296,"data_stored":340967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542191,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31936,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339904,"statfs":{"total":96636764160,"available":96604061696,"internally_reserved":0,"allocated":761856,"data_stored":314188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542192,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28396,"kb_used_data":1360,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343444,"statfs":{"total":96636764160,"available":96607686656,"internally_reserved":0,"allocated":1392640,"data_stored":923423,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T03:20:01.389 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-12T03:20:01.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.526+0000 7fa3d0752640 1 Processor -- start 2024-09-12T03:20:01.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.526+0000 7fa3d0752640 1 -- start start 2024-09-12T03:20:01.528 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.527+0000 7fa3d0752640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa3c815a8c0 0x7fa3c80cdb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:01.529 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.527+0000 7fa3d0752640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fa3c815aca0 0x7fa3c80b4710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:01.529 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.527+0000 7fa3d0752640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa3c806daf0 con 0x7fa3c8071be0 2024-09-12T03:20:01.529 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.527+0000 7fa3d0752640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7fa3c806d8a0 con 0x7fa3c815aca0 2024-09-12T03:20:01.529 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.527+0000 7fa3d0752640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fa3c806ddd0 con 0x7fa3c815a8c0 2024-09-12T03:20:01.529 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.527+0000 7fa3cecef640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fa3c815aca0 0x7fa3c80b4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:01.529 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.527+0000 7fa3cecef640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fa3c815aca0 0x7fa3c80b4710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56194/0 (socket says 172.21.15.27:56194) 2024-09-12T03:20:01.529 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.527+0000 7fa3cecef640 1 -- 172.21.15.27:0/2529180380 learned_addr learned my addr 172.21.15.27:0/2529180380 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:20:01.529 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.527+0000 7fa3ce4ee640 1 --2- 172.21.15.27:0/2529180380 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa3c815a8c0 0x7fa3c80cdb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:01.529 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.527+0000 7fa3cecef640 1 -- 172.21.15.27:0/2529180380 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa3c815a8c0 msgr2=0x7fa3c80cdb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:01.529 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.527+0000 7fa3cecef640 1 --2- 172.21.15.27:0/2529180380 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa3c815a8c0 0x7fa3c80cdb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:01.529 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.527+0000 7fa3cecef640 1 -- 172.21.15.27:0/2529180380 >> v1:172.21.15.45:6789/0 conn(0x7fa3c8071be0 legacy=0x7fa3c806e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:01.529 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.527+0000 7fa3cecef640 1 -- 172.21.15.27:0/2529180380 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3c806e0b0 con 0x7fa3c815aca0 2024-09-12T03:20:01.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.528+0000 7fa3cecef640 1 --2- 172.21.15.27:0/2529180380 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa3c815aca0 0x7fa3c80b4710 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fa3c000af60 tx=0x7fa3c002c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=249de85c638ce773 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:01.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.528+0000 7fa3cd4ec640 1 -- 172.21.15.27:0/2529180380 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3c0008500 con 0x7fa3c815aca0 2024-09-12T03:20:01.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.528+0000 7fa3cd4ec640 1 -- 172.21.15.27:0/2529180380 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3c0037070 con 0x7fa3c815aca0 2024-09-12T03:20:01.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.528+0000 7fa3cd4ec640 1 -- 172.21.15.27:0/2529180380 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3c00088c0 con 0x7fa3c815aca0 2024-09-12T03:20:01.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.529+0000 7fa3d0752640 1 -- 172.21.15.27:0/2529180380 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa3c815aca0 msgr2=0x7fa3c80b4710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:01.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.529+0000 7fa3d0752640 1 --2- 172.21.15.27:0/2529180380 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa3c815aca0 0x7fa3c80b4710 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fa3c000af60 tx=0x7fa3c002c920 comp rx=0 tx=0).stop 2024-09-12T03:20:01.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.529+0000 7fa3d0752640 1 -- 172.21.15.27:0/2529180380 shutdown_connections 2024-09-12T03:20:01.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.529+0000 7fa3d0752640 1 --2- 172.21.15.27:0/2529180380 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa3c815aca0 0x7fa3c80b4710 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:01.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.529+0000 7fa3d0752640 1 --2- 172.21.15.27:0/2529180380 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa3c815a8c0 0x7fa3c80cdb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:01.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.529+0000 7fa3d0752640 1 -- 172.21.15.27:0/2529180380 >> 172.21.15.27:0/2529180380 conn(0x7fa3c8062c60 msgr2=0x7fa3c806bb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:01.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.529+0000 7fa3d0752640 1 -- 172.21.15.27:0/2529180380 shutdown_connections 2024-09-12T03:20:01.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.529+0000 7fa3d0752640 1 -- 172.21.15.27:0/2529180380 wait complete. 2024-09-12T03:20:01.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.530+0000 7fa3d0752640 1 Processor -- start 2024-09-12T03:20:01.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.530+0000 7fa3d0752640 1 -- start start 2024-09-12T03:20:01.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.531+0000 7fa3d0752640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa3c8071be0 0x7fa3c81f4b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:01.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.531+0000 7fa3d0752640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa3c815a8c0 0x7fa3c814f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:01.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.531+0000 7fa3d0752640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa3c815aca0 0x7fa3c80c6aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:01.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.531+0000 7fa3d0752640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fa3c806f140 con 0x7fa3c815aca0 2024-09-12T03:20:01.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.531+0000 7fa3d0752640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa3c806efc0 con 0x7fa3c815a8c0 2024-09-12T03:20:01.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.531+0000 7fa3d0752640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fa3c80b3560 con 0x7fa3c8071be0 2024-09-12T03:20:01.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.531+0000 7fa3cecef640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa3c815aca0 0x7fa3c80c6aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:01.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.531+0000 7fa3cecef640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa3c815aca0 0x7fa3c80c6aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56200/0 (socket says 172.21.15.27:56200) 2024-09-12T03:20:01.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.531+0000 7fa3cecef640 1 -- 172.21.15.27:0/1352956193 learned_addr learned my addr 172.21.15.27:0/1352956193 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:20:01.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.531+0000 7fa3ce4ee640 1 --2- 172.21.15.27:0/1352956193 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa3c8071be0 0x7fa3c81f4b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:01.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.531+0000 7fa3cdced640 1 --2- 172.21.15.27:0/1352956193 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa3c815a8c0 0x7fa3c814f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:01.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.531+0000 7fa3cecef640 1 -- 172.21.15.27:0/1352956193 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa3c8071be0 msgr2=0x7fa3c81f4b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:01.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.531+0000 7fa3cecef640 1 --2- 172.21.15.27:0/1352956193 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa3c8071be0 0x7fa3c81f4b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:01.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.531+0000 7fa3cecef640 1 -- 172.21.15.27:0/1352956193 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa3c815a8c0 msgr2=0x7fa3c814f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:01.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.531+0000 7fa3cecef640 1 --2- 172.21.15.27:0/1352956193 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa3c815a8c0 0x7fa3c814f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:01.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.531+0000 7fa3cecef640 1 -- 172.21.15.27:0/1352956193 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3c81f50d0 con 0x7fa3c815aca0 2024-09-12T03:20:01.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.531+0000 7fa3cecef640 1 --2- 172.21.15.27:0/1352956193 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa3c815aca0 0x7fa3c80c6aa0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fa3c0034040 tx=0x7fa3c0004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:01.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.532+0000 7fa3baffd640 1 -- 172.21.15.27:0/1352956193 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3c000a620 con 0x7fa3c815aca0 2024-09-12T03:20:01.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.532+0000 7fa3baffd640 1 -- 172.21.15.27:0/1352956193 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3c0032630 con 0x7fa3c815aca0 2024-09-12T03:20:01.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.532+0000 7fa3d0752640 1 -- 172.21.15.27:0/1352956193 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3c814fd50 con 0x7fa3c815aca0 2024-09-12T03:20:01.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.532+0000 7fa3baffd640 1 -- 172.21.15.27:0/1352956193 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3c0032a20 con 0x7fa3c815aca0 2024-09-12T03:20:01.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.532+0000 7fa3d0752640 1 -- 172.21.15.27:0/1352956193 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3c80c5ce0 con 0x7fa3c815aca0 2024-09-12T03:20:01.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.533+0000 7fa3d0752640 1 -- 172.21.15.27:0/1352956193 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3c8072ba0 con 0x7fa3c815aca0 2024-09-12T03:20:01.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.533+0000 7fa3baffd640 1 -- 172.21.15.27:0/1352956193 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 11) ==== 90804+0+0 (secure 0 0 0) 0x7fa3c0032bc0 con 0x7fa3c815aca0 2024-09-12T03:20:01.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.534+0000 7fa3baffd640 1 --2- 172.21.15.27:0/1352956193 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fa3ac06c1c0 0x7fa3ac06e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:01.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.534+0000 7fa3baffd640 1 -- 172.21.15.27:0/1352956193 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 5709+0+0 (secure 0 0 0) 0x7fa3c00ac360 con 0x7fa3c815aca0 2024-09-12T03:20:01.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.534+0000 7fa3ce4ee640 1 --2- 172.21.15.27:0/1352956193 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fa3ac06c1c0 0x7fa3ac06e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:01.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.534+0000 7fa3ce4ee640 1 --2- 172.21.15.27:0/1352956193 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fa3ac06c1c0 0x7fa3ac06e680 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fa3c4009ad0 tx=0x7fa3c4008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:01.539 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.538+0000 7fa3baffd640 1 -- 172.21.15.27:0/1352956193 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fa3c81f50d0 con 0x7fa3c815aca0 2024-09-12T03:20:01.711 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.710+0000 7fa3d0752640 1 -- 172.21.15.27:0/1352956193 --> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa3c80b1fd0 con 0x7fa3ac06c1c0 2024-09-12T03:20:01.717 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.717+0000 7fa3baffd640 1 -- 172.21.15.27:0/1352956193 <== mgr.4102 v2:172.21.15.45:6832/3923821219 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+248831 (secure 0 0 0) 0x7fa3c80b1fd0 con 0x7fa3ac06c1c0 2024-09-12T03:20:01.720 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:20:01.721 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2024-09-12T03:20:01.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.720+0000 7fa3d0752640 1 -- 172.21.15.27:0/1352956193 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fa3ac06c1c0 msgr2=0x7fa3ac06e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:01.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.720+0000 7fa3d0752640 1 --2- 172.21.15.27:0/1352956193 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fa3ac06c1c0 0x7fa3ac06e680 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fa3c4009ad0 tx=0x7fa3c4008040 comp rx=0 tx=0).stop 2024-09-12T03:20:01.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.720+0000 7fa3d0752640 1 -- 172.21.15.27:0/1352956193 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa3c815aca0 msgr2=0x7fa3c80c6aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:01.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.720+0000 7fa3d0752640 1 --2- 172.21.15.27:0/1352956193 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa3c815aca0 0x7fa3c80c6aa0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fa3c0034040 tx=0x7fa3c0004ab0 comp rx=0 tx=0).stop 2024-09-12T03:20:01.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.720+0000 7fa3d0752640 1 -- 172.21.15.27:0/1352956193 shutdown_connections 2024-09-12T03:20:01.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.720+0000 7fa3d0752640 1 --2- 172.21.15.27:0/1352956193 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fa3ac06c1c0 0x7fa3ac06e680 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:01.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.720+0000 7fa3d0752640 1 --2- 172.21.15.27:0/1352956193 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa3c815aca0 0x7fa3c80c6aa0 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:01.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.720+0000 7fa3d0752640 1 --2- 172.21.15.27:0/1352956193 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa3c815a8c0 0x7fa3c814f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:01.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.720+0000 7fa3d0752640 1 --2- 172.21.15.27:0/1352956193 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa3c8071be0 0x7fa3c81f4b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:01.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.720+0000 7fa3d0752640 1 -- 172.21.15.27:0/1352956193 >> 172.21.15.27:0/1352956193 conn(0x7fa3c8062c60 msgr2=0x7fa3c80b4710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:01.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.720+0000 7fa3d0752640 1 -- 172.21.15.27:0/1352956193 shutdown_connections 2024-09-12T03:20:01.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.720+0000 7fa3d0752640 1 -- 172.21.15.27:0/1352956193 wait complete. 2024-09-12T03:20:01.732 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":170,"stamp":"2024-09-12T03:20:00.103531+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":591132,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":194,"num_write_kb":2420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":147,"ondisk_log_size":147,"up":154,"acting":154,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":100,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":6,"kb":754974720,"kb_used":252804,"kb_used_data":7404,"kb_used_omap":12,"kb_used_meta":245171,"kb_avail":754721916,"statfs":{"total":773094113280,"available":772835241984,"internally_reserved":0,"allocated":7581696,"data_stored":3828844,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":251055664},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":764,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":35,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001083"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.412295+0000","last_change":"2024-09-12T03:19:58.405738+0000","last_active":"2024-09-12T03:19:59.412295+0000","last_peered":"2024-09-12T03:19:59.412295+0000","last_clean":"2024-09-12T03:19:59.412295+0000","last_became_active":"2024-09-12T03:19:58.405044+0000","last_became_peered":"2024-09-12T03:19:58.405044+0000","last_unstale":"2024-09-12T03:19:59.412295+0000","last_undegraded":"2024-09-12T03:19:59.412295+0000","last_fullsized":"2024-09-12T03:19:59.412295+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:24:50.892593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.412848+0000","last_change":"2024-09-12T03:19:58.407057+0000","last_active":"2024-09-12T03:19:59.412848+0000","last_peered":"2024-09-12T03:19:59.412848+0000","last_clean":"2024-09-12T03:19:59.412848+0000","last_became_active":"2024-09-12T03:19:58.406455+0000","last_became_peered":"2024-09-12T03:19:58.406455+0000","last_unstale":"2024-09-12T03:19:59.412848+0000","last_undegraded":"2024-09-12T03:19:59.412848+0000","last_fullsized":"2024-09-12T03:19:59.412848+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:12:06.463949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.411816+0000","last_change":"2024-09-12T03:19:58.402689+0000","last_active":"2024-09-12T03:19:59.411816+0000","last_peered":"2024-09-12T03:19:59.411816+0000","last_clean":"2024-09-12T03:19:59.411816+0000","last_became_active":"2024-09-12T03:19:58.402424+0000","last_became_peered":"2024-09-12T03:19:58.402424+0000","last_unstale":"2024-09-12T03:19:59.411816+0000","last_undegraded":"2024-09-12T03:19:59.411816+0000","last_fullsized":"2024-09-12T03:19:59.411816+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:01:53.466856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:19:57.371227+0000","last_change":"2024-09-12T03:19:57.371227+0000","last_active":"2024-09-12T03:19:57.371227+0000","last_peered":"2024-09-12T03:19:57.371227+0000","last_clean":"2024-09-12T03:19:57.371227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:57.371227+0000","last_undegraded":"2024-09-12T03:19:57.371227+0000","last_fullsized":"2024-09-12T03:19:57.371227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.411395+0000","last_change":"2024-09-12T03:19:58.405012+0000","last_active":"2024-09-12T03:19:59.411395+0000","last_peered":"2024-09-12T03:19:59.411395+0000","last_clean":"2024-09-12T03:19:59.411395+0000","last_became_active":"2024-09-12T03:19:58.404706+0000","last_became_peered":"2024-09-12T03:19:58.404706+0000","last_unstale":"2024-09-12T03:19:59.411395+0000","last_undegraded":"2024-09-12T03:19:59.411395+0000","last_fullsized":"2024-09-12T03:19:59.411395+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:13:43.693628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.412119+0000","last_change":"2024-09-12T03:19:58.404152+0000","last_active":"2024-09-12T03:19:59.412119+0000","last_peered":"2024-09-12T03:19:59.412119+0000","last_clean":"2024-09-12T03:19:59.412119+0000","last_became_active":"2024-09-12T03:19:58.403865+0000","last_became_peered":"2024-09-12T03:19:58.403865+0000","last_unstale":"2024-09-12T03:19:59.412119+0000","last_undegraded":"2024-09-12T03:19:59.412119+0000","last_fullsized":"2024-09-12T03:19:59.412119+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:03:35.086339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:19:57.371227+0000","last_change":"2024-09-12T03:19:57.371227+0000","last_active":"2024-09-12T03:19:57.371227+0000","last_peered":"2024-09-12T03:19:57.371227+0000","last_clean":"2024-09-12T03:19:57.371227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:57.371227+0000","last_undegraded":"2024-09-12T03:19:57.371227+0000","last_fullsized":"2024-09-12T03:19:57.371227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.412001+0000","last_change":"2024-09-12T03:19:58.408692+0000","last_active":"2024-09-12T03:19:59.412001+0000","last_peered":"2024-09-12T03:19:59.412001+0000","last_clean":"2024-09-12T03:19:59.412001+0000","last_became_active":"2024-09-12T03:19:58.408182+0000","last_became_peered":"2024-09-12T03:19:58.408182+0000","last_unstale":"2024-09-12T03:19:59.412001+0000","last_undegraded":"2024-09-12T03:19:59.412001+0000","last_fullsized":"2024-09-12T03:19:59.412001+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:25:56.007148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.412647+0000","last_change":"2024-09-12T03:19:58.408054+0000","last_active":"2024-09-12T03:19:59.412647+0000","last_peered":"2024-09-12T03:19:59.412647+0000","last_clean":"2024-09-12T03:19:59.412647+0000","last_became_active":"2024-09-12T03:19:58.407439+0000","last_became_peered":"2024-09-12T03:19:58.407439+0000","last_unstale":"2024-09-12T03:19:59.412647+0000","last_undegraded":"2024-09-12T03:19:59.412647+0000","last_fullsized":"2024-09-12T03:19:59.412647+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:52:55.350315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.399858+0000","last_change":"2024-09-12T03:19:58.400878+0000","last_active":"2024-09-12T03:19:59.399858+0000","last_peered":"2024-09-12T03:19:59.399858+0000","last_clean":"2024-09-12T03:19:59.399858+0000","last_became_active":"2024-09-12T03:19:58.400601+0000","last_became_peered":"2024-09-12T03:19:58.400601+0000","last_unstale":"2024-09-12T03:19:59.399858+0000","last_undegraded":"2024-09-12T03:19:59.399858+0000","last_fullsized":"2024-09-12T03:19:59.399858+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:53:33.025355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.412388+0000","last_change":"2024-09-12T03:19:58.404349+0000","last_active":"2024-09-12T03:19:59.412388+0000","last_peered":"2024-09-12T03:19:59.412388+0000","last_clean":"2024-09-12T03:19:59.412388+0000","last_became_active":"2024-09-12T03:19:58.404067+0000","last_became_peered":"2024-09-12T03:19:58.404067+0000","last_unstale":"2024-09-12T03:19:59.412388+0000","last_undegraded":"2024-09-12T03:19:59.412388+0000","last_fullsized":"2024-09-12T03:19:59.412388+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:31:06.800562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:19:57.371227+0000","last_change":"2024-09-12T03:19:57.371227+0000","last_active":"2024-09-12T03:19:57.371227+0000","last_peered":"2024-09-12T03:19:57.371227+0000","last_clean":"2024-09-12T03:19:57.371227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:57.371227+0000","last_undegraded":"2024-09-12T03:19:57.371227+0000","last_fullsized":"2024-09-12T03:19:57.371227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.398467+0000","last_change":"2024-09-12T03:19:58.408147+0000","last_active":"2024-09-12T03:19:59.398467+0000","last_peered":"2024-09-12T03:19:59.398467+0000","last_clean":"2024-09-12T03:19:59.398467+0000","last_became_active":"2024-09-12T03:19:58.407855+0000","last_became_peered":"2024-09-12T03:19:58.407855+0000","last_unstale":"2024-09-12T03:19:59.398467+0000","last_undegraded":"2024-09-12T03:19:59.398467+0000","last_fullsized":"2024-09-12T03:19:59.398467+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:50:40.770697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.412570+0000","last_change":"2024-09-12T03:19:58.401979+0000","last_active":"2024-09-12T03:19:59.412570+0000","last_peered":"2024-09-12T03:19:59.412570+0000","last_clean":"2024-09-12T03:19:59.412570+0000","last_became_active":"2024-09-12T03:19:58.401689+0000","last_became_peered":"2024-09-12T03:19:58.401689+0000","last_unstale":"2024-09-12T03:19:59.412570+0000","last_undegraded":"2024-09-12T03:19:59.412570+0000","last_fullsized":"2024-09-12T03:19:59.412570+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:22:57.280328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.398306+0000","last_change":"2024-09-12T03:19:58.399312+0000","last_active":"2024-09-12T03:19:59.398306+0000","last_peered":"2024-09-12T03:19:59.398306+0000","last_clean":"2024-09-12T03:19:59.398306+0000","last_became_active":"2024-09-12T03:19:58.399050+0000","last_became_peered":"2024-09-12T03:19:58.399050+0000","last_unstale":"2024-09-12T03:19:59.398306+0000","last_undegraded":"2024-09-12T03:19:59.398306+0000","last_fullsized":"2024-09-12T03:19:59.398306+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:41:13.177460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.400875+0000","last_change":"2024-09-12T03:19:58.404580+0000","last_active":"2024-09-12T03:19:59.400875+0000","last_peered":"2024-09-12T03:19:59.400875+0000","last_clean":"2024-09-12T03:19:59.400875+0000","last_became_active":"2024-09-12T03:19:58.404320+0000","last_became_peered":"2024-09-12T03:19:58.404320+0000","last_unstale":"2024-09-12T03:19:59.400875+0000","last_undegraded":"2024-09-12T03:19:59.400875+0000","last_fullsized":"2024-09-12T03:19:59.400875+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:41:51.158105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.412705+0000","last_change":"2024-09-12T03:19:58.408389+0000","last_active":"2024-09-12T03:19:59.412705+0000","last_peered":"2024-09-12T03:19:59.412705+0000","last_clean":"2024-09-12T03:19:59.412705+0000","last_became_active":"2024-09-12T03:19:58.407797+0000","last_became_peered":"2024-09-12T03:19:58.407797+0000","last_unstale":"2024-09-12T03:19:59.412705+0000","last_undegraded":"2024-09-12T03:19:59.412705+0000","last_fullsized":"2024-09-12T03:19:59.412705+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:17:24.698351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:19:57.371227+0000","last_change":"2024-09-12T03:19:57.371227+0000","last_active":"2024-09-12T03:19:57.371227+0000","last_peered":"2024-09-12T03:19:57.371227+0000","last_clean":"2024-09-12T03:19:57.371227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:57.371227+0000","last_undegraded":"2024-09-12T03:19:57.371227+0000","last_fullsized":"2024-09-12T03:19:57.371227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.412173+0000","last_change":"2024-09-12T03:19:58.408115+0000","last_active":"2024-09-12T03:19:59.412173+0000","last_peered":"2024-09-12T03:19:59.412173+0000","last_clean":"2024-09-12T03:19:59.412173+0000","last_became_active":"2024-09-12T03:19:58.407745+0000","last_became_peered":"2024-09-12T03:19:58.407745+0000","last_unstale":"2024-09-12T03:19:59.412173+0000","last_undegraded":"2024-09-12T03:19:59.412173+0000","last_fullsized":"2024-09-12T03:19:59.412173+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:50:17.515387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.398554+0000","last_change":"2024-09-12T03:19:58.398467+0000","last_active":"2024-09-12T03:19:59.398554+0000","last_peered":"2024-09-12T03:19:59.398554+0000","last_clean":"2024-09-12T03:19:59.398554+0000","last_became_active":"2024-09-12T03:19:58.398152+0000","last_became_peered":"2024-09-12T03:19:58.398152+0000","last_unstale":"2024-09-12T03:19:59.398554+0000","last_undegraded":"2024-09-12T03:19:59.398554+0000","last_fullsized":"2024-09-12T03:19:59.398554+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:24:50.892593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.400458+0000","last_change":"2024-09-12T03:19:58.400541+0000","last_active":"2024-09-12T03:19:59.400458+0000","last_peered":"2024-09-12T03:19:59.400458+0000","last_clean":"2024-09-12T03:19:59.400458+0000","last_became_active":"2024-09-12T03:19:58.399602+0000","last_became_peered":"2024-09-12T03:19:58.399602+0000","last_unstale":"2024-09-12T03:19:59.400458+0000","last_undegraded":"2024-09-12T03:19:59.400458+0000","last_fullsized":"2024-09-12T03:19:59.400458+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:56:37.761014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.400370+0000","last_change":"2024-09-12T03:19:58.399427+0000","last_active":"2024-09-12T03:19:59.400370+0000","last_peered":"2024-09-12T03:19:59.400370+0000","last_clean":"2024-09-12T03:19:59.400370+0000","last_became_active":"2024-09-12T03:19:58.398464+0000","last_became_peered":"2024-09-12T03:19:58.398464+0000","last_unstale":"2024-09-12T03:19:59.400370+0000","last_undegraded":"2024-09-12T03:19:59.400370+0000","last_fullsized":"2024-09-12T03:19:59.400370+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:44:19.209045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.411588+0000","last_change":"2024-09-12T03:19:58.405456+0000","last_active":"2024-09-12T03:19:59.411588+0000","last_peered":"2024-09-12T03:19:59.411588+0000","last_clean":"2024-09-12T03:19:59.411588+0000","last_became_active":"2024-09-12T03:19:58.404974+0000","last_became_peered":"2024-09-12T03:19:58.404974+0000","last_unstale":"2024-09-12T03:19:59.411588+0000","last_undegraded":"2024-09-12T03:19:59.411588+0000","last_fullsized":"2024-09-12T03:19:59.411588+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:58:58.914180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.411477+0000","last_change":"2024-09-12T03:19:58.408893+0000","last_active":"2024-09-12T03:19:59.411477+0000","last_peered":"2024-09-12T03:19:59.411477+0000","last_clean":"2024-09-12T03:19:59.411477+0000","last_became_active":"2024-09-12T03:19:58.408589+0000","last_became_peered":"2024-09-12T03:19:58.408589+0000","last_unstale":"2024-09-12T03:19:59.411477+0000","last_undegraded":"2024-09-12T03:19:59.411477+0000","last_fullsized":"2024-09-12T03:19:59.411477+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:54:48.020501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.400037+0000","last_change":"2024-09-12T03:19:58.411515+0000","last_active":"2024-09-12T03:19:59.400037+0000","last_peered":"2024-09-12T03:19:59.400037+0000","last_clean":"2024-09-12T03:19:59.400037+0000","last_became_active":"2024-09-12T03:19:58.409091+0000","last_became_peered":"2024-09-12T03:19:58.409091+0000","last_unstale":"2024-09-12T03:19:59.400037+0000","last_undegraded":"2024-09-12T03:19:59.400037+0000","last_fullsized":"2024-09-12T03:19:59.400037+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:25:42.004151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:19:57.371227+0000","last_change":"2024-09-12T03:19:57.371227+0000","last_active":"2024-09-12T03:19:57.371227+0000","last_peered":"2024-09-12T03:19:57.371227+0000","last_clean":"2024-09-12T03:19:57.371227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:57.371227+0000","last_undegraded":"2024-09-12T03:19:57.371227+0000","last_fullsized":"2024-09-12T03:19:57.371227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.411966+0000","last_change":"2024-09-12T03:19:58.406104+0000","last_active":"2024-09-12T03:19:59.411966+0000","last_peered":"2024-09-12T03:19:59.411966+0000","last_clean":"2024-09-12T03:19:59.411966+0000","last_became_active":"2024-09-12T03:19:58.405546+0000","last_became_peered":"2024-09-12T03:19:58.405546+0000","last_unstale":"2024-09-12T03:19:59.411966+0000","last_undegraded":"2024-09-12T03:19:59.411966+0000","last_fullsized":"2024-09-12T03:19:59.411966+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:45:16.630721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.400401+0000","last_change":"2024-09-12T03:19:58.401764+0000","last_active":"2024-09-12T03:19:59.400401+0000","last_peered":"2024-09-12T03:19:59.400401+0000","last_clean":"2024-09-12T03:19:59.400401+0000","last_became_active":"2024-09-12T03:19:58.397972+0000","last_became_peered":"2024-09-12T03:19:58.397972+0000","last_unstale":"2024-09-12T03:19:59.400401+0000","last_undegraded":"2024-09-12T03:19:59.400401+0000","last_fullsized":"2024-09-12T03:19:59.400401+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:50:11.761046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.412120+0000","last_change":"2024-09-12T03:19:58.403549+0000","last_active":"2024-09-12T03:19:59.412120+0000","last_peered":"2024-09-12T03:19:59.412120+0000","last_clean":"2024-09-12T03:19:59.412120+0000","last_became_active":"2024-09-12T03:19:58.403294+0000","last_became_peered":"2024-09-12T03:19:58.403294+0000","last_unstale":"2024-09-12T03:19:59.412120+0000","last_undegraded":"2024-09-12T03:19:59.412120+0000","last_fullsized":"2024-09-12T03:19:59.412120+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:20:56.214083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:19:57.371227+0000","last_change":"2024-09-12T03:19:57.371227+0000","last_active":"2024-09-12T03:19:57.371227+0000","last_peered":"2024-09-12T03:19:57.371227+0000","last_clean":"2024-09-12T03:19:57.371227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:57.371227+0000","last_undegraded":"2024-09-12T03:19:57.371227+0000","last_fullsized":"2024-09-12T03:19:57.371227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:19:57.371227+0000","last_change":"2024-09-12T03:19:57.371227+0000","last_active":"2024-09-12T03:19:57.371227+0000","last_peered":"2024-09-12T03:19:57.371227+0000","last_clean":"2024-09-12T03:19:57.371227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:57.371227+0000","last_undegraded":"2024-09-12T03:19:57.371227+0000","last_fullsized":"2024-09-12T03:19:57.371227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.412055+0000","last_change":"2024-09-12T03:19:58.407063+0000","last_active":"2024-09-12T03:19:59.412055+0000","last_peered":"2024-09-12T03:19:59.412055+0000","last_clean":"2024-09-12T03:19:59.412055+0000","last_became_active":"2024-09-12T03:19:58.406116+0000","last_became_peered":"2024-09-12T03:19:58.406116+0000","last_unstale":"2024-09-12T03:19:59.412055+0000","last_undegraded":"2024-09-12T03:19:59.412055+0000","last_fullsized":"2024-09-12T03:19:59.412055+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:11:39.227764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.411730+0000","last_change":"2024-09-12T03:19:58.409587+0000","last_active":"2024-09-12T03:19:59.411730+0000","last_peered":"2024-09-12T03:19:59.411730+0000","last_clean":"2024-09-12T03:19:59.411730+0000","last_became_active":"2024-09-12T03:19:58.409275+0000","last_became_peered":"2024-09-12T03:19:58.409275+0000","last_unstale":"2024-09-12T03:19:59.411730+0000","last_undegraded":"2024-09-12T03:19:59.411730+0000","last_fullsized":"2024-09-12T03:19:59.411730+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:00:46.656402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"66'4","reported_seq":18,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.412596+0000","last_change":"2024-09-12T03:19:57.394595+0000","last_active":"2024-09-12T03:19:59.412596+0000","last_peered":"2024-09-12T03:19:59.412596+0000","last_clean":"2024-09-12T03:19:59.412596+0000","last_became_active":"2024-09-12T03:19:57.394294+0000","last_became_peered":"2024-09-12T03:19:57.394294+0000","last_unstale":"2024-09-12T03:19:59.412596+0000","last_undegraded":"2024-09-12T03:19:59.412596+0000","last_fullsized":"2024-09-12T03:19:59.412596+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_clean_scrub_stamp":"2024-09-12T03:19:56.366974+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:51:58.062647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.412771+0000","last_change":"2024-09-12T03:19:58.407115+0000","last_active":"2024-09-12T03:19:59.412771+0000","last_peered":"2024-09-12T03:19:59.412771+0000","last_clean":"2024-09-12T03:19:59.412771+0000","last_became_active":"2024-09-12T03:19:58.406683+0000","last_became_peered":"2024-09-12T03:19:58.406683+0000","last_unstale":"2024-09-12T03:19:59.412771+0000","last_undegraded":"2024-09-12T03:19:59.412771+0000","last_fullsized":"2024-09-12T03:19:59.412771+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:36:57.331135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"66'1","reported_seq":15,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.398595+0000","last_change":"2024-09-12T03:19:57.389982+0000","last_active":"2024-09-12T03:19:59.398595+0000","last_peered":"2024-09-12T03:19:59.398595+0000","last_clean":"2024-09-12T03:19:59.398595+0000","last_became_active":"2024-09-12T03:19:57.389694+0000","last_became_peered":"2024-09-12T03:19:57.389694+0000","last_unstale":"2024-09-12T03:19:59.398595+0000","last_undegraded":"2024-09-12T03:19:59.398595+0000","last_fullsized":"2024-09-12T03:19:59.398595+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_clean_scrub_stamp":"2024-09-12T03:19:56.366974+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:01:26.160193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.412792+0000","last_change":"2024-09-12T03:19:58.415175+0000","last_active":"2024-09-12T03:19:59.412792+0000","last_peered":"2024-09-12T03:19:59.412792+0000","last_clean":"2024-09-12T03:19:59.412792+0000","last_became_active":"2024-09-12T03:19:58.414899+0000","last_became_peered":"2024-09-12T03:19:58.414899+0000","last_unstale":"2024-09-12T03:19:59.412792+0000","last_undegraded":"2024-09-12T03:19:59.412792+0000","last_fullsized":"2024-09-12T03:19:59.412792+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:36:33.065448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"66'2","reported_seq":16,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.400437+0000","last_change":"2024-09-12T03:19:57.393946+0000","last_active":"2024-09-12T03:19:59.400437+0000","last_peered":"2024-09-12T03:19:59.400437+0000","last_clean":"2024-09-12T03:19:59.400437+0000","last_became_active":"2024-09-12T03:19:57.393672+0000","last_became_peered":"2024-09-12T03:19:57.393672+0000","last_unstale":"2024-09-12T03:19:59.400437+0000","last_undegraded":"2024-09-12T03:19:59.400437+0000","last_fullsized":"2024-09-12T03:19:59.400437+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_clean_scrub_stamp":"2024-09-12T03:19:56.366974+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:57:23.221949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.412495+0000","last_change":"2024-09-12T03:19:58.403124+0000","last_active":"2024-09-12T03:19:59.412495+0000","last_peered":"2024-09-12T03:19:59.412495+0000","last_clean":"2024-09-12T03:19:59.412495+0000","last_became_active":"2024-09-12T03:19:58.402859+0000","last_became_peered":"2024-09-12T03:19:58.402859+0000","last_unstale":"2024-09-12T03:19:59.412495+0000","last_undegraded":"2024-09-12T03:19:59.412495+0000","last_fullsized":"2024-09-12T03:19:59.412495+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:41:26.586794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"66'2","reported_seq":16,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.412896+0000","last_change":"2024-09-12T03:19:57.396585+0000","last_active":"2024-09-12T03:19:59.412896+0000","last_peered":"2024-09-12T03:19:59.412896+0000","last_clean":"2024-09-12T03:19:59.412896+0000","last_became_active":"2024-09-12T03:19:57.396109+0000","last_became_peered":"2024-09-12T03:19:57.396109+0000","last_unstale":"2024-09-12T03:19:59.412896+0000","last_undegraded":"2024-09-12T03:19:59.412896+0000","last_fullsized":"2024-09-12T03:19:59.412896+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_clean_scrub_stamp":"2024-09-12T03:19:56.366974+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:25:50.078358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:19:57.371227+0000","last_change":"2024-09-12T03:19:57.371227+0000","last_active":"2024-09-12T03:19:57.371227+0000","last_peered":"2024-09-12T03:19:57.371227+0000","last_clean":"2024-09-12T03:19:57.371227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:57.371227+0000","last_undegraded":"2024-09-12T03:19:57.371227+0000","last_fullsized":"2024-09-12T03:19:57.371227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1b","version":"66'1","reported_seq":15,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.412362+0000","last_change":"2024-09-12T03:19:57.391056+0000","last_active":"2024-09-12T03:19:59.412362+0000","last_peered":"2024-09-12T03:19:59.412362+0000","last_clean":"2024-09-12T03:19:59.412362+0000","last_became_active":"2024-09-12T03:19:57.390769+0000","last_became_peered":"2024-09-12T03:19:57.390769+0000","last_unstale":"2024-09-12T03:19:59.412362+0000","last_undegraded":"2024-09-12T03:19:59.412362+0000","last_fullsized":"2024-09-12T03:19:59.412362+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_clean_scrub_stamp":"2024-09-12T03:19:56.366974+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:41:12.173208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.400570+0000","last_change":"2024-09-12T03:19:58.415098+0000","last_active":"2024-09-12T03:19:59.400570+0000","last_peered":"2024-09-12T03:19:59.400570+0000","last_clean":"2024-09-12T03:19:59.400570+0000","last_became_active":"2024-09-12T03:19:58.411028+0000","last_became_peered":"2024-09-12T03:19:58.411028+0000","last_unstale":"2024-09-12T03:19:59.400570+0000","last_undegraded":"2024-09-12T03:19:59.400570+0000","last_fullsized":"2024-09-12T03:19:59.400570+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:29:23.148491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"66'1","reported_seq":15,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.412238+0000","last_change":"2024-09-12T03:19:57.395730+0000","last_active":"2024-09-12T03:19:59.412238+0000","last_peered":"2024-09-12T03:19:59.412238+0000","last_clean":"2024-09-12T03:19:59.412238+0000","last_became_active":"2024-09-12T03:19:57.395253+0000","last_became_peered":"2024-09-12T03:19:57.395253+0000","last_unstale":"2024-09-12T03:19:59.412238+0000","last_undegraded":"2024-09-12T03:19:59.412238+0000","last_fullsized":"2024-09-12T03:19:59.412238+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_clean_scrub_stamp":"2024-09-12T03:19:56.366974+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:54:51.158313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.411683+0000","last_change":"2024-09-12T03:19:58.400222+0000","last_active":"2024-09-12T03:19:59.411683+0000","last_peered":"2024-09-12T03:19:59.411683+0000","last_clean":"2024-09-12T03:19:59.411683+0000","last_became_active":"2024-09-12T03:19:58.399906+0000","last_became_peered":"2024-09-12T03:19:58.399906+0000","last_unstale":"2024-09-12T03:19:59.411683+0000","last_undegraded":"2024-09-12T03:19:59.411683+0000","last_fullsized":"2024-09-12T03:19:59.411683+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:17:22.493344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:19:56.366974+0000","last_change":"2024-09-12T03:19:56.366974+0000","last_active":"2024-09-12T03:19:56.366974+0000","last_peered":"2024-09-12T03:19:56.366974+0000","last_clean":"2024-09-12T03:19:56.366974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:56.366974+0000","last_undegraded":"2024-09-12T03:19:56.366974+0000","last_fullsized":"2024-09-12T03:19:56.366974+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_clean_scrub_stamp":"2024-09-12T03:19:56.366974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.398361+0000","last_change":"2024-09-12T03:19:58.405943+0000","last_active":"2024-09-12T03:19:59.398361+0000","last_peered":"2024-09-12T03:19:59.398361+0000","last_clean":"2024-09-12T03:19:59.398361+0000","last_became_active":"2024-09-12T03:19:58.405630+0000","last_became_peered":"2024-09-12T03:19:58.405630+0000","last_unstale":"2024-09-12T03:19:59.398361+0000","last_undegraded":"2024-09-12T03:19:59.398361+0000","last_fullsized":"2024-09-12T03:19:59.398361+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:00:46.656402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.411776+0000","last_change":"2024-09-12T03:19:57.394447+0000","last_active":"2024-09-12T03:19:59.411776+0000","last_peered":"2024-09-12T03:19:59.411776+0000","last_clean":"2024-09-12T03:19:59.411776+0000","last_became_active":"2024-09-12T03:19:57.394153+0000","last_became_peered":"2024-09-12T03:19:57.394153+0000","last_unstale":"2024-09-12T03:19:59.411776+0000","last_undegraded":"2024-09-12T03:19:59.411776+0000","last_fullsized":"2024-09-12T03:19:59.411776+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_clean_scrub_stamp":"2024-09-12T03:19:56.366974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:12:19.700994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:19:57.371227+0000","last_change":"2024-09-12T03:19:57.371227+0000","last_active":"2024-09-12T03:19:57.371227+0000","last_peered":"2024-09-12T03:19:57.371227+0000","last_clean":"2024-09-12T03:19:57.371227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:57.371227+0000","last_undegraded":"2024-09-12T03:19:57.371227+0000","last_fullsized":"2024-09-12T03:19:57.371227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.400155+0000","last_change":"2024-09-12T03:19:57.393730+0000","last_active":"2024-09-12T03:19:59.400155+0000","last_peered":"2024-09-12T03:19:59.400155+0000","last_clean":"2024-09-12T03:19:59.400155+0000","last_became_active":"2024-09-12T03:19:57.393425+0000","last_became_peered":"2024-09-12T03:19:57.393425+0000","last_unstale":"2024-09-12T03:19:59.400155+0000","last_undegraded":"2024-09-12T03:19:59.400155+0000","last_fullsized":"2024-09-12T03:19:59.400155+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_clean_scrub_stamp":"2024-09-12T03:19:56.366974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:39:01.426373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.412571+0000","last_change":"2024-09-12T03:19:58.409130+0000","last_active":"2024-09-12T03:19:59.412571+0000","last_peered":"2024-09-12T03:19:59.412571+0000","last_clean":"2024-09-12T03:19:59.412571+0000","last_became_active":"2024-09-12T03:19:58.408838+0000","last_became_peered":"2024-09-12T03:19:58.408838+0000","last_unstale":"2024-09-12T03:19:59.412571+0000","last_undegraded":"2024-09-12T03:19:59.412571+0000","last_fullsized":"2024-09-12T03:19:59.412571+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:38:31.191391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.412967+0000","last_change":"2024-09-12T03:19:57.391703+0000","last_active":"2024-09-12T03:19:59.412967+0000","last_peered":"2024-09-12T03:19:59.412967+0000","last_clean":"2024-09-12T03:19:59.412967+0000","last_became_active":"2024-09-12T03:19:57.391345+0000","last_became_peered":"2024-09-12T03:19:57.391345+0000","last_unstale":"2024-09-12T03:19:59.412967+0000","last_undegraded":"2024-09-12T03:19:59.412967+0000","last_fullsized":"2024-09-12T03:19:59.412967+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_clean_scrub_stamp":"2024-09-12T03:19:56.366974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:05:09.764085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.412786+0000","last_change":"2024-09-12T03:19:58.410704+0000","last_active":"2024-09-12T03:19:59.412786+0000","last_peered":"2024-09-12T03:19:59.412786+0000","last_clean":"2024-09-12T03:19:59.412786+0000","last_became_active":"2024-09-12T03:19:58.410390+0000","last_became_peered":"2024-09-12T03:19:58.410390+0000","last_unstale":"2024-09-12T03:19:59.412786+0000","last_undegraded":"2024-09-12T03:19:59.412786+0000","last_fullsized":"2024-09-12T03:19:59.412786+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:27:21.762076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"66'3","reported_seq":17,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.412074+0000","last_change":"2024-09-12T03:19:57.383973+0000","last_active":"2024-09-12T03:19:59.412074+0000","last_peered":"2024-09-12T03:19:59.412074+0000","last_clean":"2024-09-12T03:19:59.412074+0000","last_became_active":"2024-09-12T03:19:57.383481+0000","last_became_peered":"2024-09-12T03:19:57.383481+0000","last_unstale":"2024-09-12T03:19:59.412074+0000","last_undegraded":"2024-09-12T03:19:59.412074+0000","last_fullsized":"2024-09-12T03:19:59.412074+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_clean_scrub_stamp":"2024-09-12T03:19:56.366974+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:00:45.652150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.411674+0000","last_change":"2024-09-12T03:19:58.400811+0000","last_active":"2024-09-12T03:19:59.411674+0000","last_peered":"2024-09-12T03:19:59.411674+0000","last_clean":"2024-09-12T03:19:59.411674+0000","last_became_active":"2024-09-12T03:19:58.400542+0000","last_became_peered":"2024-09-12T03:19:58.400542+0000","last_unstale":"2024-09-12T03:19:59.411674+0000","last_undegraded":"2024-09-12T03:19:59.411674+0000","last_fullsized":"2024-09-12T03:19:59.411674+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:01:16.275564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:19:56.366974+0000","last_change":"2024-09-12T03:19:56.366974+0000","last_active":"2024-09-12T03:19:56.366974+0000","last_peered":"2024-09-12T03:19:56.366974+0000","last_clean":"2024-09-12T03:19:56.366974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:56.366974+0000","last_undegraded":"2024-09-12T03:19:56.366974+0000","last_fullsized":"2024-09-12T03:19:56.366974+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_clean_scrub_stamp":"2024-09-12T03:19:56.366974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.411683+0000","last_change":"2024-09-12T03:19:58.402659+0000","last_active":"2024-09-12T03:19:59.411683+0000","last_peered":"2024-09-12T03:19:59.411683+0000","last_clean":"2024-09-12T03:19:59.411683+0000","last_became_active":"2024-09-12T03:19:58.402358+0000","last_became_peered":"2024-09-12T03:19:58.402358+0000","last_unstale":"2024-09-12T03:19:59.411683+0000","last_undegraded":"2024-09-12T03:19:59.411683+0000","last_fullsized":"2024-09-12T03:19:59.411683+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:41:13.177460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:19:56.366974+0000","last_change":"2024-09-12T03:19:56.366974+0000","last_active":"2024-09-12T03:19:56.366974+0000","last_peered":"2024-09-12T03:19:56.366974+0000","last_clean":"2024-09-12T03:19:56.366974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:56.366974+0000","last_undegraded":"2024-09-12T03:19:56.366974+0000","last_fullsized":"2024-09-12T03:19:56.366974+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_clean_scrub_stamp":"2024-09-12T03:19:56.366974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:19:57.371227+0000","last_change":"2024-09-12T03:19:57.371227+0000","last_active":"2024-09-12T03:19:57.371227+0000","last_peered":"2024-09-12T03:19:57.371227+0000","last_clean":"2024-09-12T03:19:57.371227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:57.371227+0000","last_undegraded":"2024-09-12T03:19:57.371227+0000","last_fullsized":"2024-09-12T03:19:57.371227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.412349+0000","last_change":"2024-09-12T03:19:57.395906+0000","last_active":"2024-09-12T03:19:59.412349+0000","last_peered":"2024-09-12T03:19:59.412349+0000","last_clean":"2024-09-12T03:19:59.412349+0000","last_became_active":"2024-09-12T03:19:57.395558+0000","last_became_peered":"2024-09-12T03:19:57.395558+0000","last_unstale":"2024-09-12T03:19:59.412349+0000","last_undegraded":"2024-09-12T03:19:59.412349+0000","last_fullsized":"2024-09-12T03:19:59.412349+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_clean_scrub_stamp":"2024-09-12T03:19:56.366974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:23:25.739217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.398297+0000","last_change":"2024-09-12T03:19:58.407844+0000","last_active":"2024-09-12T03:19:59.398297+0000","last_peered":"2024-09-12T03:19:59.398297+0000","last_clean":"2024-09-12T03:19:59.398297+0000","last_became_active":"2024-09-12T03:19:58.407513+0000","last_became_peered":"2024-09-12T03:19:58.407513+0000","last_unstale":"2024-09-12T03:19:59.398297+0000","last_undegraded":"2024-09-12T03:19:59.398297+0000","last_fullsized":"2024-09-12T03:19:59.398297+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:58:58.914180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:19:56.366974+0000","last_change":"2024-09-12T03:19:56.366974+0000","last_active":"2024-09-12T03:19:56.366974+0000","last_peered":"2024-09-12T03:19:56.366974+0000","last_clean":"2024-09-12T03:19:56.366974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:56.366974+0000","last_undegraded":"2024-09-12T03:19:56.366974+0000","last_fullsized":"2024-09-12T03:19:56.366974+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_clean_scrub_stamp":"2024-09-12T03:19:56.366974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.412013+0000","last_change":"2024-09-12T03:19:58.406880+0000","last_active":"2024-09-12T03:19:59.412013+0000","last_peered":"2024-09-12T03:19:59.412013+0000","last_clean":"2024-09-12T03:19:59.412013+0000","last_became_active":"2024-09-12T03:19:58.406585+0000","last_became_peered":"2024-09-12T03:19:58.406585+0000","last_unstale":"2024-09-12T03:19:59.412013+0000","last_undegraded":"2024-09-12T03:19:59.412013+0000","last_fullsized":"2024-09-12T03:19:59.412013+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:44:18.879486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"66'8","reported_seq":22,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.400987+0000","last_change":"2024-09-12T03:19:57.397454+0000","last_active":"2024-09-12T03:19:59.400987+0000","last_peered":"2024-09-12T03:19:59.400987+0000","last_clean":"2024-09-12T03:19:59.400987+0000","last_became_active":"2024-09-12T03:19:57.397126+0000","last_became_peered":"2024-09-12T03:19:57.397126+0000","last_unstale":"2024-09-12T03:19:59.400987+0000","last_undegraded":"2024-09-12T03:19:59.400987+0000","last_fullsized":"2024-09-12T03:19:59.400987+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_clean_scrub_stamp":"2024-09-12T03:19:56.366974+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:14:13.101395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.400608+0000","last_change":"2024-09-12T03:19:58.402681+0000","last_active":"2024-09-12T03:19:59.400608+0000","last_peered":"2024-09-12T03:19:59.400608+0000","last_clean":"2024-09-12T03:19:59.400608+0000","last_became_active":"2024-09-12T03:19:58.402409+0000","last_became_peered":"2024-09-12T03:19:58.402409+0000","last_unstale":"2024-09-12T03:19:59.400608+0000","last_undegraded":"2024-09-12T03:19:59.400608+0000","last_fullsized":"2024-09-12T03:19:59.400608+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:56:05.472430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"66'1","reported_seq":15,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.412766+0000","last_change":"2024-09-12T03:19:57.394555+0000","last_active":"2024-09-12T03:19:59.412766+0000","last_peered":"2024-09-12T03:19:59.412766+0000","last_clean":"2024-09-12T03:19:59.412766+0000","last_became_active":"2024-09-12T03:19:57.394277+0000","last_became_peered":"2024-09-12T03:19:57.394277+0000","last_unstale":"2024-09-12T03:19:59.412766+0000","last_undegraded":"2024-09-12T03:19:59.412766+0000","last_fullsized":"2024-09-12T03:19:59.412766+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_clean_scrub_stamp":"2024-09-12T03:19:56.366974+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:50:52.123837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.412469+0000","last_change":"2024-09-12T03:19:58.411847+0000","last_active":"2024-09-12T03:19:59.412469+0000","last_peered":"2024-09-12T03:19:59.412469+0000","last_clean":"2024-09-12T03:19:59.412469+0000","last_became_active":"2024-09-12T03:19:58.409965+0000","last_became_peered":"2024-09-12T03:19:58.409965+0000","last_unstale":"2024-09-12T03:19:59.412469+0000","last_undegraded":"2024-09-12T03:19:59.412469+0000","last_fullsized":"2024-09-12T03:19:59.412469+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:45:22.536315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"66'2","reported_seq":16,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.400471+0000","last_change":"2024-09-12T03:19:57.394701+0000","last_active":"2024-09-12T03:19:59.400471+0000","last_peered":"2024-09-12T03:19:59.400471+0000","last_clean":"2024-09-12T03:19:59.400471+0000","last_became_active":"2024-09-12T03:19:57.394348+0000","last_became_peered":"2024-09-12T03:19:57.394348+0000","last_unstale":"2024-09-12T03:19:59.400471+0000","last_undegraded":"2024-09-12T03:19:59.400471+0000","last_fullsized":"2024-09-12T03:19:59.400471+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_clean_scrub_stamp":"2024-09-12T03:19:56.366974+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:44:36.098558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:19:57.371227+0000","last_change":"2024-09-12T03:19:57.371227+0000","last_active":"2024-09-12T03:19:57.371227+0000","last_peered":"2024-09-12T03:19:57.371227+0000","last_clean":"2024-09-12T03:19:57.371227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:57.371227+0000","last_undegraded":"2024-09-12T03:19:57.371227+0000","last_fullsized":"2024-09-12T03:19:57.371227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.d","version":"66'1","reported_seq":15,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.412373+0000","last_change":"2024-09-12T03:19:57.397975+0000","last_active":"2024-09-12T03:19:59.412373+0000","last_peered":"2024-09-12T03:19:59.412373+0000","last_clean":"2024-09-12T03:19:59.412373+0000","last_became_active":"2024-09-12T03:19:57.397637+0000","last_became_peered":"2024-09-12T03:19:57.397637+0000","last_unstale":"2024-09-12T03:19:59.412373+0000","last_undegraded":"2024-09-12T03:19:59.412373+0000","last_fullsized":"2024-09-12T03:19:59.412373+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_clean_scrub_stamp":"2024-09-12T03:19:56.366974+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:16:35.102685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.411820+0000","last_change":"2024-09-12T03:19:58.399501+0000","last_active":"2024-09-12T03:19:59.411820+0000","last_peered":"2024-09-12T03:19:59.411820+0000","last_clean":"2024-09-12T03:19:59.411820+0000","last_became_active":"2024-09-12T03:19:58.399203+0000","last_became_peered":"2024-09-12T03:19:58.399203+0000","last_unstale":"2024-09-12T03:19:59.411820+0000","last_undegraded":"2024-09-12T03:19:59.411820+0000","last_fullsized":"2024-09-12T03:19:59.411820+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:45:17.689061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.412050+0000","last_change":"2024-09-12T03:19:57.394026+0000","last_active":"2024-09-12T03:19:59.412050+0000","last_peered":"2024-09-12T03:19:59.412050+0000","last_clean":"2024-09-12T03:19:59.412050+0000","last_became_active":"2024-09-12T03:19:57.393679+0000","last_became_peered":"2024-09-12T03:19:57.393679+0000","last_unstale":"2024-09-12T03:19:59.412050+0000","last_undegraded":"2024-09-12T03:19:59.412050+0000","last_fullsized":"2024-09-12T03:19:59.412050+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_clean_scrub_stamp":"2024-09-12T03:19:56.366974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:41:50.153852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.412261+0000","last_change":"2024-09-12T03:19:58.407764+0000","last_active":"2024-09-12T03:19:59.412261+0000","last_peered":"2024-09-12T03:19:59.412261+0000","last_clean":"2024-09-12T03:19:59.412261+0000","last_became_active":"2024-09-12T03:19:58.407483+0000","last_became_peered":"2024-09-12T03:19:58.407483+0000","last_unstale":"2024-09-12T03:19:59.412261+0000","last_undegraded":"2024-09-12T03:19:59.412261+0000","last_fullsized":"2024-09-12T03:19:59.412261+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:11:19.244120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.412443+0000","last_change":"2024-09-12T03:19:57.394831+0000","last_active":"2024-09-12T03:19:59.412443+0000","last_peered":"2024-09-12T03:19:59.412443+0000","last_clean":"2024-09-12T03:19:59.412443+0000","last_became_active":"2024-09-12T03:19:57.394505+0000","last_became_peered":"2024-09-12T03:19:57.394505+0000","last_unstale":"2024-09-12T03:19:59.412443+0000","last_undegraded":"2024-09-12T03:19:59.412443+0000","last_fullsized":"2024-09-12T03:19:59.412443+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_clean_scrub_stamp":"2024-09-12T03:19:56.366974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:59:35.357929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.411253+0000","last_change":"2024-09-12T03:19:58.408696+0000","last_active":"2024-09-12T03:19:59.411253+0000","last_peered":"2024-09-12T03:19:59.411253+0000","last_clean":"2024-09-12T03:19:59.411253+0000","last_became_active":"2024-09-12T03:19:58.408393+0000","last_became_peered":"2024-09-12T03:19:58.408393+0000","last_unstale":"2024-09-12T03:19:59.411253+0000","last_undegraded":"2024-09-12T03:19:59.411253+0000","last_fullsized":"2024-09-12T03:19:59.411253+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:15:02.240922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:19:56.366974+0000","last_change":"2024-09-12T03:19:56.366974+0000","last_active":"2024-09-12T03:19:56.366974+0000","last_peered":"2024-09-12T03:19:56.366974+0000","last_clean":"2024-09-12T03:19:56.366974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:56.366974+0000","last_undegraded":"2024-09-12T03:19:56.366974+0000","last_fullsized":"2024-09-12T03:19:56.366974+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_clean_scrub_stamp":"2024-09-12T03:19:56.366974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.411564+0000","last_change":"2024-09-12T03:19:58.409403+0000","last_active":"2024-09-12T03:19:59.411564+0000","last_peered":"2024-09-12T03:19:59.411564+0000","last_clean":"2024-09-12T03:19:59.411564+0000","last_became_active":"2024-09-12T03:19:58.409087+0000","last_became_peered":"2024-09-12T03:19:58.409087+0000","last_unstale":"2024-09-12T03:19:59.411564+0000","last_undegraded":"2024-09-12T03:19:59.411564+0000","last_fullsized":"2024-09-12T03:19:59.411564+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:44:15.824316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.398237+0000","last_change":"2024-09-12T03:19:57.394055+0000","last_active":"2024-09-12T03:19:59.398237+0000","last_peered":"2024-09-12T03:19:59.398237+0000","last_clean":"2024-09-12T03:19:59.398237+0000","last_became_active":"2024-09-12T03:19:57.393767+0000","last_became_peered":"2024-09-12T03:19:57.393767+0000","last_unstale":"2024-09-12T03:19:59.398237+0000","last_undegraded":"2024-09-12T03:19:59.398237+0000","last_fullsized":"2024-09-12T03:19:59.398237+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_clean_scrub_stamp":"2024-09-12T03:19:56.366974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:34:00.504515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.398615+0000","last_change":"2024-09-12T03:19:58.400064+0000","last_active":"2024-09-12T03:19:59.398615+0000","last_peered":"2024-09-12T03:19:59.398615+0000","last_clean":"2024-09-12T03:19:59.398615+0000","last_became_active":"2024-09-12T03:19:58.399777+0000","last_became_peered":"2024-09-12T03:19:58.399777+0000","last_unstale":"2024-09-12T03:19:59.398615+0000","last_undegraded":"2024-09-12T03:19:59.398615+0000","last_fullsized":"2024-09-12T03:19:59.398615+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:31:06.800562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"66'2","reported_seq":16,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.412883+0000","last_change":"2024-09-12T03:19:57.393966+0000","last_active":"2024-09-12T03:19:59.412883+0000","last_peered":"2024-09-12T03:19:59.412883+0000","last_clean":"2024-09-12T03:19:59.412883+0000","last_became_active":"2024-09-12T03:19:57.393680+0000","last_became_peered":"2024-09-12T03:19:57.393680+0000","last_unstale":"2024-09-12T03:19:59.412883+0000","last_undegraded":"2024-09-12T03:19:59.412883+0000","last_fullsized":"2024-09-12T03:19:59.412883+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_clean_scrub_stamp":"2024-09-12T03:19:56.366974+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:05:00.306420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:19:57.371227+0000","last_change":"2024-09-12T03:19:57.371227+0000","last_active":"2024-09-12T03:19:57.371227+0000","last_peered":"2024-09-12T03:19:57.371227+0000","last_clean":"2024-09-12T03:19:57.371227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:57.371227+0000","last_undegraded":"2024-09-12T03:19:57.371227+0000","last_fullsized":"2024-09-12T03:19:57.371227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:19:56.366974+0000","last_change":"2024-09-12T03:19:56.366974+0000","last_active":"2024-09-12T03:19:56.366974+0000","last_peered":"2024-09-12T03:19:56.366974+0000","last_clean":"2024-09-12T03:19:56.366974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:56.366974+0000","last_undegraded":"2024-09-12T03:19:56.366974+0000","last_fullsized":"2024-09-12T03:19:56.366974+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_clean_scrub_stamp":"2024-09-12T03:19:56.366974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.411946+0000","last_change":"2024-09-12T03:19:58.408958+0000","last_active":"2024-09-12T03:19:59.411946+0000","last_peered":"2024-09-12T03:19:59.411946+0000","last_clean":"2024-09-12T03:19:59.411946+0000","last_became_active":"2024-09-12T03:19:58.408665+0000","last_became_peered":"2024-09-12T03:19:58.408665+0000","last_unstale":"2024-09-12T03:19:59.411946+0000","last_undegraded":"2024-09-12T03:19:59.411946+0000","last_fullsized":"2024-09-12T03:19:59.411946+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:50:40.770697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.411911+0000","last_change":"2024-09-12T03:19:57.393827+0000","last_active":"2024-09-12T03:19:59.411911+0000","last_peered":"2024-09-12T03:19:59.411911+0000","last_clean":"2024-09-12T03:19:59.411911+0000","last_became_active":"2024-09-12T03:19:57.393501+0000","last_became_peered":"2024-09-12T03:19:57.393501+0000","last_unstale":"2024-09-12T03:19:59.411911+0000","last_undegraded":"2024-09-12T03:19:59.411911+0000","last_fullsized":"2024-09-12T03:19:59.411911+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_clean_scrub_stamp":"2024-09-12T03:19:56.366974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:00:49.820829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.412577+0000","last_change":"2024-09-12T03:19:58.416479+0000","last_active":"2024-09-12T03:19:59.412577+0000","last_peered":"2024-09-12T03:19:59.412577+0000","last_clean":"2024-09-12T03:19:59.412577+0000","last_became_active":"2024-09-12T03:19:58.416145+0000","last_became_peered":"2024-09-12T03:19:58.416145+0000","last_unstale":"2024-09-12T03:19:59.412577+0000","last_undegraded":"2024-09-12T03:19:59.412577+0000","last_fullsized":"2024-09-12T03:19:59.412577+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:37:31.629037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.398503+0000","last_change":"2024-09-12T03:19:57.392451+0000","last_active":"2024-09-12T03:19:59.398503+0000","last_peered":"2024-09-12T03:19:59.398503+0000","last_clean":"2024-09-12T03:19:59.398503+0000","last_became_active":"2024-09-12T03:19:57.392127+0000","last_became_peered":"2024-09-12T03:19:57.392127+0000","last_unstale":"2024-09-12T03:19:59.398503+0000","last_undegraded":"2024-09-12T03:19:59.398503+0000","last_fullsized":"2024-09-12T03:19:59.398503+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_clean_scrub_stamp":"2024-09-12T03:19:56.366974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:11:02.225246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.412704+0000","last_change":"2024-09-12T03:19:58.407515+0000","last_active":"2024-09-12T03:19:59.412704+0000","last_peered":"2024-09-12T03:19:59.412704+0000","last_clean":"2024-09-12T03:19:59.412704+0000","last_became_active":"2024-09-12T03:19:58.407189+0000","last_became_peered":"2024-09-12T03:19:58.407189+0000","last_unstale":"2024-09-12T03:19:59.412704+0000","last_undegraded":"2024-09-12T03:19:59.412704+0000","last_fullsized":"2024-09-12T03:19:59.412704+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:57:01.617936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.411502+0000","last_change":"2024-09-12T03:19:57.391167+0000","last_active":"2024-09-12T03:19:59.411502+0000","last_peered":"2024-09-12T03:19:59.411502+0000","last_clean":"2024-09-12T03:19:59.411502+0000","last_became_active":"2024-09-12T03:19:57.390901+0000","last_became_peered":"2024-09-12T03:19:57.390901+0000","last_unstale":"2024-09-12T03:19:59.411502+0000","last_undegraded":"2024-09-12T03:19:59.411502+0000","last_fullsized":"2024-09-12T03:19:59.411502+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_clean_scrub_stamp":"2024-09-12T03:19:56.366974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:09:37.761223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.412332+0000","last_change":"2024-09-12T03:19:58.420294+0000","last_active":"2024-09-12T03:19:59.412332+0000","last_peered":"2024-09-12T03:19:59.412332+0000","last_clean":"2024-09-12T03:19:59.412332+0000","last_became_active":"2024-09-12T03:19:58.418912+0000","last_became_peered":"2024-09-12T03:19:58.418912+0000","last_unstale":"2024-09-12T03:19:59.412332+0000","last_undegraded":"2024-09-12T03:19:59.412332+0000","last_fullsized":"2024-09-12T03:19:59.412332+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:14:39.136845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:19:56.366974+0000","last_change":"2024-09-12T03:19:56.366974+0000","last_active":"2024-09-12T03:19:56.366974+0000","last_peered":"2024-09-12T03:19:56.366974+0000","last_clean":"2024-09-12T03:19:56.366974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:56.366974+0000","last_undegraded":"2024-09-12T03:19:56.366974+0000","last_fullsized":"2024-09-12T03:19:56.366974+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_clean_scrub_stamp":"2024-09-12T03:19:56.366974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:19:57.371227+0000","last_change":"2024-09-12T03:19:57.371227+0000","last_active":"2024-09-12T03:19:57.371227+0000","last_peered":"2024-09-12T03:19:57.371227+0000","last_clean":"2024-09-12T03:19:57.371227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:57.371227+0000","last_undegraded":"2024-09-12T03:19:57.371227+0000","last_fullsized":"2024-09-12T03:19:57.371227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.399794+0000","last_change":"2024-09-12T03:19:57.393150+0000","last_active":"2024-09-12T03:19:59.399794+0000","last_peered":"2024-09-12T03:19:59.399794+0000","last_clean":"2024-09-12T03:19:59.399794+0000","last_became_active":"2024-09-12T03:19:57.392692+0000","last_became_peered":"2024-09-12T03:19:57.392692+0000","last_unstale":"2024-09-12T03:19:59.399794+0000","last_undegraded":"2024-09-12T03:19:59.399794+0000","last_fullsized":"2024-09-12T03:19:59.399794+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_clean_scrub_stamp":"2024-09-12T03:19:56.366974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:44:27.441871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.411918+0000","last_change":"2024-09-12T03:19:58.407787+0000","last_active":"2024-09-12T03:19:59.411918+0000","last_peered":"2024-09-12T03:19:59.411918+0000","last_clean":"2024-09-12T03:19:59.411918+0000","last_became_active":"2024-09-12T03:19:58.407407+0000","last_became_peered":"2024-09-12T03:19:58.407407+0000","last_unstale":"2024-09-12T03:19:59.411918+0000","last_undegraded":"2024-09-12T03:19:59.411918+0000","last_fullsized":"2024-09-12T03:19:59.411918+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:38:14.768525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.412289+0000","last_change":"2024-09-12T03:19:57.391520+0000","last_active":"2024-09-12T03:19:59.412289+0000","last_peered":"2024-09-12T03:19:59.412289+0000","last_clean":"2024-09-12T03:19:59.412289+0000","last_became_active":"2024-09-12T03:19:57.391140+0000","last_became_peered":"2024-09-12T03:19:57.391140+0000","last_unstale":"2024-09-12T03:19:59.412289+0000","last_undegraded":"2024-09-12T03:19:59.412289+0000","last_fullsized":"2024-09-12T03:19:59.412289+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_clean_scrub_stamp":"2024-09-12T03:19:56.366974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:42:32.285383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.412286+0000","last_change":"2024-09-12T03:19:58.409818+0000","last_active":"2024-09-12T03:19:59.412286+0000","last_peered":"2024-09-12T03:19:59.412286+0000","last_clean":"2024-09-12T03:19:59.412286+0000","last_became_active":"2024-09-12T03:19:58.409507+0000","last_became_peered":"2024-09-12T03:19:58.409507+0000","last_unstale":"2024-09-12T03:19:59.412286+0000","last_undegraded":"2024-09-12T03:19:59.412286+0000","last_fullsized":"2024-09-12T03:19:59.412286+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:21:24.731904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"66'2","reported_seq":16,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.412444+0000","last_change":"2024-09-12T03:19:57.395758+0000","last_active":"2024-09-12T03:19:59.412444+0000","last_peered":"2024-09-12T03:19:59.412444+0000","last_clean":"2024-09-12T03:19:59.412444+0000","last_became_active":"2024-09-12T03:19:57.395422+0000","last_became_peered":"2024-09-12T03:19:57.395422+0000","last_unstale":"2024-09-12T03:19:59.412444+0000","last_undegraded":"2024-09-12T03:19:59.412444+0000","last_fullsized":"2024-09-12T03:19:59.412444+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_clean_scrub_stamp":"2024-09-12T03:19:56.366974+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:18:13.412357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":252,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.411416+0000","last_change":"2024-09-12T03:16:21.883446+0000","last_active":"2024-09-12T03:19:59.411416+0000","last_peered":"2024-09-12T03:19:59.411416+0000","last_clean":"2024-09-12T03:19:59.411416+0000","last_became_active":"2024-09-12T03:16:21.882207+0000","last_became_peered":"2024-09-12T03:16:21.882207+0000","last_unstale":"2024-09-12T03:19:59.411416+0000","last_undegraded":"2024-09-12T03:19:59.411416+0000","last_fullsized":"2024-09-12T03:19:59.411416+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:20.316426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:20.316426+0000","last_clean_scrub_stamp":"2024-09-12T03:16:20.316426+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:12:15.424320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":102,"acting":102,"num_store_stats":6},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":764,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":35,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":30,"ondisk_log_size":30,"up":50,"acting":50,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542190,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36716,"kb_used_data":1368,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335124,"statfs":{"total":96636764160,"available":96599166976,"internally_reserved":0,"allocated":1400832,"data_stored":924039,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542192,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31996,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339844,"statfs":{"total":96636764160,"available":96604000256,"internally_reserved":0,"allocated":819200,"data_stored":339447,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542190,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":36104,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335736,"statfs":{"total":96636764160,"available":96599793664,"internally_reserved":0,"allocated":761856,"data_stored":314188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542190,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31980,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339860,"statfs":{"total":96636764160,"available":96604016640,"internally_reserved":0,"allocated":802816,"data_stored":333125,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542191,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27836,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344004,"statfs":{"total":96636764160,"available":96608260096,"internally_reserved":0,"allocated":819200,"data_stored":339467,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542191,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27840,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344000,"statfs":{"total":96636764160,"available":96608256000,"internally_reserved":0,"allocated":823296,"data_stored":340967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542191,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31936,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339904,"statfs":{"total":96636764160,"available":96604061696,"internally_reserved":0,"allocated":761856,"data_stored":314188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542192,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28396,"kb_used_data":1360,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343444,"statfs":{"total":96636764160,"available":96607686656,"internally_reserved":0,"allocated":1392640,"data_stored":923423,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T03:20:01.784 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-12T03:20:01.888 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.887+0000 7f589c78d640 1 Processor -- start 2024-09-12T03:20:01.888 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.887+0000 7f589c78d640 1 -- start start 2024-09-12T03:20:01.889 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.888+0000 7f589c78d640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f589406f360 0x7f58940cef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:01.890 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.888+0000 7f589c78d640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f58940cf490 0x7f58940d2780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:01.890 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.888+0000 7f589c78d640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f589406cc40 con 0x7f58940b1ca0 2024-09-12T03:20:01.890 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.888+0000 7f589c78d640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f58940cf900 con 0x7f58940cf490 2024-09-12T03:20:01.890 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.888+0000 7f589c78d640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f58940d22f0 con 0x7f589406f360 2024-09-12T03:20:01.890 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.888+0000 7f589ad2a640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f58940cf490 0x7f58940d2780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:01.890 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.888+0000 7f589a529640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f58940b1ca0 0x7f589415f830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:34856/0 (socket says 172.21.15.27:34856) 2024-09-12T03:20:01.890 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.888+0000 7f589a529640 1 -- 172.21.15.27:0/465603057 learned_addr learned my addr 172.21.15.27:0/465603057 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:20:01.890 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.888+0000 7f5899d28640 1 --2- 172.21.15.27:0/465603057 >> v2:172.21.15.45:3300/0 conn(0x7f589406f360 0x7f58940cef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:01.890 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.889+0000 7f589ad2a640 1 -- 172.21.15.27:0/465603057 >> v2:172.21.15.45:3300/0 conn(0x7f589406f360 msgr2=0x7f58940cef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:01.890 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.889+0000 7f589ad2a640 1 --2- 172.21.15.27:0/465603057 >> v2:172.21.15.45:3300/0 conn(0x7f589406f360 0x7f58940cef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:01.890 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.889+0000 7f589ad2a640 1 -- 172.21.15.27:0/465603057 >> v1:172.21.15.27:6789/0 conn(0x7f58940b1ca0 legacy=0x7f589415f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:01.891 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.889+0000 7f589ad2a640 1 -- 172.21.15.27:0/465603057 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58940d1ce0 con 0x7f58940cf490 2024-09-12T03:20:01.892 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.889+0000 7f589ad2a640 1 --2- 172.21.15.27:0/465603057 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f58940cf490 0x7f58940d2780 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f58940b5220 tx=0x7f588c02a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4002758842dbd991 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:01.892 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.889+0000 7f5899527640 1 -- 172.21.15.27:0/465603057 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f588c007740 con 0x7f58940cf490 2024-09-12T03:20:01.892 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.889+0000 7f5899527640 1 -- 172.21.15.27:0/465603057 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f588c0078e0 con 0x7f58940cf490 2024-09-12T03:20:01.892 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.889+0000 7f5899527640 1 -- 172.21.15.27:0/465603057 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f588c007cb0 con 0x7f58940cf490 2024-09-12T03:20:01.892 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.890+0000 7f589c78d640 1 -- 172.21.15.27:0/465603057 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f58940cf490 msgr2=0x7f58940d2780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:01.892 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.890+0000 7f589c78d640 1 --2- 172.21.15.27:0/465603057 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f58940cf490 0x7f58940d2780 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f58940b5220 tx=0x7f588c02a680 comp rx=0 tx=0).stop 2024-09-12T03:20:01.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.890+0000 7f589c78d640 1 -- 172.21.15.27:0/465603057 shutdown_connections 2024-09-12T03:20:01.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.890+0000 7f589c78d640 1 --2- 172.21.15.27:0/465603057 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f58940cf490 0x7f58940d2780 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:01.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.890+0000 7f589c78d640 1 --2- 172.21.15.27:0/465603057 >> v2:172.21.15.45:3300/0 conn(0x7f589406f360 0x7f58940cef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:01.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.890+0000 7f589c78d640 1 -- 172.21.15.27:0/465603057 >> 172.21.15.27:0/465603057 conn(0x7f5894073800 msgr2=0x7f589409ba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:01.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.890+0000 7f589c78d640 1 -- 172.21.15.27:0/465603057 shutdown_connections 2024-09-12T03:20:01.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.890+0000 7f589c78d640 1 -- 172.21.15.27:0/465603057 wait complete. 2024-09-12T03:20:01.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.891+0000 7f589c78d640 1 Processor -- start 2024-09-12T03:20:01.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.891+0000 7f589c78d640 1 -- start start 2024-09-12T03:20:01.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.892+0000 7f589c78d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f589406f360 0x7f58940b75c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:01.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.892+0000 7f589c78d640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f58940b1ca0 0x7f589414e080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:01.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.892+0000 7f589c78d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f58940cf490 0x7f58940cd450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:01.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.892+0000 7f589c78d640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f589406d1c0 con 0x7f589406f360 2024-09-12T03:20:01.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.892+0000 7f589c78d640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f58940d1f80 con 0x7f58940cf490 2024-09-12T03:20:01.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.892+0000 7f589c78d640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f589406d340 con 0x7f58940b1ca0 2024-09-12T03:20:01.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.892+0000 7f589a529640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f589406f360 0x7f58940b75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:01.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.892+0000 7f589a529640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f589406f360 0x7f58940b75c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56226/0 (socket says 172.21.15.27:56226) 2024-09-12T03:20:01.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.892+0000 7f589a529640 1 -- 172.21.15.27:0/2039700238 learned_addr learned my addr 172.21.15.27:0/2039700238 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:20:01.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.892+0000 7f5899d28640 1 --2- 172.21.15.27:0/2039700238 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f58940b1ca0 0x7f589414e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:01.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.892+0000 7f589ad2a640 1 --2- 172.21.15.27:0/2039700238 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f58940cf490 0x7f58940cd450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:01.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.892+0000 7f589a529640 1 -- 172.21.15.27:0/2039700238 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f58940b1ca0 msgr2=0x7f589414e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:01.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.892+0000 7f589a529640 1 --2- 172.21.15.27:0/2039700238 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f58940b1ca0 0x7f589414e080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:01.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.892+0000 7f589a529640 1 -- 172.21.15.27:0/2039700238 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f58940cf490 msgr2=0x7f58940cd450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:01.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.892+0000 7f589a529640 1 --2- 172.21.15.27:0/2039700238 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f58940cf490 0x7f58940cd450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:01.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.892+0000 7f589a529640 1 -- 172.21.15.27:0/2039700238 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5894160850 con 0x7f589406f360 2024-09-12T03:20:01.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.892+0000 7f589ad2a640 1 --2- 172.21.15.27:0/2039700238 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f58940cf490 0x7f58940cd450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:20:01.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.893+0000 7f589a529640 1 --2- 172.21.15.27:0/2039700238 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f589406f360 0x7f58940b75c0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f589000c400 tx=0x7f589000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:01.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.893+0000 7f588affd640 1 -- 172.21.15.27:0/2039700238 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5890017070 con 0x7f589406f360 2024-09-12T03:20:01.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.893+0000 7f588affd640 1 -- 172.21.15.27:0/2039700238 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5890005ce0 con 0x7f589406f360 2024-09-12T03:20:01.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.893+0000 7f589c78d640 1 -- 172.21.15.27:0/2039700238 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58940cbbf0 con 0x7f589406f360 2024-09-12T03:20:01.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.893+0000 7f588affd640 1 -- 172.21.15.27:0/2039700238 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58900072f0 con 0x7f589406f360 2024-09-12T03:20:01.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.893+0000 7f589c78d640 1 -- 172.21.15.27:0/2039700238 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58940cb850 con 0x7f589406f360 2024-09-12T03:20:01.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.894+0000 7f589c78d640 1 -- 172.21.15.27:0/2039700238 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58940cfc40 con 0x7f589406f360 2024-09-12T03:20:01.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.894+0000 7f588affd640 1 -- 172.21.15.27:0/2039700238 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 11) ==== 90804+0+0 (secure 0 0 0) 0x7f589001a070 con 0x7f589406f360 2024-09-12T03:20:01.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.895+0000 7f588affd640 1 --2- 172.21.15.27:0/2039700238 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f587806c1c0 0x7f587806e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:01.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.895+0000 7f588affd640 1 -- 172.21.15.27:0/2039700238 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 5709+0+0 (secure 0 0 0) 0x7f589008a710 con 0x7f589406f360 2024-09-12T03:20:01.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.895+0000 7f5899d28640 1 --2- 172.21.15.27:0/2039700238 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f587806c1c0 0x7f587806e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:01.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.896+0000 7f5899d28640 1 --2- 172.21.15.27:0/2039700238 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f587806c1c0 0x7f587806e680 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f58941f7b40 tx=0x7f5884008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:01.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:01.899+0000 7f588affd640 1 -- 172.21.15.27:0/2039700238 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f5890058d80 con 0x7f589406f360 2024-09-12T03:20:02.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:02.074+0000 7f589c78d640 1 -- 172.21.15.27:0/2039700238 --> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f58940d1130 con 0x7f587806c1c0 2024-09-12T03:20:02.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:02.080+0000 7f588affd640 1 -- 172.21.15.27:0/2039700238 <== mgr.4102 v2:172.21.15.45:6832/3923821219 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+248831 (secure 0 0 0) 0x7f58940d1130 con 0x7f587806c1c0 2024-09-12T03:20:02.084 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:20:02.085 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2024-09-12T03:20:02.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:02.083+0000 7f589c78d640 1 -- 172.21.15.27:0/2039700238 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f587806c1c0 msgr2=0x7f587806e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:02.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:02.083+0000 7f589c78d640 1 --2- 172.21.15.27:0/2039700238 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f587806c1c0 0x7f587806e680 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f58941f7b40 tx=0x7f5884008040 comp rx=0 tx=0).stop 2024-09-12T03:20:02.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:02.083+0000 7f589c78d640 1 -- 172.21.15.27:0/2039700238 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f589406f360 msgr2=0x7f58940b75c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:02.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:02.083+0000 7f589c78d640 1 --2- 172.21.15.27:0/2039700238 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f589406f360 0x7f58940b75c0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f589000c400 tx=0x7f589000c8c0 comp rx=0 tx=0).stop 2024-09-12T03:20:02.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:02.083+0000 7f589c78d640 1 -- 172.21.15.27:0/2039700238 shutdown_connections 2024-09-12T03:20:02.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:02.083+0000 7f589c78d640 1 --2- 172.21.15.27:0/2039700238 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f587806c1c0 0x7f587806e680 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:02.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:02.083+0000 7f589c78d640 1 --2- 172.21.15.27:0/2039700238 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f58940cf490 0x7f58940cd450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:02.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:02.083+0000 7f589c78d640 1 --2- 172.21.15.27:0/2039700238 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f58940b1ca0 0x7f589414e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:02.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:02.083+0000 7f589c78d640 1 --2- 172.21.15.27:0/2039700238 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f589406f360 0x7f58940b75c0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:02.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:02.083+0000 7f589c78d640 1 -- 172.21.15.27:0/2039700238 >> 172.21.15.27:0/2039700238 conn(0x7f5894073800 msgr2=0x7f58940d2780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:02.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:02.084+0000 7f589c78d640 1 -- 172.21.15.27:0/2039700238 shutdown_connections 2024-09-12T03:20:02.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:02.084+0000 7f589c78d640 1 -- 172.21.15.27:0/2039700238 wait complete. 2024-09-12T03:20:02.095 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":170,"stamp":"2024-09-12T03:20:00.103531+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":591132,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":194,"num_write_kb":2420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":147,"ondisk_log_size":147,"up":154,"acting":154,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":100,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":6,"kb":754974720,"kb_used":252804,"kb_used_data":7404,"kb_used_omap":12,"kb_used_meta":245171,"kb_avail":754721916,"statfs":{"total":773094113280,"available":772835241984,"internally_reserved":0,"allocated":7581696,"data_stored":3828844,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":251055664},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":764,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":35,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001083"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.412295+0000","last_change":"2024-09-12T03:19:58.405738+0000","last_active":"2024-09-12T03:19:59.412295+0000","last_peered":"2024-09-12T03:19:59.412295+0000","last_clean":"2024-09-12T03:19:59.412295+0000","last_became_active":"2024-09-12T03:19:58.405044+0000","last_became_peered":"2024-09-12T03:19:58.405044+0000","last_unstale":"2024-09-12T03:19:59.412295+0000","last_undegraded":"2024-09-12T03:19:59.412295+0000","last_fullsized":"2024-09-12T03:19:59.412295+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:24:50.892593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.412848+0000","last_change":"2024-09-12T03:19:58.407057+0000","last_active":"2024-09-12T03:19:59.412848+0000","last_peered":"2024-09-12T03:19:59.412848+0000","last_clean":"2024-09-12T03:19:59.412848+0000","last_became_active":"2024-09-12T03:19:58.406455+0000","last_became_peered":"2024-09-12T03:19:58.406455+0000","last_unstale":"2024-09-12T03:19:59.412848+0000","last_undegraded":"2024-09-12T03:19:59.412848+0000","last_fullsized":"2024-09-12T03:19:59.412848+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:12:06.463949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.411816+0000","last_change":"2024-09-12T03:19:58.402689+0000","last_active":"2024-09-12T03:19:59.411816+0000","last_peered":"2024-09-12T03:19:59.411816+0000","last_clean":"2024-09-12T03:19:59.411816+0000","last_became_active":"2024-09-12T03:19:58.402424+0000","last_became_peered":"2024-09-12T03:19:58.402424+0000","last_unstale":"2024-09-12T03:19:59.411816+0000","last_undegraded":"2024-09-12T03:19:59.411816+0000","last_fullsized":"2024-09-12T03:19:59.411816+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:01:53.466856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:19:57.371227+0000","last_change":"2024-09-12T03:19:57.371227+0000","last_active":"2024-09-12T03:19:57.371227+0000","last_peered":"2024-09-12T03:19:57.371227+0000","last_clean":"2024-09-12T03:19:57.371227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:57.371227+0000","last_undegraded":"2024-09-12T03:19:57.371227+0000","last_fullsized":"2024-09-12T03:19:57.371227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.411395+0000","last_change":"2024-09-12T03:19:58.405012+0000","last_active":"2024-09-12T03:19:59.411395+0000","last_peered":"2024-09-12T03:19:59.411395+0000","last_clean":"2024-09-12T03:19:59.411395+0000","last_became_active":"2024-09-12T03:19:58.404706+0000","last_became_peered":"2024-09-12T03:19:58.404706+0000","last_unstale":"2024-09-12T03:19:59.411395+0000","last_undegraded":"2024-09-12T03:19:59.411395+0000","last_fullsized":"2024-09-12T03:19:59.411395+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:13:43.693628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.412119+0000","last_change":"2024-09-12T03:19:58.404152+0000","last_active":"2024-09-12T03:19:59.412119+0000","last_peered":"2024-09-12T03:19:59.412119+0000","last_clean":"2024-09-12T03:19:59.412119+0000","last_became_active":"2024-09-12T03:19:58.403865+0000","last_became_peered":"2024-09-12T03:19:58.403865+0000","last_unstale":"2024-09-12T03:19:59.412119+0000","last_undegraded":"2024-09-12T03:19:59.412119+0000","last_fullsized":"2024-09-12T03:19:59.412119+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:03:35.086339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:19:57.371227+0000","last_change":"2024-09-12T03:19:57.371227+0000","last_active":"2024-09-12T03:19:57.371227+0000","last_peered":"2024-09-12T03:19:57.371227+0000","last_clean":"2024-09-12T03:19:57.371227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:57.371227+0000","last_undegraded":"2024-09-12T03:19:57.371227+0000","last_fullsized":"2024-09-12T03:19:57.371227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.412001+0000","last_change":"2024-09-12T03:19:58.408692+0000","last_active":"2024-09-12T03:19:59.412001+0000","last_peered":"2024-09-12T03:19:59.412001+0000","last_clean":"2024-09-12T03:19:59.412001+0000","last_became_active":"2024-09-12T03:19:58.408182+0000","last_became_peered":"2024-09-12T03:19:58.408182+0000","last_unstale":"2024-09-12T03:19:59.412001+0000","last_undegraded":"2024-09-12T03:19:59.412001+0000","last_fullsized":"2024-09-12T03:19:59.412001+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:25:56.007148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.412647+0000","last_change":"2024-09-12T03:19:58.408054+0000","last_active":"2024-09-12T03:19:59.412647+0000","last_peered":"2024-09-12T03:19:59.412647+0000","last_clean":"2024-09-12T03:19:59.412647+0000","last_became_active":"2024-09-12T03:19:58.407439+0000","last_became_peered":"2024-09-12T03:19:58.407439+0000","last_unstale":"2024-09-12T03:19:59.412647+0000","last_undegraded":"2024-09-12T03:19:59.412647+0000","last_fullsized":"2024-09-12T03:19:59.412647+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:52:55.350315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.399858+0000","last_change":"2024-09-12T03:19:58.400878+0000","last_active":"2024-09-12T03:19:59.399858+0000","last_peered":"2024-09-12T03:19:59.399858+0000","last_clean":"2024-09-12T03:19:59.399858+0000","last_became_active":"2024-09-12T03:19:58.400601+0000","last_became_peered":"2024-09-12T03:19:58.400601+0000","last_unstale":"2024-09-12T03:19:59.399858+0000","last_undegraded":"2024-09-12T03:19:59.399858+0000","last_fullsized":"2024-09-12T03:19:59.399858+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:53:33.025355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.412388+0000","last_change":"2024-09-12T03:19:58.404349+0000","last_active":"2024-09-12T03:19:59.412388+0000","last_peered":"2024-09-12T03:19:59.412388+0000","last_clean":"2024-09-12T03:19:59.412388+0000","last_became_active":"2024-09-12T03:19:58.404067+0000","last_became_peered":"2024-09-12T03:19:58.404067+0000","last_unstale":"2024-09-12T03:19:59.412388+0000","last_undegraded":"2024-09-12T03:19:59.412388+0000","last_fullsized":"2024-09-12T03:19:59.412388+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:31:06.800562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:19:57.371227+0000","last_change":"2024-09-12T03:19:57.371227+0000","last_active":"2024-09-12T03:19:57.371227+0000","last_peered":"2024-09-12T03:19:57.371227+0000","last_clean":"2024-09-12T03:19:57.371227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:57.371227+0000","last_undegraded":"2024-09-12T03:19:57.371227+0000","last_fullsized":"2024-09-12T03:19:57.371227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.398467+0000","last_change":"2024-09-12T03:19:58.408147+0000","last_active":"2024-09-12T03:19:59.398467+0000","last_peered":"2024-09-12T03:19:59.398467+0000","last_clean":"2024-09-12T03:19:59.398467+0000","last_became_active":"2024-09-12T03:19:58.407855+0000","last_became_peered":"2024-09-12T03:19:58.407855+0000","last_unstale":"2024-09-12T03:19:59.398467+0000","last_undegraded":"2024-09-12T03:19:59.398467+0000","last_fullsized":"2024-09-12T03:19:59.398467+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:50:40.770697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.412570+0000","last_change":"2024-09-12T03:19:58.401979+0000","last_active":"2024-09-12T03:19:59.412570+0000","last_peered":"2024-09-12T03:19:59.412570+0000","last_clean":"2024-09-12T03:19:59.412570+0000","last_became_active":"2024-09-12T03:19:58.401689+0000","last_became_peered":"2024-09-12T03:19:58.401689+0000","last_unstale":"2024-09-12T03:19:59.412570+0000","last_undegraded":"2024-09-12T03:19:59.412570+0000","last_fullsized":"2024-09-12T03:19:59.412570+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:22:57.280328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.398306+0000","last_change":"2024-09-12T03:19:58.399312+0000","last_active":"2024-09-12T03:19:59.398306+0000","last_peered":"2024-09-12T03:19:59.398306+0000","last_clean":"2024-09-12T03:19:59.398306+0000","last_became_active":"2024-09-12T03:19:58.399050+0000","last_became_peered":"2024-09-12T03:19:58.399050+0000","last_unstale":"2024-09-12T03:19:59.398306+0000","last_undegraded":"2024-09-12T03:19:59.398306+0000","last_fullsized":"2024-09-12T03:19:59.398306+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:41:13.177460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.400875+0000","last_change":"2024-09-12T03:19:58.404580+0000","last_active":"2024-09-12T03:19:59.400875+0000","last_peered":"2024-09-12T03:19:59.400875+0000","last_clean":"2024-09-12T03:19:59.400875+0000","last_became_active":"2024-09-12T03:19:58.404320+0000","last_became_peered":"2024-09-12T03:19:58.404320+0000","last_unstale":"2024-09-12T03:19:59.400875+0000","last_undegraded":"2024-09-12T03:19:59.400875+0000","last_fullsized":"2024-09-12T03:19:59.400875+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:41:51.158105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.412705+0000","last_change":"2024-09-12T03:19:58.408389+0000","last_active":"2024-09-12T03:19:59.412705+0000","last_peered":"2024-09-12T03:19:59.412705+0000","last_clean":"2024-09-12T03:19:59.412705+0000","last_became_active":"2024-09-12T03:19:58.407797+0000","last_became_peered":"2024-09-12T03:19:58.407797+0000","last_unstale":"2024-09-12T03:19:59.412705+0000","last_undegraded":"2024-09-12T03:19:59.412705+0000","last_fullsized":"2024-09-12T03:19:59.412705+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:17:24.698351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:19:57.371227+0000","last_change":"2024-09-12T03:19:57.371227+0000","last_active":"2024-09-12T03:19:57.371227+0000","last_peered":"2024-09-12T03:19:57.371227+0000","last_clean":"2024-09-12T03:19:57.371227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:57.371227+0000","last_undegraded":"2024-09-12T03:19:57.371227+0000","last_fullsized":"2024-09-12T03:19:57.371227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.412173+0000","last_change":"2024-09-12T03:19:58.408115+0000","last_active":"2024-09-12T03:19:59.412173+0000","last_peered":"2024-09-12T03:19:59.412173+0000","last_clean":"2024-09-12T03:19:59.412173+0000","last_became_active":"2024-09-12T03:19:58.407745+0000","last_became_peered":"2024-09-12T03:19:58.407745+0000","last_unstale":"2024-09-12T03:19:59.412173+0000","last_undegraded":"2024-09-12T03:19:59.412173+0000","last_fullsized":"2024-09-12T03:19:59.412173+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:50:17.515387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.398554+0000","last_change":"2024-09-12T03:19:58.398467+0000","last_active":"2024-09-12T03:19:59.398554+0000","last_peered":"2024-09-12T03:19:59.398554+0000","last_clean":"2024-09-12T03:19:59.398554+0000","last_became_active":"2024-09-12T03:19:58.398152+0000","last_became_peered":"2024-09-12T03:19:58.398152+0000","last_unstale":"2024-09-12T03:19:59.398554+0000","last_undegraded":"2024-09-12T03:19:59.398554+0000","last_fullsized":"2024-09-12T03:19:59.398554+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:24:50.892593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.400458+0000","last_change":"2024-09-12T03:19:58.400541+0000","last_active":"2024-09-12T03:19:59.400458+0000","last_peered":"2024-09-12T03:19:59.400458+0000","last_clean":"2024-09-12T03:19:59.400458+0000","last_became_active":"2024-09-12T03:19:58.399602+0000","last_became_peered":"2024-09-12T03:19:58.399602+0000","last_unstale":"2024-09-12T03:19:59.400458+0000","last_undegraded":"2024-09-12T03:19:59.400458+0000","last_fullsized":"2024-09-12T03:19:59.400458+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:56:37.761014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.400370+0000","last_change":"2024-09-12T03:19:58.399427+0000","last_active":"2024-09-12T03:19:59.400370+0000","last_peered":"2024-09-12T03:19:59.400370+0000","last_clean":"2024-09-12T03:19:59.400370+0000","last_became_active":"2024-09-12T03:19:58.398464+0000","last_became_peered":"2024-09-12T03:19:58.398464+0000","last_unstale":"2024-09-12T03:19:59.400370+0000","last_undegraded":"2024-09-12T03:19:59.400370+0000","last_fullsized":"2024-09-12T03:19:59.400370+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:44:19.209045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.411588+0000","last_change":"2024-09-12T03:19:58.405456+0000","last_active":"2024-09-12T03:19:59.411588+0000","last_peered":"2024-09-12T03:19:59.411588+0000","last_clean":"2024-09-12T03:19:59.411588+0000","last_became_active":"2024-09-12T03:19:58.404974+0000","last_became_peered":"2024-09-12T03:19:58.404974+0000","last_unstale":"2024-09-12T03:19:59.411588+0000","last_undegraded":"2024-09-12T03:19:59.411588+0000","last_fullsized":"2024-09-12T03:19:59.411588+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:58:58.914180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.411477+0000","last_change":"2024-09-12T03:19:58.408893+0000","last_active":"2024-09-12T03:19:59.411477+0000","last_peered":"2024-09-12T03:19:59.411477+0000","last_clean":"2024-09-12T03:19:59.411477+0000","last_became_active":"2024-09-12T03:19:58.408589+0000","last_became_peered":"2024-09-12T03:19:58.408589+0000","last_unstale":"2024-09-12T03:19:59.411477+0000","last_undegraded":"2024-09-12T03:19:59.411477+0000","last_fullsized":"2024-09-12T03:19:59.411477+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:54:48.020501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.400037+0000","last_change":"2024-09-12T03:19:58.411515+0000","last_active":"2024-09-12T03:19:59.400037+0000","last_peered":"2024-09-12T03:19:59.400037+0000","last_clean":"2024-09-12T03:19:59.400037+0000","last_became_active":"2024-09-12T03:19:58.409091+0000","last_became_peered":"2024-09-12T03:19:58.409091+0000","last_unstale":"2024-09-12T03:19:59.400037+0000","last_undegraded":"2024-09-12T03:19:59.400037+0000","last_fullsized":"2024-09-12T03:19:59.400037+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:25:42.004151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:19:57.371227+0000","last_change":"2024-09-12T03:19:57.371227+0000","last_active":"2024-09-12T03:19:57.371227+0000","last_peered":"2024-09-12T03:19:57.371227+0000","last_clean":"2024-09-12T03:19:57.371227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:57.371227+0000","last_undegraded":"2024-09-12T03:19:57.371227+0000","last_fullsized":"2024-09-12T03:19:57.371227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.411966+0000","last_change":"2024-09-12T03:19:58.406104+0000","last_active":"2024-09-12T03:19:59.411966+0000","last_peered":"2024-09-12T03:19:59.411966+0000","last_clean":"2024-09-12T03:19:59.411966+0000","last_became_active":"2024-09-12T03:19:58.405546+0000","last_became_peered":"2024-09-12T03:19:58.405546+0000","last_unstale":"2024-09-12T03:19:59.411966+0000","last_undegraded":"2024-09-12T03:19:59.411966+0000","last_fullsized":"2024-09-12T03:19:59.411966+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:45:16.630721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.400401+0000","last_change":"2024-09-12T03:19:58.401764+0000","last_active":"2024-09-12T03:19:59.400401+0000","last_peered":"2024-09-12T03:19:59.400401+0000","last_clean":"2024-09-12T03:19:59.400401+0000","last_became_active":"2024-09-12T03:19:58.397972+0000","last_became_peered":"2024-09-12T03:19:58.397972+0000","last_unstale":"2024-09-12T03:19:59.400401+0000","last_undegraded":"2024-09-12T03:19:59.400401+0000","last_fullsized":"2024-09-12T03:19:59.400401+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:50:11.761046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.412120+0000","last_change":"2024-09-12T03:19:58.403549+0000","last_active":"2024-09-12T03:19:59.412120+0000","last_peered":"2024-09-12T03:19:59.412120+0000","last_clean":"2024-09-12T03:19:59.412120+0000","last_became_active":"2024-09-12T03:19:58.403294+0000","last_became_peered":"2024-09-12T03:19:58.403294+0000","last_unstale":"2024-09-12T03:19:59.412120+0000","last_undegraded":"2024-09-12T03:19:59.412120+0000","last_fullsized":"2024-09-12T03:19:59.412120+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:20:56.214083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:19:57.371227+0000","last_change":"2024-09-12T03:19:57.371227+0000","last_active":"2024-09-12T03:19:57.371227+0000","last_peered":"2024-09-12T03:19:57.371227+0000","last_clean":"2024-09-12T03:19:57.371227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:57.371227+0000","last_undegraded":"2024-09-12T03:19:57.371227+0000","last_fullsized":"2024-09-12T03:19:57.371227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:19:57.371227+0000","last_change":"2024-09-12T03:19:57.371227+0000","last_active":"2024-09-12T03:19:57.371227+0000","last_peered":"2024-09-12T03:19:57.371227+0000","last_clean":"2024-09-12T03:19:57.371227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:57.371227+0000","last_undegraded":"2024-09-12T03:19:57.371227+0000","last_fullsized":"2024-09-12T03:19:57.371227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.412055+0000","last_change":"2024-09-12T03:19:58.407063+0000","last_active":"2024-09-12T03:19:59.412055+0000","last_peered":"2024-09-12T03:19:59.412055+0000","last_clean":"2024-09-12T03:19:59.412055+0000","last_became_active":"2024-09-12T03:19:58.406116+0000","last_became_peered":"2024-09-12T03:19:58.406116+0000","last_unstale":"2024-09-12T03:19:59.412055+0000","last_undegraded":"2024-09-12T03:19:59.412055+0000","last_fullsized":"2024-09-12T03:19:59.412055+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:11:39.227764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.411730+0000","last_change":"2024-09-12T03:19:58.409587+0000","last_active":"2024-09-12T03:19:59.411730+0000","last_peered":"2024-09-12T03:19:59.411730+0000","last_clean":"2024-09-12T03:19:59.411730+0000","last_became_active":"2024-09-12T03:19:58.409275+0000","last_became_peered":"2024-09-12T03:19:58.409275+0000","last_unstale":"2024-09-12T03:19:59.411730+0000","last_undegraded":"2024-09-12T03:19:59.411730+0000","last_fullsized":"2024-09-12T03:19:59.411730+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:00:46.656402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"66'4","reported_seq":18,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.412596+0000","last_change":"2024-09-12T03:19:57.394595+0000","last_active":"2024-09-12T03:19:59.412596+0000","last_peered":"2024-09-12T03:19:59.412596+0000","last_clean":"2024-09-12T03:19:59.412596+0000","last_became_active":"2024-09-12T03:19:57.394294+0000","last_became_peered":"2024-09-12T03:19:57.394294+0000","last_unstale":"2024-09-12T03:19:59.412596+0000","last_undegraded":"2024-09-12T03:19:59.412596+0000","last_fullsized":"2024-09-12T03:19:59.412596+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_clean_scrub_stamp":"2024-09-12T03:19:56.366974+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:51:58.062647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.412771+0000","last_change":"2024-09-12T03:19:58.407115+0000","last_active":"2024-09-12T03:19:59.412771+0000","last_peered":"2024-09-12T03:19:59.412771+0000","last_clean":"2024-09-12T03:19:59.412771+0000","last_became_active":"2024-09-12T03:19:58.406683+0000","last_became_peered":"2024-09-12T03:19:58.406683+0000","last_unstale":"2024-09-12T03:19:59.412771+0000","last_undegraded":"2024-09-12T03:19:59.412771+0000","last_fullsized":"2024-09-12T03:19:59.412771+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:36:57.331135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"66'1","reported_seq":15,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.398595+0000","last_change":"2024-09-12T03:19:57.389982+0000","last_active":"2024-09-12T03:19:59.398595+0000","last_peered":"2024-09-12T03:19:59.398595+0000","last_clean":"2024-09-12T03:19:59.398595+0000","last_became_active":"2024-09-12T03:19:57.389694+0000","last_became_peered":"2024-09-12T03:19:57.389694+0000","last_unstale":"2024-09-12T03:19:59.398595+0000","last_undegraded":"2024-09-12T03:19:59.398595+0000","last_fullsized":"2024-09-12T03:19:59.398595+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_clean_scrub_stamp":"2024-09-12T03:19:56.366974+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:01:26.160193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.412792+0000","last_change":"2024-09-12T03:19:58.415175+0000","last_active":"2024-09-12T03:19:59.412792+0000","last_peered":"2024-09-12T03:19:59.412792+0000","last_clean":"2024-09-12T03:19:59.412792+0000","last_became_active":"2024-09-12T03:19:58.414899+0000","last_became_peered":"2024-09-12T03:19:58.414899+0000","last_unstale":"2024-09-12T03:19:59.412792+0000","last_undegraded":"2024-09-12T03:19:59.412792+0000","last_fullsized":"2024-09-12T03:19:59.412792+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:36:33.065448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"66'2","reported_seq":16,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.400437+0000","last_change":"2024-09-12T03:19:57.393946+0000","last_active":"2024-09-12T03:19:59.400437+0000","last_peered":"2024-09-12T03:19:59.400437+0000","last_clean":"2024-09-12T03:19:59.400437+0000","last_became_active":"2024-09-12T03:19:57.393672+0000","last_became_peered":"2024-09-12T03:19:57.393672+0000","last_unstale":"2024-09-12T03:19:59.400437+0000","last_undegraded":"2024-09-12T03:19:59.400437+0000","last_fullsized":"2024-09-12T03:19:59.400437+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_clean_scrub_stamp":"2024-09-12T03:19:56.366974+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:57:23.221949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.412495+0000","last_change":"2024-09-12T03:19:58.403124+0000","last_active":"2024-09-12T03:19:59.412495+0000","last_peered":"2024-09-12T03:19:59.412495+0000","last_clean":"2024-09-12T03:19:59.412495+0000","last_became_active":"2024-09-12T03:19:58.402859+0000","last_became_peered":"2024-09-12T03:19:58.402859+0000","last_unstale":"2024-09-12T03:19:59.412495+0000","last_undegraded":"2024-09-12T03:19:59.412495+0000","last_fullsized":"2024-09-12T03:19:59.412495+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:41:26.586794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"66'2","reported_seq":16,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.412896+0000","last_change":"2024-09-12T03:19:57.396585+0000","last_active":"2024-09-12T03:19:59.412896+0000","last_peered":"2024-09-12T03:19:59.412896+0000","last_clean":"2024-09-12T03:19:59.412896+0000","last_became_active":"2024-09-12T03:19:57.396109+0000","last_became_peered":"2024-09-12T03:19:57.396109+0000","last_unstale":"2024-09-12T03:19:59.412896+0000","last_undegraded":"2024-09-12T03:19:59.412896+0000","last_fullsized":"2024-09-12T03:19:59.412896+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_clean_scrub_stamp":"2024-09-12T03:19:56.366974+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:25:50.078358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:19:57.371227+0000","last_change":"2024-09-12T03:19:57.371227+0000","last_active":"2024-09-12T03:19:57.371227+0000","last_peered":"2024-09-12T03:19:57.371227+0000","last_clean":"2024-09-12T03:19:57.371227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:57.371227+0000","last_undegraded":"2024-09-12T03:19:57.371227+0000","last_fullsized":"2024-09-12T03:19:57.371227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1b","version":"66'1","reported_seq":15,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.412362+0000","last_change":"2024-09-12T03:19:57.391056+0000","last_active":"2024-09-12T03:19:59.412362+0000","last_peered":"2024-09-12T03:19:59.412362+0000","last_clean":"2024-09-12T03:19:59.412362+0000","last_became_active":"2024-09-12T03:19:57.390769+0000","last_became_peered":"2024-09-12T03:19:57.390769+0000","last_unstale":"2024-09-12T03:19:59.412362+0000","last_undegraded":"2024-09-12T03:19:59.412362+0000","last_fullsized":"2024-09-12T03:19:59.412362+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_clean_scrub_stamp":"2024-09-12T03:19:56.366974+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:41:12.173208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.400570+0000","last_change":"2024-09-12T03:19:58.415098+0000","last_active":"2024-09-12T03:19:59.400570+0000","last_peered":"2024-09-12T03:19:59.400570+0000","last_clean":"2024-09-12T03:19:59.400570+0000","last_became_active":"2024-09-12T03:19:58.411028+0000","last_became_peered":"2024-09-12T03:19:58.411028+0000","last_unstale":"2024-09-12T03:19:59.400570+0000","last_undegraded":"2024-09-12T03:19:59.400570+0000","last_fullsized":"2024-09-12T03:19:59.400570+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:29:23.148491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"66'1","reported_seq":15,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.412238+0000","last_change":"2024-09-12T03:19:57.395730+0000","last_active":"2024-09-12T03:19:59.412238+0000","last_peered":"2024-09-12T03:19:59.412238+0000","last_clean":"2024-09-12T03:19:59.412238+0000","last_became_active":"2024-09-12T03:19:57.395253+0000","last_became_peered":"2024-09-12T03:19:57.395253+0000","last_unstale":"2024-09-12T03:19:59.412238+0000","last_undegraded":"2024-09-12T03:19:59.412238+0000","last_fullsized":"2024-09-12T03:19:59.412238+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_clean_scrub_stamp":"2024-09-12T03:19:56.366974+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:54:51.158313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.411683+0000","last_change":"2024-09-12T03:19:58.400222+0000","last_active":"2024-09-12T03:19:59.411683+0000","last_peered":"2024-09-12T03:19:59.411683+0000","last_clean":"2024-09-12T03:19:59.411683+0000","last_became_active":"2024-09-12T03:19:58.399906+0000","last_became_peered":"2024-09-12T03:19:58.399906+0000","last_unstale":"2024-09-12T03:19:59.411683+0000","last_undegraded":"2024-09-12T03:19:59.411683+0000","last_fullsized":"2024-09-12T03:19:59.411683+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:17:22.493344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:19:56.366974+0000","last_change":"2024-09-12T03:19:56.366974+0000","last_active":"2024-09-12T03:19:56.366974+0000","last_peered":"2024-09-12T03:19:56.366974+0000","last_clean":"2024-09-12T03:19:56.366974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:56.366974+0000","last_undegraded":"2024-09-12T03:19:56.366974+0000","last_fullsized":"2024-09-12T03:19:56.366974+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_clean_scrub_stamp":"2024-09-12T03:19:56.366974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.398361+0000","last_change":"2024-09-12T03:19:58.405943+0000","last_active":"2024-09-12T03:19:59.398361+0000","last_peered":"2024-09-12T03:19:59.398361+0000","last_clean":"2024-09-12T03:19:59.398361+0000","last_became_active":"2024-09-12T03:19:58.405630+0000","last_became_peered":"2024-09-12T03:19:58.405630+0000","last_unstale":"2024-09-12T03:19:59.398361+0000","last_undegraded":"2024-09-12T03:19:59.398361+0000","last_fullsized":"2024-09-12T03:19:59.398361+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:00:46.656402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.411776+0000","last_change":"2024-09-12T03:19:57.394447+0000","last_active":"2024-09-12T03:19:59.411776+0000","last_peered":"2024-09-12T03:19:59.411776+0000","last_clean":"2024-09-12T03:19:59.411776+0000","last_became_active":"2024-09-12T03:19:57.394153+0000","last_became_peered":"2024-09-12T03:19:57.394153+0000","last_unstale":"2024-09-12T03:19:59.411776+0000","last_undegraded":"2024-09-12T03:19:59.411776+0000","last_fullsized":"2024-09-12T03:19:59.411776+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_clean_scrub_stamp":"2024-09-12T03:19:56.366974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:12:19.700994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:19:57.371227+0000","last_change":"2024-09-12T03:19:57.371227+0000","last_active":"2024-09-12T03:19:57.371227+0000","last_peered":"2024-09-12T03:19:57.371227+0000","last_clean":"2024-09-12T03:19:57.371227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:57.371227+0000","last_undegraded":"2024-09-12T03:19:57.371227+0000","last_fullsized":"2024-09-12T03:19:57.371227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.400155+0000","last_change":"2024-09-12T03:19:57.393730+0000","last_active":"2024-09-12T03:19:59.400155+0000","last_peered":"2024-09-12T03:19:59.400155+0000","last_clean":"2024-09-12T03:19:59.400155+0000","last_became_active":"2024-09-12T03:19:57.393425+0000","last_became_peered":"2024-09-12T03:19:57.393425+0000","last_unstale":"2024-09-12T03:19:59.400155+0000","last_undegraded":"2024-09-12T03:19:59.400155+0000","last_fullsized":"2024-09-12T03:19:59.400155+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_clean_scrub_stamp":"2024-09-12T03:19:56.366974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:39:01.426373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.412571+0000","last_change":"2024-09-12T03:19:58.409130+0000","last_active":"2024-09-12T03:19:59.412571+0000","last_peered":"2024-09-12T03:19:59.412571+0000","last_clean":"2024-09-12T03:19:59.412571+0000","last_became_active":"2024-09-12T03:19:58.408838+0000","last_became_peered":"2024-09-12T03:19:58.408838+0000","last_unstale":"2024-09-12T03:19:59.412571+0000","last_undegraded":"2024-09-12T03:19:59.412571+0000","last_fullsized":"2024-09-12T03:19:59.412571+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:38:31.191391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.412967+0000","last_change":"2024-09-12T03:19:57.391703+0000","last_active":"2024-09-12T03:19:59.412967+0000","last_peered":"2024-09-12T03:19:59.412967+0000","last_clean":"2024-09-12T03:19:59.412967+0000","last_became_active":"2024-09-12T03:19:57.391345+0000","last_became_peered":"2024-09-12T03:19:57.391345+0000","last_unstale":"2024-09-12T03:19:59.412967+0000","last_undegraded":"2024-09-12T03:19:59.412967+0000","last_fullsized":"2024-09-12T03:19:59.412967+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_clean_scrub_stamp":"2024-09-12T03:19:56.366974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:05:09.764085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.412786+0000","last_change":"2024-09-12T03:19:58.410704+0000","last_active":"2024-09-12T03:19:59.412786+0000","last_peered":"2024-09-12T03:19:59.412786+0000","last_clean":"2024-09-12T03:19:59.412786+0000","last_became_active":"2024-09-12T03:19:58.410390+0000","last_became_peered":"2024-09-12T03:19:58.410390+0000","last_unstale":"2024-09-12T03:19:59.412786+0000","last_undegraded":"2024-09-12T03:19:59.412786+0000","last_fullsized":"2024-09-12T03:19:59.412786+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:27:21.762076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"66'3","reported_seq":17,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.412074+0000","last_change":"2024-09-12T03:19:57.383973+0000","last_active":"2024-09-12T03:19:59.412074+0000","last_peered":"2024-09-12T03:19:59.412074+0000","last_clean":"2024-09-12T03:19:59.412074+0000","last_became_active":"2024-09-12T03:19:57.383481+0000","last_became_peered":"2024-09-12T03:19:57.383481+0000","last_unstale":"2024-09-12T03:19:59.412074+0000","last_undegraded":"2024-09-12T03:19:59.412074+0000","last_fullsized":"2024-09-12T03:19:59.412074+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_clean_scrub_stamp":"2024-09-12T03:19:56.366974+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:00:45.652150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.411674+0000","last_change":"2024-09-12T03:19:58.400811+0000","last_active":"2024-09-12T03:19:59.411674+0000","last_peered":"2024-09-12T03:19:59.411674+0000","last_clean":"2024-09-12T03:19:59.411674+0000","last_became_active":"2024-09-12T03:19:58.400542+0000","last_became_peered":"2024-09-12T03:19:58.400542+0000","last_unstale":"2024-09-12T03:19:59.411674+0000","last_undegraded":"2024-09-12T03:19:59.411674+0000","last_fullsized":"2024-09-12T03:19:59.411674+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:01:16.275564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:19:56.366974+0000","last_change":"2024-09-12T03:19:56.366974+0000","last_active":"2024-09-12T03:19:56.366974+0000","last_peered":"2024-09-12T03:19:56.366974+0000","last_clean":"2024-09-12T03:19:56.366974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:56.366974+0000","last_undegraded":"2024-09-12T03:19:56.366974+0000","last_fullsized":"2024-09-12T03:19:56.366974+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_clean_scrub_stamp":"2024-09-12T03:19:56.366974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.411683+0000","last_change":"2024-09-12T03:19:58.402659+0000","last_active":"2024-09-12T03:19:59.411683+0000","last_peered":"2024-09-12T03:19:59.411683+0000","last_clean":"2024-09-12T03:19:59.411683+0000","last_became_active":"2024-09-12T03:19:58.402358+0000","last_became_peered":"2024-09-12T03:19:58.402358+0000","last_unstale":"2024-09-12T03:19:59.411683+0000","last_undegraded":"2024-09-12T03:19:59.411683+0000","last_fullsized":"2024-09-12T03:19:59.411683+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:41:13.177460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:19:56.366974+0000","last_change":"2024-09-12T03:19:56.366974+0000","last_active":"2024-09-12T03:19:56.366974+0000","last_peered":"2024-09-12T03:19:56.366974+0000","last_clean":"2024-09-12T03:19:56.366974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:56.366974+0000","last_undegraded":"2024-09-12T03:19:56.366974+0000","last_fullsized":"2024-09-12T03:19:56.366974+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_clean_scrub_stamp":"2024-09-12T03:19:56.366974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:19:57.371227+0000","last_change":"2024-09-12T03:19:57.371227+0000","last_active":"2024-09-12T03:19:57.371227+0000","last_peered":"2024-09-12T03:19:57.371227+0000","last_clean":"2024-09-12T03:19:57.371227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:57.371227+0000","last_undegraded":"2024-09-12T03:19:57.371227+0000","last_fullsized":"2024-09-12T03:19:57.371227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.412349+0000","last_change":"2024-09-12T03:19:57.395906+0000","last_active":"2024-09-12T03:19:59.412349+0000","last_peered":"2024-09-12T03:19:59.412349+0000","last_clean":"2024-09-12T03:19:59.412349+0000","last_became_active":"2024-09-12T03:19:57.395558+0000","last_became_peered":"2024-09-12T03:19:57.395558+0000","last_unstale":"2024-09-12T03:19:59.412349+0000","last_undegraded":"2024-09-12T03:19:59.412349+0000","last_fullsized":"2024-09-12T03:19:59.412349+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_clean_scrub_stamp":"2024-09-12T03:19:56.366974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:23:25.739217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.398297+0000","last_change":"2024-09-12T03:19:58.407844+0000","last_active":"2024-09-12T03:19:59.398297+0000","last_peered":"2024-09-12T03:19:59.398297+0000","last_clean":"2024-09-12T03:19:59.398297+0000","last_became_active":"2024-09-12T03:19:58.407513+0000","last_became_peered":"2024-09-12T03:19:58.407513+0000","last_unstale":"2024-09-12T03:19:59.398297+0000","last_undegraded":"2024-09-12T03:19:59.398297+0000","last_fullsized":"2024-09-12T03:19:59.398297+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:58:58.914180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:19:56.366974+0000","last_change":"2024-09-12T03:19:56.366974+0000","last_active":"2024-09-12T03:19:56.366974+0000","last_peered":"2024-09-12T03:19:56.366974+0000","last_clean":"2024-09-12T03:19:56.366974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:56.366974+0000","last_undegraded":"2024-09-12T03:19:56.366974+0000","last_fullsized":"2024-09-12T03:19:56.366974+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_clean_scrub_stamp":"2024-09-12T03:19:56.366974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.412013+0000","last_change":"2024-09-12T03:19:58.406880+0000","last_active":"2024-09-12T03:19:59.412013+0000","last_peered":"2024-09-12T03:19:59.412013+0000","last_clean":"2024-09-12T03:19:59.412013+0000","last_became_active":"2024-09-12T03:19:58.406585+0000","last_became_peered":"2024-09-12T03:19:58.406585+0000","last_unstale":"2024-09-12T03:19:59.412013+0000","last_undegraded":"2024-09-12T03:19:59.412013+0000","last_fullsized":"2024-09-12T03:19:59.412013+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:44:18.879486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"66'8","reported_seq":22,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.400987+0000","last_change":"2024-09-12T03:19:57.397454+0000","last_active":"2024-09-12T03:19:59.400987+0000","last_peered":"2024-09-12T03:19:59.400987+0000","last_clean":"2024-09-12T03:19:59.400987+0000","last_became_active":"2024-09-12T03:19:57.397126+0000","last_became_peered":"2024-09-12T03:19:57.397126+0000","last_unstale":"2024-09-12T03:19:59.400987+0000","last_undegraded":"2024-09-12T03:19:59.400987+0000","last_fullsized":"2024-09-12T03:19:59.400987+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_clean_scrub_stamp":"2024-09-12T03:19:56.366974+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:14:13.101395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.400608+0000","last_change":"2024-09-12T03:19:58.402681+0000","last_active":"2024-09-12T03:19:59.400608+0000","last_peered":"2024-09-12T03:19:59.400608+0000","last_clean":"2024-09-12T03:19:59.400608+0000","last_became_active":"2024-09-12T03:19:58.402409+0000","last_became_peered":"2024-09-12T03:19:58.402409+0000","last_unstale":"2024-09-12T03:19:59.400608+0000","last_undegraded":"2024-09-12T03:19:59.400608+0000","last_fullsized":"2024-09-12T03:19:59.400608+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:56:05.472430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"66'1","reported_seq":15,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.412766+0000","last_change":"2024-09-12T03:19:57.394555+0000","last_active":"2024-09-12T03:19:59.412766+0000","last_peered":"2024-09-12T03:19:59.412766+0000","last_clean":"2024-09-12T03:19:59.412766+0000","last_became_active":"2024-09-12T03:19:57.394277+0000","last_became_peered":"2024-09-12T03:19:57.394277+0000","last_unstale":"2024-09-12T03:19:59.412766+0000","last_undegraded":"2024-09-12T03:19:59.412766+0000","last_fullsized":"2024-09-12T03:19:59.412766+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_clean_scrub_stamp":"2024-09-12T03:19:56.366974+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:50:52.123837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.412469+0000","last_change":"2024-09-12T03:19:58.411847+0000","last_active":"2024-09-12T03:19:59.412469+0000","last_peered":"2024-09-12T03:19:59.412469+0000","last_clean":"2024-09-12T03:19:59.412469+0000","last_became_active":"2024-09-12T03:19:58.409965+0000","last_became_peered":"2024-09-12T03:19:58.409965+0000","last_unstale":"2024-09-12T03:19:59.412469+0000","last_undegraded":"2024-09-12T03:19:59.412469+0000","last_fullsized":"2024-09-12T03:19:59.412469+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:45:22.536315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"66'2","reported_seq":16,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.400471+0000","last_change":"2024-09-12T03:19:57.394701+0000","last_active":"2024-09-12T03:19:59.400471+0000","last_peered":"2024-09-12T03:19:59.400471+0000","last_clean":"2024-09-12T03:19:59.400471+0000","last_became_active":"2024-09-12T03:19:57.394348+0000","last_became_peered":"2024-09-12T03:19:57.394348+0000","last_unstale":"2024-09-12T03:19:59.400471+0000","last_undegraded":"2024-09-12T03:19:59.400471+0000","last_fullsized":"2024-09-12T03:19:59.400471+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_clean_scrub_stamp":"2024-09-12T03:19:56.366974+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:44:36.098558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:19:57.371227+0000","last_change":"2024-09-12T03:19:57.371227+0000","last_active":"2024-09-12T03:19:57.371227+0000","last_peered":"2024-09-12T03:19:57.371227+0000","last_clean":"2024-09-12T03:19:57.371227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:57.371227+0000","last_undegraded":"2024-09-12T03:19:57.371227+0000","last_fullsized":"2024-09-12T03:19:57.371227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.d","version":"66'1","reported_seq":15,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.412373+0000","last_change":"2024-09-12T03:19:57.397975+0000","last_active":"2024-09-12T03:19:59.412373+0000","last_peered":"2024-09-12T03:19:59.412373+0000","last_clean":"2024-09-12T03:19:59.412373+0000","last_became_active":"2024-09-12T03:19:57.397637+0000","last_became_peered":"2024-09-12T03:19:57.397637+0000","last_unstale":"2024-09-12T03:19:59.412373+0000","last_undegraded":"2024-09-12T03:19:59.412373+0000","last_fullsized":"2024-09-12T03:19:59.412373+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_clean_scrub_stamp":"2024-09-12T03:19:56.366974+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:16:35.102685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.411820+0000","last_change":"2024-09-12T03:19:58.399501+0000","last_active":"2024-09-12T03:19:59.411820+0000","last_peered":"2024-09-12T03:19:59.411820+0000","last_clean":"2024-09-12T03:19:59.411820+0000","last_became_active":"2024-09-12T03:19:58.399203+0000","last_became_peered":"2024-09-12T03:19:58.399203+0000","last_unstale":"2024-09-12T03:19:59.411820+0000","last_undegraded":"2024-09-12T03:19:59.411820+0000","last_fullsized":"2024-09-12T03:19:59.411820+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:45:17.689061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.412050+0000","last_change":"2024-09-12T03:19:57.394026+0000","last_active":"2024-09-12T03:19:59.412050+0000","last_peered":"2024-09-12T03:19:59.412050+0000","last_clean":"2024-09-12T03:19:59.412050+0000","last_became_active":"2024-09-12T03:19:57.393679+0000","last_became_peered":"2024-09-12T03:19:57.393679+0000","last_unstale":"2024-09-12T03:19:59.412050+0000","last_undegraded":"2024-09-12T03:19:59.412050+0000","last_fullsized":"2024-09-12T03:19:59.412050+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_clean_scrub_stamp":"2024-09-12T03:19:56.366974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:41:50.153852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.412261+0000","last_change":"2024-09-12T03:19:58.407764+0000","last_active":"2024-09-12T03:19:59.412261+0000","last_peered":"2024-09-12T03:19:59.412261+0000","last_clean":"2024-09-12T03:19:59.412261+0000","last_became_active":"2024-09-12T03:19:58.407483+0000","last_became_peered":"2024-09-12T03:19:58.407483+0000","last_unstale":"2024-09-12T03:19:59.412261+0000","last_undegraded":"2024-09-12T03:19:59.412261+0000","last_fullsized":"2024-09-12T03:19:59.412261+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:11:19.244120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.412443+0000","last_change":"2024-09-12T03:19:57.394831+0000","last_active":"2024-09-12T03:19:59.412443+0000","last_peered":"2024-09-12T03:19:59.412443+0000","last_clean":"2024-09-12T03:19:59.412443+0000","last_became_active":"2024-09-12T03:19:57.394505+0000","last_became_peered":"2024-09-12T03:19:57.394505+0000","last_unstale":"2024-09-12T03:19:59.412443+0000","last_undegraded":"2024-09-12T03:19:59.412443+0000","last_fullsized":"2024-09-12T03:19:59.412443+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_clean_scrub_stamp":"2024-09-12T03:19:56.366974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:59:35.357929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.411253+0000","last_change":"2024-09-12T03:19:58.408696+0000","last_active":"2024-09-12T03:19:59.411253+0000","last_peered":"2024-09-12T03:19:59.411253+0000","last_clean":"2024-09-12T03:19:59.411253+0000","last_became_active":"2024-09-12T03:19:58.408393+0000","last_became_peered":"2024-09-12T03:19:58.408393+0000","last_unstale":"2024-09-12T03:19:59.411253+0000","last_undegraded":"2024-09-12T03:19:59.411253+0000","last_fullsized":"2024-09-12T03:19:59.411253+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:15:02.240922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:19:56.366974+0000","last_change":"2024-09-12T03:19:56.366974+0000","last_active":"2024-09-12T03:19:56.366974+0000","last_peered":"2024-09-12T03:19:56.366974+0000","last_clean":"2024-09-12T03:19:56.366974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:56.366974+0000","last_undegraded":"2024-09-12T03:19:56.366974+0000","last_fullsized":"2024-09-12T03:19:56.366974+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_clean_scrub_stamp":"2024-09-12T03:19:56.366974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.411564+0000","last_change":"2024-09-12T03:19:58.409403+0000","last_active":"2024-09-12T03:19:59.411564+0000","last_peered":"2024-09-12T03:19:59.411564+0000","last_clean":"2024-09-12T03:19:59.411564+0000","last_became_active":"2024-09-12T03:19:58.409087+0000","last_became_peered":"2024-09-12T03:19:58.409087+0000","last_unstale":"2024-09-12T03:19:59.411564+0000","last_undegraded":"2024-09-12T03:19:59.411564+0000","last_fullsized":"2024-09-12T03:19:59.411564+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:44:15.824316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.398237+0000","last_change":"2024-09-12T03:19:57.394055+0000","last_active":"2024-09-12T03:19:59.398237+0000","last_peered":"2024-09-12T03:19:59.398237+0000","last_clean":"2024-09-12T03:19:59.398237+0000","last_became_active":"2024-09-12T03:19:57.393767+0000","last_became_peered":"2024-09-12T03:19:57.393767+0000","last_unstale":"2024-09-12T03:19:59.398237+0000","last_undegraded":"2024-09-12T03:19:59.398237+0000","last_fullsized":"2024-09-12T03:19:59.398237+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_clean_scrub_stamp":"2024-09-12T03:19:56.366974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:34:00.504515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.398615+0000","last_change":"2024-09-12T03:19:58.400064+0000","last_active":"2024-09-12T03:19:59.398615+0000","last_peered":"2024-09-12T03:19:59.398615+0000","last_clean":"2024-09-12T03:19:59.398615+0000","last_became_active":"2024-09-12T03:19:58.399777+0000","last_became_peered":"2024-09-12T03:19:58.399777+0000","last_unstale":"2024-09-12T03:19:59.398615+0000","last_undegraded":"2024-09-12T03:19:59.398615+0000","last_fullsized":"2024-09-12T03:19:59.398615+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:31:06.800562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"66'2","reported_seq":16,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.412883+0000","last_change":"2024-09-12T03:19:57.393966+0000","last_active":"2024-09-12T03:19:59.412883+0000","last_peered":"2024-09-12T03:19:59.412883+0000","last_clean":"2024-09-12T03:19:59.412883+0000","last_became_active":"2024-09-12T03:19:57.393680+0000","last_became_peered":"2024-09-12T03:19:57.393680+0000","last_unstale":"2024-09-12T03:19:59.412883+0000","last_undegraded":"2024-09-12T03:19:59.412883+0000","last_fullsized":"2024-09-12T03:19:59.412883+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_clean_scrub_stamp":"2024-09-12T03:19:56.366974+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:05:00.306420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:19:57.371227+0000","last_change":"2024-09-12T03:19:57.371227+0000","last_active":"2024-09-12T03:19:57.371227+0000","last_peered":"2024-09-12T03:19:57.371227+0000","last_clean":"2024-09-12T03:19:57.371227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:57.371227+0000","last_undegraded":"2024-09-12T03:19:57.371227+0000","last_fullsized":"2024-09-12T03:19:57.371227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:19:56.366974+0000","last_change":"2024-09-12T03:19:56.366974+0000","last_active":"2024-09-12T03:19:56.366974+0000","last_peered":"2024-09-12T03:19:56.366974+0000","last_clean":"2024-09-12T03:19:56.366974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:56.366974+0000","last_undegraded":"2024-09-12T03:19:56.366974+0000","last_fullsized":"2024-09-12T03:19:56.366974+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_clean_scrub_stamp":"2024-09-12T03:19:56.366974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.411946+0000","last_change":"2024-09-12T03:19:58.408958+0000","last_active":"2024-09-12T03:19:59.411946+0000","last_peered":"2024-09-12T03:19:59.411946+0000","last_clean":"2024-09-12T03:19:59.411946+0000","last_became_active":"2024-09-12T03:19:58.408665+0000","last_became_peered":"2024-09-12T03:19:58.408665+0000","last_unstale":"2024-09-12T03:19:59.411946+0000","last_undegraded":"2024-09-12T03:19:59.411946+0000","last_fullsized":"2024-09-12T03:19:59.411946+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:50:40.770697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.411911+0000","last_change":"2024-09-12T03:19:57.393827+0000","last_active":"2024-09-12T03:19:59.411911+0000","last_peered":"2024-09-12T03:19:59.411911+0000","last_clean":"2024-09-12T03:19:59.411911+0000","last_became_active":"2024-09-12T03:19:57.393501+0000","last_became_peered":"2024-09-12T03:19:57.393501+0000","last_unstale":"2024-09-12T03:19:59.411911+0000","last_undegraded":"2024-09-12T03:19:59.411911+0000","last_fullsized":"2024-09-12T03:19:59.411911+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_clean_scrub_stamp":"2024-09-12T03:19:56.366974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:00:49.820829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.412577+0000","last_change":"2024-09-12T03:19:58.416479+0000","last_active":"2024-09-12T03:19:59.412577+0000","last_peered":"2024-09-12T03:19:59.412577+0000","last_clean":"2024-09-12T03:19:59.412577+0000","last_became_active":"2024-09-12T03:19:58.416145+0000","last_became_peered":"2024-09-12T03:19:58.416145+0000","last_unstale":"2024-09-12T03:19:59.412577+0000","last_undegraded":"2024-09-12T03:19:59.412577+0000","last_fullsized":"2024-09-12T03:19:59.412577+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:37:31.629037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.398503+0000","last_change":"2024-09-12T03:19:57.392451+0000","last_active":"2024-09-12T03:19:59.398503+0000","last_peered":"2024-09-12T03:19:59.398503+0000","last_clean":"2024-09-12T03:19:59.398503+0000","last_became_active":"2024-09-12T03:19:57.392127+0000","last_became_peered":"2024-09-12T03:19:57.392127+0000","last_unstale":"2024-09-12T03:19:59.398503+0000","last_undegraded":"2024-09-12T03:19:59.398503+0000","last_fullsized":"2024-09-12T03:19:59.398503+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_clean_scrub_stamp":"2024-09-12T03:19:56.366974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:11:02.225246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.412704+0000","last_change":"2024-09-12T03:19:58.407515+0000","last_active":"2024-09-12T03:19:59.412704+0000","last_peered":"2024-09-12T03:19:59.412704+0000","last_clean":"2024-09-12T03:19:59.412704+0000","last_became_active":"2024-09-12T03:19:58.407189+0000","last_became_peered":"2024-09-12T03:19:58.407189+0000","last_unstale":"2024-09-12T03:19:59.412704+0000","last_undegraded":"2024-09-12T03:19:59.412704+0000","last_fullsized":"2024-09-12T03:19:59.412704+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:57:01.617936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.411502+0000","last_change":"2024-09-12T03:19:57.391167+0000","last_active":"2024-09-12T03:19:59.411502+0000","last_peered":"2024-09-12T03:19:59.411502+0000","last_clean":"2024-09-12T03:19:59.411502+0000","last_became_active":"2024-09-12T03:19:57.390901+0000","last_became_peered":"2024-09-12T03:19:57.390901+0000","last_unstale":"2024-09-12T03:19:59.411502+0000","last_undegraded":"2024-09-12T03:19:59.411502+0000","last_fullsized":"2024-09-12T03:19:59.411502+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_clean_scrub_stamp":"2024-09-12T03:19:56.366974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:09:37.761223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.412332+0000","last_change":"2024-09-12T03:19:58.420294+0000","last_active":"2024-09-12T03:19:59.412332+0000","last_peered":"2024-09-12T03:19:59.412332+0000","last_clean":"2024-09-12T03:19:59.412332+0000","last_became_active":"2024-09-12T03:19:58.418912+0000","last_became_peered":"2024-09-12T03:19:58.418912+0000","last_unstale":"2024-09-12T03:19:59.412332+0000","last_undegraded":"2024-09-12T03:19:59.412332+0000","last_fullsized":"2024-09-12T03:19:59.412332+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:14:39.136845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:19:56.366974+0000","last_change":"2024-09-12T03:19:56.366974+0000","last_active":"2024-09-12T03:19:56.366974+0000","last_peered":"2024-09-12T03:19:56.366974+0000","last_clean":"2024-09-12T03:19:56.366974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:56.366974+0000","last_undegraded":"2024-09-12T03:19:56.366974+0000","last_fullsized":"2024-09-12T03:19:56.366974+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_clean_scrub_stamp":"2024-09-12T03:19:56.366974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:19:57.371227+0000","last_change":"2024-09-12T03:19:57.371227+0000","last_active":"2024-09-12T03:19:57.371227+0000","last_peered":"2024-09-12T03:19:57.371227+0000","last_clean":"2024-09-12T03:19:57.371227+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:19:57.371227+0000","last_undegraded":"2024-09-12T03:19:57.371227+0000","last_fullsized":"2024-09-12T03:19:57.371227+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.399794+0000","last_change":"2024-09-12T03:19:57.393150+0000","last_active":"2024-09-12T03:19:59.399794+0000","last_peered":"2024-09-12T03:19:59.399794+0000","last_clean":"2024-09-12T03:19:59.399794+0000","last_became_active":"2024-09-12T03:19:57.392692+0000","last_became_peered":"2024-09-12T03:19:57.392692+0000","last_unstale":"2024-09-12T03:19:59.399794+0000","last_undegraded":"2024-09-12T03:19:59.399794+0000","last_fullsized":"2024-09-12T03:19:59.399794+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_clean_scrub_stamp":"2024-09-12T03:19:56.366974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:44:27.441871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.411918+0000","last_change":"2024-09-12T03:19:58.407787+0000","last_active":"2024-09-12T03:19:59.411918+0000","last_peered":"2024-09-12T03:19:59.411918+0000","last_clean":"2024-09-12T03:19:59.411918+0000","last_became_active":"2024-09-12T03:19:58.407407+0000","last_became_peered":"2024-09-12T03:19:58.407407+0000","last_unstale":"2024-09-12T03:19:59.411918+0000","last_undegraded":"2024-09-12T03:19:59.411918+0000","last_fullsized":"2024-09-12T03:19:59.411918+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:38:14.768525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.412289+0000","last_change":"2024-09-12T03:19:57.391520+0000","last_active":"2024-09-12T03:19:59.412289+0000","last_peered":"2024-09-12T03:19:59.412289+0000","last_clean":"2024-09-12T03:19:59.412289+0000","last_became_active":"2024-09-12T03:19:57.391140+0000","last_became_peered":"2024-09-12T03:19:57.391140+0000","last_unstale":"2024-09-12T03:19:59.412289+0000","last_undegraded":"2024-09-12T03:19:59.412289+0000","last_fullsized":"2024-09-12T03:19:59.412289+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_clean_scrub_stamp":"2024-09-12T03:19:56.366974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:42:32.285383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.412286+0000","last_change":"2024-09-12T03:19:58.409818+0000","last_active":"2024-09-12T03:19:59.412286+0000","last_peered":"2024-09-12T03:19:59.412286+0000","last_clean":"2024-09-12T03:19:59.412286+0000","last_became_active":"2024-09-12T03:19:58.409507+0000","last_became_peered":"2024-09-12T03:19:58.409507+0000","last_unstale":"2024-09-12T03:19:59.412286+0000","last_undegraded":"2024-09-12T03:19:59.412286+0000","last_fullsized":"2024-09-12T03:19:59.412286+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:21:24.731904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"66'2","reported_seq":16,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.412444+0000","last_change":"2024-09-12T03:19:57.395758+0000","last_active":"2024-09-12T03:19:59.412444+0000","last_peered":"2024-09-12T03:19:59.412444+0000","last_clean":"2024-09-12T03:19:59.412444+0000","last_became_active":"2024-09-12T03:19:57.395422+0000","last_became_peered":"2024-09-12T03:19:57.395422+0000","last_unstale":"2024-09-12T03:19:59.412444+0000","last_undegraded":"2024-09-12T03:19:59.412444+0000","last_fullsized":"2024-09-12T03:19:59.412444+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_clean_scrub_stamp":"2024-09-12T03:19:56.366974+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:18:13.412357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":252,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.411416+0000","last_change":"2024-09-12T03:16:21.883446+0000","last_active":"2024-09-12T03:19:59.411416+0000","last_peered":"2024-09-12T03:19:59.411416+0000","last_clean":"2024-09-12T03:19:59.411416+0000","last_became_active":"2024-09-12T03:16:21.882207+0000","last_became_peered":"2024-09-12T03:16:21.882207+0000","last_unstale":"2024-09-12T03:19:59.411416+0000","last_undegraded":"2024-09-12T03:19:59.411416+0000","last_fullsized":"2024-09-12T03:19:59.411416+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:20.316426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:20.316426+0000","last_clean_scrub_stamp":"2024-09-12T03:16:20.316426+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:12:15.424320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":102,"acting":102,"num_store_stats":6},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":764,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":35,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":30,"ondisk_log_size":30,"up":50,"acting":50,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542190,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36716,"kb_used_data":1368,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335124,"statfs":{"total":96636764160,"available":96599166976,"internally_reserved":0,"allocated":1400832,"data_stored":924039,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542192,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31996,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339844,"statfs":{"total":96636764160,"available":96604000256,"internally_reserved":0,"allocated":819200,"data_stored":339447,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542190,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":36104,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335736,"statfs":{"total":96636764160,"available":96599793664,"internally_reserved":0,"allocated":761856,"data_stored":314188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542190,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31980,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339860,"statfs":{"total":96636764160,"available":96604016640,"internally_reserved":0,"allocated":802816,"data_stored":333125,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542191,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27836,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344004,"statfs":{"total":96636764160,"available":96608260096,"internally_reserved":0,"allocated":819200,"data_stored":339467,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542191,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27840,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344000,"statfs":{"total":96636764160,"available":96608256000,"internally_reserved":0,"allocated":823296,"data_stored":340967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542191,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31936,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339904,"statfs":{"total":96636764160,"available":96604061696,"internally_reserved":0,"allocated":761856,"data_stored":314188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542192,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28396,"kb_used_data":1360,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343444,"statfs":{"total":96636764160,"available":96607686656,"internally_reserved":0,"allocated":1392640,"data_stored":923423,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T03:20:02.099 INFO:tasks.ceph.ceph_manager.ceph:PG 11.3c is not active+clean 2024-09-12T03:20:02.099 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.3c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T03:19:57.371227+0000', 'last_change': '2024-09-12T03:19:57.371227+0000', 'last_active': '2024-09-12T03:19:57.371227+0000', 'last_peered': '2024-09-12T03:19:57.371227+0000', 'last_clean': '2024-09-12T03:19:57.371227+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:19:57.371227+0000', 'last_undegraded': '2024-09-12T03:19:57.371227+0000', 'last_fullsized': '2024-09-12T03:19:57.371227+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:19:57.371227+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:19:57.371227+0000', 'last_clean_scrub_stamp': '2024-09-12T03:19:57.371227+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T03:20:02.099 INFO:tasks.ceph.ceph_manager.ceph:PG 11.39 is not active+clean 2024-09-12T03:20:02.099 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.39', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T03:19:57.371227+0000', 'last_change': '2024-09-12T03:19:57.371227+0000', 'last_active': '2024-09-12T03:19:57.371227+0000', 'last_peered': '2024-09-12T03:19:57.371227+0000', 'last_clean': '2024-09-12T03:19:57.371227+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:19:57.371227+0000', 'last_undegraded': '2024-09-12T03:19:57.371227+0000', 'last_fullsized': '2024-09-12T03:19:57.371227+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:19:57.371227+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:19:57.371227+0000', 'last_clean_scrub_stamp': '2024-09-12T03:19:57.371227+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T03:20:02.099 INFO:tasks.ceph.ceph_manager.ceph:PG 11.34 is not active+clean 2024-09-12T03:20:02.100 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T03:19:57.371227+0000', 'last_change': '2024-09-12T03:19:57.371227+0000', 'last_active': '2024-09-12T03:19:57.371227+0000', 'last_peered': '2024-09-12T03:19:57.371227+0000', 'last_clean': '2024-09-12T03:19:57.371227+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:19:57.371227+0000', 'last_undegraded': '2024-09-12T03:19:57.371227+0000', 'last_fullsized': '2024-09-12T03:19:57.371227+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:19:57.371227+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:19:57.371227+0000', 'last_clean_scrub_stamp': '2024-09-12T03:19:57.371227+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T03:20:02.100 INFO:tasks.ceph.ceph_manager.ceph:PG 11.2e is not active+clean 2024-09-12T03:20:02.100 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.2e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T03:19:57.371227+0000', 'last_change': '2024-09-12T03:19:57.371227+0000', 'last_active': '2024-09-12T03:19:57.371227+0000', 'last_peered': '2024-09-12T03:19:57.371227+0000', 'last_clean': '2024-09-12T03:19:57.371227+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:19:57.371227+0000', 'last_undegraded': '2024-09-12T03:19:57.371227+0000', 'last_fullsized': '2024-09-12T03:19:57.371227+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:19:57.371227+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:19:57.371227+0000', 'last_clean_scrub_stamp': '2024-09-12T03:19:57.371227+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T03:20:02.100 INFO:tasks.ceph.ceph_manager.ceph:PG 11.26 is not active+clean 2024-09-12T03:20:02.100 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.26', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T03:19:57.371227+0000', 'last_change': '2024-09-12T03:19:57.371227+0000', 'last_active': '2024-09-12T03:19:57.371227+0000', 'last_peered': '2024-09-12T03:19:57.371227+0000', 'last_clean': '2024-09-12T03:19:57.371227+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:19:57.371227+0000', 'last_undegraded': '2024-09-12T03:19:57.371227+0000', 'last_fullsized': '2024-09-12T03:19:57.371227+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:19:57.371227+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:19:57.371227+0000', 'last_clean_scrub_stamp': '2024-09-12T03:19:57.371227+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T03:20:02.100 INFO:tasks.ceph.ceph_manager.ceph:PG 11.22 is not active+clean 2024-09-12T03:20:02.100 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T03:19:57.371227+0000', 'last_change': '2024-09-12T03:19:57.371227+0000', 'last_active': '2024-09-12T03:19:57.371227+0000', 'last_peered': '2024-09-12T03:19:57.371227+0000', 'last_clean': '2024-09-12T03:19:57.371227+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:19:57.371227+0000', 'last_undegraded': '2024-09-12T03:19:57.371227+0000', 'last_fullsized': '2024-09-12T03:19:57.371227+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:19:57.371227+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:19:57.371227+0000', 'last_clean_scrub_stamp': '2024-09-12T03:19:57.371227+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T03:20:02.100 INFO:tasks.ceph.ceph_manager.ceph:PG 11.21 is not active+clean 2024-09-12T03:20:02.100 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.21', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T03:19:57.371227+0000', 'last_change': '2024-09-12T03:19:57.371227+0000', 'last_active': '2024-09-12T03:19:57.371227+0000', 'last_peered': '2024-09-12T03:19:57.371227+0000', 'last_clean': '2024-09-12T03:19:57.371227+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:19:57.371227+0000', 'last_undegraded': '2024-09-12T03:19:57.371227+0000', 'last_fullsized': '2024-09-12T03:19:57.371227+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:19:57.371227+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:19:57.371227+0000', 'last_clean_scrub_stamp': '2024-09-12T03:19:57.371227+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T03:20:02.100 INFO:tasks.ceph.ceph_manager.ceph:PG 11.1a is not active+clean 2024-09-12T03:20:02.100 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T03:19:57.371227+0000', 'last_change': '2024-09-12T03:19:57.371227+0000', 'last_active': '2024-09-12T03:19:57.371227+0000', 'last_peered': '2024-09-12T03:19:57.371227+0000', 'last_clean': '2024-09-12T03:19:57.371227+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:19:57.371227+0000', 'last_undegraded': '2024-09-12T03:19:57.371227+0000', 'last_fullsized': '2024-09-12T03:19:57.371227+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:19:57.371227+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:19:57.371227+0000', 'last_clean_scrub_stamp': '2024-09-12T03:19:57.371227+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T03:20:02.101 INFO:tasks.ceph.ceph_manager.ceph:PG 10.19 is not active+clean 2024-09-12T03:20:02.101 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T03:19:56.366974+0000', 'last_change': '2024-09-12T03:19:56.366974+0000', 'last_active': '2024-09-12T03:19:56.366974+0000', 'last_peered': '2024-09-12T03:19:56.366974+0000', 'last_clean': '2024-09-12T03:19:56.366974+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:19:56.366974+0000', 'last_undegraded': '2024-09-12T03:19:56.366974+0000', 'last_fullsized': '2024-09-12T03:19:56.366974+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:19:56.366974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:19:56.366974+0000', 'last_clean_scrub_stamp': '2024-09-12T03:19:56.366974+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T03:20:02.101 INFO:tasks.ceph.ceph_manager.ceph:PG 11.16 is not active+clean 2024-09-12T03:20:02.101 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T03:19:57.371227+0000', 'last_change': '2024-09-12T03:19:57.371227+0000', 'last_active': '2024-09-12T03:19:57.371227+0000', 'last_peered': '2024-09-12T03:19:57.371227+0000', 'last_clean': '2024-09-12T03:19:57.371227+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:19:57.371227+0000', 'last_undegraded': '2024-09-12T03:19:57.371227+0000', 'last_fullsized': '2024-09-12T03:19:57.371227+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:19:57.371227+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:19:57.371227+0000', 'last_clean_scrub_stamp': '2024-09-12T03:19:57.371227+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T03:20:02.101 INFO:tasks.ceph.ceph_manager.ceph:PG 10.14 is not active+clean 2024-09-12T03:20:02.101 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T03:19:56.366974+0000', 'last_change': '2024-09-12T03:19:56.366974+0000', 'last_active': '2024-09-12T03:19:56.366974+0000', 'last_peered': '2024-09-12T03:19:56.366974+0000', 'last_clean': '2024-09-12T03:19:56.366974+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:19:56.366974+0000', 'last_undegraded': '2024-09-12T03:19:56.366974+0000', 'last_fullsized': '2024-09-12T03:19:56.366974+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:19:56.366974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:19:56.366974+0000', 'last_clean_scrub_stamp': '2024-09-12T03:19:56.366974+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T03:20:02.101 INFO:tasks.ceph.ceph_manager.ceph:PG 10.13 is not active+clean 2024-09-12T03:20:02.101 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T03:19:56.366974+0000', 'last_change': '2024-09-12T03:19:56.366974+0000', 'last_active': '2024-09-12T03:19:56.366974+0000', 'last_peered': '2024-09-12T03:19:56.366974+0000', 'last_clean': '2024-09-12T03:19:56.366974+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:19:56.366974+0000', 'last_undegraded': '2024-09-12T03:19:56.366974+0000', 'last_fullsized': '2024-09-12T03:19:56.366974+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:19:56.366974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:19:56.366974+0000', 'last_clean_scrub_stamp': '2024-09-12T03:19:56.366974+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T03:20:02.101 INFO:tasks.ceph.ceph_manager.ceph:PG 11.13 is not active+clean 2024-09-12T03:20:02.101 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T03:19:57.371227+0000', 'last_change': '2024-09-12T03:19:57.371227+0000', 'last_active': '2024-09-12T03:19:57.371227+0000', 'last_peered': '2024-09-12T03:19:57.371227+0000', 'last_clean': '2024-09-12T03:19:57.371227+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:19:57.371227+0000', 'last_undegraded': '2024-09-12T03:19:57.371227+0000', 'last_fullsized': '2024-09-12T03:19:57.371227+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:19:57.371227+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:19:57.371227+0000', 'last_clean_scrub_stamp': '2024-09-12T03:19:57.371227+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T03:20:02.101 INFO:tasks.ceph.ceph_manager.ceph:PG 10.11 is not active+clean 2024-09-12T03:20:02.102 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T03:19:56.366974+0000', 'last_change': '2024-09-12T03:19:56.366974+0000', 'last_active': '2024-09-12T03:19:56.366974+0000', 'last_peered': '2024-09-12T03:19:56.366974+0000', 'last_clean': '2024-09-12T03:19:56.366974+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:19:56.366974+0000', 'last_undegraded': '2024-09-12T03:19:56.366974+0000', 'last_fullsized': '2024-09-12T03:19:56.366974+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:19:56.366974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:19:56.366974+0000', 'last_clean_scrub_stamp': '2024-09-12T03:19:56.366974+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T03:20:02.102 INFO:tasks.ceph.ceph_manager.ceph:PG 11.c is not active+clean 2024-09-12T03:20:02.102 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T03:19:57.371227+0000', 'last_change': '2024-09-12T03:19:57.371227+0000', 'last_active': '2024-09-12T03:19:57.371227+0000', 'last_peered': '2024-09-12T03:19:57.371227+0000', 'last_clean': '2024-09-12T03:19:57.371227+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:19:57.371227+0000', 'last_undegraded': '2024-09-12T03:19:57.371227+0000', 'last_fullsized': '2024-09-12T03:19:57.371227+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:19:57.371227+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:19:57.371227+0000', 'last_clean_scrub_stamp': '2024-09-12T03:19:57.371227+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T03:20:02.102 INFO:tasks.ceph.ceph_manager.ceph:PG 10.9 is not active+clean 2024-09-12T03:20:02.102 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T03:19:56.366974+0000', 'last_change': '2024-09-12T03:19:56.366974+0000', 'last_active': '2024-09-12T03:19:56.366974+0000', 'last_peered': '2024-09-12T03:19:56.366974+0000', 'last_clean': '2024-09-12T03:19:56.366974+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:19:56.366974+0000', 'last_undegraded': '2024-09-12T03:19:56.366974+0000', 'last_fullsized': '2024-09-12T03:19:56.366974+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:19:56.366974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:19:56.366974+0000', 'last_clean_scrub_stamp': '2024-09-12T03:19:56.366974+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T03:20:02.102 INFO:tasks.ceph.ceph_manager.ceph:PG 11.7 is not active+clean 2024-09-12T03:20:02.102 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T03:19:57.371227+0000', 'last_change': '2024-09-12T03:19:57.371227+0000', 'last_active': '2024-09-12T03:19:57.371227+0000', 'last_peered': '2024-09-12T03:19:57.371227+0000', 'last_clean': '2024-09-12T03:19:57.371227+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:19:57.371227+0000', 'last_undegraded': '2024-09-12T03:19:57.371227+0000', 'last_fullsized': '2024-09-12T03:19:57.371227+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:19:57.371227+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:19:57.371227+0000', 'last_clean_scrub_stamp': '2024-09-12T03:19:57.371227+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T03:20:02.102 INFO:tasks.ceph.ceph_manager.ceph:PG 10.6 is not active+clean 2024-09-12T03:20:02.103 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T03:19:56.366974+0000', 'last_change': '2024-09-12T03:19:56.366974+0000', 'last_active': '2024-09-12T03:19:56.366974+0000', 'last_peered': '2024-09-12T03:19:56.366974+0000', 'last_clean': '2024-09-12T03:19:56.366974+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:19:56.366974+0000', 'last_undegraded': '2024-09-12T03:19:56.366974+0000', 'last_fullsized': '2024-09-12T03:19:56.366974+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:19:56.366974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:19:56.366974+0000', 'last_clean_scrub_stamp': '2024-09-12T03:19:56.366974+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T03:20:02.103 INFO:tasks.ceph.ceph_manager.ceph:PG 10.2 is not active+clean 2024-09-12T03:20:02.103 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T03:19:56.366974+0000', 'last_change': '2024-09-12T03:19:56.366974+0000', 'last_active': '2024-09-12T03:19:56.366974+0000', 'last_peered': '2024-09-12T03:19:56.366974+0000', 'last_clean': '2024-09-12T03:19:56.366974+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:19:56.366974+0000', 'last_undegraded': '2024-09-12T03:19:56.366974+0000', 'last_fullsized': '2024-09-12T03:19:56.366974+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:19:56.366974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:19:56.366974+0000', 'last_clean_scrub_stamp': '2024-09-12T03:19:56.366974+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T03:20:02.103 INFO:tasks.ceph.ceph_manager.ceph:PG 11.0 is not active+clean 2024-09-12T03:20:02.103 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T03:19:57.371227+0000', 'last_change': '2024-09-12T03:19:57.371227+0000', 'last_active': '2024-09-12T03:19:57.371227+0000', 'last_peered': '2024-09-12T03:19:57.371227+0000', 'last_clean': '2024-09-12T03:19:57.371227+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:19:57.371227+0000', 'last_undegraded': '2024-09-12T03:19:57.371227+0000', 'last_fullsized': '2024-09-12T03:19:57.371227+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:19:57.371227+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:19:57.371227+0000', 'last_clean_scrub_stamp': '2024-09-12T03:19:57.371227+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T03:20:02.103 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-12T03:20:02.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:02.236+0000 7f4147d91640 1 Processor -- start 2024-09-12T03:20:02.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:02.236+0000 7f4147d91640 1 -- start start 2024-09-12T03:20:02.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:02.237+0000 7f4147d91640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f414006f360 0x7f41400cef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:02.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:02.237+0000 7f4147d91640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f41400cf490 0x7f41400d2780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:02.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:02.237+0000 7f4147d91640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f414006cc40 con 0x7f41400b1ca0 2024-09-12T03:20:02.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:02.237+0000 7f4147d91640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f41400d22f0 con 0x7f414006f360 2024-09-12T03:20:02.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:02.237+0000 7f4147d91640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f41400d1ce0 con 0x7f41400cf490 2024-09-12T03:20:02.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:02.237+0000 7f414532c640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f414006f360 0x7f41400cef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:02.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:02.237+0000 7f4145b2d640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f41400b1ca0 0x7f414015f830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:34858/0 (socket says 172.21.15.27:34858) 2024-09-12T03:20:02.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:02.237+0000 7f4145b2d640 1 -- 172.21.15.27:0/2635652253 learned_addr learned my addr 172.21.15.27:0/2635652253 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:20:02.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:02.237+0000 7f414532c640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f414006f360 0x7f41400cef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56234/0 (socket says 172.21.15.27:56234) 2024-09-12T03:20:02.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:02.237+0000 7f414532c640 1 -- 172.21.15.27:0/2635652253 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f41400cf490 msgr2=0x7f41400d2780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:02.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:02.237+0000 7f414532c640 1 --2- 172.21.15.27:0/2635652253 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f41400cf490 0x7f41400d2780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:02.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:02.237+0000 7f414532c640 1 -- 172.21.15.27:0/2635652253 >> v1:172.21.15.27:6789/0 conn(0x7f41400b1ca0 legacy=0x7f414015f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:02.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:02.237+0000 7f4144b2b640 1 -- 172.21.15.27:0/2635652253 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1251173949 0 0) 0x7f414006cc40 con 0x7f41400b1ca0 2024-09-12T03:20:02.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:02.238+0000 7f414532c640 1 -- 172.21.15.27:0/2635652253 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f414006cee0 con 0x7f414006f360 2024-09-12T03:20:02.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:02.238+0000 7f414532c640 1 --2- 172.21.15.27:0/2635652253 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f414006f360 0x7f41400cef50 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f41300097a0 tx=0x7f413002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6886f1f6c4d4c837 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:02.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:02.238+0000 7f4144b2b640 1 -- 172.21.15.27:0/2635652253 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4130037070 con 0x7f414006f360 2024-09-12T03:20:02.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:02.238+0000 7f4144b2b640 1 -- 172.21.15.27:0/2635652253 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4130032440 con 0x7f414006f360 2024-09-12T03:20:02.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:02.238+0000 7f4144b2b640 1 -- 172.21.15.27:0/2635652253 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4130032810 con 0x7f414006f360 2024-09-12T03:20:02.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:02.239+0000 7f4147d91640 1 -- 172.21.15.27:0/2635652253 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f414006f360 msgr2=0x7f41400cef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:02.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:02.239+0000 7f4147d91640 1 --2- 172.21.15.27:0/2635652253 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f414006f360 0x7f41400cef50 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f41300097a0 tx=0x7f413002a180 comp rx=0 tx=0).stop 2024-09-12T03:20:02.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:02.239+0000 7f4147d91640 1 -- 172.21.15.27:0/2635652253 shutdown_connections 2024-09-12T03:20:02.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:02.239+0000 7f4147d91640 1 --2- 172.21.15.27:0/2635652253 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f41400cf490 0x7f41400d2780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:02.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:02.239+0000 7f4147d91640 1 --2- 172.21.15.27:0/2635652253 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f414006f360 0x7f41400cef50 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:02.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:02.239+0000 7f4147d91640 1 -- 172.21.15.27:0/2635652253 >> 172.21.15.27:0/2635652253 conn(0x7f4140073800 msgr2=0x7f414009ba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:02.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:02.239+0000 7f4147d91640 1 -- 172.21.15.27:0/2635652253 shutdown_connections 2024-09-12T03:20:02.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:02.239+0000 7f4147d91640 1 -- 172.21.15.27:0/2635652253 wait complete. 2024-09-12T03:20:02.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:02.240+0000 7f4147d91640 1 Processor -- start 2024-09-12T03:20:02.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:02.240+0000 7f4147d91640 1 -- start start 2024-09-12T03:20:02.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:02.241+0000 7f4147d91640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f414006f360 0x7f414009e480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:02.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:02.241+0000 7f4147d91640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f41400b1ca0 0x7f414009e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:02.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:02.241+0000 7f4147d91640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f41400cf490 0x7f414015f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:02.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:02.241+0000 7f4147d91640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f414006d340 con 0x7f41400cf490 2024-09-12T03:20:02.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:02.241+0000 7f4147d91640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f414006d1c0 con 0x7f414006f360 2024-09-12T03:20:02.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:02.241+0000 7f4147d91640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f41400cfc40 con 0x7f41400b1ca0 2024-09-12T03:20:02.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:02.241+0000 7f414632e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f41400cf490 0x7f414015f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:02.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:02.241+0000 7f414632e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f41400cf490 0x7f414015f9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56244/0 (socket says 172.21.15.27:56244) 2024-09-12T03:20:02.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:02.241+0000 7f414632e640 1 -- 172.21.15.27:0/1144378961 learned_addr learned my addr 172.21.15.27:0/1144378961 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:20:02.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:02.241+0000 7f414532c640 1 --2- 172.21.15.27:0/1144378961 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f41400b1ca0 0x7f414009e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:02.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:02.241+0000 7f414632e640 1 -- 172.21.15.27:0/1144378961 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f41400b1ca0 msgr2=0x7f414009e9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:02.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:02.241+0000 7f4145b2d640 1 --2- 172.21.15.27:0/1144378961 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f414006f360 0x7f414009e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:02.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:02.241+0000 7f414632e640 1 --2- 172.21.15.27:0/1144378961 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f41400b1ca0 0x7f414009e9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:02.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:02.241+0000 7f414632e640 1 -- 172.21.15.27:0/1144378961 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f414006f360 msgr2=0x7f414009e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:02.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:02.241+0000 7f414632e640 1 --2- 172.21.15.27:0/1144378961 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f414006f360 0x7f414009e480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:02.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:02.241+0000 7f414632e640 1 -- 172.21.15.27:0/1144378961 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f414009f1f0 con 0x7f41400cf490 2024-09-12T03:20:02.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:02.242+0000 7f414632e640 1 --2- 172.21.15.27:0/1144378961 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f41400cf490 0x7f414015f9b0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f413800b970 tx=0x7f413800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:02.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:02.242+0000 7f41367fc640 1 -- 172.21.15.27:0/1144378961 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4138019070 con 0x7f41400cf490 2024-09-12T03:20:02.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:02.242+0000 7f41367fc640 1 -- 172.21.15.27:0/1144378961 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f413800cb60 con 0x7f41400cf490 2024-09-12T03:20:02.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:02.242+0000 7f41367fc640 1 -- 172.21.15.27:0/1144378961 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4138014440 con 0x7f41400cf490 2024-09-12T03:20:02.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:02.242+0000 7f4147d91640 1 -- 172.21.15.27:0/1144378961 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f414009f4d0 con 0x7f41400cf490 2024-09-12T03:20:02.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:02.242+0000 7f4147d91640 1 -- 172.21.15.27:0/1144378961 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f414009fa90 con 0x7f41400cf490 2024-09-12T03:20:02.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:02.243+0000 7f4147d91640 1 -- 172.21.15.27:0/1144378961 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41401518d0 con 0x7f41400cf490 2024-09-12T03:20:02.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:02.243+0000 7f41367fc640 1 -- 172.21.15.27:0/1144378961 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 11) ==== 90804+0+0 (secure 0 0 0) 0x7f413801c070 con 0x7f41400cf490 2024-09-12T03:20:02.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:02.244+0000 7f41367fc640 1 --2- 172.21.15.27:0/1144378961 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f412406c190 0x7f412406e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:02.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:02.244+0000 7f41367fc640 1 -- 172.21.15.27:0/1144378961 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 5709+0+0 (secure 0 0 0) 0x7f413808b110 con 0x7f41400cf490 2024-09-12T03:20:02.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:02.244+0000 7f4145b2d640 1 --2- 172.21.15.27:0/1144378961 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f412406c190 0x7f412406e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:02.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:02.245+0000 7f4145b2d640 1 --2- 172.21.15.27:0/1144378961 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f412406c190 0x7f412406e650 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f413c000f60 tx=0x7f413c009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:02.249 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:02.248+0000 7f41367fc640 1 -- 172.21.15.27:0/1144378961 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f41401518d0 con 0x7f41400cf490 2024-09-12T03:20:02.495 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:02.494+0000 7f4147d91640 1 -- 172.21.15.27:0/1144378961 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f414006e720 con 0x7f41400cf490 2024-09-12T03:20:02.495 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:02.494+0000 7f41367fc640 1 -- 172.21.15.27:0/1144378961 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1474 (secure 0 0 0) 0x7f41380596b0 con 0x7f41400cf490 2024-09-12T03:20:02.496 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:20:02.496 INFO:teuthology.orchestra.run.smithi027.stdout:{"fsid":"d2df504e-2647-4385-a5db-9fdf69f487a0","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":234,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":67,"num_osds":8,"num_up_osds":8,"osd_up_since":1726110979,"num_in_osds":8,"osd_in_since":1726110969,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":24,"data_bytes":592778,"bytes_used":258977792,"bytes_avail":772835135488,"bytes_total":773094113280,"write_bytes_sec":2501,"read_op_per_sec":0,"write_op_per_sec":7},"fsmap":{"epoch":38,"btime":"2024-09-12T03:20:00:397329+0000","id":5,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":5,"rank":0,"name":"d","status":"up:active","gid":6066}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":8,"modified":"2024-09-12T03:19:48.100460+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-12T03:20:02.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:02.496+0000 7f4147d91640 1 -- 172.21.15.27:0/1144378961 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f412406c190 msgr2=0x7f412406e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:02.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:02.496+0000 7f4147d91640 1 --2- 172.21.15.27:0/1144378961 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f412406c190 0x7f412406e650 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f413c000f60 tx=0x7f413c009040 comp rx=0 tx=0).stop 2024-09-12T03:20:02.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:02.496+0000 7f4147d91640 1 -- 172.21.15.27:0/1144378961 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f41400cf490 msgr2=0x7f414015f9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:02.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:02.496+0000 7f4147d91640 1 --2- 172.21.15.27:0/1144378961 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f41400cf490 0x7f414015f9b0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f413800b970 tx=0x7f413800be30 comp rx=0 tx=0).stop 2024-09-12T03:20:02.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:02.496+0000 7f4147d91640 1 -- 172.21.15.27:0/1144378961 shutdown_connections 2024-09-12T03:20:02.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:02.496+0000 7f4147d91640 1 --2- 172.21.15.27:0/1144378961 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f412406c190 0x7f412406e650 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:02.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:02.496+0000 7f4147d91640 1 --2- 172.21.15.27:0/1144378961 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f41400cf490 0x7f414015f9b0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:02.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:02.496+0000 7f4147d91640 1 --2- 172.21.15.27:0/1144378961 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f41400b1ca0 0x7f414009e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:02.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:02.496+0000 7f4147d91640 1 --2- 172.21.15.27:0/1144378961 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f414006f360 0x7f414009e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:02.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:02.496+0000 7f4147d91640 1 -- 172.21.15.27:0/1144378961 >> 172.21.15.27:0/1144378961 conn(0x7f4140073800 msgr2=0x7f41400b74e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:02.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:02.496+0000 7f4147d91640 1 -- 172.21.15.27:0/1144378961 shutdown_connections 2024-09-12T03:20:02.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:02.497+0000 7f4147d91640 1 -- 172.21.15.27:0/1144378961 wait complete. 2024-09-12T03:20:02.508 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-09-12T03:20:02.508 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-12T03:20:02.610 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:02.609+0000 7fdc8f3d6640 1 Processor -- start 2024-09-12T03:20:02.610 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:02.609+0000 7fdc8f3d6640 1 -- start start 2024-09-12T03:20:02.613 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:02.610+0000 7fdc8f3d6640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fdc8815eb60 0x7fdc880d1280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:02.613 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:02.610+0000 7fdc8f3d6640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fdc880d17c0 0x7fdc880b3390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:02.613 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:02.610+0000 7fdc8f3d6640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdc880b6690 con 0x7fdc8815e780 2024-09-12T03:20:02.613 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:02.610+0000 7fdc8f3d6640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fdc880d1c30 con 0x7fdc880d17c0 2024-09-12T03:20:02.613 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:02.610+0000 7fdc8f3d6640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fdc880b7950 con 0x7fdc8815eb60 2024-09-12T03:20:02.613 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:02.610+0000 7fdc8d172640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fdc8815e780 0x7fdc880d3e70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:34868/0 (socket says 172.21.15.27:34868) 2024-09-12T03:20:02.613 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:02.610+0000 7fdc8d172640 1 -- 172.21.15.27:0/3752073734 learned_addr learned my addr 172.21.15.27:0/3752073734 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:20:02.614 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:02.610+0000 7fdc8d973640 1 --2- 172.21.15.27:0/3752073734 >> v2:172.21.15.45:3300/0 conn(0x7fdc880d17c0 0x7fdc880b3390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:02.614 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:02.610+0000 7fdc8c971640 1 --2- 172.21.15.27:0/3752073734 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fdc8815eb60 0x7fdc880d1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:02.614 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:02.610+0000 7fdc7ffff640 1 -- 172.21.15.27:0/3752073734 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3545173884 0 0) 0x7fdc880b6690 con 0x7fdc8815e780 2024-09-12T03:20:02.614 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:02.611+0000 7fdc7ffff640 1 -- 172.21.15.27:0/3752073734 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdc74003520 con 0x7fdc8815e780 2024-09-12T03:20:02.614 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:02.611+0000 7fdc7ffff640 1 -- 172.21.15.27:0/3752073734 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2041742619 0 0) 0x7fdc74003520 con 0x7fdc8815e780 2024-09-12T03:20:02.614 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:02.611+0000 7fdc7ffff640 1 -- 172.21.15.27:0/3752073734 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fdc8815eb60 msgr2=0x7fdc880d1280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:02.614 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:02.611+0000 7fdc7ffff640 1 --2- 172.21.15.27:0/3752073734 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fdc8815eb60 0x7fdc880d1280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:02.614 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:02.611+0000 7fdc7ffff640 1 -- 172.21.15.27:0/3752073734 >> v2:172.21.15.45:3300/0 conn(0x7fdc880d17c0 msgr2=0x7fdc880b3390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:02.614 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:02.611+0000 7fdc7ffff640 1 --2- 172.21.15.27:0/3752073734 >> v2:172.21.15.45:3300/0 conn(0x7fdc880d17c0 0x7fdc880b3390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:02.614 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:02.611+0000 7fdc7ffff640 1 -- 172.21.15.27:0/3752073734 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc880b6970 con 0x7fdc8815e780 2024-09-12T03:20:02.614 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:02.611+0000 7fdc7ffff640 1 -- 172.21.15.27:0/3752073734 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 847755213 0 0) 0x7fdc84001670 con 0x7fdc8815e780 2024-09-12T03:20:02.614 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:02.611+0000 7fdc7ffff640 1 -- 172.21.15.27:0/3752073734 >> v1:172.21.15.27:6789/0 conn(0x7fdc8815e780 legacy=0x7fdc880d3e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:02.614 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:02.611+0000 7fdc7ffff640 1 --2- 172.21.15.27:0/3752073734 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdc740039b0 0x7fdc74003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:02.614 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:02.611+0000 7fdc7ffff640 1 --2- 172.21.15.27:0/3752073734 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdc74004800 0x7fdc74004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:02.615 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:02.611+0000 7fdc7ffff640 1 --2- 172.21.15.27:0/3752073734 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fdc8815eb60 0x7fdc740051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:02.615 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:02.611+0000 7fdc7ffff640 1 -- 172.21.15.27:0/3752073734 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fdc880d1c30 con 0x7fdc74004800 2024-09-12T03:20:02.615 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:02.611+0000 7fdc7ffff640 1 -- 172.21.15.27:0/3752073734 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fdc7400b9c0 con 0x7fdc740039b0 2024-09-12T03:20:02.615 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:02.611+0000 7fdc7ffff640 1 -- 172.21.15.27:0/3752073734 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fdc7400bb40 con 0x7fdc8815eb60 2024-09-12T03:20:02.615 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:02.611+0000 7fdc8c971640 1 --2- 172.21.15.27:0/3752073734 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdc74004800 0x7fdc74004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:02.615 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:02.611+0000 7fdc8d172640 1 --2- 172.21.15.27:0/3752073734 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdc740039b0 0x7fdc74003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:02.615 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:02.611+0000 7fdc8d973640 1 --2- 172.21.15.27:0/3752073734 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fdc8815eb60 0x7fdc740051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:02.615 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:02.611+0000 7fdc8c971640 1 -- 172.21.15.27:0/3752073734 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fdc8815eb60 msgr2=0x7fdc740051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:02.615 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:02.611+0000 7fdc8c971640 1 --2- 172.21.15.27:0/3752073734 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fdc8815eb60 0x7fdc740051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:02.615 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:02.611+0000 7fdc8c971640 1 -- 172.21.15.27:0/3752073734 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdc740039b0 msgr2=0x7fdc74003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:02.615 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:02.611+0000 7fdc8c971640 1 --2- 172.21.15.27:0/3752073734 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdc740039b0 0x7fdc74003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:02.615 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:02.611+0000 7fdc8c971640 1 -- 172.21.15.27:0/3752073734 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc74003520 con 0x7fdc74004800 2024-09-12T03:20:02.615 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:02.611+0000 7fdc8d973640 1 --2- 172.21.15.27:0/3752073734 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fdc8815eb60 0x7fdc740051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:20:02.616 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:02.612+0000 7fdc8c971640 1 --2- 172.21.15.27:0/3752073734 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdc74004800 0x7fdc74004c80 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fdc78009ac0 tx=0x7fdc74005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c08de1b2439ab74 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:02.616 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:02.612+0000 7fdc7ffff640 1 -- 172.21.15.27:0/3752073734 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc78036070 con 0x7fdc74004800 2024-09-12T03:20:02.616 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:02.612+0000 7fdc7ffff640 1 -- 172.21.15.27:0/3752073734 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc78031400 con 0x7fdc74004800 2024-09-12T03:20:02.616 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:02.612+0000 7fdc7ffff640 1 -- 172.21.15.27:0/3752073734 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc78009690 con 0x7fdc74004800 2024-09-12T03:20:02.619 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:02.613+0000 7fdc8f3d6640 1 -- 172.21.15.27:0/3752073734 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdc74004800 msgr2=0x7fdc74004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:02.619 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:02.613+0000 7fdc8f3d6640 1 --2- 172.21.15.27:0/3752073734 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdc74004800 0x7fdc74004c80 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fdc78009ac0 tx=0x7fdc74005830 comp rx=0 tx=0).stop 2024-09-12T03:20:02.619 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:02.613+0000 7fdc8d973640 1 -- 172.21.15.27:0/3752073734 reap_dead start 2024-09-12T03:20:02.619 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:02.613+0000 7fdc8f3d6640 1 -- 172.21.15.27:0/3752073734 shutdown_connections 2024-09-12T03:20:02.619 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:02.613+0000 7fdc8f3d6640 1 -- 172.21.15.27:0/3752073734 >> 172.21.15.27:0/3752073734 conn(0x7fdc8806da60 msgr2=0x7fdc8806de70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:02.619 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:02.613+0000 7fdc8f3d6640 1 -- 172.21.15.27:0/3752073734 shutdown_connections 2024-09-12T03:20:02.619 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:02.613+0000 7fdc8f3d6640 1 -- 172.21.15.27:0/3752073734 wait complete. 2024-09-12T03:20:02.619 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:02.614+0000 7fdc8f3d6640 1 Processor -- start 2024-09-12T03:20:02.619 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:02.614+0000 7fdc8f3d6640 1 -- start start 2024-09-12T03:20:02.619 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:02.615+0000 7fdc8f3d6640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdc74004800 0x7fdc881f7500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:02.619 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:02.615+0000 7fdc8f3d6640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fdc881f7ea0 0x7fdc880cdaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:02.619 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:02.615+0000 7fdc8f3d6640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdc880ce030 0x7fdc8815a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:02.620 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:02.615+0000 7fdc8f3d6640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fdc880b6f30 con 0x7fdc880ce030 2024-09-12T03:20:02.620 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:02.615+0000 7fdc8f3d6640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fdc880b7210 con 0x7fdc74004800 2024-09-12T03:20:02.620 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:02.615+0000 7fdc8f3d6640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fdc880b6340 con 0x7fdc881f7ea0 2024-09-12T03:20:02.620 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:02.615+0000 7fdc8d973640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdc880ce030 0x7fdc8815a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:02.620 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:02.615+0000 7fdc8d973640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdc880ce030 0x7fdc8815a700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56256/0 (socket says 172.21.15.27:56256) 2024-09-12T03:20:02.620 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:02.615+0000 7fdc8d973640 1 -- 172.21.15.27:0/1264666530 learned_addr learned my addr 172.21.15.27:0/1264666530 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:20:02.620 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:02.615+0000 7fdc8c971640 1 --2- 172.21.15.27:0/1264666530 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fdc881f7ea0 0x7fdc880cdaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:02.620 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:02.615+0000 7fdc8d172640 1 --2- 172.21.15.27:0/1264666530 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdc74004800 0x7fdc881f7500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:02.620 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:02.615+0000 7fdc8d973640 1 -- 172.21.15.27:0/1264666530 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fdc881f7ea0 msgr2=0x7fdc880cdaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:02.620 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:02.615+0000 7fdc8d973640 1 --2- 172.21.15.27:0/1264666530 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fdc881f7ea0 0x7fdc880cdaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:02.620 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:02.615+0000 7fdc8d973640 1 -- 172.21.15.27:0/1264666530 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdc74004800 msgr2=0x7fdc881f7500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:02.620 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:02.615+0000 7fdc8d973640 1 --2- 172.21.15.27:0/1264666530 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdc74004800 0x7fdc881f7500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:02.620 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:02.615+0000 7fdc8d973640 1 -- 172.21.15.27:0/1264666530 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc880ccb60 con 0x7fdc880ce030 2024-09-12T03:20:02.621 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:02.616+0000 7fdc8d973640 1 --2- 172.21.15.27:0/1264666530 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdc880ce030 0x7fdc8815a700 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fdc88162480 tx=0x7fdc80010590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:02.621 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:02.616+0000 7fdc7dffb640 1 -- 172.21.15.27:0/1264666530 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc8001a030 con 0x7fdc880ce030 2024-09-12T03:20:02.621 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:02.616+0000 7fdc7dffb640 1 -- 172.21.15.27:0/1264666530 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc80010e00 con 0x7fdc880ce030 2024-09-12T03:20:02.621 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:02.616+0000 7fdc8f3d6640 1 -- 172.21.15.27:0/1264666530 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc880cce40 con 0x7fdc880ce030 2024-09-12T03:20:02.621 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:02.616+0000 7fdc7dffb640 1 -- 172.21.15.27:0/1264666530 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc80003bd0 con 0x7fdc880ce030 2024-09-12T03:20:02.621 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:02.616+0000 7fdc8f3d6640 1 -- 172.21.15.27:0/1264666530 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc881f7a40 con 0x7fdc880ce030 2024-09-12T03:20:02.621 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:02.617+0000 7fdc8f3d6640 1 -- 172.21.15.27:0/1264666530 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc880d1280 con 0x7fdc880ce030 2024-09-12T03:20:02.621 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:02.617+0000 7fdc7dffb640 1 -- 172.21.15.27:0/1264666530 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 11) ==== 90804+0+0 (secure 0 0 0) 0x7fdc8001d070 con 0x7fdc880ce030 2024-09-12T03:20:02.621 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:02.618+0000 7fdc7dffb640 1 --2- 172.21.15.27:0/1264666530 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fdc6c06c190 0x7fdc6c06e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:02.621 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:02.618+0000 7fdc7dffb640 1 -- 172.21.15.27:0/1264666530 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 5709+0+0 (secure 0 0 0) 0x7fdc8008ba10 con 0x7fdc880ce030 2024-09-12T03:20:02.622 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:02.618+0000 7fdc8d172640 1 --2- 172.21.15.27:0/1264666530 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fdc6c06c190 0x7fdc6c06e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:02.622 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:02.619+0000 7fdc8d172640 1 --2- 172.21.15.27:0/1264666530 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fdc6c06c190 0x7fdc6c06e650 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fdc880b5ad0 tx=0x7fdc84007970 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:02.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:02.622+0000 7fdc7dffb640 1 -- 172.21.15.27:0/1264666530 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fdc80059f30 con 0x7fdc880ce030 2024-09-12T03:20:02.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:02.797+0000 7fdc8f3d6640 1 -- 172.21.15.27:0/1264666530 --> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fdc88069ff0 con 0x7fdc6c06c190 2024-09-12T03:20:02.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:02.804+0000 7fdc7dffb640 1 -- 172.21.15.27:0/1264666530 <== mgr.4102 v2:172.21.15.45:6832/3923821219 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252065 (secure 0 0 0) 0x7fdc88069ff0 con 0x7fdc6c06c190 2024-09-12T03:20:02.807 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:20:02.808 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2024-09-12T03:20:02.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:02.806+0000 7fdc8f3d6640 1 -- 172.21.15.27:0/1264666530 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fdc6c06c190 msgr2=0x7fdc6c06e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:02.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:02.806+0000 7fdc8f3d6640 1 --2- 172.21.15.27:0/1264666530 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fdc6c06c190 0x7fdc6c06e650 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fdc880b5ad0 tx=0x7fdc84007970 comp rx=0 tx=0).stop 2024-09-12T03:20:02.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:02.806+0000 7fdc8f3d6640 1 -- 172.21.15.27:0/1264666530 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdc880ce030 msgr2=0x7fdc8815a700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:02.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:02.806+0000 7fdc8f3d6640 1 --2- 172.21.15.27:0/1264666530 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdc880ce030 0x7fdc8815a700 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fdc88162480 tx=0x7fdc80010590 comp rx=0 tx=0).stop 2024-09-12T03:20:02.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:02.806+0000 7fdc8f3d6640 1 -- 172.21.15.27:0/1264666530 shutdown_connections 2024-09-12T03:20:02.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:02.806+0000 7fdc8f3d6640 1 --2- 172.21.15.27:0/1264666530 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fdc6c06c190 0x7fdc6c06e650 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:02.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:02.806+0000 7fdc8f3d6640 1 --2- 172.21.15.27:0/1264666530 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdc880ce030 0x7fdc8815a700 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:02.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:02.806+0000 7fdc8f3d6640 1 --2- 172.21.15.27:0/1264666530 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fdc881f7ea0 0x7fdc880cdaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:02.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:02.806+0000 7fdc8f3d6640 1 --2- 172.21.15.27:0/1264666530 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdc74004800 0x7fdc881f7500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:02.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:02.806+0000 7fdc8f3d6640 1 -- 172.21.15.27:0/1264666530 >> 172.21.15.27:0/1264666530 conn(0x7fdc8806da60 msgr2=0x7fdc8806de40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:02.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:02.806+0000 7fdc8f3d6640 1 -- 172.21.15.27:0/1264666530 shutdown_connections 2024-09-12T03:20:02.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:02.806+0000 7fdc8f3d6640 1 -- 172.21.15.27:0/1264666530 wait complete. 2024-09-12T03:20:02.818 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":171,"stamp":"2024-09-12T03:20:02.103994+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":104,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":252908,"kb_used_data":7524,"kb_used_omap":12,"kb_used_meta":245171,"kb_avail":754721812,"statfs":{"total":773094113280,"available":772835135488,"internally_reserved":0,"allocated":7704576,"data_stored":3881496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":251055664},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.731928"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.412295+0000","last_change":"2024-09-12T03:19:58.405738+0000","last_active":"2024-09-12T03:19:59.412295+0000","last_peered":"2024-09-12T03:19:59.412295+0000","last_clean":"2024-09-12T03:19:59.412295+0000","last_became_active":"2024-09-12T03:19:58.405044+0000","last_became_peered":"2024-09-12T03:19:58.405044+0000","last_unstale":"2024-09-12T03:19:59.412295+0000","last_undegraded":"2024-09-12T03:19:59.412295+0000","last_fullsized":"2024-09-12T03:19:59.412295+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:24:50.892593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.412848+0000","last_change":"2024-09-12T03:19:58.407057+0000","last_active":"2024-09-12T03:19:59.412848+0000","last_peered":"2024-09-12T03:19:59.412848+0000","last_clean":"2024-09-12T03:19:59.412848+0000","last_became_active":"2024-09-12T03:19:58.406455+0000","last_became_peered":"2024-09-12T03:19:58.406455+0000","last_unstale":"2024-09-12T03:19:59.412848+0000","last_undegraded":"2024-09-12T03:19:59.412848+0000","last_fullsized":"2024-09-12T03:19:59.412848+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:12:06.463949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.411816+0000","last_change":"2024-09-12T03:19:58.402689+0000","last_active":"2024-09-12T03:19:59.411816+0000","last_peered":"2024-09-12T03:19:59.411816+0000","last_clean":"2024-09-12T03:19:59.411816+0000","last_became_active":"2024-09-12T03:19:58.402424+0000","last_became_peered":"2024-09-12T03:19:58.402424+0000","last_unstale":"2024-09-12T03:19:59.411816+0000","last_undegraded":"2024-09-12T03:19:59.411816+0000","last_fullsized":"2024-09-12T03:19:59.411816+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:01:53.466856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.411556+0000","last_change":"2024-09-12T03:19:58.407823+0000","last_active":"2024-09-12T03:19:59.411556+0000","last_peered":"2024-09-12T03:19:59.411556+0000","last_clean":"2024-09-12T03:19:59.411556+0000","last_became_active":"2024-09-12T03:19:58.407533+0000","last_became_peered":"2024-09-12T03:19:58.407533+0000","last_unstale":"2024-09-12T03:19:59.411556+0000","last_undegraded":"2024-09-12T03:19:59.411556+0000","last_fullsized":"2024-09-12T03:19:59.411556+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:35:40.539826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.411395+0000","last_change":"2024-09-12T03:19:58.405012+0000","last_active":"2024-09-12T03:19:59.411395+0000","last_peered":"2024-09-12T03:19:59.411395+0000","last_clean":"2024-09-12T03:19:59.411395+0000","last_became_active":"2024-09-12T03:19:58.404706+0000","last_became_peered":"2024-09-12T03:19:58.404706+0000","last_unstale":"2024-09-12T03:19:59.411395+0000","last_undegraded":"2024-09-12T03:19:59.411395+0000","last_fullsized":"2024-09-12T03:19:59.411395+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:13:43.693628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.412119+0000","last_change":"2024-09-12T03:19:58.404152+0000","last_active":"2024-09-12T03:19:59.412119+0000","last_peered":"2024-09-12T03:19:59.412119+0000","last_clean":"2024-09-12T03:19:59.412119+0000","last_became_active":"2024-09-12T03:19:58.403865+0000","last_became_peered":"2024-09-12T03:19:58.403865+0000","last_unstale":"2024-09-12T03:19:59.412119+0000","last_undegraded":"2024-09-12T03:19:59.412119+0000","last_fullsized":"2024-09-12T03:19:59.412119+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:03:35.086339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.399545+0000","last_change":"2024-09-12T03:19:58.405356+0000","last_active":"2024-09-12T03:19:59.399545+0000","last_peered":"2024-09-12T03:19:59.399545+0000","last_clean":"2024-09-12T03:19:59.399545+0000","last_became_active":"2024-09-12T03:19:58.405079+0000","last_became_peered":"2024-09-12T03:19:58.405079+0000","last_unstale":"2024-09-12T03:19:59.399545+0000","last_undegraded":"2024-09-12T03:19:59.399545+0000","last_fullsized":"2024-09-12T03:19:59.399545+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:26:48.602769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.412001+0000","last_change":"2024-09-12T03:19:58.408692+0000","last_active":"2024-09-12T03:19:59.412001+0000","last_peered":"2024-09-12T03:19:59.412001+0000","last_clean":"2024-09-12T03:19:59.412001+0000","last_became_active":"2024-09-12T03:19:58.408182+0000","last_became_peered":"2024-09-12T03:19:58.408182+0000","last_unstale":"2024-09-12T03:19:59.412001+0000","last_undegraded":"2024-09-12T03:19:59.412001+0000","last_fullsized":"2024-09-12T03:19:59.412001+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:25:56.007148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.412647+0000","last_change":"2024-09-12T03:19:58.408054+0000","last_active":"2024-09-12T03:19:59.412647+0000","last_peered":"2024-09-12T03:19:59.412647+0000","last_clean":"2024-09-12T03:19:59.412647+0000","last_became_active":"2024-09-12T03:19:58.407439+0000","last_became_peered":"2024-09-12T03:19:58.407439+0000","last_unstale":"2024-09-12T03:19:59.412647+0000","last_undegraded":"2024-09-12T03:19:59.412647+0000","last_fullsized":"2024-09-12T03:19:59.412647+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:52:55.350315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.399858+0000","last_change":"2024-09-12T03:19:58.400878+0000","last_active":"2024-09-12T03:19:59.399858+0000","last_peered":"2024-09-12T03:19:59.399858+0000","last_clean":"2024-09-12T03:19:59.399858+0000","last_became_active":"2024-09-12T03:19:58.400601+0000","last_became_peered":"2024-09-12T03:19:58.400601+0000","last_unstale":"2024-09-12T03:19:59.399858+0000","last_undegraded":"2024-09-12T03:19:59.399858+0000","last_fullsized":"2024-09-12T03:19:59.399858+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:53:33.025355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.412388+0000","last_change":"2024-09-12T03:19:58.404349+0000","last_active":"2024-09-12T03:19:59.412388+0000","last_peered":"2024-09-12T03:19:59.412388+0000","last_clean":"2024-09-12T03:19:59.412388+0000","last_became_active":"2024-09-12T03:19:58.404067+0000","last_became_peered":"2024-09-12T03:19:58.404067+0000","last_unstale":"2024-09-12T03:19:59.412388+0000","last_undegraded":"2024-09-12T03:19:59.412388+0000","last_fullsized":"2024-09-12T03:19:59.412388+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:31:06.800562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.411695+0000","last_change":"2024-09-12T03:19:58.398948+0000","last_active":"2024-09-12T03:19:59.411695+0000","last_peered":"2024-09-12T03:19:59.411695+0000","last_clean":"2024-09-12T03:19:59.411695+0000","last_became_active":"2024-09-12T03:19:58.398583+0000","last_became_peered":"2024-09-12T03:19:58.398583+0000","last_unstale":"2024-09-12T03:19:59.411695+0000","last_undegraded":"2024-09-12T03:19:59.411695+0000","last_fullsized":"2024-09-12T03:19:59.411695+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:14:21.530213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.398467+0000","last_change":"2024-09-12T03:19:58.408147+0000","last_active":"2024-09-12T03:19:59.398467+0000","last_peered":"2024-09-12T03:19:59.398467+0000","last_clean":"2024-09-12T03:19:59.398467+0000","last_became_active":"2024-09-12T03:19:58.407855+0000","last_became_peered":"2024-09-12T03:19:58.407855+0000","last_unstale":"2024-09-12T03:19:59.398467+0000","last_undegraded":"2024-09-12T03:19:59.398467+0000","last_fullsized":"2024-09-12T03:19:59.398467+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:50:40.770697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.412570+0000","last_change":"2024-09-12T03:19:58.401979+0000","last_active":"2024-09-12T03:19:59.412570+0000","last_peered":"2024-09-12T03:19:59.412570+0000","last_clean":"2024-09-12T03:19:59.412570+0000","last_became_active":"2024-09-12T03:19:58.401689+0000","last_became_peered":"2024-09-12T03:19:58.401689+0000","last_unstale":"2024-09-12T03:19:59.412570+0000","last_undegraded":"2024-09-12T03:19:59.412570+0000","last_fullsized":"2024-09-12T03:19:59.412570+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:22:57.280328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.398306+0000","last_change":"2024-09-12T03:19:58.399312+0000","last_active":"2024-09-12T03:19:59.398306+0000","last_peered":"2024-09-12T03:19:59.398306+0000","last_clean":"2024-09-12T03:19:59.398306+0000","last_became_active":"2024-09-12T03:19:58.399050+0000","last_became_peered":"2024-09-12T03:19:58.399050+0000","last_unstale":"2024-09-12T03:19:59.398306+0000","last_undegraded":"2024-09-12T03:19:59.398306+0000","last_fullsized":"2024-09-12T03:19:59.398306+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:41:13.177460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.400875+0000","last_change":"2024-09-12T03:19:58.404580+0000","last_active":"2024-09-12T03:19:59.400875+0000","last_peered":"2024-09-12T03:19:59.400875+0000","last_clean":"2024-09-12T03:19:59.400875+0000","last_became_active":"2024-09-12T03:19:58.404320+0000","last_became_peered":"2024-09-12T03:19:58.404320+0000","last_unstale":"2024-09-12T03:19:59.400875+0000","last_undegraded":"2024-09-12T03:19:59.400875+0000","last_fullsized":"2024-09-12T03:19:59.400875+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:41:51.158105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.412705+0000","last_change":"2024-09-12T03:19:58.408389+0000","last_active":"2024-09-12T03:19:59.412705+0000","last_peered":"2024-09-12T03:19:59.412705+0000","last_clean":"2024-09-12T03:19:59.412705+0000","last_became_active":"2024-09-12T03:19:58.407797+0000","last_became_peered":"2024-09-12T03:19:58.407797+0000","last_unstale":"2024-09-12T03:19:59.412705+0000","last_undegraded":"2024-09-12T03:19:59.412705+0000","last_fullsized":"2024-09-12T03:19:59.412705+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:17:24.698351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.411598+0000","last_change":"2024-09-12T03:19:58.409159+0000","last_active":"2024-09-12T03:19:59.411598+0000","last_peered":"2024-09-12T03:19:59.411598+0000","last_clean":"2024-09-12T03:19:59.411598+0000","last_became_active":"2024-09-12T03:19:58.408851+0000","last_became_peered":"2024-09-12T03:19:58.408851+0000","last_unstale":"2024-09-12T03:19:59.411598+0000","last_undegraded":"2024-09-12T03:19:59.411598+0000","last_fullsized":"2024-09-12T03:19:59.411598+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:23:52.073974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.412173+0000","last_change":"2024-09-12T03:19:58.408115+0000","last_active":"2024-09-12T03:19:59.412173+0000","last_peered":"2024-09-12T03:19:59.412173+0000","last_clean":"2024-09-12T03:19:59.412173+0000","last_became_active":"2024-09-12T03:19:58.407745+0000","last_became_peered":"2024-09-12T03:19:58.407745+0000","last_unstale":"2024-09-12T03:19:59.412173+0000","last_undegraded":"2024-09-12T03:19:59.412173+0000","last_fullsized":"2024-09-12T03:19:59.412173+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:50:17.515387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.398554+0000","last_change":"2024-09-12T03:19:58.398467+0000","last_active":"2024-09-12T03:19:59.398554+0000","last_peered":"2024-09-12T03:19:59.398554+0000","last_clean":"2024-09-12T03:19:59.398554+0000","last_became_active":"2024-09-12T03:19:58.398152+0000","last_became_peered":"2024-09-12T03:19:58.398152+0000","last_unstale":"2024-09-12T03:19:59.398554+0000","last_undegraded":"2024-09-12T03:19:59.398554+0000","last_fullsized":"2024-09-12T03:19:59.398554+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:24:50.892593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.400458+0000","last_change":"2024-09-12T03:19:58.400541+0000","last_active":"2024-09-12T03:19:59.400458+0000","last_peered":"2024-09-12T03:19:59.400458+0000","last_clean":"2024-09-12T03:19:59.400458+0000","last_became_active":"2024-09-12T03:19:58.399602+0000","last_became_peered":"2024-09-12T03:19:58.399602+0000","last_unstale":"2024-09-12T03:19:59.400458+0000","last_undegraded":"2024-09-12T03:19:59.400458+0000","last_fullsized":"2024-09-12T03:19:59.400458+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:56:37.761014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.400370+0000","last_change":"2024-09-12T03:19:58.399427+0000","last_active":"2024-09-12T03:19:59.400370+0000","last_peered":"2024-09-12T03:19:59.400370+0000","last_clean":"2024-09-12T03:19:59.400370+0000","last_became_active":"2024-09-12T03:19:58.398464+0000","last_became_peered":"2024-09-12T03:19:58.398464+0000","last_unstale":"2024-09-12T03:19:59.400370+0000","last_undegraded":"2024-09-12T03:19:59.400370+0000","last_fullsized":"2024-09-12T03:19:59.400370+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:44:19.209045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.411588+0000","last_change":"2024-09-12T03:19:58.405456+0000","last_active":"2024-09-12T03:19:59.411588+0000","last_peered":"2024-09-12T03:19:59.411588+0000","last_clean":"2024-09-12T03:19:59.411588+0000","last_became_active":"2024-09-12T03:19:58.404974+0000","last_became_peered":"2024-09-12T03:19:58.404974+0000","last_unstale":"2024-09-12T03:19:59.411588+0000","last_undegraded":"2024-09-12T03:19:59.411588+0000","last_fullsized":"2024-09-12T03:19:59.411588+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:58:58.914180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.411477+0000","last_change":"2024-09-12T03:19:58.408893+0000","last_active":"2024-09-12T03:19:59.411477+0000","last_peered":"2024-09-12T03:19:59.411477+0000","last_clean":"2024-09-12T03:19:59.411477+0000","last_became_active":"2024-09-12T03:19:58.408589+0000","last_became_peered":"2024-09-12T03:19:58.408589+0000","last_unstale":"2024-09-12T03:19:59.411477+0000","last_undegraded":"2024-09-12T03:19:59.411477+0000","last_fullsized":"2024-09-12T03:19:59.411477+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:54:48.020501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.400037+0000","last_change":"2024-09-12T03:19:58.411515+0000","last_active":"2024-09-12T03:19:59.400037+0000","last_peered":"2024-09-12T03:19:59.400037+0000","last_clean":"2024-09-12T03:19:59.400037+0000","last_became_active":"2024-09-12T03:19:58.409091+0000","last_became_peered":"2024-09-12T03:19:58.409091+0000","last_unstale":"2024-09-12T03:19:59.400037+0000","last_undegraded":"2024-09-12T03:19:59.400037+0000","last_fullsized":"2024-09-12T03:19:59.400037+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:25:42.004151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.400164+0000","last_change":"2024-09-12T03:19:58.410854+0000","last_active":"2024-09-12T03:19:59.400164+0000","last_peered":"2024-09-12T03:19:59.400164+0000","last_clean":"2024-09-12T03:19:59.400164+0000","last_became_active":"2024-09-12T03:19:58.410556+0000","last_became_peered":"2024-09-12T03:19:58.410556+0000","last_unstale":"2024-09-12T03:19:59.400164+0000","last_undegraded":"2024-09-12T03:19:59.400164+0000","last_fullsized":"2024-09-12T03:19:59.400164+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:16:53.699272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.411966+0000","last_change":"2024-09-12T03:19:58.406104+0000","last_active":"2024-09-12T03:19:59.411966+0000","last_peered":"2024-09-12T03:19:59.411966+0000","last_clean":"2024-09-12T03:19:59.411966+0000","last_became_active":"2024-09-12T03:19:58.405546+0000","last_became_peered":"2024-09-12T03:19:58.405546+0000","last_unstale":"2024-09-12T03:19:59.411966+0000","last_undegraded":"2024-09-12T03:19:59.411966+0000","last_fullsized":"2024-09-12T03:19:59.411966+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:45:16.630721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.400401+0000","last_change":"2024-09-12T03:19:58.401764+0000","last_active":"2024-09-12T03:19:59.400401+0000","last_peered":"2024-09-12T03:19:59.400401+0000","last_clean":"2024-09-12T03:19:59.400401+0000","last_became_active":"2024-09-12T03:19:58.397972+0000","last_became_peered":"2024-09-12T03:19:58.397972+0000","last_unstale":"2024-09-12T03:19:59.400401+0000","last_undegraded":"2024-09-12T03:19:59.400401+0000","last_fullsized":"2024-09-12T03:19:59.400401+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:50:11.761046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.412120+0000","last_change":"2024-09-12T03:19:58.403549+0000","last_active":"2024-09-12T03:19:59.412120+0000","last_peered":"2024-09-12T03:19:59.412120+0000","last_clean":"2024-09-12T03:19:59.412120+0000","last_became_active":"2024-09-12T03:19:58.403294+0000","last_became_peered":"2024-09-12T03:19:58.403294+0000","last_unstale":"2024-09-12T03:19:59.412120+0000","last_undegraded":"2024-09-12T03:19:59.412120+0000","last_fullsized":"2024-09-12T03:19:59.412120+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:20:56.214083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.399515+0000","last_change":"2024-09-12T03:19:58.408095+0000","last_active":"2024-09-12T03:19:59.399515+0000","last_peered":"2024-09-12T03:19:59.399515+0000","last_clean":"2024-09-12T03:19:59.399515+0000","last_became_active":"2024-09-12T03:19:58.407819+0000","last_became_peered":"2024-09-12T03:19:58.407819+0000","last_unstale":"2024-09-12T03:19:59.399515+0000","last_undegraded":"2024-09-12T03:19:59.399515+0000","last_fullsized":"2024-09-12T03:19:59.399515+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:43:53.939725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.399784+0000","last_change":"2024-09-12T03:19:58.399638+0000","last_active":"2024-09-12T03:19:59.399784+0000","last_peered":"2024-09-12T03:19:59.399784+0000","last_clean":"2024-09-12T03:19:59.399784+0000","last_became_active":"2024-09-12T03:19:58.399356+0000","last_became_peered":"2024-09-12T03:19:58.399356+0000","last_unstale":"2024-09-12T03:19:59.399784+0000","last_undegraded":"2024-09-12T03:19:59.399784+0000","last_fullsized":"2024-09-12T03:19:59.399784+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:10:16.728909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.412055+0000","last_change":"2024-09-12T03:19:58.407063+0000","last_active":"2024-09-12T03:19:59.412055+0000","last_peered":"2024-09-12T03:19:59.412055+0000","last_clean":"2024-09-12T03:19:59.412055+0000","last_became_active":"2024-09-12T03:19:58.406116+0000","last_became_peered":"2024-09-12T03:19:58.406116+0000","last_unstale":"2024-09-12T03:19:59.412055+0000","last_undegraded":"2024-09-12T03:19:59.412055+0000","last_fullsized":"2024-09-12T03:19:59.412055+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:11:39.227764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.411730+0000","last_change":"2024-09-12T03:19:58.409587+0000","last_active":"2024-09-12T03:19:59.411730+0000","last_peered":"2024-09-12T03:19:59.411730+0000","last_clean":"2024-09-12T03:19:59.411730+0000","last_became_active":"2024-09-12T03:19:58.409275+0000","last_became_peered":"2024-09-12T03:19:58.409275+0000","last_unstale":"2024-09-12T03:19:59.411730+0000","last_undegraded":"2024-09-12T03:19:59.411730+0000","last_fullsized":"2024-09-12T03:19:59.411730+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:00:46.656402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"66'4","reported_seq":18,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.412596+0000","last_change":"2024-09-12T03:19:57.394595+0000","last_active":"2024-09-12T03:19:59.412596+0000","last_peered":"2024-09-12T03:19:59.412596+0000","last_clean":"2024-09-12T03:19:59.412596+0000","last_became_active":"2024-09-12T03:19:57.394294+0000","last_became_peered":"2024-09-12T03:19:57.394294+0000","last_unstale":"2024-09-12T03:19:59.412596+0000","last_undegraded":"2024-09-12T03:19:59.412596+0000","last_fullsized":"2024-09-12T03:19:59.412596+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_clean_scrub_stamp":"2024-09-12T03:19:56.366974+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:51:58.062647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.412771+0000","last_change":"2024-09-12T03:19:58.407115+0000","last_active":"2024-09-12T03:19:59.412771+0000","last_peered":"2024-09-12T03:19:59.412771+0000","last_clean":"2024-09-12T03:19:59.412771+0000","last_became_active":"2024-09-12T03:19:58.406683+0000","last_became_peered":"2024-09-12T03:19:58.406683+0000","last_unstale":"2024-09-12T03:19:59.412771+0000","last_undegraded":"2024-09-12T03:19:59.412771+0000","last_fullsized":"2024-09-12T03:19:59.412771+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:36:57.331135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"66'1","reported_seq":15,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.398595+0000","last_change":"2024-09-12T03:19:57.389982+0000","last_active":"2024-09-12T03:19:59.398595+0000","last_peered":"2024-09-12T03:19:59.398595+0000","last_clean":"2024-09-12T03:19:59.398595+0000","last_became_active":"2024-09-12T03:19:57.389694+0000","last_became_peered":"2024-09-12T03:19:57.389694+0000","last_unstale":"2024-09-12T03:19:59.398595+0000","last_undegraded":"2024-09-12T03:19:59.398595+0000","last_fullsized":"2024-09-12T03:19:59.398595+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_clean_scrub_stamp":"2024-09-12T03:19:56.366974+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:01:26.160193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.412792+0000","last_change":"2024-09-12T03:19:58.415175+0000","last_active":"2024-09-12T03:19:59.412792+0000","last_peered":"2024-09-12T03:19:59.412792+0000","last_clean":"2024-09-12T03:19:59.412792+0000","last_became_active":"2024-09-12T03:19:58.414899+0000","last_became_peered":"2024-09-12T03:19:58.414899+0000","last_unstale":"2024-09-12T03:19:59.412792+0000","last_undegraded":"2024-09-12T03:19:59.412792+0000","last_fullsized":"2024-09-12T03:19:59.412792+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:36:33.065448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"66'2","reported_seq":16,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.400437+0000","last_change":"2024-09-12T03:19:57.393946+0000","last_active":"2024-09-12T03:19:59.400437+0000","last_peered":"2024-09-12T03:19:59.400437+0000","last_clean":"2024-09-12T03:19:59.400437+0000","last_became_active":"2024-09-12T03:19:57.393672+0000","last_became_peered":"2024-09-12T03:19:57.393672+0000","last_unstale":"2024-09-12T03:19:59.400437+0000","last_undegraded":"2024-09-12T03:19:59.400437+0000","last_fullsized":"2024-09-12T03:19:59.400437+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_clean_scrub_stamp":"2024-09-12T03:19:56.366974+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:57:23.221949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.412495+0000","last_change":"2024-09-12T03:19:58.403124+0000","last_active":"2024-09-12T03:19:59.412495+0000","last_peered":"2024-09-12T03:19:59.412495+0000","last_clean":"2024-09-12T03:19:59.412495+0000","last_became_active":"2024-09-12T03:19:58.402859+0000","last_became_peered":"2024-09-12T03:19:58.402859+0000","last_unstale":"2024-09-12T03:19:59.412495+0000","last_undegraded":"2024-09-12T03:19:59.412495+0000","last_fullsized":"2024-09-12T03:19:59.412495+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:41:26.586794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"66'2","reported_seq":16,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.412896+0000","last_change":"2024-09-12T03:19:57.396585+0000","last_active":"2024-09-12T03:19:59.412896+0000","last_peered":"2024-09-12T03:19:59.412896+0000","last_clean":"2024-09-12T03:19:59.412896+0000","last_became_active":"2024-09-12T03:19:57.396109+0000","last_became_peered":"2024-09-12T03:19:57.396109+0000","last_unstale":"2024-09-12T03:19:59.412896+0000","last_undegraded":"2024-09-12T03:19:59.412896+0000","last_fullsized":"2024-09-12T03:19:59.412896+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_clean_scrub_stamp":"2024-09-12T03:19:56.366974+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:25:50.078358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.399545+0000","last_change":"2024-09-12T03:19:58.406747+0000","last_active":"2024-09-12T03:19:59.399545+0000","last_peered":"2024-09-12T03:19:59.399545+0000","last_clean":"2024-09-12T03:19:59.399545+0000","last_became_active":"2024-09-12T03:19:58.406464+0000","last_became_peered":"2024-09-12T03:19:58.406464+0000","last_unstale":"2024-09-12T03:19:59.399545+0000","last_undegraded":"2024-09-12T03:19:59.399545+0000","last_fullsized":"2024-09-12T03:19:59.399545+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:43:08.336624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"66'1","reported_seq":15,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.412362+0000","last_change":"2024-09-12T03:19:57.391056+0000","last_active":"2024-09-12T03:19:59.412362+0000","last_peered":"2024-09-12T03:19:59.412362+0000","last_clean":"2024-09-12T03:19:59.412362+0000","last_became_active":"2024-09-12T03:19:57.390769+0000","last_became_peered":"2024-09-12T03:19:57.390769+0000","last_unstale":"2024-09-12T03:19:59.412362+0000","last_undegraded":"2024-09-12T03:19:59.412362+0000","last_fullsized":"2024-09-12T03:19:59.412362+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_clean_scrub_stamp":"2024-09-12T03:19:56.366974+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:41:12.173208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.400570+0000","last_change":"2024-09-12T03:19:58.415098+0000","last_active":"2024-09-12T03:19:59.400570+0000","last_peered":"2024-09-12T03:19:59.400570+0000","last_clean":"2024-09-12T03:19:59.400570+0000","last_became_active":"2024-09-12T03:19:58.411028+0000","last_became_peered":"2024-09-12T03:19:58.411028+0000","last_unstale":"2024-09-12T03:19:59.400570+0000","last_undegraded":"2024-09-12T03:19:59.400570+0000","last_fullsized":"2024-09-12T03:19:59.400570+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:29:23.148491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"66'1","reported_seq":15,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.412238+0000","last_change":"2024-09-12T03:19:57.395730+0000","last_active":"2024-09-12T03:19:59.412238+0000","last_peered":"2024-09-12T03:19:59.412238+0000","last_clean":"2024-09-12T03:19:59.412238+0000","last_became_active":"2024-09-12T03:19:57.395253+0000","last_became_peered":"2024-09-12T03:19:57.395253+0000","last_unstale":"2024-09-12T03:19:59.412238+0000","last_undegraded":"2024-09-12T03:19:59.412238+0000","last_fullsized":"2024-09-12T03:19:59.412238+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_clean_scrub_stamp":"2024-09-12T03:19:56.366974+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:54:51.158313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.411683+0000","last_change":"2024-09-12T03:19:58.400222+0000","last_active":"2024-09-12T03:19:59.411683+0000","last_peered":"2024-09-12T03:19:59.411683+0000","last_clean":"2024-09-12T03:19:59.411683+0000","last_became_active":"2024-09-12T03:19:58.399906+0000","last_became_peered":"2024-09-12T03:19:58.399906+0000","last_unstale":"2024-09-12T03:19:59.411683+0000","last_undegraded":"2024-09-12T03:19:59.411683+0000","last_fullsized":"2024-09-12T03:19:59.411683+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:17:22.493344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.411395+0000","last_change":"2024-09-12T03:19:57.390855+0000","last_active":"2024-09-12T03:19:59.411395+0000","last_peered":"2024-09-12T03:19:59.411395+0000","last_clean":"2024-09-12T03:19:59.411395+0000","last_became_active":"2024-09-12T03:19:57.390307+0000","last_became_peered":"2024-09-12T03:19:57.390307+0000","last_unstale":"2024-09-12T03:19:59.411395+0000","last_undegraded":"2024-09-12T03:19:59.411395+0000","last_fullsized":"2024-09-12T03:19:59.411395+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_clean_scrub_stamp":"2024-09-12T03:19:56.366974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:33:55.147981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.398361+0000","last_change":"2024-09-12T03:19:58.405943+0000","last_active":"2024-09-12T03:19:59.398361+0000","last_peered":"2024-09-12T03:19:59.398361+0000","last_clean":"2024-09-12T03:19:59.398361+0000","last_became_active":"2024-09-12T03:19:58.405630+0000","last_became_peered":"2024-09-12T03:19:58.405630+0000","last_unstale":"2024-09-12T03:19:59.398361+0000","last_undegraded":"2024-09-12T03:19:59.398361+0000","last_fullsized":"2024-09-12T03:19:59.398361+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:00:46.656402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.411776+0000","last_change":"2024-09-12T03:19:57.394447+0000","last_active":"2024-09-12T03:19:59.411776+0000","last_peered":"2024-09-12T03:19:59.411776+0000","last_clean":"2024-09-12T03:19:59.411776+0000","last_became_active":"2024-09-12T03:19:57.394153+0000","last_became_peered":"2024-09-12T03:19:57.394153+0000","last_unstale":"2024-09-12T03:19:59.411776+0000","last_undegraded":"2024-09-12T03:19:59.411776+0000","last_fullsized":"2024-09-12T03:19:59.411776+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_clean_scrub_stamp":"2024-09-12T03:19:56.366974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:12:19.700994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.400317+0000","last_change":"2024-09-12T03:19:58.412227+0000","last_active":"2024-09-12T03:19:59.400317+0000","last_peered":"2024-09-12T03:19:59.400317+0000","last_clean":"2024-09-12T03:19:59.400317+0000","last_became_active":"2024-09-12T03:19:58.411951+0000","last_became_peered":"2024-09-12T03:19:58.411951+0000","last_unstale":"2024-09-12T03:19:59.400317+0000","last_undegraded":"2024-09-12T03:19:59.400317+0000","last_fullsized":"2024-09-12T03:19:59.400317+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:54:24.310119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.400155+0000","last_change":"2024-09-12T03:19:57.393730+0000","last_active":"2024-09-12T03:19:59.400155+0000","last_peered":"2024-09-12T03:19:59.400155+0000","last_clean":"2024-09-12T03:19:59.400155+0000","last_became_active":"2024-09-12T03:19:57.393425+0000","last_became_peered":"2024-09-12T03:19:57.393425+0000","last_unstale":"2024-09-12T03:19:59.400155+0000","last_undegraded":"2024-09-12T03:19:59.400155+0000","last_fullsized":"2024-09-12T03:19:59.400155+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_clean_scrub_stamp":"2024-09-12T03:19:56.366974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:39:01.426373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.412571+0000","last_change":"2024-09-12T03:19:58.409130+0000","last_active":"2024-09-12T03:19:59.412571+0000","last_peered":"2024-09-12T03:19:59.412571+0000","last_clean":"2024-09-12T03:19:59.412571+0000","last_became_active":"2024-09-12T03:19:58.408838+0000","last_became_peered":"2024-09-12T03:19:58.408838+0000","last_unstale":"2024-09-12T03:19:59.412571+0000","last_undegraded":"2024-09-12T03:19:59.412571+0000","last_fullsized":"2024-09-12T03:19:59.412571+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:38:31.191391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.412967+0000","last_change":"2024-09-12T03:19:57.391703+0000","last_active":"2024-09-12T03:19:59.412967+0000","last_peered":"2024-09-12T03:19:59.412967+0000","last_clean":"2024-09-12T03:19:59.412967+0000","last_became_active":"2024-09-12T03:19:57.391345+0000","last_became_peered":"2024-09-12T03:19:57.391345+0000","last_unstale":"2024-09-12T03:19:59.412967+0000","last_undegraded":"2024-09-12T03:19:59.412967+0000","last_fullsized":"2024-09-12T03:19:59.412967+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_clean_scrub_stamp":"2024-09-12T03:19:56.366974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:05:09.764085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.412786+0000","last_change":"2024-09-12T03:19:58.410704+0000","last_active":"2024-09-12T03:19:59.412786+0000","last_peered":"2024-09-12T03:19:59.412786+0000","last_clean":"2024-09-12T03:19:59.412786+0000","last_became_active":"2024-09-12T03:19:58.410390+0000","last_became_peered":"2024-09-12T03:19:58.410390+0000","last_unstale":"2024-09-12T03:19:59.412786+0000","last_undegraded":"2024-09-12T03:19:59.412786+0000","last_fullsized":"2024-09-12T03:19:59.412786+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:27:21.762076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"66'3","reported_seq":17,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.412074+0000","last_change":"2024-09-12T03:19:57.383973+0000","last_active":"2024-09-12T03:19:59.412074+0000","last_peered":"2024-09-12T03:19:59.412074+0000","last_clean":"2024-09-12T03:19:59.412074+0000","last_became_active":"2024-09-12T03:19:57.383481+0000","last_became_peered":"2024-09-12T03:19:57.383481+0000","last_unstale":"2024-09-12T03:19:59.412074+0000","last_undegraded":"2024-09-12T03:19:59.412074+0000","last_fullsized":"2024-09-12T03:19:59.412074+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_clean_scrub_stamp":"2024-09-12T03:19:56.366974+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:00:45.652150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.411674+0000","last_change":"2024-09-12T03:19:58.400811+0000","last_active":"2024-09-12T03:19:59.411674+0000","last_peered":"2024-09-12T03:19:59.411674+0000","last_clean":"2024-09-12T03:19:59.411674+0000","last_became_active":"2024-09-12T03:19:58.400542+0000","last_became_peered":"2024-09-12T03:19:58.400542+0000","last_unstale":"2024-09-12T03:19:59.411674+0000","last_undegraded":"2024-09-12T03:19:59.411674+0000","last_fullsized":"2024-09-12T03:19:59.411674+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:01:16.275564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"66'6","reported_seq":20,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.411922+0000","last_change":"2024-09-12T03:19:57.390457+0000","last_active":"2024-09-12T03:19:59.411922+0000","last_peered":"2024-09-12T03:19:59.411922+0000","last_clean":"2024-09-12T03:19:59.411922+0000","last_became_active":"2024-09-12T03:19:57.390109+0000","last_became_peered":"2024-09-12T03:19:57.390109+0000","last_unstale":"2024-09-12T03:19:59.411922+0000","last_undegraded":"2024-09-12T03:19:59.411922+0000","last_fullsized":"2024-09-12T03:19:59.411922+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_clean_scrub_stamp":"2024-09-12T03:19:56.366974+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:43:23.673876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.411683+0000","last_change":"2024-09-12T03:19:58.402659+0000","last_active":"2024-09-12T03:19:59.411683+0000","last_peered":"2024-09-12T03:19:59.411683+0000","last_clean":"2024-09-12T03:19:59.411683+0000","last_became_active":"2024-09-12T03:19:58.402358+0000","last_became_peered":"2024-09-12T03:19:58.402358+0000","last_unstale":"2024-09-12T03:19:59.411683+0000","last_undegraded":"2024-09-12T03:19:59.411683+0000","last_fullsized":"2024-09-12T03:19:59.411683+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:41:13.177460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"66'2","reported_seq":16,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.411862+0000","last_change":"2024-09-12T03:19:57.391726+0000","last_active":"2024-09-12T03:19:59.411862+0000","last_peered":"2024-09-12T03:19:59.411862+0000","last_clean":"2024-09-12T03:19:59.411862+0000","last_became_active":"2024-09-12T03:19:57.391107+0000","last_became_peered":"2024-09-12T03:19:57.391107+0000","last_unstale":"2024-09-12T03:19:59.411862+0000","last_undegraded":"2024-09-12T03:19:59.411862+0000","last_fullsized":"2024-09-12T03:19:59.411862+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_clean_scrub_stamp":"2024-09-12T03:19:56.366974+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:41:29.366042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.399734+0000","last_change":"2024-09-12T03:19:58.394023+0000","last_active":"2024-09-12T03:19:59.399734+0000","last_peered":"2024-09-12T03:19:59.399734+0000","last_clean":"2024-09-12T03:19:59.399734+0000","last_became_active":"2024-09-12T03:19:58.393630+0000","last_became_peered":"2024-09-12T03:19:58.393630+0000","last_unstale":"2024-09-12T03:19:59.399734+0000","last_undegraded":"2024-09-12T03:19:59.399734+0000","last_fullsized":"2024-09-12T03:19:59.399734+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:17:36.830688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.412349+0000","last_change":"2024-09-12T03:19:57.395906+0000","last_active":"2024-09-12T03:19:59.412349+0000","last_peered":"2024-09-12T03:19:59.412349+0000","last_clean":"2024-09-12T03:19:59.412349+0000","last_became_active":"2024-09-12T03:19:57.395558+0000","last_became_peered":"2024-09-12T03:19:57.395558+0000","last_unstale":"2024-09-12T03:19:59.412349+0000","last_undegraded":"2024-09-12T03:19:59.412349+0000","last_fullsized":"2024-09-12T03:19:59.412349+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_clean_scrub_stamp":"2024-09-12T03:19:56.366974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:23:25.739217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.398297+0000","last_change":"2024-09-12T03:19:58.407844+0000","last_active":"2024-09-12T03:19:59.398297+0000","last_peered":"2024-09-12T03:19:59.398297+0000","last_clean":"2024-09-12T03:19:59.398297+0000","last_became_active":"2024-09-12T03:19:58.407513+0000","last_became_peered":"2024-09-12T03:19:58.407513+0000","last_unstale":"2024-09-12T03:19:59.398297+0000","last_undegraded":"2024-09-12T03:19:59.398297+0000","last_fullsized":"2024-09-12T03:19:59.398297+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:58:58.914180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.399665+0000","last_change":"2024-09-12T03:19:57.384562+0000","last_active":"2024-09-12T03:19:59.399665+0000","last_peered":"2024-09-12T03:19:59.399665+0000","last_clean":"2024-09-12T03:19:59.399665+0000","last_became_active":"2024-09-12T03:19:57.384227+0000","last_became_peered":"2024-09-12T03:19:57.384227+0000","last_unstale":"2024-09-12T03:19:59.399665+0000","last_undegraded":"2024-09-12T03:19:59.399665+0000","last_fullsized":"2024-09-12T03:19:59.399665+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_clean_scrub_stamp":"2024-09-12T03:19:56.366974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:52:55.095626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.412013+0000","last_change":"2024-09-12T03:19:58.406880+0000","last_active":"2024-09-12T03:19:59.412013+0000","last_peered":"2024-09-12T03:19:59.412013+0000","last_clean":"2024-09-12T03:19:59.412013+0000","last_became_active":"2024-09-12T03:19:58.406585+0000","last_became_peered":"2024-09-12T03:19:58.406585+0000","last_unstale":"2024-09-12T03:19:59.412013+0000","last_undegraded":"2024-09-12T03:19:59.412013+0000","last_fullsized":"2024-09-12T03:19:59.412013+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:44:18.879486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"66'8","reported_seq":22,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.400987+0000","last_change":"2024-09-12T03:19:57.397454+0000","last_active":"2024-09-12T03:19:59.400987+0000","last_peered":"2024-09-12T03:19:59.400987+0000","last_clean":"2024-09-12T03:19:59.400987+0000","last_became_active":"2024-09-12T03:19:57.397126+0000","last_became_peered":"2024-09-12T03:19:57.397126+0000","last_unstale":"2024-09-12T03:19:59.400987+0000","last_undegraded":"2024-09-12T03:19:59.400987+0000","last_fullsized":"2024-09-12T03:19:59.400987+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_clean_scrub_stamp":"2024-09-12T03:19:56.366974+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:14:13.101395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.400608+0000","last_change":"2024-09-12T03:19:58.402681+0000","last_active":"2024-09-12T03:19:59.400608+0000","last_peered":"2024-09-12T03:19:59.400608+0000","last_clean":"2024-09-12T03:19:59.400608+0000","last_became_active":"2024-09-12T03:19:58.402409+0000","last_became_peered":"2024-09-12T03:19:58.402409+0000","last_unstale":"2024-09-12T03:19:59.400608+0000","last_undegraded":"2024-09-12T03:19:59.400608+0000","last_fullsized":"2024-09-12T03:19:59.400608+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:56:05.472430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"66'1","reported_seq":15,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.412766+0000","last_change":"2024-09-12T03:19:57.394555+0000","last_active":"2024-09-12T03:19:59.412766+0000","last_peered":"2024-09-12T03:19:59.412766+0000","last_clean":"2024-09-12T03:19:59.412766+0000","last_became_active":"2024-09-12T03:19:57.394277+0000","last_became_peered":"2024-09-12T03:19:57.394277+0000","last_unstale":"2024-09-12T03:19:59.412766+0000","last_undegraded":"2024-09-12T03:19:59.412766+0000","last_fullsized":"2024-09-12T03:19:59.412766+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_clean_scrub_stamp":"2024-09-12T03:19:56.366974+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:50:52.123837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.412469+0000","last_change":"2024-09-12T03:19:58.411847+0000","last_active":"2024-09-12T03:19:59.412469+0000","last_peered":"2024-09-12T03:19:59.412469+0000","last_clean":"2024-09-12T03:19:59.412469+0000","last_became_active":"2024-09-12T03:19:58.409965+0000","last_became_peered":"2024-09-12T03:19:58.409965+0000","last_unstale":"2024-09-12T03:19:59.412469+0000","last_undegraded":"2024-09-12T03:19:59.412469+0000","last_fullsized":"2024-09-12T03:19:59.412469+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:45:22.536315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"66'2","reported_seq":16,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.400471+0000","last_change":"2024-09-12T03:19:57.394701+0000","last_active":"2024-09-12T03:19:59.400471+0000","last_peered":"2024-09-12T03:19:59.400471+0000","last_clean":"2024-09-12T03:19:59.400471+0000","last_became_active":"2024-09-12T03:19:57.394348+0000","last_became_peered":"2024-09-12T03:19:57.394348+0000","last_unstale":"2024-09-12T03:19:59.400471+0000","last_undegraded":"2024-09-12T03:19:59.400471+0000","last_fullsized":"2024-09-12T03:19:59.400471+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_clean_scrub_stamp":"2024-09-12T03:19:56.366974+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:44:36.098558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.411827+0000","last_change":"2024-09-12T03:19:58.400111+0000","last_active":"2024-09-12T03:19:59.411827+0000","last_peered":"2024-09-12T03:19:59.411827+0000","last_clean":"2024-09-12T03:19:59.411827+0000","last_became_active":"2024-09-12T03:19:58.399388+0000","last_became_peered":"2024-09-12T03:19:58.399388+0000","last_unstale":"2024-09-12T03:19:59.411827+0000","last_undegraded":"2024-09-12T03:19:59.411827+0000","last_fullsized":"2024-09-12T03:19:59.411827+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:06:24.498091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"66'1","reported_seq":15,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.412373+0000","last_change":"2024-09-12T03:19:57.397975+0000","last_active":"2024-09-12T03:19:59.412373+0000","last_peered":"2024-09-12T03:19:59.412373+0000","last_clean":"2024-09-12T03:19:59.412373+0000","last_became_active":"2024-09-12T03:19:57.397637+0000","last_became_peered":"2024-09-12T03:19:57.397637+0000","last_unstale":"2024-09-12T03:19:59.412373+0000","last_undegraded":"2024-09-12T03:19:59.412373+0000","last_fullsized":"2024-09-12T03:19:59.412373+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_clean_scrub_stamp":"2024-09-12T03:19:56.366974+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:16:35.102685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.411820+0000","last_change":"2024-09-12T03:19:58.399501+0000","last_active":"2024-09-12T03:19:59.411820+0000","last_peered":"2024-09-12T03:19:59.411820+0000","last_clean":"2024-09-12T03:19:59.411820+0000","last_became_active":"2024-09-12T03:19:58.399203+0000","last_became_peered":"2024-09-12T03:19:58.399203+0000","last_unstale":"2024-09-12T03:19:59.411820+0000","last_undegraded":"2024-09-12T03:19:59.411820+0000","last_fullsized":"2024-09-12T03:19:59.411820+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:45:17.689061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.412050+0000","last_change":"2024-09-12T03:19:57.394026+0000","last_active":"2024-09-12T03:19:59.412050+0000","last_peered":"2024-09-12T03:19:59.412050+0000","last_clean":"2024-09-12T03:19:59.412050+0000","last_became_active":"2024-09-12T03:19:57.393679+0000","last_became_peered":"2024-09-12T03:19:57.393679+0000","last_unstale":"2024-09-12T03:19:59.412050+0000","last_undegraded":"2024-09-12T03:19:59.412050+0000","last_fullsized":"2024-09-12T03:19:59.412050+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_clean_scrub_stamp":"2024-09-12T03:19:56.366974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:41:50.153852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.412261+0000","last_change":"2024-09-12T03:19:58.407764+0000","last_active":"2024-09-12T03:19:59.412261+0000","last_peered":"2024-09-12T03:19:59.412261+0000","last_clean":"2024-09-12T03:19:59.412261+0000","last_became_active":"2024-09-12T03:19:58.407483+0000","last_became_peered":"2024-09-12T03:19:58.407483+0000","last_unstale":"2024-09-12T03:19:59.412261+0000","last_undegraded":"2024-09-12T03:19:59.412261+0000","last_fullsized":"2024-09-12T03:19:59.412261+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:11:19.244120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.412443+0000","last_change":"2024-09-12T03:19:57.394831+0000","last_active":"2024-09-12T03:19:59.412443+0000","last_peered":"2024-09-12T03:19:59.412443+0000","last_clean":"2024-09-12T03:19:59.412443+0000","last_became_active":"2024-09-12T03:19:57.394505+0000","last_became_peered":"2024-09-12T03:19:57.394505+0000","last_unstale":"2024-09-12T03:19:59.412443+0000","last_undegraded":"2024-09-12T03:19:59.412443+0000","last_fullsized":"2024-09-12T03:19:59.412443+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_clean_scrub_stamp":"2024-09-12T03:19:56.366974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:59:35.357929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.411253+0000","last_change":"2024-09-12T03:19:58.408696+0000","last_active":"2024-09-12T03:19:59.411253+0000","last_peered":"2024-09-12T03:19:59.411253+0000","last_clean":"2024-09-12T03:19:59.411253+0000","last_became_active":"2024-09-12T03:19:58.408393+0000","last_became_peered":"2024-09-12T03:19:58.408393+0000","last_unstale":"2024-09-12T03:19:59.411253+0000","last_undegraded":"2024-09-12T03:19:59.411253+0000","last_fullsized":"2024-09-12T03:19:59.411253+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:15:02.240922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.399840+0000","last_change":"2024-09-12T03:19:57.390326+0000","last_active":"2024-09-12T03:19:59.399840+0000","last_peered":"2024-09-12T03:19:59.399840+0000","last_clean":"2024-09-12T03:19:59.399840+0000","last_became_active":"2024-09-12T03:19:57.390046+0000","last_became_peered":"2024-09-12T03:19:57.390046+0000","last_unstale":"2024-09-12T03:19:59.399840+0000","last_undegraded":"2024-09-12T03:19:59.399840+0000","last_fullsized":"2024-09-12T03:19:59.399840+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_clean_scrub_stamp":"2024-09-12T03:19:56.366974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:11:38.450285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.411564+0000","last_change":"2024-09-12T03:19:58.409403+0000","last_active":"2024-09-12T03:19:59.411564+0000","last_peered":"2024-09-12T03:19:59.411564+0000","last_clean":"2024-09-12T03:19:59.411564+0000","last_became_active":"2024-09-12T03:19:58.409087+0000","last_became_peered":"2024-09-12T03:19:58.409087+0000","last_unstale":"2024-09-12T03:19:59.411564+0000","last_undegraded":"2024-09-12T03:19:59.411564+0000","last_fullsized":"2024-09-12T03:19:59.411564+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:44:15.824316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.398237+0000","last_change":"2024-09-12T03:19:57.394055+0000","last_active":"2024-09-12T03:19:59.398237+0000","last_peered":"2024-09-12T03:19:59.398237+0000","last_clean":"2024-09-12T03:19:59.398237+0000","last_became_active":"2024-09-12T03:19:57.393767+0000","last_became_peered":"2024-09-12T03:19:57.393767+0000","last_unstale":"2024-09-12T03:19:59.398237+0000","last_undegraded":"2024-09-12T03:19:59.398237+0000","last_fullsized":"2024-09-12T03:19:59.398237+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_clean_scrub_stamp":"2024-09-12T03:19:56.366974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:34:00.504515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.398615+0000","last_change":"2024-09-12T03:19:58.400064+0000","last_active":"2024-09-12T03:19:59.398615+0000","last_peered":"2024-09-12T03:19:59.398615+0000","last_clean":"2024-09-12T03:19:59.398615+0000","last_became_active":"2024-09-12T03:19:58.399777+0000","last_became_peered":"2024-09-12T03:19:58.399777+0000","last_unstale":"2024-09-12T03:19:59.398615+0000","last_undegraded":"2024-09-12T03:19:59.398615+0000","last_fullsized":"2024-09-12T03:19:59.398615+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:31:06.800562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"66'2","reported_seq":16,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.412883+0000","last_change":"2024-09-12T03:19:57.393966+0000","last_active":"2024-09-12T03:19:59.412883+0000","last_peered":"2024-09-12T03:19:59.412883+0000","last_clean":"2024-09-12T03:19:59.412883+0000","last_became_active":"2024-09-12T03:19:57.393680+0000","last_became_peered":"2024-09-12T03:19:57.393680+0000","last_unstale":"2024-09-12T03:19:59.412883+0000","last_undegraded":"2024-09-12T03:19:59.412883+0000","last_fullsized":"2024-09-12T03:19:59.412883+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_clean_scrub_stamp":"2024-09-12T03:19:56.366974+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:05:00.306420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.400071+0000","last_change":"2024-09-12T03:19:58.404813+0000","last_active":"2024-09-12T03:19:59.400071+0000","last_peered":"2024-09-12T03:19:59.400071+0000","last_clean":"2024-09-12T03:19:59.400071+0000","last_became_active":"2024-09-12T03:19:58.404482+0000","last_became_peered":"2024-09-12T03:19:58.404482+0000","last_unstale":"2024-09-12T03:19:59.400071+0000","last_undegraded":"2024-09-12T03:19:59.400071+0000","last_fullsized":"2024-09-12T03:19:59.400071+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:11:28.704109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"67'1","reported_seq":15,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.403756+0000","last_change":"2024-09-12T03:19:57.386098+0000","last_active":"2024-09-12T03:19:59.403756+0000","last_peered":"2024-09-12T03:19:59.403756+0000","last_clean":"2024-09-12T03:19:59.403756+0000","last_became_active":"2024-09-12T03:19:57.384254+0000","last_became_peered":"2024-09-12T03:19:57.384254+0000","last_unstale":"2024-09-12T03:19:59.403756+0000","last_undegraded":"2024-09-12T03:19:59.403756+0000","last_fullsized":"2024-09-12T03:19:59.403756+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_clean_scrub_stamp":"2024-09-12T03:19:56.366974+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:40:18.965709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.411946+0000","last_change":"2024-09-12T03:19:58.408958+0000","last_active":"2024-09-12T03:19:59.411946+0000","last_peered":"2024-09-12T03:19:59.411946+0000","last_clean":"2024-09-12T03:19:59.411946+0000","last_became_active":"2024-09-12T03:19:58.408665+0000","last_became_peered":"2024-09-12T03:19:58.408665+0000","last_unstale":"2024-09-12T03:19:59.411946+0000","last_undegraded":"2024-09-12T03:19:59.411946+0000","last_fullsized":"2024-09-12T03:19:59.411946+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:50:40.770697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.411911+0000","last_change":"2024-09-12T03:19:57.393827+0000","last_active":"2024-09-12T03:19:59.411911+0000","last_peered":"2024-09-12T03:19:59.411911+0000","last_clean":"2024-09-12T03:19:59.411911+0000","last_became_active":"2024-09-12T03:19:57.393501+0000","last_became_peered":"2024-09-12T03:19:57.393501+0000","last_unstale":"2024-09-12T03:19:59.411911+0000","last_undegraded":"2024-09-12T03:19:59.411911+0000","last_fullsized":"2024-09-12T03:19:59.411911+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_clean_scrub_stamp":"2024-09-12T03:19:56.366974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:00:49.820829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.412577+0000","last_change":"2024-09-12T03:19:58.416479+0000","last_active":"2024-09-12T03:19:59.412577+0000","last_peered":"2024-09-12T03:19:59.412577+0000","last_clean":"2024-09-12T03:19:59.412577+0000","last_became_active":"2024-09-12T03:19:58.416145+0000","last_became_peered":"2024-09-12T03:19:58.416145+0000","last_unstale":"2024-09-12T03:19:59.412577+0000","last_undegraded":"2024-09-12T03:19:59.412577+0000","last_fullsized":"2024-09-12T03:19:59.412577+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:37:31.629037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.398503+0000","last_change":"2024-09-12T03:19:57.392451+0000","last_active":"2024-09-12T03:19:59.398503+0000","last_peered":"2024-09-12T03:19:59.398503+0000","last_clean":"2024-09-12T03:19:59.398503+0000","last_became_active":"2024-09-12T03:19:57.392127+0000","last_became_peered":"2024-09-12T03:19:57.392127+0000","last_unstale":"2024-09-12T03:19:59.398503+0000","last_undegraded":"2024-09-12T03:19:59.398503+0000","last_fullsized":"2024-09-12T03:19:59.398503+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_clean_scrub_stamp":"2024-09-12T03:19:56.366974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:11:02.225246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.412704+0000","last_change":"2024-09-12T03:19:58.407515+0000","last_active":"2024-09-12T03:19:59.412704+0000","last_peered":"2024-09-12T03:19:59.412704+0000","last_clean":"2024-09-12T03:19:59.412704+0000","last_became_active":"2024-09-12T03:19:58.407189+0000","last_became_peered":"2024-09-12T03:19:58.407189+0000","last_unstale":"2024-09-12T03:19:59.412704+0000","last_undegraded":"2024-09-12T03:19:59.412704+0000","last_fullsized":"2024-09-12T03:19:59.412704+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:57:01.617936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.411502+0000","last_change":"2024-09-12T03:19:57.391167+0000","last_active":"2024-09-12T03:19:59.411502+0000","last_peered":"2024-09-12T03:19:59.411502+0000","last_clean":"2024-09-12T03:19:59.411502+0000","last_became_active":"2024-09-12T03:19:57.390901+0000","last_became_peered":"2024-09-12T03:19:57.390901+0000","last_unstale":"2024-09-12T03:19:59.411502+0000","last_undegraded":"2024-09-12T03:19:59.411502+0000","last_fullsized":"2024-09-12T03:19:59.411502+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_clean_scrub_stamp":"2024-09-12T03:19:56.366974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:09:37.761223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.412332+0000","last_change":"2024-09-12T03:19:58.420294+0000","last_active":"2024-09-12T03:19:59.412332+0000","last_peered":"2024-09-12T03:19:59.412332+0000","last_clean":"2024-09-12T03:19:59.412332+0000","last_became_active":"2024-09-12T03:19:58.418912+0000","last_became_peered":"2024-09-12T03:19:58.418912+0000","last_unstale":"2024-09-12T03:19:59.412332+0000","last_undegraded":"2024-09-12T03:19:59.412332+0000","last_fullsized":"2024-09-12T03:19:59.412332+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:14:39.136845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.412515+0000","last_change":"2024-09-12T03:19:57.392009+0000","last_active":"2024-09-12T03:19:59.412515+0000","last_peered":"2024-09-12T03:19:59.412515+0000","last_clean":"2024-09-12T03:19:59.412515+0000","last_became_active":"2024-09-12T03:19:57.391668+0000","last_became_peered":"2024-09-12T03:19:57.391668+0000","last_unstale":"2024-09-12T03:19:59.412515+0000","last_undegraded":"2024-09-12T03:19:59.412515+0000","last_fullsized":"2024-09-12T03:19:59.412515+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_clean_scrub_stamp":"2024-09-12T03:19:56.366974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:41:36.777515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.411339+0000","last_change":"2024-09-12T03:19:58.403003+0000","last_active":"2024-09-12T03:19:59.411339+0000","last_peered":"2024-09-12T03:19:59.411339+0000","last_clean":"2024-09-12T03:19:59.411339+0000","last_became_active":"2024-09-12T03:19:58.402682+0000","last_became_peered":"2024-09-12T03:19:58.402682+0000","last_unstale":"2024-09-12T03:19:59.411339+0000","last_undegraded":"2024-09-12T03:19:59.411339+0000","last_fullsized":"2024-09-12T03:19:59.411339+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:55:09.544823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.399794+0000","last_change":"2024-09-12T03:19:57.393150+0000","last_active":"2024-09-12T03:19:59.399794+0000","last_peered":"2024-09-12T03:19:59.399794+0000","last_clean":"2024-09-12T03:19:59.399794+0000","last_became_active":"2024-09-12T03:19:57.392692+0000","last_became_peered":"2024-09-12T03:19:57.392692+0000","last_unstale":"2024-09-12T03:19:59.399794+0000","last_undegraded":"2024-09-12T03:19:59.399794+0000","last_fullsized":"2024-09-12T03:19:59.399794+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_clean_scrub_stamp":"2024-09-12T03:19:56.366974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:44:27.441871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.411918+0000","last_change":"2024-09-12T03:19:58.407787+0000","last_active":"2024-09-12T03:19:59.411918+0000","last_peered":"2024-09-12T03:19:59.411918+0000","last_clean":"2024-09-12T03:19:59.411918+0000","last_became_active":"2024-09-12T03:19:58.407407+0000","last_became_peered":"2024-09-12T03:19:58.407407+0000","last_unstale":"2024-09-12T03:19:59.411918+0000","last_undegraded":"2024-09-12T03:19:59.411918+0000","last_fullsized":"2024-09-12T03:19:59.411918+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:38:14.768525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.412289+0000","last_change":"2024-09-12T03:19:57.391520+0000","last_active":"2024-09-12T03:19:59.412289+0000","last_peered":"2024-09-12T03:19:59.412289+0000","last_clean":"2024-09-12T03:19:59.412289+0000","last_became_active":"2024-09-12T03:19:57.391140+0000","last_became_peered":"2024-09-12T03:19:57.391140+0000","last_unstale":"2024-09-12T03:19:59.412289+0000","last_undegraded":"2024-09-12T03:19:59.412289+0000","last_fullsized":"2024-09-12T03:19:59.412289+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_clean_scrub_stamp":"2024-09-12T03:19:56.366974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:42:32.285383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.412286+0000","last_change":"2024-09-12T03:19:58.409818+0000","last_active":"2024-09-12T03:19:59.412286+0000","last_peered":"2024-09-12T03:19:59.412286+0000","last_clean":"2024-09-12T03:19:59.412286+0000","last_became_active":"2024-09-12T03:19:58.409507+0000","last_became_peered":"2024-09-12T03:19:58.409507+0000","last_unstale":"2024-09-12T03:19:59.412286+0000","last_undegraded":"2024-09-12T03:19:59.412286+0000","last_fullsized":"2024-09-12T03:19:59.412286+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:21:24.731904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"66'2","reported_seq":16,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.412444+0000","last_change":"2024-09-12T03:19:57.395758+0000","last_active":"2024-09-12T03:19:59.412444+0000","last_peered":"2024-09-12T03:19:59.412444+0000","last_clean":"2024-09-12T03:19:59.412444+0000","last_became_active":"2024-09-12T03:19:57.395422+0000","last_became_peered":"2024-09-12T03:19:57.395422+0000","last_unstale":"2024-09-12T03:19:59.412444+0000","last_undegraded":"2024-09-12T03:19:59.412444+0000","last_fullsized":"2024-09-12T03:19:59.412444+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_clean_scrub_stamp":"2024-09-12T03:19:56.366974+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:18:13.412357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":252,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.411416+0000","last_change":"2024-09-12T03:16:21.883446+0000","last_active":"2024-09-12T03:19:59.411416+0000","last_peered":"2024-09-12T03:19:59.411416+0000","last_clean":"2024-09-12T03:19:59.411416+0000","last_became_active":"2024-09-12T03:16:21.882207+0000","last_became_peered":"2024-09-12T03:16:21.882207+0000","last_unstale":"2024-09-12T03:19:59.411416+0000","last_undegraded":"2024-09-12T03:19:59.411416+0000","last_fullsized":"2024-09-12T03:19:59.411416+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:20.316426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:20.316426+0000","last_clean_scrub_stamp":"2024-09-12T03:16:20.316426+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:12:15.424320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542190,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36716,"kb_used_data":1368,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335124,"statfs":{"total":96636764160,"available":96599166976,"internally_reserved":0,"allocated":1400832,"data_stored":924039,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542192,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31996,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339844,"statfs":{"total":96636764160,"available":96604000256,"internally_reserved":0,"allocated":819200,"data_stored":339447,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542191,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36144,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335696,"statfs":{"total":96636764160,"available":96599752704,"internally_reserved":0,"allocated":815104,"data_stored":339445,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542190,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31980,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339860,"statfs":{"total":96636764160,"available":96604016640,"internally_reserved":0,"allocated":802816,"data_stored":333125,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542191,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27836,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344004,"statfs":{"total":96636764160,"available":96608260096,"internally_reserved":0,"allocated":819200,"data_stored":339467,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542191,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27840,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344000,"statfs":{"total":96636764160,"available":96608256000,"internally_reserved":0,"allocated":823296,"data_stored":340967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542192,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32000,"kb_used_data":812,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339840,"statfs":{"total":96636764160,"available":96603996160,"internally_reserved":0,"allocated":831488,"data_stored":341583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542192,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28396,"kb_used_data":1360,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343444,"statfs":{"total":96636764160,"available":96607686656,"internally_reserved":0,"allocated":1392640,"data_stored":923423,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T03:20:05.822 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-12T03:20:05.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:05.937+0000 7f2116023640 1 Processor -- start 2024-09-12T03:20:05.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:05.937+0000 7f2116023640 1 -- start start 2024-09-12T03:20:05.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:05.938+0000 7f2116023640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f21100d1050 0x7f21100cf810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:05.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:05.938+0000 7f2116023640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f21100d3120 0x7f21100d35b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:05.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:05.938+0000 7f2116023640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f21100d1e60 con 0x7f21100d52e0 2024-09-12T03:20:05.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:05.938+0000 7f2116023640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f21100d2380 con 0x7f21100d1050 2024-09-12T03:20:05.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:05.938+0000 7f2116023640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f21100cfef0 con 0x7f21100d3120 2024-09-12T03:20:05.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:05.938+0000 7f210f7fe640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f21100d1050 0x7f21100cf810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:05.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:05.938+0000 7f210f7fe640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f21100d1050 0x7f21100cf810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.27:48638/0 (socket says 172.21.15.27:48638) 2024-09-12T03:20:05.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:05.938+0000 7f210effd640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f21100d52e0 0x7f21100d1430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.27:41172/0 (socket says 172.21.15.27:41172) 2024-09-12T03:20:05.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:05.938+0000 7f210f7fe640 1 -- 172.21.15.27:0/2616032631 learned_addr learned my addr 172.21.15.27:0/2616032631 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:20:05.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:05.938+0000 7f210ffff640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f21100d3120 0x7f21100d35b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:05.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:05.939+0000 7f210f7fe640 1 -- 172.21.15.27:0/2616032631 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f21100d3120 msgr2=0x7f21100d35b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:05.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:05.939+0000 7f210f7fe640 1 --2- 172.21.15.27:0/2616032631 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f21100d3120 0x7f21100d35b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:05.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:05.939+0000 7f210f7fe640 1 -- 172.21.15.27:0/2616032631 >> v1:172.21.15.45:6789/0 conn(0x7f21100d52e0 legacy=0x7f21100d1430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:05.942 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:05.939+0000 7f210f7fe640 1 -- 172.21.15.27:0/2616032631 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21100d2640 con 0x7f21100d1050 2024-09-12T03:20:05.942 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:05.940+0000 7f210f7fe640 1 --2- 172.21.15.27:0/2616032631 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f21100d1050 0x7f21100cf810 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f2100009870 tx=0x7f210002a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=63cc03283e98102c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:05.942 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:05.940+0000 7f210e7fc640 1 -- 172.21.15.27:0/2616032631 <== mon.1 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2100037070 con 0x7f21100d1050 2024-09-12T03:20:05.942 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:05.940+0000 7f210e7fc640 1 -- 172.21.15.27:0/2616032631 <== mon.1 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2100032440 con 0x7f21100d1050 2024-09-12T03:20:05.942 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:05.940+0000 7f210e7fc640 1 -- 172.21.15.27:0/2616032631 <== mon.1 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2100032810 con 0x7f21100d1050 2024-09-12T03:20:05.942 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:05.940+0000 7f2116023640 1 -- 172.21.15.27:0/2616032631 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f21100d1050 msgr2=0x7f21100cf810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:05.942 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:05.940+0000 7f2116023640 1 --2- 172.21.15.27:0/2616032631 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f21100d1050 0x7f21100cf810 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f2100009870 tx=0x7f210002a270 comp rx=0 tx=0).stop 2024-09-12T03:20:05.942 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:05.940+0000 7f2116023640 1 -- 172.21.15.27:0/2616032631 shutdown_connections 2024-09-12T03:20:05.942 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:05.940+0000 7f2116023640 1 --2- 172.21.15.27:0/2616032631 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f21100d3120 0x7f21100d35b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:05.942 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:05.940+0000 7f2116023640 1 --2- 172.21.15.27:0/2616032631 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f21100d1050 0x7f21100cf810 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:05.942 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:05.940+0000 7f2116023640 1 -- 172.21.15.27:0/2616032631 >> 172.21.15.27:0/2616032631 conn(0x7f211006ddf0 msgr2=0x7f211006e200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:05.942 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:05.940+0000 7f2116023640 1 -- 172.21.15.27:0/2616032631 shutdown_connections 2024-09-12T03:20:05.942 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:05.940+0000 7f2116023640 1 -- 172.21.15.27:0/2616032631 wait complete. 2024-09-12T03:20:05.943 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:05.942+0000 7f2116023640 1 Processor -- start 2024-09-12T03:20:05.943 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:05.942+0000 7f2116023640 1 -- start start 2024-09-12T03:20:05.945 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:05.943+0000 7f2116023640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f21100d1050 0x7f211015a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:05.945 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:05.943+0000 7f2116023640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f21100d3120 0x7f21100ce0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:05.945 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:05.943+0000 7f2116023640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f21100d52e0 0x7f21101626c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:05.945 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:05.943+0000 7f2116023640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f21100b7a50 con 0x7f21100d52e0 2024-09-12T03:20:05.945 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:05.943+0000 7f2116023640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f21100d2c00 con 0x7f21100d1050 2024-09-12T03:20:05.945 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:05.943+0000 7f2116023640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f21100b7bd0 con 0x7f21100d3120 2024-09-12T03:20:05.945 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:05.943+0000 7f210ffff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f21100d52e0 0x7f21101626c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:05.945 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:05.943+0000 7f210f7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f21100d1050 0x7f211015a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:05.945 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:05.943+0000 7f210ffff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f21100d52e0 0x7f21101626c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56260/0 (socket says 172.21.15.27:56260) 2024-09-12T03:20:05.945 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:05.943+0000 7f210ffff640 1 -- 172.21.15.27:0/3438884223 learned_addr learned my addr 172.21.15.27:0/3438884223 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:20:05.945 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:05.943+0000 7f210effd640 1 --2- 172.21.15.27:0/3438884223 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f21100d3120 0x7f21100ce0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:05.946 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:05.943+0000 7f210f7fe640 1 -- 172.21.15.27:0/3438884223 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f21100d3120 msgr2=0x7f21100ce0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:05.946 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:05.943+0000 7f210f7fe640 1 --2- 172.21.15.27:0/3438884223 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f21100d3120 0x7f21100ce0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:05.946 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:05.943+0000 7f210f7fe640 1 -- 172.21.15.27:0/3438884223 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f21100d52e0 msgr2=0x7f21101626c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:05.946 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:05.943+0000 7f210f7fe640 1 --2- 172.21.15.27:0/3438884223 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f21100d52e0 0x7f21101626c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:05.946 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:05.943+0000 7f210f7fe640 1 -- 172.21.15.27:0/3438884223 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f211015f9b0 con 0x7f21100d1050 2024-09-12T03:20:05.946 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:05.943+0000 7f210effd640 1 --2- 172.21.15.27:0/3438884223 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f21100d3120 0x7f21100ce0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:20:05.946 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:05.943+0000 7f210ffff640 1 --2- 172.21.15.27:0/3438884223 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f21100d52e0 0x7f21101626c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:20:05.946 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:05.943+0000 7f210f7fe640 1 --2- 172.21.15.27:0/3438884223 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f21100d1050 0x7f211015a630 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f2100031000 tx=0x7f21000046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:05.946 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:05.944+0000 7f20f7fff640 1 -- 172.21.15.27:0/3438884223 <== mon.1 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2100041020 con 0x7f21100d1050 2024-09-12T03:20:05.946 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:05.944+0000 7f20f7fff640 1 -- 172.21.15.27:0/3438884223 <== mon.1 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2100030660 con 0x7f21100d1050 2024-09-12T03:20:05.946 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:05.944+0000 7f2116023640 1 -- 172.21.15.27:0/3438884223 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f211015fc50 con 0x7f21100d1050 2024-09-12T03:20:05.946 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:05.944+0000 7f20f7fff640 1 -- 172.21.15.27:0/3438884223 <== mon.1 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2100037040 con 0x7f21100d1050 2024-09-12T03:20:05.946 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:05.944+0000 7f2116023640 1 -- 172.21.15.27:0/3438884223 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2110160210 con 0x7f21100d1050 2024-09-12T03:20:05.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:05.945+0000 7f2116023640 1 -- 172.21.15.27:0/3438884223 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f211015ec60 con 0x7f21100d1050 2024-09-12T03:20:05.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:05.945+0000 7f20f7fff640 1 -- 172.21.15.27:0/3438884223 <== mon.1 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 11) ==== 90804+0+0 (secure 0 0 0) 0x7f2100044020 con 0x7f21100d1050 2024-09-12T03:20:05.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:05.946+0000 7f20f7fff640 1 --2- 172.21.15.27:0/3438884223 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f20fc06c190 0x7f20fc06e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:05.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:05.946+0000 7f20f7fff640 1 -- 172.21.15.27:0/3438884223 <== mon.1 v2:172.21.15.45:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 5709+0+0 (secure 0 0 0) 0x7f21000ab860 con 0x7f21100d1050 2024-09-12T03:20:05.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:05.946+0000 7f210effd640 1 --2- 172.21.15.27:0/3438884223 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f20fc06c190 0x7f20fc06e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:05.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:05.947+0000 7f210effd640 1 --2- 172.21.15.27:0/3438884223 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f20fc06c190 0x7f20fc06e650 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f211015ab70 tx=0x7f2104007e80 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:05.951 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:05.951+0000 7f20f7fff640 1 -- 172.21.15.27:0/3438884223 <== mon.1 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f210004b030 con 0x7f21100d1050 2024-09-12T03:20:06.126 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:06.125+0000 7f2116023640 1 -- 172.21.15.27:0/3438884223 --> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f21100d20a0 con 0x7f20fc06c190 2024-09-12T03:20:06.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:06.132+0000 7f20f7fff640 1 -- 172.21.15.27:0/3438884223 <== mgr.4102 v2:172.21.15.45:6832/3923821219 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251957 (secure 0 0 0) 0x7f21100d20a0 con 0x7f20fc06c190 2024-09-12T03:20:06.136 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:20:06.137 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2024-09-12T03:20:06.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:06.135+0000 7f2116023640 1 -- 172.21.15.27:0/3438884223 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f20fc06c190 msgr2=0x7f20fc06e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:06.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:06.135+0000 7f2116023640 1 --2- 172.21.15.27:0/3438884223 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f20fc06c190 0x7f20fc06e650 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f211015ab70 tx=0x7f2104007e80 comp rx=0 tx=0).stop 2024-09-12T03:20:06.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:06.135+0000 7f2116023640 1 -- 172.21.15.27:0/3438884223 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f21100d1050 msgr2=0x7f211015a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:06.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:06.135+0000 7f2116023640 1 --2- 172.21.15.27:0/3438884223 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f21100d1050 0x7f211015a630 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f2100031000 tx=0x7f21000046c0 comp rx=0 tx=0).stop 2024-09-12T03:20:06.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:06.135+0000 7f2116023640 1 -- 172.21.15.27:0/3438884223 shutdown_connections 2024-09-12T03:20:06.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:06.135+0000 7f2116023640 1 --2- 172.21.15.27:0/3438884223 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f20fc06c190 0x7f20fc06e650 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:06.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:06.135+0000 7f2116023640 1 --2- 172.21.15.27:0/3438884223 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f21100d52e0 0x7f21101626c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:06.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:06.135+0000 7f2116023640 1 --2- 172.21.15.27:0/3438884223 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f21100d3120 0x7f21100ce0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:06.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:06.135+0000 7f2116023640 1 --2- 172.21.15.27:0/3438884223 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f21100d1050 0x7f211015a630 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:06.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:06.135+0000 7f2116023640 1 -- 172.21.15.27:0/3438884223 >> 172.21.15.27:0/3438884223 conn(0x7f211006ddf0 msgr2=0x7f21100d35b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:06.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:06.135+0000 7f2116023640 1 -- 172.21.15.27:0/3438884223 shutdown_connections 2024-09-12T03:20:06.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:06.135+0000 7f2116023640 1 -- 172.21.15.27:0/3438884223 wait complete. 2024-09-12T03:20:06.147 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":173,"stamp":"2024-09-12T03:20:06.104846+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":252932,"kb_used_data":7560,"kb_used_omap":12,"kb_used_meta":245171,"kb_avail":754721788,"statfs":{"total":773094113280,"available":772835110912,"internally_reserved":0,"allocated":7741440,"data_stored":3900396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":251055664},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001902"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.412295+0000","last_change":"2024-09-12T03:19:58.405738+0000","last_active":"2024-09-12T03:19:59.412295+0000","last_peered":"2024-09-12T03:19:59.412295+0000","last_clean":"2024-09-12T03:19:59.412295+0000","last_became_active":"2024-09-12T03:19:58.405044+0000","last_became_peered":"2024-09-12T03:19:58.405044+0000","last_unstale":"2024-09-12T03:19:59.412295+0000","last_undegraded":"2024-09-12T03:19:59.412295+0000","last_fullsized":"2024-09-12T03:19:59.412295+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:24:50.892593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.412848+0000","last_change":"2024-09-12T03:19:58.407057+0000","last_active":"2024-09-12T03:19:59.412848+0000","last_peered":"2024-09-12T03:19:59.412848+0000","last_clean":"2024-09-12T03:19:59.412848+0000","last_became_active":"2024-09-12T03:19:58.406455+0000","last_became_peered":"2024-09-12T03:19:58.406455+0000","last_unstale":"2024-09-12T03:19:59.412848+0000","last_undegraded":"2024-09-12T03:19:59.412848+0000","last_fullsized":"2024-09-12T03:19:59.412848+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:12:06.463949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.411816+0000","last_change":"2024-09-12T03:19:58.402689+0000","last_active":"2024-09-12T03:19:59.411816+0000","last_peered":"2024-09-12T03:19:59.411816+0000","last_clean":"2024-09-12T03:19:59.411816+0000","last_became_active":"2024-09-12T03:19:58.402424+0000","last_became_peered":"2024-09-12T03:19:58.402424+0000","last_unstale":"2024-09-12T03:19:59.411816+0000","last_undegraded":"2024-09-12T03:19:59.411816+0000","last_fullsized":"2024-09-12T03:19:59.411816+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:01:53.466856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.411556+0000","last_change":"2024-09-12T03:19:58.407823+0000","last_active":"2024-09-12T03:19:59.411556+0000","last_peered":"2024-09-12T03:19:59.411556+0000","last_clean":"2024-09-12T03:19:59.411556+0000","last_became_active":"2024-09-12T03:19:58.407533+0000","last_became_peered":"2024-09-12T03:19:58.407533+0000","last_unstale":"2024-09-12T03:19:59.411556+0000","last_undegraded":"2024-09-12T03:19:59.411556+0000","last_fullsized":"2024-09-12T03:19:59.411556+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:35:40.539826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.411395+0000","last_change":"2024-09-12T03:19:58.405012+0000","last_active":"2024-09-12T03:19:59.411395+0000","last_peered":"2024-09-12T03:19:59.411395+0000","last_clean":"2024-09-12T03:19:59.411395+0000","last_became_active":"2024-09-12T03:19:58.404706+0000","last_became_peered":"2024-09-12T03:19:58.404706+0000","last_unstale":"2024-09-12T03:19:59.411395+0000","last_undegraded":"2024-09-12T03:19:59.411395+0000","last_fullsized":"2024-09-12T03:19:59.411395+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:13:43.693628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.412119+0000","last_change":"2024-09-12T03:19:58.404152+0000","last_active":"2024-09-12T03:19:59.412119+0000","last_peered":"2024-09-12T03:19:59.412119+0000","last_clean":"2024-09-12T03:19:59.412119+0000","last_became_active":"2024-09-12T03:19:58.403865+0000","last_became_peered":"2024-09-12T03:19:58.403865+0000","last_unstale":"2024-09-12T03:19:59.412119+0000","last_undegraded":"2024-09-12T03:19:59.412119+0000","last_fullsized":"2024-09-12T03:19:59.412119+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:03:35.086339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.399545+0000","last_change":"2024-09-12T03:19:58.405356+0000","last_active":"2024-09-12T03:19:59.399545+0000","last_peered":"2024-09-12T03:19:59.399545+0000","last_clean":"2024-09-12T03:19:59.399545+0000","last_became_active":"2024-09-12T03:19:58.405079+0000","last_became_peered":"2024-09-12T03:19:58.405079+0000","last_unstale":"2024-09-12T03:19:59.399545+0000","last_undegraded":"2024-09-12T03:19:59.399545+0000","last_fullsized":"2024-09-12T03:19:59.399545+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:26:48.602769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.412001+0000","last_change":"2024-09-12T03:19:58.408692+0000","last_active":"2024-09-12T03:19:59.412001+0000","last_peered":"2024-09-12T03:19:59.412001+0000","last_clean":"2024-09-12T03:19:59.412001+0000","last_became_active":"2024-09-12T03:19:58.408182+0000","last_became_peered":"2024-09-12T03:19:58.408182+0000","last_unstale":"2024-09-12T03:19:59.412001+0000","last_undegraded":"2024-09-12T03:19:59.412001+0000","last_fullsized":"2024-09-12T03:19:59.412001+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:25:56.007148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.412647+0000","last_change":"2024-09-12T03:19:58.408054+0000","last_active":"2024-09-12T03:19:59.412647+0000","last_peered":"2024-09-12T03:19:59.412647+0000","last_clean":"2024-09-12T03:19:59.412647+0000","last_became_active":"2024-09-12T03:19:58.407439+0000","last_became_peered":"2024-09-12T03:19:58.407439+0000","last_unstale":"2024-09-12T03:19:59.412647+0000","last_undegraded":"2024-09-12T03:19:59.412647+0000","last_fullsized":"2024-09-12T03:19:59.412647+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:52:55.350315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.399858+0000","last_change":"2024-09-12T03:19:58.400878+0000","last_active":"2024-09-12T03:19:59.399858+0000","last_peered":"2024-09-12T03:19:59.399858+0000","last_clean":"2024-09-12T03:19:59.399858+0000","last_became_active":"2024-09-12T03:19:58.400601+0000","last_became_peered":"2024-09-12T03:19:58.400601+0000","last_unstale":"2024-09-12T03:19:59.399858+0000","last_undegraded":"2024-09-12T03:19:59.399858+0000","last_fullsized":"2024-09-12T03:19:59.399858+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:53:33.025355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.412388+0000","last_change":"2024-09-12T03:19:58.404349+0000","last_active":"2024-09-12T03:19:59.412388+0000","last_peered":"2024-09-12T03:19:59.412388+0000","last_clean":"2024-09-12T03:19:59.412388+0000","last_became_active":"2024-09-12T03:19:58.404067+0000","last_became_peered":"2024-09-12T03:19:58.404067+0000","last_unstale":"2024-09-12T03:19:59.412388+0000","last_undegraded":"2024-09-12T03:19:59.412388+0000","last_fullsized":"2024-09-12T03:19:59.412388+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:31:06.800562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.411695+0000","last_change":"2024-09-12T03:19:58.398948+0000","last_active":"2024-09-12T03:19:59.411695+0000","last_peered":"2024-09-12T03:19:59.411695+0000","last_clean":"2024-09-12T03:19:59.411695+0000","last_became_active":"2024-09-12T03:19:58.398583+0000","last_became_peered":"2024-09-12T03:19:58.398583+0000","last_unstale":"2024-09-12T03:19:59.411695+0000","last_undegraded":"2024-09-12T03:19:59.411695+0000","last_fullsized":"2024-09-12T03:19:59.411695+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:14:21.530213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.398467+0000","last_change":"2024-09-12T03:19:58.408147+0000","last_active":"2024-09-12T03:19:59.398467+0000","last_peered":"2024-09-12T03:19:59.398467+0000","last_clean":"2024-09-12T03:19:59.398467+0000","last_became_active":"2024-09-12T03:19:58.407855+0000","last_became_peered":"2024-09-12T03:19:58.407855+0000","last_unstale":"2024-09-12T03:19:59.398467+0000","last_undegraded":"2024-09-12T03:19:59.398467+0000","last_fullsized":"2024-09-12T03:19:59.398467+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:50:40.770697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.412570+0000","last_change":"2024-09-12T03:19:58.401979+0000","last_active":"2024-09-12T03:19:59.412570+0000","last_peered":"2024-09-12T03:19:59.412570+0000","last_clean":"2024-09-12T03:19:59.412570+0000","last_became_active":"2024-09-12T03:19:58.401689+0000","last_became_peered":"2024-09-12T03:19:58.401689+0000","last_unstale":"2024-09-12T03:19:59.412570+0000","last_undegraded":"2024-09-12T03:19:59.412570+0000","last_fullsized":"2024-09-12T03:19:59.412570+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:22:57.280328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.398306+0000","last_change":"2024-09-12T03:19:58.399312+0000","last_active":"2024-09-12T03:19:59.398306+0000","last_peered":"2024-09-12T03:19:59.398306+0000","last_clean":"2024-09-12T03:19:59.398306+0000","last_became_active":"2024-09-12T03:19:58.399050+0000","last_became_peered":"2024-09-12T03:19:58.399050+0000","last_unstale":"2024-09-12T03:19:59.398306+0000","last_undegraded":"2024-09-12T03:19:59.398306+0000","last_fullsized":"2024-09-12T03:19:59.398306+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:41:13.177460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.400875+0000","last_change":"2024-09-12T03:19:58.404580+0000","last_active":"2024-09-12T03:19:59.400875+0000","last_peered":"2024-09-12T03:19:59.400875+0000","last_clean":"2024-09-12T03:19:59.400875+0000","last_became_active":"2024-09-12T03:19:58.404320+0000","last_became_peered":"2024-09-12T03:19:58.404320+0000","last_unstale":"2024-09-12T03:19:59.400875+0000","last_undegraded":"2024-09-12T03:19:59.400875+0000","last_fullsized":"2024-09-12T03:19:59.400875+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:41:51.158105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.412705+0000","last_change":"2024-09-12T03:19:58.408389+0000","last_active":"2024-09-12T03:19:59.412705+0000","last_peered":"2024-09-12T03:19:59.412705+0000","last_clean":"2024-09-12T03:19:59.412705+0000","last_became_active":"2024-09-12T03:19:58.407797+0000","last_became_peered":"2024-09-12T03:19:58.407797+0000","last_unstale":"2024-09-12T03:19:59.412705+0000","last_undegraded":"2024-09-12T03:19:59.412705+0000","last_fullsized":"2024-09-12T03:19:59.412705+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:17:24.698351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.411598+0000","last_change":"2024-09-12T03:19:58.409159+0000","last_active":"2024-09-12T03:19:59.411598+0000","last_peered":"2024-09-12T03:19:59.411598+0000","last_clean":"2024-09-12T03:19:59.411598+0000","last_became_active":"2024-09-12T03:19:58.408851+0000","last_became_peered":"2024-09-12T03:19:58.408851+0000","last_unstale":"2024-09-12T03:19:59.411598+0000","last_undegraded":"2024-09-12T03:19:59.411598+0000","last_fullsized":"2024-09-12T03:19:59.411598+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:23:52.073974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.412173+0000","last_change":"2024-09-12T03:19:58.408115+0000","last_active":"2024-09-12T03:19:59.412173+0000","last_peered":"2024-09-12T03:19:59.412173+0000","last_clean":"2024-09-12T03:19:59.412173+0000","last_became_active":"2024-09-12T03:19:58.407745+0000","last_became_peered":"2024-09-12T03:19:58.407745+0000","last_unstale":"2024-09-12T03:19:59.412173+0000","last_undegraded":"2024-09-12T03:19:59.412173+0000","last_fullsized":"2024-09-12T03:19:59.412173+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:50:17.515387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.398554+0000","last_change":"2024-09-12T03:19:58.398467+0000","last_active":"2024-09-12T03:19:59.398554+0000","last_peered":"2024-09-12T03:19:59.398554+0000","last_clean":"2024-09-12T03:19:59.398554+0000","last_became_active":"2024-09-12T03:19:58.398152+0000","last_became_peered":"2024-09-12T03:19:58.398152+0000","last_unstale":"2024-09-12T03:19:59.398554+0000","last_undegraded":"2024-09-12T03:19:59.398554+0000","last_fullsized":"2024-09-12T03:19:59.398554+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:24:50.892593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.400458+0000","last_change":"2024-09-12T03:19:58.400541+0000","last_active":"2024-09-12T03:19:59.400458+0000","last_peered":"2024-09-12T03:19:59.400458+0000","last_clean":"2024-09-12T03:19:59.400458+0000","last_became_active":"2024-09-12T03:19:58.399602+0000","last_became_peered":"2024-09-12T03:19:58.399602+0000","last_unstale":"2024-09-12T03:19:59.400458+0000","last_undegraded":"2024-09-12T03:19:59.400458+0000","last_fullsized":"2024-09-12T03:19:59.400458+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:56:37.761014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.400370+0000","last_change":"2024-09-12T03:19:58.399427+0000","last_active":"2024-09-12T03:19:59.400370+0000","last_peered":"2024-09-12T03:19:59.400370+0000","last_clean":"2024-09-12T03:19:59.400370+0000","last_became_active":"2024-09-12T03:19:58.398464+0000","last_became_peered":"2024-09-12T03:19:58.398464+0000","last_unstale":"2024-09-12T03:19:59.400370+0000","last_undegraded":"2024-09-12T03:19:59.400370+0000","last_fullsized":"2024-09-12T03:19:59.400370+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:44:19.209045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.411588+0000","last_change":"2024-09-12T03:19:58.405456+0000","last_active":"2024-09-12T03:19:59.411588+0000","last_peered":"2024-09-12T03:19:59.411588+0000","last_clean":"2024-09-12T03:19:59.411588+0000","last_became_active":"2024-09-12T03:19:58.404974+0000","last_became_peered":"2024-09-12T03:19:58.404974+0000","last_unstale":"2024-09-12T03:19:59.411588+0000","last_undegraded":"2024-09-12T03:19:59.411588+0000","last_fullsized":"2024-09-12T03:19:59.411588+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:58:58.914180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.411477+0000","last_change":"2024-09-12T03:19:58.408893+0000","last_active":"2024-09-12T03:19:59.411477+0000","last_peered":"2024-09-12T03:19:59.411477+0000","last_clean":"2024-09-12T03:19:59.411477+0000","last_became_active":"2024-09-12T03:19:58.408589+0000","last_became_peered":"2024-09-12T03:19:58.408589+0000","last_unstale":"2024-09-12T03:19:59.411477+0000","last_undegraded":"2024-09-12T03:19:59.411477+0000","last_fullsized":"2024-09-12T03:19:59.411477+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:54:48.020501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.400037+0000","last_change":"2024-09-12T03:19:58.411515+0000","last_active":"2024-09-12T03:19:59.400037+0000","last_peered":"2024-09-12T03:19:59.400037+0000","last_clean":"2024-09-12T03:19:59.400037+0000","last_became_active":"2024-09-12T03:19:58.409091+0000","last_became_peered":"2024-09-12T03:19:58.409091+0000","last_unstale":"2024-09-12T03:19:59.400037+0000","last_undegraded":"2024-09-12T03:19:59.400037+0000","last_fullsized":"2024-09-12T03:19:59.400037+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:25:42.004151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.400164+0000","last_change":"2024-09-12T03:19:58.410854+0000","last_active":"2024-09-12T03:19:59.400164+0000","last_peered":"2024-09-12T03:19:59.400164+0000","last_clean":"2024-09-12T03:19:59.400164+0000","last_became_active":"2024-09-12T03:19:58.410556+0000","last_became_peered":"2024-09-12T03:19:58.410556+0000","last_unstale":"2024-09-12T03:19:59.400164+0000","last_undegraded":"2024-09-12T03:19:59.400164+0000","last_fullsized":"2024-09-12T03:19:59.400164+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:16:53.699272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.411966+0000","last_change":"2024-09-12T03:19:58.406104+0000","last_active":"2024-09-12T03:19:59.411966+0000","last_peered":"2024-09-12T03:19:59.411966+0000","last_clean":"2024-09-12T03:19:59.411966+0000","last_became_active":"2024-09-12T03:19:58.405546+0000","last_became_peered":"2024-09-12T03:19:58.405546+0000","last_unstale":"2024-09-12T03:19:59.411966+0000","last_undegraded":"2024-09-12T03:19:59.411966+0000","last_fullsized":"2024-09-12T03:19:59.411966+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:45:16.630721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.400401+0000","last_change":"2024-09-12T03:19:58.401764+0000","last_active":"2024-09-12T03:19:59.400401+0000","last_peered":"2024-09-12T03:19:59.400401+0000","last_clean":"2024-09-12T03:19:59.400401+0000","last_became_active":"2024-09-12T03:19:58.397972+0000","last_became_peered":"2024-09-12T03:19:58.397972+0000","last_unstale":"2024-09-12T03:19:59.400401+0000","last_undegraded":"2024-09-12T03:19:59.400401+0000","last_fullsized":"2024-09-12T03:19:59.400401+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:50:11.761046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.412120+0000","last_change":"2024-09-12T03:19:58.403549+0000","last_active":"2024-09-12T03:19:59.412120+0000","last_peered":"2024-09-12T03:19:59.412120+0000","last_clean":"2024-09-12T03:19:59.412120+0000","last_became_active":"2024-09-12T03:19:58.403294+0000","last_became_peered":"2024-09-12T03:19:58.403294+0000","last_unstale":"2024-09-12T03:19:59.412120+0000","last_undegraded":"2024-09-12T03:19:59.412120+0000","last_fullsized":"2024-09-12T03:19:59.412120+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:20:56.214083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.399515+0000","last_change":"2024-09-12T03:19:58.408095+0000","last_active":"2024-09-12T03:19:59.399515+0000","last_peered":"2024-09-12T03:19:59.399515+0000","last_clean":"2024-09-12T03:19:59.399515+0000","last_became_active":"2024-09-12T03:19:58.407819+0000","last_became_peered":"2024-09-12T03:19:58.407819+0000","last_unstale":"2024-09-12T03:19:59.399515+0000","last_undegraded":"2024-09-12T03:19:59.399515+0000","last_fullsized":"2024-09-12T03:19:59.399515+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:43:53.939725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.399784+0000","last_change":"2024-09-12T03:19:58.399638+0000","last_active":"2024-09-12T03:19:59.399784+0000","last_peered":"2024-09-12T03:19:59.399784+0000","last_clean":"2024-09-12T03:19:59.399784+0000","last_became_active":"2024-09-12T03:19:58.399356+0000","last_became_peered":"2024-09-12T03:19:58.399356+0000","last_unstale":"2024-09-12T03:19:59.399784+0000","last_undegraded":"2024-09-12T03:19:59.399784+0000","last_fullsized":"2024-09-12T03:19:59.399784+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:10:16.728909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.412055+0000","last_change":"2024-09-12T03:19:58.407063+0000","last_active":"2024-09-12T03:19:59.412055+0000","last_peered":"2024-09-12T03:19:59.412055+0000","last_clean":"2024-09-12T03:19:59.412055+0000","last_became_active":"2024-09-12T03:19:58.406116+0000","last_became_peered":"2024-09-12T03:19:58.406116+0000","last_unstale":"2024-09-12T03:19:59.412055+0000","last_undegraded":"2024-09-12T03:19:59.412055+0000","last_fullsized":"2024-09-12T03:19:59.412055+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:11:39.227764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.411730+0000","last_change":"2024-09-12T03:19:58.409587+0000","last_active":"2024-09-12T03:19:59.411730+0000","last_peered":"2024-09-12T03:19:59.411730+0000","last_clean":"2024-09-12T03:19:59.411730+0000","last_became_active":"2024-09-12T03:19:58.409275+0000","last_became_peered":"2024-09-12T03:19:58.409275+0000","last_unstale":"2024-09-12T03:19:59.411730+0000","last_undegraded":"2024-09-12T03:19:59.411730+0000","last_fullsized":"2024-09-12T03:19:59.411730+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:00:46.656402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"66'4","reported_seq":18,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.412596+0000","last_change":"2024-09-12T03:19:57.394595+0000","last_active":"2024-09-12T03:19:59.412596+0000","last_peered":"2024-09-12T03:19:59.412596+0000","last_clean":"2024-09-12T03:19:59.412596+0000","last_became_active":"2024-09-12T03:19:57.394294+0000","last_became_peered":"2024-09-12T03:19:57.394294+0000","last_unstale":"2024-09-12T03:19:59.412596+0000","last_undegraded":"2024-09-12T03:19:59.412596+0000","last_fullsized":"2024-09-12T03:19:59.412596+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_clean_scrub_stamp":"2024-09-12T03:19:56.366974+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:51:58.062647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.412771+0000","last_change":"2024-09-12T03:19:58.407115+0000","last_active":"2024-09-12T03:19:59.412771+0000","last_peered":"2024-09-12T03:19:59.412771+0000","last_clean":"2024-09-12T03:19:59.412771+0000","last_became_active":"2024-09-12T03:19:58.406683+0000","last_became_peered":"2024-09-12T03:19:58.406683+0000","last_unstale":"2024-09-12T03:19:59.412771+0000","last_undegraded":"2024-09-12T03:19:59.412771+0000","last_fullsized":"2024-09-12T03:19:59.412771+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:36:57.331135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"66'1","reported_seq":15,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.398595+0000","last_change":"2024-09-12T03:19:57.389982+0000","last_active":"2024-09-12T03:19:59.398595+0000","last_peered":"2024-09-12T03:19:59.398595+0000","last_clean":"2024-09-12T03:19:59.398595+0000","last_became_active":"2024-09-12T03:19:57.389694+0000","last_became_peered":"2024-09-12T03:19:57.389694+0000","last_unstale":"2024-09-12T03:19:59.398595+0000","last_undegraded":"2024-09-12T03:19:59.398595+0000","last_fullsized":"2024-09-12T03:19:59.398595+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_clean_scrub_stamp":"2024-09-12T03:19:56.366974+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:01:26.160193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.412792+0000","last_change":"2024-09-12T03:19:58.415175+0000","last_active":"2024-09-12T03:19:59.412792+0000","last_peered":"2024-09-12T03:19:59.412792+0000","last_clean":"2024-09-12T03:19:59.412792+0000","last_became_active":"2024-09-12T03:19:58.414899+0000","last_became_peered":"2024-09-12T03:19:58.414899+0000","last_unstale":"2024-09-12T03:19:59.412792+0000","last_undegraded":"2024-09-12T03:19:59.412792+0000","last_fullsized":"2024-09-12T03:19:59.412792+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:36:33.065448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"66'2","reported_seq":16,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.400437+0000","last_change":"2024-09-12T03:19:57.393946+0000","last_active":"2024-09-12T03:19:59.400437+0000","last_peered":"2024-09-12T03:19:59.400437+0000","last_clean":"2024-09-12T03:19:59.400437+0000","last_became_active":"2024-09-12T03:19:57.393672+0000","last_became_peered":"2024-09-12T03:19:57.393672+0000","last_unstale":"2024-09-12T03:19:59.400437+0000","last_undegraded":"2024-09-12T03:19:59.400437+0000","last_fullsized":"2024-09-12T03:19:59.400437+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_clean_scrub_stamp":"2024-09-12T03:19:56.366974+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:57:23.221949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.412495+0000","last_change":"2024-09-12T03:19:58.403124+0000","last_active":"2024-09-12T03:19:59.412495+0000","last_peered":"2024-09-12T03:19:59.412495+0000","last_clean":"2024-09-12T03:19:59.412495+0000","last_became_active":"2024-09-12T03:19:58.402859+0000","last_became_peered":"2024-09-12T03:19:58.402859+0000","last_unstale":"2024-09-12T03:19:59.412495+0000","last_undegraded":"2024-09-12T03:19:59.412495+0000","last_fullsized":"2024-09-12T03:19:59.412495+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:41:26.586794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"66'2","reported_seq":16,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.412896+0000","last_change":"2024-09-12T03:19:57.396585+0000","last_active":"2024-09-12T03:19:59.412896+0000","last_peered":"2024-09-12T03:19:59.412896+0000","last_clean":"2024-09-12T03:19:59.412896+0000","last_became_active":"2024-09-12T03:19:57.396109+0000","last_became_peered":"2024-09-12T03:19:57.396109+0000","last_unstale":"2024-09-12T03:19:59.412896+0000","last_undegraded":"2024-09-12T03:19:59.412896+0000","last_fullsized":"2024-09-12T03:19:59.412896+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_clean_scrub_stamp":"2024-09-12T03:19:56.366974+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:25:50.078358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.399545+0000","last_change":"2024-09-12T03:19:58.406747+0000","last_active":"2024-09-12T03:19:59.399545+0000","last_peered":"2024-09-12T03:19:59.399545+0000","last_clean":"2024-09-12T03:19:59.399545+0000","last_became_active":"2024-09-12T03:19:58.406464+0000","last_became_peered":"2024-09-12T03:19:58.406464+0000","last_unstale":"2024-09-12T03:19:59.399545+0000","last_undegraded":"2024-09-12T03:19:59.399545+0000","last_fullsized":"2024-09-12T03:19:59.399545+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:43:08.336624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"66'1","reported_seq":15,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.412362+0000","last_change":"2024-09-12T03:19:57.391056+0000","last_active":"2024-09-12T03:19:59.412362+0000","last_peered":"2024-09-12T03:19:59.412362+0000","last_clean":"2024-09-12T03:19:59.412362+0000","last_became_active":"2024-09-12T03:19:57.390769+0000","last_became_peered":"2024-09-12T03:19:57.390769+0000","last_unstale":"2024-09-12T03:19:59.412362+0000","last_undegraded":"2024-09-12T03:19:59.412362+0000","last_fullsized":"2024-09-12T03:19:59.412362+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_clean_scrub_stamp":"2024-09-12T03:19:56.366974+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:41:12.173208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.400570+0000","last_change":"2024-09-12T03:19:58.415098+0000","last_active":"2024-09-12T03:19:59.400570+0000","last_peered":"2024-09-12T03:19:59.400570+0000","last_clean":"2024-09-12T03:19:59.400570+0000","last_became_active":"2024-09-12T03:19:58.411028+0000","last_became_peered":"2024-09-12T03:19:58.411028+0000","last_unstale":"2024-09-12T03:19:59.400570+0000","last_undegraded":"2024-09-12T03:19:59.400570+0000","last_fullsized":"2024-09-12T03:19:59.400570+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:29:23.148491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"66'1","reported_seq":15,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.412238+0000","last_change":"2024-09-12T03:19:57.395730+0000","last_active":"2024-09-12T03:19:59.412238+0000","last_peered":"2024-09-12T03:19:59.412238+0000","last_clean":"2024-09-12T03:19:59.412238+0000","last_became_active":"2024-09-12T03:19:57.395253+0000","last_became_peered":"2024-09-12T03:19:57.395253+0000","last_unstale":"2024-09-12T03:19:59.412238+0000","last_undegraded":"2024-09-12T03:19:59.412238+0000","last_fullsized":"2024-09-12T03:19:59.412238+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_clean_scrub_stamp":"2024-09-12T03:19:56.366974+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:54:51.158313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.411683+0000","last_change":"2024-09-12T03:19:58.400222+0000","last_active":"2024-09-12T03:19:59.411683+0000","last_peered":"2024-09-12T03:19:59.411683+0000","last_clean":"2024-09-12T03:19:59.411683+0000","last_became_active":"2024-09-12T03:19:58.399906+0000","last_became_peered":"2024-09-12T03:19:58.399906+0000","last_unstale":"2024-09-12T03:19:59.411683+0000","last_undegraded":"2024-09-12T03:19:59.411683+0000","last_fullsized":"2024-09-12T03:19:59.411683+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:17:22.493344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.411395+0000","last_change":"2024-09-12T03:19:57.390855+0000","last_active":"2024-09-12T03:19:59.411395+0000","last_peered":"2024-09-12T03:19:59.411395+0000","last_clean":"2024-09-12T03:19:59.411395+0000","last_became_active":"2024-09-12T03:19:57.390307+0000","last_became_peered":"2024-09-12T03:19:57.390307+0000","last_unstale":"2024-09-12T03:19:59.411395+0000","last_undegraded":"2024-09-12T03:19:59.411395+0000","last_fullsized":"2024-09-12T03:19:59.411395+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_clean_scrub_stamp":"2024-09-12T03:19:56.366974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:33:55.147981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.398361+0000","last_change":"2024-09-12T03:19:58.405943+0000","last_active":"2024-09-12T03:19:59.398361+0000","last_peered":"2024-09-12T03:19:59.398361+0000","last_clean":"2024-09-12T03:19:59.398361+0000","last_became_active":"2024-09-12T03:19:58.405630+0000","last_became_peered":"2024-09-12T03:19:58.405630+0000","last_unstale":"2024-09-12T03:19:59.398361+0000","last_undegraded":"2024-09-12T03:19:59.398361+0000","last_fullsized":"2024-09-12T03:19:59.398361+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:00:46.656402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.411776+0000","last_change":"2024-09-12T03:19:57.394447+0000","last_active":"2024-09-12T03:19:59.411776+0000","last_peered":"2024-09-12T03:19:59.411776+0000","last_clean":"2024-09-12T03:19:59.411776+0000","last_became_active":"2024-09-12T03:19:57.394153+0000","last_became_peered":"2024-09-12T03:19:57.394153+0000","last_unstale":"2024-09-12T03:19:59.411776+0000","last_undegraded":"2024-09-12T03:19:59.411776+0000","last_fullsized":"2024-09-12T03:19:59.411776+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_clean_scrub_stamp":"2024-09-12T03:19:56.366974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:12:19.700994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.400317+0000","last_change":"2024-09-12T03:19:58.412227+0000","last_active":"2024-09-12T03:19:59.400317+0000","last_peered":"2024-09-12T03:19:59.400317+0000","last_clean":"2024-09-12T03:19:59.400317+0000","last_became_active":"2024-09-12T03:19:58.411951+0000","last_became_peered":"2024-09-12T03:19:58.411951+0000","last_unstale":"2024-09-12T03:19:59.400317+0000","last_undegraded":"2024-09-12T03:19:59.400317+0000","last_fullsized":"2024-09-12T03:19:59.400317+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:54:24.310119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.400155+0000","last_change":"2024-09-12T03:19:57.393730+0000","last_active":"2024-09-12T03:19:59.400155+0000","last_peered":"2024-09-12T03:19:59.400155+0000","last_clean":"2024-09-12T03:19:59.400155+0000","last_became_active":"2024-09-12T03:19:57.393425+0000","last_became_peered":"2024-09-12T03:19:57.393425+0000","last_unstale":"2024-09-12T03:19:59.400155+0000","last_undegraded":"2024-09-12T03:19:59.400155+0000","last_fullsized":"2024-09-12T03:19:59.400155+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_clean_scrub_stamp":"2024-09-12T03:19:56.366974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:39:01.426373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.412571+0000","last_change":"2024-09-12T03:19:58.409130+0000","last_active":"2024-09-12T03:19:59.412571+0000","last_peered":"2024-09-12T03:19:59.412571+0000","last_clean":"2024-09-12T03:19:59.412571+0000","last_became_active":"2024-09-12T03:19:58.408838+0000","last_became_peered":"2024-09-12T03:19:58.408838+0000","last_unstale":"2024-09-12T03:19:59.412571+0000","last_undegraded":"2024-09-12T03:19:59.412571+0000","last_fullsized":"2024-09-12T03:19:59.412571+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:38:31.191391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.412967+0000","last_change":"2024-09-12T03:19:57.391703+0000","last_active":"2024-09-12T03:19:59.412967+0000","last_peered":"2024-09-12T03:19:59.412967+0000","last_clean":"2024-09-12T03:19:59.412967+0000","last_became_active":"2024-09-12T03:19:57.391345+0000","last_became_peered":"2024-09-12T03:19:57.391345+0000","last_unstale":"2024-09-12T03:19:59.412967+0000","last_undegraded":"2024-09-12T03:19:59.412967+0000","last_fullsized":"2024-09-12T03:19:59.412967+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_clean_scrub_stamp":"2024-09-12T03:19:56.366974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:05:09.764085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.412786+0000","last_change":"2024-09-12T03:19:58.410704+0000","last_active":"2024-09-12T03:19:59.412786+0000","last_peered":"2024-09-12T03:19:59.412786+0000","last_clean":"2024-09-12T03:19:59.412786+0000","last_became_active":"2024-09-12T03:19:58.410390+0000","last_became_peered":"2024-09-12T03:19:58.410390+0000","last_unstale":"2024-09-12T03:19:59.412786+0000","last_undegraded":"2024-09-12T03:19:59.412786+0000","last_fullsized":"2024-09-12T03:19:59.412786+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:27:21.762076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"66'3","reported_seq":17,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.412074+0000","last_change":"2024-09-12T03:19:57.383973+0000","last_active":"2024-09-12T03:19:59.412074+0000","last_peered":"2024-09-12T03:19:59.412074+0000","last_clean":"2024-09-12T03:19:59.412074+0000","last_became_active":"2024-09-12T03:19:57.383481+0000","last_became_peered":"2024-09-12T03:19:57.383481+0000","last_unstale":"2024-09-12T03:19:59.412074+0000","last_undegraded":"2024-09-12T03:19:59.412074+0000","last_fullsized":"2024-09-12T03:19:59.412074+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_clean_scrub_stamp":"2024-09-12T03:19:56.366974+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:00:45.652150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.411674+0000","last_change":"2024-09-12T03:19:58.400811+0000","last_active":"2024-09-12T03:19:59.411674+0000","last_peered":"2024-09-12T03:19:59.411674+0000","last_clean":"2024-09-12T03:19:59.411674+0000","last_became_active":"2024-09-12T03:19:58.400542+0000","last_became_peered":"2024-09-12T03:19:58.400542+0000","last_unstale":"2024-09-12T03:19:59.411674+0000","last_undegraded":"2024-09-12T03:19:59.411674+0000","last_fullsized":"2024-09-12T03:19:59.411674+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:01:16.275564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"66'6","reported_seq":20,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.411922+0000","last_change":"2024-09-12T03:19:57.390457+0000","last_active":"2024-09-12T03:19:59.411922+0000","last_peered":"2024-09-12T03:19:59.411922+0000","last_clean":"2024-09-12T03:19:59.411922+0000","last_became_active":"2024-09-12T03:19:57.390109+0000","last_became_peered":"2024-09-12T03:19:57.390109+0000","last_unstale":"2024-09-12T03:19:59.411922+0000","last_undegraded":"2024-09-12T03:19:59.411922+0000","last_fullsized":"2024-09-12T03:19:59.411922+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_clean_scrub_stamp":"2024-09-12T03:19:56.366974+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:43:23.673876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.411683+0000","last_change":"2024-09-12T03:19:58.402659+0000","last_active":"2024-09-12T03:19:59.411683+0000","last_peered":"2024-09-12T03:19:59.411683+0000","last_clean":"2024-09-12T03:19:59.411683+0000","last_became_active":"2024-09-12T03:19:58.402358+0000","last_became_peered":"2024-09-12T03:19:58.402358+0000","last_unstale":"2024-09-12T03:19:59.411683+0000","last_undegraded":"2024-09-12T03:19:59.411683+0000","last_fullsized":"2024-09-12T03:19:59.411683+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:41:13.177460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"66'2","reported_seq":16,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.411862+0000","last_change":"2024-09-12T03:19:57.391726+0000","last_active":"2024-09-12T03:19:59.411862+0000","last_peered":"2024-09-12T03:19:59.411862+0000","last_clean":"2024-09-12T03:19:59.411862+0000","last_became_active":"2024-09-12T03:19:57.391107+0000","last_became_peered":"2024-09-12T03:19:57.391107+0000","last_unstale":"2024-09-12T03:19:59.411862+0000","last_undegraded":"2024-09-12T03:19:59.411862+0000","last_fullsized":"2024-09-12T03:19:59.411862+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_clean_scrub_stamp":"2024-09-12T03:19:56.366974+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:41:29.366042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.399734+0000","last_change":"2024-09-12T03:19:58.394023+0000","last_active":"2024-09-12T03:19:59.399734+0000","last_peered":"2024-09-12T03:19:59.399734+0000","last_clean":"2024-09-12T03:19:59.399734+0000","last_became_active":"2024-09-12T03:19:58.393630+0000","last_became_peered":"2024-09-12T03:19:58.393630+0000","last_unstale":"2024-09-12T03:19:59.399734+0000","last_undegraded":"2024-09-12T03:19:59.399734+0000","last_fullsized":"2024-09-12T03:19:59.399734+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:17:36.830688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.412349+0000","last_change":"2024-09-12T03:19:57.395906+0000","last_active":"2024-09-12T03:19:59.412349+0000","last_peered":"2024-09-12T03:19:59.412349+0000","last_clean":"2024-09-12T03:19:59.412349+0000","last_became_active":"2024-09-12T03:19:57.395558+0000","last_became_peered":"2024-09-12T03:19:57.395558+0000","last_unstale":"2024-09-12T03:19:59.412349+0000","last_undegraded":"2024-09-12T03:19:59.412349+0000","last_fullsized":"2024-09-12T03:19:59.412349+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_clean_scrub_stamp":"2024-09-12T03:19:56.366974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:23:25.739217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.398297+0000","last_change":"2024-09-12T03:19:58.407844+0000","last_active":"2024-09-12T03:19:59.398297+0000","last_peered":"2024-09-12T03:19:59.398297+0000","last_clean":"2024-09-12T03:19:59.398297+0000","last_became_active":"2024-09-12T03:19:58.407513+0000","last_became_peered":"2024-09-12T03:19:58.407513+0000","last_unstale":"2024-09-12T03:19:59.398297+0000","last_undegraded":"2024-09-12T03:19:59.398297+0000","last_fullsized":"2024-09-12T03:19:59.398297+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:58:58.914180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.399665+0000","last_change":"2024-09-12T03:19:57.384562+0000","last_active":"2024-09-12T03:19:59.399665+0000","last_peered":"2024-09-12T03:19:59.399665+0000","last_clean":"2024-09-12T03:19:59.399665+0000","last_became_active":"2024-09-12T03:19:57.384227+0000","last_became_peered":"2024-09-12T03:19:57.384227+0000","last_unstale":"2024-09-12T03:19:59.399665+0000","last_undegraded":"2024-09-12T03:19:59.399665+0000","last_fullsized":"2024-09-12T03:19:59.399665+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_clean_scrub_stamp":"2024-09-12T03:19:56.366974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:52:55.095626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.412013+0000","last_change":"2024-09-12T03:19:58.406880+0000","last_active":"2024-09-12T03:19:59.412013+0000","last_peered":"2024-09-12T03:19:59.412013+0000","last_clean":"2024-09-12T03:19:59.412013+0000","last_became_active":"2024-09-12T03:19:58.406585+0000","last_became_peered":"2024-09-12T03:19:58.406585+0000","last_unstale":"2024-09-12T03:19:59.412013+0000","last_undegraded":"2024-09-12T03:19:59.412013+0000","last_fullsized":"2024-09-12T03:19:59.412013+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:44:18.879486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"66'8","reported_seq":22,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.400987+0000","last_change":"2024-09-12T03:19:57.397454+0000","last_active":"2024-09-12T03:19:59.400987+0000","last_peered":"2024-09-12T03:19:59.400987+0000","last_clean":"2024-09-12T03:19:59.400987+0000","last_became_active":"2024-09-12T03:19:57.397126+0000","last_became_peered":"2024-09-12T03:19:57.397126+0000","last_unstale":"2024-09-12T03:19:59.400987+0000","last_undegraded":"2024-09-12T03:19:59.400987+0000","last_fullsized":"2024-09-12T03:19:59.400987+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_clean_scrub_stamp":"2024-09-12T03:19:56.366974+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:14:13.101395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.400608+0000","last_change":"2024-09-12T03:19:58.402681+0000","last_active":"2024-09-12T03:19:59.400608+0000","last_peered":"2024-09-12T03:19:59.400608+0000","last_clean":"2024-09-12T03:19:59.400608+0000","last_became_active":"2024-09-12T03:19:58.402409+0000","last_became_peered":"2024-09-12T03:19:58.402409+0000","last_unstale":"2024-09-12T03:19:59.400608+0000","last_undegraded":"2024-09-12T03:19:59.400608+0000","last_fullsized":"2024-09-12T03:19:59.400608+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:56:05.472430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"66'1","reported_seq":15,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.412766+0000","last_change":"2024-09-12T03:19:57.394555+0000","last_active":"2024-09-12T03:19:59.412766+0000","last_peered":"2024-09-12T03:19:59.412766+0000","last_clean":"2024-09-12T03:19:59.412766+0000","last_became_active":"2024-09-12T03:19:57.394277+0000","last_became_peered":"2024-09-12T03:19:57.394277+0000","last_unstale":"2024-09-12T03:19:59.412766+0000","last_undegraded":"2024-09-12T03:19:59.412766+0000","last_fullsized":"2024-09-12T03:19:59.412766+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_clean_scrub_stamp":"2024-09-12T03:19:56.366974+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:50:52.123837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.412469+0000","last_change":"2024-09-12T03:19:58.411847+0000","last_active":"2024-09-12T03:19:59.412469+0000","last_peered":"2024-09-12T03:19:59.412469+0000","last_clean":"2024-09-12T03:19:59.412469+0000","last_became_active":"2024-09-12T03:19:58.409965+0000","last_became_peered":"2024-09-12T03:19:58.409965+0000","last_unstale":"2024-09-12T03:19:59.412469+0000","last_undegraded":"2024-09-12T03:19:59.412469+0000","last_fullsized":"2024-09-12T03:19:59.412469+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:45:22.536315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"66'2","reported_seq":16,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.400471+0000","last_change":"2024-09-12T03:19:57.394701+0000","last_active":"2024-09-12T03:19:59.400471+0000","last_peered":"2024-09-12T03:19:59.400471+0000","last_clean":"2024-09-12T03:19:59.400471+0000","last_became_active":"2024-09-12T03:19:57.394348+0000","last_became_peered":"2024-09-12T03:19:57.394348+0000","last_unstale":"2024-09-12T03:19:59.400471+0000","last_undegraded":"2024-09-12T03:19:59.400471+0000","last_fullsized":"2024-09-12T03:19:59.400471+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_clean_scrub_stamp":"2024-09-12T03:19:56.366974+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:44:36.098558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.411827+0000","last_change":"2024-09-12T03:19:58.400111+0000","last_active":"2024-09-12T03:19:59.411827+0000","last_peered":"2024-09-12T03:19:59.411827+0000","last_clean":"2024-09-12T03:19:59.411827+0000","last_became_active":"2024-09-12T03:19:58.399388+0000","last_became_peered":"2024-09-12T03:19:58.399388+0000","last_unstale":"2024-09-12T03:19:59.411827+0000","last_undegraded":"2024-09-12T03:19:59.411827+0000","last_fullsized":"2024-09-12T03:19:59.411827+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:06:24.498091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"66'1","reported_seq":15,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.412373+0000","last_change":"2024-09-12T03:19:57.397975+0000","last_active":"2024-09-12T03:19:59.412373+0000","last_peered":"2024-09-12T03:19:59.412373+0000","last_clean":"2024-09-12T03:19:59.412373+0000","last_became_active":"2024-09-12T03:19:57.397637+0000","last_became_peered":"2024-09-12T03:19:57.397637+0000","last_unstale":"2024-09-12T03:19:59.412373+0000","last_undegraded":"2024-09-12T03:19:59.412373+0000","last_fullsized":"2024-09-12T03:19:59.412373+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_clean_scrub_stamp":"2024-09-12T03:19:56.366974+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:16:35.102685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.411820+0000","last_change":"2024-09-12T03:19:58.399501+0000","last_active":"2024-09-12T03:19:59.411820+0000","last_peered":"2024-09-12T03:19:59.411820+0000","last_clean":"2024-09-12T03:19:59.411820+0000","last_became_active":"2024-09-12T03:19:58.399203+0000","last_became_peered":"2024-09-12T03:19:58.399203+0000","last_unstale":"2024-09-12T03:19:59.411820+0000","last_undegraded":"2024-09-12T03:19:59.411820+0000","last_fullsized":"2024-09-12T03:19:59.411820+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:45:17.689061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.412050+0000","last_change":"2024-09-12T03:19:57.394026+0000","last_active":"2024-09-12T03:19:59.412050+0000","last_peered":"2024-09-12T03:19:59.412050+0000","last_clean":"2024-09-12T03:19:59.412050+0000","last_became_active":"2024-09-12T03:19:57.393679+0000","last_became_peered":"2024-09-12T03:19:57.393679+0000","last_unstale":"2024-09-12T03:19:59.412050+0000","last_undegraded":"2024-09-12T03:19:59.412050+0000","last_fullsized":"2024-09-12T03:19:59.412050+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_clean_scrub_stamp":"2024-09-12T03:19:56.366974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:41:50.153852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.412261+0000","last_change":"2024-09-12T03:19:58.407764+0000","last_active":"2024-09-12T03:19:59.412261+0000","last_peered":"2024-09-12T03:19:59.412261+0000","last_clean":"2024-09-12T03:19:59.412261+0000","last_became_active":"2024-09-12T03:19:58.407483+0000","last_became_peered":"2024-09-12T03:19:58.407483+0000","last_unstale":"2024-09-12T03:19:59.412261+0000","last_undegraded":"2024-09-12T03:19:59.412261+0000","last_fullsized":"2024-09-12T03:19:59.412261+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:11:19.244120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.412443+0000","last_change":"2024-09-12T03:19:57.394831+0000","last_active":"2024-09-12T03:19:59.412443+0000","last_peered":"2024-09-12T03:19:59.412443+0000","last_clean":"2024-09-12T03:19:59.412443+0000","last_became_active":"2024-09-12T03:19:57.394505+0000","last_became_peered":"2024-09-12T03:19:57.394505+0000","last_unstale":"2024-09-12T03:19:59.412443+0000","last_undegraded":"2024-09-12T03:19:59.412443+0000","last_fullsized":"2024-09-12T03:19:59.412443+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_clean_scrub_stamp":"2024-09-12T03:19:56.366974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:59:35.357929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.411253+0000","last_change":"2024-09-12T03:19:58.408696+0000","last_active":"2024-09-12T03:19:59.411253+0000","last_peered":"2024-09-12T03:19:59.411253+0000","last_clean":"2024-09-12T03:19:59.411253+0000","last_became_active":"2024-09-12T03:19:58.408393+0000","last_became_peered":"2024-09-12T03:19:58.408393+0000","last_unstale":"2024-09-12T03:19:59.411253+0000","last_undegraded":"2024-09-12T03:19:59.411253+0000","last_fullsized":"2024-09-12T03:19:59.411253+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:15:02.240922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.399840+0000","last_change":"2024-09-12T03:19:57.390326+0000","last_active":"2024-09-12T03:19:59.399840+0000","last_peered":"2024-09-12T03:19:59.399840+0000","last_clean":"2024-09-12T03:19:59.399840+0000","last_became_active":"2024-09-12T03:19:57.390046+0000","last_became_peered":"2024-09-12T03:19:57.390046+0000","last_unstale":"2024-09-12T03:19:59.399840+0000","last_undegraded":"2024-09-12T03:19:59.399840+0000","last_fullsized":"2024-09-12T03:19:59.399840+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_clean_scrub_stamp":"2024-09-12T03:19:56.366974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:11:38.450285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.411564+0000","last_change":"2024-09-12T03:19:58.409403+0000","last_active":"2024-09-12T03:19:59.411564+0000","last_peered":"2024-09-12T03:19:59.411564+0000","last_clean":"2024-09-12T03:19:59.411564+0000","last_became_active":"2024-09-12T03:19:58.409087+0000","last_became_peered":"2024-09-12T03:19:58.409087+0000","last_unstale":"2024-09-12T03:19:59.411564+0000","last_undegraded":"2024-09-12T03:19:59.411564+0000","last_fullsized":"2024-09-12T03:19:59.411564+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:44:15.824316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.398237+0000","last_change":"2024-09-12T03:19:57.394055+0000","last_active":"2024-09-12T03:19:59.398237+0000","last_peered":"2024-09-12T03:19:59.398237+0000","last_clean":"2024-09-12T03:19:59.398237+0000","last_became_active":"2024-09-12T03:19:57.393767+0000","last_became_peered":"2024-09-12T03:19:57.393767+0000","last_unstale":"2024-09-12T03:19:59.398237+0000","last_undegraded":"2024-09-12T03:19:59.398237+0000","last_fullsized":"2024-09-12T03:19:59.398237+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_clean_scrub_stamp":"2024-09-12T03:19:56.366974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:34:00.504515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.398615+0000","last_change":"2024-09-12T03:19:58.400064+0000","last_active":"2024-09-12T03:19:59.398615+0000","last_peered":"2024-09-12T03:19:59.398615+0000","last_clean":"2024-09-12T03:19:59.398615+0000","last_became_active":"2024-09-12T03:19:58.399777+0000","last_became_peered":"2024-09-12T03:19:58.399777+0000","last_unstale":"2024-09-12T03:19:59.398615+0000","last_undegraded":"2024-09-12T03:19:59.398615+0000","last_fullsized":"2024-09-12T03:19:59.398615+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:31:06.800562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"66'2","reported_seq":16,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.412883+0000","last_change":"2024-09-12T03:19:57.393966+0000","last_active":"2024-09-12T03:19:59.412883+0000","last_peered":"2024-09-12T03:19:59.412883+0000","last_clean":"2024-09-12T03:19:59.412883+0000","last_became_active":"2024-09-12T03:19:57.393680+0000","last_became_peered":"2024-09-12T03:19:57.393680+0000","last_unstale":"2024-09-12T03:19:59.412883+0000","last_undegraded":"2024-09-12T03:19:59.412883+0000","last_fullsized":"2024-09-12T03:19:59.412883+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_clean_scrub_stamp":"2024-09-12T03:19:56.366974+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:05:00.306420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.400071+0000","last_change":"2024-09-12T03:19:58.404813+0000","last_active":"2024-09-12T03:19:59.400071+0000","last_peered":"2024-09-12T03:19:59.400071+0000","last_clean":"2024-09-12T03:19:59.400071+0000","last_became_active":"2024-09-12T03:19:58.404482+0000","last_became_peered":"2024-09-12T03:19:58.404482+0000","last_unstale":"2024-09-12T03:19:59.400071+0000","last_undegraded":"2024-09-12T03:19:59.400071+0000","last_fullsized":"2024-09-12T03:19:59.400071+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:11:28.704109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"67'1","reported_seq":15,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.403756+0000","last_change":"2024-09-12T03:19:57.386098+0000","last_active":"2024-09-12T03:19:59.403756+0000","last_peered":"2024-09-12T03:19:59.403756+0000","last_clean":"2024-09-12T03:19:59.403756+0000","last_became_active":"2024-09-12T03:19:57.384254+0000","last_became_peered":"2024-09-12T03:19:57.384254+0000","last_unstale":"2024-09-12T03:19:59.403756+0000","last_undegraded":"2024-09-12T03:19:59.403756+0000","last_fullsized":"2024-09-12T03:19:59.403756+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_clean_scrub_stamp":"2024-09-12T03:19:56.366974+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:40:18.965709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.411946+0000","last_change":"2024-09-12T03:19:58.408958+0000","last_active":"2024-09-12T03:19:59.411946+0000","last_peered":"2024-09-12T03:19:59.411946+0000","last_clean":"2024-09-12T03:19:59.411946+0000","last_became_active":"2024-09-12T03:19:58.408665+0000","last_became_peered":"2024-09-12T03:19:58.408665+0000","last_unstale":"2024-09-12T03:19:59.411946+0000","last_undegraded":"2024-09-12T03:19:59.411946+0000","last_fullsized":"2024-09-12T03:19:59.411946+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:50:40.770697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.411911+0000","last_change":"2024-09-12T03:19:57.393827+0000","last_active":"2024-09-12T03:19:59.411911+0000","last_peered":"2024-09-12T03:19:59.411911+0000","last_clean":"2024-09-12T03:19:59.411911+0000","last_became_active":"2024-09-12T03:19:57.393501+0000","last_became_peered":"2024-09-12T03:19:57.393501+0000","last_unstale":"2024-09-12T03:19:59.411911+0000","last_undegraded":"2024-09-12T03:19:59.411911+0000","last_fullsized":"2024-09-12T03:19:59.411911+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_clean_scrub_stamp":"2024-09-12T03:19:56.366974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:00:49.820829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.412577+0000","last_change":"2024-09-12T03:19:58.416479+0000","last_active":"2024-09-12T03:19:59.412577+0000","last_peered":"2024-09-12T03:19:59.412577+0000","last_clean":"2024-09-12T03:19:59.412577+0000","last_became_active":"2024-09-12T03:19:58.416145+0000","last_became_peered":"2024-09-12T03:19:58.416145+0000","last_unstale":"2024-09-12T03:19:59.412577+0000","last_undegraded":"2024-09-12T03:19:59.412577+0000","last_fullsized":"2024-09-12T03:19:59.412577+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:37:31.629037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.398503+0000","last_change":"2024-09-12T03:19:57.392451+0000","last_active":"2024-09-12T03:19:59.398503+0000","last_peered":"2024-09-12T03:19:59.398503+0000","last_clean":"2024-09-12T03:19:59.398503+0000","last_became_active":"2024-09-12T03:19:57.392127+0000","last_became_peered":"2024-09-12T03:19:57.392127+0000","last_unstale":"2024-09-12T03:19:59.398503+0000","last_undegraded":"2024-09-12T03:19:59.398503+0000","last_fullsized":"2024-09-12T03:19:59.398503+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_clean_scrub_stamp":"2024-09-12T03:19:56.366974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:11:02.225246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.412704+0000","last_change":"2024-09-12T03:19:58.407515+0000","last_active":"2024-09-12T03:19:59.412704+0000","last_peered":"2024-09-12T03:19:59.412704+0000","last_clean":"2024-09-12T03:19:59.412704+0000","last_became_active":"2024-09-12T03:19:58.407189+0000","last_became_peered":"2024-09-12T03:19:58.407189+0000","last_unstale":"2024-09-12T03:19:59.412704+0000","last_undegraded":"2024-09-12T03:19:59.412704+0000","last_fullsized":"2024-09-12T03:19:59.412704+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:57:01.617936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.411502+0000","last_change":"2024-09-12T03:19:57.391167+0000","last_active":"2024-09-12T03:19:59.411502+0000","last_peered":"2024-09-12T03:19:59.411502+0000","last_clean":"2024-09-12T03:19:59.411502+0000","last_became_active":"2024-09-12T03:19:57.390901+0000","last_became_peered":"2024-09-12T03:19:57.390901+0000","last_unstale":"2024-09-12T03:19:59.411502+0000","last_undegraded":"2024-09-12T03:19:59.411502+0000","last_fullsized":"2024-09-12T03:19:59.411502+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_clean_scrub_stamp":"2024-09-12T03:19:56.366974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:09:37.761223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.412332+0000","last_change":"2024-09-12T03:19:58.420294+0000","last_active":"2024-09-12T03:19:59.412332+0000","last_peered":"2024-09-12T03:19:59.412332+0000","last_clean":"2024-09-12T03:19:59.412332+0000","last_became_active":"2024-09-12T03:19:58.418912+0000","last_became_peered":"2024-09-12T03:19:58.418912+0000","last_unstale":"2024-09-12T03:19:59.412332+0000","last_undegraded":"2024-09-12T03:19:59.412332+0000","last_fullsized":"2024-09-12T03:19:59.412332+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:14:39.136845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.412515+0000","last_change":"2024-09-12T03:19:57.392009+0000","last_active":"2024-09-12T03:19:59.412515+0000","last_peered":"2024-09-12T03:19:59.412515+0000","last_clean":"2024-09-12T03:19:59.412515+0000","last_became_active":"2024-09-12T03:19:57.391668+0000","last_became_peered":"2024-09-12T03:19:57.391668+0000","last_unstale":"2024-09-12T03:19:59.412515+0000","last_undegraded":"2024-09-12T03:19:59.412515+0000","last_fullsized":"2024-09-12T03:19:59.412515+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_clean_scrub_stamp":"2024-09-12T03:19:56.366974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:41:36.777515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.411339+0000","last_change":"2024-09-12T03:19:58.403003+0000","last_active":"2024-09-12T03:19:59.411339+0000","last_peered":"2024-09-12T03:19:59.411339+0000","last_clean":"2024-09-12T03:19:59.411339+0000","last_became_active":"2024-09-12T03:19:58.402682+0000","last_became_peered":"2024-09-12T03:19:58.402682+0000","last_unstale":"2024-09-12T03:19:59.411339+0000","last_undegraded":"2024-09-12T03:19:59.411339+0000","last_fullsized":"2024-09-12T03:19:59.411339+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:55:09.544823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.399794+0000","last_change":"2024-09-12T03:19:57.393150+0000","last_active":"2024-09-12T03:19:59.399794+0000","last_peered":"2024-09-12T03:19:59.399794+0000","last_clean":"2024-09-12T03:19:59.399794+0000","last_became_active":"2024-09-12T03:19:57.392692+0000","last_became_peered":"2024-09-12T03:19:57.392692+0000","last_unstale":"2024-09-12T03:19:59.399794+0000","last_undegraded":"2024-09-12T03:19:59.399794+0000","last_fullsized":"2024-09-12T03:19:59.399794+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_clean_scrub_stamp":"2024-09-12T03:19:56.366974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:44:27.441871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.411918+0000","last_change":"2024-09-12T03:19:58.407787+0000","last_active":"2024-09-12T03:19:59.411918+0000","last_peered":"2024-09-12T03:19:59.411918+0000","last_clean":"2024-09-12T03:19:59.411918+0000","last_became_active":"2024-09-12T03:19:58.407407+0000","last_became_peered":"2024-09-12T03:19:58.407407+0000","last_unstale":"2024-09-12T03:19:59.411918+0000","last_undegraded":"2024-09-12T03:19:59.411918+0000","last_fullsized":"2024-09-12T03:19:59.411918+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:38:14.768525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.412289+0000","last_change":"2024-09-12T03:19:57.391520+0000","last_active":"2024-09-12T03:19:59.412289+0000","last_peered":"2024-09-12T03:19:59.412289+0000","last_clean":"2024-09-12T03:19:59.412289+0000","last_became_active":"2024-09-12T03:19:57.391140+0000","last_became_peered":"2024-09-12T03:19:57.391140+0000","last_unstale":"2024-09-12T03:19:59.412289+0000","last_undegraded":"2024-09-12T03:19:59.412289+0000","last_fullsized":"2024-09-12T03:19:59.412289+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_clean_scrub_stamp":"2024-09-12T03:19:56.366974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:42:32.285383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.412286+0000","last_change":"2024-09-12T03:19:58.409818+0000","last_active":"2024-09-12T03:19:59.412286+0000","last_peered":"2024-09-12T03:19:59.412286+0000","last_clean":"2024-09-12T03:19:59.412286+0000","last_became_active":"2024-09-12T03:19:58.409507+0000","last_became_peered":"2024-09-12T03:19:58.409507+0000","last_unstale":"2024-09-12T03:19:59.412286+0000","last_undegraded":"2024-09-12T03:19:59.412286+0000","last_fullsized":"2024-09-12T03:19:59.412286+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:57.371227+0000","last_clean_scrub_stamp":"2024-09-12T03:19:57.371227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:21:24.731904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"66'2","reported_seq":16,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.412444+0000","last_change":"2024-09-12T03:19:57.395758+0000","last_active":"2024-09-12T03:19:59.412444+0000","last_peered":"2024-09-12T03:19:59.412444+0000","last_clean":"2024-09-12T03:19:59.412444+0000","last_became_active":"2024-09-12T03:19:57.395422+0000","last_became_peered":"2024-09-12T03:19:57.395422+0000","last_unstale":"2024-09-12T03:19:59.412444+0000","last_undegraded":"2024-09-12T03:19:59.412444+0000","last_fullsized":"2024-09-12T03:19:59.412444+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:19:56.366974+0000","last_clean_scrub_stamp":"2024-09-12T03:19:56.366974+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:18:13.412357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":252,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-12T03:19:59.411416+0000","last_change":"2024-09-12T03:16:21.883446+0000","last_active":"2024-09-12T03:19:59.411416+0000","last_peered":"2024-09-12T03:19:59.411416+0000","last_clean":"2024-09-12T03:19:59.411416+0000","last_became_active":"2024-09-12T03:16:21.882207+0000","last_became_peered":"2024-09-12T03:16:21.882207+0000","last_unstale":"2024-09-12T03:19:59.411416+0000","last_undegraded":"2024-09-12T03:19:59.411416+0000","last_fullsized":"2024-09-12T03:19:59.411416+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:20.316426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:20.316426+0000","last_clean_scrub_stamp":"2024-09-12T03:16:20.316426+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:12:15.424320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542191,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36732,"kb_used_data":1380,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335108,"statfs":{"total":96636764160,"available":96599150592,"internally_reserved":0,"allocated":1413120,"data_stored":930339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542193,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31996,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339844,"statfs":{"total":96636764160,"available":96604000256,"internally_reserved":0,"allocated":819200,"data_stored":339447,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542192,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36144,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335696,"statfs":{"total":96636764160,"available":96599752704,"internally_reserved":0,"allocated":815104,"data_stored":339445,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542191,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31984,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339856,"statfs":{"total":96636764160,"available":96604012544,"internally_reserved":0,"allocated":815104,"data_stored":339425,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542192,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27836,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344004,"statfs":{"total":96636764160,"available":96608260096,"internally_reserved":0,"allocated":819200,"data_stored":339467,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542192,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27840,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344000,"statfs":{"total":96636764160,"available":96608256000,"internally_reserved":0,"allocated":823296,"data_stored":340967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542193,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32000,"kb_used_data":812,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339840,"statfs":{"total":96636764160,"available":96603996160,"internally_reserved":0,"allocated":831488,"data_stored":341583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542193,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28400,"kb_used_data":1372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343440,"statfs":{"total":96636764160,"available":96607682560,"internally_reserved":0,"allocated":1404928,"data_stored":929723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T03:20:06.150 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-12T03:20:06.151 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-12T03:20:06.297 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:06.295+0000 7f08d14e4640 1 Processor -- start 2024-09-12T03:20:06.297 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:06.296+0000 7f08d14e4640 1 -- start start 2024-09-12T03:20:06.298 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:06.296+0000 7f08d14e4640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f08cc15c6c0 0x7f08cc0d00a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:06.298 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:06.296+0000 7f08d14e4640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f08cc0d05e0 0x7f08cc0d1040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:06.298 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:06.296+0000 7f08d14e4640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08cc0d0a80 con 0x7f08cc0d1580 2024-09-12T03:20:06.298 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:06.296+0000 7f08d14e4640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f08cc15caa0 con 0x7f08cc15c6c0 2024-09-12T03:20:06.298 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:06.296+0000 7f08d14e4640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f08cc0b6510 con 0x7f08cc0d05e0 2024-09-12T03:20:06.298 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:06.297+0000 7f08ca7fc640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f08cc0d05e0 0x7f08cc0d1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:06.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:06.297+0000 7f08caffd640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f08cc15c6c0 0x7f08cc0d00a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:06.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:06.297+0000 7f08caffd640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f08cc15c6c0 0x7f08cc0d00a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.27:48642/0 (socket says 172.21.15.27:48642) 2024-09-12T03:20:06.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:06.297+0000 7f08caffd640 1 -- 172.21.15.27:0/3242338681 learned_addr learned my addr 172.21.15.27:0/3242338681 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:20:06.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:06.297+0000 7f08ca7fc640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f08cc0d05e0 0x7f08cc0d1040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3301/0 says I am v2:172.21.15.27:42456/0 (socket says 172.21.15.27:42456) 2024-09-12T03:20:06.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:06.297+0000 7f08caffd640 1 -- 172.21.15.27:0/3242338681 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f08cc0d05e0 msgr2=0x7f08cc0d1040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:06.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:06.297+0000 7f08caffd640 1 --2- 172.21.15.27:0/3242338681 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f08cc0d05e0 0x7f08cc0d1040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:06.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:06.297+0000 7f08caffd640 1 -- 172.21.15.27:0/3242338681 >> v1:172.21.15.45:6789/0 conn(0x7f08cc0d1580 legacy=0x7f08cc0d24f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:06.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:06.297+0000 7f08caffd640 1 -- 172.21.15.27:0/3242338681 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08cc0d0d60 con 0x7f08cc15c6c0 2024-09-12T03:20:06.300 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:06.298+0000 7f08caffd640 1 --2- 172.21.15.27:0/3242338681 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f08cc15c6c0 0x7f08cc0d00a0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f08bc0098a0 tx=0x7f08bc02a200 comp rx=0 tx=0).ready entity=mon.1 client_cookie=29eaab65b38b2a22 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:06.300 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:06.298+0000 7f08c9ffb640 1 -- 172.21.15.27:0/3242338681 <== mon.1 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08bc037070 con 0x7f08cc15c6c0 2024-09-12T03:20:06.301 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:06.298+0000 7f08c9ffb640 1 -- 172.21.15.27:0/3242338681 <== mon.1 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08bc032440 con 0x7f08cc15c6c0 2024-09-12T03:20:06.301 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:06.298+0000 7f08c9ffb640 1 -- 172.21.15.27:0/3242338681 <== mon.1 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08bc032810 con 0x7f08cc15c6c0 2024-09-12T03:20:06.301 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:06.299+0000 7f08d14e4640 1 -- 172.21.15.27:0/3242338681 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f08cc15c6c0 msgr2=0x7f08cc0d00a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:06.301 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:06.299+0000 7f08d14e4640 1 --2- 172.21.15.27:0/3242338681 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f08cc15c6c0 0x7f08cc0d00a0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f08bc0098a0 tx=0x7f08bc02a200 comp rx=0 tx=0).stop 2024-09-12T03:20:06.301 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:06.299+0000 7f08d14e4640 1 -- 172.21.15.27:0/3242338681 shutdown_connections 2024-09-12T03:20:06.301 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:06.299+0000 7f08d14e4640 1 --2- 172.21.15.27:0/3242338681 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f08cc0d05e0 0x7f08cc0d1040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:06.301 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:06.299+0000 7f08d14e4640 1 --2- 172.21.15.27:0/3242338681 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f08cc15c6c0 0x7f08cc0d00a0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:06.301 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:06.299+0000 7f08d14e4640 1 -- 172.21.15.27:0/3242338681 >> 172.21.15.27:0/3242338681 conn(0x7f08cc06f320 msgr2=0x7f08cc06f700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:06.301 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:06.299+0000 7f08d14e4640 1 -- 172.21.15.27:0/3242338681 shutdown_connections 2024-09-12T03:20:06.301 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:06.299+0000 7f08d14e4640 1 -- 172.21.15.27:0/3242338681 wait complete. 2024-09-12T03:20:06.304 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:06.300+0000 7f08d14e4640 1 Processor -- start 2024-09-12T03:20:06.305 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:06.300+0000 7f08d14e4640 1 -- start start 2024-09-12T03:20:06.305 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:06.301+0000 7f08d14e4640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f08cc0d05e0 0x7f08cc148740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:06.305 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:06.301+0000 7f08d14e4640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f08cc0d1580 0x7f08cc0ce0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:06.305 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:06.301+0000 7f08d14e4640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f08cc15c6c0 0x7f08cc0ce600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:06.305 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:06.301+0000 7f08d14e4640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f08cc0d1a10 con 0x7f08cc0d05e0 2024-09-12T03:20:06.305 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:06.301+0000 7f08d14e4640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f08cc0b6a50 con 0x7f08cc15c6c0 2024-09-12T03:20:06.305 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:06.301+0000 7f08d14e4640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f08cc0b4400 con 0x7f08cc0d1580 2024-09-12T03:20:06.305 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:06.301+0000 7f08caffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f08cc0d05e0 0x7f08cc148740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:06.305 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:06.301+0000 7f08caffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f08cc0d05e0 0x7f08cc148740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56276/0 (socket says 172.21.15.27:56276) 2024-09-12T03:20:06.305 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:06.301+0000 7f08caffd640 1 -- 172.21.15.27:0/2456612953 learned_addr learned my addr 172.21.15.27:0/2456612953 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:20:06.306 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:06.301+0000 7f08ca7fc640 1 --2- 172.21.15.27:0/2456612953 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f08cc0d1580 0x7f08cc0ce0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:06.306 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:06.301+0000 7f08cb7fe640 1 --2- 172.21.15.27:0/2456612953 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f08cc15c6c0 0x7f08cc0ce600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:06.306 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:06.301+0000 7f08caffd640 1 -- 172.21.15.27:0/2456612953 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f08cc0d1580 msgr2=0x7f08cc0ce0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:06.306 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:06.301+0000 7f08caffd640 1 --2- 172.21.15.27:0/2456612953 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f08cc0d1580 0x7f08cc0ce0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:06.306 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:06.301+0000 7f08caffd640 1 -- 172.21.15.27:0/2456612953 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f08cc15c6c0 msgr2=0x7f08cc0ce600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:06.306 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:06.301+0000 7f08caffd640 1 --2- 172.21.15.27:0/2456612953 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f08cc15c6c0 0x7f08cc0ce600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:06.306 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:06.301+0000 7f08caffd640 1 -- 172.21.15.27:0/2456612953 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08cc0ca520 con 0x7f08cc0d05e0 2024-09-12T03:20:06.306 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:06.302+0000 7f08caffd640 1 --2- 172.21.15.27:0/2456612953 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f08cc0d05e0 0x7f08cc148740 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f08bc0098a0 tx=0x7f08bc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:06.306 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:06.302+0000 7f08b37fe640 1 -- 172.21.15.27:0/2456612953 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08bc041070 con 0x7f08cc0d05e0 2024-09-12T03:20:06.306 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:06.302+0000 7f08b37fe640 1 -- 172.21.15.27:0/2456612953 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08bc03c870 con 0x7f08cc0d05e0 2024-09-12T03:20:06.306 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:06.302+0000 7f08d14e4640 1 -- 172.21.15.27:0/2456612953 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08cc0ca800 con 0x7f08cc0d05e0 2024-09-12T03:20:06.306 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:06.302+0000 7f08b37fe640 1 -- 172.21.15.27:0/2456612953 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08bc037040 con 0x7f08cc0d05e0 2024-09-12T03:20:06.307 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:06.302+0000 7f08d14e4640 1 -- 172.21.15.27:0/2456612953 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08cc0cadc0 con 0x7f08cc0d05e0 2024-09-12T03:20:06.307 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:06.303+0000 7f08d14e4640 1 -- 172.21.15.27:0/2456612953 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08cc0cb000 con 0x7f08cc0d05e0 2024-09-12T03:20:06.307 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:06.303+0000 7f08b37fe640 1 -- 172.21.15.27:0/2456612953 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 11) ==== 90804+0+0 (secure 0 0 0) 0x7f08bc044070 con 0x7f08cc0d05e0 2024-09-12T03:20:06.307 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:06.304+0000 7f08b37fe640 1 --2- 172.21.15.27:0/2456612953 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f08b806c190 0x7f08b806e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:06.307 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:06.304+0000 7f08b37fe640 1 -- 172.21.15.27:0/2456612953 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 5709+0+0 (secure 0 0 0) 0x7f08bc0abe20 con 0x7f08cc0d05e0 2024-09-12T03:20:06.307 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:06.304+0000 7f08ca7fc640 1 --2- 172.21.15.27:0/2456612953 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f08b806c190 0x7f08b806e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:06.308 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:06.305+0000 7f08ca7fc640 1 --2- 172.21.15.27:0/2456612953 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f08b806c190 0x7f08b806e650 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f08cc0690e0 tx=0x7f08c0009190 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:06.309 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:06.308+0000 7f08b37fe640 1 -- 172.21.15.27:0/2456612953 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f08cc0cb000 con 0x7f08cc0d05e0 2024-09-12T03:20:06.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:06.553+0000 7f08d14e4640 1 -- 172.21.15.27:0/2456612953 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f08cc15dbe0 con 0x7f08cc0d05e0 2024-09-12T03:20:06.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:06.555+0000 7f08b37fe640 1 -- 172.21.15.27:0/2456612953 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v16) ==== 190+0+147 (secure 0 0 0) 0x7f08bc079d90 con 0x7f08cc0d05e0 2024-09-12T03:20:06.556 INFO:teuthology.orchestra.run.smithi027.stdout:[client.0] 2024-09-12T03:20:06.557 INFO:teuthology.orchestra.run.smithi027.stdout: key = AQDVXOJmLm+5ExAAECM4CKfuVElmp/n0b02mXg== 2024-09-12T03:20:06.557 INFO:teuthology.orchestra.run.smithi027.stdout: caps mds = "allow" 2024-09-12T03:20:06.557 INFO:teuthology.orchestra.run.smithi027.stdout: caps mon = "allow r" 2024-09-12T03:20:06.557 INFO:teuthology.orchestra.run.smithi027.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-12T03:20:06.557 INFO:teuthology.orchestra.run.smithi027.stderr:updated caps for client.0 2024-09-12T03:20:06.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:06.557+0000 7f08d14e4640 1 -- 172.21.15.27:0/2456612953 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f08b806c190 msgr2=0x7f08b806e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:06.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:06.557+0000 7f08d14e4640 1 --2- 172.21.15.27:0/2456612953 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f08b806c190 0x7f08b806e650 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f08cc0690e0 tx=0x7f08c0009190 comp rx=0 tx=0).stop 2024-09-12T03:20:06.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:06.557+0000 7f08d14e4640 1 -- 172.21.15.27:0/2456612953 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f08cc0d05e0 msgr2=0x7f08cc148740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:06.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:06.557+0000 7f08d14e4640 1 --2- 172.21.15.27:0/2456612953 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f08cc0d05e0 0x7f08cc148740 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f08bc0098a0 tx=0x7f08bc0046c0 comp rx=0 tx=0).stop 2024-09-12T03:20:06.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:06.557+0000 7f08d14e4640 1 -- 172.21.15.27:0/2456612953 shutdown_connections 2024-09-12T03:20:06.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:06.557+0000 7f08d14e4640 1 --2- 172.21.15.27:0/2456612953 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f08b806c190 0x7f08b806e650 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:06.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:06.558+0000 7f08d14e4640 1 --2- 172.21.15.27:0/2456612953 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f08cc15c6c0 0x7f08cc0ce600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:06.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:06.558+0000 7f08d14e4640 1 --2- 172.21.15.27:0/2456612953 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f08cc0d1580 0x7f08cc0ce0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:06.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:06.558+0000 7f08d14e4640 1 --2- 172.21.15.27:0/2456612953 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f08cc0d05e0 0x7f08cc148740 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:06.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:06.558+0000 7f08d14e4640 1 -- 172.21.15.27:0/2456612953 >> 172.21.15.27:0/2456612953 conn(0x7f08cc06f320 msgr2=0x7f08cc15f7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:06.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:06.558+0000 7f08d14e4640 1 -- 172.21.15.27:0/2456612953 shutdown_connections 2024-09-12T03:20:06.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:06.558+0000 7f08d14e4640 1 -- 172.21.15.27:0/2456612953 wait complete. 2024-09-12T03:20:06.570 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-12T03:20:06.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:06.672+0000 7f45195e4640 1 Processor -- start 2024-09-12T03:20:06.677 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:06.672+0000 7f45195e4640 1 -- start start 2024-09-12T03:20:06.682 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:06.672+0000 7f45195e4640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f451415e5f0 0x7f4514073250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:06.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:06.672+0000 7f45195e4640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4514073bb0 con 0x7f4514073790 2024-09-12T03:20:06.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:06.673+0000 7f45195e4640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f45140d2240 con 0x7f4514162a30 2024-09-12T03:20:06.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:06.673+0000 7f45195e4640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f45140d0da0 con 0x7f451415e5f0 2024-09-12T03:20:06.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:06.673+0000 7f45127fc640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f4514073790 0x7f4514162320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:34880/0 (socket says 172.21.15.27:34880) 2024-09-12T03:20:06.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:06.673+0000 7f45127fc640 1 -- 172.21.15.27:0/2459493020 learned_addr learned my addr 172.21.15.27:0/2459493020 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:20:06.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:06.673+0000 7f4512ffd640 1 --2- 172.21.15.27:0/2459493020 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f451415e5f0 0x7f4514073250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:06.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:06.673+0000 7f4511ffb640 1 -- 172.21.15.27:0/2459493020 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3887821964 0 0) 0x7f4514073bb0 con 0x7f4514073790 2024-09-12T03:20:06.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:06.673+0000 7f4511ffb640 1 -- 172.21.15.27:0/2459493020 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4500003520 con 0x7f4514073790 2024-09-12T03:20:06.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:06.673+0000 7f4511ffb640 1 -- 172.21.15.27:0/2459493020 <== mon.1 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1786191459 0 0) 0x7f45140d2240 con 0x7f4514162a30 2024-09-12T03:20:06.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:06.673+0000 7f4512ffd640 1 -- 172.21.15.27:0/2459493020 >> v1:172.21.15.45:6789/0 conn(0x7f4514162a30 legacy=0x7f451415e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:06.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:06.673+0000 7f4512ffd640 1 -- 172.21.15.27:0/2459493020 >> v1:172.21.15.27:6789/0 conn(0x7f4514073790 legacy=0x7f4514162320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:06.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:06.673+0000 7f4512ffd640 1 -- 172.21.15.27:0/2459493020 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4514163090 con 0x7f451415e5f0 2024-09-12T03:20:06.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:06.674+0000 7f4512ffd640 1 --2- 172.21.15.27:0/2459493020 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f451415e5f0 0x7f4514073250 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f4504009dc0 tx=0x7f450402a640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5c0dc20c2647e223 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:06.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:06.674+0000 7f4511ffb640 1 -- 172.21.15.27:0/2459493020 <== mon.2 v2:172.21.15.45:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4504035020 con 0x7f451415e5f0 2024-09-12T03:20:06.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:06.674+0000 7f4511ffb640 1 -- 172.21.15.27:0/2459493020 <== mon.2 v2:172.21.15.45:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4504007540 con 0x7f451415e5f0 2024-09-12T03:20:06.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:06.674+0000 7f4511ffb640 1 -- 172.21.15.27:0/2459493020 <== mon.2 v2:172.21.15.45:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44fc001670 con 0x7f451415e5f0 2024-09-12T03:20:06.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:06.675+0000 7f45195e4640 1 -- 172.21.15.27:0/2459493020 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f451415e5f0 msgr2=0x7f4514073250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:06.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:06.675+0000 7f45195e4640 1 --2- 172.21.15.27:0/2459493020 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f451415e5f0 0x7f4514073250 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f4504009dc0 tx=0x7f450402a640 comp rx=0 tx=0).stop 2024-09-12T03:20:06.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:06.675+0000 7f45195e4640 1 -- 172.21.15.27:0/2459493020 shutdown_connections 2024-09-12T03:20:06.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:06.675+0000 7f45195e4640 1 --2- 172.21.15.27:0/2459493020 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f451415e5f0 0x7f4514073250 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:06.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:06.675+0000 7f45195e4640 1 -- 172.21.15.27:0/2459493020 >> 172.21.15.27:0/2459493020 conn(0x7f451406f300 msgr2=0x7f451406f710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:06.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:06.675+0000 7f45195e4640 1 -- 172.21.15.27:0/2459493020 shutdown_connections 2024-09-12T03:20:06.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:06.675+0000 7f45195e4640 1 -- 172.21.15.27:0/2459493020 wait complete. 2024-09-12T03:20:06.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:06.676+0000 7f45195e4640 1 Processor -- start 2024-09-12T03:20:06.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:06.676+0000 7f45195e4640 1 -- start start 2024-09-12T03:20:06.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:06.677+0000 7f45195e4640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4514073790 0x7f451414e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:06.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:06.677+0000 7f45195e4640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f451415e5f0 0x7f4514154690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:06.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:06.677+0000 7f45195e4640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4514162a30 0x7f45141515a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:06.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:06.677+0000 7f45195e4640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f45140d0a30 con 0x7f451415e5f0 2024-09-12T03:20:06.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:06.677+0000 7f45195e4640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f45140d0750 con 0x7f4514073790 2024-09-12T03:20:06.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:06.677+0000 7f45195e4640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f45140b5620 con 0x7f4514162a30 2024-09-12T03:20:06.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:06.677+0000 7f45127fc640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f451415e5f0 0x7f4514154690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:06.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:06.677+0000 7f45127fc640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f451415e5f0 0x7f4514154690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56288/0 (socket says 172.21.15.27:56288) 2024-09-12T03:20:06.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:06.677+0000 7f45127fc640 1 -- 172.21.15.27:0/3415479275 learned_addr learned my addr 172.21.15.27:0/3415479275 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:20:06.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:06.677+0000 7f45137fe640 1 --2- 172.21.15.27:0/3415479275 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4514162a30 0x7f45141515a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:06.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:06.677+0000 7f4512ffd640 1 --2- 172.21.15.27:0/3415479275 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4514073790 0x7f451414e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:06.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:06.677+0000 7f45127fc640 1 -- 172.21.15.27:0/3415479275 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4514162a30 msgr2=0x7f45141515a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:06.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:06.677+0000 7f45127fc640 1 --2- 172.21.15.27:0/3415479275 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4514162a30 0x7f45141515a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:06.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:06.677+0000 7f45127fc640 1 -- 172.21.15.27:0/3415479275 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4514073790 msgr2=0x7f451414e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:06.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:06.677+0000 7f45127fc640 1 --2- 172.21.15.27:0/3415479275 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4514073790 0x7f451414e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:06.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:06.677+0000 7f45127fc640 1 -- 172.21.15.27:0/3415479275 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f451414f0b0 con 0x7f451415e5f0 2024-09-12T03:20:06.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:06.678+0000 7f45127fc640 1 --2- 172.21.15.27:0/3415479275 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f451415e5f0 0x7f4514154690 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f44fc00c710 tx=0x7f44fc00cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:06.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:06.678+0000 7f450b7fe640 1 -- 172.21.15.27:0/3415479275 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44fc005ce0 con 0x7f451415e5f0 2024-09-12T03:20:06.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:06.678+0000 7f450b7fe640 1 -- 172.21.15.27:0/3415479275 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44fc00ce20 con 0x7f451415e5f0 2024-09-12T03:20:06.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:06.678+0000 7f45195e4640 1 -- 172.21.15.27:0/3415479275 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f451415ab70 con 0x7f451415e5f0 2024-09-12T03:20:06.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:06.678+0000 7f450b7fe640 1 -- 172.21.15.27:0/3415479275 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44fc0072b0 con 0x7f451415e5f0 2024-09-12T03:20:06.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:06.678+0000 7f45195e4640 1 -- 172.21.15.27:0/3415479275 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f451415d810 con 0x7f451415e5f0 2024-09-12T03:20:06.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:06.679+0000 7f45195e4640 1 -- 172.21.15.27:0/3415479275 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4514073250 con 0x7f451415e5f0 2024-09-12T03:20:06.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:06.679+0000 7f450b7fe640 1 -- 172.21.15.27:0/3415479275 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 11) ==== 90804+0+0 (secure 0 0 0) 0x7f44fc007450 con 0x7f451415e5f0 2024-09-12T03:20:06.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:06.680+0000 7f450b7fe640 1 --2- 172.21.15.27:0/3415479275 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f44f406c190 0x7f44f406e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:06.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:06.680+0000 7f450b7fe640 1 -- 172.21.15.27:0/3415479275 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 5709+0+0 (secure 0 0 0) 0x7f44fc08b410 con 0x7f451415e5f0 2024-09-12T03:20:06.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:06.680+0000 7f4512ffd640 1 --2- 172.21.15.27:0/3415479275 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f44f406c190 0x7f44f406e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:06.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:06.680+0000 7f4512ffd640 1 --2- 172.21.15.27:0/3415479275 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f44f406c190 0x7f44f406e650 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f4504033000 tx=0x7f450402f000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:06.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:06.684+0000 7f450b7fe640 1 -- 172.21.15.27:0/3415479275 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f44fc0598b0 con 0x7f451415e5f0 2024-09-12T03:20:06.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:06.897+0000 7f45195e4640 1 -- 172.21.15.27:0/3415479275 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f45141608e0 con 0x7f451415e5f0 2024-09-12T03:20:06.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:06.898+0000 7f450b7fe640 1 -- 172.21.15.27:0/3415479275 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) ==== 94+0+5225 (secure 0 0 0) 0x7f44fc0591b0 con 0x7f451415e5f0 2024-09-12T03:20:06.899 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:20:06.899 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":38,"btime":"2024-09-12T03:20:00:397329+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6045,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6834/3058537826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":3058537826},{"type":"v1","addr":"172.21.15.27:6834","nonce":3058537826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":34},{"gid":6048,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/1585628536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":1585628536},{"type":"v1","addr":"172.21.15.27:6835","nonce":1585628536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":34},{"gid":6063,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6836/1421588365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":1421588365},{"type":"v1","addr":"172.21.15.45:6836","nonce":1421588365}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-12T03:19:57.738864+0000","modified":"2024-09-12T03:19:59.740115+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6066},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6066":{"gid":6066,"name":"d","rank":0,"incarnation":36,"state":"up:active","state_seq":3,"addr":"172.21.15.45:6837/2890007436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6835","nonce":2890007436},{"type":"v1","addr":"172.21.15.45:6837","nonce":2890007436}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6066,"qdb_cluster":[6066]},"id":5}]} 2024-09-12T03:20:06.899 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 38 2024-09-12T03:20:06.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:06.900+0000 7f45195e4640 1 -- 172.21.15.27:0/3415479275 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f44f406c190 msgr2=0x7f44f406e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:06.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:06.900+0000 7f45195e4640 1 --2- 172.21.15.27:0/3415479275 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f44f406c190 0x7f44f406e650 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f4504033000 tx=0x7f450402f000 comp rx=0 tx=0).stop 2024-09-12T03:20:06.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:06.900+0000 7f45195e4640 1 -- 172.21.15.27:0/3415479275 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f451415e5f0 msgr2=0x7f4514154690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:06.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:06.900+0000 7f45195e4640 1 --2- 172.21.15.27:0/3415479275 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f451415e5f0 0x7f4514154690 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f44fc00c710 tx=0x7f44fc00cbd0 comp rx=0 tx=0).stop 2024-09-12T03:20:06.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:06.900+0000 7f45195e4640 1 -- 172.21.15.27:0/3415479275 shutdown_connections 2024-09-12T03:20:06.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:06.900+0000 7f45195e4640 1 --2- 172.21.15.27:0/3415479275 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f44f406c190 0x7f44f406e650 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:06.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:06.900+0000 7f45195e4640 1 --2- 172.21.15.27:0/3415479275 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4514162a30 0x7f45141515a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:06.902 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:06.900+0000 7f45195e4640 1 --2- 172.21.15.27:0/3415479275 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f451415e5f0 0x7f4514154690 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:06.902 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:06.900+0000 7f45195e4640 1 --2- 172.21.15.27:0/3415479275 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4514073790 0x7f451414e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:06.902 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:06.900+0000 7f45195e4640 1 -- 172.21.15.27:0/3415479275 >> 172.21.15.27:0/3415479275 conn(0x7f451406f300 msgr2=0x7f45140b5d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:06.902 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:06.900+0000 7f45195e4640 1 -- 172.21.15.27:0/3415479275 shutdown_connections 2024-09-12T03:20:06.902 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:06.900+0000 7f45195e4640 1 -- 172.21.15.27:0/3415479275 wait complete. 2024-09-12T03:20:07.315 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 38, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-12T03:19:57.738864+0000', 'modified': '2024-09-12T03:19:59.740115+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6066}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6066': {'gid': 6066, 'name': 'd', 'rank': 0, 'incarnation': 36, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.45:6837/2890007436', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.45:6835', 'nonce': 2890007436}, {'type': 'v1', 'addr': '172.21.15.45:6837', 'nonce': 2890007436}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [11], 'metadata_pool': 10, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6066, 'qdb_cluster': [6066]} 2024-09-12T03:20:07.315 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-12T03:20:07.315 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-09-12T03:20:07.422 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.421+0000 7f2a00d52640 1 Processor -- start 2024-09-12T03:20:07.422 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.421+0000 7f2a00d52640 1 -- start start 2024-09-12T03:20:07.423 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.422+0000 7f2a00d52640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f29fc0d1050 0x7f29fc0cf810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:07.424 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.422+0000 7f2a00d52640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f29fc0d52e0 0x7f29fc0d1430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:07.424 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.422+0000 7f2a00d52640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f29fc0d1e60 con 0x7f29fc162520 2024-09-12T03:20:07.424 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.422+0000 7f2a00d52640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f29fc0d2380 con 0x7f29fc0d52e0 2024-09-12T03:20:07.424 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.422+0000 7f2a00d52640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f29fc0cfef0 con 0x7f29fc0d1050 2024-09-12T03:20:07.424 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.422+0000 7f29f3fff640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f29fc0d52e0 0x7f29fc0d1430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:07.424 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.422+0000 7f29fa59c640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f29fc0d1050 0x7f29fc0cf810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:07.424 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.422+0000 7f29f3fff640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f29fc0d52e0 0x7f29fc0d1430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.27:48668/0 (socket says 172.21.15.27:48668) 2024-09-12T03:20:07.424 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.422+0000 7f29f3fff640 1 -- 172.21.15.27:0/4055705026 learned_addr learned my addr 172.21.15.27:0/4055705026 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:20:07.424 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.423+0000 7f29f3fff640 1 -- 172.21.15.27:0/4055705026 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f29fc0d1050 msgr2=0x7f29fc0cf810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:07.425 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.423+0000 7f29f3fff640 1 --2- 172.21.15.27:0/4055705026 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f29fc0d1050 0x7f29fc0cf810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:07.425 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.423+0000 7f29f3fff640 1 -- 172.21.15.27:0/4055705026 >> v1:172.21.15.45:6789/0 conn(0x7f29fc162520 legacy=0x7f29fc1629d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:07.425 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.423+0000 7f29f3fff640 1 -- 172.21.15.27:0/4055705026 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29fc0d2640 con 0x7f29fc0d52e0 2024-09-12T03:20:07.426 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.423+0000 7f29f3fff640 1 --2- 172.21.15.27:0/4055705026 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f29fc0d52e0 0x7f29fc0d1430 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f29e4009870 tx=0x7f29e402a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=39968270aa424609 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:07.426 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.424+0000 7f29f9d9b640 1 -- 172.21.15.27:0/4055705026 <== mon.1 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29e4037070 con 0x7f29fc0d52e0 2024-09-12T03:20:07.426 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.424+0000 7f29f9d9b640 1 -- 172.21.15.27:0/4055705026 <== mon.1 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29e4032440 con 0x7f29fc0d52e0 2024-09-12T03:20:07.426 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.424+0000 7f29f9d9b640 1 -- 172.21.15.27:0/4055705026 <== mon.1 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29e4032810 con 0x7f29fc0d52e0 2024-09-12T03:20:07.426 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.424+0000 7f2a00d52640 1 -- 172.21.15.27:0/4055705026 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f29fc0d52e0 msgr2=0x7f29fc0d1430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:07.426 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.424+0000 7f2a00d52640 1 --2- 172.21.15.27:0/4055705026 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f29fc0d52e0 0x7f29fc0d1430 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f29e4009870 tx=0x7f29e402a270 comp rx=0 tx=0).stop 2024-09-12T03:20:07.426 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.424+0000 7f2a00d52640 1 -- 172.21.15.27:0/4055705026 shutdown_connections 2024-09-12T03:20:07.426 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.424+0000 7f2a00d52640 1 --2- 172.21.15.27:0/4055705026 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f29fc0d52e0 0x7f29fc0d1430 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:07.426 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.424+0000 7f2a00d52640 1 --2- 172.21.15.27:0/4055705026 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f29fc0d1050 0x7f29fc0cf810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:07.427 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.424+0000 7f2a00d52640 1 -- 172.21.15.27:0/4055705026 >> 172.21.15.27:0/4055705026 conn(0x7f29fc06ddf0 msgr2=0x7f29fc06e200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:07.427 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.424+0000 7f2a00d52640 1 -- 172.21.15.27:0/4055705026 shutdown_connections 2024-09-12T03:20:07.427 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.424+0000 7f2a00d52640 1 -- 172.21.15.27:0/4055705026 wait complete. 2024-09-12T03:20:07.429 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.425+0000 7f2a00d52640 1 Processor -- start 2024-09-12T03:20:07.429 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.426+0000 7f2a00d52640 1 -- start start 2024-09-12T03:20:07.429 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.426+0000 7f2a00d52640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f29fc0d1050 0x7f29fc0ca170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:07.429 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.426+0000 7f2a00d52640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f29fc0d52e0 0x7f29fc0c82a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:07.429 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.426+0000 7f2a00d52640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f29fc162520 0x7f29fc0c87e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:07.429 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.426+0000 7f2a00d52640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f29fc0d1970 con 0x7f29fc0d52e0 2024-09-12T03:20:07.429 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.426+0000 7f2a00d52640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f29fc0d2c00 con 0x7f29fc162520 2024-09-12T03:20:07.429 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.426+0000 7f2a00d52640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f29fc0b7a50 con 0x7f29fc0d1050 2024-09-12T03:20:07.429 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.426+0000 7f29f3fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f29fc0d52e0 0x7f29fc0c82a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:07.430 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.426+0000 7f29f3fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f29fc0d52e0 0x7f29fc0c82a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56294/0 (socket says 172.21.15.27:56294) 2024-09-12T03:20:07.430 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.426+0000 7f29f3fff640 1 -- 172.21.15.27:0/3981226465 learned_addr learned my addr 172.21.15.27:0/3981226465 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:20:07.430 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.426+0000 7f29fa59c640 1 --2- 172.21.15.27:0/3981226465 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f29fc0d1050 0x7f29fc0ca170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:07.430 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.426+0000 7f29fad9d640 1 --2- 172.21.15.27:0/3981226465 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f29fc162520 0x7f29fc0c87e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:07.430 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.427+0000 7f29f3fff640 1 -- 172.21.15.27:0/3981226465 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f29fc0d1050 msgr2=0x7f29fc0ca170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:07.430 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.427+0000 7f29f3fff640 1 --2- 172.21.15.27:0/3981226465 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f29fc0d1050 0x7f29fc0ca170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:07.430 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.427+0000 7f29f3fff640 1 -- 172.21.15.27:0/3981226465 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f29fc162520 msgr2=0x7f29fc0c87e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:07.430 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.427+0000 7f29f3fff640 1 --2- 172.21.15.27:0/3981226465 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f29fc162520 0x7f29fc0c87e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:07.430 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.427+0000 7f29f3fff640 1 -- 172.21.15.27:0/3981226465 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29fc0ca900 con 0x7f29fc0d52e0 2024-09-12T03:20:07.430 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.427+0000 7f29f3fff640 1 --2- 172.21.15.27:0/3981226465 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f29fc0d52e0 0x7f29fc0c82a0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f29e4000c00 tx=0x7f29e40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:07.431 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.427+0000 7f29f2ffd640 1 -- 172.21.15.27:0/3981226465 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29e4037040 con 0x7f29fc0d52e0 2024-09-12T03:20:07.431 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.427+0000 7f29f2ffd640 1 -- 172.21.15.27:0/3981226465 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29e4042070 con 0x7f29fc0d52e0 2024-09-12T03:20:07.431 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.427+0000 7f2a00d52640 1 -- 172.21.15.27:0/3981226465 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29fc0cabe0 con 0x7f29fc0d52e0 2024-09-12T03:20:07.431 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.427+0000 7f29f2ffd640 1 -- 172.21.15.27:0/3981226465 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29e4030880 con 0x7f29fc0d52e0 2024-09-12T03:20:07.431 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.427+0000 7f2a00d52640 1 -- 172.21.15.27:0/3981226465 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29fc0cb1a0 con 0x7f29fc0d52e0 2024-09-12T03:20:07.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.429+0000 7f29f2ffd640 1 -- 172.21.15.27:0/3981226465 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 11) ==== 90804+0+0 (secure 0 0 0) 0x7f29e4030a20 con 0x7f29fc0d52e0 2024-09-12T03:20:07.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.429+0000 7f29f2ffd640 1 --2- 172.21.15.27:0/3981226465 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f29dc06c190 0x7f29dc06e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:07.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.430+0000 7f29f2ffd640 1 -- 172.21.15.27:0/3981226465 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 5709+0+0 (secure 0 0 0) 0x7f29e40aba50 con 0x7f29fc0d52e0 2024-09-12T03:20:07.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.430+0000 7f29fa59c640 1 --2- 172.21.15.27:0/3981226465 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f29dc06c190 0x7f29dc06e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:07.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.430+0000 7f29fa59c640 1 --2- 172.21.15.27:0/3981226465 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f29dc06c190 0x7f29dc06e650 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f29ec009aa0 tx=0x7f29ec009190 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:07.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.430+0000 7f2a03bcb740 1 Processor -- start 2024-09-12T03:20:07.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.431+0000 7f2a03bcb740 1 -- start start 2024-09-12T03:20:07.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.431+0000 7f2a03bcb740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562242bb52c0 0x562242bfaa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:07.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.431+0000 7f2a03bcb740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562242bb2a80 0x562242bb56a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:07.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.431+0000 7f2a03bcb740 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x562242bb6a50 0x562242baf8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:07.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.431+0000 7f2a03bcb740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x562242b17520 con 0x562242bb2a80 2024-09-12T03:20:07.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.431+0000 7f2a03bcb740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x562242abf3e0 con 0x562242bb52c0 2024-09-12T03:20:07.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.431+0000 7f2a03bcb740 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x562242b134e0 con 0x562242bb6a50 2024-09-12T03:20:07.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.431+0000 7f29f3fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562242bb2a80 0x562242bb56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:07.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.431+0000 7f29f3fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562242bb2a80 0x562242bb56a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56300/0 (socket says 172.21.15.27:56300) 2024-09-12T03:20:07.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.431+0000 7f29f3fff640 1 -- 172.21.15.27:0/279590635 learned_addr learned my addr 172.21.15.27:0/279590635 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:20:07.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.432+0000 7f29fa59c640 1 --2- 172.21.15.27:0/279590635 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562242bb52c0 0x562242bfaa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:07.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.432+0000 7f29fad9d640 1 --2- 172.21.15.27:0/279590635 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x562242bb6a50 0x562242baf8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:07.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.432+0000 7f29f3fff640 1 -- 172.21.15.27:0/279590635 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x562242bb6a50 msgr2=0x562242baf8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:07.437 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.432+0000 7f29f3fff640 1 --2- 172.21.15.27:0/279590635 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x562242bb6a50 0x562242baf8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:07.437 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.432+0000 7f29f3fff640 1 -- 172.21.15.27:0/279590635 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562242bb52c0 msgr2=0x562242bfaa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:07.437 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.432+0000 7f29f3fff640 1 --2- 172.21.15.27:0/279590635 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562242bb52c0 0x562242bfaa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:07.437 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.432+0000 7f29f3fff640 1 -- 172.21.15.27:0/279590635 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562242ada2f0 con 0x562242bb2a80 2024-09-12T03:20:07.437 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.432+0000 7f29f3fff640 1 --2- 172.21.15.27:0/279590635 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562242bb2a80 0x562242bb56a0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f29e4075e00 tx=0x7f29e4079560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecb2ec1dfa8f5f9a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:07.437 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.432+0000 7f29dbfff640 1 -- 172.21.15.27:0/279590635 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29e407dca0 con 0x562242bb2a80 2024-09-12T03:20:07.437 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.432+0000 7f29dbfff640 1 -- 172.21.15.27:0/279590635 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29e40824d0 con 0x562242bb2a80 2024-09-12T03:20:07.437 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.432+0000 7f29dbfff640 1 -- 172.21.15.27:0/279590635 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29e4030d10 con 0x562242bb2a80 2024-09-12T03:20:07.437 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.433+0000 7f2a03bcb740 1 -- 172.21.15.27:0/279590635 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562242bb2a80 msgr2=0x562242bb56a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:07.437 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.433+0000 7f2a03bcb740 1 --2- 172.21.15.27:0/279590635 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562242bb2a80 0x562242bb56a0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f29e4075e00 tx=0x7f29e4079560 comp rx=0 tx=0).stop 2024-09-12T03:20:07.437 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.433+0000 7f2a03bcb740 1 -- 172.21.15.27:0/279590635 shutdown_connections 2024-09-12T03:20:07.437 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.433+0000 7f2a03bcb740 1 --2- 172.21.15.27:0/279590635 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x562242bb6a50 0x562242baf8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:07.438 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.433+0000 7f2a03bcb740 1 --2- 172.21.15.27:0/279590635 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562242bb2a80 0x562242bb56a0 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:07.438 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.433+0000 7f2a03bcb740 1 --2- 172.21.15.27:0/279590635 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562242bb52c0 0x562242bfaa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:07.438 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.433+0000 7f2a03bcb740 1 -- 172.21.15.27:0/279590635 >> 172.21.15.27:0/279590635 conn(0x562242bb1a80 msgr2=0x562242bf9f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:07.438 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.433+0000 7f2a03bcb740 1 -- 172.21.15.27:0/279590635 shutdown_connections 2024-09-12T03:20:07.438 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.433+0000 7f2a03bcb740 1 -- 172.21.15.27:0/279590635 wait complete. 2024-09-12T03:20:07.438 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.433+0000 7f2a03bcb740 1 -- start start 2024-09-12T03:20:07.438 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.433+0000 7f2a03bcb740 1 Processor -- start 2024-09-12T03:20:07.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.434+0000 7f2a03bcb740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562242bb52c0 0x562242bf6810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:07.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.434+0000 7f2a03bcb740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562242bb6a50 0x562242bf4d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:07.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.434+0000 7f2a03bcb740 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x562242bd8290 0x562242baf820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:07.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.434+0000 7f2a03bcb740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x562242b17520 con 0x562242bb52c0 2024-09-12T03:20:07.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.434+0000 7f2a03bcb740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x562242aa0450 con 0x562242bb6a50 2024-09-12T03:20:07.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.434+0000 7f2a03bcb740 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x562242bb0f20 con 0x562242bd8290 2024-09-12T03:20:07.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.434+0000 7f2a03bcb740 10 client.0 _ll_register_callbacks cb 0x562242829ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:20:07.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.434+0000 7f29fa59c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562242bb52c0 0x562242bf6810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:07.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.434+0000 7f29fa59c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562242bb52c0 0x562242bf6810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56310/0 (socket says 172.21.15.27:56310) 2024-09-12T03:20:07.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.435+0000 7f29fa59c640 1 -- 172.21.15.27:0/2768729332 learned_addr learned my addr 172.21.15.27:0/2768729332 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:20:07.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.435+0000 7f29f3fff640 1 --2- 172.21.15.27:0/2768729332 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562242bb6a50 0x562242bf4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:07.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.435+0000 7f29fad9d640 1 --2- 172.21.15.27:0/2768729332 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x562242bd8290 0x562242baf820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:07.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.435+0000 7f29fa59c640 1 -- 172.21.15.27:0/2768729332 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x562242bd8290 msgr2=0x562242baf820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:07.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.435+0000 7f29fa59c640 1 --2- 172.21.15.27:0/2768729332 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x562242bd8290 0x562242baf820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:07.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.435+0000 7f29fa59c640 1 -- 172.21.15.27:0/2768729332 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562242bb6a50 msgr2=0x562242bf4d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:07.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.435+0000 7f29fa59c640 1 --2- 172.21.15.27:0/2768729332 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562242bb6a50 0x562242bf4d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:07.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.435+0000 7f29fa59c640 1 -- 172.21.15.27:0/2768729332 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56224282a2d0 con 0x562242bb52c0 2024-09-12T03:20:07.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.435+0000 7f29fad9d640 1 --2- 172.21.15.27:0/2768729332 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x562242bd8290 0x562242baf820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:20:07.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.435+0000 7f29fa59c640 1 --2- 172.21.15.27:0/2768729332 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562242bb52c0 0x562242bf6810 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f29ec003640 tx=0x7f29ec0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79ae08b23ac3a028 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:07.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.435+0000 7f29d97fa640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-12T03:20:07.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.435+0000 7f29d97fa640 1 -- 172.21.15.27:0/2768729332 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29ec027070 con 0x562242bb52c0 2024-09-12T03:20:07.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.435+0000 7f29d97fa640 1 -- 172.21.15.27:0/2768729332 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(67..67 src has 1..67) ==== 5709+0+0 (secure 0 0 0) 0x7f29ec014440 con 0x562242bb52c0 2024-09-12T03:20:07.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.435+0000 7f2a03bcb740 1 -- 172.21.15.27:0/2768729332 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f29e40824d0 con 0x562242bb52c0 2024-09-12T03:20:07.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.435+0000 7f29d97fa640 1 -- 172.21.15.27:0/2768729332 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f29ec032050 con 0x562242bb52c0 2024-09-12T03:20:07.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.435+0000 7f2a03bcb740 10 client.6216 fetch_fsmap learned FSMap version 38 2024-09-12T03:20:07.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.435+0000 7f2a03bcb740 1 -- 172.21.15.27:0/2768729332 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=2+}) -- 0x5622428f46b0 con 0x562242bb52c0 2024-09-12T03:20:07.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.435+0000 7f29d97fa640 1 -- 172.21.15.27:0/2768729332 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29e40824d0 con 0x562242bb52c0 2024-09-12T03:20:07.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.436+0000 7f29d97fa640 1 -- 172.21.15.27:0/2768729332 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 38) ==== 3502+0+0 (secure 0 0 0) 0x7f29ec035020 con 0x562242bb52c0 2024-09-12T03:20:07.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.436+0000 7f2a03bcb740 10 client.6216 fetch_fsmap finished waiting for FSMap version 38 2024-09-12T03:20:07.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.436+0000 7f2a03bcb740 10 client.6216 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-12T03:20:07.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.436+0000 7f2a03bcb740 20 client.6216 populate_metadata read hostname 'smithi027' 2024-09-12T03:20:07.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.436+0000 7f2a03bcb740 1 --2- 172.21.15.27:0/2768729332 >> [v2:172.21.15.45:6835/2890007436,v1:172.21.15.45:6837/2890007436] conn(0x562242bc9240 0x562242bc7680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:07.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.436+0000 7f2a03bcb740 4 client.6216 mds_command: new command op to 6066 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:20:07.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.436+0000 7f2a03bcb740 1 -- 172.21.15.27:0/2768729332 --> [v2:172.21.15.45:6835/2890007436,v1:172.21.15.45:6837/2890007436] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562242b24c50 con 0x562242bc9240 2024-09-12T03:20:07.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.436+0000 7f29f3fff640 1 --2- 172.21.15.27:0/2768729332 >> [v2:172.21.15.45:6835/2890007436,v1:172.21.15.45:6837/2890007436] conn(0x562242bc9240 0x562242bc7680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:07.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.436+0000 7f29f3fff640 1 --2- 172.21.15.27:0/2768729332 >> [v2:172.21.15.45:6835/2890007436,v1:172.21.15.45:6837/2890007436] conn(0x562242bc9240 0x562242bc7680 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ed44cbefca42473e server_cookie=4e4e50d8b606513 in_seq=0 out_seq=0 2024-09-12T03:20:07.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.436+0000 7f29d97fa640 10 client.6216 ms_handle_connect on v2:172.21.15.45:6835/2890007436 2024-09-12T03:20:07.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.437+0000 7f29d97fa640 1 -- 172.21.15.27:0/2768729332 <== mds.0 v2:172.21.15.45:6835/2890007436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562242b24c50 con 0x562242bc9240 2024-09-12T03:20:07.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.437+0000 7f29d97fa640 10 client.6216 handle_command_reply: tid=0 2024-09-12T03:20:07.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.437+0000 7f2a03bcb740 1 client.6216 shutdown 2024-09-12T03:20:07.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.437+0000 7f2a03bcb740 1 -- 172.21.15.27:0/2768729332 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562242bb52c0 msgr2=0x562242bf6810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:07.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.437+0000 7f2a03bcb740 1 --2- 172.21.15.27:0/2768729332 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562242bb52c0 0x562242bf6810 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f29ec003640 tx=0x7f29ec0046c0 comp rx=0 tx=0).stop 2024-09-12T03:20:07.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.437+0000 7f2a03bcb740 1 -- 172.21.15.27:0/2768729332 shutdown_connections 2024-09-12T03:20:07.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.437+0000 7f2a03bcb740 1 --2- 172.21.15.27:0/2768729332 >> [v2:172.21.15.45:6835/2890007436,v1:172.21.15.45:6837/2890007436] conn(0x562242bc9240 0x562242bc7680 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:07.446 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.437+0000 7f2a03bcb740 1 --2- 172.21.15.27:0/2768729332 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x562242bd8290 0x562242baf820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:07.446 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.437+0000 7f2a03bcb740 1 --2- 172.21.15.27:0/2768729332 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562242bb6a50 0x562242bf4d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:07.446 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.437+0000 7f2a03bcb740 1 --2- 172.21.15.27:0/2768729332 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562242bb52c0 0x562242bf6810 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:07.446 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.437+0000 7f29d97fa640 0 client.6216 ms_handle_reset on v2:172.21.15.45:6835/2890007436 2024-09-12T03:20:07.446 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.437+0000 7f2a03bcb740 1 -- 172.21.15.27:0/2768729332 >> 172.21.15.27:0/2768729332 conn(0x562242bb2a80 msgr2=0x562242bf7fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:07.446 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.437+0000 7f2a03bcb740 1 -- 172.21.15.27:0/2768729332 shutdown_connections 2024-09-12T03:20:07.446 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.437+0000 7f2a03bcb740 1 -- 172.21.15.27:0/2768729332 wait complete. 2024-09-12T03:20:07.446 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.437+0000 7f2a03bcb740 20 client.6216 trim_cache size 0 max 16384 2024-09-12T03:20:07.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.456+0000 7f2a03bcb740 1 Processor -- start 2024-09-12T03:20:07.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.456+0000 7f2a03bcb740 1 -- start start 2024-09-12T03:20:07.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.456+0000 7f2a03bcb740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562242bb52c0 0x562242c1dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:07.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.456+0000 7f2a03bcb740 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x562242bb6a50 0x562242bfd630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:07.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.456+0000 7f2a03bcb740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562242bd8290 0x562242bdac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:07.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.456+0000 7f2a03bcb740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f29fc1ffd40 con 0x562242bd8290 2024-09-12T03:20:07.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.456+0000 7f2a03bcb740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f29fc1ff0d0 con 0x562242bb52c0 2024-09-12T03:20:07.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.456+0000 7f2a03bcb740 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f29fc1ff6f0 con 0x562242bb6a50 2024-09-12T03:20:07.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.456+0000 7f29fad9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562242bd8290 0x562242bdac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:07.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.456+0000 7f29fad9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562242bd8290 0x562242bdac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56318/0 (socket says 172.21.15.27:56318) 2024-09-12T03:20:07.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.456+0000 7f29fad9d640 1 -- 172.21.15.27:0/3098963190 learned_addr learned my addr 172.21.15.27:0/3098963190 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:20:07.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.457+0000 7f29fa59c640 1 --2- 172.21.15.27:0/3098963190 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562242bb52c0 0x562242c1dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:07.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.457+0000 7f29fad9d640 1 -- 172.21.15.27:0/3098963190 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x562242bb6a50 msgr2=0x562242bfd630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:07.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.457+0000 7f29fad9d640 1 --2- 172.21.15.27:0/3098963190 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x562242bb6a50 0x562242bfd630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:07.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.457+0000 7f29fad9d640 1 -- 172.21.15.27:0/3098963190 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562242bb52c0 msgr2=0x562242c1dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:07.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.457+0000 7f29fad9d640 1 --2- 172.21.15.27:0/3098963190 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562242bb52c0 0x562242c1dd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:07.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.457+0000 7f29fad9d640 1 -- 172.21.15.27:0/3098963190 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562242bab6c0 con 0x562242bd8290 2024-09-12T03:20:07.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.457+0000 7f29fad9d640 1 --2- 172.21.15.27:0/3098963190 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562242bd8290 0x562242bdac10 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f29f4003f70 tx=0x7f29f4001670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16427d5ef56032f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:07.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.457+0000 7f29db7fe640 1 -- 172.21.15.27:0/3098963190 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29f401a070 con 0x562242bd8290 2024-09-12T03:20:07.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.457+0000 7f29db7fe640 1 -- 172.21.15.27:0/3098963190 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29f4002080 con 0x562242bd8290 2024-09-12T03:20:07.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.457+0000 7f29db7fe640 1 -- 172.21.15.27:0/3098963190 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29f4016620 con 0x562242bd8290 2024-09-12T03:20:07.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.458+0000 7f2a03bcb740 1 -- 172.21.15.27:0/3098963190 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562242bd8290 msgr2=0x562242bdac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:07.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.458+0000 7f2a03bcb740 1 --2- 172.21.15.27:0/3098963190 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562242bd8290 0x562242bdac10 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f29f4003f70 tx=0x7f29f4001670 comp rx=0 tx=0).stop 2024-09-12T03:20:07.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.458+0000 7f2a03bcb740 1 -- 172.21.15.27:0/3098963190 shutdown_connections 2024-09-12T03:20:07.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.458+0000 7f2a03bcb740 1 --2- 172.21.15.27:0/3098963190 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562242bd8290 0x562242bdac10 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:07.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.458+0000 7f2a03bcb740 1 --2- 172.21.15.27:0/3098963190 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x562242bb6a50 0x562242bfd630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:07.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.458+0000 7f2a03bcb740 1 --2- 172.21.15.27:0/3098963190 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562242bb52c0 0x562242c1dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:07.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.458+0000 7f2a03bcb740 1 -- 172.21.15.27:0/3098963190 >> 172.21.15.27:0/3098963190 conn(0x562242bb2a80 msgr2=0x562242bf7fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:07.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.458+0000 7f2a03bcb740 1 -- 172.21.15.27:0/3098963190 shutdown_connections 2024-09-12T03:20:07.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.458+0000 7f2a03bcb740 1 -- 172.21.15.27:0/3098963190 wait complete. 2024-09-12T03:20:07.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.458+0000 7f2a03bcb740 1 -- start start 2024-09-12T03:20:07.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.458+0000 7f2a03bcb740 1 Processor -- start 2024-09-12T03:20:07.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.459+0000 7f2a03bcb740 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x562242bb52c0 0x562242bfa090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:07.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.459+0000 7f2a03bcb740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562242bb6a50 0x562242c5dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:07.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.459+0000 7f2a03bcb740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562242bd8290 0x562242bf7fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:07.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.459+0000 7f2a03bcb740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x562242bc86e0 con 0x562242bd8290 2024-09-12T03:20:07.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.459+0000 7f2a03bcb740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x562242bf7600 con 0x562242bb6a50 2024-09-12T03:20:07.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.459+0000 7f2a03bcb740 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x562242bc96c0 con 0x562242bb52c0 2024-09-12T03:20:07.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.459+0000 7f2a03bcb740 10 client.0 _ll_register_callbacks cb 0x7f29ec004120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:20:07.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.459+0000 7f29fad9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562242bd8290 0x562242bf7fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:07.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.459+0000 7f29fad9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562242bd8290 0x562242bf7fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56320/0 (socket says 172.21.15.27:56320) 2024-09-12T03:20:07.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.459+0000 7f29fad9d640 1 -- 172.21.15.27:0/2971434494 learned_addr learned my addr 172.21.15.27:0/2971434494 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:20:07.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.459+0000 7f29f3fff640 1 --2- 172.21.15.27:0/2971434494 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562242bb6a50 0x562242c5dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:07.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.459+0000 7f29fa59c640 1 --2- 172.21.15.27:0/2971434494 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x562242bb52c0 0x562242bfa090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:07.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.459+0000 7f29fad9d640 1 -- 172.21.15.27:0/2971434494 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x562242bb52c0 msgr2=0x562242bfa090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:07.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.460+0000 7f29fad9d640 1 --2- 172.21.15.27:0/2971434494 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x562242bb52c0 0x562242bfa090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:07.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.460+0000 7f29fad9d640 1 -- 172.21.15.27:0/2971434494 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562242bb6a50 msgr2=0x562242c5dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:07.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.460+0000 7f29fad9d640 1 --2- 172.21.15.27:0/2971434494 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562242bb6a50 0x562242c5dfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:07.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.460+0000 7f29fad9d640 1 -- 172.21.15.27:0/2971434494 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562242bdab50 con 0x562242bd8290 2024-09-12T03:20:07.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.460+0000 7f29fa59c640 1 --2- 172.21.15.27:0/2971434494 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x562242bb52c0 0x562242bfa090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:20:07.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.460+0000 7f29fad9d640 1 --2- 172.21.15.27:0/2971434494 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562242bd8290 0x562242bf7fb0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f29f4009010 tx=0x7f29f4001250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6eac12900b699ab server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:07.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.460+0000 7f29d97fa640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-12T03:20:07.468 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.460+0000 7f29d97fa640 1 -- 172.21.15.27:0/2971434494 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29f402e060 con 0x562242bd8290 2024-09-12T03:20:07.468 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.460+0000 7f29d97fa640 1 -- 172.21.15.27:0/2971434494 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(67..67 src has 1..67) ==== 5709+0+0 (secure 0 0 0) 0x7f29f4016620 con 0x562242bd8290 2024-09-12T03:20:07.468 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.460+0000 7f2a03bcb740 1 -- 172.21.15.27:0/2971434494 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f29f4002080 con 0x562242bd8290 2024-09-12T03:20:07.468 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.460+0000 7f29d97fa640 1 -- 172.21.15.27:0/2971434494 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f29f4034050 con 0x562242bd8290 2024-09-12T03:20:07.468 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.460+0000 7f2a03bcb740 10 client.6222 fetch_fsmap learned FSMap version 38 2024-09-12T03:20:07.468 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.460+0000 7f2a03bcb740 1 -- 172.21.15.27:0/2971434494 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=2+}) -- 0x562242bdbbd0 con 0x562242bd8290 2024-09-12T03:20:07.468 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.460+0000 7f29d97fa640 1 -- 172.21.15.27:0/2971434494 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29f4002080 con 0x562242bd8290 2024-09-12T03:20:07.468 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.460+0000 7f29d97fa640 1 -- 172.21.15.27:0/2971434494 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 38) ==== 3502+0+0 (secure 0 0 0) 0x7f29f4016820 con 0x562242bd8290 2024-09-12T03:20:07.468 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.460+0000 7f2a03bcb740 10 client.6222 fetch_fsmap finished waiting for FSMap version 38 2024-09-12T03:20:07.468 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.460+0000 7f2a03bcb740 10 client.6222 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-12T03:20:07.468 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.460+0000 7f2a03bcb740 20 client.6222 populate_metadata read hostname 'smithi027' 2024-09-12T03:20:07.468 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.460+0000 7f2a03bcb740 1 --2- 172.21.15.27:0/2971434494 >> [v2:172.21.15.45:6835/2890007436,v1:172.21.15.45:6837/2890007436] conn(0x562242c1b890 0x562242bdecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:07.468 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.460+0000 7f2a03bcb740 4 client.6222 mds_command: new command op to 6066 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-12T03:20:07.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.460+0000 7f2a03bcb740 1 -- 172.21.15.27:0/2971434494 --> [v2:172.21.15.45:6835/2890007436,v1:172.21.15.45:6837/2890007436] -- command(tid 0: {"prefix": "status"}) -- 0x562242b1efa0 con 0x562242c1b890 2024-09-12T03:20:07.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.461+0000 7f29fa59c640 1 --2- 172.21.15.27:0/2971434494 >> [v2:172.21.15.45:6835/2890007436,v1:172.21.15.45:6837/2890007436] conn(0x562242c1b890 0x562242bdecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:07.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.461+0000 7f29d97fa640 10 client.6222 ms_handle_connect on v2:172.21.15.45:6835/2890007436 2024-09-12T03:20:07.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.461+0000 7f29fa59c640 1 --2- 172.21.15.27:0/2971434494 >> [v2:172.21.15.45:6835/2890007436,v1:172.21.15.45:6837/2890007436] conn(0x562242c1b890 0x562242bdecb0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=87b31f1162af464c server_cookie=70a04280905cb66a in_seq=0 out_seq=0 2024-09-12T03:20:07.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.461+0000 7f29d97fa640 1 -- 172.21.15.27:0/2971434494 <== mds.0 v2:172.21.15.45:6835/2890007436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x562242b1efa0 con 0x562242c1b890 2024-09-12T03:20:07.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.461+0000 7f29d97fa640 10 client.6222 handle_command_reply: tid=0 2024-09-12T03:20:07.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.461+0000 7f2a03bcb740 1 client.6222 shutdown 2024-09-12T03:20:07.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.461+0000 7f2a03bcb740 1 -- 172.21.15.27:0/2971434494 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562242bd8290 msgr2=0x562242bf7fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:07.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.461+0000 7f2a03bcb740 1 --2- 172.21.15.27:0/2971434494 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562242bd8290 0x562242bf7fb0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f29f4009010 tx=0x7f29f4001250 comp rx=0 tx=0).stop 2024-09-12T03:20:07.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.461+0000 7f2a03bcb740 1 -- 172.21.15.27:0/2971434494 shutdown_connections 2024-09-12T03:20:07.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.461+0000 7f2a03bcb740 1 --2- 172.21.15.27:0/2971434494 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562242bd8290 0x562242bf7fb0 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:07.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.461+0000 7f2a03bcb740 1 --2- 172.21.15.27:0/2971434494 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562242bb6a50 0x562242c5dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:07.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.461+0000 7f2a03bcb740 1 --2- 172.21.15.27:0/2971434494 >> [v2:172.21.15.45:6835/2890007436,v1:172.21.15.45:6837/2890007436] conn(0x562242c1b890 0x562242bdecb0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:07.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.461+0000 7f2a03bcb740 1 --2- 172.21.15.27:0/2971434494 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x562242bb52c0 0x562242bfa090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:07.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.461+0000 7f29d97fa640 0 client.6222 ms_handle_reset on v2:172.21.15.45:6835/2890007436 2024-09-12T03:20:07.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.461+0000 7f2a03bcb740 1 -- 172.21.15.27:0/2971434494 >> 172.21.15.27:0/2971434494 conn(0x562242bb2a80 msgr2=0x562242baaef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:07.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.461+0000 7f2a03bcb740 1 -- 172.21.15.27:0/2971434494 shutdown_connections 2024-09-12T03:20:07.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.462+0000 7f2a03bcb740 1 -- 172.21.15.27:0/2971434494 wait complete. 2024-09-12T03:20:07.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.462+0000 7f2a03bcb740 20 client.6222 trim_cache size 0 max 16384 2024-09-12T03:20:07.470 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-12T03:20:07.470 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_fsid": "d2df504e-2647-4385-a5db-9fdf69f487a0", 2024-09-12T03:20:07.470 INFO:teuthology.orchestra.run.smithi027.stdout: "whoami": 0, 2024-09-12T03:20:07.470 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 6066, 2024-09-12T03:20:07.470 INFO:teuthology.orchestra.run.smithi027.stdout: "want_state": "up:active", 2024-09-12T03:20:07.471 INFO:teuthology.orchestra.run.smithi027.stdout: "state": "up:active", 2024-09-12T03:20:07.471 INFO:teuthology.orchestra.run.smithi027.stdout: "fs_name": "cephfs", 2024-09-12T03:20:07.471 INFO:teuthology.orchestra.run.smithi027.stdout: "rank_uptime": 9.0703092719999994, 2024-09-12T03:20:07.471 INFO:teuthology.orchestra.run.smithi027.stdout: "mdsmap_epoch": 38, 2024-09-12T03:20:07.471 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch": 67, 2024-09-12T03:20:07.471 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch_barrier": 67, 2024-09-12T03:20:07.471 INFO:teuthology.orchestra.run.smithi027.stdout: "uptime": 14.089777628 2024-09-12T03:20:07.471 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-12T03:20:07.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.462+0000 7f2a00d52640 1 -- 172.21.15.27:0/3981226465 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f29dc06c190 msgr2=0x7f29dc06e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:07.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.462+0000 7f2a00d52640 1 --2- 172.21.15.27:0/3981226465 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f29dc06c190 0x7f29dc06e650 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f29ec009aa0 tx=0x7f29ec009190 comp rx=0 tx=0).stop 2024-09-12T03:20:07.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.462+0000 7f2a00d52640 1 -- 172.21.15.27:0/3981226465 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f29fc0d52e0 msgr2=0x7f29fc0c82a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:07.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.462+0000 7f2a00d52640 1 --2- 172.21.15.27:0/3981226465 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f29fc0d52e0 0x7f29fc0c82a0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f29e4000c00 tx=0x7f29e40046c0 comp rx=0 tx=0).stop 2024-09-12T03:20:07.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.462+0000 7f2a00d52640 1 -- 172.21.15.27:0/3981226465 shutdown_connections 2024-09-12T03:20:07.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.462+0000 7f2a00d52640 1 --2- 172.21.15.27:0/3981226465 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f29dc06c190 0x7f29dc06e650 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:07.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.462+0000 7f2a00d52640 1 --2- 172.21.15.27:0/3981226465 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f29fc162520 0x7f29fc0c87e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:07.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.462+0000 7f2a00d52640 1 --2- 172.21.15.27:0/3981226465 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f29fc0d52e0 0x7f29fc0c82a0 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:07.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.462+0000 7f2a00d52640 1 --2- 172.21.15.27:0/3981226465 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f29fc0d1050 0x7f29fc0ca170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:07.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.462+0000 7f2a00d52640 1 -- 172.21.15.27:0/3981226465 >> 172.21.15.27:0/3981226465 conn(0x7f29fc06ddf0 msgr2=0x7f29fc0b9280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:07.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.462+0000 7f2a00d52640 1 -- 172.21.15.27:0/3981226465 shutdown_connections 2024-09-12T03:20:07.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.462+0000 7f2a00d52640 1 -- 172.21.15.27:0/3981226465 wait complete. 2024-09-12T03:20:07.474 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-12T03:20:07.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.575+0000 7fe4b9c65640 1 Processor -- start 2024-09-12T03:20:07.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.575+0000 7fe4b9c65640 1 -- start start 2024-09-12T03:20:07.577 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.576+0000 7fe4b9c65640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe4b40b1ca0 0x7fe4b415f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:07.577 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.576+0000 7fe4b9c65640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fe4b40cf660 0x7fe4b40d2780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:07.577 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.576+0000 7fe4b9c65640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe4b415fd70 con 0x7fe4b406f360 2024-09-12T03:20:07.577 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.576+0000 7fe4b9c65640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fe4b40d22f0 con 0x7fe4b40cf660 2024-09-12T03:20:07.577 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.576+0000 7fe4b9c65640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fe4b406cc40 con 0x7fe4b40b1ca0 2024-09-12T03:20:07.577 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.576+0000 7fe4b3fff640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fe4b40cf660 0x7fe4b40d2780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:07.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.576+0000 7fe4b37fe640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe4b40b1ca0 0x7fe4b415f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:07.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.576+0000 7fe4b3fff640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fe4b40cf660 0x7fe4b40d2780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.27:48738/0 (socket says 172.21.15.27:48738) 2024-09-12T03:20:07.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.576+0000 7fe4b3fff640 1 -- 172.21.15.27:0/401651822 learned_addr learned my addr 172.21.15.27:0/401651822 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:20:07.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.576+0000 7fe4b2ffd640 1 --1- 172.21.15.27:0/401651822 >> v1:172.21.15.45:6789/0 conn(0x7fe4b406f360 0x7fe4b40cef50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.27:41212/0 (socket says 172.21.15.27:41212) 2024-09-12T03:20:07.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.577+0000 7fe4b3fff640 1 -- 172.21.15.27:0/401651822 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe4b40b1ca0 msgr2=0x7fe4b415f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:07.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.577+0000 7fe4b3fff640 1 --2- 172.21.15.27:0/401651822 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe4b40b1ca0 0x7fe4b415f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:07.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.577+0000 7fe4b3fff640 1 -- 172.21.15.27:0/401651822 >> v1:172.21.15.45:6789/0 conn(0x7fe4b406f360 legacy=0x7fe4b40cef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:07.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.577+0000 7fe4b3fff640 1 -- 172.21.15.27:0/401651822 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4b40d1ce0 con 0x7fe4b40cf660 2024-09-12T03:20:07.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.577+0000 7fe4b27fc640 1 -- 172.21.15.27:0/401651822 <== mon.1 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2686908939 0 0) 0x7fe4a8001110 con 0x7fe4b406f360 2024-09-12T03:20:07.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.577+0000 7fe4b37fe640 1 --2- 172.21.15.27:0/401651822 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe4b40b1ca0 0x7fe4b415f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:20:07.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.578+0000 7fe4b3fff640 1 --2- 172.21.15.27:0/401651822 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe4b40cf660 0x7fe4b40d2780 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fe4ac009e80 tx=0x7fe4ac02a750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=997d3e7cf530701e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:07.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.578+0000 7fe4b27fc640 1 -- 172.21.15.27:0/401651822 <== mon.1 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4ac036070 con 0x7fe4b40cf660 2024-09-12T03:20:07.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.578+0000 7fe4b27fc640 1 -- 172.21.15.27:0/401651822 <== mon.1 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4ac031600 con 0x7fe4b40cf660 2024-09-12T03:20:07.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.578+0000 7fe4b27fc640 1 -- 172.21.15.27:0/401651822 <== mon.1 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4ac0319d0 con 0x7fe4b40cf660 2024-09-12T03:20:07.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.578+0000 7fe4b9c65640 1 -- 172.21.15.27:0/401651822 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe4b40cf660 msgr2=0x7fe4b40d2780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:07.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.578+0000 7fe4b9c65640 1 --2- 172.21.15.27:0/401651822 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe4b40cf660 0x7fe4b40d2780 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fe4ac009e80 tx=0x7fe4ac02a750 comp rx=0 tx=0).stop 2024-09-12T03:20:07.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.578+0000 7fe4b9c65640 1 -- 172.21.15.27:0/401651822 shutdown_connections 2024-09-12T03:20:07.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.578+0000 7fe4b9c65640 1 --2- 172.21.15.27:0/401651822 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe4b40cf660 0x7fe4b40d2780 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:07.581 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.578+0000 7fe4b9c65640 1 --2- 172.21.15.27:0/401651822 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe4b40b1ca0 0x7fe4b415f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:07.581 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.578+0000 7fe4b9c65640 1 -- 172.21.15.27:0/401651822 >> 172.21.15.27:0/401651822 conn(0x7fe4b4073800 msgr2=0x7fe4b409ba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:07.581 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.578+0000 7fe4b9c65640 1 -- 172.21.15.27:0/401651822 shutdown_connections 2024-09-12T03:20:07.581 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.578+0000 7fe4b9c65640 1 -- 172.21.15.27:0/401651822 wait complete. 2024-09-12T03:20:07.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.579+0000 7fe4b9c65640 1 Processor -- start 2024-09-12T03:20:07.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.580+0000 7fe4b9c65640 1 -- start start 2024-09-12T03:20:07.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.580+0000 7fe4b9c65640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe4b406f360 0x7fe4b41f4aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:07.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.580+0000 7fe4b9c65640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe4b40b1ca0 0x7fe4b40ce050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:07.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.580+0000 7fe4b9c65640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe4b40cf660 0x7fe4b41f6a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:07.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.580+0000 7fe4b9c65640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fe4b406d1c0 con 0x7fe4b406f360 2024-09-12T03:20:07.584 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.580+0000 7fe4b9c65640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe4b40d1f80 con 0x7fe4b40cf660 2024-09-12T03:20:07.584 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.580+0000 7fe4b9c65640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fe4b406d340 con 0x7fe4b40b1ca0 2024-09-12T03:20:07.584 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.581+0000 7fe4b37fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe4b406f360 0x7fe4b41f4aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:07.584 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.581+0000 7fe4b37fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe4b406f360 0x7fe4b41f4aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56332/0 (socket says 172.21.15.27:56332) 2024-09-12T03:20:07.584 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.581+0000 7fe4b37fe640 1 -- 172.21.15.27:0/3424174549 learned_addr learned my addr 172.21.15.27:0/3424174549 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:20:07.584 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.581+0000 7fe4b2ffd640 1 --2- 172.21.15.27:0/3424174549 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe4b40b1ca0 0x7fe4b40ce050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:07.584 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.581+0000 7fe4b3fff640 1 --2- 172.21.15.27:0/3424174549 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe4b40cf660 0x7fe4b41f6a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:07.584 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.581+0000 7fe4b37fe640 1 -- 172.21.15.27:0/3424174549 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe4b40b1ca0 msgr2=0x7fe4b40ce050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:07.584 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.581+0000 7fe4b37fe640 1 --2- 172.21.15.27:0/3424174549 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe4b40b1ca0 0x7fe4b40ce050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:07.584 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.581+0000 7fe4b37fe640 1 -- 172.21.15.27:0/3424174549 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe4b40cf660 msgr2=0x7fe4b41f6a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:07.584 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.581+0000 7fe4b37fe640 1 --2- 172.21.15.27:0/3424174549 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe4b40cf660 0x7fe4b41f6a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:07.584 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.581+0000 7fe4b37fe640 1 -- 172.21.15.27:0/3424174549 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4b41f4fe0 con 0x7fe4b406f360 2024-09-12T03:20:07.585 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.581+0000 7fe4b37fe640 1 --2- 172.21.15.27:0/3424174549 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe4b406f360 0x7fe4b41f4aa0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fe4a400ec10 tx=0x7fe4a400c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:07.585 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.581+0000 7fe49bfff640 1 -- 172.21.15.27:0/3424174549 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4a4017070 con 0x7fe4b406f360 2024-09-12T03:20:07.585 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.581+0000 7fe49bfff640 1 -- 172.21.15.27:0/3424174549 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4a400cc30 con 0x7fe4b406f360 2024-09-12T03:20:07.585 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.581+0000 7fe49bfff640 1 -- 172.21.15.27:0/3424174549 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4a4013470 con 0x7fe4b406f360 2024-09-12T03:20:07.585 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.581+0000 7fe4b9c65640 1 -- 172.21.15.27:0/3424174549 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4b40cb850 con 0x7fe4b406f360 2024-09-12T03:20:07.585 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.582+0000 7fe4b9c65640 1 -- 172.21.15.27:0/3424174549 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4b40cbdd0 con 0x7fe4b406f360 2024-09-12T03:20:07.586 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.583+0000 7fe4b9c65640 1 -- 172.21.15.27:0/3424174549 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4b40cfc40 con 0x7fe4b406f360 2024-09-12T03:20:07.586 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.583+0000 7fe49bfff640 1 -- 172.21.15.27:0/3424174549 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 11) ==== 90804+0+0 (secure 0 0 0) 0x7fe4a401a070 con 0x7fe4b406f360 2024-09-12T03:20:07.586 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.583+0000 7fe49bfff640 1 --2- 172.21.15.27:0/3424174549 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fe4a006c190 0x7fe4a006e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:07.586 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.583+0000 7fe49bfff640 1 -- 172.21.15.27:0/3424174549 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 5709+0+0 (secure 0 0 0) 0x7fe4a408b2f0 con 0x7fe4b406f360 2024-09-12T03:20:07.586 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.584+0000 7fe4b2ffd640 1 --2- 172.21.15.27:0/3424174549 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fe4a006c190 0x7fe4a006e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:07.586 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.584+0000 7fe4b2ffd640 1 --2- 172.21.15.27:0/3424174549 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fe4a006c190 0x7fe4a006e650 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fe4b40a0a10 tx=0x7fe4a8007b50 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:07.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.588+0000 7fe49bfff640 1 -- 172.21.15.27:0/3424174549 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fe4b41f4fe0 con 0x7fe4b406f360 2024-09-12T03:20:07.801 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.801+0000 7fe4b9c65640 1 -- 172.21.15.27:0/3424174549 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fe4b4069ff0 con 0x7fe4b406f360 2024-09-12T03:20:07.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.801+0000 7fe49bfff640 1 -- 172.21.15.27:0/3424174549 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v38) ==== 53+0+73 (secure 0 0 0) 0x7fe4a4059240 con 0x7fe4b406f360 2024-09-12T03:20:07.802 INFO:teuthology.orchestra.run.smithi027.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-12T03:20:07.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.803+0000 7fe4b9c65640 1 -- 172.21.15.27:0/3424174549 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fe4a006c190 msgr2=0x7fe4a006e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:07.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.803+0000 7fe4b9c65640 1 --2- 172.21.15.27:0/3424174549 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fe4a006c190 0x7fe4a006e650 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fe4b40a0a10 tx=0x7fe4a8007b50 comp rx=0 tx=0).stop 2024-09-12T03:20:07.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.803+0000 7fe4b9c65640 1 -- 172.21.15.27:0/3424174549 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe4b406f360 msgr2=0x7fe4b41f4aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:07.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.803+0000 7fe4b9c65640 1 --2- 172.21.15.27:0/3424174549 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe4b406f360 0x7fe4b41f4aa0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fe4a400ec10 tx=0x7fe4a400c520 comp rx=0 tx=0).stop 2024-09-12T03:20:07.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.803+0000 7fe4b9c65640 1 -- 172.21.15.27:0/3424174549 shutdown_connections 2024-09-12T03:20:07.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.803+0000 7fe4b9c65640 1 --2- 172.21.15.27:0/3424174549 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fe4a006c190 0x7fe4a006e650 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:07.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.803+0000 7fe4b9c65640 1 --2- 172.21.15.27:0/3424174549 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe4b40cf660 0x7fe4b41f6a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:07.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.803+0000 7fe4b9c65640 1 --2- 172.21.15.27:0/3424174549 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe4b40b1ca0 0x7fe4b40ce050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:07.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.803+0000 7fe4b9c65640 1 --2- 172.21.15.27:0/3424174549 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe4b406f360 0x7fe4b41f4aa0 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:07.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.803+0000 7fe4b9c65640 1 -- 172.21.15.27:0/3424174549 >> 172.21.15.27:0/3424174549 conn(0x7fe4b4073800 msgr2=0x7fe4b40b2240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:07.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.803+0000 7fe4b9c65640 1 -- 172.21.15.27:0/3424174549 shutdown_connections 2024-09-12T03:20:07.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:07.803+0000 7fe4b9c65640 1 -- 172.21.15.27:0/3424174549 wait complete. 2024-09-12T03:20:07.815 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-12T03:20:07.815 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-12T03:20:07.816 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi027.front.sepia.ceph.com 2024-09-12T03:20:07.816 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-12T03:20:07.816 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-12T03:20:07.816 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-12T03:20:07.816 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-12T03:20:07.816 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-12T03:20:07.816 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-12T03:20:07.818 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:20:07.818 DEBUG:teuthology.orchestra.run.smithi027:> ip addr 2024-09-12T03:20:07.835 INFO:teuthology.orchestra.run.smithi027.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-12T03:20:07.835 INFO:teuthology.orchestra.run.smithi027.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-12T03:20:07.835 INFO:teuthology.orchestra.run.smithi027.stdout: inet 127.0.0.1/8 scope host lo 2024-09-12T03:20:07.835 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:20:07.835 INFO:teuthology.orchestra.run.smithi027.stdout: inet6 ::1/128 scope host 2024-09-12T03:20:07.835 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:20:07.835 INFO:teuthology.orchestra.run.smithi027.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-12T03:20:07.835 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether 0c:c4:7a:6b:fe:c8 brd ff:ff:ff:ff:ff:ff 2024-09-12T03:20:07.836 INFO:teuthology.orchestra.run.smithi027.stdout: altname enp7s0f0 2024-09-12T03:20:07.836 INFO:teuthology.orchestra.run.smithi027.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-12T03:20:07.836 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether 0c:c4:7a:6b:fe:c9 brd ff:ff:ff:ff:ff:ff 2024-09-12T03:20:07.836 INFO:teuthology.orchestra.run.smithi027.stdout: altname enp7s0f1 2024-09-12T03:20:07.836 INFO:teuthology.orchestra.run.smithi027.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-12T03:20:07.836 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether ac:1f:6b:f8:39:5a brd ff:ff:ff:ff:ff:ff 2024-09-12T03:20:07.836 INFO:teuthology.orchestra.run.smithi027.stdout: altname enp1s0f0 2024-09-12T03:20:07.836 INFO:teuthology.orchestra.run.smithi027.stdout: inet 172.21.15.27/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-12T03:20:07.836 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft 42581sec preferred_lft 42581sec 2024-09-12T03:20:07.836 INFO:teuthology.orchestra.run.smithi027.stdout: inet6 fe80::ae1f:6bff:fef8:395a/64 scope link 2024-09-12T03:20:07.836 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:20:07.836 INFO:teuthology.orchestra.run.smithi027.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-12T03:20:07.836 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether ac:1f:6b:f8:39:5b brd ff:ff:ff:ff:ff:ff 2024-09-12T03:20:07.837 INFO:teuthology.orchestra.run.smithi027.stdout: altname enp1s0f1 2024-09-12T03:20:07.837 INFO:teuthology.orchestra.run.smithi027.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-12T03:20:07.837 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether 2e:7f:89:6c:33:a0 brd ff:ff:ff:ff:ff:ff 2024-09-12T03:20:07.837 INFO:teuthology.orchestra.run.smithi027.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-12T03:20:07.837 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:20:07.837 INFO:teuthology.orchestra.run.smithi027.stdout: inet6 fe80::c84b:19ff:fef0:ca74/64 scope link 2024-09-12T03:20:07.838 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:20:07.838 INFO:teuthology.orchestra.run.smithi027.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-12T03:20:07.838 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether 42:40:d7:5c:7a:8e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-12T03:20:07.838 INFO:teuthology.orchestra.run.smithi027.stdout: inet6 fe80::4040:d7ff:fe5c:7a8e/64 scope link 2024-09-12T03:20:07.838 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:20:07.838 INFO:teuthology.orchestra.run.smithi027.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-12T03:20:07.838 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether 2e:7f:89:6c:33:a0 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-12T03:20:07.838 INFO:teuthology.orchestra.run.smithi027.stdout: inet6 fe80::2c7f:89ff:fe6c:33a0/64 scope link 2024-09-12T03:20:07.838 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:20:07.839 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:20:07.839 DEBUG:teuthology.orchestra.run.smithi027:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-12T03:20:07.899 INFO:teuthology.orchestra.run.smithi027.stdout:1 2024-09-12T03:20:07.900 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:20:07.900 DEBUG:teuthology.orchestra.run.smithi027:> route 2024-09-12T03:20:07.958 INFO:teuthology.orchestra.run.smithi027.stdout:Kernel IP routing table 2024-09-12T03:20:07.958 INFO:teuthology.orchestra.run.smithi027.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-12T03:20:07.958 INFO:teuthology.orchestra.run.smithi027.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-12T03:20:07.958 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-12T03:20:07.958 INFO:teuthology.orchestra.run.smithi027.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-12T03:20:07.959 DEBUG:teuthology.orchestra.run.smithi027:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-12T03:20:07.959 DEBUG:teuthology.orchestra.run.smithi027:> set -e 2024-09-12T03:20:07.959 DEBUG:teuthology.orchestra.run.smithi027:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-12T03:20:07.959 DEBUG:teuthology.orchestra.run.smithi027:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-12T03:20:07.959 DEBUG:teuthology.orchestra.run.smithi027:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-12T03:20:07.959 DEBUG:teuthology.orchestra.run.smithi027:> ') 2024-09-12T03:20:08.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:08 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T03:20:08.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T03:20:08.209 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:20:08.209 DEBUG:teuthology.orchestra.run.smithi027:> ip netns list 2024-09-12T03:20:08.226 INFO:teuthology.orchestra.run.smithi027.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-12T03:20:08.226 INFO:teuthology.orchestra.run.smithi027.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-12T03:20:08.227 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-12T03:20:08.227 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:20:08.227 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip link set brx.0 up 2024-09-12T03:20:08.299 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-12T03:20:08.299 DEBUG:teuthology.orchestra.run.smithi027:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-12T03:20:08.358 INFO:teuthology.orchestra.run.smithi027.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-12T03:20:08.359 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-12T03:20:08.359 DEBUG:teuthology.orchestra.run.smithi027:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-12T03:20:08.417 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-09-12T03:20:08.417 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:20:08.417 DEBUG:teuthology.orchestra.run.smithi027:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-12T03:20:08.502 INFO:teuthology.orchestra.run.smithi027.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-12T03:20:08.507 INFO:teuthology.orchestra.run.smithi027.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-12T03:20:08.519 INFO:teuthology.orchestra.run.smithi027.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-12T03:20:08.520 INFO:teuthology.orchestra.run.smithi027.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-12T03:20:08.520 INFO:teuthology.orchestra.run.smithi027.stdout:mount.ceph: resolved to: "172.21.15.27:3300,172.21.15.45:3300,172.21.15.45:3301" 2024-09-12T03:20:08.520 INFO:teuthology.orchestra.run.smithi027.stdout:mount.ceph: trying mount with new device syntax: 0@d2df504e-2647-4385-a5db-9fdf69f487a0.cephfs=/ 2024-09-12T03:20:08.520 INFO:teuthology.orchestra.run.smithi027.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.27:3300/172.21.15.45:3300/172.21.15.45:3301" will pass to kernel 2024-09-12T03:20:08.520 INFO:teuthology.orchestra.run.smithi027.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-12T03:20:08.521 INFO:teuthology.orchestra.run.smithi027.stdout: You just mounted a file system that supports labels which does not 2024-09-12T03:20:08.521 INFO:teuthology.orchestra.run.smithi027.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-12T03:20:08.521 INFO:teuthology.orchestra.run.smithi027.stdout: applications will generate AVC messages and not be allowed access to 2024-09-12T03:20:08.521 INFO:teuthology.orchestra.run.smithi027.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-12T03:20:08.523 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-12T03:20:08.523 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:20:08.523 DEBUG:teuthology.orchestra.run.smithi027:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-12T03:20:08.550 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-12T03:20:08.642 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:08 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-12T03:20:08.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T03:20:08.654 INFO:teuthology.orchestra.run.smithi027.stdout:client5480 2024-09-12T03:20:08.655 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-12T03:20:08.655 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T03:20:08.671 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-12T03:20:08.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:08.813+0000 7f24c3627640 1 Processor -- start 2024-09-12T03:20:08.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:08.813+0000 7f24c3627640 1 -- start start 2024-09-12T03:20:08.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:08.814+0000 7f24c3627640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f24bc073960 0x7f24bc162320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:08.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:08.814+0000 7f24c3627640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f24bc162860 0x7f24bc15e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:08.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:08.814+0000 7f24c3627640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f24bc0d2240 con 0x7f24bc15e5f0 2024-09-12T03:20:08.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:08.814+0000 7f24c3627640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f24bc0d0da0 con 0x7f24bc162860 2024-09-12T03:20:08.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:08.814+0000 7f24c3627640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f24bc163090 con 0x7f24bc073960 2024-09-12T03:20:08.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:08.814+0000 7f24c13c3640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f24bc15e5f0 0x7f24bc073250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:34894/0 (socket says 172.21.15.27:34894) 2024-09-12T03:20:08.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:08.814+0000 7f24c13c3640 1 -- 172.21.15.27:0/4255063549 learned_addr learned my addr 172.21.15.27:0/4255063549 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:20:08.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:08.814+0000 7f24c0bc2640 1 --2- 172.21.15.27:0/4255063549 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f24bc073960 0x7f24bc162320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:08.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:08.814+0000 7f24c1bc4640 1 --2- 172.21.15.27:0/4255063549 >> v2:172.21.15.45:3300/0 conn(0x7f24bc162860 0x7f24bc15e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:08.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:08.814+0000 7f24b3fff640 1 -- 172.21.15.27:0/4255063549 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3225707473 0 0) 0x7f24bc0d2240 con 0x7f24bc15e5f0 2024-09-12T03:20:08.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:08.814+0000 7f24b3fff640 1 -- 172.21.15.27:0/4255063549 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f24a8003520 con 0x7f24bc15e5f0 2024-09-12T03:20:08.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:08.815+0000 7f24b3fff640 1 -- 172.21.15.27:0/4255063549 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 330370925 0 0) 0x7f24a8003520 con 0x7f24bc15e5f0 2024-09-12T03:20:08.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:08.815+0000 7f24b3fff640 1 -- 172.21.15.27:0/4255063549 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f24bc073960 msgr2=0x7f24bc162320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:08.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:08.815+0000 7f24b3fff640 1 --2- 172.21.15.27:0/4255063549 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f24bc073960 0x7f24bc162320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:08.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:08.815+0000 7f24b3fff640 1 -- 172.21.15.27:0/4255063549 >> v2:172.21.15.45:3300/0 conn(0x7f24bc162860 msgr2=0x7f24bc15e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:08.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:08.815+0000 7f24b3fff640 1 --2- 172.21.15.27:0/4255063549 >> v2:172.21.15.45:3300/0 conn(0x7f24bc162860 0x7f24bc15e9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:08.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:08.815+0000 7f24b3fff640 1 -- 172.21.15.27:0/4255063549 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24bc0d0470 con 0x7f24bc15e5f0 2024-09-12T03:20:08.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:08.815+0000 7f24b3fff640 1 -- 172.21.15.27:0/4255063549 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 847755213 0 0) 0x7f24b8001670 con 0x7f24bc15e5f0 2024-09-12T03:20:08.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:08.815+0000 7f24b3fff640 1 -- 172.21.15.27:0/4255063549 >> v1:172.21.15.27:6789/0 conn(0x7f24bc15e5f0 legacy=0x7f24bc073250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:08.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:08.815+0000 7f24b3fff640 1 --2- 172.21.15.27:0/4255063549 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f24a80039b0 0x7f24a8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:08.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:08.815+0000 7f24b3fff640 1 --2- 172.21.15.27:0/4255063549 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f24a8004800 0x7f24a8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:08.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:08.815+0000 7f24b3fff640 1 --2- 172.21.15.27:0/4255063549 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f24bc073960 0x7f24a80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:08.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:08.815+0000 7f24b3fff640 1 -- 172.21.15.27:0/4255063549 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f24bc0d0da0 con 0x7f24a8004800 2024-09-12T03:20:08.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:08.815+0000 7f24b3fff640 1 -- 172.21.15.27:0/4255063549 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f24bc163090 con 0x7f24a80039b0 2024-09-12T03:20:08.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:08.815+0000 7f24b3fff640 1 -- 172.21.15.27:0/4255063549 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f24a800b9c0 con 0x7f24bc073960 2024-09-12T03:20:08.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:08.815+0000 7f24c0bc2640 1 --2- 172.21.15.27:0/4255063549 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f24a8004800 0x7f24a8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:08.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:08.815+0000 7f24c13c3640 1 --2- 172.21.15.27:0/4255063549 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f24a80039b0 0x7f24a8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:08.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:08.815+0000 7f24c1bc4640 1 --2- 172.21.15.27:0/4255063549 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f24bc073960 0x7f24a80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:08.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:08.815+0000 7f24c0bc2640 1 -- 172.21.15.27:0/4255063549 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f24bc073960 msgr2=0x7f24a80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:08.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:08.815+0000 7f24c0bc2640 1 --2- 172.21.15.27:0/4255063549 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f24bc073960 0x7f24a80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:08.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:08.815+0000 7f24c0bc2640 1 -- 172.21.15.27:0/4255063549 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f24a80039b0 msgr2=0x7f24a8003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:08.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:08.815+0000 7f24c0bc2640 1 --2- 172.21.15.27:0/4255063549 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f24a80039b0 0x7f24a8003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:08.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:08.815+0000 7f24c0bc2640 1 -- 172.21.15.27:0/4255063549 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24a8003520 con 0x7f24a8004800 2024-09-12T03:20:08.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:08.815+0000 7f24c1bc4640 1 --2- 172.21.15.27:0/4255063549 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f24bc073960 0x7f24a80051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:20:08.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:08.816+0000 7f24c0bc2640 1 --2- 172.21.15.27:0/4255063549 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f24a8004800 0x7f24a8004c80 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f24ac009a90 tx=0x7f24a8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa0105b22a387706 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:08.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:08.816+0000 7f24b3fff640 1 -- 172.21.15.27:0/4255063549 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24ac036070 con 0x7f24a8004800 2024-09-12T03:20:08.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:08.816+0000 7f24b3fff640 1 -- 172.21.15.27:0/4255063549 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24ac031400 con 0x7f24a8004800 2024-09-12T03:20:08.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:08.816+0000 7f24b3fff640 1 -- 172.21.15.27:0/4255063549 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24ac009690 con 0x7f24a8004800 2024-09-12T03:20:08.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:08.817+0000 7f24c3627640 1 -- 172.21.15.27:0/4255063549 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f24a8004800 msgr2=0x7f24a8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:08.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:08.817+0000 7f24c3627640 1 --2- 172.21.15.27:0/4255063549 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f24a8004800 0x7f24a8004c80 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f24ac009a90 tx=0x7f24a8005830 comp rx=0 tx=0).stop 2024-09-12T03:20:08.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:08.817+0000 7f24c1bc4640 1 -- 172.21.15.27:0/4255063549 reap_dead start 2024-09-12T03:20:08.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:08.817+0000 7f24c3627640 1 -- 172.21.15.27:0/4255063549 shutdown_connections 2024-09-12T03:20:08.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:08.817+0000 7f24c3627640 1 -- 172.21.15.27:0/4255063549 >> 172.21.15.27:0/4255063549 conn(0x7f24bc06f300 msgr2=0x7f24bc06f710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:08.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:08.817+0000 7f24c3627640 1 -- 172.21.15.27:0/4255063549 shutdown_connections 2024-09-12T03:20:08.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:08.817+0000 7f24c3627640 1 -- 172.21.15.27:0/4255063549 wait complete. 2024-09-12T03:20:08.826 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:08.818+0000 7f24c3627640 1 Processor -- start 2024-09-12T03:20:08.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:08.818+0000 7f24c3627640 1 -- start start 2024-09-12T03:20:08.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:08.819+0000 7f24c3627640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f24a8004800 0x7f24bc1544e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:08.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:08.819+0000 7f24c3627640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f24bc156780 0x7f24bc152fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:08.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:08.819+0000 7f24c3627640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f24bc15cd00 0x7f24bc15d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:08.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:08.819+0000 7f24c3627640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f24bc0b5620 con 0x7f24bc156780 2024-09-12T03:20:08.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:08.819+0000 7f24c3627640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f24bc0d0a30 con 0x7f24a8004800 2024-09-12T03:20:08.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:08.819+0000 7f24c3627640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f24bc162cf0 con 0x7f24bc15cd00 2024-09-12T03:20:08.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:08.819+0000 7f24c0bc2640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f24bc156780 0x7f24bc152fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:08.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:08.819+0000 7f24c0bc2640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f24bc156780 0x7f24bc152fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56348/0 (socket says 172.21.15.27:56348) 2024-09-12T03:20:08.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:08.819+0000 7f24c0bc2640 1 -- 172.21.15.27:0/3416277147 learned_addr learned my addr 172.21.15.27:0/3416277147 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:20:08.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:08.819+0000 7f24c13c3640 1 --2- 172.21.15.27:0/3416277147 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f24a8004800 0x7f24bc1544e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:08.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:08.819+0000 7f24c1bc4640 1 --2- 172.21.15.27:0/3416277147 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f24bc15cd00 0x7f24bc15d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:08.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:08.819+0000 7f24c0bc2640 1 -- 172.21.15.27:0/3416277147 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f24bc15cd00 msgr2=0x7f24bc15d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:08.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:08.819+0000 7f24c0bc2640 1 --2- 172.21.15.27:0/3416277147 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f24bc15cd00 0x7f24bc15d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:08.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:08.819+0000 7f24c0bc2640 1 -- 172.21.15.27:0/3416277147 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f24a8004800 msgr2=0x7f24bc1544e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:08.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:08.819+0000 7f24c0bc2640 1 --2- 172.21.15.27:0/3416277147 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f24a8004800 0x7f24bc1544e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:08.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:08.819+0000 7f24c0bc2640 1 -- 172.21.15.27:0/3416277147 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24bc153510 con 0x7f24bc156780 2024-09-12T03:20:08.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:08.819+0000 7f24c0bc2640 1 --2- 172.21.15.27:0/3416277147 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f24bc156780 0x7f24bc152fd0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f24ac034040 tx=0x7f24ac004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:08.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:08.819+0000 7f24b1ffb640 1 -- 172.21.15.27:0/3416277147 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24ac041020 con 0x7f24bc156780 2024-09-12T03:20:08.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:08.820+0000 7f24b1ffb640 1 -- 172.21.15.27:0/3416277147 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24ac03c470 con 0x7f24bc156780 2024-09-12T03:20:08.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:08.820+0000 7f24b1ffb640 1 -- 172.21.15.27:0/3416277147 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24ac036040 con 0x7f24bc156780 2024-09-12T03:20:08.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:08.820+0000 7f24c3627640 1 -- 172.21.15.27:0/3416277147 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24bc154ad0 con 0x7f24bc156780 2024-09-12T03:20:08.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:08.820+0000 7f24c3627640 1 -- 172.21.15.27:0/3416277147 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24bc155090 con 0x7f24bc156780 2024-09-12T03:20:08.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:08.821+0000 7f24c3627640 1 -- 172.21.15.27:0/3416277147 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24bc162320 con 0x7f24bc156780 2024-09-12T03:20:08.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:08.821+0000 7f24b1ffb640 1 -- 172.21.15.27:0/3416277147 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 11) ==== 90804+0+0 (secure 0 0 0) 0x7f24ac044020 con 0x7f24bc156780 2024-09-12T03:20:08.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:08.822+0000 7f24b1ffb640 1 --2- 172.21.15.27:0/3416277147 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f24a006c1c0 0x7f24a006e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:08.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:08.822+0000 7f24b1ffb640 1 -- 172.21.15.27:0/3416277147 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 5709+0+0 (secure 0 0 0) 0x7f24ac0ab950 con 0x7f24bc156780 2024-09-12T03:20:08.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:08.822+0000 7f24c13c3640 1 --2- 172.21.15.27:0/3416277147 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f24a006c1c0 0x7f24a006e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:08.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:08.822+0000 7f24c13c3640 1 --2- 172.21.15.27:0/3416277147 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f24a006c1c0 0x7f24a006e680 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f24b80015d0 tx=0x7f24b80078f0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:08.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:08.826+0000 7f24b1ffb640 1 -- 172.21.15.27:0/3416277147 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f24ac04b030 con 0x7f24bc156780 2024-09-12T03:20:09.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:09.040+0000 7f24c3627640 1 -- 172.21.15.27:0/3416277147 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f24bc0d1ed0 con 0x7f24bc156780 2024-09-12T03:20:09.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:09.040+0000 7f24b1ffb640 1 -- 172.21.15.27:0/3416277147 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v38) ==== 53+0+73 (secure 0 0 0) 0x7f24bc153510 con 0x7f24bc156780 2024-09-12T03:20:09.042 INFO:teuthology.orchestra.run.smithi027.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-12T03:20:09.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:09.042+0000 7f24c3627640 1 -- 172.21.15.27:0/3416277147 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f24a006c1c0 msgr2=0x7f24a006e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:09.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:09.042+0000 7f24c3627640 1 --2- 172.21.15.27:0/3416277147 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f24a006c1c0 0x7f24a006e680 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f24b80015d0 tx=0x7f24b80078f0 comp rx=0 tx=0).stop 2024-09-12T03:20:09.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:09.042+0000 7f24c3627640 1 -- 172.21.15.27:0/3416277147 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f24bc156780 msgr2=0x7f24bc152fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:09.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:09.042+0000 7f24c3627640 1 --2- 172.21.15.27:0/3416277147 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f24bc156780 0x7f24bc152fd0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f24ac034040 tx=0x7f24ac004730 comp rx=0 tx=0).stop 2024-09-12T03:20:09.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:09.042+0000 7f24c3627640 1 -- 172.21.15.27:0/3416277147 shutdown_connections 2024-09-12T03:20:09.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:09.042+0000 7f24c3627640 1 --2- 172.21.15.27:0/3416277147 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f24a006c1c0 0x7f24a006e680 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:09.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:09.042+0000 7f24c3627640 1 --2- 172.21.15.27:0/3416277147 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f24bc15cd00 0x7f24bc15d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:09.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:09.042+0000 7f24c3627640 1 --2- 172.21.15.27:0/3416277147 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f24bc156780 0x7f24bc152fd0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:09.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:09.042+0000 7f24c3627640 1 --2- 172.21.15.27:0/3416277147 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f24a8004800 0x7f24bc1544e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:09.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:09.042+0000 7f24c3627640 1 -- 172.21.15.27:0/3416277147 >> 172.21.15.27:0/3416277147 conn(0x7f24bc06f300 msgr2=0x7f24bc159950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:09.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:09.042+0000 7f24c3627640 1 -- 172.21.15.27:0/3416277147 shutdown_connections 2024-09-12T03:20:09.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:09.042+0000 7f24c3627640 1 -- 172.21.15.27:0/3416277147 wait complete. 2024-09-12T03:20:09.054 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-12T03:20:09.055 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-12T03:20:09.055 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi045.front.sepia.ceph.com 2024-09-12T03:20:09.055 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-12T03:20:09.055 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-12T03:20:09.055 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-12T03:20:09.055 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-12T03:20:09.055 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-12T03:20:09.055 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-12T03:20:09.055 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:20:09.055 DEBUG:teuthology.orchestra.run.smithi045:> ip addr 2024-09-12T03:20:09.076 INFO:teuthology.orchestra.run.smithi045.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-12T03:20:09.077 INFO:teuthology.orchestra.run.smithi045.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-12T03:20:09.077 INFO:teuthology.orchestra.run.smithi045.stdout: inet 127.0.0.1/8 scope host lo 2024-09-12T03:20:09.077 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:20:09.077 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 ::1/128 scope host 2024-09-12T03:20:09.077 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:20:09.077 INFO:teuthology.orchestra.run.smithi045.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-12T03:20:09.077 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:6b:fd:a6 brd ff:ff:ff:ff:ff:ff 2024-09-12T03:20:09.077 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp7s0f0 2024-09-12T03:20:09.077 INFO:teuthology.orchestra.run.smithi045.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-12T03:20:09.077 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:6b:fd:a7 brd ff:ff:ff:ff:ff:ff 2024-09-12T03:20:09.077 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp7s0f1 2024-09-12T03:20:09.077 INFO:teuthology.orchestra.run.smithi045.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-12T03:20:09.078 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:bd:12:be brd ff:ff:ff:ff:ff:ff 2024-09-12T03:20:09.078 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp1s0f0 2024-09-12T03:20:09.078 INFO:teuthology.orchestra.run.smithi045.stdout: inet 172.21.15.45/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-12T03:20:09.078 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft 42524sec preferred_lft 42524sec 2024-09-12T03:20:09.078 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::ec4:7aff:febd:12be/64 scope link 2024-09-12T03:20:09.078 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:20:09.078 INFO:teuthology.orchestra.run.smithi045.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-12T03:20:09.078 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:bd:12:bf brd ff:ff:ff:ff:ff:ff 2024-09-12T03:20:09.078 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp1s0f1 2024-09-12T03:20:09.078 INFO:teuthology.orchestra.run.smithi045.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-12T03:20:09.078 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 52:97:33:5c:60:1b brd ff:ff:ff:ff:ff:ff 2024-09-12T03:20:09.078 INFO:teuthology.orchestra.run.smithi045.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-12T03:20:09.078 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:20:09.078 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::d8c2:5dff:fe26:78c8/64 scope link 2024-09-12T03:20:09.079 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:20:09.079 INFO:teuthology.orchestra.run.smithi045.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-12T03:20:09.079 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 5a:ec:d7:40:ef:75 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-12T03:20:09.079 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::58ec:d7ff:fe40:ef75/64 scope link 2024-09-12T03:20:09.079 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:20:09.079 INFO:teuthology.orchestra.run.smithi045.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-12T03:20:09.079 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 52:97:33:5c:60:1b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-12T03:20:09.079 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::5097:33ff:fe5c:601b/64 scope link 2024-09-12T03:20:09.079 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:20:09.079 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:20:09.079 DEBUG:teuthology.orchestra.run.smithi045:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-12T03:20:09.147 INFO:teuthology.orchestra.run.smithi045.stdout:1 2024-09-12T03:20:09.149 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:20:09.149 DEBUG:teuthology.orchestra.run.smithi045:> route 2024-09-12T03:20:09.208 INFO:teuthology.orchestra.run.smithi045.stdout:Kernel IP routing table 2024-09-12T03:20:09.208 INFO:teuthology.orchestra.run.smithi045.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-12T03:20:09.208 INFO:teuthology.orchestra.run.smithi045.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-12T03:20:09.208 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-12T03:20:09.208 INFO:teuthology.orchestra.run.smithi045.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-12T03:20:09.209 DEBUG:teuthology.orchestra.run.smithi045:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-12T03:20:09.209 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-09-12T03:20:09.209 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-12T03:20:09.209 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-12T03:20:09.209 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-12T03:20:09.209 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-09-12T03:20:09.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:20:09 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T03:20:09.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:20:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T03:20:09.349 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:20:09.350 DEBUG:teuthology.orchestra.run.smithi045:> ip netns list 2024-09-12T03:20:09.368 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-09-12T03:20:09.368 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-12T03:20:09.368 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-12T03:20:09.368 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:20:09.369 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link set brx.0 up 2024-09-12T03:20:09.436 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-12T03:20:09.436 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-12T03:20:09.493 INFO:teuthology.orchestra.run.smithi045.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-12T03:20:09.493 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-12T03:20:09.493 DEBUG:teuthology.orchestra.run.smithi045:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-12T03:20:09.550 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2024-09-12T03:20:09.550 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:20:09.551 DEBUG:teuthology.orchestra.run.smithi045:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-12T03:20:09.635 INFO:teuthology.orchestra.run.smithi045.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-12T03:20:09.635 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-12T03:20:09.643 INFO:teuthology.orchestra.run.smithi045.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-12T03:20:09.643 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-12T03:20:09.643 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: resolved to: "172.21.15.27:3300,172.21.15.45:3300,172.21.15.45:3301" 2024-09-12T03:20:09.643 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: trying mount with new device syntax: 1@d2df504e-2647-4385-a5db-9fdf69f487a0.cephfs=/ 2024-09-12T03:20:09.643 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.27:3300/172.21.15.45:3300/172.21.15.45:3301" will pass to kernel 2024-09-12T03:20:09.643 INFO:teuthology.orchestra.run.smithi045.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-09-12T03:20:09.644 INFO:teuthology.orchestra.run.smithi045.stdout: You just mounted a file system that supports labels which does not 2024-09-12T03:20:09.644 INFO:teuthology.orchestra.run.smithi045.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-12T03:20:09.644 INFO:teuthology.orchestra.run.smithi045.stdout: applications will generate AVC messages and not be allowed access to 2024-09-12T03:20:09.644 INFO:teuthology.orchestra.run.smithi045.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-12T03:20:09.645 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-12T03:20:09.645 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:20:09.646 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-12T03:20:09.671 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-12T03:20:09.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:20:09 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-12T03:20:09.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:20:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T03:20:09.769 INFO:teuthology.orchestra.run.smithi045.stdout:client6234 2024-09-12T03:20:09.769 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-12T03:20:09.769 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T03:20:09.786 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-09-12T03:20:09.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:09.898+0000 7fe86d76d640 1 Processor -- start 2024-09-12T03:20:09.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:09.898+0000 7fe86d76d640 1 -- start start 2024-09-12T03:20:09.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:09.899+0000 7fe86d76d640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fe86815e990 0x7fe8680b7600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:09.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:09.899+0000 7fe86d76d640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe8680b3d10 0x7fe8680b66c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:09.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:09.899+0000 7fe86d76d640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe86815ed70 con 0x7fe8680b6230 2024-09-12T03:20:09.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:09.899+0000 7fe86d76d640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fe8680d2a30 con 0x7fe86815e990 2024-09-12T03:20:09.902 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:09.899+0000 7fe86d76d640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fe8680d2d10 con 0x7fe8680b3d10 2024-09-12T03:20:09.902 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:09.899+0000 7fe85ffff640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fe8680b6230 0x7fe8680b3600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:50878/0 (socket says 172.21.15.27:50878) 2024-09-12T03:20:09.902 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:09.899+0000 7fe85ffff640 1 -- 172.21.15.27:0/3993126036 learned_addr learned my addr 172.21.15.27:0/3993126036 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:20:09.902 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:09.899+0000 7fe866ffd640 1 --2- 172.21.15.27:0/3993126036 >> v2:172.21.15.45:3300/0 conn(0x7fe86815e990 0x7fe8680b7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:09.902 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:09.899+0000 7fe8667fc640 1 -- 172.21.15.27:0/3993126036 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3646497863 0 0) 0x7fe86815ed70 con 0x7fe8680b6230 2024-09-12T03:20:09.902 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:09.899+0000 7fe8677fe640 1 --2- 172.21.15.27:0/3993126036 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe8680b3d10 0x7fe8680b66c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:09.902 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:09.899+0000 7fe8667fc640 1 -- 172.21.15.27:0/3993126036 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe854003520 con 0x7fe8680b6230 2024-09-12T03:20:09.902 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:09.900+0000 7fe8667fc640 1 -- 172.21.15.27:0/3993126036 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1110276789 0 0) 0x7fe854003520 con 0x7fe8680b6230 2024-09-12T03:20:09.902 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:09.900+0000 7fe8667fc640 1 -- 172.21.15.27:0/3993126036 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe8680b3d10 msgr2=0x7fe8680b66c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:09.902 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:09.900+0000 7fe8667fc640 1 --2- 172.21.15.27:0/3993126036 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe8680b3d10 0x7fe8680b66c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:09.902 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:09.900+0000 7fe8667fc640 1 -- 172.21.15.27:0/3993126036 >> v2:172.21.15.45:3300/0 conn(0x7fe86815e990 msgr2=0x7fe8680b7600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:09.902 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:09.900+0000 7fe8667fc640 1 --2- 172.21.15.27:0/3993126036 >> v2:172.21.15.45:3300/0 conn(0x7fe86815e990 0x7fe8680b7600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:09.902 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:09.900+0000 7fe8667fc640 1 -- 172.21.15.27:0/3993126036 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8680d2ff0 con 0x7fe8680b6230 2024-09-12T03:20:09.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:09.900+0000 7fe8667fc640 1 -- 172.21.15.27:0/3993126036 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 847755213 0 0) 0x7fe850001670 con 0x7fe8680b6230 2024-09-12T03:20:09.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:09.900+0000 7fe8667fc640 1 -- 172.21.15.27:0/3993126036 >> v1:172.21.15.27:6789/0 conn(0x7fe8680b6230 legacy=0x7fe8680b3600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:09.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:09.900+0000 7fe8667fc640 1 --2- 172.21.15.27:0/3993126036 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe8540039b0 0x7fe854003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:09.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:09.900+0000 7fe8667fc640 1 --2- 172.21.15.27:0/3993126036 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe8680b3d10 0x7fe854004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:09.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:09.900+0000 7fe8667fc640 1 --2- 172.21.15.27:0/3993126036 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe854004d40 0x7fe8540051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:09.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:09.900+0000 7fe8667fc640 1 -- 172.21.15.27:0/3993126036 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fe8680d2a30 con 0x7fe854004d40 2024-09-12T03:20:09.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:09.900+0000 7fe8667fc640 1 -- 172.21.15.27:0/3993126036 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe8680d2d10 con 0x7fe8540039b0 2024-09-12T03:20:09.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:09.900+0000 7fe8667fc640 1 -- 172.21.15.27:0/3993126036 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fe85400b9c0 con 0x7fe8680b3d10 2024-09-12T03:20:09.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:09.900+0000 7fe8677fe640 1 --2- 172.21.15.27:0/3993126036 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe854004d40 0x7fe8540051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:09.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:09.900+0000 7fe866ffd640 1 --2- 172.21.15.27:0/3993126036 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe8540039b0 0x7fe854003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:09.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:09.900+0000 7fe85ffff640 1 --2- 172.21.15.27:0/3993126036 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe8680b3d10 0x7fe854004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:09.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:09.900+0000 7fe8677fe640 1 -- 172.21.15.27:0/3993126036 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe8680b3d10 msgr2=0x7fe854004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:09.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:09.900+0000 7fe8677fe640 1 --2- 172.21.15.27:0/3993126036 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe8680b3d10 0x7fe854004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:09.904 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:09.900+0000 7fe8677fe640 1 -- 172.21.15.27:0/3993126036 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe8540039b0 msgr2=0x7fe854003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:09.904 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:09.900+0000 7fe8677fe640 1 --2- 172.21.15.27:0/3993126036 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe8540039b0 0x7fe854003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:09.904 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:09.900+0000 7fe8677fe640 1 -- 172.21.15.27:0/3993126036 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe854003520 con 0x7fe854004d40 2024-09-12T03:20:09.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:09.901+0000 7fe8677fe640 1 --2- 172.21.15.27:0/3993126036 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe854004d40 0x7fe8540051c0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fe860009fc0 tx=0x7fe854005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e310300f6b494c61 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:09.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:09.901+0000 7fe8667fc640 1 -- 172.21.15.27:0/3993126036 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe860009c10 con 0x7fe854004d40 2024-09-12T03:20:09.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:09.901+0000 7fe8667fc640 1 -- 172.21.15.27:0/3993126036 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe860036070 con 0x7fe854004d40 2024-09-12T03:20:09.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:09.901+0000 7fe8667fc640 1 -- 172.21.15.27:0/3993126036 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe860031880 con 0x7fe854004d40 2024-09-12T03:20:09.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:09.902+0000 7fe86d76d640 1 -- 172.21.15.27:0/3993126036 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe854004d40 msgr2=0x7fe8540051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:09.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:09.902+0000 7fe86d76d640 1 --2- 172.21.15.27:0/3993126036 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe854004d40 0x7fe8540051c0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fe860009fc0 tx=0x7fe854005830 comp rx=0 tx=0).stop 2024-09-12T03:20:09.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:09.902+0000 7fe8677fe640 1 -- 172.21.15.27:0/3993126036 reap_dead start 2024-09-12T03:20:09.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:09.902+0000 7fe86d76d640 1 -- 172.21.15.27:0/3993126036 shutdown_connections 2024-09-12T03:20:09.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:09.902+0000 7fe86d76d640 1 -- 172.21.15.27:0/3993126036 >> 172.21.15.27:0/3993126036 conn(0x7fe8680d1230 msgr2=0x7fe8680d1640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:09.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:09.902+0000 7fe86d76d640 1 -- 172.21.15.27:0/3993126036 shutdown_connections 2024-09-12T03:20:09.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:09.902+0000 7fe86d76d640 1 -- 172.21.15.27:0/3993126036 wait complete. 2024-09-12T03:20:09.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:09.903+0000 7fe86d76d640 1 Processor -- start 2024-09-12T03:20:09.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:09.903+0000 7fe86d76d640 1 -- start start 2024-09-12T03:20:09.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:09.904+0000 7fe86d76d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe854004d40 0x7fe8680cb880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:09.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:09.904+0000 7fe86d76d640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe868160350 0x7fe8681618f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:09.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:09.904+0000 7fe86d76d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe868150e20 0x7fe86815cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:09.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:09.904+0000 7fe86d76d640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fe8680b7b40 con 0x7fe854004d40 2024-09-12T03:20:09.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:09.904+0000 7fe86d76d640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe8680d35b0 con 0x7fe868150e20 2024-09-12T03:20:09.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:09.904+0000 7fe86d76d640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fe8680b7cc0 con 0x7fe868160350 2024-09-12T03:20:09.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:09.904+0000 7fe866ffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe854004d40 0x7fe8680cb880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:09.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:09.904+0000 7fe866ffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe854004d40 0x7fe8680cb880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57778/0 (socket says 172.21.15.27:57778) 2024-09-12T03:20:09.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:09.904+0000 7fe866ffd640 1 -- 172.21.15.27:0/1568392962 learned_addr learned my addr 172.21.15.27:0/1568392962 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:20:09.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:09.904+0000 7fe85ffff640 1 --2- 172.21.15.27:0/1568392962 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe868160350 0x7fe8681618f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:09.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:09.904+0000 7fe8677fe640 1 --2- 172.21.15.27:0/1568392962 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe868150e20 0x7fe86815cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:09.909 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:09.904+0000 7fe866ffd640 1 -- 172.21.15.27:0/1568392962 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe868160350 msgr2=0x7fe8681618f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:09.909 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:09.904+0000 7fe866ffd640 1 --2- 172.21.15.27:0/1568392962 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe868160350 0x7fe8681618f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:09.909 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:09.904+0000 7fe866ffd640 1 -- 172.21.15.27:0/1568392962 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe868150e20 msgr2=0x7fe86815cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:09.909 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:09.904+0000 7fe866ffd640 1 --2- 172.21.15.27:0/1568392962 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe868150e20 0x7fe86815cf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:09.909 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:09.904+0000 7fe866ffd640 1 -- 172.21.15.27:0/1568392962 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe86815b390 con 0x7fe854004d40 2024-09-12T03:20:09.909 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:09.904+0000 7fe866ffd640 1 --2- 172.21.15.27:0/1568392962 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe854004d40 0x7fe8680cb880 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fe858011aa0 tx=0x7fe858011f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:09.909 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:09.905+0000 7fe85f7fe640 1 -- 172.21.15.27:0/1568392962 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe85800fb20 con 0x7fe854004d40 2024-09-12T03:20:09.909 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:09.905+0000 7fe85f7fe640 1 -- 172.21.15.27:0/1568392962 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe85800fcc0 con 0x7fe854004d40 2024-09-12T03:20:09.909 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:09.905+0000 7fe86d76d640 1 -- 172.21.15.27:0/1568392962 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe868161e30 con 0x7fe854004d40 2024-09-12T03:20:09.909 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:09.905+0000 7fe85f7fe640 1 -- 172.21.15.27:0/1568392962 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe858014660 con 0x7fe854004d40 2024-09-12T03:20:09.909 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:09.905+0000 7fe86d76d640 1 -- 172.21.15.27:0/1568392962 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe868160b30 con 0x7fe854004d40 2024-09-12T03:20:09.909 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:09.906+0000 7fe86d76d640 1 -- 172.21.15.27:0/1568392962 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8680b7600 con 0x7fe854004d40 2024-09-12T03:20:09.909 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:09.906+0000 7fe85f7fe640 1 -- 172.21.15.27:0/1568392962 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 11) ==== 90804+0+0 (secure 0 0 0) 0x7fe858012040 con 0x7fe854004d40 2024-09-12T03:20:09.910 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:09.907+0000 7fe85f7fe640 1 --2- 172.21.15.27:0/1568392962 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fe84806c1c0 0x7fe84806e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:09.910 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:09.907+0000 7fe85f7fe640 1 -- 172.21.15.27:0/1568392962 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 5709+0+0 (secure 0 0 0) 0x7fe85808b8a0 con 0x7fe854004d40 2024-09-12T03:20:09.910 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:09.907+0000 7fe85ffff640 1 --2- 172.21.15.27:0/1568392962 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fe84806c1c0 0x7fe84806e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:09.910 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:09.907+0000 7fe85ffff640 1 --2- 172.21.15.27:0/1568392962 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fe84806c1c0 0x7fe84806e680 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fe850007fd0 tx=0x7fe8500078f0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:09.912 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:09.911+0000 7fe85f7fe640 1 -- 172.21.15.27:0/1568392962 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fe86815b390 con 0x7fe854004d40 2024-09-12T03:20:10.123 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:10.122+0000 7fe86d76d640 1 -- 172.21.15.27:0/1568392962 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7fe8680c8080 con 0x7fe854004d40 2024-09-12T03:20:10.423 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:10.422+0000 7fe85f7fe640 1 -- 172.21.15.27:0/1568392962 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v39) ==== 136+0+0 (secure 0 0 0) 0x7fe858019030 con 0x7fe854004d40 2024-09-12T03:20:10.426 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:10.425+0000 7fe86d76d640 1 -- 172.21.15.27:0/1568392962 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fe84806c1c0 msgr2=0x7fe84806e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:10.426 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:10.425+0000 7fe86d76d640 1 --2- 172.21.15.27:0/1568392962 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fe84806c1c0 0x7fe84806e680 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fe850007fd0 tx=0x7fe8500078f0 comp rx=0 tx=0).stop 2024-09-12T03:20:10.426 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:10.425+0000 7fe86d76d640 1 -- 172.21.15.27:0/1568392962 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe854004d40 msgr2=0x7fe8680cb880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:10.426 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:10.425+0000 7fe86d76d640 1 --2- 172.21.15.27:0/1568392962 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe854004d40 0x7fe8680cb880 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fe858011aa0 tx=0x7fe858011f60 comp rx=0 tx=0).stop 2024-09-12T03:20:10.426 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:10.425+0000 7fe86d76d640 1 -- 172.21.15.27:0/1568392962 shutdown_connections 2024-09-12T03:20:10.426 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:10.425+0000 7fe86d76d640 1 --2- 172.21.15.27:0/1568392962 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fe84806c1c0 0x7fe84806e680 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:10.426 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:10.425+0000 7fe86d76d640 1 --2- 172.21.15.27:0/1568392962 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe868150e20 0x7fe86815cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:10.426 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:10.425+0000 7fe86d76d640 1 --2- 172.21.15.27:0/1568392962 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe868160350 0x7fe8681618f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:10.427 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:10.425+0000 7fe86d76d640 1 --2- 172.21.15.27:0/1568392962 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe854004d40 0x7fe8680cb880 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:10.427 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:10.425+0000 7fe86d76d640 1 -- 172.21.15.27:0/1568392962 >> 172.21.15.27:0/1568392962 conn(0x7fe8680d1230 msgr2=0x7fe8680b82a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:10.427 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:10.425+0000 7fe86d76d640 1 -- 172.21.15.27:0/1568392962 shutdown_connections 2024-09-12T03:20:10.427 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:10.425+0000 7fe86d76d640 1 -- 172.21.15.27:0/1568392962 wait complete. 2024-09-12T03:20:10.438 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-12T03:20:10.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:10.541+0000 7f1b70edd640 1 Processor -- start 2024-09-12T03:20:10.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:10.541+0000 7f1b70edd640 1 -- start start 2024-09-12T03:20:10.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:10.542+0000 7f1b70edd640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1b6c162a30 0x7f1b6c1693f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:10.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:10.542+0000 7f1b70edd640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1b6c0d2240 con 0x7f1b6c15e5f0 2024-09-12T03:20:10.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:10.542+0000 7f1b70edd640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1b6c163090 con 0x7f1b6c073960 2024-09-12T03:20:10.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:10.542+0000 7f1b70edd640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f1b6c0d0da0 con 0x7f1b6c162a30 2024-09-12T03:20:10.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:10.542+0000 7f1b6a59c640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f1b6c15e5f0 0x7f1b6c073250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:50888/0 (socket says 172.21.15.27:50888) 2024-09-12T03:20:10.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:10.542+0000 7f1b6a59c640 1 -- 172.21.15.27:0/3035432633 learned_addr learned my addr 172.21.15.27:0/3035432633 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:20:10.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:10.542+0000 7f1b6ad9d640 1 --2- 172.21.15.27:0/3035432633 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1b6c162a30 0x7f1b6c1693f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:10.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:10.542+0000 7f1b6959a640 1 -- 172.21.15.27:0/3035432633 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2833777234 0 0) 0x7f1b6c0d2240 con 0x7f1b6c15e5f0 2024-09-12T03:20:10.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:10.542+0000 7f1b6959a640 1 -- 172.21.15.27:0/3035432633 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1b54003520 con 0x7f1b6c15e5f0 2024-09-12T03:20:10.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:10.543+0000 7f1b6959a640 1 -- 172.21.15.27:0/3035432633 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3295072912 0 0) 0x7f1b54003520 con 0x7f1b6c15e5f0 2024-09-12T03:20:10.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:10.543+0000 7f1b6959a640 1 -- 172.21.15.27:0/3035432633 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1b6c162a30 msgr2=0x7f1b6c1693f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:10.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:10.543+0000 7f1b6959a640 1 --2- 172.21.15.27:0/3035432633 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1b6c162a30 0x7f1b6c1693f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:10.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:10.543+0000 7f1b6959a640 1 -- 172.21.15.27:0/3035432633 >> v1:172.21.15.45:6789/0 conn(0x7f1b6c073960 legacy=0x7f1b6c162320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:10.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:10.543+0000 7f1b6959a640 1 -- 172.21.15.27:0/3035432633 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b6c0d0470 con 0x7f1b6c15e5f0 2024-09-12T03:20:10.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:10.543+0000 7f1b6959a640 1 -- 172.21.15.27:0/3035432633 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 847755213 0 0) 0x7f1b5c001670 con 0x7f1b6c15e5f0 2024-09-12T03:20:10.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:10.543+0000 7f1b6959a640 1 -- 172.21.15.27:0/3035432633 >> v1:172.21.15.27:6789/0 conn(0x7f1b6c15e5f0 legacy=0x7f1b6c073250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:10.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:10.543+0000 7f1b6959a640 1 --2- 172.21.15.27:0/3035432633 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1b540039e0 0x7f1b54003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:10.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:10.543+0000 7f1b6959a640 1 --2- 172.21.15.27:0/3035432633 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1b540043b0 0x7f1b54004880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:10.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:10.543+0000 7f1b6959a640 1 --2- 172.21.15.27:0/3035432633 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1b6c162a30 0x7f1b540050e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:10.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:10.543+0000 7f1b6959a640 1 -- 172.21.15.27:0/3035432633 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f1b6c0d0da0 con 0x7f1b540039e0 2024-09-12T03:20:10.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:10.543+0000 7f1b6959a640 1 -- 172.21.15.27:0/3035432633 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f1b5400b9f0 con 0x7f1b540043b0 2024-09-12T03:20:10.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:10.543+0000 7f1b6959a640 1 -- 172.21.15.27:0/3035432633 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f1b5400bbf0 con 0x7f1b6c162a30 2024-09-12T03:20:10.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:10.543+0000 7f1b6a59c640 1 --2- 172.21.15.27:0/3035432633 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1b540039e0 0x7f1b54003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:10.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:10.543+0000 7f1b6ad9d640 1 --2- 172.21.15.27:0/3035432633 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1b6c162a30 0x7f1b540050e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:10.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:10.543+0000 7f1b69d9b640 1 --2- 172.21.15.27:0/3035432633 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1b540043b0 0x7f1b54004880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:10.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:10.543+0000 7f1b6a59c640 1 -- 172.21.15.27:0/3035432633 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1b6c162a30 msgr2=0x7f1b540050e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:10.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:10.543+0000 7f1b6a59c640 1 --2- 172.21.15.27:0/3035432633 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1b6c162a30 0x7f1b540050e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:10.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:10.543+0000 7f1b6a59c640 1 -- 172.21.15.27:0/3035432633 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1b540043b0 msgr2=0x7f1b54004880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:10.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:10.543+0000 7f1b6a59c640 1 --2- 172.21.15.27:0/3035432633 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1b540043b0 0x7f1b54004880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:10.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:10.543+0000 7f1b6a59c640 1 -- 172.21.15.27:0/3035432633 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b54003520 con 0x7f1b540039e0 2024-09-12T03:20:10.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:10.543+0000 7f1b6ad9d640 1 --2- 172.21.15.27:0/3035432633 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1b6c162a30 0x7f1b540050e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:20:10.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:10.543+0000 7f1b69d9b640 1 --2- 172.21.15.27:0/3035432633 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1b540043b0 0x7f1b54004880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:20:10.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:10.544+0000 7f1b6a59c640 1 --2- 172.21.15.27:0/3035432633 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1b540039e0 0x7f1b54003de0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f1b5c007fd0 tx=0x7f1b54005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3b78357bb4472ea server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:10.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:10.544+0000 7f1b6959a640 1 -- 172.21.15.27:0/3035432633 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b5c007ca0 con 0x7f1b540039e0 2024-09-12T03:20:10.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:10.544+0000 7f1b6959a640 1 -- 172.21.15.27:0/3035432633 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b5c005ad0 con 0x7f1b540039e0 2024-09-12T03:20:10.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:10.544+0000 7f1b6959a640 1 -- 172.21.15.27:0/3035432633 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b5c007070 con 0x7f1b540039e0 2024-09-12T03:20:10.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:10.544+0000 7f1b70edd640 1 -- 172.21.15.27:0/3035432633 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1b540039e0 msgr2=0x7f1b54003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:10.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:10.544+0000 7f1b70edd640 1 --2- 172.21.15.27:0/3035432633 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1b540039e0 0x7f1b54003de0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f1b5c007fd0 tx=0x7f1b54005830 comp rx=0 tx=0).stop 2024-09-12T03:20:10.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:10.544+0000 7f1b6ad9d640 1 -- 172.21.15.27:0/3035432633 reap_dead start 2024-09-12T03:20:10.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:10.545+0000 7f1b70edd640 1 -- 172.21.15.27:0/3035432633 shutdown_connections 2024-09-12T03:20:10.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:10.545+0000 7f1b70edd640 1 -- 172.21.15.27:0/3035432633 >> 172.21.15.27:0/3035432633 conn(0x7f1b6c06f300 msgr2=0x7f1b6c06f710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:10.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:10.545+0000 7f1b70edd640 1 -- 172.21.15.27:0/3035432633 shutdown_connections 2024-09-12T03:20:10.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:10.545+0000 7f1b70edd640 1 -- 172.21.15.27:0/3035432633 wait complete. 2024-09-12T03:20:10.551 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:10.546+0000 7f1b70edd640 1 Processor -- start 2024-09-12T03:20:10.551 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:10.546+0000 7f1b70edd640 1 -- start start 2024-09-12T03:20:10.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:10.547+0000 7f1b70edd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1b540039e0 0x7f1b6c0b2500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:10.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:10.547+0000 7f1b70edd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1b6c15cf60 0x7f1b6c1f7450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:10.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:10.547+0000 7f1b70edd640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1b6c1f7990 0x7f1b6c155f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:10.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:10.547+0000 7f1b70edd640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f1b6c0b5620 con 0x7f1b540039e0 2024-09-12T03:20:10.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:10.547+0000 7f1b70edd640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f1b6c0d0a30 con 0x7f1b6c15cf60 2024-09-12T03:20:10.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:10.547+0000 7f1b70edd640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f1b6c15f9e0 con 0x7f1b6c1f7990 2024-09-12T03:20:10.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:10.547+0000 7f1b6a59c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1b540039e0 0x7f1b6c0b2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:10.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:10.547+0000 7f1b6a59c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1b540039e0 0x7f1b6c0b2500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57790/0 (socket says 172.21.15.27:57790) 2024-09-12T03:20:10.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:10.547+0000 7f1b6a59c640 1 -- 172.21.15.27:0/3370685811 learned_addr learned my addr 172.21.15.27:0/3370685811 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:20:10.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:10.547+0000 7f1b69d9b640 1 --2- 172.21.15.27:0/3370685811 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1b6c15cf60 0x7f1b6c1f7450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:10.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:10.547+0000 7f1b6ad9d640 1 --2- 172.21.15.27:0/3370685811 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1b6c1f7990 0x7f1b6c155f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:10.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:10.547+0000 7f1b6a59c640 1 -- 172.21.15.27:0/3370685811 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1b6c1f7990 msgr2=0x7f1b6c155f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:10.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:10.547+0000 7f1b6a59c640 1 --2- 172.21.15.27:0/3370685811 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1b6c1f7990 0x7f1b6c155f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:10.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:10.547+0000 7f1b6a59c640 1 -- 172.21.15.27:0/3370685811 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1b6c15cf60 msgr2=0x7f1b6c1f7450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:10.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:10.547+0000 7f1b6a59c640 1 --2- 172.21.15.27:0/3370685811 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1b6c15cf60 0x7f1b6c1f7450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:10.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:10.547+0000 7f1b6a59c640 1 -- 172.21.15.27:0/3370685811 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b6c1f4dc0 con 0x7f1b540039e0 2024-09-12T03:20:10.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:10.547+0000 7f1b69d9b640 1 --2- 172.21.15.27:0/3370685811 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1b6c15cf60 0x7f1b6c1f7450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:20:10.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:10.547+0000 7f1b6a59c640 1 --2- 172.21.15.27:0/3370685811 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1b540039e0 0x7f1b6c0b2500 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f1b5c00c250 tx=0x7f1b6c163260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:10.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:10.547+0000 7f1b5affd640 1 -- 172.21.15.27:0/3370685811 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b5c03f020 con 0x7f1b540039e0 2024-09-12T03:20:10.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:10.547+0000 7f1b5affd640 1 -- 172.21.15.27:0/3370685811 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b5c001430 con 0x7f1b540039e0 2024-09-12T03:20:10.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:10.547+0000 7f1b70edd640 1 -- 172.21.15.27:0/3370685811 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b6c1f7ea0 con 0x7f1b540039e0 2024-09-12T03:20:10.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:10.547+0000 7f1b5affd640 1 -- 172.21.15.27:0/3370685811 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b5c0047e0 con 0x7f1b540039e0 2024-09-12T03:20:10.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:10.548+0000 7f1b70edd640 1 -- 172.21.15.27:0/3370685811 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b6c15c760 con 0x7f1b540039e0 2024-09-12T03:20:10.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:10.549+0000 7f1b70edd640 1 -- 172.21.15.27:0/3370685811 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b6c15c550 con 0x7f1b540039e0 2024-09-12T03:20:10.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:10.549+0000 7f1b5affd640 1 -- 172.21.15.27:0/3370685811 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 11) ==== 90804+0+0 (secure 0 0 0) 0x7f1b5c042020 con 0x7f1b540039e0 2024-09-12T03:20:10.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:10.550+0000 7f1b5affd640 1 --2- 172.21.15.27:0/3370685811 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f1b4c06c1c0 0x7f1b4c06e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:10.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:10.550+0000 7f1b5affd640 1 -- 172.21.15.27:0/3370685811 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 5709+0+0 (secure 0 0 0) 0x7f1b5c0aae50 con 0x7f1b540039e0 2024-09-12T03:20:10.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:10.550+0000 7f1b69d9b640 1 --2- 172.21.15.27:0/3370685811 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f1b4c06c1c0 0x7f1b4c06e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:10.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:10.550+0000 7f1b69d9b640 1 --2- 172.21.15.27:0/3370685811 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f1b4c06c1c0 0x7f1b4c06e680 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f1b6c0b7910 tx=0x7f1b60009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:10.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:10.554+0000 7f1b5affd640 1 -- 172.21.15.27:0/3370685811 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f1b5c0793f0 con 0x7f1b540039e0 2024-09-12T03:20:10.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:10.753+0000 7f1b70edd640 1 -- 172.21.15.27:0/3370685811 --> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f1b6c0d1470 con 0x7f1b4c06c1c0 2024-09-12T03:20:10.756 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:10.756+0000 7f1b5affd640 1 -- 172.21.15.27:0/3370685811 <== mgr.4102 v2:172.21.15.45:6832/3923821219 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f1b6c0d1470 con 0x7f1b4c06c1c0 2024-09-12T03:20:10.756 INFO:teuthology.orchestra.run.smithi027.stdout:[ 2024-09-12T03:20:10.756 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:20:10.756 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "cephfs" 2024-09-12T03:20:10.756 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:20:10.756 INFO:teuthology.orchestra.run.smithi027.stdout:] 2024-09-12T03:20:10.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:10.757+0000 7f1b70edd640 1 -- 172.21.15.27:0/3370685811 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f1b4c06c1c0 msgr2=0x7f1b4c06e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:10.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:10.757+0000 7f1b70edd640 1 --2- 172.21.15.27:0/3370685811 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f1b4c06c1c0 0x7f1b4c06e680 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f1b6c0b7910 tx=0x7f1b60009040 comp rx=0 tx=0).stop 2024-09-12T03:20:10.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:10.757+0000 7f1b70edd640 1 -- 172.21.15.27:0/3370685811 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1b540039e0 msgr2=0x7f1b6c0b2500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:10.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:10.757+0000 7f1b70edd640 1 --2- 172.21.15.27:0/3370685811 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1b540039e0 0x7f1b6c0b2500 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f1b5c00c250 tx=0x7f1b6c163260 comp rx=0 tx=0).stop 2024-09-12T03:20:10.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:10.757+0000 7f1b70edd640 1 -- 172.21.15.27:0/3370685811 shutdown_connections 2024-09-12T03:20:10.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:10.757+0000 7f1b70edd640 1 --2- 172.21.15.27:0/3370685811 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f1b4c06c1c0 0x7f1b4c06e680 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:10.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:10.757+0000 7f1b70edd640 1 --2- 172.21.15.27:0/3370685811 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1b6c1f7990 0x7f1b6c155f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:10.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:10.757+0000 7f1b70edd640 1 --2- 172.21.15.27:0/3370685811 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1b6c15cf60 0x7f1b6c1f7450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:10.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:10.757+0000 7f1b70edd640 1 --2- 172.21.15.27:0/3370685811 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1b540039e0 0x7f1b6c0b2500 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:10.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:10.757+0000 7f1b70edd640 1 -- 172.21.15.27:0/3370685811 >> 172.21.15.27:0/3370685811 conn(0x7f1b6c06f300 msgr2=0x7f1b6c06f710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:10.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:10.757+0000 7f1b70edd640 1 -- 172.21.15.27:0/3370685811 shutdown_connections 2024-09-12T03:20:10.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:10.757+0000 7f1b70edd640 1 -- 172.21.15.27:0/3370685811 wait complete. 2024-09-12T03:20:10.770 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-12T03:20:10.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:10.875+0000 7f228015f640 1 Processor -- start 2024-09-12T03:20:10.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:10.875+0000 7f228015f640 1 -- start start 2024-09-12T03:20:10.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:10.876+0000 7f228015f640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2278070d30 0x7f227806cc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:10.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:10.876+0000 7f228015f640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f22780cf9f0 con 0x7f22780d0b50 2024-09-12T03:20:10.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:10.876+0000 7f228015f640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f22780cfc90 con 0x7f227806e960 2024-09-12T03:20:10.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:10.876+0000 7f228015f640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f22780d1a40 con 0x7f2278070d30 2024-09-12T03:20:10.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:10.876+0000 7f227e6fc640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f22780d0b50 0x7f2278168300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:50900/0 (socket says 172.21.15.27:50900) 2024-09-12T03:20:10.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:10.876+0000 7f227e6fc640 1 -- 172.21.15.27:0/2017221876 learned_addr learned my addr 172.21.15.27:0/2017221876 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:20:10.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:10.876+0000 7f227defb640 1 --2- 172.21.15.27:0/2017221876 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2278070d30 0x7f227806cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:10.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:10.876+0000 7f227cef9640 1 -- 172.21.15.27:0/2017221876 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2410633753 0 0) 0x7f22780cf9f0 con 0x7f22780d0b50 2024-09-12T03:20:10.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:10.876+0000 7f227cef9640 1 -- 172.21.15.27:0/2017221876 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2264003520 con 0x7f22780d0b50 2024-09-12T03:20:10.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:10.877+0000 7f227cef9640 1 -- 172.21.15.27:0/2017221876 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 850522122 0 0) 0x7f2264003520 con 0x7f22780d0b50 2024-09-12T03:20:10.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:10.877+0000 7f227cef9640 1 -- 172.21.15.27:0/2017221876 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2278070d30 msgr2=0x7f227806cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:10.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:10.877+0000 7f227cef9640 1 --2- 172.21.15.27:0/2017221876 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2278070d30 0x7f227806cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:10.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:10.877+0000 7f227cef9640 1 -- 172.21.15.27:0/2017221876 >> v1:172.21.15.45:6789/0 conn(0x7f227806e960 legacy=0x7f22780d0440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:10.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:10.877+0000 7f227cef9640 1 -- 172.21.15.27:0/2017221876 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22780cff70 con 0x7f22780d0b50 2024-09-12T03:20:10.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:10.877+0000 7f227cef9640 1 -- 172.21.15.27:0/2017221876 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 847755213 0 0) 0x7f22700019f0 con 0x7f22780d0b50 2024-09-12T03:20:10.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:10.877+0000 7f227cef9640 1 -- 172.21.15.27:0/2017221876 >> v1:172.21.15.27:6789/0 conn(0x7f22780d0b50 legacy=0x7f2278168300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:10.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:10.877+0000 7f227cef9640 1 --2- 172.21.15.27:0/2017221876 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f22640039e0 0x7f2264003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:10.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:10.877+0000 7f227cef9640 1 --2- 172.21.15.27:0/2017221876 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2278070d30 0x7f2264004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:10.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:10.877+0000 7f227cef9640 1 --2- 172.21.15.27:0/2017221876 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2264004d40 0x7f22640051a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:10.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:10.877+0000 7f227cef9640 1 -- 172.21.15.27:0/2017221876 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f22780d1a40 con 0x7f2264004d40 2024-09-12T03:20:10.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:10.877+0000 7f227cef9640 1 -- 172.21.15.27:0/2017221876 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f226400bbd0 con 0x7f22640039e0 2024-09-12T03:20:10.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:10.877+0000 7f227cef9640 1 -- 172.21.15.27:0/2017221876 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f226400bdd0 con 0x7f2278070d30 2024-09-12T03:20:10.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:10.877+0000 7f227e6fc640 1 --2- 172.21.15.27:0/2017221876 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2264004d40 0x7f22640051a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:10.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:10.877+0000 7f227d6fa640 1 --2- 172.21.15.27:0/2017221876 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2278070d30 0x7f2264004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:10.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:10.877+0000 7f227defb640 1 --2- 172.21.15.27:0/2017221876 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f22640039e0 0x7f2264003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:10.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:10.877+0000 7f227e6fc640 1 -- 172.21.15.27:0/2017221876 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2278070d30 msgr2=0x7f2264004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:10.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:10.877+0000 7f227e6fc640 1 --2- 172.21.15.27:0/2017221876 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2278070d30 0x7f2264004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:10.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:10.877+0000 7f227e6fc640 1 -- 172.21.15.27:0/2017221876 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f22640039e0 msgr2=0x7f2264003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:10.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:10.877+0000 7f227e6fc640 1 --2- 172.21.15.27:0/2017221876 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f22640039e0 0x7f2264003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:10.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:10.877+0000 7f227e6fc640 1 -- 172.21.15.27:0/2017221876 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2264003520 con 0x7f2264004d40 2024-09-12T03:20:10.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:10.877+0000 7f227d6fa640 1 --2- 172.21.15.27:0/2017221876 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2278070d30 0x7f2264004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:20:10.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:10.878+0000 7f227e6fc640 1 --2- 172.21.15.27:0/2017221876 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2264004d40 0x7f22640051a0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f227000a760 tx=0x7f2264005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5b7a5bde0e89873 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:10.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:10.878+0000 7f227cef9640 1 -- 172.21.15.27:0/2017221876 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f227000a3b0 con 0x7f2264004d40 2024-09-12T03:20:10.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:10.878+0000 7f227cef9640 1 -- 172.21.15.27:0/2017221876 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f227002f070 con 0x7f2264004d40 2024-09-12T03:20:10.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:10.878+0000 7f227cef9640 1 -- 172.21.15.27:0/2017221876 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f227002bbe0 con 0x7f2264004d40 2024-09-12T03:20:10.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:10.878+0000 7f228015f640 1 -- 172.21.15.27:0/2017221876 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2264004d40 msgr2=0x7f22640051a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:10.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:10.878+0000 7f228015f640 1 --2- 172.21.15.27:0/2017221876 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2264004d40 0x7f22640051a0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f227000a760 tx=0x7f2264005830 comp rx=0 tx=0).stop 2024-09-12T03:20:10.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:10.878+0000 7f227e6fc640 1 -- 172.21.15.27:0/2017221876 reap_dead start 2024-09-12T03:20:10.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:10.879+0000 7f228015f640 1 -- 172.21.15.27:0/2017221876 shutdown_connections 2024-09-12T03:20:10.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:10.879+0000 7f228015f640 1 -- 172.21.15.27:0/2017221876 >> 172.21.15.27:0/2017221876 conn(0x7f22780b7c10 msgr2=0x7f227809ba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:10.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:10.879+0000 7f228015f640 1 -- 172.21.15.27:0/2017221876 shutdown_connections 2024-09-12T03:20:10.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:10.879+0000 7f228015f640 1 -- 172.21.15.27:0/2017221876 wait complete. 2024-09-12T03:20:10.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:10.880+0000 7f228015f640 1 Processor -- start 2024-09-12T03:20:10.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:10.880+0000 7f228015f640 1 -- start start 2024-09-12T03:20:10.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:10.881+0000 7f228015f640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2264004d40 0x7f2278153db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:10.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:10.881+0000 7f228015f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f22780cc820 0x7f22780ce0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:10.888 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:10.881+0000 7f228015f640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f227806f360 0x7f227806f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:10.888 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:10.881+0000 7f228015f640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f22780d3340 con 0x7f227806f360 2024-09-12T03:20:10.888 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:10.881+0000 7f228015f640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f22780d31c0 con 0x7f22780cc820 2024-09-12T03:20:10.888 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:10.881+0000 7f228015f640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f227806cc40 con 0x7f2264004d40 2024-09-12T03:20:10.888 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:10.881+0000 7f227e6fc640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f227806f360 0x7f227806f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:10.888 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:10.881+0000 7f227e6fc640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f227806f360 0x7f227806f7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57804/0 (socket says 172.21.15.27:57804) 2024-09-12T03:20:10.888 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:10.881+0000 7f227e6fc640 1 -- 172.21.15.27:0/2713289268 learned_addr learned my addr 172.21.15.27:0/2713289268 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:20:10.888 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:10.881+0000 7f227defb640 1 --2- 172.21.15.27:0/2713289268 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2264004d40 0x7f2278153db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:10.888 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:10.881+0000 7f227d6fa640 1 --2- 172.21.15.27:0/2713289268 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f22780cc820 0x7f22780ce0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:10.888 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:10.881+0000 7f227e6fc640 1 -- 172.21.15.27:0/2713289268 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2264004d40 msgr2=0x7f2278153db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:10.888 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:10.881+0000 7f227e6fc640 1 --2- 172.21.15.27:0/2713289268 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2264004d40 0x7f2278153db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:10.888 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:10.881+0000 7f227e6fc640 1 -- 172.21.15.27:0/2713289268 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f22780cc820 msgr2=0x7f22780ce0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:10.889 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:10.881+0000 7f227e6fc640 1 --2- 172.21.15.27:0/2713289268 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f22780cc820 0x7f22780ce0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:10.889 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:10.881+0000 7f227e6fc640 1 -- 172.21.15.27:0/2713289268 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2278160a70 con 0x7f227806f360 2024-09-12T03:20:10.889 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:10.881+0000 7f227e6fc640 1 --2- 172.21.15.27:0/2713289268 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f227806f360 0x7f227806f7e0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f2270001ff0 tx=0x7f22780cfe60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:10.889 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:10.881+0000 7f226e7fc640 1 -- 172.21.15.27:0/2713289268 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f227000a3b0 con 0x7f227806f360 2024-09-12T03:20:10.889 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:10.881+0000 7f226e7fc640 1 -- 172.21.15.27:0/2713289268 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f227002bb90 con 0x7f227806f360 2024-09-12T03:20:10.889 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:10.881+0000 7f228015f640 1 -- 172.21.15.27:0/2713289268 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2278160ce0 con 0x7f227806f360 2024-09-12T03:20:10.889 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:10.881+0000 7f226e7fc640 1 -- 172.21.15.27:0/2713289268 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2270034470 con 0x7f227806f360 2024-09-12T03:20:10.889 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:10.882+0000 7f228015f640 1 -- 172.21.15.27:0/2713289268 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22781612a0 con 0x7f227806f360 2024-09-12T03:20:10.889 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:10.883+0000 7f228015f640 1 -- 172.21.15.27:0/2713289268 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f227806d020 con 0x7f227806f360 2024-09-12T03:20:10.889 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:10.883+0000 7f226e7fc640 1 -- 172.21.15.27:0/2713289268 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 11) ==== 90804+0+0 (secure 0 0 0) 0x7f2270046020 con 0x7f227806f360 2024-09-12T03:20:10.889 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:10.884+0000 7f226e7fc640 1 --2- 172.21.15.27:0/2713289268 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f225c06c1c0 0x7f225c06e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:10.889 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:10.884+0000 7f226e7fc640 1 -- 172.21.15.27:0/2713289268 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 5709+0+0 (secure 0 0 0) 0x7f22700aafe0 con 0x7f227806f360 2024-09-12T03:20:10.889 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:10.884+0000 7f227defb640 1 --2- 172.21.15.27:0/2713289268 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f225c06c1c0 0x7f225c06e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:10.890 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:10.885+0000 7f227defb640 1 --2- 172.21.15.27:0/2713289268 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f225c06c1c0 0x7f225c06e680 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f2278155ba0 tx=0x7f2274008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:10.890 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:10.888+0000 7f226e7fc640 1 -- 172.21.15.27:0/2713289268 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f2270078ea0 con 0x7f227806f360 2024-09-12T03:20:11.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:11.062+0000 7f228015f640 1 -- 172.21.15.27:0/2713289268 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f2278154870 con 0x7f227806f360 2024-09-12T03:20:11.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:11.063+0000 7f226e7fc640 1 -- 172.21.15.27:0/2713289268 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v16)=0 v16) ==== 122+0+0 (secure 0 0 0) 0x7f227007d1a0 con 0x7f227806f360 2024-09-12T03:20:11.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:11.064+0000 7f228015f640 1 -- 172.21.15.27:0/2713289268 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f225c06c1c0 msgr2=0x7f225c06e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:11.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:11.064+0000 7f228015f640 1 --2- 172.21.15.27:0/2713289268 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f225c06c1c0 0x7f225c06e680 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f2278155ba0 tx=0x7f2274008040 comp rx=0 tx=0).stop 2024-09-12T03:20:11.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:11.064+0000 7f228015f640 1 -- 172.21.15.27:0/2713289268 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f227806f360 msgr2=0x7f227806f7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:11.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:11.064+0000 7f228015f640 1 --2- 172.21.15.27:0/2713289268 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f227806f360 0x7f227806f7e0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f2270001ff0 tx=0x7f22780cfe60 comp rx=0 tx=0).stop 2024-09-12T03:20:11.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:11.065+0000 7f228015f640 1 -- 172.21.15.27:0/2713289268 shutdown_connections 2024-09-12T03:20:11.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:11.065+0000 7f228015f640 1 --2- 172.21.15.27:0/2713289268 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f225c06c1c0 0x7f225c06e680 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:11.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:11.065+0000 7f228015f640 1 --2- 172.21.15.27:0/2713289268 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f227806f360 0x7f227806f7e0 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:11.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:11.065+0000 7f228015f640 1 --2- 172.21.15.27:0/2713289268 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f22780cc820 0x7f22780ce0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:11.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:11.065+0000 7f228015f640 1 --2- 172.21.15.27:0/2713289268 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2264004d40 0x7f2278153db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:11.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:11.065+0000 7f228015f640 1 -- 172.21.15.27:0/2713289268 >> 172.21.15.27:0/2713289268 conn(0x7f22780b7c10 msgr2=0x7f22780af940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:11.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:11.065+0000 7f228015f640 1 -- 172.21.15.27:0/2713289268 shutdown_connections 2024-09-12T03:20:11.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:11.065+0000 7f228015f640 1 -- 172.21.15.27:0/2713289268 wait complete. 2024-09-12T03:20:11.077 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-12T03:20:11.078 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T03:20:11.096 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-12T03:20:11.096 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T03:20:11.165 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-12T03:20:11.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:11.312+0000 7f5d9ba8d640 1 Processor -- start 2024-09-12T03:20:11.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:11.312+0000 7f5d9ba8d640 1 -- start start 2024-09-12T03:20:11.314 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:11.313+0000 7f5d9ba8d640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f5d94072d70 0x7f5d940b3560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:11.314 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:11.313+0000 7f5d9ba8d640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f5d9406e9b0 0x7f5d9406ee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:11.314 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:11.313+0000 7f5d9ba8d640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5d940ce410 con 0x7f5d9415b2b0 2024-09-12T03:20:11.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:11.313+0000 7f5d9ba8d640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f5d940720a0 con 0x7f5d94072d70 2024-09-12T03:20:11.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:11.313+0000 7f5d9ba8d640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f5d9406f340 con 0x7f5d9406e9b0 2024-09-12T03:20:11.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:11.313+0000 7f5d99028640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f5d94072d70 0x7f5d940b3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:11.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:11.313+0000 7f5d99829640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f5d9415b2b0 0x7f5d94072430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:50904/0 (socket says 172.21.15.27:50904) 2024-09-12T03:20:11.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:11.313+0000 7f5d99829640 1 -- 172.21.15.27:0/1000937842 learned_addr learned my addr 172.21.15.27:0/1000937842 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:20:11.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:11.313+0000 7f5d9a02a640 1 --2- 172.21.15.27:0/1000937842 >> v2:172.21.15.45:3300/0 conn(0x7f5d9406e9b0 0x7f5d9406ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:11.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:11.313+0000 7f5d99028640 1 -- 172.21.15.27:0/1000937842 >> v2:172.21.15.45:3300/0 conn(0x7f5d9406e9b0 msgr2=0x7f5d9406ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:11.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:11.313+0000 7f5d99028640 1 --2- 172.21.15.27:0/1000937842 >> v2:172.21.15.45:3300/0 conn(0x7f5d9406e9b0 0x7f5d9406ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:11.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:11.313+0000 7f5d99028640 1 -- 172.21.15.27:0/1000937842 >> v1:172.21.15.27:6789/0 conn(0x7f5d9415b2b0 legacy=0x7f5d94072430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:11.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:11.313+0000 7f5d99028640 1 -- 172.21.15.27:0/1000937842 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d940732c0 con 0x7f5d94072d70 2024-09-12T03:20:11.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:11.313+0000 7f5d98827640 1 -- 172.21.15.27:0/1000937842 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1272835074 0 0) 0x7f5d90001110 con 0x7f5d9415b2b0 2024-09-12T03:20:11.316 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:11.314+0000 7f5d99028640 1 --2- 172.21.15.27:0/1000937842 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5d94072d70 0x7f5d940b3560 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f5d940ce1e0 tx=0x7f5d8402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=250ac1d5b64cf182 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:11.316 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:11.314+0000 7f5d98827640 1 -- 172.21.15.27:0/1000937842 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d84037070 con 0x7f5d94072d70 2024-09-12T03:20:11.317 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:11.314+0000 7f5d98827640 1 -- 172.21.15.27:0/1000937842 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d84032440 con 0x7f5d94072d70 2024-09-12T03:20:11.317 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:11.314+0000 7f5d98827640 1 -- 172.21.15.27:0/1000937842 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d84032810 con 0x7f5d94072d70 2024-09-12T03:20:11.317 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:11.315+0000 7f5d9ba8d640 1 -- 172.21.15.27:0/1000937842 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5d94072d70 msgr2=0x7f5d940b3560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:11.317 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:11.315+0000 7f5d9ba8d640 1 --2- 172.21.15.27:0/1000937842 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5d94072d70 0x7f5d940b3560 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f5d940ce1e0 tx=0x7f5d8402a1a0 comp rx=0 tx=0).stop 2024-09-12T03:20:11.317 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:11.315+0000 7f5d9ba8d640 1 -- 172.21.15.27:0/1000937842 shutdown_connections 2024-09-12T03:20:11.317 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:11.315+0000 7f5d9ba8d640 1 --2- 172.21.15.27:0/1000937842 >> v2:172.21.15.45:3300/0 conn(0x7f5d9406e9b0 0x7f5d9406ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:11.317 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:11.315+0000 7f5d9ba8d640 1 --2- 172.21.15.27:0/1000937842 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5d94072d70 0x7f5d940b3560 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:11.317 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:11.315+0000 7f5d9ba8d640 1 -- 172.21.15.27:0/1000937842 >> 172.21.15.27:0/1000937842 conn(0x7f5d94062c60 msgr2=0x7f5d9406bb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:11.317 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:11.315+0000 7f5d9ba8d640 1 -- 172.21.15.27:0/1000937842 shutdown_connections 2024-09-12T03:20:11.317 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:11.315+0000 7f5d9ba8d640 1 -- 172.21.15.27:0/1000937842 wait complete. 2024-09-12T03:20:11.319 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:11.316+0000 7f5d9ba8d640 1 Processor -- start 2024-09-12T03:20:11.319 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:11.316+0000 7f5d9ba8d640 1 -- start start 2024-09-12T03:20:11.319 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:11.317+0000 7f5d9ba8d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5d9406e9b0 0x7f5d9415a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:11.319 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:11.317+0000 7f5d9ba8d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5d94072d70 0x7f5d940ae8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:11.320 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:11.317+0000 7f5d9ba8d640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f5d9415b2b0 0x7f5d940aee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:11.320 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:11.317+0000 7f5d9ba8d640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f5d940ce690 con 0x7f5d9406e9b0 2024-09-12T03:20:11.320 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:11.317+0000 7f5d9ba8d640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f5d94071dc0 con 0x7f5d94072d70 2024-09-12T03:20:11.320 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:11.317+0000 7f5d9ba8d640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f5d9415b690 con 0x7f5d9415b2b0 2024-09-12T03:20:11.320 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:11.317+0000 7f5d99829640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5d9406e9b0 0x7f5d9415a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:11.320 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:11.317+0000 7f5d99829640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5d9406e9b0 0x7f5d9415a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57820/0 (socket says 172.21.15.27:57820) 2024-09-12T03:20:11.320 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:11.317+0000 7f5d99829640 1 -- 172.21.15.27:0/3102892687 learned_addr learned my addr 172.21.15.27:0/3102892687 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:20:11.320 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:11.317+0000 7f5d9a02a640 1 --2- 172.21.15.27:0/3102892687 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f5d9415b2b0 0x7f5d940aee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:11.320 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:11.317+0000 7f5d99028640 1 --2- 172.21.15.27:0/3102892687 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5d94072d70 0x7f5d940ae8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:11.320 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:11.317+0000 7f5d99829640 1 -- 172.21.15.27:0/3102892687 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f5d9415b2b0 msgr2=0x7f5d940aee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:11.320 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:11.317+0000 7f5d99829640 1 --2- 172.21.15.27:0/3102892687 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f5d9415b2b0 0x7f5d940aee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:11.320 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:11.317+0000 7f5d99829640 1 -- 172.21.15.27:0/3102892687 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5d94072d70 msgr2=0x7f5d940ae8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:11.320 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:11.317+0000 7f5d99829640 1 --2- 172.21.15.27:0/3102892687 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5d94072d70 0x7f5d940ae8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:11.321 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:11.317+0000 7f5d99829640 1 -- 172.21.15.27:0/3102892687 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d94150560 con 0x7f5d9406e9b0 2024-09-12T03:20:11.321 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:11.318+0000 7f5d99829640 1 --2- 172.21.15.27:0/3102892687 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5d9406e9b0 0x7f5d9415a2f0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f5d9000c430 tx=0x7f5d9000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:11.321 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:11.318+0000 7f5d8a7fc640 1 -- 172.21.15.27:0/3102892687 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d900074e0 con 0x7f5d9406e9b0 2024-09-12T03:20:11.321 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:11.318+0000 7f5d8a7fc640 1 -- 172.21.15.27:0/3102892687 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d900047e0 con 0x7f5d9406e9b0 2024-09-12T03:20:11.321 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:11.318+0000 7f5d8a7fc640 1 -- 172.21.15.27:0/3102892687 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d90004bd0 con 0x7f5d9406e9b0 2024-09-12T03:20:11.321 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:11.318+0000 7f5d9ba8d640 1 -- 172.21.15.27:0/3102892687 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d94150840 con 0x7f5d9406e9b0 2024-09-12T03:20:11.321 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:11.318+0000 7f5d9ba8d640 1 -- 172.21.15.27:0/3102892687 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d940b3690 con 0x7f5d9406e9b0 2024-09-12T03:20:11.322 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:11.319+0000 7f5d9ba8d640 1 -- 172.21.15.27:0/3102892687 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d9406ee00 con 0x7f5d9406e9b0 2024-09-12T03:20:11.322 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:11.319+0000 7f5d8a7fc640 1 -- 172.21.15.27:0/3102892687 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 11) ==== 90804+0+0 (secure 0 0 0) 0x7f5d90011070 con 0x7f5d9406e9b0 2024-09-12T03:20:11.322 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:11.320+0000 7f5d8a7fc640 1 --2- 172.21.15.27:0/3102892687 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f5d7806c190 0x7f5d7806e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:11.322 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:11.320+0000 7f5d8a7fc640 1 -- 172.21.15.27:0/3102892687 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 5709+0+0 (secure 0 0 0) 0x7f5d9008b0b0 con 0x7f5d9406e9b0 2024-09-12T03:20:11.322 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:11.320+0000 7f5d99028640 1 --2- 172.21.15.27:0/3102892687 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f5d7806c190 0x7f5d7806e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:11.322 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:11.321+0000 7f5d99028640 1 --2- 172.21.15.27:0/3102892687 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f5d7806c190 0x7f5d7806e650 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f5d84034000 tx=0x7f5d8402e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:11.325 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:11.324+0000 7f5d8a7fc640 1 -- 172.21.15.27:0/3102892687 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f5d940ce410 con 0x7f5d9406e9b0 2024-09-12T03:20:11.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:11.515+0000 7f5d9ba8d640 1 -- 172.21.15.27:0/3102892687 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f5d940b38d0 con 0x7f5d9406e9b0 2024-09-12T03:20:11.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:11.515+0000 7f5d8a7fc640 1 -- 172.21.15.27:0/3102892687 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v67) ==== 80+0+0 (secure 0 0 0) 0x7f5d900595e0 con 0x7f5d9406e9b0 2024-09-12T03:20:11.516 INFO:teuthology.orchestra.run.smithi027.stderr:listed 0 entries 2024-09-12T03:20:11.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:11.517+0000 7f5d9ba8d640 1 -- 172.21.15.27:0/3102892687 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f5d7806c190 msgr2=0x7f5d7806e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:11.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:11.517+0000 7f5d9ba8d640 1 --2- 172.21.15.27:0/3102892687 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f5d7806c190 0x7f5d7806e650 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f5d84034000 tx=0x7f5d8402e000 comp rx=0 tx=0).stop 2024-09-12T03:20:11.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:11.517+0000 7f5d9ba8d640 1 -- 172.21.15.27:0/3102892687 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5d9406e9b0 msgr2=0x7f5d9415a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:11.519 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:11.517+0000 7f5d9ba8d640 1 --2- 172.21.15.27:0/3102892687 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5d9406e9b0 0x7f5d9415a2f0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f5d9000c430 tx=0x7f5d9000c8f0 comp rx=0 tx=0).stop 2024-09-12T03:20:11.519 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:11.517+0000 7f5d9ba8d640 1 -- 172.21.15.27:0/3102892687 shutdown_connections 2024-09-12T03:20:11.519 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:11.517+0000 7f5d9ba8d640 1 --2- 172.21.15.27:0/3102892687 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f5d7806c190 0x7f5d7806e650 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:11.519 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:11.517+0000 7f5d9ba8d640 1 --2- 172.21.15.27:0/3102892687 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f5d9415b2b0 0x7f5d940aee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:11.519 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:11.517+0000 7f5d9ba8d640 1 --2- 172.21.15.27:0/3102892687 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5d94072d70 0x7f5d940ae8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:11.519 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:11.517+0000 7f5d9ba8d640 1 --2- 172.21.15.27:0/3102892687 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5d9406e9b0 0x7f5d9415a2f0 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:11.519 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:11.517+0000 7f5d9ba8d640 1 -- 172.21.15.27:0/3102892687 >> 172.21.15.27:0/3102892687 conn(0x7f5d94062c60 msgr2=0x7f5d94157700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:11.519 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:11.517+0000 7f5d9ba8d640 1 -- 172.21.15.27:0/3102892687 shutdown_connections 2024-09-12T03:20:11.519 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:11.517+0000 7f5d9ba8d640 1 -- 172.21.15.27:0/3102892687 wait complete. 2024-09-12T03:20:11.529 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-12T03:20:11.529 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:20:11.529 DEBUG:teuthology.orchestra.run.smithi027:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-12T03:20:11.560 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi027.front.sepia.ceph.com 2024-09-12T03:20:11.560 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:20:11.560 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-12T03:20:11.616 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-12T03:20:11.616 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T03:20:11.637 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-12T03:20:11.637 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T03:20:11.699 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-12T03:20:11.803 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:11.802+0000 7f595e5e8640 1 Processor -- start 2024-09-12T03:20:11.803 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:11.802+0000 7f595e5e8640 1 -- start start 2024-09-12T03:20:11.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:11.803+0000 7f595e5e8640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f59580b1ca0 0x7f595815f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:11.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:11.803+0000 7f595e5e8640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f595806f360 0x7f59580cef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:11.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:11.803+0000 7f595e5e8640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f595815fd70 con 0x7f59580cf490 2024-09-12T03:20:11.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:11.803+0000 7f595e5e8640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f59580d22f0 con 0x7f59580b1ca0 2024-09-12T03:20:11.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:11.803+0000 7f595e5e8640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f595806cc40 con 0x7f595806f360 2024-09-12T03:20:11.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:11.803+0000 7f59577fe640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f595806f360 0x7f59580cef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:11.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:11.803+0000 7f595cb85640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f59580cf490 0x7f59580d2780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.27:52288/0 (socket says 172.21.15.27:52288) 2024-09-12T03:20:11.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:11.803+0000 7f595cb85640 1 -- 172.21.15.27:0/815355900 learned_addr learned my addr 172.21.15.27:0/815355900 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:20:11.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:11.803+0000 7f5957fff640 1 --2- 172.21.15.27:0/815355900 >> v2:172.21.15.45:3300/0 conn(0x7f59580b1ca0 0x7f595815f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:11.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:11.804+0000 7f5956ffd640 1 -- 172.21.15.27:0/815355900 <== mon.1 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3172072412 0 0) 0x7f5950001490 con 0x7f59580cf490 2024-09-12T03:20:11.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:11.804+0000 7f5956ffd640 1 -- 172.21.15.27:0/815355900 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5940003520 con 0x7f59580cf490 2024-09-12T03:20:11.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:11.804+0000 7f59577fe640 1 -- 172.21.15.27:0/815355900 >> v2:172.21.15.45:3300/0 conn(0x7f59580b1ca0 msgr2=0x7f595815f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:11.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:11.804+0000 7f59577fe640 1 --2- 172.21.15.27:0/815355900 >> v2:172.21.15.45:3300/0 conn(0x7f59580b1ca0 0x7f595815f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:11.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:11.804+0000 7f59577fe640 1 -- 172.21.15.27:0/815355900 >> v1:172.21.15.45:6789/0 conn(0x7f59580cf490 legacy=0x7f59580d2780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:11.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:11.804+0000 7f59577fe640 1 -- 172.21.15.27:0/815355900 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59580d1ce0 con 0x7f595806f360 2024-09-12T03:20:11.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:11.805+0000 7f59577fe640 1 --2- 172.21.15.27:0/815355900 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f595806f360 0x7f59580cef50 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f594c0097a0 tx=0x7f594c02a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1436e08c6cc52480 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:11.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:11.805+0000 7f5956ffd640 1 -- 172.21.15.27:0/815355900 <== mon.2 v2:172.21.15.45:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f594c037070 con 0x7f595806f360 2024-09-12T03:20:11.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:11.805+0000 7f5956ffd640 1 -- 172.21.15.27:0/815355900 <== mon.2 v2:172.21.15.45:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f594c032440 con 0x7f595806f360 2024-09-12T03:20:11.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:11.805+0000 7f5956ffd640 1 -- 172.21.15.27:0/815355900 <== mon.2 v2:172.21.15.45:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f594c0327f0 con 0x7f595806f360 2024-09-12T03:20:11.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:11.805+0000 7f595e5e8640 1 -- 172.21.15.27:0/815355900 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f595806f360 msgr2=0x7f59580cef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:11.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:11.805+0000 7f595e5e8640 1 --2- 172.21.15.27:0/815355900 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f595806f360 0x7f59580cef50 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f594c0097a0 tx=0x7f594c02a180 comp rx=0 tx=0).stop 2024-09-12T03:20:11.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:11.805+0000 7f595e5e8640 1 -- 172.21.15.27:0/815355900 shutdown_connections 2024-09-12T03:20:11.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:11.805+0000 7f595e5e8640 1 --2- 172.21.15.27:0/815355900 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f595806f360 0x7f59580cef50 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:11.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:11.805+0000 7f595e5e8640 1 --2- 172.21.15.27:0/815355900 >> v2:172.21.15.45:3300/0 conn(0x7f59580b1ca0 0x7f595815f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:11.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:11.805+0000 7f595e5e8640 1 -- 172.21.15.27:0/815355900 >> 172.21.15.27:0/815355900 conn(0x7f5958073800 msgr2=0x7f595809ba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:11.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:11.805+0000 7f595e5e8640 1 -- 172.21.15.27:0/815355900 shutdown_connections 2024-09-12T03:20:11.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:11.805+0000 7f595e5e8640 1 -- 172.21.15.27:0/815355900 wait complete. 2024-09-12T03:20:11.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:11.807+0000 7f595e5e8640 1 Processor -- start 2024-09-12T03:20:11.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:11.807+0000 7f595e5e8640 1 -- start start 2024-09-12T03:20:11.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:11.807+0000 7f595e5e8640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f595806f360 0x7f5958154f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:11.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:11.807+0000 7f595e5e8640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f59580b1ca0 0x7f595809cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:11.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:11.807+0000 7f595e5e8640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f59580cf490 0x7f59580b6520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:11.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:11.807+0000 7f595e5e8640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f595806d1c0 con 0x7f59580cf490 2024-09-12T03:20:11.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:11.807+0000 7f595e5e8640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f59580d1f80 con 0x7f59580b1ca0 2024-09-12T03:20:11.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:11.807+0000 7f595e5e8640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f595806d340 con 0x7f595806f360 2024-09-12T03:20:11.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:11.808+0000 7f595cb85640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f59580cf490 0x7f59580b6520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:11.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:11.808+0000 7f595cb85640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f59580cf490 0x7f59580b6520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57836/0 (socket says 172.21.15.27:57836) 2024-09-12T03:20:11.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:11.808+0000 7f595cb85640 1 -- 172.21.15.27:0/1881681330 learned_addr learned my addr 172.21.15.27:0/1881681330 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:20:11.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:11.808+0000 7f5957fff640 1 --2- 172.21.15.27:0/1881681330 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f595806f360 0x7f5958154f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:11.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:11.808+0000 7f59577fe640 1 --2- 172.21.15.27:0/1881681330 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f59580b1ca0 0x7f595809cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:11.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:11.808+0000 7f595cb85640 1 -- 172.21.15.27:0/1881681330 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f595806f360 msgr2=0x7f5958154f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:11.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:11.808+0000 7f595cb85640 1 --2- 172.21.15.27:0/1881681330 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f595806f360 0x7f5958154f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:11.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:11.808+0000 7f595cb85640 1 -- 172.21.15.27:0/1881681330 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f59580b1ca0 msgr2=0x7f595809cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:11.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:11.808+0000 7f595cb85640 1 --2- 172.21.15.27:0/1881681330 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f59580b1ca0 0x7f595809cae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:11.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:11.808+0000 7f595cb85640 1 -- 172.21.15.27:0/1881681330 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f595809f6b0 con 0x7f59580cf490 2024-09-12T03:20:11.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:11.808+0000 7f595cb85640 1 --2- 172.21.15.27:0/1881681330 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f59580cf490 0x7f59580b6520 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f595000cb10 tx=0x7f595000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:11.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:11.808+0000 7f5954ff9640 1 -- 172.21.15.27:0/1881681330 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5950017070 con 0x7f59580cf490 2024-09-12T03:20:11.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:11.808+0000 7f5954ff9640 1 -- 172.21.15.27:0/1881681330 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f595000ac70 con 0x7f59580cf490 2024-09-12T03:20:11.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:11.808+0000 7f595e5e8640 1 -- 172.21.15.27:0/1881681330 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f595809f990 con 0x7f59580cf490 2024-09-12T03:20:11.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:11.808+0000 7f5954ff9640 1 -- 172.21.15.27:0/1881681330 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5950012640 con 0x7f59580cf490 2024-09-12T03:20:11.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:11.808+0000 7f595e5e8640 1 -- 172.21.15.27:0/1881681330 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f595809ff50 con 0x7f59580cf490 2024-09-12T03:20:11.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:11.809+0000 7f595e5e8640 1 -- 172.21.15.27:0/1881681330 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59580cfc40 con 0x7f59580cf490 2024-09-12T03:20:11.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:11.810+0000 7f5954ff9640 1 -- 172.21.15.27:0/1881681330 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 11) ==== 90804+0+0 (secure 0 0 0) 0x7f595001a070 con 0x7f59580cf490 2024-09-12T03:20:11.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:11.810+0000 7f5954ff9640 1 --2- 172.21.15.27:0/1881681330 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f594406c190 0x7f594406e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:11.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:11.810+0000 7f5954ff9640 1 -- 172.21.15.27:0/1881681330 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 5709+0+0 (secure 0 0 0) 0x7f595008b040 con 0x7f59580cf490 2024-09-12T03:20:11.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:11.811+0000 7f5957fff640 1 --2- 172.21.15.27:0/1881681330 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f594406c190 0x7f594406e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:11.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:11.811+0000 7f5957fff640 1 --2- 172.21.15.27:0/1881681330 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f594406c190 0x7f594406e650 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f5948009b40 tx=0x7f5948009190 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:11.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:11.815+0000 7f5954ff9640 1 -- 172.21.15.27:0/1881681330 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f595815fd70 con 0x7f59580cf490 2024-09-12T03:20:12.003 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:12.002+0000 7f595e5e8640 1 -- 172.21.15.27:0/1881681330 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f595809f340 con 0x7f59580cf490 2024-09-12T03:20:12.004 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:12.003+0000 7f5954ff9640 1 -- 172.21.15.27:0/1881681330 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v67) ==== 80+0+0 (secure 0 0 0) 0x7f595000ae20 con 0x7f59580cf490 2024-09-12T03:20:12.004 INFO:teuthology.orchestra.run.smithi027.stderr:listed 0 entries 2024-09-12T03:20:12.006 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:12.005+0000 7f595e5e8640 1 -- 172.21.15.27:0/1881681330 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f594406c190 msgr2=0x7f594406e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:12.006 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:12.005+0000 7f595e5e8640 1 --2- 172.21.15.27:0/1881681330 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f594406c190 0x7f594406e650 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f5948009b40 tx=0x7f5948009190 comp rx=0 tx=0).stop 2024-09-12T03:20:12.006 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:12.005+0000 7f595e5e8640 1 -- 172.21.15.27:0/1881681330 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f59580cf490 msgr2=0x7f59580b6520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:12.006 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:12.005+0000 7f595e5e8640 1 --2- 172.21.15.27:0/1881681330 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f59580cf490 0x7f59580b6520 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f595000cb10 tx=0x7f595000a060 comp rx=0 tx=0).stop 2024-09-12T03:20:12.006 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:12.005+0000 7f595e5e8640 1 -- 172.21.15.27:0/1881681330 shutdown_connections 2024-09-12T03:20:12.007 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:12.005+0000 7f595e5e8640 1 --2- 172.21.15.27:0/1881681330 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f594406c190 0x7f594406e650 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:12.007 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:12.005+0000 7f595e5e8640 1 --2- 172.21.15.27:0/1881681330 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f59580cf490 0x7f59580b6520 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:12.007 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:12.005+0000 7f595e5e8640 1 --2- 172.21.15.27:0/1881681330 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f59580b1ca0 0x7f595809cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:12.007 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:12.005+0000 7f595e5e8640 1 --2- 172.21.15.27:0/1881681330 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f595806f360 0x7f5958154f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:12.007 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:12.005+0000 7f595e5e8640 1 -- 172.21.15.27:0/1881681330 >> 172.21.15.27:0/1881681330 conn(0x7f5958073800 msgr2=0x7f59581605d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:12.007 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:12.005+0000 7f595e5e8640 1 -- 172.21.15.27:0/1881681330 shutdown_connections 2024-09-12T03:20:12.007 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:12.005+0000 7f595e5e8640 1 -- 172.21.15.27:0/1881681330 wait complete. 2024-09-12T03:20:12.017 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-09-12T03:20:12.017 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:20:12.017 DEBUG:teuthology.orchestra.run.smithi045:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-09-12T03:20:12.051 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi045.front.sepia.ceph.com 2024-09-12T03:20:12.052 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:20:12.052 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-12T03:20:12.111 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-12T03:20:12.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:12.213+0000 7f787788d640 1 Processor -- start 2024-09-12T03:20:12.215 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:12.214+0000 7f787788d640 1 -- start start 2024-09-12T03:20:12.216 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:12.215+0000 7f787788d640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f787015b2b0 0x7f78700b3560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:12.216 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:12.215+0000 7f787788d640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f7870072d70 0x7f7870072430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:12.216 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:12.215+0000 7f787788d640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f787006e9b0 0x7f787006ee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:12.216 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:12.215+0000 7f787788d640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f78700720a0 con 0x7f787006e9b0 2024-09-12T03:20:12.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:12.215+0000 7f787788d640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f78700ce410 con 0x7f7870072d70 2024-09-12T03:20:12.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:12.215+0000 7f787788d640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f7870072970 con 0x7f787015b2b0 2024-09-12T03:20:12.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:12.215+0000 7f7875e2a640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f787006e9b0 0x7f787006ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:12.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:12.215+0000 7f7875e2a640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f787006e9b0 0x7f787006ee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57844/0 (socket says 172.21.15.27:57844) 2024-09-12T03:20:12.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:12.215+0000 7f7875e2a640 1 -- 172.21.15.27:0/3175258628 learned_addr learned my addr 172.21.15.27:0/3175258628 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:20:12.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:12.215+0000 7f7875629640 1 --2- 172.21.15.27:0/3175258628 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f787015b2b0 0x7f78700b3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:12.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:12.215+0000 7f7875e2a640 1 -- 172.21.15.27:0/3175258628 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f787015b2b0 msgr2=0x7f78700b3560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:12.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:12.215+0000 7f7874e28640 1 --2- 172.21.15.27:0/3175258628 >> v2:172.21.15.45:3300/0 conn(0x7f7870072d70 0x7f7870072430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:12.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:12.215+0000 7f7875e2a640 1 --2- 172.21.15.27:0/3175258628 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f787015b2b0 0x7f78700b3560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:12.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:12.215+0000 7f7875e2a640 1 -- 172.21.15.27:0/3175258628 >> v2:172.21.15.45:3300/0 conn(0x7f7870072d70 msgr2=0x7f7870072430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:12.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:12.215+0000 7f7875e2a640 1 --2- 172.21.15.27:0/3175258628 >> v2:172.21.15.45:3300/0 conn(0x7f7870072d70 0x7f7870072430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:12.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:12.215+0000 7f7875e2a640 1 -- 172.21.15.27:0/3175258628 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78700732c0 con 0x7f787006e9b0 2024-09-12T03:20:12.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:12.215+0000 7f7875629640 1 --2- 172.21.15.27:0/3175258628 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f787015b2b0 0x7f78700b3560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:20:12.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:12.216+0000 7f7875e2a640 1 --2- 172.21.15.27:0/3175258628 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f787006e9b0 0x7f787006ee40 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f787006caa0 tx=0x7f786002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7a87467aa465c2d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:12.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:12.216+0000 7f786ffff640 1 -- 172.21.15.27:0/3175258628 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7860037070 con 0x7f787006e9b0 2024-09-12T03:20:12.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:12.216+0000 7f786ffff640 1 -- 172.21.15.27:0/3175258628 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7860032440 con 0x7f787006e9b0 2024-09-12T03:20:12.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:12.216+0000 7f786ffff640 1 -- 172.21.15.27:0/3175258628 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7860032810 con 0x7f787006e9b0 2024-09-12T03:20:12.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:12.217+0000 7f787788d640 1 -- 172.21.15.27:0/3175258628 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f787006e9b0 msgr2=0x7f787006ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:12.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:12.217+0000 7f787788d640 1 --2- 172.21.15.27:0/3175258628 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f787006e9b0 0x7f787006ee40 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f787006caa0 tx=0x7f786002a1e0 comp rx=0 tx=0).stop 2024-09-12T03:20:12.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:12.217+0000 7f787788d640 1 -- 172.21.15.27:0/3175258628 shutdown_connections 2024-09-12T03:20:12.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:12.217+0000 7f787788d640 1 --2- 172.21.15.27:0/3175258628 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f787006e9b0 0x7f787006ee40 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:12.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:12.217+0000 7f787788d640 1 --2- 172.21.15.27:0/3175258628 >> v2:172.21.15.45:3300/0 conn(0x7f7870072d70 0x7f7870072430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:12.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:12.217+0000 7f787788d640 1 --2- 172.21.15.27:0/3175258628 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f787015b2b0 0x7f78700b3560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:12.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:12.217+0000 7f787788d640 1 -- 172.21.15.27:0/3175258628 >> 172.21.15.27:0/3175258628 conn(0x7f7870062c60 msgr2=0x7f787006bb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:12.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:12.217+0000 7f787788d640 1 -- 172.21.15.27:0/3175258628 shutdown_connections 2024-09-12T03:20:12.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:12.217+0000 7f787788d640 1 -- 172.21.15.27:0/3175258628 wait complete. 2024-09-12T03:20:12.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:12.218+0000 7f787788d640 1 Processor -- start 2024-09-12T03:20:12.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:12.218+0000 7f787788d640 1 -- start start 2024-09-12T03:20:12.222 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:12.219+0000 7f787788d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f787006e9b0 0x7f7870154d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:12.222 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:12.219+0000 7f787788d640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f7870072d70 0x7f787014fe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:12.222 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:12.219+0000 7f787788d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f787015b2b0 0x7f787015b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:12.222 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:12.219+0000 7f787788d640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f78700ce690 con 0x7f787015b2b0 2024-09-12T03:20:12.223 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:12.219+0000 7f787788d640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f7870071dc0 con 0x7f787006e9b0 2024-09-12T03:20:12.223 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:12.219+0000 7f787788d640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f787015b690 con 0x7f7870072d70 2024-09-12T03:20:12.223 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:12.219+0000 7f7875e2a640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f787015b2b0 0x7f787015b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:12.223 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:12.219+0000 7f7875e2a640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f787015b2b0 0x7f787015b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57850/0 (socket says 172.21.15.27:57850) 2024-09-12T03:20:12.223 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:12.219+0000 7f7875e2a640 1 -- 172.21.15.27:0/3673083952 learned_addr learned my addr 172.21.15.27:0/3673083952 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:20:12.223 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:12.219+0000 7f7875629640 1 --2- 172.21.15.27:0/3673083952 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f787006e9b0 0x7f7870154d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:12.223 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:12.219+0000 7f7875e2a640 1 -- 172.21.15.27:0/3673083952 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f7870072d70 msgr2=0x7f787014fe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:12.223 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:12.219+0000 7f7874e28640 1 --2- 172.21.15.27:0/3673083952 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f7870072d70 0x7f787014fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:12.223 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:12.219+0000 7f7875e2a640 1 --2- 172.21.15.27:0/3673083952 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f7870072d70 0x7f787014fe20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:12.223 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:12.219+0000 7f7875e2a640 1 -- 172.21.15.27:0/3673083952 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f787006e9b0 msgr2=0x7f7870154d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:12.223 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:12.219+0000 7f7875e2a640 1 --2- 172.21.15.27:0/3673083952 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f787006e9b0 0x7f7870154d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:12.223 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:12.219+0000 7f7875e2a640 1 -- 172.21.15.27:0/3673083952 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7870150360 con 0x7f787015b2b0 2024-09-12T03:20:12.223 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:12.219+0000 7f7875629640 1 --2- 172.21.15.27:0/3673083952 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f787006e9b0 0x7f7870154d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:20:12.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:12.219+0000 7f7875e2a640 1 --2- 172.21.15.27:0/3673083952 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f787015b2b0 0x7f787015b9d0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f78600090b0 tx=0x7f78600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:12.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:12.219+0000 7f786dffb640 1 -- 172.21.15.27:0/3673083952 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7860037040 con 0x7f787015b2b0 2024-09-12T03:20:12.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:12.219+0000 7f786dffb640 1 -- 172.21.15.27:0/3673083952 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7860042070 con 0x7f787015b2b0 2024-09-12T03:20:12.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:12.219+0000 7f787788d640 1 -- 172.21.15.27:0/3673083952 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f787015c260 con 0x7f787015b2b0 2024-09-12T03:20:12.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:12.219+0000 7f786dffb640 1 -- 172.21.15.27:0/3673083952 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7860030bd0 con 0x7f787015b2b0 2024-09-12T03:20:12.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:12.220+0000 7f787788d640 1 -- 172.21.15.27:0/3673083952 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f787015c820 con 0x7f787015b2b0 2024-09-12T03:20:12.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:12.221+0000 7f787788d640 1 -- 172.21.15.27:0/3673083952 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f787015cff0 con 0x7f787015b2b0 2024-09-12T03:20:12.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:12.221+0000 7f786dffb640 1 -- 172.21.15.27:0/3673083952 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 11) ==== 90804+0+0 (secure 0 0 0) 0x7f7860047460 con 0x7f787015b2b0 2024-09-12T03:20:12.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:12.222+0000 7f786dffb640 1 --2- 172.21.15.27:0/3673083952 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f785c06c190 0x7f785c06e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:12.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:12.222+0000 7f786dffb640 1 -- 172.21.15.27:0/3673083952 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 5709+0+0 (secure 0 0 0) 0x7f78600abfb0 con 0x7f787015b2b0 2024-09-12T03:20:12.225 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:12.222+0000 7f7875629640 1 --2- 172.21.15.27:0/3673083952 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f785c06c190 0x7f785c06e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:12.225 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:12.222+0000 7f7875629640 1 --2- 172.21.15.27:0/3673083952 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f785c06c190 0x7f785c06e650 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f7870154770 tx=0x7f7868007430 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:12.227 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:12.226+0000 7f786dffb640 1 -- 172.21.15.27:0/3673083952 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f786007a550 con 0x7f787015b2b0 2024-09-12T03:20:12.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:12.443+0000 7f787788d640 1 -- 172.21.15.27:0/3673083952 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f78700718a0 con 0x7f787015b2b0 2024-09-12T03:20:12.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:12.444+0000 7f786dffb640 1 -- 172.21.15.27:0/3673083952 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+5225 (secure 0 0 0) 0x7f7860079e50 con 0x7f787015b2b0 2024-09-12T03:20:12.445 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:20:12.445 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":39,"btime":"2024-09-12T03:20:10:420320+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6045,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6834/3058537826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":3058537826},{"type":"v1","addr":"172.21.15.27:6834","nonce":3058537826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":34},{"gid":6048,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/1585628536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":1585628536},{"type":"v1","addr":"172.21.15.27:6835","nonce":1585628536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":34},{"gid":6063,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6836/1421588365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":1421588365},{"type":"v1","addr":"172.21.15.45:6836","nonce":1421588365}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-12T03:19:57.738864+0000","modified":"2024-09-12T03:19:59.740115+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6066},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6066":{"gid":6066,"name":"d","rank":0,"incarnation":36,"state":"up:active","state_seq":3,"addr":"172.21.15.45:6837/2890007436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6835","nonce":2890007436},{"type":"v1","addr":"172.21.15.45:6837","nonce":2890007436}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6066,"qdb_cluster":[6066]},"id":5}]} 2024-09-12T03:20:12.445 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 39 2024-09-12T03:20:12.447 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:12.446+0000 7f787788d640 1 -- 172.21.15.27:0/3673083952 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f785c06c190 msgr2=0x7f785c06e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:12.447 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:12.446+0000 7f787788d640 1 --2- 172.21.15.27:0/3673083952 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f785c06c190 0x7f785c06e650 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f7870154770 tx=0x7f7868007430 comp rx=0 tx=0).stop 2024-09-12T03:20:12.447 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:12.446+0000 7f787788d640 1 -- 172.21.15.27:0/3673083952 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f787015b2b0 msgr2=0x7f787015b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:12.447 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:12.446+0000 7f787788d640 1 --2- 172.21.15.27:0/3673083952 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f787015b2b0 0x7f787015b9d0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f78600090b0 tx=0x7f78600046c0 comp rx=0 tx=0).stop 2024-09-12T03:20:12.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:12.446+0000 7f787788d640 1 -- 172.21.15.27:0/3673083952 shutdown_connections 2024-09-12T03:20:12.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:12.446+0000 7f787788d640 1 --2- 172.21.15.27:0/3673083952 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7f785c06c190 0x7f785c06e650 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:12.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:12.446+0000 7f787788d640 1 --2- 172.21.15.27:0/3673083952 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f787015b2b0 0x7f787015b9d0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:12.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:12.446+0000 7f787788d640 1 --2- 172.21.15.27:0/3673083952 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f7870072d70 0x7f787014fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:12.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:12.446+0000 7f787788d640 1 --2- 172.21.15.27:0/3673083952 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f787006e9b0 0x7f7870154d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:12.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:12.446+0000 7f787788d640 1 -- 172.21.15.27:0/3673083952 >> 172.21.15.27:0/3673083952 conn(0x7f7870062c60 msgr2=0x7f78700b01d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:12.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:12.446+0000 7f787788d640 1 -- 172.21.15.27:0/3673083952 shutdown_connections 2024-09-12T03:20:12.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:12.446+0000 7f787788d640 1 -- 172.21.15.27:0/3673083952 wait complete. 2024-09-12T03:20:12.458 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json session ls 2024-09-12T03:20:12.593 INFO:teuthology.orchestra.run.smithi045.stdout:[] 2024-09-12T03:20:12.593 DEBUG:tasks.cephfs.filesystem:_json_asok output [] 2024-09-12T03:20:12.594 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs subvol56970 2024-09-12T03:20:12.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:12.699+0000 7fbe0118d640 1 Processor -- start 2024-09-12T03:20:12.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:12.699+0000 7fbe0118d640 1 -- start start 2024-09-12T03:20:12.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:12.700+0000 7fbe0118d640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fbdfc0b6230 0x7fbdfc0b66c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:12.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:12.700+0000 7fbe0118d640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fbdfc0b3600 0x7fbdfc0b3a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:12.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:12.700+0000 7fbe0118d640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbdfc15ed70 con 0x7fbdfc15e990 2024-09-12T03:20:12.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:12.700+0000 7fbe0118d640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7fbdfc0b7d10 con 0x7fbdfc0b6230 2024-09-12T03:20:12.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:12.700+0000 7fbe0118d640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fbdfc0d2a30 con 0x7fbdfc0b3600 2024-09-12T03:20:12.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:12.700+0000 7fbdfa59c640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fbdfc0b6230 0x7fbdfc0b66c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:12.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:12.700+0000 7fbdfa59c640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fbdfc0b6230 0x7fbdfc0b66c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57854/0 (socket says 172.21.15.27:57854) 2024-09-12T03:20:12.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:12.700+0000 7fbdfa59c640 1 -- 172.21.15.27:0/290908380 learned_addr learned my addr 172.21.15.27:0/290908380 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:20:12.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:12.700+0000 7fbdfb59e640 1 --2- 172.21.15.27:0/290908380 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fbdfc0b3600 0x7fbdfc0b3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:12.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:12.700+0000 7fbdfa59c640 1 -- 172.21.15.27:0/290908380 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fbdfc0b3600 msgr2=0x7fbdfc0b3a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:12.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:12.700+0000 7fbdfa59c640 1 --2- 172.21.15.27:0/290908380 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fbdfc0b3600 0x7fbdfc0b3a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:12.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:12.700+0000 7fbdfa59c640 1 -- 172.21.15.27:0/290908380 >> v1:172.21.15.45:6789/0 conn(0x7fbdfc15e990 legacy=0x7fbdfc0b7600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:12.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:12.700+0000 7fbdfa59c640 1 -- 172.21.15.27:0/290908380 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdfc0d2d10 con 0x7fbdfc0b6230 2024-09-12T03:20:12.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:12.701+0000 7fbdfa59c640 1 --2- 172.21.15.27:0/290908380 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbdfc0b6230 0x7fbdfc0b66c0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fbdf0009870 tx=0x7fbdf002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2066062392a2954b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:12.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:12.701+0000 7fbdf9d9b640 1 -- 172.21.15.27:0/290908380 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdf0037070 con 0x7fbdfc0b6230 2024-09-12T03:20:12.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:12.701+0000 7fbdf9d9b640 1 -- 172.21.15.27:0/290908380 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbdf0032440 con 0x7fbdfc0b6230 2024-09-12T03:20:12.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:12.701+0000 7fbdf9d9b640 1 -- 172.21.15.27:0/290908380 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdf0032810 con 0x7fbdfc0b6230 2024-09-12T03:20:12.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:12.702+0000 7fbe0118d640 1 -- 172.21.15.27:0/290908380 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbdfc0b6230 msgr2=0x7fbdfc0b66c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:12.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:12.702+0000 7fbe0118d640 1 --2- 172.21.15.27:0/290908380 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbdfc0b6230 0x7fbdfc0b66c0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fbdf0009870 tx=0x7fbdf002a270 comp rx=0 tx=0).stop 2024-09-12T03:20:12.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:12.702+0000 7fbe0118d640 1 -- 172.21.15.27:0/290908380 shutdown_connections 2024-09-12T03:20:12.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:12.702+0000 7fbe0118d640 1 --2- 172.21.15.27:0/290908380 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fbdfc0b3600 0x7fbdfc0b3a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:12.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:12.702+0000 7fbe0118d640 1 --2- 172.21.15.27:0/290908380 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbdfc0b6230 0x7fbdfc0b66c0 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:12.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:12.702+0000 7fbe0118d640 1 -- 172.21.15.27:0/290908380 >> 172.21.15.27:0/290908380 conn(0x7fbdfc0d1230 msgr2=0x7fbdfc0d1640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:12.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:12.702+0000 7fbe0118d640 1 -- 172.21.15.27:0/290908380 shutdown_connections 2024-09-12T03:20:12.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:12.702+0000 7fbe0118d640 1 -- 172.21.15.27:0/290908380 wait complete. 2024-09-12T03:20:12.708 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:12.703+0000 7fbe0118d640 1 Processor -- start 2024-09-12T03:20:12.708 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:12.703+0000 7fbe0118d640 1 -- start start 2024-09-12T03:20:12.708 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:12.704+0000 7fbe0118d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbdfc0b3600 0x7fbdfc1f8350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:12.708 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:12.704+0000 7fbe0118d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbdfc0b6230 0x7fbdfc15f990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:12.708 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:12.704+0000 7fbe0118d640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fbdfc15e990 0x7fbdfc0b57c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:12.708 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:12.704+0000 7fbe0118d640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fbdfc0d35b0 con 0x7fbdfc0b3600 2024-09-12T03:20:12.708 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:12.704+0000 7fbe0118d640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fbdfc0d32d0 con 0x7fbdfc0b6230 2024-09-12T03:20:12.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:12.704+0000 7fbe0118d640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fbdfc0b3f90 con 0x7fbdfc15e990 2024-09-12T03:20:12.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:12.704+0000 7fbdfad9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbdfc0b3600 0x7fbdfc1f8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:12.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:12.704+0000 7fbdfad9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbdfc0b3600 0x7fbdfc1f8350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57870/0 (socket says 172.21.15.27:57870) 2024-09-12T03:20:12.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:12.704+0000 7fbdfad9d640 1 -- 172.21.15.27:0/1429157299 learned_addr learned my addr 172.21.15.27:0/1429157299 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:20:12.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:12.704+0000 7fbdfa59c640 1 --2- 172.21.15.27:0/1429157299 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbdfc0b6230 0x7fbdfc15f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:12.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:12.704+0000 7fbdfb59e640 1 --2- 172.21.15.27:0/1429157299 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fbdfc15e990 0x7fbdfc0b57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:12.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:12.704+0000 7fbdfad9d640 1 -- 172.21.15.27:0/1429157299 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fbdfc15e990 msgr2=0x7fbdfc0b57c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:12.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:12.704+0000 7fbdfad9d640 1 --2- 172.21.15.27:0/1429157299 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fbdfc15e990 0x7fbdfc0b57c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:12.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:12.704+0000 7fbdfad9d640 1 -- 172.21.15.27:0/1429157299 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbdfc0b6230 msgr2=0x7fbdfc15f990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:12.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:12.704+0000 7fbdfad9d640 1 --2- 172.21.15.27:0/1429157299 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbdfc0b6230 0x7fbdfc15f990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:12.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:12.704+0000 7fbdfad9d640 1 -- 172.21.15.27:0/1429157299 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdfc15fed0 con 0x7fbdfc0b3600 2024-09-12T03:20:12.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:12.705+0000 7fbdfad9d640 1 --2- 172.21.15.27:0/1429157299 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbdfc0b3600 0x7fbdfc1f8350 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fbdec00c3c0 tx=0x7fbdec00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:12.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:12.705+0000 7fbde37fe640 1 -- 172.21.15.27:0/1429157299 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdec0052d0 con 0x7fbdfc0b3600 2024-09-12T03:20:12.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:12.705+0000 7fbde37fe640 1 -- 172.21.15.27:0/1429157299 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbdec005490 con 0x7fbdfc0b3600 2024-09-12T03:20:12.710 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:12.705+0000 7fbe0118d640 1 -- 172.21.15.27:0/1429157299 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbdfc1618c0 con 0x7fbdfc0b3600 2024-09-12T03:20:12.710 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:12.705+0000 7fbde37fe640 1 -- 172.21.15.27:0/1429157299 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdec007260 con 0x7fbdfc0b3600 2024-09-12T03:20:12.710 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:12.705+0000 7fbe0118d640 1 -- 172.21.15.27:0/1429157299 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbdfc15f190 con 0x7fbdfc0b3600 2024-09-12T03:20:12.710 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:12.706+0000 7fbe0118d640 1 -- 172.21.15.27:0/1429157299 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbdfc0b3a50 con 0x7fbdfc0b3600 2024-09-12T03:20:12.710 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:12.706+0000 7fbde37fe640 1 -- 172.21.15.27:0/1429157299 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 11) ==== 90804+0+0 (secure 0 0 0) 0x7fbdec007400 con 0x7fbdfc0b3600 2024-09-12T03:20:12.710 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:12.707+0000 7fbde37fe640 1 --2- 172.21.15.27:0/1429157299 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fbde806c190 0x7fbde806e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:12.710 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:12.707+0000 7fbde37fe640 1 -- 172.21.15.27:0/1429157299 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 5970+0+0 (secure 0 0 0) 0x7fbdec08a8e0 con 0x7fbdfc0b3600 2024-09-12T03:20:12.710 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:12.707+0000 7fbdfa59c640 1 --2- 172.21.15.27:0/1429157299 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fbde806c190 0x7fbde806e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:12.710 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:12.707+0000 7fbdfa59c640 1 --2- 172.21.15.27:0/1429157299 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fbde806c190 0x7fbde806e650 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fbdf0034000 tx=0x7fbdf002e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:12.712 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:12.711+0000 7fbde37fe640 1 -- 172.21.15.27:0/1429157299 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fbdec058720 con 0x7fbdfc0b3600 2024-09-12T03:20:12.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:12.913+0000 7fbe0118d640 1 -- 172.21.15.27:0/1429157299 --> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvol56970", "target": ["mon-mgr", ""]}) -- 0x7fbdfc0ba510 con 0x7fbde806c190 2024-09-12T03:20:12.920 INFO:tasks.ceph.mgr.x.smithi045.stderr:2024-09-12T03:20:12.920+0000 7fc16ffad640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:20:12.920 INFO:tasks.ceph.mgr.x.smithi045.stderr:2024-09-12T03:20:12.920+0000 7fc16ffad640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:20:12.920 INFO:tasks.ceph.mgr.x.smithi045.stderr:2024-09-12T03:20:12.920+0000 7fc16ffad640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:20:12.920 INFO:tasks.ceph.mgr.x.smithi045.stderr:2024-09-12T03:20:12.920+0000 7fc16ffad640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:20:12.920 INFO:tasks.ceph.mgr.x.smithi045.stderr:2024-09-12T03:20:12.920+0000 7fc16ffad640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:20:12.951 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:12.950+0000 7fbde37fe640 1 -- 172.21.15.27:0/1429157299 <== mgr.4102 v2:172.21.15.45:6832/3923821219 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fbdfc0ba510 con 0x7fbde806c190 2024-09-12T03:20:12.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:12.952+0000 7fbe0118d640 1 -- 172.21.15.27:0/1429157299 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fbde806c190 msgr2=0x7fbde806e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:12.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:12.952+0000 7fbe0118d640 1 --2- 172.21.15.27:0/1429157299 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fbde806c190 0x7fbde806e650 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fbdf0034000 tx=0x7fbdf002e000 comp rx=0 tx=0).stop 2024-09-12T03:20:12.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:12.952+0000 7fbe0118d640 1 -- 172.21.15.27:0/1429157299 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbdfc0b3600 msgr2=0x7fbdfc1f8350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:12.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:12.952+0000 7fbe0118d640 1 --2- 172.21.15.27:0/1429157299 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbdfc0b3600 0x7fbdfc1f8350 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fbdec00c3c0 tx=0x7fbdec00c880 comp rx=0 tx=0).stop 2024-09-12T03:20:12.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:12.952+0000 7fbe0118d640 1 -- 172.21.15.27:0/1429157299 shutdown_connections 2024-09-12T03:20:12.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:12.952+0000 7fbe0118d640 1 --2- 172.21.15.27:0/1429157299 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fbde806c190 0x7fbde806e650 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:12.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:12.952+0000 7fbe0118d640 1 --2- 172.21.15.27:0/1429157299 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fbdfc15e990 0x7fbdfc0b57c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:12.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:12.952+0000 7fbe0118d640 1 --2- 172.21.15.27:0/1429157299 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbdfc0b6230 0x7fbdfc15f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:12.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:12.952+0000 7fbe0118d640 1 --2- 172.21.15.27:0/1429157299 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbdfc0b3600 0x7fbdfc1f8350 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:12.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:12.952+0000 7fbe0118d640 1 -- 172.21.15.27:0/1429157299 >> 172.21.15.27:0/1429157299 conn(0x7fbdfc0d1230 msgr2=0x7fbdfc06d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:12.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:12.952+0000 7fbe0118d640 1 -- 172.21.15.27:0/1429157299 shutdown_connections 2024-09-12T03:20:12.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:12.952+0000 7fbe0118d640 1 -- 172.21.15.27:0/1429157299 wait complete. 2024-09-12T03:20:12.965 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-12T03:20:13.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:13.065+0000 7fa50cee6640 1 Processor -- start 2024-09-12T03:20:13.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:13.065+0000 7fa50cee6640 1 -- start start 2024-09-12T03:20:13.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:13.066+0000 7fa50cee6640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fa50815eb90 0x7fa5080d2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:13.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:13.066+0000 7fa50cee6640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa5080d5200 0x7fa5080d1010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:13.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:13.066+0000 7fa50cee6640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa5080d19c0 con 0x7fa5080d1550 2024-09-12T03:20:13.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:13.066+0000 7fa50cee6640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7fa5080b7d20 con 0x7fa50815eb90 2024-09-12T03:20:13.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:13.066+0000 7fa50cee6640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fa5080d42e0 con 0x7fa5080d5200 2024-09-12T03:20:13.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:13.067+0000 7fa50659c640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fa50815eb90 0x7fa5080d2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:13.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:13.067+0000 7fa50659c640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fa50815eb90 0x7fa5080d2df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57876/0 (socket says 172.21.15.27:57876) 2024-09-12T03:20:13.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:13.067+0000 7fa50659c640 1 -- 172.21.15.27:0/3575558427 learned_addr learned my addr 172.21.15.27:0/3575558427 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:20:13.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:13.067+0000 7fa505d9b640 1 --2- 172.21.15.27:0/3575558427 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa5080d5200 0x7fa5080d1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:13.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:13.067+0000 7fa50659c640 1 -- 172.21.15.27:0/3575558427 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa5080d5200 msgr2=0x7fa5080d1010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:13.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:13.067+0000 7fa50659c640 1 --2- 172.21.15.27:0/3575558427 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa5080d5200 0x7fa5080d1010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:13.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:13.067+0000 7fa50659c640 1 -- 172.21.15.27:0/3575558427 >> v1:172.21.15.45:6789/0 conn(0x7fa5080d1550 legacy=0x7fa5080cf200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:13.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:13.067+0000 7fa50659c640 1 -- 172.21.15.27:0/3575558427 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5080d3760 con 0x7fa50815eb90 2024-09-12T03:20:13.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:13.067+0000 7fa505d9b640 1 --2- 172.21.15.27:0/3575558427 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa5080d5200 0x7fa5080d1010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:20:13.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:13.068+0000 7fa50659c640 1 --2- 172.21.15.27:0/3575558427 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa50815eb90 0x7fa5080d2df0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fa4f8009870 tx=0x7fa4f802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2dc9761153d29b2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:13.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:13.068+0000 7fa50559a640 1 -- 172.21.15.27:0/3575558427 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4f8037070 con 0x7fa50815eb90 2024-09-12T03:20:13.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:13.068+0000 7fa50559a640 1 -- 172.21.15.27:0/3575558427 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4f8032440 con 0x7fa50815eb90 2024-09-12T03:20:13.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:13.068+0000 7fa50559a640 1 -- 172.21.15.27:0/3575558427 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4f8032810 con 0x7fa50815eb90 2024-09-12T03:20:13.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:13.068+0000 7fa50cee6640 1 -- 172.21.15.27:0/3575558427 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa50815eb90 msgr2=0x7fa5080d2df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:13.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:13.068+0000 7fa50cee6640 1 --2- 172.21.15.27:0/3575558427 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa50815eb90 0x7fa5080d2df0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fa4f8009870 tx=0x7fa4f802a270 comp rx=0 tx=0).stop 2024-09-12T03:20:13.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:13.068+0000 7fa50cee6640 1 -- 172.21.15.27:0/3575558427 shutdown_connections 2024-09-12T03:20:13.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:13.068+0000 7fa50cee6640 1 --2- 172.21.15.27:0/3575558427 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa5080d5200 0x7fa5080d1010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:13.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:13.068+0000 7fa50cee6640 1 --2- 172.21.15.27:0/3575558427 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa50815eb90 0x7fa5080d2df0 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:13.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:13.068+0000 7fa50cee6640 1 -- 172.21.15.27:0/3575558427 >> 172.21.15.27:0/3575558427 conn(0x7fa50806dce0 msgr2=0x7fa50806e0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:13.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:13.068+0000 7fa50cee6640 1 -- 172.21.15.27:0/3575558427 shutdown_connections 2024-09-12T03:20:13.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:13.069+0000 7fa50cee6640 1 -- 172.21.15.27:0/3575558427 wait complete. 2024-09-12T03:20:13.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:13.070+0000 7fa50cee6640 1 Processor -- start 2024-09-12T03:20:13.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:13.070+0000 7fa50cee6640 1 -- start start 2024-09-12T03:20:13.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:13.071+0000 7fa50cee6640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa5080d1550 0x7fa5080cdef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:13.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:13.071+0000 7fa50cee6640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa5080d5200 0x7fa508149c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:13.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:13.071+0000 7fa50cee6640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa50815eb90 0x7fa508158fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:13.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:13.071+0000 7fa50cee6640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fa5080d4000 con 0x7fa5080d1550 2024-09-12T03:20:13.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:13.071+0000 7fa50cee6640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa5080d3d20 con 0x7fa50815eb90 2024-09-12T03:20:13.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:13.071+0000 7fa50cee6640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fa5080d3330 con 0x7fa5080d5200 2024-09-12T03:20:13.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:13.071+0000 7fa50659c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa5080d1550 0x7fa5080cdef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:13.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:13.071+0000 7fa50659c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa5080d1550 0x7fa5080cdef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57880/0 (socket says 172.21.15.27:57880) 2024-09-12T03:20:13.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:13.071+0000 7fa50659c640 1 -- 172.21.15.27:0/511272715 learned_addr learned my addr 172.21.15.27:0/511272715 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:20:13.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:13.071+0000 7fa505d9b640 1 --2- 172.21.15.27:0/511272715 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa5080d5200 0x7fa508149c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:13.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:13.071+0000 7fa506d9d640 1 --2- 172.21.15.27:0/511272715 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa50815eb90 0x7fa508158fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:13.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:13.071+0000 7fa50659c640 1 -- 172.21.15.27:0/511272715 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa5080d5200 msgr2=0x7fa508149c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:13.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:13.071+0000 7fa50659c640 1 --2- 172.21.15.27:0/511272715 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa5080d5200 0x7fa508149c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:13.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:13.071+0000 7fa50659c640 1 -- 172.21.15.27:0/511272715 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa50815eb90 msgr2=0x7fa508158fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:13.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:13.071+0000 7fa50659c640 1 --2- 172.21.15.27:0/511272715 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa50815eb90 0x7fa508158fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:13.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:13.071+0000 7fa50659c640 1 -- 172.21.15.27:0/511272715 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa50814a140 con 0x7fa5080d1550 2024-09-12T03:20:13.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:13.071+0000 7fa50659c640 1 --2- 172.21.15.27:0/511272715 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa5080d1550 0x7fa5080cdef0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fa4f8000c00 tx=0x7fa4f80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:13.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:13.071+0000 7fa4f6ffd640 1 -- 172.21.15.27:0/511272715 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4f8041020 con 0x7fa5080d1550 2024-09-12T03:20:13.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:13.071+0000 7fa4f6ffd640 1 -- 172.21.15.27:0/511272715 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4f80306c0 con 0x7fa5080d1550 2024-09-12T03:20:13.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:13.071+0000 7fa50cee6640 1 -- 172.21.15.27:0/511272715 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa508159500 con 0x7fa5080d1550 2024-09-12T03:20:13.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:13.071+0000 7fa4f6ffd640 1 -- 172.21.15.27:0/511272715 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4f8037040 con 0x7fa5080d1550 2024-09-12T03:20:13.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:13.071+0000 7fa50cee6640 1 -- 172.21.15.27:0/511272715 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa508159a80 con 0x7fa5080d1550 2024-09-12T03:20:13.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:13.072+0000 7fa50cee6640 1 -- 172.21.15.27:0/511272715 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa50815ef70 con 0x7fa5080d1550 2024-09-12T03:20:13.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:13.073+0000 7fa4f6ffd640 1 -- 172.21.15.27:0/511272715 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 11) ==== 90804+0+0 (secure 0 0 0) 0x7fa4f8044020 con 0x7fa5080d1550 2024-09-12T03:20:13.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:13.073+0000 7fa4f6ffd640 1 --2- 172.21.15.27:0/511272715 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fa4e806c190 0x7fa4e806e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:13.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:13.073+0000 7fa4f6ffd640 1 -- 172.21.15.27:0/511272715 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 5970+0+0 (secure 0 0 0) 0x7fa4f80abb10 con 0x7fa5080d1550 2024-09-12T03:20:13.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:13.073+0000 7fa505d9b640 1 --2- 172.21.15.27:0/511272715 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fa4e806c190 0x7fa4e806e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:13.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:13.074+0000 7fa505d9b640 1 --2- 172.21.15.27:0/511272715 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fa4e806c190 0x7fa4e806e650 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fa5080ce630 tx=0x7fa4fc008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:13.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:13.078+0000 7fa4f6ffd640 1 -- 172.21.15.27:0/511272715 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fa4f804b030 con 0x7fa5080d1550 2024-09-12T03:20:13.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:13.291+0000 7fa50cee6640 1 -- 172.21.15.27:0/511272715 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa5081621e0 con 0x7fa5080d1550 2024-09-12T03:20:13.293 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:13.292+0000 7fa4f6ffd640 1 -- 172.21.15.27:0/511272715 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+5225 (secure 0 0 0) 0x7fa50814a140 con 0x7fa5080d1550 2024-09-12T03:20:13.293 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:20:13.293 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":39,"btime":"2024-09-12T03:20:10:420320+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6045,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6834/3058537826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":3058537826},{"type":"v1","addr":"172.21.15.27:6834","nonce":3058537826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":34},{"gid":6048,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/1585628536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":1585628536},{"type":"v1","addr":"172.21.15.27:6835","nonce":1585628536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":34},{"gid":6063,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6836/1421588365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":1421588365},{"type":"v1","addr":"172.21.15.45:6836","nonce":1421588365}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-12T03:19:57.738864+0000","modified":"2024-09-12T03:19:59.740115+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6066},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6066":{"gid":6066,"name":"d","rank":0,"incarnation":36,"state":"up:active","state_seq":3,"addr":"172.21.15.45:6837/2890007436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6835","nonce":2890007436},{"type":"v1","addr":"172.21.15.45:6837","nonce":2890007436}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6066,"qdb_cluster":[6066]},"id":5}]} 2024-09-12T03:20:13.293 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 39 2024-09-12T03:20:13.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:13.294+0000 7fa50cee6640 1 -- 172.21.15.27:0/511272715 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fa4e806c190 msgr2=0x7fa4e806e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:13.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:13.294+0000 7fa50cee6640 1 --2- 172.21.15.27:0/511272715 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fa4e806c190 0x7fa4e806e650 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fa5080ce630 tx=0x7fa4fc008040 comp rx=0 tx=0).stop 2024-09-12T03:20:13.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:13.294+0000 7fa50cee6640 1 -- 172.21.15.27:0/511272715 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa5080d1550 msgr2=0x7fa5080cdef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:13.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:13.294+0000 7fa50cee6640 1 --2- 172.21.15.27:0/511272715 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa5080d1550 0x7fa5080cdef0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fa4f8000c00 tx=0x7fa4f80046c0 comp rx=0 tx=0).stop 2024-09-12T03:20:13.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:13.294+0000 7fa50cee6640 1 -- 172.21.15.27:0/511272715 shutdown_connections 2024-09-12T03:20:13.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:13.294+0000 7fa50cee6640 1 --2- 172.21.15.27:0/511272715 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fa4e806c190 0x7fa4e806e650 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:13.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:13.294+0000 7fa50cee6640 1 --2- 172.21.15.27:0/511272715 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa50815eb90 0x7fa508158fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:13.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:13.294+0000 7fa50cee6640 1 --2- 172.21.15.27:0/511272715 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa5080d5200 0x7fa508149c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:13.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:13.294+0000 7fa50cee6640 1 --2- 172.21.15.27:0/511272715 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa5080d1550 0x7fa5080cdef0 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:13.296 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:13.294+0000 7fa50cee6640 1 -- 172.21.15.27:0/511272715 >> 172.21.15.27:0/511272715 conn(0x7fa50806dce0 msgr2=0x7fa5080cfd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:13.296 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:13.294+0000 7fa50cee6640 1 -- 172.21.15.27:0/511272715 shutdown_connections 2024-09-12T03:20:13.296 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:13.294+0000 7fa50cee6640 1 -- 172.21.15.27:0/511272715 wait complete. 2024-09-12T03:20:13.306 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json session ls 2024-09-12T03:20:13.447 INFO:teuthology.orchestra.run.smithi045.stdout:[{"id":5920,"entity":{"name":{"type":"client","num":5920},"addr":{"type":"any","addr":"172.21.15.45:0","nonce":761029391}},"state":"open","num_leases":1,"num_caps":7,"request_load_avg":14,"uptime":0.51199244200000005,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":21.974574270563117,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5920 172.21.15.45:0/761029391","completed_requests":[{"tid":14,"created_ino":"0x0"},{"tid":15,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc","ceph_version":"ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)","entity_id":"x","hostname":"smithi045","pid":"36189","root":"/"}}] 2024-09-12T03:20:13.448 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc", "ceph_version": "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "x", "hostname": "smithi045", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "36189", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 14 }, { "created_ino": "0x100000001f9", "tid": 15 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.45:0", "nonce": 761029391, "type": "any" }, "name": { "num": 5920, "type": "client" } }, "id": 5920, "inst": "client.5920 172.21.15.45:0/761029391", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 1, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 14, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 21.974574270563117 }, "state": "open", "uptime": 0.511992442 } ] 2024-09-12T03:20:13.449 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-09-12T03:20:13.552 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:13.551+0000 7fa178d59640 1 Processor -- start 2024-09-12T03:20:13.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:13.552+0000 7fa178d59640 1 -- start start 2024-09-12T03:20:13.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:13.552+0000 7fa178d59640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa17415eb90 0x7fa1740d2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:13.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:13.552+0000 7fa178d59640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fa1740d5200 0x7fa1740d1010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:13.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:13.553+0000 7fa178d59640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa1740d3760 con 0x7fa1740d1550 2024-09-12T03:20:13.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:13.553+0000 7fa178d59640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7fa1740b7d20 con 0x7fa1740d5200 2024-09-12T03:20:13.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:13.553+0000 7fa178d59640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fa1740d42e0 con 0x7fa17415eb90 2024-09-12T03:20:13.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:13.553+0000 7fa171d9b640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fa1740d5200 0x7fa1740d1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:13.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:13.553+0000 7fa172d9d640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fa1740d1550 0x7fa1740cf200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:50910/0 (socket says 172.21.15.27:50910) 2024-09-12T03:20:13.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:13.553+0000 7fa172d9d640 1 -- 172.21.15.27:0/739949781 learned_addr learned my addr 172.21.15.27:0/739949781 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:20:13.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:13.553+0000 7fa17259c640 1 --2- 172.21.15.27:0/739949781 >> v2:172.21.15.45:3300/0 conn(0x7fa17415eb90 0x7fa1740d2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:13.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:13.553+0000 7fa171d9b640 1 -- 172.21.15.27:0/739949781 >> v2:172.21.15.45:3300/0 conn(0x7fa17415eb90 msgr2=0x7fa1740d2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:13.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:13.553+0000 7fa171d9b640 1 --2- 172.21.15.27:0/739949781 >> v2:172.21.15.45:3300/0 conn(0x7fa17415eb90 0x7fa1740d2df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:13.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:13.553+0000 7fa171d9b640 1 -- 172.21.15.27:0/739949781 >> v1:172.21.15.27:6789/0 conn(0x7fa1740d1550 legacy=0x7fa1740cf200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:13.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:13.553+0000 7fa171d9b640 1 -- 172.21.15.27:0/739949781 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1740d3a40 con 0x7fa1740d5200 2024-09-12T03:20:13.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:13.553+0000 7fa17259c640 1 --2- 172.21.15.27:0/739949781 >> v2:172.21.15.45:3300/0 conn(0x7fa17415eb90 0x7fa1740d2df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:20:13.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:13.553+0000 7fa17159a640 1 -- 172.21.15.27:0/739949781 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4009280704 0 0) 0x7fa1740d3760 con 0x7fa1740d1550 2024-09-12T03:20:13.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:13.554+0000 7fa171d9b640 1 --2- 172.21.15.27:0/739949781 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa1740d5200 0x7fa1740d1010 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fa174075560 tx=0x7fa15c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29a91b5313de0d06 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:13.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:13.554+0000 7fa17159a640 1 -- 172.21.15.27:0/739949781 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa15c037070 con 0x7fa1740d5200 2024-09-12T03:20:13.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:13.554+0000 7fa17159a640 1 -- 172.21.15.27:0/739949781 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa15c032440 con 0x7fa1740d5200 2024-09-12T03:20:13.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:13.554+0000 7fa17159a640 1 -- 172.21.15.27:0/739949781 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa15c032810 con 0x7fa1740d5200 2024-09-12T03:20:13.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:13.554+0000 7fa178d59640 1 -- 172.21.15.27:0/739949781 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa1740d5200 msgr2=0x7fa1740d1010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:13.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:13.554+0000 7fa178d59640 1 --2- 172.21.15.27:0/739949781 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa1740d5200 0x7fa1740d1010 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fa174075560 tx=0x7fa15c02a1a0 comp rx=0 tx=0).stop 2024-09-12T03:20:13.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:13.554+0000 7fa178d59640 1 -- 172.21.15.27:0/739949781 shutdown_connections 2024-09-12T03:20:13.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:13.555+0000 7fa178d59640 1 --2- 172.21.15.27:0/739949781 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa1740d5200 0x7fa1740d1010 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:13.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:13.555+0000 7fa178d59640 1 --2- 172.21.15.27:0/739949781 >> v2:172.21.15.45:3300/0 conn(0x7fa17415eb90 0x7fa1740d2df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:13.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:13.555+0000 7fa178d59640 1 -- 172.21.15.27:0/739949781 >> 172.21.15.27:0/739949781 conn(0x7fa17406dce0 msgr2=0x7fa17406e0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:13.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:13.555+0000 7fa178d59640 1 -- 172.21.15.27:0/739949781 shutdown_connections 2024-09-12T03:20:13.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:13.555+0000 7fa178d59640 1 -- 172.21.15.27:0/739949781 wait complete. 2024-09-12T03:20:13.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:13.556+0000 7fa178d59640 1 Processor -- start 2024-09-12T03:20:13.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:13.556+0000 7fa178d59640 1 -- start start 2024-09-12T03:20:13.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:13.557+0000 7fa178d59640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa1740d1550 0x7fa174158bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:13.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:13.557+0000 7fa178d59640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa1740d5200 0x7fa174146130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:13.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:13.557+0000 7fa178d59640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa17415eb90 0x7fa174156bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:13.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:13.557+0000 7fa178d59640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fa1740d1960 con 0x7fa1740d1550 2024-09-12T03:20:13.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:13.557+0000 7fa178d59640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa1740d4000 con 0x7fa17415eb90 2024-09-12T03:20:13.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:13.557+0000 7fa178d59640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fa1740d3330 con 0x7fa1740d5200 2024-09-12T03:20:13.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:13.557+0000 7fa17259c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa1740d1550 0x7fa174158bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:13.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:13.557+0000 7fa17259c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa1740d1550 0x7fa174158bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57888/0 (socket says 172.21.15.27:57888) 2024-09-12T03:20:13.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:13.557+0000 7fa17259c640 1 -- 172.21.15.27:0/4259047272 learned_addr learned my addr 172.21.15.27:0/4259047272 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:20:13.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:13.557+0000 7fa172d9d640 1 --2- 172.21.15.27:0/4259047272 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa17415eb90 0x7fa174156bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:13.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:13.557+0000 7fa17259c640 1 -- 172.21.15.27:0/4259047272 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa1740d5200 msgr2=0x7fa174146130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:13.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:13.557+0000 7fa171d9b640 1 --2- 172.21.15.27:0/4259047272 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa1740d5200 0x7fa174146130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:13.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:13.557+0000 7fa17259c640 1 --2- 172.21.15.27:0/4259047272 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa1740d5200 0x7fa174146130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:13.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:13.557+0000 7fa17259c640 1 -- 172.21.15.27:0/4259047272 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa17415eb90 msgr2=0x7fa174156bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:13.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:13.557+0000 7fa17259c640 1 --2- 172.21.15.27:0/4259047272 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa17415eb90 0x7fa174156bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:13.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:13.557+0000 7fa17259c640 1 -- 172.21.15.27:0/4259047272 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1740c9530 con 0x7fa1740d1550 2024-09-12T03:20:13.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:13.557+0000 7fa172d9d640 1 --2- 172.21.15.27:0/4259047272 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa17415eb90 0x7fa174156bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:20:13.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:13.557+0000 7fa17259c640 1 --2- 172.21.15.27:0/4259047272 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa1740d1550 0x7fa174158bf0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fa16400d7d0 tx=0x7fa16400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:13.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:13.557+0000 7fa16affd640 1 -- 172.21.15.27:0/4259047272 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa164004030 con 0x7fa1740d1550 2024-09-12T03:20:13.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:13.557+0000 7fa16affd640 1 -- 172.21.15.27:0/4259047272 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1640041d0 con 0x7fa1740d1550 2024-09-12T03:20:13.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:13.557+0000 7fa16affd640 1 -- 172.21.15.27:0/4259047272 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa16400b6c0 con 0x7fa1740d1550 2024-09-12T03:20:13.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:13.557+0000 7fa178d59640 1 -- 172.21.15.27:0/4259047272 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1741595f0 con 0x7fa1740d1550 2024-09-12T03:20:13.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:13.558+0000 7fa178d59640 1 -- 172.21.15.27:0/4259047272 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa174159bb0 con 0x7fa1740d1550 2024-09-12T03:20:13.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:13.558+0000 7fa178d59640 1 -- 172.21.15.27:0/4259047272 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa17415ef70 con 0x7fa1740d1550 2024-09-12T03:20:13.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:13.559+0000 7fa16affd640 1 -- 172.21.15.27:0/4259047272 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 11) ==== 90804+0+0 (secure 0 0 0) 0x7fa16400b860 con 0x7fa1740d1550 2024-09-12T03:20:13.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:13.559+0000 7fa16affd640 1 --2- 172.21.15.27:0/4259047272 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fa15406c190 0x7fa15406e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:13.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:13.559+0000 7fa16affd640 1 -- 172.21.15.27:0/4259047272 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 5970+0+0 (secure 0 0 0) 0x7fa16408b800 con 0x7fa1740d1550 2024-09-12T03:20:13.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:13.560+0000 7fa171d9b640 1 --2- 172.21.15.27:0/4259047272 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fa15406c190 0x7fa15406e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:13.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:13.560+0000 7fa171d9b640 1 --2- 172.21.15.27:0/4259047272 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fa15406c190 0x7fa15406e650 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fa15c034000 tx=0x7fa15c02e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:13.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:13.564+0000 7fa16affd640 1 -- 172.21.15.27:0/4259047272 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fa164059bd0 con 0x7fa1740d1550 2024-09-12T03:20:13.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:13.785+0000 7fa178d59640 1 -- 172.21.15.27:0/4259047272 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7fa17415e340 con 0x7fa1740d1550 2024-09-12T03:20:13.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:13.790+0000 7fa16affd640 1 -- 172.21.15.27:0/4259047272 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v11) ==== 81+0+471936 (secure 0 0 0) 0x7fa1640594d0 con 0x7fa1740d1550 2024-09-12T03:20:13.796 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:20:13.796 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-12T03:20:13.797 INFO:teuthology.orchestra.run.smithi027.stdout: "epoch": 11, 2024-09-12T03:20:13.797 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:13.797 INFO:teuthology.orchestra.run.smithi027.stdout: "active_gid": 4102, 2024-09-12T03:20:13.797 INFO:teuthology.orchestra.run.smithi027.stdout: "active_name": "x", 2024-09-12T03:20:13.797 INFO:teuthology.orchestra.run.smithi027.stdout: "active_addrs": { 2024-09-12T03:20:13.797 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-12T03:20:13.797 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:20:13.797 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-12T03:20:13.797 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.45:6832", 2024-09-12T03:20:13.797 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3923821219 2024-09-12T03:20:13.797 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:13.797 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:20:13.798 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-12T03:20:13.798 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.45:6833", 2024-09-12T03:20:13.798 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3923821219 2024-09-12T03:20:13.798 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:20:13.798 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-12T03:20:13.798 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:13.798 INFO:teuthology.orchestra.run.smithi027.stdout: "active_addr": "172.21.15.45:6833/3923821219", 2024-09-12T03:20:13.798 INFO:teuthology.orchestra.run.smithi027.stdout: "active_change": "2024-09-12T03:16:12.041759+0000", 2024-09-12T03:20:13.798 INFO:teuthology.orchestra.run.smithi027.stdout: "active_mgr_features": 4540701547738038271, 2024-09-12T03:20:13.798 INFO:teuthology.orchestra.run.smithi027.stdout: "available": true, 2024-09-12T03:20:13.798 INFO:teuthology.orchestra.run.smithi027.stdout: "standbys": [ 2024-09-12T03:20:13.798 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:20:13.798 INFO:teuthology.orchestra.run.smithi027.stdout: "gid": 4113, 2024-09-12T03:20:13.798 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "y", 2024-09-12T03:20:13.799 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr_features": 4540701547738038271, 2024-09-12T03:20:13.799 INFO:teuthology.orchestra.run.smithi027.stdout: "available_modules": [ 2024-09-12T03:20:13.799 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:20:13.799 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "alerts", 2024-09-12T03:20:13.799 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-12T03:20:13.799 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-12T03:20:13.799 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-12T03:20:13.799 INFO:teuthology.orchestra.run.smithi027.stdout: "interval": { 2024-09-12T03:20:13.799 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "interval", 2024-09-12T03:20:13.799 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-12T03:20:13.799 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:13.799 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:13.799 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "60", 2024-09-12T03:20:13.799 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:13.800 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:13.800 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:13.800 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "How frequently to reexamine health status", 2024-09-12T03:20:13.800 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:13.800 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:13.800 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:13.800 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:13.800 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-12T03:20:13.800 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-12T03:20:13.800 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:13.800 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:13.800 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:13.800 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-12T03:20:13.800 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:13.801 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:13.801 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-12T03:20:13.801 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-12T03:20:13.801 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-12T03:20:13.801 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-12T03:20:13.801 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-12T03:20:13.801 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-12T03:20:13.801 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-12T03:20:13.801 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-12T03:20:13.801 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:13.801 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:13.801 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:13.802 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:13.802 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:13.802 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-12T03:20:13.802 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-12T03:20:13.802 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:13.802 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:13.802 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:13.802 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-12T03:20:13.802 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:13.802 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:13.802 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:13.802 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:13.802 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:13.802 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:13.803 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:13.803 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:13.803 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-12T03:20:13.803 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-12T03:20:13.803 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:13.803 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:13.803 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:13.803 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-12T03:20:13.803 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:13.803 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:13.803 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-12T03:20:13.803 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-12T03:20:13.803 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-12T03:20:13.803 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-12T03:20:13.804 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-12T03:20:13.804 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-12T03:20:13.804 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-12T03:20:13.804 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-12T03:20:13.804 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:13.804 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:13.804 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:13.804 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:13.804 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:13.804 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-12T03:20:13.804 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-12T03:20:13.804 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:13.804 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:13.804 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:13.805 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-12T03:20:13.805 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:13.805 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:13.805 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:13.805 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:13.805 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:13.805 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:13.805 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:13.805 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:13.805 INFO:teuthology.orchestra.run.smithi027.stdout: "smtp_destination": { 2024-09-12T03:20:13.805 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "smtp_destination", 2024-09-12T03:20:13.805 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:13.805 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:13.805 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:13.806 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-12T03:20:13.806 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:13.806 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:13.806 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:13.806 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Email address to send alerts to", 2024-09-12T03:20:13.806 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:13.806 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:13.806 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:13.806 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:13.806 INFO:teuthology.orchestra.run.smithi027.stdout: "smtp_from_name": { 2024-09-12T03:20:13.806 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "smtp_from_name", 2024-09-12T03:20:13.806 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:13.806 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:13.806 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:13.807 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "Ceph", 2024-09-12T03:20:13.807 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:13.807 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:13.807 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:13.807 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Email From: name", 2024-09-12T03:20:13.807 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:13.807 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:13.807 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:13.807 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:13.807 INFO:teuthology.orchestra.run.smithi027.stdout: "smtp_host": { 2024-09-12T03:20:13.807 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "smtp_host", 2024-09-12T03:20:13.807 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:13.807 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:13.807 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:13.808 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-12T03:20:13.808 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:13.808 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:13.808 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:13.808 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "SMTP server", 2024-09-12T03:20:13.808 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:13.808 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:13.808 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:13.808 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:13.808 INFO:teuthology.orchestra.run.smithi027.stdout: "smtp_password": { 2024-09-12T03:20:13.808 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "smtp_password", 2024-09-12T03:20:13.808 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:13.808 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:13.808 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:13.809 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-12T03:20:13.809 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:13.809 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:13.809 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:13.809 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Password to authenticate with", 2024-09-12T03:20:13.809 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:13.809 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:13.809 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:13.809 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:13.809 INFO:teuthology.orchestra.run.smithi027.stdout: "smtp_port": { 2024-09-12T03:20:13.809 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "smtp_port", 2024-09-12T03:20:13.809 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-12T03:20:13.809 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:13.809 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:13.810 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "465", 2024-09-12T03:20:13.810 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:13.810 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:13.810 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:13.810 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "SMTP port", 2024-09-12T03:20:13.810 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:13.810 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:13.810 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:13.810 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:13.810 INFO:teuthology.orchestra.run.smithi027.stdout: "smtp_sender": { 2024-09-12T03:20:13.810 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "smtp_sender", 2024-09-12T03:20:13.810 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:13.810 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:13.810 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:13.811 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-12T03:20:13.811 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:13.811 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:13.811 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:13.811 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "SMTP envelope sender", 2024-09-12T03:20:13.811 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:13.811 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:13.811 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:13.811 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:13.811 INFO:teuthology.orchestra.run.smithi027.stdout: "smtp_ssl": { 2024-09-12T03:20:13.811 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "smtp_ssl", 2024-09-12T03:20:13.811 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:13.812 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:13.812 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:13.812 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-12T03:20:13.812 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:13.812 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:13.812 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:13.812 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-12T03:20:13.812 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:13.812 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:13.813 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:13.813 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:13.813 INFO:teuthology.orchestra.run.smithi027.stdout: "smtp_user": { 2024-09-12T03:20:13.813 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "smtp_user", 2024-09-12T03:20:13.813 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:13.813 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:13.813 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:13.813 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-12T03:20:13.813 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:13.813 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:13.813 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:13.813 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "User to authenticate as", 2024-09-12T03:20:13.813 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:13.813 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:13.814 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:13.814 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:20:13.814 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:20:13.814 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:13.814 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:20:13.814 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "balancer", 2024-09-12T03:20:13.814 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-12T03:20:13.814 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-12T03:20:13.814 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-12T03:20:13.814 INFO:teuthology.orchestra.run.smithi027.stdout: "active": { 2024-09-12T03:20:13.814 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "active", 2024-09-12T03:20:13.814 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:13.814 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:13.814 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:13.815 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-12T03:20:13.815 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:13.815 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:13.815 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:13.815 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "automatically balance PGs across cluster", 2024-09-12T03:20:13.815 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:13.815 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:13.815 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:13.815 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:13.815 INFO:teuthology.orchestra.run.smithi027.stdout: "begin_time": { 2024-09-12T03:20:13.816 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "begin_time", 2024-09-12T03:20:13.816 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:13.816 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:13.816 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:13.816 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0000", 2024-09-12T03:20:13.816 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:13.816 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:13.816 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:13.816 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "beginning time of day to automatically balance", 2024-09-12T03:20:13.816 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-12T03:20:13.817 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:13.817 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:13.817 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:13.817 INFO:teuthology.orchestra.run.smithi027.stdout: "begin_weekday": { 2024-09-12T03:20:13.817 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "begin_weekday", 2024-09-12T03:20:13.817 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "uint", 2024-09-12T03:20:13.818 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:13.818 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:13.818 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-12T03:20:13.818 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "0", 2024-09-12T03:20:13.818 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "6", 2024-09-12T03:20:13.818 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:13.818 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-12T03:20:13.818 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-12T03:20:13.818 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:13.818 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:13.818 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:13.818 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_compat_max_iterations": { 2024-09-12T03:20:13.818 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "crush_compat_max_iterations", 2024-09-12T03:20:13.818 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "uint", 2024-09-12T03:20:13.819 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:13.819 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:13.819 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "25", 2024-09-12T03:20:13.819 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "1", 2024-09-12T03:20:13.819 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "250", 2024-09-12T03:20:13.819 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:13.819 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-12T03:20:13.819 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:13.819 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:13.819 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:13.819 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:13.819 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_compat_metrics": { 2024-09-12T03:20:13.819 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "crush_compat_metrics", 2024-09-12T03:20:13.819 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:13.820 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:13.820 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:13.820 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "pgs,objects,bytes", 2024-09-12T03:20:13.820 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:13.820 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:13.820 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:13.820 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-12T03:20:13.820 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-12T03:20:13.820 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:13.820 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:13.820 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:13.820 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_compat_step": { 2024-09-12T03:20:13.820 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "crush_compat_step", 2024-09-12T03:20:13.820 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "float", 2024-09-12T03:20:13.821 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:13.821 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:13.821 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0.5", 2024-09-12T03:20:13.821 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "0.001", 2024-09-12T03:20:13.821 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "0.999", 2024-09-12T03:20:13.821 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:13.821 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "aggressiveness of optimization", 2024-09-12T03:20:13.821 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-12T03:20:13.821 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:13.821 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:13.821 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:13.821 INFO:teuthology.orchestra.run.smithi027.stdout: "end_time": { 2024-09-12T03:20:13.821 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "end_time", 2024-09-12T03:20:13.821 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:13.822 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:13.822 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:13.822 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "2359", 2024-09-12T03:20:13.822 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:13.822 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:13.822 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:13.822 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "ending time of day to automatically balance", 2024-09-12T03:20:13.822 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-12T03:20:13.822 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:13.822 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:13.822 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:13.822 INFO:teuthology.orchestra.run.smithi027.stdout: "end_weekday": { 2024-09-12T03:20:13.822 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "end_weekday", 2024-09-12T03:20:13.822 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "uint", 2024-09-12T03:20:13.823 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:13.823 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:13.823 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-12T03:20:13.823 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "0", 2024-09-12T03:20:13.823 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "6", 2024-09-12T03:20:13.823 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:13.823 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-12T03:20:13.823 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-12T03:20:13.823 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:13.823 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:13.823 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:13.823 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-12T03:20:13.823 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-12T03:20:13.823 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:13.824 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:13.824 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:13.824 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-12T03:20:13.824 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:13.824 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:13.824 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-12T03:20:13.824 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-12T03:20:13.824 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-12T03:20:13.824 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-12T03:20:13.824 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-12T03:20:13.824 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-12T03:20:13.824 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-12T03:20:13.824 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-12T03:20:13.824 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:13.825 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:13.825 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:13.825 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:13.825 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:13.825 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-12T03:20:13.825 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-12T03:20:13.825 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:13.825 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:13.825 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:13.825 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-12T03:20:13.825 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:13.825 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:13.825 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:13.826 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:13.826 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:13.826 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:13.826 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:13.826 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:13.826 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-12T03:20:13.826 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-12T03:20:13.826 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:13.826 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:13.826 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:13.826 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-12T03:20:13.826 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:13.827 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:13.827 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-12T03:20:13.827 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-12T03:20:13.827 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-12T03:20:13.827 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-12T03:20:13.827 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-12T03:20:13.827 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-12T03:20:13.827 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-12T03:20:13.827 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-12T03:20:13.827 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:13.827 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:13.827 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:13.827 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:13.828 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:13.828 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-12T03:20:13.828 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-12T03:20:13.828 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:13.828 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:13.828 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:13.828 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-12T03:20:13.828 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:13.828 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:13.828 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:13.828 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:13.828 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:13.829 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:13.829 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:13.829 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:13.829 INFO:teuthology.orchestra.run.smithi027.stdout: "min_score": { 2024-09-12T03:20:13.829 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "min_score", 2024-09-12T03:20:13.829 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "float", 2024-09-12T03:20:13.829 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:13.829 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:13.829 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-12T03:20:13.829 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:13.829 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:13.829 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:13.830 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-12T03:20:13.830 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:13.830 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:13.830 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:13.830 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:13.830 INFO:teuthology.orchestra.run.smithi027.stdout: "mode": { 2024-09-12T03:20:13.830 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "mode", 2024-09-12T03:20:13.830 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:13.830 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:13.830 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:13.830 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "upmap", 2024-09-12T03:20:13.831 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:13.831 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:13.831 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-12T03:20:13.831 INFO:teuthology.orchestra.run.smithi027.stdout: "crush-compat", 2024-09-12T03:20:13.831 INFO:teuthology.orchestra.run.smithi027.stdout: "none", 2024-09-12T03:20:13.831 INFO:teuthology.orchestra.run.smithi027.stdout: "read", 2024-09-12T03:20:13.831 INFO:teuthology.orchestra.run.smithi027.stdout: "upmap", 2024-09-12T03:20:13.831 INFO:teuthology.orchestra.run.smithi027.stdout: "upmap-read" 2024-09-12T03:20:13.831 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-12T03:20:13.831 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Balancer mode", 2024-09-12T03:20:13.831 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:13.831 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:13.831 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:13.831 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:13.832 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_ids": { 2024-09-12T03:20:13.832 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "pool_ids", 2024-09-12T03:20:13.832 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:13.832 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:13.832 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:13.832 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-12T03:20:13.832 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:13.832 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:13.832 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:13.832 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-12T03:20:13.832 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:13.832 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:13.833 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:13.833 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:13.833 INFO:teuthology.orchestra.run.smithi027.stdout: "sleep_interval": { 2024-09-12T03:20:13.833 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sleep_interval", 2024-09-12T03:20:13.833 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-12T03:20:13.833 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:13.833 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:13.833 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "60", 2024-09-12T03:20:13.833 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:13.833 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:13.833 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:13.834 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-12T03:20:13.834 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:13.834 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:13.834 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:13.834 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:13.834 INFO:teuthology.orchestra.run.smithi027.stdout: "upmap_max_deviation": { 2024-09-12T03:20:13.834 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "upmap_max_deviation", 2024-09-12T03:20:13.834 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-12T03:20:13.834 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:13.834 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:13.835 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "5", 2024-09-12T03:20:13.835 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "1", 2024-09-12T03:20:13.835 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:13.835 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:13.835 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-12T03:20:13.835 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-12T03:20:13.835 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:13.835 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:13.835 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:13.835 INFO:teuthology.orchestra.run.smithi027.stdout: "upmap_max_optimizations": { 2024-09-12T03:20:13.835 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "upmap_max_optimizations", 2024-09-12T03:20:13.835 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "uint", 2024-09-12T03:20:13.835 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:13.835 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:13.836 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10", 2024-09-12T03:20:13.836 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:13.836 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:13.836 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:13.836 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-12T03:20:13.836 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:13.836 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:13.836 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:13.836 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:20:13.836 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:20:13.836 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:13.836 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:20:13.836 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "cephadm", 2024-09-12T03:20:13.836 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-12T03:20:13.837 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-12T03:20:13.837 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-12T03:20:13.837 INFO:teuthology.orchestra.run.smithi027.stdout: "agent_down_multiplier": { 2024-09-12T03:20:13.837 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "agent_down_multiplier", 2024-09-12T03:20:13.837 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "float", 2024-09-12T03:20:13.837 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:13.837 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:13.837 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "3.0", 2024-09-12T03:20:13.837 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:13.837 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:13.837 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:13.837 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-12T03:20:13.837 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:13.837 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:13.838 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:13.838 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:13.838 INFO:teuthology.orchestra.run.smithi027.stdout: "agent_refresh_rate": { 2024-09-12T03:20:13.838 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "agent_refresh_rate", 2024-09-12T03:20:13.838 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-12T03:20:13.838 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:13.838 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:13.838 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "20", 2024-09-12T03:20:13.838 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:13.838 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:13.838 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:13.838 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-12T03:20:13.838 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:13.838 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:13.839 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:13.839 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:13.839 INFO:teuthology.orchestra.run.smithi027.stdout: "agent_starting_port": { 2024-09-12T03:20:13.839 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "agent_starting_port", 2024-09-12T03:20:13.839 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-12T03:20:13.839 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:13.839 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:13.839 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "4721", 2024-09-12T03:20:13.839 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:13.839 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:13.839 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:13.839 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-12T03:20:13.839 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:13.840 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:13.840 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:13.840 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:13.840 INFO:teuthology.orchestra.run.smithi027.stdout: "allow_ptrace": { 2024-09-12T03:20:13.840 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "allow_ptrace", 2024-09-12T03:20:13.840 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:13.840 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:13.840 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:13.840 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-12T03:20:13.840 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:13.841 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:13.841 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:13.841 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-12T03:20:13.841 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-12T03:20:13.841 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:13.841 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:13.841 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:13.841 INFO:teuthology.orchestra.run.smithi027.stdout: "autotune_interval": { 2024-09-12T03:20:13.841 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "autotune_interval", 2024-09-12T03:20:13.841 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-12T03:20:13.842 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:13.842 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:13.842 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "600", 2024-09-12T03:20:13.842 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:13.842 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:13.842 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:13.842 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-12T03:20:13.842 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:13.842 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:13.842 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:13.842 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:13.842 INFO:teuthology.orchestra.run.smithi027.stdout: "autotune_memory_target_ratio": { 2024-09-12T03:20:13.843 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "autotune_memory_target_ratio", 2024-09-12T03:20:13.843 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "float", 2024-09-12T03:20:13.843 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:13.843 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:13.843 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0.7", 2024-09-12T03:20:13.843 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:13.843 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:13.843 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:13.843 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-12T03:20:13.843 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:13.844 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:13.844 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:13.844 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:13.844 INFO:teuthology.orchestra.run.smithi027.stdout: "cephadm_log_destination": { 2024-09-12T03:20:13.844 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "cephadm_log_destination", 2024-09-12T03:20:13.845 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:13.845 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:13.845 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:13.845 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-12T03:20:13.845 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:13.845 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:13.845 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-12T03:20:13.845 INFO:teuthology.orchestra.run.smithi027.stdout: "file", 2024-09-12T03:20:13.845 INFO:teuthology.orchestra.run.smithi027.stdout: "file,syslog", 2024-09-12T03:20:13.845 INFO:teuthology.orchestra.run.smithi027.stdout: "syslog" 2024-09-12T03:20:13.846 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-12T03:20:13.846 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-12T03:20:13.846 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:13.846 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:13.846 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:13.846 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:13.846 INFO:teuthology.orchestra.run.smithi027.stdout: "cgroups_split": { 2024-09-12T03:20:13.846 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "cgroups_split", 2024-09-12T03:20:13.846 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:13.846 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:13.846 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:13.846 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-12T03:20:13.846 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:13.846 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:13.847 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:13.847 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-12T03:20:13.847 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:13.847 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:13.847 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:13.847 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:13.848 INFO:teuthology.orchestra.run.smithi027.stdout: "config_checks_enabled": { 2024-09-12T03:20:13.848 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "config_checks_enabled", 2024-09-12T03:20:13.848 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:13.848 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:13.848 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:13.848 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-12T03:20:13.848 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:13.848 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:13.848 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:13.848 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-12T03:20:13.848 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:13.848 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:13.848 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:13.848 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:13.849 INFO:teuthology.orchestra.run.smithi027.stdout: "config_dashboard": { 2024-09-12T03:20:13.849 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "config_dashboard", 2024-09-12T03:20:13.849 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:13.849 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:13.849 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:13.849 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-12T03:20:13.849 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:13.849 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:13.849 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:13.849 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-12T03:20:13.849 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:13.849 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:13.849 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:13.849 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:13.850 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_alertmanager": { 2024-09-12T03:20:13.850 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_alertmanager", 2024-09-12T03:20:13.850 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:13.850 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:13.850 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:13.850 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-12T03:20:13.850 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:13.850 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:13.850 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:13.850 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Prometheus container image", 2024-09-12T03:20:13.850 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:13.850 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:13.850 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:13.850 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:13.851 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_base": { 2024-09-12T03:20:13.851 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_base", 2024-09-12T03:20:13.851 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:13.851 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:13.851 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:13.851 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-12T03:20:13.851 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:13.851 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:13.851 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:13.851 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Container image name, without the tag", 2024-09-12T03:20:13.851 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:13.851 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:13.852 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:13.852 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:13.852 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_elasticsearch": { 2024-09-12T03:20:13.852 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_elasticsearch", 2024-09-12T03:20:13.852 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:13.852 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:13.852 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:13.852 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-12T03:20:13.852 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:13.852 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:13.852 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:13.852 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "elasticsearch container image", 2024-09-12T03:20:13.852 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:13.852 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:13.853 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:13.853 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:13.853 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_grafana": { 2024-09-12T03:20:13.853 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_grafana", 2024-09-12T03:20:13.853 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:13.853 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:13.853 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:13.853 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-09-12T03:20:13.853 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:13.853 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:13.853 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:13.853 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Prometheus container image", 2024-09-12T03:20:13.853 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:13.853 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:13.854 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:13.854 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:13.854 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_haproxy": { 2024-09-12T03:20:13.854 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_haproxy", 2024-09-12T03:20:13.854 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:13.854 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:13.854 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:13.854 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-12T03:20:13.854 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:13.854 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:13.854 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:13.854 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "HAproxy container image", 2024-09-12T03:20:13.854 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:13.855 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:13.855 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:13.855 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:13.855 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_jaeger_agent": { 2024-09-12T03:20:13.855 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_jaeger_agent", 2024-09-12T03:20:13.855 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:13.855 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:13.855 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:13.855 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-12T03:20:13.855 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:13.855 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:13.855 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:13.855 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Jaeger agent container image", 2024-09-12T03:20:13.855 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:13.856 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:13.856 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:13.856 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:13.856 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_jaeger_collector": { 2024-09-12T03:20:13.856 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_jaeger_collector", 2024-09-12T03:20:13.856 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:13.856 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:13.856 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:13.856 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-12T03:20:13.856 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:13.856 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:13.856 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:13.856 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Jaeger collector container image", 2024-09-12T03:20:13.857 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:13.857 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:13.857 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:13.857 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:13.857 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_jaeger_query": { 2024-09-12T03:20:13.857 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_jaeger_query", 2024-09-12T03:20:13.857 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:13.857 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:13.857 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:13.857 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-12T03:20:13.857 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:13.857 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:13.857 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:13.857 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Jaeger query container image", 2024-09-12T03:20:13.858 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:13.858 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:13.858 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:13.858 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:13.858 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_keepalived": { 2024-09-12T03:20:13.858 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_keepalived", 2024-09-12T03:20:13.858 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:13.858 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:13.858 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:13.858 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-12T03:20:13.858 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:13.858 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:13.858 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:13.858 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Keepalived container image", 2024-09-12T03:20:13.859 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:13.859 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:13.859 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:13.859 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:13.859 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_loki": { 2024-09-12T03:20:13.859 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_loki", 2024-09-12T03:20:13.859 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:13.859 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:13.859 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:13.859 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-12T03:20:13.859 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:13.859 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:13.859 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:13.859 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Loki container image", 2024-09-12T03:20:13.860 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:13.860 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:13.860 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:13.860 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:13.860 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_node_exporter": { 2024-09-12T03:20:13.860 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_node_exporter", 2024-09-12T03:20:13.861 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:13.861 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:13.861 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:13.861 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-12T03:20:13.861 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:13.861 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:13.861 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:13.861 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Prometheus container image", 2024-09-12T03:20:13.861 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:13.861 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:13.861 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:13.861 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:13.861 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_nvmeof": { 2024-09-12T03:20:13.861 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_nvmeof", 2024-09-12T03:20:13.862 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:13.862 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:13.862 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:13.862 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-12T03:20:13.862 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:13.862 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:13.862 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:13.862 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Nvme-of container image", 2024-09-12T03:20:13.862 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:13.862 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:13.862 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:13.862 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:13.862 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_prometheus": { 2024-09-12T03:20:13.862 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_prometheus", 2024-09-12T03:20:13.863 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:13.863 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:13.863 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:13.863 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-12T03:20:13.863 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:13.863 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:13.863 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:13.863 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Prometheus container image", 2024-09-12T03:20:13.863 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:13.863 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:13.863 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:13.863 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:13.863 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_promtail": { 2024-09-12T03:20:13.863 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_promtail", 2024-09-12T03:20:13.864 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:13.864 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:13.864 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:13.864 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-12T03:20:13.864 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:13.864 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:13.864 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:13.864 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Promtail container image", 2024-09-12T03:20:13.864 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:13.864 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:13.864 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:13.864 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:13.864 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_samba": { 2024-09-12T03:20:13.864 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_samba", 2024-09-12T03:20:13.865 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:13.865 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:13.865 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:13.865 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-12T03:20:13.865 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:13.865 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:13.865 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:13.865 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Samba/SMB container image", 2024-09-12T03:20:13.865 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:13.865 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:13.865 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:13.865 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:13.865 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_snmp_gateway": { 2024-09-12T03:20:13.865 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_snmp_gateway", 2024-09-12T03:20:13.866 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:13.866 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:13.866 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:13.866 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-12T03:20:13.866 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:13.866 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:13.866 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:13.866 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "SNMP Gateway container image", 2024-09-12T03:20:13.866 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:13.866 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:13.866 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:13.866 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:13.866 INFO:teuthology.orchestra.run.smithi027.stdout: "container_init": { 2024-09-12T03:20:13.866 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_init", 2024-09-12T03:20:13.867 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:13.867 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:13.867 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:13.867 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-12T03:20:13.867 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:13.867 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:13.867 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:13.867 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Run podman/docker with `--init`", 2024-09-12T03:20:13.867 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:13.867 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:13.867 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:13.867 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:13.868 INFO:teuthology.orchestra.run.smithi027.stdout: "daemon_cache_timeout": { 2024-09-12T03:20:13.868 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "daemon_cache_timeout", 2024-09-12T03:20:13.868 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-12T03:20:13.868 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:13.868 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:13.868 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "600", 2024-09-12T03:20:13.868 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:13.868 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:13.868 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:13.868 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-12T03:20:13.868 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:13.868 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:13.868 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:13.868 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:13.869 INFO:teuthology.orchestra.run.smithi027.stdout: "default_cephadm_command_timeout": { 2024-09-12T03:20:13.869 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "default_cephadm_command_timeout", 2024-09-12T03:20:13.869 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-12T03:20:13.869 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:13.869 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:13.869 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "900", 2024-09-12T03:20:13.869 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:13.869 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:13.869 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:13.869 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-12T03:20:13.869 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:13.869 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:13.869 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:13.869 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:13.870 INFO:teuthology.orchestra.run.smithi027.stdout: "default_registry": { 2024-09-12T03:20:13.870 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "default_registry", 2024-09-12T03:20:13.870 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:13.870 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:13.870 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:13.870 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "docker.io", 2024-09-12T03:20:13.870 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:13.870 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:13.870 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:13.870 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-12T03:20:13.870 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:13.870 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:13.870 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:13.870 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:13.871 INFO:teuthology.orchestra.run.smithi027.stdout: "device_cache_timeout": { 2024-09-12T03:20:13.871 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "device_cache_timeout", 2024-09-12T03:20:13.871 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-12T03:20:13.871 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:13.871 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:13.871 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "1800", 2024-09-12T03:20:13.871 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:13.871 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:13.871 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:13.871 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "seconds to cache device inventory", 2024-09-12T03:20:13.871 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:13.871 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:13.871 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:13.871 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:13.872 INFO:teuthology.orchestra.run.smithi027.stdout: "device_enhanced_scan": { 2024-09-12T03:20:13.872 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "device_enhanced_scan", 2024-09-12T03:20:13.872 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:13.872 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:13.872 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:13.872 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-12T03:20:13.872 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:13.872 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:13.872 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:13.872 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-12T03:20:13.873 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:13.873 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:13.873 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:13.873 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:13.873 INFO:teuthology.orchestra.run.smithi027.stdout: "facts_cache_timeout": { 2024-09-12T03:20:13.873 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "facts_cache_timeout", 2024-09-12T03:20:13.873 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-12T03:20:13.873 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:13.873 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:13.873 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "60", 2024-09-12T03:20:13.873 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:13.873 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:13.873 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:13.874 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "seconds to cache host facts data", 2024-09-12T03:20:13.874 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:13.874 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:13.874 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:13.874 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:13.874 INFO:teuthology.orchestra.run.smithi027.stdout: "grafana_dashboards_path": { 2024-09-12T03:20:13.874 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "grafana_dashboards_path", 2024-09-12T03:20:13.874 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:13.874 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:13.874 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:13.874 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-12T03:20:13.874 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:13.874 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:13.874 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:13.875 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-12T03:20:13.875 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:13.875 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:13.875 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:13.875 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:13.875 INFO:teuthology.orchestra.run.smithi027.stdout: "host_check_interval": { 2024-09-12T03:20:13.875 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "host_check_interval", 2024-09-12T03:20:13.875 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-12T03:20:13.875 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:13.875 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:13.875 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "600", 2024-09-12T03:20:13.875 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:13.875 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:13.875 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:13.876 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "how frequently to perform a host check", 2024-09-12T03:20:13.876 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:13.876 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:13.876 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:13.876 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:13.876 INFO:teuthology.orchestra.run.smithi027.stdout: "hw_monitoring": { 2024-09-12T03:20:13.876 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "hw_monitoring", 2024-09-12T03:20:13.876 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:13.876 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:13.876 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:13.876 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-12T03:20:13.876 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:13.876 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:13.876 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:13.877 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-12T03:20:13.877 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:13.877 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:13.877 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:13.877 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:13.877 INFO:teuthology.orchestra.run.smithi027.stdout: "inventory_list_all": { 2024-09-12T03:20:13.877 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "inventory_list_all", 2024-09-12T03:20:13.877 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:13.877 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:13.877 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:13.877 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-12T03:20:13.877 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:13.877 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:13.877 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:13.878 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-12T03:20:13.878 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:13.878 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:13.878 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:13.878 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:13.878 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-12T03:20:13.878 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-12T03:20:13.878 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:13.878 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:13.878 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:13.878 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-12T03:20:13.878 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:13.878 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:13.878 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-12T03:20:13.879 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-12T03:20:13.879 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-12T03:20:13.879 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-12T03:20:13.879 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-12T03:20:13.879 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-12T03:20:13.879 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-12T03:20:13.879 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-12T03:20:13.879 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:13.879 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:13.879 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:13.879 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:13.879 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:13.879 INFO:teuthology.orchestra.run.smithi027.stdout: "log_refresh_metadata": { 2024-09-12T03:20:13.879 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_refresh_metadata", 2024-09-12T03:20:13.880 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:13.880 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:13.880 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:13.880 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-12T03:20:13.880 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:13.880 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:13.880 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:13.880 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-12T03:20:13.880 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:13.880 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:13.880 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:13.880 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:13.881 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-12T03:20:13.881 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-12T03:20:13.881 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:13.881 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:13.881 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:13.881 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-12T03:20:13.881 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:13.881 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:13.881 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:13.881 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-12T03:20:13.881 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:13.881 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:13.881 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:13.882 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:13.882 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-12T03:20:13.882 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-12T03:20:13.882 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:13.882 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:13.882 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:13.882 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-12T03:20:13.882 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:13.882 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:13.882 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-12T03:20:13.882 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-12T03:20:13.882 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-12T03:20:13.882 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-12T03:20:13.882 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-12T03:20:13.883 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-12T03:20:13.883 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-12T03:20:13.883 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-12T03:20:13.883 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:13.883 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:13.883 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:13.883 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:13.883 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:13.883 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-12T03:20:13.883 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-12T03:20:13.883 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:13.883 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:13.883 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:13.884 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-12T03:20:13.884 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:13.884 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:13.884 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:13.884 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:13.884 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:13.884 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:13.884 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:13.884 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:13.884 INFO:teuthology.orchestra.run.smithi027.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-12T03:20:13.884 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-12T03:20:13.884 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:13.884 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:13.884 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:13.885 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-12T03:20:13.885 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:13.885 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:13.885 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:13.885 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-12T03:20:13.885 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:13.885 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:13.885 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:13.885 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:13.885 INFO:teuthology.orchestra.run.smithi027.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-12T03:20:13.885 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-12T03:20:13.885 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:13.885 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:13.886 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:13.886 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "*", 2024-09-12T03:20:13.886 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:13.886 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:13.886 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:13.886 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-12T03:20:13.886 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:13.886 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:13.886 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:13.886 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:13.886 INFO:teuthology.orchestra.run.smithi027.stdout: "max_count_per_host": { 2024-09-12T03:20:13.886 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "max_count_per_host", 2024-09-12T03:20:13.886 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-12T03:20:13.886 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:13.887 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:13.887 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10", 2024-09-12T03:20:13.887 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:13.887 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:13.887 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:13.887 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "max number of daemons per service per host", 2024-09-12T03:20:13.887 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:13.887 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:13.887 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:13.887 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:13.887 INFO:teuthology.orchestra.run.smithi027.stdout: "max_osd_draining_count": { 2024-09-12T03:20:13.887 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "max_osd_draining_count", 2024-09-12T03:20:13.887 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-12T03:20:13.887 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:13.888 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:13.888 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10", 2024-09-12T03:20:13.888 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:13.888 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:13.888 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:13.888 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-12T03:20:13.888 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:13.888 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:13.888 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:13.888 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:13.888 INFO:teuthology.orchestra.run.smithi027.stdout: "migration_current": { 2024-09-12T03:20:13.888 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "migration_current", 2024-09-12T03:20:13.888 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-12T03:20:13.888 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:13.889 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:13.889 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-12T03:20:13.889 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:13.889 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:13.889 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:13.889 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "internal - do not modify", 2024-09-12T03:20:13.889 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:13.889 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:13.889 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:13.889 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:13.889 INFO:teuthology.orchestra.run.smithi027.stdout: "mode": { 2024-09-12T03:20:13.889 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "mode", 2024-09-12T03:20:13.889 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:13.889 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:13.890 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:13.890 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "root", 2024-09-12T03:20:13.890 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:13.890 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:13.890 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-12T03:20:13.890 INFO:teuthology.orchestra.run.smithi027.stdout: "cephadm-package", 2024-09-12T03:20:13.890 INFO:teuthology.orchestra.run.smithi027.stdout: "root" 2024-09-12T03:20:13.890 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-12T03:20:13.890 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "mode for remote execution of cephadm", 2024-09-12T03:20:13.890 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:13.890 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:13.890 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:13.890 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:13.890 INFO:teuthology.orchestra.run.smithi027.stdout: "oob_default_addr": { 2024-09-12T03:20:13.890 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "oob_default_addr", 2024-09-12T03:20:13.891 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:13.891 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:13.891 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:13.891 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "169.254.1.1", 2024-09-12T03:20:13.891 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:13.891 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:13.891 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:13.891 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-12T03:20:13.891 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:13.891 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:13.891 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:13.891 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:13.891 INFO:teuthology.orchestra.run.smithi027.stdout: "prometheus_alerts_path": { 2024-09-12T03:20:13.891 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "prometheus_alerts_path", 2024-09-12T03:20:13.892 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:13.892 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:13.892 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:13.892 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-12T03:20:13.892 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:13.892 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:13.892 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:13.892 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-12T03:20:13.892 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:13.892 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:13.892 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:13.892 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:13.892 INFO:teuthology.orchestra.run.smithi027.stdout: "registry_insecure": { 2024-09-12T03:20:13.893 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "registry_insecure", 2024-09-12T03:20:13.893 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:13.893 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:13.893 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:13.893 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-12T03:20:13.893 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:13.893 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:13.894 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:13.894 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-12T03:20:13.894 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:13.894 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:13.894 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:13.894 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:13.894 INFO:teuthology.orchestra.run.smithi027.stdout: "registry_password": { 2024-09-12T03:20:13.894 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "registry_password", 2024-09-12T03:20:13.894 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:13.894 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:13.894 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:13.894 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-12T03:20:13.894 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:13.895 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:13.895 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:13.895 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-12T03:20:13.895 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:13.895 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:13.895 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:13.895 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:13.895 INFO:teuthology.orchestra.run.smithi027.stdout: "registry_url": { 2024-09-12T03:20:13.895 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "registry_url", 2024-09-12T03:20:13.895 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:13.895 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:13.895 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:13.895 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-12T03:20:13.895 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:13.896 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:13.896 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:13.896 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-12T03:20:13.896 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:13.896 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:13.896 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:13.896 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:13.896 INFO:teuthology.orchestra.run.smithi027.stdout: "registry_username": { 2024-09-12T03:20:13.896 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "registry_username", 2024-09-12T03:20:13.896 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:13.896 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:13.896 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:13.896 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-12T03:20:13.896 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:13.897 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:13.897 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:13.897 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-12T03:20:13.897 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:13.897 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:13.897 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:13.897 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:13.897 INFO:teuthology.orchestra.run.smithi027.stdout: "secure_monitoring_stack": { 2024-09-12T03:20:13.897 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "secure_monitoring_stack", 2024-09-12T03:20:13.897 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:13.897 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:13.897 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:13.897 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-12T03:20:13.897 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:13.898 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:13.898 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:13.898 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-12T03:20:13.898 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:13.898 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:13.898 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:13.898 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:13.898 INFO:teuthology.orchestra.run.smithi027.stdout: "service_discovery_port": { 2024-09-12T03:20:13.898 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "service_discovery_port", 2024-09-12T03:20:13.898 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-12T03:20:13.898 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:13.898 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:13.898 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "8765", 2024-09-12T03:20:13.898 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:13.899 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:13.899 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:13.899 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "cephadm service discovery port", 2024-09-12T03:20:13.899 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:13.899 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:13.899 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:13.899 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:13.899 INFO:teuthology.orchestra.run.smithi027.stdout: "ssh_config_file": { 2024-09-12T03:20:13.899 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ssh_config_file", 2024-09-12T03:20:13.899 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:13.899 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:13.899 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:13.899 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-12T03:20:13.899 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:13.900 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:13.900 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:13.900 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-12T03:20:13.900 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:13.900 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:13.900 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:13.900 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:13.900 INFO:teuthology.orchestra.run.smithi027.stdout: "use_agent": { 2024-09-12T03:20:13.900 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "use_agent", 2024-09-12T03:20:13.900 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:13.900 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:13.900 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:13.900 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-12T03:20:13.901 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:13.901 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:13.901 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:13.901 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-12T03:20:13.901 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:13.901 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:13.901 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:13.901 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:13.901 INFO:teuthology.orchestra.run.smithi027.stdout: "use_repo_digest": { 2024-09-12T03:20:13.901 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "use_repo_digest", 2024-09-12T03:20:13.901 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:13.901 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:13.901 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:13.902 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-12T03:20:13.902 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:13.902 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:13.902 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:13.902 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-12T03:20:13.902 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:13.902 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:13.902 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:13.902 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:13.902 INFO:teuthology.orchestra.run.smithi027.stdout: "warn_on_failed_host_check": { 2024-09-12T03:20:13.902 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "warn_on_failed_host_check", 2024-09-12T03:20:13.902 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:13.902 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:13.902 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:13.903 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-12T03:20:13.903 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:13.903 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:13.903 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:13.903 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "raise a health warning if the host check fails", 2024-09-12T03:20:13.903 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:13.903 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:13.903 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:13.903 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:13.903 INFO:teuthology.orchestra.run.smithi027.stdout: "warn_on_stray_daemons": { 2024-09-12T03:20:13.903 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "warn_on_stray_daemons", 2024-09-12T03:20:13.903 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:13.903 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:13.903 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:13.904 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-12T03:20:13.904 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:13.904 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:13.904 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:13.904 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-12T03:20:13.904 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:13.904 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:13.904 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:13.904 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:13.904 INFO:teuthology.orchestra.run.smithi027.stdout: "warn_on_stray_hosts": { 2024-09-12T03:20:13.904 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "warn_on_stray_hosts", 2024-09-12T03:20:13.904 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:13.904 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:13.904 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:13.905 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-12T03:20:13.905 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:13.905 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:13.905 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:13.905 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-12T03:20:13.905 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:13.905 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:13.905 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:13.905 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:20:13.905 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:20:13.905 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:13.905 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:20:13.905 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "crash", 2024-09-12T03:20:13.906 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-12T03:20:13.906 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-12T03:20:13.906 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-12T03:20:13.906 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-12T03:20:13.906 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-12T03:20:13.906 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:13.906 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:13.906 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:13.906 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-12T03:20:13.906 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:13.906 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:13.906 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-12T03:20:13.906 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-12T03:20:13.907 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-12T03:20:13.907 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-12T03:20:13.907 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-12T03:20:13.907 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-12T03:20:13.907 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-12T03:20:13.907 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-12T03:20:13.907 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:13.907 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:13.907 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:13.907 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:13.907 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:13.907 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-12T03:20:13.907 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-12T03:20:13.907 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:13.908 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:13.908 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:13.908 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-12T03:20:13.908 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:13.908 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:13.908 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:13.908 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:13.908 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:13.908 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:13.908 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:13.908 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:13.908 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-12T03:20:13.908 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-12T03:20:13.909 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:13.909 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:13.909 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:13.909 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-12T03:20:13.909 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:13.909 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:13.909 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-12T03:20:13.909 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-12T03:20:13.909 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-12T03:20:13.909 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-12T03:20:13.909 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-12T03:20:13.909 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-12T03:20:13.909 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-12T03:20:13.909 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-12T03:20:13.910 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:13.910 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:13.910 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:13.910 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:13.910 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:13.910 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-12T03:20:13.910 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-12T03:20:13.910 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:13.910 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:13.910 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:13.910 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-12T03:20:13.910 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:13.910 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:13.910 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:13.911 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:13.911 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:13.911 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:13.911 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:13.911 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:13.911 INFO:teuthology.orchestra.run.smithi027.stdout: "retain_interval": { 2024-09-12T03:20:13.911 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "retain_interval", 2024-09-12T03:20:13.911 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-12T03:20:13.911 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:13.911 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:13.911 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "31536000", 2024-09-12T03:20:13.911 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:13.911 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:13.911 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:13.912 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-12T03:20:13.912 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:13.912 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:13.912 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:13.912 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:13.912 INFO:teuthology.orchestra.run.smithi027.stdout: "warn_recent_interval": { 2024-09-12T03:20:13.912 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "warn_recent_interval", 2024-09-12T03:20:13.912 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-12T03:20:13.912 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:13.912 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:13.912 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "1209600", 2024-09-12T03:20:13.912 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:13.912 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:13.912 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:13.913 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-12T03:20:13.913 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:13.913 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:13.913 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:13.913 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:20:13.913 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:20:13.913 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:13.913 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:20:13.913 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "dashboard", 2024-09-12T03:20:13.913 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-12T03:20:13.913 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-12T03:20:13.913 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-12T03:20:13.913 INFO:teuthology.orchestra.run.smithi027.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-12T03:20:13.914 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-12T03:20:13.914 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-12T03:20:13.914 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:13.914 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:13.914 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10", 2024-09-12T03:20:13.914 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:13.914 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:13.914 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:13.914 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:13.914 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:13.914 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:13.914 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:13.914 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:13.914 INFO:teuthology.orchestra.run.smithi027.stdout: "ALERTMANAGER_API_HOST": { 2024-09-12T03:20:13.914 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-12T03:20:13.915 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:13.915 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:13.915 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:13.915 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-12T03:20:13.915 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:13.915 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:13.915 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:13.915 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:13.915 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:13.915 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:13.915 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:13.915 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:13.915 INFO:teuthology.orchestra.run.smithi027.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-12T03:20:13.915 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-12T03:20:13.916 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:13.916 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:13.916 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:13.916 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-12T03:20:13.916 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:13.916 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:13.916 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:13.916 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:13.916 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:13.916 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:13.916 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:13.916 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:13.916 INFO:teuthology.orchestra.run.smithi027.stdout: "AUDIT_API_ENABLED": { 2024-09-12T03:20:13.917 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "AUDIT_API_ENABLED", 2024-09-12T03:20:13.917 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:13.917 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:13.917 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:13.917 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-12T03:20:13.917 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:13.917 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:13.917 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:13.917 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:13.917 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:13.917 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:13.917 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:13.917 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:13.918 INFO:teuthology.orchestra.run.smithi027.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-12T03:20:13.918 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-12T03:20:13.918 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:13.918 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:13.918 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:13.918 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-12T03:20:13.918 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:13.918 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:13.918 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:13.918 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:13.918 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:13.918 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:13.918 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:13.918 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:13.919 INFO:teuthology.orchestra.run.smithi027.stdout: "ENABLE_BROWSABLE_API": { 2024-09-12T03:20:13.919 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-12T03:20:13.919 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:13.919 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:13.919 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:13.919 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-12T03:20:13.919 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:13.919 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:13.919 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:13.919 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:13.919 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:13.919 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:13.919 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:13.920 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:13.920 INFO:teuthology.orchestra.run.smithi027.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-12T03:20:13.920 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-12T03:20:13.920 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:13.920 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:13.920 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:13.920 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-12T03:20:13.920 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:13.920 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:13.920 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:13.920 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:13.920 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:13.920 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:13.920 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:13.921 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:13.921 INFO:teuthology.orchestra.run.smithi027.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-12T03:20:13.921 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-12T03:20:13.921 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:13.921 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:13.921 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:13.921 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-12T03:20:13.921 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:13.921 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:13.921 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:13.921 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:13.921 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:13.921 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:13.921 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:13.922 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:13.922 INFO:teuthology.orchestra.run.smithi027.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-12T03:20:13.922 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-12T03:20:13.922 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:13.922 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:13.922 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:13.922 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-12T03:20:13.922 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:13.922 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:13.922 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:13.922 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:13.922 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:13.922 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:13.922 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:13.923 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:13.923 INFO:teuthology.orchestra.run.smithi027.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-12T03:20:13.923 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-12T03:20:13.923 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:13.923 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:13.923 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:13.923 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-12T03:20:13.923 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:13.923 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:13.923 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:13.923 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:13.923 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:13.923 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:13.923 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:13.924 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:13.924 INFO:teuthology.orchestra.run.smithi027.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-12T03:20:13.924 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-12T03:20:13.924 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:13.924 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:13.924 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:13.924 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-12T03:20:13.924 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:13.924 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:13.924 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:13.924 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:13.924 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:13.924 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:13.924 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:13.925 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:13.925 INFO:teuthology.orchestra.run.smithi027.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-12T03:20:13.925 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-12T03:20:13.925 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:13.925 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:13.925 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:13.925 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-12T03:20:13.925 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:13.925 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:13.925 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:13.925 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:13.925 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:13.925 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:13.925 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:13.926 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:13.926 INFO:teuthology.orchestra.run.smithi027.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-12T03:20:13.926 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-12T03:20:13.926 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:13.926 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:13.926 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:13.926 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-12T03:20:13.926 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:13.926 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:13.926 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:13.926 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:13.926 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:13.926 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:13.927 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:13.927 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:13.927 INFO:teuthology.orchestra.run.smithi027.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-12T03:20:13.927 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-12T03:20:13.927 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:13.927 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:13.927 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:13.927 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-12T03:20:13.927 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:13.927 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:13.927 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:13.927 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:13.927 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:13.927 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:13.928 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:13.928 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:13.928 INFO:teuthology.orchestra.run.smithi027.stdout: "GRAFANA_API_PASSWORD": { 2024-09-12T03:20:13.928 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-12T03:20:13.928 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:13.928 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:13.928 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:13.928 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "admin", 2024-09-12T03:20:13.928 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:13.928 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:13.928 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:13.928 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:13.928 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:13.928 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:13.929 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:13.929 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:13.929 INFO:teuthology.orchestra.run.smithi027.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-12T03:20:13.929 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-12T03:20:13.929 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:13.929 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:13.929 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:13.929 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-12T03:20:13.929 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:13.929 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:13.929 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:13.929 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:13.929 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:13.929 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:13.930 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:13.930 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:13.930 INFO:teuthology.orchestra.run.smithi027.stdout: "GRAFANA_API_URL": { 2024-09-12T03:20:13.930 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "GRAFANA_API_URL", 2024-09-12T03:20:13.930 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:13.930 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:13.930 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:13.930 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-12T03:20:13.930 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:13.930 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:13.930 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:13.930 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:13.930 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:13.930 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:13.931 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:13.931 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:13.931 INFO:teuthology.orchestra.run.smithi027.stdout: "GRAFANA_API_USERNAME": { 2024-09-12T03:20:13.931 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-12T03:20:13.931 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:13.931 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:13.931 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:13.931 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "admin", 2024-09-12T03:20:13.931 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:13.931 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:13.931 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:13.931 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:13.931 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:13.931 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:13.932 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:13.932 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:13.932 INFO:teuthology.orchestra.run.smithi027.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-12T03:20:13.932 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-12T03:20:13.932 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:13.932 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:13.932 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:13.932 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-12T03:20:13.932 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:13.932 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:13.932 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:13.932 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:13.932 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:13.932 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:13.933 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:13.933 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:13.933 INFO:teuthology.orchestra.run.smithi027.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-12T03:20:13.933 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-12T03:20:13.933 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:13.933 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:13.933 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:13.933 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-12T03:20:13.933 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:13.933 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:13.933 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:13.934 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:13.934 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:13.934 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:13.934 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:13.934 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:13.934 INFO:teuthology.orchestra.run.smithi027.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-12T03:20:13.934 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-12T03:20:13.934 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:13.934 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:13.934 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:13.934 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-12T03:20:13.934 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:13.934 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:13.935 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:13.935 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:13.935 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:13.935 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:13.935 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:13.935 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:13.935 INFO:teuthology.orchestra.run.smithi027.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-12T03:20:13.935 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-12T03:20:13.935 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:13.935 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:13.935 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:13.935 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-12T03:20:13.935 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:13.935 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:13.936 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:13.936 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:13.936 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:13.936 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:13.936 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:13.936 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:13.936 INFO:teuthology.orchestra.run.smithi027.stdout: "PROMETHEUS_API_HOST": { 2024-09-12T03:20:13.936 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-12T03:20:13.936 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:13.936 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:13.936 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:13.936 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-12T03:20:13.936 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:13.936 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:13.937 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:13.937 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:13.937 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:13.937 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:13.937 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:13.937 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:13.937 INFO:teuthology.orchestra.run.smithi027.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-12T03:20:13.937 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-12T03:20:13.937 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:13.937 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:13.937 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:13.937 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-12T03:20:13.937 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:13.937 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:13.938 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:13.938 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:13.938 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:13.938 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:13.938 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:13.938 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:13.938 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-12T03:20:13.938 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-12T03:20:13.938 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:13.938 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:13.938 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:13.938 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-12T03:20:13.938 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:13.938 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:13.939 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:13.939 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:13.939 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:13.939 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:13.939 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:13.939 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:13.939 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-12T03:20:13.939 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-12T03:20:13.939 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:13.939 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:13.939 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:13.939 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-12T03:20:13.939 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:13.939 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:13.940 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:13.940 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:13.940 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:13.940 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:13.940 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:13.940 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:13.940 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-12T03:20:13.940 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-12T03:20:13.940 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:13.940 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:13.940 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:13.940 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-12T03:20:13.940 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:13.941 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:13.941 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:13.941 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:13.941 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:13.941 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:13.941 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:13.941 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:13.941 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-12T03:20:13.941 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-12T03:20:13.941 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:13.941 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:13.941 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:13.941 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-12T03:20:13.941 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:13.942 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:13.942 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:13.942 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:13.942 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:13.942 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:13.942 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:13.942 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:13.942 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-12T03:20:13.942 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-12T03:20:13.942 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:13.942 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:13.942 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:13.942 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-12T03:20:13.942 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:13.943 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:13.943 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:13.943 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:13.943 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:13.943 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:13.943 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:13.943 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:13.943 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-12T03:20:13.943 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-12T03:20:13.943 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:13.943 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:13.943 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:13.943 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-12T03:20:13.943 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:13.944 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:13.944 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:13.944 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:13.944 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:13.944 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:13.944 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:13.944 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:13.944 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-12T03:20:13.944 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-12T03:20:13.944 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:13.945 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:13.945 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:13.945 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-12T03:20:13.945 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:13.945 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:13.945 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:13.945 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:13.945 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:13.945 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:13.945 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:13.945 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:13.945 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_ENABLED": { 2024-09-12T03:20:13.945 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-12T03:20:13.946 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:13.946 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:13.946 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:13.946 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-12T03:20:13.946 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:13.946 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:13.946 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:13.946 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:13.946 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:13.946 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:13.946 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:13.946 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:13.946 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-12T03:20:13.947 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-12T03:20:13.947 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:13.947 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:13.947 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:13.947 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-12T03:20:13.947 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:13.947 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:13.947 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:13.947 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:13.947 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:13.947 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:13.947 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:13.947 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:13.948 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-12T03:20:13.948 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-12T03:20:13.948 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-12T03:20:13.948 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:13.948 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:13.948 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10", 2024-09-12T03:20:13.948 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:13.948 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:13.948 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:13.948 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:13.948 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:13.948 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:13.948 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:13.949 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:13.949 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-12T03:20:13.949 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-12T03:20:13.949 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-12T03:20:13.949 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:13.949 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:13.949 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "8", 2024-09-12T03:20:13.949 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:13.949 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:13.949 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:13.949 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:13.949 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:13.949 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:13.949 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:13.950 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:13.950 INFO:teuthology.orchestra.run.smithi027.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-12T03:20:13.950 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-12T03:20:13.950 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-12T03:20:13.950 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:13.950 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:13.950 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "45", 2024-09-12T03:20:13.950 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:13.950 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:13.950 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:13.950 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:13.950 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:13.950 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:13.950 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:13.951 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:13.951 INFO:teuthology.orchestra.run.smithi027.stdout: "RGW_API_ACCESS_KEY": { 2024-09-12T03:20:13.951 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-12T03:20:13.951 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:13.951 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:13.951 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:13.951 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-12T03:20:13.951 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:13.951 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:13.951 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:13.951 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:13.951 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:13.951 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:13.952 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:13.952 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:13.952 INFO:teuthology.orchestra.run.smithi027.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-12T03:20:13.952 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-12T03:20:13.952 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:13.952 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:13.952 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:13.952 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "admin", 2024-09-12T03:20:13.952 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:13.952 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:13.952 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:13.952 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:13.952 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:13.952 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:13.953 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:13.953 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:13.953 INFO:teuthology.orchestra.run.smithi027.stdout: "RGW_API_SECRET_KEY": { 2024-09-12T03:20:13.953 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-12T03:20:13.953 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:13.953 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:13.953 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:13.953 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-12T03:20:13.953 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:13.953 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:13.953 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:13.953 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:13.953 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:13.954 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:13.954 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:13.954 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:13.954 INFO:teuthology.orchestra.run.smithi027.stdout: "RGW_API_SSL_VERIFY": { 2024-09-12T03:20:13.954 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-12T03:20:13.954 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:13.954 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:13.954 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:13.954 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-12T03:20:13.954 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:13.954 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:13.954 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:13.954 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:13.954 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:13.955 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:13.955 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:13.955 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:13.955 INFO:teuthology.orchestra.run.smithi027.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-12T03:20:13.955 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-12T03:20:13.955 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-12T03:20:13.955 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:13.955 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:13.955 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-12T03:20:13.955 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:13.955 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:13.955 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:13.955 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:13.955 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:13.956 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:13.956 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:13.956 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:13.956 INFO:teuthology.orchestra.run.smithi027.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-12T03:20:13.956 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-12T03:20:13.956 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-12T03:20:13.956 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:13.956 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:13.956 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10", 2024-09-12T03:20:13.956 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:13.956 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:13.956 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:13.956 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:13.956 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:13.957 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:13.957 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:13.957 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:13.957 INFO:teuthology.orchestra.run.smithi027.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-12T03:20:13.957 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-12T03:20:13.957 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-12T03:20:13.957 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:13.957 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:13.957 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "5", 2024-09-12T03:20:13.957 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:13.957 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:13.957 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:13.957 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:13.958 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:13.958 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:13.958 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:13.958 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:13.958 INFO:teuthology.orchestra.run.smithi027.stdout: "cross_origin_url": { 2024-09-12T03:20:13.958 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "cross_origin_url", 2024-09-12T03:20:13.958 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:13.958 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:13.958 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:13.958 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-12T03:20:13.958 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:13.958 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:13.958 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:13.958 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:13.959 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:13.959 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:13.959 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:13.959 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:13.959 INFO:teuthology.orchestra.run.smithi027.stdout: "crt_file": { 2024-09-12T03:20:13.959 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "crt_file", 2024-09-12T03:20:13.959 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:13.959 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:13.959 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:13.959 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-12T03:20:13.959 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:13.959 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:13.959 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:13.959 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:13.960 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:13.960 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:13.960 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:13.960 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:13.960 INFO:teuthology.orchestra.run.smithi027.stdout: "debug": { 2024-09-12T03:20:13.960 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "debug", 2024-09-12T03:20:13.960 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:13.960 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:13.960 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:13.960 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-12T03:20:13.960 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:13.960 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:13.961 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:13.961 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Enable/disable debug options", 2024-09-12T03:20:13.961 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:13.961 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:13.961 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:13.961 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:13.961 INFO:teuthology.orchestra.run.smithi027.stdout: "jwt_token_ttl": { 2024-09-12T03:20:13.961 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "jwt_token_ttl", 2024-09-12T03:20:13.961 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-12T03:20:13.961 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:13.961 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:13.962 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "28800", 2024-09-12T03:20:13.962 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:13.962 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:13.962 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:13.962 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:13.962 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:13.962 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:13.962 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:13.962 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:13.962 INFO:teuthology.orchestra.run.smithi027.stdout: "key_file": { 2024-09-12T03:20:13.962 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "key_file", 2024-09-12T03:20:13.962 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:13.962 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:13.962 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:13.963 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-12T03:20:13.963 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:13.963 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:13.963 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:13.963 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:13.963 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:13.963 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:13.963 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:13.963 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:13.963 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-12T03:20:13.963 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-12T03:20:13.963 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:13.963 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:13.963 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:13.964 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-12T03:20:13.964 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:13.964 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:13.964 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-12T03:20:13.964 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-12T03:20:13.964 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-12T03:20:13.964 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-12T03:20:13.964 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-12T03:20:13.964 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-12T03:20:13.964 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-12T03:20:13.964 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-12T03:20:13.964 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:13.964 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:13.964 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:13.965 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:13.965 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:13.965 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-12T03:20:13.965 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-12T03:20:13.965 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:13.965 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:13.965 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:13.965 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-12T03:20:13.965 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:13.965 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:13.965 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:13.965 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:13.965 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:13.965 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:13.966 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:13.966 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:13.966 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-12T03:20:13.966 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-12T03:20:13.966 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:13.966 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:13.966 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:13.966 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-12T03:20:13.966 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:13.966 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:13.966 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-12T03:20:13.966 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-12T03:20:13.966 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-12T03:20:13.966 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-12T03:20:13.967 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-12T03:20:13.967 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-12T03:20:13.967 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-12T03:20:13.967 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-12T03:20:13.967 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:13.967 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:13.967 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:13.967 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:13.967 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:13.967 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-12T03:20:13.967 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-12T03:20:13.967 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:13.967 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:13.967 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:13.968 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-12T03:20:13.968 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:13.968 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:13.968 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:13.968 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:13.968 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:13.968 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:13.968 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:13.968 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:13.968 INFO:teuthology.orchestra.run.smithi027.stdout: "motd": { 2024-09-12T03:20:13.968 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "motd", 2024-09-12T03:20:13.968 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:13.968 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:13.969 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:13.969 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-12T03:20:13.969 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:13.969 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:13.969 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:13.969 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "The message of the day", 2024-09-12T03:20:13.969 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:13.969 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:13.969 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:13.969 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:13.969 INFO:teuthology.orchestra.run.smithi027.stdout: "redirect_resolve_ip_addr": { 2024-09-12T03:20:13.969 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "redirect_resolve_ip_addr", 2024-09-12T03:20:13.969 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:13.969 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:13.970 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:13.970 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-12T03:20:13.970 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:13.970 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:13.970 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:13.970 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:13.970 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:13.970 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:13.970 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:13.970 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:13.970 INFO:teuthology.orchestra.run.smithi027.stdout: "server_addr": { 2024-09-12T03:20:13.970 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "server_addr", 2024-09-12T03:20:13.970 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:13.970 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:13.971 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:13.971 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "::", 2024-09-12T03:20:13.971 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:13.971 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:13.971 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:13.971 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:13.971 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:13.971 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:13.971 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:13.971 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:13.971 INFO:teuthology.orchestra.run.smithi027.stdout: "server_port": { 2024-09-12T03:20:13.971 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "server_port", 2024-09-12T03:20:13.971 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-12T03:20:13.971 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:13.972 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:13.972 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "8080", 2024-09-12T03:20:13.972 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:13.972 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:13.972 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:13.972 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:13.972 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:13.972 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:13.972 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:13.972 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:13.972 INFO:teuthology.orchestra.run.smithi027.stdout: "ssl": { 2024-09-12T03:20:13.972 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ssl", 2024-09-12T03:20:13.972 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:13.972 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:13.973 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:13.973 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-12T03:20:13.973 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:13.973 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:13.973 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:13.973 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:13.973 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:13.973 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:13.973 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:13.973 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:13.973 INFO:teuthology.orchestra.run.smithi027.stdout: "ssl_server_port": { 2024-09-12T03:20:13.973 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ssl_server_port", 2024-09-12T03:20:13.973 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-12T03:20:13.974 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:13.974 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:13.974 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "8443", 2024-09-12T03:20:13.974 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:13.974 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:13.974 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:13.974 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:13.974 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:13.974 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:13.974 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:13.974 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:13.974 INFO:teuthology.orchestra.run.smithi027.stdout: "standby_behaviour": { 2024-09-12T03:20:13.974 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "standby_behaviour", 2024-09-12T03:20:13.974 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:13.975 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:13.975 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:13.975 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "redirect", 2024-09-12T03:20:13.975 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:13.975 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:13.975 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-12T03:20:13.975 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-12T03:20:13.975 INFO:teuthology.orchestra.run.smithi027.stdout: "redirect" 2024-09-12T03:20:13.975 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-12T03:20:13.975 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:13.975 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:13.975 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:13.975 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:13.976 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:13.976 INFO:teuthology.orchestra.run.smithi027.stdout: "standby_error_status_code": { 2024-09-12T03:20:13.976 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "standby_error_status_code", 2024-09-12T03:20:13.976 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-12T03:20:13.976 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:13.976 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:13.976 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "500", 2024-09-12T03:20:13.976 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "400", 2024-09-12T03:20:13.976 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "599", 2024-09-12T03:20:13.976 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:13.976 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:13.976 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:13.976 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:13.976 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:13.977 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:13.977 INFO:teuthology.orchestra.run.smithi027.stdout: "url_prefix": { 2024-09-12T03:20:13.977 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "url_prefix", 2024-09-12T03:20:13.977 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:13.977 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:13.977 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:13.977 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-12T03:20:13.977 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:13.977 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:13.977 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:13.977 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:13.977 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:13.977 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:13.977 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:13.978 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:20:13.978 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:20:13.978 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:13.978 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:20:13.978 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "devicehealth", 2024-09-12T03:20:13.978 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-12T03:20:13.978 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-12T03:20:13.978 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-12T03:20:13.978 INFO:teuthology.orchestra.run.smithi027.stdout: "enable_monitoring": { 2024-09-12T03:20:13.978 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "enable_monitoring", 2024-09-12T03:20:13.978 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:13.978 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:13.978 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:13.978 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-12T03:20:13.979 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:13.979 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:13.979 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:13.979 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "monitor device health metrics", 2024-09-12T03:20:13.979 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:13.979 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:13.979 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:13.979 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:13.979 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-12T03:20:13.979 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-12T03:20:13.979 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:13.979 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:13.979 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:13.979 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-12T03:20:13.980 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:13.980 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:13.980 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-12T03:20:13.980 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-12T03:20:13.980 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-12T03:20:13.980 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-12T03:20:13.980 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-12T03:20:13.980 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-12T03:20:13.980 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-12T03:20:13.980 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-12T03:20:13.980 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:13.980 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:13.980 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:13.981 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:13.981 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:13.981 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-12T03:20:13.981 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-12T03:20:13.981 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:13.981 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:13.981 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:13.981 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-12T03:20:13.981 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:13.981 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:13.981 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:13.981 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:13.981 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:13.981 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:13.982 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:13.982 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:13.982 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-12T03:20:13.982 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-12T03:20:13.982 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:13.982 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:13.982 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:13.982 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-12T03:20:13.982 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:13.982 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:13.982 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-12T03:20:13.982 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-12T03:20:13.982 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-12T03:20:13.982 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-12T03:20:13.982 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-12T03:20:13.983 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-12T03:20:13.983 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-12T03:20:13.983 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-12T03:20:13.983 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:13.983 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:13.983 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:13.983 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:13.983 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:13.983 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-12T03:20:13.983 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-12T03:20:13.983 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:13.983 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:13.983 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:13.983 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-12T03:20:13.984 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:13.984 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:13.984 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:13.984 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:13.984 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:13.984 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:13.984 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:13.984 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:13.984 INFO:teuthology.orchestra.run.smithi027.stdout: "mark_out_threshold": { 2024-09-12T03:20:13.984 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "mark_out_threshold", 2024-09-12T03:20:13.984 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-12T03:20:13.984 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:13.984 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:13.984 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "2419200", 2024-09-12T03:20:13.985 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:13.985 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:13.985 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:13.985 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-12T03:20:13.985 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:13.985 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:13.985 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:13.985 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:13.985 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_name": { 2024-09-12T03:20:13.985 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "pool_name", 2024-09-12T03:20:13.985 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:13.985 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:13.985 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:13.986 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "device_health_metrics", 2024-09-12T03:20:13.986 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:13.986 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:13.986 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:13.986 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-12T03:20:13.986 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:13.986 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:13.986 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:13.986 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:13.986 INFO:teuthology.orchestra.run.smithi027.stdout: "retention_period": { 2024-09-12T03:20:13.986 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "retention_period", 2024-09-12T03:20:13.986 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-12T03:20:13.986 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:13.986 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:13.987 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "15552000", 2024-09-12T03:20:13.987 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:13.987 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:13.987 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:13.987 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "how long to retain device health metrics", 2024-09-12T03:20:13.987 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:13.987 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:13.987 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:13.987 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:13.987 INFO:teuthology.orchestra.run.smithi027.stdout: "scrape_frequency": { 2024-09-12T03:20:13.987 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "scrape_frequency", 2024-09-12T03:20:13.987 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-12T03:20:13.987 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:13.987 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:13.988 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "86400", 2024-09-12T03:20:13.988 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:13.988 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:13.988 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:13.988 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-12T03:20:13.988 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:13.988 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:13.988 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:13.988 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:13.988 INFO:teuthology.orchestra.run.smithi027.stdout: "self_heal": { 2024-09-12T03:20:13.988 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "self_heal", 2024-09-12T03:20:13.988 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:13.989 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:13.989 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:13.989 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-12T03:20:13.989 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:13.989 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:13.989 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:13.989 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-12T03:20:13.989 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:13.989 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:13.989 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:13.989 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:13.989 INFO:teuthology.orchestra.run.smithi027.stdout: "sleep_interval": { 2024-09-12T03:20:13.989 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sleep_interval", 2024-09-12T03:20:13.989 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-12T03:20:13.990 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:13.990 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:13.990 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "600", 2024-09-12T03:20:13.990 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:13.990 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:13.990 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:13.990 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "how frequently to wake up and check device health", 2024-09-12T03:20:13.990 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:13.990 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:13.990 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:13.990 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:13.990 INFO:teuthology.orchestra.run.smithi027.stdout: "warn_threshold": { 2024-09-12T03:20:13.990 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "warn_threshold", 2024-09-12T03:20:13.990 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-12T03:20:13.991 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:13.991 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:13.991 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "7257600", 2024-09-12T03:20:13.991 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:13.991 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:13.991 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:13.991 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-12T03:20:13.991 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:13.991 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:13.991 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:13.991 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:20:13.991 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:20:13.991 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:13.991 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:20:13.992 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "diskprediction_local", 2024-09-12T03:20:13.992 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-12T03:20:13.992 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-12T03:20:13.992 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-12T03:20:13.992 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-12T03:20:13.992 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-12T03:20:13.992 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:13.992 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:13.992 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:13.992 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-12T03:20:13.992 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:13.992 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:13.992 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-12T03:20:13.992 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-12T03:20:13.993 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-12T03:20:13.993 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-12T03:20:13.993 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-12T03:20:13.993 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-12T03:20:13.993 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-12T03:20:13.993 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-12T03:20:13.993 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:13.993 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:13.993 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:13.993 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:13.993 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:13.993 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-12T03:20:13.993 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-12T03:20:13.993 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:13.994 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:13.994 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:13.994 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-12T03:20:13.994 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:13.994 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:13.994 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:13.994 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:13.994 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:13.994 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:13.994 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:13.994 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:13.994 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-12T03:20:13.994 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-12T03:20:13.994 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:13.995 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:13.995 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:13.995 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-12T03:20:13.995 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:13.995 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:13.995 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-12T03:20:13.995 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-12T03:20:13.995 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-12T03:20:13.995 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-12T03:20:13.995 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-12T03:20:13.995 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-12T03:20:13.995 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-12T03:20:13.995 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-12T03:20:13.995 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:13.996 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:13.996 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:13.996 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:13.996 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:13.996 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-12T03:20:13.996 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-12T03:20:13.996 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:13.996 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:13.996 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:13.996 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-12T03:20:13.996 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:13.996 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:13.996 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:13.996 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:13.997 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:13.997 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:13.997 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:13.997 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:13.997 INFO:teuthology.orchestra.run.smithi027.stdout: "predict_interval": { 2024-09-12T03:20:13.997 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "predict_interval", 2024-09-12T03:20:13.997 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:13.997 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:13.997 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:13.997 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "86400", 2024-09-12T03:20:13.997 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:13.997 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:13.997 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:13.998 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:13.998 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:13.998 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:13.998 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:13.998 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:13.998 INFO:teuthology.orchestra.run.smithi027.stdout: "predictor_model": { 2024-09-12T03:20:13.998 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "predictor_model", 2024-09-12T03:20:13.998 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:13.998 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:13.998 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:13.998 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "prophetstor", 2024-09-12T03:20:13.998 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:13.998 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:13.998 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:13.998 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:13.999 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:13.999 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:13.999 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:13.999 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:13.999 INFO:teuthology.orchestra.run.smithi027.stdout: "sleep_interval": { 2024-09-12T03:20:13.999 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sleep_interval", 2024-09-12T03:20:13.999 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:13.999 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:13.999 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:13.999 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "600", 2024-09-12T03:20:13.999 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:13.999 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:13.999 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.000 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.000 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.000 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.000 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.000 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:20:14.000 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:20:14.000 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.000 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:20:14.000 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "influx", 2024-09-12T03:20:14.000 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": false, 2024-09-12T03:20:14.000 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "influxdb python module not found", 2024-09-12T03:20:14.000 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-12T03:20:14.000 INFO:teuthology.orchestra.run.smithi027.stdout: "batch_size": { 2024-09-12T03:20:14.000 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "batch_size", 2024-09-12T03:20:14.001 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-12T03:20:14.001 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.001 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.001 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "5000", 2024-09-12T03:20:14.001 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.001 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.001 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.001 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-12T03:20:14.001 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.001 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.001 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.001 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.001 INFO:teuthology.orchestra.run.smithi027.stdout: "database": { 2024-09-12T03:20:14.002 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "database", 2024-09-12T03:20:14.002 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.002 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.002 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.002 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "ceph", 2024-09-12T03:20:14.002 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.002 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.002 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.002 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-12T03:20:14.002 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.002 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.002 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.002 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.002 INFO:teuthology.orchestra.run.smithi027.stdout: "hostname": { 2024-09-12T03:20:14.003 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "hostname", 2024-09-12T03:20:14.003 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.003 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.003 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.003 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-12T03:20:14.003 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.003 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.003 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.003 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "InfluxDB server hostname", 2024-09-12T03:20:14.003 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.003 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.003 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.003 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.003 INFO:teuthology.orchestra.run.smithi027.stdout: "interval": { 2024-09-12T03:20:14.004 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "interval", 2024-09-12T03:20:14.004 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-12T03:20:14.004 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.004 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.004 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "30", 2024-09-12T03:20:14.004 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "5", 2024-09-12T03:20:14.004 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.004 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.004 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-12T03:20:14.004 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.004 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.004 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.004 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.005 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-12T03:20:14.005 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-12T03:20:14.005 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.005 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.005 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.005 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-12T03:20:14.005 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.005 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.005 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-12T03:20:14.005 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-12T03:20:14.005 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-12T03:20:14.005 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-12T03:20:14.005 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-12T03:20:14.005 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-12T03:20:14.006 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-12T03:20:14.006 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-12T03:20:14.006 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.006 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.006 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.006 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.006 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.006 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-12T03:20:14.006 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-12T03:20:14.006 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.006 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.006 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.006 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-12T03:20:14.006 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.007 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.007 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.007 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.007 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.007 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.007 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.007 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.007 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-12T03:20:14.007 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-12T03:20:14.007 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.007 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.007 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.007 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-12T03:20:14.007 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.008 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.008 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-12T03:20:14.008 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-12T03:20:14.008 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-12T03:20:14.008 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-12T03:20:14.008 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-12T03:20:14.008 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-12T03:20:14.008 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-12T03:20:14.008 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-12T03:20:14.008 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.008 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.008 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.008 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.008 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.009 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-12T03:20:14.009 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-12T03:20:14.009 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.009 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.009 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.009 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-12T03:20:14.009 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.009 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.009 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.009 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.009 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.009 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.009 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.009 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.010 INFO:teuthology.orchestra.run.smithi027.stdout: "password": { 2024-09-12T03:20:14.010 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "password", 2024-09-12T03:20:14.010 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.010 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.010 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.010 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-12T03:20:14.010 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.010 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.010 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.010 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "password of InfluxDB server user", 2024-09-12T03:20:14.010 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.010 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.010 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.010 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.011 INFO:teuthology.orchestra.run.smithi027.stdout: "port": { 2024-09-12T03:20:14.011 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "port", 2024-09-12T03:20:14.011 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-12T03:20:14.011 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.011 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.011 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "8086", 2024-09-12T03:20:14.011 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.011 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.011 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.011 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "InfluxDB server port", 2024-09-12T03:20:14.011 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.011 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.011 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.011 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.012 INFO:teuthology.orchestra.run.smithi027.stdout: "ssl": { 2024-09-12T03:20:14.012 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ssl", 2024-09-12T03:20:14.012 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.012 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.012 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.012 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "false", 2024-09-12T03:20:14.012 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.012 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.012 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.012 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-12T03:20:14.012 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.012 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.012 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.013 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.013 INFO:teuthology.orchestra.run.smithi027.stdout: "threads": { 2024-09-12T03:20:14.013 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "threads", 2024-09-12T03:20:14.013 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-12T03:20:14.013 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.013 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.013 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "5", 2024-09-12T03:20:14.013 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "1", 2024-09-12T03:20:14.013 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "32", 2024-09-12T03:20:14.013 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.013 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-12T03:20:14.013 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.013 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.013 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.014 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.014 INFO:teuthology.orchestra.run.smithi027.stdout: "username": { 2024-09-12T03:20:14.014 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "username", 2024-09-12T03:20:14.014 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.014 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.014 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.014 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-12T03:20:14.014 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.014 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.014 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.014 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "username of InfluxDB server user", 2024-09-12T03:20:14.014 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.014 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.014 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.015 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.015 INFO:teuthology.orchestra.run.smithi027.stdout: "verify_ssl": { 2024-09-12T03:20:14.015 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "verify_ssl", 2024-09-12T03:20:14.015 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.015 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.015 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.015 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "true", 2024-09-12T03:20:14.015 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.015 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.015 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.015 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-12T03:20:14.015 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.015 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.016 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.016 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:20:14.016 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:20:14.016 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.016 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:20:14.016 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "insights", 2024-09-12T03:20:14.016 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-12T03:20:14.016 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-12T03:20:14.016 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-12T03:20:14.016 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-12T03:20:14.016 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-12T03:20:14.016 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.016 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.017 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.017 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-12T03:20:14.017 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.017 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.017 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-12T03:20:14.017 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-12T03:20:14.017 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-12T03:20:14.017 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-12T03:20:14.017 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-12T03:20:14.017 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-12T03:20:14.017 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-12T03:20:14.017 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-12T03:20:14.017 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.017 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.018 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.018 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.018 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.018 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-12T03:20:14.018 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-12T03:20:14.018 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.018 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.018 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.018 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-12T03:20:14.018 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.018 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.018 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.018 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.018 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.019 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.019 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.019 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.019 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-12T03:20:14.019 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-12T03:20:14.019 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.019 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.019 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.019 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-12T03:20:14.019 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.019 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.019 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-12T03:20:14.019 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-12T03:20:14.019 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-12T03:20:14.020 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-12T03:20:14.020 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-12T03:20:14.020 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-12T03:20:14.020 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-12T03:20:14.020 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-12T03:20:14.020 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.020 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.020 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.020 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.020 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.020 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-12T03:20:14.020 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-12T03:20:14.020 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.020 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.021 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.021 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-12T03:20:14.021 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.021 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.021 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.021 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.021 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.021 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.021 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.021 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:20:14.021 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:20:14.021 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.021 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:20:14.021 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "iostat", 2024-09-12T03:20:14.022 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-12T03:20:14.022 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-12T03:20:14.022 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-12T03:20:14.022 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-12T03:20:14.022 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-12T03:20:14.022 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.022 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.022 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.022 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-12T03:20:14.022 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.022 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.022 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-12T03:20:14.022 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-12T03:20:14.022 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-12T03:20:14.023 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-12T03:20:14.023 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-12T03:20:14.023 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-12T03:20:14.023 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-12T03:20:14.023 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-12T03:20:14.023 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.023 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.023 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.023 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.023 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.023 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-12T03:20:14.023 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-12T03:20:14.023 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.023 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.024 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.024 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-12T03:20:14.024 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.024 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.024 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.024 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.024 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.024 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.024 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.024 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.024 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-12T03:20:14.024 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-12T03:20:14.024 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.024 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.025 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.025 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-12T03:20:14.025 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.025 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.025 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-12T03:20:14.025 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-12T03:20:14.025 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-12T03:20:14.025 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-12T03:20:14.025 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-12T03:20:14.025 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-12T03:20:14.025 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-12T03:20:14.025 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-12T03:20:14.025 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.025 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.026 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.026 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.026 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.026 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-12T03:20:14.026 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-12T03:20:14.026 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.026 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.026 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.026 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-12T03:20:14.026 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.026 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.026 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.026 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.026 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.027 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.027 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.027 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:20:14.027 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:20:14.027 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.027 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:20:14.027 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "localpool", 2024-09-12T03:20:14.027 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-12T03:20:14.027 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-12T03:20:14.027 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-12T03:20:14.027 INFO:teuthology.orchestra.run.smithi027.stdout: "failure_domain": { 2024-09-12T03:20:14.027 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "failure_domain", 2024-09-12T03:20:14.027 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.027 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.028 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.028 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "host", 2024-09-12T03:20:14.028 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.028 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.028 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.028 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "failure domain for any created local pool", 2024-09-12T03:20:14.028 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-12T03:20:14.028 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.028 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.029 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.029 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-12T03:20:14.029 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-12T03:20:14.029 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.029 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.029 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.029 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-12T03:20:14.029 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.029 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.029 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-12T03:20:14.029 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-12T03:20:14.029 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-12T03:20:14.029 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-12T03:20:14.030 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-12T03:20:14.030 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-12T03:20:14.030 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-12T03:20:14.030 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-12T03:20:14.030 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.030 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.030 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.030 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.030 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.030 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-12T03:20:14.030 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-12T03:20:14.030 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.030 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.030 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.031 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-12T03:20:14.031 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.031 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.031 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.031 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.031 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.031 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.031 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.031 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.031 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-12T03:20:14.031 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-12T03:20:14.031 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.031 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.032 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.032 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-12T03:20:14.032 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.032 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.032 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-12T03:20:14.032 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-12T03:20:14.032 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-12T03:20:14.032 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-12T03:20:14.032 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-12T03:20:14.032 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-12T03:20:14.032 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-12T03:20:14.032 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-12T03:20:14.032 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.032 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.033 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.033 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.033 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.033 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-12T03:20:14.033 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-12T03:20:14.033 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.033 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.033 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.033 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-12T03:20:14.033 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.033 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.033 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.034 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.034 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.034 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.034 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.034 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.034 INFO:teuthology.orchestra.run.smithi027.stdout: "min_size": { 2024-09-12T03:20:14.034 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "min_size", 2024-09-12T03:20:14.034 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-12T03:20:14.034 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.034 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.034 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-12T03:20:14.034 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.034 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.035 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.035 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "default min_size for any created local pool", 2024-09-12T03:20:14.035 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-12T03:20:14.035 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.035 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.035 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.035 INFO:teuthology.orchestra.run.smithi027.stdout: "num_rep": { 2024-09-12T03:20:14.035 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "num_rep", 2024-09-12T03:20:14.035 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-12T03:20:14.035 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.035 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.035 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "3", 2024-09-12T03:20:14.035 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.035 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.036 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.036 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "default replica count for any created local pool", 2024-09-12T03:20:14.036 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.036 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.036 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.036 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.036 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num": { 2024-09-12T03:20:14.036 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "pg_num", 2024-09-12T03:20:14.036 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-12T03:20:14.036 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.036 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.036 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "128", 2024-09-12T03:20:14.036 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.036 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.037 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.037 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "default pg_num for any created local pool", 2024-09-12T03:20:14.037 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.037 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.037 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.037 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.037 INFO:teuthology.orchestra.run.smithi027.stdout: "prefix": { 2024-09-12T03:20:14.037 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "prefix", 2024-09-12T03:20:14.037 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.037 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.037 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.037 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-12T03:20:14.037 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.040 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.040 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.040 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "name prefix for any created local pool", 2024-09-12T03:20:14.040 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.041 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.041 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.041 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.041 INFO:teuthology.orchestra.run.smithi027.stdout: "subtree": { 2024-09-12T03:20:14.041 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "subtree", 2024-09-12T03:20:14.041 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.041 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.041 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.041 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "rack", 2024-09-12T03:20:14.041 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.042 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.042 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.042 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-12T03:20:14.042 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-12T03:20:14.042 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.042 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.042 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:20:14.042 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:20:14.042 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.042 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:20:14.042 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "mds_autoscaler", 2024-09-12T03:20:14.042 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-12T03:20:14.042 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-12T03:20:14.042 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-12T03:20:14.043 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-12T03:20:14.043 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-12T03:20:14.043 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.043 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.043 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.043 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-12T03:20:14.043 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.043 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.043 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-12T03:20:14.043 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-12T03:20:14.043 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-12T03:20:14.043 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-12T03:20:14.043 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-12T03:20:14.044 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-12T03:20:14.044 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-12T03:20:14.044 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-12T03:20:14.044 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.044 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.044 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.044 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.044 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.044 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-12T03:20:14.044 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-12T03:20:14.044 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.044 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.044 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.045 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-12T03:20:14.045 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.045 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.045 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.045 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.045 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.045 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.045 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.045 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.045 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-12T03:20:14.046 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-12T03:20:14.046 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.046 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.046 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.046 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-12T03:20:14.046 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.046 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.046 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-12T03:20:14.046 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-12T03:20:14.046 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-12T03:20:14.046 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-12T03:20:14.046 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-12T03:20:14.046 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-12T03:20:14.047 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-12T03:20:14.047 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-12T03:20:14.047 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.047 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.047 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.047 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.047 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.047 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-12T03:20:14.047 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-12T03:20:14.047 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.048 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.048 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.048 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-12T03:20:14.048 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.048 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.048 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.048 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.049 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.049 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.049 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.049 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:20:14.049 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:20:14.049 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.049 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:20:14.049 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "mirroring", 2024-09-12T03:20:14.049 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-12T03:20:14.050 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-12T03:20:14.050 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-12T03:20:14.050 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-12T03:20:14.050 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-12T03:20:14.050 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.050 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.050 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.050 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-12T03:20:14.050 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.050 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.050 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-12T03:20:14.050 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-12T03:20:14.051 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-12T03:20:14.051 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-12T03:20:14.051 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-12T03:20:14.051 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-12T03:20:14.051 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-12T03:20:14.051 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-12T03:20:14.051 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.051 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.051 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.051 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.051 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.051 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-12T03:20:14.052 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-12T03:20:14.052 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.052 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.052 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.052 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-12T03:20:14.052 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.052 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.052 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.052 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.052 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.053 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.053 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.053 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.053 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-12T03:20:14.053 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-12T03:20:14.053 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.053 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.053 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.053 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-12T03:20:14.053 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.053 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.053 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-12T03:20:14.054 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-12T03:20:14.054 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-12T03:20:14.054 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-12T03:20:14.054 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-12T03:20:14.054 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-12T03:20:14.054 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-12T03:20:14.054 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-12T03:20:14.054 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.054 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.054 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.054 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.054 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.054 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-12T03:20:14.055 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-12T03:20:14.055 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.055 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.055 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.055 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-12T03:20:14.055 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.055 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.055 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.055 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.055 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.055 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.055 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.055 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:20:14.056 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:20:14.056 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.056 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:20:14.056 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "nfs", 2024-09-12T03:20:14.056 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-12T03:20:14.056 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-12T03:20:14.056 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-12T03:20:14.056 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-12T03:20:14.056 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-12T03:20:14.056 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.056 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.056 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.057 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-12T03:20:14.057 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.057 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.057 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-12T03:20:14.057 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-12T03:20:14.057 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-12T03:20:14.057 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-12T03:20:14.057 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-12T03:20:14.057 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-12T03:20:14.057 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-12T03:20:14.057 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-12T03:20:14.058 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.058 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.058 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.058 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.058 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.058 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-12T03:20:14.058 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-12T03:20:14.059 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.059 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.059 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.059 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-12T03:20:14.059 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.059 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.059 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.059 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.060 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.060 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.060 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.060 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.060 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-12T03:20:14.060 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-12T03:20:14.060 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.060 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.060 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.060 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-12T03:20:14.060 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.061 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.061 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-12T03:20:14.061 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-12T03:20:14.061 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-12T03:20:14.061 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-12T03:20:14.061 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-12T03:20:14.061 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-12T03:20:14.061 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-12T03:20:14.062 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-12T03:20:14.062 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.062 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.062 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.062 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.062 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.062 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-12T03:20:14.062 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-12T03:20:14.062 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.063 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.063 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.063 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-12T03:20:14.063 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.063 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.063 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.063 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.063 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.063 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.063 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.063 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:20:14.063 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:20:14.064 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.064 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:20:14.064 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "orchestrator", 2024-09-12T03:20:14.064 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-12T03:20:14.064 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-12T03:20:14.064 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-12T03:20:14.064 INFO:teuthology.orchestra.run.smithi027.stdout: "fail_fs": { 2024-09-12T03:20:14.064 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "fail_fs", 2024-09-12T03:20:14.064 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.064 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.065 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.065 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-12T03:20:14.065 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.065 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.065 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.065 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-12T03:20:14.065 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.065 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.065 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.065 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.066 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-12T03:20:14.066 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-12T03:20:14.066 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.066 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.066 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.066 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-12T03:20:14.066 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.066 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.066 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-12T03:20:14.067 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-12T03:20:14.067 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-12T03:20:14.067 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-12T03:20:14.067 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-12T03:20:14.067 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-12T03:20:14.067 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-12T03:20:14.067 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-12T03:20:14.067 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.067 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.067 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.068 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.068 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.068 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-12T03:20:14.068 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-12T03:20:14.068 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.068 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.068 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.068 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-12T03:20:14.068 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.068 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.068 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.068 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.069 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.069 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.069 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.069 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.069 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-12T03:20:14.069 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-12T03:20:14.069 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.069 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.069 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.069 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-12T03:20:14.069 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.070 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.070 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-12T03:20:14.070 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-12T03:20:14.070 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-12T03:20:14.070 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-12T03:20:14.070 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-12T03:20:14.070 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-12T03:20:14.070 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-12T03:20:14.070 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-12T03:20:14.070 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.070 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.071 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.071 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.071 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.071 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-12T03:20:14.071 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-12T03:20:14.071 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.071 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.071 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.071 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-12T03:20:14.071 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.071 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.071 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.072 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.072 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.072 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.072 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.072 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.072 INFO:teuthology.orchestra.run.smithi027.stdout: "orchestrator": { 2024-09-12T03:20:14.072 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "orchestrator", 2024-09-12T03:20:14.072 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.072 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.072 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.072 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-12T03:20:14.072 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.073 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.073 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-12T03:20:14.073 INFO:teuthology.orchestra.run.smithi027.stdout: "cephadm", 2024-09-12T03:20:14.073 INFO:teuthology.orchestra.run.smithi027.stdout: "rook", 2024-09-12T03:20:14.073 INFO:teuthology.orchestra.run.smithi027.stdout: "test_orchestrator" 2024-09-12T03:20:14.073 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-12T03:20:14.073 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Orchestrator backend", 2024-09-12T03:20:14.073 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.073 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.073 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.073 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:20:14.073 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:20:14.074 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.074 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:20:14.074 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "osd_perf_query", 2024-09-12T03:20:14.074 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-12T03:20:14.074 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-12T03:20:14.074 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-12T03:20:14.074 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-12T03:20:14.074 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-12T03:20:14.074 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.074 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.074 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.075 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-12T03:20:14.075 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.075 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.075 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-12T03:20:14.075 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-12T03:20:14.075 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-12T03:20:14.075 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-12T03:20:14.075 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-12T03:20:14.075 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-12T03:20:14.075 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-12T03:20:14.075 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-12T03:20:14.075 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.076 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.076 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.076 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.076 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.076 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-12T03:20:14.076 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-12T03:20:14.076 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.076 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.076 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.076 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-12T03:20:14.077 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.077 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.077 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.077 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.077 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.077 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.077 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.077 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.077 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-12T03:20:14.077 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-12T03:20:14.077 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.078 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.078 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.078 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-12T03:20:14.078 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.078 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.078 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-12T03:20:14.078 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-12T03:20:14.078 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-12T03:20:14.078 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-12T03:20:14.078 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-12T03:20:14.079 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-12T03:20:14.079 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-12T03:20:14.079 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-12T03:20:14.079 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.079 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.079 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.079 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.079 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.079 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-12T03:20:14.079 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-12T03:20:14.079 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.080 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.080 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.080 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-12T03:20:14.080 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.080 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.080 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.080 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.080 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.080 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.080 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.081 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:20:14.081 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:20:14.081 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.081 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:20:14.081 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "osd_support", 2024-09-12T03:20:14.081 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-12T03:20:14.081 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-12T03:20:14.081 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-12T03:20:14.081 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-12T03:20:14.082 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-12T03:20:14.082 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.082 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.082 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.082 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-12T03:20:14.082 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.082 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.082 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-12T03:20:14.082 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-12T03:20:14.082 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-12T03:20:14.082 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-12T03:20:14.082 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-12T03:20:14.083 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-12T03:20:14.083 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-12T03:20:14.083 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-12T03:20:14.083 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.083 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.083 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.083 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.083 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.083 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-12T03:20:14.083 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-12T03:20:14.083 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.084 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.084 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.084 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-12T03:20:14.084 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.084 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.084 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.084 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.084 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.084 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.084 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.085 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.085 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-12T03:20:14.085 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-12T03:20:14.085 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.085 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.085 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.085 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-12T03:20:14.085 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.085 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.085 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-12T03:20:14.085 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-12T03:20:14.085 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-12T03:20:14.086 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-12T03:20:14.086 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-12T03:20:14.086 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-12T03:20:14.086 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-12T03:20:14.086 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-12T03:20:14.086 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.086 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.086 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.086 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.086 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.086 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-12T03:20:14.086 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-12T03:20:14.087 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.087 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.087 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.087 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-12T03:20:14.087 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.087 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.087 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.087 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.087 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.087 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.088 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.088 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:20:14.088 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:20:14.088 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.088 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:20:14.088 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "pg_autoscaler", 2024-09-12T03:20:14.088 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-12T03:20:14.088 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-12T03:20:14.088 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-12T03:20:14.088 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-12T03:20:14.088 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-12T03:20:14.089 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.089 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.089 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.089 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-12T03:20:14.089 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.089 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.089 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-12T03:20:14.089 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-12T03:20:14.090 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-12T03:20:14.090 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-12T03:20:14.090 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-12T03:20:14.090 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-12T03:20:14.090 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-12T03:20:14.090 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-12T03:20:14.090 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.090 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.090 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.090 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.090 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.091 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-12T03:20:14.091 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-12T03:20:14.091 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.091 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.091 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.091 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-12T03:20:14.091 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.091 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.091 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.091 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.091 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.091 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.092 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.092 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.092 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-12T03:20:14.092 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-12T03:20:14.092 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.092 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.092 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.092 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-12T03:20:14.092 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.093 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.093 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-12T03:20:14.093 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-12T03:20:14.093 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-12T03:20:14.093 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-12T03:20:14.093 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-12T03:20:14.093 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-12T03:20:14.093 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-12T03:20:14.093 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-12T03:20:14.093 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.093 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.093 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.094 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.094 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.094 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-12T03:20:14.094 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-12T03:20:14.094 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.094 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.094 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.094 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-12T03:20:14.094 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.094 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.094 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.094 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.094 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.094 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.095 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.095 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.095 INFO:teuthology.orchestra.run.smithi027.stdout: "sleep_interval": { 2024-09-12T03:20:14.095 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sleep_interval", 2024-09-12T03:20:14.095 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-12T03:20:14.095 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.095 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.095 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "60", 2024-09-12T03:20:14.095 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.095 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.095 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.096 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.096 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.096 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.096 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.096 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.096 INFO:teuthology.orchestra.run.smithi027.stdout: "threshold": { 2024-09-12T03:20:14.096 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "threshold", 2024-09-12T03:20:14.096 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "float", 2024-09-12T03:20:14.096 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.097 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.097 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "3.0", 2024-09-12T03:20:14.097 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "1.0", 2024-09-12T03:20:14.097 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.097 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.097 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "scaling threshold", 2024-09-12T03:20:14.097 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-12T03:20:14.097 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.097 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.098 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:20:14.098 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:20:14.098 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.098 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:20:14.098 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "progress", 2024-09-12T03:20:14.098 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-12T03:20:14.098 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-12T03:20:14.098 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-12T03:20:14.098 INFO:teuthology.orchestra.run.smithi027.stdout: "allow_pg_recovery_event": { 2024-09-12T03:20:14.098 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "allow_pg_recovery_event", 2024-09-12T03:20:14.098 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.099 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.099 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.099 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-12T03:20:14.099 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.099 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.099 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.099 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-12T03:20:14.099 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.099 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.099 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.099 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.100 INFO:teuthology.orchestra.run.smithi027.stdout: "enabled": { 2024-09-12T03:20:14.100 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "enabled", 2024-09-12T03:20:14.100 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.100 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.100 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.100 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-12T03:20:14.100 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.100 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.100 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.100 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.100 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.100 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.100 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.101 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.101 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-12T03:20:14.101 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-12T03:20:14.101 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.101 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.101 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.101 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-12T03:20:14.101 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.101 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.101 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-12T03:20:14.101 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-12T03:20:14.101 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-12T03:20:14.102 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-12T03:20:14.102 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-12T03:20:14.102 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-12T03:20:14.102 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-12T03:20:14.102 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-12T03:20:14.102 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.102 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.102 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.102 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.102 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.103 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-12T03:20:14.103 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-12T03:20:14.103 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.103 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.103 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.103 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-12T03:20:14.103 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.103 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.104 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.104 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.104 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.104 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.104 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.104 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.104 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-12T03:20:14.105 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-12T03:20:14.105 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.105 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.105 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.105 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-12T03:20:14.105 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.105 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.105 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-12T03:20:14.106 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-12T03:20:14.106 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-12T03:20:14.106 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-12T03:20:14.106 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-12T03:20:14.106 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-12T03:20:14.106 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-12T03:20:14.106 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-12T03:20:14.106 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.106 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.106 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.106 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.107 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.107 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-12T03:20:14.107 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-12T03:20:14.107 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.107 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.107 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.107 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-12T03:20:14.107 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.107 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.107 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.107 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.107 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.108 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.108 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.108 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.108 INFO:teuthology.orchestra.run.smithi027.stdout: "max_completed_events": { 2024-09-12T03:20:14.108 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "max_completed_events", 2024-09-12T03:20:14.108 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-12T03:20:14.108 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.108 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.108 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "50", 2024-09-12T03:20:14.108 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.108 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.108 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.109 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "number of past completed events to remember", 2024-09-12T03:20:14.109 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.109 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.109 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.109 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.109 INFO:teuthology.orchestra.run.smithi027.stdout: "sleep_interval": { 2024-09-12T03:20:14.109 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sleep_interval", 2024-09-12T03:20:14.109 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-12T03:20:14.109 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.109 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.110 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "5", 2024-09-12T03:20:14.110 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.110 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.110 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.110 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "how long the module is going to sleep", 2024-09-12T03:20:14.110 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.110 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.110 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.110 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:20:14.111 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:20:14.111 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.111 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:20:14.111 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "prometheus", 2024-09-12T03:20:14.111 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-12T03:20:14.111 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-12T03:20:14.111 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-12T03:20:14.111 INFO:teuthology.orchestra.run.smithi027.stdout: "cache": { 2024-09-12T03:20:14.111 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "cache", 2024-09-12T03:20:14.111 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.112 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.112 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.112 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-12T03:20:14.112 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.112 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.112 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.112 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.112 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.112 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.112 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.112 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.113 INFO:teuthology.orchestra.run.smithi027.stdout: "exclude_perf_counters": { 2024-09-12T03:20:14.113 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "exclude_perf_counters", 2024-09-12T03:20:14.113 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.113 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.113 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.113 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-12T03:20:14.113 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.113 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.113 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.113 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-12T03:20:14.114 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-12T03:20:14.114 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.114 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.114 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.114 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-12T03:20:14.114 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-12T03:20:14.114 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.114 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.114 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.115 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-12T03:20:14.115 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.115 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.115 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-12T03:20:14.115 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-12T03:20:14.115 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-12T03:20:14.115 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-12T03:20:14.115 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-12T03:20:14.115 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-12T03:20:14.115 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-12T03:20:14.115 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-12T03:20:14.115 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.116 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.116 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.116 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.116 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.116 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-12T03:20:14.116 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-12T03:20:14.116 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.116 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.116 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.117 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-12T03:20:14.117 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.117 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.117 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.117 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.117 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.117 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.117 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.118 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.118 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-12T03:20:14.118 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-12T03:20:14.118 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.118 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.118 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.118 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-12T03:20:14.118 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.118 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.118 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-12T03:20:14.119 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-12T03:20:14.119 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-12T03:20:14.119 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-12T03:20:14.119 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-12T03:20:14.119 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-12T03:20:14.119 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-12T03:20:14.119 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-12T03:20:14.119 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.119 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.119 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.119 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.119 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.120 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-12T03:20:14.120 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-12T03:20:14.120 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.120 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.120 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.120 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-12T03:20:14.120 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.120 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.120 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.120 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.120 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.121 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.121 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.121 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.121 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_stats_pools": { 2024-09-12T03:20:14.121 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rbd_stats_pools", 2024-09-12T03:20:14.121 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.121 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.121 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.121 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-12T03:20:14.121 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.121 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.121 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.122 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.122 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.122 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.122 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.122 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.122 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-12T03:20:14.122 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-12T03:20:14.123 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-12T03:20:14.123 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.123 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.123 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "300", 2024-09-12T03:20:14.123 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.123 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.123 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.123 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.123 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.123 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.124 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.124 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.124 INFO:teuthology.orchestra.run.smithi027.stdout: "scrape_interval": { 2024-09-12T03:20:14.124 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "scrape_interval", 2024-09-12T03:20:14.124 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "float", 2024-09-12T03:20:14.124 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.124 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.124 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "15.0", 2024-09-12T03:20:14.124 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.124 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.124 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.124 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.125 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.125 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.125 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.125 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.125 INFO:teuthology.orchestra.run.smithi027.stdout: "server_addr": { 2024-09-12T03:20:14.125 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "server_addr", 2024-09-12T03:20:14.125 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.125 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.125 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.125 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "::", 2024-09-12T03:20:14.126 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.126 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.126 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.126 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-12T03:20:14.126 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.126 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.126 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.126 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.126 INFO:teuthology.orchestra.run.smithi027.stdout: "server_port": { 2024-09-12T03:20:14.127 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "server_port", 2024-09-12T03:20:14.127 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-12T03:20:14.127 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.127 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.127 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "9283", 2024-09-12T03:20:14.127 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.127 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.127 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.127 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-12T03:20:14.127 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.127 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.127 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.128 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.128 INFO:teuthology.orchestra.run.smithi027.stdout: "stale_cache_strategy": { 2024-09-12T03:20:14.128 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "stale_cache_strategy", 2024-09-12T03:20:14.128 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.128 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.128 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.128 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "log", 2024-09-12T03:20:14.128 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.128 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.128 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.128 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.129 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.129 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.129 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.129 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.129 INFO:teuthology.orchestra.run.smithi027.stdout: "standby_behaviour": { 2024-09-12T03:20:14.129 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "standby_behaviour", 2024-09-12T03:20:14.129 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.129 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.129 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.129 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "default", 2024-09-12T03:20:14.129 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.130 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.130 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-12T03:20:14.130 INFO:teuthology.orchestra.run.smithi027.stdout: "default", 2024-09-12T03:20:14.130 INFO:teuthology.orchestra.run.smithi027.stdout: "error" 2024-09-12T03:20:14.130 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-12T03:20:14.130 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.130 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.131 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.131 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.131 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.131 INFO:teuthology.orchestra.run.smithi027.stdout: "standby_error_status_code": { 2024-09-12T03:20:14.131 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "standby_error_status_code", 2024-09-12T03:20:14.131 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-12T03:20:14.131 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.131 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.131 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "500", 2024-09-12T03:20:14.132 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "400", 2024-09-12T03:20:14.132 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "599", 2024-09-12T03:20:14.132 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.132 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.132 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.132 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.132 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.132 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:20:14.132 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:20:14.132 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.133 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:20:14.133 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rbd_support", 2024-09-12T03:20:14.133 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-12T03:20:14.133 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-12T03:20:14.133 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-12T03:20:14.133 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-12T03:20:14.133 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-12T03:20:14.133 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.134 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.134 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.134 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-12T03:20:14.134 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.134 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.134 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-12T03:20:14.134 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-12T03:20:14.134 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-12T03:20:14.134 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-12T03:20:14.134 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-12T03:20:14.134 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-12T03:20:14.134 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-12T03:20:14.135 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-12T03:20:14.135 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.135 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.135 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.135 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.135 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.135 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-12T03:20:14.135 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-12T03:20:14.136 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.136 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.136 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.136 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-12T03:20:14.136 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.136 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.136 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.136 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.136 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.136 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.136 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.137 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.137 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-12T03:20:14.137 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-12T03:20:14.137 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.137 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.137 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.137 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-12T03:20:14.137 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.137 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.137 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-12T03:20:14.137 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-12T03:20:14.137 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-12T03:20:14.138 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-12T03:20:14.138 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-12T03:20:14.138 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-12T03:20:14.138 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-12T03:20:14.138 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-12T03:20:14.138 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.138 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.138 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.138 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.138 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.138 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-12T03:20:14.138 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-12T03:20:14.139 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.139 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.139 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.139 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-12T03:20:14.139 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.139 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.139 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.139 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.140 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.140 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.140 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.140 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.140 INFO:teuthology.orchestra.run.smithi027.stdout: "max_concurrent_snap_create": { 2024-09-12T03:20:14.140 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "max_concurrent_snap_create", 2024-09-12T03:20:14.140 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-12T03:20:14.140 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.140 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.140 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10", 2024-09-12T03:20:14.141 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.141 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.141 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.141 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.141 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.141 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.141 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.141 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.141 INFO:teuthology.orchestra.run.smithi027.stdout: "mirror_snapshot_schedule": { 2024-09-12T03:20:14.141 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "mirror_snapshot_schedule", 2024-09-12T03:20:14.142 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.142 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.142 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.142 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-12T03:20:14.142 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.142 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.142 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.142 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.142 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.142 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.142 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.143 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.143 INFO:teuthology.orchestra.run.smithi027.stdout: "trash_purge_schedule": { 2024-09-12T03:20:14.143 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "trash_purge_schedule", 2024-09-12T03:20:14.143 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.143 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.143 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.143 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-12T03:20:14.143 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.143 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.143 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.143 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.143 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.144 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.144 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.144 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:20:14.144 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:20:14.144 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.144 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:20:14.144 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "restful", 2024-09-12T03:20:14.144 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-12T03:20:14.144 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-12T03:20:14.144 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-12T03:20:14.144 INFO:teuthology.orchestra.run.smithi027.stdout: "enable_auth": { 2024-09-12T03:20:14.144 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "enable_auth", 2024-09-12T03:20:14.144 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.145 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.145 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.145 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-12T03:20:14.145 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.145 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.145 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.145 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.145 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.145 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.145 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.145 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.145 INFO:teuthology.orchestra.run.smithi027.stdout: "key_file": { 2024-09-12T03:20:14.145 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "key_file", 2024-09-12T03:20:14.146 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.146 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.146 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.146 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-12T03:20:14.146 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.146 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.146 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.146 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.146 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.146 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.146 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.147 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.147 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-12T03:20:14.147 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-12T03:20:14.147 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.147 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.147 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.147 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-12T03:20:14.147 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.147 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.147 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-12T03:20:14.147 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-12T03:20:14.148 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-12T03:20:14.148 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-12T03:20:14.148 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-12T03:20:14.148 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-12T03:20:14.148 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-12T03:20:14.148 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-12T03:20:14.148 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.148 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.148 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.148 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.148 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.148 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-12T03:20:14.148 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-12T03:20:14.149 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.149 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.149 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.149 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-12T03:20:14.149 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.149 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.149 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.149 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.149 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.149 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.149 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.149 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.150 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-12T03:20:14.150 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-12T03:20:14.150 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.150 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.150 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.150 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-12T03:20:14.150 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.150 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.150 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-12T03:20:14.150 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-12T03:20:14.150 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-12T03:20:14.150 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-12T03:20:14.151 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-12T03:20:14.151 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-12T03:20:14.151 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-12T03:20:14.151 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-12T03:20:14.151 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.151 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.151 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.151 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.151 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.151 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-12T03:20:14.151 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-12T03:20:14.151 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.152 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.152 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.152 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-12T03:20:14.152 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.152 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.152 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.152 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.152 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.152 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.152 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.152 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.153 INFO:teuthology.orchestra.run.smithi027.stdout: "server_addr": { 2024-09-12T03:20:14.153 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "server_addr", 2024-09-12T03:20:14.153 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.153 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.153 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.153 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-12T03:20:14.153 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.153 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.153 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.153 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.153 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.153 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.154 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.154 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.154 INFO:teuthology.orchestra.run.smithi027.stdout: "server_port": { 2024-09-12T03:20:14.154 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "server_port", 2024-09-12T03:20:14.154 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.154 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.154 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.154 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-12T03:20:14.154 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.154 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.154 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.154 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.155 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.155 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.155 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.155 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:20:14.155 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:20:14.155 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.155 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:20:14.155 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rgw", 2024-09-12T03:20:14.155 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-12T03:20:14.155 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-12T03:20:14.155 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-12T03:20:14.155 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-12T03:20:14.156 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-12T03:20:14.156 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.156 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.156 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.156 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-12T03:20:14.156 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.156 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.156 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-12T03:20:14.156 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-12T03:20:14.156 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-12T03:20:14.156 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-12T03:20:14.156 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-12T03:20:14.157 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-12T03:20:14.157 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-12T03:20:14.157 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-12T03:20:14.157 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.157 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.157 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.157 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.157 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.157 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-12T03:20:14.157 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-12T03:20:14.157 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.157 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.157 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.158 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-12T03:20:14.158 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.158 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.158 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.158 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.158 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.158 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.158 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.158 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.158 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-12T03:20:14.158 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-12T03:20:14.158 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.159 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.159 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.159 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-12T03:20:14.159 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.159 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.159 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-12T03:20:14.159 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-12T03:20:14.159 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-12T03:20:14.159 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-12T03:20:14.159 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-12T03:20:14.159 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-12T03:20:14.159 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-12T03:20:14.159 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-12T03:20:14.160 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.160 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.160 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.160 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.160 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.160 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-12T03:20:14.160 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-12T03:20:14.160 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.160 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.160 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.160 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-12T03:20:14.160 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.160 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.161 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.161 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.161 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.161 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.161 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.161 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:20:14.161 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:20:14.161 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.161 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:20:14.161 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rook", 2024-09-12T03:20:14.161 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-12T03:20:14.161 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-12T03:20:14.162 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-12T03:20:14.162 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-12T03:20:14.162 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-12T03:20:14.162 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.162 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.162 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.162 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-12T03:20:14.162 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.162 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.162 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-12T03:20:14.162 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-12T03:20:14.162 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-12T03:20:14.163 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-12T03:20:14.163 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-12T03:20:14.163 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-12T03:20:14.163 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-12T03:20:14.163 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-12T03:20:14.163 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.163 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.163 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.163 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.163 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.163 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-12T03:20:14.163 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-12T03:20:14.164 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.164 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.164 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.164 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-12T03:20:14.164 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.164 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.164 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.164 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.164 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.164 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.164 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.164 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.165 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-12T03:20:14.165 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-12T03:20:14.165 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.165 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.165 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.165 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-12T03:20:14.165 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.165 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.165 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-12T03:20:14.165 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-12T03:20:14.165 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-12T03:20:14.165 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-12T03:20:14.166 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-12T03:20:14.166 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-12T03:20:14.166 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-12T03:20:14.166 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-12T03:20:14.166 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.166 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.166 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.166 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.166 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.166 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-12T03:20:14.166 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-12T03:20:14.166 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.166 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.167 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.167 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-12T03:20:14.167 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.167 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.167 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.167 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.167 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.167 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.167 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.167 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.167 INFO:teuthology.orchestra.run.smithi027.stdout: "storage_class": { 2024-09-12T03:20:14.167 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "storage_class", 2024-09-12T03:20:14.167 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.168 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.168 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.168 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "local", 2024-09-12T03:20:14.168 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.168 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.168 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.168 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-12T03:20:14.168 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.168 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.168 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.168 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:20:14.169 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:20:14.169 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.169 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:20:14.169 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "selftest", 2024-09-12T03:20:14.169 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-12T03:20:14.169 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-12T03:20:14.169 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-12T03:20:14.169 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-12T03:20:14.169 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-12T03:20:14.169 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.169 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.169 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.169 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-12T03:20:14.170 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.170 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.170 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-12T03:20:14.170 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-12T03:20:14.170 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-12T03:20:14.170 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-12T03:20:14.170 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-12T03:20:14.170 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-12T03:20:14.170 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-12T03:20:14.170 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-12T03:20:14.170 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.170 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.170 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.170 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.171 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.171 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-12T03:20:14.171 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-12T03:20:14.171 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.171 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.171 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.171 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-12T03:20:14.171 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.171 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.171 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.171 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.172 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.172 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.172 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.172 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.172 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-12T03:20:14.172 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-12T03:20:14.172 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.172 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.172 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.172 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-12T03:20:14.172 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.172 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.173 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-12T03:20:14.173 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-12T03:20:14.173 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-12T03:20:14.173 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-12T03:20:14.173 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-12T03:20:14.173 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-12T03:20:14.173 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-12T03:20:14.173 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-12T03:20:14.173 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.173 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.173 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.173 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.174 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.174 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-12T03:20:14.174 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-12T03:20:14.174 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.174 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.174 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.174 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-12T03:20:14.174 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.174 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.174 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.174 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.174 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.174 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.174 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.175 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.175 INFO:teuthology.orchestra.run.smithi027.stdout: "roption1": { 2024-09-12T03:20:14.175 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "roption1", 2024-09-12T03:20:14.175 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.175 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.175 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.175 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-12T03:20:14.175 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.175 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.175 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.175 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.175 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.176 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.176 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.176 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.176 INFO:teuthology.orchestra.run.smithi027.stdout: "roption2": { 2024-09-12T03:20:14.176 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "roption2", 2024-09-12T03:20:14.176 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.176 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.176 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.176 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "xyz", 2024-09-12T03:20:14.176 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.176 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.176 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.176 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.177 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.177 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.177 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.177 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.177 INFO:teuthology.orchestra.run.smithi027.stdout: "rwoption1": { 2024-09-12T03:20:14.177 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rwoption1", 2024-09-12T03:20:14.177 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.177 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.177 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.177 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-12T03:20:14.177 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.177 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.177 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.177 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.178 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.178 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.178 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.178 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.178 INFO:teuthology.orchestra.run.smithi027.stdout: "rwoption2": { 2024-09-12T03:20:14.178 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rwoption2", 2024-09-12T03:20:14.178 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-12T03:20:14.178 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.178 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.178 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-12T03:20:14.178 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.178 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.179 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.179 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.179 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.179 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.179 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.179 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.179 INFO:teuthology.orchestra.run.smithi027.stdout: "rwoption3": { 2024-09-12T03:20:14.179 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rwoption3", 2024-09-12T03:20:14.179 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "float", 2024-09-12T03:20:14.179 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.179 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.179 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-12T03:20:14.180 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.180 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.180 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.180 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.180 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.180 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.180 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.180 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.180 INFO:teuthology.orchestra.run.smithi027.stdout: "rwoption4": { 2024-09-12T03:20:14.180 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rwoption4", 2024-09-12T03:20:14.180 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.180 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.181 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.181 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-12T03:20:14.181 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.181 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.181 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.181 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.181 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.181 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.181 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.181 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.181 INFO:teuthology.orchestra.run.smithi027.stdout: "rwoption5": { 2024-09-12T03:20:14.181 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rwoption5", 2024-09-12T03:20:14.182 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.182 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.182 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.182 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-12T03:20:14.182 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.182 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.182 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.182 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.182 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.182 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.182 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.182 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.183 INFO:teuthology.orchestra.run.smithi027.stdout: "rwoption6": { 2024-09-12T03:20:14.183 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rwoption6", 2024-09-12T03:20:14.183 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.183 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.183 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.183 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-12T03:20:14.183 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.183 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.183 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.183 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.183 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.183 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.183 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.184 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.184 INFO:teuthology.orchestra.run.smithi027.stdout: "rwoption7": { 2024-09-12T03:20:14.184 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rwoption7", 2024-09-12T03:20:14.184 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-12T03:20:14.184 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.184 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.184 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-12T03:20:14.184 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "1", 2024-09-12T03:20:14.184 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "42", 2024-09-12T03:20:14.184 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.184 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.184 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.185 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.185 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.185 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.185 INFO:teuthology.orchestra.run.smithi027.stdout: "testkey": { 2024-09-12T03:20:14.185 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "testkey", 2024-09-12T03:20:14.185 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.185 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.185 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.185 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-12T03:20:14.185 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.185 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.185 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.186 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.186 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.186 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.186 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.186 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.186 INFO:teuthology.orchestra.run.smithi027.stdout: "testlkey": { 2024-09-12T03:20:14.186 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "testlkey", 2024-09-12T03:20:14.186 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.186 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.186 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.186 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-12T03:20:14.186 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.186 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.187 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.187 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.187 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.187 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.187 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.187 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.187 INFO:teuthology.orchestra.run.smithi027.stdout: "testnewline": { 2024-09-12T03:20:14.187 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "testnewline", 2024-09-12T03:20:14.187 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.187 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.187 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.187 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-12T03:20:14.187 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.188 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.188 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.188 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.188 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.188 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.188 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.188 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:20:14.188 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:20:14.188 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.188 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:20:14.188 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "snap_schedule", 2024-09-12T03:20:14.188 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-12T03:20:14.189 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-12T03:20:14.189 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-12T03:20:14.189 INFO:teuthology.orchestra.run.smithi027.stdout: "allow_m_granularity": { 2024-09-12T03:20:14.189 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "allow_m_granularity", 2024-09-12T03:20:14.189 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.189 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.189 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.189 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-12T03:20:14.189 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.189 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.189 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.189 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "allow minute scheduled snapshots", 2024-09-12T03:20:14.189 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.190 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.190 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.190 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.190 INFO:teuthology.orchestra.run.smithi027.stdout: "dump_on_update": { 2024-09-12T03:20:14.190 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "dump_on_update", 2024-09-12T03:20:14.190 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.190 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.190 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.190 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-12T03:20:14.190 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.190 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.191 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.191 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "dump database to debug log on update", 2024-09-12T03:20:14.191 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.191 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.191 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.191 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.191 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-12T03:20:14.191 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-12T03:20:14.191 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.191 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.191 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.191 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-12T03:20:14.191 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.191 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.192 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-12T03:20:14.192 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-12T03:20:14.192 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-12T03:20:14.192 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-12T03:20:14.192 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-12T03:20:14.192 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-12T03:20:14.192 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-12T03:20:14.192 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-12T03:20:14.192 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.192 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.192 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.192 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.192 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.192 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-12T03:20:14.193 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-12T03:20:14.193 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.193 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.193 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.193 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-12T03:20:14.193 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.193 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.193 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.193 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.193 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.193 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.193 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.194 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.194 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-12T03:20:14.194 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-12T03:20:14.194 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.194 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.194 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.194 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-12T03:20:14.194 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.194 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.194 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-12T03:20:14.194 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-12T03:20:14.194 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-12T03:20:14.194 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-12T03:20:14.195 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-12T03:20:14.195 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-12T03:20:14.195 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-12T03:20:14.195 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-12T03:20:14.195 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.195 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.195 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.195 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.195 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.195 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-12T03:20:14.195 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-12T03:20:14.195 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.195 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.196 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.196 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-12T03:20:14.196 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.196 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.196 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.196 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.196 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.196 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.196 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.196 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:20:14.196 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:20:14.196 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.197 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:20:14.197 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "stats", 2024-09-12T03:20:14.197 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-12T03:20:14.197 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-12T03:20:14.197 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-12T03:20:14.197 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-12T03:20:14.197 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-12T03:20:14.197 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.197 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.197 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.197 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-12T03:20:14.197 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.197 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.198 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-12T03:20:14.198 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-12T03:20:14.198 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-12T03:20:14.198 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-12T03:20:14.198 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-12T03:20:14.198 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-12T03:20:14.198 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-12T03:20:14.198 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-12T03:20:14.198 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.198 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.198 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.198 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.198 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.199 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-12T03:20:14.199 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-12T03:20:14.199 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.199 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.199 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.199 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-12T03:20:14.199 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.199 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.199 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.199 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.199 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.199 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.200 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.200 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.200 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-12T03:20:14.200 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-12T03:20:14.200 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.200 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.200 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.200 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-12T03:20:14.200 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.200 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.200 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-12T03:20:14.200 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-12T03:20:14.201 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-12T03:20:14.201 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-12T03:20:14.201 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-12T03:20:14.201 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-12T03:20:14.201 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-12T03:20:14.201 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-12T03:20:14.201 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.201 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.201 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.201 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.201 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.201 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-12T03:20:14.201 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-12T03:20:14.202 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.202 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.202 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.202 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-12T03:20:14.202 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.202 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.202 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.202 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.202 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.202 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.202 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.202 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:20:14.203 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:20:14.203 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.203 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:20:14.203 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "status", 2024-09-12T03:20:14.203 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-12T03:20:14.203 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-12T03:20:14.203 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-12T03:20:14.203 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-12T03:20:14.203 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-12T03:20:14.203 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.203 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.203 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.204 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-12T03:20:14.204 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.204 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.204 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-12T03:20:14.204 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-12T03:20:14.204 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-12T03:20:14.204 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-12T03:20:14.204 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-12T03:20:14.204 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-12T03:20:14.204 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-12T03:20:14.204 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-12T03:20:14.204 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.205 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.205 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.205 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.205 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.205 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-12T03:20:14.205 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-12T03:20:14.205 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.205 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.205 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.205 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-12T03:20:14.205 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.205 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.206 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.206 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.206 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.206 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.206 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.206 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.206 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-12T03:20:14.206 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-12T03:20:14.206 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.206 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.206 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.206 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-12T03:20:14.207 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.207 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.207 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-12T03:20:14.207 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-12T03:20:14.207 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-12T03:20:14.207 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-12T03:20:14.207 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-12T03:20:14.207 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-12T03:20:14.207 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-12T03:20:14.207 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-12T03:20:14.207 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.207 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.207 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.207 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.208 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.208 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-12T03:20:14.208 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-12T03:20:14.208 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.208 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.208 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.208 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-12T03:20:14.208 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.208 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.208 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.208 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.208 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.209 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.209 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.209 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:20:14.209 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:20:14.209 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.209 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:20:14.209 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "telegraf", 2024-09-12T03:20:14.209 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-12T03:20:14.209 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-12T03:20:14.209 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-12T03:20:14.209 INFO:teuthology.orchestra.run.smithi027.stdout: "address": { 2024-09-12T03:20:14.209 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "address", 2024-09-12T03:20:14.209 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.210 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.210 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.210 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-12T03:20:14.210 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.210 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.210 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.210 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.210 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.210 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.210 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.210 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.210 INFO:teuthology.orchestra.run.smithi027.stdout: "interval": { 2024-09-12T03:20:14.211 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "interval", 2024-09-12T03:20:14.211 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-12T03:20:14.211 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.211 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.211 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "15", 2024-09-12T03:20:14.211 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.211 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.211 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.211 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.211 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.211 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.211 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.211 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.212 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-12T03:20:14.212 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-12T03:20:14.212 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.212 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.212 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.212 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-12T03:20:14.212 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.212 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.212 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-12T03:20:14.212 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-12T03:20:14.212 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-12T03:20:14.212 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-12T03:20:14.213 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-12T03:20:14.213 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-12T03:20:14.213 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-12T03:20:14.213 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-12T03:20:14.213 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.213 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.213 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.213 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.213 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.213 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-12T03:20:14.213 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-12T03:20:14.213 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.214 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.214 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.214 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-12T03:20:14.214 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.214 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.214 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.214 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.214 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.214 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.214 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.214 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.214 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-12T03:20:14.214 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-12T03:20:14.215 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.215 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.215 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.215 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-12T03:20:14.215 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.215 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.215 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-12T03:20:14.215 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-12T03:20:14.215 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-12T03:20:14.215 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-12T03:20:14.215 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-12T03:20:14.215 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-12T03:20:14.216 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-12T03:20:14.216 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-12T03:20:14.216 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.216 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.216 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.216 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.216 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.216 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-12T03:20:14.216 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-12T03:20:14.216 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.216 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.216 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.217 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-12T03:20:14.217 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.217 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.217 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.217 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.217 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.217 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.217 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.217 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:20:14.217 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:20:14.217 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.217 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:20:14.217 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "telemetry", 2024-09-12T03:20:14.217 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-12T03:20:14.218 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-12T03:20:14.218 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-12T03:20:14.218 INFO:teuthology.orchestra.run.smithi027.stdout: "channel_basic": { 2024-09-12T03:20:14.218 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "channel_basic", 2024-09-12T03:20:14.218 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.218 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.218 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.218 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-12T03:20:14.218 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.218 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.218 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.218 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-12T03:20:14.219 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.219 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.219 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.219 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.219 INFO:teuthology.orchestra.run.smithi027.stdout: "channel_crash": { 2024-09-12T03:20:14.219 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "channel_crash", 2024-09-12T03:20:14.219 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.219 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.219 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.219 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-12T03:20:14.219 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.219 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.219 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.220 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-12T03:20:14.220 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.220 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.220 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.220 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.220 INFO:teuthology.orchestra.run.smithi027.stdout: "channel_device": { 2024-09-12T03:20:14.220 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "channel_device", 2024-09-12T03:20:14.220 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.220 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.220 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.220 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-12T03:20:14.220 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.221 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.221 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.221 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-12T03:20:14.221 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.221 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.221 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.221 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.221 INFO:teuthology.orchestra.run.smithi027.stdout: "channel_ident": { 2024-09-12T03:20:14.221 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "channel_ident", 2024-09-12T03:20:14.221 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.221 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.221 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.221 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-12T03:20:14.222 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.222 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.222 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.222 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-12T03:20:14.222 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.222 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.222 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.222 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.222 INFO:teuthology.orchestra.run.smithi027.stdout: "channel_perf": { 2024-09-12T03:20:14.222 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "channel_perf", 2024-09-12T03:20:14.222 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.222 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.222 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.222 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-12T03:20:14.223 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.223 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.223 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.223 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-12T03:20:14.223 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.223 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.223 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.223 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.223 INFO:teuthology.orchestra.run.smithi027.stdout: "contact": { 2024-09-12T03:20:14.223 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "contact", 2024-09-12T03:20:14.223 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.223 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.224 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.224 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-12T03:20:14.224 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.224 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.224 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.224 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.224 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.224 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.224 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.224 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.224 INFO:teuthology.orchestra.run.smithi027.stdout: "description": { 2024-09-12T03:20:14.224 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "description", 2024-09-12T03:20:14.225 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.225 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.225 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.225 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-12T03:20:14.225 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.225 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.225 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.225 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.225 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.225 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.225 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.225 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.225 INFO:teuthology.orchestra.run.smithi027.stdout: "device_url": { 2024-09-12T03:20:14.225 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "device_url", 2024-09-12T03:20:14.226 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.226 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.226 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.226 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-12T03:20:14.226 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.226 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.226 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.226 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.226 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.226 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.226 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.226 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.226 INFO:teuthology.orchestra.run.smithi027.stdout: "enabled": { 2024-09-12T03:20:14.227 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "enabled", 2024-09-12T03:20:14.227 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.227 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.227 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.227 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-12T03:20:14.227 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.227 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.227 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.227 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.227 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.227 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.227 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.227 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.227 INFO:teuthology.orchestra.run.smithi027.stdout: "interval": { 2024-09-12T03:20:14.228 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "interval", 2024-09-12T03:20:14.228 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-12T03:20:14.228 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.228 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.228 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "24", 2024-09-12T03:20:14.228 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "8", 2024-09-12T03:20:14.228 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.228 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.228 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.228 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.228 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.228 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.228 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.229 INFO:teuthology.orchestra.run.smithi027.stdout: "last_opt_revision": { 2024-09-12T03:20:14.229 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "last_opt_revision", 2024-09-12T03:20:14.229 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-12T03:20:14.229 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.229 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.229 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "1", 2024-09-12T03:20:14.229 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.229 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.229 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.229 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.229 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.229 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.230 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.230 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.230 INFO:teuthology.orchestra.run.smithi027.stdout: "leaderboard": { 2024-09-12T03:20:14.230 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "leaderboard", 2024-09-12T03:20:14.230 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.230 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.230 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.230 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-12T03:20:14.230 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.230 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.230 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.230 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.231 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.231 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.231 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.231 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.231 INFO:teuthology.orchestra.run.smithi027.stdout: "leaderboard_description": { 2024-09-12T03:20:14.231 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "leaderboard_description", 2024-09-12T03:20:14.231 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.231 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.231 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.231 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-12T03:20:14.231 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.231 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.232 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.232 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.232 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.232 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.232 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.232 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.232 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-12T03:20:14.232 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-12T03:20:14.232 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.232 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.232 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.232 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-12T03:20:14.232 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.232 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.233 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-12T03:20:14.233 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-12T03:20:14.233 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-12T03:20:14.233 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-12T03:20:14.233 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-12T03:20:14.233 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-12T03:20:14.233 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-12T03:20:14.233 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-12T03:20:14.233 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.233 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.233 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.233 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.233 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.234 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-12T03:20:14.234 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-12T03:20:14.234 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.234 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.234 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.234 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-12T03:20:14.234 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.234 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.234 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.234 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.234 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.234 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.234 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.234 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.235 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-12T03:20:14.235 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-12T03:20:14.235 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.235 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.235 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.235 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-12T03:20:14.235 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.235 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.235 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-12T03:20:14.235 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-12T03:20:14.235 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-12T03:20:14.235 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-12T03:20:14.235 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-12T03:20:14.236 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-12T03:20:14.236 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-12T03:20:14.236 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-12T03:20:14.236 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.236 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.236 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.236 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.236 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.236 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-12T03:20:14.236 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-12T03:20:14.236 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.236 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.236 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.236 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-12T03:20:14.237 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.237 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.237 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.237 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.237 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.237 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.237 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.237 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.237 INFO:teuthology.orchestra.run.smithi027.stdout: "organization": { 2024-09-12T03:20:14.237 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "organization", 2024-09-12T03:20:14.237 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.237 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.237 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.237 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-12T03:20:14.238 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.238 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.238 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.238 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.238 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.238 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.238 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.238 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.238 INFO:teuthology.orchestra.run.smithi027.stdout: "proxy": { 2024-09-12T03:20:14.238 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "proxy", 2024-09-12T03:20:14.238 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.240 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.240 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.241 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-12T03:20:14.241 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.241 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.241 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.241 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.241 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.241 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.241 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.241 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.241 INFO:teuthology.orchestra.run.smithi027.stdout: "url": { 2024-09-12T03:20:14.241 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "url", 2024-09-12T03:20:14.241 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.241 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.241 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.242 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-12T03:20:14.242 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.242 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.242 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.242 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.242 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.242 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.242 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.242 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:20:14.242 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:20:14.242 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.242 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:20:14.242 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "test_orchestrator", 2024-09-12T03:20:14.242 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-12T03:20:14.243 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-12T03:20:14.243 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-12T03:20:14.243 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-12T03:20:14.243 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-12T03:20:14.243 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.243 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.243 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.243 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-12T03:20:14.243 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.243 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.243 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-12T03:20:14.243 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-12T03:20:14.243 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-12T03:20:14.244 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-12T03:20:14.244 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-12T03:20:14.244 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-12T03:20:14.244 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-12T03:20:14.244 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-12T03:20:14.244 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.244 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.244 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.244 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.244 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.244 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-12T03:20:14.244 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-12T03:20:14.244 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.244 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.245 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.245 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-12T03:20:14.245 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.245 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.245 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.245 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.245 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.245 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.245 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.245 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.245 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-12T03:20:14.245 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-12T03:20:14.245 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.245 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.246 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.246 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-12T03:20:14.246 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.246 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.246 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-12T03:20:14.246 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-12T03:20:14.246 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-12T03:20:14.246 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-12T03:20:14.246 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-12T03:20:14.246 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-12T03:20:14.246 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-12T03:20:14.246 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-12T03:20:14.246 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.246 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.247 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.247 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.247 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.247 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-12T03:20:14.247 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-12T03:20:14.247 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.247 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.247 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.247 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-12T03:20:14.247 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.247 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.247 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.247 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.247 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.248 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.248 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.248 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:20:14.248 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:20:14.248 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.248 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:20:14.248 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "volumes", 2024-09-12T03:20:14.248 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-12T03:20:14.248 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-12T03:20:14.248 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-12T03:20:14.248 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-12T03:20:14.248 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-12T03:20:14.248 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.249 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.249 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.249 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-12T03:20:14.249 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.249 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.249 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-12T03:20:14.249 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-12T03:20:14.249 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-12T03:20:14.249 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-12T03:20:14.249 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-12T03:20:14.249 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-12T03:20:14.249 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-12T03:20:14.249 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-12T03:20:14.250 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.250 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.250 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.250 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.250 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.250 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-12T03:20:14.250 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-12T03:20:14.250 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.250 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.250 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.250 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-12T03:20:14.250 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.250 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.250 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.251 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.251 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.251 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.251 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.251 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.251 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-12T03:20:14.251 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-12T03:20:14.251 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.251 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.251 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.251 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-12T03:20:14.251 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.251 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.251 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-12T03:20:14.252 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-12T03:20:14.252 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-12T03:20:14.252 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-12T03:20:14.252 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-12T03:20:14.252 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-12T03:20:14.252 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-12T03:20:14.252 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-12T03:20:14.252 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.252 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.252 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.252 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.252 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.252 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-12T03:20:14.253 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-12T03:20:14.253 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.253 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.253 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.253 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-12T03:20:14.253 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.253 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.253 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.253 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.253 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.253 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.253 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.253 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.253 INFO:teuthology.orchestra.run.smithi027.stdout: "max_concurrent_clones": { 2024-09-12T03:20:14.253 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "max_concurrent_clones", 2024-09-12T03:20:14.254 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-12T03:20:14.254 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.254 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.254 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "4", 2024-09-12T03:20:14.254 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.254 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.254 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.254 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-12T03:20:14.254 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.254 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.254 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.254 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.254 INFO:teuthology.orchestra.run.smithi027.stdout: "periodic_async_work": { 2024-09-12T03:20:14.254 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "periodic_async_work", 2024-09-12T03:20:14.255 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.255 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.255 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.255 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-12T03:20:14.255 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.255 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.255 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.255 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Periodically check for async work", 2024-09-12T03:20:14.255 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.255 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.255 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.255 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.255 INFO:teuthology.orchestra.run.smithi027.stdout: "snapshot_clone_delay": { 2024-09-12T03:20:14.255 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "snapshot_clone_delay", 2024-09-12T03:20:14.256 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-12T03:20:14.256 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.256 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.256 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-12T03:20:14.256 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.256 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.256 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.256 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-12T03:20:14.256 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.256 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.256 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.256 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.256 INFO:teuthology.orchestra.run.smithi027.stdout: "snapshot_clone_no_wait": { 2024-09-12T03:20:14.256 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "snapshot_clone_no_wait", 2024-09-12T03:20:14.257 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.257 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.257 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.257 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-12T03:20:14.257 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.257 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.257 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.257 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-12T03:20:14.257 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.257 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.257 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.257 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:20:14.257 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:20:14.257 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.258 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:20:14.258 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "zabbix", 2024-09-12T03:20:14.258 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-12T03:20:14.258 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-12T03:20:14.258 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-12T03:20:14.258 INFO:teuthology.orchestra.run.smithi027.stdout: "discovery_interval": { 2024-09-12T03:20:14.258 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "discovery_interval", 2024-09-12T03:20:14.258 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "uint", 2024-09-12T03:20:14.258 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.258 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.258 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "100", 2024-09-12T03:20:14.258 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.258 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.258 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.259 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.259 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.259 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.259 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.259 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.259 INFO:teuthology.orchestra.run.smithi027.stdout: "identifier": { 2024-09-12T03:20:14.259 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "identifier", 2024-09-12T03:20:14.259 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.259 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.259 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.259 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-12T03:20:14.259 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.259 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.259 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.260 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.260 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.260 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.260 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.260 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.260 INFO:teuthology.orchestra.run.smithi027.stdout: "interval": { 2024-09-12T03:20:14.260 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "interval", 2024-09-12T03:20:14.260 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-12T03:20:14.260 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.260 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.260 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "60", 2024-09-12T03:20:14.260 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.260 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.260 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.261 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.261 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.261 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.261 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.261 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.261 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-12T03:20:14.261 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-12T03:20:14.261 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.261 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.261 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.261 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-12T03:20:14.261 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.261 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.261 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-12T03:20:14.262 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-12T03:20:14.262 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-12T03:20:14.262 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-12T03:20:14.262 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-12T03:20:14.262 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-12T03:20:14.262 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-12T03:20:14.262 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-12T03:20:14.262 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.262 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.262 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.262 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.262 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.262 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-12T03:20:14.262 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-12T03:20:14.263 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.263 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.263 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.263 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-12T03:20:14.263 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.263 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.263 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.263 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.263 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.263 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.263 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.263 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.263 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-12T03:20:14.264 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-12T03:20:14.264 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.264 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.264 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.264 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-12T03:20:14.264 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.264 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.264 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-12T03:20:14.264 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-12T03:20:14.264 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-12T03:20:14.264 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-12T03:20:14.264 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-12T03:20:14.264 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-12T03:20:14.265 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-12T03:20:14.265 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-12T03:20:14.265 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.265 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.265 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.265 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.265 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.265 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-12T03:20:14.265 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-12T03:20:14.265 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.265 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.265 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.265 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-12T03:20:14.265 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.266 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.266 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.266 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.266 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.266 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.266 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.266 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.266 INFO:teuthology.orchestra.run.smithi027.stdout: "zabbix_host": { 2024-09-12T03:20:14.266 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "zabbix_host", 2024-09-12T03:20:14.266 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.266 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.266 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.266 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-12T03:20:14.266 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.267 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.267 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.267 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.267 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.267 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.267 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.267 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.267 INFO:teuthology.orchestra.run.smithi027.stdout: "zabbix_port": { 2024-09-12T03:20:14.267 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "zabbix_port", 2024-09-12T03:20:14.267 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-12T03:20:14.267 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.267 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.267 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10051", 2024-09-12T03:20:14.267 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.268 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.268 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.268 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.268 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.268 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.268 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.268 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.268 INFO:teuthology.orchestra.run.smithi027.stdout: "zabbix_sender": { 2024-09-12T03:20:14.268 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "zabbix_sender", 2024-09-12T03:20:14.268 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.268 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.268 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.268 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-12T03:20:14.268 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.269 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.269 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.269 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.269 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.269 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.269 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.269 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:20:14.269 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:20:14.269 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:20:14.269 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-12T03:20:14.269 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:20:14.269 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-12T03:20:14.269 INFO:teuthology.orchestra.run.smithi027.stdout: "modules": [ 2024-09-12T03:20:14.269 INFO:teuthology.orchestra.run.smithi027.stdout: "iostat", 2024-09-12T03:20:14.270 INFO:teuthology.orchestra.run.smithi027.stdout: "nfs", 2024-09-12T03:20:14.270 INFO:teuthology.orchestra.run.smithi027.stdout: "restful" 2024-09-12T03:20:14.270 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-12T03:20:14.270 INFO:teuthology.orchestra.run.smithi027.stdout: "available_modules": [ 2024-09-12T03:20:14.270 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:20:14.270 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "alerts", 2024-09-12T03:20:14.270 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-12T03:20:14.270 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-12T03:20:14.270 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-12T03:20:14.270 INFO:teuthology.orchestra.run.smithi027.stdout: "interval": { 2024-09-12T03:20:14.270 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "interval", 2024-09-12T03:20:14.270 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-12T03:20:14.270 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.270 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.271 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "60", 2024-09-12T03:20:14.271 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.271 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.271 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.271 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "How frequently to reexamine health status", 2024-09-12T03:20:14.271 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.271 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.271 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.271 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.271 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-12T03:20:14.271 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-12T03:20:14.271 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.271 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.271 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.272 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-12T03:20:14.272 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.272 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.272 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-12T03:20:14.272 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-12T03:20:14.272 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-12T03:20:14.272 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-12T03:20:14.272 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-12T03:20:14.272 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-12T03:20:14.272 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-12T03:20:14.272 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-12T03:20:14.272 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.272 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.272 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.273 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.273 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.273 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-12T03:20:14.273 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-12T03:20:14.273 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.273 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.273 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.273 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-12T03:20:14.273 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.273 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.273 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.273 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.273 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.273 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.274 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.274 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.274 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-12T03:20:14.274 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-12T03:20:14.274 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.274 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.274 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.274 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-12T03:20:14.274 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.274 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.274 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-12T03:20:14.274 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-12T03:20:14.274 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-12T03:20:14.274 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-12T03:20:14.275 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-12T03:20:14.275 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-12T03:20:14.275 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-12T03:20:14.275 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-12T03:20:14.275 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.275 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.275 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.275 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.275 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.275 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-12T03:20:14.275 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-12T03:20:14.275 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.275 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.275 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.276 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-12T03:20:14.276 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.276 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.276 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.276 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.276 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.276 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.276 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.276 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.276 INFO:teuthology.orchestra.run.smithi027.stdout: "smtp_destination": { 2024-09-12T03:20:14.276 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "smtp_destination", 2024-09-12T03:20:14.276 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.276 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.277 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.277 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-12T03:20:14.277 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.277 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.277 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.277 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Email address to send alerts to", 2024-09-12T03:20:14.277 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.277 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.277 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.277 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.277 INFO:teuthology.orchestra.run.smithi027.stdout: "smtp_from_name": { 2024-09-12T03:20:14.277 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "smtp_from_name", 2024-09-12T03:20:14.277 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.277 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.278 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.278 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "Ceph", 2024-09-12T03:20:14.278 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.278 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.278 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.278 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Email From: name", 2024-09-12T03:20:14.278 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.278 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.278 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.278 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.278 INFO:teuthology.orchestra.run.smithi027.stdout: "smtp_host": { 2024-09-12T03:20:14.278 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "smtp_host", 2024-09-12T03:20:14.278 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.278 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.279 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.279 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-12T03:20:14.279 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.279 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.279 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.279 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "SMTP server", 2024-09-12T03:20:14.279 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.279 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.279 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.279 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.279 INFO:teuthology.orchestra.run.smithi027.stdout: "smtp_password": { 2024-09-12T03:20:14.279 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "smtp_password", 2024-09-12T03:20:14.279 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.280 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.280 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.280 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-12T03:20:14.280 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.280 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.280 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.280 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Password to authenticate with", 2024-09-12T03:20:14.280 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.280 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.280 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.280 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.280 INFO:teuthology.orchestra.run.smithi027.stdout: "smtp_port": { 2024-09-12T03:20:14.280 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "smtp_port", 2024-09-12T03:20:14.280 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-12T03:20:14.281 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.281 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.281 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "465", 2024-09-12T03:20:14.281 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.281 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.281 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.281 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "SMTP port", 2024-09-12T03:20:14.281 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.281 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.281 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.281 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.281 INFO:teuthology.orchestra.run.smithi027.stdout: "smtp_sender": { 2024-09-12T03:20:14.281 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "smtp_sender", 2024-09-12T03:20:14.281 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.282 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.282 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.282 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-12T03:20:14.282 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.282 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.282 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.282 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "SMTP envelope sender", 2024-09-12T03:20:14.282 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.282 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.282 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.282 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.282 INFO:teuthology.orchestra.run.smithi027.stdout: "smtp_ssl": { 2024-09-12T03:20:14.282 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "smtp_ssl", 2024-09-12T03:20:14.282 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.283 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.283 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.283 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-12T03:20:14.283 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.283 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.283 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.283 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-12T03:20:14.283 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.283 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.283 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.283 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.283 INFO:teuthology.orchestra.run.smithi027.stdout: "smtp_user": { 2024-09-12T03:20:14.283 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "smtp_user", 2024-09-12T03:20:14.283 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.284 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.284 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.284 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-12T03:20:14.284 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.284 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.284 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.284 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "User to authenticate as", 2024-09-12T03:20:14.284 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.284 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.284 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.284 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:20:14.284 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:20:14.284 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.285 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:20:14.285 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "balancer", 2024-09-12T03:20:14.285 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-12T03:20:14.285 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-12T03:20:14.285 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-12T03:20:14.285 INFO:teuthology.orchestra.run.smithi027.stdout: "active": { 2024-09-12T03:20:14.285 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "active", 2024-09-12T03:20:14.285 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.285 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.285 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.285 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-12T03:20:14.285 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.285 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.285 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.286 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "automatically balance PGs across cluster", 2024-09-12T03:20:14.286 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.286 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.286 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.286 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.286 INFO:teuthology.orchestra.run.smithi027.stdout: "begin_time": { 2024-09-12T03:20:14.286 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "begin_time", 2024-09-12T03:20:14.286 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.286 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.286 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.286 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0000", 2024-09-12T03:20:14.286 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.286 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.286 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.287 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "beginning time of day to automatically balance", 2024-09-12T03:20:14.287 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-12T03:20:14.287 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.287 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.287 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.287 INFO:teuthology.orchestra.run.smithi027.stdout: "begin_weekday": { 2024-09-12T03:20:14.287 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "begin_weekday", 2024-09-12T03:20:14.287 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "uint", 2024-09-12T03:20:14.287 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.287 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.287 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-12T03:20:14.287 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "0", 2024-09-12T03:20:14.287 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "6", 2024-09-12T03:20:14.287 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.288 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-12T03:20:14.288 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-12T03:20:14.288 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.288 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.288 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.288 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_compat_max_iterations": { 2024-09-12T03:20:14.288 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "crush_compat_max_iterations", 2024-09-12T03:20:14.288 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "uint", 2024-09-12T03:20:14.288 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.288 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.288 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "25", 2024-09-12T03:20:14.288 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "1", 2024-09-12T03:20:14.288 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "250", 2024-09-12T03:20:14.288 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.289 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-12T03:20:14.289 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.289 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.289 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.289 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.289 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_compat_metrics": { 2024-09-12T03:20:14.289 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "crush_compat_metrics", 2024-09-12T03:20:14.289 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.289 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.289 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.289 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "pgs,objects,bytes", 2024-09-12T03:20:14.289 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.289 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.289 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.290 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-12T03:20:14.290 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-12T03:20:14.290 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.290 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.290 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.290 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_compat_step": { 2024-09-12T03:20:14.290 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "crush_compat_step", 2024-09-12T03:20:14.290 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "float", 2024-09-12T03:20:14.290 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.290 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.290 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0.5", 2024-09-12T03:20:14.290 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "0.001", 2024-09-12T03:20:14.290 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "0.999", 2024-09-12T03:20:14.291 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.291 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "aggressiveness of optimization", 2024-09-12T03:20:14.291 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-12T03:20:14.291 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.291 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.291 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.291 INFO:teuthology.orchestra.run.smithi027.stdout: "end_time": { 2024-09-12T03:20:14.291 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "end_time", 2024-09-12T03:20:14.291 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.291 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.291 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.291 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "2359", 2024-09-12T03:20:14.291 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.291 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.292 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.292 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "ending time of day to automatically balance", 2024-09-12T03:20:14.292 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-12T03:20:14.292 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.292 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.292 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.292 INFO:teuthology.orchestra.run.smithi027.stdout: "end_weekday": { 2024-09-12T03:20:14.292 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "end_weekday", 2024-09-12T03:20:14.292 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "uint", 2024-09-12T03:20:14.292 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.292 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.292 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-12T03:20:14.292 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "0", 2024-09-12T03:20:14.293 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "6", 2024-09-12T03:20:14.293 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.293 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-12T03:20:14.293 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-12T03:20:14.293 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.293 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.293 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.293 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-12T03:20:14.293 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-12T03:20:14.293 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.293 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.293 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.293 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-12T03:20:14.293 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.294 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.294 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-12T03:20:14.294 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-12T03:20:14.294 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-12T03:20:14.294 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-12T03:20:14.294 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-12T03:20:14.294 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-12T03:20:14.294 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-12T03:20:14.294 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-12T03:20:14.294 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.294 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.294 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.294 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.295 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.295 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-12T03:20:14.295 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-12T03:20:14.295 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.295 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.295 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.295 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-12T03:20:14.295 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.295 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.295 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.295 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.295 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.295 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.295 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.296 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.296 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-12T03:20:14.296 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-12T03:20:14.296 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.296 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.296 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.296 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-12T03:20:14.296 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.296 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.296 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-12T03:20:14.296 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-12T03:20:14.296 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-12T03:20:14.296 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-12T03:20:14.296 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-12T03:20:14.297 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-12T03:20:14.297 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-12T03:20:14.297 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-12T03:20:14.297 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.297 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.297 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.297 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.297 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.297 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-12T03:20:14.297 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-12T03:20:14.297 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.297 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.297 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.298 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-12T03:20:14.298 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.298 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.298 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.298 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.298 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.298 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.298 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.298 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.298 INFO:teuthology.orchestra.run.smithi027.stdout: "min_score": { 2024-09-12T03:20:14.298 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "min_score", 2024-09-12T03:20:14.298 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "float", 2024-09-12T03:20:14.298 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.298 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.299 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-12T03:20:14.299 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.299 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.299 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.299 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-12T03:20:14.299 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.299 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.299 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.299 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.299 INFO:teuthology.orchestra.run.smithi027.stdout: "mode": { 2024-09-12T03:20:14.299 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "mode", 2024-09-12T03:20:14.299 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.299 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.299 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.300 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "upmap", 2024-09-12T03:20:14.300 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.300 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.300 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-12T03:20:14.300 INFO:teuthology.orchestra.run.smithi027.stdout: "crush-compat", 2024-09-12T03:20:14.300 INFO:teuthology.orchestra.run.smithi027.stdout: "none", 2024-09-12T03:20:14.300 INFO:teuthology.orchestra.run.smithi027.stdout: "read", 2024-09-12T03:20:14.300 INFO:teuthology.orchestra.run.smithi027.stdout: "upmap", 2024-09-12T03:20:14.300 INFO:teuthology.orchestra.run.smithi027.stdout: "upmap-read" 2024-09-12T03:20:14.300 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-12T03:20:14.300 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Balancer mode", 2024-09-12T03:20:14.300 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.300 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.300 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.301 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.301 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_ids": { 2024-09-12T03:20:14.301 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "pool_ids", 2024-09-12T03:20:14.301 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.301 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.301 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.301 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-12T03:20:14.301 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.301 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.301 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.301 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-12T03:20:14.301 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.301 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.301 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.302 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.302 INFO:teuthology.orchestra.run.smithi027.stdout: "sleep_interval": { 2024-09-12T03:20:14.302 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sleep_interval", 2024-09-12T03:20:14.302 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-12T03:20:14.302 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.302 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.302 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "60", 2024-09-12T03:20:14.302 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.302 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.302 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.302 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-12T03:20:14.302 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.302 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.302 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.303 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.303 INFO:teuthology.orchestra.run.smithi027.stdout: "upmap_max_deviation": { 2024-09-12T03:20:14.303 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "upmap_max_deviation", 2024-09-12T03:20:14.303 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-12T03:20:14.303 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.303 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.303 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "5", 2024-09-12T03:20:14.303 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "1", 2024-09-12T03:20:14.303 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.303 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.303 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-12T03:20:14.303 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-12T03:20:14.303 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.303 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.304 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.304 INFO:teuthology.orchestra.run.smithi027.stdout: "upmap_max_optimizations": { 2024-09-12T03:20:14.304 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "upmap_max_optimizations", 2024-09-12T03:20:14.304 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "uint", 2024-09-12T03:20:14.304 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.304 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.304 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10", 2024-09-12T03:20:14.304 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.304 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.304 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.304 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-12T03:20:14.304 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.304 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.305 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.305 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:20:14.305 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:20:14.305 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.305 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:20:14.305 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "cephadm", 2024-09-12T03:20:14.305 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-12T03:20:14.305 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-12T03:20:14.305 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-12T03:20:14.305 INFO:teuthology.orchestra.run.smithi027.stdout: "agent_down_multiplier": { 2024-09-12T03:20:14.305 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "agent_down_multiplier", 2024-09-12T03:20:14.305 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "float", 2024-09-12T03:20:14.305 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.305 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.306 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "3.0", 2024-09-12T03:20:14.306 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.306 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.306 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.306 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-12T03:20:14.306 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.306 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.306 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.306 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.306 INFO:teuthology.orchestra.run.smithi027.stdout: "agent_refresh_rate": { 2024-09-12T03:20:14.306 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "agent_refresh_rate", 2024-09-12T03:20:14.306 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-12T03:20:14.306 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.306 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.307 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "20", 2024-09-12T03:20:14.307 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.307 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.307 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.307 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-12T03:20:14.307 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.307 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.307 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.307 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.307 INFO:teuthology.orchestra.run.smithi027.stdout: "agent_starting_port": { 2024-09-12T03:20:14.307 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "agent_starting_port", 2024-09-12T03:20:14.307 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-12T03:20:14.307 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.307 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.308 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "4721", 2024-09-12T03:20:14.308 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.308 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.308 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.308 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-12T03:20:14.308 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.308 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.308 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.308 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.308 INFO:teuthology.orchestra.run.smithi027.stdout: "allow_ptrace": { 2024-09-12T03:20:14.308 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "allow_ptrace", 2024-09-12T03:20:14.308 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.308 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.308 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.309 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-12T03:20:14.309 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.309 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.309 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.309 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-12T03:20:14.309 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-12T03:20:14.309 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.309 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.309 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.309 INFO:teuthology.orchestra.run.smithi027.stdout: "autotune_interval": { 2024-09-12T03:20:14.309 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "autotune_interval", 2024-09-12T03:20:14.309 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-12T03:20:14.309 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.309 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.310 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "600", 2024-09-12T03:20:14.310 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.310 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.310 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.310 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-12T03:20:14.310 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.310 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.310 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.310 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.310 INFO:teuthology.orchestra.run.smithi027.stdout: "autotune_memory_target_ratio": { 2024-09-12T03:20:14.310 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "autotune_memory_target_ratio", 2024-09-12T03:20:14.310 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "float", 2024-09-12T03:20:14.310 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.311 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.311 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0.7", 2024-09-12T03:20:14.311 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.311 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.311 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.311 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-12T03:20:14.311 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.311 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.311 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.311 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.311 INFO:teuthology.orchestra.run.smithi027.stdout: "cephadm_log_destination": { 2024-09-12T03:20:14.311 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "cephadm_log_destination", 2024-09-12T03:20:14.311 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.312 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.312 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.312 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-12T03:20:14.312 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.312 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.312 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-12T03:20:14.312 INFO:teuthology.orchestra.run.smithi027.stdout: "file", 2024-09-12T03:20:14.312 INFO:teuthology.orchestra.run.smithi027.stdout: "file,syslog", 2024-09-12T03:20:14.312 INFO:teuthology.orchestra.run.smithi027.stdout: "syslog" 2024-09-12T03:20:14.312 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-12T03:20:14.312 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-12T03:20:14.312 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.312 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.312 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.313 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.313 INFO:teuthology.orchestra.run.smithi027.stdout: "cgroups_split": { 2024-09-12T03:20:14.313 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "cgroups_split", 2024-09-12T03:20:14.313 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.313 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.313 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.313 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-12T03:20:14.313 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.313 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.313 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.313 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-12T03:20:14.313 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.313 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.313 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.314 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.314 INFO:teuthology.orchestra.run.smithi027.stdout: "config_checks_enabled": { 2024-09-12T03:20:14.314 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "config_checks_enabled", 2024-09-12T03:20:14.314 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.314 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.314 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.314 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-12T03:20:14.314 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.314 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.314 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.314 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-12T03:20:14.314 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.314 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.314 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.315 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.315 INFO:teuthology.orchestra.run.smithi027.stdout: "config_dashboard": { 2024-09-12T03:20:14.315 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "config_dashboard", 2024-09-12T03:20:14.315 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.315 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.315 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.315 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-12T03:20:14.315 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.315 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.315 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.315 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-12T03:20:14.315 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.315 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.315 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.316 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.316 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_alertmanager": { 2024-09-12T03:20:14.316 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_alertmanager", 2024-09-12T03:20:14.316 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.316 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.316 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.316 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-12T03:20:14.316 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.316 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.316 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.316 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Prometheus container image", 2024-09-12T03:20:14.316 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.316 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.316 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.317 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.317 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_base": { 2024-09-12T03:20:14.317 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_base", 2024-09-12T03:20:14.317 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.317 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.317 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.317 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-12T03:20:14.317 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.317 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.317 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.317 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Container image name, without the tag", 2024-09-12T03:20:14.317 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.317 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.317 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.318 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.318 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_elasticsearch": { 2024-09-12T03:20:14.318 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_elasticsearch", 2024-09-12T03:20:14.318 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.318 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.318 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.318 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-12T03:20:14.318 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.318 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.318 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.318 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "elasticsearch container image", 2024-09-12T03:20:14.318 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.318 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.318 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.319 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.319 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_grafana": { 2024-09-12T03:20:14.319 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_grafana", 2024-09-12T03:20:14.319 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.319 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.319 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.319 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-09-12T03:20:14.319 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.319 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.319 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.319 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Prometheus container image", 2024-09-12T03:20:14.319 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.319 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.319 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.320 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.320 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_haproxy": { 2024-09-12T03:20:14.320 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_haproxy", 2024-09-12T03:20:14.320 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.320 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.320 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.320 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-12T03:20:14.320 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.320 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.320 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.320 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "HAproxy container image", 2024-09-12T03:20:14.320 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.320 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.321 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.321 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.321 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_jaeger_agent": { 2024-09-12T03:20:14.321 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_jaeger_agent", 2024-09-12T03:20:14.321 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.321 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.321 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.321 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-12T03:20:14.321 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.321 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.321 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.321 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Jaeger agent container image", 2024-09-12T03:20:14.321 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.322 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.322 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.322 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.322 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_jaeger_collector": { 2024-09-12T03:20:14.322 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_jaeger_collector", 2024-09-12T03:20:14.322 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.322 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.322 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.322 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-12T03:20:14.322 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.322 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.322 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.322 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Jaeger collector container image", 2024-09-12T03:20:14.322 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.323 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.323 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.323 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.323 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_jaeger_query": { 2024-09-12T03:20:14.323 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_jaeger_query", 2024-09-12T03:20:14.323 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.323 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.323 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.323 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-12T03:20:14.323 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.323 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.323 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.323 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Jaeger query container image", 2024-09-12T03:20:14.323 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.324 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.324 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.324 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.324 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_keepalived": { 2024-09-12T03:20:14.324 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_keepalived", 2024-09-12T03:20:14.324 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.324 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.324 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.324 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-12T03:20:14.324 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.324 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.324 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.324 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Keepalived container image", 2024-09-12T03:20:14.325 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.325 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.325 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.325 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.325 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_loki": { 2024-09-12T03:20:14.325 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_loki", 2024-09-12T03:20:14.325 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.325 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.325 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.325 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-12T03:20:14.325 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.325 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.325 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.326 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Loki container image", 2024-09-12T03:20:14.326 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.326 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.326 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.326 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.326 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_node_exporter": { 2024-09-12T03:20:14.326 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_node_exporter", 2024-09-12T03:20:14.326 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.326 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.326 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.326 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-12T03:20:14.326 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.326 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.326 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.327 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Prometheus container image", 2024-09-12T03:20:14.327 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.327 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.327 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.327 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.327 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_nvmeof": { 2024-09-12T03:20:14.327 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_nvmeof", 2024-09-12T03:20:14.327 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.327 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.327 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.327 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-12T03:20:14.327 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.327 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.327 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.328 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Nvme-of container image", 2024-09-12T03:20:14.328 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.328 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.328 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.328 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.328 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_prometheus": { 2024-09-12T03:20:14.328 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_prometheus", 2024-09-12T03:20:14.328 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.328 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.328 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.328 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-12T03:20:14.328 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.328 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.329 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.329 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Prometheus container image", 2024-09-12T03:20:14.329 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.329 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.329 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.329 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.329 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_promtail": { 2024-09-12T03:20:14.329 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_promtail", 2024-09-12T03:20:14.329 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.329 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.329 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.329 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-12T03:20:14.329 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.329 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.330 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.330 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Promtail container image", 2024-09-12T03:20:14.330 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.330 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.330 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.330 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.330 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_samba": { 2024-09-12T03:20:14.330 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_samba", 2024-09-12T03:20:14.330 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.330 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.330 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.330 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-12T03:20:14.330 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.330 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.331 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.331 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Samba/SMB container image", 2024-09-12T03:20:14.331 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.331 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.331 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.331 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.331 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_snmp_gateway": { 2024-09-12T03:20:14.331 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_snmp_gateway", 2024-09-12T03:20:14.331 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.331 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.331 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.331 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-12T03:20:14.331 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.331 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.331 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.332 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "SNMP Gateway container image", 2024-09-12T03:20:14.332 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.332 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.332 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.332 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.332 INFO:teuthology.orchestra.run.smithi027.stdout: "container_init": { 2024-09-12T03:20:14.332 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_init", 2024-09-12T03:20:14.332 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.332 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.332 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.332 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-12T03:20:14.332 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.332 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.333 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.333 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Run podman/docker with `--init`", 2024-09-12T03:20:14.333 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.333 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.333 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.333 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.333 INFO:teuthology.orchestra.run.smithi027.stdout: "daemon_cache_timeout": { 2024-09-12T03:20:14.333 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "daemon_cache_timeout", 2024-09-12T03:20:14.333 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-12T03:20:14.333 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.333 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.333 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "600", 2024-09-12T03:20:14.333 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.333 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.334 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.334 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-12T03:20:14.334 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.334 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.334 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.334 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.334 INFO:teuthology.orchestra.run.smithi027.stdout: "default_cephadm_command_timeout": { 2024-09-12T03:20:14.334 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "default_cephadm_command_timeout", 2024-09-12T03:20:14.334 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-12T03:20:14.334 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.334 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.334 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "900", 2024-09-12T03:20:14.334 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.334 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.335 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.335 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-12T03:20:14.335 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.335 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.335 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.335 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.335 INFO:teuthology.orchestra.run.smithi027.stdout: "default_registry": { 2024-09-12T03:20:14.335 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "default_registry", 2024-09-12T03:20:14.335 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.335 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.335 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.335 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "docker.io", 2024-09-12T03:20:14.335 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.335 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.336 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.336 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-12T03:20:14.336 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.336 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.336 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.336 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.336 INFO:teuthology.orchestra.run.smithi027.stdout: "device_cache_timeout": { 2024-09-12T03:20:14.336 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "device_cache_timeout", 2024-09-12T03:20:14.336 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-12T03:20:14.336 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.336 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.336 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "1800", 2024-09-12T03:20:14.336 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.336 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.337 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.337 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "seconds to cache device inventory", 2024-09-12T03:20:14.337 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.337 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.337 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.337 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.337 INFO:teuthology.orchestra.run.smithi027.stdout: "device_enhanced_scan": { 2024-09-12T03:20:14.337 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "device_enhanced_scan", 2024-09-12T03:20:14.337 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.337 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.337 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.337 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-12T03:20:14.337 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.337 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.338 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.338 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-12T03:20:14.338 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.338 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.338 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.338 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.338 INFO:teuthology.orchestra.run.smithi027.stdout: "facts_cache_timeout": { 2024-09-12T03:20:14.338 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "facts_cache_timeout", 2024-09-12T03:20:14.338 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-12T03:20:14.338 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.338 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.338 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "60", 2024-09-12T03:20:14.338 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.338 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.339 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.339 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "seconds to cache host facts data", 2024-09-12T03:20:14.339 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.339 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.339 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.339 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.339 INFO:teuthology.orchestra.run.smithi027.stdout: "grafana_dashboards_path": { 2024-09-12T03:20:14.339 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "grafana_dashboards_path", 2024-09-12T03:20:14.339 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.339 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.339 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.339 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-12T03:20:14.339 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.340 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.340 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.340 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-12T03:20:14.340 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.340 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.340 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.340 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.340 INFO:teuthology.orchestra.run.smithi027.stdout: "host_check_interval": { 2024-09-12T03:20:14.340 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "host_check_interval", 2024-09-12T03:20:14.340 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-12T03:20:14.340 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.340 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.340 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "600", 2024-09-12T03:20:14.341 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.341 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.341 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.341 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "how frequently to perform a host check", 2024-09-12T03:20:14.341 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.341 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.341 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.341 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.341 INFO:teuthology.orchestra.run.smithi027.stdout: "hw_monitoring": { 2024-09-12T03:20:14.341 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "hw_monitoring", 2024-09-12T03:20:14.341 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.341 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.341 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.341 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-12T03:20:14.342 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.342 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.342 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.342 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-12T03:20:14.342 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.342 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.342 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.342 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.342 INFO:teuthology.orchestra.run.smithi027.stdout: "inventory_list_all": { 2024-09-12T03:20:14.342 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "inventory_list_all", 2024-09-12T03:20:14.342 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.342 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.342 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.342 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-12T03:20:14.343 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.343 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.343 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.343 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-12T03:20:14.343 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.343 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.343 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.343 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.343 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-12T03:20:14.343 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-12T03:20:14.343 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.343 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.343 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.343 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-12T03:20:14.344 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.344 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.344 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-12T03:20:14.344 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-12T03:20:14.344 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-12T03:20:14.344 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-12T03:20:14.344 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-12T03:20:14.344 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-12T03:20:14.344 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-12T03:20:14.344 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-12T03:20:14.344 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.344 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.344 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.344 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.345 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.345 INFO:teuthology.orchestra.run.smithi027.stdout: "log_refresh_metadata": { 2024-09-12T03:20:14.345 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_refresh_metadata", 2024-09-12T03:20:14.345 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.345 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.345 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.345 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-12T03:20:14.345 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.345 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.345 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.345 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-12T03:20:14.345 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.345 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.345 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.346 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.346 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-12T03:20:14.346 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-12T03:20:14.346 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.346 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.346 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.346 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-12T03:20:14.346 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.346 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.346 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.346 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-12T03:20:14.346 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.346 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.347 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.347 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.347 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-12T03:20:14.347 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-12T03:20:14.347 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.347 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.347 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.347 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-12T03:20:14.347 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.347 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.347 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-12T03:20:14.347 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-12T03:20:14.347 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-12T03:20:14.347 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-12T03:20:14.348 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-12T03:20:14.348 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-12T03:20:14.348 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-12T03:20:14.348 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-12T03:20:14.348 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.348 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.348 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.348 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.348 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.348 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-12T03:20:14.348 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-12T03:20:14.348 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.348 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.348 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.349 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-12T03:20:14.349 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.349 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.349 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.349 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.349 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.349 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.349 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.349 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.349 INFO:teuthology.orchestra.run.smithi027.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-12T03:20:14.349 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-12T03:20:14.349 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.349 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.349 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.350 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-12T03:20:14.350 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.350 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.350 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.350 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-12T03:20:14.350 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.350 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.350 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.350 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.350 INFO:teuthology.orchestra.run.smithi027.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-12T03:20:14.350 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-12T03:20:14.350 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.350 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.351 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.351 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "*", 2024-09-12T03:20:14.351 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.351 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.351 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.351 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-12T03:20:14.351 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.351 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.351 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.351 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.351 INFO:teuthology.orchestra.run.smithi027.stdout: "max_count_per_host": { 2024-09-12T03:20:14.351 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "max_count_per_host", 2024-09-12T03:20:14.351 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-12T03:20:14.351 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.351 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.352 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10", 2024-09-12T03:20:14.352 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.352 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.352 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.352 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "max number of daemons per service per host", 2024-09-12T03:20:14.352 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.352 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.352 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.352 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.352 INFO:teuthology.orchestra.run.smithi027.stdout: "max_osd_draining_count": { 2024-09-12T03:20:14.352 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "max_osd_draining_count", 2024-09-12T03:20:14.352 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-12T03:20:14.352 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.353 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.353 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10", 2024-09-12T03:20:14.353 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.353 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.353 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.353 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-12T03:20:14.353 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.353 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.353 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.353 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.353 INFO:teuthology.orchestra.run.smithi027.stdout: "migration_current": { 2024-09-12T03:20:14.353 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "migration_current", 2024-09-12T03:20:14.353 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-12T03:20:14.353 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.354 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.354 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-12T03:20:14.354 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.354 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.354 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.354 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "internal - do not modify", 2024-09-12T03:20:14.354 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.354 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.354 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.354 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.354 INFO:teuthology.orchestra.run.smithi027.stdout: "mode": { 2024-09-12T03:20:14.354 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "mode", 2024-09-12T03:20:14.354 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.354 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.355 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.355 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "root", 2024-09-12T03:20:14.355 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.355 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.355 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-12T03:20:14.355 INFO:teuthology.orchestra.run.smithi027.stdout: "cephadm-package", 2024-09-12T03:20:14.355 INFO:teuthology.orchestra.run.smithi027.stdout: "root" 2024-09-12T03:20:14.355 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-12T03:20:14.355 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "mode for remote execution of cephadm", 2024-09-12T03:20:14.355 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.355 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.355 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.355 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.356 INFO:teuthology.orchestra.run.smithi027.stdout: "oob_default_addr": { 2024-09-12T03:20:14.356 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "oob_default_addr", 2024-09-12T03:20:14.356 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.356 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.356 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.356 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "169.254.1.1", 2024-09-12T03:20:14.356 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.356 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.356 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.356 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-12T03:20:14.356 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.356 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.356 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.356 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.357 INFO:teuthology.orchestra.run.smithi027.stdout: "prometheus_alerts_path": { 2024-09-12T03:20:14.357 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "prometheus_alerts_path", 2024-09-12T03:20:14.357 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.357 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.357 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.357 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-12T03:20:14.357 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.357 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.357 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.357 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-12T03:20:14.357 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.357 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.357 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.357 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.358 INFO:teuthology.orchestra.run.smithi027.stdout: "registry_insecure": { 2024-09-12T03:20:14.358 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "registry_insecure", 2024-09-12T03:20:14.358 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.358 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.358 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.358 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-12T03:20:14.358 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.358 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.358 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.358 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-12T03:20:14.358 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.358 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.358 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.358 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.359 INFO:teuthology.orchestra.run.smithi027.stdout: "registry_password": { 2024-09-12T03:20:14.359 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "registry_password", 2024-09-12T03:20:14.359 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.359 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.359 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.359 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-12T03:20:14.359 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.359 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.359 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.359 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-12T03:20:14.359 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.359 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.359 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.359 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.360 INFO:teuthology.orchestra.run.smithi027.stdout: "registry_url": { 2024-09-12T03:20:14.360 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "registry_url", 2024-09-12T03:20:14.360 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.360 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.360 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.360 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-12T03:20:14.360 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.360 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.360 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.360 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-12T03:20:14.360 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.360 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.360 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.360 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.361 INFO:teuthology.orchestra.run.smithi027.stdout: "registry_username": { 2024-09-12T03:20:14.361 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "registry_username", 2024-09-12T03:20:14.361 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.361 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.361 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.361 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-12T03:20:14.361 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.361 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.361 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.361 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-12T03:20:14.361 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.361 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.361 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.362 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.362 INFO:teuthology.orchestra.run.smithi027.stdout: "secure_monitoring_stack": { 2024-09-12T03:20:14.362 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "secure_monitoring_stack", 2024-09-12T03:20:14.362 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.362 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.362 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.362 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-12T03:20:14.362 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.362 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.362 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.362 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-12T03:20:14.362 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.362 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.362 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.363 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.363 INFO:teuthology.orchestra.run.smithi027.stdout: "service_discovery_port": { 2024-09-12T03:20:14.363 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "service_discovery_port", 2024-09-12T03:20:14.363 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-12T03:20:14.363 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.363 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.363 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "8765", 2024-09-12T03:20:14.363 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.363 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.363 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.363 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "cephadm service discovery port", 2024-09-12T03:20:14.363 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.363 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.363 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.364 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.364 INFO:teuthology.orchestra.run.smithi027.stdout: "ssh_config_file": { 2024-09-12T03:20:14.364 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ssh_config_file", 2024-09-12T03:20:14.364 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.364 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.364 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.364 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-12T03:20:14.364 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.364 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.364 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.364 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-12T03:20:14.364 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.364 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.364 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.365 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.365 INFO:teuthology.orchestra.run.smithi027.stdout: "use_agent": { 2024-09-12T03:20:14.365 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "use_agent", 2024-09-12T03:20:14.365 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.365 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.365 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.365 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-12T03:20:14.365 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.365 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.365 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.365 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-12T03:20:14.365 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.365 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.366 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.366 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.366 INFO:teuthology.orchestra.run.smithi027.stdout: "use_repo_digest": { 2024-09-12T03:20:14.366 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "use_repo_digest", 2024-09-12T03:20:14.366 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.366 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.366 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.366 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-12T03:20:14.366 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.366 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.366 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.366 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-12T03:20:14.366 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.366 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.367 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.367 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.367 INFO:teuthology.orchestra.run.smithi027.stdout: "warn_on_failed_host_check": { 2024-09-12T03:20:14.367 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "warn_on_failed_host_check", 2024-09-12T03:20:14.367 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.367 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.367 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.367 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-12T03:20:14.367 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.367 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.367 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.367 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "raise a health warning if the host check fails", 2024-09-12T03:20:14.367 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.367 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.367 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.368 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.368 INFO:teuthology.orchestra.run.smithi027.stdout: "warn_on_stray_daemons": { 2024-09-12T03:20:14.368 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "warn_on_stray_daemons", 2024-09-12T03:20:14.368 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.368 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.368 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.368 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-12T03:20:14.368 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.368 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.368 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.368 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-12T03:20:14.368 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.368 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.369 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.369 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.369 INFO:teuthology.orchestra.run.smithi027.stdout: "warn_on_stray_hosts": { 2024-09-12T03:20:14.369 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "warn_on_stray_hosts", 2024-09-12T03:20:14.369 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.369 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.369 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.369 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-12T03:20:14.369 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.369 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.369 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.369 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-12T03:20:14.369 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.370 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.370 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.370 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:20:14.370 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:20:14.370 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.370 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:20:14.370 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "crash", 2024-09-12T03:20:14.370 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-12T03:20:14.370 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-12T03:20:14.370 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-12T03:20:14.370 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-12T03:20:14.370 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-12T03:20:14.370 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.370 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.371 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.371 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-12T03:20:14.371 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.371 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.371 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-12T03:20:14.371 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-12T03:20:14.371 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-12T03:20:14.371 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-12T03:20:14.371 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-12T03:20:14.371 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-12T03:20:14.371 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-12T03:20:14.371 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-12T03:20:14.371 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.371 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.371 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.372 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.372 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.372 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-12T03:20:14.372 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-12T03:20:14.372 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.372 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.372 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.372 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-12T03:20:14.372 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.372 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.372 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.372 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.372 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.373 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.373 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.373 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.373 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-12T03:20:14.373 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-12T03:20:14.373 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.373 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.373 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.373 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-12T03:20:14.373 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.373 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.373 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-12T03:20:14.373 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-12T03:20:14.373 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-12T03:20:14.374 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-12T03:20:14.374 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-12T03:20:14.374 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-12T03:20:14.374 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-12T03:20:14.374 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-12T03:20:14.374 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.374 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.374 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.374 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.374 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.374 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-12T03:20:14.374 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-12T03:20:14.374 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.374 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.375 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.375 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-12T03:20:14.375 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.375 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.375 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.375 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.375 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.375 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.375 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.375 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.375 INFO:teuthology.orchestra.run.smithi027.stdout: "retain_interval": { 2024-09-12T03:20:14.375 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "retain_interval", 2024-09-12T03:20:14.375 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-12T03:20:14.375 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.376 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.376 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "31536000", 2024-09-12T03:20:14.376 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.376 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.376 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.376 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-12T03:20:14.376 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.376 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.376 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.376 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.376 INFO:teuthology.orchestra.run.smithi027.stdout: "warn_recent_interval": { 2024-09-12T03:20:14.376 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "warn_recent_interval", 2024-09-12T03:20:14.376 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-12T03:20:14.377 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.377 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.377 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "1209600", 2024-09-12T03:20:14.377 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.377 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.377 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.377 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-12T03:20:14.377 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.377 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.377 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.377 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:20:14.377 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:20:14.377 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.377 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:20:14.377 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "dashboard", 2024-09-12T03:20:14.378 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-12T03:20:14.378 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-12T03:20:14.378 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-12T03:20:14.378 INFO:teuthology.orchestra.run.smithi027.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-12T03:20:14.378 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-12T03:20:14.378 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-12T03:20:14.378 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.378 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.378 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10", 2024-09-12T03:20:14.378 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.378 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.378 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.378 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.378 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.379 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.379 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.379 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.379 INFO:teuthology.orchestra.run.smithi027.stdout: "ALERTMANAGER_API_HOST": { 2024-09-12T03:20:14.379 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-12T03:20:14.379 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.379 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.379 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.379 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-12T03:20:14.379 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.379 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.379 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.379 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.379 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.380 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.380 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.380 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.380 INFO:teuthology.orchestra.run.smithi027.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-12T03:20:14.380 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-12T03:20:14.380 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.380 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.380 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.380 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-12T03:20:14.380 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.380 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.380 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.380 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.380 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.381 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.381 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.381 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.381 INFO:teuthology.orchestra.run.smithi027.stdout: "AUDIT_API_ENABLED": { 2024-09-12T03:20:14.381 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "AUDIT_API_ENABLED", 2024-09-12T03:20:14.381 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.381 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.381 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.381 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-12T03:20:14.381 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.381 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.381 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.381 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.381 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.382 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.382 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.382 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.382 INFO:teuthology.orchestra.run.smithi027.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-12T03:20:14.382 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-12T03:20:14.382 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.382 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.382 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.382 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-12T03:20:14.382 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.382 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.382 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.382 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.382 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.383 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.383 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.383 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.383 INFO:teuthology.orchestra.run.smithi027.stdout: "ENABLE_BROWSABLE_API": { 2024-09-12T03:20:14.383 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-12T03:20:14.383 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.383 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.383 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.383 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-12T03:20:14.383 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.383 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.383 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.383 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.384 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.384 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.384 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.384 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.384 INFO:teuthology.orchestra.run.smithi027.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-12T03:20:14.384 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-12T03:20:14.384 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.384 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.384 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.384 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-12T03:20:14.384 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.384 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.384 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.384 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.385 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.385 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.385 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.385 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.385 INFO:teuthology.orchestra.run.smithi027.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-12T03:20:14.385 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-12T03:20:14.385 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.385 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.385 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.385 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-12T03:20:14.385 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.385 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.385 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.386 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.386 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.386 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.386 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.386 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.386 INFO:teuthology.orchestra.run.smithi027.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-12T03:20:14.386 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-12T03:20:14.386 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.386 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.386 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.386 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-12T03:20:14.386 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.386 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.386 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.387 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.387 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.387 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.387 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.387 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.387 INFO:teuthology.orchestra.run.smithi027.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-12T03:20:14.387 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-12T03:20:14.387 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.387 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.387 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.387 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-12T03:20:14.387 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.387 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.387 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.388 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.388 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.388 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.388 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.388 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.388 INFO:teuthology.orchestra.run.smithi027.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-12T03:20:14.388 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-12T03:20:14.388 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.388 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.388 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.388 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-12T03:20:14.388 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.388 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.389 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.389 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.389 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.389 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.389 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.389 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.389 INFO:teuthology.orchestra.run.smithi027.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-12T03:20:14.389 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-12T03:20:14.389 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.389 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.389 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.389 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-12T03:20:14.389 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.389 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.390 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.390 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.390 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.390 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.390 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.390 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.390 INFO:teuthology.orchestra.run.smithi027.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-12T03:20:14.390 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-12T03:20:14.390 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.390 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.390 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.390 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-12T03:20:14.390 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.390 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.391 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.391 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.391 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.391 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.391 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.391 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.391 INFO:teuthology.orchestra.run.smithi027.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-12T03:20:14.391 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-12T03:20:14.391 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.391 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.391 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.391 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-12T03:20:14.391 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.391 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.392 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.392 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.392 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.392 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.392 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.392 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.392 INFO:teuthology.orchestra.run.smithi027.stdout: "GRAFANA_API_PASSWORD": { 2024-09-12T03:20:14.392 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-12T03:20:14.392 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.392 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.392 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.392 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "admin", 2024-09-12T03:20:14.392 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.392 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.393 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.393 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.393 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.393 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.393 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.393 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.393 INFO:teuthology.orchestra.run.smithi027.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-12T03:20:14.393 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-12T03:20:14.393 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.393 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.393 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.393 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-12T03:20:14.393 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.394 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.394 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.394 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.394 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.394 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.394 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.394 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.394 INFO:teuthology.orchestra.run.smithi027.stdout: "GRAFANA_API_URL": { 2024-09-12T03:20:14.394 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "GRAFANA_API_URL", 2024-09-12T03:20:14.394 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.394 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.394 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.394 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-12T03:20:14.394 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.395 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.395 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.395 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.395 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.395 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.395 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.395 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.395 INFO:teuthology.orchestra.run.smithi027.stdout: "GRAFANA_API_USERNAME": { 2024-09-12T03:20:14.395 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-12T03:20:14.395 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.395 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.395 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.395 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "admin", 2024-09-12T03:20:14.395 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.396 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.396 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.396 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.396 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.396 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.396 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.396 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.396 INFO:teuthology.orchestra.run.smithi027.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-12T03:20:14.396 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-12T03:20:14.396 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.396 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.396 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.396 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-12T03:20:14.397 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.397 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.397 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.397 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.397 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.397 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.397 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.397 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.397 INFO:teuthology.orchestra.run.smithi027.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-12T03:20:14.397 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-12T03:20:14.397 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.397 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.397 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.397 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-12T03:20:14.398 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.398 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.398 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.398 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.398 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.398 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.398 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.398 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.398 INFO:teuthology.orchestra.run.smithi027.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-12T03:20:14.398 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-12T03:20:14.398 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.398 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.398 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.399 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-12T03:20:14.399 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.399 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.399 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.399 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.399 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.399 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.399 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.399 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.399 INFO:teuthology.orchestra.run.smithi027.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-12T03:20:14.399 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-12T03:20:14.399 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.399 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.399 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.400 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-12T03:20:14.400 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.400 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.400 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.400 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.400 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.400 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.400 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.400 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.400 INFO:teuthology.orchestra.run.smithi027.stdout: "PROMETHEUS_API_HOST": { 2024-09-12T03:20:14.400 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-12T03:20:14.400 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.400 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.401 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.401 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-12T03:20:14.401 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.401 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.401 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.401 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.401 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.401 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.401 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.401 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.401 INFO:teuthology.orchestra.run.smithi027.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-12T03:20:14.401 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-12T03:20:14.401 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.401 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.402 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.402 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-12T03:20:14.402 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.402 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.402 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.402 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.402 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.402 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.402 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.402 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.402 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-12T03:20:14.402 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-12T03:20:14.402 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.403 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.403 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.403 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-12T03:20:14.403 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.403 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.403 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.403 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.403 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.403 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.403 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.403 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.403 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-12T03:20:14.403 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-12T03:20:14.404 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.404 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.404 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.404 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-12T03:20:14.404 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.404 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.404 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.404 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.404 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.404 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.404 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.404 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.404 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-12T03:20:14.405 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-12T03:20:14.405 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.405 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.405 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.405 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-12T03:20:14.405 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.405 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.405 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.405 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.405 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.405 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.405 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.405 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.405 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-12T03:20:14.406 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-12T03:20:14.406 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.406 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.406 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.406 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-12T03:20:14.406 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.406 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.406 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.406 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.406 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.406 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.406 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.406 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.406 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-12T03:20:14.406 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-12T03:20:14.407 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.407 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.407 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.407 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-12T03:20:14.407 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.407 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.407 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.407 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.407 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.407 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.407 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.407 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.407 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-12T03:20:14.407 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-12T03:20:14.408 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.408 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.408 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.408 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-12T03:20:14.408 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.408 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.408 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.408 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.408 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.408 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.408 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.408 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.408 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-12T03:20:14.408 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-12T03:20:14.409 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.409 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.409 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.409 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-12T03:20:14.409 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.409 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.409 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.409 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.409 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.409 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.409 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.409 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.409 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_ENABLED": { 2024-09-12T03:20:14.409 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-12T03:20:14.410 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.410 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.410 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.410 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-12T03:20:14.410 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.410 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.410 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.410 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.410 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.410 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.410 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.410 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.410 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-12T03:20:14.410 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-12T03:20:14.411 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.411 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.411 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.411 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-12T03:20:14.411 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.411 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.411 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.411 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.411 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.411 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.411 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.411 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.411 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-12T03:20:14.411 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-12T03:20:14.411 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-12T03:20:14.412 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.412 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.412 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10", 2024-09-12T03:20:14.412 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.412 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.412 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.412 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.412 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.412 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.412 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.412 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.412 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-12T03:20:14.412 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-12T03:20:14.413 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-12T03:20:14.413 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.413 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.413 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "8", 2024-09-12T03:20:14.413 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.413 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.413 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.413 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.413 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.413 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.413 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.413 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.413 INFO:teuthology.orchestra.run.smithi027.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-12T03:20:14.413 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-12T03:20:14.414 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-12T03:20:14.414 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.414 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.414 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "45", 2024-09-12T03:20:14.414 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.414 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.414 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.414 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.414 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.414 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.414 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.414 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.414 INFO:teuthology.orchestra.run.smithi027.stdout: "RGW_API_ACCESS_KEY": { 2024-09-12T03:20:14.414 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-12T03:20:14.415 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.415 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.415 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.415 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-12T03:20:14.415 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.415 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.415 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.415 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.415 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.415 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.415 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.415 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.415 INFO:teuthology.orchestra.run.smithi027.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-12T03:20:14.415 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-12T03:20:14.416 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.416 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.416 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.416 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "admin", 2024-09-12T03:20:14.416 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.416 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.416 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.416 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.416 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.416 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.416 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.416 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.416 INFO:teuthology.orchestra.run.smithi027.stdout: "RGW_API_SECRET_KEY": { 2024-09-12T03:20:14.416 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-12T03:20:14.417 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.417 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.417 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.417 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-12T03:20:14.417 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.417 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.417 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.417 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.417 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.417 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.417 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.417 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.417 INFO:teuthology.orchestra.run.smithi027.stdout: "RGW_API_SSL_VERIFY": { 2024-09-12T03:20:14.417 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-12T03:20:14.418 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.418 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.418 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.418 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-12T03:20:14.418 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.418 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.418 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.418 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.418 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.418 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.418 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.418 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.418 INFO:teuthology.orchestra.run.smithi027.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-12T03:20:14.418 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-12T03:20:14.419 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-12T03:20:14.419 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.419 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.419 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-12T03:20:14.419 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.419 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.419 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.419 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.419 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.419 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.419 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.419 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.419 INFO:teuthology.orchestra.run.smithi027.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-12T03:20:14.420 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-12T03:20:14.420 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-12T03:20:14.420 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.420 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.420 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10", 2024-09-12T03:20:14.420 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.420 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.420 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.420 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.420 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.420 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.420 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.420 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.421 INFO:teuthology.orchestra.run.smithi027.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-12T03:20:14.421 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-12T03:20:14.421 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-12T03:20:14.421 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.421 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.421 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "5", 2024-09-12T03:20:14.421 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.421 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.421 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.421 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.421 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.421 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.421 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.421 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.422 INFO:teuthology.orchestra.run.smithi027.stdout: "cross_origin_url": { 2024-09-12T03:20:14.422 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "cross_origin_url", 2024-09-12T03:20:14.422 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.422 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.422 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.422 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-12T03:20:14.422 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.422 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.422 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.422 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.422 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.422 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.422 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.422 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.423 INFO:teuthology.orchestra.run.smithi027.stdout: "crt_file": { 2024-09-12T03:20:14.423 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "crt_file", 2024-09-12T03:20:14.423 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.423 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.423 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.423 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-12T03:20:14.423 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.423 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.423 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.423 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.423 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.423 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.423 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.424 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.424 INFO:teuthology.orchestra.run.smithi027.stdout: "debug": { 2024-09-12T03:20:14.424 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "debug", 2024-09-12T03:20:14.424 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.424 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.424 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.424 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-12T03:20:14.424 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.424 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.424 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.424 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Enable/disable debug options", 2024-09-12T03:20:14.424 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.424 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.424 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.425 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.425 INFO:teuthology.orchestra.run.smithi027.stdout: "jwt_token_ttl": { 2024-09-12T03:20:14.425 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "jwt_token_ttl", 2024-09-12T03:20:14.425 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-12T03:20:14.425 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.425 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.425 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "28800", 2024-09-12T03:20:14.425 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.425 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.425 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.425 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.425 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.425 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.425 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.426 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.426 INFO:teuthology.orchestra.run.smithi027.stdout: "key_file": { 2024-09-12T03:20:14.426 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "key_file", 2024-09-12T03:20:14.426 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.426 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.426 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.426 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-12T03:20:14.426 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.426 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.426 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.426 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.426 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.426 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.426 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.427 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.427 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-12T03:20:14.427 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-12T03:20:14.427 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.427 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.427 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.427 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-12T03:20:14.427 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.427 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.427 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-12T03:20:14.427 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-12T03:20:14.427 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-12T03:20:14.427 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-12T03:20:14.427 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-12T03:20:14.428 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-12T03:20:14.428 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-12T03:20:14.428 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-12T03:20:14.428 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.428 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.428 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.428 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.428 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.428 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-12T03:20:14.428 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-12T03:20:14.428 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.428 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.428 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.428 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-12T03:20:14.429 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.429 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.429 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.429 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.429 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.429 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.429 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.429 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.429 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-12T03:20:14.429 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-12T03:20:14.429 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.429 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.429 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.430 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-12T03:20:14.430 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.430 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.430 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-12T03:20:14.430 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-12T03:20:14.430 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-12T03:20:14.430 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-12T03:20:14.430 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-12T03:20:14.430 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-12T03:20:14.430 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-12T03:20:14.430 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-12T03:20:14.430 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.430 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.430 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.431 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.431 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.431 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-12T03:20:14.431 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-12T03:20:14.431 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.431 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.431 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.431 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-12T03:20:14.431 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.431 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.431 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.431 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.431 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.432 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.432 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.432 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.432 INFO:teuthology.orchestra.run.smithi027.stdout: "motd": { 2024-09-12T03:20:14.432 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "motd", 2024-09-12T03:20:14.432 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.432 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.432 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.432 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-12T03:20:14.432 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.432 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.432 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.432 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "The message of the day", 2024-09-12T03:20:14.433 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.433 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.433 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.433 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.433 INFO:teuthology.orchestra.run.smithi027.stdout: "redirect_resolve_ip_addr": { 2024-09-12T03:20:14.433 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "redirect_resolve_ip_addr", 2024-09-12T03:20:14.433 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.433 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.433 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.433 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-12T03:20:14.433 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.433 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.433 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.433 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.434 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.434 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.434 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.434 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.434 INFO:teuthology.orchestra.run.smithi027.stdout: "server_addr": { 2024-09-12T03:20:14.434 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "server_addr", 2024-09-12T03:20:14.434 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.434 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.434 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.434 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "::", 2024-09-12T03:20:14.434 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.434 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.434 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.434 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.435 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.435 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.435 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.435 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.435 INFO:teuthology.orchestra.run.smithi027.stdout: "server_port": { 2024-09-12T03:20:14.435 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "server_port", 2024-09-12T03:20:14.435 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-12T03:20:14.435 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.435 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.435 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "8080", 2024-09-12T03:20:14.435 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.435 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.435 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.435 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.436 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.436 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.436 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.436 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.436 INFO:teuthology.orchestra.run.smithi027.stdout: "ssl": { 2024-09-12T03:20:14.436 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ssl", 2024-09-12T03:20:14.436 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.436 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.436 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.436 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-12T03:20:14.436 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.436 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.436 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.437 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.437 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.437 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.437 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.437 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.437 INFO:teuthology.orchestra.run.smithi027.stdout: "ssl_server_port": { 2024-09-12T03:20:14.437 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ssl_server_port", 2024-09-12T03:20:14.437 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-12T03:20:14.437 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.437 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.437 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "8443", 2024-09-12T03:20:14.437 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.437 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.437 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.438 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.438 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.438 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.438 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.438 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.438 INFO:teuthology.orchestra.run.smithi027.stdout: "standby_behaviour": { 2024-09-12T03:20:14.438 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "standby_behaviour", 2024-09-12T03:20:14.438 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.438 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.438 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.438 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "redirect", 2024-09-12T03:20:14.438 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.438 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.438 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-12T03:20:14.439 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-12T03:20:14.439 INFO:teuthology.orchestra.run.smithi027.stdout: "redirect" 2024-09-12T03:20:14.439 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-12T03:20:14.439 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.439 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.439 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.439 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.439 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.439 INFO:teuthology.orchestra.run.smithi027.stdout: "standby_error_status_code": { 2024-09-12T03:20:14.439 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "standby_error_status_code", 2024-09-12T03:20:14.439 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-12T03:20:14.439 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.439 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.440 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "500", 2024-09-12T03:20:14.440 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "400", 2024-09-12T03:20:14.440 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "599", 2024-09-12T03:20:14.440 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.440 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.440 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.440 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.440 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.440 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.440 INFO:teuthology.orchestra.run.smithi027.stdout: "url_prefix": { 2024-09-12T03:20:14.440 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "url_prefix", 2024-09-12T03:20:14.440 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.440 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.440 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.441 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-12T03:20:14.441 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.441 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.441 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.441 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.441 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.441 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.441 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.441 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:20:14.441 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:20:14.441 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.441 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:20:14.441 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "devicehealth", 2024-09-12T03:20:14.441 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-12T03:20:14.442 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-12T03:20:14.442 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-12T03:20:14.442 INFO:teuthology.orchestra.run.smithi027.stdout: "enable_monitoring": { 2024-09-12T03:20:14.442 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "enable_monitoring", 2024-09-12T03:20:14.442 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.442 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.442 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.442 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-12T03:20:14.442 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.442 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.442 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.442 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "monitor device health metrics", 2024-09-12T03:20:14.442 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.443 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.443 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.443 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.443 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-12T03:20:14.443 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-12T03:20:14.443 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.443 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.443 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.443 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-12T03:20:14.443 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.443 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.443 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-12T03:20:14.443 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-12T03:20:14.443 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-12T03:20:14.444 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-12T03:20:14.444 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-12T03:20:14.444 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-12T03:20:14.444 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-12T03:20:14.444 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-12T03:20:14.444 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.444 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.444 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.444 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.444 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.444 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-12T03:20:14.444 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-12T03:20:14.444 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.444 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.445 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.445 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-12T03:20:14.445 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.445 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.445 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.445 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.445 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.445 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.445 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.445 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.445 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-12T03:20:14.445 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-12T03:20:14.445 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.445 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.446 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.446 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-12T03:20:14.446 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.446 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.446 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-12T03:20:14.446 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-12T03:20:14.446 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-12T03:20:14.446 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-12T03:20:14.446 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-12T03:20:14.446 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-12T03:20:14.446 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-12T03:20:14.446 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-12T03:20:14.446 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.447 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.447 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.447 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.447 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.447 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-12T03:20:14.447 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-12T03:20:14.447 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.447 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.447 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.447 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-12T03:20:14.447 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.447 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.447 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.447 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.448 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.448 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.448 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.448 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.448 INFO:teuthology.orchestra.run.smithi027.stdout: "mark_out_threshold": { 2024-09-12T03:20:14.448 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "mark_out_threshold", 2024-09-12T03:20:14.448 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-12T03:20:14.448 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.448 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.448 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "2419200", 2024-09-12T03:20:14.448 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.448 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.448 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.449 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-12T03:20:14.449 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.449 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.449 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.449 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.449 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_name": { 2024-09-12T03:20:14.449 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "pool_name", 2024-09-12T03:20:14.449 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.449 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.449 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.452 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "device_health_metrics", 2024-09-12T03:20:14.452 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.453 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.453 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.453 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-12T03:20:14.453 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.453 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.453 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.453 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.453 INFO:teuthology.orchestra.run.smithi027.stdout: "retention_period": { 2024-09-12T03:20:14.453 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "retention_period", 2024-09-12T03:20:14.453 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-12T03:20:14.453 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.453 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.454 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "15552000", 2024-09-12T03:20:14.454 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.454 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.454 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.454 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "how long to retain device health metrics", 2024-09-12T03:20:14.454 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.454 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.454 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.454 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.454 INFO:teuthology.orchestra.run.smithi027.stdout: "scrape_frequency": { 2024-09-12T03:20:14.454 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "scrape_frequency", 2024-09-12T03:20:14.454 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-12T03:20:14.454 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.454 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.455 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "86400", 2024-09-12T03:20:14.455 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.455 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.455 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.455 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-12T03:20:14.455 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.455 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.455 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.455 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.455 INFO:teuthology.orchestra.run.smithi027.stdout: "self_heal": { 2024-09-12T03:20:14.455 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "self_heal", 2024-09-12T03:20:14.455 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.455 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.456 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.456 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-12T03:20:14.456 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.456 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.456 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.456 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-12T03:20:14.456 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.456 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.456 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.456 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.456 INFO:teuthology.orchestra.run.smithi027.stdout: "sleep_interval": { 2024-09-12T03:20:14.456 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sleep_interval", 2024-09-12T03:20:14.456 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-12T03:20:14.457 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.457 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.457 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "600", 2024-09-12T03:20:14.457 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.457 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.457 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.457 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "how frequently to wake up and check device health", 2024-09-12T03:20:14.457 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.457 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.457 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.457 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.457 INFO:teuthology.orchestra.run.smithi027.stdout: "warn_threshold": { 2024-09-12T03:20:14.457 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "warn_threshold", 2024-09-12T03:20:14.458 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-12T03:20:14.458 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.458 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.458 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "7257600", 2024-09-12T03:20:14.458 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.458 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.458 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.458 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-12T03:20:14.458 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.458 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.458 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.458 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:20:14.458 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:20:14.458 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.459 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:20:14.459 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "diskprediction_local", 2024-09-12T03:20:14.459 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-12T03:20:14.459 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-12T03:20:14.459 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-12T03:20:14.459 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-12T03:20:14.459 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-12T03:20:14.459 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.459 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.459 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.459 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-12T03:20:14.459 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.459 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.459 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-12T03:20:14.459 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-12T03:20:14.460 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-12T03:20:14.460 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-12T03:20:14.460 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-12T03:20:14.460 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-12T03:20:14.460 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-12T03:20:14.460 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-12T03:20:14.460 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.460 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.460 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.460 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.460 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.460 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-12T03:20:14.460 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-12T03:20:14.461 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.461 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.461 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.461 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-12T03:20:14.461 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.461 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.461 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.461 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.461 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.461 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.461 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.461 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.461 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-12T03:20:14.461 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-12T03:20:14.462 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.462 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.462 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.462 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-12T03:20:14.462 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.462 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.462 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-12T03:20:14.462 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-12T03:20:14.462 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-12T03:20:14.462 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-12T03:20:14.462 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-12T03:20:14.462 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-12T03:20:14.462 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-12T03:20:14.462 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-12T03:20:14.462 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.463 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.463 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.463 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.463 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.463 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-12T03:20:14.463 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-12T03:20:14.463 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.463 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.463 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.463 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-12T03:20:14.463 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.463 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.463 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.463 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.464 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.464 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.464 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.464 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.464 INFO:teuthology.orchestra.run.smithi027.stdout: "predict_interval": { 2024-09-12T03:20:14.464 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "predict_interval", 2024-09-12T03:20:14.464 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.464 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.464 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.464 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "86400", 2024-09-12T03:20:14.464 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.464 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.464 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.464 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.465 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.465 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.465 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.465 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.465 INFO:teuthology.orchestra.run.smithi027.stdout: "predictor_model": { 2024-09-12T03:20:14.465 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "predictor_model", 2024-09-12T03:20:14.465 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.465 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.465 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.465 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "prophetstor", 2024-09-12T03:20:14.465 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.465 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.465 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.465 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.466 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.466 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.466 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.466 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.466 INFO:teuthology.orchestra.run.smithi027.stdout: "sleep_interval": { 2024-09-12T03:20:14.466 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sleep_interval", 2024-09-12T03:20:14.466 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.466 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.466 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.466 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "600", 2024-09-12T03:20:14.466 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.466 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.466 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.467 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.467 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.467 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.467 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.467 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:20:14.467 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:20:14.467 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.467 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:20:14.467 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "influx", 2024-09-12T03:20:14.467 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": false, 2024-09-12T03:20:14.467 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "influxdb python module not found", 2024-09-12T03:20:14.467 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-12T03:20:14.467 INFO:teuthology.orchestra.run.smithi027.stdout: "batch_size": { 2024-09-12T03:20:14.467 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "batch_size", 2024-09-12T03:20:14.468 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-12T03:20:14.468 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.468 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.468 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "5000", 2024-09-12T03:20:14.468 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.468 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.468 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.468 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-12T03:20:14.468 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.468 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.468 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.468 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.468 INFO:teuthology.orchestra.run.smithi027.stdout: "database": { 2024-09-12T03:20:14.468 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "database", 2024-09-12T03:20:14.469 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.469 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.469 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.469 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "ceph", 2024-09-12T03:20:14.469 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.469 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.469 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.469 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-12T03:20:14.469 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.469 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.469 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.469 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.469 INFO:teuthology.orchestra.run.smithi027.stdout: "hostname": { 2024-09-12T03:20:14.469 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "hostname", 2024-09-12T03:20:14.470 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.470 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.470 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.470 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-12T03:20:14.470 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.470 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.470 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.470 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "InfluxDB server hostname", 2024-09-12T03:20:14.470 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.470 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.470 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.470 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.470 INFO:teuthology.orchestra.run.smithi027.stdout: "interval": { 2024-09-12T03:20:14.470 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "interval", 2024-09-12T03:20:14.471 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-12T03:20:14.471 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.471 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.471 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "30", 2024-09-12T03:20:14.471 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "5", 2024-09-12T03:20:14.471 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.471 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.471 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-12T03:20:14.471 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.471 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.471 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.471 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.471 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-12T03:20:14.471 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-12T03:20:14.472 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.472 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.472 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.472 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-12T03:20:14.472 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.472 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.472 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-12T03:20:14.472 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-12T03:20:14.472 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-12T03:20:14.472 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-12T03:20:14.472 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-12T03:20:14.472 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-12T03:20:14.472 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-12T03:20:14.473 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-12T03:20:14.473 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.473 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.473 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.473 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.473 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.473 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-12T03:20:14.473 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-12T03:20:14.473 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.473 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.473 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.473 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-12T03:20:14.473 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.473 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.474 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.474 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.474 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.474 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.474 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.474 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.474 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-12T03:20:14.474 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-12T03:20:14.474 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.474 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.474 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.474 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-12T03:20:14.474 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.474 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.475 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-12T03:20:14.475 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-12T03:20:14.475 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-12T03:20:14.475 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-12T03:20:14.475 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-12T03:20:14.475 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-12T03:20:14.475 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-12T03:20:14.475 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-12T03:20:14.475 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.475 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.475 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.475 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.475 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.475 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-12T03:20:14.476 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-12T03:20:14.476 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.476 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.476 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.476 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-12T03:20:14.476 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.476 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.476 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.476 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.476 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.476 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.476 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.476 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.476 INFO:teuthology.orchestra.run.smithi027.stdout: "password": { 2024-09-12T03:20:14.477 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "password", 2024-09-12T03:20:14.477 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.477 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.477 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.477 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-12T03:20:14.477 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.477 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.477 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.477 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "password of InfluxDB server user", 2024-09-12T03:20:14.477 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.477 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.477 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.477 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.477 INFO:teuthology.orchestra.run.smithi027.stdout: "port": { 2024-09-12T03:20:14.478 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "port", 2024-09-12T03:20:14.478 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-12T03:20:14.478 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.478 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.478 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "8086", 2024-09-12T03:20:14.478 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.478 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.478 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.478 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "InfluxDB server port", 2024-09-12T03:20:14.478 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.478 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.478 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.478 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.478 INFO:teuthology.orchestra.run.smithi027.stdout: "ssl": { 2024-09-12T03:20:14.479 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ssl", 2024-09-12T03:20:14.479 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.479 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.479 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.479 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "false", 2024-09-12T03:20:14.479 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.479 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.479 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.479 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-12T03:20:14.479 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.479 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.479 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.479 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.479 INFO:teuthology.orchestra.run.smithi027.stdout: "threads": { 2024-09-12T03:20:14.480 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "threads", 2024-09-12T03:20:14.480 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-12T03:20:14.480 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.480 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.480 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "5", 2024-09-12T03:20:14.480 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "1", 2024-09-12T03:20:14.480 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "32", 2024-09-12T03:20:14.480 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.480 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-12T03:20:14.480 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.480 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.480 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.480 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.480 INFO:teuthology.orchestra.run.smithi027.stdout: "username": { 2024-09-12T03:20:14.481 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "username", 2024-09-12T03:20:14.481 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.481 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.481 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.481 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-12T03:20:14.481 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.481 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.481 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.481 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "username of InfluxDB server user", 2024-09-12T03:20:14.481 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.481 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.481 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.481 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.481 INFO:teuthology.orchestra.run.smithi027.stdout: "verify_ssl": { 2024-09-12T03:20:14.482 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "verify_ssl", 2024-09-12T03:20:14.482 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.482 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.482 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.482 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "true", 2024-09-12T03:20:14.482 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.482 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.482 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.482 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-12T03:20:14.482 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.482 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.482 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.482 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:20:14.482 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:20:14.483 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.483 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:20:14.483 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "insights", 2024-09-12T03:20:14.483 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-12T03:20:14.483 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-12T03:20:14.483 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-12T03:20:14.483 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-12T03:20:14.483 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-12T03:20:14.483 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.483 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.483 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.483 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-12T03:20:14.483 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.483 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.484 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-12T03:20:14.484 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-12T03:20:14.484 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-12T03:20:14.484 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-12T03:20:14.484 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-12T03:20:14.484 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-12T03:20:14.484 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-12T03:20:14.484 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-12T03:20:14.484 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.484 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.484 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.484 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.484 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.484 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-12T03:20:14.484 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-12T03:20:14.485 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.485 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.485 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.485 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-12T03:20:14.485 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.485 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.485 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.485 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.485 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.485 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.485 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.485 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.485 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-12T03:20:14.486 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-12T03:20:14.486 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.486 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.486 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.486 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-12T03:20:14.486 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.486 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.486 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-12T03:20:14.486 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-12T03:20:14.486 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-12T03:20:14.486 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-12T03:20:14.486 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-12T03:20:14.486 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-12T03:20:14.486 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-12T03:20:14.487 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-12T03:20:14.487 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.487 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.487 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.487 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.487 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.487 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-12T03:20:14.487 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-12T03:20:14.487 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.487 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.487 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.487 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-12T03:20:14.487 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.487 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.488 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.488 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.488 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.488 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.488 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.488 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:20:14.488 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:20:14.488 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.488 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:20:14.488 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "iostat", 2024-09-12T03:20:14.488 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-12T03:20:14.488 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-12T03:20:14.488 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-12T03:20:14.488 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-12T03:20:14.489 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-12T03:20:14.489 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.489 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.489 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.489 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-12T03:20:14.489 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.489 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.489 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-12T03:20:14.489 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-12T03:20:14.489 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-12T03:20:14.489 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-12T03:20:14.489 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-12T03:20:14.490 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-12T03:20:14.490 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-12T03:20:14.490 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-12T03:20:14.490 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.490 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.490 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.490 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.490 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.490 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-12T03:20:14.490 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-12T03:20:14.490 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.490 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.490 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.490 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-12T03:20:14.491 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.491 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.491 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.491 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.491 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.491 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.491 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.491 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.491 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-12T03:20:14.491 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-12T03:20:14.491 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.491 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.491 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.492 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-12T03:20:14.492 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.492 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.492 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-12T03:20:14.492 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-12T03:20:14.492 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-12T03:20:14.492 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-12T03:20:14.492 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-12T03:20:14.492 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-12T03:20:14.492 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-12T03:20:14.492 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-12T03:20:14.492 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.492 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.493 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.493 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.493 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.493 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-12T03:20:14.493 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-12T03:20:14.493 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.493 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.493 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.493 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-12T03:20:14.493 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.493 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.493 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.493 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.493 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.494 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.494 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.494 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:20:14.494 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:20:14.494 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.494 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:20:14.494 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "localpool", 2024-09-12T03:20:14.494 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-12T03:20:14.494 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-12T03:20:14.494 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-12T03:20:14.494 INFO:teuthology.orchestra.run.smithi027.stdout: "failure_domain": { 2024-09-12T03:20:14.494 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "failure_domain", 2024-09-12T03:20:14.494 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.494 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.495 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.495 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "host", 2024-09-12T03:20:14.495 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.495 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.495 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.495 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "failure domain for any created local pool", 2024-09-12T03:20:14.495 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-12T03:20:14.495 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.495 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.495 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.495 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-12T03:20:14.495 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-12T03:20:14.495 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.495 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.495 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.496 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-12T03:20:14.496 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.496 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.496 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-12T03:20:14.496 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-12T03:20:14.496 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-12T03:20:14.496 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-12T03:20:14.496 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-12T03:20:14.496 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-12T03:20:14.496 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-12T03:20:14.496 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-12T03:20:14.496 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.496 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.497 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.497 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.497 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.497 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-12T03:20:14.497 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-12T03:20:14.497 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.497 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.497 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.497 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-12T03:20:14.497 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.497 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.497 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.497 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.497 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.498 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.498 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.498 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.498 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-12T03:20:14.498 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-12T03:20:14.498 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.498 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.498 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.498 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-12T03:20:14.498 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.498 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.498 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-12T03:20:14.498 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-12T03:20:14.498 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-12T03:20:14.499 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-12T03:20:14.499 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-12T03:20:14.499 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-12T03:20:14.499 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-12T03:20:14.499 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-12T03:20:14.499 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.499 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.499 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.499 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.499 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.499 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-12T03:20:14.499 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-12T03:20:14.499 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.499 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.500 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.500 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-12T03:20:14.500 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.500 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.500 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.500 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.500 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.500 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.500 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.500 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.500 INFO:teuthology.orchestra.run.smithi027.stdout: "min_size": { 2024-09-12T03:20:14.500 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "min_size", 2024-09-12T03:20:14.500 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-12T03:20:14.500 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.501 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.501 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-12T03:20:14.501 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.501 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.501 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.501 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "default min_size for any created local pool", 2024-09-12T03:20:14.501 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-12T03:20:14.501 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.501 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.501 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.501 INFO:teuthology.orchestra.run.smithi027.stdout: "num_rep": { 2024-09-12T03:20:14.501 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "num_rep", 2024-09-12T03:20:14.501 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-12T03:20:14.501 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.502 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.502 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "3", 2024-09-12T03:20:14.502 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.502 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.502 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.502 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "default replica count for any created local pool", 2024-09-12T03:20:14.502 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.502 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.502 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.502 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.502 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num": { 2024-09-12T03:20:14.502 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "pg_num", 2024-09-12T03:20:14.502 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-12T03:20:14.502 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.503 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.503 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "128", 2024-09-12T03:20:14.503 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.503 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.503 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.503 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "default pg_num for any created local pool", 2024-09-12T03:20:14.503 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.503 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.503 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.503 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.503 INFO:teuthology.orchestra.run.smithi027.stdout: "prefix": { 2024-09-12T03:20:14.503 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "prefix", 2024-09-12T03:20:14.503 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.504 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.504 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.504 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-12T03:20:14.504 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.504 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.504 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.504 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "name prefix for any created local pool", 2024-09-12T03:20:14.504 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.504 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.504 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.504 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.504 INFO:teuthology.orchestra.run.smithi027.stdout: "subtree": { 2024-09-12T03:20:14.504 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "subtree", 2024-09-12T03:20:14.505 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.505 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.505 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.505 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "rack", 2024-09-12T03:20:14.505 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.505 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.505 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.505 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-12T03:20:14.505 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-12T03:20:14.505 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.505 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.505 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:20:14.505 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:20:14.505 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.506 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:20:14.506 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "mds_autoscaler", 2024-09-12T03:20:14.506 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-12T03:20:14.506 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-12T03:20:14.506 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-12T03:20:14.506 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-12T03:20:14.506 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-12T03:20:14.506 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.506 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.506 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.506 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-12T03:20:14.506 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.506 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.507 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-12T03:20:14.507 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-12T03:20:14.507 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-12T03:20:14.507 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-12T03:20:14.507 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-12T03:20:14.507 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-12T03:20:14.507 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-12T03:20:14.507 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-12T03:20:14.507 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.507 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.507 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.507 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.507 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.508 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-12T03:20:14.508 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-12T03:20:14.508 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.508 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.508 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.508 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-12T03:20:14.508 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.508 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.508 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.508 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.508 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.508 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.508 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.508 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.509 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-12T03:20:14.509 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-12T03:20:14.509 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.509 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.509 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.509 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-12T03:20:14.509 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.509 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.509 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-12T03:20:14.509 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-12T03:20:14.509 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-12T03:20:14.509 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-12T03:20:14.509 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-12T03:20:14.510 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-12T03:20:14.510 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-12T03:20:14.510 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-12T03:20:14.510 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.510 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.510 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.510 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.510 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.510 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-12T03:20:14.510 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-12T03:20:14.510 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.510 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.510 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.510 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-12T03:20:14.511 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.511 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.511 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.511 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.511 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.511 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.511 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.511 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:20:14.511 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:20:14.511 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.511 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:20:14.511 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "mirroring", 2024-09-12T03:20:14.511 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-12T03:20:14.511 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-12T03:20:14.512 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-12T03:20:14.512 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-12T03:20:14.512 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-12T03:20:14.512 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.512 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.512 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.512 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-12T03:20:14.512 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.512 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.512 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-12T03:20:14.512 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-12T03:20:14.512 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-12T03:20:14.512 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-12T03:20:14.512 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-12T03:20:14.513 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-12T03:20:14.513 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-12T03:20:14.513 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-12T03:20:14.513 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.513 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.513 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.513 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.513 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.513 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-12T03:20:14.513 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-12T03:20:14.513 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.513 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.513 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.514 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-12T03:20:14.514 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.514 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.514 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.514 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.514 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.514 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.514 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.514 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.514 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-12T03:20:14.514 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-12T03:20:14.514 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.514 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.514 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.514 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-12T03:20:14.515 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.515 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.515 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-12T03:20:14.515 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-12T03:20:14.515 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-12T03:20:14.515 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-12T03:20:14.515 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-12T03:20:14.515 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-12T03:20:14.515 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-12T03:20:14.515 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-12T03:20:14.515 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.515 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.516 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.516 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.516 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.516 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-12T03:20:14.516 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-12T03:20:14.516 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.516 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.516 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.516 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-12T03:20:14.516 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.516 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.516 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.516 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.516 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.517 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.517 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.517 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:20:14.517 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:20:14.517 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.517 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:20:14.517 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "nfs", 2024-09-12T03:20:14.517 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-12T03:20:14.517 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-12T03:20:14.517 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-12T03:20:14.517 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-12T03:20:14.517 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-12T03:20:14.517 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.517 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.518 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.518 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-12T03:20:14.518 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.518 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.518 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-12T03:20:14.518 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-12T03:20:14.518 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-12T03:20:14.518 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-12T03:20:14.518 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-12T03:20:14.518 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-12T03:20:14.518 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-12T03:20:14.518 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-12T03:20:14.518 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.518 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.519 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.519 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.519 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.519 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-12T03:20:14.519 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-12T03:20:14.519 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.519 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.519 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.519 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-12T03:20:14.519 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.519 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.519 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.519 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.520 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.520 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.520 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.520 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.520 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-12T03:20:14.520 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-12T03:20:14.520 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.520 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.520 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.520 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-12T03:20:14.520 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.520 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.520 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-12T03:20:14.520 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-12T03:20:14.521 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-12T03:20:14.521 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-12T03:20:14.521 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-12T03:20:14.521 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-12T03:20:14.521 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-12T03:20:14.521 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-12T03:20:14.521 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.521 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.521 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.521 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.521 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.521 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-12T03:20:14.521 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-12T03:20:14.521 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.522 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.522 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.522 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-12T03:20:14.522 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.522 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.522 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.522 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.522 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.522 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.522 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.522 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:20:14.522 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:20:14.522 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.522 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:20:14.523 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "orchestrator", 2024-09-12T03:20:14.523 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-12T03:20:14.523 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-12T03:20:14.523 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-12T03:20:14.523 INFO:teuthology.orchestra.run.smithi027.stdout: "fail_fs": { 2024-09-12T03:20:14.523 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "fail_fs", 2024-09-12T03:20:14.523 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.523 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.523 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.523 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-12T03:20:14.523 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.523 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.523 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.524 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-12T03:20:14.524 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.524 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.524 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.524 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.524 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-12T03:20:14.524 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-12T03:20:14.524 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.524 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.524 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.524 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-12T03:20:14.524 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.524 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.525 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-12T03:20:14.525 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-12T03:20:14.525 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-12T03:20:14.525 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-12T03:20:14.525 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-12T03:20:14.525 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-12T03:20:14.525 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-12T03:20:14.525 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-12T03:20:14.525 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.525 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.525 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.525 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.525 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.525 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-12T03:20:14.526 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-12T03:20:14.526 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.526 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.526 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.526 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-12T03:20:14.526 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.526 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.526 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.526 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.526 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.526 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.526 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.526 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.526 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-12T03:20:14.527 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-12T03:20:14.527 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.527 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.527 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.527 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-12T03:20:14.527 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.527 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.527 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-12T03:20:14.527 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-12T03:20:14.527 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-12T03:20:14.527 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-12T03:20:14.527 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-12T03:20:14.527 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-12T03:20:14.527 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-12T03:20:14.528 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-12T03:20:14.528 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.528 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.528 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.528 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.528 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.528 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-12T03:20:14.528 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-12T03:20:14.528 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.528 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.528 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.528 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-12T03:20:14.528 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.529 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.529 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.529 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.529 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.529 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.529 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.529 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.529 INFO:teuthology.orchestra.run.smithi027.stdout: "orchestrator": { 2024-09-12T03:20:14.529 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "orchestrator", 2024-09-12T03:20:14.529 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.529 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.529 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.529 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-12T03:20:14.529 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.530 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.530 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-12T03:20:14.530 INFO:teuthology.orchestra.run.smithi027.stdout: "cephadm", 2024-09-12T03:20:14.530 INFO:teuthology.orchestra.run.smithi027.stdout: "rook", 2024-09-12T03:20:14.530 INFO:teuthology.orchestra.run.smithi027.stdout: "test_orchestrator" 2024-09-12T03:20:14.530 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-12T03:20:14.530 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Orchestrator backend", 2024-09-12T03:20:14.530 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.530 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.530 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.530 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:20:14.530 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:20:14.530 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.530 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:20:14.531 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "osd_perf_query", 2024-09-12T03:20:14.531 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-12T03:20:14.531 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-12T03:20:14.531 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-12T03:20:14.531 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-12T03:20:14.531 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-12T03:20:14.531 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.531 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.531 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.531 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-12T03:20:14.531 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.531 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.531 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-12T03:20:14.531 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-12T03:20:14.532 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-12T03:20:14.532 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-12T03:20:14.532 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-12T03:20:14.532 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-12T03:20:14.532 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-12T03:20:14.532 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-12T03:20:14.532 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.532 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.532 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.532 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.532 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.532 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-12T03:20:14.532 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-12T03:20:14.533 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.533 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.533 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.533 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-12T03:20:14.533 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.533 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.533 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.533 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.533 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.533 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.533 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.533 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.533 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-12T03:20:14.533 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-12T03:20:14.534 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.534 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.534 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.534 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-12T03:20:14.534 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.534 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.534 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-12T03:20:14.534 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-12T03:20:14.534 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-12T03:20:14.534 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-12T03:20:14.534 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-12T03:20:14.534 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-12T03:20:14.534 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-12T03:20:14.534 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-12T03:20:14.535 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.535 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.535 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.535 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.535 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.535 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-12T03:20:14.535 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-12T03:20:14.535 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.535 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.535 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.535 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-12T03:20:14.535 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.535 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.535 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.536 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.536 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.536 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.536 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.536 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:20:14.536 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:20:14.536 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.536 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:20:14.536 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "osd_support", 2024-09-12T03:20:14.536 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-12T03:20:14.536 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-12T03:20:14.536 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-12T03:20:14.536 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-12T03:20:14.536 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-12T03:20:14.537 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.537 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.537 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.537 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-12T03:20:14.537 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.537 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.537 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-12T03:20:14.537 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-12T03:20:14.537 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-12T03:20:14.537 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-12T03:20:14.537 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-12T03:20:14.537 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-12T03:20:14.537 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-12T03:20:14.537 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-12T03:20:14.538 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.538 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.538 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.538 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.538 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.538 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-12T03:20:14.538 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-12T03:20:14.538 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.538 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.538 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.538 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-12T03:20:14.538 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.538 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.538 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.538 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.539 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.539 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.539 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.539 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.539 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-12T03:20:14.539 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-12T03:20:14.539 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.539 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.539 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.539 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-12T03:20:14.539 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.539 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.539 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-12T03:20:14.539 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-12T03:20:14.540 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-12T03:20:14.540 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-12T03:20:14.540 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-12T03:20:14.540 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-12T03:20:14.540 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-12T03:20:14.540 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-12T03:20:14.540 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.540 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.540 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.540 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.540 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.540 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-12T03:20:14.540 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-12T03:20:14.540 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.541 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.541 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.541 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-12T03:20:14.541 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.541 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.541 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.541 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.541 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.541 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.541 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.541 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:20:14.541 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:20:14.541 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.541 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:20:14.542 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "pg_autoscaler", 2024-09-12T03:20:14.542 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-12T03:20:14.542 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-12T03:20:14.542 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-12T03:20:14.542 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-12T03:20:14.542 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-12T03:20:14.542 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.542 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.542 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.542 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-12T03:20:14.542 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.542 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.542 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-12T03:20:14.543 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-12T03:20:14.543 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-12T03:20:14.543 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-12T03:20:14.543 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-12T03:20:14.543 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-12T03:20:14.543 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-12T03:20:14.543 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-12T03:20:14.543 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.543 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.543 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.543 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.543 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.543 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-12T03:20:14.543 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-12T03:20:14.544 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.544 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.544 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.544 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-12T03:20:14.544 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.544 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.544 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.544 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.544 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.544 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.544 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.544 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.544 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-12T03:20:14.544 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-12T03:20:14.545 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.545 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.545 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.545 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-12T03:20:14.545 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.545 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.545 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-12T03:20:14.545 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-12T03:20:14.545 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-12T03:20:14.545 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-12T03:20:14.545 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-12T03:20:14.545 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-12T03:20:14.545 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-12T03:20:14.545 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-12T03:20:14.546 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.546 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.546 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.546 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.546 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.546 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-12T03:20:14.546 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-12T03:20:14.546 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.546 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.546 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.546 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-12T03:20:14.546 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.546 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.546 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.547 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.547 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.547 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.547 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.547 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.547 INFO:teuthology.orchestra.run.smithi027.stdout: "sleep_interval": { 2024-09-12T03:20:14.547 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sleep_interval", 2024-09-12T03:20:14.547 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-12T03:20:14.547 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.547 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.547 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "60", 2024-09-12T03:20:14.547 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.547 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.548 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.548 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.548 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.548 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.548 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.548 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.548 INFO:teuthology.orchestra.run.smithi027.stdout: "threshold": { 2024-09-12T03:20:14.548 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "threshold", 2024-09-12T03:20:14.548 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "float", 2024-09-12T03:20:14.548 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.548 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.548 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "3.0", 2024-09-12T03:20:14.548 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "1.0", 2024-09-12T03:20:14.548 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.549 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.549 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "scaling threshold", 2024-09-12T03:20:14.549 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-12T03:20:14.549 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.549 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.549 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:20:14.549 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:20:14.549 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.549 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:20:14.549 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "progress", 2024-09-12T03:20:14.549 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-12T03:20:14.549 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-12T03:20:14.549 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-12T03:20:14.549 INFO:teuthology.orchestra.run.smithi027.stdout: "allow_pg_recovery_event": { 2024-09-12T03:20:14.550 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "allow_pg_recovery_event", 2024-09-12T03:20:14.550 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.550 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.550 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.550 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-12T03:20:14.550 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.550 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.550 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.550 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-12T03:20:14.550 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.550 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.550 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.550 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.550 INFO:teuthology.orchestra.run.smithi027.stdout: "enabled": { 2024-09-12T03:20:14.551 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "enabled", 2024-09-12T03:20:14.551 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.551 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.551 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.551 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-12T03:20:14.551 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.551 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.551 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.551 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.551 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.551 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.551 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.551 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.551 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-12T03:20:14.552 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-12T03:20:14.552 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.552 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.552 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.552 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-12T03:20:14.552 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.552 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.552 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-12T03:20:14.552 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-12T03:20:14.552 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-12T03:20:14.552 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-12T03:20:14.552 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-12T03:20:14.552 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-12T03:20:14.552 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-12T03:20:14.553 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-12T03:20:14.553 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.553 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.553 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.553 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.553 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.553 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-12T03:20:14.553 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-12T03:20:14.553 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.553 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.553 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.553 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-12T03:20:14.553 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.553 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.554 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.554 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.554 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.554 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.554 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.554 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.554 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-12T03:20:14.554 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-12T03:20:14.554 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.554 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.554 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.554 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-12T03:20:14.554 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.554 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.555 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-12T03:20:14.555 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-12T03:20:14.555 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-12T03:20:14.555 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-12T03:20:14.555 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-12T03:20:14.555 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-12T03:20:14.555 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-12T03:20:14.555 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-12T03:20:14.555 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.555 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.555 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.555 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.555 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.555 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-12T03:20:14.555 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-12T03:20:14.556 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.556 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.556 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.556 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-12T03:20:14.556 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.556 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.556 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.556 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.556 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.556 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.556 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.556 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.556 INFO:teuthology.orchestra.run.smithi027.stdout: "max_completed_events": { 2024-09-12T03:20:14.557 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "max_completed_events", 2024-09-12T03:20:14.557 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-12T03:20:14.557 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.557 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.557 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "50", 2024-09-12T03:20:14.557 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.557 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.557 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.557 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "number of past completed events to remember", 2024-09-12T03:20:14.557 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.557 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.557 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.557 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.557 INFO:teuthology.orchestra.run.smithi027.stdout: "sleep_interval": { 2024-09-12T03:20:14.558 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sleep_interval", 2024-09-12T03:20:14.558 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-12T03:20:14.558 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.558 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.558 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "5", 2024-09-12T03:20:14.558 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.558 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.558 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.558 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "how long the module is going to sleep", 2024-09-12T03:20:14.558 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.558 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.559 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.559 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:20:14.559 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:20:14.559 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.559 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:20:14.559 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "prometheus", 2024-09-12T03:20:14.559 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-12T03:20:14.559 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-12T03:20:14.559 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-12T03:20:14.559 INFO:teuthology.orchestra.run.smithi027.stdout: "cache": { 2024-09-12T03:20:14.559 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "cache", 2024-09-12T03:20:14.559 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.559 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.560 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.560 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-12T03:20:14.560 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.560 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.560 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.560 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.560 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.560 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.560 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.560 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.560 INFO:teuthology.orchestra.run.smithi027.stdout: "exclude_perf_counters": { 2024-09-12T03:20:14.560 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "exclude_perf_counters", 2024-09-12T03:20:14.560 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.561 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.561 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.561 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-12T03:20:14.561 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.561 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.561 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.561 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-12T03:20:14.561 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-12T03:20:14.561 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.561 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.561 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.561 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-12T03:20:14.561 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-12T03:20:14.562 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.562 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.562 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.562 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-12T03:20:14.562 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.562 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.562 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-12T03:20:14.562 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-12T03:20:14.562 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-12T03:20:14.562 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-12T03:20:14.562 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-12T03:20:14.562 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-12T03:20:14.562 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-12T03:20:14.562 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-12T03:20:14.563 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.563 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.563 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.563 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.563 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.563 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-12T03:20:14.563 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-12T03:20:14.563 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.563 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.563 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.563 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-12T03:20:14.563 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.563 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.564 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.564 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.564 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.564 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.564 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.564 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.564 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-12T03:20:14.564 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-12T03:20:14.564 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.564 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.564 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.564 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-12T03:20:14.564 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.565 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.565 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-12T03:20:14.565 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-12T03:20:14.565 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-12T03:20:14.565 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-12T03:20:14.565 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-12T03:20:14.565 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-12T03:20:14.565 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-12T03:20:14.565 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-12T03:20:14.565 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.565 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.565 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.565 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.565 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.566 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-12T03:20:14.566 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-12T03:20:14.566 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.566 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.566 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.566 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-12T03:20:14.566 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.566 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.566 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.566 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.566 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.566 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.566 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.567 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.567 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_stats_pools": { 2024-09-12T03:20:14.567 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rbd_stats_pools", 2024-09-12T03:20:14.567 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.567 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.567 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.567 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-12T03:20:14.567 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.567 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.567 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.567 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.567 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.567 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.567 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.568 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.568 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-12T03:20:14.568 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-12T03:20:14.568 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-12T03:20:14.568 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.568 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.568 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "300", 2024-09-12T03:20:14.568 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.568 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.568 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.568 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.568 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.568 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.569 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.569 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.569 INFO:teuthology.orchestra.run.smithi027.stdout: "scrape_interval": { 2024-09-12T03:20:14.569 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "scrape_interval", 2024-09-12T03:20:14.569 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "float", 2024-09-12T03:20:14.569 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.569 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.569 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "15.0", 2024-09-12T03:20:14.569 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.569 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.569 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.569 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.569 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.569 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.570 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.570 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.570 INFO:teuthology.orchestra.run.smithi027.stdout: "server_addr": { 2024-09-12T03:20:14.570 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "server_addr", 2024-09-12T03:20:14.570 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.570 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.570 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.570 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "::", 2024-09-12T03:20:14.570 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.570 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.570 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.570 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-12T03:20:14.570 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.571 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.571 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.571 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.571 INFO:teuthology.orchestra.run.smithi027.stdout: "server_port": { 2024-09-12T03:20:14.571 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "server_port", 2024-09-12T03:20:14.571 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-12T03:20:14.571 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.571 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.571 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "9283", 2024-09-12T03:20:14.571 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.571 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.571 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.571 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-12T03:20:14.571 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.572 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.572 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.572 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.572 INFO:teuthology.orchestra.run.smithi027.stdout: "stale_cache_strategy": { 2024-09-12T03:20:14.572 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "stale_cache_strategy", 2024-09-12T03:20:14.572 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.572 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.572 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.572 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "log", 2024-09-12T03:20:14.572 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.572 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.572 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.572 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.572 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.573 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.573 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.573 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.573 INFO:teuthology.orchestra.run.smithi027.stdout: "standby_behaviour": { 2024-09-12T03:20:14.573 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "standby_behaviour", 2024-09-12T03:20:14.573 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.573 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.573 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.573 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "default", 2024-09-12T03:20:14.573 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.573 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.573 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-12T03:20:14.573 INFO:teuthology.orchestra.run.smithi027.stdout: "default", 2024-09-12T03:20:14.573 INFO:teuthology.orchestra.run.smithi027.stdout: "error" 2024-09-12T03:20:14.574 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-12T03:20:14.574 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.574 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.574 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.574 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.574 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.574 INFO:teuthology.orchestra.run.smithi027.stdout: "standby_error_status_code": { 2024-09-12T03:20:14.574 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "standby_error_status_code", 2024-09-12T03:20:14.574 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-12T03:20:14.574 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.574 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.574 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "500", 2024-09-12T03:20:14.574 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "400", 2024-09-12T03:20:14.574 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "599", 2024-09-12T03:20:14.575 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.575 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.575 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.575 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.575 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.575 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:20:14.575 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:20:14.575 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.575 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:20:14.575 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rbd_support", 2024-09-12T03:20:14.575 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-12T03:20:14.575 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-12T03:20:14.575 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-12T03:20:14.576 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-12T03:20:14.576 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-12T03:20:14.576 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.576 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.576 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.576 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-12T03:20:14.576 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.576 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.576 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-12T03:20:14.576 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-12T03:20:14.576 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-12T03:20:14.576 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-12T03:20:14.576 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-12T03:20:14.576 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-12T03:20:14.577 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-12T03:20:14.577 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-12T03:20:14.577 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.577 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.577 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.577 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.577 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.577 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-12T03:20:14.577 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-12T03:20:14.577 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.577 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.577 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.577 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-12T03:20:14.577 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.578 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.578 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.578 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.578 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.578 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.578 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.578 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.578 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-12T03:20:14.578 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-12T03:20:14.578 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.578 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.578 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.578 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-12T03:20:14.579 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.579 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.579 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-12T03:20:14.579 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-12T03:20:14.579 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-12T03:20:14.579 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-12T03:20:14.579 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-12T03:20:14.579 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-12T03:20:14.579 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-12T03:20:14.579 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-12T03:20:14.579 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.579 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.579 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.579 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.580 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.580 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-12T03:20:14.580 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-12T03:20:14.580 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.580 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.580 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.580 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-12T03:20:14.580 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.580 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.580 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.580 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.580 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.580 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.581 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.581 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.581 INFO:teuthology.orchestra.run.smithi027.stdout: "max_concurrent_snap_create": { 2024-09-12T03:20:14.581 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "max_concurrent_snap_create", 2024-09-12T03:20:14.581 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-12T03:20:14.581 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.581 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.581 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10", 2024-09-12T03:20:14.581 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.581 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.581 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.581 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.581 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.581 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.582 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.582 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.582 INFO:teuthology.orchestra.run.smithi027.stdout: "mirror_snapshot_schedule": { 2024-09-12T03:20:14.582 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "mirror_snapshot_schedule", 2024-09-12T03:20:14.582 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.582 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.582 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.582 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-12T03:20:14.582 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.582 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.582 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.582 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.582 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.582 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.583 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.583 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.583 INFO:teuthology.orchestra.run.smithi027.stdout: "trash_purge_schedule": { 2024-09-12T03:20:14.583 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "trash_purge_schedule", 2024-09-12T03:20:14.583 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.583 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.583 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.583 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-12T03:20:14.583 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.583 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.583 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.583 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.583 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.584 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.584 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.584 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:20:14.584 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:20:14.584 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.584 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:20:14.584 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "restful", 2024-09-12T03:20:14.584 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-12T03:20:14.584 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-12T03:20:14.584 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-12T03:20:14.584 INFO:teuthology.orchestra.run.smithi027.stdout: "enable_auth": { 2024-09-12T03:20:14.584 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "enable_auth", 2024-09-12T03:20:14.584 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.585 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.585 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.585 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-12T03:20:14.585 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.585 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.585 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.585 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.585 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.585 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.585 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.585 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.585 INFO:teuthology.orchestra.run.smithi027.stdout: "key_file": { 2024-09-12T03:20:14.585 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "key_file", 2024-09-12T03:20:14.585 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.586 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.586 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.586 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-12T03:20:14.586 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.586 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.586 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.586 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.586 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.586 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.586 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.586 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.586 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-12T03:20:14.586 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-12T03:20:14.587 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.587 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.587 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.587 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-12T03:20:14.587 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.587 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.587 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-12T03:20:14.587 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-12T03:20:14.587 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-12T03:20:14.587 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-12T03:20:14.587 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-12T03:20:14.587 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-12T03:20:14.587 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-12T03:20:14.587 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-12T03:20:14.588 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.588 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.588 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.588 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.588 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.588 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-12T03:20:14.588 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-12T03:20:14.588 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.588 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.588 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.588 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-12T03:20:14.588 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.588 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.589 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.589 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.589 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.589 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.589 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.589 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.589 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-12T03:20:14.589 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-12T03:20:14.589 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.589 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.589 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.589 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-12T03:20:14.589 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.589 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.590 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-12T03:20:14.590 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-12T03:20:14.590 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-12T03:20:14.590 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-12T03:20:14.590 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-12T03:20:14.590 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-12T03:20:14.590 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-12T03:20:14.590 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-12T03:20:14.590 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.590 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.590 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.590 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.590 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.590 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-12T03:20:14.591 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-12T03:20:14.591 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.591 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.591 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.591 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-12T03:20:14.591 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.591 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.591 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.591 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.591 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.591 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.591 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.591 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.591 INFO:teuthology.orchestra.run.smithi027.stdout: "server_addr": { 2024-09-12T03:20:14.592 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "server_addr", 2024-09-12T03:20:14.592 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.592 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.592 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.592 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-12T03:20:14.592 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.592 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.592 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.592 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.592 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.592 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.592 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.592 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.593 INFO:teuthology.orchestra.run.smithi027.stdout: "server_port": { 2024-09-12T03:20:14.593 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "server_port", 2024-09-12T03:20:14.593 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.593 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.593 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.593 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-12T03:20:14.593 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.593 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.593 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.593 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.593 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.593 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.593 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.594 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:20:14.594 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:20:14.594 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.594 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:20:14.594 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rgw", 2024-09-12T03:20:14.594 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-12T03:20:14.594 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-12T03:20:14.594 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-12T03:20:14.594 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-12T03:20:14.594 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-12T03:20:14.594 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.594 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.594 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.595 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-12T03:20:14.595 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.595 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.595 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-12T03:20:14.595 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-12T03:20:14.595 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-12T03:20:14.595 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-12T03:20:14.595 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-12T03:20:14.595 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-12T03:20:14.595 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-12T03:20:14.595 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-12T03:20:14.595 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.595 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.595 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.595 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.596 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.596 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-12T03:20:14.596 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-12T03:20:14.596 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.596 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.596 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.596 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-12T03:20:14.596 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.596 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.596 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.596 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.596 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.596 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.597 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.597 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.597 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-12T03:20:14.597 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-12T03:20:14.597 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.597 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.597 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.597 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-12T03:20:14.597 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.597 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.597 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-12T03:20:14.597 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-12T03:20:14.597 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-12T03:20:14.598 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-12T03:20:14.598 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-12T03:20:14.598 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-12T03:20:14.598 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-12T03:20:14.598 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-12T03:20:14.598 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.598 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.598 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.598 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.598 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.598 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-12T03:20:14.598 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-12T03:20:14.598 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.598 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.598 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.599 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-12T03:20:14.599 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.599 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.599 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.599 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.599 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.599 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.599 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.599 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:20:14.599 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:20:14.599 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.599 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:20:14.599 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rook", 2024-09-12T03:20:14.599 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-12T03:20:14.600 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-12T03:20:14.600 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-12T03:20:14.600 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-12T03:20:14.600 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-12T03:20:14.600 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.600 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.600 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.600 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-12T03:20:14.600 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.600 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.600 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-12T03:20:14.600 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-12T03:20:14.600 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-12T03:20:14.600 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-12T03:20:14.601 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-12T03:20:14.601 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-12T03:20:14.601 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-12T03:20:14.601 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-12T03:20:14.601 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.601 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.601 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.601 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.601 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.601 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-12T03:20:14.601 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-12T03:20:14.601 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.601 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.602 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.602 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-12T03:20:14.602 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.602 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.602 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.602 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.602 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.602 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.602 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.602 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.602 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-12T03:20:14.602 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-12T03:20:14.602 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.602 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.603 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.603 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-12T03:20:14.603 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.603 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.603 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-12T03:20:14.603 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-12T03:20:14.603 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-12T03:20:14.603 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-12T03:20:14.603 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-12T03:20:14.603 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-12T03:20:14.603 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-12T03:20:14.603 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-12T03:20:14.603 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.603 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.604 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.604 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.604 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.604 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-12T03:20:14.604 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-12T03:20:14.604 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.604 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.604 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.604 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-12T03:20:14.604 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.604 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.604 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.604 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.604 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.605 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.605 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.605 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.605 INFO:teuthology.orchestra.run.smithi027.stdout: "storage_class": { 2024-09-12T03:20:14.605 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "storage_class", 2024-09-12T03:20:14.605 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.605 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.605 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.605 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "local", 2024-09-12T03:20:14.605 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.605 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.605 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.605 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-12T03:20:14.605 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.606 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.606 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.606 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:20:14.606 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:20:14.606 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.606 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:20:14.606 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "selftest", 2024-09-12T03:20:14.606 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-12T03:20:14.606 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-12T03:20:14.606 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-12T03:20:14.606 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-12T03:20:14.606 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-12T03:20:14.606 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.606 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.607 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.607 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-12T03:20:14.607 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.607 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.607 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-12T03:20:14.607 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-12T03:20:14.607 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-12T03:20:14.607 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-12T03:20:14.607 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-12T03:20:14.607 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-12T03:20:14.607 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-12T03:20:14.607 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-12T03:20:14.607 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.607 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.607 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.608 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.608 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.608 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-12T03:20:14.608 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-12T03:20:14.608 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.608 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.608 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.608 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-12T03:20:14.608 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.608 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.608 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.608 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.608 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.609 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.609 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.609 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.609 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-12T03:20:14.609 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-12T03:20:14.609 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.609 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.609 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.609 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-12T03:20:14.609 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.609 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.609 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-12T03:20:14.609 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-12T03:20:14.609 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-12T03:20:14.609 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-12T03:20:14.610 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-12T03:20:14.610 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-12T03:20:14.610 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-12T03:20:14.610 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-12T03:20:14.610 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.610 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.610 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.610 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.610 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.610 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-12T03:20:14.610 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-12T03:20:14.610 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.610 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.611 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.611 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-12T03:20:14.611 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.611 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.611 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.611 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.611 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.611 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.611 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.611 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.611 INFO:teuthology.orchestra.run.smithi027.stdout: "roption1": { 2024-09-12T03:20:14.611 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "roption1", 2024-09-12T03:20:14.611 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.611 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.612 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.612 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-12T03:20:14.612 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.612 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.612 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.612 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.612 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.612 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.612 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.612 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.612 INFO:teuthology.orchestra.run.smithi027.stdout: "roption2": { 2024-09-12T03:20:14.612 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "roption2", 2024-09-12T03:20:14.612 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.612 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.613 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.613 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "xyz", 2024-09-12T03:20:14.613 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.613 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.613 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.613 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.613 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.613 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.613 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.613 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.613 INFO:teuthology.orchestra.run.smithi027.stdout: "rwoption1": { 2024-09-12T03:20:14.613 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rwoption1", 2024-09-12T03:20:14.613 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.614 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.614 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.614 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-12T03:20:14.614 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.614 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.614 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.614 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.614 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.614 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.614 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.614 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.614 INFO:teuthology.orchestra.run.smithi027.stdout: "rwoption2": { 2024-09-12T03:20:14.614 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rwoption2", 2024-09-12T03:20:14.614 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-12T03:20:14.615 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.615 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.615 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-12T03:20:14.615 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.615 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.615 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.615 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.615 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.615 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.615 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.615 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.615 INFO:teuthology.orchestra.run.smithi027.stdout: "rwoption3": { 2024-09-12T03:20:14.615 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rwoption3", 2024-09-12T03:20:14.615 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "float", 2024-09-12T03:20:14.616 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.616 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.616 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-12T03:20:14.616 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.616 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.616 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.616 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.616 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.616 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.616 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.616 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.616 INFO:teuthology.orchestra.run.smithi027.stdout: "rwoption4": { 2024-09-12T03:20:14.616 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rwoption4", 2024-09-12T03:20:14.616 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.617 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.617 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.617 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-12T03:20:14.617 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.617 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.617 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.617 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.617 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.617 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.617 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.617 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.617 INFO:teuthology.orchestra.run.smithi027.stdout: "rwoption5": { 2024-09-12T03:20:14.617 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rwoption5", 2024-09-12T03:20:14.617 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.618 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.618 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.618 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-12T03:20:14.618 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.618 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.618 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.618 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.618 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.618 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.618 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.618 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.618 INFO:teuthology.orchestra.run.smithi027.stdout: "rwoption6": { 2024-09-12T03:20:14.618 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rwoption6", 2024-09-12T03:20:14.618 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.619 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.619 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.619 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-12T03:20:14.619 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.619 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.619 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.619 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.619 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.619 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.619 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.619 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.619 INFO:teuthology.orchestra.run.smithi027.stdout: "rwoption7": { 2024-09-12T03:20:14.619 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rwoption7", 2024-09-12T03:20:14.619 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-12T03:20:14.620 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.620 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.620 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-12T03:20:14.620 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "1", 2024-09-12T03:20:14.620 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "42", 2024-09-12T03:20:14.620 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.620 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.620 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.620 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.620 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.620 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.620 INFO:teuthology.orchestra.run.smithi027.stdout: "testkey": { 2024-09-12T03:20:14.620 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "testkey", 2024-09-12T03:20:14.620 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.621 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.621 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.621 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-12T03:20:14.621 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.621 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.621 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.621 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.621 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.621 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.621 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.621 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.621 INFO:teuthology.orchestra.run.smithi027.stdout: "testlkey": { 2024-09-12T03:20:14.621 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "testlkey", 2024-09-12T03:20:14.621 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.622 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.622 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.622 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-12T03:20:14.622 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.622 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.622 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.622 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.622 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.622 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.622 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.622 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.622 INFO:teuthology.orchestra.run.smithi027.stdout: "testnewline": { 2024-09-12T03:20:14.622 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "testnewline", 2024-09-12T03:20:14.622 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.623 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.623 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.623 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-12T03:20:14.623 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.623 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.623 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.623 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.623 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.623 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.623 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.623 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:20:14.623 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:20:14.623 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.624 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:20:14.624 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "snap_schedule", 2024-09-12T03:20:14.624 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-12T03:20:14.624 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-12T03:20:14.624 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-12T03:20:14.624 INFO:teuthology.orchestra.run.smithi027.stdout: "allow_m_granularity": { 2024-09-12T03:20:14.624 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "allow_m_granularity", 2024-09-12T03:20:14.624 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.624 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.624 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.624 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-12T03:20:14.624 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.624 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.624 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.625 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "allow minute scheduled snapshots", 2024-09-12T03:20:14.625 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.625 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.625 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.625 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.625 INFO:teuthology.orchestra.run.smithi027.stdout: "dump_on_update": { 2024-09-12T03:20:14.625 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "dump_on_update", 2024-09-12T03:20:14.625 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.625 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.625 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.625 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-12T03:20:14.625 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.625 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.625 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.626 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "dump database to debug log on update", 2024-09-12T03:20:14.626 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.626 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.626 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.626 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.626 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-12T03:20:14.626 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-12T03:20:14.626 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.626 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.626 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.626 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-12T03:20:14.626 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.626 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.626 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-12T03:20:14.627 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-12T03:20:14.627 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-12T03:20:14.627 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-12T03:20:14.627 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-12T03:20:14.627 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-12T03:20:14.627 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-12T03:20:14.627 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-12T03:20:14.627 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.627 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.627 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.627 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.627 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.627 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-12T03:20:14.628 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-12T03:20:14.628 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.628 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.628 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.628 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-12T03:20:14.628 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.628 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.628 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.628 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.628 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.628 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.628 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.629 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.629 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-12T03:20:14.629 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-12T03:20:14.629 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.629 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.629 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.629 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-12T03:20:14.629 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.629 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.629 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-12T03:20:14.629 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-12T03:20:14.629 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-12T03:20:14.629 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-12T03:20:14.629 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-12T03:20:14.630 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-12T03:20:14.630 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-12T03:20:14.630 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-12T03:20:14.630 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.630 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.630 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.630 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.630 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.630 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-12T03:20:14.630 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-12T03:20:14.630 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.630 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.630 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.631 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-12T03:20:14.631 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.631 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.631 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.631 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.631 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.631 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.631 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.631 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:20:14.631 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:20:14.631 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.631 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:20:14.631 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "stats", 2024-09-12T03:20:14.631 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-12T03:20:14.632 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-12T03:20:14.632 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-12T03:20:14.632 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-12T03:20:14.632 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-12T03:20:14.632 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.632 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.632 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.632 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-12T03:20:14.632 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.632 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.632 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-12T03:20:14.632 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-12T03:20:14.632 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-12T03:20:14.632 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-12T03:20:14.633 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-12T03:20:14.633 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-12T03:20:14.633 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-12T03:20:14.633 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-12T03:20:14.633 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.633 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.633 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.633 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.633 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.633 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-12T03:20:14.633 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-12T03:20:14.633 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.633 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.634 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.634 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-12T03:20:14.634 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.634 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.634 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.634 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.634 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.634 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.634 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.634 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.634 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-12T03:20:14.634 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-12T03:20:14.634 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.634 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.634 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.635 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-12T03:20:14.635 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.635 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.635 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-12T03:20:14.635 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-12T03:20:14.635 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-12T03:20:14.635 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-12T03:20:14.635 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-12T03:20:14.635 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-12T03:20:14.635 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-12T03:20:14.635 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-12T03:20:14.635 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.635 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.635 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.636 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.636 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.636 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-12T03:20:14.636 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-12T03:20:14.636 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.636 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.636 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.636 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-12T03:20:14.636 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.636 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.636 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.636 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.636 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.637 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.637 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.637 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:20:14.637 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:20:14.637 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.637 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:20:14.637 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "status", 2024-09-12T03:20:14.637 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-12T03:20:14.637 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-12T03:20:14.637 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-12T03:20:14.637 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-12T03:20:14.637 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-12T03:20:14.637 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.637 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.638 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.638 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-12T03:20:14.638 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.638 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.638 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-12T03:20:14.638 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-12T03:20:14.638 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-12T03:20:14.638 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-12T03:20:14.638 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-12T03:20:14.638 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-12T03:20:14.638 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-12T03:20:14.638 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-12T03:20:14.638 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.638 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.639 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.639 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.639 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.639 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-12T03:20:14.639 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-12T03:20:14.639 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.639 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.639 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.639 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-12T03:20:14.639 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.639 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.639 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.639 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.639 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.640 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.640 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.640 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.640 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-12T03:20:14.640 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-12T03:20:14.640 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.640 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.640 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.640 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-12T03:20:14.640 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.640 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.640 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-12T03:20:14.640 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-12T03:20:14.640 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-12T03:20:14.641 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-12T03:20:14.641 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-12T03:20:14.641 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-12T03:20:14.641 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-12T03:20:14.641 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-12T03:20:14.641 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.641 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.641 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.641 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.641 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.641 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-12T03:20:14.641 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-12T03:20:14.641 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.641 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.642 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.642 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-12T03:20:14.642 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.642 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.642 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.642 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.642 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.642 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.642 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.642 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:20:14.642 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:20:14.642 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.642 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:20:14.642 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "telegraf", 2024-09-12T03:20:14.642 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-12T03:20:14.643 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-12T03:20:14.643 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-12T03:20:14.643 INFO:teuthology.orchestra.run.smithi027.stdout: "address": { 2024-09-12T03:20:14.643 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "address", 2024-09-12T03:20:14.643 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.643 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.643 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.643 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-12T03:20:14.643 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.643 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.643 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.643 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.643 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.643 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.644 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.644 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.644 INFO:teuthology.orchestra.run.smithi027.stdout: "interval": { 2024-09-12T03:20:14.644 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "interval", 2024-09-12T03:20:14.644 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-12T03:20:14.644 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.644 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.644 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "15", 2024-09-12T03:20:14.644 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.644 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.644 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.644 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.644 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.645 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.645 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.645 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.645 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-12T03:20:14.645 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-12T03:20:14.645 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.645 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.645 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.645 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-12T03:20:14.645 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.645 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.645 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-12T03:20:14.645 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-12T03:20:14.645 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-12T03:20:14.646 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-12T03:20:14.646 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-12T03:20:14.646 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-12T03:20:14.646 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-12T03:20:14.646 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-12T03:20:14.646 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.646 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.646 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.646 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.646 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.646 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-12T03:20:14.646 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-12T03:20:14.646 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.646 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.647 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.647 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-12T03:20:14.647 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.647 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.647 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.647 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.647 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.647 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.647 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.647 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.647 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-12T03:20:14.647 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-12T03:20:14.647 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.647 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.648 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.648 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-12T03:20:14.648 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.648 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.648 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-12T03:20:14.648 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-12T03:20:14.648 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-12T03:20:14.648 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-12T03:20:14.648 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-12T03:20:14.648 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-12T03:20:14.648 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-12T03:20:14.648 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-12T03:20:14.648 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.648 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.649 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.649 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.649 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.649 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-12T03:20:14.649 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-12T03:20:14.649 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.649 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.649 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.649 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-12T03:20:14.649 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.649 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.649 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.649 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.649 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.650 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.650 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.650 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:20:14.650 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:20:14.650 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.650 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:20:14.650 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "telemetry", 2024-09-12T03:20:14.650 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-12T03:20:14.650 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-12T03:20:14.650 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-12T03:20:14.650 INFO:teuthology.orchestra.run.smithi027.stdout: "channel_basic": { 2024-09-12T03:20:14.650 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "channel_basic", 2024-09-12T03:20:14.650 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.650 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.651 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.651 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-12T03:20:14.651 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.651 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.651 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.651 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-12T03:20:14.651 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.651 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.651 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.651 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.651 INFO:teuthology.orchestra.run.smithi027.stdout: "channel_crash": { 2024-09-12T03:20:14.651 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "channel_crash", 2024-09-12T03:20:14.651 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.651 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.652 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.652 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-12T03:20:14.652 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.652 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.652 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.652 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-12T03:20:14.652 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.652 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.652 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.652 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.652 INFO:teuthology.orchestra.run.smithi027.stdout: "channel_device": { 2024-09-12T03:20:14.652 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "channel_device", 2024-09-12T03:20:14.652 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.653 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.653 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.653 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-12T03:20:14.653 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.653 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.653 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.653 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-12T03:20:14.653 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.656 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.657 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.657 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.657 INFO:teuthology.orchestra.run.smithi027.stdout: "channel_ident": { 2024-09-12T03:20:14.657 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "channel_ident", 2024-09-12T03:20:14.657 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.657 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.657 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.657 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-12T03:20:14.657 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.657 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.657 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.657 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-12T03:20:14.657 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.658 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.658 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.658 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.658 INFO:teuthology.orchestra.run.smithi027.stdout: "channel_perf": { 2024-09-12T03:20:14.658 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "channel_perf", 2024-09-12T03:20:14.658 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.658 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.658 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.658 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-12T03:20:14.658 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.658 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.658 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.658 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-12T03:20:14.659 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.659 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.659 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.659 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.659 INFO:teuthology.orchestra.run.smithi027.stdout: "contact": { 2024-09-12T03:20:14.659 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "contact", 2024-09-12T03:20:14.659 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.659 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.659 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.659 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-12T03:20:14.659 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.659 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.659 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.660 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.660 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.660 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.660 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.660 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.660 INFO:teuthology.orchestra.run.smithi027.stdout: "description": { 2024-09-12T03:20:14.660 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "description", 2024-09-12T03:20:14.660 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.660 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.660 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.660 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-12T03:20:14.660 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.660 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.661 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.661 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.661 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.661 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.661 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.661 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.661 INFO:teuthology.orchestra.run.smithi027.stdout: "device_url": { 2024-09-12T03:20:14.661 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "device_url", 2024-09-12T03:20:14.661 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.661 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.661 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.661 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-12T03:20:14.661 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.661 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.662 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.662 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.662 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.662 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.662 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.662 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.662 INFO:teuthology.orchestra.run.smithi027.stdout: "enabled": { 2024-09-12T03:20:14.662 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "enabled", 2024-09-12T03:20:14.662 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.662 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.662 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.662 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-12T03:20:14.662 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.662 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.663 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.663 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.663 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.663 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.663 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.663 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.663 INFO:teuthology.orchestra.run.smithi027.stdout: "interval": { 2024-09-12T03:20:14.663 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "interval", 2024-09-12T03:20:14.663 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-12T03:20:14.663 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.663 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.663 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "24", 2024-09-12T03:20:14.663 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "8", 2024-09-12T03:20:14.663 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.664 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.664 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.664 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.664 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.664 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.664 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.664 INFO:teuthology.orchestra.run.smithi027.stdout: "last_opt_revision": { 2024-09-12T03:20:14.664 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "last_opt_revision", 2024-09-12T03:20:14.664 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-12T03:20:14.664 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.664 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.664 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "1", 2024-09-12T03:20:14.664 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.665 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.665 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.665 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.665 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.665 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.665 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.665 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.665 INFO:teuthology.orchestra.run.smithi027.stdout: "leaderboard": { 2024-09-12T03:20:14.665 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "leaderboard", 2024-09-12T03:20:14.665 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.665 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.665 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.665 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-12T03:20:14.666 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.666 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.666 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.666 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.666 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.666 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.666 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.666 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.666 INFO:teuthology.orchestra.run.smithi027.stdout: "leaderboard_description": { 2024-09-12T03:20:14.666 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "leaderboard_description", 2024-09-12T03:20:14.666 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.666 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.666 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.667 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-12T03:20:14.667 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.667 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.667 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.667 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.667 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.667 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.667 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.667 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.667 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-12T03:20:14.667 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-12T03:20:14.667 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.667 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.667 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.668 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-12T03:20:14.668 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.668 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.668 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-12T03:20:14.668 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-12T03:20:14.668 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-12T03:20:14.668 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-12T03:20:14.668 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-12T03:20:14.668 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-12T03:20:14.668 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-12T03:20:14.668 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-12T03:20:14.668 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.668 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.669 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.669 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.669 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.669 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-12T03:20:14.669 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-12T03:20:14.669 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.669 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.669 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.669 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-12T03:20:14.669 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.669 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.669 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.669 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.669 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.669 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.670 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.670 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.670 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-12T03:20:14.670 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-12T03:20:14.670 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.670 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.670 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.670 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-12T03:20:14.670 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.670 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.670 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-12T03:20:14.670 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-12T03:20:14.670 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-12T03:20:14.671 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-12T03:20:14.671 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-12T03:20:14.671 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-12T03:20:14.671 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-12T03:20:14.671 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-12T03:20:14.671 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.671 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.671 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.671 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.671 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.671 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-12T03:20:14.671 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-12T03:20:14.671 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.671 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.672 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.672 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-12T03:20:14.672 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.672 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.672 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.672 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.672 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.672 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.672 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.672 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.672 INFO:teuthology.orchestra.run.smithi027.stdout: "organization": { 2024-09-12T03:20:14.672 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "organization", 2024-09-12T03:20:14.672 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.672 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.673 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.673 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-12T03:20:14.673 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.673 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.673 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.673 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.673 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.673 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.673 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.673 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.673 INFO:teuthology.orchestra.run.smithi027.stdout: "proxy": { 2024-09-12T03:20:14.673 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "proxy", 2024-09-12T03:20:14.673 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.673 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.674 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.674 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-12T03:20:14.674 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.674 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.674 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.674 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.674 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.674 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.674 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.674 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.674 INFO:teuthology.orchestra.run.smithi027.stdout: "url": { 2024-09-12T03:20:14.674 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "url", 2024-09-12T03:20:14.674 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.674 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.675 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.675 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-12T03:20:14.675 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.675 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.675 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.675 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.675 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.675 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.675 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.675 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:20:14.675 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:20:14.675 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.675 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:20:14.675 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "test_orchestrator", 2024-09-12T03:20:14.676 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-12T03:20:14.676 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-12T03:20:14.676 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-12T03:20:14.676 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-12T03:20:14.676 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-12T03:20:14.676 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.676 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.676 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.676 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-12T03:20:14.676 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.676 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.676 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-12T03:20:14.676 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-12T03:20:14.676 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-12T03:20:14.677 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-12T03:20:14.677 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-12T03:20:14.677 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-12T03:20:14.677 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-12T03:20:14.677 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-12T03:20:14.677 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.677 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.677 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.677 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.677 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.677 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-12T03:20:14.677 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-12T03:20:14.677 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.677 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.678 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.678 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-12T03:20:14.678 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.678 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.678 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.678 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.678 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.678 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.678 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.678 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.678 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-12T03:20:14.678 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-12T03:20:14.678 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.679 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.679 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.679 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-12T03:20:14.679 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.679 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.679 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-12T03:20:14.679 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-12T03:20:14.679 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-12T03:20:14.679 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-12T03:20:14.679 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-12T03:20:14.679 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-12T03:20:14.679 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-12T03:20:14.679 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-12T03:20:14.679 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.680 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.680 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.680 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.680 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.680 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-12T03:20:14.680 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-12T03:20:14.680 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.680 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.680 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.680 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-12T03:20:14.680 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.680 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.680 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.680 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.681 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.681 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.681 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.681 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:20:14.681 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:20:14.681 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.681 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:20:14.681 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "volumes", 2024-09-12T03:20:14.681 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-12T03:20:14.681 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-12T03:20:14.681 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-12T03:20:14.681 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-12T03:20:14.681 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-12T03:20:14.681 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.682 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.682 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.682 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-12T03:20:14.682 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.682 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.682 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-12T03:20:14.682 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-12T03:20:14.682 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-12T03:20:14.682 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-12T03:20:14.682 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-12T03:20:14.682 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-12T03:20:14.682 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-12T03:20:14.682 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-12T03:20:14.682 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.683 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.683 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.683 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.683 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.683 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-12T03:20:14.683 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-12T03:20:14.683 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.683 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.683 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.683 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-12T03:20:14.683 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.683 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.683 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.683 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.684 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.684 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.684 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.684 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.684 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-12T03:20:14.684 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-12T03:20:14.684 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.684 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.684 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.684 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-12T03:20:14.684 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.684 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.684 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-12T03:20:14.684 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-12T03:20:14.685 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-12T03:20:14.685 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-12T03:20:14.685 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-12T03:20:14.685 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-12T03:20:14.685 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-12T03:20:14.685 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-12T03:20:14.685 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.685 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.685 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.685 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.685 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.685 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-12T03:20:14.685 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-12T03:20:14.686 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.686 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.686 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.686 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-12T03:20:14.686 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.686 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.686 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.686 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.686 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.686 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.686 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.686 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.686 INFO:teuthology.orchestra.run.smithi027.stdout: "max_concurrent_clones": { 2024-09-12T03:20:14.686 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "max_concurrent_clones", 2024-09-12T03:20:14.687 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-12T03:20:14.687 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.687 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.687 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "4", 2024-09-12T03:20:14.687 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.687 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.687 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.687 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-12T03:20:14.687 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.687 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.687 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.687 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.687 INFO:teuthology.orchestra.run.smithi027.stdout: "periodic_async_work": { 2024-09-12T03:20:14.687 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "periodic_async_work", 2024-09-12T03:20:14.688 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.688 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.688 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.688 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-12T03:20:14.688 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.688 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.688 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.688 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Periodically check for async work", 2024-09-12T03:20:14.688 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.688 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.688 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.688 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.688 INFO:teuthology.orchestra.run.smithi027.stdout: "snapshot_clone_delay": { 2024-09-12T03:20:14.688 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "snapshot_clone_delay", 2024-09-12T03:20:14.689 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-12T03:20:14.689 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.689 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.689 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-12T03:20:14.689 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.689 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.689 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.689 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-12T03:20:14.689 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.689 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.689 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.689 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.689 INFO:teuthology.orchestra.run.smithi027.stdout: "snapshot_clone_no_wait": { 2024-09-12T03:20:14.689 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "snapshot_clone_no_wait", 2024-09-12T03:20:14.690 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.690 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.690 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.690 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-12T03:20:14.690 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.690 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.690 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.690 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-12T03:20:14.690 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.690 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.690 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.690 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:20:14.690 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:20:14.690 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.691 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:20:14.691 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "zabbix", 2024-09-12T03:20:14.691 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-12T03:20:14.691 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-12T03:20:14.691 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-12T03:20:14.691 INFO:teuthology.orchestra.run.smithi027.stdout: "discovery_interval": { 2024-09-12T03:20:14.691 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "discovery_interval", 2024-09-12T03:20:14.691 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "uint", 2024-09-12T03:20:14.691 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.691 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.691 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "100", 2024-09-12T03:20:14.691 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.691 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.691 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.692 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.692 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.692 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.692 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.692 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.692 INFO:teuthology.orchestra.run.smithi027.stdout: "identifier": { 2024-09-12T03:20:14.692 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "identifier", 2024-09-12T03:20:14.692 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.692 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.692 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.692 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-12T03:20:14.692 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.692 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.693 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.693 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.693 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.693 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.693 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.693 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.693 INFO:teuthology.orchestra.run.smithi027.stdout: "interval": { 2024-09-12T03:20:14.693 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "interval", 2024-09-12T03:20:14.693 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-12T03:20:14.693 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.693 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.693 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "60", 2024-09-12T03:20:14.693 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.693 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.694 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.694 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.694 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.694 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.694 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.694 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.694 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-12T03:20:14.694 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-12T03:20:14.694 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.694 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.694 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.694 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-12T03:20:14.694 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.694 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.695 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-12T03:20:14.695 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-12T03:20:14.695 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-12T03:20:14.695 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-12T03:20:14.695 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-12T03:20:14.695 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-12T03:20:14.695 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-12T03:20:14.695 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-12T03:20:14.695 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.695 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.695 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.695 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.695 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.696 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-12T03:20:14.696 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-12T03:20:14.696 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.696 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.696 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.696 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-12T03:20:14.696 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.696 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.696 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.696 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.696 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.696 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.696 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.696 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.697 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-12T03:20:14.697 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-12T03:20:14.697 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.697 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.697 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.697 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-12T03:20:14.697 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.697 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.697 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-12T03:20:14.697 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-12T03:20:14.697 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-12T03:20:14.697 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-12T03:20:14.697 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-12T03:20:14.698 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-12T03:20:14.698 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-12T03:20:14.698 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-12T03:20:14.698 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.698 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.698 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.698 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.698 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.698 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-12T03:20:14.698 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-12T03:20:14.698 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-12T03:20:14.698 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.698 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-12T03:20:14.698 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-12T03:20:14.699 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.699 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.699 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.699 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.699 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.699 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.699 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.699 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.699 INFO:teuthology.orchestra.run.smithi027.stdout: "zabbix_host": { 2024-09-12T03:20:14.699 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "zabbix_host", 2024-09-12T03:20:14.699 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.699 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.699 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.699 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-12T03:20:14.700 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.700 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.700 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.700 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.700 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.700 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.700 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.700 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.700 INFO:teuthology.orchestra.run.smithi027.stdout: "zabbix_port": { 2024-09-12T03:20:14.700 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "zabbix_port", 2024-09-12T03:20:14.700 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-12T03:20:14.700 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.700 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.700 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10051", 2024-09-12T03:20:14.701 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.701 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.701 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.701 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.701 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.701 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.701 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.701 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.701 INFO:teuthology.orchestra.run.smithi027.stdout: "zabbix_sender": { 2024-09-12T03:20:14.701 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "zabbix_sender", 2024-09-12T03:20:14.701 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-12T03:20:14.701 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-12T03:20:14.701 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-12T03:20:14.702 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-12T03:20:14.702 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-12T03:20:14.702 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-12T03:20:14.702 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-12T03:20:14.702 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-12T03:20:14.702 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-12T03:20:14.702 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-12T03:20:14.702 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-12T03:20:14.702 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:20:14.702 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:20:14.702 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:20:14.702 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-12T03:20:14.702 INFO:teuthology.orchestra.run.smithi027.stdout: "services": {}, 2024-09-12T03:20:14.702 INFO:teuthology.orchestra.run.smithi027.stdout: "always_on_modules": { 2024-09-12T03:20:14.703 INFO:teuthology.orchestra.run.smithi027.stdout: "octopus": [ 2024-09-12T03:20:14.703 INFO:teuthology.orchestra.run.smithi027.stdout: "balancer", 2024-09-12T03:20:14.703 INFO:teuthology.orchestra.run.smithi027.stdout: "crash", 2024-09-12T03:20:14.703 INFO:teuthology.orchestra.run.smithi027.stdout: "devicehealth", 2024-09-12T03:20:14.703 INFO:teuthology.orchestra.run.smithi027.stdout: "orchestrator", 2024-09-12T03:20:14.703 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_autoscaler", 2024-09-12T03:20:14.703 INFO:teuthology.orchestra.run.smithi027.stdout: "progress", 2024-09-12T03:20:14.703 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_support", 2024-09-12T03:20:14.703 INFO:teuthology.orchestra.run.smithi027.stdout: "status", 2024-09-12T03:20:14.703 INFO:teuthology.orchestra.run.smithi027.stdout: "telemetry", 2024-09-12T03:20:14.703 INFO:teuthology.orchestra.run.smithi027.stdout: "volumes" 2024-09-12T03:20:14.703 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-12T03:20:14.703 INFO:teuthology.orchestra.run.smithi027.stdout: "pacific": [ 2024-09-12T03:20:14.703 INFO:teuthology.orchestra.run.smithi027.stdout: "balancer", 2024-09-12T03:20:14.704 INFO:teuthology.orchestra.run.smithi027.stdout: "crash", 2024-09-12T03:20:14.704 INFO:teuthology.orchestra.run.smithi027.stdout: "devicehealth", 2024-09-12T03:20:14.704 INFO:teuthology.orchestra.run.smithi027.stdout: "orchestrator", 2024-09-12T03:20:14.704 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_autoscaler", 2024-09-12T03:20:14.704 INFO:teuthology.orchestra.run.smithi027.stdout: "progress", 2024-09-12T03:20:14.704 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_support", 2024-09-12T03:20:14.704 INFO:teuthology.orchestra.run.smithi027.stdout: "status", 2024-09-12T03:20:14.704 INFO:teuthology.orchestra.run.smithi027.stdout: "telemetry", 2024-09-12T03:20:14.704 INFO:teuthology.orchestra.run.smithi027.stdout: "volumes" 2024-09-12T03:20:14.704 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-12T03:20:14.704 INFO:teuthology.orchestra.run.smithi027.stdout: "quincy": [ 2024-09-12T03:20:14.704 INFO:teuthology.orchestra.run.smithi027.stdout: "balancer", 2024-09-12T03:20:14.704 INFO:teuthology.orchestra.run.smithi027.stdout: "crash", 2024-09-12T03:20:14.704 INFO:teuthology.orchestra.run.smithi027.stdout: "devicehealth", 2024-09-12T03:20:14.705 INFO:teuthology.orchestra.run.smithi027.stdout: "orchestrator", 2024-09-12T03:20:14.705 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_autoscaler", 2024-09-12T03:20:14.705 INFO:teuthology.orchestra.run.smithi027.stdout: "progress", 2024-09-12T03:20:14.705 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_support", 2024-09-12T03:20:14.705 INFO:teuthology.orchestra.run.smithi027.stdout: "status", 2024-09-12T03:20:14.705 INFO:teuthology.orchestra.run.smithi027.stdout: "telemetry", 2024-09-12T03:20:14.705 INFO:teuthology.orchestra.run.smithi027.stdout: "volumes" 2024-09-12T03:20:14.705 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-12T03:20:14.705 INFO:teuthology.orchestra.run.smithi027.stdout: "reef": [ 2024-09-12T03:20:14.705 INFO:teuthology.orchestra.run.smithi027.stdout: "balancer", 2024-09-12T03:20:14.705 INFO:teuthology.orchestra.run.smithi027.stdout: "crash", 2024-09-12T03:20:14.705 INFO:teuthology.orchestra.run.smithi027.stdout: "devicehealth", 2024-09-12T03:20:14.705 INFO:teuthology.orchestra.run.smithi027.stdout: "orchestrator", 2024-09-12T03:20:14.705 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_autoscaler", 2024-09-12T03:20:14.706 INFO:teuthology.orchestra.run.smithi027.stdout: "progress", 2024-09-12T03:20:14.706 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_support", 2024-09-12T03:20:14.706 INFO:teuthology.orchestra.run.smithi027.stdout: "status", 2024-09-12T03:20:14.706 INFO:teuthology.orchestra.run.smithi027.stdout: "telemetry", 2024-09-12T03:20:14.706 INFO:teuthology.orchestra.run.smithi027.stdout: "volumes" 2024-09-12T03:20:14.706 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-12T03:20:14.706 INFO:teuthology.orchestra.run.smithi027.stdout: "squid": [ 2024-09-12T03:20:14.706 INFO:teuthology.orchestra.run.smithi027.stdout: "balancer", 2024-09-12T03:20:14.706 INFO:teuthology.orchestra.run.smithi027.stdout: "crash", 2024-09-12T03:20:14.706 INFO:teuthology.orchestra.run.smithi027.stdout: "devicehealth", 2024-09-12T03:20:14.706 INFO:teuthology.orchestra.run.smithi027.stdout: "orchestrator", 2024-09-12T03:20:14.706 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_autoscaler", 2024-09-12T03:20:14.706 INFO:teuthology.orchestra.run.smithi027.stdout: "progress", 2024-09-12T03:20:14.706 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_support", 2024-09-12T03:20:14.707 INFO:teuthology.orchestra.run.smithi027.stdout: "status", 2024-09-12T03:20:14.707 INFO:teuthology.orchestra.run.smithi027.stdout: "telemetry", 2024-09-12T03:20:14.707 INFO:teuthology.orchestra.run.smithi027.stdout: "volumes" 2024-09-12T03:20:14.707 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-12T03:20:14.707 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.707 INFO:teuthology.orchestra.run.smithi027.stdout: "last_failure_osd_epoch": 0, 2024-09-12T03:20:14.707 INFO:teuthology.orchestra.run.smithi027.stdout: "active_clients": [ 2024-09-12T03:20:14.707 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:20:14.707 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "devicehealth", 2024-09-12T03:20:14.707 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-12T03:20:14.707 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:20:14.707 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-12T03:20:14.707 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.45:0", 2024-09-12T03:20:14.707 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3383141969 2024-09-12T03:20:14.708 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:20:14.708 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-12T03:20:14.708 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.708 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:20:14.708 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "libcephsqlite", 2024-09-12T03:20:14.708 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-12T03:20:14.708 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:20:14.708 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-12T03:20:14.708 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.45:0", 2024-09-12T03:20:14.708 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 1945166272 2024-09-12T03:20:14.708 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:20:14.708 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-12T03:20:14.708 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.708 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:20:14.709 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rbd_support", 2024-09-12T03:20:14.709 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-12T03:20:14.709 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:20:14.709 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-12T03:20:14.709 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.45:0", 2024-09-12T03:20:14.709 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 687436879 2024-09-12T03:20:14.709 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:20:14.709 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-12T03:20:14.709 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.709 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:20:14.709 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "volumes", 2024-09-12T03:20:14.709 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-12T03:20:14.709 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:20:14.709 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-12T03:20:14.710 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.45:0", 2024-09-12T03:20:14.710 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 4216551134 2024-09-12T03:20:14.710 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:20:14.710 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-12T03:20:14.710 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.710 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:20:14.710 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "volumes", 2024-09-12T03:20:14.710 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-12T03:20:14.710 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:20:14.710 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-12T03:20:14.710 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.45:0", 2024-09-12T03:20:14.710 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 471646388 2024-09-12T03:20:14.710 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:20:14.710 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-12T03:20:14.711 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.711 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:20:14.711 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "volumes", 2024-09-12T03:20:14.711 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-12T03:20:14.711 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:20:14.711 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-12T03:20:14.711 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.45:0", 2024-09-12T03:20:14.711 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3677899312 2024-09-12T03:20:14.711 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:20:14.711 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-12T03:20:14.711 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.711 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:20:14.711 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "volumes", 2024-09-12T03:20:14.711 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-12T03:20:14.712 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:20:14.712 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-12T03:20:14.712 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.45:0", 2024-09-12T03:20:14.712 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 4018097172 2024-09-12T03:20:14.712 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:20:14.712 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-12T03:20:14.712 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.712 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:20:14.712 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "volumes", 2024-09-12T03:20:14.712 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-12T03:20:14.712 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:20:14.712 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-12T03:20:14.712 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.45:0", 2024-09-12T03:20:14.712 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3895125587 2024-09-12T03:20:14.713 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:20:14.713 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-12T03:20:14.713 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.713 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:20:14.713 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "volumes", 2024-09-12T03:20:14.713 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-12T03:20:14.713 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:20:14.713 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-12T03:20:14.713 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.45:0", 2024-09-12T03:20:14.713 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3205085313 2024-09-12T03:20:14.713 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:20:14.713 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-12T03:20:14.713 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:14.713 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:20:14.714 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "volumes", 2024-09-12T03:20:14.714 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-12T03:20:14.714 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:20:14.714 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-12T03:20:14.714 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.45:0", 2024-09-12T03:20:14.714 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 2632182829 2024-09-12T03:20:14.714 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:20:14.714 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-12T03:20:14.714 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:20:14.714 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-12T03:20:14.714 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-12T03:20:14.715 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:13.794+0000 7fa178d59640 1 -- 172.21.15.27:0/4259047272 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fa15406c190 msgr2=0x7fa15406e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:14.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:13.794+0000 7fa178d59640 1 --2- 172.21.15.27:0/4259047272 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fa15406c190 0x7fa15406e650 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fa15c034000 tx=0x7fa15c02e000 comp rx=0 tx=0).stop 2024-09-12T03:20:14.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:13.794+0000 7fa178d59640 1 -- 172.21.15.27:0/4259047272 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa1740d1550 msgr2=0x7fa174158bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:14.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:13.794+0000 7fa178d59640 1 --2- 172.21.15.27:0/4259047272 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa1740d1550 0x7fa174158bf0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fa16400d7d0 tx=0x7fa16400dc90 comp rx=0 tx=0).stop 2024-09-12T03:20:14.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:13.794+0000 7fa178d59640 1 -- 172.21.15.27:0/4259047272 shutdown_connections 2024-09-12T03:20:14.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:13.794+0000 7fa178d59640 1 --2- 172.21.15.27:0/4259047272 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fa15406c190 0x7fa15406e650 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:14.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:13.794+0000 7fa178d59640 1 --2- 172.21.15.27:0/4259047272 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa17415eb90 0x7fa174156bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:14.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:13.794+0000 7fa178d59640 1 --2- 172.21.15.27:0/4259047272 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa1740d5200 0x7fa174146130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:14.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:13.794+0000 7fa178d59640 1 --2- 172.21.15.27:0/4259047272 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa1740d1550 0x7fa174158bf0 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:14.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:13.794+0000 7fa178d59640 1 -- 172.21.15.27:0/4259047272 >> 172.21.15.27:0/4259047272 conn(0x7fa17406dce0 msgr2=0x7fa17415f180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:14.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:13.794+0000 7fa178d59640 1 -- 172.21.15.27:0/4259047272 shutdown_connections 2024-09-12T03:20:14.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:13.794+0000 7fa178d59640 1 -- 172.21.15.27:0/4259047272 wait complete. 2024-09-12T03:20:14.721 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr fail x 2024-09-12T03:20:14.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:14.836+0000 7fdfd3caf640 1 Processor -- start 2024-09-12T03:20:14.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:14.836+0000 7fdfd3caf640 1 -- start start 2024-09-12T03:20:14.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:14.838+0000 7fdfd3caf640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fdfcc072d70 0x7fdfcc0b3560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:14.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:14.838+0000 7fdfd3caf640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fdfcc06e9b0 0x7fdfcc06ee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:14.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:14.838+0000 7fdfd3caf640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdfcc0ce410 con 0x7fdfcc15b2b0 2024-09-12T03:20:14.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:14.838+0000 7fdfd3caf640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7fdfcc0720a0 con 0x7fdfcc072d70 2024-09-12T03:20:14.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:14.838+0000 7fdfd3caf640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fdfcc0732c0 con 0x7fdfcc06e9b0 2024-09-12T03:20:14.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:14.838+0000 7fdfd124a640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fdfcc072d70 0x7fdfcc0b3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:14.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:14.838+0000 7fdfd1a4b640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fdfcc15b2b0 0x7fdfcc072430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:50914/0 (socket says 172.21.15.27:50914) 2024-09-12T03:20:14.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:14.838+0000 7fdfd1a4b640 1 -- 172.21.15.27:0/91464576 learned_addr learned my addr 172.21.15.27:0/91464576 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:20:14.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:14.838+0000 7fdfd224c640 1 --2- 172.21.15.27:0/91464576 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fdfcc06e9b0 0x7fdfcc06ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:14.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:14.838+0000 7fdfd124a640 1 -- 172.21.15.27:0/91464576 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fdfcc06e9b0 msgr2=0x7fdfcc06ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:14.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:14.838+0000 7fdfd124a640 1 --2- 172.21.15.27:0/91464576 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fdfcc06e9b0 0x7fdfcc06ee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:14.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:14.838+0000 7fdfd124a640 1 -- 172.21.15.27:0/91464576 >> v1:172.21.15.27:6789/0 conn(0x7fdfcc15b2b0 legacy=0x7fdfcc072430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:14.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:14.838+0000 7fdfd124a640 1 -- 172.21.15.27:0/91464576 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfcc071ae0 con 0x7fdfcc072d70 2024-09-12T03:20:14.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:14.838+0000 7fdfd124a640 1 --2- 172.21.15.27:0/91464576 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdfcc072d70 0x7fdfcc0b3560 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fdfbc0097a0 tx=0x7fdfbc02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57978093d8994de server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:14.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:14.838+0000 7fdfd0a49640 1 -- 172.21.15.27:0/91464576 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfbc037070 con 0x7fdfcc072d70 2024-09-12T03:20:14.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:14.838+0000 7fdfd0a49640 1 -- 172.21.15.27:0/91464576 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdfbc032440 con 0x7fdfcc072d70 2024-09-12T03:20:14.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:14.838+0000 7fdfd0a49640 1 -- 172.21.15.27:0/91464576 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfbc032810 con 0x7fdfcc072d70 2024-09-12T03:20:14.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:14.839+0000 7fdfd3caf640 1 -- 172.21.15.27:0/91464576 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdfcc072d70 msgr2=0x7fdfcc0b3560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:14.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:14.839+0000 7fdfd3caf640 1 --2- 172.21.15.27:0/91464576 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdfcc072d70 0x7fdfcc0b3560 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fdfbc0097a0 tx=0x7fdfbc02a180 comp rx=0 tx=0).stop 2024-09-12T03:20:14.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:14.839+0000 7fdfd3caf640 1 -- 172.21.15.27:0/91464576 shutdown_connections 2024-09-12T03:20:14.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:14.839+0000 7fdfd3caf640 1 --2- 172.21.15.27:0/91464576 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fdfcc06e9b0 0x7fdfcc06ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:14.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:14.839+0000 7fdfd3caf640 1 --2- 172.21.15.27:0/91464576 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdfcc072d70 0x7fdfcc0b3560 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:14.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:14.839+0000 7fdfd3caf640 1 -- 172.21.15.27:0/91464576 >> 172.21.15.27:0/91464576 conn(0x7fdfcc062c60 msgr2=0x7fdfcc06bb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:14.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:14.839+0000 7fdfd3caf640 1 -- 172.21.15.27:0/91464576 shutdown_connections 2024-09-12T03:20:14.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:14.839+0000 7fdfd3caf640 1 -- 172.21.15.27:0/91464576 wait complete. 2024-09-12T03:20:14.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:14.840+0000 7fdfd3caf640 1 Processor -- start 2024-09-12T03:20:14.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:14.840+0000 7fdfd3caf640 1 -- start start 2024-09-12T03:20:14.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:14.841+0000 7fdfd3caf640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdfcc06e9b0 0x7fdfcc158060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:14.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:14.841+0000 7fdfd3caf640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdfcc072d70 0x7fdfcc1e4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:14.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:14.841+0000 7fdfd3caf640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fdfcc15b2b0 0x7fdfcc15de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:14.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:14.841+0000 7fdfd3caf640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fdfcc15b690 con 0x7fdfcc072d70 2024-09-12T03:20:14.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:14.841+0000 7fdfd3caf640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fdfcc0ce690 con 0x7fdfcc06e9b0 2024-09-12T03:20:14.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:14.841+0000 7fdfd3caf640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fdfcc15b810 con 0x7fdfcc15b2b0 2024-09-12T03:20:14.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:14.841+0000 7fdfd124a640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdfcc072d70 0x7fdfcc1e4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:14.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:14.841+0000 7fdfd124a640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdfcc072d70 0x7fdfcc1e4d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57914/0 (socket says 172.21.15.27:57914) 2024-09-12T03:20:14.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:14.841+0000 7fdfd124a640 1 -- 172.21.15.27:0/1182450259 learned_addr learned my addr 172.21.15.27:0/1182450259 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:20:14.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:14.841+0000 7fdfd1a4b640 1 --2- 172.21.15.27:0/1182450259 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdfcc06e9b0 0x7fdfcc158060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:14.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:14.841+0000 7fdfd124a640 1 -- 172.21.15.27:0/1182450259 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fdfcc15b2b0 msgr2=0x7fdfcc15de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:14.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:14.841+0000 7fdfd224c640 1 --2- 172.21.15.27:0/1182450259 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fdfcc15b2b0 0x7fdfcc15de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:14.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:14.841+0000 7fdfd124a640 1 --2- 172.21.15.27:0/1182450259 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fdfcc15b2b0 0x7fdfcc15de00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:14.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:14.841+0000 7fdfd124a640 1 -- 172.21.15.27:0/1182450259 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdfcc06e9b0 msgr2=0x7fdfcc158060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:14.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:14.841+0000 7fdfd124a640 1 --2- 172.21.15.27:0/1182450259 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdfcc06e9b0 0x7fdfcc158060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:14.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:14.841+0000 7fdfd124a640 1 -- 172.21.15.27:0/1182450259 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfcc156450 con 0x7fdfcc072d70 2024-09-12T03:20:14.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:14.841+0000 7fdfd1a4b640 1 --2- 172.21.15.27:0/1182450259 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdfcc06e9b0 0x7fdfcc158060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:20:14.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:14.841+0000 7fdfd224c640 1 --2- 172.21.15.27:0/1182450259 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fdfcc15b2b0 0x7fdfcc15de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:20:14.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:14.842+0000 7fdfd124a640 1 --2- 172.21.15.27:0/1182450259 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdfcc072d70 0x7fdfcc1e4d90 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fdfbc035040 tx=0x7fdfbc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:14.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:14.842+0000 7fdfc27fc640 1 -- 172.21.15.27:0/1182450259 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfbc037040 con 0x7fdfcc072d70 2024-09-12T03:20:14.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:14.842+0000 7fdfc27fc640 1 -- 172.21.15.27:0/1182450259 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdfbc042070 con 0x7fdfcc072d70 2024-09-12T03:20:14.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:14.842+0000 7fdfd3caf640 1 -- 172.21.15.27:0/1182450259 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfcc156730 con 0x7fdfcc072d70 2024-09-12T03:20:14.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:14.842+0000 7fdfc27fc640 1 -- 172.21.15.27:0/1182450259 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfbc030bb0 con 0x7fdfcc072d70 2024-09-12T03:20:14.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:14.842+0000 7fdfd3caf640 1 -- 172.21.15.27:0/1182450259 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfcc156cf0 con 0x7fdfcc072d70 2024-09-12T03:20:14.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:14.844+0000 7fdfd3caf640 1 -- 172.21.15.27:0/1182450259 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdfcc0b3560 con 0x7fdfcc072d70 2024-09-12T03:20:14.847 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:14.844+0000 7fdfc27fc640 1 -- 172.21.15.27:0/1182450259 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 12) ==== 90855+0+0 (secure 0 0 0) 0x7fdfbc047460 con 0x7fdfcc072d70 2024-09-12T03:20:14.847 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:14.845+0000 7fdfc27fc640 1 --2- 172.21.15.27:0/1182450259 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fdfb006c220 0x7fdfb006e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:14.847 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:14.845+0000 7fdfc27fc640 1 -- 172.21.15.27:0/1182450259 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 5970+0+0 (secure 0 0 0) 0x7fdfbc0ac0f0 con 0x7fdfcc072d70 2024-09-12T03:20:14.847 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:14.845+0000 7fdfd1a4b640 1 --2- 172.21.15.27:0/1182450259 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fdfb006c220 0x7fdfb006e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:14.847 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:14.845+0000 7fdfd1a4b640 1 --2- 172.21.15.27:0/1182450259 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fdfb006c220 0x7fdfb006e6e0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fdfcc0cfc70 tx=0x7fdfc8009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:14.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:14.849+0000 7fdfc27fc640 1 -- 172.21.15.27:0/1182450259 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fdfbc07a550 con 0x7fdfcc072d70 2024-09-12T03:20:15.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:15.074+0000 7fdfd3caf640 1 -- 172.21.15.27:0/1182450259 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "mgr fail", "who": "x"} v 0) -- 0x7fdfcc0b0e70 con 0x7fdfcc072d70 2024-09-12T03:20:15.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:15.079+0000 7fdfc27fc640 1 -- 172.21.15.27:0/1182450259 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "mgr fail", "who": "x"}]=0 v13) ==== 68+0+0 (secure 0 0 0) 0x7fdfbc079e50 con 0x7fdfcc072d70 2024-09-12T03:20:15.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:15.082+0000 7fdfd3caf640 1 -- 172.21.15.27:0/1182450259 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fdfb006c220 msgr2=0x7fdfb006e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:15.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:15.082+0000 7fdfd3caf640 1 --2- 172.21.15.27:0/1182450259 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fdfb006c220 0x7fdfb006e6e0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fdfcc0cfc70 tx=0x7fdfc8009040 comp rx=0 tx=0).stop 2024-09-12T03:20:15.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:15.082+0000 7fdfd3caf640 1 -- 172.21.15.27:0/1182450259 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdfcc072d70 msgr2=0x7fdfcc1e4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:15.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:15.082+0000 7fdfd3caf640 1 --2- 172.21.15.27:0/1182450259 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdfcc072d70 0x7fdfcc1e4d90 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fdfbc035040 tx=0x7fdfbc0046c0 comp rx=0 tx=0).stop 2024-09-12T03:20:15.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:15.083+0000 7fdfd3caf640 1 -- 172.21.15.27:0/1182450259 shutdown_connections 2024-09-12T03:20:15.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:15.083+0000 7fdfd3caf640 1 --2- 172.21.15.27:0/1182450259 >> [v2:172.21.15.45:6832/3923821219,v1:172.21.15.45:6833/3923821219] conn(0x7fdfb006c220 0x7fdfb006e6e0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:15.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:15.083+0000 7fdfd3caf640 1 --2- 172.21.15.27:0/1182450259 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fdfcc15b2b0 0x7fdfcc15de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:15.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:15.083+0000 7fdfd3caf640 1 --2- 172.21.15.27:0/1182450259 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdfcc072d70 0x7fdfcc1e4d90 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:15.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:15.083+0000 7fdfd3caf640 1 --2- 172.21.15.27:0/1182450259 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdfcc06e9b0 0x7fdfcc158060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:15.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:15.083+0000 7fdfd3caf640 1 -- 172.21.15.27:0/1182450259 >> 172.21.15.27:0/1182450259 conn(0x7fdfcc062c60 msgr2=0x7fdfcc0b13b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:15.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:15.083+0000 7fdfd3caf640 1 -- 172.21.15.27:0/1182450259 shutdown_connections 2024-09-12T03:20:15.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:15.083+0000 7fdfd3caf640 1 -- 172.21.15.27:0/1182450259 wait complete. 2024-09-12T03:20:15.106 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-12T03:20:15.176 INFO:tasks.ceph.mgr.y.smithi027.stderr:2024-09-12T03:20:15.176+0000 7f32608e9640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:20:15.176 INFO:tasks.ceph.mgr.y.smithi027.stderr:2024-09-12T03:20:15.176+0000 7f32608e9640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:20:15.176 INFO:tasks.ceph.mgr.y.smithi027.stderr:2024-09-12T03:20:15.176+0000 7f32608e9640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:20:15.177 INFO:tasks.ceph.mgr.y.smithi027.stderr:2024-09-12T03:20:15.176+0000 7f32608e9640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:20:15.177 INFO:tasks.ceph.mgr.y.smithi027.stderr:2024-09-12T03:20:15.176+0000 7f32608e9640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:20:15.177 INFO:tasks.ceph.mgr.y.smithi027.stderr:2024-09-12T03:20:15.176+0000 7f32638ef640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:20:15.177 INFO:tasks.ceph.mgr.y.smithi027.stderr:2024-09-12T03:20:15.176+0000 7f32638ef640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:20:15.177 INFO:tasks.ceph.mgr.y.smithi027.stderr:2024-09-12T03:20:15.176+0000 7f32638ef640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:20:15.177 INFO:tasks.ceph.mgr.y.smithi027.stderr:2024-09-12T03:20:15.176+0000 7f32638ef640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:20:15.177 INFO:tasks.ceph.mgr.y.smithi027.stderr:2024-09-12T03:20:15.176+0000 7f32638ef640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:20:15.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:15.229+0000 7f6184d2c640 1 Processor -- start 2024-09-12T03:20:15.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:15.229+0000 7f6184d2c640 1 -- start start 2024-09-12T03:20:15.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:15.230+0000 7f6184d2c640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f61800b1ca0 0x7f618015f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:15.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:15.230+0000 7f6184d2c640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f618006f360 0x7f61800cef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:15.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:15.230+0000 7f6184d2c640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f61800cf490 0x7f61800d2780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:15.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:15.230+0000 7f6184d2c640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f61800d22f0 con 0x7f61800cf490 2024-09-12T03:20:15.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:15.230+0000 7f6184d2c640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f618015fd70 con 0x7f618006f360 2024-09-12T03:20:15.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:15.230+0000 7f6184d2c640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f618006cc40 con 0x7f61800b1ca0 2024-09-12T03:20:15.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:15.230+0000 7f617ed9d640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f61800cf490 0x7f61800d2780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:15.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:15.230+0000 7f617ed9d640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f61800cf490 0x7f61800d2780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58036/0 (socket says 172.21.15.27:58036) 2024-09-12T03:20:15.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:15.230+0000 7f617ed9d640 1 -- 172.21.15.27:0/2630529015 learned_addr learned my addr 172.21.15.27:0/2630529015 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:20:15.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:15.230+0000 7f617e59c640 1 --2- 172.21.15.27:0/2630529015 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f61800b1ca0 0x7f618015f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:15.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:15.230+0000 7f617dd9b640 1 --2- 172.21.15.27:0/2630529015 >> v2:172.21.15.45:3300/0 conn(0x7f618006f360 0x7f61800cef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:15.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:15.230+0000 7f617ed9d640 1 -- 172.21.15.27:0/2630529015 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f61800b1ca0 msgr2=0x7f618015f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:15.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:15.230+0000 7f617ed9d640 1 --2- 172.21.15.27:0/2630529015 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f61800b1ca0 0x7f618015f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:15.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:15.230+0000 7f617ed9d640 1 -- 172.21.15.27:0/2630529015 >> v2:172.21.15.45:3300/0 conn(0x7f618006f360 msgr2=0x7f61800cef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:15.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:15.230+0000 7f617ed9d640 1 --2- 172.21.15.27:0/2630529015 >> v2:172.21.15.45:3300/0 conn(0x7f618006f360 0x7f61800cef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:15.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:15.230+0000 7f617ed9d640 1 -- 172.21.15.27:0/2630529015 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61800d1ce0 con 0x7f61800cf490 2024-09-12T03:20:15.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:15.230+0000 7f617dd9b640 1 --2- 172.21.15.27:0/2630529015 >> v2:172.21.15.45:3300/0 conn(0x7f618006f360 0x7f61800cef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:20:15.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:15.231+0000 7f617ed9d640 1 --2- 172.21.15.27:0/2630529015 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f61800cf490 0x7f61800d2780 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f618015c9a0 tx=0x7f617802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d3dbab12a613bde server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:15.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:15.231+0000 7f617d59a640 1 -- 172.21.15.27:0/2630529015 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6178036070 con 0x7f61800cf490 2024-09-12T03:20:15.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:15.231+0000 7f617d59a640 1 -- 172.21.15.27:0/2630529015 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6178031600 con 0x7f61800cf490 2024-09-12T03:20:15.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:15.231+0000 7f617d59a640 1 -- 172.21.15.27:0/2630529015 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61780319d0 con 0x7f61800cf490 2024-09-12T03:20:15.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:15.232+0000 7f6184d2c640 1 -- 172.21.15.27:0/2630529015 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f61800cf490 msgr2=0x7f61800d2780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:15.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:15.232+0000 7f6184d2c640 1 --2- 172.21.15.27:0/2630529015 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f61800cf490 0x7f61800d2780 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f618015c9a0 tx=0x7f617802a6a0 comp rx=0 tx=0).stop 2024-09-12T03:20:15.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:15.232+0000 7f6184d2c640 1 -- 172.21.15.27:0/2630529015 shutdown_connections 2024-09-12T03:20:15.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:15.232+0000 7f6184d2c640 1 --2- 172.21.15.27:0/2630529015 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f61800cf490 0x7f61800d2780 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:15.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:15.232+0000 7f6184d2c640 1 --2- 172.21.15.27:0/2630529015 >> v2:172.21.15.45:3300/0 conn(0x7f618006f360 0x7f61800cef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:15.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:15.232+0000 7f6184d2c640 1 --2- 172.21.15.27:0/2630529015 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f61800b1ca0 0x7f618015f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:15.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:15.232+0000 7f6184d2c640 1 -- 172.21.15.27:0/2630529015 >> 172.21.15.27:0/2630529015 conn(0x7f6180073800 msgr2=0x7f618009ba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:15.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:15.232+0000 7f6184d2c640 1 -- 172.21.15.27:0/2630529015 shutdown_connections 2024-09-12T03:20:15.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:15.232+0000 7f6184d2c640 1 -- 172.21.15.27:0/2630529015 wait complete. 2024-09-12T03:20:15.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:15.233+0000 7f6184d2c640 1 Processor -- start 2024-09-12T03:20:15.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:15.233+0000 7f6184d2c640 1 -- start start 2024-09-12T03:20:15.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:15.234+0000 7f6184d2c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f618006f360 0x7f61801f4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:15.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:15.234+0000 7f6184d2c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f61800b1ca0 0x7f6180159fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:15.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:15.234+0000 7f6184d2c640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f61800cf490 0x7f61800b2380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:15.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:15.234+0000 7f6184d2c640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f618006d1c0 con 0x7f61800b1ca0 2024-09-12T03:20:15.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:15.234+0000 7f6184d2c640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f61800d1f80 con 0x7f618006f360 2024-09-12T03:20:15.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:15.234+0000 7f6184d2c640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f618006d340 con 0x7f61800cf490 2024-09-12T03:20:15.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:15.234+0000 7f617dd9b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f61800b1ca0 0x7f6180159fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:15.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:15.234+0000 7f617dd9b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f61800b1ca0 0x7f6180159fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58040/0 (socket says 172.21.15.27:58040) 2024-09-12T03:20:15.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:15.234+0000 7f617dd9b640 1 -- 172.21.15.27:0/1407981391 learned_addr learned my addr 172.21.15.27:0/1407981391 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:20:15.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:15.234+0000 7f617e59c640 1 --2- 172.21.15.27:0/1407981391 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f618006f360 0x7f61801f4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:15.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:15.234+0000 7f617ed9d640 1 --2- 172.21.15.27:0/1407981391 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f61800cf490 0x7f61800b2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:15.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:15.234+0000 7f617dd9b640 1 -- 172.21.15.27:0/1407981391 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f61800cf490 msgr2=0x7f61800b2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:15.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:15.234+0000 7f617dd9b640 1 --2- 172.21.15.27:0/1407981391 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f61800cf490 0x7f61800b2380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:15.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:15.234+0000 7f617dd9b640 1 -- 172.21.15.27:0/1407981391 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f618006f360 msgr2=0x7f61801f4ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:15.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:15.234+0000 7f617dd9b640 1 --2- 172.21.15.27:0/1407981391 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f618006f360 0x7f61801f4ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:15.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:15.234+0000 7f617dd9b640 1 -- 172.21.15.27:0/1407981391 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f618015a6e0 con 0x7f61800b1ca0 2024-09-12T03:20:15.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:15.235+0000 7f617dd9b640 1 --2- 172.21.15.27:0/1407981391 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f61800b1ca0 0x7f6180159fe0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f617400e810 tx=0x7f617400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:15.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:15.235+0000 7f616effd640 1 -- 172.21.15.27:0/1407981391 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6174017070 con 0x7f61800b1ca0 2024-09-12T03:20:15.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:15.235+0000 7f616effd640 1 -- 172.21.15.27:0/1407981391 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f617400c6d0 con 0x7f61800b1ca0 2024-09-12T03:20:15.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:15.235+0000 7f6184d2c640 1 -- 172.21.15.27:0/1407981391 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f618015a9c0 con 0x7f61800b1ca0 2024-09-12T03:20:15.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:15.235+0000 7f616effd640 1 -- 172.21.15.27:0/1407981391 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f617400f040 con 0x7f61800b1ca0 2024-09-12T03:20:15.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:15.235+0000 7f6184d2c640 1 -- 172.21.15.27:0/1407981391 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f618015af80 con 0x7f61800b1ca0 2024-09-12T03:20:15.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:15.236+0000 7f616effd640 1 -- 172.21.15.27:0/1407981391 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 13) ==== 45274+0+0 (secure 0 0 0) 0x7f617401a070 con 0x7f61800b1ca0 2024-09-12T03:20:15.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:15.236+0000 7f616effd640 1 -- 172.21.15.27:0/1407981391 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 6529+0+0 (secure 0 0 0) 0x7f617404b220 con 0x7f61800b1ca0 2024-09-12T03:20:15.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:15.236+0000 7f6184d2c640 1 -- 172.21.15.27:0/1407981391 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61800cfc40 con 0x7f61800b1ca0 2024-09-12T03:20:15.238 INFO:tasks.ceph.mgr.x.smithi045.stderr:2024-09-12T03:20:15.235+0000 7ff5de91e040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-12T03:20:15.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:15.241+0000 7f616effd640 1 -- 172.21.15.27:0/1407981391 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f617400cac0 con 0x7f61800b1ca0 2024-09-12T03:20:15.278 INFO:tasks.ceph.mgr.x.smithi045.stderr:2024-09-12T03:20:15.278+0000 7ff5de91e040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-12T03:20:15.416 INFO:tasks.ceph.mgr.x.smithi045.stderr:2024-09-12T03:20:15.416+0000 7ff5de91e040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-12T03:20:15.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:15.451+0000 7f6184d2c640 1 -- 172.21.15.27:0/1407981391 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f61800d18b0 con 0x7f61800b1ca0 2024-09-12T03:20:15.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:15.451+0000 7f616effd640 1 -- 172.21.15.27:0/1407981391 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+5225 (secure 0 0 0) 0x7f617400cd00 con 0x7f61800b1ca0 2024-09-12T03:20:15.451 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:20:15.452 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":39,"btime":"2024-09-12T03:20:10:420320+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6045,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6834/3058537826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":3058537826},{"type":"v1","addr":"172.21.15.27:6834","nonce":3058537826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":34},{"gid":6048,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/1585628536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":1585628536},{"type":"v1","addr":"172.21.15.27:6835","nonce":1585628536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":34},{"gid":6063,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6836/1421588365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":1421588365},{"type":"v1","addr":"172.21.15.45:6836","nonce":1421588365}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-12T03:19:57.738864+0000","modified":"2024-09-12T03:19:59.740115+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6066},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6066":{"gid":6066,"name":"d","rank":0,"incarnation":36,"state":"up:active","state_seq":3,"addr":"172.21.15.45:6837/2890007436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6835","nonce":2890007436},{"type":"v1","addr":"172.21.15.45:6837","nonce":2890007436}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6066,"qdb_cluster":[6066]},"id":5}]} 2024-09-12T03:20:15.452 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 39 2024-09-12T03:20:15.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:15.453+0000 7f6184d2c640 1 -- 172.21.15.27:0/1407981391 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f61800b1ca0 msgr2=0x7f6180159fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:15.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:15.453+0000 7f6184d2c640 1 --2- 172.21.15.27:0/1407981391 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f61800b1ca0 0x7f6180159fe0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f617400e810 tx=0x7f617400ecd0 comp rx=0 tx=0).stop 2024-09-12T03:20:15.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:15.453+0000 7f6184d2c640 1 -- 172.21.15.27:0/1407981391 shutdown_connections 2024-09-12T03:20:15.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:15.453+0000 7f6184d2c640 1 --2- 172.21.15.27:0/1407981391 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f61800cf490 0x7f61800b2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:15.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:15.453+0000 7f6184d2c640 1 --2- 172.21.15.27:0/1407981391 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f61800b1ca0 0x7f6180159fe0 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:15.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:15.453+0000 7f6184d2c640 1 --2- 172.21.15.27:0/1407981391 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f618006f360 0x7f61801f4ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:15.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:15.453+0000 7f6184d2c640 1 -- 172.21.15.27:0/1407981391 >> 172.21.15.27:0/1407981391 conn(0x7f6180073800 msgr2=0x7f61800700e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:15.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:15.453+0000 7f6184d2c640 1 -- 172.21.15.27:0/1407981391 shutdown_connections 2024-09-12T03:20:15.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:15.453+0000 7f6184d2c640 1 -- 172.21.15.27:0/1407981391 wait complete. 2024-09-12T03:20:15.464 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json session ls 2024-09-12T03:20:15.468 INFO:tasks.ceph.mgr.x.smithi045.stderr:2024-09-12T03:20:15.468+0000 7ff5de91e040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-12T03:20:15.584 INFO:tasks.ceph.mgr.x.smithi045.stderr:2024-09-12T03:20:15.585+0000 7ff5de91e040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-12T03:20:15.586 INFO:teuthology.orchestra.run.smithi045.stdout:[{"id":5962,"entity":{"name":{"type":"client","num":5962},"addr":{"type":"any","addr":"172.21.15.27:0","nonce":3044361423}},"state":"open","num_leases":0,"num_caps":2,"request_load_avg":2,"uptime":0.39799412499999998,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":1.9981871137864802,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":2,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.5962 172.21.15.27:0/3044361423","completed_requests":[],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc","ceph_version":"ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)","entity_id":"y","hostname":"smithi027","pid":"36345","root":"/"}},{"id":5564,"entity":{"name":{"type":"client","num":5564},"addr":{"type":"any","addr":"172.21.15.27:0","nonce":1439268209}},"state":"open","num_leases":0,"num_caps":2,"request_load_avg":2,"uptime":0.39799412499999998,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":1.9981894221481278,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":2,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.5564 172.21.15.27:0/1439268209","completed_requests":[],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc","ceph_version":"ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)","entity_id":"y","hostname":"smithi027","pid":"36345","root":"/"}}] 2024-09-12T03:20:15.587 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc", "ceph_version": "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "y", "hostname": "smithi027", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "36345", "root": "/" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "172.21.15.27:0", "nonce": 3044361423, "type": "any" }, "name": { "num": 5962, "type": "client" } }, "id": 5962, "inst": "client.5962 172.21.15.27:0/3044361423", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 2, "num_caps": 2, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 2, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 1.9981871137864802 }, "state": "open", "uptime": 0.397994125 }, { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc", "ceph_version": "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "y", "hostname": "smithi027", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "36345", "root": "/" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "172.21.15.27:0", "nonce": 1439268209, "type": "any" }, "name": { "num": 5564, "type": "client" } }, "id": 5564, "inst": "client.5564 172.21.15.27:0/1439268209", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 2, "num_caps": 2, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 2, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 1.9981894221481278 }, "state": "open", "uptime": 0.397994125 } ] 2024-09-12T03:20:15.587 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-12T03:20:15.587 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-12T03:20:15.587 DEBUG:teuthology.orchestra.run.smithi027:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-12T03:20:15.633 DEBUG:teuthology.orchestra.run.smithi045:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-12T03:20:15.676 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-12T03:20:15.677 INFO:tasks.cephfs.mount:Terminating background process 2024-09-12T03:20:15.677 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-12T03:20:15.677 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T03:20:15.683 INFO:tasks.ceph.mgr.x.smithi045.stderr:2024-09-12T03:20:15.684+0000 7ff5de91e040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-12T03:20:15.693 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056004,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497432k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16324 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248716k,nr_inodes=812179,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497432k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-12T03:20:15.693 INFO:tasks.cephfs.mount:Terminating background process 2024-09-12T03:20:15.694 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-12T03:20:15.694 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T03:20:15.710 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056795,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16602 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249348k,nr_inodes=812337,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-12T03:20:15.710 INFO:tasks.cephfs.mount:Terminating background process 2024-09-12T03:20:15.710 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-12T03:20:15.710 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T03:20:15.750 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056004,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497432k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16324 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248716k,nr_inodes=812179,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497432k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-12T03:20:15.750 INFO:tasks.cephfs.mount:Terminating background process 2024-09-12T03:20:15.750 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-12T03:20:15.750 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T03:20:15.768 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056795,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16602 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249348k,nr_inodes=812337,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-12T03:20:15.768 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-12T03:20:15.892 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:15.892+0000 7f98931b7640 1 Processor -- start 2024-09-12T03:20:15.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:15.892+0000 7f98931b7640 1 -- start start 2024-09-12T03:20:15.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:15.893+0000 7f98931b7640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f988c0d5200 0x7f988c0d1010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:15.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:15.893+0000 7f98931b7640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f988c0d1550 0x7f988c0b8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:15.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:15.893+0000 7f98931b7640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f988c0d19c0 con 0x7f988c15eb90 2024-09-12T03:20:15.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:15.893+0000 7f98931b7640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f988c0b7d20 con 0x7f988c0d1550 2024-09-12T03:20:15.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:15.893+0000 7f98931b7640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f988c0d42e0 con 0x7f988c0d5200 2024-09-12T03:20:15.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:15.893+0000 7f9890f53640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f988c15eb90 0x7f988c0d2df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:50922/0 (socket says 172.21.15.27:50922) 2024-09-12T03:20:15.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:15.893+0000 7f9890f53640 1 -- 172.21.15.27:0/2554736302 learned_addr learned my addr 172.21.15.27:0/2554736302 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:20:15.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:15.894+0000 7f988bfff640 1 --2- 172.21.15.27:0/2554736302 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f988c0d5200 0x7f988c0d1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:15.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:15.894+0000 7f9891754640 1 --2- 172.21.15.27:0/2554736302 >> v2:172.21.15.45:3300/0 conn(0x7f988c0d1550 0x7f988c0b8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:15.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:15.894+0000 7f988b7fe640 1 -- 172.21.15.27:0/2554736302 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1046048123 0 0) 0x7f988c0d19c0 con 0x7f988c15eb90 2024-09-12T03:20:15.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:15.894+0000 7f988b7fe640 1 -- 172.21.15.27:0/2554736302 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9874003520 con 0x7f988c15eb90 2024-09-12T03:20:15.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:15.894+0000 7f9891754640 1 -- 172.21.15.27:0/2554736302 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f988c0d5200 msgr2=0x7f988c0d1010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:15.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:15.894+0000 7f9891754640 1 --2- 172.21.15.27:0/2554736302 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f988c0d5200 0x7f988c0d1010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:15.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:15.894+0000 7f9891754640 1 -- 172.21.15.27:0/2554736302 >> v1:172.21.15.27:6789/0 conn(0x7f988c15eb90 legacy=0x7f988c0d2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:15.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:15.894+0000 7f9891754640 1 -- 172.21.15.27:0/2554736302 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f988c0d3760 con 0x7f988c0d1550 2024-09-12T03:20:15.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:15.894+0000 7f988bfff640 1 --2- 172.21.15.27:0/2554736302 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f988c0d5200 0x7f988c0d1010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:20:15.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:15.895+0000 7f9891754640 1 --2- 172.21.15.27:0/2554736302 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f988c0d1550 0x7f988c0b8190 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f987c0097a0 tx=0x7f987c02a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=94bcd2a31e60b03a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:15.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:15.895+0000 7f988b7fe640 1 -- 172.21.15.27:0/2554736302 <== mon.1 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f987c037070 con 0x7f988c0d1550 2024-09-12T03:20:15.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:15.895+0000 7f988b7fe640 1 -- 172.21.15.27:0/2554736302 <== mon.1 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f987c032440 con 0x7f988c0d1550 2024-09-12T03:20:15.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:15.895+0000 7f988b7fe640 1 -- 172.21.15.27:0/2554736302 <== mon.1 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f987c0327f0 con 0x7f988c0d1550 2024-09-12T03:20:15.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:15.895+0000 7f98931b7640 1 -- 172.21.15.27:0/2554736302 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f988c0d1550 msgr2=0x7f988c0b8190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:15.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:15.895+0000 7f98931b7640 1 --2- 172.21.15.27:0/2554736302 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f988c0d1550 0x7f988c0b8190 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f987c0097a0 tx=0x7f987c02a180 comp rx=0 tx=0).stop 2024-09-12T03:20:15.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:15.895+0000 7f98931b7640 1 -- 172.21.15.27:0/2554736302 shutdown_connections 2024-09-12T03:20:15.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:15.895+0000 7f98931b7640 1 --2- 172.21.15.27:0/2554736302 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f988c0d1550 0x7f988c0b8190 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:15.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:15.895+0000 7f98931b7640 1 --2- 172.21.15.27:0/2554736302 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f988c0d5200 0x7f988c0d1010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:15.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:15.895+0000 7f98931b7640 1 -- 172.21.15.27:0/2554736302 >> 172.21.15.27:0/2554736302 conn(0x7f988c06dce0 msgr2=0x7f988c06e0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:15.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:15.895+0000 7f98931b7640 1 -- 172.21.15.27:0/2554736302 shutdown_connections 2024-09-12T03:20:15.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:15.895+0000 7f98931b7640 1 -- 172.21.15.27:0/2554736302 wait complete. 2024-09-12T03:20:15.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:15.897+0000 7f98931b7640 1 Processor -- start 2024-09-12T03:20:15.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:15.897+0000 7f98931b7640 1 -- start start 2024-09-12T03:20:15.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:15.897+0000 7f98931b7640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f988c0d1550 0x7f988c1e7ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:15.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:15.898+0000 7f98931b7640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f988c0d5200 0x7f988c160750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:15.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:15.898+0000 7f98931b7640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f988c15eb90 0x7f988c14d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:15.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:15.898+0000 7f98931b7640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f988c0d3d20 con 0x7f988c0d5200 2024-09-12T03:20:15.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:15.898+0000 7f98931b7640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f988c0d3500 con 0x7f988c15eb90 2024-09-12T03:20:15.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:15.898+0000 7f98931b7640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f988c0d4000 con 0x7f988c0d1550 2024-09-12T03:20:15.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:15.898+0000 7f988bfff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f988c0d5200 0x7f988c160750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:15.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:15.898+0000 7f9890f53640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f988c0d1550 0x7f988c1e7ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:15.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:15.898+0000 7f988bfff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f988c0d5200 0x7f988c160750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58048/0 (socket says 172.21.15.27:58048) 2024-09-12T03:20:15.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:15.898+0000 7f988bfff640 1 -- 172.21.15.27:0/85908997 learned_addr learned my addr 172.21.15.27:0/85908997 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:20:15.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:15.898+0000 7f9891754640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f988c15eb90 0x7f988c14d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:15.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:15.898+0000 7f988bfff640 1 -- 172.21.15.27:0/85908997 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f988c0d1550 msgr2=0x7f988c1e7ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:15.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:15.898+0000 7f988bfff640 1 --2- 172.21.15.27:0/85908997 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f988c0d1550 0x7f988c1e7ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:15.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:15.898+0000 7f988bfff640 1 -- 172.21.15.27:0/85908997 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f988c15eb90 msgr2=0x7f988c14d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:15.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:15.898+0000 7f988bfff640 1 --2- 172.21.15.27:0/85908997 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f988c15eb90 0x7f988c14d610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:15.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:15.898+0000 7f988bfff640 1 -- 172.21.15.27:0/85908997 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f988c1e8580 con 0x7f988c0d5200 2024-09-12T03:20:15.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:15.898+0000 7f9891754640 1 --2- 172.21.15.27:0/85908997 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f988c15eb90 0x7f988c14d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:20:15.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:15.898+0000 7f988bfff640 1 --2- 172.21.15.27:0/85908997 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f988c0d5200 0x7f988c160750 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f988000e9b0 tx=0x7f988000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:15.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:15.898+0000 7f98897fa640 1 -- 172.21.15.27:0/85908997 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9880017070 con 0x7f988c0d5200 2024-09-12T03:20:15.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:15.898+0000 7f98897fa640 1 -- 172.21.15.27:0/85908997 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f988000ca00 con 0x7f988c0d5200 2024-09-12T03:20:15.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:15.898+0000 7f98931b7640 1 -- 172.21.15.27:0/85908997 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f988c1e8860 con 0x7f988c0d5200 2024-09-12T03:20:15.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:15.898+0000 7f98897fa640 1 -- 172.21.15.27:0/85908997 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f988000cdf0 con 0x7f988c0d5200 2024-09-12T03:20:15.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:15.898+0000 7f98931b7640 1 -- 172.21.15.27:0/85908997 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f988c1e8e20 con 0x7f988c0d5200 2024-09-12T03:20:15.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:15.899+0000 7f98897fa640 1 -- 172.21.15.27:0/85908997 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 13) ==== 45274+0+0 (secure 0 0 0) 0x7f988001a070 con 0x7f988c0d5200 2024-09-12T03:20:15.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:15.899+0000 7f98931b7640 1 -- 172.21.15.27:0/85908997 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f988c15f000 con 0x7f988c0d5200 2024-09-12T03:20:15.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:15.900+0000 7f98897fa640 1 -- 172.21.15.27:0/85908997 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 6529+0+0 (secure 0 0 0) 0x7f988004b660 con 0x7f988c0d5200 2024-09-12T03:20:15.904 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:15.904+0000 7f98897fa640 1 -- 172.21.15.27:0/85908997 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f9880009360 con 0x7f988c0d5200 2024-09-12T03:20:15.965 INFO:tasks.ceph.mgr.x.smithi045.stderr:2024-09-12T03:20:15.965+0000 7ff5de91e040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-12T03:20:16.011 INFO:tasks.ceph.mgr.x.smithi045.stderr:2024-09-12T03:20:16.011+0000 7ff5de91e040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-12T03:20:16.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.082+0000 7f98897fa640 1 -- 172.21.15.27:0/85908997 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mgrmap(e 14) ==== 45401+0+0 (secure 0 0 0) 0x7f9880013940 con 0x7f988c0d5200 2024-09-12T03:20:16.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.082+0000 7f98897fa640 1 --2- 172.21.15.27:0/85908997 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f987803dff0 0x7f98780403e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:16.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.083+0000 7f9890f53640 1 --2- 172.21.15.27:0/85908997 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f987803dff0 0x7f98780403e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:16.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.084+0000 7f9890f53640 1 --2- 172.21.15.27:0/85908997 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f987803dff0 0x7f98780403e0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f9884005c00 tx=0x7f9884009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:16.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.118+0000 7f98931b7640 1 -- 172.21.15.27:0/85908997 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f988c15e340 con 0x7f988c0d5200 2024-09-12T03:20:16.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.119+0000 7f98897fa640 1 -- 172.21.15.27:0/85908997 <== mon.0 v2:172.21.15.27:3300/0 8 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+5225 (secure 0 0 0) 0x7f9880016e00 con 0x7f988c0d5200 2024-09-12T03:20:16.119 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:20:16.119 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":39,"btime":"2024-09-12T03:20:10:420320+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6045,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6834/3058537826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":3058537826},{"type":"v1","addr":"172.21.15.27:6834","nonce":3058537826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":34},{"gid":6048,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/1585628536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":1585628536},{"type":"v1","addr":"172.21.15.27:6835","nonce":1585628536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":34},{"gid":6063,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6836/1421588365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":1421588365},{"type":"v1","addr":"172.21.15.45:6836","nonce":1421588365}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-12T03:19:57.738864+0000","modified":"2024-09-12T03:19:59.740115+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6066},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6066":{"gid":6066,"name":"d","rank":0,"incarnation":36,"state":"up:active","state_seq":3,"addr":"172.21.15.45:6837/2890007436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6835","nonce":2890007436},{"type":"v1","addr":"172.21.15.45:6837","nonce":2890007436}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6066,"qdb_cluster":[6066]},"id":5}]} 2024-09-12T03:20:16.119 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 39 2024-09-12T03:20:16.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.121+0000 7f98931b7640 1 -- 172.21.15.27:0/85908997 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f987803dff0 msgr2=0x7f98780403e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:16.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.121+0000 7f98931b7640 1 --2- 172.21.15.27:0/85908997 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f987803dff0 0x7f98780403e0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f9884005c00 tx=0x7f9884009040 comp rx=0 tx=0).stop 2024-09-12T03:20:16.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.121+0000 7f98931b7640 1 -- 172.21.15.27:0/85908997 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f988c0d5200 msgr2=0x7f988c160750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:16.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.121+0000 7f98931b7640 1 --2- 172.21.15.27:0/85908997 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f988c0d5200 0x7f988c160750 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f988000e9b0 tx=0x7f988000ee70 comp rx=0 tx=0).stop 2024-09-12T03:20:16.122 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.121+0000 7f98931b7640 1 -- 172.21.15.27:0/85908997 shutdown_connections 2024-09-12T03:20:16.122 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.121+0000 7f98931b7640 1 --2- 172.21.15.27:0/85908997 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f988c15eb90 0x7f988c14d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:16.122 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.121+0000 7f98931b7640 1 --2- 172.21.15.27:0/85908997 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f987803dff0 0x7f98780403e0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:16.122 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.121+0000 7f98931b7640 1 --2- 172.21.15.27:0/85908997 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f988c0d5200 0x7f988c160750 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:16.122 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.121+0000 7f98931b7640 1 --2- 172.21.15.27:0/85908997 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f988c0d1550 0x7f988c1e7ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:16.122 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.121+0000 7f98931b7640 1 -- 172.21.15.27:0/85908997 >> 172.21.15.27:0/85908997 conn(0x7f988c06dce0 msgr2=0x7f988c161ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:16.122 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.121+0000 7f98931b7640 1 -- 172.21.15.27:0/85908997 shutdown_connections 2024-09-12T03:20:16.122 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.121+0000 7f98931b7640 1 -- 172.21.15.27:0/85908997 wait complete. 2024-09-12T03:20:16.132 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-12T03:20:16.135 INFO:tasks.ceph.mgr.x.smithi045.stderr:2024-09-12T03:20:16.135+0000 7ff5de91e040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-12T03:20:16.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.235+0000 7f95066c3640 1 Processor -- start 2024-09-12T03:20:16.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.235+0000 7f95066c3640 1 -- start start 2024-09-12T03:20:16.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.236+0000 7f95066c3640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f9500145c30 0x7f950015ca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:16.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.236+0000 7f95066c3640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f95000cda40 0x7f95000b4040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:16.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.236+0000 7f95066c3640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f95000b4d70 con 0x7f9500146010 2024-09-12T03:20:16.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.236+0000 7f95066c3640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f95000b5050 con 0x7f95000cda40 2024-09-12T03:20:16.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.236+0000 7f95066c3640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f950006d140 con 0x7f9500145c30 2024-09-12T03:20:16.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.236+0000 7f9504c60640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f95000cda40 0x7f95000b4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:16.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.236+0000 7f9504c60640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f95000cda40 0x7f95000b4040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58064/0 (socket says 172.21.15.27:58064) 2024-09-12T03:20:16.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.236+0000 7f9504c60640 1 -- 172.21.15.27:0/2567031104 learned_addr learned my addr 172.21.15.27:0/2567031104 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:20:16.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.236+0000 7f94fffff640 1 --2- 172.21.15.27:0/2567031104 >> v2:172.21.15.45:3300/0 conn(0x7f9500145c30 0x7f950015ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:16.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.236+0000 7f9504c60640 1 -- 172.21.15.27:0/2567031104 >> v2:172.21.15.45:3300/0 conn(0x7f9500145c30 msgr2=0x7f950015ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:16.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.236+0000 7f9504c60640 1 --2- 172.21.15.27:0/2567031104 >> v2:172.21.15.45:3300/0 conn(0x7f9500145c30 0x7f950015ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:16.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.236+0000 7f9504c60640 1 -- 172.21.15.27:0/2567031104 >> v1:172.21.15.27:6789/0 conn(0x7f9500146010 legacy=0x7f95000cd330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:16.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.236+0000 7f9504c60640 1 -- 172.21.15.27:0/2567031104 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f950006d420 con 0x7f95000cda40 2024-09-12T03:20:16.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.236+0000 7f94fffff640 1 --2- 172.21.15.27:0/2567031104 >> v2:172.21.15.45:3300/0 conn(0x7f9500145c30 0x7f950015ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:20:16.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.237+0000 7f9504c60640 1 --2- 172.21.15.27:0/2567031104 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f95000cda40 0x7f95000b4040 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f9500073390 tx=0x7f94f802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d37c9167e0703a0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:16.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.237+0000 7f94feffd640 1 -- 172.21.15.27:0/2567031104 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94f8036070 con 0x7f95000cda40 2024-09-12T03:20:16.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.237+0000 7f94feffd640 1 -- 172.21.15.27:0/2567031104 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f94f8031600 con 0x7f95000cda40 2024-09-12T03:20:16.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.237+0000 7f94feffd640 1 -- 172.21.15.27:0/2567031104 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94f80319d0 con 0x7f95000cda40 2024-09-12T03:20:16.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.238+0000 7f95066c3640 1 -- 172.21.15.27:0/2567031104 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f95000cda40 msgr2=0x7f95000b4040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:16.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.238+0000 7f95066c3640 1 --2- 172.21.15.27:0/2567031104 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f95000cda40 0x7f95000b4040 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f9500073390 tx=0x7f94f802a6a0 comp rx=0 tx=0).stop 2024-09-12T03:20:16.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.238+0000 7f95066c3640 1 -- 172.21.15.27:0/2567031104 shutdown_connections 2024-09-12T03:20:16.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.238+0000 7f95066c3640 1 --2- 172.21.15.27:0/2567031104 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f95000cda40 0x7f95000b4040 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:16.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.238+0000 7f95066c3640 1 --2- 172.21.15.27:0/2567031104 >> v2:172.21.15.45:3300/0 conn(0x7f9500145c30 0x7f950015ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:16.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.238+0000 7f95066c3640 1 -- 172.21.15.27:0/2567031104 >> 172.21.15.27:0/2567031104 conn(0x7f9500062c60 msgr2=0x7f950006bb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:16.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.238+0000 7f95066c3640 1 -- 172.21.15.27:0/2567031104 shutdown_connections 2024-09-12T03:20:16.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.238+0000 7f95066c3640 1 -- 172.21.15.27:0/2567031104 wait complete. 2024-09-12T03:20:16.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.239+0000 7f95066c3640 1 Processor -- start 2024-09-12T03:20:16.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.239+0000 7f95066c3640 1 -- start start 2024-09-12T03:20:16.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.240+0000 7f95066c3640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f95000cda40 0x7f950015bec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:16.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.240+0000 7f95066c3640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f9500145c30 0x7f950013f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:16.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.240+0000 7f95066c3640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9500146010 0x7f95001405f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:16.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.240+0000 7f95066c3640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f95000b4700 con 0x7f9500146010 2024-09-12T03:20:16.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.240+0000 7f95066c3640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f95000b4580 con 0x7f95000cda40 2024-09-12T03:20:16.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.240+0000 7f95066c3640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f95000b4880 con 0x7f9500145c30 2024-09-12T03:20:16.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.240+0000 7f9504c60640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9500146010 0x7f95001405f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:16.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.240+0000 7f9504c60640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9500146010 0x7f95001405f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58070/0 (socket says 172.21.15.27:58070) 2024-09-12T03:20:16.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.240+0000 7f9504c60640 1 -- 172.21.15.27:0/4203043783 learned_addr learned my addr 172.21.15.27:0/4203043783 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:20:16.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.240+0000 7f94fffff640 1 --2- 172.21.15.27:0/4203043783 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f95000cda40 0x7f950015bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:16.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.240+0000 7f94ff7fe640 1 --2- 172.21.15.27:0/4203043783 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f9500145c30 0x7f950013f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:16.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.240+0000 7f9504c60640 1 -- 172.21.15.27:0/4203043783 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f9500145c30 msgr2=0x7f950013f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:16.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.240+0000 7f9504c60640 1 --2- 172.21.15.27:0/4203043783 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f9500145c30 0x7f950013f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:16.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.240+0000 7f9504c60640 1 -- 172.21.15.27:0/4203043783 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f95000cda40 msgr2=0x7f950015bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:16.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.240+0000 7f9504c60640 1 --2- 172.21.15.27:0/4203043783 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f95000cda40 0x7f950015bec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:16.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.240+0000 7f9504c60640 1 -- 172.21.15.27:0/4203043783 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9500073050 con 0x7f9500146010 2024-09-12T03:20:16.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.241+0000 7f9504c60640 1 --2- 172.21.15.27:0/4203043783 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9500146010 0x7f95001405f0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f94f8009f40 tx=0x7f94f8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:16.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.241+0000 7f94fcff9640 1 -- 172.21.15.27:0/4203043783 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94f8036050 con 0x7f9500146010 2024-09-12T03:20:16.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.241+0000 7f94fcff9640 1 -- 172.21.15.27:0/4203043783 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f94f8042070 con 0x7f9500146010 2024-09-12T03:20:16.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.241+0000 7f94fcff9640 1 -- 172.21.15.27:0/4203043783 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94f8009510 con 0x7f9500146010 2024-09-12T03:20:16.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.241+0000 7f95066c3640 1 -- 172.21.15.27:0/4203043783 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9500072230 con 0x7f9500146010 2024-09-12T03:20:16.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.241+0000 7f95066c3640 1 -- 172.21.15.27:0/4203043783 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95000727b0 con 0x7f9500146010 2024-09-12T03:20:16.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.242+0000 7f94fcff9640 1 -- 172.21.15.27:0/4203043783 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 14) ==== 45401+0+0 (secure 0 0 0) 0x7f94f8007bd0 con 0x7f9500146010 2024-09-12T03:20:16.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.242+0000 7f94fcff9640 1 --2- 172.21.15.27:0/4203043783 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f94ec037fc0 0x7f94ec03a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:16.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.242+0000 7f94fcff9640 1 -- 172.21.15.27:0/4203043783 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 6529+0+0 (secure 0 0 0) 0x7f94f806c640 con 0x7f9500146010 2024-09-12T03:20:16.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.242+0000 7f95066c3640 1 -- 172.21.15.27:0/4203043783 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95000b4040 con 0x7f9500146010 2024-09-12T03:20:16.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.242+0000 7f94fffff640 1 --2- 172.21.15.27:0/4203043783 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f94ec037fc0 0x7f94ec03a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:16.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.242+0000 7f94fffff640 1 --2- 172.21.15.27:0/4203043783 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f94ec037fc0 0x7f94ec03a480 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f94f0006e80 tx=0x7f94f0008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:16.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.247+0000 7f94fcff9640 1 -- 172.21.15.27:0/4203043783 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f94f80463c0 con 0x7f9500146010 2024-09-12T03:20:16.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.457+0000 7f95066c3640 1 -- 172.21.15.27:0/4203043783 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f95001e5bb0 con 0x7f9500146010 2024-09-12T03:20:16.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.457+0000 7f94fcff9640 1 -- 172.21.15.27:0/4203043783 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+5225 (secure 0 0 0) 0x7f94f80465a0 con 0x7f9500146010 2024-09-12T03:20:16.457 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:20:16.457 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":39,"btime":"2024-09-12T03:20:10:420320+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6045,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6834/3058537826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":3058537826},{"type":"v1","addr":"172.21.15.27:6834","nonce":3058537826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":34},{"gid":6048,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/1585628536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":1585628536},{"type":"v1","addr":"172.21.15.27:6835","nonce":1585628536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":34},{"gid":6063,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6836/1421588365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":1421588365},{"type":"v1","addr":"172.21.15.45:6836","nonce":1421588365}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-12T03:19:57.738864+0000","modified":"2024-09-12T03:19:59.740115+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6066},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6066":{"gid":6066,"name":"d","rank":0,"incarnation":36,"state":"up:active","state_seq":3,"addr":"172.21.15.45:6837/2890007436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6835","nonce":2890007436},{"type":"v1","addr":"172.21.15.45:6837","nonce":2890007436}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6066,"qdb_cluster":[6066]},"id":5}]} 2024-09-12T03:20:16.458 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 39 2024-09-12T03:20:16.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.459+0000 7f95066c3640 1 -- 172.21.15.27:0/4203043783 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f94ec037fc0 msgr2=0x7f94ec03a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:16.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.459+0000 7f95066c3640 1 --2- 172.21.15.27:0/4203043783 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f94ec037fc0 0x7f94ec03a480 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f94f0006e80 tx=0x7f94f0008040 comp rx=0 tx=0).stop 2024-09-12T03:20:16.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.459+0000 7f95066c3640 1 -- 172.21.15.27:0/4203043783 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9500146010 msgr2=0x7f95001405f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:16.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.459+0000 7f95066c3640 1 --2- 172.21.15.27:0/4203043783 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9500146010 0x7f95001405f0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f94f8009f40 tx=0x7f94f8004a40 comp rx=0 tx=0).stop 2024-09-12T03:20:16.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.459+0000 7f95066c3640 1 -- 172.21.15.27:0/4203043783 shutdown_connections 2024-09-12T03:20:16.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.459+0000 7f95066c3640 1 --2- 172.21.15.27:0/4203043783 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f94ec037fc0 0x7f94ec03a480 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:16.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.459+0000 7f95066c3640 1 --2- 172.21.15.27:0/4203043783 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9500146010 0x7f95001405f0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:16.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.459+0000 7f95066c3640 1 --2- 172.21.15.27:0/4203043783 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f9500145c30 0x7f950013f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:16.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.459+0000 7f95066c3640 1 --2- 172.21.15.27:0/4203043783 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f95000cda40 0x7f950015bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:16.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.459+0000 7f95066c3640 1 -- 172.21.15.27:0/4203043783 >> 172.21.15.27:0/4203043783 conn(0x7f9500062c60 msgr2=0x7f95000ab710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:16.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.459+0000 7f95066c3640 1 -- 172.21.15.27:0/4203043783 shutdown_connections 2024-09-12T03:20:16.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.459+0000 7f95066c3640 1 -- 172.21.15.27:0/4203043783 wait complete. 2024-09-12T03:20:16.471 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-12T03:20:16.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.573+0000 7f1ddcdd6640 1 Processor -- start 2024-09-12T03:20:16.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.573+0000 7f1ddcdd6640 1 -- start start 2024-09-12T03:20:16.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.574+0000 7f1ddcdd6640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f1dd815a8c0 0x7f1dd80cdb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:16.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.574+0000 7f1ddcdd6640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1dd806daf0 con 0x7f1dd815aca0 2024-09-12T03:20:16.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.574+0000 7f1ddcdd6640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1dd806ddd0 con 0x7f1dd8071be0 2024-09-12T03:20:16.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.574+0000 7f1ddcdd6640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f1dd806d8a0 con 0x7f1dd815a8c0 2024-09-12T03:20:16.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.574+0000 7f1dd6d9d640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f1dd815aca0 0x7f1dd80b4710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:50936/0 (socket says 172.21.15.27:50936) 2024-09-12T03:20:16.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.574+0000 7f1dd6d9d640 1 -- 172.21.15.27:0/3184930640 learned_addr learned my addr 172.21.15.27:0/3184930640 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:20:16.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.574+0000 7f1dd659c640 1 --2- 172.21.15.27:0/3184930640 >> v2:172.21.15.45:3300/0 conn(0x7f1dd815a8c0 0x7f1dd80cdb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:16.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.575+0000 7f1dd559a640 1 -- 172.21.15.27:0/3184930640 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3678019105 0 0) 0x7f1dd806daf0 con 0x7f1dd815aca0 2024-09-12T03:20:16.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.575+0000 7f1dd559a640 1 -- 172.21.15.27:0/3184930640 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1dc4003520 con 0x7f1dd815aca0 2024-09-12T03:20:16.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.575+0000 7f1dd659c640 1 -- 172.21.15.27:0/3184930640 >> v1:172.21.15.45:6789/0 conn(0x7f1dd8071be0 legacy=0x7f1dd806e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:16.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.575+0000 7f1dd659c640 1 -- 172.21.15.27:0/3184930640 >> v1:172.21.15.27:6789/0 conn(0x7f1dd815aca0 legacy=0x7f1dd80b4710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:16.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.575+0000 7f1dd659c640 1 -- 172.21.15.27:0/3184930640 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1dd806e0b0 con 0x7f1dd815a8c0 2024-09-12T03:20:16.577 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.576+0000 7f1dd659c640 1 --2- 172.21.15.27:0/3184930640 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1dd815a8c0 0x7f1dd80cdb30 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f1dd00097a0 tx=0x7f1dd002a230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f7c46e97a3639039 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:16.577 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.576+0000 7f1dd559a640 1 -- 172.21.15.27:0/3184930640 <== mon.1 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dd0037070 con 0x7f1dd815a8c0 2024-09-12T03:20:16.577 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.576+0000 7f1dd559a640 1 -- 172.21.15.27:0/3184930640 <== mon.1 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1dd0032440 con 0x7f1dd815a8c0 2024-09-12T03:20:16.577 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.576+0000 7f1dd559a640 1 -- 172.21.15.27:0/3184930640 <== mon.1 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dd00327f0 con 0x7f1dd815a8c0 2024-09-12T03:20:16.577 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.576+0000 7f1ddcdd6640 1 -- 172.21.15.27:0/3184930640 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1dd815a8c0 msgr2=0x7f1dd80cdb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:16.577 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.576+0000 7f1ddcdd6640 1 --2- 172.21.15.27:0/3184930640 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1dd815a8c0 0x7f1dd80cdb30 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f1dd00097a0 tx=0x7f1dd002a230 comp rx=0 tx=0).stop 2024-09-12T03:20:16.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.576+0000 7f1ddcdd6640 1 -- 172.21.15.27:0/3184930640 shutdown_connections 2024-09-12T03:20:16.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.576+0000 7f1ddcdd6640 1 --2- 172.21.15.27:0/3184930640 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1dd815a8c0 0x7f1dd80cdb30 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:16.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.576+0000 7f1ddcdd6640 1 -- 172.21.15.27:0/3184930640 >> 172.21.15.27:0/3184930640 conn(0x7f1dd8062c60 msgr2=0x7f1dd806bb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:16.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.576+0000 7f1ddcdd6640 1 -- 172.21.15.27:0/3184930640 shutdown_connections 2024-09-12T03:20:16.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.576+0000 7f1ddcdd6640 1 -- 172.21.15.27:0/3184930640 wait complete. 2024-09-12T03:20:16.581 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.578+0000 7f1ddcdd6640 1 Processor -- start 2024-09-12T03:20:16.581 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.578+0000 7f1ddcdd6640 1 -- start start 2024-09-12T03:20:16.581 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.578+0000 7f1ddcdd6640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1dd8071be0 0x7f1dd80cea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:16.581 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.578+0000 7f1ddcdd6640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1dd815a8c0 0x7f1dd815c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:16.581 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.578+0000 7f1ddcdd6640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1dd815aca0 0x7f1dd815c610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:16.581 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.578+0000 7f1ddcdd6640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f1dd806f140 con 0x7f1dd8071be0 2024-09-12T03:20:16.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.578+0000 7f1ddcdd6640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f1dd806efc0 con 0x7f1dd815aca0 2024-09-12T03:20:16.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.578+0000 7f1ddcdd6640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f1dd80b3560 con 0x7f1dd815a8c0 2024-09-12T03:20:16.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.579+0000 7f1dd659c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1dd8071be0 0x7f1dd80cea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:16.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.579+0000 7f1dd659c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1dd8071be0 0x7f1dd80cea30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58076/0 (socket says 172.21.15.27:58076) 2024-09-12T03:20:16.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.579+0000 7f1dd659c640 1 -- 172.21.15.27:0/202936539 learned_addr learned my addr 172.21.15.27:0/202936539 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:20:16.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.579+0000 7f1dd6d9d640 1 --2- 172.21.15.27:0/202936539 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1dd815aca0 0x7f1dd815c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:16.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.579+0000 7f1dd5d9b640 1 --2- 172.21.15.27:0/202936539 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1dd815a8c0 0x7f1dd815c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:16.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.579+0000 7f1dd659c640 1 -- 172.21.15.27:0/202936539 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1dd815a8c0 msgr2=0x7f1dd815c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:16.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.579+0000 7f1dd659c640 1 --2- 172.21.15.27:0/202936539 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1dd815a8c0 0x7f1dd815c0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:16.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.579+0000 7f1dd659c640 1 -- 172.21.15.27:0/202936539 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1dd815aca0 msgr2=0x7f1dd815c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:16.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.579+0000 7f1dd659c640 1 --2- 172.21.15.27:0/202936539 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1dd815aca0 0x7f1dd815c610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:16.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.579+0000 7f1dd659c640 1 -- 172.21.15.27:0/202936539 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1dd80cfd50 con 0x7f1dd8071be0 2024-09-12T03:20:16.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.579+0000 7f1dd659c640 1 --2- 172.21.15.27:0/202936539 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1dd8071be0 0x7f1dd80cea30 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f1dd00097a0 tx=0x7f1dd00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:16.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.579+0000 7f1dceffd640 1 -- 172.21.15.27:0/202936539 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dd0037050 con 0x7f1dd8071be0 2024-09-12T03:20:16.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.579+0000 7f1ddcdd6640 1 -- 172.21.15.27:0/202936539 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1dd815b1e0 con 0x7f1dd8071be0 2024-09-12T03:20:16.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.579+0000 7f1dceffd640 1 -- 172.21.15.27:0/202936539 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1dd0038040 con 0x7f1dd8071be0 2024-09-12T03:20:16.584 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.579+0000 7f1dceffd640 1 -- 172.21.15.27:0/202936539 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dd0004470 con 0x7f1dd8071be0 2024-09-12T03:20:16.584 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.579+0000 7f1ddcdd6640 1 -- 172.21.15.27:0/202936539 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1dd815b7a0 con 0x7f1dd8071be0 2024-09-12T03:20:16.584 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.580+0000 7f1dceffd640 1 -- 172.21.15.27:0/202936539 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 14) ==== 45401+0+0 (secure 0 0 0) 0x7f1dd003c6d0 con 0x7f1dd8071be0 2024-09-12T03:20:16.584 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.580+0000 7f1ddcdd6640 1 -- 172.21.15.27:0/202936539 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1dd8072ba0 con 0x7f1dd8071be0 2024-09-12T03:20:16.584 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.580+0000 7f1dceffd640 1 --2- 172.21.15.27:0/202936539 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f1db8037fc0 0x7f1db803a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:16.584 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.580+0000 7f1dceffd640 1 -- 172.21.15.27:0/202936539 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 6529+0+0 (secure 0 0 0) 0x7f1dd006c3a0 con 0x7f1dd8071be0 2024-09-12T03:20:16.584 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.581+0000 7f1dd5d9b640 1 --2- 172.21.15.27:0/202936539 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f1db8037fc0 0x7f1db803a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:16.584 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.581+0000 7f1dd5d9b640 1 --2- 172.21.15.27:0/202936539 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f1db8037fc0 0x7f1db803a480 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f1dd80af6f0 tx=0x7f1dc0009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:16.585 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.585+0000 7f1dceffd640 1 -- 172.21.15.27:0/202936539 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f1dd003d3b0 con 0x7f1dd8071be0 2024-09-12T03:20:16.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.762+0000 7f1ddcdd6640 1 -- 172.21.15.27:0/202936539 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1dd81553f0 con 0x7f1dd8071be0 2024-09-12T03:20:16.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.763+0000 7f1dceffd640 1 -- 172.21.15.27:0/202936539 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v70) ==== 74+0+17919 (secure 0 0 0) 0x7f1dd003ee10 con 0x7f1dd8071be0 2024-09-12T03:20:16.763 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:20:16.763 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":70,"fsid":"d2df504e-2647-4385-a5db-9fdf69f487a0","created":"2024-09-12T03:16:07.778937+0000","modified":"2024-09-12T03:20:15.074939+0000","last_up_change":"2024-09-12T03:16:19.314778+0000","last_in_change":"2024-09-12T03:16:09.893086+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T03:16:20.097646+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-09-12T03:19:55.777868+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-09-12T03:19:56.718653+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1851116c-1fb3-4dc9-9772-f90e2ef4c355","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6817","nonce":3002321488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6819","nonce":3002321488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6823","nonce":3002321488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6821","nonce":3002321488}]},"public_addr":"172.21.15.27:6817/3002321488","cluster_addr":"172.21.15.27:6819/3002321488","heartbeat_back_addr":"172.21.15.27:6823/3002321488","heartbeat_front_addr":"172.21.15.27:6821/3002321488","state":["exists","up"]},{"osd":1,"uuid":"f38ae17d-73eb-4204-b99d-975fa806fb87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6809","nonce":1613102886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6811","nonce":1613102886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6815","nonce":1613102886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6813","nonce":1613102886}]},"public_addr":"172.21.15.27:6809/1613102886","cluster_addr":"172.21.15.27:6811/1613102886","heartbeat_back_addr":"172.21.15.27:6815/1613102886","heartbeat_front_addr":"172.21.15.27:6813/1613102886","state":["exists","up"]},{"osd":2,"uuid":"fb12dcc8-bdd0-42b8-b46d-8b10106224bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6825","nonce":3534723355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6827","nonce":3534723355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6831","nonce":3534723355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6829","nonce":3534723355}]},"public_addr":"172.21.15.27:6825/3534723355","cluster_addr":"172.21.15.27:6827/3534723355","heartbeat_back_addr":"172.21.15.27:6831/3534723355","heartbeat_front_addr":"172.21.15.27:6829/3534723355","state":["exists","up"]},{"osd":3,"uuid":"d74d3f49-9c24-4b9f-98e2-3ecd7b5ef3b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6801","nonce":80661598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6803","nonce":80661598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6807","nonce":80661598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6805","nonce":80661598}]},"public_addr":"172.21.15.27:6801/80661598","cluster_addr":"172.21.15.27:6803/80661598","heartbeat_back_addr":"172.21.15.27:6807/80661598","heartbeat_front_addr":"172.21.15.27:6805/80661598","state":["exists","up"]},{"osd":4,"uuid":"2bf123f2-1b0d-41c4-aa3b-805bab267e59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6817","nonce":3943189827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6819","nonce":3943189827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6823","nonce":3943189827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6821","nonce":3943189827}]},"public_addr":"172.21.15.45:6817/3943189827","cluster_addr":"172.21.15.45:6819/3943189827","heartbeat_back_addr":"172.21.15.45:6823/3943189827","heartbeat_front_addr":"172.21.15.45:6821/3943189827","state":["exists","up"]},{"osd":5,"uuid":"c49ba5da-91c6-4a86-a97b-4aba89c253ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6809","nonce":3620760330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6811","nonce":3620760330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6815","nonce":3620760330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6813","nonce":3620760330}]},"public_addr":"172.21.15.45:6809/3620760330","cluster_addr":"172.21.15.45:6811/3620760330","heartbeat_back_addr":"172.21.15.45:6815/3620760330","heartbeat_front_addr":"172.21.15.45:6813/3620760330","state":["exists","up"]},{"osd":6,"uuid":"b0123475-89f7-4a34-9e33-d797b99dd9dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6801","nonce":1989083510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6803","nonce":1989083510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6807","nonce":1989083510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6805","nonce":1989083510}]},"public_addr":"172.21.15.45:6801/1989083510","cluster_addr":"172.21.15.45:6803/1989083510","heartbeat_back_addr":"172.21.15.45:6807/1989083510","heartbeat_front_addr":"172.21.15.45:6805/1989083510","state":["exists","up"]},{"osd":7,"uuid":"4dad773d-b7d8-46fc-8a10-a972aad27b29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6825","nonce":1513300294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6827","nonce":1513300294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6831","nonce":1513300294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6829","nonce":1513300294}]},"public_addr":"172.21.15.45:6825/1513300294","cluster_addr":"172.21.15.45:6827/1513300294","heartbeat_back_addr":"172.21.15.45:6831/1513300294","heartbeat_front_addr":"172.21.15.45:6829/1513300294","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.881855+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:17.133252+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.735339+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.500742+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.477571+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:17.167667+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.470291+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.873743+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"11.3","mappings":[{"from":6,"to":0}]},{"pgid":"11.6","mappings":[{"from":2,"to":4}]},{"pgid":"11.a","mappings":[{"from":6,"to":1}]},{"pgid":"11.10","mappings":[{"from":6,"to":5}]},{"pgid":"11.12","mappings":[{"from":2,"to":5}]},{"pgid":"11.1c","mappings":[{"from":7,"to":1}]},{"pgid":"11.1e","mappings":[{"from":6,"to":4}]},{"pgid":"11.27","mappings":[{"from":7,"to":0}]},{"pgid":"11.33","mappings":[{"from":6,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:0/4216551134":"2024-09-13T03:20:15.074894+0000","172.21.15.45:0/471646388":"2024-09-13T03:20:15.074894+0000","172.21.15.45:0/1945166272":"2024-09-13T03:20:15.074894+0000","172.21.15.45:6832/3923821219":"2024-09-13T03:20:15.074894+0000","172.21.15.45:0/3205085313":"2024-09-13T03:20:15.074894+0000","172.21.15.45:0/3383141969":"2024-09-13T03:20:15.074894+0000","172.21.15.45:0/3895125587":"2024-09-13T03:20:15.074894+0000","172.21.15.45:0/3677899312":"2024-09-13T03:20:15.074894+0000","172.21.15.45:0/2632182829":"2024-09-13T03:20:15.074894+0000","172.21.15.45:0/687436879":"2024-09-13T03:20:15.074894+0000","172.21.15.45:0/4018097172":"2024-09-13T03:20:15.074894+0000","172.21.15.45:6833/3923821219":"2024-09-13T03:20:15.074894+0000","172.21.15.45:0/761029391":"2024-09-13T03:20:15.074894+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T03:20:16.765 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.765+0000 7f1ddcdd6640 1 -- 172.21.15.27:0/202936539 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f1db8037fc0 msgr2=0x7f1db803a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:16.765 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.765+0000 7f1ddcdd6640 1 --2- 172.21.15.27:0/202936539 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f1db8037fc0 0x7f1db803a480 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f1dd80af6f0 tx=0x7f1dc0009040 comp rx=0 tx=0).stop 2024-09-12T03:20:16.765 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.765+0000 7f1ddcdd6640 1 -- 172.21.15.27:0/202936539 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1dd8071be0 msgr2=0x7f1dd80cea30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:16.766 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.765+0000 7f1ddcdd6640 1 --2- 172.21.15.27:0/202936539 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1dd8071be0 0x7f1dd80cea30 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f1dd00097a0 tx=0x7f1dd00046c0 comp rx=0 tx=0).stop 2024-09-12T03:20:16.766 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.765+0000 7f1ddcdd6640 1 -- 172.21.15.27:0/202936539 shutdown_connections 2024-09-12T03:20:16.766 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.765+0000 7f1ddcdd6640 1 --2- 172.21.15.27:0/202936539 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1dd815aca0 0x7f1dd815c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:16.766 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.765+0000 7f1ddcdd6640 1 --2- 172.21.15.27:0/202936539 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1dd815a8c0 0x7f1dd815c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:16.766 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.765+0000 7f1ddcdd6640 1 --2- 172.21.15.27:0/202936539 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f1db8037fc0 0x7f1db803a480 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:16.766 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.765+0000 7f1ddcdd6640 1 --2- 172.21.15.27:0/202936539 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1dd8071be0 0x7f1dd80cea30 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:16.766 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.765+0000 7f1ddcdd6640 1 -- 172.21.15.27:0/202936539 >> 172.21.15.27:0/202936539 conn(0x7f1dd8062c60 msgr2=0x7f1dd8072db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:16.766 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.765+0000 7f1ddcdd6640 1 -- 172.21.15.27:0/202936539 shutdown_connections 2024-09-12T03:20:16.766 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.765+0000 7f1ddcdd6640 1 -- 172.21.15.27:0/202936539 wait complete. 2024-09-12T03:20:16.777 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-12T03:20:16.777 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-12T03:20:16.876 INFO:tasks.ceph.mgr.x.smithi045.stderr:2024-09-12T03:20:16.876+0000 7ff5de91e040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-12T03:20:16.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.880+0000 7ff7aaa27640 1 Processor -- start 2024-09-12T03:20:16.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.880+0000 7ff7aaa27640 1 -- start start 2024-09-12T03:20:16.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.881+0000 7ff7aaa27640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ff7a415e5f0 0x7ff7a4073250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:16.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.881+0000 7ff7aaa27640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7ff7a4073790 0x7ff7a4162320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:16.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.881+0000 7ff7aaa27640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ff7a4162860 0x7ff7a40d1230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:16.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.881+0000 7ff7aaa27640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7ff7a4073be0 con 0x7ff7a4073790 2024-09-12T03:20:16.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.881+0000 7ff7aaa27640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7ff7a40d0da0 con 0x7ff7a415e5f0 2024-09-12T03:20:16.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.881+0000 7ff7aaa27640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7ff7a4163090 con 0x7ff7a4162860 2024-09-12T03:20:16.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.881+0000 7ff7a37fe640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7ff7a4073790 0x7ff7a4162320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:16.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.881+0000 7ff7a37fe640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7ff7a4073790 0x7ff7a4162320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58078/0 (socket says 172.21.15.27:58078) 2024-09-12T03:20:16.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.881+0000 7ff7a37fe640 1 -- 172.21.15.27:0/2106477935 learned_addr learned my addr 172.21.15.27:0/2106477935 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:20:16.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.881+0000 7ff7a3fff640 1 --2- 172.21.15.27:0/2106477935 >> v2:172.21.15.45:3300/0 conn(0x7ff7a415e5f0 0x7ff7a4073250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:16.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.881+0000 7ff7a8fc4640 1 --2- 172.21.15.27:0/2106477935 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ff7a4162860 0x7ff7a40d1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:16.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.882+0000 7ff7a37fe640 1 -- 172.21.15.27:0/2106477935 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ff7a4162860 msgr2=0x7ff7a40d1230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:16.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.882+0000 7ff7a37fe640 1 --2- 172.21.15.27:0/2106477935 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ff7a4162860 0x7ff7a40d1230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:16.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.882+0000 7ff7a37fe640 1 -- 172.21.15.27:0/2106477935 >> v2:172.21.15.45:3300/0 conn(0x7ff7a415e5f0 msgr2=0x7ff7a4073250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:16.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.882+0000 7ff7a37fe640 1 --2- 172.21.15.27:0/2106477935 >> v2:172.21.15.45:3300/0 conn(0x7ff7a415e5f0 0x7ff7a4073250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:16.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.882+0000 7ff7a37fe640 1 -- 172.21.15.27:0/2106477935 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7a40d0470 con 0x7ff7a4073790 2024-09-12T03:20:16.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.882+0000 7ff7a37fe640 1 --2- 172.21.15.27:0/2106477935 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff7a4073790 0x7ff7a4162320 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7ff7a4063570 tx=0x7ff79802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1311fdc1380db3a6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:16.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.882+0000 7ff7a2ffd640 1 -- 172.21.15.27:0/2106477935 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff798037070 con 0x7ff7a4073790 2024-09-12T03:20:16.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.883+0000 7ff7a2ffd640 1 -- 172.21.15.27:0/2106477935 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff798032440 con 0x7ff7a4073790 2024-09-12T03:20:16.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.883+0000 7ff7a2ffd640 1 -- 172.21.15.27:0/2106477935 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff798032810 con 0x7ff7a4073790 2024-09-12T03:20:16.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.883+0000 7ff7aaa27640 1 -- 172.21.15.27:0/2106477935 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff7a4073790 msgr2=0x7ff7a4162320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:16.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.883+0000 7ff7aaa27640 1 --2- 172.21.15.27:0/2106477935 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff7a4073790 0x7ff7a4162320 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7ff7a4063570 tx=0x7ff79802a1e0 comp rx=0 tx=0).stop 2024-09-12T03:20:16.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.883+0000 7ff7aaa27640 1 -- 172.21.15.27:0/2106477935 shutdown_connections 2024-09-12T03:20:16.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.883+0000 7ff7aaa27640 1 --2- 172.21.15.27:0/2106477935 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ff7a4162860 0x7ff7a40d1230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:16.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.883+0000 7ff7aaa27640 1 --2- 172.21.15.27:0/2106477935 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff7a4073790 0x7ff7a4162320 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:16.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.883+0000 7ff7aaa27640 1 --2- 172.21.15.27:0/2106477935 >> v2:172.21.15.45:3300/0 conn(0x7ff7a415e5f0 0x7ff7a4073250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:16.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.883+0000 7ff7aaa27640 1 -- 172.21.15.27:0/2106477935 >> 172.21.15.27:0/2106477935 conn(0x7ff7a406f300 msgr2=0x7ff7a406f710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:16.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.883+0000 7ff7aaa27640 1 -- 172.21.15.27:0/2106477935 shutdown_connections 2024-09-12T03:20:16.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.883+0000 7ff7aaa27640 1 -- 172.21.15.27:0/2106477935 wait complete. 2024-09-12T03:20:16.888 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.884+0000 7ff7aaa27640 1 Processor -- start 2024-09-12T03:20:16.888 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.885+0000 7ff7aaa27640 1 -- start start 2024-09-12T03:20:16.888 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.885+0000 7ff7aaa27640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff7a4073790 0x7ff7a4152cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:16.888 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.885+0000 7ff7aaa27640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ff7a415e5f0 0x7ff7a41565b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:16.888 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.885+0000 7ff7aaa27640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff7a4162860 0x7ff7a4160540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:16.888 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.885+0000 7ff7aaa27640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7ff7a40b5620 con 0x7ff7a4162860 2024-09-12T03:20:16.888 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.885+0000 7ff7aaa27640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ff7a40d0a30 con 0x7ff7a4073790 2024-09-12T03:20:16.888 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.885+0000 7ff7aaa27640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7ff7a4162cd0 con 0x7ff7a415e5f0 2024-09-12T03:20:16.888 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.885+0000 7ff7a8fc4640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff7a4162860 0x7ff7a4160540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:16.888 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.885+0000 7ff7a8fc4640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff7a4162860 0x7ff7a4160540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58092/0 (socket says 172.21.15.27:58092) 2024-09-12T03:20:16.889 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.885+0000 7ff7a3fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff7a4073790 0x7ff7a4152cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:16.889 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.885+0000 7ff7a8fc4640 1 -- 172.21.15.27:0/1037591107 learned_addr learned my addr 172.21.15.27:0/1037591107 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:20:16.889 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.885+0000 7ff7a37fe640 1 --2- 172.21.15.27:0/1037591107 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ff7a415e5f0 0x7ff7a41565b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:16.889 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.886+0000 7ff7a8fc4640 1 -- 172.21.15.27:0/1037591107 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ff7a415e5f0 msgr2=0x7ff7a41565b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:16.889 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.886+0000 7ff7a8fc4640 1 --2- 172.21.15.27:0/1037591107 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ff7a415e5f0 0x7ff7a41565b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:16.889 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.886+0000 7ff7a8fc4640 1 -- 172.21.15.27:0/1037591107 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff7a4073790 msgr2=0x7ff7a4152cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:16.889 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.886+0000 7ff7a8fc4640 1 --2- 172.21.15.27:0/1037591107 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff7a4073790 0x7ff7a4152cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:16.889 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.886+0000 7ff7a8fc4640 1 -- 172.21.15.27:0/1037591107 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7a415f5a0 con 0x7ff7a4162860 2024-09-12T03:20:16.890 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.886+0000 7ff7a8fc4640 1 --2- 172.21.15.27:0/1037591107 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff7a4162860 0x7ff7a4160540 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7ff79c00efc0 tx=0x7ff79c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:16.890 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.886+0000 7ff7a0ff9640 1 -- 172.21.15.27:0/1037591107 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff79c018020 con 0x7ff7a4162860 2024-09-12T03:20:16.890 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.886+0000 7ff7a0ff9640 1 -- 172.21.15.27:0/1037591107 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff79c014480 con 0x7ff7a4162860 2024-09-12T03:20:16.890 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.886+0000 7ff7a0ff9640 1 -- 172.21.15.27:0/1037591107 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff79c014870 con 0x7ff7a4162860 2024-09-12T03:20:16.890 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.886+0000 7ff7aaa27640 1 -- 172.21.15.27:0/1037591107 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7a41538b0 con 0x7ff7a4162860 2024-09-12T03:20:16.890 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.886+0000 7ff7aaa27640 1 -- 172.21.15.27:0/1037591107 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7a4153490 con 0x7ff7a4162860 2024-09-12T03:20:16.890 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.887+0000 7ff7a0ff9640 1 -- 172.21.15.27:0/1037591107 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 14) ==== 45401+0+0 (secure 0 0 0) 0x7ff79c01b020 con 0x7ff7a4162860 2024-09-12T03:20:16.890 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.887+0000 7ff7aaa27640 1 -- 172.21.15.27:0/1037591107 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7a4162320 con 0x7ff7a4162860 2024-09-12T03:20:16.890 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.887+0000 7ff7a0ff9640 1 --2- 172.21.15.27:0/1037591107 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7ff790037fc0 0x7ff79003a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:16.890 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.888+0000 7ff7a0ff9640 1 -- 172.21.15.27:0/1037591107 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 6529+0+0 (secure 0 0 0) 0x7ff79c04c310 con 0x7ff7a4162860 2024-09-12T03:20:16.890 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.888+0000 7ff7a3fff640 1 --2- 172.21.15.27:0/1037591107 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7ff790037fc0 0x7ff79003a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:16.890 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.888+0000 7ff7a3fff640 1 --2- 172.21.15.27:0/1037591107 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7ff790037fc0 0x7ff79003a480 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7ff7a40b76e0 tx=0x7ff794009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:16.892 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:16.892+0000 7ff7a0ff9640 1 -- 172.21.15.27:0/1037591107 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7ff79c022030 con 0x7ff7a4162860 2024-09-12T03:20:16.922 INFO:tasks.ceph.mgr.x.smithi045.stderr:2024-09-12T03:20:16.923+0000 7ff5de91e040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-12T03:20:17.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.083+0000 7ff7a0ff9640 1 -- 172.21.15.27:0/1037591107 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mgrmap(e 15) ==== 45609+0+0 (secure 0 0 0) 0x7ff79c012050 con 0x7ff7a4162860 2024-09-12T03:20:17.110 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.110+0000 7ff7aaa27640 1 -- 172.21.15.27:0/1037591107 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff7a4158ca0 con 0x7ff7a4162860 2024-09-12T03:20:17.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.111+0000 7ff7a0ff9640 1 -- 172.21.15.27:0/1037591107 <== mon.0 v2:172.21.15.27:3300/0 8 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+5225 (secure 0 0 0) 0x7ff79c010040 con 0x7ff7a4162860 2024-09-12T03:20:17.111 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:20:17.111 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":39,"btime":"2024-09-12T03:20:10:420320+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6045,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6834/3058537826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":3058537826},{"type":"v1","addr":"172.21.15.27:6834","nonce":3058537826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":34},{"gid":6048,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/1585628536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":1585628536},{"type":"v1","addr":"172.21.15.27:6835","nonce":1585628536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":34},{"gid":6063,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6836/1421588365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":1421588365},{"type":"v1","addr":"172.21.15.45:6836","nonce":1421588365}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-12T03:19:57.738864+0000","modified":"2024-09-12T03:19:59.740115+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6066},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6066":{"gid":6066,"name":"d","rank":0,"incarnation":36,"state":"up:active","state_seq":3,"addr":"172.21.15.45:6837/2890007436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6835","nonce":2890007436},{"type":"v1","addr":"172.21.15.45:6837","nonce":2890007436}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6066,"qdb_cluster":[6066]},"id":5}]} 2024-09-12T03:20:17.112 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 39 2024-09-12T03:20:17.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.113+0000 7ff7aaa27640 1 -- 172.21.15.27:0/1037591107 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7ff790037fc0 msgr2=0x7ff79003a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:17.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.113+0000 7ff7aaa27640 1 --2- 172.21.15.27:0/1037591107 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7ff790037fc0 0x7ff79003a480 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7ff7a40b76e0 tx=0x7ff794009190 comp rx=0 tx=0).stop 2024-09-12T03:20:17.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.113+0000 7ff7aaa27640 1 -- 172.21.15.27:0/1037591107 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff7a4162860 msgr2=0x7ff7a4160540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:17.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.113+0000 7ff7aaa27640 1 --2- 172.21.15.27:0/1037591107 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff7a4162860 0x7ff7a4160540 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7ff79c00efc0 tx=0x7ff79c00c490 comp rx=0 tx=0).stop 2024-09-12T03:20:17.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.113+0000 7ff7aaa27640 1 -- 172.21.15.27:0/1037591107 shutdown_connections 2024-09-12T03:20:17.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.113+0000 7ff7aaa27640 1 --2- 172.21.15.27:0/1037591107 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7ff790037fc0 0x7ff79003a480 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:17.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.113+0000 7ff7aaa27640 1 --2- 172.21.15.27:0/1037591107 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff7a4162860 0x7ff7a4160540 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:17.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.113+0000 7ff7aaa27640 1 --2- 172.21.15.27:0/1037591107 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ff7a415e5f0 0x7ff7a41565b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:17.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.113+0000 7ff7aaa27640 1 --2- 172.21.15.27:0/1037591107 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff7a4073790 0x7ff7a4152cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:17.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.113+0000 7ff7aaa27640 1 -- 172.21.15.27:0/1037591107 >> 172.21.15.27:0/1037591107 conn(0x7ff7a406f300 msgr2=0x7ff7a40b57a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:17.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.113+0000 7ff7aaa27640 1 -- 172.21.15.27:0/1037591107 shutdown_connections 2024-09-12T03:20:17.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.113+0000 7ff7aaa27640 1 -- 172.21.15.27:0/1037591107 wait complete. 2024-09-12T03:20:17.124 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-12T03:20:17.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.232+0000 7fe82e39d640 1 Processor -- start 2024-09-12T03:20:17.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.233+0000 7fe82e39d640 1 -- start start 2024-09-12T03:20:17.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.233+0000 7fe82e39d640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fe828161d80 0x7fe8280cf100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:17.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.233+0000 7fe82e39d640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe8280d5320 0x7fe8280cf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:17.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.233+0000 7fe82e39d640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe82806c100 con 0x7fe8280b3b40 2024-09-12T03:20:17.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.233+0000 7fe82e39d640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7fe8280d4460 con 0x7fe828161d80 2024-09-12T03:20:17.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.233+0000 7fe82e39d640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fe82806c3e0 con 0x7fe8280d5320 2024-09-12T03:20:17.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.234+0000 7fe82d39b640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fe828161d80 0x7fe8280cf100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:17.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.234+0000 7fe82db9c640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fe8280b3b40 0x7fe8280d24f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:50948/0 (socket says 172.21.15.27:50948) 2024-09-12T03:20:17.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.234+0000 7fe82db9c640 1 -- 172.21.15.27:0/1010768056 learned_addr learned my addr 172.21.15.27:0/1010768056 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:20:17.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.234+0000 7fe82cb9a640 1 --2- 172.21.15.27:0/1010768056 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe8280d5320 0x7fe8280cf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:17.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.234+0000 7fe82d39b640 1 -- 172.21.15.27:0/1010768056 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe8280d5320 msgr2=0x7fe8280cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:17.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.234+0000 7fe82d39b640 1 --2- 172.21.15.27:0/1010768056 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe8280d5320 0x7fe8280cf640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:17.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.234+0000 7fe82d39b640 1 -- 172.21.15.27:0/1010768056 >> v1:172.21.15.27:6789/0 conn(0x7fe8280b3b40 legacy=0x7fe8280d24f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:17.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.234+0000 7fe827fff640 1 -- 172.21.15.27:0/1010768056 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3350834967 0 0) 0x7fe82806c100 con 0x7fe8280b3b40 2024-09-12T03:20:17.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.234+0000 7fe82d39b640 1 -- 172.21.15.27:0/1010768056 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe82806c6c0 con 0x7fe828161d80 2024-09-12T03:20:17.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.235+0000 7fe82d39b640 1 --2- 172.21.15.27:0/1010768056 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe828161d80 0x7fe8280cf100 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fe818009dc0 tx=0x7fe81802a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29106a97454ac22 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:17.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.235+0000 7fe827fff640 1 -- 172.21.15.27:0/1010768056 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe81802ae60 con 0x7fe828161d80 2024-09-12T03:20:17.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.235+0000 7fe827fff640 1 -- 172.21.15.27:0/1010768056 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe818007740 con 0x7fe828161d80 2024-09-12T03:20:17.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.235+0000 7fe827fff640 1 -- 172.21.15.27:0/1010768056 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe818007b10 con 0x7fe828161d80 2024-09-12T03:20:17.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.235+0000 7fe82e39d640 1 -- 172.21.15.27:0/1010768056 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe828161d80 msgr2=0x7fe8280cf100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:17.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.235+0000 7fe82e39d640 1 --2- 172.21.15.27:0/1010768056 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe828161d80 0x7fe8280cf100 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fe818009dc0 tx=0x7fe81802a680 comp rx=0 tx=0).stop 2024-09-12T03:20:17.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.235+0000 7fe82e39d640 1 -- 172.21.15.27:0/1010768056 shutdown_connections 2024-09-12T03:20:17.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.235+0000 7fe82e39d640 1 --2- 172.21.15.27:0/1010768056 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe8280d5320 0x7fe8280cf640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:17.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.235+0000 7fe82e39d640 1 --2- 172.21.15.27:0/1010768056 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe828161d80 0x7fe8280cf100 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:17.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.235+0000 7fe82e39d640 1 -- 172.21.15.27:0/1010768056 >> 172.21.15.27:0/1010768056 conn(0x7fe8280d45f0 msgr2=0x7fe8280d4a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:17.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.236+0000 7fe82e39d640 1 -- 172.21.15.27:0/1010768056 shutdown_connections 2024-09-12T03:20:17.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.236+0000 7fe82e39d640 1 -- 172.21.15.27:0/1010768056 wait complete. 2024-09-12T03:20:17.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.237+0000 7fe82e39d640 1 Processor -- start 2024-09-12T03:20:17.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.237+0000 7fe82e39d640 1 -- start start 2024-09-12T03:20:17.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.237+0000 7fe82e39d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe8280b3b40 0x7fe8281f7640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:17.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.237+0000 7fe82e39d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe8280d5320 0x7fe8280cc340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:17.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.237+0000 7fe82e39d640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe828161d80 0x7fe82815bee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:17.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.238+0000 7fe82e39d640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fe8280d4220 con 0x7fe8280b3b40 2024-09-12T03:20:17.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.238+0000 7fe82e39d640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe8280d40a0 con 0x7fe8280d5320 2024-09-12T03:20:17.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.238+0000 7fe82e39d640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fe8280b8900 con 0x7fe828161d80 2024-09-12T03:20:17.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.238+0000 7fe82d39b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe8280b3b40 0x7fe8281f7640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:17.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.238+0000 7fe82d39b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe8280b3b40 0x7fe8281f7640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58128/0 (socket says 172.21.15.27:58128) 2024-09-12T03:20:17.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.238+0000 7fe82d39b640 1 -- 172.21.15.27:0/85583411 learned_addr learned my addr 172.21.15.27:0/85583411 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:20:17.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.238+0000 7fe82cb9a640 1 --2- 172.21.15.27:0/85583411 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe8280d5320 0x7fe8280cc340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:17.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.238+0000 7fe82db9c640 1 --2- 172.21.15.27:0/85583411 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe828161d80 0x7fe82815bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:17.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.238+0000 7fe82d39b640 1 -- 172.21.15.27:0/85583411 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe828161d80 msgr2=0x7fe82815bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:17.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.238+0000 7fe82d39b640 1 --2- 172.21.15.27:0/85583411 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe828161d80 0x7fe82815bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:17.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.238+0000 7fe82d39b640 1 -- 172.21.15.27:0/85583411 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe8280d5320 msgr2=0x7fe8280cc340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:17.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.238+0000 7fe82d39b640 1 --2- 172.21.15.27:0/85583411 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe8280d5320 0x7fe8280cc340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:17.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.238+0000 7fe82d39b640 1 -- 172.21.15.27:0/85583411 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8281f7f40 con 0x7fe8280b3b40 2024-09-12T03:20:17.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.238+0000 7fe82d39b640 1 --2- 172.21.15.27:0/85583411 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe8280b3b40 0x7fe8281f7640 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fe818009dc0 tx=0x7fe818004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:17.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.238+0000 7fe825ffb640 1 -- 172.21.15.27:0/85583411 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe81803f070 con 0x7fe8280b3b40 2024-09-12T03:20:17.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.238+0000 7fe825ffb640 1 -- 172.21.15.27:0/85583411 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe81802ae60 con 0x7fe8280b3b40 2024-09-12T03:20:17.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.238+0000 7fe825ffb640 1 -- 172.21.15.27:0/85583411 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe818031c80 con 0x7fe8280b3b40 2024-09-12T03:20:17.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.238+0000 7fe82e39d640 1 -- 172.21.15.27:0/85583411 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8281f8220 con 0x7fe8280b3b40 2024-09-12T03:20:17.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.239+0000 7fe82e39d640 1 -- 172.21.15.27:0/85583411 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe82815b6e0 con 0x7fe8280b3b40 2024-09-12T03:20:17.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.239+0000 7fe825ffb640 1 -- 172.21.15.27:0/85583411 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 15) ==== 45609+0+0 (secure 0 0 0) 0x7fe818043410 con 0x7fe8280b3b40 2024-09-12T03:20:17.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.239+0000 7fe82e39d640 1 -- 172.21.15.27:0/85583411 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe828063330 con 0x7fe8280b3b40 2024-09-12T03:20:17.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.240+0000 7fe825ffb640 1 --2- 172.21.15.27:0/85583411 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fe814038200 0x7fe81403a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:17.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.240+0000 7fe825ffb640 1 -- 172.21.15.27:0/85583411 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 6529+0+0 (secure 0 0 0) 0x7fe81806cea0 con 0x7fe8280b3b40 2024-09-12T03:20:17.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.240+0000 7fe82cb9a640 1 --2- 172.21.15.27:0/85583411 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fe814038200 0x7fe81403a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:17.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.240+0000 7fe82cb9a640 1 --2- 172.21.15.27:0/85583411 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fe814038200 0x7fe81403a6c0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fe82815eb70 tx=0x7fe81c009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:17.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.244+0000 7fe825ffb640 1 -- 172.21.15.27:0/85583411 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fe818031e20 con 0x7fe8280b3b40 2024-09-12T03:20:17.257 INFO:tasks.ceph.mgr.x.smithi045.stderr:2024-09-12T03:20:17.257+0000 7ff5de91e040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-12T03:20:17.343 INFO:tasks.ceph.mgr.x.smithi045.stderr:2024-09-12T03:20:17.344+0000 7ff5de91e040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-12T03:20:17.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.456+0000 7fe82e39d640 1 -- 172.21.15.27:0/85583411 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe8280cf100 con 0x7fe8280b3b40 2024-09-12T03:20:17.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.456+0000 7fe825ffb640 1 -- 172.21.15.27:0/85583411 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+5225 (secure 0 0 0) 0x7fe81803a560 con 0x7fe8280b3b40 2024-09-12T03:20:17.456 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:20:17.456 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":39,"btime":"2024-09-12T03:20:10:420320+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6045,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6834/3058537826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":3058537826},{"type":"v1","addr":"172.21.15.27:6834","nonce":3058537826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":34},{"gid":6048,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/1585628536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":1585628536},{"type":"v1","addr":"172.21.15.27:6835","nonce":1585628536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":34},{"gid":6063,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6836/1421588365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":1421588365},{"type":"v1","addr":"172.21.15.45:6836","nonce":1421588365}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-12T03:19:57.738864+0000","modified":"2024-09-12T03:19:59.740115+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6066},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6066":{"gid":6066,"name":"d","rank":0,"incarnation":36,"state":"up:active","state_seq":3,"addr":"172.21.15.45:6837/2890007436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6835","nonce":2890007436},{"type":"v1","addr":"172.21.15.45:6837","nonce":2890007436}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6066,"qdb_cluster":[6066]},"id":5}]} 2024-09-12T03:20:17.457 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 39 2024-09-12T03:20:17.457 INFO:tasks.ceph.mgr.x.smithi045.stderr:2024-09-12T03:20:17.457+0000 7ff5de91e040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-12T03:20:17.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.458+0000 7fe82e39d640 1 -- 172.21.15.27:0/85583411 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fe814038200 msgr2=0x7fe81403a6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:17.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.458+0000 7fe82e39d640 1 --2- 172.21.15.27:0/85583411 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fe814038200 0x7fe81403a6c0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fe82815eb70 tx=0x7fe81c009190 comp rx=0 tx=0).stop 2024-09-12T03:20:17.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.458+0000 7fe82e39d640 1 -- 172.21.15.27:0/85583411 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe8280b3b40 msgr2=0x7fe8281f7640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:17.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.458+0000 7fe82e39d640 1 --2- 172.21.15.27:0/85583411 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe8280b3b40 0x7fe8281f7640 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fe818009dc0 tx=0x7fe818004a40 comp rx=0 tx=0).stop 2024-09-12T03:20:17.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.458+0000 7fe82e39d640 1 -- 172.21.15.27:0/85583411 shutdown_connections 2024-09-12T03:20:17.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.458+0000 7fe82e39d640 1 --2- 172.21.15.27:0/85583411 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe828161d80 0x7fe82815bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:17.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.458+0000 7fe82e39d640 1 --2- 172.21.15.27:0/85583411 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe8280d5320 0x7fe8280cc340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:17.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.458+0000 7fe82e39d640 1 --2- 172.21.15.27:0/85583411 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fe814038200 0x7fe81403a6c0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:17.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.458+0000 7fe82e39d640 1 --2- 172.21.15.27:0/85583411 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe8280b3b40 0x7fe8281f7640 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:17.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.458+0000 7fe82e39d640 1 -- 172.21.15.27:0/85583411 >> 172.21.15.27:0/85583411 conn(0x7fe8280d45f0 msgr2=0x7fe8280b4910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:17.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.458+0000 7fe82e39d640 1 -- 172.21.15.27:0/85583411 shutdown_connections 2024-09-12T03:20:17.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.458+0000 7fe82e39d640 1 -- 172.21.15.27:0/85583411 wait complete. 2024-09-12T03:20:17.469 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-12T03:20:17.502 INFO:tasks.ceph.mgr.x.smithi045.stderr:2024-09-12T03:20:17.502+0000 7ff5de91e040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-12T03:20:17.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.571+0000 7f185ceba640 1 Processor -- start 2024-09-12T03:20:17.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.571+0000 7f185ceba640 1 -- start start 2024-09-12T03:20:17.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.572+0000 7f185ceba640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1858145c30 0x7f185815ca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:17.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.572+0000 7f185ceba640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f1858146010 0x7f18580cd330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:17.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.572+0000 7f185ceba640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f18580cd870 0x7f18580b4040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:17.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.572+0000 7f185ceba640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f18580b5050 con 0x7f18580cd870 2024-09-12T03:20:17.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.572+0000 7f185ceba640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f18580b4d70 con 0x7f1858146010 2024-09-12T03:20:17.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.572+0000 7f185ceba640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f185806d140 con 0x7f1858145c30 2024-09-12T03:20:17.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.572+0000 7f1856d9d640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f18580cd870 0x7f18580b4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:17.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.572+0000 7f1856d9d640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f18580cd870 0x7f18580b4040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58140/0 (socket says 172.21.15.27:58140) 2024-09-12T03:20:17.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.572+0000 7f1856d9d640 1 -- 172.21.15.27:0/392053538 learned_addr learned my addr 172.21.15.27:0/392053538 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:20:17.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.572+0000 7f185659c640 1 --2- 172.21.15.27:0/392053538 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1858145c30 0x7f185815ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:17.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.572+0000 7f1855d9b640 1 --2- 172.21.15.27:0/392053538 >> v2:172.21.15.45:3300/0 conn(0x7f1858146010 0x7f18580cd330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:17.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.572+0000 7f1856d9d640 1 -- 172.21.15.27:0/392053538 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1858145c30 msgr2=0x7f185815ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:17.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.572+0000 7f1856d9d640 1 --2- 172.21.15.27:0/392053538 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1858145c30 0x7f185815ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:17.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.572+0000 7f1856d9d640 1 -- 172.21.15.27:0/392053538 >> v2:172.21.15.45:3300/0 conn(0x7f1858146010 msgr2=0x7f18580cd330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:17.574 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.572+0000 7f1856d9d640 1 --2- 172.21.15.27:0/392053538 >> v2:172.21.15.45:3300/0 conn(0x7f1858146010 0x7f18580cd330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:17.574 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.572+0000 7f1856d9d640 1 -- 172.21.15.27:0/392053538 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f185806d420 con 0x7f18580cd870 2024-09-12T03:20:17.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.573+0000 7f1856d9d640 1 --2- 172.21.15.27:0/392053538 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f18580cd870 0x7f18580b4040 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f185806d350 tx=0x7f185002a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70482ccae3ca8f19 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:17.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.573+0000 7f185559a640 1 -- 172.21.15.27:0/392053538 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1850036070 con 0x7f18580cd870 2024-09-12T03:20:17.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.573+0000 7f185559a640 1 -- 172.21.15.27:0/392053538 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1850031600 con 0x7f18580cd870 2024-09-12T03:20:17.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.573+0000 7f185559a640 1 -- 172.21.15.27:0/392053538 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18500319d0 con 0x7f18580cd870 2024-09-12T03:20:17.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.574+0000 7f185ceba640 1 -- 172.21.15.27:0/392053538 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f18580cd870 msgr2=0x7f18580b4040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:17.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.574+0000 7f185ceba640 1 --2- 172.21.15.27:0/392053538 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f18580cd870 0x7f18580b4040 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f185806d350 tx=0x7f185002a5f0 comp rx=0 tx=0).stop 2024-09-12T03:20:17.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.574+0000 7f185ceba640 1 -- 172.21.15.27:0/392053538 shutdown_connections 2024-09-12T03:20:17.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.574+0000 7f185ceba640 1 --2- 172.21.15.27:0/392053538 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f18580cd870 0x7f18580b4040 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:17.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.574+0000 7f185ceba640 1 --2- 172.21.15.27:0/392053538 >> v2:172.21.15.45:3300/0 conn(0x7f1858146010 0x7f18580cd330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:17.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.574+0000 7f185ceba640 1 --2- 172.21.15.27:0/392053538 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1858145c30 0x7f185815ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:17.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.574+0000 7f185ceba640 1 -- 172.21.15.27:0/392053538 >> 172.21.15.27:0/392053538 conn(0x7f1858062c60 msgr2=0x7f185806bb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:17.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.574+0000 7f185ceba640 1 -- 172.21.15.27:0/392053538 shutdown_connections 2024-09-12T03:20:17.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.574+0000 7f185ceba640 1 -- 172.21.15.27:0/392053538 wait complete. 2024-09-12T03:20:17.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.575+0000 7f185ceba640 1 Processor -- start 2024-09-12T03:20:17.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.575+0000 7f185ceba640 1 -- start start 2024-09-12T03:20:17.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.576+0000 7f185ceba640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f18580cd870 0x7f18580af2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:17.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.576+0000 7f185ceba640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1858145c30 0x7f185813da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:17.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.576+0000 7f185ceba640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1858146010 0x7f185813dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:17.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.576+0000 7f185ceba640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f18580b4700 con 0x7f1858146010 2024-09-12T03:20:17.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.576+0000 7f185ceba640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f18580b4580 con 0x7f18580cd870 2024-09-12T03:20:17.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.576+0000 7f185ceba640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f18580b4880 con 0x7f1858145c30 2024-09-12T03:20:17.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.576+0000 7f1856d9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1858146010 0x7f185813dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:17.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.576+0000 7f185659c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f18580cd870 0x7f18580af2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:17.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.576+0000 7f1856d9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1858146010 0x7f185813dfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58146/0 (socket says 172.21.15.27:58146) 2024-09-12T03:20:17.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.576+0000 7f185659c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f18580cd870 0x7f18580af2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.27:59182/0 (socket says 172.21.15.27:59182) 2024-09-12T03:20:17.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.576+0000 7f1856d9d640 1 -- 172.21.15.27:0/161756020 learned_addr learned my addr 172.21.15.27:0/161756020 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:20:17.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.576+0000 7f1855d9b640 1 --2- 172.21.15.27:0/161756020 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1858145c30 0x7f185813da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:17.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.577+0000 7f1856d9d640 1 -- 172.21.15.27:0/161756020 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1858145c30 msgr2=0x7f185813da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:17.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.577+0000 7f1856d9d640 1 --2- 172.21.15.27:0/161756020 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1858145c30 0x7f185813da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:17.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.577+0000 7f1856d9d640 1 -- 172.21.15.27:0/161756020 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f18580cd870 msgr2=0x7f18580af2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:17.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.577+0000 7f1856d9d640 1 --2- 172.21.15.27:0/161756020 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f18580cd870 0x7f18580af2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:17.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.577+0000 7f1856d9d640 1 -- 172.21.15.27:0/161756020 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18580c9a60 con 0x7f1858146010 2024-09-12T03:20:17.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.577+0000 7f185659c640 1 --2- 172.21.15.27:0/161756020 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f18580cd870 0x7f18580af2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:20:17.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.577+0000 7f1856d9d640 1 --2- 172.21.15.27:0/161756020 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1858146010 0x7f185813dfb0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f1850009f10 tx=0x7f1850004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:17.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.577+0000 7f1846ffd640 1 -- 172.21.15.27:0/161756020 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1850036050 con 0x7f1858146010 2024-09-12T03:20:17.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.577+0000 7f1846ffd640 1 -- 172.21.15.27:0/161756020 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1850042070 con 0x7f1858146010 2024-09-12T03:20:17.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.577+0000 7f185ceba640 1 -- 172.21.15.27:0/161756020 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18580c9d40 con 0x7f1858146010 2024-09-12T03:20:17.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.577+0000 7f1846ffd640 1 -- 172.21.15.27:0/161756020 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1850009510 con 0x7f1858146010 2024-09-12T03:20:17.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.577+0000 7f185ceba640 1 -- 172.21.15.27:0/161756020 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18580ca300 con 0x7f1858146010 2024-09-12T03:20:17.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.578+0000 7f1846ffd640 1 -- 172.21.15.27:0/161756020 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 15) ==== 45609+0+0 (secure 0 0 0) 0x7f1850007bf0 con 0x7f1858146010 2024-09-12T03:20:17.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.578+0000 7f185ceba640 1 -- 172.21.15.27:0/161756020 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f185815ca00 con 0x7f1858146010 2024-09-12T03:20:17.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.578+0000 7f1846ffd640 1 --2- 172.21.15.27:0/161756020 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f1838038200 0x7f183803a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:17.581 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.578+0000 7f1846ffd640 1 -- 172.21.15.27:0/161756020 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 6529+0+0 (secure 0 0 0) 0x7f185006ca90 con 0x7f1858146010 2024-09-12T03:20:17.581 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.578+0000 7f185659c640 1 --2- 172.21.15.27:0/161756020 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f1838038200 0x7f183803a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:17.581 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.579+0000 7f185659c640 1 --2- 172.21.15.27:0/161756020 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f1838038200 0x7f183803a6c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f18580afc80 tx=0x7f1848009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:17.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.583+0000 7f1846ffd640 1 -- 172.21.15.27:0/161756020 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f18500463c0 con 0x7f1858146010 2024-09-12T03:20:17.618 INFO:tasks.ceph.mgr.x.smithi045.stderr:2024-09-12T03:20:17.618+0000 7ff5de91e040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-12T03:20:17.672 INFO:tasks.ceph.mgr.x.smithi045.stderr:2024-09-12T03:20:17.672+0000 7ff5de91e040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-12T03:20:17.715 INFO:tasks.ceph.mgr.x.smithi045.stderr:2024-09-12T03:20:17.715+0000 7ff5de91e040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-12T03:20:17.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.758+0000 7f185ceba640 1 -- 172.21.15.27:0/161756020 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f18580cd180 con 0x7f1858146010 2024-09-12T03:20:17.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.760+0000 7f1846ffd640 1 -- 172.21.15.27:0/161756020 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v70) ==== 74+0+17919 (secure 0 0 0) 0x7f18500465a0 con 0x7f1858146010 2024-09-12T03:20:17.760 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:20:17.760 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":70,"fsid":"d2df504e-2647-4385-a5db-9fdf69f487a0","created":"2024-09-12T03:16:07.778937+0000","modified":"2024-09-12T03:20:15.074939+0000","last_up_change":"2024-09-12T03:16:19.314778+0000","last_in_change":"2024-09-12T03:16:09.893086+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T03:16:20.097646+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-09-12T03:19:55.777868+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-09-12T03:19:56.718653+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1851116c-1fb3-4dc9-9772-f90e2ef4c355","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6817","nonce":3002321488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6819","nonce":3002321488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6823","nonce":3002321488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6821","nonce":3002321488}]},"public_addr":"172.21.15.27:6817/3002321488","cluster_addr":"172.21.15.27:6819/3002321488","heartbeat_back_addr":"172.21.15.27:6823/3002321488","heartbeat_front_addr":"172.21.15.27:6821/3002321488","state":["exists","up"]},{"osd":1,"uuid":"f38ae17d-73eb-4204-b99d-975fa806fb87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6809","nonce":1613102886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6811","nonce":1613102886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6815","nonce":1613102886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6813","nonce":1613102886}]},"public_addr":"172.21.15.27:6809/1613102886","cluster_addr":"172.21.15.27:6811/1613102886","heartbeat_back_addr":"172.21.15.27:6815/1613102886","heartbeat_front_addr":"172.21.15.27:6813/1613102886","state":["exists","up"]},{"osd":2,"uuid":"fb12dcc8-bdd0-42b8-b46d-8b10106224bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6825","nonce":3534723355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6827","nonce":3534723355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6831","nonce":3534723355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6829","nonce":3534723355}]},"public_addr":"172.21.15.27:6825/3534723355","cluster_addr":"172.21.15.27:6827/3534723355","heartbeat_back_addr":"172.21.15.27:6831/3534723355","heartbeat_front_addr":"172.21.15.27:6829/3534723355","state":["exists","up"]},{"osd":3,"uuid":"d74d3f49-9c24-4b9f-98e2-3ecd7b5ef3b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6801","nonce":80661598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6803","nonce":80661598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6807","nonce":80661598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6805","nonce":80661598}]},"public_addr":"172.21.15.27:6801/80661598","cluster_addr":"172.21.15.27:6803/80661598","heartbeat_back_addr":"172.21.15.27:6807/80661598","heartbeat_front_addr":"172.21.15.27:6805/80661598","state":["exists","up"]},{"osd":4,"uuid":"2bf123f2-1b0d-41c4-aa3b-805bab267e59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6817","nonce":3943189827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6819","nonce":3943189827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6823","nonce":3943189827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6821","nonce":3943189827}]},"public_addr":"172.21.15.45:6817/3943189827","cluster_addr":"172.21.15.45:6819/3943189827","heartbeat_back_addr":"172.21.15.45:6823/3943189827","heartbeat_front_addr":"172.21.15.45:6821/3943189827","state":["exists","up"]},{"osd":5,"uuid":"c49ba5da-91c6-4a86-a97b-4aba89c253ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6809","nonce":3620760330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6811","nonce":3620760330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6815","nonce":3620760330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6813","nonce":3620760330}]},"public_addr":"172.21.15.45:6809/3620760330","cluster_addr":"172.21.15.45:6811/3620760330","heartbeat_back_addr":"172.21.15.45:6815/3620760330","heartbeat_front_addr":"172.21.15.45:6813/3620760330","state":["exists","up"]},{"osd":6,"uuid":"b0123475-89f7-4a34-9e33-d797b99dd9dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6801","nonce":1989083510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6803","nonce":1989083510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6807","nonce":1989083510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6805","nonce":1989083510}]},"public_addr":"172.21.15.45:6801/1989083510","cluster_addr":"172.21.15.45:6803/1989083510","heartbeat_back_addr":"172.21.15.45:6807/1989083510","heartbeat_front_addr":"172.21.15.45:6805/1989083510","state":["exists","up"]},{"osd":7,"uuid":"4dad773d-b7d8-46fc-8a10-a972aad27b29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6825","nonce":1513300294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6827","nonce":1513300294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6831","nonce":1513300294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6829","nonce":1513300294}]},"public_addr":"172.21.15.45:6825/1513300294","cluster_addr":"172.21.15.45:6827/1513300294","heartbeat_back_addr":"172.21.15.45:6831/1513300294","heartbeat_front_addr":"172.21.15.45:6829/1513300294","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.881855+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:17.133252+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.735339+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.500742+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.477571+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:17.167667+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.470291+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.873743+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"11.3","mappings":[{"from":6,"to":0}]},{"pgid":"11.6","mappings":[{"from":2,"to":4}]},{"pgid":"11.a","mappings":[{"from":6,"to":1}]},{"pgid":"11.10","mappings":[{"from":6,"to":5}]},{"pgid":"11.12","mappings":[{"from":2,"to":5}]},{"pgid":"11.1c","mappings":[{"from":7,"to":1}]},{"pgid":"11.1e","mappings":[{"from":6,"to":4}]},{"pgid":"11.27","mappings":[{"from":7,"to":0}]},{"pgid":"11.33","mappings":[{"from":6,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:0/4216551134":"2024-09-13T03:20:15.074894+0000","172.21.15.45:0/471646388":"2024-09-13T03:20:15.074894+0000","172.21.15.45:0/1945166272":"2024-09-13T03:20:15.074894+0000","172.21.15.45:6832/3923821219":"2024-09-13T03:20:15.074894+0000","172.21.15.45:0/3205085313":"2024-09-13T03:20:15.074894+0000","172.21.15.45:0/3383141969":"2024-09-13T03:20:15.074894+0000","172.21.15.45:0/3895125587":"2024-09-13T03:20:15.074894+0000","172.21.15.45:0/3677899312":"2024-09-13T03:20:15.074894+0000","172.21.15.45:0/2632182829":"2024-09-13T03:20:15.074894+0000","172.21.15.45:0/687436879":"2024-09-13T03:20:15.074894+0000","172.21.15.45:0/4018097172":"2024-09-13T03:20:15.074894+0000","172.21.15.45:6833/3923821219":"2024-09-13T03:20:15.074894+0000","172.21.15.45:0/761029391":"2024-09-13T03:20:15.074894+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T03:20:17.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.762+0000 7f185ceba640 1 -- 172.21.15.27:0/161756020 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f1838038200 msgr2=0x7f183803a6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:17.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.762+0000 7f185ceba640 1 --2- 172.21.15.27:0/161756020 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f1838038200 0x7f183803a6c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f18580afc80 tx=0x7f1848009190 comp rx=0 tx=0).stop 2024-09-12T03:20:17.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.762+0000 7f185ceba640 1 -- 172.21.15.27:0/161756020 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1858146010 msgr2=0x7f185813dfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:17.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.762+0000 7f185ceba640 1 --2- 172.21.15.27:0/161756020 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1858146010 0x7f185813dfb0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f1850009f10 tx=0x7f1850004a40 comp rx=0 tx=0).stop 2024-09-12T03:20:17.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.762+0000 7f185ceba640 1 -- 172.21.15.27:0/161756020 shutdown_connections 2024-09-12T03:20:17.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.762+0000 7f185ceba640 1 --2- 172.21.15.27:0/161756020 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f1838038200 0x7f183803a6c0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:17.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.762+0000 7f185ceba640 1 --2- 172.21.15.27:0/161756020 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1858146010 0x7f185813dfb0 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:17.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.762+0000 7f185ceba640 1 --2- 172.21.15.27:0/161756020 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1858145c30 0x7f185813da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:17.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.762+0000 7f185ceba640 1 --2- 172.21.15.27:0/161756020 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f18580cd870 0x7f18580af2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:17.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.762+0000 7f185ceba640 1 -- 172.21.15.27:0/161756020 >> 172.21.15.27:0/161756020 conn(0x7f1858062c60 msgr2=0x7f18580ab600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:17.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.762+0000 7f185ceba640 1 -- 172.21.15.27:0/161756020 shutdown_connections 2024-09-12T03:20:17.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.762+0000 7f185ceba640 1 -- 172.21.15.27:0/161756020 wait complete. 2024-09-12T03:20:17.773 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-12T03:20:17.832 INFO:tasks.ceph.mgr.x.smithi045.stderr:2024-09-12T03:20:17.832+0000 7ff5de91e040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-12T03:20:17.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.877+0000 7fa1f25c4640 1 Processor -- start 2024-09-12T03:20:17.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.877+0000 7fa1f25c4640 1 -- start start 2024-09-12T03:20:17.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.877+0000 7fa1f25c4640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa1ec145c30 0x7fa1ec15ca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:17.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.877+0000 7fa1f25c4640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fa1ec0cda40 0x7fa1ec0b4040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:17.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.877+0000 7fa1f25c4640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa1ec0b4d70 con 0x7fa1ec146010 2024-09-12T03:20:17.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.877+0000 7fa1f25c4640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7fa1ec0b5050 con 0x7fa1ec0cda40 2024-09-12T03:20:17.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.877+0000 7fa1f25c4640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fa1ec06d140 con 0x7fa1ec145c30 2024-09-12T03:20:17.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.878+0000 7fa1f0b61640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fa1ec0cda40 0x7fa1ec0b4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:17.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.878+0000 7fa1f0b61640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fa1ec0cda40 0x7fa1ec0b4040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58162/0 (socket says 172.21.15.27:58162) 2024-09-12T03:20:17.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.878+0000 7fa1f0b61640 1 -- 172.21.15.27:0/615496468 learned_addr learned my addr 172.21.15.27:0/615496468 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:20:17.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.878+0000 7fa1ebfff640 1 --2- 172.21.15.27:0/615496468 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa1ec145c30 0x7fa1ec15ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:17.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.878+0000 7fa1f0b61640 1 -- 172.21.15.27:0/615496468 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa1ec145c30 msgr2=0x7fa1ec15ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:17.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.878+0000 7fa1f0b61640 1 --2- 172.21.15.27:0/615496468 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa1ec145c30 0x7fa1ec15ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:17.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.878+0000 7fa1f0b61640 1 -- 172.21.15.27:0/615496468 >> v1:172.21.15.45:6789/0 conn(0x7fa1ec146010 legacy=0x7fa1ec0cd330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:17.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.878+0000 7fa1f0b61640 1 -- 172.21.15.27:0/615496468 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1ec06d420 con 0x7fa1ec0cda40 2024-09-12T03:20:17.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.879+0000 7fa1f0b61640 1 --2- 172.21.15.27:0/615496468 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa1ec0cda40 0x7fa1ec0b4040 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fa1e4009e50 tx=0x7fa1e402a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5727185032bf98c4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:17.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.879+0000 7fa1eaffd640 1 -- 172.21.15.27:0/615496468 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1e4036070 con 0x7fa1ec0cda40 2024-09-12T03:20:17.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.879+0000 7fa1eaffd640 1 -- 172.21.15.27:0/615496468 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1e4031600 con 0x7fa1ec0cda40 2024-09-12T03:20:17.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.879+0000 7fa1eaffd640 1 -- 172.21.15.27:0/615496468 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1e40319d0 con 0x7fa1ec0cda40 2024-09-12T03:20:17.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.879+0000 7fa1f25c4640 1 -- 172.21.15.27:0/615496468 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa1ec0cda40 msgr2=0x7fa1ec0b4040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:17.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.879+0000 7fa1f25c4640 1 --2- 172.21.15.27:0/615496468 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa1ec0cda40 0x7fa1ec0b4040 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fa1e4009e50 tx=0x7fa1e402a680 comp rx=0 tx=0).stop 2024-09-12T03:20:17.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.879+0000 7fa1f25c4640 1 -- 172.21.15.27:0/615496468 shutdown_connections 2024-09-12T03:20:17.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.879+0000 7fa1f25c4640 1 --2- 172.21.15.27:0/615496468 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa1ec0cda40 0x7fa1ec0b4040 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:17.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.879+0000 7fa1f25c4640 1 --2- 172.21.15.27:0/615496468 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa1ec145c30 0x7fa1ec15ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:17.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.879+0000 7fa1f25c4640 1 -- 172.21.15.27:0/615496468 >> 172.21.15.27:0/615496468 conn(0x7fa1ec062c60 msgr2=0x7fa1ec06bb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:17.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.879+0000 7fa1f25c4640 1 -- 172.21.15.27:0/615496468 shutdown_connections 2024-09-12T03:20:17.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.880+0000 7fa1f25c4640 1 -- 172.21.15.27:0/615496468 wait complete. 2024-09-12T03:20:17.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.881+0000 7fa1f25c4640 1 Processor -- start 2024-09-12T03:20:17.883 INFO:tasks.ceph.mgr.x.smithi045.stderr:2024-09-12T03:20:17.883+0000 7ff5de91e040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-12T03:20:17.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.881+0000 7fa1f25c4640 1 -- start start 2024-09-12T03:20:17.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.882+0000 7fa1f25c4640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa1ec0cda40 0x7fa1ec15be10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:17.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.882+0000 7fa1f25c4640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa1ec145c30 0x7fa1ec0ae790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:17.886 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.882+0000 7fa1f25c4640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa1ec146010 0x7fa1ec0aecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:17.886 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.882+0000 7fa1f25c4640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fa1ec0b4700 con 0x7fa1ec146010 2024-09-12T03:20:17.886 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.882+0000 7fa1f25c4640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa1ec0b4580 con 0x7fa1ec145c30 2024-09-12T03:20:17.886 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.882+0000 7fa1f25c4640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fa1ec0b4880 con 0x7fa1ec0cda40 2024-09-12T03:20:17.886 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.882+0000 7fa1f0b61640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa1ec146010 0x7fa1ec0aecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:17.886 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.882+0000 7fa1f0b61640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa1ec146010 0x7fa1ec0aecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58164/0 (socket says 172.21.15.27:58164) 2024-09-12T03:20:17.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.882+0000 7fa1f0b61640 1 -- 172.21.15.27:0/504698064 learned_addr learned my addr 172.21.15.27:0/504698064 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:20:17.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.882+0000 7fa1ebfff640 1 --2- 172.21.15.27:0/504698064 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa1ec0cda40 0x7fa1ec15be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:17.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.882+0000 7fa1eb7fe640 1 --2- 172.21.15.27:0/504698064 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa1ec145c30 0x7fa1ec0ae790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:17.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.882+0000 7fa1f0b61640 1 -- 172.21.15.27:0/504698064 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa1ec0cda40 msgr2=0x7fa1ec15be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:17.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.882+0000 7fa1f0b61640 1 --2- 172.21.15.27:0/504698064 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa1ec0cda40 0x7fa1ec15be10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:17.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.882+0000 7fa1f0b61640 1 -- 172.21.15.27:0/504698064 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa1ec145c30 msgr2=0x7fa1ec0ae790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:17.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.882+0000 7fa1f0b61640 1 --2- 172.21.15.27:0/504698064 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa1ec145c30 0x7fa1ec0ae790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:17.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.882+0000 7fa1f0b61640 1 -- 172.21.15.27:0/504698064 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1ec0cb5b0 con 0x7fa1ec146010 2024-09-12T03:20:17.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.882+0000 7fa1f0b61640 1 --2- 172.21.15.27:0/504698064 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa1ec146010 0x7fa1ec0aecd0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fa1e4034040 tx=0x7fa1e4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:17.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.882+0000 7fa1e8ff9640 1 -- 172.21.15.27:0/504698064 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1e4036050 con 0x7fa1ec146010 2024-09-12T03:20:17.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.882+0000 7fa1e8ff9640 1 -- 172.21.15.27:0/504698064 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1e4042070 con 0x7fa1ec146010 2024-09-12T03:20:17.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.883+0000 7fa1f25c4640 1 -- 172.21.15.27:0/504698064 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1ec071160 con 0x7fa1ec146010 2024-09-12T03:20:17.888 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.883+0000 7fa1e8ff9640 1 -- 172.21.15.27:0/504698064 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1e4009510 con 0x7fa1ec146010 2024-09-12T03:20:17.888 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.883+0000 7fa1f25c4640 1 -- 172.21.15.27:0/504698064 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1ec071720 con 0x7fa1ec146010 2024-09-12T03:20:17.888 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.883+0000 7fa1e8ff9640 1 -- 172.21.15.27:0/504698064 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 15) ==== 45609+0+0 (secure 0 0 0) 0x7fa1e4007bf0 con 0x7fa1ec146010 2024-09-12T03:20:17.888 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.883+0000 7fa1f25c4640 1 -- 172.21.15.27:0/504698064 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1ec0b4040 con 0x7fa1ec146010 2024-09-12T03:20:17.888 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.884+0000 7fa1e8ff9640 1 --2- 172.21.15.27:0/504698064 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fa1d8038200 0x7fa1d803a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:17.888 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.884+0000 7fa1e8ff9640 1 -- 172.21.15.27:0/504698064 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 6529+0+0 (secure 0 0 0) 0x7fa1e406cc20 con 0x7fa1ec146010 2024-09-12T03:20:17.888 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.884+0000 7fa1ebfff640 1 --2- 172.21.15.27:0/504698064 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fa1d8038200 0x7fa1d803a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:17.888 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.884+0000 7fa1ebfff640 1 --2- 172.21.15.27:0/504698064 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fa1d8038200 0x7fa1d803a6c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fa1dc009b40 tx=0x7fa1dc009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:17.888 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:17.888+0000 7fa1e8ff9640 1 -- 172.21.15.27:0/504698064 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fa1e40463c0 con 0x7fa1ec146010 2024-09-12T03:20:17.933 INFO:tasks.ceph.mgr.x.smithi045.stderr:2024-09-12T03:20:17.933+0000 7ff5de91e040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-12T03:20:17.988 INFO:tasks.ceph.mgr.x.smithi045.stderr:2024-09-12T03:20:17.988+0000 7ff5de91e040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-12T03:20:18.032 INFO:tasks.ceph.mgr.x.smithi045.stderr:2024-09-12T03:20:18.032+0000 7ff5de91e040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-12T03:20:18.075 INFO:tasks.ceph.mgr.x.smithi045.stderr:2024-09-12T03:20:18.075+0000 7ff5de91e040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-12T03:20:18.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:18.101+0000 7fa1f25c4640 1 -- 172.21.15.27:0/504698064 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fa1ec0cd540 con 0x7fa1ec146010 2024-09-12T03:20:18.101 INFO:tasks.ceph.mon.a.smithi027.stderr:2024-09-12T03:20:18.101+0000 7f959bfd9640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-12T03:20:18.102 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:18.102+0000 7fa1e8ff9640 1 -- 172.21.15.27:0/504698064 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v40) ==== 161+0+0 (secure 0 0 0) 0x7fa1e40465a0 con 0x7fa1ec146010 2024-09-12T03:20:18.102 INFO:teuthology.orchestra.run.smithi027.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-12T03:20:18.106 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:18.105+0000 7fa1f25c4640 1 -- 172.21.15.27:0/504698064 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fa1d8038200 msgr2=0x7fa1d803a6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:18.106 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:18.105+0000 7fa1f25c4640 1 --2- 172.21.15.27:0/504698064 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fa1d8038200 0x7fa1d803a6c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fa1dc009b40 tx=0x7fa1dc009190 comp rx=0 tx=0).stop 2024-09-12T03:20:18.106 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:18.105+0000 7fa1f25c4640 1 -- 172.21.15.27:0/504698064 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa1ec146010 msgr2=0x7fa1ec0aecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:18.106 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:18.105+0000 7fa1f25c4640 1 --2- 172.21.15.27:0/504698064 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa1ec146010 0x7fa1ec0aecd0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fa1e4034040 tx=0x7fa1e4004a40 comp rx=0 tx=0).stop 2024-09-12T03:20:18.106 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:18.106+0000 7fa1f25c4640 1 -- 172.21.15.27:0/504698064 shutdown_connections 2024-09-12T03:20:18.106 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:18.106+0000 7fa1f25c4640 1 --2- 172.21.15.27:0/504698064 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fa1d8038200 0x7fa1d803a6c0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:18.106 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:18.106+0000 7fa1f25c4640 1 --2- 172.21.15.27:0/504698064 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa1ec146010 0x7fa1ec0aecd0 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:18.106 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:18.106+0000 7fa1f25c4640 1 --2- 172.21.15.27:0/504698064 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa1ec145c30 0x7fa1ec0ae790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:18.106 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:18.106+0000 7fa1f25c4640 1 --2- 172.21.15.27:0/504698064 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa1ec0cda40 0x7fa1ec15be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:18.106 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:18.106+0000 7fa1f25c4640 1 -- 172.21.15.27:0/504698064 >> 172.21.15.27:0/504698064 conn(0x7fa1ec062c60 msgr2=0x7fa1ec0cc750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:18.106 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:18.106+0000 7fa1f25c4640 1 -- 172.21.15.27:0/504698064 shutdown_connections 2024-09-12T03:20:18.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:18.106+0000 7fa1f25c4640 1 -- 172.21.15.27:0/504698064 wait complete. 2024-09-12T03:20:18.109 INFO:tasks.ceph.mds.d.smithi045.stderr: -7497> 2024-09-12T03:20:00.369+0000 7fba813ea640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-12T03:20:18.116 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-12T03:20:18.141 INFO:tasks.ceph.mds.d.smithi045.stdout:starting mds.d at 2024-09-12T03:20:18.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:18.221+0000 7fdb9bca5640 1 Processor -- start 2024-09-12T03:20:18.222 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:18.222+0000 7fdb9bca5640 1 -- start start 2024-09-12T03:20:18.223 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:18.222+0000 7fdb9bca5640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fdb94162040 0x7fdb940d1da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:18.223 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:18.222+0000 7fdb9bca5640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fdb9406dfd0 0x7fdb940b7a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:18.223 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:18.222+0000 7fdb9bca5640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdb940d2e30 con 0x7fdb940d3790 2024-09-12T03:20:18.223 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:18.222+0000 7fdb9bca5640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fdb940b7f90 con 0x7fdb9406dfd0 2024-09-12T03:20:18.223 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:18.222+0000 7fdb9bca5640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fdb94162420 con 0x7fdb94162040 2024-09-12T03:20:18.223 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:18.223+0000 7fdb99a41640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fdb94162040 0x7fdb940d1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:18.223 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:18.223+0000 7fdb9a242640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fdb9406dfd0 0x7fdb940b7a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:18.223 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:18.223+0000 7fdb99240640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fdb940d3790 0x7fdb9406d8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.27:52334/0 (socket says 172.21.15.27:52334) 2024-09-12T03:20:18.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:18.223+0000 7fdb9a242640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fdb9406dfd0 0x7fdb940b7a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.27:59200/0 (socket says 172.21.15.27:59200) 2024-09-12T03:20:18.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:18.223+0000 7fdb99a41640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fdb94162040 0x7fdb940d1da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3301/0 says I am v2:172.21.15.27:40722/0 (socket says 172.21.15.27:40722) 2024-09-12T03:20:18.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:18.223+0000 7fdb99240640 1 -- 172.21.15.27:0/1829019021 learned_addr learned my addr 172.21.15.27:0/1829019021 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:20:18.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:18.223+0000 7fdb99a41640 1 -- 172.21.15.27:0/1829019021 >> v2:172.21.15.45:3300/0 conn(0x7fdb9406dfd0 msgr2=0x7fdb940b7a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:18.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:18.223+0000 7fdb99a41640 1 --2- 172.21.15.27:0/1829019021 >> v2:172.21.15.45:3300/0 conn(0x7fdb9406dfd0 0x7fdb940b7a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:18.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:18.223+0000 7fdb99a41640 1 -- 172.21.15.27:0/1829019021 >> v1:172.21.15.45:6789/0 conn(0x7fdb940d3790 legacy=0x7fdb9406d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:18.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:18.223+0000 7fdb98a3f640 1 -- 172.21.15.27:0/1829019021 <== mon.1 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2026238945 0 0) 0x7fdb940d2e30 con 0x7fdb940d3790 2024-09-12T03:20:18.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:18.223+0000 7fdb99a41640 1 -- 172.21.15.27:0/1829019021 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb940d30d0 con 0x7fdb94162040 2024-09-12T03:20:18.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:18.223+0000 7fdb9a242640 1 --2- 172.21.15.27:0/1829019021 >> v2:172.21.15.45:3300/0 conn(0x7fdb9406dfd0 0x7fdb940b7a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:20:18.225 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:18.224+0000 7fdb99a41640 1 --2- 172.21.15.27:0/1829019021 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fdb94162040 0x7fdb940d1da0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fdb900097a0 tx=0x7fdb9002a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9dd7eb1f9dbdc4b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:18.225 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:18.224+0000 7fdb98a3f640 1 -- 172.21.15.27:0/1829019021 <== mon.2 v2:172.21.15.45:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb90037070 con 0x7fdb94162040 2024-09-12T03:20:18.225 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:18.224+0000 7fdb98a3f640 1 -- 172.21.15.27:0/1829019021 <== mon.2 v2:172.21.15.45:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb90032440 con 0x7fdb94162040 2024-09-12T03:20:18.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:18.224+0000 7fdb98a3f640 1 -- 172.21.15.27:0/1829019021 <== mon.2 v2:172.21.15.45:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb90032810 con 0x7fdb94162040 2024-09-12T03:20:18.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:18.225+0000 7fdb9bca5640 1 -- 172.21.15.27:0/1829019021 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fdb94162040 msgr2=0x7fdb940d1da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:18.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:18.225+0000 7fdb9bca5640 1 --2- 172.21.15.27:0/1829019021 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fdb94162040 0x7fdb940d1da0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fdb900097a0 tx=0x7fdb9002a180 comp rx=0 tx=0).stop 2024-09-12T03:20:18.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:18.225+0000 7fdb9bca5640 1 -- 172.21.15.27:0/1829019021 shutdown_connections 2024-09-12T03:20:18.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:18.225+0000 7fdb9bca5640 1 --2- 172.21.15.27:0/1829019021 >> v2:172.21.15.45:3300/0 conn(0x7fdb9406dfd0 0x7fdb940b7a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:18.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:18.225+0000 7fdb9bca5640 1 --2- 172.21.15.27:0/1829019021 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fdb94162040 0x7fdb940d1da0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:18.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:18.225+0000 7fdb9bca5640 1 -- 172.21.15.27:0/1829019021 >> 172.21.15.27:0/1829019021 conn(0x7fdb940b6700 msgr2=0x7fdb940b6b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:18.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:18.225+0000 7fdb9bca5640 1 -- 172.21.15.27:0/1829019021 shutdown_connections 2024-09-12T03:20:18.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:18.225+0000 7fdb9bca5640 1 -- 172.21.15.27:0/1829019021 wait complete. 2024-09-12T03:20:18.228 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:18.226+0000 7fdb9bca5640 1 Processor -- start 2024-09-12T03:20:18.228 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:18.226+0000 7fdb9bca5640 1 -- start start 2024-09-12T03:20:18.228 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:18.227+0000 7fdb9bca5640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdb9406dfd0 0x7fdb9415e210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:18.228 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:18.227+0000 7fdb9bca5640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fdb940d3790 0x7fdb94161320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:18.228 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:18.227+0000 7fdb9bca5640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdb94162040 0x7fdb9415f940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:18.228 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:18.227+0000 7fdb9bca5640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fdb940d4530 con 0x7fdb9406dfd0 2024-09-12T03:20:18.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:18.227+0000 7fdb9bca5640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fdb940cee00 con 0x7fdb94162040 2024-09-12T03:20:18.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:18.227+0000 7fdb9bca5640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fdb940d4150 con 0x7fdb940d3790 2024-09-12T03:20:18.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:18.227+0000 7fdb99a41640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdb9406dfd0 0x7fdb9415e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:18.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:18.227+0000 7fdb99a41640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdb9406dfd0 0x7fdb9415e210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58176/0 (socket says 172.21.15.27:58176) 2024-09-12T03:20:18.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:18.227+0000 7fdb99a41640 1 -- 172.21.15.27:0/230845959 learned_addr learned my addr 172.21.15.27:0/230845959 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:20:18.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:18.227+0000 7fdb9a242640 1 --2- 172.21.15.27:0/230845959 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdb94162040 0x7fdb9415f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:18.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:18.227+0000 7fdb99240640 1 --2- 172.21.15.27:0/230845959 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fdb940d3790 0x7fdb94161320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:18.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:18.227+0000 7fdb99a41640 1 -- 172.21.15.27:0/230845959 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fdb940d3790 msgr2=0x7fdb94161320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:18.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:18.227+0000 7fdb99a41640 1 --2- 172.21.15.27:0/230845959 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fdb940d3790 0x7fdb94161320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:18.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:18.227+0000 7fdb99a41640 1 -- 172.21.15.27:0/230845959 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdb94162040 msgr2=0x7fdb9415f940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:18.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:18.227+0000 7fdb99a41640 1 --2- 172.21.15.27:0/230845959 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdb94162040 0x7fdb9415f940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:18.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:18.227+0000 7fdb99a41640 1 -- 172.21.15.27:0/230845959 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb94158210 con 0x7fdb9406dfd0 2024-09-12T03:20:18.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:18.227+0000 7fdb99a41640 1 --2- 172.21.15.27:0/230845959 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdb9406dfd0 0x7fdb9415e210 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fdb900097a0 tx=0x7fdb900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:18.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:18.228+0000 7fdb867fc640 1 -- 172.21.15.27:0/230845959 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb90037050 con 0x7fdb9406dfd0 2024-09-12T03:20:18.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:18.228+0000 7fdb867fc640 1 -- 172.21.15.27:0/230845959 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb90038040 con 0x7fdb9406dfd0 2024-09-12T03:20:18.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:18.228+0000 7fdb9bca5640 1 -- 172.21.15.27:0/230845959 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb9415fe80 con 0x7fdb9406dfd0 2024-09-12T03:20:18.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:18.228+0000 7fdb867fc640 1 -- 172.21.15.27:0/230845959 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb90004490 con 0x7fdb9406dfd0 2024-09-12T03:20:18.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:18.228+0000 7fdb9bca5640 1 -- 172.21.15.27:0/230845959 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb9415ee60 con 0x7fdb9406dfd0 2024-09-12T03:20:18.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:18.229+0000 7fdb867fc640 1 -- 172.21.15.27:0/230845959 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 15) ==== 45609+0+0 (secure 0 0 0) 0x7fdb9003c6d0 con 0x7fdb9406dfd0 2024-09-12T03:20:18.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:18.229+0000 7fdb867fc640 1 --2- 172.21.15.27:0/230845959 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fdb78038230 0x7fdb7803a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:18.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:18.229+0000 7fdb9bca5640 1 -- 172.21.15.27:0/230845959 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb940b7a50 con 0x7fdb9406dfd0 2024-09-12T03:20:18.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:18.229+0000 7fdb867fc640 1 -- 172.21.15.27:0/230845959 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 6615+0+0 (secure 0 0 0) 0x7fdb9006c530 con 0x7fdb9406dfd0 2024-09-12T03:20:18.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:18.229+0000 7fdb99240640 1 --2- 172.21.15.27:0/230845959 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fdb78038230 0x7fdb7803a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:18.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:18.229+0000 7fdb99240640 1 --2- 172.21.15.27:0/230845959 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fdb78038230 0x7fdb7803a6f0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fdb9415ddf0 tx=0x7fdb88007d50 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:18.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:18.234+0000 7fdb867fc640 1 -- 172.21.15.27:0/230845959 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fdb9004a100 con 0x7fdb9406dfd0 2024-09-12T03:20:18.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:18.448+0000 7fdb9bca5640 1 -- 172.21.15.27:0/230845959 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fdb9406e6e0 con 0x7fdb9406dfd0 2024-09-12T03:20:18.676 INFO:tasks.ceph.mgr.x.smithi045.stderr:2024-09-12T03:20:18.677+0000 7ff5de91e040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-12T03:20:19.104 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:19.104+0000 7fdb867fc640 1 -- 172.21.15.27:0/230845959 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v41) ==== 104+0+0 (secure 0 0 0) 0x7fdb9003c9c0 con 0x7fdb9406dfd0 2024-09-12T03:20:19.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:19.106+0000 7fdb9bca5640 1 -- 172.21.15.27:0/230845959 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fdb78038230 msgr2=0x7fdb7803a6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:19.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:19.106+0000 7fdb9bca5640 1 --2- 172.21.15.27:0/230845959 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fdb78038230 0x7fdb7803a6f0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fdb9415ddf0 tx=0x7fdb88007d50 comp rx=0 tx=0).stop 2024-09-12T03:20:19.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:19.106+0000 7fdb9bca5640 1 -- 172.21.15.27:0/230845959 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdb9406dfd0 msgr2=0x7fdb9415e210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:19.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:19.106+0000 7fdb9bca5640 1 --2- 172.21.15.27:0/230845959 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdb9406dfd0 0x7fdb9415e210 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fdb900097a0 tx=0x7fdb900046c0 comp rx=0 tx=0).stop 2024-09-12T03:20:19.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:19.107+0000 7fdb9bca5640 1 -- 172.21.15.27:0/230845959 shutdown_connections 2024-09-12T03:20:19.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:19.107+0000 7fdb9bca5640 1 --2- 172.21.15.27:0/230845959 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdb94162040 0x7fdb9415f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:19.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:19.107+0000 7fdb9bca5640 1 --2- 172.21.15.27:0/230845959 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fdb940d3790 0x7fdb94161320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:19.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:19.107+0000 7fdb9bca5640 1 --2- 172.21.15.27:0/230845959 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fdb78038230 0x7fdb7803a6f0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:19.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:19.107+0000 7fdb9bca5640 1 --2- 172.21.15.27:0/230845959 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdb9406dfd0 0x7fdb9415e210 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:19.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:19.107+0000 7fdb9bca5640 1 -- 172.21.15.27:0/230845959 >> 172.21.15.27:0/230845959 conn(0x7fdb940b6700 msgr2=0x7fdb940b8700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:19.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:19.107+0000 7fdb9bca5640 1 -- 172.21.15.27:0/230845959 shutdown_connections 2024-09-12T03:20:19.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:19.107+0000 7fdb9bca5640 1 -- 172.21.15.27:0/230845959 wait complete. 2024-09-12T03:20:19.125 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-12T03:20:19.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:19.238+0000 7f2c9a623640 1 Processor -- start 2024-09-12T03:20:19.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:19.238+0000 7f2c9a623640 1 -- start start 2024-09-12T03:20:19.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:19.239+0000 7f2c9a623640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2c9415e650 0x7f2c940d1d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:19.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:19.239+0000 7f2c9a623640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2c9406e690 con 0x7f2c9406db30 2024-09-12T03:20:19.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:19.239+0000 7f2c9a623640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2c940d2280 con 0x7f2c9406cc40 2024-09-12T03:20:19.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:19.239+0000 7f2c9a623640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f2c9406d850 con 0x7f2c9415e650 2024-09-12T03:20:19.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:19.239+0000 7f2c98bc0640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f2c9406db30 0x7f2c9406df80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:50960/0 (socket says 172.21.15.27:50960) 2024-09-12T03:20:19.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:19.239+0000 7f2c98bc0640 1 -- 172.21.15.27:0/1214570104 learned_addr learned my addr 172.21.15.27:0/1214570104 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:20:19.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:19.240+0000 7f2c93fff640 1 --2- 172.21.15.27:0/1214570104 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2c9415e650 0x7f2c940d1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:19.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:19.240+0000 7f2c92ffd640 1 -- 172.21.15.27:0/1214570104 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4063747366 0 0) 0x7f2c9406e690 con 0x7f2c9406db30 2024-09-12T03:20:19.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:19.240+0000 7f2c92ffd640 1 -- 172.21.15.27:0/1214570104 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2c7c003520 con 0x7f2c9406db30 2024-09-12T03:20:19.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:19.240+0000 7f2c92ffd640 1 -- 172.21.15.27:0/1214570104 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 366819753 0 0) 0x7f2c7c003520 con 0x7f2c9406db30 2024-09-12T03:20:19.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:19.240+0000 7f2c92ffd640 1 -- 172.21.15.27:0/1214570104 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2c9415e650 msgr2=0x7f2c940d1d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:19.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:19.240+0000 7f2c92ffd640 1 --2- 172.21.15.27:0/1214570104 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2c9415e650 0x7f2c940d1d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:19.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:19.240+0000 7f2c92ffd640 1 -- 172.21.15.27:0/1214570104 >> v1:172.21.15.45:6789/0 conn(0x7f2c9406cc40 legacy=0x7f2c9406d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:19.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:19.240+0000 7f2c92ffd640 1 -- 172.21.15.27:0/1214570104 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c940d1260 con 0x7f2c9406db30 2024-09-12T03:20:19.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:19.240+0000 7f2c92ffd640 1 -- 172.21.15.27:0/1214570104 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 847755213 0 0) 0x7f2c8c001a10 con 0x7f2c9406db30 2024-09-12T03:20:19.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:19.240+0000 7f2c92ffd640 1 -- 172.21.15.27:0/1214570104 >> v1:172.21.15.27:6789/0 conn(0x7f2c9406db30 legacy=0x7f2c9406df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:19.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:19.240+0000 7f2c92ffd640 1 --2- 172.21.15.27:0/1214570104 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2c7c0039e0 0x7f2c7c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:19.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:19.240+0000 7f2c92ffd640 1 --2- 172.21.15.27:0/1214570104 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2c7c004800 0x7f2c7c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:19.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:19.240+0000 7f2c92ffd640 1 --2- 172.21.15.27:0/1214570104 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2c9415e650 0x7f2c7c005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:19.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:19.240+0000 7f2c92ffd640 1 -- 172.21.15.27:0/1214570104 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f2c9406d850 con 0x7f2c7c0039e0 2024-09-12T03:20:19.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:19.240+0000 7f2c92ffd640 1 -- 172.21.15.27:0/1214570104 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f2c7c00baf0 con 0x7f2c7c004800 2024-09-12T03:20:19.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:19.240+0000 7f2c92ffd640 1 -- 172.21.15.27:0/1214570104 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f2c7c00bcf0 con 0x7f2c9415e650 2024-09-12T03:20:19.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:19.240+0000 7f2c93fff640 1 --2- 172.21.15.27:0/1214570104 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2c7c0039e0 0x7f2c7c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:19.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:19.240+0000 7f2c937fe640 1 --2- 172.21.15.27:0/1214570104 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2c7c004800 0x7f2c7c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:19.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:19.240+0000 7f2c98bc0640 1 --2- 172.21.15.27:0/1214570104 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2c9415e650 0x7f2c7c005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:19.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:19.240+0000 7f2c93fff640 1 -- 172.21.15.27:0/1214570104 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2c9415e650 msgr2=0x7f2c7c005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:19.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:19.240+0000 7f2c93fff640 1 --2- 172.21.15.27:0/1214570104 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2c9415e650 0x7f2c7c005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:19.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:19.240+0000 7f2c93fff640 1 -- 172.21.15.27:0/1214570104 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2c7c004800 msgr2=0x7f2c7c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:19.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:19.240+0000 7f2c93fff640 1 --2- 172.21.15.27:0/1214570104 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2c7c004800 0x7f2c7c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:19.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:19.240+0000 7f2c93fff640 1 -- 172.21.15.27:0/1214570104 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c7c003520 con 0x7f2c7c0039e0 2024-09-12T03:20:19.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:19.240+0000 7f2c937fe640 1 --2- 172.21.15.27:0/1214570104 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2c7c004800 0x7f2c7c004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:20:19.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:19.240+0000 7f2c98bc0640 1 --2- 172.21.15.27:0/1214570104 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2c9415e650 0x7f2c7c005200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:20:19.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:19.241+0000 7f2c93fff640 1 --2- 172.21.15.27:0/1214570104 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2c7c0039e0 0x7f2c7c003e00 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f2c84009ac0 tx=0x7f2c7c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2816101ee0a1327 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:19.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:19.241+0000 7f2c92ffd640 1 -- 172.21.15.27:0/1214570104 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c84036070 con 0x7f2c7c0039e0 2024-09-12T03:20:19.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:19.241+0000 7f2c92ffd640 1 -- 172.21.15.27:0/1214570104 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c84031400 con 0x7f2c7c0039e0 2024-09-12T03:20:19.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:19.241+0000 7f2c92ffd640 1 -- 172.21.15.27:0/1214570104 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c84009690 con 0x7f2c7c0039e0 2024-09-12T03:20:19.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:19.242+0000 7f2c9a623640 1 -- 172.21.15.27:0/1214570104 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2c7c0039e0 msgr2=0x7f2c7c003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:19.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:19.242+0000 7f2c9a623640 1 --2- 172.21.15.27:0/1214570104 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2c7c0039e0 0x7f2c7c003e00 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f2c84009ac0 tx=0x7f2c7c005830 comp rx=0 tx=0).stop 2024-09-12T03:20:19.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:19.242+0000 7f2c98bc0640 1 -- 172.21.15.27:0/1214570104 reap_dead start 2024-09-12T03:20:19.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:19.242+0000 7f2c9a623640 1 -- 172.21.15.27:0/1214570104 shutdown_connections 2024-09-12T03:20:19.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:19.242+0000 7f2c9a623640 1 -- 172.21.15.27:0/1214570104 >> 172.21.15.27:0/1214570104 conn(0x7f2c9406f320 msgr2=0x7f2c9406f700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:19.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:19.242+0000 7f2c9a623640 1 -- 172.21.15.27:0/1214570104 shutdown_connections 2024-09-12T03:20:19.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:19.242+0000 7f2c9a623640 1 -- 172.21.15.27:0/1214570104 wait complete. 2024-09-12T03:20:19.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:19.243+0000 7f2c9a623640 1 Processor -- start 2024-09-12T03:20:19.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:19.243+0000 7f2c9a623640 1 -- start start 2024-09-12T03:20:19.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:19.244+0000 7f2c9a623640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2c7c0039e0 0x7f2c94147250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:19.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:19.244+0000 7f2c9a623640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2c941e8100 0x7f2c941e8580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:19.248 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:19.244+0000 7f2c9a623640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2c94160480 0x7f2c94160900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:19.248 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:19.244+0000 7f2c9a623640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f2c9415ebb0 con 0x7f2c7c0039e0 2024-09-12T03:20:19.248 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:19.244+0000 7f2c9a623640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f2c9415ea30 con 0x7f2c94160480 2024-09-12T03:20:19.248 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:19.244+0000 7f2c9a623640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f2c940d1d40 con 0x7f2c941e8100 2024-09-12T03:20:19.248 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:19.244+0000 7f2c93fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2c7c0039e0 0x7f2c94147250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:19.248 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:19.244+0000 7f2c93fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2c7c0039e0 0x7f2c94147250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58180/0 (socket says 172.21.15.27:58180) 2024-09-12T03:20:19.248 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:19.244+0000 7f2c93fff640 1 -- 172.21.15.27:0/3065093078 learned_addr learned my addr 172.21.15.27:0/3065093078 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:20:19.248 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:19.244+0000 7f2c98bc0640 1 --2- 172.21.15.27:0/3065093078 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2c94160480 0x7f2c94160900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:19.248 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:19.244+0000 7f2c93fff640 1 -- 172.21.15.27:0/3065093078 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2c941e8100 msgr2=0x7f2c941e8580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:19.248 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:19.244+0000 7f2c93fff640 1 --2- 172.21.15.27:0/3065093078 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2c941e8100 0x7f2c941e8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:19.248 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:19.244+0000 7f2c93fff640 1 -- 172.21.15.27:0/3065093078 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2c94160480 msgr2=0x7f2c94160900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:19.248 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:19.244+0000 7f2c93fff640 1 --2- 172.21.15.27:0/3065093078 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2c94160480 0x7f2c94160900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:19.249 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:19.244+0000 7f2c93fff640 1 -- 172.21.15.27:0/3065093078 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c94157ec0 con 0x7f2c7c0039e0 2024-09-12T03:20:19.249 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:19.245+0000 7f2c93fff640 1 --2- 172.21.15.27:0/3065093078 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2c7c0039e0 0x7f2c94147250 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f2c84009be0 tx=0x7f2c84004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:19.249 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:19.245+0000 7f2c90ff9640 1 -- 172.21.15.27:0/3065093078 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c84040020 con 0x7f2c7c0039e0 2024-09-12T03:20:19.249 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:19.245+0000 7f2c90ff9640 1 -- 172.21.15.27:0/3065093078 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c84003bc0 con 0x7f2c7c0039e0 2024-09-12T03:20:19.249 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:19.245+0000 7f2c90ff9640 1 -- 172.21.15.27:0/3065093078 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c84036040 con 0x7f2c7c0039e0 2024-09-12T03:20:19.249 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:19.245+0000 7f2c9a623640 1 -- 172.21.15.27:0/3065093078 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c941581a0 con 0x7f2c7c0039e0 2024-09-12T03:20:19.249 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:19.245+0000 7f2c9a623640 1 -- 172.21.15.27:0/3065093078 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c94158760 con 0x7f2c7c0039e0 2024-09-12T03:20:19.249 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:19.246+0000 7f2c9a623640 1 -- 172.21.15.27:0/3065093078 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c9406d060 con 0x7f2c7c0039e0 2024-09-12T03:20:19.249 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:19.246+0000 7f2c90ff9640 1 -- 172.21.15.27:0/3065093078 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 90600+0+0 (secure 0 0 0) 0x7f2c84043020 con 0x7f2c7c0039e0 2024-09-12T03:20:19.249 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:19.247+0000 7f2c90ff9640 1 --2- 172.21.15.27:0/3065093078 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f2c8006bf80 0x7f2c8006e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:19.249 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:19.247+0000 7f2c937fe640 1 --2- 172.21.15.27:0/3065093078 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f2c8006bf80 0x7f2c8006e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:19.249 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:19.247+0000 7f2c90ff9640 1 -- 172.21.15.27:0/3065093078 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 6615+0+0 (secure 0 0 0) 0x7f2c840aade0 con 0x7f2c7c0039e0 2024-09-12T03:20:19.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:19.247+0000 7f2c937fe640 1 --2- 172.21.15.27:0/3065093078 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f2c8006bf80 0x7f2c8006e440 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f2c88007aa0 tx=0x7f2c88009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:19.251 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:19.251+0000 7f2c90ff9640 1 -- 172.21.15.27:0/3065093078 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f2c840044e0 con 0x7f2c7c0039e0 2024-09-12T03:20:19.427 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:19.427+0000 7f2c9a623640 1 -- 172.21.15.27:0/3065093078 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f2c941e7cb0 con 0x7f2c7c0039e0 2024-09-12T03:20:20.109 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:20.109+0000 7f2c90ff9640 1 -- 172.21.15.27:0/3065093078 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v72) ==== 181+0+0 (secure 0 0 0) 0x7f2c840792f0 con 0x7f2c7c0039e0 2024-09-12T03:20:20.110 INFO:teuthology.orchestra.run.smithi027.stderr:pool 'cephfs_metadata' removed 2024-09-12T03:20:20.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:20.114+0000 7f2c9a623640 1 -- 172.21.15.27:0/3065093078 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f2c8006bf80 msgr2=0x7f2c8006e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:20.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:20.114+0000 7f2c9a623640 1 --2- 172.21.15.27:0/3065093078 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f2c8006bf80 0x7f2c8006e440 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f2c88007aa0 tx=0x7f2c88009040 comp rx=0 tx=0).stop 2024-09-12T03:20:20.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:20.114+0000 7f2c9a623640 1 -- 172.21.15.27:0/3065093078 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2c7c0039e0 msgr2=0x7f2c94147250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:20.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:20.114+0000 7f2c9a623640 1 --2- 172.21.15.27:0/3065093078 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2c7c0039e0 0x7f2c94147250 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f2c84009be0 tx=0x7f2c84004730 comp rx=0 tx=0).stop 2024-09-12T03:20:20.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:20.114+0000 7f2c9a623640 1 -- 172.21.15.27:0/3065093078 shutdown_connections 2024-09-12T03:20:20.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:20.114+0000 7f2c9a623640 1 --2- 172.21.15.27:0/3065093078 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2c94160480 0x7f2c94160900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:20.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:20.114+0000 7f2c9a623640 1 --2- 172.21.15.27:0/3065093078 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2c941e8100 0x7f2c941e8580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:20.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:20.114+0000 7f2c9a623640 1 --2- 172.21.15.27:0/3065093078 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f2c8006bf80 0x7f2c8006e440 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:20.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:20.114+0000 7f2c9a623640 1 --2- 172.21.15.27:0/3065093078 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2c7c0039e0 0x7f2c94147250 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:20.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:20.114+0000 7f2c9a623640 1 -- 172.21.15.27:0/3065093078 >> 172.21.15.27:0/3065093078 conn(0x7f2c9406f320 msgr2=0x7f2c9406f700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:20.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:20.115+0000 7f2c9a623640 1 -- 172.21.15.27:0/3065093078 shutdown_connections 2024-09-12T03:20:20.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:20.115+0000 7f2c9a623640 1 -- 172.21.15.27:0/3065093078 wait complete. 2024-09-12T03:20:20.129 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-12T03:20:20.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:20.243+0000 7f58b941e640 1 Processor -- start 2024-09-12T03:20:20.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:20.243+0000 7f58b941e640 1 -- start start 2024-09-12T03:20:20.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:20.244+0000 7f58b941e640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f58b40d0b50 0x7f58b4169310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:20.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:20.244+0000 7f58b941e640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f58b40cf9f0 con 0x7f58b4070d30 2024-09-12T03:20:20.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:20.244+0000 7f58b941e640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f58b40cfc90 con 0x7f58b40d31c0 2024-09-12T03:20:20.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:20.244+0000 7f58b941e640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f58b40d1a40 con 0x7f58b40d0b50 2024-09-12T03:20:20.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:20.244+0000 7f58b2ffd640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f58b4070d30 0x7f58b406cc40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:33950/0 (socket says 172.21.15.27:33950) 2024-09-12T03:20:20.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:20.244+0000 7f58b2ffd640 1 -- 172.21.15.27:0/619088894 learned_addr learned my addr 172.21.15.27:0/619088894 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:20:20.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:20.244+0000 7f58b37fe640 1 --2- 172.21.15.27:0/619088894 >> v2:172.21.15.45:3300/0 conn(0x7f58b40d0b50 0x7f58b4169310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:20.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:20.244+0000 7f58b27fc640 1 -- 172.21.15.27:0/619088894 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1516921201 0 0) 0x7f58b40cf9f0 con 0x7f58b4070d30 2024-09-12T03:20:20.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:20.244+0000 7f58b27fc640 1 -- 172.21.15.27:0/619088894 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f58a0003520 con 0x7f58b4070d30 2024-09-12T03:20:20.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:20.245+0000 7f58b37fe640 1 -- 172.21.15.27:0/619088894 >> v1:172.21.15.45:6789/0 conn(0x7f58b40d31c0 legacy=0x7f58b40d0440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:20.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:20.245+0000 7f58b37fe640 1 -- 172.21.15.27:0/619088894 >> v1:172.21.15.27:6789/0 conn(0x7f58b4070d30 legacy=0x7f58b406cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:20.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:20.245+0000 7f58b37fe640 1 -- 172.21.15.27:0/619088894 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58b40cff70 con 0x7f58b40d0b50 2024-09-12T03:20:20.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:20.246+0000 7f58b37fe640 1 --2- 172.21.15.27:0/619088894 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f58b40d0b50 0x7f58b4169310 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f58ac009d90 tx=0x7f58ac02a5d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7be7aa585d244e6c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:20.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:20.246+0000 7f58b27fc640 1 -- 172.21.15.27:0/619088894 <== mon.1 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58ac031600 con 0x7f58b40d0b50 2024-09-12T03:20:20.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:20.246+0000 7f58b27fc640 1 -- 172.21.15.27:0/619088894 <== mon.1 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58ac0317a0 con 0x7f58b40d0b50 2024-09-12T03:20:20.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:20.246+0000 7f58b27fc640 1 -- 172.21.15.27:0/619088894 <== mon.1 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58ac031b70 con 0x7f58b40d0b50 2024-09-12T03:20:20.248 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:20.247+0000 7f58b941e640 1 -- 172.21.15.27:0/619088894 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f58b40d0b50 msgr2=0x7f58b4169310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:20.248 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:20.247+0000 7f58b941e640 1 --2- 172.21.15.27:0/619088894 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f58b40d0b50 0x7f58b4169310 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f58ac009d90 tx=0x7f58ac02a5d0 comp rx=0 tx=0).stop 2024-09-12T03:20:20.248 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:20.247+0000 7f58b941e640 1 -- 172.21.15.27:0/619088894 shutdown_connections 2024-09-12T03:20:20.248 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:20.247+0000 7f58b941e640 1 --2- 172.21.15.27:0/619088894 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f58b40d0b50 0x7f58b4169310 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:20.248 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:20.247+0000 7f58b941e640 1 -- 172.21.15.27:0/619088894 >> 172.21.15.27:0/619088894 conn(0x7f58b40b7c10 msgr2=0x7f58b409ba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:20.248 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:20.247+0000 7f58b941e640 1 -- 172.21.15.27:0/619088894 shutdown_connections 2024-09-12T03:20:20.248 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:20.248+0000 7f58b941e640 1 -- 172.21.15.27:0/619088894 wait complete. 2024-09-12T03:20:20.249 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:20.249+0000 7f58b941e640 1 Processor -- start 2024-09-12T03:20:20.249 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:20.249+0000 7f58b941e640 1 -- start start 2024-09-12T03:20:20.251 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:20.250+0000 7f58b941e640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f58b4070d30 0x7f58b4151140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:20.251 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:20.250+0000 7f58b941e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f58b40d0b50 0x7f58b415ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:20.251 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:20.250+0000 7f58b941e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f58b40d31c0 0x7f58b41576a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:20.251 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:20.250+0000 7f58b941e640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f58b406eae0 con 0x7f58b40d31c0 2024-09-12T03:20:20.251 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:20.250+0000 7f58b941e640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f58b406e960 con 0x7f58b40d0b50 2024-09-12T03:20:20.251 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:20.250+0000 7f58b941e640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f58b4169850 con 0x7f58b4070d30 2024-09-12T03:20:20.251 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:20.250+0000 7f58b37fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f58b40d31c0 0x7f58b41576a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:20.251 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:20.250+0000 7f58b37fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f58b40d31c0 0x7f58b41576a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46688/0 (socket says 172.21.15.27:46688) 2024-09-12T03:20:20.251 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:20.250+0000 7f58b37fe640 1 -- 172.21.15.27:0/2654650635 learned_addr learned my addr 172.21.15.27:0/2654650635 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:20:20.251 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:20.250+0000 7f58aa7fc640 1 --2- 172.21.15.27:0/2654650635 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f58b40d0b50 0x7f58b415ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:20.251 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:20.250+0000 7f58b2ffd640 1 --2- 172.21.15.27:0/2654650635 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f58b4070d30 0x7f58b4151140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:20.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:20.250+0000 7f58b37fe640 1 -- 172.21.15.27:0/2654650635 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f58b4070d30 msgr2=0x7f58b4151140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:20.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:20.250+0000 7f58b37fe640 1 --2- 172.21.15.27:0/2654650635 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f58b4070d30 0x7f58b4151140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:20.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:20.250+0000 7f58b37fe640 1 -- 172.21.15.27:0/2654650635 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f58b40d0b50 msgr2=0x7f58b415ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:20.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:20.250+0000 7f58b37fe640 1 --2- 172.21.15.27:0/2654650635 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f58b40d0b50 0x7f58b415ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:20.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:20.250+0000 7f58b37fe640 1 -- 172.21.15.27:0/2654650635 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58b40b2370 con 0x7f58b40d31c0 2024-09-12T03:20:20.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:20.251+0000 7f58b37fe640 1 --2- 172.21.15.27:0/2654650635 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f58b40d31c0 0x7f58b41576a0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f58ac009eb0 tx=0x7f58ac004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:20.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:20.251+0000 7f58abfff640 1 -- 172.21.15.27:0/2654650635 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58ac040070 con 0x7f58b40d31c0 2024-09-12T03:20:20.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:20.251+0000 7f58abfff640 1 -- 172.21.15.27:0/2654650635 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58ac030b60 con 0x7f58b40d31c0 2024-09-12T03:20:20.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:20.251+0000 7f58b941e640 1 -- 172.21.15.27:0/2654650635 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58b40b2650 con 0x7f58b40d31c0 2024-09-12T03:20:20.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:20.251+0000 7f58abfff640 1 -- 172.21.15.27:0/2654650635 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58ac009450 con 0x7f58b40d31c0 2024-09-12T03:20:20.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:20.251+0000 7f58b941e640 1 -- 172.21.15.27:0/2654650635 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58b40b2c10 con 0x7f58b40d31c0 2024-09-12T03:20:20.253 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:20.252+0000 7f58b941e640 1 -- 172.21.15.27:0/2654650635 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58b406cc40 con 0x7f58b40d31c0 2024-09-12T03:20:20.253 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:20.253+0000 7f58abfff640 1 -- 172.21.15.27:0/2654650635 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 90600+0+0 (secure 0 0 0) 0x7f58ac039040 con 0x7f58b40d31c0 2024-09-12T03:20:20.253 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:20.253+0000 7f58abfff640 1 --2- 172.21.15.27:0/2654650635 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f589406bf50 0x7f589406e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:20.253 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:20.253+0000 7f58b2ffd640 1 --2- 172.21.15.27:0/2654650635 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f589406bf50 0x7f589406e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:20.253 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:20.253+0000 7f58abfff640 1 -- 172.21.15.27:0/2654650635 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 6177+0+0 (secure 0 0 0) 0x7f58ac0ac7b0 con 0x7f58b40d31c0 2024-09-12T03:20:20.254 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:20.254+0000 7f58b2ffd640 1 --2- 172.21.15.27:0/2654650635 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f589406bf50 0x7f589406e410 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f58a40011e0 tx=0x7f58a4007d30 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:20.257 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:20.258+0000 7f58abfff640 1 -- 172.21.15.27:0/2654650635 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f58ac07a720 con 0x7f58b40d31c0 2024-09-12T03:20:20.431 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:20.431+0000 7f58b941e640 1 -- 172.21.15.27:0/2654650635 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f58b40d1230 con 0x7f58b40d31c0 2024-09-12T03:20:21.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:21.111+0000 7f58abfff640 1 -- 172.21.15.27:0/2654650635 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v73) ==== 169+0+0 (secure 0 0 0) 0x7f58ac07ea20 con 0x7f58b40d31c0 2024-09-12T03:20:21.111 INFO:teuthology.orchestra.run.smithi027.stderr:pool 'cephfs_data' removed 2024-09-12T03:20:21.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:21.113+0000 7f58b941e640 1 -- 172.21.15.27:0/2654650635 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f589406bf50 msgr2=0x7f589406e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:21.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:21.113+0000 7f58b941e640 1 --2- 172.21.15.27:0/2654650635 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f589406bf50 0x7f589406e410 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f58a40011e0 tx=0x7f58a4007d30 comp rx=0 tx=0).stop 2024-09-12T03:20:21.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:21.113+0000 7f58b941e640 1 -- 172.21.15.27:0/2654650635 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f58b40d31c0 msgr2=0x7f58b41576a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:21.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:21.113+0000 7f58b941e640 1 --2- 172.21.15.27:0/2654650635 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f58b40d31c0 0x7f58b41576a0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f58ac009eb0 tx=0x7f58ac004a40 comp rx=0 tx=0).stop 2024-09-12T03:20:21.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:21.114+0000 7f58b941e640 1 -- 172.21.15.27:0/2654650635 shutdown_connections 2024-09-12T03:20:21.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:21.114+0000 7f58b941e640 1 --2- 172.21.15.27:0/2654650635 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f589406bf50 0x7f589406e410 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:21.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:21.114+0000 7f58b941e640 1 --2- 172.21.15.27:0/2654650635 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f58b40d31c0 0x7f58b41576a0 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:21.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:21.114+0000 7f58b941e640 1 --2- 172.21.15.27:0/2654650635 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f58b40d0b50 0x7f58b415ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:21.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:21.114+0000 7f58b941e640 1 --2- 172.21.15.27:0/2654650635 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f58b4070d30 0x7f58b4151140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:21.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:21.114+0000 7f58b941e640 1 -- 172.21.15.27:0/2654650635 >> 172.21.15.27:0/2654650635 conn(0x7f58b40b7c10 msgr2=0x7f58b415e190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:21.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:21.114+0000 7f58b941e640 1 -- 172.21.15.27:0/2654650635 shutdown_connections 2024-09-12T03:20:21.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:21.114+0000 7f58b941e640 1 -- 172.21.15.27:0/2654650635 wait complete. 2024-09-12T03:20:21.131 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-09-12T03:20:21.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:21.242+0000 7f2082ca6640 1 Processor -- start 2024-09-12T03:20:21.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:21.242+0000 7f2082ca6640 1 -- start start 2024-09-12T03:20:21.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:21.243+0000 7f2082ca6640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f207c15ec30 0x7f207c0b3900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:21.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:21.243+0000 7f2082ca6640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f207c06da00 0x7f207c06deb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:21.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:21.243+0000 7f2082ca6640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f207c0ce5e0 con 0x7f207c0af9d0 2024-09-12T03:20:21.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:21.243+0000 7f2082ca6640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f207c0d02d0 con 0x7f207c06da00 2024-09-12T03:20:21.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:21.243+0000 7f2082ca6640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f207c06c650 con 0x7f207c15ec30 2024-09-12T03:20:21.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:21.243+0000 7f2081243640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f207c06da00 0x7f207c06deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:21.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:21.243+0000 7f207bfff640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f207c0af9d0 0x7f207c0b3e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:33954/0 (socket says 172.21.15.27:33954) 2024-09-12T03:20:21.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:21.243+0000 7f2081243640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f207c06da00 0x7f207c06deb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46704/0 (socket says 172.21.15.27:46704) 2024-09-12T03:20:21.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:21.243+0000 7f207bfff640 1 -- 172.21.15.27:0/3381469284 learned_addr learned my addr 172.21.15.27:0/3381469284 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:20:21.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:21.243+0000 7f2080a42640 1 --2- 172.21.15.27:0/3381469284 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f207c15ec30 0x7f207c0b3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:21.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:21.243+0000 7f2081243640 1 -- 172.21.15.27:0/3381469284 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f207c15ec30 msgr2=0x7f207c0b3900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:21.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:21.243+0000 7f2081243640 1 --2- 172.21.15.27:0/3381469284 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f207c15ec30 0x7f207c0b3900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:21.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:21.243+0000 7f2081243640 1 -- 172.21.15.27:0/3381469284 >> v1:172.21.15.27:6789/0 conn(0x7f207c0af9d0 legacy=0x7f207c0b3e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:21.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:21.243+0000 7f2081243640 1 -- 172.21.15.27:0/3381469284 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f207c0ceac0 con 0x7f207c06da00 2024-09-12T03:20:21.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:21.243+0000 7f207b7fe640 1 -- 172.21.15.27:0/3381469284 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3980287464 0 0) 0x7f207c0ce5e0 con 0x7f207c0af9d0 2024-09-12T03:20:21.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:21.243+0000 7f2080a42640 1 --2- 172.21.15.27:0/3381469284 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f207c15ec30 0x7f207c0b3900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:20:21.248 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:21.244+0000 7f2081243640 1 --2- 172.21.15.27:0/3381469284 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f207c06da00 0x7f207c06deb0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f2074009d90 tx=0x7f207402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff7ed4dfc19d494 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:21.248 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:21.244+0000 7f207b7fe640 1 -- 172.21.15.27:0/3381469284 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2074007740 con 0x7f207c06da00 2024-09-12T03:20:21.249 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:21.244+0000 7f207b7fe640 1 -- 172.21.15.27:0/3381469284 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20740078e0 con 0x7f207c06da00 2024-09-12T03:20:21.249 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:21.244+0000 7f207b7fe640 1 -- 172.21.15.27:0/3381469284 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2074007cb0 con 0x7f207c06da00 2024-09-12T03:20:21.249 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:21.245+0000 7f2082ca6640 1 -- 172.21.15.27:0/3381469284 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f207c06da00 msgr2=0x7f207c06deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:21.249 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:21.245+0000 7f2082ca6640 1 --2- 172.21.15.27:0/3381469284 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f207c06da00 0x7f207c06deb0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f2074009d90 tx=0x7f207402a660 comp rx=0 tx=0).stop 2024-09-12T03:20:21.249 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:21.245+0000 7f2082ca6640 1 -- 172.21.15.27:0/3381469284 shutdown_connections 2024-09-12T03:20:21.249 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:21.245+0000 7f2082ca6640 1 --2- 172.21.15.27:0/3381469284 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f207c06da00 0x7f207c06deb0 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:21.249 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:21.245+0000 7f2082ca6640 1 --2- 172.21.15.27:0/3381469284 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f207c15ec30 0x7f207c0b3900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:21.249 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:21.245+0000 7f2082ca6640 1 -- 172.21.15.27:0/3381469284 >> 172.21.15.27:0/3381469284 conn(0x7f207c06bb30 msgr2=0x7f207c06bf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:21.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:21.245+0000 7f2082ca6640 1 -- 172.21.15.27:0/3381469284 shutdown_connections 2024-09-12T03:20:21.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:21.245+0000 7f2082ca6640 1 -- 172.21.15.27:0/3381469284 wait complete. 2024-09-12T03:20:21.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:21.246+0000 7f2082ca6640 1 Processor -- start 2024-09-12T03:20:21.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:21.246+0000 7f2082ca6640 1 -- start start 2024-09-12T03:20:21.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:21.247+0000 7f2082ca6640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f207c06da00 0x7f207c138d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:21.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:21.247+0000 7f2082ca6640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f207c0af9d0 0x7f207c06dde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:21.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:21.247+0000 7f2082ca6640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f207c15ec30 0x7f207c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:21.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:21.247+0000 7f2082ca6640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f207c0cf140 con 0x7f207c15ec30 2024-09-12T03:20:21.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:21.247+0000 7f2082ca6640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f207c0cefc0 con 0x7f207c0af9d0 2024-09-12T03:20:21.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:21.247+0000 7f2082ca6640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f207c0cf2c0 con 0x7f207c06da00 2024-09-12T03:20:21.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:21.247+0000 7f2081243640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f207c15ec30 0x7f207c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:21.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:21.247+0000 7f2081243640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f207c15ec30 0x7f207c06e320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46718/0 (socket says 172.21.15.27:46718) 2024-09-12T03:20:21.251 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:21.247+0000 7f2081243640 1 -- 172.21.15.27:0/1684885051 learned_addr learned my addr 172.21.15.27:0/1684885051 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:20:21.251 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:21.247+0000 7f2080a42640 1 --2- 172.21.15.27:0/1684885051 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f207c06da00 0x7f207c138d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:21.251 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:21.247+0000 7f207bfff640 1 --2- 172.21.15.27:0/1684885051 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f207c0af9d0 0x7f207c06dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:21.251 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:21.248+0000 7f2081243640 1 -- 172.21.15.27:0/1684885051 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f207c06da00 msgr2=0x7f207c138d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:21.251 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:21.248+0000 7f2081243640 1 --2- 172.21.15.27:0/1684885051 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f207c06da00 0x7f207c138d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:21.251 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:21.248+0000 7f2081243640 1 -- 172.21.15.27:0/1684885051 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f207c0af9d0 msgr2=0x7f207c06dde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:21.251 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:21.248+0000 7f2081243640 1 --2- 172.21.15.27:0/1684885051 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f207c0af9d0 0x7f207c06dde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:21.251 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:21.248+0000 7f2081243640 1 -- 172.21.15.27:0/1684885051 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f207c1558c0 con 0x7f207c15ec30 2024-09-12T03:20:21.251 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:21.248+0000 7f2081243640 1 --2- 172.21.15.27:0/1684885051 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f207c15ec30 0x7f207c06e320 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f2074009d90 tx=0x7f2074004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:21.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:21.248+0000 7f20797fa640 1 -- 172.21.15.27:0/1684885051 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2074040070 con 0x7f207c15ec30 2024-09-12T03:20:21.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:21.248+0000 7f20797fa640 1 -- 172.21.15.27:0/1684885051 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2074031c90 con 0x7f207c15ec30 2024-09-12T03:20:21.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:21.248+0000 7f2082ca6640 1 -- 172.21.15.27:0/1684885051 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f207c155b60 con 0x7f207c15ec30 2024-09-12T03:20:21.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:21.248+0000 7f20797fa640 1 -- 172.21.15.27:0/1684885051 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f207403c640 con 0x7f207c15ec30 2024-09-12T03:20:21.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:21.248+0000 7f2082ca6640 1 -- 172.21.15.27:0/1684885051 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f207c156120 con 0x7f207c15ec30 2024-09-12T03:20:21.253 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:21.249+0000 7f2082ca6640 1 -- 172.21.15.27:0/1684885051 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f207c0695a0 con 0x7f207c15ec30 2024-09-12T03:20:21.253 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:21.250+0000 7f20797fa640 1 -- 172.21.15.27:0/1684885051 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 90600+0+0 (secure 0 0 0) 0x7f2074039040 con 0x7f207c15ec30 2024-09-12T03:20:21.253 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:21.250+0000 7f20797fa640 1 --2- 172.21.15.27:0/1684885051 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f206006bf50 0x7f206006e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:21.253 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:21.250+0000 7f20797fa640 1 -- 172.21.15.27:0/1684885051 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 5502+0+0 (secure 0 0 0) 0x7f20740ac530 con 0x7f207c15ec30 2024-09-12T03:20:21.253 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:21.250+0000 7f2080a42640 1 --2- 172.21.15.27:0/1684885051 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f206006bf50 0x7f206006e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:21.253 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:21.250+0000 7f2080a42640 1 --2- 172.21.15.27:0/1684885051 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f206006bf50 0x7f206006e410 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f207000aac0 tx=0x7f2070008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:21.254 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:21.254+0000 7f20797fa640 1 -- 172.21.15.27:0/1684885051 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f207407a800 con 0x7f207c15ec30 2024-09-12T03:20:21.434 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:21.434+0000 7f2082ca6640 1 -- 172.21.15.27:0/1684885051 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f207c1663b0 con 0x7f207c15ec30 2024-09-12T03:20:21.437 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:21.437+0000 7f20797fa640 1 -- 172.21.15.27:0/1684885051 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v17) ==== 104+0+0 (secure 0 0 0) 0x7f207407eb00 con 0x7f207c15ec30 2024-09-12T03:20:21.440 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:21.439+0000 7f2082ca6640 1 -- 172.21.15.27:0/1684885051 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f206006bf50 msgr2=0x7f206006e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:21.440 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:21.439+0000 7f2082ca6640 1 --2- 172.21.15.27:0/1684885051 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f206006bf50 0x7f206006e410 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f207000aac0 tx=0x7f2070008040 comp rx=0 tx=0).stop 2024-09-12T03:20:21.440 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:21.439+0000 7f2082ca6640 1 -- 172.21.15.27:0/1684885051 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f207c15ec30 msgr2=0x7f207c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:21.440 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:21.439+0000 7f2082ca6640 1 --2- 172.21.15.27:0/1684885051 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f207c15ec30 0x7f207c06e320 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f2074009d90 tx=0x7f2074004a40 comp rx=0 tx=0).stop 2024-09-12T03:20:21.440 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:21.440+0000 7f2082ca6640 1 -- 172.21.15.27:0/1684885051 shutdown_connections 2024-09-12T03:20:21.440 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:21.440+0000 7f2082ca6640 1 --2- 172.21.15.27:0/1684885051 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f206006bf50 0x7f206006e410 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:21.440 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:21.440+0000 7f2082ca6640 1 --2- 172.21.15.27:0/1684885051 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f207c15ec30 0x7f207c06e320 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:21.440 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:21.440+0000 7f2082ca6640 1 --2- 172.21.15.27:0/1684885051 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f207c0af9d0 0x7f207c06dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:21.440 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:21.440+0000 7f2082ca6640 1 --2- 172.21.15.27:0/1684885051 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f207c06da00 0x7f207c138d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:21.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:21.440+0000 7f2082ca6640 1 -- 172.21.15.27:0/1684885051 >> 172.21.15.27:0/1684885051 conn(0x7f207c06bb30 msgr2=0x7f207c0cf7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:21.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:21.440+0000 7f2082ca6640 1 -- 172.21.15.27:0/1684885051 shutdown_connections 2024-09-12T03:20:21.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:21.440+0000 7f2082ca6640 1 -- 172.21.15.27:0/1684885051 wait complete. 2024-09-12T03:20:21.450 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestMisc.test_mgr_eviction' 2024-09-12T03:20:21.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:21.555+0000 7f6ae48ba640 1 Processor -- start 2024-09-12T03:20:21.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:21.555+0000 7f6ae48ba640 1 -- start start 2024-09-12T03:20:21.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:21.556+0000 7f6ae48ba640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f6ae00b1ca0 0x7f6ae015f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:21.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:21.556+0000 7f6ae48ba640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6ae015fd70 con 0x7f6ae00cf660 2024-09-12T03:20:21.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:21.556+0000 7f6ae48ba640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6ae006cc40 con 0x7f6ae006f360 2024-09-12T03:20:21.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:21.556+0000 7f6ae48ba640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f6ae00d22f0 con 0x7f6ae00b1ca0 2024-09-12T03:20:21.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:21.556+0000 7f6ade59c640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f6ae00b1ca0 0x7f6ae015f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:21.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:21.556+0000 7f6aded9d640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f6ae00cf660 0x7f6ae00d2780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:33958/0 (socket says 172.21.15.27:33958) 2024-09-12T03:20:21.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:21.556+0000 7f6aded9d640 1 -- 172.21.15.27:0/2346451146 learned_addr learned my addr 172.21.15.27:0/2346451146 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:20:21.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:21.556+0000 7f6ade59c640 1 -- 172.21.15.27:0/2346451146 >> v1:172.21.15.45:6789/0 conn(0x7f6ae006f360 legacy=0x7f6ae00cef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:21.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:21.556+0000 7f6add59a640 1 -- 172.21.15.27:0/2346451146 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 660933789 0 0) 0x7f6ad8001490 con 0x7f6ae00cf660 2024-09-12T03:20:21.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:21.556+0000 7f6ade59c640 1 -- 172.21.15.27:0/2346451146 >> v1:172.21.15.27:6789/0 conn(0x7f6ae00cf660 legacy=0x7f6ae00d2780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:21.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:21.556+0000 7f6ade59c640 1 -- 172.21.15.27:0/2346451146 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ae00d1ce0 con 0x7f6ae00b1ca0 2024-09-12T03:20:21.558 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:21.557+0000 7f6ade59c640 1 --2- 172.21.15.27:0/2346451146 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ae00b1ca0 0x7f6ae015f830 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f6ad00097a0 tx=0x7f6ad002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e60996aee6e49809 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:21.558 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:21.557+0000 7f6add59a640 1 -- 172.21.15.27:0/2346451146 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ad0037070 con 0x7f6ae00b1ca0 2024-09-12T03:20:21.558 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:21.557+0000 7f6add59a640 1 -- 172.21.15.27:0/2346451146 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ad0032440 con 0x7f6ae00b1ca0 2024-09-12T03:20:21.558 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:21.557+0000 7f6add59a640 1 -- 172.21.15.27:0/2346451146 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ad0032810 con 0x7f6ae00b1ca0 2024-09-12T03:20:21.558 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:21.558+0000 7f6ae48ba640 1 -- 172.21.15.27:0/2346451146 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ae00b1ca0 msgr2=0x7f6ae015f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:21.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:21.558+0000 7f6ae48ba640 1 --2- 172.21.15.27:0/2346451146 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ae00b1ca0 0x7f6ae015f830 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f6ad00097a0 tx=0x7f6ad002a1e0 comp rx=0 tx=0).stop 2024-09-12T03:20:21.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:21.558+0000 7f6ae48ba640 1 -- 172.21.15.27:0/2346451146 shutdown_connections 2024-09-12T03:20:21.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:21.558+0000 7f6ae48ba640 1 --2- 172.21.15.27:0/2346451146 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ae00b1ca0 0x7f6ae015f830 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:21.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:21.558+0000 7f6ae48ba640 1 -- 172.21.15.27:0/2346451146 >> 172.21.15.27:0/2346451146 conn(0x7f6ae0073800 msgr2=0x7f6ae009ba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:21.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:21.558+0000 7f6ae48ba640 1 -- 172.21.15.27:0/2346451146 shutdown_connections 2024-09-12T03:20:21.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:21.558+0000 7f6ae48ba640 1 -- 172.21.15.27:0/2346451146 wait complete. 2024-09-12T03:20:21.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:21.559+0000 7f6ae48ba640 1 Processor -- start 2024-09-12T03:20:21.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:21.559+0000 7f6ae48ba640 1 -- start start 2024-09-12T03:20:21.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:21.560+0000 7f6ae48ba640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6ae006f360 0x7f6ae0157580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:21.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:21.560+0000 7f6ae48ba640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ae00b1ca0 0x7f6ae0157d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:21.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:21.560+0000 7f6ae48ba640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f6ae00cf660 0x7f6ae01596f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:21.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:21.560+0000 7f6ae48ba640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f6ae006d1c0 con 0x7f6ae00b1ca0 2024-09-12T03:20:21.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:21.560+0000 7f6ae48ba640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f6ae00d1f80 con 0x7f6ae006f360 2024-09-12T03:20:21.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:21.560+0000 7f6ae48ba640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f6ae006d340 con 0x7f6ae00cf660 2024-09-12T03:20:21.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:21.560+0000 7f6addd9b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ae00b1ca0 0x7f6ae0157d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:21.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:21.560+0000 7f6addd9b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ae00b1ca0 0x7f6ae0157d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46730/0 (socket says 172.21.15.27:46730) 2024-09-12T03:20:21.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:21.560+0000 7f6addd9b640 1 -- 172.21.15.27:0/3505083584 learned_addr learned my addr 172.21.15.27:0/3505083584 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:20:21.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:21.560+0000 7f6ade59c640 1 --2- 172.21.15.27:0/3505083584 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6ae006f360 0x7f6ae0157580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:21.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:21.561+0000 7f6addd9b640 1 -- 172.21.15.27:0/3505083584 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f6ae00cf660 msgr2=0x7f6ae01596f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:21.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:21.561+0000 7f6addd9b640 1 --2- 172.21.15.27:0/3505083584 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f6ae00cf660 0x7f6ae01596f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:21.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:21.561+0000 7f6addd9b640 1 -- 172.21.15.27:0/3505083584 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6ae006f360 msgr2=0x7f6ae0157580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:21.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:21.561+0000 7f6addd9b640 1 --2- 172.21.15.27:0/3505083584 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6ae006f360 0x7f6ae0157580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:21.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:21.561+0000 7f6addd9b640 1 -- 172.21.15.27:0/3505083584 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ae0157ac0 con 0x7f6ae00b1ca0 2024-09-12T03:20:21.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:21.561+0000 7f6addd9b640 1 --2- 172.21.15.27:0/3505083584 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ae00b1ca0 0x7f6ae0157d10 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f6ad400c430 tx=0x7f6ad400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:21.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:21.561+0000 7f6aceffd640 1 -- 172.21.15.27:0/3505083584 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ad4011790 con 0x7f6ae00b1ca0 2024-09-12T03:20:21.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:21.561+0000 7f6aceffd640 1 -- 172.21.15.27:0/3505083584 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ad4017070 con 0x7f6ae00b1ca0 2024-09-12T03:20:21.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:21.561+0000 7f6ae48ba640 1 -- 172.21.15.27:0/3505083584 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ae00b70e0 con 0x7f6ae00b1ca0 2024-09-12T03:20:21.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:21.561+0000 7f6aceffd640 1 -- 172.21.15.27:0/3505083584 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ad4011bb0 con 0x7f6ae00b1ca0 2024-09-12T03:20:21.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:21.561+0000 7f6ae48ba640 1 -- 172.21.15.27:0/3505083584 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ae01f4c20 con 0x7f6ae00b1ca0 2024-09-12T03:20:21.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:21.562+0000 7f6ae48ba640 1 -- 172.21.15.27:0/3505083584 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ae015ea40 con 0x7f6ae00b1ca0 2024-09-12T03:20:21.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:21.562+0000 7f6aceffd640 1 -- 172.21.15.27:0/3505083584 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 90600+0+0 (secure 0 0 0) 0x7f6ad401c460 con 0x7f6ae00b1ca0 2024-09-12T03:20:21.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:21.563+0000 7f6aceffd640 1 --2- 172.21.15.27:0/3505083584 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f6ac006bf80 0x7f6ac006e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:21.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:21.563+0000 7f6aceffd640 1 -- 172.21.15.27:0/3505083584 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 5502+0+0 (secure 0 0 0) 0x7f6ad408add0 con 0x7f6ae00b1ca0 2024-09-12T03:20:21.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:21.563+0000 7f6ade59c640 1 --2- 172.21.15.27:0/3505083584 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f6ac006bf80 0x7f6ac006e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:21.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:21.563+0000 7f6ade59c640 1 --2- 172.21.15.27:0/3505083584 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f6ac006bf80 0x7f6ac006e440 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f6ad0034000 tx=0x7f6ad002e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:21.567 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:21.567+0000 7f6aceffd640 1 -- 172.21.15.27:0/3505083584 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f6ae015ea40 con 0x7f6ae00b1ca0 2024-09-12T03:20:21.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:21.814+0000 7f6ae48ba640 1 -- 172.21.15.27:0/3505083584 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_mgr_eviction"]} v 0) -- 0x7f6ae01f4e20 con 0x7f6ae00b1ca0 2024-09-12T03:20:22.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.112+0000 7f6aceffd640 1 -- 172.21.15.27:0/3505083584 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_mgr_eviction"]}]=0 v211) ==== 131+0+0 (secure 0 0 0) 0x7f6ae01f4e20 con 0x7f6ae00b1ca0 2024-09-12T03:20:22.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.115+0000 7f6ae48ba640 1 -- 172.21.15.27:0/3505083584 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f6ac006bf80 msgr2=0x7f6ac006e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:22.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.115+0000 7f6ae48ba640 1 --2- 172.21.15.27:0/3505083584 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f6ac006bf80 0x7f6ac006e440 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f6ad0034000 tx=0x7f6ad002e000 comp rx=0 tx=0).stop 2024-09-12T03:20:22.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.115+0000 7f6ae48ba640 1 -- 172.21.15.27:0/3505083584 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ae00b1ca0 msgr2=0x7f6ae0157d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:22.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.115+0000 7f6ae48ba640 1 --2- 172.21.15.27:0/3505083584 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ae00b1ca0 0x7f6ae0157d10 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f6ad400c430 tx=0x7f6ad400c8f0 comp rx=0 tx=0).stop 2024-09-12T03:20:22.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.115+0000 7f6ae48ba640 1 -- 172.21.15.27:0/3505083584 shutdown_connections 2024-09-12T03:20:22.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.115+0000 7f6ae48ba640 1 --2- 172.21.15.27:0/3505083584 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f6ae00cf660 0x7f6ae01596f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:22.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.115+0000 7f6ae48ba640 1 --2- 172.21.15.27:0/3505083584 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f6ac006bf80 0x7f6ac006e440 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:22.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.115+0000 7f6ae48ba640 1 --2- 172.21.15.27:0/3505083584 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ae00b1ca0 0x7f6ae0157d10 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:22.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.115+0000 7f6ae48ba640 1 --2- 172.21.15.27:0/3505083584 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6ae006f360 0x7f6ae0157580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:22.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.115+0000 7f6ae48ba640 1 -- 172.21.15.27:0/3505083584 >> 172.21.15.27:0/3505083584 conn(0x7f6ae0073800 msgr2=0x7f6ae0161290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:22.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.115+0000 7f6ae48ba640 1 -- 172.21.15.27:0/3505083584 shutdown_connections 2024-09-12T03:20:22.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.115+0000 7f6ae48ba640 1 -- 172.21.15.27:0/3505083584 wait complete. 2024-09-12T03:20:22.127 INFO:tasks.cephfs_test_runner:test_mgr_eviction (tasks.cephfs.test_volumes.TestMisc) ... ok 2024-09-12T03:20:22.127 INFO:tasks.cephfs_test_runner:Starting test: test_names_can_only_be_goodchars (tasks.cephfs.test_volumes.TestMisc) Test the creating vols, subvols subvolgroups fails when their names uses 2024-09-12T03:20:22.127 INFO:tasks.cephfs_test_runner:test_names_can_only_be_goodchars (tasks.cephfs.test_volumes.TestMisc) 2024-09-12T03:20:22.128 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-12T03:20:22.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.231+0000 7fdf20186640 1 Processor -- start 2024-09-12T03:20:22.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.232+0000 7fdf20186640 1 -- start start 2024-09-12T03:20:22.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.232+0000 7fdf20186640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fdf1815eb90 0x7fdf180d2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:22.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.232+0000 7fdf20186640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fdf180d5200 0x7fdf180d1010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:22.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.232+0000 7fdf20186640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdf180d19c0 con 0x7fdf180d1550 2024-09-12T03:20:22.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.232+0000 7fdf20186640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7fdf180b7d20 con 0x7fdf1815eb90 2024-09-12T03:20:22.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.232+0000 7fdf20186640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fdf180d42e0 con 0x7fdf180d5200 2024-09-12T03:20:22.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.233+0000 7fdf1df22640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fdf1815eb90 0x7fdf180d2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:22.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.233+0000 7fdf1df22640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fdf1815eb90 0x7fdf180d2df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46742/0 (socket says 172.21.15.27:46742) 2024-09-12T03:20:22.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.233+0000 7fdf1df22640 1 -- 172.21.15.27:0/1640526061 learned_addr learned my addr 172.21.15.27:0/1640526061 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:20:22.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.233+0000 7fdf1d721640 1 --2- 172.21.15.27:0/1640526061 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fdf180d5200 0x7fdf180d1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:22.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.233+0000 7fdf1df22640 1 -- 172.21.15.27:0/1640526061 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fdf180d5200 msgr2=0x7fdf180d1010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:22.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.233+0000 7fdf1df22640 1 --2- 172.21.15.27:0/1640526061 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fdf180d5200 0x7fdf180d1010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:22.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.233+0000 7fdf1df22640 1 -- 172.21.15.27:0/1640526061 >> v1:172.21.15.45:6789/0 conn(0x7fdf180d1550 legacy=0x7fdf180cf200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:22.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.233+0000 7fdf1df22640 1 -- 172.21.15.27:0/1640526061 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf180d3760 con 0x7fdf1815eb90 2024-09-12T03:20:22.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.234+0000 7fdf1df22640 1 --2- 172.21.15.27:0/1640526061 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdf1815eb90 0x7fdf180d2df0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fdf08009e50 tx=0x7fdf0802a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f05aa126daad52f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:22.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.234+0000 7fdf1cf20640 1 -- 172.21.15.27:0/1640526061 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf08036070 con 0x7fdf1815eb90 2024-09-12T03:20:22.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.234+0000 7fdf1cf20640 1 -- 172.21.15.27:0/1640526061 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf08031600 con 0x7fdf1815eb90 2024-09-12T03:20:22.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.234+0000 7fdf1cf20640 1 -- 172.21.15.27:0/1640526061 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf080319d0 con 0x7fdf1815eb90 2024-09-12T03:20:22.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.234+0000 7fdf20186640 1 -- 172.21.15.27:0/1640526061 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdf1815eb90 msgr2=0x7fdf180d2df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:22.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.234+0000 7fdf20186640 1 --2- 172.21.15.27:0/1640526061 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdf1815eb90 0x7fdf180d2df0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fdf08009e50 tx=0x7fdf0802a6c0 comp rx=0 tx=0).stop 2024-09-12T03:20:22.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.234+0000 7fdf20186640 1 -- 172.21.15.27:0/1640526061 shutdown_connections 2024-09-12T03:20:22.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.234+0000 7fdf20186640 1 --2- 172.21.15.27:0/1640526061 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fdf180d5200 0x7fdf180d1010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:22.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.234+0000 7fdf20186640 1 --2- 172.21.15.27:0/1640526061 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdf1815eb90 0x7fdf180d2df0 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:22.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.234+0000 7fdf20186640 1 -- 172.21.15.27:0/1640526061 >> 172.21.15.27:0/1640526061 conn(0x7fdf1806dce0 msgr2=0x7fdf1806e0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:22.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.234+0000 7fdf20186640 1 -- 172.21.15.27:0/1640526061 shutdown_connections 2024-09-12T03:20:22.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.234+0000 7fdf20186640 1 -- 172.21.15.27:0/1640526061 wait complete. 2024-09-12T03:20:22.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.236+0000 7fdf20186640 1 Processor -- start 2024-09-12T03:20:22.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.236+0000 7fdf20186640 1 -- start start 2024-09-12T03:20:22.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.236+0000 7fdf20186640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdf180d1550 0x7fdf180c9390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:22.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.236+0000 7fdf20186640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fdf180d5200 0x7fdf181f7470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:22.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.236+0000 7fdf20186640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdf1815eb90 0x7fdf18153790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:22.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.236+0000 7fdf20186640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fdf180d4000 con 0x7fdf180d1550 2024-09-12T03:20:22.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.236+0000 7fdf20186640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fdf180d3d20 con 0x7fdf1815eb90 2024-09-12T03:20:22.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.236+0000 7fdf20186640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fdf180d3330 con 0x7fdf180d5200 2024-09-12T03:20:22.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.237+0000 7fdf1df22640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdf180d1550 0x7fdf180c9390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:22.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.237+0000 7fdf1df22640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdf180d1550 0x7fdf180c9390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46754/0 (socket says 172.21.15.27:46754) 2024-09-12T03:20:22.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.237+0000 7fdf1df22640 1 -- 172.21.15.27:0/3488453551 learned_addr learned my addr 172.21.15.27:0/3488453551 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:20:22.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.237+0000 7fdf1e723640 1 --2- 172.21.15.27:0/3488453551 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdf1815eb90 0x7fdf18153790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:22.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.237+0000 7fdf1d721640 1 --2- 172.21.15.27:0/3488453551 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fdf180d5200 0x7fdf181f7470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:22.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.237+0000 7fdf1df22640 1 -- 172.21.15.27:0/3488453551 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fdf180d5200 msgr2=0x7fdf181f7470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:22.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.237+0000 7fdf1df22640 1 --2- 172.21.15.27:0/3488453551 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fdf180d5200 0x7fdf181f7470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:22.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.237+0000 7fdf1df22640 1 -- 172.21.15.27:0/3488453551 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdf1815eb90 msgr2=0x7fdf18153790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:22.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.237+0000 7fdf1df22640 1 --2- 172.21.15.27:0/3488453551 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdf1815eb90 0x7fdf18153790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:22.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.237+0000 7fdf1df22640 1 -- 172.21.15.27:0/3488453551 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf180c9970 con 0x7fdf180d1550 2024-09-12T03:20:22.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.237+0000 7fdf1d721640 1 --2- 172.21.15.27:0/3488453551 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fdf180d5200 0x7fdf181f7470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:20:22.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.237+0000 7fdf1e723640 1 --2- 172.21.15.27:0/3488453551 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdf1815eb90 0x7fdf18153790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:20:22.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.237+0000 7fdf1df22640 1 --2- 172.21.15.27:0/3488453551 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdf180d1550 0x7fdf180c9390 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fdf08034040 tx=0x7fdf08004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:22.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.237+0000 7fdf167fc640 1 -- 172.21.15.27:0/3488453551 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf08036050 con 0x7fdf180d1550 2024-09-12T03:20:22.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.237+0000 7fdf167fc640 1 -- 172.21.15.27:0/3488453551 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf08042070 con 0x7fdf180d1550 2024-09-12T03:20:22.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.237+0000 7fdf167fc640 1 -- 172.21.15.27:0/3488453551 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf08009510 con 0x7fdf180d1550 2024-09-12T03:20:22.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.237+0000 7fdf20186640 1 -- 172.21.15.27:0/3488453551 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf180c9c50 con 0x7fdf180d1550 2024-09-12T03:20:22.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.237+0000 7fdf20186640 1 -- 172.21.15.27:0/3488453551 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf180ca210 con 0x7fdf180d1550 2024-09-12T03:20:22.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.238+0000 7fdf20186640 1 -- 172.21.15.27:0/3488453551 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf1815ef70 con 0x7fdf180d1550 2024-09-12T03:20:22.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.239+0000 7fdf167fc640 1 -- 172.21.15.27:0/3488453551 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 90600+0+0 (secure 0 0 0) 0x7fdf08007c50 con 0x7fdf180d1550 2024-09-12T03:20:22.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.239+0000 7fdf167fc640 1 --2- 172.21.15.27:0/3488453551 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fdf0406bf50 0x7fdf0406e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:22.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.239+0000 7fdf167fc640 1 -- 172.21.15.27:0/3488453551 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 5502+0+0 (secure 0 0 0) 0x7fdf080ac060 con 0x7fdf180d1550 2024-09-12T03:20:22.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.239+0000 7fdf1d721640 1 --2- 172.21.15.27:0/3488453551 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fdf0406bf50 0x7fdf0406e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:22.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.240+0000 7fdf1d721640 1 --2- 172.21.15.27:0/3488453551 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fdf0406bf50 0x7fdf0406e410 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fdf180cb3b0 tx=0x7fdf0c008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:22.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.244+0000 7fdf167fc640 1 -- 172.21.15.27:0/3488453551 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fdf08045360 con 0x7fdf180d1550 2024-09-12T03:20:22.420 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.420+0000 7fdf20186640 1 -- 172.21.15.27:0/3488453551 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdf180b9340 con 0x7fdf180d1550 2024-09-12T03:20:22.421 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.421+0000 7fdf167fc640 1 -- 172.21.15.27:0/3488453551 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v73) ==== 74+0+13794 (secure 0 0 0) 0x7fdf0807a950 con 0x7fdf180d1550 2024-09-12T03:20:22.421 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:20:22.421 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":73,"fsid":"d2df504e-2647-4385-a5db-9fdf69f487a0","created":"2024-09-12T03:16:07.778937+0000","modified":"2024-09-12T03:20:21.110304+0000","last_up_change":"2024-09-12T03:16:19.314778+0000","last_in_change":"2024-09-12T03:16:09.893086+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T03:16:20.097646+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1851116c-1fb3-4dc9-9772-f90e2ef4c355","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6817","nonce":3002321488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6819","nonce":3002321488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6823","nonce":3002321488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6821","nonce":3002321488}]},"public_addr":"172.21.15.27:6817/3002321488","cluster_addr":"172.21.15.27:6819/3002321488","heartbeat_back_addr":"172.21.15.27:6823/3002321488","heartbeat_front_addr":"172.21.15.27:6821/3002321488","state":["exists","up"]},{"osd":1,"uuid":"f38ae17d-73eb-4204-b99d-975fa806fb87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6809","nonce":1613102886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6811","nonce":1613102886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6815","nonce":1613102886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6813","nonce":1613102886}]},"public_addr":"172.21.15.27:6809/1613102886","cluster_addr":"172.21.15.27:6811/1613102886","heartbeat_back_addr":"172.21.15.27:6815/1613102886","heartbeat_front_addr":"172.21.15.27:6813/1613102886","state":["exists","up"]},{"osd":2,"uuid":"fb12dcc8-bdd0-42b8-b46d-8b10106224bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6825","nonce":3534723355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6827","nonce":3534723355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6831","nonce":3534723355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6829","nonce":3534723355}]},"public_addr":"172.21.15.27:6825/3534723355","cluster_addr":"172.21.15.27:6827/3534723355","heartbeat_back_addr":"172.21.15.27:6831/3534723355","heartbeat_front_addr":"172.21.15.27:6829/3534723355","state":["exists","up"]},{"osd":3,"uuid":"d74d3f49-9c24-4b9f-98e2-3ecd7b5ef3b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6801","nonce":80661598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6803","nonce":80661598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6807","nonce":80661598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6805","nonce":80661598}]},"public_addr":"172.21.15.27:6801/80661598","cluster_addr":"172.21.15.27:6803/80661598","heartbeat_back_addr":"172.21.15.27:6807/80661598","heartbeat_front_addr":"172.21.15.27:6805/80661598","state":["exists","up"]},{"osd":4,"uuid":"2bf123f2-1b0d-41c4-aa3b-805bab267e59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6817","nonce":3943189827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6819","nonce":3943189827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6823","nonce":3943189827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6821","nonce":3943189827}]},"public_addr":"172.21.15.45:6817/3943189827","cluster_addr":"172.21.15.45:6819/3943189827","heartbeat_back_addr":"172.21.15.45:6823/3943189827","heartbeat_front_addr":"172.21.15.45:6821/3943189827","state":["exists","up"]},{"osd":5,"uuid":"c49ba5da-91c6-4a86-a97b-4aba89c253ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6809","nonce":3620760330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6811","nonce":3620760330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6815","nonce":3620760330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6813","nonce":3620760330}]},"public_addr":"172.21.15.45:6809/3620760330","cluster_addr":"172.21.15.45:6811/3620760330","heartbeat_back_addr":"172.21.15.45:6815/3620760330","heartbeat_front_addr":"172.21.15.45:6813/3620760330","state":["exists","up"]},{"osd":6,"uuid":"b0123475-89f7-4a34-9e33-d797b99dd9dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6801","nonce":1989083510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6803","nonce":1989083510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6807","nonce":1989083510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6805","nonce":1989083510}]},"public_addr":"172.21.15.45:6801/1989083510","cluster_addr":"172.21.15.45:6803/1989083510","heartbeat_back_addr":"172.21.15.45:6807/1989083510","heartbeat_front_addr":"172.21.15.45:6805/1989083510","state":["exists","up"]},{"osd":7,"uuid":"4dad773d-b7d8-46fc-8a10-a972aad27b29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6825","nonce":1513300294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6827","nonce":1513300294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6831","nonce":1513300294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6829","nonce":1513300294}]},"public_addr":"172.21.15.45:6825/1513300294","cluster_addr":"172.21.15.45:6827/1513300294","heartbeat_back_addr":"172.21.15.45:6831/1513300294","heartbeat_front_addr":"172.21.15.45:6829/1513300294","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.881855+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:17.133252+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.735339+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.500742+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.477571+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:17.167667+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.470291+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.873743+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:6837/2890007436":"2024-09-13T03:20:18.102011+0000","172.21.15.45:6835/2890007436":"2024-09-13T03:20:18.102011+0000","172.21.15.45:0/4018097172":"2024-09-13T03:20:15.074894+0000","172.21.15.45:0/687436879":"2024-09-13T03:20:15.074894+0000","172.21.15.45:0/3895125587":"2024-09-13T03:20:15.074894+0000","172.21.15.45:0/2632182829":"2024-09-13T03:20:15.074894+0000","172.21.15.45:0/3383141969":"2024-09-13T03:20:15.074894+0000","172.21.15.45:6833/3923821219":"2024-09-13T03:20:15.074894+0000","172.21.15.45:0/3205085313":"2024-09-13T03:20:15.074894+0000","172.21.15.45:0/3677899312":"2024-09-13T03:20:15.074894+0000","172.21.15.45:6832/3923821219":"2024-09-13T03:20:15.074894+0000","172.21.15.45:0/1945166272":"2024-09-13T03:20:15.074894+0000","172.21.15.45:0/761029391":"2024-09-13T03:20:15.074894+0000","172.21.15.45:0/471646388":"2024-09-13T03:20:15.074894+0000","172.21.15.45:0/4216551134":"2024-09-13T03:20:15.074894+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T03:20:22.423 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.423+0000 7fdf20186640 1 -- 172.21.15.27:0/3488453551 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fdf0406bf50 msgr2=0x7fdf0406e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:22.423 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.423+0000 7fdf20186640 1 --2- 172.21.15.27:0/3488453551 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fdf0406bf50 0x7fdf0406e410 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fdf180cb3b0 tx=0x7fdf0c008040 comp rx=0 tx=0).stop 2024-09-12T03:20:22.423 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.423+0000 7fdf20186640 1 -- 172.21.15.27:0/3488453551 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdf180d1550 msgr2=0x7fdf180c9390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:22.424 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.423+0000 7fdf20186640 1 --2- 172.21.15.27:0/3488453551 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdf180d1550 0x7fdf180c9390 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fdf08034040 tx=0x7fdf08004a40 comp rx=0 tx=0).stop 2024-09-12T03:20:22.424 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.423+0000 7fdf20186640 1 -- 172.21.15.27:0/3488453551 shutdown_connections 2024-09-12T03:20:22.424 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.423+0000 7fdf20186640 1 --2- 172.21.15.27:0/3488453551 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdf1815eb90 0x7fdf18153790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:22.424 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.423+0000 7fdf20186640 1 --2- 172.21.15.27:0/3488453551 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fdf180d5200 0x7fdf181f7470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:22.424 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.423+0000 7fdf20186640 1 --2- 172.21.15.27:0/3488453551 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fdf0406bf50 0x7fdf0406e410 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:22.424 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.423+0000 7fdf20186640 1 --2- 172.21.15.27:0/3488453551 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdf180d1550 0x7fdf180c9390 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:22.424 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.423+0000 7fdf20186640 1 -- 172.21.15.27:0/3488453551 >> 172.21.15.27:0/3488453551 conn(0x7fdf1806dce0 msgr2=0x7fdf1815f180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:22.424 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.423+0000 7fdf20186640 1 -- 172.21.15.27:0/3488453551 shutdown_connections 2024-09-12T03:20:22.424 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.423+0000 7fdf20186640 1 -- 172.21.15.27:0/3488453551 wait complete. 2024-09-12T03:20:22.434 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-12T03:16:20.097646+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-12T03:20:22.435 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-12T03:20:22.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.540+0000 7f0564c2e640 1 Processor -- start 2024-09-12T03:20:22.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.540+0000 7f0564c2e640 1 -- start start 2024-09-12T03:20:22.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.541+0000 7f0564c2e640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f056015e650 0x7f05600d1d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:22.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.541+0000 7f0564c2e640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f056006cc40 0x7f056006d060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:22.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.541+0000 7f0564c2e640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f05600d2280 con 0x7f056006db30 2024-09-12T03:20:22.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.541+0000 7f0564c2e640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f056006d5a0 con 0x7f056015e650 2024-09-12T03:20:22.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.541+0000 7f0564c2e640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f056006e670 con 0x7f056006cc40 2024-09-12T03:20:22.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.541+0000 7f055e59c640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f056015e650 0x7f05600d1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:22.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.541+0000 7f055e59c640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f056015e650 0x7f05600d1d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46762/0 (socket says 172.21.15.27:46762) 2024-09-12T03:20:22.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.541+0000 7f055e59c640 1 -- 172.21.15.27:0/2903267984 learned_addr learned my addr 172.21.15.27:0/2903267984 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:20:22.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.541+0000 7f055ed9d640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f056006db30 0x7f056006df60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:33966/0 (socket says 172.21.15.27:33966) 2024-09-12T03:20:22.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.541+0000 7f055dd9b640 1 --2- 172.21.15.27:0/2903267984 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f056006cc40 0x7f056006d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:22.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.541+0000 7f055e59c640 1 -- 172.21.15.27:0/2903267984 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f056006cc40 msgr2=0x7f056006d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:22.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.541+0000 7f055e59c640 1 --2- 172.21.15.27:0/2903267984 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f056006cc40 0x7f056006d060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:22.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.541+0000 7f055e59c640 1 -- 172.21.15.27:0/2903267984 >> v1:172.21.15.27:6789/0 conn(0x7f056006db30 legacy=0x7f056006df60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:22.543 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.542+0000 7f055d59a640 1 -- 172.21.15.27:0/2903267984 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2368400239 0 0) 0x7f05600d2280 con 0x7f056006db30 2024-09-12T03:20:22.543 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.542+0000 7f055e59c640 1 -- 172.21.15.27:0/2903267984 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f056006d850 con 0x7f056015e650 2024-09-12T03:20:22.544 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.542+0000 7f055e59c640 1 --2- 172.21.15.27:0/2903267984 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f056015e650 0x7f05600d1d40 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f0550009870 tx=0x7f055002a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=328b0a92c29afea3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:22.544 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.542+0000 7f055d59a640 1 -- 172.21.15.27:0/2903267984 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0550037070 con 0x7f056015e650 2024-09-12T03:20:22.544 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.542+0000 7f055d59a640 1 -- 172.21.15.27:0/2903267984 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0550032440 con 0x7f056015e650 2024-09-12T03:20:22.544 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.542+0000 7f055d59a640 1 -- 172.21.15.27:0/2903267984 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0550032810 con 0x7f056015e650 2024-09-12T03:20:22.544 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.543+0000 7f0564c2e640 1 -- 172.21.15.27:0/2903267984 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f056015e650 msgr2=0x7f05600d1d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:22.544 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.543+0000 7f0564c2e640 1 --2- 172.21.15.27:0/2903267984 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f056015e650 0x7f05600d1d40 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f0550009870 tx=0x7f055002a1c0 comp rx=0 tx=0).stop 2024-09-12T03:20:22.545 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.543+0000 7f0564c2e640 1 -- 172.21.15.27:0/2903267984 shutdown_connections 2024-09-12T03:20:22.545 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.543+0000 7f0564c2e640 1 --2- 172.21.15.27:0/2903267984 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f056006cc40 0x7f056006d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:22.545 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.543+0000 7f0564c2e640 1 --2- 172.21.15.27:0/2903267984 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f056015e650 0x7f05600d1d40 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:22.545 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.543+0000 7f0564c2e640 1 -- 172.21.15.27:0/2903267984 >> 172.21.15.27:0/2903267984 conn(0x7f056006f320 msgr2=0x7f056006f700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:22.545 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.543+0000 7f0564c2e640 1 -- 172.21.15.27:0/2903267984 shutdown_connections 2024-09-12T03:20:22.545 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.543+0000 7f0564c2e640 1 -- 172.21.15.27:0/2903267984 wait complete. 2024-09-12T03:20:22.545 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.544+0000 7f0564c2e640 1 Processor -- start 2024-09-12T03:20:22.545 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.544+0000 7f0564c2e640 1 -- start start 2024-09-12T03:20:22.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.545+0000 7f0564c2e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f056006cc40 0x7f05600cf5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:22.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.545+0000 7f0564c2e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f056006db30 0x7f05601f48e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:22.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.545+0000 7f0564c2e640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f056015e650 0x7f05600cc5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:22.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.545+0000 7f0564c2e640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f056015ea30 con 0x7f056006db30 2024-09-12T03:20:22.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.545+0000 7f0564c2e640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f05600d1540 con 0x7f056006cc40 2024-09-12T03:20:22.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.545+0000 7f0564c2e640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f056015ebb0 con 0x7f056015e650 2024-09-12T03:20:22.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.545+0000 7f055dd9b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f056006db30 0x7f05601f48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:22.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.545+0000 7f055dd9b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f056006db30 0x7f05601f48e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46764/0 (socket says 172.21.15.27:46764) 2024-09-12T03:20:22.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.545+0000 7f055dd9b640 1 -- 172.21.15.27:0/2436888736 learned_addr learned my addr 172.21.15.27:0/2436888736 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:20:22.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.545+0000 7f055ed9d640 1 --2- 172.21.15.27:0/2436888736 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f056015e650 0x7f05600cc5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:22.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.545+0000 7f055e59c640 1 --2- 172.21.15.27:0/2436888736 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f056006cc40 0x7f05600cf5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:22.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.546+0000 7f055dd9b640 1 -- 172.21.15.27:0/2436888736 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f056015e650 msgr2=0x7f05600cc5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:22.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.546+0000 7f055dd9b640 1 --2- 172.21.15.27:0/2436888736 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f056015e650 0x7f05600cc5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:22.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.546+0000 7f055dd9b640 1 -- 172.21.15.27:0/2436888736 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f056006cc40 msgr2=0x7f05600cf5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:22.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.546+0000 7f055dd9b640 1 --2- 172.21.15.27:0/2436888736 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f056006cc40 0x7f05600cf5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:22.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.546+0000 7f055dd9b640 1 -- 172.21.15.27:0/2436888736 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05600cd0f0 con 0x7f056006db30 2024-09-12T03:20:22.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.546+0000 7f055dd9b640 1 --2- 172.21.15.27:0/2436888736 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f056006db30 0x7f05601f48e0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f055400d5f0 tx=0x7f055400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:22.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.546+0000 7f054effd640 1 -- 172.21.15.27:0/2436888736 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05540125f0 con 0x7f056006db30 2024-09-12T03:20:22.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.546+0000 7f054effd640 1 -- 172.21.15.27:0/2436888736 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0554019070 con 0x7f056006db30 2024-09-12T03:20:22.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.546+0000 7f0564c2e640 1 -- 172.21.15.27:0/2436888736 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05600cead0 con 0x7f056006db30 2024-09-12T03:20:22.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.546+0000 7f054effd640 1 -- 172.21.15.27:0/2436888736 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05540129d0 con 0x7f056006db30 2024-09-12T03:20:22.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.546+0000 7f0564c2e640 1 -- 172.21.15.27:0/2436888736 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05600cf090 con 0x7f056006db30 2024-09-12T03:20:22.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.547+0000 7f0564c2e640 1 -- 172.21.15.27:0/2436888736 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f056006d060 con 0x7f056006db30 2024-09-12T03:20:22.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.548+0000 7f054effd640 1 -- 172.21.15.27:0/2436888736 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 90600+0+0 (secure 0 0 0) 0x7f0554012b70 con 0x7f056006db30 2024-09-12T03:20:22.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.548+0000 7f054effd640 1 --2- 172.21.15.27:0/2436888736 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f054006bf50 0x7f054006e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:22.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.548+0000 7f054effd640 1 -- 172.21.15.27:0/2436888736 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 5502+0+0 (secure 0 0 0) 0x7f055408b310 con 0x7f056006db30 2024-09-12T03:20:22.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.548+0000 7f055e59c640 1 --2- 172.21.15.27:0/2436888736 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f054006bf50 0x7f054006e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:22.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.549+0000 7f055e59c640 1 --2- 172.21.15.27:0/2436888736 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f054006bf50 0x7f054006e410 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f0550034000 tx=0x7f055002e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:22.552 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.552+0000 7f054effd640 1 -- 172.21.15.27:0/2436888736 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f0554059c80 con 0x7f056006db30 2024-09-12T03:20:22.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.729+0000 7f0564c2e640 1 -- 172.21.15.27:0/2436888736 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f05600d1fc0 con 0x7f056006db30 2024-09-12T03:20:22.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.729+0000 7f054effd640 1 -- 172.21.15.27:0/2436888736 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v73) ==== 93+0+10 (secure 0 0 0) 0x7f0554059580 con 0x7f056006db30 2024-09-12T03:20:22.729 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 1 2024-09-12T03:20:22.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.731+0000 7f0564c2e640 1 -- 172.21.15.27:0/2436888736 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f054006bf50 msgr2=0x7f054006e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:22.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.731+0000 7f0564c2e640 1 --2- 172.21.15.27:0/2436888736 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f054006bf50 0x7f054006e410 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f0550034000 tx=0x7f055002e000 comp rx=0 tx=0).stop 2024-09-12T03:20:22.732 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.731+0000 7f0564c2e640 1 -- 172.21.15.27:0/2436888736 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f056006db30 msgr2=0x7f05601f48e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:22.732 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.731+0000 7f0564c2e640 1 --2- 172.21.15.27:0/2436888736 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f056006db30 0x7f05601f48e0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f055400d5f0 tx=0x7f055400dab0 comp rx=0 tx=0).stop 2024-09-12T03:20:22.732 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.731+0000 7f0564c2e640 1 -- 172.21.15.27:0/2436888736 shutdown_connections 2024-09-12T03:20:22.732 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.731+0000 7f0564c2e640 1 --2- 172.21.15.27:0/2436888736 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f056015e650 0x7f05600cc5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:22.732 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.731+0000 7f0564c2e640 1 --2- 172.21.15.27:0/2436888736 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f054006bf50 0x7f054006e410 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:22.732 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.731+0000 7f0564c2e640 1 --2- 172.21.15.27:0/2436888736 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f056006db30 0x7f05601f48e0 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:22.732 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.731+0000 7f0564c2e640 1 --2- 172.21.15.27:0/2436888736 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f056006cc40 0x7f05600cf5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:22.732 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.731+0000 7f0564c2e640 1 -- 172.21.15.27:0/2436888736 >> 172.21.15.27:0/2436888736 conn(0x7f056006f320 msgr2=0x7f05600b3820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:22.732 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.731+0000 7f0564c2e640 1 -- 172.21.15.27:0/2436888736 shutdown_connections 2024-09-12T03:20:22.732 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.731+0000 7f0564c2e640 1 -- 172.21.15.27:0/2436888736 wait complete. 2024-09-12T03:20:22.742 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestMisc.test_names_can_only_be_goodchars' 2024-09-12T03:20:22.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.845+0000 7f179c447640 1 Processor -- start 2024-09-12T03:20:22.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.845+0000 7f179c447640 1 -- start start 2024-09-12T03:20:22.847 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.846+0000 7f179c447640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f179415e5f0 0x7f1794073250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:22.847 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.846+0000 7f179c447640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f1794162a30 0x7f179415e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:22.847 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.846+0000 7f179c447640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f17940d2240 con 0x7f1794073790 2024-09-12T03:20:22.847 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.846+0000 7f179c447640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f1794162e60 con 0x7f179415e5f0 2024-09-12T03:20:22.847 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.846+0000 7f179c447640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f1794073be0 con 0x7f1794162a30 2024-09-12T03:20:22.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.846+0000 7f179a1e3640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f179415e5f0 0x7f1794073250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:22.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.846+0000 7f17999e2640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f1794073790 0x7f1794162320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:33974/0 (socket says 172.21.15.27:33974) 2024-09-12T03:20:22.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.846+0000 7f17999e2640 1 -- 172.21.15.27:0/312453834 learned_addr learned my addr 172.21.15.27:0/312453834 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:20:22.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.847+0000 7f179a9e4640 1 --2- 172.21.15.27:0/312453834 >> v2:172.21.15.45:3300/0 conn(0x7f1794162a30 0x7f179415e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:22.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.847+0000 7f179a1e3640 1 -- 172.21.15.27:0/312453834 >> v2:172.21.15.45:3300/0 conn(0x7f1794162a30 msgr2=0x7f179415e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:22.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.847+0000 7f179a1e3640 1 --2- 172.21.15.27:0/312453834 >> v2:172.21.15.45:3300/0 conn(0x7f1794162a30 0x7f179415e9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:22.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.847+0000 7f179a1e3640 1 -- 172.21.15.27:0/312453834 >> v1:172.21.15.27:6789/0 conn(0x7f1794073790 legacy=0x7f1794162320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:22.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.847+0000 7f179a1e3640 1 -- 172.21.15.27:0/312453834 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1794163090 con 0x7f179415e5f0 2024-09-12T03:20:22.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.847+0000 7f17991e1640 1 -- 172.21.15.27:0/312453834 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3172649893 0 0) 0x7f17940d2240 con 0x7f1794073790 2024-09-12T03:20:22.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.847+0000 7f179a9e4640 1 --2- 172.21.15.27:0/312453834 >> v2:172.21.15.45:3300/0 conn(0x7f1794162a30 0x7f179415e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:20:22.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.848+0000 7f179a1e3640 1 --2- 172.21.15.27:0/312453834 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f179415e5f0 0x7f1794073250 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f17940702a0 tx=0x7f179002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62e78ba79366c29d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:22.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.848+0000 7f17991e1640 1 -- 172.21.15.27:0/312453834 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1790037070 con 0x7f179415e5f0 2024-09-12T03:20:22.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.848+0000 7f17991e1640 1 -- 172.21.15.27:0/312453834 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1790032440 con 0x7f179415e5f0 2024-09-12T03:20:22.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.848+0000 7f17991e1640 1 -- 172.21.15.27:0/312453834 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1790032810 con 0x7f179415e5f0 2024-09-12T03:20:22.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.848+0000 7f179c447640 1 -- 172.21.15.27:0/312453834 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f179415e5f0 msgr2=0x7f1794073250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:22.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.848+0000 7f179c447640 1 --2- 172.21.15.27:0/312453834 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f179415e5f0 0x7f1794073250 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f17940702a0 tx=0x7f179002a1a0 comp rx=0 tx=0).stop 2024-09-12T03:20:22.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.848+0000 7f179c447640 1 -- 172.21.15.27:0/312453834 shutdown_connections 2024-09-12T03:20:22.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.848+0000 7f179c447640 1 --2- 172.21.15.27:0/312453834 >> v2:172.21.15.45:3300/0 conn(0x7f1794162a30 0x7f179415e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:22.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.848+0000 7f179c447640 1 --2- 172.21.15.27:0/312453834 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f179415e5f0 0x7f1794073250 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:22.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.848+0000 7f179c447640 1 -- 172.21.15.27:0/312453834 >> 172.21.15.27:0/312453834 conn(0x7f179406f300 msgr2=0x7f179406f710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:22.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.848+0000 7f179c447640 1 -- 172.21.15.27:0/312453834 shutdown_connections 2024-09-12T03:20:22.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.848+0000 7f179c447640 1 -- 172.21.15.27:0/312453834 wait complete. 2024-09-12T03:20:22.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.850+0000 7f179c447640 1 Processor -- start 2024-09-12T03:20:22.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.850+0000 7f179c447640 1 -- start start 2024-09-12T03:20:22.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.850+0000 7f179c447640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1794073790 0x7f17941f8240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:22.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.850+0000 7f179c447640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f179415e5f0 0x7f1794152fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:22.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.850+0000 7f179c447640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1794162a30 0x7f179415c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:22.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.850+0000 7f179c447640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f17940d0750 con 0x7f179415e5f0 2024-09-12T03:20:22.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.850+0000 7f179c447640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f17940d0da0 con 0x7f1794073790 2024-09-12T03:20:22.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.850+0000 7f179c447640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f17940d0a30 con 0x7f1794162a30 2024-09-12T03:20:22.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.850+0000 7f17999e2640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f179415e5f0 0x7f1794152fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:22.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.851+0000 7f17999e2640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f179415e5f0 0x7f1794152fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46782/0 (socket says 172.21.15.27:46782) 2024-09-12T03:20:22.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.851+0000 7f17999e2640 1 -- 172.21.15.27:0/2384126743 learned_addr learned my addr 172.21.15.27:0/2384126743 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:20:22.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.851+0000 7f179a1e3640 1 --2- 172.21.15.27:0/2384126743 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1794073790 0x7f17941f8240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:22.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.851+0000 7f179a9e4640 1 --2- 172.21.15.27:0/2384126743 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1794162a30 0x7f179415c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:22.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.851+0000 7f17999e2640 1 -- 172.21.15.27:0/2384126743 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1794162a30 msgr2=0x7f179415c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:22.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.851+0000 7f17999e2640 1 --2- 172.21.15.27:0/2384126743 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1794162a30 0x7f179415c520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:22.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.851+0000 7f17999e2640 1 -- 172.21.15.27:0/2384126743 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1794073790 msgr2=0x7f17941f8240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:22.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.851+0000 7f17999e2640 1 --2- 172.21.15.27:0/2384126743 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1794073790 0x7f17941f8240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:22.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.851+0000 7f17999e2640 1 -- 172.21.15.27:0/2384126743 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1794153510 con 0x7f179415e5f0 2024-09-12T03:20:22.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.851+0000 7f179a9e4640 1 --2- 172.21.15.27:0/2384126743 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1794162a30 0x7f179415c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:20:22.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.851+0000 7f17999e2640 1 --2- 172.21.15.27:0/2384126743 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f179415e5f0 0x7f1794152fd0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f178400c500 tx=0x7f178400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:22.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.851+0000 7f178affd640 1 -- 172.21.15.27:0/2384126743 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1784018070 con 0x7f179415e5f0 2024-09-12T03:20:22.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.851+0000 7f178affd640 1 -- 172.21.15.27:0/2384126743 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1784007580 con 0x7f179415e5f0 2024-09-12T03:20:22.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.851+0000 7f179c447640 1 -- 172.21.15.27:0/2384126743 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17941f4f90 con 0x7f179415e5f0 2024-09-12T03:20:22.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.851+0000 7f178affd640 1 -- 172.21.15.27:0/2384126743 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1784004a20 con 0x7f179415e5f0 2024-09-12T03:20:22.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.851+0000 7f179c447640 1 -- 172.21.15.27:0/2384126743 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17941f7760 con 0x7f179415e5f0 2024-09-12T03:20:22.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.852+0000 7f179c447640 1 -- 172.21.15.27:0/2384126743 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f179415ee50 con 0x7f179415e5f0 2024-09-12T03:20:22.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.853+0000 7f178affd640 1 -- 172.21.15.27:0/2384126743 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 90600+0+0 (secure 0 0 0) 0x7f178401b070 con 0x7f179415e5f0 2024-09-12T03:20:22.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.853+0000 7f178affd640 1 --2- 172.21.15.27:0/2384126743 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f177806bf50 0x7f177806e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:22.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.853+0000 7f178affd640 1 -- 172.21.15.27:0/2384126743 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 5502+0+0 (secure 0 0 0) 0x7f178408a130 con 0x7f179415e5f0 2024-09-12T03:20:22.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.853+0000 7f179a1e3640 1 --2- 172.21.15.27:0/2384126743 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f177806bf50 0x7f177806e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:22.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.854+0000 7f179a1e3640 1 --2- 172.21.15.27:0/2384126743 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f177806bf50 0x7f177806e410 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f1790034000 tx=0x7f179002e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:22.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:22.858+0000 7f178affd640 1 -- 172.21.15.27:0/2384126743 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f1794153510 con 0x7f179415e5f0 2024-09-12T03:20:23.102 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:23.102+0000 7f179c447640 1 -- 172.21.15.27:0/2384126743 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_names_can_only_be_goodchars"]} v 0) -- 0x7f1794069790 con 0x7f179415e5f0 2024-09-12T03:20:23.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:23.116+0000 7f178affd640 1 -- 172.21.15.27:0/2384126743 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_names_can_only_be_goodchars"]}]=0 v212) ==== 149+0+0 (secure 0 0 0) 0x7f17840584e0 con 0x7f179415e5f0 2024-09-12T03:20:23.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:23.119+0000 7f179c447640 1 -- 172.21.15.27:0/2384126743 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f177806bf50 msgr2=0x7f177806e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:23.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:23.119+0000 7f179c447640 1 --2- 172.21.15.27:0/2384126743 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f177806bf50 0x7f177806e410 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f1790034000 tx=0x7f179002e000 comp rx=0 tx=0).stop 2024-09-12T03:20:23.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:23.119+0000 7f179c447640 1 -- 172.21.15.27:0/2384126743 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f179415e5f0 msgr2=0x7f1794152fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:23.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:23.119+0000 7f179c447640 1 --2- 172.21.15.27:0/2384126743 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f179415e5f0 0x7f1794152fd0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f178400c500 tx=0x7f178400c9c0 comp rx=0 tx=0).stop 2024-09-12T03:20:23.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:23.119+0000 7f179c447640 1 -- 172.21.15.27:0/2384126743 shutdown_connections 2024-09-12T03:20:23.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:23.119+0000 7f179c447640 1 --2- 172.21.15.27:0/2384126743 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1794162a30 0x7f179415c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:23.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:23.119+0000 7f179c447640 1 --2- 172.21.15.27:0/2384126743 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f177806bf50 0x7f177806e410 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:23.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:23.119+0000 7f179c447640 1 --2- 172.21.15.27:0/2384126743 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f179415e5f0 0x7f1794152fd0 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:23.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:23.119+0000 7f179c447640 1 --2- 172.21.15.27:0/2384126743 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1794073790 0x7f17941f8240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:23.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:23.119+0000 7f179c447640 1 -- 172.21.15.27:0/2384126743 >> 172.21.15.27:0/2384126743 conn(0x7f179406f300 msgr2=0x7f179415f650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:23.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:23.119+0000 7f179c447640 1 -- 172.21.15.27:0/2384126743 shutdown_connections 2024-09-12T03:20:23.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:23.119+0000 7f179c447640 1 -- 172.21.15.27:0/2384126743 wait complete. 2024-09-12T03:20:23.130 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-12T03:20:23.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:23.236+0000 7f1ea55c4640 1 Processor -- start 2024-09-12T03:20:23.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:23.236+0000 7f1ea55c4640 1 -- start start 2024-09-12T03:20:23.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:23.237+0000 7f1ea55c4640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1ea015e990 0x7f1ea00b7600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:23.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:23.237+0000 7f1ea55c4640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ea015ed70 con 0x7f1ea00b3d10 2024-09-12T03:20:23.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:23.237+0000 7f1ea55c4640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ea00d2a30 con 0x7f1ea00b6230 2024-09-12T03:20:23.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:23.237+0000 7f1ea55c4640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f1ea00d2d10 con 0x7f1ea015e990 2024-09-12T03:20:23.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:23.237+0000 7f1e9f7fe640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f1ea00b3d10 0x7f1ea00d3ad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:33976/0 (socket says 172.21.15.27:33976) 2024-09-12T03:20:23.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:23.237+0000 7f1e9f7fe640 1 -- 172.21.15.27:0/1147837090 learned_addr learned my addr 172.21.15.27:0/1147837090 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:20:23.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:23.237+0000 7f1e9effd640 1 --2- 172.21.15.27:0/1147837090 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1ea015e990 0x7f1ea00b7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:23.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:23.237+0000 7f1e9dffb640 1 -- 172.21.15.27:0/1147837090 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 858330978 0 0) 0x7f1ea015ed70 con 0x7f1ea00b3d10 2024-09-12T03:20:23.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:23.237+0000 7f1e9dffb640 1 -- 172.21.15.27:0/1147837090 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1e88003520 con 0x7f1ea00b3d10 2024-09-12T03:20:23.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:23.237+0000 7f1e9dffb640 1 -- 172.21.15.27:0/1147837090 <== mon.1 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2185630614 0 0) 0x7f1ea00d2a30 con 0x7f1ea00b6230 2024-09-12T03:20:23.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:23.237+0000 7f1e9dffb640 1 -- 172.21.15.27:0/1147837090 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1ea015ed70 con 0x7f1ea00b6230 2024-09-12T03:20:23.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:23.237+0000 7f1e9effd640 1 -- 172.21.15.27:0/1147837090 >> v1:172.21.15.45:6789/0 conn(0x7f1ea00b6230 legacy=0x7f1ea00b3600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:23.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:23.237+0000 7f1e9effd640 1 -- 172.21.15.27:0/1147837090 >> v1:172.21.15.27:6789/0 conn(0x7f1ea00b3d10 legacy=0x7f1ea00d3ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:23.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:23.238+0000 7f1e9effd640 1 -- 172.21.15.27:0/1147837090 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ea00d2ff0 con 0x7f1ea015e990 2024-09-12T03:20:23.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:23.238+0000 7f1e9effd640 1 --2- 172.21.15.27:0/1147837090 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1ea015e990 0x7f1ea00b7600 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f1e900097d0 tx=0x7f1e9002a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3b565d9cd8df268c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:23.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:23.239+0000 7f1e9dffb640 1 -- 172.21.15.27:0/1147837090 <== mon.2 v2:172.21.15.45:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e90037070 con 0x7f1ea015e990 2024-09-12T03:20:23.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:23.239+0000 7f1e9dffb640 1 -- 172.21.15.27:0/1147837090 <== mon.2 v2:172.21.15.45:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e90032440 con 0x7f1ea015e990 2024-09-12T03:20:23.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:23.239+0000 7f1e9dffb640 1 -- 172.21.15.27:0/1147837090 <== mon.2 v2:172.21.15.45:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e90032810 con 0x7f1ea015e990 2024-09-12T03:20:23.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:23.239+0000 7f1ea55c4640 1 -- 172.21.15.27:0/1147837090 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1ea015e990 msgr2=0x7f1ea00b7600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:23.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:23.239+0000 7f1ea55c4640 1 --2- 172.21.15.27:0/1147837090 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1ea015e990 0x7f1ea00b7600 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f1e900097d0 tx=0x7f1e9002a210 comp rx=0 tx=0).stop 2024-09-12T03:20:23.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:23.239+0000 7f1ea55c4640 1 -- 172.21.15.27:0/1147837090 shutdown_connections 2024-09-12T03:20:23.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:23.239+0000 7f1ea55c4640 1 --2- 172.21.15.27:0/1147837090 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1ea015e990 0x7f1ea00b7600 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:23.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:23.239+0000 7f1ea55c4640 1 -- 172.21.15.27:0/1147837090 >> 172.21.15.27:0/1147837090 conn(0x7f1ea00d1230 msgr2=0x7f1ea00d1640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:23.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:23.239+0000 7f1ea55c4640 1 -- 172.21.15.27:0/1147837090 shutdown_connections 2024-09-12T03:20:23.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:23.239+0000 7f1ea55c4640 1 -- 172.21.15.27:0/1147837090 wait complete. 2024-09-12T03:20:23.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:23.241+0000 7f1ea55c4640 1 Processor -- start 2024-09-12T03:20:23.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:23.241+0000 7f1ea55c4640 1 -- start start 2024-09-12T03:20:23.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:23.241+0000 7f1ea55c4640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1ea00b3d10 0x7f1ea01f7b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:23.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:23.241+0000 7f1ea55c4640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1ea00b6230 0x7f1ea01f8040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:23.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:23.241+0000 7f1ea55c4640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1ea015e990 0x7f1ea0160780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:23.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:23.241+0000 7f1ea55c4640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f1ea00b7b40 con 0x7f1ea00b6230 2024-09-12T03:20:23.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:23.241+0000 7f1ea55c4640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f1ea00d35b0 con 0x7f1ea00b3d10 2024-09-12T03:20:23.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:23.241+0000 7f1ea55c4640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f1ea00b7cc0 con 0x7f1ea015e990 2024-09-12T03:20:23.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:23.241+0000 7f1e9e7fc640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1ea00b6230 0x7f1ea01f8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:23.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:23.242+0000 7f1e9e7fc640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1ea00b6230 0x7f1ea01f8040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46796/0 (socket says 172.21.15.27:46796) 2024-09-12T03:20:23.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:23.242+0000 7f1e9e7fc640 1 -- 172.21.15.27:0/942386141 learned_addr learned my addr 172.21.15.27:0/942386141 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:20:23.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:23.242+0000 7f1e9effd640 1 --2- 172.21.15.27:0/942386141 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1ea00b3d10 0x7f1ea01f7b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:23.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:23.242+0000 7f1e9f7fe640 1 --2- 172.21.15.27:0/942386141 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1ea015e990 0x7f1ea0160780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:23.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:23.242+0000 7f1e9e7fc640 1 -- 172.21.15.27:0/942386141 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1ea015e990 msgr2=0x7f1ea0160780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:23.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:23.242+0000 7f1e9e7fc640 1 --2- 172.21.15.27:0/942386141 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1ea015e990 0x7f1ea0160780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:23.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:23.242+0000 7f1e9e7fc640 1 -- 172.21.15.27:0/942386141 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1ea00b3d10 msgr2=0x7f1ea01f7b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:23.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:23.242+0000 7f1e9e7fc640 1 --2- 172.21.15.27:0/942386141 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1ea00b3d10 0x7f1ea01f7b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:23.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:23.242+0000 7f1e9e7fc640 1 -- 172.21.15.27:0/942386141 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ea00abe00 con 0x7f1ea00b6230 2024-09-12T03:20:23.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:23.242+0000 7f1e9effd640 1 --2- 172.21.15.27:0/942386141 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1ea00b3d10 0x7f1ea01f7b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:20:23.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:23.242+0000 7f1e9e7fc640 1 --2- 172.21.15.27:0/942386141 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1ea00b6230 0x7f1ea01f8040 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f1e9400c450 tx=0x7f1e9400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:23.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:23.242+0000 7f1e877fe640 1 -- 172.21.15.27:0/942386141 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e940052d0 con 0x7f1ea00b6230 2024-09-12T03:20:23.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:23.242+0000 7f1e877fe640 1 -- 172.21.15.27:0/942386141 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e94005470 con 0x7f1ea00b6230 2024-09-12T03:20:23.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:23.242+0000 7f1ea55c4640 1 -- 172.21.15.27:0/942386141 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ea0161370 con 0x7f1ea00b6230 2024-09-12T03:20:23.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:23.242+0000 7f1e877fe640 1 -- 172.21.15.27:0/942386141 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e94012870 con 0x7f1ea00b6230 2024-09-12T03:20:23.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:23.242+0000 7f1ea55c4640 1 -- 172.21.15.27:0/942386141 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ea01f5920 con 0x7f1ea00b6230 2024-09-12T03:20:23.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:23.243+0000 7f1ea55c4640 1 -- 172.21.15.27:0/942386141 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ea00d5900 con 0x7f1ea00b6230 2024-09-12T03:20:23.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:23.244+0000 7f1e877fe640 1 -- 172.21.15.27:0/942386141 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 90600+0+0 (secure 0 0 0) 0x7f1e94012a10 con 0x7f1ea00b6230 2024-09-12T03:20:23.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:23.244+0000 7f1e877fe640 1 --2- 172.21.15.27:0/942386141 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f1e8c06bf50 0x7f1e8c06e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:23.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:23.244+0000 7f1e9effd640 1 --2- 172.21.15.27:0/942386141 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f1e8c06bf50 0x7f1e8c06e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:23.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:23.244+0000 7f1e877fe640 1 -- 172.21.15.27:0/942386141 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 5502+0+0 (secure 0 0 0) 0x7f1e9408aba0 con 0x7f1ea00b6230 2024-09-12T03:20:23.248 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:23.245+0000 7f1e9effd640 1 --2- 172.21.15.27:0/942386141 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f1e8c06bf50 0x7f1e8c06e410 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f1e90034000 tx=0x7f1e9002e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:23.248 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:23.249+0000 7f1e877fe640 1 -- 172.21.15.27:0/942386141 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f1ea00d5900 con 0x7f1ea00b6230 2024-09-12T03:20:23.423 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:23.423+0000 7f1ea55c4640 1 -- 172.21.15.27:0/942386141 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f1ea00b75e0 con 0x7f1ea00b6230 2024-09-12T03:20:23.426 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:23.426+0000 7f1e877fe640 1 -- 172.21.15.27:0/942386141 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v18)=0 v18) ==== 122+0+0 (secure 0 0 0) 0x7f1e940595d0 con 0x7f1ea00b6230 2024-09-12T03:20:23.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:23.428+0000 7f1ea55c4640 1 -- 172.21.15.27:0/942386141 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f1e8c06bf50 msgr2=0x7f1e8c06e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:23.429 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:23.428+0000 7f1ea55c4640 1 --2- 172.21.15.27:0/942386141 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f1e8c06bf50 0x7f1e8c06e410 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f1e90034000 tx=0x7f1e9002e000 comp rx=0 tx=0).stop 2024-09-12T03:20:23.429 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:23.428+0000 7f1ea55c4640 1 -- 172.21.15.27:0/942386141 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1ea00b6230 msgr2=0x7f1ea01f8040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:23.429 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:23.428+0000 7f1ea55c4640 1 --2- 172.21.15.27:0/942386141 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1ea00b6230 0x7f1ea01f8040 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f1e9400c450 tx=0x7f1e9400c910 comp rx=0 tx=0).stop 2024-09-12T03:20:23.429 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:23.428+0000 7f1ea55c4640 1 -- 172.21.15.27:0/942386141 shutdown_connections 2024-09-12T03:20:23.429 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:23.428+0000 7f1ea55c4640 1 --2- 172.21.15.27:0/942386141 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1ea015e990 0x7f1ea0160780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:23.429 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:23.428+0000 7f1ea55c4640 1 --2- 172.21.15.27:0/942386141 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f1e8c06bf50 0x7f1e8c06e410 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:23.429 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:23.428+0000 7f1ea55c4640 1 --2- 172.21.15.27:0/942386141 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1ea00b6230 0x7f1ea01f8040 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:23.429 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:23.428+0000 7f1ea55c4640 1 --2- 172.21.15.27:0/942386141 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1ea00b3d10 0x7f1ea01f7b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:23.429 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:23.428+0000 7f1ea55c4640 1 -- 172.21.15.27:0/942386141 >> 172.21.15.27:0/942386141 conn(0x7f1ea00d1230 msgr2=0x7f1ea00b5a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:23.429 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:23.428+0000 7f1ea55c4640 1 -- 172.21.15.27:0/942386141 shutdown_connections 2024-09-12T03:20:23.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:23.428+0000 7f1ea55c4640 1 -- 172.21.15.27:0/942386141 wait complete. 2024-09-12T03:20:23.440 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-12T03:20:23.440 DEBUG:teuthology.orchestra.run.smithi027:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-12T03:20:23.479 DEBUG:teuthology.orchestra.run.smithi045:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-12T03:20:23.523 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-12T03:20:23.523 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-12T03:20:23.523 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T03:20:23.540 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056004,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497432k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16324 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248716k,nr_inodes=812179,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497432k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-12T03:20:23.540 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-12T03:20:23.540 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T03:20:23.582 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056795,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16602 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249348k,nr_inodes=812337,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-12T03:20:23.582 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-12T03:20:23.582 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T03:20:23.598 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056004,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497432k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16324 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248716k,nr_inodes=812179,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497432k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-12T03:20:23.598 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-12T03:20:23.598 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T03:20:23.643 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056795,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16602 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249348k,nr_inodes=812337,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-12T03:20:23.644 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-12T03:20:23.644 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-12T03:20:23.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:23.746+0000 7f90da149640 1 Processor -- start 2024-09-12T03:20:23.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:23.746+0000 7f90da149640 1 -- start start 2024-09-12T03:20:23.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:23.747+0000 7f90da149640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f90d4165150 0x7f90d4167540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:23.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:23.747+0000 7f90da149640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f90d406fa00 con 0x7f90d40d1450 2024-09-12T03:20:23.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:23.747+0000 7f90da149640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f90d406f720 con 0x7f90d40cef50 2024-09-12T03:20:23.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:23.747+0000 7f90da149640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f90d415ca90 con 0x7f90d4165150 2024-09-12T03:20:23.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:23.747+0000 7f90d3fff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f90d4165150 0x7f90d4167540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:23.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:23.747+0000 7f90d2ffd640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f90d40d1450 0x7f90d4070e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:33992/0 (socket says 172.21.15.27:33992) 2024-09-12T03:20:23.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:23.747+0000 7f90d2ffd640 1 -- 172.21.15.27:0/2707651013 learned_addr learned my addr 172.21.15.27:0/2707651013 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:20:23.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:23.748+0000 7f90d3fff640 1 -- 172.21.15.27:0/2707651013 >> v1:172.21.15.45:6789/0 conn(0x7f90d40cef50 legacy=0x7f90d40cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:23.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:23.748+0000 7f90d27fc640 1 -- 172.21.15.27:0/2707651013 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2534910019 0 0) 0x7f90d406fa00 con 0x7f90d40d1450 2024-09-12T03:20:23.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:23.748+0000 7f90d3fff640 1 -- 172.21.15.27:0/2707651013 >> v1:172.21.15.27:6789/0 conn(0x7f90d40d1450 legacy=0x7f90d4070e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:23.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:23.748+0000 7f90d3fff640 1 -- 172.21.15.27:0/2707651013 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90d40d2a30 con 0x7f90d4165150 2024-09-12T03:20:23.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:23.748+0000 7f90d3fff640 1 --2- 172.21.15.27:0/2707651013 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f90d4165150 0x7f90d4167540 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f90cc009d90 tx=0x7f90cc02a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93365530db4f15b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:23.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:23.748+0000 7f90d27fc640 1 -- 172.21.15.27:0/2707651013 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90cc031600 con 0x7f90d4165150 2024-09-12T03:20:23.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:23.748+0000 7f90d27fc640 1 -- 172.21.15.27:0/2707651013 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90cc0317a0 con 0x7f90d4165150 2024-09-12T03:20:23.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:23.749+0000 7f90d27fc640 1 -- 172.21.15.27:0/2707651013 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90cc031b70 con 0x7f90d4165150 2024-09-12T03:20:23.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:23.749+0000 7f90da149640 1 -- 172.21.15.27:0/2707651013 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f90d4165150 msgr2=0x7f90d4167540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:23.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:23.749+0000 7f90da149640 1 --2- 172.21.15.27:0/2707651013 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f90d4165150 0x7f90d4167540 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f90cc009d90 tx=0x7f90cc02a5d0 comp rx=0 tx=0).stop 2024-09-12T03:20:23.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:23.749+0000 7f90da149640 1 -- 172.21.15.27:0/2707651013 shutdown_connections 2024-09-12T03:20:23.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:23.749+0000 7f90da149640 1 --2- 172.21.15.27:0/2707651013 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f90d4165150 0x7f90d4167540 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:23.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:23.749+0000 7f90da149640 1 -- 172.21.15.27:0/2707651013 >> 172.21.15.27:0/2707651013 conn(0x7f90d4073800 msgr2=0x7f90d409ba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:23.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:23.749+0000 7f90da149640 1 -- 172.21.15.27:0/2707651013 shutdown_connections 2024-09-12T03:20:23.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:23.749+0000 7f90da149640 1 -- 172.21.15.27:0/2707651013 wait complete. 2024-09-12T03:20:23.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:23.750+0000 7f90da149640 1 Processor -- start 2024-09-12T03:20:23.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:23.750+0000 7f90da149640 1 -- start start 2024-09-12T03:20:23.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:23.751+0000 7f90da149640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f90d40cef50 0x7f90d40b2290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:23.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:23.751+0000 7f90da149640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f90d40d1450 0x7f90d41f57c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:23.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:23.751+0000 7f90da149640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f90d4165150 0x7f90d41f5d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:23.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:23.751+0000 7f90da149640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f90d406f4e0 con 0x7f90d4165150 2024-09-12T03:20:23.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:23.751+0000 7f90da149640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f90d406f360 con 0x7f90d40d1450 2024-09-12T03:20:23.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:23.751+0000 7f90da149640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f90d40cfbd0 con 0x7f90d40cef50 2024-09-12T03:20:23.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:23.751+0000 7f90d3fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f90d4165150 0x7f90d41f5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:23.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:23.751+0000 7f90d3fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f90d4165150 0x7f90d41f5d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46818/0 (socket says 172.21.15.27:46818) 2024-09-12T03:20:23.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:23.751+0000 7f90d3fff640 1 -- 172.21.15.27:0/4959049 learned_addr learned my addr 172.21.15.27:0/4959049 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:20:23.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:23.751+0000 7f90d2ffd640 1 --2- 172.21.15.27:0/4959049 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f90d40d1450 0x7f90d41f57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:23.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:23.751+0000 7f90d37fe640 1 --2- 172.21.15.27:0/4959049 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f90d40cef50 0x7f90d40b2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:23.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:23.752+0000 7f90d3fff640 1 -- 172.21.15.27:0/4959049 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f90d40cef50 msgr2=0x7f90d40b2290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:23.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:23.752+0000 7f90d3fff640 1 --2- 172.21.15.27:0/4959049 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f90d40cef50 0x7f90d40b2290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:23.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:23.752+0000 7f90d3fff640 1 -- 172.21.15.27:0/4959049 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f90d40d1450 msgr2=0x7f90d41f57c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:23.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:23.752+0000 7f90d3fff640 1 --2- 172.21.15.27:0/4959049 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f90d40d1450 0x7f90d41f57c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:23.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:23.752+0000 7f90d3fff640 1 -- 172.21.15.27:0/4959049 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90d4150b80 con 0x7f90d4165150 2024-09-12T03:20:23.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:23.752+0000 7f90d37fe640 1 --2- 172.21.15.27:0/4959049 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f90d40cef50 0x7f90d40b2290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:20:23.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:23.752+0000 7f90d2ffd640 1 --2- 172.21.15.27:0/4959049 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f90d40d1450 0x7f90d41f57c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:20:23.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:23.752+0000 7f90d3fff640 1 --2- 172.21.15.27:0/4959049 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f90d4165150 0x7f90d41f5d00 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f90cc009d90 tx=0x7f90cc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:23.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:23.752+0000 7f90bbfff640 1 -- 172.21.15.27:0/4959049 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90cc03f070 con 0x7f90d4165150 2024-09-12T03:20:23.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:23.752+0000 7f90bbfff640 1 -- 172.21.15.27:0/4959049 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90cc030b60 con 0x7f90d4165150 2024-09-12T03:20:23.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:23.752+0000 7f90da149640 1 -- 172.21.15.27:0/4959049 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90d40b32e0 con 0x7f90d4165150 2024-09-12T03:20:23.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:23.752+0000 7f90bbfff640 1 -- 172.21.15.27:0/4959049 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90cc043410 con 0x7f90d4165150 2024-09-12T03:20:23.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:23.752+0000 7f90da149640 1 -- 172.21.15.27:0/4959049 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90d40b27d0 con 0x7f90d4165150 2024-09-12T03:20:23.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:23.753+0000 7f90da149640 1 -- 172.21.15.27:0/4959049 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90d40cf330 con 0x7f90d4165150 2024-09-12T03:20:23.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:23.753+0000 7f90bbfff640 1 -- 172.21.15.27:0/4959049 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 90600+0+0 (secure 0 0 0) 0x7f90cc03b410 con 0x7f90d4165150 2024-09-12T03:20:23.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:23.754+0000 7f90bbfff640 1 --2- 172.21.15.27:0/4959049 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f90c006bf50 0x7f90c006e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:23.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:23.754+0000 7f90bbfff640 1 -- 172.21.15.27:0/4959049 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 5502+0+0 (secure 0 0 0) 0x7f90cc0ac360 con 0x7f90d4165150 2024-09-12T03:20:23.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:23.754+0000 7f90d37fe640 1 --2- 172.21.15.27:0/4959049 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f90c006bf50 0x7f90c006e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:23.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:23.754+0000 7f90d37fe640 1 --2- 172.21.15.27:0/4959049 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f90c006bf50 0x7f90c006e410 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f90c4007fd0 tx=0x7f90c4009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:23.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:23.758+0000 7f90bbfff640 1 -- 172.21.15.27:0/4959049 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f90cc07a6c0 con 0x7f90d4165150 2024-09-12T03:20:23.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:23.973+0000 7f90da149640 1 -- 172.21.15.27:0/4959049 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f90d4150d50 con 0x7f90d4165150 2024-09-12T03:20:23.973 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:23.973+0000 7f90bbfff640 1 -- 172.21.15.27:0/4959049 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) ==== 94+0+3911 (secure 0 0 0) 0x7f90cc07e9c0 con 0x7f90d4165150 2024-09-12T03:20:23.973 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:20:23.973 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":41,"btime":"2024-09-12T03:20:19:103521+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6028,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/3112880978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":3112880978},{"type":"v1","addr":"172.21.15.45:6833","nonce":3112880978}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":6045,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6834/3058537826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":3058537826},{"type":"v1","addr":"172.21.15.27:6834","nonce":3058537826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":34},{"gid":6048,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/1585628536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":1585628536},{"type":"v1","addr":"172.21.15.27:6835","nonce":1585628536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":34},{"gid":6063,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6836/1421588365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":1421588365},{"type":"v1","addr":"172.21.15.45:6836","nonce":1421588365}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":34}],"filesystems":[]} 2024-09-12T03:20:23.974 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 41 2024-09-12T03:20:23.975 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:23.975+0000 7f90da149640 1 -- 172.21.15.27:0/4959049 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f90c006bf50 msgr2=0x7f90c006e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:23.975 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:23.975+0000 7f90da149640 1 --2- 172.21.15.27:0/4959049 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f90c006bf50 0x7f90c006e410 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f90c4007fd0 tx=0x7f90c4009040 comp rx=0 tx=0).stop 2024-09-12T03:20:23.975 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:23.975+0000 7f90da149640 1 -- 172.21.15.27:0/4959049 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f90d4165150 msgr2=0x7f90d41f5d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:23.976 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:23.975+0000 7f90da149640 1 --2- 172.21.15.27:0/4959049 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f90d4165150 0x7f90d41f5d00 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f90cc009d90 tx=0x7f90cc004a40 comp rx=0 tx=0).stop 2024-09-12T03:20:23.976 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:23.975+0000 7f90da149640 1 -- 172.21.15.27:0/4959049 shutdown_connections 2024-09-12T03:20:23.976 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:23.975+0000 7f90da149640 1 --2- 172.21.15.27:0/4959049 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f90c006bf50 0x7f90c006e410 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:23.976 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:23.975+0000 7f90da149640 1 --2- 172.21.15.27:0/4959049 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f90d4165150 0x7f90d41f5d00 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:23.976 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:23.975+0000 7f90da149640 1 --2- 172.21.15.27:0/4959049 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f90d40d1450 0x7f90d41f57c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:23.976 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:23.975+0000 7f90da149640 1 --2- 172.21.15.27:0/4959049 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f90d40cef50 0x7f90d40b2290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:23.976 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:23.975+0000 7f90da149640 1 -- 172.21.15.27:0/4959049 >> 172.21.15.27:0/4959049 conn(0x7f90d4073800 msgr2=0x7f90d4070e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:23.976 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:23.975+0000 7f90da149640 1 -- 172.21.15.27:0/4959049 shutdown_connections 2024-09-12T03:20:23.976 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:23.975+0000 7f90da149640 1 -- 172.21.15.27:0/4959049 wait complete. 2024-09-12T03:20:23.987 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-12T03:20:23.987 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-12T03:20:23.987 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-12T03:20:23.987 INFO:teuthology.orchestra.run:waiting for 300 2024-09-12T03:20:23.987 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-12T03:20:23.987 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-12T03:20:23.987 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-12T03:20:23.988 INFO:teuthology.orchestra.run:waiting for 300 2024-09-12T03:20:23.988 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-12T03:20:23.988 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-12T03:20:23.988 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-12T03:20:23.988 INFO:teuthology.orchestra.run:waiting for 300 2024-09-12T03:20:23.988 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-12T03:20:23.988 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-12T03:20:23.989 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-12T03:20:23.989 INFO:teuthology.orchestra.run:waiting for 300 2024-09-12T03:20:23.998 INFO:tasks.ceph.mds.c:Stopped 2024-09-12T03:20:23.998 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-12T03:20:23.999 INFO:tasks.ceph.mds.a:Stopped 2024-09-12T03:20:23.999 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-12T03:20:24.000 INFO:tasks.ceph.mds.d:Stopped 2024-09-12T03:20:24.001 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-12T03:20:24.001 INFO:tasks.ceph.mds.b:Stopped 2024-09-12T03:20:24.001 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-12T03:20:24.040 INFO:tasks.ceph.mds.c:Started 2024-09-12T03:20:24.041 DEBUG:teuthology.parallel:result is None 2024-09-12T03:20:24.041 INFO:tasks.ceph.mds.a:Started 2024-09-12T03:20:24.042 DEBUG:teuthology.parallel:result is None 2024-09-12T03:20:24.044 INFO:tasks.ceph.mds.d:Started 2024-09-12T03:20:24.044 DEBUG:teuthology.parallel:result is None 2024-09-12T03:20:24.045 INFO:tasks.ceph.mds.b:Started 2024-09-12T03:20:24.050 DEBUG:teuthology.parallel:result is None 2024-09-12T03:20:24.050 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-12T03:20:24.118 INFO:tasks.ceph.mds.c.smithi027.stdout:starting mds.c at 2024-09-12T03:20:24.119 INFO:tasks.ceph.mds.a.smithi027.stdout:starting mds.a at 2024-09-12T03:20:24.130 INFO:tasks.ceph.mds.b.smithi045.stdout:starting mds.b at 2024-09-12T03:20:24.130 INFO:tasks.ceph.mds.d.smithi045.stdout:starting mds.d at 2024-09-12T03:20:24.159 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:24.159+0000 7f3d9952c640 1 Processor -- start 2024-09-12T03:20:24.159 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:24.159+0000 7f3d9952c640 1 -- start start 2024-09-12T03:20:24.160 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:24.160+0000 7f3d9952c640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f3d9415c6c0 0x7f3d94073210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:24.160 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:24.160+0000 7f3d9952c640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3d940cf8a0 con 0x7f3d940cfaf0 2024-09-12T03:20:24.160 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:24.160+0000 7f3d9952c640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3d941600f0 con 0x7f3d9415c2e0 2024-09-12T03:20:24.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:24.160+0000 7f3d9952c640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f3d94160eb0 con 0x7f3d9415c6c0 2024-09-12T03:20:24.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:24.160+0000 7f3d927fc640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f3d9415c6c0 0x7f3d94073210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:24.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:24.160+0000 7f3d937fe640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f3d940cfaf0 0x7f3d940b5bc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:34004/0 (socket says 172.21.15.27:34004) 2024-09-12T03:20:24.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:24.160+0000 7f3d937fe640 1 -- 172.21.15.27:0/3600612869 learned_addr learned my addr 172.21.15.27:0/3600612869 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:20:24.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:24.160+0000 7f3d927fc640 1 -- 172.21.15.27:0/3600612869 >> v1:172.21.15.45:6789/0 conn(0x7f3d9415c2e0 legacy=0x7f3d940cf070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:24.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:24.160+0000 7f3d927fc640 1 -- 172.21.15.27:0/3600612869 >> v1:172.21.15.27:6789/0 conn(0x7f3d940cfaf0 legacy=0x7f3d940b5bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:24.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:24.160+0000 7f3d927fc640 1 -- 172.21.15.27:0/3600612869 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d941603d0 con 0x7f3d9415c6c0 2024-09-12T03:20:24.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:24.160+0000 7f3d91ffb640 1 -- 172.21.15.27:0/3600612869 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3671099449 0 0) 0x7f3d940cf8a0 con 0x7f3d940cfaf0 2024-09-12T03:20:24.162 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:24.161+0000 7f3d927fc640 1 --2- 172.21.15.27:0/3600612869 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3d9415c6c0 0x7f3d94073210 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f3d880097a0 tx=0x7f3d8802a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7702230bfe8f07c9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:24.162 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:24.161+0000 7f3d91ffb640 1 -- 172.21.15.27:0/3600612869 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d88037070 con 0x7f3d9415c6c0 2024-09-12T03:20:24.162 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:24.161+0000 7f3d91ffb640 1 -- 172.21.15.27:0/3600612869 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d88032440 con 0x7f3d9415c6c0 2024-09-12T03:20:24.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:24.161+0000 7f3d91ffb640 1 -- 172.21.15.27:0/3600612869 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d8802ae60 con 0x7f3d9415c6c0 2024-09-12T03:20:24.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:24.162+0000 7f3d9952c640 1 -- 172.21.15.27:0/3600612869 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3d9415c6c0 msgr2=0x7f3d94073210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:24.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:24.162+0000 7f3d9952c640 1 --2- 172.21.15.27:0/3600612869 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3d9415c6c0 0x7f3d94073210 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f3d880097a0 tx=0x7f3d8802a110 comp rx=0 tx=0).stop 2024-09-12T03:20:24.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:24.162+0000 7f3d9952c640 1 -- 172.21.15.27:0/3600612869 shutdown_connections 2024-09-12T03:20:24.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:24.162+0000 7f3d9952c640 1 --2- 172.21.15.27:0/3600612869 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3d9415c6c0 0x7f3d94073210 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:24.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:24.162+0000 7f3d9952c640 1 -- 172.21.15.27:0/3600612869 >> 172.21.15.27:0/3600612869 conn(0x7f3d9406f2c0 msgr2=0x7f3d9406f6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:24.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:24.162+0000 7f3d9952c640 1 -- 172.21.15.27:0/3600612869 shutdown_connections 2024-09-12T03:20:24.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:24.162+0000 7f3d9952c640 1 -- 172.21.15.27:0/3600612869 wait complete. 2024-09-12T03:20:24.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:24.163+0000 7f3d9952c640 1 Processor -- start 2024-09-12T03:20:24.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:24.163+0000 7f3d9952c640 1 -- start start 2024-09-12T03:20:24.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:24.164+0000 7f3d9952c640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f3d940cfaf0 0x7f3d940ae8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:24.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:24.164+0000 7f3d9952c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3d9415c2e0 0x7f3d941f8510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:24.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:24.164+0000 7f3d9952c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d9415c6c0 0x7f3d940b43b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:24.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:24.164+0000 7f3d9952c640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f3d940738d0 con 0x7f3d9415c2e0 2024-09-12T03:20:24.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:24.164+0000 7f3d9952c640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f3d94073750 con 0x7f3d9415c6c0 2024-09-12T03:20:24.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:24.164+0000 7f3d9952c640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f3d94073a50 con 0x7f3d940cfaf0 2024-09-12T03:20:24.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:24.164+0000 7f3d927fc640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3d9415c2e0 0x7f3d941f8510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:24.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:24.164+0000 7f3d937fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d9415c6c0 0x7f3d940b43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:24.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:24.164+0000 7f3d927fc640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3d9415c2e0 0x7f3d941f8510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46888/0 (socket says 172.21.15.27:46888) 2024-09-12T03:20:24.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:24.164+0000 7f3d927fc640 1 -- 172.21.15.27:0/1007421783 learned_addr learned my addr 172.21.15.27:0/1007421783 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:20:24.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:24.164+0000 7f3d92ffd640 1 --2- 172.21.15.27:0/1007421783 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f3d940cfaf0 0x7f3d940ae8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:24.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:24.164+0000 7f3d927fc640 1 -- 172.21.15.27:0/1007421783 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f3d940cfaf0 msgr2=0x7f3d940ae8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:24.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:24.164+0000 7f3d927fc640 1 --2- 172.21.15.27:0/1007421783 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f3d940cfaf0 0x7f3d940ae8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:24.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:24.164+0000 7f3d927fc640 1 -- 172.21.15.27:0/1007421783 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d9415c6c0 msgr2=0x7f3d940b43b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:24.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:24.164+0000 7f3d927fc640 1 --2- 172.21.15.27:0/1007421783 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d9415c6c0 0x7f3d940b43b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:24.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:24.164+0000 7f3d927fc640 1 -- 172.21.15.27:0/1007421783 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d940aef60 con 0x7f3d9415c2e0 2024-09-12T03:20:24.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:24.165+0000 7f3d927fc640 1 --2- 172.21.15.27:0/1007421783 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3d9415c2e0 0x7f3d941f8510 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f3d880098c0 tx=0x7f3d880046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:24.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:24.165+0000 7f3d7b7fe640 1 -- 172.21.15.27:0/1007421783 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d88037040 con 0x7f3d9415c2e0 2024-09-12T03:20:24.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:24.165+0000 7f3d7b7fe640 1 -- 172.21.15.27:0/1007421783 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d88042070 con 0x7f3d9415c2e0 2024-09-12T03:20:24.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:24.165+0000 7f3d9952c640 1 -- 172.21.15.27:0/1007421783 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d940af240 con 0x7f3d9415c2e0 2024-09-12T03:20:24.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:24.165+0000 7f3d9952c640 1 -- 172.21.15.27:0/1007421783 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d941f7ff0 con 0x7f3d9415c2e0 2024-09-12T03:20:24.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:24.165+0000 7f3d7b7fe640 1 -- 172.21.15.27:0/1007421783 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d88030c70 con 0x7f3d9415c2e0 2024-09-12T03:20:24.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:24.166+0000 7f3d9952c640 1 -- 172.21.15.27:0/1007421783 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d74005290 con 0x7f3d9415c2e0 2024-09-12T03:20:24.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:24.166+0000 7f3d7b7fe640 1 -- 172.21.15.27:0/1007421783 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 90600+0+0 (secure 0 0 0) 0x7f3d88047460 con 0x7f3d9415c2e0 2024-09-12T03:20:24.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:24.167+0000 7f3d7b7fe640 1 --2- 172.21.15.27:0/1007421783 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f3d8006bf50 0x7f3d8006e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:24.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:24.167+0000 7f3d7b7fe640 1 -- 172.21.15.27:0/1007421783 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 5502+0+0 (secure 0 0 0) 0x7f3d880aba20 con 0x7f3d9415c2e0 2024-09-12T03:20:24.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:24.167+0000 7f3d92ffd640 1 --2- 172.21.15.27:0/1007421783 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f3d8006bf50 0x7f3d8006e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:24.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:24.167+0000 7f3d92ffd640 1 --2- 172.21.15.27:0/1007421783 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f3d8006bf50 0x7f3d8006e410 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f3d84000f60 tx=0x7f3d84009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:24.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:24.171+0000 7f3d7b7fe640 1 -- 172.21.15.27:0/1007421783 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f3d88079d30 con 0x7f3d9415c2e0 2024-09-12T03:20:24.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:24.359+0000 7f3d9952c640 1 -- 172.21.15.27:0/1007421783 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f3d74005850 con 0x7f3d9415c2e0 2024-09-12T03:20:25.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.128+0000 7f3d7b7fe640 1 -- 172.21.15.27:0/1007421783 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v75) ==== 97+0+0 (secure 0 0 0) 0x7f3d8807e030 con 0x7f3d9415c2e0 2024-09-12T03:20:25.128 INFO:teuthology.orchestra.run.smithi027.stderr: removed all blocklist entries 2024-09-12T03:20:25.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.132+0000 7f3d9952c640 1 -- 172.21.15.27:0/1007421783 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f3d8006bf50 msgr2=0x7f3d8006e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:25.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.132+0000 7f3d9952c640 1 --2- 172.21.15.27:0/1007421783 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f3d8006bf50 0x7f3d8006e410 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f3d84000f60 tx=0x7f3d84009040 comp rx=0 tx=0).stop 2024-09-12T03:20:25.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.132+0000 7f3d9952c640 1 -- 172.21.15.27:0/1007421783 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3d9415c2e0 msgr2=0x7f3d941f8510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:25.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.132+0000 7f3d9952c640 1 --2- 172.21.15.27:0/1007421783 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3d9415c2e0 0x7f3d941f8510 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f3d880098c0 tx=0x7f3d880046c0 comp rx=0 tx=0).stop 2024-09-12T03:20:25.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.132+0000 7f3d9952c640 1 -- 172.21.15.27:0/1007421783 shutdown_connections 2024-09-12T03:20:25.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.132+0000 7f3d9952c640 1 --2- 172.21.15.27:0/1007421783 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d9415c6c0 0x7f3d940b43b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:25.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.132+0000 7f3d9952c640 1 --2- 172.21.15.27:0/1007421783 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f3d8006bf50 0x7f3d8006e410 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:25.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.132+0000 7f3d9952c640 1 --2- 172.21.15.27:0/1007421783 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3d9415c2e0 0x7f3d941f8510 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:25.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.132+0000 7f3d9952c640 1 --2- 172.21.15.27:0/1007421783 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f3d940cfaf0 0x7f3d940ae8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:25.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.132+0000 7f3d9952c640 1 -- 172.21.15.27:0/1007421783 >> 172.21.15.27:0/1007421783 conn(0x7f3d9406f2c0 msgr2=0x7f3d940cf070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:25.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.132+0000 7f3d9952c640 1 -- 172.21.15.27:0/1007421783 shutdown_connections 2024-09-12T03:20:25.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.132+0000 7f3d9952c640 1 -- 172.21.15.27:0/1007421783 wait complete. 2024-09-12T03:20:25.147 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-12T03:20:25.255 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.255+0000 7fbde80ed640 1 Processor -- start 2024-09-12T03:20:25.256 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.256+0000 7fbde80ed640 1 -- start start 2024-09-12T03:20:25.256 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.257+0000 7fbde80ed640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fbde0073960 0x7fbde0162320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:25.256 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.257+0000 7fbde80ed640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fbde0162860 0x7fbde015e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:25.257 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.257+0000 7fbde80ed640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbde00d2240 con 0x7fbde015e5f0 2024-09-12T03:20:25.257 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.257+0000 7fbde80ed640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fbde00d0da0 con 0x7fbde0073960 2024-09-12T03:20:25.257 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.257+0000 7fbde80ed640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fbde0163090 con 0x7fbde0162860 2024-09-12T03:20:25.258 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.257+0000 7fbde668a640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fbde0162860 0x7fbde015e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:25.258 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.257+0000 7fbde5e89640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fbde015e5f0 0x7fbde0073250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.27:56224/0 (socket says 172.21.15.27:56224) 2024-09-12T03:20:25.258 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.257+0000 7fbde668a640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fbde0162860 0x7fbde015e9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3301/0 says I am v2:172.21.15.27:54072/0 (socket says 172.21.15.27:54072) 2024-09-12T03:20:25.258 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.257+0000 7fbde5688640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fbde0073960 0x7fbde0162320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:25.259 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.257+0000 7fbde5e89640 1 -- 172.21.15.27:0/964315509 learned_addr learned my addr 172.21.15.27:0/964315509 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:20:25.259 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.257+0000 7fbde4e87640 1 -- 172.21.15.27:0/964315509 <== mon.1 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 251221849 0 0) 0x7fbde00d2240 con 0x7fbde015e5f0 2024-09-12T03:20:25.259 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.257+0000 7fbde4e87640 1 -- 172.21.15.27:0/964315509 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbdc8003520 con 0x7fbde015e5f0 2024-09-12T03:20:25.259 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.257+0000 7fbde5688640 1 -- 172.21.15.27:0/964315509 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fbde0162860 msgr2=0x7fbde015e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:25.259 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.257+0000 7fbde5688640 1 --2- 172.21.15.27:0/964315509 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fbde0162860 0x7fbde015e9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:25.259 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.257+0000 7fbde5688640 1 -- 172.21.15.27:0/964315509 >> v1:172.21.15.45:6789/0 conn(0x7fbde015e5f0 legacy=0x7fbde0073250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:25.259 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.257+0000 7fbde5688640 1 -- 172.21.15.27:0/964315509 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbde00d0470 con 0x7fbde0073960 2024-09-12T03:20:25.259 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.257+0000 7fbde668a640 1 --2- 172.21.15.27:0/964315509 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fbde0162860 0x7fbde015e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:20:25.259 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.258+0000 7fbde5688640 1 --2- 172.21.15.27:0/964315509 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbde0073960 0x7fbde0162320 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fbde00d0f20 tx=0x7fbdd402a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8d043036e7e47117 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:25.259 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.258+0000 7fbde4e87640 1 -- 172.21.15.27:0/964315509 <== mon.1 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdd4037070 con 0x7fbde0073960 2024-09-12T03:20:25.259 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.258+0000 7fbde4e87640 1 -- 172.21.15.27:0/964315509 <== mon.1 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbdd4032440 con 0x7fbde0073960 2024-09-12T03:20:25.259 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.258+0000 7fbde4e87640 1 -- 172.21.15.27:0/964315509 <== mon.1 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdd4032810 con 0x7fbde0073960 2024-09-12T03:20:25.260 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.259+0000 7fbde80ed640 1 -- 172.21.15.27:0/964315509 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbde0073960 msgr2=0x7fbde0162320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:25.260 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.259+0000 7fbde80ed640 1 --2- 172.21.15.27:0/964315509 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbde0073960 0x7fbde0162320 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fbde00d0f20 tx=0x7fbdd402a180 comp rx=0 tx=0).stop 2024-09-12T03:20:25.261 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.259+0000 7fbde80ed640 1 -- 172.21.15.27:0/964315509 shutdown_connections 2024-09-12T03:20:25.261 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.259+0000 7fbde80ed640 1 --2- 172.21.15.27:0/964315509 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fbde0162860 0x7fbde015e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:25.261 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.259+0000 7fbde80ed640 1 --2- 172.21.15.27:0/964315509 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbde0073960 0x7fbde0162320 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:25.261 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.259+0000 7fbde80ed640 1 -- 172.21.15.27:0/964315509 >> 172.21.15.27:0/964315509 conn(0x7fbde006f300 msgr2=0x7fbde006f710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:25.261 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.259+0000 7fbde80ed640 1 -- 172.21.15.27:0/964315509 shutdown_connections 2024-09-12T03:20:25.261 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.259+0000 7fbde80ed640 1 -- 172.21.15.27:0/964315509 wait complete. 2024-09-12T03:20:25.261 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.260+0000 7fbde80ed640 1 Processor -- start 2024-09-12T03:20:25.261 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.260+0000 7fbde80ed640 1 -- start start 2024-09-12T03:20:25.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.261+0000 7fbde80ed640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbde0073960 0x7fbde01f8230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:25.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.261+0000 7fbde80ed640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbde015e5f0 0x7fbde015d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:25.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.261+0000 7fbde80ed640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fbde0162860 0x7fbde01f38d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:25.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.261+0000 7fbde80ed640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fbde00b5620 con 0x7fbde0073960 2024-09-12T03:20:25.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.261+0000 7fbde80ed640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fbde00d0a30 con 0x7fbde015e5f0 2024-09-12T03:20:25.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.261+0000 7fbde80ed640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fbde0162cb0 con 0x7fbde0162860 2024-09-12T03:20:25.264 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.261+0000 7fbde5e89640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbde0073960 0x7fbde01f8230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:25.264 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.261+0000 7fbde5e89640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbde0073960 0x7fbde01f8230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46904/0 (socket says 172.21.15.27:46904) 2024-09-12T03:20:25.264 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.261+0000 7fbde5e89640 1 -- 172.21.15.27:0/2833731907 learned_addr learned my addr 172.21.15.27:0/2833731907 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:20:25.264 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.261+0000 7fbde668a640 1 --2- 172.21.15.27:0/2833731907 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fbde0162860 0x7fbde01f38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:25.264 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.261+0000 7fbde5688640 1 --2- 172.21.15.27:0/2833731907 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbde015e5f0 0x7fbde015d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:25.264 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.261+0000 7fbde5e89640 1 -- 172.21.15.27:0/2833731907 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fbde0162860 msgr2=0x7fbde01f38d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:25.264 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.261+0000 7fbde5e89640 1 --2- 172.21.15.27:0/2833731907 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fbde0162860 0x7fbde01f38d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:25.264 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.261+0000 7fbde5e89640 1 -- 172.21.15.27:0/2833731907 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbde015e5f0 msgr2=0x7fbde015d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:25.264 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.261+0000 7fbde5e89640 1 --2- 172.21.15.27:0/2833731907 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbde015e5f0 0x7fbde015d210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:25.265 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.261+0000 7fbde5e89640 1 -- 172.21.15.27:0/2833731907 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbde00cf5c0 con 0x7fbde0073960 2024-09-12T03:20:25.265 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.261+0000 7fbde668a640 1 --2- 172.21.15.27:0/2833731907 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fbde0162860 0x7fbde01f38d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:20:25.265 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.262+0000 7fbde5e89640 1 --2- 172.21.15.27:0/2833731907 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbde0073960 0x7fbde01f8230 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fbdd000cb70 tx=0x7fbdd000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:25.265 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.262+0000 7fbdde7fc640 1 -- 172.21.15.27:0/2833731907 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdd0017070 con 0x7fbde0073960 2024-09-12T03:20:25.265 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.262+0000 7fbdde7fc640 1 -- 172.21.15.27:0/2833731907 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbdd000ad40 con 0x7fbde0073960 2024-09-12T03:20:25.265 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.262+0000 7fbde80ed640 1 -- 172.21.15.27:0/2833731907 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbde00cf8a0 con 0x7fbde0073960 2024-09-12T03:20:25.265 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.262+0000 7fbdde7fc640 1 -- 172.21.15.27:0/2833731907 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdd0012620 con 0x7fbde0073960 2024-09-12T03:20:25.265 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.262+0000 7fbde80ed640 1 -- 172.21.15.27:0/2833731907 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbde01f7a30 con 0x7fbde0073960 2024-09-12T03:20:25.265 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.263+0000 7fbde80ed640 1 -- 172.21.15.27:0/2833731907 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbde0162320 con 0x7fbde0073960 2024-09-12T03:20:25.265 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.263+0000 7fbdde7fc640 1 -- 172.21.15.27:0/2833731907 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 90600+0+0 (secure 0 0 0) 0x7fbdd001a070 con 0x7fbde0073960 2024-09-12T03:20:25.266 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.264+0000 7fbdde7fc640 1 --2- 172.21.15.27:0/2833731907 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fbdcc06bf80 0x7fbdcc06e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:25.266 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.264+0000 7fbdde7fc640 1 -- 172.21.15.27:0/2833731907 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 4857+0+0 (secure 0 0 0) 0x7fbdd008aa10 con 0x7fbde0073960 2024-09-12T03:20:25.266 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.264+0000 7fbde5688640 1 --2- 172.21.15.27:0/2833731907 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fbdcc06bf80 0x7fbdcc06e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:25.266 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.264+0000 7fbde5688640 1 --2- 172.21.15.27:0/2833731907 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fbdcc06bf80 0x7fbdcc06e440 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fbdd4034000 tx=0x7fbdd402e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:25.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.268+0000 7fbdde7fc640 1 -- 172.21.15.27:0/2833731907 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fbdd0059690 con 0x7fbde0073960 2024-09-12T03:20:25.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.510+0000 7fbde80ed640 1 -- 172.21.15.27:0/2833731907 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7fbde0159c70 con 0x7fbde0073960 2024-09-12T03:20:25.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.510+0000 7fbdde7fc640 1 -- 172.21.15.27:0/2833731907 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v16) ==== 80+0+5304 (secure 0 0 0) 0x7fbdd0058f90 con 0x7fbde0073960 2024-09-12T03:20:25.511 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:20:25.511 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-12T03:20:25.511 INFO:teuthology.orchestra.run.smithi027.stdout: "auth_dump": [ 2024-09-12T03:20:25.511 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:20:25.511 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "mds.a", 2024-09-12T03:20:25.511 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDUXOJmIhxjNxAAaGg8SwtW6NRe/lIgNCV20w==", 2024-09-12T03:20:25.511 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:20:25.511 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow", 2024-09-12T03:20:25.511 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow *", 2024-09-12T03:20:25.511 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow *", 2024-09-12T03:20:25.512 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2024-09-12T03:20:25.512 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:20:25.512 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:25.512 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:20:25.512 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "mds.b", 2024-09-12T03:20:25.512 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDVXOJm63kWCBAAzjfOsBsWUI+IAsuZ49yEkw==", 2024-09-12T03:20:25.512 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:20:25.512 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow", 2024-09-12T03:20:25.512 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow *", 2024-09-12T03:20:25.512 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow *", 2024-09-12T03:20:25.512 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2024-09-12T03:20:25.512 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:20:25.512 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:25.512 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:20:25.513 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "mds.c", 2024-09-12T03:20:25.513 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDVXOJm0sX7AhAAdynCcho2a/AgFF9Cw3+vAw==", 2024-09-12T03:20:25.513 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:20:25.513 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow", 2024-09-12T03:20:25.513 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow *", 2024-09-12T03:20:25.513 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow *", 2024-09-12T03:20:25.513 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2024-09-12T03:20:25.513 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:20:25.513 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:25.513 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:20:25.513 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "mds.d", 2024-09-12T03:20:25.513 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDVXOJm7Dd8DxAANHYmvKgpRZ6sVfXDSgPHWw==", 2024-09-12T03:20:25.514 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:20:25.514 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow", 2024-09-12T03:20:25.514 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow *", 2024-09-12T03:20:25.514 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow *", 2024-09-12T03:20:25.514 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2024-09-12T03:20:25.515 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:20:25.515 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:25.515 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:20:25.515 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "osd.0", 2024-09-12T03:20:25.515 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDYXOJmJTVYLxAAmo8kxRjaiwTT44dZLZlq4A==", 2024-09-12T03:20:25.515 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:20:25.515 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow profile osd", 2024-09-12T03:20:25.515 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile osd", 2024-09-12T03:20:25.515 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2024-09-12T03:20:25.515 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:20:25.515 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:25.515 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:20:25.515 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "osd.1", 2024-09-12T03:20:25.515 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDcXOJmYV0BFBAAmoGX9gf9yLoE/qkWFnv9tg==", 2024-09-12T03:20:25.516 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:20:25.516 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow profile osd", 2024-09-12T03:20:25.516 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile osd", 2024-09-12T03:20:25.516 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2024-09-12T03:20:25.516 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:20:25.516 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:25.516 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:20:25.516 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "osd.2", 2024-09-12T03:20:25.516 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDfXOJmEetiJRAAXk+AvfTfhjsB6T2UB4Z2/w==", 2024-09-12T03:20:25.516 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:20:25.517 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow profile osd", 2024-09-12T03:20:25.517 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile osd", 2024-09-12T03:20:25.517 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2024-09-12T03:20:25.517 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:20:25.517 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:25.517 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:20:25.517 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "osd.3", 2024-09-12T03:20:25.517 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDiXOJmrQbPJxAAQsi2B5a1LnPTPGZVdo7j7w==", 2024-09-12T03:20:25.517 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:20:25.517 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow profile osd", 2024-09-12T03:20:25.517 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile osd", 2024-09-12T03:20:25.517 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2024-09-12T03:20:25.517 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:20:25.517 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:25.518 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:20:25.518 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "osd.4", 2024-09-12T03:20:25.518 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDnXOJmoMr1NxAAv3u2q8hYt9w//9gz+N7UAQ==", 2024-09-12T03:20:25.519 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:20:25.519 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow profile osd", 2024-09-12T03:20:25.519 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile osd", 2024-09-12T03:20:25.519 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2024-09-12T03:20:25.519 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:20:25.519 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:25.519 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:20:25.520 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "osd.5", 2024-09-12T03:20:25.520 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDqXOJmxpUPOxAAxLH6RJsyLOZ2XMfzM0fQ1g==", 2024-09-12T03:20:25.520 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:20:25.520 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow profile osd", 2024-09-12T03:20:25.520 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile osd", 2024-09-12T03:20:25.520 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2024-09-12T03:20:25.520 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:20:25.520 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:25.520 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:20:25.520 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "osd.6", 2024-09-12T03:20:25.520 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDuXOJmuP0+LxAAMHoz5eSO81hPtkdBIS2cig==", 2024-09-12T03:20:25.520 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:20:25.520 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow profile osd", 2024-09-12T03:20:25.520 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile osd", 2024-09-12T03:20:25.521 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2024-09-12T03:20:25.521 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:20:25.521 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:25.521 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:20:25.521 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "osd.7", 2024-09-12T03:20:25.521 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDxXOJmuIC/MRAAPDM+lAHqdoQyGVODwjFp+Q==", 2024-09-12T03:20:25.521 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:20:25.522 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow profile osd", 2024-09-12T03:20:25.522 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile osd", 2024-09-12T03:20:25.522 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2024-09-12T03:20:25.522 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:20:25.522 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:25.522 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:20:25.522 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "client.0", 2024-09-12T03:20:25.522 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDVXOJmLm+5ExAAECM4CKfuVElmp/n0b02mXg==", 2024-09-12T03:20:25.522 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:20:25.522 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow", 2024-09-12T03:20:25.522 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow r", 2024-09-12T03:20:25.522 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-12T03:20:25.522 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:20:25.523 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:25.523 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:20:25.523 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "client.1", 2024-09-12T03:20:25.523 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDVXOJm61/FHBAALCQouXySfQeZ5DESGmqkzA==", 2024-09-12T03:20:25.523 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:20:25.523 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow", 2024-09-12T03:20:25.523 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow r", 2024-09-12T03:20:25.523 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow rw", 2024-09-12T03:20:25.523 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow rwx" 2024-09-12T03:20:25.523 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:20:25.523 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:25.523 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:20:25.523 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "client.2", 2024-09-12T03:20:25.523 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDVXOJmq/ZkGRAAOzmQzEIhb4b/YTxayc34Bw==", 2024-09-12T03:20:25.524 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:20:25.524 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow", 2024-09-12T03:20:25.524 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow r", 2024-09-12T03:20:25.524 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow rw", 2024-09-12T03:20:25.524 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow rwx" 2024-09-12T03:20:25.524 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:20:25.524 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:25.524 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:20:25.524 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "client.3", 2024-09-12T03:20:25.524 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDVXOJm+DlzIhAATQKZe+Zyk0MM8efTa6g9PA==", 2024-09-12T03:20:25.524 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:20:25.524 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow", 2024-09-12T03:20:25.524 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow r", 2024-09-12T03:20:25.525 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow rw", 2024-09-12T03:20:25.525 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow rwx" 2024-09-12T03:20:25.525 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:20:25.525 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:25.525 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:20:25.525 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "client.admin", 2024-09-12T03:20:25.525 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDUXOJm+JGWGhAApTQw9RtJWd/J5QzQaUFXvg==", 2024-09-12T03:20:25.525 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:20:25.525 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow *", 2024-09-12T03:20:25.525 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow *", 2024-09-12T03:20:25.525 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow *", 2024-09-12T03:20:25.525 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2024-09-12T03:20:25.525 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:20:25.525 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:25.526 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:20:25.526 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "mgr.x", 2024-09-12T03:20:25.526 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDUXOJmZcsONBAAMTY2KykuHaEkv1t6Vz5KjA==", 2024-09-12T03:20:25.526 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:20:25.526 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow *", 2024-09-12T03:20:25.526 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile mgr", 2024-09-12T03:20:25.526 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2024-09-12T03:20:25.526 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:20:25.526 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:25.526 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:20:25.526 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "mgr.y", 2024-09-12T03:20:25.526 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDUXOJmJOgZMBAAQ/JTxfhox7HgxBMtsvUCBg==", 2024-09-12T03:20:25.526 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:20:25.526 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow *", 2024-09-12T03:20:25.527 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile mgr", 2024-09-12T03:20:25.527 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2024-09-12T03:20:25.527 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:20:25.527 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:20:25.527 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-12T03:20:25.527 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-12T03:20:25.528 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.512+0000 7fbde80ed640 1 -- 172.21.15.27:0/2833731907 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fbdcc06bf80 msgr2=0x7fbdcc06e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:25.528 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.512+0000 7fbde80ed640 1 --2- 172.21.15.27:0/2833731907 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fbdcc06bf80 0x7fbdcc06e440 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fbdd4034000 tx=0x7fbdd402e000 comp rx=0 tx=0).stop 2024-09-12T03:20:25.528 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.512+0000 7fbde80ed640 1 -- 172.21.15.27:0/2833731907 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbde0073960 msgr2=0x7fbde01f8230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:25.529 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.512+0000 7fbde80ed640 1 --2- 172.21.15.27:0/2833731907 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbde0073960 0x7fbde01f8230 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fbdd000cb70 tx=0x7fbdd000a430 comp rx=0 tx=0).stop 2024-09-12T03:20:25.529 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.512+0000 7fbde80ed640 1 -- 172.21.15.27:0/2833731907 shutdown_connections 2024-09-12T03:20:25.529 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.512+0000 7fbde80ed640 1 --2- 172.21.15.27:0/2833731907 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fbde0162860 0x7fbde01f38d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:25.529 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.512+0000 7fbde80ed640 1 --2- 172.21.15.27:0/2833731907 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbde015e5f0 0x7fbde015d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:25.529 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.512+0000 7fbde80ed640 1 --2- 172.21.15.27:0/2833731907 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fbdcc06bf80 0x7fbdcc06e440 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:25.529 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.512+0000 7fbde80ed640 1 --2- 172.21.15.27:0/2833731907 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbde0073960 0x7fbde01f8230 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:25.529 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.512+0000 7fbde80ed640 1 -- 172.21.15.27:0/2833731907 >> 172.21.15.27:0/2833731907 conn(0x7fbde006f300 msgr2=0x7fbde006e140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:25.529 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.512+0000 7fbde80ed640 1 -- 172.21.15.27:0/2833731907 shutdown_connections 2024-09-12T03:20:25.529 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.512+0000 7fbde80ed640 1 -- 172.21.15.27:0/2833731907 wait complete. 2024-09-12T03:20:25.530 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-12T03:20:25.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.633+0000 7f2cd4e5e640 1 Processor -- start 2024-09-12T03:20:25.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.633+0000 7f2cd4e5e640 1 -- start start 2024-09-12T03:20:25.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.634+0000 7f2cd4e5e640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2cd006f360 0x7f2cd00cef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:25.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.634+0000 7f2cd4e5e640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f2cd00cf490 0x7f2cd00d2780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:25.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.634+0000 7f2cd4e5e640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2cd006cc40 con 0x7f2cd00b1ca0 2024-09-12T03:20:25.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.634+0000 7f2cd4e5e640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f2cd00d22f0 con 0x7f2cd00cf490 2024-09-12T03:20:25.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.634+0000 7f2cd4e5e640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f2cd00d1ce0 con 0x7f2cd006f360 2024-09-12T03:20:25.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.634+0000 7f2cced9d640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f2cd00cf490 0x7f2cd00d2780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:25.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.634+0000 7f2cce59c640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f2cd00b1ca0 0x7f2cd015f830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:34006/0 (socket says 172.21.15.27:34006) 2024-09-12T03:20:25.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.634+0000 7f2cce59c640 1 -- 172.21.15.27:0/2082999643 learned_addr learned my addr 172.21.15.27:0/2082999643 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:20:25.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.634+0000 7f2ccdd9b640 1 --2- 172.21.15.27:0/2082999643 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2cd006f360 0x7f2cd00cef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:25.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.634+0000 7f2cced9d640 1 -- 172.21.15.27:0/2082999643 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2cd006f360 msgr2=0x7f2cd00cef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:25.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.634+0000 7f2cced9d640 1 --2- 172.21.15.27:0/2082999643 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2cd006f360 0x7f2cd00cef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:25.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.634+0000 7f2cced9d640 1 -- 172.21.15.27:0/2082999643 >> v1:172.21.15.27:6789/0 conn(0x7f2cd00b1ca0 legacy=0x7f2cd015f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:25.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.634+0000 7f2cced9d640 1 -- 172.21.15.27:0/2082999643 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cd006cee0 con 0x7f2cd00cf490 2024-09-12T03:20:25.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.635+0000 7f2cced9d640 1 --2- 172.21.15.27:0/2082999643 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2cd00cf490 0x7f2cd00d2780 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f2cc80097a0 tx=0x7f2cc802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a37aa0cf01c1ae4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:25.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.635+0000 7f2ccd59a640 1 -- 172.21.15.27:0/2082999643 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cc8037070 con 0x7f2cd00cf490 2024-09-12T03:20:25.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.635+0000 7f2ccd59a640 1 -- 172.21.15.27:0/2082999643 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2cc8032440 con 0x7f2cd00cf490 2024-09-12T03:20:25.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.635+0000 7f2ccd59a640 1 -- 172.21.15.27:0/2082999643 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cc8032810 con 0x7f2cd00cf490 2024-09-12T03:20:25.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.635+0000 7f2cd4e5e640 1 -- 172.21.15.27:0/2082999643 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2cd00cf490 msgr2=0x7f2cd00d2780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:25.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.635+0000 7f2cd4e5e640 1 --2- 172.21.15.27:0/2082999643 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2cd00cf490 0x7f2cd00d2780 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f2cc80097a0 tx=0x7f2cc802a180 comp rx=0 tx=0).stop 2024-09-12T03:20:25.638 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.636+0000 7f2cd4e5e640 1 -- 172.21.15.27:0/2082999643 shutdown_connections 2024-09-12T03:20:25.638 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.636+0000 7f2cd4e5e640 1 --2- 172.21.15.27:0/2082999643 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2cd00cf490 0x7f2cd00d2780 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:25.638 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.636+0000 7f2cd4e5e640 1 --2- 172.21.15.27:0/2082999643 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2cd006f360 0x7f2cd00cef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:25.638 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.636+0000 7f2cd4e5e640 1 -- 172.21.15.27:0/2082999643 >> 172.21.15.27:0/2082999643 conn(0x7f2cd0073800 msgr2=0x7f2cd009ba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:25.638 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.636+0000 7f2cd4e5e640 1 -- 172.21.15.27:0/2082999643 shutdown_connections 2024-09-12T03:20:25.638 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.636+0000 7f2cd4e5e640 1 -- 172.21.15.27:0/2082999643 wait complete. 2024-09-12T03:20:25.642 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.637+0000 7f2cd4e5e640 1 Processor -- start 2024-09-12T03:20:25.642 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.637+0000 7f2cd4e5e640 1 -- start start 2024-09-12T03:20:25.642 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.638+0000 7f2cd4e5e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2cd006f360 0x7f2cd015d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:25.642 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.638+0000 7f2cd4e5e640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2cd00b1ca0 0x7f2cd01602a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:25.642 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.638+0000 7f2cd4e5e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2cd00cf490 0x7f2cd01607e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:25.642 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.638+0000 7f2cd4e5e640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f2cd006d340 con 0x7f2cd00cf490 2024-09-12T03:20:25.642 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.638+0000 7f2cd4e5e640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f2cd006d1c0 con 0x7f2cd006f360 2024-09-12T03:20:25.642 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.638+0000 7f2cd4e5e640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f2cd00cfc40 con 0x7f2cd00b1ca0 2024-09-12T03:20:25.642 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.638+0000 7f2cced9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2cd00cf490 0x7f2cd01607e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:25.642 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.638+0000 7f2cced9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2cd00cf490 0x7f2cd01607e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46912/0 (socket says 172.21.15.27:46912) 2024-09-12T03:20:25.643 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.638+0000 7f2cced9d640 1 -- 172.21.15.27:0/2718951071 learned_addr learned my addr 172.21.15.27:0/2718951071 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:20:25.643 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.638+0000 7f2cce59c640 1 --2- 172.21.15.27:0/2718951071 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2cd006f360 0x7f2cd015d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:25.643 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.638+0000 7f2ccdd9b640 1 --2- 172.21.15.27:0/2718951071 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2cd00b1ca0 0x7f2cd01602a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:25.643 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.638+0000 7f2cced9d640 1 -- 172.21.15.27:0/2718951071 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2cd00b1ca0 msgr2=0x7f2cd01602a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:25.643 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.638+0000 7f2cced9d640 1 --2- 172.21.15.27:0/2718951071 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2cd00b1ca0 0x7f2cd01602a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:25.643 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.638+0000 7f2cced9d640 1 -- 172.21.15.27:0/2718951071 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2cd006f360 msgr2=0x7f2cd015d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:25.643 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.638+0000 7f2cced9d640 1 --2- 172.21.15.27:0/2718951071 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2cd006f360 0x7f2cd015d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:25.643 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.638+0000 7f2cced9d640 1 -- 172.21.15.27:0/2718951071 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cd00cd3a0 con 0x7f2cd00cf490 2024-09-12T03:20:25.643 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.638+0000 7f2cced9d640 1 --2- 172.21.15.27:0/2718951071 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2cd00cf490 0x7f2cd01607e0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f2cc80097a0 tx=0x7f2cc80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:25.643 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.639+0000 7f2cc6ffd640 1 -- 172.21.15.27:0/2718951071 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cc8041070 con 0x7f2cd00cf490 2024-09-12T03:20:25.643 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.639+0000 7f2cc6ffd640 1 -- 172.21.15.27:0/2718951071 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2cc803c870 con 0x7f2cd00cf490 2024-09-12T03:20:25.643 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.639+0000 7f2cd4e5e640 1 -- 172.21.15.27:0/2718951071 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2cd00cd610 con 0x7f2cd00cf490 2024-09-12T03:20:25.643 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.639+0000 7f2cc6ffd640 1 -- 172.21.15.27:0/2718951071 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cc8037040 con 0x7f2cd00cf490 2024-09-12T03:20:25.643 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.639+0000 7f2cd4e5e640 1 -- 172.21.15.27:0/2718951071 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2cd00cdeb0 con 0x7f2cd00cf490 2024-09-12T03:20:25.644 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.640+0000 7f2cd4e5e640 1 -- 172.21.15.27:0/2718951071 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2cd006f7a0 con 0x7f2cd00cf490 2024-09-12T03:20:25.644 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.640+0000 7f2cc6ffd640 1 -- 172.21.15.27:0/2718951071 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 90600+0+0 (secure 0 0 0) 0x7f2cc8044070 con 0x7f2cd00cf490 2024-09-12T03:20:25.644 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.641+0000 7f2cc6ffd640 1 --2- 172.21.15.27:0/2718951071 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f2cb006bf80 0x7f2cb006e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:25.644 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.641+0000 7f2cc6ffd640 1 -- 172.21.15.27:0/2718951071 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 4857+0+0 (secure 0 0 0) 0x7f2cc80ab6f0 con 0x7f2cd00cf490 2024-09-12T03:20:25.644 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.641+0000 7f2cce59c640 1 --2- 172.21.15.27:0/2718951071 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f2cb006bf80 0x7f2cb006e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:25.644 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.641+0000 7f2cce59c640 1 --2- 172.21.15.27:0/2718951071 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f2cb006bf80 0x7f2cb006e440 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f2cc00012e0 tx=0x7f2cc0001a30 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:25.645 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.645+0000 7f2cc6ffd640 1 -- 172.21.15.27:0/2718951071 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f2cc8079db0 con 0x7f2cd00cf490 2024-09-12T03:20:25.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.825+0000 7f2cd4e5e640 1 -- 172.21.15.27:0/2718951071 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f2cd00711c0 con 0x7f2cd00cf490 2024-09-12T03:20:25.826 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.826+0000 7f2cc6ffd640 1 -- 172.21.15.27:0/2718951071 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v77) ==== 84+0+65 (secure 0 0 0) 0x7f2cc807e0b0 con 0x7f2cd00cf490 2024-09-12T03:20:25.826 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:20:25.826 INFO:teuthology.orchestra.run.smithi027.stdout:[ 2024-09-12T03:20:25.826 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:20:25.826 INFO:teuthology.orchestra.run.smithi027.stdout: "poolnum": 1, 2024-09-12T03:20:25.826 INFO:teuthology.orchestra.run.smithi027.stdout: "poolname": ".mgr" 2024-09-12T03:20:25.826 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:20:25.826 INFO:teuthology.orchestra.run.smithi027.stdout:] 2024-09-12T03:20:25.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.828+0000 7f2cd4e5e640 1 -- 172.21.15.27:0/2718951071 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f2cb006bf80 msgr2=0x7f2cb006e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:25.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.828+0000 7f2cd4e5e640 1 --2- 172.21.15.27:0/2718951071 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f2cb006bf80 0x7f2cb006e440 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f2cc00012e0 tx=0x7f2cc0001a30 comp rx=0 tx=0).stop 2024-09-12T03:20:25.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.828+0000 7f2cd4e5e640 1 -- 172.21.15.27:0/2718951071 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2cd00cf490 msgr2=0x7f2cd01607e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:25.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.828+0000 7f2cd4e5e640 1 --2- 172.21.15.27:0/2718951071 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2cd00cf490 0x7f2cd01607e0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f2cc80097a0 tx=0x7f2cc80046c0 comp rx=0 tx=0).stop 2024-09-12T03:20:25.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.828+0000 7f2cd4e5e640 1 -- 172.21.15.27:0/2718951071 shutdown_connections 2024-09-12T03:20:25.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.828+0000 7f2cd4e5e640 1 --2- 172.21.15.27:0/2718951071 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f2cb006bf80 0x7f2cb006e440 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:25.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.828+0000 7f2cd4e5e640 1 --2- 172.21.15.27:0/2718951071 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2cd00cf490 0x7f2cd01607e0 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:25.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.828+0000 7f2cd4e5e640 1 --2- 172.21.15.27:0/2718951071 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2cd00b1ca0 0x7f2cd01602a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:25.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.828+0000 7f2cd4e5e640 1 --2- 172.21.15.27:0/2718951071 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2cd006f360 0x7f2cd015d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:25.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.828+0000 7f2cd4e5e640 1 -- 172.21.15.27:0/2718951071 >> 172.21.15.27:0/2718951071 conn(0x7f2cd0073800 msgr2=0x7f2cd00d0440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:25.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.828+0000 7f2cd4e5e640 1 -- 172.21.15.27:0/2718951071 shutdown_connections 2024-09-12T03:20:25.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.828+0000 7f2cd4e5e640 1 -- 172.21.15.27:0/2718951071 wait complete. 2024-09-12T03:20:25.838 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-12T03:20:25.838 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-12T03:20:25.945 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.945+0000 7fe4761d6640 1 Processor -- start 2024-09-12T03:20:25.945 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.945+0000 7fe4761d6640 1 -- start start 2024-09-12T03:20:25.947 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.946+0000 7fe4761d6640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fe47015e650 0x7fe4700d1d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:25.947 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.946+0000 7fe4761d6640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe47006db30 0x7fe47006df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:25.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.946+0000 7fe4761d6640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe4700d2280 con 0x7fe47006cc40 2024-09-12T03:20:25.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.946+0000 7fe4761d6640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fe47006d850 con 0x7fe47015e650 2024-09-12T03:20:25.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.946+0000 7fe4761d6640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fe4700d1260 con 0x7fe47006db30 2024-09-12T03:20:25.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.946+0000 7fe46ffff640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe47006db30 0x7fe47006df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:25.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.946+0000 7fe46f7fe640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fe47015e650 0x7fe4700d1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:25.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.946+0000 7fe46effd640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fe47006cc40 0x7fe47006d090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.27:56228/0 (socket says 172.21.15.27:56228) 2024-09-12T03:20:25.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.946+0000 7fe46effd640 1 -- 172.21.15.27:0/3170682584 learned_addr learned my addr 172.21.15.27:0/3170682584 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:20:25.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.947+0000 7fe46ffff640 1 -- 172.21.15.27:0/3170682584 >> v2:172.21.15.45:3300/0 conn(0x7fe47015e650 msgr2=0x7fe4700d1d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:25.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.947+0000 7fe46e7fc640 1 -- 172.21.15.27:0/3170682584 <== mon.1 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1615156254 0 0) 0x7fe4700d2280 con 0x7fe47006cc40 2024-09-12T03:20:25.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.947+0000 7fe46ffff640 1 --2- 172.21.15.27:0/3170682584 >> v2:172.21.15.45:3300/0 conn(0x7fe47015e650 0x7fe4700d1d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:25.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.947+0000 7fe46ffff640 1 -- 172.21.15.27:0/3170682584 >> v1:172.21.15.45:6789/0 conn(0x7fe47006cc40 legacy=0x7fe47006d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:25.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.947+0000 7fe46ffff640 1 -- 172.21.15.27:0/3170682584 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4700d1540 con 0x7fe47006db30 2024-09-12T03:20:25.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.948+0000 7fe46ffff640 1 --2- 172.21.15.27:0/3170682584 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe47006db30 0x7fe47006df30 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fe468009d90 tx=0x7fe46802a5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=257fc11d36c05325 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:25.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.948+0000 7fe46e7fc640 1 -- 172.21.15.27:0/3170682584 <== mon.2 v2:172.21.15.45:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe468031600 con 0x7fe47006db30 2024-09-12T03:20:25.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.948+0000 7fe46e7fc640 1 -- 172.21.15.27:0/3170682584 <== mon.2 v2:172.21.15.45:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4680317a0 con 0x7fe47006db30 2024-09-12T03:20:25.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.948+0000 7fe46e7fc640 1 -- 172.21.15.27:0/3170682584 <== mon.2 v2:172.21.15.45:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe468031b70 con 0x7fe47006db30 2024-09-12T03:20:25.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.948+0000 7fe4761d6640 1 -- 172.21.15.27:0/3170682584 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe47006db30 msgr2=0x7fe47006df30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:25.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.948+0000 7fe4761d6640 1 --2- 172.21.15.27:0/3170682584 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe47006db30 0x7fe47006df30 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fe468009d90 tx=0x7fe46802a5d0 comp rx=0 tx=0).stop 2024-09-12T03:20:25.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.948+0000 7fe4761d6640 1 -- 172.21.15.27:0/3170682584 shutdown_connections 2024-09-12T03:20:25.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.948+0000 7fe4761d6640 1 --2- 172.21.15.27:0/3170682584 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe47006db30 0x7fe47006df30 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:25.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.948+0000 7fe4761d6640 1 --2- 172.21.15.27:0/3170682584 >> v2:172.21.15.45:3300/0 conn(0x7fe47015e650 0x7fe4700d1d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:25.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.948+0000 7fe4761d6640 1 -- 172.21.15.27:0/3170682584 >> 172.21.15.27:0/3170682584 conn(0x7fe47006f320 msgr2=0x7fe47006f700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:25.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.948+0000 7fe4761d6640 1 -- 172.21.15.27:0/3170682584 shutdown_connections 2024-09-12T03:20:25.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.949+0000 7fe4761d6640 1 -- 172.21.15.27:0/3170682584 wait complete. 2024-09-12T03:20:25.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.950+0000 7fe4761d6640 1 Processor -- start 2024-09-12T03:20:25.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.950+0000 7fe4761d6640 1 -- start start 2024-09-12T03:20:25.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.951+0000 7fe4761d6640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe47006cc40 0x7fe4701e9220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:25.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.951+0000 7fe4761d6640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe47006db30 0x7fe4701e6f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:25.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.951+0000 7fe4761d6640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe47015e650 0x7fe4700cede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:25.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.951+0000 7fe4761d6640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fe47006e5f0 con 0x7fe47015e650 2024-09-12T03:20:25.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.951+0000 7fe4761d6640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe47006e470 con 0x7fe47006cc40 2024-09-12T03:20:25.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.951+0000 7fe4761d6640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fe4700b55b0 con 0x7fe47006db30 2024-09-12T03:20:25.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.951+0000 7fe46ffff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe47015e650 0x7fe4700cede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:25.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.951+0000 7fe46ffff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe47015e650 0x7fe4700cede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46922/0 (socket says 172.21.15.27:46922) 2024-09-12T03:20:25.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.951+0000 7fe46ffff640 1 -- 172.21.15.27:0/369932517 learned_addr learned my addr 172.21.15.27:0/369932517 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:20:25.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.951+0000 7fe46f7fe640 1 --2- 172.21.15.27:0/369932517 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe47006cc40 0x7fe4701e9220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:25.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.951+0000 7fe46effd640 1 --2- 172.21.15.27:0/369932517 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe47006db30 0x7fe4701e6f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:25.959 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.951+0000 7fe46ffff640 1 -- 172.21.15.27:0/369932517 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe47006db30 msgr2=0x7fe4701e6f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:25.959 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.951+0000 7fe46ffff640 1 --2- 172.21.15.27:0/369932517 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe47006db30 0x7fe4701e6f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:25.959 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.951+0000 7fe46ffff640 1 -- 172.21.15.27:0/369932517 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe47006cc40 msgr2=0x7fe4701e9220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:25.959 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.951+0000 7fe46ffff640 1 --2- 172.21.15.27:0/369932517 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe47006cc40 0x7fe4701e9220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:25.959 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.951+0000 7fe46ffff640 1 -- 172.21.15.27:0/369932517 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4701e76e0 con 0x7fe47015e650 2024-09-12T03:20:25.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.951+0000 7fe46ffff640 1 --2- 172.21.15.27:0/369932517 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe47015e650 0x7fe4700cede0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fe468009d90 tx=0x7fe468004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:25.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.951+0000 7fe457fff640 1 -- 172.21.15.27:0/369932517 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe468040070 con 0x7fe47015e650 2024-09-12T03:20:25.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.952+0000 7fe457fff640 1 -- 172.21.15.27:0/369932517 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe468003b90 con 0x7fe47015e650 2024-09-12T03:20:25.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.952+0000 7fe4761d6640 1 -- 172.21.15.27:0/369932517 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4701e79c0 con 0x7fe47015e650 2024-09-12T03:20:25.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.952+0000 7fe457fff640 1 -- 172.21.15.27:0/369932517 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe46803c6f0 con 0x7fe47015e650 2024-09-12T03:20:25.961 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.952+0000 7fe4761d6640 1 -- 172.21.15.27:0/369932517 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4701e7f80 con 0x7fe47015e650 2024-09-12T03:20:25.961 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.953+0000 7fe4761d6640 1 -- 172.21.15.27:0/369932517 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe47006df30 con 0x7fe47015e650 2024-09-12T03:20:25.961 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.953+0000 7fe457fff640 1 -- 172.21.15.27:0/369932517 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 90600+0+0 (secure 0 0 0) 0x7fe468039040 con 0x7fe47015e650 2024-09-12T03:20:25.961 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.954+0000 7fe457fff640 1 --2- 172.21.15.27:0/369932517 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fe45c06bf50 0x7fe45c06e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:25.961 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.954+0000 7fe457fff640 1 -- 172.21.15.27:0/369932517 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 4857+0+0 (secure 0 0 0) 0x7fe4680ab870 con 0x7fe47015e650 2024-09-12T03:20:25.962 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.954+0000 7fe46f7fe640 1 --2- 172.21.15.27:0/369932517 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fe45c06bf50 0x7fe45c06e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:25.962 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.954+0000 7fe46f7fe640 1 --2- 172.21.15.27:0/369932517 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fe45c06bf50 0x7fe45c06e410 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fe460009aa0 tx=0x7fe460009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:25.962 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:25.958+0000 7fe457fff640 1 -- 172.21.15.27:0/369932517 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fe468079d90 con 0x7fe47015e650 2024-09-12T03:20:26.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:26.136+0000 7fe4761d6640 1 -- 172.21.15.27:0/369932517 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fe4700b7020 con 0x7fe47015e650 2024-09-12T03:20:27.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:27.136+0000 7fe457fff640 1 -- 172.21.15.27:0/369932517 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v78) ==== 138+0+0 (secure 0 0 0) 0x7fe46807e090 con 0x7fe47015e650 2024-09-12T03:20:27.136 INFO:teuthology.orchestra.run.smithi027.stderr:pool 'cephfs_metadata' created 2024-09-12T03:20:27.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:27.139+0000 7fe4761d6640 1 -- 172.21.15.27:0/369932517 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fe45c06bf50 msgr2=0x7fe45c06e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:27.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:27.139+0000 7fe4761d6640 1 --2- 172.21.15.27:0/369932517 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fe45c06bf50 0x7fe45c06e410 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fe460009aa0 tx=0x7fe460009190 comp rx=0 tx=0).stop 2024-09-12T03:20:27.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:27.139+0000 7fe4761d6640 1 -- 172.21.15.27:0/369932517 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe47015e650 msgr2=0x7fe4700cede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:27.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:27.139+0000 7fe4761d6640 1 --2- 172.21.15.27:0/369932517 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe47015e650 0x7fe4700cede0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fe468009d90 tx=0x7fe468004a40 comp rx=0 tx=0).stop 2024-09-12T03:20:27.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:27.139+0000 7fe4761d6640 1 -- 172.21.15.27:0/369932517 shutdown_connections 2024-09-12T03:20:27.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:27.139+0000 7fe4761d6640 1 --2- 172.21.15.27:0/369932517 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fe45c06bf50 0x7fe45c06e410 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:27.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:27.139+0000 7fe4761d6640 1 --2- 172.21.15.27:0/369932517 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe47015e650 0x7fe4700cede0 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:27.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:27.139+0000 7fe4761d6640 1 --2- 172.21.15.27:0/369932517 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe47006db30 0x7fe4701e6f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:27.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:27.139+0000 7fe4761d6640 1 --2- 172.21.15.27:0/369932517 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe47006cc40 0x7fe4701e9220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:27.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:27.139+0000 7fe4761d6640 1 -- 172.21.15.27:0/369932517 >> 172.21.15.27:0/369932517 conn(0x7fe47006f320 msgr2=0x7fe4700b45c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:27.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:27.139+0000 7fe4761d6640 1 -- 172.21.15.27:0/369932517 shutdown_connections 2024-09-12T03:20:27.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:27.139+0000 7fe4761d6640 1 -- 172.21.15.27:0/369932517 wait complete. 2024-09-12T03:20:27.157 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-12T03:20:27.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:27.272+0000 7f47c40a5640 1 Processor -- start 2024-09-12T03:20:27.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:27.273+0000 7f47c40a5640 1 -- start start 2024-09-12T03:20:27.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:27.274+0000 7f47c40a5640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f47bc15e650 0x7f47bc0d1d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:27.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:27.274+0000 7f47c40a5640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f47bc06db30 0x7f47bc06df80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:27.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:27.274+0000 7f47c40a5640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f47bc0d2280 con 0x7f47bc06cc40 2024-09-12T03:20:27.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:27.274+0000 7f47c40a5640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f47bc06d850 con 0x7f47bc06db30 2024-09-12T03:20:27.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:27.274+0000 7f47c40a5640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f47bc0d1260 con 0x7f47bc15e650 2024-09-12T03:20:27.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:27.274+0000 7f47c1e41640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f47bc15e650 0x7f47bc0d1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:27.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:27.274+0000 7f47c2642640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f47bc06db30 0x7f47bc06df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:27.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:27.274+0000 7f47c1640640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f47bc06cc40 0x7f47bc06d060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.27:56244/0 (socket says 172.21.15.27:56244) 2024-09-12T03:20:27.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:27.274+0000 7f47c1640640 1 -- 172.21.15.27:0/2636604222 learned_addr learned my addr 172.21.15.27:0/2636604222 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:20:27.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:27.274+0000 7f47c0e3f640 1 -- 172.21.15.27:0/2636604222 <== mon.1 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1956899359 0 0) 0x7f47bc0d2280 con 0x7f47bc06cc40 2024-09-12T03:20:27.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:27.274+0000 7f47c0e3f640 1 -- 172.21.15.27:0/2636604222 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f47a4003520 con 0x7f47bc06cc40 2024-09-12T03:20:27.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:27.275+0000 7f47c1e41640 1 -- 172.21.15.27:0/2636604222 >> v2:172.21.15.45:3300/0 conn(0x7f47bc06db30 msgr2=0x7f47bc06df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:27.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:27.275+0000 7f47c1e41640 1 --2- 172.21.15.27:0/2636604222 >> v2:172.21.15.45:3300/0 conn(0x7f47bc06db30 0x7f47bc06df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:27.276 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:27.275+0000 7f47c1e41640 1 -- 172.21.15.27:0/2636604222 >> v1:172.21.15.45:6789/0 conn(0x7f47bc06cc40 legacy=0x7f47bc06d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:27.276 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:27.275+0000 7f47c1e41640 1 -- 172.21.15.27:0/2636604222 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47bc0d1540 con 0x7f47bc15e650 2024-09-12T03:20:27.276 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:27.275+0000 7f47c2642640 1 --2- 172.21.15.27:0/2636604222 >> v2:172.21.15.45:3300/0 conn(0x7f47bc06db30 0x7f47bc06df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:20:27.276 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:27.275+0000 7f47c1e41640 1 --2- 172.21.15.27:0/2636604222 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f47bc15e650 0x7f47bc0d1d40 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f47ac009d90 tx=0x7f47ac02a5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c3b73f5f586b54f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:27.277 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:27.276+0000 7f47c0e3f640 1 -- 172.21.15.27:0/2636604222 <== mon.2 v2:172.21.15.45:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47ac031600 con 0x7f47bc15e650 2024-09-12T03:20:27.277 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:27.276+0000 7f47c0e3f640 1 -- 172.21.15.27:0/2636604222 <== mon.2 v2:172.21.15.45:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47ac0317a0 con 0x7f47bc15e650 2024-09-12T03:20:27.277 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:27.276+0000 7f47c0e3f640 1 -- 172.21.15.27:0/2636604222 <== mon.2 v2:172.21.15.45:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47ac031b50 con 0x7f47bc15e650 2024-09-12T03:20:27.277 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:27.276+0000 7f47c40a5640 1 -- 172.21.15.27:0/2636604222 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f47bc15e650 msgr2=0x7f47bc0d1d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:27.277 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:27.276+0000 7f47c40a5640 1 --2- 172.21.15.27:0/2636604222 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f47bc15e650 0x7f47bc0d1d40 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f47ac009d90 tx=0x7f47ac02a5d0 comp rx=0 tx=0).stop 2024-09-12T03:20:27.277 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:27.276+0000 7f47c40a5640 1 -- 172.21.15.27:0/2636604222 shutdown_connections 2024-09-12T03:20:27.277 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:27.276+0000 7f47c40a5640 1 --2- 172.21.15.27:0/2636604222 >> v2:172.21.15.45:3300/0 conn(0x7f47bc06db30 0x7f47bc06df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:27.277 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:27.276+0000 7f47c40a5640 1 --2- 172.21.15.27:0/2636604222 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f47bc15e650 0x7f47bc0d1d40 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:27.277 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:27.276+0000 7f47c40a5640 1 -- 172.21.15.27:0/2636604222 >> 172.21.15.27:0/2636604222 conn(0x7f47bc06f320 msgr2=0x7f47bc06f700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:27.277 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:27.276+0000 7f47c40a5640 1 -- 172.21.15.27:0/2636604222 shutdown_connections 2024-09-12T03:20:27.277 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:27.276+0000 7f47c40a5640 1 -- 172.21.15.27:0/2636604222 wait complete. 2024-09-12T03:20:27.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:27.277+0000 7f47c40a5640 1 Processor -- start 2024-09-12T03:20:27.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:27.277+0000 7f47c40a5640 1 -- start start 2024-09-12T03:20:27.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:27.278+0000 7f47c40a5640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f47bc06cc40 0x7f47bc1f3570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:27.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:27.278+0000 7f47c40a5640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f47bc06db30 0x7f47bc1e92f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:27.280 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:27.278+0000 7f47c40a5640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f47bc15e650 0x7f47bc1e9830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:27.280 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:27.278+0000 7f47c40a5640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f47bc06e640 con 0x7f47bc06cc40 2024-09-12T03:20:27.280 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:27.278+0000 7f47c40a5640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f47bc06e4c0 con 0x7f47bc15e650 2024-09-12T03:20:27.280 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:27.278+0000 7f47c40a5640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f47bc0b55b0 con 0x7f47bc06db30 2024-09-12T03:20:27.280 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:27.278+0000 7f47c1e41640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f47bc06cc40 0x7f47bc1f3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:27.280 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:27.278+0000 7f47c1e41640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f47bc06cc40 0x7f47bc1f3570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46924/0 (socket says 172.21.15.27:46924) 2024-09-12T03:20:27.280 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:27.278+0000 7f47c1e41640 1 -- 172.21.15.27:0/2429798263 learned_addr learned my addr 172.21.15.27:0/2429798263 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:20:27.280 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:27.279+0000 7f47c1640640 1 --2- 172.21.15.27:0/2429798263 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f47bc06db30 0x7f47bc1e92f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:27.280 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:27.279+0000 7f47c2642640 1 --2- 172.21.15.27:0/2429798263 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f47bc15e650 0x7f47bc1e9830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:27.280 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:27.279+0000 7f47c1e41640 1 -- 172.21.15.27:0/2429798263 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f47bc06db30 msgr2=0x7f47bc1e92f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:27.280 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:27.279+0000 7f47c1e41640 1 --2- 172.21.15.27:0/2429798263 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f47bc06db30 0x7f47bc1e92f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:27.280 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:27.279+0000 7f47c1e41640 1 -- 172.21.15.27:0/2429798263 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f47bc15e650 msgr2=0x7f47bc1e9830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:27.280 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:27.279+0000 7f47c1e41640 1 --2- 172.21.15.27:0/2429798263 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f47bc15e650 0x7f47bc1e9830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:27.281 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:27.279+0000 7f47c1e41640 1 -- 172.21.15.27:0/2429798263 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47bc0ce910 con 0x7f47bc06cc40 2024-09-12T03:20:27.281 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:27.279+0000 7f47c1e41640 1 --2- 172.21.15.27:0/2429798263 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f47bc06cc40 0x7f47bc1f3570 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f47ac009eb0 tx=0x7f47ac004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:27.281 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:27.279+0000 7f47ba7fc640 1 -- 172.21.15.27:0/2429798263 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47ac03f070 con 0x7f47bc06cc40 2024-09-12T03:20:27.281 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:27.279+0000 7f47ba7fc640 1 -- 172.21.15.27:0/2429798263 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47ac030830 con 0x7f47bc06cc40 2024-09-12T03:20:27.281 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:27.279+0000 7f47c40a5640 1 -- 172.21.15.27:0/2429798263 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47bc0cebf0 con 0x7f47bc06cc40 2024-09-12T03:20:27.281 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:27.279+0000 7f47ba7fc640 1 -- 172.21.15.27:0/2429798263 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47ac030c00 con 0x7f47bc06cc40 2024-09-12T03:20:27.281 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:27.279+0000 7f47c40a5640 1 -- 172.21.15.27:0/2429798263 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47bc0cf1b0 con 0x7f47bc06cc40 2024-09-12T03:20:27.282 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:27.280+0000 7f47c40a5640 1 -- 172.21.15.27:0/2429798263 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47bc06df80 con 0x7f47bc06cc40 2024-09-12T03:20:27.282 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:27.281+0000 7f47ba7fc640 1 -- 172.21.15.27:0/2429798263 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 90600+0+0 (secure 0 0 0) 0x7f47ac03b410 con 0x7f47bc06cc40 2024-09-12T03:20:27.282 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:27.281+0000 7f47ba7fc640 1 --2- 172.21.15.27:0/2429798263 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f47a806bf50 0x7f47a806e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:27.282 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:27.281+0000 7f47ba7fc640 1 -- 172.21.15.27:0/2429798263 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5227+0+0 (secure 0 0 0) 0x7f47ac0ac390 con 0x7f47bc06cc40 2024-09-12T03:20:27.282 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:27.281+0000 7f47c1640640 1 --2- 172.21.15.27:0/2429798263 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f47a806bf50 0x7f47a806e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:27.282 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:27.282+0000 7f47c1640640 1 --2- 172.21.15.27:0/2429798263 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f47a806bf50 0x7f47a806e410 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f47bc0b3cf0 tx=0x7f47b0009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:27.285 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:27.286+0000 7f47ba7fc640 1 -- 172.21.15.27:0/2429798263 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f47ac07ad90 con 0x7f47bc06cc40 2024-09-12T03:20:27.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:27.460+0000 7f47c40a5640 1 -- 172.21.15.27:0/2429798263 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f47bc1e9d70 con 0x7f47bc06cc40 2024-09-12T03:20:28.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:28.139+0000 7f47ba7fc640 1 -- 172.21.15.27:0/2429798263 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v79) ==== 170+0+0 (secure 0 0 0) 0x7f47ac07a690 con 0x7f47bc06cc40 2024-09-12T03:20:28.140 INFO:teuthology.orchestra.run.smithi027.stderr:pool 'cephfs_data' created 2024-09-12T03:20:28.142 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:28.142+0000 7f47c40a5640 1 -- 172.21.15.27:0/2429798263 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f47a806bf50 msgr2=0x7f47a806e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:28.142 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:28.142+0000 7f47c40a5640 1 --2- 172.21.15.27:0/2429798263 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f47a806bf50 0x7f47a806e410 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f47bc0b3cf0 tx=0x7f47b0009040 comp rx=0 tx=0).stop 2024-09-12T03:20:28.142 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:28.142+0000 7f47c40a5640 1 -- 172.21.15.27:0/2429798263 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f47bc06cc40 msgr2=0x7f47bc1f3570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:28.142 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:28.142+0000 7f47c40a5640 1 --2- 172.21.15.27:0/2429798263 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f47bc06cc40 0x7f47bc1f3570 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f47ac009eb0 tx=0x7f47ac004a40 comp rx=0 tx=0).stop 2024-09-12T03:20:28.142 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:28.142+0000 7f47c40a5640 1 -- 172.21.15.27:0/2429798263 shutdown_connections 2024-09-12T03:20:28.142 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:28.142+0000 7f47c40a5640 1 --2- 172.21.15.27:0/2429798263 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f47bc15e650 0x7f47bc1e9830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:28.143 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:28.142+0000 7f47c40a5640 1 --2- 172.21.15.27:0/2429798263 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f47bc06db30 0x7f47bc1e92f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:28.143 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:28.142+0000 7f47c40a5640 1 --2- 172.21.15.27:0/2429798263 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f47a806bf50 0x7f47a806e410 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:28.143 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:28.142+0000 7f47c40a5640 1 --2- 172.21.15.27:0/2429798263 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f47bc06cc40 0x7f47bc1f3570 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:28.143 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:28.142+0000 7f47c40a5640 1 -- 172.21.15.27:0/2429798263 >> 172.21.15.27:0/2429798263 conn(0x7f47bc06f320 msgr2=0x7f47bc161710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:28.143 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:28.142+0000 7f47c40a5640 1 -- 172.21.15.27:0/2429798263 shutdown_connections 2024-09-12T03:20:28.143 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:28.142+0000 7f47c40a5640 1 -- 172.21.15.27:0/2429798263 wait complete. 2024-09-12T03:20:28.165 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-12T03:20:28.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:28.274+0000 7f77c6fc4640 1 Processor -- start 2024-09-12T03:20:28.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:28.274+0000 7f77c6fc4640 1 -- start start 2024-09-12T03:20:28.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:28.275+0000 7f77c6fc4640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f77c0070d30 0x7f77c006cc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:28.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:28.275+0000 7f77c6fc4640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f77c006e960 0x7f77c00d0440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:28.276 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:28.275+0000 7f77c6fc4640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f77c00cf9f0 con 0x7f77c00d0980 2024-09-12T03:20:28.276 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:28.275+0000 7f77c6fc4640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f77c00d0df0 con 0x7f77c006e960 2024-09-12T03:20:28.276 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:28.275+0000 7f77c6fc4640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f77c00d1a40 con 0x7f77c0070d30 2024-09-12T03:20:28.276 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:28.275+0000 7f77c5561640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f77c00d0980 0x7f77c00d21b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:34016/0 (socket says 172.21.15.27:34016) 2024-09-12T03:20:28.276 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:28.275+0000 7f77c5561640 1 -- 172.21.15.27:0/3775473242 learned_addr learned my addr 172.21.15.27:0/3775473242 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:20:28.276 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:28.275+0000 7f77bffff640 1 --2- 172.21.15.27:0/3775473242 >> v2:172.21.15.45:3300/0 conn(0x7f77c006e960 0x7f77c00d0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:28.276 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:28.275+0000 7f77c4d60640 1 --2- 172.21.15.27:0/3775473242 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f77c0070d30 0x7f77c006cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:28.276 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:28.275+0000 7f77bf7fe640 1 -- 172.21.15.27:0/3775473242 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4133677371 0 0) 0x7f77c00cf9f0 con 0x7f77c00d0980 2024-09-12T03:20:28.276 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:28.275+0000 7f77bf7fe640 1 -- 172.21.15.27:0/3775473242 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f77b0003520 con 0x7f77c00d0980 2024-09-12T03:20:28.276 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:28.275+0000 7f77bffff640 1 -- 172.21.15.27:0/3775473242 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f77c0070d30 msgr2=0x7f77c006cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:28.276 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:28.275+0000 7f77bffff640 1 --2- 172.21.15.27:0/3775473242 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f77c0070d30 0x7f77c006cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:28.276 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:28.275+0000 7f77bffff640 1 -- 172.21.15.27:0/3775473242 >> v1:172.21.15.27:6789/0 conn(0x7f77c00d0980 legacy=0x7f77c00d21b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:28.277 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:28.275+0000 7f77bffff640 1 -- 172.21.15.27:0/3775473242 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77c00cfc90 con 0x7f77c006e960 2024-09-12T03:20:28.277 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:28.275+0000 7f77c4d60640 1 --2- 172.21.15.27:0/3775473242 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f77c0070d30 0x7f77c006cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:20:28.278 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:28.276+0000 7f77bffff640 1 --2- 172.21.15.27:0/3775473242 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f77c006e960 0x7f77c00d0440 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f77ac0097a0 tx=0x7f77ac02a060 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dfdf811515acc6d8 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:28.278 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:28.276+0000 7f77bf7fe640 1 -- 172.21.15.27:0/3775473242 <== mon.1 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77ac037070 con 0x7f77c006e960 2024-09-12T03:20:28.278 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:28.276+0000 7f77bf7fe640 1 -- 172.21.15.27:0/3775473242 <== mon.1 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77ac02adb0 con 0x7f77c006e960 2024-09-12T03:20:28.278 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:28.276+0000 7f77bf7fe640 1 -- 172.21.15.27:0/3775473242 <== mon.1 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77ac032640 con 0x7f77c006e960 2024-09-12T03:20:28.278 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:28.277+0000 7f77c6fc4640 1 -- 172.21.15.27:0/3775473242 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f77c006e960 msgr2=0x7f77c00d0440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:28.278 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:28.277+0000 7f77c6fc4640 1 --2- 172.21.15.27:0/3775473242 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f77c006e960 0x7f77c00d0440 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f77ac0097a0 tx=0x7f77ac02a060 comp rx=0 tx=0).stop 2024-09-12T03:20:28.278 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:28.277+0000 7f77c6fc4640 1 -- 172.21.15.27:0/3775473242 shutdown_connections 2024-09-12T03:20:28.278 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:28.277+0000 7f77c6fc4640 1 --2- 172.21.15.27:0/3775473242 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f77c006e960 0x7f77c00d0440 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:28.278 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:28.277+0000 7f77c6fc4640 1 --2- 172.21.15.27:0/3775473242 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f77c0070d30 0x7f77c006cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:28.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:28.277+0000 7f77c6fc4640 1 -- 172.21.15.27:0/3775473242 >> 172.21.15.27:0/3775473242 conn(0x7f77c00b7c10 msgr2=0x7f77c009ba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:28.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:28.277+0000 7f77c6fc4640 1 -- 172.21.15.27:0/3775473242 shutdown_connections 2024-09-12T03:20:28.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:28.277+0000 7f77c6fc4640 1 -- 172.21.15.27:0/3775473242 wait complete. 2024-09-12T03:20:28.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:28.278+0000 7f77c6fc4640 1 Processor -- start 2024-09-12T03:20:28.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:28.278+0000 7f77c6fc4640 1 -- start start 2024-09-12T03:20:28.281 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:28.279+0000 7f77c6fc4640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f77c006e960 0x7f77c0155330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:28.281 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:28.279+0000 7f77c6fc4640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f77c0070d30 0x7f77c015f240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:28.281 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:28.279+0000 7f77c6fc4640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f77c00d0980 0x7f77c015f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:28.281 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:28.279+0000 7f77c6fc4640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f77c006d180 con 0x7f77c0070d30 2024-09-12T03:20:28.281 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:28.279+0000 7f77c6fc4640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f77c00d1c90 con 0x7f77c006e960 2024-09-12T03:20:28.281 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:28.279+0000 7f77c6fc4640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f77c00d28c0 con 0x7f77c00d0980 2024-09-12T03:20:28.281 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:28.279+0000 7f77bffff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f77c0070d30 0x7f77c015f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:28.281 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:28.279+0000 7f77bffff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f77c0070d30 0x7f77c015f240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46940/0 (socket says 172.21.15.27:46940) 2024-09-12T03:20:28.281 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:28.279+0000 7f77bffff640 1 -- 172.21.15.27:0/3147361711 learned_addr learned my addr 172.21.15.27:0/3147361711 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:20:28.281 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:28.279+0000 7f77c4d60640 1 --2- 172.21.15.27:0/3147361711 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f77c006e960 0x7f77c0155330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:28.281 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:28.279+0000 7f77bffff640 1 -- 172.21.15.27:0/3147361711 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f77c00d0980 msgr2=0x7f77c015f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:28.281 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:28.279+0000 7f77bffff640 1 --2- 172.21.15.27:0/3147361711 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f77c00d0980 0x7f77c015f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:28.282 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:28.279+0000 7f77bffff640 1 -- 172.21.15.27:0/3147361711 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f77c006e960 msgr2=0x7f77c0155330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:28.282 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:28.279+0000 7f77bffff640 1 --2- 172.21.15.27:0/3147361711 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f77c006e960 0x7f77c0155330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:28.282 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:28.279+0000 7f77bffff640 1 -- 172.21.15.27:0/3147361711 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77c00cc990 con 0x7f77c0070d30 2024-09-12T03:20:28.282 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:28.280+0000 7f77bffff640 1 --2- 172.21.15.27:0/3147361711 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f77c0070d30 0x7f77c015f240 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f77ac02a0a0 tx=0x7f77ac0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:28.282 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:28.280+0000 7f77bd7fa640 1 -- 172.21.15.27:0/3147361711 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77ac041070 con 0x7f77c0070d30 2024-09-12T03:20:28.282 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:28.280+0000 7f77bd7fa640 1 -- 172.21.15.27:0/3147361711 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77ac03c680 con 0x7f77c0070d30 2024-09-12T03:20:28.282 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:28.280+0000 7f77c6fc4640 1 -- 172.21.15.27:0/3147361711 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77c00ce710 con 0x7f77c0070d30 2024-09-12T03:20:28.282 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:28.280+0000 7f77bd7fa640 1 -- 172.21.15.27:0/3147361711 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77ac037040 con 0x7f77c0070d30 2024-09-12T03:20:28.282 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:28.280+0000 7f77c6fc4640 1 -- 172.21.15.27:0/3147361711 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77c00cbbd0 con 0x7f77c0070d30 2024-09-12T03:20:28.282 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:28.281+0000 7f77c6fc4640 1 -- 172.21.15.27:0/3147361711 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77c00d0440 con 0x7f77c0070d30 2024-09-12T03:20:28.283 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:28.282+0000 7f77bd7fa640 1 -- 172.21.15.27:0/3147361711 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 90600+0+0 (secure 0 0 0) 0x7f77ac044070 con 0x7f77c0070d30 2024-09-12T03:20:28.283 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:28.282+0000 7f77bd7fa640 1 --2- 172.21.15.27:0/3147361711 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f77a406bf50 0x7f77a406e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:28.283 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:28.282+0000 7f77bd7fa640 1 -- 172.21.15.27:0/3147361711 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 5609+0+0 (secure 0 0 0) 0x7f77ac0ab8b0 con 0x7f77c0070d30 2024-09-12T03:20:28.283 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:28.282+0000 7f77c4d60640 1 --2- 172.21.15.27:0/3147361711 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f77a406bf50 0x7f77a406e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:28.283 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:28.282+0000 7f77c4d60640 1 --2- 172.21.15.27:0/3147361711 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f77a406bf50 0x7f77a406e410 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f77b4009560 tx=0x7f77b4008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:28.286 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:28.287+0000 7f77bd7fa640 1 -- 172.21.15.27:0/3147361711 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f77ac07a2d0 con 0x7f77c0070d30 2024-09-12T03:20:28.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:28.498+0000 7f77c6fc4640 1 -- 172.21.15.27:0/3147361711 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f77c00d1230 con 0x7f77c0070d30 2024-09-12T03:20:29.157 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.157+0000 7f77bd7fa640 1 -- 172.21.15.27:0/3147361711 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 12 and data pool 13 v44) ==== 174+0+0 (secure 0 0 0) 0x7f77ac079bd0 con 0x7f77c0070d30 2024-09-12T03:20:29.157 INFO:teuthology.orchestra.run.smithi027.stderr:new fs with metadata pool 12 and data pool 13 2024-09-12T03:20:29.164 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.162+0000 7f77c6fc4640 1 -- 172.21.15.27:0/3147361711 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f77a406bf50 msgr2=0x7f77a406e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:29.164 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.162+0000 7f77c6fc4640 1 --2- 172.21.15.27:0/3147361711 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f77a406bf50 0x7f77a406e410 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f77b4009560 tx=0x7f77b4008040 comp rx=0 tx=0).stop 2024-09-12T03:20:29.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.163+0000 7f77c6fc4640 1 -- 172.21.15.27:0/3147361711 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f77c0070d30 msgr2=0x7f77c015f240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:29.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.163+0000 7f77c6fc4640 1 --2- 172.21.15.27:0/3147361711 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f77c0070d30 0x7f77c015f240 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f77ac02a0a0 tx=0x7f77ac0046c0 comp rx=0 tx=0).stop 2024-09-12T03:20:29.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.165+0000 7f77c6fc4640 1 -- 172.21.15.27:0/3147361711 shutdown_connections 2024-09-12T03:20:29.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.165+0000 7f77c6fc4640 1 --2- 172.21.15.27:0/3147361711 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f77c00d0980 0x7f77c015f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:29.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.165+0000 7f77c6fc4640 1 --2- 172.21.15.27:0/3147361711 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f77a406bf50 0x7f77a406e410 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:29.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.165+0000 7f77c6fc4640 1 --2- 172.21.15.27:0/3147361711 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f77c0070d30 0x7f77c015f240 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:29.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.165+0000 7f77c6fc4640 1 --2- 172.21.15.27:0/3147361711 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f77c006e960 0x7f77c0155330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:29.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.165+0000 7f77c6fc4640 1 -- 172.21.15.27:0/3147361711 >> 172.21.15.27:0/3147361711 conn(0x7f77c00b7c10 msgr2=0x7f77c00ceaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:29.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.167+0000 7f77c6fc4640 1 -- 172.21.15.27:0/3147361711 shutdown_connections 2024-09-12T03:20:29.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.167+0000 7f77c6fc4640 1 -- 172.21.15.27:0/3147361711 wait complete. 2024-09-12T03:20:29.193 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-12T03:20:29.297 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.297+0000 7fddb458d640 1 Processor -- start 2024-09-12T03:20:29.297 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.297+0000 7fddb458d640 1 -- start start 2024-09-12T03:20:29.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.298+0000 7fddb458d640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fddac15eb90 0x7fddac0d2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:29.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.298+0000 7fddb458d640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fddac0d1720 0x7fddac0b8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:29.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.298+0000 7fddb458d640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fddac0d3760 con 0x7fddac0d5200 2024-09-12T03:20:29.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.298+0000 7fddb458d640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fddac0b7d20 con 0x7fddac0d1720 2024-09-12T03:20:29.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.298+0000 7fddb458d640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fddac0d42e0 con 0x7fddac15eb90 2024-09-12T03:20:29.300 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.298+0000 7fddb1b28640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fddac0d5200 0x7fddac0d1010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:57656/0 (socket says 172.21.15.27:57656) 2024-09-12T03:20:29.300 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.298+0000 7fddb1b28640 1 -- 172.21.15.27:0/3397414991 learned_addr learned my addr 172.21.15.27:0/3397414991 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:20:29.300 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.299+0000 7fddb2b2a640 1 --2- 172.21.15.27:0/3397414991 >> v2:172.21.15.45:3300/0 conn(0x7fddac0d1720 0x7fddac0b8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:29.300 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.299+0000 7fddb2329640 1 --2- 172.21.15.27:0/3397414991 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fddac15eb90 0x7fddac0d2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:29.300 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.299+0000 7fddb1327640 1 -- 172.21.15.27:0/3397414991 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4197115517 0 0) 0x7fddac0d3760 con 0x7fddac0d5200 2024-09-12T03:20:29.300 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.299+0000 7fddb1327640 1 -- 172.21.15.27:0/3397414991 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdd94003520 con 0x7fddac0d5200 2024-09-12T03:20:29.300 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.299+0000 7fddb2329640 1 -- 172.21.15.27:0/3397414991 >> v2:172.21.15.45:3300/0 conn(0x7fddac0d1720 msgr2=0x7fddac0b8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:29.300 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.299+0000 7fddb2329640 1 --2- 172.21.15.27:0/3397414991 >> v2:172.21.15.45:3300/0 conn(0x7fddac0d1720 0x7fddac0b8190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:29.300 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.299+0000 7fddb2329640 1 -- 172.21.15.27:0/3397414991 >> v1:172.21.15.27:6789/0 conn(0x7fddac0d5200 legacy=0x7fddac0d1010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:29.300 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.299+0000 7fddb2329640 1 -- 172.21.15.27:0/3397414991 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddac0d3a40 con 0x7fddac15eb90 2024-09-12T03:20:29.300 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.299+0000 7fddb2b2a640 1 --2- 172.21.15.27:0/3397414991 >> v2:172.21.15.45:3300/0 conn(0x7fddac0d1720 0x7fddac0b8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:20:29.304 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.300+0000 7fddb2329640 1 --2- 172.21.15.27:0/3397414991 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fddac15eb90 0x7fddac0d2df0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fdda40097a0 tx=0x7fdda402a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1d49acf92ad00c51 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:29.304 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.300+0000 7fddb1327640 1 -- 172.21.15.27:0/3397414991 <== mon.2 v2:172.21.15.45:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdda4037070 con 0x7fddac15eb90 2024-09-12T03:20:29.304 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.300+0000 7fddb1327640 1 -- 172.21.15.27:0/3397414991 <== mon.2 v2:172.21.15.45:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdda4032440 con 0x7fddac15eb90 2024-09-12T03:20:29.304 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.300+0000 7fddb1327640 1 -- 172.21.15.27:0/3397414991 <== mon.2 v2:172.21.15.45:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdda4032810 con 0x7fddac15eb90 2024-09-12T03:20:29.304 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.300+0000 7fddb458d640 1 -- 172.21.15.27:0/3397414991 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fddac15eb90 msgr2=0x7fddac0d2df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:29.305 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.300+0000 7fddb458d640 1 --2- 172.21.15.27:0/3397414991 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fddac15eb90 0x7fddac0d2df0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fdda40097a0 tx=0x7fdda402a180 comp rx=0 tx=0).stop 2024-09-12T03:20:29.305 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.301+0000 7fddb458d640 1 -- 172.21.15.27:0/3397414991 shutdown_connections 2024-09-12T03:20:29.305 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.301+0000 7fddb458d640 1 --2- 172.21.15.27:0/3397414991 >> v2:172.21.15.45:3300/0 conn(0x7fddac0d1720 0x7fddac0b8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:29.305 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.301+0000 7fddb458d640 1 --2- 172.21.15.27:0/3397414991 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fddac15eb90 0x7fddac0d2df0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:29.305 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.301+0000 7fddb458d640 1 -- 172.21.15.27:0/3397414991 >> 172.21.15.27:0/3397414991 conn(0x7fddac06dce0 msgr2=0x7fddac06e0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:29.305 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.301+0000 7fddb458d640 1 -- 172.21.15.27:0/3397414991 shutdown_connections 2024-09-12T03:20:29.305 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.301+0000 7fddb458d640 1 -- 172.21.15.27:0/3397414991 wait complete. 2024-09-12T03:20:29.305 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.302+0000 7fddb458d640 1 Processor -- start 2024-09-12T03:20:29.305 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.302+0000 7fddb458d640 1 -- start start 2024-09-12T03:20:29.305 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.302+0000 7fddb458d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fddac0d1720 0x7fddac0ce560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:29.306 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.303+0000 7fddb458d640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fddac0d5200 0x7fddac1534b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:29.306 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.303+0000 7fddb458d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fddac15eb90 0x7fddac152d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:29.306 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.303+0000 7fddb458d640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fddac0d3330 con 0x7fddac0d1720 2024-09-12T03:20:29.306 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.303+0000 7fddb458d640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fddac0d4000 con 0x7fddac15eb90 2024-09-12T03:20:29.306 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.303+0000 7fddb458d640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fddac0d34b0 con 0x7fddac0d5200 2024-09-12T03:20:29.306 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.303+0000 7fddb2329640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fddac0d1720 0x7fddac0ce560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:29.306 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.303+0000 7fddb2329640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fddac0d1720 0x7fddac0ce560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46838/0 (socket says 172.21.15.27:46838) 2024-09-12T03:20:29.306 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.303+0000 7fddb2329640 1 -- 172.21.15.27:0/4042357995 learned_addr learned my addr 172.21.15.27:0/4042357995 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:20:29.306 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.303+0000 7fddb2b2a640 1 --2- 172.21.15.27:0/4042357995 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fddac15eb90 0x7fddac152d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:29.306 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.303+0000 7fddb1b28640 1 --2- 172.21.15.27:0/4042357995 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fddac0d5200 0x7fddac1534b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:29.307 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.303+0000 7fddb2329640 1 -- 172.21.15.27:0/4042357995 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fddac0d5200 msgr2=0x7fddac1534b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:29.307 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.303+0000 7fddb2329640 1 --2- 172.21.15.27:0/4042357995 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fddac0d5200 0x7fddac1534b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:29.307 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.303+0000 7fddb2329640 1 -- 172.21.15.27:0/4042357995 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fddac15eb90 msgr2=0x7fddac152d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:29.307 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.303+0000 7fddb2329640 1 --2- 172.21.15.27:0/4042357995 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fddac15eb90 0x7fddac152d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:29.307 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.303+0000 7fddb2329640 1 -- 172.21.15.27:0/4042357995 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddac159610 con 0x7fddac0d1720 2024-09-12T03:20:29.307 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.303+0000 7fddb2b2a640 1 --2- 172.21.15.27:0/4042357995 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fddac15eb90 0x7fddac152d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:20:29.307 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.303+0000 7fddb2329640 1 --2- 172.21.15.27:0/4042357995 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fddac0d1720 0x7fddac0ce560 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fdda4000c00 tx=0x7fdda40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:29.307 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.303+0000 7fddaaffd640 1 -- 172.21.15.27:0/4042357995 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdda4041020 con 0x7fddac0d1720 2024-09-12T03:20:29.307 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.303+0000 7fddaaffd640 1 -- 172.21.15.27:0/4042357995 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdda403d640 con 0x7fddac0d1720 2024-09-12T03:20:29.307 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.303+0000 7fddb458d640 1 -- 172.21.15.27:0/4042357995 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fddac1598f0 con 0x7fddac0d1720 2024-09-12T03:20:29.308 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.303+0000 7fddaaffd640 1 -- 172.21.15.27:0/4042357995 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdda4037040 con 0x7fddac0d1720 2024-09-12T03:20:29.308 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.303+0000 7fddb458d640 1 -- 172.21.15.27:0/4042357995 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fddac159eb0 con 0x7fddac0d1720 2024-09-12T03:20:29.308 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.304+0000 7fddb458d640 1 -- 172.21.15.27:0/4042357995 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fddac15f090 con 0x7fddac0d1720 2024-09-12T03:20:29.309 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.305+0000 7fddaaffd640 1 -- 172.21.15.27:0/4042357995 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 90600+0+0 (secure 0 0 0) 0x7fdda4044020 con 0x7fddac0d1720 2024-09-12T03:20:29.309 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.305+0000 7fddaaffd640 1 --2- 172.21.15.27:0/4042357995 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fdd9806bf80 0x7fdd9806e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:29.309 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.305+0000 7fddaaffd640 1 -- 172.21.15.27:0/4042357995 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 5709+0+0 (secure 0 0 0) 0x7fdda40ab720 con 0x7fddac0d1720 2024-09-12T03:20:29.309 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.306+0000 7fddb1b28640 1 --2- 172.21.15.27:0/4042357995 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fdd9806bf80 0x7fdd9806e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:29.309 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.306+0000 7fddb1b28640 1 --2- 172.21.15.27:0/4042357995 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fdd9806bf80 0x7fdd9806e440 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fddac15b120 tx=0x7fdda0009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:29.310 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.310+0000 7fddaaffd640 1 -- 172.21.15.27:0/4042357995 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fdda404b030 con 0x7fddac0d1720 2024-09-12T03:20:29.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.484+0000 7fddb458d640 1 -- 172.21.15.27:0/4042357995 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fddac1539f0 con 0x7fddac0d1720 2024-09-12T03:20:29.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.485+0000 7fddaaffd640 1 -- 172.21.15.27:0/4042357995 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) ==== 74+0+16685 (secure 0 0 0) 0x7fdda4079fd0 con 0x7fddac0d1720 2024-09-12T03:20:29.486 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:20:29.486 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":80,"fsid":"d2df504e-2647-4385-a5db-9fdf69f487a0","created":"2024-09-12T03:16:07.778937+0000","modified":"2024-09-12T03:20:28.499239+0000","last_up_change":"2024-09-12T03:16:19.314778+0000","last_in_change":"2024-09-12T03:16:09.893086+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T03:16:20.097646+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-09-12T03:20:26.167239+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-09-12T03:20:27.491764+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1851116c-1fb3-4dc9-9772-f90e2ef4c355","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6817","nonce":3002321488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6819","nonce":3002321488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6823","nonce":3002321488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6821","nonce":3002321488}]},"public_addr":"172.21.15.27:6817/3002321488","cluster_addr":"172.21.15.27:6819/3002321488","heartbeat_back_addr":"172.21.15.27:6823/3002321488","heartbeat_front_addr":"172.21.15.27:6821/3002321488","state":["exists","up"]},{"osd":1,"uuid":"f38ae17d-73eb-4204-b99d-975fa806fb87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6809","nonce":1613102886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6811","nonce":1613102886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6815","nonce":1613102886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6813","nonce":1613102886}]},"public_addr":"172.21.15.27:6809/1613102886","cluster_addr":"172.21.15.27:6811/1613102886","heartbeat_back_addr":"172.21.15.27:6815/1613102886","heartbeat_front_addr":"172.21.15.27:6813/1613102886","state":["exists","up"]},{"osd":2,"uuid":"fb12dcc8-bdd0-42b8-b46d-8b10106224bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6825","nonce":3534723355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6827","nonce":3534723355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6831","nonce":3534723355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6829","nonce":3534723355}]},"public_addr":"172.21.15.27:6825/3534723355","cluster_addr":"172.21.15.27:6827/3534723355","heartbeat_back_addr":"172.21.15.27:6831/3534723355","heartbeat_front_addr":"172.21.15.27:6829/3534723355","state":["exists","up"]},{"osd":3,"uuid":"d74d3f49-9c24-4b9f-98e2-3ecd7b5ef3b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6801","nonce":80661598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6803","nonce":80661598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6807","nonce":80661598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6805","nonce":80661598}]},"public_addr":"172.21.15.27:6801/80661598","cluster_addr":"172.21.15.27:6803/80661598","heartbeat_back_addr":"172.21.15.27:6807/80661598","heartbeat_front_addr":"172.21.15.27:6805/80661598","state":["exists","up"]},{"osd":4,"uuid":"2bf123f2-1b0d-41c4-aa3b-805bab267e59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6817","nonce":3943189827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6819","nonce":3943189827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6823","nonce":3943189827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6821","nonce":3943189827}]},"public_addr":"172.21.15.45:6817/3943189827","cluster_addr":"172.21.15.45:6819/3943189827","heartbeat_back_addr":"172.21.15.45:6823/3943189827","heartbeat_front_addr":"172.21.15.45:6821/3943189827","state":["exists","up"]},{"osd":5,"uuid":"c49ba5da-91c6-4a86-a97b-4aba89c253ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6809","nonce":3620760330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6811","nonce":3620760330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6815","nonce":3620760330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6813","nonce":3620760330}]},"public_addr":"172.21.15.45:6809/3620760330","cluster_addr":"172.21.15.45:6811/3620760330","heartbeat_back_addr":"172.21.15.45:6815/3620760330","heartbeat_front_addr":"172.21.15.45:6813/3620760330","state":["exists","up"]},{"osd":6,"uuid":"b0123475-89f7-4a34-9e33-d797b99dd9dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6801","nonce":1989083510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6803","nonce":1989083510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6807","nonce":1989083510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6805","nonce":1989083510}]},"public_addr":"172.21.15.45:6801/1989083510","cluster_addr":"172.21.15.45:6803/1989083510","heartbeat_back_addr":"172.21.15.45:6807/1989083510","heartbeat_front_addr":"172.21.15.45:6805/1989083510","state":["exists","up"]},{"osd":7,"uuid":"4dad773d-b7d8-46fc-8a10-a972aad27b29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6825","nonce":1513300294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6827","nonce":1513300294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6831","nonce":1513300294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6829","nonce":1513300294}]},"public_addr":"172.21.15.45:6825/1513300294","cluster_addr":"172.21.15.45:6827/1513300294","heartbeat_back_addr":"172.21.15.45:6831/1513300294","heartbeat_front_addr":"172.21.15.45:6829/1513300294","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.881855+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:17.133252+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.735339+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.500742+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.477571+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:17.167667+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.470291+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.873743+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T03:20:29.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.487+0000 7fddb458d640 1 -- 172.21.15.27:0/4042357995 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fdd9806bf80 msgr2=0x7fdd9806e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:29.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.487+0000 7fddb458d640 1 --2- 172.21.15.27:0/4042357995 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fdd9806bf80 0x7fdd9806e440 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fddac15b120 tx=0x7fdda0009040 comp rx=0 tx=0).stop 2024-09-12T03:20:29.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.487+0000 7fddb458d640 1 -- 172.21.15.27:0/4042357995 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fddac0d1720 msgr2=0x7fddac0ce560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:29.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.487+0000 7fddb458d640 1 --2- 172.21.15.27:0/4042357995 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fddac0d1720 0x7fddac0ce560 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fdda4000c00 tx=0x7fdda40046c0 comp rx=0 tx=0).stop 2024-09-12T03:20:29.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.487+0000 7fddb458d640 1 -- 172.21.15.27:0/4042357995 shutdown_connections 2024-09-12T03:20:29.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.487+0000 7fddb458d640 1 --2- 172.21.15.27:0/4042357995 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fddac15eb90 0x7fddac152d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:29.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.487+0000 7fddb458d640 1 --2- 172.21.15.27:0/4042357995 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fddac0d5200 0x7fddac1534b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:29.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.487+0000 7fddb458d640 1 --2- 172.21.15.27:0/4042357995 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fdd9806bf80 0x7fdd9806e440 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:29.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.487+0000 7fddb458d640 1 --2- 172.21.15.27:0/4042357995 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fddac0d1720 0x7fddac0ce560 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:29.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.487+0000 7fddb458d640 1 -- 172.21.15.27:0/4042357995 >> 172.21.15.27:0/4042357995 conn(0x7fddac06dce0 msgr2=0x7fddac156a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:29.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.487+0000 7fddb458d640 1 -- 172.21.15.27:0/4042357995 shutdown_connections 2024-09-12T03:20:29.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.487+0000 7fddb458d640 1 -- 172.21.15.27:0/4042357995 wait complete. 2024-09-12T03:20:29.499 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-12T03:20:29.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.602+0000 7fecb6d8d640 1 Processor -- start 2024-09-12T03:20:29.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.602+0000 7fecb6d8d640 1 -- start start 2024-09-12T03:20:29.604 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.603+0000 7fecb6d8d640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fecb015b2b0 0x7fecb00b3560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:29.604 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.603+0000 7fecb6d8d640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fecb006e9b0 0x7fecb006ee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:29.604 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.603+0000 7fecb6d8d640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fecb00ce410 con 0x7fecb0072d70 2024-09-12T03:20:29.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.603+0000 7fecb6d8d640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7fecb00720a0 con 0x7fecb015b2b0 2024-09-12T03:20:29.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.603+0000 7fecb6d8d640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fecb00732c0 con 0x7fecb006e9b0 2024-09-12T03:20:29.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.603+0000 7fecb4b29640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fecb015b2b0 0x7fecb00b3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:29.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.603+0000 7fecaffff640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fecb0072d70 0x7fecb0072430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:57666/0 (socket says 172.21.15.27:57666) 2024-09-12T03:20:29.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.603+0000 7fecaffff640 1 -- 172.21.15.27:0/3821837106 learned_addr learned my addr 172.21.15.27:0/3821837106 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:20:29.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.603+0000 7fecb4b29640 1 -- 172.21.15.27:0/3821837106 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fecb006e9b0 msgr2=0x7fecb006ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:29.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.603+0000 7fecb4b29640 1 --2- 172.21.15.27:0/3821837106 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fecb006e9b0 0x7fecb006ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:29.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.603+0000 7fecb4b29640 1 -- 172.21.15.27:0/3821837106 >> v1:172.21.15.27:6789/0 conn(0x7fecb0072d70 legacy=0x7fecb0072430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:29.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.603+0000 7fecb4b29640 1 -- 172.21.15.27:0/3821837106 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecb0071ae0 con 0x7fecb015b2b0 2024-09-12T03:20:29.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.603+0000 7fecaf7fe640 1 -- 172.21.15.27:0/3821837106 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2427790883 0 0) 0x7fec9c001110 con 0x7fecb0072d70 2024-09-12T03:20:29.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.604+0000 7fecb4b29640 1 --2- 172.21.15.27:0/3821837106 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fecb015b2b0 0x7fecb00b3560 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7feca40097a0 tx=0x7feca402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79f39e77ea84fc06 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:29.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.604+0000 7fecaf7fe640 1 -- 172.21.15.27:0/3821837106 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feca4037070 con 0x7fecb015b2b0 2024-09-12T03:20:29.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.604+0000 7fecaf7fe640 1 -- 172.21.15.27:0/3821837106 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feca4032440 con 0x7fecb015b2b0 2024-09-12T03:20:29.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.604+0000 7fecaf7fe640 1 -- 172.21.15.27:0/3821837106 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feca4032810 con 0x7fecb015b2b0 2024-09-12T03:20:29.608 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.605+0000 7fecb6d8d640 1 -- 172.21.15.27:0/3821837106 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fecb015b2b0 msgr2=0x7fecb00b3560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:29.608 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.605+0000 7fecb6d8d640 1 --2- 172.21.15.27:0/3821837106 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fecb015b2b0 0x7fecb00b3560 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7feca40097a0 tx=0x7feca402a180 comp rx=0 tx=0).stop 2024-09-12T03:20:29.608 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.605+0000 7fecb6d8d640 1 -- 172.21.15.27:0/3821837106 shutdown_connections 2024-09-12T03:20:29.608 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.605+0000 7fecb6d8d640 1 --2- 172.21.15.27:0/3821837106 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fecb006e9b0 0x7fecb006ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:29.608 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.605+0000 7fecb6d8d640 1 --2- 172.21.15.27:0/3821837106 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fecb015b2b0 0x7fecb00b3560 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:29.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.605+0000 7fecb6d8d640 1 -- 172.21.15.27:0/3821837106 >> 172.21.15.27:0/3821837106 conn(0x7fecb0062c60 msgr2=0x7fecb006bb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:29.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.605+0000 7fecb6d8d640 1 -- 172.21.15.27:0/3821837106 shutdown_connections 2024-09-12T03:20:29.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.605+0000 7fecb6d8d640 1 -- 172.21.15.27:0/3821837106 wait complete. 2024-09-12T03:20:29.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.606+0000 7fecb6d8d640 1 Processor -- start 2024-09-12T03:20:29.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.606+0000 7fecb6d8d640 1 -- start start 2024-09-12T03:20:29.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.607+0000 7fecb6d8d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fecb006e9b0 0x7fecb01491f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:29.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.607+0000 7fecb6d8d640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fecb0072d70 0x7fecb0148c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:29.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.607+0000 7fecb6d8d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fecb015b2b0 0x7fecb014aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:29.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.607+0000 7fecb6d8d640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fecb015b690 con 0x7fecb006e9b0 2024-09-12T03:20:29.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.607+0000 7fecb6d8d640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fecb00ce690 con 0x7fecb015b2b0 2024-09-12T03:20:29.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.607+0000 7fecb6d8d640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fecb015b810 con 0x7fecb0072d70 2024-09-12T03:20:29.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.607+0000 7fecb4b29640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fecb006e9b0 0x7fecb01491f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:29.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.607+0000 7fecb4b29640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fecb006e9b0 0x7fecb01491f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46852/0 (socket says 172.21.15.27:46852) 2024-09-12T03:20:29.610 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.607+0000 7fecb4b29640 1 -- 172.21.15.27:0/2745909917 learned_addr learned my addr 172.21.15.27:0/2745909917 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:20:29.610 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.607+0000 7fecb4b29640 1 -- 172.21.15.27:0/2745909917 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fecb0072d70 msgr2=0x7fecb0148c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:29.610 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.607+0000 7fecb4b29640 1 --2- 172.21.15.27:0/2745909917 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fecb0072d70 0x7fecb0148c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:29.610 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.607+0000 7fecb4b29640 1 -- 172.21.15.27:0/2745909917 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fecb015b2b0 msgr2=0x7fecb014aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:29.610 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.607+0000 7fecb4b29640 1 --2- 172.21.15.27:0/2745909917 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fecb015b2b0 0x7fecb014aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:29.610 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.607+0000 7fecb4b29640 1 -- 172.21.15.27:0/2745909917 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecb01515f0 con 0x7fecb006e9b0 2024-09-12T03:20:29.610 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.608+0000 7fecb4b29640 1 --2- 172.21.15.27:0/2745909917 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fecb006e9b0 0x7fecb01491f0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7feca40097a0 tx=0x7feca40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:29.610 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.608+0000 7fecad7fa640 1 -- 172.21.15.27:0/2745909917 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feca4041070 con 0x7fecb006e9b0 2024-09-12T03:20:29.610 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.608+0000 7fecad7fa640 1 -- 172.21.15.27:0/2745909917 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feca403c870 con 0x7fecb006e9b0 2024-09-12T03:20:29.610 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.608+0000 7fecad7fa640 1 -- 172.21.15.27:0/2745909917 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feca4037040 con 0x7fecb006e9b0 2024-09-12T03:20:29.610 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.608+0000 7fecb6d8d640 1 -- 172.21.15.27:0/2745909917 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fecb014ccc0 con 0x7fecb006e9b0 2024-09-12T03:20:29.610 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.608+0000 7fecb6d8d640 1 -- 172.21.15.27:0/2745909917 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fecb014c9e0 con 0x7fecb006e9b0 2024-09-12T03:20:29.611 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.609+0000 7fecb6d8d640 1 -- 172.21.15.27:0/2745909917 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fecb00b3560 con 0x7fecb006e9b0 2024-09-12T03:20:29.611 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.609+0000 7fecad7fa640 1 -- 172.21.15.27:0/2745909917 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 90600+0+0 (secure 0 0 0) 0x7feca4044070 con 0x7fecb006e9b0 2024-09-12T03:20:29.611 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.610+0000 7fecad7fa640 1 --2- 172.21.15.27:0/2745909917 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fec9406bf50 0x7fec9406e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:29.611 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.610+0000 7fecad7fa640 1 -- 172.21.15.27:0/2745909917 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 5709+0+0 (secure 0 0 0) 0x7feca40abae0 con 0x7fecb006e9b0 2024-09-12T03:20:29.612 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.610+0000 7fecaffff640 1 --2- 172.21.15.27:0/2745909917 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fec9406bf50 0x7fec9406e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:29.612 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.610+0000 7fecaffff640 1 --2- 172.21.15.27:0/2745909917 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fec9406bf50 0x7fec9406e410 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fecb0071ab0 tx=0x7fec9c001310 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:29.614 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.614+0000 7fecad7fa640 1 -- 172.21.15.27:0/2745909917 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fecb01515f0 con 0x7fecb006e9b0 2024-09-12T03:20:29.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.789+0000 7fecb6d8d640 1 -- 172.21.15.27:0/2745909917 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fecb006e350 con 0x7fecb006e9b0 2024-09-12T03:20:29.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.790+0000 7fecad7fa640 1 -- 172.21.15.27:0/2745909917 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) ==== 74+0+16685 (secure 0 0 0) 0x7feca407a530 con 0x7fecb006e9b0 2024-09-12T03:20:29.790 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:20:29.791 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":80,"fsid":"d2df504e-2647-4385-a5db-9fdf69f487a0","created":"2024-09-12T03:16:07.778937+0000","modified":"2024-09-12T03:20:28.499239+0000","last_up_change":"2024-09-12T03:16:19.314778+0000","last_in_change":"2024-09-12T03:16:09.893086+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T03:16:20.097646+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-09-12T03:20:26.167239+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-09-12T03:20:27.491764+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1851116c-1fb3-4dc9-9772-f90e2ef4c355","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6817","nonce":3002321488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6819","nonce":3002321488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6823","nonce":3002321488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6821","nonce":3002321488}]},"public_addr":"172.21.15.27:6817/3002321488","cluster_addr":"172.21.15.27:6819/3002321488","heartbeat_back_addr":"172.21.15.27:6823/3002321488","heartbeat_front_addr":"172.21.15.27:6821/3002321488","state":["exists","up"]},{"osd":1,"uuid":"f38ae17d-73eb-4204-b99d-975fa806fb87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6809","nonce":1613102886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6811","nonce":1613102886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6815","nonce":1613102886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6813","nonce":1613102886}]},"public_addr":"172.21.15.27:6809/1613102886","cluster_addr":"172.21.15.27:6811/1613102886","heartbeat_back_addr":"172.21.15.27:6815/1613102886","heartbeat_front_addr":"172.21.15.27:6813/1613102886","state":["exists","up"]},{"osd":2,"uuid":"fb12dcc8-bdd0-42b8-b46d-8b10106224bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6825","nonce":3534723355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6827","nonce":3534723355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6831","nonce":3534723355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6829","nonce":3534723355}]},"public_addr":"172.21.15.27:6825/3534723355","cluster_addr":"172.21.15.27:6827/3534723355","heartbeat_back_addr":"172.21.15.27:6831/3534723355","heartbeat_front_addr":"172.21.15.27:6829/3534723355","state":["exists","up"]},{"osd":3,"uuid":"d74d3f49-9c24-4b9f-98e2-3ecd7b5ef3b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6801","nonce":80661598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6803","nonce":80661598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6807","nonce":80661598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6805","nonce":80661598}]},"public_addr":"172.21.15.27:6801/80661598","cluster_addr":"172.21.15.27:6803/80661598","heartbeat_back_addr":"172.21.15.27:6807/80661598","heartbeat_front_addr":"172.21.15.27:6805/80661598","state":["exists","up"]},{"osd":4,"uuid":"2bf123f2-1b0d-41c4-aa3b-805bab267e59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6817","nonce":3943189827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6819","nonce":3943189827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6823","nonce":3943189827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6821","nonce":3943189827}]},"public_addr":"172.21.15.45:6817/3943189827","cluster_addr":"172.21.15.45:6819/3943189827","heartbeat_back_addr":"172.21.15.45:6823/3943189827","heartbeat_front_addr":"172.21.15.45:6821/3943189827","state":["exists","up"]},{"osd":5,"uuid":"c49ba5da-91c6-4a86-a97b-4aba89c253ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6809","nonce":3620760330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6811","nonce":3620760330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6815","nonce":3620760330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6813","nonce":3620760330}]},"public_addr":"172.21.15.45:6809/3620760330","cluster_addr":"172.21.15.45:6811/3620760330","heartbeat_back_addr":"172.21.15.45:6815/3620760330","heartbeat_front_addr":"172.21.15.45:6813/3620760330","state":["exists","up"]},{"osd":6,"uuid":"b0123475-89f7-4a34-9e33-d797b99dd9dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6801","nonce":1989083510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6803","nonce":1989083510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6807","nonce":1989083510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6805","nonce":1989083510}]},"public_addr":"172.21.15.45:6801/1989083510","cluster_addr":"172.21.15.45:6803/1989083510","heartbeat_back_addr":"172.21.15.45:6807/1989083510","heartbeat_front_addr":"172.21.15.45:6805/1989083510","state":["exists","up"]},{"osd":7,"uuid":"4dad773d-b7d8-46fc-8a10-a972aad27b29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6825","nonce":1513300294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6827","nonce":1513300294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6831","nonce":1513300294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6829","nonce":1513300294}]},"public_addr":"172.21.15.45:6825/1513300294","cluster_addr":"172.21.15.45:6827/1513300294","heartbeat_back_addr":"172.21.15.45:6831/1513300294","heartbeat_front_addr":"172.21.15.45:6829/1513300294","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.881855+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:17.133252+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.735339+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.500742+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.477571+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:17.167667+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.470291+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.873743+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T03:20:29.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.792+0000 7fecb6d8d640 1 -- 172.21.15.27:0/2745909917 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fec9406bf50 msgr2=0x7fec9406e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:29.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.792+0000 7fecb6d8d640 1 --2- 172.21.15.27:0/2745909917 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fec9406bf50 0x7fec9406e410 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fecb0071ab0 tx=0x7fec9c001310 comp rx=0 tx=0).stop 2024-09-12T03:20:29.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.792+0000 7fecb6d8d640 1 -- 172.21.15.27:0/2745909917 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fecb006e9b0 msgr2=0x7fecb01491f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:29.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.792+0000 7fecb6d8d640 1 --2- 172.21.15.27:0/2745909917 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fecb006e9b0 0x7fecb01491f0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7feca40097a0 tx=0x7feca40046c0 comp rx=0 tx=0).stop 2024-09-12T03:20:29.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.792+0000 7fecb6d8d640 1 -- 172.21.15.27:0/2745909917 shutdown_connections 2024-09-12T03:20:29.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.792+0000 7fecb6d8d640 1 --2- 172.21.15.27:0/2745909917 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fecb015b2b0 0x7fecb014aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:29.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.792+0000 7fecb6d8d640 1 --2- 172.21.15.27:0/2745909917 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fecb0072d70 0x7fecb0148c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:29.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.792+0000 7fecb6d8d640 1 --2- 172.21.15.27:0/2745909917 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fec9406bf50 0x7fec9406e410 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:29.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.792+0000 7fecb6d8d640 1 --2- 172.21.15.27:0/2745909917 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fecb006e9b0 0x7fecb01491f0 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:29.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.792+0000 7fecb6d8d640 1 -- 172.21.15.27:0/2745909917 >> 172.21.15.27:0/2745909917 conn(0x7fecb0062c60 msgr2=0x7fecb006d8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:29.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.792+0000 7fecb6d8d640 1 -- 172.21.15.27:0/2745909917 shutdown_connections 2024-09-12T03:20:29.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.792+0000 7fecb6d8d640 1 -- 172.21.15.27:0/2745909917 wait complete. 2024-09-12T03:20:29.804 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-12T03:20:29.909 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.909+0000 7f029b477640 1 Processor -- start 2024-09-12T03:20:29.909 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.910+0000 7f029b477640 1 -- start start 2024-09-12T03:20:29.911 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.910+0000 7f029b477640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f029415c670 0x7f02940b4240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:29.911 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.910+0000 7f029b477640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f0294073ef0 0x7f02941607c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:29.911 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.910+0000 7f029b477640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f0294160d00 0x7f02940cef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:29.911 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.910+0000 7f029b477640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f02940cff10 con 0x7f0294160d00 2024-09-12T03:20:29.911 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.910+0000 7f029b477640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f0294160050 con 0x7f0294073ef0 2024-09-12T03:20:29.911 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.910+0000 7f029b477640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f02940b4940 con 0x7f029415c670 2024-09-12T03:20:29.911 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.911+0000 7f0299a14640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f0294160d00 0x7f02940cef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:29.911 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.911+0000 7f0299a14640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f0294160d00 0x7f02940cef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46868/0 (socket says 172.21.15.27:46868) 2024-09-12T03:20:29.912 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.911+0000 7f0299a14640 1 -- 172.21.15.27:0/1039234381 learned_addr learned my addr 172.21.15.27:0/1039234381 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:20:29.912 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.911+0000 7f0298a12640 1 --2- 172.21.15.27:0/1039234381 >> v2:172.21.15.45:3300/0 conn(0x7f0294073ef0 0x7f02941607c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:29.912 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.911+0000 7f0299213640 1 --2- 172.21.15.27:0/1039234381 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f029415c670 0x7f02940b4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:29.912 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.911+0000 7f0299a14640 1 -- 172.21.15.27:0/1039234381 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f029415c670 msgr2=0x7f02940b4240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:29.912 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.911+0000 7f0299a14640 1 --2- 172.21.15.27:0/1039234381 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f029415c670 0x7f02940b4240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:29.912 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.911+0000 7f0299a14640 1 -- 172.21.15.27:0/1039234381 >> v2:172.21.15.45:3300/0 conn(0x7f0294073ef0 msgr2=0x7f02941607c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:29.912 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.911+0000 7f0299a14640 1 --2- 172.21.15.27:0/1039234381 >> v2:172.21.15.45:3300/0 conn(0x7f0294073ef0 0x7f02941607c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:29.912 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.911+0000 7f0299a14640 1 -- 172.21.15.27:0/1039234381 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02940b4c20 con 0x7f0294160d00 2024-09-12T03:20:29.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.912+0000 7f0299a14640 1 --2- 172.21.15.27:0/1039234381 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0294160d00 0x7f02940cef90 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f0294069c20 tx=0x7f028c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b10d47bcf9af863 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:29.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.912+0000 7f0293fff640 1 -- 172.21.15.27:0/1039234381 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f028c037070 con 0x7f0294160d00 2024-09-12T03:20:29.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.912+0000 7f0293fff640 1 -- 172.21.15.27:0/1039234381 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f028c032440 con 0x7f0294160d00 2024-09-12T03:20:29.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.912+0000 7f0293fff640 1 -- 172.21.15.27:0/1039234381 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f028c032810 con 0x7f0294160d00 2024-09-12T03:20:29.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.912+0000 7f029b477640 1 -- 172.21.15.27:0/1039234381 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0294160d00 msgr2=0x7f02940cef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:29.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.912+0000 7f029b477640 1 --2- 172.21.15.27:0/1039234381 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0294160d00 0x7f02940cef90 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f0294069c20 tx=0x7f028c02a1e0 comp rx=0 tx=0).stop 2024-09-12T03:20:29.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.912+0000 7f029b477640 1 -- 172.21.15.27:0/1039234381 shutdown_connections 2024-09-12T03:20:29.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.912+0000 7f029b477640 1 --2- 172.21.15.27:0/1039234381 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0294160d00 0x7f02940cef90 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:29.915 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.912+0000 7f029b477640 1 --2- 172.21.15.27:0/1039234381 >> v2:172.21.15.45:3300/0 conn(0x7f0294073ef0 0x7f02941607c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:29.915 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.912+0000 7f029b477640 1 --2- 172.21.15.27:0/1039234381 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f029415c670 0x7f02940b4240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:29.915 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.912+0000 7f029b477640 1 -- 172.21.15.27:0/1039234381 >> 172.21.15.27:0/1039234381 conn(0x7f029406f320 msgr2=0x7f029406f700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:29.915 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.913+0000 7f029b477640 1 -- 172.21.15.27:0/1039234381 shutdown_connections 2024-09-12T03:20:29.915 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.913+0000 7f029b477640 1 -- 172.21.15.27:0/1039234381 wait complete. 2024-09-12T03:20:29.915 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.914+0000 7f029b477640 1 Processor -- start 2024-09-12T03:20:29.915 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.914+0000 7f029b477640 1 -- start start 2024-09-12T03:20:29.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.914+0000 7f029b477640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0294073ef0 0x7f029414a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:29.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.914+0000 7f029b477640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f029415c670 0x7f02940b3920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:29.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.914+0000 7f029b477640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0294160d00 0x7f02940b3e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:29.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.914+0000 7f029b477640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f02940cf4d0 con 0x7f0294160d00 2024-09-12T03:20:29.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.914+0000 7f029b477640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f02941602a0 con 0x7f0294073ef0 2024-09-12T03:20:29.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.914+0000 7f029b477640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f02940cf650 con 0x7f029415c670 2024-09-12T03:20:29.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.915+0000 7f0299a14640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0294160d00 0x7f02940b3e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:29.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.915+0000 7f0299a14640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0294160d00 0x7f02940b3e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46876/0 (socket says 172.21.15.27:46876) 2024-09-12T03:20:29.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.915+0000 7f0299a14640 1 -- 172.21.15.27:0/1579705503 learned_addr learned my addr 172.21.15.27:0/1579705503 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:20:29.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.915+0000 7f0299213640 1 --2- 172.21.15.27:0/1579705503 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0294073ef0 0x7f029414a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:29.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.915+0000 7f0298a12640 1 --2- 172.21.15.27:0/1579705503 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f029415c670 0x7f02940b3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:29.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.915+0000 7f0299a14640 1 -- 172.21.15.27:0/1579705503 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f029415c670 msgr2=0x7f02940b3920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:29.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.915+0000 7f0299a14640 1 --2- 172.21.15.27:0/1579705503 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f029415c670 0x7f02940b3920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:29.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.915+0000 7f0299a14640 1 -- 172.21.15.27:0/1579705503 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0294073ef0 msgr2=0x7f029414a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:29.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.915+0000 7f0299a14640 1 --2- 172.21.15.27:0/1579705503 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0294073ef0 0x7f029414a2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:29.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.915+0000 7f0299a14640 1 -- 172.21.15.27:0/1579705503 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02941f33b0 con 0x7f0294160d00 2024-09-12T03:20:29.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.915+0000 7f0298a12640 1 --2- 172.21.15.27:0/1579705503 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f029415c670 0x7f02940b3920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:20:29.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.915+0000 7f0299a14640 1 --2- 172.21.15.27:0/1579705503 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0294160d00 0x7f02940b3e60 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f028c02a220 tx=0x7f028c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:29.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.915+0000 7f0291ffb640 1 -- 172.21.15.27:0/1579705503 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f028c041020 con 0x7f0294160d00 2024-09-12T03:20:29.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.915+0000 7f0291ffb640 1 -- 172.21.15.27:0/1579705503 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f028c0306e0 con 0x7f0294160d00 2024-09-12T03:20:29.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.915+0000 7f029b477640 1 -- 172.21.15.27:0/1579705503 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02941f3690 con 0x7f0294160d00 2024-09-12T03:20:29.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.915+0000 7f0291ffb640 1 -- 172.21.15.27:0/1579705503 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f028c037040 con 0x7f0294160d00 2024-09-12T03:20:29.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.916+0000 7f029b477640 1 -- 172.21.15.27:0/1579705503 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02941f3c50 con 0x7f0294160d00 2024-09-12T03:20:29.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.917+0000 7f029b477640 1 -- 172.21.15.27:0/1579705503 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02941607c0 con 0x7f0294160d00 2024-09-12T03:20:29.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.917+0000 7f0291ffb640 1 -- 172.21.15.27:0/1579705503 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 90600+0+0 (secure 0 0 0) 0x7f028c044020 con 0x7f0294160d00 2024-09-12T03:20:29.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.918+0000 7f0291ffb640 1 --2- 172.21.15.27:0/1579705503 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f028006bf50 0x7f028006e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:29.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.918+0000 7f0291ffb640 1 -- 172.21.15.27:0/1579705503 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 5709+0+0 (secure 0 0 0) 0x7f028c0ab690 con 0x7f0294160d00 2024-09-12T03:20:29.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.918+0000 7f0299213640 1 --2- 172.21.15.27:0/1579705503 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f028006bf50 0x7f028006e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:29.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.918+0000 7f0299213640 1 --2- 172.21.15.27:0/1579705503 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f028006bf50 0x7f028006e410 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f0284003b80 tx=0x7f02840098d0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:29.923 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:29.922+0000 7f0291ffb640 1 -- 172.21.15.27:0/1579705503 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f028c04b030 con 0x7f0294160d00 2024-09-12T03:20:30.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:30.132+0000 7f029b477640 1 -- 172.21.15.27:0/1579705503 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f02940cf0b0 con 0x7f0294160d00 2024-09-12T03:20:30.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:30.167+0000 7f0291ffb640 1 -- 172.21.15.27:0/1579705503 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v45) ==== 118+0+0 (secure 0 0 0) 0x7f028c079fc0 con 0x7f0294160d00 2024-09-12T03:20:30.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:30.169+0000 7f029b477640 1 -- 172.21.15.27:0/1579705503 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f028006bf50 msgr2=0x7f028006e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:30.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:30.169+0000 7f029b477640 1 --2- 172.21.15.27:0/1579705503 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f028006bf50 0x7f028006e410 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f0284003b80 tx=0x7f02840098d0 comp rx=0 tx=0).stop 2024-09-12T03:20:30.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:30.169+0000 7f029b477640 1 -- 172.21.15.27:0/1579705503 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0294160d00 msgr2=0x7f02940b3e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:30.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:30.169+0000 7f029b477640 1 --2- 172.21.15.27:0/1579705503 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0294160d00 0x7f02940b3e60 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f028c02a220 tx=0x7f028c0046c0 comp rx=0 tx=0).stop 2024-09-12T03:20:30.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:30.169+0000 7f029b477640 1 -- 172.21.15.27:0/1579705503 shutdown_connections 2024-09-12T03:20:30.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:30.169+0000 7f029b477640 1 --2- 172.21.15.27:0/1579705503 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f028006bf50 0x7f028006e410 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:30.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:30.169+0000 7f029b477640 1 --2- 172.21.15.27:0/1579705503 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0294160d00 0x7f02940b3e60 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:30.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:30.169+0000 7f029b477640 1 --2- 172.21.15.27:0/1579705503 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f029415c670 0x7f02940b3920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:30.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:30.169+0000 7f029b477640 1 --2- 172.21.15.27:0/1579705503 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0294073ef0 0x7f029414a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:30.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:30.169+0000 7f029b477640 1 -- 172.21.15.27:0/1579705503 >> 172.21.15.27:0/1579705503 conn(0x7f029406f320 msgr2=0x7f02940b2cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:30.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:30.170+0000 7f029b477640 1 -- 172.21.15.27:0/1579705503 shutdown_connections 2024-09-12T03:20:30.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:30.170+0000 7f029b477640 1 -- 172.21.15.27:0/1579705503 wait complete. 2024-09-12T03:20:30.182 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-12T03:20:30.182 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-12T03:20:30.287 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:30.287+0000 7fe976a65640 1 Processor -- start 2024-09-12T03:20:30.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:30.287+0000 7fe976a65640 1 -- start start 2024-09-12T03:20:30.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:30.288+0000 7fe976a65640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fe9700d3790 0x7fe9700d1da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:30.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:30.288+0000 7fe976a65640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe9700d2e30 con 0x7fe97006dfd0 2024-09-12T03:20:30.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:30.288+0000 7fe976a65640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe970162420 con 0x7fe970162040 2024-09-12T03:20:30.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:30.288+0000 7fe976a65640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fe9700d30d0 con 0x7fe9700d3790 2024-09-12T03:20:30.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:30.288+0000 7fe96ffff640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fe97006dfd0 0x7fe9700cf320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:57682/0 (socket says 172.21.15.27:57682) 2024-09-12T03:20:30.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:30.288+0000 7fe96ffff640 1 -- 172.21.15.27:0/3285415929 learned_addr learned my addr 172.21.15.27:0/3285415929 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:20:30.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:30.289+0000 7fe975002640 1 --2- 172.21.15.27:0/3285415929 >> v2:172.21.15.45:3300/0 conn(0x7fe9700d3790 0x7fe9700d1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:30.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:30.289+0000 7fe96f7fe640 1 -- 172.21.15.27:0/3285415929 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1121754287 0 0) 0x7fe9700d2e30 con 0x7fe97006dfd0 2024-09-12T03:20:30.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:30.289+0000 7fe96f7fe640 1 -- 172.21.15.27:0/3285415929 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe958003520 con 0x7fe97006dfd0 2024-09-12T03:20:30.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:30.289+0000 7fe96f7fe640 1 -- 172.21.15.27:0/3285415929 <== mon.1 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2487026090 0 0) 0x7fe970162420 con 0x7fe970162040 2024-09-12T03:20:30.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:30.289+0000 7fe96f7fe640 1 -- 172.21.15.27:0/3285415929 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe9700d2e30 con 0x7fe970162040 2024-09-12T03:20:30.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:30.289+0000 7fe96f7fe640 1 -- 172.21.15.27:0/3285415929 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1042506836 0 0) 0x7fe958003520 con 0x7fe97006dfd0 2024-09-12T03:20:30.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:30.289+0000 7fe96f7fe640 1 -- 172.21.15.27:0/3285415929 >> v2:172.21.15.45:3300/0 conn(0x7fe9700d3790 msgr2=0x7fe9700d1da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:30.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:30.289+0000 7fe96f7fe640 1 --2- 172.21.15.27:0/3285415929 >> v2:172.21.15.45:3300/0 conn(0x7fe9700d3790 0x7fe9700d1da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:30.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:30.289+0000 7fe96f7fe640 1 -- 172.21.15.27:0/3285415929 >> v1:172.21.15.45:6789/0 conn(0x7fe970162040 legacy=0x7fe97006d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:30.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:30.289+0000 7fe96f7fe640 1 -- 172.21.15.27:0/3285415929 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9700d33b0 con 0x7fe97006dfd0 2024-09-12T03:20:30.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:30.289+0000 7fe96f7fe640 1 -- 172.21.15.27:0/3285415929 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 847755213 0 0) 0x7fe964001670 con 0x7fe97006dfd0 2024-09-12T03:20:30.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:30.289+0000 7fe96f7fe640 1 -- 172.21.15.27:0/3285415929 >> v1:172.21.15.27:6789/0 conn(0x7fe97006dfd0 legacy=0x7fe9700cf320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:30.293 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:30.289+0000 7fe96f7fe640 1 --2- 172.21.15.27:0/3285415929 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe958003b40 0x7fe958003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:30.293 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:30.289+0000 7fe96f7fe640 1 --2- 172.21.15.27:0/3285415929 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe958004800 0x7fe958004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:30.293 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:30.289+0000 7fe96f7fe640 1 --2- 172.21.15.27:0/3285415929 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe9580051c0 0x7fe95800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:30.293 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:30.289+0000 7fe96f7fe640 1 -- 172.21.15.27:0/3285415929 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fe9700d30d0 con 0x7fe9580051c0 2024-09-12T03:20:30.293 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:30.289+0000 7fe96f7fe640 1 -- 172.21.15.27:0/3285415929 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe95800c040 con 0x7fe958004800 2024-09-12T03:20:30.293 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:30.289+0000 7fe96f7fe640 1 -- 172.21.15.27:0/3285415929 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fe95800c240 con 0x7fe958003b40 2024-09-12T03:20:30.293 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:30.289+0000 7fe975002640 1 --2- 172.21.15.27:0/3285415929 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe9580051c0 0x7fe95800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:30.293 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:30.289+0000 7fe96ffff640 1 --2- 172.21.15.27:0/3285415929 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe958004800 0x7fe958004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:30.293 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:30.289+0000 7fe974801640 1 --2- 172.21.15.27:0/3285415929 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe958003b40 0x7fe958003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:30.293 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:30.290+0000 7fe975002640 1 -- 172.21.15.27:0/3285415929 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe958003b40 msgr2=0x7fe958003f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:30.293 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:30.290+0000 7fe975002640 1 --2- 172.21.15.27:0/3285415929 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe958003b40 0x7fe958003f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:30.293 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:30.290+0000 7fe975002640 1 -- 172.21.15.27:0/3285415929 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe958004800 msgr2=0x7fe958004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:30.293 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:30.290+0000 7fe975002640 1 --2- 172.21.15.27:0/3285415929 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe958004800 0x7fe958004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:30.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:30.290+0000 7fe975002640 1 -- 172.21.15.27:0/3285415929 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe958003520 con 0x7fe9580051c0 2024-09-12T03:20:30.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:30.290+0000 7fe975002640 1 -- 172.21.15.27:0/3285415929 reap_dead start 2024-09-12T03:20:30.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:30.290+0000 7fe975002640 1 --2- 172.21.15.27:0/3285415929 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe9580051c0 0x7fe95800b9c0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fe968009f90 tx=0x7fe958005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8d8fcf1f82bbec0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:30.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:30.290+0000 7fe96f7fe640 1 -- 172.21.15.27:0/3285415929 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe968035020 con 0x7fe9580051c0 2024-09-12T03:20:30.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:30.291+0000 7fe96f7fe640 1 -- 172.21.15.27:0/3285415929 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe968003b90 con 0x7fe9580051c0 2024-09-12T03:20:30.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:30.291+0000 7fe96f7fe640 1 -- 172.21.15.27:0/3285415929 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe968009c10 con 0x7fe9580051c0 2024-09-12T03:20:30.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:30.291+0000 7fe976a65640 1 -- 172.21.15.27:0/3285415929 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe9580051c0 msgr2=0x7fe95800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:30.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:30.291+0000 7fe976a65640 1 --2- 172.21.15.27:0/3285415929 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe9580051c0 0x7fe95800b9c0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fe968009f90 tx=0x7fe958005830 comp rx=0 tx=0).stop 2024-09-12T03:20:30.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:30.291+0000 7fe976a65640 1 -- 172.21.15.27:0/3285415929 shutdown_connections 2024-09-12T03:20:30.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:30.291+0000 7fe976a65640 1 --2- 172.21.15.27:0/3285415929 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe9580051c0 0x7fe95800b9c0 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:30.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:30.291+0000 7fe976a65640 1 -- 172.21.15.27:0/3285415929 >> 172.21.15.27:0/3285415929 conn(0x7fe9700b6700 msgr2=0x7fe9700b6b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:30.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:30.291+0000 7fe976a65640 1 -- 172.21.15.27:0/3285415929 shutdown_connections 2024-09-12T03:20:30.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:30.291+0000 7fe976a65640 1 -- 172.21.15.27:0/3285415929 wait complete. 2024-09-12T03:20:30.297 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:30.293+0000 7fe976a65640 1 Processor -- start 2024-09-12T03:20:30.297 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:30.293+0000 7fe976a65640 1 -- start start 2024-09-12T03:20:30.297 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:30.293+0000 7fe976a65640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe9580051c0 0x7fe9700b92d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:30.297 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:30.294+0000 7fe976a65640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe9700c9e30 0x7fe9700ca2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:30.298 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:30.294+0000 7fe976a65640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe9701f4700 0x7fe9701f4b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:30.298 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:30.294+0000 7fe976a65640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fe9700d4150 con 0x7fe9700c9e30 2024-09-12T03:20:30.298 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:30.294+0000 7fe976a65640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe9700d4530 con 0x7fe9701f4700 2024-09-12T03:20:30.298 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:30.294+0000 7fe976a65640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fe9700d42d0 con 0x7fe9580051c0 2024-09-12T03:20:30.298 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:30.294+0000 7fe96ffff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe9700c9e30 0x7fe9700ca2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:30.298 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:30.294+0000 7fe96ffff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe9700c9e30 0x7fe9700ca2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46896/0 (socket says 172.21.15.27:46896) 2024-09-12T03:20:30.298 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:30.294+0000 7fe96ffff640 1 -- 172.21.15.27:0/665555531 learned_addr learned my addr 172.21.15.27:0/665555531 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:20:30.298 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:30.294+0000 7fe975002640 1 --2- 172.21.15.27:0/665555531 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe9701f4700 0x7fe9701f4b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:30.298 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:30.294+0000 7fe974801640 1 --2- 172.21.15.27:0/665555531 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe9580051c0 0x7fe9700b92d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:30.298 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:30.294+0000 7fe96ffff640 1 -- 172.21.15.27:0/665555531 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe9580051c0 msgr2=0x7fe9700b92d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:30.298 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:30.294+0000 7fe96ffff640 1 --2- 172.21.15.27:0/665555531 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe9580051c0 0x7fe9700b92d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:30.298 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:30.294+0000 7fe96ffff640 1 -- 172.21.15.27:0/665555531 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe9701f4700 msgr2=0x7fe9701f4b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:30.298 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:30.294+0000 7fe96ffff640 1 --2- 172.21.15.27:0/665555531 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe9701f4700 0x7fe9701f4b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:30.298 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:30.294+0000 7fe96ffff640 1 -- 172.21.15.27:0/665555531 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9701f7af0 con 0x7fe9700c9e30 2024-09-12T03:20:30.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:30.294+0000 7fe96ffff640 1 --2- 172.21.15.27:0/665555531 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe9700c9e30 0x7fe9700ca2b0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fe964007290 tx=0x7fe964007690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:30.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:30.294+0000 7fe96d7fa640 1 -- 172.21.15.27:0/665555531 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe964001630 con 0x7fe9700c9e30 2024-09-12T03:20:30.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:30.294+0000 7fe96d7fa640 1 -- 172.21.15.27:0/665555531 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe964018070 con 0x7fe9700c9e30 2024-09-12T03:20:30.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:30.294+0000 7fe976a65640 1 -- 172.21.15.27:0/665555531 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9701f7d90 con 0x7fe9700c9e30 2024-09-12T03:20:30.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:30.294+0000 7fe96d7fa640 1 -- 172.21.15.27:0/665555531 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe96400fbe0 con 0x7fe9700c9e30 2024-09-12T03:20:30.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:30.294+0000 7fe976a65640 1 -- 172.21.15.27:0/665555531 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9701f8350 con 0x7fe9700c9e30 2024-09-12T03:20:30.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:30.295+0000 7fe976a65640 1 -- 172.21.15.27:0/665555531 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9701f8680 con 0x7fe9700c9e30 2024-09-12T03:20:30.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:30.296+0000 7fe96d7fa640 1 -- 172.21.15.27:0/665555531 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 90600+0+0 (secure 0 0 0) 0x7fe96401d460 con 0x7fe9700c9e30 2024-09-12T03:20:30.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:30.296+0000 7fe96d7fa640 1 --2- 172.21.15.27:0/665555531 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fe95c06bf50 0x7fe95c06e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:30.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:30.296+0000 7fe96d7fa640 1 -- 172.21.15.27:0/665555531 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 5709+0+0 (secure 0 0 0) 0x7fe96408af00 con 0x7fe9700c9e30 2024-09-12T03:20:30.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:30.296+0000 7fe974801640 1 --2- 172.21.15.27:0/665555531 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fe95c06bf50 0x7fe95c06e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:30.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:30.297+0000 7fe974801640 1 --2- 172.21.15.27:0/665555531 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fe95c06bf50 0x7fe95c06e410 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fe960000c00 tx=0x7fe960009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:30.300 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:30.301+0000 7fe96d7fa640 1 -- 172.21.15.27:0/665555531 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fe9640597a0 con 0x7fe9700c9e30 2024-09-12T03:20:30.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:30.509+0000 7fe976a65640 1 -- 172.21.15.27:0/665555531 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fe97006e6e0 con 0x7fe9700c9e30 2024-09-12T03:20:30.937 DEBUG:teuthology.orchestra.run.smithi027:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-12T03:20:30.941 DEBUG:teuthology.orchestra.run.smithi045:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-12T03:20:31.134 INFO:tasks.ceph.mds.a.smithi027.stderr:2024-09-12T03:20:31.134+0000 7f2b90b90640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-12T03:20:31.176 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.176+0000 7fe96d7fa640 1 -- 172.21.15.27:0/665555531 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v46) ==== 122+0+0 (secure 0 0 0) 0x7fe9640590a0 con 0x7fe9700c9e30 2024-09-12T03:20:31.178 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.178+0000 7fe976a65640 1 -- 172.21.15.27:0/665555531 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fe95c06bf50 msgr2=0x7fe95c06e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:31.179 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.178+0000 7fe976a65640 1 --2- 172.21.15.27:0/665555531 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fe95c06bf50 0x7fe95c06e410 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fe960000c00 tx=0x7fe960009040 comp rx=0 tx=0).stop 2024-09-12T03:20:31.179 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.178+0000 7fe976a65640 1 -- 172.21.15.27:0/665555531 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe9700c9e30 msgr2=0x7fe9700ca2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:31.179 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.178+0000 7fe976a65640 1 --2- 172.21.15.27:0/665555531 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe9700c9e30 0x7fe9700ca2b0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fe964007290 tx=0x7fe964007690 comp rx=0 tx=0).stop 2024-09-12T03:20:31.179 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.178+0000 7fe976a65640 1 -- 172.21.15.27:0/665555531 shutdown_connections 2024-09-12T03:20:31.179 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.178+0000 7fe976a65640 1 --2- 172.21.15.27:0/665555531 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe9701f4700 0x7fe9701f4b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:31.179 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.178+0000 7fe976a65640 1 --2- 172.21.15.27:0/665555531 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fe95c06bf50 0x7fe95c06e410 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:31.179 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.178+0000 7fe976a65640 1 --2- 172.21.15.27:0/665555531 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe9700c9e30 0x7fe9700ca2b0 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:31.179 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.178+0000 7fe976a65640 1 --2- 172.21.15.27:0/665555531 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe9580051c0 0x7fe9700b92d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:31.180 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.178+0000 7fe976a65640 1 -- 172.21.15.27:0/665555531 >> 172.21.15.27:0/665555531 conn(0x7fe9700b6700 msgr2=0x7fe9700b4640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:31.180 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.179+0000 7fe976a65640 1 -- 172.21.15.27:0/665555531 shutdown_connections 2024-09-12T03:20:31.180 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.179+0000 7fe976a65640 1 -- 172.21.15.27:0/665555531 wait complete. 2024-09-12T03:20:31.190 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-12T03:20:31.191 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-12T03:20:31.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.295+0000 7fd27359e640 1 Processor -- start 2024-09-12T03:20:31.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.295+0000 7fd27359e640 1 -- start start 2024-09-12T03:20:31.297 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.296+0000 7fd27359e640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fd2740b6080 0x7fd2740b64c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:31.297 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.296+0000 7fd27359e640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fd2740b77a0 0x7fd2740b7bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:31.298 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.296+0000 7fd27359e640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd2740d32b0 con 0x7fd27415e6f0 2024-09-12T03:20:31.298 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.296+0000 7fd27359e640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7fd27415ead0 con 0x7fd2740b6080 2024-09-12T03:20:31.298 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.296+0000 7fd27359e640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fd2740d3e80 con 0x7fd2740b77a0 2024-09-12T03:20:31.298 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.296+0000 7fd271d9b640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fd2740b6080 0x7fd2740b64c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:31.298 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.296+0000 7fd271d9b640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fd2740b6080 0x7fd2740b64c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46898/0 (socket says 172.21.15.27:46898) 2024-09-12T03:20:31.298 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.296+0000 7fd271d9b640 1 -- 172.21.15.27:0/3817502172 learned_addr learned my addr 172.21.15.27:0/3817502172 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:20:31.298 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.296+0000 7fd272d9d640 1 --2- 172.21.15.27:0/3817502172 >> v2:172.21.15.45:3300/0 conn(0x7fd2740b77a0 0x7fd2740b7bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:31.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.296+0000 7fd271d9b640 1 -- 172.21.15.27:0/3817502172 >> v2:172.21.15.45:3300/0 conn(0x7fd2740b77a0 msgr2=0x7fd2740b7bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:31.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.296+0000 7fd271d9b640 1 --2- 172.21.15.27:0/3817502172 >> v2:172.21.15.45:3300/0 conn(0x7fd2740b77a0 0x7fd2740b7bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:31.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.296+0000 7fd271d9b640 1 -- 172.21.15.27:0/3817502172 >> v1:172.21.15.45:6789/0 conn(0x7fd27415e6f0 legacy=0x7fd2740cf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:31.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.296+0000 7fd271d9b640 1 -- 172.21.15.27:0/3817502172 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2740d4310 con 0x7fd2740b6080 2024-09-12T03:20:31.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.297+0000 7fd271d9b640 1 --2- 172.21.15.27:0/3817502172 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd2740b6080 0x7fd2740b64c0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fd2740d0730 tx=0x7fd26802a200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26cea6dbf71f7ab3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:31.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.297+0000 7fd27159a640 1 -- 172.21.15.27:0/3817502172 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd268037070 con 0x7fd2740b6080 2024-09-12T03:20:31.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.297+0000 7fd27159a640 1 -- 172.21.15.27:0/3817502172 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd268032440 con 0x7fd2740b6080 2024-09-12T03:20:31.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.297+0000 7fd27159a640 1 -- 172.21.15.27:0/3817502172 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd268032810 con 0x7fd2740b6080 2024-09-12T03:20:31.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.297+0000 7fd27359e640 1 -- 172.21.15.27:0/3817502172 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd2740b6080 msgr2=0x7fd2740b64c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:31.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.297+0000 7fd27359e640 1 --2- 172.21.15.27:0/3817502172 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd2740b6080 0x7fd2740b64c0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fd2740d0730 tx=0x7fd26802a200 comp rx=0 tx=0).stop 2024-09-12T03:20:31.300 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.298+0000 7fd27359e640 1 -- 172.21.15.27:0/3817502172 shutdown_connections 2024-09-12T03:20:31.300 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.298+0000 7fd27359e640 1 --2- 172.21.15.27:0/3817502172 >> v2:172.21.15.45:3300/0 conn(0x7fd2740b77a0 0x7fd2740b7bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:31.300 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.298+0000 7fd27359e640 1 --2- 172.21.15.27:0/3817502172 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd2740b6080 0x7fd2740b64c0 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:31.300 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.298+0000 7fd27359e640 1 -- 172.21.15.27:0/3817502172 >> 172.21.15.27:0/3817502172 conn(0x7fd2740d1260 msgr2=0x7fd2740d1670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:31.300 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.298+0000 7fd27359e640 1 -- 172.21.15.27:0/3817502172 shutdown_connections 2024-09-12T03:20:31.300 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.298+0000 7fd27359e640 1 -- 172.21.15.27:0/3817502172 wait complete. 2024-09-12T03:20:31.303 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.299+0000 7fd27359e640 1 Processor -- start 2024-09-12T03:20:31.303 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.299+0000 7fd27359e640 1 -- start start 2024-09-12T03:20:31.303 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.300+0000 7fd27359e640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fd2740b6080 0x7fd27415f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:31.303 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.300+0000 7fd27359e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd2740b77a0 0x7fd274160560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:31.304 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.300+0000 7fd27359e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd27415e6f0 0x7fd274160aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:31.304 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.300+0000 7fd27359e640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fd2740d3830 con 0x7fd2740b77a0 2024-09-12T03:20:31.304 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.300+0000 7fd27359e640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fd2740b6a00 con 0x7fd27415e6f0 2024-09-12T03:20:31.304 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.300+0000 7fd27359e640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fd2740d3b10 con 0x7fd2740b6080 2024-09-12T03:20:31.304 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.300+0000 7fd271d9b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd2740b77a0 0x7fd274160560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:31.304 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.300+0000 7fd271d9b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd2740b77a0 0x7fd274160560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46910/0 (socket says 172.21.15.27:46910) 2024-09-12T03:20:31.304 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.300+0000 7fd271d9b640 1 -- 172.21.15.27:0/2835149409 learned_addr learned my addr 172.21.15.27:0/2835149409 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:20:31.304 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.300+0000 7fd272d9d640 1 --2- 172.21.15.27:0/2835149409 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd27415e6f0 0x7fd274160aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:31.305 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.300+0000 7fd27259c640 1 --2- 172.21.15.27:0/2835149409 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fd2740b6080 0x7fd27415f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:31.305 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.300+0000 7fd271d9b640 1 -- 172.21.15.27:0/2835149409 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fd2740b6080 msgr2=0x7fd27415f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:31.305 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.300+0000 7fd271d9b640 1 --2- 172.21.15.27:0/2835149409 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fd2740b6080 0x7fd27415f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:31.305 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.300+0000 7fd271d9b640 1 -- 172.21.15.27:0/2835149409 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd27415e6f0 msgr2=0x7fd274160aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:31.305 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.300+0000 7fd271d9b640 1 --2- 172.21.15.27:0/2835149409 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd27415e6f0 0x7fd274160aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:31.306 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.300+0000 7fd271d9b640 1 -- 172.21.15.27:0/2835149409 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd27415fc50 con 0x7fd2740b77a0 2024-09-12T03:20:31.306 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.300+0000 7fd272d9d640 1 --2- 172.21.15.27:0/2835149409 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd27415e6f0 0x7fd274160aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:20:31.306 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.300+0000 7fd271d9b640 1 --2- 172.21.15.27:0/2835149409 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd2740b77a0 0x7fd274160560 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fd268000c00 tx=0x7fd2680046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:31.306 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.300+0000 7fd262ffd640 1 -- 172.21.15.27:0/2835149409 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd268041020 con 0x7fd2740b77a0 2024-09-12T03:20:31.306 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.300+0000 7fd262ffd640 1 -- 172.21.15.27:0/2835149409 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2680306c0 con 0x7fd2740b77a0 2024-09-12T03:20:31.306 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.300+0000 7fd27359e640 1 -- 172.21.15.27:0/2835149409 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd27415ff30 con 0x7fd2740b77a0 2024-09-12T03:20:31.306 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.300+0000 7fd262ffd640 1 -- 172.21.15.27:0/2835149409 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd268037040 con 0x7fd2740b77a0 2024-09-12T03:20:31.306 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.301+0000 7fd27359e640 1 -- 172.21.15.27:0/2835149409 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2740c8110 con 0x7fd2740b77a0 2024-09-12T03:20:31.306 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.301+0000 7fd27359e640 1 -- 172.21.15.27:0/2835149409 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2740d2390 con 0x7fd2740b77a0 2024-09-12T03:20:31.306 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.302+0000 7fd262ffd640 1 -- 172.21.15.27:0/2835149409 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 90600+0+0 (secure 0 0 0) 0x7fd268044020 con 0x7fd2740b77a0 2024-09-12T03:20:31.307 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.302+0000 7fd262ffd640 1 --2- 172.21.15.27:0/2835149409 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fd25406bf50 0x7fd25406e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:31.307 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.303+0000 7fd262ffd640 1 -- 172.21.15.27:0/2835149409 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 5709+0+0 (secure 0 0 0) 0x7fd2680ab7d0 con 0x7fd2740b77a0 2024-09-12T03:20:31.307 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.303+0000 7fd27259c640 1 --2- 172.21.15.27:0/2835149409 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fd25406bf50 0x7fd25406e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:31.307 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.303+0000 7fd27259c640 1 --2- 172.21.15.27:0/2835149409 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fd25406bf50 0x7fd25406e410 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fd264004ea0 tx=0x7fd264009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:31.307 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.307+0000 7fd262ffd640 1 -- 172.21.15.27:0/2835149409 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fd26804b030 con 0x7fd2740b77a0 2024-09-12T03:20:31.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.520+0000 7fd27359e640 1 -- 172.21.15.27:0/2835149409 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd274069150 con 0x7fd2740b77a0 2024-09-12T03:20:31.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.520+0000 7fd262ffd640 1 -- 172.21.15.27:0/2835149409 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) ==== 94+0+5222 (secure 0 0 0) 0x7fd26807a100 con 0x7fd2740b77a0 2024-09-12T03:20:31.521 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:20:31.521 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":46,"btime":"2024-09-12T03:20:31:172713+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5669,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/2036680565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2036680565},{"type":"v1","addr":"172.21.15.45:6834","nonce":2036680565}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":43},{"gid":6094,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/874218126","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":874218126},{"type":"v1","addr":"172.21.15.45:6835","nonce":874218126}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":43},{"gid":6483,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6834/1466511159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":1466511159},{"type":"v1","addr":"172.21.15.27:6834","nonce":1466511159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-12T03:20:28.499550+0000","modified":"2024-09-12T03:20:30.510730+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6486},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6486":{"gid":6486,"name":"a","rank":0,"incarnation":44,"state":"up:active","state_seq":3,"addr":"172.21.15.27:6835/3209963418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":3209963418},{"type":"v1","addr":"172.21.15.27:6835","nonce":3209963418}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6486,"qdb_cluster":[6486]},"id":6}]} 2024-09-12T03:20:31.521 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 46 2024-09-12T03:20:31.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.523+0000 7fd27359e640 1 -- 172.21.15.27:0/2835149409 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fd25406bf50 msgr2=0x7fd25406e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:31.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.523+0000 7fd27359e640 1 --2- 172.21.15.27:0/2835149409 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fd25406bf50 0x7fd25406e410 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fd264004ea0 tx=0x7fd264009040 comp rx=0 tx=0).stop 2024-09-12T03:20:31.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.523+0000 7fd27359e640 1 -- 172.21.15.27:0/2835149409 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd2740b77a0 msgr2=0x7fd274160560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:31.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.523+0000 7fd27359e640 1 --2- 172.21.15.27:0/2835149409 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd2740b77a0 0x7fd274160560 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fd268000c00 tx=0x7fd2680046c0 comp rx=0 tx=0).stop 2024-09-12T03:20:31.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.523+0000 7fd27359e640 1 -- 172.21.15.27:0/2835149409 shutdown_connections 2024-09-12T03:20:31.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.523+0000 7fd27359e640 1 --2- 172.21.15.27:0/2835149409 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd27415e6f0 0x7fd274160aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:31.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.523+0000 7fd27359e640 1 --2- 172.21.15.27:0/2835149409 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fd25406bf50 0x7fd25406e410 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:31.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.523+0000 7fd27359e640 1 --2- 172.21.15.27:0/2835149409 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd2740b77a0 0x7fd274160560 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:31.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.523+0000 7fd27359e640 1 --2- 172.21.15.27:0/2835149409 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fd2740b6080 0x7fd27415f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:31.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.523+0000 7fd27359e640 1 -- 172.21.15.27:0/2835149409 >> 172.21.15.27:0/2835149409 conn(0x7fd2740d1260 msgr2=0x7fd2740b7fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:31.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.523+0000 7fd27359e640 1 -- 172.21.15.27:0/2835149409 shutdown_connections 2024-09-12T03:20:31.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.523+0000 7fd27359e640 1 -- 172.21.15.27:0/2835149409 wait complete. 2024-09-12T03:20:31.534 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-12T03:20:31.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.635+0000 7f6536331640 1 Processor -- start 2024-09-12T03:20:31.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.635+0000 7f6536331640 1 -- start start 2024-09-12T03:20:31.638 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.636+0000 7f6536331640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f65300b1ca0 0x7f653015f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:31.638 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.636+0000 7f6536331640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f65300cf660 0x7f65300d2780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:31.639 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.636+0000 7f6536331640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f653015fd70 con 0x7f653006f360 2024-09-12T03:20:31.639 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.636+0000 7f6536331640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f65300d22f0 con 0x7f65300cf660 2024-09-12T03:20:31.639 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.636+0000 7f6536331640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f653006cc40 con 0x7f65300b1ca0 2024-09-12T03:20:31.639 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.636+0000 7f65348ce640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f65300cf660 0x7f65300d2780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:31.639 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.636+0000 7f65348ce640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f65300cf660 0x7f65300d2780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46914/0 (socket says 172.21.15.27:46914) 2024-09-12T03:20:31.639 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.636+0000 7f65348ce640 1 -- 172.21.15.27:0/3673046295 learned_addr learned my addr 172.21.15.27:0/3673046295 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:20:31.639 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.637+0000 7f652ffff640 1 --2- 172.21.15.27:0/3673046295 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f65300b1ca0 0x7f653015f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:31.639 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.637+0000 7f65348ce640 1 -- 172.21.15.27:0/3673046295 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f65300b1ca0 msgr2=0x7f653015f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:31.639 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.637+0000 7f65348ce640 1 --2- 172.21.15.27:0/3673046295 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f65300b1ca0 0x7f653015f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:31.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.637+0000 7f65348ce640 1 -- 172.21.15.27:0/3673046295 >> v1:172.21.15.45:6789/0 conn(0x7f653006f360 legacy=0x7f65300cef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:31.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.637+0000 7f65348ce640 1 -- 172.21.15.27:0/3673046295 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65300d1ce0 con 0x7f65300cf660 2024-09-12T03:20:31.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.637+0000 7f652ffff640 1 --2- 172.21.15.27:0/3673046295 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f65300b1ca0 0x7f653015f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:20:31.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.637+0000 7f65348ce640 1 --2- 172.21.15.27:0/3673046295 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f65300cf660 0x7f65300d2780 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f6528009e80 tx=0x7f652802a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eddbd4a03f5d1c5e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:31.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.638+0000 7f652effd640 1 -- 172.21.15.27:0/3673046295 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6528036070 con 0x7f65300cf660 2024-09-12T03:20:31.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.638+0000 7f652effd640 1 -- 172.21.15.27:0/3673046295 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6528031600 con 0x7f65300cf660 2024-09-12T03:20:31.641 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.638+0000 7f652effd640 1 -- 172.21.15.27:0/3673046295 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65280319d0 con 0x7f65300cf660 2024-09-12T03:20:31.641 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.638+0000 7f6536331640 1 -- 172.21.15.27:0/3673046295 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f65300cf660 msgr2=0x7f65300d2780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:31.641 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.638+0000 7f6536331640 1 --2- 172.21.15.27:0/3673046295 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f65300cf660 0x7f65300d2780 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f6528009e80 tx=0x7f652802a6f0 comp rx=0 tx=0).stop 2024-09-12T03:20:31.641 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.638+0000 7f6536331640 1 -- 172.21.15.27:0/3673046295 shutdown_connections 2024-09-12T03:20:31.641 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.638+0000 7f6536331640 1 --2- 172.21.15.27:0/3673046295 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f65300cf660 0x7f65300d2780 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:31.641 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.638+0000 7f6536331640 1 --2- 172.21.15.27:0/3673046295 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f65300b1ca0 0x7f653015f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:31.641 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.638+0000 7f6536331640 1 -- 172.21.15.27:0/3673046295 >> 172.21.15.27:0/3673046295 conn(0x7f6530073800 msgr2=0x7f653009ba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:31.641 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.638+0000 7f6536331640 1 -- 172.21.15.27:0/3673046295 shutdown_connections 2024-09-12T03:20:31.642 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.638+0000 7f6536331640 1 -- 172.21.15.27:0/3673046295 wait complete. 2024-09-12T03:20:31.645 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.639+0000 7f6536331640 1 Processor -- start 2024-09-12T03:20:31.645 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.640+0000 7f6536331640 1 -- start start 2024-09-12T03:20:31.645 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.640+0000 7f6536331640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f653006f360 0x7f65301f4640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:31.645 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.640+0000 7f6536331640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f65300b1ca0 0x7f65300cb850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:31.645 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.640+0000 7f6536331640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f65300cf660 0x7f6530151fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:31.645 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.640+0000 7f6536331640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f653006d1c0 con 0x7f65300cf660 2024-09-12T03:20:31.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.640+0000 7f6536331640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f65300d1f80 con 0x7f65300b1ca0 2024-09-12T03:20:31.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.640+0000 7f6536331640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f653006d340 con 0x7f653006f360 2024-09-12T03:20:31.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.641+0000 7f65348ce640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f65300cf660 0x7f6530151fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:31.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.641+0000 7f65348ce640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f65300cf660 0x7f6530151fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46926/0 (socket says 172.21.15.27:46926) 2024-09-12T03:20:31.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.641+0000 7f65348ce640 1 -- 172.21.15.27:0/173831712 learned_addr learned my addr 172.21.15.27:0/173831712 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:20:31.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.641+0000 7f652ffff640 1 --2- 172.21.15.27:0/173831712 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f653006f360 0x7f65301f4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:31.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.641+0000 7f652f7fe640 1 --2- 172.21.15.27:0/173831712 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f65300b1ca0 0x7f65300cb850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:31.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.641+0000 7f65348ce640 1 -- 172.21.15.27:0/173831712 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f653006f360 msgr2=0x7f65301f4640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:31.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.641+0000 7f65348ce640 1 --2- 172.21.15.27:0/173831712 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f653006f360 0x7f65301f4640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:31.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.641+0000 7f65348ce640 1 -- 172.21.15.27:0/173831712 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f65300b1ca0 msgr2=0x7f65300cb850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:31.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.641+0000 7f65348ce640 1 --2- 172.21.15.27:0/173831712 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f65300b1ca0 0x7f65300cb850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:31.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.641+0000 7f65348ce640 1 -- 172.21.15.27:0/173831712 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65300ce9a0 con 0x7f65300cf660 2024-09-12T03:20:31.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.641+0000 7f652f7fe640 1 --2- 172.21.15.27:0/173831712 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f65300b1ca0 0x7f65300cb850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:20:31.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.641+0000 7f65348ce640 1 --2- 172.21.15.27:0/173831712 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f65300cf660 0x7f6530151fc0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f6528009fa0 tx=0x7f6528004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:31.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.641+0000 7f652cff9640 1 -- 172.21.15.27:0/173831712 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6528036040 con 0x7f65300cf660 2024-09-12T03:20:31.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.641+0000 7f652cff9640 1 -- 172.21.15.27:0/173831712 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6528003990 con 0x7f65300cf660 2024-09-12T03:20:31.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.641+0000 7f6536331640 1 -- 172.21.15.27:0/173831712 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65300b6110 con 0x7f65300cf660 2024-09-12T03:20:31.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.641+0000 7f652cff9640 1 -- 172.21.15.27:0/173831712 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6528009510 con 0x7f65300cf660 2024-09-12T03:20:31.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.641+0000 7f6536331640 1 -- 172.21.15.27:0/173831712 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65300b66d0 con 0x7f65300cf660 2024-09-12T03:20:31.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.642+0000 7f6536331640 1 -- 172.21.15.27:0/173831712 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65300cfc40 con 0x7f65300cf660 2024-09-12T03:20:31.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.643+0000 7f652cff9640 1 -- 172.21.15.27:0/173831712 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 90600+0+0 (secure 0 0 0) 0x7f6528007780 con 0x7f65300cf660 2024-09-12T03:20:31.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.643+0000 7f652cff9640 1 --2- 172.21.15.27:0/173831712 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f651c06bf50 0x7f651c06e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:31.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.643+0000 7f652cff9640 1 -- 172.21.15.27:0/173831712 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 5709+0+0 (secure 0 0 0) 0x7f65280ac370 con 0x7f65300cf660 2024-09-12T03:20:31.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.643+0000 7f652ffff640 1 --2- 172.21.15.27:0/173831712 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f651c06bf50 0x7f651c06e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:31.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.644+0000 7f652ffff640 1 --2- 172.21.15.27:0/173831712 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f651c06bf50 0x7f651c06e410 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f652000a990 tx=0x7f6520008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:31.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.648+0000 7f652cff9640 1 -- 172.21.15.27:0/173831712 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f653015fd70 con 0x7f65300cf660 2024-09-12T03:20:31.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.824+0000 7f6536331640 1 -- 172.21.15.27:0/173831712 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f653015e9e0 con 0x7f65300cf660 2024-09-12T03:20:31.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.825+0000 7f652cff9640 1 -- 172.21.15.27:0/173831712 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+16672 (secure 0 0 0) 0x7f652807ac20 con 0x7f65300cf660 2024-09-12T03:20:31.825 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:20:31.825 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":81,"fsid":"d2df504e-2647-4385-a5db-9fdf69f487a0","created":"2024-09-12T03:16:07.778937+0000","modified":"2024-09-12T03:20:30.167126+0000","last_up_change":"2024-09-12T03:16:19.314778+0000","last_in_change":"2024-09-12T03:16:09.893086+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T03:16:20.097646+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-09-12T03:20:26.167239+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-09-12T03:20:27.491764+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1851116c-1fb3-4dc9-9772-f90e2ef4c355","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6817","nonce":3002321488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6819","nonce":3002321488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6823","nonce":3002321488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6821","nonce":3002321488}]},"public_addr":"172.21.15.27:6817/3002321488","cluster_addr":"172.21.15.27:6819/3002321488","heartbeat_back_addr":"172.21.15.27:6823/3002321488","heartbeat_front_addr":"172.21.15.27:6821/3002321488","state":["exists","up"]},{"osd":1,"uuid":"f38ae17d-73eb-4204-b99d-975fa806fb87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6809","nonce":1613102886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6811","nonce":1613102886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6815","nonce":1613102886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6813","nonce":1613102886}]},"public_addr":"172.21.15.27:6809/1613102886","cluster_addr":"172.21.15.27:6811/1613102886","heartbeat_back_addr":"172.21.15.27:6815/1613102886","heartbeat_front_addr":"172.21.15.27:6813/1613102886","state":["exists","up"]},{"osd":2,"uuid":"fb12dcc8-bdd0-42b8-b46d-8b10106224bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6825","nonce":3534723355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6827","nonce":3534723355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6831","nonce":3534723355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6829","nonce":3534723355}]},"public_addr":"172.21.15.27:6825/3534723355","cluster_addr":"172.21.15.27:6827/3534723355","heartbeat_back_addr":"172.21.15.27:6831/3534723355","heartbeat_front_addr":"172.21.15.27:6829/3534723355","state":["exists","up"]},{"osd":3,"uuid":"d74d3f49-9c24-4b9f-98e2-3ecd7b5ef3b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6801","nonce":80661598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6803","nonce":80661598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6807","nonce":80661598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6805","nonce":80661598}]},"public_addr":"172.21.15.27:6801/80661598","cluster_addr":"172.21.15.27:6803/80661598","heartbeat_back_addr":"172.21.15.27:6807/80661598","heartbeat_front_addr":"172.21.15.27:6805/80661598","state":["exists","up"]},{"osd":4,"uuid":"2bf123f2-1b0d-41c4-aa3b-805bab267e59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6817","nonce":3943189827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6819","nonce":3943189827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6823","nonce":3943189827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6821","nonce":3943189827}]},"public_addr":"172.21.15.45:6817/3943189827","cluster_addr":"172.21.15.45:6819/3943189827","heartbeat_back_addr":"172.21.15.45:6823/3943189827","heartbeat_front_addr":"172.21.15.45:6821/3943189827","state":["exists","up"]},{"osd":5,"uuid":"c49ba5da-91c6-4a86-a97b-4aba89c253ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6809","nonce":3620760330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6811","nonce":3620760330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6815","nonce":3620760330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6813","nonce":3620760330}]},"public_addr":"172.21.15.45:6809/3620760330","cluster_addr":"172.21.15.45:6811/3620760330","heartbeat_back_addr":"172.21.15.45:6815/3620760330","heartbeat_front_addr":"172.21.15.45:6813/3620760330","state":["exists","up"]},{"osd":6,"uuid":"b0123475-89f7-4a34-9e33-d797b99dd9dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6801","nonce":1989083510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6803","nonce":1989083510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6807","nonce":1989083510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6805","nonce":1989083510}]},"public_addr":"172.21.15.45:6801/1989083510","cluster_addr":"172.21.15.45:6803/1989083510","heartbeat_back_addr":"172.21.15.45:6807/1989083510","heartbeat_front_addr":"172.21.15.45:6805/1989083510","state":["exists","up"]},{"osd":7,"uuid":"4dad773d-b7d8-46fc-8a10-a972aad27b29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6825","nonce":1513300294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6827","nonce":1513300294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6831","nonce":1513300294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6829","nonce":1513300294}]},"public_addr":"172.21.15.45:6825/1513300294","cluster_addr":"172.21.15.45:6827/1513300294","heartbeat_back_addr":"172.21.15.45:6831/1513300294","heartbeat_front_addr":"172.21.15.45:6829/1513300294","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.881855+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:17.133252+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.735339+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.500742+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.477571+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:17.167667+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.470291+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.873743+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T03:20:31.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.827+0000 7f6536331640 1 -- 172.21.15.27:0/173831712 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f651c06bf50 msgr2=0x7f651c06e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:31.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.827+0000 7f6536331640 1 --2- 172.21.15.27:0/173831712 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f651c06bf50 0x7f651c06e410 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f652000a990 tx=0x7f6520008040 comp rx=0 tx=0).stop 2024-09-12T03:20:31.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.827+0000 7f6536331640 1 -- 172.21.15.27:0/173831712 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f65300cf660 msgr2=0x7f6530151fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:31.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.827+0000 7f6536331640 1 --2- 172.21.15.27:0/173831712 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f65300cf660 0x7f6530151fc0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f6528009fa0 tx=0x7f6528004a40 comp rx=0 tx=0).stop 2024-09-12T03:20:31.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.827+0000 7f6536331640 1 -- 172.21.15.27:0/173831712 shutdown_connections 2024-09-12T03:20:31.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.827+0000 7f6536331640 1 --2- 172.21.15.27:0/173831712 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f651c06bf50 0x7f651c06e410 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:31.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.827+0000 7f6536331640 1 --2- 172.21.15.27:0/173831712 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f65300cf660 0x7f6530151fc0 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:31.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.827+0000 7f6536331640 1 --2- 172.21.15.27:0/173831712 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f65300b1ca0 0x7f65300cb850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:31.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.827+0000 7f6536331640 1 --2- 172.21.15.27:0/173831712 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f653006f360 0x7f65301f4640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:31.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.827+0000 7f6536331640 1 -- 172.21.15.27:0/173831712 >> 172.21.15.27:0/173831712 conn(0x7f6530073800 msgr2=0x7f65300cef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:31.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.827+0000 7f6536331640 1 -- 172.21.15.27:0/173831712 shutdown_connections 2024-09-12T03:20:31.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.827+0000 7f6536331640 1 -- 172.21.15.27:0/173831712 wait complete. 2024-09-12T03:20:31.838 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-12T03:20:31.838 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-12T03:20:31.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.940+0000 7fd644077640 1 Processor -- start 2024-09-12T03:20:31.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.941+0000 7fd644077640 1 -- start start 2024-09-12T03:20:31.947 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.941+0000 7fd644077640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fd63c0cf660 0x7fd63c0d2780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:31.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.941+0000 7fd644077640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd63c06cc40 con 0x7fd63c06f360 2024-09-12T03:20:31.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.942+0000 7fd644077640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd63c0d1ce0 con 0x7fd63c0b1ca0 2024-09-12T03:20:31.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.942+0000 7fd644077640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fd63c0d22f0 con 0x7fd63c0cf660 2024-09-12T03:20:31.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.942+0000 7fd641612640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fd63c06f360 0x7fd63c0cef50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:57684/0 (socket says 172.21.15.27:57684) 2024-09-12T03:20:31.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.942+0000 7fd641612640 1 -- 172.21.15.27:0/3590522882 learned_addr learned my addr 172.21.15.27:0/3590522882 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:20:31.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.942+0000 7fd642614640 1 --2- 172.21.15.27:0/3590522882 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fd63c0cf660 0x7fd63c0d2780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:31.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.942+0000 7fd640e11640 1 -- 172.21.15.27:0/3590522882 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1019669583 0 0) 0x7fd63c06cc40 con 0x7fd63c06f360 2024-09-12T03:20:31.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.942+0000 7fd640e11640 1 -- 172.21.15.27:0/3590522882 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd628003520 con 0x7fd63c06f360 2024-09-12T03:20:31.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.942+0000 7fd640e11640 1 -- 172.21.15.27:0/3590522882 <== mon.1 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 185591064 0 0) 0x7fd63c0d1ce0 con 0x7fd63c0b1ca0 2024-09-12T03:20:31.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.942+0000 7fd640e11640 1 -- 172.21.15.27:0/3590522882 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd63c06cc40 con 0x7fd63c0b1ca0 2024-09-12T03:20:31.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.942+0000 7fd640e11640 1 -- 172.21.15.27:0/3590522882 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3242830026 0 0) 0x7fd628003520 con 0x7fd63c06f360 2024-09-12T03:20:31.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.942+0000 7fd640e11640 1 -- 172.21.15.27:0/3590522882 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fd63c0cf660 msgr2=0x7fd63c0d2780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:31.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.942+0000 7fd640e11640 1 --2- 172.21.15.27:0/3590522882 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fd63c0cf660 0x7fd63c0d2780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:31.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.942+0000 7fd640e11640 1 -- 172.21.15.27:0/3590522882 >> v1:172.21.15.45:6789/0 conn(0x7fd63c0b1ca0 legacy=0x7fd63c15f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:31.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.942+0000 7fd640e11640 1 -- 172.21.15.27:0/3590522882 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd63c06cee0 con 0x7fd63c06f360 2024-09-12T03:20:31.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.942+0000 7fd640e11640 1 -- 172.21.15.27:0/3590522882 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 847755213 0 0) 0x7fd62c001670 con 0x7fd63c06f360 2024-09-12T03:20:31.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.942+0000 7fd640e11640 1 -- 172.21.15.27:0/3590522882 >> v1:172.21.15.27:6789/0 conn(0x7fd63c06f360 legacy=0x7fd63c0cef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:31.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.942+0000 7fd640e11640 1 --2- 172.21.15.27:0/3590522882 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd628003b70 0x7fd628003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:31.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.942+0000 7fd640e11640 1 --2- 172.21.15.27:0/3590522882 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fd63c0cf660 0x7fd628004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:31.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.942+0000 7fd640e11640 1 --2- 172.21.15.27:0/3590522882 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd628004d40 0x7fd6280051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:31.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.942+0000 7fd640e11640 1 -- 172.21.15.27:0/3590522882 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fd63c0d22f0 con 0x7fd628004d40 2024-09-12T03:20:31.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.942+0000 7fd640e11640 1 -- 172.21.15.27:0/3590522882 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fd62800ba70 con 0x7fd628003b70 2024-09-12T03:20:31.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.942+0000 7fd640e11640 1 -- 172.21.15.27:0/3590522882 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fd62800bc70 con 0x7fd63c0cf660 2024-09-12T03:20:31.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.942+0000 7fd642614640 1 --2- 172.21.15.27:0/3590522882 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd628004d40 0x7fd6280051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:31.950 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.942+0000 7fd641e13640 1 --2- 172.21.15.27:0/3590522882 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd628003b70 0x7fd628003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:31.950 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.942+0000 7fd641612640 1 --2- 172.21.15.27:0/3590522882 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fd63c0cf660 0x7fd628004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:31.950 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.943+0000 7fd642614640 1 -- 172.21.15.27:0/3590522882 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fd63c0cf660 msgr2=0x7fd628004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:31.950 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.943+0000 7fd642614640 1 --2- 172.21.15.27:0/3590522882 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fd63c0cf660 0x7fd628004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:31.950 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.943+0000 7fd642614640 1 -- 172.21.15.27:0/3590522882 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd628003b70 msgr2=0x7fd628003f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:31.950 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.943+0000 7fd642614640 1 --2- 172.21.15.27:0/3590522882 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd628003b70 0x7fd628003f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:31.950 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.943+0000 7fd642614640 1 -- 172.21.15.27:0/3590522882 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd628003520 con 0x7fd628004d40 2024-09-12T03:20:31.950 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.943+0000 7fd642614640 1 --2- 172.21.15.27:0/3590522882 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd628004d40 0x7fd6280051c0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fd634009f90 tx=0x7fd628005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=369e4f004c30d2f8 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:31.950 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.944+0000 7fd640e11640 1 -- 172.21.15.27:0/3590522882 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd634009c10 con 0x7fd628004d40 2024-09-12T03:20:31.950 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.944+0000 7fd640e11640 1 -- 172.21.15.27:0/3590522882 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd634036070 con 0x7fd628004d40 2024-09-12T03:20:31.950 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.944+0000 7fd640e11640 1 -- 172.21.15.27:0/3590522882 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd634031850 con 0x7fd628004d40 2024-09-12T03:20:31.950 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.944+0000 7fd644077640 1 -- 172.21.15.27:0/3590522882 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd628004d40 msgr2=0x7fd6280051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:31.950 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.944+0000 7fd644077640 1 --2- 172.21.15.27:0/3590522882 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd628004d40 0x7fd6280051c0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fd634009f90 tx=0x7fd628005830 comp rx=0 tx=0).stop 2024-09-12T03:20:31.951 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.944+0000 7fd642614640 1 -- 172.21.15.27:0/3590522882 reap_dead start 2024-09-12T03:20:31.951 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.944+0000 7fd644077640 1 -- 172.21.15.27:0/3590522882 shutdown_connections 2024-09-12T03:20:31.951 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.944+0000 7fd644077640 1 -- 172.21.15.27:0/3590522882 >> 172.21.15.27:0/3590522882 conn(0x7fd63c073800 msgr2=0x7fd63c09ba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:31.951 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.944+0000 7fd644077640 1 -- 172.21.15.27:0/3590522882 shutdown_connections 2024-09-12T03:20:31.951 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.944+0000 7fd644077640 1 -- 172.21.15.27:0/3590522882 wait complete. 2024-09-12T03:20:31.951 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.945+0000 7fd644077640 1 Processor -- start 2024-09-12T03:20:31.951 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.945+0000 7fd644077640 1 -- start start 2024-09-12T03:20:31.951 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.946+0000 7fd644077640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd628004d40 0x7fd63c1612f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:31.951 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.946+0000 7fd644077640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fd63c15f250 0x7fd63c1541c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:31.951 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.946+0000 7fd644077640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd63c0704c0 0x7fd63c070940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:31.951 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.946+0000 7fd644077640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fd63c06d340 con 0x7fd63c0704c0 2024-09-12T03:20:31.951 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.946+0000 7fd644077640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fd63c06d1c0 con 0x7fd628004d40 2024-09-12T03:20:31.951 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.946+0000 7fd644077640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fd63c0cfc40 con 0x7fd63c15f250 2024-09-12T03:20:31.951 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.946+0000 7fd642614640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd63c0704c0 0x7fd63c070940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:31.952 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.946+0000 7fd642614640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd63c0704c0 0x7fd63c070940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46946/0 (socket says 172.21.15.27:46946) 2024-09-12T03:20:31.952 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.946+0000 7fd642614640 1 -- 172.21.15.27:0/1697780659 learned_addr learned my addr 172.21.15.27:0/1697780659 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:20:31.952 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.946+0000 7fd641e13640 1 --2- 172.21.15.27:0/1697780659 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd628004d40 0x7fd63c1612f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:31.952 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.946+0000 7fd641612640 1 --2- 172.21.15.27:0/1697780659 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fd63c15f250 0x7fd63c1541c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:31.952 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.947+0000 7fd642614640 1 -- 172.21.15.27:0/1697780659 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fd63c15f250 msgr2=0x7fd63c1541c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:31.952 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.947+0000 7fd642614640 1 --2- 172.21.15.27:0/1697780659 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fd63c15f250 0x7fd63c1541c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:31.952 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.947+0000 7fd642614640 1 -- 172.21.15.27:0/1697780659 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd628004d40 msgr2=0x7fd63c1612f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:31.952 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.947+0000 7fd642614640 1 --2- 172.21.15.27:0/1697780659 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd628004d40 0x7fd63c1612f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:31.952 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.947+0000 7fd642614640 1 -- 172.21.15.27:0/1697780659 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd63c0b3a00 con 0x7fd63c0704c0 2024-09-12T03:20:31.952 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.947+0000 7fd641612640 1 --2- 172.21.15.27:0/1697780659 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fd63c15f250 0x7fd63c1541c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:20:31.952 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.947+0000 7fd642614640 1 --2- 172.21.15.27:0/1697780659 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd63c0704c0 0x7fd63c070940 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fd63400e230 tx=0x7fd634009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:31.952 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.947+0000 7fd6327fc640 1 -- 172.21.15.27:0/1697780659 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd634009c10 con 0x7fd63c0704c0 2024-09-12T03:20:31.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.947+0000 7fd6327fc640 1 -- 172.21.15.27:0/1697780659 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd634040030 con 0x7fd63c0704c0 2024-09-12T03:20:31.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.947+0000 7fd644077640 1 -- 172.21.15.27:0/1697780659 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd63c0b7500 con 0x7fd63c0704c0 2024-09-12T03:20:31.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.947+0000 7fd6327fc640 1 -- 172.21.15.27:0/1697780659 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd634003dd0 con 0x7fd63c0704c0 2024-09-12T03:20:31.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.947+0000 7fd644077640 1 -- 172.21.15.27:0/1697780659 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd63c0b3c80 con 0x7fd63c0704c0 2024-09-12T03:20:31.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.948+0000 7fd644077640 1 -- 172.21.15.27:0/1697780659 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd63c06f7d0 con 0x7fd63c0704c0 2024-09-12T03:20:31.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.948+0000 7fd6327fc640 1 -- 172.21.15.27:0/1697780659 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 90600+0+0 (secure 0 0 0) 0x7fd634005d10 con 0x7fd63c0704c0 2024-09-12T03:20:31.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.949+0000 7fd6327fc640 1 --2- 172.21.15.27:0/1697780659 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fd62006bf80 0x7fd62006e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:31.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.949+0000 7fd6327fc640 1 -- 172.21.15.27:0/1697780659 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 5709+0+0 (secure 0 0 0) 0x7fd6340ac2a0 con 0x7fd63c0704c0 2024-09-12T03:20:31.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.949+0000 7fd641e13640 1 --2- 172.21.15.27:0/1697780659 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fd62006bf80 0x7fd62006e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:31.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.949+0000 7fd641e13640 1 --2- 172.21.15.27:0/1697780659 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fd62006bf80 0x7fd62006e440 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fd63c0d1eb0 tx=0x7fd638009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:31.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:31.953+0000 7fd6327fc640 1 -- 172.21.15.27:0/1697780659 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fd63404a020 con 0x7fd63c0704c0 2024-09-12T03:20:32.127 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:32.127+0000 7fd644077640 1 -- 172.21.15.27:0/1697780659 --> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd63c0b1400 con 0x7fd62006bf80 2024-09-12T03:20:32.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:32.134+0000 7fd6327fc640 1 -- 172.21.15.27:0/1697780659 <== mgr.4113 v2:172.21.15.27:6836/1226920526 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252049 (secure 0 0 0) 0x7fd63c0b1400 con 0x7fd62006bf80 2024-09-12T03:20:32.136 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:20:32.138 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2024-09-12T03:20:32.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:32.137+0000 7fd644077640 1 -- 172.21.15.27:0/1697780659 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fd62006bf80 msgr2=0x7fd62006e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:32.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:32.137+0000 7fd644077640 1 --2- 172.21.15.27:0/1697780659 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fd62006bf80 0x7fd62006e440 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fd63c0d1eb0 tx=0x7fd638009040 comp rx=0 tx=0).stop 2024-09-12T03:20:32.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:32.137+0000 7fd644077640 1 -- 172.21.15.27:0/1697780659 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd63c0704c0 msgr2=0x7fd63c070940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:32.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:32.137+0000 7fd644077640 1 --2- 172.21.15.27:0/1697780659 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd63c0704c0 0x7fd63c070940 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fd63400e230 tx=0x7fd634009750 comp rx=0 tx=0).stop 2024-09-12T03:20:32.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:32.137+0000 7fd644077640 1 -- 172.21.15.27:0/1697780659 shutdown_connections 2024-09-12T03:20:32.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:32.137+0000 7fd644077640 1 --2- 172.21.15.27:0/1697780659 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fd62006bf80 0x7fd62006e440 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:32.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:32.137+0000 7fd644077640 1 --2- 172.21.15.27:0/1697780659 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd63c0704c0 0x7fd63c070940 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:32.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:32.137+0000 7fd644077640 1 --2- 172.21.15.27:0/1697780659 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fd63c15f250 0x7fd63c1541c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:32.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:32.137+0000 7fd644077640 1 --2- 172.21.15.27:0/1697780659 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd628004d40 0x7fd63c1612f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:32.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:32.137+0000 7fd644077640 1 -- 172.21.15.27:0/1697780659 >> 172.21.15.27:0/1697780659 conn(0x7fd63c073800 msgr2=0x7fd63c15d3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:32.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:32.137+0000 7fd644077640 1 -- 172.21.15.27:0/1697780659 shutdown_connections 2024-09-12T03:20:32.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:32.137+0000 7fd644077640 1 -- 172.21.15.27:0/1697780659 wait complete. 2024-09-12T03:20:32.148 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2024-09-12T03:20:31.097175+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":223,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":166,"ondisk_log_size":166,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":126,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":266760,"kb_used_data":8908,"kb_used_omap":12,"kb_used_meta":257651,"kb_avail":754707960,"statfs":{"total":773094113280,"available":772820951040,"internally_reserved":0,"allocated":9121792,"data_stored":4560856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":263835184},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000982"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-12T03:20:30.786784+0000","last_change":"2024-09-12T03:20:29.177288+0000","last_active":"2024-09-12T03:20:30.786784+0000","last_peered":"2024-09-12T03:20:30.786784+0000","last_clean":"2024-09-12T03:20:30.786784+0000","last_became_active":"2024-09-12T03:20:29.176231+0000","last_became_peered":"2024-09-12T03:20:29.176231+0000","last_unstale":"2024-09-12T03:20:30.786784+0000","last_undegraded":"2024-09-12T03:20:30.786784+0000","last_fullsized":"2024-09-12T03:20:30.786784+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_clean_scrub_stamp":"2024-09-12T03:20:28.136653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:02:27.543548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-12T03:20:30.174970+0000","last_change":"2024-09-12T03:20:29.177451+0000","last_active":"2024-09-12T03:20:30.174970+0000","last_peered":"2024-09-12T03:20:30.174970+0000","last_clean":"2024-09-12T03:20:30.174970+0000","last_became_active":"2024-09-12T03:20:29.176831+0000","last_became_peered":"2024-09-12T03:20:29.176831+0000","last_unstale":"2024-09-12T03:20:30.174970+0000","last_undegraded":"2024-09-12T03:20:30.174970+0000","last_fullsized":"2024-09-12T03:20:30.174970+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_clean_scrub_stamp":"2024-09-12T03:20:28.136653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:51:51.184680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:20:29.178319+0000","last_change":"2024-09-12T03:20:29.176354+0000","last_active":"2024-09-12T03:20:29.178319+0000","last_peered":"2024-09-12T03:20:29.178319+0000","last_clean":"2024-09-12T03:20:29.178319+0000","last_became_active":"2024-09-12T03:20:29.176060+0000","last_became_peered":"2024-09-12T03:20:29.176060+0000","last_unstale":"2024-09-12T03:20:29.178319+0000","last_undegraded":"2024-09-12T03:20:29.178319+0000","last_fullsized":"2024-09-12T03:20:29.178319+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_clean_scrub_stamp":"2024-09-12T03:20:28.136653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:20:39.549097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:20:29.192407+0000","last_change":"2024-09-12T03:20:29.187883+0000","last_active":"2024-09-12T03:20:29.192407+0000","last_peered":"2024-09-12T03:20:29.192407+0000","last_clean":"2024-09-12T03:20:29.192407+0000","last_became_active":"2024-09-12T03:20:29.187594+0000","last_became_peered":"2024-09-12T03:20:29.187594+0000","last_unstale":"2024-09-12T03:20:29.192407+0000","last_undegraded":"2024-09-12T03:20:29.192407+0000","last_fullsized":"2024-09-12T03:20:29.192407+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_clean_scrub_stamp":"2024-09-12T03:20:28.136653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:26:26.772575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-12T03:20:30.176147+0000","last_change":"2024-09-12T03:20:29.186478+0000","last_active":"2024-09-12T03:20:30.176147+0000","last_peered":"2024-09-12T03:20:30.176147+0000","last_clean":"2024-09-12T03:20:30.176147+0000","last_became_active":"2024-09-12T03:20:29.186196+0000","last_became_peered":"2024-09-12T03:20:29.186196+0000","last_unstale":"2024-09-12T03:20:30.176147+0000","last_undegraded":"2024-09-12T03:20:30.176147+0000","last_fullsized":"2024-09-12T03:20:30.176147+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_clean_scrub_stamp":"2024-09-12T03:20:28.136653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:17:42.735036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-12T03:20:30.175097+0000","last_change":"2024-09-12T03:20:29.178733+0000","last_active":"2024-09-12T03:20:30.175097+0000","last_peered":"2024-09-12T03:20:30.175097+0000","last_clean":"2024-09-12T03:20:30.175097+0000","last_became_active":"2024-09-12T03:20:29.178419+0000","last_became_peered":"2024-09-12T03:20:29.178419+0000","last_unstale":"2024-09-12T03:20:30.175097+0000","last_undegraded":"2024-09-12T03:20:30.175097+0000","last_fullsized":"2024-09-12T03:20:30.175097+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_clean_scrub_stamp":"2024-09-12T03:20:28.136653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:08:21.425269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:20:29.182198+0000","last_change":"2024-09-12T03:20:29.178336+0000","last_active":"2024-09-12T03:20:29.182198+0000","last_peered":"2024-09-12T03:20:29.182198+0000","last_clean":"2024-09-12T03:20:29.182198+0000","last_became_active":"2024-09-12T03:20:29.177344+0000","last_became_peered":"2024-09-12T03:20:29.177344+0000","last_unstale":"2024-09-12T03:20:29.182198+0000","last_undegraded":"2024-09-12T03:20:29.182198+0000","last_fullsized":"2024-09-12T03:20:29.182198+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_clean_scrub_stamp":"2024-09-12T03:20:28.136653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:39:33.196052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-12T03:20:30.175436+0000","last_change":"2024-09-12T03:20:29.179643+0000","last_active":"2024-09-12T03:20:30.175436+0000","last_peered":"2024-09-12T03:20:30.175436+0000","last_clean":"2024-09-12T03:20:30.175436+0000","last_became_active":"2024-09-12T03:20:29.179313+0000","last_became_peered":"2024-09-12T03:20:29.179313+0000","last_unstale":"2024-09-12T03:20:30.175436+0000","last_undegraded":"2024-09-12T03:20:30.175436+0000","last_fullsized":"2024-09-12T03:20:30.175436+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_clean_scrub_stamp":"2024-09-12T03:20:28.136653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:14:47.509935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-12T03:20:30.175154+0000","last_change":"2024-09-12T03:20:29.183146+0000","last_active":"2024-09-12T03:20:30.175154+0000","last_peered":"2024-09-12T03:20:30.175154+0000","last_clean":"2024-09-12T03:20:30.175154+0000","last_became_active":"2024-09-12T03:20:29.182856+0000","last_became_peered":"2024-09-12T03:20:29.182856+0000","last_unstale":"2024-09-12T03:20:30.175154+0000","last_undegraded":"2024-09-12T03:20:30.175154+0000","last_fullsized":"2024-09-12T03:20:30.175154+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_clean_scrub_stamp":"2024-09-12T03:20:28.136653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:51:36.544538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-12T03:20:30.786601+0000","last_change":"2024-09-12T03:20:29.178507+0000","last_active":"2024-09-12T03:20:30.786601+0000","last_peered":"2024-09-12T03:20:30.786601+0000","last_clean":"2024-09-12T03:20:30.786601+0000","last_became_active":"2024-09-12T03:20:29.178202+0000","last_became_peered":"2024-09-12T03:20:29.178202+0000","last_unstale":"2024-09-12T03:20:30.786601+0000","last_undegraded":"2024-09-12T03:20:30.786601+0000","last_fullsized":"2024-09-12T03:20:30.786601+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_clean_scrub_stamp":"2024-09-12T03:20:28.136653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:17:42.735036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-12T03:20:30.174801+0000","last_change":"2024-09-12T03:20:29.180016+0000","last_active":"2024-09-12T03:20:30.174801+0000","last_peered":"2024-09-12T03:20:30.174801+0000","last_clean":"2024-09-12T03:20:30.174801+0000","last_became_active":"2024-09-12T03:20:29.179428+0000","last_became_peered":"2024-09-12T03:20:29.179428+0000","last_unstale":"2024-09-12T03:20:30.174801+0000","last_undegraded":"2024-09-12T03:20:30.174801+0000","last_fullsized":"2024-09-12T03:20:30.174801+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_clean_scrub_stamp":"2024-09-12T03:20:28.136653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:02:05.748037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-12T03:20:30.786307+0000","last_change":"2024-09-12T03:20:29.174451+0000","last_active":"2024-09-12T03:20:30.786307+0000","last_peered":"2024-09-12T03:20:30.786307+0000","last_clean":"2024-09-12T03:20:30.786307+0000","last_became_active":"2024-09-12T03:20:29.173679+0000","last_became_peered":"2024-09-12T03:20:29.173679+0000","last_unstale":"2024-09-12T03:20:30.786307+0000","last_undegraded":"2024-09-12T03:20:30.786307+0000","last_fullsized":"2024-09-12T03:20:30.786307+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_clean_scrub_stamp":"2024-09-12T03:20:28.136653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:10:24.781478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:20:29.177573+0000","last_change":"2024-09-12T03:20:29.174921+0000","last_active":"2024-09-12T03:20:29.177573+0000","last_peered":"2024-09-12T03:20:29.177573+0000","last_clean":"2024-09-12T03:20:29.177573+0000","last_became_active":"2024-09-12T03:20:29.174603+0000","last_became_peered":"2024-09-12T03:20:29.174603+0000","last_unstale":"2024-09-12T03:20:29.177573+0000","last_undegraded":"2024-09-12T03:20:29.177573+0000","last_fullsized":"2024-09-12T03:20:29.177573+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_clean_scrub_stamp":"2024-09-12T03:20:28.136653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:10:28.631427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:20:29.192313+0000","last_change":"2024-09-12T03:20:29.187072+0000","last_active":"2024-09-12T03:20:29.192313+0000","last_peered":"2024-09-12T03:20:29.192313+0000","last_clean":"2024-09-12T03:20:29.192313+0000","last_became_active":"2024-09-12T03:20:29.186803+0000","last_became_peered":"2024-09-12T03:20:29.186803+0000","last_unstale":"2024-09-12T03:20:29.192313+0000","last_undegraded":"2024-09-12T03:20:29.192313+0000","last_fullsized":"2024-09-12T03:20:29.192313+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_clean_scrub_stamp":"2024-09-12T03:20:28.136653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:38:45.533952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-12T03:20:30.809930+0000","last_change":"2024-09-12T03:20:29.179532+0000","last_active":"2024-09-12T03:20:30.809930+0000","last_peered":"2024-09-12T03:20:30.809930+0000","last_clean":"2024-09-12T03:20:30.809930+0000","last_became_active":"2024-09-12T03:20:29.179262+0000","last_became_peered":"2024-09-12T03:20:29.179262+0000","last_unstale":"2024-09-12T03:20:30.809930+0000","last_undegraded":"2024-09-12T03:20:30.809930+0000","last_fullsized":"2024-09-12T03:20:30.809930+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_clean_scrub_stamp":"2024-09-12T03:20:28.136653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:50:43.423983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-12T03:20:30.175204+0000","last_change":"2024-09-12T03:20:29.170807+0000","last_active":"2024-09-12T03:20:30.175204+0000","last_peered":"2024-09-12T03:20:30.175204+0000","last_clean":"2024-09-12T03:20:30.175204+0000","last_became_active":"2024-09-12T03:20:29.170547+0000","last_became_peered":"2024-09-12T03:20:29.170547+0000","last_unstale":"2024-09-12T03:20:30.175204+0000","last_undegraded":"2024-09-12T03:20:30.175204+0000","last_fullsized":"2024-09-12T03:20:30.175204+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_clean_scrub_stamp":"2024-09-12T03:20:28.136653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:40:09.344605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-12T03:20:30.176131+0000","last_change":"2024-09-12T03:20:29.178480+0000","last_active":"2024-09-12T03:20:30.176131+0000","last_peered":"2024-09-12T03:20:30.176131+0000","last_clean":"2024-09-12T03:20:30.176131+0000","last_became_active":"2024-09-12T03:20:29.178229+0000","last_became_peered":"2024-09-12T03:20:29.178229+0000","last_unstale":"2024-09-12T03:20:30.176131+0000","last_undegraded":"2024-09-12T03:20:30.176131+0000","last_fullsized":"2024-09-12T03:20:30.176131+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_clean_scrub_stamp":"2024-09-12T03:20:28.136653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:10:28.631427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:20:29.182872+0000","last_change":"2024-09-12T03:20:29.178792+0000","last_active":"2024-09-12T03:20:29.182872+0000","last_peered":"2024-09-12T03:20:29.182872+0000","last_clean":"2024-09-12T03:20:29.182872+0000","last_became_active":"2024-09-12T03:20:29.178508+0000","last_became_peered":"2024-09-12T03:20:29.178508+0000","last_unstale":"2024-09-12T03:20:29.182872+0000","last_undegraded":"2024-09-12T03:20:29.182872+0000","last_fullsized":"2024-09-12T03:20:29.182872+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_clean_scrub_stamp":"2024-09-12T03:20:28.136653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:57:54.991629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:20:29.186520+0000","last_change":"2024-09-12T03:20:29.182384+0000","last_active":"2024-09-12T03:20:29.186520+0000","last_peered":"2024-09-12T03:20:29.186520+0000","last_clean":"2024-09-12T03:20:29.186520+0000","last_became_active":"2024-09-12T03:20:29.181445+0000","last_became_peered":"2024-09-12T03:20:29.181445+0000","last_unstale":"2024-09-12T03:20:29.186520+0000","last_undegraded":"2024-09-12T03:20:29.186520+0000","last_fullsized":"2024-09-12T03:20:29.186520+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_clean_scrub_stamp":"2024-09-12T03:20:28.136653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:06:02.733007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:20:29.190699+0000","last_change":"2024-09-12T03:20:29.183131+0000","last_active":"2024-09-12T03:20:29.190699+0000","last_peered":"2024-09-12T03:20:29.190699+0000","last_clean":"2024-09-12T03:20:29.190699+0000","last_became_active":"2024-09-12T03:20:29.182837+0000","last_became_peered":"2024-09-12T03:20:29.182837+0000","last_unstale":"2024-09-12T03:20:29.190699+0000","last_undegraded":"2024-09-12T03:20:29.190699+0000","last_fullsized":"2024-09-12T03:20:29.190699+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_clean_scrub_stamp":"2024-09-12T03:20:28.136653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:45:07.868238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:20:29.192441+0000","last_change":"2024-09-12T03:20:29.187883+0000","last_active":"2024-09-12T03:20:29.192441+0000","last_peered":"2024-09-12T03:20:29.192441+0000","last_clean":"2024-09-12T03:20:29.192441+0000","last_became_active":"2024-09-12T03:20:29.187616+0000","last_became_peered":"2024-09-12T03:20:29.187616+0000","last_unstale":"2024-09-12T03:20:29.192441+0000","last_undegraded":"2024-09-12T03:20:29.192441+0000","last_fullsized":"2024-09-12T03:20:29.192441+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_clean_scrub_stamp":"2024-09-12T03:20:28.136653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:12:09.993191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-12T03:20:30.174377+0000","last_change":"2024-09-12T03:20:29.189822+0000","last_active":"2024-09-12T03:20:30.174377+0000","last_peered":"2024-09-12T03:20:30.174377+0000","last_clean":"2024-09-12T03:20:30.174377+0000","last_became_active":"2024-09-12T03:20:29.189582+0000","last_became_peered":"2024-09-12T03:20:29.189582+0000","last_unstale":"2024-09-12T03:20:30.174377+0000","last_undegraded":"2024-09-12T03:20:30.174377+0000","last_fullsized":"2024-09-12T03:20:30.174377+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_clean_scrub_stamp":"2024-09-12T03:20:28.136653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:41:35.802098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-12T03:20:30.786223+0000","last_change":"2024-09-12T03:20:29.188227+0000","last_active":"2024-09-12T03:20:30.786223+0000","last_peered":"2024-09-12T03:20:30.786223+0000","last_clean":"2024-09-12T03:20:30.786223+0000","last_became_active":"2024-09-12T03:20:29.187910+0000","last_became_peered":"2024-09-12T03:20:29.187910+0000","last_unstale":"2024-09-12T03:20:30.786223+0000","last_undegraded":"2024-09-12T03:20:30.786223+0000","last_fullsized":"2024-09-12T03:20:30.786223+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_clean_scrub_stamp":"2024-09-12T03:20:28.136653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:19:22.773136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:20:29.192450+0000","last_change":"2024-09-12T03:20:29.188417+0000","last_active":"2024-09-12T03:20:29.192450+0000","last_peered":"2024-09-12T03:20:29.192450+0000","last_clean":"2024-09-12T03:20:29.192450+0000","last_became_active":"2024-09-12T03:20:29.188157+0000","last_became_peered":"2024-09-12T03:20:29.188157+0000","last_unstale":"2024-09-12T03:20:29.192450+0000","last_undegraded":"2024-09-12T03:20:29.192450+0000","last_fullsized":"2024-09-12T03:20:29.192450+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_clean_scrub_stamp":"2024-09-12T03:20:28.136653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:04:05.851766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-12T03:20:30.174471+0000","last_change":"2024-09-12T03:20:29.176464+0000","last_active":"2024-09-12T03:20:30.174471+0000","last_peered":"2024-09-12T03:20:30.174471+0000","last_clean":"2024-09-12T03:20:30.174471+0000","last_became_active":"2024-09-12T03:20:29.176182+0000","last_became_peered":"2024-09-12T03:20:29.176182+0000","last_unstale":"2024-09-12T03:20:30.174471+0000","last_undegraded":"2024-09-12T03:20:30.174471+0000","last_fullsized":"2024-09-12T03:20:30.174471+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_clean_scrub_stamp":"2024-09-12T03:20:28.136653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:31:35.182563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:20:29.188272+0000","last_change":"2024-09-12T03:20:29.187228+0000","last_active":"2024-09-12T03:20:29.188272+0000","last_peered":"2024-09-12T03:20:29.188272+0000","last_clean":"2024-09-12T03:20:29.188272+0000","last_became_active":"2024-09-12T03:20:29.186784+0000","last_became_peered":"2024-09-12T03:20:29.186784+0000","last_unstale":"2024-09-12T03:20:29.188272+0000","last_undegraded":"2024-09-12T03:20:29.188272+0000","last_fullsized":"2024-09-12T03:20:29.188272+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_clean_scrub_stamp":"2024-09-12T03:20:28.136653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:41:29.086502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-12T03:20:30.175994+0000","last_change":"2024-09-12T03:20:29.173152+0000","last_active":"2024-09-12T03:20:30.175994+0000","last_peered":"2024-09-12T03:20:30.175994+0000","last_clean":"2024-09-12T03:20:30.175994+0000","last_became_active":"2024-09-12T03:20:29.172710+0000","last_became_peered":"2024-09-12T03:20:29.172710+0000","last_unstale":"2024-09-12T03:20:30.175994+0000","last_undegraded":"2024-09-12T03:20:30.175994+0000","last_fullsized":"2024-09-12T03:20:30.175994+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_clean_scrub_stamp":"2024-09-12T03:20:28.136653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:10:24.781478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-12T03:20:30.786410+0000","last_change":"2024-09-12T03:20:29.175081+0000","last_active":"2024-09-12T03:20:30.786410+0000","last_peered":"2024-09-12T03:20:30.786410+0000","last_clean":"2024-09-12T03:20:30.786410+0000","last_became_active":"2024-09-12T03:20:29.174776+0000","last_became_peered":"2024-09-12T03:20:29.174776+0000","last_unstale":"2024-09-12T03:20:30.786410+0000","last_undegraded":"2024-09-12T03:20:30.786410+0000","last_fullsized":"2024-09-12T03:20:30.786410+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_clean_scrub_stamp":"2024-09-12T03:20:28.136653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:10:28.631427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-12T03:20:30.174640+0000","last_change":"2024-09-12T03:20:29.168919+0000","last_active":"2024-09-12T03:20:30.174640+0000","last_peered":"2024-09-12T03:20:30.174640+0000","last_clean":"2024-09-12T03:20:30.174640+0000","last_became_active":"2024-09-12T03:20:29.168636+0000","last_became_peered":"2024-09-12T03:20:29.168636+0000","last_unstale":"2024-09-12T03:20:30.174640+0000","last_undegraded":"2024-09-12T03:20:30.174640+0000","last_fullsized":"2024-09-12T03:20:30.174640+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_clean_scrub_stamp":"2024-09-12T03:20:28.136653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:18:08.957359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-12T03:20:30.174402+0000","last_change":"2024-09-12T03:20:29.188821+0000","last_active":"2024-09-12T03:20:30.174402+0000","last_peered":"2024-09-12T03:20:30.174402+0000","last_clean":"2024-09-12T03:20:30.174402+0000","last_became_active":"2024-09-12T03:20:29.185747+0000","last_became_peered":"2024-09-12T03:20:29.185747+0000","last_unstale":"2024-09-12T03:20:30.174402+0000","last_undegraded":"2024-09-12T03:20:30.174402+0000","last_fullsized":"2024-09-12T03:20:30.174402+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_clean_scrub_stamp":"2024-09-12T03:20:28.136653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:19:34.760161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-12T03:20:30.175300+0000","last_change":"2024-09-12T03:20:29.185908+0000","last_active":"2024-09-12T03:20:30.175300+0000","last_peered":"2024-09-12T03:20:30.175300+0000","last_clean":"2024-09-12T03:20:30.175300+0000","last_became_active":"2024-09-12T03:20:29.185286+0000","last_became_peered":"2024-09-12T03:20:29.185286+0000","last_unstale":"2024-09-12T03:20:30.175300+0000","last_undegraded":"2024-09-12T03:20:30.175300+0000","last_fullsized":"2024-09-12T03:20:30.175300+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_clean_scrub_stamp":"2024-09-12T03:20:28.136653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:19:22.773136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-12T03:20:30.786114+0000","last_change":"2024-09-12T03:20:29.190215+0000","last_active":"2024-09-12T03:20:30.786114+0000","last_peered":"2024-09-12T03:20:30.786114+0000","last_clean":"2024-09-12T03:20:30.786114+0000","last_became_active":"2024-09-12T03:20:29.189872+0000","last_became_peered":"2024-09-12T03:20:29.189872+0000","last_unstale":"2024-09-12T03:20:30.786114+0000","last_undegraded":"2024-09-12T03:20:30.786114+0000","last_fullsized":"2024-09-12T03:20:30.786114+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_clean_scrub_stamp":"2024-09-12T03:20:28.136653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:40:09.344605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-12T03:20:30.176017+0000","last_change":"2024-09-12T03:20:29.170126+0000","last_active":"2024-09-12T03:20:30.176017+0000","last_peered":"2024-09-12T03:20:30.176017+0000","last_clean":"2024-09-12T03:20:30.176017+0000","last_became_active":"2024-09-12T03:20:29.169854+0000","last_became_peered":"2024-09-12T03:20:29.169854+0000","last_unstale":"2024-09-12T03:20:30.176017+0000","last_undegraded":"2024-09-12T03:20:30.176017+0000","last_fullsized":"2024-09-12T03:20:30.176017+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_clean_scrub_stamp":"2024-09-12T03:20:28.136653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:21:39.137804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"80'4","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-12T03:20:30.786872+0000","last_change":"2024-09-12T03:20:28.161710+0000","last_active":"2024-09-12T03:20:30.786872+0000","last_peered":"2024-09-12T03:20:30.786872+0000","last_clean":"2024-09-12T03:20:30.786872+0000","last_became_active":"2024-09-12T03:20:28.161404+0000","last_became_peered":"2024-09-12T03:20:28.161404+0000","last_unstale":"2024-09-12T03:20:30.786872+0000","last_undegraded":"2024-09-12T03:20:30.786872+0000","last_fullsized":"2024-09-12T03:20:30.786872+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:27.129547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:27.129547+0000","last_clean_scrub_stamp":"2024-09-12T03:20:27.129547+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:51:10.529017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-12T03:20:30.786780+0000","last_change":"2024-09-12T03:20:29.187176+0000","last_active":"2024-09-12T03:20:30.786780+0000","last_peered":"2024-09-12T03:20:30.786780+0000","last_clean":"2024-09-12T03:20:30.786780+0000","last_became_active":"2024-09-12T03:20:29.186884+0000","last_became_peered":"2024-09-12T03:20:29.186884+0000","last_unstale":"2024-09-12T03:20:30.786780+0000","last_undegraded":"2024-09-12T03:20:30.786780+0000","last_fullsized":"2024-09-12T03:20:30.786780+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_clean_scrub_stamp":"2024-09-12T03:20:28.136653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:03:04.880817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"80'1","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-12T03:20:30.176341+0000","last_change":"2024-09-12T03:20:28.154730+0000","last_active":"2024-09-12T03:20:30.176341+0000","last_peered":"2024-09-12T03:20:30.176341+0000","last_clean":"2024-09-12T03:20:30.176341+0000","last_became_active":"2024-09-12T03:20:28.154397+0000","last_became_peered":"2024-09-12T03:20:28.154397+0000","last_unstale":"2024-09-12T03:20:30.176341+0000","last_undegraded":"2024-09-12T03:20:30.176341+0000","last_fullsized":"2024-09-12T03:20:30.176341+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:27.129547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:27.129547+0000","last_clean_scrub_stamp":"2024-09-12T03:20:27.129547+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:39:00.949711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-12T03:20:30.786599+0000","last_change":"2024-09-12T03:20:29.186446+0000","last_active":"2024-09-12T03:20:30.786599+0000","last_peered":"2024-09-12T03:20:30.786599+0000","last_clean":"2024-09-12T03:20:30.786599+0000","last_became_active":"2024-09-12T03:20:29.186146+0000","last_became_peered":"2024-09-12T03:20:29.186146+0000","last_unstale":"2024-09-12T03:20:30.786599+0000","last_undegraded":"2024-09-12T03:20:30.786599+0000","last_fullsized":"2024-09-12T03:20:30.786599+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_clean_scrub_stamp":"2024-09-12T03:20:28.136653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:00:06.471862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"80'2","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:20:29.183673+0000","last_change":"2024-09-12T03:20:28.161680+0000","last_active":"2024-09-12T03:20:29.183673+0000","last_peered":"2024-09-12T03:20:29.183673+0000","last_clean":"2024-09-12T03:20:29.183673+0000","last_became_active":"2024-09-12T03:20:28.161376+0000","last_became_peered":"2024-09-12T03:20:28.161376+0000","last_unstale":"2024-09-12T03:20:29.183673+0000","last_undegraded":"2024-09-12T03:20:29.183673+0000","last_fullsized":"2024-09-12T03:20:29.183673+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:27.129547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:27.129547+0000","last_clean_scrub_stamp":"2024-09-12T03:20:27.129547+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:24:19.679944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:20:29.184698+0000","last_change":"2024-09-12T03:20:29.183539+0000","last_active":"2024-09-12T03:20:29.184698+0000","last_peered":"2024-09-12T03:20:29.184698+0000","last_clean":"2024-09-12T03:20:29.184698+0000","last_became_active":"2024-09-12T03:20:29.183255+0000","last_became_peered":"2024-09-12T03:20:29.183255+0000","last_unstale":"2024-09-12T03:20:29.184698+0000","last_undegraded":"2024-09-12T03:20:29.184698+0000","last_fullsized":"2024-09-12T03:20:29.184698+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_clean_scrub_stamp":"2024-09-12T03:20:28.136653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:02:27.543548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"80'2","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:20:29.186388+0000","last_change":"2024-09-12T03:20:28.161569+0000","last_active":"2024-09-12T03:20:29.186388+0000","last_peered":"2024-09-12T03:20:29.186388+0000","last_clean":"2024-09-12T03:20:29.186388+0000","last_became_active":"2024-09-12T03:20:28.161341+0000","last_became_peered":"2024-09-12T03:20:28.161341+0000","last_unstale":"2024-09-12T03:20:29.186388+0000","last_undegraded":"2024-09-12T03:20:29.186388+0000","last_fullsized":"2024-09-12T03:20:29.186388+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:27.129547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:27.129547+0000","last_clean_scrub_stamp":"2024-09-12T03:20:27.129547+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:03:03.873710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-12T03:20:30.174508+0000","last_change":"2024-09-12T03:20:29.185780+0000","last_active":"2024-09-12T03:20:30.174508+0000","last_peered":"2024-09-12T03:20:30.174508+0000","last_clean":"2024-09-12T03:20:30.174508+0000","last_became_active":"2024-09-12T03:20:29.185359+0000","last_became_peered":"2024-09-12T03:20:29.185359+0000","last_unstale":"2024-09-12T03:20:30.174508+0000","last_undegraded":"2024-09-12T03:20:30.174508+0000","last_fullsized":"2024-09-12T03:20:30.174508+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_clean_scrub_stamp":"2024-09-12T03:20:28.136653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:40:55.997920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"80'1","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-12T03:20:30.176484+0000","last_change":"2024-09-12T03:20:28.158989+0000","last_active":"2024-09-12T03:20:30.176484+0000","last_peered":"2024-09-12T03:20:30.176484+0000","last_clean":"2024-09-12T03:20:30.176484+0000","last_became_active":"2024-09-12T03:20:28.158665+0000","last_became_peered":"2024-09-12T03:20:28.158665+0000","last_unstale":"2024-09-12T03:20:30.176484+0000","last_undegraded":"2024-09-12T03:20:30.176484+0000","last_fullsized":"2024-09-12T03:20:30.176484+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:27.129547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:27.129547+0000","last_clean_scrub_stamp":"2024-09-12T03:20:27.129547+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:41:56.345114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-12T03:20:30.176325+0000","last_change":"2024-09-12T03:20:29.182394+0000","last_active":"2024-09-12T03:20:30.176325+0000","last_peered":"2024-09-12T03:20:30.176325+0000","last_clean":"2024-09-12T03:20:30.176325+0000","last_became_active":"2024-09-12T03:20:29.182107+0000","last_became_peered":"2024-09-12T03:20:29.182107+0000","last_unstale":"2024-09-12T03:20:30.176325+0000","last_undegraded":"2024-09-12T03:20:30.176325+0000","last_fullsized":"2024-09-12T03:20:30.176325+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_clean_scrub_stamp":"2024-09-12T03:20:28.136653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:03:04.880817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"80'1","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-12T03:20:30.175369+0000","last_change":"2024-09-12T03:20:28.153237+0000","last_active":"2024-09-12T03:20:30.175369+0000","last_peered":"2024-09-12T03:20:30.175369+0000","last_clean":"2024-09-12T03:20:30.175369+0000","last_became_active":"2024-09-12T03:20:28.152963+0000","last_became_peered":"2024-09-12T03:20:28.152963+0000","last_unstale":"2024-09-12T03:20:30.175369+0000","last_undegraded":"2024-09-12T03:20:30.175369+0000","last_fullsized":"2024-09-12T03:20:30.175369+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:27.129547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:27.129547+0000","last_clean_scrub_stamp":"2024-09-12T03:20:27.129547+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:31:36.558882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:20:29.187754+0000","last_change":"2024-09-12T03:20:29.185810+0000","last_active":"2024-09-12T03:20:29.187754+0000","last_peered":"2024-09-12T03:20:29.187754+0000","last_clean":"2024-09-12T03:20:29.187754+0000","last_became_active":"2024-09-12T03:20:29.185507+0000","last_became_peered":"2024-09-12T03:20:29.185507+0000","last_unstale":"2024-09-12T03:20:29.187754+0000","last_undegraded":"2024-09-12T03:20:29.187754+0000","last_fullsized":"2024-09-12T03:20:29.187754+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_clean_scrub_stamp":"2024-09-12T03:20:28.136653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:55:30.198372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:20:29.174327+0000","last_change":"2024-09-12T03:20:28.156112+0000","last_active":"2024-09-12T03:20:29.174327+0000","last_peered":"2024-09-12T03:20:29.174327+0000","last_clean":"2024-09-12T03:20:29.174327+0000","last_became_active":"2024-09-12T03:20:28.155826+0000","last_became_peered":"2024-09-12T03:20:28.155826+0000","last_unstale":"2024-09-12T03:20:29.174327+0000","last_undegraded":"2024-09-12T03:20:29.174327+0000","last_fullsized":"2024-09-12T03:20:29.174327+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:27.129547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:27.129547+0000","last_clean_scrub_stamp":"2024-09-12T03:20:27.129547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:12:11.478865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-12T03:20:30.809964+0000","last_change":"2024-09-12T03:20:29.178853+0000","last_active":"2024-09-12T03:20:30.809964+0000","last_peered":"2024-09-12T03:20:30.809964+0000","last_clean":"2024-09-12T03:20:30.809964+0000","last_became_active":"2024-09-12T03:20:29.178567+0000","last_became_peered":"2024-09-12T03:20:29.178567+0000","last_unstale":"2024-09-12T03:20:30.809964+0000","last_undegraded":"2024-09-12T03:20:30.809964+0000","last_fullsized":"2024-09-12T03:20:30.809964+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_clean_scrub_stamp":"2024-09-12T03:20:28.136653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:06:02.733007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-12T03:20:30.174313+0000","last_change":"2024-09-12T03:20:28.158205+0000","last_active":"2024-09-12T03:20:30.174313+0000","last_peered":"2024-09-12T03:20:30.174313+0000","last_clean":"2024-09-12T03:20:30.174313+0000","last_became_active":"2024-09-12T03:20:28.157948+0000","last_became_peered":"2024-09-12T03:20:28.157948+0000","last_unstale":"2024-09-12T03:20:30.174313+0000","last_undegraded":"2024-09-12T03:20:30.174313+0000","last_fullsized":"2024-09-12T03:20:30.174313+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:27.129547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:27.129547+0000","last_clean_scrub_stamp":"2024-09-12T03:20:27.129547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:34:49.505569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-12T03:20:30.810190+0000","last_change":"2024-09-12T03:20:29.180546+0000","last_active":"2024-09-12T03:20:30.810190+0000","last_peered":"2024-09-12T03:20:30.810190+0000","last_clean":"2024-09-12T03:20:30.810190+0000","last_became_active":"2024-09-12T03:20:29.180269+0000","last_became_peered":"2024-09-12T03:20:29.180269+0000","last_unstale":"2024-09-12T03:20:30.810190+0000","last_undegraded":"2024-09-12T03:20:30.810190+0000","last_fullsized":"2024-09-12T03:20:30.810190+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_clean_scrub_stamp":"2024-09-12T03:20:28.136653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:55:11.908871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-12T03:20:30.786935+0000","last_change":"2024-09-12T03:20:28.160028+0000","last_active":"2024-09-12T03:20:30.786935+0000","last_peered":"2024-09-12T03:20:30.786935+0000","last_clean":"2024-09-12T03:20:30.786935+0000","last_became_active":"2024-09-12T03:20:28.159688+0000","last_became_peered":"2024-09-12T03:20:28.159688+0000","last_unstale":"2024-09-12T03:20:30.786935+0000","last_undegraded":"2024-09-12T03:20:30.786935+0000","last_fullsized":"2024-09-12T03:20:30.786935+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:27.129547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:27.129547+0000","last_clean_scrub_stamp":"2024-09-12T03:20:27.129547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:25:20.650913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-12T03:20:30.175336+0000","last_change":"2024-09-12T03:20:29.178739+0000","last_active":"2024-09-12T03:20:30.175336+0000","last_peered":"2024-09-12T03:20:30.175336+0000","last_clean":"2024-09-12T03:20:30.175336+0000","last_became_active":"2024-09-12T03:20:29.178409+0000","last_became_peered":"2024-09-12T03:20:29.178409+0000","last_unstale":"2024-09-12T03:20:30.175336+0000","last_undegraded":"2024-09-12T03:20:30.175336+0000","last_fullsized":"2024-09-12T03:20:30.175336+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_clean_scrub_stamp":"2024-09-12T03:20:28.136653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:15:12.656811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-12T03:20:30.786750+0000","last_change":"2024-09-12T03:20:28.160587+0000","last_active":"2024-09-12T03:20:30.786750+0000","last_peered":"2024-09-12T03:20:30.786750+0000","last_clean":"2024-09-12T03:20:30.786750+0000","last_became_active":"2024-09-12T03:20:28.160114+0000","last_became_peered":"2024-09-12T03:20:28.160114+0000","last_unstale":"2024-09-12T03:20:30.786750+0000","last_undegraded":"2024-09-12T03:20:30.786750+0000","last_fullsized":"2024-09-12T03:20:30.786750+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:27.129547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:27.129547+0000","last_clean_scrub_stamp":"2024-09-12T03:20:27.129547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:45:52.294635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:20:29.186201+0000","last_change":"2024-09-12T03:20:29.181211+0000","last_active":"2024-09-12T03:20:29.186201+0000","last_peered":"2024-09-12T03:20:29.186201+0000","last_clean":"2024-09-12T03:20:29.186201+0000","last_became_active":"2024-09-12T03:20:29.180955+0000","last_became_peered":"2024-09-12T03:20:29.180955+0000","last_unstale":"2024-09-12T03:20:29.186201+0000","last_undegraded":"2024-09-12T03:20:29.186201+0000","last_fullsized":"2024-09-12T03:20:29.186201+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_clean_scrub_stamp":"2024-09-12T03:20:28.136653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:50:43.423983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"80'3","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-12T03:20:30.786361+0000","last_change":"2024-09-12T03:20:28.153383+0000","last_active":"2024-09-12T03:20:30.786361+0000","last_peered":"2024-09-12T03:20:30.786361+0000","last_clean":"2024-09-12T03:20:30.786361+0000","last_became_active":"2024-09-12T03:20:28.153124+0000","last_became_peered":"2024-09-12T03:20:28.153124+0000","last_unstale":"2024-09-12T03:20:30.786361+0000","last_undegraded":"2024-09-12T03:20:30.786361+0000","last_fullsized":"2024-09-12T03:20:30.786361+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:27.129547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:27.129547+0000","last_clean_scrub_stamp":"2024-09-12T03:20:27.129547+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:59:28.672500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-12T03:20:30.176207+0000","last_change":"2024-09-12T03:20:29.181858+0000","last_active":"2024-09-12T03:20:30.176207+0000","last_peered":"2024-09-12T03:20:30.176207+0000","last_clean":"2024-09-12T03:20:30.176207+0000","last_became_active":"2024-09-12T03:20:29.181552+0000","last_became_peered":"2024-09-12T03:20:29.181552+0000","last_unstale":"2024-09-12T03:20:30.176207+0000","last_undegraded":"2024-09-12T03:20:30.176207+0000","last_fullsized":"2024-09-12T03:20:30.176207+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_clean_scrub_stamp":"2024-09-12T03:20:28.136653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:20:39.549097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"80'6","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-12T03:20:30.175840+0000","last_change":"2024-09-12T03:20:28.155799+0000","last_active":"2024-09-12T03:20:30.175840+0000","last_peered":"2024-09-12T03:20:30.175840+0000","last_clean":"2024-09-12T03:20:30.175840+0000","last_became_active":"2024-09-12T03:20:28.155517+0000","last_became_peered":"2024-09-12T03:20:28.155517+0000","last_unstale":"2024-09-12T03:20:30.175840+0000","last_undegraded":"2024-09-12T03:20:30.175840+0000","last_fullsized":"2024-09-12T03:20:30.175840+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:27.129547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:27.129547+0000","last_clean_scrub_stamp":"2024-09-12T03:20:27.129547+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:43:32.260885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:20:29.181931+0000","last_change":"2024-09-12T03:20:29.179290+0000","last_active":"2024-09-12T03:20:29.181931+0000","last_peered":"2024-09-12T03:20:29.181931+0000","last_clean":"2024-09-12T03:20:29.181931+0000","last_became_active":"2024-09-12T03:20:29.179000+0000","last_became_peered":"2024-09-12T03:20:29.179000+0000","last_unstale":"2024-09-12T03:20:29.181931+0000","last_undegraded":"2024-09-12T03:20:29.181931+0000","last_fullsized":"2024-09-12T03:20:29.181931+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_clean_scrub_stamp":"2024-09-12T03:20:28.136653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:37:37.651061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"80'2","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-12T03:20:30.174738+0000","last_change":"2024-09-12T03:20:28.160059+0000","last_active":"2024-09-12T03:20:30.174738+0000","last_peered":"2024-09-12T03:20:30.174738+0000","last_clean":"2024-09-12T03:20:30.174738+0000","last_became_active":"2024-09-12T03:20:28.159780+0000","last_became_peered":"2024-09-12T03:20:28.159780+0000","last_unstale":"2024-09-12T03:20:30.174738+0000","last_undegraded":"2024-09-12T03:20:30.174738+0000","last_fullsized":"2024-09-12T03:20:30.174738+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:27.129547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:27.129547+0000","last_clean_scrub_stamp":"2024-09-12T03:20:27.129547+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:06:12.007015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:20:29.187632+0000","last_change":"2024-09-12T03:20:29.185677+0000","last_active":"2024-09-12T03:20:29.187632+0000","last_peered":"2024-09-12T03:20:29.187632+0000","last_clean":"2024-09-12T03:20:29.187632+0000","last_became_active":"2024-09-12T03:20:29.185297+0000","last_became_peered":"2024-09-12T03:20:29.185297+0000","last_unstale":"2024-09-12T03:20:29.187632+0000","last_undegraded":"2024-09-12T03:20:29.187632+0000","last_fullsized":"2024-09-12T03:20:29.187632+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_clean_scrub_stamp":"2024-09-12T03:20:28.136653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:03:04.880817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-12T03:20:30.810123+0000","last_change":"2024-09-12T03:20:28.154261+0000","last_active":"2024-09-12T03:20:30.810123+0000","last_peered":"2024-09-12T03:20:30.810123+0000","last_clean":"2024-09-12T03:20:30.810123+0000","last_became_active":"2024-09-12T03:20:28.153194+0000","last_became_peered":"2024-09-12T03:20:28.153194+0000","last_unstale":"2024-09-12T03:20:30.810123+0000","last_undegraded":"2024-09-12T03:20:30.810123+0000","last_fullsized":"2024-09-12T03:20:30.810123+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:27.129547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:27.129547+0000","last_clean_scrub_stamp":"2024-09-12T03:20:27.129547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:36:18.476441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-12T03:20:30.174300+0000","last_change":"2024-09-12T03:20:29.188626+0000","last_active":"2024-09-12T03:20:30.174300+0000","last_peered":"2024-09-12T03:20:30.174300+0000","last_clean":"2024-09-12T03:20:30.174300+0000","last_became_active":"2024-09-12T03:20:29.186136+0000","last_became_peered":"2024-09-12T03:20:29.186136+0000","last_unstale":"2024-09-12T03:20:30.174300+0000","last_undegraded":"2024-09-12T03:20:30.174300+0000","last_fullsized":"2024-09-12T03:20:30.174300+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_clean_scrub_stamp":"2024-09-12T03:20:28.136653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:58:49.778272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-12T03:20:30.175618+0000","last_change":"2024-09-12T03:20:28.158821+0000","last_active":"2024-09-12T03:20:30.175618+0000","last_peered":"2024-09-12T03:20:30.175618+0000","last_clean":"2024-09-12T03:20:30.175618+0000","last_became_active":"2024-09-12T03:20:28.158517+0000","last_became_peered":"2024-09-12T03:20:28.158517+0000","last_unstale":"2024-09-12T03:20:30.175618+0000","last_undegraded":"2024-09-12T03:20:30.175618+0000","last_fullsized":"2024-09-12T03:20:30.175618+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:27.129547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:27.129547+0000","last_clean_scrub_stamp":"2024-09-12T03:20:27.129547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:12:58.865580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-12T03:20:30.174242+0000","last_change":"2024-09-12T03:20:29.176284+0000","last_active":"2024-09-12T03:20:30.174242+0000","last_peered":"2024-09-12T03:20:30.174242+0000","last_clean":"2024-09-12T03:20:30.174242+0000","last_became_active":"2024-09-12T03:20:29.175900+0000","last_became_peered":"2024-09-12T03:20:29.175900+0000","last_unstale":"2024-09-12T03:20:30.174242+0000","last_undegraded":"2024-09-12T03:20:30.174242+0000","last_fullsized":"2024-09-12T03:20:30.174242+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_clean_scrub_stamp":"2024-09-12T03:20:28.136653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:33:07.633277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"80'8","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:20:29.188145+0000","last_change":"2024-09-12T03:20:28.159183+0000","last_active":"2024-09-12T03:20:29.188145+0000","last_peered":"2024-09-12T03:20:29.188145+0000","last_clean":"2024-09-12T03:20:29.188145+0000","last_became_active":"2024-09-12T03:20:28.158552+0000","last_became_peered":"2024-09-12T03:20:28.158552+0000","last_unstale":"2024-09-12T03:20:29.188145+0000","last_undegraded":"2024-09-12T03:20:29.188145+0000","last_fullsized":"2024-09-12T03:20:29.188145+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:27.129547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:27.129547+0000","last_clean_scrub_stamp":"2024-09-12T03:20:27.129547+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:29:13.201310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:20:29.177258+0000","last_change":"2024-09-12T03:20:29.175128+0000","last_active":"2024-09-12T03:20:29.177258+0000","last_peered":"2024-09-12T03:20:29.177258+0000","last_clean":"2024-09-12T03:20:29.177258+0000","last_became_active":"2024-09-12T03:20:29.174805+0000","last_became_peered":"2024-09-12T03:20:29.174805+0000","last_unstale":"2024-09-12T03:20:29.177258+0000","last_undegraded":"2024-09-12T03:20:29.177258+0000","last_fullsized":"2024-09-12T03:20:29.177258+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_clean_scrub_stamp":"2024-09-12T03:20:28.136653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:17:42.735036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"80'1","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-12T03:20:30.786997+0000","last_change":"2024-09-12T03:20:28.158768+0000","last_active":"2024-09-12T03:20:30.786997+0000","last_peered":"2024-09-12T03:20:30.786997+0000","last_clean":"2024-09-12T03:20:30.786997+0000","last_became_active":"2024-09-12T03:20:28.158309+0000","last_became_peered":"2024-09-12T03:20:28.158309+0000","last_unstale":"2024-09-12T03:20:30.786997+0000","last_undegraded":"2024-09-12T03:20:30.786997+0000","last_fullsized":"2024-09-12T03:20:30.786997+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:27.129547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:27.129547+0000","last_clean_scrub_stamp":"2024-09-12T03:20:27.129547+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:31:36.558882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-12T03:20:30.786908+0000","last_change":"2024-09-12T03:20:29.177747+0000","last_active":"2024-09-12T03:20:30.786908+0000","last_peered":"2024-09-12T03:20:30.786908+0000","last_clean":"2024-09-12T03:20:30.786908+0000","last_became_active":"2024-09-12T03:20:29.177105+0000","last_became_peered":"2024-09-12T03:20:29.177105+0000","last_unstale":"2024-09-12T03:20:30.786908+0000","last_undegraded":"2024-09-12T03:20:30.786908+0000","last_fullsized":"2024-09-12T03:20:30.786908+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_clean_scrub_stamp":"2024-09-12T03:20:28.136653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:55:30.198372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"80'2","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:20:29.182072+0000","last_change":"2024-09-12T03:20:28.160177+0000","last_active":"2024-09-12T03:20:29.182072+0000","last_peered":"2024-09-12T03:20:29.182072+0000","last_clean":"2024-09-12T03:20:29.182072+0000","last_became_active":"2024-09-12T03:20:28.159925+0000","last_became_peered":"2024-09-12T03:20:28.159925+0000","last_unstale":"2024-09-12T03:20:29.182072+0000","last_undegraded":"2024-09-12T03:20:29.182072+0000","last_fullsized":"2024-09-12T03:20:29.182072+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:27.129547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:27.129547+0000","last_clean_scrub_stamp":"2024-09-12T03:20:27.129547+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:41:28.079395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-12T03:20:30.176318+0000","last_change":"2024-09-12T03:20:29.187004+0000","last_active":"2024-09-12T03:20:30.176318+0000","last_peered":"2024-09-12T03:20:30.176318+0000","last_clean":"2024-09-12T03:20:30.176318+0000","last_became_active":"2024-09-12T03:20:29.186689+0000","last_became_peered":"2024-09-12T03:20:29.186689+0000","last_unstale":"2024-09-12T03:20:30.176318+0000","last_undegraded":"2024-09-12T03:20:30.176318+0000","last_fullsized":"2024-09-12T03:20:30.176318+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_clean_scrub_stamp":"2024-09-12T03:20:28.136653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:02:27.543548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:20:29.171529+0000","last_change":"2024-09-12T03:20:28.156929+0000","last_active":"2024-09-12T03:20:29.171529+0000","last_peered":"2024-09-12T03:20:29.171529+0000","last_clean":"2024-09-12T03:20:29.171529+0000","last_became_active":"2024-09-12T03:20:28.156666+0000","last_became_peered":"2024-09-12T03:20:28.156666+0000","last_unstale":"2024-09-12T03:20:29.171529+0000","last_undegraded":"2024-09-12T03:20:29.171529+0000","last_fullsized":"2024-09-12T03:20:29.171529+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:27.129547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:27.129547+0000","last_clean_scrub_stamp":"2024-09-12T03:20:27.129547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:55:29.191265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:20:29.192217+0000","last_change":"2024-09-12T03:20:29.186329+0000","last_active":"2024-09-12T03:20:29.192217+0000","last_peered":"2024-09-12T03:20:29.192217+0000","last_clean":"2024-09-12T03:20:29.192217+0000","last_became_active":"2024-09-12T03:20:29.186050+0000","last_became_peered":"2024-09-12T03:20:29.186050+0000","last_unstale":"2024-09-12T03:20:29.192217+0000","last_undegraded":"2024-09-12T03:20:29.192217+0000","last_fullsized":"2024-09-12T03:20:29.192217+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_clean_scrub_stamp":"2024-09-12T03:20:28.136653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:14:44.871074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"80'2","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-12T03:20:30.810101+0000","last_change":"2024-09-12T03:20:28.161455+0000","last_active":"2024-09-12T03:20:30.810101+0000","last_peered":"2024-09-12T03:20:30.810101+0000","last_clean":"2024-09-12T03:20:30.810101+0000","last_became_active":"2024-09-12T03:20:28.161180+0000","last_became_peered":"2024-09-12T03:20:28.161180+0000","last_unstale":"2024-09-12T03:20:30.810101+0000","last_undegraded":"2024-09-12T03:20:30.810101+0000","last_fullsized":"2024-09-12T03:20:30.810101+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:27.129547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:27.129547+0000","last_clean_scrub_stamp":"2024-09-12T03:20:27.129547+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:24:19.679944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-12T03:20:30.175467+0000","last_change":"2024-09-12T03:20:29.172018+0000","last_active":"2024-09-12T03:20:30.175467+0000","last_peered":"2024-09-12T03:20:30.175467+0000","last_clean":"2024-09-12T03:20:30.175467+0000","last_became_active":"2024-09-12T03:20:29.171710+0000","last_became_peered":"2024-09-12T03:20:29.171710+0000","last_unstale":"2024-09-12T03:20:30.175467+0000","last_undegraded":"2024-09-12T03:20:30.175467+0000","last_fullsized":"2024-09-12T03:20:30.175467+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_clean_scrub_stamp":"2024-09-12T03:20:28.136653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:28:05.839309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:20:29.168591+0000","last_change":"2024-09-12T03:20:28.158357+0000","last_active":"2024-09-12T03:20:29.168591+0000","last_peered":"2024-09-12T03:20:29.168591+0000","last_clean":"2024-09-12T03:20:29.168591+0000","last_became_active":"2024-09-12T03:20:28.158079+0000","last_became_peered":"2024-09-12T03:20:28.158079+0000","last_unstale":"2024-09-12T03:20:29.168591+0000","last_undegraded":"2024-09-12T03:20:29.168591+0000","last_fullsized":"2024-09-12T03:20:29.168591+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:27.129547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:27.129547+0000","last_clean_scrub_stamp":"2024-09-12T03:20:27.129547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:17:24.670406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:20:29.184471+0000","last_change":"2024-09-12T03:20:29.180789+0000","last_active":"2024-09-12T03:20:29.184471+0000","last_peered":"2024-09-12T03:20:29.184471+0000","last_clean":"2024-09-12T03:20:29.184471+0000","last_became_active":"2024-09-12T03:20:29.180491+0000","last_became_peered":"2024-09-12T03:20:29.180491+0000","last_unstale":"2024-09-12T03:20:29.184471+0000","last_undegraded":"2024-09-12T03:20:29.184471+0000","last_fullsized":"2024-09-12T03:20:29.184471+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_clean_scrub_stamp":"2024-09-12T03:20:28.136653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:44:59.211551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:20:29.166384+0000","last_change":"2024-09-12T03:20:28.155068+0000","last_active":"2024-09-12T03:20:29.166384+0000","last_peered":"2024-09-12T03:20:29.166384+0000","last_clean":"2024-09-12T03:20:29.166384+0000","last_became_active":"2024-09-12T03:20:28.154805+0000","last_became_peered":"2024-09-12T03:20:28.154805+0000","last_unstale":"2024-09-12T03:20:29.166384+0000","last_undegraded":"2024-09-12T03:20:29.166384+0000","last_fullsized":"2024-09-12T03:20:29.166384+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:27.129547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:27.129547+0000","last_clean_scrub_stamp":"2024-09-12T03:20:27.129547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:13:40.945583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-12T03:20:30.175632+0000","last_change":"2024-09-12T03:20:29.183683+0000","last_active":"2024-09-12T03:20:30.175632+0000","last_peered":"2024-09-12T03:20:30.175632+0000","last_clean":"2024-09-12T03:20:30.175632+0000","last_became_active":"2024-09-12T03:20:29.183388+0000","last_became_peered":"2024-09-12T03:20:29.183388+0000","last_unstale":"2024-09-12T03:20:30.175632+0000","last_undegraded":"2024-09-12T03:20:30.175632+0000","last_fullsized":"2024-09-12T03:20:30.175632+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_clean_scrub_stamp":"2024-09-12T03:20:28.136653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:05:32.014494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-12T03:20:30.809812+0000","last_change":"2024-09-12T03:20:28.158296+0000","last_active":"2024-09-12T03:20:30.809812+0000","last_peered":"2024-09-12T03:20:30.809812+0000","last_clean":"2024-09-12T03:20:30.809812+0000","last_became_active":"2024-09-12T03:20:28.157999+0000","last_became_peered":"2024-09-12T03:20:28.157999+0000","last_unstale":"2024-09-12T03:20:30.809812+0000","last_undegraded":"2024-09-12T03:20:30.809812+0000","last_fullsized":"2024-09-12T03:20:30.809812+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:27.129547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:27.129547+0000","last_clean_scrub_stamp":"2024-09-12T03:20:27.129547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:41:28.079395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-12T03:20:30.176207+0000","last_change":"2024-09-12T03:20:29.166228+0000","last_active":"2024-09-12T03:20:30.176207+0000","last_peered":"2024-09-12T03:20:30.176207+0000","last_clean":"2024-09-12T03:20:30.176207+0000","last_became_active":"2024-09-12T03:20:29.165927+0000","last_became_peered":"2024-09-12T03:20:29.165927+0000","last_unstale":"2024-09-12T03:20:30.176207+0000","last_undegraded":"2024-09-12T03:20:30.176207+0000","last_fullsized":"2024-09-12T03:20:30.176207+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_clean_scrub_stamp":"2024-09-12T03:20:28.136653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:00:06.471862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"80'2","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-12T03:20:30.174583+0000","last_change":"2024-09-12T03:20:28.159053+0000","last_active":"2024-09-12T03:20:30.174583+0000","last_peered":"2024-09-12T03:20:30.174583+0000","last_clean":"2024-09-12T03:20:30.174583+0000","last_became_active":"2024-09-12T03:20:28.158729+0000","last_became_peered":"2024-09-12T03:20:28.158729+0000","last_unstale":"2024-09-12T03:20:30.174583+0000","last_undegraded":"2024-09-12T03:20:30.174583+0000","last_fullsized":"2024-09-12T03:20:30.174583+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:27.129547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:27.129547+0000","last_clean_scrub_stamp":"2024-09-12T03:20:27.129547+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:23:15.274669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:20:29.177473+0000","last_change":"2024-09-12T03:20:29.175669+0000","last_active":"2024-09-12T03:20:29.177473+0000","last_peered":"2024-09-12T03:20:29.177473+0000","last_clean":"2024-09-12T03:20:29.177473+0000","last_became_active":"2024-09-12T03:20:29.175336+0000","last_became_peered":"2024-09-12T03:20:29.175336+0000","last_unstale":"2024-09-12T03:20:29.177473+0000","last_undegraded":"2024-09-12T03:20:29.177473+0000","last_fullsized":"2024-09-12T03:20:29.177473+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_clean_scrub_stamp":"2024-09-12T03:20:28.136653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:00:06.471862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-12T03:20:30.175002+0000","last_change":"2024-09-12T03:20:28.159898+0000","last_active":"2024-09-12T03:20:30.175002+0000","last_peered":"2024-09-12T03:20:30.175002+0000","last_clean":"2024-09-12T03:20:30.175002+0000","last_became_active":"2024-09-12T03:20:28.159573+0000","last_became_peered":"2024-09-12T03:20:28.159573+0000","last_unstale":"2024-09-12T03:20:30.175002+0000","last_undegraded":"2024-09-12T03:20:30.175002+0000","last_fullsized":"2024-09-12T03:20:30.175002+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:27.129547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:27.129547+0000","last_clean_scrub_stamp":"2024-09-12T03:20:27.129547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:22:43.287524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-12T03:20:30.786644+0000","last_change":"2024-09-12T03:20:29.184755+0000","last_active":"2024-09-12T03:20:30.786644+0000","last_peered":"2024-09-12T03:20:30.786644+0000","last_clean":"2024-09-12T03:20:30.786644+0000","last_became_active":"2024-09-12T03:20:29.184474+0000","last_became_peered":"2024-09-12T03:20:29.184474+0000","last_unstale":"2024-09-12T03:20:30.786644+0000","last_undegraded":"2024-09-12T03:20:30.786644+0000","last_fullsized":"2024-09-12T03:20:30.786644+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_clean_scrub_stamp":"2024-09-12T03:20:28.136653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:20:39.549097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-12T03:20:30.786603+0000","last_change":"2024-09-12T03:20:28.154889+0000","last_active":"2024-09-12T03:20:30.786603+0000","last_peered":"2024-09-12T03:20:30.786603+0000","last_clean":"2024-09-12T03:20:30.786603+0000","last_became_active":"2024-09-12T03:20:28.154544+0000","last_became_peered":"2024-09-12T03:20:28.154544+0000","last_unstale":"2024-09-12T03:20:30.786603+0000","last_undegraded":"2024-09-12T03:20:30.786603+0000","last_fullsized":"2024-09-12T03:20:30.786603+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:27.129547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:27.129547+0000","last_clean_scrub_stamp":"2024-09-12T03:20:27.129547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:01:16.414722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-12T03:20:30.176391+0000","last_change":"2024-09-12T03:20:29.181440+0000","last_active":"2024-09-12T03:20:30.176391+0000","last_peered":"2024-09-12T03:20:30.176391+0000","last_clean":"2024-09-12T03:20:30.176391+0000","last_became_active":"2024-09-12T03:20:29.181060+0000","last_became_peered":"2024-09-12T03:20:29.181060+0000","last_unstale":"2024-09-12T03:20:30.176391+0000","last_undegraded":"2024-09-12T03:20:30.176391+0000","last_fullsized":"2024-09-12T03:20:30.176391+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_clean_scrub_stamp":"2024-09-12T03:20:28.136653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:55:30.198372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-12T03:20:30.786741+0000","last_change":"2024-09-12T03:20:28.159532+0000","last_active":"2024-09-12T03:20:30.786741+0000","last_peered":"2024-09-12T03:20:30.786741+0000","last_clean":"2024-09-12T03:20:30.786741+0000","last_became_active":"2024-09-12T03:20:28.159207+0000","last_became_peered":"2024-09-12T03:20:28.159207+0000","last_unstale":"2024-09-12T03:20:30.786741+0000","last_undegraded":"2024-09-12T03:20:30.786741+0000","last_fullsized":"2024-09-12T03:20:30.786741+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:27.129547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:27.129547+0000","last_clean_scrub_stamp":"2024-09-12T03:20:27.129547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:41:56.345114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-12T03:20:30.175277+0000","last_change":"2024-09-12T03:20:29.171340+0000","last_active":"2024-09-12T03:20:30.175277+0000","last_peered":"2024-09-12T03:20:30.175277+0000","last_clean":"2024-09-12T03:20:30.175277+0000","last_became_active":"2024-09-12T03:20:29.171051+0000","last_became_peered":"2024-09-12T03:20:29.171051+0000","last_unstale":"2024-09-12T03:20:30.175277+0000","last_undegraded":"2024-09-12T03:20:30.175277+0000","last_fullsized":"2024-09-12T03:20:30.175277+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_clean_scrub_stamp":"2024-09-12T03:20:28.136653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:30:26.379868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:20:29.174095+0000","last_change":"2024-09-12T03:20:28.158162+0000","last_active":"2024-09-12T03:20:29.174095+0000","last_peered":"2024-09-12T03:20:29.174095+0000","last_clean":"2024-09-12T03:20:29.174095+0000","last_became_active":"2024-09-12T03:20:28.157908+0000","last_became_peered":"2024-09-12T03:20:28.157908+0000","last_unstale":"2024-09-12T03:20:29.174095+0000","last_undegraded":"2024-09-12T03:20:29.174095+0000","last_fullsized":"2024-09-12T03:20:29.174095+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:27.129547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:27.129547+0000","last_clean_scrub_stamp":"2024-09-12T03:20:27.129547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:36:18.476441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:20:29.182213+0000","last_change":"2024-09-12T03:20:29.178330+0000","last_active":"2024-09-12T03:20:29.182213+0000","last_peered":"2024-09-12T03:20:29.182213+0000","last_clean":"2024-09-12T03:20:29.182213+0000","last_became_active":"2024-09-12T03:20:29.178067+0000","last_became_peered":"2024-09-12T03:20:29.178067+0000","last_unstale":"2024-09-12T03:20:29.182213+0000","last_undegraded":"2024-09-12T03:20:29.182213+0000","last_fullsized":"2024-09-12T03:20:29.182213+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_clean_scrub_stamp":"2024-09-12T03:20:28.136653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:47:28.912863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-12T03:20:30.176093+0000","last_change":"2024-09-12T03:20:28.153258+0000","last_active":"2024-09-12T03:20:30.176093+0000","last_peered":"2024-09-12T03:20:30.176093+0000","last_clean":"2024-09-12T03:20:30.176093+0000","last_became_active":"2024-09-12T03:20:28.153006+0000","last_became_peered":"2024-09-12T03:20:28.153006+0000","last_unstale":"2024-09-12T03:20:30.176093+0000","last_undegraded":"2024-09-12T03:20:30.176093+0000","last_fullsized":"2024-09-12T03:20:30.176093+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:27.129547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:27.129547+0000","last_clean_scrub_stamp":"2024-09-12T03:20:27.129547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:25:20.650913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:20:29.190369+0000","last_change":"2024-09-12T03:20:29.189865+0000","last_active":"2024-09-12T03:20:29.190369+0000","last_peered":"2024-09-12T03:20:29.190369+0000","last_clean":"2024-09-12T03:20:29.190369+0000","last_became_active":"2024-09-12T03:20:29.189592+0000","last_became_peered":"2024-09-12T03:20:29.189592+0000","last_unstale":"2024-09-12T03:20:29.190369+0000","last_undegraded":"2024-09-12T03:20:29.190369+0000","last_fullsized":"2024-09-12T03:20:29.190369+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_clean_scrub_stamp":"2024-09-12T03:20:28.136653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:24:20.687050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-12T03:20:30.786470+0000","last_change":"2024-09-12T03:20:28.159255+0000","last_active":"2024-09-12T03:20:30.786470+0000","last_peered":"2024-09-12T03:20:30.786470+0000","last_clean":"2024-09-12T03:20:30.786470+0000","last_became_active":"2024-09-12T03:20:28.158961+0000","last_became_peered":"2024-09-12T03:20:28.158961+0000","last_unstale":"2024-09-12T03:20:30.786470+0000","last_undegraded":"2024-09-12T03:20:30.786470+0000","last_fullsized":"2024-09-12T03:20:30.786470+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:27.129547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:27.129547+0000","last_clean_scrub_stamp":"2024-09-12T03:20:27.129547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:41:42.935780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-12T03:20:30.786309+0000","last_change":"2024-09-12T03:20:29.184619+0000","last_active":"2024-09-12T03:20:30.786309+0000","last_peered":"2024-09-12T03:20:30.786309+0000","last_clean":"2024-09-12T03:20:30.786309+0000","last_became_active":"2024-09-12T03:20:29.183980+0000","last_became_peered":"2024-09-12T03:20:29.183980+0000","last_unstale":"2024-09-12T03:20:30.786309+0000","last_undegraded":"2024-09-12T03:20:30.786309+0000","last_fullsized":"2024-09-12T03:20:30.786309+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_clean_scrub_stamp":"2024-09-12T03:20:28.136653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:21:39.137804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-12T03:20:30.175301+0000","last_change":"2024-09-12T03:20:28.158015+0000","last_active":"2024-09-12T03:20:30.175301+0000","last_peered":"2024-09-12T03:20:30.175301+0000","last_clean":"2024-09-12T03:20:30.175301+0000","last_became_active":"2024-09-12T03:20:28.157524+0000","last_became_peered":"2024-09-12T03:20:28.157524+0000","last_unstale":"2024-09-12T03:20:30.175301+0000","last_undegraded":"2024-09-12T03:20:30.175301+0000","last_fullsized":"2024-09-12T03:20:30.175301+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:27.129547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:27.129547+0000","last_clean_scrub_stamp":"2024-09-12T03:20:27.129547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:45:52.294635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-12T03:20:30.174983+0000","last_change":"2024-09-12T03:20:29.177151+0000","last_active":"2024-09-12T03:20:30.174983+0000","last_peered":"2024-09-12T03:20:30.174983+0000","last_clean":"2024-09-12T03:20:30.174983+0000","last_became_active":"2024-09-12T03:20:29.175828+0000","last_became_peered":"2024-09-12T03:20:29.175828+0000","last_unstale":"2024-09-12T03:20:30.174983+0000","last_undegraded":"2024-09-12T03:20:30.174983+0000","last_fullsized":"2024-09-12T03:20:30.174983+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_clean_scrub_stamp":"2024-09-12T03:20:28.136653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:52:37.273971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"80'1","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:20:29.186395+0000","last_change":"2024-09-12T03:20:28.163141+0000","last_active":"2024-09-12T03:20:29.186395+0000","last_peered":"2024-09-12T03:20:29.186395+0000","last_clean":"2024-09-12T03:20:29.186395+0000","last_became_active":"2024-09-12T03:20:28.162809+0000","last_became_peered":"2024-09-12T03:20:28.162809+0000","last_unstale":"2024-09-12T03:20:29.186395+0000","last_undegraded":"2024-09-12T03:20:29.186395+0000","last_fullsized":"2024-09-12T03:20:29.186395+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:27.129547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:27.129547+0000","last_clean_scrub_stamp":"2024-09-12T03:20:27.129547+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:21:38.130697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"70'128","reported_seq":316,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:20:29.167861+0000","last_change":"2024-09-12T03:16:21.883446+0000","last_active":"2024-09-12T03:20:29.167861+0000","last_peered":"2024-09-12T03:20:29.167861+0000","last_clean":"2024-09-12T03:20:29.167861+0000","last_became_active":"2024-09-12T03:16:21.882207+0000","last_became_peered":"2024-09-12T03:16:21.882207+0000","last_unstale":"2024-09-12T03:20:29.167861+0000","last_undegraded":"2024-09-12T03:20:29.167861+0000","last_fullsized":"2024-09-12T03:20:29.167861+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:20.316426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:20.316426+0000","last_clean_scrub_stamp":"2024-09-12T03:16:20.316426+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:12:15.424320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542196,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36892,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334948,"statfs":{"total":96636764160,"available":96598986752,"internally_reserved":0,"allocated":1576960,"data_stored":1009988,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542198,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36320,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335520,"statfs":{"total":96636764160,"available":96599572480,"internally_reserved":0,"allocated":991232,"data_stored":420244,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542197,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36336,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335504,"statfs":{"total":96636764160,"available":96599556096,"internally_reserved":0,"allocated":1007616,"data_stored":427484,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542196,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36316,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335524,"statfs":{"total":96636764160,"available":96599576576,"internally_reserved":0,"allocated":991232,"data_stored":425850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542197,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28004,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343836,"statfs":{"total":96636764160,"available":96608088064,"internally_reserved":0,"allocated":991232,"data_stored":421162,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542197,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27992,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343848,"statfs":{"total":96636764160,"available":96608100352,"internally_reserved":0,"allocated":983040,"data_stored":420132,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542198,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32172,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339668,"statfs":{"total":96636764160,"available":96603820032,"internally_reserved":0,"allocated":1003520,"data_stored":426044,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542198,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32728,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339112,"statfs":{"total":96636764160,"available":96603250688,"internally_reserved":0,"allocated":1576960,"data_stored":1009952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T03:20:32.152 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-12T03:20:32.290 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:32.290+0000 7f9621eac640 1 Processor -- start 2024-09-12T03:20:32.290 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:32.290+0000 7f9621eac640 1 -- start start 2024-09-12T03:20:32.291 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:32.291+0000 7f9621eac640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f961c158cf0 0x7f961c0b55d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:32.291 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:32.291+0000 7f9621eac640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f961c1590d0 0x7f961c162430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:32.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:32.291+0000 7f9621eac640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f961c162970 con 0x7f961c15f600 2024-09-12T03:20:32.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:32.291+0000 7f9621eac640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f961c162c10 con 0x7f961c1590d0 2024-09-12T03:20:32.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:32.291+0000 7f9621eac640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f961c162ef0 con 0x7f961c158cf0 2024-09-12T03:20:32.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:32.291+0000 7f961bfff640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f961c15f600 0x7f961c06bb30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:57688/0 (socket says 172.21.15.27:57688) 2024-09-12T03:20:32.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:32.291+0000 7f961affd640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f961c1590d0 0x7f961c162430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:32.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:32.291+0000 7f961bfff640 1 -- 172.21.15.27:0/2915718472 learned_addr learned my addr 172.21.15.27:0/2915718472 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:20:32.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:32.291+0000 7f961b7fe640 1 --2- 172.21.15.27:0/2915718472 >> v2:172.21.15.45:3300/0 conn(0x7f961c158cf0 0x7f961c0b55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:32.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:32.291+0000 7f961affd640 1 -- 172.21.15.27:0/2915718472 >> v2:172.21.15.45:3300/0 conn(0x7f961c158cf0 msgr2=0x7f961c0b55d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:32.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:32.292+0000 7f961affd640 1 --2- 172.21.15.27:0/2915718472 >> v2:172.21.15.45:3300/0 conn(0x7f961c158cf0 0x7f961c0b55d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:32.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:32.292+0000 7f961affd640 1 -- 172.21.15.27:0/2915718472 >> v1:172.21.15.27:6789/0 conn(0x7f961c15f600 legacy=0x7f961c06bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:32.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:32.292+0000 7f961affd640 1 -- 172.21.15.27:0/2915718472 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f961c1631d0 con 0x7f961c1590d0 2024-09-12T03:20:32.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:32.292+0000 7f961a7fc640 1 -- 172.21.15.27:0/2915718472 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2260602869 0 0) 0x7f961c162970 con 0x7f961c15f600 2024-09-12T03:20:32.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:32.292+0000 7f961affd640 1 --2- 172.21.15.27:0/2915718472 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f961c1590d0 0x7f961c162430 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f961c0733c0 tx=0x7f961002a080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8467cf616556f71 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:32.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:32.292+0000 7f961a7fc640 1 -- 172.21.15.27:0/2915718472 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9610037070 con 0x7f961c1590d0 2024-09-12T03:20:32.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:32.292+0000 7f961a7fc640 1 -- 172.21.15.27:0/2915718472 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f961002add0 con 0x7f961c1590d0 2024-09-12T03:20:32.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:32.292+0000 7f961a7fc640 1 -- 172.21.15.27:0/2915718472 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9610032660 con 0x7f961c1590d0 2024-09-12T03:20:32.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:32.293+0000 7f9621eac640 1 -- 172.21.15.27:0/2915718472 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f961c1590d0 msgr2=0x7f961c162430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:32.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:32.293+0000 7f9621eac640 1 --2- 172.21.15.27:0/2915718472 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f961c1590d0 0x7f961c162430 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f961c0733c0 tx=0x7f961002a080 comp rx=0 tx=0).stop 2024-09-12T03:20:32.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:32.293+0000 7f9621eac640 1 -- 172.21.15.27:0/2915718472 shutdown_connections 2024-09-12T03:20:32.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:32.293+0000 7f9621eac640 1 --2- 172.21.15.27:0/2915718472 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f961c1590d0 0x7f961c162430 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:32.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:32.293+0000 7f9621eac640 1 --2- 172.21.15.27:0/2915718472 >> v2:172.21.15.45:3300/0 conn(0x7f961c158cf0 0x7f961c0b55d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:32.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:32.293+0000 7f9621eac640 1 -- 172.21.15.27:0/2915718472 >> 172.21.15.27:0/2915718472 conn(0x7f961c06cc40 msgr2=0x7f961c06d050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:32.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:32.293+0000 7f9621eac640 1 -- 172.21.15.27:0/2915718472 shutdown_connections 2024-09-12T03:20:32.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:32.293+0000 7f9621eac640 1 -- 172.21.15.27:0/2915718472 wait complete. 2024-09-12T03:20:32.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:32.294+0000 7f9621eac640 1 Processor -- start 2024-09-12T03:20:32.296 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:32.294+0000 7f9621eac640 1 -- start start 2024-09-12T03:20:32.296 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:32.295+0000 7f9621eac640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f961c158cf0 0x7f961c15cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:32.297 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:32.295+0000 7f9621eac640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f961c1590d0 0x7f961c15e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:32.297 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:32.295+0000 7f9621eac640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f961c15f600 0x7f961c153370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:32.297 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:32.295+0000 7f9621eac640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f961c06c3c0 con 0x7f961c158cf0 2024-09-12T03:20:32.297 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:32.295+0000 7f9621eac640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f961c06c240 con 0x7f961c15f600 2024-09-12T03:20:32.297 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:32.295+0000 7f9621eac640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f961c15fa70 con 0x7f961c1590d0 2024-09-12T03:20:32.297 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:32.295+0000 7f961b7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f961c158cf0 0x7f961c15cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:32.297 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:32.295+0000 7f961b7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f961c158cf0 0x7f961c15cd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46976/0 (socket says 172.21.15.27:46976) 2024-09-12T03:20:32.297 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:32.295+0000 7f961b7fe640 1 -- 172.21.15.27:0/2001321697 learned_addr learned my addr 172.21.15.27:0/2001321697 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:20:32.297 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:32.296+0000 7f961b7fe640 1 -- 172.21.15.27:0/2001321697 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f961c1590d0 msgr2=0x7f961c15e3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:32.297 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:32.296+0000 7f961b7fe640 1 --2- 172.21.15.27:0/2001321697 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f961c1590d0 0x7f961c15e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:32.297 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:32.296+0000 7f961bfff640 1 --2- 172.21.15.27:0/2001321697 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f961c15f600 0x7f961c153370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:32.297 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:32.296+0000 7f961b7fe640 1 -- 172.21.15.27:0/2001321697 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f961c15f600 msgr2=0x7f961c153370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:32.297 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:32.296+0000 7f961b7fe640 1 --2- 172.21.15.27:0/2001321697 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f961c15f600 0x7f961c153370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:32.298 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:32.296+0000 7f961b7fe640 1 -- 172.21.15.27:0/2001321697 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f961c15d330 con 0x7f961c158cf0 2024-09-12T03:20:32.298 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:32.296+0000 7f961b7fe640 1 --2- 172.21.15.27:0/2001321697 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f961c158cf0 0x7f961c15cd50 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f960c00d800 tx=0x7f960c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:32.298 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:32.296+0000 7f9603fff640 1 -- 172.21.15.27:0/2001321697 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f960c00b6c0 con 0x7f961c158cf0 2024-09-12T03:20:32.298 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:32.296+0000 7f9603fff640 1 -- 172.21.15.27:0/2001321697 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f960c019070 con 0x7f961c158cf0 2024-09-12T03:20:32.298 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:32.296+0000 7f9603fff640 1 -- 172.21.15.27:0/2001321697 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f960c00ba80 con 0x7f961c158cf0 2024-09-12T03:20:32.298 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:32.296+0000 7f9621eac640 1 -- 172.21.15.27:0/2001321697 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f961c15d610 con 0x7f961c158cf0 2024-09-12T03:20:32.298 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:32.296+0000 7f9621eac640 1 -- 172.21.15.27:0/2001321697 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f961c15dbd0 con 0x7f961c158cf0 2024-09-12T03:20:32.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:32.297+0000 7f9621eac640 1 -- 172.21.15.27:0/2001321697 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f961c0b55d0 con 0x7f961c158cf0 2024-09-12T03:20:32.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:32.297+0000 7f9603fff640 1 -- 172.21.15.27:0/2001321697 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 90600+0+0 (secure 0 0 0) 0x7f960c00bc20 con 0x7f961c158cf0 2024-09-12T03:20:32.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:32.298+0000 7f9603fff640 1 --2- 172.21.15.27:0/2001321697 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f960806bf50 0x7f960806e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:32.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:32.298+0000 7f9603fff640 1 -- 172.21.15.27:0/2001321697 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 5709+0+0 (secure 0 0 0) 0x7f960c08b3d0 con 0x7f961c158cf0 2024-09-12T03:20:32.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:32.298+0000 7f961affd640 1 --2- 172.21.15.27:0/2001321697 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f960806bf50 0x7f960806e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:32.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:32.298+0000 7f961affd640 1 --2- 172.21.15.27:0/2001321697 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f960806bf50 0x7f960806e410 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f9610034000 tx=0x7f961002c000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:32.302 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:32.302+0000 7f9603fff640 1 -- 172.21.15.27:0/2001321697 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f960c059d00 con 0x7f961c158cf0 2024-09-12T03:20:32.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:32.475+0000 7f9621eac640 1 -- 172.21.15.27:0/2001321697 --> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f961c0d0f40 con 0x7f960806bf50 2024-09-12T03:20:32.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:32.481+0000 7f9603fff640 1 -- 172.21.15.27:0/2001321697 <== mgr.4113 v2:172.21.15.27:6836/1226920526 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252049 (secure 0 0 0) 0x7f961c0d0f40 con 0x7f960806bf50 2024-09-12T03:20:32.484 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:20:32.485 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2024-09-12T03:20:32.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:32.484+0000 7f9621eac640 1 -- 172.21.15.27:0/2001321697 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f960806bf50 msgr2=0x7f960806e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:32.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:32.484+0000 7f9621eac640 1 --2- 172.21.15.27:0/2001321697 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f960806bf50 0x7f960806e410 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f9610034000 tx=0x7f961002c000 comp rx=0 tx=0).stop 2024-09-12T03:20:32.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:32.484+0000 7f9621eac640 1 -- 172.21.15.27:0/2001321697 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f961c158cf0 msgr2=0x7f961c15cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:32.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:32.484+0000 7f9621eac640 1 --2- 172.21.15.27:0/2001321697 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f961c158cf0 0x7f961c15cd50 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f960c00d800 tx=0x7f960c00dcc0 comp rx=0 tx=0).stop 2024-09-12T03:20:32.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:32.484+0000 7f9621eac640 1 -- 172.21.15.27:0/2001321697 shutdown_connections 2024-09-12T03:20:32.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:32.484+0000 7f9621eac640 1 --2- 172.21.15.27:0/2001321697 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f961c15f600 0x7f961c153370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:32.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:32.484+0000 7f9621eac640 1 --2- 172.21.15.27:0/2001321697 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f961c1590d0 0x7f961c15e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:32.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:32.484+0000 7f9621eac640 1 --2- 172.21.15.27:0/2001321697 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f960806bf50 0x7f960806e410 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:32.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:32.484+0000 7f9621eac640 1 --2- 172.21.15.27:0/2001321697 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f961c158cf0 0x7f961c15cd50 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:32.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:32.484+0000 7f9621eac640 1 -- 172.21.15.27:0/2001321697 >> 172.21.15.27:0/2001321697 conn(0x7f961c06cc40 msgr2=0x7f961c06bb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:32.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:32.485+0000 7f9621eac640 1 -- 172.21.15.27:0/2001321697 shutdown_connections 2024-09-12T03:20:32.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:32.485+0000 7f9621eac640 1 -- 172.21.15.27:0/2001321697 wait complete. 2024-09-12T03:20:32.496 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2024-09-12T03:20:31.097175+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":223,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":166,"ondisk_log_size":166,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":126,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":266760,"kb_used_data":8908,"kb_used_omap":12,"kb_used_meta":257651,"kb_avail":754707960,"statfs":{"total":773094113280,"available":772820951040,"internally_reserved":0,"allocated":9121792,"data_stored":4560856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":263835184},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000982"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-12T03:20:30.786784+0000","last_change":"2024-09-12T03:20:29.177288+0000","last_active":"2024-09-12T03:20:30.786784+0000","last_peered":"2024-09-12T03:20:30.786784+0000","last_clean":"2024-09-12T03:20:30.786784+0000","last_became_active":"2024-09-12T03:20:29.176231+0000","last_became_peered":"2024-09-12T03:20:29.176231+0000","last_unstale":"2024-09-12T03:20:30.786784+0000","last_undegraded":"2024-09-12T03:20:30.786784+0000","last_fullsized":"2024-09-12T03:20:30.786784+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_clean_scrub_stamp":"2024-09-12T03:20:28.136653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:02:27.543548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-12T03:20:30.174970+0000","last_change":"2024-09-12T03:20:29.177451+0000","last_active":"2024-09-12T03:20:30.174970+0000","last_peered":"2024-09-12T03:20:30.174970+0000","last_clean":"2024-09-12T03:20:30.174970+0000","last_became_active":"2024-09-12T03:20:29.176831+0000","last_became_peered":"2024-09-12T03:20:29.176831+0000","last_unstale":"2024-09-12T03:20:30.174970+0000","last_undegraded":"2024-09-12T03:20:30.174970+0000","last_fullsized":"2024-09-12T03:20:30.174970+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_clean_scrub_stamp":"2024-09-12T03:20:28.136653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:51:51.184680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:20:29.178319+0000","last_change":"2024-09-12T03:20:29.176354+0000","last_active":"2024-09-12T03:20:29.178319+0000","last_peered":"2024-09-12T03:20:29.178319+0000","last_clean":"2024-09-12T03:20:29.178319+0000","last_became_active":"2024-09-12T03:20:29.176060+0000","last_became_peered":"2024-09-12T03:20:29.176060+0000","last_unstale":"2024-09-12T03:20:29.178319+0000","last_undegraded":"2024-09-12T03:20:29.178319+0000","last_fullsized":"2024-09-12T03:20:29.178319+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_clean_scrub_stamp":"2024-09-12T03:20:28.136653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:20:39.549097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:20:29.192407+0000","last_change":"2024-09-12T03:20:29.187883+0000","last_active":"2024-09-12T03:20:29.192407+0000","last_peered":"2024-09-12T03:20:29.192407+0000","last_clean":"2024-09-12T03:20:29.192407+0000","last_became_active":"2024-09-12T03:20:29.187594+0000","last_became_peered":"2024-09-12T03:20:29.187594+0000","last_unstale":"2024-09-12T03:20:29.192407+0000","last_undegraded":"2024-09-12T03:20:29.192407+0000","last_fullsized":"2024-09-12T03:20:29.192407+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_clean_scrub_stamp":"2024-09-12T03:20:28.136653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:26:26.772575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-12T03:20:30.176147+0000","last_change":"2024-09-12T03:20:29.186478+0000","last_active":"2024-09-12T03:20:30.176147+0000","last_peered":"2024-09-12T03:20:30.176147+0000","last_clean":"2024-09-12T03:20:30.176147+0000","last_became_active":"2024-09-12T03:20:29.186196+0000","last_became_peered":"2024-09-12T03:20:29.186196+0000","last_unstale":"2024-09-12T03:20:30.176147+0000","last_undegraded":"2024-09-12T03:20:30.176147+0000","last_fullsized":"2024-09-12T03:20:30.176147+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_clean_scrub_stamp":"2024-09-12T03:20:28.136653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:17:42.735036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-12T03:20:30.175097+0000","last_change":"2024-09-12T03:20:29.178733+0000","last_active":"2024-09-12T03:20:30.175097+0000","last_peered":"2024-09-12T03:20:30.175097+0000","last_clean":"2024-09-12T03:20:30.175097+0000","last_became_active":"2024-09-12T03:20:29.178419+0000","last_became_peered":"2024-09-12T03:20:29.178419+0000","last_unstale":"2024-09-12T03:20:30.175097+0000","last_undegraded":"2024-09-12T03:20:30.175097+0000","last_fullsized":"2024-09-12T03:20:30.175097+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_clean_scrub_stamp":"2024-09-12T03:20:28.136653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:08:21.425269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:20:29.182198+0000","last_change":"2024-09-12T03:20:29.178336+0000","last_active":"2024-09-12T03:20:29.182198+0000","last_peered":"2024-09-12T03:20:29.182198+0000","last_clean":"2024-09-12T03:20:29.182198+0000","last_became_active":"2024-09-12T03:20:29.177344+0000","last_became_peered":"2024-09-12T03:20:29.177344+0000","last_unstale":"2024-09-12T03:20:29.182198+0000","last_undegraded":"2024-09-12T03:20:29.182198+0000","last_fullsized":"2024-09-12T03:20:29.182198+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_clean_scrub_stamp":"2024-09-12T03:20:28.136653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:39:33.196052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-12T03:20:30.175436+0000","last_change":"2024-09-12T03:20:29.179643+0000","last_active":"2024-09-12T03:20:30.175436+0000","last_peered":"2024-09-12T03:20:30.175436+0000","last_clean":"2024-09-12T03:20:30.175436+0000","last_became_active":"2024-09-12T03:20:29.179313+0000","last_became_peered":"2024-09-12T03:20:29.179313+0000","last_unstale":"2024-09-12T03:20:30.175436+0000","last_undegraded":"2024-09-12T03:20:30.175436+0000","last_fullsized":"2024-09-12T03:20:30.175436+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_clean_scrub_stamp":"2024-09-12T03:20:28.136653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:14:47.509935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-12T03:20:30.175154+0000","last_change":"2024-09-12T03:20:29.183146+0000","last_active":"2024-09-12T03:20:30.175154+0000","last_peered":"2024-09-12T03:20:30.175154+0000","last_clean":"2024-09-12T03:20:30.175154+0000","last_became_active":"2024-09-12T03:20:29.182856+0000","last_became_peered":"2024-09-12T03:20:29.182856+0000","last_unstale":"2024-09-12T03:20:30.175154+0000","last_undegraded":"2024-09-12T03:20:30.175154+0000","last_fullsized":"2024-09-12T03:20:30.175154+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_clean_scrub_stamp":"2024-09-12T03:20:28.136653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:51:36.544538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-12T03:20:30.786601+0000","last_change":"2024-09-12T03:20:29.178507+0000","last_active":"2024-09-12T03:20:30.786601+0000","last_peered":"2024-09-12T03:20:30.786601+0000","last_clean":"2024-09-12T03:20:30.786601+0000","last_became_active":"2024-09-12T03:20:29.178202+0000","last_became_peered":"2024-09-12T03:20:29.178202+0000","last_unstale":"2024-09-12T03:20:30.786601+0000","last_undegraded":"2024-09-12T03:20:30.786601+0000","last_fullsized":"2024-09-12T03:20:30.786601+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_clean_scrub_stamp":"2024-09-12T03:20:28.136653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:17:42.735036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-12T03:20:30.174801+0000","last_change":"2024-09-12T03:20:29.180016+0000","last_active":"2024-09-12T03:20:30.174801+0000","last_peered":"2024-09-12T03:20:30.174801+0000","last_clean":"2024-09-12T03:20:30.174801+0000","last_became_active":"2024-09-12T03:20:29.179428+0000","last_became_peered":"2024-09-12T03:20:29.179428+0000","last_unstale":"2024-09-12T03:20:30.174801+0000","last_undegraded":"2024-09-12T03:20:30.174801+0000","last_fullsized":"2024-09-12T03:20:30.174801+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_clean_scrub_stamp":"2024-09-12T03:20:28.136653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:02:05.748037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-12T03:20:30.786307+0000","last_change":"2024-09-12T03:20:29.174451+0000","last_active":"2024-09-12T03:20:30.786307+0000","last_peered":"2024-09-12T03:20:30.786307+0000","last_clean":"2024-09-12T03:20:30.786307+0000","last_became_active":"2024-09-12T03:20:29.173679+0000","last_became_peered":"2024-09-12T03:20:29.173679+0000","last_unstale":"2024-09-12T03:20:30.786307+0000","last_undegraded":"2024-09-12T03:20:30.786307+0000","last_fullsized":"2024-09-12T03:20:30.786307+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_clean_scrub_stamp":"2024-09-12T03:20:28.136653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:10:24.781478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:20:29.177573+0000","last_change":"2024-09-12T03:20:29.174921+0000","last_active":"2024-09-12T03:20:29.177573+0000","last_peered":"2024-09-12T03:20:29.177573+0000","last_clean":"2024-09-12T03:20:29.177573+0000","last_became_active":"2024-09-12T03:20:29.174603+0000","last_became_peered":"2024-09-12T03:20:29.174603+0000","last_unstale":"2024-09-12T03:20:29.177573+0000","last_undegraded":"2024-09-12T03:20:29.177573+0000","last_fullsized":"2024-09-12T03:20:29.177573+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_clean_scrub_stamp":"2024-09-12T03:20:28.136653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:10:28.631427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:20:29.192313+0000","last_change":"2024-09-12T03:20:29.187072+0000","last_active":"2024-09-12T03:20:29.192313+0000","last_peered":"2024-09-12T03:20:29.192313+0000","last_clean":"2024-09-12T03:20:29.192313+0000","last_became_active":"2024-09-12T03:20:29.186803+0000","last_became_peered":"2024-09-12T03:20:29.186803+0000","last_unstale":"2024-09-12T03:20:29.192313+0000","last_undegraded":"2024-09-12T03:20:29.192313+0000","last_fullsized":"2024-09-12T03:20:29.192313+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_clean_scrub_stamp":"2024-09-12T03:20:28.136653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:38:45.533952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-12T03:20:30.809930+0000","last_change":"2024-09-12T03:20:29.179532+0000","last_active":"2024-09-12T03:20:30.809930+0000","last_peered":"2024-09-12T03:20:30.809930+0000","last_clean":"2024-09-12T03:20:30.809930+0000","last_became_active":"2024-09-12T03:20:29.179262+0000","last_became_peered":"2024-09-12T03:20:29.179262+0000","last_unstale":"2024-09-12T03:20:30.809930+0000","last_undegraded":"2024-09-12T03:20:30.809930+0000","last_fullsized":"2024-09-12T03:20:30.809930+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_clean_scrub_stamp":"2024-09-12T03:20:28.136653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:50:43.423983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-12T03:20:30.175204+0000","last_change":"2024-09-12T03:20:29.170807+0000","last_active":"2024-09-12T03:20:30.175204+0000","last_peered":"2024-09-12T03:20:30.175204+0000","last_clean":"2024-09-12T03:20:30.175204+0000","last_became_active":"2024-09-12T03:20:29.170547+0000","last_became_peered":"2024-09-12T03:20:29.170547+0000","last_unstale":"2024-09-12T03:20:30.175204+0000","last_undegraded":"2024-09-12T03:20:30.175204+0000","last_fullsized":"2024-09-12T03:20:30.175204+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_clean_scrub_stamp":"2024-09-12T03:20:28.136653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:40:09.344605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-12T03:20:30.176131+0000","last_change":"2024-09-12T03:20:29.178480+0000","last_active":"2024-09-12T03:20:30.176131+0000","last_peered":"2024-09-12T03:20:30.176131+0000","last_clean":"2024-09-12T03:20:30.176131+0000","last_became_active":"2024-09-12T03:20:29.178229+0000","last_became_peered":"2024-09-12T03:20:29.178229+0000","last_unstale":"2024-09-12T03:20:30.176131+0000","last_undegraded":"2024-09-12T03:20:30.176131+0000","last_fullsized":"2024-09-12T03:20:30.176131+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_clean_scrub_stamp":"2024-09-12T03:20:28.136653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:10:28.631427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:20:29.182872+0000","last_change":"2024-09-12T03:20:29.178792+0000","last_active":"2024-09-12T03:20:29.182872+0000","last_peered":"2024-09-12T03:20:29.182872+0000","last_clean":"2024-09-12T03:20:29.182872+0000","last_became_active":"2024-09-12T03:20:29.178508+0000","last_became_peered":"2024-09-12T03:20:29.178508+0000","last_unstale":"2024-09-12T03:20:29.182872+0000","last_undegraded":"2024-09-12T03:20:29.182872+0000","last_fullsized":"2024-09-12T03:20:29.182872+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_clean_scrub_stamp":"2024-09-12T03:20:28.136653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:57:54.991629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:20:29.186520+0000","last_change":"2024-09-12T03:20:29.182384+0000","last_active":"2024-09-12T03:20:29.186520+0000","last_peered":"2024-09-12T03:20:29.186520+0000","last_clean":"2024-09-12T03:20:29.186520+0000","last_became_active":"2024-09-12T03:20:29.181445+0000","last_became_peered":"2024-09-12T03:20:29.181445+0000","last_unstale":"2024-09-12T03:20:29.186520+0000","last_undegraded":"2024-09-12T03:20:29.186520+0000","last_fullsized":"2024-09-12T03:20:29.186520+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_clean_scrub_stamp":"2024-09-12T03:20:28.136653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:06:02.733007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:20:29.190699+0000","last_change":"2024-09-12T03:20:29.183131+0000","last_active":"2024-09-12T03:20:29.190699+0000","last_peered":"2024-09-12T03:20:29.190699+0000","last_clean":"2024-09-12T03:20:29.190699+0000","last_became_active":"2024-09-12T03:20:29.182837+0000","last_became_peered":"2024-09-12T03:20:29.182837+0000","last_unstale":"2024-09-12T03:20:29.190699+0000","last_undegraded":"2024-09-12T03:20:29.190699+0000","last_fullsized":"2024-09-12T03:20:29.190699+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_clean_scrub_stamp":"2024-09-12T03:20:28.136653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:45:07.868238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:20:29.192441+0000","last_change":"2024-09-12T03:20:29.187883+0000","last_active":"2024-09-12T03:20:29.192441+0000","last_peered":"2024-09-12T03:20:29.192441+0000","last_clean":"2024-09-12T03:20:29.192441+0000","last_became_active":"2024-09-12T03:20:29.187616+0000","last_became_peered":"2024-09-12T03:20:29.187616+0000","last_unstale":"2024-09-12T03:20:29.192441+0000","last_undegraded":"2024-09-12T03:20:29.192441+0000","last_fullsized":"2024-09-12T03:20:29.192441+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_clean_scrub_stamp":"2024-09-12T03:20:28.136653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:12:09.993191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-12T03:20:30.174377+0000","last_change":"2024-09-12T03:20:29.189822+0000","last_active":"2024-09-12T03:20:30.174377+0000","last_peered":"2024-09-12T03:20:30.174377+0000","last_clean":"2024-09-12T03:20:30.174377+0000","last_became_active":"2024-09-12T03:20:29.189582+0000","last_became_peered":"2024-09-12T03:20:29.189582+0000","last_unstale":"2024-09-12T03:20:30.174377+0000","last_undegraded":"2024-09-12T03:20:30.174377+0000","last_fullsized":"2024-09-12T03:20:30.174377+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_clean_scrub_stamp":"2024-09-12T03:20:28.136653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:41:35.802098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-12T03:20:30.786223+0000","last_change":"2024-09-12T03:20:29.188227+0000","last_active":"2024-09-12T03:20:30.786223+0000","last_peered":"2024-09-12T03:20:30.786223+0000","last_clean":"2024-09-12T03:20:30.786223+0000","last_became_active":"2024-09-12T03:20:29.187910+0000","last_became_peered":"2024-09-12T03:20:29.187910+0000","last_unstale":"2024-09-12T03:20:30.786223+0000","last_undegraded":"2024-09-12T03:20:30.786223+0000","last_fullsized":"2024-09-12T03:20:30.786223+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_clean_scrub_stamp":"2024-09-12T03:20:28.136653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:19:22.773136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:20:29.192450+0000","last_change":"2024-09-12T03:20:29.188417+0000","last_active":"2024-09-12T03:20:29.192450+0000","last_peered":"2024-09-12T03:20:29.192450+0000","last_clean":"2024-09-12T03:20:29.192450+0000","last_became_active":"2024-09-12T03:20:29.188157+0000","last_became_peered":"2024-09-12T03:20:29.188157+0000","last_unstale":"2024-09-12T03:20:29.192450+0000","last_undegraded":"2024-09-12T03:20:29.192450+0000","last_fullsized":"2024-09-12T03:20:29.192450+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_clean_scrub_stamp":"2024-09-12T03:20:28.136653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:04:05.851766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-12T03:20:30.174471+0000","last_change":"2024-09-12T03:20:29.176464+0000","last_active":"2024-09-12T03:20:30.174471+0000","last_peered":"2024-09-12T03:20:30.174471+0000","last_clean":"2024-09-12T03:20:30.174471+0000","last_became_active":"2024-09-12T03:20:29.176182+0000","last_became_peered":"2024-09-12T03:20:29.176182+0000","last_unstale":"2024-09-12T03:20:30.174471+0000","last_undegraded":"2024-09-12T03:20:30.174471+0000","last_fullsized":"2024-09-12T03:20:30.174471+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_clean_scrub_stamp":"2024-09-12T03:20:28.136653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:31:35.182563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:20:29.188272+0000","last_change":"2024-09-12T03:20:29.187228+0000","last_active":"2024-09-12T03:20:29.188272+0000","last_peered":"2024-09-12T03:20:29.188272+0000","last_clean":"2024-09-12T03:20:29.188272+0000","last_became_active":"2024-09-12T03:20:29.186784+0000","last_became_peered":"2024-09-12T03:20:29.186784+0000","last_unstale":"2024-09-12T03:20:29.188272+0000","last_undegraded":"2024-09-12T03:20:29.188272+0000","last_fullsized":"2024-09-12T03:20:29.188272+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_clean_scrub_stamp":"2024-09-12T03:20:28.136653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:41:29.086502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-12T03:20:30.175994+0000","last_change":"2024-09-12T03:20:29.173152+0000","last_active":"2024-09-12T03:20:30.175994+0000","last_peered":"2024-09-12T03:20:30.175994+0000","last_clean":"2024-09-12T03:20:30.175994+0000","last_became_active":"2024-09-12T03:20:29.172710+0000","last_became_peered":"2024-09-12T03:20:29.172710+0000","last_unstale":"2024-09-12T03:20:30.175994+0000","last_undegraded":"2024-09-12T03:20:30.175994+0000","last_fullsized":"2024-09-12T03:20:30.175994+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_clean_scrub_stamp":"2024-09-12T03:20:28.136653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:10:24.781478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-12T03:20:30.786410+0000","last_change":"2024-09-12T03:20:29.175081+0000","last_active":"2024-09-12T03:20:30.786410+0000","last_peered":"2024-09-12T03:20:30.786410+0000","last_clean":"2024-09-12T03:20:30.786410+0000","last_became_active":"2024-09-12T03:20:29.174776+0000","last_became_peered":"2024-09-12T03:20:29.174776+0000","last_unstale":"2024-09-12T03:20:30.786410+0000","last_undegraded":"2024-09-12T03:20:30.786410+0000","last_fullsized":"2024-09-12T03:20:30.786410+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_clean_scrub_stamp":"2024-09-12T03:20:28.136653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:10:28.631427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-12T03:20:30.174640+0000","last_change":"2024-09-12T03:20:29.168919+0000","last_active":"2024-09-12T03:20:30.174640+0000","last_peered":"2024-09-12T03:20:30.174640+0000","last_clean":"2024-09-12T03:20:30.174640+0000","last_became_active":"2024-09-12T03:20:29.168636+0000","last_became_peered":"2024-09-12T03:20:29.168636+0000","last_unstale":"2024-09-12T03:20:30.174640+0000","last_undegraded":"2024-09-12T03:20:30.174640+0000","last_fullsized":"2024-09-12T03:20:30.174640+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_clean_scrub_stamp":"2024-09-12T03:20:28.136653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:18:08.957359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-12T03:20:30.174402+0000","last_change":"2024-09-12T03:20:29.188821+0000","last_active":"2024-09-12T03:20:30.174402+0000","last_peered":"2024-09-12T03:20:30.174402+0000","last_clean":"2024-09-12T03:20:30.174402+0000","last_became_active":"2024-09-12T03:20:29.185747+0000","last_became_peered":"2024-09-12T03:20:29.185747+0000","last_unstale":"2024-09-12T03:20:30.174402+0000","last_undegraded":"2024-09-12T03:20:30.174402+0000","last_fullsized":"2024-09-12T03:20:30.174402+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_clean_scrub_stamp":"2024-09-12T03:20:28.136653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:19:34.760161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-12T03:20:30.175300+0000","last_change":"2024-09-12T03:20:29.185908+0000","last_active":"2024-09-12T03:20:30.175300+0000","last_peered":"2024-09-12T03:20:30.175300+0000","last_clean":"2024-09-12T03:20:30.175300+0000","last_became_active":"2024-09-12T03:20:29.185286+0000","last_became_peered":"2024-09-12T03:20:29.185286+0000","last_unstale":"2024-09-12T03:20:30.175300+0000","last_undegraded":"2024-09-12T03:20:30.175300+0000","last_fullsized":"2024-09-12T03:20:30.175300+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_clean_scrub_stamp":"2024-09-12T03:20:28.136653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:19:22.773136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-12T03:20:30.786114+0000","last_change":"2024-09-12T03:20:29.190215+0000","last_active":"2024-09-12T03:20:30.786114+0000","last_peered":"2024-09-12T03:20:30.786114+0000","last_clean":"2024-09-12T03:20:30.786114+0000","last_became_active":"2024-09-12T03:20:29.189872+0000","last_became_peered":"2024-09-12T03:20:29.189872+0000","last_unstale":"2024-09-12T03:20:30.786114+0000","last_undegraded":"2024-09-12T03:20:30.786114+0000","last_fullsized":"2024-09-12T03:20:30.786114+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_clean_scrub_stamp":"2024-09-12T03:20:28.136653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:40:09.344605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-12T03:20:30.176017+0000","last_change":"2024-09-12T03:20:29.170126+0000","last_active":"2024-09-12T03:20:30.176017+0000","last_peered":"2024-09-12T03:20:30.176017+0000","last_clean":"2024-09-12T03:20:30.176017+0000","last_became_active":"2024-09-12T03:20:29.169854+0000","last_became_peered":"2024-09-12T03:20:29.169854+0000","last_unstale":"2024-09-12T03:20:30.176017+0000","last_undegraded":"2024-09-12T03:20:30.176017+0000","last_fullsized":"2024-09-12T03:20:30.176017+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_clean_scrub_stamp":"2024-09-12T03:20:28.136653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:21:39.137804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"80'4","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-12T03:20:30.786872+0000","last_change":"2024-09-12T03:20:28.161710+0000","last_active":"2024-09-12T03:20:30.786872+0000","last_peered":"2024-09-12T03:20:30.786872+0000","last_clean":"2024-09-12T03:20:30.786872+0000","last_became_active":"2024-09-12T03:20:28.161404+0000","last_became_peered":"2024-09-12T03:20:28.161404+0000","last_unstale":"2024-09-12T03:20:30.786872+0000","last_undegraded":"2024-09-12T03:20:30.786872+0000","last_fullsized":"2024-09-12T03:20:30.786872+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:27.129547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:27.129547+0000","last_clean_scrub_stamp":"2024-09-12T03:20:27.129547+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:51:10.529017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-12T03:20:30.786780+0000","last_change":"2024-09-12T03:20:29.187176+0000","last_active":"2024-09-12T03:20:30.786780+0000","last_peered":"2024-09-12T03:20:30.786780+0000","last_clean":"2024-09-12T03:20:30.786780+0000","last_became_active":"2024-09-12T03:20:29.186884+0000","last_became_peered":"2024-09-12T03:20:29.186884+0000","last_unstale":"2024-09-12T03:20:30.786780+0000","last_undegraded":"2024-09-12T03:20:30.786780+0000","last_fullsized":"2024-09-12T03:20:30.786780+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_clean_scrub_stamp":"2024-09-12T03:20:28.136653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:03:04.880817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"80'1","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-12T03:20:30.176341+0000","last_change":"2024-09-12T03:20:28.154730+0000","last_active":"2024-09-12T03:20:30.176341+0000","last_peered":"2024-09-12T03:20:30.176341+0000","last_clean":"2024-09-12T03:20:30.176341+0000","last_became_active":"2024-09-12T03:20:28.154397+0000","last_became_peered":"2024-09-12T03:20:28.154397+0000","last_unstale":"2024-09-12T03:20:30.176341+0000","last_undegraded":"2024-09-12T03:20:30.176341+0000","last_fullsized":"2024-09-12T03:20:30.176341+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:27.129547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:27.129547+0000","last_clean_scrub_stamp":"2024-09-12T03:20:27.129547+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:39:00.949711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-12T03:20:30.786599+0000","last_change":"2024-09-12T03:20:29.186446+0000","last_active":"2024-09-12T03:20:30.786599+0000","last_peered":"2024-09-12T03:20:30.786599+0000","last_clean":"2024-09-12T03:20:30.786599+0000","last_became_active":"2024-09-12T03:20:29.186146+0000","last_became_peered":"2024-09-12T03:20:29.186146+0000","last_unstale":"2024-09-12T03:20:30.786599+0000","last_undegraded":"2024-09-12T03:20:30.786599+0000","last_fullsized":"2024-09-12T03:20:30.786599+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_clean_scrub_stamp":"2024-09-12T03:20:28.136653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:00:06.471862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"80'2","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:20:29.183673+0000","last_change":"2024-09-12T03:20:28.161680+0000","last_active":"2024-09-12T03:20:29.183673+0000","last_peered":"2024-09-12T03:20:29.183673+0000","last_clean":"2024-09-12T03:20:29.183673+0000","last_became_active":"2024-09-12T03:20:28.161376+0000","last_became_peered":"2024-09-12T03:20:28.161376+0000","last_unstale":"2024-09-12T03:20:29.183673+0000","last_undegraded":"2024-09-12T03:20:29.183673+0000","last_fullsized":"2024-09-12T03:20:29.183673+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:27.129547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:27.129547+0000","last_clean_scrub_stamp":"2024-09-12T03:20:27.129547+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:24:19.679944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:20:29.184698+0000","last_change":"2024-09-12T03:20:29.183539+0000","last_active":"2024-09-12T03:20:29.184698+0000","last_peered":"2024-09-12T03:20:29.184698+0000","last_clean":"2024-09-12T03:20:29.184698+0000","last_became_active":"2024-09-12T03:20:29.183255+0000","last_became_peered":"2024-09-12T03:20:29.183255+0000","last_unstale":"2024-09-12T03:20:29.184698+0000","last_undegraded":"2024-09-12T03:20:29.184698+0000","last_fullsized":"2024-09-12T03:20:29.184698+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_clean_scrub_stamp":"2024-09-12T03:20:28.136653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:02:27.543548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"80'2","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:20:29.186388+0000","last_change":"2024-09-12T03:20:28.161569+0000","last_active":"2024-09-12T03:20:29.186388+0000","last_peered":"2024-09-12T03:20:29.186388+0000","last_clean":"2024-09-12T03:20:29.186388+0000","last_became_active":"2024-09-12T03:20:28.161341+0000","last_became_peered":"2024-09-12T03:20:28.161341+0000","last_unstale":"2024-09-12T03:20:29.186388+0000","last_undegraded":"2024-09-12T03:20:29.186388+0000","last_fullsized":"2024-09-12T03:20:29.186388+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:27.129547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:27.129547+0000","last_clean_scrub_stamp":"2024-09-12T03:20:27.129547+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:03:03.873710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-12T03:20:30.174508+0000","last_change":"2024-09-12T03:20:29.185780+0000","last_active":"2024-09-12T03:20:30.174508+0000","last_peered":"2024-09-12T03:20:30.174508+0000","last_clean":"2024-09-12T03:20:30.174508+0000","last_became_active":"2024-09-12T03:20:29.185359+0000","last_became_peered":"2024-09-12T03:20:29.185359+0000","last_unstale":"2024-09-12T03:20:30.174508+0000","last_undegraded":"2024-09-12T03:20:30.174508+0000","last_fullsized":"2024-09-12T03:20:30.174508+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_clean_scrub_stamp":"2024-09-12T03:20:28.136653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:40:55.997920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"80'1","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-12T03:20:30.176484+0000","last_change":"2024-09-12T03:20:28.158989+0000","last_active":"2024-09-12T03:20:30.176484+0000","last_peered":"2024-09-12T03:20:30.176484+0000","last_clean":"2024-09-12T03:20:30.176484+0000","last_became_active":"2024-09-12T03:20:28.158665+0000","last_became_peered":"2024-09-12T03:20:28.158665+0000","last_unstale":"2024-09-12T03:20:30.176484+0000","last_undegraded":"2024-09-12T03:20:30.176484+0000","last_fullsized":"2024-09-12T03:20:30.176484+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:27.129547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:27.129547+0000","last_clean_scrub_stamp":"2024-09-12T03:20:27.129547+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:41:56.345114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-12T03:20:30.176325+0000","last_change":"2024-09-12T03:20:29.182394+0000","last_active":"2024-09-12T03:20:30.176325+0000","last_peered":"2024-09-12T03:20:30.176325+0000","last_clean":"2024-09-12T03:20:30.176325+0000","last_became_active":"2024-09-12T03:20:29.182107+0000","last_became_peered":"2024-09-12T03:20:29.182107+0000","last_unstale":"2024-09-12T03:20:30.176325+0000","last_undegraded":"2024-09-12T03:20:30.176325+0000","last_fullsized":"2024-09-12T03:20:30.176325+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_clean_scrub_stamp":"2024-09-12T03:20:28.136653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:03:04.880817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"80'1","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-12T03:20:30.175369+0000","last_change":"2024-09-12T03:20:28.153237+0000","last_active":"2024-09-12T03:20:30.175369+0000","last_peered":"2024-09-12T03:20:30.175369+0000","last_clean":"2024-09-12T03:20:30.175369+0000","last_became_active":"2024-09-12T03:20:28.152963+0000","last_became_peered":"2024-09-12T03:20:28.152963+0000","last_unstale":"2024-09-12T03:20:30.175369+0000","last_undegraded":"2024-09-12T03:20:30.175369+0000","last_fullsized":"2024-09-12T03:20:30.175369+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:27.129547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:27.129547+0000","last_clean_scrub_stamp":"2024-09-12T03:20:27.129547+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:31:36.558882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:20:29.187754+0000","last_change":"2024-09-12T03:20:29.185810+0000","last_active":"2024-09-12T03:20:29.187754+0000","last_peered":"2024-09-12T03:20:29.187754+0000","last_clean":"2024-09-12T03:20:29.187754+0000","last_became_active":"2024-09-12T03:20:29.185507+0000","last_became_peered":"2024-09-12T03:20:29.185507+0000","last_unstale":"2024-09-12T03:20:29.187754+0000","last_undegraded":"2024-09-12T03:20:29.187754+0000","last_fullsized":"2024-09-12T03:20:29.187754+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_clean_scrub_stamp":"2024-09-12T03:20:28.136653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:55:30.198372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:20:29.174327+0000","last_change":"2024-09-12T03:20:28.156112+0000","last_active":"2024-09-12T03:20:29.174327+0000","last_peered":"2024-09-12T03:20:29.174327+0000","last_clean":"2024-09-12T03:20:29.174327+0000","last_became_active":"2024-09-12T03:20:28.155826+0000","last_became_peered":"2024-09-12T03:20:28.155826+0000","last_unstale":"2024-09-12T03:20:29.174327+0000","last_undegraded":"2024-09-12T03:20:29.174327+0000","last_fullsized":"2024-09-12T03:20:29.174327+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:27.129547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:27.129547+0000","last_clean_scrub_stamp":"2024-09-12T03:20:27.129547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:12:11.478865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-12T03:20:30.809964+0000","last_change":"2024-09-12T03:20:29.178853+0000","last_active":"2024-09-12T03:20:30.809964+0000","last_peered":"2024-09-12T03:20:30.809964+0000","last_clean":"2024-09-12T03:20:30.809964+0000","last_became_active":"2024-09-12T03:20:29.178567+0000","last_became_peered":"2024-09-12T03:20:29.178567+0000","last_unstale":"2024-09-12T03:20:30.809964+0000","last_undegraded":"2024-09-12T03:20:30.809964+0000","last_fullsized":"2024-09-12T03:20:30.809964+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_clean_scrub_stamp":"2024-09-12T03:20:28.136653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:06:02.733007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-12T03:20:30.174313+0000","last_change":"2024-09-12T03:20:28.158205+0000","last_active":"2024-09-12T03:20:30.174313+0000","last_peered":"2024-09-12T03:20:30.174313+0000","last_clean":"2024-09-12T03:20:30.174313+0000","last_became_active":"2024-09-12T03:20:28.157948+0000","last_became_peered":"2024-09-12T03:20:28.157948+0000","last_unstale":"2024-09-12T03:20:30.174313+0000","last_undegraded":"2024-09-12T03:20:30.174313+0000","last_fullsized":"2024-09-12T03:20:30.174313+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:27.129547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:27.129547+0000","last_clean_scrub_stamp":"2024-09-12T03:20:27.129547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:34:49.505569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-12T03:20:30.810190+0000","last_change":"2024-09-12T03:20:29.180546+0000","last_active":"2024-09-12T03:20:30.810190+0000","last_peered":"2024-09-12T03:20:30.810190+0000","last_clean":"2024-09-12T03:20:30.810190+0000","last_became_active":"2024-09-12T03:20:29.180269+0000","last_became_peered":"2024-09-12T03:20:29.180269+0000","last_unstale":"2024-09-12T03:20:30.810190+0000","last_undegraded":"2024-09-12T03:20:30.810190+0000","last_fullsized":"2024-09-12T03:20:30.810190+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_clean_scrub_stamp":"2024-09-12T03:20:28.136653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:55:11.908871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-12T03:20:30.786935+0000","last_change":"2024-09-12T03:20:28.160028+0000","last_active":"2024-09-12T03:20:30.786935+0000","last_peered":"2024-09-12T03:20:30.786935+0000","last_clean":"2024-09-12T03:20:30.786935+0000","last_became_active":"2024-09-12T03:20:28.159688+0000","last_became_peered":"2024-09-12T03:20:28.159688+0000","last_unstale":"2024-09-12T03:20:30.786935+0000","last_undegraded":"2024-09-12T03:20:30.786935+0000","last_fullsized":"2024-09-12T03:20:30.786935+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:27.129547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:27.129547+0000","last_clean_scrub_stamp":"2024-09-12T03:20:27.129547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:25:20.650913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-12T03:20:30.175336+0000","last_change":"2024-09-12T03:20:29.178739+0000","last_active":"2024-09-12T03:20:30.175336+0000","last_peered":"2024-09-12T03:20:30.175336+0000","last_clean":"2024-09-12T03:20:30.175336+0000","last_became_active":"2024-09-12T03:20:29.178409+0000","last_became_peered":"2024-09-12T03:20:29.178409+0000","last_unstale":"2024-09-12T03:20:30.175336+0000","last_undegraded":"2024-09-12T03:20:30.175336+0000","last_fullsized":"2024-09-12T03:20:30.175336+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_clean_scrub_stamp":"2024-09-12T03:20:28.136653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:15:12.656811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-12T03:20:30.786750+0000","last_change":"2024-09-12T03:20:28.160587+0000","last_active":"2024-09-12T03:20:30.786750+0000","last_peered":"2024-09-12T03:20:30.786750+0000","last_clean":"2024-09-12T03:20:30.786750+0000","last_became_active":"2024-09-12T03:20:28.160114+0000","last_became_peered":"2024-09-12T03:20:28.160114+0000","last_unstale":"2024-09-12T03:20:30.786750+0000","last_undegraded":"2024-09-12T03:20:30.786750+0000","last_fullsized":"2024-09-12T03:20:30.786750+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:27.129547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:27.129547+0000","last_clean_scrub_stamp":"2024-09-12T03:20:27.129547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:45:52.294635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:20:29.186201+0000","last_change":"2024-09-12T03:20:29.181211+0000","last_active":"2024-09-12T03:20:29.186201+0000","last_peered":"2024-09-12T03:20:29.186201+0000","last_clean":"2024-09-12T03:20:29.186201+0000","last_became_active":"2024-09-12T03:20:29.180955+0000","last_became_peered":"2024-09-12T03:20:29.180955+0000","last_unstale":"2024-09-12T03:20:29.186201+0000","last_undegraded":"2024-09-12T03:20:29.186201+0000","last_fullsized":"2024-09-12T03:20:29.186201+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_clean_scrub_stamp":"2024-09-12T03:20:28.136653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:50:43.423983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"80'3","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-12T03:20:30.786361+0000","last_change":"2024-09-12T03:20:28.153383+0000","last_active":"2024-09-12T03:20:30.786361+0000","last_peered":"2024-09-12T03:20:30.786361+0000","last_clean":"2024-09-12T03:20:30.786361+0000","last_became_active":"2024-09-12T03:20:28.153124+0000","last_became_peered":"2024-09-12T03:20:28.153124+0000","last_unstale":"2024-09-12T03:20:30.786361+0000","last_undegraded":"2024-09-12T03:20:30.786361+0000","last_fullsized":"2024-09-12T03:20:30.786361+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:27.129547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:27.129547+0000","last_clean_scrub_stamp":"2024-09-12T03:20:27.129547+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:59:28.672500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-12T03:20:30.176207+0000","last_change":"2024-09-12T03:20:29.181858+0000","last_active":"2024-09-12T03:20:30.176207+0000","last_peered":"2024-09-12T03:20:30.176207+0000","last_clean":"2024-09-12T03:20:30.176207+0000","last_became_active":"2024-09-12T03:20:29.181552+0000","last_became_peered":"2024-09-12T03:20:29.181552+0000","last_unstale":"2024-09-12T03:20:30.176207+0000","last_undegraded":"2024-09-12T03:20:30.176207+0000","last_fullsized":"2024-09-12T03:20:30.176207+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_clean_scrub_stamp":"2024-09-12T03:20:28.136653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:20:39.549097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"80'6","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-12T03:20:30.175840+0000","last_change":"2024-09-12T03:20:28.155799+0000","last_active":"2024-09-12T03:20:30.175840+0000","last_peered":"2024-09-12T03:20:30.175840+0000","last_clean":"2024-09-12T03:20:30.175840+0000","last_became_active":"2024-09-12T03:20:28.155517+0000","last_became_peered":"2024-09-12T03:20:28.155517+0000","last_unstale":"2024-09-12T03:20:30.175840+0000","last_undegraded":"2024-09-12T03:20:30.175840+0000","last_fullsized":"2024-09-12T03:20:30.175840+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:27.129547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:27.129547+0000","last_clean_scrub_stamp":"2024-09-12T03:20:27.129547+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:43:32.260885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:20:29.181931+0000","last_change":"2024-09-12T03:20:29.179290+0000","last_active":"2024-09-12T03:20:29.181931+0000","last_peered":"2024-09-12T03:20:29.181931+0000","last_clean":"2024-09-12T03:20:29.181931+0000","last_became_active":"2024-09-12T03:20:29.179000+0000","last_became_peered":"2024-09-12T03:20:29.179000+0000","last_unstale":"2024-09-12T03:20:29.181931+0000","last_undegraded":"2024-09-12T03:20:29.181931+0000","last_fullsized":"2024-09-12T03:20:29.181931+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_clean_scrub_stamp":"2024-09-12T03:20:28.136653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:37:37.651061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"80'2","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-12T03:20:30.174738+0000","last_change":"2024-09-12T03:20:28.160059+0000","last_active":"2024-09-12T03:20:30.174738+0000","last_peered":"2024-09-12T03:20:30.174738+0000","last_clean":"2024-09-12T03:20:30.174738+0000","last_became_active":"2024-09-12T03:20:28.159780+0000","last_became_peered":"2024-09-12T03:20:28.159780+0000","last_unstale":"2024-09-12T03:20:30.174738+0000","last_undegraded":"2024-09-12T03:20:30.174738+0000","last_fullsized":"2024-09-12T03:20:30.174738+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:27.129547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:27.129547+0000","last_clean_scrub_stamp":"2024-09-12T03:20:27.129547+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:06:12.007015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:20:29.187632+0000","last_change":"2024-09-12T03:20:29.185677+0000","last_active":"2024-09-12T03:20:29.187632+0000","last_peered":"2024-09-12T03:20:29.187632+0000","last_clean":"2024-09-12T03:20:29.187632+0000","last_became_active":"2024-09-12T03:20:29.185297+0000","last_became_peered":"2024-09-12T03:20:29.185297+0000","last_unstale":"2024-09-12T03:20:29.187632+0000","last_undegraded":"2024-09-12T03:20:29.187632+0000","last_fullsized":"2024-09-12T03:20:29.187632+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_clean_scrub_stamp":"2024-09-12T03:20:28.136653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:03:04.880817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-12T03:20:30.810123+0000","last_change":"2024-09-12T03:20:28.154261+0000","last_active":"2024-09-12T03:20:30.810123+0000","last_peered":"2024-09-12T03:20:30.810123+0000","last_clean":"2024-09-12T03:20:30.810123+0000","last_became_active":"2024-09-12T03:20:28.153194+0000","last_became_peered":"2024-09-12T03:20:28.153194+0000","last_unstale":"2024-09-12T03:20:30.810123+0000","last_undegraded":"2024-09-12T03:20:30.810123+0000","last_fullsized":"2024-09-12T03:20:30.810123+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:27.129547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:27.129547+0000","last_clean_scrub_stamp":"2024-09-12T03:20:27.129547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:36:18.476441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-12T03:20:30.174300+0000","last_change":"2024-09-12T03:20:29.188626+0000","last_active":"2024-09-12T03:20:30.174300+0000","last_peered":"2024-09-12T03:20:30.174300+0000","last_clean":"2024-09-12T03:20:30.174300+0000","last_became_active":"2024-09-12T03:20:29.186136+0000","last_became_peered":"2024-09-12T03:20:29.186136+0000","last_unstale":"2024-09-12T03:20:30.174300+0000","last_undegraded":"2024-09-12T03:20:30.174300+0000","last_fullsized":"2024-09-12T03:20:30.174300+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_clean_scrub_stamp":"2024-09-12T03:20:28.136653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:58:49.778272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-12T03:20:30.175618+0000","last_change":"2024-09-12T03:20:28.158821+0000","last_active":"2024-09-12T03:20:30.175618+0000","last_peered":"2024-09-12T03:20:30.175618+0000","last_clean":"2024-09-12T03:20:30.175618+0000","last_became_active":"2024-09-12T03:20:28.158517+0000","last_became_peered":"2024-09-12T03:20:28.158517+0000","last_unstale":"2024-09-12T03:20:30.175618+0000","last_undegraded":"2024-09-12T03:20:30.175618+0000","last_fullsized":"2024-09-12T03:20:30.175618+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:27.129547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:27.129547+0000","last_clean_scrub_stamp":"2024-09-12T03:20:27.129547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:12:58.865580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-12T03:20:30.174242+0000","last_change":"2024-09-12T03:20:29.176284+0000","last_active":"2024-09-12T03:20:30.174242+0000","last_peered":"2024-09-12T03:20:30.174242+0000","last_clean":"2024-09-12T03:20:30.174242+0000","last_became_active":"2024-09-12T03:20:29.175900+0000","last_became_peered":"2024-09-12T03:20:29.175900+0000","last_unstale":"2024-09-12T03:20:30.174242+0000","last_undegraded":"2024-09-12T03:20:30.174242+0000","last_fullsized":"2024-09-12T03:20:30.174242+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_clean_scrub_stamp":"2024-09-12T03:20:28.136653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:33:07.633277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"80'8","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:20:29.188145+0000","last_change":"2024-09-12T03:20:28.159183+0000","last_active":"2024-09-12T03:20:29.188145+0000","last_peered":"2024-09-12T03:20:29.188145+0000","last_clean":"2024-09-12T03:20:29.188145+0000","last_became_active":"2024-09-12T03:20:28.158552+0000","last_became_peered":"2024-09-12T03:20:28.158552+0000","last_unstale":"2024-09-12T03:20:29.188145+0000","last_undegraded":"2024-09-12T03:20:29.188145+0000","last_fullsized":"2024-09-12T03:20:29.188145+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:27.129547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:27.129547+0000","last_clean_scrub_stamp":"2024-09-12T03:20:27.129547+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:29:13.201310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:20:29.177258+0000","last_change":"2024-09-12T03:20:29.175128+0000","last_active":"2024-09-12T03:20:29.177258+0000","last_peered":"2024-09-12T03:20:29.177258+0000","last_clean":"2024-09-12T03:20:29.177258+0000","last_became_active":"2024-09-12T03:20:29.174805+0000","last_became_peered":"2024-09-12T03:20:29.174805+0000","last_unstale":"2024-09-12T03:20:29.177258+0000","last_undegraded":"2024-09-12T03:20:29.177258+0000","last_fullsized":"2024-09-12T03:20:29.177258+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_clean_scrub_stamp":"2024-09-12T03:20:28.136653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:17:42.735036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"80'1","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-12T03:20:30.786997+0000","last_change":"2024-09-12T03:20:28.158768+0000","last_active":"2024-09-12T03:20:30.786997+0000","last_peered":"2024-09-12T03:20:30.786997+0000","last_clean":"2024-09-12T03:20:30.786997+0000","last_became_active":"2024-09-12T03:20:28.158309+0000","last_became_peered":"2024-09-12T03:20:28.158309+0000","last_unstale":"2024-09-12T03:20:30.786997+0000","last_undegraded":"2024-09-12T03:20:30.786997+0000","last_fullsized":"2024-09-12T03:20:30.786997+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:27.129547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:27.129547+0000","last_clean_scrub_stamp":"2024-09-12T03:20:27.129547+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:31:36.558882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-12T03:20:30.786908+0000","last_change":"2024-09-12T03:20:29.177747+0000","last_active":"2024-09-12T03:20:30.786908+0000","last_peered":"2024-09-12T03:20:30.786908+0000","last_clean":"2024-09-12T03:20:30.786908+0000","last_became_active":"2024-09-12T03:20:29.177105+0000","last_became_peered":"2024-09-12T03:20:29.177105+0000","last_unstale":"2024-09-12T03:20:30.786908+0000","last_undegraded":"2024-09-12T03:20:30.786908+0000","last_fullsized":"2024-09-12T03:20:30.786908+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_clean_scrub_stamp":"2024-09-12T03:20:28.136653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:55:30.198372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"80'2","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:20:29.182072+0000","last_change":"2024-09-12T03:20:28.160177+0000","last_active":"2024-09-12T03:20:29.182072+0000","last_peered":"2024-09-12T03:20:29.182072+0000","last_clean":"2024-09-12T03:20:29.182072+0000","last_became_active":"2024-09-12T03:20:28.159925+0000","last_became_peered":"2024-09-12T03:20:28.159925+0000","last_unstale":"2024-09-12T03:20:29.182072+0000","last_undegraded":"2024-09-12T03:20:29.182072+0000","last_fullsized":"2024-09-12T03:20:29.182072+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:27.129547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:27.129547+0000","last_clean_scrub_stamp":"2024-09-12T03:20:27.129547+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:41:28.079395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-12T03:20:30.176318+0000","last_change":"2024-09-12T03:20:29.187004+0000","last_active":"2024-09-12T03:20:30.176318+0000","last_peered":"2024-09-12T03:20:30.176318+0000","last_clean":"2024-09-12T03:20:30.176318+0000","last_became_active":"2024-09-12T03:20:29.186689+0000","last_became_peered":"2024-09-12T03:20:29.186689+0000","last_unstale":"2024-09-12T03:20:30.176318+0000","last_undegraded":"2024-09-12T03:20:30.176318+0000","last_fullsized":"2024-09-12T03:20:30.176318+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_clean_scrub_stamp":"2024-09-12T03:20:28.136653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:02:27.543548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:20:29.171529+0000","last_change":"2024-09-12T03:20:28.156929+0000","last_active":"2024-09-12T03:20:29.171529+0000","last_peered":"2024-09-12T03:20:29.171529+0000","last_clean":"2024-09-12T03:20:29.171529+0000","last_became_active":"2024-09-12T03:20:28.156666+0000","last_became_peered":"2024-09-12T03:20:28.156666+0000","last_unstale":"2024-09-12T03:20:29.171529+0000","last_undegraded":"2024-09-12T03:20:29.171529+0000","last_fullsized":"2024-09-12T03:20:29.171529+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:27.129547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:27.129547+0000","last_clean_scrub_stamp":"2024-09-12T03:20:27.129547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:55:29.191265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:20:29.192217+0000","last_change":"2024-09-12T03:20:29.186329+0000","last_active":"2024-09-12T03:20:29.192217+0000","last_peered":"2024-09-12T03:20:29.192217+0000","last_clean":"2024-09-12T03:20:29.192217+0000","last_became_active":"2024-09-12T03:20:29.186050+0000","last_became_peered":"2024-09-12T03:20:29.186050+0000","last_unstale":"2024-09-12T03:20:29.192217+0000","last_undegraded":"2024-09-12T03:20:29.192217+0000","last_fullsized":"2024-09-12T03:20:29.192217+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_clean_scrub_stamp":"2024-09-12T03:20:28.136653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:14:44.871074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"80'2","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-12T03:20:30.810101+0000","last_change":"2024-09-12T03:20:28.161455+0000","last_active":"2024-09-12T03:20:30.810101+0000","last_peered":"2024-09-12T03:20:30.810101+0000","last_clean":"2024-09-12T03:20:30.810101+0000","last_became_active":"2024-09-12T03:20:28.161180+0000","last_became_peered":"2024-09-12T03:20:28.161180+0000","last_unstale":"2024-09-12T03:20:30.810101+0000","last_undegraded":"2024-09-12T03:20:30.810101+0000","last_fullsized":"2024-09-12T03:20:30.810101+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:27.129547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:27.129547+0000","last_clean_scrub_stamp":"2024-09-12T03:20:27.129547+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:24:19.679944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-12T03:20:30.175467+0000","last_change":"2024-09-12T03:20:29.172018+0000","last_active":"2024-09-12T03:20:30.175467+0000","last_peered":"2024-09-12T03:20:30.175467+0000","last_clean":"2024-09-12T03:20:30.175467+0000","last_became_active":"2024-09-12T03:20:29.171710+0000","last_became_peered":"2024-09-12T03:20:29.171710+0000","last_unstale":"2024-09-12T03:20:30.175467+0000","last_undegraded":"2024-09-12T03:20:30.175467+0000","last_fullsized":"2024-09-12T03:20:30.175467+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_clean_scrub_stamp":"2024-09-12T03:20:28.136653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:28:05.839309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:20:29.168591+0000","last_change":"2024-09-12T03:20:28.158357+0000","last_active":"2024-09-12T03:20:29.168591+0000","last_peered":"2024-09-12T03:20:29.168591+0000","last_clean":"2024-09-12T03:20:29.168591+0000","last_became_active":"2024-09-12T03:20:28.158079+0000","last_became_peered":"2024-09-12T03:20:28.158079+0000","last_unstale":"2024-09-12T03:20:29.168591+0000","last_undegraded":"2024-09-12T03:20:29.168591+0000","last_fullsized":"2024-09-12T03:20:29.168591+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:27.129547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:27.129547+0000","last_clean_scrub_stamp":"2024-09-12T03:20:27.129547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:17:24.670406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:20:29.184471+0000","last_change":"2024-09-12T03:20:29.180789+0000","last_active":"2024-09-12T03:20:29.184471+0000","last_peered":"2024-09-12T03:20:29.184471+0000","last_clean":"2024-09-12T03:20:29.184471+0000","last_became_active":"2024-09-12T03:20:29.180491+0000","last_became_peered":"2024-09-12T03:20:29.180491+0000","last_unstale":"2024-09-12T03:20:29.184471+0000","last_undegraded":"2024-09-12T03:20:29.184471+0000","last_fullsized":"2024-09-12T03:20:29.184471+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_clean_scrub_stamp":"2024-09-12T03:20:28.136653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:44:59.211551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:20:29.166384+0000","last_change":"2024-09-12T03:20:28.155068+0000","last_active":"2024-09-12T03:20:29.166384+0000","last_peered":"2024-09-12T03:20:29.166384+0000","last_clean":"2024-09-12T03:20:29.166384+0000","last_became_active":"2024-09-12T03:20:28.154805+0000","last_became_peered":"2024-09-12T03:20:28.154805+0000","last_unstale":"2024-09-12T03:20:29.166384+0000","last_undegraded":"2024-09-12T03:20:29.166384+0000","last_fullsized":"2024-09-12T03:20:29.166384+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:27.129547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:27.129547+0000","last_clean_scrub_stamp":"2024-09-12T03:20:27.129547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:13:40.945583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-12T03:20:30.175632+0000","last_change":"2024-09-12T03:20:29.183683+0000","last_active":"2024-09-12T03:20:30.175632+0000","last_peered":"2024-09-12T03:20:30.175632+0000","last_clean":"2024-09-12T03:20:30.175632+0000","last_became_active":"2024-09-12T03:20:29.183388+0000","last_became_peered":"2024-09-12T03:20:29.183388+0000","last_unstale":"2024-09-12T03:20:30.175632+0000","last_undegraded":"2024-09-12T03:20:30.175632+0000","last_fullsized":"2024-09-12T03:20:30.175632+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_clean_scrub_stamp":"2024-09-12T03:20:28.136653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:05:32.014494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-12T03:20:30.809812+0000","last_change":"2024-09-12T03:20:28.158296+0000","last_active":"2024-09-12T03:20:30.809812+0000","last_peered":"2024-09-12T03:20:30.809812+0000","last_clean":"2024-09-12T03:20:30.809812+0000","last_became_active":"2024-09-12T03:20:28.157999+0000","last_became_peered":"2024-09-12T03:20:28.157999+0000","last_unstale":"2024-09-12T03:20:30.809812+0000","last_undegraded":"2024-09-12T03:20:30.809812+0000","last_fullsized":"2024-09-12T03:20:30.809812+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:27.129547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:27.129547+0000","last_clean_scrub_stamp":"2024-09-12T03:20:27.129547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:41:28.079395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-12T03:20:30.176207+0000","last_change":"2024-09-12T03:20:29.166228+0000","last_active":"2024-09-12T03:20:30.176207+0000","last_peered":"2024-09-12T03:20:30.176207+0000","last_clean":"2024-09-12T03:20:30.176207+0000","last_became_active":"2024-09-12T03:20:29.165927+0000","last_became_peered":"2024-09-12T03:20:29.165927+0000","last_unstale":"2024-09-12T03:20:30.176207+0000","last_undegraded":"2024-09-12T03:20:30.176207+0000","last_fullsized":"2024-09-12T03:20:30.176207+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_clean_scrub_stamp":"2024-09-12T03:20:28.136653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:00:06.471862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"80'2","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-12T03:20:30.174583+0000","last_change":"2024-09-12T03:20:28.159053+0000","last_active":"2024-09-12T03:20:30.174583+0000","last_peered":"2024-09-12T03:20:30.174583+0000","last_clean":"2024-09-12T03:20:30.174583+0000","last_became_active":"2024-09-12T03:20:28.158729+0000","last_became_peered":"2024-09-12T03:20:28.158729+0000","last_unstale":"2024-09-12T03:20:30.174583+0000","last_undegraded":"2024-09-12T03:20:30.174583+0000","last_fullsized":"2024-09-12T03:20:30.174583+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:27.129547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:27.129547+0000","last_clean_scrub_stamp":"2024-09-12T03:20:27.129547+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:23:15.274669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:20:29.177473+0000","last_change":"2024-09-12T03:20:29.175669+0000","last_active":"2024-09-12T03:20:29.177473+0000","last_peered":"2024-09-12T03:20:29.177473+0000","last_clean":"2024-09-12T03:20:29.177473+0000","last_became_active":"2024-09-12T03:20:29.175336+0000","last_became_peered":"2024-09-12T03:20:29.175336+0000","last_unstale":"2024-09-12T03:20:29.177473+0000","last_undegraded":"2024-09-12T03:20:29.177473+0000","last_fullsized":"2024-09-12T03:20:29.177473+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_clean_scrub_stamp":"2024-09-12T03:20:28.136653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:00:06.471862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-12T03:20:30.175002+0000","last_change":"2024-09-12T03:20:28.159898+0000","last_active":"2024-09-12T03:20:30.175002+0000","last_peered":"2024-09-12T03:20:30.175002+0000","last_clean":"2024-09-12T03:20:30.175002+0000","last_became_active":"2024-09-12T03:20:28.159573+0000","last_became_peered":"2024-09-12T03:20:28.159573+0000","last_unstale":"2024-09-12T03:20:30.175002+0000","last_undegraded":"2024-09-12T03:20:30.175002+0000","last_fullsized":"2024-09-12T03:20:30.175002+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:27.129547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:27.129547+0000","last_clean_scrub_stamp":"2024-09-12T03:20:27.129547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:22:43.287524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-12T03:20:30.786644+0000","last_change":"2024-09-12T03:20:29.184755+0000","last_active":"2024-09-12T03:20:30.786644+0000","last_peered":"2024-09-12T03:20:30.786644+0000","last_clean":"2024-09-12T03:20:30.786644+0000","last_became_active":"2024-09-12T03:20:29.184474+0000","last_became_peered":"2024-09-12T03:20:29.184474+0000","last_unstale":"2024-09-12T03:20:30.786644+0000","last_undegraded":"2024-09-12T03:20:30.786644+0000","last_fullsized":"2024-09-12T03:20:30.786644+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_clean_scrub_stamp":"2024-09-12T03:20:28.136653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:20:39.549097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-12T03:20:30.786603+0000","last_change":"2024-09-12T03:20:28.154889+0000","last_active":"2024-09-12T03:20:30.786603+0000","last_peered":"2024-09-12T03:20:30.786603+0000","last_clean":"2024-09-12T03:20:30.786603+0000","last_became_active":"2024-09-12T03:20:28.154544+0000","last_became_peered":"2024-09-12T03:20:28.154544+0000","last_unstale":"2024-09-12T03:20:30.786603+0000","last_undegraded":"2024-09-12T03:20:30.786603+0000","last_fullsized":"2024-09-12T03:20:30.786603+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:27.129547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:27.129547+0000","last_clean_scrub_stamp":"2024-09-12T03:20:27.129547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:01:16.414722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-12T03:20:30.176391+0000","last_change":"2024-09-12T03:20:29.181440+0000","last_active":"2024-09-12T03:20:30.176391+0000","last_peered":"2024-09-12T03:20:30.176391+0000","last_clean":"2024-09-12T03:20:30.176391+0000","last_became_active":"2024-09-12T03:20:29.181060+0000","last_became_peered":"2024-09-12T03:20:29.181060+0000","last_unstale":"2024-09-12T03:20:30.176391+0000","last_undegraded":"2024-09-12T03:20:30.176391+0000","last_fullsized":"2024-09-12T03:20:30.176391+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_clean_scrub_stamp":"2024-09-12T03:20:28.136653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:55:30.198372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-12T03:20:30.786741+0000","last_change":"2024-09-12T03:20:28.159532+0000","last_active":"2024-09-12T03:20:30.786741+0000","last_peered":"2024-09-12T03:20:30.786741+0000","last_clean":"2024-09-12T03:20:30.786741+0000","last_became_active":"2024-09-12T03:20:28.159207+0000","last_became_peered":"2024-09-12T03:20:28.159207+0000","last_unstale":"2024-09-12T03:20:30.786741+0000","last_undegraded":"2024-09-12T03:20:30.786741+0000","last_fullsized":"2024-09-12T03:20:30.786741+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:27.129547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:27.129547+0000","last_clean_scrub_stamp":"2024-09-12T03:20:27.129547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:41:56.345114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-12T03:20:30.175277+0000","last_change":"2024-09-12T03:20:29.171340+0000","last_active":"2024-09-12T03:20:30.175277+0000","last_peered":"2024-09-12T03:20:30.175277+0000","last_clean":"2024-09-12T03:20:30.175277+0000","last_became_active":"2024-09-12T03:20:29.171051+0000","last_became_peered":"2024-09-12T03:20:29.171051+0000","last_unstale":"2024-09-12T03:20:30.175277+0000","last_undegraded":"2024-09-12T03:20:30.175277+0000","last_fullsized":"2024-09-12T03:20:30.175277+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_clean_scrub_stamp":"2024-09-12T03:20:28.136653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:30:26.379868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:20:29.174095+0000","last_change":"2024-09-12T03:20:28.158162+0000","last_active":"2024-09-12T03:20:29.174095+0000","last_peered":"2024-09-12T03:20:29.174095+0000","last_clean":"2024-09-12T03:20:29.174095+0000","last_became_active":"2024-09-12T03:20:28.157908+0000","last_became_peered":"2024-09-12T03:20:28.157908+0000","last_unstale":"2024-09-12T03:20:29.174095+0000","last_undegraded":"2024-09-12T03:20:29.174095+0000","last_fullsized":"2024-09-12T03:20:29.174095+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:27.129547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:27.129547+0000","last_clean_scrub_stamp":"2024-09-12T03:20:27.129547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:36:18.476441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:20:29.182213+0000","last_change":"2024-09-12T03:20:29.178330+0000","last_active":"2024-09-12T03:20:29.182213+0000","last_peered":"2024-09-12T03:20:29.182213+0000","last_clean":"2024-09-12T03:20:29.182213+0000","last_became_active":"2024-09-12T03:20:29.178067+0000","last_became_peered":"2024-09-12T03:20:29.178067+0000","last_unstale":"2024-09-12T03:20:29.182213+0000","last_undegraded":"2024-09-12T03:20:29.182213+0000","last_fullsized":"2024-09-12T03:20:29.182213+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_clean_scrub_stamp":"2024-09-12T03:20:28.136653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:47:28.912863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-12T03:20:30.176093+0000","last_change":"2024-09-12T03:20:28.153258+0000","last_active":"2024-09-12T03:20:30.176093+0000","last_peered":"2024-09-12T03:20:30.176093+0000","last_clean":"2024-09-12T03:20:30.176093+0000","last_became_active":"2024-09-12T03:20:28.153006+0000","last_became_peered":"2024-09-12T03:20:28.153006+0000","last_unstale":"2024-09-12T03:20:30.176093+0000","last_undegraded":"2024-09-12T03:20:30.176093+0000","last_fullsized":"2024-09-12T03:20:30.176093+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:27.129547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:27.129547+0000","last_clean_scrub_stamp":"2024-09-12T03:20:27.129547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:25:20.650913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:20:29.190369+0000","last_change":"2024-09-12T03:20:29.189865+0000","last_active":"2024-09-12T03:20:29.190369+0000","last_peered":"2024-09-12T03:20:29.190369+0000","last_clean":"2024-09-12T03:20:29.190369+0000","last_became_active":"2024-09-12T03:20:29.189592+0000","last_became_peered":"2024-09-12T03:20:29.189592+0000","last_unstale":"2024-09-12T03:20:29.190369+0000","last_undegraded":"2024-09-12T03:20:29.190369+0000","last_fullsized":"2024-09-12T03:20:29.190369+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_clean_scrub_stamp":"2024-09-12T03:20:28.136653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:24:20.687050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-12T03:20:30.786470+0000","last_change":"2024-09-12T03:20:28.159255+0000","last_active":"2024-09-12T03:20:30.786470+0000","last_peered":"2024-09-12T03:20:30.786470+0000","last_clean":"2024-09-12T03:20:30.786470+0000","last_became_active":"2024-09-12T03:20:28.158961+0000","last_became_peered":"2024-09-12T03:20:28.158961+0000","last_unstale":"2024-09-12T03:20:30.786470+0000","last_undegraded":"2024-09-12T03:20:30.786470+0000","last_fullsized":"2024-09-12T03:20:30.786470+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:27.129547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:27.129547+0000","last_clean_scrub_stamp":"2024-09-12T03:20:27.129547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:41:42.935780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-12T03:20:30.786309+0000","last_change":"2024-09-12T03:20:29.184619+0000","last_active":"2024-09-12T03:20:30.786309+0000","last_peered":"2024-09-12T03:20:30.786309+0000","last_clean":"2024-09-12T03:20:30.786309+0000","last_became_active":"2024-09-12T03:20:29.183980+0000","last_became_peered":"2024-09-12T03:20:29.183980+0000","last_unstale":"2024-09-12T03:20:30.786309+0000","last_undegraded":"2024-09-12T03:20:30.786309+0000","last_fullsized":"2024-09-12T03:20:30.786309+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_clean_scrub_stamp":"2024-09-12T03:20:28.136653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:21:39.137804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-12T03:20:30.175301+0000","last_change":"2024-09-12T03:20:28.158015+0000","last_active":"2024-09-12T03:20:30.175301+0000","last_peered":"2024-09-12T03:20:30.175301+0000","last_clean":"2024-09-12T03:20:30.175301+0000","last_became_active":"2024-09-12T03:20:28.157524+0000","last_became_peered":"2024-09-12T03:20:28.157524+0000","last_unstale":"2024-09-12T03:20:30.175301+0000","last_undegraded":"2024-09-12T03:20:30.175301+0000","last_fullsized":"2024-09-12T03:20:30.175301+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:27.129547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:27.129547+0000","last_clean_scrub_stamp":"2024-09-12T03:20:27.129547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:45:52.294635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-12T03:20:30.174983+0000","last_change":"2024-09-12T03:20:29.177151+0000","last_active":"2024-09-12T03:20:30.174983+0000","last_peered":"2024-09-12T03:20:30.174983+0000","last_clean":"2024-09-12T03:20:30.174983+0000","last_became_active":"2024-09-12T03:20:29.175828+0000","last_became_peered":"2024-09-12T03:20:29.175828+0000","last_unstale":"2024-09-12T03:20:30.174983+0000","last_undegraded":"2024-09-12T03:20:30.174983+0000","last_fullsized":"2024-09-12T03:20:30.174983+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:28.136653+0000","last_clean_scrub_stamp":"2024-09-12T03:20:28.136653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:52:37.273971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"80'1","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:20:29.186395+0000","last_change":"2024-09-12T03:20:28.163141+0000","last_active":"2024-09-12T03:20:29.186395+0000","last_peered":"2024-09-12T03:20:29.186395+0000","last_clean":"2024-09-12T03:20:29.186395+0000","last_became_active":"2024-09-12T03:20:28.162809+0000","last_became_peered":"2024-09-12T03:20:28.162809+0000","last_unstale":"2024-09-12T03:20:29.186395+0000","last_undegraded":"2024-09-12T03:20:29.186395+0000","last_fullsized":"2024-09-12T03:20:29.186395+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:27.129547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:27.129547+0000","last_clean_scrub_stamp":"2024-09-12T03:20:27.129547+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:21:38.130697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"70'128","reported_seq":316,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:20:29.167861+0000","last_change":"2024-09-12T03:16:21.883446+0000","last_active":"2024-09-12T03:20:29.167861+0000","last_peered":"2024-09-12T03:20:29.167861+0000","last_clean":"2024-09-12T03:20:29.167861+0000","last_became_active":"2024-09-12T03:16:21.882207+0000","last_became_peered":"2024-09-12T03:16:21.882207+0000","last_unstale":"2024-09-12T03:20:29.167861+0000","last_undegraded":"2024-09-12T03:20:29.167861+0000","last_fullsized":"2024-09-12T03:20:29.167861+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:20.316426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:20.316426+0000","last_clean_scrub_stamp":"2024-09-12T03:16:20.316426+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:12:15.424320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542196,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36892,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334948,"statfs":{"total":96636764160,"available":96598986752,"internally_reserved":0,"allocated":1576960,"data_stored":1009988,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542198,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36320,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335520,"statfs":{"total":96636764160,"available":96599572480,"internally_reserved":0,"allocated":991232,"data_stored":420244,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542197,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36336,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335504,"statfs":{"total":96636764160,"available":96599556096,"internally_reserved":0,"allocated":1007616,"data_stored":427484,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542196,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36316,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335524,"statfs":{"total":96636764160,"available":96599576576,"internally_reserved":0,"allocated":991232,"data_stored":425850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542197,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28004,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343836,"statfs":{"total":96636764160,"available":96608088064,"internally_reserved":0,"allocated":991232,"data_stored":421162,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542197,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27992,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343848,"statfs":{"total":96636764160,"available":96608100352,"internally_reserved":0,"allocated":983040,"data_stored":420132,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542198,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32172,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339668,"statfs":{"total":96636764160,"available":96603820032,"internally_reserved":0,"allocated":1003520,"data_stored":426044,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542198,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32728,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339112,"statfs":{"total":96636764160,"available":96603250688,"internally_reserved":0,"allocated":1576960,"data_stored":1009952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T03:20:32.499 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-12T03:20:32.500 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-12T03:20:32.645 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:32.645+0000 7fb0ae778640 1 Processor -- start 2024-09-12T03:20:32.645 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:32.645+0000 7fb0ae778640 1 -- start start 2024-09-12T03:20:32.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:32.646+0000 7fb0ae778640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fb0a806c650 0x7fb0a80b4000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:32.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:32.646+0000 7fb0ae778640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb0a80cf320 0x7fb0a815f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:32.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:32.646+0000 7fb0ae778640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb0a815f600 con 0x7fb0a80cef40 2024-09-12T03:20:32.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:32.646+0000 7fb0ae778640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fb0a806dee0 con 0x7fb0a806c650 2024-09-12T03:20:32.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:32.646+0000 7fb0ae778640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fb0a806cff0 con 0x7fb0a80cf320 2024-09-12T03:20:32.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:32.646+0000 7fb0acd15640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb0a80cf320 0x7fb0a815f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:32.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:32.646+0000 7fb09ffff640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fb0a806c650 0x7fb0a80b4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:32.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:32.646+0000 7fb0acd15640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb0a80cf320 0x7fb0a815f0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3301/0 says I am v2:172.21.15.27:52752/0 (socket says 172.21.15.27:52752) 2024-09-12T03:20:32.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:32.646+0000 7fb0acd15640 1 -- 172.21.15.27:0/3115973366 learned_addr learned my addr 172.21.15.27:0/3115973366 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:20:32.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:32.647+0000 7fb0a77fe640 1 -- 172.21.15.27:0/3115973366 <== mon.1 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3372802538 0 0) 0x7fb0a815f600 con 0x7fb0a80cef40 2024-09-12T03:20:32.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:32.647+0000 7fb0a77fe640 1 -- 172.21.15.27:0/3115973366 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb094003520 con 0x7fb0a80cef40 2024-09-12T03:20:32.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:32.647+0000 7fb0acd15640 1 -- 172.21.15.27:0/3115973366 >> v2:172.21.15.45:3300/0 conn(0x7fb0a806c650 msgr2=0x7fb0a80b4000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:32.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:32.647+0000 7fb0acd15640 1 --2- 172.21.15.27:0/3115973366 >> v2:172.21.15.45:3300/0 conn(0x7fb0a806c650 0x7fb0a80b4000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:32.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:32.647+0000 7fb0acd15640 1 -- 172.21.15.27:0/3115973366 >> v1:172.21.15.45:6789/0 conn(0x7fb0a80cef40 legacy=0x7fb0a80710c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:32.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:32.647+0000 7fb0acd15640 1 -- 172.21.15.27:0/3115973366 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0a806d2d0 con 0x7fb0a80cf320 2024-09-12T03:20:32.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:32.648+0000 7fb0acd15640 1 --2- 172.21.15.27:0/3115973366 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb0a80cf320 0x7fb0a815f0c0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fb0a000d150 tx=0x7fb0a002c890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e2033abcfc016b9a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:32.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:32.648+0000 7fb0a77fe640 1 -- 172.21.15.27:0/3115973366 <== mon.2 v2:172.21.15.45:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0a0039070 con 0x7fb0a80cf320 2024-09-12T03:20:32.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:32.648+0000 7fb0a77fe640 1 -- 172.21.15.27:0/3115973366 <== mon.2 v2:172.21.15.45:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0a0009600 con 0x7fb0a80cf320 2024-09-12T03:20:32.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:32.648+0000 7fb0a77fe640 1 -- 172.21.15.27:0/3115973366 <== mon.2 v2:172.21.15.45:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0a00099d0 con 0x7fb0a80cf320 2024-09-12T03:20:32.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:32.648+0000 7fb0ae778640 1 -- 172.21.15.27:0/3115973366 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb0a80cf320 msgr2=0x7fb0a815f0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:32.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:32.648+0000 7fb0ae778640 1 --2- 172.21.15.27:0/3115973366 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb0a80cf320 0x7fb0a815f0c0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fb0a000d150 tx=0x7fb0a002c890 comp rx=0 tx=0).stop 2024-09-12T03:20:32.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:32.648+0000 7fb0ae778640 1 -- 172.21.15.27:0/3115973366 shutdown_connections 2024-09-12T03:20:32.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:32.648+0000 7fb0ae778640 1 --2- 172.21.15.27:0/3115973366 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb0a80cf320 0x7fb0a815f0c0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:32.650 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:32.648+0000 7fb0ae778640 1 --2- 172.21.15.27:0/3115973366 >> v2:172.21.15.45:3300/0 conn(0x7fb0a806c650 0x7fb0a80b4000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:32.650 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:32.648+0000 7fb0ae778640 1 -- 172.21.15.27:0/3115973366 >> 172.21.15.27:0/3115973366 conn(0x7fb0a806bb30 msgr2=0x7fb0a806bf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:32.650 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:32.648+0000 7fb0ae778640 1 -- 172.21.15.27:0/3115973366 shutdown_connections 2024-09-12T03:20:32.650 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:32.649+0000 7fb0ae778640 1 -- 172.21.15.27:0/3115973366 wait complete. 2024-09-12T03:20:32.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:32.650+0000 7fb0ae778640 1 Processor -- start 2024-09-12T03:20:32.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:32.650+0000 7fb0ae778640 1 -- start start 2024-09-12T03:20:32.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:32.650+0000 7fb0ae778640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb0a806c650 0x7fb0a80aeb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:32.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:32.650+0000 7fb0ae778640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb0a80cef40 0x7fb0a8154dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:32.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:32.650+0000 7fb0ae778640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb0a80cf320 0x7fb0a8155300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:32.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:32.650+0000 7fb0ae778640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fb0a806db70 con 0x7fb0a806c650 2024-09-12T03:20:32.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:32.650+0000 7fb0ae778640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb0a806d890 con 0x7fb0a80cef40 2024-09-12T03:20:32.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:32.650+0000 7fb0ae778640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fb0a80ce440 con 0x7fb0a80cf320 2024-09-12T03:20:32.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:32.651+0000 7fb0a7fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb0a806c650 0x7fb0a80aeb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:32.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:32.651+0000 7fb0a7fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb0a806c650 0x7fb0a80aeb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46984/0 (socket says 172.21.15.27:46984) 2024-09-12T03:20:32.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:32.651+0000 7fb0a7fff640 1 -- 172.21.15.27:0/3985544220 learned_addr learned my addr 172.21.15.27:0/3985544220 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:20:32.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:32.651+0000 7fb09ffff640 1 --2- 172.21.15.27:0/3985544220 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb0a80cef40 0x7fb0a8154dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:32.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:32.651+0000 7fb0acd15640 1 --2- 172.21.15.27:0/3985544220 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb0a80cf320 0x7fb0a8155300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:32.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:32.651+0000 7fb0a7fff640 1 -- 172.21.15.27:0/3985544220 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb0a80cf320 msgr2=0x7fb0a8155300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:32.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:32.651+0000 7fb0a7fff640 1 --2- 172.21.15.27:0/3985544220 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb0a80cf320 0x7fb0a8155300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:32.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:32.651+0000 7fb0a7fff640 1 -- 172.21.15.27:0/3985544220 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb0a80cef40 msgr2=0x7fb0a8154dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:32.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:32.651+0000 7fb0a7fff640 1 --2- 172.21.15.27:0/3985544220 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb0a80cef40 0x7fb0a8154dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:32.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:32.651+0000 7fb0a7fff640 1 -- 172.21.15.27:0/3985544220 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0a81f43f0 con 0x7fb0a806c650 2024-09-12T03:20:32.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:32.651+0000 7fb0acd15640 1 --2- 172.21.15.27:0/3985544220 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb0a80cf320 0x7fb0a8155300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:20:32.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:32.651+0000 7fb0a7fff640 1 --2- 172.21.15.27:0/3985544220 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb0a806c650 0x7fb0a80aeb70 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fb09800c500 tx=0x7fb09800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:32.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:32.651+0000 7fb0a57fa640 1 -- 172.21.15.27:0/3985544220 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0980052d0 con 0x7fb0a806c650 2024-09-12T03:20:32.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:32.651+0000 7fb0a57fa640 1 -- 172.21.15.27:0/3985544220 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb098005490 con 0x7fb0a806c650 2024-09-12T03:20:32.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:32.651+0000 7fb0ae778640 1 -- 172.21.15.27:0/3985544220 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0a81f46b0 con 0x7fb0a806c650 2024-09-12T03:20:32.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:32.651+0000 7fb0a57fa640 1 -- 172.21.15.27:0/3985544220 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb098012870 con 0x7fb0a806c650 2024-09-12T03:20:32.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:32.651+0000 7fb0ae778640 1 -- 172.21.15.27:0/3985544220 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0a81f4c70 con 0x7fb0a806c650 2024-09-12T03:20:32.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:32.652+0000 7fb0ae778640 1 -- 172.21.15.27:0/3985544220 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0a80ce7f0 con 0x7fb0a806c650 2024-09-12T03:20:32.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:32.653+0000 7fb0a57fa640 1 -- 172.21.15.27:0/3985544220 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 90600+0+0 (secure 0 0 0) 0x7fb098012a10 con 0x7fb0a806c650 2024-09-12T03:20:32.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:32.653+0000 7fb0a57fa640 1 --2- 172.21.15.27:0/3985544220 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fb08806bf50 0x7fb08806e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:32.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:32.653+0000 7fb0a57fa640 1 -- 172.21.15.27:0/3985544220 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 5709+0+0 (secure 0 0 0) 0x7fb09808ab90 con 0x7fb0a806c650 2024-09-12T03:20:32.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:32.653+0000 7fb09ffff640 1 --2- 172.21.15.27:0/3985544220 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fb08806bf50 0x7fb08806e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:32.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:32.654+0000 7fb09ffff640 1 --2- 172.21.15.27:0/3985544220 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fb08806bf50 0x7fb08806e410 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb0a80b4470 tx=0x7fb090009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:32.657 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:32.658+0000 7fb0a57fa640 1 -- 172.21.15.27:0/3985544220 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fb098059440 con 0x7fb0a806c650 2024-09-12T03:20:32.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:32.903+0000 7fb0ae778640 1 -- 172.21.15.27:0/3985544220 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7fb0a806f240 con 0x7fb0a806c650 2024-09-12T03:20:32.905 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:32.905+0000 7fb0a57fa640 1 -- 172.21.15.27:0/3985544220 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v17) ==== 190+0+147 (secure 0 0 0) 0x7fb098058d40 con 0x7fb0a806c650 2024-09-12T03:20:32.905 INFO:teuthology.orchestra.run.smithi027.stdout:[client.0] 2024-09-12T03:20:32.905 INFO:teuthology.orchestra.run.smithi027.stdout: key = AQDVXOJmLm+5ExAAECM4CKfuVElmp/n0b02mXg== 2024-09-12T03:20:32.905 INFO:teuthology.orchestra.run.smithi027.stdout: caps mds = "allow" 2024-09-12T03:20:32.905 INFO:teuthology.orchestra.run.smithi027.stdout: caps mon = "allow r" 2024-09-12T03:20:32.905 INFO:teuthology.orchestra.run.smithi027.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-12T03:20:32.905 INFO:teuthology.orchestra.run.smithi027.stderr:updated caps for client.0 2024-09-12T03:20:32.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:32.907+0000 7fb0ae778640 1 -- 172.21.15.27:0/3985544220 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fb08806bf50 msgr2=0x7fb08806e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:32.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:32.907+0000 7fb0ae778640 1 --2- 172.21.15.27:0/3985544220 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fb08806bf50 0x7fb08806e410 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb0a80b4470 tx=0x7fb090009190 comp rx=0 tx=0).stop 2024-09-12T03:20:32.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:32.907+0000 7fb0ae778640 1 -- 172.21.15.27:0/3985544220 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb0a806c650 msgr2=0x7fb0a80aeb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:32.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:32.907+0000 7fb0ae778640 1 --2- 172.21.15.27:0/3985544220 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb0a806c650 0x7fb0a80aeb70 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fb09800c500 tx=0x7fb09800c9c0 comp rx=0 tx=0).stop 2024-09-12T03:20:32.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:32.907+0000 7fb0ae778640 1 -- 172.21.15.27:0/3985544220 shutdown_connections 2024-09-12T03:20:32.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:32.907+0000 7fb0ae778640 1 --2- 172.21.15.27:0/3985544220 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb0a80cf320 0x7fb0a8155300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:32.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:32.907+0000 7fb0ae778640 1 --2- 172.21.15.27:0/3985544220 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb0a80cef40 0x7fb0a8154dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:32.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:32.907+0000 7fb0ae778640 1 --2- 172.21.15.27:0/3985544220 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fb08806bf50 0x7fb08806e410 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:32.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:32.907+0000 7fb0ae778640 1 --2- 172.21.15.27:0/3985544220 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb0a806c650 0x7fb0a80aeb70 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:32.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:32.907+0000 7fb0ae778640 1 -- 172.21.15.27:0/3985544220 >> 172.21.15.27:0/3985544220 conn(0x7fb0a806bb30 msgr2=0x7fb0a806bf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:32.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:32.907+0000 7fb0ae778640 1 -- 172.21.15.27:0/3985544220 shutdown_connections 2024-09-12T03:20:32.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:32.907+0000 7fb0ae778640 1 -- 172.21.15.27:0/3985544220 wait complete. 2024-09-12T03:20:32.918 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-12T03:20:33.020 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.020+0000 7f4720847640 1 Processor -- start 2024-09-12T03:20:33.020 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.020+0000 7f4720847640 1 -- start start 2024-09-12T03:20:33.021 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.021+0000 7f4720847640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f471c06f360 0x7f471c0cef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:33.021 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.021+0000 7f4720847640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f471c0cf490 0x7f471c0d2780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:33.021 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.021+0000 7f4720847640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f471c06cc40 con 0x7f471c0b1ca0 2024-09-12T03:20:33.022 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.021+0000 7f4720847640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f471c0d22f0 con 0x7f471c06f360 2024-09-12T03:20:33.022 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.021+0000 7f4720847640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f471c0d1ce0 con 0x7f471c0cf490 2024-09-12T03:20:33.022 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.021+0000 7f4719d9b640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f471c06f360 0x7f471c0cef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:33.022 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.021+0000 7f471a59c640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f471c0b1ca0 0x7f471c15f830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:57690/0 (socket says 172.21.15.27:57690) 2024-09-12T03:20:33.022 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.021+0000 7f471a59c640 1 -- 172.21.15.27:0/2568105419 learned_addr learned my addr 172.21.15.27:0/2568105419 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:20:33.022 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.021+0000 7f4719d9b640 1 -- 172.21.15.27:0/2568105419 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f471c0cf490 msgr2=0x7f471c0d2780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:33.022 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.021+0000 7f4719d9b640 1 --2- 172.21.15.27:0/2568105419 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f471c0cf490 0x7f471c0d2780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:33.022 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.021+0000 7f4719d9b640 1 -- 172.21.15.27:0/2568105419 >> v1:172.21.15.27:6789/0 conn(0x7f471c0b1ca0 legacy=0x7f471c15f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:33.022 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.022+0000 7f471959a640 1 -- 172.21.15.27:0/2568105419 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 364283191 0 0) 0x7f471c06cc40 con 0x7f471c0b1ca0 2024-09-12T03:20:33.022 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.022+0000 7f4719d9b640 1 -- 172.21.15.27:0/2568105419 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f471c06cee0 con 0x7f471c06f360 2024-09-12T03:20:33.023 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.022+0000 7f4719d9b640 1 --2- 172.21.15.27:0/2568105419 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f471c06f360 0x7f471c0cef50 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f47100097a0 tx=0x7f471002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecf1f81acfeb4dfd server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:33.023 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.022+0000 7f471959a640 1 -- 172.21.15.27:0/2568105419 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4710037070 con 0x7f471c06f360 2024-09-12T03:20:33.023 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.022+0000 7f471959a640 1 -- 172.21.15.27:0/2568105419 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4710032440 con 0x7f471c06f360 2024-09-12T03:20:33.024 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.022+0000 7f471959a640 1 -- 172.21.15.27:0/2568105419 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4710032810 con 0x7f471c06f360 2024-09-12T03:20:33.024 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.023+0000 7f4720847640 1 -- 172.21.15.27:0/2568105419 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f471c06f360 msgr2=0x7f471c0cef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:33.024 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.023+0000 7f4720847640 1 --2- 172.21.15.27:0/2568105419 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f471c06f360 0x7f471c0cef50 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f47100097a0 tx=0x7f471002a180 comp rx=0 tx=0).stop 2024-09-12T03:20:33.024 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.023+0000 7f4720847640 1 -- 172.21.15.27:0/2568105419 shutdown_connections 2024-09-12T03:20:33.024 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.023+0000 7f4720847640 1 --2- 172.21.15.27:0/2568105419 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f471c0cf490 0x7f471c0d2780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:33.024 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.023+0000 7f4720847640 1 --2- 172.21.15.27:0/2568105419 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f471c06f360 0x7f471c0cef50 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:33.024 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.023+0000 7f4720847640 1 -- 172.21.15.27:0/2568105419 >> 172.21.15.27:0/2568105419 conn(0x7f471c073800 msgr2=0x7f471c09ba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:33.024 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.023+0000 7f4720847640 1 -- 172.21.15.27:0/2568105419 shutdown_connections 2024-09-12T03:20:33.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.023+0000 7f4720847640 1 -- 172.21.15.27:0/2568105419 wait complete. 2024-09-12T03:20:33.026 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.024+0000 7f4720847640 1 Processor -- start 2024-09-12T03:20:33.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.024+0000 7f4720847640 1 -- start start 2024-09-12T03:20:33.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.025+0000 7f4720847640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f471c06f360 0x7f471c09e480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:33.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.025+0000 7f4720847640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f471c0b1ca0 0x7f471c09e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:33.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.025+0000 7f4720847640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f471c0cf490 0x7f471c15f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:33.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.025+0000 7f4720847640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f471c06d340 con 0x7f471c0b1ca0 2024-09-12T03:20:33.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.025+0000 7f4720847640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f471c06d1c0 con 0x7f471c0cf490 2024-09-12T03:20:33.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.025+0000 7f4720847640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f471c0cfc40 con 0x7f471c06f360 2024-09-12T03:20:33.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.025+0000 7f4719d9b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f471c0b1ca0 0x7f471c09e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:33.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.025+0000 7f4719d9b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f471c0b1ca0 0x7f471c09e9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46994/0 (socket says 172.21.15.27:46994) 2024-09-12T03:20:33.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.025+0000 7f4719d9b640 1 -- 172.21.15.27:0/617379737 learned_addr learned my addr 172.21.15.27:0/617379737 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:20:33.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.025+0000 7f471a59c640 1 --2- 172.21.15.27:0/617379737 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f471c06f360 0x7f471c09e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:33.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.025+0000 7f471ad9d640 1 --2- 172.21.15.27:0/617379737 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f471c0cf490 0x7f471c15f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:33.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.026+0000 7f4719d9b640 1 -- 172.21.15.27:0/617379737 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f471c06f360 msgr2=0x7f471c09e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:33.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.026+0000 7f4719d9b640 1 --2- 172.21.15.27:0/617379737 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f471c06f360 0x7f471c09e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:33.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.026+0000 7f4719d9b640 1 -- 172.21.15.27:0/617379737 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f471c0cf490 msgr2=0x7f471c15f9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:33.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.026+0000 7f4719d9b640 1 --2- 172.21.15.27:0/617379737 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f471c0cf490 0x7f471c15f9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:33.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.026+0000 7f4719d9b640 1 -- 172.21.15.27:0/617379737 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f471c09f1f0 con 0x7f471c0b1ca0 2024-09-12T03:20:33.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.026+0000 7f471ad9d640 1 --2- 172.21.15.27:0/617379737 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f471c0cf490 0x7f471c15f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:20:33.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.026+0000 7f471a59c640 1 --2- 172.21.15.27:0/617379737 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f471c06f360 0x7f471c09e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:20:33.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.026+0000 7f4719d9b640 1 --2- 172.21.15.27:0/617379737 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f471c0b1ca0 0x7f471c09e9c0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f4710000c00 tx=0x7f47100046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:33.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.026+0000 7f470affd640 1 -- 172.21.15.27:0/617379737 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4710041070 con 0x7f471c0b1ca0 2024-09-12T03:20:33.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.026+0000 7f470affd640 1 -- 172.21.15.27:0/617379737 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f471003c930 con 0x7f471c0b1ca0 2024-09-12T03:20:33.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.026+0000 7f4720847640 1 -- 172.21.15.27:0/617379737 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f471c09f4d0 con 0x7f471c0b1ca0 2024-09-12T03:20:33.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.026+0000 7f470affd640 1 -- 172.21.15.27:0/617379737 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4710037040 con 0x7f471c0b1ca0 2024-09-12T03:20:33.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.026+0000 7f4720847640 1 -- 172.21.15.27:0/617379737 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f471c09fa90 con 0x7f471c0b1ca0 2024-09-12T03:20:33.029 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.027+0000 7f4720847640 1 -- 172.21.15.27:0/617379737 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f471c1518d0 con 0x7f471c0b1ca0 2024-09-12T03:20:33.029 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.028+0000 7f470affd640 1 -- 172.21.15.27:0/617379737 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 90600+0+0 (secure 0 0 0) 0x7f4710044070 con 0x7f471c0b1ca0 2024-09-12T03:20:33.029 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.028+0000 7f470affd640 1 --2- 172.21.15.27:0/617379737 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f46fc06bf50 0x7f46fc06e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:33.029 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.028+0000 7f470affd640 1 -- 172.21.15.27:0/617379737 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 5709+0+0 (secure 0 0 0) 0x7f47100abc80 con 0x7f471c0b1ca0 2024-09-12T03:20:33.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.028+0000 7f471a59c640 1 --2- 172.21.15.27:0/617379737 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f46fc06bf50 0x7f46fc06e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:33.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.029+0000 7f471a59c640 1 --2- 172.21.15.27:0/617379737 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f46fc06bf50 0x7f46fc06e410 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f470c000f60 tx=0x7f470c009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:33.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.032+0000 7f470affd640 1 -- 172.21.15.27:0/617379737 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f471c1518d0 con 0x7f471c0b1ca0 2024-09-12T03:20:33.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.245+0000 7f4720847640 1 -- 172.21.15.27:0/617379737 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f471c06e720 con 0x7f471c0b1ca0 2024-09-12T03:20:33.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.245+0000 7f470affd640 1 -- 172.21.15.27:0/617379737 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) ==== 94+0+5222 (secure 0 0 0) 0x7f471007a640 con 0x7f471c0b1ca0 2024-09-12T03:20:33.245 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:20:33.245 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":46,"btime":"2024-09-12T03:20:31:172713+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5669,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/2036680565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2036680565},{"type":"v1","addr":"172.21.15.45:6834","nonce":2036680565}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":43},{"gid":6094,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/874218126","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":874218126},{"type":"v1","addr":"172.21.15.45:6835","nonce":874218126}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":43},{"gid":6483,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6834/1466511159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":1466511159},{"type":"v1","addr":"172.21.15.27:6834","nonce":1466511159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-12T03:20:28.499550+0000","modified":"2024-09-12T03:20:30.510730+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6486},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6486":{"gid":6486,"name":"a","rank":0,"incarnation":44,"state":"up:active","state_seq":3,"addr":"172.21.15.27:6835/3209963418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":3209963418},{"type":"v1","addr":"172.21.15.27:6835","nonce":3209963418}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6486,"qdb_cluster":[6486]},"id":6}]} 2024-09-12T03:20:33.246 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 46 2024-09-12T03:20:33.248 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.247+0000 7f4720847640 1 -- 172.21.15.27:0/617379737 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f46fc06bf50 msgr2=0x7f46fc06e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:33.248 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.247+0000 7f4720847640 1 --2- 172.21.15.27:0/617379737 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f46fc06bf50 0x7f46fc06e410 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f470c000f60 tx=0x7f470c009040 comp rx=0 tx=0).stop 2024-09-12T03:20:33.248 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.247+0000 7f4720847640 1 -- 172.21.15.27:0/617379737 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f471c0b1ca0 msgr2=0x7f471c09e9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:33.248 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.247+0000 7f4720847640 1 --2- 172.21.15.27:0/617379737 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f471c0b1ca0 0x7f471c09e9c0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f4710000c00 tx=0x7f47100046c0 comp rx=0 tx=0).stop 2024-09-12T03:20:33.248 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.247+0000 7f4720847640 1 -- 172.21.15.27:0/617379737 shutdown_connections 2024-09-12T03:20:33.248 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.247+0000 7f4720847640 1 --2- 172.21.15.27:0/617379737 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f471c0cf490 0x7f471c15f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:33.248 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.247+0000 7f4720847640 1 --2- 172.21.15.27:0/617379737 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f46fc06bf50 0x7f46fc06e410 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:33.248 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.247+0000 7f4720847640 1 --2- 172.21.15.27:0/617379737 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f471c0b1ca0 0x7f471c09e9c0 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:33.248 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.247+0000 7f4720847640 1 --2- 172.21.15.27:0/617379737 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f471c06f360 0x7f471c09e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:33.248 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.247+0000 7f4720847640 1 -- 172.21.15.27:0/617379737 >> 172.21.15.27:0/617379737 conn(0x7f471c073800 msgr2=0x7f471c0b74e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:33.249 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.247+0000 7f4720847640 1 -- 172.21.15.27:0/617379737 shutdown_connections 2024-09-12T03:20:33.249 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.247+0000 7f4720847640 1 -- 172.21.15.27:0/617379737 wait complete. 2024-09-12T03:20:33.661 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 46, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-12T03:20:28.499550+0000', 'modified': '2024-09-12T03:20:30.510730+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6486}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6486': {'gid': 6486, 'name': 'a', 'rank': 0, 'incarnation': 44, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.27:6835/3209963418', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6833', 'nonce': 3209963418}, {'type': 'v1', 'addr': '172.21.15.27:6835', 'nonce': 3209963418}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6486, 'qdb_cluster': [6486]} 2024-09-12T03:20:33.662 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-12T03:20:33.662 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-09-12T03:20:33.770 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.770+0000 7feab8ee4640 1 Processor -- start 2024-09-12T03:20:33.770 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.770+0000 7feab8ee4640 1 -- start start 2024-09-12T03:20:33.771 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.771+0000 7feab8ee4640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7feab415e760 0x7feab406d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:33.771 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.771+0000 7feab8ee4640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7feab406dc10 0x7feab40b43f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:33.771 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.771+0000 7feab8ee4640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feab40717e0 con 0x7feab40ae8d0 2024-09-12T03:20:33.771 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.771+0000 7feab8ee4640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7feab4071ac0 con 0x7feab415e760 2024-09-12T03:20:33.771 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.771+0000 7feab8ee4640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7feab4071da0 con 0x7feab406dc10 2024-09-12T03:20:33.772 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.771+0000 7feab1d9b640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7feab415e760 0x7feab406d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:33.772 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.771+0000 7feab1d9b640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7feab415e760 0x7feab406d6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:47008/0 (socket says 172.21.15.27:47008) 2024-09-12T03:20:33.772 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.771+0000 7feab1d9b640 1 -- 172.21.15.27:0/1721209999 learned_addr learned my addr 172.21.15.27:0/1721209999 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:20:33.772 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.771+0000 7feab2d9d640 1 --2- 172.21.15.27:0/1721209999 >> v2:172.21.15.45:3300/0 conn(0x7feab406dc10 0x7feab40b43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:33.772 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.771+0000 7feab1d9b640 1 -- 172.21.15.27:0/1721209999 >> v2:172.21.15.45:3300/0 conn(0x7feab406dc10 msgr2=0x7feab40b43f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:33.772 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.771+0000 7feab1d9b640 1 --2- 172.21.15.27:0/1721209999 >> v2:172.21.15.45:3300/0 conn(0x7feab406dc10 0x7feab40b43f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:33.772 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.771+0000 7feab1d9b640 1 -- 172.21.15.27:0/1721209999 >> v1:172.21.15.45:6789/0 conn(0x7feab40ae8d0 legacy=0x7feab40aecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:33.772 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.772+0000 7feab1d9b640 1 -- 172.21.15.27:0/1721209999 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feab4072080 con 0x7feab415e760 2024-09-12T03:20:33.773 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.772+0000 7feab1d9b640 1 --2- 172.21.15.27:0/1721209999 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feab415e760 0x7feab406d6d0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7feab40b35a0 tx=0x7feaa802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=926c16bea2e4704d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:33.773 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.772+0000 7feab159a640 1 -- 172.21.15.27:0/1721209999 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaa8037070 con 0x7feab415e760 2024-09-12T03:20:33.773 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.772+0000 7feab159a640 1 -- 172.21.15.27:0/1721209999 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feaa8032440 con 0x7feab415e760 2024-09-12T03:20:33.773 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.772+0000 7feab159a640 1 -- 172.21.15.27:0/1721209999 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaa8032810 con 0x7feab415e760 2024-09-12T03:20:33.773 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.773+0000 7feab8ee4640 1 -- 172.21.15.27:0/1721209999 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feab415e760 msgr2=0x7feab406d6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:33.774 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.773+0000 7feab8ee4640 1 --2- 172.21.15.27:0/1721209999 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feab415e760 0x7feab406d6d0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7feab40b35a0 tx=0x7feaa802a270 comp rx=0 tx=0).stop 2024-09-12T03:20:33.774 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.773+0000 7feab8ee4640 1 -- 172.21.15.27:0/1721209999 shutdown_connections 2024-09-12T03:20:33.774 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.773+0000 7feab8ee4640 1 --2- 172.21.15.27:0/1721209999 >> v2:172.21.15.45:3300/0 conn(0x7feab406dc10 0x7feab40b43f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:33.774 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.773+0000 7feab8ee4640 1 --2- 172.21.15.27:0/1721209999 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feab415e760 0x7feab406d6d0 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:33.774 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.773+0000 7feab8ee4640 1 -- 172.21.15.27:0/1721209999 >> 172.21.15.27:0/1721209999 conn(0x7feab40b2500 msgr2=0x7feab40b28e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:33.774 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.773+0000 7feab8ee4640 1 -- 172.21.15.27:0/1721209999 shutdown_connections 2024-09-12T03:20:33.774 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.773+0000 7feab8ee4640 1 -- 172.21.15.27:0/1721209999 wait complete. 2024-09-12T03:20:33.774 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.774+0000 7feab8ee4640 1 Processor -- start 2024-09-12T03:20:33.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.774+0000 7feab8ee4640 1 -- start start 2024-09-12T03:20:33.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.775+0000 7feab8ee4640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feab406dc10 0x7feab41f8500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:33.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.775+0000 7feab8ee4640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7feab40ae8d0 0x7feab4142660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:33.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.775+0000 7feab8ee4640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feab415e760 0x7feab4159370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:33.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.775+0000 7feab8ee4640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7feab40b4930 con 0x7feab415e760 2024-09-12T03:20:33.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.775+0000 7feab8ee4640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7feab4071490 con 0x7feab406dc10 2024-09-12T03:20:33.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.775+0000 7feab8ee4640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7feab40b4ab0 con 0x7feab40ae8d0 2024-09-12T03:20:33.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.775+0000 7feab2d9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feab415e760 0x7feab4159370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:33.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.775+0000 7feab2d9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feab415e760 0x7feab4159370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:47022/0 (socket says 172.21.15.27:47022) 2024-09-12T03:20:33.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.775+0000 7feab2d9d640 1 -- 172.21.15.27:0/3372559856 learned_addr learned my addr 172.21.15.27:0/3372559856 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:20:33.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.775+0000 7feab259c640 1 --2- 172.21.15.27:0/3372559856 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feab406dc10 0x7feab41f8500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:33.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.776+0000 7feab1d9b640 1 --2- 172.21.15.27:0/3372559856 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7feab40ae8d0 0x7feab4142660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:33.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.776+0000 7feab2d9d640 1 -- 172.21.15.27:0/3372559856 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7feab40ae8d0 msgr2=0x7feab4142660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:33.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.776+0000 7feab2d9d640 1 --2- 172.21.15.27:0/3372559856 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7feab40ae8d0 0x7feab4142660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:33.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.776+0000 7feab2d9d640 1 -- 172.21.15.27:0/3372559856 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feab406dc10 msgr2=0x7feab41f8500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:33.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.776+0000 7feab2d9d640 1 --2- 172.21.15.27:0/3372559856 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feab406dc10 0x7feab41f8500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:33.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.776+0000 7feab2d9d640 1 -- 172.21.15.27:0/3372559856 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feab41f8a40 con 0x7feab415e760 2024-09-12T03:20:33.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.776+0000 7feab259c640 1 --2- 172.21.15.27:0/3372559856 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feab406dc10 0x7feab41f8500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:20:33.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.776+0000 7feab2d9d640 1 --2- 172.21.15.27:0/3372559856 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feab415e760 0x7feab4159370 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7feaa400efc0 tx=0x7feaa400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:33.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.776+0000 7feaa2ffd640 1 -- 172.21.15.27:0/3372559856 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaa4018020 con 0x7feab415e760 2024-09-12T03:20:33.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.776+0000 7feaa2ffd640 1 -- 172.21.15.27:0/3372559856 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feaa4014480 con 0x7feab415e760 2024-09-12T03:20:33.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.776+0000 7feab8ee4640 1 -- 172.21.15.27:0/3372559856 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feab4144c60 con 0x7feab415e760 2024-09-12T03:20:33.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.776+0000 7feaa2ffd640 1 -- 172.21.15.27:0/3372559856 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaa4014870 con 0x7feab415e760 2024-09-12T03:20:33.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.776+0000 7feab8ee4640 1 -- 172.21.15.27:0/3372559856 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feab4145220 con 0x7feab415e760 2024-09-12T03:20:33.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.778+0000 7feaa2ffd640 1 -- 172.21.15.27:0/3372559856 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 90600+0+0 (secure 0 0 0) 0x7feaa4005d10 con 0x7feab415e760 2024-09-12T03:20:33.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.778+0000 7feaa2ffd640 1 --2- 172.21.15.27:0/3372559856 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fea9406bf50 0x7fea9406e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:33.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.779+0000 7feab259c640 1 --2- 172.21.15.27:0/3372559856 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fea9406bf50 0x7fea9406e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:33.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.779+0000 7feaa2ffd640 1 -- 172.21.15.27:0/3372559856 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 5709+0+0 (secure 0 0 0) 0x7feaa408bf50 con 0x7feab415e760 2024-09-12T03:20:33.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.779+0000 7feab259c640 1 --2- 172.21.15.27:0/3372559856 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fea9406bf50 0x7fea9406e410 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7feaac001700 tx=0x7feaac009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:33.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.779+0000 7feabbc79740 1 Processor -- start 2024-09-12T03:20:33.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.780+0000 7feabbc79740 1 -- start start 2024-09-12T03:20:33.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.780+0000 7feabbc79740 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x55de63cd52c0 0x55de63d1aa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:33.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.780+0000 7feabbc79740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55de63cd2a80 0x55de63cd56a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:33.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.780+0000 7feabbc79740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55de63cd6a50 0x55de63ccf8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:33.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.780+0000 7feabbc79740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55de63c37520 con 0x55de63cd6a50 2024-09-12T03:20:33.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.780+0000 7feabbc79740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55de63bdf3e0 con 0x55de63cd2a80 2024-09-12T03:20:33.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.780+0000 7feabbc79740 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x55de63c334e0 con 0x55de63cd52c0 2024-09-12T03:20:33.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.780+0000 7feab2d9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55de63cd6a50 0x55de63ccf8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:33.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.780+0000 7feab2d9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55de63cd6a50 0x55de63ccf8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:47026/0 (socket says 172.21.15.27:47026) 2024-09-12T03:20:33.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.780+0000 7feab2d9d640 1 -- 172.21.15.27:0/2575518330 learned_addr learned my addr 172.21.15.27:0/2575518330 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:20:33.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.780+0000 7feab259c640 1 --2- 172.21.15.27:0/2575518330 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x55de63cd52c0 0x55de63d1aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:33.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.781+0000 7feab1d9b640 1 --2- 172.21.15.27:0/2575518330 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55de63cd2a80 0x55de63cd56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:33.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.781+0000 7feab2d9d640 1 -- 172.21.15.27:0/2575518330 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x55de63cd52c0 msgr2=0x55de63d1aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:33.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.781+0000 7feab2d9d640 1 --2- 172.21.15.27:0/2575518330 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x55de63cd52c0 0x55de63d1aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:33.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.781+0000 7feab2d9d640 1 -- 172.21.15.27:0/2575518330 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55de63cd2a80 msgr2=0x55de63cd56a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:33.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.781+0000 7feab2d9d640 1 --2- 172.21.15.27:0/2575518330 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55de63cd2a80 0x55de63cd56a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:33.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.781+0000 7feab2d9d640 1 -- 172.21.15.27:0/2575518330 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55de63bfa2f0 con 0x55de63cd6a50 2024-09-12T03:20:33.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.781+0000 7feab1d9b640 1 --2- 172.21.15.27:0/2575518330 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55de63cd2a80 0x55de63cd56a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:20:33.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.781+0000 7feab2d9d640 1 --2- 172.21.15.27:0/2575518330 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55de63cd6a50 0x55de63ccf8b0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7feaa408a190 tx=0x7feaa4059fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89a16e29e40baedc server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:33.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.781+0000 7fea93fff640 1 -- 172.21.15.27:0/2575518330 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaa4072020 con 0x55de63cd6a50 2024-09-12T03:20:33.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.781+0000 7fea93fff640 1 -- 172.21.15.27:0/2575518330 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feaa4062bc0 con 0x55de63cd6a50 2024-09-12T03:20:33.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.781+0000 7fea93fff640 1 -- 172.21.15.27:0/2575518330 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaa4056650 con 0x55de63cd6a50 2024-09-12T03:20:33.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.782+0000 7feabbc79740 1 -- 172.21.15.27:0/2575518330 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55de63cd6a50 msgr2=0x55de63ccf8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:33.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.782+0000 7feabbc79740 1 --2- 172.21.15.27:0/2575518330 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55de63cd6a50 0x55de63ccf8b0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7feaa408a190 tx=0x7feaa4059fe0 comp rx=0 tx=0).stop 2024-09-12T03:20:33.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.782+0000 7feabbc79740 1 -- 172.21.15.27:0/2575518330 shutdown_connections 2024-09-12T03:20:33.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.782+0000 7feabbc79740 1 --2- 172.21.15.27:0/2575518330 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55de63cd6a50 0x55de63ccf8b0 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:33.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.782+0000 7feabbc79740 1 --2- 172.21.15.27:0/2575518330 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55de63cd2a80 0x55de63cd56a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:33.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.782+0000 7feabbc79740 1 --2- 172.21.15.27:0/2575518330 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x55de63cd52c0 0x55de63d1aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:33.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.782+0000 7feabbc79740 1 -- 172.21.15.27:0/2575518330 >> 172.21.15.27:0/2575518330 conn(0x55de63cd1a80 msgr2=0x55de63d19f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:33.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.782+0000 7feabbc79740 1 -- 172.21.15.27:0/2575518330 shutdown_connections 2024-09-12T03:20:33.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.782+0000 7feabbc79740 1 -- 172.21.15.27:0/2575518330 wait complete. 2024-09-12T03:20:33.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.782+0000 7feabbc79740 1 -- start start 2024-09-12T03:20:33.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.782+0000 7feabbc79740 1 Processor -- start 2024-09-12T03:20:33.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.783+0000 7feabbc79740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55de63cd52c0 0x55de63cff320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:33.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.783+0000 7feabbc79740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55de63cd6a50 0x55de63d17500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:33.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.783+0000 7feabbc79740 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x55de63d2a6b0 0x55de63d183a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:33.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.783+0000 7feabbc79740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55de63cd0f20 con 0x55de63cd6a50 2024-09-12T03:20:33.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.783+0000 7feabbc79740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55de63bc0450 con 0x55de63cd52c0 2024-09-12T03:20:33.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.783+0000 7feabbc79740 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x55de63d1b290 con 0x55de63d2a6b0 2024-09-12T03:20:33.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.783+0000 7feabbc79740 10 client.0 _ll_register_callbacks cb 0x55de63949ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:20:33.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.784+0000 7feab1d9b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55de63cd6a50 0x55de63d17500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:33.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.784+0000 7feab1d9b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55de63cd6a50 0x55de63d17500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:47042/0 (socket says 172.21.15.27:47042) 2024-09-12T03:20:33.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.784+0000 7feab1d9b640 1 -- 172.21.15.27:0/102334621 learned_addr learned my addr 172.21.15.27:0/102334621 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:20:33.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.784+0000 7feab259c640 1 --2- 172.21.15.27:0/102334621 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55de63cd52c0 0x55de63cff320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:33.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.784+0000 7feab2d9d640 1 --2- 172.21.15.27:0/102334621 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x55de63d2a6b0 0x55de63d183a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:33.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.784+0000 7feab1d9b640 1 -- 172.21.15.27:0/102334621 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x55de63d2a6b0 msgr2=0x55de63d183a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:33.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.784+0000 7feab1d9b640 1 --2- 172.21.15.27:0/102334621 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x55de63d2a6b0 0x55de63d183a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:33.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.784+0000 7feab1d9b640 1 -- 172.21.15.27:0/102334621 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55de63cd52c0 msgr2=0x55de63cff320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:33.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.784+0000 7feab1d9b640 1 --2- 172.21.15.27:0/102334621 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55de63cd52c0 0x55de63cff320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:33.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.784+0000 7feab1d9b640 1 -- 172.21.15.27:0/102334621 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55de6394a2d0 con 0x55de63cd6a50 2024-09-12T03:20:33.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.784+0000 7feab1d9b640 1 --2- 172.21.15.27:0/102334621 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55de63cd6a50 0x55de63d17500 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7feaa802a2b0 tx=0x7feaa8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=506e77ddc7469946 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:33.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.784+0000 7fea917fa640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-12T03:20:33.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.784+0000 7fea917fa640 1 -- 172.21.15.27:0/102334621 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaa8042070 con 0x55de63cd6a50 2024-09-12T03:20:33.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.784+0000 7fea917fa640 1 -- 172.21.15.27:0/102334621 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 5709+0+0 (secure 0 0 0) 0x7feaa8032d60 con 0x55de63cd6a50 2024-09-12T03:20:33.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.784+0000 7feabbc79740 1 -- 172.21.15.27:0/102334621 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feaa4062bc0 con 0x55de63cd6a50 2024-09-12T03:20:33.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.784+0000 7fea917fa640 1 -- 172.21.15.27:0/102334621 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7feaa804d050 con 0x55de63cd6a50 2024-09-12T03:20:33.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.784+0000 7feabbc79740 10 client.6606 fetch_fsmap learned FSMap version 46 2024-09-12T03:20:33.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.784+0000 7feabbc79740 1 -- 172.21.15.27:0/102334621 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=2+}) -- 0x55de63a146b0 con 0x55de63cd6a50 2024-09-12T03:20:33.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.784+0000 7fea917fa640 1 -- 172.21.15.27:0/102334621 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feaa4062bc0 con 0x55de63cd6a50 2024-09-12T03:20:33.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.785+0000 7fea917fa640 1 -- 172.21.15.27:0/102334621 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 46) ==== 3502+0+0 (secure 0 0 0) 0x7feaa8050020 con 0x55de63cd6a50 2024-09-12T03:20:33.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.785+0000 7feabbc79740 10 client.6606 fetch_fsmap finished waiting for FSMap version 46 2024-09-12T03:20:33.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.785+0000 7feabbc79740 10 client.6606 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-12T03:20:33.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.785+0000 7feabbc79740 20 client.6606 populate_metadata read hostname 'smithi027' 2024-09-12T03:20:33.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.785+0000 7feabbc79740 1 --2- 172.21.15.27:0/102334621 >> [v2:172.21.15.27:6833/3209963418,v1:172.21.15.27:6835/3209963418] conn(0x55de63cfd4e0 0x55de63cce910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:33.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.785+0000 7feabbc79740 4 client.6606 mds_command: new command op to 6486 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:20:33.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.785+0000 7feabbc79740 1 -- 172.21.15.27:0/102334621 --> [v2:172.21.15.27:6833/3209963418,v1:172.21.15.27:6835/3209963418] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55de63c44c50 con 0x55de63cfd4e0 2024-09-12T03:20:33.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.785+0000 7feab259c640 1 --2- 172.21.15.27:0/102334621 >> [v2:172.21.15.27:6833/3209963418,v1:172.21.15.27:6835/3209963418] conn(0x55de63cfd4e0 0x55de63cce910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:33.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.785+0000 7fea917fa640 10 client.6606 ms_handle_connect on v2:172.21.15.27:6833/3209963418 2024-09-12T03:20:33.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.785+0000 7feab259c640 1 --2- 172.21.15.27:0/102334621 >> [v2:172.21.15.27:6833/3209963418,v1:172.21.15.27:6835/3209963418] conn(0x55de63cfd4e0 0x55de63cce910 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=40939cedb3fa2dfd server_cookie=2b80ddd384c4098f in_seq=0 out_seq=0 2024-09-12T03:20:33.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.786+0000 7fea917fa640 1 -- 172.21.15.27:0/102334621 <== mds.0 v2:172.21.15.27:6833/3209963418 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55de63c44c50 con 0x55de63cfd4e0 2024-09-12T03:20:33.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.786+0000 7fea917fa640 10 client.6606 handle_command_reply: tid=0 2024-09-12T03:20:33.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.786+0000 7feabbc79740 1 client.6606 shutdown 2024-09-12T03:20:33.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.786+0000 7feabbc79740 1 -- 172.21.15.27:0/102334621 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55de63cd6a50 msgr2=0x55de63d17500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:33.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.786+0000 7feabbc79740 1 --2- 172.21.15.27:0/102334621 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55de63cd6a50 0x55de63d17500 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7feaa802a2b0 tx=0x7feaa8004730 comp rx=0 tx=0).stop 2024-09-12T03:20:33.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.786+0000 7feabbc79740 1 -- 172.21.15.27:0/102334621 shutdown_connections 2024-09-12T03:20:33.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.786+0000 7feabbc79740 1 --2- 172.21.15.27:0/102334621 >> [v2:172.21.15.27:6833/3209963418,v1:172.21.15.27:6835/3209963418] conn(0x55de63cfd4e0 0x55de63cce910 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:33.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.786+0000 7feabbc79740 1 --2- 172.21.15.27:0/102334621 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x55de63d2a6b0 0x55de63d183a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:33.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.786+0000 7feabbc79740 1 --2- 172.21.15.27:0/102334621 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55de63cd6a50 0x55de63d17500 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:33.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.786+0000 7fea917fa640 0 client.6606 ms_handle_reset on v2:172.21.15.27:6833/3209963418 2024-09-12T03:20:33.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.786+0000 7feabbc79740 1 --2- 172.21.15.27:0/102334621 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55de63cd52c0 0x55de63cff320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:33.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.786+0000 7feabbc79740 1 -- 172.21.15.27:0/102334621 >> 172.21.15.27:0/102334621 conn(0x55de63cd2a80 msgr2=0x55de63cd3020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:33.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.786+0000 7feabbc79740 1 -- 172.21.15.27:0/102334621 shutdown_connections 2024-09-12T03:20:33.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.786+0000 7feabbc79740 1 -- 172.21.15.27:0/102334621 wait complete. 2024-09-12T03:20:33.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.786+0000 7feabbc79740 20 client.6606 trim_cache size 0 max 16384 2024-09-12T03:20:33.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.804+0000 7feabbc79740 1 Processor -- start 2024-09-12T03:20:33.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.804+0000 7feabbc79740 1 -- start start 2024-09-12T03:20:33.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.805+0000 7feabbc79740 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x55de63cd52c0 0x55de63d2ba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:33.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.805+0000 7feabbc79740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55de63cd6a50 0x55de63d16fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:33.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.805+0000 7feabbc79740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55de63d2a6b0 0x55de63d8f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:33.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.805+0000 7feabbc79740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7feab41ffc80 con 0x55de63cd6a50 2024-09-12T03:20:33.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.805+0000 7feabbc79740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7feab4157070 con 0x55de63d2a6b0 2024-09-12T03:20:33.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.805+0000 7feabbc79740 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7feab41576c0 con 0x55de63cd52c0 2024-09-12T03:20:33.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.805+0000 7feab1d9b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55de63cd6a50 0x55de63d16fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:33.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.805+0000 7feab1d9b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55de63cd6a50 0x55de63d16fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:47052/0 (socket says 172.21.15.27:47052) 2024-09-12T03:20:33.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.805+0000 7feab1d9b640 1 -- 172.21.15.27:0/1013534372 learned_addr learned my addr 172.21.15.27:0/1013534372 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:20:33.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.805+0000 7feab1d9b640 1 -- 172.21.15.27:0/1013534372 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x55de63cd52c0 msgr2=0x55de63d2ba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:33.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.805+0000 7feab1d9b640 1 --2- 172.21.15.27:0/1013534372 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x55de63cd52c0 0x55de63d2ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:33.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.805+0000 7feab1d9b640 1 -- 172.21.15.27:0/1013534372 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55de63d2a6b0 msgr2=0x55de63d8f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:33.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.805+0000 7feab1d9b640 1 --2- 172.21.15.27:0/1013534372 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55de63d2a6b0 0x55de63d8f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:33.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.805+0000 7feab1d9b640 1 -- 172.21.15.27:0/1013534372 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55de63d1c350 con 0x55de63cd6a50 2024-09-12T03:20:33.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.806+0000 7feab1d9b640 1 --2- 172.21.15.27:0/1013534372 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55de63cd6a50 0x55de63d16fe0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7feaa80090b0 tx=0x7feaa803c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61c7520d429c0b4c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:33.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.806+0000 7fea937fe640 1 -- 172.21.15.27:0/1013534372 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaa8042040 con 0x55de63cd6a50 2024-09-12T03:20:33.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.806+0000 7fea937fe640 1 -- 172.21.15.27:0/1013534372 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feaa8032d60 con 0x55de63cd6a50 2024-09-12T03:20:33.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.806+0000 7fea937fe640 1 -- 172.21.15.27:0/1013534372 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaa8037040 con 0x55de63cd6a50 2024-09-12T03:20:33.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.806+0000 7feabbc79740 1 -- 172.21.15.27:0/1013534372 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55de63cd6a50 msgr2=0x55de63d16fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:33.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.806+0000 7feabbc79740 1 --2- 172.21.15.27:0/1013534372 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55de63cd6a50 0x55de63d16fe0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7feaa80090b0 tx=0x7feaa803c000 comp rx=0 tx=0).stop 2024-09-12T03:20:33.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.806+0000 7feabbc79740 1 -- 172.21.15.27:0/1013534372 shutdown_connections 2024-09-12T03:20:33.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.806+0000 7feabbc79740 1 --2- 172.21.15.27:0/1013534372 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55de63d2a6b0 0x55de63d8f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:33.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.806+0000 7feabbc79740 1 --2- 172.21.15.27:0/1013534372 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55de63cd6a50 0x55de63d16fe0 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:33.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.806+0000 7feabbc79740 1 --2- 172.21.15.27:0/1013534372 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x55de63cd52c0 0x55de63d2ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:33.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.806+0000 7feabbc79740 1 -- 172.21.15.27:0/1013534372 >> 172.21.15.27:0/1013534372 conn(0x55de63cd2a80 msgr2=0x55de63d3e3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:33.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.806+0000 7feabbc79740 1 -- 172.21.15.27:0/1013534372 shutdown_connections 2024-09-12T03:20:33.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.806+0000 7feabbc79740 1 -- 172.21.15.27:0/1013534372 wait complete. 2024-09-12T03:20:33.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.807+0000 7feabbc79740 1 -- start start 2024-09-12T03:20:33.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.807+0000 7feabbc79740 1 Processor -- start 2024-09-12T03:20:33.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.808+0000 7feabbc79740 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x55de63cd52c0 0x55de63d8bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:33.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.808+0000 7feabbc79740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55de63cd6a50 0x55de63d8f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:33.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.808+0000 7feabbc79740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55de63d2a6b0 0x55de63d19ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:33.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.808+0000 7feabbc79740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55de63cfd360 con 0x55de63d2a6b0 2024-09-12T03:20:33.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.808+0000 7feabbc79740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55de63d2b900 con 0x55de63cd6a50 2024-09-12T03:20:33.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.808+0000 7feabbc79740 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x55de63cfd920 con 0x55de63cd52c0 2024-09-12T03:20:33.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.808+0000 7feabbc79740 10 client.0 _ll_register_callbacks cb 0x7feaa8035070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:20:33.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.808+0000 7feab2d9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55de63d2a6b0 0x55de63d19ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:33.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.808+0000 7feab2d9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55de63d2a6b0 0x55de63d19ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:47056/0 (socket says 172.21.15.27:47056) 2024-09-12T03:20:33.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.808+0000 7feab2d9d640 1 -- 172.21.15.27:0/2956485481 learned_addr learned my addr 172.21.15.27:0/2956485481 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:20:33.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.808+0000 7feab1d9b640 1 --2- 172.21.15.27:0/2956485481 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55de63cd6a50 0x55de63d8f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:33.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.808+0000 7feab259c640 1 --2- 172.21.15.27:0/2956485481 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x55de63cd52c0 0x55de63d8bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:33.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.808+0000 7feab2d9d640 1 -- 172.21.15.27:0/2956485481 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x55de63cd52c0 msgr2=0x55de63d8bcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:33.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.808+0000 7feab2d9d640 1 --2- 172.21.15.27:0/2956485481 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x55de63cd52c0 0x55de63d8bcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:33.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.808+0000 7feab2d9d640 1 -- 172.21.15.27:0/2956485481 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55de63cd6a50 msgr2=0x55de63d8f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:33.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.808+0000 7feab2d9d640 1 --2- 172.21.15.27:0/2956485481 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55de63cd6a50 0x55de63d8f4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:33.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.808+0000 7feab2d9d640 1 -- 172.21.15.27:0/2956485481 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55de63d34930 con 0x55de63d2a6b0 2024-09-12T03:20:33.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.808+0000 7feab1d9b640 1 --2- 172.21.15.27:0/2956485481 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55de63cd6a50 0x55de63d8f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:20:33.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.808+0000 7feab2d9d640 1 --2- 172.21.15.27:0/2956485481 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55de63d2a6b0 0x55de63d19ac0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7feaa4058290 tx=0x7feaa4013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f37586fc8410065a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:33.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.808+0000 7fea917fa640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-12T03:20:33.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.809+0000 7fea917fa640 1 -- 172.21.15.27:0/2956485481 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaa4072040 con 0x55de63d2a6b0 2024-09-12T03:20:33.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.809+0000 7fea917fa640 1 -- 172.21.15.27:0/2956485481 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 5709+0+0 (secure 0 0 0) 0x7feaa4062930 con 0x55de63d2a6b0 2024-09-12T03:20:33.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.809+0000 7feabbc79740 1 -- 172.21.15.27:0/2956485481 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feaa8032d60 con 0x55de63d2a6b0 2024-09-12T03:20:33.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.809+0000 7fea917fa640 1 -- 172.21.15.27:0/2956485481 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7feaa4062bc0 con 0x55de63d2a6b0 2024-09-12T03:20:33.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.809+0000 7feabbc79740 10 client.6612 fetch_fsmap learned FSMap version 46 2024-09-12T03:20:33.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.809+0000 7feabbc79740 1 -- 172.21.15.27:0/2956485481 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=2+}) -- 0x55de63d8fe10 con 0x55de63d2a6b0 2024-09-12T03:20:33.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.809+0000 7fea917fa640 1 -- 172.21.15.27:0/2956485481 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feaa8032d60 con 0x55de63d2a6b0 2024-09-12T03:20:33.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.809+0000 7fea917fa640 1 -- 172.21.15.27:0/2956485481 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 46) ==== 3502+0+0 (secure 0 0 0) 0x7feaa4083bc0 con 0x55de63d2a6b0 2024-09-12T03:20:33.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.809+0000 7feabbc79740 10 client.6612 fetch_fsmap finished waiting for FSMap version 46 2024-09-12T03:20:33.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.809+0000 7feabbc79740 10 client.6612 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-12T03:20:33.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.809+0000 7feabbc79740 20 client.6612 populate_metadata read hostname 'smithi027' 2024-09-12T03:20:33.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.809+0000 7feabbc79740 1 --2- 172.21.15.27:0/2956485481 >> [v2:172.21.15.27:6833/3209963418,v1:172.21.15.27:6835/3209963418] conn(0x55de63d1a0c0 0x55de63cd3a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:33.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.809+0000 7feabbc79740 4 client.6612 mds_command: new command op to 6486 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-12T03:20:33.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.809+0000 7feabbc79740 1 -- 172.21.15.27:0/2956485481 --> [v2:172.21.15.27:6833/3209963418,v1:172.21.15.27:6835/3209963418] -- command(tid 0: {"prefix": "status"}) -- 0x55de63d00af0 con 0x55de63d1a0c0 2024-09-12T03:20:33.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.809+0000 7feab259c640 1 --2- 172.21.15.27:0/2956485481 >> [v2:172.21.15.27:6833/3209963418,v1:172.21.15.27:6835/3209963418] conn(0x55de63d1a0c0 0x55de63cd3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:33.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.809+0000 7feab259c640 1 --2- 172.21.15.27:0/2956485481 >> [v2:172.21.15.27:6833/3209963418,v1:172.21.15.27:6835/3209963418] conn(0x55de63d1a0c0 0x55de63cd3a30 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da90d0eec8ca22c5 server_cookie=990d627797440594 in_seq=0 out_seq=0 2024-09-12T03:20:33.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.809+0000 7fea917fa640 10 client.6612 ms_handle_connect on v2:172.21.15.27:6833/3209963418 2024-09-12T03:20:33.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.810+0000 7fea917fa640 1 -- 172.21.15.27:0/2956485481 <== mds.0 v2:172.21.15.27:6833/3209963418 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x55de63d00af0 con 0x55de63d1a0c0 2024-09-12T03:20:33.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.810+0000 7fea917fa640 10 client.6612 handle_command_reply: tid=0 2024-09-12T03:20:33.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.810+0000 7feabbc79740 1 client.6612 shutdown 2024-09-12T03:20:33.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.810+0000 7feabbc79740 1 -- 172.21.15.27:0/2956485481 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55de63d2a6b0 msgr2=0x55de63d19ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:33.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.810+0000 7feabbc79740 1 --2- 172.21.15.27:0/2956485481 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55de63d2a6b0 0x55de63d19ac0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7feaa4058290 tx=0x7feaa4013000 comp rx=0 tx=0).stop 2024-09-12T03:20:33.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.810+0000 7feabbc79740 1 -- 172.21.15.27:0/2956485481 shutdown_connections 2024-09-12T03:20:33.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.810+0000 7feabbc79740 1 --2- 172.21.15.27:0/2956485481 >> [v2:172.21.15.27:6833/3209963418,v1:172.21.15.27:6835/3209963418] conn(0x55de63d1a0c0 0x55de63cd3a30 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:33.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.810+0000 7feabbc79740 1 --2- 172.21.15.27:0/2956485481 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55de63d2a6b0 0x55de63d19ac0 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:33.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.810+0000 7feabbc79740 1 --2- 172.21.15.27:0/2956485481 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55de63cd6a50 0x55de63d8f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:33.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.810+0000 7feabbc79740 1 --2- 172.21.15.27:0/2956485481 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x55de63cd52c0 0x55de63d8bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:33.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.810+0000 7fea917fa640 0 client.6612 ms_handle_reset on v2:172.21.15.27:6833/3209963418 2024-09-12T03:20:33.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.810+0000 7feabbc79740 1 -- 172.21.15.27:0/2956485481 >> 172.21.15.27:0/2956485481 conn(0x55de63cd2a80 msgr2=0x55de63d16fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:33.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.810+0000 7feabbc79740 1 -- 172.21.15.27:0/2956485481 shutdown_connections 2024-09-12T03:20:33.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.810+0000 7feabbc79740 1 -- 172.21.15.27:0/2956485481 wait complete. 2024-09-12T03:20:33.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.810+0000 7feabbc79740 20 client.6612 trim_cache size 0 max 16384 2024-09-12T03:20:33.817 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-12T03:20:33.817 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_fsid": "d2df504e-2647-4385-a5db-9fdf69f487a0", 2024-09-12T03:20:33.817 INFO:teuthology.orchestra.run.smithi027.stdout: "whoami": 0, 2024-09-12T03:20:33.817 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 6486, 2024-09-12T03:20:33.817 INFO:teuthology.orchestra.run.smithi027.stdout: "want_state": "up:active", 2024-09-12T03:20:33.817 INFO:teuthology.orchestra.run.smithi027.stdout: "state": "up:active", 2024-09-12T03:20:33.818 INFO:teuthology.orchestra.run.smithi027.stdout: "fs_name": "cephfs", 2024-09-12T03:20:33.818 INFO:teuthology.orchestra.run.smithi027.stdout: "rank_uptime": 4.6524235369999998, 2024-09-12T03:20:33.818 INFO:teuthology.orchestra.run.smithi027.stdout: "mdsmap_epoch": 46, 2024-09-12T03:20:33.818 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch": 81, 2024-09-12T03:20:33.818 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch_barrier": 81, 2024-09-12T03:20:33.818 INFO:teuthology.orchestra.run.smithi027.stdout: "uptime": 9.6891737310000003 2024-09-12T03:20:33.818 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-12T03:20:33.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.810+0000 7feab8ee4640 1 -- 172.21.15.27:0/3372559856 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fea9406bf50 msgr2=0x7fea9406e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:33.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.810+0000 7feab8ee4640 1 --2- 172.21.15.27:0/3372559856 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fea9406bf50 0x7fea9406e410 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7feaac001700 tx=0x7feaac009040 comp rx=0 tx=0).stop 2024-09-12T03:20:33.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.811+0000 7feab8ee4640 1 -- 172.21.15.27:0/3372559856 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feab415e760 msgr2=0x7feab4159370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:33.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.811+0000 7feab8ee4640 1 --2- 172.21.15.27:0/3372559856 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feab415e760 0x7feab4159370 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7feaa400efc0 tx=0x7feaa400c490 comp rx=0 tx=0).stop 2024-09-12T03:20:33.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.811+0000 7feab8ee4640 1 -- 172.21.15.27:0/3372559856 shutdown_connections 2024-09-12T03:20:33.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.811+0000 7feab8ee4640 1 --2- 172.21.15.27:0/3372559856 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fea9406bf50 0x7fea9406e410 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:33.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.811+0000 7feab8ee4640 1 --2- 172.21.15.27:0/3372559856 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feab415e760 0x7feab4159370 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:33.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.811+0000 7feab8ee4640 1 --2- 172.21.15.27:0/3372559856 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7feab40ae8d0 0x7feab4142660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:33.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.811+0000 7feab8ee4640 1 --2- 172.21.15.27:0/3372559856 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feab406dc10 0x7feab41f8500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:33.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.811+0000 7feab8ee4640 1 -- 172.21.15.27:0/3372559856 >> 172.21.15.27:0/3372559856 conn(0x7feab40b2500 msgr2=0x7feab40b0c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:33.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.811+0000 7feab8ee4640 1 -- 172.21.15.27:0/3372559856 shutdown_connections 2024-09-12T03:20:33.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.811+0000 7feab8ee4640 1 -- 172.21.15.27:0/3372559856 wait complete. 2024-09-12T03:20:33.821 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-12T03:20:33.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.924+0000 7f1f3ef31640 1 Processor -- start 2024-09-12T03:20:33.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.924+0000 7f1f3ef31640 1 -- start start 2024-09-12T03:20:33.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.925+0000 7f1f3ef31640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f1f380d5200 0x7f1f380d1010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:33.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.925+0000 7f1f3ef31640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f1f380d1550 0x7f1f380b8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:33.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.925+0000 7f1f3ef31640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1f380d3760 con 0x7f1f3815eb90 2024-09-12T03:20:33.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.925+0000 7f1f3ef31640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f1f380b7d20 con 0x7f1f380d5200 2024-09-12T03:20:33.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.925+0000 7f1f3ef31640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f1f380d42e0 con 0x7f1f380d1550 2024-09-12T03:20:33.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.925+0000 7f1f3cccd640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f1f3815eb90 0x7f1f380d2df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:57700/0 (socket says 172.21.15.27:57700) 2024-09-12T03:20:33.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.925+0000 7f1f37fff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f1f380d5200 0x7f1f380d1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:33.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.925+0000 7f1f3cccd640 1 -- 172.21.15.27:0/1426305071 learned_addr learned my addr 172.21.15.27:0/1426305071 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:20:33.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.926+0000 7f1f3d4ce640 1 --2- 172.21.15.27:0/1426305071 >> v2:172.21.15.45:3300/0 conn(0x7f1f380d1550 0x7f1f380b8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:33.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.926+0000 7f1f377fe640 1 -- 172.21.15.27:0/1426305071 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2336213000 0 0) 0x7f1f380d3760 con 0x7f1f3815eb90 2024-09-12T03:20:33.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.926+0000 7f1f377fe640 1 -- 172.21.15.27:0/1426305071 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1f24003520 con 0x7f1f3815eb90 2024-09-12T03:20:33.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.926+0000 7f1f37fff640 1 -- 172.21.15.27:0/1426305071 >> v2:172.21.15.45:3300/0 conn(0x7f1f380d1550 msgr2=0x7f1f380b8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:33.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.926+0000 7f1f37fff640 1 --2- 172.21.15.27:0/1426305071 >> v2:172.21.15.45:3300/0 conn(0x7f1f380d1550 0x7f1f380b8190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:33.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.926+0000 7f1f37fff640 1 -- 172.21.15.27:0/1426305071 >> v1:172.21.15.27:6789/0 conn(0x7f1f3815eb90 legacy=0x7f1f380d2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:33.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.926+0000 7f1f37fff640 1 -- 172.21.15.27:0/1426305071 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f380d3a40 con 0x7f1f380d5200 2024-09-12T03:20:33.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.927+0000 7f1f37fff640 1 --2- 172.21.15.27:0/1426305071 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1f380d5200 0x7f1f380d1010 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f1f380c8270 tx=0x7f1f2802a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=920559d1f284ce07 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:33.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.927+0000 7f1f377fe640 1 -- 172.21.15.27:0/1426305071 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f28007740 con 0x7f1f380d5200 2024-09-12T03:20:33.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.927+0000 7f1f377fe640 1 -- 172.21.15.27:0/1426305071 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f280078e0 con 0x7f1f380d5200 2024-09-12T03:20:33.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.927+0000 7f1f377fe640 1 -- 172.21.15.27:0/1426305071 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f28007cb0 con 0x7f1f380d5200 2024-09-12T03:20:33.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.927+0000 7f1f3ef31640 1 -- 172.21.15.27:0/1426305071 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1f380d5200 msgr2=0x7f1f380d1010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:33.942 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.927+0000 7f1f3ef31640 1 --2- 172.21.15.27:0/1426305071 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1f380d5200 0x7f1f380d1010 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f1f380c8270 tx=0x7f1f2802a680 comp rx=0 tx=0).stop 2024-09-12T03:20:33.942 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.927+0000 7f1f3ef31640 1 -- 172.21.15.27:0/1426305071 shutdown_connections 2024-09-12T03:20:33.942 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.927+0000 7f1f3ef31640 1 --2- 172.21.15.27:0/1426305071 >> v2:172.21.15.45:3300/0 conn(0x7f1f380d1550 0x7f1f380b8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:33.942 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.927+0000 7f1f3ef31640 1 --2- 172.21.15.27:0/1426305071 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1f380d5200 0x7f1f380d1010 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:33.943 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.927+0000 7f1f3ef31640 1 -- 172.21.15.27:0/1426305071 >> 172.21.15.27:0/1426305071 conn(0x7f1f3806dce0 msgr2=0x7f1f3806e0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:33.943 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.927+0000 7f1f3ef31640 1 -- 172.21.15.27:0/1426305071 shutdown_connections 2024-09-12T03:20:33.943 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.927+0000 7f1f3ef31640 1 -- 172.21.15.27:0/1426305071 wait complete. 2024-09-12T03:20:33.943 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.928+0000 7f1f3ef31640 1 Processor -- start 2024-09-12T03:20:33.943 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.928+0000 7f1f3ef31640 1 -- start start 2024-09-12T03:20:33.943 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.929+0000 7f1f3ef31640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1f380d1550 0x7f1f380c7700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:33.943 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.929+0000 7f1f3ef31640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1f380d5200 0x7f1f381f7a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:33.943 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.929+0000 7f1f3ef31640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1f3815eb90 0x7f1f381f7f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:33.943 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.929+0000 7f1f3ef31640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f1f380d4000 con 0x7f1f380d1550 2024-09-12T03:20:33.943 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.929+0000 7f1f3ef31640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f1f380d3500 con 0x7f1f3815eb90 2024-09-12T03:20:33.943 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.929+0000 7f1f3ef31640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f1f380d1960 con 0x7f1f380d5200 2024-09-12T03:20:33.943 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.929+0000 7f1f3cccd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1f380d1550 0x7f1f380c7700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:33.944 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.929+0000 7f1f3cccd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1f380d1550 0x7f1f380c7700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:47076/0 (socket says 172.21.15.27:47076) 2024-09-12T03:20:33.944 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.929+0000 7f1f3cccd640 1 -- 172.21.15.27:0/831688013 learned_addr learned my addr 172.21.15.27:0/831688013 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:20:33.944 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.930+0000 7f1f3d4ce640 1 --2- 172.21.15.27:0/831688013 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1f3815eb90 0x7f1f381f7f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:33.944 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.930+0000 7f1f37fff640 1 --2- 172.21.15.27:0/831688013 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1f380d5200 0x7f1f381f7a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:33.944 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.930+0000 7f1f3cccd640 1 -- 172.21.15.27:0/831688013 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1f380d5200 msgr2=0x7f1f381f7a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:33.944 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.930+0000 7f1f3cccd640 1 --2- 172.21.15.27:0/831688013 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1f380d5200 0x7f1f381f7a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:33.944 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.930+0000 7f1f3cccd640 1 -- 172.21.15.27:0/831688013 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1f3815eb90 msgr2=0x7f1f381f7f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:33.944 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.930+0000 7f1f3cccd640 1 --2- 172.21.15.27:0/831688013 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1f3815eb90 0x7f1f381f7f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:33.944 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.930+0000 7f1f3cccd640 1 -- 172.21.15.27:0/831688013 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f3815b420 con 0x7f1f380d1550 2024-09-12T03:20:33.944 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.930+0000 7f1f3cccd640 1 --2- 172.21.15.27:0/831688013 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1f380d1550 0x7f1f380c7700 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f1f3000c460 tx=0x7f1f3000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:33.944 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.930+0000 7f1f357fa640 1 -- 172.21.15.27:0/831688013 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f30017070 con 0x7f1f380d1550 2024-09-12T03:20:33.944 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.930+0000 7f1f357fa640 1 -- 172.21.15.27:0/831688013 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f30005ce0 con 0x7f1f380d1550 2024-09-12T03:20:33.944 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.930+0000 7f1f3ef31640 1 -- 172.21.15.27:0/831688013 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f3815b700 con 0x7f1f380d1550 2024-09-12T03:20:33.944 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.930+0000 7f1f357fa640 1 -- 172.21.15.27:0/831688013 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f30007290 con 0x7f1f380d1550 2024-09-12T03:20:33.945 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.930+0000 7f1f3ef31640 1 -- 172.21.15.27:0/831688013 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f381588a0 con 0x7f1f380d1550 2024-09-12T03:20:33.945 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.931+0000 7f1f3ef31640 1 -- 172.21.15.27:0/831688013 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f3815f090 con 0x7f1f380d1550 2024-09-12T03:20:33.945 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.932+0000 7f1f357fa640 1 -- 172.21.15.27:0/831688013 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 90600+0+0 (secure 0 0 0) 0x7f1f3001a070 con 0x7f1f380d1550 2024-09-12T03:20:33.945 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.932+0000 7f1f357fa640 1 --2- 172.21.15.27:0/831688013 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f1f1806bf50 0x7f1f1806e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:33.945 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.932+0000 7f1f357fa640 1 -- 172.21.15.27:0/831688013 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 5709+0+0 (secure 0 0 0) 0x7f1f3008a320 con 0x7f1f380d1550 2024-09-12T03:20:33.945 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.932+0000 7f1f37fff640 1 --2- 172.21.15.27:0/831688013 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f1f1806bf50 0x7f1f1806e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:33.945 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.933+0000 7f1f37fff640 1 --2- 172.21.15.27:0/831688013 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f1f1806bf50 0x7f1f1806e410 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f1f28034000 tx=0x7f1f2802f000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:33.945 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:33.937+0000 7f1f357fa640 1 -- 172.21.15.27:0/831688013 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f1f30058570 con 0x7f1f380d1550 2024-09-12T03:20:34.147 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:34.147+0000 7f1f3ef31640 1 -- 172.21.15.27:0/831688013 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f1f380cfe90 con 0x7f1f380d1550 2024-09-12T03:20:34.148 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:34.147+0000 7f1f357fa640 1 -- 172.21.15.27:0/831688013 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v46) ==== 53+0+73 (secure 0 0 0) 0x7f1f3005c870 con 0x7f1f380d1550 2024-09-12T03:20:34.148 INFO:teuthology.orchestra.run.smithi027.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-12T03:20:34.150 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:34.149+0000 7f1f3ef31640 1 -- 172.21.15.27:0/831688013 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f1f1806bf50 msgr2=0x7f1f1806e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:34.150 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:34.149+0000 7f1f3ef31640 1 --2- 172.21.15.27:0/831688013 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f1f1806bf50 0x7f1f1806e410 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f1f28034000 tx=0x7f1f2802f000 comp rx=0 tx=0).stop 2024-09-12T03:20:34.150 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:34.149+0000 7f1f3ef31640 1 -- 172.21.15.27:0/831688013 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1f380d1550 msgr2=0x7f1f380c7700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:34.150 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:34.149+0000 7f1f3ef31640 1 --2- 172.21.15.27:0/831688013 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1f380d1550 0x7f1f380c7700 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f1f3000c460 tx=0x7f1f3000c920 comp rx=0 tx=0).stop 2024-09-12T03:20:34.150 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:34.149+0000 7f1f3ef31640 1 -- 172.21.15.27:0/831688013 shutdown_connections 2024-09-12T03:20:34.150 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:34.149+0000 7f1f3ef31640 1 --2- 172.21.15.27:0/831688013 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1f3815eb90 0x7f1f381f7f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:34.150 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:34.149+0000 7f1f3ef31640 1 --2- 172.21.15.27:0/831688013 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1f380d5200 0x7f1f381f7a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:34.150 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:34.149+0000 7f1f3ef31640 1 --2- 172.21.15.27:0/831688013 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f1f1806bf50 0x7f1f1806e410 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:34.150 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:34.149+0000 7f1f3ef31640 1 --2- 172.21.15.27:0/831688013 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1f380d1550 0x7f1f380c7700 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:34.151 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:34.149+0000 7f1f3ef31640 1 -- 172.21.15.27:0/831688013 >> 172.21.15.27:0/831688013 conn(0x7f1f3806dce0 msgr2=0x7f1f380b91a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:34.151 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:34.149+0000 7f1f3ef31640 1 -- 172.21.15.27:0/831688013 shutdown_connections 2024-09-12T03:20:34.151 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:34.149+0000 7f1f3ef31640 1 -- 172.21.15.27:0/831688013 wait complete. 2024-09-12T03:20:34.161 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-12T03:20:34.161 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-12T03:20:34.161 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi027.front.sepia.ceph.com 2024-09-12T03:20:34.161 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-12T03:20:34.161 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-12T03:20:34.161 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-12T03:20:34.161 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-12T03:20:34.161 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-12T03:20:34.162 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-12T03:20:34.162 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:20:34.162 DEBUG:teuthology.orchestra.run.smithi027:> ip addr 2024-09-12T03:20:34.181 INFO:teuthology.orchestra.run.smithi027.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-12T03:20:34.181 INFO:teuthology.orchestra.run.smithi027.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-12T03:20:34.181 INFO:teuthology.orchestra.run.smithi027.stdout: inet 127.0.0.1/8 scope host lo 2024-09-12T03:20:34.181 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:20:34.181 INFO:teuthology.orchestra.run.smithi027.stdout: inet6 ::1/128 scope host 2024-09-12T03:20:34.181 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:20:34.181 INFO:teuthology.orchestra.run.smithi027.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-12T03:20:34.181 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether 0c:c4:7a:6b:fe:c8 brd ff:ff:ff:ff:ff:ff 2024-09-12T03:20:34.182 INFO:teuthology.orchestra.run.smithi027.stdout: altname enp7s0f0 2024-09-12T03:20:34.182 INFO:teuthology.orchestra.run.smithi027.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-12T03:20:34.182 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether 0c:c4:7a:6b:fe:c9 brd ff:ff:ff:ff:ff:ff 2024-09-12T03:20:34.182 INFO:teuthology.orchestra.run.smithi027.stdout: altname enp7s0f1 2024-09-12T03:20:34.182 INFO:teuthology.orchestra.run.smithi027.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-12T03:20:34.182 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether ac:1f:6b:f8:39:5a brd ff:ff:ff:ff:ff:ff 2024-09-12T03:20:34.182 INFO:teuthology.orchestra.run.smithi027.stdout: altname enp1s0f0 2024-09-12T03:20:34.182 INFO:teuthology.orchestra.run.smithi027.stdout: inet 172.21.15.27/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-12T03:20:34.182 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft 42555sec preferred_lft 42555sec 2024-09-12T03:20:34.182 INFO:teuthology.orchestra.run.smithi027.stdout: inet6 fe80::ae1f:6bff:fef8:395a/64 scope link 2024-09-12T03:20:34.182 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:20:34.182 INFO:teuthology.orchestra.run.smithi027.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-12T03:20:34.182 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether ac:1f:6b:f8:39:5b brd ff:ff:ff:ff:ff:ff 2024-09-12T03:20:34.182 INFO:teuthology.orchestra.run.smithi027.stdout: altname enp1s0f1 2024-09-12T03:20:34.183 INFO:teuthology.orchestra.run.smithi027.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-12T03:20:34.183 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether 2e:7f:89:6c:33:a0 brd ff:ff:ff:ff:ff:ff 2024-09-12T03:20:34.183 INFO:teuthology.orchestra.run.smithi027.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-12T03:20:34.183 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:20:34.183 INFO:teuthology.orchestra.run.smithi027.stdout: inet6 fe80::c84b:19ff:fef0:ca74/64 scope link 2024-09-12T03:20:34.183 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:20:34.183 INFO:teuthology.orchestra.run.smithi027.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-12T03:20:34.183 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether 42:40:d7:5c:7a:8e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-12T03:20:34.183 INFO:teuthology.orchestra.run.smithi027.stdout: inet6 fe80::4040:d7ff:fe5c:7a8e/64 scope link 2024-09-12T03:20:34.183 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:20:34.183 INFO:teuthology.orchestra.run.smithi027.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-12T03:20:34.183 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether 2e:7f:89:6c:33:a0 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-12T03:20:34.183 INFO:teuthology.orchestra.run.smithi027.stdout: inet6 fe80::2c7f:89ff:fe6c:33a0/64 scope link 2024-09-12T03:20:34.184 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:20:34.184 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:20:34.184 DEBUG:teuthology.orchestra.run.smithi027:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-12T03:20:34.246 INFO:teuthology.orchestra.run.smithi027.stdout:1 2024-09-12T03:20:34.248 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:20:34.248 DEBUG:teuthology.orchestra.run.smithi027:> route 2024-09-12T03:20:34.304 INFO:teuthology.orchestra.run.smithi027.stdout:Kernel IP routing table 2024-09-12T03:20:34.304 INFO:teuthology.orchestra.run.smithi027.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-12T03:20:34.305 INFO:teuthology.orchestra.run.smithi027.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-12T03:20:34.305 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-12T03:20:34.305 INFO:teuthology.orchestra.run.smithi027.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-12T03:20:34.305 DEBUG:teuthology.orchestra.run.smithi027:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-12T03:20:34.305 DEBUG:teuthology.orchestra.run.smithi027:> set -e 2024-09-12T03:20:34.305 DEBUG:teuthology.orchestra.run.smithi027:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-12T03:20:34.305 DEBUG:teuthology.orchestra.run.smithi027:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-12T03:20:34.305 DEBUG:teuthology.orchestra.run.smithi027:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-12T03:20:34.305 DEBUG:teuthology.orchestra.run.smithi027:> ') 2024-09-12T03:20:34.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:34 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T03:20:34.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T03:20:34.434 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:20:34.434 DEBUG:teuthology.orchestra.run.smithi027:> ip netns list 2024-09-12T03:20:34.450 INFO:teuthology.orchestra.run.smithi027.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-12T03:20:34.450 INFO:teuthology.orchestra.run.smithi027.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-12T03:20:34.450 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-12T03:20:34.450 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:20:34.450 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip link set brx.0 up 2024-09-12T03:20:34.516 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-12T03:20:34.516 DEBUG:teuthology.orchestra.run.smithi027:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-12T03:20:34.572 INFO:teuthology.orchestra.run.smithi027.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-12T03:20:34.572 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-12T03:20:34.572 DEBUG:teuthology.orchestra.run.smithi027:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-12T03:20:34.627 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-09-12T03:20:34.627 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:20:34.628 DEBUG:teuthology.orchestra.run.smithi027:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-12T03:20:34.712 INFO:teuthology.orchestra.run.smithi027.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-12T03:20:34.713 INFO:teuthology.orchestra.run.smithi027.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-12T03:20:34.720 INFO:teuthology.orchestra.run.smithi027.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-12T03:20:34.721 INFO:teuthology.orchestra.run.smithi027.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-12T03:20:34.721 INFO:teuthology.orchestra.run.smithi027.stdout:mount.ceph: resolved to: "172.21.15.27:3300,172.21.15.45:3300,172.21.15.45:3301" 2024-09-12T03:20:34.721 INFO:teuthology.orchestra.run.smithi027.stdout:mount.ceph: trying mount with new device syntax: 0@d2df504e-2647-4385-a5db-9fdf69f487a0.cephfs=/ 2024-09-12T03:20:34.721 INFO:teuthology.orchestra.run.smithi027.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.27:3300/172.21.15.45:3300/172.21.15.45:3301" will pass to kernel 2024-09-12T03:20:34.721 INFO:teuthology.orchestra.run.smithi027.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-12T03:20:34.721 INFO:teuthology.orchestra.run.smithi027.stdout: You just mounted a file system that supports labels which does not 2024-09-12T03:20:34.722 INFO:teuthology.orchestra.run.smithi027.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-12T03:20:34.722 INFO:teuthology.orchestra.run.smithi027.stdout: applications will generate AVC messages and not be allowed access to 2024-09-12T03:20:34.722 INFO:teuthology.orchestra.run.smithi027.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-12T03:20:34.723 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-12T03:20:34.723 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:20:34.724 DEBUG:teuthology.orchestra.run.smithi027:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-12T03:20:34.750 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-12T03:20:34.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:34 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-12T03:20:34.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T03:20:34.854 INFO:teuthology.orchestra.run.smithi027.stdout:client5744 2024-09-12T03:20:34.855 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-12T03:20:34.855 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T03:20:34.871 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-12T03:20:35.013 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:35.013+0000 7f1e62978640 1 Processor -- start 2024-09-12T03:20:35.014 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:35.014+0000 7f1e62978640 1 -- start start 2024-09-12T03:20:35.015 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:35.014+0000 7f1e62978640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f1e5c15eb90 0x7f1e5c0d2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:35.015 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:35.014+0000 7f1e62978640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f1e5c0d1720 0x7f1e5c0b8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:35.015 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:35.014+0000 7f1e62978640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1e5c0d3760 con 0x7f1e5c0d5200 2024-09-12T03:20:35.015 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:35.014+0000 7f1e62978640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f1e5c0b7d20 con 0x7f1e5c0d1720 2024-09-12T03:20:35.015 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:35.014+0000 7f1e62978640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f1e5c0d42e0 con 0x7f1e5c15eb90 2024-09-12T03:20:35.015 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:35.015+0000 7f1e60f15640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f1e5c0d1720 0x7f1e5c0b8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:35.015 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:35.015+0000 7f1e60f15640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f1e5c0d1720 0x7f1e5c0b8190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:47078/0 (socket says 172.21.15.27:47078) 2024-09-12T03:20:35.015 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:35.015+0000 7f1e60f15640 1 -- 172.21.15.27:0/1289962129 learned_addr learned my addr 172.21.15.27:0/1289962129 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:20:35.016 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:35.015+0000 7f1e5bfff640 1 --2- 172.21.15.27:0/1289962129 >> v2:172.21.15.45:3300/0 conn(0x7f1e5c15eb90 0x7f1e5c0d2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:35.016 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:35.015+0000 7f1e60f15640 1 -- 172.21.15.27:0/1289962129 >> v2:172.21.15.45:3300/0 conn(0x7f1e5c15eb90 msgr2=0x7f1e5c0d2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:35.016 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:35.015+0000 7f1e60f15640 1 --2- 172.21.15.27:0/1289962129 >> v2:172.21.15.45:3300/0 conn(0x7f1e5c15eb90 0x7f1e5c0d2df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:35.016 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:35.015+0000 7f1e60f15640 1 -- 172.21.15.27:0/1289962129 >> v1:172.21.15.45:6789/0 conn(0x7f1e5c0d5200 legacy=0x7f1e5c0d1010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:35.016 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:35.015+0000 7f1e60f15640 1 -- 172.21.15.27:0/1289962129 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e5c0d3a40 con 0x7f1e5c0d1720 2024-09-12T03:20:35.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:35.016+0000 7f1e60f15640 1 --2- 172.21.15.27:0/1289962129 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1e5c0d1720 0x7f1e5c0b8190 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f1e5c075560 tx=0x7f1e5402a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70c55dfd18275e0d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:35.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:35.016+0000 7f1e5affd640 1 -- 172.21.15.27:0/1289962129 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e54036070 con 0x7f1e5c0d1720 2024-09-12T03:20:35.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:35.016+0000 7f1e5affd640 1 -- 172.21.15.27:0/1289962129 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e54031600 con 0x7f1e5c0d1720 2024-09-12T03:20:35.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:35.016+0000 7f1e5affd640 1 -- 172.21.15.27:0/1289962129 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e540319d0 con 0x7f1e5c0d1720 2024-09-12T03:20:35.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:35.016+0000 7f1e62978640 1 -- 172.21.15.27:0/1289962129 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1e5c0d1720 msgr2=0x7f1e5c0b8190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:35.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:35.016+0000 7f1e62978640 1 --2- 172.21.15.27:0/1289962129 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1e5c0d1720 0x7f1e5c0b8190 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f1e5c075560 tx=0x7f1e5402a730 comp rx=0 tx=0).stop 2024-09-12T03:20:35.018 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:35.016+0000 7f1e62978640 1 -- 172.21.15.27:0/1289962129 shutdown_connections 2024-09-12T03:20:35.018 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:35.016+0000 7f1e62978640 1 --2- 172.21.15.27:0/1289962129 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1e5c0d1720 0x7f1e5c0b8190 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:35.018 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:35.016+0000 7f1e62978640 1 --2- 172.21.15.27:0/1289962129 >> v2:172.21.15.45:3300/0 conn(0x7f1e5c15eb90 0x7f1e5c0d2df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:35.018 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:35.016+0000 7f1e62978640 1 -- 172.21.15.27:0/1289962129 >> 172.21.15.27:0/1289962129 conn(0x7f1e5c06dce0 msgr2=0x7f1e5c06e0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:35.018 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:35.017+0000 7f1e62978640 1 -- 172.21.15.27:0/1289962129 shutdown_connections 2024-09-12T03:20:35.018 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:35.017+0000 7f1e62978640 1 -- 172.21.15.27:0/1289962129 wait complete. 2024-09-12T03:20:35.020 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:35.018+0000 7f1e62978640 1 Processor -- start 2024-09-12T03:20:35.020 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:35.018+0000 7f1e62978640 1 -- start start 2024-09-12T03:20:35.020 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:35.019+0000 7f1e62978640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1e5c0d1720 0x7f1e5c157e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:35.020 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:35.019+0000 7f1e62978640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1e5c0d5200 0x7f1e5c0c8a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:35.020 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:35.019+0000 7f1e62978640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1e5c15eb90 0x7f1e5c0d1f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:35.020 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:35.019+0000 7f1e62978640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f1e5c0d3330 con 0x7f1e5c0d5200 2024-09-12T03:20:35.021 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:35.019+0000 7f1e62978640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f1e5c0d4000 con 0x7f1e5c0d1720 2024-09-12T03:20:35.021 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:35.019+0000 7f1e62978640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f1e5c0d34b0 con 0x7f1e5c15eb90 2024-09-12T03:20:35.021 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:35.019+0000 7f1e5b7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1e5c0d5200 0x7f1e5c0c8a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:35.021 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:35.019+0000 7f1e5b7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1e5c0d5200 0x7f1e5c0c8a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:47088/0 (socket says 172.21.15.27:47088) 2024-09-12T03:20:35.021 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:35.019+0000 7f1e5b7fe640 1 -- 172.21.15.27:0/3797515986 learned_addr learned my addr 172.21.15.27:0/3797515986 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:20:35.021 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:35.019+0000 7f1e5bfff640 1 --2- 172.21.15.27:0/3797515986 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1e5c0d1720 0x7f1e5c157e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:35.021 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:35.019+0000 7f1e60f15640 1 --2- 172.21.15.27:0/3797515986 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1e5c15eb90 0x7f1e5c0d1f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:35.021 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:35.019+0000 7f1e5b7fe640 1 -- 172.21.15.27:0/3797515986 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1e5c15eb90 msgr2=0x7f1e5c0d1f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:35.021 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:35.019+0000 7f1e5b7fe640 1 --2- 172.21.15.27:0/3797515986 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1e5c15eb90 0x7f1e5c0d1f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:35.021 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:35.019+0000 7f1e5b7fe640 1 -- 172.21.15.27:0/3797515986 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1e5c0d1720 msgr2=0x7f1e5c157e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:35.021 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:35.019+0000 7f1e5b7fe640 1 --2- 172.21.15.27:0/3797515986 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1e5c0d1720 0x7f1e5c157e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:35.021 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:35.019+0000 7f1e5b7fe640 1 -- 172.21.15.27:0/3797515986 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e5c15a870 con 0x7f1e5c0d5200 2024-09-12T03:20:35.021 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:35.019+0000 7f1e5b7fe640 1 --2- 172.21.15.27:0/3797515986 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1e5c0d5200 0x7f1e5c0c8a00 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f1e5000c470 tx=0x7f1e50001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:35.022 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:35.019+0000 7f1e58ff9640 1 -- 172.21.15.27:0/3797515986 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e500052d0 con 0x7f1e5c0d5200 2024-09-12T03:20:35.022 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:35.019+0000 7f1e58ff9640 1 -- 172.21.15.27:0/3797515986 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e50005470 con 0x7f1e5c0d5200 2024-09-12T03:20:35.022 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:35.019+0000 7f1e62978640 1 -- 172.21.15.27:0/3797515986 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e5c1597d0 con 0x7f1e5c0d5200 2024-09-12T03:20:35.022 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:35.020+0000 7f1e58ff9640 1 -- 172.21.15.27:0/3797515986 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e50007260 con 0x7f1e5c0d5200 2024-09-12T03:20:35.022 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:35.020+0000 7f1e62978640 1 -- 172.21.15.27:0/3797515986 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e5c159d90 con 0x7f1e5c0d5200 2024-09-12T03:20:35.023 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:35.021+0000 7f1e62978640 1 -- 172.21.15.27:0/3797515986 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e5c15f090 con 0x7f1e5c0d5200 2024-09-12T03:20:35.023 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:35.021+0000 7f1e58ff9640 1 -- 172.21.15.27:0/3797515986 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 90600+0+0 (secure 0 0 0) 0x7f1e50007400 con 0x7f1e5c0d5200 2024-09-12T03:20:35.023 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:35.021+0000 7f1e58ff9640 1 --2- 172.21.15.27:0/3797515986 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f1e4806bf80 0x7f1e4806e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:35.023 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:35.022+0000 7f1e5bfff640 1 --2- 172.21.15.27:0/3797515986 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f1e4806bf80 0x7f1e4806e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:35.023 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:35.022+0000 7f1e58ff9640 1 -- 172.21.15.27:0/3797515986 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 5709+0+0 (secure 0 0 0) 0x7f1e5008a450 con 0x7f1e5c0d5200 2024-09-12T03:20:35.023 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:35.022+0000 7f1e5bfff640 1 --2- 172.21.15.27:0/3797515986 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f1e4806bf80 0x7f1e4806e440 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f1e4c009b40 tx=0x7f1e4c009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:35.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:35.026+0000 7f1e58ff9640 1 -- 172.21.15.27:0/3797515986 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f1e500586a0 con 0x7f1e5c0d5200 2024-09-12T03:20:35.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:35.241+0000 7f1e62978640 1 -- 172.21.15.27:0/3797515986 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f1e5c069ea0 con 0x7f1e5c0d5200 2024-09-12T03:20:35.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:35.241+0000 7f1e58ff9640 1 -- 172.21.15.27:0/3797515986 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v46) ==== 53+0+73 (secure 0 0 0) 0x7f1e5005c9a0 con 0x7f1e5c0d5200 2024-09-12T03:20:35.241 INFO:teuthology.orchestra.run.smithi027.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-12T03:20:35.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:35.243+0000 7f1e62978640 1 -- 172.21.15.27:0/3797515986 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f1e4806bf80 msgr2=0x7f1e4806e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:35.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:35.243+0000 7f1e62978640 1 --2- 172.21.15.27:0/3797515986 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f1e4806bf80 0x7f1e4806e440 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f1e4c009b40 tx=0x7f1e4c009190 comp rx=0 tx=0).stop 2024-09-12T03:20:35.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:35.243+0000 7f1e62978640 1 -- 172.21.15.27:0/3797515986 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1e5c0d5200 msgr2=0x7f1e5c0c8a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:35.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:35.243+0000 7f1e62978640 1 --2- 172.21.15.27:0/3797515986 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1e5c0d5200 0x7f1e5c0c8a00 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f1e5000c470 tx=0x7f1e50001e70 comp rx=0 tx=0).stop 2024-09-12T03:20:35.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:35.243+0000 7f1e62978640 1 -- 172.21.15.27:0/3797515986 shutdown_connections 2024-09-12T03:20:35.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:35.243+0000 7f1e62978640 1 --2- 172.21.15.27:0/3797515986 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1e5c15eb90 0x7f1e5c0d1f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:35.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:35.243+0000 7f1e62978640 1 --2- 172.21.15.27:0/3797515986 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f1e4806bf80 0x7f1e4806e440 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:35.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:35.243+0000 7f1e62978640 1 --2- 172.21.15.27:0/3797515986 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1e5c0d5200 0x7f1e5c0c8a00 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:35.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:35.243+0000 7f1e62978640 1 --2- 172.21.15.27:0/3797515986 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1e5c0d1720 0x7f1e5c157e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:35.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:35.243+0000 7f1e62978640 1 -- 172.21.15.27:0/3797515986 >> 172.21.15.27:0/3797515986 conn(0x7f1e5c06dce0 msgr2=0x7f1e5c0b9360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:35.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:35.244+0000 7f1e62978640 1 -- 172.21.15.27:0/3797515986 shutdown_connections 2024-09-12T03:20:35.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:35.244+0000 7f1e62978640 1 -- 172.21.15.27:0/3797515986 wait complete. 2024-09-12T03:20:35.254 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-12T03:20:35.254 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-12T03:20:35.254 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi045.front.sepia.ceph.com 2024-09-12T03:20:35.254 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-12T03:20:35.254 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-12T03:20:35.254 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-12T03:20:35.254 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-12T03:20:35.255 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-12T03:20:35.255 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-12T03:20:35.255 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:20:35.255 DEBUG:teuthology.orchestra.run.smithi045:> ip addr 2024-09-12T03:20:35.275 INFO:teuthology.orchestra.run.smithi045.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-12T03:20:35.275 INFO:teuthology.orchestra.run.smithi045.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-12T03:20:35.275 INFO:teuthology.orchestra.run.smithi045.stdout: inet 127.0.0.1/8 scope host lo 2024-09-12T03:20:35.275 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:20:35.276 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 ::1/128 scope host 2024-09-12T03:20:35.276 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:20:35.276 INFO:teuthology.orchestra.run.smithi045.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-12T03:20:35.276 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:6b:fd:a6 brd ff:ff:ff:ff:ff:ff 2024-09-12T03:20:35.276 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp7s0f0 2024-09-12T03:20:35.276 INFO:teuthology.orchestra.run.smithi045.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-12T03:20:35.276 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:6b:fd:a7 brd ff:ff:ff:ff:ff:ff 2024-09-12T03:20:35.276 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp7s0f1 2024-09-12T03:20:35.276 INFO:teuthology.orchestra.run.smithi045.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-12T03:20:35.276 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:bd:12:be brd ff:ff:ff:ff:ff:ff 2024-09-12T03:20:35.276 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp1s0f0 2024-09-12T03:20:35.276 INFO:teuthology.orchestra.run.smithi045.stdout: inet 172.21.15.45/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-12T03:20:35.276 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft 42498sec preferred_lft 42498sec 2024-09-12T03:20:35.277 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::ec4:7aff:febd:12be/64 scope link 2024-09-12T03:20:35.277 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:20:35.277 INFO:teuthology.orchestra.run.smithi045.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-12T03:20:35.277 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:bd:12:bf brd ff:ff:ff:ff:ff:ff 2024-09-12T03:20:35.277 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp1s0f1 2024-09-12T03:20:35.277 INFO:teuthology.orchestra.run.smithi045.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-12T03:20:35.277 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 52:97:33:5c:60:1b brd ff:ff:ff:ff:ff:ff 2024-09-12T03:20:35.277 INFO:teuthology.orchestra.run.smithi045.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-12T03:20:35.277 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:20:35.277 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::d8c2:5dff:fe26:78c8/64 scope link 2024-09-12T03:20:35.277 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:20:35.277 INFO:teuthology.orchestra.run.smithi045.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-12T03:20:35.277 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 5a:ec:d7:40:ef:75 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-12T03:20:35.278 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::58ec:d7ff:fe40:ef75/64 scope link 2024-09-12T03:20:35.278 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:20:35.278 INFO:teuthology.orchestra.run.smithi045.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-12T03:20:35.278 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 52:97:33:5c:60:1b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-12T03:20:35.278 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::5097:33ff:fe5c:601b/64 scope link 2024-09-12T03:20:35.278 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:20:35.278 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:20:35.278 DEBUG:teuthology.orchestra.run.smithi045:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-12T03:20:35.344 INFO:teuthology.orchestra.run.smithi045.stdout:1 2024-09-12T03:20:35.346 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:20:35.346 DEBUG:teuthology.orchestra.run.smithi045:> route 2024-09-12T03:20:35.405 INFO:teuthology.orchestra.run.smithi045.stdout:Kernel IP routing table 2024-09-12T03:20:35.406 INFO:teuthology.orchestra.run.smithi045.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-12T03:20:35.406 INFO:teuthology.orchestra.run.smithi045.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-12T03:20:35.406 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-12T03:20:35.406 INFO:teuthology.orchestra.run.smithi045.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-12T03:20:35.406 DEBUG:teuthology.orchestra.run.smithi045:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-12T03:20:35.406 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-09-12T03:20:35.406 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-12T03:20:35.406 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-12T03:20:35.406 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-12T03:20:35.406 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-09-12T03:20:35.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:20:35 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T03:20:35.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:20:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T03:20:35.540 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:20:35.540 DEBUG:teuthology.orchestra.run.smithi045:> ip netns list 2024-09-12T03:20:35.556 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-09-12T03:20:35.556 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-12T03:20:35.557 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-12T03:20:35.557 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:20:35.557 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link set brx.0 up 2024-09-12T03:20:35.623 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-12T03:20:35.623 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-12T03:20:35.680 INFO:teuthology.orchestra.run.smithi045.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-12T03:20:35.680 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-12T03:20:35.680 DEBUG:teuthology.orchestra.run.smithi045:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-12T03:20:35.735 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2024-09-12T03:20:35.736 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:20:35.736 DEBUG:teuthology.orchestra.run.smithi045:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-12T03:20:35.820 INFO:teuthology.orchestra.run.smithi045.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-12T03:20:35.820 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-12T03:20:35.832 INFO:teuthology.orchestra.run.smithi045.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-12T03:20:35.833 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-12T03:20:35.833 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: resolved to: "172.21.15.27:3300,172.21.15.45:3300,172.21.15.45:3301" 2024-09-12T03:20:35.833 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: trying mount with new device syntax: 1@d2df504e-2647-4385-a5db-9fdf69f487a0.cephfs=/ 2024-09-12T03:20:35.833 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.27:3300/172.21.15.45:3300/172.21.15.45:3301" will pass to kernel 2024-09-12T03:20:35.833 INFO:teuthology.orchestra.run.smithi045.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-09-12T03:20:35.833 INFO:teuthology.orchestra.run.smithi045.stdout: You just mounted a file system that supports labels which does not 2024-09-12T03:20:35.833 INFO:teuthology.orchestra.run.smithi045.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-12T03:20:35.833 INFO:teuthology.orchestra.run.smithi045.stdout: applications will generate AVC messages and not be allowed access to 2024-09-12T03:20:35.833 INFO:teuthology.orchestra.run.smithi045.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-12T03:20:35.835 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-12T03:20:35.835 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:20:35.835 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-12T03:20:35.861 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-12T03:20:35.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:20:35 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-12T03:20:35.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:20:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T03:20:35.959 INFO:teuthology.orchestra.run.smithi045.stdout:client6211 2024-09-12T03:20:35.960 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-12T03:20:35.960 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T03:20:35.976 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-09-12T03:20:36.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:36.095+0000 7f20e174b640 1 Processor -- start 2024-09-12T03:20:36.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:36.095+0000 7f20e174b640 1 -- start start 2024-09-12T03:20:36.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:36.096+0000 7f20e174b640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f20dc070d30 0x7f20dc06cc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:36.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:36.096+0000 7f20e174b640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f20dc06e960 0x7f20dc0d0440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:36.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:36.096+0000 7f20e174b640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f20dc0cf9f0 con 0x7f20dc0d0980 2024-09-12T03:20:36.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:36.096+0000 7f20e174b640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f20dc0d1a40 con 0x7f20dc06e960 2024-09-12T03:20:36.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:36.096+0000 7f20e174b640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f20dc0cfc90 con 0x7f20dc070d30 2024-09-12T03:20:36.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:36.096+0000 7f20da7fc640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f20dc06e960 0x7f20dc0d0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:36.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:36.096+0000 7f20db7fe640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f20dc0d0980 0x7f20dc0d21b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:57712/0 (socket says 172.21.15.27:57712) 2024-09-12T03:20:36.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:36.096+0000 7f20db7fe640 1 -- 172.21.15.27:0/4003046249 learned_addr learned my addr 172.21.15.27:0/4003046249 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:20:36.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:36.096+0000 7f20d9ffb640 1 -- 172.21.15.27:0/4003046249 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1832639138 0 0) 0x7f20dc0cf9f0 con 0x7f20dc0d0980 2024-09-12T03:20:36.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:36.096+0000 7f20daffd640 1 --2- 172.21.15.27:0/4003046249 >> v2:172.21.15.45:3300/0 conn(0x7f20dc070d30 0x7f20dc06cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:36.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:36.096+0000 7f20d9ffb640 1 -- 172.21.15.27:0/4003046249 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f20c4003520 con 0x7f20dc0d0980 2024-09-12T03:20:36.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:36.097+0000 7f20da7fc640 1 -- 172.21.15.27:0/4003046249 >> v2:172.21.15.45:3300/0 conn(0x7f20dc070d30 msgr2=0x7f20dc06cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:36.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:36.097+0000 7f20da7fc640 1 --2- 172.21.15.27:0/4003046249 >> v2:172.21.15.45:3300/0 conn(0x7f20dc070d30 0x7f20dc06cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:36.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:36.097+0000 7f20da7fc640 1 -- 172.21.15.27:0/4003046249 >> v1:172.21.15.27:6789/0 conn(0x7f20dc0d0980 legacy=0x7f20dc0d21b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:36.098 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:36.097+0000 7f20da7fc640 1 -- 172.21.15.27:0/4003046249 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20dc0cff70 con 0x7f20dc06e960 2024-09-12T03:20:36.098 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:36.097+0000 7f20da7fc640 1 --2- 172.21.15.27:0/4003046249 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f20dc06e960 0x7f20dc0d0440 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f20dc0d1bc0 tx=0x7f20d002a080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=671637872ad7d402 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:36.099 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:36.097+0000 7f20d9ffb640 1 -- 172.21.15.27:0/4003046249 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20d0037070 con 0x7f20dc06e960 2024-09-12T03:20:36.099 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:36.098+0000 7f20d9ffb640 1 -- 172.21.15.27:0/4003046249 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20d002add0 con 0x7f20dc06e960 2024-09-12T03:20:36.099 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:36.098+0000 7f20d9ffb640 1 -- 172.21.15.27:0/4003046249 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20d0032640 con 0x7f20dc06e960 2024-09-12T03:20:36.099 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:36.098+0000 7f20e174b640 1 -- 172.21.15.27:0/4003046249 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f20dc06e960 msgr2=0x7f20dc0d0440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:36.099 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:36.098+0000 7f20e174b640 1 --2- 172.21.15.27:0/4003046249 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f20dc06e960 0x7f20dc0d0440 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f20dc0d1bc0 tx=0x7f20d002a080 comp rx=0 tx=0).stop 2024-09-12T03:20:36.099 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:36.098+0000 7f20e174b640 1 -- 172.21.15.27:0/4003046249 shutdown_connections 2024-09-12T03:20:36.099 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:36.098+0000 7f20e174b640 1 --2- 172.21.15.27:0/4003046249 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f20dc06e960 0x7f20dc0d0440 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:36.099 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:36.098+0000 7f20e174b640 1 --2- 172.21.15.27:0/4003046249 >> v2:172.21.15.45:3300/0 conn(0x7f20dc070d30 0x7f20dc06cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:36.099 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:36.098+0000 7f20e174b640 1 -- 172.21.15.27:0/4003046249 >> 172.21.15.27:0/4003046249 conn(0x7f20dc0b7c10 msgr2=0x7f20dc09ba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:36.099 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:36.098+0000 7f20e174b640 1 -- 172.21.15.27:0/4003046249 shutdown_connections 2024-09-12T03:20:36.100 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:36.098+0000 7f20e174b640 1 -- 172.21.15.27:0/4003046249 wait complete. 2024-09-12T03:20:36.102 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:36.100+0000 7f20e174b640 1 Processor -- start 2024-09-12T03:20:36.102 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:36.100+0000 7f20e174b640 1 -- start start 2024-09-12T03:20:36.102 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:36.100+0000 7f20e174b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f20dc06e960 0x7f20dc15b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:36.102 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:36.100+0000 7f20e174b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f20dc070d30 0x7f20dc1604e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:36.102 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:36.100+0000 7f20e174b640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f20dc0d0980 0x7f20dc15e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:36.102 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:36.100+0000 7f20e174b640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f20dc0d28c0 con 0x7f20dc06e960 2024-09-12T03:20:36.102 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:36.100+0000 7f20e174b640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f20dc06d180 con 0x7f20dc070d30 2024-09-12T03:20:36.102 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:36.100+0000 7f20e174b640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f20dc0d2a40 con 0x7f20dc0d0980 2024-09-12T03:20:36.102 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:36.100+0000 7f20daffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f20dc06e960 0x7f20dc15b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:36.102 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:36.101+0000 7f20db7fe640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f20dc0d0980 0x7f20dc15e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:36.102 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:36.101+0000 7f20daffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f20dc06e960 0x7f20dc15b1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:47102/0 (socket says 172.21.15.27:47102) 2024-09-12T03:20:36.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:36.101+0000 7f20daffd640 1 -- 172.21.15.27:0/962130556 learned_addr learned my addr 172.21.15.27:0/962130556 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:20:36.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:36.101+0000 7f20da7fc640 1 --2- 172.21.15.27:0/962130556 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f20dc070d30 0x7f20dc1604e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:36.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:36.101+0000 7f20daffd640 1 -- 172.21.15.27:0/962130556 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f20dc0d0980 msgr2=0x7f20dc15e9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:36.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:36.101+0000 7f20daffd640 1 --2- 172.21.15.27:0/962130556 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f20dc0d0980 0x7f20dc15e9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:36.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:36.101+0000 7f20daffd640 1 -- 172.21.15.27:0/962130556 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f20dc070d30 msgr2=0x7f20dc1604e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:36.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:36.101+0000 7f20daffd640 1 --2- 172.21.15.27:0/962130556 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f20dc070d30 0x7f20dc1604e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:36.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:36.101+0000 7f20daffd640 1 -- 172.21.15.27:0/962130556 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20dc15f440 con 0x7f20dc06e960 2024-09-12T03:20:36.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:36.101+0000 7f20db7fe640 1 --2- 172.21.15.27:0/962130556 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f20dc0d0980 0x7f20dc15e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:20:36.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:36.101+0000 7f20daffd640 1 --2- 172.21.15.27:0/962130556 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f20dc06e960 0x7f20dc15b1e0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f20cc00d800 tx=0x7f20cc00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:36.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:36.101+0000 7f20c37fe640 1 -- 172.21.15.27:0/962130556 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20cc00b6c0 con 0x7f20dc06e960 2024-09-12T03:20:36.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:36.101+0000 7f20c37fe640 1 -- 172.21.15.27:0/962130556 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20cc019070 con 0x7f20dc06e960 2024-09-12T03:20:36.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:36.101+0000 7f20e174b640 1 -- 172.21.15.27:0/962130556 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20dc15f720 con 0x7f20dc06e960 2024-09-12T03:20:36.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:36.101+0000 7f20c37fe640 1 -- 172.21.15.27:0/962130556 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20cc00ba80 con 0x7f20dc06e960 2024-09-12T03:20:36.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:36.101+0000 7f20e174b640 1 -- 172.21.15.27:0/962130556 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20dc15fce0 con 0x7f20dc06e960 2024-09-12T03:20:36.104 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:36.102+0000 7f20e174b640 1 -- 172.21.15.27:0/962130556 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20dc0d0440 con 0x7f20dc06e960 2024-09-12T03:20:36.104 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:36.103+0000 7f20c37fe640 1 -- 172.21.15.27:0/962130556 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 90600+0+0 (secure 0 0 0) 0x7f20cc00bc20 con 0x7f20dc06e960 2024-09-12T03:20:36.104 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:36.103+0000 7f20c37fe640 1 --2- 172.21.15.27:0/962130556 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f20c806bf50 0x7f20c806e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:36.104 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:36.103+0000 7f20c37fe640 1 -- 172.21.15.27:0/962130556 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 5709+0+0 (secure 0 0 0) 0x7f20cc08b2c0 con 0x7f20dc06e960 2024-09-12T03:20:36.105 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:36.103+0000 7f20da7fc640 1 --2- 172.21.15.27:0/962130556 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f20c806bf50 0x7f20c806e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:36.105 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:36.104+0000 7f20da7fc640 1 --2- 172.21.15.27:0/962130556 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f20c806bf50 0x7f20c806e410 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f20d0034000 tx=0x7f20d002e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:36.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:36.108+0000 7f20c37fe640 1 -- 172.21.15.27:0/962130556 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f20cc0595a0 con 0x7f20dc06e960 2024-09-12T03:20:36.324 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:36.324+0000 7f20e174b640 1 -- 172.21.15.27:0/962130556 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7f20dc0b1e00 con 0x7f20dc06e960 2024-09-12T03:20:37.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.187+0000 7f20c37fe640 1 -- 172.21.15.27:0/962130556 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v47) ==== 136+0+0 (secure 0 0 0) 0x7f20cc05d8a0 con 0x7f20dc06e960 2024-09-12T03:20:37.190 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.190+0000 7f20e174b640 1 -- 172.21.15.27:0/962130556 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f20c806bf50 msgr2=0x7f20c806e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:37.191 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.190+0000 7f20e174b640 1 --2- 172.21.15.27:0/962130556 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f20c806bf50 0x7f20c806e410 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f20d0034000 tx=0x7f20d002e000 comp rx=0 tx=0).stop 2024-09-12T03:20:37.191 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.190+0000 7f20e174b640 1 -- 172.21.15.27:0/962130556 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f20dc06e960 msgr2=0x7f20dc15b1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:37.191 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.190+0000 7f20e174b640 1 --2- 172.21.15.27:0/962130556 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f20dc06e960 0x7f20dc15b1e0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f20cc00d800 tx=0x7f20cc00dcc0 comp rx=0 tx=0).stop 2024-09-12T03:20:37.191 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.190+0000 7f20e174b640 1 -- 172.21.15.27:0/962130556 shutdown_connections 2024-09-12T03:20:37.191 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.190+0000 7f20e174b640 1 --2- 172.21.15.27:0/962130556 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f20dc0d0980 0x7f20dc15e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:37.191 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.190+0000 7f20e174b640 1 --2- 172.21.15.27:0/962130556 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f20dc070d30 0x7f20dc1604e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:37.191 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.190+0000 7f20e174b640 1 --2- 172.21.15.27:0/962130556 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f20c806bf50 0x7f20c806e410 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:37.191 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.190+0000 7f20e174b640 1 --2- 172.21.15.27:0/962130556 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f20dc06e960 0x7f20dc15b1e0 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:37.191 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.190+0000 7f20e174b640 1 -- 172.21.15.27:0/962130556 >> 172.21.15.27:0/962130556 conn(0x7f20dc0b7c10 msgr2=0x7f20dc0b70e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:37.191 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.190+0000 7f20e174b640 1 -- 172.21.15.27:0/962130556 shutdown_connections 2024-09-12T03:20:37.191 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.191+0000 7f20e174b640 1 -- 172.21.15.27:0/962130556 wait complete. 2024-09-12T03:20:37.204 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-12T03:20:37.314 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.314+0000 7f792d5b7640 1 Processor -- start 2024-09-12T03:20:37.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.315+0000 7f792d5b7640 1 -- start start 2024-09-12T03:20:37.316 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.315+0000 7f792d5b7640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f792815eb90 0x7f79280d2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:37.316 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.315+0000 7f792d5b7640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f79280d1720 0x7f79280b8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:37.316 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.315+0000 7f792d5b7640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f79280d3760 con 0x7f79280d5200 2024-09-12T03:20:37.316 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.315+0000 7f792d5b7640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f79280b7d20 con 0x7f792815eb90 2024-09-12T03:20:37.316 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.315+0000 7f792d5b7640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f79280d42e0 con 0x7f79280d1720 2024-09-12T03:20:37.316 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.316+0000 7f7926ffd640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f792815eb90 0x7f79280d2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:37.316 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.316+0000 7f79267fc640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f79280d5200 0x7f79280d1010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:57724/0 (socket says 172.21.15.27:57724) 2024-09-12T03:20:37.317 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.316+0000 7f79267fc640 1 -- 172.21.15.27:0/3615581089 learned_addr learned my addr 172.21.15.27:0/3615581089 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:20:37.317 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.316+0000 7f79277fe640 1 --2- 172.21.15.27:0/3615581089 >> v2:172.21.15.45:3300/0 conn(0x7f79280d1720 0x7f79280b8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:37.317 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.316+0000 7f7926ffd640 1 -- 172.21.15.27:0/3615581089 >> v2:172.21.15.45:3300/0 conn(0x7f79280d1720 msgr2=0x7f79280b8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:37.317 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.316+0000 7f7926ffd640 1 --2- 172.21.15.27:0/3615581089 >> v2:172.21.15.45:3300/0 conn(0x7f79280d1720 0x7f79280b8190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:37.317 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.316+0000 7f7926ffd640 1 -- 172.21.15.27:0/3615581089 >> v1:172.21.15.27:6789/0 conn(0x7f79280d5200 legacy=0x7f79280d1010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:37.317 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.316+0000 7f7926ffd640 1 -- 172.21.15.27:0/3615581089 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79280d3a40 con 0x7f792815eb90 2024-09-12T03:20:37.317 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.316+0000 7f7925ffb640 1 -- 172.21.15.27:0/3615581089 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1145124440 0 0) 0x7f79280d3760 con 0x7f79280d5200 2024-09-12T03:20:37.317 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.316+0000 7f79277fe640 1 --2- 172.21.15.27:0/3615581089 >> v2:172.21.15.45:3300/0 conn(0x7f79280d1720 0x7f79280b8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:20:37.318 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.317+0000 7f7926ffd640 1 --2- 172.21.15.27:0/3615581089 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f792815eb90 0x7f79280d2df0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f79280c8270 tx=0x7f792002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab0ffb40a0335ecd server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:37.318 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.317+0000 7f7925ffb640 1 -- 172.21.15.27:0/3615581089 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7920037070 con 0x7f792815eb90 2024-09-12T03:20:37.318 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.317+0000 7f7925ffb640 1 -- 172.21.15.27:0/3615581089 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7920032440 con 0x7f792815eb90 2024-09-12T03:20:37.318 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.317+0000 7f7925ffb640 1 -- 172.21.15.27:0/3615581089 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7920032810 con 0x7f792815eb90 2024-09-12T03:20:37.318 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.317+0000 7f792d5b7640 1 -- 172.21.15.27:0/3615581089 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f792815eb90 msgr2=0x7f79280d2df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:37.318 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.317+0000 7f792d5b7640 1 --2- 172.21.15.27:0/3615581089 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f792815eb90 0x7f79280d2df0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f79280c8270 tx=0x7f792002a1a0 comp rx=0 tx=0).stop 2024-09-12T03:20:37.319 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.317+0000 7f792d5b7640 1 -- 172.21.15.27:0/3615581089 shutdown_connections 2024-09-12T03:20:37.319 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.317+0000 7f792d5b7640 1 --2- 172.21.15.27:0/3615581089 >> v2:172.21.15.45:3300/0 conn(0x7f79280d1720 0x7f79280b8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:37.319 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.317+0000 7f792d5b7640 1 --2- 172.21.15.27:0/3615581089 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f792815eb90 0x7f79280d2df0 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:37.319 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.317+0000 7f792d5b7640 1 -- 172.21.15.27:0/3615581089 >> 172.21.15.27:0/3615581089 conn(0x7f792806dce0 msgr2=0x7f792806e0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:37.319 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.317+0000 7f792d5b7640 1 -- 172.21.15.27:0/3615581089 shutdown_connections 2024-09-12T03:20:37.319 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.317+0000 7f792d5b7640 1 -- 172.21.15.27:0/3615581089 wait complete. 2024-09-12T03:20:37.321 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.319+0000 7f792d5b7640 1 Processor -- start 2024-09-12T03:20:37.321 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.319+0000 7f792d5b7640 1 -- start start 2024-09-12T03:20:37.321 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.320+0000 7f792d5b7640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f79280d1720 0x7f79281f7690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:37.321 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.320+0000 7f792d5b7640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f79280d5200 0x7f79280acbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:37.322 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.320+0000 7f792d5b7640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f792815eb90 0x7f79280ad100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:37.322 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.320+0000 7f792d5b7640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f79280d3330 con 0x7f79280d1720 2024-09-12T03:20:37.322 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.320+0000 7f792d5b7640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f79280d4000 con 0x7f79280d5200 2024-09-12T03:20:37.322 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.320+0000 7f792d5b7640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f79280d34b0 con 0x7f792815eb90 2024-09-12T03:20:37.322 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.320+0000 7f7926ffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f79280d1720 0x7f79281f7690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:37.322 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.320+0000 7f7926ffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f79280d1720 0x7f79281f7690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:47124/0 (socket says 172.21.15.27:47124) 2024-09-12T03:20:37.322 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.320+0000 7f7926ffd640 1 -- 172.21.15.27:0/1259346584 learned_addr learned my addr 172.21.15.27:0/1259346584 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:20:37.322 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.320+0000 7f79277fe640 1 --2- 172.21.15.27:0/1259346584 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f792815eb90 0x7f79280ad100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:37.322 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.320+0000 7f79267fc640 1 --2- 172.21.15.27:0/1259346584 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f79280d5200 0x7f79280acbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:37.322 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.320+0000 7f7926ffd640 1 -- 172.21.15.27:0/1259346584 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f792815eb90 msgr2=0x7f79280ad100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:37.322 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.320+0000 7f7926ffd640 1 --2- 172.21.15.27:0/1259346584 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f792815eb90 0x7f79280ad100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:37.322 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.320+0000 7f7926ffd640 1 -- 172.21.15.27:0/1259346584 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f79280d5200 msgr2=0x7f79280acbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:37.322 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.320+0000 7f7926ffd640 1 --2- 172.21.15.27:0/1259346584 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f79280d5200 0x7f79280acbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:37.323 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.320+0000 7f7926ffd640 1 -- 172.21.15.27:0/1259346584 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79280c8f50 con 0x7f79280d1720 2024-09-12T03:20:37.323 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.320+0000 7f79267fc640 1 --2- 172.21.15.27:0/1259346584 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f79280d5200 0x7f79280acbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:20:37.323 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.320+0000 7f7926ffd640 1 --2- 172.21.15.27:0/1259346584 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f79280d1720 0x7f79281f7690 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f7920000c00 tx=0x7f79200046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:37.323 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.320+0000 7f791f7fe640 1 -- 172.21.15.27:0/1259346584 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7920037040 con 0x7f79280d1720 2024-09-12T03:20:37.323 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.320+0000 7f791f7fe640 1 -- 172.21.15.27:0/1259346584 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7920042070 con 0x7f79280d1720 2024-09-12T03:20:37.323 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.320+0000 7f791f7fe640 1 -- 172.21.15.27:0/1259346584 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7920030bd0 con 0x7f79280d1720 2024-09-12T03:20:37.323 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.320+0000 7f792d5b7640 1 -- 172.21.15.27:0/1259346584 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79280c9230 con 0x7f79280d1720 2024-09-12T03:20:37.323 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.320+0000 7f792d5b7640 1 -- 172.21.15.27:0/1259346584 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79280c97f0 con 0x7f79280d1720 2024-09-12T03:20:37.323 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.321+0000 7f792d5b7640 1 -- 172.21.15.27:0/1259346584 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f792815f090 con 0x7f79280d1720 2024-09-12T03:20:37.324 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.322+0000 7f791f7fe640 1 -- 172.21.15.27:0/1259346584 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 90600+0+0 (secure 0 0 0) 0x7f7920047460 con 0x7f79280d1720 2024-09-12T03:20:37.324 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.322+0000 7f791f7fe640 1 --2- 172.21.15.27:0/1259346584 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f790806bf80 0x7f790806e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:37.324 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.323+0000 7f791f7fe640 1 -- 172.21.15.27:0/1259346584 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 5709+0+0 (secure 0 0 0) 0x7f79200abba0 con 0x7f79280d1720 2024-09-12T03:20:37.324 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.323+0000 7f79267fc640 1 --2- 172.21.15.27:0/1259346584 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f790806bf80 0x7f790806e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:37.324 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.323+0000 7f79267fc640 1 --2- 172.21.15.27:0/1259346584 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f790806bf80 0x7f790806e440 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f79280cb890 tx=0x7f7910007c10 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:37.327 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.327+0000 7f791f7fe640 1 -- 172.21.15.27:0/1259346584 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f7920032e20 con 0x7f79280d1720 2024-09-12T03:20:37.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.524+0000 7f792d5b7640 1 -- 172.21.15.27:0/1259346584 --> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f79280ce410 con 0x7f790806bf80 2024-09-12T03:20:37.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.525+0000 7f791f7fe640 1 -- 172.21.15.27:0/1259346584 <== mgr.4113 v2:172.21.15.27:6836/1226920526 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f79280ce410 con 0x7f790806bf80 2024-09-12T03:20:37.525 INFO:teuthology.orchestra.run.smithi027.stdout:[ 2024-09-12T03:20:37.525 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:20:37.526 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "cephfs" 2024-09-12T03:20:37.526 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:20:37.526 INFO:teuthology.orchestra.run.smithi027.stdout:] 2024-09-12T03:20:37.528 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.527+0000 7f792d5b7640 1 -- 172.21.15.27:0/1259346584 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f790806bf80 msgr2=0x7f790806e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:37.528 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.527+0000 7f792d5b7640 1 --2- 172.21.15.27:0/1259346584 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f790806bf80 0x7f790806e440 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f79280cb890 tx=0x7f7910007c10 comp rx=0 tx=0).stop 2024-09-12T03:20:37.528 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.527+0000 7f792d5b7640 1 -- 172.21.15.27:0/1259346584 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f79280d1720 msgr2=0x7f79281f7690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:37.528 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.527+0000 7f792d5b7640 1 --2- 172.21.15.27:0/1259346584 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f79280d1720 0x7f79281f7690 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f7920000c00 tx=0x7f79200046c0 comp rx=0 tx=0).stop 2024-09-12T03:20:37.528 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.527+0000 7f792d5b7640 1 -- 172.21.15.27:0/1259346584 shutdown_connections 2024-09-12T03:20:37.528 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.527+0000 7f792d5b7640 1 --2- 172.21.15.27:0/1259346584 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f792815eb90 0x7f79280ad100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:37.528 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.527+0000 7f792d5b7640 1 --2- 172.21.15.27:0/1259346584 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f79280d5200 0x7f79280acbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:37.528 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.527+0000 7f792d5b7640 1 --2- 172.21.15.27:0/1259346584 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f790806bf80 0x7f790806e440 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:37.528 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.527+0000 7f792d5b7640 1 --2- 172.21.15.27:0/1259346584 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f79280d1720 0x7f79281f7690 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:37.528 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.527+0000 7f792d5b7640 1 -- 172.21.15.27:0/1259346584 >> 172.21.15.27:0/1259346584 conn(0x7f792806dce0 msgr2=0x7f79280b8090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:37.529 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.528+0000 7f792d5b7640 1 -- 172.21.15.27:0/1259346584 shutdown_connections 2024-09-12T03:20:37.529 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.528+0000 7f792d5b7640 1 -- 172.21.15.27:0/1259346584 wait complete. 2024-09-12T03:20:37.539 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-12T03:20:37.644 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.643+0000 7fb70b3d6640 1 Processor -- start 2024-09-12T03:20:37.644 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.643+0000 7fb70b3d6640 1 -- start start 2024-09-12T03:20:37.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.644+0000 7fb70b3d6640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fb7040b6240 0x7fb7040b6620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:37.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.644+0000 7fb70b3d6640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fb7040b87d0 0x7fb7040d4040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:37.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.644+0000 7fb70b3d6640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb7040b6b60 con 0x7fb7040d3c20 2024-09-12T03:20:37.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.644+0000 7fb70b3d6640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7fb7040d2930 con 0x7fb7040b6240 2024-09-12T03:20:37.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.644+0000 7fb70b3d6640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fb7040d2c10 con 0x7fb7040b87d0 2024-09-12T03:20:37.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.644+0000 7fb709172640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fb7040b6240 0x7fb7040b6620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:37.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.644+0000 7fb709172640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fb7040b6240 0x7fb7040b6620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:47134/0 (socket says 172.21.15.27:47134) 2024-09-12T03:20:37.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.644+0000 7fb709172640 1 -- 172.21.15.27:0/2453212411 learned_addr learned my addr 172.21.15.27:0/2453212411 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:20:37.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.645+0000 7fb709973640 1 --2- 172.21.15.27:0/2453212411 >> v2:172.21.15.45:3300/0 conn(0x7fb7040b87d0 0x7fb7040d4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:37.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.645+0000 7fb709172640 1 -- 172.21.15.27:0/2453212411 >> v2:172.21.15.45:3300/0 conn(0x7fb7040b87d0 msgr2=0x7fb7040d4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:37.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.645+0000 7fb709172640 1 --2- 172.21.15.27:0/2453212411 >> v2:172.21.15.45:3300/0 conn(0x7fb7040b87d0 0x7fb7040d4040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:37.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.645+0000 7fb709172640 1 -- 172.21.15.27:0/2453212411 >> v1:172.21.15.45:6789/0 conn(0x7fb7040d3c20 legacy=0x7fb70406d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:37.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.645+0000 7fb709172640 1 -- 172.21.15.27:0/2453212411 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7040d2ef0 con 0x7fb7040b6240 2024-09-12T03:20:37.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.645+0000 7fb709172640 1 --2- 172.21.15.27:0/2453212411 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb7040b6240 0x7fb7040b6620 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fb70408dae0 tx=0x7fb70002a780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ceddc48147ac769e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:37.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.646+0000 7fb6fbfff640 1 -- 172.21.15.27:0/2453212411 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb700036070 con 0x7fb7040b6240 2024-09-12T03:20:37.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.646+0000 7fb6fbfff640 1 -- 172.21.15.27:0/2453212411 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb700031600 con 0x7fb7040b6240 2024-09-12T03:20:37.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.646+0000 7fb6fbfff640 1 -- 172.21.15.27:0/2453212411 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7000319d0 con 0x7fb7040b6240 2024-09-12T03:20:37.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.646+0000 7fb70b3d6640 1 -- 172.21.15.27:0/2453212411 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb7040b6240 msgr2=0x7fb7040b6620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:37.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.646+0000 7fb70b3d6640 1 --2- 172.21.15.27:0/2453212411 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb7040b6240 0x7fb7040b6620 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fb70408dae0 tx=0x7fb70002a780 comp rx=0 tx=0).stop 2024-09-12T03:20:37.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.646+0000 7fb70b3d6640 1 -- 172.21.15.27:0/2453212411 shutdown_connections 2024-09-12T03:20:37.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.646+0000 7fb70b3d6640 1 --2- 172.21.15.27:0/2453212411 >> v2:172.21.15.45:3300/0 conn(0x7fb7040b87d0 0x7fb7040d4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:37.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.646+0000 7fb70b3d6640 1 --2- 172.21.15.27:0/2453212411 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb7040b6240 0x7fb7040b6620 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:37.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.646+0000 7fb70b3d6640 1 -- 172.21.15.27:0/2453212411 >> 172.21.15.27:0/2453212411 conn(0x7fb7040d1420 msgr2=0x7fb7040d1830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:37.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.646+0000 7fb70b3d6640 1 -- 172.21.15.27:0/2453212411 shutdown_connections 2024-09-12T03:20:37.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.646+0000 7fb70b3d6640 1 -- 172.21.15.27:0/2453212411 wait complete. 2024-09-12T03:20:37.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.647+0000 7fb70b3d6640 1 Processor -- start 2024-09-12T03:20:37.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.648+0000 7fb70b3d6640 1 -- start start 2024-09-12T03:20:37.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.648+0000 7fb70b3d6640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb7040b6240 0x7fb704151b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:37.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.648+0000 7fb70b3d6640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb7040b87d0 0x7fb7040c91e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:37.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.648+0000 7fb70b3d6640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb7040d3c20 0x7fb7040ca090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:37.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.648+0000 7fb70b3d6640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fb70415ece0 con 0x7fb7040d3c20 2024-09-12T03:20:37.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.648+0000 7fb70b3d6640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb70415eb60 con 0x7fb7040b6240 2024-09-12T03:20:37.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.648+0000 7fb70b3d6640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fb7040cfc70 con 0x7fb7040b87d0 2024-09-12T03:20:37.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.648+0000 7fb709973640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb7040d3c20 0x7fb7040ca090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:37.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.648+0000 7fb709973640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb7040d3c20 0x7fb7040ca090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:47136/0 (socket says 172.21.15.27:47136) 2024-09-12T03:20:37.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.648+0000 7fb709973640 1 -- 172.21.15.27:0/4063961288 learned_addr learned my addr 172.21.15.27:0/4063961288 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:20:37.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.648+0000 7fb709172640 1 --2- 172.21.15.27:0/4063961288 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb7040b6240 0x7fb704151b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:37.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.649+0000 7fb708971640 1 --2- 172.21.15.27:0/4063961288 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb7040b87d0 0x7fb7040c91e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:37.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.649+0000 7fb709973640 1 -- 172.21.15.27:0/4063961288 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb7040b87d0 msgr2=0x7fb7040c91e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:37.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.649+0000 7fb709973640 1 --2- 172.21.15.27:0/4063961288 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb7040b87d0 0x7fb7040c91e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:37.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.649+0000 7fb709973640 1 -- 172.21.15.27:0/4063961288 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb7040b6240 msgr2=0x7fb704151b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:37.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.649+0000 7fb709973640 1 --2- 172.21.15.27:0/4063961288 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb7040b6240 0x7fb704151b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:37.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.649+0000 7fb709973640 1 -- 172.21.15.27:0/4063961288 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7040c9b10 con 0x7fb7040d3c20 2024-09-12T03:20:37.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.649+0000 7fb709973640 1 --2- 172.21.15.27:0/4063961288 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb7040d3c20 0x7fb7040ca090 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7fb6fc00d800 tx=0x7fb6fc00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:37.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.649+0000 7fb6f9ffb640 1 -- 172.21.15.27:0/4063961288 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6fc00b660 con 0x7fb7040d3c20 2024-09-12T03:20:37.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.649+0000 7fb6f9ffb640 1 -- 172.21.15.27:0/4063961288 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6fc019070 con 0x7fb7040d3c20 2024-09-12T03:20:37.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.649+0000 7fb70b3d6640 1 -- 172.21.15.27:0/4063961288 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7040cbaa0 con 0x7fb7040d3c20 2024-09-12T03:20:37.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.649+0000 7fb6f9ffb640 1 -- 172.21.15.27:0/4063961288 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6fc00ba60 con 0x7fb7040d3c20 2024-09-12T03:20:37.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.649+0000 7fb70b3d6640 1 -- 172.21.15.27:0/4063961288 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7040cc060 con 0x7fb7040d3c20 2024-09-12T03:20:37.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.650+0000 7fb70b3d6640 1 -- 172.21.15.27:0/4063961288 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7040b6620 con 0x7fb7040d3c20 2024-09-12T03:20:37.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.651+0000 7fb6f9ffb640 1 -- 172.21.15.27:0/4063961288 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 90600+0+0 (secure 0 0 0) 0x7fb6fc00bc00 con 0x7fb7040d3c20 2024-09-12T03:20:37.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.651+0000 7fb6f9ffb640 1 --2- 172.21.15.27:0/4063961288 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fb6e806bf80 0x7fb6e806e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:37.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.651+0000 7fb6f9ffb640 1 -- 172.21.15.27:0/4063961288 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 5709+0+0 (secure 0 0 0) 0x7fb6fc08b1c0 con 0x7fb7040d3c20 2024-09-12T03:20:37.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.651+0000 7fb709172640 1 --2- 172.21.15.27:0/4063961288 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fb6e806bf80 0x7fb6e806e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:37.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.652+0000 7fb709172640 1 --2- 172.21.15.27:0/4063961288 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fb6e806bf80 0x7fb6e806e440 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fb700033000 tx=0x7fb700009000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:37.657 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.656+0000 7fb6f9ffb640 1 -- 172.21.15.27:0/4063961288 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fb6fc0594a0 con 0x7fb7040d3c20 2024-09-12T03:20:37.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.832+0000 7fb70b3d6640 1 -- 172.21.15.27:0/4063961288 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fb7040ba370 con 0x7fb7040d3c20 2024-09-12T03:20:37.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.832+0000 7fb6f9ffb640 1 -- 172.21.15.27:0/4063961288 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v18)=0 v18) ==== 122+0+0 (secure 0 0 0) 0x7fb6fc05d7a0 con 0x7fb7040d3c20 2024-09-12T03:20:37.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.834+0000 7fb70b3d6640 1 -- 172.21.15.27:0/4063961288 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fb6e806bf80 msgr2=0x7fb6e806e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:37.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.834+0000 7fb70b3d6640 1 --2- 172.21.15.27:0/4063961288 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fb6e806bf80 0x7fb6e806e440 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fb700033000 tx=0x7fb700009000 comp rx=0 tx=0).stop 2024-09-12T03:20:37.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.834+0000 7fb70b3d6640 1 -- 172.21.15.27:0/4063961288 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb7040d3c20 msgr2=0x7fb7040ca090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:37.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.834+0000 7fb70b3d6640 1 --2- 172.21.15.27:0/4063961288 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb7040d3c20 0x7fb7040ca090 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7fb6fc00d800 tx=0x7fb6fc00dcc0 comp rx=0 tx=0).stop 2024-09-12T03:20:37.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.834+0000 7fb70b3d6640 1 -- 172.21.15.27:0/4063961288 shutdown_connections 2024-09-12T03:20:37.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.834+0000 7fb70b3d6640 1 --2- 172.21.15.27:0/4063961288 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fb6e806bf80 0x7fb6e806e440 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:37.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.834+0000 7fb70b3d6640 1 --2- 172.21.15.27:0/4063961288 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb7040d3c20 0x7fb7040ca090 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:37.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.834+0000 7fb70b3d6640 1 --2- 172.21.15.27:0/4063961288 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb7040b87d0 0x7fb7040c91e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:37.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.834+0000 7fb70b3d6640 1 --2- 172.21.15.27:0/4063961288 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb7040b6240 0x7fb704151b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:37.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.834+0000 7fb70b3d6640 1 -- 172.21.15.27:0/4063961288 >> 172.21.15.27:0/4063961288 conn(0x7fb7040d1420 msgr2=0x7fb7040b4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:37.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.834+0000 7fb70b3d6640 1 -- 172.21.15.27:0/4063961288 shutdown_connections 2024-09-12T03:20:37.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.834+0000 7fb70b3d6640 1 -- 172.21.15.27:0/4063961288 wait complete. 2024-09-12T03:20:37.846 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume create 'abcd@#' 2024-09-12T03:20:37.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.978+0000 7f1ae9e0e640 1 Processor -- start 2024-09-12T03:20:37.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.978+0000 7f1ae9e0e640 1 -- start start 2024-09-12T03:20:37.982 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.979+0000 7f1ae9e0e640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f1ae415e650 0x7f1ae40d1d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:37.982 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.979+0000 7f1ae9e0e640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1ae406cc40 0x7f1ae406d060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:37.982 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.979+0000 7f1ae9e0e640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f1ae406db30 0x7f1ae406df60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:37.982 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.979+0000 7f1ae9e0e640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f1ae40d2280 con 0x7f1ae406db30 2024-09-12T03:20:37.982 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.979+0000 7f1ae9e0e640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f1ae406d5a0 con 0x7f1ae415e650 2024-09-12T03:20:37.982 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.979+0000 7f1ae9e0e640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f1ae406d850 con 0x7f1ae406cc40 2024-09-12T03:20:37.982 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.979+0000 7f1ae3fff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f1ae406db30 0x7f1ae406df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:37.982 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.979+0000 7f1ae3fff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f1ae406db30 0x7f1ae406df60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:47150/0 (socket says 172.21.15.27:47150) 2024-09-12T03:20:37.982 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.979+0000 7f1ae3fff640 1 -- 172.21.15.27:0/57818528 learned_addr learned my addr 172.21.15.27:0/57818528 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:20:37.982 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.979+0000 7f1ae2ffd640 1 --2- 172.21.15.27:0/57818528 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1ae406cc40 0x7f1ae406d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:37.982 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.979+0000 7f1ae37fe640 1 --2- 172.21.15.27:0/57818528 >> v2:172.21.15.45:3300/0 conn(0x7f1ae415e650 0x7f1ae40d1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:37.983 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.979+0000 7f1ae3fff640 1 -- 172.21.15.27:0/57818528 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1ae406cc40 msgr2=0x7f1ae406d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:37.983 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.979+0000 7f1ae3fff640 1 --2- 172.21.15.27:0/57818528 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1ae406cc40 0x7f1ae406d060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:37.983 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.979+0000 7f1ae3fff640 1 -- 172.21.15.27:0/57818528 >> v2:172.21.15.45:3300/0 conn(0x7f1ae415e650 msgr2=0x7f1ae40d1d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:37.983 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.979+0000 7f1ae3fff640 1 --2- 172.21.15.27:0/57818528 >> v2:172.21.15.45:3300/0 conn(0x7f1ae415e650 0x7f1ae40d1d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:37.983 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.979+0000 7f1ae3fff640 1 -- 172.21.15.27:0/57818528 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ae40d1260 con 0x7f1ae406db30 2024-09-12T03:20:37.983 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.979+0000 7f1ae2ffd640 1 --2- 172.21.15.27:0/57818528 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1ae406cc40 0x7f1ae406d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:20:37.983 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.980+0000 7f1ae3fff640 1 --2- 172.21.15.27:0/57818528 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1ae406db30 0x7f1ae406df60 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f1ae4075c50 tx=0x7f1adc02a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7080b17f60094900 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:37.983 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.980+0000 7f1ae27fc640 1 -- 172.21.15.27:0/57818528 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1adc036070 con 0x7f1ae406db30 2024-09-12T03:20:37.983 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.980+0000 7f1ae27fc640 1 -- 172.21.15.27:0/57818528 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1adc031600 con 0x7f1ae406db30 2024-09-12T03:20:37.983 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.980+0000 7f1ae27fc640 1 -- 172.21.15.27:0/57818528 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1adc0319d0 con 0x7f1ae406db30 2024-09-12T03:20:37.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.981+0000 7f1ae9e0e640 1 -- 172.21.15.27:0/57818528 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1ae406db30 msgr2=0x7f1ae406df60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:37.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.981+0000 7f1ae9e0e640 1 --2- 172.21.15.27:0/57818528 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1ae406db30 0x7f1ae406df60 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f1ae4075c50 tx=0x7f1adc02a610 comp rx=0 tx=0).stop 2024-09-12T03:20:37.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.981+0000 7f1ae9e0e640 1 -- 172.21.15.27:0/57818528 shutdown_connections 2024-09-12T03:20:37.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.981+0000 7f1ae9e0e640 1 --2- 172.21.15.27:0/57818528 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1ae406db30 0x7f1ae406df60 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:37.988 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.981+0000 7f1ae9e0e640 1 --2- 172.21.15.27:0/57818528 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1ae406cc40 0x7f1ae406d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:37.988 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.981+0000 7f1ae9e0e640 1 --2- 172.21.15.27:0/57818528 >> v2:172.21.15.45:3300/0 conn(0x7f1ae415e650 0x7f1ae40d1d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:37.988 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.981+0000 7f1ae9e0e640 1 -- 172.21.15.27:0/57818528 >> 172.21.15.27:0/57818528 conn(0x7f1ae406f320 msgr2=0x7f1ae406f700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:37.988 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.981+0000 7f1ae9e0e640 1 -- 172.21.15.27:0/57818528 shutdown_connections 2024-09-12T03:20:37.988 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.981+0000 7f1ae9e0e640 1 -- 172.21.15.27:0/57818528 wait complete. 2024-09-12T03:20:37.988 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.982+0000 7f1ae9e0e640 1 Processor -- start 2024-09-12T03:20:37.988 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.982+0000 7f1ae9e0e640 1 -- start start 2024-09-12T03:20:37.988 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.983+0000 7f1ae9e0e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1ae406cc40 0x7f1ae4154af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:37.989 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.983+0000 7f1ae9e0e640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1ae406db30 0x7f1ae414b060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:37.989 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.983+0000 7f1ae9e0e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1ae415e650 0x7f1ae4158b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:37.989 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.983+0000 7f1ae9e0e640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f1ae415ebb0 con 0x7f1ae415e650 2024-09-12T03:20:37.989 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.983+0000 7f1ae9e0e640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f1ae415ea30 con 0x7f1ae406cc40 2024-09-12T03:20:37.989 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.983+0000 7f1ae9e0e640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f1ae406e4a0 con 0x7f1ae406db30 2024-09-12T03:20:37.989 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.983+0000 7f1ae3fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1ae415e650 0x7f1ae4158b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:37.989 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.983+0000 7f1ae3fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1ae415e650 0x7f1ae4158b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:47162/0 (socket says 172.21.15.27:47162) 2024-09-12T03:20:37.989 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.983+0000 7f1ae3fff640 1 -- 172.21.15.27:0/4241402247 learned_addr learned my addr 172.21.15.27:0/4241402247 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:20:37.989 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.983+0000 7f1ae37fe640 1 --2- 172.21.15.27:0/4241402247 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1ae406cc40 0x7f1ae4154af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:37.989 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.983+0000 7f1ae2ffd640 1 --2- 172.21.15.27:0/4241402247 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1ae406db30 0x7f1ae414b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:37.989 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.983+0000 7f1ae3fff640 1 -- 172.21.15.27:0/4241402247 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1ae406db30 msgr2=0x7f1ae414b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:37.989 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.983+0000 7f1ae3fff640 1 --2- 172.21.15.27:0/4241402247 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1ae406db30 0x7f1ae414b060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:37.990 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.983+0000 7f1ae3fff640 1 -- 172.21.15.27:0/4241402247 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1ae406cc40 msgr2=0x7f1ae4154af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:37.990 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.983+0000 7f1ae3fff640 1 --2- 172.21.15.27:0/4241402247 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1ae406cc40 0x7f1ae4154af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:37.990 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.983+0000 7f1ae3fff640 1 -- 172.21.15.27:0/4241402247 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ae415b280 con 0x7f1ae415e650 2024-09-12T03:20:37.990 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.983+0000 7f1ae2ffd640 1 --2- 172.21.15.27:0/4241402247 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1ae406db30 0x7f1ae414b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:20:37.990 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.984+0000 7f1ae3fff640 1 --2- 172.21.15.27:0/4241402247 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1ae415e650 0x7f1ae4158b10 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f1adc02ac00 tx=0x7f1adc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:37.990 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.984+0000 7f1acbfff640 1 -- 172.21.15.27:0/4241402247 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1adc03f020 con 0x7f1ae415e650 2024-09-12T03:20:37.990 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.984+0000 7f1acbfff640 1 -- 172.21.15.27:0/4241402247 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1adc037040 con 0x7f1ae415e650 2024-09-12T03:20:37.990 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.984+0000 7f1acbfff640 1 -- 172.21.15.27:0/4241402247 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1adc036070 con 0x7f1ae415e650 2024-09-12T03:20:37.990 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.984+0000 7f1ae9e0e640 1 -- 172.21.15.27:0/4241402247 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ae41f6330 con 0x7f1ae415e650 2024-09-12T03:20:37.990 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.984+0000 7f1ae9e0e640 1 -- 172.21.15.27:0/4241402247 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ae41f68f0 con 0x7f1ae415e650 2024-09-12T03:20:37.990 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.985+0000 7f1ae9e0e640 1 -- 172.21.15.27:0/4241402247 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ae406df60 con 0x7f1ae415e650 2024-09-12T03:20:37.990 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.986+0000 7f1acbfff640 1 -- 172.21.15.27:0/4241402247 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 90600+0+0 (secure 0 0 0) 0x7f1adc042020 con 0x7f1ae415e650 2024-09-12T03:20:37.990 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.986+0000 7f1acbfff640 1 --2- 172.21.15.27:0/4241402247 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f1ad006bf50 0x7f1ad006e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:37.991 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.986+0000 7f1acbfff640 1 -- 172.21.15.27:0/4241402247 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 5709+0+0 (secure 0 0 0) 0x7f1adc0ab560 con 0x7f1ae415e650 2024-09-12T03:20:37.991 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.986+0000 7f1ae37fe640 1 --2- 172.21.15.27:0/4241402247 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f1ad006bf50 0x7f1ad006e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:37.992 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.986+0000 7f1ae37fe640 1 --2- 172.21.15.27:0/4241402247 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f1ad006bf50 0x7f1ad006e410 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f1ad400aac0 tx=0x7f1ad4008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:37.992 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:37.990+0000 7f1acbfff640 1 -- 172.21.15.27:0/4241402247 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f1adc049030 con 0x7f1ae415e650 2024-09-12T03:20:38.189 INFO:teuthology.orchestra.run.smithi027.stderr:Invalid command: invalid chars @# in abcd@# 2024-09-12T03:20:38.190 INFO:teuthology.orchestra.run.smithi027.stderr:fs volume create [] : Create a CephFS volume 2024-09-12T03:20:38.190 INFO:teuthology.orchestra.run.smithi027.stderr:Error EINVAL: invalid command 2024-09-12T03:20:38.193 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-12T03:20:38.193 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume create testvol 2024-09-12T03:20:38.306 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:38.304+0000 7f1ed72c0640 1 Processor -- start 2024-09-12T03:20:38.306 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:38.304+0000 7f1ed72c0640 1 -- start start 2024-09-12T03:20:38.307 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:38.305+0000 7f1ed72c0640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1ed0162040 0x7f1ed00d1da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:38.307 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:38.305+0000 7f1ed72c0640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f1ed006dfd0 0x7f1ed00b7a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:38.307 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:38.305+0000 7f1ed72c0640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ed00d2e30 con 0x7f1ed00d3790 2024-09-12T03:20:38.307 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:38.305+0000 7f1ed72c0640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f1ed00b7f90 con 0x7f1ed006dfd0 2024-09-12T03:20:38.307 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:38.305+0000 7f1ed72c0640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f1ed0162420 con 0x7f1ed0162040 2024-09-12T03:20:38.307 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:38.305+0000 7f1ed585d640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f1ed006dfd0 0x7f1ed00b7a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:38.307 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:38.305+0000 7f1ed485b640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f1ed00d3790 0x7f1ed006d8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:57726/0 (socket says 172.21.15.27:57726) 2024-09-12T03:20:38.308 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:38.305+0000 7f1ed485b640 1 -- 172.21.15.27:0/605918641 learned_addr learned my addr 172.21.15.27:0/605918641 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:20:38.308 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:38.306+0000 7f1ed505c640 1 --2- 172.21.15.27:0/605918641 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1ed0162040 0x7f1ed00d1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:38.308 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:38.306+0000 7f1ed585d640 1 -- 172.21.15.27:0/605918641 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1ed0162040 msgr2=0x7f1ed00d1da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:38.308 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:38.306+0000 7f1ed585d640 1 --2- 172.21.15.27:0/605918641 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1ed0162040 0x7f1ed00d1da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:38.308 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:38.306+0000 7f1ed585d640 1 -- 172.21.15.27:0/605918641 >> v1:172.21.15.27:6789/0 conn(0x7f1ed00d3790 legacy=0x7f1ed006d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:38.308 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:38.306+0000 7f1ecffff640 1 -- 172.21.15.27:0/605918641 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 397210722 0 0) 0x7f1ed00d2e30 con 0x7f1ed00d3790 2024-09-12T03:20:38.308 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:38.306+0000 7f1ed585d640 1 -- 172.21.15.27:0/605918641 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ed00d30d0 con 0x7f1ed006dfd0 2024-09-12T03:20:38.309 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:38.306+0000 7f1ed585d640 1 --2- 172.21.15.27:0/605918641 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1ed006dfd0 0x7f1ed00b7a50 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f1ec0009d90 tx=0x7f1ec002a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d977cc3eaa28b6c5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:38.309 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:38.307+0000 7f1ecffff640 1 -- 172.21.15.27:0/605918641 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ec0007740 con 0x7f1ed006dfd0 2024-09-12T03:20:38.310 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:38.307+0000 7f1ecffff640 1 -- 172.21.15.27:0/605918641 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ec00078e0 con 0x7f1ed006dfd0 2024-09-12T03:20:38.310 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:38.307+0000 7f1ecffff640 1 -- 172.21.15.27:0/605918641 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ec0007cb0 con 0x7f1ed006dfd0 2024-09-12T03:20:38.310 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:38.307+0000 7f1ed72c0640 1 -- 172.21.15.27:0/605918641 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1ed006dfd0 msgr2=0x7f1ed00b7a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:38.310 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:38.307+0000 7f1ed72c0640 1 --2- 172.21.15.27:0/605918641 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1ed006dfd0 0x7f1ed00b7a50 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f1ec0009d90 tx=0x7f1ec002a660 comp rx=0 tx=0).stop 2024-09-12T03:20:38.310 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:38.307+0000 7f1ed72c0640 1 -- 172.21.15.27:0/605918641 shutdown_connections 2024-09-12T03:20:38.310 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:38.307+0000 7f1ed72c0640 1 --2- 172.21.15.27:0/605918641 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1ed006dfd0 0x7f1ed00b7a50 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:38.310 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:38.307+0000 7f1ed72c0640 1 --2- 172.21.15.27:0/605918641 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1ed0162040 0x7f1ed00d1da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:38.310 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:38.307+0000 7f1ed72c0640 1 -- 172.21.15.27:0/605918641 >> 172.21.15.27:0/605918641 conn(0x7f1ed00b6700 msgr2=0x7f1ed00b6b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:38.310 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:38.307+0000 7f1ed72c0640 1 -- 172.21.15.27:0/605918641 shutdown_connections 2024-09-12T03:20:38.310 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:38.307+0000 7f1ed72c0640 1 -- 172.21.15.27:0/605918641 wait complete. 2024-09-12T03:20:38.310 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:38.308+0000 7f1ed72c0640 1 Processor -- start 2024-09-12T03:20:38.310 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:38.309+0000 7f1ed72c0640 1 -- start start 2024-09-12T03:20:38.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:38.309+0000 7f1ed72c0640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1ed006dfd0 0x7f1ed014fb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:38.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:38.309+0000 7f1ed72c0640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1ed00d3790 0x7f1ed00ccbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:38.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:38.309+0000 7f1ed72c0640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1ed0162040 0x7f1ed015b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:38.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:38.309+0000 7f1ed72c0640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f1ed00d4530 con 0x7f1ed006dfd0 2024-09-12T03:20:38.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:38.309+0000 7f1ed72c0640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f1ed00cee00 con 0x7f1ed0162040 2024-09-12T03:20:38.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:38.309+0000 7f1ed72c0640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f1ed00d4150 con 0x7f1ed00d3790 2024-09-12T03:20:38.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:38.309+0000 7f1ed505c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1ed006dfd0 0x7f1ed014fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:38.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:38.309+0000 7f1ed505c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1ed006dfd0 0x7f1ed014fb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:47178/0 (socket says 172.21.15.27:47178) 2024-09-12T03:20:38.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:38.309+0000 7f1ed505c640 1 -- 172.21.15.27:0/3104340149 learned_addr learned my addr 172.21.15.27:0/3104340149 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:20:38.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:38.310+0000 7f1ed485b640 1 --2- 172.21.15.27:0/3104340149 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1ed00d3790 0x7f1ed00ccbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:38.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:38.310+0000 7f1ed505c640 1 -- 172.21.15.27:0/3104340149 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1ed00d3790 msgr2=0x7f1ed00ccbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:38.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:38.310+0000 7f1ed505c640 1 --2- 172.21.15.27:0/3104340149 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1ed00d3790 0x7f1ed00ccbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:38.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:38.310+0000 7f1ed505c640 1 -- 172.21.15.27:0/3104340149 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1ed0162040 msgr2=0x7f1ed015b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:38.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:38.310+0000 7f1ed505c640 1 --2- 172.21.15.27:0/3104340149 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1ed0162040 0x7f1ed015b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:38.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:38.310+0000 7f1ed505c640 1 -- 172.21.15.27:0/3104340149 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ed00cc430 con 0x7f1ed006dfd0 2024-09-12T03:20:38.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:38.310+0000 7f1ed505c640 1 --2- 172.21.15.27:0/3104340149 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1ed006dfd0 0x7f1ed014fb90 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f1ec800d620 tx=0x7f1ec800dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:38.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:38.310+0000 7f1ecdffb640 1 -- 172.21.15.27:0/3104340149 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ec800b620 con 0x7f1ed006dfd0 2024-09-12T03:20:38.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:38.310+0000 7f1ecdffb640 1 -- 172.21.15.27:0/3104340149 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ec8019070 con 0x7f1ed006dfd0 2024-09-12T03:20:38.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:38.310+0000 7f1ed72c0640 1 -- 172.21.15.27:0/3104340149 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ed00ce5a0 con 0x7f1ed006dfd0 2024-09-12T03:20:38.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:38.310+0000 7f1ecdffb640 1 -- 172.21.15.27:0/3104340149 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ec800ba00 con 0x7f1ed006dfd0 2024-09-12T03:20:38.314 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:38.310+0000 7f1ed72c0640 1 -- 172.21.15.27:0/3104340149 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ed00cc6d0 con 0x7f1ed006dfd0 2024-09-12T03:20:38.314 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:38.311+0000 7f1ed72c0640 1 -- 172.21.15.27:0/3104340149 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ed00b7a50 con 0x7f1ed006dfd0 2024-09-12T03:20:38.314 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:38.312+0000 7f1ecdffb640 1 -- 172.21.15.27:0/3104340149 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 90600+0+0 (secure 0 0 0) 0x7f1ec800bba0 con 0x7f1ed006dfd0 2024-09-12T03:20:38.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:38.312+0000 7f1ecdffb640 1 --2- 172.21.15.27:0/3104340149 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f1ebc06bf50 0x7f1ebc06e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:38.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:38.312+0000 7f1ecdffb640 1 -- 172.21.15.27:0/3104340149 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 5709+0+0 (secure 0 0 0) 0x7f1ec808b370 con 0x7f1ed006dfd0 2024-09-12T03:20:38.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:38.312+0000 7f1ed485b640 1 --2- 172.21.15.27:0/3104340149 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f1ebc06bf50 0x7f1ebc06e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:38.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:38.313+0000 7f1ed485b640 1 --2- 172.21.15.27:0/3104340149 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f1ebc06bf50 0x7f1ebc06e410 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f1ec4007fd0 tx=0x7f1ec4009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:38.317 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:38.316+0000 7f1ecdffb640 1 -- 172.21.15.27:0/3104340149 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f1ec8059ca0 con 0x7f1ed006dfd0 2024-09-12T03:20:38.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:38.516+0000 7f1ed72c0640 1 -- 172.21.15.27:0/3104340149 --> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] -- mgr_command(tid 0: {"prefix": "fs volume create", "name": "testvol", "target": ["mon-mgr", ""]}) -- 0x7f1ed00ce350 con 0x7f1ebc06bf50 2024-09-12T03:20:41.216 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:41.215+0000 7f1ecdffb640 1 -- 172.21.15.27:0/3104340149 <== mgr.4113 v2:172.21.15.27:6836/1226920526 1 ==== mgr_command_reply(tid 0: 0 Volume created successfully (no MDS daemons created)) ==== 60+0+0 (secure 0 0 0) 0x7f1ed00ce350 con 0x7f1ebc06bf50 2024-09-12T03:20:41.217 INFO:teuthology.orchestra.run.smithi027.stderr:Volume created successfully (no MDS daemons created) 2024-09-12T03:20:41.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:41.219+0000 7f1ed72c0640 1 -- 172.21.15.27:0/3104340149 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f1ebc06bf50 msgr2=0x7f1ebc06e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:41.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:41.219+0000 7f1ed72c0640 1 --2- 172.21.15.27:0/3104340149 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f1ebc06bf50 0x7f1ebc06e410 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f1ec4007fd0 tx=0x7f1ec4009040 comp rx=0 tx=0).stop 2024-09-12T03:20:41.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:41.220+0000 7f1ed72c0640 1 -- 172.21.15.27:0/3104340149 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1ed006dfd0 msgr2=0x7f1ed014fb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:41.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:41.220+0000 7f1ed72c0640 1 --2- 172.21.15.27:0/3104340149 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1ed006dfd0 0x7f1ed014fb90 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f1ec800d620 tx=0x7f1ec800dae0 comp rx=0 tx=0).stop 2024-09-12T03:20:41.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:41.220+0000 7f1ed72c0640 1 -- 172.21.15.27:0/3104340149 shutdown_connections 2024-09-12T03:20:41.222 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:41.220+0000 7f1ed72c0640 1 --2- 172.21.15.27:0/3104340149 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1ed0162040 0x7f1ed015b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:41.222 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:41.220+0000 7f1ed72c0640 1 --2- 172.21.15.27:0/3104340149 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1ed00d3790 0x7f1ed00ccbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:41.222 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:41.220+0000 7f1ed72c0640 1 --2- 172.21.15.27:0/3104340149 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f1ebc06bf50 0x7f1ebc06e410 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:41.222 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:41.220+0000 7f1ed72c0640 1 --2- 172.21.15.27:0/3104340149 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1ed006dfd0 0x7f1ed014fb90 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:41.222 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:41.220+0000 7f1ed72c0640 1 -- 172.21.15.27:0/3104340149 >> 172.21.15.27:0/3104340149 conn(0x7f1ed00b6700 msgr2=0x7f1ed00b5710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:41.223 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:41.222+0000 7f1ed72c0640 1 -- 172.21.15.27:0/3104340149 shutdown_connections 2024-09-12T03:20:41.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:41.223+0000 7f1ed72c0640 1 -- 172.21.15.27:0/3104340149 wait complete. 2024-09-12T03:20:41.241 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create testvol 'abcd@#' 2024-09-12T03:20:41.352 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:41.351+0000 7f8fec5f1640 1 Processor -- start 2024-09-12T03:20:41.352 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:41.351+0000 7f8fec5f1640 1 -- start start 2024-09-12T03:20:41.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:41.352+0000 7f8fec5f1640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f8fe4162040 0x7f8fe40d1da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:41.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:41.352+0000 7f8fec5f1640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f8fe40d3790 0x7f8fe40b7a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:41.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:41.352+0000 7f8fec5f1640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f8fe406d8c0 0x7f8fe406dd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:41.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:41.352+0000 7f8fec5f1640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f8fe40b7f90 con 0x7f8fe40d3790 2024-09-12T03:20:41.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:41.352+0000 7f8fec5f1640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f8fe40d2e30 con 0x7f8fe4162040 2024-09-12T03:20:41.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:41.352+0000 7f8fec5f1640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f8fe4162420 con 0x7f8fe406d8c0 2024-09-12T03:20:41.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:41.352+0000 7f8fe9b8c640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f8fe40d3790 0x7f8fe40b7a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:41.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:41.353+0000 7f8fe9b8c640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f8fe40d3790 0x7f8fe40b7a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42502/0 (socket says 172.21.15.27:42502) 2024-09-12T03:20:41.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:41.353+0000 7f8fe9b8c640 1 -- 172.21.15.27:0/1226714896 learned_addr learned my addr 172.21.15.27:0/1226714896 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:20:41.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:41.353+0000 7f8feab8e640 1 --2- 172.21.15.27:0/1226714896 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f8fe406d8c0 0x7f8fe406dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:41.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:41.353+0000 7f8fea38d640 1 --2- 172.21.15.27:0/1226714896 >> v2:172.21.15.45:3300/0 conn(0x7f8fe4162040 0x7f8fe40d1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:41.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:41.353+0000 7f8fe9b8c640 1 -- 172.21.15.27:0/1226714896 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f8fe406d8c0 msgr2=0x7f8fe406dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:41.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:41.353+0000 7f8fe9b8c640 1 --2- 172.21.15.27:0/1226714896 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f8fe406d8c0 0x7f8fe406dd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:41.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:41.353+0000 7f8fe9b8c640 1 -- 172.21.15.27:0/1226714896 >> v2:172.21.15.45:3300/0 conn(0x7f8fe4162040 msgr2=0x7f8fe40d1da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:41.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:41.353+0000 7f8fe9b8c640 1 --2- 172.21.15.27:0/1226714896 >> v2:172.21.15.45:3300/0 conn(0x7f8fe4162040 0x7f8fe40d1da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:41.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:41.353+0000 7f8fe9b8c640 1 -- 172.21.15.27:0/1226714896 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fe40d30d0 con 0x7f8fe40d3790 2024-09-12T03:20:41.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:41.353+0000 7f8feab8e640 1 --2- 172.21.15.27:0/1226714896 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f8fe406d8c0 0x7f8fe406dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:20:41.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:41.354+0000 7f8fe9b8c640 1 --2- 172.21.15.27:0/1226714896 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8fe40d3790 0x7f8fe40b7a50 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f8fe40d17b0 tx=0x7f8fd802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2673984ac458cfc6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:41.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:41.354+0000 7f8fe938b640 1 -- 172.21.15.27:0/1226714896 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fd8037070 con 0x7f8fe40d3790 2024-09-12T03:20:41.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:41.354+0000 7f8fe938b640 1 -- 172.21.15.27:0/1226714896 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8fd8032440 con 0x7f8fe40d3790 2024-09-12T03:20:41.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:41.354+0000 7f8fe938b640 1 -- 172.21.15.27:0/1226714896 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fd8032810 con 0x7f8fe40d3790 2024-09-12T03:20:41.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:41.354+0000 7f8fec5f1640 1 -- 172.21.15.27:0/1226714896 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8fe40d3790 msgr2=0x7f8fe40b7a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:41.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:41.354+0000 7f8fec5f1640 1 --2- 172.21.15.27:0/1226714896 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8fe40d3790 0x7f8fe40b7a50 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f8fe40d17b0 tx=0x7f8fd802a1e0 comp rx=0 tx=0).stop 2024-09-12T03:20:41.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:41.354+0000 7f8fec5f1640 1 -- 172.21.15.27:0/1226714896 shutdown_connections 2024-09-12T03:20:41.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:41.354+0000 7f8fec5f1640 1 --2- 172.21.15.27:0/1226714896 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f8fe406d8c0 0x7f8fe406dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:41.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:41.354+0000 7f8fec5f1640 1 --2- 172.21.15.27:0/1226714896 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8fe40d3790 0x7f8fe40b7a50 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:41.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:41.354+0000 7f8fec5f1640 1 --2- 172.21.15.27:0/1226714896 >> v2:172.21.15.45:3300/0 conn(0x7f8fe4162040 0x7f8fe40d1da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:41.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:41.354+0000 7f8fec5f1640 1 -- 172.21.15.27:0/1226714896 >> 172.21.15.27:0/1226714896 conn(0x7f8fe40b6700 msgr2=0x7f8fe40b6b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:41.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:41.355+0000 7f8fec5f1640 1 -- 172.21.15.27:0/1226714896 shutdown_connections 2024-09-12T03:20:41.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:41.355+0000 7f8fec5f1640 1 -- 172.21.15.27:0/1226714896 wait complete. 2024-09-12T03:20:41.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:41.356+0000 7f8fec5f1640 1 Processor -- start 2024-09-12T03:20:41.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:41.356+0000 7f8fec5f1640 1 -- start start 2024-09-12T03:20:41.359 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:41.356+0000 7f8fec5f1640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8fe406d8c0 0x7f8fe4160670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:41.360 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:41.356+0000 7f8fec5f1640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f8fe40d3790 0x7f8fe41f7da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:41.360 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:41.356+0000 7f8fec5f1640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8fe4162040 0x7f8fe41f82e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:41.360 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:41.356+0000 7f8fec5f1640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f8fe40d4530 con 0x7f8fe4162040 2024-09-12T03:20:41.360 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:41.356+0000 7f8fec5f1640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f8fe40cee00 con 0x7f8fe406d8c0 2024-09-12T03:20:41.360 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:41.356+0000 7f8fec5f1640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f8fe40d4150 con 0x7f8fe40d3790 2024-09-12T03:20:41.360 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:41.357+0000 7f8feab8e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8fe4162040 0x7f8fe41f82e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:41.360 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:41.357+0000 7f8feab8e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8fe4162040 0x7f8fe41f82e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42514/0 (socket says 172.21.15.27:42514) 2024-09-12T03:20:41.360 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:41.357+0000 7f8feab8e640 1 -- 172.21.15.27:0/3513229055 learned_addr learned my addr 172.21.15.27:0/3513229055 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:20:41.360 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:41.357+0000 7f8fea38d640 1 --2- 172.21.15.27:0/3513229055 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8fe406d8c0 0x7f8fe4160670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:41.360 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:41.357+0000 7f8fe9b8c640 1 --2- 172.21.15.27:0/3513229055 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f8fe40d3790 0x7f8fe41f7da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:41.360 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:41.357+0000 7f8feab8e640 1 -- 172.21.15.27:0/3513229055 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f8fe40d3790 msgr2=0x7f8fe41f7da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:41.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:41.357+0000 7f8feab8e640 1 --2- 172.21.15.27:0/3513229055 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f8fe40d3790 0x7f8fe41f7da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:41.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:41.357+0000 7f8feab8e640 1 -- 172.21.15.27:0/3513229055 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8fe406d8c0 msgr2=0x7f8fe4160670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:41.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:41.357+0000 7f8feab8e640 1 --2- 172.21.15.27:0/3513229055 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8fe406d8c0 0x7f8fe4160670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:41.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:41.357+0000 7f8feab8e640 1 -- 172.21.15.27:0/3513229055 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fe415f5d0 con 0x7f8fe4162040 2024-09-12T03:20:41.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:41.357+0000 7f8feab8e640 1 --2- 172.21.15.27:0/3513229055 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8fe4162040 0x7f8fe41f82e0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f8fd4007e10 tx=0x7f8fd400d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:41.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:41.357+0000 7f8fe2ffd640 1 -- 172.21.15.27:0/3513229055 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fd401a070 con 0x7f8fe4162040 2024-09-12T03:20:41.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:41.357+0000 7f8fe2ffd640 1 -- 172.21.15.27:0/3513229055 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8fd4003b90 con 0x7f8fe4162040 2024-09-12T03:20:41.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:41.357+0000 7f8fec5f1640 1 -- 172.21.15.27:0/3513229055 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fe415f8b0 con 0x7f8fe4162040 2024-09-12T03:20:41.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:41.357+0000 7f8fe2ffd640 1 -- 172.21.15.27:0/3513229055 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fd4004030 con 0x7f8fe4162040 2024-09-12T03:20:41.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:41.357+0000 7f8fec5f1640 1 -- 172.21.15.27:0/3513229055 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fe415fe70 con 0x7f8fe4162040 2024-09-12T03:20:41.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:41.358+0000 7f8fec5f1640 1 -- 172.21.15.27:0/3513229055 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8fe40b7a50 con 0x7f8fe4162040 2024-09-12T03:20:41.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:41.359+0000 7f8fe2ffd640 1 -- 172.21.15.27:0/3513229055 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 90600+0+0 (secure 0 0 0) 0x7f8fd401d070 con 0x7f8fe4162040 2024-09-12T03:20:41.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:41.359+0000 7f8fe2ffd640 1 --2- 172.21.15.27:0/3513229055 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f8fd006bf50 0x7f8fd006e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:41.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:41.360+0000 7f8fe2ffd640 1 -- 172.21.15.27:0/3513229055 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 6543+0+0 (secure 0 0 0) 0x7f8fd408ba00 con 0x7f8fe4162040 2024-09-12T03:20:41.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:41.360+0000 7f8fea38d640 1 --2- 172.21.15.27:0/3513229055 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f8fd006bf50 0x7f8fd006e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:41.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:41.360+0000 7f8fea38d640 1 --2- 172.21.15.27:0/3513229055 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f8fd006bf50 0x7f8fd006e410 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f8fdc009aa0 tx=0x7f8fdc009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:41.364 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:41.363+0000 7f8fe2ffd640 1 -- 172.21.15.27:0/3513229055 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f8fd4024020 con 0x7f8fe4162040 2024-09-12T03:20:41.561 INFO:teuthology.orchestra.run.smithi027.stderr:Invalid command: invalid chars #@ in abcd@# 2024-09-12T03:20:41.561 INFO:teuthology.orchestra.run.smithi027.stderr:fs subvolumegroup create [] [] [] [] [] : Create a CephFS subvolume group in a volume, and optionally, with a specific data pool layout, and a specific numeric mode 2024-09-12T03:20:41.561 INFO:teuthology.orchestra.run.smithi027.stderr:Error EINVAL: invalid command 2024-09-12T03:20:41.565 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-12T03:20:41.566 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create testvol 'abcd@#' 2024-09-12T03:20:41.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:41.670+0000 7f41c29c4640 1 Processor -- start 2024-09-12T03:20:41.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:41.670+0000 7f41c29c4640 1 -- start start 2024-09-12T03:20:41.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:41.671+0000 7f41c29c4640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f41bc0b6540 0x7f41bc0af370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:41.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:41.671+0000 7f41c29c4640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f41bc0b5940 con 0x7f41bc15e900 2024-09-12T03:20:41.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:41.671+0000 7f41c29c4640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f41bc0af950 con 0x7f41bc06d730 2024-09-12T03:20:41.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:41.671+0000 7f41c29c4640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f41bc15ece0 con 0x7f41bc0b6540 2024-09-12T03:20:41.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:41.671+0000 7f41bb7fe640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f41bc0b6540 0x7f41bc0af370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:41.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:41.671+0000 7f41bbfff640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f41bc15e900 0x7f41bc0b5e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:47190/0 (socket says 172.21.15.27:47190) 2024-09-12T03:20:41.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:41.671+0000 7f41bbfff640 1 -- 172.21.15.27:0/1160247658 learned_addr learned my addr 172.21.15.27:0/1160247658 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:20:41.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:41.671+0000 7f41bb7fe640 1 -- 172.21.15.27:0/1160247658 >> v1:172.21.15.45:6789/0 conn(0x7f41bc06d730 legacy=0x7f41bc06ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:41.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:41.672+0000 7f41bb7fe640 1 -- 172.21.15.27:0/1160247658 >> v1:172.21.15.27:6789/0 conn(0x7f41bc15e900 legacy=0x7f41bc0b5e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:41.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:41.672+0000 7f41bb7fe640 1 -- 172.21.15.27:0/1160247658 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41bc0afc30 con 0x7f41bc0b6540 2024-09-12T03:20:41.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:41.672+0000 7f41baffd640 1 -- 172.21.15.27:0/1160247658 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4164221293 0 0) 0x7f41ac001110 con 0x7f41bc15e900 2024-09-12T03:20:41.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:41.672+0000 7f41bb7fe640 1 --2- 172.21.15.27:0/1160247658 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f41bc0b6540 0x7f41bc0af370 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f41b00097a0 tx=0x7f41b002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=969ee2add7eef657 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:41.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:41.672+0000 7f41baffd640 1 -- 172.21.15.27:0/1160247658 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41b0037070 con 0x7f41bc0b6540 2024-09-12T03:20:41.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:41.672+0000 7f41baffd640 1 -- 172.21.15.27:0/1160247658 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41b0032440 con 0x7f41bc0b6540 2024-09-12T03:20:41.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:41.672+0000 7f41baffd640 1 -- 172.21.15.27:0/1160247658 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41b0032810 con 0x7f41bc0b6540 2024-09-12T03:20:41.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:41.673+0000 7f41c29c4640 1 -- 172.21.15.27:0/1160247658 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f41bc0b6540 msgr2=0x7f41bc0af370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:41.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:41.673+0000 7f41c29c4640 1 --2- 172.21.15.27:0/1160247658 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f41bc0b6540 0x7f41bc0af370 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f41b00097a0 tx=0x7f41b002a1e0 comp rx=0 tx=0).stop 2024-09-12T03:20:41.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:41.673+0000 7f41c29c4640 1 -- 172.21.15.27:0/1160247658 shutdown_connections 2024-09-12T03:20:41.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:41.673+0000 7f41c29c4640 1 --2- 172.21.15.27:0/1160247658 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f41bc0b6540 0x7f41bc0af370 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:41.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:41.673+0000 7f41c29c4640 1 -- 172.21.15.27:0/1160247658 >> 172.21.15.27:0/1160247658 conn(0x7f41bc06cc40 msgr2=0x7f41bc06d020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:41.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:41.673+0000 7f41c29c4640 1 -- 172.21.15.27:0/1160247658 shutdown_connections 2024-09-12T03:20:41.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:41.673+0000 7f41c29c4640 1 -- 172.21.15.27:0/1160247658 wait complete. 2024-09-12T03:20:41.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:41.674+0000 7f41c29c4640 1 Processor -- start 2024-09-12T03:20:41.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:41.674+0000 7f41c29c4640 1 -- start start 2024-09-12T03:20:41.678 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:41.675+0000 7f41c29c4640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f41bc06d730 0x7f41bc148d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:41.678 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:41.675+0000 7f41c29c4640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f41bc0b6540 0x7f41bc152d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:41.678 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:41.675+0000 7f41c29c4640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f41bc15e900 0x7f41bc1552d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:41.678 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:41.675+0000 7f41c29c4640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f41bc0b01f0 con 0x7f41bc06d730 2024-09-12T03:20:41.678 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:41.675+0000 7f41c29c4640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f41bc06db40 con 0x7f41bc15e900 2024-09-12T03:20:41.678 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:41.675+0000 7f41c29c4640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f41bc06e880 con 0x7f41bc0b6540 2024-09-12T03:20:41.678 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:41.675+0000 7f41bbfff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f41bc06d730 0x7f41bc148d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:41.678 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:41.675+0000 7f41bbfff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f41bc06d730 0x7f41bc148d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42522/0 (socket says 172.21.15.27:42522) 2024-09-12T03:20:41.679 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:41.675+0000 7f41bbfff640 1 -- 172.21.15.27:0/1917852155 learned_addr learned my addr 172.21.15.27:0/1917852155 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:20:41.679 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:41.675+0000 7f41bbfff640 1 -- 172.21.15.27:0/1917852155 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f41bc0b6540 msgr2=0x7f41bc152d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:41.679 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:41.675+0000 7f41bbfff640 1 --2- 172.21.15.27:0/1917852155 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f41bc0b6540 0x7f41bc152d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:41.679 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:41.675+0000 7f41bbfff640 1 -- 172.21.15.27:0/1917852155 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f41bc15e900 msgr2=0x7f41bc1552d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:41.679 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:41.675+0000 7f41bbfff640 1 --2- 172.21.15.27:0/1917852155 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f41bc15e900 0x7f41bc1552d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:41.679 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:41.676+0000 7f41bbfff640 1 -- 172.21.15.27:0/1917852155 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41bc15fff0 con 0x7f41bc06d730 2024-09-12T03:20:41.679 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:41.676+0000 7f41bbfff640 1 --2- 172.21.15.27:0/1917852155 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f41bc06d730 0x7f41bc148d80 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f41ac00c430 tx=0x7f41ac00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:41.679 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:41.676+0000 7f41b8ff9640 1 -- 172.21.15.27:0/1917852155 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41ac0074e0 con 0x7f41bc06d730 2024-09-12T03:20:41.679 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:41.676+0000 7f41b8ff9640 1 -- 172.21.15.27:0/1917852155 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41ac0047e0 con 0x7f41bc06d730 2024-09-12T03:20:41.679 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:41.676+0000 7f41c29c4640 1 -- 172.21.15.27:0/1917852155 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41bc1602d0 con 0x7f41bc06d730 2024-09-12T03:20:41.679 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:41.676+0000 7f41b8ff9640 1 -- 172.21.15.27:0/1917852155 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41ac004bd0 con 0x7f41bc06d730 2024-09-12T03:20:41.679 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:41.676+0000 7f41c29c4640 1 -- 172.21.15.27:0/1917852155 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41bc160890 con 0x7f41bc06d730 2024-09-12T03:20:41.679 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:41.677+0000 7f41c29c4640 1 -- 172.21.15.27:0/1917852155 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41bc069560 con 0x7f41bc06d730 2024-09-12T03:20:41.680 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:41.678+0000 7f41b8ff9640 1 -- 172.21.15.27:0/1917852155 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 90600+0+0 (secure 0 0 0) 0x7f41ac011070 con 0x7f41bc06d730 2024-09-12T03:20:41.680 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:41.678+0000 7f41b8ff9640 1 --2- 172.21.15.27:0/1917852155 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f41a806bf50 0x7f41a806e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:41.680 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:41.678+0000 7f41bb7fe640 1 --2- 172.21.15.27:0/1917852155 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f41a806bf50 0x7f41a806e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:41.680 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:41.678+0000 7f41b8ff9640 1 -- 172.21.15.27:0/1917852155 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 6543+0+0 (secure 0 0 0) 0x7f41ac08af20 con 0x7f41bc06d730 2024-09-12T03:20:41.681 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:41.679+0000 7f41bb7fe640 1 --2- 172.21.15.27:0/1917852155 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f41a806bf50 0x7f41a806e410 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f41b0034000 tx=0x7f41b002c000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:41.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:41.683+0000 7f41b8ff9640 1 -- 172.21.15.27:0/1917852155 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f41bc069560 con 0x7f41bc06d730 2024-09-12T03:20:41.885 INFO:teuthology.orchestra.run.smithi027.stderr:Invalid command: invalid chars #@ in abcd@# 2024-09-12T03:20:41.885 INFO:teuthology.orchestra.run.smithi027.stderr:fs subvolume create [] [] [] [] [] [] [--namespace-isolated] : Create a CephFS subvolume in a volume, and optionally, with a specific size (in bytes), a specific data pool layout, a specific mode, in a specific subvolume group and in separate RADOS namespace 2024-09-12T03:20:41.885 INFO:teuthology.orchestra.run.smithi027.stderr:Error EINVAL: invalid command 2024-09-12T03:20:41.888 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-12T03:20:41.889 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume rm testvol --yes-i-really-mean-it 2024-09-12T03:20:41.994 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:41.994+0000 7f56e3842640 1 Processor -- start 2024-09-12T03:20:41.995 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:41.994+0000 7f56e3842640 1 -- start start 2024-09-12T03:20:41.996 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:41.995+0000 7f56e3842640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f56dc06c650 0x7f56dc0b4000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:41.996 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:41.995+0000 7f56e3842640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f56dc06cff0 con 0x7f56dc0cef40 2024-09-12T03:20:41.996 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:41.995+0000 7f56e3842640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f56dc06d2d0 con 0x7f56dc0cf320 2024-09-12T03:20:41.997 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:41.995+0000 7f56e3842640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f56dc06dee0 con 0x7f56dc06c650 2024-09-12T03:20:41.997 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:41.995+0000 7f56e15de640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f56dc0cef40 0x7f56dc0710c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:47206/0 (socket says 172.21.15.27:47206) 2024-09-12T03:20:41.997 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:41.995+0000 7f56e15de640 1 -- 172.21.15.27:0/3850935119 learned_addr learned my addr 172.21.15.27:0/3850935119 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:20:41.997 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:41.995+0000 7f56e0ddd640 1 --2- 172.21.15.27:0/3850935119 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f56dc06c650 0x7f56dc0b4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:41.997 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:41.995+0000 7f56d3fff640 1 -- 172.21.15.27:0/3850935119 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 680907140 0 0) 0x7f56dc06cff0 con 0x7f56dc0cef40 2024-09-12T03:20:41.997 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:41.995+0000 7f56d3fff640 1 -- 172.21.15.27:0/3850935119 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f56c8003440 con 0x7f56dc0cef40 2024-09-12T03:20:41.997 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:41.995+0000 7f56d3fff640 1 -- 172.21.15.27:0/3850935119 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3802074981 0 0) 0x7f56c8003440 con 0x7f56dc0cef40 2024-09-12T03:20:41.997 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:41.995+0000 7f56e0ddd640 1 -- 172.21.15.27:0/3850935119 >> v1:172.21.15.45:6789/0 conn(0x7f56dc0cf320 legacy=0x7f56dc15f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:41.997 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:41.995+0000 7f56e0ddd640 1 -- 172.21.15.27:0/3850935119 >> v1:172.21.15.27:6789/0 conn(0x7f56dc0cef40 legacy=0x7f56dc0710c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:41.997 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:41.995+0000 7f56e0ddd640 1 -- 172.21.15.27:0/3850935119 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56dc06d5b0 con 0x7f56dc06c650 2024-09-12T03:20:41.999 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:41.996+0000 7f56e0ddd640 1 --2- 172.21.15.27:0/3850935119 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f56dc06c650 0x7f56dc0b4000 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f56cc0097d0 tx=0x7f56cc02a030 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eb5baa9070cb02d3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:41.999 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:41.996+0000 7f56d3fff640 1 -- 172.21.15.27:0/3850935119 <== mon.2 v2:172.21.15.45:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56cc037070 con 0x7f56dc06c650 2024-09-12T03:20:41.999 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:41.996+0000 7f56d3fff640 1 -- 172.21.15.27:0/3850935119 <== mon.2 v2:172.21.15.45:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56cc02aac0 con 0x7f56dc06c650 2024-09-12T03:20:41.999 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:41.996+0000 7f56d3fff640 1 -- 172.21.15.27:0/3850935119 <== mon.2 v2:172.21.15.45:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56d8001670 con 0x7f56dc06c650 2024-09-12T03:20:41.999 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:41.997+0000 7f56e3842640 1 -- 172.21.15.27:0/3850935119 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f56dc06c650 msgr2=0x7f56dc0b4000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:41.999 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:41.997+0000 7f56e3842640 1 --2- 172.21.15.27:0/3850935119 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f56dc06c650 0x7f56dc0b4000 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f56cc0097d0 tx=0x7f56cc02a030 comp rx=0 tx=0).stop 2024-09-12T03:20:42.000 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:41.997+0000 7f56e3842640 1 -- 172.21.15.27:0/3850935119 shutdown_connections 2024-09-12T03:20:42.000 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:41.997+0000 7f56e3842640 1 --2- 172.21.15.27:0/3850935119 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f56dc06c650 0x7f56dc0b4000 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:42.000 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:41.997+0000 7f56e3842640 1 -- 172.21.15.27:0/3850935119 >> 172.21.15.27:0/3850935119 conn(0x7f56dc06bb30 msgr2=0x7f56dc06bf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:42.000 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:41.997+0000 7f56e3842640 1 -- 172.21.15.27:0/3850935119 shutdown_connections 2024-09-12T03:20:42.000 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:41.997+0000 7f56e3842640 1 -- 172.21.15.27:0/3850935119 wait complete. 2024-09-12T03:20:42.000 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:41.998+0000 7f56e3842640 1 Processor -- start 2024-09-12T03:20:42.000 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:41.998+0000 7f56e3842640 1 -- start start 2024-09-12T03:20:42.003 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:41.999+0000 7f56e3842640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f56dc06c650 0x7f56dc1f80e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:42.004 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:41.999+0000 7f56e3842640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f56dc0cef40 0x7f56dc151da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:42.004 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:41.999+0000 7f56e3842640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f56dc0cf320 0x7f56dc0725d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:42.004 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:41.999+0000 7f56e3842640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f56dc0ce440 con 0x7f56dc0cef40 2024-09-12T03:20:42.004 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:41.999+0000 7f56e3842640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f56dc06db70 con 0x7f56dc0cf320 2024-09-12T03:20:42.004 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:41.999+0000 7f56e3842640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f56dc0ce5c0 con 0x7f56dc06c650 2024-09-12T03:20:42.004 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:41.999+0000 7f56e0ddd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f56dc0cef40 0x7f56dc151da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:42.004 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:41.999+0000 7f56e0ddd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f56dc0cef40 0x7f56dc151da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42530/0 (socket says 172.21.15.27:42530) 2024-09-12T03:20:42.004 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:41.999+0000 7f56e0ddd640 1 -- 172.21.15.27:0/2265437135 learned_addr learned my addr 172.21.15.27:0/2265437135 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:20:42.004 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:41.999+0000 7f56e15de640 1 --2- 172.21.15.27:0/2265437135 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f56dc06c650 0x7f56dc1f80e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:42.004 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:41.999+0000 7f56e1ddf640 1 --2- 172.21.15.27:0/2265437135 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f56dc0cf320 0x7f56dc0725d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:42.004 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:41.999+0000 7f56e0ddd640 1 -- 172.21.15.27:0/2265437135 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f56dc06c650 msgr2=0x7f56dc1f80e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:42.004 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:41.999+0000 7f56e0ddd640 1 --2- 172.21.15.27:0/2265437135 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f56dc06c650 0x7f56dc1f80e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:42.005 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:41.999+0000 7f56e0ddd640 1 -- 172.21.15.27:0/2265437135 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f56dc0cf320 msgr2=0x7f56dc0725d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:42.005 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:41.999+0000 7f56e0ddd640 1 --2- 172.21.15.27:0/2265437135 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f56dc0cf320 0x7f56dc0725d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:42.005 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:41.999+0000 7f56e0ddd640 1 -- 172.21.15.27:0/2265437135 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56dc1f8910 con 0x7f56dc0cef40 2024-09-12T03:20:42.005 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:42.000+0000 7f56e0ddd640 1 --2- 172.21.15.27:0/2265437135 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f56dc0cef40 0x7f56dc151da0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f56cc035040 tx=0x7f56cc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:42.005 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:42.000+0000 7f56d1ffb640 1 -- 172.21.15.27:0/2265437135 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56cc042020 con 0x7f56dc0cef40 2024-09-12T03:20:42.005 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:42.000+0000 7f56d1ffb640 1 -- 172.21.15.27:0/2265437135 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56cc003a40 con 0x7f56dc0cef40 2024-09-12T03:20:42.005 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:42.000+0000 7f56e3842640 1 -- 172.21.15.27:0/2265437135 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56dc1f8bf0 con 0x7f56dc0cef40 2024-09-12T03:20:42.005 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:42.000+0000 7f56d1ffb640 1 -- 172.21.15.27:0/2265437135 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56cc037040 con 0x7f56dc0cef40 2024-09-12T03:20:42.005 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:42.000+0000 7f56e3842640 1 -- 172.21.15.27:0/2265437135 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56dc1f8620 con 0x7f56dc0cef40 2024-09-12T03:20:42.005 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:42.001+0000 7f56e3842640 1 -- 172.21.15.27:0/2265437135 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56dc06cff0 con 0x7f56dc0cef40 2024-09-12T03:20:42.006 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:42.001+0000 7f56d1ffb640 1 -- 172.21.15.27:0/2265437135 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 90600+0+0 (secure 0 0 0) 0x7f56cc045020 con 0x7f56dc0cef40 2024-09-12T03:20:42.006 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:42.002+0000 7f56d1ffb640 1 --2- 172.21.15.27:0/2265437135 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f56c006bf50 0x7f56c006e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:42.006 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:42.002+0000 7f56d1ffb640 1 -- 172.21.15.27:0/2265437135 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 6543+0+0 (secure 0 0 0) 0x7f56cc0ab8d0 con 0x7f56dc0cef40 2024-09-12T03:20:42.006 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:42.002+0000 7f56e15de640 1 --2- 172.21.15.27:0/2265437135 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f56c006bf50 0x7f56c006e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:42.006 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:42.002+0000 7f56e15de640 1 --2- 172.21.15.27:0/2265437135 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f56c006bf50 0x7f56c006e410 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f56d8005900 tx=0x7f56d8001bc0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:42.007 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:42.006+0000 7f56d1ffb640 1 -- 172.21.15.27:0/2265437135 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f56cc04c030 con 0x7f56dc0cef40 2024-09-12T03:20:42.206 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:42.205+0000 7f56e3842640 1 -- 172.21.15.27:0/2265437135 --> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] -- mgr_command(tid 0: {"prefix": "fs volume rm", "vol_name": "testvol", "yes-i-really-mean-it": "--yes-i-really-mean-it", "target": ["mon-mgr", ""]}) -- 0x7f56dc06f240 con 0x7f56c006bf50 2024-09-12T03:20:42.229 INFO:tasks.ceph.mon.a.smithi027.stderr:2024-09-12T03:20:42.228+0000 7f95997d4640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-12T03:20:42.260 INFO:tasks.ceph.mds.c.smithi027.stdout:starting mds.c at 2024-09-12T03:20:45.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:45.246+0000 7f56d1ffb640 1 -- 172.21.15.27:0/2265437135 <== mgr.4113 v2:172.21.15.27:6836/1226920526 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+78 (secure 0 0 0) 0x7f56dc06f240 con 0x7f56c006bf50 2024-09-12T03:20:45.247 INFO:teuthology.orchestra.run.smithi027.stdout:metadata pool: cephfs.testvol.meta data pool: ['cephfs.testvol.data'] removed 2024-09-12T03:20:45.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:45.249+0000 7f56e3842640 1 -- 172.21.15.27:0/2265437135 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f56c006bf50 msgr2=0x7f56c006e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:45.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:45.249+0000 7f56e3842640 1 --2- 172.21.15.27:0/2265437135 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f56c006bf50 0x7f56c006e410 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f56d8005900 tx=0x7f56d8001bc0 comp rx=0 tx=0).stop 2024-09-12T03:20:45.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:45.249+0000 7f56e3842640 1 -- 172.21.15.27:0/2265437135 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f56dc0cef40 msgr2=0x7f56dc151da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:45.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:45.249+0000 7f56e3842640 1 --2- 172.21.15.27:0/2265437135 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f56dc0cef40 0x7f56dc151da0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f56cc035040 tx=0x7f56cc0046c0 comp rx=0 tx=0).stop 2024-09-12T03:20:45.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:45.249+0000 7f56e3842640 1 -- 172.21.15.27:0/2265437135 shutdown_connections 2024-09-12T03:20:45.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:45.249+0000 7f56e3842640 1 --2- 172.21.15.27:0/2265437135 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f56dc0cf320 0x7f56dc0725d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:45.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:45.249+0000 7f56e3842640 1 --2- 172.21.15.27:0/2265437135 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f56c006bf50 0x7f56c006e410 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:45.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:45.249+0000 7f56e3842640 1 --2- 172.21.15.27:0/2265437135 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f56dc0cef40 0x7f56dc151da0 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:45.251 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:45.249+0000 7f56e3842640 1 --2- 172.21.15.27:0/2265437135 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f56dc06c650 0x7f56dc1f80e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:45.251 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:45.249+0000 7f56e3842640 1 -- 172.21.15.27:0/2265437135 >> 172.21.15.27:0/2265437135 conn(0x7f56dc06bb30 msgr2=0x7f56dc0b1910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:45.251 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:45.249+0000 7f56e3842640 1 -- 172.21.15.27:0/2265437135 shutdown_connections 2024-09-12T03:20:45.251 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:45.249+0000 7f56e3842640 1 -- 172.21.15.27:0/2265437135 wait complete. 2024-09-12T03:20:45.263 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-12T03:20:45.263 DEBUG:teuthology.orchestra.run.smithi027:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-12T03:20:45.315 DEBUG:teuthology.orchestra.run.smithi045:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-12T03:20:45.370 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-12T03:20:45.370 INFO:tasks.cephfs.mount:Terminating background process 2024-09-12T03:20:45.370 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-12T03:20:45.370 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T03:20:45.387 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-12T03:20:45.387 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T03:20:45.444 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-12T03:20:45.589 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:45.588+0000 7fc86e665640 1 Processor -- start 2024-09-12T03:20:45.589 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:45.588+0000 7fc86e665640 1 -- start start 2024-09-12T03:20:45.590 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:45.589+0000 7fc86e665640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fc86815b2b0 0x7fc8680b3560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:45.590 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:45.589+0000 7fc86e665640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc86806e9b0 0x7fc86806ee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:45.591 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:45.589+0000 7fc86e665640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc8680ce410 con 0x7fc868072d70 2024-09-12T03:20:45.591 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:45.589+0000 7fc86e665640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7fc8680720a0 con 0x7fc86815b2b0 2024-09-12T03:20:45.591 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:45.589+0000 7fc86e665640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fc86806f340 con 0x7fc86806e9b0 2024-09-12T03:20:45.591 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:45.589+0000 7fc867fff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fc86815b2b0 0x7fc8680b3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:45.591 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:45.589+0000 7fc867fff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fc86815b2b0 0x7fc8680b3560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42550/0 (socket says 172.21.15.27:42550) 2024-09-12T03:20:45.591 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:45.589+0000 7fc867fff640 1 -- 172.21.15.27:0/3170597327 learned_addr learned my addr 172.21.15.27:0/3170597327 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:20:45.591 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:45.589+0000 7fc86cc02640 1 --2- 172.21.15.27:0/3170597327 >> v2:172.21.15.45:3300/0 conn(0x7fc86806e9b0 0x7fc86806ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:45.591 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:45.589+0000 7fc867fff640 1 -- 172.21.15.27:0/3170597327 >> v2:172.21.15.45:3300/0 conn(0x7fc86806e9b0 msgr2=0x7fc86806ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:45.591 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:45.589+0000 7fc867fff640 1 --2- 172.21.15.27:0/3170597327 >> v2:172.21.15.45:3300/0 conn(0x7fc86806e9b0 0x7fc86806ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:45.591 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:45.589+0000 7fc867fff640 1 -- 172.21.15.27:0/3170597327 >> v1:172.21.15.45:6789/0 conn(0x7fc868072d70 legacy=0x7fc868072430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:45.591 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:45.589+0000 7fc867fff640 1 -- 172.21.15.27:0/3170597327 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8680732c0 con 0x7fc86815b2b0 2024-09-12T03:20:45.591 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:45.589+0000 7fc86cc02640 1 --2- 172.21.15.27:0/3170597327 >> v2:172.21.15.45:3300/0 conn(0x7fc86806e9b0 0x7fc86806ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:20:45.593 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:45.590+0000 7fc867fff640 1 --2- 172.21.15.27:0/3170597327 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc86815b2b0 0x7fc8680b3560 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fc8680ce1e0 tx=0x7fc85802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2368c99ddb7687a5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:45.593 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:45.590+0000 7fc866ffd640 1 -- 172.21.15.27:0/3170597327 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc858037070 con 0x7fc86815b2b0 2024-09-12T03:20:45.593 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:45.590+0000 7fc866ffd640 1 -- 172.21.15.27:0/3170597327 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc858032440 con 0x7fc86815b2b0 2024-09-12T03:20:45.593 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:45.590+0000 7fc866ffd640 1 -- 172.21.15.27:0/3170597327 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc858032810 con 0x7fc86815b2b0 2024-09-12T03:20:45.593 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:45.591+0000 7fc86e665640 1 -- 172.21.15.27:0/3170597327 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc86815b2b0 msgr2=0x7fc8680b3560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:45.593 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:45.591+0000 7fc86e665640 1 --2- 172.21.15.27:0/3170597327 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc86815b2b0 0x7fc8680b3560 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fc8680ce1e0 tx=0x7fc85802a230 comp rx=0 tx=0).stop 2024-09-12T03:20:45.593 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:45.591+0000 7fc86e665640 1 -- 172.21.15.27:0/3170597327 shutdown_connections 2024-09-12T03:20:45.593 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:45.591+0000 7fc86e665640 1 --2- 172.21.15.27:0/3170597327 >> v2:172.21.15.45:3300/0 conn(0x7fc86806e9b0 0x7fc86806ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:45.593 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:45.591+0000 7fc86e665640 1 --2- 172.21.15.27:0/3170597327 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc86815b2b0 0x7fc8680b3560 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:45.593 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:45.591+0000 7fc86e665640 1 -- 172.21.15.27:0/3170597327 >> 172.21.15.27:0/3170597327 conn(0x7fc868062c60 msgr2=0x7fc86806bb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:45.593 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:45.591+0000 7fc86e665640 1 -- 172.21.15.27:0/3170597327 shutdown_connections 2024-09-12T03:20:45.594 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:45.591+0000 7fc86e665640 1 -- 172.21.15.27:0/3170597327 wait complete. 2024-09-12T03:20:45.594 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:45.592+0000 7fc86e665640 1 Processor -- start 2024-09-12T03:20:45.596 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:45.592+0000 7fc86e665640 1 -- start start 2024-09-12T03:20:45.596 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:45.593+0000 7fc86e665640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc86806e9b0 0x7fc868149110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:45.596 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:45.593+0000 7fc86e665640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc868072d70 0x7fc868147970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:45.596 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:45.593+0000 7fc86e665640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fc86815b2b0 0x7fc8680ae8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:45.596 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:45.593+0000 7fc86e665640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fc8680ce690 con 0x7fc868072d70 2024-09-12T03:20:45.596 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:45.593+0000 7fc86e665640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc868071dc0 con 0x7fc86806e9b0 2024-09-12T03:20:45.596 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:45.593+0000 7fc86e665640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fc86815b690 con 0x7fc86815b2b0 2024-09-12T03:20:45.596 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:45.593+0000 7fc8677fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc868072d70 0x7fc868147970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:45.596 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:45.593+0000 7fc8677fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc868072d70 0x7fc868147970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42554/0 (socket says 172.21.15.27:42554) 2024-09-12T03:20:45.596 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:45.593+0000 7fc8677fe640 1 -- 172.21.15.27:0/659997923 learned_addr learned my addr 172.21.15.27:0/659997923 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:20:45.596 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:45.593+0000 7fc867fff640 1 --2- 172.21.15.27:0/659997923 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc86806e9b0 0x7fc868149110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:45.596 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:45.593+0000 7fc86cc02640 1 --2- 172.21.15.27:0/659997923 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fc86815b2b0 0x7fc8680ae8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:45.596 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:45.594+0000 7fc8677fe640 1 -- 172.21.15.27:0/659997923 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fc86815b2b0 msgr2=0x7fc8680ae8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:45.597 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:45.594+0000 7fc8677fe640 1 --2- 172.21.15.27:0/659997923 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fc86815b2b0 0x7fc8680ae8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:45.597 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:45.594+0000 7fc8677fe640 1 -- 172.21.15.27:0/659997923 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc86806e9b0 msgr2=0x7fc868149110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:45.597 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:45.594+0000 7fc8677fe640 1 --2- 172.21.15.27:0/659997923 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc86806e9b0 0x7fc868149110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:45.597 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:45.594+0000 7fc8677fe640 1 -- 172.21.15.27:0/659997923 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc868148070 con 0x7fc868072d70 2024-09-12T03:20:45.597 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:45.594+0000 7fc8677fe640 1 --2- 172.21.15.27:0/659997923 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc868072d70 0x7fc868147970 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7fc85c00c430 tx=0x7fc85c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:45.597 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:45.594+0000 7fc864ff9640 1 -- 172.21.15.27:0/659997923 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc85c011790 con 0x7fc868072d70 2024-09-12T03:20:45.597 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:45.594+0000 7fc864ff9640 1 -- 172.21.15.27:0/659997923 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc85c017070 con 0x7fc868072d70 2024-09-12T03:20:45.597 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:45.594+0000 7fc86e665640 1 -- 172.21.15.27:0/659997923 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc868148350 con 0x7fc868072d70 2024-09-12T03:20:45.597 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:45.594+0000 7fc864ff9640 1 -- 172.21.15.27:0/659997923 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc85c011bb0 con 0x7fc868072d70 2024-09-12T03:20:45.597 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:45.594+0000 7fc86e665640 1 -- 172.21.15.27:0/659997923 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc868148910 con 0x7fc868072d70 2024-09-12T03:20:45.597 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:45.595+0000 7fc86e665640 1 -- 172.21.15.27:0/659997923 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8680b4c40 con 0x7fc868072d70 2024-09-12T03:20:45.598 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:45.596+0000 7fc864ff9640 1 -- 172.21.15.27:0/659997923 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 90600+0+0 (secure 0 0 0) 0x7fc85c01c460 con 0x7fc868072d70 2024-09-12T03:20:45.598 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:45.596+0000 7fc864ff9640 1 --2- 172.21.15.27:0/659997923 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fc85406bf50 0x7fc85406e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:45.598 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:45.596+0000 7fc864ff9640 1 -- 172.21.15.27:0/659997923 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5795+0+0 (secure 0 0 0) 0x7fc85c08ad40 con 0x7fc868072d70 2024-09-12T03:20:45.598 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:45.596+0000 7fc867fff640 1 --2- 172.21.15.27:0/659997923 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fc85406bf50 0x7fc85406e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:45.598 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:45.596+0000 7fc867fff640 1 --2- 172.21.15.27:0/659997923 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fc85406bf50 0x7fc85406e410 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fc858034000 tx=0x7fc85802e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:45.601 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:45.600+0000 7fc864ff9640 1 -- 172.21.15.27:0/659997923 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fc85c058fc0 con 0x7fc868072d70 2024-09-12T03:20:45.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:45.785+0000 7fc86e665640 1 -- 172.21.15.27:0/659997923 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fc8680b4960 con 0x7fc868072d70 2024-09-12T03:20:45.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:45.786+0000 7fc864ff9640 1 -- 172.21.15.27:0/659997923 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v89) ==== 80+0+122 (secure 0 0 0) 0x7fc85c05d2c0 con 0x7fc868072d70 2024-09-12T03:20:45.787 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:6834/1466511159 2024-09-13T03:20:42.229493+0000 2024-09-12T03:20:45.787 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:6832/1466511159 2024-09-13T03:20:42.229493+0000 2024-09-12T03:20:45.787 INFO:teuthology.orchestra.run.smithi027.stderr:listed 2 entries 2024-09-12T03:20:45.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:45.788+0000 7fc86e665640 1 -- 172.21.15.27:0/659997923 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fc85406bf50 msgr2=0x7fc85406e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:45.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:45.788+0000 7fc86e665640 1 --2- 172.21.15.27:0/659997923 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fc85406bf50 0x7fc85406e410 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fc858034000 tx=0x7fc85802e000 comp rx=0 tx=0).stop 2024-09-12T03:20:45.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:45.788+0000 7fc86e665640 1 -- 172.21.15.27:0/659997923 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc868072d70 msgr2=0x7fc868147970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:45.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:45.788+0000 7fc86e665640 1 --2- 172.21.15.27:0/659997923 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc868072d70 0x7fc868147970 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7fc85c00c430 tx=0x7fc85c00c8f0 comp rx=0 tx=0).stop 2024-09-12T03:20:45.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:45.788+0000 7fc86e665640 1 -- 172.21.15.27:0/659997923 shutdown_connections 2024-09-12T03:20:45.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:45.788+0000 7fc86e665640 1 --2- 172.21.15.27:0/659997923 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fc86815b2b0 0x7fc8680ae8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:45.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:45.788+0000 7fc86e665640 1 --2- 172.21.15.27:0/659997923 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fc85406bf50 0x7fc85406e410 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:45.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:45.788+0000 7fc86e665640 1 --2- 172.21.15.27:0/659997923 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc868072d70 0x7fc868147970 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:45.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:45.788+0000 7fc86e665640 1 --2- 172.21.15.27:0/659997923 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc86806e9b0 0x7fc868149110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:45.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:45.788+0000 7fc86e665640 1 -- 172.21.15.27:0/659997923 >> 172.21.15.27:0/659997923 conn(0x7fc868062c60 msgr2=0x7fc8680b5440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:45.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:45.788+0000 7fc86e665640 1 -- 172.21.15.27:0/659997923 shutdown_connections 2024-09-12T03:20:45.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:45.788+0000 7fc86e665640 1 -- 172.21.15.27:0/659997923 wait complete. 2024-09-12T03:20:45.799 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-12T03:20:45.800 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:20:45.800 DEBUG:teuthology.orchestra.run.smithi027:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-12T03:20:45.829 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi027.front.sepia.ceph.com 2024-09-12T03:20:45.830 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:20:45.830 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-12T03:20:45.886 INFO:tasks.cephfs.mount:Terminating background process 2024-09-12T03:20:45.886 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-12T03:20:45.886 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T03:20:45.906 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-12T03:20:45.907 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T03:20:45.966 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-12T03:20:46.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.068+0000 7f82cd1c0640 1 Processor -- start 2024-09-12T03:20:46.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.068+0000 7f82cd1c0640 1 -- start start 2024-09-12T03:20:46.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.069+0000 7f82cd1c0640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f82c8071be0 0x7f82c806e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:46.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.069+0000 7f82cd1c0640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f82c815aca0 0x7f82c806edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:46.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.069+0000 7f82cd1c0640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f82c806daf0 con 0x7f82c815a8c0 2024-09-12T03:20:46.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.069+0000 7f82cd1c0640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f82c806d8a0 con 0x7f82c8071be0 2024-09-12T03:20:46.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.069+0000 7f82cd1c0640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f82c806ddd0 con 0x7f82c815aca0 2024-09-12T03:20:46.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.069+0000 7f82c759e640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f82c815aca0 0x7f82c806edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:46.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.069+0000 7f82c659c640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f82c8071be0 0x7f82c806e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:46.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.069+0000 7f82c759e640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f82c815aca0 0x7f82c806edf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3301/0 says I am v2:172.21.15.27:42916/0 (socket says 172.21.15.27:42916) 2024-09-12T03:20:46.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.069+0000 7f82c759e640 1 -- 172.21.15.27:0/2651217257 learned_addr learned my addr 172.21.15.27:0/2651217257 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:20:46.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.069+0000 7f82c6d9d640 1 --1- 172.21.15.27:0/2651217257 >> v1:172.21.15.45:6789/0 conn(0x7f82c815a8c0 0x7f82c80cdb30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.27:38208/0 (socket says 172.21.15.27:38208) 2024-09-12T03:20:46.072 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.070+0000 7f82c659c640 1 -- 172.21.15.27:0/2651217257 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f82c815aca0 msgr2=0x7f82c806edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:46.072 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.070+0000 7f82c659c640 1 --2- 172.21.15.27:0/2651217257 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f82c815aca0 0x7f82c806edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:46.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.070+0000 7f82c659c640 1 -- 172.21.15.27:0/2651217257 >> v1:172.21.15.45:6789/0 conn(0x7f82c815a8c0 legacy=0x7f82c80cdb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:46.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.070+0000 7f82c5d9b640 1 -- 172.21.15.27:0/2651217257 <== mon.1 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 761837829 0 0) 0x7f82c806daf0 con 0x7f82c815a8c0 2024-09-12T03:20:46.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.070+0000 7f82c659c640 1 -- 172.21.15.27:0/2651217257 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82c806e0b0 con 0x7f82c8071be0 2024-09-12T03:20:46.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.070+0000 7f82c759e640 1 --2- 172.21.15.27:0/2651217257 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f82c815aca0 0x7f82c806edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:20:46.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.070+0000 7f82c659c640 1 --2- 172.21.15.27:0/2651217257 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f82c8071be0 0x7f82c806e8b0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f82bc0097a0 tx=0x7f82bc02a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=51f1f09efe12459d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:46.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.071+0000 7f82c5d9b640 1 -- 172.21.15.27:0/2651217257 <== mon.1 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82bc037070 con 0x7f82c8071be0 2024-09-12T03:20:46.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.071+0000 7f82c5d9b640 1 -- 172.21.15.27:0/2651217257 <== mon.1 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82bc032440 con 0x7f82c8071be0 2024-09-12T03:20:46.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.071+0000 7f82c5d9b640 1 -- 172.21.15.27:0/2651217257 <== mon.1 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82bc032810 con 0x7f82c8071be0 2024-09-12T03:20:46.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.071+0000 7f82cd1c0640 1 -- 172.21.15.27:0/2651217257 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f82c8071be0 msgr2=0x7f82c806e8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:46.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.071+0000 7f82cd1c0640 1 --2- 172.21.15.27:0/2651217257 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f82c8071be0 0x7f82c806e8b0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f82bc0097a0 tx=0x7f82bc02a180 comp rx=0 tx=0).stop 2024-09-12T03:20:46.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.071+0000 7f82cd1c0640 1 -- 172.21.15.27:0/2651217257 shutdown_connections 2024-09-12T03:20:46.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.071+0000 7f82cd1c0640 1 --2- 172.21.15.27:0/2651217257 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f82c815aca0 0x7f82c806edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:46.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.071+0000 7f82cd1c0640 1 --2- 172.21.15.27:0/2651217257 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f82c8071be0 0x7f82c806e8b0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:46.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.071+0000 7f82cd1c0640 1 -- 172.21.15.27:0/2651217257 >> 172.21.15.27:0/2651217257 conn(0x7f82c8062c60 msgr2=0x7f82c806bb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:46.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.071+0000 7f82cd1c0640 1 -- 172.21.15.27:0/2651217257 shutdown_connections 2024-09-12T03:20:46.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.071+0000 7f82cd1c0640 1 -- 172.21.15.27:0/2651217257 wait complete. 2024-09-12T03:20:46.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.073+0000 7f82cd1c0640 1 Processor -- start 2024-09-12T03:20:46.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.073+0000 7f82cd1c0640 1 -- start start 2024-09-12T03:20:46.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.073+0000 7f82cd1c0640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f82c8071be0 0x7f82c80b4980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:46.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.073+0000 7f82cd1c0640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f82c815a8c0 0x7f82c8152450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:46.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.073+0000 7f82cd1c0640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f82c815aca0 0x7f82c8152990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:46.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.073+0000 7f82cd1c0640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f82c80ce3c0 con 0x7f82c815aca0 2024-09-12T03:20:46.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.073+0000 7f82cd1c0640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f82c80ce240 con 0x7f82c815a8c0 2024-09-12T03:20:46.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.073+0000 7f82cd1c0640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f82c80b3560 con 0x7f82c8071be0 2024-09-12T03:20:46.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.074+0000 7f82c759e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f82c815aca0 0x7f82c8152990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:46.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.074+0000 7f82c659c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f82c815a8c0 0x7f82c8152450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:46.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.074+0000 7f82c759e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f82c815aca0 0x7f82c8152990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42564/0 (socket says 172.21.15.27:42564) 2024-09-12T03:20:46.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.074+0000 7f82c6d9d640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f82c8071be0 0x7f82c80b4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:46.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.074+0000 7f82c759e640 1 -- 172.21.15.27:0/752558543 learned_addr learned my addr 172.21.15.27:0/752558543 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:20:46.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.074+0000 7f82c759e640 1 -- 172.21.15.27:0/752558543 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f82c8071be0 msgr2=0x7f82c80b4980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:46.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.074+0000 7f82c759e640 1 --2- 172.21.15.27:0/752558543 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f82c8071be0 0x7f82c80b4980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:46.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.074+0000 7f82c759e640 1 -- 172.21.15.27:0/752558543 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f82c815a8c0 msgr2=0x7f82c8152450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:46.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.074+0000 7f82c759e640 1 --2- 172.21.15.27:0/752558543 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f82c815a8c0 0x7f82c8152450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:46.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.074+0000 7f82c759e640 1 -- 172.21.15.27:0/752558543 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82c814f8a0 con 0x7f82c815aca0 2024-09-12T03:20:46.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.074+0000 7f82c659c640 1 --2- 172.21.15.27:0/752558543 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f82c815a8c0 0x7f82c8152450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:20:46.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.074+0000 7f82c759e640 1 --2- 172.21.15.27:0/752558543 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f82c815aca0 0x7f82c8152990 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f82c00072e0 tx=0x7f82c000e550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:46.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.074+0000 7f82af7fe640 1 -- 172.21.15.27:0/752558543 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82c000b310 con 0x7f82c815aca0 2024-09-12T03:20:46.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.074+0000 7f82af7fe640 1 -- 172.21.15.27:0/752558543 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82c0003990 con 0x7f82c815aca0 2024-09-12T03:20:46.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.074+0000 7f82cd1c0640 1 -- 172.21.15.27:0/752558543 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82c81f3e70 con 0x7f82c815aca0 2024-09-12T03:20:46.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.074+0000 7f82af7fe640 1 -- 172.21.15.27:0/752558543 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82c0003d60 con 0x7f82c815aca0 2024-09-12T03:20:46.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.074+0000 7f82cd1c0640 1 -- 172.21.15.27:0/752558543 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82c8140930 con 0x7f82c815aca0 2024-09-12T03:20:46.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.075+0000 7f82cd1c0640 1 -- 172.21.15.27:0/752558543 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82c8072ba0 con 0x7f82c815aca0 2024-09-12T03:20:46.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.076+0000 7f82af7fe640 1 -- 172.21.15.27:0/752558543 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 90600+0+0 (secure 0 0 0) 0x7f82c0014050 con 0x7f82c815aca0 2024-09-12T03:20:46.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.076+0000 7f82af7fe640 1 --2- 172.21.15.27:0/752558543 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f82b406bf50 0x7f82b406e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:46.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.077+0000 7f82c6d9d640 1 --2- 172.21.15.27:0/752558543 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f82b406bf50 0x7f82b406e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:46.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.077+0000 7f82af7fe640 1 -- 172.21.15.27:0/752558543 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5795+0+0 (secure 0 0 0) 0x7f82c008d590 con 0x7f82c815aca0 2024-09-12T03:20:46.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.077+0000 7f82c6d9d640 1 --2- 172.21.15.27:0/752558543 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f82b406bf50 0x7f82b406e410 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f82b8000f60 tx=0x7f82b8007d50 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:46.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.081+0000 7f82af7fe640 1 -- 172.21.15.27:0/752558543 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f82c814f8a0 con 0x7f82c815aca0 2024-09-12T03:20:46.269 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.268+0000 7f82cd1c0640 1 -- 172.21.15.27:0/752558543 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f82c8072750 con 0x7f82c815aca0 2024-09-12T03:20:46.269 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.269+0000 7f82af7fe640 1 -- 172.21.15.27:0/752558543 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v89) ==== 80+0+122 (secure 0 0 0) 0x7f82c0012040 con 0x7f82c815aca0 2024-09-12T03:20:46.270 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:6834/1466511159 2024-09-13T03:20:42.229493+0000 2024-09-12T03:20:46.270 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:6832/1466511159 2024-09-13T03:20:42.229493+0000 2024-09-12T03:20:46.270 INFO:teuthology.orchestra.run.smithi027.stderr:listed 2 entries 2024-09-12T03:20:46.272 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.271+0000 7f82cd1c0640 1 -- 172.21.15.27:0/752558543 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f82b406bf50 msgr2=0x7f82b406e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:46.272 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.271+0000 7f82cd1c0640 1 --2- 172.21.15.27:0/752558543 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f82b406bf50 0x7f82b406e410 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f82b8000f60 tx=0x7f82b8007d50 comp rx=0 tx=0).stop 2024-09-12T03:20:46.272 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.271+0000 7f82cd1c0640 1 -- 172.21.15.27:0/752558543 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f82c815aca0 msgr2=0x7f82c8152990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:46.272 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.271+0000 7f82cd1c0640 1 --2- 172.21.15.27:0/752558543 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f82c815aca0 0x7f82c8152990 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f82c00072e0 tx=0x7f82c000e550 comp rx=0 tx=0).stop 2024-09-12T03:20:46.272 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.271+0000 7f82cd1c0640 1 -- 172.21.15.27:0/752558543 shutdown_connections 2024-09-12T03:20:46.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.271+0000 7f82cd1c0640 1 --2- 172.21.15.27:0/752558543 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f82b406bf50 0x7f82b406e410 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:46.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.271+0000 7f82cd1c0640 1 --2- 172.21.15.27:0/752558543 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f82c815aca0 0x7f82c8152990 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:46.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.271+0000 7f82cd1c0640 1 --2- 172.21.15.27:0/752558543 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f82c815a8c0 0x7f82c8152450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:46.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.271+0000 7f82cd1c0640 1 --2- 172.21.15.27:0/752558543 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f82c8071be0 0x7f82c80b4980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:46.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.271+0000 7f82cd1c0640 1 -- 172.21.15.27:0/752558543 >> 172.21.15.27:0/752558543 conn(0x7f82c8062c60 msgr2=0x7f82c8159140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:46.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.271+0000 7f82cd1c0640 1 -- 172.21.15.27:0/752558543 shutdown_connections 2024-09-12T03:20:46.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.271+0000 7f82cd1c0640 1 -- 172.21.15.27:0/752558543 wait complete. 2024-09-12T03:20:46.283 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-09-12T03:20:46.283 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:20:46.283 DEBUG:teuthology.orchestra.run.smithi045:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-09-12T03:20:46.317 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi045.front.sepia.ceph.com 2024-09-12T03:20:46.317 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:20:46.317 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-12T03:20:46.376 INFO:tasks.cephfs.mount:Terminating background process 2024-09-12T03:20:46.377 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-12T03:20:46.377 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T03:20:46.393 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056004,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497432k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16324 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248716k,nr_inodes=812179,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497432k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-12T03:20:46.393 INFO:tasks.cephfs.mount:Terminating background process 2024-09-12T03:20:46.393 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-12T03:20:46.394 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T03:20:46.434 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056795,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16602 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249348k,nr_inodes=812337,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-12T03:20:46.435 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-12T03:20:46.538 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.537+0000 7f18e6ce4640 1 Processor -- start 2024-09-12T03:20:46.538 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.537+0000 7f18e6ce4640 1 -- start start 2024-09-12T03:20:46.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.538+0000 7f18e6ce4640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f18e0146020 0x7f18e00d1720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:46.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.538+0000 7f18e6ce4640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f18e00d2100 con 0x7f18e00d1c60 2024-09-12T03:20:46.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.538+0000 7f18e6ce4640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f18e00b6fb0 con 0x7f18e0073960 2024-09-12T03:20:46.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.538+0000 7f18e6ce4640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f18e0146400 con 0x7f18e0146020 2024-09-12T03:20:46.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.538+0000 7f18dffff640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f18e00d1c60 0x7f18e0073250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:47224/0 (socket says 172.21.15.27:47224) 2024-09-12T03:20:46.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.538+0000 7f18dffff640 1 -- 172.21.15.27:0/3257444477 learned_addr learned my addr 172.21.15.27:0/3257444477 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:20:46.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.538+0000 7f18e4a80640 1 --2- 172.21.15.27:0/3257444477 >> v2:172.21.15.45:3300/0 conn(0x7f18e0146020 0x7f18e00d1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:46.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.538+0000 7f18df7fe640 1 -- 172.21.15.27:0/3257444477 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3486578927 0 0) 0x7f18e00d2100 con 0x7f18e00d1c60 2024-09-12T03:20:46.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.538+0000 7f18df7fe640 1 -- 172.21.15.27:0/3257444477 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f18d0003520 con 0x7f18e00d1c60 2024-09-12T03:20:46.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.539+0000 7f18df7fe640 1 -- 172.21.15.27:0/3257444477 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3503198659 0 0) 0x7f18d0003520 con 0x7f18e00d1c60 2024-09-12T03:20:46.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.539+0000 7f18df7fe640 1 -- 172.21.15.27:0/3257444477 >> v2:172.21.15.45:3300/0 conn(0x7f18e0146020 msgr2=0x7f18e00d1720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:46.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.539+0000 7f18df7fe640 1 --2- 172.21.15.27:0/3257444477 >> v2:172.21.15.45:3300/0 conn(0x7f18e0146020 0x7f18e00d1720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:46.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.539+0000 7f18df7fe640 1 -- 172.21.15.27:0/3257444477 >> v1:172.21.15.45:6789/0 conn(0x7f18e0073960 legacy=0x7f18e015cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:46.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.539+0000 7f18df7fe640 1 -- 172.21.15.27:0/3257444477 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18e00b7290 con 0x7f18e00d1c60 2024-09-12T03:20:46.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.539+0000 7f18df7fe640 1 -- 172.21.15.27:0/3257444477 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 847755213 0 0) 0x7f18cc001670 con 0x7f18e00d1c60 2024-09-12T03:20:46.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.539+0000 7f18df7fe640 1 -- 172.21.15.27:0/3257444477 >> v1:172.21.15.27:6789/0 conn(0x7f18e00d1c60 legacy=0x7f18e0073250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:46.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.539+0000 7f18df7fe640 1 --2- 172.21.15.27:0/3257444477 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f18d00039b0 0x7f18d0003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:46.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.539+0000 7f18df7fe640 1 --2- 172.21.15.27:0/3257444477 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f18d0004800 0x7f18d0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:46.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.539+0000 7f18df7fe640 1 --2- 172.21.15.27:0/3257444477 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f18d00051c0 0x7f18d000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:46.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.539+0000 7f18df7fe640 1 -- 172.21.15.27:0/3257444477 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f18e0146400 con 0x7f18d00039b0 2024-09-12T03:20:46.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.539+0000 7f18df7fe640 1 -- 172.21.15.27:0/3257444477 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f18d000c090 con 0x7f18d0004800 2024-09-12T03:20:46.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.539+0000 7f18df7fe640 1 -- 172.21.15.27:0/3257444477 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f18d000c290 con 0x7f18d00051c0 2024-09-12T03:20:46.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.539+0000 7f18e4a80640 1 --2- 172.21.15.27:0/3257444477 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f18d00039b0 0x7f18d0003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:46.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.539+0000 7f18dffff640 1 --2- 172.21.15.27:0/3257444477 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f18d0004800 0x7f18d0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:46.543 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.539+0000 7f18e5281640 1 --2- 172.21.15.27:0/3257444477 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f18d00051c0 0x7f18d000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:46.543 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.539+0000 7f18e4a80640 1 -- 172.21.15.27:0/3257444477 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f18d00051c0 msgr2=0x7f18d000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:46.543 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.539+0000 7f18e4a80640 1 --2- 172.21.15.27:0/3257444477 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f18d00051c0 0x7f18d000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:46.543 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.539+0000 7f18e4a80640 1 -- 172.21.15.27:0/3257444477 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f18d0004800 msgr2=0x7f18d0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:46.543 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.539+0000 7f18e4a80640 1 --2- 172.21.15.27:0/3257444477 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f18d0004800 0x7f18d0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:46.543 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.539+0000 7f18e4a80640 1 -- 172.21.15.27:0/3257444477 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18d0003520 con 0x7f18d00039b0 2024-09-12T03:20:46.543 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.539+0000 7f18e5281640 1 -- 172.21.15.27:0/3257444477 reap_dead start 2024-09-12T03:20:46.543 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.540+0000 7f18e4a80640 1 --2- 172.21.15.27:0/3257444477 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f18d00039b0 0x7f18d0003df0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f18d4009ac0 tx=0x7f18d0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c6ea59c56bc5a26 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:46.543 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.540+0000 7f18df7fe640 1 -- 172.21.15.27:0/3257444477 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18d4036070 con 0x7f18d00039b0 2024-09-12T03:20:46.543 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.540+0000 7f18df7fe640 1 -- 172.21.15.27:0/3257444477 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f18d4031400 con 0x7f18d00039b0 2024-09-12T03:20:46.543 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.540+0000 7f18df7fe640 1 -- 172.21.15.27:0/3257444477 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18d4009690 con 0x7f18d00039b0 2024-09-12T03:20:46.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.540+0000 7f18e6ce4640 1 -- 172.21.15.27:0/3257444477 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f18d00039b0 msgr2=0x7f18d0003df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:46.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.540+0000 7f18e6ce4640 1 --2- 172.21.15.27:0/3257444477 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f18d00039b0 0x7f18d0003df0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f18d4009ac0 tx=0x7f18d0005830 comp rx=0 tx=0).stop 2024-09-12T03:20:46.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.541+0000 7f18e6ce4640 1 -- 172.21.15.27:0/3257444477 shutdown_connections 2024-09-12T03:20:46.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.541+0000 7f18e6ce4640 1 --2- 172.21.15.27:0/3257444477 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f18d00039b0 0x7f18d0003df0 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:46.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.541+0000 7f18e6ce4640 1 -- 172.21.15.27:0/3257444477 >> 172.21.15.27:0/3257444477 conn(0x7f18e006f300 msgr2=0x7f18e006f710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:46.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.541+0000 7f18e6ce4640 1 -- 172.21.15.27:0/3257444477 shutdown_connections 2024-09-12T03:20:46.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.541+0000 7f18e6ce4640 1 -- 172.21.15.27:0/3257444477 wait complete. 2024-09-12T03:20:46.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.542+0000 7f18e6ce4640 1 Processor -- start 2024-09-12T03:20:46.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.542+0000 7f18e6ce4640 1 -- start start 2024-09-12T03:20:46.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.543+0000 7f18e6ce4640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f18d00039b0 0x7f18e015bf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:46.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.543+0000 7f18e6ce4640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f18e013c430 0x7f18e013e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:46.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.543+0000 7f18e6ce4640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f18e013c8b0 0x7f18e013eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:46.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.543+0000 7f18e6ce4640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f18e0070b70 con 0x7f18e013c430 2024-09-12T03:20:46.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.543+0000 7f18e6ce4640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f18e00b6c60 con 0x7f18e013c8b0 2024-09-12T03:20:46.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.543+0000 7f18e6ce4640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f18e0070cf0 con 0x7f18d00039b0 2024-09-12T03:20:46.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.543+0000 7f18dffff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f18e013c430 0x7f18e013e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:46.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.543+0000 7f18dffff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f18e013c430 0x7f18e013e960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42582/0 (socket says 172.21.15.27:42582) 2024-09-12T03:20:46.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.543+0000 7f18dffff640 1 -- 172.21.15.27:0/2190354476 learned_addr learned my addr 172.21.15.27:0/2190354476 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:20:46.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.543+0000 7f18e4a80640 1 --2- 172.21.15.27:0/2190354476 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f18d00039b0 0x7f18e015bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:46.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.543+0000 7f18e5281640 1 --2- 172.21.15.27:0/2190354476 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f18e013c8b0 0x7f18e013eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:46.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.543+0000 7f18dffff640 1 -- 172.21.15.27:0/2190354476 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f18d00039b0 msgr2=0x7f18e015bf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:46.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.543+0000 7f18dffff640 1 --2- 172.21.15.27:0/2190354476 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f18d00039b0 0x7f18e015bf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:46.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.543+0000 7f18dffff640 1 -- 172.21.15.27:0/2190354476 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f18e013c8b0 msgr2=0x7f18e013eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:46.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.543+0000 7f18dffff640 1 --2- 172.21.15.27:0/2190354476 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f18e013c8b0 0x7f18e013eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:46.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.543+0000 7f18dffff640 1 -- 172.21.15.27:0/2190354476 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18e015c490 con 0x7f18e013c430 2024-09-12T03:20:46.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.543+0000 7f18dffff640 1 --2- 172.21.15.27:0/2190354476 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f18e013c430 0x7f18e013e960 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f18cc0072f0 tx=0x7f18cc0077b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:46.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.543+0000 7f18dd7fa640 1 -- 172.21.15.27:0/2190354476 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18cc01a070 con 0x7f18e013c430 2024-09-12T03:20:46.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.543+0000 7f18dd7fa640 1 -- 172.21.15.27:0/2190354476 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f18cc00fa80 con 0x7f18e013c430 2024-09-12T03:20:46.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.543+0000 7f18e6ce4640 1 -- 172.21.15.27:0/2190354476 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18e013ce60 con 0x7f18e013c430 2024-09-12T03:20:46.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.543+0000 7f18dd7fa640 1 -- 172.21.15.27:0/2190354476 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18cc00fe70 con 0x7f18e013c430 2024-09-12T03:20:46.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.544+0000 7f18e6ce4640 1 -- 172.21.15.27:0/2190354476 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18e013d420 con 0x7f18e013c430 2024-09-12T03:20:46.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.544+0000 7f18e6ce4640 1 -- 172.21.15.27:0/2190354476 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18e00b2540 con 0x7f18e013c430 2024-09-12T03:20:46.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.545+0000 7f18dd7fa640 1 -- 172.21.15.27:0/2190354476 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 90600+0+0 (secure 0 0 0) 0x7f18cc01d070 con 0x7f18e013c430 2024-09-12T03:20:46.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.545+0000 7f18dd7fa640 1 --2- 172.21.15.27:0/2190354476 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f18c406bf80 0x7f18c406e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:46.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.545+0000 7f18dd7fa640 1 -- 172.21.15.27:0/2190354476 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5795+0+0 (secure 0 0 0) 0x7f18cc08aa40 con 0x7f18e013c430 2024-09-12T03:20:46.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.546+0000 7f18e4a80640 1 --2- 172.21.15.27:0/2190354476 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f18c406bf80 0x7f18c406e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:46.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.546+0000 7f18e4a80640 1 --2- 172.21.15.27:0/2190354476 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f18c406bf80 0x7f18c406e440 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f18d4009ac0 tx=0x7f18d4008030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:46.551 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.550+0000 7f18dd7fa640 1 -- 172.21.15.27:0/2190354476 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f18cc058cc0 con 0x7f18e013c430 2024-09-12T03:20:46.764 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.763+0000 7f18e6ce4640 1 -- 172.21.15.27:0/2190354476 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f18e00b3390 con 0x7f18e013c430 2024-09-12T03:20:46.764 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.763+0000 7f18dd7fa640 1 -- 172.21.15.27:0/2190354476 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 51 v51) ==== 94+0+5219 (secure 0 0 0) 0x7f18cc05cfc0 con 0x7f18e013c430 2024-09-12T03:20:46.764 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:20:46.765 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":51,"btime":"2024-09-12T03:20:43:233119+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5669,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/2036680565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2036680565},{"type":"v1","addr":"172.21.15.45:6834","nonce":2036680565}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":43},{"gid":6094,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/874218126","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":874218126},{"type":"v1","addr":"172.21.15.45:6835","nonce":874218126}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":43},{"gid":6693,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6834/658418980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":658418980},{"type":"v1","addr":"172.21.15.27:6834","nonce":658418980}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":51}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-12T03:20:28.499550+0000","modified":"2024-09-12T03:20:30.510730+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6486},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6486":{"gid":6486,"name":"a","rank":0,"incarnation":44,"state":"up:active","state_seq":3,"addr":"172.21.15.27:6835/3209963418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":3209963418},{"type":"v1","addr":"172.21.15.27:6835","nonce":3209963418}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6486,"qdb_cluster":[6486]},"id":6}]} 2024-09-12T03:20:46.765 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 51 2024-09-12T03:20:46.767 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.765+0000 7f18e6ce4640 1 -- 172.21.15.27:0/2190354476 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f18c406bf80 msgr2=0x7f18c406e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:46.767 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.765+0000 7f18e6ce4640 1 --2- 172.21.15.27:0/2190354476 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f18c406bf80 0x7f18c406e440 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f18d4009ac0 tx=0x7f18d4008030 comp rx=0 tx=0).stop 2024-09-12T03:20:46.767 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.765+0000 7f18e6ce4640 1 -- 172.21.15.27:0/2190354476 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f18e013c430 msgr2=0x7f18e013e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:46.767 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.765+0000 7f18e6ce4640 1 --2- 172.21.15.27:0/2190354476 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f18e013c430 0x7f18e013e960 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f18cc0072f0 tx=0x7f18cc0077b0 comp rx=0 tx=0).stop 2024-09-12T03:20:46.767 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.766+0000 7f18e6ce4640 1 -- 172.21.15.27:0/2190354476 shutdown_connections 2024-09-12T03:20:46.767 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.766+0000 7f18e6ce4640 1 --2- 172.21.15.27:0/2190354476 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f18e013c8b0 0x7f18e013eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:46.767 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.766+0000 7f18e6ce4640 1 --2- 172.21.15.27:0/2190354476 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f18c406bf80 0x7f18c406e440 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:46.767 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.766+0000 7f18e6ce4640 1 --2- 172.21.15.27:0/2190354476 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f18e013c430 0x7f18e013e960 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:46.767 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.766+0000 7f18e6ce4640 1 --2- 172.21.15.27:0/2190354476 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f18d00039b0 0x7f18e015bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:46.767 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.766+0000 7f18e6ce4640 1 -- 172.21.15.27:0/2190354476 >> 172.21.15.27:0/2190354476 conn(0x7f18e006f300 msgr2=0x7f18e00b39d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:46.768 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.766+0000 7f18e6ce4640 1 -- 172.21.15.27:0/2190354476 shutdown_connections 2024-09-12T03:20:46.768 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.766+0000 7f18e6ce4640 1 -- 172.21.15.27:0/2190354476 wait complete. 2024-09-12T03:20:46.778 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-12T03:20:46.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.880+0000 7fe3cddbd640 1 Processor -- start 2024-09-12T03:20:46.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.880+0000 7fe3cddbd640 1 -- start start 2024-09-12T03:20:46.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.880+0000 7fe3cddbd640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fe3c80d5200 0x7fe3c80d1010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:46.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.880+0000 7fe3cddbd640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fe3c80d1550 0x7fe3c80b8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:46.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.880+0000 7fe3cddbd640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe3c80d3760 con 0x7fe3c815eb90 2024-09-12T03:20:46.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.880+0000 7fe3cddbd640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7fe3c80b7d20 con 0x7fe3c80d1550 2024-09-12T03:20:46.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.880+0000 7fe3cddbd640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fe3c80d42e0 con 0x7fe3c80d5200 2024-09-12T03:20:46.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.881+0000 7fe3c77fe640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fe3c815eb90 0x7fe3c80d2df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:47240/0 (socket says 172.21.15.27:47240) 2024-09-12T03:20:46.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.881+0000 7fe3c77fe640 1 -- 172.21.15.27:0/2817874743 learned_addr learned my addr 172.21.15.27:0/2817874743 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:20:46.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.881+0000 7fe3c7fff640 1 --2- 172.21.15.27:0/2817874743 >> v2:172.21.15.27:3300/0 conn(0x7fe3c80d1550 0x7fe3c80b8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:46.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.881+0000 7fe3c6ffd640 1 --2- 172.21.15.27:0/2817874743 >> v2:172.21.15.45:3300/0 conn(0x7fe3c80d5200 0x7fe3c80d1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:46.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.881+0000 7fe3c7fff640 1 -- 172.21.15.27:0/2817874743 >> v2:172.21.15.45:3300/0 conn(0x7fe3c80d5200 msgr2=0x7fe3c80d1010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:46.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.881+0000 7fe3c7fff640 1 --2- 172.21.15.27:0/2817874743 >> v2:172.21.15.45:3300/0 conn(0x7fe3c80d5200 0x7fe3c80d1010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:46.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.881+0000 7fe3c7fff640 1 -- 172.21.15.27:0/2817874743 >> v1:172.21.15.27:6789/0 conn(0x7fe3c815eb90 legacy=0x7fe3c80d2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:46.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.881+0000 7fe3c7fff640 1 -- 172.21.15.27:0/2817874743 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3c80d3a40 con 0x7fe3c80d1550 2024-09-12T03:20:46.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.881+0000 7fe3c67fc640 1 -- 172.21.15.27:0/2817874743 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 149141301 0 0) 0x7fe3c80d3760 con 0x7fe3c815eb90 2024-09-12T03:20:46.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.882+0000 7fe3c7fff640 1 --2- 172.21.15.27:0/2817874743 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe3c80d1550 0x7fe3c80b8190 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fe3c80d4750 tx=0x7fe3c002a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49d989c8aa64bfce server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:46.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.882+0000 7fe3c67fc640 1 -- 172.21.15.27:0/2817874743 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3c0007740 con 0x7fe3c80d1550 2024-09-12T03:20:46.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.882+0000 7fe3c67fc640 1 -- 172.21.15.27:0/2817874743 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe3c00078e0 con 0x7fe3c80d1550 2024-09-12T03:20:46.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.882+0000 7fe3c67fc640 1 -- 172.21.15.27:0/2817874743 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3c0007cb0 con 0x7fe3c80d1550 2024-09-12T03:20:46.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.882+0000 7fe3cddbd640 1 -- 172.21.15.27:0/2817874743 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe3c80d1550 msgr2=0x7fe3c80b8190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:46.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.882+0000 7fe3cddbd640 1 --2- 172.21.15.27:0/2817874743 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe3c80d1550 0x7fe3c80b8190 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fe3c80d4750 tx=0x7fe3c002a680 comp rx=0 tx=0).stop 2024-09-12T03:20:46.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.882+0000 7fe3cddbd640 1 -- 172.21.15.27:0/2817874743 shutdown_connections 2024-09-12T03:20:46.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.882+0000 7fe3cddbd640 1 --2- 172.21.15.27:0/2817874743 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe3c80d1550 0x7fe3c80b8190 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:46.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.882+0000 7fe3cddbd640 1 --2- 172.21.15.27:0/2817874743 >> v2:172.21.15.45:3300/0 conn(0x7fe3c80d5200 0x7fe3c80d1010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:46.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.882+0000 7fe3cddbd640 1 -- 172.21.15.27:0/2817874743 >> 172.21.15.27:0/2817874743 conn(0x7fe3c806dce0 msgr2=0x7fe3c806e0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:46.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.882+0000 7fe3cddbd640 1 -- 172.21.15.27:0/2817874743 shutdown_connections 2024-09-12T03:20:46.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.882+0000 7fe3cddbd640 1 -- 172.21.15.27:0/2817874743 wait complete. 2024-09-12T03:20:46.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.884+0000 7fe3cddbd640 1 Processor -- start 2024-09-12T03:20:46.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.884+0000 7fe3cddbd640 1 -- start start 2024-09-12T03:20:46.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.885+0000 7fe3cddbd640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe3c80d1550 0x7fe3c8155f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:46.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.885+0000 7fe3cddbd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe3c80d5200 0x7fe3c815f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:46.888 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.885+0000 7fe3cddbd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe3c815eb90 0x7fe3c80c8340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:46.888 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.885+0000 7fe3cddbd640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fe3c80d4000 con 0x7fe3c80d5200 2024-09-12T03:20:46.888 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.885+0000 7fe3cddbd640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe3c80d3500 con 0x7fe3c815eb90 2024-09-12T03:20:46.888 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.885+0000 7fe3cddbd640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fe3c80d1960 con 0x7fe3c80d1550 2024-09-12T03:20:46.888 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.885+0000 7fe3c6ffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe3c80d5200 0x7fe3c815f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:46.888 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.885+0000 7fe3c6ffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe3c80d5200 0x7fe3c815f2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42600/0 (socket says 172.21.15.27:42600) 2024-09-12T03:20:46.888 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.885+0000 7fe3c6ffd640 1 -- 172.21.15.27:0/3818446927 learned_addr learned my addr 172.21.15.27:0/3818446927 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:20:46.888 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.885+0000 7fe3c77fe640 1 --2- 172.21.15.27:0/3818446927 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe3c80d1550 0x7fe3c8155f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:46.888 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.885+0000 7fe3c7fff640 1 --2- 172.21.15.27:0/3818446927 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe3c815eb90 0x7fe3c80c8340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:46.888 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.885+0000 7fe3c6ffd640 1 -- 172.21.15.27:0/3818446927 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe3c80d1550 msgr2=0x7fe3c8155f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:46.888 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.885+0000 7fe3c6ffd640 1 --2- 172.21.15.27:0/3818446927 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe3c80d1550 0x7fe3c8155f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:46.889 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.885+0000 7fe3c6ffd640 1 -- 172.21.15.27:0/3818446927 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe3c815eb90 msgr2=0x7fe3c80c8340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:46.889 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.885+0000 7fe3c6ffd640 1 --2- 172.21.15.27:0/3818446927 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe3c815eb90 0x7fe3c80c8340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:46.889 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.885+0000 7fe3c6ffd640 1 -- 172.21.15.27:0/3818446927 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3c815fcf0 con 0x7fe3c80d5200 2024-09-12T03:20:46.889 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.885+0000 7fe3c77fe640 1 --2- 172.21.15.27:0/3818446927 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe3c80d1550 0x7fe3c8155f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:20:46.889 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.885+0000 7fe3c6ffd640 1 --2- 172.21.15.27:0/3818446927 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe3c80d5200 0x7fe3c815f2a0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7fe3bc00d800 tx=0x7fe3bc00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:46.889 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.885+0000 7fe3affff640 1 -- 172.21.15.27:0/3818446927 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3bc00b6c0 con 0x7fe3c80d5200 2024-09-12T03:20:46.889 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.885+0000 7fe3affff640 1 -- 172.21.15.27:0/3818446927 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe3bc019070 con 0x7fe3c80d5200 2024-09-12T03:20:46.889 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.885+0000 7fe3affff640 1 -- 172.21.15.27:0/3818446927 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3bc00ba80 con 0x7fe3c80d5200 2024-09-12T03:20:46.889 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.885+0000 7fe3cddbd640 1 -- 172.21.15.27:0/3818446927 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3c815ffd0 con 0x7fe3c80d5200 2024-09-12T03:20:46.889 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.885+0000 7fe3cddbd640 1 -- 172.21.15.27:0/3818446927 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3c8160590 con 0x7fe3c80d5200 2024-09-12T03:20:46.889 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.887+0000 7fe3cddbd640 1 -- 172.21.15.27:0/3818446927 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3c815f090 con 0x7fe3c80d5200 2024-09-12T03:20:46.890 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.887+0000 7fe3affff640 1 -- 172.21.15.27:0/3818446927 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 90600+0+0 (secure 0 0 0) 0x7fe3bc00bc20 con 0x7fe3c80d5200 2024-09-12T03:20:46.890 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.887+0000 7fe3affff640 1 --2- 172.21.15.27:0/3818446927 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fe3b406bf50 0x7fe3b406e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:46.890 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.888+0000 7fe3affff640 1 -- 172.21.15.27:0/3818446927 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5795+0+0 (secure 0 0 0) 0x7fe3bc08b490 con 0x7fe3c80d5200 2024-09-12T03:20:46.890 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.888+0000 7fe3c77fe640 1 --2- 172.21.15.27:0/3818446927 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fe3b406bf50 0x7fe3b406e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:46.890 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.888+0000 7fe3c77fe640 1 --2- 172.21.15.27:0/3818446927 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fe3b406bf50 0x7fe3b406e410 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe3b8000f60 tx=0x7fe3b8007c30 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:46.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:46.892+0000 7fe3affff640 1 -- 172.21.15.27:0/3818446927 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fe3bc059d60 con 0x7fe3c80d5200 2024-09-12T03:20:47.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.102+0000 7fe3cddbd640 1 -- 172.21.15.27:0/3818446927 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe3c80c8030 con 0x7fe3c80d5200 2024-09-12T03:20:47.104 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.103+0000 7fe3affff640 1 -- 172.21.15.27:0/3818446927 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 51 v51) ==== 94+0+5219 (secure 0 0 0) 0x7fe3bc059660 con 0x7fe3c80d5200 2024-09-12T03:20:47.104 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:20:47.104 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":51,"btime":"2024-09-12T03:20:43:233119+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5669,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/2036680565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2036680565},{"type":"v1","addr":"172.21.15.45:6834","nonce":2036680565}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":43},{"gid":6094,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/874218126","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":874218126},{"type":"v1","addr":"172.21.15.45:6835","nonce":874218126}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":43},{"gid":6693,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6834/658418980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":658418980},{"type":"v1","addr":"172.21.15.27:6834","nonce":658418980}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":51}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-12T03:20:28.499550+0000","modified":"2024-09-12T03:20:30.510730+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6486},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6486":{"gid":6486,"name":"a","rank":0,"incarnation":44,"state":"up:active","state_seq":3,"addr":"172.21.15.27:6835/3209963418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":3209963418},{"type":"v1","addr":"172.21.15.27:6835","nonce":3209963418}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6486,"qdb_cluster":[6486]},"id":6}]} 2024-09-12T03:20:47.104 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 51 2024-09-12T03:20:47.106 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.105+0000 7fe3cddbd640 1 -- 172.21.15.27:0/3818446927 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fe3b406bf50 msgr2=0x7fe3b406e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:47.106 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.105+0000 7fe3cddbd640 1 --2- 172.21.15.27:0/3818446927 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fe3b406bf50 0x7fe3b406e410 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe3b8000f60 tx=0x7fe3b8007c30 comp rx=0 tx=0).stop 2024-09-12T03:20:47.106 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.105+0000 7fe3cddbd640 1 -- 172.21.15.27:0/3818446927 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe3c80d5200 msgr2=0x7fe3c815f2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:47.106 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.105+0000 7fe3cddbd640 1 --2- 172.21.15.27:0/3818446927 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe3c80d5200 0x7fe3c815f2a0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7fe3bc00d800 tx=0x7fe3bc00dcc0 comp rx=0 tx=0).stop 2024-09-12T03:20:47.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.105+0000 7fe3cddbd640 1 -- 172.21.15.27:0/3818446927 shutdown_connections 2024-09-12T03:20:47.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.105+0000 7fe3cddbd640 1 --2- 172.21.15.27:0/3818446927 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe3c815eb90 0x7fe3c80c8340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:47.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.105+0000 7fe3cddbd640 1 --2- 172.21.15.27:0/3818446927 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fe3b406bf50 0x7fe3b406e410 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:47.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.105+0000 7fe3cddbd640 1 --2- 172.21.15.27:0/3818446927 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe3c80d5200 0x7fe3c815f2a0 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:47.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.105+0000 7fe3cddbd640 1 --2- 172.21.15.27:0/3818446927 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe3c80d1550 0x7fe3c8155f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:47.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.105+0000 7fe3cddbd640 1 -- 172.21.15.27:0/3818446927 >> 172.21.15.27:0/3818446927 conn(0x7fe3c806dce0 msgr2=0x7fe3c80cfcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:47.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.105+0000 7fe3cddbd640 1 -- 172.21.15.27:0/3818446927 shutdown_connections 2024-09-12T03:20:47.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.105+0000 7fe3cddbd640 1 -- 172.21.15.27:0/3818446927 wait complete. 2024-09-12T03:20:47.117 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-12T03:20:47.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.218+0000 7fe21d511640 1 Processor -- start 2024-09-12T03:20:47.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.219+0000 7fe21d511640 1 -- start start 2024-09-12T03:20:47.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.219+0000 7fe21d511640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe2180cda40 0x7fe21815ca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:47.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.219+0000 7fe21d511640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe2180b4d70 con 0x7fe218146010 2024-09-12T03:20:47.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.219+0000 7fe21d511640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe21806d140 con 0x7fe218145c30 2024-09-12T03:20:47.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.219+0000 7fe21d511640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fe2180b5050 con 0x7fe2180cda40 2024-09-12T03:20:47.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.220+0000 7fe2177fe640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fe218146010 0x7fe2180b4040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:47252/0 (socket says 172.21.15.27:47252) 2024-09-12T03:20:47.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.220+0000 7fe2177fe640 1 -- 172.21.15.27:0/2953606090 learned_addr learned my addr 172.21.15.27:0/2953606090 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:20:47.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.220+0000 7fe2167fc640 1 --2- 172.21.15.27:0/2953606090 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe2180cda40 0x7fe21815ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:47.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.220+0000 7fe215ffb640 1 -- 172.21.15.27:0/2953606090 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 972378718 0 0) 0x7fe2180b4d70 con 0x7fe218146010 2024-09-12T03:20:47.225 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.220+0000 7fe215ffb640 1 -- 172.21.15.27:0/2953606090 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe200003520 con 0x7fe218146010 2024-09-12T03:20:47.225 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.220+0000 7fe215ffb640 1 -- 172.21.15.27:0/2953606090 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2140535826 0 0) 0x7fe200003520 con 0x7fe218146010 2024-09-12T03:20:47.225 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.220+0000 7fe215ffb640 1 -- 172.21.15.27:0/2953606090 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe2180cda40 msgr2=0x7fe21815ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:47.225 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.220+0000 7fe215ffb640 1 --2- 172.21.15.27:0/2953606090 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe2180cda40 0x7fe21815ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:47.225 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.220+0000 7fe215ffb640 1 -- 172.21.15.27:0/2953606090 >> v1:172.21.15.45:6789/0 conn(0x7fe218145c30 legacy=0x7fe2180cd330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:47.225 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.220+0000 7fe215ffb640 1 -- 172.21.15.27:0/2953606090 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe21806d420 con 0x7fe218146010 2024-09-12T03:20:47.225 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.220+0000 7fe215ffb640 1 -- 172.21.15.27:0/2953606090 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 847755213 0 0) 0x7fe2100019f0 con 0x7fe218146010 2024-09-12T03:20:47.225 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.220+0000 7fe215ffb640 1 -- 172.21.15.27:0/2953606090 >> v1:172.21.15.27:6789/0 conn(0x7fe218146010 legacy=0x7fe2180b4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:47.225 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.220+0000 7fe215ffb640 1 --2- 172.21.15.27:0/2953606090 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe2180cda40 0x7fe200003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:47.225 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.220+0000 7fe215ffb640 1 --2- 172.21.15.27:0/2953606090 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe200003f80 0x7fe200004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:47.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.220+0000 7fe215ffb640 1 --2- 172.21.15.27:0/2953606090 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe200004d40 0x7fe2000051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:47.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.220+0000 7fe215ffb640 1 -- 172.21.15.27:0/2953606090 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fe2180b5050 con 0x7fe200003f80 2024-09-12T03:20:47.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.220+0000 7fe215ffb640 1 -- 172.21.15.27:0/2953606090 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe20000bb80 con 0x7fe200004d40 2024-09-12T03:20:47.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.220+0000 7fe215ffb640 1 -- 172.21.15.27:0/2953606090 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fe20000bd80 con 0x7fe2180cda40 2024-09-12T03:20:47.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.220+0000 7fe2167fc640 1 --2- 172.21.15.27:0/2953606090 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe200003f80 0x7fe200004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:47.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.220+0000 7fe216ffd640 1 --2- 172.21.15.27:0/2953606090 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe2180cda40 0x7fe200003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:47.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.220+0000 7fe2177fe640 1 --2- 172.21.15.27:0/2953606090 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe200004d40 0x7fe2000051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:47.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.221+0000 7fe2167fc640 1 -- 172.21.15.27:0/2953606090 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe2180cda40 msgr2=0x7fe200003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:47.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.221+0000 7fe2167fc640 1 --2- 172.21.15.27:0/2953606090 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe2180cda40 0x7fe200003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:47.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.221+0000 7fe2167fc640 1 -- 172.21.15.27:0/2953606090 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe200004d40 msgr2=0x7fe2000051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:47.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.221+0000 7fe2167fc640 1 --2- 172.21.15.27:0/2953606090 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe200004d40 0x7fe2000051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:47.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.221+0000 7fe2167fc640 1 -- 172.21.15.27:0/2953606090 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe200003520 con 0x7fe200003f80 2024-09-12T03:20:47.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.221+0000 7fe216ffd640 1 --2- 172.21.15.27:0/2953606090 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe2180cda40 0x7fe200003a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:20:47.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.221+0000 7fe2177fe640 1 --2- 172.21.15.27:0/2953606090 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe200004d40 0x7fe2000051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:20:47.227 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.221+0000 7fe2167fc640 1 --2- 172.21.15.27:0/2953606090 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe200003f80 0x7fe200004800 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fe20c009ac0 tx=0x7fe200005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39fdd71a28737bf2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:47.227 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.222+0000 7fe215ffb640 1 -- 172.21.15.27:0/2953606090 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe20c036070 con 0x7fe200003f80 2024-09-12T03:20:47.227 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.222+0000 7fe215ffb640 1 -- 172.21.15.27:0/2953606090 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe20c031400 con 0x7fe200003f80 2024-09-12T03:20:47.227 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.222+0000 7fe215ffb640 1 -- 172.21.15.27:0/2953606090 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe20c009690 con 0x7fe200003f80 2024-09-12T03:20:47.227 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.222+0000 7fe21d511640 1 -- 172.21.15.27:0/2953606090 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe200003f80 msgr2=0x7fe200004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:47.227 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.222+0000 7fe21d511640 1 --2- 172.21.15.27:0/2953606090 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe200003f80 0x7fe200004800 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fe20c009ac0 tx=0x7fe200005830 comp rx=0 tx=0).stop 2024-09-12T03:20:47.227 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.222+0000 7fe2177fe640 1 -- 172.21.15.27:0/2953606090 reap_dead start 2024-09-12T03:20:47.227 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.222+0000 7fe21d511640 1 -- 172.21.15.27:0/2953606090 shutdown_connections 2024-09-12T03:20:47.227 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.222+0000 7fe21d511640 1 -- 172.21.15.27:0/2953606090 >> 172.21.15.27:0/2953606090 conn(0x7fe218062c60 msgr2=0x7fe21806bb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:47.227 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.222+0000 7fe21d511640 1 -- 172.21.15.27:0/2953606090 shutdown_connections 2024-09-12T03:20:47.227 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.222+0000 7fe21d511640 1 -- 172.21.15.27:0/2953606090 wait complete. 2024-09-12T03:20:47.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.223+0000 7fe21d511640 1 Processor -- start 2024-09-12T03:20:47.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.223+0000 7fe21d511640 1 -- start start 2024-09-12T03:20:47.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.224+0000 7fe21d511640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe200003f80 0x7fe218071030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:47.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.224+0000 7fe21d511640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe2181f8830 0x7fe2180c5740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:47.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.224+0000 7fe21d511640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe218132720 0x7fe218132ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:47.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.224+0000 7fe21d511640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fe21815cb80 con 0x7fe200003f80 2024-09-12T03:20:47.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.224+0000 7fe21d511640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe21815ca00 con 0x7fe218132720 2024-09-12T03:20:47.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.224+0000 7fe21d511640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fe21815cd00 con 0x7fe2181f8830 2024-09-12T03:20:47.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.224+0000 7fe216ffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe200003f80 0x7fe218071030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:47.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.224+0000 7fe216ffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe200003f80 0x7fe218071030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42622/0 (socket says 172.21.15.27:42622) 2024-09-12T03:20:47.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.224+0000 7fe216ffd640 1 -- 172.21.15.27:0/1948893757 learned_addr learned my addr 172.21.15.27:0/1948893757 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:20:47.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.224+0000 7fe2177fe640 1 --2- 172.21.15.27:0/1948893757 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe218132720 0x7fe218132ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:47.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.224+0000 7fe2167fc640 1 --2- 172.21.15.27:0/1948893757 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe2181f8830 0x7fe2180c5740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:47.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.225+0000 7fe216ffd640 1 -- 172.21.15.27:0/1948893757 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe2181f8830 msgr2=0x7fe2180c5740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:47.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.225+0000 7fe216ffd640 1 --2- 172.21.15.27:0/1948893757 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe2181f8830 0x7fe2180c5740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:47.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.225+0000 7fe216ffd640 1 -- 172.21.15.27:0/1948893757 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe218132720 msgr2=0x7fe218132ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:47.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.225+0000 7fe216ffd640 1 --2- 172.21.15.27:0/1948893757 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe218132720 0x7fe218132ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:47.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.225+0000 7fe216ffd640 1 -- 172.21.15.27:0/1948893757 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2181f8cb0 con 0x7fe200003f80 2024-09-12T03:20:47.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.225+0000 7fe216ffd640 1 --2- 172.21.15.27:0/1948893757 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe200003f80 0x7fe218071030 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7fe20800e600 tx=0x7fe20800eac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:47.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.225+0000 7fe1ff7fe640 1 -- 172.21.15.27:0/1948893757 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2080052e0 con 0x7fe200003f80 2024-09-12T03:20:47.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.225+0000 7fe1ff7fe640 1 -- 172.21.15.27:0/1948893757 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe208018070 con 0x7fe200003f80 2024-09-12T03:20:47.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.225+0000 7fe21d511640 1 -- 172.21.15.27:0/1948893757 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe218071760 con 0x7fe200003f80 2024-09-12T03:20:47.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.225+0000 7fe1ff7fe640 1 -- 172.21.15.27:0/1948893757 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe208003810 con 0x7fe200003f80 2024-09-12T03:20:47.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.225+0000 7fe21d511640 1 -- 172.21.15.27:0/1948893757 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe218071d20 con 0x7fe200003f80 2024-09-12T03:20:47.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.226+0000 7fe21d511640 1 -- 172.21.15.27:0/1948893757 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2180b47e0 con 0x7fe200003f80 2024-09-12T03:20:47.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.226+0000 7fe1ff7fe640 1 -- 172.21.15.27:0/1948893757 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 90600+0+0 (secure 0 0 0) 0x7fe2080039b0 con 0x7fe200003f80 2024-09-12T03:20:47.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.227+0000 7fe1ff7fe640 1 --2- 172.21.15.27:0/1948893757 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fe20406bf50 0x7fe20406e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:47.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.227+0000 7fe1ff7fe640 1 -- 172.21.15.27:0/1948893757 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5795+0+0 (secure 0 0 0) 0x7fe20808ad90 con 0x7fe200003f80 2024-09-12T03:20:47.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.227+0000 7fe2167fc640 1 --2- 172.21.15.27:0/1948893757 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fe20406bf50 0x7fe20406e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:47.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.227+0000 7fe2167fc640 1 --2- 172.21.15.27:0/1948893757 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fe20406bf50 0x7fe20406e410 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fe20c033000 tx=0x7fe20c008030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:47.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.231+0000 7fe1ff7fe640 1 -- 172.21.15.27:0/1948893757 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fe208059660 con 0x7fe200003f80 2024-09-12T03:20:47.407 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.406+0000 7fe21d511640 1 -- 172.21.15.27:0/1948893757 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe2180ab570 con 0x7fe200003f80 2024-09-12T03:20:47.408 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.407+0000 7fe1ff7fe640 1 -- 172.21.15.27:0/1948893757 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+16801 (secure 0 0 0) 0x7fe208058f60 con 0x7fe200003f80 2024-09-12T03:20:47.408 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:20:47.408 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":89,"fsid":"d2df504e-2647-4385-a5db-9fdf69f487a0","created":"2024-09-12T03:16:07.778937+0000","modified":"2024-09-12T03:20:45.239116+0000","last_up_change":"2024-09-12T03:16:19.314778+0000","last_in_change":"2024-09-12T03:16:09.893086+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T03:16:20.097646+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-09-12T03:20:26.167239+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-09-12T03:20:27.491764+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1851116c-1fb3-4dc9-9772-f90e2ef4c355","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6817","nonce":3002321488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6819","nonce":3002321488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6823","nonce":3002321488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6821","nonce":3002321488}]},"public_addr":"172.21.15.27:6817/3002321488","cluster_addr":"172.21.15.27:6819/3002321488","heartbeat_back_addr":"172.21.15.27:6823/3002321488","heartbeat_front_addr":"172.21.15.27:6821/3002321488","state":["exists","up"]},{"osd":1,"uuid":"f38ae17d-73eb-4204-b99d-975fa806fb87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6809","nonce":1613102886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6811","nonce":1613102886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6815","nonce":1613102886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6813","nonce":1613102886}]},"public_addr":"172.21.15.27:6809/1613102886","cluster_addr":"172.21.15.27:6811/1613102886","heartbeat_back_addr":"172.21.15.27:6815/1613102886","heartbeat_front_addr":"172.21.15.27:6813/1613102886","state":["exists","up"]},{"osd":2,"uuid":"fb12dcc8-bdd0-42b8-b46d-8b10106224bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6825","nonce":3534723355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6827","nonce":3534723355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6831","nonce":3534723355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6829","nonce":3534723355}]},"public_addr":"172.21.15.27:6825/3534723355","cluster_addr":"172.21.15.27:6827/3534723355","heartbeat_back_addr":"172.21.15.27:6831/3534723355","heartbeat_front_addr":"172.21.15.27:6829/3534723355","state":["exists","up"]},{"osd":3,"uuid":"d74d3f49-9c24-4b9f-98e2-3ecd7b5ef3b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6801","nonce":80661598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6803","nonce":80661598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6807","nonce":80661598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6805","nonce":80661598}]},"public_addr":"172.21.15.27:6801/80661598","cluster_addr":"172.21.15.27:6803/80661598","heartbeat_back_addr":"172.21.15.27:6807/80661598","heartbeat_front_addr":"172.21.15.27:6805/80661598","state":["exists","up"]},{"osd":4,"uuid":"2bf123f2-1b0d-41c4-aa3b-805bab267e59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6817","nonce":3943189827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6819","nonce":3943189827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6823","nonce":3943189827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6821","nonce":3943189827}]},"public_addr":"172.21.15.45:6817/3943189827","cluster_addr":"172.21.15.45:6819/3943189827","heartbeat_back_addr":"172.21.15.45:6823/3943189827","heartbeat_front_addr":"172.21.15.45:6821/3943189827","state":["exists","up"]},{"osd":5,"uuid":"c49ba5da-91c6-4a86-a97b-4aba89c253ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6809","nonce":3620760330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6811","nonce":3620760330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6815","nonce":3620760330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6813","nonce":3620760330}]},"public_addr":"172.21.15.45:6809/3620760330","cluster_addr":"172.21.15.45:6811/3620760330","heartbeat_back_addr":"172.21.15.45:6815/3620760330","heartbeat_front_addr":"172.21.15.45:6813/3620760330","state":["exists","up"]},{"osd":6,"uuid":"b0123475-89f7-4a34-9e33-d797b99dd9dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6801","nonce":1989083510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6803","nonce":1989083510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6807","nonce":1989083510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6805","nonce":1989083510}]},"public_addr":"172.21.15.45:6801/1989083510","cluster_addr":"172.21.15.45:6803/1989083510","heartbeat_back_addr":"172.21.15.45:6807/1989083510","heartbeat_front_addr":"172.21.15.45:6805/1989083510","state":["exists","up"]},{"osd":7,"uuid":"4dad773d-b7d8-46fc-8a10-a972aad27b29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6825","nonce":1513300294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6827","nonce":1513300294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6831","nonce":1513300294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6829","nonce":1513300294}]},"public_addr":"172.21.15.45:6825/1513300294","cluster_addr":"172.21.15.45:6827/1513300294","heartbeat_back_addr":"172.21.15.45:6831/1513300294","heartbeat_front_addr":"172.21.15.45:6829/1513300294","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.881855+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:17.133252+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.735339+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.500742+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.477571+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:17.167667+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.470291+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.873743+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.27:6834/1466511159":"2024-09-13T03:20:42.229493+0000","172.21.15.27:6832/1466511159":"2024-09-13T03:20:42.229493+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T03:20:47.410 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.409+0000 7fe21d511640 1 -- 172.21.15.27:0/1948893757 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fe20406bf50 msgr2=0x7fe20406e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:47.410 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.409+0000 7fe21d511640 1 --2- 172.21.15.27:0/1948893757 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fe20406bf50 0x7fe20406e410 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fe20c033000 tx=0x7fe20c008030 comp rx=0 tx=0).stop 2024-09-12T03:20:47.410 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.409+0000 7fe21d511640 1 -- 172.21.15.27:0/1948893757 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe200003f80 msgr2=0x7fe218071030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:47.410 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.409+0000 7fe21d511640 1 --2- 172.21.15.27:0/1948893757 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe200003f80 0x7fe218071030 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7fe20800e600 tx=0x7fe20800eac0 comp rx=0 tx=0).stop 2024-09-12T03:20:47.410 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.409+0000 7fe21d511640 1 -- 172.21.15.27:0/1948893757 shutdown_connections 2024-09-12T03:20:47.410 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.409+0000 7fe21d511640 1 --2- 172.21.15.27:0/1948893757 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe218132720 0x7fe218132ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:47.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.409+0000 7fe21d511640 1 --2- 172.21.15.27:0/1948893757 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe2181f8830 0x7fe2180c5740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:47.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.409+0000 7fe21d511640 1 --2- 172.21.15.27:0/1948893757 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fe20406bf50 0x7fe20406e410 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:47.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.409+0000 7fe21d511640 1 --2- 172.21.15.27:0/1948893757 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe200003f80 0x7fe218071030 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:47.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.409+0000 7fe21d511640 1 -- 172.21.15.27:0/1948893757 >> 172.21.15.27:0/1948893757 conn(0x7fe218062c60 msgr2=0x7fe2180b14c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:47.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.409+0000 7fe21d511640 1 -- 172.21.15.27:0/1948893757 shutdown_connections 2024-09-12T03:20:47.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.409+0000 7fe21d511640 1 -- 172.21.15.27:0/1948893757 wait complete. 2024-09-12T03:20:47.421 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-12T03:20:47.421 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-12T03:20:47.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.522+0000 7f9233152640 1 Processor -- start 2024-09-12T03:20:47.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.522+0000 7f9233152640 1 -- start start 2024-09-12T03:20:47.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.523+0000 7f9233152640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f922c0b1ca0 0x7f922c15f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:47.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.523+0000 7f9233152640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f922c15fd70 con 0x7f922c06f360 2024-09-12T03:20:47.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.523+0000 7f9233152640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f922c06cc40 con 0x7f922c0cf660 2024-09-12T03:20:47.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.523+0000 7f9233152640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f922c0d22f0 con 0x7f922c0b1ca0 2024-09-12T03:20:47.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.523+0000 7f922bfff640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f922c06f360 0x7f922c0cef50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:47266/0 (socket says 172.21.15.27:47266) 2024-09-12T03:20:47.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.523+0000 7f922bfff640 1 -- 172.21.15.27:0/3306792200 learned_addr learned my addr 172.21.15.27:0/3306792200 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:20:47.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.523+0000 7f9230eee640 1 --2- 172.21.15.27:0/3306792200 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f922c0b1ca0 0x7f922c15f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:47.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.523+0000 7f922b7fe640 1 -- 172.21.15.27:0/3306792200 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2234856373 0 0) 0x7f9220001110 con 0x7f922c06f360 2024-09-12T03:20:47.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.523+0000 7f922b7fe640 1 -- 172.21.15.27:0/3306792200 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9214003520 con 0x7f922c06f360 2024-09-12T03:20:47.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.524+0000 7f922b7fe640 1 -- 172.21.15.27:0/3306792200 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1946246621 0 0) 0x7f9214003520 con 0x7f922c06f360 2024-09-12T03:20:47.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.524+0000 7f922b7fe640 1 -- 172.21.15.27:0/3306792200 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f922c0b1ca0 msgr2=0x7f922c15f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:47.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.524+0000 7f922b7fe640 1 --2- 172.21.15.27:0/3306792200 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f922c0b1ca0 0x7f922c15f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:47.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.524+0000 7f922b7fe640 1 -- 172.21.15.27:0/3306792200 >> v1:172.21.15.45:6789/0 conn(0x7f922c0cf660 legacy=0x7f922c0d2780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:47.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.524+0000 7f922b7fe640 1 -- 172.21.15.27:0/3306792200 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f922c0d1ce0 con 0x7f922c06f360 2024-09-12T03:20:47.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.524+0000 7f922b7fe640 1 -- 172.21.15.27:0/3306792200 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 847755213 0 0) 0x7f9220001840 con 0x7f922c06f360 2024-09-12T03:20:47.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.524+0000 7f922b7fe640 1 -- 172.21.15.27:0/3306792200 >> v1:172.21.15.27:6789/0 conn(0x7f922c06f360 legacy=0x7f922c0cef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:47.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.524+0000 7f922b7fe640 1 --2- 172.21.15.27:0/3306792200 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f922c0b1ca0 0x7f9214003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:47.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.524+0000 7f922b7fe640 1 --2- 172.21.15.27:0/3306792200 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9214003f80 0x7f9214004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:47.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.524+0000 7f922b7fe640 1 --2- 172.21.15.27:0/3306792200 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9214004d40 0x7f92140051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:47.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.524+0000 7f922b7fe640 1 -- 172.21.15.27:0/3306792200 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f922c0d22f0 con 0x7f9214004d40 2024-09-12T03:20:47.528 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.524+0000 7f922b7fe640 1 -- 172.21.15.27:0/3306792200 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f921400bad0 con 0x7f9214003f80 2024-09-12T03:20:47.528 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.524+0000 7f922b7fe640 1 -- 172.21.15.27:0/3306792200 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f921400bcd0 con 0x7f922c0b1ca0 2024-09-12T03:20:47.528 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.524+0000 7f92316ef640 1 --2- 172.21.15.27:0/3306792200 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9214004d40 0x7f92140051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:47.528 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.524+0000 7f922bfff640 1 --2- 172.21.15.27:0/3306792200 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9214003f80 0x7f9214004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:47.528 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.524+0000 7f9230eee640 1 --2- 172.21.15.27:0/3306792200 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f922c0b1ca0 0x7f9214003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:47.528 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.524+0000 7f92316ef640 1 -- 172.21.15.27:0/3306792200 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f922c0b1ca0 msgr2=0x7f9214003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:47.528 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.524+0000 7f92316ef640 1 --2- 172.21.15.27:0/3306792200 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f922c0b1ca0 0x7f9214003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:47.528 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.524+0000 7f92316ef640 1 -- 172.21.15.27:0/3306792200 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9214003f80 msgr2=0x7f9214004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:47.528 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.524+0000 7f92316ef640 1 --2- 172.21.15.27:0/3306792200 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9214003f80 0x7f9214004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:47.528 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.524+0000 7f92316ef640 1 -- 172.21.15.27:0/3306792200 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9214003520 con 0x7f9214004d40 2024-09-12T03:20:47.528 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.525+0000 7f92316ef640 1 --2- 172.21.15.27:0/3306792200 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9214004d40 0x7f92140051c0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f922c069d10 tx=0x7f9214005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=498f8a6837528c1d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:47.528 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.525+0000 7f922b7fe640 1 -- 172.21.15.27:0/3306792200 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9224001a70 con 0x7f9214004d40 2024-09-12T03:20:47.529 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.525+0000 7f922b7fe640 1 -- 172.21.15.27:0/3306792200 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f922402add0 con 0x7f9214004d40 2024-09-12T03:20:47.529 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.525+0000 7f922b7fe640 1 -- 172.21.15.27:0/3306792200 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9224003bb0 con 0x7f9214004d40 2024-09-12T03:20:47.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.526+0000 7f9233152640 1 -- 172.21.15.27:0/3306792200 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9214004d40 msgr2=0x7f92140051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:47.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.526+0000 7f9233152640 1 --2- 172.21.15.27:0/3306792200 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9214004d40 0x7f92140051c0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f922c069d10 tx=0x7f9214005830 comp rx=0 tx=0).stop 2024-09-12T03:20:47.532 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.526+0000 7f92316ef640 1 -- 172.21.15.27:0/3306792200 reap_dead start 2024-09-12T03:20:47.532 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.526+0000 7f9233152640 1 -- 172.21.15.27:0/3306792200 shutdown_connections 2024-09-12T03:20:47.532 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.526+0000 7f9233152640 1 -- 172.21.15.27:0/3306792200 >> 172.21.15.27:0/3306792200 conn(0x7f922c073800 msgr2=0x7f922c09ba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:47.532 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.526+0000 7f9233152640 1 -- 172.21.15.27:0/3306792200 shutdown_connections 2024-09-12T03:20:47.532 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.526+0000 7f9233152640 1 -- 172.21.15.27:0/3306792200 wait complete. 2024-09-12T03:20:47.532 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.527+0000 7f9233152640 1 Processor -- start 2024-09-12T03:20:47.532 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.527+0000 7f9233152640 1 -- start start 2024-09-12T03:20:47.532 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.528+0000 7f9233152640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f922c15a040 0x7f922c0b2140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:47.532 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.528+0000 7f9233152640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f922c0b2680 0x7f922c0707f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:47.532 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.528+0000 7f9233152640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f922c070d30 0x7f922c0b54d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:47.532 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.528+0000 7f9233152640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f922c06d1c0 con 0x7f922c15a040 2024-09-12T03:20:47.532 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.528+0000 7f9233152640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f922c0d1f80 con 0x7f922c070d30 2024-09-12T03:20:47.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.528+0000 7f9233152640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f922c06d340 con 0x7f922c0b2680 2024-09-12T03:20:47.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.528+0000 7f9230eee640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f922c15a040 0x7f922c0b2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:47.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.528+0000 7f9230eee640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f922c15a040 0x7f922c0b2140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42626/0 (socket says 172.21.15.27:42626) 2024-09-12T03:20:47.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.528+0000 7f9230eee640 1 -- 172.21.15.27:0/2961257384 learned_addr learned my addr 172.21.15.27:0/2961257384 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:20:47.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.528+0000 7f92316ef640 1 --2- 172.21.15.27:0/2961257384 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f922c070d30 0x7f922c0b54d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:47.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.528+0000 7f922bfff640 1 --2- 172.21.15.27:0/2961257384 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f922c0b2680 0x7f922c0707f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:47.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.528+0000 7f9230eee640 1 -- 172.21.15.27:0/2961257384 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f922c0b2680 msgr2=0x7f922c0707f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:47.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.528+0000 7f9230eee640 1 --2- 172.21.15.27:0/2961257384 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f922c0b2680 0x7f922c0707f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:47.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.528+0000 7f9230eee640 1 -- 172.21.15.27:0/2961257384 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f922c070d30 msgr2=0x7f922c0b54d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:47.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.528+0000 7f9230eee640 1 --2- 172.21.15.27:0/2961257384 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f922c070d30 0x7f922c0b54d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:47.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.528+0000 7f9230eee640 1 -- 172.21.15.27:0/2961257384 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f922c0cc890 con 0x7f922c15a040 2024-09-12T03:20:47.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.528+0000 7f922bfff640 1 --2- 172.21.15.27:0/2961257384 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f922c0b2680 0x7f922c0707f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:20:47.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.528+0000 7f9230eee640 1 --2- 172.21.15.27:0/2961257384 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f922c15a040 0x7f922c0b2140 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f921c0119a0 tx=0x7f921c011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:47.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.529+0000 7f92297fa640 1 -- 172.21.15.27:0/2961257384 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f921c00fa90 con 0x7f922c15a040 2024-09-12T03:20:47.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.529+0000 7f92297fa640 1 -- 172.21.15.27:0/2961257384 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f921c00fc30 con 0x7f922c15a040 2024-09-12T03:20:47.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.529+0000 7f9233152640 1 -- 172.21.15.27:0/2961257384 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f922c0b2c00 con 0x7f922c15a040 2024-09-12T03:20:47.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.529+0000 7f92297fa640 1 -- 172.21.15.27:0/2961257384 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f921c014440 con 0x7f922c15a040 2024-09-12T03:20:47.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.529+0000 7f9233152640 1 -- 172.21.15.27:0/2961257384 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f922c0b31c0 con 0x7f922c15a040 2024-09-12T03:20:47.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.530+0000 7f9233152640 1 -- 172.21.15.27:0/2961257384 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f922c0cfc40 con 0x7f922c15a040 2024-09-12T03:20:47.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.530+0000 7f92297fa640 1 -- 172.21.15.27:0/2961257384 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 90600+0+0 (secure 0 0 0) 0x7f921c012040 con 0x7f922c15a040 2024-09-12T03:20:47.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.531+0000 7f92297fa640 1 --2- 172.21.15.27:0/2961257384 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f921806bf50 0x7f921806e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:47.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.531+0000 7f922bfff640 1 --2- 172.21.15.27:0/2961257384 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f921806bf50 0x7f921806e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:47.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.531+0000 7f92297fa640 1 -- 172.21.15.27:0/2961257384 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5795+0+0 (secure 0 0 0) 0x7f921c08b620 con 0x7f922c15a040 2024-09-12T03:20:47.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.532+0000 7f922bfff640 1 --2- 172.21.15.27:0/2961257384 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f921806bf50 0x7f921806e410 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f922c0d1340 tx=0x7f9220009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:47.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.535+0000 7f92297fa640 1 -- 172.21.15.27:0/2961257384 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f921c019030 con 0x7f922c15a040 2024-09-12T03:20:47.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.744+0000 7f9233152640 1 -- 172.21.15.27:0/2961257384 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f922c071110 con 0x7f922c15a040 2024-09-12T03:20:47.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.744+0000 7f92297fa640 1 -- 172.21.15.27:0/2961257384 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 51 v51) ==== 94+0+5219 (secure 0 0 0) 0x7f921c059e60 con 0x7f922c15a040 2024-09-12T03:20:47.746 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:20:47.746 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":51,"btime":"2024-09-12T03:20:43:233119+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5669,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/2036680565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2036680565},{"type":"v1","addr":"172.21.15.45:6834","nonce":2036680565}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":43},{"gid":6094,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/874218126","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":874218126},{"type":"v1","addr":"172.21.15.45:6835","nonce":874218126}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":43},{"gid":6693,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6834/658418980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":658418980},{"type":"v1","addr":"172.21.15.27:6834","nonce":658418980}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":51}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-12T03:20:28.499550+0000","modified":"2024-09-12T03:20:30.510730+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6486},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6486":{"gid":6486,"name":"a","rank":0,"incarnation":44,"state":"up:active","state_seq":3,"addr":"172.21.15.27:6835/3209963418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":3209963418},{"type":"v1","addr":"172.21.15.27:6835","nonce":3209963418}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6486,"qdb_cluster":[6486]},"id":6}]} 2024-09-12T03:20:47.746 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 51 2024-09-12T03:20:47.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.746+0000 7f9233152640 1 -- 172.21.15.27:0/2961257384 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f921806bf50 msgr2=0x7f921806e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:47.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.746+0000 7f9233152640 1 --2- 172.21.15.27:0/2961257384 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f921806bf50 0x7f921806e410 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f922c0d1340 tx=0x7f9220009040 comp rx=0 tx=0).stop 2024-09-12T03:20:47.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.746+0000 7f9233152640 1 -- 172.21.15.27:0/2961257384 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f922c15a040 msgr2=0x7f922c0b2140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:47.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.746+0000 7f9233152640 1 --2- 172.21.15.27:0/2961257384 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f922c15a040 0x7f922c0b2140 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f921c0119a0 tx=0x7f921c011e60 comp rx=0 tx=0).stop 2024-09-12T03:20:47.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.746+0000 7f9233152640 1 -- 172.21.15.27:0/2961257384 shutdown_connections 2024-09-12T03:20:47.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.746+0000 7f9233152640 1 --2- 172.21.15.27:0/2961257384 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f922c070d30 0x7f922c0b54d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:47.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.746+0000 7f9233152640 1 --2- 172.21.15.27:0/2961257384 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f922c0b2680 0x7f922c0707f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:47.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.746+0000 7f9233152640 1 --2- 172.21.15.27:0/2961257384 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f921806bf50 0x7f921806e410 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:47.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.746+0000 7f9233152640 1 --2- 172.21.15.27:0/2961257384 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f922c15a040 0x7f922c0b2140 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:47.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.747+0000 7f9233152640 1 -- 172.21.15.27:0/2961257384 >> 172.21.15.27:0/2961257384 conn(0x7f922c073800 msgr2=0x7f922c1615d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:47.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.747+0000 7f9233152640 1 -- 172.21.15.27:0/2961257384 shutdown_connections 2024-09-12T03:20:47.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.747+0000 7f9233152640 1 -- 172.21.15.27:0/2961257384 wait complete. 2024-09-12T03:20:47.758 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-12T03:20:47.860 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.859+0000 7f6a771e8640 1 Processor -- start 2024-09-12T03:20:47.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.860+0000 7f6a771e8640 1 -- start start 2024-09-12T03:20:47.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.860+0000 7f6a771e8640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f6a7015eb90 0x7f6a700d2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:47.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.860+0000 7f6a771e8640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f6a700d5200 0x7f6a700d1010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:47.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.860+0000 7f6a771e8640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f6a700d1550 0x7f6a700cf200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:47.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.860+0000 7f6a771e8640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f6a700b7d20 con 0x7f6a7015eb90 2024-09-12T03:20:47.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.860+0000 7f6a771e8640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f6a700d42e0 con 0x7f6a700d5200 2024-09-12T03:20:47.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.860+0000 7f6a771e8640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f6a700d3760 con 0x7f6a700d1550 2024-09-12T03:20:47.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.860+0000 7f6a74f84640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f6a7015eb90 0x7f6a700d2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:47.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.860+0000 7f6a74f84640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f6a7015eb90 0x7f6a700d2df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42640/0 (socket says 172.21.15.27:42640) 2024-09-12T03:20:47.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.861+0000 7f6a74f84640 1 -- 172.21.15.27:0/2195858922 learned_addr learned my addr 172.21.15.27:0/2195858922 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:20:47.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.861+0000 7f6a6ffff640 1 --2- 172.21.15.27:0/2195858922 >> v2:172.21.15.45:3300/0 conn(0x7f6a700d5200 0x7f6a700d1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:47.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.861+0000 7f6a75785640 1 --2- 172.21.15.27:0/2195858922 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f6a700d1550 0x7f6a700cf200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:47.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.861+0000 7f6a74f84640 1 -- 172.21.15.27:0/2195858922 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f6a700d1550 msgr2=0x7f6a700cf200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:47.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.861+0000 7f6a74f84640 1 --2- 172.21.15.27:0/2195858922 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f6a700d1550 0x7f6a700cf200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:47.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.861+0000 7f6a74f84640 1 -- 172.21.15.27:0/2195858922 >> v2:172.21.15.45:3300/0 conn(0x7f6a700d5200 msgr2=0x7f6a700d1010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:47.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.861+0000 7f6a74f84640 1 --2- 172.21.15.27:0/2195858922 >> v2:172.21.15.45:3300/0 conn(0x7f6a700d5200 0x7f6a700d1010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:47.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.861+0000 7f6a74f84640 1 -- 172.21.15.27:0/2195858922 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a700d3a40 con 0x7f6a7015eb90 2024-09-12T03:20:47.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.861+0000 7f6a6ffff640 1 --2- 172.21.15.27:0/2195858922 >> v2:172.21.15.45:3300/0 conn(0x7f6a700d5200 0x7f6a700d1010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:20:47.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.861+0000 7f6a75785640 1 --2- 172.21.15.27:0/2195858922 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f6a700d1550 0x7f6a700cf200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:20:47.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.862+0000 7f6a74f84640 1 --2- 172.21.15.27:0/2195858922 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6a7015eb90 0x7f6a700d2df0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f6a700d4750 tx=0x7f6a6802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ca13a4f67734127 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:47.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.862+0000 7f6a6f7fe640 1 -- 172.21.15.27:0/2195858922 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a68036070 con 0x7f6a7015eb90 2024-09-12T03:20:47.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.862+0000 7f6a6f7fe640 1 -- 172.21.15.27:0/2195858922 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a68031600 con 0x7f6a7015eb90 2024-09-12T03:20:47.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.862+0000 7f6a6f7fe640 1 -- 172.21.15.27:0/2195858922 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a680319d0 con 0x7f6a7015eb90 2024-09-12T03:20:47.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.862+0000 7f6a771e8640 1 -- 172.21.15.27:0/2195858922 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6a7015eb90 msgr2=0x7f6a700d2df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:47.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.862+0000 7f6a771e8640 1 --2- 172.21.15.27:0/2195858922 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6a7015eb90 0x7f6a700d2df0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f6a700d4750 tx=0x7f6a6802a6a0 comp rx=0 tx=0).stop 2024-09-12T03:20:47.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.862+0000 7f6a771e8640 1 -- 172.21.15.27:0/2195858922 shutdown_connections 2024-09-12T03:20:47.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.862+0000 7f6a771e8640 1 --2- 172.21.15.27:0/2195858922 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f6a700d1550 0x7f6a700cf200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:47.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.862+0000 7f6a771e8640 1 --2- 172.21.15.27:0/2195858922 >> v2:172.21.15.45:3300/0 conn(0x7f6a700d5200 0x7f6a700d1010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:47.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.862+0000 7f6a771e8640 1 --2- 172.21.15.27:0/2195858922 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6a7015eb90 0x7f6a700d2df0 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:47.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.862+0000 7f6a771e8640 1 -- 172.21.15.27:0/2195858922 >> 172.21.15.27:0/2195858922 conn(0x7f6a7006dce0 msgr2=0x7f6a7006e0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:47.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.862+0000 7f6a771e8640 1 -- 172.21.15.27:0/2195858922 shutdown_connections 2024-09-12T03:20:47.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.862+0000 7f6a771e8640 1 -- 172.21.15.27:0/2195858922 wait complete. 2024-09-12T03:20:47.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.863+0000 7f6a771e8640 1 Processor -- start 2024-09-12T03:20:47.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.864+0000 7f6a771e8640 1 -- start start 2024-09-12T03:20:47.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.864+0000 7f6a771e8640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6a700d1550 0x7f6a700ada20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:47.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.864+0000 7f6a771e8640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6a700d5200 0x7f6a700adf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:47.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.864+0000 7f6a771e8640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f6a7015eb90 0x7f6a700cc390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:47.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.864+0000 7f6a771e8640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f6a700d19a0 con 0x7f6a700d1550 2024-09-12T03:20:47.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.864+0000 7f6a771e8640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f6a700d4000 con 0x7f6a700d5200 2024-09-12T03:20:47.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.864+0000 7f6a771e8640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f6a700d3330 con 0x7f6a7015eb90 2024-09-12T03:20:47.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.864+0000 7f6a74f84640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6a700d1550 0x7f6a700ada20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:47.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.865+0000 7f6a74f84640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6a700d1550 0x7f6a700ada20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42642/0 (socket says 172.21.15.27:42642) 2024-09-12T03:20:47.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.865+0000 7f6a74f84640 1 -- 172.21.15.27:0/3691559618 learned_addr learned my addr 172.21.15.27:0/3691559618 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:20:47.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.865+0000 7f6a6ffff640 1 --2- 172.21.15.27:0/3691559618 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6a700d5200 0x7f6a700adf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:47.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.865+0000 7f6a75785640 1 --2- 172.21.15.27:0/3691559618 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f6a7015eb90 0x7f6a700cc390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:47.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.865+0000 7f6a74f84640 1 -- 172.21.15.27:0/3691559618 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f6a7015eb90 msgr2=0x7f6a700cc390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:47.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.865+0000 7f6a74f84640 1 --2- 172.21.15.27:0/3691559618 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f6a7015eb90 0x7f6a700cc390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:47.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.865+0000 7f6a74f84640 1 -- 172.21.15.27:0/3691559618 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6a700d5200 msgr2=0x7f6a700adf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:47.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.865+0000 7f6a74f84640 1 --2- 172.21.15.27:0/3691559618 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6a700d5200 0x7f6a700adf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:47.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.865+0000 7f6a74f84640 1 -- 172.21.15.27:0/3691559618 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a700cd0a0 con 0x7f6a700d1550 2024-09-12T03:20:47.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.865+0000 7f6a74f84640 1 --2- 172.21.15.27:0/3691559618 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6a700d1550 0x7f6a700ada20 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f6a68000f80 tx=0x7f6a68004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:47.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.865+0000 7f6a6d7fa640 1 -- 172.21.15.27:0/3691559618 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a68036050 con 0x7f6a700d1550 2024-09-12T03:20:47.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.865+0000 7f6a6d7fa640 1 -- 172.21.15.27:0/3691559618 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a68042070 con 0x7f6a700d1550 2024-09-12T03:20:47.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.865+0000 7f6a6d7fa640 1 -- 172.21.15.27:0/3691559618 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a68009510 con 0x7f6a700d1550 2024-09-12T03:20:47.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.865+0000 7f6a771e8640 1 -- 172.21.15.27:0/3691559618 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a700cd380 con 0x7f6a700d1550 2024-09-12T03:20:47.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.865+0000 7f6a771e8640 1 -- 172.21.15.27:0/3691559618 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a7014d0f0 con 0x7f6a700d1550 2024-09-12T03:20:47.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.866+0000 7f6a771e8640 1 -- 172.21.15.27:0/3691559618 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a7015ef70 con 0x7f6a700d1550 2024-09-12T03:20:47.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.867+0000 7f6a6d7fa640 1 -- 172.21.15.27:0/3691559618 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 90600+0+0 (secure 0 0 0) 0x7f6a68007c70 con 0x7f6a700d1550 2024-09-12T03:20:47.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.867+0000 7f6a6d7fa640 1 --2- 172.21.15.27:0/3691559618 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f6a5006bf50 0x7f6a5006e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:47.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.868+0000 7f6a6ffff640 1 --2- 172.21.15.27:0/3691559618 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f6a5006bf50 0x7f6a5006e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:47.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.868+0000 7f6a6d7fa640 1 -- 172.21.15.27:0/3691559618 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5795+0+0 (secure 0 0 0) 0x7f6a680ac020 con 0x7f6a700d1550 2024-09-12T03:20:47.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.868+0000 7f6a6ffff640 1 --2- 172.21.15.27:0/3691559618 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f6a5006bf50 0x7f6a5006e410 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f6a700b7350 tx=0x7f6a60008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:47.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:47.872+0000 7f6a6d7fa640 1 -- 172.21.15.27:0/3691559618 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f6a6807a750 con 0x7f6a700d1550 2024-09-12T03:20:48.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.082+0000 7f6a771e8640 1 -- 172.21.15.27:0/3691559618 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6a700cd550 con 0x7f6a700d1550 2024-09-12T03:20:48.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.082+0000 7f6a6d7fa640 1 -- 172.21.15.27:0/3691559618 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 51 v51) ==== 94+0+5219 (secure 0 0 0) 0x7f6a6807a050 con 0x7f6a700d1550 2024-09-12T03:20:48.083 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:20:48.084 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":51,"btime":"2024-09-12T03:20:43:233119+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5669,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/2036680565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2036680565},{"type":"v1","addr":"172.21.15.45:6834","nonce":2036680565}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":43},{"gid":6094,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/874218126","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":874218126},{"type":"v1","addr":"172.21.15.45:6835","nonce":874218126}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":43},{"gid":6693,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6834/658418980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":658418980},{"type":"v1","addr":"172.21.15.27:6834","nonce":658418980}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":51}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-12T03:20:28.499550+0000","modified":"2024-09-12T03:20:30.510730+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6486},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6486":{"gid":6486,"name":"a","rank":0,"incarnation":44,"state":"up:active","state_seq":3,"addr":"172.21.15.27:6835/3209963418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":3209963418},{"type":"v1","addr":"172.21.15.27:6835","nonce":3209963418}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6486,"qdb_cluster":[6486]},"id":6}]} 2024-09-12T03:20:48.084 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 51 2024-09-12T03:20:48.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.084+0000 7f6a771e8640 1 -- 172.21.15.27:0/3691559618 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f6a5006bf50 msgr2=0x7f6a5006e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:48.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.084+0000 7f6a771e8640 1 --2- 172.21.15.27:0/3691559618 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f6a5006bf50 0x7f6a5006e410 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f6a700b7350 tx=0x7f6a60008040 comp rx=0 tx=0).stop 2024-09-12T03:20:48.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.084+0000 7f6a771e8640 1 -- 172.21.15.27:0/3691559618 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6a700d1550 msgr2=0x7f6a700ada20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:48.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.084+0000 7f6a771e8640 1 --2- 172.21.15.27:0/3691559618 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6a700d1550 0x7f6a700ada20 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f6a68000f80 tx=0x7f6a68004a40 comp rx=0 tx=0).stop 2024-09-12T03:20:48.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.084+0000 7f6a771e8640 1 -- 172.21.15.27:0/3691559618 shutdown_connections 2024-09-12T03:20:48.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.084+0000 7f6a771e8640 1 --2- 172.21.15.27:0/3691559618 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f6a7015eb90 0x7f6a700cc390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:48.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.084+0000 7f6a771e8640 1 --2- 172.21.15.27:0/3691559618 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6a700d5200 0x7f6a700adf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:48.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.084+0000 7f6a771e8640 1 --2- 172.21.15.27:0/3691559618 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f6a5006bf50 0x7f6a5006e410 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:48.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.084+0000 7f6a771e8640 1 --2- 172.21.15.27:0/3691559618 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6a700d1550 0x7f6a700ada20 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:48.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.085+0000 7f6a771e8640 1 -- 172.21.15.27:0/3691559618 >> 172.21.15.27:0/3691559618 conn(0x7f6a7006dce0 msgr2=0x7f6a700ca350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:48.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.085+0000 7f6a771e8640 1 -- 172.21.15.27:0/3691559618 shutdown_connections 2024-09-12T03:20:48.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.085+0000 7f6a771e8640 1 -- 172.21.15.27:0/3691559618 wait complete. 2024-09-12T03:20:48.097 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-12T03:20:48.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.199+0000 7fa3b2f5f640 1 Processor -- start 2024-09-12T03:20:48.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.199+0000 7fa3b2f5f640 1 -- start start 2024-09-12T03:20:48.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.200+0000 7fa3b2f5f640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fa3ac073960 0x7fa3ac162320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:48.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.200+0000 7fa3b2f5f640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa3ac0d2240 con 0x7fa3ac162860 2024-09-12T03:20:48.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.200+0000 7fa3b2f5f640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa3ac163090 con 0x7fa3ac15e5f0 2024-09-12T03:20:48.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.200+0000 7fa3b2f5f640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7fa3ac0d0da0 con 0x7fa3ac073960 2024-09-12T03:20:48.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.200+0000 7fa3a3fff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fa3ac073960 0x7fa3ac162320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:48.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.200+0000 7fa3a3fff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fa3ac073960 0x7fa3ac162320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42652/0 (socket says 172.21.15.27:42652) 2024-09-12T03:20:48.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.200+0000 7fa3b14fc640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fa3ac162860 0x7fa3ac15e9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:47274/0 (socket says 172.21.15.27:47274) 2024-09-12T03:20:48.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.200+0000 7fa3a3fff640 1 -- 172.21.15.27:0/2838804902 learned_addr learned my addr 172.21.15.27:0/2838804902 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:20:48.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.200+0000 7fa3a3fff640 1 -- 172.21.15.27:0/2838804902 >> v1:172.21.15.45:6789/0 conn(0x7fa3ac15e5f0 legacy=0x7fa3ac073250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:48.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.200+0000 7fa3a3fff640 1 -- 172.21.15.27:0/2838804902 >> v1:172.21.15.27:6789/0 conn(0x7fa3ac162860 legacy=0x7fa3ac15e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:48.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.200+0000 7fa3a3fff640 1 -- 172.21.15.27:0/2838804902 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3ac0d0470 con 0x7fa3ac073960 2024-09-12T03:20:48.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.201+0000 7fa3a3fff640 1 --2- 172.21.15.27:0/2838804902 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa3ac073960 0x7fa3ac162320 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7fa39c009870 tx=0x7fa39c02a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db8758d7b799a81c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:48.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.201+0000 7fa3a37fe640 1 -- 172.21.15.27:0/2838804902 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa39c037070 con 0x7fa3ac073960 2024-09-12T03:20:48.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.201+0000 7fa3a37fe640 1 -- 172.21.15.27:0/2838804902 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa39c032440 con 0x7fa3ac073960 2024-09-12T03:20:48.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.201+0000 7fa3a37fe640 1 -- 172.21.15.27:0/2838804902 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa39c032810 con 0x7fa3ac073960 2024-09-12T03:20:48.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.202+0000 7fa3b2f5f640 1 -- 172.21.15.27:0/2838804902 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa3ac073960 msgr2=0x7fa3ac162320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:48.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.202+0000 7fa3b2f5f640 1 --2- 172.21.15.27:0/2838804902 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa3ac073960 0x7fa3ac162320 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7fa39c009870 tx=0x7fa39c02a2a0 comp rx=0 tx=0).stop 2024-09-12T03:20:48.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.202+0000 7fa3b2f5f640 1 -- 172.21.15.27:0/2838804902 shutdown_connections 2024-09-12T03:20:48.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.202+0000 7fa3b2f5f640 1 --2- 172.21.15.27:0/2838804902 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa3ac073960 0x7fa3ac162320 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:48.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.202+0000 7fa3b2f5f640 1 -- 172.21.15.27:0/2838804902 >> 172.21.15.27:0/2838804902 conn(0x7fa3ac06f300 msgr2=0x7fa3ac06f710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:48.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.202+0000 7fa3b2f5f640 1 -- 172.21.15.27:0/2838804902 shutdown_connections 2024-09-12T03:20:48.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.202+0000 7fa3b2f5f640 1 -- 172.21.15.27:0/2838804902 wait complete. 2024-09-12T03:20:48.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.203+0000 7fa3b2f5f640 1 Processor -- start 2024-09-12T03:20:48.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.203+0000 7fa3b2f5f640 1 -- start start 2024-09-12T03:20:48.207 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.204+0000 7fa3b2f5f640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa3ac073960 0x7fa3ac15c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:48.207 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.204+0000 7fa3b2f5f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa3ac15e5f0 0x7fa3ac148e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:48.207 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.204+0000 7fa3b2f5f640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa3ac162860 0x7fa3ac160f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:48.207 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.204+0000 7fa3b2f5f640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fa3ac0b5620 con 0x7fa3ac162860 2024-09-12T03:20:48.207 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.204+0000 7fa3b2f5f640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa3ac0d0a30 con 0x7fa3ac15e5f0 2024-09-12T03:20:48.207 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.204+0000 7fa3b2f5f640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fa3ac162c70 con 0x7fa3ac073960 2024-09-12T03:20:48.207 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.204+0000 7fa3b14fc640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa3ac162860 0x7fa3ac160f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:48.207 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.204+0000 7fa3b14fc640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa3ac162860 0x7fa3ac160f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42662/0 (socket says 172.21.15.27:42662) 2024-09-12T03:20:48.207 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.204+0000 7fa3b14fc640 1 -- 172.21.15.27:0/2867561695 learned_addr learned my addr 172.21.15.27:0/2867561695 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:20:48.208 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.204+0000 7fa3a3fff640 1 --2- 172.21.15.27:0/2867561695 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa3ac15e5f0 0x7fa3ac148e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:48.208 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.204+0000 7fa3b14fc640 1 -- 172.21.15.27:0/2867561695 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa3ac073960 msgr2=0x7fa3ac15c5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:48.208 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.204+0000 7fa3b0cfb640 1 --2- 172.21.15.27:0/2867561695 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa3ac073960 0x7fa3ac15c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:48.208 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.204+0000 7fa3b14fc640 1 --2- 172.21.15.27:0/2867561695 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa3ac073960 0x7fa3ac15c5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:48.208 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.204+0000 7fa3b14fc640 1 -- 172.21.15.27:0/2867561695 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa3ac15e5f0 msgr2=0x7fa3ac148e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:48.208 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.204+0000 7fa3b14fc640 1 --2- 172.21.15.27:0/2867561695 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa3ac15e5f0 0x7fa3ac148e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:48.208 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.204+0000 7fa3b14fc640 1 -- 172.21.15.27:0/2867561695 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3ac149390 con 0x7fa3ac162860 2024-09-12T03:20:48.208 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.204+0000 7fa3a3fff640 1 --2- 172.21.15.27:0/2867561695 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa3ac15e5f0 0x7fa3ac148e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:20:48.208 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.204+0000 7fa3b14fc640 1 --2- 172.21.15.27:0/2867561695 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa3ac162860 0x7fa3ac160f70 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7fa3a400c6b0 tx=0x7fa3a400cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:48.208 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.204+0000 7fa3a17fa640 1 -- 172.21.15.27:0/2867561695 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3a4018070 con 0x7fa3ac162860 2024-09-12T03:20:48.208 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.205+0000 7fa3a17fa640 1 -- 172.21.15.27:0/2867561695 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3a4004da0 con 0x7fa3ac162860 2024-09-12T03:20:48.208 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.205+0000 7fa3b2f5f640 1 -- 172.21.15.27:0/2867561695 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3ac1f4f10 con 0x7fa3ac162860 2024-09-12T03:20:48.208 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.205+0000 7fa3a17fa640 1 -- 172.21.15.27:0/2867561695 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3a400a880 con 0x7fa3ac162860 2024-09-12T03:20:48.209 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.205+0000 7fa3b2f5f640 1 -- 172.21.15.27:0/2867561695 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3ac15bac0 con 0x7fa3ac162860 2024-09-12T03:20:48.209 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.206+0000 7fa3b2f5f640 1 -- 172.21.15.27:0/2867561695 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3ac162320 con 0x7fa3ac162860 2024-09-12T03:20:48.209 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.206+0000 7fa3a17fa640 1 -- 172.21.15.27:0/2867561695 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 90600+0+0 (secure 0 0 0) 0x7fa3a401b070 con 0x7fa3ac162860 2024-09-12T03:20:48.209 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.207+0000 7fa3a17fa640 1 --2- 172.21.15.27:0/2867561695 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fa39806bf50 0x7fa39806e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:48.209 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.207+0000 7fa3a17fa640 1 -- 172.21.15.27:0/2867561695 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5795+0+0 (secure 0 0 0) 0x7fa3a408a570 con 0x7fa3ac162860 2024-09-12T03:20:48.210 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.207+0000 7fa3b0cfb640 1 --2- 172.21.15.27:0/2867561695 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fa39806bf50 0x7fa39806e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:48.210 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.207+0000 7fa3b0cfb640 1 --2- 172.21.15.27:0/2867561695 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fa39806bf50 0x7fa39806e410 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fa3ac15f130 tx=0x7fa3a8009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:48.212 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.211+0000 7fa3a17fa640 1 -- 172.21.15.27:0/2867561695 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fa3a4058e40 con 0x7fa3ac162860 2024-09-12T03:20:48.389 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.388+0000 7fa3b2f5f640 1 -- 172.21.15.27:0/2867561695 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa3ac069790 con 0x7fa3ac162860 2024-09-12T03:20:48.390 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.389+0000 7fa3a17fa640 1 -- 172.21.15.27:0/2867561695 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+16801 (secure 0 0 0) 0x7fa3a4058740 con 0x7fa3ac162860 2024-09-12T03:20:48.391 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:20:48.391 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":89,"fsid":"d2df504e-2647-4385-a5db-9fdf69f487a0","created":"2024-09-12T03:16:07.778937+0000","modified":"2024-09-12T03:20:45.239116+0000","last_up_change":"2024-09-12T03:16:19.314778+0000","last_in_change":"2024-09-12T03:16:09.893086+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T03:16:20.097646+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-09-12T03:20:26.167239+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-09-12T03:20:27.491764+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1851116c-1fb3-4dc9-9772-f90e2ef4c355","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6817","nonce":3002321488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6819","nonce":3002321488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6823","nonce":3002321488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6821","nonce":3002321488}]},"public_addr":"172.21.15.27:6817/3002321488","cluster_addr":"172.21.15.27:6819/3002321488","heartbeat_back_addr":"172.21.15.27:6823/3002321488","heartbeat_front_addr":"172.21.15.27:6821/3002321488","state":["exists","up"]},{"osd":1,"uuid":"f38ae17d-73eb-4204-b99d-975fa806fb87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6809","nonce":1613102886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6811","nonce":1613102886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6815","nonce":1613102886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6813","nonce":1613102886}]},"public_addr":"172.21.15.27:6809/1613102886","cluster_addr":"172.21.15.27:6811/1613102886","heartbeat_back_addr":"172.21.15.27:6815/1613102886","heartbeat_front_addr":"172.21.15.27:6813/1613102886","state":["exists","up"]},{"osd":2,"uuid":"fb12dcc8-bdd0-42b8-b46d-8b10106224bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6825","nonce":3534723355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6827","nonce":3534723355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6831","nonce":3534723355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6829","nonce":3534723355}]},"public_addr":"172.21.15.27:6825/3534723355","cluster_addr":"172.21.15.27:6827/3534723355","heartbeat_back_addr":"172.21.15.27:6831/3534723355","heartbeat_front_addr":"172.21.15.27:6829/3534723355","state":["exists","up"]},{"osd":3,"uuid":"d74d3f49-9c24-4b9f-98e2-3ecd7b5ef3b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6801","nonce":80661598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6803","nonce":80661598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6807","nonce":80661598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6805","nonce":80661598}]},"public_addr":"172.21.15.27:6801/80661598","cluster_addr":"172.21.15.27:6803/80661598","heartbeat_back_addr":"172.21.15.27:6807/80661598","heartbeat_front_addr":"172.21.15.27:6805/80661598","state":["exists","up"]},{"osd":4,"uuid":"2bf123f2-1b0d-41c4-aa3b-805bab267e59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6817","nonce":3943189827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6819","nonce":3943189827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6823","nonce":3943189827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6821","nonce":3943189827}]},"public_addr":"172.21.15.45:6817/3943189827","cluster_addr":"172.21.15.45:6819/3943189827","heartbeat_back_addr":"172.21.15.45:6823/3943189827","heartbeat_front_addr":"172.21.15.45:6821/3943189827","state":["exists","up"]},{"osd":5,"uuid":"c49ba5da-91c6-4a86-a97b-4aba89c253ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6809","nonce":3620760330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6811","nonce":3620760330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6815","nonce":3620760330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6813","nonce":3620760330}]},"public_addr":"172.21.15.45:6809/3620760330","cluster_addr":"172.21.15.45:6811/3620760330","heartbeat_back_addr":"172.21.15.45:6815/3620760330","heartbeat_front_addr":"172.21.15.45:6813/3620760330","state":["exists","up"]},{"osd":6,"uuid":"b0123475-89f7-4a34-9e33-d797b99dd9dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6801","nonce":1989083510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6803","nonce":1989083510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6807","nonce":1989083510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6805","nonce":1989083510}]},"public_addr":"172.21.15.45:6801/1989083510","cluster_addr":"172.21.15.45:6803/1989083510","heartbeat_back_addr":"172.21.15.45:6807/1989083510","heartbeat_front_addr":"172.21.15.45:6805/1989083510","state":["exists","up"]},{"osd":7,"uuid":"4dad773d-b7d8-46fc-8a10-a972aad27b29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6825","nonce":1513300294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6827","nonce":1513300294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6831","nonce":1513300294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6829","nonce":1513300294}]},"public_addr":"172.21.15.45:6825/1513300294","cluster_addr":"172.21.15.45:6827/1513300294","heartbeat_back_addr":"172.21.15.45:6831/1513300294","heartbeat_front_addr":"172.21.15.45:6829/1513300294","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.881855+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:17.133252+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.735339+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.500742+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.477571+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:17.167667+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.470291+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.873743+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.27:6834/1466511159":"2024-09-13T03:20:42.229493+0000","172.21.15.27:6832/1466511159":"2024-09-13T03:20:42.229493+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T03:20:48.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.392+0000 7fa3b2f5f640 1 -- 172.21.15.27:0/2867561695 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fa39806bf50 msgr2=0x7fa39806e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:48.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.392+0000 7fa3b2f5f640 1 --2- 172.21.15.27:0/2867561695 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fa39806bf50 0x7fa39806e410 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fa3ac15f130 tx=0x7fa3a8009040 comp rx=0 tx=0).stop 2024-09-12T03:20:48.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.392+0000 7fa3b2f5f640 1 -- 172.21.15.27:0/2867561695 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa3ac162860 msgr2=0x7fa3ac160f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:48.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.392+0000 7fa3b2f5f640 1 --2- 172.21.15.27:0/2867561695 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa3ac162860 0x7fa3ac160f70 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7fa3a400c6b0 tx=0x7fa3a400cb70 comp rx=0 tx=0).stop 2024-09-12T03:20:48.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.392+0000 7fa3b2f5f640 1 -- 172.21.15.27:0/2867561695 shutdown_connections 2024-09-12T03:20:48.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.392+0000 7fa3b2f5f640 1 --2- 172.21.15.27:0/2867561695 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fa39806bf50 0x7fa39806e410 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:48.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.392+0000 7fa3b2f5f640 1 --2- 172.21.15.27:0/2867561695 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa3ac162860 0x7fa3ac160f70 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:48.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.392+0000 7fa3b2f5f640 1 --2- 172.21.15.27:0/2867561695 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa3ac15e5f0 0x7fa3ac148e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:48.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.392+0000 7fa3b2f5f640 1 --2- 172.21.15.27:0/2867561695 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa3ac073960 0x7fa3ac15c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:48.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.392+0000 7fa3b2f5f640 1 -- 172.21.15.27:0/2867561695 >> 172.21.15.27:0/2867561695 conn(0x7fa3ac06f300 msgr2=0x7fa3ac159c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:48.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.392+0000 7fa3b2f5f640 1 -- 172.21.15.27:0/2867561695 shutdown_connections 2024-09-12T03:20:48.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.392+0000 7fa3b2f5f640 1 -- 172.21.15.27:0/2867561695 wait complete. 2024-09-12T03:20:48.404 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-12T03:20:48.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.506+0000 7fdbb79fd640 1 Processor -- start 2024-09-12T03:20:48.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.506+0000 7fdbb79fd640 1 -- start start 2024-09-12T03:20:48.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.507+0000 7fdbb79fd640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fdbb015b2b0 0x7fdbb00b3560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:48.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.507+0000 7fdbb79fd640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fdbb0072d70 0x7fdbb0072430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:48.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.507+0000 7fdbb79fd640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdbb00ce410 con 0x7fdbb006e9b0 2024-09-12T03:20:48.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.507+0000 7fdbb79fd640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fdbb00720a0 con 0x7fdbb0072d70 2024-09-12T03:20:48.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.507+0000 7fdbb79fd640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fdbb0072970 con 0x7fdbb015b2b0 2024-09-12T03:20:48.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.507+0000 7fdbb5f9a640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fdbb006e9b0 0x7fdbb00cebb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:47278/0 (socket says 172.21.15.27:47278) 2024-09-12T03:20:48.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.507+0000 7fdbb5f9a640 1 -- 172.21.15.27:0/712910057 learned_addr learned my addr 172.21.15.27:0/712910057 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:20:48.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.507+0000 7fdbb4f98640 1 --2- 172.21.15.27:0/712910057 >> v2:172.21.15.45:3300/0 conn(0x7fdbb0072d70 0x7fdbb0072430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:48.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.507+0000 7fdbb5799640 1 --2- 172.21.15.27:0/712910057 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fdbb015b2b0 0x7fdbb00b3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:48.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.507+0000 7fdba7fff640 1 -- 172.21.15.27:0/712910057 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2699717929 0 0) 0x7fdba8001490 con 0x7fdbb006e9b0 2024-09-12T03:20:48.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.507+0000 7fdba7fff640 1 -- 172.21.15.27:0/712910057 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdb9c003520 con 0x7fdbb006e9b0 2024-09-12T03:20:48.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.508+0000 7fdba7fff640 1 -- 172.21.15.27:0/712910057 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 781232819 0 0) 0x7fdb9c003520 con 0x7fdbb006e9b0 2024-09-12T03:20:48.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.508+0000 7fdba7fff640 1 -- 172.21.15.27:0/712910057 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fdbb015b2b0 msgr2=0x7fdbb00b3560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:48.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.508+0000 7fdba7fff640 1 --2- 172.21.15.27:0/712910057 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fdbb015b2b0 0x7fdbb00b3560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:48.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.508+0000 7fdba7fff640 1 -- 172.21.15.27:0/712910057 >> v2:172.21.15.45:3300/0 conn(0x7fdbb0072d70 msgr2=0x7fdbb0072430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:48.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.508+0000 7fdba7fff640 1 --2- 172.21.15.27:0/712910057 >> v2:172.21.15.45:3300/0 conn(0x7fdbb0072d70 0x7fdbb0072430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:48.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.508+0000 7fdba7fff640 1 -- 172.21.15.27:0/712910057 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbb00732c0 con 0x7fdbb006e9b0 2024-09-12T03:20:48.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.508+0000 7fdba7fff640 1 -- 172.21.15.27:0/712910057 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 847755213 0 0) 0x7fdba8001bc0 con 0x7fdbb006e9b0 2024-09-12T03:20:48.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.508+0000 7fdba7fff640 1 -- 172.21.15.27:0/712910057 >> v1:172.21.15.27:6789/0 conn(0x7fdbb006e9b0 legacy=0x7fdbb00cebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:48.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.508+0000 7fdba7fff640 1 --2- 172.21.15.27:0/712910057 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdb9c0039b0 0x7fdb9c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:48.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.508+0000 7fdba7fff640 1 --2- 172.21.15.27:0/712910057 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fdbb015b2b0 0x7fdb9c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:48.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.508+0000 7fdba7fff640 1 --2- 172.21.15.27:0/712910057 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdb9c004d40 0x7fdb9c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:48.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.508+0000 7fdba7fff640 1 -- 172.21.15.27:0/712910057 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fdbb00720a0 con 0x7fdb9c0039b0 2024-09-12T03:20:48.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.508+0000 7fdba7fff640 1 -- 172.21.15.27:0/712910057 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fdbb0072970 con 0x7fdb9c004d40 2024-09-12T03:20:48.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.508+0000 7fdba7fff640 1 -- 172.21.15.27:0/712910057 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fdb9c00b9c0 con 0x7fdbb015b2b0 2024-09-12T03:20:48.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.508+0000 7fdbb4f98640 1 --2- 172.21.15.27:0/712910057 >> v2:172.21.15.45:3300/0 conn(0x7fdbb0072d70 0x7fdbb0072430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:20:48.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.508+0000 7fdbb5799640 1 --2- 172.21.15.27:0/712910057 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdb9c0039b0 0x7fdb9c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:48.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.508+0000 7fdbb5f9a640 1 --2- 172.21.15.27:0/712910057 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdb9c004d40 0x7fdb9c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:48.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.508+0000 7fdbb4f98640 1 --2- 172.21.15.27:0/712910057 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fdbb015b2b0 0x7fdb9c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:48.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.508+0000 7fdbb5799640 1 -- 172.21.15.27:0/712910057 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fdbb015b2b0 msgr2=0x7fdb9c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:48.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.508+0000 7fdbb5799640 1 --2- 172.21.15.27:0/712910057 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fdbb015b2b0 0x7fdb9c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:48.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.508+0000 7fdbb5799640 1 -- 172.21.15.27:0/712910057 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdb9c004d40 msgr2=0x7fdb9c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:48.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.508+0000 7fdbb5799640 1 --2- 172.21.15.27:0/712910057 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdb9c004d40 0x7fdb9c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:48.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.508+0000 7fdbb5799640 1 -- 172.21.15.27:0/712910057 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb9c003520 con 0x7fdb9c0039b0 2024-09-12T03:20:48.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.509+0000 7fdbb5799640 1 --2- 172.21.15.27:0/712910057 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdb9c0039b0 0x7fdb9c003dd0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7fdbac009ac0 tx=0x7fdb9c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df0cfafa1e323132 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:48.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.509+0000 7fdba7fff640 1 -- 172.21.15.27:0/712910057 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbac036070 con 0x7fdb9c0039b0 2024-09-12T03:20:48.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.509+0000 7fdba7fff640 1 -- 172.21.15.27:0/712910057 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdbac031400 con 0x7fdb9c0039b0 2024-09-12T03:20:48.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.509+0000 7fdba7fff640 1 -- 172.21.15.27:0/712910057 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbac009690 con 0x7fdb9c0039b0 2024-09-12T03:20:48.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.510+0000 7fdbb79fd640 1 -- 172.21.15.27:0/712910057 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdb9c0039b0 msgr2=0x7fdb9c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:48.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.510+0000 7fdbb79fd640 1 --2- 172.21.15.27:0/712910057 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdb9c0039b0 0x7fdb9c003dd0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7fdbac009ac0 tx=0x7fdb9c005830 comp rx=0 tx=0).stop 2024-09-12T03:20:48.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.510+0000 7fdbb5f9a640 1 -- 172.21.15.27:0/712910057 reap_dead start 2024-09-12T03:20:48.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.510+0000 7fdbb79fd640 1 -- 172.21.15.27:0/712910057 shutdown_connections 2024-09-12T03:20:48.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.510+0000 7fdbb79fd640 1 -- 172.21.15.27:0/712910057 >> 172.21.15.27:0/712910057 conn(0x7fdbb0062c60 msgr2=0x7fdbb006bb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:48.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.510+0000 7fdbb79fd640 1 -- 172.21.15.27:0/712910057 shutdown_connections 2024-09-12T03:20:48.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.510+0000 7fdbb79fd640 1 -- 172.21.15.27:0/712910057 wait complete. 2024-09-12T03:20:48.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.511+0000 7fdbb79fd640 1 Processor -- start 2024-09-12T03:20:48.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.511+0000 7fdbb79fd640 1 -- start start 2024-09-12T03:20:48.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.512+0000 7fdbb79fd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdb9c0039b0 0x7fdbb00b5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:48.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.512+0000 7fdbb79fd640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fdbb01497e0 0x7fdbb01477d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:48.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.512+0000 7fdbb79fd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdbb0147d10 0x7fdbb014f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:48.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.512+0000 7fdbb79fd640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fdbb00ce690 con 0x7fdb9c0039b0 2024-09-12T03:20:48.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.512+0000 7fdbb79fd640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fdbb0071dc0 con 0x7fdbb0147d10 2024-09-12T03:20:48.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.512+0000 7fdbb79fd640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fdbb015b690 con 0x7fdbb01497e0 2024-09-12T03:20:48.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.512+0000 7fdbb5799640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdb9c0039b0 0x7fdbb00b5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:48.519 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.512+0000 7fdbb5799640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdb9c0039b0 0x7fdbb00b5170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42682/0 (socket says 172.21.15.27:42682) 2024-09-12T03:20:48.519 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.512+0000 7fdbb5799640 1 -- 172.21.15.27:0/3253688140 learned_addr learned my addr 172.21.15.27:0/3253688140 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:20:48.519 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.512+0000 7fdbb5799640 1 -- 172.21.15.27:0/3253688140 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fdbb01497e0 msgr2=0x7fdbb01477d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:48.519 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.512+0000 7fdbb5799640 1 --2- 172.21.15.27:0/3253688140 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fdbb01497e0 0x7fdbb01477d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:48.519 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.512+0000 7fdbb5799640 1 -- 172.21.15.27:0/3253688140 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdbb0147d10 msgr2=0x7fdbb014f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:48.519 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.512+0000 7fdbb5799640 1 --2- 172.21.15.27:0/3253688140 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdbb0147d10 0x7fdbb014f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:48.519 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.512+0000 7fdbb5799640 1 -- 172.21.15.27:0/3253688140 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbb015cd60 con 0x7fdb9c0039b0 2024-09-12T03:20:48.519 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.512+0000 7fdbb5799640 1 --2- 172.21.15.27:0/3253688140 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdb9c0039b0 0x7fdbb00b5170 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fdbac009ac0 tx=0x7fdbac004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:48.519 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.512+0000 7fdba5ffb640 1 -- 172.21.15.27:0/3253688140 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbac040020 con 0x7fdb9c0039b0 2024-09-12T03:20:48.519 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.512+0000 7fdba5ffb640 1 -- 172.21.15.27:0/3253688140 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdbac003bc0 con 0x7fdb9c0039b0 2024-09-12T03:20:48.519 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.512+0000 7fdba5ffb640 1 -- 172.21.15.27:0/3253688140 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbac036040 con 0x7fdb9c0039b0 2024-09-12T03:20:48.519 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.512+0000 7fdbb79fd640 1 -- 172.21.15.27:0/3253688140 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbb015c4c0 con 0x7fdb9c0039b0 2024-09-12T03:20:48.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.513+0000 7fdbb79fd640 1 -- 172.21.15.27:0/3253688140 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbb015d250 con 0x7fdb9c0039b0 2024-09-12T03:20:48.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.514+0000 7fdbb79fd640 1 -- 172.21.15.27:0/3253688140 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdbb015c8b0 con 0x7fdb9c0039b0 2024-09-12T03:20:48.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.514+0000 7fdba5ffb640 1 -- 172.21.15.27:0/3253688140 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 90600+0+0 (secure 0 0 0) 0x7fdbac043020 con 0x7fdb9c0039b0 2024-09-12T03:20:48.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.515+0000 7fdba5ffb640 1 --2- 172.21.15.27:0/3253688140 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fdb9406bf80 0x7fdb9406e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:48.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.515+0000 7fdbb4f98640 1 --2- 172.21.15.27:0/3253688140 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fdb9406bf80 0x7fdb9406e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:48.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.515+0000 7fdba5ffb640 1 -- 172.21.15.27:0/3253688140 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5795+0+0 (secure 0 0 0) 0x7fdbac0ab100 con 0x7fdb9c0039b0 2024-09-12T03:20:48.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.515+0000 7fdbb4f98640 1 --2- 172.21.15.27:0/3253688140 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fdb9406bf80 0x7fdb9406e440 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fdbb0149be0 tx=0x7fdba0008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:48.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.519+0000 7fdba5ffb640 1 -- 172.21.15.27:0/3253688140 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fdbac04a070 con 0x7fdb9c0039b0 2024-09-12T03:20:48.732 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.731+0000 7fdbb79fd640 1 -- 172.21.15.27:0/3253688140 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fdbb0072430 con 0x7fdb9c0039b0 2024-09-12T03:20:48.732 INFO:tasks.ceph.mon.a.smithi027.stderr:2024-09-12T03:20:48.731+0000 7f959bfd9640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-12T03:20:48.733 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.733+0000 7fdba5ffb640 1 -- 172.21.15.27:0/3253688140 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v52) ==== 161+0+0 (secure 0 0 0) 0x7fdbac0044e0 con 0x7fdb9c0039b0 2024-09-12T03:20:48.733 INFO:teuthology.orchestra.run.smithi027.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-12T03:20:48.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.736+0000 7fdbb79fd640 1 -- 172.21.15.27:0/3253688140 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fdb9406bf80 msgr2=0x7fdb9406e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:48.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.736+0000 7fdbb79fd640 1 --2- 172.21.15.27:0/3253688140 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fdb9406bf80 0x7fdb9406e440 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fdbb0149be0 tx=0x7fdba0008040 comp rx=0 tx=0).stop 2024-09-12T03:20:48.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.736+0000 7fdbb79fd640 1 -- 172.21.15.27:0/3253688140 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdb9c0039b0 msgr2=0x7fdbb00b5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:48.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.736+0000 7fdbb79fd640 1 --2- 172.21.15.27:0/3253688140 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdb9c0039b0 0x7fdbb00b5170 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fdbac009ac0 tx=0x7fdbac004730 comp rx=0 tx=0).stop 2024-09-12T03:20:48.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.736+0000 7fdbb79fd640 1 -- 172.21.15.27:0/3253688140 shutdown_connections 2024-09-12T03:20:48.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.736+0000 7fdbb79fd640 1 --2- 172.21.15.27:0/3253688140 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdbb0147d10 0x7fdbb014f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:48.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.736+0000 7fdbb79fd640 1 --2- 172.21.15.27:0/3253688140 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fdbb01497e0 0x7fdbb01477d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:48.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.736+0000 7fdbb79fd640 1 --2- 172.21.15.27:0/3253688140 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fdb9406bf80 0x7fdb9406e440 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:48.738 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.736+0000 7fdbb79fd640 1 --2- 172.21.15.27:0/3253688140 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdb9c0039b0 0x7fdbb00b5170 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:48.738 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.736+0000 7fdbb79fd640 1 -- 172.21.15.27:0/3253688140 >> 172.21.15.27:0/3253688140 conn(0x7fdbb0062c60 msgr2=0x7fdbb00b12b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:48.738 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.736+0000 7fdbb79fd640 1 -- 172.21.15.27:0/3253688140 shutdown_connections 2024-09-12T03:20:48.738 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.736+0000 7fdbb79fd640 1 -- 172.21.15.27:0/3253688140 wait complete. 2024-09-12T03:20:48.738 INFO:tasks.ceph.mds.a.smithi027.stderr: -2602> 2024-09-12T03:20:31.134+0000 7f2b90b90640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-12T03:20:48.749 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-12T03:20:48.766 INFO:tasks.ceph.mds.a.smithi027.stdout:starting mds.a at 2024-09-12T03:20:48.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.854+0000 7f94d4fcf640 1 Processor -- start 2024-09-12T03:20:48.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.854+0000 7f94d4fcf640 1 -- start start 2024-09-12T03:20:48.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.855+0000 7f94d4fcf640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f94d015e990 0x7f94d00d1d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:48.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.855+0000 7f94d4fcf640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f94d00b6d50 0x7f94d00d2f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:48.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.855+0000 7f94d4fcf640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f94d00d2720 con 0x7f94d00b6200 2024-09-12T03:20:48.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.855+0000 7f94d4fcf640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f94d00cf910 con 0x7f94d00b6d50 2024-09-12T03:20:48.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.855+0000 7f94d4fcf640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f94d00cfbf0 con 0x7f94d015e990 2024-09-12T03:20:48.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.855+0000 7f94ce59c640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f94d015e990 0x7f94d00d1d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:48.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.855+0000 7f94ced9d640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f94d00b6d50 0x7f94d00d2f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:48.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.855+0000 7f94ce59c640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f94d015e990 0x7f94d00d1d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3301/0 says I am v2:172.21.15.27:43090/0 (socket says 172.21.15.27:43090) 2024-09-12T03:20:48.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.855+0000 7f94ced9d640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f94d00b6d50 0x7f94d00d2f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.27:49134/0 (socket says 172.21.15.27:49134) 2024-09-12T03:20:48.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.855+0000 7f94cdd9b640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f94d00b6200 0x7f94d00b6640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.27:38252/0 (socket says 172.21.15.27:38252) 2024-09-12T03:20:48.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.855+0000 7f94ce59c640 1 -- 172.21.15.27:0/3623578175 learned_addr learned my addr 172.21.15.27:0/3623578175 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:20:48.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.856+0000 7f94ce59c640 1 -- 172.21.15.27:0/3623578175 >> v2:172.21.15.45:3300/0 conn(0x7f94d00b6d50 msgr2=0x7f94d00d2f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:48.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.856+0000 7f94ce59c640 1 --2- 172.21.15.27:0/3623578175 >> v2:172.21.15.45:3300/0 conn(0x7f94d00b6d50 0x7f94d00d2f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:48.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.856+0000 7f94ce59c640 1 -- 172.21.15.27:0/3623578175 >> v1:172.21.15.45:6789/0 conn(0x7f94d00b6200 legacy=0x7f94d00b6640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:48.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.856+0000 7f94ce59c640 1 -- 172.21.15.27:0/3623578175 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94d00d29e0 con 0x7f94d015e990 2024-09-12T03:20:48.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.856+0000 7f94ced9d640 1 --2- 172.21.15.27:0/3623578175 >> v2:172.21.15.45:3300/0 conn(0x7f94d00b6d50 0x7f94d00d2f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:20:48.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.857+0000 7f94ce59c640 1 --2- 172.21.15.27:0/3623578175 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f94d015e990 0x7f94d00d1d20 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f94c0009870 tx=0x7f94c002a130 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8597c7f3493379d1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:48.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.857+0000 7f94cd59a640 1 -- 172.21.15.27:0/3623578175 <== mon.2 v2:172.21.15.45:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94c0037070 con 0x7f94d015e990 2024-09-12T03:20:48.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.857+0000 7f94cd59a640 1 -- 172.21.15.27:0/3623578175 <== mon.2 v2:172.21.15.45:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f94c0032440 con 0x7f94d015e990 2024-09-12T03:20:48.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.857+0000 7f94cd59a640 1 -- 172.21.15.27:0/3623578175 <== mon.2 v2:172.21.15.45:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94c0032810 con 0x7f94d015e990 2024-09-12T03:20:48.860 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.857+0000 7f94d4fcf640 1 -- 172.21.15.27:0/3623578175 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f94d015e990 msgr2=0x7f94d00d1d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:48.860 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.857+0000 7f94d4fcf640 1 --2- 172.21.15.27:0/3623578175 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f94d015e990 0x7f94d00d1d20 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f94c0009870 tx=0x7f94c002a130 comp rx=0 tx=0).stop 2024-09-12T03:20:48.860 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.857+0000 7f94d4fcf640 1 -- 172.21.15.27:0/3623578175 shutdown_connections 2024-09-12T03:20:48.860 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.857+0000 7f94d4fcf640 1 --2- 172.21.15.27:0/3623578175 >> v2:172.21.15.45:3300/0 conn(0x7f94d00b6d50 0x7f94d00d2f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:48.860 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.857+0000 7f94d4fcf640 1 --2- 172.21.15.27:0/3623578175 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f94d015e990 0x7f94d00d1d20 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:48.860 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.857+0000 7f94d4fcf640 1 -- 172.21.15.27:0/3623578175 >> 172.21.15.27:0/3623578175 conn(0x7f94d00d1200 msgr2=0x7f94d00d1610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:48.860 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.857+0000 7f94d4fcf640 1 -- 172.21.15.27:0/3623578175 shutdown_connections 2024-09-12T03:20:48.860 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.857+0000 7f94d4fcf640 1 -- 172.21.15.27:0/3623578175 wait complete. 2024-09-12T03:20:48.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.859+0000 7f94d4fcf640 1 Processor -- start 2024-09-12T03:20:48.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.859+0000 7f94d4fcf640 1 -- start start 2024-09-12T03:20:48.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.860+0000 7f94d4fcf640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f94d00b6200 0x7f94d01557e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:48.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.860+0000 7f94d4fcf640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f94d00b6d50 0x7f94d0160f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:48.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.860+0000 7f94d4fcf640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f94d015e990 0x7f94d01614b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:48.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.860+0000 7f94d4fcf640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f94d00d0050 con 0x7f94d00b6200 2024-09-12T03:20:48.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.860+0000 7f94d4fcf640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f94d00cfed0 con 0x7f94d015e990 2024-09-12T03:20:48.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.860+0000 7f94d4fcf640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f94d00d3440 con 0x7f94d00b6d50 2024-09-12T03:20:48.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.860+0000 7f94ce59c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f94d00b6200 0x7f94d01557e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:48.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.860+0000 7f94ce59c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f94d00b6200 0x7f94d01557e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42716/0 (socket says 172.21.15.27:42716) 2024-09-12T03:20:48.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.860+0000 7f94ce59c640 1 -- 172.21.15.27:0/3000773837 learned_addr learned my addr 172.21.15.27:0/3000773837 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:20:48.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.860+0000 7f94ced9d640 1 --2- 172.21.15.27:0/3000773837 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f94d015e990 0x7f94d01614b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:48.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.860+0000 7f94cdd9b640 1 --2- 172.21.15.27:0/3000773837 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f94d00b6d50 0x7f94d0160f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:48.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.860+0000 7f94ce59c640 1 -- 172.21.15.27:0/3000773837 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f94d00b6d50 msgr2=0x7f94d0160f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:48.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.860+0000 7f94ce59c640 1 --2- 172.21.15.27:0/3000773837 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f94d00b6d50 0x7f94d0160f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:48.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.860+0000 7f94ce59c640 1 -- 172.21.15.27:0/3000773837 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f94d015e990 msgr2=0x7f94d01614b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:48.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.860+0000 7f94ce59c640 1 --2- 172.21.15.27:0/3000773837 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f94d015e990 0x7f94d01614b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:48.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.860+0000 7f94ce59c640 1 -- 172.21.15.27:0/3000773837 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94d01621b0 con 0x7f94d00b6200 2024-09-12T03:20:48.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.860+0000 7f94ce59c640 1 --2- 172.21.15.27:0/3000773837 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f94d00b6200 0x7f94d01557e0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f94c002a170 tx=0x7f94c00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:48.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.860+0000 7f94beffd640 1 -- 172.21.15.27:0/3000773837 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94c0041030 con 0x7f94d00b6200 2024-09-12T03:20:48.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.860+0000 7f94beffd640 1 -- 172.21.15.27:0/3000773837 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f94c0030440 con 0x7f94d00b6200 2024-09-12T03:20:48.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.860+0000 7f94beffd640 1 -- 172.21.15.27:0/3000773837 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94c0037040 con 0x7f94d00b6200 2024-09-12T03:20:48.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.860+0000 7f94d4fcf640 1 -- 172.21.15.27:0/3000773837 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94d00ca550 con 0x7f94d00b6200 2024-09-12T03:20:48.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.861+0000 7f94d4fcf640 1 -- 172.21.15.27:0/3000773837 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94d00cab10 con 0x7f94d00b6200 2024-09-12T03:20:48.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.862+0000 7f94d4fcf640 1 -- 172.21.15.27:0/3000773837 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94d00d2f00 con 0x7f94d00b6200 2024-09-12T03:20:48.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.862+0000 7f94beffd640 1 -- 172.21.15.27:0/3000773837 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 90600+0+0 (secure 0 0 0) 0x7f94c0044070 con 0x7f94d00b6200 2024-09-12T03:20:48.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.863+0000 7f94beffd640 1 --2- 172.21.15.27:0/3000773837 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f94b006bf50 0x7f94b006e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:48.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.863+0000 7f94cdd9b640 1 --2- 172.21.15.27:0/3000773837 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f94b006bf50 0x7f94b006e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:48.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.863+0000 7f94beffd640 1 -- 172.21.15.27:0/3000773837 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5881+0+0 (secure 0 0 0) 0x7f94c00ab590 con 0x7f94d00b6200 2024-09-12T03:20:48.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.863+0000 7f94cdd9b640 1 --2- 172.21.15.27:0/3000773837 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f94b006bf50 0x7f94b006e410 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f94d0159bd0 tx=0x7f94c4007e80 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:48.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:48.867+0000 7f94beffd640 1 -- 172.21.15.27:0/3000773837 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f94c0079d90 con 0x7f94d00b6200 2024-09-12T03:20:49.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:49.075+0000 7f94d4fcf640 1 -- 172.21.15.27:0/3000773837 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f94d015f110 con 0x7f94d00b6200 2024-09-12T03:20:49.738 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:49.737+0000 7f94beffd640 1 -- 172.21.15.27:0/3000773837 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v53) ==== 104+0+0 (secure 0 0 0) 0x7f94c0079690 con 0x7f94d00b6200 2024-09-12T03:20:49.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:49.740+0000 7f94d4fcf640 1 -- 172.21.15.27:0/3000773837 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f94b006bf50 msgr2=0x7f94b006e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:49.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:49.740+0000 7f94d4fcf640 1 --2- 172.21.15.27:0/3000773837 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f94b006bf50 0x7f94b006e410 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f94d0159bd0 tx=0x7f94c4007e80 comp rx=0 tx=0).stop 2024-09-12T03:20:49.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:49.740+0000 7f94d4fcf640 1 -- 172.21.15.27:0/3000773837 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f94d00b6200 msgr2=0x7f94d01557e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:49.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:49.740+0000 7f94d4fcf640 1 --2- 172.21.15.27:0/3000773837 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f94d00b6200 0x7f94d01557e0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f94c002a170 tx=0x7f94c00046c0 comp rx=0 tx=0).stop 2024-09-12T03:20:49.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:49.740+0000 7f94d4fcf640 1 -- 172.21.15.27:0/3000773837 shutdown_connections 2024-09-12T03:20:49.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:49.740+0000 7f94d4fcf640 1 --2- 172.21.15.27:0/3000773837 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f94d015e990 0x7f94d01614b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:49.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:49.740+0000 7f94d4fcf640 1 --2- 172.21.15.27:0/3000773837 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f94d00b6d50 0x7f94d0160f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:49.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:49.740+0000 7f94d4fcf640 1 --2- 172.21.15.27:0/3000773837 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f94b006bf50 0x7f94b006e410 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:49.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:49.740+0000 7f94d4fcf640 1 --2- 172.21.15.27:0/3000773837 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f94d00b6200 0x7f94d01557e0 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:49.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:49.740+0000 7f94d4fcf640 1 -- 172.21.15.27:0/3000773837 >> 172.21.15.27:0/3000773837 conn(0x7f94d00d1200 msgr2=0x7f94d00b93a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:49.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:49.740+0000 7f94d4fcf640 1 -- 172.21.15.27:0/3000773837 shutdown_connections 2024-09-12T03:20:49.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:49.740+0000 7f94d4fcf640 1 -- 172.21.15.27:0/3000773837 wait complete. 2024-09-12T03:20:49.755 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-12T03:20:49.910 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:49.909+0000 7f2e5c36d640 1 Processor -- start 2024-09-12T03:20:49.910 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:49.909+0000 7f2e5c36d640 1 -- start start 2024-09-12T03:20:49.911 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:49.910+0000 7f2e5c36d640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f2e5415e990 0x7f2e540b7600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:49.911 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:49.910+0000 7f2e5c36d640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f2e540b6230 0x7f2e540b66c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:49.911 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:49.910+0000 7f2e5c36d640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2e5415ed70 con 0x7f2e540b3600 2024-09-12T03:20:49.911 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:49.910+0000 7f2e5c36d640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f2e540d2a30 con 0x7f2e540b6230 2024-09-12T03:20:49.911 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:49.910+0000 7f2e5c36d640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f2e540d2d10 con 0x7f2e5415e990 2024-09-12T03:20:49.912 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:49.910+0000 7f2e59908640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f2e540b6230 0x7f2e540b66c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:49.912 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:49.910+0000 7f2e5a90a640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f2e540b3600 0x7f2e540b3a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:58096/0 (socket says 172.21.15.27:58096) 2024-09-12T03:20:49.912 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:49.910+0000 7f2e5a90a640 1 -- 172.21.15.27:0/841678798 learned_addr learned my addr 172.21.15.27:0/841678798 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:20:49.912 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:49.910+0000 7f2e5a109640 1 --2- 172.21.15.27:0/841678798 >> v2:172.21.15.45:3300/0 conn(0x7f2e5415e990 0x7f2e540b7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:49.912 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:49.910+0000 7f2e59908640 1 -- 172.21.15.27:0/841678798 >> v2:172.21.15.45:3300/0 conn(0x7f2e5415e990 msgr2=0x7f2e540b7600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:49.912 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:49.910+0000 7f2e59908640 1 --2- 172.21.15.27:0/841678798 >> v2:172.21.15.45:3300/0 conn(0x7f2e5415e990 0x7f2e540b7600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:49.912 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:49.910+0000 7f2e59908640 1 -- 172.21.15.27:0/841678798 >> v1:172.21.15.27:6789/0 conn(0x7f2e540b3600 legacy=0x7f2e540b3a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:49.912 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:49.910+0000 7f2e59107640 1 -- 172.21.15.27:0/841678798 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2777228799 0 0) 0x7f2e5415ed70 con 0x7f2e540b3600 2024-09-12T03:20:49.912 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:49.910+0000 7f2e59908640 1 -- 172.21.15.27:0/841678798 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e540d2ff0 con 0x7f2e540b6230 2024-09-12T03:20:49.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:49.911+0000 7f2e59908640 1 --2- 172.21.15.27:0/841678798 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2e540b6230 0x7f2e540b66c0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f2e540d2270 tx=0x7f2e5002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=501e513b2c5604d5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:49.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:49.911+0000 7f2e59107640 1 -- 172.21.15.27:0/841678798 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e50037070 con 0x7f2e540b6230 2024-09-12T03:20:49.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:49.911+0000 7f2e59107640 1 -- 172.21.15.27:0/841678798 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e50032440 con 0x7f2e540b6230 2024-09-12T03:20:49.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:49.911+0000 7f2e59107640 1 -- 172.21.15.27:0/841678798 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e50032810 con 0x7f2e540b6230 2024-09-12T03:20:49.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:49.912+0000 7f2e5c36d640 1 -- 172.21.15.27:0/841678798 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2e540b6230 msgr2=0x7f2e540b66c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:49.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:49.912+0000 7f2e5c36d640 1 --2- 172.21.15.27:0/841678798 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2e540b6230 0x7f2e540b66c0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f2e540d2270 tx=0x7f2e5002a1a0 comp rx=0 tx=0).stop 2024-09-12T03:20:49.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:49.912+0000 7f2e5c36d640 1 -- 172.21.15.27:0/841678798 shutdown_connections 2024-09-12T03:20:49.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:49.912+0000 7f2e5c36d640 1 --2- 172.21.15.27:0/841678798 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2e540b6230 0x7f2e540b66c0 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:49.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:49.912+0000 7f2e5c36d640 1 --2- 172.21.15.27:0/841678798 >> v2:172.21.15.45:3300/0 conn(0x7f2e5415e990 0x7f2e540b7600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:49.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:49.912+0000 7f2e5c36d640 1 -- 172.21.15.27:0/841678798 >> 172.21.15.27:0/841678798 conn(0x7f2e540d1230 msgr2=0x7f2e540d1640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:49.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:49.912+0000 7f2e5c36d640 1 -- 172.21.15.27:0/841678798 shutdown_connections 2024-09-12T03:20:49.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:49.912+0000 7f2e5c36d640 1 -- 172.21.15.27:0/841678798 wait complete. 2024-09-12T03:20:49.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:49.913+0000 7f2e5c36d640 1 Processor -- start 2024-09-12T03:20:49.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:49.913+0000 7f2e5c36d640 1 -- start start 2024-09-12T03:20:49.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:49.914+0000 7f2e5c36d640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2e540b3600 0x7f2e540cb880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:49.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:49.914+0000 7f2e5c36d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2e540b6230 0x7f2e540cbdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:49.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:49.914+0000 7f2e5c36d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2e5415e990 0x7f2e540cd990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:49.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:49.914+0000 7f2e5c36d640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f2e540b7b40 con 0x7f2e540b6230 2024-09-12T03:20:49.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:49.914+0000 7f2e5c36d640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f2e540d35b0 con 0x7f2e5415e990 2024-09-12T03:20:49.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:49.914+0000 7f2e5c36d640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f2e540b7cc0 con 0x7f2e540b3600 2024-09-12T03:20:49.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:49.914+0000 7f2e59908640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2e540b6230 0x7f2e540cbdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:49.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:49.914+0000 7f2e59908640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2e540b6230 0x7f2e540cbdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54904/0 (socket says 172.21.15.27:54904) 2024-09-12T03:20:49.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:49.914+0000 7f2e59908640 1 -- 172.21.15.27:0/3798992599 learned_addr learned my addr 172.21.15.27:0/3798992599 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:20:49.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:49.914+0000 7f2e5a90a640 1 --2- 172.21.15.27:0/3798992599 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2e5415e990 0x7f2e540cd990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:49.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:49.914+0000 7f2e5a109640 1 --2- 172.21.15.27:0/3798992599 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2e540b3600 0x7f2e540cb880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:49.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:49.914+0000 7f2e59908640 1 -- 172.21.15.27:0/3798992599 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2e540b3600 msgr2=0x7f2e540cb880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:49.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:49.914+0000 7f2e59908640 1 --2- 172.21.15.27:0/3798992599 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2e540b3600 0x7f2e540cb880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:49.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:49.914+0000 7f2e59908640 1 -- 172.21.15.27:0/3798992599 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2e5415e990 msgr2=0x7f2e540cd990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:49.918 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:49.914+0000 7f2e59908640 1 --2- 172.21.15.27:0/3798992599 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2e5415e990 0x7f2e540cd990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:49.918 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:49.914+0000 7f2e59908640 1 -- 172.21.15.27:0/3798992599 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e540ce710 con 0x7f2e540b6230 2024-09-12T03:20:49.918 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:49.914+0000 7f2e59908640 1 --2- 172.21.15.27:0/3798992599 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2e540b6230 0x7f2e540cbdc0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f2e50000c00 tx=0x7f2e500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:49.918 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:49.915+0000 7f2e46ffd640 1 -- 172.21.15.27:0/3798992599 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e50037050 con 0x7f2e540b6230 2024-09-12T03:20:49.918 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:49.915+0000 7f2e46ffd640 1 -- 172.21.15.27:0/3798992599 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e50038040 con 0x7f2e540b6230 2024-09-12T03:20:49.918 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:49.915+0000 7f2e5c36d640 1 -- 172.21.15.27:0/3798992599 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e5415ac90 con 0x7f2e540b6230 2024-09-12T03:20:49.918 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:49.915+0000 7f2e46ffd640 1 -- 172.21.15.27:0/3798992599 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e50004450 con 0x7f2e540b6230 2024-09-12T03:20:49.918 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:49.915+0000 7f2e5c36d640 1 -- 172.21.15.27:0/3798992599 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e540c7ea0 con 0x7f2e540b6230 2024-09-12T03:20:49.918 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:49.916+0000 7f2e5c36d640 1 -- 172.21.15.27:0/3798992599 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e540b7600 con 0x7f2e540b6230 2024-09-12T03:20:49.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:49.916+0000 7f2e46ffd640 1 -- 172.21.15.27:0/3798992599 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 90600+0+0 (secure 0 0 0) 0x7f2e50030980 con 0x7f2e540b6230 2024-09-12T03:20:49.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:49.917+0000 7f2e46ffd640 1 --2- 172.21.15.27:0/3798992599 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f2e3806bf50 0x7f2e3806e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:49.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:49.917+0000 7f2e46ffd640 1 -- 172.21.15.27:0/3798992599 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5881+0+0 (secure 0 0 0) 0x7f2e500abd50 con 0x7f2e540b6230 2024-09-12T03:20:49.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:49.917+0000 7f2e5a109640 1 --2- 172.21.15.27:0/3798992599 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f2e3806bf50 0x7f2e3806e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:49.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:49.917+0000 7f2e5a109640 1 --2- 172.21.15.27:0/3798992599 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f2e3806bf50 0x7f2e3806e410 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f2e48006fd0 tx=0x7f2e48008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:49.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:49.921+0000 7f2e46ffd640 1 -- 172.21.15.27:0/3798992599 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f2e500af030 con 0x7f2e540b6230 2024-09-12T03:20:50.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:50.096+0000 7f2e5c36d640 1 -- 172.21.15.27:0/3798992599 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f2e5415aa60 con 0x7f2e540b6230 2024-09-12T03:20:50.740 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:50.739+0000 7f2e46ffd640 1 -- 172.21.15.27:0/3798992599 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v91) ==== 181+0+0 (secure 0 0 0) 0x7f2e5007a660 con 0x7f2e540b6230 2024-09-12T03:20:50.740 INFO:teuthology.orchestra.run.smithi027.stderr:pool 'cephfs_metadata' removed 2024-09-12T03:20:50.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:50.742+0000 7f2e5c36d640 1 -- 172.21.15.27:0/3798992599 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f2e3806bf50 msgr2=0x7f2e3806e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:50.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:50.742+0000 7f2e5c36d640 1 --2- 172.21.15.27:0/3798992599 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f2e3806bf50 0x7f2e3806e410 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f2e48006fd0 tx=0x7f2e48008040 comp rx=0 tx=0).stop 2024-09-12T03:20:50.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:50.742+0000 7f2e5c36d640 1 -- 172.21.15.27:0/3798992599 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2e540b6230 msgr2=0x7f2e540cbdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:50.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:50.742+0000 7f2e5c36d640 1 --2- 172.21.15.27:0/3798992599 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2e540b6230 0x7f2e540cbdc0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f2e50000c00 tx=0x7f2e500046c0 comp rx=0 tx=0).stop 2024-09-12T03:20:50.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:50.742+0000 7f2e5c36d640 1 -- 172.21.15.27:0/3798992599 shutdown_connections 2024-09-12T03:20:50.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:50.742+0000 7f2e5c36d640 1 --2- 172.21.15.27:0/3798992599 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2e5415e990 0x7f2e540cd990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:50.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:50.743+0000 7f2e5c36d640 1 --2- 172.21.15.27:0/3798992599 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f2e3806bf50 0x7f2e3806e410 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:50.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:50.743+0000 7f2e5c36d640 1 --2- 172.21.15.27:0/3798992599 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2e540b6230 0x7f2e540cbdc0 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:50.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:50.743+0000 7f2e5c36d640 1 --2- 172.21.15.27:0/3798992599 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2e540b3600 0x7f2e540cb880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:50.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:50.743+0000 7f2e5c36d640 1 -- 172.21.15.27:0/3798992599 >> 172.21.15.27:0/3798992599 conn(0x7f2e540d1230 msgr2=0x7f2e540d4b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:50.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:50.743+0000 7f2e5c36d640 1 -- 172.21.15.27:0/3798992599 shutdown_connections 2024-09-12T03:20:50.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:50.743+0000 7f2e5c36d640 1 -- 172.21.15.27:0/3798992599 wait complete. 2024-09-12T03:20:50.758 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-12T03:20:50.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:50.869+0000 7fb81288d640 1 Processor -- start 2024-09-12T03:20:50.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:50.869+0000 7fb81288d640 1 -- start start 2024-09-12T03:20:50.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:50.870+0000 7fb81288d640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb80c15e650 0x7fb80c0d1d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:50.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:50.870+0000 7fb81288d640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fb80c06db30 0x7fb80c06df80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:50.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:50.870+0000 7fb81288d640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb80c0d2280 con 0x7fb80c06cc40 2024-09-12T03:20:50.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:50.870+0000 7fb81288d640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7fb80c06d850 con 0x7fb80c06db30 2024-09-12T03:20:50.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:50.870+0000 7fb81288d640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fb80c0d1260 con 0x7fb80c15e650 2024-09-12T03:20:50.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:50.870+0000 7fb810e2a640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fb80c06db30 0x7fb80c06df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:50.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:50.870+0000 7fb810e2a640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fb80c06db30 0x7fb80c06df80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54910/0 (socket says 172.21.15.27:54910) 2024-09-12T03:20:50.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:50.870+0000 7fb810e2a640 1 -- 172.21.15.27:0/110466066 learned_addr learned my addr 172.21.15.27:0/110466066 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:20:50.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:50.870+0000 7fb80bfff640 1 --2- 172.21.15.27:0/110466066 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb80c15e650 0x7fb80c0d1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:50.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:50.870+0000 7fb810e2a640 1 -- 172.21.15.27:0/110466066 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb80c15e650 msgr2=0x7fb80c0d1d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:50.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:50.870+0000 7fb810e2a640 1 --2- 172.21.15.27:0/110466066 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb80c15e650 0x7fb80c0d1d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:50.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:50.870+0000 7fb810e2a640 1 -- 172.21.15.27:0/110466066 >> v1:172.21.15.45:6789/0 conn(0x7fb80c06cc40 legacy=0x7fb80c06d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:50.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:50.870+0000 7fb810e2a640 1 -- 172.21.15.27:0/110466066 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb80c0d1540 con 0x7fb80c06db30 2024-09-12T03:20:50.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:50.871+0000 7fb810e2a640 1 --2- 172.21.15.27:0/110466066 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb80c06db30 0x7fb80c06df80 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7fb804009e50 tx=0x7fb80402a630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fa18d36aff6cbf server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:50.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:50.871+0000 7fb80affd640 1 -- 172.21.15.27:0/110466066 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb804036070 con 0x7fb80c06db30 2024-09-12T03:20:50.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:50.871+0000 7fb80affd640 1 -- 172.21.15.27:0/110466066 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb804031600 con 0x7fb80c06db30 2024-09-12T03:20:50.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:50.871+0000 7fb80affd640 1 -- 172.21.15.27:0/110466066 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8040319d0 con 0x7fb80c06db30 2024-09-12T03:20:50.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:50.872+0000 7fb81288d640 1 -- 172.21.15.27:0/110466066 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb80c06db30 msgr2=0x7fb80c06df80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:50.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:50.872+0000 7fb81288d640 1 --2- 172.21.15.27:0/110466066 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb80c06db30 0x7fb80c06df80 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7fb804009e50 tx=0x7fb80402a630 comp rx=0 tx=0).stop 2024-09-12T03:20:50.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:50.872+0000 7fb81288d640 1 -- 172.21.15.27:0/110466066 shutdown_connections 2024-09-12T03:20:50.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:50.872+0000 7fb81288d640 1 --2- 172.21.15.27:0/110466066 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb80c06db30 0x7fb80c06df80 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:50.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:50.872+0000 7fb81288d640 1 --2- 172.21.15.27:0/110466066 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb80c15e650 0x7fb80c0d1d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:50.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:50.872+0000 7fb81288d640 1 -- 172.21.15.27:0/110466066 >> 172.21.15.27:0/110466066 conn(0x7fb80c06f320 msgr2=0x7fb80c06f700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:50.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:50.872+0000 7fb81288d640 1 -- 172.21.15.27:0/110466066 shutdown_connections 2024-09-12T03:20:50.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:50.872+0000 7fb81288d640 1 -- 172.21.15.27:0/110466066 wait complete. 2024-09-12T03:20:50.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:50.873+0000 7fb81288d640 1 Processor -- start 2024-09-12T03:20:50.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:50.873+0000 7fb81288d640 1 -- start start 2024-09-12T03:20:50.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:50.874+0000 7fb81288d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb80c06cc40 0x7fb80c161910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:50.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:50.874+0000 7fb81288d640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb80c06db30 0x7fb80c0cb930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:50.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:50.874+0000 7fb81288d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb80c15e650 0x7fb80c14c210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:50.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:50.874+0000 7fb81288d640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fb80c06e640 con 0x7fb80c06cc40 2024-09-12T03:20:50.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:50.874+0000 7fb81288d640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb80c06e4c0 con 0x7fb80c15e650 2024-09-12T03:20:50.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:50.874+0000 7fb81288d640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fb80c0b55b0 con 0x7fb80c06db30 2024-09-12T03:20:50.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:50.874+0000 7fb80bfff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb80c06cc40 0x7fb80c161910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:50.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:50.874+0000 7fb80bfff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb80c06cc40 0x7fb80c161910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54926/0 (socket says 172.21.15.27:54926) 2024-09-12T03:20:50.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:50.874+0000 7fb80bfff640 1 -- 172.21.15.27:0/954494244 learned_addr learned my addr 172.21.15.27:0/954494244 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:20:50.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:50.874+0000 7fb810e2a640 1 --2- 172.21.15.27:0/954494244 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb80c15e650 0x7fb80c14c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:50.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:50.874+0000 7fb80bfff640 1 -- 172.21.15.27:0/954494244 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb80c06db30 msgr2=0x7fb80c0cb930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:50.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:50.874+0000 7fb80bfff640 1 --2- 172.21.15.27:0/954494244 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb80c06db30 0x7fb80c0cb930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:50.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:50.874+0000 7fb80bfff640 1 -- 172.21.15.27:0/954494244 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb80c15e650 msgr2=0x7fb80c14c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:50.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:50.874+0000 7fb80bfff640 1 --2- 172.21.15.27:0/954494244 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb80c15e650 0x7fb80c14c210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:50.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:50.874+0000 7fb80bfff640 1 -- 172.21.15.27:0/954494244 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb80c15b8f0 con 0x7fb80c06cc40 2024-09-12T03:20:50.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:50.875+0000 7fb80bfff640 1 --2- 172.21.15.27:0/954494244 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb80c06cc40 0x7fb80c161910 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fb7fc00d5f0 tx=0x7fb7fc00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:50.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:50.875+0000 7fb808ff9640 1 -- 172.21.15.27:0/954494244 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7fc0125f0 con 0x7fb80c06cc40 2024-09-12T03:20:50.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:50.875+0000 7fb808ff9640 1 -- 172.21.15.27:0/954494244 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7fc019070 con 0x7fb80c06cc40 2024-09-12T03:20:50.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:50.875+0000 7fb81288d640 1 -- 172.21.15.27:0/954494244 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb80c14c750 con 0x7fb80c06cc40 2024-09-12T03:20:50.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:50.875+0000 7fb808ff9640 1 -- 172.21.15.27:0/954494244 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7fc0129d0 con 0x7fb80c06cc40 2024-09-12T03:20:50.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:50.875+0000 7fb81288d640 1 -- 172.21.15.27:0/954494244 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb80c0cc2a0 con 0x7fb80c06cc40 2024-09-12T03:20:50.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:50.876+0000 7fb81288d640 1 -- 172.21.15.27:0/954494244 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb80c06df80 con 0x7fb80c06cc40 2024-09-12T03:20:50.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:50.876+0000 7fb808ff9640 1 -- 172.21.15.27:0/954494244 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 90600+0+0 (secure 0 0 0) 0x7fb7fc012b70 con 0x7fb80c06cc40 2024-09-12T03:20:50.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:50.877+0000 7fb808ff9640 1 --2- 172.21.15.27:0/954494244 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fb7f806bf50 0x7fb7f806e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:50.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:50.877+0000 7fb808ff9640 1 -- 172.21.15.27:0/954494244 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5443+0+0 (secure 0 0 0) 0x7fb7fc08b260 con 0x7fb80c06cc40 2024-09-12T03:20:50.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:50.877+0000 7fb80b7fe640 1 --2- 172.21.15.27:0/954494244 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fb7f806bf50 0x7fb7f806e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:50.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:50.877+0000 7fb80b7fe640 1 --2- 172.21.15.27:0/954494244 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fb7f806bf50 0x7fb7f806e410 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fb80c160930 tx=0x7fb8000079d0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:50.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:50.881+0000 7fb808ff9640 1 -- 172.21.15.27:0/954494244 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fb80c15b8f0 con 0x7fb80c06cc40 2024-09-12T03:20:51.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:51.056+0000 7fb81288d640 1 -- 172.21.15.27:0/954494244 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fb80c0b4db0 con 0x7fb80c06cc40 2024-09-12T03:20:51.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:51.745+0000 7fb808ff9640 1 -- 172.21.15.27:0/954494244 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v92) ==== 169+0+0 (secure 0 0 0) 0x7fb7fc059c90 con 0x7fb80c06cc40 2024-09-12T03:20:51.746 INFO:teuthology.orchestra.run.smithi027.stderr:pool 'cephfs_data' removed 2024-09-12T03:20:51.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:51.748+0000 7fb81288d640 1 -- 172.21.15.27:0/954494244 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fb7f806bf50 msgr2=0x7fb7f806e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:51.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:51.748+0000 7fb81288d640 1 --2- 172.21.15.27:0/954494244 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fb7f806bf50 0x7fb7f806e410 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fb80c160930 tx=0x7fb8000079d0 comp rx=0 tx=0).stop 2024-09-12T03:20:51.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:51.748+0000 7fb81288d640 1 -- 172.21.15.27:0/954494244 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb80c06cc40 msgr2=0x7fb80c161910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:51.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:51.748+0000 7fb81288d640 1 --2- 172.21.15.27:0/954494244 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb80c06cc40 0x7fb80c161910 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fb7fc00d5f0 tx=0x7fb7fc00dab0 comp rx=0 tx=0).stop 2024-09-12T03:20:51.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:51.749+0000 7fb81288d640 1 -- 172.21.15.27:0/954494244 shutdown_connections 2024-09-12T03:20:51.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:51.749+0000 7fb81288d640 1 --2- 172.21.15.27:0/954494244 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb80c15e650 0x7fb80c14c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:51.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:51.749+0000 7fb81288d640 1 --2- 172.21.15.27:0/954494244 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb80c06db30 0x7fb80c0cb930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:51.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:51.749+0000 7fb81288d640 1 --2- 172.21.15.27:0/954494244 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fb7f806bf50 0x7fb7f806e410 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:51.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:51.749+0000 7fb81288d640 1 --2- 172.21.15.27:0/954494244 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb80c06cc40 0x7fb80c161910 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:51.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:51.749+0000 7fb81288d640 1 -- 172.21.15.27:0/954494244 >> 172.21.15.27:0/954494244 conn(0x7fb80c06f320 msgr2=0x7fb80c15bcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:51.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:51.749+0000 7fb81288d640 1 -- 172.21.15.27:0/954494244 shutdown_connections 2024-09-12T03:20:51.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:51.749+0000 7fb81288d640 1 -- 172.21.15.27:0/954494244 wait complete. 2024-09-12T03:20:51.764 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-09-12T03:20:51.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:51.870+0000 7f5ebd8dd640 1 Processor -- start 2024-09-12T03:20:51.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:51.870+0000 7f5ebd8dd640 1 -- start start 2024-09-12T03:20:51.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:51.871+0000 7f5ebd8dd640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f5eb806cc40 0x7f5eb806d060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:51.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:51.871+0000 7f5ebd8dd640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5eb806e690 con 0x7f5eb815e650 2024-09-12T03:20:51.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:51.871+0000 7f5ebd8dd640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5eb806d5a0 con 0x7f5eb806db30 2024-09-12T03:20:51.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:51.871+0000 7f5ebd8dd640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f5eb806d850 con 0x7f5eb806cc40 2024-09-12T03:20:51.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:51.871+0000 7f5eb6ffd640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f5eb815e650 0x7f5eb80d1d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:58110/0 (socket says 172.21.15.27:58110) 2024-09-12T03:20:51.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:51.871+0000 7f5eb6ffd640 1 -- 172.21.15.27:0/1107912927 learned_addr learned my addr 172.21.15.27:0/1107912927 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:20:51.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:51.871+0000 7f5eb5ffb640 1 -- 172.21.15.27:0/1107912927 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2770200662 0 0) 0x7f5eb806e690 con 0x7f5eb815e650 2024-09-12T03:20:51.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:51.871+0000 7f5eb67fc640 1 --2- 172.21.15.27:0/1107912927 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f5eb806cc40 0x7f5eb806d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:51.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:51.871+0000 7f5eb5ffb640 1 -- 172.21.15.27:0/1107912927 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5ea0003520 con 0x7f5eb815e650 2024-09-12T03:20:51.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:51.871+0000 7f5eb5ffb640 1 -- 172.21.15.27:0/1107912927 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3272910067 0 0) 0x7f5ea0003520 con 0x7f5eb815e650 2024-09-12T03:20:51.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:51.871+0000 7f5eb5ffb640 1 -- 172.21.15.27:0/1107912927 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f5eb806cc40 msgr2=0x7f5eb806d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:51.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:51.871+0000 7f5eb5ffb640 1 --2- 172.21.15.27:0/1107912927 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f5eb806cc40 0x7f5eb806d060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:51.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:51.871+0000 7f5eb5ffb640 1 -- 172.21.15.27:0/1107912927 >> v1:172.21.15.45:6789/0 conn(0x7f5eb806db30 legacy=0x7f5eb806df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:51.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:51.871+0000 7f5eb5ffb640 1 -- 172.21.15.27:0/1107912927 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5eb80d1260 con 0x7f5eb815e650 2024-09-12T03:20:51.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:51.871+0000 7f5eb5ffb640 1 -- 172.21.15.27:0/1107912927 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 847755213 0 0) 0x7f5ea8001690 con 0x7f5eb815e650 2024-09-12T03:20:51.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:51.871+0000 7f5eb5ffb640 1 -- 172.21.15.27:0/1107912927 >> v1:172.21.15.27:6789/0 conn(0x7f5eb815e650 legacy=0x7f5eb80d1d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:51.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:51.871+0000 7f5eb5ffb640 1 --2- 172.21.15.27:0/1107912927 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ea00039e0 0x7f5ea0003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:51.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:51.871+0000 7f5eb5ffb640 1 --2- 172.21.15.27:0/1107912927 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f5eb806cc40 0x7f5ea0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:51.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:51.871+0000 7f5eb5ffb640 1 --2- 172.21.15.27:0/1107912927 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5ea0004d40 0x7f5ea00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:51.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:51.872+0000 7f5eb5ffb640 1 -- 172.21.15.27:0/1107912927 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f5eb806d850 con 0x7f5ea0004d40 2024-09-12T03:20:51.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:51.872+0000 7f5eb5ffb640 1 -- 172.21.15.27:0/1107912927 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f5ea000bb20 con 0x7f5ea00039e0 2024-09-12T03:20:51.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:51.872+0000 7f5eb5ffb640 1 -- 172.21.15.27:0/1107912927 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f5ea000bd20 con 0x7f5eb806cc40 2024-09-12T03:20:51.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:51.872+0000 7f5eb77fe640 1 --2- 172.21.15.27:0/1107912927 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5ea0004d40 0x7f5ea00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:51.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:51.872+0000 7f5eb67fc640 1 --2- 172.21.15.27:0/1107912927 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f5eb806cc40 0x7f5ea0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:51.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:51.872+0000 7f5eb6ffd640 1 --2- 172.21.15.27:0/1107912927 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ea00039e0 0x7f5ea0003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:51.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:51.872+0000 7f5eb77fe640 1 -- 172.21.15.27:0/1107912927 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f5eb806cc40 msgr2=0x7f5ea0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:51.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:51.872+0000 7f5eb77fe640 1 --2- 172.21.15.27:0/1107912927 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f5eb806cc40 0x7f5ea0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:51.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:51.872+0000 7f5eb77fe640 1 -- 172.21.15.27:0/1107912927 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ea00039e0 msgr2=0x7f5ea0003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:51.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:51.872+0000 7f5eb77fe640 1 --2- 172.21.15.27:0/1107912927 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ea00039e0 0x7f5ea0003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:51.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:51.872+0000 7f5eb77fe640 1 -- 172.21.15.27:0/1107912927 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ea0003520 con 0x7f5ea0004d40 2024-09-12T03:20:51.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:51.872+0000 7f5eb6ffd640 1 --2- 172.21.15.27:0/1107912927 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ea00039e0 0x7f5ea0003e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:20:51.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:51.873+0000 7f5eb77fe640 1 --2- 172.21.15.27:0/1107912927 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5ea0004d40 0x7f5ea00051c0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f5eb0001e50 tx=0x7f5ea0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a93afe059b0cf8de server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:51.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:51.873+0000 7f5eb5ffb640 1 -- 172.21.15.27:0/1107912927 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eb0001a70 con 0x7f5ea0004d40 2024-09-12T03:20:51.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:51.873+0000 7f5eb5ffb640 1 -- 172.21.15.27:0/1107912927 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5eb0003970 con 0x7f5ea0004d40 2024-09-12T03:20:51.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:51.873+0000 7f5eb5ffb640 1 -- 172.21.15.27:0/1107912927 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eb0003d60 con 0x7f5ea0004d40 2024-09-12T03:20:51.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:51.873+0000 7f5ebd8dd640 1 -- 172.21.15.27:0/1107912927 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5ea0004d40 msgr2=0x7f5ea00051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:51.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:51.873+0000 7f5ebd8dd640 1 --2- 172.21.15.27:0/1107912927 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5ea0004d40 0x7f5ea00051c0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f5eb0001e50 tx=0x7f5ea0005830 comp rx=0 tx=0).stop 2024-09-12T03:20:51.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:51.873+0000 7f5eb77fe640 1 -- 172.21.15.27:0/1107912927 reap_dead start 2024-09-12T03:20:51.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:51.873+0000 7f5ebd8dd640 1 -- 172.21.15.27:0/1107912927 shutdown_connections 2024-09-12T03:20:51.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:51.873+0000 7f5ebd8dd640 1 -- 172.21.15.27:0/1107912927 >> 172.21.15.27:0/1107912927 conn(0x7f5eb806f320 msgr2=0x7f5eb806f700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:51.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:51.873+0000 7f5ebd8dd640 1 -- 172.21.15.27:0/1107912927 shutdown_connections 2024-09-12T03:20:51.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:51.873+0000 7f5ebd8dd640 1 -- 172.21.15.27:0/1107912927 wait complete. 2024-09-12T03:20:51.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:51.874+0000 7f5ebd8dd640 1 Processor -- start 2024-09-12T03:20:51.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:51.875+0000 7f5ebd8dd640 1 -- start start 2024-09-12T03:20:51.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:51.875+0000 7f5ebd8dd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5ea0004d40 0x7f5eb814bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:51.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:51.875+0000 7f5ebd8dd640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f5eb8149140 0x7f5eb80b3b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:51.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:51.875+0000 7f5ebd8dd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5eb814a750 0x7f5eb814ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:51.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:51.875+0000 7f5ebd8dd640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f5eb815ebb0 con 0x7f5ea0004d40 2024-09-12T03:20:51.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:51.875+0000 7f5ebd8dd640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f5eb815ea30 con 0x7f5eb814a750 2024-09-12T03:20:51.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:51.875+0000 7f5ebd8dd640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f5eb806d060 con 0x7f5eb8149140 2024-09-12T03:20:51.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:51.876+0000 7f5eb6ffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5ea0004d40 0x7f5eb814bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:51.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:51.876+0000 7f5eb6ffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5ea0004d40 0x7f5eb814bbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54942/0 (socket says 172.21.15.27:54942) 2024-09-12T03:20:51.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:51.876+0000 7f5eb6ffd640 1 -- 172.21.15.27:0/741925188 learned_addr learned my addr 172.21.15.27:0/741925188 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:20:51.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:51.876+0000 7f5eb77fe640 1 --2- 172.21.15.27:0/741925188 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5eb814a750 0x7f5eb814ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:51.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:51.876+0000 7f5eb67fc640 1 --2- 172.21.15.27:0/741925188 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f5eb8149140 0x7f5eb80b3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:51.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:51.876+0000 7f5eb6ffd640 1 -- 172.21.15.27:0/741925188 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f5eb8149140 msgr2=0x7f5eb80b3b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:51.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:51.876+0000 7f5eb6ffd640 1 --2- 172.21.15.27:0/741925188 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f5eb8149140 0x7f5eb80b3b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:51.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:51.876+0000 7f5eb6ffd640 1 -- 172.21.15.27:0/741925188 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5eb814a750 msgr2=0x7f5eb814ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:51.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:51.876+0000 7f5eb6ffd640 1 --2- 172.21.15.27:0/741925188 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5eb814a750 0x7f5eb814ac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:51.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:51.876+0000 7f5eb6ffd640 1 -- 172.21.15.27:0/741925188 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5eb814c0f0 con 0x7f5ea0004d40 2024-09-12T03:20:51.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:51.876+0000 7f5eb6ffd640 1 --2- 172.21.15.27:0/741925188 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5ea0004d40 0x7f5eb814bbb0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f5ea800e800 tx=0x7f5ea800ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:51.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:51.876+0000 7f5e9ffff640 1 -- 172.21.15.27:0/741925188 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ea800ee50 con 0x7f5ea0004d40 2024-09-12T03:20:51.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:51.876+0000 7f5e9ffff640 1 -- 172.21.15.27:0/741925188 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ea8018070 con 0x7f5ea0004d40 2024-09-12T03:20:51.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:51.876+0000 7f5ebd8dd640 1 -- 172.21.15.27:0/741925188 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5eb81496b0 con 0x7f5ea0004d40 2024-09-12T03:20:51.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:51.876+0000 7f5e9ffff640 1 -- 172.21.15.27:0/741925188 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ea800ca70 con 0x7f5ea0004d40 2024-09-12T03:20:51.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:51.876+0000 7f5ebd8dd640 1 -- 172.21.15.27:0/741925188 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5eb8149c70 con 0x7f5ea0004d40 2024-09-12T03:20:51.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:51.877+0000 7f5ebd8dd640 1 -- 172.21.15.27:0/741925188 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5eb80d1d40 con 0x7f5ea0004d40 2024-09-12T03:20:51.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:51.878+0000 7f5e9ffff640 1 -- 172.21.15.27:0/741925188 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 90600+0+0 (secure 0 0 0) 0x7f5ea800cc10 con 0x7f5ea0004d40 2024-09-12T03:20:51.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:51.878+0000 7f5e9ffff640 1 --2- 172.21.15.27:0/741925188 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f5ea406bf80 0x7f5ea406e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:51.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:51.878+0000 7f5e9ffff640 1 -- 172.21.15.27:0/741925188 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5029+0+0 (secure 0 0 0) 0x7f5ea808a770 con 0x7f5ea0004d40 2024-09-12T03:20:51.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:51.878+0000 7f5eb67fc640 1 --2- 172.21.15.27:0/741925188 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f5ea406bf80 0x7f5ea406e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:51.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:51.879+0000 7f5eb67fc640 1 --2- 172.21.15.27:0/741925188 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f5ea406bf80 0x7f5ea406e440 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f5eb81ea350 tx=0x7f5eac009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:51.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:51.882+0000 7f5e9ffff640 1 -- 172.21.15.27:0/741925188 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f5ea80592b0 con 0x7f5ea0004d40 2024-09-12T03:20:52.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:52.058+0000 7f5ebd8dd640 1 -- 172.21.15.27:0/741925188 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f5eb806faf0 con 0x7f5ea0004d40 2024-09-12T03:20:52.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:52.061+0000 7f5e9ffff640 1 -- 172.21.15.27:0/741925188 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v19) ==== 104+0+0 (secure 0 0 0) 0x7f5ea8058bb0 con 0x7f5ea0004d40 2024-09-12T03:20:52.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:52.063+0000 7f5ebd8dd640 1 -- 172.21.15.27:0/741925188 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f5ea406bf80 msgr2=0x7f5ea406e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:52.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:52.063+0000 7f5ebd8dd640 1 --2- 172.21.15.27:0/741925188 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f5ea406bf80 0x7f5ea406e440 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f5eb81ea350 tx=0x7f5eac009190 comp rx=0 tx=0).stop 2024-09-12T03:20:52.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:52.063+0000 7f5ebd8dd640 1 -- 172.21.15.27:0/741925188 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5ea0004d40 msgr2=0x7f5eb814bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:52.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:52.063+0000 7f5ebd8dd640 1 --2- 172.21.15.27:0/741925188 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5ea0004d40 0x7f5eb814bbb0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f5ea800e800 tx=0x7f5ea800ec00 comp rx=0 tx=0).stop 2024-09-12T03:20:52.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:52.063+0000 7f5ebd8dd640 1 -- 172.21.15.27:0/741925188 shutdown_connections 2024-09-12T03:20:52.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:52.063+0000 7f5ebd8dd640 1 --2- 172.21.15.27:0/741925188 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5eb814a750 0x7f5eb814ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:52.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:52.063+0000 7f5ebd8dd640 1 --2- 172.21.15.27:0/741925188 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f5eb8149140 0x7f5eb80b3b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:52.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:52.063+0000 7f5ebd8dd640 1 --2- 172.21.15.27:0/741925188 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f5ea406bf80 0x7f5ea406e440 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:52.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:52.063+0000 7f5ebd8dd640 1 --2- 172.21.15.27:0/741925188 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5ea0004d40 0x7f5eb814bbb0 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:52.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:52.063+0000 7f5ebd8dd640 1 -- 172.21.15.27:0/741925188 >> 172.21.15.27:0/741925188 conn(0x7f5eb806f320 msgr2=0x7f5eb80736c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:52.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:52.064+0000 7f5ebd8dd640 1 -- 172.21.15.27:0/741925188 shutdown_connections 2024-09-12T03:20:52.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:52.064+0000 7f5ebd8dd640 1 -- 172.21.15.27:0/741925188 wait complete. 2024-09-12T03:20:52.076 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestMisc.test_names_can_only_be_goodchars' 2024-09-12T03:20:52.179 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:52.178+0000 7f102a9fd640 1 Processor -- start 2024-09-12T03:20:52.179 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:52.178+0000 7f102a9fd640 1 -- start start 2024-09-12T03:20:52.181 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:52.179+0000 7f102a9fd640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f10240ae8d0 0x7f10240728b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:52.181 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:52.179+0000 7f102a9fd640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f1024072df0 0x7f102406e930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:52.181 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:52.179+0000 7f102a9fd640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f10240aee90 con 0x7f10240b5a70 2024-09-12T03:20:52.181 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:52.179+0000 7f102a9fd640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f1024073240 con 0x7f1024072df0 2024-09-12T03:20:52.181 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:52.179+0000 7f102a9fd640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f10240af170 con 0x7f10240ae8d0 2024-09-12T03:20:52.182 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:52.179+0000 7f1028f9a640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f1024072df0 0x7f102406e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:52.182 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:52.179+0000 7f1028f9a640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f1024072df0 0x7f102406e930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54956/0 (socket says 172.21.15.27:54956) 2024-09-12T03:20:52.182 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:52.179+0000 7f1028f9a640 1 -- 172.21.15.27:0/2240553230 learned_addr learned my addr 172.21.15.27:0/2240553230 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:20:52.182 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:52.179+0000 7f10237fe640 1 --2- 172.21.15.27:0/2240553230 >> v2:172.21.15.45:3300/0 conn(0x7f10240ae8d0 0x7f10240728b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:52.182 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:52.179+0000 7f1028f9a640 1 -- 172.21.15.27:0/2240553230 >> v2:172.21.15.45:3300/0 conn(0x7f10240ae8d0 msgr2=0x7f10240728b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:52.182 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:52.179+0000 7f1028f9a640 1 --2- 172.21.15.27:0/2240553230 >> v2:172.21.15.45:3300/0 conn(0x7f10240ae8d0 0x7f10240728b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:52.182 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:52.179+0000 7f1028f9a640 1 -- 172.21.15.27:0/2240553230 >> v1:172.21.15.45:6789/0 conn(0x7f10240b5a70 legacy=0x7f102415f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:52.182 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:52.179+0000 7f1028f9a640 1 -- 172.21.15.27:0/2240553230 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10240af450 con 0x7f1024072df0 2024-09-12T03:20:52.182 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:52.180+0000 7f1028f9a640 1 --2- 172.21.15.27:0/2240553230 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1024072df0 0x7f102406e930 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f1024062840 tx=0x7f101c02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec5900e45d32f41f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:52.182 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:52.180+0000 7f1022ffd640 1 -- 172.21.15.27:0/2240553230 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f101c036070 con 0x7f1024072df0 2024-09-12T03:20:52.182 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:52.180+0000 7f1022ffd640 1 -- 172.21.15.27:0/2240553230 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f101c031600 con 0x7f1024072df0 2024-09-12T03:20:52.182 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:52.180+0000 7f1022ffd640 1 -- 172.21.15.27:0/2240553230 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f101c0319d0 con 0x7f1024072df0 2024-09-12T03:20:52.183 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:52.181+0000 7f102a9fd640 1 -- 172.21.15.27:0/2240553230 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1024072df0 msgr2=0x7f102406e930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:52.183 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:52.181+0000 7f102a9fd640 1 --2- 172.21.15.27:0/2240553230 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1024072df0 0x7f102406e930 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f1024062840 tx=0x7f101c02a6a0 comp rx=0 tx=0).stop 2024-09-12T03:20:52.183 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:52.181+0000 7f102a9fd640 1 -- 172.21.15.27:0/2240553230 shutdown_connections 2024-09-12T03:20:52.183 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:52.181+0000 7f102a9fd640 1 --2- 172.21.15.27:0/2240553230 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1024072df0 0x7f102406e930 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:52.184 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:52.181+0000 7f102a9fd640 1 --2- 172.21.15.27:0/2240553230 >> v2:172.21.15.45:3300/0 conn(0x7f10240ae8d0 0x7f10240728b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:52.184 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:52.181+0000 7f102a9fd640 1 -- 172.21.15.27:0/2240553230 >> 172.21.15.27:0/2240553230 conn(0x7f102406ca30 msgr2=0x7f102406ce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:52.184 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:52.181+0000 7f102a9fd640 1 -- 172.21.15.27:0/2240553230 shutdown_connections 2024-09-12T03:20:52.184 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:52.181+0000 7f102a9fd640 1 -- 172.21.15.27:0/2240553230 wait complete. 2024-09-12T03:20:52.184 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:52.182+0000 7f102a9fd640 1 Processor -- start 2024-09-12T03:20:52.184 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:52.182+0000 7f102a9fd640 1 -- start start 2024-09-12T03:20:52.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:52.183+0000 7f102a9fd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1024072df0 0x7f102414d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:52.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:52.183+0000 7f102a9fd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f10240ae8d0 0x7f10241505d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:52.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:52.183+0000 7f102a9fd640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f10240b5a70 0x7f1024160760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:52.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:52.183+0000 7f102a9fd640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f102406eff0 con 0x7f10240ae8d0 2024-09-12T03:20:52.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:52.183+0000 7f102a9fd640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f102406ee70 con 0x7f1024072df0 2024-09-12T03:20:52.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:52.183+0000 7f102a9fd640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f102406f170 con 0x7f10240b5a70 2024-09-12T03:20:52.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:52.183+0000 7f10237fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f10240ae8d0 0x7f10241505d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:52.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:52.183+0000 7f10237fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f10240ae8d0 0x7f10241505d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54968/0 (socket says 172.21.15.27:54968) 2024-09-12T03:20:52.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:52.183+0000 7f10237fe640 1 -- 172.21.15.27:0/3061042882 learned_addr learned my addr 172.21.15.27:0/3061042882 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:20:52.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:52.183+0000 7f1023fff640 1 --2- 172.21.15.27:0/3061042882 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1024072df0 0x7f102414d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:52.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:52.183+0000 7f1028f9a640 1 --2- 172.21.15.27:0/3061042882 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f10240b5a70 0x7f1024160760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:52.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:52.183+0000 7f10237fe640 1 -- 172.21.15.27:0/3061042882 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f10240b5a70 msgr2=0x7f1024160760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:52.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:52.183+0000 7f10237fe640 1 --2- 172.21.15.27:0/3061042882 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f10240b5a70 0x7f1024160760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:52.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:52.183+0000 7f10237fe640 1 -- 172.21.15.27:0/3061042882 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1024072df0 msgr2=0x7f102414d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:52.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:52.183+0000 7f10237fe640 1 --2- 172.21.15.27:0/3061042882 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1024072df0 0x7f102414d870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:52.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:52.183+0000 7f10237fe640 1 -- 172.21.15.27:0/3061042882 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10240cdba0 con 0x7f10240ae8d0 2024-09-12T03:20:52.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:52.183+0000 7f10237fe640 1 --2- 172.21.15.27:0/3061042882 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f10240ae8d0 0x7f10241505d0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f101800d800 tx=0x7f101800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:52.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:52.184+0000 7f1020ff9640 1 -- 172.21.15.27:0/3061042882 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f101800b6c0 con 0x7f10240ae8d0 2024-09-12T03:20:52.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:52.184+0000 7f1020ff9640 1 -- 172.21.15.27:0/3061042882 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1018019070 con 0x7f10240ae8d0 2024-09-12T03:20:52.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:52.184+0000 7f1020ff9640 1 -- 172.21.15.27:0/3061042882 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f101800bac0 con 0x7f10240ae8d0 2024-09-12T03:20:52.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:52.184+0000 7f102a9fd640 1 -- 172.21.15.27:0/3061042882 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1024159520 con 0x7f10240ae8d0 2024-09-12T03:20:52.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:52.184+0000 7f102a9fd640 1 -- 172.21.15.27:0/3061042882 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1024159ae0 con 0x7f10240ae8d0 2024-09-12T03:20:52.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:52.185+0000 7f102a9fd640 1 -- 172.21.15.27:0/3061042882 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f102406e930 con 0x7f10240ae8d0 2024-09-12T03:20:52.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:52.185+0000 7f1020ff9640 1 -- 172.21.15.27:0/3061042882 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 90600+0+0 (secure 0 0 0) 0x7f101800bc60 con 0x7f10240ae8d0 2024-09-12T03:20:52.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:52.186+0000 7f1020ff9640 1 --2- 172.21.15.27:0/3061042882 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f101006bf50 0x7f101006e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:52.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:52.186+0000 7f1020ff9640 1 -- 172.21.15.27:0/3061042882 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5029+0+0 (secure 0 0 0) 0x7f101808b190 con 0x7f10240ae8d0 2024-09-12T03:20:52.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:52.186+0000 7f1023fff640 1 --2- 172.21.15.27:0/3061042882 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f101006bf50 0x7f101006e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:52.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:52.186+0000 7f1023fff640 1 --2- 172.21.15.27:0/3061042882 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f101006bf50 0x7f101006e410 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f1014007fd0 tx=0x7f1014009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:52.190 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:52.190+0000 7f1020ff9640 1 -- 172.21.15.27:0/3061042882 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f1018059d60 con 0x7f10240ae8d0 2024-09-12T03:20:52.439 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:52.438+0000 7f102a9fd640 1 -- 172.21.15.27:0/3061042882 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_names_can_only_be_goodchars"]} v 0) -- 0x7f102414de30 con 0x7f10240ae8d0 2024-09-12T03:20:52.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:52.746+0000 7f1020ff9640 1 -- 172.21.15.27:0/3061042882 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_names_can_only_be_goodchars"]}]=0 v240) ==== 146+0+0 (secure 0 0 0) 0x7f1018059660 con 0x7f10240ae8d0 2024-09-12T03:20:52.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:52.748+0000 7f102a9fd640 1 -- 172.21.15.27:0/3061042882 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f101006bf50 msgr2=0x7f101006e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:52.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:52.748+0000 7f102a9fd640 1 --2- 172.21.15.27:0/3061042882 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f101006bf50 0x7f101006e410 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f1014007fd0 tx=0x7f1014009040 comp rx=0 tx=0).stop 2024-09-12T03:20:52.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:52.748+0000 7f102a9fd640 1 -- 172.21.15.27:0/3061042882 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f10240ae8d0 msgr2=0x7f10241505d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:52.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:52.748+0000 7f102a9fd640 1 --2- 172.21.15.27:0/3061042882 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f10240ae8d0 0x7f10241505d0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f101800d800 tx=0x7f101800dcc0 comp rx=0 tx=0).stop 2024-09-12T03:20:52.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:52.748+0000 7f102a9fd640 1 -- 172.21.15.27:0/3061042882 shutdown_connections 2024-09-12T03:20:52.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:52.748+0000 7f102a9fd640 1 --2- 172.21.15.27:0/3061042882 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f10240b5a70 0x7f1024160760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:52.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:52.748+0000 7f102a9fd640 1 --2- 172.21.15.27:0/3061042882 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f101006bf50 0x7f101006e410 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:52.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:52.748+0000 7f102a9fd640 1 --2- 172.21.15.27:0/3061042882 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f10240ae8d0 0x7f10241505d0 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:52.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:52.748+0000 7f102a9fd640 1 --2- 172.21.15.27:0/3061042882 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1024072df0 0x7f102414d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:52.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:52.748+0000 7f102a9fd640 1 -- 172.21.15.27:0/3061042882 >> 172.21.15.27:0/3061042882 conn(0x7f102406ca30 msgr2=0x7f102414e070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:52.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:52.748+0000 7f102a9fd640 1 -- 172.21.15.27:0/3061042882 shutdown_connections 2024-09-12T03:20:52.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:52.748+0000 7f102a9fd640 1 -- 172.21.15.27:0/3061042882 wait complete. 2024-09-12T03:20:52.760 INFO:tasks.cephfs_test_runner:Test the creating vols, subvols subvolgroups fails when their names uses ... ok 2024-09-12T03:20:52.761 INFO:tasks.cephfs_test_runner:Starting test: test_subvolume_no_upgrade_v1_sanity (tasks.cephfs.test_volumes.TestMisc) poor man's upgrade test -- theme continues... 2024-09-12T03:20:52.761 INFO:tasks.cephfs_test_runner:test_subvolume_no_upgrade_v1_sanity (tasks.cephfs.test_volumes.TestMisc) 2024-09-12T03:20:52.761 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-12T03:20:52.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:52.862+0000 7f7d2d015640 1 Processor -- start 2024-09-12T03:20:52.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:52.862+0000 7f7d2d015640 1 -- start start 2024-09-12T03:20:52.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:52.863+0000 7f7d2d015640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f7d280d5200 0x7f7d280d1010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:52.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:52.863+0000 7f7d2d015640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7d280d3760 con 0x7f7d280d1550 2024-09-12T03:20:52.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:52.863+0000 7f7d2d015640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7d280d3a40 con 0x7f7d2815eb90 2024-09-12T03:20:52.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:52.863+0000 7f7d2d015640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f7d280b7d20 con 0x7f7d280d5200 2024-09-12T03:20:52.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:52.863+0000 7f7d2759e640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f7d280d1550 0x7f7d280b8190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:58124/0 (socket says 172.21.15.27:58124) 2024-09-12T03:20:52.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:52.863+0000 7f7d2759e640 1 -- 172.21.15.27:0/1303927435 learned_addr learned my addr 172.21.15.27:0/1303927435 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:20:52.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:52.863+0000 7f7d2659c640 1 --2- 172.21.15.27:0/1303927435 >> v2:172.21.15.45:3300/0 conn(0x7f7d280d5200 0x7f7d280d1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:52.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:52.864+0000 7f7d25d9b640 1 -- 172.21.15.27:0/1303927435 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3827582573 0 0) 0x7f7d280d3760 con 0x7f7d280d1550 2024-09-12T03:20:52.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:52.864+0000 7f7d25d9b640 1 -- 172.21.15.27:0/1303927435 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7d14003520 con 0x7f7d280d1550 2024-09-12T03:20:52.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:52.864+0000 7f7d25d9b640 1 -- 172.21.15.27:0/1303927435 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 74763872 0 0) 0x7f7d14003520 con 0x7f7d280d1550 2024-09-12T03:20:52.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:52.864+0000 7f7d2659c640 1 -- 172.21.15.27:0/1303927435 >> v1:172.21.15.45:6789/0 conn(0x7f7d2815eb90 legacy=0x7f7d280d2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:52.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:52.864+0000 7f7d2659c640 1 -- 172.21.15.27:0/1303927435 >> v1:172.21.15.27:6789/0 conn(0x7f7d280d1550 legacy=0x7f7d280b8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:52.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:52.864+0000 7f7d2659c640 1 -- 172.21.15.27:0/1303927435 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d280d3d20 con 0x7f7d280d5200 2024-09-12T03:20:52.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:52.865+0000 7f7d2659c640 1 --2- 172.21.15.27:0/1303927435 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7d280d5200 0x7f7d280d1010 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f7d100097a0 tx=0x7f7d1002a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4da825537db5dae4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:52.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:52.865+0000 7f7d25d9b640 1 -- 172.21.15.27:0/1303927435 <== mon.1 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d10037070 con 0x7f7d280d5200 2024-09-12T03:20:52.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:52.865+0000 7f7d25d9b640 1 -- 172.21.15.27:0/1303927435 <== mon.1 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d1002ac50 con 0x7f7d280d5200 2024-09-12T03:20:52.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:52.865+0000 7f7d25d9b640 1 -- 172.21.15.27:0/1303927435 <== mon.1 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d180019f0 con 0x7f7d280d5200 2024-09-12T03:20:52.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:52.865+0000 7f7d2d015640 1 -- 172.21.15.27:0/1303927435 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7d280d5200 msgr2=0x7f7d280d1010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:52.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:52.865+0000 7f7d2d015640 1 --2- 172.21.15.27:0/1303927435 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7d280d5200 0x7f7d280d1010 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f7d100097a0 tx=0x7f7d1002a1c0 comp rx=0 tx=0).stop 2024-09-12T03:20:52.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:52.865+0000 7f7d2d015640 1 -- 172.21.15.27:0/1303927435 shutdown_connections 2024-09-12T03:20:52.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:52.865+0000 7f7d2d015640 1 --2- 172.21.15.27:0/1303927435 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7d280d5200 0x7f7d280d1010 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:52.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:52.865+0000 7f7d2d015640 1 -- 172.21.15.27:0/1303927435 >> 172.21.15.27:0/1303927435 conn(0x7f7d2806dce0 msgr2=0x7f7d2806e0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:52.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:52.865+0000 7f7d2d015640 1 -- 172.21.15.27:0/1303927435 shutdown_connections 2024-09-12T03:20:52.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:52.865+0000 7f7d2d015640 1 -- 172.21.15.27:0/1303927435 wait complete. 2024-09-12T03:20:52.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:52.867+0000 7f7d2d015640 1 Processor -- start 2024-09-12T03:20:52.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:52.867+0000 7f7d2d015640 1 -- start start 2024-09-12T03:20:52.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:52.868+0000 7f7d2d015640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7d280d1550 0x7f7d2815b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:52.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:52.868+0000 7f7d2d015640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f7d280d5200 0x7f7d280c8d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:52.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:52.868+0000 7f7d2d015640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7d2815eb90 0x7f7d2814eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:52.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:52.868+0000 7f7d2d015640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f7d280d3500 con 0x7f7d2815eb90 2024-09-12T03:20:52.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:52.868+0000 7f7d2d015640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f7d280d42e0 con 0x7f7d280d1550 2024-09-12T03:20:52.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:52.868+0000 7f7d2d015640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f7d280d1960 con 0x7f7d280d5200 2024-09-12T03:20:52.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:52.868+0000 7f7d2759e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7d2815eb90 0x7f7d2814eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:52.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:52.868+0000 7f7d2759e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7d2815eb90 0x7f7d2814eb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54980/0 (socket says 172.21.15.27:54980) 2024-09-12T03:20:52.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:52.868+0000 7f7d2759e640 1 -- 172.21.15.27:0/1807473314 learned_addr learned my addr 172.21.15.27:0/1807473314 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:20:52.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:52.868+0000 7f7d26d9d640 1 --2- 172.21.15.27:0/1807473314 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7d280d1550 0x7f7d2815b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:52.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:52.868+0000 7f7d2659c640 1 --2- 172.21.15.27:0/1807473314 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f7d280d5200 0x7f7d280c8d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:52.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:52.868+0000 7f7d2759e640 1 -- 172.21.15.27:0/1807473314 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f7d280d5200 msgr2=0x7f7d280c8d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:52.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:52.868+0000 7f7d2759e640 1 --2- 172.21.15.27:0/1807473314 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f7d280d5200 0x7f7d280c8d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:52.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:52.868+0000 7f7d2759e640 1 -- 172.21.15.27:0/1807473314 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7d280d1550 msgr2=0x7f7d2815b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:52.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:52.868+0000 7f7d2759e640 1 --2- 172.21.15.27:0/1807473314 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7d280d1550 0x7f7d2815b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:52.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:52.868+0000 7f7d2759e640 1 -- 172.21.15.27:0/1807473314 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d280c93e0 con 0x7f7d2815eb90 2024-09-12T03:20:52.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:52.868+0000 7f7d2759e640 1 --2- 172.21.15.27:0/1807473314 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7d2815eb90 0x7f7d2814eb20 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f7d1800cb40 tx=0x7f7d1800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:52.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:52.869+0000 7f7d1f7fe640 1 -- 172.21.15.27:0/1807473314 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d18017070 con 0x7f7d2815eb90 2024-09-12T03:20:52.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:52.869+0000 7f7d1f7fe640 1 -- 172.21.15.27:0/1807473314 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d1800ad80 con 0x7f7d2815eb90 2024-09-12T03:20:52.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:52.869+0000 7f7d1f7fe640 1 -- 172.21.15.27:0/1807473314 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d18012640 con 0x7f7d2815eb90 2024-09-12T03:20:52.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:52.869+0000 7f7d2d015640 1 -- 172.21.15.27:0/1807473314 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d280c96c0 con 0x7f7d2815eb90 2024-09-12T03:20:52.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:52.869+0000 7f7d2d015640 1 -- 172.21.15.27:0/1807473314 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d28158a30 con 0x7f7d2815eb90 2024-09-12T03:20:52.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:52.870+0000 7f7d2d015640 1 -- 172.21.15.27:0/1807473314 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d280693d0 con 0x7f7d2815eb90 2024-09-12T03:20:52.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:52.870+0000 7f7d1f7fe640 1 -- 172.21.15.27:0/1807473314 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 90600+0+0 (secure 0 0 0) 0x7f7d1801a070 con 0x7f7d2815eb90 2024-09-12T03:20:52.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:52.870+0000 7f7d1f7fe640 1 --2- 172.21.15.27:0/1807473314 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f7d0806bf50 0x7f7d0806e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:52.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:52.871+0000 7f7d1f7fe640 1 -- 172.21.15.27:0/1807473314 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5029+0+0 (secure 0 0 0) 0x7f7d1808ab00 con 0x7f7d2815eb90 2024-09-12T03:20:52.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:52.871+0000 7f7d26d9d640 1 --2- 172.21.15.27:0/1807473314 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f7d0806bf50 0x7f7d0806e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:52.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:52.871+0000 7f7d26d9d640 1 --2- 172.21.15.27:0/1807473314 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f7d0806bf50 0x7f7d0806e410 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f7d20000f60 tx=0x7f7d20009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:52.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:52.875+0000 7f7d1f7fe640 1 -- 172.21.15.27:0/1807473314 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f7d18058fe0 con 0x7f7d2815eb90 2024-09-12T03:20:53.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.054+0000 7f7d2d015640 1 -- 172.21.15.27:0/1807473314 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7d280b9d10 con 0x7f7d2815eb90 2024-09-12T03:20:53.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.054+0000 7f7d1f7fe640 1 -- 172.21.15.27:0/1807473314 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+13115 (secure 0 0 0) 0x7f7d1805d2e0 con 0x7f7d2815eb90 2024-09-12T03:20:53.055 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:20:53.056 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":92,"fsid":"d2df504e-2647-4385-a5db-9fdf69f487a0","created":"2024-09-12T03:16:07.778937+0000","modified":"2024-09-12T03:20:51.741079+0000","last_up_change":"2024-09-12T03:16:19.314778+0000","last_in_change":"2024-09-12T03:16:09.893086+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T03:16:20.097646+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1851116c-1fb3-4dc9-9772-f90e2ef4c355","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6817","nonce":3002321488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6819","nonce":3002321488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6823","nonce":3002321488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6821","nonce":3002321488}]},"public_addr":"172.21.15.27:6817/3002321488","cluster_addr":"172.21.15.27:6819/3002321488","heartbeat_back_addr":"172.21.15.27:6823/3002321488","heartbeat_front_addr":"172.21.15.27:6821/3002321488","state":["exists","up"]},{"osd":1,"uuid":"f38ae17d-73eb-4204-b99d-975fa806fb87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6809","nonce":1613102886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6811","nonce":1613102886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6815","nonce":1613102886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6813","nonce":1613102886}]},"public_addr":"172.21.15.27:6809/1613102886","cluster_addr":"172.21.15.27:6811/1613102886","heartbeat_back_addr":"172.21.15.27:6815/1613102886","heartbeat_front_addr":"172.21.15.27:6813/1613102886","state":["exists","up"]},{"osd":2,"uuid":"fb12dcc8-bdd0-42b8-b46d-8b10106224bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6825","nonce":3534723355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6827","nonce":3534723355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6831","nonce":3534723355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6829","nonce":3534723355}]},"public_addr":"172.21.15.27:6825/3534723355","cluster_addr":"172.21.15.27:6827/3534723355","heartbeat_back_addr":"172.21.15.27:6831/3534723355","heartbeat_front_addr":"172.21.15.27:6829/3534723355","state":["exists","up"]},{"osd":3,"uuid":"d74d3f49-9c24-4b9f-98e2-3ecd7b5ef3b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6801","nonce":80661598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6803","nonce":80661598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6807","nonce":80661598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6805","nonce":80661598}]},"public_addr":"172.21.15.27:6801/80661598","cluster_addr":"172.21.15.27:6803/80661598","heartbeat_back_addr":"172.21.15.27:6807/80661598","heartbeat_front_addr":"172.21.15.27:6805/80661598","state":["exists","up"]},{"osd":4,"uuid":"2bf123f2-1b0d-41c4-aa3b-805bab267e59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6817","nonce":3943189827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6819","nonce":3943189827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6823","nonce":3943189827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6821","nonce":3943189827}]},"public_addr":"172.21.15.45:6817/3943189827","cluster_addr":"172.21.15.45:6819/3943189827","heartbeat_back_addr":"172.21.15.45:6823/3943189827","heartbeat_front_addr":"172.21.15.45:6821/3943189827","state":["exists","up"]},{"osd":5,"uuid":"c49ba5da-91c6-4a86-a97b-4aba89c253ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6809","nonce":3620760330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6811","nonce":3620760330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6815","nonce":3620760330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6813","nonce":3620760330}]},"public_addr":"172.21.15.45:6809/3620760330","cluster_addr":"172.21.15.45:6811/3620760330","heartbeat_back_addr":"172.21.15.45:6815/3620760330","heartbeat_front_addr":"172.21.15.45:6813/3620760330","state":["exists","up"]},{"osd":6,"uuid":"b0123475-89f7-4a34-9e33-d797b99dd9dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6801","nonce":1989083510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6803","nonce":1989083510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6807","nonce":1989083510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6805","nonce":1989083510}]},"public_addr":"172.21.15.45:6801/1989083510","cluster_addr":"172.21.15.45:6803/1989083510","heartbeat_back_addr":"172.21.15.45:6807/1989083510","heartbeat_front_addr":"172.21.15.45:6805/1989083510","state":["exists","up"]},{"osd":7,"uuid":"4dad773d-b7d8-46fc-8a10-a972aad27b29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6825","nonce":1513300294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6827","nonce":1513300294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6831","nonce":1513300294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6829","nonce":1513300294}]},"public_addr":"172.21.15.45:6825/1513300294","cluster_addr":"172.21.15.45:6827/1513300294","heartbeat_back_addr":"172.21.15.45:6831/1513300294","heartbeat_front_addr":"172.21.15.45:6829/1513300294","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.881855+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:17.133252+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.735339+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.500742+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.477571+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:17.167667+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.470291+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.873743+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.27:6835/3209963418":"2024-09-13T03:20:48.733056+0000","172.21.15.27:6833/3209963418":"2024-09-13T03:20:48.733056+0000","172.21.15.27:6834/1466511159":"2024-09-13T03:20:42.229493+0000","172.21.15.27:6832/1466511159":"2024-09-13T03:20:42.229493+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T03:20:53.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.057+0000 7f7d2d015640 1 -- 172.21.15.27:0/1807473314 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f7d0806bf50 msgr2=0x7f7d0806e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:53.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.057+0000 7f7d2d015640 1 --2- 172.21.15.27:0/1807473314 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f7d0806bf50 0x7f7d0806e410 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f7d20000f60 tx=0x7f7d20009040 comp rx=0 tx=0).stop 2024-09-12T03:20:53.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.057+0000 7f7d2d015640 1 -- 172.21.15.27:0/1807473314 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7d2815eb90 msgr2=0x7f7d2814eb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:53.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.057+0000 7f7d2d015640 1 --2- 172.21.15.27:0/1807473314 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7d2815eb90 0x7f7d2814eb20 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f7d1800cb40 tx=0x7f7d1800a060 comp rx=0 tx=0).stop 2024-09-12T03:20:53.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.057+0000 7f7d2d015640 1 -- 172.21.15.27:0/1807473314 shutdown_connections 2024-09-12T03:20:53.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.057+0000 7f7d2d015640 1 --2- 172.21.15.27:0/1807473314 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f7d0806bf50 0x7f7d0806e410 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:53.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.057+0000 7f7d2d015640 1 --2- 172.21.15.27:0/1807473314 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7d2815eb90 0x7f7d2814eb20 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:53.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.057+0000 7f7d2d015640 1 --2- 172.21.15.27:0/1807473314 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f7d280d5200 0x7f7d280c8d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:53.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.057+0000 7f7d2d015640 1 --2- 172.21.15.27:0/1807473314 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7d280d1550 0x7f7d2815b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:53.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.057+0000 7f7d2d015640 1 -- 172.21.15.27:0/1807473314 >> 172.21.15.27:0/1807473314 conn(0x7f7d2806dce0 msgr2=0x7f7d2815f340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:53.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.057+0000 7f7d2d015640 1 -- 172.21.15.27:0/1807473314 shutdown_connections 2024-09-12T03:20:53.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.057+0000 7f7d2d015640 1 -- 172.21.15.27:0/1807473314 wait complete. 2024-09-12T03:20:53.069 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-12T03:16:20.097646+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-12T03:20:53.069 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-12T03:20:53.176 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.175+0000 7fe8d8aaf640 1 Processor -- start 2024-09-12T03:20:53.176 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.175+0000 7fe8d8aaf640 1 -- start start 2024-09-12T03:20:53.179 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.176+0000 7fe8d8aaf640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe8d415c6d0 0x7fe8d40d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:53.179 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.176+0000 7fe8d8aaf640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fe8d41600c0 0x7fe8d4160550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:53.179 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.176+0000 7fe8d8aaf640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe8d40d0ef0 con 0x7fe8d4070fa0 2024-09-12T03:20:53.180 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.176+0000 7fe8d8aaf640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fe8d40713e0 con 0x7fe8d41600c0 2024-09-12T03:20:53.180 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.176+0000 7fe8d8aaf640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fe8d40d1720 con 0x7fe8d415c6d0 2024-09-12T03:20:53.180 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.176+0000 7fe8d1d9b640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fe8d4070fa0 0x7fe8d406f360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:58134/0 (socket says 172.21.15.27:58134) 2024-09-12T03:20:53.180 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.176+0000 7fe8d1d9b640 1 -- 172.21.15.27:0/716103676 learned_addr learned my addr 172.21.15.27:0/716103676 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:20:53.180 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.176+0000 7fe8d2d9d640 1 --2- 172.21.15.27:0/716103676 >> v2:172.21.15.45:3300/0 conn(0x7fe8d41600c0 0x7fe8d4160550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:53.180 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.176+0000 7fe8d259c640 1 --2- 172.21.15.27:0/716103676 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe8d415c6d0 0x7fe8d40d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:53.180 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.176+0000 7fe8d159a640 1 -- 172.21.15.27:0/716103676 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1093510874 0 0) 0x7fe8d40d0ef0 con 0x7fe8d4070fa0 2024-09-12T03:20:53.180 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.176+0000 7fe8d159a640 1 -- 172.21.15.27:0/716103676 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe8bc003520 con 0x7fe8d4070fa0 2024-09-12T03:20:53.180 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.176+0000 7fe8d159a640 1 -- 172.21.15.27:0/716103676 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 35892875 0 0) 0x7fe8bc003520 con 0x7fe8d4070fa0 2024-09-12T03:20:53.180 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.176+0000 7fe8d159a640 1 -- 172.21.15.27:0/716103676 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe8d415c6d0 msgr2=0x7fe8d40d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:53.180 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.176+0000 7fe8d159a640 1 --2- 172.21.15.27:0/716103676 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe8d415c6d0 0x7fe8d40d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:53.180 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.176+0000 7fe8d159a640 1 -- 172.21.15.27:0/716103676 >> v2:172.21.15.45:3300/0 conn(0x7fe8d41600c0 msgr2=0x7fe8d4160550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:53.180 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.176+0000 7fe8d159a640 1 --2- 172.21.15.27:0/716103676 >> v2:172.21.15.45:3300/0 conn(0x7fe8d41600c0 0x7fe8d4160550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:53.181 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.176+0000 7fe8d159a640 1 -- 172.21.15.27:0/716103676 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8d40d1160 con 0x7fe8d4070fa0 2024-09-12T03:20:53.181 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.176+0000 7fe8d159a640 1 -- 172.21.15.27:0/716103676 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 847755213 0 0) 0x7fe8c8001670 con 0x7fe8d4070fa0 2024-09-12T03:20:53.181 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.176+0000 7fe8d159a640 1 -- 172.21.15.27:0/716103676 >> v1:172.21.15.27:6789/0 conn(0x7fe8d4070fa0 legacy=0x7fe8d406f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:53.181 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.176+0000 7fe8d159a640 1 --2- 172.21.15.27:0/716103676 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe8d415c6d0 0x7fe8bc0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:53.181 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.176+0000 7fe8d159a640 1 --2- 172.21.15.27:0/716103676 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe8bc003f30 0x7fe8bc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:53.181 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.176+0000 7fe8d159a640 1 --2- 172.21.15.27:0/716103676 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe8bc004d40 0x7fe8bc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:53.181 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.176+0000 7fe8d159a640 1 -- 172.21.15.27:0/716103676 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fe8d40713e0 con 0x7fe8bc003f30 2024-09-12T03:20:53.181 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.176+0000 7fe8d159a640 1 -- 172.21.15.27:0/716103676 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe8d40d1720 con 0x7fe8bc004d40 2024-09-12T03:20:53.181 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.177+0000 7fe8d159a640 1 -- 172.21.15.27:0/716103676 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fe8bc00baa0 con 0x7fe8d415c6d0 2024-09-12T03:20:53.181 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.177+0000 7fe8d2d9d640 1 --2- 172.21.15.27:0/716103676 >> v2:172.21.15.45:3300/0 conn(0x7fe8d41600c0 0x7fe8d4160550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:20:53.181 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.177+0000 7fe8d1d9b640 1 --2- 172.21.15.27:0/716103676 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe8bc003f30 0x7fe8bc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:53.181 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.177+0000 7fe8d259c640 1 --2- 172.21.15.27:0/716103676 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe8d415c6d0 0x7fe8bc0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:53.181 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.177+0000 7fe8d2d9d640 1 --2- 172.21.15.27:0/716103676 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe8bc004d40 0x7fe8bc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:53.181 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.177+0000 7fe8d1d9b640 1 -- 172.21.15.27:0/716103676 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe8d415c6d0 msgr2=0x7fe8bc0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:53.182 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.177+0000 7fe8d1d9b640 1 --2- 172.21.15.27:0/716103676 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe8d415c6d0 0x7fe8bc0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:53.182 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.177+0000 7fe8d1d9b640 1 -- 172.21.15.27:0/716103676 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe8bc004d40 msgr2=0x7fe8bc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:53.182 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.177+0000 7fe8d1d9b640 1 --2- 172.21.15.27:0/716103676 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe8bc004d40 0x7fe8bc0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:53.182 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.177+0000 7fe8d1d9b640 1 -- 172.21.15.27:0/716103676 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8bc003520 con 0x7fe8bc003f30 2024-09-12T03:20:53.182 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.177+0000 7fe8d2d9d640 1 --2- 172.21.15.27:0/716103676 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe8bc004d40 0x7fe8bc0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:20:53.182 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.178+0000 7fe8d1d9b640 1 --2- 172.21.15.27:0/716103676 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe8bc003f30 0x7fe8bc004800 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7fe8c800c240 tx=0x7fe8bc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74893faceb43aeca server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:53.182 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.178+0000 7fe8d159a640 1 -- 172.21.15.27:0/716103676 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8c8007d60 con 0x7fe8bc003f30 2024-09-12T03:20:53.182 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.178+0000 7fe8d159a640 1 -- 172.21.15.27:0/716103676 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8c8005ad0 con 0x7fe8bc003f30 2024-09-12T03:20:53.182 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.178+0000 7fe8d159a640 1 -- 172.21.15.27:0/716103676 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8c8007070 con 0x7fe8bc003f30 2024-09-12T03:20:53.182 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.178+0000 7fe8d8aaf640 1 -- 172.21.15.27:0/716103676 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe8bc003f30 msgr2=0x7fe8bc004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:53.182 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.178+0000 7fe8d8aaf640 1 --2- 172.21.15.27:0/716103676 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe8bc003f30 0x7fe8bc004800 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7fe8c800c240 tx=0x7fe8bc005830 comp rx=0 tx=0).stop 2024-09-12T03:20:53.182 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.178+0000 7fe8d2d9d640 1 -- 172.21.15.27:0/716103676 reap_dead start 2024-09-12T03:20:53.182 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.179+0000 7fe8d8aaf640 1 -- 172.21.15.27:0/716103676 shutdown_connections 2024-09-12T03:20:53.183 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.179+0000 7fe8d8aaf640 1 -- 172.21.15.27:0/716103676 >> 172.21.15.27:0/716103676 conn(0x7fe8d40b7c10 msgr2=0x7fe8d409ba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:53.183 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.179+0000 7fe8d8aaf640 1 -- 172.21.15.27:0/716103676 shutdown_connections 2024-09-12T03:20:53.183 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.179+0000 7fe8d8aaf640 1 -- 172.21.15.27:0/716103676 wait complete. 2024-09-12T03:20:53.185 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.180+0000 7fe8d8aaf640 1 Processor -- start 2024-09-12T03:20:53.185 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.180+0000 7fe8d8aaf640 1 -- start start 2024-09-12T03:20:53.185 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.181+0000 7fe8d8aaf640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe8bc003f30 0x7fe8d4151ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:53.185 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.181+0000 7fe8d8aaf640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe8d41543d0 0x7fe8d4148e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:53.185 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.181+0000 7fe8d8aaf640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe8d40b38f0 0x7fe8d40b3d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:53.185 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.181+0000 7fe8d8aaf640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fe8d40d1de0 con 0x7fe8d40b38f0 2024-09-12T03:20:53.185 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.181+0000 7fe8d8aaf640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe8d40ceb10 con 0x7fe8d41543d0 2024-09-12T03:20:53.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.181+0000 7fe8d8aaf640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fe8d40d1f60 con 0x7fe8bc003f30 2024-09-12T03:20:53.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.181+0000 7fe8d2d9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe8d40b38f0 0x7fe8d40b3d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:53.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.181+0000 7fe8d2d9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe8d40b38f0 0x7fe8d40b3d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55002/0 (socket says 172.21.15.27:55002) 2024-09-12T03:20:53.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.181+0000 7fe8d2d9d640 1 -- 172.21.15.27:0/682506900 learned_addr learned my addr 172.21.15.27:0/682506900 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:20:53.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.181+0000 7fe8d1d9b640 1 --2- 172.21.15.27:0/682506900 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe8d41543d0 0x7fe8d4148e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:53.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.181+0000 7fe8d259c640 1 --2- 172.21.15.27:0/682506900 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe8bc003f30 0x7fe8d4151ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:53.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.181+0000 7fe8d2d9d640 1 -- 172.21.15.27:0/682506900 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe8bc003f30 msgr2=0x7fe8d4151ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:53.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.181+0000 7fe8d2d9d640 1 --2- 172.21.15.27:0/682506900 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe8bc003f30 0x7fe8d4151ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:53.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.181+0000 7fe8d2d9d640 1 -- 172.21.15.27:0/682506900 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe8d41543d0 msgr2=0x7fe8d4148e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:53.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.181+0000 7fe8d2d9d640 1 --2- 172.21.15.27:0/682506900 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe8d41543d0 0x7fe8d4148e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:53.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.181+0000 7fe8d2d9d640 1 -- 172.21.15.27:0/682506900 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8d4152510 con 0x7fe8d40b38f0 2024-09-12T03:20:53.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.181+0000 7fe8d2d9d640 1 --2- 172.21.15.27:0/682506900 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe8d40b38f0 0x7fe8d40b3d70 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7fe8cc00f380 tx=0x7fe8cc00f780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:53.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.182+0000 7fe8c2ffd640 1 -- 172.21.15.27:0/682506900 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8cc016620 con 0x7fe8d40b38f0 2024-09-12T03:20:53.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.182+0000 7fe8c2ffd640 1 -- 172.21.15.27:0/682506900 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8cc01b070 con 0x7fe8d40b38f0 2024-09-12T03:20:53.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.182+0000 7fe8c2ffd640 1 -- 172.21.15.27:0/682506900 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8cc0169e0 con 0x7fe8d40b38f0 2024-09-12T03:20:53.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.182+0000 7fe8d8aaf640 1 -- 172.21.15.27:0/682506900 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8d41527f0 con 0x7fe8d40b38f0 2024-09-12T03:20:53.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.182+0000 7fe8d8aaf640 1 -- 172.21.15.27:0/682506900 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8d4152db0 con 0x7fe8d40b38f0 2024-09-12T03:20:53.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.183+0000 7fe8d8aaf640 1 -- 172.21.15.27:0/682506900 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8d4159700 con 0x7fe8d40b38f0 2024-09-12T03:20:53.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.183+0000 7fe8c2ffd640 1 -- 172.21.15.27:0/682506900 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 90600+0+0 (secure 0 0 0) 0x7fe8cc016b80 con 0x7fe8d40b38f0 2024-09-12T03:20:53.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.184+0000 7fe8c2ffd640 1 --2- 172.21.15.27:0/682506900 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fe8b406bf80 0x7fe8b406e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:53.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.184+0000 7fe8c2ffd640 1 -- 172.21.15.27:0/682506900 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5029+0+0 (secure 0 0 0) 0x7fe8cc08bb50 con 0x7fe8d40b38f0 2024-09-12T03:20:53.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.184+0000 7fe8d259c640 1 --2- 172.21.15.27:0/682506900 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fe8b406bf80 0x7fe8b406e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:53.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.184+0000 7fe8d259c640 1 --2- 172.21.15.27:0/682506900 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fe8b406bf80 0x7fe8b406e440 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fe8c4009a60 tx=0x7fe8c4009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:53.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.188+0000 7fe8c2ffd640 1 -- 172.21.15.27:0/682506900 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fe8d4159700 con 0x7fe8d40b38f0 2024-09-12T03:20:53.364 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.362+0000 7fe8d8aaf640 1 -- 172.21.15.27:0/682506900 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fe8d4160550 con 0x7fe8d40b38f0 2024-09-12T03:20:53.364 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.363+0000 7fe8c2ffd640 1 -- 172.21.15.27:0/682506900 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v92) ==== 93+0+10 (secure 0 0 0) 0x7fe8cc05a580 con 0x7fe8d40b38f0 2024-09-12T03:20:53.364 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 1 2024-09-12T03:20:53.366 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.365+0000 7fe8d8aaf640 1 -- 172.21.15.27:0/682506900 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fe8b406bf80 msgr2=0x7fe8b406e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:53.366 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.365+0000 7fe8d8aaf640 1 --2- 172.21.15.27:0/682506900 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fe8b406bf80 0x7fe8b406e440 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fe8c4009a60 tx=0x7fe8c4009190 comp rx=0 tx=0).stop 2024-09-12T03:20:53.366 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.365+0000 7fe8d8aaf640 1 -- 172.21.15.27:0/682506900 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe8d40b38f0 msgr2=0x7fe8d40b3d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:53.367 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.365+0000 7fe8d8aaf640 1 --2- 172.21.15.27:0/682506900 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe8d40b38f0 0x7fe8d40b3d70 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7fe8cc00f380 tx=0x7fe8cc00f780 comp rx=0 tx=0).stop 2024-09-12T03:20:53.367 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.365+0000 7fe8d8aaf640 1 -- 172.21.15.27:0/682506900 shutdown_connections 2024-09-12T03:20:53.367 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.365+0000 7fe8d8aaf640 1 --2- 172.21.15.27:0/682506900 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fe8b406bf80 0x7fe8b406e440 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:53.367 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.365+0000 7fe8d8aaf640 1 --2- 172.21.15.27:0/682506900 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe8d40b38f0 0x7fe8d40b3d70 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:53.367 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.365+0000 7fe8d8aaf640 1 --2- 172.21.15.27:0/682506900 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe8d41543d0 0x7fe8d4148e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:53.367 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.365+0000 7fe8d8aaf640 1 --2- 172.21.15.27:0/682506900 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe8bc003f30 0x7fe8d4151ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:53.367 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.365+0000 7fe8d8aaf640 1 -- 172.21.15.27:0/682506900 >> 172.21.15.27:0/682506900 conn(0x7fe8d40b7c10 msgr2=0x7fe8d40d2360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:53.367 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.365+0000 7fe8d8aaf640 1 -- 172.21.15.27:0/682506900 shutdown_connections 2024-09-12T03:20:53.367 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.365+0000 7fe8d8aaf640 1 -- 172.21.15.27:0/682506900 wait complete. 2024-09-12T03:20:53.377 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_no_upgrade_v1_sanity' 2024-09-12T03:20:53.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.478+0000 7fc96fc0e640 1 Processor -- start 2024-09-12T03:20:53.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.478+0000 7fc96fc0e640 1 -- start start 2024-09-12T03:20:53.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.479+0000 7fc96fc0e640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fc968162d90 0x7fc9680d1040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:53.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.479+0000 7fc96fc0e640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fc9680d1580 0x7fc9680d4d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:53.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.479+0000 7fc96fc0e640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc9680d5270 0x7fc9681669f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:53.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.479+0000 7fc96fc0e640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7fc9680d1bc0 con 0x7fc968162d90 2024-09-12T03:20:53.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.479+0000 7fc96fc0e640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fc9680d1ea0 con 0x7fc9680d5270 2024-09-12T03:20:53.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.479+0000 7fc96fc0e640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fc9680d2a90 con 0x7fc9680d1580 2024-09-12T03:20:53.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.479+0000 7fc96d9aa640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fc968162d90 0x7fc9680d1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:53.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.479+0000 7fc96d9aa640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fc968162d90 0x7fc9680d1040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55004/0 (socket says 172.21.15.27:55004) 2024-09-12T03:20:53.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.479+0000 7fc96d9aa640 1 -- 172.21.15.27:0/1181111197 learned_addr learned my addr 172.21.15.27:0/1181111197 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:20:53.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.480+0000 7fc96d1a9640 1 --2- 172.21.15.27:0/1181111197 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fc9680d1580 0x7fc9680d4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:53.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.480+0000 7fc96e1ab640 1 --2- 172.21.15.27:0/1181111197 >> v2:172.21.15.45:3300/0 conn(0x7fc9680d5270 0x7fc9681669f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:53.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.480+0000 7fc96d9aa640 1 -- 172.21.15.27:0/1181111197 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fc9680d1580 msgr2=0x7fc9680d4d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:53.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.480+0000 7fc96d9aa640 1 --2- 172.21.15.27:0/1181111197 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fc9680d1580 0x7fc9680d4d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:53.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.480+0000 7fc96d9aa640 1 -- 172.21.15.27:0/1181111197 >> v2:172.21.15.45:3300/0 conn(0x7fc9680d5270 msgr2=0x7fc9681669f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:53.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.480+0000 7fc96d9aa640 1 --2- 172.21.15.27:0/1181111197 >> v2:172.21.15.45:3300/0 conn(0x7fc9680d5270 0x7fc9681669f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:53.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.480+0000 7fc96d9aa640 1 -- 172.21.15.27:0/1181111197 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9680d2d70 con 0x7fc968162d90 2024-09-12T03:20:53.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.480+0000 7fc96d1a9640 1 --2- 172.21.15.27:0/1181111197 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fc9680d1580 0x7fc9680d4d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:20:53.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.480+0000 7fc96d9aa640 1 --2- 172.21.15.27:0/1181111197 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc968162d90 0x7fc9680d1040 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7fc9680cfd20 tx=0x7fc96002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1748ab37ea9f7fb9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:53.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.481+0000 7fc96c9a8640 1 -- 172.21.15.27:0/1181111197 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc960037070 con 0x7fc968162d90 2024-09-12T03:20:53.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.481+0000 7fc96c9a8640 1 -- 172.21.15.27:0/1181111197 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc960032440 con 0x7fc968162d90 2024-09-12T03:20:53.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.481+0000 7fc96c9a8640 1 -- 172.21.15.27:0/1181111197 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc960032810 con 0x7fc968162d90 2024-09-12T03:20:53.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.481+0000 7fc96fc0e640 1 -- 172.21.15.27:0/1181111197 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc968162d90 msgr2=0x7fc9680d1040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:53.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.481+0000 7fc96fc0e640 1 --2- 172.21.15.27:0/1181111197 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc968162d90 0x7fc9680d1040 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7fc9680cfd20 tx=0x7fc96002a1e0 comp rx=0 tx=0).stop 2024-09-12T03:20:53.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.481+0000 7fc96fc0e640 1 -- 172.21.15.27:0/1181111197 shutdown_connections 2024-09-12T03:20:53.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.481+0000 7fc96fc0e640 1 --2- 172.21.15.27:0/1181111197 >> v2:172.21.15.45:3300/0 conn(0x7fc9680d5270 0x7fc9681669f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:53.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.481+0000 7fc96fc0e640 1 --2- 172.21.15.27:0/1181111197 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fc9680d1580 0x7fc9680d4d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:53.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.481+0000 7fc96fc0e640 1 --2- 172.21.15.27:0/1181111197 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc968162d90 0x7fc9680d1040 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:53.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.481+0000 7fc96fc0e640 1 -- 172.21.15.27:0/1181111197 >> 172.21.15.27:0/1181111197 conn(0x7fc96806dce0 msgr2=0x7fc96806e0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:53.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.481+0000 7fc96fc0e640 1 -- 172.21.15.27:0/1181111197 shutdown_connections 2024-09-12T03:20:53.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.481+0000 7fc96fc0e640 1 -- 172.21.15.27:0/1181111197 wait complete. 2024-09-12T03:20:53.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.482+0000 7fc96fc0e640 1 Processor -- start 2024-09-12T03:20:53.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.482+0000 7fc96fc0e640 1 -- start start 2024-09-12T03:20:53.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.483+0000 7fc96fc0e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc9680d1580 0x7fc96814e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:53.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.483+0000 7fc96fc0e640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fc9680d5270 0x7fc968070a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:53.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.483+0000 7fc96fc0e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc968162d90 0x7fc96806ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:53.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.483+0000 7fc96fc0e640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fc9681632f0 con 0x7fc9680d1580 2024-09-12T03:20:53.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.483+0000 7fc96fc0e640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc968163170 con 0x7fc968162d90 2024-09-12T03:20:53.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.483+0000 7fc96fc0e640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fc968166f30 con 0x7fc9680d5270 2024-09-12T03:20:53.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.483+0000 7fc96d9aa640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc9680d1580 0x7fc96814e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:53.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.483+0000 7fc96d9aa640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc9680d1580 0x7fc96814e800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55018/0 (socket says 172.21.15.27:55018) 2024-09-12T03:20:53.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.483+0000 7fc96d9aa640 1 -- 172.21.15.27:0/3772910721 learned_addr learned my addr 172.21.15.27:0/3772910721 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:20:53.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.483+0000 7fc96e1ab640 1 --2- 172.21.15.27:0/3772910721 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc968162d90 0x7fc96806ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:53.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.483+0000 7fc96d1a9640 1 --2- 172.21.15.27:0/3772910721 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fc9680d5270 0x7fc968070a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:53.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.484+0000 7fc96d9aa640 1 -- 172.21.15.27:0/3772910721 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fc9680d5270 msgr2=0x7fc968070a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:53.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.484+0000 7fc96d9aa640 1 --2- 172.21.15.27:0/3772910721 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fc9680d5270 0x7fc968070a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:53.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.484+0000 7fc96d9aa640 1 -- 172.21.15.27:0/3772910721 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc968162d90 msgr2=0x7fc96806ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:53.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.484+0000 7fc96d9aa640 1 --2- 172.21.15.27:0/3772910721 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc968162d90 0x7fc96806ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:53.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.484+0000 7fc96d9aa640 1 -- 172.21.15.27:0/3772910721 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc96814f000 con 0x7fc9680d1580 2024-09-12T03:20:53.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.484+0000 7fc96d9aa640 1 --2- 172.21.15.27:0/3772910721 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc9680d1580 0x7fc96814e800 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7fc96002a220 tx=0x7fc9600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:53.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.484+0000 7fc9667fc640 1 -- 172.21.15.27:0/3772910721 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc960041020 con 0x7fc9680d1580 2024-09-12T03:20:53.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.484+0000 7fc9667fc640 1 -- 172.21.15.27:0/3772910721 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc9600306c0 con 0x7fc9680d1580 2024-09-12T03:20:53.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.484+0000 7fc96fc0e640 1 -- 172.21.15.27:0/3772910721 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc96814f2e0 con 0x7fc9680d1580 2024-09-12T03:20:53.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.484+0000 7fc9667fc640 1 -- 172.21.15.27:0/3772910721 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc960037040 con 0x7fc9680d1580 2024-09-12T03:20:53.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.484+0000 7fc96fc0e640 1 -- 172.21.15.27:0/3772910721 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc96814f8a0 con 0x7fc9680d1580 2024-09-12T03:20:53.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.485+0000 7fc96fc0e640 1 -- 172.21.15.27:0/3772910721 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9681669f0 con 0x7fc9680d1580 2024-09-12T03:20:53.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.486+0000 7fc9667fc640 1 -- 172.21.15.27:0/3772910721 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 90600+0+0 (secure 0 0 0) 0x7fc960044020 con 0x7fc9680d1580 2024-09-12T03:20:53.490 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.486+0000 7fc9667fc640 1 --2- 172.21.15.27:0/3772910721 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fc95406bf50 0x7fc95406e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:53.490 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.486+0000 7fc9667fc640 1 -- 172.21.15.27:0/3772910721 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5029+0+0 (secure 0 0 0) 0x7fc9600ab450 con 0x7fc9680d1580 2024-09-12T03:20:53.490 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.486+0000 7fc96d1a9640 1 --2- 172.21.15.27:0/3772910721 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fc95406bf50 0x7fc95406e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:53.490 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.486+0000 7fc96d1a9640 1 --2- 172.21.15.27:0/3772910721 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fc95406bf50 0x7fc95406e410 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fc96815f4b0 tx=0x7fc95c008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:53.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.490+0000 7fc9667fc640 1 -- 172.21.15.27:0/3772910721 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fc96004b030 con 0x7fc9680d1580 2024-09-12T03:20:53.733 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.732+0000 7fc96fc0e640 1 -- 172.21.15.27:0/3772910721 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_no_upgrade_v1_sanity"]} v 0) -- 0x7fc9680b3ec0 con 0x7fc9680d1580 2024-09-12T03:20:53.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.750+0000 7fc9667fc640 1 -- 172.21.15.27:0/3772910721 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_no_upgrade_v1_sanity"]}]=0 v241) ==== 152+0+0 (secure 0 0 0) 0x7fc960079940 con 0x7fc9680d1580 2024-09-12T03:20:53.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.752+0000 7fc96fc0e640 1 -- 172.21.15.27:0/3772910721 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fc95406bf50 msgr2=0x7fc95406e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:53.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.752+0000 7fc96fc0e640 1 --2- 172.21.15.27:0/3772910721 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fc95406bf50 0x7fc95406e410 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fc96815f4b0 tx=0x7fc95c008040 comp rx=0 tx=0).stop 2024-09-12T03:20:53.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.752+0000 7fc96fc0e640 1 -- 172.21.15.27:0/3772910721 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc9680d1580 msgr2=0x7fc96814e800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:53.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.752+0000 7fc96fc0e640 1 --2- 172.21.15.27:0/3772910721 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc9680d1580 0x7fc96814e800 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7fc96002a220 tx=0x7fc9600046c0 comp rx=0 tx=0).stop 2024-09-12T03:20:53.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.752+0000 7fc96fc0e640 1 -- 172.21.15.27:0/3772910721 shutdown_connections 2024-09-12T03:20:53.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.752+0000 7fc96fc0e640 1 --2- 172.21.15.27:0/3772910721 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc968162d90 0x7fc96806ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:53.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.752+0000 7fc96fc0e640 1 --2- 172.21.15.27:0/3772910721 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fc9680d5270 0x7fc968070a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:53.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.752+0000 7fc96fc0e640 1 --2- 172.21.15.27:0/3772910721 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fc95406bf50 0x7fc95406e410 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:53.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.752+0000 7fc96fc0e640 1 --2- 172.21.15.27:0/3772910721 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc9680d1580 0x7fc96814e800 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:53.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.752+0000 7fc96fc0e640 1 -- 172.21.15.27:0/3772910721 >> 172.21.15.27:0/3772910721 conn(0x7fc96806dce0 msgr2=0x7fc968160fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:53.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.752+0000 7fc96fc0e640 1 -- 172.21.15.27:0/3772910721 shutdown_connections 2024-09-12T03:20:53.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.752+0000 7fc96fc0e640 1 -- 172.21.15.27:0/3772910721 wait complete. 2024-09-12T03:20:53.764 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-12T03:20:53.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.869+0000 7f137292c640 1 Processor -- start 2024-09-12T03:20:53.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.869+0000 7f137292c640 1 -- start start 2024-09-12T03:20:53.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.869+0000 7f137292c640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f136c06dfd0 0x7f136c0d1da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:53.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.869+0000 7f137292c640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f136c0d3790 0x7f136c0b7a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:53.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.869+0000 7f137292c640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f136c0d2e30 con 0x7f136c162040 2024-09-12T03:20:53.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.869+0000 7f137292c640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f136c0b7f90 con 0x7f136c0d3790 2024-09-12T03:20:53.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.869+0000 7f137292c640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f136c162420 con 0x7f136c06dfd0 2024-09-12T03:20:53.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.870+0000 7f136b7fe640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f136c06dfd0 0x7f136c0d1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:53.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.870+0000 7f136b7fe640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f136c06dfd0 0x7f136c0d1da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3301/0 says I am v2:172.21.15.27:60140/0 (socket says 172.21.15.27:60140) 2024-09-12T03:20:53.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.870+0000 7f136b7fe640 1 -- 172.21.15.27:0/140926667 learned_addr learned my addr 172.21.15.27:0/140926667 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:20:53.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.870+0000 7f1370ec9640 1 --2- 172.21.15.27:0/140926667 >> v2:172.21.15.45:3300/0 conn(0x7f136c0d3790 0x7f136c0b7a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:53.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.870+0000 7f136b7fe640 1 -- 172.21.15.27:0/140926667 >> v2:172.21.15.45:3300/0 conn(0x7f136c0d3790 msgr2=0x7f136c0b7a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:53.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.870+0000 7f136b7fe640 1 --2- 172.21.15.27:0/140926667 >> v2:172.21.15.45:3300/0 conn(0x7f136c0d3790 0x7f136c0b7a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:53.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.870+0000 7f136b7fe640 1 -- 172.21.15.27:0/140926667 >> v1:172.21.15.45:6789/0 conn(0x7f136c162040 legacy=0x7f136c06d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:53.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.870+0000 7f136b7fe640 1 -- 172.21.15.27:0/140926667 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f136c0d30d0 con 0x7f136c06dfd0 2024-09-12T03:20:53.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.870+0000 7f1370ec9640 1 --2- 172.21.15.27:0/140926667 >> v2:172.21.15.45:3300/0 conn(0x7f136c0d3790 0x7f136c0b7a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:20:53.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.871+0000 7f136b7fe640 1 --2- 172.21.15.27:0/140926667 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f136c06dfd0 0x7f136c0d1da0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f1360009870 tx=0x7f136002a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=53968f1c34c25014 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:53.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.871+0000 7f136affd640 1 -- 172.21.15.27:0/140926667 <== mon.2 v2:172.21.15.45:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1360037070 con 0x7f136c06dfd0 2024-09-12T03:20:53.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.871+0000 7f136affd640 1 -- 172.21.15.27:0/140926667 <== mon.2 v2:172.21.15.45:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1360032440 con 0x7f136c06dfd0 2024-09-12T03:20:53.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.871+0000 7f136affd640 1 -- 172.21.15.27:0/140926667 <== mon.2 v2:172.21.15.45:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1360032810 con 0x7f136c06dfd0 2024-09-12T03:20:53.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.872+0000 7f137292c640 1 -- 172.21.15.27:0/140926667 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f136c06dfd0 msgr2=0x7f136c0d1da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:53.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.872+0000 7f137292c640 1 --2- 172.21.15.27:0/140926667 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f136c06dfd0 0x7f136c0d1da0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f1360009870 tx=0x7f136002a270 comp rx=0 tx=0).stop 2024-09-12T03:20:53.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.872+0000 7f137292c640 1 -- 172.21.15.27:0/140926667 shutdown_connections 2024-09-12T03:20:53.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.872+0000 7f137292c640 1 --2- 172.21.15.27:0/140926667 >> v2:172.21.15.45:3300/0 conn(0x7f136c0d3790 0x7f136c0b7a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:53.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.872+0000 7f137292c640 1 --2- 172.21.15.27:0/140926667 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f136c06dfd0 0x7f136c0d1da0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:53.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.872+0000 7f137292c640 1 -- 172.21.15.27:0/140926667 >> 172.21.15.27:0/140926667 conn(0x7f136c0b6700 msgr2=0x7f136c0b6b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:53.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.872+0000 7f137292c640 1 -- 172.21.15.27:0/140926667 shutdown_connections 2024-09-12T03:20:53.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.872+0000 7f137292c640 1 -- 172.21.15.27:0/140926667 wait complete. 2024-09-12T03:20:53.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.873+0000 7f137292c640 1 Processor -- start 2024-09-12T03:20:53.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.873+0000 7f137292c640 1 -- start start 2024-09-12T03:20:53.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.874+0000 7f137292c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f136c06dfd0 0x7f136c158f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:53.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.874+0000 7f137292c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f136c0d3790 0x7f136c0c8bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:53.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.874+0000 7f137292c640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f136c162040 0x7f136c0c9d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:53.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.874+0000 7f137292c640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f136c0d4530 con 0x7f136c06dfd0 2024-09-12T03:20:53.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.874+0000 7f137292c640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f136c0cee00 con 0x7f136c0d3790 2024-09-12T03:20:53.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.874+0000 7f137292c640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f136c0d4150 con 0x7f136c162040 2024-09-12T03:20:53.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.874+0000 7f136bfff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f136c06dfd0 0x7f136c158f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:53.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.874+0000 7f136bfff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f136c06dfd0 0x7f136c158f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55024/0 (socket says 172.21.15.27:55024) 2024-09-12T03:20:53.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.874+0000 7f136bfff640 1 -- 172.21.15.27:0/2246123260 learned_addr learned my addr 172.21.15.27:0/2246123260 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:20:53.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.874+0000 7f136b7fe640 1 --2- 172.21.15.27:0/2246123260 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f136c0d3790 0x7f136c0c8bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:53.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.874+0000 7f136bfff640 1 -- 172.21.15.27:0/2246123260 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f136c162040 msgr2=0x7f136c0c9d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:53.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.874+0000 7f1370ec9640 1 --2- 172.21.15.27:0/2246123260 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f136c162040 0x7f136c0c9d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:53.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.874+0000 7f136bfff640 1 --2- 172.21.15.27:0/2246123260 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f136c162040 0x7f136c0c9d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:53.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.874+0000 7f136bfff640 1 -- 172.21.15.27:0/2246123260 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f136c0d3790 msgr2=0x7f136c0c8bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:53.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.874+0000 7f136bfff640 1 --2- 172.21.15.27:0/2246123260 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f136c0d3790 0x7f136c0c8bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:53.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.874+0000 7f136bfff640 1 -- 172.21.15.27:0/2246123260 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f136c0c9950 con 0x7f136c06dfd0 2024-09-12T03:20:53.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.874+0000 7f136bfff640 1 --2- 172.21.15.27:0/2246123260 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f136c06dfd0 0x7f136c158f40 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f135c00c430 tx=0x7f135c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:53.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.875+0000 7f1368ff9640 1 -- 172.21.15.27:0/2246123260 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f135c0052d0 con 0x7f136c06dfd0 2024-09-12T03:20:53.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.875+0000 7f1368ff9640 1 -- 172.21.15.27:0/2246123260 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f135c005490 con 0x7f136c06dfd0 2024-09-12T03:20:53.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.875+0000 7f1368ff9640 1 -- 172.21.15.27:0/2246123260 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f135c007260 con 0x7f136c06dfd0 2024-09-12T03:20:53.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.875+0000 7f137292c640 1 -- 172.21.15.27:0/2246123260 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f136c15f9a0 con 0x7f136c06dfd0 2024-09-12T03:20:53.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.875+0000 7f137292c640 1 -- 172.21.15.27:0/2246123260 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f136c0c9390 con 0x7f136c06dfd0 2024-09-12T03:20:53.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.876+0000 7f137292c640 1 -- 172.21.15.27:0/2246123260 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f136c06e6e0 con 0x7f136c06dfd0 2024-09-12T03:20:53.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.876+0000 7f1368ff9640 1 -- 172.21.15.27:0/2246123260 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 90600+0+0 (secure 0 0 0) 0x7f135c007400 con 0x7f136c06dfd0 2024-09-12T03:20:53.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.877+0000 7f1368ff9640 1 --2- 172.21.15.27:0/2246123260 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f135806bf50 0x7f135806e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:53.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.877+0000 7f1368ff9640 1 -- 172.21.15.27:0/2246123260 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5029+0+0 (secure 0 0 0) 0x7f135c08a240 con 0x7f136c06dfd0 2024-09-12T03:20:53.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.877+0000 7f136b7fe640 1 --2- 172.21.15.27:0/2246123260 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f135806bf50 0x7f135806e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:53.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.878+0000 7f136b7fe640 1 --2- 172.21.15.27:0/2246123260 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f135806bf50 0x7f135806e410 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f1360034000 tx=0x7f136002e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:53.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:53.882+0000 7f1368ff9640 1 -- 172.21.15.27:0/2246123260 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f135c058e10 con 0x7f136c06dfd0 2024-09-12T03:20:54.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:54.058+0000 7f137292c640 1 -- 172.21.15.27:0/2246123260 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f136c0cfa40 con 0x7f136c06dfd0 2024-09-12T03:20:54.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:54.061+0000 7f1368ff9640 1 -- 172.21.15.27:0/2246123260 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v20)=0 v20) ==== 122+0+0 (secure 0 0 0) 0x7f135c058710 con 0x7f136c06dfd0 2024-09-12T03:20:54.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:54.063+0000 7f137292c640 1 -- 172.21.15.27:0/2246123260 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f135806bf50 msgr2=0x7f135806e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:54.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:54.063+0000 7f137292c640 1 --2- 172.21.15.27:0/2246123260 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f135806bf50 0x7f135806e410 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f1360034000 tx=0x7f136002e000 comp rx=0 tx=0).stop 2024-09-12T03:20:54.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:54.063+0000 7f137292c640 1 -- 172.21.15.27:0/2246123260 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f136c06dfd0 msgr2=0x7f136c158f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:54.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:54.063+0000 7f137292c640 1 --2- 172.21.15.27:0/2246123260 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f136c06dfd0 0x7f136c158f40 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f135c00c430 tx=0x7f135c00c8f0 comp rx=0 tx=0).stop 2024-09-12T03:20:54.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:54.064+0000 7f137292c640 1 -- 172.21.15.27:0/2246123260 shutdown_connections 2024-09-12T03:20:54.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:54.064+0000 7f137292c640 1 --2- 172.21.15.27:0/2246123260 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f136c162040 0x7f136c0c9d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:54.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:54.064+0000 7f137292c640 1 --2- 172.21.15.27:0/2246123260 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f136c0d3790 0x7f136c0c8bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:54.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:54.064+0000 7f137292c640 1 --2- 172.21.15.27:0/2246123260 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f135806bf50 0x7f135806e410 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:54.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:54.064+0000 7f137292c640 1 --2- 172.21.15.27:0/2246123260 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f136c06dfd0 0x7f136c158f40 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:54.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:54.064+0000 7f137292c640 1 -- 172.21.15.27:0/2246123260 >> 172.21.15.27:0/2246123260 conn(0x7f136c0b6700 msgr2=0x7f136c0b9100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:54.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:54.064+0000 7f137292c640 1 -- 172.21.15.27:0/2246123260 shutdown_connections 2024-09-12T03:20:54.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:54.064+0000 7f137292c640 1 -- 172.21.15.27:0/2246123260 wait complete. 2024-09-12T03:20:54.076 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-12T03:20:54.076 DEBUG:teuthology.orchestra.run.smithi027:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-12T03:20:54.119 DEBUG:teuthology.orchestra.run.smithi045:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-12T03:20:54.164 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-12T03:20:54.164 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-12T03:20:54.165 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T03:20:54.181 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056004,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497432k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16324 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248716k,nr_inodes=812179,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497432k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-12T03:20:54.181 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-12T03:20:54.181 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T03:20:54.199 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056795,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16602 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249348k,nr_inodes=812337,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-12T03:20:54.199 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-12T03:20:54.199 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T03:20:54.236 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056004,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497432k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16324 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248716k,nr_inodes=812179,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497432k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-12T03:20:54.237 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-12T03:20:54.237 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T03:20:54.257 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056795,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16602 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249348k,nr_inodes=812337,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-12T03:20:54.257 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-12T03:20:54.258 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-12T03:20:54.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:54.378+0000 7f26062b7640 1 Processor -- start 2024-09-12T03:20:54.379 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:54.378+0000 7f26062b7640 1 -- start start 2024-09-12T03:20:54.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:54.379+0000 7f26062b7640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f26000d5200 0x7f26000d1010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:54.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:54.379+0000 7f26062b7640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f26000d1550 0x7f26000b8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:54.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:54.379+0000 7f26062b7640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f26000d3760 con 0x7f260015eb90 2024-09-12T03:20:54.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:54.379+0000 7f26062b7640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f26000b7d20 con 0x7f26000d5200 2024-09-12T03:20:54.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:54.379+0000 7f26062b7640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f26000d42e0 con 0x7f26000d1550 2024-09-12T03:20:54.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:54.379+0000 7f25ff7fe640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f26000d5200 0x7f26000d1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:54.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:54.379+0000 7f25fffff640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f260015eb90 0x7f26000d2df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:58150/0 (socket says 172.21.15.27:58150) 2024-09-12T03:20:54.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:54.379+0000 7f25fffff640 1 -- 172.21.15.27:0/635170672 learned_addr learned my addr 172.21.15.27:0/635170672 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:20:54.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:54.380+0000 7f2604854640 1 --2- 172.21.15.27:0/635170672 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f26000d1550 0x7f26000b8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:54.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:54.380+0000 7f25ff7fe640 1 -- 172.21.15.27:0/635170672 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f26000d1550 msgr2=0x7f26000b8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:54.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:54.380+0000 7f25ff7fe640 1 --2- 172.21.15.27:0/635170672 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f26000d1550 0x7f26000b8190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:54.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:54.380+0000 7f25ff7fe640 1 -- 172.21.15.27:0/635170672 >> v1:172.21.15.27:6789/0 conn(0x7f260015eb90 legacy=0x7f26000d2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:54.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:54.380+0000 7f25feffd640 1 -- 172.21.15.27:0/635170672 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 169502211 0 0) 0x7f26000d3760 con 0x7f260015eb90 2024-09-12T03:20:54.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:54.380+0000 7f25ff7fe640 1 -- 172.21.15.27:0/635170672 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26000d3a40 con 0x7f26000d5200 2024-09-12T03:20:54.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:54.381+0000 7f25ff7fe640 1 --2- 172.21.15.27:0/635170672 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f26000d5200 0x7f26000d1010 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f25f40097a0 tx=0x7f25f402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20fa530920da40ba server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:54.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:54.381+0000 7f25feffd640 1 -- 172.21.15.27:0/635170672 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25f4037070 con 0x7f26000d5200 2024-09-12T03:20:54.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:54.381+0000 7f25feffd640 1 -- 172.21.15.27:0/635170672 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25f4032440 con 0x7f26000d5200 2024-09-12T03:20:54.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:54.381+0000 7f25feffd640 1 -- 172.21.15.27:0/635170672 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25f4032810 con 0x7f26000d5200 2024-09-12T03:20:54.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:54.381+0000 7f26062b7640 1 -- 172.21.15.27:0/635170672 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f26000d5200 msgr2=0x7f26000d1010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:54.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:54.381+0000 7f26062b7640 1 --2- 172.21.15.27:0/635170672 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f26000d5200 0x7f26000d1010 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f25f40097a0 tx=0x7f25f402a180 comp rx=0 tx=0).stop 2024-09-12T03:20:54.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:54.381+0000 7f26062b7640 1 -- 172.21.15.27:0/635170672 shutdown_connections 2024-09-12T03:20:54.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:54.381+0000 7f26062b7640 1 --2- 172.21.15.27:0/635170672 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f26000d1550 0x7f26000b8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:54.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:54.381+0000 7f26062b7640 1 --2- 172.21.15.27:0/635170672 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f26000d5200 0x7f26000d1010 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:54.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:54.381+0000 7f26062b7640 1 -- 172.21.15.27:0/635170672 >> 172.21.15.27:0/635170672 conn(0x7f260006dce0 msgr2=0x7f260006e0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:54.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:54.381+0000 7f26062b7640 1 -- 172.21.15.27:0/635170672 shutdown_connections 2024-09-12T03:20:54.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:54.381+0000 7f26062b7640 1 -- 172.21.15.27:0/635170672 wait complete. 2024-09-12T03:20:54.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:54.382+0000 7f26062b7640 1 Processor -- start 2024-09-12T03:20:54.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:54.383+0000 7f26062b7640 1 -- start start 2024-09-12T03:20:54.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:54.383+0000 7f26062b7640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f26000d1550 0x7f260014dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:54.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:54.383+0000 7f26062b7640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f26000d5200 0x7f2600159c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:54.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:54.383+0000 7f26062b7640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f260015eb90 0x7f26000caf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:54.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:54.383+0000 7f26062b7640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f26000d4000 con 0x7f26000d1550 2024-09-12T03:20:54.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:54.383+0000 7f26062b7640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f26000d3500 con 0x7f260015eb90 2024-09-12T03:20:54.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:54.383+0000 7f26062b7640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f26000d19a0 con 0x7f26000d5200 2024-09-12T03:20:54.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:54.384+0000 7f25fffff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f26000d1550 0x7f260014dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:54.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:54.384+0000 7f25fffff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f26000d1550 0x7f260014dbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55036/0 (socket says 172.21.15.27:55036) 2024-09-12T03:20:54.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:54.384+0000 7f25fffff640 1 -- 172.21.15.27:0/3670785184 learned_addr learned my addr 172.21.15.27:0/3670785184 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:20:54.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:54.384+0000 7f25ff7fe640 1 --2- 172.21.15.27:0/3670785184 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f26000d5200 0x7f2600159c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:54.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:54.384+0000 7f2604854640 1 --2- 172.21.15.27:0/3670785184 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f260015eb90 0x7f26000caf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:54.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:54.384+0000 7f25fffff640 1 -- 172.21.15.27:0/3670785184 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f26000d5200 msgr2=0x7f2600159c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:54.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:54.384+0000 7f25fffff640 1 --2- 172.21.15.27:0/3670785184 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f26000d5200 0x7f2600159c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:54.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:54.384+0000 7f25fffff640 1 -- 172.21.15.27:0/3670785184 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f260015eb90 msgr2=0x7f26000caf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:54.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:54.384+0000 7f25fffff640 1 --2- 172.21.15.27:0/3670785184 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f260015eb90 0x7f26000caf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:54.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:54.384+0000 7f25fffff640 1 -- 172.21.15.27:0/3670785184 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26001f43d0 con 0x7f26000d1550 2024-09-12T03:20:54.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:54.384+0000 7f2604854640 1 --2- 172.21.15.27:0/3670785184 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f260015eb90 0x7f26000caf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:20:54.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:54.384+0000 7f25fffff640 1 --2- 172.21.15.27:0/3670785184 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f26000d1550 0x7f260014dbd0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f25f000c430 tx=0x7f25f000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:54.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:54.384+0000 7f25fcff9640 1 -- 172.21.15.27:0/3670785184 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25f00074e0 con 0x7f26000d1550 2024-09-12T03:20:54.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:54.384+0000 7f25fcff9640 1 -- 172.21.15.27:0/3670785184 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25f00047e0 con 0x7f26000d1550 2024-09-12T03:20:54.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:54.384+0000 7f25fcff9640 1 -- 172.21.15.27:0/3670785184 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25f0004bd0 con 0x7f26000d1550 2024-09-12T03:20:54.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:54.384+0000 7f26062b7640 1 -- 172.21.15.27:0/3670785184 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26001f46b0 con 0x7f26000d1550 2024-09-12T03:20:54.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:54.384+0000 7f26062b7640 1 -- 172.21.15.27:0/3670785184 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26001f4c70 con 0x7f26000d1550 2024-09-12T03:20:54.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:54.385+0000 7f26062b7640 1 -- 172.21.15.27:0/3670785184 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2600160210 con 0x7f26000d1550 2024-09-12T03:20:54.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:54.386+0000 7f25fcff9640 1 -- 172.21.15.27:0/3670785184 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 90600+0+0 (secure 0 0 0) 0x7f25f0011070 con 0x7f26000d1550 2024-09-12T03:20:54.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:54.386+0000 7f25fcff9640 1 --2- 172.21.15.27:0/3670785184 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f25ec06bf50 0x7f25ec06e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:54.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:54.386+0000 7f25fcff9640 1 -- 172.21.15.27:0/3670785184 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5029+0+0 (secure 0 0 0) 0x7f25f008a980 con 0x7f26000d1550 2024-09-12T03:20:54.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:54.386+0000 7f25ff7fe640 1 --2- 172.21.15.27:0/3670785184 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f25ec06bf50 0x7f25ec06e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:54.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:54.387+0000 7f25ff7fe640 1 --2- 172.21.15.27:0/3670785184 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f25ec06bf50 0x7f25ec06e410 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f25f4034000 tx=0x7f25f402e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:54.390 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:54.391+0000 7f25fcff9640 1 -- 172.21.15.27:0/3670785184 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f25f00594c0 con 0x7f26000d1550 2024-09-12T03:20:54.604 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:54.604+0000 7f26062b7640 1 -- 172.21.15.27:0/3670785184 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f260014e110 con 0x7f26000d1550 2024-09-12T03:20:54.604 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:54.605+0000 7f25fcff9640 1 -- 172.21.15.27:0/3670785184 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) ==== 94+0+3905 (secure 0 0 0) 0x7f25f0058dc0 con 0x7f26000d1550 2024-09-12T03:20:54.605 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:20:54.605 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":53,"btime":"2024-09-12T03:20:49:735003+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5669,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/2036680565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2036680565},{"type":"v1","addr":"172.21.15.45:6834","nonce":2036680565}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":43},{"gid":6094,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/874218126","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":874218126},{"type":"v1","addr":"172.21.15.45:6835","nonce":874218126}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":43},{"gid":6693,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6834/658418980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":658418980},{"type":"v1","addr":"172.21.15.27:6834","nonce":658418980}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":51},{"gid":6756,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/3457531209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":3457531209},{"type":"v1","addr":"172.21.15.27:6835","nonce":3457531209}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":53}],"filesystems":[]} 2024-09-12T03:20:54.605 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 53 2024-09-12T03:20:54.607 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:54.607+0000 7f26062b7640 1 -- 172.21.15.27:0/3670785184 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f25ec06bf50 msgr2=0x7f25ec06e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:54.607 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:54.607+0000 7f26062b7640 1 --2- 172.21.15.27:0/3670785184 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f25ec06bf50 0x7f25ec06e410 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f25f4034000 tx=0x7f25f402e000 comp rx=0 tx=0).stop 2024-09-12T03:20:54.607 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:54.607+0000 7f26062b7640 1 -- 172.21.15.27:0/3670785184 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f26000d1550 msgr2=0x7f260014dbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:54.607 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:54.607+0000 7f26062b7640 1 --2- 172.21.15.27:0/3670785184 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f26000d1550 0x7f260014dbd0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f25f000c430 tx=0x7f25f000c8f0 comp rx=0 tx=0).stop 2024-09-12T03:20:54.607 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:54.607+0000 7f26062b7640 1 -- 172.21.15.27:0/3670785184 shutdown_connections 2024-09-12T03:20:54.608 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:54.607+0000 7f26062b7640 1 --2- 172.21.15.27:0/3670785184 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f260015eb90 0x7f26000caf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:54.608 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:54.607+0000 7f26062b7640 1 --2- 172.21.15.27:0/3670785184 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f26000d5200 0x7f2600159c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:54.608 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:54.607+0000 7f26062b7640 1 --2- 172.21.15.27:0/3670785184 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f25ec06bf50 0x7f25ec06e410 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:54.608 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:54.607+0000 7f26062b7640 1 --2- 172.21.15.27:0/3670785184 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f26000d1550 0x7f260014dbd0 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:54.608 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:54.607+0000 7f26062b7640 1 -- 172.21.15.27:0/3670785184 >> 172.21.15.27:0/3670785184 conn(0x7f260006dce0 msgr2=0x7f26000cfcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:54.608 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:54.607+0000 7f26062b7640 1 -- 172.21.15.27:0/3670785184 shutdown_connections 2024-09-12T03:20:54.608 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:54.607+0000 7f26062b7640 1 -- 172.21.15.27:0/3670785184 wait complete. 2024-09-12T03:20:54.619 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-12T03:20:54.619 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-12T03:20:54.619 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-12T03:20:54.619 INFO:teuthology.orchestra.run:waiting for 300 2024-09-12T03:20:54.620 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-12T03:20:54.620 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-12T03:20:54.620 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-12T03:20:54.620 INFO:teuthology.orchestra.run:waiting for 300 2024-09-12T03:20:54.620 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-12T03:20:54.620 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-12T03:20:54.621 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-12T03:20:54.621 INFO:teuthology.orchestra.run:waiting for 300 2024-09-12T03:20:54.621 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-12T03:20:54.621 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-12T03:20:54.621 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-12T03:20:54.621 INFO:teuthology.orchestra.run:waiting for 300 2024-09-12T03:20:54.629 INFO:tasks.ceph.mds.a:Stopped 2024-09-12T03:20:54.629 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-12T03:20:54.630 INFO:tasks.ceph.mds.c:Stopped 2024-09-12T03:20:54.630 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-12T03:20:54.633 INFO:tasks.ceph.mds.b:Stopped 2024-09-12T03:20:54.633 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-12T03:20:54.634 INFO:tasks.ceph.mds.d:Stopped 2024-09-12T03:20:54.634 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-12T03:20:54.672 INFO:tasks.ceph.mds.a:Started 2024-09-12T03:20:54.673 DEBUG:teuthology.parallel:result is None 2024-09-12T03:20:54.673 INFO:tasks.ceph.mds.c:Started 2024-09-12T03:20:54.673 DEBUG:teuthology.parallel:result is None 2024-09-12T03:20:54.675 INFO:tasks.ceph.mds.b:Started 2024-09-12T03:20:54.676 DEBUG:teuthology.parallel:result is None 2024-09-12T03:20:54.676 INFO:tasks.ceph.mds.d:Started 2024-09-12T03:20:54.677 DEBUG:teuthology.parallel:result is None 2024-09-12T03:20:54.677 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-12T03:20:54.751 INFO:tasks.ceph.mds.c.smithi027.stdout:starting mds.c at 2024-09-12T03:20:54.751 INFO:tasks.ceph.mds.a.smithi027.stdout:starting mds.a at 2024-09-12T03:20:54.763 INFO:tasks.ceph.mds.b.smithi045.stdout:starting mds.b at 2024-09-12T03:20:54.763 INFO:tasks.ceph.mds.d.smithi045.stdout:starting mds.d at 2024-09-12T03:20:54.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:54.786+0000 7fd9bdc27640 1 Processor -- start 2024-09-12T03:20:54.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:54.786+0000 7fd9bdc27640 1 -- start start 2024-09-12T03:20:54.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:54.787+0000 7fd9bdc27640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fd9b815eb90 0x7fd9b80d2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:54.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:54.787+0000 7fd9bdc27640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd9b80d3760 con 0x7fd9b80d1720 2024-09-12T03:20:54.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:54.787+0000 7fd9bdc27640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd9b80d3a40 con 0x7fd9b80d5200 2024-09-12T03:20:54.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:54.787+0000 7fd9bdc27640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fd9b80b7d20 con 0x7fd9b815eb90 2024-09-12T03:20:54.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:54.787+0000 7fd9b7fff640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fd9b80d1720 0x7fd9b80b8190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:58154/0 (socket says 172.21.15.27:58154) 2024-09-12T03:20:54.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:54.787+0000 7fd9b7fff640 1 -- 172.21.15.27:0/38238785 learned_addr learned my addr 172.21.15.27:0/38238785 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:20:54.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:54.787+0000 7fd9b77fe640 1 --2- 172.21.15.27:0/38238785 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fd9b815eb90 0x7fd9b80d2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:54.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:54.788+0000 7fd9b67fc640 1 -- 172.21.15.27:0/38238785 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1655533321 0 0) 0x7fd9b80d3760 con 0x7fd9b80d1720 2024-09-12T03:20:54.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:54.788+0000 7fd9b67fc640 1 -- 172.21.15.27:0/38238785 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd9a4003520 con 0x7fd9b80d1720 2024-09-12T03:20:54.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:54.788+0000 7fd9b67fc640 1 -- 172.21.15.27:0/38238785 <== mon.1 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 809817681 0 0) 0x7fd9b80d3a40 con 0x7fd9b80d5200 2024-09-12T03:20:54.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:54.788+0000 7fd9b67fc640 1 -- 172.21.15.27:0/38238785 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd9b80d3760 con 0x7fd9b80d5200 2024-09-12T03:20:54.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:54.788+0000 7fd9b77fe640 1 -- 172.21.15.27:0/38238785 >> v1:172.21.15.45:6789/0 conn(0x7fd9b80d5200 legacy=0x7fd9b80d1010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:54.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:54.788+0000 7fd9b77fe640 1 -- 172.21.15.27:0/38238785 >> v1:172.21.15.27:6789/0 conn(0x7fd9b80d1720 legacy=0x7fd9b80b8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:54.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:54.788+0000 7fd9b77fe640 1 -- 172.21.15.27:0/38238785 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9b80d3d20 con 0x7fd9b815eb90 2024-09-12T03:20:54.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:54.788+0000 7fd9b77fe640 1 --2- 172.21.15.27:0/38238785 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fd9b815eb90 0x7fd9b80d2df0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fd9a80097d0 tx=0x7fd9a802a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cda6fab56492bc67 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:54.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:54.789+0000 7fd9b67fc640 1 -- 172.21.15.27:0/38238785 <== mon.2 v2:172.21.15.45:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9a8037070 con 0x7fd9b815eb90 2024-09-12T03:20:54.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:54.789+0000 7fd9b67fc640 1 -- 172.21.15.27:0/38238785 <== mon.2 v2:172.21.15.45:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9a8032440 con 0x7fd9b815eb90 2024-09-12T03:20:54.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:54.789+0000 7fd9b67fc640 1 -- 172.21.15.27:0/38238785 <== mon.2 v2:172.21.15.45:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9a8032810 con 0x7fd9b815eb90 2024-09-12T03:20:54.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:54.789+0000 7fd9bdc27640 1 -- 172.21.15.27:0/38238785 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fd9b815eb90 msgr2=0x7fd9b80d2df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:54.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:54.789+0000 7fd9bdc27640 1 --2- 172.21.15.27:0/38238785 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fd9b815eb90 0x7fd9b80d2df0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fd9a80097d0 tx=0x7fd9a802a210 comp rx=0 tx=0).stop 2024-09-12T03:20:54.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:54.789+0000 7fd9bdc27640 1 -- 172.21.15.27:0/38238785 shutdown_connections 2024-09-12T03:20:54.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:54.789+0000 7fd9bdc27640 1 --2- 172.21.15.27:0/38238785 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fd9b815eb90 0x7fd9b80d2df0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:54.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:54.789+0000 7fd9bdc27640 1 -- 172.21.15.27:0/38238785 >> 172.21.15.27:0/38238785 conn(0x7fd9b806dce0 msgr2=0x7fd9b806e0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:54.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:54.789+0000 7fd9bdc27640 1 -- 172.21.15.27:0/38238785 shutdown_connections 2024-09-12T03:20:54.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:54.789+0000 7fd9bdc27640 1 -- 172.21.15.27:0/38238785 wait complete. 2024-09-12T03:20:54.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:54.790+0000 7fd9bdc27640 1 Processor -- start 2024-09-12T03:20:54.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:54.791+0000 7fd9bdc27640 1 -- start start 2024-09-12T03:20:54.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:54.791+0000 7fd9bdc27640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fd9b80d1720 0x7fd9b81585b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:54.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:54.791+0000 7fd9bdc27640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd9b80d5200 0x7fd9b814ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:54.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:54.791+0000 7fd9bdc27640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd9b815eb90 0x7fd9b8159c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:54.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:54.791+0000 7fd9bdc27640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fd9b80d3330 con 0x7fd9b815eb90 2024-09-12T03:20:54.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:54.791+0000 7fd9bdc27640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fd9b80d42e0 con 0x7fd9b80d5200 2024-09-12T03:20:54.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:54.791+0000 7fd9bdc27640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fd9b80d34b0 con 0x7fd9b80d1720 2024-09-12T03:20:54.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:54.791+0000 7fd9b7fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd9b815eb90 0x7fd9b8159c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:54.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:54.791+0000 7fd9b7fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd9b815eb90 0x7fd9b8159c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55088/0 (socket says 172.21.15.27:55088) 2024-09-12T03:20:54.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:54.791+0000 7fd9b77fe640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fd9b80d1720 0x7fd9b81585b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:54.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:54.791+0000 7fd9b7fff640 1 -- 172.21.15.27:0/1513259317 learned_addr learned my addr 172.21.15.27:0/1513259317 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:20:54.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:54.791+0000 7fd9b6ffd640 1 --2- 172.21.15.27:0/1513259317 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd9b80d5200 0x7fd9b814ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:54.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:54.792+0000 7fd9b7fff640 1 -- 172.21.15.27:0/1513259317 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fd9b80d1720 msgr2=0x7fd9b81585b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:54.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:54.792+0000 7fd9b7fff640 1 --2- 172.21.15.27:0/1513259317 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fd9b80d1720 0x7fd9b81585b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:54.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:54.792+0000 7fd9b7fff640 1 -- 172.21.15.27:0/1513259317 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd9b80d5200 msgr2=0x7fd9b814ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:54.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:54.792+0000 7fd9b7fff640 1 --2- 172.21.15.27:0/1513259317 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd9b80d5200 0x7fd9b814ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:54.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:54.792+0000 7fd9b7fff640 1 -- 172.21.15.27:0/1513259317 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9b8158bd0 con 0x7fd9b815eb90 2024-09-12T03:20:54.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:54.792+0000 7fd9b7fff640 1 --2- 172.21.15.27:0/1513259317 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd9b815eb90 0x7fd9b8159c70 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7fd9b000caa0 tx=0x7fd9b000cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:54.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:54.792+0000 7fd9affff640 1 -- 172.21.15.27:0/1513259317 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9b0017070 con 0x7fd9b815eb90 2024-09-12T03:20:54.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:54.792+0000 7fd9affff640 1 -- 172.21.15.27:0/1513259317 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9b000ac20 con 0x7fd9b815eb90 2024-09-12T03:20:54.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:54.792+0000 7fd9bdc27640 1 -- 172.21.15.27:0/1513259317 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9b8158eb0 con 0x7fd9b815eb90 2024-09-12T03:20:54.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:54.792+0000 7fd9affff640 1 -- 172.21.15.27:0/1513259317 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9b0012400 con 0x7fd9b815eb90 2024-09-12T03:20:54.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:54.792+0000 7fd9bdc27640 1 -- 172.21.15.27:0/1513259317 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9b8159470 con 0x7fd9b815eb90 2024-09-12T03:20:54.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:54.793+0000 7fd9bdc27640 1 -- 172.21.15.27:0/1513259317 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9b80d2df0 con 0x7fd9b815eb90 2024-09-12T03:20:54.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:54.794+0000 7fd9affff640 1 -- 172.21.15.27:0/1513259317 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 90600+0+0 (secure 0 0 0) 0x7fd9b001a070 con 0x7fd9b815eb90 2024-09-12T03:20:54.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:54.794+0000 7fd9affff640 1 --2- 172.21.15.27:0/1513259317 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fd99806bf80 0x7fd99806e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:54.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:54.794+0000 7fd9b77fe640 1 --2- 172.21.15.27:0/1513259317 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fd99806bf80 0x7fd99806e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:54.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:54.794+0000 7fd9affff640 1 -- 172.21.15.27:0/1513259317 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5029+0+0 (secure 0 0 0) 0x7fd9b008aa70 con 0x7fd9b815eb90 2024-09-12T03:20:54.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:54.795+0000 7fd9b77fe640 1 --2- 172.21.15.27:0/1513259317 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fd99806bf80 0x7fd99806e440 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fd9a8034000 tx=0x7fd9a802e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:54.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:54.799+0000 7fd9affff640 1 -- 172.21.15.27:0/1513259317 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fd9b0058ee0 con 0x7fd9b815eb90 2024-09-12T03:20:54.984 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:54.984+0000 7fd9bdc27640 1 -- 172.21.15.27:0/1513259317 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7fd9b8161fd0 con 0x7fd9b815eb90 2024-09-12T03:20:55.764 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:55.764+0000 7fd9affff640 1 -- 172.21.15.27:0/1513259317 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v94) ==== 97+0+0 (secure 0 0 0) 0x7fd9b005d1e0 con 0x7fd9b815eb90 2024-09-12T03:20:55.764 INFO:teuthology.orchestra.run.smithi027.stderr: removed all blocklist entries 2024-09-12T03:20:55.767 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:55.767+0000 7fd9bdc27640 1 -- 172.21.15.27:0/1513259317 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fd99806bf80 msgr2=0x7fd99806e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:55.767 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:55.767+0000 7fd9bdc27640 1 --2- 172.21.15.27:0/1513259317 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fd99806bf80 0x7fd99806e440 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fd9a8034000 tx=0x7fd9a802e000 comp rx=0 tx=0).stop 2024-09-12T03:20:55.768 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:55.767+0000 7fd9bdc27640 1 -- 172.21.15.27:0/1513259317 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd9b815eb90 msgr2=0x7fd9b8159c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:55.768 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:55.767+0000 7fd9bdc27640 1 --2- 172.21.15.27:0/1513259317 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd9b815eb90 0x7fd9b8159c70 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7fd9b000caa0 tx=0x7fd9b000cf60 comp rx=0 tx=0).stop 2024-09-12T03:20:55.768 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:55.767+0000 7fd9bdc27640 1 -- 172.21.15.27:0/1513259317 shutdown_connections 2024-09-12T03:20:55.768 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:55.767+0000 7fd9bdc27640 1 --2- 172.21.15.27:0/1513259317 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fd99806bf80 0x7fd99806e440 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:55.768 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:55.767+0000 7fd9bdc27640 1 --2- 172.21.15.27:0/1513259317 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd9b815eb90 0x7fd9b8159c70 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:55.768 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:55.767+0000 7fd9bdc27640 1 --2- 172.21.15.27:0/1513259317 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd9b80d5200 0x7fd9b814ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:55.768 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:55.767+0000 7fd9bdc27640 1 --2- 172.21.15.27:0/1513259317 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fd9b80d1720 0x7fd9b81585b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:55.768 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:55.767+0000 7fd9bdc27640 1 -- 172.21.15.27:0/1513259317 >> 172.21.15.27:0/1513259317 conn(0x7fd9b806dce0 msgr2=0x7fd9b80d1010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:55.768 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:55.768+0000 7fd9bdc27640 1 -- 172.21.15.27:0/1513259317 shutdown_connections 2024-09-12T03:20:55.768 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:55.768+0000 7fd9bdc27640 1 -- 172.21.15.27:0/1513259317 wait complete. 2024-09-12T03:20:55.783 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-12T03:20:55.892 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:55.892+0000 7f4c2579f640 1 Processor -- start 2024-09-12T03:20:55.892 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:55.892+0000 7f4c2579f640 1 -- start start 2024-09-12T03:20:55.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:55.893+0000 7f4c2579f640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4c2006f2d0 0x7f4c2006f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:55.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:55.893+0000 7f4c2579f640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f4c200725b0 0x7f4c200cf250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:55.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:55.893+0000 7f4c2579f640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4c2006fbf0 con 0x7f4c2015c740 2024-09-12T03:20:55.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:55.893+0000 7f4c2579f640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f4c20072b90 con 0x7f4c200725b0 2024-09-12T03:20:55.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:55.893+0000 7f4c2579f640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f4c20072e70 con 0x7f4c2006f2d0 2024-09-12T03:20:55.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:55.893+0000 7f4c1e7fc640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f4c2015c740 0x7f4c20071ea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:58164/0 (socket says 172.21.15.27:58164) 2024-09-12T03:20:55.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:55.893+0000 7f4c1e7fc640 1 -- 172.21.15.27:0/795477815 learned_addr learned my addr 172.21.15.27:0/795477815 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:20:55.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:55.893+0000 7f4c1effd640 1 --2- 172.21.15.27:0/795477815 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4c2006f2d0 0x7f4c2006f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:55.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:55.893+0000 7f4c1f7fe640 1 --2- 172.21.15.27:0/795477815 >> v2:172.21.15.45:3300/0 conn(0x7f4c200725b0 0x7f4c200cf250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:55.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:55.893+0000 7f4c1dffb640 1 -- 172.21.15.27:0/795477815 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3341426052 0 0) 0x7f4c2006fbf0 con 0x7f4c2015c740 2024-09-12T03:20:55.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:55.893+0000 7f4c1dffb640 1 -- 172.21.15.27:0/795477815 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4c0c003520 con 0x7f4c2015c740 2024-09-12T03:20:55.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:55.893+0000 7f4c1dffb640 1 -- 172.21.15.27:0/795477815 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 780390758 0 0) 0x7f4c0c003520 con 0x7f4c2015c740 2024-09-12T03:20:55.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:55.893+0000 7f4c1dffb640 1 -- 172.21.15.27:0/795477815 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4c2006f2d0 msgr2=0x7f4c2006f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:55.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:55.893+0000 7f4c1dffb640 1 --2- 172.21.15.27:0/795477815 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4c2006f2d0 0x7f4c2006f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:55.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:55.893+0000 7f4c1dffb640 1 -- 172.21.15.27:0/795477815 >> v2:172.21.15.45:3300/0 conn(0x7f4c200725b0 msgr2=0x7f4c200cf250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:55.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:55.893+0000 7f4c1dffb640 1 --2- 172.21.15.27:0/795477815 >> v2:172.21.15.45:3300/0 conn(0x7f4c200725b0 0x7f4c200cf250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:55.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:55.893+0000 7f4c1dffb640 1 -- 172.21.15.27:0/795477815 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c200cea50 con 0x7f4c2015c740 2024-09-12T03:20:55.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:55.893+0000 7f4c1dffb640 1 -- 172.21.15.27:0/795477815 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 847755213 0 0) 0x7f4c08001670 con 0x7f4c2015c740 2024-09-12T03:20:55.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:55.893+0000 7f4c1dffb640 1 -- 172.21.15.27:0/795477815 >> v1:172.21.15.27:6789/0 conn(0x7f4c2015c740 legacy=0x7f4c20071ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:55.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:55.893+0000 7f4c1dffb640 1 --2- 172.21.15.27:0/795477815 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4c0c0039b0 0x7f4c0c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:55.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:55.893+0000 7f4c1dffb640 1 --2- 172.21.15.27:0/795477815 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4c0c004800 0x7f4c0c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:55.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:55.893+0000 7f4c1dffb640 1 --2- 172.21.15.27:0/795477815 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4c2006f2d0 0x7f4c0c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:55.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:55.893+0000 7f4c1dffb640 1 -- 172.21.15.27:0/795477815 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f4c20072b90 con 0x7f4c0c0039b0 2024-09-12T03:20:55.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:55.893+0000 7f4c1dffb640 1 -- 172.21.15.27:0/795477815 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f4c20072e70 con 0x7f4c0c004800 2024-09-12T03:20:55.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:55.893+0000 7f4c1dffb640 1 -- 172.21.15.27:0/795477815 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f4c0c00ba40 con 0x7f4c2006f2d0 2024-09-12T03:20:55.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:55.894+0000 7f4c1effd640 1 --2- 172.21.15.27:0/795477815 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4c0c0039b0 0x7f4c0c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:55.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:55.894+0000 7f4c1e7fc640 1 --2- 172.21.15.27:0/795477815 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4c0c004800 0x7f4c0c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:55.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:55.894+0000 7f4c1f7fe640 1 --2- 172.21.15.27:0/795477815 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4c2006f2d0 0x7f4c0c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:55.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:55.894+0000 7f4c1effd640 1 -- 172.21.15.27:0/795477815 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4c2006f2d0 msgr2=0x7f4c0c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:55.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:55.894+0000 7f4c1effd640 1 --2- 172.21.15.27:0/795477815 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4c2006f2d0 0x7f4c0c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:55.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:55.894+0000 7f4c1effd640 1 -- 172.21.15.27:0/795477815 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4c0c004800 msgr2=0x7f4c0c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:55.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:55.894+0000 7f4c1effd640 1 --2- 172.21.15.27:0/795477815 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4c0c004800 0x7f4c0c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:55.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:55.894+0000 7f4c1effd640 1 -- 172.21.15.27:0/795477815 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c0c003520 con 0x7f4c0c0039b0 2024-09-12T03:20:55.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:55.894+0000 7f4c1e7fc640 1 --2- 172.21.15.27:0/795477815 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4c0c004800 0x7f4c0c004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:20:55.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:55.894+0000 7f4c1effd640 1 --2- 172.21.15.27:0/795477815 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4c0c0039b0 0x7f4c0c003dd0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f4c10009fc0 tx=0x7f4c0c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42cfa25b901c546 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:55.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:55.895+0000 7f4c1dffb640 1 -- 172.21.15.27:0/795477815 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c10009c10 con 0x7f4c0c0039b0 2024-09-12T03:20:55.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:55.895+0000 7f4c1dffb640 1 -- 172.21.15.27:0/795477815 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c10036070 con 0x7f4c0c0039b0 2024-09-12T03:20:55.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:55.895+0000 7f4c1dffb640 1 -- 172.21.15.27:0/795477815 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c10031880 con 0x7f4c0c0039b0 2024-09-12T03:20:55.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:55.895+0000 7f4c2579f640 1 -- 172.21.15.27:0/795477815 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4c0c0039b0 msgr2=0x7f4c0c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:55.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:55.895+0000 7f4c2579f640 1 --2- 172.21.15.27:0/795477815 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4c0c0039b0 0x7f4c0c003dd0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f4c10009fc0 tx=0x7f4c0c005830 comp rx=0 tx=0).stop 2024-09-12T03:20:55.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:55.895+0000 7f4c1f7fe640 1 -- 172.21.15.27:0/795477815 reap_dead start 2024-09-12T03:20:55.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:55.895+0000 7f4c2579f640 1 -- 172.21.15.27:0/795477815 shutdown_connections 2024-09-12T03:20:55.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:55.895+0000 7f4c2579f640 1 -- 172.21.15.27:0/795477815 >> 172.21.15.27:0/795477815 conn(0x7f4c200b2500 msgr2=0x7f4c200b28e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:55.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:55.895+0000 7f4c2579f640 1 -- 172.21.15.27:0/795477815 shutdown_connections 2024-09-12T03:20:55.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:55.895+0000 7f4c2579f640 1 -- 172.21.15.27:0/795477815 wait complete. 2024-09-12T03:20:55.902 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:55.897+0000 7f4c2579f640 1 Processor -- start 2024-09-12T03:20:55.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:55.897+0000 7f4c2579f640 1 -- start start 2024-09-12T03:20:55.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:55.897+0000 7f4c2579f640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4c0c0039b0 0x7f4c200b1c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:55.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:55.897+0000 7f4c2579f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4c20150cc0 0x7f4c20154e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:55.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:55.897+0000 7f4c2579f640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4c2014b960 0x7f4c20158010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:55.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:55.897+0000 7f4c2579f640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f4c200ce7f0 con 0x7f4c2014b960 2024-09-12T03:20:55.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:55.897+0000 7f4c2579f640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f4c200ce670 con 0x7f4c20150cc0 2024-09-12T03:20:55.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:55.897+0000 7f4c2579f640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f4c2006f6b0 con 0x7f4c0c0039b0 2024-09-12T03:20:55.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:55.898+0000 7f4c1f7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4c2014b960 0x7f4c20158010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:55.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:55.898+0000 7f4c1f7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4c2014b960 0x7f4c20158010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55102/0 (socket says 172.21.15.27:55102) 2024-09-12T03:20:55.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:55.898+0000 7f4c1f7fe640 1 -- 172.21.15.27:0/1936231687 learned_addr learned my addr 172.21.15.27:0/1936231687 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:20:55.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:55.898+0000 7f4c1e7fc640 1 --2- 172.21.15.27:0/1936231687 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4c20150cc0 0x7f4c20154e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:55.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:55.898+0000 7f4c1effd640 1 --2- 172.21.15.27:0/1936231687 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4c0c0039b0 0x7f4c200b1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:55.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:55.898+0000 7f4c1f7fe640 1 -- 172.21.15.27:0/1936231687 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4c0c0039b0 msgr2=0x7f4c200b1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:55.904 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:55.898+0000 7f4c1f7fe640 1 --2- 172.21.15.27:0/1936231687 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4c0c0039b0 0x7f4c200b1c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:55.904 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:55.898+0000 7f4c1f7fe640 1 -- 172.21.15.27:0/1936231687 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4c20150cc0 msgr2=0x7f4c20154e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:55.904 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:55.898+0000 7f4c1f7fe640 1 --2- 172.21.15.27:0/1936231687 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4c20150cc0 0x7f4c20154e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:55.904 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:55.898+0000 7f4c1f7fe640 1 -- 172.21.15.27:0/1936231687 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c20153de0 con 0x7f4c2014b960 2024-09-12T03:20:55.904 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:55.898+0000 7f4c1e7fc640 1 --2- 172.21.15.27:0/1936231687 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4c20150cc0 0x7f4c20154e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:20:55.904 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:55.898+0000 7f4c1effd640 1 --2- 172.21.15.27:0/1936231687 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4c0c0039b0 0x7f4c200b1c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:20:55.904 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:55.898+0000 7f4c1f7fe640 1 --2- 172.21.15.27:0/1936231687 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4c2014b960 0x7f4c20158010 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f4c200cdb40 tx=0x7f4c18009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:55.904 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:55.898+0000 7f4c177fe640 1 -- 172.21.15.27:0/1936231687 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c18019020 con 0x7f4c2014b960 2024-09-12T03:20:55.904 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:55.898+0000 7f4c177fe640 1 -- 172.21.15.27:0/1936231687 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c18003610 con 0x7f4c2014b960 2024-09-12T03:20:55.904 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:55.898+0000 7f4c2579f640 1 -- 172.21.15.27:0/1936231687 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c201540c0 con 0x7f4c2014b960 2024-09-12T03:20:55.904 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:55.898+0000 7f4c177fe640 1 -- 172.21.15.27:0/1936231687 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c18003a40 con 0x7f4c2014b960 2024-09-12T03:20:55.904 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:55.898+0000 7f4c2579f640 1 -- 172.21.15.27:0/1936231687 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c20154680 con 0x7f4c2014b960 2024-09-12T03:20:55.904 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:55.899+0000 7f4c2579f640 1 -- 172.21.15.27:0/1936231687 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c2006fa40 con 0x7f4c2014b960 2024-09-12T03:20:55.905 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:55.900+0000 7f4c177fe640 1 -- 172.21.15.27:0/1936231687 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 90600+0+0 (secure 0 0 0) 0x7f4c1801c020 con 0x7f4c2014b960 2024-09-12T03:20:55.905 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:55.900+0000 7f4c177fe640 1 --2- 172.21.15.27:0/1936231687 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f4c0006bf80 0x7f4c0006e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:55.905 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:55.900+0000 7f4c177fe640 1 -- 172.21.15.27:0/1936231687 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 4857+0+0 (secure 0 0 0) 0x7f4c1808b120 con 0x7f4c2014b960 2024-09-12T03:20:55.905 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:55.900+0000 7f4c1effd640 1 --2- 172.21.15.27:0/1936231687 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f4c0006bf80 0x7f4c0006e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:55.905 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:55.901+0000 7f4c1effd640 1 --2- 172.21.15.27:0/1936231687 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f4c0006bf80 0x7f4c0006e440 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f4c10035000 tx=0x7f4c1002f000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:55.905 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:55.905+0000 7f4c177fe640 1 -- 172.21.15.27:0/1936231687 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f4c18023020 con 0x7f4c2014b960 2024-09-12T03:20:56.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:56.146+0000 7f4c2579f640 1 -- 172.21.15.27:0/1936231687 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f4c2006e9e0 con 0x7f4c2014b960 2024-09-12T03:20:56.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:56.146+0000 7f4c177fe640 1 -- 172.21.15.27:0/1936231687 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v17) ==== 80+0+5304 (secure 0 0 0) 0x7f4c18059750 con 0x7f4c2014b960 2024-09-12T03:20:56.146 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:20:56.146 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-12T03:20:56.147 INFO:teuthology.orchestra.run.smithi027.stdout: "auth_dump": [ 2024-09-12T03:20:56.147 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:20:56.147 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "mds.a", 2024-09-12T03:20:56.147 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDUXOJmIhxjNxAAaGg8SwtW6NRe/lIgNCV20w==", 2024-09-12T03:20:56.147 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:20:56.147 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow", 2024-09-12T03:20:56.147 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow *", 2024-09-12T03:20:56.147 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow *", 2024-09-12T03:20:56.147 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2024-09-12T03:20:56.147 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:20:56.147 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:56.147 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:20:56.147 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "mds.b", 2024-09-12T03:20:56.147 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDVXOJm63kWCBAAzjfOsBsWUI+IAsuZ49yEkw==", 2024-09-12T03:20:56.148 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:20:56.148 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow", 2024-09-12T03:20:56.148 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow *", 2024-09-12T03:20:56.148 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow *", 2024-09-12T03:20:56.148 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2024-09-12T03:20:56.148 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:20:56.148 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:56.148 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:20:56.148 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "mds.c", 2024-09-12T03:20:56.148 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDVXOJm0sX7AhAAdynCcho2a/AgFF9Cw3+vAw==", 2024-09-12T03:20:56.148 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:20:56.148 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow", 2024-09-12T03:20:56.148 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow *", 2024-09-12T03:20:56.149 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow *", 2024-09-12T03:20:56.149 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2024-09-12T03:20:56.149 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:20:56.149 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:56.149 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:20:56.149 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "mds.d", 2024-09-12T03:20:56.149 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDVXOJm7Dd8DxAANHYmvKgpRZ6sVfXDSgPHWw==", 2024-09-12T03:20:56.149 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:20:56.149 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow", 2024-09-12T03:20:56.149 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow *", 2024-09-12T03:20:56.149 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow *", 2024-09-12T03:20:56.149 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2024-09-12T03:20:56.149 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:20:56.150 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:56.150 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:20:56.150 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "osd.0", 2024-09-12T03:20:56.150 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDYXOJmJTVYLxAAmo8kxRjaiwTT44dZLZlq4A==", 2024-09-12T03:20:56.150 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:20:56.150 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow profile osd", 2024-09-12T03:20:56.150 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile osd", 2024-09-12T03:20:56.150 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2024-09-12T03:20:56.150 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:20:56.150 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:56.150 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:20:56.150 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "osd.1", 2024-09-12T03:20:56.150 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDcXOJmYV0BFBAAmoGX9gf9yLoE/qkWFnv9tg==", 2024-09-12T03:20:56.150 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:20:56.151 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow profile osd", 2024-09-12T03:20:56.151 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile osd", 2024-09-12T03:20:56.151 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2024-09-12T03:20:56.151 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:20:56.151 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:56.151 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:20:56.151 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "osd.2", 2024-09-12T03:20:56.151 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDfXOJmEetiJRAAXk+AvfTfhjsB6T2UB4Z2/w==", 2024-09-12T03:20:56.151 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:20:56.151 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow profile osd", 2024-09-12T03:20:56.152 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile osd", 2024-09-12T03:20:56.152 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2024-09-12T03:20:56.152 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:20:56.152 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:56.152 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:20:56.152 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "osd.3", 2024-09-12T03:20:56.152 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDiXOJmrQbPJxAAQsi2B5a1LnPTPGZVdo7j7w==", 2024-09-12T03:20:56.152 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:20:56.152 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow profile osd", 2024-09-12T03:20:56.152 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile osd", 2024-09-12T03:20:56.153 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2024-09-12T03:20:56.153 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:20:56.153 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:56.153 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:20:56.153 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "osd.4", 2024-09-12T03:20:56.153 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDnXOJmoMr1NxAAv3u2q8hYt9w//9gz+N7UAQ==", 2024-09-12T03:20:56.153 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:20:56.153 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow profile osd", 2024-09-12T03:20:56.153 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile osd", 2024-09-12T03:20:56.153 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2024-09-12T03:20:56.153 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:20:56.153 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:56.153 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:20:56.154 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "osd.5", 2024-09-12T03:20:56.154 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDqXOJmxpUPOxAAxLH6RJsyLOZ2XMfzM0fQ1g==", 2024-09-12T03:20:56.154 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:20:56.154 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow profile osd", 2024-09-12T03:20:56.154 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile osd", 2024-09-12T03:20:56.154 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2024-09-12T03:20:56.154 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:20:56.154 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:56.154 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:20:56.154 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "osd.6", 2024-09-12T03:20:56.154 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDuXOJmuP0+LxAAMHoz5eSO81hPtkdBIS2cig==", 2024-09-12T03:20:56.155 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:20:56.155 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow profile osd", 2024-09-12T03:20:56.155 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile osd", 2024-09-12T03:20:56.155 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2024-09-12T03:20:56.155 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:20:56.155 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:56.155 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:20:56.155 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "osd.7", 2024-09-12T03:20:56.155 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDxXOJmuIC/MRAAPDM+lAHqdoQyGVODwjFp+Q==", 2024-09-12T03:20:56.155 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:20:56.155 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow profile osd", 2024-09-12T03:20:56.155 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile osd", 2024-09-12T03:20:56.155 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2024-09-12T03:20:56.155 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:20:56.156 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:56.156 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:20:56.156 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "client.0", 2024-09-12T03:20:56.156 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDVXOJmLm+5ExAAECM4CKfuVElmp/n0b02mXg==", 2024-09-12T03:20:56.156 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:20:56.156 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow", 2024-09-12T03:20:56.156 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow r", 2024-09-12T03:20:56.156 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-12T03:20:56.156 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:20:56.156 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:56.156 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:20:56.156 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "client.1", 2024-09-12T03:20:56.156 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDVXOJm61/FHBAALCQouXySfQeZ5DESGmqkzA==", 2024-09-12T03:20:56.158 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:20:56.158 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow", 2024-09-12T03:20:56.158 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow r", 2024-09-12T03:20:56.158 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow rw", 2024-09-12T03:20:56.158 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow rwx" 2024-09-12T03:20:56.158 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:20:56.158 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:56.159 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:20:56.159 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "client.2", 2024-09-12T03:20:56.159 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDVXOJmq/ZkGRAAOzmQzEIhb4b/YTxayc34Bw==", 2024-09-12T03:20:56.159 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:20:56.160 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow", 2024-09-12T03:20:56.161 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow r", 2024-09-12T03:20:56.161 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow rw", 2024-09-12T03:20:56.161 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow rwx" 2024-09-12T03:20:56.161 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:20:56.161 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:56.161 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:20:56.161 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "client.3", 2024-09-12T03:20:56.161 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDVXOJm+DlzIhAATQKZe+Zyk0MM8efTa6g9PA==", 2024-09-12T03:20:56.161 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:20:56.161 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow", 2024-09-12T03:20:56.161 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow r", 2024-09-12T03:20:56.161 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow rw", 2024-09-12T03:20:56.161 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow rwx" 2024-09-12T03:20:56.162 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:20:56.162 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:56.162 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:20:56.162 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "client.admin", 2024-09-12T03:20:56.162 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDUXOJm+JGWGhAApTQw9RtJWd/J5QzQaUFXvg==", 2024-09-12T03:20:56.162 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:20:56.162 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow *", 2024-09-12T03:20:56.162 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow *", 2024-09-12T03:20:56.162 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow *", 2024-09-12T03:20:56.162 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2024-09-12T03:20:56.162 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:20:56.162 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:56.162 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:20:56.163 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "mgr.x", 2024-09-12T03:20:56.163 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDUXOJmZcsONBAAMTY2KykuHaEkv1t6Vz5KjA==", 2024-09-12T03:20:56.163 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:20:56.163 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow *", 2024-09-12T03:20:56.163 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile mgr", 2024-09-12T03:20:56.163 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2024-09-12T03:20:56.163 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:20:56.163 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:20:56.163 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:20:56.163 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "mgr.y", 2024-09-12T03:20:56.163 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDUXOJmJOgZMBAAQ/JTxfhox7HgxBMtsvUCBg==", 2024-09-12T03:20:56.163 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:20:56.163 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow *", 2024-09-12T03:20:56.164 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile mgr", 2024-09-12T03:20:56.164 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2024-09-12T03:20:56.164 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:20:56.164 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:20:56.164 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-12T03:20:56.164 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-12T03:20:56.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:56.148+0000 7f4c2579f640 1 -- 172.21.15.27:0/1936231687 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f4c0006bf80 msgr2=0x7f4c0006e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:56.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:56.148+0000 7f4c2579f640 1 --2- 172.21.15.27:0/1936231687 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f4c0006bf80 0x7f4c0006e440 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f4c10035000 tx=0x7f4c1002f000 comp rx=0 tx=0).stop 2024-09-12T03:20:56.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:56.148+0000 7f4c2579f640 1 -- 172.21.15.27:0/1936231687 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4c2014b960 msgr2=0x7f4c20158010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:56.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:56.148+0000 7f4c2579f640 1 --2- 172.21.15.27:0/1936231687 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4c2014b960 0x7f4c20158010 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f4c200cdb40 tx=0x7f4c18009f90 comp rx=0 tx=0).stop 2024-09-12T03:20:56.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:56.148+0000 7f4c2579f640 1 -- 172.21.15.27:0/1936231687 shutdown_connections 2024-09-12T03:20:56.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:56.148+0000 7f4c2579f640 1 --2- 172.21.15.27:0/1936231687 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f4c0006bf80 0x7f4c0006e440 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:56.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:56.148+0000 7f4c2579f640 1 --2- 172.21.15.27:0/1936231687 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4c2014b960 0x7f4c20158010 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:56.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:56.148+0000 7f4c2579f640 1 --2- 172.21.15.27:0/1936231687 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4c20150cc0 0x7f4c20154e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:56.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:56.148+0000 7f4c2579f640 1 --2- 172.21.15.27:0/1936231687 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4c0c0039b0 0x7f4c200b1c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:56.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:56.148+0000 7f4c2579f640 1 -- 172.21.15.27:0/1936231687 >> 172.21.15.27:0/1936231687 conn(0x7f4c200b2500 msgr2=0x7f4c2006d4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:56.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:56.148+0000 7f4c2579f640 1 -- 172.21.15.27:0/1936231687 shutdown_connections 2024-09-12T03:20:56.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:56.149+0000 7f4c2579f640 1 -- 172.21.15.27:0/1936231687 wait complete. 2024-09-12T03:20:56.167 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-12T03:20:56.269 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:56.269+0000 7f5e827c4640 1 Processor -- start 2024-09-12T03:20:56.269 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:56.269+0000 7f5e827c4640 1 -- start start 2024-09-12T03:20:56.272 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:56.269+0000 7f5e827c4640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f5e7c0d5200 0x7f5e7c0d1010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:56.272 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:56.269+0000 7f5e827c4640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5e7c0d19c0 con 0x7f5e7c0d1550 2024-09-12T03:20:56.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:56.269+0000 7f5e827c4640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5e7c0d3760 con 0x7f5e7c15eb90 2024-09-12T03:20:56.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:56.269+0000 7f5e827c4640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f5e7c0b7d20 con 0x7f5e7c0d5200 2024-09-12T03:20:56.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:56.270+0000 7f5e80d61640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f5e7c0d1550 0x7f5e7c0b8190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:58172/0 (socket says 172.21.15.27:58172) 2024-09-12T03:20:56.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:56.270+0000 7f5e80d61640 1 -- 172.21.15.27:0/2636497196 learned_addr learned my addr 172.21.15.27:0/2636497196 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:20:56.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:56.270+0000 7f5e7b7fe640 1 --2- 172.21.15.27:0/2636497196 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f5e7c0d5200 0x7f5e7c0d1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:56.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:56.270+0000 7f5e7affd640 1 -- 172.21.15.27:0/2636497196 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4078957151 0 0) 0x7f5e7c0d19c0 con 0x7f5e7c0d1550 2024-09-12T03:20:56.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:56.270+0000 7f5e7affd640 1 -- 172.21.15.27:0/2636497196 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5e68003520 con 0x7f5e7c0d1550 2024-09-12T03:20:56.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:56.270+0000 7f5e7affd640 1 -- 172.21.15.27:0/2636497196 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3358529552 0 0) 0x7f5e68003520 con 0x7f5e7c0d1550 2024-09-12T03:20:56.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:56.270+0000 7f5e7affd640 1 -- 172.21.15.27:0/2636497196 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f5e7c0d5200 msgr2=0x7f5e7c0d1010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:56.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:56.270+0000 7f5e7affd640 1 --2- 172.21.15.27:0/2636497196 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f5e7c0d5200 0x7f5e7c0d1010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:56.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:56.270+0000 7f5e7affd640 1 -- 172.21.15.27:0/2636497196 >> v1:172.21.15.45:6789/0 conn(0x7f5e7c15eb90 legacy=0x7f5e7c0d2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:56.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:56.270+0000 7f5e7affd640 1 -- 172.21.15.27:0/2636497196 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e7c0d3a40 con 0x7f5e7c0d1550 2024-09-12T03:20:56.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:56.270+0000 7f5e7affd640 1 -- 172.21.15.27:0/2636497196 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 847755213 0 0) 0x7f5e740019f0 con 0x7f5e7c0d1550 2024-09-12T03:20:56.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:56.270+0000 7f5e7affd640 1 -- 172.21.15.27:0/2636497196 >> v1:172.21.15.27:6789/0 conn(0x7f5e7c0d1550 legacy=0x7f5e7c0b8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:56.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:56.270+0000 7f5e7affd640 1 --2- 172.21.15.27:0/2636497196 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5e680039e0 0x7f5e68003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:56.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:56.270+0000 7f5e7affd640 1 --2- 172.21.15.27:0/2636497196 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5e68004390 0x7f5e68004860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:56.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:56.270+0000 7f5e7affd640 1 --2- 172.21.15.27:0/2636497196 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f5e7c0d5200 0x7f5e680050c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:56.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:56.270+0000 7f5e7affd640 1 -- 172.21.15.27:0/2636497196 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f5e6800b9c0 con 0x7f5e68004390 2024-09-12T03:20:56.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:56.270+0000 7f5e7affd640 1 -- 172.21.15.27:0/2636497196 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f5e6800bb40 con 0x7f5e680039e0 2024-09-12T03:20:56.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:56.270+0000 7f5e7affd640 1 -- 172.21.15.27:0/2636497196 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f5e6800bd40 con 0x7f5e7c0d5200 2024-09-12T03:20:56.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:56.270+0000 7f5e7b7fe640 1 --2- 172.21.15.27:0/2636497196 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5e68004390 0x7f5e68004860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:56.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:56.270+0000 7f5e7bfff640 1 --2- 172.21.15.27:0/2636497196 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5e680039e0 0x7f5e68003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:56.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:56.270+0000 7f5e80d61640 1 --2- 172.21.15.27:0/2636497196 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f5e7c0d5200 0x7f5e680050c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:56.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:56.271+0000 7f5e7b7fe640 1 -- 172.21.15.27:0/2636497196 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f5e7c0d5200 msgr2=0x7f5e680050c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:56.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:56.271+0000 7f5e7b7fe640 1 --2- 172.21.15.27:0/2636497196 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f5e7c0d5200 0x7f5e680050c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:56.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:56.271+0000 7f5e7b7fe640 1 -- 172.21.15.27:0/2636497196 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5e680039e0 msgr2=0x7f5e68003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:56.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:56.271+0000 7f5e7b7fe640 1 --2- 172.21.15.27:0/2636497196 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5e680039e0 0x7f5e68003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:56.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:56.271+0000 7f5e7b7fe640 1 -- 172.21.15.27:0/2636497196 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e68003520 con 0x7f5e68004390 2024-09-12T03:20:56.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:56.271+0000 7f5e80d61640 1 --2- 172.21.15.27:0/2636497196 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f5e7c0d5200 0x7f5e680050c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:20:56.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:56.271+0000 7f5e7b7fe640 1 --2- 172.21.15.27:0/2636497196 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5e68004390 0x7f5e68004860 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f5e64009a90 tx=0x7f5e68005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10e78dd3e082ce46 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:56.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:56.271+0000 7f5e7affd640 1 -- 172.21.15.27:0/2636497196 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e64036070 con 0x7f5e68004390 2024-09-12T03:20:56.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:56.271+0000 7f5e7affd640 1 -- 172.21.15.27:0/2636497196 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e64031400 con 0x7f5e68004390 2024-09-12T03:20:56.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:56.271+0000 7f5e7affd640 1 -- 172.21.15.27:0/2636497196 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e64009690 con 0x7f5e68004390 2024-09-12T03:20:56.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:56.272+0000 7f5e827c4640 1 -- 172.21.15.27:0/2636497196 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5e68004390 msgr2=0x7f5e68004860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:56.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:56.272+0000 7f5e827c4640 1 --2- 172.21.15.27:0/2636497196 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5e68004390 0x7f5e68004860 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f5e64009a90 tx=0x7f5e68005830 comp rx=0 tx=0).stop 2024-09-12T03:20:56.276 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:56.272+0000 7f5e80d61640 1 -- 172.21.15.27:0/2636497196 reap_dead start 2024-09-12T03:20:56.276 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:56.272+0000 7f5e827c4640 1 -- 172.21.15.27:0/2636497196 shutdown_connections 2024-09-12T03:20:56.276 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:56.272+0000 7f5e827c4640 1 -- 172.21.15.27:0/2636497196 >> 172.21.15.27:0/2636497196 conn(0x7f5e7c06dce0 msgr2=0x7f5e7c06e0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:56.276 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:56.272+0000 7f5e827c4640 1 -- 172.21.15.27:0/2636497196 shutdown_connections 2024-09-12T03:20:56.276 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:56.272+0000 7f5e827c4640 1 -- 172.21.15.27:0/2636497196 wait complete. 2024-09-12T03:20:56.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:56.274+0000 7f5e827c4640 1 Processor -- start 2024-09-12T03:20:56.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:56.274+0000 7f5e827c4640 1 -- start start 2024-09-12T03:20:56.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:56.275+0000 7f5e827c4640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5e68004390 0x7f5e7c15ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:56.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:56.275+0000 7f5e827c4640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f5e7c0cced0 0x7f5e7c0cd350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:56.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:56.275+0000 7f5e827c4640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5e7c1e8160 0x7f5e7c1e85e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:56.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:56.275+0000 7f5e827c4640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f5e7c0d3500 con 0x7f5e68004390 2024-09-12T03:20:56.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:56.275+0000 7f5e827c4640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f5e7c0d42e0 con 0x7f5e7c1e8160 2024-09-12T03:20:56.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:56.275+0000 7f5e827c4640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f5e7c0d4000 con 0x7f5e7c0cced0 2024-09-12T03:20:56.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:56.275+0000 7f5e7bfff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5e68004390 0x7f5e7c15ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:56.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:56.275+0000 7f5e7bfff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5e68004390 0x7f5e7c15ada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55118/0 (socket says 172.21.15.27:55118) 2024-09-12T03:20:56.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:56.275+0000 7f5e7bfff640 1 -- 172.21.15.27:0/2491503050 learned_addr learned my addr 172.21.15.27:0/2491503050 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:20:56.280 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:56.275+0000 7f5e7b7fe640 1 --2- 172.21.15.27:0/2491503050 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f5e7c0cced0 0x7f5e7c0cd350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:56.280 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:56.275+0000 7f5e80d61640 1 --2- 172.21.15.27:0/2491503050 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5e7c1e8160 0x7f5e7c1e85e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:56.280 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:56.275+0000 7f5e7bfff640 1 -- 172.21.15.27:0/2491503050 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f5e7c0cced0 msgr2=0x7f5e7c0cd350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:56.280 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:56.275+0000 7f5e7bfff640 1 --2- 172.21.15.27:0/2491503050 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f5e7c0cced0 0x7f5e7c0cd350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:56.280 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:56.275+0000 7f5e7bfff640 1 -- 172.21.15.27:0/2491503050 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5e7c1e8160 msgr2=0x7f5e7c1e85e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:56.280 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:56.275+0000 7f5e7bfff640 1 --2- 172.21.15.27:0/2491503050 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5e7c1e8160 0x7f5e7c1e85e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:56.280 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:56.275+0000 7f5e7bfff640 1 -- 172.21.15.27:0/2491503050 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e7c1e8b20 con 0x7f5e68004390 2024-09-12T03:20:56.280 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:56.275+0000 7f5e80d61640 1 --2- 172.21.15.27:0/2491503050 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5e7c1e8160 0x7f5e7c1e85e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:20:56.280 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:56.275+0000 7f5e7bfff640 1 --2- 172.21.15.27:0/2491503050 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5e68004390 0x7f5e7c15ada0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f5e6c00f7c0 tx=0x7f5e6c00fc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:56.280 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:56.275+0000 7f5e78ff9640 1 -- 172.21.15.27:0/2491503050 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e6c0052e0 con 0x7f5e68004390 2024-09-12T03:20:56.280 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:56.275+0000 7f5e78ff9640 1 -- 172.21.15.27:0/2491503050 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e6c018070 con 0x7f5e68004390 2024-09-12T03:20:56.280 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:56.275+0000 7f5e78ff9640 1 -- 172.21.15.27:0/2491503050 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e6c00d910 con 0x7f5e68004390 2024-09-12T03:20:56.280 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:56.276+0000 7f5e827c4640 1 -- 172.21.15.27:0/2491503050 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e7c15b2e0 con 0x7f5e68004390 2024-09-12T03:20:56.281 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:56.276+0000 7f5e827c4640 1 -- 172.21.15.27:0/2491503050 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e7c0c8620 con 0x7f5e68004390 2024-09-12T03:20:56.281 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:56.276+0000 7f5e827c4640 1 -- 172.21.15.27:0/2491503050 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e7c15f0e0 con 0x7f5e68004390 2024-09-12T03:20:56.281 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:56.277+0000 7f5e78ff9640 1 -- 172.21.15.27:0/2491503050 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 90600+0+0 (secure 0 0 0) 0x7f5e6c00dab0 con 0x7f5e68004390 2024-09-12T03:20:56.281 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:56.278+0000 7f5e78ff9640 1 --2- 172.21.15.27:0/2491503050 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f5e5c06bf80 0x7f5e5c06e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:56.281 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:56.278+0000 7f5e78ff9640 1 -- 172.21.15.27:0/2491503050 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 4857+0+0 (secure 0 0 0) 0x7f5e6c08aaf0 con 0x7f5e68004390 2024-09-12T03:20:56.281 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:56.278+0000 7f5e7b7fe640 1 --2- 172.21.15.27:0/2491503050 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f5e5c06bf80 0x7f5e5c06e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:56.281 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:56.278+0000 7f5e7b7fe640 1 --2- 172.21.15.27:0/2491503050 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f5e5c06bf80 0x7f5e5c06e440 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f5e64009a90 tx=0x7f5e64008030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:56.282 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:56.282+0000 7f5e78ff9640 1 -- 172.21.15.27:0/2491503050 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f5e6c0596e0 con 0x7f5e68004390 2024-09-12T03:20:56.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:56.461+0000 7f5e827c4640 1 -- 172.21.15.27:0/2491503050 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f5e7c0b9a40 con 0x7f5e68004390 2024-09-12T03:20:56.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:56.461+0000 7f5e78ff9640 1 -- 172.21.15.27:0/2491503050 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v96) ==== 84+0+65 (secure 0 0 0) 0x7f5e6c058fe0 con 0x7f5e68004390 2024-09-12T03:20:56.461 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:20:56.462 INFO:teuthology.orchestra.run.smithi027.stdout:[ 2024-09-12T03:20:56.462 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:20:56.462 INFO:teuthology.orchestra.run.smithi027.stdout: "poolnum": 1, 2024-09-12T03:20:56.462 INFO:teuthology.orchestra.run.smithi027.stdout: "poolname": ".mgr" 2024-09-12T03:20:56.462 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:20:56.462 INFO:teuthology.orchestra.run.smithi027.stdout:] 2024-09-12T03:20:56.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:56.464+0000 7f5e827c4640 1 -- 172.21.15.27:0/2491503050 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f5e5c06bf80 msgr2=0x7f5e5c06e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:56.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:56.464+0000 7f5e827c4640 1 --2- 172.21.15.27:0/2491503050 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f5e5c06bf80 0x7f5e5c06e440 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f5e64009a90 tx=0x7f5e64008030 comp rx=0 tx=0).stop 2024-09-12T03:20:56.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:56.464+0000 7f5e827c4640 1 -- 172.21.15.27:0/2491503050 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5e68004390 msgr2=0x7f5e7c15ada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:56.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:56.464+0000 7f5e827c4640 1 --2- 172.21.15.27:0/2491503050 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5e68004390 0x7f5e7c15ada0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f5e6c00f7c0 tx=0x7f5e6c00fc80 comp rx=0 tx=0).stop 2024-09-12T03:20:56.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:56.464+0000 7f5e827c4640 1 -- 172.21.15.27:0/2491503050 shutdown_connections 2024-09-12T03:20:56.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:56.464+0000 7f5e827c4640 1 --2- 172.21.15.27:0/2491503050 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5e7c1e8160 0x7f5e7c1e85e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:56.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:56.464+0000 7f5e827c4640 1 --2- 172.21.15.27:0/2491503050 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f5e7c0cced0 0x7f5e7c0cd350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:56.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:56.464+0000 7f5e827c4640 1 --2- 172.21.15.27:0/2491503050 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f5e5c06bf80 0x7f5e5c06e440 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:56.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:56.464+0000 7f5e827c4640 1 --2- 172.21.15.27:0/2491503050 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5e68004390 0x7f5e7c15ada0 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:56.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:56.464+0000 7f5e827c4640 1 -- 172.21.15.27:0/2491503050 >> 172.21.15.27:0/2491503050 conn(0x7f5e7c06dce0 msgr2=0x7f5e7c0d1f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:56.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:56.464+0000 7f5e827c4640 1 -- 172.21.15.27:0/2491503050 shutdown_connections 2024-09-12T03:20:56.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:56.464+0000 7f5e827c4640 1 -- 172.21.15.27:0/2491503050 wait complete. 2024-09-12T03:20:56.474 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-12T03:20:56.475 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-12T03:20:56.581 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:56.581+0000 7f43d3b11640 1 Processor -- start 2024-09-12T03:20:56.581 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:56.581+0000 7f43d3b11640 1 -- start start 2024-09-12T03:20:56.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:56.582+0000 7f43d3b11640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f43cc15ec30 0x7f43cc0b3900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:56.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:56.582+0000 7f43d3b11640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f43cc0af9d0 0x7f43cc0b3e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:56.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:56.582+0000 7f43d3b11640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f43cc0b4380 0x7f43cc15f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:56.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:56.582+0000 7f43d3b11640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f43cc0d02d0 con 0x7f43cc0af9d0 2024-09-12T03:20:56.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:56.582+0000 7f43d3b11640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f43cc06c650 con 0x7f43cc0b4380 2024-09-12T03:20:56.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:56.582+0000 7f43d3b11640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f43cc0ce5e0 con 0x7f43cc15ec30 2024-09-12T03:20:56.584 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:56.582+0000 7f43d10ac640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f43cc0af9d0 0x7f43cc0b3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:56.584 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:56.582+0000 7f43d10ac640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f43cc0af9d0 0x7f43cc0b3e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55122/0 (socket says 172.21.15.27:55122) 2024-09-12T03:20:56.584 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:56.582+0000 7f43d10ac640 1 -- 172.21.15.27:0/1880981261 learned_addr learned my addr 172.21.15.27:0/1880981261 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:20:56.584 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:56.582+0000 7f43d18ad640 1 --2- 172.21.15.27:0/1880981261 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f43cc15ec30 0x7f43cc0b3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:56.584 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:56.582+0000 7f43d20ae640 1 --2- 172.21.15.27:0/1880981261 >> v2:172.21.15.45:3300/0 conn(0x7f43cc0b4380 0x7f43cc15f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:56.584 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:56.582+0000 7f43d10ac640 1 -- 172.21.15.27:0/1880981261 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f43cc15ec30 msgr2=0x7f43cc0b3900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:56.584 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:56.582+0000 7f43d10ac640 1 --2- 172.21.15.27:0/1880981261 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f43cc15ec30 0x7f43cc0b3900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:56.584 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:56.582+0000 7f43d10ac640 1 -- 172.21.15.27:0/1880981261 >> v2:172.21.15.45:3300/0 conn(0x7f43cc0b4380 msgr2=0x7f43cc15f2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:56.584 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:56.582+0000 7f43d10ac640 1 --2- 172.21.15.27:0/1880981261 >> v2:172.21.15.45:3300/0 conn(0x7f43cc0b4380 0x7f43cc15f2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:56.584 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:56.582+0000 7f43d10ac640 1 -- 172.21.15.27:0/1880981261 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43cc0ceac0 con 0x7f43cc0af9d0 2024-09-12T03:20:56.584 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:56.582+0000 7f43d18ad640 1 --2- 172.21.15.27:0/1880981261 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f43cc15ec30 0x7f43cc0b3900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:20:56.584 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:56.583+0000 7f43d10ac640 1 --2- 172.21.15.27:0/1880981261 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43cc0af9d0 0x7f43cc0b3e40 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f43cc051050 tx=0x7f43bc02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a07254bee276aca server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:56.584 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:56.583+0000 7f43d08ab640 1 -- 172.21.15.27:0/1880981261 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43bc037070 con 0x7f43cc0af9d0 2024-09-12T03:20:56.585 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:56.583+0000 7f43d08ab640 1 -- 172.21.15.27:0/1880981261 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43bc032440 con 0x7f43cc0af9d0 2024-09-12T03:20:56.585 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:56.583+0000 7f43d08ab640 1 -- 172.21.15.27:0/1880981261 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43bc032810 con 0x7f43cc0af9d0 2024-09-12T03:20:56.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:56.584+0000 7f43d3b11640 1 -- 172.21.15.27:0/1880981261 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43cc0af9d0 msgr2=0x7f43cc0b3e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:56.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:56.584+0000 7f43d3b11640 1 --2- 172.21.15.27:0/1880981261 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43cc0af9d0 0x7f43cc0b3e40 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f43cc051050 tx=0x7f43bc02a1e0 comp rx=0 tx=0).stop 2024-09-12T03:20:56.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:56.584+0000 7f43d3b11640 1 -- 172.21.15.27:0/1880981261 shutdown_connections 2024-09-12T03:20:56.589 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:56.584+0000 7f43d3b11640 1 --2- 172.21.15.27:0/1880981261 >> v2:172.21.15.45:3300/0 conn(0x7f43cc0b4380 0x7f43cc15f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:56.589 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:56.584+0000 7f43d3b11640 1 --2- 172.21.15.27:0/1880981261 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43cc0af9d0 0x7f43cc0b3e40 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:56.589 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:56.584+0000 7f43d3b11640 1 --2- 172.21.15.27:0/1880981261 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f43cc15ec30 0x7f43cc0b3900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:56.589 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:56.584+0000 7f43d3b11640 1 -- 172.21.15.27:0/1880981261 >> 172.21.15.27:0/1880981261 conn(0x7f43cc06bb30 msgr2=0x7f43cc06bf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:56.589 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:56.584+0000 7f43d3b11640 1 -- 172.21.15.27:0/1880981261 shutdown_connections 2024-09-12T03:20:56.589 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:56.584+0000 7f43d3b11640 1 -- 172.21.15.27:0/1880981261 wait complete. 2024-09-12T03:20:56.589 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:56.585+0000 7f43d3b11640 1 Processor -- start 2024-09-12T03:20:56.589 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:56.585+0000 7f43d3b11640 1 -- start start 2024-09-12T03:20:56.589 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:56.586+0000 7f43d3b11640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f43cc0af9d0 0x7f43cc155b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:56.589 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:56.586+0000 7f43d3b11640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f43cc0b4380 0x7f43cc14f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:56.589 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:56.586+0000 7f43d3b11640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43cc15ec30 0x7f43cc159950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:56.589 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:56.586+0000 7f43d3b11640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f43cc0cfff0 con 0x7f43cc15ec30 2024-09-12T03:20:56.589 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:56.586+0000 7f43d3b11640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f43cc06c8a0 con 0x7f43cc0af9d0 2024-09-12T03:20:56.590 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:56.586+0000 7f43d3b11640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f43cc0715f0 con 0x7f43cc0b4380 2024-09-12T03:20:56.590 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:56.586+0000 7f43d20ae640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43cc15ec30 0x7f43cc159950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:56.590 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:56.586+0000 7f43d20ae640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43cc15ec30 0x7f43cc159950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55134/0 (socket says 172.21.15.27:55134) 2024-09-12T03:20:56.590 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:56.586+0000 7f43d20ae640 1 -- 172.21.15.27:0/2110328485 learned_addr learned my addr 172.21.15.27:0/2110328485 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:20:56.590 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:56.586+0000 7f43d18ad640 1 --2- 172.21.15.27:0/2110328485 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f43cc0af9d0 0x7f43cc155b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:56.590 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:56.586+0000 7f43d10ac640 1 --2- 172.21.15.27:0/2110328485 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f43cc0b4380 0x7f43cc14f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:56.590 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:56.586+0000 7f43d20ae640 1 -- 172.21.15.27:0/2110328485 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f43cc0b4380 msgr2=0x7f43cc14f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:56.590 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:56.586+0000 7f43d20ae640 1 --2- 172.21.15.27:0/2110328485 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f43cc0b4380 0x7f43cc14f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:56.590 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:56.586+0000 7f43d20ae640 1 -- 172.21.15.27:0/2110328485 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f43cc0af9d0 msgr2=0x7f43cc155b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:56.591 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:56.586+0000 7f43d20ae640 1 --2- 172.21.15.27:0/2110328485 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f43cc0af9d0 0x7f43cc155b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:56.591 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:56.586+0000 7f43d20ae640 1 -- 172.21.15.27:0/2110328485 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43cc154ab0 con 0x7f43cc15ec30 2024-09-12T03:20:56.591 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:56.586+0000 7f43d20ae640 1 --2- 172.21.15.27:0/2110328485 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43cc15ec30 0x7f43cc159950 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f43c4007e60 tx=0x7f43c400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:56.591 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:56.587+0000 7f43c27fc640 1 -- 172.21.15.27:0/2110328485 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43c4014620 con 0x7f43cc15ec30 2024-09-12T03:20:56.591 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:56.587+0000 7f43c27fc640 1 -- 172.21.15.27:0/2110328485 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43c4019070 con 0x7f43cc15ec30 2024-09-12T03:20:56.591 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:56.587+0000 7f43c27fc640 1 -- 172.21.15.27:0/2110328485 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43c4014a20 con 0x7f43cc15ec30 2024-09-12T03:20:56.591 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:56.587+0000 7f43d3b11640 1 -- 172.21.15.27:0/2110328485 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43cc154d90 con 0x7f43cc15ec30 2024-09-12T03:20:56.591 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:56.587+0000 7f43d3b11640 1 -- 172.21.15.27:0/2110328485 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43cc155350 con 0x7f43cc15ec30 2024-09-12T03:20:56.591 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:56.588+0000 7f43d3b11640 1 -- 172.21.15.27:0/2110328485 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43cc0b3900 con 0x7f43cc15ec30 2024-09-12T03:20:56.591 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:56.588+0000 7f43c27fc640 1 -- 172.21.15.27:0/2110328485 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 90600+0+0 (secure 0 0 0) 0x7f43c4014bc0 con 0x7f43cc15ec30 2024-09-12T03:20:56.592 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:56.589+0000 7f43c27fc640 1 --2- 172.21.15.27:0/2110328485 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f43b006bf80 0x7f43b006e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:56.592 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:56.589+0000 7f43c27fc640 1 -- 172.21.15.27:0/2110328485 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 4857+0+0 (secure 0 0 0) 0x7f43c408b410 con 0x7f43cc15ec30 2024-09-12T03:20:56.592 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:56.589+0000 7f43d18ad640 1 --2- 172.21.15.27:0/2110328485 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f43b006bf80 0x7f43b006e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:56.593 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:56.589+0000 7f43d18ad640 1 --2- 172.21.15.27:0/2110328485 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f43b006bf80 0x7f43b006e440 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f43cc0b55a0 tx=0x7f43c8008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:56.593 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:56.593+0000 7f43c27fc640 1 -- 172.21.15.27:0/2110328485 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f43c405a000 con 0x7f43cc15ec30 2024-09-12T03:20:56.774 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:56.774+0000 7f43d3b11640 1 -- 172.21.15.27:0/2110328485 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f43cc0af740 con 0x7f43cc15ec30 2024-09-12T03:20:57.769 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:57.769+0000 7f43c27fc640 1 -- 172.21.15.27:0/2110328485 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v97) ==== 138+0+0 (secure 0 0 0) 0x7f43c4059900 con 0x7f43cc15ec30 2024-09-12T03:20:57.769 INFO:teuthology.orchestra.run.smithi027.stderr:pool 'cephfs_metadata' created 2024-09-12T03:20:57.772 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:57.772+0000 7f43d3b11640 1 -- 172.21.15.27:0/2110328485 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f43b006bf80 msgr2=0x7f43b006e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:57.772 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:57.772+0000 7f43d3b11640 1 --2- 172.21.15.27:0/2110328485 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f43b006bf80 0x7f43b006e440 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f43cc0b55a0 tx=0x7f43c8008040 comp rx=0 tx=0).stop 2024-09-12T03:20:57.772 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:57.772+0000 7f43d3b11640 1 -- 172.21.15.27:0/2110328485 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43cc15ec30 msgr2=0x7f43cc159950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:57.773 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:57.772+0000 7f43d3b11640 1 --2- 172.21.15.27:0/2110328485 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43cc15ec30 0x7f43cc159950 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f43c4007e60 tx=0x7f43c400e4b0 comp rx=0 tx=0).stop 2024-09-12T03:20:57.773 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:57.772+0000 7f43d3b11640 1 -- 172.21.15.27:0/2110328485 shutdown_connections 2024-09-12T03:20:57.773 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:57.772+0000 7f43d3b11640 1 --2- 172.21.15.27:0/2110328485 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f43b006bf80 0x7f43b006e440 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:57.773 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:57.772+0000 7f43d3b11640 1 --2- 172.21.15.27:0/2110328485 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43cc15ec30 0x7f43cc159950 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:57.773 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:57.772+0000 7f43d3b11640 1 --2- 172.21.15.27:0/2110328485 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f43cc0b4380 0x7f43cc14f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:57.773 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:57.772+0000 7f43d3b11640 1 --2- 172.21.15.27:0/2110328485 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f43cc0af9d0 0x7f43cc155b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:57.773 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:57.772+0000 7f43d3b11640 1 -- 172.21.15.27:0/2110328485 >> 172.21.15.27:0/2110328485 conn(0x7f43cc06bb30 msgr2=0x7f43cc15da50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:57.773 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:57.772+0000 7f43d3b11640 1 -- 172.21.15.27:0/2110328485 shutdown_connections 2024-09-12T03:20:57.773 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:57.773+0000 7f43d3b11640 1 -- 172.21.15.27:0/2110328485 wait complete. 2024-09-12T03:20:57.790 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-12T03:20:57.905 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:57.905+0000 7f0da5f49640 1 Processor -- start 2024-09-12T03:20:57.906 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:57.905+0000 7f0da5f49640 1 -- start start 2024-09-12T03:20:57.909 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:57.906+0000 7f0da5f49640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f0da00b6240 0x7f0da00b6620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:57.909 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:57.906+0000 7f0da5f49640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f0da00b87d0 0x7f0da00d4040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:57.910 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:57.906+0000 7f0da5f49640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0da00b6b60 con 0x7f0da00d3c20 2024-09-12T03:20:57.910 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:57.906+0000 7f0da5f49640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f0da00d2930 con 0x7f0da00b6240 2024-09-12T03:20:57.910 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:57.906+0000 7f0da5f49640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f0da00d2c10 con 0x7f0da00b87d0 2024-09-12T03:20:57.910 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:57.906+0000 7f0d9f7fe640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f0da00b6240 0x7f0da00b6620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:57.910 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:57.906+0000 7f0d9f7fe640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f0da00b6240 0x7f0da00b6620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55136/0 (socket says 172.21.15.27:55136) 2024-09-12T03:20:57.910 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:57.906+0000 7f0d9f7fe640 1 -- 172.21.15.27:0/1724042273 learned_addr learned my addr 172.21.15.27:0/1724042273 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:20:57.910 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:57.907+0000 7f0d9ffff640 1 --2- 172.21.15.27:0/1724042273 >> v2:172.21.15.45:3300/0 conn(0x7f0da00b87d0 0x7f0da00d4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:57.910 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:57.907+0000 7f0d9f7fe640 1 -- 172.21.15.27:0/1724042273 >> v2:172.21.15.45:3300/0 conn(0x7f0da00b87d0 msgr2=0x7f0da00d4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:57.910 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:57.907+0000 7f0d9f7fe640 1 --2- 172.21.15.27:0/1724042273 >> v2:172.21.15.45:3300/0 conn(0x7f0da00b87d0 0x7f0da00d4040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:57.911 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:57.907+0000 7f0d9f7fe640 1 -- 172.21.15.27:0/1724042273 >> v1:172.21.15.45:6789/0 conn(0x7f0da00d3c20 legacy=0x7f0da006d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:57.911 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:57.907+0000 7f0d9f7fe640 1 -- 172.21.15.27:0/1724042273 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0da00d2ef0 con 0x7f0da00b6240 2024-09-12T03:20:57.911 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:57.907+0000 7f0d9f7fe640 1 --2- 172.21.15.27:0/1724042273 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0da00b6240 0x7f0da00b6620 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f0da008dae0 tx=0x7f0d9002a2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2556872a122e39bd server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:57.911 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:57.908+0000 7f0d9e7fc640 1 -- 172.21.15.27:0/1724042273 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d90037070 con 0x7f0da00b6240 2024-09-12T03:20:57.911 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:57.908+0000 7f0d9e7fc640 1 -- 172.21.15.27:0/1724042273 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d90032440 con 0x7f0da00b6240 2024-09-12T03:20:57.911 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:57.908+0000 7f0d9e7fc640 1 -- 172.21.15.27:0/1724042273 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d90032810 con 0x7f0da00b6240 2024-09-12T03:20:57.911 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:57.908+0000 7f0da5f49640 1 -- 172.21.15.27:0/1724042273 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0da00b6240 msgr2=0x7f0da00b6620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:57.911 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:57.908+0000 7f0da5f49640 1 --2- 172.21.15.27:0/1724042273 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0da00b6240 0x7f0da00b6620 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f0da008dae0 tx=0x7f0d9002a2c0 comp rx=0 tx=0).stop 2024-09-12T03:20:57.912 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:57.908+0000 7f0da5f49640 1 -- 172.21.15.27:0/1724042273 shutdown_connections 2024-09-12T03:20:57.912 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:57.908+0000 7f0da5f49640 1 --2- 172.21.15.27:0/1724042273 >> v2:172.21.15.45:3300/0 conn(0x7f0da00b87d0 0x7f0da00d4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:57.912 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:57.908+0000 7f0da5f49640 1 --2- 172.21.15.27:0/1724042273 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0da00b6240 0x7f0da00b6620 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:57.912 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:57.908+0000 7f0da5f49640 1 -- 172.21.15.27:0/1724042273 >> 172.21.15.27:0/1724042273 conn(0x7f0da00d1420 msgr2=0x7f0da00d1830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:57.912 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:57.908+0000 7f0da5f49640 1 -- 172.21.15.27:0/1724042273 shutdown_connections 2024-09-12T03:20:57.912 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:57.908+0000 7f0da5f49640 1 -- 172.21.15.27:0/1724042273 wait complete. 2024-09-12T03:20:57.912 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:57.909+0000 7f0da5f49640 1 Processor -- start 2024-09-12T03:20:57.912 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:57.910+0000 7f0da5f49640 1 -- start start 2024-09-12T03:20:57.915 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:57.910+0000 7f0da5f49640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0da00b6240 0x7f0da0155e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:57.915 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:57.910+0000 7f0da5f49640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f0da00b87d0 0x7f0da015cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:57.915 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:57.910+0000 7f0da5f49640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0da00d3c20 0x7f0da00b4120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:57.915 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:57.910+0000 7f0da5f49640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f0da015ece0 con 0x7f0da00b6240 2024-09-12T03:20:57.915 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:57.910+0000 7f0da5f49640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f0da015eb60 con 0x7f0da00d3c20 2024-09-12T03:20:57.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:57.910+0000 7f0da5f49640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f0da00cfc70 con 0x7f0da00b87d0 2024-09-12T03:20:57.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:57.910+0000 7f0d9f7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0da00b6240 0x7f0da0155e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:57.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:57.910+0000 7f0d9f7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0da00b6240 0x7f0da0155e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55140/0 (socket says 172.21.15.27:55140) 2024-09-12T03:20:57.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:57.911+0000 7f0d9f7fe640 1 -- 172.21.15.27:0/4034660456 learned_addr learned my addr 172.21.15.27:0/4034660456 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:20:57.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:57.911+0000 7f0d9effd640 1 --2- 172.21.15.27:0/4034660456 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f0da00b87d0 0x7f0da015cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:57.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:57.911+0000 7f0d9ffff640 1 --2- 172.21.15.27:0/4034660456 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0da00d3c20 0x7f0da00b4120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:57.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:57.911+0000 7f0d9f7fe640 1 -- 172.21.15.27:0/4034660456 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f0da00b87d0 msgr2=0x7f0da015cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:57.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:57.911+0000 7f0d9f7fe640 1 --2- 172.21.15.27:0/4034660456 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f0da00b87d0 0x7f0da015cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:57.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:57.911+0000 7f0d9f7fe640 1 -- 172.21.15.27:0/4034660456 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0da00d3c20 msgr2=0x7f0da00b4120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:57.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:57.911+0000 7f0d9f7fe640 1 --2- 172.21.15.27:0/4034660456 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0da00d3c20 0x7f0da00b4120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:57.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:57.911+0000 7f0d9f7fe640 1 -- 172.21.15.27:0/4034660456 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0da00cbe50 con 0x7f0da00b6240 2024-09-12T03:20:57.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:57.911+0000 7f0d9f7fe640 1 --2- 172.21.15.27:0/4034660456 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0da00b6240 0x7f0da0155e90 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f0d90000c00 tx=0x7f0d900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:57.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:57.911+0000 7f0d87fff640 1 -- 172.21.15.27:0/4034660456 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d90041070 con 0x7f0da00b6240 2024-09-12T03:20:57.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:57.911+0000 7f0d87fff640 1 -- 172.21.15.27:0/4034660456 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d9003c870 con 0x7f0da00b6240 2024-09-12T03:20:57.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:57.911+0000 7f0da5f49640 1 -- 172.21.15.27:0/4034660456 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0da00cc130 con 0x7f0da00b6240 2024-09-12T03:20:57.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:57.911+0000 7f0d87fff640 1 -- 172.21.15.27:0/4034660456 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d90037040 con 0x7f0da00b6240 2024-09-12T03:20:57.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:57.911+0000 7f0da5f49640 1 -- 172.21.15.27:0/4034660456 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0da00cc6f0 con 0x7f0da00b6240 2024-09-12T03:20:57.918 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:57.912+0000 7f0da5f49640 1 -- 172.21.15.27:0/4034660456 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0da00b6620 con 0x7f0da00b6240 2024-09-12T03:20:57.918 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:57.913+0000 7f0d87fff640 1 -- 172.21.15.27:0/4034660456 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 90600+0+0 (secure 0 0 0) 0x7f0d90044070 con 0x7f0da00b6240 2024-09-12T03:20:57.918 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:57.913+0000 7f0d87fff640 1 --2- 172.21.15.27:0/4034660456 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f0d8c06bf50 0x7f0d8c06e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:57.918 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:57.913+0000 7f0d87fff640 1 -- 172.21.15.27:0/4034660456 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 5227+0+0 (secure 0 0 0) 0x7f0d900aba70 con 0x7f0da00b6240 2024-09-12T03:20:57.918 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:57.914+0000 7f0d9effd640 1 --2- 172.21.15.27:0/4034660456 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f0d8c06bf50 0x7f0d8c06e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:57.918 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:57.914+0000 7f0d9effd640 1 --2- 172.21.15.27:0/4034660456 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f0d8c06bf50 0x7f0d8c06e410 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f0da00c8e90 tx=0x7f0d94009060 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:57.918 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:57.918+0000 7f0d87fff640 1 -- 172.21.15.27:0/4034660456 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f0d9007a050 con 0x7f0da00b6240 2024-09-12T03:20:58.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:58.097+0000 7f0da5f49640 1 -- 172.21.15.27:0/4034660456 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f0da0069f80 con 0x7f0da00b6240 2024-09-12T03:20:58.771 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:58.771+0000 7f0d87fff640 1 -- 172.21.15.27:0/4034660456 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v98) ==== 170+0+0 (secure 0 0 0) 0x7f0d9007e350 con 0x7f0da00b6240 2024-09-12T03:20:58.771 INFO:teuthology.orchestra.run.smithi027.stderr:pool 'cephfs_data' created 2024-09-12T03:20:58.774 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:58.774+0000 7f0da5f49640 1 -- 172.21.15.27:0/4034660456 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f0d8c06bf50 msgr2=0x7f0d8c06e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:58.774 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:58.774+0000 7f0da5f49640 1 --2- 172.21.15.27:0/4034660456 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f0d8c06bf50 0x7f0d8c06e410 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f0da00c8e90 tx=0x7f0d94009060 comp rx=0 tx=0).stop 2024-09-12T03:20:58.774 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:58.774+0000 7f0da5f49640 1 -- 172.21.15.27:0/4034660456 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0da00b6240 msgr2=0x7f0da0155e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:58.774 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:58.774+0000 7f0da5f49640 1 --2- 172.21.15.27:0/4034660456 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0da00b6240 0x7f0da0155e90 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f0d90000c00 tx=0x7f0d900046c0 comp rx=0 tx=0).stop 2024-09-12T03:20:58.774 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:58.774+0000 7f0da5f49640 1 -- 172.21.15.27:0/4034660456 shutdown_connections 2024-09-12T03:20:58.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:58.774+0000 7f0da5f49640 1 --2- 172.21.15.27:0/4034660456 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0da00d3c20 0x7f0da00b4120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:58.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:58.774+0000 7f0da5f49640 1 --2- 172.21.15.27:0/4034660456 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f0da00b87d0 0x7f0da015cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:58.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:58.774+0000 7f0da5f49640 1 --2- 172.21.15.27:0/4034660456 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f0d8c06bf50 0x7f0d8c06e410 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:58.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:58.774+0000 7f0da5f49640 1 --2- 172.21.15.27:0/4034660456 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0da00b6240 0x7f0da0155e90 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:58.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:58.774+0000 7f0da5f49640 1 -- 172.21.15.27:0/4034660456 >> 172.21.15.27:0/4034660456 conn(0x7f0da00d1420 msgr2=0x7f0da015bcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:58.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:58.775+0000 7f0da5f49640 1 -- 172.21.15.27:0/4034660456 shutdown_connections 2024-09-12T03:20:58.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:58.775+0000 7f0da5f49640 1 -- 172.21.15.27:0/4034660456 wait complete. 2024-09-12T03:20:58.801 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-12T03:20:58.909 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:58.909+0000 7f1fae21e640 1 Processor -- start 2024-09-12T03:20:58.909 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:58.909+0000 7f1fae21e640 1 -- start start 2024-09-12T03:20:58.911 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:58.910+0000 7f1fae21e640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1fa806c650 0x7f1fa80cf410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:58.911 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:58.910+0000 7f1fae21e640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1fa80ce650 con 0x7f1fa815d560 2024-09-12T03:20:58.911 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:58.910+0000 7f1fae21e640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1fa80ce930 con 0x7f1fa815a270 2024-09-12T03:20:58.912 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:58.910+0000 7f1fae21e640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f1fa80cef80 con 0x7f1fa806c650 2024-09-12T03:20:58.912 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:58.910+0000 7f1fa7fff640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f1fa815d560 0x7f1fa80726a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:58184/0 (socket says 172.21.15.27:58184) 2024-09-12T03:20:58.912 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:58.910+0000 7f1fa7fff640 1 -- 172.21.15.27:0/3141109682 learned_addr learned my addr 172.21.15.27:0/3141109682 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:20:58.912 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:58.910+0000 7f1fa6ffd640 1 --2- 172.21.15.27:0/3141109682 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1fa806c650 0x7f1fa80cf410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:58.912 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:58.910+0000 7f1fa67fc640 1 -- 172.21.15.27:0/3141109682 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1577074118 0 0) 0x7f1fa80ce650 con 0x7f1fa815d560 2024-09-12T03:20:58.912 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:58.910+0000 7f1fa67fc640 1 -- 172.21.15.27:0/3141109682 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1f90003520 con 0x7f1fa815d560 2024-09-12T03:20:58.912 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:58.911+0000 7f1fa67fc640 1 -- 172.21.15.27:0/3141109682 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 299969835 0 0) 0x7f1f90003520 con 0x7f1fa815d560 2024-09-12T03:20:58.912 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:58.911+0000 7f1fa67fc640 1 -- 172.21.15.27:0/3141109682 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1fa806c650 msgr2=0x7f1fa80cf410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:58.912 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:58.911+0000 7f1fa67fc640 1 --2- 172.21.15.27:0/3141109682 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1fa806c650 0x7f1fa80cf410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:58.912 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:58.911+0000 7f1fa67fc640 1 -- 172.21.15.27:0/3141109682 >> v1:172.21.15.45:6789/0 conn(0x7f1fa815a270 legacy=0x7f1fa806cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:58.912 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:58.911+0000 7f1fa67fc640 1 -- 172.21.15.27:0/3141109682 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fa80cec10 con 0x7f1fa815d560 2024-09-12T03:20:58.912 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:58.911+0000 7f1fa67fc640 1 -- 172.21.15.27:0/3141109682 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 847755213 0 0) 0x7f1fa00019f0 con 0x7f1fa815d560 2024-09-12T03:20:58.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:58.911+0000 7f1fa67fc640 1 -- 172.21.15.27:0/3141109682 >> v1:172.21.15.27:6789/0 conn(0x7f1fa815d560 legacy=0x7f1fa80726a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:58.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:58.911+0000 7f1fa67fc640 1 --2- 172.21.15.27:0/3141109682 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1fa806c650 0x7f1f900039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:58.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:58.911+0000 7f1fa67fc640 1 --2- 172.21.15.27:0/3141109682 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1f90003f60 0x7f1f90004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:58.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:58.911+0000 7f1fa67fc640 1 --2- 172.21.15.27:0/3141109682 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1f90004d40 0x7f1f900051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:58.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:58.911+0000 7f1fa67fc640 1 -- 172.21.15.27:0/3141109682 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f1fa80cef80 con 0x7f1f90004d40 2024-09-12T03:20:58.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:58.911+0000 7f1fa67fc640 1 -- 172.21.15.27:0/3141109682 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f1f9000baa0 con 0x7f1f90003f60 2024-09-12T03:20:58.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:58.911+0000 7f1fa67fc640 1 -- 172.21.15.27:0/3141109682 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f1f9000bca0 con 0x7f1fa806c650 2024-09-12T03:20:58.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:58.911+0000 7f1fa77fe640 1 --2- 172.21.15.27:0/3141109682 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1fa806c650 0x7f1f900039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:58.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:58.911+0000 7f1fa7fff640 1 --2- 172.21.15.27:0/3141109682 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1f90004d40 0x7f1f900051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:58.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:58.911+0000 7f1fa6ffd640 1 --2- 172.21.15.27:0/3141109682 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1f90003f60 0x7f1f90004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:58.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:58.911+0000 7f1fa77fe640 1 -- 172.21.15.27:0/3141109682 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1f90003f60 msgr2=0x7f1f90004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:58.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:58.911+0000 7f1fa77fe640 1 --2- 172.21.15.27:0/3141109682 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1f90003f60 0x7f1f90004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:58.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:58.911+0000 7f1fa77fe640 1 -- 172.21.15.27:0/3141109682 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1f90004d40 msgr2=0x7f1f900051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:58.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:58.911+0000 7f1fa77fe640 1 --2- 172.21.15.27:0/3141109682 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1f90004d40 0x7f1f900051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:58.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:58.911+0000 7f1fa77fe640 1 -- 172.21.15.27:0/3141109682 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f90003520 con 0x7f1fa806c650 2024-09-12T03:20:58.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:58.911+0000 7f1fa7fff640 1 --2- 172.21.15.27:0/3141109682 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1f90004d40 0x7f1f900051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:20:58.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:58.912+0000 7f1fa77fe640 1 --2- 172.21.15.27:0/3141109682 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1fa806c650 0x7f1f900039e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f1f98001750 tx=0x7f1f90005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=56781d2175a1ef7a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:58.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:58.912+0000 7f1fa67fc640 1 -- 172.21.15.27:0/3141109682 <== mon.2 v2:172.21.15.45:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f98001370 con 0x7f1fa806c650 2024-09-12T03:20:58.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:58.912+0000 7f1fa67fc640 1 -- 172.21.15.27:0/3141109682 <== mon.2 v2:172.21.15.45:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f980052d0 con 0x7f1fa806c650 2024-09-12T03:20:58.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:58.912+0000 7f1fa67fc640 1 -- 172.21.15.27:0/3141109682 <== mon.2 v2:172.21.15.45:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f9802abe0 con 0x7f1fa806c650 2024-09-12T03:20:58.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:58.913+0000 7f1fae21e640 1 -- 172.21.15.27:0/3141109682 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1fa806c650 msgr2=0x7f1f900039e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:58.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:58.913+0000 7f1fae21e640 1 --2- 172.21.15.27:0/3141109682 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1fa806c650 0x7f1f900039e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f1f98001750 tx=0x7f1f90005830 comp rx=0 tx=0).stop 2024-09-12T03:20:58.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:58.913+0000 7f1fa7fff640 1 -- 172.21.15.27:0/3141109682 reap_dead start 2024-09-12T03:20:58.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:58.913+0000 7f1fae21e640 1 -- 172.21.15.27:0/3141109682 shutdown_connections 2024-09-12T03:20:58.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:58.913+0000 7f1fae21e640 1 -- 172.21.15.27:0/3141109682 >> 172.21.15.27:0/3141109682 conn(0x7f1fa806bb30 msgr2=0x7f1fa806bf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:58.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:58.913+0000 7f1fae21e640 1 -- 172.21.15.27:0/3141109682 shutdown_connections 2024-09-12T03:20:58.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:58.913+0000 7f1fae21e640 1 -- 172.21.15.27:0/3141109682 wait complete. 2024-09-12T03:20:58.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:58.914+0000 7f1fae21e640 1 Processor -- start 2024-09-12T03:20:58.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:58.914+0000 7f1fae21e640 1 -- start start 2024-09-12T03:20:58.918 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:58.915+0000 7f1fae21e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1fa806c650 0x7f1fa8154cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:58.918 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:58.915+0000 7f1fae21e640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1fa81e6c50 0x7f1fa81592c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:58.918 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:58.915+0000 7f1fae21e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1fa815bc60 0x7f1fa815c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:58.918 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:58.915+0000 7f1fae21e640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f1fa8072db0 con 0x7f1fa806c650 2024-09-12T03:20:58.918 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:58.915+0000 7f1fae21e640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f1fa8073320 con 0x7f1fa815bc60 2024-09-12T03:20:58.918 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:58.915+0000 7f1fae21e640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f1fa8072f30 con 0x7f1fa81e6c50 2024-09-12T03:20:58.918 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:58.915+0000 7f1fa77fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1fa806c650 0x7f1fa8154cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:58.918 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:58.915+0000 7f1fa77fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1fa806c650 0x7f1fa8154cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55162/0 (socket says 172.21.15.27:55162) 2024-09-12T03:20:58.918 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:58.915+0000 7f1fa77fe640 1 -- 172.21.15.27:0/2160131941 learned_addr learned my addr 172.21.15.27:0/2160131941 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:20:58.918 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:58.915+0000 7f1fa6ffd640 1 --2- 172.21.15.27:0/2160131941 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1fa81e6c50 0x7f1fa81592c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:58.918 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:58.915+0000 7f1fa77fe640 1 -- 172.21.15.27:0/2160131941 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1fa81e6c50 msgr2=0x7f1fa81592c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:58.918 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:58.915+0000 7f1fa77fe640 1 --2- 172.21.15.27:0/2160131941 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1fa81e6c50 0x7f1fa81592c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:58.918 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:58.915+0000 7f1fa7fff640 1 --2- 172.21.15.27:0/2160131941 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1fa815bc60 0x7f1fa815c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:58.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:58.915+0000 7f1fa77fe640 1 -- 172.21.15.27:0/2160131941 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1fa815bc60 msgr2=0x7f1fa815c0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:58.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:58.915+0000 7f1fa77fe640 1 --2- 172.21.15.27:0/2160131941 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1fa815bc60 0x7f1fa815c0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:58.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:58.915+0000 7f1fa77fe640 1 -- 172.21.15.27:0/2160131941 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fa81f8250 con 0x7f1fa806c650 2024-09-12T03:20:58.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:58.915+0000 7f1fa77fe640 1 --2- 172.21.15.27:0/2160131941 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1fa806c650 0x7f1fa8154cd0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f1f98008fd0 tx=0x7f1f98001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:58.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:58.916+0000 7f1f8ffff640 1 -- 172.21.15.27:0/2160131941 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f98042070 con 0x7f1fa806c650 2024-09-12T03:20:58.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:58.916+0000 7f1f8ffff640 1 -- 172.21.15.27:0/2160131941 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f9802be50 con 0x7f1fa806c650 2024-09-12T03:20:58.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:58.916+0000 7f1fae21e640 1 -- 172.21.15.27:0/2160131941 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fa81f8530 con 0x7f1fa806c650 2024-09-12T03:20:58.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:58.916+0000 7f1f8ffff640 1 -- 172.21.15.27:0/2160131941 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f98037040 con 0x7f1fa806c650 2024-09-12T03:20:58.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:58.916+0000 7f1fae21e640 1 -- 172.21.15.27:0/2160131941 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fa81f8af0 con 0x7f1fa806c650 2024-09-12T03:20:58.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:58.917+0000 7f1fae21e640 1 -- 172.21.15.27:0/2160131941 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1fa80ce650 con 0x7f1fa806c650 2024-09-12T03:20:58.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:58.917+0000 7f1f8ffff640 1 -- 172.21.15.27:0/2160131941 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 90600+0+0 (secure 0 0 0) 0x7f1f98045070 con 0x7f1fa806c650 2024-09-12T03:20:58.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:58.918+0000 7f1f8ffff640 1 --2- 172.21.15.27:0/2160131941 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f1f9406bf50 0x7f1f9406e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:58.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:58.918+0000 7f1f8ffff640 1 -- 172.21.15.27:0/2160131941 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 5609+0+0 (secure 0 0 0) 0x7f1f980aaa70 con 0x7f1fa806c650 2024-09-12T03:20:58.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:58.918+0000 7f1fa6ffd640 1 --2- 172.21.15.27:0/2160131941 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f1f9406bf50 0x7f1f9406e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:58.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:58.918+0000 7f1fa6ffd640 1 --2- 172.21.15.27:0/2160131941 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f1f9406bf50 0x7f1f9406e410 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f1fa8158ca0 tx=0x7f1f9c008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:58.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:58.922+0000 7f1f8ffff640 1 -- 172.21.15.27:0/2160131941 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f1f98079450 con 0x7f1fa806c650 2024-09-12T03:20:59.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:59.132+0000 7f1fae21e640 1 -- 172.21.15.27:0/2160131941 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f1fa80ccdb0 con 0x7f1fa806c650 2024-09-12T03:20:59.801 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:59.800+0000 7f1f8ffff640 1 -- 172.21.15.27:0/2160131941 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 16 and data pool 17 v56) ==== 174+0+0 (secure 0 0 0) 0x7f1f98078d50 con 0x7f1fa806c650 2024-09-12T03:20:59.801 INFO:teuthology.orchestra.run.smithi027.stderr:new fs with metadata pool 16 and data pool 17 2024-09-12T03:20:59.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:59.812+0000 7f1fae21e640 1 -- 172.21.15.27:0/2160131941 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f1f9406bf50 msgr2=0x7f1f9406e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:59.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:59.812+0000 7f1fae21e640 1 --2- 172.21.15.27:0/2160131941 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f1f9406bf50 0x7f1f9406e410 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f1fa8158ca0 tx=0x7f1f9c008040 comp rx=0 tx=0).stop 2024-09-12T03:20:59.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:59.812+0000 7f1fae21e640 1 -- 172.21.15.27:0/2160131941 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1fa806c650 msgr2=0x7f1fa8154cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:59.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:59.812+0000 7f1fae21e640 1 --2- 172.21.15.27:0/2160131941 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1fa806c650 0x7f1fa8154cd0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f1f98008fd0 tx=0x7f1f98001120 comp rx=0 tx=0).stop 2024-09-12T03:20:59.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:59.814+0000 7f1fae21e640 1 -- 172.21.15.27:0/2160131941 shutdown_connections 2024-09-12T03:20:59.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:59.814+0000 7f1fae21e640 1 --2- 172.21.15.27:0/2160131941 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1fa815bc60 0x7f1fa815c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:59.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:59.814+0000 7f1fae21e640 1 --2- 172.21.15.27:0/2160131941 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1fa81e6c50 0x7f1fa81592c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:59.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:59.814+0000 7f1fae21e640 1 --2- 172.21.15.27:0/2160131941 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f1f9406bf50 0x7f1f9406e410 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:59.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:59.814+0000 7f1fae21e640 1 --2- 172.21.15.27:0/2160131941 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1fa806c650 0x7f1fa8154cd0 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:59.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:59.814+0000 7f1fae21e640 1 -- 172.21.15.27:0/2160131941 >> 172.21.15.27:0/2160131941 conn(0x7f1fa806bb30 msgr2=0x7f1fa8072660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:59.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:59.817+0000 7f1fae21e640 1 -- 172.21.15.27:0/2160131941 shutdown_connections 2024-09-12T03:20:59.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:59.818+0000 7f1fae21e640 1 -- 172.21.15.27:0/2160131941 wait complete. 2024-09-12T03:20:59.837 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-12T03:20:59.944 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:59.944+0000 7f6ccb28d640 1 Processor -- start 2024-09-12T03:20:59.944 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:59.944+0000 7f6ccb28d640 1 -- start start 2024-09-12T03:20:59.945 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:59.945+0000 7f6ccb28d640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f6cc4145c30 0x7f6cc415ca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:59.946 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:59.945+0000 7f6ccb28d640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f6cc40cda40 0x7f6cc40b4040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:59.946 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:59.945+0000 7f6ccb28d640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6cc40b4d70 con 0x7f6cc4146010 2024-09-12T03:20:59.946 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:59.945+0000 7f6ccb28d640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f6cc40b5050 con 0x7f6cc40cda40 2024-09-12T03:20:59.946 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:59.945+0000 7f6ccb28d640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f6cc406d140 con 0x7f6cc4145c30 2024-09-12T03:20:59.946 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:59.945+0000 7f6cc982a640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f6cc40cda40 0x7f6cc40b4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:59.946 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:59.945+0000 7f6cc8828640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f6cc4146010 0x7f6cc40cd330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:47782/0 (socket says 172.21.15.27:47782) 2024-09-12T03:20:59.946 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:59.945+0000 7f6cc8828640 1 -- 172.21.15.27:0/3805035521 learned_addr learned my addr 172.21.15.27:0/3805035521 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:20:59.946 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:59.945+0000 7f6cc982a640 1 -- 172.21.15.27:0/3805035521 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f6cc4145c30 msgr2=0x7f6cc415ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:59.946 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:59.945+0000 7f6cc982a640 1 --2- 172.21.15.27:0/3805035521 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f6cc4145c30 0x7f6cc415ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:59.946 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:59.945+0000 7f6cc982a640 1 -- 172.21.15.27:0/3805035521 >> v1:172.21.15.27:6789/0 conn(0x7f6cc4146010 legacy=0x7f6cc40cd330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:59.946 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:59.946+0000 7f6cc982a640 1 -- 172.21.15.27:0/3805035521 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cc406d420 con 0x7f6cc40cda40 2024-09-12T03:20:59.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:59.946+0000 7f6cc982a640 1 --2- 172.21.15.27:0/3805035521 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6cc40cda40 0x7f6cc40b4040 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f6cb4009d90 tx=0x7f6cb402a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f23b471eda8b3691 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:59.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:59.946+0000 7f6cc3fff640 1 -- 172.21.15.27:0/3805035521 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cb4031600 con 0x7f6cc40cda40 2024-09-12T03:20:59.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:59.946+0000 7f6cc3fff640 1 -- 172.21.15.27:0/3805035521 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6cb40317a0 con 0x7f6cc40cda40 2024-09-12T03:20:59.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:59.946+0000 7f6cc3fff640 1 -- 172.21.15.27:0/3805035521 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cb4031b70 con 0x7f6cc40cda40 2024-09-12T03:20:59.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:59.947+0000 7f6ccb28d640 1 -- 172.21.15.27:0/3805035521 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6cc40cda40 msgr2=0x7f6cc40b4040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:59.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:59.947+0000 7f6ccb28d640 1 --2- 172.21.15.27:0/3805035521 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6cc40cda40 0x7f6cc40b4040 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f6cb4009d90 tx=0x7f6cb402a5d0 comp rx=0 tx=0).stop 2024-09-12T03:20:59.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:59.947+0000 7f6ccb28d640 1 -- 172.21.15.27:0/3805035521 shutdown_connections 2024-09-12T03:20:59.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:59.947+0000 7f6ccb28d640 1 --2- 172.21.15.27:0/3805035521 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6cc40cda40 0x7f6cc40b4040 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:59.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:59.947+0000 7f6ccb28d640 1 --2- 172.21.15.27:0/3805035521 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f6cc4145c30 0x7f6cc415ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:59.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:59.947+0000 7f6ccb28d640 1 -- 172.21.15.27:0/3805035521 >> 172.21.15.27:0/3805035521 conn(0x7f6cc4062c60 msgr2=0x7f6cc406bb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:59.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:59.947+0000 7f6ccb28d640 1 -- 172.21.15.27:0/3805035521 shutdown_connections 2024-09-12T03:20:59.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:59.947+0000 7f6ccb28d640 1 -- 172.21.15.27:0/3805035521 wait complete. 2024-09-12T03:20:59.950 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:59.948+0000 7f6ccb28d640 1 Processor -- start 2024-09-12T03:20:59.950 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:59.948+0000 7f6ccb28d640 1 -- start start 2024-09-12T03:20:59.952 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:59.949+0000 7f6ccb28d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6cc40cda40 0x7f6cc4072b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:59.952 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:59.949+0000 7f6ccb28d640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f6cc4145c30 0x7f6cc41f4920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:59.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:59.949+0000 7f6ccb28d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6cc4146010 0x7f6cc41f4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:59.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:59.949+0000 7f6ccb28d640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f6cc40b4700 con 0x7f6cc40cda40 2024-09-12T03:20:59.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:59.949+0000 7f6ccb28d640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f6cc40b4580 con 0x7f6cc4146010 2024-09-12T03:20:59.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:59.949+0000 7f6ccb28d640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f6cc40b4880 con 0x7f6cc4145c30 2024-09-12T03:20:59.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:59.949+0000 7f6cc9029640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6cc40cda40 0x7f6cc4072b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:59.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:59.949+0000 7f6cc9029640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6cc40cda40 0x7f6cc4072b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38432/0 (socket says 172.21.15.27:38432) 2024-09-12T03:20:59.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:59.949+0000 7f6cc9029640 1 -- 172.21.15.27:0/3214981830 learned_addr learned my addr 172.21.15.27:0/3214981830 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:20:59.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:59.949+0000 7f6cc9029640 1 -- 172.21.15.27:0/3214981830 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f6cc4145c30 msgr2=0x7f6cc41f4920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:59.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:59.949+0000 7f6cc9029640 1 --2- 172.21.15.27:0/3214981830 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f6cc4145c30 0x7f6cc41f4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:59.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:59.949+0000 7f6cc9029640 1 -- 172.21.15.27:0/3214981830 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6cc4146010 msgr2=0x7f6cc41f4e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:59.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:59.949+0000 7f6cc9029640 1 --2- 172.21.15.27:0/3214981830 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6cc4146010 0x7f6cc41f4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:59.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:59.949+0000 7f6cc9029640 1 -- 172.21.15.27:0/3214981830 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cc41f5490 con 0x7f6cc40cda40 2024-09-12T03:20:59.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:59.950+0000 7f6cc9029640 1 --2- 172.21.15.27:0/3214981830 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6cc40cda40 0x7f6cc4072b40 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f6cbc00b4a0 tx=0x7f6cbc00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:59.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:59.950+0000 7f6cc1ffb640 1 -- 172.21.15.27:0/3214981830 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cbc014070 con 0x7f6cc40cda40 2024-09-12T03:20:59.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:59.950+0000 7f6cc1ffb640 1 -- 172.21.15.27:0/3214981830 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6cbc010610 con 0x7f6cc40cda40 2024-09-12T03:20:59.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:59.950+0000 7f6ccb28d640 1 -- 172.21.15.27:0/3214981830 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6cc41f5770 con 0x7f6cc40cda40 2024-09-12T03:20:59.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:59.950+0000 7f6cc1ffb640 1 -- 172.21.15.27:0/3214981830 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cbc010a00 con 0x7f6cc40cda40 2024-09-12T03:20:59.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:59.950+0000 7f6ccb28d640 1 -- 172.21.15.27:0/3214981830 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6cc40ae0c0 con 0x7f6cc40cda40 2024-09-12T03:20:59.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:59.951+0000 7f6ccb28d640 1 -- 172.21.15.27:0/3214981830 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6cc40b4040 con 0x7f6cc40cda40 2024-09-12T03:20:59.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:59.952+0000 7f6cc1ffb640 1 -- 172.21.15.27:0/3214981830 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 90600+0+0 (secure 0 0 0) 0x7f6cbc017070 con 0x7f6cc40cda40 2024-09-12T03:20:59.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:59.952+0000 7f6cc1ffb640 1 --2- 172.21.15.27:0/3214981830 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f6cb006bf50 0x7f6cb006e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:59.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:59.952+0000 7f6cc1ffb640 1 -- 172.21.15.27:0/3214981830 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 5709+0+0 (secure 0 0 0) 0x7f6cbc08aa80 con 0x7f6cc40cda40 2024-09-12T03:20:59.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:59.952+0000 7f6cc8828640 1 --2- 172.21.15.27:0/3214981830 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f6cb006bf50 0x7f6cb006e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:59.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:59.953+0000 7f6cc8828640 1 --2- 172.21.15.27:0/3214981830 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f6cb006bf50 0x7f6cb006e410 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f6cc40afc20 tx=0x7f6cb8007a40 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:59.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:20:59.956+0000 7f6cc1ffb640 1 -- 172.21.15.27:0/3214981830 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f6cbc059320 con 0x7f6cc40cda40 2024-09-12T03:21:00.130 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.130+0000 7f6ccb28d640 1 -- 172.21.15.27:0/3214981830 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6cc40abaf0 con 0x7f6cc40cda40 2024-09-12T03:21:00.131 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.131+0000 7f6cc1ffb640 1 -- 172.21.15.27:0/3214981830 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v99) ==== 74+0+16641 (secure 0 0 0) 0x7f6cbc058c20 con 0x7f6cc40cda40 2024-09-12T03:21:00.132 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:21:00.132 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":99,"fsid":"d2df504e-2647-4385-a5db-9fdf69f487a0","created":"2024-09-12T03:16:07.778937+0000","modified":"2024-09-12T03:20:59.133496+0000","last_up_change":"2024-09-12T03:16:19.314778+0000","last_in_change":"2024-09-12T03:16:09.893086+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T03:16:20.097646+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-09-12T03:20:56.805568+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-09-12T03:20:58.128117+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1851116c-1fb3-4dc9-9772-f90e2ef4c355","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6817","nonce":3002321488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6819","nonce":3002321488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6823","nonce":3002321488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6821","nonce":3002321488}]},"public_addr":"172.21.15.27:6817/3002321488","cluster_addr":"172.21.15.27:6819/3002321488","heartbeat_back_addr":"172.21.15.27:6823/3002321488","heartbeat_front_addr":"172.21.15.27:6821/3002321488","state":["exists","up"]},{"osd":1,"uuid":"f38ae17d-73eb-4204-b99d-975fa806fb87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6809","nonce":1613102886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6811","nonce":1613102886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6815","nonce":1613102886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6813","nonce":1613102886}]},"public_addr":"172.21.15.27:6809/1613102886","cluster_addr":"172.21.15.27:6811/1613102886","heartbeat_back_addr":"172.21.15.27:6815/1613102886","heartbeat_front_addr":"172.21.15.27:6813/1613102886","state":["exists","up"]},{"osd":2,"uuid":"fb12dcc8-bdd0-42b8-b46d-8b10106224bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6825","nonce":3534723355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6827","nonce":3534723355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6831","nonce":3534723355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6829","nonce":3534723355}]},"public_addr":"172.21.15.27:6825/3534723355","cluster_addr":"172.21.15.27:6827/3534723355","heartbeat_back_addr":"172.21.15.27:6831/3534723355","heartbeat_front_addr":"172.21.15.27:6829/3534723355","state":["exists","up"]},{"osd":3,"uuid":"d74d3f49-9c24-4b9f-98e2-3ecd7b5ef3b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6801","nonce":80661598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6803","nonce":80661598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6807","nonce":80661598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6805","nonce":80661598}]},"public_addr":"172.21.15.27:6801/80661598","cluster_addr":"172.21.15.27:6803/80661598","heartbeat_back_addr":"172.21.15.27:6807/80661598","heartbeat_front_addr":"172.21.15.27:6805/80661598","state":["exists","up"]},{"osd":4,"uuid":"2bf123f2-1b0d-41c4-aa3b-805bab267e59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6817","nonce":3943189827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6819","nonce":3943189827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6823","nonce":3943189827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6821","nonce":3943189827}]},"public_addr":"172.21.15.45:6817/3943189827","cluster_addr":"172.21.15.45:6819/3943189827","heartbeat_back_addr":"172.21.15.45:6823/3943189827","heartbeat_front_addr":"172.21.15.45:6821/3943189827","state":["exists","up"]},{"osd":5,"uuid":"c49ba5da-91c6-4a86-a97b-4aba89c253ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6809","nonce":3620760330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6811","nonce":3620760330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6815","nonce":3620760330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6813","nonce":3620760330}]},"public_addr":"172.21.15.45:6809/3620760330","cluster_addr":"172.21.15.45:6811/3620760330","heartbeat_back_addr":"172.21.15.45:6815/3620760330","heartbeat_front_addr":"172.21.15.45:6813/3620760330","state":["exists","up"]},{"osd":6,"uuid":"b0123475-89f7-4a34-9e33-d797b99dd9dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6801","nonce":1989083510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6803","nonce":1989083510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6807","nonce":1989083510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6805","nonce":1989083510}]},"public_addr":"172.21.15.45:6801/1989083510","cluster_addr":"172.21.15.45:6803/1989083510","heartbeat_back_addr":"172.21.15.45:6807/1989083510","heartbeat_front_addr":"172.21.15.45:6805/1989083510","state":["exists","up"]},{"osd":7,"uuid":"4dad773d-b7d8-46fc-8a10-a972aad27b29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6825","nonce":1513300294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6827","nonce":1513300294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6831","nonce":1513300294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6829","nonce":1513300294}]},"public_addr":"172.21.15.45:6825/1513300294","cluster_addr":"172.21.15.45:6827/1513300294","heartbeat_back_addr":"172.21.15.45:6831/1513300294","heartbeat_front_addr":"172.21.15.45:6829/1513300294","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.881855+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:17.133252+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.735339+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.500742+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.477571+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:17.167667+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.470291+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.873743+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T03:21:00.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.133+0000 7f6ccb28d640 1 -- 172.21.15.27:0/3214981830 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f6cb006bf50 msgr2=0x7f6cb006e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:00.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.133+0000 7f6ccb28d640 1 --2- 172.21.15.27:0/3214981830 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f6cb006bf50 0x7f6cb006e410 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f6cc40afc20 tx=0x7f6cb8007a40 comp rx=0 tx=0).stop 2024-09-12T03:21:00.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.133+0000 7f6ccb28d640 1 -- 172.21.15.27:0/3214981830 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6cc40cda40 msgr2=0x7f6cc4072b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:00.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.133+0000 7f6ccb28d640 1 --2- 172.21.15.27:0/3214981830 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6cc40cda40 0x7f6cc4072b40 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f6cbc00b4a0 tx=0x7f6cbc00b960 comp rx=0 tx=0).stop 2024-09-12T03:21:00.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.133+0000 7f6ccb28d640 1 -- 172.21.15.27:0/3214981830 shutdown_connections 2024-09-12T03:21:00.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.133+0000 7f6ccb28d640 1 --2- 172.21.15.27:0/3214981830 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6cc4146010 0x7f6cc41f4e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:00.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.133+0000 7f6ccb28d640 1 --2- 172.21.15.27:0/3214981830 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f6cc4145c30 0x7f6cc41f4920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:00.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.133+0000 7f6ccb28d640 1 --2- 172.21.15.27:0/3214981830 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f6cb006bf50 0x7f6cb006e410 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:00.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.133+0000 7f6ccb28d640 1 --2- 172.21.15.27:0/3214981830 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6cc40cda40 0x7f6cc4072b40 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:00.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.134+0000 7f6ccb28d640 1 -- 172.21.15.27:0/3214981830 >> 172.21.15.27:0/3214981830 conn(0x7f6cc4062c60 msgr2=0x7f6cc40b2b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:00.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.134+0000 7f6ccb28d640 1 -- 172.21.15.27:0/3214981830 shutdown_connections 2024-09-12T03:21:00.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.134+0000 7f6ccb28d640 1 -- 172.21.15.27:0/3214981830 wait complete. 2024-09-12T03:21:00.146 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-12T03:21:00.248 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.248+0000 7f753c074640 1 Processor -- start 2024-09-12T03:21:00.248 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.248+0000 7f753c074640 1 -- start start 2024-09-12T03:21:00.249 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.249+0000 7f753c074640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f7534158cf0 0x7f75340b55d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:00.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.249+0000 7f753c074640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f753415f600 0x7f753406bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:00.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.249+0000 7f753c074640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7534162c10 con 0x7f75341590d0 2024-09-12T03:21:00.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.249+0000 7f753c074640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f7534162ef0 con 0x7f753415f600 2024-09-12T03:21:00.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.249+0000 7f753c074640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f75341631d0 con 0x7f7534158cf0 2024-09-12T03:21:00.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.249+0000 7f7539e10640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f7534158cf0 0x7f75340b55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:00.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.249+0000 7f753a611640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f753415f600 0x7f753406bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:00.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.249+0000 7f7539e10640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f7534158cf0 0x7f75340b55d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3301/0 says I am v2:172.21.15.27:60606/0 (socket says 172.21.15.27:60606) 2024-09-12T03:21:00.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.249+0000 7f7539e10640 1 -- 172.21.15.27:0/2680942672 learned_addr learned my addr 172.21.15.27:0/2680942672 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:21:00.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.249+0000 7f753a611640 1 --2- 172.21.15.27:0/2680942672 >> v2:172.21.15.45:3300/0 conn(0x7f753415f600 0x7f753406bb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.27:42068/0 (socket says 172.21.15.27:42068) 2024-09-12T03:21:00.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.250+0000 7f7539e10640 1 -- 172.21.15.27:0/2680942672 >> v2:172.21.15.45:3300/0 conn(0x7f753415f600 msgr2=0x7f753406bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:00.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.250+0000 7f7539e10640 1 --2- 172.21.15.27:0/2680942672 >> v2:172.21.15.45:3300/0 conn(0x7f753415f600 0x7f753406bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:00.251 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.250+0000 7f7539e10640 1 -- 172.21.15.27:0/2680942672 >> v1:172.21.15.45:6789/0 conn(0x7f75341590d0 legacy=0x7f7534162430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:00.251 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.250+0000 7f7539e10640 1 -- 172.21.15.27:0/2680942672 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f753415edc0 con 0x7f7534158cf0 2024-09-12T03:21:00.251 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.250+0000 7f753a611640 1 --2- 172.21.15.27:0/2680942672 >> v2:172.21.15.45:3300/0 conn(0x7f753415f600 0x7f753406bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:21:00.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.250+0000 7f7539e10640 1 --2- 172.21.15.27:0/2680942672 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f7534158cf0 0x7f75340b55d0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f7530009870 tx=0x7f753002a130 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bace41228af64847 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:00.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.251+0000 7f7538e0e640 1 -- 172.21.15.27:0/2680942672 <== mon.2 v2:172.21.15.45:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7530037070 con 0x7f7534158cf0 2024-09-12T03:21:00.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.251+0000 7f7538e0e640 1 -- 172.21.15.27:0/2680942672 <== mon.2 v2:172.21.15.45:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7530032440 con 0x7f7534158cf0 2024-09-12T03:21:00.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.251+0000 7f7538e0e640 1 -- 172.21.15.27:0/2680942672 <== mon.2 v2:172.21.15.45:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7530032810 con 0x7f7534158cf0 2024-09-12T03:21:00.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.251+0000 7f753c074640 1 -- 172.21.15.27:0/2680942672 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f7534158cf0 msgr2=0x7f75340b55d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:00.253 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.251+0000 7f753c074640 1 --2- 172.21.15.27:0/2680942672 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f7534158cf0 0x7f75340b55d0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f7530009870 tx=0x7f753002a130 comp rx=0 tx=0).stop 2024-09-12T03:21:00.253 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.251+0000 7f753c074640 1 -- 172.21.15.27:0/2680942672 shutdown_connections 2024-09-12T03:21:00.253 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.251+0000 7f753c074640 1 --2- 172.21.15.27:0/2680942672 >> v2:172.21.15.45:3300/0 conn(0x7f753415f600 0x7f753406bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:00.253 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.251+0000 7f753c074640 1 --2- 172.21.15.27:0/2680942672 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f7534158cf0 0x7f75340b55d0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:00.253 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.251+0000 7f753c074640 1 -- 172.21.15.27:0/2680942672 >> 172.21.15.27:0/2680942672 conn(0x7f753406cc40 msgr2=0x7f753406d050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:00.253 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.251+0000 7f753c074640 1 -- 172.21.15.27:0/2680942672 shutdown_connections 2024-09-12T03:21:00.253 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.251+0000 7f753c074640 1 -- 172.21.15.27:0/2680942672 wait complete. 2024-09-12T03:21:00.253 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.252+0000 7f753c074640 1 Processor -- start 2024-09-12T03:21:00.253 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.252+0000 7f753c074640 1 -- start start 2024-09-12T03:21:00.255 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.253+0000 7f753c074640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7534158cf0 0x7f7534150ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:00.255 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.253+0000 7f753c074640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f75341590d0 0x7f7534151ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:00.255 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.253+0000 7f753c074640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f753415f600 0x7f75341542b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:00.255 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.253+0000 7f753c074640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f753406c070 con 0x7f753415f600 2024-09-12T03:21:00.255 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.253+0000 7f753c074640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f753415fca0 con 0x7f7534158cf0 2024-09-12T03:21:00.255 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.253+0000 7f753c074640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f753406c1f0 con 0x7f75341590d0 2024-09-12T03:21:00.255 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.253+0000 7f753a611640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f753415f600 0x7f75341542b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:00.255 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.253+0000 7f753a611640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f753415f600 0x7f75341542b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38434/0 (socket says 172.21.15.27:38434) 2024-09-12T03:21:00.255 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.253+0000 7f753a611640 1 -- 172.21.15.27:0/1947596014 learned_addr learned my addr 172.21.15.27:0/1947596014 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:21:00.255 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.253+0000 7f7539e10640 1 --2- 172.21.15.27:0/1947596014 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7534158cf0 0x7f7534150ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:00.256 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.253+0000 7f753960f640 1 --2- 172.21.15.27:0/1947596014 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f75341590d0 0x7f7534151ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:00.256 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.254+0000 7f753a611640 1 -- 172.21.15.27:0/1947596014 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f75341590d0 msgr2=0x7f7534151ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:00.256 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.254+0000 7f753a611640 1 --2- 172.21.15.27:0/1947596014 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f75341590d0 0x7f7534151ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:00.256 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.254+0000 7f753a611640 1 -- 172.21.15.27:0/1947596014 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7534158cf0 msgr2=0x7f7534150ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:00.256 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.254+0000 7f753a611640 1 --2- 172.21.15.27:0/1947596014 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7534158cf0 0x7f7534150ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:00.256 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.254+0000 7f753a611640 1 -- 172.21.15.27:0/1947596014 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7534152000 con 0x7f753415f600 2024-09-12T03:21:00.256 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.254+0000 7f753a611640 1 --2- 172.21.15.27:0/1947596014 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f753415f600 0x7f75341542b0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f752c009e60 tx=0x7f752c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:00.256 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.254+0000 7f752a7fc640 1 -- 172.21.15.27:0/1947596014 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f752c018030 con 0x7f753415f600 2024-09-12T03:21:00.256 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.254+0000 7f752a7fc640 1 -- 172.21.15.27:0/1947596014 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f752c00cdf0 con 0x7f753415f600 2024-09-12T03:21:00.256 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.254+0000 7f752a7fc640 1 -- 172.21.15.27:0/1947596014 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f752c003bb0 con 0x7f753415f600 2024-09-12T03:21:00.256 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.254+0000 7f753c074640 1 -- 172.21.15.27:0/1947596014 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7534159710 con 0x7f753415f600 2024-09-12T03:21:00.256 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.254+0000 7f753c074640 1 -- 172.21.15.27:0/1947596014 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7534159cd0 con 0x7f753415f600 2024-09-12T03:21:00.256 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.255+0000 7f753c074640 1 -- 172.21.15.27:0/1947596014 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75340b2940 con 0x7f753415f600 2024-09-12T03:21:00.262 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.256+0000 7f752a7fc640 1 -- 172.21.15.27:0/1947596014 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 90600+0+0 (secure 0 0 0) 0x7f752c005d10 con 0x7f753415f600 2024-09-12T03:21:00.262 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.256+0000 7f752a7fc640 1 --2- 172.21.15.27:0/1947596014 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f751806bf50 0x7f751806e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:00.262 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.256+0000 7f752a7fc640 1 -- 172.21.15.27:0/1947596014 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 5709+0+0 (secure 0 0 0) 0x7f752c08b960 con 0x7f753415f600 2024-09-12T03:21:00.262 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.256+0000 7f7539e10640 1 --2- 172.21.15.27:0/1947596014 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f751806bf50 0x7f751806e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:00.262 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.257+0000 7f7539e10640 1 --2- 172.21.15.27:0/1947596014 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f751806bf50 0x7f751806e410 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f7530034000 tx=0x7f753002e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:00.262 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.261+0000 7f752a7fc640 1 -- 172.21.15.27:0/1947596014 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f75340b2940 con 0x7f753415f600 2024-09-12T03:21:00.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.436+0000 7f753c074640 1 -- 172.21.15.27:0/1947596014 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f753415c840 con 0x7f753415f600 2024-09-12T03:21:00.437 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.437+0000 7f752a7fc640 1 -- 172.21.15.27:0/1947596014 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v99) ==== 74+0+16641 (secure 0 0 0) 0x7f752c022070 con 0x7f753415f600 2024-09-12T03:21:00.437 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:21:00.437 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":99,"fsid":"d2df504e-2647-4385-a5db-9fdf69f487a0","created":"2024-09-12T03:16:07.778937+0000","modified":"2024-09-12T03:20:59.133496+0000","last_up_change":"2024-09-12T03:16:19.314778+0000","last_in_change":"2024-09-12T03:16:09.893086+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T03:16:20.097646+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-09-12T03:20:56.805568+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-09-12T03:20:58.128117+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1851116c-1fb3-4dc9-9772-f90e2ef4c355","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6817","nonce":3002321488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6819","nonce":3002321488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6823","nonce":3002321488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6821","nonce":3002321488}]},"public_addr":"172.21.15.27:6817/3002321488","cluster_addr":"172.21.15.27:6819/3002321488","heartbeat_back_addr":"172.21.15.27:6823/3002321488","heartbeat_front_addr":"172.21.15.27:6821/3002321488","state":["exists","up"]},{"osd":1,"uuid":"f38ae17d-73eb-4204-b99d-975fa806fb87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6809","nonce":1613102886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6811","nonce":1613102886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6815","nonce":1613102886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6813","nonce":1613102886}]},"public_addr":"172.21.15.27:6809/1613102886","cluster_addr":"172.21.15.27:6811/1613102886","heartbeat_back_addr":"172.21.15.27:6815/1613102886","heartbeat_front_addr":"172.21.15.27:6813/1613102886","state":["exists","up"]},{"osd":2,"uuid":"fb12dcc8-bdd0-42b8-b46d-8b10106224bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6825","nonce":3534723355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6827","nonce":3534723355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6831","nonce":3534723355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6829","nonce":3534723355}]},"public_addr":"172.21.15.27:6825/3534723355","cluster_addr":"172.21.15.27:6827/3534723355","heartbeat_back_addr":"172.21.15.27:6831/3534723355","heartbeat_front_addr":"172.21.15.27:6829/3534723355","state":["exists","up"]},{"osd":3,"uuid":"d74d3f49-9c24-4b9f-98e2-3ecd7b5ef3b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6801","nonce":80661598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6803","nonce":80661598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6807","nonce":80661598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6805","nonce":80661598}]},"public_addr":"172.21.15.27:6801/80661598","cluster_addr":"172.21.15.27:6803/80661598","heartbeat_back_addr":"172.21.15.27:6807/80661598","heartbeat_front_addr":"172.21.15.27:6805/80661598","state":["exists","up"]},{"osd":4,"uuid":"2bf123f2-1b0d-41c4-aa3b-805bab267e59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6817","nonce":3943189827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6819","nonce":3943189827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6823","nonce":3943189827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6821","nonce":3943189827}]},"public_addr":"172.21.15.45:6817/3943189827","cluster_addr":"172.21.15.45:6819/3943189827","heartbeat_back_addr":"172.21.15.45:6823/3943189827","heartbeat_front_addr":"172.21.15.45:6821/3943189827","state":["exists","up"]},{"osd":5,"uuid":"c49ba5da-91c6-4a86-a97b-4aba89c253ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6809","nonce":3620760330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6811","nonce":3620760330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6815","nonce":3620760330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6813","nonce":3620760330}]},"public_addr":"172.21.15.45:6809/3620760330","cluster_addr":"172.21.15.45:6811/3620760330","heartbeat_back_addr":"172.21.15.45:6815/3620760330","heartbeat_front_addr":"172.21.15.45:6813/3620760330","state":["exists","up"]},{"osd":6,"uuid":"b0123475-89f7-4a34-9e33-d797b99dd9dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6801","nonce":1989083510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6803","nonce":1989083510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6807","nonce":1989083510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6805","nonce":1989083510}]},"public_addr":"172.21.15.45:6801/1989083510","cluster_addr":"172.21.15.45:6803/1989083510","heartbeat_back_addr":"172.21.15.45:6807/1989083510","heartbeat_front_addr":"172.21.15.45:6805/1989083510","state":["exists","up"]},{"osd":7,"uuid":"4dad773d-b7d8-46fc-8a10-a972aad27b29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6825","nonce":1513300294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6827","nonce":1513300294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6831","nonce":1513300294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6829","nonce":1513300294}]},"public_addr":"172.21.15.45:6825/1513300294","cluster_addr":"172.21.15.45:6827/1513300294","heartbeat_back_addr":"172.21.15.45:6831/1513300294","heartbeat_front_addr":"172.21.15.45:6829/1513300294","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.881855+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:17.133252+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.735339+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.500742+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.477571+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:17.167667+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.470291+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.873743+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T03:21:00.439 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.439+0000 7f753c074640 1 -- 172.21.15.27:0/1947596014 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f751806bf50 msgr2=0x7f751806e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:00.439 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.439+0000 7f753c074640 1 --2- 172.21.15.27:0/1947596014 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f751806bf50 0x7f751806e410 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f7530034000 tx=0x7f753002e000 comp rx=0 tx=0).stop 2024-09-12T03:21:00.439 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.439+0000 7f753c074640 1 -- 172.21.15.27:0/1947596014 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f753415f600 msgr2=0x7f75341542b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:00.439 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.439+0000 7f753c074640 1 --2- 172.21.15.27:0/1947596014 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f753415f600 0x7f75341542b0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f752c009e60 tx=0x7f752c00c490 comp rx=0 tx=0).stop 2024-09-12T03:21:00.439 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.439+0000 7f753c074640 1 -- 172.21.15.27:0/1947596014 shutdown_connections 2024-09-12T03:21:00.439 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.439+0000 7f753c074640 1 --2- 172.21.15.27:0/1947596014 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f751806bf50 0x7f751806e410 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:00.439 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.439+0000 7f753c074640 1 --2- 172.21.15.27:0/1947596014 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f753415f600 0x7f75341542b0 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:00.439 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.439+0000 7f753c074640 1 --2- 172.21.15.27:0/1947596014 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f75341590d0 0x7f7534151ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:00.440 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.439+0000 7f753c074640 1 --2- 172.21.15.27:0/1947596014 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7534158cf0 0x7f7534150ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:00.440 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.439+0000 7f753c074640 1 -- 172.21.15.27:0/1947596014 >> 172.21.15.27:0/1947596014 conn(0x7f753406cc40 msgr2=0x7f75340b32e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:00.440 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.439+0000 7f753c074640 1 -- 172.21.15.27:0/1947596014 shutdown_connections 2024-09-12T03:21:00.440 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.439+0000 7f753c074640 1 -- 172.21.15.27:0/1947596014 wait complete. 2024-09-12T03:21:00.450 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-12T03:21:00.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.556+0000 7f670e4d2640 1 Processor -- start 2024-09-12T03:21:00.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.556+0000 7f670e4d2640 1 -- start start 2024-09-12T03:21:00.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.557+0000 7f670e4d2640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f67080d3790 0x7f67080d1da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:00.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.557+0000 7f670e4d2640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f67080d2e30 con 0x7f6708162040 2024-09-12T03:21:00.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.557+0000 7f670e4d2640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6708162420 con 0x7f670806dfd0 2024-09-12T03:21:00.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.557+0000 7f670e4d2640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f67080d30d0 con 0x7f67080d3790 2024-09-12T03:21:00.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.557+0000 7f6707fff640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f6708162040 0x7f670806d8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:47792/0 (socket says 172.21.15.27:47792) 2024-09-12T03:21:00.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.557+0000 7f6707fff640 1 -- 172.21.15.27:0/3530677348 learned_addr learned my addr 172.21.15.27:0/3530677348 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:21:00.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.557+0000 7f6706ffd640 1 -- 172.21.15.27:0/3530677348 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3295420991 0 0) 0x7f67080d2e30 con 0x7f6708162040 2024-09-12T03:21:00.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.557+0000 7f670ca6f640 1 --2- 172.21.15.27:0/3530677348 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f67080d3790 0x7f67080d1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:00.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.557+0000 7f6706ffd640 1 -- 172.21.15.27:0/3530677348 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f66f0003520 con 0x7f6708162040 2024-09-12T03:21:00.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.557+0000 7f6706ffd640 1 -- 172.21.15.27:0/3530677348 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2101734517 0 0) 0x7f66f0003520 con 0x7f6708162040 2024-09-12T03:21:00.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.557+0000 7f6706ffd640 1 -- 172.21.15.27:0/3530677348 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f67080d3790 msgr2=0x7f67080d1da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:00.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.557+0000 7f6706ffd640 1 --2- 172.21.15.27:0/3530677348 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f67080d3790 0x7f67080d1da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:00.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.557+0000 7f6706ffd640 1 -- 172.21.15.27:0/3530677348 >> v1:172.21.15.45:6789/0 conn(0x7f670806dfd0 legacy=0x7f67080cf320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:00.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.557+0000 7f6706ffd640 1 -- 172.21.15.27:0/3530677348 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67080d33b0 con 0x7f6708162040 2024-09-12T03:21:00.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.557+0000 7f6706ffd640 1 -- 172.21.15.27:0/3530677348 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 847755213 0 0) 0x7f66f8001670 con 0x7f6708162040 2024-09-12T03:21:00.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.557+0000 7f670ca6f640 1 --2- 172.21.15.27:0/3530677348 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f67080d3790 0x7f67080d1da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:21:00.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.557+0000 7f6706ffd640 1 -- 172.21.15.27:0/3530677348 >> v1:172.21.15.27:6789/0 conn(0x7f6708162040 legacy=0x7f670806d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:00.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.557+0000 7f6706ffd640 1 --2- 172.21.15.27:0/3530677348 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f66f00039e0 0x7f66f0003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:00.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.557+0000 7f6706ffd640 1 --2- 172.21.15.27:0/3530677348 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f66f0004800 0x7f66f0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:00.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.557+0000 7f6706ffd640 1 --2- 172.21.15.27:0/3530677348 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f67080d3790 0x7f66f00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:00.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.557+0000 7f6706ffd640 1 -- 172.21.15.27:0/3530677348 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f67080d30d0 con 0x7f66f0004800 2024-09-12T03:21:00.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.557+0000 7f6706ffd640 1 -- 172.21.15.27:0/3530677348 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f66f000bac0 con 0x7f66f00039e0 2024-09-12T03:21:00.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.557+0000 7f6706ffd640 1 -- 172.21.15.27:0/3530677348 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f66f000bcc0 con 0x7f67080d3790 2024-09-12T03:21:00.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.557+0000 7f67077fe640 1 --2- 172.21.15.27:0/3530677348 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f66f0004800 0x7f66f0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:00.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.557+0000 7f6707fff640 1 --2- 172.21.15.27:0/3530677348 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f66f00039e0 0x7f66f0003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:00.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.558+0000 7f670ca6f640 1 --2- 172.21.15.27:0/3530677348 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f67080d3790 0x7f66f00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:00.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.558+0000 7f67077fe640 1 -- 172.21.15.27:0/3530677348 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f67080d3790 msgr2=0x7f66f00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:00.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.558+0000 7f67077fe640 1 --2- 172.21.15.27:0/3530677348 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f67080d3790 0x7f66f00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:00.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.558+0000 7f67077fe640 1 -- 172.21.15.27:0/3530677348 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f66f00039e0 msgr2=0x7f66f0003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:00.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.558+0000 7f67077fe640 1 --2- 172.21.15.27:0/3530677348 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f66f00039e0 0x7f66f0003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:00.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.558+0000 7f67077fe640 1 -- 172.21.15.27:0/3530677348 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66f0003520 con 0x7f66f0004800 2024-09-12T03:21:00.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.559+0000 7f67077fe640 1 --2- 172.21.15.27:0/3530677348 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f66f0004800 0x7f66f0004c80 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f66fc001750 tx=0x7f66f0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b08000ca8587134 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:00.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.559+0000 7f6706ffd640 1 -- 172.21.15.27:0/3530677348 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66fc001370 con 0x7f66f0004800 2024-09-12T03:21:00.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.559+0000 7f6706ffd640 1 -- 172.21.15.27:0/3530677348 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66fc0052d0 con 0x7f66f0004800 2024-09-12T03:21:00.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.559+0000 7f6706ffd640 1 -- 172.21.15.27:0/3530677348 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66fc02abe0 con 0x7f66f0004800 2024-09-12T03:21:00.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.559+0000 7f670e4d2640 1 -- 172.21.15.27:0/3530677348 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f66f0004800 msgr2=0x7f66f0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:00.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.559+0000 7f670e4d2640 1 --2- 172.21.15.27:0/3530677348 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f66f0004800 0x7f66f0004c80 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f66fc001750 tx=0x7f66f0005830 comp rx=0 tx=0).stop 2024-09-12T03:21:00.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.559+0000 7f670ca6f640 1 -- 172.21.15.27:0/3530677348 reap_dead start 2024-09-12T03:21:00.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.559+0000 7f670e4d2640 1 -- 172.21.15.27:0/3530677348 shutdown_connections 2024-09-12T03:21:00.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.559+0000 7f670e4d2640 1 -- 172.21.15.27:0/3530677348 >> 172.21.15.27:0/3530677348 conn(0x7f67080b6700 msgr2=0x7f67080b6b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:00.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.559+0000 7f670e4d2640 1 -- 172.21.15.27:0/3530677348 shutdown_connections 2024-09-12T03:21:00.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.559+0000 7f670e4d2640 1 -- 172.21.15.27:0/3530677348 wait complete. 2024-09-12T03:21:00.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.560+0000 7f670e4d2640 1 Processor -- start 2024-09-12T03:21:00.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.561+0000 7f670e4d2640 1 -- start start 2024-09-12T03:21:00.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.561+0000 7f670e4d2640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f66f0004800 0x7f6708157cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:00.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.561+0000 7f670e4d2640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f670814ec80 0x7f6708159f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:00.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.561+0000 7f670e4d2640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6708158650 0x7f67080b8e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:00.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.561+0000 7f670e4d2640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f67080d4150 con 0x7f6708158650 2024-09-12T03:21:00.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.561+0000 7f670e4d2640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f67080d4530 con 0x7f66f0004800 2024-09-12T03:21:00.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.561+0000 7f670e4d2640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f67080d42d0 con 0x7f670814ec80 2024-09-12T03:21:00.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.561+0000 7f670ca6f640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6708158650 0x7f67080b8e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:00.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.561+0000 7f670ca6f640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6708158650 0x7f67080b8e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38466/0 (socket says 172.21.15.27:38466) 2024-09-12T03:21:00.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.561+0000 7f670ca6f640 1 -- 172.21.15.27:0/1202615824 learned_addr learned my addr 172.21.15.27:0/1202615824 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:21:00.567 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.562+0000 7f6707fff640 1 --2- 172.21.15.27:0/1202615824 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f66f0004800 0x7f6708157cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:00.567 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.562+0000 7f67077fe640 1 --2- 172.21.15.27:0/1202615824 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f670814ec80 0x7f6708159f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:00.567 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.562+0000 7f670ca6f640 1 -- 172.21.15.27:0/1202615824 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f670814ec80 msgr2=0x7f6708159f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:00.567 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.562+0000 7f670ca6f640 1 --2- 172.21.15.27:0/1202615824 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f670814ec80 0x7f6708159f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:00.567 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.562+0000 7f670ca6f640 1 -- 172.21.15.27:0/1202615824 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f66f0004800 msgr2=0x7f6708157cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:00.567 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.562+0000 7f670ca6f640 1 --2- 172.21.15.27:0/1202615824 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f66f0004800 0x7f6708157cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:00.567 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.562+0000 7f670ca6f640 1 -- 172.21.15.27:0/1202615824 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f670815a480 con 0x7f6708158650 2024-09-12T03:21:00.567 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.562+0000 7f67077fe640 1 --2- 172.21.15.27:0/1202615824 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f670814ec80 0x7f6708159f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:21:00.567 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.562+0000 7f6707fff640 1 --2- 172.21.15.27:0/1202615824 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f66f0004800 0x7f6708157cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:21:00.567 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.562+0000 7f670ca6f640 1 --2- 172.21.15.27:0/1202615824 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6708158650 0x7f67080b8e00 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f670808dcf0 tx=0x7f6700010530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:00.567 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.562+0000 7f6704ff9640 1 -- 172.21.15.27:0/1202615824 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f670001c070 con 0x7f6708158650 2024-09-12T03:21:00.567 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.562+0000 7f6704ff9640 1 -- 172.21.15.27:0/1202615824 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6700010cc0 con 0x7f6708158650 2024-09-12T03:21:00.567 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.562+0000 7f6704ff9640 1 -- 172.21.15.27:0/1202615824 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6700003bf0 con 0x7f6708158650 2024-09-12T03:21:00.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.562+0000 7f670e4d2640 1 -- 172.21.15.27:0/1202615824 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f670814f100 con 0x7f6708158650 2024-09-12T03:21:00.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.562+0000 7f670e4d2640 1 -- 172.21.15.27:0/1202615824 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6708159180 con 0x7f6708158650 2024-09-12T03:21:00.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.563+0000 7f670e4d2640 1 -- 172.21.15.27:0/1202615824 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f670806e6e0 con 0x7f6708158650 2024-09-12T03:21:00.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.564+0000 7f6704ff9640 1 -- 172.21.15.27:0/1202615824 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 90600+0+0 (secure 0 0 0) 0x7f670001f070 con 0x7f6708158650 2024-09-12T03:21:00.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.564+0000 7f6704ff9640 1 --2- 172.21.15.27:0/1202615824 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f66f406bf80 0x7f66f406e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:00.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.564+0000 7f6704ff9640 1 -- 172.21.15.27:0/1202615824 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 5709+0+0 (secure 0 0 0) 0x7f670008b1f0 con 0x7f6708158650 2024-09-12T03:21:00.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.564+0000 7f6707fff640 1 --2- 172.21.15.27:0/1202615824 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f66f406bf80 0x7f66f406e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:00.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.565+0000 7f6707fff640 1 --2- 172.21.15.27:0/1202615824 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f66f406bf80 0x7f66f406e440 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f67080b9b60 tx=0x7f66f8009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:00.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.568+0000 7f6704ff9640 1 -- 172.21.15.27:0/1202615824 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f6700059b20 con 0x7f6708158650 2024-09-12T03:21:00.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.782+0000 7f670e4d2640 1 -- 172.21.15.27:0/1202615824 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f670815f6d0 con 0x7f6708158650 2024-09-12T03:21:00.801 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.801+0000 7f6704ff9640 1 -- 172.21.15.27:0/1202615824 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v57) ==== 118+0+0 (secure 0 0 0) 0x7f6700059420 con 0x7f6708158650 2024-09-12T03:21:00.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.803+0000 7f670e4d2640 1 -- 172.21.15.27:0/1202615824 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f66f406bf80 msgr2=0x7f66f406e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:00.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.803+0000 7f670e4d2640 1 --2- 172.21.15.27:0/1202615824 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f66f406bf80 0x7f66f406e440 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f67080b9b60 tx=0x7f66f8009040 comp rx=0 tx=0).stop 2024-09-12T03:21:00.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.803+0000 7f670e4d2640 1 -- 172.21.15.27:0/1202615824 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6708158650 msgr2=0x7f67080b8e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:00.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.803+0000 7f670e4d2640 1 --2- 172.21.15.27:0/1202615824 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6708158650 0x7f67080b8e00 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f670808dcf0 tx=0x7f6700010530 comp rx=0 tx=0).stop 2024-09-12T03:21:00.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.803+0000 7f670e4d2640 1 -- 172.21.15.27:0/1202615824 shutdown_connections 2024-09-12T03:21:00.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.803+0000 7f670e4d2640 1 --2- 172.21.15.27:0/1202615824 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f66f406bf80 0x7f66f406e440 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:00.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.803+0000 7f670e4d2640 1 --2- 172.21.15.27:0/1202615824 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6708158650 0x7f67080b8e00 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:00.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.803+0000 7f670e4d2640 1 --2- 172.21.15.27:0/1202615824 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f670814ec80 0x7f6708159f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:00.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.804+0000 7f670e4d2640 1 --2- 172.21.15.27:0/1202615824 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f66f0004800 0x7f6708157cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:00.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.804+0000 7f670e4d2640 1 -- 172.21.15.27:0/1202615824 >> 172.21.15.27:0/1202615824 conn(0x7f67080b6700 msgr2=0x7f67080d1fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:00.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.804+0000 7f670e4d2640 1 -- 172.21.15.27:0/1202615824 shutdown_connections 2024-09-12T03:21:00.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.804+0000 7f670e4d2640 1 -- 172.21.15.27:0/1202615824 wait complete. 2024-09-12T03:21:00.815 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-12T03:21:00.815 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-12T03:21:00.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.920+0000 7fa831c82640 1 Processor -- start 2024-09-12T03:21:00.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.920+0000 7fa831c82640 1 -- start start 2024-09-12T03:21:00.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.921+0000 7fa831c82640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa82c15b3f0 0x7fa82c06eb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:00.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.921+0000 7fa831c82640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fa82c06fb90 0x7fa82c06ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:00.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.921+0000 7fa831c82640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa82c06f990 con 0x7fa82c070510 2024-09-12T03:21:00.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.921+0000 7fa831c82640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7fa82c15b7d0 con 0x7fa82c06fb90 2024-09-12T03:21:00.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.921+0000 7fa831c82640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fa82c06f170 con 0x7fa82c15b3f0 2024-09-12T03:21:00.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.921+0000 7fa82affd640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fa82c06fb90 0x7fa82c06ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:00.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.921+0000 7fa82affd640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fa82c06fb90 0x7fa82c06ffd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38482/0 (socket says 172.21.15.27:38482) 2024-09-12T03:21:00.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.921+0000 7fa82affd640 1 -- 172.21.15.27:0/4279736618 learned_addr learned my addr 172.21.15.27:0/4279736618 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:21:00.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.921+0000 7fa82b7fe640 1 --2- 172.21.15.27:0/4279736618 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa82c15b3f0 0x7fa82c06eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:00.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.921+0000 7fa82affd640 1 -- 172.21.15.27:0/4279736618 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa82c15b3f0 msgr2=0x7fa82c06eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:00.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.921+0000 7fa82affd640 1 --2- 172.21.15.27:0/4279736618 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa82c15b3f0 0x7fa82c06eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:00.923 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.922+0000 7fa82affd640 1 -- 172.21.15.27:0/4279736618 >> v1:172.21.15.45:6789/0 conn(0x7fa82c070510 legacy=0x7fa82c0d03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:00.923 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.922+0000 7fa82affd640 1 -- 172.21.15.27:0/4279736618 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa82c0cf610 con 0x7fa82c06fb90 2024-09-12T03:21:00.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.922+0000 7fa82affd640 1 --2- 172.21.15.27:0/4279736618 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa82c06fb90 0x7fa82c06ffd0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7fa820009870 tx=0x7fa82002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d312df35abba4f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:00.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.922+0000 7fa82a7fc640 1 -- 172.21.15.27:0/4279736618 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa820037070 con 0x7fa82c06fb90 2024-09-12T03:21:00.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.922+0000 7fa82a7fc640 1 -- 172.21.15.27:0/4279736618 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa820032440 con 0x7fa82c06fb90 2024-09-12T03:21:00.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.922+0000 7fa82a7fc640 1 -- 172.21.15.27:0/4279736618 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa820032810 con 0x7fa82c06fb90 2024-09-12T03:21:00.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.923+0000 7fa831c82640 1 -- 172.21.15.27:0/4279736618 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa82c06fb90 msgr2=0x7fa82c06ffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:00.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.923+0000 7fa831c82640 1 --2- 172.21.15.27:0/4279736618 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa82c06fb90 0x7fa82c06ffd0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7fa820009870 tx=0x7fa82002a270 comp rx=0 tx=0).stop 2024-09-12T03:21:00.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.923+0000 7fa831c82640 1 -- 172.21.15.27:0/4279736618 shutdown_connections 2024-09-12T03:21:00.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.923+0000 7fa831c82640 1 --2- 172.21.15.27:0/4279736618 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa82c06fb90 0x7fa82c06ffd0 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:00.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.923+0000 7fa831c82640 1 --2- 172.21.15.27:0/4279736618 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa82c15b3f0 0x7fa82c06eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:00.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.923+0000 7fa831c82640 1 -- 172.21.15.27:0/4279736618 >> 172.21.15.27:0/4279736618 conn(0x7fa82c06cc40 msgr2=0x7fa82c06d020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:00.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.923+0000 7fa831c82640 1 -- 172.21.15.27:0/4279736618 shutdown_connections 2024-09-12T03:21:00.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.923+0000 7fa831c82640 1 -- 172.21.15.27:0/4279736618 wait complete. 2024-09-12T03:21:00.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.924+0000 7fa831c82640 1 Processor -- start 2024-09-12T03:21:00.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.924+0000 7fa831c82640 1 -- start start 2024-09-12T03:21:00.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.925+0000 7fa831c82640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa82c06fb90 0x7fa82c15be20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:00.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.925+0000 7fa831c82640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa82c070510 0x7fa82c1f4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:00.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.925+0000 7fa831c82640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa82c15b3f0 0x7fa82c1f5220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:00.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.925+0000 7fa831c82640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fa82c0cfeb0 con 0x7fa82c06fb90 2024-09-12T03:21:00.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.925+0000 7fa831c82640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa82c0cfbd0 con 0x7fa82c070510 2024-09-12T03:21:00.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.925+0000 7fa831c82640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fa82c070920 con 0x7fa82c15b3f0 2024-09-12T03:21:00.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.925+0000 7fa82b7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa82c06fb90 0x7fa82c15be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:00.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.925+0000 7fa82b7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa82c06fb90 0x7fa82c15be20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38496/0 (socket says 172.21.15.27:38496) 2024-09-12T03:21:00.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.925+0000 7fa82b7fe640 1 -- 172.21.15.27:0/2429556810 learned_addr learned my addr 172.21.15.27:0/2429556810 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:21:00.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.925+0000 7fa82affd640 1 --2- 172.21.15.27:0/2429556810 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa82c070510 0x7fa82c1f4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:00.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.925+0000 7fa82bfff640 1 --2- 172.21.15.27:0/2429556810 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa82c15b3f0 0x7fa82c1f5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:00.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.926+0000 7fa82b7fe640 1 -- 172.21.15.27:0/2429556810 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa82c15b3f0 msgr2=0x7fa82c1f5220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:00.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.926+0000 7fa82b7fe640 1 --2- 172.21.15.27:0/2429556810 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa82c15b3f0 0x7fa82c1f5220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:00.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.926+0000 7fa82b7fe640 1 -- 172.21.15.27:0/2429556810 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa82c070510 msgr2=0x7fa82c1f4ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:00.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.926+0000 7fa82b7fe640 1 --2- 172.21.15.27:0/2429556810 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa82c070510 0x7fa82c1f4ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:00.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.926+0000 7fa82b7fe640 1 -- 172.21.15.27:0/2429556810 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa82c15c9f0 con 0x7fa82c06fb90 2024-09-12T03:21:00.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.926+0000 7fa82bfff640 1 --2- 172.21.15.27:0/2429556810 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa82c15b3f0 0x7fa82c1f5220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:21:00.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.926+0000 7fa82b7fe640 1 --2- 172.21.15.27:0/2429556810 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa82c06fb90 0x7fa82c15be20 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7fa81c00d800 tx=0x7fa81c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:00.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.926+0000 7fa813fff640 1 -- 172.21.15.27:0/2429556810 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa81c00b6c0 con 0x7fa82c06fb90 2024-09-12T03:21:00.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.926+0000 7fa813fff640 1 -- 172.21.15.27:0/2429556810 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa81c019070 con 0x7fa82c06fb90 2024-09-12T03:21:00.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.926+0000 7fa813fff640 1 -- 172.21.15.27:0/2429556810 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa81c00bac0 con 0x7fa82c06fb90 2024-09-12T03:21:00.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.926+0000 7fa831c82640 1 -- 172.21.15.27:0/2429556810 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa82c15c430 con 0x7fa82c06fb90 2024-09-12T03:21:00.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.926+0000 7fa831c82640 1 -- 172.21.15.27:0/2429556810 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa82c1e5bc0 con 0x7fa82c06fb90 2024-09-12T03:21:00.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.927+0000 7fa831c82640 1 -- 172.21.15.27:0/2429556810 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa82c0b0ae0 con 0x7fa82c06fb90 2024-09-12T03:21:00.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.927+0000 7fa813fff640 1 -- 172.21.15.27:0/2429556810 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 90600+0+0 (secure 0 0 0) 0x7fa81c00bc60 con 0x7fa82c06fb90 2024-09-12T03:21:00.930 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.928+0000 7fa813fff640 1 --2- 172.21.15.27:0/2429556810 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fa81806bf50 0x7fa81806e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:00.930 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.928+0000 7fa813fff640 1 -- 172.21.15.27:0/2429556810 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5709+0+0 (secure 0 0 0) 0x7fa81c08b3f0 con 0x7fa82c06fb90 2024-09-12T03:21:00.930 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.928+0000 7fa82affd640 1 --2- 172.21.15.27:0/2429556810 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fa81806bf50 0x7fa81806e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:00.930 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.928+0000 7fa82affd640 1 --2- 172.21.15.27:0/2429556810 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fa81806bf50 0x7fa81806e410 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fa820034000 tx=0x7fa82002e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:00.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:00.932+0000 7fa813fff640 1 -- 172.21.15.27:0/2429556810 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fa81c059c90 con 0x7fa82c06fb90 2024-09-12T03:21:01.001 DEBUG:teuthology.orchestra.run.smithi027:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-12T03:21:01.004 DEBUG:teuthology.orchestra.run.smithi045:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-12T03:21:01.147 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:01.147+0000 7fa831c82640 1 -- 172.21.15.27:0/2429556810 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fa82c0b2430 con 0x7fa82c06fb90 2024-09-12T03:21:01.766 INFO:tasks.ceph.mds.a.smithi027.stderr:2024-09-12T03:21:01.766+0000 7f37935dc640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-12T03:21:01.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:01.806+0000 7fa813fff640 1 -- 172.21.15.27:0/2429556810 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v58) ==== 122+0+0 (secure 0 0 0) 0x7fa81c059590 con 0x7fa82c06fb90 2024-09-12T03:21:01.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:01.808+0000 7fa831c82640 1 -- 172.21.15.27:0/2429556810 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fa81806bf50 msgr2=0x7fa81806e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:01.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:01.808+0000 7fa831c82640 1 --2- 172.21.15.27:0/2429556810 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fa81806bf50 0x7fa81806e410 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fa820034000 tx=0x7fa82002e000 comp rx=0 tx=0).stop 2024-09-12T03:21:01.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:01.809+0000 7fa831c82640 1 -- 172.21.15.27:0/2429556810 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa82c06fb90 msgr2=0x7fa82c15be20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:01.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:01.809+0000 7fa831c82640 1 --2- 172.21.15.27:0/2429556810 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa82c06fb90 0x7fa82c15be20 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7fa81c00d800 tx=0x7fa81c00dcc0 comp rx=0 tx=0).stop 2024-09-12T03:21:01.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:01.809+0000 7fa831c82640 1 -- 172.21.15.27:0/2429556810 shutdown_connections 2024-09-12T03:21:01.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:01.809+0000 7fa831c82640 1 --2- 172.21.15.27:0/2429556810 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa82c15b3f0 0x7fa82c1f5220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:01.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:01.809+0000 7fa831c82640 1 --2- 172.21.15.27:0/2429556810 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa82c070510 0x7fa82c1f4ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:01.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:01.809+0000 7fa831c82640 1 --2- 172.21.15.27:0/2429556810 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fa81806bf50 0x7fa81806e410 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:01.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:01.809+0000 7fa831c82640 1 --2- 172.21.15.27:0/2429556810 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa82c06fb90 0x7fa82c15be20 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:01.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:01.809+0000 7fa831c82640 1 -- 172.21.15.27:0/2429556810 >> 172.21.15.27:0/2429556810 conn(0x7fa82c06cc40 msgr2=0x7fa82c0721d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:01.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:01.809+0000 7fa831c82640 1 -- 172.21.15.27:0/2429556810 shutdown_connections 2024-09-12T03:21:01.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:01.809+0000 7fa831c82640 1 -- 172.21.15.27:0/2429556810 wait complete. 2024-09-12T03:21:01.821 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-12T03:21:01.822 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-12T03:21:01.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:01.929+0000 7f1c3260e640 1 Processor -- start 2024-09-12T03:21:01.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:01.929+0000 7f1c3260e640 1 -- start start 2024-09-12T03:21:01.930 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:01.929+0000 7f1c3260e640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f1c2c0d5200 0x7f1c2c0d1010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:01.930 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:01.929+0000 7f1c3260e640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1c2c0d1550 0x7f1c2c0b8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:01.930 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:01.929+0000 7f1c3260e640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1c2c0d3760 con 0x7f1c2c15eb90 2024-09-12T03:21:01.930 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:01.929+0000 7f1c3260e640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f1c2c0b7d20 con 0x7f1c2c0d5200 2024-09-12T03:21:01.930 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:01.929+0000 7f1c3260e640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f1c2c0d42e0 con 0x7f1c2c0d1550 2024-09-12T03:21:01.930 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:01.930+0000 7f1c2b7fe640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f1c2c0d5200 0x7f1c2c0d1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:01.930 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:01.930+0000 7f1c30bab640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1c2c0d1550 0x7f1c2c0b8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:01.930 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:01.930+0000 7f1c30bab640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1c2c0d1550 0x7f1c2c0b8190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3301/0 says I am v2:172.21.15.27:60666/0 (socket says 172.21.15.27:60666) 2024-09-12T03:21:01.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:01.930+0000 7f1c30bab640 1 -- 172.21.15.27:0/1858374356 learned_addr learned my addr 172.21.15.27:0/1858374356 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:21:01.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:01.930+0000 7f1c2bfff640 1 --1- 172.21.15.27:0/1858374356 >> v1:172.21.15.45:6789/0 conn(0x7f1c2c15eb90 0x7f1c2c0d2df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.27:36302/0 (socket says 172.21.15.27:36302) 2024-09-12T03:21:01.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:01.930+0000 7f1c30bab640 1 -- 172.21.15.27:0/1858374356 >> v2:172.21.15.45:3300/0 conn(0x7f1c2c0d5200 msgr2=0x7f1c2c0d1010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:01.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:01.930+0000 7f1c30bab640 1 --2- 172.21.15.27:0/1858374356 >> v2:172.21.15.45:3300/0 conn(0x7f1c2c0d5200 0x7f1c2c0d1010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:01.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:01.930+0000 7f1c30bab640 1 -- 172.21.15.27:0/1858374356 >> v1:172.21.15.45:6789/0 conn(0x7f1c2c15eb90 legacy=0x7f1c2c0d2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:01.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:01.930+0000 7f1c30bab640 1 -- 172.21.15.27:0/1858374356 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c2c0d3a40 con 0x7f1c2c0d1550 2024-09-12T03:21:01.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:01.930+0000 7f1c2b7fe640 1 --2- 172.21.15.27:0/1858374356 >> v2:172.21.15.45:3300/0 conn(0x7f1c2c0d5200 0x7f1c2c0d1010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:21:01.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:01.931+0000 7f1c30bab640 1 --2- 172.21.15.27:0/1858374356 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1c2c0d1550 0x7f1c2c0b8190 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f1c24009e50 tx=0x7f1c2402a710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9742bc2bdf974f6b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:01.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:01.931+0000 7f1c2affd640 1 -- 172.21.15.27:0/1858374356 <== mon.2 v2:172.21.15.45:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c24036070 con 0x7f1c2c0d1550 2024-09-12T03:21:01.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:01.931+0000 7f1c2affd640 1 -- 172.21.15.27:0/1858374356 <== mon.2 v2:172.21.15.45:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c24031600 con 0x7f1c2c0d1550 2024-09-12T03:21:01.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:01.931+0000 7f1c2affd640 1 -- 172.21.15.27:0/1858374356 <== mon.2 v2:172.21.15.45:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c240319d0 con 0x7f1c2c0d1550 2024-09-12T03:21:01.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:01.932+0000 7f1c3260e640 1 -- 172.21.15.27:0/1858374356 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1c2c0d1550 msgr2=0x7f1c2c0b8190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:01.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:01.932+0000 7f1c3260e640 1 --2- 172.21.15.27:0/1858374356 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1c2c0d1550 0x7f1c2c0b8190 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f1c24009e50 tx=0x7f1c2402a710 comp rx=0 tx=0).stop 2024-09-12T03:21:01.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:01.932+0000 7f1c3260e640 1 -- 172.21.15.27:0/1858374356 shutdown_connections 2024-09-12T03:21:01.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:01.932+0000 7f1c3260e640 1 --2- 172.21.15.27:0/1858374356 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1c2c0d1550 0x7f1c2c0b8190 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:01.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:01.932+0000 7f1c3260e640 1 --2- 172.21.15.27:0/1858374356 >> v2:172.21.15.45:3300/0 conn(0x7f1c2c0d5200 0x7f1c2c0d1010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:01.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:01.932+0000 7f1c3260e640 1 -- 172.21.15.27:0/1858374356 >> 172.21.15.27:0/1858374356 conn(0x7f1c2c06dce0 msgr2=0x7f1c2c06e0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:01.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:01.932+0000 7f1c3260e640 1 -- 172.21.15.27:0/1858374356 shutdown_connections 2024-09-12T03:21:01.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:01.932+0000 7f1c3260e640 1 -- 172.21.15.27:0/1858374356 wait complete. 2024-09-12T03:21:01.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:01.933+0000 7f1c3260e640 1 Processor -- start 2024-09-12T03:21:01.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:01.933+0000 7f1c3260e640 1 -- start start 2024-09-12T03:21:01.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:01.934+0000 7f1c3260e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1c2c0d1550 0x7f1c2c15b500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:01.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:01.934+0000 7f1c3260e640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1c2c0d5200 0x7f1c2c158da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:01.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:01.934+0000 7f1c3260e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1c2c15eb90 0x7f1c2c1592e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:01.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:01.934+0000 7f1c3260e640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f1c2c0d4000 con 0x7f1c2c15eb90 2024-09-12T03:21:01.936 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:01.934+0000 7f1c3260e640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f1c2c0d3500 con 0x7f1c2c0d1550 2024-09-12T03:21:01.936 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:01.934+0000 7f1c3260e640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f1c2c0d19a0 con 0x7f1c2c0d5200 2024-09-12T03:21:01.936 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:01.934+0000 7f1c30bab640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1c2c15eb90 0x7f1c2c1592e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:01.936 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:01.934+0000 7f1c30bab640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1c2c15eb90 0x7f1c2c1592e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38512/0 (socket says 172.21.15.27:38512) 2024-09-12T03:21:01.936 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:01.934+0000 7f1c30bab640 1 -- 172.21.15.27:0/1482381229 learned_addr learned my addr 172.21.15.27:0/1482381229 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:21:01.936 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:01.934+0000 7f1c2bfff640 1 --2- 172.21.15.27:0/1482381229 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1c2c0d1550 0x7f1c2c15b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:01.936 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:01.934+0000 7f1c2b7fe640 1 --2- 172.21.15.27:0/1482381229 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1c2c0d5200 0x7f1c2c158da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:01.936 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:01.934+0000 7f1c30bab640 1 -- 172.21.15.27:0/1482381229 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1c2c0d5200 msgr2=0x7f1c2c158da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:01.936 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:01.934+0000 7f1c30bab640 1 --2- 172.21.15.27:0/1482381229 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1c2c0d5200 0x7f1c2c158da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:01.936 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:01.934+0000 7f1c30bab640 1 -- 172.21.15.27:0/1482381229 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1c2c0d1550 msgr2=0x7f1c2c15b500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:01.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:01.934+0000 7f1c30bab640 1 --2- 172.21.15.27:0/1482381229 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1c2c0d1550 0x7f1c2c15b500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:01.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:01.934+0000 7f1c30bab640 1 -- 172.21.15.27:0/1482381229 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c2c15ba40 con 0x7f1c2c15eb90 2024-09-12T03:21:01.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:01.935+0000 7f1c30bab640 1 --2- 172.21.15.27:0/1482381229 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1c2c15eb90 0x7f1c2c1592e0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f1c24034040 tx=0x7f1c24004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:01.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:01.935+0000 7f1c28ff9640 1 -- 172.21.15.27:0/1482381229 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c24036050 con 0x7f1c2c15eb90 2024-09-12T03:21:01.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:01.935+0000 7f1c28ff9640 1 -- 172.21.15.27:0/1482381229 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c24042070 con 0x7f1c2c15eb90 2024-09-12T03:21:01.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:01.935+0000 7f1c3260e640 1 -- 172.21.15.27:0/1482381229 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c2c1f4a30 con 0x7f1c2c15eb90 2024-09-12T03:21:01.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:01.935+0000 7f1c28ff9640 1 -- 172.21.15.27:0/1482381229 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c24009510 con 0x7f1c2c15eb90 2024-09-12T03:21:01.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:01.935+0000 7f1c3260e640 1 -- 172.21.15.27:0/1482381229 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c2c1f4ff0 con 0x7f1c2c15eb90 2024-09-12T03:21:01.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:01.936+0000 7f1c3260e640 1 -- 172.21.15.27:0/1482381229 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c2c15f090 con 0x7f1c2c15eb90 2024-09-12T03:21:01.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:01.936+0000 7f1c28ff9640 1 -- 172.21.15.27:0/1482381229 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 90600+0+0 (secure 0 0 0) 0x7f1c24007bf0 con 0x7f1c2c15eb90 2024-09-12T03:21:01.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:01.937+0000 7f1c28ff9640 1 --2- 172.21.15.27:0/1482381229 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f1c1806bf50 0x7f1c1806e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:01.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:01.937+0000 7f1c28ff9640 1 -- 172.21.15.27:0/1482381229 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5709+0+0 (secure 0 0 0) 0x7f1c240abd60 con 0x7f1c2c15eb90 2024-09-12T03:21:01.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:01.937+0000 7f1c2bfff640 1 --2- 172.21.15.27:0/1482381229 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f1c1806bf50 0x7f1c1806e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:01.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:01.937+0000 7f1c2bfff640 1 --2- 172.21.15.27:0/1482381229 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f1c1806bf50 0x7f1c1806e410 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f1c2c161e70 tx=0x7f1c1c007e80 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:01.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:01.941+0000 7f1c28ff9640 1 -- 172.21.15.27:0/1482381229 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f1c2407a050 con 0x7f1c2c15eb90 2024-09-12T03:21:02.152 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.152+0000 7f1c3260e640 1 -- 172.21.15.27:0/1482381229 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1c2c0cfee0 con 0x7f1c2c15eb90 2024-09-12T03:21:02.152 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.152+0000 7f1c28ff9640 1 -- 172.21.15.27:0/1482381229 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) ==== 94+0+5222 (secure 0 0 0) 0x7f1c2407e350 con 0x7f1c2c15eb90 2024-09-12T03:21:02.153 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:21:02.153 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":58,"btime":"2024-09-12T03:21:01:802409+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5885,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/2343815091","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2343815091},{"type":"v1","addr":"172.21.15.45:6834","nonce":2343815091}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":55},{"gid":6376,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/512483568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":512483568},{"type":"v1","addr":"172.21.15.45:6835","nonce":512483568}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":55},{"gid":6825,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6834/1888895227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":1888895227},{"type":"v1","addr":"172.21.15.27:6834","nonce":1888895227}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":54}],"filesystems":[{"mdsmap":{"epoch":58,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-12T03:20:59.133824+0000","modified":"2024-09-12T03:21:01.148633+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6828},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6828":{"gid":6828,"name":"a","rank":0,"incarnation":56,"state":"up:active","state_seq":3,"addr":"172.21.15.27:6835/1379166526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":1379166526},{"type":"v1","addr":"172.21.15.27:6835","nonce":1379166526}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6828,"qdb_cluster":[6828]},"id":8}]} 2024-09-12T03:21:02.153 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 58 2024-09-12T03:21:02.155 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.154+0000 7f1c3260e640 1 -- 172.21.15.27:0/1482381229 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f1c1806bf50 msgr2=0x7f1c1806e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:02.155 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.154+0000 7f1c3260e640 1 --2- 172.21.15.27:0/1482381229 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f1c1806bf50 0x7f1c1806e410 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f1c2c161e70 tx=0x7f1c1c007e80 comp rx=0 tx=0).stop 2024-09-12T03:21:02.155 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.154+0000 7f1c3260e640 1 -- 172.21.15.27:0/1482381229 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1c2c15eb90 msgr2=0x7f1c2c1592e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:02.155 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.154+0000 7f1c3260e640 1 --2- 172.21.15.27:0/1482381229 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1c2c15eb90 0x7f1c2c1592e0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f1c24034040 tx=0x7f1c24004a40 comp rx=0 tx=0).stop 2024-09-12T03:21:02.155 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.155+0000 7f1c3260e640 1 -- 172.21.15.27:0/1482381229 shutdown_connections 2024-09-12T03:21:02.155 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.155+0000 7f1c3260e640 1 --2- 172.21.15.27:0/1482381229 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f1c1806bf50 0x7f1c1806e410 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:02.155 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.155+0000 7f1c3260e640 1 --2- 172.21.15.27:0/1482381229 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1c2c15eb90 0x7f1c2c1592e0 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:02.155 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.155+0000 7f1c3260e640 1 --2- 172.21.15.27:0/1482381229 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1c2c0d5200 0x7f1c2c158da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:02.155 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.155+0000 7f1c3260e640 1 --2- 172.21.15.27:0/1482381229 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1c2c0d1550 0x7f1c2c15b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:02.155 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.155+0000 7f1c3260e640 1 -- 172.21.15.27:0/1482381229 >> 172.21.15.27:0/1482381229 conn(0x7f1c2c06dce0 msgr2=0x7f1c2c1611a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:02.156 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.155+0000 7f1c3260e640 1 -- 172.21.15.27:0/1482381229 shutdown_connections 2024-09-12T03:21:02.156 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.155+0000 7f1c3260e640 1 -- 172.21.15.27:0/1482381229 wait complete. 2024-09-12T03:21:02.166 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-12T03:21:02.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.268+0000 7f72f23f6640 1 Processor -- start 2024-09-12T03:21:02.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.268+0000 7f72f23f6640 1 -- start start 2024-09-12T03:21:02.269 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.269+0000 7f72f23f6640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f72ec0ae8d0 0x7f72ec0728b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:02.269 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.269+0000 7f72f23f6640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f72ec072df0 0x7f72ec06e930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:02.269 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.269+0000 7f72f23f6640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f72ec0aee90 con 0x7f72ec0b5a70 2024-09-12T03:21:02.270 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.269+0000 7f72f23f6640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f72ec073240 con 0x7f72ec0ae8d0 2024-09-12T03:21:02.270 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.269+0000 7f72f23f6640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f72ec0af170 con 0x7f72ec072df0 2024-09-12T03:21:02.270 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.269+0000 7f72eb7fe640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f72ec0ae8d0 0x7f72ec0728b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:02.270 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.269+0000 7f72f0993640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f72ec072df0 0x7f72ec06e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:02.270 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.270+0000 7f72ebfff640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f72ec0b5a70 0x7f72ec15f600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.27:36306/0 (socket says 172.21.15.27:36306) 2024-09-12T03:21:02.270 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.270+0000 7f72ebfff640 1 -- 172.21.15.27:0/3037570809 learned_addr learned my addr 172.21.15.27:0/3037570809 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:21:02.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.270+0000 7f72eaffd640 1 -- 172.21.15.27:0/3037570809 <== mon.1 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 846018955 0 0) 0x7f72ec0aee90 con 0x7f72ec0b5a70 2024-09-12T03:21:02.272 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.270+0000 7f72eaffd640 1 -- 172.21.15.27:0/3037570809 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f72d4003520 con 0x7f72ec0b5a70 2024-09-12T03:21:02.272 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.270+0000 7f72eb7fe640 1 -- 172.21.15.27:0/3037570809 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f72ec072df0 msgr2=0x7f72ec06e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:02.272 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.270+0000 7f72eb7fe640 1 --2- 172.21.15.27:0/3037570809 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f72ec072df0 0x7f72ec06e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:02.272 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.270+0000 7f72eb7fe640 1 -- 172.21.15.27:0/3037570809 >> v1:172.21.15.45:6789/0 conn(0x7f72ec0b5a70 legacy=0x7f72ec15f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:02.272 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.270+0000 7f72eb7fe640 1 -- 172.21.15.27:0/3037570809 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72ec0af450 con 0x7f72ec0ae8d0 2024-09-12T03:21:02.272 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.271+0000 7f72eb7fe640 1 --2- 172.21.15.27:0/3037570809 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f72ec0ae8d0 0x7f72ec0728b0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f72e00097a0 tx=0x7f72e002a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6764a57463c4237a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:02.272 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.271+0000 7f72eaffd640 1 -- 172.21.15.27:0/3037570809 <== mon.1 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72e0037070 con 0x7f72ec0ae8d0 2024-09-12T03:21:02.272 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.271+0000 7f72eaffd640 1 -- 172.21.15.27:0/3037570809 <== mon.1 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f72e002ae50 con 0x7f72ec0ae8d0 2024-09-12T03:21:02.272 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.271+0000 7f72eaffd640 1 -- 172.21.15.27:0/3037570809 <== mon.1 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72e0032660 con 0x7f72ec0ae8d0 2024-09-12T03:21:02.272 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.271+0000 7f72f23f6640 1 -- 172.21.15.27:0/3037570809 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f72ec0ae8d0 msgr2=0x7f72ec0728b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:02.272 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.271+0000 7f72f23f6640 1 --2- 172.21.15.27:0/3037570809 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f72ec0ae8d0 0x7f72ec0728b0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f72e00097a0 tx=0x7f72e002a100 comp rx=0 tx=0).stop 2024-09-12T03:21:02.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.272+0000 7f72f23f6640 1 -- 172.21.15.27:0/3037570809 shutdown_connections 2024-09-12T03:21:02.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.272+0000 7f72f23f6640 1 --2- 172.21.15.27:0/3037570809 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f72ec072df0 0x7f72ec06e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:02.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.272+0000 7f72f23f6640 1 --2- 172.21.15.27:0/3037570809 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f72ec0ae8d0 0x7f72ec0728b0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:02.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.272+0000 7f72f23f6640 1 -- 172.21.15.27:0/3037570809 >> 172.21.15.27:0/3037570809 conn(0x7f72ec06ca30 msgr2=0x7f72ec06ce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:02.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.272+0000 7f72f23f6640 1 -- 172.21.15.27:0/3037570809 shutdown_connections 2024-09-12T03:21:02.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.272+0000 7f72f23f6640 1 -- 172.21.15.27:0/3037570809 wait complete. 2024-09-12T03:21:02.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.273+0000 7f72f23f6640 1 Processor -- start 2024-09-12T03:21:02.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.273+0000 7f72f23f6640 1 -- start start 2024-09-12T03:21:02.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.274+0000 7f72f23f6640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f72ec072df0 0x7f72ec15cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:02.276 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.274+0000 7f72f23f6640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f72ec0ae8d0 0x7f72ec15af60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:02.276 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.274+0000 7f72f23f6640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f72ec0b5a70 0x7f72ec15b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:02.276 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.274+0000 7f72f23f6640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f72ec06eff0 con 0x7f72ec072df0 2024-09-12T03:21:02.276 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.274+0000 7f72f23f6640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f72ec06ee70 con 0x7f72ec0b5a70 2024-09-12T03:21:02.276 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.274+0000 7f72f23f6640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f72ec06f170 con 0x7f72ec0ae8d0 2024-09-12T03:21:02.276 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.274+0000 7f72ebfff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f72ec072df0 0x7f72ec15cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:02.276 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.274+0000 7f72ebfff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f72ec072df0 0x7f72ec15cc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38514/0 (socket says 172.21.15.27:38514) 2024-09-12T03:21:02.276 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.274+0000 7f72ebfff640 1 -- 172.21.15.27:0/2724339667 learned_addr learned my addr 172.21.15.27:0/2724339667 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:21:02.276 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.274+0000 7f72f0993640 1 --2- 172.21.15.27:0/2724339667 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f72ec0b5a70 0x7f72ec15b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:02.276 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.274+0000 7f72eb7fe640 1 --2- 172.21.15.27:0/2724339667 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f72ec0ae8d0 0x7f72ec15af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:02.276 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.274+0000 7f72ebfff640 1 -- 172.21.15.27:0/2724339667 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f72ec0ae8d0 msgr2=0x7f72ec15af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:02.276 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.274+0000 7f72ebfff640 1 --2- 172.21.15.27:0/2724339667 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f72ec0ae8d0 0x7f72ec15af60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:02.277 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.274+0000 7f72ebfff640 1 -- 172.21.15.27:0/2724339667 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f72ec0b5a70 msgr2=0x7f72ec15b4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:02.277 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.274+0000 7f72ebfff640 1 --2- 172.21.15.27:0/2724339667 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f72ec0b5a70 0x7f72ec15b4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:02.277 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.274+0000 7f72ebfff640 1 -- 172.21.15.27:0/2724339667 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72ec15bcd0 con 0x7f72ec072df0 2024-09-12T03:21:02.277 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.274+0000 7f72ebfff640 1 --2- 172.21.15.27:0/2724339667 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f72ec072df0 0x7f72ec15cc60 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f72dc00c400 tx=0x7f72dc00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:02.277 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.274+0000 7f72e8ff9640 1 -- 172.21.15.27:0/2724339667 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72dc017070 con 0x7f72ec072df0 2024-09-12T03:21:02.277 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.275+0000 7f72e8ff9640 1 -- 172.21.15.27:0/2724339667 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f72dc005d00 con 0x7f72ec072df0 2024-09-12T03:21:02.277 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.275+0000 7f72f23f6640 1 -- 172.21.15.27:0/2724339667 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72ec15bfb0 con 0x7f72ec072df0 2024-09-12T03:21:02.277 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.275+0000 7f72e8ff9640 1 -- 172.21.15.27:0/2724339667 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72dc007310 con 0x7f72ec072df0 2024-09-12T03:21:02.277 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.275+0000 7f72f23f6640 1 -- 172.21.15.27:0/2724339667 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72ec0cdfe0 con 0x7f72ec072df0 2024-09-12T03:21:02.277 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.275+0000 7f72f23f6640 1 -- 172.21.15.27:0/2724339667 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72ec06e930 con 0x7f72ec072df0 2024-09-12T03:21:02.278 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.276+0000 7f72e8ff9640 1 -- 172.21.15.27:0/2724339667 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 90600+0+0 (secure 0 0 0) 0x7f72dc01a070 con 0x7f72ec072df0 2024-09-12T03:21:02.278 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.277+0000 7f72e8ff9640 1 --2- 172.21.15.27:0/2724339667 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f72d806bf50 0x7f72d806e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:02.278 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.277+0000 7f72e8ff9640 1 -- 172.21.15.27:0/2724339667 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5709+0+0 (secure 0 0 0) 0x7f72dc08a2e0 con 0x7f72ec072df0 2024-09-12T03:21:02.278 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.277+0000 7f72eb7fe640 1 --2- 172.21.15.27:0/2724339667 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f72d806bf50 0x7f72d806e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:02.278 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.277+0000 7f72eb7fe640 1 --2- 172.21.15.27:0/2724339667 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f72d806bf50 0x7f72d806e410 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f72e0034000 tx=0x7f72e002e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:02.281 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.281+0000 7f72e8ff9640 1 -- 172.21.15.27:0/2724339667 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f72dc058540 con 0x7f72ec072df0 2024-09-12T03:21:02.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.457+0000 7f72f23f6640 1 -- 172.21.15.27:0/2724339667 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f72ec072870 con 0x7f72ec072df0 2024-09-12T03:21:02.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.458+0000 7f72e8ff9640 1 -- 172.21.15.27:0/2724339667 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v100) ==== 74+0+16630 (secure 0 0 0) 0x7f72dc05c840 con 0x7f72ec072df0 2024-09-12T03:21:02.459 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:21:02.459 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":100,"fsid":"d2df504e-2647-4385-a5db-9fdf69f487a0","created":"2024-09-12T03:16:07.778937+0000","modified":"2024-09-12T03:21:00.796149+0000","last_up_change":"2024-09-12T03:16:19.314778+0000","last_in_change":"2024-09-12T03:16:09.893086+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T03:16:20.097646+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-09-12T03:20:56.805568+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-09-12T03:20:58.128117+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"100","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1851116c-1fb3-4dc9-9772-f90e2ef4c355","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6817","nonce":3002321488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6819","nonce":3002321488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6823","nonce":3002321488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6821","nonce":3002321488}]},"public_addr":"172.21.15.27:6817/3002321488","cluster_addr":"172.21.15.27:6819/3002321488","heartbeat_back_addr":"172.21.15.27:6823/3002321488","heartbeat_front_addr":"172.21.15.27:6821/3002321488","state":["exists","up"]},{"osd":1,"uuid":"f38ae17d-73eb-4204-b99d-975fa806fb87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6809","nonce":1613102886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6811","nonce":1613102886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6815","nonce":1613102886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6813","nonce":1613102886}]},"public_addr":"172.21.15.27:6809/1613102886","cluster_addr":"172.21.15.27:6811/1613102886","heartbeat_back_addr":"172.21.15.27:6815/1613102886","heartbeat_front_addr":"172.21.15.27:6813/1613102886","state":["exists","up"]},{"osd":2,"uuid":"fb12dcc8-bdd0-42b8-b46d-8b10106224bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6825","nonce":3534723355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6827","nonce":3534723355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6831","nonce":3534723355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6829","nonce":3534723355}]},"public_addr":"172.21.15.27:6825/3534723355","cluster_addr":"172.21.15.27:6827/3534723355","heartbeat_back_addr":"172.21.15.27:6831/3534723355","heartbeat_front_addr":"172.21.15.27:6829/3534723355","state":["exists","up"]},{"osd":3,"uuid":"d74d3f49-9c24-4b9f-98e2-3ecd7b5ef3b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6801","nonce":80661598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6803","nonce":80661598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6807","nonce":80661598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6805","nonce":80661598}]},"public_addr":"172.21.15.27:6801/80661598","cluster_addr":"172.21.15.27:6803/80661598","heartbeat_back_addr":"172.21.15.27:6807/80661598","heartbeat_front_addr":"172.21.15.27:6805/80661598","state":["exists","up"]},{"osd":4,"uuid":"2bf123f2-1b0d-41c4-aa3b-805bab267e59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6817","nonce":3943189827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6819","nonce":3943189827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6823","nonce":3943189827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6821","nonce":3943189827}]},"public_addr":"172.21.15.45:6817/3943189827","cluster_addr":"172.21.15.45:6819/3943189827","heartbeat_back_addr":"172.21.15.45:6823/3943189827","heartbeat_front_addr":"172.21.15.45:6821/3943189827","state":["exists","up"]},{"osd":5,"uuid":"c49ba5da-91c6-4a86-a97b-4aba89c253ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6809","nonce":3620760330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6811","nonce":3620760330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6815","nonce":3620760330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6813","nonce":3620760330}]},"public_addr":"172.21.15.45:6809/3620760330","cluster_addr":"172.21.15.45:6811/3620760330","heartbeat_back_addr":"172.21.15.45:6815/3620760330","heartbeat_front_addr":"172.21.15.45:6813/3620760330","state":["exists","up"]},{"osd":6,"uuid":"b0123475-89f7-4a34-9e33-d797b99dd9dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6801","nonce":1989083510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6803","nonce":1989083510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6807","nonce":1989083510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6805","nonce":1989083510}]},"public_addr":"172.21.15.45:6801/1989083510","cluster_addr":"172.21.15.45:6803/1989083510","heartbeat_back_addr":"172.21.15.45:6807/1989083510","heartbeat_front_addr":"172.21.15.45:6805/1989083510","state":["exists","up"]},{"osd":7,"uuid":"4dad773d-b7d8-46fc-8a10-a972aad27b29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6825","nonce":1513300294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6827","nonce":1513300294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6831","nonce":1513300294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6829","nonce":1513300294}]},"public_addr":"172.21.15.45:6825/1513300294","cluster_addr":"172.21.15.45:6827/1513300294","heartbeat_back_addr":"172.21.15.45:6831/1513300294","heartbeat_front_addr":"172.21.15.45:6829/1513300294","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.881855+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:17.133252+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.735339+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.500742+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.477571+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:17.167667+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.470291+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.873743+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T03:21:02.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.461+0000 7f72f23f6640 1 -- 172.21.15.27:0/2724339667 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f72d806bf50 msgr2=0x7f72d806e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:02.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.461+0000 7f72f23f6640 1 --2- 172.21.15.27:0/2724339667 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f72d806bf50 0x7f72d806e410 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f72e0034000 tx=0x7f72e002e000 comp rx=0 tx=0).stop 2024-09-12T03:21:02.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.461+0000 7f72f23f6640 1 -- 172.21.15.27:0/2724339667 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f72ec072df0 msgr2=0x7f72ec15cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:02.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.461+0000 7f72f23f6640 1 --2- 172.21.15.27:0/2724339667 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f72ec072df0 0x7f72ec15cc60 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f72dc00c400 tx=0x7f72dc00c8c0 comp rx=0 tx=0).stop 2024-09-12T03:21:02.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.461+0000 7f72f23f6640 1 -- 172.21.15.27:0/2724339667 shutdown_connections 2024-09-12T03:21:02.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.461+0000 7f72f23f6640 1 --2- 172.21.15.27:0/2724339667 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f72ec0b5a70 0x7f72ec15b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:02.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.461+0000 7f72f23f6640 1 --2- 172.21.15.27:0/2724339667 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f72ec0ae8d0 0x7f72ec15af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:02.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.461+0000 7f72f23f6640 1 --2- 172.21.15.27:0/2724339667 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f72d806bf50 0x7f72d806e410 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:02.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.461+0000 7f72f23f6640 1 --2- 172.21.15.27:0/2724339667 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f72ec072df0 0x7f72ec15cc60 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:02.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.461+0000 7f72f23f6640 1 -- 172.21.15.27:0/2724339667 >> 172.21.15.27:0/2724339667 conn(0x7f72ec06ca30 msgr2=0x7f72ec0718a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:02.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.461+0000 7f72f23f6640 1 -- 172.21.15.27:0/2724339667 shutdown_connections 2024-09-12T03:21:02.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.461+0000 7f72f23f6640 1 -- 172.21.15.27:0/2724339667 wait complete. 2024-09-12T03:21:02.472 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-12T03:21:02.472 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-12T03:21:02.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.575+0000 7efcce6eb640 1 Processor -- start 2024-09-12T03:21:02.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.575+0000 7efcce6eb640 1 -- start start 2024-09-12T03:21:02.585 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.576+0000 7efcce6eb640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7efcc815e780 0x7efcc80d3e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:02.585 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.576+0000 7efcce6eb640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7efcc80d1990 0x7efcc80b3390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:02.585 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.576+0000 7efcce6eb640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efcc80b6690 con 0x7efcc80d43b0 2024-09-12T03:21:02.586 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.576+0000 7efcce6eb640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7efcc80b7950 con 0x7efcc815e780 2024-09-12T03:21:02.586 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.576+0000 7efcce6eb640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7efcc80b7210 con 0x7efcc80d1990 2024-09-12T03:21:02.586 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.576+0000 7efcc7fff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7efcc815e780 0x7efcc80d3e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:02.586 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.576+0000 7efcc7fff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7efcc815e780 0x7efcc80d3e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38530/0 (socket says 172.21.15.27:38530) 2024-09-12T03:21:02.586 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.576+0000 7efcc7fff640 1 -- 172.21.15.27:0/2439043754 learned_addr learned my addr 172.21.15.27:0/2439043754 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:21:02.586 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.576+0000 7efcccc88640 1 --2- 172.21.15.27:0/2439043754 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7efcc80d1990 0x7efcc80b3390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:02.587 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.576+0000 7efcc7fff640 1 -- 172.21.15.27:0/2439043754 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7efcc80d1990 msgr2=0x7efcc80b3390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:02.587 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.576+0000 7efcc7fff640 1 --2- 172.21.15.27:0/2439043754 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7efcc80d1990 0x7efcc80b3390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:02.587 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.576+0000 7efcc7fff640 1 -- 172.21.15.27:0/2439043754 >> v1:172.21.15.45:6789/0 conn(0x7efcc80d43b0 legacy=0x7efcc80d1280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:02.587 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.576+0000 7efcc7fff640 1 -- 172.21.15.27:0/2439043754 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcc80b6970 con 0x7efcc815e780 2024-09-12T03:21:02.587 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.576+0000 7efcccc88640 1 --2- 172.21.15.27:0/2439043754 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7efcc80d1990 0x7efcc80b3390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:21:02.587 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.577+0000 7efcc7fff640 1 --2- 172.21.15.27:0/2439043754 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efcc815e780 0x7efcc80d3e70 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7efcb8009870 tx=0x7efcb802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bea2270014efb50 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:02.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.577+0000 7efcc6ffd640 1 -- 172.21.15.27:0/2439043754 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcb8037070 con 0x7efcc815e780 2024-09-12T03:21:02.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.577+0000 7efcc6ffd640 1 -- 172.21.15.27:0/2439043754 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efcb8032440 con 0x7efcc815e780 2024-09-12T03:21:02.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.577+0000 7efcc6ffd640 1 -- 172.21.15.27:0/2439043754 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcb8032810 con 0x7efcc815e780 2024-09-12T03:21:02.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.578+0000 7efcce6eb640 1 -- 172.21.15.27:0/2439043754 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efcc815e780 msgr2=0x7efcc80d3e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:02.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.578+0000 7efcce6eb640 1 --2- 172.21.15.27:0/2439043754 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efcc815e780 0x7efcc80d3e70 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7efcb8009870 tx=0x7efcb802a210 comp rx=0 tx=0).stop 2024-09-12T03:21:02.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.578+0000 7efcce6eb640 1 -- 172.21.15.27:0/2439043754 shutdown_connections 2024-09-12T03:21:02.589 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.578+0000 7efcce6eb640 1 --2- 172.21.15.27:0/2439043754 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7efcc80d1990 0x7efcc80b3390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:02.589 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.578+0000 7efcce6eb640 1 --2- 172.21.15.27:0/2439043754 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efcc815e780 0x7efcc80d3e70 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:02.589 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.578+0000 7efcce6eb640 1 -- 172.21.15.27:0/2439043754 >> 172.21.15.27:0/2439043754 conn(0x7efcc806da60 msgr2=0x7efcc806de70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:02.589 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.578+0000 7efcce6eb640 1 -- 172.21.15.27:0/2439043754 shutdown_connections 2024-09-12T03:21:02.589 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.578+0000 7efcce6eb640 1 -- 172.21.15.27:0/2439043754 wait complete. 2024-09-12T03:21:02.590 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.579+0000 7efcce6eb640 1 Processor -- start 2024-09-12T03:21:02.590 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.579+0000 7efcce6eb640 1 -- start start 2024-09-12T03:21:02.590 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.580+0000 7efcce6eb640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efcc80d1990 0x7efcc80cbe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:02.590 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.580+0000 7efcce6eb640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7efcc80d43b0 0x7efcc80cc5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:02.590 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.580+0000 7efcce6eb640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efcc815e780 0x7efcc814bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:02.590 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.580+0000 7efcce6eb640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7efcc80b6340 con 0x7efcc815e780 2024-09-12T03:21:02.591 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.580+0000 7efcce6eb640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7efcc80b6f30 con 0x7efcc80d1990 2024-09-12T03:21:02.591 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.580+0000 7efcce6eb640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7efcc815eb60 con 0x7efcc80d43b0 2024-09-12T03:21:02.591 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.580+0000 7efcccc88640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efcc815e780 0x7efcc814bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:02.591 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.580+0000 7efcccc88640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efcc815e780 0x7efcc814bcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38542/0 (socket says 172.21.15.27:38542) 2024-09-12T03:21:02.591 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.580+0000 7efcccc88640 1 -- 172.21.15.27:0/2314434878 learned_addr learned my addr 172.21.15.27:0/2314434878 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:21:02.591 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.580+0000 7efcc7fff640 1 --2- 172.21.15.27:0/2314434878 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efcc80d1990 0x7efcc80cbe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:02.592 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.580+0000 7efcc77fe640 1 --2- 172.21.15.27:0/2314434878 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7efcc80d43b0 0x7efcc80cc5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:02.592 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.580+0000 7efcccc88640 1 -- 172.21.15.27:0/2314434878 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7efcc80d43b0 msgr2=0x7efcc80cc5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:02.592 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.581+0000 7efcccc88640 1 --2- 172.21.15.27:0/2314434878 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7efcc80d43b0 0x7efcc80cc5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:02.592 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.581+0000 7efcccc88640 1 -- 172.21.15.27:0/2314434878 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efcc80d1990 msgr2=0x7efcc80cbe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:02.592 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.581+0000 7efcccc88640 1 --2- 172.21.15.27:0/2314434878 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efcc80d1990 0x7efcc80cbe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:02.592 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.581+0000 7efcccc88640 1 -- 172.21.15.27:0/2314434878 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcc80cce00 con 0x7efcc815e780 2024-09-12T03:21:02.593 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.581+0000 7efcccc88640 1 --2- 172.21.15.27:0/2314434878 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efcc815e780 0x7efcc814bcd0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7efcc000ce70 tx=0x7efcc000a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:02.593 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.581+0000 7efcc4ff9640 1 -- 172.21.15.27:0/2314434878 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcc0003b90 con 0x7efcc815e780 2024-09-12T03:21:02.593 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.581+0000 7efcc4ff9640 1 -- 172.21.15.27:0/2314434878 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efcc0003d50 con 0x7efcc815e780 2024-09-12T03:21:02.593 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.581+0000 7efcce6eb640 1 -- 172.21.15.27:0/2314434878 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efcc815af80 con 0x7efcc815e780 2024-09-12T03:21:02.593 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.581+0000 7efcc4ff9640 1 -- 172.21.15.27:0/2314434878 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcc0004260 con 0x7efcc815e780 2024-09-12T03:21:02.594 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.581+0000 7efcce6eb640 1 -- 172.21.15.27:0/2314434878 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efcc80ccaf0 con 0x7efcc815e780 2024-09-12T03:21:02.594 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.582+0000 7efcce6eb640 1 -- 172.21.15.27:0/2314434878 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efcc80d3e70 con 0x7efcc815e780 2024-09-12T03:21:02.594 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.582+0000 7efcc4ff9640 1 -- 172.21.15.27:0/2314434878 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 90600+0+0 (secure 0 0 0) 0x7efcc0004400 con 0x7efcc815e780 2024-09-12T03:21:02.594 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.583+0000 7efcc4ff9640 1 --2- 172.21.15.27:0/2314434878 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7efcb406bf50 0x7efcb406e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:02.594 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.583+0000 7efcc4ff9640 1 -- 172.21.15.27:0/2314434878 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5709+0+0 (secure 0 0 0) 0x7efcc008b890 con 0x7efcc815e780 2024-09-12T03:21:02.594 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.583+0000 7efcc7fff640 1 --2- 172.21.15.27:0/2314434878 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7efcb406bf50 0x7efcb406e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:02.595 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.583+0000 7efcc7fff640 1 --2- 172.21.15.27:0/2314434878 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7efcb406bf50 0x7efcb406e410 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7efcb8034000 tx=0x7efcb802e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:02.595 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.587+0000 7efcc4ff9640 1 -- 172.21.15.27:0/2314434878 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7efcc0059af0 con 0x7efcc815e780 2024-09-12T03:21:02.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.759+0000 7efcce6eb640 1 -- 172.21.15.27:0/2314434878 --> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7efcc80d1320 con 0x7efcb406bf50 2024-09-12T03:21:02.765 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.765+0000 7efcc4ff9640 1 -- 172.21.15.27:0/2314434878 <== mgr.4113 v2:172.21.15.27:6836/1226920526 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252186 (secure 0 0 0) 0x7efcc80d1320 con 0x7efcb406bf50 2024-09-12T03:21:02.769 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:21:02.771 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2024-09-12T03:21:02.774 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.768+0000 7efcce6eb640 1 -- 172.21.15.27:0/2314434878 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7efcb406bf50 msgr2=0x7efcb406e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:02.774 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.768+0000 7efcce6eb640 1 --2- 172.21.15.27:0/2314434878 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7efcb406bf50 0x7efcb406e410 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7efcb8034000 tx=0x7efcb802e000 comp rx=0 tx=0).stop 2024-09-12T03:21:02.774 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.768+0000 7efcce6eb640 1 -- 172.21.15.27:0/2314434878 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efcc815e780 msgr2=0x7efcc814bcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:02.774 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.768+0000 7efcce6eb640 1 --2- 172.21.15.27:0/2314434878 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efcc815e780 0x7efcc814bcd0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7efcc000ce70 tx=0x7efcc000a710 comp rx=0 tx=0).stop 2024-09-12T03:21:02.774 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.768+0000 7efcce6eb640 1 -- 172.21.15.27:0/2314434878 shutdown_connections 2024-09-12T03:21:02.774 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.768+0000 7efcce6eb640 1 --2- 172.21.15.27:0/2314434878 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7efcb406bf50 0x7efcb406e410 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:02.774 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.768+0000 7efcce6eb640 1 --2- 172.21.15.27:0/2314434878 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efcc815e780 0x7efcc814bcd0 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:02.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.768+0000 7efcce6eb640 1 --2- 172.21.15.27:0/2314434878 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7efcc80d43b0 0x7efcc80cc5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:02.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.768+0000 7efcce6eb640 1 --2- 172.21.15.27:0/2314434878 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efcc80d1990 0x7efcc80cbe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:02.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.768+0000 7efcce6eb640 1 -- 172.21.15.27:0/2314434878 >> 172.21.15.27:0/2314434878 conn(0x7efcc806da60 msgr2=0x7efcc80d4870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:02.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.768+0000 7efcce6eb640 1 -- 172.21.15.27:0/2314434878 shutdown_connections 2024-09-12T03:21:02.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.768+0000 7efcce6eb640 1 -- 172.21.15.27:0/2314434878 wait complete. 2024-09-12T03:21:02.779 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":57,"stamp":"2024-09-12T03:21:01.104013+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":224,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":167,"ondisk_log_size":167,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":115,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":268708,"kb_used_data":10860,"kb_used_omap":12,"kb_used_meta":257651,"kb_avail":754706012,"statfs":{"total":773094113280,"available":772818956288,"internally_reserved":0,"allocated":11120640,"data_stored":5516016,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":263835184},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3],"upper_bound":8},"perf_stat":{"commit_latency_ms":20,"apply_latency_ms":20,"commit_latency_ns":20000000,"apply_latency_ns":20000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000762"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-12T03:21:01.015811+0000","last_change":"2024-09-12T03:20:59.812495+0000","last_active":"2024-09-12T03:21:01.015811+0000","last_peered":"2024-09-12T03:21:01.015811+0000","last_clean":"2024-09-12T03:21:01.015811+0000","last_became_active":"2024-09-12T03:20:59.812227+0000","last_became_peered":"2024-09-12T03:20:59.812227+0000","last_unstale":"2024-09-12T03:21:01.015811+0000","last_undegraded":"2024-09-12T03:21:01.015811+0000","last_fullsized":"2024-09-12T03:21:01.015811+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_clean_scrub_stamp":"2024-09-12T03:20:58.769862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:40:38.653233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-12T03:21:00.801750+0000","last_change":"2024-09-12T03:20:59.804237+0000","last_active":"2024-09-12T03:21:00.801750+0000","last_peered":"2024-09-12T03:21:00.801750+0000","last_clean":"2024-09-12T03:21:00.801750+0000","last_became_active":"2024-09-12T03:20:59.803405+0000","last_became_peered":"2024-09-12T03:20:59.803405+0000","last_unstale":"2024-09-12T03:21:00.801750+0000","last_undegraded":"2024-09-12T03:21:00.801750+0000","last_fullsized":"2024-09-12T03:21:00.801750+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_clean_scrub_stamp":"2024-09-12T03:20:58.769862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:32:54.597742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-12T03:21:01.016035+0000","last_change":"2024-09-12T03:20:59.810587+0000","last_active":"2024-09-12T03:21:01.016035+0000","last_peered":"2024-09-12T03:21:01.016035+0000","last_clean":"2024-09-12T03:21:01.016035+0000","last_became_active":"2024-09-12T03:20:59.810237+0000","last_became_peered":"2024-09-12T03:20:59.810237+0000","last_unstale":"2024-09-12T03:21:01.016035+0000","last_undegraded":"2024-09-12T03:21:01.016035+0000","last_fullsized":"2024-09-12T03:21:01.016035+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_clean_scrub_stamp":"2024-09-12T03:20:58.769862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:27:31.328878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-12T03:21:01.016212+0000","last_change":"2024-09-12T03:20:59.820197+0000","last_active":"2024-09-12T03:21:01.016212+0000","last_peered":"2024-09-12T03:21:01.016212+0000","last_clean":"2024-09-12T03:21:01.016212+0000","last_became_active":"2024-09-12T03:20:59.819922+0000","last_became_peered":"2024-09-12T03:20:59.819922+0000","last_unstale":"2024-09-12T03:21:01.016212+0000","last_undegraded":"2024-09-12T03:21:01.016212+0000","last_fullsized":"2024-09-12T03:21:01.016212+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_clean_scrub_stamp":"2024-09-12T03:20:58.769862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:41:57.265774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-12T03:21:01.014231+0000","last_change":"2024-09-12T03:20:59.812365+0000","last_active":"2024-09-12T03:21:01.014231+0000","last_peered":"2024-09-12T03:21:01.014231+0000","last_clean":"2024-09-12T03:21:01.014231+0000","last_became_active":"2024-09-12T03:20:59.812073+0000","last_became_peered":"2024-09-12T03:20:59.812073+0000","last_unstale":"2024-09-12T03:21:01.014231+0000","last_undegraded":"2024-09-12T03:21:01.014231+0000","last_fullsized":"2024-09-12T03:21:01.014231+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_clean_scrub_stamp":"2024-09-12T03:20:58.769862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:51:14.057192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-12T03:21:01.016019+0000","last_change":"2024-09-12T03:20:59.812997+0000","last_active":"2024-09-12T03:21:01.016019+0000","last_peered":"2024-09-12T03:21:01.016019+0000","last_clean":"2024-09-12T03:21:01.016019+0000","last_became_active":"2024-09-12T03:20:59.812693+0000","last_became_peered":"2024-09-12T03:20:59.812693+0000","last_unstale":"2024-09-12T03:21:01.016019+0000","last_undegraded":"2024-09-12T03:21:01.016019+0000","last_fullsized":"2024-09-12T03:21:01.016019+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_clean_scrub_stamp":"2024-09-12T03:20:58.769862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:47:32.534307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-12T03:21:00.800567+0000","last_change":"2024-09-12T03:20:59.802840+0000","last_active":"2024-09-12T03:21:00.800567+0000","last_peered":"2024-09-12T03:21:00.800567+0000","last_clean":"2024-09-12T03:21:00.800567+0000","last_became_active":"2024-09-12T03:20:59.802530+0000","last_became_peered":"2024-09-12T03:20:59.802530+0000","last_unstale":"2024-09-12T03:21:00.800567+0000","last_undegraded":"2024-09-12T03:21:00.800567+0000","last_fullsized":"2024-09-12T03:21:00.800567+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_clean_scrub_stamp":"2024-09-12T03:20:58.769862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:08:29.553346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-12T03:21:01.015363+0000","last_change":"2024-09-12T03:20:59.805399+0000","last_active":"2024-09-12T03:21:01.015363+0000","last_peered":"2024-09-12T03:21:01.015363+0000","last_clean":"2024-09-12T03:21:01.015363+0000","last_became_active":"2024-09-12T03:20:59.804834+0000","last_became_peered":"2024-09-12T03:20:59.804834+0000","last_unstale":"2024-09-12T03:21:01.015363+0000","last_undegraded":"2024-09-12T03:21:01.015363+0000","last_fullsized":"2024-09-12T03:21:01.015363+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_clean_scrub_stamp":"2024-09-12T03:20:58.769862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:18:54.207971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-12T03:21:01.015949+0000","last_change":"2024-09-12T03:20:59.824528+0000","last_active":"2024-09-12T03:21:01.015949+0000","last_peered":"2024-09-12T03:21:01.015949+0000","last_clean":"2024-09-12T03:21:01.015949+0000","last_became_active":"2024-09-12T03:20:59.824204+0000","last_became_peered":"2024-09-12T03:20:59.824204+0000","last_unstale":"2024-09-12T03:21:01.015949+0000","last_undegraded":"2024-09-12T03:21:01.015949+0000","last_fullsized":"2024-09-12T03:21:01.015949+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_clean_scrub_stamp":"2024-09-12T03:20:58.769862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:19:55.126748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-12T03:21:01.015355+0000","last_change":"2024-09-12T03:20:59.815730+0000","last_active":"2024-09-12T03:21:01.015355+0000","last_peered":"2024-09-12T03:21:01.015355+0000","last_clean":"2024-09-12T03:21:01.015355+0000","last_became_active":"2024-09-12T03:20:59.815367+0000","last_became_peered":"2024-09-12T03:20:59.815367+0000","last_unstale":"2024-09-12T03:21:01.015355+0000","last_undegraded":"2024-09-12T03:21:01.015355+0000","last_fullsized":"2024-09-12T03:21:01.015355+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_clean_scrub_stamp":"2024-09-12T03:20:58.769862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:32:00.897353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-12T03:21:00.801476+0000","last_change":"2024-09-12T03:20:59.799545+0000","last_active":"2024-09-12T03:21:00.801476+0000","last_peered":"2024-09-12T03:21:00.801476+0000","last_clean":"2024-09-12T03:21:00.801476+0000","last_became_active":"2024-09-12T03:20:59.799234+0000","last_became_peered":"2024-09-12T03:20:59.799234+0000","last_unstale":"2024-09-12T03:21:00.801476+0000","last_undegraded":"2024-09-12T03:21:00.801476+0000","last_fullsized":"2024-09-12T03:21:00.801476+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_clean_scrub_stamp":"2024-09-12T03:20:58.769862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:56:50.124923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-12T03:21:01.016330+0000","last_change":"2024-09-12T03:20:59.816949+0000","last_active":"2024-09-12T03:21:01.016330+0000","last_peered":"2024-09-12T03:21:01.016330+0000","last_clean":"2024-09-12T03:21:01.016330+0000","last_became_active":"2024-09-12T03:20:59.816264+0000","last_became_peered":"2024-09-12T03:20:59.816264+0000","last_unstale":"2024-09-12T03:21:01.016330+0000","last_undegraded":"2024-09-12T03:21:01.016330+0000","last_fullsized":"2024-09-12T03:21:01.016330+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_clean_scrub_stamp":"2024-09-12T03:20:58.769862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:06:53.605499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-12T03:21:01.015881+0000","last_change":"2024-09-12T03:20:59.802684+0000","last_active":"2024-09-12T03:21:01.015881+0000","last_peered":"2024-09-12T03:21:01.015881+0000","last_clean":"2024-09-12T03:21:01.015881+0000","last_became_active":"2024-09-12T03:20:59.802364+0000","last_became_peered":"2024-09-12T03:20:59.802364+0000","last_unstale":"2024-09-12T03:21:01.015881+0000","last_undegraded":"2024-09-12T03:21:01.015881+0000","last_fullsized":"2024-09-12T03:21:01.015881+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_clean_scrub_stamp":"2024-09-12T03:20:58.769862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:49:14.153740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-12T03:21:01.015351+0000","last_change":"2024-09-12T03:20:59.803518+0000","last_active":"2024-09-12T03:21:01.015351+0000","last_peered":"2024-09-12T03:21:01.015351+0000","last_clean":"2024-09-12T03:21:01.015351+0000","last_became_active":"2024-09-12T03:20:59.802943+0000","last_became_peered":"2024-09-12T03:20:59.802943+0000","last_unstale":"2024-09-12T03:21:01.015351+0000","last_undegraded":"2024-09-12T03:21:01.015351+0000","last_fullsized":"2024-09-12T03:21:01.015351+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_clean_scrub_stamp":"2024-09-12T03:20:58.769862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:39:28.867757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-12T03:21:01.016020+0000","last_change":"2024-09-12T03:20:59.811528+0000","last_active":"2024-09-12T03:21:01.016020+0000","last_peered":"2024-09-12T03:21:01.016020+0000","last_clean":"2024-09-12T03:21:01.016020+0000","last_became_active":"2024-09-12T03:20:59.811243+0000","last_became_peered":"2024-09-12T03:20:59.811243+0000","last_unstale":"2024-09-12T03:21:01.016020+0000","last_undegraded":"2024-09-12T03:21:01.016020+0000","last_fullsized":"2024-09-12T03:21:01.016020+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_clean_scrub_stamp":"2024-09-12T03:20:58.769862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:59:11.158965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-12T03:21:01.014058+0000","last_change":"2024-09-12T03:20:59.820472+0000","last_active":"2024-09-12T03:21:01.014058+0000","last_peered":"2024-09-12T03:21:01.014058+0000","last_clean":"2024-09-12T03:21:01.014058+0000","last_became_active":"2024-09-12T03:20:59.819864+0000","last_became_peered":"2024-09-12T03:20:59.819864+0000","last_unstale":"2024-09-12T03:21:01.014058+0000","last_undegraded":"2024-09-12T03:21:01.014058+0000","last_fullsized":"2024-09-12T03:21:01.014058+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_clean_scrub_stamp":"2024-09-12T03:20:58.769862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:15:57.664758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-12T03:21:00.801781+0000","last_change":"2024-09-12T03:20:59.806723+0000","last_active":"2024-09-12T03:21:00.801781+0000","last_peered":"2024-09-12T03:21:00.801781+0000","last_clean":"2024-09-12T03:21:00.801781+0000","last_became_active":"2024-09-12T03:20:59.806418+0000","last_became_peered":"2024-09-12T03:20:59.806418+0000","last_unstale":"2024-09-12T03:21:00.801781+0000","last_undegraded":"2024-09-12T03:21:00.801781+0000","last_fullsized":"2024-09-12T03:21:00.801781+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_clean_scrub_stamp":"2024-09-12T03:20:58.769862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:49:14.153740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-12T03:21:00.807037+0000","last_change":"2024-09-12T03:20:59.804307+0000","last_active":"2024-09-12T03:21:00.807037+0000","last_peered":"2024-09-12T03:21:00.807037+0000","last_clean":"2024-09-12T03:21:00.807037+0000","last_became_active":"2024-09-12T03:20:59.803978+0000","last_became_peered":"2024-09-12T03:20:59.803978+0000","last_unstale":"2024-09-12T03:21:00.807037+0000","last_undegraded":"2024-09-12T03:21:00.807037+0000","last_fullsized":"2024-09-12T03:21:00.807037+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_clean_scrub_stamp":"2024-09-12T03:20:58.769862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:39:27.157965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-12T03:21:01.016171+0000","last_change":"2024-09-12T03:20:59.805203+0000","last_active":"2024-09-12T03:21:01.016171+0000","last_peered":"2024-09-12T03:21:01.016171+0000","last_clean":"2024-09-12T03:21:01.016171+0000","last_became_active":"2024-09-12T03:20:59.804424+0000","last_became_peered":"2024-09-12T03:20:59.804424+0000","last_unstale":"2024-09-12T03:21:01.016171+0000","last_undegraded":"2024-09-12T03:21:01.016171+0000","last_fullsized":"2024-09-12T03:21:01.016171+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_clean_scrub_stamp":"2024-09-12T03:20:58.769862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:29:56.581778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-12T03:21:00.801274+0000","last_change":"2024-09-12T03:20:59.809588+0000","last_active":"2024-09-12T03:21:00.801274+0000","last_peered":"2024-09-12T03:21:00.801274+0000","last_clean":"2024-09-12T03:21:00.801274+0000","last_became_active":"2024-09-12T03:20:59.808976+0000","last_became_peered":"2024-09-12T03:20:59.808976+0000","last_unstale":"2024-09-12T03:21:00.801274+0000","last_undegraded":"2024-09-12T03:21:00.801274+0000","last_fullsized":"2024-09-12T03:21:00.801274+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_clean_scrub_stamp":"2024-09-12T03:20:58.769862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:39:28.867757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-12T03:21:00.801335+0000","last_change":"2024-09-12T03:20:59.806035+0000","last_active":"2024-09-12T03:21:00.801335+0000","last_peered":"2024-09-12T03:21:00.801335+0000","last_clean":"2024-09-12T03:21:00.801335+0000","last_became_active":"2024-09-12T03:20:59.805734+0000","last_became_peered":"2024-09-12T03:20:59.805734+0000","last_unstale":"2024-09-12T03:21:00.801335+0000","last_undegraded":"2024-09-12T03:21:00.801335+0000","last_fullsized":"2024-09-12T03:21:00.801335+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_clean_scrub_stamp":"2024-09-12T03:20:58.769862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:18:54.207971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-12T03:21:00.799520+0000","last_change":"2024-09-12T03:20:59.804655+0000","last_active":"2024-09-12T03:21:00.799520+0000","last_peered":"2024-09-12T03:21:00.799520+0000","last_clean":"2024-09-12T03:21:00.799520+0000","last_became_active":"2024-09-12T03:20:59.804358+0000","last_became_peered":"2024-09-12T03:20:59.804358+0000","last_unstale":"2024-09-12T03:21:00.799520+0000","last_undegraded":"2024-09-12T03:21:00.799520+0000","last_fullsized":"2024-09-12T03:21:00.799520+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_clean_scrub_stamp":"2024-09-12T03:20:58.769862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:14:46.054149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-12T03:21:00.800507+0000","last_change":"2024-09-12T03:20:59.808182+0000","last_active":"2024-09-12T03:21:00.800507+0000","last_peered":"2024-09-12T03:21:00.800507+0000","last_clean":"2024-09-12T03:21:00.800507+0000","last_became_active":"2024-09-12T03:20:59.807909+0000","last_became_peered":"2024-09-12T03:20:59.807909+0000","last_unstale":"2024-09-12T03:21:00.800507+0000","last_undegraded":"2024-09-12T03:21:00.800507+0000","last_fullsized":"2024-09-12T03:21:00.800507+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_clean_scrub_stamp":"2024-09-12T03:20:58.769862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:39:27.157965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-12T03:21:00.799250+0000","last_change":"2024-09-12T03:20:59.809254+0000","last_active":"2024-09-12T03:21:00.799250+0000","last_peered":"2024-09-12T03:21:00.799250+0000","last_clean":"2024-09-12T03:21:00.799250+0000","last_became_active":"2024-09-12T03:20:59.808969+0000","last_became_peered":"2024-09-12T03:20:59.808969+0000","last_unstale":"2024-09-12T03:21:00.799250+0000","last_undegraded":"2024-09-12T03:21:00.799250+0000","last_fullsized":"2024-09-12T03:21:00.799250+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_clean_scrub_stamp":"2024-09-12T03:20:58.769862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:39:28.867757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-12T03:21:00.801841+0000","last_change":"2024-09-12T03:20:59.806550+0000","last_active":"2024-09-12T03:21:00.801841+0000","last_peered":"2024-09-12T03:21:00.801841+0000","last_clean":"2024-09-12T03:21:00.801841+0000","last_became_active":"2024-09-12T03:20:59.806245+0000","last_became_peered":"2024-09-12T03:20:59.806245+0000","last_unstale":"2024-09-12T03:21:00.801841+0000","last_undegraded":"2024-09-12T03:21:00.801841+0000","last_fullsized":"2024-09-12T03:21:00.801841+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_clean_scrub_stamp":"2024-09-12T03:20:58.769862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:14:46.054149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-12T03:21:01.015501+0000","last_change":"2024-09-12T03:20:59.815020+0000","last_active":"2024-09-12T03:21:01.015501+0000","last_peered":"2024-09-12T03:21:01.015501+0000","last_clean":"2024-09-12T03:21:01.015501+0000","last_became_active":"2024-09-12T03:20:59.814707+0000","last_became_peered":"2024-09-12T03:20:59.814707+0000","last_unstale":"2024-09-12T03:21:01.015501+0000","last_undegraded":"2024-09-12T03:21:01.015501+0000","last_fullsized":"2024-09-12T03:21:01.015501+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_clean_scrub_stamp":"2024-09-12T03:20:58.769862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:53:47.188882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-12T03:21:01.015746+0000","last_change":"2024-09-12T03:20:59.821070+0000","last_active":"2024-09-12T03:21:01.015746+0000","last_peered":"2024-09-12T03:21:01.015746+0000","last_clean":"2024-09-12T03:21:01.015746+0000","last_became_active":"2024-09-12T03:20:59.820747+0000","last_became_peered":"2024-09-12T03:20:59.820747+0000","last_unstale":"2024-09-12T03:21:01.015746+0000","last_undegraded":"2024-09-12T03:21:01.015746+0000","last_fullsized":"2024-09-12T03:21:01.015746+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_clean_scrub_stamp":"2024-09-12T03:20:58.769862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:28:03.723307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-12T03:21:00.799493+0000","last_change":"2024-09-12T03:20:59.818013+0000","last_active":"2024-09-12T03:21:00.799493+0000","last_peered":"2024-09-12T03:21:00.799493+0000","last_clean":"2024-09-12T03:21:00.799493+0000","last_became_active":"2024-09-12T03:20:59.817704+0000","last_became_peered":"2024-09-12T03:20:59.817704+0000","last_unstale":"2024-09-12T03:21:00.799493+0000","last_undegraded":"2024-09-12T03:21:00.799493+0000","last_fullsized":"2024-09-12T03:21:00.799493+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_clean_scrub_stamp":"2024-09-12T03:20:58.769862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:49:14.153740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-12T03:21:00.800544+0000","last_change":"2024-09-12T03:20:59.809481+0000","last_active":"2024-09-12T03:21:00.800544+0000","last_peered":"2024-09-12T03:21:00.800544+0000","last_clean":"2024-09-12T03:21:00.800544+0000","last_became_active":"2024-09-12T03:20:59.809200+0000","last_became_peered":"2024-09-12T03:20:59.809200+0000","last_unstale":"2024-09-12T03:21:00.800544+0000","last_undegraded":"2024-09-12T03:21:00.800544+0000","last_fullsized":"2024-09-12T03:21:00.800544+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_clean_scrub_stamp":"2024-09-12T03:20:58.769862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:36:44.063303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-12T03:21:00.800761+0000","last_change":"2024-09-12T03:20:59.798156+0000","last_active":"2024-09-12T03:21:00.800761+0000","last_peered":"2024-09-12T03:21:00.800761+0000","last_clean":"2024-09-12T03:21:00.800761+0000","last_became_active":"2024-09-12T03:20:59.797821+0000","last_became_peered":"2024-09-12T03:20:59.797821+0000","last_unstale":"2024-09-12T03:21:00.800761+0000","last_undegraded":"2024-09-12T03:21:00.800761+0000","last_fullsized":"2024-09-12T03:21:00.800761+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_clean_scrub_stamp":"2024-09-12T03:20:58.769862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:35:07.183939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-12T03:21:01.014769+0000","last_change":"2024-09-12T03:20:59.814559+0000","last_active":"2024-09-12T03:21:01.014769+0000","last_peered":"2024-09-12T03:21:01.014769+0000","last_clean":"2024-09-12T03:21:01.014769+0000","last_became_active":"2024-09-12T03:20:59.813964+0000","last_became_peered":"2024-09-12T03:20:59.813964+0000","last_unstale":"2024-09-12T03:21:01.014769+0000","last_undegraded":"2024-09-12T03:21:01.014769+0000","last_fullsized":"2024-09-12T03:21:01.014769+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_clean_scrub_stamp":"2024-09-12T03:20:58.769862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:42:06.157939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-12T03:21:01.015130+0000","last_change":"2024-09-12T03:20:59.812258+0000","last_active":"2024-09-12T03:21:01.015130+0000","last_peered":"2024-09-12T03:21:01.015130+0000","last_clean":"2024-09-12T03:21:01.015130+0000","last_became_active":"2024-09-12T03:20:59.811984+0000","last_became_peered":"2024-09-12T03:20:59.811984+0000","last_unstale":"2024-09-12T03:21:01.015130+0000","last_undegraded":"2024-09-12T03:21:01.015130+0000","last_fullsized":"2024-09-12T03:21:01.015130+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_clean_scrub_stamp":"2024-09-12T03:20:58.769862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:54:36.464412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-12T03:21:01.015511+0000","last_change":"2024-09-12T03:20:59.801304+0000","last_active":"2024-09-12T03:21:01.015511+0000","last_peered":"2024-09-12T03:21:01.015511+0000","last_clean":"2024-09-12T03:21:01.015511+0000","last_became_active":"2024-09-12T03:20:59.800985+0000","last_became_peered":"2024-09-12T03:20:59.800985+0000","last_unstale":"2024-09-12T03:21:01.015511+0000","last_undegraded":"2024-09-12T03:21:01.015511+0000","last_fullsized":"2024-09-12T03:21:01.015511+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_clean_scrub_stamp":"2024-09-12T03:20:58.769862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:42:58.503627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"99'4","reported_seq":18,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-12T03:21:00.801463+0000","last_change":"2024-09-12T03:20:58.784264+0000","last_active":"2024-09-12T03:21:00.801463+0000","last_peered":"2024-09-12T03:21:00.801463+0000","last_clean":"2024-09-12T03:21:00.801463+0000","last_became_active":"2024-09-12T03:20:58.783932+0000","last_became_peered":"2024-09-12T03:20:58.783932+0000","last_unstale":"2024-09-12T03:21:00.801463+0000","last_undegraded":"2024-09-12T03:21:00.801463+0000","last_fullsized":"2024-09-12T03:21:00.801463+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:57.764885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:57.764885+0000","last_clean_scrub_stamp":"2024-09-12T03:20:57.764885+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:47:31.529330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-12T03:21:00.807152+0000","last_change":"2024-09-12T03:20:59.805989+0000","last_active":"2024-09-12T03:21:00.807152+0000","last_peered":"2024-09-12T03:21:00.807152+0000","last_clean":"2024-09-12T03:21:00.807152+0000","last_became_active":"2024-09-12T03:20:59.804997+0000","last_became_peered":"2024-09-12T03:20:59.804997+0000","last_unstale":"2024-09-12T03:21:00.807152+0000","last_undegraded":"2024-09-12T03:21:00.807152+0000","last_fullsized":"2024-09-12T03:21:00.807152+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_clean_scrub_stamp":"2024-09-12T03:20:58.769862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:05:44.345724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1e","version":"99'1","reported_seq":15,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-12T03:21:01.015925+0000","last_change":"2024-09-12T03:20:58.791049+0000","last_active":"2024-09-12T03:21:01.015925+0000","last_peered":"2024-09-12T03:21:01.015925+0000","last_clean":"2024-09-12T03:21:01.015925+0000","last_became_active":"2024-09-12T03:20:58.790738+0000","last_became_peered":"2024-09-12T03:20:58.790738+0000","last_unstale":"2024-09-12T03:21:01.015925+0000","last_undegraded":"2024-09-12T03:21:01.015925+0000","last_fullsized":"2024-09-12T03:21:01.015925+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:57.764885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:57.764885+0000","last_clean_scrub_stamp":"2024-09-12T03:20:57.764885+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:50:28.197551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-12T03:21:01.015694+0000","last_change":"2024-09-12T03:20:59.814043+0000","last_active":"2024-09-12T03:21:01.015694+0000","last_peered":"2024-09-12T03:21:01.015694+0000","last_clean":"2024-09-12T03:21:01.015694+0000","last_became_active":"2024-09-12T03:20:59.813786+0000","last_became_peered":"2024-09-12T03:20:59.813786+0000","last_unstale":"2024-09-12T03:21:01.015694+0000","last_undegraded":"2024-09-12T03:21:01.015694+0000","last_fullsized":"2024-09-12T03:21:01.015694+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_clean_scrub_stamp":"2024-09-12T03:20:58.769862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:39:40.342252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1d","version":"99'2","reported_seq":16,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-12T03:21:01.014180+0000","last_change":"2024-09-12T03:20:58.795147+0000","last_active":"2024-09-12T03:21:01.014180+0000","last_peered":"2024-09-12T03:21:01.014180+0000","last_clean":"2024-09-12T03:21:01.014180+0000","last_became_active":"2024-09-12T03:20:58.794857+0000","last_became_peered":"2024-09-12T03:20:58.794857+0000","last_unstale":"2024-09-12T03:21:01.014180+0000","last_undegraded":"2024-09-12T03:21:01.014180+0000","last_fullsized":"2024-09-12T03:21:01.014180+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:57.764885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:57.764885+0000","last_clean_scrub_stamp":"2024-09-12T03:20:57.764885+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:36:49.111778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-12T03:21:01.016133+0000","last_change":"2024-09-12T03:20:59.810539+0000","last_active":"2024-09-12T03:21:01.016133+0000","last_peered":"2024-09-12T03:21:01.016133+0000","last_clean":"2024-09-12T03:21:01.016133+0000","last_became_active":"2024-09-12T03:20:59.810157+0000","last_became_peered":"2024-09-12T03:20:59.810157+0000","last_unstale":"2024-09-12T03:21:01.016133+0000","last_undegraded":"2024-09-12T03:21:01.016133+0000","last_fullsized":"2024-09-12T03:21:01.016133+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_clean_scrub_stamp":"2024-09-12T03:20:58.769862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:09:34.594263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1c","version":"99'2","reported_seq":16,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-12T03:21:01.014129+0000","last_change":"2024-09-12T03:20:58.795764+0000","last_active":"2024-09-12T03:21:01.014129+0000","last_peered":"2024-09-12T03:21:01.014129+0000","last_clean":"2024-09-12T03:21:01.014129+0000","last_became_active":"2024-09-12T03:20:58.795493+0000","last_became_peered":"2024-09-12T03:20:58.795493+0000","last_unstale":"2024-09-12T03:21:01.014129+0000","last_undegraded":"2024-09-12T03:21:01.014129+0000","last_fullsized":"2024-09-12T03:21:01.014129+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:57.764885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:57.764885+0000","last_clean_scrub_stamp":"2024-09-12T03:20:57.764885+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:24:50.315281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-12T03:21:01.014970+0000","last_change":"2024-09-12T03:20:59.822730+0000","last_active":"2024-09-12T03:21:01.014970+0000","last_peered":"2024-09-12T03:21:01.014970+0000","last_clean":"2024-09-12T03:21:01.014970+0000","last_became_active":"2024-09-12T03:20:59.822390+0000","last_became_peered":"2024-09-12T03:20:59.822390+0000","last_unstale":"2024-09-12T03:21:01.014970+0000","last_undegraded":"2024-09-12T03:21:01.014970+0000","last_fullsized":"2024-09-12T03:21:01.014970+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_clean_scrub_stamp":"2024-09-12T03:20:58.769862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:37:40.335331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1b","version":"99'1","reported_seq":15,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-12T03:21:00.800574+0000","last_change":"2024-09-12T03:20:58.785429+0000","last_active":"2024-09-12T03:21:00.800574+0000","last_peered":"2024-09-12T03:21:00.800574+0000","last_clean":"2024-09-12T03:21:00.800574+0000","last_became_active":"2024-09-12T03:20:58.785117+0000","last_became_peered":"2024-09-12T03:20:58.785117+0000","last_unstale":"2024-09-12T03:21:00.800574+0000","last_undegraded":"2024-09-12T03:21:00.800574+0000","last_fullsized":"2024-09-12T03:21:00.800574+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:57.764885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:57.764885+0000","last_clean_scrub_stamp":"2024-09-12T03:20:57.764885+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:42:57.498649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-12T03:21:00.800707+0000","last_change":"2024-09-12T03:20:59.802898+0000","last_active":"2024-09-12T03:21:00.800707+0000","last_peered":"2024-09-12T03:21:00.800707+0000","last_clean":"2024-09-12T03:21:00.800707+0000","last_became_active":"2024-09-12T03:20:59.802618+0000","last_became_peered":"2024-09-12T03:20:59.802618+0000","last_unstale":"2024-09-12T03:21:00.800707+0000","last_undegraded":"2024-09-12T03:21:00.800707+0000","last_fullsized":"2024-09-12T03:21:00.800707+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_clean_scrub_stamp":"2024-09-12T03:20:58.769862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:05:44.345724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1a","version":"99'1","reported_seq":15,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-12T03:21:00.799762+0000","last_change":"2024-09-12T03:20:58.789541+0000","last_active":"2024-09-12T03:21:00.799762+0000","last_peered":"2024-09-12T03:21:00.799762+0000","last_clean":"2024-09-12T03:21:00.799762+0000","last_became_active":"2024-09-12T03:20:58.789287+0000","last_became_peered":"2024-09-12T03:20:58.789287+0000","last_unstale":"2024-09-12T03:21:00.799762+0000","last_undegraded":"2024-09-12T03:21:00.799762+0000","last_fullsized":"2024-09-12T03:21:00.799762+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:57.764885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:57.764885+0000","last_clean_scrub_stamp":"2024-09-12T03:20:57.764885+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:48:02.512942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-12T03:21:01.014207+0000","last_change":"2024-09-12T03:20:59.815523+0000","last_active":"2024-09-12T03:21:01.014207+0000","last_peered":"2024-09-12T03:21:01.014207+0000","last_clean":"2024-09-12T03:21:01.014207+0000","last_became_active":"2024-09-12T03:20:59.815218+0000","last_became_peered":"2024-09-12T03:20:59.815218+0000","last_unstale":"2024-09-12T03:21:01.014207+0000","last_undegraded":"2024-09-12T03:21:01.014207+0000","last_fullsized":"2024-09-12T03:21:01.014207+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_clean_scrub_stamp":"2024-09-12T03:20:58.769862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:32:58.743319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":14,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-12T03:21:00.800617+0000","last_change":"2024-09-12T03:20:58.785470+0000","last_active":"2024-09-12T03:21:00.800617+0000","last_peered":"2024-09-12T03:21:00.800617+0000","last_clean":"2024-09-12T03:21:00.800617+0000","last_became_active":"2024-09-12T03:20:58.784580+0000","last_became_peered":"2024-09-12T03:20:58.784580+0000","last_unstale":"2024-09-12T03:21:00.800617+0000","last_undegraded":"2024-09-12T03:21:00.800617+0000","last_fullsized":"2024-09-12T03:21:00.800617+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:57.764885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:57.764885+0000","last_clean_scrub_stamp":"2024-09-12T03:20:57.764885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:39:08.649257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-12T03:21:00.800217+0000","last_change":"2024-09-12T03:20:59.802879+0000","last_active":"2024-09-12T03:21:00.800217+0000","last_peered":"2024-09-12T03:21:00.800217+0000","last_clean":"2024-09-12T03:21:00.800217+0000","last_became_active":"2024-09-12T03:20:59.802205+0000","last_became_peered":"2024-09-12T03:20:59.802205+0000","last_unstale":"2024-09-12T03:21:00.800217+0000","last_undegraded":"2024-09-12T03:21:00.800217+0000","last_fullsized":"2024-09-12T03:21:00.800217+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_clean_scrub_stamp":"2024-09-12T03:20:58.769862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:06:03.121556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":14,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-12T03:21:01.014239+0000","last_change":"2024-09-12T03:20:58.787702+0000","last_active":"2024-09-12T03:21:01.014239+0000","last_peered":"2024-09-12T03:21:01.014239+0000","last_clean":"2024-09-12T03:21:01.014239+0000","last_became_active":"2024-09-12T03:20:58.787402+0000","last_became_peered":"2024-09-12T03:20:58.787402+0000","last_unstale":"2024-09-12T03:21:01.014239+0000","last_undegraded":"2024-09-12T03:21:01.014239+0000","last_fullsized":"2024-09-12T03:21:01.014239+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:57.764885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:57.764885+0000","last_clean_scrub_stamp":"2024-09-12T03:20:57.764885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:22:16.377445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-12T03:21:00.801150+0000","last_change":"2024-09-12T03:20:59.804631+0000","last_active":"2024-09-12T03:21:00.801150+0000","last_peered":"2024-09-12T03:21:00.801150+0000","last_clean":"2024-09-12T03:21:00.801150+0000","last_became_active":"2024-09-12T03:20:59.804347+0000","last_became_peered":"2024-09-12T03:20:59.804347+0000","last_unstale":"2024-09-12T03:21:00.801150+0000","last_undegraded":"2024-09-12T03:21:00.801150+0000","last_fullsized":"2024-09-12T03:21:00.801150+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_clean_scrub_stamp":"2024-09-12T03:20:58.769862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:34:02.340684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":14,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-12T03:21:01.014396+0000","last_change":"2024-09-12T03:20:58.795794+0000","last_active":"2024-09-12T03:21:01.014396+0000","last_peered":"2024-09-12T03:21:01.014396+0000","last_clean":"2024-09-12T03:21:01.014396+0000","last_became_active":"2024-09-12T03:20:58.795488+0000","last_became_peered":"2024-09-12T03:20:58.795488+0000","last_unstale":"2024-09-12T03:21:01.014396+0000","last_undegraded":"2024-09-12T03:21:01.014396+0000","last_fullsized":"2024-09-12T03:21:01.014396+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:57.764885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:57.764885+0000","last_clean_scrub_stamp":"2024-09-12T03:20:57.764885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:55:59.826603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-12T03:21:00.807213+0000","last_change":"2024-09-12T03:20:59.818970+0000","last_active":"2024-09-12T03:21:00.807213+0000","last_peered":"2024-09-12T03:21:00.807213+0000","last_clean":"2024-09-12T03:21:00.807213+0000","last_became_active":"2024-09-12T03:20:59.818675+0000","last_became_peered":"2024-09-12T03:20:59.818675+0000","last_unstale":"2024-09-12T03:21:00.807213+0000","last_undegraded":"2024-09-12T03:21:00.807213+0000","last_fullsized":"2024-09-12T03:21:00.807213+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_clean_scrub_stamp":"2024-09-12T03:20:58.769862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:47:28.898685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":14,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-12T03:21:00.799540+0000","last_change":"2024-09-12T03:20:58.790159+0000","last_active":"2024-09-12T03:21:00.799540+0000","last_peered":"2024-09-12T03:21:00.799540+0000","last_clean":"2024-09-12T03:21:00.799540+0000","last_became_active":"2024-09-12T03:20:58.789913+0000","last_became_peered":"2024-09-12T03:20:58.789913+0000","last_unstale":"2024-09-12T03:21:00.799540+0000","last_undegraded":"2024-09-12T03:21:00.799540+0000","last_fullsized":"2024-09-12T03:21:00.799540+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:57.764885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:57.764885+0000","last_clean_scrub_stamp":"2024-09-12T03:20:57.764885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:59:23.802632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-12T03:21:00.799571+0000","last_change":"2024-09-12T03:20:59.799033+0000","last_active":"2024-09-12T03:21:00.799571+0000","last_peered":"2024-09-12T03:21:00.799571+0000","last_clean":"2024-09-12T03:21:00.799571+0000","last_became_active":"2024-09-12T03:20:59.798728+0000","last_became_peered":"2024-09-12T03:20:59.798728+0000","last_unstale":"2024-09-12T03:21:00.799571+0000","last_undegraded":"2024-09-12T03:21:00.799571+0000","last_fullsized":"2024-09-12T03:21:00.799571+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_clean_scrub_stamp":"2024-09-12T03:20:58.769862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:27:31.328878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.15","version":"99'3","reported_seq":17,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-12T03:21:01.015364+0000","last_change":"2024-09-12T03:20:58.782723+0000","last_active":"2024-09-12T03:21:01.015364+0000","last_peered":"2024-09-12T03:21:01.015364+0000","last_clean":"2024-09-12T03:21:01.015364+0000","last_became_active":"2024-09-12T03:20:58.782357+0000","last_became_peered":"2024-09-12T03:20:58.782357+0000","last_unstale":"2024-09-12T03:21:01.015364+0000","last_undegraded":"2024-09-12T03:21:01.015364+0000","last_fullsized":"2024-09-12T03:21:01.015364+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:57.764885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:57.764885+0000","last_clean_scrub_stamp":"2024-09-12T03:20:57.764885+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:08:51.994300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-12T03:21:00.800898+0000","last_change":"2024-09-12T03:20:59.807060+0000","last_active":"2024-09-12T03:21:00.800898+0000","last_peered":"2024-09-12T03:21:00.800898+0000","last_clean":"2024-09-12T03:21:00.800898+0000","last_became_active":"2024-09-12T03:20:59.806759+0000","last_became_peered":"2024-09-12T03:20:59.806759+0000","last_unstale":"2024-09-12T03:21:00.800898+0000","last_undegraded":"2024-09-12T03:21:00.800898+0000","last_fullsized":"2024-09-12T03:21:00.800898+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_clean_scrub_stamp":"2024-09-12T03:20:58.769862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:56:40.237221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"99'6","reported_seq":20,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-12T03:21:01.016379+0000","last_change":"2024-09-12T03:20:58.787890+0000","last_active":"2024-09-12T03:21:01.016379+0000","last_peered":"2024-09-12T03:21:01.016379+0000","last_clean":"2024-09-12T03:21:01.016379+0000","last_became_active":"2024-09-12T03:20:58.787589+0000","last_became_peered":"2024-09-12T03:20:58.787589+0000","last_unstale":"2024-09-12T03:21:01.016379+0000","last_undegraded":"2024-09-12T03:21:01.016379+0000","last_fullsized":"2024-09-12T03:21:01.016379+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:57.764885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:57.764885+0000","last_clean_scrub_stamp":"2024-09-12T03:20:57.764885+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:33:43.591218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-12T03:21:00.799672+0000","last_change":"2024-09-12T03:20:59.796560+0000","last_active":"2024-09-12T03:21:00.799672+0000","last_peered":"2024-09-12T03:21:00.799672+0000","last_clean":"2024-09-12T03:21:00.799672+0000","last_became_active":"2024-09-12T03:20:59.796040+0000","last_became_peered":"2024-09-12T03:20:59.796040+0000","last_unstale":"2024-09-12T03:21:00.799672+0000","last_undegraded":"2024-09-12T03:21:00.799672+0000","last_fullsized":"2024-09-12T03:21:00.799672+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_clean_scrub_stamp":"2024-09-12T03:20:58.769862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:09:34.594263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"99'2","reported_seq":16,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-12T03:21:01.015764+0000","last_change":"2024-09-12T03:20:58.783863+0000","last_active":"2024-09-12T03:21:01.015764+0000","last_peered":"2024-09-12T03:21:01.015764+0000","last_clean":"2024-09-12T03:21:01.015764+0000","last_became_active":"2024-09-12T03:20:58.783563+0000","last_became_peered":"2024-09-12T03:20:58.783563+0000","last_unstale":"2024-09-12T03:21:01.015764+0000","last_undegraded":"2024-09-12T03:21:01.015764+0000","last_fullsized":"2024-09-12T03:21:01.015764+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:57.764885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:57.764885+0000","last_clean_scrub_stamp":"2024-09-12T03:20:57.764885+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:09:22.594580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-12T03:21:00.799804+0000","last_change":"2024-09-12T03:20:59.806451+0000","last_active":"2024-09-12T03:21:00.799804+0000","last_peered":"2024-09-12T03:21:00.799804+0000","last_clean":"2024-09-12T03:21:00.799804+0000","last_became_active":"2024-09-12T03:20:59.806161+0000","last_became_peered":"2024-09-12T03:20:59.806161+0000","last_unstale":"2024-09-12T03:21:00.799804+0000","last_undegraded":"2024-09-12T03:21:00.799804+0000","last_fullsized":"2024-09-12T03:21:00.799804+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_clean_scrub_stamp":"2024-09-12T03:20:58.769862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:29:56.581778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":14,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-12T03:21:00.806861+0000","last_change":"2024-09-12T03:20:58.793160+0000","last_active":"2024-09-12T03:21:00.806861+0000","last_peered":"2024-09-12T03:21:00.806861+0000","last_clean":"2024-09-12T03:21:00.806861+0000","last_became_active":"2024-09-12T03:20:58.792865+0000","last_became_peered":"2024-09-12T03:20:58.792865+0000","last_unstale":"2024-09-12T03:21:00.806861+0000","last_undegraded":"2024-09-12T03:21:00.806861+0000","last_fullsized":"2024-09-12T03:21:00.806861+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:57.764885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:57.764885+0000","last_clean_scrub_stamp":"2024-09-12T03:20:57.764885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:50:28.197551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-12T03:21:00.799283+0000","last_change":"2024-09-12T03:20:59.798514+0000","last_active":"2024-09-12T03:21:00.799283+0000","last_peered":"2024-09-12T03:21:00.799283+0000","last_clean":"2024-09-12T03:21:00.799283+0000","last_became_active":"2024-09-12T03:20:59.798213+0000","last_became_peered":"2024-09-12T03:20:59.798213+0000","last_unstale":"2024-09-12T03:21:00.799283+0000","last_undegraded":"2024-09-12T03:21:00.799283+0000","last_fullsized":"2024-09-12T03:21:00.799283+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_clean_scrub_stamp":"2024-09-12T03:20:58.769862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:18:54.207971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"99'8","reported_seq":22,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-12T03:21:01.014573+0000","last_change":"2024-09-12T03:20:58.794537+0000","last_active":"2024-09-12T03:21:01.014573+0000","last_peered":"2024-09-12T03:21:01.014573+0000","last_clean":"2024-09-12T03:21:01.014573+0000","last_became_active":"2024-09-12T03:20:58.794158+0000","last_became_peered":"2024-09-12T03:20:58.794158+0000","last_unstale":"2024-09-12T03:21:01.014573+0000","last_undegraded":"2024-09-12T03:21:01.014573+0000","last_fullsized":"2024-09-12T03:21:01.014573+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:57.764885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:57.764885+0000","last_clean_scrub_stamp":"2024-09-12T03:20:57.764885+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:03:34.509048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-12T03:21:01.014540+0000","last_change":"2024-09-12T03:20:59.817332+0000","last_active":"2024-09-12T03:21:01.014540+0000","last_peered":"2024-09-12T03:21:01.014540+0000","last_clean":"2024-09-12T03:21:01.014540+0000","last_became_active":"2024-09-12T03:20:59.816867+0000","last_became_peered":"2024-09-12T03:20:59.816867+0000","last_unstale":"2024-09-12T03:21:01.014540+0000","last_undegraded":"2024-09-12T03:21:01.014540+0000","last_fullsized":"2024-09-12T03:21:01.014540+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_clean_scrub_stamp":"2024-09-12T03:20:58.769862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:06:33.366216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"99'1","reported_seq":15,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-12T03:21:01.016171+0000","last_change":"2024-09-12T03:20:58.783826+0000","last_active":"2024-09-12T03:21:01.016171+0000","last_peered":"2024-09-12T03:21:01.016171+0000","last_clean":"2024-09-12T03:21:01.016171+0000","last_became_active":"2024-09-12T03:20:58.783545+0000","last_became_peered":"2024-09-12T03:20:58.783545+0000","last_unstale":"2024-09-12T03:21:01.016171+0000","last_undegraded":"2024-09-12T03:21:01.016171+0000","last_fullsized":"2024-09-12T03:21:01.016171+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:57.764885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:57.764885+0000","last_clean_scrub_stamp":"2024-09-12T03:20:57.764885+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:24:04.897701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-12T03:21:01.015501+0000","last_change":"2024-09-12T03:20:59.821031+0000","last_active":"2024-09-12T03:21:01.015501+0000","last_peered":"2024-09-12T03:21:01.015501+0000","last_clean":"2024-09-12T03:21:01.015501+0000","last_became_active":"2024-09-12T03:20:59.820707+0000","last_became_peered":"2024-09-12T03:20:59.820707+0000","last_unstale":"2024-09-12T03:21:01.015501+0000","last_undegraded":"2024-09-12T03:21:01.015501+0000","last_fullsized":"2024-09-12T03:21:01.015501+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_clean_scrub_stamp":"2024-09-12T03:20:58.769862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:42:13.017144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.e","version":"99'2","reported_seq":16,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-12T03:21:01.014519+0000","last_change":"2024-09-12T03:20:58.791853+0000","last_active":"2024-09-12T03:21:01.014519+0000","last_peered":"2024-09-12T03:21:01.014519+0000","last_clean":"2024-09-12T03:21:01.014519+0000","last_became_active":"2024-09-12T03:20:58.791554+0000","last_became_peered":"2024-09-12T03:20:58.791554+0000","last_unstale":"2024-09-12T03:21:01.014519+0000","last_undegraded":"2024-09-12T03:21:01.014519+0000","last_fullsized":"2024-09-12T03:21:01.014519+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:57.764885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:57.764885+0000","last_clean_scrub_stamp":"2024-09-12T03:20:57.764885+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:12:42.114203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-12T03:21:01.015906+0000","last_change":"2024-09-12T03:20:59.826574+0000","last_active":"2024-09-12T03:21:01.015906+0000","last_peered":"2024-09-12T03:21:01.015906+0000","last_clean":"2024-09-12T03:21:01.015906+0000","last_became_active":"2024-09-12T03:20:59.826350+0000","last_became_peered":"2024-09-12T03:20:59.826350+0000","last_unstale":"2024-09-12T03:21:01.015906+0000","last_undegraded":"2024-09-12T03:21:01.015906+0000","last_fullsized":"2024-09-12T03:21:01.015906+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_clean_scrub_stamp":"2024-09-12T03:20:58.769862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:14:46.054149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.d","version":"99'1","reported_seq":15,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-12T03:21:00.806986+0000","last_change":"2024-09-12T03:20:58.791889+0000","last_active":"2024-09-12T03:21:00.806986+0000","last_peered":"2024-09-12T03:21:00.806986+0000","last_clean":"2024-09-12T03:21:00.806986+0000","last_became_active":"2024-09-12T03:20:58.791589+0000","last_became_peered":"2024-09-12T03:20:58.791589+0000","last_unstale":"2024-09-12T03:21:00.806986+0000","last_undegraded":"2024-09-12T03:21:00.806986+0000","last_fullsized":"2024-09-12T03:21:00.806986+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:57.764885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:57.764885+0000","last_clean_scrub_stamp":"2024-09-12T03:20:57.764885+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:59:23.802632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-12T03:21:01.016230+0000","last_change":"2024-09-12T03:20:59.807777+0000","last_active":"2024-09-12T03:21:01.016230+0000","last_peered":"2024-09-12T03:21:01.016230+0000","last_clean":"2024-09-12T03:21:01.016230+0000","last_became_active":"2024-09-12T03:20:59.807351+0000","last_became_peered":"2024-09-12T03:20:59.807351+0000","last_unstale":"2024-09-12T03:21:01.016230+0000","last_undegraded":"2024-09-12T03:21:01.016230+0000","last_fullsized":"2024-09-12T03:21:01.016230+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_clean_scrub_stamp":"2024-09-12T03:20:58.769862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:01:46.395221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":14,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-12T03:21:00.800556+0000","last_change":"2024-09-12T03:20:58.786401+0000","last_active":"2024-09-12T03:21:00.800556+0000","last_peered":"2024-09-12T03:21:00.800556+0000","last_clean":"2024-09-12T03:21:00.800556+0000","last_became_active":"2024-09-12T03:20:58.786052+0000","last_became_peered":"2024-09-12T03:20:58.786052+0000","last_unstale":"2024-09-12T03:21:00.800556+0000","last_undegraded":"2024-09-12T03:21:00.800556+0000","last_fullsized":"2024-09-12T03:21:00.800556+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:57.764885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:57.764885+0000","last_clean_scrub_stamp":"2024-09-12T03:20:57.764885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:42:12.012166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-12T03:21:01.015691+0000","last_change":"2024-09-12T03:20:59.803179+0000","last_active":"2024-09-12T03:21:01.015691+0000","last_peered":"2024-09-12T03:21:01.015691+0000","last_clean":"2024-09-12T03:21:01.015691+0000","last_became_active":"2024-09-12T03:20:59.802868+0000","last_became_peered":"2024-09-12T03:20:59.802868+0000","last_unstale":"2024-09-12T03:21:01.015691+0000","last_undegraded":"2024-09-12T03:21:01.015691+0000","last_fullsized":"2024-09-12T03:21:01.015691+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_clean_scrub_stamp":"2024-09-12T03:20:58.769862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:32:54.597742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"99'2","reported_seq":16,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-12T03:21:01.015853+0000","last_change":"2024-09-12T03:20:58.792805+0000","last_active":"2024-09-12T03:21:01.015853+0000","last_peered":"2024-09-12T03:21:01.015853+0000","last_clean":"2024-09-12T03:21:01.015853+0000","last_became_active":"2024-09-12T03:20:58.792525+0000","last_became_peered":"2024-09-12T03:20:58.792525+0000","last_unstale":"2024-09-12T03:21:01.015853+0000","last_undegraded":"2024-09-12T03:21:01.015853+0000","last_fullsized":"2024-09-12T03:21:01.015853+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:57.764885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:57.764885+0000","last_clean_scrub_stamp":"2024-09-12T03:20:57.764885+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:45:19.481652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-12T03:21:01.015116+0000","last_change":"2024-09-12T03:20:59.819374+0000","last_active":"2024-09-12T03:21:01.015116+0000","last_peered":"2024-09-12T03:21:01.015116+0000","last_clean":"2024-09-12T03:21:01.015116+0000","last_became_active":"2024-09-12T03:20:59.818991+0000","last_became_peered":"2024-09-12T03:20:59.818991+0000","last_unstale":"2024-09-12T03:21:01.015116+0000","last_undegraded":"2024-09-12T03:21:01.015116+0000","last_fullsized":"2024-09-12T03:21:01.015116+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_clean_scrub_stamp":"2024-09-12T03:20:58.769862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:06:24.788751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":14,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-12T03:21:01.013950+0000","last_change":"2024-09-12T03:20:58.795383+0000","last_active":"2024-09-12T03:21:01.013950+0000","last_peered":"2024-09-12T03:21:01.013950+0000","last_clean":"2024-09-12T03:21:01.013950+0000","last_became_active":"2024-09-12T03:20:58.795051+0000","last_became_peered":"2024-09-12T03:20:58.795051+0000","last_unstale":"2024-09-12T03:21:01.013950+0000","last_undegraded":"2024-09-12T03:21:01.013950+0000","last_fullsized":"2024-09-12T03:21:01.013950+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:57.764885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:57.764885+0000","last_clean_scrub_stamp":"2024-09-12T03:20:57.764885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:29:43.836647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-12T03:21:01.015328+0000","last_change":"2024-09-12T03:20:59.805532+0000","last_active":"2024-09-12T03:21:01.015328+0000","last_peered":"2024-09-12T03:21:01.015328+0000","last_clean":"2024-09-12T03:21:01.015328+0000","last_became_active":"2024-09-12T03:20:59.804500+0000","last_became_peered":"2024-09-12T03:20:59.804500+0000","last_unstale":"2024-09-12T03:21:01.015328+0000","last_undegraded":"2024-09-12T03:21:01.015328+0000","last_fullsized":"2024-09-12T03:21:01.015328+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_clean_scrub_stamp":"2024-09-12T03:20:58.769862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:28:50.840481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":14,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-12T03:21:01.016032+0000","last_change":"2024-09-12T03:20:58.793856+0000","last_active":"2024-09-12T03:21:01.016032+0000","last_peered":"2024-09-12T03:21:01.016032+0000","last_clean":"2024-09-12T03:21:01.016032+0000","last_became_active":"2024-09-12T03:20:58.793590+0000","last_became_peered":"2024-09-12T03:20:58.793590+0000","last_unstale":"2024-09-12T03:21:01.016032+0000","last_undegraded":"2024-09-12T03:21:01.016032+0000","last_fullsized":"2024-09-12T03:21:01.016032+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:57.764885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:57.764885+0000","last_clean_scrub_stamp":"2024-09-12T03:20:57.764885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:55:34.361301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-12T03:21:00.799416+0000","last_change":"2024-09-12T03:20:59.817900+0000","last_active":"2024-09-12T03:21:00.799416+0000","last_peered":"2024-09-12T03:21:00.799416+0000","last_clean":"2024-09-12T03:21:00.799416+0000","last_became_active":"2024-09-12T03:20:59.817549+0000","last_became_peered":"2024-09-12T03:20:59.817549+0000","last_unstale":"2024-09-12T03:21:00.799416+0000","last_undegraded":"2024-09-12T03:21:00.799416+0000","last_fullsized":"2024-09-12T03:21:00.799416+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_clean_scrub_stamp":"2024-09-12T03:20:58.769862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:32:54.597742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":14,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-12T03:21:01.014303+0000","last_change":"2024-09-12T03:20:58.790115+0000","last_active":"2024-09-12T03:21:01.014303+0000","last_peered":"2024-09-12T03:21:01.014303+0000","last_clean":"2024-09-12T03:21:01.014303+0000","last_became_active":"2024-09-12T03:20:58.789669+0000","last_became_peered":"2024-09-12T03:20:58.789669+0000","last_unstale":"2024-09-12T03:21:01.014303+0000","last_undegraded":"2024-09-12T03:21:01.014303+0000","last_fullsized":"2024-09-12T03:21:01.014303+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:57.764885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:57.764885+0000","last_clean_scrub_stamp":"2024-09-12T03:20:57.764885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:08:37.714256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-12T03:21:00.807130+0000","last_change":"2024-09-12T03:20:59.816082+0000","last_active":"2024-09-12T03:21:00.807130+0000","last_peered":"2024-09-12T03:21:00.807130+0000","last_clean":"2024-09-12T03:21:00.807130+0000","last_became_active":"2024-09-12T03:20:59.815781+0000","last_became_peered":"2024-09-12T03:20:59.815781+0000","last_unstale":"2024-09-12T03:21:00.807130+0000","last_undegraded":"2024-09-12T03:21:00.807130+0000","last_fullsized":"2024-09-12T03:21:00.807130+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_clean_scrub_stamp":"2024-09-12T03:20:58.769862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:36:44.063303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.7","version":"99'2","reported_seq":16,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-12T03:21:00.799474+0000","last_change":"2024-09-12T03:20:58.793458+0000","last_active":"2024-09-12T03:21:00.799474+0000","last_peered":"2024-09-12T03:21:00.799474+0000","last_clean":"2024-09-12T03:21:00.799474+0000","last_became_active":"2024-09-12T03:20:58.793160+0000","last_became_peered":"2024-09-12T03:20:58.793160+0000","last_unstale":"2024-09-12T03:21:00.799474+0000","last_undegraded":"2024-09-12T03:21:00.799474+0000","last_fullsized":"2024-09-12T03:21:00.799474+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:57.764885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:57.764885+0000","last_clean_scrub_stamp":"2024-09-12T03:20:57.764885+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:50:28.197551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-12T03:21:01.016094+0000","last_change":"2024-09-12T03:20:59.823804+0000","last_active":"2024-09-12T03:21:01.016094+0000","last_peered":"2024-09-12T03:21:01.016094+0000","last_clean":"2024-09-12T03:21:01.016094+0000","last_became_active":"2024-09-12T03:20:59.822379+0000","last_became_peered":"2024-09-12T03:20:59.822379+0000","last_unstale":"2024-09-12T03:21:01.016094+0000","last_undegraded":"2024-09-12T03:21:01.016094+0000","last_fullsized":"2024-09-12T03:21:01.016094+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_clean_scrub_stamp":"2024-09-12T03:20:58.769862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:48:20.080140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.6","version":"100'1","reported_seq":15,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-12T03:21:00.808400+0000","last_change":"2024-09-12T03:20:58.791357+0000","last_active":"2024-09-12T03:21:00.808400+0000","last_peered":"2024-09-12T03:21:00.808400+0000","last_clean":"2024-09-12T03:21:00.808400+0000","last_became_active":"2024-09-12T03:20:58.790915+0000","last_became_peered":"2024-09-12T03:20:58.790915+0000","last_unstale":"2024-09-12T03:21:00.808400+0000","last_undegraded":"2024-09-12T03:21:00.808400+0000","last_fullsized":"2024-09-12T03:21:00.808400+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:57.764885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:57.764885+0000","last_clean_scrub_stamp":"2024-09-12T03:20:57.764885+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:30:52.998555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-12T03:21:01.016296+0000","last_change":"2024-09-12T03:20:59.816598+0000","last_active":"2024-09-12T03:21:01.016296+0000","last_peered":"2024-09-12T03:21:01.016296+0000","last_clean":"2024-09-12T03:21:01.016296+0000","last_became_active":"2024-09-12T03:20:59.815748+0000","last_became_peered":"2024-09-12T03:20:59.815748+0000","last_unstale":"2024-09-12T03:21:01.016296+0000","last_undegraded":"2024-09-12T03:21:01.016296+0000","last_fullsized":"2024-09-12T03:21:01.016296+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_clean_scrub_stamp":"2024-09-12T03:20:58.769862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:07:35.076120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":14,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-12T03:21:01.015450+0000","last_change":"2024-09-12T03:20:58.789754+0000","last_active":"2024-09-12T03:21:01.015450+0000","last_peered":"2024-09-12T03:21:01.015450+0000","last_clean":"2024-09-12T03:21:01.015450+0000","last_became_active":"2024-09-12T03:20:58.789480+0000","last_became_peered":"2024-09-12T03:20:58.789480+0000","last_unstale":"2024-09-12T03:21:01.015450+0000","last_undegraded":"2024-09-12T03:21:01.015450+0000","last_fullsized":"2024-09-12T03:21:01.015450+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:57.764885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:57.764885+0000","last_clean_scrub_stamp":"2024-09-12T03:20:57.764885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:15:30.656163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-12T03:21:00.806976+0000","last_change":"2024-09-12T03:20:59.815751+0000","last_active":"2024-09-12T03:21:00.806976+0000","last_peered":"2024-09-12T03:21:00.806976+0000","last_clean":"2024-09-12T03:21:00.806976+0000","last_became_active":"2024-09-12T03:20:59.815436+0000","last_became_peered":"2024-09-12T03:20:59.815436+0000","last_unstale":"2024-09-12T03:21:00.806976+0000","last_undegraded":"2024-09-12T03:21:00.806976+0000","last_fullsized":"2024-09-12T03:21:00.806976+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_clean_scrub_stamp":"2024-09-12T03:20:58.769862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:54:52.453406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":14,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-12T03:21:00.799670+0000","last_change":"2024-09-12T03:20:58.788558+0000","last_active":"2024-09-12T03:21:00.799670+0000","last_peered":"2024-09-12T03:21:00.799670+0000","last_clean":"2024-09-12T03:21:00.799670+0000","last_became_active":"2024-09-12T03:20:58.788292+0000","last_became_peered":"2024-09-12T03:20:58.788292+0000","last_unstale":"2024-09-12T03:21:00.799670+0000","last_undegraded":"2024-09-12T03:21:00.799670+0000","last_fullsized":"2024-09-12T03:21:00.799670+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:57.764885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:57.764885+0000","last_clean_scrub_stamp":"2024-09-12T03:20:57.764885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:20:43.484861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-12T03:21:00.801042+0000","last_change":"2024-09-12T03:20:59.796569+0000","last_active":"2024-09-12T03:21:00.801042+0000","last_peered":"2024-09-12T03:21:00.801042+0000","last_clean":"2024-09-12T03:21:00.801042+0000","last_became_active":"2024-09-12T03:20:59.796255+0000","last_became_peered":"2024-09-12T03:20:59.796255+0000","last_unstale":"2024-09-12T03:21:00.801042+0000","last_undegraded":"2024-09-12T03:21:00.801042+0000","last_fullsized":"2024-09-12T03:21:00.801042+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_clean_scrub_stamp":"2024-09-12T03:20:58.769862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:47:28.898685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":14,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-12T03:21:00.800819+0000","last_change":"2024-09-12T03:20:58.788555+0000","last_active":"2024-09-12T03:21:00.800819+0000","last_peered":"2024-09-12T03:21:00.800819+0000","last_clean":"2024-09-12T03:21:00.800819+0000","last_became_active":"2024-09-12T03:20:58.785915+0000","last_became_peered":"2024-09-12T03:20:58.785915+0000","last_unstale":"2024-09-12T03:21:00.800819+0000","last_undegraded":"2024-09-12T03:21:00.800819+0000","last_fullsized":"2024-09-12T03:21:00.800819+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:57.764885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:57.764885+0000","last_clean_scrub_stamp":"2024-09-12T03:20:57.764885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:28:02.718330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-12T03:21:00.801235+0000","last_change":"2024-09-12T03:20:59.801009+0000","last_active":"2024-09-12T03:21:00.801235+0000","last_peered":"2024-09-12T03:21:00.801235+0000","last_clean":"2024-09-12T03:21:00.801235+0000","last_became_active":"2024-09-12T03:20:59.800693+0000","last_became_peered":"2024-09-12T03:20:59.800693+0000","last_unstale":"2024-09-12T03:21:00.801235+0000","last_undegraded":"2024-09-12T03:21:00.801235+0000","last_fullsized":"2024-09-12T03:21:00.801235+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_clean_scrub_stamp":"2024-09-12T03:20:58.769862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:28:50.840481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":14,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-12T03:21:00.801035+0000","last_change":"2024-09-12T03:20:58.785215+0000","last_active":"2024-09-12T03:21:00.801035+0000","last_peered":"2024-09-12T03:21:00.801035+0000","last_clean":"2024-09-12T03:21:00.801035+0000","last_became_active":"2024-09-12T03:20:58.784904+0000","last_became_peered":"2024-09-12T03:20:58.784904+0000","last_unstale":"2024-09-12T03:21:00.801035+0000","last_undegraded":"2024-09-12T03:21:00.801035+0000","last_fullsized":"2024-09-12T03:21:00.801035+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:57.764885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:57.764885+0000","last_clean_scrub_stamp":"2024-09-12T03:20:57.764885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:16:31.679549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-12T03:21:01.015434+0000","last_change":"2024-09-12T03:20:59.805632+0000","last_active":"2024-09-12T03:21:01.015434+0000","last_peered":"2024-09-12T03:21:01.015434+0000","last_clean":"2024-09-12T03:21:01.015434+0000","last_became_active":"2024-09-12T03:20:59.805096+0000","last_became_peered":"2024-09-12T03:20:59.805096+0000","last_unstale":"2024-09-12T03:21:01.015434+0000","last_undegraded":"2024-09-12T03:21:01.015434+0000","last_fullsized":"2024-09-12T03:21:01.015434+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_clean_scrub_stamp":"2024-09-12T03:20:58.769862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:56:50.124923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":14,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-12T03:21:01.013912+0000","last_change":"2024-09-12T03:20:58.794661+0000","last_active":"2024-09-12T03:21:01.013912+0000","last_peered":"2024-09-12T03:21:01.013912+0000","last_clean":"2024-09-12T03:21:01.013912+0000","last_became_active":"2024-09-12T03:20:58.794283+0000","last_became_peered":"2024-09-12T03:20:58.794283+0000","last_unstale":"2024-09-12T03:21:01.013912+0000","last_undegraded":"2024-09-12T03:21:01.013912+0000","last_fullsized":"2024-09-12T03:21:01.013912+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:57.764885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:57.764885+0000","last_clean_scrub_stamp":"2024-09-12T03:20:57.764885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:41:58.714733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-12T03:21:01.014766+0000","last_change":"2024-09-12T03:20:59.814345+0000","last_active":"2024-09-12T03:21:01.014766+0000","last_peered":"2024-09-12T03:21:01.014766+0000","last_clean":"2024-09-12T03:21:01.014766+0000","last_became_active":"2024-09-12T03:20:59.813587+0000","last_became_peered":"2024-09-12T03:20:59.813587+0000","last_unstale":"2024-09-12T03:21:01.014766+0000","last_undegraded":"2024-09-12T03:21:01.014766+0000","last_fullsized":"2024-09-12T03:21:01.014766+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_clean_scrub_stamp":"2024-09-12T03:20:58.769862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:13:37.100603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":14,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-12T03:21:01.014329+0000","last_change":"2024-09-12T03:20:58.789932+0000","last_active":"2024-09-12T03:21:01.014329+0000","last_peered":"2024-09-12T03:21:01.014329+0000","last_clean":"2024-09-12T03:21:01.014329+0000","last_became_active":"2024-09-12T03:20:58.789318+0000","last_became_peered":"2024-09-12T03:20:58.789318+0000","last_unstale":"2024-09-12T03:21:01.014329+0000","last_undegraded":"2024-09-12T03:21:01.014329+0000","last_fullsized":"2024-09-12T03:21:01.014329+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:57.764885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:57.764885+0000","last_clean_scrub_stamp":"2024-09-12T03:20:57.764885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:39:31.290244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-12T03:21:00.799333+0000","last_change":"2024-09-12T03:20:59.807209+0000","last_active":"2024-09-12T03:21:00.799333+0000","last_peered":"2024-09-12T03:21:00.799333+0000","last_clean":"2024-09-12T03:21:00.799333+0000","last_became_active":"2024-09-12T03:20:59.806924+0000","last_became_peered":"2024-09-12T03:20:59.806924+0000","last_unstale":"2024-09-12T03:21:00.799333+0000","last_undegraded":"2024-09-12T03:21:00.799333+0000","last_fullsized":"2024-09-12T03:21:00.799333+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_clean_scrub_stamp":"2024-09-12T03:20:58.769862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:56:50.124923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":14,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-12T03:21:00.800687+0000","last_change":"2024-09-12T03:20:58.785446+0000","last_active":"2024-09-12T03:21:00.800687+0000","last_peered":"2024-09-12T03:21:00.800687+0000","last_clean":"2024-09-12T03:21:00.800687+0000","last_became_active":"2024-09-12T03:20:58.785143+0000","last_became_peered":"2024-09-12T03:20:58.785143+0000","last_unstale":"2024-09-12T03:21:00.800687+0000","last_undegraded":"2024-09-12T03:21:00.800687+0000","last_fullsized":"2024-09-12T03:21:00.800687+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:57.764885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:57.764885+0000","last_clean_scrub_stamp":"2024-09-12T03:20:57.764885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:39:39.337275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"70'128","reported_seq":355,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-12T03:21:01.015473+0000","last_change":"2024-09-12T03:16:21.883446+0000","last_active":"2024-09-12T03:21:01.015473+0000","last_peered":"2024-09-12T03:21:01.015473+0000","last_clean":"2024-09-12T03:21:01.015473+0000","last_became_active":"2024-09-12T03:16:21.882207+0000","last_became_peered":"2024-09-12T03:16:21.882207+0000","last_unstale":"2024-09-12T03:21:01.015473+0000","last_undegraded":"2024-09-12T03:21:01.015473+0000","last_fullsized":"2024-09-12T03:21:01.015473+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:20.316426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:20.316426+0000","last_clean_scrub_stamp":"2024-09-12T03:16:20.316426+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:12:15.424320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542202,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37144,"kb_used_data":1788,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334696,"statfs":{"total":96636764160,"available":96598728704,"internally_reserved":0,"allocated":1830912,"data_stored":1129929,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542204,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36564,"kb_used_data":1216,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335276,"statfs":{"total":96636764160,"available":96599322624,"internally_reserved":0,"allocated":1245184,"data_stored":545315,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542203,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36556,"kb_used_data":1208,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335284,"statfs":{"total":96636764160,"available":96599330816,"internally_reserved":0,"allocated":1236992,"data_stored":539549,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3],"upper_bound":8},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542202,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36548,"kb_used_data":1200,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335292,"statfs":{"total":96636764160,"available":96599339008,"internally_reserved":0,"allocated":1228800,"data_stored":538945,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542203,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28256,"kb_used_data":1228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343584,"statfs":{"total":96636764160,"available":96607830016,"internally_reserved":0,"allocated":1257472,"data_stored":546927,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542203,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28236,"kb_used_data":1200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343604,"statfs":{"total":96636764160,"available":96607850496,"internally_reserved":0,"allocated":1228800,"data_stored":538945,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542204,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32396,"kb_used_data":1208,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339444,"statfs":{"total":96636764160,"available":96603590656,"internally_reserved":0,"allocated":1236992,"data_stored":539069,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542204,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33008,"kb_used_data":1812,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338832,"statfs":{"total":96636764160,"available":96602963968,"internally_reserved":0,"allocated":1855488,"data_stored":1137337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T03:21:02.786 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-12T03:21:02.923 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.923+0000 7f40d35c4640 1 Processor -- start 2024-09-12T03:21:02.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.923+0000 7f40d35c4640 1 -- start start 2024-09-12T03:21:02.930 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.924+0000 7f40d35c4640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f40cc0d1450 0x7f40cc070e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:02.930 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.924+0000 7f40d35c4640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f40cc164140 0x7f40cc166530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:02.930 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.924+0000 7f40d35c4640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f40cc06fa00 con 0x7f40cc0cef50 2024-09-12T03:21:02.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.924+0000 7f40d35c4640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f40cc15ca90 con 0x7f40cc0d1450 2024-09-12T03:21:02.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.924+0000 7f40d35c4640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f40cc0713c0 con 0x7f40cc164140 2024-09-12T03:21:02.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.924+0000 7f40d0b5f640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f40cc0d1450 0x7f40cc070e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:02.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.925+0000 7f40d0b5f640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f40cc0d1450 0x7f40cc070e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38546/0 (socket says 172.21.15.27:38546) 2024-09-12T03:21:02.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.925+0000 7f40d0b5f640 1 -- 172.21.15.27:0/4043507517 learned_addr learned my addr 172.21.15.27:0/4043507517 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:21:02.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.925+0000 7f40d1b61640 1 --2- 172.21.15.27:0/4043507517 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f40cc164140 0x7f40cc166530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:02.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.925+0000 7f40d0b5f640 1 -- 172.21.15.27:0/4043507517 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f40cc164140 msgr2=0x7f40cc166530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:02.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.925+0000 7f40d0b5f640 1 --2- 172.21.15.27:0/4043507517 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f40cc164140 0x7f40cc166530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:02.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.925+0000 7f40d0b5f640 1 -- 172.21.15.27:0/4043507517 >> v1:172.21.15.45:6789/0 conn(0x7f40cc0cef50 legacy=0x7f40cc0cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:02.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.925+0000 7f40d0b5f640 1 -- 172.21.15.27:0/4043507517 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40cc06f720 con 0x7f40cc0d1450 2024-09-12T03:21:02.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.926+0000 7f40d0b5f640 1 --2- 172.21.15.27:0/4043507517 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f40cc0d1450 0x7f40cc070e80 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f40bc009870 tx=0x7f40bc02a130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d130e0667b129eb server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:02.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.926+0000 7f40c3fff640 1 -- 172.21.15.27:0/4043507517 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40bc037070 con 0x7f40cc0d1450 2024-09-12T03:21:02.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.926+0000 7f40c3fff640 1 -- 172.21.15.27:0/4043507517 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40bc032440 con 0x7f40cc0d1450 2024-09-12T03:21:02.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.926+0000 7f40c3fff640 1 -- 172.21.15.27:0/4043507517 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40bc032810 con 0x7f40cc0d1450 2024-09-12T03:21:02.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.926+0000 7f40d35c4640 1 -- 172.21.15.27:0/4043507517 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f40cc0d1450 msgr2=0x7f40cc070e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:02.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.926+0000 7f40d35c4640 1 --2- 172.21.15.27:0/4043507517 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f40cc0d1450 0x7f40cc070e80 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f40bc009870 tx=0x7f40bc02a130 comp rx=0 tx=0).stop 2024-09-12T03:21:02.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.926+0000 7f40d35c4640 1 -- 172.21.15.27:0/4043507517 shutdown_connections 2024-09-12T03:21:02.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.926+0000 7f40d35c4640 1 --2- 172.21.15.27:0/4043507517 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f40cc164140 0x7f40cc166530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:02.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.926+0000 7f40d35c4640 1 --2- 172.21.15.27:0/4043507517 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f40cc0d1450 0x7f40cc070e80 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:02.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.926+0000 7f40d35c4640 1 -- 172.21.15.27:0/4043507517 >> 172.21.15.27:0/4043507517 conn(0x7f40cc073800 msgr2=0x7f40cc09ba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:02.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.926+0000 7f40d35c4640 1 -- 172.21.15.27:0/4043507517 shutdown_connections 2024-09-12T03:21:02.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.926+0000 7f40d35c4640 1 -- 172.21.15.27:0/4043507517 wait complete. 2024-09-12T03:21:02.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.927+0000 7f40d35c4640 1 Processor -- start 2024-09-12T03:21:02.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.928+0000 7f40d35c4640 1 -- start start 2024-09-12T03:21:02.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.928+0000 7f40d35c4640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f40cc0cef50 0x7f40cc14c210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:02.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.928+0000 7f40d35c4640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f40cc0d1450 0x7f40cc158e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:02.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.928+0000 7f40d35c4640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f40cc164140 0x7f40cc0b6f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:02.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.928+0000 7f40d35c4640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f40cc06f360 con 0x7f40cc0cef50 2024-09-12T03:21:02.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.928+0000 7f40d35c4640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f40cc0d2d10 con 0x7f40cc0d1450 2024-09-12T03:21:02.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.928+0000 7f40d35c4640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f40cc06f4e0 con 0x7f40cc164140 2024-09-12T03:21:02.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.928+0000 7f40d1360640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f40cc0cef50 0x7f40cc14c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:02.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.928+0000 7f40d1360640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f40cc0cef50 0x7f40cc14c210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38556/0 (socket says 172.21.15.27:38556) 2024-09-12T03:21:02.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.928+0000 7f40d1360640 1 -- 172.21.15.27:0/635486187 learned_addr learned my addr 172.21.15.27:0/635486187 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:21:02.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.929+0000 7f40d0b5f640 1 --2- 172.21.15.27:0/635486187 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f40cc0d1450 0x7f40cc158e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:02.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.929+0000 7f40d1b61640 1 --2- 172.21.15.27:0/635486187 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f40cc164140 0x7f40cc0b6f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:02.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.929+0000 7f40d1360640 1 -- 172.21.15.27:0/635486187 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f40cc164140 msgr2=0x7f40cc0b6f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:02.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.929+0000 7f40d1360640 1 --2- 172.21.15.27:0/635486187 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f40cc164140 0x7f40cc0b6f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:02.934 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.929+0000 7f40d1360640 1 -- 172.21.15.27:0/635486187 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f40cc0d1450 msgr2=0x7f40cc158e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:02.934 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.929+0000 7f40d1360640 1 --2- 172.21.15.27:0/635486187 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f40cc0d1450 0x7f40cc158e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:02.934 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.929+0000 7f40d1360640 1 -- 172.21.15.27:0/635486187 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40cc1594c0 con 0x7f40cc0cef50 2024-09-12T03:21:02.934 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.929+0000 7f40d1360640 1 --2- 172.21.15.27:0/635486187 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f40cc0cef50 0x7f40cc14c210 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f40c800c390 tx=0x7f40c800c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:02.934 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.929+0000 7f40c1ffb640 1 -- 172.21.15.27:0/635486187 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40c8017070 con 0x7f40cc0cef50 2024-09-12T03:21:02.934 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.929+0000 7f40c1ffb640 1 -- 172.21.15.27:0/635486187 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40c80052d0 con 0x7f40cc0cef50 2024-09-12T03:21:02.934 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.929+0000 7f40d35c4640 1 -- 172.21.15.27:0/635486187 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40cc153aa0 con 0x7f40cc0cef50 2024-09-12T03:21:02.934 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.929+0000 7f40c1ffb640 1 -- 172.21.15.27:0/635486187 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40c8003830 con 0x7f40cc0cef50 2024-09-12T03:21:02.934 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.929+0000 7f40d35c4640 1 -- 172.21.15.27:0/635486187 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40cc154060 con 0x7f40cc0cef50 2024-09-12T03:21:02.934 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.930+0000 7f40d35c4640 1 -- 172.21.15.27:0/635486187 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40cc0cfbd0 con 0x7f40cc0cef50 2024-09-12T03:21:02.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.931+0000 7f40c1ffb640 1 -- 172.21.15.27:0/635486187 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 90600+0+0 (secure 0 0 0) 0x7f40c801a070 con 0x7f40cc0cef50 2024-09-12T03:21:02.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.931+0000 7f40c1ffb640 1 --2- 172.21.15.27:0/635486187 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f40b006bf50 0x7f40b006e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:02.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.931+0000 7f40c1ffb640 1 -- 172.21.15.27:0/635486187 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5709+0+0 (secure 0 0 0) 0x7f40c808a2b0 con 0x7f40cc0cef50 2024-09-12T03:21:02.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.931+0000 7f40d0b5f640 1 --2- 172.21.15.27:0/635486187 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f40b006bf50 0x7f40b006e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:02.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.932+0000 7f40d0b5f640 1 --2- 172.21.15.27:0/635486187 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f40b006bf50 0x7f40b006e410 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f40bc034000 tx=0x7f40bc02e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:02.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:02.936+0000 7f40c1ffb640 1 -- 172.21.15.27:0/635486187 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f40c8058500 con 0x7f40cc0cef50 2024-09-12T03:21:03.109 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:03.109+0000 7f40d35c4640 1 -- 172.21.15.27:0/635486187 --> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f40cc0b4fc0 con 0x7f40b006bf50 2024-09-12T03:21:03.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:03.115+0000 7f40c1ffb640 1 -- 172.21.15.27:0/635486187 <== mgr.4113 v2:172.21.15.27:6836/1226920526 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252186 (secure 0 0 0) 0x7f40cc0b4fc0 con 0x7f40b006bf50 2024-09-12T03:21:03.120 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:21:03.122 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2024-09-12T03:21:03.125 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:03.119+0000 7f40d35c4640 1 -- 172.21.15.27:0/635486187 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f40b006bf50 msgr2=0x7f40b006e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:03.125 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:03.119+0000 7f40d35c4640 1 --2- 172.21.15.27:0/635486187 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f40b006bf50 0x7f40b006e410 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f40bc034000 tx=0x7f40bc02e000 comp rx=0 tx=0).stop 2024-09-12T03:21:03.125 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:03.119+0000 7f40d35c4640 1 -- 172.21.15.27:0/635486187 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f40cc0cef50 msgr2=0x7f40cc14c210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:03.126 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:03.119+0000 7f40d35c4640 1 --2- 172.21.15.27:0/635486187 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f40cc0cef50 0x7f40cc14c210 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f40c800c390 tx=0x7f40c800c850 comp rx=0 tx=0).stop 2024-09-12T03:21:03.126 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:03.119+0000 7f40d35c4640 1 -- 172.21.15.27:0/635486187 shutdown_connections 2024-09-12T03:21:03.126 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:03.119+0000 7f40d35c4640 1 --2- 172.21.15.27:0/635486187 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f40cc164140 0x7f40cc0b6f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:03.126 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:03.119+0000 7f40d35c4640 1 --2- 172.21.15.27:0/635486187 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f40cc0d1450 0x7f40cc158e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:03.127 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:03.119+0000 7f40d35c4640 1 --2- 172.21.15.27:0/635486187 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f40b006bf50 0x7f40b006e410 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:03.127 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:03.119+0000 7f40d35c4640 1 --2- 172.21.15.27:0/635486187 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f40cc0cef50 0x7f40cc14c210 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:03.127 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:03.119+0000 7f40d35c4640 1 -- 172.21.15.27:0/635486187 >> 172.21.15.27:0/635486187 conn(0x7f40cc073800 msgr2=0x7f40cc0b3830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:03.127 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:03.119+0000 7f40d35c4640 1 -- 172.21.15.27:0/635486187 shutdown_connections 2024-09-12T03:21:03.127 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:03.119+0000 7f40d35c4640 1 -- 172.21.15.27:0/635486187 wait complete. 2024-09-12T03:21:03.129 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":58,"stamp":"2024-09-12T03:21:03.104412+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":224,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":167,"ondisk_log_size":167,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":115,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":268708,"kb_used_data":10860,"kb_used_omap":12,"kb_used_meta":257651,"kb_avail":754706012,"statfs":{"total":773094113280,"available":772818956288,"internally_reserved":0,"allocated":11120640,"data_stored":5516016,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":263835184},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3],"upper_bound":8},"perf_stat":{"commit_latency_ms":20,"apply_latency_ms":20,"commit_latency_ns":20000000,"apply_latency_ns":20000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.333662"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-12T03:21:01.015811+0000","last_change":"2024-09-12T03:20:59.812495+0000","last_active":"2024-09-12T03:21:01.015811+0000","last_peered":"2024-09-12T03:21:01.015811+0000","last_clean":"2024-09-12T03:21:01.015811+0000","last_became_active":"2024-09-12T03:20:59.812227+0000","last_became_peered":"2024-09-12T03:20:59.812227+0000","last_unstale":"2024-09-12T03:21:01.015811+0000","last_undegraded":"2024-09-12T03:21:01.015811+0000","last_fullsized":"2024-09-12T03:21:01.015811+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_clean_scrub_stamp":"2024-09-12T03:20:58.769862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:40:38.653233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-12T03:21:00.801750+0000","last_change":"2024-09-12T03:20:59.804237+0000","last_active":"2024-09-12T03:21:00.801750+0000","last_peered":"2024-09-12T03:21:00.801750+0000","last_clean":"2024-09-12T03:21:00.801750+0000","last_became_active":"2024-09-12T03:20:59.803405+0000","last_became_peered":"2024-09-12T03:20:59.803405+0000","last_unstale":"2024-09-12T03:21:00.801750+0000","last_undegraded":"2024-09-12T03:21:00.801750+0000","last_fullsized":"2024-09-12T03:21:00.801750+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_clean_scrub_stamp":"2024-09-12T03:20:58.769862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:32:54.597742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-12T03:21:01.016035+0000","last_change":"2024-09-12T03:20:59.810587+0000","last_active":"2024-09-12T03:21:01.016035+0000","last_peered":"2024-09-12T03:21:01.016035+0000","last_clean":"2024-09-12T03:21:01.016035+0000","last_became_active":"2024-09-12T03:20:59.810237+0000","last_became_peered":"2024-09-12T03:20:59.810237+0000","last_unstale":"2024-09-12T03:21:01.016035+0000","last_undegraded":"2024-09-12T03:21:01.016035+0000","last_fullsized":"2024-09-12T03:21:01.016035+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_clean_scrub_stamp":"2024-09-12T03:20:58.769862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:27:31.328878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-12T03:21:01.016212+0000","last_change":"2024-09-12T03:20:59.820197+0000","last_active":"2024-09-12T03:21:01.016212+0000","last_peered":"2024-09-12T03:21:01.016212+0000","last_clean":"2024-09-12T03:21:01.016212+0000","last_became_active":"2024-09-12T03:20:59.819922+0000","last_became_peered":"2024-09-12T03:20:59.819922+0000","last_unstale":"2024-09-12T03:21:01.016212+0000","last_undegraded":"2024-09-12T03:21:01.016212+0000","last_fullsized":"2024-09-12T03:21:01.016212+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_clean_scrub_stamp":"2024-09-12T03:20:58.769862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:41:57.265774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-12T03:21:01.014231+0000","last_change":"2024-09-12T03:20:59.812365+0000","last_active":"2024-09-12T03:21:01.014231+0000","last_peered":"2024-09-12T03:21:01.014231+0000","last_clean":"2024-09-12T03:21:01.014231+0000","last_became_active":"2024-09-12T03:20:59.812073+0000","last_became_peered":"2024-09-12T03:20:59.812073+0000","last_unstale":"2024-09-12T03:21:01.014231+0000","last_undegraded":"2024-09-12T03:21:01.014231+0000","last_fullsized":"2024-09-12T03:21:01.014231+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_clean_scrub_stamp":"2024-09-12T03:20:58.769862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:51:14.057192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-12T03:21:01.016019+0000","last_change":"2024-09-12T03:20:59.812997+0000","last_active":"2024-09-12T03:21:01.016019+0000","last_peered":"2024-09-12T03:21:01.016019+0000","last_clean":"2024-09-12T03:21:01.016019+0000","last_became_active":"2024-09-12T03:20:59.812693+0000","last_became_peered":"2024-09-12T03:20:59.812693+0000","last_unstale":"2024-09-12T03:21:01.016019+0000","last_undegraded":"2024-09-12T03:21:01.016019+0000","last_fullsized":"2024-09-12T03:21:01.016019+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_clean_scrub_stamp":"2024-09-12T03:20:58.769862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:47:32.534307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-12T03:21:00.800567+0000","last_change":"2024-09-12T03:20:59.802840+0000","last_active":"2024-09-12T03:21:00.800567+0000","last_peered":"2024-09-12T03:21:00.800567+0000","last_clean":"2024-09-12T03:21:00.800567+0000","last_became_active":"2024-09-12T03:20:59.802530+0000","last_became_peered":"2024-09-12T03:20:59.802530+0000","last_unstale":"2024-09-12T03:21:00.800567+0000","last_undegraded":"2024-09-12T03:21:00.800567+0000","last_fullsized":"2024-09-12T03:21:00.800567+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_clean_scrub_stamp":"2024-09-12T03:20:58.769862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:08:29.553346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-12T03:21:01.015363+0000","last_change":"2024-09-12T03:20:59.805399+0000","last_active":"2024-09-12T03:21:01.015363+0000","last_peered":"2024-09-12T03:21:01.015363+0000","last_clean":"2024-09-12T03:21:01.015363+0000","last_became_active":"2024-09-12T03:20:59.804834+0000","last_became_peered":"2024-09-12T03:20:59.804834+0000","last_unstale":"2024-09-12T03:21:01.015363+0000","last_undegraded":"2024-09-12T03:21:01.015363+0000","last_fullsized":"2024-09-12T03:21:01.015363+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_clean_scrub_stamp":"2024-09-12T03:20:58.769862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:18:54.207971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-12T03:21:01.015949+0000","last_change":"2024-09-12T03:20:59.824528+0000","last_active":"2024-09-12T03:21:01.015949+0000","last_peered":"2024-09-12T03:21:01.015949+0000","last_clean":"2024-09-12T03:21:01.015949+0000","last_became_active":"2024-09-12T03:20:59.824204+0000","last_became_peered":"2024-09-12T03:20:59.824204+0000","last_unstale":"2024-09-12T03:21:01.015949+0000","last_undegraded":"2024-09-12T03:21:01.015949+0000","last_fullsized":"2024-09-12T03:21:01.015949+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_clean_scrub_stamp":"2024-09-12T03:20:58.769862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:19:55.126748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-12T03:21:01.015355+0000","last_change":"2024-09-12T03:20:59.815730+0000","last_active":"2024-09-12T03:21:01.015355+0000","last_peered":"2024-09-12T03:21:01.015355+0000","last_clean":"2024-09-12T03:21:01.015355+0000","last_became_active":"2024-09-12T03:20:59.815367+0000","last_became_peered":"2024-09-12T03:20:59.815367+0000","last_unstale":"2024-09-12T03:21:01.015355+0000","last_undegraded":"2024-09-12T03:21:01.015355+0000","last_fullsized":"2024-09-12T03:21:01.015355+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_clean_scrub_stamp":"2024-09-12T03:20:58.769862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:32:00.897353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-12T03:21:00.801476+0000","last_change":"2024-09-12T03:20:59.799545+0000","last_active":"2024-09-12T03:21:00.801476+0000","last_peered":"2024-09-12T03:21:00.801476+0000","last_clean":"2024-09-12T03:21:00.801476+0000","last_became_active":"2024-09-12T03:20:59.799234+0000","last_became_peered":"2024-09-12T03:20:59.799234+0000","last_unstale":"2024-09-12T03:21:00.801476+0000","last_undegraded":"2024-09-12T03:21:00.801476+0000","last_fullsized":"2024-09-12T03:21:00.801476+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_clean_scrub_stamp":"2024-09-12T03:20:58.769862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:56:50.124923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-12T03:21:01.016330+0000","last_change":"2024-09-12T03:20:59.816949+0000","last_active":"2024-09-12T03:21:01.016330+0000","last_peered":"2024-09-12T03:21:01.016330+0000","last_clean":"2024-09-12T03:21:01.016330+0000","last_became_active":"2024-09-12T03:20:59.816264+0000","last_became_peered":"2024-09-12T03:20:59.816264+0000","last_unstale":"2024-09-12T03:21:01.016330+0000","last_undegraded":"2024-09-12T03:21:01.016330+0000","last_fullsized":"2024-09-12T03:21:01.016330+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_clean_scrub_stamp":"2024-09-12T03:20:58.769862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:06:53.605499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-12T03:21:01.015881+0000","last_change":"2024-09-12T03:20:59.802684+0000","last_active":"2024-09-12T03:21:01.015881+0000","last_peered":"2024-09-12T03:21:01.015881+0000","last_clean":"2024-09-12T03:21:01.015881+0000","last_became_active":"2024-09-12T03:20:59.802364+0000","last_became_peered":"2024-09-12T03:20:59.802364+0000","last_unstale":"2024-09-12T03:21:01.015881+0000","last_undegraded":"2024-09-12T03:21:01.015881+0000","last_fullsized":"2024-09-12T03:21:01.015881+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_clean_scrub_stamp":"2024-09-12T03:20:58.769862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:49:14.153740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-12T03:21:01.015351+0000","last_change":"2024-09-12T03:20:59.803518+0000","last_active":"2024-09-12T03:21:01.015351+0000","last_peered":"2024-09-12T03:21:01.015351+0000","last_clean":"2024-09-12T03:21:01.015351+0000","last_became_active":"2024-09-12T03:20:59.802943+0000","last_became_peered":"2024-09-12T03:20:59.802943+0000","last_unstale":"2024-09-12T03:21:01.015351+0000","last_undegraded":"2024-09-12T03:21:01.015351+0000","last_fullsized":"2024-09-12T03:21:01.015351+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_clean_scrub_stamp":"2024-09-12T03:20:58.769862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:39:28.867757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-12T03:21:01.016020+0000","last_change":"2024-09-12T03:20:59.811528+0000","last_active":"2024-09-12T03:21:01.016020+0000","last_peered":"2024-09-12T03:21:01.016020+0000","last_clean":"2024-09-12T03:21:01.016020+0000","last_became_active":"2024-09-12T03:20:59.811243+0000","last_became_peered":"2024-09-12T03:20:59.811243+0000","last_unstale":"2024-09-12T03:21:01.016020+0000","last_undegraded":"2024-09-12T03:21:01.016020+0000","last_fullsized":"2024-09-12T03:21:01.016020+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_clean_scrub_stamp":"2024-09-12T03:20:58.769862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:59:11.158965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-12T03:21:01.014058+0000","last_change":"2024-09-12T03:20:59.820472+0000","last_active":"2024-09-12T03:21:01.014058+0000","last_peered":"2024-09-12T03:21:01.014058+0000","last_clean":"2024-09-12T03:21:01.014058+0000","last_became_active":"2024-09-12T03:20:59.819864+0000","last_became_peered":"2024-09-12T03:20:59.819864+0000","last_unstale":"2024-09-12T03:21:01.014058+0000","last_undegraded":"2024-09-12T03:21:01.014058+0000","last_fullsized":"2024-09-12T03:21:01.014058+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_clean_scrub_stamp":"2024-09-12T03:20:58.769862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:15:57.664758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-12T03:21:00.801781+0000","last_change":"2024-09-12T03:20:59.806723+0000","last_active":"2024-09-12T03:21:00.801781+0000","last_peered":"2024-09-12T03:21:00.801781+0000","last_clean":"2024-09-12T03:21:00.801781+0000","last_became_active":"2024-09-12T03:20:59.806418+0000","last_became_peered":"2024-09-12T03:20:59.806418+0000","last_unstale":"2024-09-12T03:21:00.801781+0000","last_undegraded":"2024-09-12T03:21:00.801781+0000","last_fullsized":"2024-09-12T03:21:00.801781+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_clean_scrub_stamp":"2024-09-12T03:20:58.769862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:49:14.153740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-12T03:21:00.807037+0000","last_change":"2024-09-12T03:20:59.804307+0000","last_active":"2024-09-12T03:21:00.807037+0000","last_peered":"2024-09-12T03:21:00.807037+0000","last_clean":"2024-09-12T03:21:00.807037+0000","last_became_active":"2024-09-12T03:20:59.803978+0000","last_became_peered":"2024-09-12T03:20:59.803978+0000","last_unstale":"2024-09-12T03:21:00.807037+0000","last_undegraded":"2024-09-12T03:21:00.807037+0000","last_fullsized":"2024-09-12T03:21:00.807037+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_clean_scrub_stamp":"2024-09-12T03:20:58.769862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:39:27.157965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-12T03:21:01.016171+0000","last_change":"2024-09-12T03:20:59.805203+0000","last_active":"2024-09-12T03:21:01.016171+0000","last_peered":"2024-09-12T03:21:01.016171+0000","last_clean":"2024-09-12T03:21:01.016171+0000","last_became_active":"2024-09-12T03:20:59.804424+0000","last_became_peered":"2024-09-12T03:20:59.804424+0000","last_unstale":"2024-09-12T03:21:01.016171+0000","last_undegraded":"2024-09-12T03:21:01.016171+0000","last_fullsized":"2024-09-12T03:21:01.016171+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_clean_scrub_stamp":"2024-09-12T03:20:58.769862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:29:56.581778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-12T03:21:00.801274+0000","last_change":"2024-09-12T03:20:59.809588+0000","last_active":"2024-09-12T03:21:00.801274+0000","last_peered":"2024-09-12T03:21:00.801274+0000","last_clean":"2024-09-12T03:21:00.801274+0000","last_became_active":"2024-09-12T03:20:59.808976+0000","last_became_peered":"2024-09-12T03:20:59.808976+0000","last_unstale":"2024-09-12T03:21:00.801274+0000","last_undegraded":"2024-09-12T03:21:00.801274+0000","last_fullsized":"2024-09-12T03:21:00.801274+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_clean_scrub_stamp":"2024-09-12T03:20:58.769862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:39:28.867757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-12T03:21:00.801335+0000","last_change":"2024-09-12T03:20:59.806035+0000","last_active":"2024-09-12T03:21:00.801335+0000","last_peered":"2024-09-12T03:21:00.801335+0000","last_clean":"2024-09-12T03:21:00.801335+0000","last_became_active":"2024-09-12T03:20:59.805734+0000","last_became_peered":"2024-09-12T03:20:59.805734+0000","last_unstale":"2024-09-12T03:21:00.801335+0000","last_undegraded":"2024-09-12T03:21:00.801335+0000","last_fullsized":"2024-09-12T03:21:00.801335+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_clean_scrub_stamp":"2024-09-12T03:20:58.769862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:18:54.207971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-12T03:21:00.799520+0000","last_change":"2024-09-12T03:20:59.804655+0000","last_active":"2024-09-12T03:21:00.799520+0000","last_peered":"2024-09-12T03:21:00.799520+0000","last_clean":"2024-09-12T03:21:00.799520+0000","last_became_active":"2024-09-12T03:20:59.804358+0000","last_became_peered":"2024-09-12T03:20:59.804358+0000","last_unstale":"2024-09-12T03:21:00.799520+0000","last_undegraded":"2024-09-12T03:21:00.799520+0000","last_fullsized":"2024-09-12T03:21:00.799520+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_clean_scrub_stamp":"2024-09-12T03:20:58.769862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:14:46.054149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-12T03:21:00.800507+0000","last_change":"2024-09-12T03:20:59.808182+0000","last_active":"2024-09-12T03:21:00.800507+0000","last_peered":"2024-09-12T03:21:00.800507+0000","last_clean":"2024-09-12T03:21:00.800507+0000","last_became_active":"2024-09-12T03:20:59.807909+0000","last_became_peered":"2024-09-12T03:20:59.807909+0000","last_unstale":"2024-09-12T03:21:00.800507+0000","last_undegraded":"2024-09-12T03:21:00.800507+0000","last_fullsized":"2024-09-12T03:21:00.800507+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_clean_scrub_stamp":"2024-09-12T03:20:58.769862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:39:27.157965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-12T03:21:00.799250+0000","last_change":"2024-09-12T03:20:59.809254+0000","last_active":"2024-09-12T03:21:00.799250+0000","last_peered":"2024-09-12T03:21:00.799250+0000","last_clean":"2024-09-12T03:21:00.799250+0000","last_became_active":"2024-09-12T03:20:59.808969+0000","last_became_peered":"2024-09-12T03:20:59.808969+0000","last_unstale":"2024-09-12T03:21:00.799250+0000","last_undegraded":"2024-09-12T03:21:00.799250+0000","last_fullsized":"2024-09-12T03:21:00.799250+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_clean_scrub_stamp":"2024-09-12T03:20:58.769862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:39:28.867757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-12T03:21:00.801841+0000","last_change":"2024-09-12T03:20:59.806550+0000","last_active":"2024-09-12T03:21:00.801841+0000","last_peered":"2024-09-12T03:21:00.801841+0000","last_clean":"2024-09-12T03:21:00.801841+0000","last_became_active":"2024-09-12T03:20:59.806245+0000","last_became_peered":"2024-09-12T03:20:59.806245+0000","last_unstale":"2024-09-12T03:21:00.801841+0000","last_undegraded":"2024-09-12T03:21:00.801841+0000","last_fullsized":"2024-09-12T03:21:00.801841+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_clean_scrub_stamp":"2024-09-12T03:20:58.769862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:14:46.054149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-12T03:21:01.015501+0000","last_change":"2024-09-12T03:20:59.815020+0000","last_active":"2024-09-12T03:21:01.015501+0000","last_peered":"2024-09-12T03:21:01.015501+0000","last_clean":"2024-09-12T03:21:01.015501+0000","last_became_active":"2024-09-12T03:20:59.814707+0000","last_became_peered":"2024-09-12T03:20:59.814707+0000","last_unstale":"2024-09-12T03:21:01.015501+0000","last_undegraded":"2024-09-12T03:21:01.015501+0000","last_fullsized":"2024-09-12T03:21:01.015501+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_clean_scrub_stamp":"2024-09-12T03:20:58.769862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:53:47.188882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-12T03:21:01.015746+0000","last_change":"2024-09-12T03:20:59.821070+0000","last_active":"2024-09-12T03:21:01.015746+0000","last_peered":"2024-09-12T03:21:01.015746+0000","last_clean":"2024-09-12T03:21:01.015746+0000","last_became_active":"2024-09-12T03:20:59.820747+0000","last_became_peered":"2024-09-12T03:20:59.820747+0000","last_unstale":"2024-09-12T03:21:01.015746+0000","last_undegraded":"2024-09-12T03:21:01.015746+0000","last_fullsized":"2024-09-12T03:21:01.015746+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_clean_scrub_stamp":"2024-09-12T03:20:58.769862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:28:03.723307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-12T03:21:00.799493+0000","last_change":"2024-09-12T03:20:59.818013+0000","last_active":"2024-09-12T03:21:00.799493+0000","last_peered":"2024-09-12T03:21:00.799493+0000","last_clean":"2024-09-12T03:21:00.799493+0000","last_became_active":"2024-09-12T03:20:59.817704+0000","last_became_peered":"2024-09-12T03:20:59.817704+0000","last_unstale":"2024-09-12T03:21:00.799493+0000","last_undegraded":"2024-09-12T03:21:00.799493+0000","last_fullsized":"2024-09-12T03:21:00.799493+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_clean_scrub_stamp":"2024-09-12T03:20:58.769862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:49:14.153740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-12T03:21:00.800544+0000","last_change":"2024-09-12T03:20:59.809481+0000","last_active":"2024-09-12T03:21:00.800544+0000","last_peered":"2024-09-12T03:21:00.800544+0000","last_clean":"2024-09-12T03:21:00.800544+0000","last_became_active":"2024-09-12T03:20:59.809200+0000","last_became_peered":"2024-09-12T03:20:59.809200+0000","last_unstale":"2024-09-12T03:21:00.800544+0000","last_undegraded":"2024-09-12T03:21:00.800544+0000","last_fullsized":"2024-09-12T03:21:00.800544+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_clean_scrub_stamp":"2024-09-12T03:20:58.769862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:36:44.063303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-12T03:21:00.800761+0000","last_change":"2024-09-12T03:20:59.798156+0000","last_active":"2024-09-12T03:21:00.800761+0000","last_peered":"2024-09-12T03:21:00.800761+0000","last_clean":"2024-09-12T03:21:00.800761+0000","last_became_active":"2024-09-12T03:20:59.797821+0000","last_became_peered":"2024-09-12T03:20:59.797821+0000","last_unstale":"2024-09-12T03:21:00.800761+0000","last_undegraded":"2024-09-12T03:21:00.800761+0000","last_fullsized":"2024-09-12T03:21:00.800761+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_clean_scrub_stamp":"2024-09-12T03:20:58.769862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:35:07.183939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-12T03:21:01.014769+0000","last_change":"2024-09-12T03:20:59.814559+0000","last_active":"2024-09-12T03:21:01.014769+0000","last_peered":"2024-09-12T03:21:01.014769+0000","last_clean":"2024-09-12T03:21:01.014769+0000","last_became_active":"2024-09-12T03:20:59.813964+0000","last_became_peered":"2024-09-12T03:20:59.813964+0000","last_unstale":"2024-09-12T03:21:01.014769+0000","last_undegraded":"2024-09-12T03:21:01.014769+0000","last_fullsized":"2024-09-12T03:21:01.014769+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_clean_scrub_stamp":"2024-09-12T03:20:58.769862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:42:06.157939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-12T03:21:01.015130+0000","last_change":"2024-09-12T03:20:59.812258+0000","last_active":"2024-09-12T03:21:01.015130+0000","last_peered":"2024-09-12T03:21:01.015130+0000","last_clean":"2024-09-12T03:21:01.015130+0000","last_became_active":"2024-09-12T03:20:59.811984+0000","last_became_peered":"2024-09-12T03:20:59.811984+0000","last_unstale":"2024-09-12T03:21:01.015130+0000","last_undegraded":"2024-09-12T03:21:01.015130+0000","last_fullsized":"2024-09-12T03:21:01.015130+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_clean_scrub_stamp":"2024-09-12T03:20:58.769862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:54:36.464412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-12T03:21:01.015511+0000","last_change":"2024-09-12T03:20:59.801304+0000","last_active":"2024-09-12T03:21:01.015511+0000","last_peered":"2024-09-12T03:21:01.015511+0000","last_clean":"2024-09-12T03:21:01.015511+0000","last_became_active":"2024-09-12T03:20:59.800985+0000","last_became_peered":"2024-09-12T03:20:59.800985+0000","last_unstale":"2024-09-12T03:21:01.015511+0000","last_undegraded":"2024-09-12T03:21:01.015511+0000","last_fullsized":"2024-09-12T03:21:01.015511+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_clean_scrub_stamp":"2024-09-12T03:20:58.769862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:42:58.503627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"99'4","reported_seq":18,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-12T03:21:00.801463+0000","last_change":"2024-09-12T03:20:58.784264+0000","last_active":"2024-09-12T03:21:00.801463+0000","last_peered":"2024-09-12T03:21:00.801463+0000","last_clean":"2024-09-12T03:21:00.801463+0000","last_became_active":"2024-09-12T03:20:58.783932+0000","last_became_peered":"2024-09-12T03:20:58.783932+0000","last_unstale":"2024-09-12T03:21:00.801463+0000","last_undegraded":"2024-09-12T03:21:00.801463+0000","last_fullsized":"2024-09-12T03:21:00.801463+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:57.764885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:57.764885+0000","last_clean_scrub_stamp":"2024-09-12T03:20:57.764885+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:47:31.529330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-12T03:21:00.807152+0000","last_change":"2024-09-12T03:20:59.805989+0000","last_active":"2024-09-12T03:21:00.807152+0000","last_peered":"2024-09-12T03:21:00.807152+0000","last_clean":"2024-09-12T03:21:00.807152+0000","last_became_active":"2024-09-12T03:20:59.804997+0000","last_became_peered":"2024-09-12T03:20:59.804997+0000","last_unstale":"2024-09-12T03:21:00.807152+0000","last_undegraded":"2024-09-12T03:21:00.807152+0000","last_fullsized":"2024-09-12T03:21:00.807152+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_clean_scrub_stamp":"2024-09-12T03:20:58.769862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:05:44.345724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1e","version":"99'1","reported_seq":15,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-12T03:21:01.015925+0000","last_change":"2024-09-12T03:20:58.791049+0000","last_active":"2024-09-12T03:21:01.015925+0000","last_peered":"2024-09-12T03:21:01.015925+0000","last_clean":"2024-09-12T03:21:01.015925+0000","last_became_active":"2024-09-12T03:20:58.790738+0000","last_became_peered":"2024-09-12T03:20:58.790738+0000","last_unstale":"2024-09-12T03:21:01.015925+0000","last_undegraded":"2024-09-12T03:21:01.015925+0000","last_fullsized":"2024-09-12T03:21:01.015925+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:57.764885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:57.764885+0000","last_clean_scrub_stamp":"2024-09-12T03:20:57.764885+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:50:28.197551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-12T03:21:01.015694+0000","last_change":"2024-09-12T03:20:59.814043+0000","last_active":"2024-09-12T03:21:01.015694+0000","last_peered":"2024-09-12T03:21:01.015694+0000","last_clean":"2024-09-12T03:21:01.015694+0000","last_became_active":"2024-09-12T03:20:59.813786+0000","last_became_peered":"2024-09-12T03:20:59.813786+0000","last_unstale":"2024-09-12T03:21:01.015694+0000","last_undegraded":"2024-09-12T03:21:01.015694+0000","last_fullsized":"2024-09-12T03:21:01.015694+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_clean_scrub_stamp":"2024-09-12T03:20:58.769862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:39:40.342252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1d","version":"99'2","reported_seq":16,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-12T03:21:01.014180+0000","last_change":"2024-09-12T03:20:58.795147+0000","last_active":"2024-09-12T03:21:01.014180+0000","last_peered":"2024-09-12T03:21:01.014180+0000","last_clean":"2024-09-12T03:21:01.014180+0000","last_became_active":"2024-09-12T03:20:58.794857+0000","last_became_peered":"2024-09-12T03:20:58.794857+0000","last_unstale":"2024-09-12T03:21:01.014180+0000","last_undegraded":"2024-09-12T03:21:01.014180+0000","last_fullsized":"2024-09-12T03:21:01.014180+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:57.764885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:57.764885+0000","last_clean_scrub_stamp":"2024-09-12T03:20:57.764885+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:36:49.111778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-12T03:21:01.016133+0000","last_change":"2024-09-12T03:20:59.810539+0000","last_active":"2024-09-12T03:21:01.016133+0000","last_peered":"2024-09-12T03:21:01.016133+0000","last_clean":"2024-09-12T03:21:01.016133+0000","last_became_active":"2024-09-12T03:20:59.810157+0000","last_became_peered":"2024-09-12T03:20:59.810157+0000","last_unstale":"2024-09-12T03:21:01.016133+0000","last_undegraded":"2024-09-12T03:21:01.016133+0000","last_fullsized":"2024-09-12T03:21:01.016133+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_clean_scrub_stamp":"2024-09-12T03:20:58.769862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:09:34.594263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1c","version":"99'2","reported_seq":16,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-12T03:21:01.014129+0000","last_change":"2024-09-12T03:20:58.795764+0000","last_active":"2024-09-12T03:21:01.014129+0000","last_peered":"2024-09-12T03:21:01.014129+0000","last_clean":"2024-09-12T03:21:01.014129+0000","last_became_active":"2024-09-12T03:20:58.795493+0000","last_became_peered":"2024-09-12T03:20:58.795493+0000","last_unstale":"2024-09-12T03:21:01.014129+0000","last_undegraded":"2024-09-12T03:21:01.014129+0000","last_fullsized":"2024-09-12T03:21:01.014129+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:57.764885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:57.764885+0000","last_clean_scrub_stamp":"2024-09-12T03:20:57.764885+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:24:50.315281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-12T03:21:01.014970+0000","last_change":"2024-09-12T03:20:59.822730+0000","last_active":"2024-09-12T03:21:01.014970+0000","last_peered":"2024-09-12T03:21:01.014970+0000","last_clean":"2024-09-12T03:21:01.014970+0000","last_became_active":"2024-09-12T03:20:59.822390+0000","last_became_peered":"2024-09-12T03:20:59.822390+0000","last_unstale":"2024-09-12T03:21:01.014970+0000","last_undegraded":"2024-09-12T03:21:01.014970+0000","last_fullsized":"2024-09-12T03:21:01.014970+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_clean_scrub_stamp":"2024-09-12T03:20:58.769862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:37:40.335331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1b","version":"99'1","reported_seq":15,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-12T03:21:00.800574+0000","last_change":"2024-09-12T03:20:58.785429+0000","last_active":"2024-09-12T03:21:00.800574+0000","last_peered":"2024-09-12T03:21:00.800574+0000","last_clean":"2024-09-12T03:21:00.800574+0000","last_became_active":"2024-09-12T03:20:58.785117+0000","last_became_peered":"2024-09-12T03:20:58.785117+0000","last_unstale":"2024-09-12T03:21:00.800574+0000","last_undegraded":"2024-09-12T03:21:00.800574+0000","last_fullsized":"2024-09-12T03:21:00.800574+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:57.764885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:57.764885+0000","last_clean_scrub_stamp":"2024-09-12T03:20:57.764885+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:42:57.498649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-12T03:21:00.800707+0000","last_change":"2024-09-12T03:20:59.802898+0000","last_active":"2024-09-12T03:21:00.800707+0000","last_peered":"2024-09-12T03:21:00.800707+0000","last_clean":"2024-09-12T03:21:00.800707+0000","last_became_active":"2024-09-12T03:20:59.802618+0000","last_became_peered":"2024-09-12T03:20:59.802618+0000","last_unstale":"2024-09-12T03:21:00.800707+0000","last_undegraded":"2024-09-12T03:21:00.800707+0000","last_fullsized":"2024-09-12T03:21:00.800707+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_clean_scrub_stamp":"2024-09-12T03:20:58.769862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:05:44.345724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1a","version":"99'1","reported_seq":15,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-12T03:21:00.799762+0000","last_change":"2024-09-12T03:20:58.789541+0000","last_active":"2024-09-12T03:21:00.799762+0000","last_peered":"2024-09-12T03:21:00.799762+0000","last_clean":"2024-09-12T03:21:00.799762+0000","last_became_active":"2024-09-12T03:20:58.789287+0000","last_became_peered":"2024-09-12T03:20:58.789287+0000","last_unstale":"2024-09-12T03:21:00.799762+0000","last_undegraded":"2024-09-12T03:21:00.799762+0000","last_fullsized":"2024-09-12T03:21:00.799762+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:57.764885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:57.764885+0000","last_clean_scrub_stamp":"2024-09-12T03:20:57.764885+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:48:02.512942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-12T03:21:01.014207+0000","last_change":"2024-09-12T03:20:59.815523+0000","last_active":"2024-09-12T03:21:01.014207+0000","last_peered":"2024-09-12T03:21:01.014207+0000","last_clean":"2024-09-12T03:21:01.014207+0000","last_became_active":"2024-09-12T03:20:59.815218+0000","last_became_peered":"2024-09-12T03:20:59.815218+0000","last_unstale":"2024-09-12T03:21:01.014207+0000","last_undegraded":"2024-09-12T03:21:01.014207+0000","last_fullsized":"2024-09-12T03:21:01.014207+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_clean_scrub_stamp":"2024-09-12T03:20:58.769862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:32:58.743319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":14,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-12T03:21:00.800617+0000","last_change":"2024-09-12T03:20:58.785470+0000","last_active":"2024-09-12T03:21:00.800617+0000","last_peered":"2024-09-12T03:21:00.800617+0000","last_clean":"2024-09-12T03:21:00.800617+0000","last_became_active":"2024-09-12T03:20:58.784580+0000","last_became_peered":"2024-09-12T03:20:58.784580+0000","last_unstale":"2024-09-12T03:21:00.800617+0000","last_undegraded":"2024-09-12T03:21:00.800617+0000","last_fullsized":"2024-09-12T03:21:00.800617+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:57.764885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:57.764885+0000","last_clean_scrub_stamp":"2024-09-12T03:20:57.764885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:39:08.649257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-12T03:21:00.800217+0000","last_change":"2024-09-12T03:20:59.802879+0000","last_active":"2024-09-12T03:21:00.800217+0000","last_peered":"2024-09-12T03:21:00.800217+0000","last_clean":"2024-09-12T03:21:00.800217+0000","last_became_active":"2024-09-12T03:20:59.802205+0000","last_became_peered":"2024-09-12T03:20:59.802205+0000","last_unstale":"2024-09-12T03:21:00.800217+0000","last_undegraded":"2024-09-12T03:21:00.800217+0000","last_fullsized":"2024-09-12T03:21:00.800217+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_clean_scrub_stamp":"2024-09-12T03:20:58.769862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:06:03.121556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":14,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-12T03:21:01.014239+0000","last_change":"2024-09-12T03:20:58.787702+0000","last_active":"2024-09-12T03:21:01.014239+0000","last_peered":"2024-09-12T03:21:01.014239+0000","last_clean":"2024-09-12T03:21:01.014239+0000","last_became_active":"2024-09-12T03:20:58.787402+0000","last_became_peered":"2024-09-12T03:20:58.787402+0000","last_unstale":"2024-09-12T03:21:01.014239+0000","last_undegraded":"2024-09-12T03:21:01.014239+0000","last_fullsized":"2024-09-12T03:21:01.014239+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:57.764885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:57.764885+0000","last_clean_scrub_stamp":"2024-09-12T03:20:57.764885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:22:16.377445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-12T03:21:00.801150+0000","last_change":"2024-09-12T03:20:59.804631+0000","last_active":"2024-09-12T03:21:00.801150+0000","last_peered":"2024-09-12T03:21:00.801150+0000","last_clean":"2024-09-12T03:21:00.801150+0000","last_became_active":"2024-09-12T03:20:59.804347+0000","last_became_peered":"2024-09-12T03:20:59.804347+0000","last_unstale":"2024-09-12T03:21:00.801150+0000","last_undegraded":"2024-09-12T03:21:00.801150+0000","last_fullsized":"2024-09-12T03:21:00.801150+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_clean_scrub_stamp":"2024-09-12T03:20:58.769862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:34:02.340684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":14,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-12T03:21:01.014396+0000","last_change":"2024-09-12T03:20:58.795794+0000","last_active":"2024-09-12T03:21:01.014396+0000","last_peered":"2024-09-12T03:21:01.014396+0000","last_clean":"2024-09-12T03:21:01.014396+0000","last_became_active":"2024-09-12T03:20:58.795488+0000","last_became_peered":"2024-09-12T03:20:58.795488+0000","last_unstale":"2024-09-12T03:21:01.014396+0000","last_undegraded":"2024-09-12T03:21:01.014396+0000","last_fullsized":"2024-09-12T03:21:01.014396+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:57.764885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:57.764885+0000","last_clean_scrub_stamp":"2024-09-12T03:20:57.764885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:55:59.826603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-12T03:21:00.807213+0000","last_change":"2024-09-12T03:20:59.818970+0000","last_active":"2024-09-12T03:21:00.807213+0000","last_peered":"2024-09-12T03:21:00.807213+0000","last_clean":"2024-09-12T03:21:00.807213+0000","last_became_active":"2024-09-12T03:20:59.818675+0000","last_became_peered":"2024-09-12T03:20:59.818675+0000","last_unstale":"2024-09-12T03:21:00.807213+0000","last_undegraded":"2024-09-12T03:21:00.807213+0000","last_fullsized":"2024-09-12T03:21:00.807213+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_clean_scrub_stamp":"2024-09-12T03:20:58.769862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:47:28.898685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":14,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-12T03:21:00.799540+0000","last_change":"2024-09-12T03:20:58.790159+0000","last_active":"2024-09-12T03:21:00.799540+0000","last_peered":"2024-09-12T03:21:00.799540+0000","last_clean":"2024-09-12T03:21:00.799540+0000","last_became_active":"2024-09-12T03:20:58.789913+0000","last_became_peered":"2024-09-12T03:20:58.789913+0000","last_unstale":"2024-09-12T03:21:00.799540+0000","last_undegraded":"2024-09-12T03:21:00.799540+0000","last_fullsized":"2024-09-12T03:21:00.799540+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:57.764885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:57.764885+0000","last_clean_scrub_stamp":"2024-09-12T03:20:57.764885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:59:23.802632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-12T03:21:00.799571+0000","last_change":"2024-09-12T03:20:59.799033+0000","last_active":"2024-09-12T03:21:00.799571+0000","last_peered":"2024-09-12T03:21:00.799571+0000","last_clean":"2024-09-12T03:21:00.799571+0000","last_became_active":"2024-09-12T03:20:59.798728+0000","last_became_peered":"2024-09-12T03:20:59.798728+0000","last_unstale":"2024-09-12T03:21:00.799571+0000","last_undegraded":"2024-09-12T03:21:00.799571+0000","last_fullsized":"2024-09-12T03:21:00.799571+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_clean_scrub_stamp":"2024-09-12T03:20:58.769862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:27:31.328878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.15","version":"99'3","reported_seq":17,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-12T03:21:01.015364+0000","last_change":"2024-09-12T03:20:58.782723+0000","last_active":"2024-09-12T03:21:01.015364+0000","last_peered":"2024-09-12T03:21:01.015364+0000","last_clean":"2024-09-12T03:21:01.015364+0000","last_became_active":"2024-09-12T03:20:58.782357+0000","last_became_peered":"2024-09-12T03:20:58.782357+0000","last_unstale":"2024-09-12T03:21:01.015364+0000","last_undegraded":"2024-09-12T03:21:01.015364+0000","last_fullsized":"2024-09-12T03:21:01.015364+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:57.764885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:57.764885+0000","last_clean_scrub_stamp":"2024-09-12T03:20:57.764885+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:08:51.994300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-12T03:21:00.800898+0000","last_change":"2024-09-12T03:20:59.807060+0000","last_active":"2024-09-12T03:21:00.800898+0000","last_peered":"2024-09-12T03:21:00.800898+0000","last_clean":"2024-09-12T03:21:00.800898+0000","last_became_active":"2024-09-12T03:20:59.806759+0000","last_became_peered":"2024-09-12T03:20:59.806759+0000","last_unstale":"2024-09-12T03:21:00.800898+0000","last_undegraded":"2024-09-12T03:21:00.800898+0000","last_fullsized":"2024-09-12T03:21:00.800898+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_clean_scrub_stamp":"2024-09-12T03:20:58.769862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:56:40.237221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"99'6","reported_seq":20,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-12T03:21:01.016379+0000","last_change":"2024-09-12T03:20:58.787890+0000","last_active":"2024-09-12T03:21:01.016379+0000","last_peered":"2024-09-12T03:21:01.016379+0000","last_clean":"2024-09-12T03:21:01.016379+0000","last_became_active":"2024-09-12T03:20:58.787589+0000","last_became_peered":"2024-09-12T03:20:58.787589+0000","last_unstale":"2024-09-12T03:21:01.016379+0000","last_undegraded":"2024-09-12T03:21:01.016379+0000","last_fullsized":"2024-09-12T03:21:01.016379+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:57.764885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:57.764885+0000","last_clean_scrub_stamp":"2024-09-12T03:20:57.764885+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:33:43.591218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-12T03:21:00.799672+0000","last_change":"2024-09-12T03:20:59.796560+0000","last_active":"2024-09-12T03:21:00.799672+0000","last_peered":"2024-09-12T03:21:00.799672+0000","last_clean":"2024-09-12T03:21:00.799672+0000","last_became_active":"2024-09-12T03:20:59.796040+0000","last_became_peered":"2024-09-12T03:20:59.796040+0000","last_unstale":"2024-09-12T03:21:00.799672+0000","last_undegraded":"2024-09-12T03:21:00.799672+0000","last_fullsized":"2024-09-12T03:21:00.799672+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_clean_scrub_stamp":"2024-09-12T03:20:58.769862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:09:34.594263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"99'2","reported_seq":16,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-12T03:21:01.015764+0000","last_change":"2024-09-12T03:20:58.783863+0000","last_active":"2024-09-12T03:21:01.015764+0000","last_peered":"2024-09-12T03:21:01.015764+0000","last_clean":"2024-09-12T03:21:01.015764+0000","last_became_active":"2024-09-12T03:20:58.783563+0000","last_became_peered":"2024-09-12T03:20:58.783563+0000","last_unstale":"2024-09-12T03:21:01.015764+0000","last_undegraded":"2024-09-12T03:21:01.015764+0000","last_fullsized":"2024-09-12T03:21:01.015764+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:57.764885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:57.764885+0000","last_clean_scrub_stamp":"2024-09-12T03:20:57.764885+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:09:22.594580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-12T03:21:00.799804+0000","last_change":"2024-09-12T03:20:59.806451+0000","last_active":"2024-09-12T03:21:00.799804+0000","last_peered":"2024-09-12T03:21:00.799804+0000","last_clean":"2024-09-12T03:21:00.799804+0000","last_became_active":"2024-09-12T03:20:59.806161+0000","last_became_peered":"2024-09-12T03:20:59.806161+0000","last_unstale":"2024-09-12T03:21:00.799804+0000","last_undegraded":"2024-09-12T03:21:00.799804+0000","last_fullsized":"2024-09-12T03:21:00.799804+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_clean_scrub_stamp":"2024-09-12T03:20:58.769862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:29:56.581778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":14,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-12T03:21:00.806861+0000","last_change":"2024-09-12T03:20:58.793160+0000","last_active":"2024-09-12T03:21:00.806861+0000","last_peered":"2024-09-12T03:21:00.806861+0000","last_clean":"2024-09-12T03:21:00.806861+0000","last_became_active":"2024-09-12T03:20:58.792865+0000","last_became_peered":"2024-09-12T03:20:58.792865+0000","last_unstale":"2024-09-12T03:21:00.806861+0000","last_undegraded":"2024-09-12T03:21:00.806861+0000","last_fullsized":"2024-09-12T03:21:00.806861+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:57.764885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:57.764885+0000","last_clean_scrub_stamp":"2024-09-12T03:20:57.764885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:50:28.197551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-12T03:21:00.799283+0000","last_change":"2024-09-12T03:20:59.798514+0000","last_active":"2024-09-12T03:21:00.799283+0000","last_peered":"2024-09-12T03:21:00.799283+0000","last_clean":"2024-09-12T03:21:00.799283+0000","last_became_active":"2024-09-12T03:20:59.798213+0000","last_became_peered":"2024-09-12T03:20:59.798213+0000","last_unstale":"2024-09-12T03:21:00.799283+0000","last_undegraded":"2024-09-12T03:21:00.799283+0000","last_fullsized":"2024-09-12T03:21:00.799283+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_clean_scrub_stamp":"2024-09-12T03:20:58.769862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:18:54.207971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"99'8","reported_seq":22,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-12T03:21:01.014573+0000","last_change":"2024-09-12T03:20:58.794537+0000","last_active":"2024-09-12T03:21:01.014573+0000","last_peered":"2024-09-12T03:21:01.014573+0000","last_clean":"2024-09-12T03:21:01.014573+0000","last_became_active":"2024-09-12T03:20:58.794158+0000","last_became_peered":"2024-09-12T03:20:58.794158+0000","last_unstale":"2024-09-12T03:21:01.014573+0000","last_undegraded":"2024-09-12T03:21:01.014573+0000","last_fullsized":"2024-09-12T03:21:01.014573+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:57.764885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:57.764885+0000","last_clean_scrub_stamp":"2024-09-12T03:20:57.764885+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:03:34.509048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-12T03:21:01.014540+0000","last_change":"2024-09-12T03:20:59.817332+0000","last_active":"2024-09-12T03:21:01.014540+0000","last_peered":"2024-09-12T03:21:01.014540+0000","last_clean":"2024-09-12T03:21:01.014540+0000","last_became_active":"2024-09-12T03:20:59.816867+0000","last_became_peered":"2024-09-12T03:20:59.816867+0000","last_unstale":"2024-09-12T03:21:01.014540+0000","last_undegraded":"2024-09-12T03:21:01.014540+0000","last_fullsized":"2024-09-12T03:21:01.014540+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_clean_scrub_stamp":"2024-09-12T03:20:58.769862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:06:33.366216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"99'1","reported_seq":15,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-12T03:21:01.016171+0000","last_change":"2024-09-12T03:20:58.783826+0000","last_active":"2024-09-12T03:21:01.016171+0000","last_peered":"2024-09-12T03:21:01.016171+0000","last_clean":"2024-09-12T03:21:01.016171+0000","last_became_active":"2024-09-12T03:20:58.783545+0000","last_became_peered":"2024-09-12T03:20:58.783545+0000","last_unstale":"2024-09-12T03:21:01.016171+0000","last_undegraded":"2024-09-12T03:21:01.016171+0000","last_fullsized":"2024-09-12T03:21:01.016171+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:57.764885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:57.764885+0000","last_clean_scrub_stamp":"2024-09-12T03:20:57.764885+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:24:04.897701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-12T03:21:01.015501+0000","last_change":"2024-09-12T03:20:59.821031+0000","last_active":"2024-09-12T03:21:01.015501+0000","last_peered":"2024-09-12T03:21:01.015501+0000","last_clean":"2024-09-12T03:21:01.015501+0000","last_became_active":"2024-09-12T03:20:59.820707+0000","last_became_peered":"2024-09-12T03:20:59.820707+0000","last_unstale":"2024-09-12T03:21:01.015501+0000","last_undegraded":"2024-09-12T03:21:01.015501+0000","last_fullsized":"2024-09-12T03:21:01.015501+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_clean_scrub_stamp":"2024-09-12T03:20:58.769862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:42:13.017144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.e","version":"99'2","reported_seq":16,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-12T03:21:01.014519+0000","last_change":"2024-09-12T03:20:58.791853+0000","last_active":"2024-09-12T03:21:01.014519+0000","last_peered":"2024-09-12T03:21:01.014519+0000","last_clean":"2024-09-12T03:21:01.014519+0000","last_became_active":"2024-09-12T03:20:58.791554+0000","last_became_peered":"2024-09-12T03:20:58.791554+0000","last_unstale":"2024-09-12T03:21:01.014519+0000","last_undegraded":"2024-09-12T03:21:01.014519+0000","last_fullsized":"2024-09-12T03:21:01.014519+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:57.764885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:57.764885+0000","last_clean_scrub_stamp":"2024-09-12T03:20:57.764885+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:12:42.114203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-12T03:21:01.015906+0000","last_change":"2024-09-12T03:20:59.826574+0000","last_active":"2024-09-12T03:21:01.015906+0000","last_peered":"2024-09-12T03:21:01.015906+0000","last_clean":"2024-09-12T03:21:01.015906+0000","last_became_active":"2024-09-12T03:20:59.826350+0000","last_became_peered":"2024-09-12T03:20:59.826350+0000","last_unstale":"2024-09-12T03:21:01.015906+0000","last_undegraded":"2024-09-12T03:21:01.015906+0000","last_fullsized":"2024-09-12T03:21:01.015906+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_clean_scrub_stamp":"2024-09-12T03:20:58.769862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:14:46.054149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.d","version":"99'1","reported_seq":15,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-12T03:21:00.806986+0000","last_change":"2024-09-12T03:20:58.791889+0000","last_active":"2024-09-12T03:21:00.806986+0000","last_peered":"2024-09-12T03:21:00.806986+0000","last_clean":"2024-09-12T03:21:00.806986+0000","last_became_active":"2024-09-12T03:20:58.791589+0000","last_became_peered":"2024-09-12T03:20:58.791589+0000","last_unstale":"2024-09-12T03:21:00.806986+0000","last_undegraded":"2024-09-12T03:21:00.806986+0000","last_fullsized":"2024-09-12T03:21:00.806986+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:57.764885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:57.764885+0000","last_clean_scrub_stamp":"2024-09-12T03:20:57.764885+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:59:23.802632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-12T03:21:01.016230+0000","last_change":"2024-09-12T03:20:59.807777+0000","last_active":"2024-09-12T03:21:01.016230+0000","last_peered":"2024-09-12T03:21:01.016230+0000","last_clean":"2024-09-12T03:21:01.016230+0000","last_became_active":"2024-09-12T03:20:59.807351+0000","last_became_peered":"2024-09-12T03:20:59.807351+0000","last_unstale":"2024-09-12T03:21:01.016230+0000","last_undegraded":"2024-09-12T03:21:01.016230+0000","last_fullsized":"2024-09-12T03:21:01.016230+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_clean_scrub_stamp":"2024-09-12T03:20:58.769862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:01:46.395221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":14,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-12T03:21:00.800556+0000","last_change":"2024-09-12T03:20:58.786401+0000","last_active":"2024-09-12T03:21:00.800556+0000","last_peered":"2024-09-12T03:21:00.800556+0000","last_clean":"2024-09-12T03:21:00.800556+0000","last_became_active":"2024-09-12T03:20:58.786052+0000","last_became_peered":"2024-09-12T03:20:58.786052+0000","last_unstale":"2024-09-12T03:21:00.800556+0000","last_undegraded":"2024-09-12T03:21:00.800556+0000","last_fullsized":"2024-09-12T03:21:00.800556+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:57.764885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:57.764885+0000","last_clean_scrub_stamp":"2024-09-12T03:20:57.764885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:42:12.012166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-12T03:21:01.015691+0000","last_change":"2024-09-12T03:20:59.803179+0000","last_active":"2024-09-12T03:21:01.015691+0000","last_peered":"2024-09-12T03:21:01.015691+0000","last_clean":"2024-09-12T03:21:01.015691+0000","last_became_active":"2024-09-12T03:20:59.802868+0000","last_became_peered":"2024-09-12T03:20:59.802868+0000","last_unstale":"2024-09-12T03:21:01.015691+0000","last_undegraded":"2024-09-12T03:21:01.015691+0000","last_fullsized":"2024-09-12T03:21:01.015691+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_clean_scrub_stamp":"2024-09-12T03:20:58.769862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:32:54.597742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"99'2","reported_seq":16,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-12T03:21:01.015853+0000","last_change":"2024-09-12T03:20:58.792805+0000","last_active":"2024-09-12T03:21:01.015853+0000","last_peered":"2024-09-12T03:21:01.015853+0000","last_clean":"2024-09-12T03:21:01.015853+0000","last_became_active":"2024-09-12T03:20:58.792525+0000","last_became_peered":"2024-09-12T03:20:58.792525+0000","last_unstale":"2024-09-12T03:21:01.015853+0000","last_undegraded":"2024-09-12T03:21:01.015853+0000","last_fullsized":"2024-09-12T03:21:01.015853+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:57.764885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:57.764885+0000","last_clean_scrub_stamp":"2024-09-12T03:20:57.764885+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:45:19.481652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-12T03:21:01.015116+0000","last_change":"2024-09-12T03:20:59.819374+0000","last_active":"2024-09-12T03:21:01.015116+0000","last_peered":"2024-09-12T03:21:01.015116+0000","last_clean":"2024-09-12T03:21:01.015116+0000","last_became_active":"2024-09-12T03:20:59.818991+0000","last_became_peered":"2024-09-12T03:20:59.818991+0000","last_unstale":"2024-09-12T03:21:01.015116+0000","last_undegraded":"2024-09-12T03:21:01.015116+0000","last_fullsized":"2024-09-12T03:21:01.015116+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_clean_scrub_stamp":"2024-09-12T03:20:58.769862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:06:24.788751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":14,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-12T03:21:01.013950+0000","last_change":"2024-09-12T03:20:58.795383+0000","last_active":"2024-09-12T03:21:01.013950+0000","last_peered":"2024-09-12T03:21:01.013950+0000","last_clean":"2024-09-12T03:21:01.013950+0000","last_became_active":"2024-09-12T03:20:58.795051+0000","last_became_peered":"2024-09-12T03:20:58.795051+0000","last_unstale":"2024-09-12T03:21:01.013950+0000","last_undegraded":"2024-09-12T03:21:01.013950+0000","last_fullsized":"2024-09-12T03:21:01.013950+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:57.764885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:57.764885+0000","last_clean_scrub_stamp":"2024-09-12T03:20:57.764885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:29:43.836647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-12T03:21:01.015328+0000","last_change":"2024-09-12T03:20:59.805532+0000","last_active":"2024-09-12T03:21:01.015328+0000","last_peered":"2024-09-12T03:21:01.015328+0000","last_clean":"2024-09-12T03:21:01.015328+0000","last_became_active":"2024-09-12T03:20:59.804500+0000","last_became_peered":"2024-09-12T03:20:59.804500+0000","last_unstale":"2024-09-12T03:21:01.015328+0000","last_undegraded":"2024-09-12T03:21:01.015328+0000","last_fullsized":"2024-09-12T03:21:01.015328+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_clean_scrub_stamp":"2024-09-12T03:20:58.769862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:28:50.840481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":14,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-12T03:21:01.016032+0000","last_change":"2024-09-12T03:20:58.793856+0000","last_active":"2024-09-12T03:21:01.016032+0000","last_peered":"2024-09-12T03:21:01.016032+0000","last_clean":"2024-09-12T03:21:01.016032+0000","last_became_active":"2024-09-12T03:20:58.793590+0000","last_became_peered":"2024-09-12T03:20:58.793590+0000","last_unstale":"2024-09-12T03:21:01.016032+0000","last_undegraded":"2024-09-12T03:21:01.016032+0000","last_fullsized":"2024-09-12T03:21:01.016032+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:57.764885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:57.764885+0000","last_clean_scrub_stamp":"2024-09-12T03:20:57.764885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:55:34.361301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-12T03:21:00.799416+0000","last_change":"2024-09-12T03:20:59.817900+0000","last_active":"2024-09-12T03:21:00.799416+0000","last_peered":"2024-09-12T03:21:00.799416+0000","last_clean":"2024-09-12T03:21:00.799416+0000","last_became_active":"2024-09-12T03:20:59.817549+0000","last_became_peered":"2024-09-12T03:20:59.817549+0000","last_unstale":"2024-09-12T03:21:00.799416+0000","last_undegraded":"2024-09-12T03:21:00.799416+0000","last_fullsized":"2024-09-12T03:21:00.799416+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_clean_scrub_stamp":"2024-09-12T03:20:58.769862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:32:54.597742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":14,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-12T03:21:01.014303+0000","last_change":"2024-09-12T03:20:58.790115+0000","last_active":"2024-09-12T03:21:01.014303+0000","last_peered":"2024-09-12T03:21:01.014303+0000","last_clean":"2024-09-12T03:21:01.014303+0000","last_became_active":"2024-09-12T03:20:58.789669+0000","last_became_peered":"2024-09-12T03:20:58.789669+0000","last_unstale":"2024-09-12T03:21:01.014303+0000","last_undegraded":"2024-09-12T03:21:01.014303+0000","last_fullsized":"2024-09-12T03:21:01.014303+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:57.764885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:57.764885+0000","last_clean_scrub_stamp":"2024-09-12T03:20:57.764885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:08:37.714256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-12T03:21:00.807130+0000","last_change":"2024-09-12T03:20:59.816082+0000","last_active":"2024-09-12T03:21:00.807130+0000","last_peered":"2024-09-12T03:21:00.807130+0000","last_clean":"2024-09-12T03:21:00.807130+0000","last_became_active":"2024-09-12T03:20:59.815781+0000","last_became_peered":"2024-09-12T03:20:59.815781+0000","last_unstale":"2024-09-12T03:21:00.807130+0000","last_undegraded":"2024-09-12T03:21:00.807130+0000","last_fullsized":"2024-09-12T03:21:00.807130+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_clean_scrub_stamp":"2024-09-12T03:20:58.769862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:36:44.063303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.7","version":"99'2","reported_seq":16,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-12T03:21:00.799474+0000","last_change":"2024-09-12T03:20:58.793458+0000","last_active":"2024-09-12T03:21:00.799474+0000","last_peered":"2024-09-12T03:21:00.799474+0000","last_clean":"2024-09-12T03:21:00.799474+0000","last_became_active":"2024-09-12T03:20:58.793160+0000","last_became_peered":"2024-09-12T03:20:58.793160+0000","last_unstale":"2024-09-12T03:21:00.799474+0000","last_undegraded":"2024-09-12T03:21:00.799474+0000","last_fullsized":"2024-09-12T03:21:00.799474+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:57.764885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:57.764885+0000","last_clean_scrub_stamp":"2024-09-12T03:20:57.764885+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:50:28.197551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-12T03:21:01.016094+0000","last_change":"2024-09-12T03:20:59.823804+0000","last_active":"2024-09-12T03:21:01.016094+0000","last_peered":"2024-09-12T03:21:01.016094+0000","last_clean":"2024-09-12T03:21:01.016094+0000","last_became_active":"2024-09-12T03:20:59.822379+0000","last_became_peered":"2024-09-12T03:20:59.822379+0000","last_unstale":"2024-09-12T03:21:01.016094+0000","last_undegraded":"2024-09-12T03:21:01.016094+0000","last_fullsized":"2024-09-12T03:21:01.016094+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_clean_scrub_stamp":"2024-09-12T03:20:58.769862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:48:20.080140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.6","version":"100'1","reported_seq":15,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-12T03:21:00.808400+0000","last_change":"2024-09-12T03:20:58.791357+0000","last_active":"2024-09-12T03:21:00.808400+0000","last_peered":"2024-09-12T03:21:00.808400+0000","last_clean":"2024-09-12T03:21:00.808400+0000","last_became_active":"2024-09-12T03:20:58.790915+0000","last_became_peered":"2024-09-12T03:20:58.790915+0000","last_unstale":"2024-09-12T03:21:00.808400+0000","last_undegraded":"2024-09-12T03:21:00.808400+0000","last_fullsized":"2024-09-12T03:21:00.808400+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:57.764885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:57.764885+0000","last_clean_scrub_stamp":"2024-09-12T03:20:57.764885+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:30:52.998555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-12T03:21:01.016296+0000","last_change":"2024-09-12T03:20:59.816598+0000","last_active":"2024-09-12T03:21:01.016296+0000","last_peered":"2024-09-12T03:21:01.016296+0000","last_clean":"2024-09-12T03:21:01.016296+0000","last_became_active":"2024-09-12T03:20:59.815748+0000","last_became_peered":"2024-09-12T03:20:59.815748+0000","last_unstale":"2024-09-12T03:21:01.016296+0000","last_undegraded":"2024-09-12T03:21:01.016296+0000","last_fullsized":"2024-09-12T03:21:01.016296+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_clean_scrub_stamp":"2024-09-12T03:20:58.769862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:07:35.076120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":14,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-12T03:21:01.015450+0000","last_change":"2024-09-12T03:20:58.789754+0000","last_active":"2024-09-12T03:21:01.015450+0000","last_peered":"2024-09-12T03:21:01.015450+0000","last_clean":"2024-09-12T03:21:01.015450+0000","last_became_active":"2024-09-12T03:20:58.789480+0000","last_became_peered":"2024-09-12T03:20:58.789480+0000","last_unstale":"2024-09-12T03:21:01.015450+0000","last_undegraded":"2024-09-12T03:21:01.015450+0000","last_fullsized":"2024-09-12T03:21:01.015450+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:57.764885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:57.764885+0000","last_clean_scrub_stamp":"2024-09-12T03:20:57.764885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:15:30.656163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-12T03:21:00.806976+0000","last_change":"2024-09-12T03:20:59.815751+0000","last_active":"2024-09-12T03:21:00.806976+0000","last_peered":"2024-09-12T03:21:00.806976+0000","last_clean":"2024-09-12T03:21:00.806976+0000","last_became_active":"2024-09-12T03:20:59.815436+0000","last_became_peered":"2024-09-12T03:20:59.815436+0000","last_unstale":"2024-09-12T03:21:00.806976+0000","last_undegraded":"2024-09-12T03:21:00.806976+0000","last_fullsized":"2024-09-12T03:21:00.806976+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_clean_scrub_stamp":"2024-09-12T03:20:58.769862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:54:52.453406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":14,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-12T03:21:00.799670+0000","last_change":"2024-09-12T03:20:58.788558+0000","last_active":"2024-09-12T03:21:00.799670+0000","last_peered":"2024-09-12T03:21:00.799670+0000","last_clean":"2024-09-12T03:21:00.799670+0000","last_became_active":"2024-09-12T03:20:58.788292+0000","last_became_peered":"2024-09-12T03:20:58.788292+0000","last_unstale":"2024-09-12T03:21:00.799670+0000","last_undegraded":"2024-09-12T03:21:00.799670+0000","last_fullsized":"2024-09-12T03:21:00.799670+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:57.764885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:57.764885+0000","last_clean_scrub_stamp":"2024-09-12T03:20:57.764885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:20:43.484861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-12T03:21:00.801042+0000","last_change":"2024-09-12T03:20:59.796569+0000","last_active":"2024-09-12T03:21:00.801042+0000","last_peered":"2024-09-12T03:21:00.801042+0000","last_clean":"2024-09-12T03:21:00.801042+0000","last_became_active":"2024-09-12T03:20:59.796255+0000","last_became_peered":"2024-09-12T03:20:59.796255+0000","last_unstale":"2024-09-12T03:21:00.801042+0000","last_undegraded":"2024-09-12T03:21:00.801042+0000","last_fullsized":"2024-09-12T03:21:00.801042+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_clean_scrub_stamp":"2024-09-12T03:20:58.769862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:47:28.898685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":14,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-12T03:21:00.800819+0000","last_change":"2024-09-12T03:20:58.788555+0000","last_active":"2024-09-12T03:21:00.800819+0000","last_peered":"2024-09-12T03:21:00.800819+0000","last_clean":"2024-09-12T03:21:00.800819+0000","last_became_active":"2024-09-12T03:20:58.785915+0000","last_became_peered":"2024-09-12T03:20:58.785915+0000","last_unstale":"2024-09-12T03:21:00.800819+0000","last_undegraded":"2024-09-12T03:21:00.800819+0000","last_fullsized":"2024-09-12T03:21:00.800819+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:57.764885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:57.764885+0000","last_clean_scrub_stamp":"2024-09-12T03:20:57.764885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:28:02.718330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-12T03:21:00.801235+0000","last_change":"2024-09-12T03:20:59.801009+0000","last_active":"2024-09-12T03:21:00.801235+0000","last_peered":"2024-09-12T03:21:00.801235+0000","last_clean":"2024-09-12T03:21:00.801235+0000","last_became_active":"2024-09-12T03:20:59.800693+0000","last_became_peered":"2024-09-12T03:20:59.800693+0000","last_unstale":"2024-09-12T03:21:00.801235+0000","last_undegraded":"2024-09-12T03:21:00.801235+0000","last_fullsized":"2024-09-12T03:21:00.801235+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_clean_scrub_stamp":"2024-09-12T03:20:58.769862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:28:50.840481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":14,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-12T03:21:00.801035+0000","last_change":"2024-09-12T03:20:58.785215+0000","last_active":"2024-09-12T03:21:00.801035+0000","last_peered":"2024-09-12T03:21:00.801035+0000","last_clean":"2024-09-12T03:21:00.801035+0000","last_became_active":"2024-09-12T03:20:58.784904+0000","last_became_peered":"2024-09-12T03:20:58.784904+0000","last_unstale":"2024-09-12T03:21:00.801035+0000","last_undegraded":"2024-09-12T03:21:00.801035+0000","last_fullsized":"2024-09-12T03:21:00.801035+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:57.764885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:57.764885+0000","last_clean_scrub_stamp":"2024-09-12T03:20:57.764885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:16:31.679549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-12T03:21:01.015434+0000","last_change":"2024-09-12T03:20:59.805632+0000","last_active":"2024-09-12T03:21:01.015434+0000","last_peered":"2024-09-12T03:21:01.015434+0000","last_clean":"2024-09-12T03:21:01.015434+0000","last_became_active":"2024-09-12T03:20:59.805096+0000","last_became_peered":"2024-09-12T03:20:59.805096+0000","last_unstale":"2024-09-12T03:21:01.015434+0000","last_undegraded":"2024-09-12T03:21:01.015434+0000","last_fullsized":"2024-09-12T03:21:01.015434+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_clean_scrub_stamp":"2024-09-12T03:20:58.769862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:56:50.124923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":14,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-12T03:21:01.013912+0000","last_change":"2024-09-12T03:20:58.794661+0000","last_active":"2024-09-12T03:21:01.013912+0000","last_peered":"2024-09-12T03:21:01.013912+0000","last_clean":"2024-09-12T03:21:01.013912+0000","last_became_active":"2024-09-12T03:20:58.794283+0000","last_became_peered":"2024-09-12T03:20:58.794283+0000","last_unstale":"2024-09-12T03:21:01.013912+0000","last_undegraded":"2024-09-12T03:21:01.013912+0000","last_fullsized":"2024-09-12T03:21:01.013912+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:57.764885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:57.764885+0000","last_clean_scrub_stamp":"2024-09-12T03:20:57.764885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:41:58.714733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-12T03:21:01.014766+0000","last_change":"2024-09-12T03:20:59.814345+0000","last_active":"2024-09-12T03:21:01.014766+0000","last_peered":"2024-09-12T03:21:01.014766+0000","last_clean":"2024-09-12T03:21:01.014766+0000","last_became_active":"2024-09-12T03:20:59.813587+0000","last_became_peered":"2024-09-12T03:20:59.813587+0000","last_unstale":"2024-09-12T03:21:01.014766+0000","last_undegraded":"2024-09-12T03:21:01.014766+0000","last_fullsized":"2024-09-12T03:21:01.014766+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_clean_scrub_stamp":"2024-09-12T03:20:58.769862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:13:37.100603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":14,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-12T03:21:01.014329+0000","last_change":"2024-09-12T03:20:58.789932+0000","last_active":"2024-09-12T03:21:01.014329+0000","last_peered":"2024-09-12T03:21:01.014329+0000","last_clean":"2024-09-12T03:21:01.014329+0000","last_became_active":"2024-09-12T03:20:58.789318+0000","last_became_peered":"2024-09-12T03:20:58.789318+0000","last_unstale":"2024-09-12T03:21:01.014329+0000","last_undegraded":"2024-09-12T03:21:01.014329+0000","last_fullsized":"2024-09-12T03:21:01.014329+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:57.764885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:57.764885+0000","last_clean_scrub_stamp":"2024-09-12T03:20:57.764885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:39:31.290244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-12T03:21:00.799333+0000","last_change":"2024-09-12T03:20:59.807209+0000","last_active":"2024-09-12T03:21:00.799333+0000","last_peered":"2024-09-12T03:21:00.799333+0000","last_clean":"2024-09-12T03:21:00.799333+0000","last_became_active":"2024-09-12T03:20:59.806924+0000","last_became_peered":"2024-09-12T03:20:59.806924+0000","last_unstale":"2024-09-12T03:21:00.799333+0000","last_undegraded":"2024-09-12T03:21:00.799333+0000","last_fullsized":"2024-09-12T03:21:00.799333+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:58.769862+0000","last_clean_scrub_stamp":"2024-09-12T03:20:58.769862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:56:50.124923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":14,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-12T03:21:00.800687+0000","last_change":"2024-09-12T03:20:58.785446+0000","last_active":"2024-09-12T03:21:00.800687+0000","last_peered":"2024-09-12T03:21:00.800687+0000","last_clean":"2024-09-12T03:21:00.800687+0000","last_became_active":"2024-09-12T03:20:58.785143+0000","last_became_peered":"2024-09-12T03:20:58.785143+0000","last_unstale":"2024-09-12T03:21:00.800687+0000","last_undegraded":"2024-09-12T03:21:00.800687+0000","last_fullsized":"2024-09-12T03:21:00.800687+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:20:57.764885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:20:57.764885+0000","last_clean_scrub_stamp":"2024-09-12T03:20:57.764885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:39:39.337275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"70'128","reported_seq":355,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-12T03:21:01.015473+0000","last_change":"2024-09-12T03:16:21.883446+0000","last_active":"2024-09-12T03:21:01.015473+0000","last_peered":"2024-09-12T03:21:01.015473+0000","last_clean":"2024-09-12T03:21:01.015473+0000","last_became_active":"2024-09-12T03:16:21.882207+0000","last_became_peered":"2024-09-12T03:16:21.882207+0000","last_unstale":"2024-09-12T03:21:01.015473+0000","last_undegraded":"2024-09-12T03:21:01.015473+0000","last_fullsized":"2024-09-12T03:21:01.015473+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:20.316426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:20.316426+0000","last_clean_scrub_stamp":"2024-09-12T03:16:20.316426+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:12:15.424320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542202,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37144,"kb_used_data":1788,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334696,"statfs":{"total":96636764160,"available":96598728704,"internally_reserved":0,"allocated":1830912,"data_stored":1129929,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542204,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36564,"kb_used_data":1216,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335276,"statfs":{"total":96636764160,"available":96599322624,"internally_reserved":0,"allocated":1245184,"data_stored":545315,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542203,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36556,"kb_used_data":1208,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335284,"statfs":{"total":96636764160,"available":96599330816,"internally_reserved":0,"allocated":1236992,"data_stored":539549,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3],"upper_bound":8},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542202,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36548,"kb_used_data":1200,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335292,"statfs":{"total":96636764160,"available":96599339008,"internally_reserved":0,"allocated":1228800,"data_stored":538945,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542203,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28256,"kb_used_data":1228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343584,"statfs":{"total":96636764160,"available":96607830016,"internally_reserved":0,"allocated":1257472,"data_stored":546927,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542203,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28236,"kb_used_data":1200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343604,"statfs":{"total":96636764160,"available":96607850496,"internally_reserved":0,"allocated":1228800,"data_stored":538945,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542204,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32396,"kb_used_data":1208,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339444,"statfs":{"total":96636764160,"available":96603590656,"internally_reserved":0,"allocated":1236992,"data_stored":539069,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542204,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33008,"kb_used_data":1812,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338832,"statfs":{"total":96636764160,"available":96602963968,"internally_reserved":0,"allocated":1855488,"data_stored":1137337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T03:21:03.133 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-12T03:21:03.134 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-12T03:21:03.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:03.279+0000 7f0b668c9640 1 Processor -- start 2024-09-12T03:21:03.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:03.279+0000 7f0b668c9640 1 -- start start 2024-09-12T03:21:03.281 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:03.280+0000 7f0b668c9640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f0b6015e530 0x7f0b600aff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:03.281 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:03.280+0000 7f0b668c9640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f0b6015e910 0x7f0b600b4710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:03.281 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:03.280+0000 7f0b668c9640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f0b600aeb80 0x7f0b600722f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:03.281 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:03.280+0000 7f0b668c9640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f0b600b07e0 con 0x7f0b6015e910 2024-09-12T03:21:03.281 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:03.280+0000 7f0b668c9640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f0b600b4c50 con 0x7f0b600aeb80 2024-09-12T03:21:03.281 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:03.280+0000 7f0b668c9640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f0b6006dc20 con 0x7f0b6015e530 2024-09-12T03:21:03.281 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:03.280+0000 7f0b5f7fe640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f0b6015e910 0x7f0b600b4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:03.281 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:03.280+0000 7f0b5f7fe640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f0b6015e910 0x7f0b600b4710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38572/0 (socket says 172.21.15.27:38572) 2024-09-12T03:21:03.281 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:03.280+0000 7f0b5f7fe640 1 -- 172.21.15.27:0/973131686 learned_addr learned my addr 172.21.15.27:0/973131686 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:21:03.281 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:03.281+0000 7f0b64e66640 1 --2- 172.21.15.27:0/973131686 >> v2:172.21.15.45:3300/0 conn(0x7f0b600aeb80 0x7f0b600722f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:03.281 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:03.281+0000 7f0b5f7fe640 1 -- 172.21.15.27:0/973131686 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f0b6015e530 msgr2=0x7f0b600aff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:03.282 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:03.281+0000 7f0b5ffff640 1 --2- 172.21.15.27:0/973131686 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f0b6015e530 0x7f0b600aff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:03.282 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:03.281+0000 7f0b5f7fe640 1 --2- 172.21.15.27:0/973131686 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f0b6015e530 0x7f0b600aff20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:03.282 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:03.281+0000 7f0b5f7fe640 1 -- 172.21.15.27:0/973131686 >> v2:172.21.15.45:3300/0 conn(0x7f0b600aeb80 msgr2=0x7f0b600722f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:03.282 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:03.281+0000 7f0b5f7fe640 1 --2- 172.21.15.27:0/973131686 >> v2:172.21.15.45:3300/0 conn(0x7f0b600aeb80 0x7f0b600722f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:03.282 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:03.281+0000 7f0b5f7fe640 1 -- 172.21.15.27:0/973131686 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b600b0500 con 0x7f0b6015e910 2024-09-12T03:21:03.283 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:03.281+0000 7f0b5f7fe640 1 --2- 172.21.15.27:0/973131686 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0b6015e910 0x7f0b600b4710 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f0b600aeaf0 tx=0x7f0b5402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f21cc6aeb61922b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:03.283 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:03.282+0000 7f0b5effd640 1 -- 172.21.15.27:0/973131686 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b54037070 con 0x7f0b6015e910 2024-09-12T03:21:03.283 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:03.282+0000 7f0b5effd640 1 -- 172.21.15.27:0/973131686 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b54032440 con 0x7f0b6015e910 2024-09-12T03:21:03.283 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:03.282+0000 7f0b5effd640 1 -- 172.21.15.27:0/973131686 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b54032810 con 0x7f0b6015e910 2024-09-12T03:21:03.284 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:03.282+0000 7f0b668c9640 1 -- 172.21.15.27:0/973131686 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0b6015e910 msgr2=0x7f0b600b4710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:03.284 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:03.282+0000 7f0b668c9640 1 --2- 172.21.15.27:0/973131686 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0b6015e910 0x7f0b600b4710 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f0b600aeaf0 tx=0x7f0b5402a1e0 comp rx=0 tx=0).stop 2024-09-12T03:21:03.284 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:03.282+0000 7f0b668c9640 1 -- 172.21.15.27:0/973131686 shutdown_connections 2024-09-12T03:21:03.284 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:03.282+0000 7f0b668c9640 1 --2- 172.21.15.27:0/973131686 >> v2:172.21.15.45:3300/0 conn(0x7f0b600aeb80 0x7f0b600722f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:03.284 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:03.282+0000 7f0b668c9640 1 --2- 172.21.15.27:0/973131686 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0b6015e910 0x7f0b600b4710 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:03.284 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:03.282+0000 7f0b668c9640 1 --2- 172.21.15.27:0/973131686 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f0b6015e530 0x7f0b600aff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:03.284 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:03.282+0000 7f0b668c9640 1 -- 172.21.15.27:0/973131686 >> 172.21.15.27:0/973131686 conn(0x7f0b6006bb30 msgr2=0x7f0b6006bf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:03.284 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:03.282+0000 7f0b668c9640 1 -- 172.21.15.27:0/973131686 shutdown_connections 2024-09-12T03:21:03.284 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:03.282+0000 7f0b668c9640 1 -- 172.21.15.27:0/973131686 wait complete. 2024-09-12T03:21:03.284 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:03.283+0000 7f0b668c9640 1 Processor -- start 2024-09-12T03:21:03.284 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:03.284+0000 7f0b668c9640 1 -- start start 2024-09-12T03:21:03.286 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:03.284+0000 7f0b668c9640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f0b600aeb80 0x7f0b601475c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:03.286 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:03.284+0000 7f0b668c9640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0b6015e530 0x7f0b60162a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:03.286 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:03.284+0000 7f0b668c9640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0b6015e910 0x7f0b601548c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:03.286 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:03.284+0000 7f0b668c9640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f0b60071ff0 con 0x7f0b6015e910 2024-09-12T03:21:03.286 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:03.284+0000 7f0b668c9640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f0b6006e1a0 con 0x7f0b6015e530 2024-09-12T03:21:03.287 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:03.284+0000 7f0b668c9640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f0b60072830 con 0x7f0b600aeb80 2024-09-12T03:21:03.287 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:03.285+0000 7f0b64e66640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0b6015e910 0x7f0b601548c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:03.287 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:03.285+0000 7f0b64e66640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0b6015e910 0x7f0b601548c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38588/0 (socket says 172.21.15.27:38588) 2024-09-12T03:21:03.287 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:03.285+0000 7f0b64e66640 1 -- 172.21.15.27:0/3445839322 learned_addr learned my addr 172.21.15.27:0/3445839322 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:21:03.287 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:03.285+0000 7f0b5ffff640 1 --2- 172.21.15.27:0/3445839322 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f0b600aeb80 0x7f0b601475c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:03.287 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:03.285+0000 7f0b5f7fe640 1 --2- 172.21.15.27:0/3445839322 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0b6015e530 0x7f0b60162a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:03.287 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:03.285+0000 7f0b64e66640 1 -- 172.21.15.27:0/3445839322 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f0b600aeb80 msgr2=0x7f0b601475c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:03.287 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:03.285+0000 7f0b64e66640 1 --2- 172.21.15.27:0/3445839322 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f0b600aeb80 0x7f0b601475c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:03.287 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:03.285+0000 7f0b64e66640 1 -- 172.21.15.27:0/3445839322 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0b6015e530 msgr2=0x7f0b60162a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:03.287 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:03.285+0000 7f0b64e66640 1 --2- 172.21.15.27:0/3445839322 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0b6015e530 0x7f0b60162a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:03.287 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:03.285+0000 7f0b64e66640 1 -- 172.21.15.27:0/3445839322 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b601f4660 con 0x7f0b6015e910 2024-09-12T03:21:03.287 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:03.285+0000 7f0b5f7fe640 1 --2- 172.21.15.27:0/3445839322 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0b6015e530 0x7f0b60162a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:21:03.287 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:03.285+0000 7f0b64e66640 1 --2- 172.21.15.27:0/3445839322 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0b6015e910 0x7f0b601548c0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f0b5800cd70 tx=0x7f0b5800a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:03.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:03.285+0000 7f0b5cff9640 1 -- 172.21.15.27:0/3445839322 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b58003b90 con 0x7f0b6015e910 2024-09-12T03:21:03.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:03.285+0000 7f0b5cff9640 1 -- 172.21.15.27:0/3445839322 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b58003d30 con 0x7f0b6015e910 2024-09-12T03:21:03.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:03.285+0000 7f0b668c9640 1 -- 172.21.15.27:0/3445839322 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b6015f870 con 0x7f0b6015e910 2024-09-12T03:21:03.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:03.285+0000 7f0b5cff9640 1 -- 172.21.15.27:0/3445839322 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b58004240 con 0x7f0b6015e910 2024-09-12T03:21:03.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:03.285+0000 7f0b668c9640 1 -- 172.21.15.27:0/3445839322 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b60151ff0 con 0x7f0b6015e910 2024-09-12T03:21:03.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:03.286+0000 7f0b668c9640 1 -- 172.21.15.27:0/3445839322 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b600aff20 con 0x7f0b6015e910 2024-09-12T03:21:03.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:03.287+0000 7f0b5cff9640 1 -- 172.21.15.27:0/3445839322 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 90600+0+0 (secure 0 0 0) 0x7f0b580043e0 con 0x7f0b6015e910 2024-09-12T03:21:03.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:03.287+0000 7f0b5cff9640 1 --2- 172.21.15.27:0/3445839322 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f0b4c06bf50 0x7f0b4c06e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:03.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:03.287+0000 7f0b5cff9640 1 -- 172.21.15.27:0/3445839322 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5709+0+0 (secure 0 0 0) 0x7f0b5808bab0 con 0x7f0b6015e910 2024-09-12T03:21:03.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:03.287+0000 7f0b5ffff640 1 --2- 172.21.15.27:0/3445839322 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f0b4c06bf50 0x7f0b4c06e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:03.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:03.288+0000 7f0b5ffff640 1 --2- 172.21.15.27:0/3445839322 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f0b4c06bf50 0x7f0b4c06e410 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f0b5000aa40 tx=0x7f0b50008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:03.291 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:03.291+0000 7f0b5cff9640 1 -- 172.21.15.27:0/3445839322 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f0b6015f870 con 0x7f0b6015e910 2024-09-12T03:21:03.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:03.530+0000 7f0b668c9640 1 -- 172.21.15.27:0/3445839322 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f0b60068f80 con 0x7f0b6015e910 2024-09-12T03:21:03.577 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:03.533+0000 7f0b5cff9640 1 -- 172.21.15.27:0/3445839322 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v18) ==== 190+0+147 (secure 0 0 0) 0x7f0b5805a2d0 con 0x7f0b6015e910 2024-09-12T03:21:03.578 INFO:teuthology.orchestra.run.smithi027.stdout:[client.0] 2024-09-12T03:21:03.578 INFO:teuthology.orchestra.run.smithi027.stdout: key = AQDVXOJmLm+5ExAAECM4CKfuVElmp/n0b02mXg== 2024-09-12T03:21:03.578 INFO:teuthology.orchestra.run.smithi027.stdout: caps mds = "allow" 2024-09-12T03:21:03.578 INFO:teuthology.orchestra.run.smithi027.stdout: caps mon = "allow r" 2024-09-12T03:21:03.578 INFO:teuthology.orchestra.run.smithi027.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-12T03:21:03.580 INFO:teuthology.orchestra.run.smithi027.stderr:updated caps for client.0 2024-09-12T03:21:03.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:03.535+0000 7f0b668c9640 1 -- 172.21.15.27:0/3445839322 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f0b4c06bf50 msgr2=0x7f0b4c06e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:03.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:03.535+0000 7f0b668c9640 1 --2- 172.21.15.27:0/3445839322 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f0b4c06bf50 0x7f0b4c06e410 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f0b5000aa40 tx=0x7f0b50008040 comp rx=0 tx=0).stop 2024-09-12T03:21:03.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:03.535+0000 7f0b668c9640 1 -- 172.21.15.27:0/3445839322 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0b6015e910 msgr2=0x7f0b601548c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:03.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:03.535+0000 7f0b668c9640 1 --2- 172.21.15.27:0/3445839322 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0b6015e910 0x7f0b601548c0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f0b5800cd70 tx=0x7f0b5800a660 comp rx=0 tx=0).stop 2024-09-12T03:21:03.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:03.535+0000 7f0b668c9640 1 -- 172.21.15.27:0/3445839322 shutdown_connections 2024-09-12T03:21:03.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:03.535+0000 7f0b668c9640 1 --2- 172.21.15.27:0/3445839322 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f0b4c06bf50 0x7f0b4c06e410 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:03.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:03.535+0000 7f0b668c9640 1 --2- 172.21.15.27:0/3445839322 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0b6015e910 0x7f0b601548c0 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:03.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:03.535+0000 7f0b668c9640 1 --2- 172.21.15.27:0/3445839322 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0b6015e530 0x7f0b60162a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:03.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:03.535+0000 7f0b668c9640 1 --2- 172.21.15.27:0/3445839322 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f0b600aeb80 0x7f0b601475c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:03.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:03.535+0000 7f0b668c9640 1 -- 172.21.15.27:0/3445839322 >> 172.21.15.27:0/3445839322 conn(0x7f0b6006bb30 msgr2=0x7f0b600b4de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:03.581 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:03.535+0000 7f0b668c9640 1 -- 172.21.15.27:0/3445839322 shutdown_connections 2024-09-12T03:21:03.581 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:03.535+0000 7f0b668c9640 1 -- 172.21.15.27:0/3445839322 wait complete. 2024-09-12T03:21:03.581 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-12T03:21:03.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:03.684+0000 7fc78b21e640 1 Processor -- start 2024-09-12T03:21:03.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:03.684+0000 7fc78b21e640 1 -- start start 2024-09-12T03:21:03.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:03.685+0000 7fc78b21e640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fc784158cf0 0x7fc7840b55d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:03.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:03.685+0000 7fc78b21e640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc7841590d0 0x7fc784162430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:03.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:03.685+0000 7fc78b21e640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fc78415f600 0x7fc78406bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:03.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:03.685+0000 7fc78b21e640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7fc784162970 con 0x7fc784158cf0 2024-09-12T03:21:03.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:03.685+0000 7fc78b21e640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fc784162c10 con 0x7fc7841590d0 2024-09-12T03:21:03.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:03.685+0000 7fc78b21e640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fc784162ef0 con 0x7fc78415f600 2024-09-12T03:21:03.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:03.685+0000 7fc788fba640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fc784158cf0 0x7fc7840b55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:03.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:03.685+0000 7fc788fba640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fc784158cf0 0x7fc7840b55d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38594/0 (socket says 172.21.15.27:38594) 2024-09-12T03:21:03.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:03.685+0000 7fc788fba640 1 -- 172.21.15.27:0/1595033200 learned_addr learned my addr 172.21.15.27:0/1595033200 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:21:03.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:03.685+0000 7fc7897bb640 1 --2- 172.21.15.27:0/1595033200 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fc78415f600 0x7fc78406bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:03.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:03.685+0000 7fc783fff640 1 --2- 172.21.15.27:0/1595033200 >> v2:172.21.15.45:3300/0 conn(0x7fc7841590d0 0x7fc784162430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:03.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:03.685+0000 7fc788fba640 1 -- 172.21.15.27:0/1595033200 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fc78415f600 msgr2=0x7fc78406bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:03.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:03.685+0000 7fc788fba640 1 --2- 172.21.15.27:0/1595033200 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fc78415f600 0x7fc78406bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:03.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:03.685+0000 7fc788fba640 1 -- 172.21.15.27:0/1595033200 >> v2:172.21.15.45:3300/0 conn(0x7fc7841590d0 msgr2=0x7fc784162430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:03.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:03.685+0000 7fc788fba640 1 --2- 172.21.15.27:0/1595033200 >> v2:172.21.15.45:3300/0 conn(0x7fc7841590d0 0x7fc784162430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:03.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:03.685+0000 7fc788fba640 1 -- 172.21.15.27:0/1595033200 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7841631d0 con 0x7fc784158cf0 2024-09-12T03:21:03.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:03.686+0000 7fc788fba640 1 --2- 172.21.15.27:0/1595033200 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc784158cf0 0x7fc7840b55d0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7fc7840b1210 tx=0x7fc77802a0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dbb87d20eb76118 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:03.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:03.686+0000 7fc7837fe640 1 -- 172.21.15.27:0/1595033200 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc778037070 con 0x7fc784158cf0 2024-09-12T03:21:03.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:03.686+0000 7fc7837fe640 1 -- 172.21.15.27:0/1595033200 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc77802ae00 con 0x7fc784158cf0 2024-09-12T03:21:03.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:03.686+0000 7fc7837fe640 1 -- 172.21.15.27:0/1595033200 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc778032660 con 0x7fc784158cf0 2024-09-12T03:21:03.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:03.687+0000 7fc78b21e640 1 -- 172.21.15.27:0/1595033200 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc784158cf0 msgr2=0x7fc7840b55d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:03.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:03.687+0000 7fc78b21e640 1 --2- 172.21.15.27:0/1595033200 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc784158cf0 0x7fc7840b55d0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7fc7840b1210 tx=0x7fc77802a0b0 comp rx=0 tx=0).stop 2024-09-12T03:21:03.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:03.687+0000 7fc78b21e640 1 -- 172.21.15.27:0/1595033200 shutdown_connections 2024-09-12T03:21:03.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:03.687+0000 7fc78b21e640 1 --2- 172.21.15.27:0/1595033200 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fc78415f600 0x7fc78406bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:03.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:03.687+0000 7fc78b21e640 1 --2- 172.21.15.27:0/1595033200 >> v2:172.21.15.45:3300/0 conn(0x7fc7841590d0 0x7fc784162430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:03.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:03.687+0000 7fc78b21e640 1 --2- 172.21.15.27:0/1595033200 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc784158cf0 0x7fc7840b55d0 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:03.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:03.687+0000 7fc78b21e640 1 -- 172.21.15.27:0/1595033200 >> 172.21.15.27:0/1595033200 conn(0x7fc78406cc40 msgr2=0x7fc78406d050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:03.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:03.687+0000 7fc78b21e640 1 -- 172.21.15.27:0/1595033200 shutdown_connections 2024-09-12T03:21:03.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:03.687+0000 7fc78b21e640 1 -- 172.21.15.27:0/1595033200 wait complete. 2024-09-12T03:21:03.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:03.688+0000 7fc78b21e640 1 Processor -- start 2024-09-12T03:21:03.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:03.688+0000 7fc78b21e640 1 -- start start 2024-09-12T03:21:03.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:03.689+0000 7fc78b21e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc784158cf0 0x7fc78415c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:03.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:03.689+0000 7fc78b21e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc7841590d0 0x7fc784152420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:03.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:03.689+0000 7fc78b21e640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fc78415f600 0x7fc784147020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:03.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:03.689+0000 7fc78b21e640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fc78415fbf0 con 0x7fc784158cf0 2024-09-12T03:21:03.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:03.689+0000 7fc78b21e640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc78415fa70 con 0x7fc7841590d0 2024-09-12T03:21:03.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:03.689+0000 7fc78b21e640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fc78415fd70 con 0x7fc78415f600 2024-09-12T03:21:03.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:03.689+0000 7fc788fba640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc784158cf0 0x7fc78415c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:03.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:03.689+0000 7fc788fba640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc784158cf0 0x7fc78415c300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38608/0 (socket says 172.21.15.27:38608) 2024-09-12T03:21:03.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:03.689+0000 7fc788fba640 1 -- 172.21.15.27:0/3572757902 learned_addr learned my addr 172.21.15.27:0/3572757902 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:21:03.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:03.689+0000 7fc783fff640 1 --2- 172.21.15.27:0/3572757902 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc7841590d0 0x7fc784152420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:03.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:03.689+0000 7fc7897bb640 1 --2- 172.21.15.27:0/3572757902 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fc78415f600 0x7fc784147020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:03.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:03.689+0000 7fc788fba640 1 -- 172.21.15.27:0/3572757902 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fc78415f600 msgr2=0x7fc784147020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:03.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:03.689+0000 7fc788fba640 1 --2- 172.21.15.27:0/3572757902 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fc78415f600 0x7fc784147020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:03.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:03.689+0000 7fc788fba640 1 -- 172.21.15.27:0/3572757902 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc7841590d0 msgr2=0x7fc784152420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:03.693 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:03.689+0000 7fc788fba640 1 --2- 172.21.15.27:0/3572757902 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc7841590d0 0x7fc784152420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:03.693 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:03.689+0000 7fc788fba640 1 -- 172.21.15.27:0/3572757902 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7841595a0 con 0x7fc784158cf0 2024-09-12T03:21:03.693 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:03.690+0000 7fc788fba640 1 --2- 172.21.15.27:0/3572757902 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc784158cf0 0x7fc78415c300 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7fc778000c00 tx=0x7fc7780046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:03.693 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:03.690+0000 7fc7817fa640 1 -- 172.21.15.27:0/3572757902 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc778041030 con 0x7fc784158cf0 2024-09-12T03:21:03.693 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:03.690+0000 7fc7817fa640 1 -- 172.21.15.27:0/3572757902 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc77802ac00 con 0x7fc784158cf0 2024-09-12T03:21:03.693 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:03.690+0000 7fc7817fa640 1 -- 172.21.15.27:0/3572757902 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc778037040 con 0x7fc784158cf0 2024-09-12T03:21:03.693 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:03.690+0000 7fc78b21e640 1 -- 172.21.15.27:0/3572757902 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc784147560 con 0x7fc784158cf0 2024-09-12T03:21:03.693 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:03.690+0000 7fc78b21e640 1 -- 172.21.15.27:0/3572757902 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc784152e50 con 0x7fc784158cf0 2024-09-12T03:21:03.693 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:03.691+0000 7fc78b21e640 1 -- 172.21.15.27:0/3572757902 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7840b55d0 con 0x7fc784158cf0 2024-09-12T03:21:03.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:03.691+0000 7fc7817fa640 1 -- 172.21.15.27:0/3572757902 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 90600+0+0 (secure 0 0 0) 0x7fc778044070 con 0x7fc784158cf0 2024-09-12T03:21:03.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:03.692+0000 7fc7817fa640 1 --2- 172.21.15.27:0/3572757902 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fc76806bf50 0x7fc76806e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:03.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:03.692+0000 7fc7817fa640 1 -- 172.21.15.27:0/3572757902 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5709+0+0 (secure 0 0 0) 0x7fc7780ab260 con 0x7fc784158cf0 2024-09-12T03:21:03.695 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:03.692+0000 7fc783fff640 1 --2- 172.21.15.27:0/3572757902 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fc76806bf50 0x7fc76806e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:03.695 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:03.692+0000 7fc783fff640 1 --2- 172.21.15.27:0/3572757902 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fc76806bf50 0x7fc76806e410 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fc784157100 tx=0x7fc770008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:03.696 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:03.696+0000 7fc7817fa640 1 -- 172.21.15.27:0/3572757902 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fc7780794c0 con 0x7fc784158cf0 2024-09-12T03:21:03.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:03.913+0000 7fc78b21e640 1 -- 172.21.15.27:0/3572757902 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc78415d370 con 0x7fc784158cf0 2024-09-12T03:21:03.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:03.914+0000 7fc7817fa640 1 -- 172.21.15.27:0/3572757902 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) ==== 94+0+5222 (secure 0 0 0) 0x7fc77807d7c0 con 0x7fc784158cf0 2024-09-12T03:21:03.914 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:21:03.914 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":58,"btime":"2024-09-12T03:21:01:802409+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5885,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/2343815091","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2343815091},{"type":"v1","addr":"172.21.15.45:6834","nonce":2343815091}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":55},{"gid":6376,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/512483568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":512483568},{"type":"v1","addr":"172.21.15.45:6835","nonce":512483568}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":55},{"gid":6825,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6834/1888895227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":1888895227},{"type":"v1","addr":"172.21.15.27:6834","nonce":1888895227}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":54}],"filesystems":[{"mdsmap":{"epoch":58,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-12T03:20:59.133824+0000","modified":"2024-09-12T03:21:01.148633+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6828},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6828":{"gid":6828,"name":"a","rank":0,"incarnation":56,"state":"up:active","state_seq":3,"addr":"172.21.15.27:6835/1379166526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":1379166526},{"type":"v1","addr":"172.21.15.27:6835","nonce":1379166526}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6828,"qdb_cluster":[6828]},"id":8}]} 2024-09-12T03:21:03.914 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 58 2024-09-12T03:21:03.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:03.916+0000 7fc78b21e640 1 -- 172.21.15.27:0/3572757902 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fc76806bf50 msgr2=0x7fc76806e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:03.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:03.916+0000 7fc78b21e640 1 --2- 172.21.15.27:0/3572757902 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fc76806bf50 0x7fc76806e410 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fc784157100 tx=0x7fc770008040 comp rx=0 tx=0).stop 2024-09-12T03:21:03.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:03.916+0000 7fc78b21e640 1 -- 172.21.15.27:0/3572757902 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc784158cf0 msgr2=0x7fc78415c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:03.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:03.916+0000 7fc78b21e640 1 --2- 172.21.15.27:0/3572757902 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc784158cf0 0x7fc78415c300 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7fc778000c00 tx=0x7fc7780046c0 comp rx=0 tx=0).stop 2024-09-12T03:21:03.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:03.916+0000 7fc78b21e640 1 -- 172.21.15.27:0/3572757902 shutdown_connections 2024-09-12T03:21:03.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:03.916+0000 7fc78b21e640 1 --2- 172.21.15.27:0/3572757902 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fc78415f600 0x7fc784147020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:03.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:03.916+0000 7fc78b21e640 1 --2- 172.21.15.27:0/3572757902 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc7841590d0 0x7fc784152420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:03.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:03.916+0000 7fc78b21e640 1 --2- 172.21.15.27:0/3572757902 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fc76806bf50 0x7fc76806e410 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:03.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:03.916+0000 7fc78b21e640 1 --2- 172.21.15.27:0/3572757902 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc784158cf0 0x7fc78415c300 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:03.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:03.916+0000 7fc78b21e640 1 -- 172.21.15.27:0/3572757902 >> 172.21.15.27:0/3572757902 conn(0x7fc78406cc40 msgr2=0x7fc784157930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:03.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:03.916+0000 7fc78b21e640 1 -- 172.21.15.27:0/3572757902 shutdown_connections 2024-09-12T03:21:03.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:03.916+0000 7fc78b21e640 1 -- 172.21.15.27:0/3572757902 wait complete. 2024-09-12T03:21:04.330 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 58, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-12T03:20:59.133824+0000', 'modified': '2024-09-12T03:21:01.148633+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6828}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6828': {'gid': 6828, 'name': 'a', 'rank': 0, 'incarnation': 56, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.27:6835/1379166526', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6833', 'nonce': 1379166526}, {'type': 'v1', 'addr': '172.21.15.27:6835', 'nonce': 1379166526}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6828, 'qdb_cluster': [6828]} 2024-09-12T03:21:04.330 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-12T03:21:04.330 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-09-12T03:21:04.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.436+0000 7f38ae5d6640 1 Processor -- start 2024-09-12T03:21:04.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.436+0000 7f38ae5d6640 1 -- start start 2024-09-12T03:21:04.437 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.437+0000 7f38ae5d6640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f38a80b6470 0x7f38a80d2710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:04.437 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.437+0000 7f38ae5d6640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f38a80b6850 0x7f38a80d2c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:04.437 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.437+0000 7f38ae5d6640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f38a80d1ec0 con 0x7f38a80d3190 2024-09-12T03:21:04.437 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.437+0000 7f38ae5d6640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f38a80cf980 con 0x7f38a80b6850 2024-09-12T03:21:04.437 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.437+0000 7f38ae5d6640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f38a80b79b0 con 0x7f38a80b6470 2024-09-12T03:21:04.438 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.437+0000 7f38a77fe640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f38a80b6850 0x7f38a80d2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:04.438 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.437+0000 7f38a7fff640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f38a80b6470 0x7f38a80d2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:04.438 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.437+0000 7f38a77fe640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f38a80b6850 0x7f38a80d2c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.27:42184/0 (socket says 172.21.15.27:42184) 2024-09-12T03:21:04.438 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.437+0000 7f38acb73640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f38a80d3190 0x7f38a8161b80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.27:36338/0 (socket says 172.21.15.27:36338) 2024-09-12T03:21:04.438 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.437+0000 7f38a77fe640 1 -- 172.21.15.27:0/3842337038 learned_addr learned my addr 172.21.15.27:0/3842337038 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:21:04.438 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.437+0000 7f38a7fff640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f38a80b6470 0x7f38a80d2710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3301/0 says I am v2:172.21.15.27:60806/0 (socket says 172.21.15.27:60806) 2024-09-12T03:21:04.438 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.438+0000 7f38a7fff640 1 -- 172.21.15.27:0/3842337038 >> v2:172.21.15.45:3300/0 conn(0x7f38a80b6850 msgr2=0x7f38a80d2c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:04.438 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.438+0000 7f38a7fff640 1 --2- 172.21.15.27:0/3842337038 >> v2:172.21.15.45:3300/0 conn(0x7f38a80b6850 0x7f38a80d2c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:04.438 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.438+0000 7f38a7fff640 1 -- 172.21.15.27:0/3842337038 >> v1:172.21.15.45:6789/0 conn(0x7f38a80d3190 legacy=0x7f38a8161b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:04.439 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.438+0000 7f38a7fff640 1 -- 172.21.15.27:0/3842337038 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38a80cb580 con 0x7f38a80b6470 2024-09-12T03:21:04.440 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.439+0000 7f38a7fff640 1 --2- 172.21.15.27:0/3842337038 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f38a80b6470 0x7f38a80d2710 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f38980097a0 tx=0x7f389802a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=657868c6f26dceda server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:04.440 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.439+0000 7f38a6ffd640 1 -- 172.21.15.27:0/3842337038 <== mon.2 v2:172.21.15.45:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3898037070 con 0x7f38a80b6470 2024-09-12T03:21:04.440 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.439+0000 7f38a6ffd640 1 -- 172.21.15.27:0/3842337038 <== mon.2 v2:172.21.15.45:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3898032440 con 0x7f38a80b6470 2024-09-12T03:21:04.440 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.439+0000 7f38a6ffd640 1 -- 172.21.15.27:0/3842337038 <== mon.2 v2:172.21.15.45:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3898032810 con 0x7f38a80b6470 2024-09-12T03:21:04.440 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.439+0000 7f38ae5d6640 1 -- 172.21.15.27:0/3842337038 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f38a80b6470 msgr2=0x7f38a80d2710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:04.440 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.439+0000 7f38ae5d6640 1 --2- 172.21.15.27:0/3842337038 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f38a80b6470 0x7f38a80d2710 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f38980097a0 tx=0x7f389802a180 comp rx=0 tx=0).stop 2024-09-12T03:21:04.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.439+0000 7f38ae5d6640 1 -- 172.21.15.27:0/3842337038 shutdown_connections 2024-09-12T03:21:04.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.439+0000 7f38ae5d6640 1 --2- 172.21.15.27:0/3842337038 >> v2:172.21.15.45:3300/0 conn(0x7f38a80b6850 0x7f38a80d2c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:04.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.439+0000 7f38ae5d6640 1 --2- 172.21.15.27:0/3842337038 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f38a80b6470 0x7f38a80d2710 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:04.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.439+0000 7f38ae5d6640 1 -- 172.21.15.27:0/3842337038 >> 172.21.15.27:0/3842337038 conn(0x7f38a80d1200 msgr2=0x7f38a80d1610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:04.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.439+0000 7f38ae5d6640 1 -- 172.21.15.27:0/3842337038 shutdown_connections 2024-09-12T03:21:04.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.439+0000 7f38ae5d6640 1 -- 172.21.15.27:0/3842337038 wait complete. 2024-09-12T03:21:04.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.441+0000 7f38ae5d6640 1 Processor -- start 2024-09-12T03:21:04.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.441+0000 7f38ae5d6640 1 -- start start 2024-09-12T03:21:04.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.442+0000 7f38ae5d6640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f38a80b6470 0x7f38a815db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:04.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.442+0000 7f38ae5d6640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f38a80b6850 0x7f38a80ae110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:04.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.442+0000 7f38ae5d6640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f38a80d3190 0x7f38a80ae650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:04.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.442+0000 7f38ae5d6640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f38a806d9d0 con 0x7f38a80b6850 2024-09-12T03:21:04.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.442+0000 7f38ae5d6640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f38a806d850 con 0x7f38a80d3190 2024-09-12T03:21:04.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.442+0000 7f38ae5d6640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f38a80d3fa0 con 0x7f38a80b6470 2024-09-12T03:21:04.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.442+0000 7f38a77fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f38a80b6850 0x7f38a80ae110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:04.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.442+0000 7f38a77fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f38a80b6850 0x7f38a80ae110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38614/0 (socket says 172.21.15.27:38614) 2024-09-12T03:21:04.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.442+0000 7f38a77fe640 1 -- 172.21.15.27:0/2040557241 learned_addr learned my addr 172.21.15.27:0/2040557241 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:21:04.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.442+0000 7f38a7fff640 1 --2- 172.21.15.27:0/2040557241 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f38a80b6470 0x7f38a815db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:04.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.442+0000 7f38acb73640 1 --2- 172.21.15.27:0/2040557241 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f38a80d3190 0x7f38a80ae650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:04.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.442+0000 7f38a77fe640 1 -- 172.21.15.27:0/2040557241 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f38a80b6470 msgr2=0x7f38a815db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:04.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.442+0000 7f38a77fe640 1 --2- 172.21.15.27:0/2040557241 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f38a80b6470 0x7f38a815db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:04.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.442+0000 7f38a77fe640 1 -- 172.21.15.27:0/2040557241 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f38a80d3190 msgr2=0x7f38a80ae650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:04.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.442+0000 7f38a77fe640 1 --2- 172.21.15.27:0/2040557241 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f38a80d3190 0x7f38a80ae650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:04.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.442+0000 7f38a77fe640 1 -- 172.21.15.27:0/2040557241 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38a815e620 con 0x7f38a80b6850 2024-09-12T03:21:04.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.442+0000 7f38a77fe640 1 --2- 172.21.15.27:0/2040557241 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f38a80b6850 0x7f38a80ae110 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f389000e870 tx=0x7f389000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:04.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.442+0000 7f38a4ff9640 1 -- 172.21.15.27:0/2040557241 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3890017070 con 0x7f38a80b6850 2024-09-12T03:21:04.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.442+0000 7f38a4ff9640 1 -- 172.21.15.27:0/2040557241 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f389000c8e0 con 0x7f38a80b6850 2024-09-12T03:21:04.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.443+0000 7f38ae5d6640 1 -- 172.21.15.27:0/2040557241 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38a80ab800 con 0x7f38a80b6850 2024-09-12T03:21:04.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.443+0000 7f38a4ff9640 1 -- 172.21.15.27:0/2040557241 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f389000cc90 con 0x7f38a80b6850 2024-09-12T03:21:04.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.443+0000 7f38ae5d6640 1 -- 172.21.15.27:0/2040557241 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38a80abdc0 con 0x7f38a80b6850 2024-09-12T03:21:04.446 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.444+0000 7f38a4ff9640 1 -- 172.21.15.27:0/2040557241 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 90600+0+0 (secure 0 0 0) 0x7f389001a070 con 0x7f38a80b6850 2024-09-12T03:21:04.446 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.445+0000 7f38a4ff9640 1 --2- 172.21.15.27:0/2040557241 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f388806bf80 0x7f388806e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:04.446 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.445+0000 7f38a4ff9640 1 -- 172.21.15.27:0/2040557241 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5709+0+0 (secure 0 0 0) 0x7f389008adc0 con 0x7f38a80b6850 2024-09-12T03:21:04.446 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.445+0000 7f38a7fff640 1 --2- 172.21.15.27:0/2040557241 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f388806bf80 0x7f388806e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:04.446 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.445+0000 7f38a7fff640 1 --2- 172.21.15.27:0/2040557241 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f388806bf80 0x7f388806e440 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f3898034000 tx=0x7f389802e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:04.446 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.446+0000 7f38b1453740 1 Processor -- start 2024-09-12T03:21:04.446 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.446+0000 7f38b1453740 1 -- start start 2024-09-12T03:21:04.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.446+0000 7f38b1453740 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x561618e2b2c0 0x561618e70a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:04.449 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.447+0000 7f38b1453740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561618e28a80 0x561618e2b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:04.449 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.447+0000 7f38b1453740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561618e2ca50 0x561618e258b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:04.449 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.447+0000 7f38b1453740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x561618d8d520 con 0x561618e2ca50 2024-09-12T03:21:04.449 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.447+0000 7f38b1453740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x561618d353e0 con 0x561618e28a80 2024-09-12T03:21:04.449 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.447+0000 7f38b1453740 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x561618d894e0 con 0x561618e2b2c0 2024-09-12T03:21:04.449 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.447+0000 7f38acb73640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561618e2ca50 0x561618e258b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:04.449 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.447+0000 7f38acb73640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561618e2ca50 0x561618e258b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38622/0 (socket says 172.21.15.27:38622) 2024-09-12T03:21:04.449 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.447+0000 7f38acb73640 1 -- 172.21.15.27:0/4161244326 learned_addr learned my addr 172.21.15.27:0/4161244326 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:21:04.449 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.447+0000 7f38a7fff640 1 --2- 172.21.15.27:0/4161244326 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x561618e2b2c0 0x561618e70a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:04.449 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.447+0000 7f38a77fe640 1 --2- 172.21.15.27:0/4161244326 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561618e28a80 0x561618e2b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:04.449 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.447+0000 7f38acb73640 1 -- 172.21.15.27:0/4161244326 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x561618e2b2c0 msgr2=0x561618e70a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:04.449 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.447+0000 7f38acb73640 1 --2- 172.21.15.27:0/4161244326 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x561618e2b2c0 0x561618e70a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:04.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.447+0000 7f38acb73640 1 -- 172.21.15.27:0/4161244326 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561618e28a80 msgr2=0x561618e2b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:04.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.447+0000 7f38acb73640 1 --2- 172.21.15.27:0/4161244326 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561618e28a80 0x561618e2b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:04.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.447+0000 7f38acb73640 1 -- 172.21.15.27:0/4161244326 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561618d502f0 con 0x561618e2ca50 2024-09-12T03:21:04.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.447+0000 7f38a77fe640 1 --2- 172.21.15.27:0/4161244326 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561618e28a80 0x561618e2b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:21:04.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.447+0000 7f38acb73640 1 --2- 172.21.15.27:0/4161244326 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561618e2ca50 0x561618e258b0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f38a0001cd0 tx=0x7f38a0002190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e8b0fb35d3dc225 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:04.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.447+0000 7f389dffb640 1 -- 172.21.15.27:0/4161244326 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38a0019070 con 0x561618e2ca50 2024-09-12T03:21:04.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.447+0000 7f389dffb640 1 -- 172.21.15.27:0/4161244326 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38a00078b0 con 0x561618e2ca50 2024-09-12T03:21:04.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.447+0000 7f389dffb640 1 -- 172.21.15.27:0/4161244326 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38a0014650 con 0x561618e2ca50 2024-09-12T03:21:04.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.448+0000 7f38b1453740 1 -- 172.21.15.27:0/4161244326 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561618e2ca50 msgr2=0x561618e258b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:04.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.448+0000 7f38b1453740 1 --2- 172.21.15.27:0/4161244326 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561618e2ca50 0x561618e258b0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f38a0001cd0 tx=0x7f38a0002190 comp rx=0 tx=0).stop 2024-09-12T03:21:04.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.448+0000 7f38b1453740 1 -- 172.21.15.27:0/4161244326 shutdown_connections 2024-09-12T03:21:04.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.448+0000 7f38b1453740 1 --2- 172.21.15.27:0/4161244326 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561618e2ca50 0x561618e258b0 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:04.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.448+0000 7f38b1453740 1 --2- 172.21.15.27:0/4161244326 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561618e28a80 0x561618e2b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:04.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.448+0000 7f38b1453740 1 --2- 172.21.15.27:0/4161244326 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x561618e2b2c0 0x561618e70a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:04.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.448+0000 7f38b1453740 1 -- 172.21.15.27:0/4161244326 >> 172.21.15.27:0/4161244326 conn(0x561618e27a80 msgr2=0x561618e6ff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:04.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.448+0000 7f38b1453740 1 -- 172.21.15.27:0/4161244326 shutdown_connections 2024-09-12T03:21:04.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.448+0000 7f38b1453740 1 -- 172.21.15.27:0/4161244326 wait complete. 2024-09-12T03:21:04.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.448+0000 7f38b1453740 1 -- start start 2024-09-12T03:21:04.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.449+0000 7f38b1453740 1 Processor -- start 2024-09-12T03:21:04.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.449+0000 7f38b1453740 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x561618e2b2c0 0x561618e55930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:04.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.449+0000 7f38b1453740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561618e2ca50 0x561618e4fd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:04.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.449+0000 7f38b1453740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561618e50770 0x561618e83160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:04.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.449+0000 7f38b1453740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x561618e26f20 con 0x561618e50770 2024-09-12T03:21:04.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.449+0000 7f38b1453740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x561618d16450 con 0x561618e2ca50 2024-09-12T03:21:04.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.449+0000 7f38b1453740 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x561618e71290 con 0x561618e2b2c0 2024-09-12T03:21:04.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.450+0000 7f38b1453740 10 client.0 _ll_register_callbacks cb 0x561618a9fac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:21:04.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.450+0000 7f38acb73640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561618e50770 0x561618e83160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:04.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.450+0000 7f38acb73640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561618e50770 0x561618e83160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38624/0 (socket says 172.21.15.27:38624) 2024-09-12T03:21:04.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.450+0000 7f38acb73640 1 -- 172.21.15.27:0/361364031 learned_addr learned my addr 172.21.15.27:0/361364031 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:21:04.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.450+0000 7f38a7fff640 1 --2- 172.21.15.27:0/361364031 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x561618e2b2c0 0x561618e55930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:04.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.450+0000 7f38a77fe640 1 --2- 172.21.15.27:0/361364031 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561618e2ca50 0x561618e4fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:04.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.450+0000 7f38acb73640 1 -- 172.21.15.27:0/361364031 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x561618e2b2c0 msgr2=0x561618e55930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:04.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.450+0000 7f38acb73640 1 --2- 172.21.15.27:0/361364031 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x561618e2b2c0 0x561618e55930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:04.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.450+0000 7f38acb73640 1 -- 172.21.15.27:0/361364031 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561618e2ca50 msgr2=0x561618e4fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:04.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.450+0000 7f38acb73640 1 --2- 172.21.15.27:0/361364031 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561618e2ca50 0x561618e4fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:04.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.450+0000 7f38acb73640 1 -- 172.21.15.27:0/361364031 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561618aa02d0 con 0x561618e50770 2024-09-12T03:21:04.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.450+0000 7f38acb73640 1 --2- 172.21.15.27:0/361364031 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561618e50770 0x561618e83160 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f38a00012e0 tx=0x7f38a0001250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3afe850acba7d5fe server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:04.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.450+0000 7f3886ffd640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-12T03:21:04.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.450+0000 7f3886ffd640 1 -- 172.21.15.27:0/361364031 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38a0023030 con 0x561618e50770 2024-09-12T03:21:04.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.450+0000 7f3886ffd640 1 -- 172.21.15.27:0/361364031 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 5709+0+0 (secure 0 0 0) 0x7f38a0015bd0 con 0x561618e50770 2024-09-12T03:21:04.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.450+0000 7f38b1453740 1 -- 172.21.15.27:0/361364031 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f38a00078b0 con 0x561618e50770 2024-09-12T03:21:04.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.450+0000 7f3886ffd640 1 -- 172.21.15.27:0/361364031 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f38a002c050 con 0x561618e50770 2024-09-12T03:21:04.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.450+0000 7f38b1453740 10 client.6936 fetch_fsmap learned FSMap version 58 2024-09-12T03:21:04.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.450+0000 7f38b1453740 1 -- 172.21.15.27:0/361364031 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=2+}) -- 0x561618b6a6b0 con 0x561618e50770 2024-09-12T03:21:04.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.451+0000 7f3886ffd640 1 -- 172.21.15.27:0/361364031 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38a00078b0 con 0x561618e50770 2024-09-12T03:21:04.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.451+0000 7f3886ffd640 1 -- 172.21.15.27:0/361364031 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 58) ==== 3502+0+0 (secure 0 0 0) 0x7f38a002f020 con 0x561618e50770 2024-09-12T03:21:04.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.451+0000 7f38b1453740 10 client.6936 fetch_fsmap finished waiting for FSMap version 58 2024-09-12T03:21:04.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.451+0000 7f38b1453740 10 client.6936 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-12T03:21:04.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.451+0000 7f38b1453740 20 client.6936 populate_metadata read hostname 'smithi027' 2024-09-12T03:21:04.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.451+0000 7f38b1453740 1 --2- 172.21.15.27:0/361364031 >> [v2:172.21.15.27:6833/1379166526,v1:172.21.15.27:6835/1379166526] conn(0x561618e8b420 0x561618e8d810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:04.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.451+0000 7f38b1453740 4 client.6936 mds_command: new command op to 6828 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:21:04.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.451+0000 7f38b1453740 1 -- 172.21.15.27:0/361364031 --> [v2:172.21.15.27:6833/1379166526,v1:172.21.15.27:6835/1379166526] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561618d9ac50 con 0x561618e8b420 2024-09-12T03:21:04.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.451+0000 7f38a7fff640 1 --2- 172.21.15.27:0/361364031 >> [v2:172.21.15.27:6833/1379166526,v1:172.21.15.27:6835/1379166526] conn(0x561618e8b420 0x561618e8d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:04.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.451+0000 7f3886ffd640 10 client.6936 ms_handle_connect on v2:172.21.15.27:6833/1379166526 2024-09-12T03:21:04.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.451+0000 7f38a7fff640 1 --2- 172.21.15.27:0/361364031 >> [v2:172.21.15.27:6833/1379166526,v1:172.21.15.27:6835/1379166526] conn(0x561618e8b420 0x561618e8d810 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=96f299adc6c6dcad server_cookie=33f4cbbac9044da6 in_seq=0 out_seq=0 2024-09-12T03:21:04.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.452+0000 7f3886ffd640 1 -- 172.21.15.27:0/361364031 <== mds.0 v2:172.21.15.27:6833/1379166526 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561618d9ac50 con 0x561618e8b420 2024-09-12T03:21:04.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.452+0000 7f3886ffd640 10 client.6936 handle_command_reply: tid=0 2024-09-12T03:21:04.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.452+0000 7f38b1453740 1 client.6936 shutdown 2024-09-12T03:21:04.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.452+0000 7f38b1453740 1 -- 172.21.15.27:0/361364031 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561618e50770 msgr2=0x561618e83160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:04.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.452+0000 7f38b1453740 1 --2- 172.21.15.27:0/361364031 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561618e50770 0x561618e83160 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f38a00012e0 tx=0x7f38a0001250 comp rx=0 tx=0).stop 2024-09-12T03:21:04.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.452+0000 7f38b1453740 1 -- 172.21.15.27:0/361364031 shutdown_connections 2024-09-12T03:21:04.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.452+0000 7f38b1453740 1 --2- 172.21.15.27:0/361364031 >> [v2:172.21.15.27:6833/1379166526,v1:172.21.15.27:6835/1379166526] conn(0x561618e8b420 0x561618e8d810 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:04.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.452+0000 7f38b1453740 1 --2- 172.21.15.27:0/361364031 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561618e50770 0x561618e83160 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:04.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.452+0000 7f38b1453740 1 --2- 172.21.15.27:0/361364031 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561618e2ca50 0x561618e4fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:04.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.452+0000 7f38b1453740 1 --2- 172.21.15.27:0/361364031 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x561618e2b2c0 0x561618e55930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:04.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.452+0000 7f3886ffd640 0 client.6936 ms_handle_reset on v2:172.21.15.27:6833/1379166526 2024-09-12T03:21:04.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.452+0000 7f38b1453740 1 -- 172.21.15.27:0/361364031 >> 172.21.15.27:0/361364031 conn(0x561618e28a80 msgr2=0x561618e6dfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:04.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.452+0000 7f38b1453740 1 -- 172.21.15.27:0/361364031 shutdown_connections 2024-09-12T03:21:04.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.452+0000 7f38b1453740 1 -- 172.21.15.27:0/361364031 wait complete. 2024-09-12T03:21:04.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.452+0000 7f38b1453740 20 client.6936 trim_cache size 0 max 16384 2024-09-12T03:21:04.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.470+0000 7f38b1453740 1 Processor -- start 2024-09-12T03:21:04.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.470+0000 7f38b1453740 1 -- start start 2024-09-12T03:21:04.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.471+0000 7f38b1453740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561618e2b2c0 0x561618ee4a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:04.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.471+0000 7f38b1453740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561618e2ca50 0x561618e6dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:04.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.471+0000 7f38b1453740 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x561618e50770 0x561618e6e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:04.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.471+0000 7f38b1453740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f38a81f89e0 con 0x561618e2b2c0 2024-09-12T03:21:04.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.471+0000 7f38b1453740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f38a81f7d40 con 0x561618e2ca50 2024-09-12T03:21:04.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.471+0000 7f38b1453740 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f38a81f8390 con 0x561618e50770 2024-09-12T03:21:04.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.471+0000 7f38a7fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561618e2b2c0 0x561618ee4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:04.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.471+0000 7f38a7fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561618e2b2c0 0x561618ee4a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38628/0 (socket says 172.21.15.27:38628) 2024-09-12T03:21:04.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.471+0000 7f38a7fff640 1 -- 172.21.15.27:0/2781902652 learned_addr learned my addr 172.21.15.27:0/2781902652 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:21:04.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.471+0000 7f38a7fff640 1 -- 172.21.15.27:0/2781902652 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x561618e50770 msgr2=0x561618e6e4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:04.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.471+0000 7f38acb73640 1 --2- 172.21.15.27:0/2781902652 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x561618e50770 0x561618e6e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:04.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.471+0000 7f38a7fff640 1 --2- 172.21.15.27:0/2781902652 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x561618e50770 0x561618e6e4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:04.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.471+0000 7f38a7fff640 1 -- 172.21.15.27:0/2781902652 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561618e2ca50 msgr2=0x561618e6dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:04.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.471+0000 7f38a77fe640 1 --2- 172.21.15.27:0/2781902652 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561618e2ca50 0x561618e6dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:04.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.471+0000 7f38a7fff640 1 --2- 172.21.15.27:0/2781902652 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561618e2ca50 0x561618e6dfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:04.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.471+0000 7f38a7fff640 1 -- 172.21.15.27:0/2781902652 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561618e26850 con 0x561618e2b2c0 2024-09-12T03:21:04.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.471+0000 7f38a77fe640 1 --2- 172.21.15.27:0/2781902652 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561618e2ca50 0x561618e6dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:21:04.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.471+0000 7f38a7fff640 1 --2- 172.21.15.27:0/2781902652 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561618e2b2c0 0x561618ee4a00 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f38980098c0 tx=0x7f389802c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f5c54ac3006dc50 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:04.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.472+0000 7f389d7fa640 1 -- 172.21.15.27:0/2781902652 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3898058070 con 0x561618e2b2c0 2024-09-12T03:21:04.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.472+0000 7f389d7fa640 1 -- 172.21.15.27:0/2781902652 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3898032b10 con 0x561618e2b2c0 2024-09-12T03:21:04.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.472+0000 7f389d7fa640 1 -- 172.21.15.27:0/2781902652 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3898037040 con 0x561618e2b2c0 2024-09-12T03:21:04.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.472+0000 7f38b1453740 1 -- 172.21.15.27:0/2781902652 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561618e2b2c0 msgr2=0x561618ee4a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:04.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.472+0000 7f38b1453740 1 --2- 172.21.15.27:0/2781902652 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561618e2b2c0 0x561618ee4a00 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f38980098c0 tx=0x7f389802c000 comp rx=0 tx=0).stop 2024-09-12T03:21:04.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.472+0000 7f38b1453740 1 -- 172.21.15.27:0/2781902652 shutdown_connections 2024-09-12T03:21:04.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.472+0000 7f38b1453740 1 --2- 172.21.15.27:0/2781902652 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x561618e50770 0x561618e6e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:04.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.472+0000 7f38b1453740 1 --2- 172.21.15.27:0/2781902652 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561618e2ca50 0x561618e6dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:04.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.472+0000 7f38b1453740 1 --2- 172.21.15.27:0/2781902652 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561618e2b2c0 0x561618ee4a00 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:04.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.472+0000 7f38b1453740 1 -- 172.21.15.27:0/2781902652 >> 172.21.15.27:0/2781902652 conn(0x561618e28a80 msgr2=0x561618e71a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:04.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.472+0000 7f38b1453740 1 -- 172.21.15.27:0/2781902652 shutdown_connections 2024-09-12T03:21:04.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.472+0000 7f38b1453740 1 -- 172.21.15.27:0/2781902652 wait complete. 2024-09-12T03:21:04.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.473+0000 7f38b1453740 1 -- start start 2024-09-12T03:21:04.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.473+0000 7f38b1453740 1 Processor -- start 2024-09-12T03:21:04.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.474+0000 7f38b1453740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561618e2b2c0 0x561618e6e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:04.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.474+0000 7f38b1453740 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x561618e2ca50 0x561618e248d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:04.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.474+0000 7f38b1453740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561618e50770 0x561618e24e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:04.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.474+0000 7f38b1453740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x561618e50280 con 0x561618e2b2c0 2024-09-12T03:21:04.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.474+0000 7f38b1453740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x561618e52ed0 con 0x561618e50770 2024-09-12T03:21:04.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.474+0000 7f38b1453740 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x561618e58660 con 0x561618e2ca50 2024-09-12T03:21:04.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.474+0000 7f38b1453740 10 client.0 _ll_register_callbacks cb 0x7f38a0017070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:21:04.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.474+0000 7f38a7fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561618e2b2c0 0x561618e6e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:04.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.474+0000 7f38a7fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561618e2b2c0 0x561618e6e1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38640/0 (socket says 172.21.15.27:38640) 2024-09-12T03:21:04.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.474+0000 7f38a7fff640 1 -- 172.21.15.27:0/2364815194 learned_addr learned my addr 172.21.15.27:0/2364815194 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:21:04.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.474+0000 7f38a77fe640 1 --2- 172.21.15.27:0/2364815194 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x561618e2ca50 0x561618e248d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:04.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.474+0000 7f38acb73640 1 --2- 172.21.15.27:0/2364815194 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561618e50770 0x561618e24e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:04.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.474+0000 7f38a7fff640 1 -- 172.21.15.27:0/2364815194 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x561618e2ca50 msgr2=0x561618e248d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:04.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.474+0000 7f38a7fff640 1 --2- 172.21.15.27:0/2364815194 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x561618e2ca50 0x561618e248d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:04.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.474+0000 7f38a7fff640 1 -- 172.21.15.27:0/2364815194 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561618e50770 msgr2=0x561618e24e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:04.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.474+0000 7f38a7fff640 1 --2- 172.21.15.27:0/2364815194 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561618e50770 0x561618e24e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:04.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.474+0000 7f38a7fff640 1 -- 172.21.15.27:0/2364815194 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561618e25990 con 0x561618e2b2c0 2024-09-12T03:21:04.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.474+0000 7f38acb73640 1 --2- 172.21.15.27:0/2364815194 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561618e50770 0x561618e24e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:21:04.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.474+0000 7f38a7fff640 1 --2- 172.21.15.27:0/2364815194 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561618e2b2c0 0x561618e6e1a0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f3898000c00 tx=0x7f3898004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb05c09be13d444e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:04.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.474+0000 7f3886ffd640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-12T03:21:04.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.474+0000 7f3886ffd640 1 -- 172.21.15.27:0/2364815194 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3898058030 con 0x561618e2b2c0 2024-09-12T03:21:04.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.474+0000 7f3886ffd640 1 -- 172.21.15.27:0/2364815194 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 5709+0+0 (secure 0 0 0) 0x7f3898050070 con 0x561618e2b2c0 2024-09-12T03:21:04.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.474+0000 7f38b1453740 1 -- 172.21.15.27:0/2364815194 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3898032b10 con 0x561618e2b2c0 2024-09-12T03:21:04.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.475+0000 7f3886ffd640 1 -- 172.21.15.27:0/2364815194 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f3898037070 con 0x561618e2b2c0 2024-09-12T03:21:04.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.475+0000 7f38b1453740 10 client.6942 fetch_fsmap learned FSMap version 58 2024-09-12T03:21:04.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.475+0000 7f38b1453740 1 -- 172.21.15.27:0/2364815194 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=2+}) -- 0x561618ee5800 con 0x561618e2b2c0 2024-09-12T03:21:04.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.475+0000 7f3886ffd640 1 -- 172.21.15.27:0/2364815194 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3898032b10 con 0x561618e2b2c0 2024-09-12T03:21:04.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.475+0000 7f3886ffd640 1 -- 172.21.15.27:0/2364815194 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 58) ==== 3502+0+0 (secure 0 0 0) 0x7f389803c440 con 0x561618e2b2c0 2024-09-12T03:21:04.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.475+0000 7f38b1453740 10 client.6942 fetch_fsmap finished waiting for FSMap version 58 2024-09-12T03:21:04.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.475+0000 7f38b1453740 10 client.6942 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-12T03:21:04.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.475+0000 7f38b1453740 20 client.6942 populate_metadata read hostname 'smithi027' 2024-09-12T03:21:04.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.475+0000 7f38b1453740 1 --2- 172.21.15.27:0/2364815194 >> [v2:172.21.15.27:6833/1379166526,v1:172.21.15.27:6835/1379166526] conn(0x561618e82650 0x561618e82a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:04.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.475+0000 7f38b1453740 4 client.6942 mds_command: new command op to 6828 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-12T03:21:04.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.475+0000 7f38b1453740 1 -- 172.21.15.27:0/2364815194 --> [v2:172.21.15.27:6833/1379166526,v1:172.21.15.27:6835/1379166526] -- command(tid 0: {"prefix": "status"}) -- 0x561618d94fa0 con 0x561618e82650 2024-09-12T03:21:04.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.475+0000 7f38a77fe640 1 --2- 172.21.15.27:0/2364815194 >> [v2:172.21.15.27:6833/1379166526,v1:172.21.15.27:6835/1379166526] conn(0x561618e82650 0x561618e82a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:04.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.475+0000 7f3886ffd640 10 client.6942 ms_handle_connect on v2:172.21.15.27:6833/1379166526 2024-09-12T03:21:04.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.475+0000 7f38a77fe640 1 --2- 172.21.15.27:0/2364815194 >> [v2:172.21.15.27:6833/1379166526,v1:172.21.15.27:6835/1379166526] conn(0x561618e82650 0x561618e82a30 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb365f82f0d882 server_cookie=c4abe9b1d24abc08 in_seq=0 out_seq=0 2024-09-12T03:21:04.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.476+0000 7f3886ffd640 1 -- 172.21.15.27:0/2364815194 <== mds.0 v2:172.21.15.27:6833/1379166526 1 ==== command_reply(tid 0: 0 ) ==== 8+0+333 (crc 0 0 0) 0x561618d94fa0 con 0x561618e82650 2024-09-12T03:21:04.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.476+0000 7f3886ffd640 10 client.6942 handle_command_reply: tid=0 2024-09-12T03:21:04.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.476+0000 7f38b1453740 1 client.6942 shutdown 2024-09-12T03:21:04.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.476+0000 7f38b1453740 1 -- 172.21.15.27:0/2364815194 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561618e2b2c0 msgr2=0x561618e6e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:04.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.476+0000 7f38b1453740 1 --2- 172.21.15.27:0/2364815194 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561618e2b2c0 0x561618e6e1a0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f3898000c00 tx=0x7f3898004730 comp rx=0 tx=0).stop 2024-09-12T03:21:04.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.476+0000 7f38b1453740 1 -- 172.21.15.27:0/2364815194 shutdown_connections 2024-09-12T03:21:04.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.476+0000 7f38b1453740 1 --2- 172.21.15.27:0/2364815194 >> [v2:172.21.15.27:6833/1379166526,v1:172.21.15.27:6835/1379166526] conn(0x561618e82650 0x561618e82a30 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:04.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.476+0000 7f38b1453740 1 --2- 172.21.15.27:0/2364815194 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561618e50770 0x561618e24e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:04.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.476+0000 7f38b1453740 1 --2- 172.21.15.27:0/2364815194 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x561618e2ca50 0x561618e248d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:04.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.476+0000 7f38b1453740 1 --2- 172.21.15.27:0/2364815194 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561618e2b2c0 0x561618e6e1a0 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:04.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.476+0000 7f3886ffd640 0 client.6942 ms_handle_reset on v2:172.21.15.27:6833/1379166526 2024-09-12T03:21:04.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.476+0000 7f38b1453740 1 -- 172.21.15.27:0/2364815194 >> 172.21.15.27:0/2364815194 conn(0x561618e28a80 msgr2=0x561618e71a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:04.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.476+0000 7f38b1453740 1 -- 172.21.15.27:0/2364815194 shutdown_connections 2024-09-12T03:21:04.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.476+0000 7f38b1453740 1 -- 172.21.15.27:0/2364815194 wait complete. 2024-09-12T03:21:04.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.476+0000 7f38b1453740 20 client.6942 trim_cache size 0 max 16384 2024-09-12T03:21:04.482 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-12T03:21:04.483 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_fsid": "d2df504e-2647-4385-a5db-9fdf69f487a0", 2024-09-12T03:21:04.483 INFO:teuthology.orchestra.run.smithi027.stdout: "whoami": 0, 2024-09-12T03:21:04.483 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 6828, 2024-09-12T03:21:04.483 INFO:teuthology.orchestra.run.smithi027.stdout: "want_state": "up:active", 2024-09-12T03:21:04.483 INFO:teuthology.orchestra.run.smithi027.stdout: "state": "up:active", 2024-09-12T03:21:04.483 INFO:teuthology.orchestra.run.smithi027.stdout: "fs_name": "cephfs", 2024-09-12T03:21:04.483 INFO:teuthology.orchestra.run.smithi027.stdout: "rank_uptime": 4.6792246359999998, 2024-09-12T03:21:04.483 INFO:teuthology.orchestra.run.smithi027.stdout: "mdsmap_epoch": 58, 2024-09-12T03:21:04.483 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch": 100, 2024-09-12T03:21:04.483 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch_barrier": 100, 2024-09-12T03:21:04.483 INFO:teuthology.orchestra.run.smithi027.stdout: "uptime": 9.7216420820000007 2024-09-12T03:21:04.483 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-12T03:21:04.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.476+0000 7f38ae5d6640 1 -- 172.21.15.27:0/2040557241 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f388806bf80 msgr2=0x7f388806e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:04.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.476+0000 7f38ae5d6640 1 --2- 172.21.15.27:0/2040557241 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f388806bf80 0x7f388806e440 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f3898034000 tx=0x7f389802e000 comp rx=0 tx=0).stop 2024-09-12T03:21:04.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.476+0000 7f38ae5d6640 1 -- 172.21.15.27:0/2040557241 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f38a80b6850 msgr2=0x7f38a80ae110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:04.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.476+0000 7f38ae5d6640 1 --2- 172.21.15.27:0/2040557241 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f38a80b6850 0x7f38a80ae110 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f389000e870 tx=0x7f389000ed30 comp rx=0 tx=0).stop 2024-09-12T03:21:04.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.476+0000 7f38ae5d6640 1 -- 172.21.15.27:0/2040557241 shutdown_connections 2024-09-12T03:21:04.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.476+0000 7f38ae5d6640 1 --2- 172.21.15.27:0/2040557241 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f38a80d3190 0x7f38a80ae650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:04.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.476+0000 7f38ae5d6640 1 --2- 172.21.15.27:0/2040557241 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f388806bf80 0x7f388806e440 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:04.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.477+0000 7f38ae5d6640 1 --2- 172.21.15.27:0/2040557241 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f38a80b6850 0x7f38a80ae110 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:04.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.477+0000 7f38ae5d6640 1 --2- 172.21.15.27:0/2040557241 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f38a80b6470 0x7f38a815db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:04.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.477+0000 7f38ae5d6640 1 -- 172.21.15.27:0/2040557241 >> 172.21.15.27:0/2040557241 conn(0x7f38a80d1200 msgr2=0x7f38a80b3c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:04.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.477+0000 7f38ae5d6640 1 -- 172.21.15.27:0/2040557241 shutdown_connections 2024-09-12T03:21:04.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.477+0000 7f38ae5d6640 1 -- 172.21.15.27:0/2040557241 wait complete. 2024-09-12T03:21:04.487 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-12T03:21:04.589 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.589+0000 7ffa4a52c640 1 Processor -- start 2024-09-12T03:21:04.589 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.589+0000 7ffa4a52c640 1 -- start start 2024-09-12T03:21:04.590 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.590+0000 7ffa4a52c640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7ffa4406f360 0x7ffa440cef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:04.591 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.590+0000 7ffa4a52c640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ffa440cf490 0x7ffa440d2780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:04.591 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.590+0000 7ffa4a52c640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffa4406cc40 con 0x7ffa440b1ca0 2024-09-12T03:21:04.591 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.590+0000 7ffa4a52c640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7ffa440cf900 con 0x7ffa4406f360 2024-09-12T03:21:04.591 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.590+0000 7ffa4a52c640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7ffa440d22f0 con 0x7ffa440cf490 2024-09-12T03:21:04.591 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.590+0000 7ffa437fe640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7ffa4406f360 0x7ffa440cef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:04.591 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.590+0000 7ffa437fe640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7ffa4406f360 0x7ffa440cef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38652/0 (socket says 172.21.15.27:38652) 2024-09-12T03:21:04.591 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.590+0000 7ffa437fe640 1 -- 172.21.15.27:0/4290219066 learned_addr learned my addr 172.21.15.27:0/4290219066 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:21:04.591 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.590+0000 7ffa437fe640 1 -- 172.21.15.27:0/4290219066 >> v2:172.21.15.45:3300/0 conn(0x7ffa440cf490 msgr2=0x7ffa440d2780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:04.591 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.590+0000 7ffa48ac9640 1 --2- 172.21.15.27:0/4290219066 >> v2:172.21.15.45:3300/0 conn(0x7ffa440cf490 0x7ffa440d2780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:04.591 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.590+0000 7ffa437fe640 1 --2- 172.21.15.27:0/4290219066 >> v2:172.21.15.45:3300/0 conn(0x7ffa440cf490 0x7ffa440d2780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:04.591 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.590+0000 7ffa437fe640 1 -- 172.21.15.27:0/4290219066 >> v1:172.21.15.45:6789/0 conn(0x7ffa440b1ca0 legacy=0x7ffa4415f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:04.591 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.590+0000 7ffa437fe640 1 -- 172.21.15.27:0/4290219066 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa440d1ce0 con 0x7ffa4406f360 2024-09-12T03:21:04.593 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.591+0000 7ffa437fe640 1 --2- 172.21.15.27:0/4290219066 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffa4406f360 0x7ffa440cef50 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7ffa44069cf0 tx=0x7ffa3802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63410506765f756f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:04.593 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.591+0000 7ffa42ffd640 1 -- 172.21.15.27:0/4290219066 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa38037070 con 0x7ffa4406f360 2024-09-12T03:21:04.593 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.591+0000 7ffa42ffd640 1 -- 172.21.15.27:0/4290219066 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa38032440 con 0x7ffa4406f360 2024-09-12T03:21:04.593 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.591+0000 7ffa42ffd640 1 -- 172.21.15.27:0/4290219066 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa38032810 con 0x7ffa4406f360 2024-09-12T03:21:04.593 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.592+0000 7ffa4a52c640 1 -- 172.21.15.27:0/4290219066 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffa4406f360 msgr2=0x7ffa440cef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:04.593 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.592+0000 7ffa4a52c640 1 --2- 172.21.15.27:0/4290219066 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffa4406f360 0x7ffa440cef50 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7ffa44069cf0 tx=0x7ffa3802a230 comp rx=0 tx=0).stop 2024-09-12T03:21:04.593 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.592+0000 7ffa4a52c640 1 -- 172.21.15.27:0/4290219066 shutdown_connections 2024-09-12T03:21:04.593 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.592+0000 7ffa4a52c640 1 --2- 172.21.15.27:0/4290219066 >> v2:172.21.15.45:3300/0 conn(0x7ffa440cf490 0x7ffa440d2780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:04.593 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.592+0000 7ffa4a52c640 1 --2- 172.21.15.27:0/4290219066 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffa4406f360 0x7ffa440cef50 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:04.593 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.592+0000 7ffa4a52c640 1 -- 172.21.15.27:0/4290219066 >> 172.21.15.27:0/4290219066 conn(0x7ffa44073800 msgr2=0x7ffa4409ba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:04.594 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.592+0000 7ffa4a52c640 1 -- 172.21.15.27:0/4290219066 shutdown_connections 2024-09-12T03:21:04.594 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.592+0000 7ffa4a52c640 1 -- 172.21.15.27:0/4290219066 wait complete. 2024-09-12T03:21:04.594 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.593+0000 7ffa4a52c640 1 Processor -- start 2024-09-12T03:21:04.594 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.593+0000 7ffa4a52c640 1 -- start start 2024-09-12T03:21:04.595 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.594+0000 7ffa4a52c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffa4406f360 0x7ffa440afe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:04.595 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.594+0000 7ffa4a52c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffa440b1ca0 0x7ffa440b7000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:04.595 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.594+0000 7ffa4a52c640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ffa440cf490 0x7ffa440b7540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:04.596 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.594+0000 7ffa4a52c640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7ffa4406d1c0 con 0x7ffa440b1ca0 2024-09-12T03:21:04.596 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.594+0000 7ffa4a52c640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ffa440d1f80 con 0x7ffa4406f360 2024-09-12T03:21:04.596 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.594+0000 7ffa4a52c640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7ffa4406d340 con 0x7ffa440cf490 2024-09-12T03:21:04.596 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.594+0000 7ffa437fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffa440b1ca0 0x7ffa440b7000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:04.596 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.594+0000 7ffa437fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffa440b1ca0 0x7ffa440b7000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38658/0 (socket says 172.21.15.27:38658) 2024-09-12T03:21:04.596 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.594+0000 7ffa437fe640 1 -- 172.21.15.27:0/2975099274 learned_addr learned my addr 172.21.15.27:0/2975099274 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:21:04.596 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.594+0000 7ffa48ac9640 1 --2- 172.21.15.27:0/2975099274 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ffa440cf490 0x7ffa440b7540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:04.596 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.594+0000 7ffa437fe640 1 -- 172.21.15.27:0/2975099274 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ffa440cf490 msgr2=0x7ffa440b7540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:04.596 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.594+0000 7ffa43fff640 1 --2- 172.21.15.27:0/2975099274 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffa4406f360 0x7ffa440afe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:04.596 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.594+0000 7ffa437fe640 1 --2- 172.21.15.27:0/2975099274 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ffa440cf490 0x7ffa440b7540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:04.596 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.594+0000 7ffa437fe640 1 -- 172.21.15.27:0/2975099274 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffa4406f360 msgr2=0x7ffa440afe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:04.596 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.594+0000 7ffa437fe640 1 --2- 172.21.15.27:0/2975099274 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffa4406f360 0x7ffa440afe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:04.596 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.594+0000 7ffa437fe640 1 -- 172.21.15.27:0/2975099274 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa44160d60 con 0x7ffa440b1ca0 2024-09-12T03:21:04.597 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.595+0000 7ffa437fe640 1 --2- 172.21.15.27:0/2975099274 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffa440b1ca0 0x7ffa440b7000 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7ffa38000c00 tx=0x7ffa380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:04.597 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.595+0000 7ffa40ff9640 1 -- 172.21.15.27:0/2975099274 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa38041020 con 0x7ffa440b1ca0 2024-09-12T03:21:04.597 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.595+0000 7ffa40ff9640 1 -- 172.21.15.27:0/2975099274 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa38030970 con 0x7ffa440b1ca0 2024-09-12T03:21:04.597 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.595+0000 7ffa4a52c640 1 -- 172.21.15.27:0/2975099274 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa44161040 con 0x7ffa440b1ca0 2024-09-12T03:21:04.597 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.595+0000 7ffa40ff9640 1 -- 172.21.15.27:0/2975099274 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa38037040 con 0x7ffa440b1ca0 2024-09-12T03:21:04.597 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.595+0000 7ffa4a52c640 1 -- 172.21.15.27:0/2975099274 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa44161600 con 0x7ffa440b1ca0 2024-09-12T03:21:04.598 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.596+0000 7ffa4a52c640 1 -- 172.21.15.27:0/2975099274 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa440cfc40 con 0x7ffa440b1ca0 2024-09-12T03:21:04.598 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.597+0000 7ffa40ff9640 1 -- 172.21.15.27:0/2975099274 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 90600+0+0 (secure 0 0 0) 0x7ffa38044020 con 0x7ffa440b1ca0 2024-09-12T03:21:04.598 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.597+0000 7ffa40ff9640 1 --2- 172.21.15.27:0/2975099274 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7ffa3006bf50 0x7ffa3006e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:04.598 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.597+0000 7ffa40ff9640 1 -- 172.21.15.27:0/2975099274 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5709+0+0 (secure 0 0 0) 0x7ffa380aba10 con 0x7ffa440b1ca0 2024-09-12T03:21:04.598 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.597+0000 7ffa43fff640 1 --2- 172.21.15.27:0/2975099274 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7ffa3006bf50 0x7ffa3006e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:04.598 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.597+0000 7ffa43fff640 1 --2- 172.21.15.27:0/2975099274 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7ffa3006bf50 0x7ffa3006e410 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7ffa340015d0 tx=0x7ffa34009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:04.601 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.601+0000 7ffa40ff9640 1 -- 172.21.15.27:0/2975099274 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7ffa3804b030 con 0x7ffa440b1ca0 2024-09-12T03:21:04.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.813+0000 7ffa4a52c640 1 -- 172.21.15.27:0/2975099274 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7ffa4406e8c0 con 0x7ffa440b1ca0 2024-09-12T03:21:04.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.813+0000 7ffa40ff9640 1 -- 172.21.15.27:0/2975099274 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v58) ==== 53+0+73 (secure 0 0 0) 0x7ffa3807a2b0 con 0x7ffa440b1ca0 2024-09-12T03:21:04.813 INFO:teuthology.orchestra.run.smithi027.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-12T03:21:04.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.815+0000 7ffa4a52c640 1 -- 172.21.15.27:0/2975099274 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7ffa3006bf50 msgr2=0x7ffa3006e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:04.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.815+0000 7ffa4a52c640 1 --2- 172.21.15.27:0/2975099274 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7ffa3006bf50 0x7ffa3006e410 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7ffa340015d0 tx=0x7ffa34009040 comp rx=0 tx=0).stop 2024-09-12T03:21:04.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.815+0000 7ffa4a52c640 1 -- 172.21.15.27:0/2975099274 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffa440b1ca0 msgr2=0x7ffa440b7000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:04.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.815+0000 7ffa4a52c640 1 --2- 172.21.15.27:0/2975099274 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffa440b1ca0 0x7ffa440b7000 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7ffa38000c00 tx=0x7ffa380046c0 comp rx=0 tx=0).stop 2024-09-12T03:21:04.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.815+0000 7ffa4a52c640 1 -- 172.21.15.27:0/2975099274 shutdown_connections 2024-09-12T03:21:04.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.815+0000 7ffa4a52c640 1 --2- 172.21.15.27:0/2975099274 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ffa440cf490 0x7ffa440b7540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:04.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.815+0000 7ffa4a52c640 1 --2- 172.21.15.27:0/2975099274 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7ffa3006bf50 0x7ffa3006e410 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:04.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.815+0000 7ffa4a52c640 1 --2- 172.21.15.27:0/2975099274 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffa440b1ca0 0x7ffa440b7000 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:04.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.815+0000 7ffa4a52c640 1 --2- 172.21.15.27:0/2975099274 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffa4406f360 0x7ffa440afe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:04.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.815+0000 7ffa4a52c640 1 -- 172.21.15.27:0/2975099274 >> 172.21.15.27:0/2975099274 conn(0x7ffa44073800 msgr2=0x7ffa4406f740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:04.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.815+0000 7ffa4a52c640 1 -- 172.21.15.27:0/2975099274 shutdown_connections 2024-09-12T03:21:04.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:04.815+0000 7ffa4a52c640 1 -- 172.21.15.27:0/2975099274 wait complete. 2024-09-12T03:21:04.826 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-12T03:21:04.826 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-12T03:21:04.826 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi027.front.sepia.ceph.com 2024-09-12T03:21:04.827 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-12T03:21:04.827 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-12T03:21:04.827 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-12T03:21:04.827 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-12T03:21:04.827 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-12T03:21:04.827 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-12T03:21:04.827 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:21:04.827 DEBUG:teuthology.orchestra.run.smithi027:> ip addr 2024-09-12T03:21:04.844 INFO:teuthology.orchestra.run.smithi027.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-12T03:21:04.844 INFO:teuthology.orchestra.run.smithi027.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-12T03:21:04.844 INFO:teuthology.orchestra.run.smithi027.stdout: inet 127.0.0.1/8 scope host lo 2024-09-12T03:21:04.844 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:21:04.845 INFO:teuthology.orchestra.run.smithi027.stdout: inet6 ::1/128 scope host 2024-09-12T03:21:04.845 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:21:04.845 INFO:teuthology.orchestra.run.smithi027.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-12T03:21:04.845 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether 0c:c4:7a:6b:fe:c8 brd ff:ff:ff:ff:ff:ff 2024-09-12T03:21:04.845 INFO:teuthology.orchestra.run.smithi027.stdout: altname enp7s0f0 2024-09-12T03:21:04.845 INFO:teuthology.orchestra.run.smithi027.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-12T03:21:04.845 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether 0c:c4:7a:6b:fe:c9 brd ff:ff:ff:ff:ff:ff 2024-09-12T03:21:04.845 INFO:teuthology.orchestra.run.smithi027.stdout: altname enp7s0f1 2024-09-12T03:21:04.845 INFO:teuthology.orchestra.run.smithi027.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-12T03:21:04.845 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether ac:1f:6b:f8:39:5a brd ff:ff:ff:ff:ff:ff 2024-09-12T03:21:04.845 INFO:teuthology.orchestra.run.smithi027.stdout: altname enp1s0f0 2024-09-12T03:21:04.845 INFO:teuthology.orchestra.run.smithi027.stdout: inet 172.21.15.27/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-12T03:21:04.846 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft 42524sec preferred_lft 42524sec 2024-09-12T03:21:04.846 INFO:teuthology.orchestra.run.smithi027.stdout: inet6 fe80::ae1f:6bff:fef8:395a/64 scope link 2024-09-12T03:21:04.846 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:21:04.846 INFO:teuthology.orchestra.run.smithi027.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-12T03:21:04.846 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether ac:1f:6b:f8:39:5b brd ff:ff:ff:ff:ff:ff 2024-09-12T03:21:04.846 INFO:teuthology.orchestra.run.smithi027.stdout: altname enp1s0f1 2024-09-12T03:21:04.846 INFO:teuthology.orchestra.run.smithi027.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-12T03:21:04.846 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether 2e:7f:89:6c:33:a0 brd ff:ff:ff:ff:ff:ff 2024-09-12T03:21:04.846 INFO:teuthology.orchestra.run.smithi027.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-12T03:21:04.846 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:21:04.846 INFO:teuthology.orchestra.run.smithi027.stdout: inet6 fe80::c84b:19ff:fef0:ca74/64 scope link 2024-09-12T03:21:04.847 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:21:04.847 INFO:teuthology.orchestra.run.smithi027.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-12T03:21:04.847 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether 42:40:d7:5c:7a:8e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-12T03:21:04.847 INFO:teuthology.orchestra.run.smithi027.stdout: inet6 fe80::4040:d7ff:fe5c:7a8e/64 scope link 2024-09-12T03:21:04.847 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:21:04.847 INFO:teuthology.orchestra.run.smithi027.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-12T03:21:04.847 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether 2e:7f:89:6c:33:a0 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-12T03:21:04.847 INFO:teuthology.orchestra.run.smithi027.stdout: inet6 fe80::2c7f:89ff:fe6c:33a0/64 scope link 2024-09-12T03:21:04.847 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:21:04.848 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:21:04.848 DEBUG:teuthology.orchestra.run.smithi027:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-12T03:21:04.909 INFO:teuthology.orchestra.run.smithi027.stdout:1 2024-09-12T03:21:04.911 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:21:04.911 DEBUG:teuthology.orchestra.run.smithi027:> route 2024-09-12T03:21:04.969 INFO:teuthology.orchestra.run.smithi027.stdout:Kernel IP routing table 2024-09-12T03:21:04.970 INFO:teuthology.orchestra.run.smithi027.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-12T03:21:04.970 INFO:teuthology.orchestra.run.smithi027.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-12T03:21:04.970 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-12T03:21:04.970 INFO:teuthology.orchestra.run.smithi027.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-12T03:21:04.970 DEBUG:teuthology.orchestra.run.smithi027:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-12T03:21:04.970 DEBUG:teuthology.orchestra.run.smithi027:> set -e 2024-09-12T03:21:04.970 DEBUG:teuthology.orchestra.run.smithi027:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-12T03:21:04.970 DEBUG:teuthology.orchestra.run.smithi027:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-12T03:21:04.971 DEBUG:teuthology.orchestra.run.smithi027:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-12T03:21:04.971 DEBUG:teuthology.orchestra.run.smithi027:> ') 2024-09-12T03:21:05.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:05 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T03:21:05.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T03:21:05.100 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:21:05.100 DEBUG:teuthology.orchestra.run.smithi027:> ip netns list 2024-09-12T03:21:05.116 INFO:teuthology.orchestra.run.smithi027.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-12T03:21:05.116 INFO:teuthology.orchestra.run.smithi027.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-12T03:21:05.116 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-12T03:21:05.117 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:21:05.117 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip link set brx.0 up 2024-09-12T03:21:05.183 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-12T03:21:05.183 DEBUG:teuthology.orchestra.run.smithi027:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-12T03:21:05.240 INFO:teuthology.orchestra.run.smithi027.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-12T03:21:05.240 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-12T03:21:05.240 DEBUG:teuthology.orchestra.run.smithi027:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-12T03:21:05.297 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-09-12T03:21:05.297 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:21:05.297 DEBUG:teuthology.orchestra.run.smithi027:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-12T03:21:05.381 INFO:teuthology.orchestra.run.smithi027.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-12T03:21:05.381 INFO:teuthology.orchestra.run.smithi027.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-12T03:21:05.389 INFO:teuthology.orchestra.run.smithi027.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-12T03:21:05.389 INFO:teuthology.orchestra.run.smithi027.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-12T03:21:05.389 INFO:teuthology.orchestra.run.smithi027.stdout:mount.ceph: resolved to: "172.21.15.27:3300,172.21.15.45:3300,172.21.15.45:3301" 2024-09-12T03:21:05.389 INFO:teuthology.orchestra.run.smithi027.stdout:mount.ceph: trying mount with new device syntax: 0@d2df504e-2647-4385-a5db-9fdf69f487a0.cephfs=/ 2024-09-12T03:21:05.389 INFO:teuthology.orchestra.run.smithi027.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.27:3300/172.21.15.45:3300/172.21.15.45:3301" will pass to kernel 2024-09-12T03:21:05.390 INFO:teuthology.orchestra.run.smithi027.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-12T03:21:05.390 INFO:teuthology.orchestra.run.smithi027.stdout: You just mounted a file system that supports labels which does not 2024-09-12T03:21:05.390 INFO:teuthology.orchestra.run.smithi027.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-12T03:21:05.390 INFO:teuthology.orchestra.run.smithi027.stdout: applications will generate AVC messages and not be allowed access to 2024-09-12T03:21:05.390 INFO:teuthology.orchestra.run.smithi027.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-12T03:21:05.392 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-12T03:21:05.392 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:21:05.392 DEBUG:teuthology.orchestra.run.smithi027:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-12T03:21:05.418 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-12T03:21:05.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:05 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-12T03:21:05.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T03:21:05.520 INFO:teuthology.orchestra.run.smithi027.stdout:client5978 2024-09-12T03:21:05.521 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-12T03:21:05.521 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T03:21:05.537 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-12T03:21:05.680 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:05.680+0000 7f162f482640 1 Processor -- start 2024-09-12T03:21:05.680 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:05.680+0000 7f162f482640 1 -- start start 2024-09-12T03:21:05.682 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:05.681+0000 7f162f482640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f16280ae8d0 0x7f16280728b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:05.682 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:05.681+0000 7f162f482640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f1628072df0 0x7f162806e930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:05.682 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:05.681+0000 7f162f482640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f16280aee90 con 0x7f16280b5a70 2024-09-12T03:21:05.682 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:05.681+0000 7f162f482640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f1628073240 con 0x7f1628072df0 2024-09-12T03:21:05.682 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:05.681+0000 7f162f482640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f16280af170 con 0x7f16280ae8d0 2024-09-12T03:21:05.682 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:05.681+0000 7f162da1f640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f1628072df0 0x7f162806e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:05.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:05.681+0000 7f162da1f640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f1628072df0 0x7f162806e930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38672/0 (socket says 172.21.15.27:38672) 2024-09-12T03:21:05.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:05.681+0000 7f162da1f640 1 -- 172.21.15.27:0/4128862109 learned_addr learned my addr 172.21.15.27:0/4128862109 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:21:05.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:05.681+0000 7f162ca1d640 1 --2- 172.21.15.27:0/4128862109 >> v2:172.21.15.45:3300/0 conn(0x7f16280ae8d0 0x7f16280728b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:05.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:05.681+0000 7f162da1f640 1 -- 172.21.15.27:0/4128862109 >> v2:172.21.15.45:3300/0 conn(0x7f16280ae8d0 msgr2=0x7f16280728b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:05.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:05.681+0000 7f162da1f640 1 --2- 172.21.15.27:0/4128862109 >> v2:172.21.15.45:3300/0 conn(0x7f16280ae8d0 0x7f16280728b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:05.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:05.681+0000 7f162da1f640 1 -- 172.21.15.27:0/4128862109 >> v1:172.21.15.45:6789/0 conn(0x7f16280b5a70 legacy=0x7f162815f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:05.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:05.681+0000 7f162da1f640 1 -- 172.21.15.27:0/4128862109 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16280af450 con 0x7f1628072df0 2024-09-12T03:21:05.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:05.682+0000 7f162da1f640 1 --2- 172.21.15.27:0/4128862109 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1628072df0 0x7f162806e930 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f1628062840 tx=0x7f162002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4381066209914a2b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:05.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:05.682+0000 7f161ffff640 1 -- 172.21.15.27:0/4128862109 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1620036070 con 0x7f1628072df0 2024-09-12T03:21:05.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:05.682+0000 7f161ffff640 1 -- 172.21.15.27:0/4128862109 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1620031600 con 0x7f1628072df0 2024-09-12T03:21:05.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:05.682+0000 7f161ffff640 1 -- 172.21.15.27:0/4128862109 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16200319d0 con 0x7f1628072df0 2024-09-12T03:21:05.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:05.683+0000 7f162f482640 1 -- 172.21.15.27:0/4128862109 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1628072df0 msgr2=0x7f162806e930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:05.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:05.683+0000 7f162f482640 1 --2- 172.21.15.27:0/4128862109 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1628072df0 0x7f162806e930 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f1628062840 tx=0x7f162002a6a0 comp rx=0 tx=0).stop 2024-09-12T03:21:05.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:05.683+0000 7f162f482640 1 -- 172.21.15.27:0/4128862109 shutdown_connections 2024-09-12T03:21:05.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:05.683+0000 7f162f482640 1 --2- 172.21.15.27:0/4128862109 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1628072df0 0x7f162806e930 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:05.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:05.683+0000 7f162f482640 1 --2- 172.21.15.27:0/4128862109 >> v2:172.21.15.45:3300/0 conn(0x7f16280ae8d0 0x7f16280728b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:05.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:05.683+0000 7f162f482640 1 -- 172.21.15.27:0/4128862109 >> 172.21.15.27:0/4128862109 conn(0x7f162806ca30 msgr2=0x7f162806ce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:05.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:05.683+0000 7f162f482640 1 -- 172.21.15.27:0/4128862109 shutdown_connections 2024-09-12T03:21:05.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:05.683+0000 7f162f482640 1 -- 172.21.15.27:0/4128862109 wait complete. 2024-09-12T03:21:05.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:05.684+0000 7f162f482640 1 Processor -- start 2024-09-12T03:21:05.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:05.684+0000 7f162f482640 1 -- start start 2024-09-12T03:21:05.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:05.685+0000 7f162f482640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1628072df0 0x7f162814d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:05.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:05.685+0000 7f162f482640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f16280ae8d0 0x7f16281505d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:05.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:05.685+0000 7f162f482640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f16280b5a70 0x7f1628160760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:05.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:05.685+0000 7f162f482640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f162806eff0 con 0x7f16280ae8d0 2024-09-12T03:21:05.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:05.685+0000 7f162f482640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f162806ee70 con 0x7f1628072df0 2024-09-12T03:21:05.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:05.685+0000 7f162f482640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f162806f170 con 0x7f16280b5a70 2024-09-12T03:21:05.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:05.685+0000 7f162ca1d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f16280ae8d0 0x7f16281505d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:05.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:05.685+0000 7f162ca1d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f16280ae8d0 0x7f16281505d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38680/0 (socket says 172.21.15.27:38680) 2024-09-12T03:21:05.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:05.685+0000 7f162ca1d640 1 -- 172.21.15.27:0/316513571 learned_addr learned my addr 172.21.15.27:0/316513571 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:21:05.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:05.685+0000 7f162da1f640 1 --2- 172.21.15.27:0/316513571 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f16280b5a70 0x7f1628160760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:05.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:05.685+0000 7f162d21e640 1 --2- 172.21.15.27:0/316513571 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1628072df0 0x7f162814d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:05.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:05.685+0000 7f162ca1d640 1 -- 172.21.15.27:0/316513571 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f16280b5a70 msgr2=0x7f1628160760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:05.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:05.685+0000 7f162ca1d640 1 --2- 172.21.15.27:0/316513571 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f16280b5a70 0x7f1628160760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:05.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:05.685+0000 7f162ca1d640 1 -- 172.21.15.27:0/316513571 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1628072df0 msgr2=0x7f162814d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:05.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:05.685+0000 7f162ca1d640 1 --2- 172.21.15.27:0/316513571 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1628072df0 0x7f162814d870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:05.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:05.685+0000 7f162ca1d640 1 -- 172.21.15.27:0/316513571 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16280cdba0 con 0x7f16280ae8d0 2024-09-12T03:21:05.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:05.685+0000 7f162d21e640 1 --2- 172.21.15.27:0/316513571 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1628072df0 0x7f162814d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:21:05.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:05.685+0000 7f162ca1d640 1 --2- 172.21.15.27:0/316513571 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f16280ae8d0 0x7f16281505d0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f161800e930 tx=0x7f161800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:05.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:05.686+0000 7f161dffb640 1 -- 172.21.15.27:0/316513571 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1618017070 con 0x7f16280ae8d0 2024-09-12T03:21:05.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:05.686+0000 7f161dffb640 1 -- 172.21.15.27:0/316513571 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f161800ca00 con 0x7f16280ae8d0 2024-09-12T03:21:05.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:05.686+0000 7f161dffb640 1 -- 172.21.15.27:0/316513571 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f161800cdf0 con 0x7f16280ae8d0 2024-09-12T03:21:05.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:05.686+0000 7f162f482640 1 -- 172.21.15.27:0/316513571 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1628159520 con 0x7f16280ae8d0 2024-09-12T03:21:05.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:05.686+0000 7f162f482640 1 -- 172.21.15.27:0/316513571 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1628159ae0 con 0x7f16280ae8d0 2024-09-12T03:21:05.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:05.687+0000 7f162f482640 1 -- 172.21.15.27:0/316513571 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f162806e930 con 0x7f16280ae8d0 2024-09-12T03:21:05.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:05.687+0000 7f161dffb640 1 -- 172.21.15.27:0/316513571 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 90600+0+0 (secure 0 0 0) 0x7f161801a070 con 0x7f16280ae8d0 2024-09-12T03:21:05.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:05.688+0000 7f161dffb640 1 --2- 172.21.15.27:0/316513571 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f160c06bf50 0x7f160c06e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:05.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:05.688+0000 7f161dffb640 1 -- 172.21.15.27:0/316513571 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5709+0+0 (secure 0 0 0) 0x7f161808afb0 con 0x7f16280ae8d0 2024-09-12T03:21:05.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:05.688+0000 7f162d21e640 1 --2- 172.21.15.27:0/316513571 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f160c06bf50 0x7f160c06e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:05.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:05.688+0000 7f162d21e640 1 --2- 172.21.15.27:0/316513571 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f160c06bf50 0x7f160c06e410 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f162815acd0 tx=0x7f1624009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:05.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:05.692+0000 7f161dffb640 1 -- 172.21.15.27:0/316513571 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f16180599a0 con 0x7f16280ae8d0 2024-09-12T03:21:05.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:05.901+0000 7f162f482640 1 -- 172.21.15.27:0/316513571 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f162814de30 con 0x7f16280ae8d0 2024-09-12T03:21:05.902 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:05.902+0000 7f161dffb640 1 -- 172.21.15.27:0/316513571 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v58) ==== 53+0+73 (secure 0 0 0) 0x7f16180592a0 con 0x7f16280ae8d0 2024-09-12T03:21:05.902 INFO:teuthology.orchestra.run.smithi027.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-12T03:21:05.904 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:05.904+0000 7f162f482640 1 -- 172.21.15.27:0/316513571 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f160c06bf50 msgr2=0x7f160c06e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:05.904 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:05.904+0000 7f162f482640 1 --2- 172.21.15.27:0/316513571 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f160c06bf50 0x7f160c06e410 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f162815acd0 tx=0x7f1624009040 comp rx=0 tx=0).stop 2024-09-12T03:21:05.904 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:05.904+0000 7f162f482640 1 -- 172.21.15.27:0/316513571 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f16280ae8d0 msgr2=0x7f16281505d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:05.904 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:05.904+0000 7f162f482640 1 --2- 172.21.15.27:0/316513571 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f16280ae8d0 0x7f16281505d0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f161800e930 tx=0x7f161800edf0 comp rx=0 tx=0).stop 2024-09-12T03:21:05.904 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:05.904+0000 7f162f482640 1 -- 172.21.15.27:0/316513571 shutdown_connections 2024-09-12T03:21:05.904 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:05.904+0000 7f162f482640 1 --2- 172.21.15.27:0/316513571 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f16280b5a70 0x7f1628160760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:05.905 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:05.904+0000 7f162f482640 1 --2- 172.21.15.27:0/316513571 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f160c06bf50 0x7f160c06e410 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:05.905 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:05.904+0000 7f162f482640 1 --2- 172.21.15.27:0/316513571 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f16280ae8d0 0x7f16281505d0 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:05.905 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:05.904+0000 7f162f482640 1 --2- 172.21.15.27:0/316513571 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1628072df0 0x7f162814d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:05.905 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:05.904+0000 7f162f482640 1 -- 172.21.15.27:0/316513571 >> 172.21.15.27:0/316513571 conn(0x7f162806ca30 msgr2=0x7f162814e070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:05.905 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:05.904+0000 7f162f482640 1 -- 172.21.15.27:0/316513571 shutdown_connections 2024-09-12T03:21:05.905 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:05.904+0000 7f162f482640 1 -- 172.21.15.27:0/316513571 wait complete. 2024-09-12T03:21:05.915 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-12T03:21:05.915 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-12T03:21:05.915 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi045.front.sepia.ceph.com 2024-09-12T03:21:05.915 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-12T03:21:05.915 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-12T03:21:05.915 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-12T03:21:05.915 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-12T03:21:05.915 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-12T03:21:05.915 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-12T03:21:05.916 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:21:05.916 DEBUG:teuthology.orchestra.run.smithi045:> ip addr 2024-09-12T03:21:05.936 INFO:teuthology.orchestra.run.smithi045.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-12T03:21:05.936 INFO:teuthology.orchestra.run.smithi045.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-12T03:21:05.936 INFO:teuthology.orchestra.run.smithi045.stdout: inet 127.0.0.1/8 scope host lo 2024-09-12T03:21:05.936 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:21:05.936 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 ::1/128 scope host 2024-09-12T03:21:05.937 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:21:05.937 INFO:teuthology.orchestra.run.smithi045.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-12T03:21:05.937 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:6b:fd:a6 brd ff:ff:ff:ff:ff:ff 2024-09-12T03:21:05.937 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp7s0f0 2024-09-12T03:21:05.937 INFO:teuthology.orchestra.run.smithi045.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-12T03:21:05.937 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:6b:fd:a7 brd ff:ff:ff:ff:ff:ff 2024-09-12T03:21:05.937 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp7s0f1 2024-09-12T03:21:05.937 INFO:teuthology.orchestra.run.smithi045.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-12T03:21:05.937 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:bd:12:be brd ff:ff:ff:ff:ff:ff 2024-09-12T03:21:05.937 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp1s0f0 2024-09-12T03:21:05.937 INFO:teuthology.orchestra.run.smithi045.stdout: inet 172.21.15.45/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-12T03:21:05.937 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft 42468sec preferred_lft 42468sec 2024-09-12T03:21:05.938 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::ec4:7aff:febd:12be/64 scope link 2024-09-12T03:21:05.938 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:21:05.938 INFO:teuthology.orchestra.run.smithi045.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-12T03:21:05.938 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:bd:12:bf brd ff:ff:ff:ff:ff:ff 2024-09-12T03:21:05.938 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp1s0f1 2024-09-12T03:21:05.938 INFO:teuthology.orchestra.run.smithi045.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-12T03:21:05.938 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 52:97:33:5c:60:1b brd ff:ff:ff:ff:ff:ff 2024-09-12T03:21:05.938 INFO:teuthology.orchestra.run.smithi045.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-12T03:21:05.938 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:21:05.938 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::d8c2:5dff:fe26:78c8/64 scope link 2024-09-12T03:21:05.938 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:21:05.938 INFO:teuthology.orchestra.run.smithi045.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-12T03:21:05.939 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 5a:ec:d7:40:ef:75 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-12T03:21:05.939 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::58ec:d7ff:fe40:ef75/64 scope link 2024-09-12T03:21:05.939 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:21:05.939 INFO:teuthology.orchestra.run.smithi045.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-12T03:21:05.939 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 52:97:33:5c:60:1b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-12T03:21:05.939 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::5097:33ff:fe5c:601b/64 scope link 2024-09-12T03:21:05.939 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:21:05.939 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:21:05.939 DEBUG:teuthology.orchestra.run.smithi045:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-12T03:21:06.006 INFO:teuthology.orchestra.run.smithi045.stdout:1 2024-09-12T03:21:06.009 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:21:06.009 DEBUG:teuthology.orchestra.run.smithi045:> route 2024-09-12T03:21:06.068 INFO:teuthology.orchestra.run.smithi045.stdout:Kernel IP routing table 2024-09-12T03:21:06.069 INFO:teuthology.orchestra.run.smithi045.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-12T03:21:06.069 INFO:teuthology.orchestra.run.smithi045.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-12T03:21:06.069 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-12T03:21:06.069 INFO:teuthology.orchestra.run.smithi045.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-12T03:21:06.069 DEBUG:teuthology.orchestra.run.smithi045:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-12T03:21:06.069 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-09-12T03:21:06.069 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-12T03:21:06.069 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-12T03:21:06.069 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-12T03:21:06.070 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-09-12T03:21:06.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:21:06 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T03:21:06.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:21:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T03:21:06.207 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:21:06.207 DEBUG:teuthology.orchestra.run.smithi045:> ip netns list 2024-09-12T03:21:06.224 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-09-12T03:21:06.224 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-12T03:21:06.224 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-12T03:21:06.224 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:21:06.224 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link set brx.0 up 2024-09-12T03:21:06.290 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-12T03:21:06.291 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-12T03:21:06.347 INFO:teuthology.orchestra.run.smithi045.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-12T03:21:06.347 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-12T03:21:06.347 DEBUG:teuthology.orchestra.run.smithi045:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-12T03:21:06.405 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2024-09-12T03:21:06.405 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:21:06.406 DEBUG:teuthology.orchestra.run.smithi045:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-12T03:21:06.491 INFO:teuthology.orchestra.run.smithi045.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-12T03:21:06.491 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-12T03:21:06.499 INFO:teuthology.orchestra.run.smithi045.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-12T03:21:06.499 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-12T03:21:06.499 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: resolved to: "172.21.15.27:3300,172.21.15.45:3300,172.21.15.45:3301" 2024-09-12T03:21:06.499 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: trying mount with new device syntax: 1@d2df504e-2647-4385-a5db-9fdf69f487a0.cephfs=/ 2024-09-12T03:21:06.500 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.27:3300/172.21.15.45:3300/172.21.15.45:3301" will pass to kernel 2024-09-12T03:21:06.500 INFO:teuthology.orchestra.run.smithi045.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-09-12T03:21:06.500 INFO:teuthology.orchestra.run.smithi045.stdout: You just mounted a file system that supports labels which does not 2024-09-12T03:21:06.500 INFO:teuthology.orchestra.run.smithi045.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-12T03:21:06.500 INFO:teuthology.orchestra.run.smithi045.stdout: applications will generate AVC messages and not be allowed access to 2024-09-12T03:21:06.501 INFO:teuthology.orchestra.run.smithi045.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-12T03:21:06.502 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-12T03:21:06.502 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:21:06.503 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-12T03:21:06.528 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-12T03:21:06.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:21:06 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-12T03:21:06.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:21:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T03:21:06.626 INFO:teuthology.orchestra.run.smithi045.stdout:client5984 2024-09-12T03:21:06.626 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-12T03:21:06.626 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T03:21:06.643 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-09-12T03:21:06.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:06.758+0000 7f03566ba640 1 Processor -- start 2024-09-12T03:21:06.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:06.758+0000 7f03566ba640 1 -- start start 2024-09-12T03:21:06.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:06.759+0000 7f03566ba640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f03500b6440 0x7f0350162840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:06.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:06.759+0000 7f03566ba640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f03500d3810 0x7f0350162d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:06.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:06.759+0000 7f03566ba640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f03500d4310 con 0x7f03500b6820 2024-09-12T03:21:06.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:06.759+0000 7f03566ba640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f03500b7b30 con 0x7f03500b6440 2024-09-12T03:21:06.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:06.759+0000 7f03566ba640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f03500d1e90 con 0x7f03500d3810 2024-09-12T03:21:06.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:06.760+0000 7f034f7fe640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f03500d3810 0x7f0350162d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:06.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:06.760+0000 7f034ffff640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f03500b6440 0x7f0350162840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:06.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:06.760+0000 7f0354c57640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f03500b6820 0x7f03500d2320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.27:36362/0 (socket says 172.21.15.27:36362) 2024-09-12T03:21:06.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:06.760+0000 7f0354c57640 1 -- 172.21.15.27:0/2763234961 learned_addr learned my addr 172.21.15.27:0/2763234961 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:21:06.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:06.760+0000 7f034effd640 1 -- 172.21.15.27:0/2763234961 <== mon.1 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2811361697 0 0) 0x7f03500d4310 con 0x7f03500b6820 2024-09-12T03:21:06.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:06.760+0000 7f034effd640 1 -- 172.21.15.27:0/2763234961 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0338003520 con 0x7f03500b6820 2024-09-12T03:21:06.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:06.760+0000 7f034f7fe640 1 -- 172.21.15.27:0/2763234961 >> v2:172.21.15.45:3300/0 conn(0x7f03500b6440 msgr2=0x7f0350162840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:06.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:06.760+0000 7f034f7fe640 1 --2- 172.21.15.27:0/2763234961 >> v2:172.21.15.45:3300/0 conn(0x7f03500b6440 0x7f0350162840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:06.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:06.760+0000 7f034f7fe640 1 -- 172.21.15.27:0/2763234961 >> v1:172.21.15.45:6789/0 conn(0x7f03500b6820 legacy=0x7f03500d2320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:06.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:06.760+0000 7f034f7fe640 1 -- 172.21.15.27:0/2763234961 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03500b7750 con 0x7f03500d3810 2024-09-12T03:21:06.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:06.760+0000 7f034ffff640 1 --2- 172.21.15.27:0/2763234961 >> v2:172.21.15.45:3300/0 conn(0x7f03500b6440 0x7f0350162840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:21:06.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:06.761+0000 7f034f7fe640 1 --2- 172.21.15.27:0/2763234961 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f03500d3810 0x7f0350162d80 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f03440097a0 tx=0x7f034402a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=85fe4db75d4970e2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:06.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:06.761+0000 7f034effd640 1 -- 172.21.15.27:0/2763234961 <== mon.2 v2:172.21.15.45:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0344037070 con 0x7f03500d3810 2024-09-12T03:21:06.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:06.761+0000 7f034effd640 1 -- 172.21.15.27:0/2763234961 <== mon.2 v2:172.21.15.45:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0344032440 con 0x7f03500d3810 2024-09-12T03:21:06.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:06.761+0000 7f034effd640 1 -- 172.21.15.27:0/2763234961 <== mon.2 v2:172.21.15.45:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03440327f0 con 0x7f03500d3810 2024-09-12T03:21:06.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:06.762+0000 7f03566ba640 1 -- 172.21.15.27:0/2763234961 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f03500d3810 msgr2=0x7f0350162d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:06.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:06.762+0000 7f03566ba640 1 --2- 172.21.15.27:0/2763234961 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f03500d3810 0x7f0350162d80 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f03440097a0 tx=0x7f034402a180 comp rx=0 tx=0).stop 2024-09-12T03:21:06.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:06.762+0000 7f03566ba640 1 -- 172.21.15.27:0/2763234961 shutdown_connections 2024-09-12T03:21:06.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:06.762+0000 7f03566ba640 1 --2- 172.21.15.27:0/2763234961 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f03500d3810 0x7f0350162d80 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:06.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:06.762+0000 7f03566ba640 1 --2- 172.21.15.27:0/2763234961 >> v2:172.21.15.45:3300/0 conn(0x7f03500b6440 0x7f0350162840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:06.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:06.762+0000 7f03566ba640 1 -- 172.21.15.27:0/2763234961 >> 172.21.15.27:0/2763234961 conn(0x7f03500d11d0 msgr2=0x7f03500d15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:06.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:06.762+0000 7f03566ba640 1 -- 172.21.15.27:0/2763234961 shutdown_connections 2024-09-12T03:21:06.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:06.762+0000 7f03566ba640 1 -- 172.21.15.27:0/2763234961 wait complete. 2024-09-12T03:21:06.766 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:06.763+0000 7f03566ba640 1 Processor -- start 2024-09-12T03:21:06.766 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:06.763+0000 7f03566ba640 1 -- start start 2024-09-12T03:21:06.766 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:06.764+0000 7f03566ba640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f03500b6440 0x7f03501f3de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:06.766 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:06.764+0000 7f03566ba640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f03500b6820 0x7f03500ca8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:06.766 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:06.764+0000 7f03566ba640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f03500d3810 0x7f0350147b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:06.766 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:06.764+0000 7f03566ba640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f03500b8fd0 con 0x7f03500b6440 2024-09-12T03:21:06.766 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:06.764+0000 7f03566ba640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f03500d3f70 con 0x7f03500d3810 2024-09-12T03:21:06.767 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:06.764+0000 7f03566ba640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f03500b9150 con 0x7f03500b6820 2024-09-12T03:21:06.767 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:06.764+0000 7f034ffff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f03500b6440 0x7f03501f3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:06.767 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:06.764+0000 7f034ffff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f03500b6440 0x7f03501f3de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38682/0 (socket says 172.21.15.27:38682) 2024-09-12T03:21:06.767 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:06.764+0000 7f034ffff640 1 -- 172.21.15.27:0/4213668931 learned_addr learned my addr 172.21.15.27:0/4213668931 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:21:06.767 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:06.764+0000 7f0354c57640 1 --2- 172.21.15.27:0/4213668931 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f03500d3810 0x7f0350147b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:06.767 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:06.764+0000 7f034f7fe640 1 --2- 172.21.15.27:0/4213668931 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f03500b6820 0x7f03500ca8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:06.767 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:06.764+0000 7f034ffff640 1 -- 172.21.15.27:0/4213668931 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f03500b6820 msgr2=0x7f03500ca8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:06.767 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:06.764+0000 7f034ffff640 1 --2- 172.21.15.27:0/4213668931 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f03500b6820 0x7f03500ca8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:06.767 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:06.764+0000 7f034ffff640 1 -- 172.21.15.27:0/4213668931 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f03500d3810 msgr2=0x7f0350147b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:06.767 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:06.764+0000 7f034ffff640 1 --2- 172.21.15.27:0/4213668931 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f03500d3810 0x7f0350147b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:06.767 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:06.764+0000 7f034ffff640 1 -- 172.21.15.27:0/4213668931 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0350146ae0 con 0x7f03500b6440 2024-09-12T03:21:06.767 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:06.764+0000 7f034ffff640 1 --2- 172.21.15.27:0/4213668931 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f03500b6440 0x7f03501f3de0 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f034000e960 tx=0x7f034000ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:06.767 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:06.765+0000 7f034cff9640 1 -- 172.21.15.27:0/4213668931 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0340017070 con 0x7f03500b6440 2024-09-12T03:21:06.768 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:06.765+0000 7f034cff9640 1 -- 172.21.15.27:0/4213668931 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f034000ca00 con 0x7f03500b6440 2024-09-12T03:21:06.768 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:06.765+0000 7f03566ba640 1 -- 172.21.15.27:0/4213668931 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0350146dc0 con 0x7f03500b6440 2024-09-12T03:21:06.768 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:06.765+0000 7f034cff9640 1 -- 172.21.15.27:0/4213668931 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f034000cdd0 con 0x7f03500b6440 2024-09-12T03:21:06.768 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:06.765+0000 7f03566ba640 1 -- 172.21.15.27:0/4213668931 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0350147660 con 0x7f03500b6440 2024-09-12T03:21:06.768 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:06.766+0000 7f03566ba640 1 -- 172.21.15.27:0/4213668931 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0350162840 con 0x7f03500b6440 2024-09-12T03:21:06.768 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:06.766+0000 7f034cff9640 1 -- 172.21.15.27:0/4213668931 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 90600+0+0 (secure 0 0 0) 0x7f034001a070 con 0x7f03500b6440 2024-09-12T03:21:06.769 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:06.767+0000 7f034cff9640 1 --2- 172.21.15.27:0/4213668931 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f033c06bf50 0x7f033c06e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:06.769 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:06.767+0000 7f034cff9640 1 -- 172.21.15.27:0/4213668931 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5709+0+0 (secure 0 0 0) 0x7f034008b190 con 0x7f03500b6440 2024-09-12T03:21:06.769 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:06.767+0000 7f034f7fe640 1 --2- 172.21.15.27:0/4213668931 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f033c06bf50 0x7f033c06e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:06.769 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:06.767+0000 7f034f7fe640 1 --2- 172.21.15.27:0/4213668931 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f033c06bf50 0x7f033c06e410 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f0344034000 tx=0x7f034402e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:06.771 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:06.771+0000 7f034cff9640 1 -- 172.21.15.27:0/4213668931 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f03400593e0 con 0x7f03500b6440 2024-09-12T03:21:06.990 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:06.990+0000 7f03566ba640 1 -- 172.21.15.27:0/4213668931 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7f0350146f90 con 0x7f03500b6440 2024-09-12T03:21:07.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:07.823+0000 7f034cff9640 1 -- 172.21.15.27:0/4213668931 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v59) ==== 136+0+0 (secure 0 0 0) 0x7f034005d6e0 con 0x7f03500b6440 2024-09-12T03:21:07.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:07.826+0000 7f03566ba640 1 -- 172.21.15.27:0/4213668931 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f033c06bf50 msgr2=0x7f033c06e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:07.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:07.826+0000 7f03566ba640 1 --2- 172.21.15.27:0/4213668931 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f033c06bf50 0x7f033c06e410 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f0344034000 tx=0x7f034402e000 comp rx=0 tx=0).stop 2024-09-12T03:21:07.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:07.826+0000 7f03566ba640 1 -- 172.21.15.27:0/4213668931 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f03500b6440 msgr2=0x7f03501f3de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:07.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:07.826+0000 7f03566ba640 1 --2- 172.21.15.27:0/4213668931 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f03500b6440 0x7f03501f3de0 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f034000e960 tx=0x7f034000ee20 comp rx=0 tx=0).stop 2024-09-12T03:21:07.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:07.826+0000 7f03566ba640 1 -- 172.21.15.27:0/4213668931 shutdown_connections 2024-09-12T03:21:07.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:07.826+0000 7f03566ba640 1 --2- 172.21.15.27:0/4213668931 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f03500d3810 0x7f0350147b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:07.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:07.826+0000 7f03566ba640 1 --2- 172.21.15.27:0/4213668931 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f03500b6820 0x7f03500ca8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:07.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:07.826+0000 7f03566ba640 1 --2- 172.21.15.27:0/4213668931 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f033c06bf50 0x7f033c06e410 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:07.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:07.826+0000 7f03566ba640 1 --2- 172.21.15.27:0/4213668931 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f03500b6440 0x7f03501f3de0 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:07.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:07.826+0000 7f03566ba640 1 -- 172.21.15.27:0/4213668931 >> 172.21.15.27:0/4213668931 conn(0x7f03500d11d0 msgr2=0x7f03501581b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:07.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:07.826+0000 7f03566ba640 1 -- 172.21.15.27:0/4213668931 shutdown_connections 2024-09-12T03:21:07.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:07.826+0000 7f03566ba640 1 -- 172.21.15.27:0/4213668931 wait complete. 2024-09-12T03:21:07.842 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-12T03:21:07.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:07.956+0000 7fa3d3c65640 1 Processor -- start 2024-09-12T03:21:07.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:07.956+0000 7fa3d3c65640 1 -- start start 2024-09-12T03:21:07.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:07.957+0000 7fa3d3c65640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fa3cc0b5a70 0x7fa3cc15f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:07.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:07.957+0000 7fa3d3c65640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa3cc0ae8d0 0x7fa3cc0728b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:07.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:07.957+0000 7fa3d3c65640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa3cc15fb40 con 0x7fa3cc072df0 2024-09-12T03:21:07.959 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:07.957+0000 7fa3d3c65640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7fa3cc0aee90 con 0x7fa3cc0b5a70 2024-09-12T03:21:07.959 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:07.957+0000 7fa3d3c65640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fa3cc0af170 con 0x7fa3cc0ae8d0 2024-09-12T03:21:07.959 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:07.957+0000 7fa3d1a01640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fa3cc0b5a70 0x7fa3cc15f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:07.959 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:07.957+0000 7fa3d1a01640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fa3cc0b5a70 0x7fa3cc15f600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38690/0 (socket says 172.21.15.27:38690) 2024-09-12T03:21:07.959 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:07.957+0000 7fa3d1a01640 1 -- 172.21.15.27:0/1765724164 learned_addr learned my addr 172.21.15.27:0/1765724164 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:21:07.959 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:07.957+0000 7fa3d1200640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa3cc0ae8d0 0x7fa3cc0728b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:07.959 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:07.958+0000 7fa3d1a01640 1 -- 172.21.15.27:0/1765724164 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa3cc0ae8d0 msgr2=0x7fa3cc0728b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:07.959 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:07.958+0000 7fa3d1a01640 1 --2- 172.21.15.27:0/1765724164 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa3cc0ae8d0 0x7fa3cc0728b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:07.959 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:07.958+0000 7fa3d1a01640 1 -- 172.21.15.27:0/1765724164 >> v1:172.21.15.45:6789/0 conn(0x7fa3cc072df0 legacy=0x7fa3cc1621c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:07.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:07.958+0000 7fa3d1a01640 1 -- 172.21.15.27:0/1765724164 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3cc0af450 con 0x7fa3cc0b5a70 2024-09-12T03:21:07.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:07.958+0000 7fa3d1a01640 1 --2- 172.21.15.27:0/1765724164 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa3cc0b5a70 0x7fa3cc15f600 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7fa3c8009870 tx=0x7fa3c802a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d202ef743574e1b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:07.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:07.958+0000 7fa3d09ff640 1 -- 172.21.15.27:0/1765724164 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3c8037070 con 0x7fa3cc0b5a70 2024-09-12T03:21:07.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:07.958+0000 7fa3d09ff640 1 -- 172.21.15.27:0/1765724164 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3c8032440 con 0x7fa3cc0b5a70 2024-09-12T03:21:07.961 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:07.958+0000 7fa3d09ff640 1 -- 172.21.15.27:0/1765724164 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3c8032810 con 0x7fa3cc0b5a70 2024-09-12T03:21:07.961 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:07.959+0000 7fa3d3c65640 1 -- 172.21.15.27:0/1765724164 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa3cc0b5a70 msgr2=0x7fa3cc15f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:07.964 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:07.959+0000 7fa3d3c65640 1 --2- 172.21.15.27:0/1765724164 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa3cc0b5a70 0x7fa3cc15f600 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7fa3c8009870 tx=0x7fa3c802a1c0 comp rx=0 tx=0).stop 2024-09-12T03:21:07.964 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:07.959+0000 7fa3d3c65640 1 -- 172.21.15.27:0/1765724164 shutdown_connections 2024-09-12T03:21:07.964 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:07.959+0000 7fa3d3c65640 1 --2- 172.21.15.27:0/1765724164 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa3cc0ae8d0 0x7fa3cc0728b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:07.964 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:07.959+0000 7fa3d3c65640 1 --2- 172.21.15.27:0/1765724164 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa3cc0b5a70 0x7fa3cc15f600 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:07.964 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:07.959+0000 7fa3d3c65640 1 -- 172.21.15.27:0/1765724164 >> 172.21.15.27:0/1765724164 conn(0x7fa3cc06ca30 msgr2=0x7fa3cc06ce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:07.964 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:07.959+0000 7fa3d3c65640 1 -- 172.21.15.27:0/1765724164 shutdown_connections 2024-09-12T03:21:07.964 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:07.959+0000 7fa3d3c65640 1 -- 172.21.15.27:0/1765724164 wait complete. 2024-09-12T03:21:07.964 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:07.961+0000 7fa3d3c65640 1 Processor -- start 2024-09-12T03:21:07.964 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:07.961+0000 7fa3d3c65640 1 -- start start 2024-09-12T03:21:07.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:07.961+0000 7fa3d3c65640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa3cc072df0 0x7fa3cc14df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:07.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:07.961+0000 7fa3d3c65640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa3cc0ae8d0 0x7fa3cc15f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:07.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:07.961+0000 7fa3d3c65640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa3cc0b5a70 0x7fa3cc160d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:07.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:07.961+0000 7fa3d3c65640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fa3cc06fac0 con 0x7fa3cc072df0 2024-09-12T03:21:07.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:07.961+0000 7fa3d3c65640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa3cc06f940 con 0x7fa3cc0b5a70 2024-09-12T03:21:07.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:07.961+0000 7fa3d3c65640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fa3cc06fc40 con 0x7fa3cc0ae8d0 2024-09-12T03:21:07.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:07.961+0000 7fa3d1a01640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa3cc072df0 0x7fa3cc14df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:07.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:07.961+0000 7fa3d1a01640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa3cc072df0 0x7fa3cc14df50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38704/0 (socket says 172.21.15.27:38704) 2024-09-12T03:21:07.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:07.961+0000 7fa3d1a01640 1 -- 172.21.15.27:0/624321759 learned_addr learned my addr 172.21.15.27:0/624321759 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:21:07.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:07.962+0000 7fa3d1200640 1 --2- 172.21.15.27:0/624321759 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa3cc0ae8d0 0x7fa3cc15f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:07.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:07.962+0000 7fa3d2202640 1 --2- 172.21.15.27:0/624321759 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa3cc0b5a70 0x7fa3cc160d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:07.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:07.962+0000 7fa3d1a01640 1 -- 172.21.15.27:0/624321759 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa3cc0ae8d0 msgr2=0x7fa3cc15f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:07.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:07.962+0000 7fa3d1a01640 1 --2- 172.21.15.27:0/624321759 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa3cc0ae8d0 0x7fa3cc15f0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:07.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:07.962+0000 7fa3d1a01640 1 -- 172.21.15.27:0/624321759 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa3cc0b5a70 msgr2=0x7fa3cc160d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:07.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:07.962+0000 7fa3d1a01640 1 --2- 172.21.15.27:0/624321759 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa3cc0b5a70 0x7fa3cc160d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:07.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:07.962+0000 7fa3d1a01640 1 -- 172.21.15.27:0/624321759 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3cc152050 con 0x7fa3cc072df0 2024-09-12T03:21:07.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:07.962+0000 7fa3d1200640 1 --2- 172.21.15.27:0/624321759 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa3cc0ae8d0 0x7fa3cc15f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:21:07.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:07.962+0000 7fa3d1a01640 1 --2- 172.21.15.27:0/624321759 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa3cc072df0 0x7fa3cc14df50 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7fa3c8000c00 tx=0x7fa3c80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:07.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:07.962+0000 7fa3be7fc640 1 -- 172.21.15.27:0/624321759 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3c8037040 con 0x7fa3cc072df0 2024-09-12T03:21:07.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:07.962+0000 7fa3be7fc640 1 -- 172.21.15.27:0/624321759 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3c8042070 con 0x7fa3cc072df0 2024-09-12T03:21:07.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:07.962+0000 7fa3d3c65640 1 -- 172.21.15.27:0/624321759 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3cc152330 con 0x7fa3cc072df0 2024-09-12T03:21:07.967 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:07.962+0000 7fa3be7fc640 1 -- 172.21.15.27:0/624321759 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3c8030880 con 0x7fa3cc072df0 2024-09-12T03:21:07.968 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:07.962+0000 7fa3d3c65640 1 -- 172.21.15.27:0/624321759 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3cc1528f0 con 0x7fa3cc072df0 2024-09-12T03:21:07.968 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:07.963+0000 7fa3d3c65640 1 -- 172.21.15.27:0/624321759 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3cc0b4740 con 0x7fa3cc072df0 2024-09-12T03:21:07.968 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:07.964+0000 7fa3be7fc640 1 -- 172.21.15.27:0/624321759 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 90600+0+0 (secure 0 0 0) 0x7fa3c802ad10 con 0x7fa3cc072df0 2024-09-12T03:21:07.968 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:07.964+0000 7fa3be7fc640 1 --2- 172.21.15.27:0/624321759 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fa3b006bf80 0x7fa3b006e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:07.969 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:07.964+0000 7fa3be7fc640 1 -- 172.21.15.27:0/624321759 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5709+0+0 (secure 0 0 0) 0x7fa3c80ab440 con 0x7fa3cc072df0 2024-09-12T03:21:07.969 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:07.964+0000 7fa3d1200640 1 --2- 172.21.15.27:0/624321759 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fa3b006bf80 0x7fa3b006e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:07.969 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:07.965+0000 7fa3d1200640 1 --2- 172.21.15.27:0/624321759 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fa3b006bf80 0x7fa3b006e440 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fa3cc151e90 tx=0x7fa3c0008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:07.969 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:07.968+0000 7fa3be7fc640 1 -- 172.21.15.27:0/624321759 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fa3c8079d70 con 0x7fa3cc072df0 2024-09-12T03:21:08.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:08.167+0000 7fa3d3c65640 1 -- 172.21.15.27:0/624321759 --> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7fa3cc0b4180 con 0x7fa3b006bf80 2024-09-12T03:21:08.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:08.168+0000 7fa3be7fc640 1 -- 172.21.15.27:0/624321759 <== mgr.4113 v2:172.21.15.27:6836/1226920526 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7fa3cc0b4180 con 0x7fa3b006bf80 2024-09-12T03:21:08.169 INFO:teuthology.orchestra.run.smithi027.stdout:[ 2024-09-12T03:21:08.169 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:21:08.169 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "cephfs" 2024-09-12T03:21:08.169 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:21:08.169 INFO:teuthology.orchestra.run.smithi027.stdout:] 2024-09-12T03:21:08.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:08.171+0000 7fa3d3c65640 1 -- 172.21.15.27:0/624321759 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fa3b006bf80 msgr2=0x7fa3b006e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:08.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:08.171+0000 7fa3d3c65640 1 --2- 172.21.15.27:0/624321759 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fa3b006bf80 0x7fa3b006e440 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fa3cc151e90 tx=0x7fa3c0008040 comp rx=0 tx=0).stop 2024-09-12T03:21:08.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:08.171+0000 7fa3d3c65640 1 -- 172.21.15.27:0/624321759 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa3cc072df0 msgr2=0x7fa3cc14df50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:08.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:08.171+0000 7fa3d3c65640 1 --2- 172.21.15.27:0/624321759 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa3cc072df0 0x7fa3cc14df50 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7fa3c8000c00 tx=0x7fa3c80046c0 comp rx=0 tx=0).stop 2024-09-12T03:21:08.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:08.171+0000 7fa3d3c65640 1 -- 172.21.15.27:0/624321759 shutdown_connections 2024-09-12T03:21:08.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:08.171+0000 7fa3d3c65640 1 --2- 172.21.15.27:0/624321759 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa3cc0b5a70 0x7fa3cc160d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:08.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:08.171+0000 7fa3d3c65640 1 --2- 172.21.15.27:0/624321759 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa3cc0ae8d0 0x7fa3cc15f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:08.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:08.171+0000 7fa3d3c65640 1 --2- 172.21.15.27:0/624321759 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fa3b006bf80 0x7fa3b006e440 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:08.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:08.171+0000 7fa3d3c65640 1 --2- 172.21.15.27:0/624321759 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa3cc072df0 0x7fa3cc14df50 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:08.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:08.171+0000 7fa3d3c65640 1 -- 172.21.15.27:0/624321759 >> 172.21.15.27:0/624321759 conn(0x7fa3cc06ca30 msgr2=0x7fa3cc0b4c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:08.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:08.171+0000 7fa3d3c65640 1 -- 172.21.15.27:0/624321759 shutdown_connections 2024-09-12T03:21:08.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:08.171+0000 7fa3d3c65640 1 -- 172.21.15.27:0/624321759 wait complete. 2024-09-12T03:21:08.182 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-12T03:21:08.300 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:08.299+0000 7fb98e04c640 1 Processor -- start 2024-09-12T03:21:08.300 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:08.299+0000 7fb98e04c640 1 -- start start 2024-09-12T03:21:08.302 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:08.300+0000 7fb98e04c640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fb98815e530 0x7fb9880aff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:08.302 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:08.300+0000 7fb98e04c640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fb98815e910 0x7fb9880b4710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:08.302 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:08.300+0000 7fb98e04c640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb98806dc20 con 0x7fb9880aeb80 2024-09-12T03:21:08.302 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:08.300+0000 7fb98e04c640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7fb9880b07e0 con 0x7fb98815e910 2024-09-12T03:21:08.303 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:08.300+0000 7fb98e04c640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fb9880b4c50 con 0x7fb98815e530 2024-09-12T03:21:08.303 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:08.300+0000 7fb986ffd640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fb98815e910 0x7fb9880b4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:08.303 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:08.300+0000 7fb987fff640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fb9880aeb80 0x7fb9880722f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:47798/0 (socket says 172.21.15.27:47798) 2024-09-12T03:21:08.303 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:08.300+0000 7fb987fff640 1 -- 172.21.15.27:0/1041058816 learned_addr learned my addr 172.21.15.27:0/1041058816 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:21:08.303 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:08.300+0000 7fb9877fe640 1 --2- 172.21.15.27:0/1041058816 >> v2:172.21.15.45:3300/0 conn(0x7fb98815e530 0x7fb9880aff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:08.303 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:08.300+0000 7fb986ffd640 1 -- 172.21.15.27:0/1041058816 >> v2:172.21.15.45:3300/0 conn(0x7fb98815e530 msgr2=0x7fb9880aff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:08.304 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:08.300+0000 7fb986ffd640 1 --2- 172.21.15.27:0/1041058816 >> v2:172.21.15.45:3300/0 conn(0x7fb98815e530 0x7fb9880aff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:08.304 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:08.300+0000 7fb986ffd640 1 -- 172.21.15.27:0/1041058816 >> v1:172.21.15.27:6789/0 conn(0x7fb9880aeb80 legacy=0x7fb9880722f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:08.304 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:08.301+0000 7fb986ffd640 1 -- 172.21.15.27:0/1041058816 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9880b0500 con 0x7fb98815e910 2024-09-12T03:21:08.304 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:08.301+0000 7fb9867fc640 1 -- 172.21.15.27:0/1041058816 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1283073793 0 0) 0x7fb98806dc20 con 0x7fb9880aeb80 2024-09-12T03:21:08.304 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:08.301+0000 7fb986ffd640 1 --2- 172.21.15.27:0/1041058816 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb98815e910 0x7fb9880b4710 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7fb9880b3b40 tx=0x7fb97802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27b7e97f97a66965 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:08.304 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:08.301+0000 7fb9867fc640 1 -- 172.21.15.27:0/1041058816 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb978037070 con 0x7fb98815e910 2024-09-12T03:21:08.304 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:08.301+0000 7fb9867fc640 1 -- 172.21.15.27:0/1041058816 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb978032440 con 0x7fb98815e910 2024-09-12T03:21:08.304 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:08.301+0000 7fb9867fc640 1 -- 172.21.15.27:0/1041058816 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb978032810 con 0x7fb98815e910 2024-09-12T03:21:08.304 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:08.302+0000 7fb98e04c640 1 -- 172.21.15.27:0/1041058816 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb98815e910 msgr2=0x7fb9880b4710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:08.304 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:08.302+0000 7fb98e04c640 1 --2- 172.21.15.27:0/1041058816 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb98815e910 0x7fb9880b4710 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7fb9880b3b40 tx=0x7fb97802a1a0 comp rx=0 tx=0).stop 2024-09-12T03:21:08.305 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:08.302+0000 7fb98e04c640 1 -- 172.21.15.27:0/1041058816 shutdown_connections 2024-09-12T03:21:08.305 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:08.302+0000 7fb98e04c640 1 --2- 172.21.15.27:0/1041058816 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb98815e910 0x7fb9880b4710 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:08.305 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:08.302+0000 7fb98e04c640 1 --2- 172.21.15.27:0/1041058816 >> v2:172.21.15.45:3300/0 conn(0x7fb98815e530 0x7fb9880aff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:08.305 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:08.302+0000 7fb98e04c640 1 -- 172.21.15.27:0/1041058816 >> 172.21.15.27:0/1041058816 conn(0x7fb98806bb30 msgr2=0x7fb98806bf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:08.305 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:08.302+0000 7fb98e04c640 1 -- 172.21.15.27:0/1041058816 shutdown_connections 2024-09-12T03:21:08.305 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:08.302+0000 7fb98e04c640 1 -- 172.21.15.27:0/1041058816 wait complete. 2024-09-12T03:21:08.310 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:08.303+0000 7fb98e04c640 1 Processor -- start 2024-09-12T03:21:08.311 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:08.303+0000 7fb98e04c640 1 -- start start 2024-09-12T03:21:08.311 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:08.304+0000 7fb98e04c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb9880aeb80 0x7fb988144650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:08.311 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:08.304+0000 7fb98e04c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb98815e530 0x7fb9880cdc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:08.311 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:08.304+0000 7fb98e04c640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb98815e910 0x7fb9880b0d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:08.311 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:08.304+0000 7fb98e04c640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fb988071ff0 con 0x7fb9880aeb80 2024-09-12T03:21:08.311 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:08.304+0000 7fb98e04c640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb98806e1a0 con 0x7fb98815e530 2024-09-12T03:21:08.311 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:08.304+0000 7fb98e04c640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fb988072a00 con 0x7fb98815e910 2024-09-12T03:21:08.311 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:08.304+0000 7fb9877fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb9880aeb80 0x7fb988144650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:08.311 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:08.304+0000 7fb9877fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb9880aeb80 0x7fb988144650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38718/0 (socket says 172.21.15.27:38718) 2024-09-12T03:21:08.311 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:08.304+0000 7fb9877fe640 1 -- 172.21.15.27:0/2716117185 learned_addr learned my addr 172.21.15.27:0/2716117185 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:21:08.311 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:08.304+0000 7fb986ffd640 1 --2- 172.21.15.27:0/2716117185 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb98815e530 0x7fb9880cdc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:08.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:08.304+0000 7fb987fff640 1 --2- 172.21.15.27:0/2716117185 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb98815e910 0x7fb9880b0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:08.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:08.304+0000 7fb9877fe640 1 -- 172.21.15.27:0/2716117185 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb98815e910 msgr2=0x7fb9880b0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:08.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:08.305+0000 7fb9877fe640 1 --2- 172.21.15.27:0/2716117185 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb98815e910 0x7fb9880b0d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:08.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:08.305+0000 7fb9877fe640 1 -- 172.21.15.27:0/2716117185 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb98815e530 msgr2=0x7fb9880cdc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:08.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:08.305+0000 7fb9877fe640 1 --2- 172.21.15.27:0/2716117185 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb98815e530 0x7fb9880cdc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:08.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:08.305+0000 7fb9877fe640 1 -- 172.21.15.27:0/2716117185 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9880ce530 con 0x7fb9880aeb80 2024-09-12T03:21:08.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:08.305+0000 7fb986ffd640 1 --2- 172.21.15.27:0/2716117185 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb98815e530 0x7fb9880cdc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:21:08.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:08.305+0000 7fb987fff640 1 --2- 172.21.15.27:0/2716117185 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb98815e910 0x7fb9880b0d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:21:08.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:08.305+0000 7fb9877fe640 1 --2- 172.21.15.27:0/2716117185 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb9880aeb80 0x7fb988144650 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7fb97c00d830 tx=0x7fb97c00dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:08.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:08.305+0000 7fb96ffff640 1 -- 172.21.15.27:0/2716117185 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb97c00b6c0 con 0x7fb9880aeb80 2024-09-12T03:21:08.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:08.305+0000 7fb96ffff640 1 -- 172.21.15.27:0/2716117185 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb97c019070 con 0x7fb9880aeb80 2024-09-12T03:21:08.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:08.305+0000 7fb98e04c640 1 -- 172.21.15.27:0/2716117185 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9880ce810 con 0x7fb9880aeb80 2024-09-12T03:21:08.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:08.305+0000 7fb96ffff640 1 -- 172.21.15.27:0/2716117185 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb97c00ba80 con 0x7fb9880aeb80 2024-09-12T03:21:08.314 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:08.305+0000 7fb98e04c640 1 -- 172.21.15.27:0/2716117185 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9880cedd0 con 0x7fb9880aeb80 2024-09-12T03:21:08.314 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:08.306+0000 7fb98e04c640 1 -- 172.21.15.27:0/2716117185 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9880aff20 con 0x7fb9880aeb80 2024-09-12T03:21:08.314 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:08.306+0000 7fb96ffff640 1 -- 172.21.15.27:0/2716117185 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 90600+0+0 (secure 0 0 0) 0x7fb97c01e460 con 0x7fb9880aeb80 2024-09-12T03:21:08.314 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:08.307+0000 7fb96ffff640 1 --2- 172.21.15.27:0/2716117185 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fb97406bf50 0x7fb97406e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:08.314 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:08.307+0000 7fb96ffff640 1 -- 172.21.15.27:0/2716117185 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5709+0+0 (secure 0 0 0) 0x7fb97c08b720 con 0x7fb9880aeb80 2024-09-12T03:21:08.314 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:08.307+0000 7fb986ffd640 1 --2- 172.21.15.27:0/2716117185 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fb97406bf50 0x7fb97406e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:08.314 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:08.307+0000 7fb986ffd640 1 --2- 172.21.15.27:0/2716117185 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fb97406bf50 0x7fb97406e410 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fb978034000 tx=0x7fb97802e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:08.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:08.311+0000 7fb96ffff640 1 -- 172.21.15.27:0/2716117185 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fb97c059fc0 con 0x7fb9880aeb80 2024-09-12T03:21:08.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:08.485+0000 7fb98e04c640 1 -- 172.21.15.27:0/2716117185 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fb9881631a0 con 0x7fb9880aeb80 2024-09-12T03:21:08.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:08.486+0000 7fb96ffff640 1 -- 172.21.15.27:0/2716117185 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v20)=0 v20) ==== 122+0+0 (secure 0 0 0) 0x7fb97c0598c0 con 0x7fb9880aeb80 2024-09-12T03:21:08.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:08.487+0000 7fb98e04c640 1 -- 172.21.15.27:0/2716117185 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fb97406bf50 msgr2=0x7fb97406e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:08.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:08.487+0000 7fb98e04c640 1 --2- 172.21.15.27:0/2716117185 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fb97406bf50 0x7fb97406e410 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fb978034000 tx=0x7fb97802e000 comp rx=0 tx=0).stop 2024-09-12T03:21:08.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:08.487+0000 7fb98e04c640 1 -- 172.21.15.27:0/2716117185 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb9880aeb80 msgr2=0x7fb988144650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:08.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:08.487+0000 7fb98e04c640 1 --2- 172.21.15.27:0/2716117185 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb9880aeb80 0x7fb988144650 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7fb97c00d830 tx=0x7fb97c00dcf0 comp rx=0 tx=0).stop 2024-09-12T03:21:08.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:08.488+0000 7fb98e04c640 1 -- 172.21.15.27:0/2716117185 shutdown_connections 2024-09-12T03:21:08.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:08.488+0000 7fb98e04c640 1 --2- 172.21.15.27:0/2716117185 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb98815e910 0x7fb9880b0d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:08.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:08.488+0000 7fb98e04c640 1 --2- 172.21.15.27:0/2716117185 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb98815e530 0x7fb9880cdc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:08.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:08.488+0000 7fb98e04c640 1 --2- 172.21.15.27:0/2716117185 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fb97406bf50 0x7fb97406e410 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:08.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:08.488+0000 7fb98e04c640 1 --2- 172.21.15.27:0/2716117185 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb9880aeb80 0x7fb988144650 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:08.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:08.488+0000 7fb98e04c640 1 -- 172.21.15.27:0/2716117185 >> 172.21.15.27:0/2716117185 conn(0x7fb98806bb30 msgr2=0x7fb9880722d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:08.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:08.488+0000 7fb98e04c640 1 -- 172.21.15.27:0/2716117185 shutdown_connections 2024-09-12T03:21:08.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:08.488+0000 7fb98e04c640 1 -- 172.21.15.27:0/2716117185 wait complete. 2024-09-12T03:21:08.499 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:21:08.499 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p volumes/_nogroup/subvol84490/a842cae4-61b6-4f73-a96f-03c1581e2365) 2024-09-12T03:21:08.531 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:21:08.531 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p volumes/_nogroup/subvol84490/a842cae4-61b6-4f73-a96f-03c1581e2365/.snap/fake) 2024-09-12T03:21:08.607 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:21:08.607 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.layout.pool .) 2024-09-12T03:21:08.665 INFO:teuthology.orchestra.run.smithi027.stdout:cephfs_data 2024-09-12T03:21:08.665 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:21:08.665 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo setfattr -n ceph.dir.layout.pool -v cephfs_data volumes/_nogroup/subvol84490/a842cae4-61b6-4f73-a96f-03c1581e2365) 2024-09-12T03:21:08.735 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-12T03:21:08.735 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd of=/home/ubuntu/cephtest/mnt.0/volumes/_nogroup/subvol84490/.meta 2024-09-12T03:21:08.805 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol84490 2024-09-12T03:21:08.950 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:08.950+0000 7f35d6223640 1 Processor -- start 2024-09-12T03:21:08.950 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:08.950+0000 7f35d6223640 1 -- start start 2024-09-12T03:21:08.951 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:08.951+0000 7f35d6223640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f35d00af9d0 0x7f35d00b4010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:08.951 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:08.951+0000 7f35d6223640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f35d006da00 0x7f35d006deb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:08.951 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:08.951+0000 7f35d6223640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f35d00ce5e0 con 0x7f35d015ec30 2024-09-12T03:21:08.951 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:08.951+0000 7f35d6223640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f35d00d02d0 con 0x7f35d006da00 2024-09-12T03:21:08.951 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:08.951+0000 7f35d6223640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f35d006c650 con 0x7f35d00af9d0 2024-09-12T03:21:08.951 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:08.951+0000 7f35cffff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f35d006da00 0x7f35d006deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:08.952 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:08.951+0000 7f35cf7fe640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f35d015ec30 0x7f35d00b3900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:47814/0 (socket says 172.21.15.27:47814) 2024-09-12T03:21:08.952 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:08.951+0000 7f35cf7fe640 1 -- 172.21.15.27:0/1899312263 learned_addr learned my addr 172.21.15.27:0/1899312263 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:21:08.952 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:08.951+0000 7f35cffff640 1 -- 172.21.15.27:0/1899312263 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f35d00af9d0 msgr2=0x7f35d00b4010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:08.952 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:08.951+0000 7f35cffff640 1 --2- 172.21.15.27:0/1899312263 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f35d00af9d0 0x7f35d00b4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:08.952 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:08.951+0000 7f35cffff640 1 -- 172.21.15.27:0/1899312263 >> v1:172.21.15.27:6789/0 conn(0x7f35d015ec30 legacy=0x7f35d00b3900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:08.952 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:08.951+0000 7f35cffff640 1 -- 172.21.15.27:0/1899312263 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35d00ceac0 con 0x7f35d006da00 2024-09-12T03:21:08.952 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:08.951+0000 7f35ce7fc640 1 -- 172.21.15.27:0/1899312263 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2634976112 0 0) 0x7f35d00ce5e0 con 0x7f35d015ec30 2024-09-12T03:21:08.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:08.952+0000 7f35cffff640 1 --2- 172.21.15.27:0/1899312263 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f35d006da00 0x7f35d006deb0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f35c8009d90 tx=0x7f35c802a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d2ecf7d83b6eb04 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:08.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:08.952+0000 7f35ce7fc640 1 -- 172.21.15.27:0/1899312263 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35c8007740 con 0x7f35d006da00 2024-09-12T03:21:08.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:08.952+0000 7f35ce7fc640 1 -- 172.21.15.27:0/1899312263 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35c80078e0 con 0x7f35d006da00 2024-09-12T03:21:08.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:08.952+0000 7f35ce7fc640 1 -- 172.21.15.27:0/1899312263 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35c8007cb0 con 0x7f35d006da00 2024-09-12T03:21:08.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:08.953+0000 7f35d6223640 1 -- 172.21.15.27:0/1899312263 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f35d006da00 msgr2=0x7f35d006deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:08.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:08.953+0000 7f35d6223640 1 --2- 172.21.15.27:0/1899312263 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f35d006da00 0x7f35d006deb0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f35c8009d90 tx=0x7f35c802a660 comp rx=0 tx=0).stop 2024-09-12T03:21:08.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:08.953+0000 7f35d6223640 1 -- 172.21.15.27:0/1899312263 shutdown_connections 2024-09-12T03:21:08.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:08.953+0000 7f35d6223640 1 --2- 172.21.15.27:0/1899312263 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f35d006da00 0x7f35d006deb0 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:08.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:08.953+0000 7f35d6223640 1 --2- 172.21.15.27:0/1899312263 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f35d00af9d0 0x7f35d00b4010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:08.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:08.953+0000 7f35d6223640 1 -- 172.21.15.27:0/1899312263 >> 172.21.15.27:0/1899312263 conn(0x7f35d006bb30 msgr2=0x7f35d006bf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:08.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:08.953+0000 7f35d6223640 1 -- 172.21.15.27:0/1899312263 shutdown_connections 2024-09-12T03:21:08.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:08.953+0000 7f35d6223640 1 -- 172.21.15.27:0/1899312263 wait complete. 2024-09-12T03:21:08.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:08.954+0000 7f35d6223640 1 Processor -- start 2024-09-12T03:21:08.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:08.954+0000 7f35d6223640 1 -- start start 2024-09-12T03:21:08.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:08.955+0000 7f35d6223640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f35d006da00 0x7f35d01f5f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:08.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:08.955+0000 7f35d6223640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35d00af9d0 0x7f35d01569a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:08.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:08.955+0000 7f35d6223640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f35d015ec30 0x7f35d0149ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:08.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:08.955+0000 7f35d6223640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f35d00cf140 con 0x7f35d006da00 2024-09-12T03:21:08.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:08.955+0000 7f35d6223640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f35d00cefc0 con 0x7f35d00af9d0 2024-09-12T03:21:08.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:08.955+0000 7f35d6223640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f35d00cf2c0 con 0x7f35d015ec30 2024-09-12T03:21:08.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:08.955+0000 7f35cf7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f35d006da00 0x7f35d01f5f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:08.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:08.955+0000 7f35cf7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f35d006da00 0x7f35d01f5f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38740/0 (socket says 172.21.15.27:38740) 2024-09-12T03:21:08.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:08.955+0000 7f35cf7fe640 1 -- 172.21.15.27:0/2899224961 learned_addr learned my addr 172.21.15.27:0/2899224961 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:21:08.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:08.955+0000 7f35cffff640 1 --2- 172.21.15.27:0/2899224961 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f35d015ec30 0x7f35d0149ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:08.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:08.955+0000 7f35ceffd640 1 --2- 172.21.15.27:0/2899224961 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35d00af9d0 0x7f35d01569a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:08.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:08.955+0000 7f35cf7fe640 1 -- 172.21.15.27:0/2899224961 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f35d015ec30 msgr2=0x7f35d0149ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:08.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:08.955+0000 7f35cf7fe640 1 --2- 172.21.15.27:0/2899224961 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f35d015ec30 0x7f35d0149ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:08.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:08.955+0000 7f35cf7fe640 1 -- 172.21.15.27:0/2899224961 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35d00af9d0 msgr2=0x7f35d01569a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:08.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:08.955+0000 7f35cf7fe640 1 --2- 172.21.15.27:0/2899224961 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35d00af9d0 0x7f35d01569a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:08.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:08.955+0000 7f35cf7fe640 1 -- 172.21.15.27:0/2899224961 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35d01f64f0 con 0x7f35d006da00 2024-09-12T03:21:08.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:08.955+0000 7f35cf7fe640 1 --2- 172.21.15.27:0/2899224961 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f35d006da00 0x7f35d01f5f10 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f35c000c460 tx=0x7f35c000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:08.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:08.956+0000 7f35b7fff640 1 -- 172.21.15.27:0/2899224961 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35c0018070 con 0x7f35d006da00 2024-09-12T03:21:08.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:08.956+0000 7f35b7fff640 1 -- 172.21.15.27:0/2899224961 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35c0007500 con 0x7f35d006da00 2024-09-12T03:21:08.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:08.956+0000 7f35d6223640 1 -- 172.21.15.27:0/2899224961 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35d01f67d0 con 0x7f35d006da00 2024-09-12T03:21:08.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:08.956+0000 7f35b7fff640 1 -- 172.21.15.27:0/2899224961 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35c0004a20 con 0x7f35d006da00 2024-09-12T03:21:08.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:08.956+0000 7f35d6223640 1 -- 172.21.15.27:0/2899224961 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35d01f6d90 con 0x7f35d006da00 2024-09-12T03:21:08.959 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:08.957+0000 7f35d6223640 1 -- 172.21.15.27:0/2899224961 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35d00695a0 con 0x7f35d006da00 2024-09-12T03:21:08.959 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:08.957+0000 7f35b7fff640 1 -- 172.21.15.27:0/2899224961 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 90600+0+0 (secure 0 0 0) 0x7f35c001b070 con 0x7f35d006da00 2024-09-12T03:21:08.959 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:08.958+0000 7f35b7fff640 1 --2- 172.21.15.27:0/2899224961 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f35bc06bf50 0x7f35bc06e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:08.959 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:08.958+0000 7f35b7fff640 1 -- 172.21.15.27:0/2899224961 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5709+0+0 (secure 0 0 0) 0x7f35c008a260 con 0x7f35d006da00 2024-09-12T03:21:08.959 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:08.958+0000 7f35ceffd640 1 --2- 172.21.15.27:0/2899224961 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f35bc06bf50 0x7f35bc06e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:08.959 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:08.958+0000 7f35ceffd640 1 --2- 172.21.15.27:0/2899224961 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f35bc06bf50 0x7f35bc06e410 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f35d0151250 tx=0x7f35c4008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:08.961 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:08.962+0000 7f35b7fff640 1 -- 172.21.15.27:0/2899224961 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f35c0058b00 con 0x7f35d006da00 2024-09-12T03:21:09.162 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:09.163+0000 7f35d6223640 1 -- 172.21.15.27:0/2899224961 --> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol84490", "target": ["mon-mgr", ""]}) -- 0x7f35d015e020 con 0x7f35bc06bf50 2024-09-12T03:21:09.166 INFO:tasks.ceph.mgr.y.smithi027.stderr:2024-09-12T03:21:09.166+0000 7f325d8e3640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:21:09.166 INFO:tasks.ceph.mgr.y.smithi027.stderr:2024-09-12T03:21:09.166+0000 7f325d8e3640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:21:09.166 INFO:tasks.ceph.mgr.y.smithi027.stderr:2024-09-12T03:21:09.166+0000 7f325d8e3640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:21:09.167 INFO:tasks.ceph.mgr.y.smithi027.stderr:2024-09-12T03:21:09.166+0000 7f325d8e3640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:21:09.167 INFO:tasks.ceph.mgr.y.smithi027.stderr:2024-09-12T03:21:09.166+0000 7f325d8e3640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:21:09.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:09.200+0000 7f35b7fff640 1 -- 172.21.15.27:0/2899224961 <== mgr.4113 v2:172.21.15.27:6836/1226920526 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+67 (secure 0 0 0) 0x7f35d015e020 con 0x7f35bc06bf50 2024-09-12T03:21:09.200 INFO:teuthology.orchestra.run.smithi027.stdout:/volumes/_nogroup/subvol84490/a842cae4-61b6-4f73-a96f-03c1581e2365 2024-09-12T03:21:09.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:09.202+0000 7f35d6223640 1 -- 172.21.15.27:0/2899224961 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f35bc06bf50 msgr2=0x7f35bc06e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:09.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:09.202+0000 7f35d6223640 1 --2- 172.21.15.27:0/2899224961 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f35bc06bf50 0x7f35bc06e410 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f35d0151250 tx=0x7f35c4008040 comp rx=0 tx=0).stop 2024-09-12T03:21:09.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:09.202+0000 7f35d6223640 1 -- 172.21.15.27:0/2899224961 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f35d006da00 msgr2=0x7f35d01f5f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:09.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:09.202+0000 7f35d6223640 1 --2- 172.21.15.27:0/2899224961 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f35d006da00 0x7f35d01f5f10 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f35c000c460 tx=0x7f35c000c920 comp rx=0 tx=0).stop 2024-09-12T03:21:09.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:09.202+0000 7f35d6223640 1 -- 172.21.15.27:0/2899224961 shutdown_connections 2024-09-12T03:21:09.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:09.202+0000 7f35d6223640 1 --2- 172.21.15.27:0/2899224961 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f35d015ec30 0x7f35d0149ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:09.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:09.202+0000 7f35d6223640 1 --2- 172.21.15.27:0/2899224961 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35d00af9d0 0x7f35d01569a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:09.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:09.202+0000 7f35d6223640 1 --2- 172.21.15.27:0/2899224961 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f35bc06bf50 0x7f35bc06e410 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:09.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:09.202+0000 7f35d6223640 1 --2- 172.21.15.27:0/2899224961 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f35d006da00 0x7f35d01f5f10 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:09.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:09.202+0000 7f35d6223640 1 -- 172.21.15.27:0/2899224961 >> 172.21.15.27:0/2899224961 conn(0x7f35d006bb30 msgr2=0x7f35d0071f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:09.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:09.202+0000 7f35d6223640 1 -- 172.21.15.27:0/2899224961 shutdown_connections 2024-09-12T03:21:09.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:09.202+0000 7f35d6223640 1 -- 172.21.15.27:0/2899224961 wait complete. 2024-09-12T03:21:09.213 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume ls cephfs 2024-09-12T03:21:09.318 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:09.317+0000 7f0073ce6640 1 Processor -- start 2024-09-12T03:21:09.318 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:09.318+0000 7f0073ce6640 1 -- start start 2024-09-12T03:21:09.319 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:09.319+0000 7f0073ce6640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f006c162040 0x7f006c0d1da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:09.319 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:09.319+0000 7f0073ce6640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f006c0d3790 0x7f006c0b7a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:09.319 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:09.319+0000 7f0073ce6640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f006c06d8c0 0x7f006c06dd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:09.319 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:09.319+0000 7f0073ce6640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f006c0b7f90 con 0x7f006c06d8c0 2024-09-12T03:21:09.319 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:09.319+0000 7f0073ce6640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f006c0d2e30 con 0x7f006c0d3790 2024-09-12T03:21:09.319 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:09.319+0000 7f0073ce6640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f006c162420 con 0x7f006c162040 2024-09-12T03:21:09.320 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:09.319+0000 7f0072283640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f006c06d8c0 0x7f006c06dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:09.320 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:09.319+0000 7f0072283640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f006c06d8c0 0x7f006c06dd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57338/0 (socket says 172.21.15.27:57338) 2024-09-12T03:21:09.320 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:09.319+0000 7f0072283640 1 -- 172.21.15.27:0/2176022854 learned_addr learned my addr 172.21.15.27:0/2176022854 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:21:09.320 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:09.319+0000 7f0071a82640 1 --2- 172.21.15.27:0/2176022854 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f006c162040 0x7f006c0d1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:09.320 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:09.319+0000 7f0071281640 1 --2- 172.21.15.27:0/2176022854 >> v2:172.21.15.45:3300/0 conn(0x7f006c0d3790 0x7f006c0b7a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:09.320 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:09.319+0000 7f0072283640 1 -- 172.21.15.27:0/2176022854 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f006c162040 msgr2=0x7f006c0d1da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:09.320 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:09.319+0000 7f0072283640 1 --2- 172.21.15.27:0/2176022854 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f006c162040 0x7f006c0d1da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:09.320 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:09.319+0000 7f0072283640 1 -- 172.21.15.27:0/2176022854 >> v2:172.21.15.45:3300/0 conn(0x7f006c0d3790 msgr2=0x7f006c0b7a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:09.320 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:09.319+0000 7f0072283640 1 --2- 172.21.15.27:0/2176022854 >> v2:172.21.15.45:3300/0 conn(0x7f006c0d3790 0x7f006c0b7a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:09.320 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:09.319+0000 7f0072283640 1 -- 172.21.15.27:0/2176022854 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f006c0d30d0 con 0x7f006c06d8c0 2024-09-12T03:21:09.320 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:09.319+0000 7f0071a82640 1 --2- 172.21.15.27:0/2176022854 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f006c162040 0x7f006c0d1da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:21:09.322 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:09.320+0000 7f0072283640 1 --2- 172.21.15.27:0/2176022854 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f006c06d8c0 0x7f006c06dd70 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f006c0d58b0 tx=0x7f006402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e22db2094353cbbd server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:09.322 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:09.320+0000 7f0070a80640 1 -- 172.21.15.27:0/2176022854 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0064036070 con 0x7f006c06d8c0 2024-09-12T03:21:09.322 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:09.320+0000 7f0070a80640 1 -- 172.21.15.27:0/2176022854 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0064031600 con 0x7f006c06d8c0 2024-09-12T03:21:09.322 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:09.320+0000 7f0070a80640 1 -- 172.21.15.27:0/2176022854 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00640319d0 con 0x7f006c06d8c0 2024-09-12T03:21:09.322 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:09.320+0000 7f0073ce6640 1 -- 172.21.15.27:0/2176022854 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f006c06d8c0 msgr2=0x7f006c06dd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:09.322 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:09.320+0000 7f0073ce6640 1 --2- 172.21.15.27:0/2176022854 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f006c06d8c0 0x7f006c06dd70 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f006c0d58b0 tx=0x7f006402a6a0 comp rx=0 tx=0).stop 2024-09-12T03:21:09.322 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:09.321+0000 7f0073ce6640 1 -- 172.21.15.27:0/2176022854 shutdown_connections 2024-09-12T03:21:09.322 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:09.321+0000 7f0073ce6640 1 --2- 172.21.15.27:0/2176022854 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f006c06d8c0 0x7f006c06dd70 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:09.322 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:09.321+0000 7f0073ce6640 1 --2- 172.21.15.27:0/2176022854 >> v2:172.21.15.45:3300/0 conn(0x7f006c0d3790 0x7f006c0b7a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:09.322 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:09.321+0000 7f0073ce6640 1 --2- 172.21.15.27:0/2176022854 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f006c162040 0x7f006c0d1da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:09.322 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:09.321+0000 7f0073ce6640 1 -- 172.21.15.27:0/2176022854 >> 172.21.15.27:0/2176022854 conn(0x7f006c0b6700 msgr2=0x7f006c0b6b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:09.323 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:09.321+0000 7f0073ce6640 1 -- 172.21.15.27:0/2176022854 shutdown_connections 2024-09-12T03:21:09.323 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:09.321+0000 7f0073ce6640 1 -- 172.21.15.27:0/2176022854 wait complete. 2024-09-12T03:21:09.323 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:09.322+0000 7f0073ce6640 1 Processor -- start 2024-09-12T03:21:09.323 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:09.322+0000 7f0073ce6640 1 -- start start 2024-09-12T03:21:09.325 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:09.323+0000 7f0073ce6640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f006c06d8c0 0x7f006c0ce390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:09.325 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:09.323+0000 7f0073ce6640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f006c0d3790 0x7f006c15f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:09.325 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:09.323+0000 7f0073ce6640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f006c162040 0x7f006c0cac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:09.325 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:09.323+0000 7f0073ce6640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f006c0d4530 con 0x7f006c06d8c0 2024-09-12T03:21:09.325 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:09.323+0000 7f0073ce6640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f006c0cee00 con 0x7f006c0d3790 2024-09-12T03:21:09.325 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:09.323+0000 7f0073ce6640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f006c0d4150 con 0x7f006c162040 2024-09-12T03:21:09.325 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:09.323+0000 7f0071a82640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f006c06d8c0 0x7f006c0ce390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:09.325 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:09.323+0000 7f0071a82640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f006c06d8c0 0x7f006c0ce390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57344/0 (socket says 172.21.15.27:57344) 2024-09-12T03:21:09.326 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:09.323+0000 7f0071a82640 1 -- 172.21.15.27:0/3030504220 learned_addr learned my addr 172.21.15.27:0/3030504220 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:21:09.326 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:09.323+0000 7f0071281640 1 --2- 172.21.15.27:0/3030504220 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f006c0d3790 0x7f006c15f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:09.326 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:09.323+0000 7f0072283640 1 --2- 172.21.15.27:0/3030504220 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f006c162040 0x7f006c0cac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:09.326 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:09.323+0000 7f0071a82640 1 -- 172.21.15.27:0/3030504220 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f006c162040 msgr2=0x7f006c0cac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:09.326 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:09.323+0000 7f0071a82640 1 --2- 172.21.15.27:0/3030504220 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f006c162040 0x7f006c0cac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:09.326 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:09.323+0000 7f0071a82640 1 -- 172.21.15.27:0/3030504220 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f006c0d3790 msgr2=0x7f006c15f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:09.326 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:09.323+0000 7f0071a82640 1 --2- 172.21.15.27:0/3030504220 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f006c0d3790 0x7f006c15f410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:09.326 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:09.323+0000 7f0071a82640 1 -- 172.21.15.27:0/3030504220 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f006c146a50 con 0x7f006c06d8c0 2024-09-12T03:21:09.326 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:09.323+0000 7f0071281640 1 --2- 172.21.15.27:0/3030504220 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f006c0d3790 0x7f006c15f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:21:09.326 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:09.323+0000 7f0071a82640 1 --2- 172.21.15.27:0/3030504220 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f006c06d8c0 0x7f006c0ce390 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f006800e990 tx=0x7f006800ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:09.326 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:09.323+0000 7f005e7fc640 1 -- 172.21.15.27:0/3030504220 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0068017070 con 0x7f006c06d8c0 2024-09-12T03:21:09.326 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:09.323+0000 7f005e7fc640 1 -- 172.21.15.27:0/3030504220 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f006800c9e0 con 0x7f006c06d8c0 2024-09-12T03:21:09.326 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:09.323+0000 7f0073ce6640 1 -- 172.21.15.27:0/3030504220 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f006c146d30 con 0x7f006c06d8c0 2024-09-12T03:21:09.326 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:09.323+0000 7f005e7fc640 1 -- 172.21.15.27:0/3030504220 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f006800cdb0 con 0x7f006c06d8c0 2024-09-12T03:21:09.327 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:09.323+0000 7f0073ce6640 1 -- 172.21.15.27:0/3030504220 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f006c1472f0 con 0x7f006c06d8c0 2024-09-12T03:21:09.327 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:09.324+0000 7f0073ce6640 1 -- 172.21.15.27:0/3030504220 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f006c0b7a50 con 0x7f006c06d8c0 2024-09-12T03:21:09.327 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:09.325+0000 7f005e7fc640 1 -- 172.21.15.27:0/3030504220 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 90600+0+0 (secure 0 0 0) 0x7f006801a070 con 0x7f006c06d8c0 2024-09-12T03:21:09.327 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:09.325+0000 7f005e7fc640 1 --2- 172.21.15.27:0/3030504220 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f005006bf80 0x7f005006e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:09.327 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:09.325+0000 7f0071281640 1 --2- 172.21.15.27:0/3030504220 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f005006bf80 0x7f005006e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:09.327 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:09.325+0000 7f005e7fc640 1 -- 172.21.15.27:0/3030504220 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5709+0+0 (secure 0 0 0) 0x7f006808af60 con 0x7f006c06d8c0 2024-09-12T03:21:09.327 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:09.326+0000 7f0071281640 1 --2- 172.21.15.27:0/3030504220 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f005006bf80 0x7f005006e440 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f006c0cd450 tx=0x7f0060008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:09.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:09.330+0000 7f005e7fc640 1 -- 172.21.15.27:0/3030504220 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f0068059770 con 0x7f006c06d8c0 2024-09-12T03:21:09.528 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:09.529+0000 7f0073ce6640 1 -- 172.21.15.27:0/3030504220 --> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] -- mgr_command(tid 0: {"prefix": "fs subvolume ls", "vol_name": "cephfs", "target": ["mon-mgr", ""]}) -- 0x7f006c15f950 con 0x7f005006bf80 2024-09-12T03:21:09.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:09.532+0000 7f005e7fc640 1 -- 172.21.15.27:0/3030504220 <== mgr.4113 v2:172.21.15.27:6836/1226920526 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+46 (secure 0 0 0) 0x7f006c15f950 con 0x7f005006bf80 2024-09-12T03:21:09.532 INFO:teuthology.orchestra.run.smithi027.stdout:[ 2024-09-12T03:21:09.532 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:21:09.532 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "subvol84490" 2024-09-12T03:21:09.532 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:21:09.532 INFO:teuthology.orchestra.run.smithi027.stdout:] 2024-09-12T03:21:09.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:09.534+0000 7f0073ce6640 1 -- 172.21.15.27:0/3030504220 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f005006bf80 msgr2=0x7f005006e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:09.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:09.534+0000 7f0073ce6640 1 --2- 172.21.15.27:0/3030504220 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f005006bf80 0x7f005006e440 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f006c0cd450 tx=0x7f0060008040 comp rx=0 tx=0).stop 2024-09-12T03:21:09.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:09.534+0000 7f0073ce6640 1 -- 172.21.15.27:0/3030504220 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f006c06d8c0 msgr2=0x7f006c0ce390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:09.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:09.534+0000 7f0073ce6640 1 --2- 172.21.15.27:0/3030504220 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f006c06d8c0 0x7f006c0ce390 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f006800e990 tx=0x7f006800ee50 comp rx=0 tx=0).stop 2024-09-12T03:21:09.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:09.534+0000 7f0073ce6640 1 -- 172.21.15.27:0/3030504220 shutdown_connections 2024-09-12T03:21:09.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:09.534+0000 7f0073ce6640 1 --2- 172.21.15.27:0/3030504220 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f006c162040 0x7f006c0cac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:09.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:09.534+0000 7f0073ce6640 1 --2- 172.21.15.27:0/3030504220 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f006c0d3790 0x7f006c15f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:09.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:09.534+0000 7f0073ce6640 1 --2- 172.21.15.27:0/3030504220 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f005006bf80 0x7f005006e440 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:09.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:09.534+0000 7f0073ce6640 1 --2- 172.21.15.27:0/3030504220 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f006c06d8c0 0x7f006c0ce390 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:09.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:09.534+0000 7f0073ce6640 1 -- 172.21.15.27:0/3030504220 >> 172.21.15.27:0/3030504220 conn(0x7f006c0b6700 msgr2=0x7f006c0b8ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:09.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:09.534+0000 7f0073ce6640 1 -- 172.21.15.27:0/3030504220 shutdown_connections 2024-09-12T03:21:09.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:09.534+0000 7f0073ce6640 1 -- 172.21.15.27:0/3030504220 wait complete. 2024-09-12T03:21:09.545 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume info cephfs subvol84490 2024-09-12T03:21:09.650 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:09.650+0000 7ff75e20e640 1 Processor -- start 2024-09-12T03:21:09.650 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:09.650+0000 7ff75e20e640 1 -- start start 2024-09-12T03:21:09.651 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:09.651+0000 7ff75e20e640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ff75806cc40 0x7ff75806d060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:09.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:09.651+0000 7ff75e20e640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7ff75806db30 0x7ff75806df80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:09.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:09.651+0000 7ff75e20e640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff75806d5a0 con 0x7ff75815e650 2024-09-12T03:21:09.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:09.651+0000 7ff75e20e640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7ff75806d850 con 0x7ff75806db30 2024-09-12T03:21:09.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:09.651+0000 7ff75e20e640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7ff7580d1260 con 0x7ff75806cc40 2024-09-12T03:21:09.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:09.651+0000 7ff757fff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7ff75806db30 0x7ff75806df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:09.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:09.651+0000 7ff7577fe640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7ff75815e650 0x7ff7580d1d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:40682/0 (socket says 172.21.15.27:40682) 2024-09-12T03:21:09.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:09.651+0000 7ff7577fe640 1 -- 172.21.15.27:0/2262110770 learned_addr learned my addr 172.21.15.27:0/2262110770 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:21:09.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:09.652+0000 7ff756ffd640 1 --2- 172.21.15.27:0/2262110770 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ff75806cc40 0x7ff75806d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:09.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:09.652+0000 7ff757fff640 1 -- 172.21.15.27:0/2262110770 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ff75806cc40 msgr2=0x7ff75806d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:09.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:09.652+0000 7ff757fff640 1 --2- 172.21.15.27:0/2262110770 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ff75806cc40 0x7ff75806d060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:09.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:09.652+0000 7ff757fff640 1 -- 172.21.15.27:0/2262110770 >> v1:172.21.15.27:6789/0 conn(0x7ff75815e650 legacy=0x7ff7580d1d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:09.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:09.652+0000 7ff757fff640 1 -- 172.21.15.27:0/2262110770 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7580d1540 con 0x7ff75806db30 2024-09-12T03:21:09.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:09.652+0000 7ff7567fc640 1 -- 172.21.15.27:0/2262110770 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4130247152 0 0) 0x7ff75806d5a0 con 0x7ff75815e650 2024-09-12T03:21:09.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:09.652+0000 7ff756ffd640 1 --2- 172.21.15.27:0/2262110770 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ff75806cc40 0x7ff75806d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:21:09.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:09.652+0000 7ff757fff640 1 --2- 172.21.15.27:0/2262110770 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff75806db30 0x7ff75806df80 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7ff750009d90 tx=0x7ff75002a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=655dd8c6d30fed19 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:09.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:09.653+0000 7ff7567fc640 1 -- 172.21.15.27:0/2262110770 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff750031600 con 0x7ff75806db30 2024-09-12T03:21:09.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:09.653+0000 7ff7567fc640 1 -- 172.21.15.27:0/2262110770 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7500317a0 con 0x7ff75806db30 2024-09-12T03:21:09.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:09.653+0000 7ff7567fc640 1 -- 172.21.15.27:0/2262110770 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff750031b70 con 0x7ff75806db30 2024-09-12T03:21:09.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:09.653+0000 7ff75e20e640 1 -- 172.21.15.27:0/2262110770 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff75806db30 msgr2=0x7ff75806df80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:09.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:09.653+0000 7ff75e20e640 1 --2- 172.21.15.27:0/2262110770 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff75806db30 0x7ff75806df80 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7ff750009d90 tx=0x7ff75002a5d0 comp rx=0 tx=0).stop 2024-09-12T03:21:09.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:09.653+0000 7ff75e20e640 1 -- 172.21.15.27:0/2262110770 shutdown_connections 2024-09-12T03:21:09.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:09.653+0000 7ff75e20e640 1 --2- 172.21.15.27:0/2262110770 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff75806db30 0x7ff75806df80 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:09.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:09.653+0000 7ff75e20e640 1 --2- 172.21.15.27:0/2262110770 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ff75806cc40 0x7ff75806d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:09.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:09.653+0000 7ff75e20e640 1 -- 172.21.15.27:0/2262110770 >> 172.21.15.27:0/2262110770 conn(0x7ff75806f320 msgr2=0x7ff75806f700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:09.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:09.653+0000 7ff75e20e640 1 -- 172.21.15.27:0/2262110770 shutdown_connections 2024-09-12T03:21:09.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:09.653+0000 7ff75e20e640 1 -- 172.21.15.27:0/2262110770 wait complete. 2024-09-12T03:21:09.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:09.654+0000 7ff75e20e640 1 Processor -- start 2024-09-12T03:21:09.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:09.655+0000 7ff75e20e640 1 -- start start 2024-09-12T03:21:09.657 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:09.655+0000 7ff75e20e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff75806cc40 0x7ff7581f4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:09.657 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:09.655+0000 7ff75e20e640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ff75806db30 0x7ff758151f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:09.657 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:09.655+0000 7ff75e20e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff75815e650 0x7ff7581f38d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:09.657 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:09.655+0000 7ff75e20e640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7ff75806e640 con 0x7ff75806cc40 2024-09-12T03:21:09.657 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:09.655+0000 7ff75e20e640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ff75806e4c0 con 0x7ff75815e650 2024-09-12T03:21:09.657 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:09.655+0000 7ff75e20e640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7ff7580b55b0 con 0x7ff75806db30 2024-09-12T03:21:09.657 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:09.655+0000 7ff7577fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff75806cc40 0x7ff7581f4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:09.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:09.655+0000 7ff7577fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff75806cc40 0x7ff7581f4bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57360/0 (socket says 172.21.15.27:57360) 2024-09-12T03:21:09.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:09.655+0000 7ff7577fe640 1 -- 172.21.15.27:0/4165879355 learned_addr learned my addr 172.21.15.27:0/4165879355 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:21:09.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:09.656+0000 7ff756ffd640 1 --2- 172.21.15.27:0/4165879355 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ff75806db30 0x7ff758151f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:09.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:09.656+0000 7ff757fff640 1 --2- 172.21.15.27:0/4165879355 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff75815e650 0x7ff7581f38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:09.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:09.656+0000 7ff7577fe640 1 -- 172.21.15.27:0/4165879355 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ff75806db30 msgr2=0x7ff758151f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:09.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:09.656+0000 7ff7577fe640 1 --2- 172.21.15.27:0/4165879355 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ff75806db30 0x7ff758151f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:09.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:09.656+0000 7ff7577fe640 1 -- 172.21.15.27:0/4165879355 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff75815e650 msgr2=0x7ff7581f38d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:09.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:09.656+0000 7ff7577fe640 1 --2- 172.21.15.27:0/4165879355 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff75815e650 0x7ff7581f38d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:09.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:09.656+0000 7ff7577fe640 1 -- 172.21.15.27:0/4165879355 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7581524a0 con 0x7ff75806cc40 2024-09-12T03:21:09.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:09.656+0000 7ff757fff640 1 --2- 172.21.15.27:0/4165879355 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff75815e650 0x7ff7581f38d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:21:09.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:09.656+0000 7ff7577fe640 1 --2- 172.21.15.27:0/4165879355 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff75806cc40 0x7ff7581f4bd0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7ff74800c430 tx=0x7ff74800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:09.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:09.656+0000 7ff73ffff640 1 -- 172.21.15.27:0/4165879355 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7480074e0 con 0x7ff75806cc40 2024-09-12T03:21:09.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:09.656+0000 7ff73ffff640 1 -- 172.21.15.27:0/4165879355 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7480047e0 con 0x7ff75806cc40 2024-09-12T03:21:09.659 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:09.656+0000 7ff73ffff640 1 -- 172.21.15.27:0/4165879355 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff748004bd0 con 0x7ff75806cc40 2024-09-12T03:21:09.659 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:09.656+0000 7ff75e20e640 1 -- 172.21.15.27:0/4165879355 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7581f3e10 con 0x7ff75806cc40 2024-09-12T03:21:09.659 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:09.656+0000 7ff75e20e640 1 -- 172.21.15.27:0/4165879355 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7581f79a0 con 0x7ff75806cc40 2024-09-12T03:21:09.659 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:09.657+0000 7ff75e20e640 1 -- 172.21.15.27:0/4165879355 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7580d36b0 con 0x7ff75806cc40 2024-09-12T03:21:09.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:09.658+0000 7ff73ffff640 1 -- 172.21.15.27:0/4165879355 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 90600+0+0 (secure 0 0 0) 0x7ff748011070 con 0x7ff75806cc40 2024-09-12T03:21:09.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:09.658+0000 7ff73ffff640 1 --2- 172.21.15.27:0/4165879355 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7ff74406bf50 0x7ff74406e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:09.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:09.658+0000 7ff73ffff640 1 -- 172.21.15.27:0/4165879355 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5709+0+0 (secure 0 0 0) 0x7ff74808ab70 con 0x7ff75806cc40 2024-09-12T03:21:09.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:09.658+0000 7ff756ffd640 1 --2- 172.21.15.27:0/4165879355 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7ff74406bf50 0x7ff74406e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:09.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:09.659+0000 7ff756ffd640 1 --2- 172.21.15.27:0/4165879355 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7ff74406bf50 0x7ff74406e410 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ff7581f5110 tx=0x7ff74c008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:09.662 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:09.662+0000 7ff73ffff640 1 -- 172.21.15.27:0/4165879355 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7ff7580d36b0 con 0x7ff75806cc40 2024-09-12T03:21:09.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:09.865+0000 7ff75e20e640 1 -- 172.21.15.27:0/4165879355 --> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] -- mgr_command(tid 0: {"prefix": "fs subvolume info", "vol_name": "cephfs", "sub_name": "subvol84490", "target": ["mon-mgr", ""]}) -- 0x7ff75806d2b0 con 0x7ff74406bf50 2024-09-12T03:21:09.886 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:09.886+0000 7ff73ffff640 1 -- 172.21.15.27:0/4165879355 <== mgr.4113 v2:172.21.15.27:6836/1226920526 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+684 (secure 0 0 0) 0x7ff75806d2b0 con 0x7ff74406bf50 2024-09-12T03:21:09.887 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-12T03:21:09.887 INFO:teuthology.orchestra.run.smithi027.stdout: "atime": "2024-09-12 03:21:08", 2024-09-12T03:21:09.887 INFO:teuthology.orchestra.run.smithi027.stdout: "bytes_pcent": "undefined", 2024-09-12T03:21:09.887 INFO:teuthology.orchestra.run.smithi027.stdout: "bytes_quota": "infinite", 2024-09-12T03:21:09.887 INFO:teuthology.orchestra.run.smithi027.stdout: "bytes_used": 0, 2024-09-12T03:21:09.887 INFO:teuthology.orchestra.run.smithi027.stdout: "created_at": "2024-09-12 03:21:08", 2024-09-12T03:21:09.887 INFO:teuthology.orchestra.run.smithi027.stdout: "ctime": "2024-09-12 03:21:09", 2024-09-12T03:21:09.888 INFO:teuthology.orchestra.run.smithi027.stdout: "data_pool": "cephfs_data", 2024-09-12T03:21:09.888 INFO:teuthology.orchestra.run.smithi027.stdout: "features": [ 2024-09-12T03:21:09.888 INFO:teuthology.orchestra.run.smithi027.stdout: "snapshot-clone", 2024-09-12T03:21:09.888 INFO:teuthology.orchestra.run.smithi027.stdout: "snapshot-autoprotect" 2024-09-12T03:21:09.888 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-12T03:21:09.889 INFO:teuthology.orchestra.run.smithi027.stdout: "flavor": 1, 2024-09-12T03:21:09.889 INFO:teuthology.orchestra.run.smithi027.stdout: "gid": 0, 2024-09-12T03:21:09.889 INFO:teuthology.orchestra.run.smithi027.stdout: "mode": 16877, 2024-09-12T03:21:09.889 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_addrs": [ 2024-09-12T03:21:09.889 INFO:teuthology.orchestra.run.smithi027.stdout: "172.21.15.27:6789", 2024-09-12T03:21:09.889 INFO:teuthology.orchestra.run.smithi027.stdout: "172.21.15.45:6789", 2024-09-12T03:21:09.889 INFO:teuthology.orchestra.run.smithi027.stdout: "172.21.15.45:6790" 2024-09-12T03:21:09.889 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-12T03:21:09.889 INFO:teuthology.orchestra.run.smithi027.stdout: "mtime": "2024-09-12 03:21:08", 2024-09-12T03:21:09.889 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/volumes/_nogroup/subvol84490/a842cae4-61b6-4f73-a96f-03c1581e2365", 2024-09-12T03:21:09.889 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_namespace": "", 2024-09-12T03:21:09.889 INFO:teuthology.orchestra.run.smithi027.stdout: "state": "complete", 2024-09-12T03:21:09.890 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "subvolume", 2024-09-12T03:21:09.890 INFO:teuthology.orchestra.run.smithi027.stdout: "uid": 0 2024-09-12T03:21:09.890 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-12T03:21:09.891 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:09.888+0000 7ff75e20e640 1 -- 172.21.15.27:0/4165879355 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7ff74406bf50 msgr2=0x7ff74406e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:09.891 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:09.888+0000 7ff75e20e640 1 --2- 172.21.15.27:0/4165879355 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7ff74406bf50 0x7ff74406e410 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ff7581f5110 tx=0x7ff74c008040 comp rx=0 tx=0).stop 2024-09-12T03:21:09.891 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:09.888+0000 7ff75e20e640 1 -- 172.21.15.27:0/4165879355 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff75806cc40 msgr2=0x7ff7581f4bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:09.891 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:09.888+0000 7ff75e20e640 1 --2- 172.21.15.27:0/4165879355 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff75806cc40 0x7ff7581f4bd0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7ff74800c430 tx=0x7ff74800c8f0 comp rx=0 tx=0).stop 2024-09-12T03:21:09.891 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:09.888+0000 7ff75e20e640 1 -- 172.21.15.27:0/4165879355 shutdown_connections 2024-09-12T03:21:09.891 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:09.888+0000 7ff75e20e640 1 --2- 172.21.15.27:0/4165879355 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff75815e650 0x7ff7581f38d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:09.891 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:09.888+0000 7ff75e20e640 1 --2- 172.21.15.27:0/4165879355 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ff75806db30 0x7ff758151f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:09.892 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:09.888+0000 7ff75e20e640 1 --2- 172.21.15.27:0/4165879355 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7ff74406bf50 0x7ff74406e410 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:09.892 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:09.888+0000 7ff75e20e640 1 --2- 172.21.15.27:0/4165879355 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff75806cc40 0x7ff7581f4bd0 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:09.892 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:09.888+0000 7ff75e20e640 1 -- 172.21.15.27:0/4165879355 >> 172.21.15.27:0/4165879355 conn(0x7ff75806f320 msgr2=0x7ff7580b3230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:09.892 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:09.889+0000 7ff75e20e640 1 -- 172.21.15.27:0/4165879355 shutdown_connections 2024-09-12T03:21:09.892 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:09.889+0000 7ff75e20e640 1 -- 172.21.15.27:0/4165879355 wait complete. 2024-09-12T03:21:09.899 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume resize cephfs subvol84490 10485760 2024-09-12T03:21:10.006 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.006+0000 7fec79359640 1 Processor -- start 2024-09-12T03:21:10.006 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.006+0000 7fec79359640 1 -- start start 2024-09-12T03:21:10.007 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.007+0000 7fec79359640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fec7415a270 0x7fec7406cff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:10.007 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.007+0000 7fec79359640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fec7406d530 0x7fec740cf410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:10.007 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.007+0000 7fec79359640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fec740ce650 con 0x7fec7406c650 2024-09-12T03:21:10.007 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.007+0000 7fec79359640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fec740cef80 con 0x7fec7415a270 2024-09-12T03:21:10.007 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.007+0000 7fec79359640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fec740ce930 con 0x7fec7406d530 2024-09-12T03:21:10.008 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.007+0000 7fec72ffd640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fec7415a270 0x7fec7406cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:10.008 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.007+0000 7fec727fc640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fec7406d530 0x7fec740cf410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:10.008 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.007+0000 7fec727fc640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fec7406d530 0x7fec740cf410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3301/0 says I am v2:172.21.15.27:34386/0 (socket says 172.21.15.27:34386) 2024-09-12T03:21:10.008 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.007+0000 7fec72ffd640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fec7415a270 0x7fec7406cff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.27:41642/0 (socket says 172.21.15.27:41642) 2024-09-12T03:21:10.008 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.007+0000 7fec727fc640 1 -- 172.21.15.27:0/2960338017 learned_addr learned my addr 172.21.15.27:0/2960338017 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:21:10.008 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.007+0000 7fec737fe640 1 --1- 172.21.15.27:0/2960338017 >> v1:172.21.15.45:6789/0 conn(0x7fec7406c650 0x7fec740726a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.27:43188/0 (socket says 172.21.15.27:43188) 2024-09-12T03:21:10.008 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.008+0000 7fec72ffd640 1 -- 172.21.15.27:0/2960338017 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fec7406d530 msgr2=0x7fec740cf410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:10.008 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.008+0000 7fec72ffd640 1 --2- 172.21.15.27:0/2960338017 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fec7406d530 0x7fec740cf410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:10.008 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.008+0000 7fec72ffd640 1 -- 172.21.15.27:0/2960338017 >> v1:172.21.15.45:6789/0 conn(0x7fec7406c650 legacy=0x7fec740726a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:10.009 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.008+0000 7fec72ffd640 1 -- 172.21.15.27:0/2960338017 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec740cec10 con 0x7fec7415a270 2024-09-12T03:21:10.009 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.008+0000 7fec727fc640 1 --2- 172.21.15.27:0/2960338017 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fec7406d530 0x7fec740cf410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:21:10.010 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.009+0000 7fec72ffd640 1 --2- 172.21.15.27:0/2960338017 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fec7415a270 0x7fec7406cff0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fec6c00dbc0 tx=0x7fec6c02e480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dd005a471050225a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:10.010 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.009+0000 7fec71ffb640 1 -- 172.21.15.27:0/2960338017 <== mon.1 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec6c03a070 con 0x7fec7415a270 2024-09-12T03:21:10.010 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.009+0000 7fec71ffb640 1 -- 172.21.15.27:0/2960338017 <== mon.1 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec6c00b650 con 0x7fec7415a270 2024-09-12T03:21:10.010 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.009+0000 7fec71ffb640 1 -- 172.21.15.27:0/2960338017 <== mon.1 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec6c00ba20 con 0x7fec7415a270 2024-09-12T03:21:10.010 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.009+0000 7fec79359640 1 -- 172.21.15.27:0/2960338017 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fec7415a270 msgr2=0x7fec7406cff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:10.010 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.009+0000 7fec79359640 1 --2- 172.21.15.27:0/2960338017 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fec7415a270 0x7fec7406cff0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fec6c00dbc0 tx=0x7fec6c02e480 comp rx=0 tx=0).stop 2024-09-12T03:21:10.010 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.009+0000 7fec79359640 1 -- 172.21.15.27:0/2960338017 shutdown_connections 2024-09-12T03:21:10.010 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.009+0000 7fec79359640 1 --2- 172.21.15.27:0/2960338017 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fec7406d530 0x7fec740cf410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:10.010 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.009+0000 7fec79359640 1 --2- 172.21.15.27:0/2960338017 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fec7415a270 0x7fec7406cff0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:10.010 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.009+0000 7fec79359640 1 -- 172.21.15.27:0/2960338017 >> 172.21.15.27:0/2960338017 conn(0x7fec7406bb30 msgr2=0x7fec7406bf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:10.011 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.009+0000 7fec79359640 1 -- 172.21.15.27:0/2960338017 shutdown_connections 2024-09-12T03:21:10.011 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.009+0000 7fec79359640 1 -- 172.21.15.27:0/2960338017 wait complete. 2024-09-12T03:21:10.013 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.011+0000 7fec79359640 1 Processor -- start 2024-09-12T03:21:10.013 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.011+0000 7fec79359640 1 -- start start 2024-09-12T03:21:10.013 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.011+0000 7fec79359640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fec7406c650 0x7fec74147e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:10.013 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.011+0000 7fec79359640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fec7406d530 0x7fec7415f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:10.013 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.011+0000 7fec79359640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fec7415a270 0x7fec74158400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:10.013 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.012+0000 7fec79359640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fec74072f30 con 0x7fec7406c650 2024-09-12T03:21:10.013 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.012+0000 7fec79359640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fec74072db0 con 0x7fec7406d530 2024-09-12T03:21:10.014 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.012+0000 7fec79359640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fec740730b0 con 0x7fec7415a270 2024-09-12T03:21:10.014 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.012+0000 7fec72ffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fec7406c650 0x7fec74147e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:10.014 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.012+0000 7fec72ffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fec7406c650 0x7fec74147e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57372/0 (socket says 172.21.15.27:57372) 2024-09-12T03:21:10.014 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.012+0000 7fec72ffd640 1 -- 172.21.15.27:0/1782051097 learned_addr learned my addr 172.21.15.27:0/1782051097 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:21:10.014 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.012+0000 7fec737fe640 1 --2- 172.21.15.27:0/1782051097 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fec7415a270 0x7fec74158400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:10.014 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.012+0000 7fec727fc640 1 --2- 172.21.15.27:0/1782051097 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fec7406d530 0x7fec7415f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:10.014 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.012+0000 7fec72ffd640 1 -- 172.21.15.27:0/1782051097 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fec7415a270 msgr2=0x7fec74158400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:10.014 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.012+0000 7fec72ffd640 1 --2- 172.21.15.27:0/1782051097 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fec7415a270 0x7fec74158400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:10.014 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.012+0000 7fec72ffd640 1 -- 172.21.15.27:0/1782051097 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fec7406d530 msgr2=0x7fec7415f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:10.014 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.012+0000 7fec72ffd640 1 --2- 172.21.15.27:0/1782051097 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fec7406d530 0x7fec7415f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:10.014 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.012+0000 7fec72ffd640 1 -- 172.21.15.27:0/1782051097 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec741567d0 con 0x7fec7406c650 2024-09-12T03:21:10.014 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.012+0000 7fec72ffd640 1 --2- 172.21.15.27:0/1782051097 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fec7406c650 0x7fec74147e30 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7fec6c00dbc0 tx=0x7fec6c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:10.014 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.012+0000 7fec6b7fe640 1 -- 172.21.15.27:0/1782051097 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec6c043020 con 0x7fec7406c650 2024-09-12T03:21:10.015 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.012+0000 7fec6b7fe640 1 -- 172.21.15.27:0/1782051097 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec6c03b040 con 0x7fec7406c650 2024-09-12T03:21:10.015 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.012+0000 7fec79359640 1 -- 172.21.15.27:0/1782051097 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec74156ab0 con 0x7fec7406c650 2024-09-12T03:21:10.015 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.012+0000 7fec6b7fe640 1 -- 172.21.15.27:0/1782051097 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec6c03a070 con 0x7fec7406c650 2024-09-12T03:21:10.015 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.013+0000 7fec79359640 1 -- 172.21.15.27:0/1782051097 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec74157070 con 0x7fec7406c650 2024-09-12T03:21:10.016 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.014+0000 7fec79359640 1 -- 172.21.15.27:0/1782051097 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec740ccdb0 con 0x7fec7406c650 2024-09-12T03:21:10.016 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.014+0000 7fec6b7fe640 1 -- 172.21.15.27:0/1782051097 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 90600+0+0 (secure 0 0 0) 0x7fec6c046020 con 0x7fec7406c650 2024-09-12T03:21:10.016 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.014+0000 7fec6b7fe640 1 --2- 172.21.15.27:0/1782051097 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fec5406bf50 0x7fec5406e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:10.016 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.015+0000 7fec6b7fe640 1 -- 172.21.15.27:0/1782051097 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5709+0+0 (secure 0 0 0) 0x7fec6c0af490 con 0x7fec7406c650 2024-09-12T03:21:10.016 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.015+0000 7fec727fc640 1 --2- 172.21.15.27:0/1782051097 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fec5406bf50 0x7fec5406e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:10.016 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.015+0000 7fec727fc640 1 --2- 172.21.15.27:0/1782051097 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fec5406bf50 0x7fec5406e410 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fec74157e60 tx=0x7fec5c009240 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:10.019 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.019+0000 7fec6b7fe640 1 -- 172.21.15.27:0/1782051097 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fec6c04d030 con 0x7fec7406c650 2024-09-12T03:21:10.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.226+0000 7fec79359640 1 -- 172.21.15.27:0/1782051097 --> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] -- mgr_command(tid 0: {"prefix": "fs subvolume resize", "vol_name": "cephfs", "sub_name": "subvol84490", "new_size": "10485760", "target": ["mon-mgr", ""]}) -- 0x7fec74063320 con 0x7fec5406bf50 2024-09-12T03:21:10.249 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.249+0000 7fec6b7fe640 1 -- 172.21.15.27:0/1782051097 <== mgr.4113 v2:172.21.15.27:6836/1226920526 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+128 (secure 0 0 0) 0x7fec5c011a90 con 0x7fec5406bf50 2024-09-12T03:21:10.250 INFO:teuthology.orchestra.run.smithi027.stdout:[ 2024-09-12T03:21:10.250 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:21:10.250 INFO:teuthology.orchestra.run.smithi027.stdout: "bytes_used": 0 2024-09-12T03:21:10.250 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:21:10.250 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:21:10.250 INFO:teuthology.orchestra.run.smithi027.stdout: "bytes_quota": 10485760 2024-09-12T03:21:10.250 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:21:10.250 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:21:10.250 INFO:teuthology.orchestra.run.smithi027.stdout: "bytes_pcent": "0.00" 2024-09-12T03:21:10.251 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:21:10.251 INFO:teuthology.orchestra.run.smithi027.stdout:] 2024-09-12T03:21:10.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.251+0000 7fec79359640 1 -- 172.21.15.27:0/1782051097 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fec5406bf50 msgr2=0x7fec5406e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:10.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.251+0000 7fec79359640 1 --2- 172.21.15.27:0/1782051097 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fec5406bf50 0x7fec5406e410 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fec74157e60 tx=0x7fec5c009240 comp rx=0 tx=0).stop 2024-09-12T03:21:10.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.251+0000 7fec79359640 1 -- 172.21.15.27:0/1782051097 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fec7406c650 msgr2=0x7fec74147e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:10.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.251+0000 7fec79359640 1 --2- 172.21.15.27:0/1782051097 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fec7406c650 0x7fec74147e30 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7fec6c00dbc0 tx=0x7fec6c0046c0 comp rx=0 tx=0).stop 2024-09-12T03:21:10.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.251+0000 7fec79359640 1 -- 172.21.15.27:0/1782051097 shutdown_connections 2024-09-12T03:21:10.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.251+0000 7fec79359640 1 --2- 172.21.15.27:0/1782051097 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fec7415a270 0x7fec74158400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:10.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.251+0000 7fec79359640 1 --2- 172.21.15.27:0/1782051097 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fec7406d530 0x7fec7415f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:10.253 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.251+0000 7fec79359640 1 --2- 172.21.15.27:0/1782051097 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fec5406bf50 0x7fec5406e410 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:10.253 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.251+0000 7fec79359640 1 --2- 172.21.15.27:0/1782051097 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fec7406c650 0x7fec74147e30 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:10.253 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.251+0000 7fec79359640 1 -- 172.21.15.27:0/1782051097 >> 172.21.15.27:0/1782051097 conn(0x7fec7406bb30 msgr2=0x7fec7406bf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:10.253 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.252+0000 7fec79359640 1 -- 172.21.15.27:0/1782051097 shutdown_connections 2024-09-12T03:21:10.253 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.252+0000 7fec79359640 1 -- 172.21.15.27:0/1782051097 wait complete. 2024-09-12T03:21:10.263 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume info cephfs subvol84490 2024-09-12T03:21:10.369 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.369+0000 7fedaba53640 1 Processor -- start 2024-09-12T03:21:10.369 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.369+0000 7fedaba53640 1 -- start start 2024-09-12T03:21:10.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.370+0000 7fedaba53640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7feda415e650 0x7feda40d1d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:10.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.370+0000 7fedaba53640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feda40d2280 con 0x7feda406db30 2024-09-12T03:21:10.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.370+0000 7fedaba53640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feda406e670 con 0x7feda406cc40 2024-09-12T03:21:10.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.370+0000 7fedaba53640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7feda406d850 con 0x7feda415e650 2024-09-12T03:21:10.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.370+0000 7feda9ff0640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7feda406db30 0x7feda406df60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:40686/0 (socket says 172.21.15.27:40686) 2024-09-12T03:21:10.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.370+0000 7feda9ff0640 1 -- 172.21.15.27:0/1157224284 learned_addr learned my addr 172.21.15.27:0/1157224284 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:21:10.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.370+0000 7feda97ef640 1 --2- 172.21.15.27:0/1157224284 >> v2:172.21.15.45:3300/0 conn(0x7feda415e650 0x7feda40d1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:10.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.370+0000 7fed9bfff640 1 -- 172.21.15.27:0/1157224284 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1268516470 0 0) 0x7feda40d2280 con 0x7feda406db30 2024-09-12T03:21:10.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.370+0000 7fed9bfff640 1 -- 172.21.15.27:0/1157224284 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fed90003520 con 0x7feda406db30 2024-09-12T03:21:10.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.370+0000 7fed9bfff640 1 -- 172.21.15.27:0/1157224284 <== mon.1 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2648867464 0 0) 0x7feda406e670 con 0x7feda406cc40 2024-09-12T03:21:10.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.370+0000 7fed9bfff640 1 -- 172.21.15.27:0/1157224284 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7feda40d2280 con 0x7feda406cc40 2024-09-12T03:21:10.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.370+0000 7fed9bfff640 1 -- 172.21.15.27:0/1157224284 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2563216895 0 0) 0x7fed90003520 con 0x7feda406db30 2024-09-12T03:21:10.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.370+0000 7fed9bfff640 1 -- 172.21.15.27:0/1157224284 >> v2:172.21.15.45:3300/0 conn(0x7feda415e650 msgr2=0x7feda40d1d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:10.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.370+0000 7fed9bfff640 1 --2- 172.21.15.27:0/1157224284 >> v2:172.21.15.45:3300/0 conn(0x7feda415e650 0x7feda40d1d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:10.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.370+0000 7fed9bfff640 1 -- 172.21.15.27:0/1157224284 >> v1:172.21.15.45:6789/0 conn(0x7feda406cc40 legacy=0x7feda406d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:10.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.370+0000 7fed9bfff640 1 -- 172.21.15.27:0/1157224284 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feda40d1260 con 0x7feda406db30 2024-09-12T03:21:10.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.370+0000 7fed9bfff640 1 -- 172.21.15.27:0/1157224284 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 847755213 0 0) 0x7fed9c0019f0 con 0x7feda406db30 2024-09-12T03:21:10.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.370+0000 7feda97ef640 1 --2- 172.21.15.27:0/1157224284 >> v2:172.21.15.45:3300/0 conn(0x7feda415e650 0x7feda40d1d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:21:10.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.371+0000 7fed9bfff640 1 -- 172.21.15.27:0/1157224284 >> v1:172.21.15.27:6789/0 conn(0x7feda406db30 legacy=0x7feda406df60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:10.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.371+0000 7fed9bfff640 1 --2- 172.21.15.27:0/1157224284 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fed90003b40 0x7fed90003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:10.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.371+0000 7fed9bfff640 1 --2- 172.21.15.27:0/1157224284 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fed90004800 0x7fed90004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:10.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.371+0000 7fed9bfff640 1 --2- 172.21.15.27:0/1157224284 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fed900051c0 0x7fed9000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:10.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.371+0000 7fed9bfff640 1 -- 172.21.15.27:0/1157224284 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7feda406d850 con 0x7fed900051c0 2024-09-12T03:21:10.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.371+0000 7fed9bfff640 1 -- 172.21.15.27:0/1157224284 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fed9000c0f0 con 0x7fed90004800 2024-09-12T03:21:10.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.371+0000 7fed9bfff640 1 -- 172.21.15.27:0/1157224284 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fed9000c2f0 con 0x7fed90003b40 2024-09-12T03:21:10.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.371+0000 7feda9ff0640 1 --2- 172.21.15.27:0/1157224284 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fed900051c0 0x7fed9000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:10.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.371+0000 7feda8fee640 1 --2- 172.21.15.27:0/1157224284 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fed90004800 0x7fed90004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:10.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.371+0000 7feda97ef640 1 --2- 172.21.15.27:0/1157224284 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fed90003b40 0x7fed90003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:10.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.371+0000 7feda9ff0640 1 -- 172.21.15.27:0/1157224284 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fed90003b40 msgr2=0x7fed90003f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:10.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.371+0000 7feda9ff0640 1 --2- 172.21.15.27:0/1157224284 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fed90003b40 0x7fed90003f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:10.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.371+0000 7feda9ff0640 1 -- 172.21.15.27:0/1157224284 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fed90004800 msgr2=0x7fed90004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:10.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.371+0000 7feda9ff0640 1 --2- 172.21.15.27:0/1157224284 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fed90004800 0x7fed90004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:10.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.371+0000 7feda9ff0640 1 -- 172.21.15.27:0/1157224284 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed90003520 con 0x7fed900051c0 2024-09-12T03:21:10.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.372+0000 7feda9ff0640 1 -- 172.21.15.27:0/1157224284 reap_dead start 2024-09-12T03:21:10.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.372+0000 7feda9ff0640 1 --2- 172.21.15.27:0/1157224284 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fed900051c0 0x7fed9000b9c0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7fed9c00a7f0 tx=0x7fed90005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79a4800e571634b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:10.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.372+0000 7fed9bfff640 1 -- 172.21.15.27:0/1157224284 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed9c00a410 con 0x7fed900051c0 2024-09-12T03:21:10.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.372+0000 7fed9bfff640 1 -- 172.21.15.27:0/1157224284 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed9c02f070 con 0x7fed900051c0 2024-09-12T03:21:10.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.372+0000 7fed9bfff640 1 -- 172.21.15.27:0/1157224284 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed9c02bc70 con 0x7fed900051c0 2024-09-12T03:21:10.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.372+0000 7fedaba53640 1 -- 172.21.15.27:0/1157224284 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fed900051c0 msgr2=0x7fed9000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:10.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.373+0000 7fedaba53640 1 --2- 172.21.15.27:0/1157224284 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fed900051c0 0x7fed9000b9c0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7fed9c00a7f0 tx=0x7fed90005830 comp rx=0 tx=0).stop 2024-09-12T03:21:10.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.373+0000 7fedaba53640 1 -- 172.21.15.27:0/1157224284 shutdown_connections 2024-09-12T03:21:10.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.373+0000 7fedaba53640 1 --2- 172.21.15.27:0/1157224284 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fed900051c0 0x7fed9000b9c0 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:10.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.373+0000 7fedaba53640 1 -- 172.21.15.27:0/1157224284 >> 172.21.15.27:0/1157224284 conn(0x7feda406f320 msgr2=0x7feda406f700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:10.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.373+0000 7fedaba53640 1 -- 172.21.15.27:0/1157224284 shutdown_connections 2024-09-12T03:21:10.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.373+0000 7fedaba53640 1 -- 172.21.15.27:0/1157224284 wait complete. 2024-09-12T03:21:10.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.374+0000 7fedaba53640 1 Processor -- start 2024-09-12T03:21:10.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.374+0000 7fedaba53640 1 -- start start 2024-09-12T03:21:10.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.375+0000 7fedaba53640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fed900051c0 0x7feda415b000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:10.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.375+0000 7fedaba53640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feda41566e0 0x7feda4156b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:10.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.375+0000 7fedaba53640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7feda41e6be0 0x7feda41e7060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:10.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.375+0000 7fedaba53640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7feda415ebb0 con 0x7feda41566e0 2024-09-12T03:21:10.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.375+0000 7fedaba53640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7feda415ea30 con 0x7fed900051c0 2024-09-12T03:21:10.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.375+0000 7fedaba53640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7feda40d1d40 con 0x7feda41e6be0 2024-09-12T03:21:10.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.375+0000 7feda8fee640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feda41566e0 0x7feda4156b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:10.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.375+0000 7feda8fee640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feda41566e0 0x7feda4156b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57380/0 (socket says 172.21.15.27:57380) 2024-09-12T03:21:10.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.375+0000 7feda8fee640 1 -- 172.21.15.27:0/1128037009 learned_addr learned my addr 172.21.15.27:0/1128037009 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:21:10.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.375+0000 7feda9ff0640 1 --2- 172.21.15.27:0/1128037009 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7feda41e6be0 0x7feda41e7060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:10.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.375+0000 7feda97ef640 1 --2- 172.21.15.27:0/1128037009 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fed900051c0 0x7feda415b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:10.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.375+0000 7feda8fee640 1 -- 172.21.15.27:0/1128037009 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7feda41e6be0 msgr2=0x7feda41e7060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:10.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.375+0000 7feda8fee640 1 --2- 172.21.15.27:0/1128037009 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7feda41e6be0 0x7feda41e7060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:10.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.375+0000 7feda8fee640 1 -- 172.21.15.27:0/1128037009 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fed900051c0 msgr2=0x7feda415b000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:10.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.375+0000 7feda8fee640 1 --2- 172.21.15.27:0/1128037009 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fed900051c0 0x7feda415b000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:10.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.375+0000 7feda8fee640 1 -- 172.21.15.27:0/1128037009 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feda41548c0 con 0x7feda41566e0 2024-09-12T03:21:10.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.375+0000 7feda8fee640 1 --2- 172.21.15.27:0/1128037009 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feda41566e0 0x7feda4156b40 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7fed9400f7a0 tx=0x7fed9400fc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:10.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.375+0000 7fed99ffb640 1 -- 172.21.15.27:0/1128037009 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed9401a070 con 0x7feda41566e0 2024-09-12T03:21:10.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.375+0000 7fed99ffb640 1 -- 172.21.15.27:0/1128037009 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed9400fe50 con 0x7feda41566e0 2024-09-12T03:21:10.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.375+0000 7fed99ffb640 1 -- 172.21.15.27:0/1128037009 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed94007050 con 0x7feda41566e0 2024-09-12T03:21:10.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.375+0000 7fedaba53640 1 -- 172.21.15.27:0/1128037009 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feda415b540 con 0x7feda41566e0 2024-09-12T03:21:10.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.376+0000 7fedaba53640 1 -- 172.21.15.27:0/1128037009 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feda4157530 con 0x7feda41566e0 2024-09-12T03:21:10.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.377+0000 7fedaba53640 1 -- 172.21.15.27:0/1128037009 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feda406d090 con 0x7feda41566e0 2024-09-12T03:21:10.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.377+0000 7fed99ffb640 1 -- 172.21.15.27:0/1128037009 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 90600+0+0 (secure 0 0 0) 0x7fed9401d070 con 0x7feda41566e0 2024-09-12T03:21:10.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.377+0000 7fed99ffb640 1 --2- 172.21.15.27:0/1128037009 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fed8806bf50 0x7fed8806e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:10.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.378+0000 7fed99ffb640 1 -- 172.21.15.27:0/1128037009 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5709+0+0 (secure 0 0 0) 0x7fed9408a780 con 0x7feda41566e0 2024-09-12T03:21:10.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.378+0000 7feda97ef640 1 --2- 172.21.15.27:0/1128037009 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fed8806bf50 0x7fed8806e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:10.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.378+0000 7feda97ef640 1 --2- 172.21.15.27:0/1128037009 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fed8806bf50 0x7fed8806e410 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7feda0009c10 tx=0x7feda0008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:10.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.382+0000 7fed99ffb640 1 -- 172.21.15.27:0/1128037009 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fed94059020 con 0x7feda41566e0 2024-09-12T03:21:10.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.583+0000 7fedaba53640 1 -- 172.21.15.27:0/1128037009 --> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] -- mgr_command(tid 0: {"prefix": "fs subvolume info", "vol_name": "cephfs", "sub_name": "subvol84490", "target": ["mon-mgr", ""]}) -- 0x7feda40b5000 con 0x7fed8806bf50 2024-09-12T03:21:10.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.609+0000 7fed99ffb640 1 -- 172.21.15.27:0/1128037009 <== mgr.4113 v2:172.21.15.27:6836/1226920526 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+677 (secure 0 0 0) 0x7feda40b5000 con 0x7fed8806bf50 2024-09-12T03:21:10.609 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-12T03:21:10.610 INFO:teuthology.orchestra.run.smithi027.stdout: "atime": "2024-09-12 03:21:08", 2024-09-12T03:21:10.610 INFO:teuthology.orchestra.run.smithi027.stdout: "bytes_pcent": "0.00", 2024-09-12T03:21:10.610 INFO:teuthology.orchestra.run.smithi027.stdout: "bytes_quota": 10485760, 2024-09-12T03:21:10.610 INFO:teuthology.orchestra.run.smithi027.stdout: "bytes_used": 0, 2024-09-12T03:21:10.610 INFO:teuthology.orchestra.run.smithi027.stdout: "created_at": "2024-09-12 03:21:08", 2024-09-12T03:21:10.610 INFO:teuthology.orchestra.run.smithi027.stdout: "ctime": "2024-09-12 03:21:10", 2024-09-12T03:21:10.610 INFO:teuthology.orchestra.run.smithi027.stdout: "data_pool": "cephfs_data", 2024-09-12T03:21:10.610 INFO:teuthology.orchestra.run.smithi027.stdout: "features": [ 2024-09-12T03:21:10.610 INFO:teuthology.orchestra.run.smithi027.stdout: "snapshot-clone", 2024-09-12T03:21:10.610 INFO:teuthology.orchestra.run.smithi027.stdout: "snapshot-autoprotect" 2024-09-12T03:21:10.610 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-12T03:21:10.610 INFO:teuthology.orchestra.run.smithi027.stdout: "flavor": 1, 2024-09-12T03:21:10.611 INFO:teuthology.orchestra.run.smithi027.stdout: "gid": 0, 2024-09-12T03:21:10.611 INFO:teuthology.orchestra.run.smithi027.stdout: "mode": 16877, 2024-09-12T03:21:10.611 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_addrs": [ 2024-09-12T03:21:10.611 INFO:teuthology.orchestra.run.smithi027.stdout: "172.21.15.27:6789", 2024-09-12T03:21:10.611 INFO:teuthology.orchestra.run.smithi027.stdout: "172.21.15.45:6789", 2024-09-12T03:21:10.611 INFO:teuthology.orchestra.run.smithi027.stdout: "172.21.15.45:6790" 2024-09-12T03:21:10.611 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-12T03:21:10.611 INFO:teuthology.orchestra.run.smithi027.stdout: "mtime": "2024-09-12 03:21:08", 2024-09-12T03:21:10.611 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/volumes/_nogroup/subvol84490/a842cae4-61b6-4f73-a96f-03c1581e2365", 2024-09-12T03:21:10.611 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_namespace": "", 2024-09-12T03:21:10.611 INFO:teuthology.orchestra.run.smithi027.stdout: "state": "complete", 2024-09-12T03:21:10.611 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "subvolume", 2024-09-12T03:21:10.611 INFO:teuthology.orchestra.run.smithi027.stdout: "uid": 0 2024-09-12T03:21:10.612 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-12T03:21:10.613 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.611+0000 7fedaba53640 1 -- 172.21.15.27:0/1128037009 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fed8806bf50 msgr2=0x7fed8806e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:10.613 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.611+0000 7fedaba53640 1 --2- 172.21.15.27:0/1128037009 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fed8806bf50 0x7fed8806e410 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7feda0009c10 tx=0x7feda0008040 comp rx=0 tx=0).stop 2024-09-12T03:21:10.613 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.611+0000 7fedaba53640 1 -- 172.21.15.27:0/1128037009 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feda41566e0 msgr2=0x7feda4156b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:10.613 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.611+0000 7fedaba53640 1 --2- 172.21.15.27:0/1128037009 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feda41566e0 0x7feda4156b40 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7fed9400f7a0 tx=0x7fed9400fc60 comp rx=0 tx=0).stop 2024-09-12T03:21:10.613 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.611+0000 7fedaba53640 1 -- 172.21.15.27:0/1128037009 shutdown_connections 2024-09-12T03:21:10.613 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.611+0000 7fedaba53640 1 --2- 172.21.15.27:0/1128037009 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7feda41e6be0 0x7feda41e7060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:10.613 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.611+0000 7fedaba53640 1 --2- 172.21.15.27:0/1128037009 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fed8806bf50 0x7fed8806e410 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:10.613 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.611+0000 7fedaba53640 1 --2- 172.21.15.27:0/1128037009 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feda41566e0 0x7feda4156b40 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:10.613 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.611+0000 7fedaba53640 1 --2- 172.21.15.27:0/1128037009 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fed900051c0 0x7feda415b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:10.614 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.611+0000 7fedaba53640 1 -- 172.21.15.27:0/1128037009 >> 172.21.15.27:0/1128037009 conn(0x7feda406f320 msgr2=0x7feda415ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:10.614 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.611+0000 7fedaba53640 1 -- 172.21.15.27:0/1128037009 shutdown_connections 2024-09-12T03:21:10.614 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.611+0000 7fedaba53640 1 -- 172.21.15.27:0/1128037009 wait complete. 2024-09-12T03:21:10.622 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs subvol84490 --mode 777 --uid 1000 --gid 1000 2024-09-12T03:21:10.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.734+0000 7f8bae477640 1 Processor -- start 2024-09-12T03:21:10.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.734+0000 7f8bae477640 1 -- start start 2024-09-12T03:21:10.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.735+0000 7f8bae477640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f8ba81557e0 0x7f8ba8072ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:10.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.735+0000 7f8bae477640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f8ba815fc70 0x7f8ba815c470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:10.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.735+0000 7f8bae477640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8ba80730b0 con 0x7f8ba8155bc0 2024-09-12T03:21:10.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.735+0000 7f8bae477640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f8ba8072920 con 0x7f8ba815fc70 2024-09-12T03:21:10.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.735+0000 7f8bae477640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f8ba815c9b0 con 0x7f8ba81557e0 2024-09-12T03:21:10.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.735+0000 7f8b9ffff640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f8ba815fc70 0x7f8ba815c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:10.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.735+0000 7f8ba7fff640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f8ba81557e0 0x7f8ba8072ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:10.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.735+0000 7f8ba7fff640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f8ba81557e0 0x7f8ba8072ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3301/0 says I am v2:172.21.15.27:34436/0 (socket says 172.21.15.27:34436) 2024-09-12T03:21:10.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.735+0000 7f8ba7fff640 1 -- 172.21.15.27:0/3586382837 learned_addr learned my addr 172.21.15.27:0/3586382837 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:21:10.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.736+0000 7f8ba7fff640 1 -- 172.21.15.27:0/3586382837 >> v2:172.21.15.45:3300/0 conn(0x7f8ba815fc70 msgr2=0x7f8ba815c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:10.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.736+0000 7f8ba7fff640 1 --2- 172.21.15.27:0/3586382837 >> v2:172.21.15.45:3300/0 conn(0x7f8ba815fc70 0x7f8ba815c470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:10.738 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.736+0000 7f8ba7fff640 1 -- 172.21.15.27:0/3586382837 >> v1:172.21.15.45:6789/0 conn(0x7f8ba8155bc0 legacy=0x7f8ba80b4770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:10.738 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.736+0000 7f8ba7fff640 1 -- 172.21.15.27:0/3586382837 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ba815f190 con 0x7f8ba81557e0 2024-09-12T03:21:10.738 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.736+0000 7f8ba7fff640 1 --2- 172.21.15.27:0/3586382837 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f8ba81557e0 0x7f8ba8072ad0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f8b9800bb10 tx=0x7f8b9802c920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6f8f1082f4cf898b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:10.738 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.737+0000 7f8ba77fe640 1 -- 172.21.15.27:0/3586382837 <== mon.2 v2:172.21.15.45:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b98036020 con 0x7f8ba81557e0 2024-09-12T03:21:10.738 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.737+0000 7f8ba77fe640 1 -- 172.21.15.27:0/3586382837 <== mon.2 v2:172.21.15.45:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b98032820 con 0x7f8ba81557e0 2024-09-12T03:21:10.738 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.737+0000 7f8ba77fe640 1 -- 172.21.15.27:0/3586382837 <== mon.2 v2:172.21.15.45:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b98032bf0 con 0x7f8ba81557e0 2024-09-12T03:21:10.740 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.737+0000 7f8bae477640 1 -- 172.21.15.27:0/3586382837 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f8ba81557e0 msgr2=0x7f8ba8072ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:10.740 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.737+0000 7f8bae477640 1 --2- 172.21.15.27:0/3586382837 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f8ba81557e0 0x7f8ba8072ad0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f8b9800bb10 tx=0x7f8b9802c920 comp rx=0 tx=0).stop 2024-09-12T03:21:10.740 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.737+0000 7f8bae477640 1 -- 172.21.15.27:0/3586382837 shutdown_connections 2024-09-12T03:21:10.740 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.737+0000 7f8bae477640 1 --2- 172.21.15.27:0/3586382837 >> v2:172.21.15.45:3300/0 conn(0x7f8ba815fc70 0x7f8ba815c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:10.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.737+0000 7f8bae477640 1 --2- 172.21.15.27:0/3586382837 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f8ba81557e0 0x7f8ba8072ad0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:10.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.737+0000 7f8bae477640 1 -- 172.21.15.27:0/3586382837 >> 172.21.15.27:0/3586382837 conn(0x7f8ba80b2260 msgr2=0x7f8ba80b2640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:10.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.737+0000 7f8bae477640 1 -- 172.21.15.27:0/3586382837 shutdown_connections 2024-09-12T03:21:10.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.737+0000 7f8bae477640 1 -- 172.21.15.27:0/3586382837 wait complete. 2024-09-12T03:21:10.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.738+0000 7f8bae477640 1 Processor -- start 2024-09-12T03:21:10.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.738+0000 7f8bae477640 1 -- start start 2024-09-12T03:21:10.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.739+0000 7f8bae477640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f8ba81557e0 0x7f8ba80b0f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:10.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.739+0000 7f8bae477640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8ba8155bc0 0x7f8ba80ac090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:10.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.739+0000 7f8bae477640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ba815fc70 0x7f8ba8154370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:10.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.739+0000 7f8bae477640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f8ba80b4e80 con 0x7f8ba8155bc0 2024-09-12T03:21:10.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.739+0000 7f8bae477640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f8ba815f750 con 0x7f8ba815fc70 2024-09-12T03:21:10.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.739+0000 7f8bae477640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f8ba80b5000 con 0x7f8ba81557e0 2024-09-12T03:21:10.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.739+0000 7f8b9ffff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8ba8155bc0 0x7f8ba80ac090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:10.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.739+0000 7f8b9ffff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8ba8155bc0 0x7f8ba80ac090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57382/0 (socket says 172.21.15.27:57382) 2024-09-12T03:21:10.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.739+0000 7f8b9ffff640 1 -- 172.21.15.27:0/4292515885 learned_addr learned my addr 172.21.15.27:0/4292515885 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:21:10.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.739+0000 7f8ba7fff640 1 --2- 172.21.15.27:0/4292515885 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f8ba81557e0 0x7f8ba80b0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:10.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.739+0000 7f8baca14640 1 --2- 172.21.15.27:0/4292515885 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ba815fc70 0x7f8ba8154370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:10.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.740+0000 7f8b9ffff640 1 -- 172.21.15.27:0/4292515885 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f8ba81557e0 msgr2=0x7f8ba80b0f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:10.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.740+0000 7f8b9ffff640 1 --2- 172.21.15.27:0/4292515885 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f8ba81557e0 0x7f8ba80b0f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:10.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.740+0000 7f8b9ffff640 1 -- 172.21.15.27:0/4292515885 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ba815fc70 msgr2=0x7f8ba8154370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:10.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.740+0000 7f8b9ffff640 1 --2- 172.21.15.27:0/4292515885 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ba815fc70 0x7f8ba8154370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:10.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.740+0000 7f8b9ffff640 1 -- 172.21.15.27:0/4292515885 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ba80ac790 con 0x7f8ba8155bc0 2024-09-12T03:21:10.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.740+0000 7f8b9ffff640 1 --2- 172.21.15.27:0/4292515885 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8ba8155bc0 0x7f8ba80ac090 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f8b9000e8a0 tx=0x7f8b9000ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:10.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.740+0000 7f8ba57fa640 1 -- 172.21.15.27:0/4292515885 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b90017070 con 0x7f8ba8155bc0 2024-09-12T03:21:10.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.740+0000 7f8ba57fa640 1 -- 172.21.15.27:0/4292515885 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b9000ee90 con 0x7f8ba8155bc0 2024-09-12T03:21:10.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.740+0000 7f8bae477640 1 -- 172.21.15.27:0/4292515885 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ba80aca70 con 0x7f8ba8155bc0 2024-09-12T03:21:10.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.740+0000 7f8ba57fa640 1 -- 172.21.15.27:0/4292515885 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b9000cb60 con 0x7f8ba8155bc0 2024-09-12T03:21:10.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.740+0000 7f8bae477640 1 -- 172.21.15.27:0/4292515885 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ba81564b0 con 0x7f8ba8155bc0 2024-09-12T03:21:10.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.741+0000 7f8bae477640 1 -- 172.21.15.27:0/4292515885 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ba806ea00 con 0x7f8ba8155bc0 2024-09-12T03:21:10.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.742+0000 7f8ba57fa640 1 -- 172.21.15.27:0/4292515885 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 90600+0+0 (secure 0 0 0) 0x7f8b9001a070 con 0x7f8ba8155bc0 2024-09-12T03:21:10.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.742+0000 7f8ba57fa640 1 --2- 172.21.15.27:0/4292515885 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f8b8806bf50 0x7f8b8806e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:10.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.742+0000 7f8ba57fa640 1 -- 172.21.15.27:0/4292515885 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5709+0+0 (secure 0 0 0) 0x7f8b9008b120 con 0x7f8ba8155bc0 2024-09-12T03:21:10.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.742+0000 7f8ba7fff640 1 --2- 172.21.15.27:0/4292515885 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f8b8806bf50 0x7f8b8806e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:10.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.743+0000 7f8ba7fff640 1 --2- 172.21.15.27:0/4292515885 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f8b8806bf50 0x7f8b8806e410 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f8b98037000 tx=0x7f8b9803d040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:10.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.746+0000 7f8ba57fa640 1 -- 172.21.15.27:0/4292515885 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f8b90059370 con 0x7f8ba8155bc0 2024-09-12T03:21:10.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.841+0000 7f8ba57fa640 1 -- 172.21.15.27:0/4292515885 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mgrmap(e 17) ==== 90651+0+0 (secure 0 0 0) 0x7f8b90089640 con 0x7f8ba8155bc0 2024-09-12T03:21:10.951 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.951+0000 7f8bae477640 1 -- 172.21.15.27:0/4292515885 --> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvol84490", "mode": "777", "uid": 1000, "gid": 1000, "target": ["mon-mgr", ""]}) -- 0x7f8ba80b3820 con 0x7f8b8806bf50 2024-09-12T03:21:10.973 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.973+0000 7f8ba57fa640 1 -- 172.21.15.27:0/4292515885 <== mgr.4113 v2:172.21.15.27:6836/1226920526 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f8ba80b3820 con 0x7f8b8806bf50 2024-09-12T03:21:10.976 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.976+0000 7f8bae477640 1 -- 172.21.15.27:0/4292515885 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f8b8806bf50 msgr2=0x7f8b8806e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:10.976 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.976+0000 7f8bae477640 1 --2- 172.21.15.27:0/4292515885 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f8b8806bf50 0x7f8b8806e410 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f8b98037000 tx=0x7f8b9803d040 comp rx=0 tx=0).stop 2024-09-12T03:21:10.976 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.976+0000 7f8bae477640 1 -- 172.21.15.27:0/4292515885 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8ba8155bc0 msgr2=0x7f8ba80ac090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:10.976 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.976+0000 7f8bae477640 1 --2- 172.21.15.27:0/4292515885 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8ba8155bc0 0x7f8ba80ac090 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f8b9000e8a0 tx=0x7f8b9000ed60 comp rx=0 tx=0).stop 2024-09-12T03:21:10.976 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.976+0000 7f8bae477640 1 -- 172.21.15.27:0/4292515885 shutdown_connections 2024-09-12T03:21:10.976 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.976+0000 7f8bae477640 1 --2- 172.21.15.27:0/4292515885 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ba815fc70 0x7f8ba8154370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:10.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.976+0000 7f8bae477640 1 --2- 172.21.15.27:0/4292515885 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f8b8806bf50 0x7f8b8806e410 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:10.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.976+0000 7f8bae477640 1 --2- 172.21.15.27:0/4292515885 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8ba8155bc0 0x7f8ba80ac090 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:10.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.976+0000 7f8bae477640 1 --2- 172.21.15.27:0/4292515885 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f8ba81557e0 0x7f8ba80b0f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:10.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.976+0000 7f8bae477640 1 -- 172.21.15.27:0/4292515885 >> 172.21.15.27:0/4292515885 conn(0x7f8ba80b2260 msgr2=0x7f8ba8071d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:10.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.976+0000 7f8bae477640 1 -- 172.21.15.27:0/4292515885 shutdown_connections 2024-09-12T03:21:10.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:10.976+0000 7f8bae477640 1 -- 172.21.15.27:0/4292515885 wait complete. 2024-09-12T03:21:10.986 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol84490 2024-09-12T03:21:11.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:11.093+0000 7f717660e640 1 Processor -- start 2024-09-12T03:21:11.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:11.093+0000 7f717660e640 1 -- start start 2024-09-12T03:21:11.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:11.094+0000 7f717660e640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f717015cc90 0x7f717006dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:11.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:11.094+0000 7f717660e640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f717006e4e0 0x7f7170160380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:11.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:11.094+0000 7f717660e640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f717006bcd0 con 0x7f717013f5a0 2024-09-12T03:21:11.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:11.094+0000 7f717660e640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f7170070b40 con 0x7f717006e4e0 2024-09-12T03:21:11.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:11.094+0000 7f717660e640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f717006e910 con 0x7f717015cc90 2024-09-12T03:21:11.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:11.094+0000 7f716ffff640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f717013f5a0 0x7f7170070430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:40696/0 (socket says 172.21.15.27:40696) 2024-09-12T03:21:11.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:11.094+0000 7f716ffff640 1 -- 172.21.15.27:0/1295377613 learned_addr learned my addr 172.21.15.27:0/1295377613 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:21:11.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:11.094+0000 7f7174bab640 1 --2- 172.21.15.27:0/1295377613 >> v2:172.21.15.27:3300/0 conn(0x7f717006e4e0 0x7f7170160380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:11.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:11.094+0000 7f716f7fe640 1 --2- 172.21.15.27:0/1295377613 >> v2:172.21.15.45:3300/0 conn(0x7f717015cc90 0x7f717006dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:11.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:11.094+0000 7f716effd640 1 -- 172.21.15.27:0/1295377613 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 172142095 0 0) 0x7f717006bcd0 con 0x7f717013f5a0 2024-09-12T03:21:11.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:11.094+0000 7f716effd640 1 -- 172.21.15.27:0/1295377613 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7158003520 con 0x7f717013f5a0 2024-09-12T03:21:11.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:11.095+0000 7f7174bab640 1 -- 172.21.15.27:0/1295377613 >> v2:172.21.15.45:3300/0 conn(0x7f717015cc90 msgr2=0x7f717006dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:11.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:11.095+0000 7f7174bab640 1 --2- 172.21.15.27:0/1295377613 >> v2:172.21.15.45:3300/0 conn(0x7f717015cc90 0x7f717006dfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:11.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:11.095+0000 7f7174bab640 1 -- 172.21.15.27:0/1295377613 >> v1:172.21.15.27:6789/0 conn(0x7f717013f5a0 legacy=0x7f7170070430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:11.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:11.095+0000 7f7174bab640 1 -- 172.21.15.27:0/1295377613 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f717006bfb0 con 0x7f717006e4e0 2024-09-12T03:21:11.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:11.095+0000 7f7174bab640 1 --2- 172.21.15.27:0/1295377613 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f717006e4e0 0x7f7170160380 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f7170063350 tx=0x7f716802a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91ad4e20cb15d983 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:11.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:11.095+0000 7f716effd640 1 -- 172.21.15.27:0/1295377613 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7168007740 con 0x7f717006e4e0 2024-09-12T03:21:11.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:11.095+0000 7f716effd640 1 -- 172.21.15.27:0/1295377613 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71680078e0 con 0x7f717006e4e0 2024-09-12T03:21:11.098 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:11.095+0000 7f716effd640 1 -- 172.21.15.27:0/1295377613 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7168007c90 con 0x7f717006e4e0 2024-09-12T03:21:11.098 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:11.096+0000 7f717660e640 1 -- 172.21.15.27:0/1295377613 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f717006e4e0 msgr2=0x7f7170160380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:11.098 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:11.096+0000 7f717660e640 1 --2- 172.21.15.27:0/1295377613 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f717006e4e0 0x7f7170160380 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f7170063350 tx=0x7f716802a5f0 comp rx=0 tx=0).stop 2024-09-12T03:21:11.098 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:11.096+0000 7f717660e640 1 -- 172.21.15.27:0/1295377613 shutdown_connections 2024-09-12T03:21:11.098 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:11.096+0000 7f717660e640 1 --2- 172.21.15.27:0/1295377613 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f717006e4e0 0x7f7170160380 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:11.098 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:11.096+0000 7f717660e640 1 --2- 172.21.15.27:0/1295377613 >> v2:172.21.15.45:3300/0 conn(0x7f717015cc90 0x7f717006dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:11.098 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:11.096+0000 7f717660e640 1 -- 172.21.15.27:0/1295377613 >> 172.21.15.27:0/1295377613 conn(0x7f71700b3370 msgr2=0x7f71700b3780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:11.098 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:11.096+0000 7f717660e640 1 -- 172.21.15.27:0/1295377613 shutdown_connections 2024-09-12T03:21:11.098 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:11.096+0000 7f717660e640 1 -- 172.21.15.27:0/1295377613 wait complete. 2024-09-12T03:21:11.098 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:11.097+0000 7f717660e640 1 Processor -- start 2024-09-12T03:21:11.098 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:11.098+0000 7f717660e640 1 -- start start 2024-09-12T03:21:11.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:11.098+0000 7f717660e640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f717006e4e0 0x7f71700ac4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:11.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:11.098+0000 7f717660e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f717013f5a0 0x7f71700cd0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:11.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:11.098+0000 7f717660e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f717015cc90 0x7f71700aee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:11.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:11.098+0000 7f717660e640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f717013f980 con 0x7f717013f5a0 2024-09-12T03:21:11.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:11.098+0000 7f717660e640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f717006c570 con 0x7f717015cc90 2024-09-12T03:21:11.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:11.098+0000 7f717660e640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f717013fb00 con 0x7f717006e4e0 2024-09-12T03:21:11.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:11.098+0000 7f716f7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f717013f5a0 0x7f71700cd0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:11.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:11.098+0000 7f716f7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f717013f5a0 0x7f71700cd0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57400/0 (socket says 172.21.15.27:57400) 2024-09-12T03:21:11.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:11.098+0000 7f716f7fe640 1 -- 172.21.15.27:0/186780909 learned_addr learned my addr 172.21.15.27:0/186780909 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:21:11.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:11.099+0000 7f7174bab640 1 --2- 172.21.15.27:0/186780909 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f717015cc90 0x7f71700aee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:11.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:11.099+0000 7f716ffff640 1 --2- 172.21.15.27:0/186780909 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f717006e4e0 0x7f71700ac4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:11.102 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:11.099+0000 7f716f7fe640 1 -- 172.21.15.27:0/186780909 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f717006e4e0 msgr2=0x7f71700ac4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:11.102 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:11.099+0000 7f716f7fe640 1 --2- 172.21.15.27:0/186780909 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f717006e4e0 0x7f71700ac4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:11.102 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:11.099+0000 7f716f7fe640 1 -- 172.21.15.27:0/186780909 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f717015cc90 msgr2=0x7f71700aee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:11.102 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:11.099+0000 7f716f7fe640 1 --2- 172.21.15.27:0/186780909 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f717015cc90 0x7f71700aee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:11.102 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:11.099+0000 7f716f7fe640 1 -- 172.21.15.27:0/186780909 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71700cd620 con 0x7f717013f5a0 2024-09-12T03:21:11.102 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:11.099+0000 7f716ffff640 1 --2- 172.21.15.27:0/186780909 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f717006e4e0 0x7f71700ac4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:21:11.102 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:11.099+0000 7f716f7fe640 1 --2- 172.21.15.27:0/186780909 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f717013f5a0 0x7f71700cd0e0 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f716400e9b0 tx=0x7f716400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:11.102 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:11.099+0000 7f716cff9640 1 -- 172.21.15.27:0/186780909 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7164017070 con 0x7f717013f5a0 2024-09-12T03:21:11.102 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:11.099+0000 7f716cff9640 1 -- 172.21.15.27:0/186780909 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f716400c9a0 con 0x7f717013f5a0 2024-09-12T03:21:11.102 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:11.099+0000 7f716cff9640 1 -- 172.21.15.27:0/186780909 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f716400cd90 con 0x7f717013f5a0 2024-09-12T03:21:11.102 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:11.099+0000 7f717660e640 1 -- 172.21.15.27:0/186780909 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71701432c0 con 0x7f717013f5a0 2024-09-12T03:21:11.102 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:11.099+0000 7f717660e640 1 -- 172.21.15.27:0/186780909 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7170143880 con 0x7f717013f5a0 2024-09-12T03:21:11.102 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:11.100+0000 7f717660e640 1 -- 172.21.15.27:0/186780909 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71700b3fe0 con 0x7f717013f5a0 2024-09-12T03:21:11.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:11.100+0000 7f716cff9640 1 -- 172.21.15.27:0/186780909 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) ==== 90651+0+0 (secure 0 0 0) 0x7f716401a070 con 0x7f717013f5a0 2024-09-12T03:21:11.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:11.101+0000 7f716cff9640 1 --2- 172.21.15.27:0/186780909 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f715c06bfe0 0x7f715c06e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:11.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:11.101+0000 7f716cff9640 1 -- 172.21.15.27:0/186780909 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5709+0+0 (secure 0 0 0) 0x7f716408b110 con 0x7f717013f5a0 2024-09-12T03:21:11.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:11.101+0000 7f716ffff640 1 --2- 172.21.15.27:0/186780909 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f715c06bfe0 0x7f715c06e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:11.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:11.101+0000 7f716ffff640 1 --2- 172.21.15.27:0/186780909 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f715c06bfe0 0x7f715c06e4a0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f7160005880 tx=0x7f7160009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:11.105 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:11.106+0000 7f716cff9640 1 -- 172.21.15.27:0/186780909 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f71640598d0 con 0x7f717013f5a0 2024-09-12T03:21:11.308 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:11.307+0000 7f717660e640 1 -- 172.21.15.27:0/186780909 --> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol84490", "target": ["mon-mgr", ""]}) -- 0x7f71700702a0 con 0x7f715c06bfe0 2024-09-12T03:21:11.333 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:11.333+0000 7f716cff9640 1 -- 172.21.15.27:0/186780909 <== mgr.4113 v2:172.21.15.27:6836/1226920526 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+67 (secure 0 0 0) 0x7f71700702a0 con 0x7f715c06bfe0 2024-09-12T03:21:11.333 INFO:teuthology.orchestra.run.smithi027.stdout:/volumes/_nogroup/subvol84490/a842cae4-61b6-4f73-a96f-03c1581e2365 2024-09-12T03:21:11.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:11.335+0000 7f717660e640 1 -- 172.21.15.27:0/186780909 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f715c06bfe0 msgr2=0x7f715c06e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:11.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:11.335+0000 7f717660e640 1 --2- 172.21.15.27:0/186780909 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f715c06bfe0 0x7f715c06e4a0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f7160005880 tx=0x7f7160009040 comp rx=0 tx=0).stop 2024-09-12T03:21:11.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:11.335+0000 7f717660e640 1 -- 172.21.15.27:0/186780909 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f717013f5a0 msgr2=0x7f71700cd0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:11.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:11.335+0000 7f717660e640 1 --2- 172.21.15.27:0/186780909 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f717013f5a0 0x7f71700cd0e0 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f716400e9b0 tx=0x7f716400ee70 comp rx=0 tx=0).stop 2024-09-12T03:21:11.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:11.335+0000 7f717660e640 1 -- 172.21.15.27:0/186780909 shutdown_connections 2024-09-12T03:21:11.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:11.335+0000 7f717660e640 1 --2- 172.21.15.27:0/186780909 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f717015cc90 0x7f71700aee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:11.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:11.335+0000 7f717660e640 1 --2- 172.21.15.27:0/186780909 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f715c06bfe0 0x7f715c06e4a0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:11.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:11.335+0000 7f717660e640 1 --2- 172.21.15.27:0/186780909 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f717013f5a0 0x7f71700cd0e0 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:11.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:11.335+0000 7f717660e640 1 --2- 172.21.15.27:0/186780909 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f717006e4e0 0x7f71700ac4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:11.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:11.335+0000 7f717660e640 1 -- 172.21.15.27:0/186780909 >> 172.21.15.27:0/186780909 conn(0x7f71700b3370 msgr2=0x7f71700b1930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:11.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:11.335+0000 7f717660e640 1 -- 172.21.15.27:0/186780909 shutdown_connections 2024-09-12T03:21:11.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:11.335+0000 7f717660e640 1 -- 172.21.15.27:0/186780909 wait complete. 2024-09-12T03:21:11.346 DEBUG:tasks.cephfs.test_volumes:filling subvolume subvol84490 with 8 files each 1MB size under directory volumes/_nogroup/subvol84490/a842cae4-61b6-4f73-a96f-03c1581e2365 2024-09-12T03:21:11.346 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-12T03:21:11.346 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T03:21:11.363 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:21:11.363 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol84490/a842cae4-61b6-4f73-a96f-03c1581e2365/subvolume_file.0 bs=1M conv=fdatasync count=1 seek=0) 2024-09-12T03:21:11.433 INFO:teuthology.orchestra.run.smithi027.stderr:1+0 records in 2024-09-12T03:21:11.433 INFO:teuthology.orchestra.run.smithi027.stderr:1+0 records out 2024-09-12T03:21:11.434 INFO:teuthology.orchestra.run.smithi027.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104181 s, 101 MB/s 2024-09-12T03:21:11.434 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-12T03:21:11.434 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T03:21:11.490 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:21:11.490 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol84490/a842cae4-61b6-4f73-a96f-03c1581e2365/subvolume_file.1 bs=1M conv=fdatasync count=1 seek=0) 2024-09-12T03:21:11.553 INFO:teuthology.orchestra.run.smithi027.stderr:1+0 records in 2024-09-12T03:21:11.554 INFO:teuthology.orchestra.run.smithi027.stderr:1+0 records out 2024-09-12T03:21:11.554 INFO:teuthology.orchestra.run.smithi027.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00798384 s, 131 MB/s 2024-09-12T03:21:11.555 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-12T03:21:11.555 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T03:21:11.611 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:21:11.611 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol84490/a842cae4-61b6-4f73-a96f-03c1581e2365/subvolume_file.2 bs=1M conv=fdatasync count=1 seek=0) 2024-09-12T03:21:11.677 INFO:teuthology.orchestra.run.smithi027.stderr:1+0 records in 2024-09-12T03:21:11.677 INFO:teuthology.orchestra.run.smithi027.stderr:1+0 records out 2024-09-12T03:21:11.678 INFO:teuthology.orchestra.run.smithi027.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00916848 s, 114 MB/s 2024-09-12T03:21:11.678 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-12T03:21:11.678 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T03:21:11.734 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:21:11.734 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol84490/a842cae4-61b6-4f73-a96f-03c1581e2365/subvolume_file.3 bs=1M conv=fdatasync count=1 seek=0) 2024-09-12T03:21:11.798 INFO:teuthology.orchestra.run.smithi027.stderr:1+0 records in 2024-09-12T03:21:11.799 INFO:teuthology.orchestra.run.smithi027.stderr:1+0 records out 2024-09-12T03:21:11.799 INFO:teuthology.orchestra.run.smithi027.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00902909 s, 116 MB/s 2024-09-12T03:21:11.800 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-12T03:21:11.800 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T03:21:11.856 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:21:11.856 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol84490/a842cae4-61b6-4f73-a96f-03c1581e2365/subvolume_file.4 bs=1M conv=fdatasync count=1 seek=0) 2024-09-12T03:21:11.920 INFO:teuthology.orchestra.run.smithi027.stderr:1+0 records in 2024-09-12T03:21:11.920 INFO:teuthology.orchestra.run.smithi027.stderr:1+0 records out 2024-09-12T03:21:11.920 INFO:teuthology.orchestra.run.smithi027.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00787637 s, 133 MB/s 2024-09-12T03:21:11.921 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-12T03:21:11.921 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T03:21:11.977 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:21:11.978 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol84490/a842cae4-61b6-4f73-a96f-03c1581e2365/subvolume_file.5 bs=1M conv=fdatasync count=1 seek=0) 2024-09-12T03:21:12.044 INFO:teuthology.orchestra.run.smithi027.stderr:1+0 records in 2024-09-12T03:21:12.044 INFO:teuthology.orchestra.run.smithi027.stderr:1+0 records out 2024-09-12T03:21:12.044 INFO:teuthology.orchestra.run.smithi027.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00902943 s, 116 MB/s 2024-09-12T03:21:12.044 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-12T03:21:12.044 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T03:21:12.101 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:21:12.101 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol84490/a842cae4-61b6-4f73-a96f-03c1581e2365/subvolume_file.6 bs=1M conv=fdatasync count=1 seek=0) 2024-09-12T03:21:12.167 INFO:teuthology.orchestra.run.smithi027.stderr:1+0 records in 2024-09-12T03:21:12.167 INFO:teuthology.orchestra.run.smithi027.stderr:1+0 records out 2024-09-12T03:21:12.167 INFO:teuthology.orchestra.run.smithi027.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00948295 s, 111 MB/s 2024-09-12T03:21:12.168 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-12T03:21:12.168 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T03:21:12.224 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:21:12.225 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol84490/a842cae4-61b6-4f73-a96f-03c1581e2365/subvolume_file.7 bs=1M conv=fdatasync count=1 seek=0) 2024-09-12T03:21:12.291 INFO:teuthology.orchestra.run.smithi027.stderr:1+0 records in 2024-09-12T03:21:12.291 INFO:teuthology.orchestra.run.smithi027.stderr:1+0 records out 2024-09-12T03:21:12.291 INFO:teuthology.orchestra.run.smithi027.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00964456 s, 109 MB/s 2024-09-12T03:21:12.292 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot create cephfs subvol84490 subvol_snap39590 2024-09-12T03:21:12.440 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:12.440+0000 7fbc929c0640 1 Processor -- start 2024-09-12T03:21:12.440 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:12.440+0000 7fbc929c0640 1 -- start start 2024-09-12T03:21:12.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:12.440+0000 7fbc929c0640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fbc8c162040 0x7fbc8c0d1da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:12.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:12.440+0000 7fbc929c0640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fbc8c06dfd0 0x7fbc8c0b7a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:12.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:12.440+0000 7fbc929c0640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbc8c0d2e30 con 0x7fbc8c0d3790 2024-09-12T03:21:12.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:12.440+0000 7fbc929c0640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7fbc8c0b7f90 con 0x7fbc8c06dfd0 2024-09-12T03:21:12.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:12.440+0000 7fbc929c0640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fbc8c162420 con 0x7fbc8c162040 2024-09-12T03:21:12.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:12.441+0000 7fbc90f5d640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fbc8c06dfd0 0x7fbc8c0b7a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:12.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:12.441+0000 7fbc90f5d640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fbc8c06dfd0 0x7fbc8c0b7a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57412/0 (socket says 172.21.15.27:57412) 2024-09-12T03:21:12.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:12.441+0000 7fbc90f5d640 1 -- 172.21.15.27:0/943463814 learned_addr learned my addr 172.21.15.27:0/943463814 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:21:12.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:12.441+0000 7fbc90f5d640 1 -- 172.21.15.27:0/943463814 >> v2:172.21.15.45:3300/0 conn(0x7fbc8c162040 msgr2=0x7fbc8c0d1da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:12.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:12.441+0000 7fbc90f5d640 1 --2- 172.21.15.27:0/943463814 >> v2:172.21.15.45:3300/0 conn(0x7fbc8c162040 0x7fbc8c0d1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:12.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:12.441+0000 7fbc90f5d640 1 -- 172.21.15.27:0/943463814 >> v1:172.21.15.45:6789/0 conn(0x7fbc8c0d3790 legacy=0x7fbc8c06d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:12.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:12.441+0000 7fbc90f5d640 1 -- 172.21.15.27:0/943463814 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc8c0d30d0 con 0x7fbc8c06dfd0 2024-09-12T03:21:12.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:12.442+0000 7fbc90f5d640 1 --2- 172.21.15.27:0/943463814 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbc8c06dfd0 0x7fbc8c0b7a50 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7fbc8c0d17b0 tx=0x7fbc8402a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e67f773b5adf669f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:12.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:12.442+0000 7fbc8b7fe640 1 -- 172.21.15.27:0/943463814 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc84036070 con 0x7fbc8c06dfd0 2024-09-12T03:21:12.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:12.442+0000 7fbc8b7fe640 1 -- 172.21.15.27:0/943463814 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc84031600 con 0x7fbc8c06dfd0 2024-09-12T03:21:12.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:12.442+0000 7fbc8b7fe640 1 -- 172.21.15.27:0/943463814 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc840319d0 con 0x7fbc8c06dfd0 2024-09-12T03:21:12.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:12.442+0000 7fbc929c0640 1 -- 172.21.15.27:0/943463814 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbc8c06dfd0 msgr2=0x7fbc8c0b7a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:12.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:12.442+0000 7fbc929c0640 1 --2- 172.21.15.27:0/943463814 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbc8c06dfd0 0x7fbc8c0b7a50 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7fbc8c0d17b0 tx=0x7fbc8402a6f0 comp rx=0 tx=0).stop 2024-09-12T03:21:12.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:12.442+0000 7fbc929c0640 1 -- 172.21.15.27:0/943463814 shutdown_connections 2024-09-12T03:21:12.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:12.442+0000 7fbc929c0640 1 --2- 172.21.15.27:0/943463814 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbc8c06dfd0 0x7fbc8c0b7a50 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:12.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:12.442+0000 7fbc929c0640 1 --2- 172.21.15.27:0/943463814 >> v2:172.21.15.45:3300/0 conn(0x7fbc8c162040 0x7fbc8c0d1da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:12.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:12.442+0000 7fbc929c0640 1 -- 172.21.15.27:0/943463814 >> 172.21.15.27:0/943463814 conn(0x7fbc8c0b6700 msgr2=0x7fbc8c0b6b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:12.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:12.442+0000 7fbc929c0640 1 -- 172.21.15.27:0/943463814 shutdown_connections 2024-09-12T03:21:12.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:12.442+0000 7fbc929c0640 1 -- 172.21.15.27:0/943463814 wait complete. 2024-09-12T03:21:12.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:12.444+0000 7fbc929c0640 1 Processor -- start 2024-09-12T03:21:12.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:12.444+0000 7fbc929c0640 1 -- start start 2024-09-12T03:21:12.446 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:12.444+0000 7fbc929c0640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbc8c06dfd0 0x7fbc8c158840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:12.446 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:12.444+0000 7fbc929c0640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fbc8c0d3790 0x7fbc8c0ce3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:12.446 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:12.444+0000 7fbc929c0640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbc8c162040 0x7fbc8c1ea810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:12.446 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:12.444+0000 7fbc929c0640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fbc8c0d4530 con 0x7fbc8c06dfd0 2024-09-12T03:21:12.446 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:12.444+0000 7fbc929c0640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fbc8c0cee00 con 0x7fbc8c162040 2024-09-12T03:21:12.446 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:12.444+0000 7fbc929c0640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fbc8c0d4150 con 0x7fbc8c0d3790 2024-09-12T03:21:12.446 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:12.444+0000 7fbc8bfff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbc8c06dfd0 0x7fbc8c158840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:12.447 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:12.444+0000 7fbc8bfff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbc8c06dfd0 0x7fbc8c158840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57416/0 (socket says 172.21.15.27:57416) 2024-09-12T03:21:12.447 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:12.444+0000 7fbc8bfff640 1 -- 172.21.15.27:0/1386446071 learned_addr learned my addr 172.21.15.27:0/1386446071 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:21:12.447 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:12.445+0000 7fbc90f5d640 1 --2- 172.21.15.27:0/1386446071 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbc8c162040 0x7fbc8c1ea810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:12.447 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:12.445+0000 7fbc83fff640 1 --2- 172.21.15.27:0/1386446071 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fbc8c0d3790 0x7fbc8c0ce3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:12.447 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:12.445+0000 7fbc8bfff640 1 -- 172.21.15.27:0/1386446071 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fbc8c0d3790 msgr2=0x7fbc8c0ce3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:12.447 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:12.445+0000 7fbc8bfff640 1 --2- 172.21.15.27:0/1386446071 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fbc8c0d3790 0x7fbc8c0ce3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:12.447 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:12.445+0000 7fbc8bfff640 1 -- 172.21.15.27:0/1386446071 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbc8c162040 msgr2=0x7fbc8c1ea810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:12.447 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:12.445+0000 7fbc8bfff640 1 --2- 172.21.15.27:0/1386446071 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbc8c162040 0x7fbc8c1ea810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:12.447 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:12.445+0000 7fbc8bfff640 1 -- 172.21.15.27:0/1386446071 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc8c0cd540 con 0x7fbc8c06dfd0 2024-09-12T03:21:12.447 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:12.445+0000 7fbc8bfff640 1 --2- 172.21.15.27:0/1386446071 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbc8c06dfd0 0x7fbc8c158840 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7fbc7c00b410 tx=0x7fbc7c00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:12.447 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:12.445+0000 7fbc897fa640 1 -- 172.21.15.27:0/1386446071 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc7c015020 con 0x7fbc8c06dfd0 2024-09-12T03:21:12.447 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:12.445+0000 7fbc897fa640 1 -- 172.21.15.27:0/1386446071 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc7c011850 con 0x7fbc8c06dfd0 2024-09-12T03:21:12.447 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:12.445+0000 7fbc929c0640 1 -- 172.21.15.27:0/1386446071 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc8c0cd820 con 0x7fbc8c06dfd0 2024-09-12T03:21:12.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:12.445+0000 7fbc897fa640 1 -- 172.21.15.27:0/1386446071 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc7c011c40 con 0x7fbc8c06dfd0 2024-09-12T03:21:12.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:12.445+0000 7fbc929c0640 1 -- 172.21.15.27:0/1386446071 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc8c1e74e0 con 0x7fbc8c06dfd0 2024-09-12T03:21:12.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:12.446+0000 7fbc929c0640 1 -- 172.21.15.27:0/1386446071 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc8c0b7a50 con 0x7fbc8c06dfd0 2024-09-12T03:21:12.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:12.447+0000 7fbc897fa640 1 -- 172.21.15.27:0/1386446071 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) ==== 90651+0+0 (secure 0 0 0) 0x7fbc7c018020 con 0x7fbc8c06dfd0 2024-09-12T03:21:12.449 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:12.447+0000 7fbc897fa640 1 --2- 172.21.15.27:0/1386446071 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fbc6c06bfe0 0x7fbc6c06e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:12.449 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:12.447+0000 7fbc897fa640 1 -- 172.21.15.27:0/1386446071 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5709+0+0 (secure 0 0 0) 0x7fbc7c08b080 con 0x7fbc8c06dfd0 2024-09-12T03:21:12.449 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:12.447+0000 7fbc83fff640 1 --2- 172.21.15.27:0/1386446071 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fbc6c06bfe0 0x7fbc6c06e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:12.449 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:12.448+0000 7fbc83fff640 1 --2- 172.21.15.27:0/1386446071 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fbc6c06bfe0 0x7fbc6c06e4a0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fbc8c0d55c0 tx=0x7fbc74009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:12.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:12.451+0000 7fbc897fa640 1 -- 172.21.15.27:0/1386446071 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fbc7c01f020 con 0x7fbc8c06dfd0 2024-09-12T03:21:12.657 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:12.657+0000 7fbc929c0640 1 -- 172.21.15.27:0/1386446071 --> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot create", "vol_name": "cephfs", "sub_name": "subvol84490", "snap_name": "subvol_snap39590", "target": ["mon-mgr", ""]}) -- 0x7fbc8c0ba370 con 0x7fbc6c06bfe0 2024-09-12T03:21:12.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:12.684+0000 7fbc897fa640 1 -- 172.21.15.27:0/1386446071 <== mgr.4113 v2:172.21.15.27:6836/1226920526 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fbc740012e0 con 0x7fbc6c06bfe0 2024-09-12T03:21:12.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:12.686+0000 7fbc929c0640 1 -- 172.21.15.27:0/1386446071 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fbc6c06bfe0 msgr2=0x7fbc6c06e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:12.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:12.686+0000 7fbc929c0640 1 --2- 172.21.15.27:0/1386446071 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fbc6c06bfe0 0x7fbc6c06e4a0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fbc8c0d55c0 tx=0x7fbc74009040 comp rx=0 tx=0).stop 2024-09-12T03:21:12.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:12.686+0000 7fbc929c0640 1 -- 172.21.15.27:0/1386446071 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbc8c06dfd0 msgr2=0x7fbc8c158840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:12.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:12.686+0000 7fbc929c0640 1 --2- 172.21.15.27:0/1386446071 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbc8c06dfd0 0x7fbc8c158840 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7fbc7c00b410 tx=0x7fbc7c00b8d0 comp rx=0 tx=0).stop 2024-09-12T03:21:12.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:12.686+0000 7fbc929c0640 1 -- 172.21.15.27:0/1386446071 shutdown_connections 2024-09-12T03:21:12.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:12.686+0000 7fbc929c0640 1 --2- 172.21.15.27:0/1386446071 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbc8c162040 0x7fbc8c1ea810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:12.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:12.686+0000 7fbc929c0640 1 --2- 172.21.15.27:0/1386446071 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fbc8c0d3790 0x7fbc8c0ce3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:12.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:12.686+0000 7fbc929c0640 1 --2- 172.21.15.27:0/1386446071 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fbc6c06bfe0 0x7fbc6c06e4a0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:12.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:12.686+0000 7fbc929c0640 1 --2- 172.21.15.27:0/1386446071 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbc8c06dfd0 0x7fbc8c158840 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:12.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:12.686+0000 7fbc929c0640 1 -- 172.21.15.27:0/1386446071 >> 172.21.15.27:0/1386446071 conn(0x7fbc8c0b6700 msgr2=0x7fbc8c0d46b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:12.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:12.686+0000 7fbc929c0640 1 -- 172.21.15.27:0/1386446071 shutdown_connections 2024-09-12T03:21:12.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:12.686+0000 7fbc929c0640 1 -- 172.21.15.27:0/1386446071 wait complete. 2024-09-12T03:21:12.697 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs subvol84490 subvol_snap39590 subvol_clone8340 2024-09-12T03:21:12.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:12.805+0000 7f19796ba640 1 Processor -- start 2024-09-12T03:21:12.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:12.806+0000 7f19796ba640 1 -- start start 2024-09-12T03:21:12.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:12.806+0000 7f19796ba640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f1974162040 0x7f19740d1da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:12.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:12.806+0000 7f19796ba640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f197406dfd0 0x7f19740b7a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:12.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:12.806+0000 7f19796ba640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f19740d2e30 con 0x7f19740d3790 2024-09-12T03:21:12.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:12.806+0000 7f19796ba640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f19740b7f90 con 0x7f1974162040 2024-09-12T03:21:12.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:12.806+0000 7f19796ba640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f1974162420 con 0x7f197406dfd0 2024-09-12T03:21:12.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:12.806+0000 7f1972ffd640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f1974162040 0x7f19740d1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:12.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:12.806+0000 7f1972ffd640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f1974162040 0x7f19740d1da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57420/0 (socket says 172.21.15.27:57420) 2024-09-12T03:21:12.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:12.806+0000 7f1972ffd640 1 -- 172.21.15.27:0/1165428443 learned_addr learned my addr 172.21.15.27:0/1165428443 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:21:12.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:12.807+0000 7f19737fe640 1 --2- 172.21.15.27:0/1165428443 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f197406dfd0 0x7f19740b7a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:12.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:12.807+0000 7f1972ffd640 1 -- 172.21.15.27:0/1165428443 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f197406dfd0 msgr2=0x7f19740b7a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:12.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:12.807+0000 7f1972ffd640 1 --2- 172.21.15.27:0/1165428443 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f197406dfd0 0x7f19740b7a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:12.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:12.807+0000 7f1972ffd640 1 -- 172.21.15.27:0/1165428443 >> v1:172.21.15.45:6789/0 conn(0x7f19740d3790 legacy=0x7f197406d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:12.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:12.807+0000 7f1972ffd640 1 -- 172.21.15.27:0/1165428443 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19740d30d0 con 0x7f1974162040 2024-09-12T03:21:12.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:12.807+0000 7f19737fe640 1 --2- 172.21.15.27:0/1165428443 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f197406dfd0 0x7f19740b7a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:21:12.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:12.808+0000 7f1972ffd640 1 --2- 172.21.15.27:0/1165428443 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1974162040 0x7f19740d1da0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f196c009870 tx=0x7f196c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1b7e03bb6e8ab4e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:12.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:12.808+0000 7f1971ffb640 1 -- 172.21.15.27:0/1165428443 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f196c037070 con 0x7f1974162040 2024-09-12T03:21:12.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:12.808+0000 7f1971ffb640 1 -- 172.21.15.27:0/1165428443 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f196c032440 con 0x7f1974162040 2024-09-12T03:21:12.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:12.808+0000 7f1971ffb640 1 -- 172.21.15.27:0/1165428443 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f196c032810 con 0x7f1974162040 2024-09-12T03:21:12.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:12.808+0000 7f19796ba640 1 -- 172.21.15.27:0/1165428443 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1974162040 msgr2=0x7f19740d1da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:12.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:12.808+0000 7f19796ba640 1 --2- 172.21.15.27:0/1165428443 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1974162040 0x7f19740d1da0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f196c009870 tx=0x7f196c02a270 comp rx=0 tx=0).stop 2024-09-12T03:21:12.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:12.808+0000 7f19796ba640 1 -- 172.21.15.27:0/1165428443 shutdown_connections 2024-09-12T03:21:12.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:12.808+0000 7f19796ba640 1 --2- 172.21.15.27:0/1165428443 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f197406dfd0 0x7f19740b7a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:12.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:12.808+0000 7f19796ba640 1 --2- 172.21.15.27:0/1165428443 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1974162040 0x7f19740d1da0 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:12.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:12.808+0000 7f19796ba640 1 -- 172.21.15.27:0/1165428443 >> 172.21.15.27:0/1165428443 conn(0x7f19740b6700 msgr2=0x7f19740b6b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:12.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:12.808+0000 7f19796ba640 1 -- 172.21.15.27:0/1165428443 shutdown_connections 2024-09-12T03:21:12.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:12.808+0000 7f19796ba640 1 -- 172.21.15.27:0/1165428443 wait complete. 2024-09-12T03:21:12.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:12.810+0000 7f19796ba640 1 Processor -- start 2024-09-12T03:21:12.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:12.810+0000 7f19796ba640 1 -- start start 2024-09-12T03:21:12.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:12.810+0000 7f19796ba640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f197406dfd0 0x7f197415fe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:12.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:12.810+0000 7f19796ba640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f19740d3790 0x7f197415eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:12.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:12.810+0000 7f19796ba640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1974162040 0x7f19741f82a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:12.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:12.810+0000 7f19796ba640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f19740d4530 con 0x7f19740d3790 2024-09-12T03:21:12.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:12.810+0000 7f19796ba640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f19740cee00 con 0x7f1974162040 2024-09-12T03:21:12.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:12.810+0000 7f19796ba640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f19740d4150 con 0x7f197406dfd0 2024-09-12T03:21:12.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:12.811+0000 7f19727fc640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f19740d3790 0x7f197415eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:12.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:12.811+0000 7f19727fc640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f19740d3790 0x7f197415eba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57436/0 (socket says 172.21.15.27:57436) 2024-09-12T03:21:12.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:12.811+0000 7f19727fc640 1 -- 172.21.15.27:0/3558509847 learned_addr learned my addr 172.21.15.27:0/3558509847 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:21:12.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:12.811+0000 7f1972ffd640 1 --2- 172.21.15.27:0/3558509847 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f197406dfd0 0x7f197415fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:12.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:12.811+0000 7f19737fe640 1 --2- 172.21.15.27:0/3558509847 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1974162040 0x7f19741f82a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:12.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:12.811+0000 7f19727fc640 1 -- 172.21.15.27:0/3558509847 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f197406dfd0 msgr2=0x7f197415fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:12.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:12.811+0000 7f19727fc640 1 --2- 172.21.15.27:0/3558509847 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f197406dfd0 0x7f197415fe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:12.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:12.811+0000 7f19727fc640 1 -- 172.21.15.27:0/3558509847 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1974162040 msgr2=0x7f19741f82a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:12.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:12.811+0000 7f19727fc640 1 --2- 172.21.15.27:0/3558509847 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1974162040 0x7f19741f82a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:12.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:12.811+0000 7f19727fc640 1 -- 172.21.15.27:0/3558509847 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f197415e120 con 0x7f19740d3790 2024-09-12T03:21:12.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:12.811+0000 7f19727fc640 1 --2- 172.21.15.27:0/3558509847 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f19740d3790 0x7f197415eba0 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f195c00c470 tx=0x7f195c001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:12.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:12.811+0000 7f196b7fe640 1 -- 172.21.15.27:0/3558509847 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f195c0052d0 con 0x7f19740d3790 2024-09-12T03:21:12.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:12.811+0000 7f196b7fe640 1 -- 172.21.15.27:0/3558509847 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f195c005470 con 0x7f19740d3790 2024-09-12T03:21:12.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:12.811+0000 7f19796ba640 1 -- 172.21.15.27:0/3558509847 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19741f8d60 con 0x7f19740d3790 2024-09-12T03:21:12.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:12.811+0000 7f196b7fe640 1 -- 172.21.15.27:0/3558509847 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f195c007260 con 0x7f19740d3790 2024-09-12T03:21:12.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:12.811+0000 7f19796ba640 1 -- 172.21.15.27:0/3558509847 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f197415e6a0 con 0x7f19740d3790 2024-09-12T03:21:12.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:12.812+0000 7f19796ba640 1 -- 172.21.15.27:0/3558509847 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19741f9150 con 0x7f19740d3790 2024-09-12T03:21:12.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:12.813+0000 7f196b7fe640 1 -- 172.21.15.27:0/3558509847 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) ==== 90651+0+0 (secure 0 0 0) 0x7f195c007400 con 0x7f19740d3790 2024-09-12T03:21:12.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:12.813+0000 7f196b7fe640 1 --2- 172.21.15.27:0/3558509847 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f195406bfe0 0x7f195406e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:12.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:12.813+0000 7f196b7fe640 1 -- 172.21.15.27:0/3558509847 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5709+0+0 (secure 0 0 0) 0x7f195c08a4e0 con 0x7f19740d3790 2024-09-12T03:21:12.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:12.813+0000 7f1972ffd640 1 --2- 172.21.15.27:0/3558509847 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f195406bfe0 0x7f195406e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:12.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:12.814+0000 7f1972ffd640 1 --2- 172.21.15.27:0/3558509847 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f195406bfe0 0x7f195406e4a0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f196c034000 tx=0x7f196c02e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:12.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:12.818+0000 7f196b7fe640 1 -- 172.21.15.27:0/3558509847 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f195c058670 con 0x7f19740d3790 2024-09-12T03:21:13.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:13.028+0000 7f19796ba640 1 -- 172.21.15.27:0/3558509847 --> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "subvol84490", "snap_name": "subvol_snap39590", "target_sub_name": "subvol_clone8340", "target": ["mon-mgr", ""]}) -- 0x7f19740b7a50 con 0x7f195406bfe0 2024-09-12T03:21:13.109 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:13.109+0000 7f196b7fe640 1 -- 172.21.15.27:0/3558509847 <== mgr.4113 v2:172.21.15.27:6836/1226920526 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f19740b7a50 con 0x7f195406bfe0 2024-09-12T03:21:13.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:13.111+0000 7f19796ba640 1 -- 172.21.15.27:0/3558509847 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f195406bfe0 msgr2=0x7f195406e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:13.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:13.111+0000 7f19796ba640 1 --2- 172.21.15.27:0/3558509847 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f195406bfe0 0x7f195406e4a0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f196c034000 tx=0x7f196c02e000 comp rx=0 tx=0).stop 2024-09-12T03:21:13.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:13.111+0000 7f19796ba640 1 -- 172.21.15.27:0/3558509847 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f19740d3790 msgr2=0x7f197415eba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:13.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:13.111+0000 7f19796ba640 1 --2- 172.21.15.27:0/3558509847 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f19740d3790 0x7f197415eba0 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f195c00c470 tx=0x7f195c001e70 comp rx=0 tx=0).stop 2024-09-12T03:21:13.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:13.111+0000 7f19796ba640 1 -- 172.21.15.27:0/3558509847 shutdown_connections 2024-09-12T03:21:13.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:13.111+0000 7f19796ba640 1 --2- 172.21.15.27:0/3558509847 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1974162040 0x7f19741f82a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:13.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:13.111+0000 7f19796ba640 1 --2- 172.21.15.27:0/3558509847 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f195406bfe0 0x7f195406e4a0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:13.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:13.111+0000 7f19796ba640 1 --2- 172.21.15.27:0/3558509847 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f19740d3790 0x7f197415eba0 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:13.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:13.111+0000 7f19796ba640 1 --2- 172.21.15.27:0/3558509847 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f197406dfd0 0x7f197415fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:13.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:13.111+0000 7f19796ba640 1 -- 172.21.15.27:0/3558509847 >> 172.21.15.27:0/3558509847 conn(0x7f19740b6700 msgr2=0x7f197415d8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:13.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:13.111+0000 7f19796ba640 1 -- 172.21.15.27:0/3558509847 shutdown_connections 2024-09-12T03:21:13.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:13.111+0000 7f19796ba640 1 -- 172.21.15.27:0/3558509847 wait complete. 2024-09-12T03:21:13.113 INFO:tasks.ceph.mgr.y.smithi027.stderr:2024-09-12T03:21:13.113+0000 7f32638ef640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:21:13.113 INFO:tasks.ceph.mgr.y.smithi027.stderr:2024-09-12T03:21:13.113+0000 7f32638ef640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:21:13.113 INFO:tasks.ceph.mgr.y.smithi027.stderr:2024-09-12T03:21:13.113+0000 7f32638ef640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:21:13.113 INFO:tasks.ceph.mgr.y.smithi027.stderr:2024-09-12T03:21:13.113+0000 7f32638ef640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:21:13.113 INFO:tasks.ceph.mgr.y.smithi027.stderr:2024-09-12T03:21:13.113+0000 7f32638ef640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:21:13.125 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone8340 2024-09-12T03:21:13.126 INFO:tasks.ceph.mgr.y.smithi027.stderr:2024-09-12T03:21:13.126+0000 7f32630ee640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:21:13.126 INFO:tasks.ceph.mgr.y.smithi027.stderr:2024-09-12T03:21:13.126+0000 7f32630ee640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:21:13.126 INFO:tasks.ceph.mgr.y.smithi027.stderr:2024-09-12T03:21:13.126+0000 7f32630ee640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:21:13.126 INFO:tasks.ceph.mgr.y.smithi027.stderr:2024-09-12T03:21:13.126+0000 7f32630ee640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:21:13.126 INFO:tasks.ceph.mgr.y.smithi027.stderr:2024-09-12T03:21:13.126+0000 7f32630ee640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:21:13.254 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:13.254+0000 7f170b30a640 1 Processor -- start 2024-09-12T03:21:13.254 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:13.254+0000 7f170b30a640 1 -- start start 2024-09-12T03:21:13.255 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:13.255+0000 7f170b30a640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f170415e900 0x7f170406cc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:13.255 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:13.255+0000 7f170b30a640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f17040b1ae0 0x7f170406e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:13.255 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:13.255+0000 7f170b30a640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f170415ece0 con 0x7f170406fe70 2024-09-12T03:21:13.255 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:13.255+0000 7f170b30a640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f170406d540 con 0x7f17040b1ae0 2024-09-12T03:21:13.256 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:13.255+0000 7f170b30a640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f170406d820 con 0x7f170415e900 2024-09-12T03:21:13.256 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:13.255+0000 7f17088a5640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f17040b1ae0 0x7f170406e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:13.256 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:13.255+0000 7f17090a6640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f170415e900 0x7f170406cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:13.256 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:13.256+0000 7f17098a7640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f170406fe70 0x7f17040af520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.27:43242/0 (socket says 172.21.15.27:43242) 2024-09-12T03:21:13.256 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:13.256+0000 7f17088a5640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f17040b1ae0 0x7f170406e5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.27:41812/0 (socket says 172.21.15.27:41812) 2024-09-12T03:21:13.256 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:13.256+0000 7f17098a7640 1 -- 172.21.15.27:0/2413356010 learned_addr learned my addr 172.21.15.27:0/2413356010 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:21:13.257 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:13.256+0000 7f17088a5640 1 -- 172.21.15.27:0/2413356010 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f170415e900 msgr2=0x7f170406cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:13.257 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:13.256+0000 7f17088a5640 1 --2- 172.21.15.27:0/2413356010 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f170415e900 0x7f170406cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:13.257 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:13.256+0000 7f17088a5640 1 -- 172.21.15.27:0/2413356010 >> v1:172.21.15.45:6789/0 conn(0x7f170406fe70 legacy=0x7f17040af520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:13.257 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:13.256+0000 7f17088a5640 1 -- 172.21.15.27:0/2413356010 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f170406db00 con 0x7f17040b1ae0 2024-09-12T03:21:13.257 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:13.256+0000 7f16fbfff640 1 -- 172.21.15.27:0/2413356010 <== mon.1 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3876111041 0 0) 0x7f170415ece0 con 0x7f170406fe70 2024-09-12T03:21:13.257 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:13.257+0000 7f17088a5640 1 --2- 172.21.15.27:0/2413356010 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f17040b1ae0 0x7f170406e5e0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f16f40097a0 tx=0x7f16f402a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=102a1568351eba4b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:13.257 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:13.257+0000 7f16fbfff640 1 -- 172.21.15.27:0/2413356010 <== mon.1 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16f4037070 con 0x7f17040b1ae0 2024-09-12T03:21:13.257 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:13.257+0000 7f16fbfff640 1 -- 172.21.15.27:0/2413356010 <== mon.1 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16f4032440 con 0x7f17040b1ae0 2024-09-12T03:21:13.258 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:13.257+0000 7f16fbfff640 1 -- 172.21.15.27:0/2413356010 <== mon.1 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16f4032810 con 0x7f17040b1ae0 2024-09-12T03:21:13.258 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:13.258+0000 7f170b30a640 1 -- 172.21.15.27:0/2413356010 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f17040b1ae0 msgr2=0x7f170406e5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:13.258 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:13.258+0000 7f170b30a640 1 --2- 172.21.15.27:0/2413356010 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f17040b1ae0 0x7f170406e5e0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f16f40097a0 tx=0x7f16f402a180 comp rx=0 tx=0).stop 2024-09-12T03:21:13.259 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:13.258+0000 7f170b30a640 1 -- 172.21.15.27:0/2413356010 shutdown_connections 2024-09-12T03:21:13.259 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:13.258+0000 7f170b30a640 1 --2- 172.21.15.27:0/2413356010 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f17040b1ae0 0x7f170406e5e0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:13.259 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:13.258+0000 7f170b30a640 1 --2- 172.21.15.27:0/2413356010 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f170415e900 0x7f170406cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:13.259 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:13.258+0000 7f170b30a640 1 -- 172.21.15.27:0/2413356010 >> 172.21.15.27:0/2413356010 conn(0x7f1704073800 msgr2=0x7f170409ba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:13.259 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:13.258+0000 7f170b30a640 1 -- 172.21.15.27:0/2413356010 shutdown_connections 2024-09-12T03:21:13.259 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:13.258+0000 7f170b30a640 1 -- 172.21.15.27:0/2413356010 wait complete. 2024-09-12T03:21:13.259 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:13.259+0000 7f170b30a640 1 Processor -- start 2024-09-12T03:21:13.259 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:13.260+0000 7f170b30a640 1 -- start start 2024-09-12T03:21:13.261 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:13.260+0000 7f170b30a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f170406fe70 0x7f1704157cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:13.262 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:13.260+0000 7f170b30a640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f17040b1ae0 0x7f1704156450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:13.262 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:13.260+0000 7f170b30a640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f170415e900 0x7f170409ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:13.262 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:13.260+0000 7f170b30a640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f170406d300 con 0x7f170415e900 2024-09-12T03:21:13.262 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:13.260+0000 7f170b30a640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f170406d180 con 0x7f170406fe70 2024-09-12T03:21:13.262 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:13.260+0000 7f170b30a640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f170406eb20 con 0x7f17040b1ae0 2024-09-12T03:21:13.262 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:13.261+0000 7f17098a7640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f170415e900 0x7f170409ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:13.262 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:13.261+0000 7f17098a7640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f170415e900 0x7f170409ffd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57468/0 (socket says 172.21.15.27:57468) 2024-09-12T03:21:13.262 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:13.261+0000 7f17098a7640 1 -- 172.21.15.27:0/1994787516 learned_addr learned my addr 172.21.15.27:0/1994787516 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:21:13.262 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:13.261+0000 7f17088a5640 1 --2- 172.21.15.27:0/1994787516 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f17040b1ae0 0x7f1704156450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:13.262 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:13.261+0000 7f17098a7640 1 -- 172.21.15.27:0/1994787516 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f17040b1ae0 msgr2=0x7f1704156450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:13.262 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:13.261+0000 7f17098a7640 1 --2- 172.21.15.27:0/1994787516 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f17040b1ae0 0x7f1704156450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:13.262 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:13.261+0000 7f17098a7640 1 -- 172.21.15.27:0/1994787516 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f170406fe70 msgr2=0x7f1704157cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:13.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:13.261+0000 7f17098a7640 1 --2- 172.21.15.27:0/1994787516 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f170406fe70 0x7f1704157cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:13.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:13.261+0000 7f17098a7640 1 -- 172.21.15.27:0/1994787516 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1704156c50 con 0x7f170415e900 2024-09-12T03:21:13.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:13.261+0000 7f17098a7640 1 --2- 172.21.15.27:0/1994787516 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f170415e900 0x7f170409ffd0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f16fc00c700 tx=0x7f16fc00cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:13.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:13.261+0000 7f16f9ffb640 1 -- 172.21.15.27:0/1994787516 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16fc018070 con 0x7f170415e900 2024-09-12T03:21:13.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:13.261+0000 7f16f9ffb640 1 -- 172.21.15.27:0/1994787516 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16fc00ce10 con 0x7f170415e900 2024-09-12T03:21:13.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:13.261+0000 7f170b30a640 1 -- 172.21.15.27:0/1994787516 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1704156f30 con 0x7f170415e900 2024-09-12T03:21:13.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:13.261+0000 7f16f9ffb640 1 -- 172.21.15.27:0/1994787516 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16fc00a880 con 0x7f170415e900 2024-09-12T03:21:13.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:13.262+0000 7f170b30a640 1 -- 172.21.15.27:0/1994787516 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17041574f0 con 0x7f170415e900 2024-09-12T03:21:13.264 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:13.263+0000 7f170b30a640 1 -- 172.21.15.27:0/1994787516 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f170406e5e0 con 0x7f170415e900 2024-09-12T03:21:13.264 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:13.263+0000 7f16f9ffb640 1 -- 172.21.15.27:0/1994787516 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) ==== 90651+0+0 (secure 0 0 0) 0x7f16fc00aa20 con 0x7f170415e900 2024-09-12T03:21:13.264 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:13.264+0000 7f16f9ffb640 1 --2- 172.21.15.27:0/1994787516 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f16e806bfe0 0x7f16e806e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:13.264 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:13.264+0000 7f16f9ffb640 1 -- 172.21.15.27:0/1994787516 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5709+0+0 (secure 0 0 0) 0x7f16fc08a660 con 0x7f170415e900 2024-09-12T03:21:13.264 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:13.264+0000 7f17090a6640 1 --2- 172.21.15.27:0/1994787516 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f16e806bfe0 0x7f16e806e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:13.264 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:13.264+0000 7f17090a6640 1 --2- 172.21.15.27:0/1994787516 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f16e806bfe0 0x7f16e806e4a0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f1700006a90 tx=0x7f1700006a20 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:13.269 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:13.269+0000 7f16f9ffb640 1 -- 172.21.15.27:0/1994787516 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f16fc058750 con 0x7f170415e900 2024-09-12T03:21:13.494 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:13.494+0000 7f170b30a640 1 -- 172.21.15.27:0/1994787516 --> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone8340", "target": ["mon-mgr", ""]}) -- 0x7f17040713c0 con 0x7f16e806bfe0 2024-09-12T03:21:14.768 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:14.768+0000 7f16f9ffb640 1 -- 172.21.15.27:0/1994787516 <== mgr.4113 v2:172.21.15.27:6836/1226920526 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+46 (secure 0 0 0) 0x7f17040713c0 con 0x7f16e806bfe0 2024-09-12T03:21:14.768 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-12T03:21:14.768 INFO:teuthology.orchestra.run.smithi027.stdout: "status": { 2024-09-12T03:21:14.768 INFO:teuthology.orchestra.run.smithi027.stdout: "state": "complete" 2024-09-12T03:21:14.768 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:21:14.769 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-12T03:21:14.771 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:14.771+0000 7f170b30a640 1 -- 172.21.15.27:0/1994787516 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f16e806bfe0 msgr2=0x7f16e806e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:14.771 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:14.771+0000 7f170b30a640 1 --2- 172.21.15.27:0/1994787516 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f16e806bfe0 0x7f16e806e4a0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f1700006a90 tx=0x7f1700006a20 comp rx=0 tx=0).stop 2024-09-12T03:21:14.771 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:14.771+0000 7f170b30a640 1 -- 172.21.15.27:0/1994787516 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f170415e900 msgr2=0x7f170409ffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:14.771 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:14.771+0000 7f170b30a640 1 --2- 172.21.15.27:0/1994787516 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f170415e900 0x7f170409ffd0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f16fc00c700 tx=0x7f16fc00cbc0 comp rx=0 tx=0).stop 2024-09-12T03:21:14.771 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:14.771+0000 7f170b30a640 1 -- 172.21.15.27:0/1994787516 shutdown_connections 2024-09-12T03:21:14.771 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:14.771+0000 7f170b30a640 1 --2- 172.21.15.27:0/1994787516 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f16e806bfe0 0x7f16e806e4a0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:14.771 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:14.771+0000 7f170b30a640 1 --2- 172.21.15.27:0/1994787516 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f170415e900 0x7f170409ffd0 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:14.772 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:14.771+0000 7f170b30a640 1 --2- 172.21.15.27:0/1994787516 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f17040b1ae0 0x7f1704156450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:14.772 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:14.771+0000 7f170b30a640 1 --2- 172.21.15.27:0/1994787516 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f170406fe70 0x7f1704157cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:14.772 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:14.771+0000 7f170b30a640 1 -- 172.21.15.27:0/1994787516 >> 172.21.15.27:0/1994787516 conn(0x7f1704073800 msgr2=0x7f17040b2170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:14.772 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:14.771+0000 7f170b30a640 1 -- 172.21.15.27:0/1994787516 shutdown_connections 2024-09-12T03:21:14.772 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:14.771+0000 7f170b30a640 1 -- 172.21.15.27:0/1994787516 wait complete. 2024-09-12T03:21:14.785 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:21:14.785 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cat ./volumes/_nogroup/subvol_clone8340/.meta) 2024-09-12T03:21:14.815 INFO:teuthology.orchestra.run.smithi027.stdout:[GLOBAL] 2024-09-12T03:21:14.815 INFO:teuthology.orchestra.run.smithi027.stdout:version = 2 2024-09-12T03:21:14.815 INFO:teuthology.orchestra.run.smithi027.stdout:type = clone 2024-09-12T03:21:14.815 INFO:teuthology.orchestra.run.smithi027.stdout:path = /volumes/_nogroup/subvol_clone8340/fe9d5d1c-b87b-434b-8988-e68eb0a4b017 2024-09-12T03:21:14.815 INFO:teuthology.orchestra.run.smithi027.stdout:state = complete 2024-09-12T03:21:14.815 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:21:14.817 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol84490 2024-09-12T03:21:14.967 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:14.967+0000 7f642eb9f640 1 Processor -- start 2024-09-12T03:21:14.967 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:14.967+0000 7f642eb9f640 1 -- start start 2024-09-12T03:21:14.969 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:14.968+0000 7f642eb9f640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f6428161b20 0x7f64280d3e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:14.969 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:14.968+0000 7f642eb9f640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f6428161f00 0x7f64280cf580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:14.969 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:14.968+0000 7f642eb9f640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f64280cfac0 0x7f64280b7770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:14.969 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:14.968+0000 7f642eb9f640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f64280d4390 con 0x7f64280cfac0 2024-09-12T03:21:14.969 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:14.968+0000 7f642eb9f640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f64280d3b90 con 0x7f6428161b20 2024-09-12T03:21:14.970 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:14.968+0000 7f642eb9f640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f642806c900 con 0x7f6428161f00 2024-09-12T03:21:14.970 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:14.968+0000 7f642e39e640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f64280cfac0 0x7f64280b7770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:14.970 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:14.968+0000 7f642e39e640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f64280cfac0 0x7f64280b7770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57476/0 (socket says 172.21.15.27:57476) 2024-09-12T03:21:14.970 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:14.968+0000 7f642e39e640 1 -- 172.21.15.27:0/4020237841 learned_addr learned my addr 172.21.15.27:0/4020237841 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:21:14.970 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:14.968+0000 7f642d39c640 1 --2- 172.21.15.27:0/4020237841 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f6428161f00 0x7f64280cf580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:14.970 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:14.968+0000 7f642db9d640 1 --2- 172.21.15.27:0/4020237841 >> v2:172.21.15.45:3300/0 conn(0x7f6428161b20 0x7f64280d3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:14.970 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:14.968+0000 7f642e39e640 1 -- 172.21.15.27:0/4020237841 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f6428161f00 msgr2=0x7f64280cf580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:14.970 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:14.968+0000 7f642e39e640 1 --2- 172.21.15.27:0/4020237841 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f6428161f00 0x7f64280cf580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:14.970 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:14.968+0000 7f642e39e640 1 -- 172.21.15.27:0/4020237841 >> v2:172.21.15.45:3300/0 conn(0x7f6428161b20 msgr2=0x7f64280d3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:14.970 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:14.968+0000 7f642e39e640 1 --2- 172.21.15.27:0/4020237841 >> v2:172.21.15.45:3300/0 conn(0x7f6428161b20 0x7f64280d3e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:14.970 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:14.969+0000 7f642e39e640 1 -- 172.21.15.27:0/4020237841 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6428063070 con 0x7f64280cfac0 2024-09-12T03:21:14.970 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:14.969+0000 7f642d39c640 1 --2- 172.21.15.27:0/4020237841 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f6428161f00 0x7f64280cf580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:21:14.970 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:14.969+0000 7f642e39e640 1 --2- 172.21.15.27:0/4020237841 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f64280cfac0 0x7f64280b7770 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f642806da70 tx=0x7f642002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f70c50fcb7295b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:14.971 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:14.969+0000 7f642cb9b640 1 -- 172.21.15.27:0/4020237841 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6420036070 con 0x7f64280cfac0 2024-09-12T03:21:14.971 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:14.969+0000 7f642cb9b640 1 -- 172.21.15.27:0/4020237841 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6420031600 con 0x7f64280cfac0 2024-09-12T03:21:14.971 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:14.969+0000 7f642cb9b640 1 -- 172.21.15.27:0/4020237841 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64200319d0 con 0x7f64280cfac0 2024-09-12T03:21:14.973 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:14.970+0000 7f642eb9f640 1 -- 172.21.15.27:0/4020237841 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f64280cfac0 msgr2=0x7f64280b7770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:14.973 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:14.970+0000 7f642eb9f640 1 --2- 172.21.15.27:0/4020237841 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f64280cfac0 0x7f64280b7770 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f642806da70 tx=0x7f642002a6a0 comp rx=0 tx=0).stop 2024-09-12T03:21:14.973 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:14.970+0000 7f642eb9f640 1 -- 172.21.15.27:0/4020237841 shutdown_connections 2024-09-12T03:21:14.974 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:14.970+0000 7f642eb9f640 1 --2- 172.21.15.27:0/4020237841 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f64280cfac0 0x7f64280b7770 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:14.974 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:14.970+0000 7f642eb9f640 1 --2- 172.21.15.27:0/4020237841 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f6428161f00 0x7f64280cf580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:14.974 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:14.970+0000 7f642eb9f640 1 --2- 172.21.15.27:0/4020237841 >> v2:172.21.15.45:3300/0 conn(0x7f6428161b20 0x7f64280d3e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:14.974 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:14.970+0000 7f642eb9f640 1 -- 172.21.15.27:0/4020237841 >> 172.21.15.27:0/4020237841 conn(0x7f64280d4640 msgr2=0x7f64280d4a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:14.974 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:14.970+0000 7f642eb9f640 1 -- 172.21.15.27:0/4020237841 shutdown_connections 2024-09-12T03:21:14.974 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:14.970+0000 7f642eb9f640 1 -- 172.21.15.27:0/4020237841 wait complete. 2024-09-12T03:21:14.974 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:14.971+0000 7f642eb9f640 1 Processor -- start 2024-09-12T03:21:14.974 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:14.971+0000 7f642eb9f640 1 -- start start 2024-09-12T03:21:14.974 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:14.972+0000 7f642eb9f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6428161b20 0x7f642814ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:14.974 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:14.972+0000 7f642eb9f640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f6428161f00 0x7f6428154a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:14.974 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:14.972+0000 7f642eb9f640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6428154fb0 0x7f642815ff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:14.974 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:14.972+0000 7f642eb9f640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f642806cb50 con 0x7f6428154fb0 2024-09-12T03:21:14.975 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:14.972+0000 7f642eb9f640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f64280d3700 con 0x7f6428161b20 2024-09-12T03:21:14.975 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:14.972+0000 7f642eb9f640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f642806ce30 con 0x7f6428161f00 2024-09-12T03:21:14.975 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:14.972+0000 7f642e39e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6428154fb0 0x7f642815ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:14.975 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:14.972+0000 7f642e39e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6428154fb0 0x7f642815ff50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57480/0 (socket says 172.21.15.27:57480) 2024-09-12T03:21:14.975 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:14.972+0000 7f642e39e640 1 -- 172.21.15.27:0/3946669552 learned_addr learned my addr 172.21.15.27:0/3946669552 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:21:14.975 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:14.972+0000 7f642db9d640 1 --2- 172.21.15.27:0/3946669552 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6428161b20 0x7f642814ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:14.975 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:14.972+0000 7f642d39c640 1 --2- 172.21.15.27:0/3946669552 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f6428161f00 0x7f6428154a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:14.975 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:14.972+0000 7f642e39e640 1 -- 172.21.15.27:0/3946669552 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f6428161f00 msgr2=0x7f6428154a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:14.975 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:14.972+0000 7f642e39e640 1 --2- 172.21.15.27:0/3946669552 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f6428161f00 0x7f6428154a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:14.975 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:14.972+0000 7f642e39e640 1 -- 172.21.15.27:0/3946669552 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6428161b20 msgr2=0x7f642814ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:14.975 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:14.972+0000 7f642e39e640 1 --2- 172.21.15.27:0/3946669552 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6428161b20 0x7f642814ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:14.975 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:14.972+0000 7f642e39e640 1 -- 172.21.15.27:0/3946669552 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6428148340 con 0x7f6428154fb0 2024-09-12T03:21:14.975 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:14.972+0000 7f642db9d640 1 --2- 172.21.15.27:0/3946669552 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6428161b20 0x7f642814ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:21:14.976 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:14.973+0000 7f642e39e640 1 --2- 172.21.15.27:0/3946669552 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6428154fb0 0x7f642815ff50 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f642002ac90 tx=0x7f6420004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:14.976 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:14.973+0000 7f641e7fc640 1 -- 172.21.15.27:0/3946669552 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6420041020 con 0x7f6428154fb0 2024-09-12T03:21:14.976 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:14.973+0000 7f641e7fc640 1 -- 172.21.15.27:0/3946669552 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6420007780 con 0x7f6428154fb0 2024-09-12T03:21:14.976 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:14.973+0000 7f642eb9f640 1 -- 172.21.15.27:0/3946669552 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64280cc5f0 con 0x7f6428154fb0 2024-09-12T03:21:14.976 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:14.973+0000 7f641e7fc640 1 -- 172.21.15.27:0/3946669552 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6420036040 con 0x7f6428154fb0 2024-09-12T03:21:14.976 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:14.973+0000 7f642eb9f640 1 -- 172.21.15.27:0/3946669552 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64280ccbb0 con 0x7f6428154fb0 2024-09-12T03:21:14.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:14.974+0000 7f642eb9f640 1 -- 172.21.15.27:0/3946669552 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64280d3e50 con 0x7f6428154fb0 2024-09-12T03:21:14.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:14.974+0000 7f641e7fc640 1 -- 172.21.15.27:0/3946669552 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 18) ==== 90753+0+0 (secure 0 0 0) 0x7f6420044020 con 0x7f6428154fb0 2024-09-12T03:21:14.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:14.975+0000 7f641e7fc640 1 --2- 172.21.15.27:0/3946669552 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f640c06c100 0x7f640c06e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:14.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:14.975+0000 7f641e7fc640 1 -- 172.21.15.27:0/3946669552 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5709+0+0 (secure 0 0 0) 0x7f64200ac120 con 0x7f6428154fb0 2024-09-12T03:21:14.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:14.975+0000 7f642db9d640 1 --2- 172.21.15.27:0/3946669552 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f640c06c100 0x7f640c06e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:14.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:14.975+0000 7f642db9d640 1 --2- 172.21.15.27:0/3946669552 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f640c06c100 0x7f640c06e5c0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f6424006fd0 tx=0x7f6424008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:14.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:14.979+0000 7f641e7fc640 1 -- 172.21.15.27:0/3946669552 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f642004b030 con 0x7f6428154fb0 2024-09-12T03:21:15.179 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:15.179+0000 7f642eb9f640 1 -- 172.21.15.27:0/3946669552 --> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol84490", "target": ["mon-mgr", ""]}) -- 0x7f64280d5230 con 0x7f640c06c100 2024-09-12T03:21:15.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:15.189+0000 7f641e7fc640 1 -- 172.21.15.27:0/3946669552 <== mgr.4113 v2:172.21.15.27:6836/1226920526 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+67 (secure 0 0 0) 0x7f64280d5230 con 0x7f640c06c100 2024-09-12T03:21:15.189 INFO:teuthology.orchestra.run.smithi027.stdout:/volumes/_nogroup/subvol84490/a842cae4-61b6-4f73-a96f-03c1581e2365 2024-09-12T03:21:15.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:15.192+0000 7f642eb9f640 1 -- 172.21.15.27:0/3946669552 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f640c06c100 msgr2=0x7f640c06e5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:15.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:15.192+0000 7f642eb9f640 1 --2- 172.21.15.27:0/3946669552 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f640c06c100 0x7f640c06e5c0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f6424006fd0 tx=0x7f6424008040 comp rx=0 tx=0).stop 2024-09-12T03:21:15.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:15.192+0000 7f642eb9f640 1 -- 172.21.15.27:0/3946669552 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6428154fb0 msgr2=0x7f642815ff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:15.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:15.192+0000 7f642eb9f640 1 --2- 172.21.15.27:0/3946669552 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6428154fb0 0x7f642815ff50 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f642002ac90 tx=0x7f6420004a40 comp rx=0 tx=0).stop 2024-09-12T03:21:15.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:15.192+0000 7f642eb9f640 1 -- 172.21.15.27:0/3946669552 shutdown_connections 2024-09-12T03:21:15.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:15.192+0000 7f642eb9f640 1 --2- 172.21.15.27:0/3946669552 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f640c06c100 0x7f640c06e5c0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:15.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:15.192+0000 7f642eb9f640 1 --2- 172.21.15.27:0/3946669552 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6428154fb0 0x7f642815ff50 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:15.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:15.192+0000 7f642eb9f640 1 --2- 172.21.15.27:0/3946669552 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f6428161f00 0x7f6428154a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:15.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:15.192+0000 7f642eb9f640 1 --2- 172.21.15.27:0/3946669552 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6428161b20 0x7f642814ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:15.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:15.192+0000 7f642eb9f640 1 -- 172.21.15.27:0/3946669552 >> 172.21.15.27:0/3946669552 conn(0x7f64280d4640 msgr2=0x7f64280caa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:15.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:15.192+0000 7f642eb9f640 1 -- 172.21.15.27:0/3946669552 shutdown_connections 2024-09-12T03:21:15.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:15.192+0000 7f642eb9f640 1 -- 172.21.15.27:0/3946669552 wait complete. 2024-09-12T03:21:15.203 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol_clone8340 2024-09-12T03:21:15.308 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:15.308+0000 7fe0a4465640 1 Processor -- start 2024-09-12T03:21:15.309 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:15.309+0000 7fe0a4465640 1 -- start start 2024-09-12T03:21:15.310 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:15.309+0000 7fe0a4465640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fe09c15e990 0x7fe09c0b7600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:15.310 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:15.309+0000 7fe0a4465640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe09c0b6230 0x7fe09c0b6690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:15.310 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:15.310+0000 7fe0a4465640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe09c15ed70 con 0x7fe09c0b3600 2024-09-12T03:21:15.310 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:15.310+0000 7fe0a4465640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fe09c0b6bd0 con 0x7fe09c15e990 2024-09-12T03:21:15.310 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:15.310+0000 7fe0a4465640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fe09c0d2a30 con 0x7fe09c0b6230 2024-09-12T03:21:15.310 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:15.310+0000 7fe0a2a02640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fe09c0b3600 0x7fe09c0b3a70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:40698/0 (socket says 172.21.15.27:40698) 2024-09-12T03:21:15.310 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:15.310+0000 7fe0a2a02640 1 -- 172.21.15.27:0/4230517838 learned_addr learned my addr 172.21.15.27:0/4230517838 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:21:15.311 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:15.310+0000 7fe0a1a00640 1 --2- 172.21.15.27:0/4230517838 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe09c0b6230 0x7fe09c0b6690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:15.311 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:15.310+0000 7fe0a2201640 1 --2- 172.21.15.27:0/4230517838 >> v2:172.21.15.45:3300/0 conn(0x7fe09c15e990 0x7fe09c0b7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:15.311 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:15.310+0000 7fe0a11ff640 1 -- 172.21.15.27:0/4230517838 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3836265285 0 0) 0x7fe09c15ed70 con 0x7fe09c0b3600 2024-09-12T03:21:15.311 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:15.310+0000 7fe0a11ff640 1 -- 172.21.15.27:0/4230517838 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe084003520 con 0x7fe09c0b3600 2024-09-12T03:21:15.311 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:15.310+0000 7fe0a2201640 1 -- 172.21.15.27:0/4230517838 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe09c0b6230 msgr2=0x7fe09c0b6690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:15.311 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:15.310+0000 7fe0a2201640 1 --2- 172.21.15.27:0/4230517838 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe09c0b6230 0x7fe09c0b6690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:15.311 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:15.310+0000 7fe0a2201640 1 -- 172.21.15.27:0/4230517838 >> v1:172.21.15.27:6789/0 conn(0x7fe09c0b3600 legacy=0x7fe09c0b3a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:15.311 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:15.310+0000 7fe0a2201640 1 -- 172.21.15.27:0/4230517838 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe09c0d2d10 con 0x7fe09c15e990 2024-09-12T03:21:15.311 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:15.310+0000 7fe0a1a00640 1 --2- 172.21.15.27:0/4230517838 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe09c0b6230 0x7fe09c0b6690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:21:15.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:15.311+0000 7fe0a2201640 1 --2- 172.21.15.27:0/4230517838 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe09c15e990 0x7fe09c0b7600 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fe08c0097a0 tx=0x7fe08c02a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2cac927638d5693b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:15.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:15.311+0000 7fe0a11ff640 1 -- 172.21.15.27:0/4230517838 <== mon.1 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe08c037070 con 0x7fe09c15e990 2024-09-12T03:21:15.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:15.311+0000 7fe0a11ff640 1 -- 172.21.15.27:0/4230517838 <== mon.1 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe08c032440 con 0x7fe09c15e990 2024-09-12T03:21:15.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:15.311+0000 7fe0a11ff640 1 -- 172.21.15.27:0/4230517838 <== mon.1 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe08c0327f0 con 0x7fe09c15e990 2024-09-12T03:21:15.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:15.312+0000 7fe0a4465640 1 -- 172.21.15.27:0/4230517838 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe09c15e990 msgr2=0x7fe09c0b7600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:15.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:15.312+0000 7fe0a4465640 1 --2- 172.21.15.27:0/4230517838 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe09c15e990 0x7fe09c0b7600 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fe08c0097a0 tx=0x7fe08c02a180 comp rx=0 tx=0).stop 2024-09-12T03:21:15.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:15.312+0000 7fe0a4465640 1 -- 172.21.15.27:0/4230517838 shutdown_connections 2024-09-12T03:21:15.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:15.312+0000 7fe0a4465640 1 --2- 172.21.15.27:0/4230517838 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe09c0b6230 0x7fe09c0b6690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:15.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:15.312+0000 7fe0a4465640 1 --2- 172.21.15.27:0/4230517838 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe09c15e990 0x7fe09c0b7600 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:15.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:15.312+0000 7fe0a4465640 1 -- 172.21.15.27:0/4230517838 >> 172.21.15.27:0/4230517838 conn(0x7fe09c0d1230 msgr2=0x7fe09c0d1640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:15.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:15.312+0000 7fe0a4465640 1 -- 172.21.15.27:0/4230517838 shutdown_connections 2024-09-12T03:21:15.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:15.312+0000 7fe0a4465640 1 -- 172.21.15.27:0/4230517838 wait complete. 2024-09-12T03:21:15.316 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:15.313+0000 7fe0a4465640 1 Processor -- start 2024-09-12T03:21:15.316 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:15.314+0000 7fe0a4465640 1 -- start start 2024-09-12T03:21:15.316 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:15.314+0000 7fe0a4465640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe09c0b3600 0x7fe09c15fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:15.316 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:15.314+0000 7fe0a4465640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe09c0b6230 0x7fe09c14e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:15.316 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:15.314+0000 7fe0a4465640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe09c15e990 0x7fe09c0ab270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:15.316 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:15.314+0000 7fe0a4465640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fe09c0d35b0 con 0x7fe09c0b6230 2024-09-12T03:21:15.316 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:15.314+0000 7fe0a4465640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe09c0d32d0 con 0x7fe09c0b3600 2024-09-12T03:21:15.316 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:15.314+0000 7fe0a4465640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fe09c0b7b40 con 0x7fe09c15e990 2024-09-12T03:21:15.316 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:15.314+0000 7fe0a1a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe09c0b6230 0x7fe09c14e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:15.317 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:15.314+0000 7fe0a1a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe09c0b6230 0x7fe09c14e010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57492/0 (socket says 172.21.15.27:57492) 2024-09-12T03:21:15.317 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:15.314+0000 7fe0a1a00640 1 -- 172.21.15.27:0/3650175787 learned_addr learned my addr 172.21.15.27:0/3650175787 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:21:15.317 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:15.315+0000 7fe0a2201640 1 --2- 172.21.15.27:0/3650175787 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe09c0b3600 0x7fe09c15fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:15.317 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:15.315+0000 7fe0a2a02640 1 --2- 172.21.15.27:0/3650175787 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe09c15e990 0x7fe09c0ab270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:15.317 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:15.315+0000 7fe0a1a00640 1 -- 172.21.15.27:0/3650175787 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe09c15e990 msgr2=0x7fe09c0ab270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:15.317 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:15.315+0000 7fe0a1a00640 1 --2- 172.21.15.27:0/3650175787 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe09c15e990 0x7fe09c0ab270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:15.317 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:15.315+0000 7fe0a1a00640 1 -- 172.21.15.27:0/3650175787 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe09c0b3600 msgr2=0x7fe09c15fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:15.317 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:15.315+0000 7fe0a1a00640 1 --2- 172.21.15.27:0/3650175787 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe09c0b3600 0x7fe09c15fb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:15.317 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:15.315+0000 7fe0a1a00640 1 -- 172.21.15.27:0/3650175787 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe09c14ea90 con 0x7fe09c0b6230 2024-09-12T03:21:15.317 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:15.315+0000 7fe0a2201640 1 --2- 172.21.15.27:0/3650175787 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe09c0b3600 0x7fe09c15fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:21:15.317 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:15.315+0000 7fe0a2a02640 1 --2- 172.21.15.27:0/3650175787 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe09c15e990 0x7fe09c0ab270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:21:15.317 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:15.315+0000 7fe0a1a00640 1 --2- 172.21.15.27:0/3650175787 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe09c0b6230 0x7fe09c14e010 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7fe09000e960 tx=0x7fe09000ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:15.317 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:15.315+0000 7fe09affd640 1 -- 172.21.15.27:0/3650175787 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe090017070 con 0x7fe09c0b6230 2024-09-12T03:21:15.318 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:15.315+0000 7fe09affd640 1 -- 172.21.15.27:0/3650175787 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe09000ca00 con 0x7fe09c0b6230 2024-09-12T03:21:15.318 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:15.315+0000 7fe0a4465640 1 -- 172.21.15.27:0/3650175787 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe09c14ed70 con 0x7fe09c0b6230 2024-09-12T03:21:15.318 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:15.315+0000 7fe09affd640 1 -- 172.21.15.27:0/3650175787 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe09000cdd0 con 0x7fe09c0b6230 2024-09-12T03:21:15.318 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:15.315+0000 7fe0a4465640 1 -- 172.21.15.27:0/3650175787 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe09c14f330 con 0x7fe09c0b6230 2024-09-12T03:21:15.318 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:15.316+0000 7fe0a4465640 1 -- 172.21.15.27:0/3650175787 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe09c0b6690 con 0x7fe09c0b6230 2024-09-12T03:21:15.319 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:15.317+0000 7fe09affd640 1 -- 172.21.15.27:0/3650175787 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 18) ==== 90753+0+0 (secure 0 0 0) 0x7fe09001a070 con 0x7fe09c0b6230 2024-09-12T03:21:15.319 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:15.317+0000 7fe09affd640 1 --2- 172.21.15.27:0/3650175787 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fe08806c100 0x7fe08806e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:15.319 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:15.317+0000 7fe09affd640 1 -- 172.21.15.27:0/3650175787 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5709+0+0 (secure 0 0 0) 0x7fe09008b5d0 con 0x7fe09c0b6230 2024-09-12T03:21:15.319 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:15.317+0000 7fe0a2201640 1 --2- 172.21.15.27:0/3650175787 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fe08806c100 0x7fe08806e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:15.319 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:15.318+0000 7fe0a2201640 1 --2- 172.21.15.27:0/3650175787 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fe08806c100 0x7fe08806e5c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fe08c034000 tx=0x7fe08c02e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:15.321 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:15.321+0000 7fe09affd640 1 -- 172.21.15.27:0/3650175787 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fe090059b60 con 0x7fe09c0b6230 2024-09-12T03:21:15.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:15.527+0000 7fe0a4465640 1 -- 172.21.15.27:0/3650175787 --> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol_clone8340", "target": ["mon-mgr", ""]}) -- 0x7fe09c0d1820 con 0x7fe08806c100 2024-09-12T03:21:15.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:15.531+0000 7fe09affd640 1 -- 172.21.15.27:0/3650175787 <== mgr.4113 v2:172.21.15.27:6836/1226920526 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+72 (secure 0 0 0) 0x7fe09c0d1820 con 0x7fe08806c100 2024-09-12T03:21:15.531 INFO:teuthology.orchestra.run.smithi027.stdout:/volumes/_nogroup/subvol_clone8340/fe9d5d1c-b87b-434b-8988-e68eb0a4b017 2024-09-12T03:21:15.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:15.533+0000 7fe0a4465640 1 -- 172.21.15.27:0/3650175787 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fe08806c100 msgr2=0x7fe08806e5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:15.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:15.533+0000 7fe0a4465640 1 --2- 172.21.15.27:0/3650175787 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fe08806c100 0x7fe08806e5c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fe08c034000 tx=0x7fe08c02e000 comp rx=0 tx=0).stop 2024-09-12T03:21:15.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:15.533+0000 7fe0a4465640 1 -- 172.21.15.27:0/3650175787 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe09c0b6230 msgr2=0x7fe09c14e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:15.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:15.533+0000 7fe0a4465640 1 --2- 172.21.15.27:0/3650175787 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe09c0b6230 0x7fe09c14e010 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7fe09000e960 tx=0x7fe09000ee20 comp rx=0 tx=0).stop 2024-09-12T03:21:15.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:15.534+0000 7fe0a4465640 1 -- 172.21.15.27:0/3650175787 shutdown_connections 2024-09-12T03:21:15.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:15.534+0000 7fe0a4465640 1 --2- 172.21.15.27:0/3650175787 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe09c15e990 0x7fe09c0ab270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:15.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:15.534+0000 7fe0a4465640 1 --2- 172.21.15.27:0/3650175787 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fe08806c100 0x7fe08806e5c0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:15.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:15.534+0000 7fe0a4465640 1 --2- 172.21.15.27:0/3650175787 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe09c0b6230 0x7fe09c14e010 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:15.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:15.534+0000 7fe0a4465640 1 --2- 172.21.15.27:0/3650175787 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe09c0b3600 0x7fe09c15fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:15.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:15.534+0000 7fe0a4465640 1 -- 172.21.15.27:0/3650175787 >> 172.21.15.27:0/3650175787 conn(0x7fe09c0d1230 msgr2=0x7fe09c0b5910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:15.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:15.534+0000 7fe0a4465640 1 -- 172.21.15.27:0/3650175787 shutdown_connections 2024-09-12T03:21:15.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:15.534+0000 7fe0a4465640 1 -- 172.21.15.27:0/3650175787 wait complete. 2024-09-12T03:21:15.544 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:21:15.545 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.rentries volumes/_nogroup/subvol84490/a842cae4-61b6-4f73-a96f-03c1581e2365/.snap/subvol_snap39590) 2024-09-12T03:21:15.562 INFO:teuthology.orchestra.run.smithi027.stdout:9 2024-09-12T03:21:15.562 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:21:15.562 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.rentries volumes/_nogroup/subvol_clone8340/fe9d5d1c-b87b-434b-8988-e68eb0a4b017) 2024-09-12T03:21:15.622 INFO:teuthology.orchestra.run.smithi027.stdout:9 2024-09-12T03:21:15.622 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume info cephfs subvol_clone8340 2024-09-12T03:21:15.767 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:15.767+0000 7efc6f0a5640 1 Processor -- start 2024-09-12T03:21:15.768 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:15.767+0000 7efc6f0a5640 1 -- start start 2024-09-12T03:21:15.769 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:15.768+0000 7efc6f0a5640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7efc68070d30 0x7efc6806cc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:15.769 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:15.768+0000 7efc6f0a5640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7efc680d0b50 0x7efc68168300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:15.769 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:15.768+0000 7efc6f0a5640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efc680cf9f0 con 0x7efc6806e960 2024-09-12T03:21:15.769 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:15.768+0000 7efc6f0a5640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7efc680d1a40 con 0x7efc68070d30 2024-09-12T03:21:15.769 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:15.768+0000 7efc6f0a5640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7efc680cfc90 con 0x7efc680d0b50 2024-09-12T03:21:15.769 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:15.768+0000 7efc6ce41640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7efc68070d30 0x7efc6806cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:15.769 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:15.768+0000 7efc6ce41640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7efc68070d30 0x7efc6806cc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57496/0 (socket says 172.21.15.27:57496) 2024-09-12T03:21:15.769 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:15.768+0000 7efc6ce41640 1 -- 172.21.15.27:0/273002653 learned_addr learned my addr 172.21.15.27:0/273002653 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:21:15.769 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:15.769+0000 7efc6d642640 1 --2- 172.21.15.27:0/273002653 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7efc680d0b50 0x7efc68168300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:15.769 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:15.769+0000 7efc6ce41640 1 -- 172.21.15.27:0/273002653 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7efc680d0b50 msgr2=0x7efc68168300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:15.770 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:15.769+0000 7efc6ce41640 1 --2- 172.21.15.27:0/273002653 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7efc680d0b50 0x7efc68168300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:15.770 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:15.769+0000 7efc6ce41640 1 -- 172.21.15.27:0/273002653 >> v1:172.21.15.45:6789/0 conn(0x7efc6806e960 legacy=0x7efc680d0440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:15.770 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:15.769+0000 7efc6ce41640 1 -- 172.21.15.27:0/273002653 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc680cff70 con 0x7efc68070d30 2024-09-12T03:21:15.770 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:15.769+0000 7efc6d642640 1 --2- 172.21.15.27:0/273002653 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7efc680d0b50 0x7efc68168300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:21:15.771 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:15.769+0000 7efc6ce41640 1 --2- 172.21.15.27:0/273002653 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efc68070d30 0x7efc6806cc40 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7efc68073770 tx=0x7efc5802a0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e17d3fc445d8af3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:15.771 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:15.769+0000 7efc677fe640 1 -- 172.21.15.27:0/273002653 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc58037070 con 0x7efc68070d30 2024-09-12T03:21:15.771 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:15.770+0000 7efc677fe640 1 -- 172.21.15.27:0/273002653 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc5802adc0 con 0x7efc68070d30 2024-09-12T03:21:15.771 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:15.770+0000 7efc677fe640 1 -- 172.21.15.27:0/273002653 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc58032660 con 0x7efc68070d30 2024-09-12T03:21:15.772 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:15.770+0000 7efc6f0a5640 1 -- 172.21.15.27:0/273002653 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efc68070d30 msgr2=0x7efc6806cc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:15.772 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:15.770+0000 7efc6f0a5640 1 --2- 172.21.15.27:0/273002653 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efc68070d30 0x7efc6806cc40 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7efc68073770 tx=0x7efc5802a0b0 comp rx=0 tx=0).stop 2024-09-12T03:21:15.772 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:15.770+0000 7efc6f0a5640 1 -- 172.21.15.27:0/273002653 shutdown_connections 2024-09-12T03:21:15.772 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:15.770+0000 7efc6f0a5640 1 --2- 172.21.15.27:0/273002653 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7efc680d0b50 0x7efc68168300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:15.772 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:15.770+0000 7efc6f0a5640 1 --2- 172.21.15.27:0/273002653 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efc68070d30 0x7efc6806cc40 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:15.772 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:15.770+0000 7efc6f0a5640 1 -- 172.21.15.27:0/273002653 >> 172.21.15.27:0/273002653 conn(0x7efc680b7c10 msgr2=0x7efc6809ba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:15.772 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:15.770+0000 7efc6f0a5640 1 -- 172.21.15.27:0/273002653 shutdown_connections 2024-09-12T03:21:15.772 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:15.770+0000 7efc6f0a5640 1 -- 172.21.15.27:0/273002653 wait complete. 2024-09-12T03:21:15.772 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:15.771+0000 7efc6f0a5640 1 Processor -- start 2024-09-12T03:21:15.772 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:15.771+0000 7efc6f0a5640 1 -- start start 2024-09-12T03:21:15.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:15.772+0000 7efc6f0a5640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7efc6806e960 0x7efc680b2b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:15.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:15.772+0000 7efc6f0a5640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efc68070d30 0x7efc68153ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:15.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:15.772+0000 7efc6f0a5640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efc680d0b50 0x7efc68153740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:15.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:15.772+0000 7efc6f0a5640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7efc680d31c0 con 0x7efc68070d30 2024-09-12T03:21:15.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:15.772+0000 7efc6f0a5640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7efc6806d180 con 0x7efc680d0b50 2024-09-12T03:21:15.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:15.772+0000 7efc6f0a5640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7efc680d3340 con 0x7efc6806e960 2024-09-12T03:21:15.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:15.772+0000 7efc67fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efc68070d30 0x7efc68153ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:15.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:15.772+0000 7efc67fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efc68070d30 0x7efc68153ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57502/0 (socket says 172.21.15.27:57502) 2024-09-12T03:21:15.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:15.772+0000 7efc67fff640 1 -- 172.21.15.27:0/1722576618 learned_addr learned my addr 172.21.15.27:0/1722576618 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:21:15.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:15.772+0000 7efc6d642640 1 --2- 172.21.15.27:0/1722576618 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efc680d0b50 0x7efc68153740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:15.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:15.772+0000 7efc6ce41640 1 --2- 172.21.15.27:0/1722576618 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7efc6806e960 0x7efc680b2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:15.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:15.773+0000 7efc67fff640 1 -- 172.21.15.27:0/1722576618 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7efc6806e960 msgr2=0x7efc680b2b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:15.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:15.773+0000 7efc67fff640 1 --2- 172.21.15.27:0/1722576618 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7efc6806e960 0x7efc680b2b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:15.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:15.773+0000 7efc67fff640 1 -- 172.21.15.27:0/1722576618 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efc680d0b50 msgr2=0x7efc68153740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:15.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:15.773+0000 7efc67fff640 1 --2- 172.21.15.27:0/1722576618 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efc680d0b50 0x7efc68153740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:15.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:15.773+0000 7efc67fff640 1 -- 172.21.15.27:0/1722576618 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc680ce5a0 con 0x7efc68070d30 2024-09-12T03:21:15.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:15.773+0000 7efc6ce41640 1 --2- 172.21.15.27:0/1722576618 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7efc6806e960 0x7efc680b2b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:21:15.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:15.773+0000 7efc67fff640 1 --2- 172.21.15.27:0/1722576618 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efc68070d30 0x7efc68153ff0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7efc5000c7f0 tx=0x7efc5000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:15.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:15.773+0000 7efc657fa640 1 -- 172.21.15.27:0/1722576618 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc50016020 con 0x7efc68070d30 2024-09-12T03:21:15.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:15.773+0000 7efc657fa640 1 -- 172.21.15.27:0/1722576618 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc5000aa50 con 0x7efc68070d30 2024-09-12T03:21:15.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:15.773+0000 7efc6f0a5640 1 -- 172.21.15.27:0/1722576618 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc68158780 con 0x7efc68070d30 2024-09-12T03:21:15.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:15.773+0000 7efc657fa640 1 -- 172.21.15.27:0/1722576618 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc5000ae40 con 0x7efc68070d30 2024-09-12T03:21:15.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:15.773+0000 7efc6f0a5640 1 -- 172.21.15.27:0/1722576618 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc68158d40 con 0x7efc68070d30 2024-09-12T03:21:15.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:15.774+0000 7efc6f0a5640 1 -- 172.21.15.27:0/1722576618 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc6806cc40 con 0x7efc68070d30 2024-09-12T03:21:15.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:15.774+0000 7efc657fa640 1 -- 172.21.15.27:0/1722576618 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 18) ==== 90753+0+0 (secure 0 0 0) 0x7efc50019020 con 0x7efc68070d30 2024-09-12T03:21:15.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:15.775+0000 7efc657fa640 1 --2- 172.21.15.27:0/1722576618 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7efc4806c100 0x7efc4806e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:15.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:15.775+0000 7efc657fa640 1 -- 172.21.15.27:0/1722576618 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5709+0+0 (secure 0 0 0) 0x7efc5008b250 con 0x7efc68070d30 2024-09-12T03:21:15.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:15.775+0000 7efc6ce41640 1 --2- 172.21.15.27:0/1722576618 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7efc4806c100 0x7efc4806e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:15.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:15.775+0000 7efc6ce41640 1 --2- 172.21.15.27:0/1722576618 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7efc4806c100 0x7efc4806e5c0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7efc58034000 tx=0x7efc5802e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:15.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:15.779+0000 7efc657fa640 1 -- 172.21.15.27:0/1722576618 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7efc50020030 con 0x7efc68070d30 2024-09-12T03:21:15.994 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:15.994+0000 7efc6f0a5640 1 -- 172.21.15.27:0/1722576618 --> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] -- mgr_command(tid 0: {"prefix": "fs subvolume info", "vol_name": "cephfs", "sub_name": "subvol_clone8340", "target": ["mon-mgr", ""]}) -- 0x7efc680d1230 con 0x7efc4806c100 2024-09-12T03:21:16.000 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:16.000+0000 7efc657fa640 1 -- 172.21.15.27:0/1722576618 <== mgr.4113 v2:172.21.15.27:6836/1226920526 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+721 (secure 0 0 0) 0x7efc680d1230 con 0x7efc4806c100 2024-09-12T03:21:16.000 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-12T03:21:16.000 INFO:teuthology.orchestra.run.smithi027.stdout: "atime": "2024-09-12 03:21:08", 2024-09-12T03:21:16.000 INFO:teuthology.orchestra.run.smithi027.stdout: "bytes_pcent": "80.00", 2024-09-12T03:21:16.000 INFO:teuthology.orchestra.run.smithi027.stdout: "bytes_quota": 10485760, 2024-09-12T03:21:16.000 INFO:teuthology.orchestra.run.smithi027.stdout: "bytes_used": 8388608, 2024-09-12T03:21:16.000 INFO:teuthology.orchestra.run.smithi027.stdout: "created_at": "2024-09-12 03:21:13", 2024-09-12T03:21:16.001 INFO:teuthology.orchestra.run.smithi027.stdout: "ctime": "2024-09-12 03:21:13", 2024-09-12T03:21:16.001 INFO:teuthology.orchestra.run.smithi027.stdout: "data_pool": "cephfs_data", 2024-09-12T03:21:16.001 INFO:teuthology.orchestra.run.smithi027.stdout: "features": [ 2024-09-12T03:21:16.001 INFO:teuthology.orchestra.run.smithi027.stdout: "snapshot-clone", 2024-09-12T03:21:16.001 INFO:teuthology.orchestra.run.smithi027.stdout: "snapshot-autoprotect", 2024-09-12T03:21:16.001 INFO:teuthology.orchestra.run.smithi027.stdout: "snapshot-retention" 2024-09-12T03:21:16.001 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-12T03:21:16.001 INFO:teuthology.orchestra.run.smithi027.stdout: "flavor": 2, 2024-09-12T03:21:16.001 INFO:teuthology.orchestra.run.smithi027.stdout: "gid": 1000, 2024-09-12T03:21:16.001 INFO:teuthology.orchestra.run.smithi027.stdout: "mode": 16895, 2024-09-12T03:21:16.001 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_addrs": [ 2024-09-12T03:21:16.001 INFO:teuthology.orchestra.run.smithi027.stdout: "172.21.15.27:6789", 2024-09-12T03:21:16.001 INFO:teuthology.orchestra.run.smithi027.stdout: "172.21.15.45:6789", 2024-09-12T03:21:16.001 INFO:teuthology.orchestra.run.smithi027.stdout: "172.21.15.45:6790" 2024-09-12T03:21:16.002 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-12T03:21:16.002 INFO:teuthology.orchestra.run.smithi027.stdout: "mtime": "2024-09-12 03:21:12", 2024-09-12T03:21:16.002 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/volumes/_nogroup/subvol_clone8340/fe9d5d1c-b87b-434b-8988-e68eb0a4b017", 2024-09-12T03:21:16.002 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_namespace": "", 2024-09-12T03:21:16.002 INFO:teuthology.orchestra.run.smithi027.stdout: "state": "complete", 2024-09-12T03:21:16.002 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "clone", 2024-09-12T03:21:16.002 INFO:teuthology.orchestra.run.smithi027.stdout: "uid": 1000 2024-09-12T03:21:16.002 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-12T03:21:16.003 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:16.002+0000 7efc6f0a5640 1 -- 172.21.15.27:0/1722576618 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7efc4806c100 msgr2=0x7efc4806e5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:16.003 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:16.002+0000 7efc6f0a5640 1 --2- 172.21.15.27:0/1722576618 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7efc4806c100 0x7efc4806e5c0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7efc58034000 tx=0x7efc5802e000 comp rx=0 tx=0).stop 2024-09-12T03:21:16.003 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:16.002+0000 7efc6f0a5640 1 -- 172.21.15.27:0/1722576618 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efc68070d30 msgr2=0x7efc68153ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:16.003 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:16.002+0000 7efc6f0a5640 1 --2- 172.21.15.27:0/1722576618 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efc68070d30 0x7efc68153ff0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7efc5000c7f0 tx=0x7efc5000ccb0 comp rx=0 tx=0).stop 2024-09-12T03:21:16.003 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:16.002+0000 7efc6f0a5640 1 -- 172.21.15.27:0/1722576618 shutdown_connections 2024-09-12T03:21:16.003 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:16.002+0000 7efc6f0a5640 1 --2- 172.21.15.27:0/1722576618 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efc680d0b50 0x7efc68153740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:16.003 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:16.002+0000 7efc6f0a5640 1 --2- 172.21.15.27:0/1722576618 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7efc4806c100 0x7efc4806e5c0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:16.004 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:16.002+0000 7efc6f0a5640 1 --2- 172.21.15.27:0/1722576618 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efc68070d30 0x7efc68153ff0 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:16.004 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:16.002+0000 7efc6f0a5640 1 --2- 172.21.15.27:0/1722576618 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7efc6806e960 0x7efc680b2b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:16.004 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:16.002+0000 7efc6f0a5640 1 -- 172.21.15.27:0/1722576618 >> 172.21.15.27:0/1722576618 conn(0x7efc680b7c10 msgr2=0x7efc68155350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:16.004 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:16.002+0000 7efc6f0a5640 1 -- 172.21.15.27:0/1722576618 shutdown_connections 2024-09-12T03:21:16.004 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:16.002+0000 7efc6f0a5640 1 -- 172.21.15.27:0/1722576618 wait complete. 2024-09-12T03:21:16.013 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:21:16.013 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.quota.max_bytes volumes/_nogroup/subvol84490/a842cae4-61b6-4f73-a96f-03c1581e2365/.snap/subvol_snap39590) 2024-09-12T03:21:16.030 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T03:21:16.030 INFO:teuthology.orchestra.run.smithi027.stderr:volumes/_nogroup/subvol84490/a842cae4-61b6-4f73-a96f-03c1581e2365/.snap/subvol_snap39590: ceph.quota.max_bytes: No such attribute 2024-09-12T03:21:16.030 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:21:16.031 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.layout.pool volumes/_nogroup/subvol84490/a842cae4-61b6-4f73-a96f-03c1581e2365/.snap/subvol_snap39590) 2024-09-12T03:21:16.087 INFO:teuthology.orchestra.run.smithi027.stdout:cephfs_data 2024-09-12T03:21:16.087 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:21:16.087 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.layout.pool_namespace volumes/_nogroup/subvol84490/a842cae4-61b6-4f73-a96f-03c1581e2365/.snap/subvol_snap39590) 2024-09-12T03:21:16.143 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:21:16.144 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec find volumes/_nogroup/subvol84490/a842cae4-61b6-4f73-a96f-03c1581e2365/.snap/subvol_snap39590) 2024-09-12T03:21:16.201 INFO:teuthology.orchestra.run.smithi027.stdout:volumes/_nogroup/subvol84490/a842cae4-61b6-4f73-a96f-03c1581e2365/.snap/subvol_snap39590 2024-09-12T03:21:16.201 INFO:teuthology.orchestra.run.smithi027.stdout:volumes/_nogroup/subvol84490/a842cae4-61b6-4f73-a96f-03c1581e2365/.snap/subvol_snap39590/subvolume_file.2 2024-09-12T03:21:16.201 INFO:teuthology.orchestra.run.smithi027.stdout:volumes/_nogroup/subvol84490/a842cae4-61b6-4f73-a96f-03c1581e2365/.snap/subvol_snap39590/subvolume_file.5 2024-09-12T03:21:16.201 INFO:teuthology.orchestra.run.smithi027.stdout:volumes/_nogroup/subvol84490/a842cae4-61b6-4f73-a96f-03c1581e2365/.snap/subvol_snap39590/subvolume_file.4 2024-09-12T03:21:16.201 INFO:teuthology.orchestra.run.smithi027.stdout:volumes/_nogroup/subvol84490/a842cae4-61b6-4f73-a96f-03c1581e2365/.snap/subvol_snap39590/subvolume_file.7 2024-09-12T03:21:16.201 INFO:teuthology.orchestra.run.smithi027.stdout:volumes/_nogroup/subvol84490/a842cae4-61b6-4f73-a96f-03c1581e2365/.snap/subvol_snap39590/subvolume_file.6 2024-09-12T03:21:16.202 INFO:teuthology.orchestra.run.smithi027.stdout:volumes/_nogroup/subvol84490/a842cae4-61b6-4f73-a96f-03c1581e2365/.snap/subvol_snap39590/subvolume_file.0 2024-09-12T03:21:16.202 INFO:teuthology.orchestra.run.smithi027.stdout:volumes/_nogroup/subvol84490/a842cae4-61b6-4f73-a96f-03c1581e2365/.snap/subvol_snap39590/subvolume_file.3 2024-09-12T03:21:16.202 INFO:teuthology.orchestra.run.smithi027.stdout:volumes/_nogroup/subvol84490/a842cae4-61b6-4f73-a96f-03c1581e2365/.snap/subvol_snap39590/subvolume_file.1 2024-09-12T03:21:16.202 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:21:16.202 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol84490/a842cae4-61b6-4f73-a96f-03c1581e2365/.snap/subvol_snap39590) 2024-09-12T03:21:16.258 INFO:teuthology.orchestra.run.smithi027.stdout:41ff 2024-09-12T03:21:16.258 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:21:16.258 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol_clone8340/fe9d5d1c-b87b-434b-8988-e68eb0a4b017/) 2024-09-12T03:21:16.314 INFO:teuthology.orchestra.run.smithi027.stdout:41ff 2024-09-12T03:21:16.314 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:21:16.314 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol84490/a842cae4-61b6-4f73-a96f-03c1581e2365/.snap/subvol_snap39590) 2024-09-12T03:21:16.369 INFO:teuthology.orchestra.run.smithi027.stdout:1000 2024-09-12T03:21:16.370 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:21:16.370 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol_clone8340/fe9d5d1c-b87b-434b-8988-e68eb0a4b017/) 2024-09-12T03:21:16.425 INFO:teuthology.orchestra.run.smithi027.stdout:1000 2024-09-12T03:21:16.426 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:21:16.426 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol84490/a842cae4-61b6-4f73-a96f-03c1581e2365/.snap/subvol_snap39590) 2024-09-12T03:21:16.481 INFO:teuthology.orchestra.run.smithi027.stdout:1000 2024-09-12T03:21:16.482 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:21:16.482 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol_clone8340/fe9d5d1c-b87b-434b-8988-e68eb0a4b017/) 2024-09-12T03:21:16.538 INFO:teuthology.orchestra.run.smithi027.stdout:1000 2024-09-12T03:21:16.538 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:21:16.538 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol84490/a842cae4-61b6-4f73-a96f-03c1581e2365/.snap/subvol_snap39590) 2024-09-12T03:21:16.593 INFO:teuthology.orchestra.run.smithi027.stdout:1726111272 2024-09-12T03:21:16.594 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:21:16.594 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol_clone8340/fe9d5d1c-b87b-434b-8988-e68eb0a4b017/) 2024-09-12T03:21:16.650 INFO:teuthology.orchestra.run.smithi027.stdout:1726111272 2024-09-12T03:21:16.650 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:21:16.650 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol84490/a842cae4-61b6-4f73-a96f-03c1581e2365/.snap/subvol_snap39590/subvolume_file.2) 2024-09-12T03:21:16.707 INFO:teuthology.orchestra.run.smithi027.stdout:81a4 2024-09-12T03:21:16.707 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:21:16.707 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol_clone8340/fe9d5d1c-b87b-434b-8988-e68eb0a4b017/subvolume_file.2) 2024-09-12T03:21:16.771 INFO:teuthology.orchestra.run.smithi027.stdout:81a4 2024-09-12T03:21:16.772 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:21:16.772 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol84490/a842cae4-61b6-4f73-a96f-03c1581e2365/.snap/subvol_snap39590/subvolume_file.2) 2024-09-12T03:21:16.829 INFO:teuthology.orchestra.run.smithi027.stdout:1000 2024-09-12T03:21:16.829 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:21:16.829 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol_clone8340/fe9d5d1c-b87b-434b-8988-e68eb0a4b017/subvolume_file.2) 2024-09-12T03:21:16.885 INFO:teuthology.orchestra.run.smithi027.stdout:1000 2024-09-12T03:21:16.886 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:21:16.886 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol84490/a842cae4-61b6-4f73-a96f-03c1581e2365/.snap/subvol_snap39590/subvolume_file.2) 2024-09-12T03:21:16.941 INFO:teuthology.orchestra.run.smithi027.stdout:1333 2024-09-12T03:21:16.942 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:21:16.942 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol_clone8340/fe9d5d1c-b87b-434b-8988-e68eb0a4b017/subvolume_file.2) 2024-09-12T03:21:16.998 INFO:teuthology.orchestra.run.smithi027.stdout:1333 2024-09-12T03:21:16.998 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:21:16.998 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol84490/a842cae4-61b6-4f73-a96f-03c1581e2365/.snap/subvol_snap39590/subvolume_file.2) 2024-09-12T03:21:17.054 INFO:teuthology.orchestra.run.smithi027.stdout:1726111271 2024-09-12T03:21:17.054 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:21:17.055 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol_clone8340/fe9d5d1c-b87b-434b-8988-e68eb0a4b017/subvolume_file.2) 2024-09-12T03:21:17.111 INFO:teuthology.orchestra.run.smithi027.stdout:1726111271 2024-09-12T03:21:17.112 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:21:17.112 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol84490/a842cae4-61b6-4f73-a96f-03c1581e2365/.snap/subvol_snap39590/subvolume_file.5) 2024-09-12T03:21:17.168 INFO:teuthology.orchestra.run.smithi027.stdout:81a4 2024-09-12T03:21:17.169 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:21:17.169 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol_clone8340/fe9d5d1c-b87b-434b-8988-e68eb0a4b017/subvolume_file.5) 2024-09-12T03:21:17.228 INFO:teuthology.orchestra.run.smithi027.stdout:81a4 2024-09-12T03:21:17.228 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:21:17.229 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol84490/a842cae4-61b6-4f73-a96f-03c1581e2365/.snap/subvol_snap39590/subvolume_file.5) 2024-09-12T03:21:17.285 INFO:teuthology.orchestra.run.smithi027.stdout:1000 2024-09-12T03:21:17.285 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:21:17.285 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol_clone8340/fe9d5d1c-b87b-434b-8988-e68eb0a4b017/subvolume_file.5) 2024-09-12T03:21:17.341 INFO:teuthology.orchestra.run.smithi027.stdout:1000 2024-09-12T03:21:17.342 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:21:17.342 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol84490/a842cae4-61b6-4f73-a96f-03c1581e2365/.snap/subvol_snap39590/subvolume_file.5) 2024-09-12T03:21:17.397 INFO:teuthology.orchestra.run.smithi027.stdout:1333 2024-09-12T03:21:17.398 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:21:17.398 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol_clone8340/fe9d5d1c-b87b-434b-8988-e68eb0a4b017/subvolume_file.5) 2024-09-12T03:21:17.453 INFO:teuthology.orchestra.run.smithi027.stdout:1333 2024-09-12T03:21:17.458 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:21:17.458 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol84490/a842cae4-61b6-4f73-a96f-03c1581e2365/.snap/subvol_snap39590/subvolume_file.5) 2024-09-12T03:21:17.509 INFO:teuthology.orchestra.run.smithi027.stdout:1726111272 2024-09-12T03:21:17.510 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:21:17.510 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol_clone8340/fe9d5d1c-b87b-434b-8988-e68eb0a4b017/subvolume_file.5) 2024-09-12T03:21:17.566 INFO:teuthology.orchestra.run.smithi027.stdout:1726111272 2024-09-12T03:21:17.566 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:21:17.566 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol84490/a842cae4-61b6-4f73-a96f-03c1581e2365/.snap/subvol_snap39590/subvolume_file.4) 2024-09-12T03:21:17.623 INFO:teuthology.orchestra.run.smithi027.stdout:81a4 2024-09-12T03:21:17.624 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:21:17.624 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol_clone8340/fe9d5d1c-b87b-434b-8988-e68eb0a4b017/subvolume_file.4) 2024-09-12T03:21:17.682 INFO:teuthology.orchestra.run.smithi027.stdout:81a4 2024-09-12T03:21:17.683 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:21:17.683 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol84490/a842cae4-61b6-4f73-a96f-03c1581e2365/.snap/subvol_snap39590/subvolume_file.4) 2024-09-12T03:21:17.738 INFO:teuthology.orchestra.run.smithi027.stdout:1000 2024-09-12T03:21:17.739 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:21:17.739 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol_clone8340/fe9d5d1c-b87b-434b-8988-e68eb0a4b017/subvolume_file.4) 2024-09-12T03:21:17.795 INFO:teuthology.orchestra.run.smithi027.stdout:1000 2024-09-12T03:21:17.795 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:21:17.795 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol84490/a842cae4-61b6-4f73-a96f-03c1581e2365/.snap/subvol_snap39590/subvolume_file.4) 2024-09-12T03:21:17.851 INFO:teuthology.orchestra.run.smithi027.stdout:1333 2024-09-12T03:21:17.851 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:21:17.851 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol_clone8340/fe9d5d1c-b87b-434b-8988-e68eb0a4b017/subvolume_file.4) 2024-09-12T03:21:17.907 INFO:teuthology.orchestra.run.smithi027.stdout:1333 2024-09-12T03:21:17.907 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:21:17.907 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol84490/a842cae4-61b6-4f73-a96f-03c1581e2365/.snap/subvol_snap39590/subvolume_file.4) 2024-09-12T03:21:17.963 INFO:teuthology.orchestra.run.smithi027.stdout:1726111271 2024-09-12T03:21:17.963 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:21:17.963 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol_clone8340/fe9d5d1c-b87b-434b-8988-e68eb0a4b017/subvolume_file.4) 2024-09-12T03:21:18.019 INFO:teuthology.orchestra.run.smithi027.stdout:1726111271 2024-09-12T03:21:18.019 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:21:18.019 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol84490/a842cae4-61b6-4f73-a96f-03c1581e2365/.snap/subvol_snap39590/subvolume_file.7) 2024-09-12T03:21:18.076 INFO:teuthology.orchestra.run.smithi027.stdout:81a4 2024-09-12T03:21:18.076 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:21:18.076 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol_clone8340/fe9d5d1c-b87b-434b-8988-e68eb0a4b017/subvolume_file.7) 2024-09-12T03:21:18.135 INFO:teuthology.orchestra.run.smithi027.stdout:81a4 2024-09-12T03:21:18.136 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:21:18.136 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol84490/a842cae4-61b6-4f73-a96f-03c1581e2365/.snap/subvol_snap39590/subvolume_file.7) 2024-09-12T03:21:18.191 INFO:teuthology.orchestra.run.smithi027.stdout:1000 2024-09-12T03:21:18.192 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:21:18.192 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol_clone8340/fe9d5d1c-b87b-434b-8988-e68eb0a4b017/subvolume_file.7) 2024-09-12T03:21:18.247 INFO:teuthology.orchestra.run.smithi027.stdout:1000 2024-09-12T03:21:18.248 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:21:18.248 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol84490/a842cae4-61b6-4f73-a96f-03c1581e2365/.snap/subvol_snap39590/subvolume_file.7) 2024-09-12T03:21:18.304 INFO:teuthology.orchestra.run.smithi027.stdout:1333 2024-09-12T03:21:18.304 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:21:18.304 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol_clone8340/fe9d5d1c-b87b-434b-8988-e68eb0a4b017/subvolume_file.7) 2024-09-12T03:21:18.361 INFO:teuthology.orchestra.run.smithi027.stdout:1333 2024-09-12T03:21:18.361 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:21:18.361 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol84490/a842cae4-61b6-4f73-a96f-03c1581e2365/.snap/subvol_snap39590/subvolume_file.7) 2024-09-12T03:21:18.417 INFO:teuthology.orchestra.run.smithi027.stdout:1726111272 2024-09-12T03:21:18.417 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:21:18.417 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol_clone8340/fe9d5d1c-b87b-434b-8988-e68eb0a4b017/subvolume_file.7) 2024-09-12T03:21:18.478 INFO:teuthology.orchestra.run.smithi027.stdout:1726111272 2024-09-12T03:21:18.479 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:21:18.479 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol84490/a842cae4-61b6-4f73-a96f-03c1581e2365/.snap/subvol_snap39590/subvolume_file.6) 2024-09-12T03:21:18.535 INFO:teuthology.orchestra.run.smithi027.stdout:81a4 2024-09-12T03:21:18.536 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:21:18.536 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol_clone8340/fe9d5d1c-b87b-434b-8988-e68eb0a4b017/subvolume_file.6) 2024-09-12T03:21:18.594 INFO:teuthology.orchestra.run.smithi027.stdout:81a4 2024-09-12T03:21:18.595 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:21:18.595 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol84490/a842cae4-61b6-4f73-a96f-03c1581e2365/.snap/subvol_snap39590/subvolume_file.6) 2024-09-12T03:21:18.651 INFO:teuthology.orchestra.run.smithi027.stdout:1000 2024-09-12T03:21:18.651 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:21:18.651 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol_clone8340/fe9d5d1c-b87b-434b-8988-e68eb0a4b017/subvolume_file.6) 2024-09-12T03:21:18.706 INFO:teuthology.orchestra.run.smithi027.stdout:1000 2024-09-12T03:21:18.707 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:21:18.707 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol84490/a842cae4-61b6-4f73-a96f-03c1581e2365/.snap/subvol_snap39590/subvolume_file.6) 2024-09-12T03:21:18.762 INFO:teuthology.orchestra.run.smithi027.stdout:1333 2024-09-12T03:21:18.763 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:21:18.763 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol_clone8340/fe9d5d1c-b87b-434b-8988-e68eb0a4b017/subvolume_file.6) 2024-09-12T03:21:18.818 INFO:teuthology.orchestra.run.smithi027.stdout:1333 2024-09-12T03:21:18.819 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:21:18.819 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol84490/a842cae4-61b6-4f73-a96f-03c1581e2365/.snap/subvol_snap39590/subvolume_file.6) 2024-09-12T03:21:18.876 INFO:teuthology.orchestra.run.smithi027.stdout:1726111272 2024-09-12T03:21:18.876 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:21:18.876 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol_clone8340/fe9d5d1c-b87b-434b-8988-e68eb0a4b017/subvolume_file.6) 2024-09-12T03:21:18.932 INFO:teuthology.orchestra.run.smithi027.stdout:1726111272 2024-09-12T03:21:18.932 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:21:18.933 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol84490/a842cae4-61b6-4f73-a96f-03c1581e2365/.snap/subvol_snap39590/subvolume_file.0) 2024-09-12T03:21:18.989 INFO:teuthology.orchestra.run.smithi027.stdout:81a4 2024-09-12T03:21:18.989 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:21:18.989 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol_clone8340/fe9d5d1c-b87b-434b-8988-e68eb0a4b017/subvolume_file.0) 2024-09-12T03:21:19.049 INFO:teuthology.orchestra.run.smithi027.stdout:81a4 2024-09-12T03:21:19.049 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:21:19.049 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol84490/a842cae4-61b6-4f73-a96f-03c1581e2365/.snap/subvol_snap39590/subvolume_file.0) 2024-09-12T03:21:19.105 INFO:teuthology.orchestra.run.smithi027.stdout:1000 2024-09-12T03:21:19.105 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:21:19.106 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol_clone8340/fe9d5d1c-b87b-434b-8988-e68eb0a4b017/subvolume_file.0) 2024-09-12T03:21:19.162 INFO:teuthology.orchestra.run.smithi027.stdout:1000 2024-09-12T03:21:19.162 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:21:19.162 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol84490/a842cae4-61b6-4f73-a96f-03c1581e2365/.snap/subvol_snap39590/subvolume_file.0) 2024-09-12T03:21:19.219 INFO:teuthology.orchestra.run.smithi027.stdout:1333 2024-09-12T03:21:19.219 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:21:19.219 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol_clone8340/fe9d5d1c-b87b-434b-8988-e68eb0a4b017/subvolume_file.0) 2024-09-12T03:21:19.275 INFO:teuthology.orchestra.run.smithi027.stdout:1333 2024-09-12T03:21:19.275 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:21:19.275 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol84490/a842cae4-61b6-4f73-a96f-03c1581e2365/.snap/subvol_snap39590/subvolume_file.0) 2024-09-12T03:21:19.331 INFO:teuthology.orchestra.run.smithi027.stdout:1726111271 2024-09-12T03:21:19.331 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:21:19.332 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol_clone8340/fe9d5d1c-b87b-434b-8988-e68eb0a4b017/subvolume_file.0) 2024-09-12T03:21:19.388 INFO:teuthology.orchestra.run.smithi027.stdout:1726111271 2024-09-12T03:21:19.389 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:21:19.389 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol84490/a842cae4-61b6-4f73-a96f-03c1581e2365/.snap/subvol_snap39590/subvolume_file.3) 2024-09-12T03:21:19.445 INFO:teuthology.orchestra.run.smithi027.stdout:81a4 2024-09-12T03:21:19.446 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:21:19.446 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol_clone8340/fe9d5d1c-b87b-434b-8988-e68eb0a4b017/subvolume_file.3) 2024-09-12T03:21:19.504 INFO:teuthology.orchestra.run.smithi027.stdout:81a4 2024-09-12T03:21:19.504 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:21:19.504 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol84490/a842cae4-61b6-4f73-a96f-03c1581e2365/.snap/subvol_snap39590/subvolume_file.3) 2024-09-12T03:21:19.561 INFO:teuthology.orchestra.run.smithi027.stdout:1000 2024-09-12T03:21:19.561 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:21:19.561 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol_clone8340/fe9d5d1c-b87b-434b-8988-e68eb0a4b017/subvolume_file.3) 2024-09-12T03:21:19.616 INFO:teuthology.orchestra.run.smithi027.stdout:1000 2024-09-12T03:21:19.617 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:21:19.617 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol84490/a842cae4-61b6-4f73-a96f-03c1581e2365/.snap/subvol_snap39590/subvolume_file.3) 2024-09-12T03:21:19.672 INFO:teuthology.orchestra.run.smithi027.stdout:1333 2024-09-12T03:21:19.673 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:21:19.673 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol_clone8340/fe9d5d1c-b87b-434b-8988-e68eb0a4b017/subvolume_file.3) 2024-09-12T03:21:19.728 INFO:teuthology.orchestra.run.smithi027.stdout:1333 2024-09-12T03:21:19.729 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:21:19.729 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol84490/a842cae4-61b6-4f73-a96f-03c1581e2365/.snap/subvol_snap39590/subvolume_file.3) 2024-09-12T03:21:19.785 INFO:teuthology.orchestra.run.smithi027.stdout:1726111271 2024-09-12T03:21:19.786 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:21:19.786 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol_clone8340/fe9d5d1c-b87b-434b-8988-e68eb0a4b017/subvolume_file.3) 2024-09-12T03:21:19.842 INFO:teuthology.orchestra.run.smithi027.stdout:1726111271 2024-09-12T03:21:19.843 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:21:19.843 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol84490/a842cae4-61b6-4f73-a96f-03c1581e2365/.snap/subvol_snap39590/subvolume_file.1) 2024-09-12T03:21:19.901 INFO:teuthology.orchestra.run.smithi027.stdout:81a4 2024-09-12T03:21:19.901 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:21:19.901 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol_clone8340/fe9d5d1c-b87b-434b-8988-e68eb0a4b017/subvolume_file.1) 2024-09-12T03:21:19.957 INFO:teuthology.orchestra.run.smithi027.stdout:81a4 2024-09-12T03:21:19.957 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:21:19.958 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol84490/a842cae4-61b6-4f73-a96f-03c1581e2365/.snap/subvol_snap39590/subvolume_file.1) 2024-09-12T03:21:20.014 INFO:teuthology.orchestra.run.smithi027.stdout:1000 2024-09-12T03:21:20.014 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:21:20.014 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol_clone8340/fe9d5d1c-b87b-434b-8988-e68eb0a4b017/subvolume_file.1) 2024-09-12T03:21:20.071 INFO:teuthology.orchestra.run.smithi027.stdout:1000 2024-09-12T03:21:20.072 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:21:20.072 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol84490/a842cae4-61b6-4f73-a96f-03c1581e2365/.snap/subvol_snap39590/subvolume_file.1) 2024-09-12T03:21:20.128 INFO:teuthology.orchestra.run.smithi027.stdout:1333 2024-09-12T03:21:20.129 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:21:20.129 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol_clone8340/fe9d5d1c-b87b-434b-8988-e68eb0a4b017/subvolume_file.1) 2024-09-12T03:21:20.185 INFO:teuthology.orchestra.run.smithi027.stdout:1333 2024-09-12T03:21:20.185 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:21:20.185 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol84490/a842cae4-61b6-4f73-a96f-03c1581e2365/.snap/subvol_snap39590/subvolume_file.1) 2024-09-12T03:21:20.243 INFO:teuthology.orchestra.run.smithi027.stdout:1726111271 2024-09-12T03:21:20.243 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:21:20.243 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol_clone8340/fe9d5d1c-b87b-434b-8988-e68eb0a4b017/subvolume_file.1) 2024-09-12T03:21:20.300 INFO:teuthology.orchestra.run.smithi027.stdout:1726111271 2024-09-12T03:21:20.300 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs subvol84490 fake subvol_clone64301 2024-09-12T03:21:20.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:20.449+0000 7fa2e28eb640 1 Processor -- start 2024-09-12T03:21:20.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:20.449+0000 7fa2e28eb640 1 -- start start 2024-09-12T03:21:20.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:20.450+0000 7fa2e28eb640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa2dc15a270 0x7fa2dc06cff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:20.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:20.450+0000 7fa2e28eb640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa2dc06d530 0x7fa2dc0cf410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:20.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:20.450+0000 7fa2e28eb640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fa2dc06c650 0x7fa2dc0cf950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:20.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:20.450+0000 7fa2e28eb640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7fa2dc0cef80 con 0x7fa2dc06c650 2024-09-12T03:21:20.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:20.450+0000 7fa2e28eb640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fa2dc0ce650 con 0x7fa2dc06d530 2024-09-12T03:21:20.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:20.450+0000 7fa2e28eb640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fa2dc0ce930 con 0x7fa2dc15a270 2024-09-12T03:21:20.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:20.450+0000 7fa2e0e88640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fa2dc06c650 0x7fa2dc0cf950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:20.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:20.450+0000 7fa2e0e88640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fa2dc06c650 0x7fa2dc0cf950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57022/0 (socket says 172.21.15.27:57022) 2024-09-12T03:21:20.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:20.450+0000 7fa2e0e88640 1 -- 172.21.15.27:0/1777442453 learned_addr learned my addr 172.21.15.27:0/1777442453 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:21:20.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:20.450+0000 7fa2e0e88640 1 -- 172.21.15.27:0/1777442453 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa2dc15a270 msgr2=0x7fa2dc06cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:20.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:20.450+0000 7fa2e0e88640 1 --2- 172.21.15.27:0/1777442453 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa2dc15a270 0x7fa2dc06cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:20.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:20.450+0000 7fa2e0e88640 1 -- 172.21.15.27:0/1777442453 >> v2:172.21.15.45:3300/0 conn(0x7fa2dc06d530 msgr2=0x7fa2dc0cf410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:20.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:20.450+0000 7fa2e0e88640 1 --2- 172.21.15.27:0/1777442453 >> v2:172.21.15.45:3300/0 conn(0x7fa2dc06d530 0x7fa2dc0cf410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:20.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:20.450+0000 7fa2e0e88640 1 -- 172.21.15.27:0/1777442453 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2dc0cec10 con 0x7fa2dc06c650 2024-09-12T03:21:20.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:20.451+0000 7fa2e0e88640 1 --2- 172.21.15.27:0/1777442453 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa2dc06c650 0x7fa2dc0cf950 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7fa2dc073920 tx=0x7fa2d402a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71f3b5b3a66f1d84 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:20.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:20.451+0000 7fa2daffd640 1 -- 172.21.15.27:0/1777442453 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2d4036070 con 0x7fa2dc06c650 2024-09-12T03:21:20.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:20.451+0000 7fa2daffd640 1 -- 172.21.15.27:0/1777442453 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2d4031600 con 0x7fa2dc06c650 2024-09-12T03:21:20.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:20.451+0000 7fa2daffd640 1 -- 172.21.15.27:0/1777442453 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2d40319d0 con 0x7fa2dc06c650 2024-09-12T03:21:20.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:20.452+0000 7fa2e28eb640 1 -- 172.21.15.27:0/1777442453 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa2dc06c650 msgr2=0x7fa2dc0cf950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:20.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:20.452+0000 7fa2e28eb640 1 --2- 172.21.15.27:0/1777442453 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa2dc06c650 0x7fa2dc0cf950 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7fa2dc073920 tx=0x7fa2d402a6c0 comp rx=0 tx=0).stop 2024-09-12T03:21:20.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:20.452+0000 7fa2e28eb640 1 -- 172.21.15.27:0/1777442453 shutdown_connections 2024-09-12T03:21:20.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:20.452+0000 7fa2e28eb640 1 --2- 172.21.15.27:0/1777442453 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa2dc06c650 0x7fa2dc0cf950 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:20.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:20.452+0000 7fa2e28eb640 1 --2- 172.21.15.27:0/1777442453 >> v2:172.21.15.45:3300/0 conn(0x7fa2dc06d530 0x7fa2dc0cf410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:20.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:20.452+0000 7fa2e28eb640 1 --2- 172.21.15.27:0/1777442453 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa2dc15a270 0x7fa2dc06cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:20.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:20.452+0000 7fa2e28eb640 1 -- 172.21.15.27:0/1777442453 >> 172.21.15.27:0/1777442453 conn(0x7fa2dc06bb30 msgr2=0x7fa2dc06bf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:20.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:20.452+0000 7fa2e28eb640 1 -- 172.21.15.27:0/1777442453 shutdown_connections 2024-09-12T03:21:20.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:20.452+0000 7fa2e28eb640 1 -- 172.21.15.27:0/1777442453 wait complete. 2024-09-12T03:21:20.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:20.453+0000 7fa2e28eb640 1 Processor -- start 2024-09-12T03:21:20.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:20.453+0000 7fa2e28eb640 1 -- start start 2024-09-12T03:21:20.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:20.454+0000 7fa2e28eb640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa2dc06c650 0x7fa2dc0afc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:20.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:20.454+0000 7fa2e28eb640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa2dc06d530 0x7fa2dc151060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:20.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:20.454+0000 7fa2e28eb640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa2dc15a270 0x7fa2dc146f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:20.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:20.454+0000 7fa2e28eb640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fa2dc15d6e0 con 0x7fa2dc15a270 2024-09-12T03:21:20.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:20.454+0000 7fa2e28eb640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa2dc15d560 con 0x7fa2dc06c650 2024-09-12T03:21:20.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:20.454+0000 7fa2e28eb640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fa2dc15d860 con 0x7fa2dc06d530 2024-09-12T03:21:20.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:20.454+0000 7fa2e0e88640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa2dc15a270 0x7fa2dc146f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:20.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:20.454+0000 7fa2e0e88640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa2dc15a270 0x7fa2dc146f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57030/0 (socket says 172.21.15.27:57030) 2024-09-12T03:21:20.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:20.454+0000 7fa2e0e88640 1 -- 172.21.15.27:0/1725977054 learned_addr learned my addr 172.21.15.27:0/1725977054 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:21:20.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:20.454+0000 7fa2db7fe640 1 --2- 172.21.15.27:0/1725977054 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa2dc06d530 0x7fa2dc151060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:20.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:20.454+0000 7fa2dbfff640 1 --2- 172.21.15.27:0/1725977054 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa2dc06c650 0x7fa2dc0afc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:20.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:20.454+0000 7fa2e0e88640 1 -- 172.21.15.27:0/1725977054 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa2dc06d530 msgr2=0x7fa2dc151060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:20.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:20.454+0000 7fa2e0e88640 1 --2- 172.21.15.27:0/1725977054 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa2dc06d530 0x7fa2dc151060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:20.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:20.454+0000 7fa2e0e88640 1 -- 172.21.15.27:0/1725977054 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa2dc06c650 msgr2=0x7fa2dc0afc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:20.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:20.454+0000 7fa2e0e88640 1 --2- 172.21.15.27:0/1725977054 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa2dc06c650 0x7fa2dc0afc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:20.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:20.454+0000 7fa2e0e88640 1 -- 172.21.15.27:0/1725977054 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2dc155120 con 0x7fa2dc15a270 2024-09-12T03:21:20.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:20.454+0000 7fa2dbfff640 1 --2- 172.21.15.27:0/1725977054 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa2dc06c650 0x7fa2dc0afc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:21:20.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:20.454+0000 7fa2db7fe640 1 --2- 172.21.15.27:0/1725977054 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa2dc06d530 0x7fa2dc151060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:21:20.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:20.455+0000 7fa2e0e88640 1 --2- 172.21.15.27:0/1725977054 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa2dc15a270 0x7fa2dc146f20 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7fa2d4034040 tx=0x7fa2d4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:20.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:20.455+0000 7fa2d8ff9640 1 -- 172.21.15.27:0/1725977054 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2d4041020 con 0x7fa2dc15a270 2024-09-12T03:21:20.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:20.455+0000 7fa2d8ff9640 1 -- 172.21.15.27:0/1725977054 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2d40077a0 con 0x7fa2dc15a270 2024-09-12T03:21:20.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:20.455+0000 7fa2e28eb640 1 -- 172.21.15.27:0/1725977054 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2dc155400 con 0x7fa2dc15a270 2024-09-12T03:21:20.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:20.455+0000 7fa2d8ff9640 1 -- 172.21.15.27:0/1725977054 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2d4036040 con 0x7fa2dc15a270 2024-09-12T03:21:20.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:20.455+0000 7fa2e28eb640 1 -- 172.21.15.27:0/1725977054 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2dc1559c0 con 0x7fa2dc15a270 2024-09-12T03:21:20.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:20.456+0000 7fa2e28eb640 1 -- 172.21.15.27:0/1725977054 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2dc0ccdb0 con 0x7fa2dc15a270 2024-09-12T03:21:20.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:20.456+0000 7fa2d8ff9640 1 -- 172.21.15.27:0/1725977054 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 18) ==== 90753+0+0 (secure 0 0 0) 0x7fa2d4044020 con 0x7fa2dc15a270 2024-09-12T03:21:20.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:20.457+0000 7fa2d8ff9640 1 --2- 172.21.15.27:0/1725977054 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fa2c806c100 0x7fa2c806e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:20.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:20.457+0000 7fa2d8ff9640 1 -- 172.21.15.27:0/1725977054 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5767+0+0 (secure 0 0 0) 0x7fa2d40ac1e0 con 0x7fa2dc15a270 2024-09-12T03:21:20.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:20.457+0000 7fa2dbfff640 1 --2- 172.21.15.27:0/1725977054 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fa2c806c100 0x7fa2c806e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:20.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:20.457+0000 7fa2dbfff640 1 --2- 172.21.15.27:0/1725977054 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fa2c806c100 0x7fa2c806e5c0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fa2cc009930 tx=0x7fa2cc008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:20.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:20.461+0000 7fa2d8ff9640 1 -- 172.21.15.27:0/1725977054 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fa2d404b030 con 0x7fa2dc15a270 2024-09-12T03:21:20.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:20.672+0000 7fa2e28eb640 1 -- 172.21.15.27:0/1725977054 --> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "subvol84490", "snap_name": "fake", "target_sub_name": "subvol_clone64301", "target": ["mon-mgr", ""]}) -- 0x7fa2dc063320 con 0x7fa2c806c100 2024-09-12T03:21:20.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:20.724+0000 7fa2d8ff9640 1 -- 172.21.15.27:0/1725977054 <== mgr.4113 v2:172.21.15.27:6836/1226920526 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fa2cc009140 con 0x7fa2c806c100 2024-09-12T03:21:20.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:20.727+0000 7fa2e28eb640 1 -- 172.21.15.27:0/1725977054 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fa2c806c100 msgr2=0x7fa2c806e5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:20.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:20.727+0000 7fa2e28eb640 1 --2- 172.21.15.27:0/1725977054 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fa2c806c100 0x7fa2c806e5c0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fa2cc009930 tx=0x7fa2cc008040 comp rx=0 tx=0).stop 2024-09-12T03:21:20.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:20.727+0000 7fa2e28eb640 1 -- 172.21.15.27:0/1725977054 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa2dc15a270 msgr2=0x7fa2dc146f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:20.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:20.727+0000 7fa2e28eb640 1 --2- 172.21.15.27:0/1725977054 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa2dc15a270 0x7fa2dc146f20 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7fa2d4034040 tx=0x7fa2d4004a40 comp rx=0 tx=0).stop 2024-09-12T03:21:20.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:20.727+0000 7fa2e28eb640 1 -- 172.21.15.27:0/1725977054 shutdown_connections 2024-09-12T03:21:20.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:20.727+0000 7fa2e28eb640 1 --2- 172.21.15.27:0/1725977054 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fa2c806c100 0x7fa2c806e5c0 secure :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fa2cc009930 tx=0x7fa2cc008040 comp rx=0 tx=0).stop 2024-09-12T03:21:20.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:20.727+0000 7fa2e28eb640 1 --2- 172.21.15.27:0/1725977054 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa2dc15a270 0x7fa2dc146f20 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:20.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:20.727+0000 7fa2e28eb640 1 --2- 172.21.15.27:0/1725977054 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa2dc06d530 0x7fa2dc151060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:20.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:20.727+0000 7fa2e28eb640 1 --2- 172.21.15.27:0/1725977054 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa2dc06c650 0x7fa2dc0afc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:20.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:20.727+0000 7fa2e28eb640 1 -- 172.21.15.27:0/1725977054 >> 172.21.15.27:0/1725977054 conn(0x7fa2dc06bb30 msgr2=0x7fa2dc06d930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:20.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:20.727+0000 7fa2e28eb640 1 -- 172.21.15.27:0/1725977054 shutdown_connections 2024-09-12T03:21:20.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:20.727+0000 7fa2e28eb640 1 -- 172.21.15.27:0/1725977054 wait complete. 2024-09-12T03:21:20.740 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone64301 2024-09-12T03:21:20.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:20.844+0000 7f466d931640 1 Processor -- start 2024-09-12T03:21:20.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:20.844+0000 7f466d931640 1 -- start start 2024-09-12T03:21:20.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:20.845+0000 7f466d931640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f4668162040 0x7f46680d1da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:20.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:20.845+0000 7f466d931640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f46680d2e30 con 0x7f466806dfd0 2024-09-12T03:21:20.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:20.845+0000 7f466d931640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4668162420 con 0x7f46680d3790 2024-09-12T03:21:20.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:20.845+0000 7f466d931640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f46680d30d0 con 0x7f4668162040 2024-09-12T03:21:20.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:20.845+0000 7f46677fe640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f466806dfd0 0x7f46680cf320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:33722/0 (socket says 172.21.15.27:33722) 2024-09-12T03:21:20.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:20.845+0000 7f46677fe640 1 -- 172.21.15.27:0/3397912913 learned_addr learned my addr 172.21.15.27:0/3397912913 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:21:20.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:20.846+0000 7f4666ffd640 1 --2- 172.21.15.27:0/3397912913 >> v2:172.21.15.45:3300/0 conn(0x7f4668162040 0x7f46680d1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:20.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:20.846+0000 7f4665ffb640 1 -- 172.21.15.27:0/3397912913 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2817447704 0 0) 0x7f46680d2e30 con 0x7f466806dfd0 2024-09-12T03:21:20.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:20.846+0000 7f4665ffb640 1 -- 172.21.15.27:0/3397912913 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4654003520 con 0x7f466806dfd0 2024-09-12T03:21:20.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:20.846+0000 7f4665ffb640 1 -- 172.21.15.27:0/3397912913 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2285935174 0 0) 0x7f4654003520 con 0x7f466806dfd0 2024-09-12T03:21:20.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:20.846+0000 7f4665ffb640 1 -- 172.21.15.27:0/3397912913 >> v2:172.21.15.45:3300/0 conn(0x7f4668162040 msgr2=0x7f46680d1da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:20.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:20.846+0000 7f4665ffb640 1 --2- 172.21.15.27:0/3397912913 >> v2:172.21.15.45:3300/0 conn(0x7f4668162040 0x7f46680d1da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:20.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:20.846+0000 7f4665ffb640 1 -- 172.21.15.27:0/3397912913 >> v1:172.21.15.45:6789/0 conn(0x7f46680d3790 legacy=0x7f466806d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:20.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:20.846+0000 7f4665ffb640 1 -- 172.21.15.27:0/3397912913 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46680d33b0 con 0x7f466806dfd0 2024-09-12T03:21:20.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:20.846+0000 7f4665ffb640 1 -- 172.21.15.27:0/3397912913 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 847755213 0 0) 0x7f46580019f0 con 0x7f466806dfd0 2024-09-12T03:21:20.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:20.846+0000 7f4665ffb640 1 -- 172.21.15.27:0/3397912913 >> v1:172.21.15.27:6789/0 conn(0x7f466806dfd0 legacy=0x7f46680cf320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:20.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:20.846+0000 7f4665ffb640 1 --2- 172.21.15.27:0/3397912913 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f46540039b0 0x7f4654003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:20.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:20.846+0000 7f4665ffb640 1 --2- 172.21.15.27:0/3397912913 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4654004800 0x7f4654004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:20.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:20.846+0000 7f4665ffb640 1 --2- 172.21.15.27:0/3397912913 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f46540051c0 0x7f465400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:20.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:20.846+0000 7f4665ffb640 1 -- 172.21.15.27:0/3397912913 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f46680d30d0 con 0x7f4654004800 2024-09-12T03:21:20.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:20.846+0000 7f4665ffb640 1 -- 172.21.15.27:0/3397912913 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f465400c060 con 0x7f46540039b0 2024-09-12T03:21:20.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:20.846+0000 7f4665ffb640 1 -- 172.21.15.27:0/3397912913 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f465400c260 con 0x7f46540051c0 2024-09-12T03:21:20.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:20.846+0000 7f46667fc640 1 --2- 172.21.15.27:0/3397912913 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4654004800 0x7f4654004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:20.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:20.846+0000 7f4666ffd640 1 --2- 172.21.15.27:0/3397912913 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f46540039b0 0x7f4654003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:20.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:20.846+0000 7f46677fe640 1 --2- 172.21.15.27:0/3397912913 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f46540051c0 0x7f465400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:20.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:20.846+0000 7f46667fc640 1 -- 172.21.15.27:0/3397912913 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f46540051c0 msgr2=0x7f465400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:20.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:20.846+0000 7f46667fc640 1 --2- 172.21.15.27:0/3397912913 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f46540051c0 0x7f465400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:20.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:20.846+0000 7f46667fc640 1 -- 172.21.15.27:0/3397912913 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f46540039b0 msgr2=0x7f4654003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:20.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:20.846+0000 7f46667fc640 1 --2- 172.21.15.27:0/3397912913 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f46540039b0 0x7f4654003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:20.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:20.846+0000 7f46667fc640 1 -- 172.21.15.27:0/3397912913 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4654003520 con 0x7f4654004800 2024-09-12T03:21:20.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:20.846+0000 7f4666ffd640 1 --2- 172.21.15.27:0/3397912913 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f46540039b0 0x7f4654003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:21:20.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:20.846+0000 7f46677fe640 1 --2- 172.21.15.27:0/3397912913 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f46540051c0 0x7f465400b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:21:20.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:20.846+0000 7f46677fe640 1 -- 172.21.15.27:0/3397912913 reap_dead start 2024-09-12T03:21:20.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:20.847+0000 7f46667fc640 1 --2- 172.21.15.27:0/3397912913 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4654004800 0x7f4654004c80 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f4650001720 tx=0x7f4654005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93d86cae7c2f1923 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:20.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:20.847+0000 7f4665ffb640 1 -- 172.21.15.27:0/3397912913 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4650001370 con 0x7f4654004800 2024-09-12T03:21:20.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:20.847+0000 7f4665ffb640 1 -- 172.21.15.27:0/3397912913 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46500052d0 con 0x7f4654004800 2024-09-12T03:21:20.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:20.847+0000 7f4665ffb640 1 -- 172.21.15.27:0/3397912913 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f465002ab90 con 0x7f4654004800 2024-09-12T03:21:20.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:20.848+0000 7f466d931640 1 -- 172.21.15.27:0/3397912913 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4654004800 msgr2=0x7f4654004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:20.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:20.848+0000 7f466d931640 1 --2- 172.21.15.27:0/3397912913 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4654004800 0x7f4654004c80 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f4650001720 tx=0x7f4654005830 comp rx=0 tx=0).stop 2024-09-12T03:21:20.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:20.848+0000 7f466d931640 1 -- 172.21.15.27:0/3397912913 shutdown_connections 2024-09-12T03:21:20.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:20.848+0000 7f466d931640 1 --2- 172.21.15.27:0/3397912913 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4654004800 0x7f4654004c80 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:20.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:20.848+0000 7f466d931640 1 -- 172.21.15.27:0/3397912913 >> 172.21.15.27:0/3397912913 conn(0x7f46680b6700 msgr2=0x7f46680b6b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:20.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:20.848+0000 7f466d931640 1 -- 172.21.15.27:0/3397912913 shutdown_connections 2024-09-12T03:21:20.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:20.848+0000 7f466d931640 1 -- 172.21.15.27:0/3397912913 wait complete. 2024-09-12T03:21:20.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:20.849+0000 7f466d931640 1 Processor -- start 2024-09-12T03:21:20.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:20.849+0000 7f466d931640 1 -- start start 2024-09-12T03:21:20.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:20.850+0000 7f466d931640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4654004800 0x7f46680ab220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:20.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:20.850+0000 7f466d931640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f46681f7f70 0x7f46680b90f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:20.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:20.850+0000 7f466d931640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f466814f860 0x7f466814fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:20.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:20.850+0000 7f466d931640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f46680d4150 con 0x7f4654004800 2024-09-12T03:21:20.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:20.850+0000 7f466d931640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f46680d4530 con 0x7f466814f860 2024-09-12T03:21:20.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:20.850+0000 7f466d931640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f46680d42d0 con 0x7f46681f7f70 2024-09-12T03:21:20.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:20.850+0000 7f4666ffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4654004800 0x7f46680ab220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:20.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:20.850+0000 7f4666ffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4654004800 0x7f46680ab220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57042/0 (socket says 172.21.15.27:57042) 2024-09-12T03:21:20.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:20.850+0000 7f4666ffd640 1 -- 172.21.15.27:0/2207279884 learned_addr learned my addr 172.21.15.27:0/2207279884 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:21:20.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:20.850+0000 7f4666ffd640 1 -- 172.21.15.27:0/2207279884 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f46681f7f70 msgr2=0x7f46680b90f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:20.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:20.850+0000 7f46667fc640 1 --2- 172.21.15.27:0/2207279884 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f46681f7f70 0x7f46680b90f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:20.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:20.850+0000 7f4666ffd640 1 --2- 172.21.15.27:0/2207279884 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f46681f7f70 0x7f46680b90f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:20.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:20.850+0000 7f46677fe640 1 --2- 172.21.15.27:0/2207279884 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f466814f860 0x7f466814fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:20.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:20.851+0000 7f4666ffd640 1 -- 172.21.15.27:0/2207279884 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f466814f860 msgr2=0x7f466814fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:20.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:20.851+0000 7f4666ffd640 1 --2- 172.21.15.27:0/2207279884 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f466814f860 0x7f466814fce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:20.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:20.851+0000 7f4666ffd640 1 -- 172.21.15.27:0/2207279884 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f466815fa10 con 0x7f4654004800 2024-09-12T03:21:20.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:20.851+0000 7f4666ffd640 1 --2- 172.21.15.27:0/2207279884 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4654004800 0x7f46680ab220 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f46600116d0 tx=0x7f4660011b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:20.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:20.851+0000 7f465ffff640 1 -- 172.21.15.27:0/2207279884 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f466000f9c0 con 0x7f4654004800 2024-09-12T03:21:20.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:20.851+0000 7f465ffff640 1 -- 172.21.15.27:0/2207279884 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f466000fb60 con 0x7f4654004800 2024-09-12T03:21:20.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:20.851+0000 7f466d931640 1 -- 172.21.15.27:0/2207279884 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46681f84e0 con 0x7f4654004800 2024-09-12T03:21:20.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:20.851+0000 7f465ffff640 1 -- 172.21.15.27:0/2207279884 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4660015460 con 0x7f4654004800 2024-09-12T03:21:20.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:20.851+0000 7f466d931640 1 -- 172.21.15.27:0/2207279884 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46681f8aa0 con 0x7f4654004800 2024-09-12T03:21:20.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:20.852+0000 7f466d931640 1 -- 172.21.15.27:0/2207279884 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f466806e6e0 con 0x7f4654004800 2024-09-12T03:21:20.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:20.853+0000 7f465ffff640 1 -- 172.21.15.27:0/2207279884 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 18) ==== 90753+0+0 (secure 0 0 0) 0x7f4660015600 con 0x7f4654004800 2024-09-12T03:21:20.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:20.853+0000 7f465ffff640 1 --2- 172.21.15.27:0/2207279884 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f464806c100 0x7f464806e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:20.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:20.853+0000 7f465ffff640 1 -- 172.21.15.27:0/2207279884 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5767+0+0 (secure 0 0 0) 0x7f466008bdb0 con 0x7f4654004800 2024-09-12T03:21:20.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:20.853+0000 7f46667fc640 1 --2- 172.21.15.27:0/2207279884 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f464806c100 0x7f464806e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:20.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:20.854+0000 7f46667fc640 1 --2- 172.21.15.27:0/2207279884 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f464806c100 0x7f464806e5c0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f4650032000 tx=0x7f465002e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:20.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:20.857+0000 7f465ffff640 1 -- 172.21.15.27:0/2207279884 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f4660019030 con 0x7f4654004800 2024-09-12T03:21:21.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:21.067+0000 7f466d931640 1 -- 172.21.15.27:0/2207279884 --> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone64301", "target": ["mon-mgr", ""]}) -- 0x7f46680cb560 con 0x7f464806c100 2024-09-12T03:21:24.774 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:24.773+0000 7f465ffff640 1 -- 172.21.15.27:0/2207279884 <== mgr.4113 v2:172.21.15.27:6836/1226920526 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+153 (secure 0 0 0) 0x7f46680cb560 con 0x7f464806c100 2024-09-12T03:21:24.774 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-12T03:21:24.774 INFO:teuthology.orchestra.run.smithi027.stdout: "status": { 2024-09-12T03:21:24.774 INFO:teuthology.orchestra.run.smithi027.stdout: "state": "pending", 2024-09-12T03:21:24.774 INFO:teuthology.orchestra.run.smithi027.stdout: "source": { 2024-09-12T03:21:24.775 INFO:teuthology.orchestra.run.smithi027.stdout: "volume": "cephfs", 2024-09-12T03:21:24.775 INFO:teuthology.orchestra.run.smithi027.stdout: "subvolume": "subvol84490", 2024-09-12T03:21:24.775 INFO:teuthology.orchestra.run.smithi027.stdout: "snapshot": "fake" 2024-09-12T03:21:24.775 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:21:24.775 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:21:24.775 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-12T03:21:24.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:24.777+0000 7f466d931640 1 -- 172.21.15.27:0/2207279884 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f464806c100 msgr2=0x7f464806e5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:24.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:24.777+0000 7f466d931640 1 --2- 172.21.15.27:0/2207279884 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f464806c100 0x7f464806e5c0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f4650032000 tx=0x7f465002e000 comp rx=0 tx=0).stop 2024-09-12T03:21:24.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:24.777+0000 7f466d931640 1 -- 172.21.15.27:0/2207279884 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4654004800 msgr2=0x7f46680ab220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:24.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:24.777+0000 7f466d931640 1 --2- 172.21.15.27:0/2207279884 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4654004800 0x7f46680ab220 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f46600116d0 tx=0x7f4660011b90 comp rx=0 tx=0).stop 2024-09-12T03:21:24.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:24.777+0000 7f466d931640 1 -- 172.21.15.27:0/2207279884 shutdown_connections 2024-09-12T03:21:24.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:24.777+0000 7f466d931640 1 --2- 172.21.15.27:0/2207279884 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f466814f860 0x7f466814fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:24.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:24.777+0000 7f466d931640 1 --2- 172.21.15.27:0/2207279884 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f46681f7f70 0x7f46680b90f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:24.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:24.777+0000 7f466d931640 1 --2- 172.21.15.27:0/2207279884 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f464806c100 0x7f464806e5c0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:24.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:24.777+0000 7f466d931640 1 --2- 172.21.15.27:0/2207279884 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4654004800 0x7f46680ab220 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:24.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:24.777+0000 7f466d931640 1 -- 172.21.15.27:0/2207279884 >> 172.21.15.27:0/2207279884 conn(0x7f46680b6700 msgr2=0x7f46680cc880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:24.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:24.778+0000 7f466d931640 1 -- 172.21.15.27:0/2207279884 shutdown_connections 2024-09-12T03:21:24.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:24.778+0000 7f466d931640 1 -- 172.21.15.27:0/2207279884 wait complete. 2024-09-12T03:21:25.796 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone64301 2024-09-12T03:21:25.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:25.915+0000 7fa73ec7b640 1 Processor -- start 2024-09-12T03:21:25.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:25.915+0000 7fa73ec7b640 1 -- start start 2024-09-12T03:21:25.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:25.916+0000 7fa73ec7b640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa7380d3c20 0x7fa7380d4040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:25.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:25.916+0000 7fa73ec7b640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa7380b77c0 0x7fa7380b7c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:25.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:25.916+0000 7fa73ec7b640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa7380d2930 con 0x7fa7380b6240 2024-09-12T03:21:25.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:25.916+0000 7fa73ec7b640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fa7380d2c10 con 0x7fa7380d3c20 2024-09-12T03:21:25.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:25.916+0000 7fa73ec7b640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fa7380d2ef0 con 0x7fa7380b77c0 2024-09-12T03:21:25.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:25.916+0000 7fa73ca17640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fa7380b6240 0x7fa7380b6620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:33726/0 (socket says 172.21.15.27:33726) 2024-09-12T03:21:25.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:25.916+0000 7fa73ca17640 1 -- 172.21.15.27:0/3974109629 learned_addr learned my addr 172.21.15.27:0/3974109629 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:21:25.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:25.916+0000 7fa737fff640 1 --2- 172.21.15.27:0/3974109629 >> v2:172.21.15.45:3300/0 conn(0x7fa7380d3c20 0x7fa7380d4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:25.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:25.916+0000 7fa73d218640 1 --2- 172.21.15.27:0/3974109629 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa7380b77c0 0x7fa7380b7c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:25.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:25.916+0000 7fa7377fe640 1 -- 172.21.15.27:0/3974109629 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2175937382 0 0) 0x7fa7380d2930 con 0x7fa7380b6240 2024-09-12T03:21:25.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:25.916+0000 7fa7377fe640 1 -- 172.21.15.27:0/3974109629 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa728003520 con 0x7fa7380b6240 2024-09-12T03:21:25.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:25.916+0000 7fa7377fe640 1 -- 172.21.15.27:0/3974109629 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 524139210 0 0) 0x7fa728003520 con 0x7fa7380b6240 2024-09-12T03:21:25.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:25.916+0000 7fa7377fe640 1 -- 172.21.15.27:0/3974109629 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa7380b77c0 msgr2=0x7fa7380b7c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:25.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:25.916+0000 7fa7377fe640 1 --2- 172.21.15.27:0/3974109629 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa7380b77c0 0x7fa7380b7c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:25.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:25.917+0000 7fa7377fe640 1 -- 172.21.15.27:0/3974109629 >> v2:172.21.15.45:3300/0 conn(0x7fa7380d3c20 msgr2=0x7fa7380d4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:25.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:25.917+0000 7fa7377fe640 1 --2- 172.21.15.27:0/3974109629 >> v2:172.21.15.45:3300/0 conn(0x7fa7380d3c20 0x7fa7380d4040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:25.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:25.917+0000 7fa7377fe640 1 -- 172.21.15.27:0/3974109629 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7380d31d0 con 0x7fa7380b6240 2024-09-12T03:21:25.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:25.917+0000 7fa7377fe640 1 -- 172.21.15.27:0/3974109629 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 847755213 0 0) 0x7fa72c001670 con 0x7fa7380b6240 2024-09-12T03:21:25.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:25.917+0000 7fa737fff640 1 --2- 172.21.15.27:0/3974109629 >> v2:172.21.15.45:3300/0 conn(0x7fa7380d3c20 0x7fa7380d4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:21:25.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:25.917+0000 7fa73d218640 1 --2- 172.21.15.27:0/3974109629 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa7380b77c0 0x7fa7380b7c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:21:25.923 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:25.917+0000 7fa7377fe640 1 -- 172.21.15.27:0/3974109629 >> v1:172.21.15.27:6789/0 conn(0x7fa7380b6240 legacy=0x7fa7380b6620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:25.923 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:25.917+0000 7fa7377fe640 1 --2- 172.21.15.27:0/3974109629 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa7280039b0 0x7fa728003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:25.923 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:25.917+0000 7fa7377fe640 1 --2- 172.21.15.27:0/3974109629 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa728004800 0x7fa728004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:25.923 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:25.917+0000 7fa7377fe640 1 --2- 172.21.15.27:0/3974109629 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa7380b77c0 0x7fa7280051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:25.923 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:25.917+0000 7fa7377fe640 1 -- 172.21.15.27:0/3974109629 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fa7380d2c10 con 0x7fa7280039b0 2024-09-12T03:21:25.923 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:25.917+0000 7fa7377fe640 1 -- 172.21.15.27:0/3974109629 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa7380d2ef0 con 0x7fa728004800 2024-09-12T03:21:25.923 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:25.917+0000 7fa7377fe640 1 -- 172.21.15.27:0/3974109629 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fa72800b9c0 con 0x7fa7380b77c0 2024-09-12T03:21:25.923 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:25.917+0000 7fa73ca17640 1 --2- 172.21.15.27:0/3974109629 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa7280039b0 0x7fa728003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:25.923 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:25.917+0000 7fa737fff640 1 --2- 172.21.15.27:0/3974109629 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa728004800 0x7fa728004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:25.923 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:25.917+0000 7fa73d218640 1 --2- 172.21.15.27:0/3974109629 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa7380b77c0 0x7fa7280051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:25.923 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:25.917+0000 7fa73ca17640 1 -- 172.21.15.27:0/3974109629 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa7380b77c0 msgr2=0x7fa7280051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:25.923 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:25.917+0000 7fa73ca17640 1 --2- 172.21.15.27:0/3974109629 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa7380b77c0 0x7fa7280051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:25.923 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:25.917+0000 7fa73ca17640 1 -- 172.21.15.27:0/3974109629 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa728004800 msgr2=0x7fa728004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:25.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:25.917+0000 7fa73ca17640 1 --2- 172.21.15.27:0/3974109629 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa728004800 0x7fa728004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:25.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:25.917+0000 7fa73ca17640 1 -- 172.21.15.27:0/3974109629 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa728003520 con 0x7fa7280039b0 2024-09-12T03:21:25.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:25.917+0000 7fa73d218640 1 --2- 172.21.15.27:0/3974109629 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa7380b77c0 0x7fa7280051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:21:25.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:25.918+0000 7fa73ca17640 1 --2- 172.21.15.27:0/3974109629 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa7280039b0 0x7fa728003dd0 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7fa72c00c210 tx=0x7fa728005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c218a9daba2821 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:25.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:25.918+0000 7fa7377fe640 1 -- 172.21.15.27:0/3974109629 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa72c007d50 con 0x7fa7280039b0 2024-09-12T03:21:25.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:25.918+0000 7fa7377fe640 1 -- 172.21.15.27:0/3974109629 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa72c005af0 con 0x7fa7280039b0 2024-09-12T03:21:25.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:25.918+0000 7fa7377fe640 1 -- 172.21.15.27:0/3974109629 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa72c007070 con 0x7fa7280039b0 2024-09-12T03:21:25.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:25.918+0000 7fa73ec7b640 1 -- 172.21.15.27:0/3974109629 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa7280039b0 msgr2=0x7fa728003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:25.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:25.918+0000 7fa73ec7b640 1 --2- 172.21.15.27:0/3974109629 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa7280039b0 0x7fa728003dd0 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7fa72c00c210 tx=0x7fa728005830 comp rx=0 tx=0).stop 2024-09-12T03:21:25.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:25.918+0000 7fa73d218640 1 -- 172.21.15.27:0/3974109629 reap_dead start 2024-09-12T03:21:25.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:25.919+0000 7fa73ec7b640 1 -- 172.21.15.27:0/3974109629 shutdown_connections 2024-09-12T03:21:25.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:25.919+0000 7fa73ec7b640 1 -- 172.21.15.27:0/3974109629 >> 172.21.15.27:0/3974109629 conn(0x7fa7380d1420 msgr2=0x7fa7380d1830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:25.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:25.919+0000 7fa73ec7b640 1 -- 172.21.15.27:0/3974109629 shutdown_connections 2024-09-12T03:21:25.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:25.919+0000 7fa73ec7b640 1 -- 172.21.15.27:0/3974109629 wait complete. 2024-09-12T03:21:25.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:25.920+0000 7fa73ec7b640 1 Processor -- start 2024-09-12T03:21:25.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:25.920+0000 7fa73ec7b640 1 -- start start 2024-09-12T03:21:25.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:25.921+0000 7fa73ec7b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa7280039b0 0x7fa73814dc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:25.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:25.921+0000 7fa73ec7b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa7380cb0d0 0x7fa7380ca700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:25.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:25.921+0000 7fa73ec7b640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa73814e640 0x7fa73814eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:25.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:25.921+0000 7fa73ec7b640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fa7380cfdf0 con 0x7fa7380cb0d0 2024-09-12T03:21:25.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:25.921+0000 7fa73ec7b640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa7380cfc70 con 0x7fa7280039b0 2024-09-12T03:21:25.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:25.921+0000 7fa73ec7b640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fa7380b7c50 con 0x7fa73814e640 2024-09-12T03:21:25.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:25.921+0000 7fa737fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa7380cb0d0 0x7fa7380ca700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:25.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:25.921+0000 7fa737fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa7380cb0d0 0x7fa7380ca700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57046/0 (socket says 172.21.15.27:57046) 2024-09-12T03:21:25.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:25.921+0000 7fa737fff640 1 -- 172.21.15.27:0/46770984 learned_addr learned my addr 172.21.15.27:0/46770984 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:21:25.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:25.921+0000 7fa73d218640 1 --2- 172.21.15.27:0/46770984 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa73814e640 0x7fa73814eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:25.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:25.921+0000 7fa73ca17640 1 --2- 172.21.15.27:0/46770984 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa7280039b0 0x7fa73814dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:25.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:25.921+0000 7fa737fff640 1 -- 172.21.15.27:0/46770984 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa73814e640 msgr2=0x7fa73814eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:25.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:25.921+0000 7fa737fff640 1 --2- 172.21.15.27:0/46770984 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa73814e640 0x7fa73814eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:25.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:25.921+0000 7fa737fff640 1 -- 172.21.15.27:0/46770984 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa7280039b0 msgr2=0x7fa73814dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:25.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:25.921+0000 7fa737fff640 1 --2- 172.21.15.27:0/46770984 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa7280039b0 0x7fa73814dc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:25.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:25.921+0000 7fa737fff640 1 -- 172.21.15.27:0/46770984 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7380c9660 con 0x7fa7380cb0d0 2024-09-12T03:21:25.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:25.921+0000 7fa737fff640 1 --2- 172.21.15.27:0/46770984 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa7380cb0d0 0x7fa7380ca700 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7fa724011d00 tx=0x7fa72400f610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:25.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:25.922+0000 7fa7357fa640 1 -- 172.21.15.27:0/46770984 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa72400fcc0 con 0x7fa7380cb0d0 2024-09-12T03:21:25.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:25.922+0000 7fa7357fa640 1 -- 172.21.15.27:0/46770984 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa72400fe60 con 0x7fa7380cb0d0 2024-09-12T03:21:25.930 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:25.922+0000 7fa73ec7b640 1 -- 172.21.15.27:0/46770984 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7380c9940 con 0x7fa7380cb0d0 2024-09-12T03:21:25.930 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:25.922+0000 7fa7357fa640 1 -- 172.21.15.27:0/46770984 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa724004270 con 0x7fa7380cb0d0 2024-09-12T03:21:25.930 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:25.922+0000 7fa73ec7b640 1 -- 172.21.15.27:0/46770984 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7380c9f00 con 0x7fa7380cb0d0 2024-09-12T03:21:25.930 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:25.923+0000 7fa73ec7b640 1 -- 172.21.15.27:0/46770984 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7380d4040 con 0x7fa7380cb0d0 2024-09-12T03:21:25.930 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:25.923+0000 7fa7357fa640 1 -- 172.21.15.27:0/46770984 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 18) ==== 90753+0+0 (secure 0 0 0) 0x7fa724012040 con 0x7fa7380cb0d0 2024-09-12T03:21:25.930 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:25.924+0000 7fa7357fa640 1 --2- 172.21.15.27:0/46770984 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fa71c06c130 0x7fa71c06e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:25.930 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:25.924+0000 7fa7357fa640 1 -- 172.21.15.27:0/46770984 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5767+0+0 (secure 0 0 0) 0x7fa72408b760 con 0x7fa7380cb0d0 2024-09-12T03:21:25.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:25.924+0000 7fa73ca17640 1 --2- 172.21.15.27:0/46770984 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fa71c06c130 0x7fa71c06e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:25.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:25.924+0000 7fa73ca17640 1 --2- 172.21.15.27:0/46770984 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fa71c06c130 0x7fa71c06e5f0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fa72c033000 tx=0x7fa72c009030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:25.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:25.928+0000 7fa7357fa640 1 -- 172.21.15.27:0/46770984 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fa724019070 con 0x7fa7380cb0d0 2024-09-12T03:21:26.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:26.140+0000 7fa73ec7b640 1 -- 172.21.15.27:0/46770984 --> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone64301", "target": ["mon-mgr", ""]}) -- 0x7fa73815e1f0 con 0x7fa71c06c130 2024-09-12T03:21:29.767 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:29.766+0000 7fa7357fa640 1 -- 172.21.15.27:0/46770984 <== mgr.4113 v2:172.21.15.27:6836/1226920526 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+46 (secure 0 0 0) 0x7fa73815e1f0 con 0x7fa71c06c130 2024-09-12T03:21:29.767 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-12T03:21:29.768 INFO:teuthology.orchestra.run.smithi027.stdout: "status": { 2024-09-12T03:21:29.768 INFO:teuthology.orchestra.run.smithi027.stdout: "state": "complete" 2024-09-12T03:21:29.768 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:21:29.768 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-12T03:21:29.770 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:29.769+0000 7fa73ec7b640 1 -- 172.21.15.27:0/46770984 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fa71c06c130 msgr2=0x7fa71c06e5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:29.770 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:29.769+0000 7fa73ec7b640 1 --2- 172.21.15.27:0/46770984 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fa71c06c130 0x7fa71c06e5f0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fa72c033000 tx=0x7fa72c009030 comp rx=0 tx=0).stop 2024-09-12T03:21:29.770 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:29.769+0000 7fa73ec7b640 1 -- 172.21.15.27:0/46770984 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa7380cb0d0 msgr2=0x7fa7380ca700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:29.770 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:29.769+0000 7fa73ec7b640 1 --2- 172.21.15.27:0/46770984 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa7380cb0d0 0x7fa7380ca700 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7fa724011d00 tx=0x7fa72400f610 comp rx=0 tx=0).stop 2024-09-12T03:21:29.770 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:29.769+0000 7fa73ec7b640 1 -- 172.21.15.27:0/46770984 shutdown_connections 2024-09-12T03:21:29.770 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:29.769+0000 7fa73ec7b640 1 --2- 172.21.15.27:0/46770984 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa73814e640 0x7fa73814eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:29.770 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:29.769+0000 7fa73ec7b640 1 --2- 172.21.15.27:0/46770984 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fa71c06c130 0x7fa71c06e5f0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:29.770 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:29.769+0000 7fa73ec7b640 1 --2- 172.21.15.27:0/46770984 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa7380cb0d0 0x7fa7380ca700 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:29.771 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:29.769+0000 7fa73ec7b640 1 --2- 172.21.15.27:0/46770984 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa7280039b0 0x7fa73814dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:29.771 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:29.769+0000 7fa73ec7b640 1 -- 172.21.15.27:0/46770984 >> 172.21.15.27:0/46770984 conn(0x7fa7380d1420 msgr2=0x7fa7380b6620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:29.771 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:29.769+0000 7fa73ec7b640 1 -- 172.21.15.27:0/46770984 shutdown_connections 2024-09-12T03:21:29.771 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:29.769+0000 7fa73ec7b640 1 -- 172.21.15.27:0/46770984 wait complete. 2024-09-12T03:21:29.784 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:21:29.784 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cat ./volumes/_nogroup/subvol_clone64301/.meta) 2024-09-12T03:21:29.814 INFO:teuthology.orchestra.run.smithi027.stdout:[GLOBAL] 2024-09-12T03:21:29.814 INFO:teuthology.orchestra.run.smithi027.stdout:version = 2 2024-09-12T03:21:29.815 INFO:teuthology.orchestra.run.smithi027.stdout:type = clone 2024-09-12T03:21:29.815 INFO:teuthology.orchestra.run.smithi027.stdout:path = /volumes/_nogroup/subvol_clone64301/d0856545-5b52-4552-a01f-e85b6b1a49f5 2024-09-12T03:21:29.815 INFO:teuthology.orchestra.run.smithi027.stdout:state = complete 2024-09-12T03:21:29.815 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:21:29.816 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot info cephfs subvol84490 subvol_snap39590 2024-09-12T03:21:29.968 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:29.967+0000 7fef7b44c640 1 Processor -- start 2024-09-12T03:21:29.968 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:29.967+0000 7fef7b44c640 1 -- start start 2024-09-12T03:21:29.970 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:29.968+0000 7fef7b44c640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fef740b6240 0x7fef740b6620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:29.970 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:29.968+0000 7fef7b44c640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fef740d3c20 0x7fef740d4040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:29.971 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:29.968+0000 7fef7b44c640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fef740b6b60 con 0x7fef7406d960 2024-09-12T03:21:29.971 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:29.968+0000 7fef7b44c640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7fef740d2930 con 0x7fef740b6240 2024-09-12T03:21:29.971 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:29.968+0000 7fef7b44c640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fef740d2c10 con 0x7fef740d3c20 2024-09-12T03:21:29.971 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:29.968+0000 7fef791e8640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fef740b6240 0x7fef740b6620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:29.971 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:29.968+0000 7fef791e8640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fef740b6240 0x7fef740b6620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41746/0 (socket says 172.21.15.27:41746) 2024-09-12T03:21:29.971 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:29.968+0000 7fef791e8640 1 -- 172.21.15.27:0/2216717722 learned_addr learned my addr 172.21.15.27:0/2216717722 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:21:29.971 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:29.969+0000 7fef789e7640 1 --2- 172.21.15.27:0/2216717722 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fef740d3c20 0x7fef740d4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:29.971 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:29.969+0000 7fef791e8640 1 -- 172.21.15.27:0/2216717722 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fef740d3c20 msgr2=0x7fef740d4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:29.971 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:29.969+0000 7fef791e8640 1 --2- 172.21.15.27:0/2216717722 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fef740d3c20 0x7fef740d4040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:29.971 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:29.969+0000 7fef791e8640 1 -- 172.21.15.27:0/2216717722 >> v1:172.21.15.45:6789/0 conn(0x7fef7406d960 legacy=0x7fef7406de10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:29.971 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:29.969+0000 7fef791e8640 1 -- 172.21.15.27:0/2216717722 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef740d2ef0 con 0x7fef740b6240 2024-09-12T03:21:29.971 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:29.969+0000 7fef789e7640 1 --2- 172.21.15.27:0/2216717722 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fef740d3c20 0x7fef740d4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:21:29.971 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:29.969+0000 7fef791e8640 1 --2- 172.21.15.27:0/2216717722 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fef740b6240 0x7fef740b6620 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7fef64009e50 tx=0x7fef6402a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10cadd19a9c848bf server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:29.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:29.970+0000 7fef73fff640 1 -- 172.21.15.27:0/2216717722 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef64036070 con 0x7fef740b6240 2024-09-12T03:21:29.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:29.970+0000 7fef73fff640 1 -- 172.21.15.27:0/2216717722 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef64031600 con 0x7fef740b6240 2024-09-12T03:21:29.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:29.970+0000 7fef73fff640 1 -- 172.21.15.27:0/2216717722 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef640319d0 con 0x7fef740b6240 2024-09-12T03:21:29.974 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:29.970+0000 7fef7b44c640 1 -- 172.21.15.27:0/2216717722 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fef740b6240 msgr2=0x7fef740b6620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:29.975 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:29.970+0000 7fef7b44c640 1 --2- 172.21.15.27:0/2216717722 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fef740b6240 0x7fef740b6620 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7fef64009e50 tx=0x7fef6402a7a0 comp rx=0 tx=0).stop 2024-09-12T03:21:29.975 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:29.970+0000 7fef7b44c640 1 -- 172.21.15.27:0/2216717722 shutdown_connections 2024-09-12T03:21:29.975 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:29.970+0000 7fef7b44c640 1 --2- 172.21.15.27:0/2216717722 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fef740d3c20 0x7fef740d4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:29.975 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:29.970+0000 7fef7b44c640 1 --2- 172.21.15.27:0/2216717722 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fef740b6240 0x7fef740b6620 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:29.975 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:29.970+0000 7fef7b44c640 1 -- 172.21.15.27:0/2216717722 >> 172.21.15.27:0/2216717722 conn(0x7fef740d1420 msgr2=0x7fef740d1830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:29.975 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:29.970+0000 7fef7b44c640 1 -- 172.21.15.27:0/2216717722 shutdown_connections 2024-09-12T03:21:29.975 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:29.970+0000 7fef7b44c640 1 -- 172.21.15.27:0/2216717722 wait complete. 2024-09-12T03:21:29.975 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:29.972+0000 7fef7b44c640 1 Processor -- start 2024-09-12T03:21:29.975 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:29.972+0000 7fef7b44c640 1 -- start start 2024-09-12T03:21:29.975 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:29.972+0000 7fef7b44c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fef7406d960 0x7fef740cab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:29.976 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:29.972+0000 7fef7b44c640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fef740b6240 0x7fef74157980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:29.976 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:29.972+0000 7fef7b44c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef740d3c20 0x7fef74157ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:29.976 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:29.972+0000 7fef7b44c640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fef7415ece0 con 0x7fef7406d960 2024-09-12T03:21:29.976 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:29.972+0000 7fef7b44c640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fef7415eb60 con 0x7fef740d3c20 2024-09-12T03:21:29.976 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:29.972+0000 7fef7b44c640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fef740cfc70 con 0x7fef740b6240 2024-09-12T03:21:29.976 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:29.972+0000 7fef791e8640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fef7406d960 0x7fef740cab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:29.976 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:29.972+0000 7fef791e8640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fef7406d960 0x7fef740cab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41756/0 (socket says 172.21.15.27:41756) 2024-09-12T03:21:29.976 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:29.972+0000 7fef791e8640 1 -- 172.21.15.27:0/2749711363 learned_addr learned my addr 172.21.15.27:0/2749711363 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:21:29.976 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:29.973+0000 7fef789e7640 1 --2- 172.21.15.27:0/2749711363 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fef740b6240 0x7fef74157980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:29.976 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:29.973+0000 7fef799e9640 1 --2- 172.21.15.27:0/2749711363 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef740d3c20 0x7fef74157ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:29.976 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:29.973+0000 7fef791e8640 1 -- 172.21.15.27:0/2749711363 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fef740b6240 msgr2=0x7fef74157980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:29.976 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:29.973+0000 7fef791e8640 1 --2- 172.21.15.27:0/2749711363 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fef740b6240 0x7fef74157980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:29.976 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:29.973+0000 7fef791e8640 1 -- 172.21.15.27:0/2749711363 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef740d3c20 msgr2=0x7fef74157ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:29.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:29.973+0000 7fef791e8640 1 --2- 172.21.15.27:0/2749711363 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef740d3c20 0x7fef74157ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:29.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:29.973+0000 7fef791e8640 1 -- 172.21.15.27:0/2749711363 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef740cdf00 con 0x7fef7406d960 2024-09-12T03:21:29.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:29.973+0000 7fef799e9640 1 --2- 172.21.15.27:0/2749711363 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef740d3c20 0x7fef74157ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:21:29.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:29.973+0000 7fef791e8640 1 --2- 172.21.15.27:0/2749711363 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fef7406d960 0x7fef740cab70 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7fef64000f80 tx=0x7fef64004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:29.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:29.973+0000 7fef71ffb640 1 -- 172.21.15.27:0/2749711363 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef64036050 con 0x7fef7406d960 2024-09-12T03:21:29.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:29.973+0000 7fef71ffb640 1 -- 172.21.15.27:0/2749711363 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef64042070 con 0x7fef7406d960 2024-09-12T03:21:29.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:29.973+0000 7fef71ffb640 1 -- 172.21.15.27:0/2749711363 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef64009510 con 0x7fef7406d960 2024-09-12T03:21:29.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:29.973+0000 7fef7b44c640 1 -- 172.21.15.27:0/2749711363 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef740ce1e0 con 0x7fef7406d960 2024-09-12T03:21:29.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:29.973+0000 7fef7b44c640 1 -- 172.21.15.27:0/2749711363 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef740ce7a0 con 0x7fef7406d960 2024-09-12T03:21:29.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:29.975+0000 7fef7b44c640 1 -- 172.21.15.27:0/2749711363 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef740b6620 con 0x7fef7406d960 2024-09-12T03:21:29.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:29.975+0000 7fef71ffb640 1 -- 172.21.15.27:0/2749711363 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 18) ==== 90753+0+0 (secure 0 0 0) 0x7fef64007c50 con 0x7fef7406d960 2024-09-12T03:21:29.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:29.976+0000 7fef71ffb640 1 --2- 172.21.15.27:0/2749711363 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fef6006c100 0x7fef6006e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:29.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:29.976+0000 7fef789e7640 1 --2- 172.21.15.27:0/2749711363 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fef6006c100 0x7fef6006e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:29.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:29.976+0000 7fef71ffb640 1 -- 172.21.15.27:0/2749711363 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5767+0+0 (secure 0 0 0) 0x7fef640ac260 con 0x7fef7406d960 2024-09-12T03:21:29.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:29.976+0000 7fef789e7640 1 --2- 172.21.15.27:0/2749711363 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fef6006c100 0x7fef6006e5c0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fef740c8730 tx=0x7fef68008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:29.981 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:29.980+0000 7fef71ffb640 1 -- 172.21.15.27:0/2749711363 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fef6407a950 con 0x7fef7406d960 2024-09-12T03:21:30.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:30.187+0000 7fef7b44c640 1 -- 172.21.15.27:0/2749711363 --> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot info", "vol_name": "cephfs", "sub_name": "subvol84490", "snap_name": "subvol_snap39590", "target": ["mon-mgr", ""]}) -- 0x7fef740b5be0 con 0x7fef6006c100 2024-09-12T03:21:30.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:30.202+0000 7fef71ffb640 1 -- 172.21.15.27:0/2749711363 <== mgr.4113 v2:172.21.15.27:6836/1226920526 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+115 (secure 0 0 0) 0x7fef740b5be0 con 0x7fef6006c100 2024-09-12T03:21:30.203 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-12T03:21:30.203 INFO:teuthology.orchestra.run.smithi027.stdout: "created_at": "2024-09-12 03:21:12.679811", 2024-09-12T03:21:30.203 INFO:teuthology.orchestra.run.smithi027.stdout: "data_pool": "cephfs_data", 2024-09-12T03:21:30.203 INFO:teuthology.orchestra.run.smithi027.stdout: "has_pending_clones": "no" 2024-09-12T03:21:30.203 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-12T03:21:30.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:30.204+0000 7fef7b44c640 1 -- 172.21.15.27:0/2749711363 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fef6006c100 msgr2=0x7fef6006e5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:30.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:30.204+0000 7fef7b44c640 1 --2- 172.21.15.27:0/2749711363 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fef6006c100 0x7fef6006e5c0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fef740c8730 tx=0x7fef68008040 comp rx=0 tx=0).stop 2024-09-12T03:21:30.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:30.204+0000 7fef7b44c640 1 -- 172.21.15.27:0/2749711363 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fef7406d960 msgr2=0x7fef740cab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:30.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:30.204+0000 7fef7b44c640 1 --2- 172.21.15.27:0/2749711363 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fef7406d960 0x7fef740cab70 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7fef64000f80 tx=0x7fef64004a40 comp rx=0 tx=0).stop 2024-09-12T03:21:30.206 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:30.204+0000 7fef7b44c640 1 -- 172.21.15.27:0/2749711363 shutdown_connections 2024-09-12T03:21:30.206 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:30.204+0000 7fef7b44c640 1 --2- 172.21.15.27:0/2749711363 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef740d3c20 0x7fef74157ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:30.206 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:30.204+0000 7fef7b44c640 1 --2- 172.21.15.27:0/2749711363 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fef740b6240 0x7fef74157980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:30.206 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:30.204+0000 7fef7b44c640 1 --2- 172.21.15.27:0/2749711363 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fef6006c100 0x7fef6006e5c0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:30.206 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:30.204+0000 7fef7b44c640 1 --2- 172.21.15.27:0/2749711363 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fef7406d960 0x7fef740cab70 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:30.206 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:30.204+0000 7fef7b44c640 1 -- 172.21.15.27:0/2749711363 >> 172.21.15.27:0/2749711363 conn(0x7fef740d1420 msgr2=0x7fef740d1800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:30.206 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:30.204+0000 7fef7b44c640 1 -- 172.21.15.27:0/2749711363 shutdown_connections 2024-09-12T03:21:30.206 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:30.204+0000 7fef7b44c640 1 -- 172.21.15.27:0/2749711363 wait complete. 2024-09-12T03:21:30.216 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot ls cephfs subvol84490 2024-09-12T03:21:30.323 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:30.322+0000 7f33de2a4640 1 Processor -- start 2024-09-12T03:21:30.323 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:30.322+0000 7f33de2a4640 1 -- start start 2024-09-12T03:21:30.324 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:30.323+0000 7f33de2a4640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f33d815c6c0 0x7f33d80d00a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:30.324 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:30.323+0000 7f33de2a4640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f33d80d1750 0x7f33d80b6f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:30.324 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:30.323+0000 7f33de2a4640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f33d80d0a80 con 0x7f33d80d05e0 2024-09-12T03:21:30.325 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:30.323+0000 7f33de2a4640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f33d815caa0 con 0x7f33d815c6c0 2024-09-12T03:21:30.325 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:30.323+0000 7f33de2a4640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f33d80b6510 con 0x7f33d80d1750 2024-09-12T03:21:30.325 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:30.323+0000 7f33d7fff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f33d815c6c0 0x7f33d80d00a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:30.325 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:30.323+0000 7f33d77fe640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f33d80d05e0 0x7f33d80d1040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:51792/0 (socket says 172.21.15.27:51792) 2024-09-12T03:21:30.325 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:30.323+0000 7f33d77fe640 1 -- 172.21.15.27:0/3633828484 learned_addr learned my addr 172.21.15.27:0/3633828484 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:21:30.325 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:30.323+0000 7f33dc841640 1 --2- 172.21.15.27:0/3633828484 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f33d80d1750 0x7f33d80b6f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:30.325 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:30.323+0000 7f33d7fff640 1 -- 172.21.15.27:0/3633828484 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f33d80d1750 msgr2=0x7f33d80b6f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:30.325 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:30.323+0000 7f33d7fff640 1 --2- 172.21.15.27:0/3633828484 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f33d80d1750 0x7f33d80b6f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:30.325 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:30.323+0000 7f33d7fff640 1 -- 172.21.15.27:0/3633828484 >> v1:172.21.15.27:6789/0 conn(0x7f33d80d05e0 legacy=0x7f33d80d1040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:30.325 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:30.323+0000 7f33d7fff640 1 -- 172.21.15.27:0/3633828484 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33d80d0d60 con 0x7f33d815c6c0 2024-09-12T03:21:30.325 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:30.323+0000 7f33d6ffd640 1 -- 172.21.15.27:0/3633828484 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2730197605 0 0) 0x7f33d80d0a80 con 0x7f33d80d05e0 2024-09-12T03:21:30.326 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:30.324+0000 7f33d7fff640 1 --2- 172.21.15.27:0/3633828484 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f33d815c6c0 0x7f33d80d00a0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f33c80097d0 tx=0x7f33c802a130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f4f875a5f422a15 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:30.326 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:30.324+0000 7f33d6ffd640 1 -- 172.21.15.27:0/3633828484 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33c8037070 con 0x7f33d815c6c0 2024-09-12T03:21:30.327 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:30.324+0000 7f33d6ffd640 1 -- 172.21.15.27:0/3633828484 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f33c802ae40 con 0x7f33d815c6c0 2024-09-12T03:21:30.327 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:30.324+0000 7f33d6ffd640 1 -- 172.21.15.27:0/3633828484 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33c8032660 con 0x7f33d815c6c0 2024-09-12T03:21:30.327 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:30.325+0000 7f33de2a4640 1 -- 172.21.15.27:0/3633828484 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f33d815c6c0 msgr2=0x7f33d80d00a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:30.327 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:30.325+0000 7f33de2a4640 1 --2- 172.21.15.27:0/3633828484 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f33d815c6c0 0x7f33d80d00a0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f33c80097d0 tx=0x7f33c802a130 comp rx=0 tx=0).stop 2024-09-12T03:21:30.327 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:30.325+0000 7f33de2a4640 1 -- 172.21.15.27:0/3633828484 shutdown_connections 2024-09-12T03:21:30.327 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:30.325+0000 7f33de2a4640 1 --2- 172.21.15.27:0/3633828484 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f33d80d1750 0x7f33d80b6f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:30.327 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:30.325+0000 7f33de2a4640 1 --2- 172.21.15.27:0/3633828484 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f33d815c6c0 0x7f33d80d00a0 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:30.327 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:30.325+0000 7f33de2a4640 1 -- 172.21.15.27:0/3633828484 >> 172.21.15.27:0/3633828484 conn(0x7f33d806f320 msgr2=0x7f33d806f700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:30.327 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:30.325+0000 7f33de2a4640 1 -- 172.21.15.27:0/3633828484 shutdown_connections 2024-09-12T03:21:30.327 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:30.325+0000 7f33de2a4640 1 -- 172.21.15.27:0/3633828484 wait complete. 2024-09-12T03:21:30.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:30.326+0000 7f33de2a4640 1 Processor -- start 2024-09-12T03:21:30.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:30.326+0000 7f33de2a4640 1 -- start start 2024-09-12T03:21:30.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:30.327+0000 7f33de2a4640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f33d80d05e0 0x7f33d806cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:30.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:30.327+0000 7f33de2a4640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f33d80d1750 0x7f33d814ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:30.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:30.327+0000 7f33de2a4640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f33d815c6c0 0x7f33d8147900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:30.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:30.327+0000 7f33de2a4640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f33d80b4400 con 0x7f33d815c6c0 2024-09-12T03:21:30.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:30.327+0000 7f33de2a4640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f33d80b6a50 con 0x7f33d80d05e0 2024-09-12T03:21:30.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:30.327+0000 7f33de2a4640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f33d80b4580 con 0x7f33d80d1750 2024-09-12T03:21:30.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:30.327+0000 7f33dc841640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f33d815c6c0 0x7f33d8147900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:30.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:30.327+0000 7f33dc841640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f33d815c6c0 0x7f33d8147900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41770/0 (socket says 172.21.15.27:41770) 2024-09-12T03:21:30.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:30.327+0000 7f33dc841640 1 -- 172.21.15.27:0/3026806418 learned_addr learned my addr 172.21.15.27:0/3026806418 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:21:30.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:30.327+0000 7f33d7fff640 1 --2- 172.21.15.27:0/3026806418 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f33d80d05e0 0x7f33d806cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:30.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:30.327+0000 7f33d77fe640 1 --2- 172.21.15.27:0/3026806418 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f33d80d1750 0x7f33d814ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:30.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:30.328+0000 7f33dc841640 1 -- 172.21.15.27:0/3026806418 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f33d80d1750 msgr2=0x7f33d814ace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:30.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:30.328+0000 7f33dc841640 1 --2- 172.21.15.27:0/3026806418 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f33d80d1750 0x7f33d814ace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:30.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:30.328+0000 7f33dc841640 1 -- 172.21.15.27:0/3026806418 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f33d80d05e0 msgr2=0x7f33d806cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:30.331 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:30.328+0000 7f33dc841640 1 --2- 172.21.15.27:0/3026806418 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f33d80d05e0 0x7f33d806cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:30.331 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:30.328+0000 7f33dc841640 1 -- 172.21.15.27:0/3026806418 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33d806d260 con 0x7f33d815c6c0 2024-09-12T03:21:30.331 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:30.328+0000 7f33dc841640 1 --2- 172.21.15.27:0/3026806418 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f33d815c6c0 0x7f33d8147900 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f33d000efc0 tx=0x7f33d000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:30.331 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:30.328+0000 7f33d4ff9640 1 -- 172.21.15.27:0/3026806418 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33d0018020 con 0x7f33d815c6c0 2024-09-12T03:21:30.331 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:30.328+0000 7f33d4ff9640 1 -- 172.21.15.27:0/3026806418 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f33d0009b10 con 0x7f33d815c6c0 2024-09-12T03:21:30.331 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:30.328+0000 7f33de2a4640 1 -- 172.21.15.27:0/3026806418 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33d806d540 con 0x7f33d815c6c0 2024-09-12T03:21:30.331 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:30.328+0000 7f33d4ff9640 1 -- 172.21.15.27:0/3026806418 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33d00146c0 con 0x7f33d815c6c0 2024-09-12T03:21:30.331 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:30.328+0000 7f33de2a4640 1 -- 172.21.15.27:0/3026806418 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33d806db00 con 0x7f33d815c6c0 2024-09-12T03:21:30.332 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:30.329+0000 7f33de2a4640 1 -- 172.21.15.27:0/3026806418 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33d80d00a0 con 0x7f33d815c6c0 2024-09-12T03:21:30.332 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:30.329+0000 7f33d4ff9640 1 -- 172.21.15.27:0/3026806418 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 18) ==== 90753+0+0 (secure 0 0 0) 0x7f33d0005d10 con 0x7f33d815c6c0 2024-09-12T03:21:30.332 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:30.330+0000 7f33d4ff9640 1 --2- 172.21.15.27:0/3026806418 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f33c406c100 0x7f33c406e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:30.332 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:30.330+0000 7f33d4ff9640 1 -- 172.21.15.27:0/3026806418 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5767+0+0 (secure 0 0 0) 0x7f33d008bca0 con 0x7f33d815c6c0 2024-09-12T03:21:30.332 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:30.330+0000 7f33d7fff640 1 --2- 172.21.15.27:0/3026806418 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f33c406c100 0x7f33c406e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:30.332 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:30.330+0000 7f33d7fff640 1 --2- 172.21.15.27:0/3026806418 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f33c406c100 0x7f33c406e5c0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f33c8034000 tx=0x7f33c802e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:30.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:30.334+0000 7f33d4ff9640 1 -- 172.21.15.27:0/3026806418 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f33d0022070 con 0x7f33d815c6c0 2024-09-12T03:21:30.539 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:30.538+0000 7f33de2a4640 1 -- 172.21.15.27:0/3026806418 --> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot ls", "vol_name": "cephfs", "sub_name": "subvol84490", "target": ["mon-mgr", ""]}) -- 0x7f33d80b7920 con 0x7f33c406c100 2024-09-12T03:21:30.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:30.549+0000 7f33d4ff9640 1 -- 172.21.15.27:0/3026806418 <== mgr.4113 v2:172.21.15.27:6836/1226920526 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+87 (secure 0 0 0) 0x7f33d80b7920 con 0x7f33c406c100 2024-09-12T03:21:30.551 INFO:teuthology.orchestra.run.smithi027.stdout:[ 2024-09-12T03:21:30.551 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:21:30.551 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "fake" 2024-09-12T03:21:30.551 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:21:30.551 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:21:30.551 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "subvol_snap39590" 2024-09-12T03:21:30.551 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:21:30.551 INFO:teuthology.orchestra.run.smithi027.stdout:] 2024-09-12T03:21:30.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:30.552+0000 7f33de2a4640 1 -- 172.21.15.27:0/3026806418 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f33c406c100 msgr2=0x7f33c406e5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:30.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:30.552+0000 7f33de2a4640 1 --2- 172.21.15.27:0/3026806418 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f33c406c100 0x7f33c406e5c0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f33c8034000 tx=0x7f33c802e000 comp rx=0 tx=0).stop 2024-09-12T03:21:30.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:30.552+0000 7f33de2a4640 1 -- 172.21.15.27:0/3026806418 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f33d815c6c0 msgr2=0x7f33d8147900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:30.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:30.552+0000 7f33de2a4640 1 --2- 172.21.15.27:0/3026806418 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f33d815c6c0 0x7f33d8147900 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f33d000efc0 tx=0x7f33d000c490 comp rx=0 tx=0).stop 2024-09-12T03:21:30.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:30.552+0000 7f33de2a4640 1 -- 172.21.15.27:0/3026806418 shutdown_connections 2024-09-12T03:21:30.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:30.552+0000 7f33de2a4640 1 --2- 172.21.15.27:0/3026806418 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f33c406c100 0x7f33c406e5c0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:30.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:30.552+0000 7f33de2a4640 1 --2- 172.21.15.27:0/3026806418 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f33d815c6c0 0x7f33d8147900 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:30.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:30.552+0000 7f33de2a4640 1 --2- 172.21.15.27:0/3026806418 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f33d80d1750 0x7f33d814ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:30.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:30.552+0000 7f33de2a4640 1 --2- 172.21.15.27:0/3026806418 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f33d80d05e0 0x7f33d806cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:30.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:30.552+0000 7f33de2a4640 1 -- 172.21.15.27:0/3026806418 >> 172.21.15.27:0/3026806418 conn(0x7f33d806f320 msgr2=0x7f33d80b8120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:30.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:30.552+0000 7f33de2a4640 1 -- 172.21.15.27:0/3026806418 shutdown_connections 2024-09-12T03:21:30.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:30.552+0000 7f33de2a4640 1 -- 172.21.15.27:0/3026806418 wait complete. 2024-09-12T03:21:30.563 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm cephfs subvol84490 subvol_snap39590 2024-09-12T03:21:30.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:30.670+0000 7fd3522a6640 1 Processor -- start 2024-09-12T03:21:30.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:30.670+0000 7fd3522a6640 1 -- start start 2024-09-12T03:21:30.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:30.671+0000 7fd3522a6640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fd34c0d3810 0x7fd34c162f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:30.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:30.671+0000 7fd3522a6640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fd34c0d3330 0x7fd34c0b7fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:30.674 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:30.671+0000 7fd3522a6640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd34c0d4310 con 0x7fd34c0b6440 2024-09-12T03:21:30.674 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:30.671+0000 7fd3522a6640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7fd34c0b7b30 con 0x7fd34c0d3330 2024-09-12T03:21:30.674 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:30.671+0000 7fd3522a6640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fd34c0d1e90 con 0x7fd34c0d3810 2024-09-12T03:21:30.674 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:30.671+0000 7fd350843640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fd34c0d3330 0x7fd34c0b7fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:30.674 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:30.671+0000 7fd34bfff640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fd34c0b6440 0x7fd34c162840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:51804/0 (socket says 172.21.15.27:51804) 2024-09-12T03:21:30.674 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:30.671+0000 7fd34bfff640 1 -- 172.21.15.27:0/1999421293 learned_addr learned my addr 172.21.15.27:0/1999421293 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:21:30.674 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:30.672+0000 7fd34b7fe640 1 --2- 172.21.15.27:0/1999421293 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fd34c0d3810 0x7fd34c162f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:30.674 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:30.672+0000 7fd350843640 1 -- 172.21.15.27:0/1999421293 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fd34c0d3810 msgr2=0x7fd34c162f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:30.674 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:30.672+0000 7fd350843640 1 --2- 172.21.15.27:0/1999421293 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fd34c0d3810 0x7fd34c162f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:30.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:30.672+0000 7fd350843640 1 -- 172.21.15.27:0/1999421293 >> v1:172.21.15.27:6789/0 conn(0x7fd34c0b6440 legacy=0x7fd34c162840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:30.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:30.672+0000 7fd350843640 1 -- 172.21.15.27:0/1999421293 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd34c0b7750 con 0x7fd34c0d3330 2024-09-12T03:21:30.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:30.672+0000 7fd34b7fe640 1 --2- 172.21.15.27:0/1999421293 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fd34c0d3810 0x7fd34c162f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:21:30.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:30.672+0000 7fd350843640 1 --2- 172.21.15.27:0/1999421293 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd34c0d3330 0x7fd34c0b7fc0 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7fd344009d90 tx=0x7fd34402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3809337467fe39f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:30.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:30.673+0000 7fd34affd640 1 -- 172.21.15.27:0/1999421293 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd344007740 con 0x7fd34c0d3330 2024-09-12T03:21:30.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:30.673+0000 7fd34affd640 1 -- 172.21.15.27:0/1999421293 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3440078e0 con 0x7fd34c0d3330 2024-09-12T03:21:30.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:30.673+0000 7fd34affd640 1 -- 172.21.15.27:0/1999421293 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd344007cb0 con 0x7fd34c0d3330 2024-09-12T03:21:30.678 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:30.673+0000 7fd3522a6640 1 -- 172.21.15.27:0/1999421293 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd34c0d3330 msgr2=0x7fd34c0b7fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:30.678 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:30.673+0000 7fd3522a6640 1 --2- 172.21.15.27:0/1999421293 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd34c0d3330 0x7fd34c0b7fc0 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7fd344009d90 tx=0x7fd34402a660 comp rx=0 tx=0).stop 2024-09-12T03:21:30.678 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:30.673+0000 7fd3522a6640 1 -- 172.21.15.27:0/1999421293 shutdown_connections 2024-09-12T03:21:30.678 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:30.673+0000 7fd3522a6640 1 --2- 172.21.15.27:0/1999421293 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd34c0d3330 0x7fd34c0b7fc0 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:30.678 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:30.673+0000 7fd3522a6640 1 --2- 172.21.15.27:0/1999421293 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fd34c0d3810 0x7fd34c162f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:30.679 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:30.673+0000 7fd3522a6640 1 -- 172.21.15.27:0/1999421293 >> 172.21.15.27:0/1999421293 conn(0x7fd34c0d11d0 msgr2=0x7fd34c0d15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:30.679 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:30.673+0000 7fd3522a6640 1 -- 172.21.15.27:0/1999421293 shutdown_connections 2024-09-12T03:21:30.679 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:30.673+0000 7fd3522a6640 1 -- 172.21.15.27:0/1999421293 wait complete. 2024-09-12T03:21:30.679 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:30.674+0000 7fd3522a6640 1 Processor -- start 2024-09-12T03:21:30.679 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:30.675+0000 7fd3522a6640 1 -- start start 2024-09-12T03:21:30.679 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:30.675+0000 7fd3522a6640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd34c0b6440 0x7fd34c1f8140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:30.679 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:30.675+0000 7fd3522a6640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fd34c0d3330 0x7fd34c15b050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:30.679 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:30.675+0000 7fd3522a6640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd34c0d3810 0x7fd34c15c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:30.679 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:30.675+0000 7fd3522a6640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fd34c163490 con 0x7fd34c0d3810 2024-09-12T03:21:30.679 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:30.675+0000 7fd3522a6640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fd34c0d3f70 con 0x7fd34c0b6440 2024-09-12T03:21:30.679 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:30.675+0000 7fd3522a6640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fd34c163610 con 0x7fd34c0d3330 2024-09-12T03:21:30.679 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:30.675+0000 7fd350843640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd34c0d3810 0x7fd34c15c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:30.680 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:30.676+0000 7fd350843640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd34c0d3810 0x7fd34c15c9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41788/0 (socket says 172.21.15.27:41788) 2024-09-12T03:21:30.680 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:30.676+0000 7fd350843640 1 -- 172.21.15.27:0/3270291871 learned_addr learned my addr 172.21.15.27:0/3270291871 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:21:30.680 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:30.676+0000 7fd34bfff640 1 --2- 172.21.15.27:0/3270291871 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd34c0b6440 0x7fd34c1f8140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:30.680 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:30.676+0000 7fd34b7fe640 1 --2- 172.21.15.27:0/3270291871 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fd34c0d3330 0x7fd34c15b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:30.680 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:30.676+0000 7fd350843640 1 -- 172.21.15.27:0/3270291871 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fd34c0d3330 msgr2=0x7fd34c15b050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:30.680 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:30.676+0000 7fd350843640 1 --2- 172.21.15.27:0/3270291871 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fd34c0d3330 0x7fd34c15b050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:30.680 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:30.676+0000 7fd350843640 1 -- 172.21.15.27:0/3270291871 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd34c0b6440 msgr2=0x7fd34c1f8140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:30.680 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:30.676+0000 7fd350843640 1 --2- 172.21.15.27:0/3270291871 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd34c0b6440 0x7fd34c1f8140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:30.680 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:30.676+0000 7fd350843640 1 -- 172.21.15.27:0/3270291871 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd34c0ce320 con 0x7fd34c0d3810 2024-09-12T03:21:30.680 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:30.676+0000 7fd350843640 1 --2- 172.21.15.27:0/3270291871 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd34c0d3810 0x7fd34c15c9d0 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7fd34402ac50 tx=0x7fd344004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:30.680 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:30.676+0000 7fd348ff9640 1 -- 172.21.15.27:0/3270291871 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd34403f070 con 0x7fd34c0d3810 2024-09-12T03:21:30.680 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:30.676+0000 7fd348ff9640 1 -- 172.21.15.27:0/3270291871 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd344031bc0 con 0x7fd34c0d3810 2024-09-12T03:21:30.680 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:30.676+0000 7fd3522a6640 1 -- 172.21.15.27:0/3270291871 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd34c0ce600 con 0x7fd34c0d3810 2024-09-12T03:21:30.681 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:30.676+0000 7fd348ff9640 1 -- 172.21.15.27:0/3270291871 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd344044460 con 0x7fd34c0d3810 2024-09-12T03:21:30.681 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:30.676+0000 7fd3522a6640 1 -- 172.21.15.27:0/3270291871 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd34c0cebc0 con 0x7fd34c0d3810 2024-09-12T03:21:30.681 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:30.677+0000 7fd3522a6640 1 -- 172.21.15.27:0/3270291871 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd34c0b6820 con 0x7fd34c0d3810 2024-09-12T03:21:30.681 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:30.678+0000 7fd348ff9640 1 -- 172.21.15.27:0/3270291871 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 18) ==== 90753+0+0 (secure 0 0 0) 0x7fd344007790 con 0x7fd34c0d3810 2024-09-12T03:21:30.681 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:30.678+0000 7fd348ff9640 1 --2- 172.21.15.27:0/3270291871 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fd33806c100 0x7fd33806e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:30.682 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:30.679+0000 7fd34bfff640 1 --2- 172.21.15.27:0/3270291871 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fd33806c100 0x7fd33806e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:30.682 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:30.679+0000 7fd348ff9640 1 -- 172.21.15.27:0/3270291871 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5767+0+0 (secure 0 0 0) 0x7fd3440acba0 con 0x7fd34c0d3810 2024-09-12T03:21:30.682 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:30.679+0000 7fd34bfff640 1 --2- 172.21.15.27:0/3270291871 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fd33806c100 0x7fd33806e5c0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fd34c0c7ed0 tx=0x7fd33c009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:30.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:30.683+0000 7fd348ff9640 1 -- 172.21.15.27:0/3270291871 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fd34407b140 con 0x7fd34c0d3810 2024-09-12T03:21:30.892 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:30.891+0000 7fd3522a6640 1 -- 172.21.15.27:0/3270291871 --> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "vol_name": "cephfs", "sub_name": "subvol84490", "snap_name": "subvol_snap39590", "target": ["mon-mgr", ""]}) -- 0x7fd34c0ba550 con 0x7fd33806c100 2024-09-12T03:21:30.918 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:30.918+0000 7fd348ff9640 1 -- 172.21.15.27:0/3270291871 <== mgr.4113 v2:172.21.15.27:6836/1226920526 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fd34c0ba550 con 0x7fd33806c100 2024-09-12T03:21:30.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:30.920+0000 7fd3522a6640 1 -- 172.21.15.27:0/3270291871 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fd33806c100 msgr2=0x7fd33806e5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:30.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:30.920+0000 7fd3522a6640 1 --2- 172.21.15.27:0/3270291871 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fd33806c100 0x7fd33806e5c0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fd34c0c7ed0 tx=0x7fd33c009040 comp rx=0 tx=0).stop 2024-09-12T03:21:30.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:30.920+0000 7fd3522a6640 1 -- 172.21.15.27:0/3270291871 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd34c0d3810 msgr2=0x7fd34c15c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:30.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:30.920+0000 7fd3522a6640 1 --2- 172.21.15.27:0/3270291871 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd34c0d3810 0x7fd34c15c9d0 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7fd34402ac50 tx=0x7fd344004a40 comp rx=0 tx=0).stop 2024-09-12T03:21:30.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:30.920+0000 7fd3522a6640 1 -- 172.21.15.27:0/3270291871 shutdown_connections 2024-09-12T03:21:30.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:30.920+0000 7fd3522a6640 1 --2- 172.21.15.27:0/3270291871 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fd33806c100 0x7fd33806e5c0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:30.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:30.920+0000 7fd3522a6640 1 --2- 172.21.15.27:0/3270291871 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd34c0d3810 0x7fd34c15c9d0 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:30.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:30.920+0000 7fd3522a6640 1 --2- 172.21.15.27:0/3270291871 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fd34c0d3330 0x7fd34c15b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:30.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:30.920+0000 7fd3522a6640 1 --2- 172.21.15.27:0/3270291871 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd34c0b6440 0x7fd34c1f8140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:30.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:30.920+0000 7fd3522a6640 1 -- 172.21.15.27:0/3270291871 >> 172.21.15.27:0/3270291871 conn(0x7fd34c0d11d0 msgr2=0x7fd34c0b91c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:30.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:30.920+0000 7fd3522a6640 1 -- 172.21.15.27:0/3270291871 shutdown_connections 2024-09-12T03:21:30.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:30.920+0000 7fd3522a6640 1 -- 172.21.15.27:0/3270291871 wait complete. 2024-09-12T03:21:30.931 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm cephfs subvol84490 fake 2024-09-12T03:21:31.038 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.037+0000 7f351f28d640 1 Processor -- start 2024-09-12T03:21:31.039 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.038+0000 7f351f28d640 1 -- start start 2024-09-12T03:21:31.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.038+0000 7f351f28d640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f351815e990 0x7f35180b7600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:31.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.038+0000 7f351f28d640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f35180b6230 0x7f35180b6690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:31.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.038+0000 7f351f28d640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f35180b3600 0x7f35180b3a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:31.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.038+0000 7f351f28d640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f35180b6bd0 con 0x7f35180b3600 2024-09-12T03:21:31.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.038+0000 7f351f28d640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f351815ed70 con 0x7f35180b6230 2024-09-12T03:21:31.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.038+0000 7f351f28d640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f35180b3fd0 con 0x7f351815e990 2024-09-12T03:21:31.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.039+0000 7f351d82a640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f35180b3600 0x7f35180b3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:31.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.039+0000 7f351d82a640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f35180b3600 0x7f35180b3a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41796/0 (socket says 172.21.15.27:41796) 2024-09-12T03:21:31.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.039+0000 7f351d82a640 1 -- 172.21.15.27:0/3228304492 learned_addr learned my addr 172.21.15.27:0/3228304492 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:21:31.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.039+0000 7f351d029640 1 --2- 172.21.15.27:0/3228304492 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f351815e990 0x7f35180b7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:31.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.039+0000 7f351c828640 1 --2- 172.21.15.27:0/3228304492 >> v2:172.21.15.45:3300/0 conn(0x7f35180b6230 0x7f35180b6690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:31.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.039+0000 7f351d82a640 1 -- 172.21.15.27:0/3228304492 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f351815e990 msgr2=0x7f35180b7600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:31.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.039+0000 7f351d82a640 1 --2- 172.21.15.27:0/3228304492 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f351815e990 0x7f35180b7600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:31.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.039+0000 7f351d82a640 1 -- 172.21.15.27:0/3228304492 >> v2:172.21.15.45:3300/0 conn(0x7f35180b6230 msgr2=0x7f35180b6690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:31.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.039+0000 7f351d82a640 1 --2- 172.21.15.27:0/3228304492 >> v2:172.21.15.45:3300/0 conn(0x7f35180b6230 0x7f35180b6690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:31.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.039+0000 7f351d82a640 1 -- 172.21.15.27:0/3228304492 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35180d2a30 con 0x7f35180b3600 2024-09-12T03:21:31.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.040+0000 7f351d82a640 1 --2- 172.21.15.27:0/3228304492 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f35180b3600 0x7f35180b3a90 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f351808d0e0 tx=0x7f351002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf7ad5c554e940e7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:31.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.040+0000 7f350ffff640 1 -- 172.21.15.27:0/3228304492 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3510036070 con 0x7f35180b3600 2024-09-12T03:21:31.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.040+0000 7f350ffff640 1 -- 172.21.15.27:0/3228304492 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3510031600 con 0x7f35180b3600 2024-09-12T03:21:31.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.040+0000 7f350ffff640 1 -- 172.21.15.27:0/3228304492 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35100319d0 con 0x7f35180b3600 2024-09-12T03:21:31.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.040+0000 7f351f28d640 1 -- 172.21.15.27:0/3228304492 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f35180b3600 msgr2=0x7f35180b3a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:31.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.040+0000 7f351f28d640 1 --2- 172.21.15.27:0/3228304492 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f35180b3600 0x7f35180b3a90 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f351808d0e0 tx=0x7f351002a6a0 comp rx=0 tx=0).stop 2024-09-12T03:21:31.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.040+0000 7f351f28d640 1 -- 172.21.15.27:0/3228304492 shutdown_connections 2024-09-12T03:21:31.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.040+0000 7f351f28d640 1 --2- 172.21.15.27:0/3228304492 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f35180b3600 0x7f35180b3a90 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:31.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.040+0000 7f351f28d640 1 --2- 172.21.15.27:0/3228304492 >> v2:172.21.15.45:3300/0 conn(0x7f35180b6230 0x7f35180b6690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:31.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.040+0000 7f351f28d640 1 --2- 172.21.15.27:0/3228304492 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f351815e990 0x7f35180b7600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:31.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.040+0000 7f351f28d640 1 -- 172.21.15.27:0/3228304492 >> 172.21.15.27:0/3228304492 conn(0x7f35180d1230 msgr2=0x7f35180d1640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:31.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.040+0000 7f351f28d640 1 -- 172.21.15.27:0/3228304492 shutdown_connections 2024-09-12T03:21:31.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.041+0000 7f351f28d640 1 -- 172.21.15.27:0/3228304492 wait complete. 2024-09-12T03:21:31.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.042+0000 7f351f28d640 1 Processor -- start 2024-09-12T03:21:31.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.042+0000 7f351f28d640 1 -- start start 2024-09-12T03:21:31.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.042+0000 7f351f28d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35180b3600 0x7f35181f6ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:31.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.042+0000 7f351f28d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f35180b6230 0x7f35180b8630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:31.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.042+0000 7f351f28d640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f351815e990 0x7f35180b8b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:31.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.042+0000 7f351f28d640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f35180d32d0 con 0x7f35180b6230 2024-09-12T03:21:31.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.042+0000 7f351f28d640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f35180d2ff0 con 0x7f35180b3600 2024-09-12T03:21:31.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.042+0000 7f351f28d640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f35180d35b0 con 0x7f351815e990 2024-09-12T03:21:31.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.043+0000 7f351c828640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f35180b6230 0x7f35180b8630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:31.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.043+0000 7f351c828640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f35180b6230 0x7f35180b8630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41810/0 (socket says 172.21.15.27:41810) 2024-09-12T03:21:31.048 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.043+0000 7f351c828640 1 -- 172.21.15.27:0/3863133642 learned_addr learned my addr 172.21.15.27:0/3863133642 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:21:31.048 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.043+0000 7f351d029640 1 --2- 172.21.15.27:0/3863133642 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35180b3600 0x7f35181f6ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:31.048 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.043+0000 7f351d82a640 1 --2- 172.21.15.27:0/3863133642 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f351815e990 0x7f35180b8b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:31.048 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.043+0000 7f351c828640 1 -- 172.21.15.27:0/3863133642 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f351815e990 msgr2=0x7f35180b8b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:31.048 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.043+0000 7f351c828640 1 --2- 172.21.15.27:0/3863133642 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f351815e990 0x7f35180b8b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:31.048 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.043+0000 7f351c828640 1 -- 172.21.15.27:0/3863133642 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35180b3600 msgr2=0x7f35181f6ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:31.048 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.043+0000 7f351c828640 1 --2- 172.21.15.27:0/3863133642 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35180b3600 0x7f35181f6ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:31.048 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.043+0000 7f351c828640 1 -- 172.21.15.27:0/3863133642 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f351815fc00 con 0x7f35180b6230 2024-09-12T03:21:31.048 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.043+0000 7f351c828640 1 --2- 172.21.15.27:0/3863133642 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f35180b6230 0x7f35180b8630 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f350800c7f0 tx=0x7f350800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:31.048 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.043+0000 7f350dffb640 1 -- 172.21.15.27:0/3863133642 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3508015070 con 0x7f35180b6230 2024-09-12T03:21:31.048 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.043+0000 7f350dffb640 1 -- 172.21.15.27:0/3863133642 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3508004220 con 0x7f35180b6230 2024-09-12T03:21:31.048 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.043+0000 7f351f28d640 1 -- 172.21.15.27:0/3863133642 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3518160430 con 0x7f35180b6230 2024-09-12T03:21:31.048 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.043+0000 7f350dffb640 1 -- 172.21.15.27:0/3863133642 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3508007620 con 0x7f35180b6230 2024-09-12T03:21:31.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.044+0000 7f351f28d640 1 -- 172.21.15.27:0/3863133642 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35181609f0 con 0x7f35180b6230 2024-09-12T03:21:31.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.044+0000 7f351f28d640 1 -- 172.21.15.27:0/3863133642 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35180b3a90 con 0x7f35180b6230 2024-09-12T03:21:31.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.045+0000 7f350dffb640 1 -- 172.21.15.27:0/3863133642 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 18) ==== 90753+0+0 (secure 0 0 0) 0x7f35080079d0 con 0x7f35180b6230 2024-09-12T03:21:31.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.045+0000 7f350dffb640 1 --2- 172.21.15.27:0/3863133642 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f34fc06c130 0x7f34fc06e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:31.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.046+0000 7f350dffb640 1 -- 172.21.15.27:0/3863133642 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5767+0+0 (secure 0 0 0) 0x7f350808b370 con 0x7f35180b6230 2024-09-12T03:21:31.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.046+0000 7f351d029640 1 --2- 172.21.15.27:0/3863133642 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f34fc06c130 0x7f34fc06e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:31.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.046+0000 7f351d029640 1 --2- 172.21.15.27:0/3863133642 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f34fc06c130 0x7f34fc06e5f0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f351400aac0 tx=0x7f3514008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:31.050 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.050+0000 7f350dffb640 1 -- 172.21.15.27:0/3863133642 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f3508059230 con 0x7f35180b6230 2024-09-12T03:21:31.067 DEBUG:teuthology.orchestra.run.smithi027:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-12T03:21:31.069 DEBUG:teuthology.orchestra.run.smithi045:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-12T03:21:31.258 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.257+0000 7f351f28d640 1 -- 172.21.15.27:0/3863133642 --> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "vol_name": "cephfs", "sub_name": "subvol84490", "snap_name": "fake", "target": ["mon-mgr", ""]}) -- 0x7f351806e4e0 con 0x7f34fc06c130 2024-09-12T03:21:31.287 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.286+0000 7f350dffb640 1 -- 172.21.15.27:0/3863133642 <== mgr.4113 v2:172.21.15.27:6836/1226920526 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f351806e4e0 con 0x7f34fc06c130 2024-09-12T03:21:31.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.288+0000 7f351f28d640 1 -- 172.21.15.27:0/3863133642 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f34fc06c130 msgr2=0x7f34fc06e5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:31.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.288+0000 7f351f28d640 1 --2- 172.21.15.27:0/3863133642 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f34fc06c130 0x7f34fc06e5f0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f351400aac0 tx=0x7f3514008040 comp rx=0 tx=0).stop 2024-09-12T03:21:31.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.288+0000 7f351f28d640 1 -- 172.21.15.27:0/3863133642 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f35180b6230 msgr2=0x7f35180b8630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:31.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.288+0000 7f351f28d640 1 --2- 172.21.15.27:0/3863133642 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f35180b6230 0x7f35180b8630 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f350800c7f0 tx=0x7f350800ccb0 comp rx=0 tx=0).stop 2024-09-12T03:21:31.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.288+0000 7f351f28d640 1 -- 172.21.15.27:0/3863133642 shutdown_connections 2024-09-12T03:21:31.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.288+0000 7f351f28d640 1 --2- 172.21.15.27:0/3863133642 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f351815e990 0x7f35180b8b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:31.290 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.288+0000 7f351f28d640 1 --2- 172.21.15.27:0/3863133642 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f34fc06c130 0x7f34fc06e5f0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:31.290 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.288+0000 7f351f28d640 1 --2- 172.21.15.27:0/3863133642 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f35180b6230 0x7f35180b8630 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:31.290 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.288+0000 7f351f28d640 1 --2- 172.21.15.27:0/3863133642 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35180b3600 0x7f35181f6ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:31.290 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.288+0000 7f351f28d640 1 -- 172.21.15.27:0/3863133642 >> 172.21.15.27:0/3863133642 conn(0x7f35180d1230 msgr2=0x7f351815c1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:31.290 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.288+0000 7f351f28d640 1 -- 172.21.15.27:0/3863133642 shutdown_connections 2024-09-12T03:21:31.290 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.288+0000 7f351f28d640 1 -- 172.21.15.27:0/3863133642 wait complete. 2024-09-12T03:21:31.299 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:21:31.299 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cat ./volumes/_nogroup/subvol84490/.meta) 2024-09-12T03:21:31.326 INFO:teuthology.orchestra.run.smithi027.stdout:[GLOBAL] 2024-09-12T03:21:31.326 INFO:teuthology.orchestra.run.smithi027.stdout:version = 1 2024-09-12T03:21:31.326 INFO:teuthology.orchestra.run.smithi027.stdout:type = subvolume 2024-09-12T03:21:31.326 INFO:teuthology.orchestra.run.smithi027.stdout:path = /volumes/_nogroup/subvol84490/a842cae4-61b6-4f73-a96f-03c1581e2365 2024-09-12T03:21:31.326 INFO:teuthology.orchestra.run.smithi027.stdout:state = complete 2024-09-12T03:21:31.326 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:21:31.327 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol84490 2024-09-12T03:21:31.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.473+0000 7f3a8ad86640 1 Processor -- start 2024-09-12T03:21:31.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.473+0000 7f3a8ad86640 1 -- start start 2024-09-12T03:21:31.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.474+0000 7f3a8ad86640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f3a84162040 0x7f3a840d1da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:31.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.474+0000 7f3a8ad86640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f3a840d3790 0x7f3a840b7a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:31.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.474+0000 7f3a8ad86640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3a840d2e30 con 0x7f3a8406d8c0 2024-09-12T03:21:31.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.474+0000 7f3a8ad86640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f3a840b7f90 con 0x7f3a840d3790 2024-09-12T03:21:31.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.474+0000 7f3a8ad86640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f3a84162420 con 0x7f3a84162040 2024-09-12T03:21:31.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.474+0000 7f3a83fff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f3a840d3790 0x7f3a840b7a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:31.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.474+0000 7f3a89323640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f3a8406d8c0 0x7f3a840cf320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:51816/0 (socket says 172.21.15.27:51816) 2024-09-12T03:21:31.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.474+0000 7f3a89323640 1 -- 172.21.15.27:0/2429081189 learned_addr learned my addr 172.21.15.27:0/2429081189 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:21:31.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.474+0000 7f3a88b22640 1 --2- 172.21.15.27:0/2429081189 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f3a84162040 0x7f3a840d1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:31.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.474+0000 7f3a83fff640 1 -- 172.21.15.27:0/2429081189 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f3a84162040 msgr2=0x7f3a840d1da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:31.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.474+0000 7f3a83fff640 1 --2- 172.21.15.27:0/2429081189 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f3a84162040 0x7f3a840d1da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:31.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.474+0000 7f3a83fff640 1 -- 172.21.15.27:0/2429081189 >> v1:172.21.15.27:6789/0 conn(0x7f3a8406d8c0 legacy=0x7f3a840cf320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:31.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.474+0000 7f3a83fff640 1 -- 172.21.15.27:0/2429081189 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a840d30d0 con 0x7f3a840d3790 2024-09-12T03:21:31.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.475+0000 7f3a83fff640 1 --2- 172.21.15.27:0/2429081189 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3a840d3790 0x7f3a840b7a50 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f3a700097a0 tx=0x7f3a7002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49dc3bd60c492038 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:31.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.475+0000 7f3a837fe640 1 -- 172.21.15.27:0/2429081189 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a70037070 con 0x7f3a840d3790 2024-09-12T03:21:31.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.475+0000 7f3a837fe640 1 -- 172.21.15.27:0/2429081189 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a70032440 con 0x7f3a840d3790 2024-09-12T03:21:31.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.475+0000 7f3a837fe640 1 -- 172.21.15.27:0/2429081189 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a70032810 con 0x7f3a840d3790 2024-09-12T03:21:31.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.475+0000 7f3a8ad86640 1 -- 172.21.15.27:0/2429081189 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3a840d3790 msgr2=0x7f3a840b7a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:31.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.475+0000 7f3a8ad86640 1 --2- 172.21.15.27:0/2429081189 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3a840d3790 0x7f3a840b7a50 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f3a700097a0 tx=0x7f3a7002a180 comp rx=0 tx=0).stop 2024-09-12T03:21:31.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.476+0000 7f3a8ad86640 1 -- 172.21.15.27:0/2429081189 shutdown_connections 2024-09-12T03:21:31.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.476+0000 7f3a8ad86640 1 --2- 172.21.15.27:0/2429081189 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3a840d3790 0x7f3a840b7a50 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:31.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.476+0000 7f3a8ad86640 1 --2- 172.21.15.27:0/2429081189 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f3a84162040 0x7f3a840d1da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:31.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.476+0000 7f3a8ad86640 1 -- 172.21.15.27:0/2429081189 >> 172.21.15.27:0/2429081189 conn(0x7f3a840b6700 msgr2=0x7f3a840b6b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:31.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.476+0000 7f3a8ad86640 1 -- 172.21.15.27:0/2429081189 shutdown_connections 2024-09-12T03:21:31.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.476+0000 7f3a8ad86640 1 -- 172.21.15.27:0/2429081189 wait complete. 2024-09-12T03:21:31.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.477+0000 7f3a8ad86640 1 Processor -- start 2024-09-12T03:21:31.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.477+0000 7f3a8ad86640 1 -- start start 2024-09-12T03:21:31.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.478+0000 7f3a8ad86640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f3a8406d8c0 0x7f3a84148620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:31.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.478+0000 7f3a8ad86640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3a840d3790 0x7f3a841597a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:31.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.478+0000 7f3a8ad86640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3a84162040 0x7f3a84159ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:31.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.478+0000 7f3a8ad86640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f3a840d4530 con 0x7f3a840d3790 2024-09-12T03:21:31.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.478+0000 7f3a8ad86640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f3a840cee00 con 0x7f3a84162040 2024-09-12T03:21:31.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.478+0000 7f3a8ad86640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f3a840d4150 con 0x7f3a8406d8c0 2024-09-12T03:21:31.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.478+0000 7f3a83fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3a840d3790 0x7f3a841597a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:31.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.478+0000 7f3a83fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3a840d3790 0x7f3a841597a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41828/0 (socket says 172.21.15.27:41828) 2024-09-12T03:21:31.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.478+0000 7f3a83fff640 1 -- 172.21.15.27:0/2545677449 learned_addr learned my addr 172.21.15.27:0/2545677449 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:21:31.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.478+0000 7f3a88b22640 1 --2- 172.21.15.27:0/2545677449 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f3a8406d8c0 0x7f3a84148620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:31.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.478+0000 7f3a89323640 1 --2- 172.21.15.27:0/2545677449 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3a84162040 0x7f3a84159ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:31.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.478+0000 7f3a83fff640 1 -- 172.21.15.27:0/2545677449 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f3a8406d8c0 msgr2=0x7f3a84148620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:31.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.478+0000 7f3a83fff640 1 --2- 172.21.15.27:0/2545677449 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f3a8406d8c0 0x7f3a84148620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:31.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.478+0000 7f3a83fff640 1 -- 172.21.15.27:0/2545677449 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3a84162040 msgr2=0x7f3a84159ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:31.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.478+0000 7f3a83fff640 1 --2- 172.21.15.27:0/2545677449 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3a84162040 0x7f3a84159ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:31.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.478+0000 7f3a83fff640 1 -- 172.21.15.27:0/2545677449 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a841583e0 con 0x7f3a840d3790 2024-09-12T03:21:31.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.478+0000 7f3a83fff640 1 --2- 172.21.15.27:0/2545677449 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3a840d3790 0x7f3a841597a0 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f3a70000c00 tx=0x7f3a700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:31.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.478+0000 7f3a817fa640 1 -- 172.21.15.27:0/2545677449 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a70041070 con 0x7f3a840d3790 2024-09-12T03:21:31.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.479+0000 7f3a8ad86640 1 -- 172.21.15.27:0/2545677449 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a8415a310 con 0x7f3a840d3790 2024-09-12T03:21:31.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.479+0000 7f3a817fa640 1 -- 172.21.15.27:0/2545677449 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a7003c930 con 0x7f3a840d3790 2024-09-12T03:21:31.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.479+0000 7f3a817fa640 1 -- 172.21.15.27:0/2545677449 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a70037040 con 0x7f3a840d3790 2024-09-12T03:21:31.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.479+0000 7f3a8ad86640 1 -- 172.21.15.27:0/2545677449 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a8415a8d0 con 0x7f3a840d3790 2024-09-12T03:21:31.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.480+0000 7f3a8ad86640 1 -- 172.21.15.27:0/2545677449 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a840cf040 con 0x7f3a840d3790 2024-09-12T03:21:31.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.480+0000 7f3a817fa640 1 -- 172.21.15.27:0/2545677449 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 18) ==== 90753+0+0 (secure 0 0 0) 0x7f3a70044070 con 0x7f3a840d3790 2024-09-12T03:21:31.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.481+0000 7f3a817fa640 1 --2- 172.21.15.27:0/2545677449 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f3a6806c100 0x7f3a6806e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:31.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.481+0000 7f3a817fa640 1 -- 172.21.15.27:0/2545677449 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5767+0+0 (secure 0 0 0) 0x7f3a700abf70 con 0x7f3a840d3790 2024-09-12T03:21:31.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.481+0000 7f3a88b22640 1 --2- 172.21.15.27:0/2545677449 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f3a6806c100 0x7f3a6806e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:31.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.481+0000 7f3a88b22640 1 --2- 172.21.15.27:0/2545677449 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f3a6806c100 0x7f3a6806e5c0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f3a78006fd0 tx=0x7f3a78008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:31.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.485+0000 7f3a817fa640 1 -- 172.21.15.27:0/2545677449 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f3a840cf040 con 0x7f3a840d3790 2024-09-12T03:21:31.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.687+0000 7f3a8ad86640 1 -- 172.21.15.27:0/2545677449 --> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol84490", "target": ["mon-mgr", ""]}) -- 0x7f3a840d0180 con 0x7f3a6806c100 2024-09-12T03:21:31.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.722+0000 7f3a817fa640 1 -- 172.21.15.27:0/2545677449 <== mgr.4113 v2:172.21.15.27:6836/1226920526 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f3a840d0180 con 0x7f3a6806c100 2024-09-12T03:21:31.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.725+0000 7f3a8ad86640 1 -- 172.21.15.27:0/2545677449 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f3a6806c100 msgr2=0x7f3a6806e5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:31.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.725+0000 7f3a8ad86640 1 --2- 172.21.15.27:0/2545677449 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f3a6806c100 0x7f3a6806e5c0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f3a78006fd0 tx=0x7f3a78008040 comp rx=0 tx=0).stop 2024-09-12T03:21:31.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.725+0000 7f3a8ad86640 1 -- 172.21.15.27:0/2545677449 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3a840d3790 msgr2=0x7f3a841597a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:31.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.725+0000 7f3a8ad86640 1 --2- 172.21.15.27:0/2545677449 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3a840d3790 0x7f3a841597a0 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f3a70000c00 tx=0x7f3a700046c0 comp rx=0 tx=0).stop 2024-09-12T03:21:31.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.725+0000 7f3a8ad86640 1 -- 172.21.15.27:0/2545677449 shutdown_connections 2024-09-12T03:21:31.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.725+0000 7f3a8ad86640 1 --2- 172.21.15.27:0/2545677449 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3a84162040 0x7f3a84159ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:31.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.725+0000 7f3a8ad86640 1 --2- 172.21.15.27:0/2545677449 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f3a6806c100 0x7f3a6806e5c0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:31.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.725+0000 7f3a8ad86640 1 --2- 172.21.15.27:0/2545677449 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3a840d3790 0x7f3a841597a0 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:31.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.725+0000 7f3a8ad86640 1 --2- 172.21.15.27:0/2545677449 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f3a8406d8c0 0x7f3a84148620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:31.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.725+0000 7f3a8ad86640 1 -- 172.21.15.27:0/2545677449 >> 172.21.15.27:0/2545677449 conn(0x7f3a840b6700 msgr2=0x7f3a840cc1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:31.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.725+0000 7f3a8ad86640 1 -- 172.21.15.27:0/2545677449 shutdown_connections 2024-09-12T03:21:31.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.725+0000 7f3a8ad86640 1 -- 172.21.15.27:0/2545677449 wait complete. 2024-09-12T03:21:31.727 INFO:tasks.ceph.mgr.y.smithi027.stderr:2024-09-12T03:21:31.726+0000 7f32608e9640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:21:31.727 INFO:tasks.ceph.mgr.y.smithi027.stderr:2024-09-12T03:21:31.726+0000 7f32608e9640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:21:31.727 INFO:tasks.ceph.mgr.y.smithi027.stderr:2024-09-12T03:21:31.726+0000 7f32608e9640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:21:31.727 INFO:tasks.ceph.mgr.y.smithi027.stderr:2024-09-12T03:21:31.726+0000 7f32608e9640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:21:31.727 INFO:tasks.ceph.mgr.y.smithi027.stderr:2024-09-12T03:21:31.726+0000 7f32608e9640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:21:31.739 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol_clone8340 2024-09-12T03:21:31.743 INFO:tasks.ceph.mgr.y.smithi027.stderr:2024-09-12T03:21:31.742+0000 7f325f8e7640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:21:31.743 INFO:tasks.ceph.mgr.y.smithi027.stderr:2024-09-12T03:21:31.742+0000 7f325f8e7640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:21:31.743 INFO:tasks.ceph.mgr.y.smithi027.stderr:2024-09-12T03:21:31.742+0000 7f325f8e7640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:21:31.743 INFO:tasks.ceph.mgr.y.smithi027.stderr:2024-09-12T03:21:31.742+0000 7f325f8e7640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:21:31.743 INFO:tasks.ceph.mgr.y.smithi027.stderr:2024-09-12T03:21:31.742+0000 7f325f8e7640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:21:31.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.851+0000 7f129c6e6640 1 Processor -- start 2024-09-12T03:21:31.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.851+0000 7f129c6e6640 1 -- start start 2024-09-12T03:21:31.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.852+0000 7f129c6e6640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f129406dfd0 0x7f12940d1da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:31.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.852+0000 7f129c6e6640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f12940d3790 0x7f12940b7a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:31.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.852+0000 7f129c6e6640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f12940d2e30 con 0x7f1294162040 2024-09-12T03:21:31.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.852+0000 7f129c6e6640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f12940b7f90 con 0x7f12940d3790 2024-09-12T03:21:31.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.852+0000 7f129c6e6640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f1294162420 con 0x7f129406dfd0 2024-09-12T03:21:31.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.852+0000 7f129a482640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f1294162040 0x7f129406d8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:51830/0 (socket says 172.21.15.27:51830) 2024-09-12T03:21:31.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.852+0000 7f129ac83640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f12940d3790 0x7f12940b7a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:31.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.852+0000 7f129a482640 1 -- 172.21.15.27:0/3311542780 learned_addr learned my addr 172.21.15.27:0/3311542780 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:21:31.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.853+0000 7f1299480640 1 -- 172.21.15.27:0/3311542780 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1992006905 0 0) 0x7f12940d2e30 con 0x7f1294162040 2024-09-12T03:21:31.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.853+0000 7f1299480640 1 -- 172.21.15.27:0/3311542780 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f127c003520 con 0x7f1294162040 2024-09-12T03:21:31.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.853+0000 7f129ac83640 1 -- 172.21.15.27:0/3311542780 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f129406dfd0 msgr2=0x7f12940d1da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:31.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.853+0000 7f1299c81640 1 --2- 172.21.15.27:0/3311542780 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f129406dfd0 0x7f12940d1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:31.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.853+0000 7f129ac83640 1 --2- 172.21.15.27:0/3311542780 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f129406dfd0 0x7f12940d1da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:31.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.853+0000 7f129ac83640 1 -- 172.21.15.27:0/3311542780 >> v1:172.21.15.27:6789/0 conn(0x7f1294162040 legacy=0x7f129406d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:31.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.853+0000 7f129ac83640 1 -- 172.21.15.27:0/3311542780 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12940d30d0 con 0x7f12940d3790 2024-09-12T03:21:31.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.853+0000 7f129ac83640 1 --2- 172.21.15.27:0/3311542780 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f12940d3790 0x7f12940b7a50 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f1284009d90 tx=0x7f128402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b035f2978abe683b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:31.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.854+0000 7f1299480640 1 -- 172.21.15.27:0/3311542780 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1284007740 con 0x7f12940d3790 2024-09-12T03:21:31.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.854+0000 7f1299480640 1 -- 172.21.15.27:0/3311542780 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f12840078e0 con 0x7f12940d3790 2024-09-12T03:21:31.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.854+0000 7f1299480640 1 -- 172.21.15.27:0/3311542780 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1284007cb0 con 0x7f12940d3790 2024-09-12T03:21:31.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.854+0000 7f129c6e6640 1 -- 172.21.15.27:0/3311542780 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f12940d3790 msgr2=0x7f12940b7a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:31.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.854+0000 7f129c6e6640 1 --2- 172.21.15.27:0/3311542780 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f12940d3790 0x7f12940b7a50 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f1284009d90 tx=0x7f128402a660 comp rx=0 tx=0).stop 2024-09-12T03:21:31.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.854+0000 7f129c6e6640 1 -- 172.21.15.27:0/3311542780 shutdown_connections 2024-09-12T03:21:31.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.854+0000 7f129c6e6640 1 --2- 172.21.15.27:0/3311542780 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f12940d3790 0x7f12940b7a50 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:31.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.854+0000 7f129c6e6640 1 --2- 172.21.15.27:0/3311542780 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f129406dfd0 0x7f12940d1da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:31.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.854+0000 7f129c6e6640 1 -- 172.21.15.27:0/3311542780 >> 172.21.15.27:0/3311542780 conn(0x7f12940b6700 msgr2=0x7f12940b6b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:31.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.854+0000 7f129c6e6640 1 -- 172.21.15.27:0/3311542780 shutdown_connections 2024-09-12T03:21:31.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.854+0000 7f129c6e6640 1 -- 172.21.15.27:0/3311542780 wait complete. 2024-09-12T03:21:31.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.855+0000 7f129c6e6640 1 Processor -- start 2024-09-12T03:21:31.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.856+0000 7f129c6e6640 1 -- start start 2024-09-12T03:21:31.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.856+0000 7f129c6e6640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f129406dfd0 0x7f1294157c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:31.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.856+0000 7f129c6e6640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f12940d3790 0x7f12940ccb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:31.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.856+0000 7f129c6e6640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1294162040 0x7f12940cd050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:31.862 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.856+0000 7f129c6e6640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f12940d4530 con 0x7f12940d3790 2024-09-12T03:21:31.862 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.856+0000 7f129c6e6640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f12940cee00 con 0x7f129406dfd0 2024-09-12T03:21:31.862 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.856+0000 7f129c6e6640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f12940d4150 con 0x7f1294162040 2024-09-12T03:21:31.862 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.856+0000 7f1299c81640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f12940d3790 0x7f12940ccb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:31.862 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.856+0000 7f1299c81640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f12940d3790 0x7f12940ccb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41870/0 (socket says 172.21.15.27:41870) 2024-09-12T03:21:31.862 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.856+0000 7f1299c81640 1 -- 172.21.15.27:0/4102673455 learned_addr learned my addr 172.21.15.27:0/4102673455 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:21:31.862 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.857+0000 7f129ac83640 1 --2- 172.21.15.27:0/4102673455 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1294162040 0x7f12940cd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:31.862 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.857+0000 7f129a482640 1 --2- 172.21.15.27:0/4102673455 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f129406dfd0 0x7f1294157c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:31.862 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.857+0000 7f1299c81640 1 -- 172.21.15.27:0/4102673455 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1294162040 msgr2=0x7f12940cd050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:31.862 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.857+0000 7f1299c81640 1 --2- 172.21.15.27:0/4102673455 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1294162040 0x7f12940cd050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:31.862 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.857+0000 7f1299c81640 1 -- 172.21.15.27:0/4102673455 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f129406dfd0 msgr2=0x7f1294157c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:31.862 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.857+0000 7f1299c81640 1 --2- 172.21.15.27:0/4102673455 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f129406dfd0 0x7f1294157c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:31.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.857+0000 7f1299c81640 1 -- 172.21.15.27:0/4102673455 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f129415e460 con 0x7f12940d3790 2024-09-12T03:21:31.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.857+0000 7f129a482640 1 --2- 172.21.15.27:0/4102673455 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f129406dfd0 0x7f1294157c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:21:31.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.857+0000 7f1299c81640 1 --2- 172.21.15.27:0/4102673455 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f12940d3790 0x7f12940ccb10 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f128800c7f0 tx=0x7f128800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:31.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.857+0000 7f1292ffd640 1 -- 172.21.15.27:0/4102673455 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1288015070 con 0x7f12940d3790 2024-09-12T03:21:31.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.857+0000 7f1292ffd640 1 -- 172.21.15.27:0/4102673455 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1288004220 con 0x7f12940d3790 2024-09-12T03:21:31.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.857+0000 7f129c6e6640 1 -- 172.21.15.27:0/4102673455 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1294161690 con 0x7f12940d3790 2024-09-12T03:21:31.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.857+0000 7f1292ffd640 1 -- 172.21.15.27:0/4102673455 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1288007620 con 0x7f12940d3790 2024-09-12T03:21:31.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.857+0000 7f129c6e6640 1 -- 172.21.15.27:0/4102673455 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12941589f0 con 0x7f12940d3790 2024-09-12T03:21:31.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.858+0000 7f129c6e6640 1 -- 172.21.15.27:0/4102673455 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f129406e6e0 con 0x7f12940d3790 2024-09-12T03:21:31.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.858+0000 7f1292ffd640 1 -- 172.21.15.27:0/4102673455 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 18) ==== 90753+0+0 (secure 0 0 0) 0x7f12880079d0 con 0x7f12940d3790 2024-09-12T03:21:31.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.859+0000 7f1292ffd640 1 --2- 172.21.15.27:0/4102673455 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f128006c100 0x7f128006e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:31.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.859+0000 7f129a482640 1 --2- 172.21.15.27:0/4102673455 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f128006c100 0x7f128006e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:31.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.859+0000 7f1292ffd640 1 -- 172.21.15.27:0/4102673455 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5767+0+0 (secure 0 0 0) 0x7f128808b720 con 0x7f12940d3790 2024-09-12T03:21:31.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.860+0000 7f129a482640 1 --2- 172.21.15.27:0/4102673455 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f128006c100 0x7f128006e5c0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f128c005880 tx=0x7f128c009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:31.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:31.864+0000 7f1292ffd640 1 -- 172.21.15.27:0/4102673455 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f1288059c30 con 0x7f12940d3790 2024-09-12T03:21:32.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:32.063+0000 7f129c6e6640 1 -- 172.21.15.27:0/4102673455 --> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol_clone8340", "target": ["mon-mgr", ""]}) -- 0x7f12940b45d0 con 0x7f128006c100 2024-09-12T03:21:32.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:32.078+0000 7f1292ffd640 1 -- 172.21.15.27:0/4102673455 <== mgr.4113 v2:172.21.15.27:6836/1226920526 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f12940b45d0 con 0x7f128006c100 2024-09-12T03:21:32.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:32.080+0000 7f129c6e6640 1 -- 172.21.15.27:0/4102673455 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f128006c100 msgr2=0x7f128006e5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:32.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:32.080+0000 7f129c6e6640 1 --2- 172.21.15.27:0/4102673455 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f128006c100 0x7f128006e5c0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f128c005880 tx=0x7f128c009040 comp rx=0 tx=0).stop 2024-09-12T03:21:32.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:32.081+0000 7f129c6e6640 1 -- 172.21.15.27:0/4102673455 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f12940d3790 msgr2=0x7f12940ccb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:32.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:32.081+0000 7f129c6e6640 1 --2- 172.21.15.27:0/4102673455 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f12940d3790 0x7f12940ccb10 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f128800c7f0 tx=0x7f128800ccb0 comp rx=0 tx=0).stop 2024-09-12T03:21:32.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:32.081+0000 7f129c6e6640 1 -- 172.21.15.27:0/4102673455 shutdown_connections 2024-09-12T03:21:32.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:32.081+0000 7f129c6e6640 1 --2- 172.21.15.27:0/4102673455 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1294162040 0x7f12940cd050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:32.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:32.081+0000 7f129c6e6640 1 --2- 172.21.15.27:0/4102673455 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f128006c100 0x7f128006e5c0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:32.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:32.081+0000 7f129c6e6640 1 --2- 172.21.15.27:0/4102673455 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f12940d3790 0x7f12940ccb10 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:32.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:32.081+0000 7f129c6e6640 1 --2- 172.21.15.27:0/4102673455 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f129406dfd0 0x7f1294157c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:32.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:32.081+0000 7f129c6e6640 1 -- 172.21.15.27:0/4102673455 >> 172.21.15.27:0/4102673455 conn(0x7f12940b6700 msgr2=0x7f12940b9c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:32.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:32.081+0000 7f129c6e6640 1 -- 172.21.15.27:0/4102673455 shutdown_connections 2024-09-12T03:21:32.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:32.081+0000 7f129c6e6640 1 -- 172.21.15.27:0/4102673455 wait complete. 2024-09-12T03:21:32.094 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol_clone64301 2024-09-12T03:21:32.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:32.242+0000 7f6a5596f640 1 Processor -- start 2024-09-12T03:21:32.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:32.242+0000 7f6a5596f640 1 -- start start 2024-09-12T03:21:32.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:32.242+0000 7f6a5596f640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f6a5015e900 0x7f6a500af370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:32.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:32.242+0000 7f6a5596f640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f6a5006d730 0x7f6a500b5e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:32.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:32.242+0000 7f6a5596f640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6a500b5940 con 0x7f6a500b6370 2024-09-12T03:21:32.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:32.242+0000 7f6a5596f640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f6a5015ece0 con 0x7f6a5006d730 2024-09-12T03:21:32.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:32.242+0000 7f6a5596f640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f6a500b67a0 con 0x7f6a5015e900 2024-09-12T03:21:32.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:32.243+0000 7f6a4e7fc640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f6a5006d730 0x7f6a500b5e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:32.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:32.243+0000 7f6a4f7fe640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f6a500b6370 0x7f6a500b4600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:51840/0 (socket says 172.21.15.27:51840) 2024-09-12T03:21:32.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:32.243+0000 7f6a4e7fc640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f6a5006d730 0x7f6a500b5e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41874/0 (socket says 172.21.15.27:41874) 2024-09-12T03:21:32.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:32.243+0000 7f6a4f7fe640 1 -- 172.21.15.27:0/2476864688 learned_addr learned my addr 172.21.15.27:0/2476864688 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:21:32.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:32.243+0000 7f6a4effd640 1 --2- 172.21.15.27:0/2476864688 >> v2:172.21.15.45:3300/0 conn(0x7f6a5015e900 0x7f6a500af370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:32.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:32.243+0000 7f6a4e7fc640 1 -- 172.21.15.27:0/2476864688 >> v2:172.21.15.45:3300/0 conn(0x7f6a5015e900 msgr2=0x7f6a500af370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:32.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:32.243+0000 7f6a4e7fc640 1 --2- 172.21.15.27:0/2476864688 >> v2:172.21.15.45:3300/0 conn(0x7f6a5015e900 0x7f6a500af370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:32.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:32.243+0000 7f6a4dffb640 1 -- 172.21.15.27:0/2476864688 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3099343658 0 0) 0x7f6a48001490 con 0x7f6a500b6370 2024-09-12T03:21:32.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:32.243+0000 7f6a4e7fc640 1 -- 172.21.15.27:0/2476864688 >> v1:172.21.15.27:6789/0 conn(0x7f6a500b6370 legacy=0x7f6a500b4600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:32.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:32.243+0000 7f6a4e7fc640 1 -- 172.21.15.27:0/2476864688 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a500af950 con 0x7f6a5006d730 2024-09-12T03:21:32.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:32.244+0000 7f6a4e7fc640 1 --2- 172.21.15.27:0/2476864688 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6a5006d730 0x7f6a500b5e30 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f6a50051020 tx=0x7f6a4402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b24e027f75e0387 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:32.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:32.244+0000 7f6a4dffb640 1 -- 172.21.15.27:0/2476864688 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a44037070 con 0x7f6a5006d730 2024-09-12T03:21:32.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:32.244+0000 7f6a4dffb640 1 -- 172.21.15.27:0/2476864688 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a44032440 con 0x7f6a5006d730 2024-09-12T03:21:32.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:32.244+0000 7f6a4dffb640 1 -- 172.21.15.27:0/2476864688 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a44032810 con 0x7f6a5006d730 2024-09-12T03:21:32.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:32.244+0000 7f6a5596f640 1 -- 172.21.15.27:0/2476864688 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6a5006d730 msgr2=0x7f6a500b5e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:32.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:32.244+0000 7f6a5596f640 1 --2- 172.21.15.27:0/2476864688 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6a5006d730 0x7f6a500b5e30 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f6a50051020 tx=0x7f6a4402a1e0 comp rx=0 tx=0).stop 2024-09-12T03:21:32.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:32.245+0000 7f6a5596f640 1 -- 172.21.15.27:0/2476864688 shutdown_connections 2024-09-12T03:21:32.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:32.245+0000 7f6a5596f640 1 --2- 172.21.15.27:0/2476864688 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6a5006d730 0x7f6a500b5e30 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:32.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:32.245+0000 7f6a5596f640 1 --2- 172.21.15.27:0/2476864688 >> v2:172.21.15.45:3300/0 conn(0x7f6a5015e900 0x7f6a500af370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:32.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:32.245+0000 7f6a5596f640 1 -- 172.21.15.27:0/2476864688 >> 172.21.15.27:0/2476864688 conn(0x7f6a5006cc40 msgr2=0x7f6a5006d020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:32.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:32.245+0000 7f6a5596f640 1 -- 172.21.15.27:0/2476864688 shutdown_connections 2024-09-12T03:21:32.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:32.245+0000 7f6a5596f640 1 -- 172.21.15.27:0/2476864688 wait complete. 2024-09-12T03:21:32.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:32.246+0000 7f6a5596f640 1 Processor -- start 2024-09-12T03:21:32.249 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:32.246+0000 7f6a5596f640 1 -- start start 2024-09-12T03:21:32.249 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:32.247+0000 7f6a5596f640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6a5006d730 0x7f6a5015fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:32.249 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:32.247+0000 7f6a5596f640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f6a500b6370 0x7f6a501575c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:32.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:32.247+0000 7f6a5596f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6a5015e900 0x7f6a50157b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:32.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:32.247+0000 7f6a5596f640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f6a500b01f0 con 0x7f6a5006d730 2024-09-12T03:21:32.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:32.247+0000 7f6a5596f640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f6a500aff10 con 0x7f6a5015e900 2024-09-12T03:21:32.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:32.247+0000 7f6a5596f640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f6a5006e880 con 0x7f6a500b6370 2024-09-12T03:21:32.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:32.247+0000 7f6a4effd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6a5006d730 0x7f6a5015fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:32.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:32.247+0000 7f6a4effd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6a5006d730 0x7f6a5015fcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41876/0 (socket says 172.21.15.27:41876) 2024-09-12T03:21:32.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:32.247+0000 7f6a4effd640 1 -- 172.21.15.27:0/67143097 learned_addr learned my addr 172.21.15.27:0/67143097 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:21:32.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:32.247+0000 7f6a4e7fc640 1 --2- 172.21.15.27:0/67143097 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f6a500b6370 0x7f6a501575c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:32.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:32.247+0000 7f6a4f7fe640 1 --2- 172.21.15.27:0/67143097 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6a5015e900 0x7f6a50157b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:32.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:32.247+0000 7f6a4effd640 1 -- 172.21.15.27:0/67143097 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f6a500b6370 msgr2=0x7f6a501575c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:32.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:32.247+0000 7f6a4effd640 1 --2- 172.21.15.27:0/67143097 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f6a500b6370 0x7f6a501575c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:32.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:32.247+0000 7f6a4effd640 1 -- 172.21.15.27:0/67143097 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6a5015e900 msgr2=0x7f6a50157b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:32.251 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:32.247+0000 7f6a4effd640 1 --2- 172.21.15.27:0/67143097 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6a5015e900 0x7f6a50157b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:32.251 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:32.247+0000 7f6a4effd640 1 -- 172.21.15.27:0/67143097 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a50156d40 con 0x7f6a5006d730 2024-09-12T03:21:32.251 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:32.247+0000 7f6a4f7fe640 1 --2- 172.21.15.27:0/67143097 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6a5015e900 0x7f6a50157b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:21:32.251 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:32.247+0000 7f6a4e7fc640 1 --2- 172.21.15.27:0/67143097 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f6a500b6370 0x7f6a501575c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:21:32.251 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:32.247+0000 7f6a4effd640 1 --2- 172.21.15.27:0/67143097 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6a5006d730 0x7f6a5015fcd0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f6a4000c820 tx=0x7f6a4000cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:32.251 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:32.247+0000 7f6a37fff640 1 -- 172.21.15.27:0/67143097 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a40015070 con 0x7f6a5006d730 2024-09-12T03:21:32.251 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:32.247+0000 7f6a37fff640 1 -- 172.21.15.27:0/67143097 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a40004020 con 0x7f6a5006d730 2024-09-12T03:21:32.251 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:32.247+0000 7f6a5596f640 1 -- 172.21.15.27:0/67143097 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a501f4c60 con 0x7f6a5006d730 2024-09-12T03:21:32.251 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:32.248+0000 7f6a37fff640 1 -- 172.21.15.27:0/67143097 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a400043f0 con 0x7f6a5006d730 2024-09-12T03:21:32.251 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:32.248+0000 7f6a5596f640 1 -- 172.21.15.27:0/67143097 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a501f5220 con 0x7f6a5006d730 2024-09-12T03:21:32.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:32.248+0000 7f6a5596f640 1 -- 172.21.15.27:0/67143097 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a50069560 con 0x7f6a5006d730 2024-09-12T03:21:32.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:32.249+0000 7f6a37fff640 1 -- 172.21.15.27:0/67143097 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 19) ==== 90855+0+0 (secure 0 0 0) 0x7f6a40007830 con 0x7f6a5006d730 2024-09-12T03:21:32.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:32.249+0000 7f6a37fff640 1 --2- 172.21.15.27:0/67143097 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f6a3c06c220 0x7f6a3c06e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:32.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:32.250+0000 7f6a37fff640 1 -- 172.21.15.27:0/67143097 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5767+0+0 (secure 0 0 0) 0x7f6a4008b4c0 con 0x7f6a5006d730 2024-09-12T03:21:32.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:32.250+0000 7f6a4e7fc640 1 --2- 172.21.15.27:0/67143097 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f6a3c06c220 0x7f6a3c06e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:32.253 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:32.250+0000 7f6a4e7fc640 1 --2- 172.21.15.27:0/67143097 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f6a3c06c220 0x7f6a3c06e6e0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f6a44034000 tx=0x7f6a4402e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:32.255 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:32.254+0000 7f6a37fff640 1 -- 172.21.15.27:0/67143097 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f6a50069560 con 0x7f6a5006d730 2024-09-12T03:21:32.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:32.453+0000 7f6a5596f640 1 -- 172.21.15.27:0/67143097 --> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol_clone64301", "target": ["mon-mgr", ""]}) -- 0x7f6a500b5e30 con 0x7f6a3c06c220 2024-09-12T03:21:32.468 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:32.467+0000 7f6a37fff640 1 -- 172.21.15.27:0/67143097 <== mgr.4113 v2:172.21.15.27:6836/1226920526 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f6a500b5e30 con 0x7f6a3c06c220 2024-09-12T03:21:32.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:32.469+0000 7f6a5596f640 1 -- 172.21.15.27:0/67143097 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f6a3c06c220 msgr2=0x7f6a3c06e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:32.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:32.469+0000 7f6a5596f640 1 --2- 172.21.15.27:0/67143097 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f6a3c06c220 0x7f6a3c06e6e0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f6a44034000 tx=0x7f6a4402e000 comp rx=0 tx=0).stop 2024-09-12T03:21:32.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:32.469+0000 7f6a5596f640 1 -- 172.21.15.27:0/67143097 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6a5006d730 msgr2=0x7f6a5015fcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:32.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:32.469+0000 7f6a5596f640 1 --2- 172.21.15.27:0/67143097 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6a5006d730 0x7f6a5015fcd0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f6a4000c820 tx=0x7f6a4000cce0 comp rx=0 tx=0).stop 2024-09-12T03:21:32.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:32.469+0000 7f6a5596f640 1 -- 172.21.15.27:0/67143097 shutdown_connections 2024-09-12T03:21:32.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:32.469+0000 7f6a5596f640 1 --2- 172.21.15.27:0/67143097 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6a5015e900 0x7f6a50157b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:32.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:32.469+0000 7f6a5596f640 1 --2- 172.21.15.27:0/67143097 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f6a500b6370 0x7f6a501575c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:32.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:32.469+0000 7f6a5596f640 1 --2- 172.21.15.27:0/67143097 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f6a3c06c220 0x7f6a3c06e6e0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:32.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:32.469+0000 7f6a5596f640 1 --2- 172.21.15.27:0/67143097 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6a5006d730 0x7f6a5015fcd0 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:32.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:32.469+0000 7f6a5596f640 1 -- 172.21.15.27:0/67143097 >> 172.21.15.27:0/67143097 conn(0x7f6a5006cc40 msgr2=0x7f6a500721d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:32.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:32.469+0000 7f6a5596f640 1 -- 172.21.15.27:0/67143097 shutdown_connections 2024-09-12T03:21:32.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:32.469+0000 7f6a5596f640 1 -- 172.21.15.27:0/67143097 wait complete. 2024-09-12T03:21:32.484 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-09-12T03:21:32.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:32 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-12T03:21:32.538 INFO:teuthology.orchestra.run.smithi027.stdout:2 2024-09-12T03:21:32.539 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T03:21:32.545 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-12T03:21:32.545 DEBUG:teuthology.orchestra.run.smithi027:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-12T03:21:32.593 DEBUG:teuthology.orchestra.run.smithi045:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-12T03:21:32.643 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-12T03:21:32.644 INFO:tasks.cephfs.mount:Terminating background process 2024-09-12T03:21:32.644 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-12T03:21:32.644 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T03:21:32.660 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-12T03:21:32.661 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T03:21:32.717 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-12T03:21:32.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:32.860+0000 7f9278131640 1 Processor -- start 2024-09-12T03:21:32.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:32.860+0000 7f9278131640 1 -- start start 2024-09-12T03:21:32.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:32.861+0000 7f9278131640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f927015e780 0x7f92700d3e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:32.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:32.861+0000 7f9278131640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f92700d43b0 0x7f92700d1280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:32.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:32.861+0000 7f9278131640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f92700b6690 con 0x7f927015eb60 2024-09-12T03:21:32.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:32.861+0000 7f9278131640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f92700b7950 con 0x7f927015e780 2024-09-12T03:21:32.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:32.861+0000 7f9278131640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f92700b7210 con 0x7f92700d43b0 2024-09-12T03:21:32.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:32.862+0000 7f92756cc640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f92700d43b0 0x7f92700d1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:32.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:32.862+0000 7f9275ecd640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f927015e780 0x7f92700d3e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:32.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:32.862+0000 7f92766ce640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f927015eb60 0x7f92700b3390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.27:43916/0 (socket says 172.21.15.27:43916) 2024-09-12T03:21:32.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:32.862+0000 7f92756cc640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f92700d43b0 0x7f92700d1280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3301/0 says I am v2:172.21.15.27:44910/0 (socket says 172.21.15.27:44910) 2024-09-12T03:21:32.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:32.862+0000 7f92766ce640 1 -- 172.21.15.27:0/3769629858 learned_addr learned my addr 172.21.15.27:0/3769629858 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:21:32.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:32.862+0000 7f92756cc640 1 -- 172.21.15.27:0/3769629858 >> v2:172.21.15.45:3300/0 conn(0x7f927015e780 msgr2=0x7f92700d3e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:32.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:32.862+0000 7f92756cc640 1 --2- 172.21.15.27:0/3769629858 >> v2:172.21.15.45:3300/0 conn(0x7f927015e780 0x7f92700d3e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:32.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:32.862+0000 7f92756cc640 1 -- 172.21.15.27:0/3769629858 >> v1:172.21.15.45:6789/0 conn(0x7f927015eb60 legacy=0x7f92700b3390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:32.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:32.862+0000 7f92756cc640 1 -- 172.21.15.27:0/3769629858 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92700b6970 con 0x7f92700d43b0 2024-09-12T03:21:32.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:32.863+0000 7f92756cc640 1 --2- 172.21.15.27:0/3769629858 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f92700d43b0 0x7f92700d1280 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f92640097a0 tx=0x7f926402a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b430428047fdc0aa server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:32.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:32.863+0000 7f9274ecb640 1 -- 172.21.15.27:0/3769629858 <== mon.2 v2:172.21.15.45:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9264037070 con 0x7f92700d43b0 2024-09-12T03:21:32.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:32.863+0000 7f9274ecb640 1 -- 172.21.15.27:0/3769629858 <== mon.2 v2:172.21.15.45:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9264032440 con 0x7f92700d43b0 2024-09-12T03:21:32.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:32.863+0000 7f9274ecb640 1 -- 172.21.15.27:0/3769629858 <== mon.2 v2:172.21.15.45:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9264032810 con 0x7f92700d43b0 2024-09-12T03:21:32.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:32.863+0000 7f9278131640 1 -- 172.21.15.27:0/3769629858 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f92700d43b0 msgr2=0x7f92700d1280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:32.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:32.863+0000 7f9278131640 1 --2- 172.21.15.27:0/3769629858 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f92700d43b0 0x7f92700d1280 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f92640097a0 tx=0x7f926402a180 comp rx=0 tx=0).stop 2024-09-12T03:21:32.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:32.863+0000 7f9278131640 1 -- 172.21.15.27:0/3769629858 shutdown_connections 2024-09-12T03:21:32.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:32.863+0000 7f9278131640 1 --2- 172.21.15.27:0/3769629858 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f92700d43b0 0x7f92700d1280 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:32.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:32.863+0000 7f9278131640 1 --2- 172.21.15.27:0/3769629858 >> v2:172.21.15.45:3300/0 conn(0x7f927015e780 0x7f92700d3e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:32.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:32.863+0000 7f9278131640 1 -- 172.21.15.27:0/3769629858 >> 172.21.15.27:0/3769629858 conn(0x7f927006da60 msgr2=0x7f927006de70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:32.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:32.863+0000 7f9278131640 1 -- 172.21.15.27:0/3769629858 shutdown_connections 2024-09-12T03:21:32.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:32.864+0000 7f9278131640 1 -- 172.21.15.27:0/3769629858 wait complete. 2024-09-12T03:21:32.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:32.865+0000 7f9278131640 1 Processor -- start 2024-09-12T03:21:32.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:32.865+0000 7f9278131640 1 -- start start 2024-09-12T03:21:32.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:32.865+0000 7f9278131640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f92700d43b0 0x7f92701f53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:32.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:32.865+0000 7f9278131640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f927015e780 0x7f9270161130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:32.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:32.865+0000 7f9278131640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f927015eb60 0x7f92700cb990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:32.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:32.865+0000 7f9278131640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f92700b6340 con 0x7f927015e780 2024-09-12T03:21:32.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:32.865+0000 7f9278131640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f92700b6f30 con 0x7f92700d43b0 2024-09-12T03:21:32.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:32.865+0000 7f9278131640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f92700d17c0 con 0x7f927015eb60 2024-09-12T03:21:32.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:32.866+0000 7f92756cc640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f927015e780 0x7f9270161130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:32.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:32.866+0000 7f9275ecd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f92700d43b0 0x7f92701f53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:32.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:32.866+0000 7f92756cc640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f927015e780 0x7f9270161130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41878/0 (socket says 172.21.15.27:41878) 2024-09-12T03:21:32.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:32.866+0000 7f9275ecd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f92700d43b0 0x7f92701f53c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.27:41970/0 (socket says 172.21.15.27:41970) 2024-09-12T03:21:32.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:32.866+0000 7f92756cc640 1 -- 172.21.15.27:0/2931883792 learned_addr learned my addr 172.21.15.27:0/2931883792 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:21:32.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:32.866+0000 7f92766ce640 1 --2- 172.21.15.27:0/2931883792 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f927015eb60 0x7f92700cb990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:32.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:32.866+0000 7f92756cc640 1 -- 172.21.15.27:0/2931883792 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f927015eb60 msgr2=0x7f92700cb990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:32.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:32.866+0000 7f92756cc640 1 --2- 172.21.15.27:0/2931883792 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f927015eb60 0x7f92700cb990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:32.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:32.866+0000 7f92756cc640 1 -- 172.21.15.27:0/2931883792 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f92700d43b0 msgr2=0x7f92701f53c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:32.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:32.866+0000 7f92756cc640 1 --2- 172.21.15.27:0/2931883792 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f92700d43b0 0x7f92701f53c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:32.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:32.866+0000 7f92756cc640 1 -- 172.21.15.27:0/2931883792 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92700cc610 con 0x7f927015e780 2024-09-12T03:21:32.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:32.866+0000 7f92756cc640 1 --2- 172.21.15.27:0/2931883792 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f927015e780 0x7f9270161130 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f926402a1c0 tx=0x7f92640046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:32.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:32.866+0000 7f926e7fc640 1 -- 172.21.15.27:0/2931883792 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9264041020 con 0x7f927015e780 2024-09-12T03:21:32.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:32.866+0000 7f926e7fc640 1 -- 172.21.15.27:0/2931883792 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9264030700 con 0x7f927015e780 2024-09-12T03:21:32.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:32.866+0000 7f9278131640 1 -- 172.21.15.27:0/2931883792 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92700cd310 con 0x7f927015e780 2024-09-12T03:21:32.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:32.866+0000 7f926e7fc640 1 -- 172.21.15.27:0/2931883792 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9264037040 con 0x7f927015e780 2024-09-12T03:21:32.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:32.866+0000 7f9278131640 1 -- 172.21.15.27:0/2931883792 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9270161670 con 0x7f927015e780 2024-09-12T03:21:32.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:32.867+0000 7f9278131640 1 -- 172.21.15.27:0/2931883792 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92700d1b50 con 0x7f927015e780 2024-09-12T03:21:32.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:32.868+0000 7f926e7fc640 1 -- 172.21.15.27:0/2931883792 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 19) ==== 90855+0+0 (secure 0 0 0) 0x7f9264044020 con 0x7f927015e780 2024-09-12T03:21:32.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:32.868+0000 7f926e7fc640 1 --2- 172.21.15.27:0/2931883792 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f925c06c250 0x7f925c06e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:32.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:32.869+0000 7f926e7fc640 1 -- 172.21.15.27:0/2931883792 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5767+0+0 (secure 0 0 0) 0x7f92640abba0 con 0x7f927015e780 2024-09-12T03:21:32.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:32.869+0000 7f9275ecd640 1 --2- 172.21.15.27:0/2931883792 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f925c06c250 0x7f925c06e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:32.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:32.869+0000 7f9275ecd640 1 --2- 172.21.15.27:0/2931883792 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f925c06c250 0x7f925c06e710 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f92700cfaf0 tx=0x7f926000b040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:32.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:32.873+0000 7f926e7fc640 1 -- 172.21.15.27:0/2931883792 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f926404b030 con 0x7f927015e780 2024-09-12T03:21:33.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:33.062+0000 7f9278131640 1 -- 172.21.15.27:0/2931883792 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f92700cb630 con 0x7f927015e780 2024-09-12T03:21:33.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:33.063+0000 7f926e7fc640 1 -- 172.21.15.27:0/2931883792 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v102) ==== 80+0+0 (secure 0 0 0) 0x7f9264079970 con 0x7f927015e780 2024-09-12T03:21:33.064 INFO:teuthology.orchestra.run.smithi027.stderr:listed 0 entries 2024-09-12T03:21:33.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:33.065+0000 7f9278131640 1 -- 172.21.15.27:0/2931883792 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f925c06c250 msgr2=0x7f925c06e710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:33.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:33.065+0000 7f9278131640 1 --2- 172.21.15.27:0/2931883792 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f925c06c250 0x7f925c06e710 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f92700cfaf0 tx=0x7f926000b040 comp rx=0 tx=0).stop 2024-09-12T03:21:33.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:33.065+0000 7f9278131640 1 -- 172.21.15.27:0/2931883792 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f927015e780 msgr2=0x7f9270161130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:33.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:33.065+0000 7f9278131640 1 --2- 172.21.15.27:0/2931883792 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f927015e780 0x7f9270161130 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f926402a1c0 tx=0x7f92640046c0 comp rx=0 tx=0).stop 2024-09-12T03:21:33.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:33.065+0000 7f9278131640 1 -- 172.21.15.27:0/2931883792 shutdown_connections 2024-09-12T03:21:33.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:33.065+0000 7f9278131640 1 --2- 172.21.15.27:0/2931883792 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f927015eb60 0x7f92700cb990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:33.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:33.065+0000 7f9278131640 1 --2- 172.21.15.27:0/2931883792 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f925c06c250 0x7f925c06e710 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:33.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:33.065+0000 7f9278131640 1 --2- 172.21.15.27:0/2931883792 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f927015e780 0x7f9270161130 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:33.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:33.065+0000 7f9278131640 1 --2- 172.21.15.27:0/2931883792 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f92700d43b0 0x7f92701f53c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:33.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:33.065+0000 7f9278131640 1 -- 172.21.15.27:0/2931883792 >> 172.21.15.27:0/2931883792 conn(0x7f927006da60 msgr2=0x7f927006de40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:33.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:33.065+0000 7f9278131640 1 -- 172.21.15.27:0/2931883792 shutdown_connections 2024-09-12T03:21:33.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:33.065+0000 7f9278131640 1 -- 172.21.15.27:0/2931883792 wait complete. 2024-09-12T03:21:33.077 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-12T03:21:33.077 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:21:33.077 DEBUG:teuthology.orchestra.run.smithi027:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-12T03:21:33.126 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi027.front.sepia.ceph.com 2024-09-12T03:21:33.126 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:21:33.127 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-12T03:21:33.154 INFO:tasks.cephfs.mount:Terminating background process 2024-09-12T03:21:33.155 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-12T03:21:33.155 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T03:21:33.173 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-12T03:21:33.173 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T03:21:33.232 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-12T03:21:33.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:33.334+0000 7f2b00911640 1 Processor -- start 2024-09-12T03:21:33.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:33.334+0000 7f2b00911640 1 -- start start 2024-09-12T03:21:33.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:33.336+0000 7f2b00911640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f2afc0b1ca0 0x7f2afc15f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:33.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:33.336+0000 7f2b00911640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f2afc06f360 0x7f2afc0cef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:33.337 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:33.336+0000 7f2b00911640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2afc0cf490 0x7f2afc0d2780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:33.337 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:33.336+0000 7f2b00911640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f2afc0d22f0 con 0x7f2afc06f360 2024-09-12T03:21:33.337 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:33.336+0000 7f2b00911640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f2afc15fd70 con 0x7f2afc0b1ca0 2024-09-12T03:21:33.337 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:33.336+0000 7f2b00911640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f2afc06cc40 con 0x7f2afc0cf490 2024-09-12T03:21:33.337 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:33.336+0000 7f2af3fff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f2afc06f360 0x7f2afc0cef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:33.337 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:33.336+0000 7f2af3fff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f2afc06f360 0x7f2afc0cef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41886/0 (socket says 172.21.15.27:41886) 2024-09-12T03:21:33.337 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:33.336+0000 7f2af3fff640 1 -- 172.21.15.27:0/256830411 learned_addr learned my addr 172.21.15.27:0/256830411 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:21:33.337 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:33.336+0000 7f2afa59c640 1 --2- 172.21.15.27:0/256830411 >> v2:172.21.15.45:3300/0 conn(0x7f2afc0b1ca0 0x7f2afc15f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:33.337 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:33.336+0000 7f2afad9d640 1 --2- 172.21.15.27:0/256830411 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2afc0cf490 0x7f2afc0d2780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:33.337 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:33.336+0000 7f2af3fff640 1 -- 172.21.15.27:0/256830411 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2afc0cf490 msgr2=0x7f2afc0d2780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:33.337 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:33.336+0000 7f2af3fff640 1 --2- 172.21.15.27:0/256830411 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2afc0cf490 0x7f2afc0d2780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:33.337 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:33.336+0000 7f2af3fff640 1 -- 172.21.15.27:0/256830411 >> v2:172.21.15.45:3300/0 conn(0x7f2afc0b1ca0 msgr2=0x7f2afc15f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:33.337 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:33.336+0000 7f2af3fff640 1 --2- 172.21.15.27:0/256830411 >> v2:172.21.15.45:3300/0 conn(0x7f2afc0b1ca0 0x7f2afc15f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:33.338 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:33.336+0000 7f2af3fff640 1 -- 172.21.15.27:0/256830411 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2afc0d1ce0 con 0x7f2afc06f360 2024-09-12T03:21:33.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:33.336+0000 7f2af3fff640 1 --2- 172.21.15.27:0/256830411 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2afc06f360 0x7f2afc0cef50 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f2afc073ed0 tx=0x7f2ae402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f794a294a046716 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:33.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:33.336+0000 7f2af9d9b640 1 -- 172.21.15.27:0/256830411 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ae4037070 con 0x7f2afc06f360 2024-09-12T03:21:33.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:33.336+0000 7f2af9d9b640 1 -- 172.21.15.27:0/256830411 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ae4032440 con 0x7f2afc06f360 2024-09-12T03:21:33.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:33.336+0000 7f2af9d9b640 1 -- 172.21.15.27:0/256830411 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ae4032810 con 0x7f2afc06f360 2024-09-12T03:21:33.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:33.337+0000 7f2b00911640 1 -- 172.21.15.27:0/256830411 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2afc06f360 msgr2=0x7f2afc0cef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:33.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:33.337+0000 7f2b00911640 1 --2- 172.21.15.27:0/256830411 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2afc06f360 0x7f2afc0cef50 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f2afc073ed0 tx=0x7f2ae402a1e0 comp rx=0 tx=0).stop 2024-09-12T03:21:33.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:33.337+0000 7f2b00911640 1 -- 172.21.15.27:0/256830411 shutdown_connections 2024-09-12T03:21:33.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:33.337+0000 7f2b00911640 1 --2- 172.21.15.27:0/256830411 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2afc0cf490 0x7f2afc0d2780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:33.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:33.337+0000 7f2b00911640 1 --2- 172.21.15.27:0/256830411 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2afc06f360 0x7f2afc0cef50 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:33.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:33.337+0000 7f2b00911640 1 --2- 172.21.15.27:0/256830411 >> v2:172.21.15.45:3300/0 conn(0x7f2afc0b1ca0 0x7f2afc15f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:33.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:33.337+0000 7f2b00911640 1 -- 172.21.15.27:0/256830411 >> 172.21.15.27:0/256830411 conn(0x7f2afc073800 msgr2=0x7f2afc09ba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:33.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:33.337+0000 7f2b00911640 1 -- 172.21.15.27:0/256830411 shutdown_connections 2024-09-12T03:21:33.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:33.337+0000 7f2b00911640 1 -- 172.21.15.27:0/256830411 wait complete. 2024-09-12T03:21:33.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:33.338+0000 7f2b00911640 1 Processor -- start 2024-09-12T03:21:33.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:33.338+0000 7f2b00911640 1 -- start start 2024-09-12T03:21:33.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:33.339+0000 7f2b00911640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2afc06f360 0x7f2afc1f7ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:33.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:33.339+0000 7f2b00911640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2afc0b1ca0 0x7f2afc15cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:33.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:33.339+0000 7f2b00911640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2afc0cf490 0x7f2afc0b0500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:33.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:33.339+0000 7f2b00911640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f2afc06d1c0 con 0x7f2afc06f360 2024-09-12T03:21:33.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:33.339+0000 7f2b00911640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f2afc0d1f80 con 0x7f2afc0cf490 2024-09-12T03:21:33.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:33.339+0000 7f2b00911640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f2afc06d340 con 0x7f2afc0b1ca0 2024-09-12T03:21:33.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:33.339+0000 7f2afa59c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2afc06f360 0x7f2afc1f7ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:33.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:33.339+0000 7f2afa59c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2afc06f360 0x7f2afc1f7ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41896/0 (socket says 172.21.15.27:41896) 2024-09-12T03:21:33.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:33.339+0000 7f2afa59c640 1 -- 172.21.15.27:0/1926474480 learned_addr learned my addr 172.21.15.27:0/1926474480 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:21:33.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:33.339+0000 7f2af3fff640 1 --2- 172.21.15.27:0/1926474480 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2afc0b1ca0 0x7f2afc15cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:33.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:33.339+0000 7f2afad9d640 1 --2- 172.21.15.27:0/1926474480 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2afc0cf490 0x7f2afc0b0500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:33.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:33.339+0000 7f2afa59c640 1 -- 172.21.15.27:0/1926474480 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2afc0b1ca0 msgr2=0x7f2afc15cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:33.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:33.339+0000 7f2afa59c640 1 --2- 172.21.15.27:0/1926474480 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2afc0b1ca0 0x7f2afc15cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:33.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:33.339+0000 7f2afa59c640 1 -- 172.21.15.27:0/1926474480 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2afc0cf490 msgr2=0x7f2afc0b0500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:33.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:33.339+0000 7f2afa59c640 1 --2- 172.21.15.27:0/1926474480 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2afc0cf490 0x7f2afc0b0500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:33.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:33.339+0000 7f2afa59c640 1 -- 172.21.15.27:0/1926474480 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2afc1f8e40 con 0x7f2afc06f360 2024-09-12T03:21:33.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:33.340+0000 7f2afa59c640 1 --2- 172.21.15.27:0/1926474480 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2afc06f360 0x7f2afc1f7ff0 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f2aec00e8c0 tx=0x7f2aec00ed80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:33.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:33.340+0000 7f2af2ffd640 1 -- 172.21.15.27:0/1926474480 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2aec017070 con 0x7f2afc06f360 2024-09-12T03:21:33.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:33.340+0000 7f2af2ffd640 1 -- 172.21.15.27:0/1926474480 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2aec00c940 con 0x7f2afc06f360 2024-09-12T03:21:33.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:33.340+0000 7f2af2ffd640 1 -- 172.21.15.27:0/1926474480 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2aec00cd10 con 0x7f2afc06f360 2024-09-12T03:21:33.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:33.340+0000 7f2b00911640 1 -- 172.21.15.27:0/1926474480 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2afc1f8880 con 0x7f2afc06f360 2024-09-12T03:21:33.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:33.340+0000 7f2b00911640 1 -- 172.21.15.27:0/1926474480 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2afc15c790 con 0x7f2afc06f360 2024-09-12T03:21:33.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:33.341+0000 7f2b00911640 1 -- 172.21.15.27:0/1926474480 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2afc0cfc40 con 0x7f2afc06f360 2024-09-12T03:21:33.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:33.341+0000 7f2af2ffd640 1 -- 172.21.15.27:0/1926474480 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 19) ==== 90855+0+0 (secure 0 0 0) 0x7f2aec01a070 con 0x7f2afc06f360 2024-09-12T03:21:33.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:33.342+0000 7f2af2ffd640 1 --2- 172.21.15.27:0/1926474480 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f2adc06c220 0x7f2adc06e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:33.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:33.342+0000 7f2af2ffd640 1 -- 172.21.15.27:0/1926474480 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5767+0+0 (secure 0 0 0) 0x7f2aec08b450 con 0x7f2afc06f360 2024-09-12T03:21:33.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:33.342+0000 7f2af3fff640 1 --2- 172.21.15.27:0/1926474480 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f2adc06c220 0x7f2adc06e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:33.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:33.343+0000 7f2af3fff640 1 --2- 172.21.15.27:0/1926474480 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f2adc06c220 0x7f2adc06e6e0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f2ae4034000 tx=0x7f2ae402e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:33.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:33.346+0000 7f2af2ffd640 1 -- 172.21.15.27:0/1926474480 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f2aec059220 con 0x7f2afc06f360 2024-09-12T03:21:33.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:33.533+0000 7f2b00911640 1 -- 172.21.15.27:0/1926474480 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f2afc15fa50 con 0x7f2afc06f360 2024-09-12T03:21:33.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:33.534+0000 7f2af2ffd640 1 -- 172.21.15.27:0/1926474480 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v102) ==== 80+0+0 (secure 0 0 0) 0x7f2aec05d520 con 0x7f2afc06f360 2024-09-12T03:21:33.535 INFO:teuthology.orchestra.run.smithi027.stderr:listed 0 entries 2024-09-12T03:21:33.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:33.537+0000 7f2b00911640 1 -- 172.21.15.27:0/1926474480 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f2adc06c220 msgr2=0x7f2adc06e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:33.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:33.537+0000 7f2b00911640 1 --2- 172.21.15.27:0/1926474480 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f2adc06c220 0x7f2adc06e6e0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f2ae4034000 tx=0x7f2ae402e000 comp rx=0 tx=0).stop 2024-09-12T03:21:33.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:33.537+0000 7f2b00911640 1 -- 172.21.15.27:0/1926474480 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2afc06f360 msgr2=0x7f2afc1f7ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:33.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:33.537+0000 7f2b00911640 1 --2- 172.21.15.27:0/1926474480 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2afc06f360 0x7f2afc1f7ff0 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f2aec00e8c0 tx=0x7f2aec00ed80 comp rx=0 tx=0).stop 2024-09-12T03:21:33.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:33.537+0000 7f2b00911640 1 -- 172.21.15.27:0/1926474480 shutdown_connections 2024-09-12T03:21:33.538 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:33.537+0000 7f2b00911640 1 --2- 172.21.15.27:0/1926474480 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2afc0cf490 0x7f2afc0b0500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:33.538 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:33.537+0000 7f2b00911640 1 --2- 172.21.15.27:0/1926474480 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2afc0b1ca0 0x7f2afc15cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:33.538 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:33.537+0000 7f2b00911640 1 --2- 172.21.15.27:0/1926474480 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f2adc06c220 0x7f2adc06e6e0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:33.538 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:33.537+0000 7f2b00911640 1 --2- 172.21.15.27:0/1926474480 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2afc06f360 0x7f2afc1f7ff0 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:33.538 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:33.537+0000 7f2b00911640 1 -- 172.21.15.27:0/1926474480 >> 172.21.15.27:0/1926474480 conn(0x7f2afc073800 msgr2=0x7f2afc161350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:33.538 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:33.537+0000 7f2b00911640 1 -- 172.21.15.27:0/1926474480 shutdown_connections 2024-09-12T03:21:33.538 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:33.537+0000 7f2b00911640 1 -- 172.21.15.27:0/1926474480 wait complete. 2024-09-12T03:21:33.547 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-09-12T03:21:33.547 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:21:33.548 DEBUG:teuthology.orchestra.run.smithi045:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-09-12T03:21:33.581 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi045.front.sepia.ceph.com 2024-09-12T03:21:33.581 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:21:33.581 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-12T03:21:33.640 INFO:tasks.cephfs.mount:Terminating background process 2024-09-12T03:21:33.640 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-12T03:21:33.640 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T03:21:33.658 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056004,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497432k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16324 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248716k,nr_inodes=812179,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497432k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-12T03:21:33.658 INFO:tasks.cephfs.mount:Terminating background process 2024-09-12T03:21:33.658 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-12T03:21:33.659 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T03:21:33.698 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056795,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16602 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249348k,nr_inodes=812337,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-12T03:21:33.699 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-12T03:21:33.801 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:33.801+0000 7fc7a12b7640 1 Processor -- start 2024-09-12T03:21:33.801 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:33.801+0000 7fc7a12b7640 1 -- start start 2024-09-12T03:21:33.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:33.802+0000 7fc7a12b7640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc79c15eb00 0x7fc79c0cede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:33.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:33.802+0000 7fc7a12b7640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fc79c0d3dd0 0x7fc79c0b75e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:33.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:33.802+0000 7fc7a12b7640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc79c0cf320 con 0x7fc79c0b7b20 2024-09-12T03:21:33.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:33.802+0000 7fc7a12b7640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fc79c0cfb80 con 0x7fc79c15eb00 2024-09-12T03:21:33.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:33.802+0000 7fc7a12b7640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fc79c0cf5c0 con 0x7fc79c0d3dd0 2024-09-12T03:21:33.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:33.802+0000 7fc79b7fe640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fc79c0b7b20 0x7fc79c0b3500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:51844/0 (socket says 172.21.15.27:51844) 2024-09-12T03:21:33.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:33.802+0000 7fc79b7fe640 1 -- 172.21.15.27:0/268937267 learned_addr learned my addr 172.21.15.27:0/268937267 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:21:33.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:33.802+0000 7fc79a7fc640 1 --2- 172.21.15.27:0/268937267 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fc79c0d3dd0 0x7fc79c0b75e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:33.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:33.802+0000 7fc79affd640 1 --2- 172.21.15.27:0/268937267 >> v2:172.21.15.45:3300/0 conn(0x7fc79c15eb00 0x7fc79c0cede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:33.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:33.802+0000 7fc799ffb640 1 -- 172.21.15.27:0/268937267 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3898828329 0 0) 0x7fc79c0cf320 con 0x7fc79c0b7b20 2024-09-12T03:21:33.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:33.802+0000 7fc799ffb640 1 -- 172.21.15.27:0/268937267 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc788003520 con 0x7fc79c0b7b20 2024-09-12T03:21:33.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:33.803+0000 7fc799ffb640 1 -- 172.21.15.27:0/268937267 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1696570096 0 0) 0x7fc788003520 con 0x7fc79c0b7b20 2024-09-12T03:21:33.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:33.803+0000 7fc799ffb640 1 -- 172.21.15.27:0/268937267 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fc79c0d3dd0 msgr2=0x7fc79c0b75e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:33.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:33.803+0000 7fc799ffb640 1 --2- 172.21.15.27:0/268937267 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fc79c0d3dd0 0x7fc79c0b75e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:33.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:33.803+0000 7fc799ffb640 1 -- 172.21.15.27:0/268937267 >> v2:172.21.15.45:3300/0 conn(0x7fc79c15eb00 msgr2=0x7fc79c0cede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:33.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:33.803+0000 7fc799ffb640 1 --2- 172.21.15.27:0/268937267 >> v2:172.21.15.45:3300/0 conn(0x7fc79c15eb00 0x7fc79c0cede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:33.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:33.803+0000 7fc799ffb640 1 -- 172.21.15.27:0/268937267 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc79c0cf8a0 con 0x7fc79c0b7b20 2024-09-12T03:21:33.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:33.803+0000 7fc799ffb640 1 -- 172.21.15.27:0/268937267 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 847755213 0 0) 0x7fc794001670 con 0x7fc79c0b7b20 2024-09-12T03:21:33.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:33.803+0000 7fc799ffb640 1 -- 172.21.15.27:0/268937267 >> v1:172.21.15.27:6789/0 conn(0x7fc79c0b7b20 legacy=0x7fc79c0b3500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:33.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:33.803+0000 7fc799ffb640 1 --2- 172.21.15.27:0/268937267 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc7880039b0 0x7fc788003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:33.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:33.803+0000 7fc799ffb640 1 --2- 172.21.15.27:0/268937267 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc788004800 0x7fc788004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:33.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:33.803+0000 7fc799ffb640 1 --2- 172.21.15.27:0/268937267 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fc79c0d3dd0 0x7fc7880051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:33.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:33.803+0000 7fc799ffb640 1 -- 172.21.15.27:0/268937267 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fc79c0cfb80 con 0x7fc788004800 2024-09-12T03:21:33.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:33.803+0000 7fc799ffb640 1 -- 172.21.15.27:0/268937267 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc79c0cf5c0 con 0x7fc7880039b0 2024-09-12T03:21:33.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:33.803+0000 7fc799ffb640 1 -- 172.21.15.27:0/268937267 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fc78800b9c0 con 0x7fc79c0d3dd0 2024-09-12T03:21:33.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:33.803+0000 7fc79a7fc640 1 --2- 172.21.15.27:0/268937267 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc788004800 0x7fc788004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:33.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:33.803+0000 7fc79affd640 1 --2- 172.21.15.27:0/268937267 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc7880039b0 0x7fc788003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:33.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:33.803+0000 7fc79b7fe640 1 --2- 172.21.15.27:0/268937267 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fc79c0d3dd0 0x7fc7880051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:33.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:33.803+0000 7fc79a7fc640 1 -- 172.21.15.27:0/268937267 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fc79c0d3dd0 msgr2=0x7fc7880051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:33.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:33.803+0000 7fc79a7fc640 1 --2- 172.21.15.27:0/268937267 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fc79c0d3dd0 0x7fc7880051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:33.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:33.803+0000 7fc79a7fc640 1 -- 172.21.15.27:0/268937267 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc7880039b0 msgr2=0x7fc788003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:33.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:33.803+0000 7fc79a7fc640 1 --2- 172.21.15.27:0/268937267 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc7880039b0 0x7fc788003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:33.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:33.803+0000 7fc79a7fc640 1 -- 172.21.15.27:0/268937267 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc788003520 con 0x7fc788004800 2024-09-12T03:21:33.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:33.804+0000 7fc79a7fc640 1 --2- 172.21.15.27:0/268937267 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc788004800 0x7fc788004c80 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7fc784009ac0 tx=0x7fc788005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=659a1ec5bc4da60d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:33.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:33.804+0000 7fc799ffb640 1 -- 172.21.15.27:0/268937267 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc784036070 con 0x7fc788004800 2024-09-12T03:21:33.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:33.804+0000 7fc799ffb640 1 -- 172.21.15.27:0/268937267 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc784031400 con 0x7fc788004800 2024-09-12T03:21:33.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:33.804+0000 7fc799ffb640 1 -- 172.21.15.27:0/268937267 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc784009690 con 0x7fc788004800 2024-09-12T03:21:33.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:33.805+0000 7fc7a12b7640 1 -- 172.21.15.27:0/268937267 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc788004800 msgr2=0x7fc788004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:33.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:33.805+0000 7fc7a12b7640 1 --2- 172.21.15.27:0/268937267 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc788004800 0x7fc788004c80 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7fc784009ac0 tx=0x7fc788005830 comp rx=0 tx=0).stop 2024-09-12T03:21:33.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:33.805+0000 7fc79b7fe640 1 -- 172.21.15.27:0/268937267 reap_dead start 2024-09-12T03:21:33.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:33.805+0000 7fc7a12b7640 1 -- 172.21.15.27:0/268937267 shutdown_connections 2024-09-12T03:21:33.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:33.805+0000 7fc7a12b7640 1 -- 172.21.15.27:0/268937267 >> 172.21.15.27:0/268937267 conn(0x7fc79c0d4790 msgr2=0x7fc79c0d4ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:33.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:33.805+0000 7fc7a12b7640 1 -- 172.21.15.27:0/268937267 shutdown_connections 2024-09-12T03:21:33.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:33.805+0000 7fc7a12b7640 1 -- 172.21.15.27:0/268937267 wait complete. 2024-09-12T03:21:33.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:33.806+0000 7fc7a12b7640 1 Processor -- start 2024-09-12T03:21:33.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:33.806+0000 7fc7a12b7640 1 -- start start 2024-09-12T03:21:33.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:33.807+0000 7fc7a12b7640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc788004800 0x7fc79c15fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:33.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:33.807+0000 7fc7a12b7640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fc79c0cc270 0x7fc79c0b6be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:33.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:33.807+0000 7fc7a12b7640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc79c155f50 0x7fc79c0cd570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:33.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:33.807+0000 7fc7a12b7640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fc79c0b3c10 con 0x7fc788004800 2024-09-12T03:21:33.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:33.807+0000 7fc7a12b7640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc79c0d2cf0 con 0x7fc79c155f50 2024-09-12T03:21:33.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:33.807+0000 7fc7a12b7640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fc79c0b3d90 con 0x7fc79c0cc270 2024-09-12T03:21:33.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:33.807+0000 7fc79affd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc788004800 0x7fc79c15fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:33.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:33.807+0000 7fc79affd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc788004800 0x7fc79c15fd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41916/0 (socket says 172.21.15.27:41916) 2024-09-12T03:21:33.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:33.807+0000 7fc79affd640 1 -- 172.21.15.27:0/3657835034 learned_addr learned my addr 172.21.15.27:0/3657835034 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:21:33.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:33.807+0000 7fc79a7fc640 1 --2- 172.21.15.27:0/3657835034 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fc79c0cc270 0x7fc79c0b6be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:33.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:33.807+0000 7fc79b7fe640 1 --2- 172.21.15.27:0/3657835034 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc79c155f50 0x7fc79c0cd570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:33.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:33.807+0000 7fc79affd640 1 -- 172.21.15.27:0/3657835034 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fc79c0cc270 msgr2=0x7fc79c0b6be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:33.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:33.807+0000 7fc79affd640 1 --2- 172.21.15.27:0/3657835034 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fc79c0cc270 0x7fc79c0b6be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:33.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:33.807+0000 7fc79affd640 1 -- 172.21.15.27:0/3657835034 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc79c155f50 msgr2=0x7fc79c0cd570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:33.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:33.807+0000 7fc79affd640 1 --2- 172.21.15.27:0/3657835034 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc79c155f50 0x7fc79c0cd570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:33.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:33.807+0000 7fc79affd640 1 -- 172.21.15.27:0/3657835034 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc79c0ccbe0 con 0x7fc788004800 2024-09-12T03:21:33.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:33.807+0000 7fc79b7fe640 1 --2- 172.21.15.27:0/3657835034 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc79c155f50 0x7fc79c0cd570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:21:33.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:33.807+0000 7fc79affd640 1 --2- 172.21.15.27:0/3657835034 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc788004800 0x7fc79c15fd40 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7fc78c011e30 tx=0x7fc78c00f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:33.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:33.808+0000 7fc7937fe640 1 -- 172.21.15.27:0/3657835034 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc78c015630 con 0x7fc788004800 2024-09-12T03:21:33.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:33.808+0000 7fc7937fe640 1 -- 172.21.15.27:0/3657835034 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc78c0157d0 con 0x7fc788004800 2024-09-12T03:21:33.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:33.808+0000 7fc7937fe640 1 -- 172.21.15.27:0/3657835034 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc78c015ba0 con 0x7fc788004800 2024-09-12T03:21:33.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:33.808+0000 7fc7a12b7640 1 -- 172.21.15.27:0/3657835034 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc79c0cc900 con 0x7fc788004800 2024-09-12T03:21:33.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:33.808+0000 7fc7a12b7640 1 -- 172.21.15.27:0/3657835034 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc79c0b4400 con 0x7fc788004800 2024-09-12T03:21:33.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:33.809+0000 7fc7a12b7640 1 -- 172.21.15.27:0/3657835034 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc79c0b75e0 con 0x7fc788004800 2024-09-12T03:21:33.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:33.809+0000 7fc7937fe640 1 -- 172.21.15.27:0/3657835034 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 19) ==== 90855+0+0 (secure 0 0 0) 0x7fc78c01d400 con 0x7fc788004800 2024-09-12T03:21:33.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:33.810+0000 7fc7937fe640 1 --2- 172.21.15.27:0/3657835034 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fc77c06c250 0x7fc77c06e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:33.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:33.810+0000 7fc7937fe640 1 -- 172.21.15.27:0/3657835034 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5767+0+0 (secure 0 0 0) 0x7fc78c08c810 con 0x7fc788004800 2024-09-12T03:21:33.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:33.810+0000 7fc79a7fc640 1 --2- 172.21.15.27:0/3657835034 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fc77c06c250 0x7fc77c06e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:33.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:33.810+0000 7fc79a7fc640 1 --2- 172.21.15.27:0/3657835034 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fc77c06c250 0x7fc77c06e710 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fc784033000 tx=0x7fc784008030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:33.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:33.814+0000 7fc7937fe640 1 -- 172.21.15.27:0/3657835034 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fc78c019030 con 0x7fc788004800 2024-09-12T03:21:34.029 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.029+0000 7fc7a12b7640 1 -- 172.21.15.27:0/3657835034 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc79c15e060 con 0x7fc788004800 2024-09-12T03:21:34.050 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.030+0000 7fc7937fe640 1 -- 172.21.15.27:0/3657835034 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 59 v59) ==== 94+0+5222 (secure 0 0 0) 0x7fc78c010040 con 0x7fc788004800 2024-09-12T03:21:34.051 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:21:34.051 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":59,"btime":"2024-09-12T03:21:07:819282+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5885,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/2343815091","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2343815091},{"type":"v1","addr":"172.21.15.45:6834","nonce":2343815091}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":55},{"gid":6376,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/512483568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":512483568},{"type":"v1","addr":"172.21.15.45:6835","nonce":512483568}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":55},{"gid":6825,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6834/1888895227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":1888895227},{"type":"v1","addr":"172.21.15.27:6834","nonce":1888895227}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":54}],"filesystems":[{"mdsmap":{"epoch":58,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-12T03:20:59.133824+0000","modified":"2024-09-12T03:21:01.148633+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6828},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6828":{"gid":6828,"name":"a","rank":0,"incarnation":56,"state":"up:active","state_seq":3,"addr":"172.21.15.27:6835/1379166526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":1379166526},{"type":"v1","addr":"172.21.15.27:6835","nonce":1379166526}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6828,"qdb_cluster":[6828]},"id":8}]} 2024-09-12T03:21:34.052 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 59 2024-09-12T03:21:34.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.032+0000 7fc7a12b7640 1 -- 172.21.15.27:0/3657835034 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fc77c06c250 msgr2=0x7fc77c06e710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:34.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.032+0000 7fc7a12b7640 1 --2- 172.21.15.27:0/3657835034 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fc77c06c250 0x7fc77c06e710 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fc784033000 tx=0x7fc784008030 comp rx=0 tx=0).stop 2024-09-12T03:21:34.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.032+0000 7fc7a12b7640 1 -- 172.21.15.27:0/3657835034 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc788004800 msgr2=0x7fc79c15fd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:34.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.032+0000 7fc7a12b7640 1 --2- 172.21.15.27:0/3657835034 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc788004800 0x7fc79c15fd40 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7fc78c011e30 tx=0x7fc78c00f580 comp rx=0 tx=0).stop 2024-09-12T03:21:34.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.032+0000 7fc7a12b7640 1 -- 172.21.15.27:0/3657835034 shutdown_connections 2024-09-12T03:21:34.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.032+0000 7fc7a12b7640 1 --2- 172.21.15.27:0/3657835034 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc79c155f50 0x7fc79c0cd570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:34.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.032+0000 7fc7a12b7640 1 --2- 172.21.15.27:0/3657835034 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fc79c0cc270 0x7fc79c0b6be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:34.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.032+0000 7fc7a12b7640 1 --2- 172.21.15.27:0/3657835034 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fc77c06c250 0x7fc77c06e710 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:34.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.032+0000 7fc7a12b7640 1 --2- 172.21.15.27:0/3657835034 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc788004800 0x7fc79c15fd40 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:34.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.032+0000 7fc7a12b7640 1 -- 172.21.15.27:0/3657835034 >> 172.21.15.27:0/3657835034 conn(0x7fc79c0d4790 msgr2=0x7fc79c0d3110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:34.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.032+0000 7fc7a12b7640 1 -- 172.21.15.27:0/3657835034 shutdown_connections 2024-09-12T03:21:34.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.032+0000 7fc7a12b7640 1 -- 172.21.15.27:0/3657835034 wait complete. 2024-09-12T03:21:34.054 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-12T03:21:34.157 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.157+0000 7f976218d640 1 Processor -- start 2024-09-12T03:21:34.157 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.157+0000 7f976218d640 1 -- start start 2024-09-12T03:21:34.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.158+0000 7f976218d640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f975c158cf0 0x7f975c0b55d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:34.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.158+0000 7f976218d640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f975c1590d0 0x7f975c162430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:34.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.158+0000 7f976218d640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f975c162970 con 0x7f975c15f600 2024-09-12T03:21:34.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.158+0000 7f976218d640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f975c162c10 con 0x7f975c1590d0 2024-09-12T03:21:34.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.158+0000 7f976218d640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f975c162ef0 con 0x7f975c158cf0 2024-09-12T03:21:34.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.158+0000 7f975bfff640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f975c15f600 0x7f975c06bb30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:51848/0 (socket says 172.21.15.27:51848) 2024-09-12T03:21:34.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.158+0000 7f975bfff640 1 -- 172.21.15.27:0/705052432 learned_addr learned my addr 172.21.15.27:0/705052432 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:21:34.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.158+0000 7f975affd640 1 --2- 172.21.15.27:0/705052432 >> v2:172.21.15.45:3300/0 conn(0x7f975c1590d0 0x7f975c162430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:34.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.158+0000 7f975b7fe640 1 --2- 172.21.15.27:0/705052432 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f975c158cf0 0x7f975c0b55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:34.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.158+0000 7f975a7fc640 1 -- 172.21.15.27:0/705052432 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4094593615 0 0) 0x7f975c162970 con 0x7f975c15f600 2024-09-12T03:21:34.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.158+0000 7f975a7fc640 1 -- 172.21.15.27:0/705052432 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9744003520 con 0x7f975c15f600 2024-09-12T03:21:34.162 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.159+0000 7f975a7fc640 1 -- 172.21.15.27:0/705052432 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 323212952 0 0) 0x7f9744003520 con 0x7f975c15f600 2024-09-12T03:21:34.162 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.159+0000 7f975a7fc640 1 -- 172.21.15.27:0/705052432 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f975c158cf0 msgr2=0x7f975c0b55d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:34.162 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.159+0000 7f975a7fc640 1 --2- 172.21.15.27:0/705052432 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f975c158cf0 0x7f975c0b55d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:34.162 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.159+0000 7f975a7fc640 1 -- 172.21.15.27:0/705052432 >> v2:172.21.15.45:3300/0 conn(0x7f975c1590d0 msgr2=0x7f975c162430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:34.162 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.159+0000 7f975a7fc640 1 --2- 172.21.15.27:0/705052432 >> v2:172.21.15.45:3300/0 conn(0x7f975c1590d0 0x7f975c162430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:34.162 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.159+0000 7f975a7fc640 1 -- 172.21.15.27:0/705052432 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f975c1631d0 con 0x7f975c15f600 2024-09-12T03:21:34.162 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.159+0000 7f975a7fc640 1 -- 172.21.15.27:0/705052432 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 847755213 0 0) 0x7f9754001a10 con 0x7f975c15f600 2024-09-12T03:21:34.162 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.159+0000 7f975a7fc640 1 -- 172.21.15.27:0/705052432 >> v1:172.21.15.27:6789/0 conn(0x7f975c15f600 legacy=0x7f975c06bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:34.162 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.159+0000 7f975a7fc640 1 --2- 172.21.15.27:0/705052432 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f97440039b0 0x7f9744003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:34.162 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.159+0000 7f975a7fc640 1 --2- 172.21.15.27:0/705052432 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9744004800 0x7f9744004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:34.162 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.159+0000 7f975a7fc640 1 --2- 172.21.15.27:0/705052432 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f975c158cf0 0x7f97440051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:34.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.159+0000 7f975a7fc640 1 -- 172.21.15.27:0/705052432 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f975c162c10 con 0x7f9744004800 2024-09-12T03:21:34.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.159+0000 7f975a7fc640 1 -- 172.21.15.27:0/705052432 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f975c162ef0 con 0x7f97440039b0 2024-09-12T03:21:34.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.159+0000 7f975a7fc640 1 -- 172.21.15.27:0/705052432 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f974400ba40 con 0x7f975c158cf0 2024-09-12T03:21:34.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.159+0000 7f975affd640 1 --2- 172.21.15.27:0/705052432 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9744004800 0x7f9744004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:34.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.159+0000 7f975b7fe640 1 --2- 172.21.15.27:0/705052432 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f97440039b0 0x7f9744003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:34.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.159+0000 7f975bfff640 1 --2- 172.21.15.27:0/705052432 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f975c158cf0 0x7f97440051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:34.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.159+0000 7f975affd640 1 -- 172.21.15.27:0/705052432 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f975c158cf0 msgr2=0x7f97440051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:34.164 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.159+0000 7f975affd640 1 --2- 172.21.15.27:0/705052432 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f975c158cf0 0x7f97440051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:34.164 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.159+0000 7f975affd640 1 -- 172.21.15.27:0/705052432 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f97440039b0 msgr2=0x7f9744003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:34.164 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.159+0000 7f975affd640 1 --2- 172.21.15.27:0/705052432 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f97440039b0 0x7f9744003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:34.164 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.159+0000 7f975affd640 1 -- 172.21.15.27:0/705052432 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9744003520 con 0x7f9744004800 2024-09-12T03:21:34.164 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.160+0000 7f975affd640 1 --2- 172.21.15.27:0/705052432 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9744004800 0x7f9744004c80 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f9750009a90 tx=0x7f9744005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1589ee89cf1abcf server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:34.164 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.160+0000 7f975a7fc640 1 -- 172.21.15.27:0/705052432 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9750036070 con 0x7f9744004800 2024-09-12T03:21:34.164 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.160+0000 7f975a7fc640 1 -- 172.21.15.27:0/705052432 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9750031400 con 0x7f9744004800 2024-09-12T03:21:34.164 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.160+0000 7f975a7fc640 1 -- 172.21.15.27:0/705052432 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9750009690 con 0x7f9744004800 2024-09-12T03:21:34.164 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.160+0000 7f976218d640 1 -- 172.21.15.27:0/705052432 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9744004800 msgr2=0x7f9744004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:34.164 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.160+0000 7f976218d640 1 --2- 172.21.15.27:0/705052432 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9744004800 0x7f9744004c80 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f9750009a90 tx=0x7f9744005830 comp rx=0 tx=0).stop 2024-09-12T03:21:34.164 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.160+0000 7f975bfff640 1 -- 172.21.15.27:0/705052432 reap_dead start 2024-09-12T03:21:34.164 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.160+0000 7f976218d640 1 -- 172.21.15.27:0/705052432 shutdown_connections 2024-09-12T03:21:34.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.160+0000 7f976218d640 1 -- 172.21.15.27:0/705052432 >> 172.21.15.27:0/705052432 conn(0x7f975c06cc40 msgr2=0x7f975c06d050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:34.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.161+0000 7f976218d640 1 -- 172.21.15.27:0/705052432 shutdown_connections 2024-09-12T03:21:34.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.161+0000 7f976218d640 1 -- 172.21.15.27:0/705052432 wait complete. 2024-09-12T03:21:34.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.162+0000 7f976218d640 1 Processor -- start 2024-09-12T03:21:34.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.162+0000 7f976218d640 1 -- start start 2024-09-12T03:21:34.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.163+0000 7f976218d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9744004800 0x7f975c15c250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:34.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.163+0000 7f976218d640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f975c146d60 0x7f975c0b0a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:34.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.163+0000 7f976218d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f975c15a220 0x7f975c15a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:34.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.163+0000 7f976218d640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f975c06c3c0 con 0x7f9744004800 2024-09-12T03:21:34.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.163+0000 7f976218d640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f975c06c240 con 0x7f975c15a220 2024-09-12T03:21:34.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.163+0000 7f976218d640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f975c15fa10 con 0x7f975c146d60 2024-09-12T03:21:34.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.163+0000 7f975b7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9744004800 0x7f975c15c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:34.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.163+0000 7f975b7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9744004800 0x7f975c15c250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41932/0 (socket says 172.21.15.27:41932) 2024-09-12T03:21:34.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.163+0000 7f975b7fe640 1 -- 172.21.15.27:0/1147918308 learned_addr learned my addr 172.21.15.27:0/1147918308 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:21:34.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.163+0000 7f975bfff640 1 --2- 172.21.15.27:0/1147918308 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f975c15a220 0x7f975c15a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:34.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.163+0000 7f975affd640 1 --2- 172.21.15.27:0/1147918308 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f975c146d60 0x7f975c0b0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:34.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.163+0000 7f975b7fe640 1 -- 172.21.15.27:0/1147918308 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f975c146d60 msgr2=0x7f975c0b0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:34.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.163+0000 7f975b7fe640 1 --2- 172.21.15.27:0/1147918308 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f975c146d60 0x7f975c0b0a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:34.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.163+0000 7f975b7fe640 1 -- 172.21.15.27:0/1147918308 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f975c15a220 msgr2=0x7f975c15a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:34.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.163+0000 7f975b7fe640 1 --2- 172.21.15.27:0/1147918308 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f975c15a220 0x7f975c15a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:34.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.163+0000 7f975b7fe640 1 -- 172.21.15.27:0/1147918308 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f975c0af9d0 con 0x7f9744004800 2024-09-12T03:21:34.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.163+0000 7f975b7fe640 1 --2- 172.21.15.27:0/1147918308 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9744004800 0x7f975c15c250 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f974c0119a0 tx=0x7f974c011da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:34.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.163+0000 7f9743fff640 1 -- 172.21.15.27:0/1147918308 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f974c00fa30 con 0x7f9744004800 2024-09-12T03:21:34.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.163+0000 7f9743fff640 1 -- 172.21.15.27:0/1147918308 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f974c00fbd0 con 0x7f9744004800 2024-09-12T03:21:34.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.163+0000 7f9743fff640 1 -- 172.21.15.27:0/1147918308 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f974c014440 con 0x7f9744004800 2024-09-12T03:21:34.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.163+0000 7f976218d640 1 -- 172.21.15.27:0/1147918308 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f975c0afcb0 con 0x7f9744004800 2024-09-12T03:21:34.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.164+0000 7f976218d640 1 -- 172.21.15.27:0/1147918308 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f975c0b0270 con 0x7f9744004800 2024-09-12T03:21:34.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.164+0000 7f976218d640 1 -- 172.21.15.27:0/1147918308 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f975c0b55d0 con 0x7f9744004800 2024-09-12T03:21:34.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.165+0000 7f9743fff640 1 -- 172.21.15.27:0/1147918308 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 19) ==== 90855+0+0 (secure 0 0 0) 0x7f974c012040 con 0x7f9744004800 2024-09-12T03:21:34.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.165+0000 7f9743fff640 1 --2- 172.21.15.27:0/1147918308 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f974806c220 0x7f974806e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:34.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.165+0000 7f9743fff640 1 -- 172.21.15.27:0/1147918308 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5767+0+0 (secure 0 0 0) 0x7f974c08bbe0 con 0x7f9744004800 2024-09-12T03:21:34.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.166+0000 7f975affd640 1 --2- 172.21.15.27:0/1147918308 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f974806c220 0x7f974806e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:34.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.166+0000 7f975affd640 1 --2- 172.21.15.27:0/1147918308 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f974806c220 0x7f974806e6e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f9750033000 tx=0x7f9750008030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:34.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.170+0000 7f9743fff640 1 -- 172.21.15.27:0/1147918308 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f974c019030 con 0x7f9744004800 2024-09-12T03:21:34.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.381+0000 7f976218d640 1 -- 172.21.15.27:0/1147918308 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f975c1574f0 con 0x7f9744004800 2024-09-12T03:21:34.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.381+0000 7f9743fff640 1 -- 172.21.15.27:0/1147918308 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 59 v59) ==== 94+0+5222 (secure 0 0 0) 0x7f974c059f70 con 0x7f9744004800 2024-09-12T03:21:34.381 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:21:34.382 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":59,"btime":"2024-09-12T03:21:07:819282+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5885,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/2343815091","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2343815091},{"type":"v1","addr":"172.21.15.45:6834","nonce":2343815091}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":55},{"gid":6376,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/512483568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":512483568},{"type":"v1","addr":"172.21.15.45:6835","nonce":512483568}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":55},{"gid":6825,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6834/1888895227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":1888895227},{"type":"v1","addr":"172.21.15.27:6834","nonce":1888895227}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":54}],"filesystems":[{"mdsmap":{"epoch":58,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-12T03:20:59.133824+0000","modified":"2024-09-12T03:21:01.148633+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6828},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6828":{"gid":6828,"name":"a","rank":0,"incarnation":56,"state":"up:active","state_seq":3,"addr":"172.21.15.27:6835/1379166526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":1379166526},{"type":"v1","addr":"172.21.15.27:6835","nonce":1379166526}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6828,"qdb_cluster":[6828]},"id":8}]} 2024-09-12T03:21:34.382 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 59 2024-09-12T03:21:34.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.383+0000 7f976218d640 1 -- 172.21.15.27:0/1147918308 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f974806c220 msgr2=0x7f974806e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:34.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.383+0000 7f976218d640 1 --2- 172.21.15.27:0/1147918308 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f974806c220 0x7f974806e6e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f9750033000 tx=0x7f9750008030 comp rx=0 tx=0).stop 2024-09-12T03:21:34.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.383+0000 7f976218d640 1 -- 172.21.15.27:0/1147918308 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9744004800 msgr2=0x7f975c15c250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:34.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.383+0000 7f976218d640 1 --2- 172.21.15.27:0/1147918308 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9744004800 0x7f975c15c250 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f974c0119a0 tx=0x7f974c011da0 comp rx=0 tx=0).stop 2024-09-12T03:21:34.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.383+0000 7f976218d640 1 -- 172.21.15.27:0/1147918308 shutdown_connections 2024-09-12T03:21:34.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.383+0000 7f976218d640 1 --2- 172.21.15.27:0/1147918308 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f975c15a220 0x7f975c15a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:34.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.383+0000 7f976218d640 1 --2- 172.21.15.27:0/1147918308 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f975c146d60 0x7f975c0b0a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:34.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.383+0000 7f976218d640 1 --2- 172.21.15.27:0/1147918308 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f974806c220 0x7f974806e6e0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:34.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.383+0000 7f976218d640 1 --2- 172.21.15.27:0/1147918308 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9744004800 0x7f975c15c250 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:34.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.383+0000 7f976218d640 1 -- 172.21.15.27:0/1147918308 >> 172.21.15.27:0/1147918308 conn(0x7f975c06cc40 msgr2=0x7f975c06f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:34.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.383+0000 7f976218d640 1 -- 172.21.15.27:0/1147918308 shutdown_connections 2024-09-12T03:21:34.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.383+0000 7f976218d640 1 -- 172.21.15.27:0/1147918308 wait complete. 2024-09-12T03:21:34.395 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-12T03:21:34.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.497+0000 7fc27d711640 1 Processor -- start 2024-09-12T03:21:34.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.497+0000 7fc27d711640 1 -- start start 2024-09-12T03:21:34.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.497+0000 7fc27d711640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc27815b2b0 0x7fc2780b3560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:34.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.497+0000 7fc27d711640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc2780ce410 con 0x7fc278072d70 2024-09-12T03:21:34.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.497+0000 7fc27d711640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc2780732c0 con 0x7fc27806e9b0 2024-09-12T03:21:34.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.497+0000 7fc27d711640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fc2780720a0 con 0x7fc27815b2b0 2024-09-12T03:21:34.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.498+0000 7fc2767fc640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fc278072d70 0x7fc278072430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:51852/0 (socket says 172.21.15.27:51852) 2024-09-12T03:21:34.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.498+0000 7fc2767fc640 1 -- 172.21.15.27:0/3023533252 learned_addr learned my addr 172.21.15.27:0/3023533252 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:21:34.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.498+0000 7fc276ffd640 1 --2- 172.21.15.27:0/3023533252 >> v2:172.21.15.45:3300/0 conn(0x7fc27815b2b0 0x7fc2780b3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:34.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.498+0000 7fc275ffb640 1 -- 172.21.15.27:0/3023533252 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4269609315 0 0) 0x7fc26c001110 con 0x7fc278072d70 2024-09-12T03:21:34.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.498+0000 7fc275ffb640 1 -- 172.21.15.27:0/3023533252 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc260003520 con 0x7fc278072d70 2024-09-12T03:21:34.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.498+0000 7fc276ffd640 1 -- 172.21.15.27:0/3023533252 >> v1:172.21.15.45:6789/0 conn(0x7fc27806e9b0 legacy=0x7fc2780cebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:34.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.498+0000 7fc276ffd640 1 -- 172.21.15.27:0/3023533252 >> v1:172.21.15.27:6789/0 conn(0x7fc278072d70 legacy=0x7fc278072430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:34.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.498+0000 7fc276ffd640 1 -- 172.21.15.27:0/3023533252 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc278071ae0 con 0x7fc27815b2b0 2024-09-12T03:21:34.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.499+0000 7fc276ffd640 1 --2- 172.21.15.27:0/3023533252 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc27815b2b0 0x7fc2780b3560 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fc268009d90 tx=0x7fc26802a6f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bfdf4446b67f741 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:34.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.499+0000 7fc275ffb640 1 -- 172.21.15.27:0/3023533252 <== mon.1 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc268007740 con 0x7fc27815b2b0 2024-09-12T03:21:34.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.499+0000 7fc275ffb640 1 -- 172.21.15.27:0/3023533252 <== mon.1 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2680078e0 con 0x7fc27815b2b0 2024-09-12T03:21:34.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.499+0000 7fc275ffb640 1 -- 172.21.15.27:0/3023533252 <== mon.1 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc268007c90 con 0x7fc27815b2b0 2024-09-12T03:21:34.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.500+0000 7fc27d711640 1 -- 172.21.15.27:0/3023533252 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc27815b2b0 msgr2=0x7fc2780b3560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:34.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.500+0000 7fc27d711640 1 --2- 172.21.15.27:0/3023533252 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc27815b2b0 0x7fc2780b3560 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fc268009d90 tx=0x7fc26802a6f0 comp rx=0 tx=0).stop 2024-09-12T03:21:34.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.500+0000 7fc27d711640 1 -- 172.21.15.27:0/3023533252 shutdown_connections 2024-09-12T03:21:34.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.500+0000 7fc27d711640 1 --2- 172.21.15.27:0/3023533252 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc27815b2b0 0x7fc2780b3560 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:34.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.500+0000 7fc27d711640 1 -- 172.21.15.27:0/3023533252 >> 172.21.15.27:0/3023533252 conn(0x7fc278062c60 msgr2=0x7fc27806bb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:34.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.500+0000 7fc27d711640 1 -- 172.21.15.27:0/3023533252 shutdown_connections 2024-09-12T03:21:34.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.500+0000 7fc27d711640 1 -- 172.21.15.27:0/3023533252 wait complete. 2024-09-12T03:21:34.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.501+0000 7fc27d711640 1 Processor -- start 2024-09-12T03:21:34.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.501+0000 7fc27d711640 1 -- start start 2024-09-12T03:21:34.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.502+0000 7fc27d711640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc27806e9b0 0x7fc2780cc140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:34.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.502+0000 7fc27d711640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fc278072d70 0x7fc2780acee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:34.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.502+0000 7fc27d711640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc27815b2b0 0x7fc2781f56a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:34.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.502+0000 7fc27d711640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fc27815b690 con 0x7fc27815b2b0 2024-09-12T03:21:34.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.502+0000 7fc27d711640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc2780ce690 con 0x7fc27806e9b0 2024-09-12T03:21:34.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.502+0000 7fc27d711640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fc27815b810 con 0x7fc278072d70 2024-09-12T03:21:34.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.502+0000 7fc2777fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc27815b2b0 0x7fc2781f56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:34.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.502+0000 7fc2777fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc27815b2b0 0x7fc2781f56a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41940/0 (socket says 172.21.15.27:41940) 2024-09-12T03:21:34.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.502+0000 7fc2777fe640 1 -- 172.21.15.27:0/1622680183 learned_addr learned my addr 172.21.15.27:0/1622680183 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:21:34.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.502+0000 7fc276ffd640 1 --2- 172.21.15.27:0/1622680183 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc27806e9b0 0x7fc2780cc140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:34.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.502+0000 7fc2767fc640 1 --2- 172.21.15.27:0/1622680183 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fc278072d70 0x7fc2780acee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:34.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.502+0000 7fc2777fe640 1 -- 172.21.15.27:0/1622680183 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fc278072d70 msgr2=0x7fc2780acee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:34.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.502+0000 7fc2777fe640 1 --2- 172.21.15.27:0/1622680183 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fc278072d70 0x7fc2780acee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:34.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.502+0000 7fc2777fe640 1 -- 172.21.15.27:0/1622680183 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc27806e9b0 msgr2=0x7fc2780cc140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:34.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.502+0000 7fc2777fe640 1 --2- 172.21.15.27:0/1622680183 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc27806e9b0 0x7fc2780cc140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:34.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.502+0000 7fc2777fe640 1 -- 172.21.15.27:0/1622680183 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2780cc680 con 0x7fc27815b2b0 2024-09-12T03:21:34.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.502+0000 7fc2777fe640 1 --2- 172.21.15.27:0/1622680183 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc27815b2b0 0x7fc2781f56a0 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7fc270001eb0 tx=0x7fc27000c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:34.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.503+0000 7fc25f7fe640 1 -- 172.21.15.27:0/1622680183 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc270017070 con 0x7fc27815b2b0 2024-09-12T03:21:34.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.503+0000 7fc25f7fe640 1 -- 172.21.15.27:0/1622680183 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2700052d0 con 0x7fc27815b2b0 2024-09-12T03:21:34.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.503+0000 7fc27d711640 1 -- 172.21.15.27:0/1622680183 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2780ad750 con 0x7fc27815b2b0 2024-09-12T03:21:34.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.503+0000 7fc25f7fe640 1 -- 172.21.15.27:0/1622680183 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc270003810 con 0x7fc27815b2b0 2024-09-12T03:21:34.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.503+0000 7fc27d711640 1 -- 172.21.15.27:0/1622680183 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2780add10 con 0x7fc27815b2b0 2024-09-12T03:21:34.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.504+0000 7fc27d711640 1 -- 172.21.15.27:0/1622680183 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2780b3560 con 0x7fc27815b2b0 2024-09-12T03:21:34.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.504+0000 7fc25f7fe640 1 -- 172.21.15.27:0/1622680183 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 19) ==== 90855+0+0 (secure 0 0 0) 0x7fc27001a070 con 0x7fc27815b2b0 2024-09-12T03:21:34.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.505+0000 7fc25f7fe640 1 --2- 172.21.15.27:0/1622680183 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fc26406c220 0x7fc26406e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:34.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.505+0000 7fc276ffd640 1 --2- 172.21.15.27:0/1622680183 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fc26406c220 0x7fc26406e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:34.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.505+0000 7fc25f7fe640 1 -- 172.21.15.27:0/1622680183 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5767+0+0 (secure 0 0 0) 0x7fc27008a8d0 con 0x7fc27815b2b0 2024-09-12T03:21:34.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.505+0000 7fc276ffd640 1 --2- 172.21.15.27:0/1622680183 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fc26406c220 0x7fc26406e6e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fc268034000 tx=0x7fc268033040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:34.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.509+0000 7fc25f7fe640 1 -- 172.21.15.27:0/1622680183 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fc270058cf0 con 0x7fc27815b2b0 2024-09-12T03:21:34.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.684+0000 7fc27d711640 1 -- 172.21.15.27:0/1622680183 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc27806bdf0 con 0x7fc27815b2b0 2024-09-12T03:21:34.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.685+0000 7fc25f7fe640 1 -- 172.21.15.27:0/1622680183 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v102) ==== 74+0+16726 (secure 0 0 0) 0x7fc2700585f0 con 0x7fc27815b2b0 2024-09-12T03:21:34.685 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:21:34.685 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":102,"fsid":"d2df504e-2647-4385-a5db-9fdf69f487a0","created":"2024-09-12T03:16:07.778937+0000","modified":"2024-09-12T03:21:16.858878+0000","last_up_change":"2024-09-12T03:16:19.314778+0000","last_in_change":"2024-09-12T03:16:09.893086+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T03:16:20.097646+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-09-12T03:20:56.805568+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-09-12T03:20:58.128117+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"100","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1851116c-1fb3-4dc9-9772-f90e2ef4c355","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6817","nonce":3002321488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6819","nonce":3002321488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6823","nonce":3002321488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6821","nonce":3002321488}]},"public_addr":"172.21.15.27:6817/3002321488","cluster_addr":"172.21.15.27:6819/3002321488","heartbeat_back_addr":"172.21.15.27:6823/3002321488","heartbeat_front_addr":"172.21.15.27:6821/3002321488","state":["exists","up"]},{"osd":1,"uuid":"f38ae17d-73eb-4204-b99d-975fa806fb87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6809","nonce":1613102886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6811","nonce":1613102886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6815","nonce":1613102886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6813","nonce":1613102886}]},"public_addr":"172.21.15.27:6809/1613102886","cluster_addr":"172.21.15.27:6811/1613102886","heartbeat_back_addr":"172.21.15.27:6815/1613102886","heartbeat_front_addr":"172.21.15.27:6813/1613102886","state":["exists","up"]},{"osd":2,"uuid":"fb12dcc8-bdd0-42b8-b46d-8b10106224bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6825","nonce":3534723355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6827","nonce":3534723355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6831","nonce":3534723355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6829","nonce":3534723355}]},"public_addr":"172.21.15.27:6825/3534723355","cluster_addr":"172.21.15.27:6827/3534723355","heartbeat_back_addr":"172.21.15.27:6831/3534723355","heartbeat_front_addr":"172.21.15.27:6829/3534723355","state":["exists","up"]},{"osd":3,"uuid":"d74d3f49-9c24-4b9f-98e2-3ecd7b5ef3b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6801","nonce":80661598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6803","nonce":80661598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6807","nonce":80661598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6805","nonce":80661598}]},"public_addr":"172.21.15.27:6801/80661598","cluster_addr":"172.21.15.27:6803/80661598","heartbeat_back_addr":"172.21.15.27:6807/80661598","heartbeat_front_addr":"172.21.15.27:6805/80661598","state":["exists","up"]},{"osd":4,"uuid":"2bf123f2-1b0d-41c4-aa3b-805bab267e59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6817","nonce":3943189827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6819","nonce":3943189827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6823","nonce":3943189827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6821","nonce":3943189827}]},"public_addr":"172.21.15.45:6817/3943189827","cluster_addr":"172.21.15.45:6819/3943189827","heartbeat_back_addr":"172.21.15.45:6823/3943189827","heartbeat_front_addr":"172.21.15.45:6821/3943189827","state":["exists","up"]},{"osd":5,"uuid":"c49ba5da-91c6-4a86-a97b-4aba89c253ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6809","nonce":3620760330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6811","nonce":3620760330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6815","nonce":3620760330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6813","nonce":3620760330}]},"public_addr":"172.21.15.45:6809/3620760330","cluster_addr":"172.21.15.45:6811/3620760330","heartbeat_back_addr":"172.21.15.45:6815/3620760330","heartbeat_front_addr":"172.21.15.45:6813/3620760330","state":["exists","up"]},{"osd":6,"uuid":"b0123475-89f7-4a34-9e33-d797b99dd9dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6801","nonce":1989083510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6803","nonce":1989083510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6807","nonce":1989083510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6805","nonce":1989083510}]},"public_addr":"172.21.15.45:6801/1989083510","cluster_addr":"172.21.15.45:6803/1989083510","heartbeat_back_addr":"172.21.15.45:6807/1989083510","heartbeat_front_addr":"172.21.15.45:6805/1989083510","state":["exists","up"]},{"osd":7,"uuid":"4dad773d-b7d8-46fc-8a10-a972aad27b29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6825","nonce":1513300294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6827","nonce":1513300294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6831","nonce":1513300294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6829","nonce":1513300294}]},"public_addr":"172.21.15.45:6825/1513300294","cluster_addr":"172.21.15.45:6827/1513300294","heartbeat_back_addr":"172.21.15.45:6831/1513300294","heartbeat_front_addr":"172.21.15.45:6829/1513300294","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.881855+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:17.133252+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.735339+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.500742+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.477571+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:17.167667+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.470291+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.873743+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"17.1a","mappings":[{"from":0,"to":2}]},{"pgid":"17.3f","mappings":[{"from":0,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T03:21:34.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.687+0000 7fc27d711640 1 -- 172.21.15.27:0/1622680183 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fc26406c220 msgr2=0x7fc26406e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:34.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.687+0000 7fc27d711640 1 --2- 172.21.15.27:0/1622680183 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fc26406c220 0x7fc26406e6e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fc268034000 tx=0x7fc268033040 comp rx=0 tx=0).stop 2024-09-12T03:21:34.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.687+0000 7fc27d711640 1 -- 172.21.15.27:0/1622680183 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc27815b2b0 msgr2=0x7fc2781f56a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:34.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.687+0000 7fc27d711640 1 --2- 172.21.15.27:0/1622680183 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc27815b2b0 0x7fc2781f56a0 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7fc270001eb0 tx=0x7fc27000c890 comp rx=0 tx=0).stop 2024-09-12T03:21:34.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.687+0000 7fc27d711640 1 -- 172.21.15.27:0/1622680183 shutdown_connections 2024-09-12T03:21:34.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.687+0000 7fc27d711640 1 --2- 172.21.15.27:0/1622680183 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fc26406c220 0x7fc26406e6e0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:34.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.687+0000 7fc27d711640 1 --2- 172.21.15.27:0/1622680183 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc27815b2b0 0x7fc2781f56a0 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:34.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.687+0000 7fc27d711640 1 --2- 172.21.15.27:0/1622680183 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fc278072d70 0x7fc2780acee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:34.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.687+0000 7fc27d711640 1 --2- 172.21.15.27:0/1622680183 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc27806e9b0 0x7fc2780cc140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:34.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.687+0000 7fc27d711640 1 -- 172.21.15.27:0/1622680183 >> 172.21.15.27:0/1622680183 conn(0x7fc278062c60 msgr2=0x7fc2780b01d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:34.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.687+0000 7fc27d711640 1 -- 172.21.15.27:0/1622680183 shutdown_connections 2024-09-12T03:21:34.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.687+0000 7fc27d711640 1 -- 172.21.15.27:0/1622680183 wait complete. 2024-09-12T03:21:34.699 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-12T03:21:34.699 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-12T03:21:34.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.802+0000 7f897ec17640 1 Processor -- start 2024-09-12T03:21:34.803 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.802+0000 7f897ec17640 1 -- start start 2024-09-12T03:21:34.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.803+0000 7f897ec17640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f897815e780 0x7f89780d3e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:34.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.803+0000 7f897ec17640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f89780b6690 con 0x7f89780d43b0 2024-09-12T03:21:34.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.803+0000 7f897ec17640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f89780b6970 con 0x7f89780d1990 2024-09-12T03:21:34.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.803+0000 7f897ec17640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f89780b7950 con 0x7f897815e780 2024-09-12T03:21:34.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.803+0000 7f897c9b3640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f897815e780 0x7f89780d3e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:34.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.803+0000 7f8977fff640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f89780d43b0 0x7f89780d1280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:51868/0 (socket says 172.21.15.27:51868) 2024-09-12T03:21:34.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.803+0000 7f8977fff640 1 -- 172.21.15.27:0/676095187 learned_addr learned my addr 172.21.15.27:0/676095187 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:21:34.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.804+0000 7f897c9b3640 1 -- 172.21.15.27:0/676095187 >> v1:172.21.15.45:6789/0 conn(0x7f89780d1990 legacy=0x7f89780b3390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:34.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.804+0000 7f897c9b3640 1 -- 172.21.15.27:0/676095187 >> v1:172.21.15.27:6789/0 conn(0x7f89780d43b0 legacy=0x7f89780d1280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:34.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.804+0000 7f897c9b3640 1 -- 172.21.15.27:0/676095187 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89780b6c50 con 0x7f897815e780 2024-09-12T03:21:34.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.804+0000 7f897c9b3640 1 --2- 172.21.15.27:0/676095187 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f897815e780 0x7f89780d3e70 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f89680097a0 tx=0x7f896802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddc3e8af5ab68532 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:34.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.805+0000 7f89777fe640 1 -- 172.21.15.27:0/676095187 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8968037070 con 0x7f897815e780 2024-09-12T03:21:34.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.805+0000 7f89777fe640 1 -- 172.21.15.27:0/676095187 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8968032440 con 0x7f897815e780 2024-09-12T03:21:34.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.805+0000 7f89777fe640 1 -- 172.21.15.27:0/676095187 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8968032810 con 0x7f897815e780 2024-09-12T03:21:34.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.805+0000 7f897ec17640 1 -- 172.21.15.27:0/676095187 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f897815e780 msgr2=0x7f89780d3e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:34.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.805+0000 7f897ec17640 1 --2- 172.21.15.27:0/676095187 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f897815e780 0x7f89780d3e70 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f89680097a0 tx=0x7f896802a1a0 comp rx=0 tx=0).stop 2024-09-12T03:21:34.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.805+0000 7f897ec17640 1 -- 172.21.15.27:0/676095187 shutdown_connections 2024-09-12T03:21:34.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.805+0000 7f897ec17640 1 --2- 172.21.15.27:0/676095187 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f897815e780 0x7f89780d3e70 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:34.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.805+0000 7f897ec17640 1 -- 172.21.15.27:0/676095187 >> 172.21.15.27:0/676095187 conn(0x7f897806da60 msgr2=0x7f897806de70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:34.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.805+0000 7f897ec17640 1 -- 172.21.15.27:0/676095187 shutdown_connections 2024-09-12T03:21:34.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.805+0000 7f897ec17640 1 -- 172.21.15.27:0/676095187 wait complete. 2024-09-12T03:21:34.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.806+0000 7f897ec17640 1 Processor -- start 2024-09-12T03:21:34.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.806+0000 7f897ec17640 1 -- start start 2024-09-12T03:21:34.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.807+0000 7f897ec17640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f89780d1990 0x7f89780cf0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:34.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.807+0000 7f897ec17640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f89780d43b0 0x7f89780cf610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:34.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.807+0000 7f897ec17640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f897815e780 0x7f89781f4260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:34.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.807+0000 7f897ec17640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f89780b6340 con 0x7f89780d1990 2024-09-12T03:21:34.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.807+0000 7f897ec17640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f89780b7210 con 0x7f89780d43b0 2024-09-12T03:21:34.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.807+0000 7f897ec17640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f89780b8dd0 con 0x7f897815e780 2024-09-12T03:21:34.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.807+0000 7f897c9b3640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f89780d1990 0x7f89780cf0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:34.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.807+0000 7f897c9b3640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f89780d1990 0x7f89780cf0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41952/0 (socket says 172.21.15.27:41952) 2024-09-12T03:21:34.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.808+0000 7f897c9b3640 1 -- 172.21.15.27:0/207093871 learned_addr learned my addr 172.21.15.27:0/207093871 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:21:34.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.808+0000 7f8977fff640 1 --2- 172.21.15.27:0/207093871 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f89780d43b0 0x7f89780cf610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:34.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.808+0000 7f897d1b4640 1 --2- 172.21.15.27:0/207093871 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f897815e780 0x7f89781f4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:34.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.808+0000 7f897c9b3640 1 -- 172.21.15.27:0/207093871 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f897815e780 msgr2=0x7f89781f4260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:34.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.808+0000 7f897c9b3640 1 --2- 172.21.15.27:0/207093871 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f897815e780 0x7f89781f4260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:34.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.808+0000 7f897c9b3640 1 -- 172.21.15.27:0/207093871 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f89780d43b0 msgr2=0x7f89780cf610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:34.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.808+0000 7f897c9b3640 1 --2- 172.21.15.27:0/207093871 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f89780d43b0 0x7f89780cf610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:34.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.808+0000 7f897c9b3640 1 -- 172.21.15.27:0/207093871 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89781e8130 con 0x7f89780d1990 2024-09-12T03:21:34.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.808+0000 7f8977fff640 1 --2- 172.21.15.27:0/207093871 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f89780d43b0 0x7f89780cf610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:21:34.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.808+0000 7f897d1b4640 1 --2- 172.21.15.27:0/207093871 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f897815e780 0x7f89781f4260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:21:34.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.808+0000 7f897c9b3640 1 --2- 172.21.15.27:0/207093871 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f89780d1990 0x7f89780cf0d0 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f8968035040 tx=0x7f89680046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:34.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.808+0000 7f89757fa640 1 -- 172.21.15.27:0/207093871 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8968037040 con 0x7f89780d1990 2024-09-12T03:21:34.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.808+0000 7f89757fa640 1 -- 172.21.15.27:0/207093871 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8968030770 con 0x7f89780d1990 2024-09-12T03:21:34.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.808+0000 7f897ec17640 1 -- 172.21.15.27:0/207093871 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89781e8410 con 0x7f89780d1990 2024-09-12T03:21:34.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.808+0000 7f89757fa640 1 -- 172.21.15.27:0/207093871 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8968030b80 con 0x7f89780d1990 2024-09-12T03:21:34.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.808+0000 7f897ec17640 1 -- 172.21.15.27:0/207093871 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89781e89d0 con 0x7f89780d1990 2024-09-12T03:21:34.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.809+0000 7f897ec17640 1 -- 172.21.15.27:0/207093871 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f897815eb60 con 0x7f89780d1990 2024-09-12T03:21:34.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.810+0000 7f89757fa640 1 -- 172.21.15.27:0/207093871 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 19) ==== 90855+0+0 (secure 0 0 0) 0x7f8968044020 con 0x7f89780d1990 2024-09-12T03:21:34.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.810+0000 7f89757fa640 1 --2- 172.21.15.27:0/207093871 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f896406c220 0x7f896406e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:34.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.810+0000 7f89757fa640 1 -- 172.21.15.27:0/207093871 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5767+0+0 (secure 0 0 0) 0x7f89680aba30 con 0x7f89780d1990 2024-09-12T03:21:34.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.810+0000 7f8977fff640 1 --2- 172.21.15.27:0/207093871 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f896406c220 0x7f896406e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:34.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.811+0000 7f8977fff640 1 --2- 172.21.15.27:0/207093871 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f896406c220 0x7f896406e6e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f89781e9c10 tx=0x7f896c009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:34.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:34.815+0000 7f89757fa640 1 -- 172.21.15.27:0/207093871 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f8968032e20 con 0x7f89780d1990 2024-09-12T03:21:35.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.027+0000 7f897ec17640 1 -- 172.21.15.27:0/207093871 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f89781e7e20 con 0x7f89780d1990 2024-09-12T03:21:35.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.028+0000 7f89757fa640 1 -- 172.21.15.27:0/207093871 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 59 v59) ==== 94+0+5222 (secure 0 0 0) 0x7f8968079f60 con 0x7f89780d1990 2024-09-12T03:21:35.028 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:21:35.028 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":59,"btime":"2024-09-12T03:21:07:819282+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5885,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/2343815091","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2343815091},{"type":"v1","addr":"172.21.15.45:6834","nonce":2343815091}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":55},{"gid":6376,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/512483568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":512483568},{"type":"v1","addr":"172.21.15.45:6835","nonce":512483568}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":55},{"gid":6825,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6834/1888895227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":1888895227},{"type":"v1","addr":"172.21.15.27:6834","nonce":1888895227}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":54}],"filesystems":[{"mdsmap":{"epoch":58,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-12T03:20:59.133824+0000","modified":"2024-09-12T03:21:01.148633+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6828},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6828":{"gid":6828,"name":"a","rank":0,"incarnation":56,"state":"up:active","state_seq":3,"addr":"172.21.15.27:6835/1379166526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":1379166526},{"type":"v1","addr":"172.21.15.27:6835","nonce":1379166526}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6828,"qdb_cluster":[6828]},"id":8}]} 2024-09-12T03:21:35.029 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 59 2024-09-12T03:21:35.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.030+0000 7f897ec17640 1 -- 172.21.15.27:0/207093871 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f896406c220 msgr2=0x7f896406e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:35.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.030+0000 7f897ec17640 1 --2- 172.21.15.27:0/207093871 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f896406c220 0x7f896406e6e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f89781e9c10 tx=0x7f896c009040 comp rx=0 tx=0).stop 2024-09-12T03:21:35.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.030+0000 7f897ec17640 1 -- 172.21.15.27:0/207093871 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f89780d1990 msgr2=0x7f89780cf0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:35.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.030+0000 7f897ec17640 1 --2- 172.21.15.27:0/207093871 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f89780d1990 0x7f89780cf0d0 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f8968035040 tx=0x7f89680046c0 comp rx=0 tx=0).stop 2024-09-12T03:21:35.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.030+0000 7f897ec17640 1 -- 172.21.15.27:0/207093871 shutdown_connections 2024-09-12T03:21:35.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.030+0000 7f897ec17640 1 --2- 172.21.15.27:0/207093871 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f897815e780 0x7f89781f4260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:35.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.030+0000 7f897ec17640 1 --2- 172.21.15.27:0/207093871 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f89780d43b0 0x7f89780cf610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:35.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.030+0000 7f897ec17640 1 --2- 172.21.15.27:0/207093871 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f896406c220 0x7f896406e6e0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:35.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.030+0000 7f897ec17640 1 --2- 172.21.15.27:0/207093871 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f89780d1990 0x7f89780cf0d0 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:35.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.030+0000 7f897ec17640 1 -- 172.21.15.27:0/207093871 >> 172.21.15.27:0/207093871 conn(0x7f897806da60 msgr2=0x7f89781619a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:35.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.030+0000 7f897ec17640 1 -- 172.21.15.27:0/207093871 shutdown_connections 2024-09-12T03:21:35.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.030+0000 7f897ec17640 1 -- 172.21.15.27:0/207093871 wait complete. 2024-09-12T03:21:35.041 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-12T03:21:35.143 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.143+0000 7fc0cd89b640 1 Processor -- start 2024-09-12T03:21:35.143 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.143+0000 7fc0cd89b640 1 -- start start 2024-09-12T03:21:35.145 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.144+0000 7fc0cd89b640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fc0c815eb90 0x7fc0c80d2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:35.145 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.144+0000 7fc0cd89b640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fc0c80d5200 0x7fc0c80d1010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:35.145 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.144+0000 7fc0cd89b640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc0c80d19c0 con 0x7fc0c80d1550 2024-09-12T03:21:35.145 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.144+0000 7fc0cd89b640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7fc0c80b7d20 con 0x7fc0c815eb90 2024-09-12T03:21:35.145 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.144+0000 7fc0cd89b640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fc0c80d42e0 con 0x7fc0c80d5200 2024-09-12T03:21:35.145 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.144+0000 7fc0c6ffd640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fc0c815eb90 0x7fc0c80d2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:35.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.144+0000 7fc0c77fe640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fc0c80d1550 0x7fc0c80cf200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:51874/0 (socket says 172.21.15.27:51874) 2024-09-12T03:21:35.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.144+0000 7fc0c77fe640 1 -- 172.21.15.27:0/4178510746 learned_addr learned my addr 172.21.15.27:0/4178510746 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:21:35.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.144+0000 7fc0c6ffd640 1 -- 172.21.15.27:0/4178510746 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fc0c80d5200 msgr2=0x7fc0c80d1010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:35.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.144+0000 7fc0c67fc640 1 --2- 172.21.15.27:0/4178510746 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fc0c80d5200 0x7fc0c80d1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:35.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.144+0000 7fc0c5ffb640 1 -- 172.21.15.27:0/4178510746 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1846482933 0 0) 0x7fc0c80d19c0 con 0x7fc0c80d1550 2024-09-12T03:21:35.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.144+0000 7fc0c6ffd640 1 --2- 172.21.15.27:0/4178510746 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fc0c80d5200 0x7fc0c80d1010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:35.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.144+0000 7fc0c6ffd640 1 -- 172.21.15.27:0/4178510746 >> v1:172.21.15.27:6789/0 conn(0x7fc0c80d1550 legacy=0x7fc0c80cf200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:35.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.144+0000 7fc0c6ffd640 1 -- 172.21.15.27:0/4178510746 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0c80d3760 con 0x7fc0c815eb90 2024-09-12T03:21:35.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.144+0000 7fc0c67fc640 1 --2- 172.21.15.27:0/4178510746 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fc0c80d5200 0x7fc0c80d1010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:21:35.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.145+0000 7fc0c6ffd640 1 --2- 172.21.15.27:0/4178510746 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc0c815eb90 0x7fc0c80d2df0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7fc0b80097a0 tx=0x7fc0b802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86696b89cc82c7a9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:35.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.145+0000 7fc0c5ffb640 1 -- 172.21.15.27:0/4178510746 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0b8037070 con 0x7fc0c815eb90 2024-09-12T03:21:35.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.145+0000 7fc0c5ffb640 1 -- 172.21.15.27:0/4178510746 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0b8032440 con 0x7fc0c815eb90 2024-09-12T03:21:35.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.145+0000 7fc0c5ffb640 1 -- 172.21.15.27:0/4178510746 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0b8032810 con 0x7fc0c815eb90 2024-09-12T03:21:35.149 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.146+0000 7fc0cd89b640 1 -- 172.21.15.27:0/4178510746 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc0c815eb90 msgr2=0x7fc0c80d2df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:35.149 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.146+0000 7fc0cd89b640 1 --2- 172.21.15.27:0/4178510746 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc0c815eb90 0x7fc0c80d2df0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7fc0b80097a0 tx=0x7fc0b802a180 comp rx=0 tx=0).stop 2024-09-12T03:21:35.149 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.146+0000 7fc0cd89b640 1 -- 172.21.15.27:0/4178510746 shutdown_connections 2024-09-12T03:21:35.149 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.146+0000 7fc0cd89b640 1 --2- 172.21.15.27:0/4178510746 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fc0c80d5200 0x7fc0c80d1010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:35.149 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.146+0000 7fc0cd89b640 1 --2- 172.21.15.27:0/4178510746 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc0c815eb90 0x7fc0c80d2df0 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:35.149 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.146+0000 7fc0cd89b640 1 -- 172.21.15.27:0/4178510746 >> 172.21.15.27:0/4178510746 conn(0x7fc0c806dce0 msgr2=0x7fc0c806e0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:35.149 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.146+0000 7fc0cd89b640 1 -- 172.21.15.27:0/4178510746 shutdown_connections 2024-09-12T03:21:35.150 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.146+0000 7fc0cd89b640 1 -- 172.21.15.27:0/4178510746 wait complete. 2024-09-12T03:21:35.150 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.147+0000 7fc0cd89b640 1 Processor -- start 2024-09-12T03:21:35.150 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.147+0000 7fc0cd89b640 1 -- start start 2024-09-12T03:21:35.150 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.148+0000 7fc0cd89b640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fc0c80d1550 0x7fc0c8156b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:35.150 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.148+0000 7fc0cd89b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc0c80d5200 0x7fc0c80cbfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:35.150 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.148+0000 7fc0cd89b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc0c815eb90 0x7fc0c80cc4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:35.150 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.148+0000 7fc0cd89b640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fc0c80d4000 con 0x7fc0c815eb90 2024-09-12T03:21:35.150 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.148+0000 7fc0cd89b640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc0c80d3d20 con 0x7fc0c80d5200 2024-09-12T03:21:35.150 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.148+0000 7fc0cd89b640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fc0c80d3330 con 0x7fc0c80d1550 2024-09-12T03:21:35.150 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.148+0000 7fc0c77fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc0c815eb90 0x7fc0c80cc4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:35.150 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.148+0000 7fc0c77fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc0c815eb90 0x7fc0c80cc4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41974/0 (socket says 172.21.15.27:41974) 2024-09-12T03:21:35.150 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.148+0000 7fc0c77fe640 1 -- 172.21.15.27:0/558179713 learned_addr learned my addr 172.21.15.27:0/558179713 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:21:35.150 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.148+0000 7fc0c6ffd640 1 --2- 172.21.15.27:0/558179713 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fc0c80d1550 0x7fc0c8156b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:35.150 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.148+0000 7fc0c67fc640 1 --2- 172.21.15.27:0/558179713 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc0c80d5200 0x7fc0c80cbfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:35.151 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.148+0000 7fc0c77fe640 1 -- 172.21.15.27:0/558179713 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fc0c80d1550 msgr2=0x7fc0c8156b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:35.151 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.148+0000 7fc0c77fe640 1 --2- 172.21.15.27:0/558179713 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fc0c80d1550 0x7fc0c8156b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:35.151 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.148+0000 7fc0c77fe640 1 -- 172.21.15.27:0/558179713 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc0c80d5200 msgr2=0x7fc0c80cbfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:35.151 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.148+0000 7fc0c77fe640 1 --2- 172.21.15.27:0/558179713 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc0c80d5200 0x7fc0c80cbfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:35.151 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.148+0000 7fc0c77fe640 1 -- 172.21.15.27:0/558179713 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0c814d050 con 0x7fc0c815eb90 2024-09-12T03:21:35.151 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.148+0000 7fc0c6ffd640 1 --2- 172.21.15.27:0/558179713 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fc0c80d1550 0x7fc0c8156b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:21:35.151 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.148+0000 7fc0c67fc640 1 --2- 172.21.15.27:0/558179713 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc0c80d5200 0x7fc0c80cbfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:21:35.151 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.149+0000 7fc0c77fe640 1 --2- 172.21.15.27:0/558179713 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc0c815eb90 0x7fc0c80cc4f0 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7fc0c000c830 tx=0x7fc0c000ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:35.151 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.149+0000 7fc0affff640 1 -- 172.21.15.27:0/558179713 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0c0018070 con 0x7fc0c815eb90 2024-09-12T03:21:35.151 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.149+0000 7fc0affff640 1 -- 172.21.15.27:0/558179713 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0c000ce20 con 0x7fc0c815eb90 2024-09-12T03:21:35.151 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.149+0000 7fc0cd89b640 1 -- 172.21.15.27:0/558179713 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0c80cac90 con 0x7fc0c815eb90 2024-09-12T03:21:35.151 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.149+0000 7fc0affff640 1 -- 172.21.15.27:0/558179713 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0c000a880 con 0x7fc0c815eb90 2024-09-12T03:21:35.151 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.149+0000 7fc0cd89b640 1 -- 172.21.15.27:0/558179713 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0c80ca1b0 con 0x7fc0c815eb90 2024-09-12T03:21:35.152 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.150+0000 7fc0cd89b640 1 -- 172.21.15.27:0/558179713 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0c815ef70 con 0x7fc0c815eb90 2024-09-12T03:21:35.152 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.150+0000 7fc0affff640 1 -- 172.21.15.27:0/558179713 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 19) ==== 90855+0+0 (secure 0 0 0) 0x7fc0c001b070 con 0x7fc0c815eb90 2024-09-12T03:21:35.153 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.151+0000 7fc0affff640 1 --2- 172.21.15.27:0/558179713 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fc0b406c250 0x7fc0b406e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:35.153 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.151+0000 7fc0affff640 1 -- 172.21.15.27:0/558179713 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5767+0+0 (secure 0 0 0) 0x7fc0c008aab0 con 0x7fc0c815eb90 2024-09-12T03:21:35.153 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.151+0000 7fc0c6ffd640 1 --2- 172.21.15.27:0/558179713 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fc0b406c250 0x7fc0b406e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:35.153 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.151+0000 7fc0c6ffd640 1 --2- 172.21.15.27:0/558179713 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fc0b406c250 0x7fc0b406e710 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fc0b8034000 tx=0x7fc0b802e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:35.155 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.155+0000 7fc0affff640 1 -- 172.21.15.27:0/558179713 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fc0c0058f00 con 0x7fc0c815eb90 2024-09-12T03:21:35.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.380+0000 7fc0cd89b640 1 -- 172.21.15.27:0/558179713 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc0c80d2fa0 con 0x7fc0c815eb90 2024-09-12T03:21:35.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.381+0000 7fc0affff640 1 -- 172.21.15.27:0/558179713 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 59 v59) ==== 94+0+5222 (secure 0 0 0) 0x7fc0c0058800 con 0x7fc0c815eb90 2024-09-12T03:21:35.381 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:21:35.381 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":59,"btime":"2024-09-12T03:21:07:819282+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5885,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/2343815091","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2343815091},{"type":"v1","addr":"172.21.15.45:6834","nonce":2343815091}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":55},{"gid":6376,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/512483568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":512483568},{"type":"v1","addr":"172.21.15.45:6835","nonce":512483568}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":55},{"gid":6825,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6834/1888895227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":1888895227},{"type":"v1","addr":"172.21.15.27:6834","nonce":1888895227}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":54}],"filesystems":[{"mdsmap":{"epoch":58,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-12T03:20:59.133824+0000","modified":"2024-09-12T03:21:01.148633+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6828},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6828":{"gid":6828,"name":"a","rank":0,"incarnation":56,"state":"up:active","state_seq":3,"addr":"172.21.15.27:6835/1379166526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":1379166526},{"type":"v1","addr":"172.21.15.27:6835","nonce":1379166526}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6828,"qdb_cluster":[6828]},"id":8}]} 2024-09-12T03:21:35.381 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 59 2024-09-12T03:21:35.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.383+0000 7fc0cd89b640 1 -- 172.21.15.27:0/558179713 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fc0b406c250 msgr2=0x7fc0b406e710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:35.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.383+0000 7fc0cd89b640 1 --2- 172.21.15.27:0/558179713 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fc0b406c250 0x7fc0b406e710 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fc0b8034000 tx=0x7fc0b802e000 comp rx=0 tx=0).stop 2024-09-12T03:21:35.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.383+0000 7fc0cd89b640 1 -- 172.21.15.27:0/558179713 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc0c815eb90 msgr2=0x7fc0c80cc4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:35.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.383+0000 7fc0cd89b640 1 --2- 172.21.15.27:0/558179713 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc0c815eb90 0x7fc0c80cc4f0 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7fc0c000c830 tx=0x7fc0c000ccf0 comp rx=0 tx=0).stop 2024-09-12T03:21:35.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.383+0000 7fc0cd89b640 1 -- 172.21.15.27:0/558179713 shutdown_connections 2024-09-12T03:21:35.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.383+0000 7fc0cd89b640 1 --2- 172.21.15.27:0/558179713 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fc0b406c250 0x7fc0b406e710 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:35.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.383+0000 7fc0cd89b640 1 --2- 172.21.15.27:0/558179713 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc0c815eb90 0x7fc0c80cc4f0 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:35.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.383+0000 7fc0cd89b640 1 --2- 172.21.15.27:0/558179713 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc0c80d5200 0x7fc0c80cbfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:35.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.383+0000 7fc0cd89b640 1 --2- 172.21.15.27:0/558179713 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fc0c80d1550 0x7fc0c8156b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:35.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.383+0000 7fc0cd89b640 1 -- 172.21.15.27:0/558179713 >> 172.21.15.27:0/558179713 conn(0x7fc0c806dce0 msgr2=0x7fc0c815fd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:35.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.383+0000 7fc0cd89b640 1 -- 172.21.15.27:0/558179713 shutdown_connections 2024-09-12T03:21:35.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.383+0000 7fc0cd89b640 1 -- 172.21.15.27:0/558179713 wait complete. 2024-09-12T03:21:35.394 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-12T03:21:35.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.495+0000 7f7f8a47b640 1 Processor -- start 2024-09-12T03:21:35.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.496+0000 7f7f8a47b640 1 -- start start 2024-09-12T03:21:35.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.496+0000 7f7f8a47b640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f7f840b91a0 0x7f7f84169880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:35.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.496+0000 7f7f8a47b640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7f840d3760 con 0x7f7f8415eb90 2024-09-12T03:21:35.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.496+0000 7f7f8a47b640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7f840d3a40 con 0x7f7f840d5200 2024-09-12T03:21:35.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.496+0000 7f7f8a47b640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f7f840b7d20 con 0x7f7f840b91a0 2024-09-12T03:21:35.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.496+0000 7f7f83fff640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f7f8415eb90 0x7f7f840d2df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:51884/0 (socket says 172.21.15.27:51884) 2024-09-12T03:21:35.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.496+0000 7f7f83fff640 1 -- 172.21.15.27:0/171329340 learned_addr learned my addr 172.21.15.27:0/171329340 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:21:35.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.497+0000 7f7f82ffd640 1 -- 172.21.15.27:0/171329340 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1880447563 0 0) 0x7f7f840d3760 con 0x7f7f8415eb90 2024-09-12T03:21:35.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.497+0000 7f7f82ffd640 1 -- 172.21.15.27:0/171329340 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7f6c003520 con 0x7f7f8415eb90 2024-09-12T03:21:35.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.497+0000 7f7f88a18640 1 --2- 172.21.15.27:0/171329340 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f7f840b91a0 0x7f7f84169880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:35.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.497+0000 7f7f82ffd640 1 -- 172.21.15.27:0/171329340 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1088196647 0 0) 0x7f7f6c003520 con 0x7f7f8415eb90 2024-09-12T03:21:35.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.497+0000 7f7f82ffd640 1 -- 172.21.15.27:0/171329340 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f7f840b91a0 msgr2=0x7f7f84169880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:35.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.497+0000 7f7f82ffd640 1 --2- 172.21.15.27:0/171329340 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f7f840b91a0 0x7f7f84169880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:35.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.497+0000 7f7f82ffd640 1 -- 172.21.15.27:0/171329340 >> v1:172.21.15.45:6789/0 conn(0x7f7f840d5200 legacy=0x7f7f840d1010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:35.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.497+0000 7f7f82ffd640 1 -- 172.21.15.27:0/171329340 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f840d3d20 con 0x7f7f8415eb90 2024-09-12T03:21:35.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.497+0000 7f7f82ffd640 1 -- 172.21.15.27:0/171329340 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 847755213 0 0) 0x7f7f74001670 con 0x7f7f8415eb90 2024-09-12T03:21:35.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.497+0000 7f7f82ffd640 1 -- 172.21.15.27:0/171329340 >> v1:172.21.15.27:6789/0 conn(0x7f7f8415eb90 legacy=0x7f7f840d2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:35.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.497+0000 7f7f82ffd640 1 --2- 172.21.15.27:0/171329340 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7f6c0039e0 0x7f7f6c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:35.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.497+0000 7f7f82ffd640 1 --2- 172.21.15.27:0/171329340 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7f6c004800 0x7f7f6c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:35.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.497+0000 7f7f82ffd640 1 --2- 172.21.15.27:0/171329340 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f7f840b91a0 0x7f7f6c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:35.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.497+0000 7f7f82ffd640 1 -- 172.21.15.27:0/171329340 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f7f6c00ba70 con 0x7f7f6c0039e0 2024-09-12T03:21:35.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.497+0000 7f7f82ffd640 1 -- 172.21.15.27:0/171329340 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f7f6c00bbf0 con 0x7f7f6c004800 2024-09-12T03:21:35.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.497+0000 7f7f82ffd640 1 -- 172.21.15.27:0/171329340 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f7f6c00bdf0 con 0x7f7f840b91a0 2024-09-12T03:21:35.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.497+0000 7f7f83fff640 1 --2- 172.21.15.27:0/171329340 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7f6c0039e0 0x7f7f6c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:35.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.497+0000 7f7f88a18640 1 --2- 172.21.15.27:0/171329340 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f7f840b91a0 0x7f7f6c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:35.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.497+0000 7f7f837fe640 1 --2- 172.21.15.27:0/171329340 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7f6c004800 0x7f7f6c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:35.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.498+0000 7f7f83fff640 1 -- 172.21.15.27:0/171329340 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f7f840b91a0 msgr2=0x7f7f6c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:35.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.498+0000 7f7f83fff640 1 --2- 172.21.15.27:0/171329340 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f7f840b91a0 0x7f7f6c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:35.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.498+0000 7f7f83fff640 1 -- 172.21.15.27:0/171329340 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7f6c004800 msgr2=0x7f7f6c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:35.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.498+0000 7f7f83fff640 1 --2- 172.21.15.27:0/171329340 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7f6c004800 0x7f7f6c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:35.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.498+0000 7f7f83fff640 1 -- 172.21.15.27:0/171329340 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f6c003520 con 0x7f7f6c0039e0 2024-09-12T03:21:35.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.498+0000 7f7f83fff640 1 --2- 172.21.15.27:0/171329340 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7f6c0039e0 0x7f7f6c003e00 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f7f7400c210 tx=0x7f7f6c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dfed15910cbbdfb server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:35.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.498+0000 7f7f82ffd640 1 -- 172.21.15.27:0/171329340 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f74007d10 con 0x7f7f6c0039e0 2024-09-12T03:21:35.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.498+0000 7f7f82ffd640 1 -- 172.21.15.27:0/171329340 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f74005ad0 con 0x7f7f6c0039e0 2024-09-12T03:21:35.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.498+0000 7f7f82ffd640 1 -- 172.21.15.27:0/171329340 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f74007070 con 0x7f7f6c0039e0 2024-09-12T03:21:35.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.499+0000 7f7f8a47b640 1 -- 172.21.15.27:0/171329340 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7f6c0039e0 msgr2=0x7f7f6c003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:35.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.499+0000 7f7f8a47b640 1 --2- 172.21.15.27:0/171329340 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7f6c0039e0 0x7f7f6c003e00 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f7f7400c210 tx=0x7f7f6c005830 comp rx=0 tx=0).stop 2024-09-12T03:21:35.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.499+0000 7f7f88a18640 1 -- 172.21.15.27:0/171329340 reap_dead start 2024-09-12T03:21:35.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.499+0000 7f7f8a47b640 1 -- 172.21.15.27:0/171329340 shutdown_connections 2024-09-12T03:21:35.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.499+0000 7f7f8a47b640 1 -- 172.21.15.27:0/171329340 >> 172.21.15.27:0/171329340 conn(0x7f7f8406dce0 msgr2=0x7f7f8406e0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:35.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.499+0000 7f7f8a47b640 1 -- 172.21.15.27:0/171329340 shutdown_connections 2024-09-12T03:21:35.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.499+0000 7f7f8a47b640 1 -- 172.21.15.27:0/171329340 wait complete. 2024-09-12T03:21:35.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.500+0000 7f7f8a47b640 1 Processor -- start 2024-09-12T03:21:35.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.500+0000 7f7f8a47b640 1 -- start start 2024-09-12T03:21:35.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.501+0000 7f7f8a47b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7f6c0039e0 0x7f7f840cf0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:35.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.501+0000 7f7f8a47b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7f8414c790 0x7f7f8414cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:35.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.501+0000 7f7f8a47b640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f7f841f4a20 0x7f7f841f4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:35.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.501+0000 7f7f8a47b640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f7f840d3500 con 0x7f7f8414c790 2024-09-12T03:21:35.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.501+0000 7f7f8a47b640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f7f840d42e0 con 0x7f7f6c0039e0 2024-09-12T03:21:35.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.501+0000 7f7f8a47b640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f7f8415ef70 con 0x7f7f841f4a20 2024-09-12T03:21:35.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.501+0000 7f7f837fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7f8414c790 0x7f7f8414cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:35.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.501+0000 7f7f837fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7f8414c790 0x7f7f8414cc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41986/0 (socket says 172.21.15.27:41986) 2024-09-12T03:21:35.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.501+0000 7f7f837fe640 1 -- 172.21.15.27:0/4048750625 learned_addr learned my addr 172.21.15.27:0/4048750625 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:21:35.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.501+0000 7f7f83fff640 1 --2- 172.21.15.27:0/4048750625 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7f6c0039e0 0x7f7f840cf0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:35.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.501+0000 7f7f88a18640 1 --2- 172.21.15.27:0/4048750625 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f7f841f4a20 0x7f7f841f4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:35.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.502+0000 7f7f837fe640 1 -- 172.21.15.27:0/4048750625 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f7f841f4a20 msgr2=0x7f7f841f4ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:35.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.502+0000 7f7f837fe640 1 --2- 172.21.15.27:0/4048750625 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f7f841f4a20 0x7f7f841f4ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:35.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.502+0000 7f7f837fe640 1 -- 172.21.15.27:0/4048750625 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7f6c0039e0 msgr2=0x7f7f840cf0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:35.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.502+0000 7f7f837fe640 1 --2- 172.21.15.27:0/4048750625 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7f6c0039e0 0x7f7f840cf0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:35.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.502+0000 7f7f837fe640 1 -- 172.21.15.27:0/4048750625 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f841f58d0 con 0x7f7f8414c790 2024-09-12T03:21:35.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.502+0000 7f7f837fe640 1 --2- 172.21.15.27:0/4048750625 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7f8414c790 0x7f7f8414cc10 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f7f7800e4e0 tx=0x7f7f7800e9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:35.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.502+0000 7f7f80ff9640 1 -- 172.21.15.27:0/4048750625 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f780052e0 con 0x7f7f8414c790 2024-09-12T03:21:35.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.502+0000 7f7f80ff9640 1 -- 172.21.15.27:0/4048750625 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f78018070 con 0x7f7f8414c790 2024-09-12T03:21:35.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.502+0000 7f7f8a47b640 1 -- 172.21.15.27:0/4048750625 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f841596f0 con 0x7f7f8414c790 2024-09-12T03:21:35.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.502+0000 7f7f80ff9640 1 -- 172.21.15.27:0/4048750625 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f78003810 con 0x7f7f8414c790 2024-09-12T03:21:35.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.502+0000 7f7f8a47b640 1 -- 172.21.15.27:0/4048750625 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f84159c90 con 0x7f7f8414c790 2024-09-12T03:21:35.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.503+0000 7f7f8a47b640 1 -- 172.21.15.27:0/4048750625 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f840d17b0 con 0x7f7f8414c790 2024-09-12T03:21:35.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.504+0000 7f7f80ff9640 1 -- 172.21.15.27:0/4048750625 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 19) ==== 90855+0+0 (secure 0 0 0) 0x7f7f780039b0 con 0x7f7f8414c790 2024-09-12T03:21:35.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.504+0000 7f7f80ff9640 1 --2- 172.21.15.27:0/4048750625 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f7f7006c220 0x7f7f7006e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:35.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.504+0000 7f7f80ff9640 1 -- 172.21.15.27:0/4048750625 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5823+0+0 (secure 0 0 0) 0x7f7f7808b140 con 0x7f7f8414c790 2024-09-12T03:21:35.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.504+0000 7f7f83fff640 1 --2- 172.21.15.27:0/4048750625 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f7f7006c220 0x7f7f7006e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:35.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.505+0000 7f7f83fff640 1 --2- 172.21.15.27:0/4048750625 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f7f7006c220 0x7f7f7006e6e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f7f74032000 tx=0x7f7f74034040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:35.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.509+0000 7f7f80ff9640 1 -- 172.21.15.27:0/4048750625 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f7f78058f10 con 0x7f7f8414c790 2024-09-12T03:21:35.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.684+0000 7f7f8a47b640 1 -- 172.21.15.27:0/4048750625 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7f841610e0 con 0x7f7f8414c790 2024-09-12T03:21:35.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.686+0000 7f7f80ff9640 1 -- 172.21.15.27:0/4048750625 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v103) ==== 74+0+16837 (secure 0 0 0) 0x7f7f7805d210 con 0x7f7f8414c790 2024-09-12T03:21:35.686 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:21:35.686 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":103,"fsid":"d2df504e-2647-4385-a5db-9fdf69f487a0","created":"2024-09-12T03:16:07.778937+0000","modified":"2024-09-12T03:21:35.197714+0000","last_up_change":"2024-09-12T03:16:19.314778+0000","last_in_change":"2024-09-12T03:16:09.893086+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T03:16:20.097646+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-09-12T03:20:56.805568+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-09-12T03:20:58.128117+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":5,"snap_epoch":103,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1851116c-1fb3-4dc9-9772-f90e2ef4c355","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6817","nonce":3002321488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6819","nonce":3002321488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6823","nonce":3002321488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6821","nonce":3002321488}]},"public_addr":"172.21.15.27:6817/3002321488","cluster_addr":"172.21.15.27:6819/3002321488","heartbeat_back_addr":"172.21.15.27:6823/3002321488","heartbeat_front_addr":"172.21.15.27:6821/3002321488","state":["exists","up"]},{"osd":1,"uuid":"f38ae17d-73eb-4204-b99d-975fa806fb87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6809","nonce":1613102886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6811","nonce":1613102886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6815","nonce":1613102886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6813","nonce":1613102886}]},"public_addr":"172.21.15.27:6809/1613102886","cluster_addr":"172.21.15.27:6811/1613102886","heartbeat_back_addr":"172.21.15.27:6815/1613102886","heartbeat_front_addr":"172.21.15.27:6813/1613102886","state":["exists","up"]},{"osd":2,"uuid":"fb12dcc8-bdd0-42b8-b46d-8b10106224bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6825","nonce":3534723355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6827","nonce":3534723355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6831","nonce":3534723355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6829","nonce":3534723355}]},"public_addr":"172.21.15.27:6825/3534723355","cluster_addr":"172.21.15.27:6827/3534723355","heartbeat_back_addr":"172.21.15.27:6831/3534723355","heartbeat_front_addr":"172.21.15.27:6829/3534723355","state":["exists","up"]},{"osd":3,"uuid":"d74d3f49-9c24-4b9f-98e2-3ecd7b5ef3b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6801","nonce":80661598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6803","nonce":80661598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6807","nonce":80661598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6805","nonce":80661598}]},"public_addr":"172.21.15.27:6801/80661598","cluster_addr":"172.21.15.27:6803/80661598","heartbeat_back_addr":"172.21.15.27:6807/80661598","heartbeat_front_addr":"172.21.15.27:6805/80661598","state":["exists","up"]},{"osd":4,"uuid":"2bf123f2-1b0d-41c4-aa3b-805bab267e59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6817","nonce":3943189827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6819","nonce":3943189827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6823","nonce":3943189827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6821","nonce":3943189827}]},"public_addr":"172.21.15.45:6817/3943189827","cluster_addr":"172.21.15.45:6819/3943189827","heartbeat_back_addr":"172.21.15.45:6823/3943189827","heartbeat_front_addr":"172.21.15.45:6821/3943189827","state":["exists","up"]},{"osd":5,"uuid":"c49ba5da-91c6-4a86-a97b-4aba89c253ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6809","nonce":3620760330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6811","nonce":3620760330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6815","nonce":3620760330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6813","nonce":3620760330}]},"public_addr":"172.21.15.45:6809/3620760330","cluster_addr":"172.21.15.45:6811/3620760330","heartbeat_back_addr":"172.21.15.45:6815/3620760330","heartbeat_front_addr":"172.21.15.45:6813/3620760330","state":["exists","up"]},{"osd":6,"uuid":"b0123475-89f7-4a34-9e33-d797b99dd9dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6801","nonce":1989083510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6803","nonce":1989083510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6807","nonce":1989083510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6805","nonce":1989083510}]},"public_addr":"172.21.15.45:6801/1989083510","cluster_addr":"172.21.15.45:6803/1989083510","heartbeat_back_addr":"172.21.15.45:6807/1989083510","heartbeat_front_addr":"172.21.15.45:6805/1989083510","state":["exists","up"]},{"osd":7,"uuid":"4dad773d-b7d8-46fc-8a10-a972aad27b29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6825","nonce":1513300294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6827","nonce":1513300294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6831","nonce":1513300294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6829","nonce":1513300294}]},"public_addr":"172.21.15.45:6825/1513300294","cluster_addr":"172.21.15.45:6827/1513300294","heartbeat_back_addr":"172.21.15.45:6831/1513300294","heartbeat_front_addr":"172.21.15.45:6829/1513300294","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.881855+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:17.133252+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.735339+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.500742+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.477571+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:17.167667+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.470291+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.873743+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"17.1a","mappings":[{"from":0,"to":2}]},{"pgid":"17.3f","mappings":[{"from":0,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[{"pool":17,"snaps":[{"begin":2,"length":4}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T03:21:35.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.688+0000 7f7f8a47b640 1 -- 172.21.15.27:0/4048750625 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f7f7006c220 msgr2=0x7f7f7006e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:35.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.688+0000 7f7f8a47b640 1 --2- 172.21.15.27:0/4048750625 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f7f7006c220 0x7f7f7006e6e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f7f74032000 tx=0x7f7f74034040 comp rx=0 tx=0).stop 2024-09-12T03:21:35.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.688+0000 7f7f8a47b640 1 -- 172.21.15.27:0/4048750625 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7f8414c790 msgr2=0x7f7f8414cc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:35.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.688+0000 7f7f8a47b640 1 --2- 172.21.15.27:0/4048750625 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7f8414c790 0x7f7f8414cc10 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f7f7800e4e0 tx=0x7f7f7800e9a0 comp rx=0 tx=0).stop 2024-09-12T03:21:35.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.688+0000 7f7f8a47b640 1 -- 172.21.15.27:0/4048750625 shutdown_connections 2024-09-12T03:21:35.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.688+0000 7f7f8a47b640 1 --2- 172.21.15.27:0/4048750625 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f7f841f4a20 0x7f7f841f4ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:35.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.688+0000 7f7f8a47b640 1 --2- 172.21.15.27:0/4048750625 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f7f7006c220 0x7f7f7006e6e0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:35.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.688+0000 7f7f8a47b640 1 --2- 172.21.15.27:0/4048750625 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7f8414c790 0x7f7f8414cc10 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:35.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.688+0000 7f7f8a47b640 1 --2- 172.21.15.27:0/4048750625 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7f6c0039e0 0x7f7f840cf0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:35.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.688+0000 7f7f8a47b640 1 -- 172.21.15.27:0/4048750625 >> 172.21.15.27:0/4048750625 conn(0x7f7f8406dce0 msgr2=0x7f7f8406e0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:35.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.688+0000 7f7f8a47b640 1 -- 172.21.15.27:0/4048750625 shutdown_connections 2024-09-12T03:21:35.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.688+0000 7f7f8a47b640 1 -- 172.21.15.27:0/4048750625 wait complete. 2024-09-12T03:21:35.699 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-12T03:21:35.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.802+0000 7f4c3d4a5640 1 Processor -- start 2024-09-12T03:21:35.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.802+0000 7f4c3d4a5640 1 -- start start 2024-09-12T03:21:35.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.803+0000 7f4c3d4a5640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f4c381590d0 0x7f4c380b55d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:35.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.803+0000 7f4c3d4a5640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f4c3815f600 0x7f4c3806bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:35.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.803+0000 7f4c3d4a5640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4c38162c10 con 0x7f4c38158cf0 2024-09-12T03:21:35.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.803+0000 7f4c3d4a5640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f4c38162ef0 con 0x7f4c3815f600 2024-09-12T03:21:35.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.803+0000 7f4c3d4a5640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f4c381631d0 con 0x7f4c381590d0 2024-09-12T03:21:35.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.803+0000 7f4c377fe640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f4c3815f600 0x7f4c3806bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:35.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.803+0000 7f4c377fe640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f4c3815f600 0x7f4c3806bb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41998/0 (socket says 172.21.15.27:41998) 2024-09-12T03:21:35.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.803+0000 7f4c377fe640 1 -- 172.21.15.27:0/1291485071 learned_addr learned my addr 172.21.15.27:0/1291485071 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:21:35.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.803+0000 7f4c367fc640 1 --2- 172.21.15.27:0/1291485071 >> v2:172.21.15.45:3300/0 conn(0x7f4c381590d0 0x7f4c380b55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:35.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.803+0000 7f4c377fe640 1 -- 172.21.15.27:0/1291485071 >> v2:172.21.15.45:3300/0 conn(0x7f4c381590d0 msgr2=0x7f4c380b55d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:35.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.803+0000 7f4c377fe640 1 --2- 172.21.15.27:0/1291485071 >> v2:172.21.15.45:3300/0 conn(0x7f4c381590d0 0x7f4c380b55d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:35.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.804+0000 7f4c377fe640 1 -- 172.21.15.27:0/1291485071 >> v1:172.21.15.45:6789/0 conn(0x7f4c38158cf0 legacy=0x7f4c38162430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:35.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.803+0000 7f4c35ffb640 1 -- 172.21.15.27:0/1291485071 <== mon.1 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1136882498 0 0) 0x7f4c38162c10 con 0x7f4c38158cf0 2024-09-12T03:21:35.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.804+0000 7f4c377fe640 1 -- 172.21.15.27:0/1291485071 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c3815edc0 con 0x7f4c3815f600 2024-09-12T03:21:35.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.804+0000 7f4c377fe640 1 --2- 172.21.15.27:0/1291485071 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4c3815f600 0x7f4c3806bb30 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f4c381594b0 tx=0x7f4c2802a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9295d3100a529487 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:35.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.804+0000 7f4c35ffb640 1 -- 172.21.15.27:0/1291485071 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c28036070 con 0x7f4c3815f600 2024-09-12T03:21:35.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.804+0000 7f4c35ffb640 1 -- 172.21.15.27:0/1291485071 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c28031600 con 0x7f4c3815f600 2024-09-12T03:21:35.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.804+0000 7f4c35ffb640 1 -- 172.21.15.27:0/1291485071 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c280319d0 con 0x7f4c3815f600 2024-09-12T03:21:35.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.805+0000 7f4c3d4a5640 1 -- 172.21.15.27:0/1291485071 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4c3815f600 msgr2=0x7f4c3806bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:35.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.805+0000 7f4c3d4a5640 1 --2- 172.21.15.27:0/1291485071 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4c3815f600 0x7f4c3806bb30 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f4c381594b0 tx=0x7f4c2802a610 comp rx=0 tx=0).stop 2024-09-12T03:21:35.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.805+0000 7f4c3d4a5640 1 -- 172.21.15.27:0/1291485071 shutdown_connections 2024-09-12T03:21:35.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.805+0000 7f4c3d4a5640 1 --2- 172.21.15.27:0/1291485071 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4c3815f600 0x7f4c3806bb30 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:35.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.805+0000 7f4c3d4a5640 1 --2- 172.21.15.27:0/1291485071 >> v2:172.21.15.45:3300/0 conn(0x7f4c381590d0 0x7f4c380b55d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:35.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.805+0000 7f4c3d4a5640 1 -- 172.21.15.27:0/1291485071 >> 172.21.15.27:0/1291485071 conn(0x7f4c3806cc40 msgr2=0x7f4c3806d050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:35.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.805+0000 7f4c3d4a5640 1 -- 172.21.15.27:0/1291485071 shutdown_connections 2024-09-12T03:21:35.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.805+0000 7f4c3d4a5640 1 -- 172.21.15.27:0/1291485071 wait complete. 2024-09-12T03:21:35.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.806+0000 7f4c3d4a5640 1 Processor -- start 2024-09-12T03:21:35.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.806+0000 7f4c3d4a5640 1 -- start start 2024-09-12T03:21:35.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.807+0000 7f4c3d4a5640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4c38158cf0 0x7f4c38159ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:35.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.807+0000 7f4c3d4a5640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4c381590d0 0x7f4c380b2290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:35.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.807+0000 7f4c3d4a5640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4c3815f600 0x7f4c380b27d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:35.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.807+0000 7f4c3d4a5640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f4c3806c070 con 0x7f4c381590d0 2024-09-12T03:21:35.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.807+0000 7f4c3d4a5640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f4c3815fca0 con 0x7f4c38158cf0 2024-09-12T03:21:35.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.807+0000 7f4c3d4a5640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f4c3806c1f0 con 0x7f4c3815f600 2024-09-12T03:21:35.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.807+0000 7f4c367fc640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4c381590d0 0x7f4c380b2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:35.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.807+0000 7f4c367fc640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4c381590d0 0x7f4c380b2290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42008/0 (socket says 172.21.15.27:42008) 2024-09-12T03:21:35.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.807+0000 7f4c367fc640 1 -- 172.21.15.27:0/1919406094 learned_addr learned my addr 172.21.15.27:0/1919406094 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:21:35.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.807+0000 7f4c36ffd640 1 --2- 172.21.15.27:0/1919406094 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4c38158cf0 0x7f4c38159ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:35.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.807+0000 7f4c377fe640 1 --2- 172.21.15.27:0/1919406094 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4c3815f600 0x7f4c380b27d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:35.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.808+0000 7f4c367fc640 1 -- 172.21.15.27:0/1919406094 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4c3815f600 msgr2=0x7f4c380b27d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:35.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.808+0000 7f4c367fc640 1 --2- 172.21.15.27:0/1919406094 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4c3815f600 0x7f4c380b27d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:35.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.808+0000 7f4c367fc640 1 -- 172.21.15.27:0/1919406094 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4c38158cf0 msgr2=0x7f4c38159ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:35.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.808+0000 7f4c367fc640 1 --2- 172.21.15.27:0/1919406094 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4c38158cf0 0x7f4c38159ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:35.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.808+0000 7f4c367fc640 1 -- 172.21.15.27:0/1919406094 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c381f7b30 con 0x7f4c381590d0 2024-09-12T03:21:35.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.808+0000 7f4c367fc640 1 --2- 172.21.15.27:0/1919406094 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4c381590d0 0x7f4c380b2290 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f4c2000e8d0 tx=0x7f4c2000ed90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:35.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.808+0000 7f4c2f7fe640 1 -- 172.21.15.27:0/1919406094 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c20017070 con 0x7f4c381590d0 2024-09-12T03:21:35.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.808+0000 7f4c2f7fe640 1 -- 172.21.15.27:0/1919406094 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c2000c940 con 0x7f4c381590d0 2024-09-12T03:21:35.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.808+0000 7f4c3d4a5640 1 -- 172.21.15.27:0/1919406094 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c381f7e10 con 0x7f4c381590d0 2024-09-12T03:21:35.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.808+0000 7f4c2f7fe640 1 -- 172.21.15.27:0/1919406094 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c2000cd30 con 0x7f4c381590d0 2024-09-12T03:21:35.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.808+0000 7f4c3d4a5640 1 -- 172.21.15.27:0/1919406094 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c381f83d0 con 0x7f4c381590d0 2024-09-12T03:21:35.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.809+0000 7f4c3d4a5640 1 -- 172.21.15.27:0/1919406094 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c38154460 con 0x7f4c381590d0 2024-09-12T03:21:35.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.810+0000 7f4c2f7fe640 1 -- 172.21.15.27:0/1919406094 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 19) ==== 90855+0+0 (secure 0 0 0) 0x7f4c2001a070 con 0x7f4c381590d0 2024-09-12T03:21:35.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.810+0000 7f4c2f7fe640 1 --2- 172.21.15.27:0/1919406094 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f4c1806c220 0x7f4c1806e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:35.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.810+0000 7f4c2f7fe640 1 -- 172.21.15.27:0/1919406094 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5823+0+0 (secure 0 0 0) 0x7f4c2008b8d0 con 0x7f4c381590d0 2024-09-12T03:21:35.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.810+0000 7f4c36ffd640 1 --2- 172.21.15.27:0/1919406094 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f4c1806c220 0x7f4c1806e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:35.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.810+0000 7f4c36ffd640 1 --2- 172.21.15.27:0/1919406094 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f4c1806c220 0x7f4c1806e6e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f4c30000f60 tx=0x7f4c30009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:35.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:35.814+0000 7f4c2f7fe640 1 -- 172.21.15.27:0/1919406094 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f4c20059670 con 0x7f4c381590d0 2024-09-12T03:21:36.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:36.026+0000 7f4c3d4a5640 1 -- 172.21.15.27:0/1919406094 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f4c3806bb30 con 0x7f4c381590d0 2024-09-12T03:21:36.026 INFO:tasks.ceph.mon.a.smithi027.stderr:2024-09-12T03:21:36.026+0000 7f959bfd9640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-12T03:21:36.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:36.027+0000 7f4c2f7fe640 1 -- 172.21.15.27:0/1919406094 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v60) ==== 161+0+0 (secure 0 0 0) 0x7f4c2008b070 con 0x7f4c381590d0 2024-09-12T03:21:36.027 INFO:teuthology.orchestra.run.smithi027.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-12T03:21:36.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:36.030+0000 7f4c3d4a5640 1 -- 172.21.15.27:0/1919406094 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f4c1806c220 msgr2=0x7f4c1806e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:36.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:36.030+0000 7f4c3d4a5640 1 --2- 172.21.15.27:0/1919406094 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f4c1806c220 0x7f4c1806e6e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f4c30000f60 tx=0x7f4c30009040 comp rx=0 tx=0).stop 2024-09-12T03:21:36.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:36.030+0000 7f4c3d4a5640 1 -- 172.21.15.27:0/1919406094 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4c381590d0 msgr2=0x7f4c380b2290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:36.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:36.030+0000 7f4c3d4a5640 1 --2- 172.21.15.27:0/1919406094 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4c381590d0 0x7f4c380b2290 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f4c2000e8d0 tx=0x7f4c2000ed90 comp rx=0 tx=0).stop 2024-09-12T03:21:36.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:36.030+0000 7f4c3d4a5640 1 -- 172.21.15.27:0/1919406094 shutdown_connections 2024-09-12T03:21:36.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:36.030+0000 7f4c3d4a5640 1 --2- 172.21.15.27:0/1919406094 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4c3815f600 0x7f4c380b27d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:36.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:36.030+0000 7f4c3d4a5640 1 --2- 172.21.15.27:0/1919406094 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f4c1806c220 0x7f4c1806e6e0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:36.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:36.030+0000 7f4c3d4a5640 1 --2- 172.21.15.27:0/1919406094 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4c381590d0 0x7f4c380b2290 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:36.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:36.030+0000 7f4c3d4a5640 1 --2- 172.21.15.27:0/1919406094 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4c38158cf0 0x7f4c38159ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:36.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:36.030+0000 7f4c3d4a5640 1 -- 172.21.15.27:0/1919406094 >> 172.21.15.27:0/1919406094 conn(0x7f4c3806cc40 msgr2=0x7f4c38161170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:36.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:36.030+0000 7f4c3d4a5640 1 -- 172.21.15.27:0/1919406094 shutdown_connections 2024-09-12T03:21:36.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:36.030+0000 7f4c3d4a5640 1 -- 172.21.15.27:0/1919406094 wait complete. 2024-09-12T03:21:36.043 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-12T03:21:36.069 INFO:tasks.ceph.mds.a.smithi027.stdout:starting mds.a at 2024-09-12T03:21:36.148 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:36.148+0000 7f641b2c9640 1 Processor -- start 2024-09-12T03:21:36.148 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:36.148+0000 7f641b2c9640 1 -- start start 2024-09-12T03:21:36.149 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:36.149+0000 7f641b2c9640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f641406dfd0 0x7f64140d1da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:36.150 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:36.149+0000 7f641b2c9640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f64140d3790 0x7f64140b7a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:36.150 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:36.149+0000 7f641b2c9640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f64140d2e30 con 0x7f6414162040 2024-09-12T03:21:36.150 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:36.149+0000 7f641b2c9640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f64140b7f90 con 0x7f64140d3790 2024-09-12T03:21:36.150 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:36.149+0000 7f641b2c9640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f6414162420 con 0x7f641406dfd0 2024-09-12T03:21:36.150 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:36.149+0000 7f6419866640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f64140d3790 0x7f64140b7a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:36.150 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:36.149+0000 7f6419065640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f6414162040 0x7f641406d8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:51908/0 (socket says 172.21.15.27:51908) 2024-09-12T03:21:36.150 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:36.149+0000 7f6419065640 1 -- 172.21.15.27:0/3533911099 learned_addr learned my addr 172.21.15.27:0/3533911099 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:21:36.150 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:36.150+0000 7f6418864640 1 --2- 172.21.15.27:0/3533911099 >> v2:172.21.15.45:3300/0 conn(0x7f641406dfd0 0x7f64140d1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:36.150 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:36.150+0000 7f6419866640 1 -- 172.21.15.27:0/3533911099 >> v2:172.21.15.45:3300/0 conn(0x7f641406dfd0 msgr2=0x7f64140d1da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:36.150 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:36.150+0000 7f6419866640 1 --2- 172.21.15.27:0/3533911099 >> v2:172.21.15.45:3300/0 conn(0x7f641406dfd0 0x7f64140d1da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:36.150 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:36.150+0000 7f6419866640 1 -- 172.21.15.27:0/3533911099 >> v1:172.21.15.27:6789/0 conn(0x7f6414162040 legacy=0x7f641406d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:36.151 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:36.150+0000 7f6419866640 1 -- 172.21.15.27:0/3533911099 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64140d30d0 con 0x7f64140d3790 2024-09-12T03:21:36.152 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:36.150+0000 7f6419866640 1 --2- 172.21.15.27:0/3533911099 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f64140d3790 0x7f64140b7a50 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f641406e630 tx=0x7f640c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10fc68f992ae57b2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:36.152 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:36.151+0000 7f6413fff640 1 -- 172.21.15.27:0/3533911099 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f640c037070 con 0x7f64140d3790 2024-09-12T03:21:36.152 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:36.151+0000 7f6413fff640 1 -- 172.21.15.27:0/3533911099 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f640c032440 con 0x7f64140d3790 2024-09-12T03:21:36.152 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:36.151+0000 7f6413fff640 1 -- 172.21.15.27:0/3533911099 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f640c032810 con 0x7f64140d3790 2024-09-12T03:21:36.152 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:36.151+0000 7f641b2c9640 1 -- 172.21.15.27:0/3533911099 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f64140d3790 msgr2=0x7f64140b7a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:36.152 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:36.151+0000 7f641b2c9640 1 --2- 172.21.15.27:0/3533911099 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f64140d3790 0x7f64140b7a50 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f641406e630 tx=0x7f640c02a1a0 comp rx=0 tx=0).stop 2024-09-12T03:21:36.152 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:36.151+0000 7f641b2c9640 1 -- 172.21.15.27:0/3533911099 shutdown_connections 2024-09-12T03:21:36.152 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:36.151+0000 7f641b2c9640 1 --2- 172.21.15.27:0/3533911099 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f64140d3790 0x7f64140b7a50 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:36.153 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:36.151+0000 7f641b2c9640 1 --2- 172.21.15.27:0/3533911099 >> v2:172.21.15.45:3300/0 conn(0x7f641406dfd0 0x7f64140d1da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:36.153 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:36.151+0000 7f641b2c9640 1 -- 172.21.15.27:0/3533911099 >> 172.21.15.27:0/3533911099 conn(0x7f64140b6700 msgr2=0x7f64140b6b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:36.153 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:36.151+0000 7f641b2c9640 1 -- 172.21.15.27:0/3533911099 shutdown_connections 2024-09-12T03:21:36.153 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:36.151+0000 7f641b2c9640 1 -- 172.21.15.27:0/3533911099 wait complete. 2024-09-12T03:21:36.155 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:36.152+0000 7f641b2c9640 1 Processor -- start 2024-09-12T03:21:36.155 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:36.153+0000 7f641b2c9640 1 -- start start 2024-09-12T03:21:36.155 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:36.153+0000 7f641b2c9640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f641406dfd0 0x7f64141f5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:36.155 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:36.153+0000 7f641b2c9640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f64140d3790 0x7f64141f7ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:36.155 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:36.153+0000 7f641b2c9640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f6414162040 0x7f64141f8220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:36.156 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:36.153+0000 7f641b2c9640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f64140d4530 con 0x7f641406dfd0 2024-09-12T03:21:36.156 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:36.153+0000 7f641b2c9640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f64140cee00 con 0x7f64140d3790 2024-09-12T03:21:36.156 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:36.153+0000 7f641b2c9640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f64140d4150 con 0x7f6414162040 2024-09-12T03:21:36.156 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:36.153+0000 7f6419065640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f641406dfd0 0x7f64141f5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:36.156 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:36.153+0000 7f6419065640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f641406dfd0 0x7f64141f5170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42036/0 (socket says 172.21.15.27:42036) 2024-09-12T03:21:36.156 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:36.153+0000 7f6419065640 1 -- 172.21.15.27:0/2699044261 learned_addr learned my addr 172.21.15.27:0/2699044261 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:21:36.156 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:36.154+0000 7f6418864640 1 --2- 172.21.15.27:0/2699044261 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f64140d3790 0x7f64141f7ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:36.156 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:36.154+0000 7f6419866640 1 --2- 172.21.15.27:0/2699044261 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f6414162040 0x7f64141f8220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:36.156 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:36.154+0000 7f6419065640 1 -- 172.21.15.27:0/2699044261 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f6414162040 msgr2=0x7f64141f8220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:36.156 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:36.154+0000 7f6419065640 1 --2- 172.21.15.27:0/2699044261 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f6414162040 0x7f64141f8220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:36.156 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:36.154+0000 7f6419065640 1 -- 172.21.15.27:0/2699044261 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f64140d3790 msgr2=0x7f64141f7ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:36.157 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:36.154+0000 7f6419065640 1 --2- 172.21.15.27:0/2699044261 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f64140d3790 0x7f64141f7ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:36.157 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:36.154+0000 7f6419065640 1 -- 172.21.15.27:0/2699044261 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64141f56b0 con 0x7f641406dfd0 2024-09-12T03:21:36.157 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:36.154+0000 7f6418864640 1 --2- 172.21.15.27:0/2699044261 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f64140d3790 0x7f64141f7ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:21:36.157 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:36.154+0000 7f6419065640 1 --2- 172.21.15.27:0/2699044261 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f641406dfd0 0x7f64141f5170 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f640400caa0 tx=0x7f640400cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:36.157 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:36.154+0000 7f6411ffb640 1 -- 172.21.15.27:0/2699044261 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6404017070 con 0x7f641406dfd0 2024-09-12T03:21:36.157 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:36.154+0000 7f6411ffb640 1 -- 172.21.15.27:0/2699044261 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f640400ac00 con 0x7f641406dfd0 2024-09-12T03:21:36.157 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:36.154+0000 7f641b2c9640 1 -- 172.21.15.27:0/2699044261 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6414157b50 con 0x7f641406dfd0 2024-09-12T03:21:36.157 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:36.154+0000 7f6411ffb640 1 -- 172.21.15.27:0/2699044261 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6404012400 con 0x7f641406dfd0 2024-09-12T03:21:36.157 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:36.154+0000 7f641b2c9640 1 -- 172.21.15.27:0/2699044261 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64141583f0 con 0x7f641406dfd0 2024-09-12T03:21:36.157 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:36.155+0000 7f641b2c9640 1 -- 172.21.15.27:0/2699044261 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f641406e6e0 con 0x7f641406dfd0 2024-09-12T03:21:36.158 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:36.156+0000 7f6411ffb640 1 -- 172.21.15.27:0/2699044261 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 19) ==== 90855+0+0 (secure 0 0 0) 0x7f640401a070 con 0x7f641406dfd0 2024-09-12T03:21:36.158 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:36.156+0000 7f6411ffb640 1 --2- 172.21.15.27:0/2699044261 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f640006c220 0x7f640006e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:36.158 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:36.156+0000 7f6411ffb640 1 -- 172.21.15.27:0/2699044261 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5881+0+0 (secure 0 0 0) 0x7f640408b350 con 0x7f641406dfd0 2024-09-12T03:21:36.158 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:36.156+0000 7f6418864640 1 --2- 172.21.15.27:0/2699044261 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f640006c220 0x7f640006e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:36.158 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:36.157+0000 7f6418864640 1 --2- 172.21.15.27:0/2699044261 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f640006c220 0x7f640006e6e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f64140ba160 tx=0x7f6408008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:36.160 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:36.161+0000 7f6411ffb640 1 -- 172.21.15.27:0/2699044261 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f64141f56b0 con 0x7f641406dfd0 2024-09-12T03:21:36.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:36.375+0000 7f641b2c9640 1 -- 172.21.15.27:0/2699044261 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f641406d9f0 con 0x7f641406dfd0 2024-09-12T03:21:37.029 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:37.029+0000 7f6411ffb640 1 -- 172.21.15.27:0/2699044261 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v61) ==== 104+0+0 (secure 0 0 0) 0x7f6404059670 con 0x7f641406dfd0 2024-09-12T03:21:37.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:37.032+0000 7f641b2c9640 1 -- 172.21.15.27:0/2699044261 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f640006c220 msgr2=0x7f640006e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:37.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:37.032+0000 7f641b2c9640 1 --2- 172.21.15.27:0/2699044261 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f640006c220 0x7f640006e6e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f64140ba160 tx=0x7f6408008040 comp rx=0 tx=0).stop 2024-09-12T03:21:37.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:37.032+0000 7f641b2c9640 1 -- 172.21.15.27:0/2699044261 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f641406dfd0 msgr2=0x7f64141f5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:37.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:37.032+0000 7f641b2c9640 1 --2- 172.21.15.27:0/2699044261 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f641406dfd0 0x7f64141f5170 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f640400caa0 tx=0x7f640400cf60 comp rx=0 tx=0).stop 2024-09-12T03:21:37.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:37.032+0000 7f641b2c9640 1 -- 172.21.15.27:0/2699044261 shutdown_connections 2024-09-12T03:21:37.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:37.032+0000 7f641b2c9640 1 --2- 172.21.15.27:0/2699044261 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f6414162040 0x7f64141f8220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:37.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:37.032+0000 7f641b2c9640 1 --2- 172.21.15.27:0/2699044261 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f64140d3790 0x7f64141f7ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:37.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:37.032+0000 7f641b2c9640 1 --2- 172.21.15.27:0/2699044261 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f640006c220 0x7f640006e6e0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:37.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:37.032+0000 7f641b2c9640 1 --2- 172.21.15.27:0/2699044261 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f641406dfd0 0x7f64141f5170 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:37.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:37.032+0000 7f641b2c9640 1 -- 172.21.15.27:0/2699044261 >> 172.21.15.27:0/2699044261 conn(0x7f64140b6700 msgr2=0x7f64140b4b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:37.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:37.032+0000 7f641b2c9640 1 -- 172.21.15.27:0/2699044261 shutdown_connections 2024-09-12T03:21:37.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:37.032+0000 7f641b2c9640 1 -- 172.21.15.27:0/2699044261 wait complete. 2024-09-12T03:21:37.046 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-12T03:21:37.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:37.202+0000 7fbd33942640 1 Processor -- start 2024-09-12T03:21:37.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:37.202+0000 7fbd33942640 1 -- start start 2024-09-12T03:21:37.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:37.203+0000 7fbd33942640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fbd2c15ec30 0x7fbd2c0b3900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:37.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:37.203+0000 7fbd33942640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fbd2c0af9d0 0x7fbd2c0b3e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:37.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:37.203+0000 7fbd33942640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbd2c0ce5e0 con 0x7fbd2c0b4380 2024-09-12T03:21:37.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:37.203+0000 7fbd33942640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7fbd2c0d02d0 con 0x7fbd2c15ec30 2024-09-12T03:21:37.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:37.203+0000 7fbd33942640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fbd2c06c650 con 0x7fbd2c0af9d0 2024-09-12T03:21:37.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:37.203+0000 7fbd316de640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fbd2c15ec30 0x7fbd2c0b3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:37.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:37.203+0000 7fbd316de640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fbd2c15ec30 0x7fbd2c0b3900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42048/0 (socket says 172.21.15.27:42048) 2024-09-12T03:21:37.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:37.203+0000 7fbd316de640 1 -- 172.21.15.27:0/3620306112 learned_addr learned my addr 172.21.15.27:0/3620306112 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:21:37.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:37.204+0000 7fbd316de640 1 -- 172.21.15.27:0/3620306112 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fbd2c0af9d0 msgr2=0x7fbd2c0b3e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:37.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:37.204+0000 7fbd30edd640 1 --2- 172.21.15.27:0/3620306112 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fbd2c0af9d0 0x7fbd2c0b3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:37.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:37.204+0000 7fbd316de640 1 --2- 172.21.15.27:0/3620306112 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fbd2c0af9d0 0x7fbd2c0b3e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:37.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:37.204+0000 7fbd316de640 1 -- 172.21.15.27:0/3620306112 >> v1:172.21.15.45:6789/0 conn(0x7fbd2c0b4380 legacy=0x7fbd2c15f2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:37.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:37.204+0000 7fbd316de640 1 -- 172.21.15.27:0/3620306112 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd2c15f010 con 0x7fbd2c15ec30 2024-09-12T03:21:37.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:37.204+0000 7fbd30edd640 1 --2- 172.21.15.27:0/3620306112 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fbd2c0af9d0 0x7fbd2c0b3e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:21:37.206 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:37.204+0000 7fbd316de640 1 --2- 172.21.15.27:0/3620306112 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbd2c15ec30 0x7fbd2c0b3900 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7fbd1c009870 tx=0x7fbd1c02a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8852c71843d12c50 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:37.206 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:37.205+0000 7fbd2bfff640 1 -- 172.21.15.27:0/3620306112 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd1c037070 con 0x7fbd2c15ec30 2024-09-12T03:21:37.206 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:37.205+0000 7fbd2bfff640 1 -- 172.21.15.27:0/3620306112 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd1c032440 con 0x7fbd2c15ec30 2024-09-12T03:21:37.206 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:37.205+0000 7fbd2bfff640 1 -- 172.21.15.27:0/3620306112 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd1c032810 con 0x7fbd2c15ec30 2024-09-12T03:21:37.206 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:37.205+0000 7fbd33942640 1 -- 172.21.15.27:0/3620306112 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbd2c15ec30 msgr2=0x7fbd2c0b3900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:37.206 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:37.205+0000 7fbd33942640 1 --2- 172.21.15.27:0/3620306112 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbd2c15ec30 0x7fbd2c0b3900 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7fbd1c009870 tx=0x7fbd1c02a250 comp rx=0 tx=0).stop 2024-09-12T03:21:37.207 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:37.205+0000 7fbd33942640 1 -- 172.21.15.27:0/3620306112 shutdown_connections 2024-09-12T03:21:37.207 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:37.205+0000 7fbd33942640 1 --2- 172.21.15.27:0/3620306112 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fbd2c0af9d0 0x7fbd2c0b3e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:37.207 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:37.205+0000 7fbd33942640 1 --2- 172.21.15.27:0/3620306112 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbd2c15ec30 0x7fbd2c0b3900 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:37.207 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:37.205+0000 7fbd33942640 1 -- 172.21.15.27:0/3620306112 >> 172.21.15.27:0/3620306112 conn(0x7fbd2c06bb30 msgr2=0x7fbd2c06bf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:37.207 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:37.205+0000 7fbd33942640 1 -- 172.21.15.27:0/3620306112 shutdown_connections 2024-09-12T03:21:37.207 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:37.205+0000 7fbd33942640 1 -- 172.21.15.27:0/3620306112 wait complete. 2024-09-12T03:21:37.209 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:37.207+0000 7fbd33942640 1 Processor -- start 2024-09-12T03:21:37.209 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:37.207+0000 7fbd33942640 1 -- start start 2024-09-12T03:21:37.209 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:37.207+0000 7fbd33942640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbd2c0af9d0 0x7fbd2c0b19a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:37.209 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:37.208+0000 7fbd33942640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fbd2c0b4380 0x7fbd2c14b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:37.209 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:37.208+0000 7fbd33942640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbd2c15ec30 0x7fbd2c06db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:37.209 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:37.208+0000 7fbd33942640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fbd2c0715f0 con 0x7fbd2c15ec30 2024-09-12T03:21:37.209 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:37.208+0000 7fbd33942640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fbd2c0cfff0 con 0x7fbd2c0af9d0 2024-09-12T03:21:37.209 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:37.208+0000 7fbd33942640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fbd2c15f9b0 con 0x7fbd2c0b4380 2024-09-12T03:21:37.210 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:37.208+0000 7fbd31edf640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbd2c15ec30 0x7fbd2c06db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:37.210 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:37.208+0000 7fbd31edf640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbd2c15ec30 0x7fbd2c06db90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42064/0 (socket says 172.21.15.27:42064) 2024-09-12T03:21:37.210 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:37.208+0000 7fbd31edf640 1 -- 172.21.15.27:0/290219926 learned_addr learned my addr 172.21.15.27:0/290219926 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:21:37.210 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:37.208+0000 7fbd316de640 1 --2- 172.21.15.27:0/290219926 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbd2c0af9d0 0x7fbd2c0b19a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:37.210 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:37.208+0000 7fbd30edd640 1 --2- 172.21.15.27:0/290219926 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fbd2c0b4380 0x7fbd2c14b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:37.210 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:37.208+0000 7fbd31edf640 1 -- 172.21.15.27:0/290219926 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fbd2c0b4380 msgr2=0x7fbd2c14b6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:37.210 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:37.208+0000 7fbd31edf640 1 --2- 172.21.15.27:0/290219926 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fbd2c0b4380 0x7fbd2c14b6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:37.210 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:37.208+0000 7fbd31edf640 1 -- 172.21.15.27:0/290219926 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbd2c0af9d0 msgr2=0x7fbd2c0b19a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:37.210 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:37.208+0000 7fbd31edf640 1 --2- 172.21.15.27:0/290219926 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbd2c0af9d0 0x7fbd2c0b19a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:37.210 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:37.208+0000 7fbd31edf640 1 -- 172.21.15.27:0/290219926 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd2c151bc0 con 0x7fbd2c15ec30 2024-09-12T03:21:37.210 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:37.208+0000 7fbd31edf640 1 --2- 172.21.15.27:0/290219926 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbd2c15ec30 0x7fbd2c06db90 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7fbd2400c7d0 tx=0x7fbd2400cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:37.210 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:37.208+0000 7fbd29ffb640 1 -- 172.21.15.27:0/290219926 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd24003b70 con 0x7fbd2c15ec30 2024-09-12T03:21:37.211 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:37.208+0000 7fbd29ffb640 1 -- 172.21.15.27:0/290219926 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd24003d30 con 0x7fbd2c15ec30 2024-09-12T03:21:37.211 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:37.208+0000 7fbd33942640 1 -- 172.21.15.27:0/290219926 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd2c151ea0 con 0x7fbd2c15ec30 2024-09-12T03:21:37.211 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:37.208+0000 7fbd29ffb640 1 -- 172.21.15.27:0/290219926 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd2400aa90 con 0x7fbd2c15ec30 2024-09-12T03:21:37.211 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:37.209+0000 7fbd33942640 1 -- 172.21.15.27:0/290219926 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd2c0b1ee0 con 0x7fbd2c15ec30 2024-09-12T03:21:37.211 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:37.210+0000 7fbd33942640 1 -- 172.21.15.27:0/290219926 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd2c0ce8f0 con 0x7fbd2c15ec30 2024-09-12T03:21:37.212 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:37.210+0000 7fbd29ffb640 1 -- 172.21.15.27:0/290219926 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 19) ==== 90855+0+0 (secure 0 0 0) 0x7fbd2400ac30 con 0x7fbd2c15ec30 2024-09-12T03:21:37.212 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:37.210+0000 7fbd29ffb640 1 --2- 172.21.15.27:0/290219926 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fbd1806c220 0x7fbd1806e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:37.212 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:37.211+0000 7fbd29ffb640 1 -- 172.21.15.27:0/290219926 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5881+0+0 (secure 0 0 0) 0x7fbd2408b390 con 0x7fbd2c15ec30 2024-09-12T03:21:37.212 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:37.211+0000 7fbd316de640 1 --2- 172.21.15.27:0/290219926 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fbd1806c220 0x7fbd1806e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:37.212 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:37.211+0000 7fbd316de640 1 --2- 172.21.15.27:0/290219926 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fbd1806c220 0x7fbd1806e6e0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fbd1c034000 tx=0x7fbd1c02e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:37.215 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:37.215+0000 7fbd29ffb640 1 -- 172.21.15.27:0/290219926 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fbd240590f0 con 0x7fbd2c15ec30 2024-09-12T03:21:37.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:37.393+0000 7fbd33942640 1 -- 172.21.15.27:0/290219926 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fbd2c15fd70 con 0x7fbd2c15ec30 2024-09-12T03:21:38.210 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:38.210+0000 7fbd29ffb640 1 -- 172.21.15.27:0/290219926 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v105) ==== 181+0+0 (secure 0 0 0) 0x7fbd2405d3f0 con 0x7fbd2c15ec30 2024-09-12T03:21:38.210 INFO:teuthology.orchestra.run.smithi027.stderr:pool 'cephfs_metadata' removed 2024-09-12T03:21:38.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:38.214+0000 7fbd33942640 1 -- 172.21.15.27:0/290219926 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fbd1806c220 msgr2=0x7fbd1806e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:38.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:38.214+0000 7fbd33942640 1 --2- 172.21.15.27:0/290219926 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fbd1806c220 0x7fbd1806e6e0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fbd1c034000 tx=0x7fbd1c02e000 comp rx=0 tx=0).stop 2024-09-12T03:21:38.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:38.214+0000 7fbd33942640 1 -- 172.21.15.27:0/290219926 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbd2c15ec30 msgr2=0x7fbd2c06db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:38.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:38.214+0000 7fbd33942640 1 --2- 172.21.15.27:0/290219926 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbd2c15ec30 0x7fbd2c06db90 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7fbd2400c7d0 tx=0x7fbd2400cc90 comp rx=0 tx=0).stop 2024-09-12T03:21:38.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:38.214+0000 7fbd33942640 1 -- 172.21.15.27:0/290219926 shutdown_connections 2024-09-12T03:21:38.215 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:38.214+0000 7fbd33942640 1 --2- 172.21.15.27:0/290219926 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fbd1806c220 0x7fbd1806e6e0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:38.215 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:38.214+0000 7fbd33942640 1 --2- 172.21.15.27:0/290219926 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbd2c15ec30 0x7fbd2c06db90 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:38.215 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:38.214+0000 7fbd33942640 1 --2- 172.21.15.27:0/290219926 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fbd2c0b4380 0x7fbd2c14b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:38.215 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:38.214+0000 7fbd33942640 1 --2- 172.21.15.27:0/290219926 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbd2c0af9d0 0x7fbd2c0b19a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:38.215 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:38.214+0000 7fbd33942640 1 -- 172.21.15.27:0/290219926 >> 172.21.15.27:0/290219926 conn(0x7fbd2c06bb30 msgr2=0x7fbd2c06bf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:38.215 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:38.215+0000 7fbd33942640 1 -- 172.21.15.27:0/290219926 shutdown_connections 2024-09-12T03:21:38.215 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:38.215+0000 7fbd33942640 1 -- 172.21.15.27:0/290219926 wait complete. 2024-09-12T03:21:38.229 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-12T03:21:38.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:38.342+0000 7f6a3c52c640 1 Processor -- start 2024-09-12T03:21:38.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:38.342+0000 7f6a3c52c640 1 -- start start 2024-09-12T03:21:38.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:38.343+0000 7f6a3c52c640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f6a3415a270 0x7f6a3406cff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:38.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:38.343+0000 7f6a3c52c640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f6a3406d530 0x7f6a340cf410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:38.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:38.343+0000 7f6a3c52c640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6a340ce650 con 0x7f6a3406c650 2024-09-12T03:21:38.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:38.343+0000 7f6a3c52c640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f6a340cef80 con 0x7f6a3415a270 2024-09-12T03:21:38.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:38.343+0000 7f6a3c52c640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f6a340ce930 con 0x7f6a3406d530 2024-09-12T03:21:38.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:38.343+0000 7f6a3a2c8640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f6a3415a270 0x7f6a3406cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:38.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:38.343+0000 7f6a3aac9640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f6a3406c650 0x7f6a340726a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:51918/0 (socket says 172.21.15.27:51918) 2024-09-12T03:21:38.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:38.343+0000 7f6a3aac9640 1 -- 172.21.15.27:0/2948974874 learned_addr learned my addr 172.21.15.27:0/2948974874 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:21:38.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:38.343+0000 7f6a39ac7640 1 --2- 172.21.15.27:0/2948974874 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f6a3406d530 0x7f6a340cf410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:38.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:38.343+0000 7f6a3a2c8640 1 -- 172.21.15.27:0/2948974874 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f6a3406d530 msgr2=0x7f6a340cf410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:38.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:38.343+0000 7f6a3a2c8640 1 --2- 172.21.15.27:0/2948974874 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f6a3406d530 0x7f6a340cf410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:38.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:38.343+0000 7f6a392c6640 1 -- 172.21.15.27:0/2948974874 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2085631234 0 0) 0x7f6a340ce650 con 0x7f6a3406c650 2024-09-12T03:21:38.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:38.343+0000 7f6a3a2c8640 1 -- 172.21.15.27:0/2948974874 >> v1:172.21.15.27:6789/0 conn(0x7f6a3406c650 legacy=0x7f6a340726a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:38.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:38.343+0000 7f6a3a2c8640 1 -- 172.21.15.27:0/2948974874 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a340cec10 con 0x7f6a3415a270 2024-09-12T03:21:38.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:38.343+0000 7f6a39ac7640 1 --2- 172.21.15.27:0/2948974874 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f6a3406d530 0x7f6a340cf410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:21:38.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:38.344+0000 7f6a3a2c8640 1 --2- 172.21.15.27:0/2948974874 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6a3415a270 0x7f6a3406cff0 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f6a300097a0 tx=0x7f6a3002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96628f9d67fd81d4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:38.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:38.344+0000 7f6a392c6640 1 -- 172.21.15.27:0/2948974874 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a30037070 con 0x7f6a3415a270 2024-09-12T03:21:38.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:38.344+0000 7f6a392c6640 1 -- 172.21.15.27:0/2948974874 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a30032440 con 0x7f6a3415a270 2024-09-12T03:21:38.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:38.344+0000 7f6a392c6640 1 -- 172.21.15.27:0/2948974874 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a30032810 con 0x7f6a3415a270 2024-09-12T03:21:38.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:38.345+0000 7f6a3c52c640 1 -- 172.21.15.27:0/2948974874 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6a3415a270 msgr2=0x7f6a3406cff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:38.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:38.345+0000 7f6a3c52c640 1 --2- 172.21.15.27:0/2948974874 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6a3415a270 0x7f6a3406cff0 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f6a300097a0 tx=0x7f6a3002a180 comp rx=0 tx=0).stop 2024-09-12T03:21:38.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:38.345+0000 7f6a3c52c640 1 -- 172.21.15.27:0/2948974874 shutdown_connections 2024-09-12T03:21:38.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:38.345+0000 7f6a3c52c640 1 --2- 172.21.15.27:0/2948974874 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f6a3406d530 0x7f6a340cf410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:38.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:38.345+0000 7f6a3c52c640 1 --2- 172.21.15.27:0/2948974874 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6a3415a270 0x7f6a3406cff0 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:38.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:38.345+0000 7f6a3c52c640 1 -- 172.21.15.27:0/2948974874 >> 172.21.15.27:0/2948974874 conn(0x7f6a3406bb30 msgr2=0x7f6a3406bf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:38.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:38.345+0000 7f6a3c52c640 1 -- 172.21.15.27:0/2948974874 shutdown_connections 2024-09-12T03:21:38.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:38.345+0000 7f6a3c52c640 1 -- 172.21.15.27:0/2948974874 wait complete. 2024-09-12T03:21:38.348 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:38.346+0000 7f6a3c52c640 1 Processor -- start 2024-09-12T03:21:38.348 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:38.346+0000 7f6a3c52c640 1 -- start start 2024-09-12T03:21:38.348 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:38.347+0000 7f6a3c52c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6a3406c650 0x7f6a341e7f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:38.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:38.347+0000 7f6a3c52c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6a3406d530 0x7f6a34148e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:38.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:38.347+0000 7f6a3c52c640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f6a3415a270 0x7f6a340aa4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:38.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:38.347+0000 7f6a3c52c640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f6a34072f30 con 0x7f6a3406d530 2024-09-12T03:21:38.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:38.347+0000 7f6a3c52c640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f6a34072db0 con 0x7f6a3406c650 2024-09-12T03:21:38.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:38.347+0000 7f6a3c52c640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f6a340730b0 con 0x7f6a3415a270 2024-09-12T03:21:38.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:38.347+0000 7f6a39ac7640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6a3406d530 0x7f6a34148e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:38.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:38.347+0000 7f6a39ac7640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6a3406d530 0x7f6a34148e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42084/0 (socket says 172.21.15.27:42084) 2024-09-12T03:21:38.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:38.347+0000 7f6a39ac7640 1 -- 172.21.15.27:0/1078399665 learned_addr learned my addr 172.21.15.27:0/1078399665 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:21:38.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:38.347+0000 7f6a3aac9640 1 --2- 172.21.15.27:0/1078399665 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f6a3415a270 0x7f6a340aa4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:38.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:38.347+0000 7f6a3a2c8640 1 --2- 172.21.15.27:0/1078399665 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6a3406c650 0x7f6a341e7f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:38.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:38.347+0000 7f6a39ac7640 1 -- 172.21.15.27:0/1078399665 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f6a3415a270 msgr2=0x7f6a340aa4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:38.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:38.347+0000 7f6a39ac7640 1 --2- 172.21.15.27:0/1078399665 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f6a3415a270 0x7f6a340aa4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:38.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:38.347+0000 7f6a39ac7640 1 -- 172.21.15.27:0/1078399665 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6a3406c650 msgr2=0x7f6a341e7f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:38.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:38.347+0000 7f6a39ac7640 1 --2- 172.21.15.27:0/1078399665 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6a3406c650 0x7f6a341e7f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:38.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:38.347+0000 7f6a39ac7640 1 -- 172.21.15.27:0/1078399665 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a341e6e80 con 0x7f6a3406d530 2024-09-12T03:21:38.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:38.347+0000 7f6a3aac9640 1 --2- 172.21.15.27:0/1078399665 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f6a3415a270 0x7f6a340aa4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:21:38.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:38.347+0000 7f6a3a2c8640 1 --2- 172.21.15.27:0/1078399665 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6a3406c650 0x7f6a341e7f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:21:38.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:38.348+0000 7f6a39ac7640 1 --2- 172.21.15.27:0/1078399665 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6a3406d530 0x7f6a34148e40 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f6a2400c7f0 tx=0x7f6a2400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:38.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:38.348+0000 7f6a2affd640 1 -- 172.21.15.27:0/1078399665 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a24015070 con 0x7f6a3406d530 2024-09-12T03:21:38.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:38.348+0000 7f6a2affd640 1 -- 172.21.15.27:0/1078399665 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a24007a30 con 0x7f6a3406d530 2024-09-12T03:21:38.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:38.348+0000 7f6a2affd640 1 -- 172.21.15.27:0/1078399665 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a24007e20 con 0x7f6a3406d530 2024-09-12T03:21:38.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:38.348+0000 7f6a3c52c640 1 -- 172.21.15.27:0/1078399665 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a341e7160 con 0x7f6a3406d530 2024-09-12T03:21:38.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:38.348+0000 7f6a3c52c640 1 -- 172.21.15.27:0/1078399665 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a341e7720 con 0x7f6a3406d530 2024-09-12T03:21:38.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:38.349+0000 7f6a3c52c640 1 -- 172.21.15.27:0/1078399665 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a3415d560 con 0x7f6a3406d530 2024-09-12T03:21:38.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:38.349+0000 7f6a2affd640 1 -- 172.21.15.27:0/1078399665 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 19) ==== 90855+0+0 (secure 0 0 0) 0x7f6a24018070 con 0x7f6a3406d530 2024-09-12T03:21:38.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:38.350+0000 7f6a2affd640 1 --2- 172.21.15.27:0/1078399665 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f6a1806c220 0x7f6a1806e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:38.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:38.350+0000 7f6a2affd640 1 -- 172.21.15.27:0/1078399665 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5443+0+0 (secure 0 0 0) 0x7f6a2408b300 con 0x7f6a3406d530 2024-09-12T03:21:38.352 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:38.350+0000 7f6a3a2c8640 1 --2- 172.21.15.27:0/1078399665 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f6a1806c220 0x7f6a1806e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:38.352 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:38.350+0000 7f6a3a2c8640 1 --2- 172.21.15.27:0/1078399665 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f6a1806c220 0x7f6a1806e6e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f6a30034000 tx=0x7f6a3002e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:38.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:38.354+0000 7f6a2affd640 1 -- 172.21.15.27:0/1078399665 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f6a24059630 con 0x7f6a3406d530 2024-09-12T03:21:38.528 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:38.528+0000 7f6a3c52c640 1 -- 172.21.15.27:0/1078399665 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f6a340cf870 con 0x7f6a3406d530 2024-09-12T03:21:39.215 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:39.215+0000 7f6a2affd640 1 -- 172.21.15.27:0/1078399665 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v106) ==== 169+0+0 (secure 0 0 0) 0x7f6a24058f30 con 0x7f6a3406d530 2024-09-12T03:21:39.216 INFO:teuthology.orchestra.run.smithi027.stderr:pool 'cephfs_data' removed 2024-09-12T03:21:39.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:39.219+0000 7f6a3c52c640 1 -- 172.21.15.27:0/1078399665 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f6a1806c220 msgr2=0x7f6a1806e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:39.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:39.219+0000 7f6a3c52c640 1 --2- 172.21.15.27:0/1078399665 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f6a1806c220 0x7f6a1806e6e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f6a30034000 tx=0x7f6a3002e000 comp rx=0 tx=0).stop 2024-09-12T03:21:39.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:39.219+0000 7f6a3c52c640 1 -- 172.21.15.27:0/1078399665 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6a3406d530 msgr2=0x7f6a34148e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:39.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:39.219+0000 7f6a3c52c640 1 --2- 172.21.15.27:0/1078399665 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6a3406d530 0x7f6a34148e40 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f6a2400c7f0 tx=0x7f6a2400ccb0 comp rx=0 tx=0).stop 2024-09-12T03:21:39.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:39.220+0000 7f6a3c52c640 1 -- 172.21.15.27:0/1078399665 shutdown_connections 2024-09-12T03:21:39.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:39.220+0000 7f6a3c52c640 1 --2- 172.21.15.27:0/1078399665 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f6a3415a270 0x7f6a340aa4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:39.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:39.220+0000 7f6a3c52c640 1 --2- 172.21.15.27:0/1078399665 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f6a1806c220 0x7f6a1806e6e0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:39.222 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:39.220+0000 7f6a3c52c640 1 --2- 172.21.15.27:0/1078399665 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6a3406d530 0x7f6a34148e40 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:39.222 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:39.220+0000 7f6a3c52c640 1 --2- 172.21.15.27:0/1078399665 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6a3406c650 0x7f6a341e7f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:39.222 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:39.220+0000 7f6a3c52c640 1 -- 172.21.15.27:0/1078399665 >> 172.21.15.27:0/1078399665 conn(0x7f6a3406bb30 msgr2=0x7f6a340b1af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:39.222 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:39.220+0000 7f6a3c52c640 1 -- 172.21.15.27:0/1078399665 shutdown_connections 2024-09-12T03:21:39.222 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:39.220+0000 7f6a3c52c640 1 -- 172.21.15.27:0/1078399665 wait complete. 2024-09-12T03:21:39.234 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-09-12T03:21:39.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:39.354+0000 7f5482842640 1 Processor -- start 2024-09-12T03:21:39.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:39.354+0000 7f5482842640 1 -- start start 2024-09-12T03:21:39.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:39.355+0000 7f5482842640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f547c06e550 0x7f547c15ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:39.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:39.355+0000 7f5482842640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f547c0aedd0 con 0x7f547c072610 2024-09-12T03:21:39.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:39.355+0000 7f5482842640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f547c0af0b0 con 0x7f547c0ae8d0 2024-09-12T03:21:39.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:39.355+0000 7f5482842640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f547c0af390 con 0x7f547c06e550 2024-09-12T03:21:39.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:39.355+0000 7f547bfff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f547c06e550 0x7f547c15ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:39.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:39.355+0000 7f5480ddf640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f547c072610 0x7f547c072ac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:34576/0 (socket says 172.21.15.27:34576) 2024-09-12T03:21:39.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:39.355+0000 7f547bfff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f547c06e550 0x7f547c15ebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:50320/0 (socket says 172.21.15.27:50320) 2024-09-12T03:21:39.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:39.355+0000 7f5480ddf640 1 -- 172.21.15.27:0/3294874141 learned_addr learned my addr 172.21.15.27:0/3294874141 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:21:39.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:39.356+0000 7f547bfff640 1 -- 172.21.15.27:0/3294874141 >> v1:172.21.15.45:6789/0 conn(0x7f547c0ae8d0 legacy=0x7f547c06e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:39.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:39.356+0000 7f547bfff640 1 -- 172.21.15.27:0/3294874141 >> v1:172.21.15.27:6789/0 conn(0x7f547c072610 legacy=0x7f547c072ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:39.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:39.356+0000 7f547bfff640 1 -- 172.21.15.27:0/3294874141 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f547c0af670 con 0x7f547c06e550 2024-09-12T03:21:39.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:39.356+0000 7f547bfff640 1 --2- 172.21.15.27:0/3294874141 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f547c06e550 0x7f547c15ebb0 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f546c0097a0 tx=0x7f546c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=591f0de1a7160b58 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:39.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:39.357+0000 7f547affd640 1 -- 172.21.15.27:0/3294874141 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f546c037070 con 0x7f547c06e550 2024-09-12T03:21:39.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:39.357+0000 7f547affd640 1 -- 172.21.15.27:0/3294874141 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f546c032440 con 0x7f547c06e550 2024-09-12T03:21:39.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:39.357+0000 7f547affd640 1 -- 172.21.15.27:0/3294874141 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f546c032810 con 0x7f547c06e550 2024-09-12T03:21:39.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:39.357+0000 7f5482842640 1 -- 172.21.15.27:0/3294874141 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f547c06e550 msgr2=0x7f547c15ebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:39.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:39.357+0000 7f5482842640 1 --2- 172.21.15.27:0/3294874141 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f547c06e550 0x7f547c15ebb0 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f546c0097a0 tx=0x7f546c02a1e0 comp rx=0 tx=0).stop 2024-09-12T03:21:39.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:39.357+0000 7f5482842640 1 -- 172.21.15.27:0/3294874141 shutdown_connections 2024-09-12T03:21:39.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:39.357+0000 7f5482842640 1 --2- 172.21.15.27:0/3294874141 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f547c06e550 0x7f547c15ebb0 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:39.359 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:39.357+0000 7f5482842640 1 -- 172.21.15.27:0/3294874141 >> 172.21.15.27:0/3294874141 conn(0x7f547c0b26c0 msgr2=0x7f547c0b2aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:39.359 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:39.357+0000 7f5482842640 1 -- 172.21.15.27:0/3294874141 shutdown_connections 2024-09-12T03:21:39.359 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:39.357+0000 7f5482842640 1 -- 172.21.15.27:0/3294874141 wait complete. 2024-09-12T03:21:39.360 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:39.359+0000 7f5482842640 1 Processor -- start 2024-09-12T03:21:39.360 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:39.359+0000 7f5482842640 1 -- start start 2024-09-12T03:21:39.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:39.360+0000 7f5482842640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f547c06e550 0x7f547c146830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:39.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:39.360+0000 7f5482842640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f547c072610 0x7f547c1e92d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:39.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:39.360+0000 7f5482842640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f547c0ae8d0 0x7f547c0b1960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:39.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:39.360+0000 7f5482842640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f547c0b3480 con 0x7f547c072610 2024-09-12T03:21:39.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:39.360+0000 7f5482842640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f547c0b3300 con 0x7f547c0ae8d0 2024-09-12T03:21:39.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:39.360+0000 7f5482842640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f547c0b39c0 con 0x7f547c06e550 2024-09-12T03:21:39.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:39.361+0000 7f547b7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f547c072610 0x7f547c1e92d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:39.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:39.361+0000 7f547b7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f547c072610 0x7f547c1e92d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:50322/0 (socket says 172.21.15.27:50322) 2024-09-12T03:21:39.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:39.361+0000 7f547b7fe640 1 -- 172.21.15.27:0/704712394 learned_addr learned my addr 172.21.15.27:0/704712394 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:21:39.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:39.361+0000 7f5480ddf640 1 --2- 172.21.15.27:0/704712394 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f547c0ae8d0 0x7f547c0b1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:39.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:39.361+0000 7f547bfff640 1 --2- 172.21.15.27:0/704712394 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f547c06e550 0x7f547c146830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:39.363 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:39.361+0000 7f547b7fe640 1 -- 172.21.15.27:0/704712394 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f547c06e550 msgr2=0x7f547c146830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:39.363 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:39.361+0000 7f547b7fe640 1 --2- 172.21.15.27:0/704712394 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f547c06e550 0x7f547c146830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:39.363 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:39.361+0000 7f547b7fe640 1 -- 172.21.15.27:0/704712394 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f547c0ae8d0 msgr2=0x7f547c0b1960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:39.363 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:39.361+0000 7f547b7fe640 1 --2- 172.21.15.27:0/704712394 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f547c0ae8d0 0x7f547c0b1960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:39.363 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:39.361+0000 7f547b7fe640 1 -- 172.21.15.27:0/704712394 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f547c146d70 con 0x7f547c072610 2024-09-12T03:21:39.364 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:39.361+0000 7f547b7fe640 1 --2- 172.21.15.27:0/704712394 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f547c072610 0x7f547c1e92d0 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f547000c430 tx=0x7f547000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:39.364 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:39.362+0000 7f5478ff9640 1 -- 172.21.15.27:0/704712394 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5470011790 con 0x7f547c072610 2024-09-12T03:21:39.364 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:39.362+0000 7f5478ff9640 1 -- 172.21.15.27:0/704712394 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5470017070 con 0x7f547c072610 2024-09-12T03:21:39.364 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:39.362+0000 7f5482842640 1 -- 172.21.15.27:0/704712394 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f547c154c80 con 0x7f547c072610 2024-09-12T03:21:39.364 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:39.362+0000 7f5478ff9640 1 -- 172.21.15.27:0/704712394 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5470011bb0 con 0x7f547c072610 2024-09-12T03:21:39.364 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:39.362+0000 7f5482842640 1 -- 172.21.15.27:0/704712394 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f547c1541a0 con 0x7f547c072610 2024-09-12T03:21:39.364 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:39.363+0000 7f5482842640 1 -- 172.21.15.27:0/704712394 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f547c15ebb0 con 0x7f547c072610 2024-09-12T03:21:39.364 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:39.364+0000 7f5478ff9640 1 -- 172.21.15.27:0/704712394 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 19) ==== 90855+0+0 (secure 0 0 0) 0x7f547001c460 con 0x7f547c072610 2024-09-12T03:21:39.365 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:39.364+0000 7f5478ff9640 1 --2- 172.21.15.27:0/704712394 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f546806c220 0x7f546806e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:39.365 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:39.364+0000 7f5478ff9640 1 -- 172.21.15.27:0/704712394 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 4971+0+0 (secure 0 0 0) 0x7f547008b040 con 0x7f547c072610 2024-09-12T03:21:39.365 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:39.364+0000 7f547bfff640 1 --2- 172.21.15.27:0/704712394 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f546806c220 0x7f546806e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:39.365 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:39.365+0000 7f547bfff640 1 --2- 172.21.15.27:0/704712394 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f546806c220 0x7f546806e6e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f546c034000 tx=0x7f546c02e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:39.370 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:39.368+0000 7f5478ff9640 1 -- 172.21.15.27:0/704712394 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f547c146d70 con 0x7f547c072610 2024-09-12T03:21:39.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:39.541+0000 7f5482842640 1 -- 172.21.15.27:0/704712394 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f547c160c30 con 0x7f547c072610 2024-09-12T03:21:39.544 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:39.545+0000 7f5478ff9640 1 -- 172.21.15.27:0/704712394 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v21) ==== 104+0+0 (secure 0 0 0) 0x7f54700590a0 con 0x7f547c072610 2024-09-12T03:21:39.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:39.547+0000 7f5482842640 1 -- 172.21.15.27:0/704712394 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f546806c220 msgr2=0x7f546806e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:39.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:39.547+0000 7f5482842640 1 --2- 172.21.15.27:0/704712394 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f546806c220 0x7f546806e6e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f546c034000 tx=0x7f546c02e000 comp rx=0 tx=0).stop 2024-09-12T03:21:39.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:39.547+0000 7f5482842640 1 -- 172.21.15.27:0/704712394 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f547c072610 msgr2=0x7f547c1e92d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:39.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:39.547+0000 7f5482842640 1 --2- 172.21.15.27:0/704712394 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f547c072610 0x7f547c1e92d0 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f547000c430 tx=0x7f547000c8f0 comp rx=0 tx=0).stop 2024-09-12T03:21:39.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:39.547+0000 7f5482842640 1 -- 172.21.15.27:0/704712394 shutdown_connections 2024-09-12T03:21:39.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:39.547+0000 7f5482842640 1 --2- 172.21.15.27:0/704712394 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f547c0ae8d0 0x7f547c0b1960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:39.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:39.547+0000 7f5482842640 1 --2- 172.21.15.27:0/704712394 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f546806c220 0x7f546806e6e0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:39.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:39.547+0000 7f5482842640 1 --2- 172.21.15.27:0/704712394 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f547c072610 0x7f547c1e92d0 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:39.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:39.547+0000 7f5482842640 1 --2- 172.21.15.27:0/704712394 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f547c06e550 0x7f547c146830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:39.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:39.547+0000 7f5482842640 1 -- 172.21.15.27:0/704712394 >> 172.21.15.27:0/704712394 conn(0x7f547c0b26c0 msgr2=0x7f547c15fda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:39.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:39.547+0000 7f5482842640 1 -- 172.21.15.27:0/704712394 shutdown_connections 2024-09-12T03:21:39.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:39.547+0000 7f5482842640 1 -- 172.21.15.27:0/704712394 wait complete. 2024-09-12T03:21:39.558 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_no_upgrade_v1_sanity' 2024-09-12T03:21:39.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:39.661+0000 7f0ea5482640 1 Processor -- start 2024-09-12T03:21:39.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:39.661+0000 7f0ea5482640 1 -- start start 2024-09-12T03:21:39.662 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:39.662+0000 7f0ea5482640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f0ea015eb90 0x7f0ea00d2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:39.662 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:39.662+0000 7f0ea5482640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f0ea00d5200 0x7f0ea00d1010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:39.663 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:39.662+0000 7f0ea5482640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0ea00d3760 con 0x7f0ea00d1550 2024-09-12T03:21:39.663 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:39.662+0000 7f0ea5482640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f0ea00b7d20 con 0x7f0ea00d5200 2024-09-12T03:21:39.663 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:39.662+0000 7f0ea5482640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f0ea00d42e0 con 0x7f0ea015eb90 2024-09-12T03:21:39.663 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:39.662+0000 7f0e9e7fc640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f0ea00d5200 0x7f0ea00d1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:39.663 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:39.662+0000 7f0e9f7fe640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f0ea00d1550 0x7f0ea00cf200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:34578/0 (socket says 172.21.15.27:34578) 2024-09-12T03:21:39.663 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:39.662+0000 7f0e9f7fe640 1 -- 172.21.15.27:0/710152842 learned_addr learned my addr 172.21.15.27:0/710152842 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:21:39.663 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:39.663+0000 7f0e9e7fc640 1 -- 172.21.15.27:0/710152842 >> v2:172.21.15.45:3300/0 conn(0x7f0ea015eb90 msgr2=0x7f0ea00d2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:39.663 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:39.663+0000 7f0e9e7fc640 1 --2- 172.21.15.27:0/710152842 >> v2:172.21.15.45:3300/0 conn(0x7f0ea015eb90 0x7f0ea00d2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:39.663 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:39.663+0000 7f0e9e7fc640 1 -- 172.21.15.27:0/710152842 >> v1:172.21.15.27:6789/0 conn(0x7f0ea00d1550 legacy=0x7f0ea00cf200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:39.664 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:39.663+0000 7f0e9e7fc640 1 -- 172.21.15.27:0/710152842 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ea00d3a40 con 0x7f0ea00d5200 2024-09-12T03:21:39.664 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:39.663+0000 7f0e9dffb640 1 -- 172.21.15.27:0/710152842 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1337850793 0 0) 0x7f0ea00d3760 con 0x7f0ea00d1550 2024-09-12T03:21:39.665 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:39.663+0000 7f0e9e7fc640 1 --2- 172.21.15.27:0/710152842 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0ea00d5200 0x7f0ea00d1010 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f0ea0075560 tx=0x7f0e9402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56ad5126bea1c9c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:39.665 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:39.664+0000 7f0e9dffb640 1 -- 172.21.15.27:0/710152842 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e94037070 con 0x7f0ea00d5200 2024-09-12T03:21:39.665 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:39.664+0000 7f0e9dffb640 1 -- 172.21.15.27:0/710152842 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e94032440 con 0x7f0ea00d5200 2024-09-12T03:21:39.665 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:39.664+0000 7f0e9dffb640 1 -- 172.21.15.27:0/710152842 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e94032810 con 0x7f0ea00d5200 2024-09-12T03:21:39.665 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:39.664+0000 7f0ea5482640 1 -- 172.21.15.27:0/710152842 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0ea00d5200 msgr2=0x7f0ea00d1010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:39.665 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:39.664+0000 7f0ea5482640 1 --2- 172.21.15.27:0/710152842 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0ea00d5200 0x7f0ea00d1010 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f0ea0075560 tx=0x7f0e9402a1a0 comp rx=0 tx=0).stop 2024-09-12T03:21:39.665 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:39.664+0000 7f0ea5482640 1 -- 172.21.15.27:0/710152842 shutdown_connections 2024-09-12T03:21:39.665 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:39.664+0000 7f0ea5482640 1 --2- 172.21.15.27:0/710152842 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0ea00d5200 0x7f0ea00d1010 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:39.665 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:39.664+0000 7f0ea5482640 1 --2- 172.21.15.27:0/710152842 >> v2:172.21.15.45:3300/0 conn(0x7f0ea015eb90 0x7f0ea00d2df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:39.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:39.664+0000 7f0ea5482640 1 -- 172.21.15.27:0/710152842 >> 172.21.15.27:0/710152842 conn(0x7f0ea006dce0 msgr2=0x7f0ea006e0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:39.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:39.664+0000 7f0ea5482640 1 -- 172.21.15.27:0/710152842 shutdown_connections 2024-09-12T03:21:39.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:39.664+0000 7f0ea5482640 1 -- 172.21.15.27:0/710152842 wait complete. 2024-09-12T03:21:39.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:39.665+0000 7f0ea5482640 1 Processor -- start 2024-09-12T03:21:39.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:39.666+0000 7f0ea5482640 1 -- start start 2024-09-12T03:21:39.669 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:39.666+0000 7f0ea5482640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0ea00d1550 0x7f0ea014d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:39.669 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:39.666+0000 7f0ea5482640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0ea00d5200 0x7f0ea00ca1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:39.669 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:39.666+0000 7f0ea5482640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f0ea015eb90 0x7f0ea00ca710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:39.669 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:39.666+0000 7f0ea5482640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f0ea00d1960 con 0x7f0ea00d1550 2024-09-12T03:21:39.669 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:39.666+0000 7f0ea5482640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f0ea00d4000 con 0x7f0ea00d5200 2024-09-12T03:21:39.669 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:39.666+0000 7f0ea5482640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f0ea00d3330 con 0x7f0ea015eb90 2024-09-12T03:21:39.669 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:39.666+0000 7f0e9effd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0ea00d1550 0x7f0ea014d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:39.669 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:39.666+0000 7f0e9effd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0ea00d1550 0x7f0ea014d350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:50330/0 (socket says 172.21.15.27:50330) 2024-09-12T03:21:39.669 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:39.666+0000 7f0e9effd640 1 -- 172.21.15.27:0/449042949 learned_addr learned my addr 172.21.15.27:0/449042949 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:21:39.669 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:39.667+0000 7f0e9e7fc640 1 --2- 172.21.15.27:0/449042949 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0ea00d5200 0x7f0ea00ca1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:39.669 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:39.667+0000 7f0e9f7fe640 1 --2- 172.21.15.27:0/449042949 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f0ea015eb90 0x7f0ea00ca710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:39.669 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:39.667+0000 7f0e9effd640 1 -- 172.21.15.27:0/449042949 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f0ea015eb90 msgr2=0x7f0ea00ca710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:39.669 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:39.667+0000 7f0e9effd640 1 --2- 172.21.15.27:0/449042949 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f0ea015eb90 0x7f0ea00ca710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:39.670 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:39.667+0000 7f0e9effd640 1 -- 172.21.15.27:0/449042949 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0ea00d5200 msgr2=0x7f0ea00ca1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:39.670 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:39.667+0000 7f0e9effd640 1 --2- 172.21.15.27:0/449042949 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0ea00d5200 0x7f0ea00ca1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:39.670 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:39.667+0000 7f0e9effd640 1 -- 172.21.15.27:0/449042949 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ea015d000 con 0x7f0ea00d1550 2024-09-12T03:21:39.670 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:39.667+0000 7f0e9e7fc640 1 --2- 172.21.15.27:0/449042949 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0ea00d5200 0x7f0ea00ca1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:21:39.670 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:39.667+0000 7f0e9f7fe640 1 --2- 172.21.15.27:0/449042949 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f0ea015eb90 0x7f0ea00ca710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:21:39.670 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:39.667+0000 7f0e9effd640 1 --2- 172.21.15.27:0/449042949 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0ea00d1550 0x7f0ea014d350 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f0e9800b410 tx=0x7f0e9800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:39.670 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:39.667+0000 7f0e877fe640 1 -- 172.21.15.27:0/449042949 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e98015020 con 0x7f0ea00d1550 2024-09-12T03:21:39.670 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:39.667+0000 7f0e877fe640 1 -- 172.21.15.27:0/449042949 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e98011850 con 0x7f0ea00d1550 2024-09-12T03:21:39.670 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:39.667+0000 7f0ea5482640 1 -- 172.21.15.27:0/449042949 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ea00cad70 con 0x7f0ea00d1550 2024-09-12T03:21:39.670 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:39.667+0000 7f0e877fe640 1 -- 172.21.15.27:0/449042949 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e98011c40 con 0x7f0ea00d1550 2024-09-12T03:21:39.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:39.667+0000 7f0ea5482640 1 -- 172.21.15.27:0/449042949 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ea014de00 con 0x7f0ea00d1550 2024-09-12T03:21:39.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:39.668+0000 7f0ea5482640 1 -- 172.21.15.27:0/449042949 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ea00cb290 con 0x7f0ea00d1550 2024-09-12T03:21:39.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:39.669+0000 7f0e877fe640 1 -- 172.21.15.27:0/449042949 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 19) ==== 90855+0+0 (secure 0 0 0) 0x7f0e98018020 con 0x7f0ea00d1550 2024-09-12T03:21:39.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:39.669+0000 7f0e877fe640 1 --2- 172.21.15.27:0/449042949 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f0e8c06c220 0x7f0e8c06e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:39.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:39.669+0000 7f0e9e7fc640 1 --2- 172.21.15.27:0/449042949 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f0e8c06c220 0x7f0e8c06e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:39.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:39.669+0000 7f0e877fe640 1 -- 172.21.15.27:0/449042949 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 4971+0+0 (secure 0 0 0) 0x7f0e9808b1e0 con 0x7f0ea00d1550 2024-09-12T03:21:39.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:39.670+0000 7f0e9e7fc640 1 --2- 172.21.15.27:0/449042949 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f0e8c06c220 0x7f0e8c06e6e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f0e94034000 tx=0x7f0e9402e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:39.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:39.674+0000 7f0e877fe640 1 -- 172.21.15.27:0/449042949 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f0ea00cb290 con 0x7f0ea00d1550 2024-09-12T03:21:39.918 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:39.918+0000 7f0ea5482640 1 -- 172.21.15.27:0/449042949 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_no_upgrade_v1_sanity"]} v 0) -- 0x7f0ea0160190 con 0x7f0ea00d1550 2024-09-12T03:21:40.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.217+0000 7f0e877fe640 1 -- 172.21.15.27:0/449042949 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_no_upgrade_v1_sanity"]}]=0 v282) ==== 149+0+0 (secure 0 0 0) 0x7f0e9801f020 con 0x7f0ea00d1550 2024-09-12T03:21:40.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.219+0000 7f0ea5482640 1 -- 172.21.15.27:0/449042949 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f0e8c06c220 msgr2=0x7f0e8c06e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:40.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.219+0000 7f0ea5482640 1 --2- 172.21.15.27:0/449042949 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f0e8c06c220 0x7f0e8c06e6e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f0e94034000 tx=0x7f0e9402e000 comp rx=0 tx=0).stop 2024-09-12T03:21:40.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.219+0000 7f0ea5482640 1 -- 172.21.15.27:0/449042949 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0ea00d1550 msgr2=0x7f0ea014d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:40.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.219+0000 7f0ea5482640 1 --2- 172.21.15.27:0/449042949 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0ea00d1550 0x7f0ea014d350 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f0e9800b410 tx=0x7f0e9800b8d0 comp rx=0 tx=0).stop 2024-09-12T03:21:40.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.219+0000 7f0ea5482640 1 -- 172.21.15.27:0/449042949 shutdown_connections 2024-09-12T03:21:40.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.219+0000 7f0ea5482640 1 --2- 172.21.15.27:0/449042949 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f0ea015eb90 0x7f0ea00ca710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:40.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.219+0000 7f0ea5482640 1 --2- 172.21.15.27:0/449042949 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0ea00d5200 0x7f0ea00ca1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:40.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.219+0000 7f0ea5482640 1 --2- 172.21.15.27:0/449042949 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f0e8c06c220 0x7f0e8c06e6e0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:40.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.219+0000 7f0ea5482640 1 --2- 172.21.15.27:0/449042949 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0ea00d1550 0x7f0ea014d350 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:40.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.219+0000 7f0ea5482640 1 -- 172.21.15.27:0/449042949 >> 172.21.15.27:0/449042949 conn(0x7f0ea006dce0 msgr2=0x7f0ea00cfcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:40.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.219+0000 7f0ea5482640 1 -- 172.21.15.27:0/449042949 shutdown_connections 2024-09-12T03:21:40.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.219+0000 7f0ea5482640 1 -- 172.21.15.27:0/449042949 wait complete. 2024-09-12T03:21:40.231 INFO:tasks.cephfs_test_runner:poor man's upgrade test -- theme continues... ... ok 2024-09-12T03:21:40.231 INFO:tasks.cephfs_test_runner:Starting test: test_subvolume_no_upgrade_v1_to_v2 (tasks.cephfs.test_volumes.TestMisc) poor man's upgrade test -- theme continues... 2024-09-12T03:21:40.231 INFO:tasks.cephfs_test_runner:test_subvolume_no_upgrade_v1_to_v2 (tasks.cephfs.test_volumes.TestMisc) 2024-09-12T03:21:40.232 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-12T03:21:40.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.336+0000 7f2a9775f640 1 Processor -- start 2024-09-12T03:21:40.337 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.336+0000 7f2a9775f640 1 -- start start 2024-09-12T03:21:40.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.337+0000 7f2a9775f640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2a9015b2b0 0x7f2a900b3560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:40.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.337+0000 7f2a9775f640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2a900ce410 con 0x7f2a90072d70 2024-09-12T03:21:40.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.337+0000 7f2a9775f640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2a900732c0 con 0x7f2a9006e9b0 2024-09-12T03:21:40.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.337+0000 7f2a9775f640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f2a900720a0 con 0x7f2a9015b2b0 2024-09-12T03:21:40.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.337+0000 7f2a94cfa640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f2a90072d70 0x7f2a90072430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:34586/0 (socket says 172.21.15.27:34586) 2024-09-12T03:21:40.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.337+0000 7f2a94cfa640 1 -- 172.21.15.27:0/1730924597 learned_addr learned my addr 172.21.15.27:0/1730924597 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:21:40.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.338+0000 7f2a954fb640 1 --2- 172.21.15.27:0/1730924597 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2a9015b2b0 0x7f2a900b3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:40.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.338+0000 7f2a8ffff640 1 -- 172.21.15.27:0/1730924597 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1933574579 0 0) 0x7f2a84001110 con 0x7f2a90072d70 2024-09-12T03:21:40.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.338+0000 7f2a8ffff640 1 -- 172.21.15.27:0/1730924597 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2a78003520 con 0x7f2a90072d70 2024-09-12T03:21:40.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.338+0000 7f2a8ffff640 1 -- 172.21.15.27:0/1730924597 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1655261951 0 0) 0x7f2a78003520 con 0x7f2a90072d70 2024-09-12T03:21:40.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.338+0000 7f2a8ffff640 1 -- 172.21.15.27:0/1730924597 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2a9015b2b0 msgr2=0x7f2a900b3560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:40.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.338+0000 7f2a8ffff640 1 --2- 172.21.15.27:0/1730924597 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2a9015b2b0 0x7f2a900b3560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:40.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.338+0000 7f2a8ffff640 1 -- 172.21.15.27:0/1730924597 >> v1:172.21.15.45:6789/0 conn(0x7f2a9006e9b0 legacy=0x7f2a900cebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:40.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.338+0000 7f2a8ffff640 1 -- 172.21.15.27:0/1730924597 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a90071ae0 con 0x7f2a90072d70 2024-09-12T03:21:40.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.338+0000 7f2a8ffff640 1 -- 172.21.15.27:0/1730924597 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 847755213 0 0) 0x7f2a84001840 con 0x7f2a90072d70 2024-09-12T03:21:40.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.338+0000 7f2a8ffff640 1 -- 172.21.15.27:0/1730924597 >> v1:172.21.15.27:6789/0 conn(0x7f2a90072d70 legacy=0x7f2a90072430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:40.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.338+0000 7f2a8ffff640 1 --2- 172.21.15.27:0/1730924597 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2a780039e0 0x7f2a78003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:40.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.338+0000 7f2a8ffff640 1 --2- 172.21.15.27:0/1730924597 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2a78004390 0x7f2a78004860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:40.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.338+0000 7f2a8ffff640 1 --2- 172.21.15.27:0/1730924597 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2a9015b2b0 0x7f2a780050c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:40.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.338+0000 7f2a8ffff640 1 -- 172.21.15.27:0/1730924597 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f2a900720a0 con 0x7f2a78004390 2024-09-12T03:21:40.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.338+0000 7f2a8ffff640 1 -- 172.21.15.27:0/1730924597 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f2a7800b9c0 con 0x7f2a780039e0 2024-09-12T03:21:40.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.338+0000 7f2a8ffff640 1 -- 172.21.15.27:0/1730924597 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f2a7800ddd0 con 0x7f2a9015b2b0 2024-09-12T03:21:40.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.338+0000 7f2a94cfa640 1 --2- 172.21.15.27:0/1730924597 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2a78004390 0x7f2a78004860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:40.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.338+0000 7f2a954fb640 1 --2- 172.21.15.27:0/1730924597 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2a780039e0 0x7f2a78003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:40.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.338+0000 7f2a95cfc640 1 --2- 172.21.15.27:0/1730924597 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2a9015b2b0 0x7f2a780050c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:40.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.338+0000 7f2a94cfa640 1 -- 172.21.15.27:0/1730924597 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2a9015b2b0 msgr2=0x7f2a780050c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:40.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.338+0000 7f2a94cfa640 1 --2- 172.21.15.27:0/1730924597 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2a9015b2b0 0x7f2a780050c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:40.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.338+0000 7f2a94cfa640 1 -- 172.21.15.27:0/1730924597 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2a780039e0 msgr2=0x7f2a78003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:40.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.338+0000 7f2a94cfa640 1 --2- 172.21.15.27:0/1730924597 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2a780039e0 0x7f2a78003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:40.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.338+0000 7f2a94cfa640 1 -- 172.21.15.27:0/1730924597 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a78003520 con 0x7f2a78004390 2024-09-12T03:21:40.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.339+0000 7f2a94cfa640 1 --2- 172.21.15.27:0/1730924597 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2a78004390 0x7f2a78004860 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f2a8400c4c0 tx=0x7f2a78005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4475dea01aff8712 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:40.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.339+0000 7f2a8ffff640 1 -- 172.21.15.27:0/1730924597 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a8400c0e0 con 0x7f2a78004390 2024-09-12T03:21:40.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.339+0000 7f2a8ffff640 1 -- 172.21.15.27:0/1730924597 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a84034070 con 0x7f2a78004390 2024-09-12T03:21:40.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.339+0000 7f2a8ffff640 1 -- 172.21.15.27:0/1730924597 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a84005d10 con 0x7f2a78004390 2024-09-12T03:21:40.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.340+0000 7f2a9775f640 1 -- 172.21.15.27:0/1730924597 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2a78004390 msgr2=0x7f2a78004860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:40.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.340+0000 7f2a9775f640 1 --2- 172.21.15.27:0/1730924597 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2a78004390 0x7f2a78004860 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f2a8400c4c0 tx=0x7f2a78005830 comp rx=0 tx=0).stop 2024-09-12T03:21:40.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.340+0000 7f2a95cfc640 1 -- 172.21.15.27:0/1730924597 reap_dead start 2024-09-12T03:21:40.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.340+0000 7f2a9775f640 1 -- 172.21.15.27:0/1730924597 shutdown_connections 2024-09-12T03:21:40.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.340+0000 7f2a9775f640 1 -- 172.21.15.27:0/1730924597 >> 172.21.15.27:0/1730924597 conn(0x7f2a90062c60 msgr2=0x7f2a9006bb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:40.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.340+0000 7f2a9775f640 1 -- 172.21.15.27:0/1730924597 shutdown_connections 2024-09-12T03:21:40.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.340+0000 7f2a9775f640 1 -- 172.21.15.27:0/1730924597 wait complete. 2024-09-12T03:21:40.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.341+0000 7f2a9775f640 1 Processor -- start 2024-09-12T03:21:40.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.341+0000 7f2a9775f640 1 -- start start 2024-09-12T03:21:40.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.342+0000 7f2a9775f640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2a78004390 0x7f2a9014af60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:40.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.342+0000 7f2a9775f640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2a9014ff90 0x7f2a90152260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:40.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.342+0000 7f2a9775f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2a900b47c0 0x7f2a900b4c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:40.348 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.342+0000 7f2a9775f640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f2a9015b690 con 0x7f2a78004390 2024-09-12T03:21:40.348 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.342+0000 7f2a9775f640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f2a900ce690 con 0x7f2a900b47c0 2024-09-12T03:21:40.348 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.342+0000 7f2a9775f640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f2a9015b810 con 0x7f2a9014ff90 2024-09-12T03:21:40.348 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.342+0000 7f2a954fb640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2a78004390 0x7f2a9014af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:40.348 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.342+0000 7f2a954fb640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2a78004390 0x7f2a9014af60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:50338/0 (socket says 172.21.15.27:50338) 2024-09-12T03:21:40.348 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.342+0000 7f2a954fb640 1 -- 172.21.15.27:0/3269019415 learned_addr learned my addr 172.21.15.27:0/3269019415 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:21:40.348 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.342+0000 7f2a94cfa640 1 --2- 172.21.15.27:0/3269019415 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2a9014ff90 0x7f2a90152260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:40.348 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.342+0000 7f2a95cfc640 1 --2- 172.21.15.27:0/3269019415 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2a900b47c0 0x7f2a900b4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:40.348 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.342+0000 7f2a954fb640 1 -- 172.21.15.27:0/3269019415 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2a9014ff90 msgr2=0x7f2a90152260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:40.348 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.342+0000 7f2a954fb640 1 --2- 172.21.15.27:0/3269019415 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2a9014ff90 0x7f2a90152260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:40.348 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.342+0000 7f2a954fb640 1 -- 172.21.15.27:0/3269019415 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2a900b47c0 msgr2=0x7f2a900b4c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:40.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.342+0000 7f2a954fb640 1 --2- 172.21.15.27:0/3269019415 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2a900b47c0 0x7f2a900b4c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:40.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.342+0000 7f2a954fb640 1 -- 172.21.15.27:0/3269019415 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a90150500 con 0x7f2a78004390 2024-09-12T03:21:40.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.342+0000 7f2a954fb640 1 --2- 172.21.15.27:0/3269019415 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2a78004390 0x7f2a9014af60 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f2a80011e90 tx=0x7f2a8000f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:40.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.343+0000 7f2a8dffb640 1 -- 172.21.15.27:0/3269019415 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a80015630 con 0x7f2a78004390 2024-09-12T03:21:40.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.343+0000 7f2a8dffb640 1 -- 172.21.15.27:0/3269019415 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a800157d0 con 0x7f2a78004390 2024-09-12T03:21:40.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.343+0000 7f2a8dffb640 1 -- 172.21.15.27:0/3269019415 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a80015ba0 con 0x7f2a78004390 2024-09-12T03:21:40.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.343+0000 7f2a9775f640 1 -- 172.21.15.27:0/3269019415 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a901507e0 con 0x7f2a78004390 2024-09-12T03:21:40.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.343+0000 7f2a9775f640 1 -- 172.21.15.27:0/3269019415 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a9015cca0 con 0x7f2a78004390 2024-09-12T03:21:40.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.344+0000 7f2a9775f640 1 -- 172.21.15.27:0/3269019415 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a900b3560 con 0x7f2a78004390 2024-09-12T03:21:40.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.344+0000 7f2a8dffb640 1 -- 172.21.15.27:0/3269019415 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 19) ==== 90855+0+0 (secure 0 0 0) 0x7f2a8001d400 con 0x7f2a78004390 2024-09-12T03:21:40.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.345+0000 7f2a8dffb640 1 --2- 172.21.15.27:0/3269019415 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f2a7c06c250 0x7f2a7c06e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:40.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.345+0000 7f2a8dffb640 1 -- 172.21.15.27:0/3269019415 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 4971+0+0 (secure 0 0 0) 0x7f2a8008c630 con 0x7f2a78004390 2024-09-12T03:21:40.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.345+0000 7f2a94cfa640 1 --2- 172.21.15.27:0/3269019415 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f2a7c06c250 0x7f2a7c06e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:40.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.345+0000 7f2a94cfa640 1 --2- 172.21.15.27:0/3269019415 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f2a7c06c250 0x7f2a7c06e710 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f2a8400c4c0 tx=0x7f2a8402e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:40.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.349+0000 7f2a8dffb640 1 -- 172.21.15.27:0/3269019415 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f2a80019030 con 0x7f2a78004390 2024-09-12T03:21:40.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.525+0000 7f2a9775f640 1 -- 172.21.15.27:0/3269019415 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2a900ceef0 con 0x7f2a78004390 2024-09-12T03:21:40.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.526+0000 7f2a8dffb640 1 -- 172.21.15.27:0/3269019415 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v106) ==== 74+0+13032 (secure 0 0 0) 0x7f2a80010040 con 0x7f2a78004390 2024-09-12T03:21:40.526 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:21:40.526 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":106,"fsid":"d2df504e-2647-4385-a5db-9fdf69f487a0","created":"2024-09-12T03:16:07.778937+0000","modified":"2024-09-12T03:21:39.210825+0000","last_up_change":"2024-09-12T03:16:19.314778+0000","last_in_change":"2024-09-12T03:16:09.893086+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T03:16:20.097646+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1851116c-1fb3-4dc9-9772-f90e2ef4c355","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6817","nonce":3002321488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6819","nonce":3002321488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6823","nonce":3002321488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6821","nonce":3002321488}]},"public_addr":"172.21.15.27:6817/3002321488","cluster_addr":"172.21.15.27:6819/3002321488","heartbeat_back_addr":"172.21.15.27:6823/3002321488","heartbeat_front_addr":"172.21.15.27:6821/3002321488","state":["exists","up"]},{"osd":1,"uuid":"f38ae17d-73eb-4204-b99d-975fa806fb87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6809","nonce":1613102886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6811","nonce":1613102886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6815","nonce":1613102886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6813","nonce":1613102886}]},"public_addr":"172.21.15.27:6809/1613102886","cluster_addr":"172.21.15.27:6811/1613102886","heartbeat_back_addr":"172.21.15.27:6815/1613102886","heartbeat_front_addr":"172.21.15.27:6813/1613102886","state":["exists","up"]},{"osd":2,"uuid":"fb12dcc8-bdd0-42b8-b46d-8b10106224bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6825","nonce":3534723355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6827","nonce":3534723355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6831","nonce":3534723355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6829","nonce":3534723355}]},"public_addr":"172.21.15.27:6825/3534723355","cluster_addr":"172.21.15.27:6827/3534723355","heartbeat_back_addr":"172.21.15.27:6831/3534723355","heartbeat_front_addr":"172.21.15.27:6829/3534723355","state":["exists","up"]},{"osd":3,"uuid":"d74d3f49-9c24-4b9f-98e2-3ecd7b5ef3b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6801","nonce":80661598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6803","nonce":80661598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6807","nonce":80661598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6805","nonce":80661598}]},"public_addr":"172.21.15.27:6801/80661598","cluster_addr":"172.21.15.27:6803/80661598","heartbeat_back_addr":"172.21.15.27:6807/80661598","heartbeat_front_addr":"172.21.15.27:6805/80661598","state":["exists","up"]},{"osd":4,"uuid":"2bf123f2-1b0d-41c4-aa3b-805bab267e59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6817","nonce":3943189827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6819","nonce":3943189827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6823","nonce":3943189827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6821","nonce":3943189827}]},"public_addr":"172.21.15.45:6817/3943189827","cluster_addr":"172.21.15.45:6819/3943189827","heartbeat_back_addr":"172.21.15.45:6823/3943189827","heartbeat_front_addr":"172.21.15.45:6821/3943189827","state":["exists","up"]},{"osd":5,"uuid":"c49ba5da-91c6-4a86-a97b-4aba89c253ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6809","nonce":3620760330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6811","nonce":3620760330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6815","nonce":3620760330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6813","nonce":3620760330}]},"public_addr":"172.21.15.45:6809/3620760330","cluster_addr":"172.21.15.45:6811/3620760330","heartbeat_back_addr":"172.21.15.45:6815/3620760330","heartbeat_front_addr":"172.21.15.45:6813/3620760330","state":["exists","up"]},{"osd":6,"uuid":"b0123475-89f7-4a34-9e33-d797b99dd9dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6801","nonce":1989083510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6803","nonce":1989083510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6807","nonce":1989083510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6805","nonce":1989083510}]},"public_addr":"172.21.15.45:6801/1989083510","cluster_addr":"172.21.15.45:6803/1989083510","heartbeat_back_addr":"172.21.15.45:6807/1989083510","heartbeat_front_addr":"172.21.15.45:6805/1989083510","state":["exists","up"]},{"osd":7,"uuid":"4dad773d-b7d8-46fc-8a10-a972aad27b29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6825","nonce":1513300294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6827","nonce":1513300294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6831","nonce":1513300294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6829","nonce":1513300294}]},"public_addr":"172.21.15.45:6825/1513300294","cluster_addr":"172.21.15.45:6827/1513300294","heartbeat_back_addr":"172.21.15.45:6831/1513300294","heartbeat_front_addr":"172.21.15.45:6829/1513300294","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.881855+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:17.133252+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.735339+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.500742+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.477571+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:17.167667+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.470291+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.873743+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.27:6835/1379166526":"2024-09-13T03:21:36.026922+0000","172.21.15.27:6833/1379166526":"2024-09-13T03:21:36.026922+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T03:21:40.528 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.528+0000 7f2a9775f640 1 -- 172.21.15.27:0/3269019415 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f2a7c06c250 msgr2=0x7f2a7c06e710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:40.529 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.528+0000 7f2a9775f640 1 --2- 172.21.15.27:0/3269019415 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f2a7c06c250 0x7f2a7c06e710 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f2a8400c4c0 tx=0x7f2a8402e000 comp rx=0 tx=0).stop 2024-09-12T03:21:40.529 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.528+0000 7f2a9775f640 1 -- 172.21.15.27:0/3269019415 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2a78004390 msgr2=0x7f2a9014af60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:40.529 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.528+0000 7f2a9775f640 1 --2- 172.21.15.27:0/3269019415 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2a78004390 0x7f2a9014af60 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f2a80011e90 tx=0x7f2a8000f580 comp rx=0 tx=0).stop 2024-09-12T03:21:40.529 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.528+0000 7f2a9775f640 1 -- 172.21.15.27:0/3269019415 shutdown_connections 2024-09-12T03:21:40.529 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.528+0000 7f2a9775f640 1 --2- 172.21.15.27:0/3269019415 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2a900b47c0 0x7f2a900b4c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:40.529 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.528+0000 7f2a9775f640 1 --2- 172.21.15.27:0/3269019415 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2a9014ff90 0x7f2a90152260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:40.529 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.528+0000 7f2a9775f640 1 --2- 172.21.15.27:0/3269019415 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f2a7c06c250 0x7f2a7c06e710 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:40.529 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.528+0000 7f2a9775f640 1 --2- 172.21.15.27:0/3269019415 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2a78004390 0x7f2a9014af60 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:40.529 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.528+0000 7f2a9775f640 1 -- 172.21.15.27:0/3269019415 >> 172.21.15.27:0/3269019415 conn(0x7f2a90062c60 msgr2=0x7f2a9014b880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:40.529 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.528+0000 7f2a9775f640 1 -- 172.21.15.27:0/3269019415 shutdown_connections 2024-09-12T03:21:40.529 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.528+0000 7f2a9775f640 1 -- 172.21.15.27:0/3269019415 wait complete. 2024-09-12T03:21:40.539 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-12T03:16:20.097646+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-12T03:21:40.540 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-12T03:21:40.645 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.645+0000 7f46551af640 1 Processor -- start 2024-09-12T03:21:40.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.645+0000 7f46551af640 1 -- start start 2024-09-12T03:21:40.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.646+0000 7f46551af640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f4650070d30 0x7f465006cc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:40.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.646+0000 7f46551af640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f46500d0b50 0x7f4650168300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:40.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.646+0000 7f46551af640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f46500cf9f0 con 0x7f465006e960 2024-09-12T03:21:40.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.646+0000 7f46551af640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f46500d1a40 con 0x7f4650070d30 2024-09-12T03:21:40.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.646+0000 7f46551af640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f46500cfc90 con 0x7f46500d0b50 2024-09-12T03:21:40.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.646+0000 7f464ed9d640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f4650070d30 0x7f465006cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:40.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.646+0000 7f464e59c640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f465006e960 0x7f46500d0440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:34598/0 (socket says 172.21.15.27:34598) 2024-09-12T03:21:40.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.646+0000 7f464e59c640 1 -- 172.21.15.27:0/3828677772 learned_addr learned my addr 172.21.15.27:0/3828677772 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:21:40.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.646+0000 7f464ed9d640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f4650070d30 0x7f465006cc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:50342/0 (socket says 172.21.15.27:50342) 2024-09-12T03:21:40.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.646+0000 7f464f59e640 1 --2- 172.21.15.27:0/3828677772 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f46500d0b50 0x7f4650168300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:40.650 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.647+0000 7f464ed9d640 1 -- 172.21.15.27:0/3828677772 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f46500d0b50 msgr2=0x7f4650168300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:40.650 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.647+0000 7f464ed9d640 1 --2- 172.21.15.27:0/3828677772 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f46500d0b50 0x7f4650168300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:40.650 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.647+0000 7f464ed9d640 1 -- 172.21.15.27:0/3828677772 >> v1:172.21.15.27:6789/0 conn(0x7f465006e960 legacy=0x7f46500d0440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:40.650 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.647+0000 7f464dd9b640 1 -- 172.21.15.27:0/3828677772 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 773614786 0 0) 0x7f46500cf9f0 con 0x7f465006e960 2024-09-12T03:21:40.650 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.647+0000 7f464ed9d640 1 -- 172.21.15.27:0/3828677772 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46500cff70 con 0x7f4650070d30 2024-09-12T03:21:40.650 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.647+0000 7f464f59e640 1 --2- 172.21.15.27:0/3828677772 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f46500d0b50 0x7f4650168300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:21:40.650 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.647+0000 7f464ed9d640 1 --2- 172.21.15.27:0/3828677772 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4650070d30 0x7f465006cc40 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f4640009d90 tx=0x7f464002a530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a50a9fc9a4d9033a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:40.650 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.648+0000 7f464dd9b640 1 -- 172.21.15.27:0/3828677772 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4640007740 con 0x7f4650070d30 2024-09-12T03:21:40.651 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.648+0000 7f464dd9b640 1 -- 172.21.15.27:0/3828677772 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46400078e0 con 0x7f4650070d30 2024-09-12T03:21:40.651 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.648+0000 7f464dd9b640 1 -- 172.21.15.27:0/3828677772 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4640007cb0 con 0x7f4650070d30 2024-09-12T03:21:40.651 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.648+0000 7f46551af640 1 -- 172.21.15.27:0/3828677772 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4650070d30 msgr2=0x7f465006cc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:40.651 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.648+0000 7f46551af640 1 --2- 172.21.15.27:0/3828677772 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4650070d30 0x7f465006cc40 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f4640009d90 tx=0x7f464002a530 comp rx=0 tx=0).stop 2024-09-12T03:21:40.651 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.648+0000 7f46551af640 1 -- 172.21.15.27:0/3828677772 shutdown_connections 2024-09-12T03:21:40.651 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.648+0000 7f46551af640 1 --2- 172.21.15.27:0/3828677772 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f46500d0b50 0x7f4650168300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:40.651 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.648+0000 7f46551af640 1 --2- 172.21.15.27:0/3828677772 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4650070d30 0x7f465006cc40 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:40.651 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.648+0000 7f46551af640 1 -- 172.21.15.27:0/3828677772 >> 172.21.15.27:0/3828677772 conn(0x7f46500b7c10 msgr2=0x7f465009ba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:40.651 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.648+0000 7f46551af640 1 -- 172.21.15.27:0/3828677772 shutdown_connections 2024-09-12T03:21:40.651 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.648+0000 7f46551af640 1 -- 172.21.15.27:0/3828677772 wait complete. 2024-09-12T03:21:40.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.650+0000 7f46551af640 1 Processor -- start 2024-09-12T03:21:40.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.650+0000 7f46551af640 1 -- start start 2024-09-12T03:21:40.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.650+0000 7f46551af640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f465006e960 0x7f465015ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:40.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.650+0000 7f46551af640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4650070d30 0x7f46501f55e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:40.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.650+0000 7f46551af640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f46500d0b50 0x7f46501611e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:40.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.650+0000 7f46551af640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f46500d31c0 con 0x7f4650070d30 2024-09-12T03:21:40.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.650+0000 7f46551af640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f465006d180 con 0x7f46500d0b50 2024-09-12T03:21:40.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.650+0000 7f46551af640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f46500d3340 con 0x7f465006e960 2024-09-12T03:21:40.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.651+0000 7f464e59c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4650070d30 0x7f46501f55e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:40.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.651+0000 7f464e59c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4650070d30 0x7f46501f55e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:50344/0 (socket says 172.21.15.27:50344) 2024-09-12T03:21:40.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.651+0000 7f464e59c640 1 -- 172.21.15.27:0/2108423764 learned_addr learned my addr 172.21.15.27:0/2108423764 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:21:40.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.651+0000 7f464f59e640 1 --2- 172.21.15.27:0/2108423764 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f46500d0b50 0x7f46501611e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:40.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.651+0000 7f464ed9d640 1 --2- 172.21.15.27:0/2108423764 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f465006e960 0x7f465015ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:40.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.651+0000 7f464e59c640 1 -- 172.21.15.27:0/2108423764 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f465006e960 msgr2=0x7f465015ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:40.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.651+0000 7f464e59c640 1 --2- 172.21.15.27:0/2108423764 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f465006e960 0x7f465015ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:40.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.651+0000 7f464e59c640 1 -- 172.21.15.27:0/2108423764 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f46500d0b50 msgr2=0x7f46501611e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:40.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.651+0000 7f464e59c640 1 --2- 172.21.15.27:0/2108423764 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f46500d0b50 0x7f46501611e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:40.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.651+0000 7f464e59c640 1 -- 172.21.15.27:0/2108423764 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46501f5b20 con 0x7f4650070d30 2024-09-12T03:21:40.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.651+0000 7f464e59c640 1 --2- 172.21.15.27:0/2108423764 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4650070d30 0x7f46501f55e0 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f463800c4a0 tx=0x7f463800c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:40.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.651+0000 7f46477fe640 1 -- 172.21.15.27:0/2108423764 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4638017070 con 0x7f4650070d30 2024-09-12T03:21:40.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.651+0000 7f46477fe640 1 -- 172.21.15.27:0/2108423764 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4638005ce0 con 0x7f4650070d30 2024-09-12T03:21:40.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.651+0000 7f46477fe640 1 -- 172.21.15.27:0/2108423764 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46380072b0 con 0x7f4650070d30 2024-09-12T03:21:40.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.651+0000 7f46551af640 1 -- 172.21.15.27:0/2108423764 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f465015ed40 con 0x7f4650070d30 2024-09-12T03:21:40.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.651+0000 7f46551af640 1 -- 172.21.15.27:0/2108423764 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f465015f300 con 0x7f4650070d30 2024-09-12T03:21:40.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.652+0000 7f46551af640 1 -- 172.21.15.27:0/2108423764 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f465006cc40 con 0x7f4650070d30 2024-09-12T03:21:40.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.653+0000 7f46477fe640 1 -- 172.21.15.27:0/2108423764 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 19) ==== 90855+0+0 (secure 0 0 0) 0x7f463801a070 con 0x7f4650070d30 2024-09-12T03:21:40.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.653+0000 7f46477fe640 1 --2- 172.21.15.27:0/2108423764 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f463006c220 0x7f463006e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:40.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.653+0000 7f46477fe640 1 -- 172.21.15.27:0/2108423764 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 4971+0+0 (secure 0 0 0) 0x7f463808a640 con 0x7f4650070d30 2024-09-12T03:21:40.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.653+0000 7f464ed9d640 1 --2- 172.21.15.27:0/2108423764 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f463006c220 0x7f463006e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:40.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.654+0000 7f464ed9d640 1 --2- 172.21.15.27:0/2108423764 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f463006c220 0x7f463006e6e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f4640034000 tx=0x7f464002f000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:40.657 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.658+0000 7f46477fe640 1 -- 172.21.15.27:0/2108423764 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f46501f5b20 con 0x7f4650070d30 2024-09-12T03:21:40.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.835+0000 7f46551af640 1 -- 172.21.15.27:0/2108423764 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f4650160fc0 con 0x7f4650070d30 2024-09-12T03:21:40.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.835+0000 7f46477fe640 1 -- 172.21.15.27:0/2108423764 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v106) ==== 93+0+10 (secure 0 0 0) 0x7f4638058cf0 con 0x7f4650070d30 2024-09-12T03:21:40.835 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 1 2024-09-12T03:21:40.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.837+0000 7f46551af640 1 -- 172.21.15.27:0/2108423764 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f463006c220 msgr2=0x7f463006e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:40.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.837+0000 7f46551af640 1 --2- 172.21.15.27:0/2108423764 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f463006c220 0x7f463006e6e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f4640034000 tx=0x7f464002f000 comp rx=0 tx=0).stop 2024-09-12T03:21:40.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.837+0000 7f46551af640 1 -- 172.21.15.27:0/2108423764 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4650070d30 msgr2=0x7f46501f55e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:40.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.837+0000 7f46551af640 1 --2- 172.21.15.27:0/2108423764 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4650070d30 0x7f46501f55e0 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f463800c4a0 tx=0x7f463800c960 comp rx=0 tx=0).stop 2024-09-12T03:21:40.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.837+0000 7f46551af640 1 -- 172.21.15.27:0/2108423764 shutdown_connections 2024-09-12T03:21:40.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.837+0000 7f46551af640 1 --2- 172.21.15.27:0/2108423764 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f46500d0b50 0x7f46501611e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:40.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.837+0000 7f46551af640 1 --2- 172.21.15.27:0/2108423764 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f463006c220 0x7f463006e6e0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:40.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.837+0000 7f46551af640 1 --2- 172.21.15.27:0/2108423764 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4650070d30 0x7f46501f55e0 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:40.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.837+0000 7f46551af640 1 --2- 172.21.15.27:0/2108423764 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f465006e960 0x7f465015ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:40.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.837+0000 7f46551af640 1 -- 172.21.15.27:0/2108423764 >> 172.21.15.27:0/2108423764 conn(0x7f46500b7c10 msgr2=0x7f465006f3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:40.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.837+0000 7f46551af640 1 -- 172.21.15.27:0/2108423764 shutdown_connections 2024-09-12T03:21:40.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.837+0000 7f46551af640 1 -- 172.21.15.27:0/2108423764 wait complete. 2024-09-12T03:21:40.848 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_no_upgrade_v1_to_v2' 2024-09-12T03:21:40.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.953+0000 7f8b9ac23640 1 Processor -- start 2024-09-12T03:21:40.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.953+0000 7f8b9ac23640 1 -- start start 2024-09-12T03:21:40.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.954+0000 7f8b9ac23640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f8b940cf660 0x7f8b940d2780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:40.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.954+0000 7f8b9ac23640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8b9406cc40 con 0x7f8b940b1ca0 2024-09-12T03:21:40.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.954+0000 7f8b9ac23640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8b940d1ce0 con 0x7f8b9406f360 2024-09-12T03:21:40.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.954+0000 7f8b9ac23640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f8b940d22f0 con 0x7f8b940cf660 2024-09-12T03:21:40.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.954+0000 7f8b989bf640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f8b940b1ca0 0x7f8b9415f830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:34612/0 (socket says 172.21.15.27:34612) 2024-09-12T03:21:40.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.954+0000 7f8b989bf640 1 -- 172.21.15.27:0/2787179483 learned_addr learned my addr 172.21.15.27:0/2787179483 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:21:40.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.954+0000 7f8b8b7fe640 1 -- 172.21.15.27:0/2787179483 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 470487638 0 0) 0x7f8b9406cc40 con 0x7f8b940b1ca0 2024-09-12T03:21:40.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.954+0000 7f8b991c0640 1 --2- 172.21.15.27:0/2787179483 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f8b940cf660 0x7f8b940d2780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:40.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.954+0000 7f8b8b7fe640 1 -- 172.21.15.27:0/2787179483 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8b7c003520 con 0x7f8b940b1ca0 2024-09-12T03:21:40.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.954+0000 7f8b8b7fe640 1 -- 172.21.15.27:0/2787179483 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1486759050 0 0) 0x7f8b7c003520 con 0x7f8b940b1ca0 2024-09-12T03:21:40.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.955+0000 7f8b8b7fe640 1 -- 172.21.15.27:0/2787179483 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f8b940cf660 msgr2=0x7f8b940d2780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:40.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.955+0000 7f8b8b7fe640 1 --2- 172.21.15.27:0/2787179483 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f8b940cf660 0x7f8b940d2780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:40.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.955+0000 7f8b8b7fe640 1 -- 172.21.15.27:0/2787179483 >> v1:172.21.15.45:6789/0 conn(0x7f8b9406f360 legacy=0x7f8b940cef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:40.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.955+0000 7f8b8b7fe640 1 -- 172.21.15.27:0/2787179483 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b9406cee0 con 0x7f8b940b1ca0 2024-09-12T03:21:40.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.955+0000 7f8b8b7fe640 1 -- 172.21.15.27:0/2787179483 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 847755213 0 0) 0x7f8b900019f0 con 0x7f8b940b1ca0 2024-09-12T03:21:40.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.955+0000 7f8b8b7fe640 1 -- 172.21.15.27:0/2787179483 >> v1:172.21.15.27:6789/0 conn(0x7f8b940b1ca0 legacy=0x7f8b9415f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:40.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.955+0000 7f8b8b7fe640 1 --2- 172.21.15.27:0/2787179483 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8b7c0039e0 0x7f8b7c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:40.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.955+0000 7f8b8b7fe640 1 --2- 172.21.15.27:0/2787179483 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8b7c004800 0x7f8b7c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:40.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.955+0000 7f8b8b7fe640 1 --2- 172.21.15.27:0/2787179483 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f8b940cf660 0x7f8b7c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:40.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.955+0000 7f8b8b7fe640 1 -- 172.21.15.27:0/2787179483 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f8b940d22f0 con 0x7f8b7c0039e0 2024-09-12T03:21:40.959 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.955+0000 7f8b8b7fe640 1 -- 172.21.15.27:0/2787179483 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f8b7c00bad0 con 0x7f8b7c004800 2024-09-12T03:21:40.959 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.955+0000 7f8b8b7fe640 1 -- 172.21.15.27:0/2787179483 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f8b7c00bcd0 con 0x7f8b940cf660 2024-09-12T03:21:40.959 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.955+0000 7f8b989bf640 1 --2- 172.21.15.27:0/2787179483 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8b7c0039e0 0x7f8b7c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:40.959 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.955+0000 7f8b8bfff640 1 --2- 172.21.15.27:0/2787179483 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8b7c004800 0x7f8b7c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:40.959 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.955+0000 7f8b991c0640 1 --2- 172.21.15.27:0/2787179483 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f8b940cf660 0x7f8b7c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:40.959 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.955+0000 7f8b989bf640 1 -- 172.21.15.27:0/2787179483 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f8b940cf660 msgr2=0x7f8b7c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:40.959 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.955+0000 7f8b989bf640 1 --2- 172.21.15.27:0/2787179483 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f8b940cf660 0x7f8b7c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:40.959 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.955+0000 7f8b989bf640 1 -- 172.21.15.27:0/2787179483 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8b7c004800 msgr2=0x7f8b7c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:40.959 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.955+0000 7f8b989bf640 1 --2- 172.21.15.27:0/2787179483 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8b7c004800 0x7f8b7c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:40.959 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.955+0000 7f8b989bf640 1 -- 172.21.15.27:0/2787179483 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b7c003520 con 0x7f8b7c0039e0 2024-09-12T03:21:40.959 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.955+0000 7f8b8bfff640 1 --2- 172.21.15.27:0/2787179483 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8b7c004800 0x7f8b7c004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:21:40.959 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.956+0000 7f8b989bf640 1 --2- 172.21.15.27:0/2787179483 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8b7c0039e0 0x7f8b7c003e00 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f8b9000a760 tx=0x7f8b7c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f85b626a0dca8187 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:40.959 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.956+0000 7f8b8b7fe640 1 -- 172.21.15.27:0/2787179483 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b9000a3b0 con 0x7f8b7c0039e0 2024-09-12T03:21:40.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.956+0000 7f8b8b7fe640 1 -- 172.21.15.27:0/2787179483 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b9002f070 con 0x7f8b7c0039e0 2024-09-12T03:21:40.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.956+0000 7f8b8b7fe640 1 -- 172.21.15.27:0/2787179483 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b9002bbc0 con 0x7f8b7c0039e0 2024-09-12T03:21:40.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.956+0000 7f8b9ac23640 1 -- 172.21.15.27:0/2787179483 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8b7c0039e0 msgr2=0x7f8b7c003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:40.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.956+0000 7f8b9ac23640 1 --2- 172.21.15.27:0/2787179483 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8b7c0039e0 0x7f8b7c003e00 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f8b9000a760 tx=0x7f8b7c005830 comp rx=0 tx=0).stop 2024-09-12T03:21:40.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.956+0000 7f8b991c0640 1 -- 172.21.15.27:0/2787179483 reap_dead start 2024-09-12T03:21:40.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.956+0000 7f8b9ac23640 1 -- 172.21.15.27:0/2787179483 shutdown_connections 2024-09-12T03:21:40.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.956+0000 7f8b9ac23640 1 -- 172.21.15.27:0/2787179483 >> 172.21.15.27:0/2787179483 conn(0x7f8b94073800 msgr2=0x7f8b9409ba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:40.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.956+0000 7f8b9ac23640 1 -- 172.21.15.27:0/2787179483 shutdown_connections 2024-09-12T03:21:40.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.957+0000 7f8b9ac23640 1 -- 172.21.15.27:0/2787179483 wait complete. 2024-09-12T03:21:40.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.958+0000 7f8b9ac23640 1 Processor -- start 2024-09-12T03:21:40.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.958+0000 7f8b9ac23640 1 -- start start 2024-09-12T03:21:40.963 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.959+0000 7f8b9ac23640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8b7c0039e0 0x7f8b94153160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:40.963 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.959+0000 7f8b9ac23640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8b940cc230 0x7f8b940b3f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:40.963 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.959+0000 7f8b9ac23640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f8b940b4460 0x7f8b9415db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:40.963 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.959+0000 7f8b9ac23640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f8b9406d340 con 0x7f8b7c0039e0 2024-09-12T03:21:40.963 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.959+0000 7f8b9ac23640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f8b9406d1c0 con 0x7f8b940cc230 2024-09-12T03:21:40.963 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.959+0000 7f8b9ac23640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f8b940cfc40 con 0x7f8b940b4460 2024-09-12T03:21:40.963 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.959+0000 7f8b989bf640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8b7c0039e0 0x7f8b94153160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:40.963 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.959+0000 7f8b989bf640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8b7c0039e0 0x7f8b94153160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:50364/0 (socket says 172.21.15.27:50364) 2024-09-12T03:21:40.963 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.959+0000 7f8b989bf640 1 -- 172.21.15.27:0/1479888167 learned_addr learned my addr 172.21.15.27:0/1479888167 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:21:40.964 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.959+0000 7f8b8bfff640 1 --2- 172.21.15.27:0/1479888167 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8b940cc230 0x7f8b940b3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:40.964 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.959+0000 7f8b991c0640 1 --2- 172.21.15.27:0/1479888167 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f8b940b4460 0x7f8b9415db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:40.964 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.959+0000 7f8b989bf640 1 -- 172.21.15.27:0/1479888167 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f8b940b4460 msgr2=0x7f8b9415db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:40.964 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.959+0000 7f8b989bf640 1 --2- 172.21.15.27:0/1479888167 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f8b940b4460 0x7f8b9415db30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:40.964 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.959+0000 7f8b989bf640 1 -- 172.21.15.27:0/1479888167 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8b940cc230 msgr2=0x7f8b940b3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:40.964 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.959+0000 7f8b989bf640 1 --2- 172.21.15.27:0/1479888167 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8b940cc230 0x7f8b940b3f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:40.964 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.959+0000 7f8b989bf640 1 -- 172.21.15.27:0/1479888167 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b940cd790 con 0x7f8b7c0039e0 2024-09-12T03:21:40.964 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.959+0000 7f8b991c0640 1 --2- 172.21.15.27:0/1479888167 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f8b940b4460 0x7f8b9415db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:21:40.964 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.959+0000 7f8b8bfff640 1 --2- 172.21.15.27:0/1479888167 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8b940cc230 0x7f8b940b3f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:21:40.964 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.959+0000 7f8b989bf640 1 --2- 172.21.15.27:0/1479888167 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8b7c0039e0 0x7f8b94153160 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f8b90005c20 tx=0x7f8b9415edb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:40.964 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.959+0000 7f8b897fa640 1 -- 172.21.15.27:0/1479888167 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b9002bb70 con 0x7f8b7c0039e0 2024-09-12T03:21:40.964 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.959+0000 7f8b897fa640 1 -- 172.21.15.27:0/1479888167 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b9002bd10 con 0x7f8b7c0039e0 2024-09-12T03:21:40.964 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.959+0000 7f8b9ac23640 1 -- 172.21.15.27:0/1479888167 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b940cc6f0 con 0x7f8b7c0039e0 2024-09-12T03:21:40.964 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.959+0000 7f8b897fa640 1 -- 172.21.15.27:0/1479888167 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b90034670 con 0x7f8b7c0039e0 2024-09-12T03:21:40.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.959+0000 7f8b9ac23640 1 -- 172.21.15.27:0/1479888167 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b940cccb0 con 0x7f8b7c0039e0 2024-09-12T03:21:40.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.960+0000 7f8b9ac23640 1 -- 172.21.15.27:0/1479888167 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b9406f7d0 con 0x7f8b7c0039e0 2024-09-12T03:21:40.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.961+0000 7f8b897fa640 1 -- 172.21.15.27:0/1479888167 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 19) ==== 90855+0+0 (secure 0 0 0) 0x7f8b90046060 con 0x7f8b7c0039e0 2024-09-12T03:21:40.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.961+0000 7f8b897fa640 1 --2- 172.21.15.27:0/1479888167 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f8b8006c220 0x7f8b8006e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:40.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.961+0000 7f8b897fa640 1 -- 172.21.15.27:0/1479888167 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 4971+0+0 (secure 0 0 0) 0x7f8b900ab130 con 0x7f8b7c0039e0 2024-09-12T03:21:40.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.961+0000 7f8b8bfff640 1 --2- 172.21.15.27:0/1479888167 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f8b8006c220 0x7f8b8006e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:40.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.962+0000 7f8b8bfff640 1 --2- 172.21.15.27:0/1479888167 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f8b8006c220 0x7f8b8006e6e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f8b940b3e00 tx=0x7f8b84009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:40.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:40.966+0000 7f8b897fa640 1 -- 172.21.15.27:0/1479888167 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f8b940cd790 con 0x7f8b7c0039e0 2024-09-12T03:21:41.209 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:41.209+0000 7f8b9ac23640 1 -- 172.21.15.27:0/1479888167 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_no_upgrade_v1_to_v2"]} v 0) -- 0x7f8b9406ea70 con 0x7f8b7c0039e0 2024-09-12T03:21:41.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:41.221+0000 7f8b897fa640 1 -- 172.21.15.27:0/1479888167 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_no_upgrade_v1_to_v2"]}]=0 v283) ==== 151+0+0 (secure 0 0 0) 0x7f8b900798f0 con 0x7f8b7c0039e0 2024-09-12T03:21:41.225 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:41.224+0000 7f8b9ac23640 1 -- 172.21.15.27:0/1479888167 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f8b8006c220 msgr2=0x7f8b8006e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:41.225 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:41.224+0000 7f8b9ac23640 1 --2- 172.21.15.27:0/1479888167 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f8b8006c220 0x7f8b8006e6e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f8b940b3e00 tx=0x7f8b84009040 comp rx=0 tx=0).stop 2024-09-12T03:21:41.225 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:41.224+0000 7f8b9ac23640 1 -- 172.21.15.27:0/1479888167 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8b7c0039e0 msgr2=0x7f8b94153160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:41.225 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:41.224+0000 7f8b9ac23640 1 --2- 172.21.15.27:0/1479888167 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8b7c0039e0 0x7f8b94153160 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f8b90005c20 tx=0x7f8b9415edb0 comp rx=0 tx=0).stop 2024-09-12T03:21:41.225 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:41.224+0000 7f8b9ac23640 1 -- 172.21.15.27:0/1479888167 shutdown_connections 2024-09-12T03:21:41.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:41.224+0000 7f8b9ac23640 1 --2- 172.21.15.27:0/1479888167 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f8b940b4460 0x7f8b9415db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:41.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:41.224+0000 7f8b9ac23640 1 --2- 172.21.15.27:0/1479888167 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8b940cc230 0x7f8b940b3f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:41.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:41.224+0000 7f8b9ac23640 1 --2- 172.21.15.27:0/1479888167 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f8b8006c220 0x7f8b8006e6e0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:41.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:41.224+0000 7f8b9ac23640 1 --2- 172.21.15.27:0/1479888167 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8b7c0039e0 0x7f8b94153160 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:41.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:41.224+0000 7f8b9ac23640 1 -- 172.21.15.27:0/1479888167 >> 172.21.15.27:0/1479888167 conn(0x7f8b94073800 msgr2=0x7f8b94070870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:41.227 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:41.224+0000 7f8b9ac23640 1 -- 172.21.15.27:0/1479888167 shutdown_connections 2024-09-12T03:21:41.227 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:41.224+0000 7f8b9ac23640 1 -- 172.21.15.27:0/1479888167 wait complete. 2024-09-12T03:21:41.235 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-12T03:21:41.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:41.340+0000 7f01452ba640 1 Processor -- start 2024-09-12T03:21:41.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:41.341+0000 7f01452ba640 1 -- start start 2024-09-12T03:21:41.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:41.341+0000 7f01452ba640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f014015e650 0x7f01400d1d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:41.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:41.341+0000 7f01452ba640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f014006e690 con 0x7f014006db30 2024-09-12T03:21:41.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:41.341+0000 7f01452ba640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f01400d2280 con 0x7f014006cc40 2024-09-12T03:21:41.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:41.341+0000 7f01452ba640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f014006d850 con 0x7f014015e650 2024-09-12T03:21:41.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:41.342+0000 7f013f7fe640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f014006db30 0x7f014006df80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:34616/0 (socket says 172.21.15.27:34616) 2024-09-12T03:21:41.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:41.342+0000 7f013f7fe640 1 -- 172.21.15.27:0/778754382 learned_addr learned my addr 172.21.15.27:0/778754382 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:21:41.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:41.342+0000 7f013effd640 1 --2- 172.21.15.27:0/778754382 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f014015e650 0x7f01400d1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:41.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:41.342+0000 7f013dffb640 1 -- 172.21.15.27:0/778754382 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2002418076 0 0) 0x7f014006e690 con 0x7f014006db30 2024-09-12T03:21:41.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:41.342+0000 7f013dffb640 1 -- 172.21.15.27:0/778754382 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0128003520 con 0x7f014006db30 2024-09-12T03:21:41.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:41.342+0000 7f013dffb640 1 -- 172.21.15.27:0/778754382 <== mon.1 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 356749457 0 0) 0x7f01400d2280 con 0x7f014006cc40 2024-09-12T03:21:41.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:41.342+0000 7f013dffb640 1 -- 172.21.15.27:0/778754382 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f014006e690 con 0x7f014006cc40 2024-09-12T03:21:41.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:41.342+0000 7f013dffb640 1 -- 172.21.15.27:0/778754382 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3834988970 0 0) 0x7f0128003520 con 0x7f014006db30 2024-09-12T03:21:41.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:41.342+0000 7f013dffb640 1 -- 172.21.15.27:0/778754382 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f014015e650 msgr2=0x7f01400d1d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:41.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:41.342+0000 7f013dffb640 1 --2- 172.21.15.27:0/778754382 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f014015e650 0x7f01400d1d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:41.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:41.342+0000 7f013dffb640 1 -- 172.21.15.27:0/778754382 >> v1:172.21.15.45:6789/0 conn(0x7f014006cc40 legacy=0x7f014006d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:41.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:41.342+0000 7f013dffb640 1 -- 172.21.15.27:0/778754382 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01400d1260 con 0x7f014006db30 2024-09-12T03:21:41.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:41.342+0000 7f013dffb640 1 -- 172.21.15.27:0/778754382 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 847755213 0 0) 0x7f0138001a10 con 0x7f014006db30 2024-09-12T03:21:41.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:41.342+0000 7f013effd640 1 --2- 172.21.15.27:0/778754382 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f014015e650 0x7f01400d1d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:21:41.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:41.342+0000 7f013dffb640 1 -- 172.21.15.27:0/778754382 >> v1:172.21.15.27:6789/0 conn(0x7f014006db30 legacy=0x7f014006df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:41.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:41.342+0000 7f013dffb640 1 --2- 172.21.15.27:0/778754382 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0128003b70 0x7f0128003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:41.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:41.342+0000 7f013dffb640 1 --2- 172.21.15.27:0/778754382 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0128004800 0x7f0128004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:41.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:41.342+0000 7f013dffb640 1 --2- 172.21.15.27:0/778754382 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f014015e650 0x7f01280051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:41.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:41.342+0000 7f013dffb640 1 -- 172.21.15.27:0/778754382 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f014006d850 con 0x7f0128004800 2024-09-12T03:21:41.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:41.342+0000 7f013dffb640 1 -- 172.21.15.27:0/778754382 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f012800baa0 con 0x7f0128003b70 2024-09-12T03:21:41.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:41.342+0000 7f013dffb640 1 -- 172.21.15.27:0/778754382 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f012800bca0 con 0x7f014015e650 2024-09-12T03:21:41.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:41.342+0000 7f013e7fc640 1 --2- 172.21.15.27:0/778754382 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0128004800 0x7f0128004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:41.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:41.343+0000 7f013effd640 1 --2- 172.21.15.27:0/778754382 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0128003b70 0x7f0128003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:41.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:41.343+0000 7f013f7fe640 1 --2- 172.21.15.27:0/778754382 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f014015e650 0x7f01280051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:41.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:41.343+0000 7f013e7fc640 1 -- 172.21.15.27:0/778754382 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f014015e650 msgr2=0x7f01280051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:41.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:41.343+0000 7f013e7fc640 1 --2- 172.21.15.27:0/778754382 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f014015e650 0x7f01280051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:41.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:41.343+0000 7f013e7fc640 1 -- 172.21.15.27:0/778754382 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0128003b70 msgr2=0x7f0128003f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:41.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:41.343+0000 7f013e7fc640 1 --2- 172.21.15.27:0/778754382 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0128003b70 0x7f0128003f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:41.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:41.343+0000 7f013e7fc640 1 -- 172.21.15.27:0/778754382 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0128003520 con 0x7f0128004800 2024-09-12T03:21:41.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:41.343+0000 7f013f7fe640 1 --2- 172.21.15.27:0/778754382 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f014015e650 0x7f01280051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:21:41.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:41.343+0000 7f013effd640 1 --2- 172.21.15.27:0/778754382 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0128003b70 0x7f0128003f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:21:41.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:41.343+0000 7f013e7fc640 1 --2- 172.21.15.27:0/778754382 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0128004800 0x7f0128004c80 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f01340018a0 tx=0x7f0128005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ca456a77fda7a9a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:41.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:41.344+0000 7f013dffb640 1 -- 172.21.15.27:0/778754382 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01340014f0 con 0x7f0128004800 2024-09-12T03:21:41.348 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:41.344+0000 7f013dffb640 1 -- 172.21.15.27:0/778754382 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01340052d0 con 0x7f0128004800 2024-09-12T03:21:41.348 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:41.344+0000 7f013dffb640 1 -- 172.21.15.27:0/778754382 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f013402ac40 con 0x7f0128004800 2024-09-12T03:21:41.348 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:41.344+0000 7f01452ba640 1 -- 172.21.15.27:0/778754382 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0128004800 msgr2=0x7f0128004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:41.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:41.344+0000 7f01452ba640 1 --2- 172.21.15.27:0/778754382 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0128004800 0x7f0128004c80 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f01340018a0 tx=0x7f0128005830 comp rx=0 tx=0).stop 2024-09-12T03:21:41.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:41.344+0000 7f013f7fe640 1 -- 172.21.15.27:0/778754382 reap_dead start 2024-09-12T03:21:41.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:41.344+0000 7f01452ba640 1 -- 172.21.15.27:0/778754382 shutdown_connections 2024-09-12T03:21:41.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:41.344+0000 7f01452ba640 1 -- 172.21.15.27:0/778754382 >> 172.21.15.27:0/778754382 conn(0x7f014006f320 msgr2=0x7f014006f700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:41.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:41.344+0000 7f01452ba640 1 -- 172.21.15.27:0/778754382 shutdown_connections 2024-09-12T03:21:41.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:41.344+0000 7f01452ba640 1 -- 172.21.15.27:0/778754382 wait complete. 2024-09-12T03:21:41.352 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:41.346+0000 7f01452ba640 1 Processor -- start 2024-09-12T03:21:41.352 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:41.346+0000 7f01452ba640 1 -- start start 2024-09-12T03:21:41.352 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:41.346+0000 7f01452ba640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f0128004800 0x7f01401543b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:41.352 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:41.346+0000 7f01452ba640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0140159890 0x7f0140159d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:41.353 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:41.346+0000 7f01452ba640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0140157fe0 0x7f0140158460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:41.353 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:41.346+0000 7f01452ba640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f014015ebb0 con 0x7f0140157fe0 2024-09-12T03:21:41.353 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:41.346+0000 7f01452ba640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f014015ea30 con 0x7f0140159890 2024-09-12T03:21:41.353 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:41.346+0000 7f01452ba640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f01400d1d40 con 0x7f0128004800 2024-09-12T03:21:41.353 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:41.346+0000 7f013f7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0140157fe0 0x7f0140158460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:41.353 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:41.346+0000 7f013f7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0140157fe0 0x7f0140158460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:50372/0 (socket says 172.21.15.27:50372) 2024-09-12T03:21:41.353 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:41.346+0000 7f013f7fe640 1 -- 172.21.15.27:0/844151662 learned_addr learned my addr 172.21.15.27:0/844151662 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:21:41.353 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:41.347+0000 7f013effd640 1 --2- 172.21.15.27:0/844151662 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f0128004800 0x7f01401543b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:41.353 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:41.347+0000 7f013e7fc640 1 --2- 172.21.15.27:0/844151662 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0140159890 0x7f0140159d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:41.353 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:41.347+0000 7f013f7fe640 1 -- 172.21.15.27:0/844151662 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f0128004800 msgr2=0x7f01401543b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:41.353 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:41.347+0000 7f013f7fe640 1 --2- 172.21.15.27:0/844151662 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f0128004800 0x7f01401543b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:41.353 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:41.347+0000 7f013f7fe640 1 -- 172.21.15.27:0/844151662 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0140159890 msgr2=0x7f0140159d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:41.353 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:41.347+0000 7f013f7fe640 1 --2- 172.21.15.27:0/844151662 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0140159890 0x7f0140159d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:41.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:41.347+0000 7f013f7fe640 1 -- 172.21.15.27:0/844151662 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0140158bf0 con 0x7f0140157fe0 2024-09-12T03:21:41.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:41.347+0000 7f013f7fe640 1 --2- 172.21.15.27:0/844151662 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0140157fe0 0x7f0140158460 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f0138005fd0 tx=0x7f0138007b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:41.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:41.347+0000 7f01277fe640 1 -- 172.21.15.27:0/844151662 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f013800e9c0 con 0x7f0140157fe0 2024-09-12T03:21:41.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:41.347+0000 7f01277fe640 1 -- 172.21.15.27:0/844151662 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0138018070 con 0x7f0140157fe0 2024-09-12T03:21:41.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:41.347+0000 7f01452ba640 1 -- 172.21.15.27:0/844151662 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0140158ed0 con 0x7f0140157fe0 2024-09-12T03:21:41.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:41.347+0000 7f01277fe640 1 -- 172.21.15.27:0/844151662 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f013800ede0 con 0x7f0140157fe0 2024-09-12T03:21:41.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:41.347+0000 7f01452ba640 1 -- 172.21.15.27:0/844151662 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0140159490 con 0x7f0140157fe0 2024-09-12T03:21:41.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:41.348+0000 7f01452ba640 1 -- 172.21.15.27:0/844151662 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01400d2150 con 0x7f0140157fe0 2024-09-12T03:21:41.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:41.348+0000 7f01277fe640 1 -- 172.21.15.27:0/844151662 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 19) ==== 90855+0+0 (secure 0 0 0) 0x7f013801d460 con 0x7f0140157fe0 2024-09-12T03:21:41.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:41.349+0000 7f01277fe640 1 --2- 172.21.15.27:0/844151662 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f012c06c220 0x7f012c06e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:41.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:41.349+0000 7f01277fe640 1 -- 172.21.15.27:0/844151662 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 4971+0+0 (secure 0 0 0) 0x7f013808b2d0 con 0x7f0140157fe0 2024-09-12T03:21:41.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:41.349+0000 7f013effd640 1 --2- 172.21.15.27:0/844151662 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f012c06c220 0x7f012c06e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:41.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:41.349+0000 7f013effd640 1 --2- 172.21.15.27:0/844151662 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f012c06c220 0x7f012c06e6e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f0130009560 tx=0x7f0130008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:41.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:41.353+0000 7f01277fe640 1 -- 172.21.15.27:0/844151662 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f0138059b10 con 0x7f0140157fe0 2024-09-12T03:21:41.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:41.534+0000 7f01452ba640 1 -- 172.21.15.27:0/844151662 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f01400b23c0 con 0x7f0140157fe0 2024-09-12T03:21:41.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:41.537+0000 7f01277fe640 1 -- 172.21.15.27:0/844151662 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v22)=0 v22) ==== 122+0+0 (secure 0 0 0) 0x7f0138059410 con 0x7f0140157fe0 2024-09-12T03:21:41.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:41.539+0000 7f01452ba640 1 -- 172.21.15.27:0/844151662 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f012c06c220 msgr2=0x7f012c06e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:41.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:41.539+0000 7f01452ba640 1 --2- 172.21.15.27:0/844151662 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f012c06c220 0x7f012c06e6e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f0130009560 tx=0x7f0130008040 comp rx=0 tx=0).stop 2024-09-12T03:21:41.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:41.539+0000 7f01452ba640 1 -- 172.21.15.27:0/844151662 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0140157fe0 msgr2=0x7f0140158460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:41.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:41.539+0000 7f01452ba640 1 --2- 172.21.15.27:0/844151662 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0140157fe0 0x7f0140158460 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f0138005fd0 tx=0x7f0138007b10 comp rx=0 tx=0).stop 2024-09-12T03:21:41.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:41.539+0000 7f01452ba640 1 -- 172.21.15.27:0/844151662 shutdown_connections 2024-09-12T03:21:41.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:41.539+0000 7f01452ba640 1 --2- 172.21.15.27:0/844151662 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f012c06c220 0x7f012c06e6e0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:41.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:41.539+0000 7f01452ba640 1 --2- 172.21.15.27:0/844151662 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0140157fe0 0x7f0140158460 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:41.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:41.539+0000 7f01452ba640 1 --2- 172.21.15.27:0/844151662 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0140159890 0x7f0140159d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:41.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:41.539+0000 7f01452ba640 1 --2- 172.21.15.27:0/844151662 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f0128004800 0x7f01401543b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:41.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:41.539+0000 7f01452ba640 1 -- 172.21.15.27:0/844151662 >> 172.21.15.27:0/844151662 conn(0x7f014006f320 msgr2=0x7f01400736c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:41.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:41.540+0000 7f01452ba640 1 -- 172.21.15.27:0/844151662 shutdown_connections 2024-09-12T03:21:41.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:41.540+0000 7f01452ba640 1 -- 172.21.15.27:0/844151662 wait complete. 2024-09-12T03:21:41.550 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-12T03:21:41.551 DEBUG:teuthology.orchestra.run.smithi027:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-12T03:21:41.592 DEBUG:teuthology.orchestra.run.smithi045:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-12T03:21:41.636 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-12T03:21:41.637 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-12T03:21:41.637 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T03:21:41.653 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056004,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497432k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16324 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248716k,nr_inodes=812179,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497432k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-12T03:21:41.653 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-12T03:21:41.653 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T03:21:41.695 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056795,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16602 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249348k,nr_inodes=812337,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-12T03:21:41.695 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-12T03:21:41.695 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T03:21:41.711 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056004,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497432k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16324 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248716k,nr_inodes=812179,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497432k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-12T03:21:41.711 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-12T03:21:41.711 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T03:21:41.753 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056795,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16602 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249348k,nr_inodes=812337,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-12T03:21:41.753 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-12T03:21:41.753 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-12T03:21:41.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:41.855+0000 7fc418923640 1 Processor -- start 2024-09-12T03:21:41.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:41.855+0000 7fc418923640 1 -- start start 2024-09-12T03:21:41.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:41.856+0000 7fc418923640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc41415eb90 0x7fc4140d2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:41.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:41.856+0000 7fc418923640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc4140d3760 con 0x7fc4140d5200 2024-09-12T03:21:41.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:41.856+0000 7fc418923640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc4140d3a40 con 0x7fc4140d1720 2024-09-12T03:21:41.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:41.856+0000 7fc418923640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fc4140b7d20 con 0x7fc41415eb90 2024-09-12T03:21:41.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:41.856+0000 7fc411d9b640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fc4140d5200 0x7fc4140d1010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:34620/0 (socket says 172.21.15.27:34620) 2024-09-12T03:21:41.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:41.856+0000 7fc411d9b640 1 -- 172.21.15.27:0/1492538640 learned_addr learned my addr 172.21.15.27:0/1492538640 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:21:41.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:41.857+0000 7fc41259c640 1 --2- 172.21.15.27:0/1492538640 >> v2:172.21.15.45:3300/0 conn(0x7fc41415eb90 0x7fc4140d2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:41.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:41.857+0000 7fc41159a640 1 -- 172.21.15.27:0/1492538640 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1068954027 0 0) 0x7fc4140d3760 con 0x7fc4140d5200 2024-09-12T03:21:41.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:41.857+0000 7fc41159a640 1 -- 172.21.15.27:0/1492538640 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc400003520 con 0x7fc4140d5200 2024-09-12T03:21:41.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:41.857+0000 7fc41159a640 1 -- 172.21.15.27:0/1492538640 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1233054408 0 0) 0x7fc400003520 con 0x7fc4140d5200 2024-09-12T03:21:41.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:41.857+0000 7fc41159a640 1 -- 172.21.15.27:0/1492538640 >> v2:172.21.15.45:3300/0 conn(0x7fc41415eb90 msgr2=0x7fc4140d2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:41.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:41.857+0000 7fc41159a640 1 --2- 172.21.15.27:0/1492538640 >> v2:172.21.15.45:3300/0 conn(0x7fc41415eb90 0x7fc4140d2df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:41.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:41.857+0000 7fc41159a640 1 -- 172.21.15.27:0/1492538640 >> v1:172.21.15.45:6789/0 conn(0x7fc4140d1720 legacy=0x7fc4140b8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:41.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:41.857+0000 7fc41159a640 1 -- 172.21.15.27:0/1492538640 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4140d3d20 con 0x7fc4140d5200 2024-09-12T03:21:41.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:41.857+0000 7fc41159a640 1 -- 172.21.15.27:0/1492538640 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 847755213 0 0) 0x7fc3fc001670 con 0x7fc4140d5200 2024-09-12T03:21:41.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:41.857+0000 7fc41159a640 1 -- 172.21.15.27:0/1492538640 >> v1:172.21.15.27:6789/0 conn(0x7fc4140d5200 legacy=0x7fc4140d1010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:41.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:41.857+0000 7fc41159a640 1 --2- 172.21.15.27:0/1492538640 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc4000039b0 0x7fc400003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:41.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:41.857+0000 7fc41159a640 1 --2- 172.21.15.27:0/1492538640 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc400004800 0x7fc400004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:41.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:41.857+0000 7fc41159a640 1 --2- 172.21.15.27:0/1492538640 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fc4000051c0 0x7fc40000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:41.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:41.857+0000 7fc41159a640 1 -- 172.21.15.27:0/1492538640 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fc40000c030 con 0x7fc4000039b0 2024-09-12T03:21:41.860 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:41.857+0000 7fc41159a640 1 -- 172.21.15.27:0/1492538640 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc40000c1e0 con 0x7fc400004800 2024-09-12T03:21:41.860 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:41.857+0000 7fc41159a640 1 -- 172.21.15.27:0/1492538640 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fc40000c3e0 con 0x7fc4000051c0 2024-09-12T03:21:41.860 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:41.857+0000 7fc41259c640 1 --2- 172.21.15.27:0/1492538640 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc4000039b0 0x7fc400003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:41.860 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:41.857+0000 7fc411d9b640 1 --2- 172.21.15.27:0/1492538640 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc400004800 0x7fc400004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:41.860 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:41.857+0000 7fc412d9d640 1 --2- 172.21.15.27:0/1492538640 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fc4000051c0 0x7fc40000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:41.860 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:41.858+0000 7fc41259c640 1 -- 172.21.15.27:0/1492538640 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fc4000051c0 msgr2=0x7fc40000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:41.860 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:41.858+0000 7fc41259c640 1 --2- 172.21.15.27:0/1492538640 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fc4000051c0 0x7fc40000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:41.860 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:41.858+0000 7fc41259c640 1 -- 172.21.15.27:0/1492538640 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc400004800 msgr2=0x7fc400004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:41.860 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:41.858+0000 7fc41259c640 1 --2- 172.21.15.27:0/1492538640 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc400004800 0x7fc400004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:41.860 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:41.858+0000 7fc41259c640 1 -- 172.21.15.27:0/1492538640 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc400003520 con 0x7fc4000039b0 2024-09-12T03:21:41.860 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:41.858+0000 7fc412d9d640 1 -- 172.21.15.27:0/1492538640 reap_dead start 2024-09-12T03:21:41.860 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:41.858+0000 7fc41259c640 1 --2- 172.21.15.27:0/1492538640 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc4000039b0 0x7fc400003dd0 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7fc40c009ac0 tx=0x7fc400005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28d46f920280a362 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:41.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:41.858+0000 7fc41159a640 1 -- 172.21.15.27:0/1492538640 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc40c036070 con 0x7fc4000039b0 2024-09-12T03:21:41.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:41.858+0000 7fc41159a640 1 -- 172.21.15.27:0/1492538640 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc40c031400 con 0x7fc4000039b0 2024-09-12T03:21:41.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:41.858+0000 7fc41159a640 1 -- 172.21.15.27:0/1492538640 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc40c009690 con 0x7fc4000039b0 2024-09-12T03:21:41.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:41.859+0000 7fc418923640 1 -- 172.21.15.27:0/1492538640 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc4000039b0 msgr2=0x7fc400003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:41.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:41.859+0000 7fc418923640 1 --2- 172.21.15.27:0/1492538640 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc4000039b0 0x7fc400003dd0 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7fc40c009ac0 tx=0x7fc400005830 comp rx=0 tx=0).stop 2024-09-12T03:21:41.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:41.859+0000 7fc418923640 1 -- 172.21.15.27:0/1492538640 shutdown_connections 2024-09-12T03:21:41.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:41.859+0000 7fc418923640 1 --2- 172.21.15.27:0/1492538640 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc4000039b0 0x7fc400003dd0 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:41.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:41.859+0000 7fc418923640 1 -- 172.21.15.27:0/1492538640 >> 172.21.15.27:0/1492538640 conn(0x7fc41406dce0 msgr2=0x7fc41406e0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:41.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:41.859+0000 7fc418923640 1 -- 172.21.15.27:0/1492538640 shutdown_connections 2024-09-12T03:21:41.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:41.859+0000 7fc418923640 1 -- 172.21.15.27:0/1492538640 wait complete. 2024-09-12T03:21:41.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:41.860+0000 7fc418923640 1 Processor -- start 2024-09-12T03:21:41.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:41.860+0000 7fc418923640 1 -- start start 2024-09-12T03:21:41.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:41.861+0000 7fc418923640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc4000039b0 0x7fc4140c99d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:41.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:41.861+0000 7fc418923640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc4141596a0 0x7fc4140cae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:41.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:41.861+0000 7fc418923640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fc414157d30 0x7fc414159c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:41.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:41.861+0000 7fc418923640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fc41415ef70 con 0x7fc4000039b0 2024-09-12T03:21:41.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:41.861+0000 7fc418923640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc4140d42e0 con 0x7fc4141596a0 2024-09-12T03:21:41.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:41.861+0000 7fc418923640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fc41415f0f0 con 0x7fc414157d30 2024-09-12T03:21:41.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:41.861+0000 7fc41259c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc4000039b0 0x7fc4140c99d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:41.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:41.861+0000 7fc41259c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc4000039b0 0x7fc4140c99d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:50392/0 (socket says 172.21.15.27:50392) 2024-09-12T03:21:41.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:41.861+0000 7fc41259c640 1 -- 172.21.15.27:0/2791471005 learned_addr learned my addr 172.21.15.27:0/2791471005 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:21:41.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:41.861+0000 7fc412d9d640 1 --2- 172.21.15.27:0/2791471005 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fc414157d30 0x7fc414159c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:41.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:41.861+0000 7fc411d9b640 1 --2- 172.21.15.27:0/2791471005 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc4141596a0 0x7fc4140cae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:41.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:41.862+0000 7fc41259c640 1 -- 172.21.15.27:0/2791471005 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fc414157d30 msgr2=0x7fc414159c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:41.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:41.862+0000 7fc41259c640 1 --2- 172.21.15.27:0/2791471005 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fc414157d30 0x7fc414159c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:41.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:41.862+0000 7fc41259c640 1 -- 172.21.15.27:0/2791471005 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc4141596a0 msgr2=0x7fc4140cae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:41.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:41.862+0000 7fc41259c640 1 --2- 172.21.15.27:0/2791471005 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc4141596a0 0x7fc4140cae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:41.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:41.862+0000 7fc41259c640 1 -- 172.21.15.27:0/2791471005 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4140caa10 con 0x7fc4000039b0 2024-09-12T03:21:41.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:41.862+0000 7fc41259c640 1 --2- 172.21.15.27:0/2791471005 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc4000039b0 0x7fc4140c99d0 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7fc40c009be0 tx=0x7fc40c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:41.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:41.862+0000 7fc40affd640 1 -- 172.21.15.27:0/2791471005 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc40c03f020 con 0x7fc4000039b0 2024-09-12T03:21:41.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:41.862+0000 7fc40affd640 1 -- 172.21.15.27:0/2791471005 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc40c037040 con 0x7fc4000039b0 2024-09-12T03:21:41.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:41.862+0000 7fc40affd640 1 -- 172.21.15.27:0/2791471005 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc40c036070 con 0x7fc4000039b0 2024-09-12T03:21:41.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:41.862+0000 7fc418923640 1 -- 172.21.15.27:0/2791471005 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4140c8ef0 con 0x7fc4000039b0 2024-09-12T03:21:41.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:41.862+0000 7fc418923640 1 -- 172.21.15.27:0/2791471005 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4140ca6c0 con 0x7fc4000039b0 2024-09-12T03:21:41.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:41.863+0000 7fc418923640 1 -- 172.21.15.27:0/2791471005 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4140c8d00 con 0x7fc4000039b0 2024-09-12T03:21:41.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:41.864+0000 7fc40affd640 1 -- 172.21.15.27:0/2791471005 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 19) ==== 90855+0+0 (secure 0 0 0) 0x7fc40c042020 con 0x7fc4000039b0 2024-09-12T03:21:41.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:41.864+0000 7fc40affd640 1 --2- 172.21.15.27:0/2791471005 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fc3f406c250 0x7fc3f406e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:41.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:41.864+0000 7fc40affd640 1 -- 172.21.15.27:0/2791471005 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 4971+0+0 (secure 0 0 0) 0x7fc40c0ab0d0 con 0x7fc4000039b0 2024-09-12T03:21:41.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:41.864+0000 7fc411d9b640 1 --2- 172.21.15.27:0/2791471005 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fc3f406c250 0x7fc3f406e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:41.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:41.865+0000 7fc411d9b640 1 --2- 172.21.15.27:0/2791471005 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fc3f406c250 0x7fc3f406e710 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fc4140d0060 tx=0x7fc3fc0078f0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:41.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:41.868+0000 7fc40affd640 1 -- 172.21.15.27:0/2791471005 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fc40c049030 con 0x7fc4000039b0 2024-09-12T03:21:42.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:42.083+0000 7fc418923640 1 -- 172.21.15.27:0/2791471005 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc414162050 con 0x7fc4000039b0 2024-09-12T03:21:42.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:42.084+0000 7fc40affd640 1 -- 172.21.15.27:0/2791471005 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 61 v61) ==== 94+0+3908 (secure 0 0 0) 0x7fc40c079130 con 0x7fc4000039b0 2024-09-12T03:21:42.084 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:21:42.085 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":61,"btime":"2024-09-12T03:21:37:028481+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5885,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/2343815091","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2343815091},{"type":"v1","addr":"172.21.15.45:6834","nonce":2343815091}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":55},{"gid":6376,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/512483568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":512483568},{"type":"v1","addr":"172.21.15.45:6835","nonce":512483568}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":55},{"gid":6825,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6834/1888895227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":1888895227},{"type":"v1","addr":"172.21.15.27:6834","nonce":1888895227}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":54},{"gid":7218,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/4067351360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":4067351360},{"type":"v1","addr":"172.21.15.27:6835","nonce":4067351360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":61}],"filesystems":[]} 2024-09-12T03:21:42.085 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 61 2024-09-12T03:21:42.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:42.086+0000 7fc418923640 1 -- 172.21.15.27:0/2791471005 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fc3f406c250 msgr2=0x7fc3f406e710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:42.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:42.086+0000 7fc418923640 1 --2- 172.21.15.27:0/2791471005 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fc3f406c250 0x7fc3f406e710 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fc4140d0060 tx=0x7fc3fc0078f0 comp rx=0 tx=0).stop 2024-09-12T03:21:42.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:42.086+0000 7fc418923640 1 -- 172.21.15.27:0/2791471005 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc4000039b0 msgr2=0x7fc4140c99d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:42.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:42.086+0000 7fc418923640 1 --2- 172.21.15.27:0/2791471005 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc4000039b0 0x7fc4140c99d0 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7fc40c009be0 tx=0x7fc40c004730 comp rx=0 tx=0).stop 2024-09-12T03:21:42.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:42.086+0000 7fc418923640 1 -- 172.21.15.27:0/2791471005 shutdown_connections 2024-09-12T03:21:42.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:42.086+0000 7fc418923640 1 --2- 172.21.15.27:0/2791471005 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fc414157d30 0x7fc414159c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:42.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:42.086+0000 7fc418923640 1 --2- 172.21.15.27:0/2791471005 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc4141596a0 0x7fc4140cae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:42.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:42.086+0000 7fc418923640 1 --2- 172.21.15.27:0/2791471005 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fc3f406c250 0x7fc3f406e710 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:42.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:42.086+0000 7fc418923640 1 --2- 172.21.15.27:0/2791471005 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc4000039b0 0x7fc4140c99d0 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:42.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:42.086+0000 7fc418923640 1 -- 172.21.15.27:0/2791471005 >> 172.21.15.27:0/2791471005 conn(0x7fc41406dce0 msgr2=0x7fc4140cde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:42.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:42.086+0000 7fc418923640 1 -- 172.21.15.27:0/2791471005 shutdown_connections 2024-09-12T03:21:42.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:42.086+0000 7fc418923640 1 -- 172.21.15.27:0/2791471005 wait complete. 2024-09-12T03:21:42.097 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-12T03:21:42.097 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-12T03:21:42.097 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-12T03:21:42.097 INFO:teuthology.orchestra.run:waiting for 300 2024-09-12T03:21:42.097 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-12T03:21:42.098 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-12T03:21:42.098 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-12T03:21:42.098 INFO:teuthology.orchestra.run:waiting for 300 2024-09-12T03:21:42.098 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-12T03:21:42.098 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-12T03:21:42.098 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-12T03:21:42.098 INFO:teuthology.orchestra.run:waiting for 300 2024-09-12T03:21:42.099 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-12T03:21:42.099 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-12T03:21:42.099 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-12T03:21:42.099 INFO:teuthology.orchestra.run:waiting for 300 2024-09-12T03:21:42.109 INFO:tasks.ceph.mds.a:Stopped 2024-09-12T03:21:42.109 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-12T03:21:42.110 INFO:tasks.ceph.mds.c:Stopped 2024-09-12T03:21:42.110 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-12T03:21:42.111 INFO:tasks.ceph.mds.b:Stopped 2024-09-12T03:21:42.111 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-12T03:21:42.112 INFO:tasks.ceph.mds.d:Stopped 2024-09-12T03:21:42.112 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-12T03:21:42.151 INFO:tasks.ceph.mds.a:Started 2024-09-12T03:21:42.152 DEBUG:teuthology.parallel:result is None 2024-09-12T03:21:42.152 INFO:tasks.ceph.mds.c:Started 2024-09-12T03:21:42.152 DEBUG:teuthology.parallel:result is None 2024-09-12T03:21:42.154 INFO:tasks.ceph.mds.b:Started 2024-09-12T03:21:42.154 DEBUG:teuthology.parallel:result is None 2024-09-12T03:21:42.155 INFO:tasks.ceph.mds.d:Started 2024-09-12T03:21:42.155 DEBUG:teuthology.parallel:result is None 2024-09-12T03:21:42.156 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-12T03:21:42.230 INFO:tasks.ceph.mds.c.smithi027.stdout:starting mds.c at 2024-09-12T03:21:42.230 INFO:tasks.ceph.mds.a.smithi027.stdout:starting mds.a at 2024-09-12T03:21:42.240 INFO:tasks.ceph.mds.d.smithi045.stdout:starting mds.d at 2024-09-12T03:21:42.240 INFO:tasks.ceph.mds.b.smithi045.stdout:starting mds.b at 2024-09-12T03:21:42.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:42.263+0000 7fbc91c5c640 1 Processor -- start 2024-09-12T03:21:42.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:42.263+0000 7fbc91c5c640 1 -- start start 2024-09-12T03:21:42.265 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:42.264+0000 7fbc91c5c640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fbc8c0b91a0 0x7fbc8c169880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:42.265 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:42.264+0000 7fbc91c5c640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbc8c0d3760 con 0x7fbc8c0d5200 2024-09-12T03:21:42.265 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:42.264+0000 7fbc91c5c640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbc8c0d3a40 con 0x7fbc8c15eb90 2024-09-12T03:21:42.265 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:42.264+0000 7fbc91c5c640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7fbc8c0b7d20 con 0x7fbc8c0b91a0 2024-09-12T03:21:42.265 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:42.264+0000 7fbc82ffd640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fbc8c0d5200 0x7fbc8c0d1010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:34648/0 (socket says 172.21.15.27:34648) 2024-09-12T03:21:42.265 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:42.264+0000 7fbc82ffd640 1 -- 172.21.15.27:0/21271234 learned_addr learned my addr 172.21.15.27:0/21271234 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:21:42.265 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:42.264+0000 7fbc8bfff640 1 --2- 172.21.15.27:0/21271234 >> v2:172.21.15.27:3300/0 conn(0x7fbc8c0b91a0 0x7fbc8c169880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:42.266 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:42.264+0000 7fbc8bfff640 1 -- 172.21.15.27:0/21271234 >> v1:172.21.15.45:6789/0 conn(0x7fbc8c15eb90 legacy=0x7fbc8c0d2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:42.266 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:42.264+0000 7fbc8bfff640 1 -- 172.21.15.27:0/21271234 >> v1:172.21.15.27:6789/0 conn(0x7fbc8c0d5200 legacy=0x7fbc8c0d1010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:42.266 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:42.264+0000 7fbc8bfff640 1 -- 172.21.15.27:0/21271234 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc8c0d3d20 con 0x7fbc8c0b91a0 2024-09-12T03:21:42.266 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:42.264+0000 7fbc8affd640 1 -- 172.21.15.27:0/21271234 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1848068085 0 0) 0x7fbc8c0d3760 con 0x7fbc8c0d5200 2024-09-12T03:21:42.266 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:42.265+0000 7fbc8bfff640 1 --2- 172.21.15.27:0/21271234 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbc8c0b91a0 0x7fbc8c169880 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7fbc84009d90 tx=0x7fbc8402a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cc07441eb201248 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:42.266 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:42.265+0000 7fbc8affd640 1 -- 172.21.15.27:0/21271234 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc84007740 con 0x7fbc8c0b91a0 2024-09-12T03:21:42.266 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:42.265+0000 7fbc8affd640 1 -- 172.21.15.27:0/21271234 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc840078e0 con 0x7fbc8c0b91a0 2024-09-12T03:21:42.266 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:42.265+0000 7fbc8affd640 1 -- 172.21.15.27:0/21271234 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc84007cb0 con 0x7fbc8c0b91a0 2024-09-12T03:21:42.267 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:42.266+0000 7fbc91c5c640 1 -- 172.21.15.27:0/21271234 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbc8c0b91a0 msgr2=0x7fbc8c169880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:42.267 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:42.266+0000 7fbc91c5c640 1 --2- 172.21.15.27:0/21271234 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbc8c0b91a0 0x7fbc8c169880 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7fbc84009d90 tx=0x7fbc8402a6f0 comp rx=0 tx=0).stop 2024-09-12T03:21:42.267 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:42.266+0000 7fbc91c5c640 1 -- 172.21.15.27:0/21271234 shutdown_connections 2024-09-12T03:21:42.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:42.266+0000 7fbc91c5c640 1 --2- 172.21.15.27:0/21271234 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbc8c0b91a0 0x7fbc8c169880 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:42.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:42.266+0000 7fbc91c5c640 1 -- 172.21.15.27:0/21271234 >> 172.21.15.27:0/21271234 conn(0x7fbc8c06dce0 msgr2=0x7fbc8c06e0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:42.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:42.266+0000 7fbc91c5c640 1 -- 172.21.15.27:0/21271234 shutdown_connections 2024-09-12T03:21:42.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:42.266+0000 7fbc91c5c640 1 -- 172.21.15.27:0/21271234 wait complete. 2024-09-12T03:21:42.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:42.267+0000 7fbc91c5c640 1 Processor -- start 2024-09-12T03:21:42.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:42.267+0000 7fbc91c5c640 1 -- start start 2024-09-12T03:21:42.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:42.268+0000 7fbc91c5c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbc8c0b91a0 0x7fbc8c14ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:42.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:42.268+0000 7fbc91c5c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbc8c0d5200 0x7fbc8c156a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:42.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:42.268+0000 7fbc91c5c640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fbc8c15eb90 0x7fbc8c156f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:42.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:42.268+0000 7fbc91c5c640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fbc8c0d3500 con 0x7fbc8c0b91a0 2024-09-12T03:21:42.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:42.268+0000 7fbc91c5c640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fbc8c0d42e0 con 0x7fbc8c0d5200 2024-09-12T03:21:42.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:42.268+0000 7fbc91c5c640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fbc8c15ef70 con 0x7fbc8c15eb90 2024-09-12T03:21:42.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:42.268+0000 7fbc8b7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbc8c0b91a0 0x7fbc8c14ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:42.272 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:42.268+0000 7fbc8b7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbc8c0b91a0 0x7fbc8c14ec30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:50424/0 (socket says 172.21.15.27:50424) 2024-09-12T03:21:42.272 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:42.268+0000 7fbc8b7fe640 1 -- 172.21.15.27:0/530118013 learned_addr learned my addr 172.21.15.27:0/530118013 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:21:42.272 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:42.268+0000 7fbc82ffd640 1 --2- 172.21.15.27:0/530118013 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbc8c0d5200 0x7fbc8c156a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:42.272 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:42.268+0000 7fbc8bfff640 1 --2- 172.21.15.27:0/530118013 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fbc8c15eb90 0x7fbc8c156f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:42.272 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:42.268+0000 7fbc8b7fe640 1 -- 172.21.15.27:0/530118013 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fbc8c15eb90 msgr2=0x7fbc8c156f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:42.272 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:42.268+0000 7fbc8b7fe640 1 --2- 172.21.15.27:0/530118013 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fbc8c15eb90 0x7fbc8c156f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:42.272 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:42.268+0000 7fbc8b7fe640 1 -- 172.21.15.27:0/530118013 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbc8c0d5200 msgr2=0x7fbc8c156a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:42.272 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:42.268+0000 7fbc8b7fe640 1 --2- 172.21.15.27:0/530118013 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbc8c0d5200 0x7fbc8c156a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:42.272 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:42.268+0000 7fbc8b7fe640 1 -- 172.21.15.27:0/530118013 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc8c0cad00 con 0x7fbc8c0b91a0 2024-09-12T03:21:42.272 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:42.268+0000 7fbc82ffd640 1 --2- 172.21.15.27:0/530118013 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbc8c0d5200 0x7fbc8c156a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:21:42.272 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:42.269+0000 7fbc8b7fe640 1 --2- 172.21.15.27:0/530118013 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbc8c0b91a0 0x7fbc8c14ec30 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7fbc7c00c400 tx=0x7fbc7c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:42.272 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:42.269+0000 7fbc88ff9640 1 -- 172.21.15.27:0/530118013 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc7c017070 con 0x7fbc8c0b91a0 2024-09-12T03:21:42.272 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:42.269+0000 7fbc88ff9640 1 -- 172.21.15.27:0/530118013 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc7c0052d0 con 0x7fbc8c0b91a0 2024-09-12T03:21:42.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:42.269+0000 7fbc88ff9640 1 -- 172.21.15.27:0/530118013 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc7c003830 con 0x7fbc8c0b91a0 2024-09-12T03:21:42.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:42.269+0000 7fbc91c5c640 1 -- 172.21.15.27:0/530118013 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc8c0ca0f0 con 0x7fbc8c0b91a0 2024-09-12T03:21:42.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:42.269+0000 7fbc91c5c640 1 -- 172.21.15.27:0/530118013 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc8c0ca6b0 con 0x7fbc8c0b91a0 2024-09-12T03:21:42.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:42.270+0000 7fbc91c5c640 1 -- 172.21.15.27:0/530118013 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc8c0d2df0 con 0x7fbc8c0b91a0 2024-09-12T03:21:42.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:42.270+0000 7fbc88ff9640 1 -- 172.21.15.27:0/530118013 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 19) ==== 90855+0+0 (secure 0 0 0) 0x7fbc7c01a070 con 0x7fbc8c0b91a0 2024-09-12T03:21:42.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:42.271+0000 7fbc88ff9640 1 --2- 172.21.15.27:0/530118013 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fbc6c06c220 0x7fbc6c06e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:42.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:42.271+0000 7fbc88ff9640 1 -- 172.21.15.27:0/530118013 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 4971+0+0 (secure 0 0 0) 0x7fbc7c08a6f0 con 0x7fbc8c0b91a0 2024-09-12T03:21:42.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:42.271+0000 7fbc82ffd640 1 --2- 172.21.15.27:0/530118013 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fbc6c06c220 0x7fbc6c06e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:42.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:42.271+0000 7fbc82ffd640 1 --2- 172.21.15.27:0/530118013 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fbc6c06c220 0x7fbc6c06e6e0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fbc8c152fe0 tx=0x7fbc74007c50 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:42.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:42.275+0000 7fbc88ff9640 1 -- 172.21.15.27:0/530118013 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fbc7c058d10 con 0x7fbc8c0b91a0 2024-09-12T03:21:42.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:42.461+0000 7fbc91c5c640 1 -- 172.21.15.27:0/530118013 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7fbc8c0d32e0 con 0x7fbc8c0b91a0 2024-09-12T03:21:43.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:43.234+0000 7fbc88ff9640 1 -- 172.21.15.27:0/530118013 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v108) ==== 97+0+0 (secure 0 0 0) 0x7fbc7c058610 con 0x7fbc8c0b91a0 2024-09-12T03:21:43.234 INFO:teuthology.orchestra.run.smithi027.stderr: removed all blocklist entries 2024-09-12T03:21:43.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:43.238+0000 7fbc91c5c640 1 -- 172.21.15.27:0/530118013 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fbc6c06c220 msgr2=0x7fbc6c06e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:43.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:43.238+0000 7fbc91c5c640 1 --2- 172.21.15.27:0/530118013 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fbc6c06c220 0x7fbc6c06e6e0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fbc8c152fe0 tx=0x7fbc74007c50 comp rx=0 tx=0).stop 2024-09-12T03:21:43.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:43.238+0000 7fbc91c5c640 1 -- 172.21.15.27:0/530118013 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbc8c0b91a0 msgr2=0x7fbc8c14ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:43.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:43.238+0000 7fbc91c5c640 1 --2- 172.21.15.27:0/530118013 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbc8c0b91a0 0x7fbc8c14ec30 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7fbc7c00c400 tx=0x7fbc7c00c8c0 comp rx=0 tx=0).stop 2024-09-12T03:21:43.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:43.238+0000 7fbc91c5c640 1 -- 172.21.15.27:0/530118013 shutdown_connections 2024-09-12T03:21:43.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:43.238+0000 7fbc91c5c640 1 --2- 172.21.15.27:0/530118013 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fbc8c15eb90 0x7fbc8c156f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:43.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:43.238+0000 7fbc91c5c640 1 --2- 172.21.15.27:0/530118013 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbc8c0d5200 0x7fbc8c156a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:43.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:43.238+0000 7fbc91c5c640 1 --2- 172.21.15.27:0/530118013 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fbc6c06c220 0x7fbc6c06e6e0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:43.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:43.238+0000 7fbc91c5c640 1 --2- 172.21.15.27:0/530118013 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbc8c0b91a0 0x7fbc8c14ec30 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:43.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:43.238+0000 7fbc91c5c640 1 -- 172.21.15.27:0/530118013 >> 172.21.15.27:0/530118013 conn(0x7fbc8c06dce0 msgr2=0x7fbc8c0d1010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:43.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:43.238+0000 7fbc91c5c640 1 -- 172.21.15.27:0/530118013 shutdown_connections 2024-09-12T03:21:43.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:43.238+0000 7fbc91c5c640 1 -- 172.21.15.27:0/530118013 wait complete. 2024-09-12T03:21:43.253 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-12T03:21:43.360 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:43.360+0000 7f475cc8d640 1 Processor -- start 2024-09-12T03:21:43.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:43.360+0000 7f475cc8d640 1 -- start start 2024-09-12T03:21:43.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:43.361+0000 7f475cc8d640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f475815eb90 0x7f47580d2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:43.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:43.361+0000 7f475cc8d640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f47580d5200 0x7f47580d1010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:43.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:43.361+0000 7f475cc8d640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f47580d3760 con 0x7f47580d1550 2024-09-12T03:21:43.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:43.361+0000 7f475cc8d640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f47580b7d20 con 0x7f475815eb90 2024-09-12T03:21:43.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:43.361+0000 7f475cc8d640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f47580d42e0 con 0x7f47580d5200 2024-09-12T03:21:43.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:43.362+0000 7f475659c640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f475815eb90 0x7f47580d2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:43.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:43.362+0000 7f475659c640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f475815eb90 0x7f47580d2df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:50432/0 (socket says 172.21.15.27:50432) 2024-09-12T03:21:43.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:43.362+0000 7f475659c640 1 -- 172.21.15.27:0/1805189606 learned_addr learned my addr 172.21.15.27:0/1805189606 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:21:43.363 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:43.362+0000 7f4755d9b640 1 --2- 172.21.15.27:0/1805189606 >> v2:172.21.15.45:3300/0 conn(0x7f47580d5200 0x7f47580d1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:43.363 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:43.362+0000 7f475659c640 1 -- 172.21.15.27:0/1805189606 >> v2:172.21.15.45:3300/0 conn(0x7f47580d5200 msgr2=0x7f47580d1010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:43.363 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:43.362+0000 7f475659c640 1 --2- 172.21.15.27:0/1805189606 >> v2:172.21.15.45:3300/0 conn(0x7f47580d5200 0x7f47580d1010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:43.363 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:43.362+0000 7f475659c640 1 -- 172.21.15.27:0/1805189606 >> v1:172.21.15.45:6789/0 conn(0x7f47580d1550 legacy=0x7f47580cf200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:43.363 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:43.362+0000 7f475659c640 1 -- 172.21.15.27:0/1805189606 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47580d3a40 con 0x7f475815eb90 2024-09-12T03:21:43.363 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:43.362+0000 7f4755d9b640 1 --2- 172.21.15.27:0/1805189606 >> v2:172.21.15.45:3300/0 conn(0x7f47580d5200 0x7f47580d1010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:21:43.364 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:43.363+0000 7f475659c640 1 --2- 172.21.15.27:0/1805189606 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f475815eb90 0x7f47580d2df0 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f47580d4750 tx=0x7f475002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72d9c926a4e82dd server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:43.364 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:43.363+0000 7f475559a640 1 -- 172.21.15.27:0/1805189606 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4750037070 con 0x7f475815eb90 2024-09-12T03:21:43.364 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:43.363+0000 7f475559a640 1 -- 172.21.15.27:0/1805189606 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4750032440 con 0x7f475815eb90 2024-09-12T03:21:43.364 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:43.363+0000 7f475559a640 1 -- 172.21.15.27:0/1805189606 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4750032810 con 0x7f475815eb90 2024-09-12T03:21:43.364 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:43.363+0000 7f475cc8d640 1 -- 172.21.15.27:0/1805189606 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f475815eb90 msgr2=0x7f47580d2df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:43.365 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:43.363+0000 7f475cc8d640 1 --2- 172.21.15.27:0/1805189606 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f475815eb90 0x7f47580d2df0 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f47580d4750 tx=0x7f475002a230 comp rx=0 tx=0).stop 2024-09-12T03:21:43.365 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:43.363+0000 7f475cc8d640 1 -- 172.21.15.27:0/1805189606 shutdown_connections 2024-09-12T03:21:43.365 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:43.363+0000 7f475cc8d640 1 --2- 172.21.15.27:0/1805189606 >> v2:172.21.15.45:3300/0 conn(0x7f47580d5200 0x7f47580d1010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:43.365 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:43.363+0000 7f475cc8d640 1 --2- 172.21.15.27:0/1805189606 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f475815eb90 0x7f47580d2df0 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:43.365 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:43.363+0000 7f475cc8d640 1 -- 172.21.15.27:0/1805189606 >> 172.21.15.27:0/1805189606 conn(0x7f475806dce0 msgr2=0x7f475806e0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:43.365 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:43.363+0000 7f475cc8d640 1 -- 172.21.15.27:0/1805189606 shutdown_connections 2024-09-12T03:21:43.365 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:43.364+0000 7f475cc8d640 1 -- 172.21.15.27:0/1805189606 wait complete. 2024-09-12T03:21:43.367 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:43.365+0000 7f475cc8d640 1 Processor -- start 2024-09-12T03:21:43.367 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:43.365+0000 7f475cc8d640 1 -- start start 2024-09-12T03:21:43.367 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:43.366+0000 7f475cc8d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f47580d1550 0x7f47580c9390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:43.367 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:43.366+0000 7f475cc8d640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f47580d5200 0x7f47580c98d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:43.367 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:43.366+0000 7f475cc8d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f475815eb90 0x7f4758156bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:43.367 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:43.366+0000 7f475cc8d640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f47580d1960 con 0x7f47580d1550 2024-09-12T03:21:43.367 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:43.366+0000 7f475cc8d640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f47580d4000 con 0x7f475815eb90 2024-09-12T03:21:43.368 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:43.366+0000 7f475cc8d640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f47580d3330 con 0x7f47580d5200 2024-09-12T03:21:43.368 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:43.366+0000 7f475659c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f47580d1550 0x7f47580c9390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:43.368 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:43.366+0000 7f475659c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f47580d1550 0x7f47580c9390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:50442/0 (socket says 172.21.15.27:50442) 2024-09-12T03:21:43.368 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:43.366+0000 7f475659c640 1 -- 172.21.15.27:0/662932211 learned_addr learned my addr 172.21.15.27:0/662932211 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:21:43.368 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:43.366+0000 7f4755d9b640 1 --2- 172.21.15.27:0/662932211 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f47580d5200 0x7f47580c98d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:43.368 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:43.366+0000 7f4756d9d640 1 --2- 172.21.15.27:0/662932211 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f475815eb90 0x7f4758156bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:43.368 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:43.366+0000 7f475659c640 1 -- 172.21.15.27:0/662932211 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f47580d5200 msgr2=0x7f47580c98d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:43.368 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:43.366+0000 7f475659c640 1 --2- 172.21.15.27:0/662932211 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f47580d5200 0x7f47580c98d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:43.368 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:43.366+0000 7f475659c640 1 -- 172.21.15.27:0/662932211 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f475815eb90 msgr2=0x7f4758156bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:43.368 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:43.366+0000 7f475659c640 1 --2- 172.21.15.27:0/662932211 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f475815eb90 0x7f4758156bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:43.368 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:43.366+0000 7f475659c640 1 -- 172.21.15.27:0/662932211 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47581590b0 con 0x7f47580d1550 2024-09-12T03:21:43.368 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:43.366+0000 7f4755d9b640 1 --2- 172.21.15.27:0/662932211 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f47580d5200 0x7f47580c98d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:21:43.368 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:43.366+0000 7f4756d9d640 1 --2- 172.21.15.27:0/662932211 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f475815eb90 0x7f4758156bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:21:43.369 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:43.366+0000 7f475659c640 1 --2- 172.21.15.27:0/662932211 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f47580d1550 0x7f47580c9390 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f4750000c00 tx=0x7f47500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:43.369 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:43.366+0000 7f474effd640 1 -- 172.21.15.27:0/662932211 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4750037040 con 0x7f47580d1550 2024-09-12T03:21:43.369 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:43.366+0000 7f474effd640 1 -- 172.21.15.27:0/662932211 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4750042070 con 0x7f47580d1550 2024-09-12T03:21:43.369 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:43.366+0000 7f474effd640 1 -- 172.21.15.27:0/662932211 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4750030bb0 con 0x7f47580d1550 2024-09-12T03:21:43.369 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:43.366+0000 7f475cc8d640 1 -- 172.21.15.27:0/662932211 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47580c9f00 con 0x7f47580d1550 2024-09-12T03:21:43.369 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:43.366+0000 7f475cc8d640 1 -- 172.21.15.27:0/662932211 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47580ca4c0 con 0x7f47580d1550 2024-09-12T03:21:43.369 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:43.367+0000 7f475cc8d640 1 -- 172.21.15.27:0/662932211 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f475815ef70 con 0x7f47580d1550 2024-09-12T03:21:43.370 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:43.368+0000 7f474effd640 1 -- 172.21.15.27:0/662932211 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 19) ==== 90855+0+0 (secure 0 0 0) 0x7f4750047460 con 0x7f47580d1550 2024-09-12T03:21:43.370 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:43.368+0000 7f474effd640 1 --2- 172.21.15.27:0/662932211 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f473806c220 0x7f473806e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:43.370 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:43.368+0000 7f474effd640 1 -- 172.21.15.27:0/662932211 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 4885+0+0 (secure 0 0 0) 0x7f47500abd30 con 0x7f47580d1550 2024-09-12T03:21:43.370 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:43.369+0000 7f4755d9b640 1 --2- 172.21.15.27:0/662932211 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f473806c220 0x7f473806e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:43.370 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:43.369+0000 7f4755d9b640 1 --2- 172.21.15.27:0/662932211 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f473806c220 0x7f473806e6e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f47581f7950 tx=0x7f4740008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:43.372 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:43.373+0000 7f474effd640 1 -- 172.21.15.27:0/662932211 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f4750045360 con 0x7f47580d1550 2024-09-12T03:21:43.613 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:43.613+0000 7f475cc8d640 1 -- 172.21.15.27:0/662932211 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f475815ff90 con 0x7f47580d1550 2024-09-12T03:21:43.614 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:43.614+0000 7f474effd640 1 -- 172.21.15.27:0/662932211 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v18) ==== 80+0+5304 (secure 0 0 0) 0x7f475007a5d0 con 0x7f47580d1550 2024-09-12T03:21:43.614 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:21:43.614 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-12T03:21:43.614 INFO:teuthology.orchestra.run.smithi027.stdout: "auth_dump": [ 2024-09-12T03:21:43.614 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:21:43.614 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "mds.a", 2024-09-12T03:21:43.614 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDUXOJmIhxjNxAAaGg8SwtW6NRe/lIgNCV20w==", 2024-09-12T03:21:43.614 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:21:43.614 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow", 2024-09-12T03:21:43.615 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow *", 2024-09-12T03:21:43.615 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow *", 2024-09-12T03:21:43.615 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2024-09-12T03:21:43.615 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:21:43.615 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:21:43.615 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:21:43.615 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "mds.b", 2024-09-12T03:21:43.615 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDVXOJm63kWCBAAzjfOsBsWUI+IAsuZ49yEkw==", 2024-09-12T03:21:43.615 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:21:43.615 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow", 2024-09-12T03:21:43.615 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow *", 2024-09-12T03:21:43.615 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow *", 2024-09-12T03:21:43.615 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2024-09-12T03:21:43.615 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:21:43.616 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:21:43.616 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:21:43.616 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "mds.c", 2024-09-12T03:21:43.616 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDVXOJm0sX7AhAAdynCcho2a/AgFF9Cw3+vAw==", 2024-09-12T03:21:43.616 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:21:43.616 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow", 2024-09-12T03:21:43.616 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow *", 2024-09-12T03:21:43.616 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow *", 2024-09-12T03:21:43.616 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2024-09-12T03:21:43.616 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:21:43.616 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:21:43.616 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:21:43.616 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "mds.d", 2024-09-12T03:21:43.617 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDVXOJm7Dd8DxAANHYmvKgpRZ6sVfXDSgPHWw==", 2024-09-12T03:21:43.617 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:21:43.617 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow", 2024-09-12T03:21:43.617 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow *", 2024-09-12T03:21:43.617 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow *", 2024-09-12T03:21:43.617 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2024-09-12T03:21:43.617 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:21:43.617 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:21:43.617 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:21:43.617 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "osd.0", 2024-09-12T03:21:43.617 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDYXOJmJTVYLxAAmo8kxRjaiwTT44dZLZlq4A==", 2024-09-12T03:21:43.617 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:21:43.617 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow profile osd", 2024-09-12T03:21:43.617 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile osd", 2024-09-12T03:21:43.618 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2024-09-12T03:21:43.618 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:21:43.618 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:21:43.618 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:21:43.618 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "osd.1", 2024-09-12T03:21:43.618 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDcXOJmYV0BFBAAmoGX9gf9yLoE/qkWFnv9tg==", 2024-09-12T03:21:43.618 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:21:43.618 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow profile osd", 2024-09-12T03:21:43.618 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile osd", 2024-09-12T03:21:43.618 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2024-09-12T03:21:43.618 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:21:43.618 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:21:43.618 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:21:43.618 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "osd.2", 2024-09-12T03:21:43.619 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDfXOJmEetiJRAAXk+AvfTfhjsB6T2UB4Z2/w==", 2024-09-12T03:21:43.619 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:21:43.619 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow profile osd", 2024-09-12T03:21:43.619 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile osd", 2024-09-12T03:21:43.619 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2024-09-12T03:21:43.619 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:21:43.619 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:21:43.619 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:21:43.619 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "osd.3", 2024-09-12T03:21:43.619 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDiXOJmrQbPJxAAQsi2B5a1LnPTPGZVdo7j7w==", 2024-09-12T03:21:43.619 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:21:43.619 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow profile osd", 2024-09-12T03:21:43.619 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile osd", 2024-09-12T03:21:43.619 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2024-09-12T03:21:43.620 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:21:43.620 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:21:43.620 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:21:43.620 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "osd.4", 2024-09-12T03:21:43.620 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDnXOJmoMr1NxAAv3u2q8hYt9w//9gz+N7UAQ==", 2024-09-12T03:21:43.620 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:21:43.620 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow profile osd", 2024-09-12T03:21:43.620 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile osd", 2024-09-12T03:21:43.620 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2024-09-12T03:21:43.620 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:21:43.620 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:21:43.620 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:21:43.620 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "osd.5", 2024-09-12T03:21:43.621 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDqXOJmxpUPOxAAxLH6RJsyLOZ2XMfzM0fQ1g==", 2024-09-12T03:21:43.621 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:21:43.621 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow profile osd", 2024-09-12T03:21:43.621 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile osd", 2024-09-12T03:21:43.621 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2024-09-12T03:21:43.621 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:21:43.621 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:21:43.621 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:21:43.621 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "osd.6", 2024-09-12T03:21:43.621 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDuXOJmuP0+LxAAMHoz5eSO81hPtkdBIS2cig==", 2024-09-12T03:21:43.621 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:21:43.621 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow profile osd", 2024-09-12T03:21:43.621 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile osd", 2024-09-12T03:21:43.622 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2024-09-12T03:21:43.622 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:21:43.622 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:21:43.622 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:21:43.622 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "osd.7", 2024-09-12T03:21:43.622 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDxXOJmuIC/MRAAPDM+lAHqdoQyGVODwjFp+Q==", 2024-09-12T03:21:43.622 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:21:43.622 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow profile osd", 2024-09-12T03:21:43.622 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile osd", 2024-09-12T03:21:43.622 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2024-09-12T03:21:43.622 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:21:43.622 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:21:43.622 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:21:43.622 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "client.0", 2024-09-12T03:21:43.623 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDVXOJmLm+5ExAAECM4CKfuVElmp/n0b02mXg==", 2024-09-12T03:21:43.623 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:21:43.623 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow", 2024-09-12T03:21:43.623 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow r", 2024-09-12T03:21:43.623 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-12T03:21:43.623 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:21:43.623 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:21:43.623 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:21:43.623 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "client.1", 2024-09-12T03:21:43.623 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDVXOJm61/FHBAALCQouXySfQeZ5DESGmqkzA==", 2024-09-12T03:21:43.623 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:21:43.623 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow", 2024-09-12T03:21:43.623 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow r", 2024-09-12T03:21:43.624 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow rw", 2024-09-12T03:21:43.624 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow rwx" 2024-09-12T03:21:43.624 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:21:43.624 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:21:43.624 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:21:43.624 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "client.2", 2024-09-12T03:21:43.624 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDVXOJmq/ZkGRAAOzmQzEIhb4b/YTxayc34Bw==", 2024-09-12T03:21:43.624 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:21:43.624 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow", 2024-09-12T03:21:43.624 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow r", 2024-09-12T03:21:43.624 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow rw", 2024-09-12T03:21:43.624 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow rwx" 2024-09-12T03:21:43.624 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:21:43.624 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:21:43.625 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:21:43.625 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "client.3", 2024-09-12T03:21:43.625 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDVXOJm+DlzIhAATQKZe+Zyk0MM8efTa6g9PA==", 2024-09-12T03:21:43.625 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:21:43.625 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow", 2024-09-12T03:21:43.625 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow r", 2024-09-12T03:21:43.625 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow rw", 2024-09-12T03:21:43.625 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow rwx" 2024-09-12T03:21:43.625 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:21:43.625 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:21:43.625 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:21:43.625 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "client.admin", 2024-09-12T03:21:43.625 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDUXOJm+JGWGhAApTQw9RtJWd/J5QzQaUFXvg==", 2024-09-12T03:21:43.625 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:21:43.626 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow *", 2024-09-12T03:21:43.626 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow *", 2024-09-12T03:21:43.626 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow *", 2024-09-12T03:21:43.626 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2024-09-12T03:21:43.626 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:21:43.626 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:21:43.626 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:21:43.626 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "mgr.x", 2024-09-12T03:21:43.626 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDUXOJmZcsONBAAMTY2KykuHaEkv1t6Vz5KjA==", 2024-09-12T03:21:43.626 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:21:43.626 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow *", 2024-09-12T03:21:43.626 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile mgr", 2024-09-12T03:21:43.626 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2024-09-12T03:21:43.627 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:21:43.627 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:21:43.627 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:21:43.627 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "mgr.y", 2024-09-12T03:21:43.627 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDUXOJmJOgZMBAAQ/JTxfhox7HgxBMtsvUCBg==", 2024-09-12T03:21:43.627 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:21:43.627 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow *", 2024-09-12T03:21:43.627 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile mgr", 2024-09-12T03:21:43.627 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2024-09-12T03:21:43.627 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:21:43.627 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:21:43.627 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-12T03:21:43.627 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-12T03:21:43.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:43.616+0000 7f475cc8d640 1 -- 172.21.15.27:0/662932211 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f473806c220 msgr2=0x7f473806e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:43.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:43.616+0000 7f475cc8d640 1 --2- 172.21.15.27:0/662932211 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f473806c220 0x7f473806e6e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f47581f7950 tx=0x7f4740008040 comp rx=0 tx=0).stop 2024-09-12T03:21:43.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:43.616+0000 7f475cc8d640 1 -- 172.21.15.27:0/662932211 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f47580d1550 msgr2=0x7f47580c9390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:43.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:43.616+0000 7f475cc8d640 1 --2- 172.21.15.27:0/662932211 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f47580d1550 0x7f47580c9390 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f4750000c00 tx=0x7f47500046c0 comp rx=0 tx=0).stop 2024-09-12T03:21:43.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:43.616+0000 7f475cc8d640 1 -- 172.21.15.27:0/662932211 shutdown_connections 2024-09-12T03:21:43.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:43.616+0000 7f475cc8d640 1 --2- 172.21.15.27:0/662932211 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f475815eb90 0x7f4758156bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:43.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:43.616+0000 7f475cc8d640 1 --2- 172.21.15.27:0/662932211 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f47580d5200 0x7f47580c98d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:43.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:43.616+0000 7f475cc8d640 1 --2- 172.21.15.27:0/662932211 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f473806c220 0x7f473806e6e0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:43.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:43.616+0000 7f475cc8d640 1 --2- 172.21.15.27:0/662932211 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f47580d1550 0x7f47580c9390 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:43.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:43.616+0000 7f475cc8d640 1 -- 172.21.15.27:0/662932211 >> 172.21.15.27:0/662932211 conn(0x7f475806dce0 msgr2=0x7f475815f180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:43.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:43.616+0000 7f475cc8d640 1 -- 172.21.15.27:0/662932211 shutdown_connections 2024-09-12T03:21:43.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:43.616+0000 7f475cc8d640 1 -- 172.21.15.27:0/662932211 wait complete. 2024-09-12T03:21:43.630 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-12T03:21:43.732 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:43.732+0000 7f8662417640 1 Processor -- start 2024-09-12T03:21:43.733 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:43.732+0000 7f8662417640 1 -- start start 2024-09-12T03:21:43.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:43.733+0000 7f8662417640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f865c0d5200 0x7f865c0d1010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:43.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:43.733+0000 7f8662417640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f865c0d1550 0x7f865c0b8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:43.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:43.733+0000 7f8662417640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f865c0d19c0 con 0x7f865c15eb90 2024-09-12T03:21:43.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:43.733+0000 7f8662417640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f865c0b7d20 con 0x7f865c0d1550 2024-09-12T03:21:43.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:43.733+0000 7f8662417640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f865c0d42e0 con 0x7f865c0d5200 2024-09-12T03:21:43.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:43.733+0000 7f86609b4640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f865c0d1550 0x7f865c0b8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:43.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:43.734+0000 7f865bfff640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f865c15eb90 0x7f865c0d2df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:34652/0 (socket says 172.21.15.27:34652) 2024-09-12T03:21:43.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:43.734+0000 7f865bfff640 1 -- 172.21.15.27:0/3006024514 learned_addr learned my addr 172.21.15.27:0/3006024514 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:21:43.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:43.734+0000 7f865b7fe640 1 --2- 172.21.15.27:0/3006024514 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f865c0d5200 0x7f865c0d1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:43.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:43.734+0000 7f86609b4640 1 -- 172.21.15.27:0/3006024514 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f865c0d5200 msgr2=0x7f865c0d1010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:43.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:43.734+0000 7f86609b4640 1 --2- 172.21.15.27:0/3006024514 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f865c0d5200 0x7f865c0d1010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:43.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:43.734+0000 7f86609b4640 1 -- 172.21.15.27:0/3006024514 >> v1:172.21.15.27:6789/0 conn(0x7f865c15eb90 legacy=0x7f865c0d2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:43.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:43.734+0000 7f865affd640 1 -- 172.21.15.27:0/3006024514 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2533845011 0 0) 0x7f865c0d19c0 con 0x7f865c15eb90 2024-09-12T03:21:43.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:43.734+0000 7f86609b4640 1 -- 172.21.15.27:0/3006024514 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f865c0d3760 con 0x7f865c0d1550 2024-09-12T03:21:43.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:43.735+0000 7f86609b4640 1 --2- 172.21.15.27:0/3006024514 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f865c0d1550 0x7f865c0b8190 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f8654009d90 tx=0x7f865402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=995c062debc5bc26 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:43.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:43.735+0000 7f865affd640 1 -- 172.21.15.27:0/3006024514 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8654007740 con 0x7f865c0d1550 2024-09-12T03:21:43.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:43.735+0000 7f865affd640 1 -- 172.21.15.27:0/3006024514 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86540078e0 con 0x7f865c0d1550 2024-09-12T03:21:43.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:43.735+0000 7f865affd640 1 -- 172.21.15.27:0/3006024514 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8654007cb0 con 0x7f865c0d1550 2024-09-12T03:21:43.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:43.735+0000 7f8662417640 1 -- 172.21.15.27:0/3006024514 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f865c0d1550 msgr2=0x7f865c0b8190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:43.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:43.735+0000 7f8662417640 1 --2- 172.21.15.27:0/3006024514 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f865c0d1550 0x7f865c0b8190 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f8654009d90 tx=0x7f865402a660 comp rx=0 tx=0).stop 2024-09-12T03:21:43.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:43.735+0000 7f8662417640 1 -- 172.21.15.27:0/3006024514 shutdown_connections 2024-09-12T03:21:43.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:43.735+0000 7f8662417640 1 --2- 172.21.15.27:0/3006024514 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f865c0d1550 0x7f865c0b8190 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:43.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:43.735+0000 7f8662417640 1 --2- 172.21.15.27:0/3006024514 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f865c0d5200 0x7f865c0d1010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:43.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:43.735+0000 7f8662417640 1 -- 172.21.15.27:0/3006024514 >> 172.21.15.27:0/3006024514 conn(0x7f865c06dce0 msgr2=0x7f865c06e0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:43.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:43.735+0000 7f8662417640 1 -- 172.21.15.27:0/3006024514 shutdown_connections 2024-09-12T03:21:43.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:43.735+0000 7f8662417640 1 -- 172.21.15.27:0/3006024514 wait complete. 2024-09-12T03:21:43.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:43.737+0000 7f8662417640 1 Processor -- start 2024-09-12T03:21:43.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:43.737+0000 7f8662417640 1 -- start start 2024-09-12T03:21:43.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:43.737+0000 7f8662417640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f865c0d1550 0x7f865c0ab210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:43.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:43.737+0000 7f8662417640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f865c0d5200 0x7f865c0ac890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:43.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:43.737+0000 7f8662417640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f865c15eb90 0x7f865c0cdab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:43.740 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:43.737+0000 7f8662417640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f865c0d3d20 con 0x7f865c15eb90 2024-09-12T03:21:43.740 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:43.738+0000 7f8662417640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f865c0d3500 con 0x7f865c0d5200 2024-09-12T03:21:43.740 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:43.738+0000 7f8662417640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f865c0d4000 con 0x7f865c0d1550 2024-09-12T03:21:43.740 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:43.738+0000 7f86609b4640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f865c15eb90 0x7f865c0cdab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:43.740 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:43.738+0000 7f86609b4640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f865c15eb90 0x7f865c0cdab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:50458/0 (socket says 172.21.15.27:50458) 2024-09-12T03:21:43.740 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:43.738+0000 7f86609b4640 1 -- 172.21.15.27:0/4018124332 learned_addr learned my addr 172.21.15.27:0/4018124332 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:21:43.740 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:43.738+0000 7f865b7fe640 1 --2- 172.21.15.27:0/4018124332 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f865c0d5200 0x7f865c0ac890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:43.740 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:43.738+0000 7f865bfff640 1 --2- 172.21.15.27:0/4018124332 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f865c0d1550 0x7f865c0ab210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:43.740 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:43.738+0000 7f86609b4640 1 -- 172.21.15.27:0/4018124332 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f865c0d1550 msgr2=0x7f865c0ab210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:43.740 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:43.738+0000 7f86609b4640 1 --2- 172.21.15.27:0/4018124332 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f865c0d1550 0x7f865c0ab210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:43.740 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:43.738+0000 7f86609b4640 1 -- 172.21.15.27:0/4018124332 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f865c0d5200 msgr2=0x7f865c0ac890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:43.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:43.738+0000 7f86609b4640 1 --2- 172.21.15.27:0/4018124332 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f865c0d5200 0x7f865c0ac890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:43.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:43.738+0000 7f86609b4640 1 -- 172.21.15.27:0/4018124332 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f865c0ab7f0 con 0x7f865c15eb90 2024-09-12T03:21:43.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:43.738+0000 7f865b7fe640 1 --2- 172.21.15.27:0/4018124332 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f865c0d5200 0x7f865c0ac890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:21:43.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:43.738+0000 7f86609b4640 1 --2- 172.21.15.27:0/4018124332 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f865c15eb90 0x7f865c0cdab0 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f8654009d90 tx=0x7f8654004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:43.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:43.738+0000 7f8658ff9640 1 -- 172.21.15.27:0/4018124332 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8654040070 con 0x7f865c15eb90 2024-09-12T03:21:43.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:43.738+0000 7f8658ff9640 1 -- 172.21.15.27:0/4018124332 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8654031bc0 con 0x7f865c15eb90 2024-09-12T03:21:43.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:43.738+0000 7f8662417640 1 -- 172.21.15.27:0/4018124332 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f865c0abad0 con 0x7f865c15eb90 2024-09-12T03:21:43.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:43.738+0000 7f8658ff9640 1 -- 172.21.15.27:0/4018124332 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8654009450 con 0x7f865c15eb90 2024-09-12T03:21:43.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:43.738+0000 7f8662417640 1 -- 172.21.15.27:0/4018124332 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f865c0ac090 con 0x7f865c15eb90 2024-09-12T03:21:43.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:43.739+0000 7f8662417640 1 -- 172.21.15.27:0/4018124332 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f865c15f050 con 0x7f865c15eb90 2024-09-12T03:21:43.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:43.740+0000 7f8658ff9640 1 -- 172.21.15.27:0/4018124332 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 19) ==== 90855+0+0 (secure 0 0 0) 0x7f8654039040 con 0x7f865c15eb90 2024-09-12T03:21:43.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:43.740+0000 7f8658ff9640 1 --2- 172.21.15.27:0/4018124332 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f864806c220 0x7f864806e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:43.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:43.740+0000 7f8658ff9640 1 -- 172.21.15.27:0/4018124332 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 4885+0+0 (secure 0 0 0) 0x7f86540ac7b0 con 0x7f865c15eb90 2024-09-12T03:21:43.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:43.741+0000 7f865bfff640 1 --2- 172.21.15.27:0/4018124332 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f864806c220 0x7f864806e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:43.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:43.741+0000 7f865bfff640 1 --2- 172.21.15.27:0/4018124332 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f864806c220 0x7f864806e6e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f864c000f60 tx=0x7f864c009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:43.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:43.745+0000 7f8658ff9640 1 -- 172.21.15.27:0/4018124332 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f865407ada0 con 0x7f865c15eb90 2024-09-12T03:21:43.923 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:43.922+0000 7f8662417640 1 -- 172.21.15.27:0/4018124332 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f865c161610 con 0x7f865c15eb90 2024-09-12T03:21:43.923 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:43.922+0000 7f8658ff9640 1 -- 172.21.15.27:0/4018124332 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v110) ==== 84+0+65 (secure 0 0 0) 0x7f8654038e10 con 0x7f865c15eb90 2024-09-12T03:21:43.923 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:21:43.923 INFO:teuthology.orchestra.run.smithi027.stdout:[ 2024-09-12T03:21:43.924 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:21:43.924 INFO:teuthology.orchestra.run.smithi027.stdout: "poolnum": 1, 2024-09-12T03:21:43.924 INFO:teuthology.orchestra.run.smithi027.stdout: "poolname": ".mgr" 2024-09-12T03:21:43.924 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:21:43.924 INFO:teuthology.orchestra.run.smithi027.stdout:] 2024-09-12T03:21:43.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:43.924+0000 7f8662417640 1 -- 172.21.15.27:0/4018124332 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f864806c220 msgr2=0x7f864806e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:43.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:43.924+0000 7f8662417640 1 --2- 172.21.15.27:0/4018124332 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f864806c220 0x7f864806e6e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f864c000f60 tx=0x7f864c009040 comp rx=0 tx=0).stop 2024-09-12T03:21:43.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:43.924+0000 7f8662417640 1 -- 172.21.15.27:0/4018124332 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f865c15eb90 msgr2=0x7f865c0cdab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:43.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:43.924+0000 7f8662417640 1 --2- 172.21.15.27:0/4018124332 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f865c15eb90 0x7f865c0cdab0 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f8654009d90 tx=0x7f8654004a40 comp rx=0 tx=0).stop 2024-09-12T03:21:43.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:43.924+0000 7f8662417640 1 -- 172.21.15.27:0/4018124332 shutdown_connections 2024-09-12T03:21:43.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:43.924+0000 7f8662417640 1 --2- 172.21.15.27:0/4018124332 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f864806c220 0x7f864806e6e0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:43.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:43.924+0000 7f8662417640 1 --2- 172.21.15.27:0/4018124332 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f865c15eb90 0x7f865c0cdab0 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:43.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:43.924+0000 7f8662417640 1 --2- 172.21.15.27:0/4018124332 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f865c0d5200 0x7f865c0ac890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:43.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:43.924+0000 7f8662417640 1 --2- 172.21.15.27:0/4018124332 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f865c0d1550 0x7f865c0ab210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:43.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:43.924+0000 7f8662417640 1 -- 172.21.15.27:0/4018124332 >> 172.21.15.27:0/4018124332 conn(0x7f865c06dce0 msgr2=0x7f865c160710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:43.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:43.925+0000 7f8662417640 1 -- 172.21.15.27:0/4018124332 shutdown_connections 2024-09-12T03:21:43.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:43.925+0000 7f8662417640 1 -- 172.21.15.27:0/4018124332 wait complete. 2024-09-12T03:21:43.935 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-12T03:21:43.935 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-12T03:21:44.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:44.042+0000 7f870607b640 1 Processor -- start 2024-09-12T03:21:44.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:44.042+0000 7f870607b640 1 -- start start 2024-09-12T03:21:44.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:44.043+0000 7f870607b640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f870015e650 0x7f87000d1d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:44.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:44.043+0000 7f870607b640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f87000d2280 con 0x7f870006db30 2024-09-12T03:21:44.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:44.043+0000 7f870607b640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f870006e670 con 0x7f870006cc40 2024-09-12T03:21:44.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:44.043+0000 7f870607b640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f870006d850 con 0x7f870015e650 2024-09-12T03:21:44.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:44.043+0000 7f86fffff640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f870006db30 0x7f870006df60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:34658/0 (socket says 172.21.15.27:34658) 2024-09-12T03:21:44.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:44.043+0000 7f86fffff640 1 -- 172.21.15.27:0/1375189212 learned_addr learned my addr 172.21.15.27:0/1375189212 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:21:44.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:44.043+0000 7f86ff7fe640 1 --2- 172.21.15.27:0/1375189212 >> v2:172.21.15.45:3300/0 conn(0x7f870015e650 0x7f87000d1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:44.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:44.043+0000 7f86fe7fc640 1 -- 172.21.15.27:0/1375189212 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1322981158 0 0) 0x7f87000d2280 con 0x7f870006db30 2024-09-12T03:21:44.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:44.043+0000 7f86fe7fc640 1 -- 172.21.15.27:0/1375189212 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f86e8003520 con 0x7f870006db30 2024-09-12T03:21:44.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:44.043+0000 7f86fe7fc640 1 -- 172.21.15.27:0/1375189212 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2398880888 0 0) 0x7f86e8003520 con 0x7f870006db30 2024-09-12T03:21:44.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:44.043+0000 7f86fe7fc640 1 -- 172.21.15.27:0/1375189212 >> v2:172.21.15.45:3300/0 conn(0x7f870015e650 msgr2=0x7f87000d1d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:44.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:44.043+0000 7f86fe7fc640 1 --2- 172.21.15.27:0/1375189212 >> v2:172.21.15.45:3300/0 conn(0x7f870015e650 0x7f87000d1d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:44.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:44.043+0000 7f86fe7fc640 1 -- 172.21.15.27:0/1375189212 >> v1:172.21.15.45:6789/0 conn(0x7f870006cc40 legacy=0x7f870006d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:44.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:44.043+0000 7f86fe7fc640 1 -- 172.21.15.27:0/1375189212 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87000d1260 con 0x7f870006db30 2024-09-12T03:21:44.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:44.043+0000 7f86fe7fc640 1 -- 172.21.15.27:0/1375189212 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 847755213 0 0) 0x7f86f80019f0 con 0x7f870006db30 2024-09-12T03:21:44.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:44.043+0000 7f86fe7fc640 1 -- 172.21.15.27:0/1375189212 >> v1:172.21.15.27:6789/0 conn(0x7f870006db30 legacy=0x7f870006df60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:44.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:44.043+0000 7f86fe7fc640 1 --2- 172.21.15.27:0/1375189212 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f86e8003a00 0x7f86e8003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:44.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:44.044+0000 7f86fe7fc640 1 --2- 172.21.15.27:0/1375189212 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f86e8004800 0x7f86e8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:44.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:44.044+0000 7f86fe7fc640 1 --2- 172.21.15.27:0/1375189212 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f86e8005210 0x7f86e800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:44.046 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:44.044+0000 7f86fe7fc640 1 -- 172.21.15.27:0/1375189212 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f870006d850 con 0x7f86e8005210 2024-09-12T03:21:44.046 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:44.044+0000 7f86fe7fc640 1 -- 172.21.15.27:0/1375189212 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f86e800c160 con 0x7f86e8003a00 2024-09-12T03:21:44.046 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:44.044+0000 7f86fe7fc640 1 -- 172.21.15.27:0/1375189212 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f86e800c360 con 0x7f86e8004800 2024-09-12T03:21:44.046 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:44.044+0000 7f86fffff640 1 --2- 172.21.15.27:0/1375189212 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f86e8005210 0x7f86e800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:44.046 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:44.044+0000 7f86ff7fe640 1 --2- 172.21.15.27:0/1375189212 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f86e8003a00 0x7f86e8003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:44.046 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:44.044+0000 7f86feffd640 1 --2- 172.21.15.27:0/1375189212 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f86e8004800 0x7f86e8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:44.046 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:44.044+0000 7f86fffff640 1 -- 172.21.15.27:0/1375189212 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f86e8004800 msgr2=0x7f86e8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:44.046 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:44.044+0000 7f86fffff640 1 --2- 172.21.15.27:0/1375189212 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f86e8004800 0x7f86e8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:44.046 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:44.044+0000 7f86fffff640 1 -- 172.21.15.27:0/1375189212 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f86e8003a00 msgr2=0x7f86e8003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:44.046 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:44.044+0000 7f86fffff640 1 --2- 172.21.15.27:0/1375189212 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f86e8003a00 0x7f86e8003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:44.046 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:44.044+0000 7f86fffff640 1 -- 172.21.15.27:0/1375189212 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86e8003520 con 0x7f86e8005210 2024-09-12T03:21:44.046 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:44.045+0000 7f86fffff640 1 -- 172.21.15.27:0/1375189212 reap_dead start 2024-09-12T03:21:44.050 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:44.045+0000 7f86fffff640 1 --2- 172.21.15.27:0/1375189212 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f86e8005210 0x7f86e800b9c0 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f86f800a760 tx=0x7f86e8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fbced4d5a0db04c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:44.050 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:44.045+0000 7f86fe7fc640 1 -- 172.21.15.27:0/1375189212 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86f800a3b0 con 0x7f86e8005210 2024-09-12T03:21:44.050 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:44.045+0000 7f86fe7fc640 1 -- 172.21.15.27:0/1375189212 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86f802f070 con 0x7f86e8005210 2024-09-12T03:21:44.050 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:44.045+0000 7f86fe7fc640 1 -- 172.21.15.27:0/1375189212 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86f802bba0 con 0x7f86e8005210 2024-09-12T03:21:44.050 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:44.045+0000 7f870607b640 1 -- 172.21.15.27:0/1375189212 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f86e8005210 msgr2=0x7f86e800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:44.050 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:44.045+0000 7f870607b640 1 --2- 172.21.15.27:0/1375189212 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f86e8005210 0x7f86e800b9c0 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f86f800a760 tx=0x7f86e8005830 comp rx=0 tx=0).stop 2024-09-12T03:21:44.050 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:44.045+0000 7f870607b640 1 -- 172.21.15.27:0/1375189212 shutdown_connections 2024-09-12T03:21:44.050 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:44.045+0000 7f870607b640 1 --2- 172.21.15.27:0/1375189212 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f86e8005210 0x7f86e800b9c0 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:44.050 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:44.045+0000 7f870607b640 1 -- 172.21.15.27:0/1375189212 >> 172.21.15.27:0/1375189212 conn(0x7f870006f320 msgr2=0x7f870006f700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:44.050 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:44.045+0000 7f870607b640 1 -- 172.21.15.27:0/1375189212 shutdown_connections 2024-09-12T03:21:44.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:44.045+0000 7f870607b640 1 -- 172.21.15.27:0/1375189212 wait complete. 2024-09-12T03:21:44.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:44.047+0000 7f870607b640 1 Processor -- start 2024-09-12T03:21:44.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:44.047+0000 7f870607b640 1 -- start start 2024-09-12T03:21:44.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:44.047+0000 7f870607b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f86e8005210 0x7f87001481f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:44.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:44.047+0000 7f870607b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f87000cb930 0x7f87000cbdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:44.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:44.047+0000 7f870607b640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f8700156c20 0x7f87001570a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:44.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:44.047+0000 7f870607b640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f870015ebb0 con 0x7f87000cb930 2024-09-12T03:21:44.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:44.047+0000 7f870607b640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f870015ea30 con 0x7f86e8005210 2024-09-12T03:21:44.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:44.047+0000 7f870607b640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f87000d1d40 con 0x7f8700156c20 2024-09-12T03:21:44.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:44.048+0000 7f86feffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f87000cb930 0x7f87000cbdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:44.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:44.048+0000 7f86feffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f87000cb930 0x7f87000cbdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:50472/0 (socket says 172.21.15.27:50472) 2024-09-12T03:21:44.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:44.048+0000 7f86feffd640 1 -- 172.21.15.27:0/2889606515 learned_addr learned my addr 172.21.15.27:0/2889606515 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:21:44.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:44.048+0000 7f86ff7fe640 1 --2- 172.21.15.27:0/2889606515 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f86e8005210 0x7f87001481f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:44.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:44.048+0000 7f86fffff640 1 --2- 172.21.15.27:0/2889606515 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f8700156c20 0x7f87001570a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:44.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:44.048+0000 7f86feffd640 1 -- 172.21.15.27:0/2889606515 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f8700156c20 msgr2=0x7f87001570a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:44.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:44.048+0000 7f86feffd640 1 --2- 172.21.15.27:0/2889606515 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f8700156c20 0x7f87001570a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:44.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:44.048+0000 7f86feffd640 1 -- 172.21.15.27:0/2889606515 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f86e8005210 msgr2=0x7f87001481f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:44.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:44.048+0000 7f86feffd640 1 --2- 172.21.15.27:0/2889606515 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f86e8005210 0x7f87001481f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:44.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:44.048+0000 7f86feffd640 1 -- 172.21.15.27:0/2889606515 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87000cdcc0 con 0x7f87000cb930 2024-09-12T03:21:44.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:44.048+0000 7f86fffff640 1 --2- 172.21.15.27:0/2889606515 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f8700156c20 0x7f87001570a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:21:44.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:44.048+0000 7f86feffd640 1 --2- 172.21.15.27:0/2889606515 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f87000cb930 0x7f87000cbdb0 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f86f400e4d0 tx=0x7f86f400e8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:44.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:44.048+0000 7f86e7fff640 1 -- 172.21.15.27:0/2889606515 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86f4019070 con 0x7f87000cb930 2024-09-12T03:21:44.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:44.048+0000 7f86e7fff640 1 -- 172.21.15.27:0/2889606515 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86f40052e0 con 0x7f87000cb930 2024-09-12T03:21:44.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:44.048+0000 7f86e7fff640 1 -- 172.21.15.27:0/2889606515 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86f4003810 con 0x7f87000cb930 2024-09-12T03:21:44.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:44.048+0000 7f870607b640 1 -- 172.21.15.27:0/2889606515 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87000cdf60 con 0x7f87000cb930 2024-09-12T03:21:44.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:44.048+0000 7f870607b640 1 -- 172.21.15.27:0/2889606515 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87000ce520 con 0x7f87000cb930 2024-09-12T03:21:44.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:44.049+0000 7f870607b640 1 -- 172.21.15.27:0/2889606515 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87000d20d0 con 0x7f87000cb930 2024-09-12T03:21:44.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:44.050+0000 7f86e7fff640 1 -- 172.21.15.27:0/2889606515 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 19) ==== 90855+0+0 (secure 0 0 0) 0x7f86f401c070 con 0x7f87000cb930 2024-09-12T03:21:44.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:44.050+0000 7f86e7fff640 1 --2- 172.21.15.27:0/2889606515 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f86ec06c220 0x7f86ec06e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:44.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:44.050+0000 7f86e7fff640 1 -- 172.21.15.27:0/2889606515 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 4885+0+0 (secure 0 0 0) 0x7f86f408a740 con 0x7f87000cb930 2024-09-12T03:21:44.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:44.050+0000 7f86ff7fe640 1 --2- 172.21.15.27:0/2889606515 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f86ec06c220 0x7f86ec06e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:44.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:44.051+0000 7f86ff7fe640 1 --2- 172.21.15.27:0/2889606515 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f86ec06c220 0x7f86ec06e6e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f86f0009b40 tx=0x7f86f0009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:44.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:44.055+0000 7f86e7fff640 1 -- 172.21.15.27:0/2889606515 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f86f4058880 con 0x7f87000cb930 2024-09-12T03:21:44.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:44.232+0000 7f870607b640 1 -- 172.21.15.27:0/2889606515 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f870015c370 con 0x7f87000cb930 2024-09-12T03:21:45.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:45.236+0000 7f86e7fff640 1 -- 172.21.15.27:0/2889606515 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v111) ==== 138+0+0 (secure 0 0 0) 0x7f86f405cb80 con 0x7f87000cb930 2024-09-12T03:21:45.237 INFO:teuthology.orchestra.run.smithi027.stderr:pool 'cephfs_metadata' created 2024-09-12T03:21:45.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:45.240+0000 7f870607b640 1 -- 172.21.15.27:0/2889606515 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f86ec06c220 msgr2=0x7f86ec06e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:45.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:45.240+0000 7f870607b640 1 --2- 172.21.15.27:0/2889606515 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f86ec06c220 0x7f86ec06e6e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f86f0009b40 tx=0x7f86f0009190 comp rx=0 tx=0).stop 2024-09-12T03:21:45.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:45.240+0000 7f870607b640 1 -- 172.21.15.27:0/2889606515 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f87000cb930 msgr2=0x7f87000cbdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:45.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:45.240+0000 7f870607b640 1 --2- 172.21.15.27:0/2889606515 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f87000cb930 0x7f87000cbdb0 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f86f400e4d0 tx=0x7f86f400e8d0 comp rx=0 tx=0).stop 2024-09-12T03:21:45.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:45.240+0000 7f870607b640 1 -- 172.21.15.27:0/2889606515 shutdown_connections 2024-09-12T03:21:45.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:45.240+0000 7f870607b640 1 --2- 172.21.15.27:0/2889606515 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f8700156c20 0x7f87001570a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:45.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:45.240+0000 7f870607b640 1 --2- 172.21.15.27:0/2889606515 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f86ec06c220 0x7f86ec06e6e0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:45.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:45.240+0000 7f870607b640 1 --2- 172.21.15.27:0/2889606515 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f87000cb930 0x7f87000cbdb0 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:45.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:45.240+0000 7f870607b640 1 --2- 172.21.15.27:0/2889606515 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f86e8005210 0x7f87001481f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:45.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:45.240+0000 7f870607b640 1 -- 172.21.15.27:0/2889606515 >> 172.21.15.27:0/2889606515 conn(0x7f870006f320 msgr2=0x7f8700074c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:45.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:45.240+0000 7f870607b640 1 -- 172.21.15.27:0/2889606515 shutdown_connections 2024-09-12T03:21:45.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:45.241+0000 7f870607b640 1 -- 172.21.15.27:0/2889606515 wait complete. 2024-09-12T03:21:45.257 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-12T03:21:45.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:45.374+0000 7fdfcebc4640 1 Processor -- start 2024-09-12T03:21:45.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:45.374+0000 7fdfcebc4640 1 -- start start 2024-09-12T03:21:45.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:45.375+0000 7fdfcebc4640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fdfc8162040 0x7fdfc80d1da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:45.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:45.375+0000 7fdfcebc4640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fdfc806dfd0 0x7fdfc80b7a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:45.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:45.375+0000 7fdfcebc4640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdfc80d2e30 con 0x7fdfc80d3790 2024-09-12T03:21:45.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:45.375+0000 7fdfcebc4640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7fdfc80b7f90 con 0x7fdfc806dfd0 2024-09-12T03:21:45.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:45.375+0000 7fdfcebc4640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fdfc8162420 con 0x7fdfc8162040 2024-09-12T03:21:45.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:45.375+0000 7fdfcd161640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fdfc806dfd0 0x7fdfc80b7a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:45.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:45.375+0000 7fdfcd161640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fdfc806dfd0 0x7fdfc80b7a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:50480/0 (socket says 172.21.15.27:50480) 2024-09-12T03:21:45.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:45.375+0000 7fdfcd161640 1 -- 172.21.15.27:0/1072976588 learned_addr learned my addr 172.21.15.27:0/1072976588 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:21:45.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:45.375+0000 7fdfcc960640 1 --2- 172.21.15.27:0/1072976588 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fdfc8162040 0x7fdfc80d1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:45.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:45.375+0000 7fdfcd161640 1 -- 172.21.15.27:0/1072976588 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fdfc8162040 msgr2=0x7fdfc80d1da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:45.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:45.375+0000 7fdfcd161640 1 --2- 172.21.15.27:0/1072976588 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fdfc8162040 0x7fdfc80d1da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:45.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:45.376+0000 7fdfcd161640 1 -- 172.21.15.27:0/1072976588 >> v1:172.21.15.45:6789/0 conn(0x7fdfc80d3790 legacy=0x7fdfc806d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:45.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:45.376+0000 7fdfcd161640 1 -- 172.21.15.27:0/1072976588 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfc80d30d0 con 0x7fdfc806dfd0 2024-09-12T03:21:45.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:45.376+0000 7fdfcc960640 1 --2- 172.21.15.27:0/1072976588 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fdfc8162040 0x7fdfc80d1da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:21:45.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:45.376+0000 7fdfcd161640 1 --2- 172.21.15.27:0/1072976588 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdfc806dfd0 0x7fdfc80b7a50 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7fdfc0009e50 tx=0x7fdfc002a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e1946f9a2a2dffb server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:45.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:45.376+0000 7fdfc77fe640 1 -- 172.21.15.27:0/1072976588 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfc0036070 con 0x7fdfc806dfd0 2024-09-12T03:21:45.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:45.376+0000 7fdfc77fe640 1 -- 172.21.15.27:0/1072976588 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdfc0031600 con 0x7fdfc806dfd0 2024-09-12T03:21:45.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:45.376+0000 7fdfc77fe640 1 -- 172.21.15.27:0/1072976588 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfc00319d0 con 0x7fdfc806dfd0 2024-09-12T03:21:45.379 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:45.377+0000 7fdfcebc4640 1 -- 172.21.15.27:0/1072976588 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdfc806dfd0 msgr2=0x7fdfc80b7a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:45.379 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:45.377+0000 7fdfcebc4640 1 --2- 172.21.15.27:0/1072976588 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdfc806dfd0 0x7fdfc80b7a50 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7fdfc0009e50 tx=0x7fdfc002a710 comp rx=0 tx=0).stop 2024-09-12T03:21:45.379 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:45.377+0000 7fdfcebc4640 1 -- 172.21.15.27:0/1072976588 shutdown_connections 2024-09-12T03:21:45.379 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:45.377+0000 7fdfcebc4640 1 --2- 172.21.15.27:0/1072976588 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdfc806dfd0 0x7fdfc80b7a50 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:45.379 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:45.377+0000 7fdfcebc4640 1 --2- 172.21.15.27:0/1072976588 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fdfc8162040 0x7fdfc80d1da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:45.379 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:45.377+0000 7fdfcebc4640 1 -- 172.21.15.27:0/1072976588 >> 172.21.15.27:0/1072976588 conn(0x7fdfc80b6700 msgr2=0x7fdfc80b6b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:45.379 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:45.377+0000 7fdfcebc4640 1 -- 172.21.15.27:0/1072976588 shutdown_connections 2024-09-12T03:21:45.379 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:45.377+0000 7fdfcebc4640 1 -- 172.21.15.27:0/1072976588 wait complete. 2024-09-12T03:21:45.379 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:45.378+0000 7fdfcebc4640 1 Processor -- start 2024-09-12T03:21:45.379 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:45.378+0000 7fdfcebc4640 1 -- start start 2024-09-12T03:21:45.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:45.379+0000 7fdfcebc4640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdfc806dfd0 0x7fdfc815bde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:45.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:45.379+0000 7fdfcebc4640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdfc80d3790 0x7fdfc80cd2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:45.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:45.379+0000 7fdfcebc4640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fdfc8162040 0x7fdfc81f4620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:45.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:45.379+0000 7fdfcebc4640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fdfc80d4530 con 0x7fdfc80d3790 2024-09-12T03:21:45.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:45.379+0000 7fdfcebc4640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fdfc80cee00 con 0x7fdfc806dfd0 2024-09-12T03:21:45.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:45.379+0000 7fdfcebc4640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fdfc80d4150 con 0x7fdfc8162040 2024-09-12T03:21:45.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:45.379+0000 7fdfc7fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdfc80d3790 0x7fdfc80cd2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:45.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:45.379+0000 7fdfc7fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdfc80d3790 0x7fdfc80cd2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:50496/0 (socket says 172.21.15.27:50496) 2024-09-12T03:21:45.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:45.379+0000 7fdfc7fff640 1 -- 172.21.15.27:0/465255469 learned_addr learned my addr 172.21.15.27:0/465255469 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:21:45.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:45.380+0000 7fdfcd161640 1 --2- 172.21.15.27:0/465255469 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fdfc8162040 0x7fdfc81f4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:45.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:45.380+0000 7fdfcc960640 1 --2- 172.21.15.27:0/465255469 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdfc806dfd0 0x7fdfc815bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:45.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:45.380+0000 7fdfc7fff640 1 -- 172.21.15.27:0/465255469 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fdfc8162040 msgr2=0x7fdfc81f4620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:45.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:45.380+0000 7fdfc7fff640 1 --2- 172.21.15.27:0/465255469 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fdfc8162040 0x7fdfc81f4620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:45.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:45.380+0000 7fdfc7fff640 1 -- 172.21.15.27:0/465255469 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdfc806dfd0 msgr2=0x7fdfc815bde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:45.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:45.380+0000 7fdfc7fff640 1 --2- 172.21.15.27:0/465255469 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdfc806dfd0 0x7fdfc815bde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:45.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:45.380+0000 7fdfc7fff640 1 -- 172.21.15.27:0/465255469 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfc815c320 con 0x7fdfc80d3790 2024-09-12T03:21:45.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:45.380+0000 7fdfc7fff640 1 --2- 172.21.15.27:0/465255469 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdfc80d3790 0x7fdfc80cd2e0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7fdfbc00c3a0 tx=0x7fdfbc00c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:45.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:45.380+0000 7fdfc57fa640 1 -- 172.21.15.27:0/465255469 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfbc0052d0 con 0x7fdfc80d3790 2024-09-12T03:21:45.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:45.380+0000 7fdfc57fa640 1 -- 172.21.15.27:0/465255469 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdfbc005490 con 0x7fdfc80d3790 2024-09-12T03:21:45.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:45.380+0000 7fdfcebc4640 1 -- 172.21.15.27:0/465255469 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfc81f5130 con 0x7fdfc80d3790 2024-09-12T03:21:45.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:45.380+0000 7fdfc57fa640 1 -- 172.21.15.27:0/465255469 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfbc007260 con 0x7fdfc80d3790 2024-09-12T03:21:45.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:45.380+0000 7fdfcebc4640 1 -- 172.21.15.27:0/465255469 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfc80ccae0 con 0x7fdfc80d3790 2024-09-12T03:21:45.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:45.381+0000 7fdfcebc4640 1 -- 172.21.15.27:0/465255469 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdfc80cc8d0 con 0x7fdfc80d3790 2024-09-12T03:21:45.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:45.382+0000 7fdfc57fa640 1 -- 172.21.15.27:0/465255469 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 19) ==== 90855+0+0 (secure 0 0 0) 0x7fdfbc007400 con 0x7fdfc80d3790 2024-09-12T03:21:45.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:45.382+0000 7fdfc57fa640 1 --2- 172.21.15.27:0/465255469 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fdfb406c250 0x7fdfb406e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:45.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:45.382+0000 7fdfc57fa640 1 -- 172.21.15.27:0/465255469 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 5255+0+0 (secure 0 0 0) 0x7fdfbc08a8a0 con 0x7fdfc80d3790 2024-09-12T03:21:45.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:45.382+0000 7fdfcc960640 1 --2- 172.21.15.27:0/465255469 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fdfb406c250 0x7fdfb406e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:45.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:45.383+0000 7fdfcc960640 1 --2- 172.21.15.27:0/465255469 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fdfb406c250 0x7fdfb406e710 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fdfc815adf0 tx=0x7fdfb8008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:45.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:45.387+0000 7fdfc57fa640 1 -- 172.21.15.27:0/465255469 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fdfbc058e30 con 0x7fdfc80d3790 2024-09-12T03:21:45.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:45.562+0000 7fdfcebc4640 1 -- 172.21.15.27:0/465255469 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fdfc806e6e0 con 0x7fdfc80d3790 2024-09-12T03:21:46.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:46.239+0000 7fdfc57fa640 1 -- 172.21.15.27:0/465255469 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v112) ==== 170+0+0 (secure 0 0 0) 0x7fdfbc058730 con 0x7fdfc80d3790 2024-09-12T03:21:46.239 INFO:teuthology.orchestra.run.smithi027.stderr:pool 'cephfs_data' created 2024-09-12T03:21:46.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:46.241+0000 7fdfcebc4640 1 -- 172.21.15.27:0/465255469 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fdfb406c250 msgr2=0x7fdfb406e710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:46.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:46.241+0000 7fdfcebc4640 1 --2- 172.21.15.27:0/465255469 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fdfb406c250 0x7fdfb406e710 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fdfc815adf0 tx=0x7fdfb8008040 comp rx=0 tx=0).stop 2024-09-12T03:21:46.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:46.241+0000 7fdfcebc4640 1 -- 172.21.15.27:0/465255469 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdfc80d3790 msgr2=0x7fdfc80cd2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:46.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:46.241+0000 7fdfcebc4640 1 --2- 172.21.15.27:0/465255469 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdfc80d3790 0x7fdfc80cd2e0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7fdfbc00c3a0 tx=0x7fdfbc00c860 comp rx=0 tx=0).stop 2024-09-12T03:21:46.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:46.241+0000 7fdfcebc4640 1 -- 172.21.15.27:0/465255469 shutdown_connections 2024-09-12T03:21:46.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:46.241+0000 7fdfcebc4640 1 --2- 172.21.15.27:0/465255469 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fdfc8162040 0x7fdfc81f4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:46.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:46.241+0000 7fdfcebc4640 1 --2- 172.21.15.27:0/465255469 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fdfb406c250 0x7fdfb406e710 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:46.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:46.241+0000 7fdfcebc4640 1 --2- 172.21.15.27:0/465255469 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdfc80d3790 0x7fdfc80cd2e0 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:46.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:46.241+0000 7fdfcebc4640 1 --2- 172.21.15.27:0/465255469 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdfc806dfd0 0x7fdfc815bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:46.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:46.241+0000 7fdfcebc4640 1 -- 172.21.15.27:0/465255469 >> 172.21.15.27:0/465255469 conn(0x7fdfc80b6700 msgr2=0x7fdfc80b4b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:46.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:46.242+0000 7fdfcebc4640 1 -- 172.21.15.27:0/465255469 shutdown_connections 2024-09-12T03:21:46.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:46.242+0000 7fdfcebc4640 1 -- 172.21.15.27:0/465255469 wait complete. 2024-09-12T03:21:46.268 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-12T03:21:46.379 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:46.379+0000 7f39206ba640 1 Processor -- start 2024-09-12T03:21:46.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:46.379+0000 7f39206ba640 1 -- start start 2024-09-12T03:21:46.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:46.380+0000 7f39206ba640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f391806dfd0 0x7f39180d1da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:46.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:46.380+0000 7f39206ba640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f39180d3790 0x7f39180b7a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:46.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:46.380+0000 7f39206ba640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f39180d2e30 con 0x7f3918162040 2024-09-12T03:21:46.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:46.380+0000 7f39206ba640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f39180b7f90 con 0x7f39180d3790 2024-09-12T03:21:46.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:46.380+0000 7f39206ba640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f3918162420 con 0x7f391806dfd0 2024-09-12T03:21:46.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:46.381+0000 7f391dc55640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f391806dfd0 0x7f39180d1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:46.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:46.381+0000 7f391dc55640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f391806dfd0 0x7f39180d1da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3301/0 says I am v2:172.21.15.27:35994/0 (socket says 172.21.15.27:35994) 2024-09-12T03:21:46.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:46.381+0000 7f391dc55640 1 -- 172.21.15.27:0/462431804 learned_addr learned my addr 172.21.15.27:0/462431804 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:21:46.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:46.381+0000 7f391ec57640 1 --2- 172.21.15.27:0/462431804 >> v2:172.21.15.45:3300/0 conn(0x7f39180d3790 0x7f39180b7a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:46.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:46.381+0000 7f391dc55640 1 -- 172.21.15.27:0/462431804 >> v2:172.21.15.45:3300/0 conn(0x7f39180d3790 msgr2=0x7f39180b7a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:46.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:46.381+0000 7f391dc55640 1 --2- 172.21.15.27:0/462431804 >> v2:172.21.15.45:3300/0 conn(0x7f39180d3790 0x7f39180b7a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:46.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:46.381+0000 7f391dc55640 1 -- 172.21.15.27:0/462431804 >> v1:172.21.15.45:6789/0 conn(0x7f3918162040 legacy=0x7f391806d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:46.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:46.381+0000 7f391dc55640 1 -- 172.21.15.27:0/462431804 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39180d30d0 con 0x7f391806dfd0 2024-09-12T03:21:46.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:46.382+0000 7f391dc55640 1 --2- 172.21.15.27:0/462431804 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f391806dfd0 0x7f39180d1da0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f3908009870 tx=0x7f390802a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e7642f5aa734bf25 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:46.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:46.382+0000 7f391d454640 1 -- 172.21.15.27:0/462431804 <== mon.2 v2:172.21.15.45:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3908037070 con 0x7f391806dfd0 2024-09-12T03:21:46.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:46.382+0000 7f391d454640 1 -- 172.21.15.27:0/462431804 <== mon.2 v2:172.21.15.45:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3908032440 con 0x7f391806dfd0 2024-09-12T03:21:46.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:46.382+0000 7f391d454640 1 -- 172.21.15.27:0/462431804 <== mon.2 v2:172.21.15.45:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3908032810 con 0x7f391806dfd0 2024-09-12T03:21:46.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:46.383+0000 7f39206ba640 1 -- 172.21.15.27:0/462431804 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f391806dfd0 msgr2=0x7f39180d1da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:46.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:46.383+0000 7f39206ba640 1 --2- 172.21.15.27:0/462431804 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f391806dfd0 0x7f39180d1da0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f3908009870 tx=0x7f390802a270 comp rx=0 tx=0).stop 2024-09-12T03:21:46.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:46.383+0000 7f39206ba640 1 -- 172.21.15.27:0/462431804 shutdown_connections 2024-09-12T03:21:46.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:46.383+0000 7f39206ba640 1 --2- 172.21.15.27:0/462431804 >> v2:172.21.15.45:3300/0 conn(0x7f39180d3790 0x7f39180b7a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:46.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:46.383+0000 7f39206ba640 1 --2- 172.21.15.27:0/462431804 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f391806dfd0 0x7f39180d1da0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:46.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:46.383+0000 7f39206ba640 1 -- 172.21.15.27:0/462431804 >> 172.21.15.27:0/462431804 conn(0x7f39180b6700 msgr2=0x7f39180b6b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:46.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:46.383+0000 7f39206ba640 1 -- 172.21.15.27:0/462431804 shutdown_connections 2024-09-12T03:21:46.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:46.383+0000 7f39206ba640 1 -- 172.21.15.27:0/462431804 wait complete. 2024-09-12T03:21:46.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:46.384+0000 7f39206ba640 1 Processor -- start 2024-09-12T03:21:46.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:46.384+0000 7f39206ba640 1 -- start start 2024-09-12T03:21:46.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:46.385+0000 7f39206ba640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f391806dfd0 0x7f39181f3cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:46.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:46.385+0000 7f39206ba640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f39180d3790 0x7f39181f81f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:46.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:46.385+0000 7f39206ba640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f3918162040 0x7f39180b8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:46.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:46.385+0000 7f39206ba640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f39180d4530 con 0x7f391806dfd0 2024-09-12T03:21:46.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:46.385+0000 7f39206ba640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f39180cee00 con 0x7f39180d3790 2024-09-12T03:21:46.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:46.385+0000 7f39206ba640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f39180d4150 con 0x7f3918162040 2024-09-12T03:21:46.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:46.385+0000 7f391e456640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f391806dfd0 0x7f39181f3cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:46.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:46.385+0000 7f391e456640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f391806dfd0 0x7f39181f3cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:50502/0 (socket says 172.21.15.27:50502) 2024-09-12T03:21:46.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:46.385+0000 7f391e456640 1 -- 172.21.15.27:0/1071691265 learned_addr learned my addr 172.21.15.27:0/1071691265 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:21:46.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:46.385+0000 7f391ec57640 1 --2- 172.21.15.27:0/1071691265 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f3918162040 0x7f39180b8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:46.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:46.385+0000 7f391dc55640 1 --2- 172.21.15.27:0/1071691265 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f39180d3790 0x7f39181f81f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:46.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:46.385+0000 7f391e456640 1 -- 172.21.15.27:0/1071691265 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f3918162040 msgr2=0x7f39180b8ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:46.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:46.385+0000 7f391e456640 1 --2- 172.21.15.27:0/1071691265 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f3918162040 0x7f39180b8ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:46.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:46.385+0000 7f391e456640 1 -- 172.21.15.27:0/1071691265 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f39180d3790 msgr2=0x7f39181f81f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:46.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:46.385+0000 7f391e456640 1 --2- 172.21.15.27:0/1071691265 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f39180d3790 0x7f39181f81f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:46.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:46.385+0000 7f391e456640 1 -- 172.21.15.27:0/1071691265 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39181f7a10 con 0x7f391806dfd0 2024-09-12T03:21:46.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:46.386+0000 7f391e456640 1 --2- 172.21.15.27:0/1071691265 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f391806dfd0 0x7f39181f3cf0 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f391400c3c0 tx=0x7f391400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:46.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:46.386+0000 7f390effd640 1 -- 172.21.15.27:0/1071691265 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39140052d0 con 0x7f391806dfd0 2024-09-12T03:21:46.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:46.386+0000 7f390effd640 1 -- 172.21.15.27:0/1071691265 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3914005490 con 0x7f391806dfd0 2024-09-12T03:21:46.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:46.386+0000 7f39206ba640 1 -- 172.21.15.27:0/1071691265 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39181611f0 con 0x7f391806dfd0 2024-09-12T03:21:46.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:46.386+0000 7f390effd640 1 -- 172.21.15.27:0/1071691265 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3914007260 con 0x7f391806dfd0 2024-09-12T03:21:46.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:46.386+0000 7f39206ba640 1 -- 172.21.15.27:0/1071691265 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39181f7cd0 con 0x7f391806dfd0 2024-09-12T03:21:46.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:46.387+0000 7f39206ba640 1 -- 172.21.15.27:0/1071691265 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f391806e6e0 con 0x7f391806dfd0 2024-09-12T03:21:46.389 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:46.387+0000 7f390effd640 1 -- 172.21.15.27:0/1071691265 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 19) ==== 90855+0+0 (secure 0 0 0) 0x7f3914007400 con 0x7f391806dfd0 2024-09-12T03:21:46.389 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:46.388+0000 7f390effd640 1 --2- 172.21.15.27:0/1071691265 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f38fc06c220 0x7f38fc06e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:46.389 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:46.388+0000 7f390effd640 1 -- 172.21.15.27:0/1071691265 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 5637+0+0 (secure 0 0 0) 0x7f391408a970 con 0x7f391806dfd0 2024-09-12T03:21:46.389 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:46.388+0000 7f391dc55640 1 --2- 172.21.15.27:0/1071691265 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f38fc06c220 0x7f38fc06e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:46.389 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:46.388+0000 7f391dc55640 1 --2- 172.21.15.27:0/1071691265 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f38fc06c220 0x7f38fc06e6e0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f3908034000 tx=0x7f390802e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:46.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:46.392+0000 7f390effd640 1 -- 172.21.15.27:0/1071691265 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f3914058730 con 0x7f391806dfd0 2024-09-12T03:21:46.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:46.602+0000 7f39206ba640 1 -- 172.21.15.27:0/1071691265 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f39181f7f10 con 0x7f391806dfd0 2024-09-12T03:21:47.274 INFO:teuthology.orchestra.run.smithi027.stderr:new fs with metadata pool 18 and data pool 19 2024-09-12T03:21:47.277 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:47.274+0000 7f390effd640 1 -- 172.21.15.27:0/1071691265 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 18 and data pool 19 v64) ==== 174+0+0 (secure 0 0 0) 0x7f391405ca30 con 0x7f391806dfd0 2024-09-12T03:21:47.284 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:47.284+0000 7f39206ba640 1 -- 172.21.15.27:0/1071691265 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f38fc06c220 msgr2=0x7f38fc06e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:47.284 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:47.284+0000 7f39206ba640 1 --2- 172.21.15.27:0/1071691265 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f38fc06c220 0x7f38fc06e6e0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f3908034000 tx=0x7f390802e000 comp rx=0 tx=0).stop 2024-09-12T03:21:47.284 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:47.284+0000 7f39206ba640 1 -- 172.21.15.27:0/1071691265 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f391806dfd0 msgr2=0x7f39181f3cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:47.284 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:47.284+0000 7f39206ba640 1 --2- 172.21.15.27:0/1071691265 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f391806dfd0 0x7f39181f3cf0 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f391400c3c0 tx=0x7f391400c880 comp rx=0 tx=0).stop 2024-09-12T03:21:47.284 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:47.284+0000 7f39206ba640 1 -- 172.21.15.27:0/1071691265 shutdown_connections 2024-09-12T03:21:47.285 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:47.284+0000 7f39206ba640 1 --2- 172.21.15.27:0/1071691265 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f3918162040 0x7f39180b8ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:47.285 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:47.284+0000 7f39206ba640 1 --2- 172.21.15.27:0/1071691265 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f39180d3790 0x7f39181f81f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:47.285 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:47.284+0000 7f39206ba640 1 --2- 172.21.15.27:0/1071691265 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f38fc06c220 0x7f38fc06e6e0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:47.285 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:47.284+0000 7f39206ba640 1 --2- 172.21.15.27:0/1071691265 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f391806dfd0 0x7f39181f3cf0 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:47.285 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:47.284+0000 7f39206ba640 1 -- 172.21.15.27:0/1071691265 >> 172.21.15.27:0/1071691265 conn(0x7f39180b6700 msgr2=0x7f39180b8140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:47.287 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:47.286+0000 7f39206ba640 1 -- 172.21.15.27:0/1071691265 shutdown_connections 2024-09-12T03:21:47.287 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:47.287+0000 7f39206ba640 1 -- 172.21.15.27:0/1071691265 wait complete. 2024-09-12T03:21:47.302 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-12T03:21:47.410 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:47.410+0000 7f6313142640 1 Processor -- start 2024-09-12T03:21:47.410 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:47.410+0000 7f6313142640 1 -- start start 2024-09-12T03:21:47.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:47.411+0000 7f6313142640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f630c158cf0 0x7f630c0b55d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:47.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:47.411+0000 7f6313142640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f630c15f600 0x7f630c06bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:47.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:47.411+0000 7f6313142640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f630c162c10 con 0x7f630c1590d0 2024-09-12T03:21:47.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:47.411+0000 7f6313142640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f630c162ef0 con 0x7f630c15f600 2024-09-12T03:21:47.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:47.411+0000 7f6313142640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f630c1631d0 con 0x7f630c158cf0 2024-09-12T03:21:47.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:47.412+0000 7f6310ede640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f630c158cf0 0x7f630c0b55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:47.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:47.412+0000 7f63116df640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f630c15f600 0x7f630c06bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:47.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:47.412+0000 7f6310ede640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f630c158cf0 0x7f630c0b55d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3301/0 says I am v2:172.21.15.27:36014/0 (socket says 172.21.15.27:36014) 2024-09-12T03:21:47.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:47.412+0000 7f6310ede640 1 -- 172.21.15.27:0/3457355761 learned_addr learned my addr 172.21.15.27:0/3457355761 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:21:47.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:47.412+0000 7f6310ede640 1 -- 172.21.15.27:0/3457355761 >> v2:172.21.15.45:3300/0 conn(0x7f630c15f600 msgr2=0x7f630c06bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:47.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:47.412+0000 7f6310ede640 1 --2- 172.21.15.27:0/3457355761 >> v2:172.21.15.45:3300/0 conn(0x7f630c15f600 0x7f630c06bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:47.413 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:47.412+0000 7f6310ede640 1 -- 172.21.15.27:0/3457355761 >> v1:172.21.15.45:6789/0 conn(0x7f630c1590d0 legacy=0x7f630c162430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:47.413 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:47.412+0000 7f6310ede640 1 -- 172.21.15.27:0/3457355761 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f630c15edc0 con 0x7f630c158cf0 2024-09-12T03:21:47.413 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:47.412+0000 7f63116df640 1 --2- 172.21.15.27:0/3457355761 >> v2:172.21.15.45:3300/0 conn(0x7f630c15f600 0x7f630c06bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:21:47.414 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:47.413+0000 7f6310ede640 1 --2- 172.21.15.27:0/3457355761 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f630c158cf0 0x7f630c0b55d0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f62fc009870 tx=0x7f62fc02a130 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9169abef1b8960c9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:47.414 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:47.413+0000 7f630b7fe640 1 -- 172.21.15.27:0/3457355761 <== mon.2 v2:172.21.15.45:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62fc037070 con 0x7f630c158cf0 2024-09-12T03:21:47.414 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:47.413+0000 7f630b7fe640 1 -- 172.21.15.27:0/3457355761 <== mon.2 v2:172.21.15.45:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62fc032440 con 0x7f630c158cf0 2024-09-12T03:21:47.414 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:47.413+0000 7f630b7fe640 1 -- 172.21.15.27:0/3457355761 <== mon.2 v2:172.21.15.45:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62fc032810 con 0x7f630c158cf0 2024-09-12T03:21:47.414 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:47.414+0000 7f6313142640 1 -- 172.21.15.27:0/3457355761 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f630c158cf0 msgr2=0x7f630c0b55d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:47.414 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:47.414+0000 7f6313142640 1 --2- 172.21.15.27:0/3457355761 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f630c158cf0 0x7f630c0b55d0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f62fc009870 tx=0x7f62fc02a130 comp rx=0 tx=0).stop 2024-09-12T03:21:47.414 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:47.414+0000 7f6313142640 1 -- 172.21.15.27:0/3457355761 shutdown_connections 2024-09-12T03:21:47.414 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:47.414+0000 7f6313142640 1 --2- 172.21.15.27:0/3457355761 >> v2:172.21.15.45:3300/0 conn(0x7f630c15f600 0x7f630c06bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:47.415 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:47.414+0000 7f6313142640 1 --2- 172.21.15.27:0/3457355761 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f630c158cf0 0x7f630c0b55d0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:47.415 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:47.414+0000 7f6313142640 1 -- 172.21.15.27:0/3457355761 >> 172.21.15.27:0/3457355761 conn(0x7f630c06cc40 msgr2=0x7f630c06d050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:47.415 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:47.414+0000 7f6313142640 1 -- 172.21.15.27:0/3457355761 shutdown_connections 2024-09-12T03:21:47.415 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:47.414+0000 7f6313142640 1 -- 172.21.15.27:0/3457355761 wait complete. 2024-09-12T03:21:47.415 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:47.415+0000 7f6313142640 1 Processor -- start 2024-09-12T03:21:47.415 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:47.415+0000 7f6313142640 1 -- start start 2024-09-12T03:21:47.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:47.416+0000 7f6313142640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f630c158cf0 0x7f630c159500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:47.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:47.416+0000 7f6313142640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f630c1590d0 0x7f630c0b07f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:47.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:47.416+0000 7f6313142640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f630c15f600 0x7f630c15a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:47.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:47.416+0000 7f6313142640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f630c06c070 con 0x7f630c15f600 2024-09-12T03:21:47.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:47.416+0000 7f6313142640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f630c15fca0 con 0x7f630c1590d0 2024-09-12T03:21:47.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:47.416+0000 7f6313142640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f630c06c1f0 con 0x7f630c158cf0 2024-09-12T03:21:47.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:47.416+0000 7f63116df640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f630c15f600 0x7f630c15a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:47.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:47.416+0000 7f63116df640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f630c15f600 0x7f630c15a320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:50506/0 (socket says 172.21.15.27:50506) 2024-09-12T03:21:47.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:47.416+0000 7f63116df640 1 -- 172.21.15.27:0/3149163067 learned_addr learned my addr 172.21.15.27:0/3149163067 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:21:47.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:47.417+0000 7f6310ede640 1 --2- 172.21.15.27:0/3149163067 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f630c158cf0 0x7f630c159500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:47.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:47.417+0000 7f630bfff640 1 --2- 172.21.15.27:0/3149163067 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f630c1590d0 0x7f630c0b07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:47.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:47.417+0000 7f63116df640 1 -- 172.21.15.27:0/3149163067 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f630c158cf0 msgr2=0x7f630c159500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:47.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:47.417+0000 7f63116df640 1 --2- 172.21.15.27:0/3149163067 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f630c158cf0 0x7f630c159500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:47.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:47.417+0000 7f63116df640 1 -- 172.21.15.27:0/3149163067 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f630c1590d0 msgr2=0x7f630c0b07f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:47.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:47.417+0000 7f63116df640 1 --2- 172.21.15.27:0/3149163067 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f630c1590d0 0x7f630c0b07f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:47.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:47.417+0000 7f63116df640 1 -- 172.21.15.27:0/3149163067 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f630c0af750 con 0x7f630c15f600 2024-09-12T03:21:47.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:47.417+0000 7f63116df640 1 --2- 172.21.15.27:0/3149163067 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f630c15f600 0x7f630c15a320 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f630400efc0 tx=0x7f6304009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:47.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:47.417+0000 7f63097fa640 1 -- 172.21.15.27:0/3149163067 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6304018030 con 0x7f630c15f600 2024-09-12T03:21:47.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:47.417+0000 7f63097fa640 1 -- 172.21.15.27:0/3149163067 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f630400cde0 con 0x7f630c15f600 2024-09-12T03:21:47.419 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:47.417+0000 7f63097fa640 1 -- 172.21.15.27:0/3149163067 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6304003bb0 con 0x7f630c15f600 2024-09-12T03:21:47.419 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:47.417+0000 7f6313142640 1 -- 172.21.15.27:0/3149163067 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f630c0afa30 con 0x7f630c15f600 2024-09-12T03:21:47.419 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:47.417+0000 7f6313142640 1 -- 172.21.15.27:0/3149163067 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f630c0afff0 con 0x7f630c15f600 2024-09-12T03:21:47.419 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:47.418+0000 7f6313142640 1 -- 172.21.15.27:0/3149163067 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f630c06bb30 con 0x7f630c15f600 2024-09-12T03:21:47.420 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:47.419+0000 7f63097fa640 1 -- 172.21.15.27:0/3149163067 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 19) ==== 90855+0+0 (secure 0 0 0) 0x7f630401b070 con 0x7f630c15f600 2024-09-12T03:21:47.420 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:47.419+0000 7f63097fa640 1 --2- 172.21.15.27:0/3149163067 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f62f806c220 0x7f62f806e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:47.420 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:47.419+0000 7f63097fa640 1 -- 172.21.15.27:0/3149163067 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 5737+0+0 (secure 0 0 0) 0x7f630408c110 con 0x7f630c15f600 2024-09-12T03:21:47.420 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:47.419+0000 7f6310ede640 1 --2- 172.21.15.27:0/3149163067 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f62f806c220 0x7f62f806e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:47.420 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:47.420+0000 7f6310ede640 1 --2- 172.21.15.27:0/3149163067 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f62f806c220 0x7f62f806e6e0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f62fc034000 tx=0x7f62fc02e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:47.423 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:47.424+0000 7f63097fa640 1 -- 172.21.15.27:0/3149163067 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f6304010040 con 0x7f630c15f600 2024-09-12T03:21:47.598 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:47.598+0000 7f6313142640 1 -- 172.21.15.27:0/3149163067 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f630c1607d0 con 0x7f630c15f600 2024-09-12T03:21:47.599 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:47.599+0000 7f63097fa640 1 -- 172.21.15.27:0/3149163067 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v113) ==== 74+0+16748 (secure 0 0 0) 0x7f630405a3b0 con 0x7f630c15f600 2024-09-12T03:21:47.600 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:21:47.600 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":113,"fsid":"d2df504e-2647-4385-a5db-9fdf69f487a0","created":"2024-09-12T03:16:07.778937+0000","modified":"2024-09-12T03:21:46.603559+0000","last_up_change":"2024-09-12T03:16:19.314778+0000","last_in_change":"2024-09-12T03:16:09.893086+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T03:16:20.097646+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-09-12T03:21:44.263535+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-09-12T03:21:45.593648+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1851116c-1fb3-4dc9-9772-f90e2ef4c355","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6817","nonce":3002321488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6819","nonce":3002321488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6823","nonce":3002321488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6821","nonce":3002321488}]},"public_addr":"172.21.15.27:6817/3002321488","cluster_addr":"172.21.15.27:6819/3002321488","heartbeat_back_addr":"172.21.15.27:6823/3002321488","heartbeat_front_addr":"172.21.15.27:6821/3002321488","state":["exists","up"]},{"osd":1,"uuid":"f38ae17d-73eb-4204-b99d-975fa806fb87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6809","nonce":1613102886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6811","nonce":1613102886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6815","nonce":1613102886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6813","nonce":1613102886}]},"public_addr":"172.21.15.27:6809/1613102886","cluster_addr":"172.21.15.27:6811/1613102886","heartbeat_back_addr":"172.21.15.27:6815/1613102886","heartbeat_front_addr":"172.21.15.27:6813/1613102886","state":["exists","up"]},{"osd":2,"uuid":"fb12dcc8-bdd0-42b8-b46d-8b10106224bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6825","nonce":3534723355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6827","nonce":3534723355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6831","nonce":3534723355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6829","nonce":3534723355}]},"public_addr":"172.21.15.27:6825/3534723355","cluster_addr":"172.21.15.27:6827/3534723355","heartbeat_back_addr":"172.21.15.27:6831/3534723355","heartbeat_front_addr":"172.21.15.27:6829/3534723355","state":["exists","up"]},{"osd":3,"uuid":"d74d3f49-9c24-4b9f-98e2-3ecd7b5ef3b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6801","nonce":80661598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6803","nonce":80661598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6807","nonce":80661598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6805","nonce":80661598}]},"public_addr":"172.21.15.27:6801/80661598","cluster_addr":"172.21.15.27:6803/80661598","heartbeat_back_addr":"172.21.15.27:6807/80661598","heartbeat_front_addr":"172.21.15.27:6805/80661598","state":["exists","up"]},{"osd":4,"uuid":"2bf123f2-1b0d-41c4-aa3b-805bab267e59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6817","nonce":3943189827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6819","nonce":3943189827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6823","nonce":3943189827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6821","nonce":3943189827}]},"public_addr":"172.21.15.45:6817/3943189827","cluster_addr":"172.21.15.45:6819/3943189827","heartbeat_back_addr":"172.21.15.45:6823/3943189827","heartbeat_front_addr":"172.21.15.45:6821/3943189827","state":["exists","up"]},{"osd":5,"uuid":"c49ba5da-91c6-4a86-a97b-4aba89c253ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6809","nonce":3620760330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6811","nonce":3620760330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6815","nonce":3620760330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6813","nonce":3620760330}]},"public_addr":"172.21.15.45:6809/3620760330","cluster_addr":"172.21.15.45:6811/3620760330","heartbeat_back_addr":"172.21.15.45:6815/3620760330","heartbeat_front_addr":"172.21.15.45:6813/3620760330","state":["exists","up"]},{"osd":6,"uuid":"b0123475-89f7-4a34-9e33-d797b99dd9dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6801","nonce":1989083510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6803","nonce":1989083510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6807","nonce":1989083510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6805","nonce":1989083510}]},"public_addr":"172.21.15.45:6801/1989083510","cluster_addr":"172.21.15.45:6803/1989083510","heartbeat_back_addr":"172.21.15.45:6807/1989083510","heartbeat_front_addr":"172.21.15.45:6805/1989083510","state":["exists","up"]},{"osd":7,"uuid":"4dad773d-b7d8-46fc-8a10-a972aad27b29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6825","nonce":1513300294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6827","nonce":1513300294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6831","nonce":1513300294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6829","nonce":1513300294}]},"public_addr":"172.21.15.45:6825/1513300294","cluster_addr":"172.21.15.45:6827/1513300294","heartbeat_back_addr":"172.21.15.45:6831/1513300294","heartbeat_front_addr":"172.21.15.45:6829/1513300294","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.881855+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:17.133252+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.735339+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.500742+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.477571+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:17.167667+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.470291+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.873743+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T03:21:47.601 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:47.601+0000 7f6313142640 1 -- 172.21.15.27:0/3149163067 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f62f806c220 msgr2=0x7f62f806e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:47.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:47.601+0000 7f6313142640 1 --2- 172.21.15.27:0/3149163067 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f62f806c220 0x7f62f806e6e0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f62fc034000 tx=0x7f62fc02e000 comp rx=0 tx=0).stop 2024-09-12T03:21:47.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:47.601+0000 7f6313142640 1 -- 172.21.15.27:0/3149163067 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f630c15f600 msgr2=0x7f630c15a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:47.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:47.601+0000 7f6313142640 1 --2- 172.21.15.27:0/3149163067 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f630c15f600 0x7f630c15a320 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f630400efc0 tx=0x7f6304009f90 comp rx=0 tx=0).stop 2024-09-12T03:21:47.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:47.601+0000 7f6313142640 1 -- 172.21.15.27:0/3149163067 shutdown_connections 2024-09-12T03:21:47.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:47.601+0000 7f6313142640 1 --2- 172.21.15.27:0/3149163067 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f62f806c220 0x7f62f806e6e0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:47.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:47.601+0000 7f6313142640 1 --2- 172.21.15.27:0/3149163067 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f630c15f600 0x7f630c15a320 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:47.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:47.601+0000 7f6313142640 1 --2- 172.21.15.27:0/3149163067 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f630c1590d0 0x7f630c0b07f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:47.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:47.601+0000 7f6313142640 1 --2- 172.21.15.27:0/3149163067 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f630c158cf0 0x7f630c159500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:47.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:47.601+0000 7f6313142640 1 -- 172.21.15.27:0/3149163067 >> 172.21.15.27:0/3149163067 conn(0x7f630c06cc40 msgr2=0x7f630c1572d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:47.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:47.601+0000 7f6313142640 1 -- 172.21.15.27:0/3149163067 shutdown_connections 2024-09-12T03:21:47.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:47.602+0000 7f6313142640 1 -- 172.21.15.27:0/3149163067 wait complete. 2024-09-12T03:21:47.614 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-12T03:21:47.718 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:47.716+0000 7ff987a82640 1 Processor -- start 2024-09-12T03:21:47.718 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:47.717+0000 7ff987a82640 1 -- start start 2024-09-12T03:21:47.718 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:47.717+0000 7ff987a82640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ff9800b91a0 0x7ff980169880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:47.718 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:47.717+0000 7ff987a82640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff9800d3760 con 0x7ff9800d5200 2024-09-12T03:21:47.718 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:47.717+0000 7ff987a82640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff9800d3a40 con 0x7ff98015eb90 2024-09-12T03:21:47.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:47.717+0000 7ff987a82640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7ff9800b7d20 con 0x7ff9800b91a0 2024-09-12T03:21:47.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:47.718+0000 7ff98501d640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7ff9800d5200 0x7ff9800d1010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:34670/0 (socket says 172.21.15.27:34670) 2024-09-12T03:21:47.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:47.718+0000 7ff98501d640 1 -- 172.21.15.27:0/3897713323 learned_addr learned my addr 172.21.15.27:0/3897713323 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:21:47.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:47.718+0000 7ff98601f640 1 --2- 172.21.15.27:0/3897713323 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ff9800b91a0 0x7ff980169880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:47.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:47.718+0000 7ff98481c640 1 -- 172.21.15.27:0/3897713323 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3005382541 0 0) 0x7ff9800d3760 con 0x7ff9800d5200 2024-09-12T03:21:47.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:47.718+0000 7ff98481c640 1 -- 172.21.15.27:0/3897713323 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff968003520 con 0x7ff9800d5200 2024-09-12T03:21:47.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:47.718+0000 7ff98481c640 1 -- 172.21.15.27:0/3897713323 <== mon.1 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3325206001 0 0) 0x7ff9800d3a40 con 0x7ff98015eb90 2024-09-12T03:21:47.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:47.718+0000 7ff98481c640 1 -- 172.21.15.27:0/3897713323 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff9800d3760 con 0x7ff98015eb90 2024-09-12T03:21:47.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:47.718+0000 7ff98601f640 1 -- 172.21.15.27:0/3897713323 >> v1:172.21.15.45:6789/0 conn(0x7ff98015eb90 legacy=0x7ff9800d2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:47.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:47.718+0000 7ff98601f640 1 -- 172.21.15.27:0/3897713323 >> v1:172.21.15.27:6789/0 conn(0x7ff9800d5200 legacy=0x7ff9800d1010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:47.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:47.718+0000 7ff98601f640 1 -- 172.21.15.27:0/3897713323 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9800d3d20 con 0x7ff9800b91a0 2024-09-12T03:21:47.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:47.719+0000 7ff98601f640 1 --2- 172.21.15.27:0/3897713323 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ff9800b91a0 0x7ff980169880 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7ff970009dc0 tx=0x7ff97002a6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f8791ad6cbc19828 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:47.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:47.719+0000 7ff98481c640 1 -- 172.21.15.27:0/3897713323 <== mon.2 v2:172.21.15.45:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff970007740 con 0x7ff9800b91a0 2024-09-12T03:21:47.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:47.719+0000 7ff98481c640 1 -- 172.21.15.27:0/3897713323 <== mon.2 v2:172.21.15.45:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9700078e0 con 0x7ff9800b91a0 2024-09-12T03:21:47.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:47.719+0000 7ff98481c640 1 -- 172.21.15.27:0/3897713323 <== mon.2 v2:172.21.15.45:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff970007c90 con 0x7ff9800b91a0 2024-09-12T03:21:47.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:47.720+0000 7ff987a82640 1 -- 172.21.15.27:0/3897713323 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ff9800b91a0 msgr2=0x7ff980169880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:47.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:47.720+0000 7ff987a82640 1 --2- 172.21.15.27:0/3897713323 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ff9800b91a0 0x7ff980169880 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7ff970009dc0 tx=0x7ff97002a6d0 comp rx=0 tx=0).stop 2024-09-12T03:21:47.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:47.720+0000 7ff987a82640 1 -- 172.21.15.27:0/3897713323 shutdown_connections 2024-09-12T03:21:47.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:47.720+0000 7ff987a82640 1 --2- 172.21.15.27:0/3897713323 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ff9800b91a0 0x7ff980169880 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:47.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:47.720+0000 7ff987a82640 1 -- 172.21.15.27:0/3897713323 >> 172.21.15.27:0/3897713323 conn(0x7ff98006dce0 msgr2=0x7ff98006e0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:47.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:47.720+0000 7ff987a82640 1 -- 172.21.15.27:0/3897713323 shutdown_connections 2024-09-12T03:21:47.722 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:47.720+0000 7ff987a82640 1 -- 172.21.15.27:0/3897713323 wait complete. 2024-09-12T03:21:47.722 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:47.721+0000 7ff987a82640 1 Processor -- start 2024-09-12T03:21:47.722 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:47.721+0000 7ff987a82640 1 -- start start 2024-09-12T03:21:47.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:47.722+0000 7ff987a82640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff9800b91a0 0x7ff980149c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:47.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:47.722+0000 7ff987a82640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ff9800d5200 0x7ff9800c9da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:47.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:47.722+0000 7ff987a82640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff98015eb90 0x7ff9800cee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:47.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:47.722+0000 7ff987a82640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7ff9800d3500 con 0x7ff9800b91a0 2024-09-12T03:21:47.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:47.722+0000 7ff987a82640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ff9800d42e0 con 0x7ff98015eb90 2024-09-12T03:21:47.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:47.722+0000 7ff987a82640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7ff98015ef70 con 0x7ff9800d5200 2024-09-12T03:21:47.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:47.722+0000 7ff98581e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff9800b91a0 0x7ff980149c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:47.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:47.722+0000 7ff98581e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff9800b91a0 0x7ff980149c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:50520/0 (socket says 172.21.15.27:50520) 2024-09-12T03:21:47.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:47.722+0000 7ff98581e640 1 -- 172.21.15.27:0/342745824 learned_addr learned my addr 172.21.15.27:0/342745824 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:21:47.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:47.722+0000 7ff98581e640 1 -- 172.21.15.27:0/342745824 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ff9800d5200 msgr2=0x7ff9800c9da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:47.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:47.722+0000 7ff98581e640 1 --2- 172.21.15.27:0/342745824 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ff9800d5200 0x7ff9800c9da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:47.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:47.722+0000 7ff98581e640 1 -- 172.21.15.27:0/342745824 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff98015eb90 msgr2=0x7ff9800cee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:47.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:47.722+0000 7ff98581e640 1 --2- 172.21.15.27:0/342745824 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff98015eb90 0x7ff9800cee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:47.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:47.722+0000 7ff98581e640 1 -- 172.21.15.27:0/342745824 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9800ca6a0 con 0x7ff9800b91a0 2024-09-12T03:21:47.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:47.722+0000 7ff98581e640 1 --2- 172.21.15.27:0/342745824 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff9800b91a0 0x7ff980149c00 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7ff97800c430 tx=0x7ff97800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:47.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:47.722+0000 7ff97e7fc640 1 -- 172.21.15.27:0/342745824 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff978017070 con 0x7ff9800b91a0 2024-09-12T03:21:47.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:47.722+0000 7ff97e7fc640 1 -- 172.21.15.27:0/342745824 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9780052f0 con 0x7ff9800b91a0 2024-09-12T03:21:47.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:47.722+0000 7ff987a82640 1 -- 172.21.15.27:0/342745824 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9800ca980 con 0x7ff9800b91a0 2024-09-12T03:21:47.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:47.722+0000 7ff97e7fc640 1 -- 172.21.15.27:0/342745824 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff978003830 con 0x7ff9800b91a0 2024-09-12T03:21:47.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:47.723+0000 7ff987a82640 1 -- 172.21.15.27:0/342745824 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff980158650 con 0x7ff9800b91a0 2024-09-12T03:21:47.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:47.723+0000 7ff987a82640 1 -- 172.21.15.27:0/342745824 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff980158850 con 0x7ff9800b91a0 2024-09-12T03:21:47.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:47.724+0000 7ff97e7fc640 1 -- 172.21.15.27:0/342745824 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 19) ==== 90855+0+0 (secure 0 0 0) 0x7ff97801a070 con 0x7ff9800b91a0 2024-09-12T03:21:47.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:47.725+0000 7ff97e7fc640 1 --2- 172.21.15.27:0/342745824 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7ff96c06c250 0x7ff96c06e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:47.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:47.725+0000 7ff98501d640 1 --2- 172.21.15.27:0/342745824 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7ff96c06c250 0x7ff96c06e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:47.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:47.725+0000 7ff97e7fc640 1 -- 172.21.15.27:0/342745824 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 5737+0+0 (secure 0 0 0) 0x7ff97808a9e0 con 0x7ff9800b91a0 2024-09-12T03:21:47.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:47.725+0000 7ff98501d640 1 --2- 172.21.15.27:0/342745824 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7ff96c06c250 0x7ff96c06e710 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7ff9800b99c0 tx=0x7ff974007930 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:47.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:47.729+0000 7ff97e7fc640 1 -- 172.21.15.27:0/342745824 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7ff980158850 con 0x7ff9800b91a0 2024-09-12T03:21:47.906 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:47.906+0000 7ff987a82640 1 -- 172.21.15.27:0/342745824 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff9800d17b0 con 0x7ff9800b91a0 2024-09-12T03:21:47.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:47.907+0000 7ff97e7fc640 1 -- 172.21.15.27:0/342745824 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v113) ==== 74+0+16748 (secure 0 0 0) 0x7ff978058d10 con 0x7ff9800b91a0 2024-09-12T03:21:47.907 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:21:47.908 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":113,"fsid":"d2df504e-2647-4385-a5db-9fdf69f487a0","created":"2024-09-12T03:16:07.778937+0000","modified":"2024-09-12T03:21:46.603559+0000","last_up_change":"2024-09-12T03:16:19.314778+0000","last_in_change":"2024-09-12T03:16:09.893086+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T03:16:20.097646+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-09-12T03:21:44.263535+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-09-12T03:21:45.593648+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1851116c-1fb3-4dc9-9772-f90e2ef4c355","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6817","nonce":3002321488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6819","nonce":3002321488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6823","nonce":3002321488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6821","nonce":3002321488}]},"public_addr":"172.21.15.27:6817/3002321488","cluster_addr":"172.21.15.27:6819/3002321488","heartbeat_back_addr":"172.21.15.27:6823/3002321488","heartbeat_front_addr":"172.21.15.27:6821/3002321488","state":["exists","up"]},{"osd":1,"uuid":"f38ae17d-73eb-4204-b99d-975fa806fb87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6809","nonce":1613102886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6811","nonce":1613102886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6815","nonce":1613102886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6813","nonce":1613102886}]},"public_addr":"172.21.15.27:6809/1613102886","cluster_addr":"172.21.15.27:6811/1613102886","heartbeat_back_addr":"172.21.15.27:6815/1613102886","heartbeat_front_addr":"172.21.15.27:6813/1613102886","state":["exists","up"]},{"osd":2,"uuid":"fb12dcc8-bdd0-42b8-b46d-8b10106224bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6825","nonce":3534723355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6827","nonce":3534723355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6831","nonce":3534723355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6829","nonce":3534723355}]},"public_addr":"172.21.15.27:6825/3534723355","cluster_addr":"172.21.15.27:6827/3534723355","heartbeat_back_addr":"172.21.15.27:6831/3534723355","heartbeat_front_addr":"172.21.15.27:6829/3534723355","state":["exists","up"]},{"osd":3,"uuid":"d74d3f49-9c24-4b9f-98e2-3ecd7b5ef3b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6801","nonce":80661598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6803","nonce":80661598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6807","nonce":80661598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6805","nonce":80661598}]},"public_addr":"172.21.15.27:6801/80661598","cluster_addr":"172.21.15.27:6803/80661598","heartbeat_back_addr":"172.21.15.27:6807/80661598","heartbeat_front_addr":"172.21.15.27:6805/80661598","state":["exists","up"]},{"osd":4,"uuid":"2bf123f2-1b0d-41c4-aa3b-805bab267e59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6817","nonce":3943189827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6819","nonce":3943189827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6823","nonce":3943189827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6821","nonce":3943189827}]},"public_addr":"172.21.15.45:6817/3943189827","cluster_addr":"172.21.15.45:6819/3943189827","heartbeat_back_addr":"172.21.15.45:6823/3943189827","heartbeat_front_addr":"172.21.15.45:6821/3943189827","state":["exists","up"]},{"osd":5,"uuid":"c49ba5da-91c6-4a86-a97b-4aba89c253ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6809","nonce":3620760330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6811","nonce":3620760330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6815","nonce":3620760330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6813","nonce":3620760330}]},"public_addr":"172.21.15.45:6809/3620760330","cluster_addr":"172.21.15.45:6811/3620760330","heartbeat_back_addr":"172.21.15.45:6815/3620760330","heartbeat_front_addr":"172.21.15.45:6813/3620760330","state":["exists","up"]},{"osd":6,"uuid":"b0123475-89f7-4a34-9e33-d797b99dd9dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6801","nonce":1989083510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6803","nonce":1989083510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6807","nonce":1989083510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6805","nonce":1989083510}]},"public_addr":"172.21.15.45:6801/1989083510","cluster_addr":"172.21.15.45:6803/1989083510","heartbeat_back_addr":"172.21.15.45:6807/1989083510","heartbeat_front_addr":"172.21.15.45:6805/1989083510","state":["exists","up"]},{"osd":7,"uuid":"4dad773d-b7d8-46fc-8a10-a972aad27b29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6825","nonce":1513300294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6827","nonce":1513300294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6831","nonce":1513300294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6829","nonce":1513300294}]},"public_addr":"172.21.15.45:6825/1513300294","cluster_addr":"172.21.15.45:6827/1513300294","heartbeat_back_addr":"172.21.15.45:6831/1513300294","heartbeat_front_addr":"172.21.15.45:6829/1513300294","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.881855+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:17.133252+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.735339+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.500742+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.477571+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:17.167667+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.470291+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.873743+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T03:21:47.910 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:47.909+0000 7ff987a82640 1 -- 172.21.15.27:0/342745824 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7ff96c06c250 msgr2=0x7ff96c06e710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:47.910 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:47.909+0000 7ff987a82640 1 --2- 172.21.15.27:0/342745824 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7ff96c06c250 0x7ff96c06e710 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7ff9800b99c0 tx=0x7ff974007930 comp rx=0 tx=0).stop 2024-09-12T03:21:47.910 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:47.909+0000 7ff987a82640 1 -- 172.21.15.27:0/342745824 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff9800b91a0 msgr2=0x7ff980149c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:47.910 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:47.909+0000 7ff987a82640 1 --2- 172.21.15.27:0/342745824 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff9800b91a0 0x7ff980149c00 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7ff97800c430 tx=0x7ff97800c8f0 comp rx=0 tx=0).stop 2024-09-12T03:21:47.910 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:47.909+0000 7ff987a82640 1 -- 172.21.15.27:0/342745824 shutdown_connections 2024-09-12T03:21:47.910 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:47.909+0000 7ff987a82640 1 --2- 172.21.15.27:0/342745824 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff98015eb90 0x7ff9800cee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:47.910 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:47.909+0000 7ff987a82640 1 --2- 172.21.15.27:0/342745824 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ff9800d5200 0x7ff9800c9da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:47.910 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:47.909+0000 7ff987a82640 1 --2- 172.21.15.27:0/342745824 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7ff96c06c250 0x7ff96c06e710 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:47.910 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:47.909+0000 7ff987a82640 1 --2- 172.21.15.27:0/342745824 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff9800b91a0 0x7ff980149c00 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:47.910 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:47.909+0000 7ff987a82640 1 -- 172.21.15.27:0/342745824 >> 172.21.15.27:0/342745824 conn(0x7ff98006dce0 msgr2=0x7ff9800d1010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:47.910 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:47.909+0000 7ff987a82640 1 -- 172.21.15.27:0/342745824 shutdown_connections 2024-09-12T03:21:47.911 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:47.910+0000 7ff987a82640 1 -- 172.21.15.27:0/342745824 wait complete. 2024-09-12T03:21:47.921 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-12T03:21:48.026 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:48.026+0000 7f4c63503640 1 Processor -- start 2024-09-12T03:21:48.026 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:48.026+0000 7f4c63503640 1 -- start start 2024-09-12T03:21:48.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:48.027+0000 7f4c63503640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4c5c15e650 0x7f4c5c0d1d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:48.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:48.027+0000 7f4c63503640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f4c5c06cc40 0x7f4c5c06d060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:48.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:48.027+0000 7f4c63503640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4c5c0d2280 con 0x7f4c5c06db30 2024-09-12T03:21:48.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:48.027+0000 7f4c63503640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f4c5c06d5a0 con 0x7f4c5c06cc40 2024-09-12T03:21:48.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:48.027+0000 7f4c63503640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f4c5c06e670 con 0x7f4c5c15e650 2024-09-12T03:21:48.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:48.027+0000 7f4c61aa0640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f4c5c06db30 0x7f4c5c06df60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:34686/0 (socket says 172.21.15.27:34686) 2024-09-12T03:21:48.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:48.027+0000 7f4c61aa0640 1 -- 172.21.15.27:0/825227660 learned_addr learned my addr 172.21.15.27:0/825227660 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:21:48.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:48.028+0000 7f4c6129f640 1 --2- 172.21.15.27:0/825227660 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4c5c15e650 0x7f4c5c0d1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:48.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:48.028+0000 7f4c60a9e640 1 --2- 172.21.15.27:0/825227660 >> v2:172.21.15.45:3300/0 conn(0x7f4c5c06cc40 0x7f4c5c06d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:48.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:48.028+0000 7f4c5bfff640 1 -- 172.21.15.27:0/825227660 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 752234895 0 0) 0x7f4c5c0d2280 con 0x7f4c5c06db30 2024-09-12T03:21:48.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:48.028+0000 7f4c5bfff640 1 -- 172.21.15.27:0/825227660 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4c44003520 con 0x7f4c5c06db30 2024-09-12T03:21:48.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:48.028+0000 7f4c6129f640 1 -- 172.21.15.27:0/825227660 >> v2:172.21.15.45:3300/0 conn(0x7f4c5c06cc40 msgr2=0x7f4c5c06d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:48.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:48.028+0000 7f4c6129f640 1 --2- 172.21.15.27:0/825227660 >> v2:172.21.15.45:3300/0 conn(0x7f4c5c06cc40 0x7f4c5c06d060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:48.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:48.028+0000 7f4c6129f640 1 -- 172.21.15.27:0/825227660 >> v1:172.21.15.27:6789/0 conn(0x7f4c5c06db30 legacy=0x7f4c5c06df60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:48.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:48.028+0000 7f4c6129f640 1 -- 172.21.15.27:0/825227660 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c5c06d850 con 0x7f4c5c15e650 2024-09-12T03:21:48.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:48.028+0000 7f4c60a9e640 1 --2- 172.21.15.27:0/825227660 >> v2:172.21.15.45:3300/0 conn(0x7f4c5c06cc40 0x7f4c5c06d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:21:48.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:48.029+0000 7f4c6129f640 1 --2- 172.21.15.27:0/825227660 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4c5c15e650 0x7f4c5c0d1d40 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f4c4c009d90 tx=0x7f4c4c02a5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=24981aaf699fec6b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:48.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:48.029+0000 7f4c5bfff640 1 -- 172.21.15.27:0/825227660 <== mon.2 v2:172.21.15.45:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c4c031600 con 0x7f4c5c15e650 2024-09-12T03:21:48.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:48.029+0000 7f4c5bfff640 1 -- 172.21.15.27:0/825227660 <== mon.2 v2:172.21.15.45:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c4c0317a0 con 0x7f4c5c15e650 2024-09-12T03:21:48.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:48.029+0000 7f4c5bfff640 1 -- 172.21.15.27:0/825227660 <== mon.2 v2:172.21.15.45:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c4c031b50 con 0x7f4c5c15e650 2024-09-12T03:21:48.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:48.029+0000 7f4c63503640 1 -- 172.21.15.27:0/825227660 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4c5c15e650 msgr2=0x7f4c5c0d1d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:48.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:48.029+0000 7f4c63503640 1 --2- 172.21.15.27:0/825227660 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4c5c15e650 0x7f4c5c0d1d40 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f4c4c009d90 tx=0x7f4c4c02a5d0 comp rx=0 tx=0).stop 2024-09-12T03:21:48.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:48.030+0000 7f4c63503640 1 -- 172.21.15.27:0/825227660 shutdown_connections 2024-09-12T03:21:48.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:48.030+0000 7f4c63503640 1 --2- 172.21.15.27:0/825227660 >> v2:172.21.15.45:3300/0 conn(0x7f4c5c06cc40 0x7f4c5c06d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:48.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:48.030+0000 7f4c63503640 1 --2- 172.21.15.27:0/825227660 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4c5c15e650 0x7f4c5c0d1d40 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:48.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:48.030+0000 7f4c63503640 1 -- 172.21.15.27:0/825227660 >> 172.21.15.27:0/825227660 conn(0x7f4c5c06f320 msgr2=0x7f4c5c06f700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:48.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:48.030+0000 7f4c63503640 1 -- 172.21.15.27:0/825227660 shutdown_connections 2024-09-12T03:21:48.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:48.030+0000 7f4c63503640 1 -- 172.21.15.27:0/825227660 wait complete. 2024-09-12T03:21:48.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:48.031+0000 7f4c63503640 1 Processor -- start 2024-09-12T03:21:48.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:48.031+0000 7f4c63503640 1 -- start start 2024-09-12T03:21:48.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:48.032+0000 7f4c63503640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4c5c06cc40 0x7f4c5c14d770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:48.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:48.032+0000 7f4c63503640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4c5c06db30 0x7f4c5c160fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:48.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:48.032+0000 7f4c63503640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4c5c15e650 0x7f4c5c161510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:48.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:48.032+0000 7f4c63503640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f4c5c15ea30 con 0x7f4c5c15e650 2024-09-12T03:21:48.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:48.032+0000 7f4c63503640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f4c5c0d1540 con 0x7f4c5c06cc40 2024-09-12T03:21:48.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:48.032+0000 7f4c63503640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f4c5c15ebb0 con 0x7f4c5c06db30 2024-09-12T03:21:48.037 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:48.032+0000 7f4c61aa0640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4c5c15e650 0x7f4c5c161510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:48.037 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:48.032+0000 7f4c61aa0640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4c5c15e650 0x7f4c5c161510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:50536/0 (socket says 172.21.15.27:50536) 2024-09-12T03:21:48.037 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:48.032+0000 7f4c61aa0640 1 -- 172.21.15.27:0/2449262496 learned_addr learned my addr 172.21.15.27:0/2449262496 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:21:48.037 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:48.032+0000 7f4c6129f640 1 --2- 172.21.15.27:0/2449262496 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4c5c06cc40 0x7f4c5c14d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:48.037 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:48.032+0000 7f4c60a9e640 1 --2- 172.21.15.27:0/2449262496 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4c5c06db30 0x7f4c5c160fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:48.037 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:48.032+0000 7f4c61aa0640 1 -- 172.21.15.27:0/2449262496 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4c5c06db30 msgr2=0x7f4c5c160fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:48.037 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:48.032+0000 7f4c61aa0640 1 --2- 172.21.15.27:0/2449262496 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4c5c06db30 0x7f4c5c160fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:48.037 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:48.032+0000 7f4c61aa0640 1 -- 172.21.15.27:0/2449262496 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4c5c06cc40 msgr2=0x7f4c5c14d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:48.037 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:48.032+0000 7f4c61aa0640 1 --2- 172.21.15.27:0/2449262496 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4c5c06cc40 0x7f4c5c14d770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:48.037 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:48.032+0000 7f4c61aa0640 1 -- 172.21.15.27:0/2449262496 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c5c0ce9f0 con 0x7f4c5c15e650 2024-09-12T03:21:48.037 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:48.032+0000 7f4c6129f640 1 --2- 172.21.15.27:0/2449262496 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4c5c06cc40 0x7f4c5c14d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:21:48.037 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:48.032+0000 7f4c61aa0640 1 --2- 172.21.15.27:0/2449262496 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4c5c15e650 0x7f4c5c161510 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f4c5400c3f0 tx=0x7f4c5400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:48.037 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:48.032+0000 7f4c59ffb640 1 -- 172.21.15.27:0/2449262496 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c54017070 con 0x7f4c5c15e650 2024-09-12T03:21:48.038 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:48.032+0000 7f4c59ffb640 1 -- 172.21.15.27:0/2449262496 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c54005ce0 con 0x7f4c5c15e650 2024-09-12T03:21:48.038 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:48.032+0000 7f4c63503640 1 -- 172.21.15.27:0/2449262496 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c5c0cecd0 con 0x7f4c5c15e650 2024-09-12T03:21:48.038 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:48.032+0000 7f4c59ffb640 1 -- 172.21.15.27:0/2449262496 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c54007310 con 0x7f4c5c15e650 2024-09-12T03:21:48.038 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:48.033+0000 7f4c63503640 1 -- 172.21.15.27:0/2449262496 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c5c0cf290 con 0x7f4c5c15e650 2024-09-12T03:21:48.038 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:48.034+0000 7f4c63503640 1 -- 172.21.15.27:0/2449262496 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c5c161f60 con 0x7f4c5c15e650 2024-09-12T03:21:48.038 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:48.034+0000 7f4c59ffb640 1 -- 172.21.15.27:0/2449262496 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 19) ==== 90855+0+0 (secure 0 0 0) 0x7f4c5401a070 con 0x7f4c5c15e650 2024-09-12T03:21:48.038 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:48.035+0000 7f4c59ffb640 1 --2- 172.21.15.27:0/2449262496 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f4c4806c220 0x7f4c4806e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:48.038 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:48.035+0000 7f4c6129f640 1 --2- 172.21.15.27:0/2449262496 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f4c4806c220 0x7f4c4806e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:48.038 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:48.035+0000 7f4c59ffb640 1 -- 172.21.15.27:0/2449262496 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 5737+0+0 (secure 0 0 0) 0x7f4c5408a950 con 0x7f4c5c15e650 2024-09-12T03:21:48.038 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:48.035+0000 7f4c6129f640 1 --2- 172.21.15.27:0/2449262496 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f4c4806c220 0x7f4c4806e6e0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f4c4c034000 tx=0x7f4c4c02f000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:48.039 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:48.039+0000 7f4c59ffb640 1 -- 172.21.15.27:0/2449262496 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f4c5c161f60 con 0x7f4c5c15e650 2024-09-12T03:21:48.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:48.252+0000 7f4c63503640 1 -- 172.21.15.27:0/2449262496 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f4c5c06d060 con 0x7f4c5c15e650 2024-09-12T03:21:48.611 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:48.611+0000 7f4c59ffb640 1 -- 172.21.15.27:0/2449262496 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v66) ==== 118+0+0 (secure 0 0 0) 0x7f4c54058d00 con 0x7f4c5c15e650 2024-09-12T03:21:48.614 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:48.614+0000 7f4c63503640 1 -- 172.21.15.27:0/2449262496 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f4c4806c220 msgr2=0x7f4c4806e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:48.614 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:48.614+0000 7f4c63503640 1 --2- 172.21.15.27:0/2449262496 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f4c4806c220 0x7f4c4806e6e0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f4c4c034000 tx=0x7f4c4c02f000 comp rx=0 tx=0).stop 2024-09-12T03:21:48.614 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:48.614+0000 7f4c63503640 1 -- 172.21.15.27:0/2449262496 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4c5c15e650 msgr2=0x7f4c5c161510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:48.614 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:48.614+0000 7f4c63503640 1 --2- 172.21.15.27:0/2449262496 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4c5c15e650 0x7f4c5c161510 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f4c5400c3f0 tx=0x7f4c5400c8b0 comp rx=0 tx=0).stop 2024-09-12T03:21:48.615 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:48.614+0000 7f4c63503640 1 -- 172.21.15.27:0/2449262496 shutdown_connections 2024-09-12T03:21:48.615 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:48.614+0000 7f4c63503640 1 --2- 172.21.15.27:0/2449262496 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f4c4806c220 0x7f4c4806e6e0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:48.615 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:48.614+0000 7f4c63503640 1 --2- 172.21.15.27:0/2449262496 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4c5c15e650 0x7f4c5c161510 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:48.615 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:48.614+0000 7f4c63503640 1 --2- 172.21.15.27:0/2449262496 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4c5c06db30 0x7f4c5c160fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:48.615 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:48.614+0000 7f4c63503640 1 --2- 172.21.15.27:0/2449262496 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4c5c06cc40 0x7f4c5c14d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:48.615 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:48.614+0000 7f4c63503640 1 -- 172.21.15.27:0/2449262496 >> 172.21.15.27:0/2449262496 conn(0x7f4c5c06f320 msgr2=0x7f4c5c0b3880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:48.615 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:48.614+0000 7f4c63503640 1 -- 172.21.15.27:0/2449262496 shutdown_connections 2024-09-12T03:21:48.615 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:48.614+0000 7f4c63503640 1 -- 172.21.15.27:0/2449262496 wait complete. 2024-09-12T03:21:48.625 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-12T03:21:48.626 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-12T03:21:48.733 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:48.733+0000 7fc1aba30640 1 Processor -- start 2024-09-12T03:21:48.733 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:48.733+0000 7fc1aba30640 1 -- start start 2024-09-12T03:21:48.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:48.734+0000 7fc1aba30640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fc1a415e650 0x7fc1a40d1d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:48.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:48.734+0000 7fc1aba30640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc1a406db30 0x7fc1a406df60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:48.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:48.734+0000 7fc1aba30640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc1a40d2280 con 0x7fc1a406cc40 2024-09-12T03:21:48.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:48.734+0000 7fc1aba30640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7fc1a406d850 con 0x7fc1a415e650 2024-09-12T03:21:48.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:48.734+0000 7fc1aba30640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fc1a40d1260 con 0x7fc1a406db30 2024-09-12T03:21:48.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:48.734+0000 7fc1a97cc640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fc1a415e650 0x7fc1a40d1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:48.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:48.734+0000 7fc1a97cc640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fc1a415e650 0x7fc1a40d1d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:50544/0 (socket says 172.21.15.27:50544) 2024-09-12T03:21:48.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:48.734+0000 7fc1a97cc640 1 -- 172.21.15.27:0/3616558317 learned_addr learned my addr 172.21.15.27:0/3616558317 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:21:48.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:48.734+0000 7fc1a9fcd640 1 --2- 172.21.15.27:0/3616558317 >> v2:172.21.15.45:3300/0 conn(0x7fc1a406db30 0x7fc1a406df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:48.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:48.735+0000 7fc1a97cc640 1 -- 172.21.15.27:0/3616558317 >> v2:172.21.15.45:3300/0 conn(0x7fc1a406db30 msgr2=0x7fc1a406df60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:48.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:48.735+0000 7fc1a97cc640 1 --2- 172.21.15.27:0/3616558317 >> v2:172.21.15.45:3300/0 conn(0x7fc1a406db30 0x7fc1a406df60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:48.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:48.735+0000 7fc1a97cc640 1 -- 172.21.15.27:0/3616558317 >> v1:172.21.15.45:6789/0 conn(0x7fc1a406cc40 legacy=0x7fc1a406d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:48.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:48.735+0000 7fc1a97cc640 1 -- 172.21.15.27:0/3616558317 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1a40d1540 con 0x7fc1a415e650 2024-09-12T03:21:48.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:48.735+0000 7fc1a97cc640 1 --2- 172.21.15.27:0/3616558317 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc1a415e650 0x7fc1a40d1d40 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7fc1a41621b0 tx=0x7fc19402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd821496f8269bf0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:48.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:48.735+0000 7fc1a3fff640 1 -- 172.21.15.27:0/3616558317 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc194036070 con 0x7fc1a415e650 2024-09-12T03:21:48.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:48.735+0000 7fc1a3fff640 1 -- 172.21.15.27:0/3616558317 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc194031600 con 0x7fc1a415e650 2024-09-12T03:21:48.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:48.735+0000 7fc1a3fff640 1 -- 172.21.15.27:0/3616558317 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1940319d0 con 0x7fc1a415e650 2024-09-12T03:21:48.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:48.736+0000 7fc1aba30640 1 -- 172.21.15.27:0/3616558317 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc1a415e650 msgr2=0x7fc1a40d1d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:48.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:48.736+0000 7fc1aba30640 1 --2- 172.21.15.27:0/3616558317 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc1a415e650 0x7fc1a40d1d40 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7fc1a41621b0 tx=0x7fc19402a6a0 comp rx=0 tx=0).stop 2024-09-12T03:21:48.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:48.736+0000 7fc1aba30640 1 -- 172.21.15.27:0/3616558317 shutdown_connections 2024-09-12T03:21:48.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:48.736+0000 7fc1aba30640 1 --2- 172.21.15.27:0/3616558317 >> v2:172.21.15.45:3300/0 conn(0x7fc1a406db30 0x7fc1a406df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:48.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:48.736+0000 7fc1aba30640 1 --2- 172.21.15.27:0/3616558317 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc1a415e650 0x7fc1a40d1d40 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:48.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:48.736+0000 7fc1aba30640 1 -- 172.21.15.27:0/3616558317 >> 172.21.15.27:0/3616558317 conn(0x7fc1a406f320 msgr2=0x7fc1a406f700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:48.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:48.736+0000 7fc1aba30640 1 -- 172.21.15.27:0/3616558317 shutdown_connections 2024-09-12T03:21:48.738 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:48.736+0000 7fc1aba30640 1 -- 172.21.15.27:0/3616558317 wait complete. 2024-09-12T03:21:48.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:48.737+0000 7fc1aba30640 1 Processor -- start 2024-09-12T03:21:48.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:48.737+0000 7fc1aba30640 1 -- start start 2024-09-12T03:21:48.740 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:48.738+0000 7fc1aba30640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fc1a406cc40 0x7fc1a41542b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:48.740 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:48.738+0000 7fc1aba30640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc1a406db30 0x7fc1a4159f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:48.740 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:48.738+0000 7fc1aba30640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc1a415e650 0x7fc1a415a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:48.740 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:48.738+0000 7fc1aba30640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fc1a406e620 con 0x7fc1a415e650 2024-09-12T03:21:48.740 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:48.738+0000 7fc1aba30640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc1a406e4a0 con 0x7fc1a406db30 2024-09-12T03:21:48.740 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:48.738+0000 7fc1aba30640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fc1a40b55b0 con 0x7fc1a406cc40 2024-09-12T03:21:48.740 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:48.738+0000 7fc1a9fcd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc1a415e650 0x7fc1a415a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:48.740 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:48.738+0000 7fc1a9fcd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc1a415e650 0x7fc1a415a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:50548/0 (socket says 172.21.15.27:50548) 2024-09-12T03:21:48.740 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:48.738+0000 7fc1a9fcd640 1 -- 172.21.15.27:0/1761538276 learned_addr learned my addr 172.21.15.27:0/1761538276 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:21:48.740 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:48.738+0000 7fc1a97cc640 1 --2- 172.21.15.27:0/1761538276 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fc1a406cc40 0x7fc1a41542b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:48.740 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:48.738+0000 7fc1a8fcb640 1 --2- 172.21.15.27:0/1761538276 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc1a406db30 0x7fc1a4159f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:48.740 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:48.739+0000 7fc1a9fcd640 1 -- 172.21.15.27:0/1761538276 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fc1a406cc40 msgr2=0x7fc1a41542b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:48.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:48.739+0000 7fc1a9fcd640 1 --2- 172.21.15.27:0/1761538276 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fc1a406cc40 0x7fc1a41542b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:48.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:48.739+0000 7fc1a9fcd640 1 -- 172.21.15.27:0/1761538276 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc1a406db30 msgr2=0x7fc1a4159f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:48.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:48.739+0000 7fc1a9fcd640 1 --2- 172.21.15.27:0/1761538276 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc1a406db30 0x7fc1a4159f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:48.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:48.739+0000 7fc1a9fcd640 1 -- 172.21.15.27:0/1761538276 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1a415a9a0 con 0x7fc1a415e650 2024-09-12T03:21:48.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:48.739+0000 7fc1a9fcd640 1 --2- 172.21.15.27:0/1761538276 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc1a415e650 0x7fc1a415a460 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7fc19c00e890 tx=0x7fc19c00ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:48.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:48.739+0000 7fc1a1ffb640 1 -- 172.21.15.27:0/1761538276 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc19c017070 con 0x7fc1a415e650 2024-09-12T03:21:48.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:48.739+0000 7fc1a1ffb640 1 -- 172.21.15.27:0/1761538276 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc19c00c8e0 con 0x7fc1a415e650 2024-09-12T03:21:48.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:48.739+0000 7fc1aba30640 1 -- 172.21.15.27:0/1761538276 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1a415ac50 con 0x7fc1a415e650 2024-09-12T03:21:48.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:48.739+0000 7fc1a1ffb640 1 -- 172.21.15.27:0/1761538276 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc19c00ccd0 con 0x7fc1a415e650 2024-09-12T03:21:48.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:48.739+0000 7fc1aba30640 1 -- 172.21.15.27:0/1761538276 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1a4155200 con 0x7fc1a415e650 2024-09-12T03:21:48.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:48.740+0000 7fc1aba30640 1 -- 172.21.15.27:0/1761538276 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1a406df60 con 0x7fc1a415e650 2024-09-12T03:21:48.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:48.741+0000 7fc1a1ffb640 1 -- 172.21.15.27:0/1761538276 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 19) ==== 90855+0+0 (secure 0 0 0) 0x7fc19c01a070 con 0x7fc1a415e650 2024-09-12T03:21:48.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:48.741+0000 7fc1a1ffb640 1 --2- 172.21.15.27:0/1761538276 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fc19006c250 0x7fc19006e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:48.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:48.741+0000 7fc1a1ffb640 1 -- 172.21.15.27:0/1761538276 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5737+0+0 (secure 0 0 0) 0x7fc19c08b460 con 0x7fc1a415e650 2024-09-12T03:21:48.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:48.741+0000 7fc1a97cc640 1 --2- 172.21.15.27:0/1761538276 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fc19006c250 0x7fc19006e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:48.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:48.742+0000 7fc1a97cc640 1 --2- 172.21.15.27:0/1761538276 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fc19006c250 0x7fc19006e710 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fc194033000 tx=0x7fc194009000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:48.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:48.746+0000 7fc1a1ffb640 1 -- 172.21.15.27:0/1761538276 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fc19c0591c0 con 0x7fc1a415e650 2024-09-12T03:21:48.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:48.955+0000 7fc1aba30640 1 -- 172.21.15.27:0/1761538276 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fc1a4074210 con 0x7fc1a415e650 2024-09-12T03:21:49.270 INFO:tasks.ceph.mds.a.smithi027.stderr:2024-09-12T03:21:49.270+0000 7fec0688a640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-12T03:21:49.616 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:49.616+0000 7fc1a1ffb640 1 -- 172.21.15.27:0/1761538276 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v67) ==== 122+0+0 (secure 0 0 0) 0x7fc19c05d4c0 con 0x7fc1a415e650 2024-09-12T03:21:49.619 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:49.618+0000 7fc1aba30640 1 -- 172.21.15.27:0/1761538276 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fc19006c250 msgr2=0x7fc19006e710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:49.619 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:49.618+0000 7fc1aba30640 1 --2- 172.21.15.27:0/1761538276 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fc19006c250 0x7fc19006e710 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fc194033000 tx=0x7fc194009000 comp rx=0 tx=0).stop 2024-09-12T03:21:49.619 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:49.618+0000 7fc1aba30640 1 -- 172.21.15.27:0/1761538276 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc1a415e650 msgr2=0x7fc1a415a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:49.619 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:49.618+0000 7fc1aba30640 1 --2- 172.21.15.27:0/1761538276 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc1a415e650 0x7fc1a415a460 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7fc19c00e890 tx=0x7fc19c00ed50 comp rx=0 tx=0).stop 2024-09-12T03:21:49.619 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:49.619+0000 7fc1aba30640 1 -- 172.21.15.27:0/1761538276 shutdown_connections 2024-09-12T03:21:49.619 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:49.619+0000 7fc1aba30640 1 --2- 172.21.15.27:0/1761538276 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fc19006c250 0x7fc19006e710 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:49.619 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:49.619+0000 7fc1aba30640 1 --2- 172.21.15.27:0/1761538276 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc1a415e650 0x7fc1a415a460 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:49.619 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:49.619+0000 7fc1aba30640 1 --2- 172.21.15.27:0/1761538276 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc1a406db30 0x7fc1a4159f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:49.620 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:49.619+0000 7fc1aba30640 1 --2- 172.21.15.27:0/1761538276 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fc1a406cc40 0x7fc1a41542b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:49.620 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:49.619+0000 7fc1aba30640 1 -- 172.21.15.27:0/1761538276 >> 172.21.15.27:0/1761538276 conn(0x7fc1a406f320 msgr2=0x7fc1a40b45c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:49.620 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:49.619+0000 7fc1aba30640 1 -- 172.21.15.27:0/1761538276 shutdown_connections 2024-09-12T03:21:49.620 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:49.619+0000 7fc1aba30640 1 -- 172.21.15.27:0/1761538276 wait complete. 2024-09-12T03:21:49.632 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-12T03:21:49.632 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-12T03:21:49.740 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:49.740+0000 7f0d1bb11640 1 Processor -- start 2024-09-12T03:21:49.740 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:49.740+0000 7f0d1bb11640 1 -- start start 2024-09-12T03:21:49.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:49.741+0000 7f0d1bb11640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f0d140b1ca0 0x7f0d1415f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:49.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:49.741+0000 7f0d1bb11640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f0d1406f360 0x7f0d140cef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:49.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:49.741+0000 7f0d1bb11640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f0d140cf490 0x7f0d140d2780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:49.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:49.741+0000 7f0d1bb11640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f0d140d22f0 con 0x7f0d140b1ca0 2024-09-12T03:21:49.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:49.741+0000 7f0d1bb11640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f0d1415fd70 con 0x7f0d1406f360 2024-09-12T03:21:49.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:49.741+0000 7f0d1bb11640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f0d1406cc40 con 0x7f0d140cf490 2024-09-12T03:21:49.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:49.741+0000 7f0d198ad640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f0d140b1ca0 0x7f0d1415f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:49.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:49.741+0000 7f0d198ad640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f0d140b1ca0 0x7f0d1415f830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39434/0 (socket says 172.21.15.27:39434) 2024-09-12T03:21:49.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:49.741+0000 7f0d198ad640 1 -- 172.21.15.27:0/986137577 learned_addr learned my addr 172.21.15.27:0/986137577 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:21:49.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:49.741+0000 7f0d190ac640 1 --2- 172.21.15.27:0/986137577 >> v2:172.21.15.45:3300/0 conn(0x7f0d1406f360 0x7f0d140cef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:49.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:49.741+0000 7f0d1a0ae640 1 --2- 172.21.15.27:0/986137577 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f0d140cf490 0x7f0d140d2780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:49.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:49.742+0000 7f0d198ad640 1 -- 172.21.15.27:0/986137577 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f0d140cf490 msgr2=0x7f0d140d2780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:49.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:49.742+0000 7f0d198ad640 1 --2- 172.21.15.27:0/986137577 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f0d140cf490 0x7f0d140d2780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:49.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:49.742+0000 7f0d198ad640 1 -- 172.21.15.27:0/986137577 >> v2:172.21.15.45:3300/0 conn(0x7f0d1406f360 msgr2=0x7f0d140cef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:49.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:49.742+0000 7f0d198ad640 1 --2- 172.21.15.27:0/986137577 >> v2:172.21.15.45:3300/0 conn(0x7f0d1406f360 0x7f0d140cef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:49.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:49.742+0000 7f0d198ad640 1 -- 172.21.15.27:0/986137577 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d140d1ce0 con 0x7f0d140b1ca0 2024-09-12T03:21:49.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:49.742+0000 7f0d198ad640 1 --2- 172.21.15.27:0/986137577 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0d140b1ca0 0x7f0d1415f830 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f0d140b5220 tx=0x7f0d1002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab16a8010c87cff9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:49.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:49.742+0000 7f0d188ab640 1 -- 172.21.15.27:0/986137577 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d10037070 con 0x7f0d140b1ca0 2024-09-12T03:21:49.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:49.743+0000 7f0d188ab640 1 -- 172.21.15.27:0/986137577 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d10032440 con 0x7f0d140b1ca0 2024-09-12T03:21:49.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:49.743+0000 7f0d188ab640 1 -- 172.21.15.27:0/986137577 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d10032810 con 0x7f0d140b1ca0 2024-09-12T03:21:49.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:49.743+0000 7f0d1bb11640 1 -- 172.21.15.27:0/986137577 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0d140b1ca0 msgr2=0x7f0d1415f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:49.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:49.743+0000 7f0d1bb11640 1 --2- 172.21.15.27:0/986137577 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0d140b1ca0 0x7f0d1415f830 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f0d140b5220 tx=0x7f0d1002a1e0 comp rx=0 tx=0).stop 2024-09-12T03:21:49.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:49.743+0000 7f0d1bb11640 1 -- 172.21.15.27:0/986137577 shutdown_connections 2024-09-12T03:21:49.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:49.743+0000 7f0d1bb11640 1 --2- 172.21.15.27:0/986137577 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f0d140cf490 0x7f0d140d2780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:49.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:49.743+0000 7f0d1bb11640 1 --2- 172.21.15.27:0/986137577 >> v2:172.21.15.45:3300/0 conn(0x7f0d1406f360 0x7f0d140cef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:49.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:49.743+0000 7f0d1bb11640 1 --2- 172.21.15.27:0/986137577 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0d140b1ca0 0x7f0d1415f830 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:49.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:49.743+0000 7f0d1bb11640 1 -- 172.21.15.27:0/986137577 >> 172.21.15.27:0/986137577 conn(0x7f0d14073800 msgr2=0x7f0d1409ba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:49.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:49.743+0000 7f0d1bb11640 1 -- 172.21.15.27:0/986137577 shutdown_connections 2024-09-12T03:21:49.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:49.743+0000 7f0d1bb11640 1 -- 172.21.15.27:0/986137577 wait complete. 2024-09-12T03:21:49.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:49.744+0000 7f0d1bb11640 1 Processor -- start 2024-09-12T03:21:49.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:49.744+0000 7f0d1bb11640 1 -- start start 2024-09-12T03:21:49.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:49.745+0000 7f0d1bb11640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f0d1406f360 0x7f0d1415b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:49.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:49.745+0000 7f0d1bb11640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0d140b1ca0 0x7f0d141554a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:49.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:49.745+0000 7f0d1bb11640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0d140cf490 0x7f0d1415d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:49.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:49.745+0000 7f0d1bb11640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f0d1406d1c0 con 0x7f0d140cf490 2024-09-12T03:21:49.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:49.745+0000 7f0d1bb11640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f0d140d1f80 con 0x7f0d140b1ca0 2024-09-12T03:21:49.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:49.745+0000 7f0d1bb11640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f0d1406d340 con 0x7f0d1406f360 2024-09-12T03:21:49.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:49.745+0000 7f0d1a0ae640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0d140cf490 0x7f0d1415d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:49.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:49.745+0000 7f0d1a0ae640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0d140cf490 0x7f0d1415d270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39444/0 (socket says 172.21.15.27:39444) 2024-09-12T03:21:49.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:49.745+0000 7f0d1a0ae640 1 -- 172.21.15.27:0/2188800983 learned_addr learned my addr 172.21.15.27:0/2188800983 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:21:49.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:49.745+0000 7f0d198ad640 1 --2- 172.21.15.27:0/2188800983 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f0d1406f360 0x7f0d1415b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:49.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:49.745+0000 7f0d190ac640 1 --2- 172.21.15.27:0/2188800983 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0d140b1ca0 0x7f0d141554a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:49.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:49.746+0000 7f0d1a0ae640 1 -- 172.21.15.27:0/2188800983 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f0d1406f360 msgr2=0x7f0d1415b890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:49.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:49.746+0000 7f0d1a0ae640 1 --2- 172.21.15.27:0/2188800983 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f0d1406f360 0x7f0d1415b890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:49.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:49.746+0000 7f0d1a0ae640 1 -- 172.21.15.27:0/2188800983 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0d140b1ca0 msgr2=0x7f0d141554a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:49.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:49.746+0000 7f0d1a0ae640 1 --2- 172.21.15.27:0/2188800983 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0d140b1ca0 0x7f0d141554a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:49.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:49.746+0000 7f0d1a0ae640 1 -- 172.21.15.27:0/2188800983 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d1415a7f0 con 0x7f0d140cf490 2024-09-12T03:21:49.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:49.746+0000 7f0d1a0ae640 1 --2- 172.21.15.27:0/2188800983 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0d140cf490 0x7f0d1415d270 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f0d0c009d30 tx=0x7f0d0c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:49.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:49.746+0000 7f0d0a7fc640 1 -- 172.21.15.27:0/2188800983 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d0c018020 con 0x7f0d140cf490 2024-09-12T03:21:49.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:49.746+0000 7f0d0a7fc640 1 -- 172.21.15.27:0/2188800983 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d0c014480 con 0x7f0d140cf490 2024-09-12T03:21:49.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:49.746+0000 7f0d1bb11640 1 -- 172.21.15.27:0/2188800983 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d1415aad0 con 0x7f0d140cf490 2024-09-12T03:21:49.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:49.746+0000 7f0d0a7fc640 1 -- 172.21.15.27:0/2188800983 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d0c014870 con 0x7f0d140cf490 2024-09-12T03:21:49.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:49.746+0000 7f0d1bb11640 1 -- 172.21.15.27:0/2188800983 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d1415b090 con 0x7f0d140cf490 2024-09-12T03:21:49.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:49.747+0000 7f0d1bb11640 1 -- 172.21.15.27:0/2188800983 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d140cfc40 con 0x7f0d140cf490 2024-09-12T03:21:49.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:49.748+0000 7f0d0a7fc640 1 -- 172.21.15.27:0/2188800983 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 19) ==== 90855+0+0 (secure 0 0 0) 0x7f0d0c005d10 con 0x7f0d140cf490 2024-09-12T03:21:49.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:49.748+0000 7f0d0a7fc640 1 --2- 172.21.15.27:0/2188800983 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f0cf806c220 0x7f0cf806e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:49.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:49.748+0000 7f0d198ad640 1 --2- 172.21.15.27:0/2188800983 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f0cf806c220 0x7f0cf806e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:49.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:49.748+0000 7f0d0a7fc640 1 -- 172.21.15.27:0/2188800983 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5737+0+0 (secure 0 0 0) 0x7f0d0c08c0f0 con 0x7f0d140cf490 2024-09-12T03:21:49.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:49.748+0000 7f0d198ad640 1 --2- 172.21.15.27:0/2188800983 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f0cf806c220 0x7f0cf806e6e0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f0d10034000 tx=0x7f0d1002e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:49.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:49.752+0000 7f0d0a7fc640 1 -- 172.21.15.27:0/2188800983 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f0d0c022070 con 0x7f0d140cf490 2024-09-12T03:21:49.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:49.965+0000 7f0d1bb11640 1 -- 172.21.15.27:0/2188800983 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0d140d0850 con 0x7f0d140cf490 2024-09-12T03:21:49.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:49.966+0000 7f0d0a7fc640 1 -- 172.21.15.27:0/2188800983 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) ==== 94+0+5225 (secure 0 0 0) 0x7f0d0c010040 con 0x7f0d140cf490 2024-09-12T03:21:49.966 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:21:49.966 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":67,"btime":"2024-09-12T03:21:49:612947+0000","default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6206,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/2249506350","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2249506350},{"type":"v1","addr":"172.21.15.45:6834","nonce":2249506350}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63},{"gid":6209,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/2292759672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":2292759672},{"type":"v1","addr":"172.21.15.45:6835","nonce":2292759672}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63},{"gid":7299,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6834/4021825986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":4021825986},{"type":"v1","addr":"172.21.15.27:6834","nonce":4021825986}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":67,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-12T03:21:46.603883+0000","modified":"2024-09-12T03:21:48.956285+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7302},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7302":{"gid":7302,"name":"a","rank":0,"incarnation":64,"state":"up:active","state_seq":3,"addr":"172.21.15.27:6835/1760654641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":1760654641},{"type":"v1","addr":"172.21.15.27:6835","nonce":1760654641}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7302,"qdb_cluster":[7302]},"id":9}]} 2024-09-12T03:21:49.966 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 67 2024-09-12T03:21:49.968 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:49.968+0000 7f0d1bb11640 1 -- 172.21.15.27:0/2188800983 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f0cf806c220 msgr2=0x7f0cf806e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:49.968 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:49.968+0000 7f0d1bb11640 1 --2- 172.21.15.27:0/2188800983 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f0cf806c220 0x7f0cf806e6e0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f0d10034000 tx=0x7f0d1002e000 comp rx=0 tx=0).stop 2024-09-12T03:21:49.968 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:49.968+0000 7f0d1bb11640 1 -- 172.21.15.27:0/2188800983 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0d140cf490 msgr2=0x7f0d1415d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:49.968 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:49.968+0000 7f0d1bb11640 1 --2- 172.21.15.27:0/2188800983 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0d140cf490 0x7f0d1415d270 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f0d0c009d30 tx=0x7f0d0c00c490 comp rx=0 tx=0).stop 2024-09-12T03:21:49.968 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:49.968+0000 7f0d1bb11640 1 -- 172.21.15.27:0/2188800983 shutdown_connections 2024-09-12T03:21:49.969 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:49.968+0000 7f0d1bb11640 1 --2- 172.21.15.27:0/2188800983 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f0cf806c220 0x7f0cf806e6e0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:49.969 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:49.968+0000 7f0d1bb11640 1 --2- 172.21.15.27:0/2188800983 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0d140cf490 0x7f0d1415d270 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:49.969 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:49.968+0000 7f0d1bb11640 1 --2- 172.21.15.27:0/2188800983 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0d140b1ca0 0x7f0d141554a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:49.969 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:49.968+0000 7f0d1bb11640 1 --2- 172.21.15.27:0/2188800983 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f0d1406f360 0x7f0d1415b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:49.969 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:49.968+0000 7f0d1bb11640 1 -- 172.21.15.27:0/2188800983 >> 172.21.15.27:0/2188800983 conn(0x7f0d14073800 msgr2=0x7f0d141605c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:49.969 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:49.968+0000 7f0d1bb11640 1 -- 172.21.15.27:0/2188800983 shutdown_connections 2024-09-12T03:21:49.969 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:49.968+0000 7f0d1bb11640 1 -- 172.21.15.27:0/2188800983 wait complete. 2024-09-12T03:21:49.979 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-12T03:21:50.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.081+0000 7efced39f640 1 Processor -- start 2024-09-12T03:21:50.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.081+0000 7efced39f640 1 -- start start 2024-09-12T03:21:50.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.082+0000 7efced39f640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7efce815e770 0x7efce80d3a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:50.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.082+0000 7efced39f640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7efce80d3fc0 0x7efce80d1090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:50.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.082+0000 7efced39f640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efce80d2790 con 0x7efce80d15d0 2024-09-12T03:21:50.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.082+0000 7efced39f640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7efce80d4400 con 0x7efce80d3fc0 2024-09-12T03:21:50.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.082+0000 7efced39f640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7efce80d2a70 con 0x7efce815e770 2024-09-12T03:21:50.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.082+0000 7efce77fe640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7efce80d3fc0 0x7efce80d1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:50.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.082+0000 7efce77fe640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7efce80d3fc0 0x7efce80d1090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39446/0 (socket says 172.21.15.27:39446) 2024-09-12T03:21:50.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.082+0000 7efce77fe640 1 -- 172.21.15.27:0/977079761 learned_addr learned my addr 172.21.15.27:0/977079761 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:21:50.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.082+0000 7efce7fff640 1 --2- 172.21.15.27:0/977079761 >> v2:172.21.15.45:3300/0 conn(0x7efce815e770 0x7efce80d3a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:50.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.082+0000 7efce77fe640 1 -- 172.21.15.27:0/977079761 >> v2:172.21.15.45:3300/0 conn(0x7efce815e770 msgr2=0x7efce80d3a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:50.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.082+0000 7efce77fe640 1 --2- 172.21.15.27:0/977079761 >> v2:172.21.15.45:3300/0 conn(0x7efce815e770 0x7efce80d3a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:50.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.083+0000 7efce77fe640 1 -- 172.21.15.27:0/977079761 >> v1:172.21.15.45:6789/0 conn(0x7efce80d15d0 legacy=0x7efce806bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:50.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.083+0000 7efce77fe640 1 -- 172.21.15.27:0/977079761 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efce80d2d50 con 0x7efce80d3fc0 2024-09-12T03:21:50.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.083+0000 7efce77fe640 1 --2- 172.21.15.27:0/977079761 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efce80d3fc0 0x7efce80d1090 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7efce80d5600 tx=0x7efcdc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c90264b0545ded1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:50.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.083+0000 7efce6ffd640 1 -- 172.21.15.27:0/977079761 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcdc037070 con 0x7efce80d3fc0 2024-09-12T03:21:50.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.083+0000 7efce6ffd640 1 -- 172.21.15.27:0/977079761 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efcdc032440 con 0x7efce80d3fc0 2024-09-12T03:21:50.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.084+0000 7efce6ffd640 1 -- 172.21.15.27:0/977079761 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcdc032810 con 0x7efce80d3fc0 2024-09-12T03:21:50.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.084+0000 7efced39f640 1 -- 172.21.15.27:0/977079761 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efce80d3fc0 msgr2=0x7efce80d1090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:50.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.084+0000 7efced39f640 1 --2- 172.21.15.27:0/977079761 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efce80d3fc0 0x7efce80d1090 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7efce80d5600 tx=0x7efcdc02a270 comp rx=0 tx=0).stop 2024-09-12T03:21:50.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.084+0000 7efced39f640 1 -- 172.21.15.27:0/977079761 shutdown_connections 2024-09-12T03:21:50.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.084+0000 7efced39f640 1 --2- 172.21.15.27:0/977079761 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efce80d3fc0 0x7efce80d1090 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:50.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.084+0000 7efced39f640 1 --2- 172.21.15.27:0/977079761 >> v2:172.21.15.45:3300/0 conn(0x7efce815e770 0x7efce80d3a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:50.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.084+0000 7efced39f640 1 -- 172.21.15.27:0/977079761 >> 172.21.15.27:0/977079761 conn(0x7efce80b6100 msgr2=0x7efce80b6510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:50.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.084+0000 7efced39f640 1 -- 172.21.15.27:0/977079761 shutdown_connections 2024-09-12T03:21:50.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.084+0000 7efced39f640 1 -- 172.21.15.27:0/977079761 wait complete. 2024-09-12T03:21:50.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.085+0000 7efced39f640 1 Processor -- start 2024-09-12T03:21:50.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.086+0000 7efced39f640 1 -- start start 2024-09-12T03:21:50.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.086+0000 7efced39f640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efce80d15d0 0x7efce80abb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:50.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.086+0000 7efced39f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efce80d3fc0 0x7efce80ad360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:50.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.086+0000 7efced39f640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7efce815e770 0x7efce80ad8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:50.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.086+0000 7efced39f640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7efce80d1a40 con 0x7efce80d15d0 2024-09-12T03:21:50.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.086+0000 7efced39f640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7efce80d3310 con 0x7efce80d3fc0 2024-09-12T03:21:50.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.086+0000 7efced39f640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7efce80d1bc0 con 0x7efce815e770 2024-09-12T03:21:50.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.086+0000 7efce7fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efce80d15d0 0x7efce80abb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:50.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.086+0000 7efce7fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efce80d15d0 0x7efce80abb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39452/0 (socket says 172.21.15.27:39452) 2024-09-12T03:21:50.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.086+0000 7efce7fff640 1 -- 172.21.15.27:0/1723804758 learned_addr learned my addr 172.21.15.27:0/1723804758 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:21:50.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.086+0000 7efce77fe640 1 --2- 172.21.15.27:0/1723804758 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efce80d3fc0 0x7efce80ad360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:50.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.086+0000 7efcecb9e640 1 --2- 172.21.15.27:0/1723804758 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7efce815e770 0x7efce80ad8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:50.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.087+0000 7efce7fff640 1 -- 172.21.15.27:0/1723804758 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7efce815e770 msgr2=0x7efce80ad8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:50.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.087+0000 7efce7fff640 1 --2- 172.21.15.27:0/1723804758 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7efce815e770 0x7efce80ad8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:50.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.087+0000 7efce7fff640 1 -- 172.21.15.27:0/1723804758 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efce80d3fc0 msgr2=0x7efce80ad360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:50.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.087+0000 7efce7fff640 1 --2- 172.21.15.27:0/1723804758 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efce80d3fc0 0x7efce80ad360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:50.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.087+0000 7efce7fff640 1 -- 172.21.15.27:0/1723804758 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efce80ac4a0 con 0x7efce80d15d0 2024-09-12T03:21:50.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.087+0000 7efce7fff640 1 --2- 172.21.15.27:0/1723804758 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efce80d15d0 0x7efce80abb30 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7efcd800e870 tx=0x7efcd800ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:50.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.087+0000 7efce4ff9640 1 -- 172.21.15.27:0/1723804758 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcd8017070 con 0x7efce80d15d0 2024-09-12T03:21:50.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.087+0000 7efce4ff9640 1 -- 172.21.15.27:0/1723804758 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efcd800c940 con 0x7efce80d15d0 2024-09-12T03:21:50.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.087+0000 7efced39f640 1 -- 172.21.15.27:0/1723804758 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efce80ac780 con 0x7efce80d15d0 2024-09-12T03:21:50.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.087+0000 7efce4ff9640 1 -- 172.21.15.27:0/1723804758 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcd800cd30 con 0x7efce80d15d0 2024-09-12T03:21:50.090 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.087+0000 7efced39f640 1 -- 172.21.15.27:0/1723804758 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efce80cc250 con 0x7efce80d15d0 2024-09-12T03:21:50.090 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.088+0000 7efced39f640 1 -- 172.21.15.27:0/1723804758 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efce8068d80 con 0x7efce80d15d0 2024-09-12T03:21:50.090 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.089+0000 7efce4ff9640 1 -- 172.21.15.27:0/1723804758 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 19) ==== 90855+0+0 (secure 0 0 0) 0x7efcd801a070 con 0x7efce80d15d0 2024-09-12T03:21:50.090 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.089+0000 7efce4ff9640 1 --2- 172.21.15.27:0/1723804758 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7efcd406c250 0x7efcd406e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:50.090 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.089+0000 7efce4ff9640 1 -- 172.21.15.27:0/1723804758 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5737+0+0 (secure 0 0 0) 0x7efcd808b6a0 con 0x7efce80d15d0 2024-09-12T03:21:50.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.089+0000 7efce77fe640 1 --2- 172.21.15.27:0/1723804758 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7efcd406c250 0x7efcd406e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:50.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.090+0000 7efce77fe640 1 --2- 172.21.15.27:0/1723804758 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7efcd406c250 0x7efcd406e710 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7efcdc034000 tx=0x7efcdc02e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:50.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.093+0000 7efce4ff9640 1 -- 172.21.15.27:0/1723804758 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7efce8068d80 con 0x7efce80d15d0 2024-09-12T03:21:50.269 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.269+0000 7efced39f640 1 -- 172.21.15.27:0/1723804758 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7efce80d1090 con 0x7efce80d15d0 2024-09-12T03:21:50.270 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.271+0000 7efce4ff9640 1 -- 172.21.15.27:0/1723804758 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v114) ==== 74+0+16735 (secure 0 0 0) 0x7efcd8059490 con 0x7efce80d15d0 2024-09-12T03:21:50.271 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:21:50.271 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":114,"fsid":"d2df504e-2647-4385-a5db-9fdf69f487a0","created":"2024-09-12T03:16:07.778937+0000","modified":"2024-09-12T03:21:48.262478+0000","last_up_change":"2024-09-12T03:16:19.314778+0000","last_in_change":"2024-09-12T03:16:09.893086+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T03:16:20.097646+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-09-12T03:21:44.263535+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-09-12T03:21:45.593648+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"114","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1851116c-1fb3-4dc9-9772-f90e2ef4c355","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6817","nonce":3002321488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6819","nonce":3002321488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6823","nonce":3002321488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6821","nonce":3002321488}]},"public_addr":"172.21.15.27:6817/3002321488","cluster_addr":"172.21.15.27:6819/3002321488","heartbeat_back_addr":"172.21.15.27:6823/3002321488","heartbeat_front_addr":"172.21.15.27:6821/3002321488","state":["exists","up"]},{"osd":1,"uuid":"f38ae17d-73eb-4204-b99d-975fa806fb87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6809","nonce":1613102886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6811","nonce":1613102886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6815","nonce":1613102886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6813","nonce":1613102886}]},"public_addr":"172.21.15.27:6809/1613102886","cluster_addr":"172.21.15.27:6811/1613102886","heartbeat_back_addr":"172.21.15.27:6815/1613102886","heartbeat_front_addr":"172.21.15.27:6813/1613102886","state":["exists","up"]},{"osd":2,"uuid":"fb12dcc8-bdd0-42b8-b46d-8b10106224bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6825","nonce":3534723355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6827","nonce":3534723355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6831","nonce":3534723355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6829","nonce":3534723355}]},"public_addr":"172.21.15.27:6825/3534723355","cluster_addr":"172.21.15.27:6827/3534723355","heartbeat_back_addr":"172.21.15.27:6831/3534723355","heartbeat_front_addr":"172.21.15.27:6829/3534723355","state":["exists","up"]},{"osd":3,"uuid":"d74d3f49-9c24-4b9f-98e2-3ecd7b5ef3b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6801","nonce":80661598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6803","nonce":80661598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6807","nonce":80661598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6805","nonce":80661598}]},"public_addr":"172.21.15.27:6801/80661598","cluster_addr":"172.21.15.27:6803/80661598","heartbeat_back_addr":"172.21.15.27:6807/80661598","heartbeat_front_addr":"172.21.15.27:6805/80661598","state":["exists","up"]},{"osd":4,"uuid":"2bf123f2-1b0d-41c4-aa3b-805bab267e59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6817","nonce":3943189827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6819","nonce":3943189827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6823","nonce":3943189827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6821","nonce":3943189827}]},"public_addr":"172.21.15.45:6817/3943189827","cluster_addr":"172.21.15.45:6819/3943189827","heartbeat_back_addr":"172.21.15.45:6823/3943189827","heartbeat_front_addr":"172.21.15.45:6821/3943189827","state":["exists","up"]},{"osd":5,"uuid":"c49ba5da-91c6-4a86-a97b-4aba89c253ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6809","nonce":3620760330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6811","nonce":3620760330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6815","nonce":3620760330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6813","nonce":3620760330}]},"public_addr":"172.21.15.45:6809/3620760330","cluster_addr":"172.21.15.45:6811/3620760330","heartbeat_back_addr":"172.21.15.45:6815/3620760330","heartbeat_front_addr":"172.21.15.45:6813/3620760330","state":["exists","up"]},{"osd":6,"uuid":"b0123475-89f7-4a34-9e33-d797b99dd9dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6801","nonce":1989083510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6803","nonce":1989083510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6807","nonce":1989083510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6805","nonce":1989083510}]},"public_addr":"172.21.15.45:6801/1989083510","cluster_addr":"172.21.15.45:6803/1989083510","heartbeat_back_addr":"172.21.15.45:6807/1989083510","heartbeat_front_addr":"172.21.15.45:6805/1989083510","state":["exists","up"]},{"osd":7,"uuid":"4dad773d-b7d8-46fc-8a10-a972aad27b29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6825","nonce":1513300294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6827","nonce":1513300294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6831","nonce":1513300294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6829","nonce":1513300294}]},"public_addr":"172.21.15.45:6825/1513300294","cluster_addr":"172.21.15.45:6827/1513300294","heartbeat_back_addr":"172.21.15.45:6831/1513300294","heartbeat_front_addr":"172.21.15.45:6829/1513300294","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.881855+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:17.133252+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.735339+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.500742+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.477571+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:17.167667+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.470291+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.873743+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T03:21:50.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.273+0000 7efced39f640 1 -- 172.21.15.27:0/1723804758 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7efcd406c250 msgr2=0x7efcd406e710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:50.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.273+0000 7efced39f640 1 --2- 172.21.15.27:0/1723804758 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7efcd406c250 0x7efcd406e710 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7efcdc034000 tx=0x7efcdc02e000 comp rx=0 tx=0).stop 2024-09-12T03:21:50.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.273+0000 7efced39f640 1 -- 172.21.15.27:0/1723804758 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efce80d15d0 msgr2=0x7efce80abb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:50.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.273+0000 7efced39f640 1 --2- 172.21.15.27:0/1723804758 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efce80d15d0 0x7efce80abb30 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7efcd800e870 tx=0x7efcd800ed30 comp rx=0 tx=0).stop 2024-09-12T03:21:50.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.273+0000 7efced39f640 1 -- 172.21.15.27:0/1723804758 shutdown_connections 2024-09-12T03:21:50.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.273+0000 7efced39f640 1 --2- 172.21.15.27:0/1723804758 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7efce815e770 0x7efce80ad8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:50.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.273+0000 7efced39f640 1 --2- 172.21.15.27:0/1723804758 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efce80d3fc0 0x7efce80ad360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:50.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.273+0000 7efced39f640 1 --2- 172.21.15.27:0/1723804758 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7efcd406c250 0x7efcd406e710 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:50.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.273+0000 7efced39f640 1 --2- 172.21.15.27:0/1723804758 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efce80d15d0 0x7efce80abb30 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:50.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.273+0000 7efced39f640 1 -- 172.21.15.27:0/1723804758 >> 172.21.15.27:0/1723804758 conn(0x7efce80b6100 msgr2=0x7efce80b5840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:50.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.273+0000 7efced39f640 1 -- 172.21.15.27:0/1723804758 shutdown_connections 2024-09-12T03:21:50.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.273+0000 7efced39f640 1 -- 172.21.15.27:0/1723804758 wait complete. 2024-09-12T03:21:50.288 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-12T03:21:50.289 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-12T03:21:50.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.391+0000 7f5165b6d640 1 Processor -- start 2024-09-12T03:21:50.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.391+0000 7f5165b6d640 1 -- start start 2024-09-12T03:21:50.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.392+0000 7f5165b6d640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f51600b1ca0 0x7f516015f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:50.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.392+0000 7f5165b6d640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f516006f360 0x7f51600cef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:50.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.392+0000 7f5165b6d640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f516015fd70 con 0x7f51600cf490 2024-09-12T03:21:50.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.392+0000 7f5165b6d640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f51600cf900 con 0x7f51600b1ca0 2024-09-12T03:21:50.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.392+0000 7f5165b6d640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f51600d22f0 con 0x7f516006f360 2024-09-12T03:21:50.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.392+0000 7f515f7fe640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f51600b1ca0 0x7f516015f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:50.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.392+0000 7f515ffff640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f51600cf490 0x7f51600d2780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:56320/0 (socket says 172.21.15.27:56320) 2024-09-12T03:21:50.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.392+0000 7f515ffff640 1 -- 172.21.15.27:0/3309237751 learned_addr learned my addr 172.21.15.27:0/3309237751 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:21:50.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.392+0000 7f515effd640 1 --2- 172.21.15.27:0/3309237751 >> v2:172.21.15.45:3300/0 conn(0x7f516006f360 0x7f51600cef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:50.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.392+0000 7f515f7fe640 1 -- 172.21.15.27:0/3309237751 >> v2:172.21.15.45:3300/0 conn(0x7f516006f360 msgr2=0x7f51600cef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:50.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.392+0000 7f515f7fe640 1 --2- 172.21.15.27:0/3309237751 >> v2:172.21.15.45:3300/0 conn(0x7f516006f360 0x7f51600cef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:50.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.392+0000 7f515f7fe640 1 -- 172.21.15.27:0/3309237751 >> v1:172.21.15.27:6789/0 conn(0x7f51600cf490 legacy=0x7f51600d2780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:50.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.392+0000 7f515e7fc640 1 -- 172.21.15.27:0/3309237751 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1834209424 0 0) 0x7f5158001490 con 0x7f51600cf490 2024-09-12T03:21:50.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.392+0000 7f515f7fe640 1 -- 172.21.15.27:0/3309237751 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f516006cc40 con 0x7f51600b1ca0 2024-09-12T03:21:50.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.392+0000 7f515effd640 1 --2- 172.21.15.27:0/3309237751 >> v2:172.21.15.45:3300/0 conn(0x7f516006f360 0x7f51600cef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:21:50.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.393+0000 7f515f7fe640 1 --2- 172.21.15.27:0/3309237751 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f51600b1ca0 0x7f516015f830 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f51600b5220 tx=0x7f515002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7974831bbac6feec server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:50.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.393+0000 7f515e7fc640 1 -- 172.21.15.27:0/3309237751 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5150037070 con 0x7f51600b1ca0 2024-09-12T03:21:50.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.393+0000 7f515e7fc640 1 -- 172.21.15.27:0/3309237751 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5150032440 con 0x7f51600b1ca0 2024-09-12T03:21:50.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.393+0000 7f515e7fc640 1 -- 172.21.15.27:0/3309237751 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5150032810 con 0x7f51600b1ca0 2024-09-12T03:21:50.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.394+0000 7f5165b6d640 1 -- 172.21.15.27:0/3309237751 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f51600b1ca0 msgr2=0x7f516015f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:50.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.394+0000 7f5165b6d640 1 --2- 172.21.15.27:0/3309237751 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f51600b1ca0 0x7f516015f830 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f51600b5220 tx=0x7f515002a1a0 comp rx=0 tx=0).stop 2024-09-12T03:21:50.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.394+0000 7f5165b6d640 1 -- 172.21.15.27:0/3309237751 shutdown_connections 2024-09-12T03:21:50.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.394+0000 7f5165b6d640 1 --2- 172.21.15.27:0/3309237751 >> v2:172.21.15.45:3300/0 conn(0x7f516006f360 0x7f51600cef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:50.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.394+0000 7f5165b6d640 1 --2- 172.21.15.27:0/3309237751 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f51600b1ca0 0x7f516015f830 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:50.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.394+0000 7f5165b6d640 1 -- 172.21.15.27:0/3309237751 >> 172.21.15.27:0/3309237751 conn(0x7f5160073800 msgr2=0x7f516009ba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:50.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.394+0000 7f5165b6d640 1 -- 172.21.15.27:0/3309237751 shutdown_connections 2024-09-12T03:21:50.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.394+0000 7f5165b6d640 1 -- 172.21.15.27:0/3309237751 wait complete. 2024-09-12T03:21:50.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.395+0000 7f5165b6d640 1 Processor -- start 2024-09-12T03:21:50.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.395+0000 7f5165b6d640 1 -- start start 2024-09-12T03:21:50.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.396+0000 7f5165b6d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f516006f360 0x7f5160152f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:50.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.396+0000 7f5165b6d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f51600b1ca0 0x7f51600cd710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:50.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.396+0000 7f5165b6d640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f51600cf490 0x7f51600b6520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:50.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.396+0000 7f5165b6d640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f51600d1f80 con 0x7f51600b1ca0 2024-09-12T03:21:50.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.396+0000 7f5165b6d640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f51600d1ce0 con 0x7f516006f360 2024-09-12T03:21:50.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.396+0000 7f5165b6d640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f516006d1c0 con 0x7f51600cf490 2024-09-12T03:21:50.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.396+0000 7f515effd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f51600b1ca0 0x7f51600cd710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:50.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.396+0000 7f515effd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f51600b1ca0 0x7f51600cd710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39472/0 (socket says 172.21.15.27:39472) 2024-09-12T03:21:50.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.396+0000 7f515effd640 1 -- 172.21.15.27:0/4240500650 learned_addr learned my addr 172.21.15.27:0/4240500650 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:21:50.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.396+0000 7f515f7fe640 1 --2- 172.21.15.27:0/4240500650 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f516006f360 0x7f5160152f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:50.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.396+0000 7f515ffff640 1 --2- 172.21.15.27:0/4240500650 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f51600cf490 0x7f51600b6520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:50.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.397+0000 7f515effd640 1 -- 172.21.15.27:0/4240500650 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f51600cf490 msgr2=0x7f51600b6520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:50.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.397+0000 7f515effd640 1 --2- 172.21.15.27:0/4240500650 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f51600cf490 0x7f51600b6520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:50.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.397+0000 7f515effd640 1 -- 172.21.15.27:0/4240500650 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f516006f360 msgr2=0x7f5160152f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:50.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.397+0000 7f515effd640 1 --2- 172.21.15.27:0/4240500650 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f516006f360 0x7f5160152f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:50.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.397+0000 7f515effd640 1 -- 172.21.15.27:0/4240500650 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51600cb850 con 0x7f51600b1ca0 2024-09-12T03:21:50.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.397+0000 7f515effd640 1 --2- 172.21.15.27:0/4240500650 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f51600b1ca0 0x7f51600cd710 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f515400c820 tx=0x7f515400cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:50.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.397+0000 7f5147fff640 1 -- 172.21.15.27:0/4240500650 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5154015070 con 0x7f51600b1ca0 2024-09-12T03:21:50.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.397+0000 7f5147fff640 1 -- 172.21.15.27:0/4240500650 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5154007a30 con 0x7f51600b1ca0 2024-09-12T03:21:50.400 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.397+0000 7f5165b6d640 1 -- 172.21.15.27:0/4240500650 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51600cbb10 con 0x7f51600b1ca0 2024-09-12T03:21:50.400 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.397+0000 7f5147fff640 1 -- 172.21.15.27:0/4240500650 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5154007de0 con 0x7f51600b1ca0 2024-09-12T03:21:50.400 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.397+0000 7f5165b6d640 1 -- 172.21.15.27:0/4240500650 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51600cc0d0 con 0x7f51600b1ca0 2024-09-12T03:21:50.400 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.398+0000 7f5165b6d640 1 -- 172.21.15.27:0/4240500650 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51600cfc40 con 0x7f51600b1ca0 2024-09-12T03:21:50.401 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.398+0000 7f5147fff640 1 -- 172.21.15.27:0/4240500650 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 19) ==== 90855+0+0 (secure 0 0 0) 0x7f5154018070 con 0x7f51600b1ca0 2024-09-12T03:21:50.401 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.399+0000 7f5147fff640 1 --2- 172.21.15.27:0/4240500650 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f514c06c220 0x7f514c06e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:50.401 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.399+0000 7f515f7fe640 1 --2- 172.21.15.27:0/4240500650 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f514c06c220 0x7f514c06e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:50.401 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.399+0000 7f5147fff640 1 -- 172.21.15.27:0/4240500650 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5737+0+0 (secure 0 0 0) 0x7f515408b4e0 con 0x7f51600b1ca0 2024-09-12T03:21:50.401 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.400+0000 7f515f7fe640 1 --2- 172.21.15.27:0/4240500650 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f514c06c220 0x7f514c06e6e0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f5150034000 tx=0x7f515002e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:50.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.403+0000 7f5147fff640 1 -- 172.21.15.27:0/4240500650 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f5154059130 con 0x7f51600b1ca0 2024-09-12T03:21:50.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.575+0000 7f5165b6d640 1 -- 172.21.15.27:0/4240500650 --> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f51600b77a0 con 0x7f514c06c220 2024-09-12T03:21:50.581 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.581+0000 7f5147fff640 1 -- 172.21.15.27:0/4240500650 <== mgr.4113 v2:172.21.15.27:6836/1226920526 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+240244 (secure 0 0 0) 0x7f51600b77a0 con 0x7f514c06c220 2024-09-12T03:21:50.584 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:21:50.585 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2024-09-12T03:21:50.587 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.584+0000 7f5165b6d640 1 -- 172.21.15.27:0/4240500650 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f514c06c220 msgr2=0x7f514c06e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:50.587 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.584+0000 7f5165b6d640 1 --2- 172.21.15.27:0/4240500650 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f514c06c220 0x7f514c06e6e0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f5150034000 tx=0x7f515002e000 comp rx=0 tx=0).stop 2024-09-12T03:21:50.587 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.584+0000 7f5165b6d640 1 -- 172.21.15.27:0/4240500650 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f51600b1ca0 msgr2=0x7f51600cd710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:50.587 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.584+0000 7f5165b6d640 1 --2- 172.21.15.27:0/4240500650 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f51600b1ca0 0x7f51600cd710 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f515400c820 tx=0x7f515400cce0 comp rx=0 tx=0).stop 2024-09-12T03:21:50.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.584+0000 7f5165b6d640 1 -- 172.21.15.27:0/4240500650 shutdown_connections 2024-09-12T03:21:50.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.584+0000 7f5165b6d640 1 --2- 172.21.15.27:0/4240500650 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f51600cf490 0x7f51600b6520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:50.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.584+0000 7f5165b6d640 1 --2- 172.21.15.27:0/4240500650 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f514c06c220 0x7f514c06e6e0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:50.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.584+0000 7f5165b6d640 1 --2- 172.21.15.27:0/4240500650 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f51600b1ca0 0x7f51600cd710 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:50.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.584+0000 7f5165b6d640 1 --2- 172.21.15.27:0/4240500650 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f516006f360 0x7f5160152f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:50.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.584+0000 7f5165b6d640 1 -- 172.21.15.27:0/4240500650 >> 172.21.15.27:0/4240500650 conn(0x7f5160073800 msgr2=0x7f516015f250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:50.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.584+0000 7f5165b6d640 1 -- 172.21.15.27:0/4240500650 shutdown_connections 2024-09-12T03:21:50.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.585+0000 7f5165b6d640 1 -- 172.21.15.27:0/4240500650 wait complete. 2024-09-12T03:21:50.595 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":95,"stamp":"2024-09-12T03:21:49.115228+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":8,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":278136,"kb_used_data":11964,"kb_used_omap":12,"kb_used_meta":265971,"kb_avail":754696584,"statfs":{"total":773094113280,"available":772809302016,"internally_reserved":0,"allocated":12251136,"data_stored":6038243,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":272354864},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000971"},"pg_stats":[{"pgid":"19.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:21:46.237118+0000","last_change":"2024-09-12T03:21:46.237118+0000","last_active":"2024-09-12T03:21:46.237118+0000","last_peered":"2024-09-12T03:21:46.237118+0000","last_clean":"2024-09-12T03:21:46.237118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:21:46.237118+0000","last_undegraded":"2024-09-12T03:21:46.237118+0000","last_fullsized":"2024-09-12T03:21:46.237118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:21:46.237118+0000","last_change":"2024-09-12T03:21:46.237118+0000","last_active":"2024-09-12T03:21:46.237118+0000","last_peered":"2024-09-12T03:21:46.237118+0000","last_clean":"2024-09-12T03:21:46.237118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:21:46.237118+0000","last_undegraded":"2024-09-12T03:21:46.237118+0000","last_fullsized":"2024-09-12T03:21:46.237118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:21:46.237118+0000","last_change":"2024-09-12T03:21:46.237118+0000","last_active":"2024-09-12T03:21:46.237118+0000","last_peered":"2024-09-12T03:21:46.237118+0000","last_clean":"2024-09-12T03:21:46.237118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:21:46.237118+0000","last_undegraded":"2024-09-12T03:21:46.237118+0000","last_fullsized":"2024-09-12T03:21:46.237118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:21:46.237118+0000","last_change":"2024-09-12T03:21:46.237118+0000","last_active":"2024-09-12T03:21:46.237118+0000","last_peered":"2024-09-12T03:21:46.237118+0000","last_clean":"2024-09-12T03:21:46.237118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:21:46.237118+0000","last_undegraded":"2024-09-12T03:21:46.237118+0000","last_fullsized":"2024-09-12T03:21:46.237118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:21:46.237118+0000","last_change":"2024-09-12T03:21:46.237118+0000","last_active":"2024-09-12T03:21:46.237118+0000","last_peered":"2024-09-12T03:21:46.237118+0000","last_clean":"2024-09-12T03:21:46.237118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:21:46.237118+0000","last_undegraded":"2024-09-12T03:21:46.237118+0000","last_fullsized":"2024-09-12T03:21:46.237118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:21:46.237118+0000","last_change":"2024-09-12T03:21:46.237118+0000","last_active":"2024-09-12T03:21:46.237118+0000","last_peered":"2024-09-12T03:21:46.237118+0000","last_clean":"2024-09-12T03:21:46.237118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:21:46.237118+0000","last_undegraded":"2024-09-12T03:21:46.237118+0000","last_fullsized":"2024-09-12T03:21:46.237118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:21:46.237118+0000","last_change":"2024-09-12T03:21:46.237118+0000","last_active":"2024-09-12T03:21:46.237118+0000","last_peered":"2024-09-12T03:21:46.237118+0000","last_clean":"2024-09-12T03:21:46.237118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:21:46.237118+0000","last_undegraded":"2024-09-12T03:21:46.237118+0000","last_fullsized":"2024-09-12T03:21:46.237118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:21:46.237118+0000","last_change":"2024-09-12T03:21:46.237118+0000","last_active":"2024-09-12T03:21:46.237118+0000","last_peered":"2024-09-12T03:21:46.237118+0000","last_clean":"2024-09-12T03:21:46.237118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:21:46.237118+0000","last_undegraded":"2024-09-12T03:21:46.237118+0000","last_fullsized":"2024-09-12T03:21:46.237118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:21:46.237118+0000","last_change":"2024-09-12T03:21:46.237118+0000","last_active":"2024-09-12T03:21:46.237118+0000","last_peered":"2024-09-12T03:21:46.237118+0000","last_clean":"2024-09-12T03:21:46.237118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:21:46.237118+0000","last_undegraded":"2024-09-12T03:21:46.237118+0000","last_fullsized":"2024-09-12T03:21:46.237118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:21:46.237118+0000","last_change":"2024-09-12T03:21:46.237118+0000","last_active":"2024-09-12T03:21:46.237118+0000","last_peered":"2024-09-12T03:21:46.237118+0000","last_clean":"2024-09-12T03:21:46.237118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:21:46.237118+0000","last_undegraded":"2024-09-12T03:21:46.237118+0000","last_fullsized":"2024-09-12T03:21:46.237118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:21:46.237118+0000","last_change":"2024-09-12T03:21:46.237118+0000","last_active":"2024-09-12T03:21:46.237118+0000","last_peered":"2024-09-12T03:21:46.237118+0000","last_clean":"2024-09-12T03:21:46.237118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:21:46.237118+0000","last_undegraded":"2024-09-12T03:21:46.237118+0000","last_fullsized":"2024-09-12T03:21:46.237118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:21:46.237118+0000","last_change":"2024-09-12T03:21:46.237118+0000","last_active":"2024-09-12T03:21:46.237118+0000","last_peered":"2024-09-12T03:21:46.237118+0000","last_clean":"2024-09-12T03:21:46.237118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:21:46.237118+0000","last_undegraded":"2024-09-12T03:21:46.237118+0000","last_fullsized":"2024-09-12T03:21:46.237118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:21:46.237118+0000","last_change":"2024-09-12T03:21:46.237118+0000","last_active":"2024-09-12T03:21:46.237118+0000","last_peered":"2024-09-12T03:21:46.237118+0000","last_clean":"2024-09-12T03:21:46.237118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:21:46.237118+0000","last_undegraded":"2024-09-12T03:21:46.237118+0000","last_fullsized":"2024-09-12T03:21:46.237118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:21:46.237118+0000","last_change":"2024-09-12T03:21:46.237118+0000","last_active":"2024-09-12T03:21:46.237118+0000","last_peered":"2024-09-12T03:21:46.237118+0000","last_clean":"2024-09-12T03:21:46.237118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:21:46.237118+0000","last_undegraded":"2024-09-12T03:21:46.237118+0000","last_fullsized":"2024-09-12T03:21:46.237118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:21:46.237118+0000","last_change":"2024-09-12T03:21:46.237118+0000","last_active":"2024-09-12T03:21:46.237118+0000","last_peered":"2024-09-12T03:21:46.237118+0000","last_clean":"2024-09-12T03:21:46.237118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:21:46.237118+0000","last_undegraded":"2024-09-12T03:21:46.237118+0000","last_fullsized":"2024-09-12T03:21:46.237118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:21:46.237118+0000","last_change":"2024-09-12T03:21:46.237118+0000","last_active":"2024-09-12T03:21:46.237118+0000","last_peered":"2024-09-12T03:21:46.237118+0000","last_clean":"2024-09-12T03:21:46.237118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:21:46.237118+0000","last_undegraded":"2024-09-12T03:21:46.237118+0000","last_fullsized":"2024-09-12T03:21:46.237118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:21:46.237118+0000","last_change":"2024-09-12T03:21:46.237118+0000","last_active":"2024-09-12T03:21:46.237118+0000","last_peered":"2024-09-12T03:21:46.237118+0000","last_clean":"2024-09-12T03:21:46.237118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:21:46.237118+0000","last_undegraded":"2024-09-12T03:21:46.237118+0000","last_fullsized":"2024-09-12T03:21:46.237118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:21:46.237118+0000","last_change":"2024-09-12T03:21:46.237118+0000","last_active":"2024-09-12T03:21:46.237118+0000","last_peered":"2024-09-12T03:21:46.237118+0000","last_clean":"2024-09-12T03:21:46.237118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:21:46.237118+0000","last_undegraded":"2024-09-12T03:21:46.237118+0000","last_fullsized":"2024-09-12T03:21:46.237118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:21:46.237118+0000","last_change":"2024-09-12T03:21:46.237118+0000","last_active":"2024-09-12T03:21:46.237118+0000","last_peered":"2024-09-12T03:21:46.237118+0000","last_clean":"2024-09-12T03:21:46.237118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:21:46.237118+0000","last_undegraded":"2024-09-12T03:21:46.237118+0000","last_fullsized":"2024-09-12T03:21:46.237118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:21:46.237118+0000","last_change":"2024-09-12T03:21:46.237118+0000","last_active":"2024-09-12T03:21:46.237118+0000","last_peered":"2024-09-12T03:21:46.237118+0000","last_clean":"2024-09-12T03:21:46.237118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:21:46.237118+0000","last_undegraded":"2024-09-12T03:21:46.237118+0000","last_fullsized":"2024-09-12T03:21:46.237118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:21:46.237118+0000","last_change":"2024-09-12T03:21:46.237118+0000","last_active":"2024-09-12T03:21:46.237118+0000","last_peered":"2024-09-12T03:21:46.237118+0000","last_clean":"2024-09-12T03:21:46.237118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:21:46.237118+0000","last_undegraded":"2024-09-12T03:21:46.237118+0000","last_fullsized":"2024-09-12T03:21:46.237118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:21:46.237118+0000","last_change":"2024-09-12T03:21:46.237118+0000","last_active":"2024-09-12T03:21:46.237118+0000","last_peered":"2024-09-12T03:21:46.237118+0000","last_clean":"2024-09-12T03:21:46.237118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:21:46.237118+0000","last_undegraded":"2024-09-12T03:21:46.237118+0000","last_fullsized":"2024-09-12T03:21:46.237118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:21:46.237118+0000","last_change":"2024-09-12T03:21:46.237118+0000","last_active":"2024-09-12T03:21:46.237118+0000","last_peered":"2024-09-12T03:21:46.237118+0000","last_clean":"2024-09-12T03:21:46.237118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:21:46.237118+0000","last_undegraded":"2024-09-12T03:21:46.237118+0000","last_fullsized":"2024-09-12T03:21:46.237118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:21:46.237118+0000","last_change":"2024-09-12T03:21:46.237118+0000","last_active":"2024-09-12T03:21:46.237118+0000","last_peered":"2024-09-12T03:21:46.237118+0000","last_clean":"2024-09-12T03:21:46.237118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:21:46.237118+0000","last_undegraded":"2024-09-12T03:21:46.237118+0000","last_fullsized":"2024-09-12T03:21:46.237118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:21:46.237118+0000","last_change":"2024-09-12T03:21:46.237118+0000","last_active":"2024-09-12T03:21:46.237118+0000","last_peered":"2024-09-12T03:21:46.237118+0000","last_clean":"2024-09-12T03:21:46.237118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:21:46.237118+0000","last_undegraded":"2024-09-12T03:21:46.237118+0000","last_fullsized":"2024-09-12T03:21:46.237118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:21:46.237118+0000","last_change":"2024-09-12T03:21:46.237118+0000","last_active":"2024-09-12T03:21:46.237118+0000","last_peered":"2024-09-12T03:21:46.237118+0000","last_clean":"2024-09-12T03:21:46.237118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:21:46.237118+0000","last_undegraded":"2024-09-12T03:21:46.237118+0000","last_fullsized":"2024-09-12T03:21:46.237118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:21:46.237118+0000","last_change":"2024-09-12T03:21:46.237118+0000","last_active":"2024-09-12T03:21:46.237118+0000","last_peered":"2024-09-12T03:21:46.237118+0000","last_clean":"2024-09-12T03:21:46.237118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:21:46.237118+0000","last_undegraded":"2024-09-12T03:21:46.237118+0000","last_fullsized":"2024-09-12T03:21:46.237118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:21:46.237118+0000","last_change":"2024-09-12T03:21:46.237118+0000","last_active":"2024-09-12T03:21:46.237118+0000","last_peered":"2024-09-12T03:21:46.237118+0000","last_clean":"2024-09-12T03:21:46.237118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:21:46.237118+0000","last_undegraded":"2024-09-12T03:21:46.237118+0000","last_fullsized":"2024-09-12T03:21:46.237118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:21:46.237118+0000","last_change":"2024-09-12T03:21:46.237118+0000","last_active":"2024-09-12T03:21:46.237118+0000","last_peered":"2024-09-12T03:21:46.237118+0000","last_clean":"2024-09-12T03:21:46.237118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:21:46.237118+0000","last_undegraded":"2024-09-12T03:21:46.237118+0000","last_fullsized":"2024-09-12T03:21:46.237118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:21:46.237118+0000","last_change":"2024-09-12T03:21:46.237118+0000","last_active":"2024-09-12T03:21:46.237118+0000","last_peered":"2024-09-12T03:21:46.237118+0000","last_clean":"2024-09-12T03:21:46.237118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:21:46.237118+0000","last_undegraded":"2024-09-12T03:21:46.237118+0000","last_fullsized":"2024-09-12T03:21:46.237118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:21:46.237118+0000","last_change":"2024-09-12T03:21:46.237118+0000","last_active":"2024-09-12T03:21:46.237118+0000","last_peered":"2024-09-12T03:21:46.237118+0000","last_clean":"2024-09-12T03:21:46.237118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:21:46.237118+0000","last_undegraded":"2024-09-12T03:21:46.237118+0000","last_fullsized":"2024-09-12T03:21:46.237118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:21:46.237118+0000","last_change":"2024-09-12T03:21:46.237118+0000","last_active":"2024-09-12T03:21:46.237118+0000","last_peered":"2024-09-12T03:21:46.237118+0000","last_clean":"2024-09-12T03:21:46.237118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:21:46.237118+0000","last_undegraded":"2024-09-12T03:21:46.237118+0000","last_fullsized":"2024-09-12T03:21:46.237118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:21:46.237118+0000","last_change":"2024-09-12T03:21:46.237118+0000","last_active":"2024-09-12T03:21:46.237118+0000","last_peered":"2024-09-12T03:21:46.237118+0000","last_clean":"2024-09-12T03:21:46.237118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:21:46.237118+0000","last_undegraded":"2024-09-12T03:21:46.237118+0000","last_fullsized":"2024-09-12T03:21:46.237118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-12T03:21:45.241963+0000","last_change":"2024-09-12T03:21:45.241963+0000","last_active":"2024-09-12T03:21:45.231023+0000","last_peered":"2024-09-12T03:21:45.231023+0000","last_clean":"2024-09-12T03:21:45.231023+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:21:45.241963+0000","last_undegraded":"2024-09-12T03:21:45.241963+0000","last_fullsized":"2024-09-12T03:21:45.241963+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_clean_scrub_stamp":"2024-09-12T03:21:45.231023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:21:46.237118+0000","last_change":"2024-09-12T03:21:46.237118+0000","last_active":"2024-09-12T03:21:46.237118+0000","last_peered":"2024-09-12T03:21:46.237118+0000","last_clean":"2024-09-12T03:21:46.237118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:21:46.237118+0000","last_undegraded":"2024-09-12T03:21:46.237118+0000","last_fullsized":"2024-09-12T03:21:46.237118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-12T03:21:45.244081+0000","last_change":"2024-09-12T03:21:45.244081+0000","last_active":"2024-09-12T03:21:45.231023+0000","last_peered":"2024-09-12T03:21:45.231023+0000","last_clean":"2024-09-12T03:21:45.231023+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:21:45.244081+0000","last_undegraded":"2024-09-12T03:21:45.244081+0000","last_fullsized":"2024-09-12T03:21:45.244081+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_clean_scrub_stamp":"2024-09-12T03:21:45.231023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:21:46.237118+0000","last_change":"2024-09-12T03:21:46.237118+0000","last_active":"2024-09-12T03:21:46.237118+0000","last_peered":"2024-09-12T03:21:46.237118+0000","last_clean":"2024-09-12T03:21:46.237118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:21:46.237118+0000","last_undegraded":"2024-09-12T03:21:46.237118+0000","last_fullsized":"2024-09-12T03:21:46.237118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-12T03:21:45.243768+0000","last_change":"2024-09-12T03:21:45.243768+0000","last_active":"2024-09-12T03:21:45.231023+0000","last_peered":"2024-09-12T03:21:45.231023+0000","last_clean":"2024-09-12T03:21:45.231023+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:21:45.243768+0000","last_undegraded":"2024-09-12T03:21:45.243768+0000","last_fullsized":"2024-09-12T03:21:45.243768+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_clean_scrub_stamp":"2024-09-12T03:21:45.231023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:21:46.237118+0000","last_change":"2024-09-12T03:21:46.237118+0000","last_active":"2024-09-12T03:21:46.237118+0000","last_peered":"2024-09-12T03:21:46.237118+0000","last_clean":"2024-09-12T03:21:46.237118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:21:46.237118+0000","last_undegraded":"2024-09-12T03:21:46.237118+0000","last_fullsized":"2024-09-12T03:21:46.237118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-12T03:21:45.250356+0000","last_change":"2024-09-12T03:21:45.250356+0000","last_active":"2024-09-12T03:21:45.231023+0000","last_peered":"2024-09-12T03:21:45.231023+0000","last_clean":"2024-09-12T03:21:45.231023+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:21:45.250356+0000","last_undegraded":"2024-09-12T03:21:45.250356+0000","last_fullsized":"2024-09-12T03:21:45.250356+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_clean_scrub_stamp":"2024-09-12T03:21:45.231023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:21:46.237118+0000","last_change":"2024-09-12T03:21:46.237118+0000","last_active":"2024-09-12T03:21:46.237118+0000","last_peered":"2024-09-12T03:21:46.237118+0000","last_clean":"2024-09-12T03:21:46.237118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:21:46.237118+0000","last_undegraded":"2024-09-12T03:21:46.237118+0000","last_fullsized":"2024-09-12T03:21:46.237118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-12T03:21:45.251259+0000","last_change":"2024-09-12T03:21:45.251259+0000","last_active":"2024-09-12T03:21:45.231023+0000","last_peered":"2024-09-12T03:21:45.231023+0000","last_clean":"2024-09-12T03:21:45.231023+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:21:45.251259+0000","last_undegraded":"2024-09-12T03:21:45.251259+0000","last_fullsized":"2024-09-12T03:21:45.251259+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_clean_scrub_stamp":"2024-09-12T03:21:45.231023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:21:46.237118+0000","last_change":"2024-09-12T03:21:46.237118+0000","last_active":"2024-09-12T03:21:46.237118+0000","last_peered":"2024-09-12T03:21:46.237118+0000","last_clean":"2024-09-12T03:21:46.237118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:21:46.237118+0000","last_undegraded":"2024-09-12T03:21:46.237118+0000","last_fullsized":"2024-09-12T03:21:46.237118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-12T03:21:45.245853+0000","last_change":"2024-09-12T03:21:45.245853+0000","last_active":"2024-09-12T03:21:45.231023+0000","last_peered":"2024-09-12T03:21:45.231023+0000","last_clean":"2024-09-12T03:21:45.231023+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:21:45.245853+0000","last_undegraded":"2024-09-12T03:21:45.245853+0000","last_fullsized":"2024-09-12T03:21:45.245853+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_clean_scrub_stamp":"2024-09-12T03:21:45.231023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:21:46.237118+0000","last_change":"2024-09-12T03:21:46.237118+0000","last_active":"2024-09-12T03:21:46.237118+0000","last_peered":"2024-09-12T03:21:46.237118+0000","last_clean":"2024-09-12T03:21:46.237118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:21:46.237118+0000","last_undegraded":"2024-09-12T03:21:46.237118+0000","last_fullsized":"2024-09-12T03:21:46.237118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-12T03:21:45.239959+0000","last_change":"2024-09-12T03:21:45.239959+0000","last_active":"2024-09-12T03:21:45.231023+0000","last_peered":"2024-09-12T03:21:45.231023+0000","last_clean":"2024-09-12T03:21:45.231023+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:21:45.239959+0000","last_undegraded":"2024-09-12T03:21:45.239959+0000","last_fullsized":"2024-09-12T03:21:45.239959+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_clean_scrub_stamp":"2024-09-12T03:21:45.231023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:21:46.237118+0000","last_change":"2024-09-12T03:21:46.237118+0000","last_active":"2024-09-12T03:21:46.237118+0000","last_peered":"2024-09-12T03:21:46.237118+0000","last_clean":"2024-09-12T03:21:46.237118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:21:46.237118+0000","last_undegraded":"2024-09-12T03:21:46.237118+0000","last_fullsized":"2024-09-12T03:21:46.237118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-12T03:21:45.250250+0000","last_change":"2024-09-12T03:21:45.250250+0000","last_active":"2024-09-12T03:21:45.231023+0000","last_peered":"2024-09-12T03:21:45.231023+0000","last_clean":"2024-09-12T03:21:45.231023+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:21:45.250250+0000","last_undegraded":"2024-09-12T03:21:45.250250+0000","last_fullsized":"2024-09-12T03:21:45.250250+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_clean_scrub_stamp":"2024-09-12T03:21:45.231023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:21:46.237118+0000","last_change":"2024-09-12T03:21:46.237118+0000","last_active":"2024-09-12T03:21:46.237118+0000","last_peered":"2024-09-12T03:21:46.237118+0000","last_clean":"2024-09-12T03:21:46.237118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:21:46.237118+0000","last_undegraded":"2024-09-12T03:21:46.237118+0000","last_fullsized":"2024-09-12T03:21:46.237118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-12T03:21:45.244419+0000","last_change":"2024-09-12T03:21:45.244419+0000","last_active":"2024-09-12T03:21:45.231023+0000","last_peered":"2024-09-12T03:21:45.231023+0000","last_clean":"2024-09-12T03:21:45.231023+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:21:45.244419+0000","last_undegraded":"2024-09-12T03:21:45.244419+0000","last_fullsized":"2024-09-12T03:21:45.244419+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_clean_scrub_stamp":"2024-09-12T03:21:45.231023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:21:46.237118+0000","last_change":"2024-09-12T03:21:46.237118+0000","last_active":"2024-09-12T03:21:46.237118+0000","last_peered":"2024-09-12T03:21:46.237118+0000","last_clean":"2024-09-12T03:21:46.237118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:21:46.237118+0000","last_undegraded":"2024-09-12T03:21:46.237118+0000","last_fullsized":"2024-09-12T03:21:46.237118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-12T03:21:45.245487+0000","last_change":"2024-09-12T03:21:45.245487+0000","last_active":"2024-09-12T03:21:45.231023+0000","last_peered":"2024-09-12T03:21:45.231023+0000","last_clean":"2024-09-12T03:21:45.231023+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:21:45.245487+0000","last_undegraded":"2024-09-12T03:21:45.245487+0000","last_fullsized":"2024-09-12T03:21:45.245487+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_clean_scrub_stamp":"2024-09-12T03:21:45.231023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:21:46.237118+0000","last_change":"2024-09-12T03:21:46.237118+0000","last_active":"2024-09-12T03:21:46.237118+0000","last_peered":"2024-09-12T03:21:46.237118+0000","last_clean":"2024-09-12T03:21:46.237118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:21:46.237118+0000","last_undegraded":"2024-09-12T03:21:46.237118+0000","last_fullsized":"2024-09-12T03:21:46.237118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-12T03:21:45.243429+0000","last_change":"2024-09-12T03:21:45.243429+0000","last_active":"2024-09-12T03:21:45.231023+0000","last_peered":"2024-09-12T03:21:45.231023+0000","last_clean":"2024-09-12T03:21:45.231023+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:21:45.243429+0000","last_undegraded":"2024-09-12T03:21:45.243429+0000","last_fullsized":"2024-09-12T03:21:45.243429+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_clean_scrub_stamp":"2024-09-12T03:21:45.231023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:21:46.237118+0000","last_change":"2024-09-12T03:21:46.237118+0000","last_active":"2024-09-12T03:21:46.237118+0000","last_peered":"2024-09-12T03:21:46.237118+0000","last_clean":"2024-09-12T03:21:46.237118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:21:46.237118+0000","last_undegraded":"2024-09-12T03:21:46.237118+0000","last_fullsized":"2024-09-12T03:21:46.237118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-12T03:21:45.239970+0000","last_change":"2024-09-12T03:21:45.239970+0000","last_active":"2024-09-12T03:21:45.231023+0000","last_peered":"2024-09-12T03:21:45.231023+0000","last_clean":"2024-09-12T03:21:45.231023+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:21:45.239970+0000","last_undegraded":"2024-09-12T03:21:45.239970+0000","last_fullsized":"2024-09-12T03:21:45.239970+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_clean_scrub_stamp":"2024-09-12T03:21:45.231023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:21:46.237118+0000","last_change":"2024-09-12T03:21:46.237118+0000","last_active":"2024-09-12T03:21:46.237118+0000","last_peered":"2024-09-12T03:21:46.237118+0000","last_clean":"2024-09-12T03:21:46.237118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:21:46.237118+0000","last_undegraded":"2024-09-12T03:21:46.237118+0000","last_fullsized":"2024-09-12T03:21:46.237118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-12T03:21:45.241108+0000","last_change":"2024-09-12T03:21:45.241108+0000","last_active":"2024-09-12T03:21:45.231023+0000","last_peered":"2024-09-12T03:21:45.231023+0000","last_clean":"2024-09-12T03:21:45.231023+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:21:45.241108+0000","last_undegraded":"2024-09-12T03:21:45.241108+0000","last_fullsized":"2024-09-12T03:21:45.241108+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_clean_scrub_stamp":"2024-09-12T03:21:45.231023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:21:46.237118+0000","last_change":"2024-09-12T03:21:46.237118+0000","last_active":"2024-09-12T03:21:46.237118+0000","last_peered":"2024-09-12T03:21:46.237118+0000","last_clean":"2024-09-12T03:21:46.237118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:21:46.237118+0000","last_undegraded":"2024-09-12T03:21:46.237118+0000","last_fullsized":"2024-09-12T03:21:46.237118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-12T03:21:45.250329+0000","last_change":"2024-09-12T03:21:45.250329+0000","last_active":"2024-09-12T03:21:45.231023+0000","last_peered":"2024-09-12T03:21:45.231023+0000","last_clean":"2024-09-12T03:21:45.231023+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:21:45.250329+0000","last_undegraded":"2024-09-12T03:21:45.250329+0000","last_fullsized":"2024-09-12T03:21:45.250329+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_clean_scrub_stamp":"2024-09-12T03:21:45.231023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:21:46.237118+0000","last_change":"2024-09-12T03:21:46.237118+0000","last_active":"2024-09-12T03:21:46.237118+0000","last_peered":"2024-09-12T03:21:46.237118+0000","last_clean":"2024-09-12T03:21:46.237118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:21:46.237118+0000","last_undegraded":"2024-09-12T03:21:46.237118+0000","last_fullsized":"2024-09-12T03:21:46.237118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-12T03:21:45.239970+0000","last_change":"2024-09-12T03:21:45.239970+0000","last_active":"2024-09-12T03:21:45.231023+0000","last_peered":"2024-09-12T03:21:45.231023+0000","last_clean":"2024-09-12T03:21:45.231023+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:21:45.239970+0000","last_undegraded":"2024-09-12T03:21:45.239970+0000","last_fullsized":"2024-09-12T03:21:45.239970+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_clean_scrub_stamp":"2024-09-12T03:21:45.231023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:21:46.237118+0000","last_change":"2024-09-12T03:21:46.237118+0000","last_active":"2024-09-12T03:21:46.237118+0000","last_peered":"2024-09-12T03:21:46.237118+0000","last_clean":"2024-09-12T03:21:46.237118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:21:46.237118+0000","last_undegraded":"2024-09-12T03:21:46.237118+0000","last_fullsized":"2024-09-12T03:21:46.237118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-12T03:21:45.243798+0000","last_change":"2024-09-12T03:21:45.243798+0000","last_active":"2024-09-12T03:21:45.231023+0000","last_peered":"2024-09-12T03:21:45.231023+0000","last_clean":"2024-09-12T03:21:45.231023+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:21:45.243798+0000","last_undegraded":"2024-09-12T03:21:45.243798+0000","last_fullsized":"2024-09-12T03:21:45.243798+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_clean_scrub_stamp":"2024-09-12T03:21:45.231023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:21:46.237118+0000","last_change":"2024-09-12T03:21:46.237118+0000","last_active":"2024-09-12T03:21:46.237118+0000","last_peered":"2024-09-12T03:21:46.237118+0000","last_clean":"2024-09-12T03:21:46.237118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:21:46.237118+0000","last_undegraded":"2024-09-12T03:21:46.237118+0000","last_fullsized":"2024-09-12T03:21:46.237118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-12T03:21:45.243777+0000","last_change":"2024-09-12T03:21:45.243777+0000","last_active":"2024-09-12T03:21:45.231023+0000","last_peered":"2024-09-12T03:21:45.231023+0000","last_clean":"2024-09-12T03:21:45.231023+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:21:45.243777+0000","last_undegraded":"2024-09-12T03:21:45.243777+0000","last_fullsized":"2024-09-12T03:21:45.243777+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_clean_scrub_stamp":"2024-09-12T03:21:45.231023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:21:46.237118+0000","last_change":"2024-09-12T03:21:46.237118+0000","last_active":"2024-09-12T03:21:46.237118+0000","last_peered":"2024-09-12T03:21:46.237118+0000","last_clean":"2024-09-12T03:21:46.237118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:21:46.237118+0000","last_undegraded":"2024-09-12T03:21:46.237118+0000","last_fullsized":"2024-09-12T03:21:46.237118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-12T03:21:45.239959+0000","last_change":"2024-09-12T03:21:45.239959+0000","last_active":"2024-09-12T03:21:45.231023+0000","last_peered":"2024-09-12T03:21:45.231023+0000","last_clean":"2024-09-12T03:21:45.231023+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:21:45.239959+0000","last_undegraded":"2024-09-12T03:21:45.239959+0000","last_fullsized":"2024-09-12T03:21:45.239959+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_clean_scrub_stamp":"2024-09-12T03:21:45.231023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:21:46.237118+0000","last_change":"2024-09-12T03:21:46.237118+0000","last_active":"2024-09-12T03:21:46.237118+0000","last_peered":"2024-09-12T03:21:46.237118+0000","last_clean":"2024-09-12T03:21:46.237118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:21:46.237118+0000","last_undegraded":"2024-09-12T03:21:46.237118+0000","last_fullsized":"2024-09-12T03:21:46.237118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-12T03:21:45.244432+0000","last_change":"2024-09-12T03:21:45.244432+0000","last_active":"2024-09-12T03:21:45.231023+0000","last_peered":"2024-09-12T03:21:45.231023+0000","last_clean":"2024-09-12T03:21:45.231023+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:21:45.244432+0000","last_undegraded":"2024-09-12T03:21:45.244432+0000","last_fullsized":"2024-09-12T03:21:45.244432+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_clean_scrub_stamp":"2024-09-12T03:21:45.231023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:21:46.237118+0000","last_change":"2024-09-12T03:21:46.237118+0000","last_active":"2024-09-12T03:21:46.237118+0000","last_peered":"2024-09-12T03:21:46.237118+0000","last_clean":"2024-09-12T03:21:46.237118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:21:46.237118+0000","last_undegraded":"2024-09-12T03:21:46.237118+0000","last_fullsized":"2024-09-12T03:21:46.237118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-12T03:21:45.243467+0000","last_change":"2024-09-12T03:21:45.243467+0000","last_active":"2024-09-12T03:21:45.231023+0000","last_peered":"2024-09-12T03:21:45.231023+0000","last_clean":"2024-09-12T03:21:45.231023+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:21:45.243467+0000","last_undegraded":"2024-09-12T03:21:45.243467+0000","last_fullsized":"2024-09-12T03:21:45.243467+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_clean_scrub_stamp":"2024-09-12T03:21:45.231023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:21:46.237118+0000","last_change":"2024-09-12T03:21:46.237118+0000","last_active":"2024-09-12T03:21:46.237118+0000","last_peered":"2024-09-12T03:21:46.237118+0000","last_clean":"2024-09-12T03:21:46.237118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:21:46.237118+0000","last_undegraded":"2024-09-12T03:21:46.237118+0000","last_fullsized":"2024-09-12T03:21:46.237118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-12T03:21:45.245048+0000","last_change":"2024-09-12T03:21:45.245048+0000","last_active":"2024-09-12T03:21:45.231023+0000","last_peered":"2024-09-12T03:21:45.231023+0000","last_clean":"2024-09-12T03:21:45.231023+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:21:45.245048+0000","last_undegraded":"2024-09-12T03:21:45.245048+0000","last_fullsized":"2024-09-12T03:21:45.245048+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_clean_scrub_stamp":"2024-09-12T03:21:45.231023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:21:46.237118+0000","last_change":"2024-09-12T03:21:46.237118+0000","last_active":"2024-09-12T03:21:46.237118+0000","last_peered":"2024-09-12T03:21:46.237118+0000","last_clean":"2024-09-12T03:21:46.237118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:21:46.237118+0000","last_undegraded":"2024-09-12T03:21:46.237118+0000","last_fullsized":"2024-09-12T03:21:46.237118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-12T03:21:45.245336+0000","last_change":"2024-09-12T03:21:45.245336+0000","last_active":"2024-09-12T03:21:45.231023+0000","last_peered":"2024-09-12T03:21:45.231023+0000","last_clean":"2024-09-12T03:21:45.231023+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:21:45.245336+0000","last_undegraded":"2024-09-12T03:21:45.245336+0000","last_fullsized":"2024-09-12T03:21:45.245336+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_clean_scrub_stamp":"2024-09-12T03:21:45.231023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:21:46.237118+0000","last_change":"2024-09-12T03:21:46.237118+0000","last_active":"2024-09-12T03:21:46.237118+0000","last_peered":"2024-09-12T03:21:46.237118+0000","last_clean":"2024-09-12T03:21:46.237118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:21:46.237118+0000","last_undegraded":"2024-09-12T03:21:46.237118+0000","last_fullsized":"2024-09-12T03:21:46.237118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-12T03:21:45.245243+0000","last_change":"2024-09-12T03:21:45.245243+0000","last_active":"2024-09-12T03:21:45.231023+0000","last_peered":"2024-09-12T03:21:45.231023+0000","last_clean":"2024-09-12T03:21:45.231023+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:21:45.245243+0000","last_undegraded":"2024-09-12T03:21:45.245243+0000","last_fullsized":"2024-09-12T03:21:45.245243+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_clean_scrub_stamp":"2024-09-12T03:21:45.231023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:21:46.237118+0000","last_change":"2024-09-12T03:21:46.237118+0000","last_active":"2024-09-12T03:21:46.237118+0000","last_peered":"2024-09-12T03:21:46.237118+0000","last_clean":"2024-09-12T03:21:46.237118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:21:46.237118+0000","last_undegraded":"2024-09-12T03:21:46.237118+0000","last_fullsized":"2024-09-12T03:21:46.237118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-12T03:21:45.240778+0000","last_change":"2024-09-12T03:21:45.240778+0000","last_active":"2024-09-12T03:21:45.231023+0000","last_peered":"2024-09-12T03:21:45.231023+0000","last_clean":"2024-09-12T03:21:45.231023+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:21:45.240778+0000","last_undegraded":"2024-09-12T03:21:45.240778+0000","last_fullsized":"2024-09-12T03:21:45.240778+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_clean_scrub_stamp":"2024-09-12T03:21:45.231023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:21:46.237118+0000","last_change":"2024-09-12T03:21:46.237118+0000","last_active":"2024-09-12T03:21:46.237118+0000","last_peered":"2024-09-12T03:21:46.237118+0000","last_clean":"2024-09-12T03:21:46.237118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:21:46.237118+0000","last_undegraded":"2024-09-12T03:21:46.237118+0000","last_fullsized":"2024-09-12T03:21:46.237118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-12T03:21:45.242034+0000","last_change":"2024-09-12T03:21:45.242034+0000","last_active":"2024-09-12T03:21:45.231023+0000","last_peered":"2024-09-12T03:21:45.231023+0000","last_clean":"2024-09-12T03:21:45.231023+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:21:45.242034+0000","last_undegraded":"2024-09-12T03:21:45.242034+0000","last_fullsized":"2024-09-12T03:21:45.242034+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_clean_scrub_stamp":"2024-09-12T03:21:45.231023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:21:46.237118+0000","last_change":"2024-09-12T03:21:46.237118+0000","last_active":"2024-09-12T03:21:46.237118+0000","last_peered":"2024-09-12T03:21:46.237118+0000","last_clean":"2024-09-12T03:21:46.237118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:21:46.237118+0000","last_undegraded":"2024-09-12T03:21:46.237118+0000","last_fullsized":"2024-09-12T03:21:46.237118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-12T03:21:45.250336+0000","last_change":"2024-09-12T03:21:45.250336+0000","last_active":"2024-09-12T03:21:45.231023+0000","last_peered":"2024-09-12T03:21:45.231023+0000","last_clean":"2024-09-12T03:21:45.231023+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:21:45.250336+0000","last_undegraded":"2024-09-12T03:21:45.250336+0000","last_fullsized":"2024-09-12T03:21:45.250336+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_clean_scrub_stamp":"2024-09-12T03:21:45.231023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:21:46.237118+0000","last_change":"2024-09-12T03:21:46.237118+0000","last_active":"2024-09-12T03:21:46.237118+0000","last_peered":"2024-09-12T03:21:46.237118+0000","last_clean":"2024-09-12T03:21:46.237118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:21:46.237118+0000","last_undegraded":"2024-09-12T03:21:46.237118+0000","last_fullsized":"2024-09-12T03:21:46.237118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-12T03:21:45.243710+0000","last_change":"2024-09-12T03:21:45.243710+0000","last_active":"2024-09-12T03:21:45.231023+0000","last_peered":"2024-09-12T03:21:45.231023+0000","last_clean":"2024-09-12T03:21:45.231023+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:21:45.243710+0000","last_undegraded":"2024-09-12T03:21:45.243710+0000","last_fullsized":"2024-09-12T03:21:45.243710+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_clean_scrub_stamp":"2024-09-12T03:21:45.231023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:21:46.237118+0000","last_change":"2024-09-12T03:21:46.237118+0000","last_active":"2024-09-12T03:21:46.237118+0000","last_peered":"2024-09-12T03:21:46.237118+0000","last_clean":"2024-09-12T03:21:46.237118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:21:46.237118+0000","last_undegraded":"2024-09-12T03:21:46.237118+0000","last_fullsized":"2024-09-12T03:21:46.237118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-12T03:21:45.250313+0000","last_change":"2024-09-12T03:21:45.250313+0000","last_active":"2024-09-12T03:21:45.231023+0000","last_peered":"2024-09-12T03:21:45.231023+0000","last_clean":"2024-09-12T03:21:45.231023+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:21:45.250313+0000","last_undegraded":"2024-09-12T03:21:45.250313+0000","last_fullsized":"2024-09-12T03:21:45.250313+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_clean_scrub_stamp":"2024-09-12T03:21:45.231023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:21:46.237118+0000","last_change":"2024-09-12T03:21:46.237118+0000","last_active":"2024-09-12T03:21:46.237118+0000","last_peered":"2024-09-12T03:21:46.237118+0000","last_clean":"2024-09-12T03:21:46.237118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:21:46.237118+0000","last_undegraded":"2024-09-12T03:21:46.237118+0000","last_fullsized":"2024-09-12T03:21:46.237118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-12T03:21:45.241930+0000","last_change":"2024-09-12T03:21:45.241930+0000","last_active":"2024-09-12T03:21:45.231023+0000","last_peered":"2024-09-12T03:21:45.231023+0000","last_clean":"2024-09-12T03:21:45.231023+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:21:45.241930+0000","last_undegraded":"2024-09-12T03:21:45.241930+0000","last_fullsized":"2024-09-12T03:21:45.241930+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_clean_scrub_stamp":"2024-09-12T03:21:45.231023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:21:46.237118+0000","last_change":"2024-09-12T03:21:46.237118+0000","last_active":"2024-09-12T03:21:46.237118+0000","last_peered":"2024-09-12T03:21:46.237118+0000","last_clean":"2024-09-12T03:21:46.237118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:21:46.237118+0000","last_undegraded":"2024-09-12T03:21:46.237118+0000","last_fullsized":"2024-09-12T03:21:46.237118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-12T03:21:45.244859+0000","last_change":"2024-09-12T03:21:45.244859+0000","last_active":"2024-09-12T03:21:45.231023+0000","last_peered":"2024-09-12T03:21:45.231023+0000","last_clean":"2024-09-12T03:21:45.231023+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:21:45.244859+0000","last_undegraded":"2024-09-12T03:21:45.244859+0000","last_fullsized":"2024-09-12T03:21:45.244859+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_clean_scrub_stamp":"2024-09-12T03:21:45.231023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:21:46.237118+0000","last_change":"2024-09-12T03:21:46.237118+0000","last_active":"2024-09-12T03:21:46.237118+0000","last_peered":"2024-09-12T03:21:46.237118+0000","last_clean":"2024-09-12T03:21:46.237118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:21:46.237118+0000","last_undegraded":"2024-09-12T03:21:46.237118+0000","last_fullsized":"2024-09-12T03:21:46.237118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-12T03:21:45.243709+0000","last_change":"2024-09-12T03:21:45.243709+0000","last_active":"2024-09-12T03:21:45.231023+0000","last_peered":"2024-09-12T03:21:45.231023+0000","last_clean":"2024-09-12T03:21:45.231023+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:21:45.243709+0000","last_undegraded":"2024-09-12T03:21:45.243709+0000","last_fullsized":"2024-09-12T03:21:45.243709+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_clean_scrub_stamp":"2024-09-12T03:21:45.231023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:21:46.237118+0000","last_change":"2024-09-12T03:21:46.237118+0000","last_active":"2024-09-12T03:21:46.237118+0000","last_peered":"2024-09-12T03:21:46.237118+0000","last_clean":"2024-09-12T03:21:46.237118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:21:46.237118+0000","last_undegraded":"2024-09-12T03:21:46.237118+0000","last_fullsized":"2024-09-12T03:21:46.237118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-12T03:21:45.242027+0000","last_change":"2024-09-12T03:21:45.242027+0000","last_active":"2024-09-12T03:21:45.231023+0000","last_peered":"2024-09-12T03:21:45.231023+0000","last_clean":"2024-09-12T03:21:45.231023+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:21:45.242027+0000","last_undegraded":"2024-09-12T03:21:45.242027+0000","last_fullsized":"2024-09-12T03:21:45.242027+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_clean_scrub_stamp":"2024-09-12T03:21:45.231023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"70'128","reported_seq":375,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-12T03:21:45.242677+0000","last_change":"2024-09-12T03:16:21.883446+0000","last_active":"2024-09-12T03:21:45.242677+0000","last_peered":"2024-09-12T03:21:45.242677+0000","last_clean":"2024-09-12T03:21:45.242677+0000","last_became_active":"2024-09-12T03:16:21.882207+0000","last_became_peered":"2024-09-12T03:16:21.882207+0000","last_unstale":"2024-09-12T03:21:45.242677+0000","last_undegraded":"2024-09-12T03:21:45.242677+0000","last_fullsized":"2024-09-12T03:21:45.242677+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:20.316426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:20.316426+0000","last_clean_scrub_stamp":"2024-09-12T03:16:20.316426+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:12:15.424320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":19,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542211,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37288,"kb_used_data":1932,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334552,"statfs":{"total":96636764160,"available":96598581248,"internally_reserved":0,"allocated":1978368,"data_stored":1198287,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542213,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36708,"kb_used_data":1352,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335132,"statfs":{"total":96636764160,"available":96599175168,"internally_reserved":0,"allocated":1384448,"data_stored":607919,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542212,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36700,"kb_used_data":1352,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335140,"statfs":{"total":96636764160,"available":96599183360,"internally_reserved":0,"allocated":1384448,"data_stored":607919,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542211,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36700,"kb_used_data":1352,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335140,"statfs":{"total":96636764160,"available":96599183360,"internally_reserved":0,"allocated":1384448,"data_stored":607919,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542212,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32544,"kb_used_data":1352,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339296,"statfs":{"total":96636764160,"available":96603439104,"internally_reserved":0,"allocated":1384448,"data_stored":607919,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542212,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32548,"kb_used_data":1352,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339292,"statfs":{"total":96636764160,"available":96603435008,"internally_reserved":0,"allocated":1384448,"data_stored":607919,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542213,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32528,"kb_used_data":1340,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339312,"statfs":{"total":96636764160,"available":96603455488,"internally_reserved":0,"allocated":1372160,"data_stored":602074,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542213,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33120,"kb_used_data":1932,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338720,"statfs":{"total":96636764160,"available":96602849280,"internally_reserved":0,"allocated":1978368,"data_stored":1198287,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T03:21:50.600 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-12T03:21:50.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.739+0000 7fef971a1640 1 Processor -- start 2024-09-12T03:21:50.740 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.739+0000 7fef971a1640 1 -- start start 2024-09-12T03:21:50.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.740+0000 7fef971a1640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fef900b1ca0 0x7fef9015f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:50.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.740+0000 7fef971a1640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fef9015fd70 con 0x7fef900cf660 2024-09-12T03:21:50.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.740+0000 7fef971a1640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fef9006cc40 con 0x7fef9006f360 2024-09-12T03:21:50.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.740+0000 7fef971a1640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7fef900d22f0 con 0x7fef900b1ca0 2024-09-12T03:21:50.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.740+0000 7fef94f3d640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fef900b1ca0 0x7fef9015f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:50.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.740+0000 7fef9573e640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fef900cf660 0x7fef900d2780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:56334/0 (socket says 172.21.15.27:56334) 2024-09-12T03:21:50.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.740+0000 7fef9573e640 1 -- 172.21.15.27:0/1289363039 learned_addr learned my addr 172.21.15.27:0/1289363039 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:21:50.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.740+0000 7fef94f3d640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fef900b1ca0 0x7fef9015f830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39474/0 (socket says 172.21.15.27:39474) 2024-09-12T03:21:50.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.741+0000 7fef94f3d640 1 -- 172.21.15.27:0/1289363039 >> v1:172.21.15.45:6789/0 conn(0x7fef9006f360 legacy=0x7fef900cef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:50.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.741+0000 7fef94f3d640 1 -- 172.21.15.27:0/1289363039 >> v1:172.21.15.27:6789/0 conn(0x7fef900cf660 legacy=0x7fef900d2780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:50.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.741+0000 7fef94f3d640 1 -- 172.21.15.27:0/1289363039 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef900d1ce0 con 0x7fef900b1ca0 2024-09-12T03:21:50.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.741+0000 7fef94f3d640 1 --2- 172.21.15.27:0/1289363039 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fef900b1ca0 0x7fef9015f830 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7fef800097a0 tx=0x7fef8002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e78dc28679a272e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:50.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.741+0000 7fef8f7fe640 1 -- 172.21.15.27:0/1289363039 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef80037070 con 0x7fef900b1ca0 2024-09-12T03:21:50.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.741+0000 7fef8f7fe640 1 -- 172.21.15.27:0/1289363039 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef80032440 con 0x7fef900b1ca0 2024-09-12T03:21:50.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.741+0000 7fef8f7fe640 1 -- 172.21.15.27:0/1289363039 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef80032810 con 0x7fef900b1ca0 2024-09-12T03:21:50.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.742+0000 7fef971a1640 1 -- 172.21.15.27:0/1289363039 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fef900b1ca0 msgr2=0x7fef9015f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:50.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.742+0000 7fef971a1640 1 --2- 172.21.15.27:0/1289363039 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fef900b1ca0 0x7fef9015f830 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7fef800097a0 tx=0x7fef8002a230 comp rx=0 tx=0).stop 2024-09-12T03:21:50.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.742+0000 7fef971a1640 1 -- 172.21.15.27:0/1289363039 shutdown_connections 2024-09-12T03:21:50.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.742+0000 7fef971a1640 1 --2- 172.21.15.27:0/1289363039 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fef900b1ca0 0x7fef9015f830 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:50.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.742+0000 7fef971a1640 1 -- 172.21.15.27:0/1289363039 >> 172.21.15.27:0/1289363039 conn(0x7fef90073800 msgr2=0x7fef9009ba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:50.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.742+0000 7fef971a1640 1 -- 172.21.15.27:0/1289363039 shutdown_connections 2024-09-12T03:21:50.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.742+0000 7fef971a1640 1 -- 172.21.15.27:0/1289363039 wait complete. 2024-09-12T03:21:50.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.743+0000 7fef971a1640 1 Processor -- start 2024-09-12T03:21:50.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.743+0000 7fef971a1640 1 -- start start 2024-09-12T03:21:50.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.744+0000 7fef971a1640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fef9006f360 0x7fef90156090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:50.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.744+0000 7fef971a1640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef900b1ca0 0x7fef900b07a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:50.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.744+0000 7fef971a1640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fef900cf660 0x7fef900ce380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:50.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.744+0000 7fef971a1640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fef9006d1c0 con 0x7fef900cf660 2024-09-12T03:21:50.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.744+0000 7fef971a1640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fef900d1f80 con 0x7fef900b1ca0 2024-09-12T03:21:50.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.744+0000 7fef971a1640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fef9006d340 con 0x7fef9006f360 2024-09-12T03:21:50.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.744+0000 7fef9573e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fef900cf660 0x7fef900ce380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:50.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.744+0000 7fef9573e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fef900cf660 0x7fef900ce380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39478/0 (socket says 172.21.15.27:39478) 2024-09-12T03:21:50.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.744+0000 7fef9573e640 1 -- 172.21.15.27:0/2525188153 learned_addr learned my addr 172.21.15.27:0/2525188153 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:21:50.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.744+0000 7fef8ffff640 1 --2- 172.21.15.27:0/2525188153 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef900b1ca0 0x7fef900b07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:50.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.744+0000 7fef94f3d640 1 --2- 172.21.15.27:0/2525188153 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fef9006f360 0x7fef90156090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:50.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.745+0000 7fef9573e640 1 -- 172.21.15.27:0/2525188153 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fef9006f360 msgr2=0x7fef90156090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:50.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.745+0000 7fef9573e640 1 --2- 172.21.15.27:0/2525188153 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fef9006f360 0x7fef90156090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:50.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.745+0000 7fef9573e640 1 -- 172.21.15.27:0/2525188153 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef900b1ca0 msgr2=0x7fef900b07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:50.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.745+0000 7fef9573e640 1 --2- 172.21.15.27:0/2525188153 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef900b1ca0 0x7fef900b07a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:50.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.745+0000 7fef9573e640 1 -- 172.21.15.27:0/2525188153 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef900ce8c0 con 0x7fef900cf660 2024-09-12T03:21:50.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.745+0000 7fef9573e640 1 --2- 172.21.15.27:0/2525188153 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fef900cf660 0x7fef900ce380 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7fef8800ca20 tx=0x7fef8800cee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:50.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.745+0000 7fef8d7fa640 1 -- 172.21.15.27:0/2525188153 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef88017070 con 0x7fef900cf660 2024-09-12T03:21:50.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.745+0000 7fef8d7fa640 1 -- 172.21.15.27:0/2525188153 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef8800abd0 con 0x7fef900cf660 2024-09-12T03:21:50.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.745+0000 7fef971a1640 1 -- 172.21.15.27:0/2525188153 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef900b0e40 con 0x7fef900cf660 2024-09-12T03:21:50.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.745+0000 7fef8d7fa640 1 -- 172.21.15.27:0/2525188153 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef88012400 con 0x7fef900cf660 2024-09-12T03:21:50.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.745+0000 7fef971a1640 1 -- 172.21.15.27:0/2525188153 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef900b1400 con 0x7fef900cf660 2024-09-12T03:21:50.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.746+0000 7fef971a1640 1 -- 172.21.15.27:0/2525188153 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef900cfc40 con 0x7fef900cf660 2024-09-12T03:21:50.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.747+0000 7fef8d7fa640 1 -- 172.21.15.27:0/2525188153 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 19) ==== 90855+0+0 (secure 0 0 0) 0x7fef8801a070 con 0x7fef900cf660 2024-09-12T03:21:50.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.747+0000 7fef8d7fa640 1 --2- 172.21.15.27:0/2525188153 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fef7c06c220 0x7fef7c06e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:50.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.747+0000 7fef8d7fa640 1 -- 172.21.15.27:0/2525188153 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5737+0+0 (secure 0 0 0) 0x7fef8808b0c0 con 0x7fef900cf660 2024-09-12T03:21:50.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.747+0000 7fef94f3d640 1 --2- 172.21.15.27:0/2525188153 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fef7c06c220 0x7fef7c06e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:50.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.747+0000 7fef94f3d640 1 --2- 172.21.15.27:0/2525188153 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fef7c06c220 0x7fef7c06e6e0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fef80034000 tx=0x7fef8002e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:50.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.751+0000 7fef8d7fa640 1 -- 172.21.15.27:0/2525188153 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fef9015fd70 con 0x7fef900cf660 2024-09-12T03:21:50.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.925+0000 7fef971a1640 1 -- 172.21.15.27:0/2525188153 --> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fef900b3f40 con 0x7fef7c06c220 2024-09-12T03:21:50.930 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.930+0000 7fef8d7fa640 1 -- 172.21.15.27:0/2525188153 <== mgr.4113 v2:172.21.15.27:6836/1226920526 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+240244 (secure 0 0 0) 0x7fef900b3f40 con 0x7fef7c06c220 2024-09-12T03:21:50.933 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:21:50.934 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2024-09-12T03:21:50.936 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.934+0000 7fef971a1640 1 -- 172.21.15.27:0/2525188153 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fef7c06c220 msgr2=0x7fef7c06e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:50.936 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.934+0000 7fef971a1640 1 --2- 172.21.15.27:0/2525188153 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fef7c06c220 0x7fef7c06e6e0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fef80034000 tx=0x7fef8002e000 comp rx=0 tx=0).stop 2024-09-12T03:21:50.936 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.934+0000 7fef971a1640 1 -- 172.21.15.27:0/2525188153 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fef900cf660 msgr2=0x7fef900ce380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:50.936 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.934+0000 7fef971a1640 1 --2- 172.21.15.27:0/2525188153 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fef900cf660 0x7fef900ce380 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7fef8800ca20 tx=0x7fef8800cee0 comp rx=0 tx=0).stop 2024-09-12T03:21:50.936 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.934+0000 7fef971a1640 1 -- 172.21.15.27:0/2525188153 shutdown_connections 2024-09-12T03:21:50.936 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.934+0000 7fef971a1640 1 --2- 172.21.15.27:0/2525188153 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fef7c06c220 0x7fef7c06e6e0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:50.936 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.934+0000 7fef971a1640 1 --2- 172.21.15.27:0/2525188153 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fef900cf660 0x7fef900ce380 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:50.936 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.934+0000 7fef971a1640 1 --2- 172.21.15.27:0/2525188153 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef900b1ca0 0x7fef900b07a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:50.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.934+0000 7fef971a1640 1 --2- 172.21.15.27:0/2525188153 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fef9006f360 0x7fef90156090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:50.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.934+0000 7fef971a1640 1 -- 172.21.15.27:0/2525188153 >> 172.21.15.27:0/2525188153 conn(0x7fef90073800 msgr2=0x7fef900cef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:50.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.934+0000 7fef971a1640 1 -- 172.21.15.27:0/2525188153 shutdown_connections 2024-09-12T03:21:50.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:50.934+0000 7fef971a1640 1 -- 172.21.15.27:0/2525188153 wait complete. 2024-09-12T03:21:50.945 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":95,"stamp":"2024-09-12T03:21:49.115228+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":8,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":278136,"kb_used_data":11964,"kb_used_omap":12,"kb_used_meta":265971,"kb_avail":754696584,"statfs":{"total":773094113280,"available":772809302016,"internally_reserved":0,"allocated":12251136,"data_stored":6038243,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":272354864},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000971"},"pg_stats":[{"pgid":"19.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:21:46.237118+0000","last_change":"2024-09-12T03:21:46.237118+0000","last_active":"2024-09-12T03:21:46.237118+0000","last_peered":"2024-09-12T03:21:46.237118+0000","last_clean":"2024-09-12T03:21:46.237118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:21:46.237118+0000","last_undegraded":"2024-09-12T03:21:46.237118+0000","last_fullsized":"2024-09-12T03:21:46.237118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:21:46.237118+0000","last_change":"2024-09-12T03:21:46.237118+0000","last_active":"2024-09-12T03:21:46.237118+0000","last_peered":"2024-09-12T03:21:46.237118+0000","last_clean":"2024-09-12T03:21:46.237118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:21:46.237118+0000","last_undegraded":"2024-09-12T03:21:46.237118+0000","last_fullsized":"2024-09-12T03:21:46.237118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:21:46.237118+0000","last_change":"2024-09-12T03:21:46.237118+0000","last_active":"2024-09-12T03:21:46.237118+0000","last_peered":"2024-09-12T03:21:46.237118+0000","last_clean":"2024-09-12T03:21:46.237118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:21:46.237118+0000","last_undegraded":"2024-09-12T03:21:46.237118+0000","last_fullsized":"2024-09-12T03:21:46.237118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:21:46.237118+0000","last_change":"2024-09-12T03:21:46.237118+0000","last_active":"2024-09-12T03:21:46.237118+0000","last_peered":"2024-09-12T03:21:46.237118+0000","last_clean":"2024-09-12T03:21:46.237118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:21:46.237118+0000","last_undegraded":"2024-09-12T03:21:46.237118+0000","last_fullsized":"2024-09-12T03:21:46.237118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:21:46.237118+0000","last_change":"2024-09-12T03:21:46.237118+0000","last_active":"2024-09-12T03:21:46.237118+0000","last_peered":"2024-09-12T03:21:46.237118+0000","last_clean":"2024-09-12T03:21:46.237118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:21:46.237118+0000","last_undegraded":"2024-09-12T03:21:46.237118+0000","last_fullsized":"2024-09-12T03:21:46.237118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:21:46.237118+0000","last_change":"2024-09-12T03:21:46.237118+0000","last_active":"2024-09-12T03:21:46.237118+0000","last_peered":"2024-09-12T03:21:46.237118+0000","last_clean":"2024-09-12T03:21:46.237118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:21:46.237118+0000","last_undegraded":"2024-09-12T03:21:46.237118+0000","last_fullsized":"2024-09-12T03:21:46.237118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:21:46.237118+0000","last_change":"2024-09-12T03:21:46.237118+0000","last_active":"2024-09-12T03:21:46.237118+0000","last_peered":"2024-09-12T03:21:46.237118+0000","last_clean":"2024-09-12T03:21:46.237118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:21:46.237118+0000","last_undegraded":"2024-09-12T03:21:46.237118+0000","last_fullsized":"2024-09-12T03:21:46.237118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:21:46.237118+0000","last_change":"2024-09-12T03:21:46.237118+0000","last_active":"2024-09-12T03:21:46.237118+0000","last_peered":"2024-09-12T03:21:46.237118+0000","last_clean":"2024-09-12T03:21:46.237118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:21:46.237118+0000","last_undegraded":"2024-09-12T03:21:46.237118+0000","last_fullsized":"2024-09-12T03:21:46.237118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:21:46.237118+0000","last_change":"2024-09-12T03:21:46.237118+0000","last_active":"2024-09-12T03:21:46.237118+0000","last_peered":"2024-09-12T03:21:46.237118+0000","last_clean":"2024-09-12T03:21:46.237118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:21:46.237118+0000","last_undegraded":"2024-09-12T03:21:46.237118+0000","last_fullsized":"2024-09-12T03:21:46.237118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:21:46.237118+0000","last_change":"2024-09-12T03:21:46.237118+0000","last_active":"2024-09-12T03:21:46.237118+0000","last_peered":"2024-09-12T03:21:46.237118+0000","last_clean":"2024-09-12T03:21:46.237118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:21:46.237118+0000","last_undegraded":"2024-09-12T03:21:46.237118+0000","last_fullsized":"2024-09-12T03:21:46.237118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:21:46.237118+0000","last_change":"2024-09-12T03:21:46.237118+0000","last_active":"2024-09-12T03:21:46.237118+0000","last_peered":"2024-09-12T03:21:46.237118+0000","last_clean":"2024-09-12T03:21:46.237118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:21:46.237118+0000","last_undegraded":"2024-09-12T03:21:46.237118+0000","last_fullsized":"2024-09-12T03:21:46.237118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:21:46.237118+0000","last_change":"2024-09-12T03:21:46.237118+0000","last_active":"2024-09-12T03:21:46.237118+0000","last_peered":"2024-09-12T03:21:46.237118+0000","last_clean":"2024-09-12T03:21:46.237118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:21:46.237118+0000","last_undegraded":"2024-09-12T03:21:46.237118+0000","last_fullsized":"2024-09-12T03:21:46.237118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:21:46.237118+0000","last_change":"2024-09-12T03:21:46.237118+0000","last_active":"2024-09-12T03:21:46.237118+0000","last_peered":"2024-09-12T03:21:46.237118+0000","last_clean":"2024-09-12T03:21:46.237118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:21:46.237118+0000","last_undegraded":"2024-09-12T03:21:46.237118+0000","last_fullsized":"2024-09-12T03:21:46.237118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:21:46.237118+0000","last_change":"2024-09-12T03:21:46.237118+0000","last_active":"2024-09-12T03:21:46.237118+0000","last_peered":"2024-09-12T03:21:46.237118+0000","last_clean":"2024-09-12T03:21:46.237118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:21:46.237118+0000","last_undegraded":"2024-09-12T03:21:46.237118+0000","last_fullsized":"2024-09-12T03:21:46.237118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:21:46.237118+0000","last_change":"2024-09-12T03:21:46.237118+0000","last_active":"2024-09-12T03:21:46.237118+0000","last_peered":"2024-09-12T03:21:46.237118+0000","last_clean":"2024-09-12T03:21:46.237118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:21:46.237118+0000","last_undegraded":"2024-09-12T03:21:46.237118+0000","last_fullsized":"2024-09-12T03:21:46.237118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:21:46.237118+0000","last_change":"2024-09-12T03:21:46.237118+0000","last_active":"2024-09-12T03:21:46.237118+0000","last_peered":"2024-09-12T03:21:46.237118+0000","last_clean":"2024-09-12T03:21:46.237118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:21:46.237118+0000","last_undegraded":"2024-09-12T03:21:46.237118+0000","last_fullsized":"2024-09-12T03:21:46.237118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:21:46.237118+0000","last_change":"2024-09-12T03:21:46.237118+0000","last_active":"2024-09-12T03:21:46.237118+0000","last_peered":"2024-09-12T03:21:46.237118+0000","last_clean":"2024-09-12T03:21:46.237118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:21:46.237118+0000","last_undegraded":"2024-09-12T03:21:46.237118+0000","last_fullsized":"2024-09-12T03:21:46.237118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:21:46.237118+0000","last_change":"2024-09-12T03:21:46.237118+0000","last_active":"2024-09-12T03:21:46.237118+0000","last_peered":"2024-09-12T03:21:46.237118+0000","last_clean":"2024-09-12T03:21:46.237118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:21:46.237118+0000","last_undegraded":"2024-09-12T03:21:46.237118+0000","last_fullsized":"2024-09-12T03:21:46.237118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:21:46.237118+0000","last_change":"2024-09-12T03:21:46.237118+0000","last_active":"2024-09-12T03:21:46.237118+0000","last_peered":"2024-09-12T03:21:46.237118+0000","last_clean":"2024-09-12T03:21:46.237118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:21:46.237118+0000","last_undegraded":"2024-09-12T03:21:46.237118+0000","last_fullsized":"2024-09-12T03:21:46.237118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:21:46.237118+0000","last_change":"2024-09-12T03:21:46.237118+0000","last_active":"2024-09-12T03:21:46.237118+0000","last_peered":"2024-09-12T03:21:46.237118+0000","last_clean":"2024-09-12T03:21:46.237118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:21:46.237118+0000","last_undegraded":"2024-09-12T03:21:46.237118+0000","last_fullsized":"2024-09-12T03:21:46.237118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:21:46.237118+0000","last_change":"2024-09-12T03:21:46.237118+0000","last_active":"2024-09-12T03:21:46.237118+0000","last_peered":"2024-09-12T03:21:46.237118+0000","last_clean":"2024-09-12T03:21:46.237118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:21:46.237118+0000","last_undegraded":"2024-09-12T03:21:46.237118+0000","last_fullsized":"2024-09-12T03:21:46.237118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:21:46.237118+0000","last_change":"2024-09-12T03:21:46.237118+0000","last_active":"2024-09-12T03:21:46.237118+0000","last_peered":"2024-09-12T03:21:46.237118+0000","last_clean":"2024-09-12T03:21:46.237118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:21:46.237118+0000","last_undegraded":"2024-09-12T03:21:46.237118+0000","last_fullsized":"2024-09-12T03:21:46.237118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:21:46.237118+0000","last_change":"2024-09-12T03:21:46.237118+0000","last_active":"2024-09-12T03:21:46.237118+0000","last_peered":"2024-09-12T03:21:46.237118+0000","last_clean":"2024-09-12T03:21:46.237118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:21:46.237118+0000","last_undegraded":"2024-09-12T03:21:46.237118+0000","last_fullsized":"2024-09-12T03:21:46.237118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:21:46.237118+0000","last_change":"2024-09-12T03:21:46.237118+0000","last_active":"2024-09-12T03:21:46.237118+0000","last_peered":"2024-09-12T03:21:46.237118+0000","last_clean":"2024-09-12T03:21:46.237118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:21:46.237118+0000","last_undegraded":"2024-09-12T03:21:46.237118+0000","last_fullsized":"2024-09-12T03:21:46.237118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:21:46.237118+0000","last_change":"2024-09-12T03:21:46.237118+0000","last_active":"2024-09-12T03:21:46.237118+0000","last_peered":"2024-09-12T03:21:46.237118+0000","last_clean":"2024-09-12T03:21:46.237118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:21:46.237118+0000","last_undegraded":"2024-09-12T03:21:46.237118+0000","last_fullsized":"2024-09-12T03:21:46.237118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:21:46.237118+0000","last_change":"2024-09-12T03:21:46.237118+0000","last_active":"2024-09-12T03:21:46.237118+0000","last_peered":"2024-09-12T03:21:46.237118+0000","last_clean":"2024-09-12T03:21:46.237118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:21:46.237118+0000","last_undegraded":"2024-09-12T03:21:46.237118+0000","last_fullsized":"2024-09-12T03:21:46.237118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:21:46.237118+0000","last_change":"2024-09-12T03:21:46.237118+0000","last_active":"2024-09-12T03:21:46.237118+0000","last_peered":"2024-09-12T03:21:46.237118+0000","last_clean":"2024-09-12T03:21:46.237118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:21:46.237118+0000","last_undegraded":"2024-09-12T03:21:46.237118+0000","last_fullsized":"2024-09-12T03:21:46.237118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:21:46.237118+0000","last_change":"2024-09-12T03:21:46.237118+0000","last_active":"2024-09-12T03:21:46.237118+0000","last_peered":"2024-09-12T03:21:46.237118+0000","last_clean":"2024-09-12T03:21:46.237118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:21:46.237118+0000","last_undegraded":"2024-09-12T03:21:46.237118+0000","last_fullsized":"2024-09-12T03:21:46.237118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:21:46.237118+0000","last_change":"2024-09-12T03:21:46.237118+0000","last_active":"2024-09-12T03:21:46.237118+0000","last_peered":"2024-09-12T03:21:46.237118+0000","last_clean":"2024-09-12T03:21:46.237118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:21:46.237118+0000","last_undegraded":"2024-09-12T03:21:46.237118+0000","last_fullsized":"2024-09-12T03:21:46.237118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:21:46.237118+0000","last_change":"2024-09-12T03:21:46.237118+0000","last_active":"2024-09-12T03:21:46.237118+0000","last_peered":"2024-09-12T03:21:46.237118+0000","last_clean":"2024-09-12T03:21:46.237118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:21:46.237118+0000","last_undegraded":"2024-09-12T03:21:46.237118+0000","last_fullsized":"2024-09-12T03:21:46.237118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:21:46.237118+0000","last_change":"2024-09-12T03:21:46.237118+0000","last_active":"2024-09-12T03:21:46.237118+0000","last_peered":"2024-09-12T03:21:46.237118+0000","last_clean":"2024-09-12T03:21:46.237118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:21:46.237118+0000","last_undegraded":"2024-09-12T03:21:46.237118+0000","last_fullsized":"2024-09-12T03:21:46.237118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:21:46.237118+0000","last_change":"2024-09-12T03:21:46.237118+0000","last_active":"2024-09-12T03:21:46.237118+0000","last_peered":"2024-09-12T03:21:46.237118+0000","last_clean":"2024-09-12T03:21:46.237118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:21:46.237118+0000","last_undegraded":"2024-09-12T03:21:46.237118+0000","last_fullsized":"2024-09-12T03:21:46.237118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:21:46.237118+0000","last_change":"2024-09-12T03:21:46.237118+0000","last_active":"2024-09-12T03:21:46.237118+0000","last_peered":"2024-09-12T03:21:46.237118+0000","last_clean":"2024-09-12T03:21:46.237118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:21:46.237118+0000","last_undegraded":"2024-09-12T03:21:46.237118+0000","last_fullsized":"2024-09-12T03:21:46.237118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-12T03:21:45.241963+0000","last_change":"2024-09-12T03:21:45.241963+0000","last_active":"2024-09-12T03:21:45.231023+0000","last_peered":"2024-09-12T03:21:45.231023+0000","last_clean":"2024-09-12T03:21:45.231023+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:21:45.241963+0000","last_undegraded":"2024-09-12T03:21:45.241963+0000","last_fullsized":"2024-09-12T03:21:45.241963+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_clean_scrub_stamp":"2024-09-12T03:21:45.231023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:21:46.237118+0000","last_change":"2024-09-12T03:21:46.237118+0000","last_active":"2024-09-12T03:21:46.237118+0000","last_peered":"2024-09-12T03:21:46.237118+0000","last_clean":"2024-09-12T03:21:46.237118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:21:46.237118+0000","last_undegraded":"2024-09-12T03:21:46.237118+0000","last_fullsized":"2024-09-12T03:21:46.237118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-12T03:21:45.244081+0000","last_change":"2024-09-12T03:21:45.244081+0000","last_active":"2024-09-12T03:21:45.231023+0000","last_peered":"2024-09-12T03:21:45.231023+0000","last_clean":"2024-09-12T03:21:45.231023+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:21:45.244081+0000","last_undegraded":"2024-09-12T03:21:45.244081+0000","last_fullsized":"2024-09-12T03:21:45.244081+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_clean_scrub_stamp":"2024-09-12T03:21:45.231023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:21:46.237118+0000","last_change":"2024-09-12T03:21:46.237118+0000","last_active":"2024-09-12T03:21:46.237118+0000","last_peered":"2024-09-12T03:21:46.237118+0000","last_clean":"2024-09-12T03:21:46.237118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:21:46.237118+0000","last_undegraded":"2024-09-12T03:21:46.237118+0000","last_fullsized":"2024-09-12T03:21:46.237118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-12T03:21:45.243768+0000","last_change":"2024-09-12T03:21:45.243768+0000","last_active":"2024-09-12T03:21:45.231023+0000","last_peered":"2024-09-12T03:21:45.231023+0000","last_clean":"2024-09-12T03:21:45.231023+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:21:45.243768+0000","last_undegraded":"2024-09-12T03:21:45.243768+0000","last_fullsized":"2024-09-12T03:21:45.243768+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_clean_scrub_stamp":"2024-09-12T03:21:45.231023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:21:46.237118+0000","last_change":"2024-09-12T03:21:46.237118+0000","last_active":"2024-09-12T03:21:46.237118+0000","last_peered":"2024-09-12T03:21:46.237118+0000","last_clean":"2024-09-12T03:21:46.237118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:21:46.237118+0000","last_undegraded":"2024-09-12T03:21:46.237118+0000","last_fullsized":"2024-09-12T03:21:46.237118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-12T03:21:45.250356+0000","last_change":"2024-09-12T03:21:45.250356+0000","last_active":"2024-09-12T03:21:45.231023+0000","last_peered":"2024-09-12T03:21:45.231023+0000","last_clean":"2024-09-12T03:21:45.231023+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:21:45.250356+0000","last_undegraded":"2024-09-12T03:21:45.250356+0000","last_fullsized":"2024-09-12T03:21:45.250356+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_clean_scrub_stamp":"2024-09-12T03:21:45.231023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:21:46.237118+0000","last_change":"2024-09-12T03:21:46.237118+0000","last_active":"2024-09-12T03:21:46.237118+0000","last_peered":"2024-09-12T03:21:46.237118+0000","last_clean":"2024-09-12T03:21:46.237118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:21:46.237118+0000","last_undegraded":"2024-09-12T03:21:46.237118+0000","last_fullsized":"2024-09-12T03:21:46.237118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-12T03:21:45.251259+0000","last_change":"2024-09-12T03:21:45.251259+0000","last_active":"2024-09-12T03:21:45.231023+0000","last_peered":"2024-09-12T03:21:45.231023+0000","last_clean":"2024-09-12T03:21:45.231023+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:21:45.251259+0000","last_undegraded":"2024-09-12T03:21:45.251259+0000","last_fullsized":"2024-09-12T03:21:45.251259+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_clean_scrub_stamp":"2024-09-12T03:21:45.231023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:21:46.237118+0000","last_change":"2024-09-12T03:21:46.237118+0000","last_active":"2024-09-12T03:21:46.237118+0000","last_peered":"2024-09-12T03:21:46.237118+0000","last_clean":"2024-09-12T03:21:46.237118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:21:46.237118+0000","last_undegraded":"2024-09-12T03:21:46.237118+0000","last_fullsized":"2024-09-12T03:21:46.237118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-12T03:21:45.245853+0000","last_change":"2024-09-12T03:21:45.245853+0000","last_active":"2024-09-12T03:21:45.231023+0000","last_peered":"2024-09-12T03:21:45.231023+0000","last_clean":"2024-09-12T03:21:45.231023+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:21:45.245853+0000","last_undegraded":"2024-09-12T03:21:45.245853+0000","last_fullsized":"2024-09-12T03:21:45.245853+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_clean_scrub_stamp":"2024-09-12T03:21:45.231023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:21:46.237118+0000","last_change":"2024-09-12T03:21:46.237118+0000","last_active":"2024-09-12T03:21:46.237118+0000","last_peered":"2024-09-12T03:21:46.237118+0000","last_clean":"2024-09-12T03:21:46.237118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:21:46.237118+0000","last_undegraded":"2024-09-12T03:21:46.237118+0000","last_fullsized":"2024-09-12T03:21:46.237118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-12T03:21:45.239959+0000","last_change":"2024-09-12T03:21:45.239959+0000","last_active":"2024-09-12T03:21:45.231023+0000","last_peered":"2024-09-12T03:21:45.231023+0000","last_clean":"2024-09-12T03:21:45.231023+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:21:45.239959+0000","last_undegraded":"2024-09-12T03:21:45.239959+0000","last_fullsized":"2024-09-12T03:21:45.239959+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_clean_scrub_stamp":"2024-09-12T03:21:45.231023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:21:46.237118+0000","last_change":"2024-09-12T03:21:46.237118+0000","last_active":"2024-09-12T03:21:46.237118+0000","last_peered":"2024-09-12T03:21:46.237118+0000","last_clean":"2024-09-12T03:21:46.237118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:21:46.237118+0000","last_undegraded":"2024-09-12T03:21:46.237118+0000","last_fullsized":"2024-09-12T03:21:46.237118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-12T03:21:45.250250+0000","last_change":"2024-09-12T03:21:45.250250+0000","last_active":"2024-09-12T03:21:45.231023+0000","last_peered":"2024-09-12T03:21:45.231023+0000","last_clean":"2024-09-12T03:21:45.231023+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:21:45.250250+0000","last_undegraded":"2024-09-12T03:21:45.250250+0000","last_fullsized":"2024-09-12T03:21:45.250250+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_clean_scrub_stamp":"2024-09-12T03:21:45.231023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:21:46.237118+0000","last_change":"2024-09-12T03:21:46.237118+0000","last_active":"2024-09-12T03:21:46.237118+0000","last_peered":"2024-09-12T03:21:46.237118+0000","last_clean":"2024-09-12T03:21:46.237118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:21:46.237118+0000","last_undegraded":"2024-09-12T03:21:46.237118+0000","last_fullsized":"2024-09-12T03:21:46.237118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-12T03:21:45.244419+0000","last_change":"2024-09-12T03:21:45.244419+0000","last_active":"2024-09-12T03:21:45.231023+0000","last_peered":"2024-09-12T03:21:45.231023+0000","last_clean":"2024-09-12T03:21:45.231023+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:21:45.244419+0000","last_undegraded":"2024-09-12T03:21:45.244419+0000","last_fullsized":"2024-09-12T03:21:45.244419+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_clean_scrub_stamp":"2024-09-12T03:21:45.231023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:21:46.237118+0000","last_change":"2024-09-12T03:21:46.237118+0000","last_active":"2024-09-12T03:21:46.237118+0000","last_peered":"2024-09-12T03:21:46.237118+0000","last_clean":"2024-09-12T03:21:46.237118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:21:46.237118+0000","last_undegraded":"2024-09-12T03:21:46.237118+0000","last_fullsized":"2024-09-12T03:21:46.237118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-12T03:21:45.245487+0000","last_change":"2024-09-12T03:21:45.245487+0000","last_active":"2024-09-12T03:21:45.231023+0000","last_peered":"2024-09-12T03:21:45.231023+0000","last_clean":"2024-09-12T03:21:45.231023+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:21:45.245487+0000","last_undegraded":"2024-09-12T03:21:45.245487+0000","last_fullsized":"2024-09-12T03:21:45.245487+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_clean_scrub_stamp":"2024-09-12T03:21:45.231023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:21:46.237118+0000","last_change":"2024-09-12T03:21:46.237118+0000","last_active":"2024-09-12T03:21:46.237118+0000","last_peered":"2024-09-12T03:21:46.237118+0000","last_clean":"2024-09-12T03:21:46.237118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:21:46.237118+0000","last_undegraded":"2024-09-12T03:21:46.237118+0000","last_fullsized":"2024-09-12T03:21:46.237118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-12T03:21:45.243429+0000","last_change":"2024-09-12T03:21:45.243429+0000","last_active":"2024-09-12T03:21:45.231023+0000","last_peered":"2024-09-12T03:21:45.231023+0000","last_clean":"2024-09-12T03:21:45.231023+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:21:45.243429+0000","last_undegraded":"2024-09-12T03:21:45.243429+0000","last_fullsized":"2024-09-12T03:21:45.243429+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_clean_scrub_stamp":"2024-09-12T03:21:45.231023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:21:46.237118+0000","last_change":"2024-09-12T03:21:46.237118+0000","last_active":"2024-09-12T03:21:46.237118+0000","last_peered":"2024-09-12T03:21:46.237118+0000","last_clean":"2024-09-12T03:21:46.237118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:21:46.237118+0000","last_undegraded":"2024-09-12T03:21:46.237118+0000","last_fullsized":"2024-09-12T03:21:46.237118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-12T03:21:45.239970+0000","last_change":"2024-09-12T03:21:45.239970+0000","last_active":"2024-09-12T03:21:45.231023+0000","last_peered":"2024-09-12T03:21:45.231023+0000","last_clean":"2024-09-12T03:21:45.231023+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:21:45.239970+0000","last_undegraded":"2024-09-12T03:21:45.239970+0000","last_fullsized":"2024-09-12T03:21:45.239970+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_clean_scrub_stamp":"2024-09-12T03:21:45.231023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:21:46.237118+0000","last_change":"2024-09-12T03:21:46.237118+0000","last_active":"2024-09-12T03:21:46.237118+0000","last_peered":"2024-09-12T03:21:46.237118+0000","last_clean":"2024-09-12T03:21:46.237118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:21:46.237118+0000","last_undegraded":"2024-09-12T03:21:46.237118+0000","last_fullsized":"2024-09-12T03:21:46.237118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-12T03:21:45.241108+0000","last_change":"2024-09-12T03:21:45.241108+0000","last_active":"2024-09-12T03:21:45.231023+0000","last_peered":"2024-09-12T03:21:45.231023+0000","last_clean":"2024-09-12T03:21:45.231023+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:21:45.241108+0000","last_undegraded":"2024-09-12T03:21:45.241108+0000","last_fullsized":"2024-09-12T03:21:45.241108+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_clean_scrub_stamp":"2024-09-12T03:21:45.231023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:21:46.237118+0000","last_change":"2024-09-12T03:21:46.237118+0000","last_active":"2024-09-12T03:21:46.237118+0000","last_peered":"2024-09-12T03:21:46.237118+0000","last_clean":"2024-09-12T03:21:46.237118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:21:46.237118+0000","last_undegraded":"2024-09-12T03:21:46.237118+0000","last_fullsized":"2024-09-12T03:21:46.237118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-12T03:21:45.250329+0000","last_change":"2024-09-12T03:21:45.250329+0000","last_active":"2024-09-12T03:21:45.231023+0000","last_peered":"2024-09-12T03:21:45.231023+0000","last_clean":"2024-09-12T03:21:45.231023+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:21:45.250329+0000","last_undegraded":"2024-09-12T03:21:45.250329+0000","last_fullsized":"2024-09-12T03:21:45.250329+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_clean_scrub_stamp":"2024-09-12T03:21:45.231023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:21:46.237118+0000","last_change":"2024-09-12T03:21:46.237118+0000","last_active":"2024-09-12T03:21:46.237118+0000","last_peered":"2024-09-12T03:21:46.237118+0000","last_clean":"2024-09-12T03:21:46.237118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:21:46.237118+0000","last_undegraded":"2024-09-12T03:21:46.237118+0000","last_fullsized":"2024-09-12T03:21:46.237118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-12T03:21:45.239970+0000","last_change":"2024-09-12T03:21:45.239970+0000","last_active":"2024-09-12T03:21:45.231023+0000","last_peered":"2024-09-12T03:21:45.231023+0000","last_clean":"2024-09-12T03:21:45.231023+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:21:45.239970+0000","last_undegraded":"2024-09-12T03:21:45.239970+0000","last_fullsized":"2024-09-12T03:21:45.239970+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_clean_scrub_stamp":"2024-09-12T03:21:45.231023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:21:46.237118+0000","last_change":"2024-09-12T03:21:46.237118+0000","last_active":"2024-09-12T03:21:46.237118+0000","last_peered":"2024-09-12T03:21:46.237118+0000","last_clean":"2024-09-12T03:21:46.237118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:21:46.237118+0000","last_undegraded":"2024-09-12T03:21:46.237118+0000","last_fullsized":"2024-09-12T03:21:46.237118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-12T03:21:45.243798+0000","last_change":"2024-09-12T03:21:45.243798+0000","last_active":"2024-09-12T03:21:45.231023+0000","last_peered":"2024-09-12T03:21:45.231023+0000","last_clean":"2024-09-12T03:21:45.231023+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:21:45.243798+0000","last_undegraded":"2024-09-12T03:21:45.243798+0000","last_fullsized":"2024-09-12T03:21:45.243798+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_clean_scrub_stamp":"2024-09-12T03:21:45.231023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:21:46.237118+0000","last_change":"2024-09-12T03:21:46.237118+0000","last_active":"2024-09-12T03:21:46.237118+0000","last_peered":"2024-09-12T03:21:46.237118+0000","last_clean":"2024-09-12T03:21:46.237118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:21:46.237118+0000","last_undegraded":"2024-09-12T03:21:46.237118+0000","last_fullsized":"2024-09-12T03:21:46.237118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-12T03:21:45.243777+0000","last_change":"2024-09-12T03:21:45.243777+0000","last_active":"2024-09-12T03:21:45.231023+0000","last_peered":"2024-09-12T03:21:45.231023+0000","last_clean":"2024-09-12T03:21:45.231023+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:21:45.243777+0000","last_undegraded":"2024-09-12T03:21:45.243777+0000","last_fullsized":"2024-09-12T03:21:45.243777+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_clean_scrub_stamp":"2024-09-12T03:21:45.231023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:21:46.237118+0000","last_change":"2024-09-12T03:21:46.237118+0000","last_active":"2024-09-12T03:21:46.237118+0000","last_peered":"2024-09-12T03:21:46.237118+0000","last_clean":"2024-09-12T03:21:46.237118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:21:46.237118+0000","last_undegraded":"2024-09-12T03:21:46.237118+0000","last_fullsized":"2024-09-12T03:21:46.237118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-12T03:21:45.239959+0000","last_change":"2024-09-12T03:21:45.239959+0000","last_active":"2024-09-12T03:21:45.231023+0000","last_peered":"2024-09-12T03:21:45.231023+0000","last_clean":"2024-09-12T03:21:45.231023+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:21:45.239959+0000","last_undegraded":"2024-09-12T03:21:45.239959+0000","last_fullsized":"2024-09-12T03:21:45.239959+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_clean_scrub_stamp":"2024-09-12T03:21:45.231023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:21:46.237118+0000","last_change":"2024-09-12T03:21:46.237118+0000","last_active":"2024-09-12T03:21:46.237118+0000","last_peered":"2024-09-12T03:21:46.237118+0000","last_clean":"2024-09-12T03:21:46.237118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:21:46.237118+0000","last_undegraded":"2024-09-12T03:21:46.237118+0000","last_fullsized":"2024-09-12T03:21:46.237118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-12T03:21:45.244432+0000","last_change":"2024-09-12T03:21:45.244432+0000","last_active":"2024-09-12T03:21:45.231023+0000","last_peered":"2024-09-12T03:21:45.231023+0000","last_clean":"2024-09-12T03:21:45.231023+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:21:45.244432+0000","last_undegraded":"2024-09-12T03:21:45.244432+0000","last_fullsized":"2024-09-12T03:21:45.244432+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_clean_scrub_stamp":"2024-09-12T03:21:45.231023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:21:46.237118+0000","last_change":"2024-09-12T03:21:46.237118+0000","last_active":"2024-09-12T03:21:46.237118+0000","last_peered":"2024-09-12T03:21:46.237118+0000","last_clean":"2024-09-12T03:21:46.237118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:21:46.237118+0000","last_undegraded":"2024-09-12T03:21:46.237118+0000","last_fullsized":"2024-09-12T03:21:46.237118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-12T03:21:45.243467+0000","last_change":"2024-09-12T03:21:45.243467+0000","last_active":"2024-09-12T03:21:45.231023+0000","last_peered":"2024-09-12T03:21:45.231023+0000","last_clean":"2024-09-12T03:21:45.231023+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:21:45.243467+0000","last_undegraded":"2024-09-12T03:21:45.243467+0000","last_fullsized":"2024-09-12T03:21:45.243467+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_clean_scrub_stamp":"2024-09-12T03:21:45.231023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:21:46.237118+0000","last_change":"2024-09-12T03:21:46.237118+0000","last_active":"2024-09-12T03:21:46.237118+0000","last_peered":"2024-09-12T03:21:46.237118+0000","last_clean":"2024-09-12T03:21:46.237118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:21:46.237118+0000","last_undegraded":"2024-09-12T03:21:46.237118+0000","last_fullsized":"2024-09-12T03:21:46.237118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-12T03:21:45.245048+0000","last_change":"2024-09-12T03:21:45.245048+0000","last_active":"2024-09-12T03:21:45.231023+0000","last_peered":"2024-09-12T03:21:45.231023+0000","last_clean":"2024-09-12T03:21:45.231023+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:21:45.245048+0000","last_undegraded":"2024-09-12T03:21:45.245048+0000","last_fullsized":"2024-09-12T03:21:45.245048+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_clean_scrub_stamp":"2024-09-12T03:21:45.231023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:21:46.237118+0000","last_change":"2024-09-12T03:21:46.237118+0000","last_active":"2024-09-12T03:21:46.237118+0000","last_peered":"2024-09-12T03:21:46.237118+0000","last_clean":"2024-09-12T03:21:46.237118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:21:46.237118+0000","last_undegraded":"2024-09-12T03:21:46.237118+0000","last_fullsized":"2024-09-12T03:21:46.237118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-12T03:21:45.245336+0000","last_change":"2024-09-12T03:21:45.245336+0000","last_active":"2024-09-12T03:21:45.231023+0000","last_peered":"2024-09-12T03:21:45.231023+0000","last_clean":"2024-09-12T03:21:45.231023+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:21:45.245336+0000","last_undegraded":"2024-09-12T03:21:45.245336+0000","last_fullsized":"2024-09-12T03:21:45.245336+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_clean_scrub_stamp":"2024-09-12T03:21:45.231023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:21:46.237118+0000","last_change":"2024-09-12T03:21:46.237118+0000","last_active":"2024-09-12T03:21:46.237118+0000","last_peered":"2024-09-12T03:21:46.237118+0000","last_clean":"2024-09-12T03:21:46.237118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:21:46.237118+0000","last_undegraded":"2024-09-12T03:21:46.237118+0000","last_fullsized":"2024-09-12T03:21:46.237118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-12T03:21:45.245243+0000","last_change":"2024-09-12T03:21:45.245243+0000","last_active":"2024-09-12T03:21:45.231023+0000","last_peered":"2024-09-12T03:21:45.231023+0000","last_clean":"2024-09-12T03:21:45.231023+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:21:45.245243+0000","last_undegraded":"2024-09-12T03:21:45.245243+0000","last_fullsized":"2024-09-12T03:21:45.245243+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_clean_scrub_stamp":"2024-09-12T03:21:45.231023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:21:46.237118+0000","last_change":"2024-09-12T03:21:46.237118+0000","last_active":"2024-09-12T03:21:46.237118+0000","last_peered":"2024-09-12T03:21:46.237118+0000","last_clean":"2024-09-12T03:21:46.237118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:21:46.237118+0000","last_undegraded":"2024-09-12T03:21:46.237118+0000","last_fullsized":"2024-09-12T03:21:46.237118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-12T03:21:45.240778+0000","last_change":"2024-09-12T03:21:45.240778+0000","last_active":"2024-09-12T03:21:45.231023+0000","last_peered":"2024-09-12T03:21:45.231023+0000","last_clean":"2024-09-12T03:21:45.231023+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:21:45.240778+0000","last_undegraded":"2024-09-12T03:21:45.240778+0000","last_fullsized":"2024-09-12T03:21:45.240778+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_clean_scrub_stamp":"2024-09-12T03:21:45.231023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:21:46.237118+0000","last_change":"2024-09-12T03:21:46.237118+0000","last_active":"2024-09-12T03:21:46.237118+0000","last_peered":"2024-09-12T03:21:46.237118+0000","last_clean":"2024-09-12T03:21:46.237118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:21:46.237118+0000","last_undegraded":"2024-09-12T03:21:46.237118+0000","last_fullsized":"2024-09-12T03:21:46.237118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-12T03:21:45.242034+0000","last_change":"2024-09-12T03:21:45.242034+0000","last_active":"2024-09-12T03:21:45.231023+0000","last_peered":"2024-09-12T03:21:45.231023+0000","last_clean":"2024-09-12T03:21:45.231023+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:21:45.242034+0000","last_undegraded":"2024-09-12T03:21:45.242034+0000","last_fullsized":"2024-09-12T03:21:45.242034+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_clean_scrub_stamp":"2024-09-12T03:21:45.231023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:21:46.237118+0000","last_change":"2024-09-12T03:21:46.237118+0000","last_active":"2024-09-12T03:21:46.237118+0000","last_peered":"2024-09-12T03:21:46.237118+0000","last_clean":"2024-09-12T03:21:46.237118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:21:46.237118+0000","last_undegraded":"2024-09-12T03:21:46.237118+0000","last_fullsized":"2024-09-12T03:21:46.237118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-12T03:21:45.250336+0000","last_change":"2024-09-12T03:21:45.250336+0000","last_active":"2024-09-12T03:21:45.231023+0000","last_peered":"2024-09-12T03:21:45.231023+0000","last_clean":"2024-09-12T03:21:45.231023+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:21:45.250336+0000","last_undegraded":"2024-09-12T03:21:45.250336+0000","last_fullsized":"2024-09-12T03:21:45.250336+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_clean_scrub_stamp":"2024-09-12T03:21:45.231023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:21:46.237118+0000","last_change":"2024-09-12T03:21:46.237118+0000","last_active":"2024-09-12T03:21:46.237118+0000","last_peered":"2024-09-12T03:21:46.237118+0000","last_clean":"2024-09-12T03:21:46.237118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:21:46.237118+0000","last_undegraded":"2024-09-12T03:21:46.237118+0000","last_fullsized":"2024-09-12T03:21:46.237118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-12T03:21:45.243710+0000","last_change":"2024-09-12T03:21:45.243710+0000","last_active":"2024-09-12T03:21:45.231023+0000","last_peered":"2024-09-12T03:21:45.231023+0000","last_clean":"2024-09-12T03:21:45.231023+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:21:45.243710+0000","last_undegraded":"2024-09-12T03:21:45.243710+0000","last_fullsized":"2024-09-12T03:21:45.243710+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_clean_scrub_stamp":"2024-09-12T03:21:45.231023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:21:46.237118+0000","last_change":"2024-09-12T03:21:46.237118+0000","last_active":"2024-09-12T03:21:46.237118+0000","last_peered":"2024-09-12T03:21:46.237118+0000","last_clean":"2024-09-12T03:21:46.237118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:21:46.237118+0000","last_undegraded":"2024-09-12T03:21:46.237118+0000","last_fullsized":"2024-09-12T03:21:46.237118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-12T03:21:45.250313+0000","last_change":"2024-09-12T03:21:45.250313+0000","last_active":"2024-09-12T03:21:45.231023+0000","last_peered":"2024-09-12T03:21:45.231023+0000","last_clean":"2024-09-12T03:21:45.231023+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:21:45.250313+0000","last_undegraded":"2024-09-12T03:21:45.250313+0000","last_fullsized":"2024-09-12T03:21:45.250313+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_clean_scrub_stamp":"2024-09-12T03:21:45.231023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:21:46.237118+0000","last_change":"2024-09-12T03:21:46.237118+0000","last_active":"2024-09-12T03:21:46.237118+0000","last_peered":"2024-09-12T03:21:46.237118+0000","last_clean":"2024-09-12T03:21:46.237118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:21:46.237118+0000","last_undegraded":"2024-09-12T03:21:46.237118+0000","last_fullsized":"2024-09-12T03:21:46.237118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-12T03:21:45.241930+0000","last_change":"2024-09-12T03:21:45.241930+0000","last_active":"2024-09-12T03:21:45.231023+0000","last_peered":"2024-09-12T03:21:45.231023+0000","last_clean":"2024-09-12T03:21:45.231023+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:21:45.241930+0000","last_undegraded":"2024-09-12T03:21:45.241930+0000","last_fullsized":"2024-09-12T03:21:45.241930+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_clean_scrub_stamp":"2024-09-12T03:21:45.231023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:21:46.237118+0000","last_change":"2024-09-12T03:21:46.237118+0000","last_active":"2024-09-12T03:21:46.237118+0000","last_peered":"2024-09-12T03:21:46.237118+0000","last_clean":"2024-09-12T03:21:46.237118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:21:46.237118+0000","last_undegraded":"2024-09-12T03:21:46.237118+0000","last_fullsized":"2024-09-12T03:21:46.237118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-12T03:21:45.244859+0000","last_change":"2024-09-12T03:21:45.244859+0000","last_active":"2024-09-12T03:21:45.231023+0000","last_peered":"2024-09-12T03:21:45.231023+0000","last_clean":"2024-09-12T03:21:45.231023+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:21:45.244859+0000","last_undegraded":"2024-09-12T03:21:45.244859+0000","last_fullsized":"2024-09-12T03:21:45.244859+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_clean_scrub_stamp":"2024-09-12T03:21:45.231023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:21:46.237118+0000","last_change":"2024-09-12T03:21:46.237118+0000","last_active":"2024-09-12T03:21:46.237118+0000","last_peered":"2024-09-12T03:21:46.237118+0000","last_clean":"2024-09-12T03:21:46.237118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:21:46.237118+0000","last_undegraded":"2024-09-12T03:21:46.237118+0000","last_fullsized":"2024-09-12T03:21:46.237118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-12T03:21:45.243709+0000","last_change":"2024-09-12T03:21:45.243709+0000","last_active":"2024-09-12T03:21:45.231023+0000","last_peered":"2024-09-12T03:21:45.231023+0000","last_clean":"2024-09-12T03:21:45.231023+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:21:45.243709+0000","last_undegraded":"2024-09-12T03:21:45.243709+0000","last_fullsized":"2024-09-12T03:21:45.243709+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_clean_scrub_stamp":"2024-09-12T03:21:45.231023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T03:21:46.237118+0000","last_change":"2024-09-12T03:21:46.237118+0000","last_active":"2024-09-12T03:21:46.237118+0000","last_peered":"2024-09-12T03:21:46.237118+0000","last_clean":"2024-09-12T03:21:46.237118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:21:46.237118+0000","last_undegraded":"2024-09-12T03:21:46.237118+0000","last_fullsized":"2024-09-12T03:21:46.237118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-12T03:21:45.242027+0000","last_change":"2024-09-12T03:21:45.242027+0000","last_active":"2024-09-12T03:21:45.231023+0000","last_peered":"2024-09-12T03:21:45.231023+0000","last_clean":"2024-09-12T03:21:45.231023+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:21:45.242027+0000","last_undegraded":"2024-09-12T03:21:45.242027+0000","last_fullsized":"2024-09-12T03:21:45.242027+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_clean_scrub_stamp":"2024-09-12T03:21:45.231023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"70'128","reported_seq":375,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-12T03:21:45.242677+0000","last_change":"2024-09-12T03:16:21.883446+0000","last_active":"2024-09-12T03:21:45.242677+0000","last_peered":"2024-09-12T03:21:45.242677+0000","last_clean":"2024-09-12T03:21:45.242677+0000","last_became_active":"2024-09-12T03:16:21.882207+0000","last_became_peered":"2024-09-12T03:16:21.882207+0000","last_unstale":"2024-09-12T03:21:45.242677+0000","last_undegraded":"2024-09-12T03:21:45.242677+0000","last_fullsized":"2024-09-12T03:21:45.242677+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:20.316426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:20.316426+0000","last_clean_scrub_stamp":"2024-09-12T03:16:20.316426+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:12:15.424320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":19,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542211,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37288,"kb_used_data":1932,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334552,"statfs":{"total":96636764160,"available":96598581248,"internally_reserved":0,"allocated":1978368,"data_stored":1198287,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542213,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36708,"kb_used_data":1352,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335132,"statfs":{"total":96636764160,"available":96599175168,"internally_reserved":0,"allocated":1384448,"data_stored":607919,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542212,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36700,"kb_used_data":1352,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335140,"statfs":{"total":96636764160,"available":96599183360,"internally_reserved":0,"allocated":1384448,"data_stored":607919,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542211,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36700,"kb_used_data":1352,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335140,"statfs":{"total":96636764160,"available":96599183360,"internally_reserved":0,"allocated":1384448,"data_stored":607919,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542212,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32544,"kb_used_data":1352,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339296,"statfs":{"total":96636764160,"available":96603439104,"internally_reserved":0,"allocated":1384448,"data_stored":607919,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542212,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32548,"kb_used_data":1352,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339292,"statfs":{"total":96636764160,"available":96603435008,"internally_reserved":0,"allocated":1384448,"data_stored":607919,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542213,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32528,"kb_used_data":1340,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339312,"statfs":{"total":96636764160,"available":96603455488,"internally_reserved":0,"allocated":1372160,"data_stored":602074,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542213,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33120,"kb_used_data":1932,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338720,"statfs":{"total":96636764160,"available":96602849280,"internally_reserved":0,"allocated":1978368,"data_stored":1198287,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T03:21:50.948 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-12T03:21:51.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.087+0000 7f7471842640 1 Processor -- start 2024-09-12T03:21:51.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.087+0000 7f7471842640 1 -- start start 2024-09-12T03:21:51.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.088+0000 7f7471842640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f746c15e780 0x7f746c0d3e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:51.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.088+0000 7f7471842640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f746c0d43b0 0x7f746c0d1280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:51.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.088+0000 7f7471842640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f746c15eb60 0x7f746c0d17c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:51.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.088+0000 7f7471842640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f746c0b7950 con 0x7f746c15eb60 2024-09-12T03:21:51.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.088+0000 7f7471842640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f746c0b7210 con 0x7f746c0d43b0 2024-09-12T03:21:51.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.088+0000 7f7471842640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f746c0b6690 con 0x7f746c15e780 2024-09-12T03:21:51.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.088+0000 7f746b7fe640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f746c15eb60 0x7f746c0d17c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:51.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.088+0000 7f746b7fe640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f746c15eb60 0x7f746c0d17c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39494/0 (socket says 172.21.15.27:39494) 2024-09-12T03:21:51.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.088+0000 7f746b7fe640 1 -- 172.21.15.27:0/2764832265 learned_addr learned my addr 172.21.15.27:0/2764832265 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:21:51.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.088+0000 7f746a7fc640 1 --2- 172.21.15.27:0/2764832265 >> v2:172.21.15.45:3300/0 conn(0x7f746c0d43b0 0x7f746c0d1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:51.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.088+0000 7f746affd640 1 --2- 172.21.15.27:0/2764832265 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f746c15e780 0x7f746c0d3e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:51.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.088+0000 7f746b7fe640 1 -- 172.21.15.27:0/2764832265 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f746c15e780 msgr2=0x7f746c0d3e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:51.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.088+0000 7f746b7fe640 1 --2- 172.21.15.27:0/2764832265 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f746c15e780 0x7f746c0d3e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:51.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.088+0000 7f746b7fe640 1 -- 172.21.15.27:0/2764832265 >> v2:172.21.15.45:3300/0 conn(0x7f746c0d43b0 msgr2=0x7f746c0d1280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:51.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.088+0000 7f746b7fe640 1 --2- 172.21.15.27:0/2764832265 >> v2:172.21.15.45:3300/0 conn(0x7f746c0d43b0 0x7f746c0d1280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:51.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.088+0000 7f746b7fe640 1 -- 172.21.15.27:0/2764832265 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f746c0b6970 con 0x7f746c15eb60 2024-09-12T03:21:51.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.089+0000 7f746b7fe640 1 --2- 172.21.15.27:0/2764832265 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f746c15eb60 0x7f746c0d17c0 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f746c0d59c0 tx=0x7f746402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ba24b708986cb34 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:51.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.089+0000 7f7469ffb640 1 -- 172.21.15.27:0/2764832265 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7464036070 con 0x7f746c15eb60 2024-09-12T03:21:51.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.089+0000 7f7469ffb640 1 -- 172.21.15.27:0/2764832265 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7464031600 con 0x7f746c15eb60 2024-09-12T03:21:51.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.089+0000 7f7469ffb640 1 -- 172.21.15.27:0/2764832265 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74640319d0 con 0x7f746c15eb60 2024-09-12T03:21:51.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.090+0000 7f7471842640 1 -- 172.21.15.27:0/2764832265 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f746c15eb60 msgr2=0x7f746c0d17c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:51.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.090+0000 7f7471842640 1 --2- 172.21.15.27:0/2764832265 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f746c15eb60 0x7f746c0d17c0 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f746c0d59c0 tx=0x7f746402a6a0 comp rx=0 tx=0).stop 2024-09-12T03:21:51.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.090+0000 7f7471842640 1 -- 172.21.15.27:0/2764832265 shutdown_connections 2024-09-12T03:21:51.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.090+0000 7f7471842640 1 --2- 172.21.15.27:0/2764832265 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f746c15eb60 0x7f746c0d17c0 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:51.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.090+0000 7f7471842640 1 --2- 172.21.15.27:0/2764832265 >> v2:172.21.15.45:3300/0 conn(0x7f746c0d43b0 0x7f746c0d1280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:51.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.090+0000 7f7471842640 1 --2- 172.21.15.27:0/2764832265 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f746c15e780 0x7f746c0d3e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:51.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.090+0000 7f7471842640 1 -- 172.21.15.27:0/2764832265 >> 172.21.15.27:0/2764832265 conn(0x7f746c06da60 msgr2=0x7f746c06de70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:51.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.090+0000 7f7471842640 1 -- 172.21.15.27:0/2764832265 shutdown_connections 2024-09-12T03:21:51.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.090+0000 7f7471842640 1 -- 172.21.15.27:0/2764832265 wait complete. 2024-09-12T03:21:51.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.091+0000 7f7471842640 1 Processor -- start 2024-09-12T03:21:51.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.091+0000 7f7471842640 1 -- start start 2024-09-12T03:21:51.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.092+0000 7f7471842640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f746c0d43b0 0x7f746c157f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:51.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.092+0000 7f7471842640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f746c15e780 0x7f746c161140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:51.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.092+0000 7f7471842640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f746c15eb60 0x7f746c161680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:51.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.092+0000 7f7471842640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f746c0b6340 con 0x7f746c15e780 2024-09-12T03:21:51.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.092+0000 7f7471842640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f746c0b6f30 con 0x7f746c0d43b0 2024-09-12T03:21:51.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.092+0000 7f7471842640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f746c0b3390 con 0x7f746c15eb60 2024-09-12T03:21:51.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.092+0000 7f746a7fc640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f746c15e780 0x7f746c161140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:51.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.092+0000 7f746a7fc640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f746c15e780 0x7f746c161140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39504/0 (socket says 172.21.15.27:39504) 2024-09-12T03:21:51.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.092+0000 7f746a7fc640 1 -- 172.21.15.27:0/1931313321 learned_addr learned my addr 172.21.15.27:0/1931313321 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:21:51.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.093+0000 7f746affd640 1 --2- 172.21.15.27:0/1931313321 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f746c0d43b0 0x7f746c157f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:51.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.093+0000 7f746b7fe640 1 --2- 172.21.15.27:0/1931313321 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f746c15eb60 0x7f746c161680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:51.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.093+0000 7f746a7fc640 1 -- 172.21.15.27:0/1931313321 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f746c15eb60 msgr2=0x7f746c161680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:51.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.093+0000 7f746a7fc640 1 --2- 172.21.15.27:0/1931313321 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f746c15eb60 0x7f746c161680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:51.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.093+0000 7f746a7fc640 1 -- 172.21.15.27:0/1931313321 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f746c0d43b0 msgr2=0x7f746c157f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:51.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.093+0000 7f746a7fc640 1 --2- 172.21.15.27:0/1931313321 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f746c0d43b0 0x7f746c157f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:51.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.093+0000 7f746a7fc640 1 -- 172.21.15.27:0/1931313321 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f746c1f42f0 con 0x7f746c15e780 2024-09-12T03:21:51.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.093+0000 7f746a7fc640 1 --2- 172.21.15.27:0/1931313321 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f746c15e780 0x7f746c161140 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f746000e860 tx=0x7f746000ed20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:51.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.093+0000 7f7453fff640 1 -- 172.21.15.27:0/1931313321 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7460017070 con 0x7f746c15e780 2024-09-12T03:21:51.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.094+0000 7f7453fff640 1 -- 172.21.15.27:0/1931313321 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f746000c940 con 0x7f746c15e780 2024-09-12T03:21:51.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.094+0000 7f7471842640 1 -- 172.21.15.27:0/1931313321 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f746c1f45d0 con 0x7f746c15e780 2024-09-12T03:21:51.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.094+0000 7f7453fff640 1 -- 172.21.15.27:0/1931313321 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f746000cd10 con 0x7f746c15e780 2024-09-12T03:21:51.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.094+0000 7f7471842640 1 -- 172.21.15.27:0/1931313321 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f746c1f4b90 con 0x7f746c15e780 2024-09-12T03:21:51.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.095+0000 7f7471842640 1 -- 172.21.15.27:0/1931313321 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f746c0d3e70 con 0x7f746c15e780 2024-09-12T03:21:51.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.095+0000 7f7453fff640 1 -- 172.21.15.27:0/1931313321 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 19) ==== 90855+0+0 (secure 0 0 0) 0x7f746001a070 con 0x7f746c15e780 2024-09-12T03:21:51.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.096+0000 7f7453fff640 1 --2- 172.21.15.27:0/1931313321 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f745806c220 0x7f745806e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:51.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.096+0000 7f7453fff640 1 -- 172.21.15.27:0/1931313321 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5737+0+0 (secure 0 0 0) 0x7f746008b5e0 con 0x7f746c15e780 2024-09-12T03:21:51.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.096+0000 7f746affd640 1 --2- 172.21.15.27:0/1931313321 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f745806c220 0x7f745806e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:51.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.096+0000 7f746affd640 1 --2- 172.21.15.27:0/1931313321 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f745806c220 0x7f745806e6e0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f745c009b20 tx=0x7f745c009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:51.100 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.100+0000 7f7453fff640 1 -- 172.21.15.27:0/1931313321 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f7460059340 con 0x7f746c15e780 2024-09-12T03:21:51.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.275+0000 7f7471842640 1 -- 172.21.15.27:0/1931313321 --> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f746c069ff0 con 0x7f745806c220 2024-09-12T03:21:51.281 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.281+0000 7f7453fff640 1 -- 172.21.15.27:0/1931313321 <== mgr.4113 v2:172.21.15.27:6836/1226920526 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252482 (secure 0 0 0) 0x7f746c069ff0 con 0x7f745806c220 2024-09-12T03:21:51.284 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:21:51.285 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2024-09-12T03:21:51.287 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.284+0000 7f7471842640 1 -- 172.21.15.27:0/1931313321 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f745806c220 msgr2=0x7f745806e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:51.287 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.284+0000 7f7471842640 1 --2- 172.21.15.27:0/1931313321 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f745806c220 0x7f745806e6e0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f745c009b20 tx=0x7f745c009190 comp rx=0 tx=0).stop 2024-09-12T03:21:51.287 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.284+0000 7f7471842640 1 -- 172.21.15.27:0/1931313321 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f746c15e780 msgr2=0x7f746c161140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:51.287 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.284+0000 7f7471842640 1 --2- 172.21.15.27:0/1931313321 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f746c15e780 0x7f746c161140 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f746000e860 tx=0x7f746000ed20 comp rx=0 tx=0).stop 2024-09-12T03:21:51.287 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.284+0000 7f7471842640 1 -- 172.21.15.27:0/1931313321 shutdown_connections 2024-09-12T03:21:51.287 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.284+0000 7f7471842640 1 --2- 172.21.15.27:0/1931313321 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f746c15eb60 0x7f746c161680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:51.287 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.284+0000 7f7471842640 1 --2- 172.21.15.27:0/1931313321 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f745806c220 0x7f745806e6e0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:51.287 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.284+0000 7f7471842640 1 --2- 172.21.15.27:0/1931313321 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f746c15e780 0x7f746c161140 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:51.287 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.284+0000 7f7471842640 1 --2- 172.21.15.27:0/1931313321 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f746c0d43b0 0x7f746c157f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:51.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.284+0000 7f7471842640 1 -- 172.21.15.27:0/1931313321 >> 172.21.15.27:0/1931313321 conn(0x7f746c06da60 msgr2=0x7f746c0b9420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:51.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.284+0000 7f7471842640 1 -- 172.21.15.27:0/1931313321 shutdown_connections 2024-09-12T03:21:51.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.284+0000 7f7471842640 1 -- 172.21.15.27:0/1931313321 wait complete. 2024-09-12T03:21:51.295 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":96,"stamp":"2024-09-12T03:21:51.115694+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":224,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":167,"ondisk_log_size":167,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":278496,"kb_used_data":12328,"kb_used_omap":12,"kb_used_meta":265971,"kb_avail":754696224,"statfs":{"total":773094113280,"available":772808933376,"internally_reserved":0,"allocated":12623872,"data_stored":6204380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":272354864},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":22,"apply_latency_ms":22,"commit_latency_ns":22000000,"apply_latency_ns":22000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.878018"},"pg_stats":[{"pgid":"19.3f","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.266518+0000","last_change":"2024-09-12T03:21:47.276464+0000","last_active":"2024-09-12T03:21:48.266518+0000","last_peered":"2024-09-12T03:21:48.266518+0000","last_clean":"2024-09-12T03:21:48.266518+0000","last_became_active":"2024-09-12T03:21:47.276166+0000","last_became_peered":"2024-09-12T03:21:47.276166+0000","last_unstale":"2024-09-12T03:21:48.266518+0000","last_undegraded":"2024-09-12T03:21:48.266518+0000","last_fullsized":"2024-09-12T03:21:48.266518+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:28:18.796133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3e","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.724634+0000","last_change":"2024-09-12T03:21:47.279512+0000","last_active":"2024-09-12T03:21:48.724634+0000","last_peered":"2024-09-12T03:21:48.724634+0000","last_clean":"2024-09-12T03:21:48.724634+0000","last_became_active":"2024-09-12T03:21:47.279237+0000","last_became_peered":"2024-09-12T03:21:47.279237+0000","last_unstale":"2024-09-12T03:21:48.724634+0000","last_undegraded":"2024-09-12T03:21:48.724634+0000","last_fullsized":"2024-09-12T03:21:48.724634+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:21:31.235603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3d","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.598938+0000","last_change":"2024-09-12T03:21:47.276803+0000","last_active":"2024-09-12T03:21:48.598938+0000","last_peered":"2024-09-12T03:21:48.598938+0000","last_clean":"2024-09-12T03:21:48.598938+0000","last_became_active":"2024-09-12T03:21:47.276525+0000","last_became_peered":"2024-09-12T03:21:47.276525+0000","last_unstale":"2024-09-12T03:21:48.598938+0000","last_undegraded":"2024-09-12T03:21:48.598938+0000","last_fullsized":"2024-09-12T03:21:48.598938+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:27:48.066859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.3c","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.266405+0000","last_change":"2024-09-12T03:21:47.277021+0000","last_active":"2024-09-12T03:21:48.266405+0000","last_peered":"2024-09-12T03:21:48.266405+0000","last_clean":"2024-09-12T03:21:48.266405+0000","last_became_active":"2024-09-12T03:21:47.275542+0000","last_became_peered":"2024-09-12T03:21:47.275542+0000","last_unstale":"2024-09-12T03:21:48.266405+0000","last_undegraded":"2024-09-12T03:21:48.266405+0000","last_fullsized":"2024-09-12T03:21:48.266405+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:40:16.335013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3b","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.273675+0000","last_change":"2024-09-12T03:21:47.274034+0000","last_active":"2024-09-12T03:21:48.273675+0000","last_peered":"2024-09-12T03:21:48.273675+0000","last_clean":"2024-09-12T03:21:48.273675+0000","last_became_active":"2024-09-12T03:21:47.273718+0000","last_became_peered":"2024-09-12T03:21:47.273718+0000","last_unstale":"2024-09-12T03:21:48.273675+0000","last_undegraded":"2024-09-12T03:21:48.273675+0000","last_fullsized":"2024-09-12T03:21:48.273675+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:07:27.386745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.3a","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.267033+0000","last_change":"2024-09-12T03:21:47.279653+0000","last_active":"2024-09-12T03:21:48.267033+0000","last_peered":"2024-09-12T03:21:48.267033+0000","last_clean":"2024-09-12T03:21:48.267033+0000","last_became_active":"2024-09-12T03:21:47.279340+0000","last_became_peered":"2024-09-12T03:21:47.279340+0000","last_unstale":"2024-09-12T03:21:48.267033+0000","last_undegraded":"2024-09-12T03:21:48.267033+0000","last_fullsized":"2024-09-12T03:21:48.267033+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:07:12.256007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.39","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.266736+0000","last_change":"2024-09-12T03:21:47.272385+0000","last_active":"2024-09-12T03:21:48.266736+0000","last_peered":"2024-09-12T03:21:48.266736+0000","last_clean":"2024-09-12T03:21:48.266736+0000","last_became_active":"2024-09-12T03:21:47.272056+0000","last_became_peered":"2024-09-12T03:21:47.272056+0000","last_unstale":"2024-09-12T03:21:48.266736+0000","last_undegraded":"2024-09-12T03:21:48.266736+0000","last_fullsized":"2024-09-12T03:21:48.266736+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:14:24.567859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.38","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.598920+0000","last_change":"2024-09-12T03:21:47.280231+0000","last_active":"2024-09-12T03:21:48.598920+0000","last_peered":"2024-09-12T03:21:48.598920+0000","last_clean":"2024-09-12T03:21:48.598920+0000","last_became_active":"2024-09-12T03:21:47.279961+0000","last_became_peered":"2024-09-12T03:21:47.279961+0000","last_unstale":"2024-09-12T03:21:48.598920+0000","last_undegraded":"2024-09-12T03:21:48.598920+0000","last_fullsized":"2024-09-12T03:21:48.598920+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:35:57.506654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.37","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.599097+0000","last_change":"2024-09-12T03:21:47.274676+0000","last_active":"2024-09-12T03:21:48.599097+0000","last_peered":"2024-09-12T03:21:48.599097+0000","last_clean":"2024-09-12T03:21:48.599097+0000","last_became_active":"2024-09-12T03:21:47.274394+0000","last_became_peered":"2024-09-12T03:21:47.274394+0000","last_unstale":"2024-09-12T03:21:48.599097+0000","last_undegraded":"2024-09-12T03:21:48.599097+0000","last_fullsized":"2024-09-12T03:21:48.599097+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:33:46.210575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.36","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.724767+0000","last_change":"2024-09-12T03:21:47.271566+0000","last_active":"2024-09-12T03:21:48.724767+0000","last_peered":"2024-09-12T03:21:48.724767+0000","last_clean":"2024-09-12T03:21:48.724767+0000","last_became_active":"2024-09-12T03:21:47.271260+0000","last_became_peered":"2024-09-12T03:21:47.271260+0000","last_unstale":"2024-09-12T03:21:48.724767+0000","last_undegraded":"2024-09-12T03:21:48.724767+0000","last_fullsized":"2024-09-12T03:21:48.724767+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:39:28.014015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.35","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.725510+0000","last_change":"2024-09-12T03:21:47.286426+0000","last_active":"2024-09-12T03:21:48.725510+0000","last_peered":"2024-09-12T03:21:48.725510+0000","last_clean":"2024-09-12T03:21:48.725510+0000","last_became_active":"2024-09-12T03:21:47.285471+0000","last_became_peered":"2024-09-12T03:21:47.285471+0000","last_unstale":"2024-09-12T03:21:48.725510+0000","last_undegraded":"2024-09-12T03:21:48.725510+0000","last_fullsized":"2024-09-12T03:21:48.725510+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:23:56.306775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.34","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.598790+0000","last_change":"2024-09-12T03:21:47.272716+0000","last_active":"2024-09-12T03:21:48.598790+0000","last_peered":"2024-09-12T03:21:48.598790+0000","last_clean":"2024-09-12T03:21:48.598790+0000","last_became_active":"2024-09-12T03:21:47.269444+0000","last_became_peered":"2024-09-12T03:21:47.269444+0000","last_unstale":"2024-09-12T03:21:48.598790+0000","last_undegraded":"2024-09-12T03:21:48.598790+0000","last_fullsized":"2024-09-12T03:21:48.598790+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:29:24.947639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.33","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.266441+0000","last_change":"2024-09-12T03:21:47.286251+0000","last_active":"2024-09-12T03:21:48.266441+0000","last_peered":"2024-09-12T03:21:48.266441+0000","last_clean":"2024-09-12T03:21:48.266441+0000","last_became_active":"2024-09-12T03:21:47.285959+0000","last_became_peered":"2024-09-12T03:21:47.285959+0000","last_unstale":"2024-09-12T03:21:48.266441+0000","last_undegraded":"2024-09-12T03:21:48.266441+0000","last_fullsized":"2024-09-12T03:21:48.266441+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:19:41.675227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.32","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.724693+0000","last_change":"2024-09-12T03:21:47.280594+0000","last_active":"2024-09-12T03:21:48.724693+0000","last_peered":"2024-09-12T03:21:48.724693+0000","last_clean":"2024-09-12T03:21:48.724693+0000","last_became_active":"2024-09-12T03:21:47.280250+0000","last_became_peered":"2024-09-12T03:21:47.280250+0000","last_unstale":"2024-09-12T03:21:48.724693+0000","last_undegraded":"2024-09-12T03:21:48.724693+0000","last_fullsized":"2024-09-12T03:21:48.724693+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:43:43.310362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.31","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.266159+0000","last_change":"2024-09-12T03:21:47.278494+0000","last_active":"2024-09-12T03:21:48.266159+0000","last_peered":"2024-09-12T03:21:48.266159+0000","last_clean":"2024-09-12T03:21:48.266159+0000","last_became_active":"2024-09-12T03:21:47.278228+0000","last_became_peered":"2024-09-12T03:21:47.278228+0000","last_unstale":"2024-09-12T03:21:48.266159+0000","last_undegraded":"2024-09-12T03:21:48.266159+0000","last_fullsized":"2024-09-12T03:21:48.266159+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:06:31.812980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.30","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.725675+0000","last_change":"2024-09-12T03:21:47.274919+0000","last_active":"2024-09-12T03:21:48.725675+0000","last_peered":"2024-09-12T03:21:48.725675+0000","last_clean":"2024-09-12T03:21:48.725675+0000","last_became_active":"2024-09-12T03:21:47.274611+0000","last_became_peered":"2024-09-12T03:21:47.274611+0000","last_unstale":"2024-09-12T03:21:48.725675+0000","last_undegraded":"2024-09-12T03:21:48.725675+0000","last_fullsized":"2024-09-12T03:21:48.725675+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:21:08.369709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2f","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.267173+0000","last_change":"2024-09-12T03:21:47.278293+0000","last_active":"2024-09-12T03:21:48.267173+0000","last_peered":"2024-09-12T03:21:48.267173+0000","last_clean":"2024-09-12T03:21:48.267173+0000","last_became_active":"2024-09-12T03:21:47.277990+0000","last_became_peered":"2024-09-12T03:21:47.277990+0000","last_unstale":"2024-09-12T03:21:48.267173+0000","last_undegraded":"2024-09-12T03:21:48.267173+0000","last_fullsized":"2024-09-12T03:21:48.267173+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:32:48.364609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.2e","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.724475+0000","last_change":"2024-09-12T03:21:47.267120+0000","last_active":"2024-09-12T03:21:48.724475+0000","last_peered":"2024-09-12T03:21:48.724475+0000","last_clean":"2024-09-12T03:21:48.724475+0000","last_became_active":"2024-09-12T03:21:47.266838+0000","last_became_peered":"2024-09-12T03:21:47.266838+0000","last_unstale":"2024-09-12T03:21:48.724475+0000","last_undegraded":"2024-09-12T03:21:48.724475+0000","last_fullsized":"2024-09-12T03:21:48.724475+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:20:42.594004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2d","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.725652+0000","last_change":"2024-09-12T03:21:47.281390+0000","last_active":"2024-09-12T03:21:48.725652+0000","last_peered":"2024-09-12T03:21:48.725652+0000","last_clean":"2024-09-12T03:21:48.725652+0000","last_became_active":"2024-09-12T03:21:47.281102+0000","last_became_peered":"2024-09-12T03:21:47.281102+0000","last_unstale":"2024-09-12T03:21:48.725652+0000","last_undegraded":"2024-09-12T03:21:48.725652+0000","last_fullsized":"2024-09-12T03:21:48.725652+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:49:41.579264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2c","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.724874+0000","last_change":"2024-09-12T03:21:47.271524+0000","last_active":"2024-09-12T03:21:48.724874+0000","last_peered":"2024-09-12T03:21:48.724874+0000","last_clean":"2024-09-12T03:21:48.724874+0000","last_became_active":"2024-09-12T03:21:47.271217+0000","last_became_peered":"2024-09-12T03:21:47.271217+0000","last_unstale":"2024-09-12T03:21:48.724874+0000","last_undegraded":"2024-09-12T03:21:48.724874+0000","last_fullsized":"2024-09-12T03:21:48.724874+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:08:22.543376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2b","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.273593+0000","last_change":"2024-09-12T03:21:47.274734+0000","last_active":"2024-09-12T03:21:48.273593+0000","last_peered":"2024-09-12T03:21:48.273593+0000","last_clean":"2024-09-12T03:21:48.273593+0000","last_became_active":"2024-09-12T03:21:47.274409+0000","last_became_peered":"2024-09-12T03:21:47.274409+0000","last_unstale":"2024-09-12T03:21:48.273593+0000","last_undegraded":"2024-09-12T03:21:48.273593+0000","last_fullsized":"2024-09-12T03:21:48.273593+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:00:08.281678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2a","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.724647+0000","last_change":"2024-09-12T03:21:47.285810+0000","last_active":"2024-09-12T03:21:48.724647+0000","last_peered":"2024-09-12T03:21:48.724647+0000","last_clean":"2024-09-12T03:21:48.724647+0000","last_became_active":"2024-09-12T03:21:47.285464+0000","last_became_peered":"2024-09-12T03:21:47.285464+0000","last_unstale":"2024-09-12T03:21:48.724647+0000","last_undegraded":"2024-09-12T03:21:48.724647+0000","last_fullsized":"2024-09-12T03:21:48.724647+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:27:08.761287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.29","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.598960+0000","last_change":"2024-09-12T03:21:47.274207+0000","last_active":"2024-09-12T03:21:48.598960+0000","last_peered":"2024-09-12T03:21:48.598960+0000","last_clean":"2024-09-12T03:21:48.598960+0000","last_became_active":"2024-09-12T03:21:47.273730+0000","last_became_peered":"2024-09-12T03:21:47.273730+0000","last_unstale":"2024-09-12T03:21:48.598960+0000","last_undegraded":"2024-09-12T03:21:48.598960+0000","last_fullsized":"2024-09-12T03:21:48.598960+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:24:38.533228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.28","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.724965+0000","last_change":"2024-09-12T03:21:47.275142+0000","last_active":"2024-09-12T03:21:48.724965+0000","last_peered":"2024-09-12T03:21:48.724965+0000","last_clean":"2024-09-12T03:21:48.724965+0000","last_became_active":"2024-09-12T03:21:47.274110+0000","last_became_peered":"2024-09-12T03:21:47.274110+0000","last_unstale":"2024-09-12T03:21:48.724965+0000","last_undegraded":"2024-09-12T03:21:48.724965+0000","last_fullsized":"2024-09-12T03:21:48.724965+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:37:40.004179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.27","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.724485+0000","last_change":"2024-09-12T03:21:47.269287+0000","last_active":"2024-09-12T03:21:48.724485+0000","last_peered":"2024-09-12T03:21:48.724485+0000","last_clean":"2024-09-12T03:21:48.724485+0000","last_became_active":"2024-09-12T03:21:47.268980+0000","last_became_peered":"2024-09-12T03:21:47.268980+0000","last_unstale":"2024-09-12T03:21:48.724485+0000","last_undegraded":"2024-09-12T03:21:48.724485+0000","last_fullsized":"2024-09-12T03:21:48.724485+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:05:34.381423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.26","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.724624+0000","last_change":"2024-09-12T03:21:47.278973+0000","last_active":"2024-09-12T03:21:48.724624+0000","last_peered":"2024-09-12T03:21:48.724624+0000","last_clean":"2024-09-12T03:21:48.724624+0000","last_became_active":"2024-09-12T03:21:47.278643+0000","last_became_peered":"2024-09-12T03:21:47.278643+0000","last_unstale":"2024-09-12T03:21:48.724624+0000","last_undegraded":"2024-09-12T03:21:48.724624+0000","last_fullsized":"2024-09-12T03:21:48.724624+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:49:07.547396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.25","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.266439+0000","last_change":"2024-09-12T03:21:47.279216+0000","last_active":"2024-09-12T03:21:48.266439+0000","last_peered":"2024-09-12T03:21:48.266439+0000","last_clean":"2024-09-12T03:21:48.266439+0000","last_became_active":"2024-09-12T03:21:47.278935+0000","last_became_peered":"2024-09-12T03:21:47.278935+0000","last_unstale":"2024-09-12T03:21:48.266439+0000","last_undegraded":"2024-09-12T03:21:48.266439+0000","last_fullsized":"2024-09-12T03:21:48.266439+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:57:37.592178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.24","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.267673+0000","last_change":"2024-09-12T03:21:47.278666+0000","last_active":"2024-09-12T03:21:48.267673+0000","last_peered":"2024-09-12T03:21:48.267673+0000","last_clean":"2024-09-12T03:21:48.267673+0000","last_became_active":"2024-09-12T03:21:47.278307+0000","last_became_peered":"2024-09-12T03:21:47.278307+0000","last_unstale":"2024-09-12T03:21:48.267673+0000","last_undegraded":"2024-09-12T03:21:48.267673+0000","last_fullsized":"2024-09-12T03:21:48.267673+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:39:28.014015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.23","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.266509+0000","last_change":"2024-09-12T03:21:47.281506+0000","last_active":"2024-09-12T03:21:48.266509+0000","last_peered":"2024-09-12T03:21:48.266509+0000","last_clean":"2024-09-12T03:21:48.266509+0000","last_became_active":"2024-09-12T03:21:47.281234+0000","last_became_peered":"2024-09-12T03:21:47.281234+0000","last_unstale":"2024-09-12T03:21:48.266509+0000","last_undegraded":"2024-09-12T03:21:48.266509+0000","last_fullsized":"2024-09-12T03:21:48.266509+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:15:33.521405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.22","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.266213+0000","last_change":"2024-09-12T03:21:47.277446+0000","last_active":"2024-09-12T03:21:48.266213+0000","last_peered":"2024-09-12T03:21:48.266213+0000","last_clean":"2024-09-12T03:21:48.266213+0000","last_became_active":"2024-09-12T03:21:47.276925+0000","last_became_peered":"2024-09-12T03:21:47.276925+0000","last_unstale":"2024-09-12T03:21:48.266213+0000","last_undegraded":"2024-09-12T03:21:48.266213+0000","last_fullsized":"2024-09-12T03:21:48.266213+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:29:38.307737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.21","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.273547+0000","last_change":"2024-09-12T03:21:47.283522+0000","last_active":"2024-09-12T03:21:48.273547+0000","last_peered":"2024-09-12T03:21:48.273547+0000","last_clean":"2024-09-12T03:21:48.273547+0000","last_became_active":"2024-09-12T03:21:47.283218+0000","last_became_peered":"2024-09-12T03:21:47.283218+0000","last_unstale":"2024-09-12T03:21:48.273547+0000","last_undegraded":"2024-09-12T03:21:48.273547+0000","last_fullsized":"2024-09-12T03:21:48.273547+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:07:41.072755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.20","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.266742+0000","last_change":"2024-09-12T03:21:47.280790+0000","last_active":"2024-09-12T03:21:48.266742+0000","last_peered":"2024-09-12T03:21:48.266742+0000","last_clean":"2024-09-12T03:21:48.266742+0000","last_became_active":"2024-09-12T03:21:47.280508+0000","last_became_peered":"2024-09-12T03:21:47.280508+0000","last_unstale":"2024-09-12T03:21:48.266742+0000","last_undegraded":"2024-09-12T03:21:48.266742+0000","last_fullsized":"2024-09-12T03:21:48.266742+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:38:27.802587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1e","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.724761+0000","last_change":"2024-09-12T03:21:47.274189+0000","last_active":"2024-09-12T03:21:48.724761+0000","last_peered":"2024-09-12T03:21:48.724761+0000","last_clean":"2024-09-12T03:21:48.724761+0000","last_became_active":"2024-09-12T03:21:47.273852+0000","last_became_peered":"2024-09-12T03:21:47.273852+0000","last_unstale":"2024-09-12T03:21:48.724761+0000","last_undegraded":"2024-09-12T03:21:48.724761+0000","last_fullsized":"2024-09-12T03:21:48.724761+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:42:44.733030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1f","version":"113'4","reported_seq":18,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.273969+0000","last_change":"2024-09-12T03:21:46.265345+0000","last_active":"2024-09-12T03:21:48.273969+0000","last_peered":"2024-09-12T03:21:48.273969+0000","last_clean":"2024-09-12T03:21:48.273969+0000","last_became_active":"2024-09-12T03:21:46.265014+0000","last_became_peered":"2024-09-12T03:21:46.265014+0000","last_unstale":"2024-09-12T03:21:48.273969+0000","last_undegraded":"2024-09-12T03:21:48.273969+0000","last_fullsized":"2024-09-12T03:21:48.273969+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_clean_scrub_stamp":"2024-09-12T03:21:45.231023+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:16:44.125918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.1f","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.599128+0000","last_change":"2024-09-12T03:21:47.278091+0000","last_active":"2024-09-12T03:21:48.599128+0000","last_peered":"2024-09-12T03:21:48.599128+0000","last_clean":"2024-09-12T03:21:48.599128+0000","last_became_active":"2024-09-12T03:21:47.277783+0000","last_became_peered":"2024-09-12T03:21:47.277783+0000","last_unstale":"2024-09-12T03:21:48.599128+0000","last_undegraded":"2024-09-12T03:21:48.599128+0000","last_fullsized":"2024-09-12T03:21:48.599128+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:42:53.625195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1e","version":"113'1","reported_seq":15,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.267990+0000","last_change":"2024-09-12T03:21:46.251686+0000","last_active":"2024-09-12T03:21:48.267990+0000","last_peered":"2024-09-12T03:21:48.267990+0000","last_clean":"2024-09-12T03:21:48.267990+0000","last_became_active":"2024-09-12T03:21:46.251055+0000","last_became_peered":"2024-09-12T03:21:46.251055+0000","last_unstale":"2024-09-12T03:21:48.267990+0000","last_undegraded":"2024-09-12T03:21:48.267990+0000","last_fullsized":"2024-09-12T03:21:48.267990+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_clean_scrub_stamp":"2024-09-12T03:21:45.231023+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:01:01.329193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.273734+0000","last_change":"2024-09-12T03:21:47.275088+0000","last_active":"2024-09-12T03:21:48.273734+0000","last_peered":"2024-09-12T03:21:48.273734+0000","last_clean":"2024-09-12T03:21:48.273734+0000","last_became_active":"2024-09-12T03:21:47.274745+0000","last_became_peered":"2024-09-12T03:21:47.274745+0000","last_unstale":"2024-09-12T03:21:48.273734+0000","last_undegraded":"2024-09-12T03:21:48.273734+0000","last_fullsized":"2024-09-12T03:21:48.273734+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:49:39.489484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1d","version":"113'2","reported_seq":16,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.725704+0000","last_change":"2024-09-12T03:21:46.258040+0000","last_active":"2024-09-12T03:21:48.725704+0000","last_peered":"2024-09-12T03:21:48.725704+0000","last_clean":"2024-09-12T03:21:48.725704+0000","last_became_active":"2024-09-12T03:21:46.257747+0000","last_became_peered":"2024-09-12T03:21:46.257747+0000","last_unstale":"2024-09-12T03:21:48.725704+0000","last_undegraded":"2024-09-12T03:21:48.725704+0000","last_fullsized":"2024-09-12T03:21:48.725704+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_clean_scrub_stamp":"2024-09-12T03:21:45.231023+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:07:24.863710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.1d","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.273966+0000","last_change":"2024-09-12T03:21:47.273642+0000","last_active":"2024-09-12T03:21:48.273966+0000","last_peered":"2024-09-12T03:21:48.273966+0000","last_clean":"2024-09-12T03:21:48.273966+0000","last_became_active":"2024-09-12T03:21:47.272464+0000","last_became_peered":"2024-09-12T03:21:47.272464+0000","last_unstale":"2024-09-12T03:21:48.273966+0000","last_undegraded":"2024-09-12T03:21:48.273966+0000","last_fullsized":"2024-09-12T03:21:48.273966+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:55:33.036487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1c","version":"113'2","reported_seq":16,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.725395+0000","last_change":"2024-09-12T03:21:46.266144+0000","last_active":"2024-09-12T03:21:48.725395+0000","last_peered":"2024-09-12T03:21:48.725395+0000","last_clean":"2024-09-12T03:21:48.725395+0000","last_became_active":"2024-09-12T03:21:46.265841+0000","last_became_peered":"2024-09-12T03:21:46.265841+0000","last_unstale":"2024-09-12T03:21:48.725395+0000","last_undegraded":"2024-09-12T03:21:48.725395+0000","last_fullsized":"2024-09-12T03:21:48.725395+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_clean_scrub_stamp":"2024-09-12T03:21:45.231023+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:46:45.472576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.725331+0000","last_change":"2024-09-12T03:21:47.263438+0000","last_active":"2024-09-12T03:21:48.725331+0000","last_peered":"2024-09-12T03:21:48.725331+0000","last_clean":"2024-09-12T03:21:48.725331+0000","last_became_active":"2024-09-12T03:21:47.263100+0000","last_became_peered":"2024-09-12T03:21:47.263100+0000","last_unstale":"2024-09-12T03:21:48.725331+0000","last_undegraded":"2024-09-12T03:21:48.725331+0000","last_fullsized":"2024-09-12T03:21:48.725331+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:45:51.865826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1b","version":"113'1","reported_seq":15,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.724274+0000","last_change":"2024-09-12T03:21:46.264671+0000","last_active":"2024-09-12T03:21:48.724274+0000","last_peered":"2024-09-12T03:21:48.724274+0000","last_clean":"2024-09-12T03:21:48.724274+0000","last_became_active":"2024-09-12T03:21:46.264133+0000","last_became_peered":"2024-09-12T03:21:46.264133+0000","last_unstale":"2024-09-12T03:21:48.724274+0000","last_undegraded":"2024-09-12T03:21:48.724274+0000","last_fullsized":"2024-09-12T03:21:48.724274+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_clean_scrub_stamp":"2024-09-12T03:21:45.231023+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:16:07.856351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1b","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.267682+0000","last_change":"2024-09-12T03:21:47.277615+0000","last_active":"2024-09-12T03:21:48.267682+0000","last_peered":"2024-09-12T03:21:48.267682+0000","last_clean":"2024-09-12T03:21:48.267682+0000","last_became_active":"2024-09-12T03:21:47.277321+0000","last_became_peered":"2024-09-12T03:21:47.277321+0000","last_unstale":"2024-09-12T03:21:48.267682+0000","last_undegraded":"2024-09-12T03:21:48.267682+0000","last_fullsized":"2024-09-12T03:21:48.267682+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:20:42.594004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1a","version":"113'1","reported_seq":15,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.725273+0000","last_change":"2024-09-12T03:21:46.263053+0000","last_active":"2024-09-12T03:21:48.725273+0000","last_peered":"2024-09-12T03:21:48.725273+0000","last_clean":"2024-09-12T03:21:48.725273+0000","last_became_active":"2024-09-12T03:21:46.262771+0000","last_became_peered":"2024-09-12T03:21:46.262771+0000","last_unstale":"2024-09-12T03:21:48.725273+0000","last_undegraded":"2024-09-12T03:21:48.725273+0000","last_fullsized":"2024-09-12T03:21:48.725273+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_clean_scrub_stamp":"2024-09-12T03:21:45.231023+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:35:56.500559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.599066+0000","last_change":"2024-09-12T03:21:47.281084+0000","last_active":"2024-09-12T03:21:48.599066+0000","last_peered":"2024-09-12T03:21:48.599066+0000","last_clean":"2024-09-12T03:21:48.599066+0000","last_became_active":"2024-09-12T03:21:47.280767+0000","last_became_peered":"2024-09-12T03:21:47.280767+0000","last_unstale":"2024-09-12T03:21:48.599066+0000","last_undegraded":"2024-09-12T03:21:48.599066+0000","last_fullsized":"2024-09-12T03:21:48.599066+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:16:45.132013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":14,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.266696+0000","last_change":"2024-09-12T03:21:46.253748+0000","last_active":"2024-09-12T03:21:48.266696+0000","last_peered":"2024-09-12T03:21:48.266696+0000","last_clean":"2024-09-12T03:21:48.266696+0000","last_became_active":"2024-09-12T03:21:46.253381+0000","last_became_peered":"2024-09-12T03:21:46.253381+0000","last_unstale":"2024-09-12T03:21:48.266696+0000","last_undegraded":"2024-09-12T03:21:48.266696+0000","last_fullsized":"2024-09-12T03:21:48.266696+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_clean_scrub_stamp":"2024-09-12T03:21:45.231023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:33:59.776477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.724917+0000","last_change":"2024-09-12T03:21:47.278433+0000","last_active":"2024-09-12T03:21:48.724917+0000","last_peered":"2024-09-12T03:21:48.724917+0000","last_clean":"2024-09-12T03:21:48.724917+0000","last_became_active":"2024-09-12T03:21:47.278146+0000","last_became_peered":"2024-09-12T03:21:47.278146+0000","last_unstale":"2024-09-12T03:21:48.724917+0000","last_undegraded":"2024-09-12T03:21:48.724917+0000","last_fullsized":"2024-09-12T03:21:48.724917+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:02:37.440594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":14,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.724426+0000","last_change":"2024-09-12T03:21:46.264728+0000","last_active":"2024-09-12T03:21:48.724426+0000","last_peered":"2024-09-12T03:21:48.724426+0000","last_clean":"2024-09-12T03:21:48.724426+0000","last_became_active":"2024-09-12T03:21:46.263798+0000","last_became_peered":"2024-09-12T03:21:46.263798+0000","last_unstale":"2024-09-12T03:21:48.724426+0000","last_undegraded":"2024-09-12T03:21:48.724426+0000","last_fullsized":"2024-09-12T03:21:48.724426+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_clean_scrub_stamp":"2024-09-12T03:21:45.231023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:12:16.266409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.266463+0000","last_change":"2024-09-12T03:21:47.277867+0000","last_active":"2024-09-12T03:21:48.266463+0000","last_peered":"2024-09-12T03:21:48.266463+0000","last_clean":"2024-09-12T03:21:48.266463+0000","last_became_active":"2024-09-12T03:21:47.277538+0000","last_became_peered":"2024-09-12T03:21:47.277538+0000","last_unstale":"2024-09-12T03:21:48.266463+0000","last_undegraded":"2024-09-12T03:21:48.266463+0000","last_fullsized":"2024-09-12T03:21:48.266463+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:33:42.064998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":14,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.268143+0000","last_change":"2024-09-12T03:21:46.257496+0000","last_active":"2024-09-12T03:21:48.268143+0000","last_peered":"2024-09-12T03:21:48.268143+0000","last_clean":"2024-09-12T03:21:48.268143+0000","last_became_active":"2024-09-12T03:21:46.257152+0000","last_became_peered":"2024-09-12T03:21:46.257152+0000","last_unstale":"2024-09-12T03:21:48.268143+0000","last_undegraded":"2024-09-12T03:21:48.268143+0000","last_fullsized":"2024-09-12T03:21:48.268143+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_clean_scrub_stamp":"2024-09-12T03:21:45.231023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:29:42.420816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.268097+0000","last_change":"2024-09-12T03:21:47.277451+0000","last_active":"2024-09-12T03:21:48.268097+0000","last_peered":"2024-09-12T03:21:48.268097+0000","last_clean":"2024-09-12T03:21:48.268097+0000","last_became_active":"2024-09-12T03:21:47.277147+0000","last_became_peered":"2024-09-12T03:21:47.277147+0000","last_unstale":"2024-09-12T03:21:48.268097+0000","last_undegraded":"2024-09-12T03:21:48.268097+0000","last_fullsized":"2024-09-12T03:21:48.268097+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:02:37.440594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":14,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.724804+0000","last_change":"2024-09-12T03:21:46.254525+0000","last_active":"2024-09-12T03:21:48.724804+0000","last_peered":"2024-09-12T03:21:48.724804+0000","last_clean":"2024-09-12T03:21:48.724804+0000","last_became_active":"2024-09-12T03:21:46.254190+0000","last_became_peered":"2024-09-12T03:21:46.254190+0000","last_unstale":"2024-09-12T03:21:48.724804+0000","last_undegraded":"2024-09-12T03:21:48.724804+0000","last_fullsized":"2024-09-12T03:21:48.724804+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_clean_scrub_stamp":"2024-09-12T03:21:45.231023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:16:44.125918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.14","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.273888+0000","last_change":"2024-09-12T03:21:47.278622+0000","last_active":"2024-09-12T03:21:48.273888+0000","last_peered":"2024-09-12T03:21:48.273888+0000","last_clean":"2024-09-12T03:21:48.273888+0000","last_became_active":"2024-09-12T03:21:47.278333+0000","last_became_peered":"2024-09-12T03:21:47.278333+0000","last_unstale":"2024-09-12T03:21:48.273888+0000","last_undegraded":"2024-09-12T03:21:48.273888+0000","last_fullsized":"2024-09-12T03:21:48.273888+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:37:21.066404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.15","version":"113'3","reported_seq":17,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.266666+0000","last_change":"2024-09-12T03:21:46.252566+0000","last_active":"2024-09-12T03:21:48.266666+0000","last_peered":"2024-09-12T03:21:48.266666+0000","last_clean":"2024-09-12T03:21:48.266666+0000","last_became_active":"2024-09-12T03:21:46.252247+0000","last_became_peered":"2024-09-12T03:21:46.252247+0000","last_unstale":"2024-09-12T03:21:48.266666+0000","last_undegraded":"2024-09-12T03:21:48.266666+0000","last_fullsized":"2024-09-12T03:21:48.266666+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_clean_scrub_stamp":"2024-09-12T03:21:45.231023+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:09:06.932896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.15","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.274070+0000","last_change":"2024-09-12T03:21:47.271363+0000","last_active":"2024-09-12T03:21:48.274070+0000","last_peered":"2024-09-12T03:21:48.274070+0000","last_clean":"2024-09-12T03:21:48.274070+0000","last_became_active":"2024-09-12T03:21:47.271060+0000","last_became_peered":"2024-09-12T03:21:47.271060+0000","last_unstale":"2024-09-12T03:21:48.274070+0000","last_undegraded":"2024-09-12T03:21:48.274070+0000","last_fullsized":"2024-09-12T03:21:48.274070+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:26:02.679898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.14","version":"113'6","reported_seq":20,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.598854+0000","last_change":"2024-09-12T03:21:46.255120+0000","last_active":"2024-09-12T03:21:48.598854+0000","last_peered":"2024-09-12T03:21:48.598854+0000","last_clean":"2024-09-12T03:21:48.598854+0000","last_became_active":"2024-09-12T03:21:46.254812+0000","last_became_peered":"2024-09-12T03:21:46.254812+0000","last_unstale":"2024-09-12T03:21:48.598854+0000","last_undegraded":"2024-09-12T03:21:48.598854+0000","last_fullsized":"2024-09-12T03:21:48.598854+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_clean_scrub_stamp":"2024-09-12T03:21:45.231023+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:24:52.363839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.13","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.266605+0000","last_change":"2024-09-12T03:21:47.283523+0000","last_active":"2024-09-12T03:21:48.266605+0000","last_peered":"2024-09-12T03:21:48.266605+0000","last_clean":"2024-09-12T03:21:48.266605+0000","last_became_active":"2024-09-12T03:21:47.279638+0000","last_became_peered":"2024-09-12T03:21:47.279638+0000","last_unstale":"2024-09-12T03:21:48.266605+0000","last_undegraded":"2024-09-12T03:21:48.266605+0000","last_fullsized":"2024-09-12T03:21:48.266605+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:10:22.061518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":14,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.266298+0000","last_change":"2024-09-12T03:21:46.260129+0000","last_active":"2024-09-12T03:21:48.266298+0000","last_peered":"2024-09-12T03:21:48.266298+0000","last_clean":"2024-09-12T03:21:48.266298+0000","last_became_active":"2024-09-12T03:21:46.259758+0000","last_became_peered":"2024-09-12T03:21:46.259758+0000","last_unstale":"2024-09-12T03:21:48.266298+0000","last_undegraded":"2024-09-12T03:21:48.266298+0000","last_fullsized":"2024-09-12T03:21:48.266298+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_clean_scrub_stamp":"2024-09-12T03:21:45.231023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:39:56.115395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.10","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.724236+0000","last_change":"2024-09-12T03:21:47.278630+0000","last_active":"2024-09-12T03:21:48.724236+0000","last_peered":"2024-09-12T03:21:48.724236+0000","last_clean":"2024-09-12T03:21:48.724236+0000","last_became_active":"2024-09-12T03:21:47.278333+0000","last_became_peered":"2024-09-12T03:21:47.278333+0000","last_unstale":"2024-09-12T03:21:48.724236+0000","last_undegraded":"2024-09-12T03:21:48.724236+0000","last_fullsized":"2024-09-12T03:21:48.724236+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:31:24.916429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":14,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.725104+0000","last_change":"2024-09-12T03:21:46.263275+0000","last_active":"2024-09-12T03:21:48.725104+0000","last_peered":"2024-09-12T03:21:48.725104+0000","last_clean":"2024-09-12T03:21:48.725104+0000","last_became_active":"2024-09-12T03:21:46.262982+0000","last_became_peered":"2024-09-12T03:21:46.262982+0000","last_unstale":"2024-09-12T03:21:48.725104+0000","last_undegraded":"2024-09-12T03:21:48.725104+0000","last_fullsized":"2024-09-12T03:21:48.725104+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_clean_scrub_stamp":"2024-09-12T03:21:45.231023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:35:47.022417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.599034+0000","last_change":"2024-09-12T03:21:47.274722+0000","last_active":"2024-09-12T03:21:48.599034+0000","last_peered":"2024-09-12T03:21:48.599034+0000","last_clean":"2024-09-12T03:21:48.599034+0000","last_became_active":"2024-09-12T03:21:47.274319+0000","last_became_peered":"2024-09-12T03:21:47.274319+0000","last_unstale":"2024-09-12T03:21:48.599034+0000","last_undegraded":"2024-09-12T03:21:48.599034+0000","last_fullsized":"2024-09-12T03:21:48.599034+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:36:45.878775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.10","version":"113'8","reported_seq":22,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.599010+0000","last_change":"2024-09-12T03:21:46.254895+0000","last_active":"2024-09-12T03:21:48.599010+0000","last_peered":"2024-09-12T03:21:48.599010+0000","last_clean":"2024-09-12T03:21:48.599010+0000","last_became_active":"2024-09-12T03:21:46.254582+0000","last_became_peered":"2024-09-12T03:21:46.254582+0000","last_unstale":"2024-09-12T03:21:48.599010+0000","last_undegraded":"2024-09-12T03:21:48.599010+0000","last_fullsized":"2024-09-12T03:21:48.599010+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_clean_scrub_stamp":"2024-09-12T03:21:45.231023+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:40:18.756382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.e","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.598877+0000","last_change":"2024-09-12T03:21:47.279358+0000","last_active":"2024-09-12T03:21:48.598877+0000","last_peered":"2024-09-12T03:21:48.598877+0000","last_clean":"2024-09-12T03:21:48.598877+0000","last_became_active":"2024-09-12T03:21:47.279082+0000","last_became_peered":"2024-09-12T03:21:47.279082+0000","last_unstale":"2024-09-12T03:21:48.598877+0000","last_undegraded":"2024-09-12T03:21:48.598877+0000","last_fullsized":"2024-09-12T03:21:48.598877+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:36:02.708847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.f","version":"113'1","reported_seq":15,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.268246+0000","last_change":"2024-09-12T03:21:46.255444+0000","last_active":"2024-09-12T03:21:48.268246+0000","last_peered":"2024-09-12T03:21:48.268246+0000","last_clean":"2024-09-12T03:21:48.268246+0000","last_became_active":"2024-09-12T03:21:46.255111+0000","last_became_peered":"2024-09-12T03:21:46.255111+0000","last_unstale":"2024-09-12T03:21:48.268246+0000","last_undegraded":"2024-09-12T03:21:48.268246+0000","last_fullsized":"2024-09-12T03:21:48.268246+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_clean_scrub_stamp":"2024-09-12T03:21:45.231023+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:01:19.852084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.f","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.268199+0000","last_change":"2024-09-12T03:21:47.275553+0000","last_active":"2024-09-12T03:21:48.268199+0000","last_peered":"2024-09-12T03:21:48.268199+0000","last_clean":"2024-09-12T03:21:48.268199+0000","last_became_active":"2024-09-12T03:21:47.275232+0000","last_became_peered":"2024-09-12T03:21:47.275232+0000","last_unstale":"2024-09-12T03:21:48.268199+0000","last_undegraded":"2024-09-12T03:21:48.268199+0000","last_fullsized":"2024-09-12T03:21:48.268199+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:42:51.860983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.e","version":"113'2","reported_seq":16,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.725806+0000","last_change":"2024-09-12T03:21:46.253780+0000","last_active":"2024-09-12T03:21:48.725806+0000","last_peered":"2024-09-12T03:21:48.725806+0000","last_clean":"2024-09-12T03:21:48.725806+0000","last_became_active":"2024-09-12T03:21:46.253480+0000","last_became_peered":"2024-09-12T03:21:46.253480+0000","last_unstale":"2024-09-12T03:21:48.725806+0000","last_undegraded":"2024-09-12T03:21:48.725806+0000","last_fullsized":"2024-09-12T03:21:48.725806+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_clean_scrub_stamp":"2024-09-12T03:21:45.231023+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:38:27.071851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.725011+0000","last_change":"2024-09-12T03:21:47.279125+0000","last_active":"2024-09-12T03:21:48.725011+0000","last_peered":"2024-09-12T03:21:48.725011+0000","last_clean":"2024-09-12T03:21:48.725011+0000","last_became_active":"2024-09-12T03:21:47.278602+0000","last_became_peered":"2024-09-12T03:21:47.278602+0000","last_unstale":"2024-09-12T03:21:48.725011+0000","last_undegraded":"2024-09-12T03:21:48.725011+0000","last_fullsized":"2024-09-12T03:21:48.725011+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:07:41.072755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.d","version":"113'1","reported_seq":15,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.267378+0000","last_change":"2024-09-12T03:21:46.251440+0000","last_active":"2024-09-12T03:21:48.267378+0000","last_peered":"2024-09-12T03:21:48.267378+0000","last_clean":"2024-09-12T03:21:48.267378+0000","last_became_active":"2024-09-12T03:21:46.251107+0000","last_became_peered":"2024-09-12T03:21:46.251107+0000","last_unstale":"2024-09-12T03:21:48.267378+0000","last_undegraded":"2024-09-12T03:21:48.267378+0000","last_fullsized":"2024-09-12T03:21:48.267378+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_clean_scrub_stamp":"2024-09-12T03:21:45.231023+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:29:23.941544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.725059+0000","last_change":"2024-09-12T03:21:47.278687+0000","last_active":"2024-09-12T03:21:48.725059+0000","last_peered":"2024-09-12T03:21:48.725059+0000","last_clean":"2024-09-12T03:21:48.725059+0000","last_became_active":"2024-09-12T03:21:47.277859+0000","last_became_peered":"2024-09-12T03:21:47.277859+0000","last_unstale":"2024-09-12T03:21:48.725059+0000","last_undegraded":"2024-09-12T03:21:48.725059+0000","last_fullsized":"2024-09-12T03:21:48.725059+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:17:45.434501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":14,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.267750+0000","last_change":"2024-09-12T03:21:46.249240+0000","last_active":"2024-09-12T03:21:48.267750+0000","last_peered":"2024-09-12T03:21:48.267750+0000","last_clean":"2024-09-12T03:21:48.267750+0000","last_became_active":"2024-09-12T03:21:46.248701+0000","last_became_peered":"2024-09-12T03:21:46.248701+0000","last_unstale":"2024-09-12T03:21:48.267750+0000","last_undegraded":"2024-09-12T03:21:48.267750+0000","last_fullsized":"2024-09-12T03:21:48.267750+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_clean_scrub_stamp":"2024-09-12T03:21:45.231023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:50:01.855914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.266214+0000","last_change":"2024-09-12T03:21:47.282909+0000","last_active":"2024-09-12T03:21:48.266214+0000","last_peered":"2024-09-12T03:21:48.266214+0000","last_clean":"2024-09-12T03:21:48.266214+0000","last_became_active":"2024-09-12T03:21:47.282527+0000","last_became_peered":"2024-09-12T03:21:47.282527+0000","last_unstale":"2024-09-12T03:21:48.266214+0000","last_undegraded":"2024-09-12T03:21:48.266214+0000","last_fullsized":"2024-09-12T03:21:48.266214+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:48:16.365941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.b","version":"113'2","reported_seq":16,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.267395+0000","last_change":"2024-09-12T03:21:46.262019+0000","last_active":"2024-09-12T03:21:48.267395+0000","last_peered":"2024-09-12T03:21:48.267395+0000","last_clean":"2024-09-12T03:21:48.267395+0000","last_became_active":"2024-09-12T03:21:46.261674+0000","last_became_peered":"2024-09-12T03:21:46.261674+0000","last_unstale":"2024-09-12T03:21:48.267395+0000","last_undegraded":"2024-09-12T03:21:48.267395+0000","last_fullsized":"2024-09-12T03:21:48.267395+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_clean_scrub_stamp":"2024-09-12T03:21:45.231023+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:16:38.849993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.b","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.267739+0000","last_change":"2024-09-12T03:21:47.272033+0000","last_active":"2024-09-12T03:21:48.267739+0000","last_peered":"2024-09-12T03:21:48.267739+0000","last_clean":"2024-09-12T03:21:48.267739+0000","last_became_active":"2024-09-12T03:21:47.271715+0000","last_became_peered":"2024-09-12T03:21:47.271715+0000","last_unstale":"2024-09-12T03:21:48.267739+0000","last_undegraded":"2024-09-12T03:21:48.267739+0000","last_fullsized":"2024-09-12T03:21:48.267739+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:49:07.547396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":14,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.725365+0000","last_change":"2024-09-12T03:21:46.255383+0000","last_active":"2024-09-12T03:21:48.725365+0000","last_peered":"2024-09-12T03:21:48.725365+0000","last_clean":"2024-09-12T03:21:48.725365+0000","last_became_active":"2024-09-12T03:21:46.254479+0000","last_became_peered":"2024-09-12T03:21:46.254479+0000","last_unstale":"2024-09-12T03:21:48.725365+0000","last_undegraded":"2024-09-12T03:21:48.725365+0000","last_fullsized":"2024-09-12T03:21:48.725365+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_clean_scrub_stamp":"2024-09-12T03:21:45.231023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:36:44.872680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.724377+0000","last_change":"2024-09-12T03:21:47.276243+0000","last_active":"2024-09-12T03:21:48.724377+0000","last_peered":"2024-09-12T03:21:48.724377+0000","last_clean":"2024-09-12T03:21:48.724377+0000","last_became_active":"2024-09-12T03:21:47.275973+0000","last_became_peered":"2024-09-12T03:21:47.275973+0000","last_unstale":"2024-09-12T03:21:48.724377+0000","last_undegraded":"2024-09-12T03:21:48.724377+0000","last_fullsized":"2024-09-12T03:21:48.724377+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:40:21.225897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":14,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.725355+0000","last_change":"2024-09-12T03:21:46.256740+0000","last_active":"2024-09-12T03:21:48.725355+0000","last_peered":"2024-09-12T03:21:48.725355+0000","last_clean":"2024-09-12T03:21:48.725355+0000","last_became_active":"2024-09-12T03:21:46.256157+0000","last_became_peered":"2024-09-12T03:21:46.256157+0000","last_unstale":"2024-09-12T03:21:48.725355+0000","last_undegraded":"2024-09-12T03:21:48.725355+0000","last_fullsized":"2024-09-12T03:21:48.725355+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_clean_scrub_stamp":"2024-09-12T03:21:45.231023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:27:47.060764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.273562+0000","last_change":"2024-09-12T03:21:47.282425+0000","last_active":"2024-09-12T03:21:48.273562+0000","last_peered":"2024-09-12T03:21:48.273562+0000","last_clean":"2024-09-12T03:21:48.273562+0000","last_became_active":"2024-09-12T03:21:47.282126+0000","last_became_peered":"2024-09-12T03:21:47.282126+0000","last_unstale":"2024-09-12T03:21:48.273562+0000","last_undegraded":"2024-09-12T03:21:48.273562+0000","last_fullsized":"2024-09-12T03:21:48.273562+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:45:51.865826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":14,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.725014+0000","last_change":"2024-09-12T03:21:46.253061+0000","last_active":"2024-09-12T03:21:48.725014+0000","last_peered":"2024-09-12T03:21:48.725014+0000","last_clean":"2024-09-12T03:21:48.725014+0000","last_became_active":"2024-09-12T03:21:46.252712+0000","last_became_peered":"2024-09-12T03:21:46.252712+0000","last_unstale":"2024-09-12T03:21:48.725014+0000","last_undegraded":"2024-09-12T03:21:48.725014+0000","last_fullsized":"2024-09-12T03:21:48.725014+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_clean_scrub_stamp":"2024-09-12T03:21:45.231023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:56:16.255201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.267454+0000","last_change":"2024-09-12T03:21:47.266290+0000","last_active":"2024-09-12T03:21:48.267454+0000","last_peered":"2024-09-12T03:21:48.267454+0000","last_clean":"2024-09-12T03:21:48.267454+0000","last_became_active":"2024-09-12T03:21:47.264199+0000","last_became_peered":"2024-09-12T03:21:47.264199+0000","last_unstale":"2024-09-12T03:21:48.267454+0000","last_undegraded":"2024-09-12T03:21:48.267454+0000","last_fullsized":"2024-09-12T03:21:48.267454+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:41:26.120489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.7","version":"113'2","reported_seq":16,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.266586+0000","last_change":"2024-09-12T03:21:46.260654+0000","last_active":"2024-09-12T03:21:48.266586+0000","last_peered":"2024-09-12T03:21:48.266586+0000","last_clean":"2024-09-12T03:21:48.266586+0000","last_became_active":"2024-09-12T03:21:46.260300+0000","last_became_peered":"2024-09-12T03:21:46.260300+0000","last_unstale":"2024-09-12T03:21:48.266586+0000","last_undegraded":"2024-09-12T03:21:48.266586+0000","last_fullsized":"2024-09-12T03:21:48.266586+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_clean_scrub_stamp":"2024-09-12T03:21:45.231023+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:42:29.591906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.7","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.266487+0000","last_change":"2024-09-12T03:21:47.282067+0000","last_active":"2024-09-12T03:21:48.266487+0000","last_peered":"2024-09-12T03:21:48.266487+0000","last_clean":"2024-09-12T03:21:48.266487+0000","last_became_active":"2024-09-12T03:21:47.276839+0000","last_became_peered":"2024-09-12T03:21:47.276839+0000","last_unstale":"2024-09-12T03:21:48.266487+0000","last_undegraded":"2024-09-12T03:21:48.266487+0000","last_fullsized":"2024-09-12T03:21:48.266487+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:50:01.620996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.6","version":"114'1","reported_seq":15,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.273910+0000","last_change":"2024-09-12T03:21:46.259229+0000","last_active":"2024-09-12T03:21:48.273910+0000","last_peered":"2024-09-12T03:21:48.273910+0000","last_clean":"2024-09-12T03:21:48.273910+0000","last_became_active":"2024-09-12T03:21:46.258467+0000","last_became_peered":"2024-09-12T03:21:46.258467+0000","last_unstale":"2024-09-12T03:21:48.273910+0000","last_undegraded":"2024-09-12T03:21:48.273910+0000","last_fullsized":"2024-09-12T03:21:48.273910+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_clean_scrub_stamp":"2024-09-12T03:21:45.231023+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:36:44.872680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.725277+0000","last_change":"2024-09-12T03:21:47.273262+0000","last_active":"2024-09-12T03:21:48.725277+0000","last_peered":"2024-09-12T03:21:48.725277+0000","last_clean":"2024-09-12T03:21:48.725277+0000","last_became_active":"2024-09-12T03:21:47.272986+0000","last_became_peered":"2024-09-12T03:21:47.272986+0000","last_unstale":"2024-09-12T03:21:48.725277+0000","last_undegraded":"2024-09-12T03:21:48.725277+0000","last_fullsized":"2024-09-12T03:21:48.725277+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:22:36.819977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":14,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.725080+0000","last_change":"2024-09-12T03:21:46.264077+0000","last_active":"2024-09-12T03:21:48.725080+0000","last_peered":"2024-09-12T03:21:48.725080+0000","last_clean":"2024-09-12T03:21:48.725080+0000","last_became_active":"2024-09-12T03:21:46.262611+0000","last_became_peered":"2024-09-12T03:21:46.262611+0000","last_unstale":"2024-09-12T03:21:48.725080+0000","last_undegraded":"2024-09-12T03:21:48.725080+0000","last_fullsized":"2024-09-12T03:21:48.725080+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_clean_scrub_stamp":"2024-09-12T03:21:45.231023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:02:58.180807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.267440+0000","last_change":"2024-09-12T03:21:47.280655+0000","last_active":"2024-09-12T03:21:48.267440+0000","last_peered":"2024-09-12T03:21:48.267440+0000","last_clean":"2024-09-12T03:21:48.267440+0000","last_became_active":"2024-09-12T03:21:47.280133+0000","last_became_peered":"2024-09-12T03:21:47.280133+0000","last_unstale":"2024-09-12T03:21:48.267440+0000","last_undegraded":"2024-09-12T03:21:48.267440+0000","last_fullsized":"2024-09-12T03:21:48.267440+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:54:34.656137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":14,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.725663+0000","last_change":"2024-09-12T03:21:46.265178+0000","last_active":"2024-09-12T03:21:48.725663+0000","last_peered":"2024-09-12T03:21:48.725663+0000","last_clean":"2024-09-12T03:21:48.725663+0000","last_became_active":"2024-09-12T03:21:46.264842+0000","last_became_peered":"2024-09-12T03:21:46.264842+0000","last_unstale":"2024-09-12T03:21:48.725663+0000","last_undegraded":"2024-09-12T03:21:48.725663+0000","last_fullsized":"2024-09-12T03:21:48.725663+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_clean_scrub_stamp":"2024-09-12T03:21:45.231023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:51:50.034404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.273569+0000","last_change":"2024-09-12T03:21:47.275959+0000","last_active":"2024-09-12T03:21:48.273569+0000","last_peered":"2024-09-12T03:21:48.273569+0000","last_clean":"2024-09-12T03:21:48.273569+0000","last_became_active":"2024-09-12T03:21:47.274773+0000","last_became_peered":"2024-09-12T03:21:47.274773+0000","last_unstale":"2024-09-12T03:21:48.273569+0000","last_undegraded":"2024-09-12T03:21:48.273569+0000","last_fullsized":"2024-09-12T03:21:48.273569+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:21:08.369709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":14,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.724357+0000","last_change":"2024-09-12T03:21:46.265078+0000","last_active":"2024-09-12T03:21:48.724357+0000","last_peered":"2024-09-12T03:21:48.724357+0000","last_clean":"2024-09-12T03:21:48.724357+0000","last_became_active":"2024-09-12T03:21:46.264065+0000","last_became_peered":"2024-09-12T03:21:46.264065+0000","last_unstale":"2024-09-12T03:21:48.724357+0000","last_undegraded":"2024-09-12T03:21:48.724357+0000","last_fullsized":"2024-09-12T03:21:48.724357+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_clean_scrub_stamp":"2024-09-12T03:21:45.231023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:34:03.877327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.267328+0000","last_change":"2024-09-12T03:21:47.283216+0000","last_active":"2024-09-12T03:21:48.267328+0000","last_peered":"2024-09-12T03:21:48.267328+0000","last_clean":"2024-09-12T03:21:48.267328+0000","last_became_active":"2024-09-12T03:21:47.282949+0000","last_became_peered":"2024-09-12T03:21:47.282949+0000","last_unstale":"2024-09-12T03:21:48.267328+0000","last_undegraded":"2024-09-12T03:21:48.267328+0000","last_fullsized":"2024-09-12T03:21:48.267328+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:43:43.310362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":14,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.273529+0000","last_change":"2024-09-12T03:21:46.260679+0000","last_active":"2024-09-12T03:21:48.273529+0000","last_peered":"2024-09-12T03:21:48.273529+0000","last_clean":"2024-09-12T03:21:48.273529+0000","last_became_active":"2024-09-12T03:21:46.260354+0000","last_became_peered":"2024-09-12T03:21:46.260354+0000","last_unstale":"2024-09-12T03:21:48.273529+0000","last_undegraded":"2024-09-12T03:21:48.273529+0000","last_fullsized":"2024-09-12T03:21:48.273529+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_clean_scrub_stamp":"2024-09-12T03:21:45.231023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:24:37.527133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.725752+0000","last_change":"2024-09-12T03:21:47.270434+0000","last_active":"2024-09-12T03:21:48.725752+0000","last_peered":"2024-09-12T03:21:48.725752+0000","last_clean":"2024-09-12T03:21:48.725752+0000","last_became_active":"2024-09-12T03:21:47.270141+0000","last_became_peered":"2024-09-12T03:21:47.270141+0000","last_unstale":"2024-09-12T03:21:48.725752+0000","last_undegraded":"2024-09-12T03:21:48.725752+0000","last_fullsized":"2024-09-12T03:21:48.725752+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:00:08.281678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":14,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.725446+0000","last_change":"2024-09-12T03:21:46.256501+0000","last_active":"2024-09-12T03:21:48.725446+0000","last_peered":"2024-09-12T03:21:48.725446+0000","last_clean":"2024-09-12T03:21:48.725446+0000","last_became_active":"2024-09-12T03:21:46.256069+0000","last_became_peered":"2024-09-12T03:21:46.256069+0000","last_unstale":"2024-09-12T03:21:48.725446+0000","last_undegraded":"2024-09-12T03:21:48.725446+0000","last_fullsized":"2024-09-12T03:21:48.725446+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_clean_scrub_stamp":"2024-09-12T03:21:45.231023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:24:37.527133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.725055+0000","last_change":"2024-09-12T03:21:47.269440+0000","last_active":"2024-09-12T03:21:48.725055+0000","last_peered":"2024-09-12T03:21:48.725055+0000","last_clean":"2024-09-12T03:21:48.725055+0000","last_became_active":"2024-09-12T03:21:47.269145+0000","last_became_peered":"2024-09-12T03:21:47.269145+0000","last_unstale":"2024-09-12T03:21:48.725055+0000","last_undegraded":"2024-09-12T03:21:48.725055+0000","last_fullsized":"2024-09-12T03:21:48.725055+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:42:51.860983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":14,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.725107+0000","last_change":"2024-09-12T03:21:46.263800+0000","last_active":"2024-09-12T03:21:48.725107+0000","last_peered":"2024-09-12T03:21:48.725107+0000","last_clean":"2024-09-12T03:21:48.725107+0000","last_became_active":"2024-09-12T03:21:46.263423+0000","last_became_peered":"2024-09-12T03:21:46.263423+0000","last_unstale":"2024-09-12T03:21:48.725107+0000","last_undegraded":"2024-09-12T03:21:48.725107+0000","last_fullsized":"2024-09-12T03:21:48.725107+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_clean_scrub_stamp":"2024-09-12T03:21:45.231023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:06:20.324237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.273458+0000","last_change":"2024-09-12T03:21:47.281952+0000","last_active":"2024-09-12T03:21:48.273458+0000","last_peered":"2024-09-12T03:21:48.273458+0000","last_clean":"2024-09-12T03:21:48.273458+0000","last_became_active":"2024-09-12T03:21:47.279248+0000","last_became_peered":"2024-09-12T03:21:47.279248+0000","last_unstale":"2024-09-12T03:21:48.273458+0000","last_undegraded":"2024-09-12T03:21:48.273458+0000","last_fullsized":"2024-09-12T03:21:48.273458+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:08:22.543376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"113'2","reported_seq":16,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.273663+0000","last_change":"2024-09-12T03:21:46.265296+0000","last_active":"2024-09-12T03:21:48.273663+0000","last_peered":"2024-09-12T03:21:48.273663+0000","last_clean":"2024-09-12T03:21:48.273663+0000","last_became_active":"2024-09-12T03:21:46.264982+0000","last_became_peered":"2024-09-12T03:21:46.264982+0000","last_unstale":"2024-09-12T03:21:48.273663+0000","last_undegraded":"2024-09-12T03:21:48.273663+0000","last_fullsized":"2024-09-12T03:21:48.273663+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_clean_scrub_stamp":"2024-09-12T03:21:45.231023+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:33:45.204480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"70'128","reported_seq":381,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.273224+0000","last_change":"2024-09-12T03:16:21.883446+0000","last_active":"2024-09-12T03:21:48.273224+0000","last_peered":"2024-09-12T03:21:48.273224+0000","last_clean":"2024-09-12T03:21:48.273224+0000","last_became_active":"2024-09-12T03:16:21.882207+0000","last_became_peered":"2024-09-12T03:16:21.882207+0000","last_unstale":"2024-09-12T03:21:48.273224+0000","last_undegraded":"2024-09-12T03:21:48.273224+0000","last_fullsized":"2024-09-12T03:21:48.273224+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:20.316426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:20.316426+0000","last_clean_scrub_stamp":"2024-09-12T03:16:20.316426+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:12:15.424320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":19,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542212,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37348,"kb_used_data":1992,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334492,"statfs":{"total":96636764160,"available":96598519808,"internally_reserved":0,"allocated":2039808,"data_stored":1219971,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542214,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36748,"kb_used_data":1400,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335092,"statfs":{"total":96636764160,"available":96599134208,"internally_reserved":0,"allocated":1433600,"data_stored":628965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542213,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36748,"kb_used_data":1396,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335092,"statfs":{"total":96636764160,"available":96599134208,"internally_reserved":0,"allocated":1429504,"data_stored":627465,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542212,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36740,"kb_used_data":1392,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335100,"statfs":{"total":96636764160,"available":96599142400,"internally_reserved":0,"allocated":1425408,"data_stored":627423,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542213,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32584,"kb_used_data":1396,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339256,"statfs":{"total":96636764160,"available":96603398144,"internally_reserved":0,"allocated":1429504,"data_stored":627513,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542213,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32592,"kb_used_data":1396,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339248,"statfs":{"total":96636764160,"available":96603389952,"internally_reserved":0,"allocated":1429504,"data_stored":627969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542214,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32580,"kb_used_data":1388,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339260,"statfs":{"total":96636764160,"available":96603402240,"internally_reserved":0,"allocated":1421312,"data_stored":627353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542214,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33156,"kb_used_data":1968,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338684,"statfs":{"total":96636764160,"available":96602812416,"internally_reserved":0,"allocated":2015232,"data_stored":1217721,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T03:21:51.300 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-12T03:21:51.437 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.437+0000 7f83c396d640 1 Processor -- start 2024-09-12T03:21:51.437 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.437+0000 7f83c396d640 1 -- start start 2024-09-12T03:21:51.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.438+0000 7f83c396d640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f83bc0b1ca0 0x7f83bc15f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:51.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.438+0000 7f83c396d640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f83bc06f360 0x7f83bc0cef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:51.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.438+0000 7f83c396d640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f83bc0cf490 0x7f83bc0d2780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:51.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.438+0000 7f83c396d640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f83bc0d22f0 con 0x7f83bc0b1ca0 2024-09-12T03:21:51.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.438+0000 7f83c396d640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f83bc15fd70 con 0x7f83bc06f360 2024-09-12T03:21:51.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.438+0000 7f83c396d640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f83bc06cc40 con 0x7f83bc0cf490 2024-09-12T03:21:51.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.438+0000 7f83c1709640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f83bc0b1ca0 0x7f83bc15f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:51.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.438+0000 7f83c1709640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f83bc0b1ca0 0x7f83bc15f830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39516/0 (socket says 172.21.15.27:39516) 2024-09-12T03:21:51.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.438+0000 7f83c1709640 1 -- 172.21.15.27:0/3564473544 learned_addr learned my addr 172.21.15.27:0/3564473544 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:21:51.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.438+0000 7f83c0f08640 1 --2- 172.21.15.27:0/3564473544 >> v2:172.21.15.45:3300/0 conn(0x7f83bc06f360 0x7f83bc0cef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:51.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.438+0000 7f83c1f0a640 1 --2- 172.21.15.27:0/3564473544 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f83bc0cf490 0x7f83bc0d2780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:51.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.438+0000 7f83c1709640 1 -- 172.21.15.27:0/3564473544 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f83bc0cf490 msgr2=0x7f83bc0d2780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:51.446 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.438+0000 7f83c1709640 1 --2- 172.21.15.27:0/3564473544 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f83bc0cf490 0x7f83bc0d2780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:51.446 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.438+0000 7f83c1709640 1 -- 172.21.15.27:0/3564473544 >> v2:172.21.15.45:3300/0 conn(0x7f83bc06f360 msgr2=0x7f83bc0cef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:51.446 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.438+0000 7f83c1709640 1 --2- 172.21.15.27:0/3564473544 >> v2:172.21.15.45:3300/0 conn(0x7f83bc06f360 0x7f83bc0cef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:51.446 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.438+0000 7f83c1709640 1 -- 172.21.15.27:0/3564473544 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83bc0d1ce0 con 0x7f83bc0b1ca0 2024-09-12T03:21:51.446 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.438+0000 7f83c1f0a640 1 --2- 172.21.15.27:0/3564473544 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f83bc0cf490 0x7f83bc0d2780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:21:51.446 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.439+0000 7f83c1709640 1 --2- 172.21.15.27:0/3564473544 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f83bc0b1ca0 0x7f83bc15f830 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f83bc0b5220 tx=0x7f83b802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2e1f495557bf865 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:51.446 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.439+0000 7f83b3fff640 1 -- 172.21.15.27:0/3564473544 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83b8037070 con 0x7f83bc0b1ca0 2024-09-12T03:21:51.446 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.439+0000 7f83b3fff640 1 -- 172.21.15.27:0/3564473544 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83b8032440 con 0x7f83bc0b1ca0 2024-09-12T03:21:51.446 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.439+0000 7f83b3fff640 1 -- 172.21.15.27:0/3564473544 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83b8032810 con 0x7f83bc0b1ca0 2024-09-12T03:21:51.446 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.440+0000 7f83c396d640 1 -- 172.21.15.27:0/3564473544 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f83bc0b1ca0 msgr2=0x7f83bc15f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:51.447 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.440+0000 7f83c396d640 1 --2- 172.21.15.27:0/3564473544 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f83bc0b1ca0 0x7f83bc15f830 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f83bc0b5220 tx=0x7f83b802a1e0 comp rx=0 tx=0).stop 2024-09-12T03:21:51.447 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.440+0000 7f83c396d640 1 -- 172.21.15.27:0/3564473544 shutdown_connections 2024-09-12T03:21:51.447 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.440+0000 7f83c396d640 1 --2- 172.21.15.27:0/3564473544 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f83bc0cf490 0x7f83bc0d2780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:51.447 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.440+0000 7f83c396d640 1 --2- 172.21.15.27:0/3564473544 >> v2:172.21.15.45:3300/0 conn(0x7f83bc06f360 0x7f83bc0cef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:51.447 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.440+0000 7f83c396d640 1 --2- 172.21.15.27:0/3564473544 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f83bc0b1ca0 0x7f83bc15f830 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:51.447 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.440+0000 7f83c396d640 1 -- 172.21.15.27:0/3564473544 >> 172.21.15.27:0/3564473544 conn(0x7f83bc073800 msgr2=0x7f83bc09ba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:51.447 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.440+0000 7f83c396d640 1 -- 172.21.15.27:0/3564473544 shutdown_connections 2024-09-12T03:21:51.447 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.440+0000 7f83c396d640 1 -- 172.21.15.27:0/3564473544 wait complete. 2024-09-12T03:21:51.447 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.441+0000 7f83c396d640 1 Processor -- start 2024-09-12T03:21:51.447 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.441+0000 7f83c396d640 1 -- start start 2024-09-12T03:21:51.447 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.442+0000 7f83c396d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f83bc06f360 0x7f83bc1f7c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:51.447 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.442+0000 7f83c396d640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f83bc0b1ca0 0x7f83bc070770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:51.447 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.442+0000 7f83c396d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f83bc0cf490 0x7f83bc153b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:51.447 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.442+0000 7f83c396d640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f83bc06d1c0 con 0x7f83bc06f360 2024-09-12T03:21:51.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.442+0000 7f83c396d640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f83bc0d1f80 con 0x7f83bc0cf490 2024-09-12T03:21:51.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.442+0000 7f83c396d640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f83bc06d340 con 0x7f83bc0b1ca0 2024-09-12T03:21:51.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.442+0000 7f83c1709640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f83bc06f360 0x7f83bc1f7c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:51.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.442+0000 7f83c1709640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f83bc06f360 0x7f83bc1f7c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39518/0 (socket says 172.21.15.27:39518) 2024-09-12T03:21:51.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.442+0000 7f83c1709640 1 -- 172.21.15.27:0/865384329 learned_addr learned my addr 172.21.15.27:0/865384329 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:21:51.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.442+0000 7f83c1709640 1 -- 172.21.15.27:0/865384329 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f83bc0b1ca0 msgr2=0x7f83bc070770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:51.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.442+0000 7f83c1709640 1 --2- 172.21.15.27:0/865384329 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f83bc0b1ca0 0x7f83bc070770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:51.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.442+0000 7f83c1709640 1 -- 172.21.15.27:0/865384329 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f83bc0cf490 msgr2=0x7f83bc153b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:51.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.442+0000 7f83c1709640 1 --2- 172.21.15.27:0/865384329 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f83bc0cf490 0x7f83bc153b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:51.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.442+0000 7f83c1709640 1 -- 172.21.15.27:0/865384329 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83bc0cb850 con 0x7f83bc06f360 2024-09-12T03:21:51.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.442+0000 7f83c1709640 1 --2- 172.21.15.27:0/865384329 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f83bc06f360 0x7f83bc1f7c20 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f83b8000c00 tx=0x7f83b80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:51.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.443+0000 7f83b1ffb640 1 -- 172.21.15.27:0/865384329 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83b8037040 con 0x7f83bc06f360 2024-09-12T03:21:51.449 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.443+0000 7f83b1ffb640 1 -- 172.21.15.27:0/865384329 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83b8042070 con 0x7f83bc06f360 2024-09-12T03:21:51.449 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.443+0000 7f83c396d640 1 -- 172.21.15.27:0/865384329 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83bc1f8450 con 0x7f83bc06f360 2024-09-12T03:21:51.449 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.443+0000 7f83b1ffb640 1 -- 172.21.15.27:0/865384329 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83b8030880 con 0x7f83bc06f360 2024-09-12T03:21:51.449 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.443+0000 7f83c396d640 1 -- 172.21.15.27:0/865384329 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83bc1f8a10 con 0x7f83bc06f360 2024-09-12T03:21:51.449 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.444+0000 7f83c396d640 1 -- 172.21.15.27:0/865384329 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83bc0cfc40 con 0x7f83bc06f360 2024-09-12T03:21:51.449 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.444+0000 7f83b1ffb640 1 -- 172.21.15.27:0/865384329 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 19) ==== 90855+0+0 (secure 0 0 0) 0x7f83b8030a20 con 0x7f83bc06f360 2024-09-12T03:21:51.449 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.445+0000 7f83b1ffb640 1 --2- 172.21.15.27:0/865384329 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f83a006c220 0x7f83a006e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:51.449 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.445+0000 7f83b1ffb640 1 -- 172.21.15.27:0/865384329 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5737+0+0 (secure 0 0 0) 0x7f83b80abcc0 con 0x7f83bc06f360 2024-09-12T03:21:51.449 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.445+0000 7f83c0f08640 1 --2- 172.21.15.27:0/865384329 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f83a006c220 0x7f83a006e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:51.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.445+0000 7f83c0f08640 1 --2- 172.21.15.27:0/865384329 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f83a006c220 0x7f83a006e6e0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f83bc06eb00 tx=0x7f83ac009500 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:51.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.449+0000 7f83b1ffb640 1 -- 172.21.15.27:0/865384329 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f83b80b2080 con 0x7f83bc06f360 2024-09-12T03:21:51.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.688+0000 7f83c396d640 1 -- 172.21.15.27:0/865384329 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f83bc0758e0 con 0x7f83bc06f360 2024-09-12T03:21:51.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.689+0000 7f83b1ffb640 1 -- 172.21.15.27:0/865384329 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1605 (secure 0 0 0) 0x7f83b807a210 con 0x7f83bc06f360 2024-09-12T03:21:51.689 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:21:51.689 INFO:teuthology.orchestra.run.smithi027.stdout:{"fsid":"d2df504e-2647-4385-a5db-9fdf69f487a0","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":343,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":114,"num_osds":8,"num_up_osds":8,"osd_up_since":1726110979,"num_in_osds":8,"osd_in_since":1726110969,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":24,"data_bytes":592778,"bytes_used":285179904,"bytes_avail":772808933376,"bytes_total":773094113280,"write_bytes_sec":2438,"read_op_per_sec":0,"write_op_per_sec":7},"fsmap":{"epoch":67,"btime":"2024-09-12T03:21:49:612947+0000","id":9,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":9,"rank":0,"name":"a","status":"up:active","gid":7302}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":13,"modified":"2024-09-12T03:21:21.107492+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-12T03:21:51.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.691+0000 7f83c396d640 1 -- 172.21.15.27:0/865384329 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f83a006c220 msgr2=0x7f83a006e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:51.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.691+0000 7f83c396d640 1 --2- 172.21.15.27:0/865384329 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f83a006c220 0x7f83a006e6e0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f83bc06eb00 tx=0x7f83ac009500 comp rx=0 tx=0).stop 2024-09-12T03:21:51.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.691+0000 7f83c396d640 1 -- 172.21.15.27:0/865384329 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f83bc06f360 msgr2=0x7f83bc1f7c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:51.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.691+0000 7f83c396d640 1 --2- 172.21.15.27:0/865384329 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f83bc06f360 0x7f83bc1f7c20 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f83b8000c00 tx=0x7f83b80046c0 comp rx=0 tx=0).stop 2024-09-12T03:21:51.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.691+0000 7f83c396d640 1 -- 172.21.15.27:0/865384329 shutdown_connections 2024-09-12T03:21:51.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.691+0000 7f83c396d640 1 --2- 172.21.15.27:0/865384329 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f83bc0cf490 0x7f83bc153b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:51.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.691+0000 7f83c396d640 1 --2- 172.21.15.27:0/865384329 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f83bc0b1ca0 0x7f83bc070770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:51.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.691+0000 7f83c396d640 1 --2- 172.21.15.27:0/865384329 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f83a006c220 0x7f83a006e6e0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:51.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.691+0000 7f83c396d640 1 --2- 172.21.15.27:0/865384329 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f83bc06f360 0x7f83bc1f7c20 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:51.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.691+0000 7f83c396d640 1 -- 172.21.15.27:0/865384329 >> 172.21.15.27:0/865384329 conn(0x7f83bc073800 msgr2=0x7f83bc0b7090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:51.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.691+0000 7f83c396d640 1 -- 172.21.15.27:0/865384329 shutdown_connections 2024-09-12T03:21:51.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.691+0000 7f83c396d640 1 -- 172.21.15.27:0/865384329 wait complete. 2024-09-12T03:21:51.702 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-09-12T03:21:51.702 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-12T03:21:51.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.805+0000 7f6c6f7fd640 1 Processor -- start 2024-09-12T03:21:51.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.805+0000 7f6c6f7fd640 1 -- start start 2024-09-12T03:21:51.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.806+0000 7f6c6f7fd640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f6c680cf660 0x7f6c680d2780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:51.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.806+0000 7f6c6f7fd640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6c6806cc40 con 0x7f6c680b1ca0 2024-09-12T03:21:51.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.806+0000 7f6c6f7fd640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6c680d1ce0 con 0x7f6c6806f360 2024-09-12T03:21:51.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.806+0000 7f6c6f7fd640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f6c680d22f0 con 0x7f6c680cf660 2024-09-12T03:21:51.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.806+0000 7f6c6d599640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f6c680b1ca0 0x7f6c6815f830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:56350/0 (socket says 172.21.15.27:56350) 2024-09-12T03:21:51.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.806+0000 7f6c6d599640 1 -- 172.21.15.27:0/3800505871 learned_addr learned my addr 172.21.15.27:0/3800505871 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:21:51.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.806+0000 7f6c6dd9a640 1 --2- 172.21.15.27:0/3800505871 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f6c680cf660 0x7f6c680d2780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:51.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.806+0000 7f6c5ffff640 1 -- 172.21.15.27:0/3800505871 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2310743074 0 0) 0x7f6c6806cc40 con 0x7f6c680b1ca0 2024-09-12T03:21:51.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.806+0000 7f6c5ffff640 1 -- 172.21.15.27:0/3800505871 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6c54003520 con 0x7f6c680b1ca0 2024-09-12T03:21:51.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.806+0000 7f6c5ffff640 1 -- 172.21.15.27:0/3800505871 <== mon.1 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3864013873 0 0) 0x7f6c680d1ce0 con 0x7f6c6806f360 2024-09-12T03:21:51.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.806+0000 7f6c5ffff640 1 -- 172.21.15.27:0/3800505871 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6c6806cc40 con 0x7f6c6806f360 2024-09-12T03:21:51.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.806+0000 7f6c6dd9a640 1 -- 172.21.15.27:0/3800505871 >> v1:172.21.15.45:6789/0 conn(0x7f6c6806f360 legacy=0x7f6c680cef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:51.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.806+0000 7f6c6dd9a640 1 -- 172.21.15.27:0/3800505871 >> v1:172.21.15.27:6789/0 conn(0x7f6c680b1ca0 legacy=0x7f6c6815f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:51.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.806+0000 7f6c6dd9a640 1 -- 172.21.15.27:0/3800505871 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c6806cee0 con 0x7f6c680cf660 2024-09-12T03:21:51.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.807+0000 7f6c6dd9a640 1 --2- 172.21.15.27:0/3800505871 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f6c680cf660 0x7f6c680d2780 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f6c60009dc0 tx=0x7f6c6002a6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fb004904b3767e0c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:51.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.807+0000 7f6c5ffff640 1 -- 172.21.15.27:0/3800505871 <== mon.2 v2:172.21.15.45:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c60007740 con 0x7f6c680cf660 2024-09-12T03:21:51.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.807+0000 7f6c5ffff640 1 -- 172.21.15.27:0/3800505871 <== mon.2 v2:172.21.15.45:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c600078e0 con 0x7f6c680cf660 2024-09-12T03:21:51.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.808+0000 7f6c5ffff640 1 -- 172.21.15.27:0/3800505871 <== mon.2 v2:172.21.15.45:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c60007c90 con 0x7f6c680cf660 2024-09-12T03:21:51.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.808+0000 7f6c6f7fd640 1 -- 172.21.15.27:0/3800505871 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f6c680cf660 msgr2=0x7f6c680d2780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:51.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.808+0000 7f6c6f7fd640 1 --2- 172.21.15.27:0/3800505871 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f6c680cf660 0x7f6c680d2780 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f6c60009dc0 tx=0x7f6c6002a6d0 comp rx=0 tx=0).stop 2024-09-12T03:21:51.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.808+0000 7f6c6f7fd640 1 -- 172.21.15.27:0/3800505871 shutdown_connections 2024-09-12T03:21:51.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.808+0000 7f6c6f7fd640 1 --2- 172.21.15.27:0/3800505871 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f6c680cf660 0x7f6c680d2780 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:51.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.808+0000 7f6c6f7fd640 1 -- 172.21.15.27:0/3800505871 >> 172.21.15.27:0/3800505871 conn(0x7f6c68073800 msgr2=0x7f6c6809ba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:51.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.808+0000 7f6c6f7fd640 1 -- 172.21.15.27:0/3800505871 shutdown_connections 2024-09-12T03:21:51.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.808+0000 7f6c6f7fd640 1 -- 172.21.15.27:0/3800505871 wait complete. 2024-09-12T03:21:51.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.809+0000 7f6c6f7fd640 1 Processor -- start 2024-09-12T03:21:51.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.809+0000 7f6c6f7fd640 1 -- start start 2024-09-12T03:21:51.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.810+0000 7f6c6f7fd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c6806f360 0x7f6c6815a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:51.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.810+0000 7f6c6f7fd640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f6c680b1ca0 0x7f6c680b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:51.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.810+0000 7f6c6f7fd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6c680cf660 0x7f6c68156c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:51.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.810+0000 7f6c6f7fd640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f6c6806d340 con 0x7f6c6806f360 2024-09-12T03:21:51.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.810+0000 7f6c6f7fd640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f6c6806d1c0 con 0x7f6c680cf660 2024-09-12T03:21:51.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.810+0000 7f6c6f7fd640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f6c680cfc40 con 0x7f6c680b1ca0 2024-09-12T03:21:51.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.810+0000 7f6c6d599640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c6806f360 0x7f6c6815a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:51.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.810+0000 7f6c6d599640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c6806f360 0x7f6c6815a9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39522/0 (socket says 172.21.15.27:39522) 2024-09-12T03:21:51.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.810+0000 7f6c6d599640 1 -- 172.21.15.27:0/297746676 learned_addr learned my addr 172.21.15.27:0/297746676 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:21:51.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.810+0000 7f6c6dd9a640 1 --2- 172.21.15.27:0/297746676 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6c680cf660 0x7f6c68156c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:51.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.810+0000 7f6c6cd98640 1 --2- 172.21.15.27:0/297746676 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f6c680b1ca0 0x7f6c680b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:51.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.811+0000 7f6c6d599640 1 -- 172.21.15.27:0/297746676 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f6c680b1ca0 msgr2=0x7f6c680b2080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:51.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.811+0000 7f6c6d599640 1 --2- 172.21.15.27:0/297746676 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f6c680b1ca0 0x7f6c680b2080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:51.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.811+0000 7f6c6d599640 1 -- 172.21.15.27:0/297746676 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6c680cf660 msgr2=0x7f6c68156c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:51.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.811+0000 7f6c6d599640 1 --2- 172.21.15.27:0/297746676 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6c680cf660 0x7f6c68156c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:51.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.811+0000 7f6c6d599640 1 -- 172.21.15.27:0/297746676 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c68158f50 con 0x7f6c6806f360 2024-09-12T03:21:51.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.811+0000 7f6c6d599640 1 --2- 172.21.15.27:0/297746676 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c6806f360 0x7f6c6815a9e0 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f6c6400c500 tx=0x7f6c6400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:51.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.811+0000 7f6c5dffb640 1 -- 172.21.15.27:0/297746676 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c64017070 con 0x7f6c6806f360 2024-09-12T03:21:51.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.811+0000 7f6c5dffb640 1 -- 172.21.15.27:0/297746676 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c64005ce0 con 0x7f6c6806f360 2024-09-12T03:21:51.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.811+0000 7f6c5dffb640 1 -- 172.21.15.27:0/297746676 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c640072f0 con 0x7f6c6806f360 2024-09-12T03:21:51.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.811+0000 7f6c6f7fd640 1 -- 172.21.15.27:0/297746676 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c68157eb0 con 0x7f6c6806f360 2024-09-12T03:21:51.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.811+0000 7f6c6f7fd640 1 -- 172.21.15.27:0/297746676 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c68158470 con 0x7f6c6806f360 2024-09-12T03:21:51.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.812+0000 7f6c6f7fd640 1 -- 172.21.15.27:0/297746676 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c6806f7d0 con 0x7f6c6806f360 2024-09-12T03:21:51.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.812+0000 7f6c5dffb640 1 -- 172.21.15.27:0/297746676 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 19) ==== 90855+0+0 (secure 0 0 0) 0x7f6c6401a070 con 0x7f6c6806f360 2024-09-12T03:21:51.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.813+0000 7f6c5dffb640 1 --2- 172.21.15.27:0/297746676 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f6c4c06c220 0x7f6c4c06e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:51.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.813+0000 7f6c5dffb640 1 -- 172.21.15.27:0/297746676 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5737+0+0 (secure 0 0 0) 0x7f6c6408a8b0 con 0x7f6c6806f360 2024-09-12T03:21:51.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.813+0000 7f6c6cd98640 1 --2- 172.21.15.27:0/297746676 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f6c4c06c220 0x7f6c4c06e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:51.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.813+0000 7f6c6cd98640 1 --2- 172.21.15.27:0/297746676 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f6c4c06c220 0x7f6c4c06e6e0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f6c58007fd0 tx=0x7f6c58009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:51.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.817+0000 7f6c5dffb640 1 -- 172.21.15.27:0/297746676 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f6c640586a0 con 0x7f6c6806f360 2024-09-12T03:21:51.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.987+0000 7f6c6f7fd640 1 -- 172.21.15.27:0/297746676 --> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6c68158810 con 0x7f6c4c06c220 2024-09-12T03:21:51.993 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.993+0000 7f6c5dffb640 1 -- 172.21.15.27:0/297746676 <== mgr.4113 v2:172.21.15.27:6836/1226920526 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252482 (secure 0 0 0) 0x7f6c68158810 con 0x7f6c4c06c220 2024-09-12T03:21:51.996 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:21:51.997 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2024-09-12T03:21:51.998 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.997+0000 7f6c6f7fd640 1 -- 172.21.15.27:0/297746676 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f6c4c06c220 msgr2=0x7f6c4c06e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:51.998 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.997+0000 7f6c6f7fd640 1 --2- 172.21.15.27:0/297746676 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f6c4c06c220 0x7f6c4c06e6e0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f6c58007fd0 tx=0x7f6c58009040 comp rx=0 tx=0).stop 2024-09-12T03:21:51.999 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.997+0000 7f6c6f7fd640 1 -- 172.21.15.27:0/297746676 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c6806f360 msgr2=0x7f6c6815a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:51.999 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.997+0000 7f6c6f7fd640 1 --2- 172.21.15.27:0/297746676 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c6806f360 0x7f6c6815a9e0 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f6c6400c500 tx=0x7f6c6400c9c0 comp rx=0 tx=0).stop 2024-09-12T03:21:51.999 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.997+0000 7f6c6f7fd640 1 -- 172.21.15.27:0/297746676 shutdown_connections 2024-09-12T03:21:51.999 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.997+0000 7f6c6f7fd640 1 --2- 172.21.15.27:0/297746676 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6c680cf660 0x7f6c68156c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:51.999 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.997+0000 7f6c6f7fd640 1 --2- 172.21.15.27:0/297746676 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f6c680b1ca0 0x7f6c680b2080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:51.999 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.997+0000 7f6c6f7fd640 1 --2- 172.21.15.27:0/297746676 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f6c4c06c220 0x7f6c4c06e6e0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:51.999 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.997+0000 7f6c6f7fd640 1 --2- 172.21.15.27:0/297746676 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c6806f360 0x7f6c6815a9e0 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:51.999 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.997+0000 7f6c6f7fd640 1 -- 172.21.15.27:0/297746676 >> 172.21.15.27:0/297746676 conn(0x7f6c68073800 msgr2=0x7f6c6815d990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:51.999 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.997+0000 7f6c6f7fd640 1 -- 172.21.15.27:0/297746676 shutdown_connections 2024-09-12T03:21:51.999 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:51.997+0000 7f6c6f7fd640 1 -- 172.21.15.27:0/297746676 wait complete. 2024-09-12T03:21:52.008 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":96,"stamp":"2024-09-12T03:21:51.115694+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":224,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":167,"ondisk_log_size":167,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":278496,"kb_used_data":12328,"kb_used_omap":12,"kb_used_meta":265971,"kb_avail":754696224,"statfs":{"total":773094113280,"available":772808933376,"internally_reserved":0,"allocated":12623872,"data_stored":6204380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":272354864},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":22,"apply_latency_ms":22,"commit_latency_ns":22000000,"apply_latency_ns":22000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.878018"},"pg_stats":[{"pgid":"19.3f","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.266518+0000","last_change":"2024-09-12T03:21:47.276464+0000","last_active":"2024-09-12T03:21:48.266518+0000","last_peered":"2024-09-12T03:21:48.266518+0000","last_clean":"2024-09-12T03:21:48.266518+0000","last_became_active":"2024-09-12T03:21:47.276166+0000","last_became_peered":"2024-09-12T03:21:47.276166+0000","last_unstale":"2024-09-12T03:21:48.266518+0000","last_undegraded":"2024-09-12T03:21:48.266518+0000","last_fullsized":"2024-09-12T03:21:48.266518+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:28:18.796133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3e","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.724634+0000","last_change":"2024-09-12T03:21:47.279512+0000","last_active":"2024-09-12T03:21:48.724634+0000","last_peered":"2024-09-12T03:21:48.724634+0000","last_clean":"2024-09-12T03:21:48.724634+0000","last_became_active":"2024-09-12T03:21:47.279237+0000","last_became_peered":"2024-09-12T03:21:47.279237+0000","last_unstale":"2024-09-12T03:21:48.724634+0000","last_undegraded":"2024-09-12T03:21:48.724634+0000","last_fullsized":"2024-09-12T03:21:48.724634+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:21:31.235603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3d","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.598938+0000","last_change":"2024-09-12T03:21:47.276803+0000","last_active":"2024-09-12T03:21:48.598938+0000","last_peered":"2024-09-12T03:21:48.598938+0000","last_clean":"2024-09-12T03:21:48.598938+0000","last_became_active":"2024-09-12T03:21:47.276525+0000","last_became_peered":"2024-09-12T03:21:47.276525+0000","last_unstale":"2024-09-12T03:21:48.598938+0000","last_undegraded":"2024-09-12T03:21:48.598938+0000","last_fullsized":"2024-09-12T03:21:48.598938+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:27:48.066859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.3c","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.266405+0000","last_change":"2024-09-12T03:21:47.277021+0000","last_active":"2024-09-12T03:21:48.266405+0000","last_peered":"2024-09-12T03:21:48.266405+0000","last_clean":"2024-09-12T03:21:48.266405+0000","last_became_active":"2024-09-12T03:21:47.275542+0000","last_became_peered":"2024-09-12T03:21:47.275542+0000","last_unstale":"2024-09-12T03:21:48.266405+0000","last_undegraded":"2024-09-12T03:21:48.266405+0000","last_fullsized":"2024-09-12T03:21:48.266405+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:40:16.335013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3b","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.273675+0000","last_change":"2024-09-12T03:21:47.274034+0000","last_active":"2024-09-12T03:21:48.273675+0000","last_peered":"2024-09-12T03:21:48.273675+0000","last_clean":"2024-09-12T03:21:48.273675+0000","last_became_active":"2024-09-12T03:21:47.273718+0000","last_became_peered":"2024-09-12T03:21:47.273718+0000","last_unstale":"2024-09-12T03:21:48.273675+0000","last_undegraded":"2024-09-12T03:21:48.273675+0000","last_fullsized":"2024-09-12T03:21:48.273675+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:07:27.386745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.3a","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.267033+0000","last_change":"2024-09-12T03:21:47.279653+0000","last_active":"2024-09-12T03:21:48.267033+0000","last_peered":"2024-09-12T03:21:48.267033+0000","last_clean":"2024-09-12T03:21:48.267033+0000","last_became_active":"2024-09-12T03:21:47.279340+0000","last_became_peered":"2024-09-12T03:21:47.279340+0000","last_unstale":"2024-09-12T03:21:48.267033+0000","last_undegraded":"2024-09-12T03:21:48.267033+0000","last_fullsized":"2024-09-12T03:21:48.267033+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:07:12.256007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.39","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.266736+0000","last_change":"2024-09-12T03:21:47.272385+0000","last_active":"2024-09-12T03:21:48.266736+0000","last_peered":"2024-09-12T03:21:48.266736+0000","last_clean":"2024-09-12T03:21:48.266736+0000","last_became_active":"2024-09-12T03:21:47.272056+0000","last_became_peered":"2024-09-12T03:21:47.272056+0000","last_unstale":"2024-09-12T03:21:48.266736+0000","last_undegraded":"2024-09-12T03:21:48.266736+0000","last_fullsized":"2024-09-12T03:21:48.266736+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:14:24.567859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.38","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.598920+0000","last_change":"2024-09-12T03:21:47.280231+0000","last_active":"2024-09-12T03:21:48.598920+0000","last_peered":"2024-09-12T03:21:48.598920+0000","last_clean":"2024-09-12T03:21:48.598920+0000","last_became_active":"2024-09-12T03:21:47.279961+0000","last_became_peered":"2024-09-12T03:21:47.279961+0000","last_unstale":"2024-09-12T03:21:48.598920+0000","last_undegraded":"2024-09-12T03:21:48.598920+0000","last_fullsized":"2024-09-12T03:21:48.598920+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:35:57.506654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.37","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.599097+0000","last_change":"2024-09-12T03:21:47.274676+0000","last_active":"2024-09-12T03:21:48.599097+0000","last_peered":"2024-09-12T03:21:48.599097+0000","last_clean":"2024-09-12T03:21:48.599097+0000","last_became_active":"2024-09-12T03:21:47.274394+0000","last_became_peered":"2024-09-12T03:21:47.274394+0000","last_unstale":"2024-09-12T03:21:48.599097+0000","last_undegraded":"2024-09-12T03:21:48.599097+0000","last_fullsized":"2024-09-12T03:21:48.599097+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:33:46.210575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.36","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.724767+0000","last_change":"2024-09-12T03:21:47.271566+0000","last_active":"2024-09-12T03:21:48.724767+0000","last_peered":"2024-09-12T03:21:48.724767+0000","last_clean":"2024-09-12T03:21:48.724767+0000","last_became_active":"2024-09-12T03:21:47.271260+0000","last_became_peered":"2024-09-12T03:21:47.271260+0000","last_unstale":"2024-09-12T03:21:48.724767+0000","last_undegraded":"2024-09-12T03:21:48.724767+0000","last_fullsized":"2024-09-12T03:21:48.724767+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:39:28.014015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.35","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.725510+0000","last_change":"2024-09-12T03:21:47.286426+0000","last_active":"2024-09-12T03:21:48.725510+0000","last_peered":"2024-09-12T03:21:48.725510+0000","last_clean":"2024-09-12T03:21:48.725510+0000","last_became_active":"2024-09-12T03:21:47.285471+0000","last_became_peered":"2024-09-12T03:21:47.285471+0000","last_unstale":"2024-09-12T03:21:48.725510+0000","last_undegraded":"2024-09-12T03:21:48.725510+0000","last_fullsized":"2024-09-12T03:21:48.725510+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:23:56.306775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.34","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.598790+0000","last_change":"2024-09-12T03:21:47.272716+0000","last_active":"2024-09-12T03:21:48.598790+0000","last_peered":"2024-09-12T03:21:48.598790+0000","last_clean":"2024-09-12T03:21:48.598790+0000","last_became_active":"2024-09-12T03:21:47.269444+0000","last_became_peered":"2024-09-12T03:21:47.269444+0000","last_unstale":"2024-09-12T03:21:48.598790+0000","last_undegraded":"2024-09-12T03:21:48.598790+0000","last_fullsized":"2024-09-12T03:21:48.598790+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:29:24.947639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.33","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.266441+0000","last_change":"2024-09-12T03:21:47.286251+0000","last_active":"2024-09-12T03:21:48.266441+0000","last_peered":"2024-09-12T03:21:48.266441+0000","last_clean":"2024-09-12T03:21:48.266441+0000","last_became_active":"2024-09-12T03:21:47.285959+0000","last_became_peered":"2024-09-12T03:21:47.285959+0000","last_unstale":"2024-09-12T03:21:48.266441+0000","last_undegraded":"2024-09-12T03:21:48.266441+0000","last_fullsized":"2024-09-12T03:21:48.266441+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:19:41.675227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.32","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.724693+0000","last_change":"2024-09-12T03:21:47.280594+0000","last_active":"2024-09-12T03:21:48.724693+0000","last_peered":"2024-09-12T03:21:48.724693+0000","last_clean":"2024-09-12T03:21:48.724693+0000","last_became_active":"2024-09-12T03:21:47.280250+0000","last_became_peered":"2024-09-12T03:21:47.280250+0000","last_unstale":"2024-09-12T03:21:48.724693+0000","last_undegraded":"2024-09-12T03:21:48.724693+0000","last_fullsized":"2024-09-12T03:21:48.724693+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:43:43.310362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.31","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.266159+0000","last_change":"2024-09-12T03:21:47.278494+0000","last_active":"2024-09-12T03:21:48.266159+0000","last_peered":"2024-09-12T03:21:48.266159+0000","last_clean":"2024-09-12T03:21:48.266159+0000","last_became_active":"2024-09-12T03:21:47.278228+0000","last_became_peered":"2024-09-12T03:21:47.278228+0000","last_unstale":"2024-09-12T03:21:48.266159+0000","last_undegraded":"2024-09-12T03:21:48.266159+0000","last_fullsized":"2024-09-12T03:21:48.266159+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:06:31.812980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.30","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.725675+0000","last_change":"2024-09-12T03:21:47.274919+0000","last_active":"2024-09-12T03:21:48.725675+0000","last_peered":"2024-09-12T03:21:48.725675+0000","last_clean":"2024-09-12T03:21:48.725675+0000","last_became_active":"2024-09-12T03:21:47.274611+0000","last_became_peered":"2024-09-12T03:21:47.274611+0000","last_unstale":"2024-09-12T03:21:48.725675+0000","last_undegraded":"2024-09-12T03:21:48.725675+0000","last_fullsized":"2024-09-12T03:21:48.725675+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:21:08.369709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2f","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.267173+0000","last_change":"2024-09-12T03:21:47.278293+0000","last_active":"2024-09-12T03:21:48.267173+0000","last_peered":"2024-09-12T03:21:48.267173+0000","last_clean":"2024-09-12T03:21:48.267173+0000","last_became_active":"2024-09-12T03:21:47.277990+0000","last_became_peered":"2024-09-12T03:21:47.277990+0000","last_unstale":"2024-09-12T03:21:48.267173+0000","last_undegraded":"2024-09-12T03:21:48.267173+0000","last_fullsized":"2024-09-12T03:21:48.267173+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:32:48.364609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.2e","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.724475+0000","last_change":"2024-09-12T03:21:47.267120+0000","last_active":"2024-09-12T03:21:48.724475+0000","last_peered":"2024-09-12T03:21:48.724475+0000","last_clean":"2024-09-12T03:21:48.724475+0000","last_became_active":"2024-09-12T03:21:47.266838+0000","last_became_peered":"2024-09-12T03:21:47.266838+0000","last_unstale":"2024-09-12T03:21:48.724475+0000","last_undegraded":"2024-09-12T03:21:48.724475+0000","last_fullsized":"2024-09-12T03:21:48.724475+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:20:42.594004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2d","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.725652+0000","last_change":"2024-09-12T03:21:47.281390+0000","last_active":"2024-09-12T03:21:48.725652+0000","last_peered":"2024-09-12T03:21:48.725652+0000","last_clean":"2024-09-12T03:21:48.725652+0000","last_became_active":"2024-09-12T03:21:47.281102+0000","last_became_peered":"2024-09-12T03:21:47.281102+0000","last_unstale":"2024-09-12T03:21:48.725652+0000","last_undegraded":"2024-09-12T03:21:48.725652+0000","last_fullsized":"2024-09-12T03:21:48.725652+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:49:41.579264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2c","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.724874+0000","last_change":"2024-09-12T03:21:47.271524+0000","last_active":"2024-09-12T03:21:48.724874+0000","last_peered":"2024-09-12T03:21:48.724874+0000","last_clean":"2024-09-12T03:21:48.724874+0000","last_became_active":"2024-09-12T03:21:47.271217+0000","last_became_peered":"2024-09-12T03:21:47.271217+0000","last_unstale":"2024-09-12T03:21:48.724874+0000","last_undegraded":"2024-09-12T03:21:48.724874+0000","last_fullsized":"2024-09-12T03:21:48.724874+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:08:22.543376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2b","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.273593+0000","last_change":"2024-09-12T03:21:47.274734+0000","last_active":"2024-09-12T03:21:48.273593+0000","last_peered":"2024-09-12T03:21:48.273593+0000","last_clean":"2024-09-12T03:21:48.273593+0000","last_became_active":"2024-09-12T03:21:47.274409+0000","last_became_peered":"2024-09-12T03:21:47.274409+0000","last_unstale":"2024-09-12T03:21:48.273593+0000","last_undegraded":"2024-09-12T03:21:48.273593+0000","last_fullsized":"2024-09-12T03:21:48.273593+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:00:08.281678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2a","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.724647+0000","last_change":"2024-09-12T03:21:47.285810+0000","last_active":"2024-09-12T03:21:48.724647+0000","last_peered":"2024-09-12T03:21:48.724647+0000","last_clean":"2024-09-12T03:21:48.724647+0000","last_became_active":"2024-09-12T03:21:47.285464+0000","last_became_peered":"2024-09-12T03:21:47.285464+0000","last_unstale":"2024-09-12T03:21:48.724647+0000","last_undegraded":"2024-09-12T03:21:48.724647+0000","last_fullsized":"2024-09-12T03:21:48.724647+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:27:08.761287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.29","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.598960+0000","last_change":"2024-09-12T03:21:47.274207+0000","last_active":"2024-09-12T03:21:48.598960+0000","last_peered":"2024-09-12T03:21:48.598960+0000","last_clean":"2024-09-12T03:21:48.598960+0000","last_became_active":"2024-09-12T03:21:47.273730+0000","last_became_peered":"2024-09-12T03:21:47.273730+0000","last_unstale":"2024-09-12T03:21:48.598960+0000","last_undegraded":"2024-09-12T03:21:48.598960+0000","last_fullsized":"2024-09-12T03:21:48.598960+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:24:38.533228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.28","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.724965+0000","last_change":"2024-09-12T03:21:47.275142+0000","last_active":"2024-09-12T03:21:48.724965+0000","last_peered":"2024-09-12T03:21:48.724965+0000","last_clean":"2024-09-12T03:21:48.724965+0000","last_became_active":"2024-09-12T03:21:47.274110+0000","last_became_peered":"2024-09-12T03:21:47.274110+0000","last_unstale":"2024-09-12T03:21:48.724965+0000","last_undegraded":"2024-09-12T03:21:48.724965+0000","last_fullsized":"2024-09-12T03:21:48.724965+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:37:40.004179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.27","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.724485+0000","last_change":"2024-09-12T03:21:47.269287+0000","last_active":"2024-09-12T03:21:48.724485+0000","last_peered":"2024-09-12T03:21:48.724485+0000","last_clean":"2024-09-12T03:21:48.724485+0000","last_became_active":"2024-09-12T03:21:47.268980+0000","last_became_peered":"2024-09-12T03:21:47.268980+0000","last_unstale":"2024-09-12T03:21:48.724485+0000","last_undegraded":"2024-09-12T03:21:48.724485+0000","last_fullsized":"2024-09-12T03:21:48.724485+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:05:34.381423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.26","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.724624+0000","last_change":"2024-09-12T03:21:47.278973+0000","last_active":"2024-09-12T03:21:48.724624+0000","last_peered":"2024-09-12T03:21:48.724624+0000","last_clean":"2024-09-12T03:21:48.724624+0000","last_became_active":"2024-09-12T03:21:47.278643+0000","last_became_peered":"2024-09-12T03:21:47.278643+0000","last_unstale":"2024-09-12T03:21:48.724624+0000","last_undegraded":"2024-09-12T03:21:48.724624+0000","last_fullsized":"2024-09-12T03:21:48.724624+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:49:07.547396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.25","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.266439+0000","last_change":"2024-09-12T03:21:47.279216+0000","last_active":"2024-09-12T03:21:48.266439+0000","last_peered":"2024-09-12T03:21:48.266439+0000","last_clean":"2024-09-12T03:21:48.266439+0000","last_became_active":"2024-09-12T03:21:47.278935+0000","last_became_peered":"2024-09-12T03:21:47.278935+0000","last_unstale":"2024-09-12T03:21:48.266439+0000","last_undegraded":"2024-09-12T03:21:48.266439+0000","last_fullsized":"2024-09-12T03:21:48.266439+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:57:37.592178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.24","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.267673+0000","last_change":"2024-09-12T03:21:47.278666+0000","last_active":"2024-09-12T03:21:48.267673+0000","last_peered":"2024-09-12T03:21:48.267673+0000","last_clean":"2024-09-12T03:21:48.267673+0000","last_became_active":"2024-09-12T03:21:47.278307+0000","last_became_peered":"2024-09-12T03:21:47.278307+0000","last_unstale":"2024-09-12T03:21:48.267673+0000","last_undegraded":"2024-09-12T03:21:48.267673+0000","last_fullsized":"2024-09-12T03:21:48.267673+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:39:28.014015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.23","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.266509+0000","last_change":"2024-09-12T03:21:47.281506+0000","last_active":"2024-09-12T03:21:48.266509+0000","last_peered":"2024-09-12T03:21:48.266509+0000","last_clean":"2024-09-12T03:21:48.266509+0000","last_became_active":"2024-09-12T03:21:47.281234+0000","last_became_peered":"2024-09-12T03:21:47.281234+0000","last_unstale":"2024-09-12T03:21:48.266509+0000","last_undegraded":"2024-09-12T03:21:48.266509+0000","last_fullsized":"2024-09-12T03:21:48.266509+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:15:33.521405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.22","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.266213+0000","last_change":"2024-09-12T03:21:47.277446+0000","last_active":"2024-09-12T03:21:48.266213+0000","last_peered":"2024-09-12T03:21:48.266213+0000","last_clean":"2024-09-12T03:21:48.266213+0000","last_became_active":"2024-09-12T03:21:47.276925+0000","last_became_peered":"2024-09-12T03:21:47.276925+0000","last_unstale":"2024-09-12T03:21:48.266213+0000","last_undegraded":"2024-09-12T03:21:48.266213+0000","last_fullsized":"2024-09-12T03:21:48.266213+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:29:38.307737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.21","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.273547+0000","last_change":"2024-09-12T03:21:47.283522+0000","last_active":"2024-09-12T03:21:48.273547+0000","last_peered":"2024-09-12T03:21:48.273547+0000","last_clean":"2024-09-12T03:21:48.273547+0000","last_became_active":"2024-09-12T03:21:47.283218+0000","last_became_peered":"2024-09-12T03:21:47.283218+0000","last_unstale":"2024-09-12T03:21:48.273547+0000","last_undegraded":"2024-09-12T03:21:48.273547+0000","last_fullsized":"2024-09-12T03:21:48.273547+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:07:41.072755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.20","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.266742+0000","last_change":"2024-09-12T03:21:47.280790+0000","last_active":"2024-09-12T03:21:48.266742+0000","last_peered":"2024-09-12T03:21:48.266742+0000","last_clean":"2024-09-12T03:21:48.266742+0000","last_became_active":"2024-09-12T03:21:47.280508+0000","last_became_peered":"2024-09-12T03:21:47.280508+0000","last_unstale":"2024-09-12T03:21:48.266742+0000","last_undegraded":"2024-09-12T03:21:48.266742+0000","last_fullsized":"2024-09-12T03:21:48.266742+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:38:27.802587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1e","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.724761+0000","last_change":"2024-09-12T03:21:47.274189+0000","last_active":"2024-09-12T03:21:48.724761+0000","last_peered":"2024-09-12T03:21:48.724761+0000","last_clean":"2024-09-12T03:21:48.724761+0000","last_became_active":"2024-09-12T03:21:47.273852+0000","last_became_peered":"2024-09-12T03:21:47.273852+0000","last_unstale":"2024-09-12T03:21:48.724761+0000","last_undegraded":"2024-09-12T03:21:48.724761+0000","last_fullsized":"2024-09-12T03:21:48.724761+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:42:44.733030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1f","version":"113'4","reported_seq":18,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.273969+0000","last_change":"2024-09-12T03:21:46.265345+0000","last_active":"2024-09-12T03:21:48.273969+0000","last_peered":"2024-09-12T03:21:48.273969+0000","last_clean":"2024-09-12T03:21:48.273969+0000","last_became_active":"2024-09-12T03:21:46.265014+0000","last_became_peered":"2024-09-12T03:21:46.265014+0000","last_unstale":"2024-09-12T03:21:48.273969+0000","last_undegraded":"2024-09-12T03:21:48.273969+0000","last_fullsized":"2024-09-12T03:21:48.273969+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_clean_scrub_stamp":"2024-09-12T03:21:45.231023+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:16:44.125918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.1f","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.599128+0000","last_change":"2024-09-12T03:21:47.278091+0000","last_active":"2024-09-12T03:21:48.599128+0000","last_peered":"2024-09-12T03:21:48.599128+0000","last_clean":"2024-09-12T03:21:48.599128+0000","last_became_active":"2024-09-12T03:21:47.277783+0000","last_became_peered":"2024-09-12T03:21:47.277783+0000","last_unstale":"2024-09-12T03:21:48.599128+0000","last_undegraded":"2024-09-12T03:21:48.599128+0000","last_fullsized":"2024-09-12T03:21:48.599128+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:42:53.625195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1e","version":"113'1","reported_seq":15,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.267990+0000","last_change":"2024-09-12T03:21:46.251686+0000","last_active":"2024-09-12T03:21:48.267990+0000","last_peered":"2024-09-12T03:21:48.267990+0000","last_clean":"2024-09-12T03:21:48.267990+0000","last_became_active":"2024-09-12T03:21:46.251055+0000","last_became_peered":"2024-09-12T03:21:46.251055+0000","last_unstale":"2024-09-12T03:21:48.267990+0000","last_undegraded":"2024-09-12T03:21:48.267990+0000","last_fullsized":"2024-09-12T03:21:48.267990+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_clean_scrub_stamp":"2024-09-12T03:21:45.231023+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:01:01.329193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.273734+0000","last_change":"2024-09-12T03:21:47.275088+0000","last_active":"2024-09-12T03:21:48.273734+0000","last_peered":"2024-09-12T03:21:48.273734+0000","last_clean":"2024-09-12T03:21:48.273734+0000","last_became_active":"2024-09-12T03:21:47.274745+0000","last_became_peered":"2024-09-12T03:21:47.274745+0000","last_unstale":"2024-09-12T03:21:48.273734+0000","last_undegraded":"2024-09-12T03:21:48.273734+0000","last_fullsized":"2024-09-12T03:21:48.273734+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:49:39.489484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1d","version":"113'2","reported_seq":16,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.725704+0000","last_change":"2024-09-12T03:21:46.258040+0000","last_active":"2024-09-12T03:21:48.725704+0000","last_peered":"2024-09-12T03:21:48.725704+0000","last_clean":"2024-09-12T03:21:48.725704+0000","last_became_active":"2024-09-12T03:21:46.257747+0000","last_became_peered":"2024-09-12T03:21:46.257747+0000","last_unstale":"2024-09-12T03:21:48.725704+0000","last_undegraded":"2024-09-12T03:21:48.725704+0000","last_fullsized":"2024-09-12T03:21:48.725704+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_clean_scrub_stamp":"2024-09-12T03:21:45.231023+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:07:24.863710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.1d","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.273966+0000","last_change":"2024-09-12T03:21:47.273642+0000","last_active":"2024-09-12T03:21:48.273966+0000","last_peered":"2024-09-12T03:21:48.273966+0000","last_clean":"2024-09-12T03:21:48.273966+0000","last_became_active":"2024-09-12T03:21:47.272464+0000","last_became_peered":"2024-09-12T03:21:47.272464+0000","last_unstale":"2024-09-12T03:21:48.273966+0000","last_undegraded":"2024-09-12T03:21:48.273966+0000","last_fullsized":"2024-09-12T03:21:48.273966+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:55:33.036487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1c","version":"113'2","reported_seq":16,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.725395+0000","last_change":"2024-09-12T03:21:46.266144+0000","last_active":"2024-09-12T03:21:48.725395+0000","last_peered":"2024-09-12T03:21:48.725395+0000","last_clean":"2024-09-12T03:21:48.725395+0000","last_became_active":"2024-09-12T03:21:46.265841+0000","last_became_peered":"2024-09-12T03:21:46.265841+0000","last_unstale":"2024-09-12T03:21:48.725395+0000","last_undegraded":"2024-09-12T03:21:48.725395+0000","last_fullsized":"2024-09-12T03:21:48.725395+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_clean_scrub_stamp":"2024-09-12T03:21:45.231023+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:46:45.472576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.725331+0000","last_change":"2024-09-12T03:21:47.263438+0000","last_active":"2024-09-12T03:21:48.725331+0000","last_peered":"2024-09-12T03:21:48.725331+0000","last_clean":"2024-09-12T03:21:48.725331+0000","last_became_active":"2024-09-12T03:21:47.263100+0000","last_became_peered":"2024-09-12T03:21:47.263100+0000","last_unstale":"2024-09-12T03:21:48.725331+0000","last_undegraded":"2024-09-12T03:21:48.725331+0000","last_fullsized":"2024-09-12T03:21:48.725331+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:45:51.865826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1b","version":"113'1","reported_seq":15,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.724274+0000","last_change":"2024-09-12T03:21:46.264671+0000","last_active":"2024-09-12T03:21:48.724274+0000","last_peered":"2024-09-12T03:21:48.724274+0000","last_clean":"2024-09-12T03:21:48.724274+0000","last_became_active":"2024-09-12T03:21:46.264133+0000","last_became_peered":"2024-09-12T03:21:46.264133+0000","last_unstale":"2024-09-12T03:21:48.724274+0000","last_undegraded":"2024-09-12T03:21:48.724274+0000","last_fullsized":"2024-09-12T03:21:48.724274+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_clean_scrub_stamp":"2024-09-12T03:21:45.231023+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:16:07.856351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1b","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.267682+0000","last_change":"2024-09-12T03:21:47.277615+0000","last_active":"2024-09-12T03:21:48.267682+0000","last_peered":"2024-09-12T03:21:48.267682+0000","last_clean":"2024-09-12T03:21:48.267682+0000","last_became_active":"2024-09-12T03:21:47.277321+0000","last_became_peered":"2024-09-12T03:21:47.277321+0000","last_unstale":"2024-09-12T03:21:48.267682+0000","last_undegraded":"2024-09-12T03:21:48.267682+0000","last_fullsized":"2024-09-12T03:21:48.267682+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:20:42.594004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1a","version":"113'1","reported_seq":15,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.725273+0000","last_change":"2024-09-12T03:21:46.263053+0000","last_active":"2024-09-12T03:21:48.725273+0000","last_peered":"2024-09-12T03:21:48.725273+0000","last_clean":"2024-09-12T03:21:48.725273+0000","last_became_active":"2024-09-12T03:21:46.262771+0000","last_became_peered":"2024-09-12T03:21:46.262771+0000","last_unstale":"2024-09-12T03:21:48.725273+0000","last_undegraded":"2024-09-12T03:21:48.725273+0000","last_fullsized":"2024-09-12T03:21:48.725273+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_clean_scrub_stamp":"2024-09-12T03:21:45.231023+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:35:56.500559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.599066+0000","last_change":"2024-09-12T03:21:47.281084+0000","last_active":"2024-09-12T03:21:48.599066+0000","last_peered":"2024-09-12T03:21:48.599066+0000","last_clean":"2024-09-12T03:21:48.599066+0000","last_became_active":"2024-09-12T03:21:47.280767+0000","last_became_peered":"2024-09-12T03:21:47.280767+0000","last_unstale":"2024-09-12T03:21:48.599066+0000","last_undegraded":"2024-09-12T03:21:48.599066+0000","last_fullsized":"2024-09-12T03:21:48.599066+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:16:45.132013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":14,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.266696+0000","last_change":"2024-09-12T03:21:46.253748+0000","last_active":"2024-09-12T03:21:48.266696+0000","last_peered":"2024-09-12T03:21:48.266696+0000","last_clean":"2024-09-12T03:21:48.266696+0000","last_became_active":"2024-09-12T03:21:46.253381+0000","last_became_peered":"2024-09-12T03:21:46.253381+0000","last_unstale":"2024-09-12T03:21:48.266696+0000","last_undegraded":"2024-09-12T03:21:48.266696+0000","last_fullsized":"2024-09-12T03:21:48.266696+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_clean_scrub_stamp":"2024-09-12T03:21:45.231023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:33:59.776477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.724917+0000","last_change":"2024-09-12T03:21:47.278433+0000","last_active":"2024-09-12T03:21:48.724917+0000","last_peered":"2024-09-12T03:21:48.724917+0000","last_clean":"2024-09-12T03:21:48.724917+0000","last_became_active":"2024-09-12T03:21:47.278146+0000","last_became_peered":"2024-09-12T03:21:47.278146+0000","last_unstale":"2024-09-12T03:21:48.724917+0000","last_undegraded":"2024-09-12T03:21:48.724917+0000","last_fullsized":"2024-09-12T03:21:48.724917+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:02:37.440594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":14,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.724426+0000","last_change":"2024-09-12T03:21:46.264728+0000","last_active":"2024-09-12T03:21:48.724426+0000","last_peered":"2024-09-12T03:21:48.724426+0000","last_clean":"2024-09-12T03:21:48.724426+0000","last_became_active":"2024-09-12T03:21:46.263798+0000","last_became_peered":"2024-09-12T03:21:46.263798+0000","last_unstale":"2024-09-12T03:21:48.724426+0000","last_undegraded":"2024-09-12T03:21:48.724426+0000","last_fullsized":"2024-09-12T03:21:48.724426+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_clean_scrub_stamp":"2024-09-12T03:21:45.231023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:12:16.266409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.266463+0000","last_change":"2024-09-12T03:21:47.277867+0000","last_active":"2024-09-12T03:21:48.266463+0000","last_peered":"2024-09-12T03:21:48.266463+0000","last_clean":"2024-09-12T03:21:48.266463+0000","last_became_active":"2024-09-12T03:21:47.277538+0000","last_became_peered":"2024-09-12T03:21:47.277538+0000","last_unstale":"2024-09-12T03:21:48.266463+0000","last_undegraded":"2024-09-12T03:21:48.266463+0000","last_fullsized":"2024-09-12T03:21:48.266463+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:33:42.064998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":14,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.268143+0000","last_change":"2024-09-12T03:21:46.257496+0000","last_active":"2024-09-12T03:21:48.268143+0000","last_peered":"2024-09-12T03:21:48.268143+0000","last_clean":"2024-09-12T03:21:48.268143+0000","last_became_active":"2024-09-12T03:21:46.257152+0000","last_became_peered":"2024-09-12T03:21:46.257152+0000","last_unstale":"2024-09-12T03:21:48.268143+0000","last_undegraded":"2024-09-12T03:21:48.268143+0000","last_fullsized":"2024-09-12T03:21:48.268143+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_clean_scrub_stamp":"2024-09-12T03:21:45.231023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:29:42.420816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.268097+0000","last_change":"2024-09-12T03:21:47.277451+0000","last_active":"2024-09-12T03:21:48.268097+0000","last_peered":"2024-09-12T03:21:48.268097+0000","last_clean":"2024-09-12T03:21:48.268097+0000","last_became_active":"2024-09-12T03:21:47.277147+0000","last_became_peered":"2024-09-12T03:21:47.277147+0000","last_unstale":"2024-09-12T03:21:48.268097+0000","last_undegraded":"2024-09-12T03:21:48.268097+0000","last_fullsized":"2024-09-12T03:21:48.268097+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:02:37.440594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":14,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.724804+0000","last_change":"2024-09-12T03:21:46.254525+0000","last_active":"2024-09-12T03:21:48.724804+0000","last_peered":"2024-09-12T03:21:48.724804+0000","last_clean":"2024-09-12T03:21:48.724804+0000","last_became_active":"2024-09-12T03:21:46.254190+0000","last_became_peered":"2024-09-12T03:21:46.254190+0000","last_unstale":"2024-09-12T03:21:48.724804+0000","last_undegraded":"2024-09-12T03:21:48.724804+0000","last_fullsized":"2024-09-12T03:21:48.724804+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_clean_scrub_stamp":"2024-09-12T03:21:45.231023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:16:44.125918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.14","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.273888+0000","last_change":"2024-09-12T03:21:47.278622+0000","last_active":"2024-09-12T03:21:48.273888+0000","last_peered":"2024-09-12T03:21:48.273888+0000","last_clean":"2024-09-12T03:21:48.273888+0000","last_became_active":"2024-09-12T03:21:47.278333+0000","last_became_peered":"2024-09-12T03:21:47.278333+0000","last_unstale":"2024-09-12T03:21:48.273888+0000","last_undegraded":"2024-09-12T03:21:48.273888+0000","last_fullsized":"2024-09-12T03:21:48.273888+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:37:21.066404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.15","version":"113'3","reported_seq":17,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.266666+0000","last_change":"2024-09-12T03:21:46.252566+0000","last_active":"2024-09-12T03:21:48.266666+0000","last_peered":"2024-09-12T03:21:48.266666+0000","last_clean":"2024-09-12T03:21:48.266666+0000","last_became_active":"2024-09-12T03:21:46.252247+0000","last_became_peered":"2024-09-12T03:21:46.252247+0000","last_unstale":"2024-09-12T03:21:48.266666+0000","last_undegraded":"2024-09-12T03:21:48.266666+0000","last_fullsized":"2024-09-12T03:21:48.266666+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_clean_scrub_stamp":"2024-09-12T03:21:45.231023+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:09:06.932896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.15","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.274070+0000","last_change":"2024-09-12T03:21:47.271363+0000","last_active":"2024-09-12T03:21:48.274070+0000","last_peered":"2024-09-12T03:21:48.274070+0000","last_clean":"2024-09-12T03:21:48.274070+0000","last_became_active":"2024-09-12T03:21:47.271060+0000","last_became_peered":"2024-09-12T03:21:47.271060+0000","last_unstale":"2024-09-12T03:21:48.274070+0000","last_undegraded":"2024-09-12T03:21:48.274070+0000","last_fullsized":"2024-09-12T03:21:48.274070+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:26:02.679898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.14","version":"113'6","reported_seq":20,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.598854+0000","last_change":"2024-09-12T03:21:46.255120+0000","last_active":"2024-09-12T03:21:48.598854+0000","last_peered":"2024-09-12T03:21:48.598854+0000","last_clean":"2024-09-12T03:21:48.598854+0000","last_became_active":"2024-09-12T03:21:46.254812+0000","last_became_peered":"2024-09-12T03:21:46.254812+0000","last_unstale":"2024-09-12T03:21:48.598854+0000","last_undegraded":"2024-09-12T03:21:48.598854+0000","last_fullsized":"2024-09-12T03:21:48.598854+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_clean_scrub_stamp":"2024-09-12T03:21:45.231023+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:24:52.363839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.13","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.266605+0000","last_change":"2024-09-12T03:21:47.283523+0000","last_active":"2024-09-12T03:21:48.266605+0000","last_peered":"2024-09-12T03:21:48.266605+0000","last_clean":"2024-09-12T03:21:48.266605+0000","last_became_active":"2024-09-12T03:21:47.279638+0000","last_became_peered":"2024-09-12T03:21:47.279638+0000","last_unstale":"2024-09-12T03:21:48.266605+0000","last_undegraded":"2024-09-12T03:21:48.266605+0000","last_fullsized":"2024-09-12T03:21:48.266605+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:10:22.061518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":14,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.266298+0000","last_change":"2024-09-12T03:21:46.260129+0000","last_active":"2024-09-12T03:21:48.266298+0000","last_peered":"2024-09-12T03:21:48.266298+0000","last_clean":"2024-09-12T03:21:48.266298+0000","last_became_active":"2024-09-12T03:21:46.259758+0000","last_became_peered":"2024-09-12T03:21:46.259758+0000","last_unstale":"2024-09-12T03:21:48.266298+0000","last_undegraded":"2024-09-12T03:21:48.266298+0000","last_fullsized":"2024-09-12T03:21:48.266298+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_clean_scrub_stamp":"2024-09-12T03:21:45.231023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:39:56.115395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.10","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.724236+0000","last_change":"2024-09-12T03:21:47.278630+0000","last_active":"2024-09-12T03:21:48.724236+0000","last_peered":"2024-09-12T03:21:48.724236+0000","last_clean":"2024-09-12T03:21:48.724236+0000","last_became_active":"2024-09-12T03:21:47.278333+0000","last_became_peered":"2024-09-12T03:21:47.278333+0000","last_unstale":"2024-09-12T03:21:48.724236+0000","last_undegraded":"2024-09-12T03:21:48.724236+0000","last_fullsized":"2024-09-12T03:21:48.724236+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:31:24.916429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":14,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.725104+0000","last_change":"2024-09-12T03:21:46.263275+0000","last_active":"2024-09-12T03:21:48.725104+0000","last_peered":"2024-09-12T03:21:48.725104+0000","last_clean":"2024-09-12T03:21:48.725104+0000","last_became_active":"2024-09-12T03:21:46.262982+0000","last_became_peered":"2024-09-12T03:21:46.262982+0000","last_unstale":"2024-09-12T03:21:48.725104+0000","last_undegraded":"2024-09-12T03:21:48.725104+0000","last_fullsized":"2024-09-12T03:21:48.725104+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_clean_scrub_stamp":"2024-09-12T03:21:45.231023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:35:47.022417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.599034+0000","last_change":"2024-09-12T03:21:47.274722+0000","last_active":"2024-09-12T03:21:48.599034+0000","last_peered":"2024-09-12T03:21:48.599034+0000","last_clean":"2024-09-12T03:21:48.599034+0000","last_became_active":"2024-09-12T03:21:47.274319+0000","last_became_peered":"2024-09-12T03:21:47.274319+0000","last_unstale":"2024-09-12T03:21:48.599034+0000","last_undegraded":"2024-09-12T03:21:48.599034+0000","last_fullsized":"2024-09-12T03:21:48.599034+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:36:45.878775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.10","version":"113'8","reported_seq":22,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.599010+0000","last_change":"2024-09-12T03:21:46.254895+0000","last_active":"2024-09-12T03:21:48.599010+0000","last_peered":"2024-09-12T03:21:48.599010+0000","last_clean":"2024-09-12T03:21:48.599010+0000","last_became_active":"2024-09-12T03:21:46.254582+0000","last_became_peered":"2024-09-12T03:21:46.254582+0000","last_unstale":"2024-09-12T03:21:48.599010+0000","last_undegraded":"2024-09-12T03:21:48.599010+0000","last_fullsized":"2024-09-12T03:21:48.599010+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_clean_scrub_stamp":"2024-09-12T03:21:45.231023+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:40:18.756382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.e","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.598877+0000","last_change":"2024-09-12T03:21:47.279358+0000","last_active":"2024-09-12T03:21:48.598877+0000","last_peered":"2024-09-12T03:21:48.598877+0000","last_clean":"2024-09-12T03:21:48.598877+0000","last_became_active":"2024-09-12T03:21:47.279082+0000","last_became_peered":"2024-09-12T03:21:47.279082+0000","last_unstale":"2024-09-12T03:21:48.598877+0000","last_undegraded":"2024-09-12T03:21:48.598877+0000","last_fullsized":"2024-09-12T03:21:48.598877+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:36:02.708847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.f","version":"113'1","reported_seq":15,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.268246+0000","last_change":"2024-09-12T03:21:46.255444+0000","last_active":"2024-09-12T03:21:48.268246+0000","last_peered":"2024-09-12T03:21:48.268246+0000","last_clean":"2024-09-12T03:21:48.268246+0000","last_became_active":"2024-09-12T03:21:46.255111+0000","last_became_peered":"2024-09-12T03:21:46.255111+0000","last_unstale":"2024-09-12T03:21:48.268246+0000","last_undegraded":"2024-09-12T03:21:48.268246+0000","last_fullsized":"2024-09-12T03:21:48.268246+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_clean_scrub_stamp":"2024-09-12T03:21:45.231023+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:01:19.852084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.f","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.268199+0000","last_change":"2024-09-12T03:21:47.275553+0000","last_active":"2024-09-12T03:21:48.268199+0000","last_peered":"2024-09-12T03:21:48.268199+0000","last_clean":"2024-09-12T03:21:48.268199+0000","last_became_active":"2024-09-12T03:21:47.275232+0000","last_became_peered":"2024-09-12T03:21:47.275232+0000","last_unstale":"2024-09-12T03:21:48.268199+0000","last_undegraded":"2024-09-12T03:21:48.268199+0000","last_fullsized":"2024-09-12T03:21:48.268199+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:42:51.860983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.e","version":"113'2","reported_seq":16,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.725806+0000","last_change":"2024-09-12T03:21:46.253780+0000","last_active":"2024-09-12T03:21:48.725806+0000","last_peered":"2024-09-12T03:21:48.725806+0000","last_clean":"2024-09-12T03:21:48.725806+0000","last_became_active":"2024-09-12T03:21:46.253480+0000","last_became_peered":"2024-09-12T03:21:46.253480+0000","last_unstale":"2024-09-12T03:21:48.725806+0000","last_undegraded":"2024-09-12T03:21:48.725806+0000","last_fullsized":"2024-09-12T03:21:48.725806+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_clean_scrub_stamp":"2024-09-12T03:21:45.231023+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:38:27.071851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.725011+0000","last_change":"2024-09-12T03:21:47.279125+0000","last_active":"2024-09-12T03:21:48.725011+0000","last_peered":"2024-09-12T03:21:48.725011+0000","last_clean":"2024-09-12T03:21:48.725011+0000","last_became_active":"2024-09-12T03:21:47.278602+0000","last_became_peered":"2024-09-12T03:21:47.278602+0000","last_unstale":"2024-09-12T03:21:48.725011+0000","last_undegraded":"2024-09-12T03:21:48.725011+0000","last_fullsized":"2024-09-12T03:21:48.725011+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:07:41.072755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.d","version":"113'1","reported_seq":15,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.267378+0000","last_change":"2024-09-12T03:21:46.251440+0000","last_active":"2024-09-12T03:21:48.267378+0000","last_peered":"2024-09-12T03:21:48.267378+0000","last_clean":"2024-09-12T03:21:48.267378+0000","last_became_active":"2024-09-12T03:21:46.251107+0000","last_became_peered":"2024-09-12T03:21:46.251107+0000","last_unstale":"2024-09-12T03:21:48.267378+0000","last_undegraded":"2024-09-12T03:21:48.267378+0000","last_fullsized":"2024-09-12T03:21:48.267378+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_clean_scrub_stamp":"2024-09-12T03:21:45.231023+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:29:23.941544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.725059+0000","last_change":"2024-09-12T03:21:47.278687+0000","last_active":"2024-09-12T03:21:48.725059+0000","last_peered":"2024-09-12T03:21:48.725059+0000","last_clean":"2024-09-12T03:21:48.725059+0000","last_became_active":"2024-09-12T03:21:47.277859+0000","last_became_peered":"2024-09-12T03:21:47.277859+0000","last_unstale":"2024-09-12T03:21:48.725059+0000","last_undegraded":"2024-09-12T03:21:48.725059+0000","last_fullsized":"2024-09-12T03:21:48.725059+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:17:45.434501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":14,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.267750+0000","last_change":"2024-09-12T03:21:46.249240+0000","last_active":"2024-09-12T03:21:48.267750+0000","last_peered":"2024-09-12T03:21:48.267750+0000","last_clean":"2024-09-12T03:21:48.267750+0000","last_became_active":"2024-09-12T03:21:46.248701+0000","last_became_peered":"2024-09-12T03:21:46.248701+0000","last_unstale":"2024-09-12T03:21:48.267750+0000","last_undegraded":"2024-09-12T03:21:48.267750+0000","last_fullsized":"2024-09-12T03:21:48.267750+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_clean_scrub_stamp":"2024-09-12T03:21:45.231023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:50:01.855914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.266214+0000","last_change":"2024-09-12T03:21:47.282909+0000","last_active":"2024-09-12T03:21:48.266214+0000","last_peered":"2024-09-12T03:21:48.266214+0000","last_clean":"2024-09-12T03:21:48.266214+0000","last_became_active":"2024-09-12T03:21:47.282527+0000","last_became_peered":"2024-09-12T03:21:47.282527+0000","last_unstale":"2024-09-12T03:21:48.266214+0000","last_undegraded":"2024-09-12T03:21:48.266214+0000","last_fullsized":"2024-09-12T03:21:48.266214+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:48:16.365941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.b","version":"113'2","reported_seq":16,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.267395+0000","last_change":"2024-09-12T03:21:46.262019+0000","last_active":"2024-09-12T03:21:48.267395+0000","last_peered":"2024-09-12T03:21:48.267395+0000","last_clean":"2024-09-12T03:21:48.267395+0000","last_became_active":"2024-09-12T03:21:46.261674+0000","last_became_peered":"2024-09-12T03:21:46.261674+0000","last_unstale":"2024-09-12T03:21:48.267395+0000","last_undegraded":"2024-09-12T03:21:48.267395+0000","last_fullsized":"2024-09-12T03:21:48.267395+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_clean_scrub_stamp":"2024-09-12T03:21:45.231023+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:16:38.849993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.b","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.267739+0000","last_change":"2024-09-12T03:21:47.272033+0000","last_active":"2024-09-12T03:21:48.267739+0000","last_peered":"2024-09-12T03:21:48.267739+0000","last_clean":"2024-09-12T03:21:48.267739+0000","last_became_active":"2024-09-12T03:21:47.271715+0000","last_became_peered":"2024-09-12T03:21:47.271715+0000","last_unstale":"2024-09-12T03:21:48.267739+0000","last_undegraded":"2024-09-12T03:21:48.267739+0000","last_fullsized":"2024-09-12T03:21:48.267739+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:49:07.547396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":14,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.725365+0000","last_change":"2024-09-12T03:21:46.255383+0000","last_active":"2024-09-12T03:21:48.725365+0000","last_peered":"2024-09-12T03:21:48.725365+0000","last_clean":"2024-09-12T03:21:48.725365+0000","last_became_active":"2024-09-12T03:21:46.254479+0000","last_became_peered":"2024-09-12T03:21:46.254479+0000","last_unstale":"2024-09-12T03:21:48.725365+0000","last_undegraded":"2024-09-12T03:21:48.725365+0000","last_fullsized":"2024-09-12T03:21:48.725365+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_clean_scrub_stamp":"2024-09-12T03:21:45.231023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:36:44.872680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.724377+0000","last_change":"2024-09-12T03:21:47.276243+0000","last_active":"2024-09-12T03:21:48.724377+0000","last_peered":"2024-09-12T03:21:48.724377+0000","last_clean":"2024-09-12T03:21:48.724377+0000","last_became_active":"2024-09-12T03:21:47.275973+0000","last_became_peered":"2024-09-12T03:21:47.275973+0000","last_unstale":"2024-09-12T03:21:48.724377+0000","last_undegraded":"2024-09-12T03:21:48.724377+0000","last_fullsized":"2024-09-12T03:21:48.724377+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:40:21.225897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":14,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.725355+0000","last_change":"2024-09-12T03:21:46.256740+0000","last_active":"2024-09-12T03:21:48.725355+0000","last_peered":"2024-09-12T03:21:48.725355+0000","last_clean":"2024-09-12T03:21:48.725355+0000","last_became_active":"2024-09-12T03:21:46.256157+0000","last_became_peered":"2024-09-12T03:21:46.256157+0000","last_unstale":"2024-09-12T03:21:48.725355+0000","last_undegraded":"2024-09-12T03:21:48.725355+0000","last_fullsized":"2024-09-12T03:21:48.725355+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_clean_scrub_stamp":"2024-09-12T03:21:45.231023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:27:47.060764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.273562+0000","last_change":"2024-09-12T03:21:47.282425+0000","last_active":"2024-09-12T03:21:48.273562+0000","last_peered":"2024-09-12T03:21:48.273562+0000","last_clean":"2024-09-12T03:21:48.273562+0000","last_became_active":"2024-09-12T03:21:47.282126+0000","last_became_peered":"2024-09-12T03:21:47.282126+0000","last_unstale":"2024-09-12T03:21:48.273562+0000","last_undegraded":"2024-09-12T03:21:48.273562+0000","last_fullsized":"2024-09-12T03:21:48.273562+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:45:51.865826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":14,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.725014+0000","last_change":"2024-09-12T03:21:46.253061+0000","last_active":"2024-09-12T03:21:48.725014+0000","last_peered":"2024-09-12T03:21:48.725014+0000","last_clean":"2024-09-12T03:21:48.725014+0000","last_became_active":"2024-09-12T03:21:46.252712+0000","last_became_peered":"2024-09-12T03:21:46.252712+0000","last_unstale":"2024-09-12T03:21:48.725014+0000","last_undegraded":"2024-09-12T03:21:48.725014+0000","last_fullsized":"2024-09-12T03:21:48.725014+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_clean_scrub_stamp":"2024-09-12T03:21:45.231023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:56:16.255201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.267454+0000","last_change":"2024-09-12T03:21:47.266290+0000","last_active":"2024-09-12T03:21:48.267454+0000","last_peered":"2024-09-12T03:21:48.267454+0000","last_clean":"2024-09-12T03:21:48.267454+0000","last_became_active":"2024-09-12T03:21:47.264199+0000","last_became_peered":"2024-09-12T03:21:47.264199+0000","last_unstale":"2024-09-12T03:21:48.267454+0000","last_undegraded":"2024-09-12T03:21:48.267454+0000","last_fullsized":"2024-09-12T03:21:48.267454+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:41:26.120489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.7","version":"113'2","reported_seq":16,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.266586+0000","last_change":"2024-09-12T03:21:46.260654+0000","last_active":"2024-09-12T03:21:48.266586+0000","last_peered":"2024-09-12T03:21:48.266586+0000","last_clean":"2024-09-12T03:21:48.266586+0000","last_became_active":"2024-09-12T03:21:46.260300+0000","last_became_peered":"2024-09-12T03:21:46.260300+0000","last_unstale":"2024-09-12T03:21:48.266586+0000","last_undegraded":"2024-09-12T03:21:48.266586+0000","last_fullsized":"2024-09-12T03:21:48.266586+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_clean_scrub_stamp":"2024-09-12T03:21:45.231023+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:42:29.591906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.7","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.266487+0000","last_change":"2024-09-12T03:21:47.282067+0000","last_active":"2024-09-12T03:21:48.266487+0000","last_peered":"2024-09-12T03:21:48.266487+0000","last_clean":"2024-09-12T03:21:48.266487+0000","last_became_active":"2024-09-12T03:21:47.276839+0000","last_became_peered":"2024-09-12T03:21:47.276839+0000","last_unstale":"2024-09-12T03:21:48.266487+0000","last_undegraded":"2024-09-12T03:21:48.266487+0000","last_fullsized":"2024-09-12T03:21:48.266487+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:50:01.620996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.6","version":"114'1","reported_seq":15,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.273910+0000","last_change":"2024-09-12T03:21:46.259229+0000","last_active":"2024-09-12T03:21:48.273910+0000","last_peered":"2024-09-12T03:21:48.273910+0000","last_clean":"2024-09-12T03:21:48.273910+0000","last_became_active":"2024-09-12T03:21:46.258467+0000","last_became_peered":"2024-09-12T03:21:46.258467+0000","last_unstale":"2024-09-12T03:21:48.273910+0000","last_undegraded":"2024-09-12T03:21:48.273910+0000","last_fullsized":"2024-09-12T03:21:48.273910+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_clean_scrub_stamp":"2024-09-12T03:21:45.231023+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:36:44.872680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.725277+0000","last_change":"2024-09-12T03:21:47.273262+0000","last_active":"2024-09-12T03:21:48.725277+0000","last_peered":"2024-09-12T03:21:48.725277+0000","last_clean":"2024-09-12T03:21:48.725277+0000","last_became_active":"2024-09-12T03:21:47.272986+0000","last_became_peered":"2024-09-12T03:21:47.272986+0000","last_unstale":"2024-09-12T03:21:48.725277+0000","last_undegraded":"2024-09-12T03:21:48.725277+0000","last_fullsized":"2024-09-12T03:21:48.725277+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:22:36.819977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":14,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.725080+0000","last_change":"2024-09-12T03:21:46.264077+0000","last_active":"2024-09-12T03:21:48.725080+0000","last_peered":"2024-09-12T03:21:48.725080+0000","last_clean":"2024-09-12T03:21:48.725080+0000","last_became_active":"2024-09-12T03:21:46.262611+0000","last_became_peered":"2024-09-12T03:21:46.262611+0000","last_unstale":"2024-09-12T03:21:48.725080+0000","last_undegraded":"2024-09-12T03:21:48.725080+0000","last_fullsized":"2024-09-12T03:21:48.725080+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_clean_scrub_stamp":"2024-09-12T03:21:45.231023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:02:58.180807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.267440+0000","last_change":"2024-09-12T03:21:47.280655+0000","last_active":"2024-09-12T03:21:48.267440+0000","last_peered":"2024-09-12T03:21:48.267440+0000","last_clean":"2024-09-12T03:21:48.267440+0000","last_became_active":"2024-09-12T03:21:47.280133+0000","last_became_peered":"2024-09-12T03:21:47.280133+0000","last_unstale":"2024-09-12T03:21:48.267440+0000","last_undegraded":"2024-09-12T03:21:48.267440+0000","last_fullsized":"2024-09-12T03:21:48.267440+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:54:34.656137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":14,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.725663+0000","last_change":"2024-09-12T03:21:46.265178+0000","last_active":"2024-09-12T03:21:48.725663+0000","last_peered":"2024-09-12T03:21:48.725663+0000","last_clean":"2024-09-12T03:21:48.725663+0000","last_became_active":"2024-09-12T03:21:46.264842+0000","last_became_peered":"2024-09-12T03:21:46.264842+0000","last_unstale":"2024-09-12T03:21:48.725663+0000","last_undegraded":"2024-09-12T03:21:48.725663+0000","last_fullsized":"2024-09-12T03:21:48.725663+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_clean_scrub_stamp":"2024-09-12T03:21:45.231023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:51:50.034404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.273569+0000","last_change":"2024-09-12T03:21:47.275959+0000","last_active":"2024-09-12T03:21:48.273569+0000","last_peered":"2024-09-12T03:21:48.273569+0000","last_clean":"2024-09-12T03:21:48.273569+0000","last_became_active":"2024-09-12T03:21:47.274773+0000","last_became_peered":"2024-09-12T03:21:47.274773+0000","last_unstale":"2024-09-12T03:21:48.273569+0000","last_undegraded":"2024-09-12T03:21:48.273569+0000","last_fullsized":"2024-09-12T03:21:48.273569+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:21:08.369709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":14,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.724357+0000","last_change":"2024-09-12T03:21:46.265078+0000","last_active":"2024-09-12T03:21:48.724357+0000","last_peered":"2024-09-12T03:21:48.724357+0000","last_clean":"2024-09-12T03:21:48.724357+0000","last_became_active":"2024-09-12T03:21:46.264065+0000","last_became_peered":"2024-09-12T03:21:46.264065+0000","last_unstale":"2024-09-12T03:21:48.724357+0000","last_undegraded":"2024-09-12T03:21:48.724357+0000","last_fullsized":"2024-09-12T03:21:48.724357+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_clean_scrub_stamp":"2024-09-12T03:21:45.231023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:34:03.877327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.267328+0000","last_change":"2024-09-12T03:21:47.283216+0000","last_active":"2024-09-12T03:21:48.267328+0000","last_peered":"2024-09-12T03:21:48.267328+0000","last_clean":"2024-09-12T03:21:48.267328+0000","last_became_active":"2024-09-12T03:21:47.282949+0000","last_became_peered":"2024-09-12T03:21:47.282949+0000","last_unstale":"2024-09-12T03:21:48.267328+0000","last_undegraded":"2024-09-12T03:21:48.267328+0000","last_fullsized":"2024-09-12T03:21:48.267328+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:43:43.310362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":14,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.273529+0000","last_change":"2024-09-12T03:21:46.260679+0000","last_active":"2024-09-12T03:21:48.273529+0000","last_peered":"2024-09-12T03:21:48.273529+0000","last_clean":"2024-09-12T03:21:48.273529+0000","last_became_active":"2024-09-12T03:21:46.260354+0000","last_became_peered":"2024-09-12T03:21:46.260354+0000","last_unstale":"2024-09-12T03:21:48.273529+0000","last_undegraded":"2024-09-12T03:21:48.273529+0000","last_fullsized":"2024-09-12T03:21:48.273529+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_clean_scrub_stamp":"2024-09-12T03:21:45.231023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:24:37.527133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.725752+0000","last_change":"2024-09-12T03:21:47.270434+0000","last_active":"2024-09-12T03:21:48.725752+0000","last_peered":"2024-09-12T03:21:48.725752+0000","last_clean":"2024-09-12T03:21:48.725752+0000","last_became_active":"2024-09-12T03:21:47.270141+0000","last_became_peered":"2024-09-12T03:21:47.270141+0000","last_unstale":"2024-09-12T03:21:48.725752+0000","last_undegraded":"2024-09-12T03:21:48.725752+0000","last_fullsized":"2024-09-12T03:21:48.725752+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:00:08.281678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":14,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.725446+0000","last_change":"2024-09-12T03:21:46.256501+0000","last_active":"2024-09-12T03:21:48.725446+0000","last_peered":"2024-09-12T03:21:48.725446+0000","last_clean":"2024-09-12T03:21:48.725446+0000","last_became_active":"2024-09-12T03:21:46.256069+0000","last_became_peered":"2024-09-12T03:21:46.256069+0000","last_unstale":"2024-09-12T03:21:48.725446+0000","last_undegraded":"2024-09-12T03:21:48.725446+0000","last_fullsized":"2024-09-12T03:21:48.725446+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_clean_scrub_stamp":"2024-09-12T03:21:45.231023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:24:37.527133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.725055+0000","last_change":"2024-09-12T03:21:47.269440+0000","last_active":"2024-09-12T03:21:48.725055+0000","last_peered":"2024-09-12T03:21:48.725055+0000","last_clean":"2024-09-12T03:21:48.725055+0000","last_became_active":"2024-09-12T03:21:47.269145+0000","last_became_peered":"2024-09-12T03:21:47.269145+0000","last_unstale":"2024-09-12T03:21:48.725055+0000","last_undegraded":"2024-09-12T03:21:48.725055+0000","last_fullsized":"2024-09-12T03:21:48.725055+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:42:51.860983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":14,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.725107+0000","last_change":"2024-09-12T03:21:46.263800+0000","last_active":"2024-09-12T03:21:48.725107+0000","last_peered":"2024-09-12T03:21:48.725107+0000","last_clean":"2024-09-12T03:21:48.725107+0000","last_became_active":"2024-09-12T03:21:46.263423+0000","last_became_peered":"2024-09-12T03:21:46.263423+0000","last_unstale":"2024-09-12T03:21:48.725107+0000","last_undegraded":"2024-09-12T03:21:48.725107+0000","last_fullsized":"2024-09-12T03:21:48.725107+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_clean_scrub_stamp":"2024-09-12T03:21:45.231023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:06:20.324237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.273458+0000","last_change":"2024-09-12T03:21:47.281952+0000","last_active":"2024-09-12T03:21:48.273458+0000","last_peered":"2024-09-12T03:21:48.273458+0000","last_clean":"2024-09-12T03:21:48.273458+0000","last_became_active":"2024-09-12T03:21:47.279248+0000","last_became_peered":"2024-09-12T03:21:47.279248+0000","last_unstale":"2024-09-12T03:21:48.273458+0000","last_undegraded":"2024-09-12T03:21:48.273458+0000","last_fullsized":"2024-09-12T03:21:48.273458+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:08:22.543376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"113'2","reported_seq":16,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.273663+0000","last_change":"2024-09-12T03:21:46.265296+0000","last_active":"2024-09-12T03:21:48.273663+0000","last_peered":"2024-09-12T03:21:48.273663+0000","last_clean":"2024-09-12T03:21:48.273663+0000","last_became_active":"2024-09-12T03:21:46.264982+0000","last_became_peered":"2024-09-12T03:21:46.264982+0000","last_unstale":"2024-09-12T03:21:48.273663+0000","last_undegraded":"2024-09-12T03:21:48.273663+0000","last_fullsized":"2024-09-12T03:21:48.273663+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_clean_scrub_stamp":"2024-09-12T03:21:45.231023+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:33:45.204480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"70'128","reported_seq":381,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.273224+0000","last_change":"2024-09-12T03:16:21.883446+0000","last_active":"2024-09-12T03:21:48.273224+0000","last_peered":"2024-09-12T03:21:48.273224+0000","last_clean":"2024-09-12T03:21:48.273224+0000","last_became_active":"2024-09-12T03:16:21.882207+0000","last_became_peered":"2024-09-12T03:16:21.882207+0000","last_unstale":"2024-09-12T03:21:48.273224+0000","last_undegraded":"2024-09-12T03:21:48.273224+0000","last_fullsized":"2024-09-12T03:21:48.273224+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:20.316426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:20.316426+0000","last_clean_scrub_stamp":"2024-09-12T03:16:20.316426+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:12:15.424320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":19,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542212,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37348,"kb_used_data":1992,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334492,"statfs":{"total":96636764160,"available":96598519808,"internally_reserved":0,"allocated":2039808,"data_stored":1219971,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542214,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36748,"kb_used_data":1400,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335092,"statfs":{"total":96636764160,"available":96599134208,"internally_reserved":0,"allocated":1433600,"data_stored":628965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542213,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36748,"kb_used_data":1396,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335092,"statfs":{"total":96636764160,"available":96599134208,"internally_reserved":0,"allocated":1429504,"data_stored":627465,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542212,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36740,"kb_used_data":1392,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335100,"statfs":{"total":96636764160,"available":96599142400,"internally_reserved":0,"allocated":1425408,"data_stored":627423,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542213,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32584,"kb_used_data":1396,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339256,"statfs":{"total":96636764160,"available":96603398144,"internally_reserved":0,"allocated":1429504,"data_stored":627513,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542213,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32592,"kb_used_data":1396,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339248,"statfs":{"total":96636764160,"available":96603389952,"internally_reserved":0,"allocated":1429504,"data_stored":627969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542214,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32580,"kb_used_data":1388,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339260,"statfs":{"total":96636764160,"available":96603402240,"internally_reserved":0,"allocated":1421312,"data_stored":627353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542214,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33156,"kb_used_data":1968,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338684,"statfs":{"total":96636764160,"available":96602812416,"internally_reserved":0,"allocated":2015232,"data_stored":1217721,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T03:21:55.012 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-12T03:21:55.129 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:55.128+0000 7f0d7eaf1640 1 Processor -- start 2024-09-12T03:21:55.129 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:55.129+0000 7f0d7eaf1640 1 -- start start 2024-09-12T03:21:55.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:55.129+0000 7f0d7eaf1640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f0d78073960 0x7f0d78162320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:55.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:55.129+0000 7f0d7eaf1640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0d780d2240 con 0x7f0d7815e5f0 2024-09-12T03:21:55.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:55.129+0000 7f0d7eaf1640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0d78163090 con 0x7f0d78162860 2024-09-12T03:21:55.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:55.130+0000 7f0d7eaf1640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f0d780d0da0 con 0x7f0d78073960 2024-09-12T03:21:55.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:55.130+0000 7f0d77fff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f0d78073960 0x7f0d78162320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:55.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:55.130+0000 7f0d7c88d640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f0d7815e5f0 0x7f0d78073250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:56364/0 (socket says 172.21.15.27:56364) 2024-09-12T03:21:55.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:55.130+0000 7f0d7c88d640 1 -- 172.21.15.27:0/581537163 learned_addr learned my addr 172.21.15.27:0/581537163 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:21:55.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:55.130+0000 7f0d77fff640 1 -- 172.21.15.27:0/581537163 >> v1:172.21.15.45:6789/0 conn(0x7f0d78162860 legacy=0x7f0d7815e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:55.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:55.130+0000 7f0d77fff640 1 -- 172.21.15.27:0/581537163 >> v1:172.21.15.27:6789/0 conn(0x7f0d7815e5f0 legacy=0x7f0d78073250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:55.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:55.130+0000 7f0d77fff640 1 -- 172.21.15.27:0/581537163 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d780d0470 con 0x7f0d78073960 2024-09-12T03:21:55.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:55.131+0000 7f0d77fff640 1 --2- 172.21.15.27:0/581537163 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0d78073960 0x7f0d78162320 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f0d640097a0 tx=0x7f0d6402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2949e0d8bd94c5c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:55.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:55.131+0000 7f0d777fe640 1 -- 172.21.15.27:0/581537163 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d64037070 con 0x7f0d78073960 2024-09-12T03:21:55.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:55.131+0000 7f0d777fe640 1 -- 172.21.15.27:0/581537163 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d64032440 con 0x7f0d78073960 2024-09-12T03:21:55.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:55.131+0000 7f0d777fe640 1 -- 172.21.15.27:0/581537163 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d64032810 con 0x7f0d78073960 2024-09-12T03:21:55.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:55.131+0000 7f0d7eaf1640 1 -- 172.21.15.27:0/581537163 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0d78073960 msgr2=0x7f0d78162320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:55.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:55.131+0000 7f0d7eaf1640 1 --2- 172.21.15.27:0/581537163 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0d78073960 0x7f0d78162320 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f0d640097a0 tx=0x7f0d6402a1e0 comp rx=0 tx=0).stop 2024-09-12T03:21:55.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:55.132+0000 7f0d7eaf1640 1 -- 172.21.15.27:0/581537163 shutdown_connections 2024-09-12T03:21:55.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:55.132+0000 7f0d7eaf1640 1 --2- 172.21.15.27:0/581537163 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0d78073960 0x7f0d78162320 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:55.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:55.132+0000 7f0d7eaf1640 1 -- 172.21.15.27:0/581537163 >> 172.21.15.27:0/581537163 conn(0x7f0d7806f300 msgr2=0x7f0d7806f710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:55.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:55.132+0000 7f0d7eaf1640 1 -- 172.21.15.27:0/581537163 shutdown_connections 2024-09-12T03:21:55.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:55.132+0000 7f0d7eaf1640 1 -- 172.21.15.27:0/581537163 wait complete. 2024-09-12T03:21:55.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:55.133+0000 7f0d7eaf1640 1 Processor -- start 2024-09-12T03:21:55.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:55.133+0000 7f0d7eaf1640 1 -- start start 2024-09-12T03:21:55.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:55.134+0000 7f0d7eaf1640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0d78073960 0x7f0d7815cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:55.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:55.134+0000 7f0d7eaf1640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0d7815e5f0 0x7f0d781e8d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:55.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:55.134+0000 7f0d7eaf1640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f0d78162860 0x7f0d781e9280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:55.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:55.134+0000 7f0d7eaf1640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f0d780b5620 con 0x7f0d7815e5f0 2024-09-12T03:21:55.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:55.134+0000 7f0d7eaf1640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f0d780d0a30 con 0x7f0d78073960 2024-09-12T03:21:55.138 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:55.134+0000 7f0d7eaf1640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f0d78162c70 con 0x7f0d78162860 2024-09-12T03:21:55.138 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:55.134+0000 7f0d77fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0d7815e5f0 0x7f0d781e8d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:55.138 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:55.134+0000 7f0d77fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0d7815e5f0 0x7f0d781e8d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39542/0 (socket says 172.21.15.27:39542) 2024-09-12T03:21:55.138 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:55.134+0000 7f0d77fff640 1 -- 172.21.15.27:0/1275887400 learned_addr learned my addr 172.21.15.27:0/1275887400 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:21:55.138 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:55.134+0000 7f0d7c88d640 1 --2- 172.21.15.27:0/1275887400 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0d78073960 0x7f0d7815cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:55.138 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:55.134+0000 7f0d77fff640 1 -- 172.21.15.27:0/1275887400 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f0d78162860 msgr2=0x7f0d781e9280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:55.138 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:55.134+0000 7f0d7d08e640 1 --2- 172.21.15.27:0/1275887400 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f0d78162860 0x7f0d781e9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:55.138 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:55.134+0000 7f0d77fff640 1 --2- 172.21.15.27:0/1275887400 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f0d78162860 0x7f0d781e9280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:55.138 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:55.134+0000 7f0d77fff640 1 -- 172.21.15.27:0/1275887400 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0d78073960 msgr2=0x7f0d7815cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:55.138 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:55.134+0000 7f0d77fff640 1 --2- 172.21.15.27:0/1275887400 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0d78073960 0x7f0d7815cf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:55.138 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:55.134+0000 7f0d77fff640 1 -- 172.21.15.27:0/1275887400 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d781561a0 con 0x7f0d7815e5f0 2024-09-12T03:21:55.138 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:55.135+0000 7f0d77fff640 1 --2- 172.21.15.27:0/1275887400 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0d7815e5f0 0x7f0d781e8d40 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f0d64000c00 tx=0x7f0d640046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:55.138 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:55.135+0000 7f0d757fa640 1 -- 172.21.15.27:0/1275887400 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d64037040 con 0x7f0d7815e5f0 2024-09-12T03:21:55.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:55.135+0000 7f0d757fa640 1 -- 172.21.15.27:0/1275887400 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d64042070 con 0x7f0d7815e5f0 2024-09-12T03:21:55.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:55.135+0000 7f0d757fa640 1 -- 172.21.15.27:0/1275887400 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d64030a70 con 0x7f0d7815e5f0 2024-09-12T03:21:55.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:55.135+0000 7f0d7eaf1640 1 -- 172.21.15.27:0/1275887400 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d781e9920 con 0x7f0d7815e5f0 2024-09-12T03:21:55.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:55.135+0000 7f0d7eaf1640 1 -- 172.21.15.27:0/1275887400 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d7815a750 con 0x7f0d7815e5f0 2024-09-12T03:21:55.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:55.136+0000 7f0d7eaf1640 1 -- 172.21.15.27:0/1275887400 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d78159dc0 con 0x7f0d7815e5f0 2024-09-12T03:21:55.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:55.136+0000 7f0d757fa640 1 -- 172.21.15.27:0/1275887400 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 19) ==== 90855+0+0 (secure 0 0 0) 0x7f0d64030c10 con 0x7f0d7815e5f0 2024-09-12T03:21:55.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:55.137+0000 7f0d757fa640 1 --2- 172.21.15.27:0/1275887400 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f0d5c06c220 0x7f0d5c06e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:55.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:55.137+0000 7f0d757fa640 1 -- 172.21.15.27:0/1275887400 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5737+0+0 (secure 0 0 0) 0x7f0d640abf90 con 0x7f0d7815e5f0 2024-09-12T03:21:55.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:55.137+0000 7f0d7c88d640 1 --2- 172.21.15.27:0/1275887400 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f0d5c06c220 0x7f0d5c06e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:55.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:55.137+0000 7f0d7c88d640 1 --2- 172.21.15.27:0/1275887400 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f0d5c06c220 0x7f0d5c06e6e0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f0d6c000f60 tx=0x7f0d6c009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:55.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:55.142+0000 7f0d757fa640 1 -- 172.21.15.27:0/1275887400 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f0d781561a0 con 0x7f0d7815e5f0 2024-09-12T03:21:55.316 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:55.317+0000 7f0d7eaf1640 1 -- 172.21.15.27:0/1275887400 --> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f0d78159800 con 0x7f0d5c06c220 2024-09-12T03:21:55.322 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:55.323+0000 7f0d757fa640 1 -- 172.21.15.27:0/1275887400 <== mgr.4113 v2:172.21.15.27:6836/1226920526 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252482 (secure 0 0 0) 0x7f0d78159800 con 0x7f0d5c06c220 2024-09-12T03:21:55.325 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:21:55.326 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2024-09-12T03:21:55.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:55.325+0000 7f0d7eaf1640 1 -- 172.21.15.27:0/1275887400 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f0d5c06c220 msgr2=0x7f0d5c06e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:55.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:55.325+0000 7f0d7eaf1640 1 --2- 172.21.15.27:0/1275887400 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f0d5c06c220 0x7f0d5c06e6e0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f0d6c000f60 tx=0x7f0d6c009040 comp rx=0 tx=0).stop 2024-09-12T03:21:55.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:55.325+0000 7f0d7eaf1640 1 -- 172.21.15.27:0/1275887400 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0d7815e5f0 msgr2=0x7f0d781e8d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:55.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:55.325+0000 7f0d7eaf1640 1 --2- 172.21.15.27:0/1275887400 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0d7815e5f0 0x7f0d781e8d40 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f0d64000c00 tx=0x7f0d640046c0 comp rx=0 tx=0).stop 2024-09-12T03:21:55.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:55.326+0000 7f0d7eaf1640 1 -- 172.21.15.27:0/1275887400 shutdown_connections 2024-09-12T03:21:55.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:55.326+0000 7f0d7eaf1640 1 --2- 172.21.15.27:0/1275887400 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f0d78162860 0x7f0d781e9280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:55.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:55.326+0000 7f0d7eaf1640 1 --2- 172.21.15.27:0/1275887400 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f0d5c06c220 0x7f0d5c06e6e0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:55.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:55.326+0000 7f0d7eaf1640 1 --2- 172.21.15.27:0/1275887400 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0d7815e5f0 0x7f0d781e8d40 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:55.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:55.326+0000 7f0d7eaf1640 1 --2- 172.21.15.27:0/1275887400 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0d78073960 0x7f0d7815cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:55.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:55.326+0000 7f0d7eaf1640 1 -- 172.21.15.27:0/1275887400 >> 172.21.15.27:0/1275887400 conn(0x7f0d7806f300 msgr2=0x7f0d7806e140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:55.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:55.326+0000 7f0d7eaf1640 1 -- 172.21.15.27:0/1275887400 shutdown_connections 2024-09-12T03:21:55.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:55.326+0000 7f0d7eaf1640 1 -- 172.21.15.27:0/1275887400 wait complete. 2024-09-12T03:21:55.337 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":98,"stamp":"2024-09-12T03:21:55.116550+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":224,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":167,"ondisk_log_size":167,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":278496,"kb_used_data":12328,"kb_used_omap":12,"kb_used_meta":265971,"kb_avail":754696224,"statfs":{"total":773094113280,"available":772808933376,"internally_reserved":0,"allocated":12623872,"data_stored":6204380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":272354864},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":22,"apply_latency_ms":22,"commit_latency_ns":22000000,"apply_latency_ns":22000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001795"},"pg_stats":[{"pgid":"19.3f","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.266518+0000","last_change":"2024-09-12T03:21:47.276464+0000","last_active":"2024-09-12T03:21:48.266518+0000","last_peered":"2024-09-12T03:21:48.266518+0000","last_clean":"2024-09-12T03:21:48.266518+0000","last_became_active":"2024-09-12T03:21:47.276166+0000","last_became_peered":"2024-09-12T03:21:47.276166+0000","last_unstale":"2024-09-12T03:21:48.266518+0000","last_undegraded":"2024-09-12T03:21:48.266518+0000","last_fullsized":"2024-09-12T03:21:48.266518+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:28:18.796133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3e","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.724634+0000","last_change":"2024-09-12T03:21:47.279512+0000","last_active":"2024-09-12T03:21:48.724634+0000","last_peered":"2024-09-12T03:21:48.724634+0000","last_clean":"2024-09-12T03:21:48.724634+0000","last_became_active":"2024-09-12T03:21:47.279237+0000","last_became_peered":"2024-09-12T03:21:47.279237+0000","last_unstale":"2024-09-12T03:21:48.724634+0000","last_undegraded":"2024-09-12T03:21:48.724634+0000","last_fullsized":"2024-09-12T03:21:48.724634+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:21:31.235603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3d","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.598938+0000","last_change":"2024-09-12T03:21:47.276803+0000","last_active":"2024-09-12T03:21:48.598938+0000","last_peered":"2024-09-12T03:21:48.598938+0000","last_clean":"2024-09-12T03:21:48.598938+0000","last_became_active":"2024-09-12T03:21:47.276525+0000","last_became_peered":"2024-09-12T03:21:47.276525+0000","last_unstale":"2024-09-12T03:21:48.598938+0000","last_undegraded":"2024-09-12T03:21:48.598938+0000","last_fullsized":"2024-09-12T03:21:48.598938+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:27:48.066859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.3c","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.266405+0000","last_change":"2024-09-12T03:21:47.277021+0000","last_active":"2024-09-12T03:21:48.266405+0000","last_peered":"2024-09-12T03:21:48.266405+0000","last_clean":"2024-09-12T03:21:48.266405+0000","last_became_active":"2024-09-12T03:21:47.275542+0000","last_became_peered":"2024-09-12T03:21:47.275542+0000","last_unstale":"2024-09-12T03:21:48.266405+0000","last_undegraded":"2024-09-12T03:21:48.266405+0000","last_fullsized":"2024-09-12T03:21:48.266405+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:40:16.335013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3b","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.273675+0000","last_change":"2024-09-12T03:21:47.274034+0000","last_active":"2024-09-12T03:21:48.273675+0000","last_peered":"2024-09-12T03:21:48.273675+0000","last_clean":"2024-09-12T03:21:48.273675+0000","last_became_active":"2024-09-12T03:21:47.273718+0000","last_became_peered":"2024-09-12T03:21:47.273718+0000","last_unstale":"2024-09-12T03:21:48.273675+0000","last_undegraded":"2024-09-12T03:21:48.273675+0000","last_fullsized":"2024-09-12T03:21:48.273675+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:07:27.386745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.3a","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.267033+0000","last_change":"2024-09-12T03:21:47.279653+0000","last_active":"2024-09-12T03:21:48.267033+0000","last_peered":"2024-09-12T03:21:48.267033+0000","last_clean":"2024-09-12T03:21:48.267033+0000","last_became_active":"2024-09-12T03:21:47.279340+0000","last_became_peered":"2024-09-12T03:21:47.279340+0000","last_unstale":"2024-09-12T03:21:48.267033+0000","last_undegraded":"2024-09-12T03:21:48.267033+0000","last_fullsized":"2024-09-12T03:21:48.267033+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:07:12.256007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.39","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.266736+0000","last_change":"2024-09-12T03:21:47.272385+0000","last_active":"2024-09-12T03:21:48.266736+0000","last_peered":"2024-09-12T03:21:48.266736+0000","last_clean":"2024-09-12T03:21:48.266736+0000","last_became_active":"2024-09-12T03:21:47.272056+0000","last_became_peered":"2024-09-12T03:21:47.272056+0000","last_unstale":"2024-09-12T03:21:48.266736+0000","last_undegraded":"2024-09-12T03:21:48.266736+0000","last_fullsized":"2024-09-12T03:21:48.266736+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:14:24.567859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.38","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.598920+0000","last_change":"2024-09-12T03:21:47.280231+0000","last_active":"2024-09-12T03:21:48.598920+0000","last_peered":"2024-09-12T03:21:48.598920+0000","last_clean":"2024-09-12T03:21:48.598920+0000","last_became_active":"2024-09-12T03:21:47.279961+0000","last_became_peered":"2024-09-12T03:21:47.279961+0000","last_unstale":"2024-09-12T03:21:48.598920+0000","last_undegraded":"2024-09-12T03:21:48.598920+0000","last_fullsized":"2024-09-12T03:21:48.598920+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:35:57.506654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.37","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.599097+0000","last_change":"2024-09-12T03:21:47.274676+0000","last_active":"2024-09-12T03:21:48.599097+0000","last_peered":"2024-09-12T03:21:48.599097+0000","last_clean":"2024-09-12T03:21:48.599097+0000","last_became_active":"2024-09-12T03:21:47.274394+0000","last_became_peered":"2024-09-12T03:21:47.274394+0000","last_unstale":"2024-09-12T03:21:48.599097+0000","last_undegraded":"2024-09-12T03:21:48.599097+0000","last_fullsized":"2024-09-12T03:21:48.599097+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:33:46.210575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.36","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.724767+0000","last_change":"2024-09-12T03:21:47.271566+0000","last_active":"2024-09-12T03:21:48.724767+0000","last_peered":"2024-09-12T03:21:48.724767+0000","last_clean":"2024-09-12T03:21:48.724767+0000","last_became_active":"2024-09-12T03:21:47.271260+0000","last_became_peered":"2024-09-12T03:21:47.271260+0000","last_unstale":"2024-09-12T03:21:48.724767+0000","last_undegraded":"2024-09-12T03:21:48.724767+0000","last_fullsized":"2024-09-12T03:21:48.724767+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:39:28.014015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.35","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.725510+0000","last_change":"2024-09-12T03:21:47.286426+0000","last_active":"2024-09-12T03:21:48.725510+0000","last_peered":"2024-09-12T03:21:48.725510+0000","last_clean":"2024-09-12T03:21:48.725510+0000","last_became_active":"2024-09-12T03:21:47.285471+0000","last_became_peered":"2024-09-12T03:21:47.285471+0000","last_unstale":"2024-09-12T03:21:48.725510+0000","last_undegraded":"2024-09-12T03:21:48.725510+0000","last_fullsized":"2024-09-12T03:21:48.725510+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:23:56.306775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.34","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.598790+0000","last_change":"2024-09-12T03:21:47.272716+0000","last_active":"2024-09-12T03:21:48.598790+0000","last_peered":"2024-09-12T03:21:48.598790+0000","last_clean":"2024-09-12T03:21:48.598790+0000","last_became_active":"2024-09-12T03:21:47.269444+0000","last_became_peered":"2024-09-12T03:21:47.269444+0000","last_unstale":"2024-09-12T03:21:48.598790+0000","last_undegraded":"2024-09-12T03:21:48.598790+0000","last_fullsized":"2024-09-12T03:21:48.598790+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:29:24.947639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.33","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.266441+0000","last_change":"2024-09-12T03:21:47.286251+0000","last_active":"2024-09-12T03:21:48.266441+0000","last_peered":"2024-09-12T03:21:48.266441+0000","last_clean":"2024-09-12T03:21:48.266441+0000","last_became_active":"2024-09-12T03:21:47.285959+0000","last_became_peered":"2024-09-12T03:21:47.285959+0000","last_unstale":"2024-09-12T03:21:48.266441+0000","last_undegraded":"2024-09-12T03:21:48.266441+0000","last_fullsized":"2024-09-12T03:21:48.266441+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:19:41.675227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.32","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.724693+0000","last_change":"2024-09-12T03:21:47.280594+0000","last_active":"2024-09-12T03:21:48.724693+0000","last_peered":"2024-09-12T03:21:48.724693+0000","last_clean":"2024-09-12T03:21:48.724693+0000","last_became_active":"2024-09-12T03:21:47.280250+0000","last_became_peered":"2024-09-12T03:21:47.280250+0000","last_unstale":"2024-09-12T03:21:48.724693+0000","last_undegraded":"2024-09-12T03:21:48.724693+0000","last_fullsized":"2024-09-12T03:21:48.724693+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:43:43.310362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.31","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.266159+0000","last_change":"2024-09-12T03:21:47.278494+0000","last_active":"2024-09-12T03:21:48.266159+0000","last_peered":"2024-09-12T03:21:48.266159+0000","last_clean":"2024-09-12T03:21:48.266159+0000","last_became_active":"2024-09-12T03:21:47.278228+0000","last_became_peered":"2024-09-12T03:21:47.278228+0000","last_unstale":"2024-09-12T03:21:48.266159+0000","last_undegraded":"2024-09-12T03:21:48.266159+0000","last_fullsized":"2024-09-12T03:21:48.266159+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:06:31.812980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.30","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.725675+0000","last_change":"2024-09-12T03:21:47.274919+0000","last_active":"2024-09-12T03:21:48.725675+0000","last_peered":"2024-09-12T03:21:48.725675+0000","last_clean":"2024-09-12T03:21:48.725675+0000","last_became_active":"2024-09-12T03:21:47.274611+0000","last_became_peered":"2024-09-12T03:21:47.274611+0000","last_unstale":"2024-09-12T03:21:48.725675+0000","last_undegraded":"2024-09-12T03:21:48.725675+0000","last_fullsized":"2024-09-12T03:21:48.725675+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:21:08.369709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2f","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.267173+0000","last_change":"2024-09-12T03:21:47.278293+0000","last_active":"2024-09-12T03:21:48.267173+0000","last_peered":"2024-09-12T03:21:48.267173+0000","last_clean":"2024-09-12T03:21:48.267173+0000","last_became_active":"2024-09-12T03:21:47.277990+0000","last_became_peered":"2024-09-12T03:21:47.277990+0000","last_unstale":"2024-09-12T03:21:48.267173+0000","last_undegraded":"2024-09-12T03:21:48.267173+0000","last_fullsized":"2024-09-12T03:21:48.267173+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:32:48.364609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.2e","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.724475+0000","last_change":"2024-09-12T03:21:47.267120+0000","last_active":"2024-09-12T03:21:48.724475+0000","last_peered":"2024-09-12T03:21:48.724475+0000","last_clean":"2024-09-12T03:21:48.724475+0000","last_became_active":"2024-09-12T03:21:47.266838+0000","last_became_peered":"2024-09-12T03:21:47.266838+0000","last_unstale":"2024-09-12T03:21:48.724475+0000","last_undegraded":"2024-09-12T03:21:48.724475+0000","last_fullsized":"2024-09-12T03:21:48.724475+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:20:42.594004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2d","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.725652+0000","last_change":"2024-09-12T03:21:47.281390+0000","last_active":"2024-09-12T03:21:48.725652+0000","last_peered":"2024-09-12T03:21:48.725652+0000","last_clean":"2024-09-12T03:21:48.725652+0000","last_became_active":"2024-09-12T03:21:47.281102+0000","last_became_peered":"2024-09-12T03:21:47.281102+0000","last_unstale":"2024-09-12T03:21:48.725652+0000","last_undegraded":"2024-09-12T03:21:48.725652+0000","last_fullsized":"2024-09-12T03:21:48.725652+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:49:41.579264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2c","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.724874+0000","last_change":"2024-09-12T03:21:47.271524+0000","last_active":"2024-09-12T03:21:48.724874+0000","last_peered":"2024-09-12T03:21:48.724874+0000","last_clean":"2024-09-12T03:21:48.724874+0000","last_became_active":"2024-09-12T03:21:47.271217+0000","last_became_peered":"2024-09-12T03:21:47.271217+0000","last_unstale":"2024-09-12T03:21:48.724874+0000","last_undegraded":"2024-09-12T03:21:48.724874+0000","last_fullsized":"2024-09-12T03:21:48.724874+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:08:22.543376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2b","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.273593+0000","last_change":"2024-09-12T03:21:47.274734+0000","last_active":"2024-09-12T03:21:48.273593+0000","last_peered":"2024-09-12T03:21:48.273593+0000","last_clean":"2024-09-12T03:21:48.273593+0000","last_became_active":"2024-09-12T03:21:47.274409+0000","last_became_peered":"2024-09-12T03:21:47.274409+0000","last_unstale":"2024-09-12T03:21:48.273593+0000","last_undegraded":"2024-09-12T03:21:48.273593+0000","last_fullsized":"2024-09-12T03:21:48.273593+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:00:08.281678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2a","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.724647+0000","last_change":"2024-09-12T03:21:47.285810+0000","last_active":"2024-09-12T03:21:48.724647+0000","last_peered":"2024-09-12T03:21:48.724647+0000","last_clean":"2024-09-12T03:21:48.724647+0000","last_became_active":"2024-09-12T03:21:47.285464+0000","last_became_peered":"2024-09-12T03:21:47.285464+0000","last_unstale":"2024-09-12T03:21:48.724647+0000","last_undegraded":"2024-09-12T03:21:48.724647+0000","last_fullsized":"2024-09-12T03:21:48.724647+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:27:08.761287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.29","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.598960+0000","last_change":"2024-09-12T03:21:47.274207+0000","last_active":"2024-09-12T03:21:48.598960+0000","last_peered":"2024-09-12T03:21:48.598960+0000","last_clean":"2024-09-12T03:21:48.598960+0000","last_became_active":"2024-09-12T03:21:47.273730+0000","last_became_peered":"2024-09-12T03:21:47.273730+0000","last_unstale":"2024-09-12T03:21:48.598960+0000","last_undegraded":"2024-09-12T03:21:48.598960+0000","last_fullsized":"2024-09-12T03:21:48.598960+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:24:38.533228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.28","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.724965+0000","last_change":"2024-09-12T03:21:47.275142+0000","last_active":"2024-09-12T03:21:48.724965+0000","last_peered":"2024-09-12T03:21:48.724965+0000","last_clean":"2024-09-12T03:21:48.724965+0000","last_became_active":"2024-09-12T03:21:47.274110+0000","last_became_peered":"2024-09-12T03:21:47.274110+0000","last_unstale":"2024-09-12T03:21:48.724965+0000","last_undegraded":"2024-09-12T03:21:48.724965+0000","last_fullsized":"2024-09-12T03:21:48.724965+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:37:40.004179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.27","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.724485+0000","last_change":"2024-09-12T03:21:47.269287+0000","last_active":"2024-09-12T03:21:48.724485+0000","last_peered":"2024-09-12T03:21:48.724485+0000","last_clean":"2024-09-12T03:21:48.724485+0000","last_became_active":"2024-09-12T03:21:47.268980+0000","last_became_peered":"2024-09-12T03:21:47.268980+0000","last_unstale":"2024-09-12T03:21:48.724485+0000","last_undegraded":"2024-09-12T03:21:48.724485+0000","last_fullsized":"2024-09-12T03:21:48.724485+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:05:34.381423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.26","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.724624+0000","last_change":"2024-09-12T03:21:47.278973+0000","last_active":"2024-09-12T03:21:48.724624+0000","last_peered":"2024-09-12T03:21:48.724624+0000","last_clean":"2024-09-12T03:21:48.724624+0000","last_became_active":"2024-09-12T03:21:47.278643+0000","last_became_peered":"2024-09-12T03:21:47.278643+0000","last_unstale":"2024-09-12T03:21:48.724624+0000","last_undegraded":"2024-09-12T03:21:48.724624+0000","last_fullsized":"2024-09-12T03:21:48.724624+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:49:07.547396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.25","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.266439+0000","last_change":"2024-09-12T03:21:47.279216+0000","last_active":"2024-09-12T03:21:48.266439+0000","last_peered":"2024-09-12T03:21:48.266439+0000","last_clean":"2024-09-12T03:21:48.266439+0000","last_became_active":"2024-09-12T03:21:47.278935+0000","last_became_peered":"2024-09-12T03:21:47.278935+0000","last_unstale":"2024-09-12T03:21:48.266439+0000","last_undegraded":"2024-09-12T03:21:48.266439+0000","last_fullsized":"2024-09-12T03:21:48.266439+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:57:37.592178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.24","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.267673+0000","last_change":"2024-09-12T03:21:47.278666+0000","last_active":"2024-09-12T03:21:48.267673+0000","last_peered":"2024-09-12T03:21:48.267673+0000","last_clean":"2024-09-12T03:21:48.267673+0000","last_became_active":"2024-09-12T03:21:47.278307+0000","last_became_peered":"2024-09-12T03:21:47.278307+0000","last_unstale":"2024-09-12T03:21:48.267673+0000","last_undegraded":"2024-09-12T03:21:48.267673+0000","last_fullsized":"2024-09-12T03:21:48.267673+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:39:28.014015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.23","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.266509+0000","last_change":"2024-09-12T03:21:47.281506+0000","last_active":"2024-09-12T03:21:48.266509+0000","last_peered":"2024-09-12T03:21:48.266509+0000","last_clean":"2024-09-12T03:21:48.266509+0000","last_became_active":"2024-09-12T03:21:47.281234+0000","last_became_peered":"2024-09-12T03:21:47.281234+0000","last_unstale":"2024-09-12T03:21:48.266509+0000","last_undegraded":"2024-09-12T03:21:48.266509+0000","last_fullsized":"2024-09-12T03:21:48.266509+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:15:33.521405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.22","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.266213+0000","last_change":"2024-09-12T03:21:47.277446+0000","last_active":"2024-09-12T03:21:48.266213+0000","last_peered":"2024-09-12T03:21:48.266213+0000","last_clean":"2024-09-12T03:21:48.266213+0000","last_became_active":"2024-09-12T03:21:47.276925+0000","last_became_peered":"2024-09-12T03:21:47.276925+0000","last_unstale":"2024-09-12T03:21:48.266213+0000","last_undegraded":"2024-09-12T03:21:48.266213+0000","last_fullsized":"2024-09-12T03:21:48.266213+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:29:38.307737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.21","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.273547+0000","last_change":"2024-09-12T03:21:47.283522+0000","last_active":"2024-09-12T03:21:48.273547+0000","last_peered":"2024-09-12T03:21:48.273547+0000","last_clean":"2024-09-12T03:21:48.273547+0000","last_became_active":"2024-09-12T03:21:47.283218+0000","last_became_peered":"2024-09-12T03:21:47.283218+0000","last_unstale":"2024-09-12T03:21:48.273547+0000","last_undegraded":"2024-09-12T03:21:48.273547+0000","last_fullsized":"2024-09-12T03:21:48.273547+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:07:41.072755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.20","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.266742+0000","last_change":"2024-09-12T03:21:47.280790+0000","last_active":"2024-09-12T03:21:48.266742+0000","last_peered":"2024-09-12T03:21:48.266742+0000","last_clean":"2024-09-12T03:21:48.266742+0000","last_became_active":"2024-09-12T03:21:47.280508+0000","last_became_peered":"2024-09-12T03:21:47.280508+0000","last_unstale":"2024-09-12T03:21:48.266742+0000","last_undegraded":"2024-09-12T03:21:48.266742+0000","last_fullsized":"2024-09-12T03:21:48.266742+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:38:27.802587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1e","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.724761+0000","last_change":"2024-09-12T03:21:47.274189+0000","last_active":"2024-09-12T03:21:48.724761+0000","last_peered":"2024-09-12T03:21:48.724761+0000","last_clean":"2024-09-12T03:21:48.724761+0000","last_became_active":"2024-09-12T03:21:47.273852+0000","last_became_peered":"2024-09-12T03:21:47.273852+0000","last_unstale":"2024-09-12T03:21:48.724761+0000","last_undegraded":"2024-09-12T03:21:48.724761+0000","last_fullsized":"2024-09-12T03:21:48.724761+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:42:44.733030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1f","version":"113'4","reported_seq":18,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.273969+0000","last_change":"2024-09-12T03:21:46.265345+0000","last_active":"2024-09-12T03:21:48.273969+0000","last_peered":"2024-09-12T03:21:48.273969+0000","last_clean":"2024-09-12T03:21:48.273969+0000","last_became_active":"2024-09-12T03:21:46.265014+0000","last_became_peered":"2024-09-12T03:21:46.265014+0000","last_unstale":"2024-09-12T03:21:48.273969+0000","last_undegraded":"2024-09-12T03:21:48.273969+0000","last_fullsized":"2024-09-12T03:21:48.273969+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_clean_scrub_stamp":"2024-09-12T03:21:45.231023+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:16:44.125918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.1f","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.599128+0000","last_change":"2024-09-12T03:21:47.278091+0000","last_active":"2024-09-12T03:21:48.599128+0000","last_peered":"2024-09-12T03:21:48.599128+0000","last_clean":"2024-09-12T03:21:48.599128+0000","last_became_active":"2024-09-12T03:21:47.277783+0000","last_became_peered":"2024-09-12T03:21:47.277783+0000","last_unstale":"2024-09-12T03:21:48.599128+0000","last_undegraded":"2024-09-12T03:21:48.599128+0000","last_fullsized":"2024-09-12T03:21:48.599128+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:42:53.625195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1e","version":"113'1","reported_seq":15,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.267990+0000","last_change":"2024-09-12T03:21:46.251686+0000","last_active":"2024-09-12T03:21:48.267990+0000","last_peered":"2024-09-12T03:21:48.267990+0000","last_clean":"2024-09-12T03:21:48.267990+0000","last_became_active":"2024-09-12T03:21:46.251055+0000","last_became_peered":"2024-09-12T03:21:46.251055+0000","last_unstale":"2024-09-12T03:21:48.267990+0000","last_undegraded":"2024-09-12T03:21:48.267990+0000","last_fullsized":"2024-09-12T03:21:48.267990+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_clean_scrub_stamp":"2024-09-12T03:21:45.231023+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:01:01.329193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.273734+0000","last_change":"2024-09-12T03:21:47.275088+0000","last_active":"2024-09-12T03:21:48.273734+0000","last_peered":"2024-09-12T03:21:48.273734+0000","last_clean":"2024-09-12T03:21:48.273734+0000","last_became_active":"2024-09-12T03:21:47.274745+0000","last_became_peered":"2024-09-12T03:21:47.274745+0000","last_unstale":"2024-09-12T03:21:48.273734+0000","last_undegraded":"2024-09-12T03:21:48.273734+0000","last_fullsized":"2024-09-12T03:21:48.273734+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:49:39.489484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1d","version":"113'2","reported_seq":16,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.725704+0000","last_change":"2024-09-12T03:21:46.258040+0000","last_active":"2024-09-12T03:21:48.725704+0000","last_peered":"2024-09-12T03:21:48.725704+0000","last_clean":"2024-09-12T03:21:48.725704+0000","last_became_active":"2024-09-12T03:21:46.257747+0000","last_became_peered":"2024-09-12T03:21:46.257747+0000","last_unstale":"2024-09-12T03:21:48.725704+0000","last_undegraded":"2024-09-12T03:21:48.725704+0000","last_fullsized":"2024-09-12T03:21:48.725704+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_clean_scrub_stamp":"2024-09-12T03:21:45.231023+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:07:24.863710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.1d","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.273966+0000","last_change":"2024-09-12T03:21:47.273642+0000","last_active":"2024-09-12T03:21:48.273966+0000","last_peered":"2024-09-12T03:21:48.273966+0000","last_clean":"2024-09-12T03:21:48.273966+0000","last_became_active":"2024-09-12T03:21:47.272464+0000","last_became_peered":"2024-09-12T03:21:47.272464+0000","last_unstale":"2024-09-12T03:21:48.273966+0000","last_undegraded":"2024-09-12T03:21:48.273966+0000","last_fullsized":"2024-09-12T03:21:48.273966+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:55:33.036487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1c","version":"113'2","reported_seq":16,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.725395+0000","last_change":"2024-09-12T03:21:46.266144+0000","last_active":"2024-09-12T03:21:48.725395+0000","last_peered":"2024-09-12T03:21:48.725395+0000","last_clean":"2024-09-12T03:21:48.725395+0000","last_became_active":"2024-09-12T03:21:46.265841+0000","last_became_peered":"2024-09-12T03:21:46.265841+0000","last_unstale":"2024-09-12T03:21:48.725395+0000","last_undegraded":"2024-09-12T03:21:48.725395+0000","last_fullsized":"2024-09-12T03:21:48.725395+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_clean_scrub_stamp":"2024-09-12T03:21:45.231023+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:46:45.472576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.725331+0000","last_change":"2024-09-12T03:21:47.263438+0000","last_active":"2024-09-12T03:21:48.725331+0000","last_peered":"2024-09-12T03:21:48.725331+0000","last_clean":"2024-09-12T03:21:48.725331+0000","last_became_active":"2024-09-12T03:21:47.263100+0000","last_became_peered":"2024-09-12T03:21:47.263100+0000","last_unstale":"2024-09-12T03:21:48.725331+0000","last_undegraded":"2024-09-12T03:21:48.725331+0000","last_fullsized":"2024-09-12T03:21:48.725331+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:45:51.865826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1b","version":"113'1","reported_seq":15,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.724274+0000","last_change":"2024-09-12T03:21:46.264671+0000","last_active":"2024-09-12T03:21:48.724274+0000","last_peered":"2024-09-12T03:21:48.724274+0000","last_clean":"2024-09-12T03:21:48.724274+0000","last_became_active":"2024-09-12T03:21:46.264133+0000","last_became_peered":"2024-09-12T03:21:46.264133+0000","last_unstale":"2024-09-12T03:21:48.724274+0000","last_undegraded":"2024-09-12T03:21:48.724274+0000","last_fullsized":"2024-09-12T03:21:48.724274+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_clean_scrub_stamp":"2024-09-12T03:21:45.231023+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:16:07.856351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1b","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.267682+0000","last_change":"2024-09-12T03:21:47.277615+0000","last_active":"2024-09-12T03:21:48.267682+0000","last_peered":"2024-09-12T03:21:48.267682+0000","last_clean":"2024-09-12T03:21:48.267682+0000","last_became_active":"2024-09-12T03:21:47.277321+0000","last_became_peered":"2024-09-12T03:21:47.277321+0000","last_unstale":"2024-09-12T03:21:48.267682+0000","last_undegraded":"2024-09-12T03:21:48.267682+0000","last_fullsized":"2024-09-12T03:21:48.267682+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:20:42.594004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1a","version":"113'1","reported_seq":15,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.725273+0000","last_change":"2024-09-12T03:21:46.263053+0000","last_active":"2024-09-12T03:21:48.725273+0000","last_peered":"2024-09-12T03:21:48.725273+0000","last_clean":"2024-09-12T03:21:48.725273+0000","last_became_active":"2024-09-12T03:21:46.262771+0000","last_became_peered":"2024-09-12T03:21:46.262771+0000","last_unstale":"2024-09-12T03:21:48.725273+0000","last_undegraded":"2024-09-12T03:21:48.725273+0000","last_fullsized":"2024-09-12T03:21:48.725273+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_clean_scrub_stamp":"2024-09-12T03:21:45.231023+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:35:56.500559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.599066+0000","last_change":"2024-09-12T03:21:47.281084+0000","last_active":"2024-09-12T03:21:48.599066+0000","last_peered":"2024-09-12T03:21:48.599066+0000","last_clean":"2024-09-12T03:21:48.599066+0000","last_became_active":"2024-09-12T03:21:47.280767+0000","last_became_peered":"2024-09-12T03:21:47.280767+0000","last_unstale":"2024-09-12T03:21:48.599066+0000","last_undegraded":"2024-09-12T03:21:48.599066+0000","last_fullsized":"2024-09-12T03:21:48.599066+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:16:45.132013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":14,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.266696+0000","last_change":"2024-09-12T03:21:46.253748+0000","last_active":"2024-09-12T03:21:48.266696+0000","last_peered":"2024-09-12T03:21:48.266696+0000","last_clean":"2024-09-12T03:21:48.266696+0000","last_became_active":"2024-09-12T03:21:46.253381+0000","last_became_peered":"2024-09-12T03:21:46.253381+0000","last_unstale":"2024-09-12T03:21:48.266696+0000","last_undegraded":"2024-09-12T03:21:48.266696+0000","last_fullsized":"2024-09-12T03:21:48.266696+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_clean_scrub_stamp":"2024-09-12T03:21:45.231023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:33:59.776477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.724917+0000","last_change":"2024-09-12T03:21:47.278433+0000","last_active":"2024-09-12T03:21:48.724917+0000","last_peered":"2024-09-12T03:21:48.724917+0000","last_clean":"2024-09-12T03:21:48.724917+0000","last_became_active":"2024-09-12T03:21:47.278146+0000","last_became_peered":"2024-09-12T03:21:47.278146+0000","last_unstale":"2024-09-12T03:21:48.724917+0000","last_undegraded":"2024-09-12T03:21:48.724917+0000","last_fullsized":"2024-09-12T03:21:48.724917+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:02:37.440594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":14,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.724426+0000","last_change":"2024-09-12T03:21:46.264728+0000","last_active":"2024-09-12T03:21:48.724426+0000","last_peered":"2024-09-12T03:21:48.724426+0000","last_clean":"2024-09-12T03:21:48.724426+0000","last_became_active":"2024-09-12T03:21:46.263798+0000","last_became_peered":"2024-09-12T03:21:46.263798+0000","last_unstale":"2024-09-12T03:21:48.724426+0000","last_undegraded":"2024-09-12T03:21:48.724426+0000","last_fullsized":"2024-09-12T03:21:48.724426+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_clean_scrub_stamp":"2024-09-12T03:21:45.231023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:12:16.266409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.266463+0000","last_change":"2024-09-12T03:21:47.277867+0000","last_active":"2024-09-12T03:21:48.266463+0000","last_peered":"2024-09-12T03:21:48.266463+0000","last_clean":"2024-09-12T03:21:48.266463+0000","last_became_active":"2024-09-12T03:21:47.277538+0000","last_became_peered":"2024-09-12T03:21:47.277538+0000","last_unstale":"2024-09-12T03:21:48.266463+0000","last_undegraded":"2024-09-12T03:21:48.266463+0000","last_fullsized":"2024-09-12T03:21:48.266463+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:33:42.064998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":14,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.268143+0000","last_change":"2024-09-12T03:21:46.257496+0000","last_active":"2024-09-12T03:21:48.268143+0000","last_peered":"2024-09-12T03:21:48.268143+0000","last_clean":"2024-09-12T03:21:48.268143+0000","last_became_active":"2024-09-12T03:21:46.257152+0000","last_became_peered":"2024-09-12T03:21:46.257152+0000","last_unstale":"2024-09-12T03:21:48.268143+0000","last_undegraded":"2024-09-12T03:21:48.268143+0000","last_fullsized":"2024-09-12T03:21:48.268143+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_clean_scrub_stamp":"2024-09-12T03:21:45.231023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:29:42.420816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.268097+0000","last_change":"2024-09-12T03:21:47.277451+0000","last_active":"2024-09-12T03:21:48.268097+0000","last_peered":"2024-09-12T03:21:48.268097+0000","last_clean":"2024-09-12T03:21:48.268097+0000","last_became_active":"2024-09-12T03:21:47.277147+0000","last_became_peered":"2024-09-12T03:21:47.277147+0000","last_unstale":"2024-09-12T03:21:48.268097+0000","last_undegraded":"2024-09-12T03:21:48.268097+0000","last_fullsized":"2024-09-12T03:21:48.268097+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:02:37.440594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":14,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.724804+0000","last_change":"2024-09-12T03:21:46.254525+0000","last_active":"2024-09-12T03:21:48.724804+0000","last_peered":"2024-09-12T03:21:48.724804+0000","last_clean":"2024-09-12T03:21:48.724804+0000","last_became_active":"2024-09-12T03:21:46.254190+0000","last_became_peered":"2024-09-12T03:21:46.254190+0000","last_unstale":"2024-09-12T03:21:48.724804+0000","last_undegraded":"2024-09-12T03:21:48.724804+0000","last_fullsized":"2024-09-12T03:21:48.724804+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_clean_scrub_stamp":"2024-09-12T03:21:45.231023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:16:44.125918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.14","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.273888+0000","last_change":"2024-09-12T03:21:47.278622+0000","last_active":"2024-09-12T03:21:48.273888+0000","last_peered":"2024-09-12T03:21:48.273888+0000","last_clean":"2024-09-12T03:21:48.273888+0000","last_became_active":"2024-09-12T03:21:47.278333+0000","last_became_peered":"2024-09-12T03:21:47.278333+0000","last_unstale":"2024-09-12T03:21:48.273888+0000","last_undegraded":"2024-09-12T03:21:48.273888+0000","last_fullsized":"2024-09-12T03:21:48.273888+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:37:21.066404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.15","version":"113'3","reported_seq":17,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.266666+0000","last_change":"2024-09-12T03:21:46.252566+0000","last_active":"2024-09-12T03:21:48.266666+0000","last_peered":"2024-09-12T03:21:48.266666+0000","last_clean":"2024-09-12T03:21:48.266666+0000","last_became_active":"2024-09-12T03:21:46.252247+0000","last_became_peered":"2024-09-12T03:21:46.252247+0000","last_unstale":"2024-09-12T03:21:48.266666+0000","last_undegraded":"2024-09-12T03:21:48.266666+0000","last_fullsized":"2024-09-12T03:21:48.266666+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_clean_scrub_stamp":"2024-09-12T03:21:45.231023+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:09:06.932896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.15","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.274070+0000","last_change":"2024-09-12T03:21:47.271363+0000","last_active":"2024-09-12T03:21:48.274070+0000","last_peered":"2024-09-12T03:21:48.274070+0000","last_clean":"2024-09-12T03:21:48.274070+0000","last_became_active":"2024-09-12T03:21:47.271060+0000","last_became_peered":"2024-09-12T03:21:47.271060+0000","last_unstale":"2024-09-12T03:21:48.274070+0000","last_undegraded":"2024-09-12T03:21:48.274070+0000","last_fullsized":"2024-09-12T03:21:48.274070+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:26:02.679898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.14","version":"113'6","reported_seq":20,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.598854+0000","last_change":"2024-09-12T03:21:46.255120+0000","last_active":"2024-09-12T03:21:48.598854+0000","last_peered":"2024-09-12T03:21:48.598854+0000","last_clean":"2024-09-12T03:21:48.598854+0000","last_became_active":"2024-09-12T03:21:46.254812+0000","last_became_peered":"2024-09-12T03:21:46.254812+0000","last_unstale":"2024-09-12T03:21:48.598854+0000","last_undegraded":"2024-09-12T03:21:48.598854+0000","last_fullsized":"2024-09-12T03:21:48.598854+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_clean_scrub_stamp":"2024-09-12T03:21:45.231023+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:24:52.363839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.13","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.266605+0000","last_change":"2024-09-12T03:21:47.283523+0000","last_active":"2024-09-12T03:21:48.266605+0000","last_peered":"2024-09-12T03:21:48.266605+0000","last_clean":"2024-09-12T03:21:48.266605+0000","last_became_active":"2024-09-12T03:21:47.279638+0000","last_became_peered":"2024-09-12T03:21:47.279638+0000","last_unstale":"2024-09-12T03:21:48.266605+0000","last_undegraded":"2024-09-12T03:21:48.266605+0000","last_fullsized":"2024-09-12T03:21:48.266605+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:10:22.061518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":14,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.266298+0000","last_change":"2024-09-12T03:21:46.260129+0000","last_active":"2024-09-12T03:21:48.266298+0000","last_peered":"2024-09-12T03:21:48.266298+0000","last_clean":"2024-09-12T03:21:48.266298+0000","last_became_active":"2024-09-12T03:21:46.259758+0000","last_became_peered":"2024-09-12T03:21:46.259758+0000","last_unstale":"2024-09-12T03:21:48.266298+0000","last_undegraded":"2024-09-12T03:21:48.266298+0000","last_fullsized":"2024-09-12T03:21:48.266298+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_clean_scrub_stamp":"2024-09-12T03:21:45.231023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:39:56.115395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.10","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.724236+0000","last_change":"2024-09-12T03:21:47.278630+0000","last_active":"2024-09-12T03:21:48.724236+0000","last_peered":"2024-09-12T03:21:48.724236+0000","last_clean":"2024-09-12T03:21:48.724236+0000","last_became_active":"2024-09-12T03:21:47.278333+0000","last_became_peered":"2024-09-12T03:21:47.278333+0000","last_unstale":"2024-09-12T03:21:48.724236+0000","last_undegraded":"2024-09-12T03:21:48.724236+0000","last_fullsized":"2024-09-12T03:21:48.724236+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:31:24.916429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":14,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.725104+0000","last_change":"2024-09-12T03:21:46.263275+0000","last_active":"2024-09-12T03:21:48.725104+0000","last_peered":"2024-09-12T03:21:48.725104+0000","last_clean":"2024-09-12T03:21:48.725104+0000","last_became_active":"2024-09-12T03:21:46.262982+0000","last_became_peered":"2024-09-12T03:21:46.262982+0000","last_unstale":"2024-09-12T03:21:48.725104+0000","last_undegraded":"2024-09-12T03:21:48.725104+0000","last_fullsized":"2024-09-12T03:21:48.725104+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_clean_scrub_stamp":"2024-09-12T03:21:45.231023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:35:47.022417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.599034+0000","last_change":"2024-09-12T03:21:47.274722+0000","last_active":"2024-09-12T03:21:48.599034+0000","last_peered":"2024-09-12T03:21:48.599034+0000","last_clean":"2024-09-12T03:21:48.599034+0000","last_became_active":"2024-09-12T03:21:47.274319+0000","last_became_peered":"2024-09-12T03:21:47.274319+0000","last_unstale":"2024-09-12T03:21:48.599034+0000","last_undegraded":"2024-09-12T03:21:48.599034+0000","last_fullsized":"2024-09-12T03:21:48.599034+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:36:45.878775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.10","version":"113'8","reported_seq":22,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.599010+0000","last_change":"2024-09-12T03:21:46.254895+0000","last_active":"2024-09-12T03:21:48.599010+0000","last_peered":"2024-09-12T03:21:48.599010+0000","last_clean":"2024-09-12T03:21:48.599010+0000","last_became_active":"2024-09-12T03:21:46.254582+0000","last_became_peered":"2024-09-12T03:21:46.254582+0000","last_unstale":"2024-09-12T03:21:48.599010+0000","last_undegraded":"2024-09-12T03:21:48.599010+0000","last_fullsized":"2024-09-12T03:21:48.599010+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_clean_scrub_stamp":"2024-09-12T03:21:45.231023+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:40:18.756382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.e","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.598877+0000","last_change":"2024-09-12T03:21:47.279358+0000","last_active":"2024-09-12T03:21:48.598877+0000","last_peered":"2024-09-12T03:21:48.598877+0000","last_clean":"2024-09-12T03:21:48.598877+0000","last_became_active":"2024-09-12T03:21:47.279082+0000","last_became_peered":"2024-09-12T03:21:47.279082+0000","last_unstale":"2024-09-12T03:21:48.598877+0000","last_undegraded":"2024-09-12T03:21:48.598877+0000","last_fullsized":"2024-09-12T03:21:48.598877+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:36:02.708847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.f","version":"113'1","reported_seq":15,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.268246+0000","last_change":"2024-09-12T03:21:46.255444+0000","last_active":"2024-09-12T03:21:48.268246+0000","last_peered":"2024-09-12T03:21:48.268246+0000","last_clean":"2024-09-12T03:21:48.268246+0000","last_became_active":"2024-09-12T03:21:46.255111+0000","last_became_peered":"2024-09-12T03:21:46.255111+0000","last_unstale":"2024-09-12T03:21:48.268246+0000","last_undegraded":"2024-09-12T03:21:48.268246+0000","last_fullsized":"2024-09-12T03:21:48.268246+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_clean_scrub_stamp":"2024-09-12T03:21:45.231023+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:01:19.852084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.f","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.268199+0000","last_change":"2024-09-12T03:21:47.275553+0000","last_active":"2024-09-12T03:21:48.268199+0000","last_peered":"2024-09-12T03:21:48.268199+0000","last_clean":"2024-09-12T03:21:48.268199+0000","last_became_active":"2024-09-12T03:21:47.275232+0000","last_became_peered":"2024-09-12T03:21:47.275232+0000","last_unstale":"2024-09-12T03:21:48.268199+0000","last_undegraded":"2024-09-12T03:21:48.268199+0000","last_fullsized":"2024-09-12T03:21:48.268199+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:42:51.860983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.e","version":"113'2","reported_seq":16,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.725806+0000","last_change":"2024-09-12T03:21:46.253780+0000","last_active":"2024-09-12T03:21:48.725806+0000","last_peered":"2024-09-12T03:21:48.725806+0000","last_clean":"2024-09-12T03:21:48.725806+0000","last_became_active":"2024-09-12T03:21:46.253480+0000","last_became_peered":"2024-09-12T03:21:46.253480+0000","last_unstale":"2024-09-12T03:21:48.725806+0000","last_undegraded":"2024-09-12T03:21:48.725806+0000","last_fullsized":"2024-09-12T03:21:48.725806+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_clean_scrub_stamp":"2024-09-12T03:21:45.231023+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:38:27.071851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.725011+0000","last_change":"2024-09-12T03:21:47.279125+0000","last_active":"2024-09-12T03:21:48.725011+0000","last_peered":"2024-09-12T03:21:48.725011+0000","last_clean":"2024-09-12T03:21:48.725011+0000","last_became_active":"2024-09-12T03:21:47.278602+0000","last_became_peered":"2024-09-12T03:21:47.278602+0000","last_unstale":"2024-09-12T03:21:48.725011+0000","last_undegraded":"2024-09-12T03:21:48.725011+0000","last_fullsized":"2024-09-12T03:21:48.725011+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:07:41.072755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.d","version":"113'1","reported_seq":15,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.267378+0000","last_change":"2024-09-12T03:21:46.251440+0000","last_active":"2024-09-12T03:21:48.267378+0000","last_peered":"2024-09-12T03:21:48.267378+0000","last_clean":"2024-09-12T03:21:48.267378+0000","last_became_active":"2024-09-12T03:21:46.251107+0000","last_became_peered":"2024-09-12T03:21:46.251107+0000","last_unstale":"2024-09-12T03:21:48.267378+0000","last_undegraded":"2024-09-12T03:21:48.267378+0000","last_fullsized":"2024-09-12T03:21:48.267378+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_clean_scrub_stamp":"2024-09-12T03:21:45.231023+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:29:23.941544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.725059+0000","last_change":"2024-09-12T03:21:47.278687+0000","last_active":"2024-09-12T03:21:48.725059+0000","last_peered":"2024-09-12T03:21:48.725059+0000","last_clean":"2024-09-12T03:21:48.725059+0000","last_became_active":"2024-09-12T03:21:47.277859+0000","last_became_peered":"2024-09-12T03:21:47.277859+0000","last_unstale":"2024-09-12T03:21:48.725059+0000","last_undegraded":"2024-09-12T03:21:48.725059+0000","last_fullsized":"2024-09-12T03:21:48.725059+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:17:45.434501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":14,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.267750+0000","last_change":"2024-09-12T03:21:46.249240+0000","last_active":"2024-09-12T03:21:48.267750+0000","last_peered":"2024-09-12T03:21:48.267750+0000","last_clean":"2024-09-12T03:21:48.267750+0000","last_became_active":"2024-09-12T03:21:46.248701+0000","last_became_peered":"2024-09-12T03:21:46.248701+0000","last_unstale":"2024-09-12T03:21:48.267750+0000","last_undegraded":"2024-09-12T03:21:48.267750+0000","last_fullsized":"2024-09-12T03:21:48.267750+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_clean_scrub_stamp":"2024-09-12T03:21:45.231023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:50:01.855914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.266214+0000","last_change":"2024-09-12T03:21:47.282909+0000","last_active":"2024-09-12T03:21:48.266214+0000","last_peered":"2024-09-12T03:21:48.266214+0000","last_clean":"2024-09-12T03:21:48.266214+0000","last_became_active":"2024-09-12T03:21:47.282527+0000","last_became_peered":"2024-09-12T03:21:47.282527+0000","last_unstale":"2024-09-12T03:21:48.266214+0000","last_undegraded":"2024-09-12T03:21:48.266214+0000","last_fullsized":"2024-09-12T03:21:48.266214+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:48:16.365941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.b","version":"113'2","reported_seq":16,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.267395+0000","last_change":"2024-09-12T03:21:46.262019+0000","last_active":"2024-09-12T03:21:48.267395+0000","last_peered":"2024-09-12T03:21:48.267395+0000","last_clean":"2024-09-12T03:21:48.267395+0000","last_became_active":"2024-09-12T03:21:46.261674+0000","last_became_peered":"2024-09-12T03:21:46.261674+0000","last_unstale":"2024-09-12T03:21:48.267395+0000","last_undegraded":"2024-09-12T03:21:48.267395+0000","last_fullsized":"2024-09-12T03:21:48.267395+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_clean_scrub_stamp":"2024-09-12T03:21:45.231023+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:16:38.849993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.b","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.267739+0000","last_change":"2024-09-12T03:21:47.272033+0000","last_active":"2024-09-12T03:21:48.267739+0000","last_peered":"2024-09-12T03:21:48.267739+0000","last_clean":"2024-09-12T03:21:48.267739+0000","last_became_active":"2024-09-12T03:21:47.271715+0000","last_became_peered":"2024-09-12T03:21:47.271715+0000","last_unstale":"2024-09-12T03:21:48.267739+0000","last_undegraded":"2024-09-12T03:21:48.267739+0000","last_fullsized":"2024-09-12T03:21:48.267739+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:49:07.547396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":14,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.725365+0000","last_change":"2024-09-12T03:21:46.255383+0000","last_active":"2024-09-12T03:21:48.725365+0000","last_peered":"2024-09-12T03:21:48.725365+0000","last_clean":"2024-09-12T03:21:48.725365+0000","last_became_active":"2024-09-12T03:21:46.254479+0000","last_became_peered":"2024-09-12T03:21:46.254479+0000","last_unstale":"2024-09-12T03:21:48.725365+0000","last_undegraded":"2024-09-12T03:21:48.725365+0000","last_fullsized":"2024-09-12T03:21:48.725365+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_clean_scrub_stamp":"2024-09-12T03:21:45.231023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:36:44.872680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.724377+0000","last_change":"2024-09-12T03:21:47.276243+0000","last_active":"2024-09-12T03:21:48.724377+0000","last_peered":"2024-09-12T03:21:48.724377+0000","last_clean":"2024-09-12T03:21:48.724377+0000","last_became_active":"2024-09-12T03:21:47.275973+0000","last_became_peered":"2024-09-12T03:21:47.275973+0000","last_unstale":"2024-09-12T03:21:48.724377+0000","last_undegraded":"2024-09-12T03:21:48.724377+0000","last_fullsized":"2024-09-12T03:21:48.724377+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:40:21.225897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":14,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.725355+0000","last_change":"2024-09-12T03:21:46.256740+0000","last_active":"2024-09-12T03:21:48.725355+0000","last_peered":"2024-09-12T03:21:48.725355+0000","last_clean":"2024-09-12T03:21:48.725355+0000","last_became_active":"2024-09-12T03:21:46.256157+0000","last_became_peered":"2024-09-12T03:21:46.256157+0000","last_unstale":"2024-09-12T03:21:48.725355+0000","last_undegraded":"2024-09-12T03:21:48.725355+0000","last_fullsized":"2024-09-12T03:21:48.725355+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_clean_scrub_stamp":"2024-09-12T03:21:45.231023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:27:47.060764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.273562+0000","last_change":"2024-09-12T03:21:47.282425+0000","last_active":"2024-09-12T03:21:48.273562+0000","last_peered":"2024-09-12T03:21:48.273562+0000","last_clean":"2024-09-12T03:21:48.273562+0000","last_became_active":"2024-09-12T03:21:47.282126+0000","last_became_peered":"2024-09-12T03:21:47.282126+0000","last_unstale":"2024-09-12T03:21:48.273562+0000","last_undegraded":"2024-09-12T03:21:48.273562+0000","last_fullsized":"2024-09-12T03:21:48.273562+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:45:51.865826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":14,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.725014+0000","last_change":"2024-09-12T03:21:46.253061+0000","last_active":"2024-09-12T03:21:48.725014+0000","last_peered":"2024-09-12T03:21:48.725014+0000","last_clean":"2024-09-12T03:21:48.725014+0000","last_became_active":"2024-09-12T03:21:46.252712+0000","last_became_peered":"2024-09-12T03:21:46.252712+0000","last_unstale":"2024-09-12T03:21:48.725014+0000","last_undegraded":"2024-09-12T03:21:48.725014+0000","last_fullsized":"2024-09-12T03:21:48.725014+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_clean_scrub_stamp":"2024-09-12T03:21:45.231023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:56:16.255201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.267454+0000","last_change":"2024-09-12T03:21:47.266290+0000","last_active":"2024-09-12T03:21:48.267454+0000","last_peered":"2024-09-12T03:21:48.267454+0000","last_clean":"2024-09-12T03:21:48.267454+0000","last_became_active":"2024-09-12T03:21:47.264199+0000","last_became_peered":"2024-09-12T03:21:47.264199+0000","last_unstale":"2024-09-12T03:21:48.267454+0000","last_undegraded":"2024-09-12T03:21:48.267454+0000","last_fullsized":"2024-09-12T03:21:48.267454+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:41:26.120489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.7","version":"113'2","reported_seq":16,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.266586+0000","last_change":"2024-09-12T03:21:46.260654+0000","last_active":"2024-09-12T03:21:48.266586+0000","last_peered":"2024-09-12T03:21:48.266586+0000","last_clean":"2024-09-12T03:21:48.266586+0000","last_became_active":"2024-09-12T03:21:46.260300+0000","last_became_peered":"2024-09-12T03:21:46.260300+0000","last_unstale":"2024-09-12T03:21:48.266586+0000","last_undegraded":"2024-09-12T03:21:48.266586+0000","last_fullsized":"2024-09-12T03:21:48.266586+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_clean_scrub_stamp":"2024-09-12T03:21:45.231023+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:42:29.591906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.7","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.266487+0000","last_change":"2024-09-12T03:21:47.282067+0000","last_active":"2024-09-12T03:21:48.266487+0000","last_peered":"2024-09-12T03:21:48.266487+0000","last_clean":"2024-09-12T03:21:48.266487+0000","last_became_active":"2024-09-12T03:21:47.276839+0000","last_became_peered":"2024-09-12T03:21:47.276839+0000","last_unstale":"2024-09-12T03:21:48.266487+0000","last_undegraded":"2024-09-12T03:21:48.266487+0000","last_fullsized":"2024-09-12T03:21:48.266487+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:50:01.620996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.6","version":"114'1","reported_seq":15,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.273910+0000","last_change":"2024-09-12T03:21:46.259229+0000","last_active":"2024-09-12T03:21:48.273910+0000","last_peered":"2024-09-12T03:21:48.273910+0000","last_clean":"2024-09-12T03:21:48.273910+0000","last_became_active":"2024-09-12T03:21:46.258467+0000","last_became_peered":"2024-09-12T03:21:46.258467+0000","last_unstale":"2024-09-12T03:21:48.273910+0000","last_undegraded":"2024-09-12T03:21:48.273910+0000","last_fullsized":"2024-09-12T03:21:48.273910+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_clean_scrub_stamp":"2024-09-12T03:21:45.231023+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:36:44.872680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.725277+0000","last_change":"2024-09-12T03:21:47.273262+0000","last_active":"2024-09-12T03:21:48.725277+0000","last_peered":"2024-09-12T03:21:48.725277+0000","last_clean":"2024-09-12T03:21:48.725277+0000","last_became_active":"2024-09-12T03:21:47.272986+0000","last_became_peered":"2024-09-12T03:21:47.272986+0000","last_unstale":"2024-09-12T03:21:48.725277+0000","last_undegraded":"2024-09-12T03:21:48.725277+0000","last_fullsized":"2024-09-12T03:21:48.725277+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:22:36.819977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":14,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.725080+0000","last_change":"2024-09-12T03:21:46.264077+0000","last_active":"2024-09-12T03:21:48.725080+0000","last_peered":"2024-09-12T03:21:48.725080+0000","last_clean":"2024-09-12T03:21:48.725080+0000","last_became_active":"2024-09-12T03:21:46.262611+0000","last_became_peered":"2024-09-12T03:21:46.262611+0000","last_unstale":"2024-09-12T03:21:48.725080+0000","last_undegraded":"2024-09-12T03:21:48.725080+0000","last_fullsized":"2024-09-12T03:21:48.725080+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_clean_scrub_stamp":"2024-09-12T03:21:45.231023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:02:58.180807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.267440+0000","last_change":"2024-09-12T03:21:47.280655+0000","last_active":"2024-09-12T03:21:48.267440+0000","last_peered":"2024-09-12T03:21:48.267440+0000","last_clean":"2024-09-12T03:21:48.267440+0000","last_became_active":"2024-09-12T03:21:47.280133+0000","last_became_peered":"2024-09-12T03:21:47.280133+0000","last_unstale":"2024-09-12T03:21:48.267440+0000","last_undegraded":"2024-09-12T03:21:48.267440+0000","last_fullsized":"2024-09-12T03:21:48.267440+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:54:34.656137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":14,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.725663+0000","last_change":"2024-09-12T03:21:46.265178+0000","last_active":"2024-09-12T03:21:48.725663+0000","last_peered":"2024-09-12T03:21:48.725663+0000","last_clean":"2024-09-12T03:21:48.725663+0000","last_became_active":"2024-09-12T03:21:46.264842+0000","last_became_peered":"2024-09-12T03:21:46.264842+0000","last_unstale":"2024-09-12T03:21:48.725663+0000","last_undegraded":"2024-09-12T03:21:48.725663+0000","last_fullsized":"2024-09-12T03:21:48.725663+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_clean_scrub_stamp":"2024-09-12T03:21:45.231023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:51:50.034404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.273569+0000","last_change":"2024-09-12T03:21:47.275959+0000","last_active":"2024-09-12T03:21:48.273569+0000","last_peered":"2024-09-12T03:21:48.273569+0000","last_clean":"2024-09-12T03:21:48.273569+0000","last_became_active":"2024-09-12T03:21:47.274773+0000","last_became_peered":"2024-09-12T03:21:47.274773+0000","last_unstale":"2024-09-12T03:21:48.273569+0000","last_undegraded":"2024-09-12T03:21:48.273569+0000","last_fullsized":"2024-09-12T03:21:48.273569+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:21:08.369709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":14,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.724357+0000","last_change":"2024-09-12T03:21:46.265078+0000","last_active":"2024-09-12T03:21:48.724357+0000","last_peered":"2024-09-12T03:21:48.724357+0000","last_clean":"2024-09-12T03:21:48.724357+0000","last_became_active":"2024-09-12T03:21:46.264065+0000","last_became_peered":"2024-09-12T03:21:46.264065+0000","last_unstale":"2024-09-12T03:21:48.724357+0000","last_undegraded":"2024-09-12T03:21:48.724357+0000","last_fullsized":"2024-09-12T03:21:48.724357+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_clean_scrub_stamp":"2024-09-12T03:21:45.231023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:34:03.877327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.267328+0000","last_change":"2024-09-12T03:21:47.283216+0000","last_active":"2024-09-12T03:21:48.267328+0000","last_peered":"2024-09-12T03:21:48.267328+0000","last_clean":"2024-09-12T03:21:48.267328+0000","last_became_active":"2024-09-12T03:21:47.282949+0000","last_became_peered":"2024-09-12T03:21:47.282949+0000","last_unstale":"2024-09-12T03:21:48.267328+0000","last_undegraded":"2024-09-12T03:21:48.267328+0000","last_fullsized":"2024-09-12T03:21:48.267328+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:43:43.310362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":14,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.273529+0000","last_change":"2024-09-12T03:21:46.260679+0000","last_active":"2024-09-12T03:21:48.273529+0000","last_peered":"2024-09-12T03:21:48.273529+0000","last_clean":"2024-09-12T03:21:48.273529+0000","last_became_active":"2024-09-12T03:21:46.260354+0000","last_became_peered":"2024-09-12T03:21:46.260354+0000","last_unstale":"2024-09-12T03:21:48.273529+0000","last_undegraded":"2024-09-12T03:21:48.273529+0000","last_fullsized":"2024-09-12T03:21:48.273529+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_clean_scrub_stamp":"2024-09-12T03:21:45.231023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:24:37.527133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.725752+0000","last_change":"2024-09-12T03:21:47.270434+0000","last_active":"2024-09-12T03:21:48.725752+0000","last_peered":"2024-09-12T03:21:48.725752+0000","last_clean":"2024-09-12T03:21:48.725752+0000","last_became_active":"2024-09-12T03:21:47.270141+0000","last_became_peered":"2024-09-12T03:21:47.270141+0000","last_unstale":"2024-09-12T03:21:48.725752+0000","last_undegraded":"2024-09-12T03:21:48.725752+0000","last_fullsized":"2024-09-12T03:21:48.725752+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:00:08.281678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":14,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.725446+0000","last_change":"2024-09-12T03:21:46.256501+0000","last_active":"2024-09-12T03:21:48.725446+0000","last_peered":"2024-09-12T03:21:48.725446+0000","last_clean":"2024-09-12T03:21:48.725446+0000","last_became_active":"2024-09-12T03:21:46.256069+0000","last_became_peered":"2024-09-12T03:21:46.256069+0000","last_unstale":"2024-09-12T03:21:48.725446+0000","last_undegraded":"2024-09-12T03:21:48.725446+0000","last_fullsized":"2024-09-12T03:21:48.725446+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_clean_scrub_stamp":"2024-09-12T03:21:45.231023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:24:37.527133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.725055+0000","last_change":"2024-09-12T03:21:47.269440+0000","last_active":"2024-09-12T03:21:48.725055+0000","last_peered":"2024-09-12T03:21:48.725055+0000","last_clean":"2024-09-12T03:21:48.725055+0000","last_became_active":"2024-09-12T03:21:47.269145+0000","last_became_peered":"2024-09-12T03:21:47.269145+0000","last_unstale":"2024-09-12T03:21:48.725055+0000","last_undegraded":"2024-09-12T03:21:48.725055+0000","last_fullsized":"2024-09-12T03:21:48.725055+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:42:51.860983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":14,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.725107+0000","last_change":"2024-09-12T03:21:46.263800+0000","last_active":"2024-09-12T03:21:48.725107+0000","last_peered":"2024-09-12T03:21:48.725107+0000","last_clean":"2024-09-12T03:21:48.725107+0000","last_became_active":"2024-09-12T03:21:46.263423+0000","last_became_peered":"2024-09-12T03:21:46.263423+0000","last_unstale":"2024-09-12T03:21:48.725107+0000","last_undegraded":"2024-09-12T03:21:48.725107+0000","last_fullsized":"2024-09-12T03:21:48.725107+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_clean_scrub_stamp":"2024-09-12T03:21:45.231023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:06:20.324237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.273458+0000","last_change":"2024-09-12T03:21:47.281952+0000","last_active":"2024-09-12T03:21:48.273458+0000","last_peered":"2024-09-12T03:21:48.273458+0000","last_clean":"2024-09-12T03:21:48.273458+0000","last_became_active":"2024-09-12T03:21:47.279248+0000","last_became_peered":"2024-09-12T03:21:47.279248+0000","last_unstale":"2024-09-12T03:21:48.273458+0000","last_undegraded":"2024-09-12T03:21:48.273458+0000","last_fullsized":"2024-09-12T03:21:48.273458+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:46.237118+0000","last_clean_scrub_stamp":"2024-09-12T03:21:46.237118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:08:22.543376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"113'2","reported_seq":16,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.273663+0000","last_change":"2024-09-12T03:21:46.265296+0000","last_active":"2024-09-12T03:21:48.273663+0000","last_peered":"2024-09-12T03:21:48.273663+0000","last_clean":"2024-09-12T03:21:48.273663+0000","last_became_active":"2024-09-12T03:21:46.264982+0000","last_became_peered":"2024-09-12T03:21:46.264982+0000","last_unstale":"2024-09-12T03:21:48.273663+0000","last_undegraded":"2024-09-12T03:21:48.273663+0000","last_fullsized":"2024-09-12T03:21:48.273663+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:21:45.231023+0000","last_clean_scrub_stamp":"2024-09-12T03:21:45.231023+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:33:45.204480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"70'128","reported_seq":381,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-12T03:21:48.273224+0000","last_change":"2024-09-12T03:16:21.883446+0000","last_active":"2024-09-12T03:21:48.273224+0000","last_peered":"2024-09-12T03:21:48.273224+0000","last_clean":"2024-09-12T03:21:48.273224+0000","last_became_active":"2024-09-12T03:16:21.882207+0000","last_became_peered":"2024-09-12T03:16:21.882207+0000","last_unstale":"2024-09-12T03:21:48.273224+0000","last_undegraded":"2024-09-12T03:21:48.273224+0000","last_fullsized":"2024-09-12T03:21:48.273224+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:20.316426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:20.316426+0000","last_clean_scrub_stamp":"2024-09-12T03:16:20.316426+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:12:15.424320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":19,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542212,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37348,"kb_used_data":1992,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334492,"statfs":{"total":96636764160,"available":96598519808,"internally_reserved":0,"allocated":2039808,"data_stored":1219971,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542214,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36748,"kb_used_data":1400,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335092,"statfs":{"total":96636764160,"available":96599134208,"internally_reserved":0,"allocated":1433600,"data_stored":628965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542213,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36748,"kb_used_data":1396,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335092,"statfs":{"total":96636764160,"available":96599134208,"internally_reserved":0,"allocated":1429504,"data_stored":627465,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542212,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36740,"kb_used_data":1392,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335100,"statfs":{"total":96636764160,"available":96599142400,"internally_reserved":0,"allocated":1425408,"data_stored":627423,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542213,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32584,"kb_used_data":1396,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339256,"statfs":{"total":96636764160,"available":96603398144,"internally_reserved":0,"allocated":1429504,"data_stored":627513,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542213,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32592,"kb_used_data":1396,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339248,"statfs":{"total":96636764160,"available":96603389952,"internally_reserved":0,"allocated":1429504,"data_stored":627969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542214,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32580,"kb_used_data":1388,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339260,"statfs":{"total":96636764160,"available":96603402240,"internally_reserved":0,"allocated":1421312,"data_stored":627353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542214,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33156,"kb_used_data":1968,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338684,"statfs":{"total":96636764160,"available":96602812416,"internally_reserved":0,"allocated":2015232,"data_stored":1217721,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T03:21:55.341 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-12T03:21:55.341 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-12T03:21:55.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:55.487+0000 7fb7354c9640 1 Processor -- start 2024-09-12T03:21:55.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:55.488+0000 7fb7354c9640 1 -- start start 2024-09-12T03:21:55.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:55.488+0000 7fb7354c9640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fb730162040 0x7fb7300d1da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:55.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:55.488+0000 7fb7354c9640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb7300d3790 0x7fb7300b7a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:55.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:55.488+0000 7fb7354c9640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb7300d2e30 con 0x7fb73006d8c0 2024-09-12T03:21:55.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:55.488+0000 7fb7354c9640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fb7300b7f90 con 0x7fb730162040 2024-09-12T03:21:55.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:55.488+0000 7fb7354c9640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fb730162420 con 0x7fb7300d3790 2024-09-12T03:21:55.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:55.489+0000 7fb72effd640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fb730162040 0x7fb7300d1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:55.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:55.489+0000 7fb72e7fc640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb7300d3790 0x7fb7300b7a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:55.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:55.489+0000 7fb72effd640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fb730162040 0x7fb7300d1da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.27:54136/0 (socket says 172.21.15.27:54136) 2024-09-12T03:21:55.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:55.489+0000 7fb72effd640 1 -- 172.21.15.27:0/100769664 learned_addr learned my addr 172.21.15.27:0/100769664 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:21:55.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:55.489+0000 7fb72f7fe640 1 --1- 172.21.15.27:0/100769664 >> v1:172.21.15.45:6789/0 conn(0x7fb73006d8c0 0x7fb7300cf320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.27:36878/0 (socket says 172.21.15.27:36878) 2024-09-12T03:21:55.490 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:55.489+0000 7fb72effd640 1 -- 172.21.15.27:0/100769664 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb7300d3790 msgr2=0x7fb7300b7a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:55.490 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:55.489+0000 7fb72effd640 1 --2- 172.21.15.27:0/100769664 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb7300d3790 0x7fb7300b7a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:55.490 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:55.489+0000 7fb72effd640 1 -- 172.21.15.27:0/100769664 >> v1:172.21.15.45:6789/0 conn(0x7fb73006d8c0 legacy=0x7fb7300cf320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:55.490 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:55.489+0000 7fb72effd640 1 -- 172.21.15.27:0/100769664 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7300d30d0 con 0x7fb730162040 2024-09-12T03:21:55.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:55.490+0000 7fb72effd640 1 --2- 172.21.15.27:0/100769664 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb730162040 0x7fb7300d1da0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fb720009e50 tx=0x7fb72002a710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f6d1861c9035290c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:55.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:55.490+0000 7fb72dffb640 1 -- 172.21.15.27:0/100769664 <== mon.1 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb720036070 con 0x7fb730162040 2024-09-12T03:21:55.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:55.490+0000 7fb72dffb640 1 -- 172.21.15.27:0/100769664 <== mon.1 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb720031600 con 0x7fb730162040 2024-09-12T03:21:55.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:55.490+0000 7fb72dffb640 1 -- 172.21.15.27:0/100769664 <== mon.1 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7200319d0 con 0x7fb730162040 2024-09-12T03:21:55.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:55.491+0000 7fb7354c9640 1 -- 172.21.15.27:0/100769664 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb730162040 msgr2=0x7fb7300d1da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:55.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:55.491+0000 7fb7354c9640 1 --2- 172.21.15.27:0/100769664 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb730162040 0x7fb7300d1da0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fb720009e50 tx=0x7fb72002a710 comp rx=0 tx=0).stop 2024-09-12T03:21:55.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:55.491+0000 7fb7354c9640 1 -- 172.21.15.27:0/100769664 shutdown_connections 2024-09-12T03:21:55.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:55.491+0000 7fb7354c9640 1 --2- 172.21.15.27:0/100769664 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb7300d3790 0x7fb7300b7a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:55.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:55.491+0000 7fb7354c9640 1 --2- 172.21.15.27:0/100769664 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb730162040 0x7fb7300d1da0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:55.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:55.491+0000 7fb7354c9640 1 -- 172.21.15.27:0/100769664 >> 172.21.15.27:0/100769664 conn(0x7fb7300b6700 msgr2=0x7fb7300b6b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:55.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:55.491+0000 7fb7354c9640 1 -- 172.21.15.27:0/100769664 shutdown_connections 2024-09-12T03:21:55.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:55.491+0000 7fb7354c9640 1 -- 172.21.15.27:0/100769664 wait complete. 2024-09-12T03:21:55.494 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:55.492+0000 7fb7354c9640 1 Processor -- start 2024-09-12T03:21:55.494 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:55.492+0000 7fb7354c9640 1 -- start start 2024-09-12T03:21:55.494 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:55.493+0000 7fb7354c9640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb73006d8c0 0x7fb730157c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:55.494 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:55.493+0000 7fb7354c9640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb7300d3790 0x7fb7300b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:55.494 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:55.493+0000 7fb7354c9640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb730162040 0x7fb7300b8f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:55.494 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:55.493+0000 7fb7354c9640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fb7300d4530 con 0x7fb73006d8c0 2024-09-12T03:21:55.494 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:55.493+0000 7fb7354c9640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb7300cee00 con 0x7fb730162040 2024-09-12T03:21:55.495 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:55.493+0000 7fb7354c9640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fb7300d4150 con 0x7fb7300d3790 2024-09-12T03:21:55.495 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:55.493+0000 7fb72effd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb73006d8c0 0x7fb730157c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:55.495 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:55.493+0000 7fb72effd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb73006d8c0 0x7fb730157c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39544/0 (socket says 172.21.15.27:39544) 2024-09-12T03:21:55.495 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:55.493+0000 7fb72effd640 1 -- 172.21.15.27:0/3089580543 learned_addr learned my addr 172.21.15.27:0/3089580543 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:21:55.495 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:55.493+0000 7fb72f7fe640 1 --2- 172.21.15.27:0/3089580543 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb730162040 0x7fb7300b8f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:55.495 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:55.493+0000 7fb72e7fc640 1 --2- 172.21.15.27:0/3089580543 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb7300d3790 0x7fb7300b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:55.495 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:55.493+0000 7fb72effd640 1 -- 172.21.15.27:0/3089580543 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb7300d3790 msgr2=0x7fb7300b8a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:55.495 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:55.493+0000 7fb72effd640 1 --2- 172.21.15.27:0/3089580543 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb7300d3790 0x7fb7300b8a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:55.495 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:55.493+0000 7fb72effd640 1 -- 172.21.15.27:0/3089580543 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb730162040 msgr2=0x7fb7300b8f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:55.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:55.493+0000 7fb72effd640 1 --2- 172.21.15.27:0/3089580543 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb730162040 0x7fb7300b8f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:55.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:55.493+0000 7fb72effd640 1 -- 172.21.15.27:0/3089580543 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7301ead70 con 0x7fb73006d8c0 2024-09-12T03:21:55.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:55.493+0000 7fb72f7fe640 1 --2- 172.21.15.27:0/3089580543 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb730162040 0x7fb7300b8f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:21:55.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:55.493+0000 7fb72effd640 1 --2- 172.21.15.27:0/3089580543 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb73006d8c0 0x7fb730157c80 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7fb720009e50 tx=0x7fb720004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:55.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:55.494+0000 7fb7177fe640 1 -- 172.21.15.27:0/3089580543 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb720041020 con 0x7fb73006d8c0 2024-09-12T03:21:55.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:55.494+0000 7fb7177fe640 1 -- 172.21.15.27:0/3089580543 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb720007820 con 0x7fb73006d8c0 2024-09-12T03:21:55.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:55.494+0000 7fb7177fe640 1 -- 172.21.15.27:0/3089580543 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb720036040 con 0x7fb73006d8c0 2024-09-12T03:21:55.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:55.494+0000 7fb7354c9640 1 -- 172.21.15.27:0/3089580543 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb730160310 con 0x7fb73006d8c0 2024-09-12T03:21:55.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:55.494+0000 7fb7354c9640 1 -- 172.21.15.27:0/3089580543 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb730158ad0 con 0x7fb73006d8c0 2024-09-12T03:21:55.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:55.495+0000 7fb7354c9640 1 -- 172.21.15.27:0/3089580543 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7300ceb00 con 0x7fb73006d8c0 2024-09-12T03:21:55.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:55.495+0000 7fb7177fe640 1 -- 172.21.15.27:0/3089580543 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 19) ==== 90855+0+0 (secure 0 0 0) 0x7fb720044020 con 0x7fb73006d8c0 2024-09-12T03:21:55.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:55.495+0000 7fb7177fe640 1 --2- 172.21.15.27:0/3089580543 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fb71c06c220 0x7fb71c06e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:55.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:55.496+0000 7fb7177fe640 1 -- 172.21.15.27:0/3089580543 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5737+0+0 (secure 0 0 0) 0x7fb7200ac370 con 0x7fb73006d8c0 2024-09-12T03:21:55.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:55.496+0000 7fb72e7fc640 1 --2- 172.21.15.27:0/3089580543 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fb71c06c220 0x7fb71c06e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:55.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:55.496+0000 7fb72e7fc640 1 --2- 172.21.15.27:0/3089580543 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fb71c06c220 0x7fb71c06e6e0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fb7301ea800 tx=0x7fb724008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:55.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:55.500+0000 7fb7177fe640 1 -- 172.21.15.27:0/3089580543 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fb7300ceb00 con 0x7fb73006d8c0 2024-09-12T03:21:55.740 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:55.740+0000 7fb7354c9640 1 -- 172.21.15.27:0/3089580543 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7fb73006e6e0 con 0x7fb73006d8c0 2024-09-12T03:21:55.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:55.744+0000 7fb7177fe640 1 -- 172.21.15.27:0/3089580543 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v19) ==== 190+0+147 (secure 0 0 0) 0x7fb72004b030 con 0x7fb73006d8c0 2024-09-12T03:21:55.745 INFO:teuthology.orchestra.run.smithi027.stdout:[client.0] 2024-09-12T03:21:55.745 INFO:teuthology.orchestra.run.smithi027.stdout: key = AQDVXOJmLm+5ExAAECM4CKfuVElmp/n0b02mXg== 2024-09-12T03:21:55.745 INFO:teuthology.orchestra.run.smithi027.stdout: caps mds = "allow" 2024-09-12T03:21:55.745 INFO:teuthology.orchestra.run.smithi027.stdout: caps mon = "allow r" 2024-09-12T03:21:55.745 INFO:teuthology.orchestra.run.smithi027.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-12T03:21:55.745 INFO:teuthology.orchestra.run.smithi027.stderr:updated caps for client.0 2024-09-12T03:21:55.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:55.747+0000 7fb7354c9640 1 -- 172.21.15.27:0/3089580543 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fb71c06c220 msgr2=0x7fb71c06e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:55.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:55.747+0000 7fb7354c9640 1 --2- 172.21.15.27:0/3089580543 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fb71c06c220 0x7fb71c06e6e0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fb7301ea800 tx=0x7fb724008040 comp rx=0 tx=0).stop 2024-09-12T03:21:55.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:55.747+0000 7fb7354c9640 1 -- 172.21.15.27:0/3089580543 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb73006d8c0 msgr2=0x7fb730157c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:55.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:55.747+0000 7fb7354c9640 1 --2- 172.21.15.27:0/3089580543 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb73006d8c0 0x7fb730157c80 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7fb720009e50 tx=0x7fb720004a40 comp rx=0 tx=0).stop 2024-09-12T03:21:55.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:55.747+0000 7fb7354c9640 1 -- 172.21.15.27:0/3089580543 shutdown_connections 2024-09-12T03:21:55.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:55.747+0000 7fb7354c9640 1 --2- 172.21.15.27:0/3089580543 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb730162040 0x7fb7300b8f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:55.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:55.747+0000 7fb7354c9640 1 --2- 172.21.15.27:0/3089580543 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb7300d3790 0x7fb7300b8a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:55.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:55.747+0000 7fb7354c9640 1 --2- 172.21.15.27:0/3089580543 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fb71c06c220 0x7fb71c06e6e0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:55.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:55.747+0000 7fb7354c9640 1 --2- 172.21.15.27:0/3089580543 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb73006d8c0 0x7fb730157c80 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:55.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:55.747+0000 7fb7354c9640 1 -- 172.21.15.27:0/3089580543 >> 172.21.15.27:0/3089580543 conn(0x7fb7300b6700 msgr2=0x7fb7300b5a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:55.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:55.748+0000 7fb7354c9640 1 -- 172.21.15.27:0/3089580543 shutdown_connections 2024-09-12T03:21:55.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:55.748+0000 7fb7354c9640 1 -- 172.21.15.27:0/3089580543 wait complete. 2024-09-12T03:21:55.759 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-12T03:21:55.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:55.861+0000 7f7eb332a640 1 Processor -- start 2024-09-12T03:21:55.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:55.861+0000 7f7eb332a640 1 -- start start 2024-09-12T03:21:55.862 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:55.862+0000 7f7eb332a640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f7eac0b1ca0 0x7f7eac15f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:55.862 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:55.862+0000 7f7eb332a640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7eac15fd70 con 0x7f7eac06f360 2024-09-12T03:21:55.862 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:55.862+0000 7f7eb332a640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7eac06cc40 con 0x7f7eac0cf660 2024-09-12T03:21:55.862 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:55.862+0000 7f7eb332a640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f7eac0d22f0 con 0x7f7eac0b1ca0 2024-09-12T03:21:55.862 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:55.862+0000 7f7eb10c6640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f7eac0b1ca0 0x7f7eac15f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:55.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:55.862+0000 7f7eb08c5640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f7eac06f360 0x7f7eac0cef50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:56374/0 (socket says 172.21.15.27:56374) 2024-09-12T03:21:55.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:55.862+0000 7f7eb08c5640 1 -- 172.21.15.27:0/4145140188 learned_addr learned my addr 172.21.15.27:0/4145140188 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:21:55.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:55.862+0000 7f7eb10c6640 1 -- 172.21.15.27:0/4145140188 >> v1:172.21.15.45:6789/0 conn(0x7f7eac0cf660 legacy=0x7f7eac0d2780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:55.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:55.862+0000 7f7eb10c6640 1 -- 172.21.15.27:0/4145140188 >> v1:172.21.15.27:6789/0 conn(0x7f7eac06f360 legacy=0x7f7eac0cef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:55.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:55.862+0000 7f7ea3fff640 1 -- 172.21.15.27:0/4145140188 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 939606006 0 0) 0x7f7e9c001110 con 0x7f7eac06f360 2024-09-12T03:21:55.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:55.862+0000 7f7eb10c6640 1 -- 172.21.15.27:0/4145140188 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7eac0d1ce0 con 0x7f7eac0b1ca0 2024-09-12T03:21:55.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:55.863+0000 7f7eb10c6640 1 --2- 172.21.15.27:0/4145140188 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7eac0b1ca0 0x7f7eac15f830 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f7ea8009d90 tx=0x7f7ea802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccb748807718191f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:55.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:55.863+0000 7f7ea3fff640 1 -- 172.21.15.27:0/4145140188 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ea8007740 con 0x7f7eac0b1ca0 2024-09-12T03:21:55.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:55.863+0000 7f7ea3fff640 1 -- 172.21.15.27:0/4145140188 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ea80078e0 con 0x7f7eac0b1ca0 2024-09-12T03:21:55.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:55.863+0000 7f7ea3fff640 1 -- 172.21.15.27:0/4145140188 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ea8007cb0 con 0x7f7eac0b1ca0 2024-09-12T03:21:55.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:55.864+0000 7f7eb332a640 1 -- 172.21.15.27:0/4145140188 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7eac0b1ca0 msgr2=0x7f7eac15f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:55.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:55.864+0000 7f7eb332a640 1 --2- 172.21.15.27:0/4145140188 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7eac0b1ca0 0x7f7eac15f830 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f7ea8009d90 tx=0x7f7ea802a6a0 comp rx=0 tx=0).stop 2024-09-12T03:21:55.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:55.864+0000 7f7eb332a640 1 -- 172.21.15.27:0/4145140188 shutdown_connections 2024-09-12T03:21:55.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:55.864+0000 7f7eb332a640 1 --2- 172.21.15.27:0/4145140188 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7eac0b1ca0 0x7f7eac15f830 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:55.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:55.864+0000 7f7eb332a640 1 -- 172.21.15.27:0/4145140188 >> 172.21.15.27:0/4145140188 conn(0x7f7eac073800 msgr2=0x7f7eac09ba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:55.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:55.864+0000 7f7eb332a640 1 -- 172.21.15.27:0/4145140188 shutdown_connections 2024-09-12T03:21:55.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:55.864+0000 7f7eb332a640 1 -- 172.21.15.27:0/4145140188 wait complete. 2024-09-12T03:21:55.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:55.865+0000 7f7eb332a640 1 Processor -- start 2024-09-12T03:21:55.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:55.865+0000 7f7eb332a640 1 -- start start 2024-09-12T03:21:55.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:55.866+0000 7f7eb332a640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7eac06f360 0x7f7eac153f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:55.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:55.866+0000 7f7eb332a640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f7eac0b1ca0 0x7f7eac0b54d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:55.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:55.866+0000 7f7eb332a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7eac0cf660 0x7f7eac0ce090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:55.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:55.866+0000 7f7eb332a640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f7eac06d1c0 con 0x7f7eac06f360 2024-09-12T03:21:55.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:55.866+0000 7f7eb332a640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f7eac0d1f80 con 0x7f7eac0cf660 2024-09-12T03:21:55.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:55.866+0000 7f7eb332a640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f7eac06d340 con 0x7f7eac0b1ca0 2024-09-12T03:21:55.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:55.866+0000 7f7eb10c6640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7eac06f360 0x7f7eac153f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:55.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:55.866+0000 7f7eb10c6640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7eac06f360 0x7f7eac153f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39562/0 (socket says 172.21.15.27:39562) 2024-09-12T03:21:55.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:55.866+0000 7f7eb10c6640 1 -- 172.21.15.27:0/2116693165 learned_addr learned my addr 172.21.15.27:0/2116693165 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:21:55.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:55.866+0000 7f7eb18c7640 1 --2- 172.21.15.27:0/2116693165 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7eac0cf660 0x7f7eac0ce090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:55.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:55.866+0000 7f7eb08c5640 1 --2- 172.21.15.27:0/2116693165 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f7eac0b1ca0 0x7f7eac0b54d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:55.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:55.866+0000 7f7eb10c6640 1 -- 172.21.15.27:0/2116693165 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f7eac0b1ca0 msgr2=0x7f7eac0b54d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:55.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:55.866+0000 7f7eb10c6640 1 --2- 172.21.15.27:0/2116693165 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f7eac0b1ca0 0x7f7eac0b54d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:55.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:55.866+0000 7f7eb10c6640 1 -- 172.21.15.27:0/2116693165 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7eac0cf660 msgr2=0x7f7eac0ce090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:55.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:55.866+0000 7f7eb10c6640 1 --2- 172.21.15.27:0/2116693165 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7eac0cf660 0x7f7eac0ce090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:55.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:55.866+0000 7f7eb10c6640 1 -- 172.21.15.27:0/2116693165 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7eac154480 con 0x7f7eac06f360 2024-09-12T03:21:55.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:55.866+0000 7f7eb08c5640 1 --2- 172.21.15.27:0/2116693165 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f7eac0b1ca0 0x7f7eac0b54d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:21:55.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:55.866+0000 7f7eb18c7640 1 --2- 172.21.15.27:0/2116693165 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7eac0cf660 0x7f7eac0ce090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:21:55.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:55.866+0000 7f7eb10c6640 1 --2- 172.21.15.27:0/2116693165 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7eac06f360 0x7f7eac153f40 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f7ea8009d90 tx=0x7f7ea8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:55.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:55.867+0000 7f7ea1ffb640 1 -- 172.21.15.27:0/2116693165 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ea8040070 con 0x7f7eac06f360 2024-09-12T03:21:55.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:55.867+0000 7f7ea1ffb640 1 -- 172.21.15.27:0/2116693165 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ea8031c40 con 0x7f7eac06f360 2024-09-12T03:21:55.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:55.867+0000 7f7eb332a640 1 -- 172.21.15.27:0/2116693165 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7eac0b6bd0 con 0x7f7eac06f360 2024-09-12T03:21:55.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:55.867+0000 7f7ea1ffb640 1 -- 172.21.15.27:0/2116693165 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ea803c420 con 0x7f7eac06f360 2024-09-12T03:21:55.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:55.867+0000 7f7eb332a640 1 -- 172.21.15.27:0/2116693165 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7eac0b7190 con 0x7f7eac06f360 2024-09-12T03:21:55.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:55.868+0000 7f7eb332a640 1 -- 172.21.15.27:0/2116693165 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7eac15dce0 con 0x7f7eac06f360 2024-09-12T03:21:55.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:55.868+0000 7f7ea1ffb640 1 -- 172.21.15.27:0/2116693165 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 19) ==== 90855+0+0 (secure 0 0 0) 0x7f7ea8039040 con 0x7f7eac06f360 2024-09-12T03:21:55.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:55.869+0000 7f7ea1ffb640 1 --2- 172.21.15.27:0/2116693165 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f7e9006c250 0x7f7e9006e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:55.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:55.869+0000 7f7ea1ffb640 1 -- 172.21.15.27:0/2116693165 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5737+0+0 (secure 0 0 0) 0x7f7ea80acd90 con 0x7f7eac06f360 2024-09-12T03:21:55.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:55.869+0000 7f7eb08c5640 1 --2- 172.21.15.27:0/2116693165 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f7e9006c250 0x7f7e9006e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:55.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:55.869+0000 7f7eb08c5640 1 --2- 172.21.15.27:0/2116693165 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f7e9006c250 0x7f7e9006e710 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f7e9c0013b0 tx=0x7f7e9c009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:55.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:55.873+0000 7f7ea1ffb640 1 -- 172.21.15.27:0/2116693165 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f7ea807a9e0 con 0x7f7eac06f360 2024-09-12T03:21:56.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.084+0000 7f7eb332a640 1 -- 172.21.15.27:0/2116693165 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7eac0cfc40 con 0x7f7eac06f360 2024-09-12T03:21:56.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.084+0000 7f7ea1ffb640 1 -- 172.21.15.27:0/2116693165 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) ==== 94+0+5225 (secure 0 0 0) 0x7f7ea807ece0 con 0x7f7eac06f360 2024-09-12T03:21:56.084 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:21:56.084 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":67,"btime":"2024-09-12T03:21:49:612947+0000","default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6206,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/2249506350","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2249506350},{"type":"v1","addr":"172.21.15.45:6834","nonce":2249506350}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63},{"gid":6209,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/2292759672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":2292759672},{"type":"v1","addr":"172.21.15.45:6835","nonce":2292759672}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63},{"gid":7299,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6834/4021825986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":4021825986},{"type":"v1","addr":"172.21.15.27:6834","nonce":4021825986}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":67,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-12T03:21:46.603883+0000","modified":"2024-09-12T03:21:48.956285+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7302},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7302":{"gid":7302,"name":"a","rank":0,"incarnation":64,"state":"up:active","state_seq":3,"addr":"172.21.15.27:6835/1760654641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":1760654641},{"type":"v1","addr":"172.21.15.27:6835","nonce":1760654641}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7302,"qdb_cluster":[7302]},"id":9}]} 2024-09-12T03:21:56.085 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 67 2024-09-12T03:21:56.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.086+0000 7f7eb332a640 1 -- 172.21.15.27:0/2116693165 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f7e9006c250 msgr2=0x7f7e9006e710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:56.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.086+0000 7f7eb332a640 1 --2- 172.21.15.27:0/2116693165 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f7e9006c250 0x7f7e9006e710 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f7e9c0013b0 tx=0x7f7e9c009040 comp rx=0 tx=0).stop 2024-09-12T03:21:56.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.086+0000 7f7eb332a640 1 -- 172.21.15.27:0/2116693165 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7eac06f360 msgr2=0x7f7eac153f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:56.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.086+0000 7f7eb332a640 1 --2- 172.21.15.27:0/2116693165 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7eac06f360 0x7f7eac153f40 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f7ea8009d90 tx=0x7f7ea8004a40 comp rx=0 tx=0).stop 2024-09-12T03:21:56.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.086+0000 7f7eb332a640 1 -- 172.21.15.27:0/2116693165 shutdown_connections 2024-09-12T03:21:56.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.086+0000 7f7eb332a640 1 --2- 172.21.15.27:0/2116693165 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7eac0cf660 0x7f7eac0ce090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:56.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.086+0000 7f7eb332a640 1 --2- 172.21.15.27:0/2116693165 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f7eac0b1ca0 0x7f7eac0b54d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:56.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.086+0000 7f7eb332a640 1 --2- 172.21.15.27:0/2116693165 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f7e9006c250 0x7f7e9006e710 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:56.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.086+0000 7f7eb332a640 1 --2- 172.21.15.27:0/2116693165 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7eac06f360 0x7f7eac153f40 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:56.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.086+0000 7f7eb332a640 1 -- 172.21.15.27:0/2116693165 >> 172.21.15.27:0/2116693165 conn(0x7f7eac073800 msgr2=0x7f7eac155f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:56.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.086+0000 7f7eb332a640 1 -- 172.21.15.27:0/2116693165 shutdown_connections 2024-09-12T03:21:56.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.086+0000 7f7eb332a640 1 -- 172.21.15.27:0/2116693165 wait complete. 2024-09-12T03:21:56.501 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 67, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-12T03:21:46.603883+0000', 'modified': '2024-09-12T03:21:48.956285+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7302}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7302': {'gid': 7302, 'name': 'a', 'rank': 0, 'incarnation': 64, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.27:6835/1760654641', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6833', 'nonce': 1760654641}, {'type': 'v1', 'addr': '172.21.15.27:6835', 'nonce': 1760654641}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [19], 'metadata_pool': 18, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 7302, 'qdb_cluster': [7302]} 2024-09-12T03:21:56.501 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-12T03:21:56.502 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-09-12T03:21:56.608 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.607+0000 7fd939e65640 1 Processor -- start 2024-09-12T03:21:56.608 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.607+0000 7fd939e65640 1 -- start start 2024-09-12T03:21:56.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.608+0000 7fd939e65640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fd934072fc0 0x7fd93406dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:56.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.608+0000 7fd939e65640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd934073b50 con 0x7fd93406fcf0 2024-09-12T03:21:56.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.608+0000 7fd939e65640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd934073870 con 0x7fd93415a1f0 2024-09-12T03:21:56.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.608+0000 7fd939e65640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7fd9340ce3e0 con 0x7fd934072fc0 2024-09-12T03:21:56.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.608+0000 7fd932ffd640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fd934072fc0 0x7fd93406dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:56.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.608+0000 7fd933fff640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fd93406fcf0 0x7fd93415cce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:56380/0 (socket says 172.21.15.27:56380) 2024-09-12T03:21:56.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.608+0000 7fd933fff640 1 -- 172.21.15.27:0/3231192025 learned_addr learned my addr 172.21.15.27:0/3231192025 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:21:56.611 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.609+0000 7fd932ffd640 1 -- 172.21.15.27:0/3231192025 >> v1:172.21.15.45:6789/0 conn(0x7fd93415a1f0 legacy=0x7fd9340728b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:56.611 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.609+0000 7fd932ffd640 1 -- 172.21.15.27:0/3231192025 >> v1:172.21.15.27:6789/0 conn(0x7fd93406fcf0 legacy=0x7fd93415cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:56.611 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.609+0000 7fd932ffd640 1 -- 172.21.15.27:0/3231192025 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9340ce6c0 con 0x7fd934072fc0 2024-09-12T03:21:56.611 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.609+0000 7fd9327fc640 1 -- 172.21.15.27:0/3231192025 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3965888448 0 0) 0x7fd934073b50 con 0x7fd93406fcf0 2024-09-12T03:21:56.611 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.609+0000 7fd932ffd640 1 --2- 172.21.15.27:0/3231192025 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd934072fc0 0x7fd93406dac0 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7fd9280097a0 tx=0x7fd92802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b777878132fe0c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:56.611 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.609+0000 7fd9327fc640 1 -- 172.21.15.27:0/3231192025 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd928037070 con 0x7fd934072fc0 2024-09-12T03:21:56.611 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.609+0000 7fd9327fc640 1 -- 172.21.15.27:0/3231192025 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd928032440 con 0x7fd934072fc0 2024-09-12T03:21:56.611 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.610+0000 7fd9327fc640 1 -- 172.21.15.27:0/3231192025 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd928032810 con 0x7fd934072fc0 2024-09-12T03:21:56.611 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.610+0000 7fd939e65640 1 -- 172.21.15.27:0/3231192025 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd934072fc0 msgr2=0x7fd93406dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:56.611 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.610+0000 7fd939e65640 1 --2- 172.21.15.27:0/3231192025 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd934072fc0 0x7fd93406dac0 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7fd9280097a0 tx=0x7fd92802a230 comp rx=0 tx=0).stop 2024-09-12T03:21:56.612 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.610+0000 7fd939e65640 1 -- 172.21.15.27:0/3231192025 shutdown_connections 2024-09-12T03:21:56.612 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.610+0000 7fd939e65640 1 --2- 172.21.15.27:0/3231192025 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd934072fc0 0x7fd93406dac0 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:56.612 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.610+0000 7fd939e65640 1 -- 172.21.15.27:0/3231192025 >> 172.21.15.27:0/3231192025 conn(0x7fd93406ca30 msgr2=0x7fd93406ce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:56.612 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.610+0000 7fd939e65640 1 -- 172.21.15.27:0/3231192025 shutdown_connections 2024-09-12T03:21:56.612 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.610+0000 7fd939e65640 1 -- 172.21.15.27:0/3231192025 wait complete. 2024-09-12T03:21:56.615 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.611+0000 7fd939e65640 1 Processor -- start 2024-09-12T03:21:56.616 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.611+0000 7fd939e65640 1 -- start start 2024-09-12T03:21:56.616 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.612+0000 7fd939e65640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fd93406fcf0 0x7fd934145860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:56.616 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.612+0000 7fd939e65640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd934072fc0 0x7fd934158e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:56.616 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.612+0000 7fd939e65640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd93415a1f0 0x7fd9341593b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:56.616 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.612+0000 7fd939e65640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fd934070100 con 0x7fd93415a1f0 2024-09-12T03:21:56.616 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.612+0000 7fd939e65640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fd934073520 con 0x7fd934072fc0 2024-09-12T03:21:56.616 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.612+0000 7fd939e65640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fd9340b02a0 con 0x7fd93406fcf0 2024-09-12T03:21:56.616 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.612+0000 7fd933fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd93415a1f0 0x7fd9341593b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:56.616 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.612+0000 7fd933fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd93415a1f0 0x7fd9341593b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39582/0 (socket says 172.21.15.27:39582) 2024-09-12T03:21:56.616 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.612+0000 7fd933fff640 1 -- 172.21.15.27:0/3463754147 learned_addr learned my addr 172.21.15.27:0/3463754147 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:21:56.616 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.612+0000 7fd932ffd640 1 --2- 172.21.15.27:0/3463754147 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd934072fc0 0x7fd934158e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:56.617 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.612+0000 7fd9337fe640 1 --2- 172.21.15.27:0/3463754147 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fd93406fcf0 0x7fd934145860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:56.617 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.613+0000 7fd933fff640 1 -- 172.21.15.27:0/3463754147 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fd93406fcf0 msgr2=0x7fd934145860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:56.617 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.613+0000 7fd933fff640 1 --2- 172.21.15.27:0/3463754147 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fd93406fcf0 0x7fd934145860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:56.617 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.613+0000 7fd933fff640 1 -- 172.21.15.27:0/3463754147 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd934072fc0 msgr2=0x7fd934158e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:56.617 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.613+0000 7fd933fff640 1 --2- 172.21.15.27:0/3463754147 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd934072fc0 0x7fd934158e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:56.617 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.613+0000 7fd933fff640 1 -- 172.21.15.27:0/3463754147 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9341e5c30 con 0x7fd93415a1f0 2024-09-12T03:21:56.617 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.613+0000 7fd933fff640 1 --2- 172.21.15.27:0/3463754147 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd93415a1f0 0x7fd9341593b0 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7fd92c00c700 tx=0x7fd92c00cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:56.617 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.613+0000 7fd91bfff640 1 -- 172.21.15.27:0/3463754147 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd92c018070 con 0x7fd93415a1f0 2024-09-12T03:21:56.617 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.613+0000 7fd91bfff640 1 -- 172.21.15.27:0/3463754147 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd92c00ce10 con 0x7fd93415a1f0 2024-09-12T03:21:56.617 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.613+0000 7fd91bfff640 1 -- 172.21.15.27:0/3463754147 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd92c00a860 con 0x7fd93415a1f0 2024-09-12T03:21:56.617 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.613+0000 7fd939e65640 1 -- 172.21.15.27:0/3463754147 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9341e5950 con 0x7fd93415a1f0 2024-09-12T03:21:56.617 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.613+0000 7fd939e65640 1 -- 172.21.15.27:0/3463754147 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9341465a0 con 0x7fd93415a1f0 2024-09-12T03:21:56.617 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.615+0000 7fd91bfff640 1 -- 172.21.15.27:0/3463754147 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 19) ==== 90855+0+0 (secure 0 0 0) 0x7fd92c01b070 con 0x7fd93415a1f0 2024-09-12T03:21:56.620 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.615+0000 7fd91bfff640 1 --2- 172.21.15.27:0/3463754147 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fd92006c220 0x7fd92006e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:56.620 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.615+0000 7fd9337fe640 1 --2- 172.21.15.27:0/3463754147 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fd92006c220 0x7fd92006e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:56.621 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.615+0000 7fd91bfff640 1 -- 172.21.15.27:0/3463754147 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5737+0+0 (secure 0 0 0) 0x7fd92c08aa00 con 0x7fd93415a1f0 2024-09-12T03:21:56.621 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.616+0000 7fd9337fe640 1 --2- 172.21.15.27:0/3463754147 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fd92006c220 0x7fd92006e6e0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fd924000f60 tx=0x7fd924009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:56.621 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.616+0000 7fd93cc07740 1 Processor -- start 2024-09-12T03:21:56.621 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.616+0000 7fd93cc07740 1 -- start start 2024-09-12T03:21:56.621 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.617+0000 7fd93cc07740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fa053f9120 0x55fa053f9500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:56.621 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.617+0000 7fd93cc07740 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x55fa05413ae0 0x55fa05415290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:56.621 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.617+0000 7fd93cc07740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fa053cca80 0x55fa054157d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:56.621 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.617+0000 7fd93cc07740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55fa05331520 con 0x55fa053f9120 2024-09-12T03:21:56.621 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.617+0000 7fd93cc07740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55fa052d93e0 con 0x55fa053cca80 2024-09-12T03:21:56.621 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.617+0000 7fd93cc07740 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x55fa0532d4e0 con 0x55fa05413ae0 2024-09-12T03:21:56.621 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.617+0000 7fd9337fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fa053f9120 0x55fa053f9500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:56.621 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.617+0000 7fd9337fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fa053f9120 0x55fa053f9500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39596/0 (socket says 172.21.15.27:39596) 2024-09-12T03:21:56.621 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.617+0000 7fd9337fe640 1 -- 172.21.15.27:0/3191467807 learned_addr learned my addr 172.21.15.27:0/3191467807 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:21:56.622 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.617+0000 7fd9337fe640 1 -- 172.21.15.27:0/3191467807 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x55fa05413ae0 msgr2=0x55fa05415290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:56.622 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.617+0000 7fd9337fe640 1 --2- 172.21.15.27:0/3191467807 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x55fa05413ae0 0x55fa05415290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:56.622 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.617+0000 7fd9337fe640 1 -- 172.21.15.27:0/3191467807 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fa053cca80 msgr2=0x55fa054157d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:56.622 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.617+0000 7fd9337fe640 1 --2- 172.21.15.27:0/3191467807 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fa053cca80 0x55fa054157d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:56.622 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.617+0000 7fd9337fe640 1 -- 172.21.15.27:0/3191467807 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa052f42f0 con 0x55fa053f9120 2024-09-12T03:21:56.622 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.618+0000 7fd9337fe640 1 --2- 172.21.15.27:0/3191467807 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fa053f9120 0x55fa053f9500 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7fd92400fba0 tx=0x7fd924012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c268a12fbcc5314 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:56.622 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.618+0000 7fd9197fa640 1 -- 172.21.15.27:0/3191467807 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd924025070 con 0x55fa053f9120 2024-09-12T03:21:56.622 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.618+0000 7fd9197fa640 1 -- 172.21.15.27:0/3191467807 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd92401b050 con 0x55fa053f9120 2024-09-12T03:21:56.622 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.618+0000 7fd9197fa640 1 -- 172.21.15.27:0/3191467807 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd924020410 con 0x55fa053f9120 2024-09-12T03:21:56.622 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.618+0000 7fd93cc07740 1 -- 172.21.15.27:0/3191467807 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fa053f9120 msgr2=0x55fa053f9500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:56.622 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.618+0000 7fd93cc07740 1 --2- 172.21.15.27:0/3191467807 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fa053f9120 0x55fa053f9500 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7fd92400fba0 tx=0x7fd924012000 comp rx=0 tx=0).stop 2024-09-12T03:21:56.622 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.619+0000 7fd93cc07740 1 -- 172.21.15.27:0/3191467807 shutdown_connections 2024-09-12T03:21:56.622 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.619+0000 7fd93cc07740 1 --2- 172.21.15.27:0/3191467807 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fa053cca80 0x55fa054157d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:56.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.619+0000 7fd93cc07740 1 --2- 172.21.15.27:0/3191467807 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x55fa05413ae0 0x55fa05415290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:56.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.619+0000 7fd93cc07740 1 --2- 172.21.15.27:0/3191467807 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fa053f9120 0x55fa053f9500 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:56.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.619+0000 7fd93cc07740 1 -- 172.21.15.27:0/3191467807 >> 172.21.15.27:0/3191467807 conn(0x55fa053cba80 msgr2=0x55fa05413f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:56.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.619+0000 7fd93cc07740 1 -- 172.21.15.27:0/3191467807 shutdown_connections 2024-09-12T03:21:56.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.619+0000 7fd93cc07740 1 -- 172.21.15.27:0/3191467807 wait complete. 2024-09-12T03:21:56.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.619+0000 7fd93cc07740 1 -- start start 2024-09-12T03:21:56.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.619+0000 7fd93cc07740 1 Processor -- start 2024-09-12T03:21:56.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.620+0000 7fd93cc07740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fa053f9120 0x55fa053fc300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:56.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.620+0000 7fd93cc07740 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x55fa05413ae0 0x55fa05412480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:56.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.620+0000 7fd93cc07740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fa053f5cf0 0x55fa054129c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:56.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.620+0000 7fd93cc07740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55fa053f9ee0 con 0x55fa053f9120 2024-09-12T03:21:56.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.620+0000 7fd93cc07740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55fa052ba450 con 0x55fa053f5cf0 2024-09-12T03:21:56.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.620+0000 7fd93cc07740 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x55fa05414770 con 0x55fa05413ae0 2024-09-12T03:21:56.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.620+0000 7fd93cc07740 10 client.0 _ll_register_callbacks cb 0x55fa05043ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:21:56.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.620+0000 7fd9337fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fa053f9120 0x55fa053fc300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:56.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.620+0000 7fd9337fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fa053f9120 0x55fa053fc300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39610/0 (socket says 172.21.15.27:39610) 2024-09-12T03:21:56.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.620+0000 7fd9337fe640 1 -- 172.21.15.27:0/2827151815 learned_addr learned my addr 172.21.15.27:0/2827151815 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:21:56.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.620+0000 7fd932ffd640 1 --2- 172.21.15.27:0/2827151815 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x55fa05413ae0 0x55fa05412480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:56.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.620+0000 7fd933fff640 1 --2- 172.21.15.27:0/2827151815 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fa053f5cf0 0x55fa054129c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:56.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.620+0000 7fd9337fe640 1 -- 172.21.15.27:0/2827151815 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x55fa05413ae0 msgr2=0x55fa05412480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:56.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.620+0000 7fd9337fe640 1 --2- 172.21.15.27:0/2827151815 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x55fa05413ae0 0x55fa05412480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:56.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.620+0000 7fd9337fe640 1 -- 172.21.15.27:0/2827151815 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fa053f5cf0 msgr2=0x55fa054129c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:56.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.620+0000 7fd9337fe640 1 --2- 172.21.15.27:0/2827151815 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fa053f5cf0 0x55fa054129c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:56.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.620+0000 7fd9337fe640 1 -- 172.21.15.27:0/2827151815 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa050442d0 con 0x55fa053f9120 2024-09-12T03:21:56.628 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.620+0000 7fd933fff640 1 --2- 172.21.15.27:0/2827151815 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fa053f5cf0 0x55fa054129c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:21:56.628 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.620+0000 7fd932ffd640 1 --2- 172.21.15.27:0/2827151815 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x55fa05413ae0 0x55fa05412480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:21:56.628 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.621+0000 7fd9337fe640 1 --2- 172.21.15.27:0/2827151815 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fa053f9120 0x55fa053fc300 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7fd92400f5a0 tx=0x7fd924001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42f499008500c6ec server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:56.628 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.621+0000 7fd9127fc640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-12T03:21:56.628 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.621+0000 7fd9127fc640 1 -- 172.21.15.27:0/2827151815 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd924030020 con 0x55fa053f9120 2024-09-12T03:21:56.628 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.621+0000 7fd93cc07740 1 -- 172.21.15.27:0/2827151815 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd92401b050 con 0x55fa053f9120 2024-09-12T03:21:56.628 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.621+0000 7fd9127fc640 1 -- 172.21.15.27:0/2827151815 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 5737+0+0 (secure 0 0 0) 0x7fd924020a40 con 0x55fa053f9120 2024-09-12T03:21:56.628 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.621+0000 7fd9127fc640 1 -- 172.21.15.27:0/2827151815 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7fd92403a050 con 0x55fa053f9120 2024-09-12T03:21:56.628 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.621+0000 7fd93cc07740 10 client.7458 fetch_fsmap learned FSMap version 67 2024-09-12T03:21:56.628 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.621+0000 7fd93cc07740 1 -- 172.21.15.27:0/2827151815 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=2+}) -- 0x55fa0510e6b0 con 0x55fa053f9120 2024-09-12T03:21:56.628 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.621+0000 7fd9127fc640 1 -- 172.21.15.27:0/2827151815 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd92401b050 con 0x55fa053f9120 2024-09-12T03:21:56.628 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.621+0000 7fd9127fc640 1 -- 172.21.15.27:0/2827151815 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 67) ==== 3502+0+0 (secure 0 0 0) 0x7fd92403d020 con 0x55fa053f9120 2024-09-12T03:21:56.628 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.621+0000 7fd93cc07740 10 client.7458 fetch_fsmap finished waiting for FSMap version 67 2024-09-12T03:21:56.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.621+0000 7fd93cc07740 10 client.7458 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-12T03:21:56.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.621+0000 7fd93cc07740 20 client.7458 populate_metadata read hostname 'smithi027' 2024-09-12T03:21:56.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.621+0000 7fd93cc07740 1 --2- 172.21.15.27:0/2827151815 >> [v2:172.21.15.27:6833/1760654641,v1:172.21.15.27:6835/1760654641] conn(0x55fa053cd0a0 0x55fa053fadf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:56.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.621+0000 7fd93cc07740 4 client.7458 mds_command: new command op to 7302 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:21:56.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.621+0000 7fd93cc07740 1 -- 172.21.15.27:0/2827151815 --> [v2:172.21.15.27:6833/1760654641,v1:172.21.15.27:6835/1760654641] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fa0533ec50 con 0x55fa053cd0a0 2024-09-12T03:21:56.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.621+0000 7fd932ffd640 1 --2- 172.21.15.27:0/2827151815 >> [v2:172.21.15.27:6833/1760654641,v1:172.21.15.27:6835/1760654641] conn(0x55fa053cd0a0 0x55fa053fadf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:56.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.622+0000 7fd9127fc640 10 client.7458 ms_handle_connect on v2:172.21.15.27:6833/1760654641 2024-09-12T03:21:56.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.622+0000 7fd932ffd640 1 --2- 172.21.15.27:0/2827151815 >> [v2:172.21.15.27:6833/1760654641,v1:172.21.15.27:6835/1760654641] conn(0x55fa053cd0a0 0x55fa053fadf0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bed478f96355758e server_cookie=831db5a01a89b249 in_seq=0 out_seq=0 2024-09-12T03:21:56.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.623+0000 7fd9127fc640 1 -- 172.21.15.27:0/2827151815 <== mds.0 v2:172.21.15.27:6833/1760654641 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55fa0533ec50 con 0x55fa053cd0a0 2024-09-12T03:21:56.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.623+0000 7fd9127fc640 10 client.7458 handle_command_reply: tid=0 2024-09-12T03:21:56.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.623+0000 7fd93cc07740 1 client.7458 shutdown 2024-09-12T03:21:56.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.623+0000 7fd93cc07740 1 -- 172.21.15.27:0/2827151815 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fa053f9120 msgr2=0x55fa053fc300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:56.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.623+0000 7fd93cc07740 1 --2- 172.21.15.27:0/2827151815 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fa053f9120 0x55fa053fc300 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7fd92400f5a0 tx=0x7fd924001120 comp rx=0 tx=0).stop 2024-09-12T03:21:56.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.623+0000 7fd93cc07740 1 -- 172.21.15.27:0/2827151815 shutdown_connections 2024-09-12T03:21:56.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.623+0000 7fd93cc07740 1 --2- 172.21.15.27:0/2827151815 >> [v2:172.21.15.27:6833/1760654641,v1:172.21.15.27:6835/1760654641] conn(0x55fa053cd0a0 0x55fa053fadf0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:56.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.623+0000 7fd93cc07740 1 --2- 172.21.15.27:0/2827151815 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fa053f5cf0 0x55fa054129c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:56.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.623+0000 7fd93cc07740 1 --2- 172.21.15.27:0/2827151815 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x55fa05413ae0 0x55fa05412480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:56.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.623+0000 7fd93cc07740 1 --2- 172.21.15.27:0/2827151815 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fa053f9120 0x55fa053fc300 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:56.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.623+0000 7fd9127fc640 0 client.7458 ms_handle_reset on v2:172.21.15.27:6833/1760654641 2024-09-12T03:21:56.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.623+0000 7fd93cc07740 1 -- 172.21.15.27:0/2827151815 >> 172.21.15.27:0/2827151815 conn(0x55fa053cca80 msgr2=0x55fa05417440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:56.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.623+0000 7fd93cc07740 1 -- 172.21.15.27:0/2827151815 shutdown_connections 2024-09-12T03:21:56.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.623+0000 7fd93cc07740 1 -- 172.21.15.27:0/2827151815 wait complete. 2024-09-12T03:21:56.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.623+0000 7fd93cc07740 20 client.7458 trim_cache size 0 max 16384 2024-09-12T03:21:56.642 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.641+0000 7fd93cc07740 1 Processor -- start 2024-09-12T03:21:56.642 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.641+0000 7fd93cc07740 1 -- start start 2024-09-12T03:21:56.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.642+0000 7fd93cc07740 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x55fa053f5cf0 0x55fa05432970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:56.650 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.642+0000 7fd93cc07740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fa053f9120 0x55fa0543c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:56.650 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.642+0000 7fd93cc07740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fa05413ae0 0x55fa0543caf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:56.650 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.642+0000 7fd93cc07740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fd9340b5240 con 0x55fa05413ae0 2024-09-12T03:21:56.650 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.642+0000 7fd93cc07740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fd934149050 con 0x55fa053f9120 2024-09-12T03:21:56.650 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.642+0000 7fd93cc07740 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fd934147b30 con 0x55fa053f5cf0 2024-09-12T03:21:56.650 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.642+0000 7fd933fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fa05413ae0 0x55fa0543caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:56.650 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.642+0000 7fd933fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fa05413ae0 0x55fa0543caf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39612/0 (socket says 172.21.15.27:39612) 2024-09-12T03:21:56.650 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.642+0000 7fd933fff640 1 -- 172.21.15.27:0/3341779467 learned_addr learned my addr 172.21.15.27:0/3341779467 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:21:56.650 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.642+0000 7fd932ffd640 1 --2- 172.21.15.27:0/3341779467 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fa053f9120 0x55fa0543c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:56.651 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.642+0000 7fd933fff640 1 -- 172.21.15.27:0/3341779467 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x55fa053f5cf0 msgr2=0x55fa05432970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:56.651 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.642+0000 7fd9337fe640 1 --2- 172.21.15.27:0/3341779467 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x55fa053f5cf0 0x55fa05432970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:56.651 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.642+0000 7fd933fff640 1 --2- 172.21.15.27:0/3341779467 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x55fa053f5cf0 0x55fa05432970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:56.651 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.642+0000 7fd933fff640 1 -- 172.21.15.27:0/3341779467 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fa053f9120 msgr2=0x55fa0543c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:56.651 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.642+0000 7fd933fff640 1 --2- 172.21.15.27:0/3341779467 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fa053f9120 0x55fa0543c5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:56.651 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.642+0000 7fd933fff640 1 -- 172.21.15.27:0/3341779467 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa053cb810 con 0x55fa05413ae0 2024-09-12T03:21:56.651 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.642+0000 7fd932ffd640 1 --2- 172.21.15.27:0/3341779467 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fa053f9120 0x55fa0543c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:21:56.651 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.642+0000 7fd9337fe640 1 --2- 172.21.15.27:0/3341779467 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x55fa053f5cf0 0x55fa05432970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:21:56.651 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.642+0000 7fd933fff640 1 --2- 172.21.15.27:0/3341779467 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fa05413ae0 0x55fa0543caf0 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7fd92c020030 tx=0x7fd92c05f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f706ed5df38503b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:56.651 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.642+0000 7fd918ff9640 1 -- 172.21.15.27:0/3341779467 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd92c061320 con 0x55fa05413ae0 2024-09-12T03:21:56.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.643+0000 7fd918ff9640 1 -- 172.21.15.27:0/3341779467 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd92c054e60 con 0x55fa05413ae0 2024-09-12T03:21:56.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.643+0000 7fd918ff9640 1 -- 172.21.15.27:0/3341779467 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd92c06ce10 con 0x55fa05413ae0 2024-09-12T03:21:56.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.643+0000 7fd93cc07740 1 -- 172.21.15.27:0/3341779467 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fa05413ae0 msgr2=0x55fa0543caf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:56.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.643+0000 7fd93cc07740 1 --2- 172.21.15.27:0/3341779467 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fa05413ae0 0x55fa0543caf0 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7fd92c020030 tx=0x7fd92c05f040 comp rx=0 tx=0).stop 2024-09-12T03:21:56.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.643+0000 7fd93cc07740 1 -- 172.21.15.27:0/3341779467 shutdown_connections 2024-09-12T03:21:56.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.643+0000 7fd93cc07740 1 --2- 172.21.15.27:0/3341779467 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fa05413ae0 0x55fa0543caf0 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:56.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.643+0000 7fd93cc07740 1 --2- 172.21.15.27:0/3341779467 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fa053f9120 0x55fa0543c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:56.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.643+0000 7fd93cc07740 1 --2- 172.21.15.27:0/3341779467 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x55fa053f5cf0 0x55fa05432970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:56.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.643+0000 7fd93cc07740 1 -- 172.21.15.27:0/3341779467 >> 172.21.15.27:0/3341779467 conn(0x55fa053cca80 msgr2=0x55fa05417440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:56.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.643+0000 7fd93cc07740 1 -- 172.21.15.27:0/3341779467 shutdown_connections 2024-09-12T03:21:56.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.643+0000 7fd93cc07740 1 -- 172.21.15.27:0/3341779467 wait complete. 2024-09-12T03:21:56.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.644+0000 7fd93cc07740 1 -- start start 2024-09-12T03:21:56.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.644+0000 7fd93cc07740 1 Processor -- start 2024-09-12T03:21:56.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.645+0000 7fd93cc07740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fa053f5cf0 0x55fa0548a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:56.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.645+0000 7fd93cc07740 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x55fa053f9120 0x55fa05411f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:56.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.645+0000 7fd93cc07740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fa05413ae0 0x55fa054124b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:56.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.645+0000 7fd93cc07740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55fa053ccf20 con 0x55fa05413ae0 2024-09-12T03:21:56.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.645+0000 7fd93cc07740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55fa053fa380 con 0x55fa053f5cf0 2024-09-12T03:21:56.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.645+0000 7fd93cc07740 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x55fa053cd510 con 0x55fa053f9120 2024-09-12T03:21:56.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.645+0000 7fd93cc07740 10 client.0 _ll_register_callbacks cb 0x7fd924023070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:21:56.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.645+0000 7fd933fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fa05413ae0 0x55fa054124b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:56.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.645+0000 7fd933fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fa05413ae0 0x55fa054124b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39614/0 (socket says 172.21.15.27:39614) 2024-09-12T03:21:56.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.645+0000 7fd933fff640 1 -- 172.21.15.27:0/3436973366 learned_addr learned my addr 172.21.15.27:0/3436973366 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:21:56.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.645+0000 7fd9337fe640 1 --2- 172.21.15.27:0/3436973366 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fa053f5cf0 0x55fa0548a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:56.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.645+0000 7fd932ffd640 1 --2- 172.21.15.27:0/3436973366 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x55fa053f9120 0x55fa05411f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:56.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.645+0000 7fd933fff640 1 -- 172.21.15.27:0/3436973366 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x55fa053f9120 msgr2=0x55fa05411f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:56.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.645+0000 7fd933fff640 1 --2- 172.21.15.27:0/3436973366 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x55fa053f9120 0x55fa05411f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:56.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.645+0000 7fd933fff640 1 -- 172.21.15.27:0/3436973366 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fa053f5cf0 msgr2=0x55fa0548a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:56.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.645+0000 7fd933fff640 1 --2- 172.21.15.27:0/3436973366 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fa053f5cf0 0x55fa0548a0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:56.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.645+0000 7fd933fff640 1 -- 172.21.15.27:0/3436973366 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa05412d80 con 0x55fa05413ae0 2024-09-12T03:21:56.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.645+0000 7fd933fff640 1 --2- 172.21.15.27:0/3436973366 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fa05413ae0 0x55fa054124b0 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7fd92c0589b0 tx=0x7fd92c05c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3927abe95270ebec server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:56.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.645+0000 7fd9127fc640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-12T03:21:56.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.645+0000 7fd9127fc640 1 -- 172.21.15.27:0/3436973366 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd92c061320 con 0x55fa05413ae0 2024-09-12T03:21:56.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.645+0000 7fd9127fc640 1 -- 172.21.15.27:0/3436973366 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 5737+0+0 (secure 0 0 0) 0x7fd92c054540 con 0x55fa05413ae0 2024-09-12T03:21:56.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.645+0000 7fd93cc07740 1 -- 172.21.15.27:0/3436973366 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd92c054e60 con 0x55fa05413ae0 2024-09-12T03:21:56.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.646+0000 7fd9127fc640 1 -- 172.21.15.27:0/3436973366 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7fd92c0547a0 con 0x55fa05413ae0 2024-09-12T03:21:56.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.646+0000 7fd93cc07740 10 client.7464 fetch_fsmap learned FSMap version 67 2024-09-12T03:21:56.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.646+0000 7fd93cc07740 1 -- 172.21.15.27:0/3436973366 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=2+}) -- 0x55fa05429470 con 0x55fa05413ae0 2024-09-12T03:21:56.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.646+0000 7fd9127fc640 1 -- 172.21.15.27:0/3436973366 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd92c054e60 con 0x55fa05413ae0 2024-09-12T03:21:56.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.646+0000 7fd9127fc640 1 -- 172.21.15.27:0/3436973366 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 67) ==== 3502+0+0 (secure 0 0 0) 0x7fd92c077ae0 con 0x55fa05413ae0 2024-09-12T03:21:56.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.646+0000 7fd93cc07740 10 client.7464 fetch_fsmap finished waiting for FSMap version 67 2024-09-12T03:21:56.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.646+0000 7fd93cc07740 10 client.7464 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-12T03:21:56.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.646+0000 7fd93cc07740 20 client.7464 populate_metadata read hostname 'smithi027' 2024-09-12T03:21:56.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.646+0000 7fd93cc07740 1 --2- 172.21.15.27:0/3436973366 >> [v2:172.21.15.27:6833/1760654641,v1:172.21.15.27:6835/1760654641] conn(0x55fa05410250 0x55fa05410630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:56.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.646+0000 7fd93cc07740 4 client.7464 mds_command: new command op to 7302 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-12T03:21:56.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.646+0000 7fd93cc07740 1 -- 172.21.15.27:0/3436973366 --> [v2:172.21.15.27:6833/1760654641,v1:172.21.15.27:6835/1760654641] -- command(tid 0: {"prefix": "status"}) -- 0x55fa05338fa0 con 0x55fa05410250 2024-09-12T03:21:56.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.646+0000 7fd9337fe640 1 --2- 172.21.15.27:0/3436973366 >> [v2:172.21.15.27:6833/1760654641,v1:172.21.15.27:6835/1760654641] conn(0x55fa05410250 0x55fa05410630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:56.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.646+0000 7fd9127fc640 10 client.7464 ms_handle_connect on v2:172.21.15.27:6833/1760654641 2024-09-12T03:21:56.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.646+0000 7fd9337fe640 1 --2- 172.21.15.27:0/3436973366 >> [v2:172.21.15.27:6833/1760654641,v1:172.21.15.27:6835/1760654641] conn(0x55fa05410250 0x55fa05410630 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b8467fcb57ce403 server_cookie=252bbf59a190c5cc in_seq=0 out_seq=0 2024-09-12T03:21:56.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.646+0000 7fd9127fc640 1 -- 172.21.15.27:0/3436973366 <== mds.0 v2:172.21.15.27:6833/1760654641 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55fa05338fa0 con 0x55fa05410250 2024-09-12T03:21:56.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.646+0000 7fd9127fc640 10 client.7464 handle_command_reply: tid=0 2024-09-12T03:21:56.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.646+0000 7fd93cc07740 1 client.7464 shutdown 2024-09-12T03:21:56.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.647+0000 7fd93cc07740 1 -- 172.21.15.27:0/3436973366 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fa05413ae0 msgr2=0x55fa054124b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:56.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.647+0000 7fd93cc07740 1 --2- 172.21.15.27:0/3436973366 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fa05413ae0 0x55fa054124b0 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7fd92c0589b0 tx=0x7fd92c05c040 comp rx=0 tx=0).stop 2024-09-12T03:21:56.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.647+0000 7fd93cc07740 1 -- 172.21.15.27:0/3436973366 shutdown_connections 2024-09-12T03:21:56.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.647+0000 7fd93cc07740 1 --2- 172.21.15.27:0/3436973366 >> [v2:172.21.15.27:6833/1760654641,v1:172.21.15.27:6835/1760654641] conn(0x55fa05410250 0x55fa05410630 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:56.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.647+0000 7fd93cc07740 1 --2- 172.21.15.27:0/3436973366 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fa05413ae0 0x55fa054124b0 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:56.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.647+0000 7fd9127fc640 0 client.7464 ms_handle_reset on v2:172.21.15.27:6833/1760654641 2024-09-12T03:21:56.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.647+0000 7fd93cc07740 1 --2- 172.21.15.27:0/3436973366 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x55fa053f9120 0x55fa05411f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:56.657 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.647+0000 7fd93cc07740 1 --2- 172.21.15.27:0/3436973366 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fa053f5cf0 0x55fa0548a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:56.657 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.647+0000 7fd93cc07740 1 -- 172.21.15.27:0/3436973366 >> 172.21.15.27:0/3436973366 conn(0x55fa053cca80 msgr2=0x55fa05418f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:56.657 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.647+0000 7fd93cc07740 1 -- 172.21.15.27:0/3436973366 shutdown_connections 2024-09-12T03:21:56.657 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.647+0000 7fd93cc07740 1 -- 172.21.15.27:0/3436973366 wait complete. 2024-09-12T03:21:56.657 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.647+0000 7fd93cc07740 20 client.7464 trim_cache size 0 max 16384 2024-09-12T03:21:56.657 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-12T03:21:56.658 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_fsid": "d2df504e-2647-4385-a5db-9fdf69f487a0", 2024-09-12T03:21:56.658 INFO:teuthology.orchestra.run.smithi027.stdout: "whoami": 0, 2024-09-12T03:21:56.658 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 7302, 2024-09-12T03:21:56.658 INFO:teuthology.orchestra.run.smithi027.stdout: "want_state": "up:active", 2024-09-12T03:21:56.658 INFO:teuthology.orchestra.run.smithi027.stdout: "state": "up:active", 2024-09-12T03:21:56.658 INFO:teuthology.orchestra.run.smithi027.stdout: "fs_name": "cephfs", 2024-09-12T03:21:56.658 INFO:teuthology.orchestra.run.smithi027.stdout: "rank_uptime": 9.3843062219999993, 2024-09-12T03:21:56.658 INFO:teuthology.orchestra.run.smithi027.stdout: "mdsmap_epoch": 67, 2024-09-12T03:21:56.658 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch": 114, 2024-09-12T03:21:56.658 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch_barrier": 114, 2024-09-12T03:21:56.658 INFO:teuthology.orchestra.run.smithi027.stdout: "uptime": 14.415272089 2024-09-12T03:21:56.659 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-12T03:21:56.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.647+0000 7fd939e65640 1 -- 172.21.15.27:0/3463754147 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fd92006c220 msgr2=0x7fd92006e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:56.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.647+0000 7fd939e65640 1 --2- 172.21.15.27:0/3463754147 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fd92006c220 0x7fd92006e6e0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fd924000f60 tx=0x7fd924009040 comp rx=0 tx=0).stop 2024-09-12T03:21:56.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.647+0000 7fd939e65640 1 -- 172.21.15.27:0/3463754147 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd93415a1f0 msgr2=0x7fd9341593b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:56.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.647+0000 7fd939e65640 1 --2- 172.21.15.27:0/3463754147 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd93415a1f0 0x7fd9341593b0 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7fd92c00c700 tx=0x7fd92c00cbc0 comp rx=0 tx=0).stop 2024-09-12T03:21:56.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.647+0000 7fd939e65640 1 -- 172.21.15.27:0/3463754147 shutdown_connections 2024-09-12T03:21:56.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.647+0000 7fd939e65640 1 --2- 172.21.15.27:0/3463754147 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fd92006c220 0x7fd92006e6e0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:56.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.647+0000 7fd939e65640 1 --2- 172.21.15.27:0/3463754147 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd93415a1f0 0x7fd9341593b0 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:56.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.647+0000 7fd939e65640 1 --2- 172.21.15.27:0/3463754147 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd934072fc0 0x7fd934158e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:56.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.647+0000 7fd939e65640 1 --2- 172.21.15.27:0/3463754147 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fd93406fcf0 0x7fd934145860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:56.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.647+0000 7fd939e65640 1 -- 172.21.15.27:0/3463754147 >> 172.21.15.27:0/3463754147 conn(0x7fd93406ca30 msgr2=0x7fd9340718a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:56.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.648+0000 7fd939e65640 1 -- 172.21.15.27:0/3463754147 shutdown_connections 2024-09-12T03:21:56.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.648+0000 7fd939e65640 1 -- 172.21.15.27:0/3463754147 wait complete. 2024-09-12T03:21:56.662 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-12T03:21:56.765 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.764+0000 7f6a753a6640 1 Processor -- start 2024-09-12T03:21:56.765 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.764+0000 7f6a753a6640 1 -- start start 2024-09-12T03:21:56.774 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.765+0000 7f6a753a6640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f6a7015e780 0x7f6a700d3e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:56.774 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.765+0000 7f6a753a6640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f6a700d1990 0x7f6a700b3390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:56.774 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.765+0000 7f6a753a6640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6a700b6690 con 0x7f6a700d43b0 2024-09-12T03:21:56.774 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.765+0000 7f6a753a6640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f6a700b7950 con 0x7f6a700d1990 2024-09-12T03:21:56.774 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.765+0000 7f6a753a6640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f6a700b7210 con 0x7f6a7015e780 2024-09-12T03:21:56.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.765+0000 7f6a6f7fe640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f6a700d1990 0x7f6a700b3390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:56.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.765+0000 7f6a6f7fe640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f6a700d1990 0x7f6a700b3390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39628/0 (socket says 172.21.15.27:39628) 2024-09-12T03:21:56.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.765+0000 7f6a6f7fe640 1 -- 172.21.15.27:0/3735232607 learned_addr learned my addr 172.21.15.27:0/3735232607 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:21:56.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.765+0000 7f6a6f7fe640 1 -- 172.21.15.27:0/3735232607 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f6a7015e780 msgr2=0x7f6a700d3e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:56.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.765+0000 7f6a6f7fe640 1 --2- 172.21.15.27:0/3735232607 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f6a7015e780 0x7f6a700d3e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:56.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.765+0000 7f6a6f7fe640 1 -- 172.21.15.27:0/3735232607 >> v1:172.21.15.45:6789/0 conn(0x7f6a700d43b0 legacy=0x7f6a700d1280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:56.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.765+0000 7f6a6f7fe640 1 -- 172.21.15.27:0/3735232607 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a700b6970 con 0x7f6a700d1990 2024-09-12T03:21:56.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.766+0000 7f6a6f7fe640 1 --2- 172.21.15.27:0/3735232607 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6a700d1990 0x7f6a700b3390 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f6a68009e50 tx=0x7f6a6802a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdfcb6695c681ac7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:56.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.766+0000 7f6a6dffb640 1 -- 172.21.15.27:0/3735232607 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a68036070 con 0x7f6a700d1990 2024-09-12T03:21:56.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.766+0000 7f6a6dffb640 1 -- 172.21.15.27:0/3735232607 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a68031600 con 0x7f6a700d1990 2024-09-12T03:21:56.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.766+0000 7f6a6dffb640 1 -- 172.21.15.27:0/3735232607 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a680319d0 con 0x7f6a700d1990 2024-09-12T03:21:56.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.766+0000 7f6a753a6640 1 -- 172.21.15.27:0/3735232607 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6a700d1990 msgr2=0x7f6a700b3390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:56.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.766+0000 7f6a753a6640 1 --2- 172.21.15.27:0/3735232607 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6a700d1990 0x7f6a700b3390 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f6a68009e50 tx=0x7f6a6802a6c0 comp rx=0 tx=0).stop 2024-09-12T03:21:56.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.767+0000 7f6a753a6640 1 -- 172.21.15.27:0/3735232607 shutdown_connections 2024-09-12T03:21:56.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.767+0000 7f6a753a6640 1 --2- 172.21.15.27:0/3735232607 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6a700d1990 0x7f6a700b3390 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:56.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.767+0000 7f6a753a6640 1 --2- 172.21.15.27:0/3735232607 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f6a7015e780 0x7f6a700d3e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:56.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.767+0000 7f6a753a6640 1 -- 172.21.15.27:0/3735232607 >> 172.21.15.27:0/3735232607 conn(0x7f6a7006da60 msgr2=0x7f6a7006de70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:56.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.767+0000 7f6a753a6640 1 -- 172.21.15.27:0/3735232607 shutdown_connections 2024-09-12T03:21:56.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.767+0000 7f6a753a6640 1 -- 172.21.15.27:0/3735232607 wait complete. 2024-09-12T03:21:56.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.768+0000 7f6a753a6640 1 Processor -- start 2024-09-12T03:21:56.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.768+0000 7f6a753a6640 1 -- start start 2024-09-12T03:21:56.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.769+0000 7f6a753a6640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6a700d1990 0x7f6a7015da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:56.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.769+0000 7f6a753a6640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6a700d43b0 0x7f6a701f47e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:56.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.769+0000 7f6a753a6640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f6a7015e780 0x7f6a700ae120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:56.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.769+0000 7f6a753a6640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f6a700b6340 con 0x7f6a700d1990 2024-09-12T03:21:56.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.769+0000 7f6a753a6640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f6a700b6f30 con 0x7f6a700d43b0 2024-09-12T03:21:56.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.769+0000 7f6a753a6640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f6a7015eb60 con 0x7f6a7015e780 2024-09-12T03:21:56.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.769+0000 7f6a6effd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6a700d1990 0x7f6a7015da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:56.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.769+0000 7f6a6effd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6a700d1990 0x7f6a7015da30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39634/0 (socket says 172.21.15.27:39634) 2024-09-12T03:21:56.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.769+0000 7f6a6effd640 1 -- 172.21.15.27:0/282901371 learned_addr learned my addr 172.21.15.27:0/282901371 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:21:56.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.769+0000 7f6a6effd640 1 -- 172.21.15.27:0/282901371 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f6a7015e780 msgr2=0x7f6a700ae120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:56.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.769+0000 7f6a6f7fe640 1 --2- 172.21.15.27:0/282901371 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f6a7015e780 0x7f6a700ae120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:56.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.769+0000 7f6a6e7fc640 1 --2- 172.21.15.27:0/282901371 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6a700d43b0 0x7f6a701f47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:56.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.769+0000 7f6a6effd640 1 --2- 172.21.15.27:0/282901371 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f6a7015e780 0x7f6a700ae120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:56.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.769+0000 7f6a6effd640 1 -- 172.21.15.27:0/282901371 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6a700d43b0 msgr2=0x7f6a701f47e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:56.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.769+0000 7f6a6effd640 1 --2- 172.21.15.27:0/282901371 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6a700d43b0 0x7f6a701f47e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:56.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.769+0000 7f6a6effd640 1 -- 172.21.15.27:0/282901371 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a700ac470 con 0x7f6a700d1990 2024-09-12T03:21:56.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.769+0000 7f6a6e7fc640 1 --2- 172.21.15.27:0/282901371 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6a700d43b0 0x7f6a701f47e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:21:56.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.769+0000 7f6a6f7fe640 1 --2- 172.21.15.27:0/282901371 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f6a7015e780 0x7f6a700ae120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:21:56.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.769+0000 7f6a6effd640 1 --2- 172.21.15.27:0/282901371 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6a700d1990 0x7f6a7015da30 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f6a6000b410 tx=0x7f6a6000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:56.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.769+0000 7f6a577fe640 1 -- 172.21.15.27:0/282901371 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a60015020 con 0x7f6a700d1990 2024-09-12T03:21:56.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.769+0000 7f6a577fe640 1 -- 172.21.15.27:0/282901371 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a60011850 con 0x7f6a700d1990 2024-09-12T03:21:56.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.769+0000 7f6a753a6640 1 -- 172.21.15.27:0/282901371 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a700ac750 con 0x7f6a700d1990 2024-09-12T03:21:56.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.769+0000 7f6a577fe640 1 -- 172.21.15.27:0/282901371 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a60011c40 con 0x7f6a700d1990 2024-09-12T03:21:56.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.769+0000 7f6a753a6640 1 -- 172.21.15.27:0/282901371 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a700acd10 con 0x7f6a700d1990 2024-09-12T03:21:56.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.770+0000 7f6a753a6640 1 -- 172.21.15.27:0/282901371 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a700d3e70 con 0x7f6a700d1990 2024-09-12T03:21:56.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.771+0000 7f6a577fe640 1 -- 172.21.15.27:0/282901371 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 19) ==== 90855+0+0 (secure 0 0 0) 0x7f6a60018020 con 0x7f6a700d1990 2024-09-12T03:21:56.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.771+0000 7f6a577fe640 1 --2- 172.21.15.27:0/282901371 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f6a5c06c220 0x7f6a5c06e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:56.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.772+0000 7f6a577fe640 1 -- 172.21.15.27:0/282901371 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5737+0+0 (secure 0 0 0) 0x7f6a6008b500 con 0x7f6a700d1990 2024-09-12T03:21:56.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.772+0000 7f6a6e7fc640 1 --2- 172.21.15.27:0/282901371 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f6a5c06c220 0x7f6a5c06e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:56.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.772+0000 7f6a6e7fc640 1 --2- 172.21.15.27:0/282901371 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f6a5c06c220 0x7f6a5c06e6e0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f6a700b4e90 tx=0x7f6a64009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:56.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.776+0000 7f6a577fe640 1 -- 172.21.15.27:0/282901371 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f6a6001f020 con 0x7f6a700d1990 2024-09-12T03:21:56.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.986+0000 7f6a753a6640 1 -- 172.21.15.27:0/282901371 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f6a70161f40 con 0x7f6a700d1990 2024-09-12T03:21:56.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.986+0000 7f6a577fe640 1 -- 172.21.15.27:0/282901371 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v67) ==== 53+0+73 (secure 0 0 0) 0x7f6a600172c0 con 0x7f6a700d1990 2024-09-12T03:21:56.988 INFO:teuthology.orchestra.run.smithi027.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-12T03:21:56.989 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.988+0000 7f6a753a6640 1 -- 172.21.15.27:0/282901371 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f6a5c06c220 msgr2=0x7f6a5c06e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:56.989 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.988+0000 7f6a753a6640 1 --2- 172.21.15.27:0/282901371 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f6a5c06c220 0x7f6a5c06e6e0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f6a700b4e90 tx=0x7f6a64009040 comp rx=0 tx=0).stop 2024-09-12T03:21:56.990 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.988+0000 7f6a753a6640 1 -- 172.21.15.27:0/282901371 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6a700d1990 msgr2=0x7f6a7015da30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:56.990 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.988+0000 7f6a753a6640 1 --2- 172.21.15.27:0/282901371 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6a700d1990 0x7f6a7015da30 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f6a6000b410 tx=0x7f6a6000b8d0 comp rx=0 tx=0).stop 2024-09-12T03:21:56.990 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.988+0000 7f6a753a6640 1 -- 172.21.15.27:0/282901371 shutdown_connections 2024-09-12T03:21:56.990 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.988+0000 7f6a753a6640 1 --2- 172.21.15.27:0/282901371 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f6a7015e780 0x7f6a700ae120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:56.990 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.988+0000 7f6a753a6640 1 --2- 172.21.15.27:0/282901371 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6a700d43b0 0x7f6a701f47e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:56.990 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.988+0000 7f6a753a6640 1 --2- 172.21.15.27:0/282901371 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f6a5c06c220 0x7f6a5c06e6e0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:56.990 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.988+0000 7f6a753a6640 1 --2- 172.21.15.27:0/282901371 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6a700d1990 0x7f6a7015da30 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:56.990 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.988+0000 7f6a753a6640 1 -- 172.21.15.27:0/282901371 >> 172.21.15.27:0/282901371 conn(0x7f6a7006da60 msgr2=0x7f6a701e9080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:56.990 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.988+0000 7f6a753a6640 1 -- 172.21.15.27:0/282901371 shutdown_connections 2024-09-12T03:21:56.990 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:56.988+0000 7f6a753a6640 1 -- 172.21.15.27:0/282901371 wait complete. 2024-09-12T03:21:57.000 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-12T03:21:57.000 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-12T03:21:57.000 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi027.front.sepia.ceph.com 2024-09-12T03:21:57.000 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-12T03:21:57.000 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-12T03:21:57.001 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-12T03:21:57.001 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-12T03:21:57.001 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-12T03:21:57.001 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-12T03:21:57.001 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:21:57.001 DEBUG:teuthology.orchestra.run.smithi027:> ip addr 2024-09-12T03:21:57.018 INFO:teuthology.orchestra.run.smithi027.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-12T03:21:57.018 INFO:teuthology.orchestra.run.smithi027.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-12T03:21:57.018 INFO:teuthology.orchestra.run.smithi027.stdout: inet 127.0.0.1/8 scope host lo 2024-09-12T03:21:57.018 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:21:57.018 INFO:teuthology.orchestra.run.smithi027.stdout: inet6 ::1/128 scope host 2024-09-12T03:21:57.018 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:21:57.018 INFO:teuthology.orchestra.run.smithi027.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-12T03:21:57.018 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether 0c:c4:7a:6b:fe:c8 brd ff:ff:ff:ff:ff:ff 2024-09-12T03:21:57.018 INFO:teuthology.orchestra.run.smithi027.stdout: altname enp7s0f0 2024-09-12T03:21:57.019 INFO:teuthology.orchestra.run.smithi027.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-12T03:21:57.019 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether 0c:c4:7a:6b:fe:c9 brd ff:ff:ff:ff:ff:ff 2024-09-12T03:21:57.019 INFO:teuthology.orchestra.run.smithi027.stdout: altname enp7s0f1 2024-09-12T03:21:57.019 INFO:teuthology.orchestra.run.smithi027.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-12T03:21:57.019 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether ac:1f:6b:f8:39:5a brd ff:ff:ff:ff:ff:ff 2024-09-12T03:21:57.019 INFO:teuthology.orchestra.run.smithi027.stdout: altname enp1s0f0 2024-09-12T03:21:57.019 INFO:teuthology.orchestra.run.smithi027.stdout: inet 172.21.15.27/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-12T03:21:57.019 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft 42472sec preferred_lft 42472sec 2024-09-12T03:21:57.020 INFO:teuthology.orchestra.run.smithi027.stdout: inet6 fe80::ae1f:6bff:fef8:395a/64 scope link 2024-09-12T03:21:57.020 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:21:57.020 INFO:teuthology.orchestra.run.smithi027.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-12T03:21:57.020 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether ac:1f:6b:f8:39:5b brd ff:ff:ff:ff:ff:ff 2024-09-12T03:21:57.020 INFO:teuthology.orchestra.run.smithi027.stdout: altname enp1s0f1 2024-09-12T03:21:57.020 INFO:teuthology.orchestra.run.smithi027.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-12T03:21:57.020 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether 2e:7f:89:6c:33:a0 brd ff:ff:ff:ff:ff:ff 2024-09-12T03:21:57.020 INFO:teuthology.orchestra.run.smithi027.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-12T03:21:57.020 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:21:57.021 INFO:teuthology.orchestra.run.smithi027.stdout: inet6 fe80::c84b:19ff:fef0:ca74/64 scope link 2024-09-12T03:21:57.021 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:21:57.021 INFO:teuthology.orchestra.run.smithi027.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-12T03:21:57.021 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether 42:40:d7:5c:7a:8e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-12T03:21:57.022 INFO:teuthology.orchestra.run.smithi027.stdout: inet6 fe80::4040:d7ff:fe5c:7a8e/64 scope link 2024-09-12T03:21:57.022 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:21:57.022 INFO:teuthology.orchestra.run.smithi027.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-12T03:21:57.022 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether 2e:7f:89:6c:33:a0 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-12T03:21:57.022 INFO:teuthology.orchestra.run.smithi027.stdout: inet6 fe80::2c7f:89ff:fe6c:33a0/64 scope link 2024-09-12T03:21:57.023 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:21:57.023 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:21:57.023 DEBUG:teuthology.orchestra.run.smithi027:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-12T03:21:57.082 INFO:teuthology.orchestra.run.smithi027.stdout:1 2024-09-12T03:21:57.084 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:21:57.084 DEBUG:teuthology.orchestra.run.smithi027:> route 2024-09-12T03:21:57.141 INFO:teuthology.orchestra.run.smithi027.stdout:Kernel IP routing table 2024-09-12T03:21:57.141 INFO:teuthology.orchestra.run.smithi027.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-12T03:21:57.141 INFO:teuthology.orchestra.run.smithi027.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-12T03:21:57.142 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-12T03:21:57.142 INFO:teuthology.orchestra.run.smithi027.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-12T03:21:57.142 DEBUG:teuthology.orchestra.run.smithi027:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-12T03:21:57.142 DEBUG:teuthology.orchestra.run.smithi027:> set -e 2024-09-12T03:21:57.142 DEBUG:teuthology.orchestra.run.smithi027:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-12T03:21:57.142 DEBUG:teuthology.orchestra.run.smithi027:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-12T03:21:57.142 DEBUG:teuthology.orchestra.run.smithi027:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-12T03:21:57.142 DEBUG:teuthology.orchestra.run.smithi027:> ') 2024-09-12T03:21:57.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:57 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T03:21:57.264 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T03:21:57.271 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:21:57.271 DEBUG:teuthology.orchestra.run.smithi027:> ip netns list 2024-09-12T03:21:57.287 INFO:teuthology.orchestra.run.smithi027.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-12T03:21:57.287 INFO:teuthology.orchestra.run.smithi027.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-12T03:21:57.287 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-12T03:21:57.287 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:21:57.287 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip link set brx.0 up 2024-09-12T03:21:57.352 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-12T03:21:57.353 DEBUG:teuthology.orchestra.run.smithi027:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-12T03:21:57.408 INFO:teuthology.orchestra.run.smithi027.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-12T03:21:57.409 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-12T03:21:57.409 DEBUG:teuthology.orchestra.run.smithi027:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-12T03:21:57.464 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-09-12T03:21:57.464 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:21:57.464 DEBUG:teuthology.orchestra.run.smithi027:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-12T03:21:57.549 INFO:teuthology.orchestra.run.smithi027.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-12T03:21:57.549 INFO:teuthology.orchestra.run.smithi027.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-12T03:21:57.557 INFO:teuthology.orchestra.run.smithi027.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-12T03:21:57.557 INFO:teuthology.orchestra.run.smithi027.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-12T03:21:57.557 INFO:teuthology.orchestra.run.smithi027.stdout:mount.ceph: resolved to: "172.21.15.27:3300,172.21.15.45:3300,172.21.15.45:3301" 2024-09-12T03:21:57.557 INFO:teuthology.orchestra.run.smithi027.stdout:mount.ceph: trying mount with new device syntax: 0@d2df504e-2647-4385-a5db-9fdf69f487a0.cephfs=/ 2024-09-12T03:21:57.557 INFO:teuthology.orchestra.run.smithi027.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.27:3300/172.21.15.45:3300/172.21.15.45:3301" will pass to kernel 2024-09-12T03:21:57.557 INFO:teuthology.orchestra.run.smithi027.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-12T03:21:57.557 INFO:teuthology.orchestra.run.smithi027.stdout: You just mounted a file system that supports labels which does not 2024-09-12T03:21:57.558 INFO:teuthology.orchestra.run.smithi027.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-12T03:21:57.558 INFO:teuthology.orchestra.run.smithi027.stdout: applications will generate AVC messages and not be allowed access to 2024-09-12T03:21:57.558 INFO:teuthology.orchestra.run.smithi027.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-12T03:21:57.559 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-12T03:21:57.559 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:21:57.559 DEBUG:teuthology.orchestra.run.smithi027:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-12T03:21:57.585 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-12T03:21:57.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:57 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-12T03:21:57.680 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T03:21:57.686 INFO:teuthology.orchestra.run.smithi027.stdout:client6853 2024-09-12T03:21:57.686 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-12T03:21:57.686 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T03:21:57.702 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-12T03:21:57.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:57.844+0000 7fe5fec8d640 1 Processor -- start 2024-09-12T03:21:57.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:57.844+0000 7fe5fec8d640 1 -- start start 2024-09-12T03:21:57.847 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:57.845+0000 7fe5fec8d640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fe5f815a8c0 0x7fe5f80cdb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:57.847 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:57.845+0000 7fe5fec8d640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe5f815aca0 0x7fe5f80b4710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:57.847 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:57.845+0000 7fe5fec8d640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe5f806daf0 con 0x7fe5f8071be0 2024-09-12T03:21:57.847 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:57.845+0000 7fe5fec8d640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fe5f806d8a0 con 0x7fe5f815a8c0 2024-09-12T03:21:57.847 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:57.845+0000 7fe5fec8d640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fe5f806ddd0 con 0x7fe5f815aca0 2024-09-12T03:21:57.847 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:57.845+0000 7fe5fd22a640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe5f815aca0 0x7fe5f80b4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:57.847 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:57.845+0000 7fe5fca29640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fe5f815a8c0 0x7fe5f80cdb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:57.847 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:57.845+0000 7fe5f7fff640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fe5f8071be0 0x7fe5f806e8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.27:36918/0 (socket says 172.21.15.27:36918) 2024-09-12T03:21:57.847 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:57.845+0000 7fe5f7fff640 1 -- 172.21.15.27:0/3956802933 learned_addr learned my addr 172.21.15.27:0/3956802933 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:21:57.847 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:57.845+0000 7fe5fca29640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fe5f815a8c0 0x7fe5f80cdb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.27:54200/0 (socket says 172.21.15.27:54200) 2024-09-12T03:21:57.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:57.846+0000 7fe5fca29640 1 -- 172.21.15.27:0/3956802933 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe5f815aca0 msgr2=0x7fe5f80b4710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:57.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:57.846+0000 7fe5fca29640 1 --2- 172.21.15.27:0/3956802933 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe5f815aca0 0x7fe5f80b4710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:57.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:57.846+0000 7fe5fca29640 1 -- 172.21.15.27:0/3956802933 >> v1:172.21.15.45:6789/0 conn(0x7fe5f8071be0 legacy=0x7fe5f806e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:57.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:57.846+0000 7fe5fca29640 1 -- 172.21.15.27:0/3956802933 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5f806e0b0 con 0x7fe5f815a8c0 2024-09-12T03:21:57.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:57.847+0000 7fe5fca29640 1 --2- 172.21.15.27:0/3956802933 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe5f815a8c0 0x7fe5f80cdb30 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fe5f806da20 tx=0x7fe5e802a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=869c22d3cef6a141 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:57.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:57.847+0000 7fe5f77fe640 1 -- 172.21.15.27:0/3956802933 <== mon.1 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5e8037070 con 0x7fe5f815a8c0 2024-09-12T03:21:57.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:57.847+0000 7fe5f77fe640 1 -- 172.21.15.27:0/3956802933 <== mon.1 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5e8032440 con 0x7fe5f815a8c0 2024-09-12T03:21:57.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:57.847+0000 7fe5f77fe640 1 -- 172.21.15.27:0/3956802933 <== mon.1 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5e8032810 con 0x7fe5f815a8c0 2024-09-12T03:21:57.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:57.847+0000 7fe5fec8d640 1 -- 172.21.15.27:0/3956802933 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe5f815a8c0 msgr2=0x7fe5f80cdb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:57.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:57.847+0000 7fe5fec8d640 1 --2- 172.21.15.27:0/3956802933 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe5f815a8c0 0x7fe5f80cdb30 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fe5f806da20 tx=0x7fe5e802a180 comp rx=0 tx=0).stop 2024-09-12T03:21:57.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:57.847+0000 7fe5fec8d640 1 -- 172.21.15.27:0/3956802933 shutdown_connections 2024-09-12T03:21:57.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:57.847+0000 7fe5fec8d640 1 --2- 172.21.15.27:0/3956802933 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe5f815aca0 0x7fe5f80b4710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:57.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:57.847+0000 7fe5fec8d640 1 --2- 172.21.15.27:0/3956802933 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe5f815a8c0 0x7fe5f80cdb30 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:57.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:57.847+0000 7fe5fec8d640 1 -- 172.21.15.27:0/3956802933 >> 172.21.15.27:0/3956802933 conn(0x7fe5f8062c60 msgr2=0x7fe5f806bb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:57.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:57.847+0000 7fe5fec8d640 1 -- 172.21.15.27:0/3956802933 shutdown_connections 2024-09-12T03:21:57.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:57.847+0000 7fe5fec8d640 1 -- 172.21.15.27:0/3956802933 wait complete. 2024-09-12T03:21:57.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:57.849+0000 7fe5fec8d640 1 Processor -- start 2024-09-12T03:21:57.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:57.849+0000 7fe5fec8d640 1 -- start start 2024-09-12T03:21:57.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:57.849+0000 7fe5fec8d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe5f8071be0 0x7fe5f80b4980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:57.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:57.849+0000 7fe5fec8d640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe5f815a8c0 0x7fe5f80ce940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:57.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:57.849+0000 7fe5fec8d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe5f815aca0 0x7fe5f814dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:57.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:57.849+0000 7fe5fec8d640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fe5f806f140 con 0x7fe5f815aca0 2024-09-12T03:21:57.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:57.850+0000 7fe5fec8d640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe5f806efc0 con 0x7fe5f8071be0 2024-09-12T03:21:57.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:57.850+0000 7fe5fec8d640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fe5f80b3560 con 0x7fe5f815a8c0 2024-09-12T03:21:57.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:57.850+0000 7fe5fd22a640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe5f815aca0 0x7fe5f814dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:57.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:57.850+0000 7fe5fd22a640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe5f815aca0 0x7fe5f814dc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39640/0 (socket says 172.21.15.27:39640) 2024-09-12T03:21:57.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:57.850+0000 7fe5fd22a640 1 -- 172.21.15.27:0/2065948674 learned_addr learned my addr 172.21.15.27:0/2065948674 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:21:57.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:57.850+0000 7fe5f7fff640 1 --2- 172.21.15.27:0/2065948674 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe5f815a8c0 0x7fe5f80ce940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:57.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:57.850+0000 7fe5fca29640 1 --2- 172.21.15.27:0/2065948674 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe5f8071be0 0x7fe5f80b4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:57.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:57.850+0000 7fe5fd22a640 1 -- 172.21.15.27:0/2065948674 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe5f815a8c0 msgr2=0x7fe5f80ce940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:57.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:57.850+0000 7fe5fd22a640 1 --2- 172.21.15.27:0/2065948674 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe5f815a8c0 0x7fe5f80ce940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:57.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:57.850+0000 7fe5fd22a640 1 -- 172.21.15.27:0/2065948674 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe5f8071be0 msgr2=0x7fe5f80b4980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:57.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:57.850+0000 7fe5fd22a640 1 --2- 172.21.15.27:0/2065948674 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe5f8071be0 0x7fe5f80b4980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:57.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:57.850+0000 7fe5fd22a640 1 -- 172.21.15.27:0/2065948674 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5f814f3f0 con 0x7fe5f815aca0 2024-09-12T03:21:57.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:57.850+0000 7fe5fd22a640 1 --2- 172.21.15.27:0/2065948674 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe5f815aca0 0x7fe5f814dc00 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7fe5f000e7c0 tx=0x7fe5f000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:57.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:57.850+0000 7fe5f57fa640 1 -- 172.21.15.27:0/2065948674 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5f0017070 con 0x7fe5f815aca0 2024-09-12T03:21:57.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:57.850+0000 7fe5fec8d640 1 -- 172.21.15.27:0/2065948674 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5f814f6d0 con 0x7fe5f815aca0 2024-09-12T03:21:57.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:57.850+0000 7fe5f57fa640 1 -- 172.21.15.27:0/2065948674 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5f000c730 con 0x7fe5f815aca0 2024-09-12T03:21:57.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:57.850+0000 7fe5f57fa640 1 -- 172.21.15.27:0/2065948674 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5f000cb20 con 0x7fe5f815aca0 2024-09-12T03:21:57.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:57.851+0000 7fe5fec8d640 1 -- 172.21.15.27:0/2065948674 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5f8152e00 con 0x7fe5f815aca0 2024-09-12T03:21:57.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:57.852+0000 7fe5fec8d640 1 -- 172.21.15.27:0/2065948674 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5f8072ba0 con 0x7fe5f815aca0 2024-09-12T03:21:57.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:57.852+0000 7fe5f57fa640 1 -- 172.21.15.27:0/2065948674 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 19) ==== 90855+0+0 (secure 0 0 0) 0x7fe5f001a070 con 0x7fe5f815aca0 2024-09-12T03:21:57.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:57.853+0000 7fe5f57fa640 1 --2- 172.21.15.27:0/2065948674 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fe5d806c220 0x7fe5d806e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:57.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:57.853+0000 7fe5f57fa640 1 -- 172.21.15.27:0/2065948674 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5737+0+0 (secure 0 0 0) 0x7fe5f008b610 con 0x7fe5f815aca0 2024-09-12T03:21:57.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:57.853+0000 7fe5fca29640 1 --2- 172.21.15.27:0/2065948674 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fe5d806c220 0x7fe5d806e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:57.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:57.853+0000 7fe5fca29640 1 --2- 172.21.15.27:0/2065948674 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fe5d806c220 0x7fe5d806e6e0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fe5e8034000 tx=0x7fe5e802e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:57.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:57.857+0000 7fe5f57fa640 1 -- 172.21.15.27:0/2065948674 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fe5f0059480 con 0x7fe5f815aca0 2024-09-12T03:21:58.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:58.072+0000 7fe5fec8d640 1 -- 172.21.15.27:0/2065948674 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fe5f80cdc90 con 0x7fe5f815aca0 2024-09-12T03:21:58.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:58.072+0000 7fe5f57fa640 1 -- 172.21.15.27:0/2065948674 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v67) ==== 53+0+73 (secure 0 0 0) 0x7fe5f005d780 con 0x7fe5f815aca0 2024-09-12T03:21:58.073 INFO:teuthology.orchestra.run.smithi027.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-12T03:21:58.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:58.074+0000 7fe5fec8d640 1 -- 172.21.15.27:0/2065948674 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fe5d806c220 msgr2=0x7fe5d806e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:58.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:58.074+0000 7fe5fec8d640 1 --2- 172.21.15.27:0/2065948674 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fe5d806c220 0x7fe5d806e6e0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fe5e8034000 tx=0x7fe5e802e000 comp rx=0 tx=0).stop 2024-09-12T03:21:58.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:58.074+0000 7fe5fec8d640 1 -- 172.21.15.27:0/2065948674 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe5f815aca0 msgr2=0x7fe5f814dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:58.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:58.074+0000 7fe5fec8d640 1 --2- 172.21.15.27:0/2065948674 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe5f815aca0 0x7fe5f814dc00 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7fe5f000e7c0 tx=0x7fe5f000ec80 comp rx=0 tx=0).stop 2024-09-12T03:21:58.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:58.074+0000 7fe5fec8d640 1 -- 172.21.15.27:0/2065948674 shutdown_connections 2024-09-12T03:21:58.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:58.074+0000 7fe5fec8d640 1 --2- 172.21.15.27:0/2065948674 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fe5d806c220 0x7fe5d806e6e0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:58.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:58.074+0000 7fe5fec8d640 1 --2- 172.21.15.27:0/2065948674 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe5f815aca0 0x7fe5f814dc00 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:58.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:58.074+0000 7fe5fec8d640 1 --2- 172.21.15.27:0/2065948674 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe5f815a8c0 0x7fe5f80ce940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:58.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:58.074+0000 7fe5fec8d640 1 --2- 172.21.15.27:0/2065948674 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe5f8071be0 0x7fe5f80b4980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:58.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:58.074+0000 7fe5fec8d640 1 -- 172.21.15.27:0/2065948674 >> 172.21.15.27:0/2065948674 conn(0x7fe5f8062c60 msgr2=0x7fe5f806d010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:58.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:58.074+0000 7fe5fec8d640 1 -- 172.21.15.27:0/2065948674 shutdown_connections 2024-09-12T03:21:58.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:58.075+0000 7fe5fec8d640 1 -- 172.21.15.27:0/2065948674 wait complete. 2024-09-12T03:21:58.086 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-12T03:21:58.087 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-12T03:21:58.087 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi045.front.sepia.ceph.com 2024-09-12T03:21:58.087 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-12T03:21:58.087 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-12T03:21:58.087 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-12T03:21:58.087 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-12T03:21:58.087 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-12T03:21:58.087 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-12T03:21:58.087 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:21:58.087 DEBUG:teuthology.orchestra.run.smithi045:> ip addr 2024-09-12T03:21:58.108 INFO:teuthology.orchestra.run.smithi045.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-12T03:21:58.108 INFO:teuthology.orchestra.run.smithi045.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-12T03:21:58.108 INFO:teuthology.orchestra.run.smithi045.stdout: inet 127.0.0.1/8 scope host lo 2024-09-12T03:21:58.108 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:21:58.109 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 ::1/128 scope host 2024-09-12T03:21:58.109 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:21:58.109 INFO:teuthology.orchestra.run.smithi045.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-12T03:21:58.109 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:6b:fd:a6 brd ff:ff:ff:ff:ff:ff 2024-09-12T03:21:58.109 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp7s0f0 2024-09-12T03:21:58.109 INFO:teuthology.orchestra.run.smithi045.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-12T03:21:58.109 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:6b:fd:a7 brd ff:ff:ff:ff:ff:ff 2024-09-12T03:21:58.109 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp7s0f1 2024-09-12T03:21:58.109 INFO:teuthology.orchestra.run.smithi045.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-12T03:21:58.109 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:bd:12:be brd ff:ff:ff:ff:ff:ff 2024-09-12T03:21:58.109 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp1s0f0 2024-09-12T03:21:58.109 INFO:teuthology.orchestra.run.smithi045.stdout: inet 172.21.15.45/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-12T03:21:58.110 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft 42415sec preferred_lft 42415sec 2024-09-12T03:21:58.110 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::ec4:7aff:febd:12be/64 scope link 2024-09-12T03:21:58.110 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:21:58.110 INFO:teuthology.orchestra.run.smithi045.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-12T03:21:58.110 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:bd:12:bf brd ff:ff:ff:ff:ff:ff 2024-09-12T03:21:58.110 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp1s0f1 2024-09-12T03:21:58.110 INFO:teuthology.orchestra.run.smithi045.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-12T03:21:58.110 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 52:97:33:5c:60:1b brd ff:ff:ff:ff:ff:ff 2024-09-12T03:21:58.110 INFO:teuthology.orchestra.run.smithi045.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-12T03:21:58.110 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:21:58.111 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::d8c2:5dff:fe26:78c8/64 scope link 2024-09-12T03:21:58.111 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:21:58.111 INFO:teuthology.orchestra.run.smithi045.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-12T03:21:58.111 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 5a:ec:d7:40:ef:75 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-12T03:21:58.111 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::58ec:d7ff:fe40:ef75/64 scope link 2024-09-12T03:21:58.111 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:21:58.111 INFO:teuthology.orchestra.run.smithi045.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-12T03:21:58.111 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 52:97:33:5c:60:1b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-12T03:21:58.111 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::5097:33ff:fe5c:601b/64 scope link 2024-09-12T03:21:58.111 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:21:58.112 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:21:58.112 DEBUG:teuthology.orchestra.run.smithi045:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-12T03:21:58.177 INFO:teuthology.orchestra.run.smithi045.stdout:1 2024-09-12T03:21:58.179 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:21:58.179 DEBUG:teuthology.orchestra.run.smithi045:> route 2024-09-12T03:21:58.239 INFO:teuthology.orchestra.run.smithi045.stdout:Kernel IP routing table 2024-09-12T03:21:58.239 INFO:teuthology.orchestra.run.smithi045.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-12T03:21:58.239 INFO:teuthology.orchestra.run.smithi045.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-12T03:21:58.239 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-12T03:21:58.239 INFO:teuthology.orchestra.run.smithi045.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-12T03:21:58.239 DEBUG:teuthology.orchestra.run.smithi045:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-12T03:21:58.239 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-09-12T03:21:58.239 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-12T03:21:58.240 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-12T03:21:58.240 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-12T03:21:58.240 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-09-12T03:21:58.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:21:58 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T03:21:58.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:21:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T03:21:58.376 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:21:58.376 DEBUG:teuthology.orchestra.run.smithi045:> ip netns list 2024-09-12T03:21:58.397 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-09-12T03:21:58.397 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-12T03:21:58.397 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-12T03:21:58.397 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:21:58.398 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link set brx.0 up 2024-09-12T03:21:58.464 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-12T03:21:58.464 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-12T03:21:58.521 INFO:teuthology.orchestra.run.smithi045.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-12T03:21:58.521 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-12T03:21:58.521 DEBUG:teuthology.orchestra.run.smithi045:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-12T03:21:58.577 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2024-09-12T03:21:58.577 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:21:58.577 DEBUG:teuthology.orchestra.run.smithi045:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-12T03:21:58.663 INFO:teuthology.orchestra.run.smithi045.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-12T03:21:58.664 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-12T03:21:58.672 INFO:teuthology.orchestra.run.smithi045.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-12T03:21:58.672 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-12T03:21:58.672 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: resolved to: "172.21.15.27:3300,172.21.15.45:3300,172.21.15.45:3301" 2024-09-12T03:21:58.672 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: trying mount with new device syntax: 1@d2df504e-2647-4385-a5db-9fdf69f487a0.cephfs=/ 2024-09-12T03:21:58.672 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.27:3300/172.21.15.45:3300/172.21.15.45:3301" will pass to kernel 2024-09-12T03:21:58.673 INFO:teuthology.orchestra.run.smithi045.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-09-12T03:21:58.673 INFO:teuthology.orchestra.run.smithi045.stdout: You just mounted a file system that supports labels which does not 2024-09-12T03:21:58.673 INFO:teuthology.orchestra.run.smithi045.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-12T03:21:58.673 INFO:teuthology.orchestra.run.smithi045.stdout: applications will generate AVC messages and not be allowed access to 2024-09-12T03:21:58.673 INFO:teuthology.orchestra.run.smithi045.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-12T03:21:58.674 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-12T03:21:58.674 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:21:58.674 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-12T03:21:58.699 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-12T03:21:58.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:21:58 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-12T03:21:58.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:21:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T03:21:58.798 INFO:teuthology.orchestra.run.smithi045.stdout:client6865 2024-09-12T03:21:58.798 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-12T03:21:58.798 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T03:21:58.815 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-09-12T03:21:58.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:58.924+0000 7f58cc342640 1 Processor -- start 2024-09-12T03:21:58.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:58.924+0000 7f58cc342640 1 -- start start 2024-09-12T03:21:58.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:58.925+0000 7f58cc342640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f58c4162040 0x7f58c40d1da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:58.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:58.925+0000 7f58cc342640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f58c406dfd0 0x7f58c40b7a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:58.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:58.925+0000 7f58cc342640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f58c40d2e30 con 0x7f58c40d3790 2024-09-12T03:21:58.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:58.925+0000 7f58cc342640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f58c40b7f90 con 0x7f58c406dfd0 2024-09-12T03:21:58.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:58.925+0000 7f58cc342640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f58c4162420 con 0x7f58c4162040 2024-09-12T03:21:58.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:58.925+0000 7f58c98dd640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f58c40d3790 0x7f58c406d8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:56386/0 (socket says 172.21.15.27:56386) 2024-09-12T03:21:58.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:58.925+0000 7f58c98dd640 1 -- 172.21.15.27:0/3119290724 learned_addr learned my addr 172.21.15.27:0/3119290724 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:21:58.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:58.925+0000 7f58ca8df640 1 --2- 172.21.15.27:0/3119290724 >> v2:172.21.15.45:3300/0 conn(0x7f58c406dfd0 0x7f58c40b7a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:58.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:58.925+0000 7f58ca0de640 1 --2- 172.21.15.27:0/3119290724 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f58c4162040 0x7f58c40d1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:58.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:58.926+0000 7f58c90dc640 1 -- 172.21.15.27:0/3119290724 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4065679191 0 0) 0x7f58c40d2e30 con 0x7f58c40d3790 2024-09-12T03:21:58.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:58.926+0000 7f58c90dc640 1 -- 172.21.15.27:0/3119290724 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f58ac003520 con 0x7f58c40d3790 2024-09-12T03:21:58.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:58.926+0000 7f58ca8df640 1 -- 172.21.15.27:0/3119290724 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f58c4162040 msgr2=0x7f58c40d1da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:58.930 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:58.926+0000 7f58ca8df640 1 --2- 172.21.15.27:0/3119290724 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f58c4162040 0x7f58c40d1da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:58.930 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:58.926+0000 7f58ca8df640 1 -- 172.21.15.27:0/3119290724 >> v1:172.21.15.27:6789/0 conn(0x7f58c40d3790 legacy=0x7f58c406d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:58.930 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:58.926+0000 7f58ca8df640 1 -- 172.21.15.27:0/3119290724 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58c40d30d0 con 0x7f58c406dfd0 2024-09-12T03:21:58.930 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:58.926+0000 7f58ca0de640 1 --2- 172.21.15.27:0/3119290724 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f58c4162040 0x7f58c40d1da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:21:58.930 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:58.927+0000 7f58ca8df640 1 --2- 172.21.15.27:0/3119290724 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f58c406dfd0 0x7f58c40b7a50 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f58bc0097a0 tx=0x7f58bc02a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a3d1873492c3a7e7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:58.930 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:58.927+0000 7f58c90dc640 1 -- 172.21.15.27:0/3119290724 <== mon.1 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58bc037070 con 0x7f58c406dfd0 2024-09-12T03:21:58.930 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:58.927+0000 7f58c90dc640 1 -- 172.21.15.27:0/3119290724 <== mon.1 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58bc032440 con 0x7f58c406dfd0 2024-09-12T03:21:58.930 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:58.927+0000 7f58c90dc640 1 -- 172.21.15.27:0/3119290724 <== mon.1 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58bc0327f0 con 0x7f58c406dfd0 2024-09-12T03:21:58.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:58.927+0000 7f58cc342640 1 -- 172.21.15.27:0/3119290724 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f58c406dfd0 msgr2=0x7f58c40b7a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:58.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:58.927+0000 7f58cc342640 1 --2- 172.21.15.27:0/3119290724 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f58c406dfd0 0x7f58c40b7a50 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f58bc0097a0 tx=0x7f58bc02a180 comp rx=0 tx=0).stop 2024-09-12T03:21:58.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:58.927+0000 7f58cc342640 1 -- 172.21.15.27:0/3119290724 shutdown_connections 2024-09-12T03:21:58.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:58.927+0000 7f58cc342640 1 --2- 172.21.15.27:0/3119290724 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f58c406dfd0 0x7f58c40b7a50 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:58.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:58.927+0000 7f58cc342640 1 --2- 172.21.15.27:0/3119290724 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f58c4162040 0x7f58c40d1da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:58.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:58.927+0000 7f58cc342640 1 -- 172.21.15.27:0/3119290724 >> 172.21.15.27:0/3119290724 conn(0x7f58c40b6700 msgr2=0x7f58c40b6b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:58.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:58.927+0000 7f58cc342640 1 -- 172.21.15.27:0/3119290724 shutdown_connections 2024-09-12T03:21:58.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:58.927+0000 7f58cc342640 1 -- 172.21.15.27:0/3119290724 wait complete. 2024-09-12T03:21:58.934 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:58.928+0000 7f58cc342640 1 Processor -- start 2024-09-12T03:21:58.934 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:58.929+0000 7f58cc342640 1 -- start start 2024-09-12T03:21:58.934 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:58.929+0000 7f58cc342640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f58c406dfd0 0x7f58c40ca640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:58.934 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:58.929+0000 7f58cc342640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f58c40d3790 0x7f58c414de20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:58.934 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:58.929+0000 7f58cc342640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f58c4162040 0x7f58c40cc340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:58.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:58.929+0000 7f58cc342640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f58c40d4530 con 0x7f58c4162040 2024-09-12T03:21:58.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:58.929+0000 7f58cc342640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f58c40cee00 con 0x7f58c40d3790 2024-09-12T03:21:58.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:58.929+0000 7f58cc342640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f58c40d4150 con 0x7f58c406dfd0 2024-09-12T03:21:58.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:58.929+0000 7f58ca8df640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f58c4162040 0x7f58c40cc340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:58.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:58.929+0000 7f58ca8df640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f58c4162040 0x7f58c40cc340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39648/0 (socket says 172.21.15.27:39648) 2024-09-12T03:21:58.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:58.929+0000 7f58ca8df640 1 -- 172.21.15.27:0/1544849613 learned_addr learned my addr 172.21.15.27:0/1544849613 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:21:58.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:58.929+0000 7f58c98dd640 1 --2- 172.21.15.27:0/1544849613 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f58c40d3790 0x7f58c414de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:58.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:58.929+0000 7f58ca0de640 1 --2- 172.21.15.27:0/1544849613 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f58c406dfd0 0x7f58c40ca640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:58.936 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:58.930+0000 7f58ca8df640 1 -- 172.21.15.27:0/1544849613 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f58c406dfd0 msgr2=0x7f58c40ca640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:58.936 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:58.930+0000 7f58ca8df640 1 --2- 172.21.15.27:0/1544849613 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f58c406dfd0 0x7f58c40ca640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:58.936 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:58.930+0000 7f58ca8df640 1 -- 172.21.15.27:0/1544849613 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f58c40d3790 msgr2=0x7f58c414de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:58.936 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:58.930+0000 7f58ca8df640 1 --2- 172.21.15.27:0/1544849613 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f58c40d3790 0x7f58c414de20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:58.936 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:58.930+0000 7f58ca8df640 1 -- 172.21.15.27:0/1544849613 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58c414e360 con 0x7f58c4162040 2024-09-12T03:21:58.936 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:58.930+0000 7f58ca8df640 1 --2- 172.21.15.27:0/1544849613 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f58c4162040 0x7f58c40cc340 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f58bc035040 tx=0x7f58bc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:58.936 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:58.930+0000 7f58c2ffd640 1 -- 172.21.15.27:0/1544849613 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58bc037040 con 0x7f58c4162040 2024-09-12T03:21:58.936 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:58.930+0000 7f58cc342640 1 -- 172.21.15.27:0/1544849613 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58c414b770 con 0x7f58c4162040 2024-09-12T03:21:58.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:58.930+0000 7f58c2ffd640 1 -- 172.21.15.27:0/1544849613 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58bc042070 con 0x7f58c4162040 2024-09-12T03:21:58.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:58.930+0000 7f58c2ffd640 1 -- 172.21.15.27:0/1544849613 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58bc0044e0 con 0x7f58c4162040 2024-09-12T03:21:58.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:58.930+0000 7f58cc342640 1 -- 172.21.15.27:0/1544849613 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58c414bd30 con 0x7f58c4162040 2024-09-12T03:21:58.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:58.931+0000 7f58cc342640 1 -- 172.21.15.27:0/1544849613 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58c40b7a50 con 0x7f58c4162040 2024-09-12T03:21:58.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:58.932+0000 7f58c2ffd640 1 -- 172.21.15.27:0/1544849613 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 19) ==== 90855+0+0 (secure 0 0 0) 0x7f58bc032830 con 0x7f58c4162040 2024-09-12T03:21:58.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:58.932+0000 7f58c2ffd640 1 --2- 172.21.15.27:0/1544849613 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f58b006c220 0x7f58b006e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:58.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:58.932+0000 7f58c2ffd640 1 -- 172.21.15.27:0/1544849613 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5737+0+0 (secure 0 0 0) 0x7f58bc0abcc0 con 0x7f58c4162040 2024-09-12T03:21:58.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:58.932+0000 7f58ca0de640 1 --2- 172.21.15.27:0/1544849613 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f58b006c220 0x7f58b006e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:58.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:58.933+0000 7f58ca0de640 1 --2- 172.21.15.27:0/1544849613 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f58b006c220 0x7f58b006e6e0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f58b4009bd0 tx=0x7f58b400b040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:58.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:58.937+0000 7f58c2ffd640 1 -- 172.21.15.27:0/1544849613 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f58bc0b2080 con 0x7f58c4162040 2024-09-12T03:21:59.152 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:59.151+0000 7f58cc342640 1 -- 172.21.15.27:0/1544849613 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7f58c406e6e0 con 0x7f58c4162040 2024-09-12T03:21:59.303 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:59.302+0000 7f58c2ffd640 1 -- 172.21.15.27:0/1544849613 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v68) ==== 136+0+0 (secure 0 0 0) 0x7f58bc079bd0 con 0x7f58c4162040 2024-09-12T03:21:59.306 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:59.305+0000 7f58cc342640 1 -- 172.21.15.27:0/1544849613 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f58b006c220 msgr2=0x7f58b006e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:59.306 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:59.305+0000 7f58cc342640 1 --2- 172.21.15.27:0/1544849613 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f58b006c220 0x7f58b006e6e0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f58b4009bd0 tx=0x7f58b400b040 comp rx=0 tx=0).stop 2024-09-12T03:21:59.306 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:59.305+0000 7f58cc342640 1 -- 172.21.15.27:0/1544849613 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f58c4162040 msgr2=0x7f58c40cc340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:59.306 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:59.305+0000 7f58cc342640 1 --2- 172.21.15.27:0/1544849613 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f58c4162040 0x7f58c40cc340 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f58bc035040 tx=0x7f58bc0046c0 comp rx=0 tx=0).stop 2024-09-12T03:21:59.306 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:59.305+0000 7f58cc342640 1 -- 172.21.15.27:0/1544849613 shutdown_connections 2024-09-12T03:21:59.306 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:59.305+0000 7f58cc342640 1 --2- 172.21.15.27:0/1544849613 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f58b006c220 0x7f58b006e6e0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:59.306 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:59.305+0000 7f58cc342640 1 --2- 172.21.15.27:0/1544849613 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f58c4162040 0x7f58c40cc340 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:59.307 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:59.305+0000 7f58cc342640 1 --2- 172.21.15.27:0/1544849613 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f58c40d3790 0x7f58c414de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:59.307 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:59.305+0000 7f58cc342640 1 --2- 172.21.15.27:0/1544849613 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f58c406dfd0 0x7f58c40ca640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:59.307 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:59.305+0000 7f58cc342640 1 -- 172.21.15.27:0/1544849613 >> 172.21.15.27:0/1544849613 conn(0x7f58c40b6700 msgr2=0x7f58c40cfe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:59.307 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:59.305+0000 7f58cc342640 1 -- 172.21.15.27:0/1544849613 shutdown_connections 2024-09-12T03:21:59.307 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:59.305+0000 7f58cc342640 1 -- 172.21.15.27:0/1544849613 wait complete. 2024-09-12T03:21:59.317 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-12T03:21:59.421 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:59.420+0000 7f7dbe986640 1 Processor -- start 2024-09-12T03:21:59.421 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:59.420+0000 7f7dbe986640 1 -- start start 2024-09-12T03:21:59.425 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:59.421+0000 7f7dbe986640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f7db8072d70 0x7f7db80b3560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:59.425 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:59.421+0000 7f7dbe986640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f7db806e9b0 0x7f7db806ee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:59.425 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:59.421+0000 7f7dbe986640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7db80ce410 con 0x7f7db815b2b0 2024-09-12T03:21:59.425 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:59.421+0000 7f7dbe986640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f7db80720a0 con 0x7f7db8072d70 2024-09-12T03:21:59.425 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:59.421+0000 7f7dbe986640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f7db80732c0 con 0x7f7db806e9b0 2024-09-12T03:21:59.425 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:59.421+0000 7f7db7fff640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f7db815b2b0 0x7f7db8072430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:51214/0 (socket says 172.21.15.27:51214) 2024-09-12T03:21:59.426 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:59.421+0000 7f7db7fff640 1 -- 172.21.15.27:0/1190718031 learned_addr learned my addr 172.21.15.27:0/1190718031 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:21:59.426 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:59.421+0000 7f7db77fe640 1 --2- 172.21.15.27:0/1190718031 >> v2:172.21.15.45:3300/0 conn(0x7f7db8072d70 0x7f7db80b3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:59.426 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:59.421+0000 7f7dbcf23640 1 --2- 172.21.15.27:0/1190718031 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f7db806e9b0 0x7f7db806ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:59.426 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:59.421+0000 7f7db6ffd640 1 -- 172.21.15.27:0/1190718031 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2527259060 0 0) 0x7f7da8001110 con 0x7f7db815b2b0 2024-09-12T03:21:59.426 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:59.421+0000 7f7db6ffd640 1 -- 172.21.15.27:0/1190718031 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7da0003520 con 0x7f7db815b2b0 2024-09-12T03:21:59.426 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:59.422+0000 7f7db6ffd640 1 -- 172.21.15.27:0/1190718031 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3320634011 0 0) 0x7f7da0003520 con 0x7f7db815b2b0 2024-09-12T03:21:59.426 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:59.422+0000 7f7db6ffd640 1 -- 172.21.15.27:0/1190718031 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f7db806e9b0 msgr2=0x7f7db806ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:59.426 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:59.422+0000 7f7db6ffd640 1 --2- 172.21.15.27:0/1190718031 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f7db806e9b0 0x7f7db806ee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:59.426 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:59.422+0000 7f7db6ffd640 1 -- 172.21.15.27:0/1190718031 >> v2:172.21.15.45:3300/0 conn(0x7f7db8072d70 msgr2=0x7f7db80b3560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:59.426 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:59.422+0000 7f7db6ffd640 1 --2- 172.21.15.27:0/1190718031 >> v2:172.21.15.45:3300/0 conn(0x7f7db8072d70 0x7f7db80b3560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:59.426 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:59.422+0000 7f7db6ffd640 1 -- 172.21.15.27:0/1190718031 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7db8071ae0 con 0x7f7db815b2b0 2024-09-12T03:21:59.426 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:59.422+0000 7f7db6ffd640 1 -- 172.21.15.27:0/1190718031 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 847755213 0 0) 0x7f7da8001840 con 0x7f7db815b2b0 2024-09-12T03:21:59.427 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:59.422+0000 7f7db6ffd640 1 -- 172.21.15.27:0/1190718031 >> v1:172.21.15.27:6789/0 conn(0x7f7db815b2b0 legacy=0x7f7db8072430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:59.427 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:59.422+0000 7f7db6ffd640 1 --2- 172.21.15.27:0/1190718031 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f7db806e9b0 0x7f7da00039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:59.427 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:59.422+0000 7f7db6ffd640 1 --2- 172.21.15.27:0/1190718031 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7da0003f30 0x7f7da0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:59.427 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:59.422+0000 7f7db6ffd640 1 --2- 172.21.15.27:0/1190718031 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7da0004d40 0x7f7da00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:59.427 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:59.422+0000 7f7db6ffd640 1 -- 172.21.15.27:0/1190718031 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f7db80720a0 con 0x7f7da0004d40 2024-09-12T03:21:59.427 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:59.422+0000 7f7db6ffd640 1 -- 172.21.15.27:0/1190718031 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f7db80732c0 con 0x7f7da0003f30 2024-09-12T03:21:59.427 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:59.422+0000 7f7db6ffd640 1 -- 172.21.15.27:0/1190718031 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f7da000b9c0 con 0x7f7db806e9b0 2024-09-12T03:21:59.427 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:59.422+0000 7f7dbcf23640 1 --2- 172.21.15.27:0/1190718031 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7da0004d40 0x7f7da00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:59.427 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:59.422+0000 7f7db7fff640 1 --2- 172.21.15.27:0/1190718031 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f7db806e9b0 0x7f7da00039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:59.427 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:59.422+0000 7f7db77fe640 1 --2- 172.21.15.27:0/1190718031 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7da0003f30 0x7f7da0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:59.427 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:59.422+0000 7f7dbcf23640 1 -- 172.21.15.27:0/1190718031 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f7db806e9b0 msgr2=0x7f7da00039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:59.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:59.422+0000 7f7dbcf23640 1 --2- 172.21.15.27:0/1190718031 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f7db806e9b0 0x7f7da00039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:59.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:59.422+0000 7f7dbcf23640 1 -- 172.21.15.27:0/1190718031 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7da0003f30 msgr2=0x7f7da0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:59.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:59.422+0000 7f7dbcf23640 1 --2- 172.21.15.27:0/1190718031 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7da0003f30 0x7f7da0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:59.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:59.422+0000 7f7dbcf23640 1 -- 172.21.15.27:0/1190718031 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7da0003520 con 0x7f7da0004d40 2024-09-12T03:21:59.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:59.423+0000 7f7dbcf23640 1 --2- 172.21.15.27:0/1190718031 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7da0004d40 0x7f7da00051c0 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f7db0009fc0 tx=0x7f7da0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2578300355501399 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:59.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:59.423+0000 7f7db6ffd640 1 -- 172.21.15.27:0/1190718031 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7db0009c10 con 0x7f7da0004d40 2024-09-12T03:21:59.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:59.423+0000 7f7db6ffd640 1 -- 172.21.15.27:0/1190718031 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7db0036070 con 0x7f7da0004d40 2024-09-12T03:21:59.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:59.423+0000 7f7db6ffd640 1 -- 172.21.15.27:0/1190718031 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7db0031670 con 0x7f7da0004d40 2024-09-12T03:21:59.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:59.423+0000 7f7dbe986640 1 -- 172.21.15.27:0/1190718031 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7da0004d40 msgr2=0x7f7da00051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:59.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:59.423+0000 7f7dbe986640 1 --2- 172.21.15.27:0/1190718031 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7da0004d40 0x7f7da00051c0 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f7db0009fc0 tx=0x7f7da0005830 comp rx=0 tx=0).stop 2024-09-12T03:21:59.429 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:59.423+0000 7f7dbcf23640 1 -- 172.21.15.27:0/1190718031 reap_dead start 2024-09-12T03:21:59.429 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:59.424+0000 7f7dbe986640 1 -- 172.21.15.27:0/1190718031 shutdown_connections 2024-09-12T03:21:59.429 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:59.424+0000 7f7dbe986640 1 -- 172.21.15.27:0/1190718031 >> 172.21.15.27:0/1190718031 conn(0x7f7db8062c60 msgr2=0x7f7db806bb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:59.429 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:59.424+0000 7f7dbe986640 1 -- 172.21.15.27:0/1190718031 shutdown_connections 2024-09-12T03:21:59.429 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:59.424+0000 7f7dbe986640 1 -- 172.21.15.27:0/1190718031 wait complete. 2024-09-12T03:21:59.432 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:59.425+0000 7f7dbe986640 1 Processor -- start 2024-09-12T03:21:59.432 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:59.425+0000 7f7dbe986640 1 -- start start 2024-09-12T03:21:59.432 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:59.426+0000 7f7dbe986640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7da0004d40 0x7f7db8142330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:59.432 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:59.426+0000 7f7dbe986640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f7db815a5b0 0x7f7db8141b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:59.432 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:59.426+0000 7f7dbe986640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7db8143ab0 0x7f7db80b1080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:59.432 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:59.426+0000 7f7dbe986640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f7db815b690 con 0x7f7db8143ab0 2024-09-12T03:21:59.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:59.426+0000 7f7dbe986640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f7db80ce690 con 0x7f7da0004d40 2024-09-12T03:21:59.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:59.426+0000 7f7dbe986640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f7db815b810 con 0x7f7db815a5b0 2024-09-12T03:21:59.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:59.426+0000 7f7dbcf23640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7db8143ab0 0x7f7db80b1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:59.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:59.426+0000 7f7dbcf23640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7db8143ab0 0x7f7db80b1080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42332/0 (socket says 172.21.15.27:42332) 2024-09-12T03:21:59.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:59.426+0000 7f7dbcf23640 1 -- 172.21.15.27:0/1935177704 learned_addr learned my addr 172.21.15.27:0/1935177704 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:21:59.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:59.426+0000 7f7db7fff640 1 --2- 172.21.15.27:0/1935177704 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7da0004d40 0x7f7db8142330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:59.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:59.426+0000 7f7db77fe640 1 --2- 172.21.15.27:0/1935177704 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f7db815a5b0 0x7f7db8141b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:59.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:59.426+0000 7f7dbcf23640 1 -- 172.21.15.27:0/1935177704 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f7db815a5b0 msgr2=0x7f7db8141b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:59.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:59.426+0000 7f7dbcf23640 1 --2- 172.21.15.27:0/1935177704 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f7db815a5b0 0x7f7db8141b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:59.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:59.426+0000 7f7dbcf23640 1 -- 172.21.15.27:0/1935177704 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7da0004d40 msgr2=0x7f7db8142330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:59.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:59.426+0000 7f7dbcf23640 1 --2- 172.21.15.27:0/1935177704 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7da0004d40 0x7f7db8142330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:59.434 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:59.426+0000 7f7dbcf23640 1 -- 172.21.15.27:0/1935177704 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7db8142a10 con 0x7f7db8143ab0 2024-09-12T03:21:59.434 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:59.426+0000 7f7dbcf23640 1 --2- 172.21.15.27:0/1935177704 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7db8143ab0 0x7f7db80b1080 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f7db000e230 tx=0x7f7db0009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:59.434 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:59.427+0000 7f7db4ff9640 1 -- 172.21.15.27:0/1935177704 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7db0009c10 con 0x7f7db8143ab0 2024-09-12T03:21:59.434 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:59.427+0000 7f7db4ff9640 1 -- 172.21.15.27:0/1935177704 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7db0031dd0 con 0x7f7db8143ab0 2024-09-12T03:21:59.434 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:59.427+0000 7f7db4ff9640 1 -- 172.21.15.27:0/1935177704 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7db0003bd0 con 0x7f7db8143ab0 2024-09-12T03:21:59.434 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:59.427+0000 7f7dbe986640 1 -- 172.21.15.27:0/1935177704 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7db8142cf0 con 0x7f7db8143ab0 2024-09-12T03:21:59.434 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:59.427+0000 7f7dbe986640 1 -- 172.21.15.27:0/1935177704 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7db81432b0 con 0x7f7db8143ab0 2024-09-12T03:21:59.434 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:59.428+0000 7f7dbe986640 1 -- 172.21.15.27:0/1935177704 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7db80b1f00 con 0x7f7db8143ab0 2024-09-12T03:21:59.434 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:59.428+0000 7f7db4ff9640 1 -- 172.21.15.27:0/1935177704 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 19) ==== 90855+0+0 (secure 0 0 0) 0x7f7db0005d10 con 0x7f7db8143ab0 2024-09-12T03:21:59.434 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:59.429+0000 7f7db4ff9640 1 --2- 172.21.15.27:0/1935177704 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f7da406c250 0x7f7da406e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:59.434 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:59.429+0000 7f7db4ff9640 1 -- 172.21.15.27:0/1935177704 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5737+0+0 (secure 0 0 0) 0x7f7db00ac2e0 con 0x7f7db8143ab0 2024-09-12T03:21:59.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:59.429+0000 7f7db7fff640 1 --2- 172.21.15.27:0/1935177704 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f7da406c250 0x7f7da406e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:59.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:59.429+0000 7f7db7fff640 1 --2- 172.21.15.27:0/1935177704 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f7da406c250 0x7f7da406e710 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f7da80012b0 tx=0x7f7da8009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:59.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:59.433+0000 7f7db4ff9640 1 -- 172.21.15.27:0/1935177704 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f7db007a600 con 0x7f7db8143ab0 2024-09-12T03:21:59.634 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:59.633+0000 7f7dbe986640 1 -- 172.21.15.27:0/1935177704 --> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f7db8075030 con 0x7f7da406c250 2024-09-12T03:21:59.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:59.635+0000 7f7db4ff9640 1 -- 172.21.15.27:0/1935177704 <== mgr.4113 v2:172.21.15.27:6836/1226920526 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f7db8075030 con 0x7f7da406c250 2024-09-12T03:21:59.636 INFO:teuthology.orchestra.run.smithi027.stdout:[ 2024-09-12T03:21:59.636 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:21:59.636 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "cephfs" 2024-09-12T03:21:59.636 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:21:59.637 INFO:teuthology.orchestra.run.smithi027.stdout:] 2024-09-12T03:21:59.638 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:59.637+0000 7f7dbe986640 1 -- 172.21.15.27:0/1935177704 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f7da406c250 msgr2=0x7f7da406e710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:59.638 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:59.637+0000 7f7dbe986640 1 --2- 172.21.15.27:0/1935177704 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f7da406c250 0x7f7da406e710 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f7da80012b0 tx=0x7f7da8009040 comp rx=0 tx=0).stop 2024-09-12T03:21:59.638 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:59.637+0000 7f7dbe986640 1 -- 172.21.15.27:0/1935177704 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7db8143ab0 msgr2=0x7f7db80b1080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:59.639 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:59.637+0000 7f7dbe986640 1 --2- 172.21.15.27:0/1935177704 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7db8143ab0 0x7f7db80b1080 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f7db000e230 tx=0x7f7db0009750 comp rx=0 tx=0).stop 2024-09-12T03:21:59.639 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:59.637+0000 7f7dbe986640 1 -- 172.21.15.27:0/1935177704 shutdown_connections 2024-09-12T03:21:59.639 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:59.637+0000 7f7dbe986640 1 --2- 172.21.15.27:0/1935177704 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f7da406c250 0x7f7da406e710 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:59.639 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:59.637+0000 7f7dbe986640 1 --2- 172.21.15.27:0/1935177704 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7db8143ab0 0x7f7db80b1080 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:59.639 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:59.637+0000 7f7dbe986640 1 --2- 172.21.15.27:0/1935177704 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f7db815a5b0 0x7f7db8141b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:59.639 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:59.637+0000 7f7dbe986640 1 --2- 172.21.15.27:0/1935177704 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7da0004d40 0x7f7db8142330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:59.639 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:59.637+0000 7f7dbe986640 1 -- 172.21.15.27:0/1935177704 >> 172.21.15.27:0/1935177704 conn(0x7f7db8062c60 msgr2=0x7f7db815eec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:59.639 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:59.637+0000 7f7dbe986640 1 -- 172.21.15.27:0/1935177704 shutdown_connections 2024-09-12T03:21:59.639 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:59.637+0000 7f7dbe986640 1 -- 172.21.15.27:0/1935177704 wait complete. 2024-09-12T03:21:59.649 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-12T03:21:59.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:59.754+0000 7f2bbd3c4640 1 Processor -- start 2024-09-12T03:21:59.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:59.754+0000 7f2bbd3c4640 1 -- start start 2024-09-12T03:21:59.756 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:59.754+0000 7f2bbd3c4640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2bb815e650 0x7f2bb80d1d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:59.756 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:59.754+0000 7f2bbd3c4640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f2bb806db30 0x7f2bb806df80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:59.756 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:59.754+0000 7f2bbd3c4640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2bb80d2280 con 0x7f2bb806cc40 2024-09-12T03:21:59.756 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:59.754+0000 7f2bbd3c4640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f2bb806d850 con 0x7f2bb806db30 2024-09-12T03:21:59.756 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:59.754+0000 7f2bbd3c4640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f2bb80d1260 con 0x7f2bb815e650 2024-09-12T03:21:59.756 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:59.755+0000 7f2bb6ffd640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2bb815e650 0x7f2bb80d1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:59.756 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:59.755+0000 7f2bb77fe640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f2bb806db30 0x7f2bb806df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:59.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:59.755+0000 7f2bb77fe640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f2bb806db30 0x7f2bb806df80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.27:46318/0 (socket says 172.21.15.27:46318) 2024-09-12T03:21:59.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:59.755+0000 7f2bb6ffd640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2bb815e650 0x7f2bb80d1d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3301/0 says I am v2:172.21.15.27:35742/0 (socket says 172.21.15.27:35742) 2024-09-12T03:21:59.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:59.755+0000 7f2bb67fc640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f2bb806cc40 0x7f2bb806d060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.27:40980/0 (socket says 172.21.15.27:40980) 2024-09-12T03:21:59.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:59.755+0000 7f2bb77fe640 1 -- 172.21.15.27:0/1073450032 learned_addr learned my addr 172.21.15.27:0/1073450032 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:21:59.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:59.755+0000 7f2bb6ffd640 1 -- 172.21.15.27:0/1073450032 >> v2:172.21.15.45:3300/0 conn(0x7f2bb806db30 msgr2=0x7f2bb806df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:59.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:59.755+0000 7f2bb6ffd640 1 --2- 172.21.15.27:0/1073450032 >> v2:172.21.15.45:3300/0 conn(0x7f2bb806db30 0x7f2bb806df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:59.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:59.755+0000 7f2bb6ffd640 1 -- 172.21.15.27:0/1073450032 >> v1:172.21.15.45:6789/0 conn(0x7f2bb806cc40 legacy=0x7f2bb806d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:59.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:59.755+0000 7f2bb6ffd640 1 -- 172.21.15.27:0/1073450032 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bb80d1540 con 0x7f2bb815e650 2024-09-12T03:21:59.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:59.755+0000 7f2bb77fe640 1 --2- 172.21.15.27:0/1073450032 >> v2:172.21.15.45:3300/0 conn(0x7f2bb806db30 0x7f2bb806df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:21:59.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:59.756+0000 7f2bb6ffd640 1 --2- 172.21.15.27:0/1073450032 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2bb815e650 0x7f2bb80d1d40 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f2ba80097a0 tx=0x7f2ba802a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=26a1f11f09333ae3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:59.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:59.756+0000 7f2bb5ffb640 1 -- 172.21.15.27:0/1073450032 <== mon.2 v2:172.21.15.45:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ba8037070 con 0x7f2bb815e650 2024-09-12T03:21:59.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:59.756+0000 7f2bb5ffb640 1 -- 172.21.15.27:0/1073450032 <== mon.2 v2:172.21.15.45:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ba802ae50 con 0x7f2bb815e650 2024-09-12T03:21:59.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:59.756+0000 7f2bb5ffb640 1 -- 172.21.15.27:0/1073450032 <== mon.2 v2:172.21.15.45:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ba8032660 con 0x7f2bb815e650 2024-09-12T03:21:59.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:59.757+0000 7f2bbd3c4640 1 -- 172.21.15.27:0/1073450032 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2bb815e650 msgr2=0x7f2bb80d1d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:59.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:59.757+0000 7f2bbd3c4640 1 --2- 172.21.15.27:0/1073450032 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2bb815e650 0x7f2bb80d1d40 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f2ba80097a0 tx=0x7f2ba802a100 comp rx=0 tx=0).stop 2024-09-12T03:21:59.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:59.757+0000 7f2bbd3c4640 1 -- 172.21.15.27:0/1073450032 shutdown_connections 2024-09-12T03:21:59.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:59.757+0000 7f2bbd3c4640 1 --2- 172.21.15.27:0/1073450032 >> v2:172.21.15.45:3300/0 conn(0x7f2bb806db30 0x7f2bb806df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:59.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:59.757+0000 7f2bbd3c4640 1 --2- 172.21.15.27:0/1073450032 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2bb815e650 0x7f2bb80d1d40 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:59.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:59.757+0000 7f2bbd3c4640 1 -- 172.21.15.27:0/1073450032 >> 172.21.15.27:0/1073450032 conn(0x7f2bb806f320 msgr2=0x7f2bb806f700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:59.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:59.757+0000 7f2bbd3c4640 1 -- 172.21.15.27:0/1073450032 shutdown_connections 2024-09-12T03:21:59.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:59.757+0000 7f2bbd3c4640 1 -- 172.21.15.27:0/1073450032 wait complete. 2024-09-12T03:21:59.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:59.758+0000 7f2bbd3c4640 1 Processor -- start 2024-09-12T03:21:59.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:59.758+0000 7f2bbd3c4640 1 -- start start 2024-09-12T03:21:59.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:59.759+0000 7f2bbd3c4640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2bb806cc40 0x7f2bb8155560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:59.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:59.759+0000 7f2bbd3c4640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2bb806db30 0x7f2bb8160230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:59.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:59.759+0000 7f2bbd3c4640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2bb815e650 0x7f2bb8160770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:59.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:59.759+0000 7f2bbd3c4640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f2bb806e640 con 0x7f2bb806db30 2024-09-12T03:21:59.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:59.759+0000 7f2bbd3c4640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f2bb806e4c0 con 0x7f2bb806cc40 2024-09-12T03:21:59.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:59.759+0000 7f2bbd3c4640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f2bb80b55b0 con 0x7f2bb815e650 2024-09-12T03:21:59.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:59.759+0000 7f2bb67fc640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2bb806db30 0x7f2bb8160230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:59.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:59.759+0000 7f2bb67fc640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2bb806db30 0x7f2bb8160230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42342/0 (socket says 172.21.15.27:42342) 2024-09-12T03:21:59.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:59.759+0000 7f2bb67fc640 1 -- 172.21.15.27:0/1527192066 learned_addr learned my addr 172.21.15.27:0/1527192066 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:21:59.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:59.759+0000 7f2bb6ffd640 1 --2- 172.21.15.27:0/1527192066 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2bb806cc40 0x7f2bb8155560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:59.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:59.759+0000 7f2bb77fe640 1 --2- 172.21.15.27:0/1527192066 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2bb815e650 0x7f2bb8160770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:59.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:59.759+0000 7f2bb67fc640 1 -- 172.21.15.27:0/1527192066 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2bb815e650 msgr2=0x7f2bb8160770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:59.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:59.759+0000 7f2bb67fc640 1 --2- 172.21.15.27:0/1527192066 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2bb815e650 0x7f2bb8160770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:59.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:59.759+0000 7f2bb67fc640 1 -- 172.21.15.27:0/1527192066 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2bb806cc40 msgr2=0x7f2bb8155560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:59.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:59.759+0000 7f2bb67fc640 1 --2- 172.21.15.27:0/1527192066 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2bb806cc40 0x7f2bb8155560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:59.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:59.759+0000 7f2bb67fc640 1 -- 172.21.15.27:0/1527192066 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bb8159630 con 0x7f2bb806db30 2024-09-12T03:21:59.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:59.760+0000 7f2bb67fc640 1 --2- 172.21.15.27:0/1527192066 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2bb806db30 0x7f2bb8160230 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f2ba000c400 tx=0x7f2ba000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:59.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:59.760+0000 7f2baf7fe640 1 -- 172.21.15.27:0/1527192066 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ba00074e0 con 0x7f2bb806db30 2024-09-12T03:21:59.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:59.760+0000 7f2baf7fe640 1 -- 172.21.15.27:0/1527192066 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ba0007680 con 0x7f2bb806db30 2024-09-12T03:21:59.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:59.760+0000 7f2bbd3c4640 1 -- 172.21.15.27:0/1527192066 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2bb8158470 con 0x7f2bb806db30 2024-09-12T03:21:59.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:59.760+0000 7f2baf7fe640 1 -- 172.21.15.27:0/1527192066 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ba0004a00 con 0x7f2bb806db30 2024-09-12T03:21:59.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:59.760+0000 7f2bbd3c4640 1 -- 172.21.15.27:0/1527192066 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2bb8158a30 con 0x7f2bb806db30 2024-09-12T03:21:59.764 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:59.761+0000 7f2bbd3c4640 1 -- 172.21.15.27:0/1527192066 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2bb806df80 con 0x7f2bb806db30 2024-09-12T03:21:59.764 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:59.762+0000 7f2baf7fe640 1 -- 172.21.15.27:0/1527192066 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 19) ==== 90855+0+0 (secure 0 0 0) 0x7f2ba001a020 con 0x7f2bb806db30 2024-09-12T03:21:59.764 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:59.762+0000 7f2baf7fe640 1 --2- 172.21.15.27:0/1527192066 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f2b9806c220 0x7f2b9806e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:59.764 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:59.762+0000 7f2bb6ffd640 1 --2- 172.21.15.27:0/1527192066 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f2b9806c220 0x7f2b9806e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:59.764 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:59.762+0000 7f2baf7fe640 1 -- 172.21.15.27:0/1527192066 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5737+0+0 (secure 0 0 0) 0x7f2ba008acd0 con 0x7f2bb806db30 2024-09-12T03:21:59.764 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:59.763+0000 7f2bb6ffd640 1 --2- 172.21.15.27:0/1527192066 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f2b9806c220 0x7f2b9806e6e0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f2ba8034000 tx=0x7f2ba802e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:59.767 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:59.767+0000 7f2baf7fe640 1 -- 172.21.15.27:0/1527192066 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f2ba0058a30 con 0x7f2bb806db30 2024-09-12T03:21:59.944 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:59.942+0000 7f2bbd3c4640 1 -- 172.21.15.27:0/1527192066 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f2bb806d570 con 0x7f2bb806db30 2024-09-12T03:21:59.945 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:59.943+0000 7f2baf7fe640 1 -- 172.21.15.27:0/1527192066 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v22)=0 v22) ==== 122+0+0 (secure 0 0 0) 0x7f2ba005cd30 con 0x7f2bb806db30 2024-09-12T03:21:59.946 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:59.945+0000 7f2bbd3c4640 1 -- 172.21.15.27:0/1527192066 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f2b9806c220 msgr2=0x7f2b9806e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:59.946 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:59.945+0000 7f2bbd3c4640 1 --2- 172.21.15.27:0/1527192066 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f2b9806c220 0x7f2b9806e6e0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f2ba8034000 tx=0x7f2ba802e000 comp rx=0 tx=0).stop 2024-09-12T03:21:59.946 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:59.945+0000 7f2bbd3c4640 1 -- 172.21.15.27:0/1527192066 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2bb806db30 msgr2=0x7f2bb8160230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:59.946 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:59.945+0000 7f2bbd3c4640 1 --2- 172.21.15.27:0/1527192066 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2bb806db30 0x7f2bb8160230 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f2ba000c400 tx=0x7f2ba000c8c0 comp rx=0 tx=0).stop 2024-09-12T03:21:59.946 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:59.945+0000 7f2bbd3c4640 1 -- 172.21.15.27:0/1527192066 shutdown_connections 2024-09-12T03:21:59.946 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:59.945+0000 7f2bbd3c4640 1 --2- 172.21.15.27:0/1527192066 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2bb815e650 0x7f2bb8160770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:59.947 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:59.945+0000 7f2bbd3c4640 1 --2- 172.21.15.27:0/1527192066 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f2b9806c220 0x7f2b9806e6e0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:59.947 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:59.945+0000 7f2bbd3c4640 1 --2- 172.21.15.27:0/1527192066 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2bb806db30 0x7f2bb8160230 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:59.947 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:59.945+0000 7f2bbd3c4640 1 --2- 172.21.15.27:0/1527192066 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2bb806cc40 0x7f2bb8155560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:59.947 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:59.945+0000 7f2bbd3c4640 1 -- 172.21.15.27:0/1527192066 >> 172.21.15.27:0/1527192066 conn(0x7f2bb806f320 msgr2=0x7f2bb806f700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:59.947 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:59.945+0000 7f2bbd3c4640 1 -- 172.21.15.27:0/1527192066 shutdown_connections 2024-09-12T03:21:59.947 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:21:59.945+0000 7f2bbd3c4640 1 -- 172.21.15.27:0/1527192066 wait complete. 2024-09-12T03:21:59.958 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:21:59.958 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p volumes/_nogroup/subvol43450/11044fc2-4a1f-4c04-99ec-43281cd1bd6a) 2024-09-12T03:21:59.991 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:21:59.991 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p volumes/_nogroup/subvol43450/11044fc2-4a1f-4c04-99ec-43281cd1bd6a/.snap/fake) 2024-09-12T03:22:00.077 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:22:00.078 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.layout.pool .) 2024-09-12T03:22:00.137 INFO:teuthology.orchestra.run.smithi027.stdout:cephfs_data 2024-09-12T03:22:00.137 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:22:00.137 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo setfattr -n ceph.dir.layout.pool -v cephfs_data volumes/_nogroup/subvol43450/11044fc2-4a1f-4c04-99ec-43281cd1bd6a) 2024-09-12T03:22:00.207 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-12T03:22:00.207 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd of=/home/ubuntu/cephtest/mnt.0/volumes/_nogroup/subvol43450/.meta 2024-09-12T03:22:00.276 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:22:00.277 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p volumes/subvol_grp35010/subvol35811/56ab55bb-6958-4c5b-b500-89b3151fce9c) 2024-09-12T03:22:00.345 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:22:00.345 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p volumes/subvol_grp35010/subvol35811/56ab55bb-6958-4c5b-b500-89b3151fce9c/.snap/fake) 2024-09-12T03:22:00.418 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:22:00.419 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.layout.pool .) 2024-09-12T03:22:00.475 INFO:teuthology.orchestra.run.smithi027.stdout:cephfs_data 2024-09-12T03:22:00.475 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:22:00.476 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo setfattr -n ceph.dir.layout.pool -v cephfs_data volumes/subvol_grp35010/subvol35811/56ab55bb-6958-4c5b-b500-89b3151fce9c) 2024-09-12T03:22:00.546 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-12T03:22:00.546 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd of=/home/ubuntu/cephtest/mnt.0/volumes/subvol_grp35010/subvol35811/.meta 2024-09-12T03:22:00.612 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:22:00.613 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p volumes/_nogroup/subvol7372/7f4a6d24-0de7-4198-a773-301b654f504b) 2024-09-12T03:22:00.681 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:22:00.681 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.layout.pool .) 2024-09-12T03:22:00.737 INFO:teuthology.orchestra.run.smithi027.stdout:cephfs_data 2024-09-12T03:22:00.737 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:22:00.737 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo setfattr -n ceph.dir.layout.pool -v cephfs_data volumes/_nogroup/subvol7372/7f4a6d24-0de7-4198-a773-301b654f504b) 2024-09-12T03:22:00.808 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-12T03:22:00.808 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd of=/home/ubuntu/cephtest/mnt.0/volumes/_nogroup/subvol7372/.meta 2024-09-12T03:22:00.876 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol43450 2024-09-12T03:22:01.021 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.020+0000 7f95781c4640 1 Processor -- start 2024-09-12T03:22:01.021 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.020+0000 7f95781c4640 1 -- start start 2024-09-12T03:22:01.022 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.021+0000 7f95781c4640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f95701645c0 0x7f95701669b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:01.023 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.021+0000 7f95781c4640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f95700b4720 con 0x7f95700af9d0 2024-09-12T03:22:01.023 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.021+0000 7f95781c4640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f95700ce5e0 con 0x7f957015ec30 2024-09-12T03:22:01.023 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.021+0000 7f95781c4640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f95700d02d0 con 0x7f95701645c0 2024-09-12T03:22:01.023 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.021+0000 7f9576761640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f95701645c0 0x7f95701669b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:01.023 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.021+0000 7f957575f640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f95700af9d0 0x7f95700b4010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:51222/0 (socket says 172.21.15.27:51222) 2024-09-12T03:22:01.023 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.021+0000 7f9576761640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f95701645c0 0x7f95701669b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42348/0 (socket says 172.21.15.27:42348) 2024-09-12T03:22:01.023 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.021+0000 7f957575f640 1 -- 172.21.15.27:0/460896398 learned_addr learned my addr 172.21.15.27:0/460896398 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:22:01.023 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.022+0000 7f9576761640 1 -- 172.21.15.27:0/460896398 >> v1:172.21.15.45:6789/0 conn(0x7f957015ec30 legacy=0x7f95700b3900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:01.023 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.022+0000 7f9576761640 1 -- 172.21.15.27:0/460896398 >> v1:172.21.15.27:6789/0 conn(0x7f95700af9d0 legacy=0x7f95700b4010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:01.024 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.022+0000 7f9574f5e640 1 -- 172.21.15.27:0/460896398 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 567625954 0 0) 0x7f95700b4720 con 0x7f95700af9d0 2024-09-12T03:22:01.024 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.022+0000 7f9576761640 1 -- 172.21.15.27:0/460896398 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95700ceac0 con 0x7f95701645c0 2024-09-12T03:22:01.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.022+0000 7f9576761640 1 --2- 172.21.15.27:0/460896398 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f95701645c0 0x7f95701669b0 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f9560009d90 tx=0x7f956002a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac3a5f53a7becd server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:01.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.023+0000 7f9574f5e640 1 -- 172.21.15.27:0/460896398 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9560007740 con 0x7f95701645c0 2024-09-12T03:22:01.026 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.023+0000 7f9574f5e640 1 -- 172.21.15.27:0/460896398 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95600078e0 con 0x7f95701645c0 2024-09-12T03:22:01.026 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.023+0000 7f9574f5e640 1 -- 172.21.15.27:0/460896398 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9560007cb0 con 0x7f95701645c0 2024-09-12T03:22:01.026 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.023+0000 7f95781c4640 1 -- 172.21.15.27:0/460896398 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f95701645c0 msgr2=0x7f95701669b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:01.026 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.023+0000 7f95781c4640 1 --2- 172.21.15.27:0/460896398 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f95701645c0 0x7f95701669b0 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f9560009d90 tx=0x7f956002a6f0 comp rx=0 tx=0).stop 2024-09-12T03:22:01.026 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.023+0000 7f95781c4640 1 -- 172.21.15.27:0/460896398 shutdown_connections 2024-09-12T03:22:01.026 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.023+0000 7f95781c4640 1 --2- 172.21.15.27:0/460896398 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f95701645c0 0x7f95701669b0 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:01.026 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.023+0000 7f95781c4640 1 -- 172.21.15.27:0/460896398 >> 172.21.15.27:0/460896398 conn(0x7f957006bb30 msgr2=0x7f957006bf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:01.026 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.023+0000 7f95781c4640 1 -- 172.21.15.27:0/460896398 shutdown_connections 2024-09-12T03:22:01.026 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.023+0000 7f95781c4640 1 -- 172.21.15.27:0/460896398 wait complete. 2024-09-12T03:22:01.026 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.024+0000 7f95781c4640 1 Processor -- start 2024-09-12T03:22:01.026 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.024+0000 7f95781c4640 1 -- start start 2024-09-12T03:22:01.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.025+0000 7f95781c4640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f95700af9d0 0x7f957015da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:01.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.025+0000 7f95781c4640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f957015ec30 0x7f9570155720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:01.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.025+0000 7f95781c4640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f95701645c0 0x7f9570146200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:01.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.025+0000 7f95781c4640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f957006c8a0 con 0x7f95700af9d0 2024-09-12T03:22:01.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.025+0000 7f95781c4640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f957006c650 con 0x7f95701645c0 2024-09-12T03:22:01.029 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.025+0000 7f95781c4640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f95700cfff0 con 0x7f957015ec30 2024-09-12T03:22:01.029 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.025+0000 7f9575f60640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f95700af9d0 0x7f957015da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:01.029 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.025+0000 7f9575f60640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f95700af9d0 0x7f957015da00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42350/0 (socket says 172.21.15.27:42350) 2024-09-12T03:22:01.029 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.025+0000 7f9575f60640 1 -- 172.21.15.27:0/1044645744 learned_addr learned my addr 172.21.15.27:0/1044645744 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:22:01.029 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.026+0000 7f9575f60640 1 -- 172.21.15.27:0/1044645744 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f957015ec30 msgr2=0x7f9570155720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:01.029 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.026+0000 7f9575f60640 1 --2- 172.21.15.27:0/1044645744 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f957015ec30 0x7f9570155720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:01.029 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.026+0000 7f9575f60640 1 -- 172.21.15.27:0/1044645744 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f95701645c0 msgr2=0x7f9570146200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:01.029 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.026+0000 7f9575f60640 1 --2- 172.21.15.27:0/1044645744 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f95701645c0 0x7f9570146200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:01.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.026+0000 7f9575f60640 1 -- 172.21.15.27:0/1044645744 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f957015c4b0 con 0x7f95700af9d0 2024-09-12T03:22:01.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.026+0000 7f9575f60640 1 --2- 172.21.15.27:0/1044645744 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f95700af9d0 0x7f957015da00 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f956800c480 tx=0x7f956800c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:01.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.026+0000 7f956e7fc640 1 -- 172.21.15.27:0/1044645744 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95680052d0 con 0x7f95700af9d0 2024-09-12T03:22:01.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.026+0000 7f956e7fc640 1 -- 172.21.15.27:0/1044645744 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9568005470 con 0x7f95700af9d0 2024-09-12T03:22:01.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.026+0000 7f95781c4640 1 -- 172.21.15.27:0/1044645744 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f957015df40 con 0x7f95700af9d0 2024-09-12T03:22:01.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.026+0000 7f956e7fc640 1 -- 172.21.15.27:0/1044645744 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9568012870 con 0x7f95700af9d0 2024-09-12T03:22:01.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.026+0000 7f95781c4640 1 -- 172.21.15.27:0/1044645744 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f957014e0a0 con 0x7f95700af9d0 2024-09-12T03:22:01.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.027+0000 7f95781c4640 1 -- 172.21.15.27:0/1044645744 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95700715f0 con 0x7f95700af9d0 2024-09-12T03:22:01.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.028+0000 7f956e7fc640 1 -- 172.21.15.27:0/1044645744 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 19) ==== 90855+0+0 (secure 0 0 0) 0x7f9568012a10 con 0x7f95700af9d0 2024-09-12T03:22:01.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.028+0000 7f956e7fc640 1 --2- 172.21.15.27:0/1044645744 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f955c06c220 0x7f955c06e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:01.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.028+0000 7f956e7fc640 1 -- 172.21.15.27:0/1044645744 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5737+0+0 (secure 0 0 0) 0x7f956808b1f0 con 0x7f95700af9d0 2024-09-12T03:22:01.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.028+0000 7f957575f640 1 --2- 172.21.15.27:0/1044645744 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f955c06c220 0x7f955c06e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:01.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.028+0000 7f957575f640 1 --2- 172.21.15.27:0/1044645744 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f955c06c220 0x7f955c06e6e0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f9564000c00 tx=0x7f95640079c0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:01.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.032+0000 7f956e7fc640 1 -- 172.21.15.27:0/1044645744 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f957015c4b0 con 0x7f95700af9d0 2024-09-12T03:22:01.132 DEBUG:teuthology.orchestra.run.smithi027:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-12T03:22:01.134 DEBUG:teuthology.orchestra.run.smithi045:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-12T03:22:01.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.237+0000 7f95781c4640 1 -- 172.21.15.27:0/1044645744 --> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol43450", "target": ["mon-mgr", ""]}) -- 0x7f957006f240 con 0x7f955c06c220 2024-09-12T03:22:01.243 INFO:tasks.ceph.mgr.y.smithi027.stderr:2024-09-12T03:22:01.242+0000 7f325d8e3640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:22:01.243 INFO:tasks.ceph.mgr.y.smithi027.stderr:2024-09-12T03:22:01.242+0000 7f325d8e3640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:22:01.243 INFO:tasks.ceph.mgr.y.smithi027.stderr:2024-09-12T03:22:01.242+0000 7f325d8e3640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:22:01.243 INFO:tasks.ceph.mgr.y.smithi027.stderr:2024-09-12T03:22:01.242+0000 7f325d8e3640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:22:01.243 INFO:tasks.ceph.mgr.y.smithi027.stderr:2024-09-12T03:22:01.242+0000 7f325d8e3640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:22:01.280 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.279+0000 7f956e7fc640 1 -- 172.21.15.27:0/1044645744 <== mgr.4113 v2:172.21.15.27:6836/1226920526 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+67 (secure 0 0 0) 0x7f957006f240 con 0x7f955c06c220 2024-09-12T03:22:01.280 INFO:teuthology.orchestra.run.smithi027.stdout:/volumes/_nogroup/subvol43450/11044fc2-4a1f-4c04-99ec-43281cd1bd6a 2024-09-12T03:22:01.282 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.281+0000 7f95781c4640 1 -- 172.21.15.27:0/1044645744 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f955c06c220 msgr2=0x7f955c06e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:01.283 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.281+0000 7f95781c4640 1 --2- 172.21.15.27:0/1044645744 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f955c06c220 0x7f955c06e6e0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f9564000c00 tx=0x7f95640079c0 comp rx=0 tx=0).stop 2024-09-12T03:22:01.283 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.281+0000 7f95781c4640 1 -- 172.21.15.27:0/1044645744 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f95700af9d0 msgr2=0x7f957015da00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:01.283 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.281+0000 7f95781c4640 1 --2- 172.21.15.27:0/1044645744 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f95700af9d0 0x7f957015da00 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f956800c480 tx=0x7f956800c940 comp rx=0 tx=0).stop 2024-09-12T03:22:01.283 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.281+0000 7f95781c4640 1 -- 172.21.15.27:0/1044645744 shutdown_connections 2024-09-12T03:22:01.283 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.281+0000 7f95781c4640 1 --2- 172.21.15.27:0/1044645744 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f95701645c0 0x7f9570146200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:01.283 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.281+0000 7f95781c4640 1 --2- 172.21.15.27:0/1044645744 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f957015ec30 0x7f9570155720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:01.283 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.281+0000 7f95781c4640 1 --2- 172.21.15.27:0/1044645744 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f955c06c220 0x7f955c06e6e0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:01.283 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.281+0000 7f95781c4640 1 --2- 172.21.15.27:0/1044645744 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f95700af9d0 0x7f957015da00 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:01.283 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.281+0000 7f95781c4640 1 -- 172.21.15.27:0/1044645744 >> 172.21.15.27:0/1044645744 conn(0x7f957006bb30 msgr2=0x7f9570150250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:01.283 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.281+0000 7f95781c4640 1 -- 172.21.15.27:0/1044645744 shutdown_connections 2024-09-12T03:22:01.283 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.281+0000 7f95781c4640 1 -- 172.21.15.27:0/1044645744 wait complete. 2024-09-12T03:22:01.293 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol35811 subvol_grp35010 2024-09-12T03:22:01.400 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.399+0000 7f961099f640 1 Processor -- start 2024-09-12T03:22:01.400 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.399+0000 7f961099f640 1 -- start start 2024-09-12T03:22:01.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.400+0000 7f961099f640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f960c0b6540 0x7f960c0af370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:01.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.400+0000 7f961099f640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f960c0b5940 con 0x7f960c06d730 2024-09-12T03:22:01.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.400+0000 7f961099f640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f960c0af950 con 0x7f960c15e900 2024-09-12T03:22:01.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.400+0000 7f961099f640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f960c15ece0 con 0x7f960c0b6540 2024-09-12T03:22:01.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.400+0000 7f960ad9d640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f960c06d730 0x7f960c06ee80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:51226/0 (socket says 172.21.15.27:51226) 2024-09-12T03:22:01.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.400+0000 7f960ad9d640 1 -- 172.21.15.27:0/531135736 learned_addr learned my addr 172.21.15.27:0/531135736 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:22:01.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.400+0000 7f960959a640 1 -- 172.21.15.27:0/531135736 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 382537922 0 0) 0x7f9604001490 con 0x7f960c06d730 2024-09-12T03:22:01.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.400+0000 7f960959a640 1 -- 172.21.15.27:0/531135736 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f95f4003520 con 0x7f960c06d730 2024-09-12T03:22:01.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.400+0000 7f9609d9b640 1 --2- 172.21.15.27:0/531135736 >> v2:172.21.15.45:3300/0 conn(0x7f960c0b6540 0x7f960c0af370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:01.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.400+0000 7f960959a640 1 -- 172.21.15.27:0/531135736 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1274428348 0 0) 0x7f95f4003520 con 0x7f960c06d730 2024-09-12T03:22:01.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.400+0000 7f960959a640 1 -- 172.21.15.27:0/531135736 >> v2:172.21.15.45:3300/0 conn(0x7f960c0b6540 msgr2=0x7f960c0af370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:01.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.400+0000 7f960959a640 1 --2- 172.21.15.27:0/531135736 >> v2:172.21.15.45:3300/0 conn(0x7f960c0b6540 0x7f960c0af370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:01.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.400+0000 7f960959a640 1 -- 172.21.15.27:0/531135736 >> v1:172.21.15.45:6789/0 conn(0x7f960c15e900 legacy=0x7f960c0b5e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:01.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.400+0000 7f960959a640 1 -- 172.21.15.27:0/531135736 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f960c0afc30 con 0x7f960c06d730 2024-09-12T03:22:01.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.400+0000 7f960959a640 1 -- 172.21.15.27:0/531135736 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 847755213 0 0) 0x7f9604001bc0 con 0x7f960c06d730 2024-09-12T03:22:01.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.400+0000 7f960959a640 1 -- 172.21.15.27:0/531135736 >> v1:172.21.15.27:6789/0 conn(0x7f960c06d730 legacy=0x7f960c06ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:01.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.400+0000 7f960959a640 1 --2- 172.21.15.27:0/531135736 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f95f40039b0 0x7f95f4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:01.405 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.401+0000 7f960959a640 1 --2- 172.21.15.27:0/531135736 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f95f4004800 0x7f95f4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:01.405 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.401+0000 7f960959a640 1 --2- 172.21.15.27:0/531135736 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f95f40051c0 0x7f95f400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:01.405 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.401+0000 7f960959a640 1 -- 172.21.15.27:0/531135736 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f95f400c0e0 con 0x7f95f4004800 2024-09-12T03:22:01.405 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.401+0000 7f960959a640 1 -- 172.21.15.27:0/531135736 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f95f400c290 con 0x7f95f40051c0 2024-09-12T03:22:01.405 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.401+0000 7f960959a640 1 -- 172.21.15.27:0/531135736 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f95f400c490 con 0x7f95f40039b0 2024-09-12T03:22:01.405 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.401+0000 7f9609d9b640 1 --2- 172.21.15.27:0/531135736 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f95f4004800 0x7f95f4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:01.405 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.401+0000 7f960a59c640 1 --2- 172.21.15.27:0/531135736 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f95f40039b0 0x7f95f4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:01.405 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.401+0000 7f960ad9d640 1 --2- 172.21.15.27:0/531135736 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f95f40051c0 0x7f95f400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:01.405 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.401+0000 7f9609d9b640 1 -- 172.21.15.27:0/531135736 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f95f40039b0 msgr2=0x7f95f4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:01.405 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.401+0000 7f9609d9b640 1 --2- 172.21.15.27:0/531135736 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f95f40039b0 0x7f95f4003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:01.405 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.401+0000 7f9609d9b640 1 -- 172.21.15.27:0/531135736 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f95f40051c0 msgr2=0x7f95f400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:01.405 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.401+0000 7f9609d9b640 1 --2- 172.21.15.27:0/531135736 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f95f40051c0 0x7f95f400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:01.405 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.401+0000 7f960ad9d640 1 -- 172.21.15.27:0/531135736 reap_dead start 2024-09-12T03:22:01.406 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.401+0000 7f9609d9b640 1 -- 172.21.15.27:0/531135736 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95f4003520 con 0x7f95f4004800 2024-09-12T03:22:01.406 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.402+0000 7f9609d9b640 1 --2- 172.21.15.27:0/531135736 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f95f4004800 0x7f95f4004c80 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f960000ba30 tx=0x7f95f4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2580a0fea9916ed5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:01.406 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.402+0000 7f960959a640 1 -- 172.21.15.27:0/531135736 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9600030070 con 0x7f95f4004800 2024-09-12T03:22:01.406 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.402+0000 7f960959a640 1 -- 172.21.15.27:0/531135736 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f960002ade0 con 0x7f95f4004800 2024-09-12T03:22:01.406 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.402+0000 7f960959a640 1 -- 172.21.15.27:0/531135736 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f960000b600 con 0x7f95f4004800 2024-09-12T03:22:01.406 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.402+0000 7f961099f640 1 -- 172.21.15.27:0/531135736 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f95f4004800 msgr2=0x7f95f4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:01.406 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.402+0000 7f961099f640 1 --2- 172.21.15.27:0/531135736 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f95f4004800 0x7f95f4004c80 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f960000ba30 tx=0x7f95f4005830 comp rx=0 tx=0).stop 2024-09-12T03:22:01.406 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.402+0000 7f961099f640 1 -- 172.21.15.27:0/531135736 shutdown_connections 2024-09-12T03:22:01.406 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.402+0000 7f961099f640 1 --2- 172.21.15.27:0/531135736 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f95f4004800 0x7f95f4004c80 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:01.406 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.402+0000 7f961099f640 1 -- 172.21.15.27:0/531135736 >> 172.21.15.27:0/531135736 conn(0x7f960c06cc40 msgr2=0x7f960c06d020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:01.406 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.402+0000 7f961099f640 1 -- 172.21.15.27:0/531135736 shutdown_connections 2024-09-12T03:22:01.406 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.402+0000 7f961099f640 1 -- 172.21.15.27:0/531135736 wait complete. 2024-09-12T03:22:01.409 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.404+0000 7f961099f640 1 Processor -- start 2024-09-12T03:22:01.409 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.404+0000 7f961099f640 1 -- start start 2024-09-12T03:22:01.409 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.404+0000 7f961099f640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f95f4004800 0x7f960c14ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:01.409 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.404+0000 7f961099f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f960c06f210 0x7f960c06f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:01.409 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.404+0000 7f961099f640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f960c0c6c40 0x7f960c0c70f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:01.409 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.404+0000 7f961099f640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f960c0b01f0 con 0x7f960c0c6c40 2024-09-12T03:22:01.409 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.405+0000 7f961099f640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f960c06db40 con 0x7f960c06f210 2024-09-12T03:22:01.409 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.405+0000 7f961099f640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f960c06e880 con 0x7f95f4004800 2024-09-12T03:22:01.410 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.405+0000 7f960ad9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f960c0c6c40 0x7f960c0c70f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:01.410 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.405+0000 7f960ad9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f960c0c6c40 0x7f960c0c70f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42366/0 (socket says 172.21.15.27:42366) 2024-09-12T03:22:01.410 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.405+0000 7f960ad9d640 1 -- 172.21.15.27:0/277300730 learned_addr learned my addr 172.21.15.27:0/277300730 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:22:01.410 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.405+0000 7f960a59c640 1 --2- 172.21.15.27:0/277300730 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f95f4004800 0x7f960c14ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:01.410 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.405+0000 7f9609d9b640 1 --2- 172.21.15.27:0/277300730 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f960c06f210 0x7f960c06f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:01.410 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.405+0000 7f960ad9d640 1 -- 172.21.15.27:0/277300730 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f95f4004800 msgr2=0x7f960c14ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:01.410 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.405+0000 7f960ad9d640 1 --2- 172.21.15.27:0/277300730 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f95f4004800 0x7f960c14ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:01.410 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.405+0000 7f960ad9d640 1 -- 172.21.15.27:0/277300730 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f960c06f210 msgr2=0x7f960c06f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:01.410 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.405+0000 7f960ad9d640 1 --2- 172.21.15.27:0/277300730 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f960c06f210 0x7f960c06f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:01.410 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.405+0000 7f960ad9d640 1 -- 172.21.15.27:0/277300730 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f960c1f54f0 con 0x7f960c0c6c40 2024-09-12T03:22:01.410 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.405+0000 7f9609d9b640 1 --2- 172.21.15.27:0/277300730 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f960c06f210 0x7f960c06f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:22:01.410 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.405+0000 7f960ad9d640 1 --2- 172.21.15.27:0/277300730 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f960c0c6c40 0x7f960c0c70f0 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f960c0735f0 tx=0x7f9604007a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:01.410 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.405+0000 7f95faffd640 1 -- 172.21.15.27:0/277300730 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9604019070 con 0x7f960c0c6c40 2024-09-12T03:22:01.410 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.405+0000 7f95faffd640 1 -- 172.21.15.27:0/277300730 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f960400e9e0 con 0x7f960c0c6c40 2024-09-12T03:22:01.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.405+0000 7f961099f640 1 -- 172.21.15.27:0/277300730 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f960c1f57d0 con 0x7f960c0c6c40 2024-09-12T03:22:01.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.405+0000 7f95faffd640 1 -- 172.21.15.27:0/277300730 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f960400edd0 con 0x7f960c0c6c40 2024-09-12T03:22:01.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.405+0000 7f961099f640 1 -- 172.21.15.27:0/277300730 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f960c14d3a0 con 0x7f960c0c6c40 2024-09-12T03:22:01.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.406+0000 7f961099f640 1 -- 172.21.15.27:0/277300730 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f960c0af370 con 0x7f960c0c6c40 2024-09-12T03:22:01.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.407+0000 7f95faffd640 1 -- 172.21.15.27:0/277300730 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 19) ==== 90855+0+0 (secure 0 0 0) 0x7f960401c070 con 0x7f960c0c6c40 2024-09-12T03:22:01.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.407+0000 7f95faffd640 1 --2- 172.21.15.27:0/277300730 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f95ec06c250 0x7f95ec06e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:01.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.408+0000 7f95faffd640 1 -- 172.21.15.27:0/277300730 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5737+0+0 (secure 0 0 0) 0x7f960408af40 con 0x7f960c0c6c40 2024-09-12T03:22:01.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.408+0000 7f960a59c640 1 --2- 172.21.15.27:0/277300730 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f95ec06c250 0x7f95ec06e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:01.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.408+0000 7f960a59c640 1 --2- 172.21.15.27:0/277300730 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f95ec06c250 0x7f95ec06e710 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f960c0b2380 tx=0x7f95fc009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:01.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.412+0000 7f95faffd640 1 -- 172.21.15.27:0/277300730 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f960c0b5940 con 0x7f960c0c6c40 2024-09-12T03:22:01.613 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.612+0000 7f961099f640 1 -- 172.21.15.27:0/277300730 --> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol35811", "group_name": "subvol_grp35010", "target": ["mon-mgr", ""]}) -- 0x7f960c1f52a0 con 0x7f95ec06c250 2024-09-12T03:22:01.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.646+0000 7f95faffd640 1 -- 172.21.15.27:0/277300730 <== mgr.4113 v2:172.21.15.27:6836/1226920526 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+74 (secure 0 0 0) 0x7f960c1f52a0 con 0x7f95ec06c250 2024-09-12T03:22:01.648 INFO:teuthology.orchestra.run.smithi027.stdout:/volumes/subvol_grp35010/subvol35811/56ab55bb-6958-4c5b-b500-89b3151fce9c 2024-09-12T03:22:01.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.648+0000 7f961099f640 1 -- 172.21.15.27:0/277300730 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f95ec06c250 msgr2=0x7f95ec06e710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:01.650 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.648+0000 7f961099f640 1 --2- 172.21.15.27:0/277300730 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f95ec06c250 0x7f95ec06e710 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f960c0b2380 tx=0x7f95fc009040 comp rx=0 tx=0).stop 2024-09-12T03:22:01.650 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.648+0000 7f961099f640 1 -- 172.21.15.27:0/277300730 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f960c0c6c40 msgr2=0x7f960c0c70f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:01.650 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.648+0000 7f961099f640 1 --2- 172.21.15.27:0/277300730 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f960c0c6c40 0x7f960c0c70f0 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f960c0735f0 tx=0x7f9604007a80 comp rx=0 tx=0).stop 2024-09-12T03:22:01.650 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.648+0000 7f961099f640 1 -- 172.21.15.27:0/277300730 shutdown_connections 2024-09-12T03:22:01.650 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.648+0000 7f961099f640 1 --2- 172.21.15.27:0/277300730 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f95ec06c250 0x7f95ec06e710 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:01.650 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.648+0000 7f961099f640 1 --2- 172.21.15.27:0/277300730 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f960c0c6c40 0x7f960c0c70f0 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:01.650 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.648+0000 7f961099f640 1 --2- 172.21.15.27:0/277300730 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f960c06f210 0x7f960c06f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:01.650 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.648+0000 7f961099f640 1 --2- 172.21.15.27:0/277300730 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f95f4004800 0x7f960c14ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:01.650 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.648+0000 7f961099f640 1 -- 172.21.15.27:0/277300730 >> 172.21.15.27:0/277300730 conn(0x7f960c06cc40 msgr2=0x7f960c0b2570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:01.650 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.648+0000 7f961099f640 1 -- 172.21.15.27:0/277300730 shutdown_connections 2024-09-12T03:22:01.650 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.648+0000 7f961099f640 1 -- 172.21.15.27:0/277300730 wait complete. 2024-09-12T03:22:01.660 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol7372 2024-09-12T03:22:01.769 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.768+0000 7f1eb069b640 1 Processor -- start 2024-09-12T03:22:01.769 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.768+0000 7f1eb069b640 1 -- start start 2024-09-12T03:22:01.770 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.769+0000 7f1eb069b640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f1ea80d31c0 0x7f1ea80d0440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:01.770 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.769+0000 7f1eb069b640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1ea806e960 0x7f1ea80d0980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:01.770 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.769+0000 7f1eb069b640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ea80cf9f0 con 0x7f1ea8070d30 2024-09-12T03:22:01.770 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.769+0000 7f1eb069b640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f1ea80d1a40 con 0x7f1ea80d31c0 2024-09-12T03:22:01.770 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.769+0000 7f1eb069b640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f1ea80cfc90 con 0x7f1ea806e960 2024-09-12T03:22:01.771 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.770+0000 7f1eaec38640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1ea806e960 0x7f1ea80d0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:01.771 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.770+0000 7f1eae437640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f1ea8070d30 0x7f1ea806cc40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.27:41012/0 (socket says 172.21.15.27:41012) 2024-09-12T03:22:01.771 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.770+0000 7f1eadc36640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f1ea80d31c0 0x7f1ea80d0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:01.771 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.770+0000 7f1eae437640 1 -- 172.21.15.27:0/4041504210 learned_addr learned my addr 172.21.15.27:0/4041504210 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:22:01.771 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.770+0000 7f1ead435640 1 -- 172.21.15.27:0/4041504210 <== mon.1 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1269103841 0 0) 0x7f1ea80cf9f0 con 0x7f1ea8070d30 2024-09-12T03:22:01.771 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.770+0000 7f1ead435640 1 -- 172.21.15.27:0/4041504210 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1e94003520 con 0x7f1ea8070d30 2024-09-12T03:22:01.772 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.770+0000 7f1eaec38640 1 -- 172.21.15.27:0/4041504210 >> v2:172.21.15.45:3300/0 conn(0x7f1ea80d31c0 msgr2=0x7f1ea80d0440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:01.772 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.770+0000 7f1eaec38640 1 --2- 172.21.15.27:0/4041504210 >> v2:172.21.15.45:3300/0 conn(0x7f1ea80d31c0 0x7f1ea80d0440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:01.772 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.770+0000 7f1eaec38640 1 -- 172.21.15.27:0/4041504210 >> v1:172.21.15.45:6789/0 conn(0x7f1ea8070d30 legacy=0x7f1ea806cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:01.772 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.770+0000 7f1eaec38640 1 -- 172.21.15.27:0/4041504210 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ea80cff70 con 0x7f1ea806e960 2024-09-12T03:22:01.772 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.770+0000 7f1eadc36640 1 --2- 172.21.15.27:0/4041504210 >> v2:172.21.15.45:3300/0 conn(0x7f1ea80d31c0 0x7f1ea80d0440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:22:01.773 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.771+0000 7f1eaec38640 1 --2- 172.21.15.27:0/4041504210 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1ea806e960 0x7f1ea80d0980 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f1ea80d1bc0 tx=0x7f1ea002a530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8ef8e68431ea8c5a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:01.773 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.771+0000 7f1ead435640 1 -- 172.21.15.27:0/4041504210 <== mon.2 v2:172.21.15.45:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ea0007740 con 0x7f1ea806e960 2024-09-12T03:22:01.773 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.771+0000 7f1ead435640 1 -- 172.21.15.27:0/4041504210 <== mon.2 v2:172.21.15.45:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ea00078e0 con 0x7f1ea806e960 2024-09-12T03:22:01.773 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.771+0000 7f1ead435640 1 -- 172.21.15.27:0/4041504210 <== mon.2 v2:172.21.15.45:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ea0007c90 con 0x7f1ea806e960 2024-09-12T03:22:01.774 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.772+0000 7f1eb069b640 1 -- 172.21.15.27:0/4041504210 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1ea806e960 msgr2=0x7f1ea80d0980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:01.774 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.772+0000 7f1eb069b640 1 --2- 172.21.15.27:0/4041504210 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1ea806e960 0x7f1ea80d0980 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f1ea80d1bc0 tx=0x7f1ea002a530 comp rx=0 tx=0).stop 2024-09-12T03:22:01.774 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.772+0000 7f1eb069b640 1 -- 172.21.15.27:0/4041504210 shutdown_connections 2024-09-12T03:22:01.774 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.772+0000 7f1eb069b640 1 --2- 172.21.15.27:0/4041504210 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1ea806e960 0x7f1ea80d0980 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:01.774 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.772+0000 7f1eb069b640 1 --2- 172.21.15.27:0/4041504210 >> v2:172.21.15.45:3300/0 conn(0x7f1ea80d31c0 0x7f1ea80d0440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:01.774 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.772+0000 7f1eb069b640 1 -- 172.21.15.27:0/4041504210 >> 172.21.15.27:0/4041504210 conn(0x7f1ea80b7c10 msgr2=0x7f1ea809ba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:01.774 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.772+0000 7f1eb069b640 1 -- 172.21.15.27:0/4041504210 shutdown_connections 2024-09-12T03:22:01.774 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.772+0000 7f1eb069b640 1 -- 172.21.15.27:0/4041504210 wait complete. 2024-09-12T03:22:01.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.773+0000 7f1eb069b640 1 Processor -- start 2024-09-12T03:22:01.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.773+0000 7f1eb069b640 1 -- start start 2024-09-12T03:22:01.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.774+0000 7f1eb069b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1ea806e960 0x7f1ea80cc260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:01.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.774+0000 7f1eb069b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1ea8070d30 0x7f1ea8152270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:01.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.774+0000 7f1eb069b640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1ea80d31c0 0x7f1ea815bc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:01.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.774+0000 7f1eb069b640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f1ea8168480 con 0x7f1ea806e960 2024-09-12T03:22:01.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.774+0000 7f1eb069b640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f1ea8168300 con 0x7f1ea8070d30 2024-09-12T03:22:01.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.774+0000 7f1eb069b640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f1ea8168600 con 0x7f1ea80d31c0 2024-09-12T03:22:01.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.774+0000 7f1eae437640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1ea806e960 0x7f1ea80cc260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:01.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.774+0000 7f1eae437640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1ea806e960 0x7f1ea80cc260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42382/0 (socket says 172.21.15.27:42382) 2024-09-12T03:22:01.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.774+0000 7f1eae437640 1 -- 172.21.15.27:0/1036842189 learned_addr learned my addr 172.21.15.27:0/1036842189 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:22:01.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.774+0000 7f1eadc36640 1 --2- 172.21.15.27:0/1036842189 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1ea8070d30 0x7f1ea8152270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:01.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.774+0000 7f1eaec38640 1 --2- 172.21.15.27:0/1036842189 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1ea80d31c0 0x7f1ea815bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:01.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.774+0000 7f1eae437640 1 -- 172.21.15.27:0/1036842189 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1ea80d31c0 msgr2=0x7f1ea815bc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:01.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.774+0000 7f1eae437640 1 --2- 172.21.15.27:0/1036842189 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1ea80d31c0 0x7f1ea815bc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:01.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.774+0000 7f1eae437640 1 -- 172.21.15.27:0/1036842189 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1ea8070d30 msgr2=0x7f1ea8152270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:01.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.774+0000 7f1eae437640 1 --2- 172.21.15.27:0/1036842189 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1ea8070d30 0x7f1ea8152270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:01.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.774+0000 7f1eae437640 1 -- 172.21.15.27:0/1036842189 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ea80cc840 con 0x7f1ea806e960 2024-09-12T03:22:01.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.775+0000 7f1eae437640 1 --2- 172.21.15.27:0/1036842189 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1ea806e960 0x7f1ea80cc260 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f1ea400c460 tx=0x7f1ea400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:01.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.775+0000 7f1e9effd640 1 -- 172.21.15.27:0/1036842189 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ea4017070 con 0x7f1ea806e960 2024-09-12T03:22:01.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.775+0000 7f1e9effd640 1 -- 172.21.15.27:0/1036842189 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ea4005ce0 con 0x7f1ea806e960 2024-09-12T03:22:01.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.775+0000 7f1e9effd640 1 -- 172.21.15.27:0/1036842189 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ea4007290 con 0x7f1ea806e960 2024-09-12T03:22:01.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.775+0000 7f1eb069b640 1 -- 172.21.15.27:0/1036842189 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ea80ccb20 con 0x7f1ea806e960 2024-09-12T03:22:01.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.775+0000 7f1eb069b640 1 -- 172.21.15.27:0/1036842189 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ea80cd0e0 con 0x7f1ea806e960 2024-09-12T03:22:01.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.776+0000 7f1eb069b640 1 -- 172.21.15.27:0/1036842189 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ea806cc40 con 0x7f1ea806e960 2024-09-12T03:22:01.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.776+0000 7f1e9effd640 1 -- 172.21.15.27:0/1036842189 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 19) ==== 90855+0+0 (secure 0 0 0) 0x7f1ea401a070 con 0x7f1ea806e960 2024-09-12T03:22:01.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.777+0000 7f1e9effd640 1 --2- 172.21.15.27:0/1036842189 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f1e8c06c220 0x7f1e8c06e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:01.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.777+0000 7f1eadc36640 1 --2- 172.21.15.27:0/1036842189 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f1e8c06c220 0x7f1e8c06e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:01.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.777+0000 7f1e9effd640 1 -- 172.21.15.27:0/1036842189 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5737+0+0 (secure 0 0 0) 0x7f1ea408a870 con 0x7f1ea806e960 2024-09-12T03:22:01.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.777+0000 7f1eadc36640 1 --2- 172.21.15.27:0/1036842189 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f1e8c06c220 0x7f1e8c06e6e0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f1e98009560 tx=0x7f1e98009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:01.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.781+0000 7f1e9effd640 1 -- 172.21.15.27:0/1036842189 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f1ea40585d0 con 0x7f1ea806e960 2024-09-12T03:22:01.992 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:01.991+0000 7f1eb069b640 1 -- 172.21.15.27:0/1036842189 --> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol7372", "target": ["mon-mgr", ""]}) -- 0x7f1ea815b5e0 con 0x7f1e8c06c220 2024-09-12T03:22:02.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:02.016+0000 7f1e9effd640 1 -- 172.21.15.27:0/1036842189 <== mgr.4113 v2:172.21.15.27:6836/1226920526 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+144 (secure 0 0 0) 0x7f1ea815b5e0 con 0x7f1e8c06c220 2024-09-12T03:22:02.017 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-12T03:22:02.017 INFO:teuthology.orchestra.run.smithi027.stdout: "status": { 2024-09-12T03:22:02.017 INFO:teuthology.orchestra.run.smithi027.stdout: "state": "pending", 2024-09-12T03:22:02.017 INFO:teuthology.orchestra.run.smithi027.stdout: "source": { 2024-09-12T03:22:02.017 INFO:teuthology.orchestra.run.smithi027.stdout: "volume": "fake", 2024-09-12T03:22:02.017 INFO:teuthology.orchestra.run.smithi027.stdout: "subvolume": "fake", 2024-09-12T03:22:02.017 INFO:teuthology.orchestra.run.smithi027.stdout: "snapshot": "fake" 2024-09-12T03:22:02.018 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:22:02.018 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:22:02.018 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-12T03:22:02.019 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:02.018+0000 7f1eb069b640 1 -- 172.21.15.27:0/1036842189 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f1e8c06c220 msgr2=0x7f1e8c06e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:02.019 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:02.018+0000 7f1eb069b640 1 --2- 172.21.15.27:0/1036842189 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f1e8c06c220 0x7f1e8c06e6e0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f1e98009560 tx=0x7f1e98009190 comp rx=0 tx=0).stop 2024-09-12T03:22:02.019 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:02.018+0000 7f1eb069b640 1 -- 172.21.15.27:0/1036842189 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1ea806e960 msgr2=0x7f1ea80cc260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:02.019 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:02.018+0000 7f1eb069b640 1 --2- 172.21.15.27:0/1036842189 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1ea806e960 0x7f1ea80cc260 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f1ea400c460 tx=0x7f1ea400c920 comp rx=0 tx=0).stop 2024-09-12T03:22:02.019 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:02.018+0000 7f1eb069b640 1 -- 172.21.15.27:0/1036842189 shutdown_connections 2024-09-12T03:22:02.019 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:02.018+0000 7f1eb069b640 1 --2- 172.21.15.27:0/1036842189 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1ea80d31c0 0x7f1ea815bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:02.019 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:02.018+0000 7f1eb069b640 1 --2- 172.21.15.27:0/1036842189 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1ea8070d30 0x7f1ea8152270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:02.019 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:02.018+0000 7f1eb069b640 1 --2- 172.21.15.27:0/1036842189 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f1e8c06c220 0x7f1e8c06e6e0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:02.020 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:02.018+0000 7f1eb069b640 1 --2- 172.21.15.27:0/1036842189 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1ea806e960 0x7f1ea80cc260 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:02.020 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:02.018+0000 7f1eb069b640 1 -- 172.21.15.27:0/1036842189 >> 172.21.15.27:0/1036842189 conn(0x7f1ea80b7c10 msgr2=0x7f1ea80b0be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:02.020 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:02.018+0000 7f1eb069b640 1 -- 172.21.15.27:0/1036842189 shutdown_connections 2024-09-12T03:22:02.020 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:02.018+0000 7f1eb069b640 1 -- 172.21.15.27:0/1036842189 wait complete. 2024-09-12T03:22:02.030 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm cephfs subvol43450 fake 2024-09-12T03:22:02.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:02.136+0000 7f3e7e386640 1 Processor -- start 2024-09-12T03:22:02.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:02.136+0000 7f3e7e386640 1 -- start start 2024-09-12T03:22:02.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:02.137+0000 7f3e7e386640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f3e780b55e0 0x7f3e7806c650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:02.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:02.137+0000 7f3e7e386640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f3e780b35f0 0x7f3e781638c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:02.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:02.137+0000 7f3e7e386640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3e78163f20 con 0x7f3e781634e0 2024-09-12T03:22:02.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:02.137+0000 7f3e7e386640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f3e7806ea70 con 0x7f3e780b55e0 2024-09-12T03:22:02.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:02.137+0000 7f3e7e386640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f3e7806ed50 con 0x7f3e780b35f0 2024-09-12T03:22:02.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:02.137+0000 7f3e7c923640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f3e780b35f0 0x7f3e781638c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:02.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:02.137+0000 7f3e777fe640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f3e780b55e0 0x7f3e7806c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:02.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:02.137+0000 7f3e77fff640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f3e781634e0 0x7f3e780b4ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.27:41022/0 (socket says 172.21.15.27:41022) 2024-09-12T03:22:02.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:02.137+0000 7f3e77fff640 1 -- 172.21.15.27:0/1733771408 learned_addr learned my addr 172.21.15.27:0/1733771408 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:22:02.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:02.138+0000 7f3e76ffd640 1 -- 172.21.15.27:0/1733771408 <== mon.1 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3528054230 0 0) 0x7f3e78163f20 con 0x7f3e781634e0 2024-09-12T03:22:02.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:02.138+0000 7f3e76ffd640 1 -- 172.21.15.27:0/1733771408 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3e60003520 con 0x7f3e781634e0 2024-09-12T03:22:02.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:02.138+0000 7f3e777fe640 1 -- 172.21.15.27:0/1733771408 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f3e780b35f0 msgr2=0x7f3e781638c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:02.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:02.138+0000 7f3e777fe640 1 --2- 172.21.15.27:0/1733771408 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f3e780b35f0 0x7f3e781638c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:02.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:02.138+0000 7f3e777fe640 1 -- 172.21.15.27:0/1733771408 >> v1:172.21.15.45:6789/0 conn(0x7f3e781634e0 legacy=0x7f3e780b4ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:02.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:02.138+0000 7f3e777fe640 1 -- 172.21.15.27:0/1733771408 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e7806e3e0 con 0x7f3e780b55e0 2024-09-12T03:22:02.143 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:02.139+0000 7f3e777fe640 1 --2- 172.21.15.27:0/1733771408 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3e780b55e0 0x7f3e7806c650 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f3e6c0097a0 tx=0x7f3e6c02a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d4965dfd65934e54 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:02.144 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:02.139+0000 7f3e76ffd640 1 -- 172.21.15.27:0/1733771408 <== mon.1 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e6c037070 con 0x7f3e780b55e0 2024-09-12T03:22:02.144 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:02.139+0000 7f3e76ffd640 1 -- 172.21.15.27:0/1733771408 <== mon.1 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e6c032440 con 0x7f3e780b55e0 2024-09-12T03:22:02.144 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:02.139+0000 7f3e76ffd640 1 -- 172.21.15.27:0/1733771408 <== mon.1 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e6c032810 con 0x7f3e780b55e0 2024-09-12T03:22:02.144 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:02.139+0000 7f3e7e386640 1 -- 172.21.15.27:0/1733771408 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3e780b55e0 msgr2=0x7f3e7806c650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:02.144 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:02.139+0000 7f3e7e386640 1 --2- 172.21.15.27:0/1733771408 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3e780b55e0 0x7f3e7806c650 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f3e6c0097a0 tx=0x7f3e6c02a180 comp rx=0 tx=0).stop 2024-09-12T03:22:02.144 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:02.139+0000 7f3e7e386640 1 -- 172.21.15.27:0/1733771408 shutdown_connections 2024-09-12T03:22:02.144 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:02.139+0000 7f3e7e386640 1 --2- 172.21.15.27:0/1733771408 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f3e780b35f0 0x7f3e781638c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:02.144 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:02.139+0000 7f3e7e386640 1 --2- 172.21.15.27:0/1733771408 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3e780b55e0 0x7f3e7806c650 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:02.144 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:02.139+0000 7f3e7e386640 1 -- 172.21.15.27:0/1733771408 >> 172.21.15.27:0/1733771408 conn(0x7f3e7806bb30 msgr2=0x7f3e7806bf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:02.145 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:02.139+0000 7f3e7e386640 1 -- 172.21.15.27:0/1733771408 shutdown_connections 2024-09-12T03:22:02.145 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:02.139+0000 7f3e7e386640 1 -- 172.21.15.27:0/1733771408 wait complete. 2024-09-12T03:22:02.145 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:02.140+0000 7f3e7e386640 1 Processor -- start 2024-09-12T03:22:02.145 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:02.141+0000 7f3e7e386640 1 -- start start 2024-09-12T03:22:02.145 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:02.141+0000 7f3e7e386640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3e780b35f0 0x7f3e7814a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:02.145 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:02.141+0000 7f3e7e386640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3e780b55e0 0x7f3e780b0360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:02.145 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:02.141+0000 7f3e7e386640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f3e781634e0 0x7f3e780b08a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:02.145 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:02.141+0000 7f3e7e386640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f3e78160250 con 0x7f3e780b55e0 2024-09-12T03:22:02.145 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:02.141+0000 7f3e7e386640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f3e781600d0 con 0x7f3e780b35f0 2024-09-12T03:22:02.145 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:02.141+0000 7f3e7e386640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f3e781603d0 con 0x7f3e781634e0 2024-09-12T03:22:02.145 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:02.142+0000 7f3e777fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3e780b55e0 0x7f3e780b0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:02.145 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:02.142+0000 7f3e777fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3e780b55e0 0x7f3e780b0360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42384/0 (socket says 172.21.15.27:42384) 2024-09-12T03:22:02.145 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:02.142+0000 7f3e777fe640 1 -- 172.21.15.27:0/2191070219 learned_addr learned my addr 172.21.15.27:0/2191070219 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:22:02.145 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:02.142+0000 7f3e77fff640 1 --2- 172.21.15.27:0/2191070219 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3e780b35f0 0x7f3e7814a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:02.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:02.142+0000 7f3e7c923640 1 --2- 172.21.15.27:0/2191070219 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f3e781634e0 0x7f3e780b08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:02.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:02.142+0000 7f3e777fe640 1 -- 172.21.15.27:0/2191070219 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f3e781634e0 msgr2=0x7f3e780b08a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:02.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:02.142+0000 7f3e777fe640 1 --2- 172.21.15.27:0/2191070219 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f3e781634e0 0x7f3e780b08a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:02.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:02.142+0000 7f3e777fe640 1 -- 172.21.15.27:0/2191070219 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3e780b35f0 msgr2=0x7f3e7814a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:02.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:02.142+0000 7f3e777fe640 1 --2- 172.21.15.27:0/2191070219 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3e780b35f0 0x7f3e7814a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:02.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:02.142+0000 7f3e777fe640 1 -- 172.21.15.27:0/2191070219 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e78161700 con 0x7f3e780b55e0 2024-09-12T03:22:02.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:02.142+0000 7f3e777fe640 1 --2- 172.21.15.27:0/2191070219 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3e780b55e0 0x7f3e780b0360 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f3e6c035040 tx=0x7f3e6c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:02.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:02.142+0000 7f3e74ff9640 1 -- 172.21.15.27:0/2191070219 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e6c041030 con 0x7f3e780b55e0 2024-09-12T03:22:02.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:02.142+0000 7f3e74ff9640 1 -- 172.21.15.27:0/2191070219 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e6c03d4e0 con 0x7f3e780b55e0 2024-09-12T03:22:02.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:02.142+0000 7f3e7e386640 1 -- 172.21.15.27:0/2191070219 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e781619e0 con 0x7f3e780b55e0 2024-09-12T03:22:02.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:02.142+0000 7f3e74ff9640 1 -- 172.21.15.27:0/2191070219 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e6c037040 con 0x7f3e780b55e0 2024-09-12T03:22:02.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:02.142+0000 7f3e7e386640 1 -- 172.21.15.27:0/2191070219 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e78161fa0 con 0x7f3e780b55e0 2024-09-12T03:22:02.147 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:02.143+0000 7f3e7e386640 1 -- 172.21.15.27:0/2191070219 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e780b1bb0 con 0x7f3e780b55e0 2024-09-12T03:22:02.147 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:02.144+0000 7f3e74ff9640 1 -- 172.21.15.27:0/2191070219 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 19) ==== 90855+0+0 (secure 0 0 0) 0x7f3e6c044070 con 0x7f3e780b55e0 2024-09-12T03:22:02.147 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:02.144+0000 7f3e74ff9640 1 --2- 172.21.15.27:0/2191070219 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f3e6406c220 0x7f3e6406e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:02.147 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:02.144+0000 7f3e74ff9640 1 -- 172.21.15.27:0/2191070219 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5737+0+0 (secure 0 0 0) 0x7f3e6c0aba40 con 0x7f3e780b55e0 2024-09-12T03:22:02.148 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:02.144+0000 7f3e77fff640 1 --2- 172.21.15.27:0/2191070219 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f3e6406c220 0x7f3e6406e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:02.148 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:02.145+0000 7f3e77fff640 1 --2- 172.21.15.27:0/2191070219 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f3e6406c220 0x7f3e6406e6e0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f3e68005880 tx=0x7f3e68009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:02.149 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:02.149+0000 7f3e74ff9640 1 -- 172.21.15.27:0/2191070219 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f3e780b1bb0 con 0x7f3e780b55e0 2024-09-12T03:22:02.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:02.314+0000 7f3e74ff9640 1 -- 172.21.15.27:0/2191070219 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mgrmap(e 20) ==== 90906+0+0 (secure 0 0 0) 0x7f3e6c075f30 con 0x7f3e780b55e0 2024-09-12T03:22:02.364 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:02.363+0000 7f3e7e386640 1 -- 172.21.15.27:0/2191070219 --> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "vol_name": "cephfs", "sub_name": "subvol43450", "snap_name": "fake", "target": ["mon-mgr", ""]}) -- 0x7f3e7806f240 con 0x7f3e6406c220 2024-09-12T03:22:02.401 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:02.400+0000 7f3e74ff9640 1 -- 172.21.15.27:0/2191070219 <== mgr.4113 v2:172.21.15.27:6836/1226920526 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f3e7806f240 con 0x7f3e6406c220 2024-09-12T03:22:02.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:02.402+0000 7f3e7e386640 1 -- 172.21.15.27:0/2191070219 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f3e6406c220 msgr2=0x7f3e6406e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:02.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:02.402+0000 7f3e7e386640 1 --2- 172.21.15.27:0/2191070219 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f3e6406c220 0x7f3e6406e6e0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f3e68005880 tx=0x7f3e68009040 comp rx=0 tx=0).stop 2024-09-12T03:22:02.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:02.402+0000 7f3e7e386640 1 -- 172.21.15.27:0/2191070219 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3e780b55e0 msgr2=0x7f3e780b0360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:02.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:02.402+0000 7f3e7e386640 1 --2- 172.21.15.27:0/2191070219 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3e780b55e0 0x7f3e780b0360 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f3e6c035040 tx=0x7f3e6c0046c0 comp rx=0 tx=0).stop 2024-09-12T03:22:02.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:02.402+0000 7f3e7e386640 1 -- 172.21.15.27:0/2191070219 shutdown_connections 2024-09-12T03:22:02.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:02.402+0000 7f3e7e386640 1 --2- 172.21.15.27:0/2191070219 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f3e781634e0 0x7f3e780b08a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:02.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:02.402+0000 7f3e7e386640 1 --2- 172.21.15.27:0/2191070219 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f3e6406c220 0x7f3e6406e6e0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:02.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:02.402+0000 7f3e7e386640 1 --2- 172.21.15.27:0/2191070219 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3e780b55e0 0x7f3e780b0360 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:02.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:02.402+0000 7f3e7e386640 1 --2- 172.21.15.27:0/2191070219 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3e780b35f0 0x7f3e7814a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:02.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:02.402+0000 7f3e7e386640 1 -- 172.21.15.27:0/2191070219 >> 172.21.15.27:0/2191070219 conn(0x7f3e7806bb30 msgr2=0x7f3e780b14a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:02.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:02.402+0000 7f3e7e386640 1 -- 172.21.15.27:0/2191070219 shutdown_connections 2024-09-12T03:22:02.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:02.403+0000 7f3e7e386640 1 -- 172.21.15.27:0/2191070219 wait complete. 2024-09-12T03:22:02.414 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm cephfs subvol35811 fake subvol_grp35010 2024-09-12T03:22:02.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:02.522+0000 7f3acc217640 1 Processor -- start 2024-09-12T03:22:02.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:02.522+0000 7f3acc217640 1 -- start start 2024-09-12T03:22:02.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:02.523+0000 7f3acc217640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f3ac415e530 0x7f3ac40aff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:02.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:02.523+0000 7f3acc217640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f3ac415e910 0x7f3ac40b4710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:02.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:02.523+0000 7f3acc217640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f3ac40aeb80 0x7f3ac40722f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:02.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:02.523+0000 7f3acc217640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f3ac40b07e0 con 0x7f3ac415e530 2024-09-12T03:22:02.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:02.523+0000 7f3acc217640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f3ac40b4c50 con 0x7f3ac40aeb80 2024-09-12T03:22:02.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:02.523+0000 7f3acc217640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f3ac406dc20 con 0x7f3ac415e910 2024-09-12T03:22:02.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:02.523+0000 7f3ac9fb3640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f3ac415e530 0x7f3ac40aff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:02.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:02.523+0000 7f3ac9fb3640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f3ac415e530 0x7f3ac40aff20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42390/0 (socket says 172.21.15.27:42390) 2024-09-12T03:22:02.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:02.523+0000 7f3ac9fb3640 1 -- 172.21.15.27:0/2316927647 learned_addr learned my addr 172.21.15.27:0/2316927647 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:22:02.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:02.523+0000 7f3aca7b4640 1 --2- 172.21.15.27:0/2316927647 >> v2:172.21.15.45:3300/0 conn(0x7f3ac40aeb80 0x7f3ac40722f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:02.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:02.523+0000 7f3ac9fb3640 1 -- 172.21.15.27:0/2316927647 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f3ac415e910 msgr2=0x7f3ac40b4710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:02.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:02.523+0000 7f3ac9fb3640 1 --2- 172.21.15.27:0/2316927647 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f3ac415e910 0x7f3ac40b4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:02.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:02.523+0000 7f3ac9fb3640 1 -- 172.21.15.27:0/2316927647 >> v2:172.21.15.45:3300/0 conn(0x7f3ac40aeb80 msgr2=0x7f3ac40722f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:02.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:02.523+0000 7f3ac9fb3640 1 --2- 172.21.15.27:0/2316927647 >> v2:172.21.15.45:3300/0 conn(0x7f3ac40aeb80 0x7f3ac40722f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:02.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:02.523+0000 7f3ac9fb3640 1 -- 172.21.15.27:0/2316927647 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ac40b0500 con 0x7f3ac415e530 2024-09-12T03:22:02.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:02.524+0000 7f3ac9fb3640 1 --2- 172.21.15.27:0/2316927647 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3ac415e530 0x7f3ac40aff20 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f3ac40aeaf0 tx=0x7f3ac002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58d4c0500be9e8d7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:02.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:02.524+0000 7f3ac8fb1640 1 -- 172.21.15.27:0/2316927647 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ac0037070 con 0x7f3ac415e530 2024-09-12T03:22:02.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:02.524+0000 7f3ac8fb1640 1 -- 172.21.15.27:0/2316927647 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ac0032440 con 0x7f3ac415e530 2024-09-12T03:22:02.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:02.524+0000 7f3ac8fb1640 1 -- 172.21.15.27:0/2316927647 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ac0032810 con 0x7f3ac415e530 2024-09-12T03:22:02.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:02.525+0000 7f3acc217640 1 -- 172.21.15.27:0/2316927647 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3ac415e530 msgr2=0x7f3ac40aff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:02.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:02.525+0000 7f3acc217640 1 --2- 172.21.15.27:0/2316927647 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3ac415e530 0x7f3ac40aff20 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f3ac40aeaf0 tx=0x7f3ac002a210 comp rx=0 tx=0).stop 2024-09-12T03:22:02.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:02.525+0000 7f3acc217640 1 -- 172.21.15.27:0/2316927647 shutdown_connections 2024-09-12T03:22:02.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:02.525+0000 7f3acc217640 1 --2- 172.21.15.27:0/2316927647 >> v2:172.21.15.45:3300/0 conn(0x7f3ac40aeb80 0x7f3ac40722f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:02.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:02.525+0000 7f3acc217640 1 --2- 172.21.15.27:0/2316927647 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f3ac415e910 0x7f3ac40b4710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:02.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:02.525+0000 7f3acc217640 1 --2- 172.21.15.27:0/2316927647 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3ac415e530 0x7f3ac40aff20 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:02.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:02.525+0000 7f3acc217640 1 -- 172.21.15.27:0/2316927647 >> 172.21.15.27:0/2316927647 conn(0x7f3ac406bb30 msgr2=0x7f3ac406bf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:02.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:02.525+0000 7f3acc217640 1 -- 172.21.15.27:0/2316927647 shutdown_connections 2024-09-12T03:22:02.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:02.525+0000 7f3acc217640 1 -- 172.21.15.27:0/2316927647 wait complete. 2024-09-12T03:22:02.529 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:02.526+0000 7f3acc217640 1 Processor -- start 2024-09-12T03:22:02.529 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:02.526+0000 7f3acc217640 1 -- start start 2024-09-12T03:22:02.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:02.527+0000 7f3acc217640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3ac40aeb80 0x7f3ac4143d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:02.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:02.527+0000 7f3acc217640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f3ac415e530 0x7f3ac40cc960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:02.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:02.527+0000 7f3acc217640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3ac415e910 0x7f3ac40b1e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:02.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:02.527+0000 7f3acc217640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f3ac4071ff0 con 0x7f3ac415e910 2024-09-12T03:22:02.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:02.527+0000 7f3acc217640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f3ac406e1a0 con 0x7f3ac40aeb80 2024-09-12T03:22:02.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:02.527+0000 7f3acc217640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f3ac4072830 con 0x7f3ac415e530 2024-09-12T03:22:02.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:02.527+0000 7f3aca7b4640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3ac415e910 0x7f3ac40b1e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:02.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:02.527+0000 7f3aca7b4640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3ac415e910 0x7f3ac40b1e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42398/0 (socket says 172.21.15.27:42398) 2024-09-12T03:22:02.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:02.527+0000 7f3aca7b4640 1 -- 172.21.15.27:0/100280920 learned_addr learned my addr 172.21.15.27:0/100280920 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:22:02.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:02.527+0000 7f3ac9fb3640 1 --2- 172.21.15.27:0/100280920 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3ac40aeb80 0x7f3ac4143d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:02.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:02.527+0000 7f3ac97b2640 1 --2- 172.21.15.27:0/100280920 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f3ac415e530 0x7f3ac40cc960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:02.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:02.527+0000 7f3aca7b4640 1 -- 172.21.15.27:0/100280920 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f3ac415e530 msgr2=0x7f3ac40cc960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:02.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:02.527+0000 7f3aca7b4640 1 --2- 172.21.15.27:0/100280920 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f3ac415e530 0x7f3ac40cc960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:02.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:02.527+0000 7f3aca7b4640 1 -- 172.21.15.27:0/100280920 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3ac40aeb80 msgr2=0x7f3ac4143d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:02.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:02.527+0000 7f3aca7b4640 1 --2- 172.21.15.27:0/100280920 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3ac40aeb80 0x7f3ac4143d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:02.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:02.527+0000 7f3aca7b4640 1 -- 172.21.15.27:0/100280920 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ac40cb620 con 0x7f3ac415e910 2024-09-12T03:22:02.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:02.527+0000 7f3ac97b2640 1 --2- 172.21.15.27:0/100280920 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f3ac415e530 0x7f3ac40cc960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:22:02.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:02.528+0000 7f3aca7b4640 1 --2- 172.21.15.27:0/100280920 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3ac415e910 0x7f3ac40b1e30 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f3abc00cd70 tx=0x7f3abc00a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:02.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:02.528+0000 7f3aba7fc640 1 -- 172.21.15.27:0/100280920 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3abc003b90 con 0x7f3ac415e910 2024-09-12T03:22:02.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:02.528+0000 7f3aba7fc640 1 -- 172.21.15.27:0/100280920 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3abc003d30 con 0x7f3ac415e910 2024-09-12T03:22:02.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:02.528+0000 7f3aba7fc640 1 -- 172.21.15.27:0/100280920 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3abc004240 con 0x7f3ac415e910 2024-09-12T03:22:02.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:02.528+0000 7f3acc217640 1 -- 172.21.15.27:0/100280920 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ac40cb8c0 con 0x7f3ac415e910 2024-09-12T03:22:02.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:02.528+0000 7f3acc217640 1 -- 172.21.15.27:0/100280920 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ac40cbe80 con 0x7f3ac415e910 2024-09-12T03:22:02.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:02.529+0000 7f3acc217640 1 -- 172.21.15.27:0/100280920 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ac4069550 con 0x7f3ac415e910 2024-09-12T03:22:02.532 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:02.529+0000 7f3aba7fc640 1 -- 172.21.15.27:0/100280920 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 20) ==== 90906+0+0 (secure 0 0 0) 0x7f3abc0043e0 con 0x7f3ac415e910 2024-09-12T03:22:02.532 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:02.530+0000 7f3aba7fc640 1 --2- 172.21.15.27:0/100280920 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f3aa806c2b0 0x7f3aa806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:02.532 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:02.530+0000 7f3ac9fb3640 1 --2- 172.21.15.27:0/100280920 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f3aa806c2b0 0x7f3aa806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:02.532 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:02.530+0000 7f3aba7fc640 1 -- 172.21.15.27:0/100280920 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5737+0+0 (secure 0 0 0) 0x7f3abc08c240 con 0x7f3ac415e910 2024-09-12T03:22:02.532 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:02.530+0000 7f3ac9fb3640 1 --2- 172.21.15.27:0/100280920 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f3aa806c2b0 0x7f3aa806e770 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f3ac0034000 tx=0x7f3ac002e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:02.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:02.534+0000 7f3aba7fc640 1 -- 172.21.15.27:0/100280920 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f3ac4069550 con 0x7f3ac415e910 2024-09-12T03:22:02.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:02.743+0000 7f3acc217640 1 -- 172.21.15.27:0/100280920 --> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "vol_name": "cephfs", "sub_name": "subvol35811", "snap_name": "fake", "group_name": "subvol_grp35010", "target": ["mon-mgr", ""]}) -- 0x7f3ac40aff20 con 0x7f3aa806c2b0 2024-09-12T03:22:02.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:02.778+0000 7f3aba7fc640 1 -- 172.21.15.27:0/100280920 <== mgr.4113 v2:172.21.15.27:6836/1226920526 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f3ac40aff20 con 0x7f3aa806c2b0 2024-09-12T03:22:02.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:02.780+0000 7f3acc217640 1 -- 172.21.15.27:0/100280920 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f3aa806c2b0 msgr2=0x7f3aa806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:02.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:02.780+0000 7f3acc217640 1 --2- 172.21.15.27:0/100280920 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f3aa806c2b0 0x7f3aa806e770 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f3ac0034000 tx=0x7f3ac002e000 comp rx=0 tx=0).stop 2024-09-12T03:22:02.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:02.780+0000 7f3acc217640 1 -- 172.21.15.27:0/100280920 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3ac415e910 msgr2=0x7f3ac40b1e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:02.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:02.780+0000 7f3acc217640 1 --2- 172.21.15.27:0/100280920 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3ac415e910 0x7f3ac40b1e30 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f3abc00cd70 tx=0x7f3abc00a660 comp rx=0 tx=0).stop 2024-09-12T03:22:02.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:02.780+0000 7f3acc217640 1 -- 172.21.15.27:0/100280920 shutdown_connections 2024-09-12T03:22:02.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:02.780+0000 7f3acc217640 1 --2- 172.21.15.27:0/100280920 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f3aa806c2b0 0x7f3aa806e770 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:02.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:02.780+0000 7f3acc217640 1 --2- 172.21.15.27:0/100280920 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3ac415e910 0x7f3ac40b1e30 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:02.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:02.780+0000 7f3acc217640 1 --2- 172.21.15.27:0/100280920 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f3ac415e530 0x7f3ac40cc960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:02.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:02.780+0000 7f3acc217640 1 --2- 172.21.15.27:0/100280920 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3ac40aeb80 0x7f3ac4143d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:02.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:02.780+0000 7f3acc217640 1 -- 172.21.15.27:0/100280920 >> 172.21.15.27:0/100280920 conn(0x7f3ac406bb30 msgr2=0x7f3ac40cdaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:02.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:02.780+0000 7f3acc217640 1 -- 172.21.15.27:0/100280920 shutdown_connections 2024-09-12T03:22:02.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:02.780+0000 7f3acc217640 1 -- 172.21.15.27:0/100280920 wait complete. 2024-09-12T03:22:02.791 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:22:02.791 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cat ./volumes/_nogroup/subvol43450/.meta) 2024-09-12T03:22:02.818 INFO:teuthology.orchestra.run.smithi027.stdout:[GLOBAL] 2024-09-12T03:22:02.818 INFO:teuthology.orchestra.run.smithi027.stdout:version = 1 2024-09-12T03:22:02.818 INFO:teuthology.orchestra.run.smithi027.stdout:type = subvolume 2024-09-12T03:22:02.819 INFO:teuthology.orchestra.run.smithi027.stdout:path = /volumes/_nogroup/subvol43450/11044fc2-4a1f-4c04-99ec-43281cd1bd6a 2024-09-12T03:22:02.819 INFO:teuthology.orchestra.run.smithi027.stdout:state = complete 2024-09-12T03:22:02.819 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:22:02.820 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:22:02.820 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cat ./volumes/subvol_grp35010/subvol35811/.meta) 2024-09-12T03:22:02.886 INFO:teuthology.orchestra.run.smithi027.stdout:[GLOBAL] 2024-09-12T03:22:02.886 INFO:teuthology.orchestra.run.smithi027.stdout:version = 1 2024-09-12T03:22:02.886 INFO:teuthology.orchestra.run.smithi027.stdout:type = subvolume 2024-09-12T03:22:02.886 INFO:teuthology.orchestra.run.smithi027.stdout:path = /volumes/subvol_grp35010/subvol35811/56ab55bb-6958-4c5b-b500-89b3151fce9c 2024-09-12T03:22:02.886 INFO:teuthology.orchestra.run.smithi027.stdout:state = complete 2024-09-12T03:22:02.886 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:22:02.887 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol43450 2024-09-12T03:22:03.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.032+0000 7f39713df640 1 Processor -- start 2024-09-12T03:22:03.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.032+0000 7f39713df640 1 -- start start 2024-09-12T03:22:03.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.033+0000 7f39713df640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f396c06f990 0x7f396c06e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:03.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.033+0000 7f39713df640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f396c0701c0 0x7f396c0705e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:03.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.033+0000 7f39713df640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f396c06ed60 con 0x7f396c0725b0 2024-09-12T03:22:03.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.033+0000 7f39713df640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f396c070b20 con 0x7f396c06f990 2024-09-12T03:22:03.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.033+0000 7f39713df640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f396c15ef90 con 0x7f396c0701c0 2024-09-12T03:22:03.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.033+0000 7f396affd640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f396c06f990 0x7f396c06e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:03.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.033+0000 7f396affd640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f396c06f990 0x7f396c06e820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42412/0 (socket says 172.21.15.27:42412) 2024-09-12T03:22:03.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.033+0000 7f396affd640 1 -- 172.21.15.27:0/1213385339 learned_addr learned my addr 172.21.15.27:0/1213385339 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:22:03.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.034+0000 7f396a7fc640 1 --2- 172.21.15.27:0/1213385339 >> v2:172.21.15.45:3300/0 conn(0x7f396c0701c0 0x7f396c0705e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:03.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.034+0000 7f396affd640 1 -- 172.21.15.27:0/1213385339 >> v2:172.21.15.45:3300/0 conn(0x7f396c0701c0 msgr2=0x7f396c0705e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:03.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.034+0000 7f396affd640 1 --2- 172.21.15.27:0/1213385339 >> v2:172.21.15.45:3300/0 conn(0x7f396c0701c0 0x7f396c0705e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:03.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.034+0000 7f396affd640 1 -- 172.21.15.27:0/1213385339 >> v1:172.21.15.45:6789/0 conn(0x7f396c0725b0 legacy=0x7f396c0729b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:03.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.034+0000 7f396affd640 1 -- 172.21.15.27:0/1213385339 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f396c15f270 con 0x7f396c06f990 2024-09-12T03:22:03.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.034+0000 7f396a7fc640 1 --2- 172.21.15.27:0/1213385339 >> v2:172.21.15.45:3300/0 conn(0x7f396c0701c0 0x7f396c0705e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:22:03.037 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.034+0000 7f396affd640 1 --2- 172.21.15.27:0/1213385339 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f396c06f990 0x7f396c06e820 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f396c0b0a70 tx=0x7f395c02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fa94436c4d7459f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:03.037 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.035+0000 7f3969ffb640 1 -- 172.21.15.27:0/1213385339 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f395c036070 con 0x7f396c06f990 2024-09-12T03:22:03.037 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.035+0000 7f3969ffb640 1 -- 172.21.15.27:0/1213385339 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f395c031600 con 0x7f396c06f990 2024-09-12T03:22:03.038 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.035+0000 7f3969ffb640 1 -- 172.21.15.27:0/1213385339 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f395c0319d0 con 0x7f396c06f990 2024-09-12T03:22:03.038 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.035+0000 7f39713df640 1 -- 172.21.15.27:0/1213385339 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f396c06f990 msgr2=0x7f396c06e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:03.038 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.035+0000 7f39713df640 1 --2- 172.21.15.27:0/1213385339 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f396c06f990 0x7f396c06e820 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f396c0b0a70 tx=0x7f395c02a6a0 comp rx=0 tx=0).stop 2024-09-12T03:22:03.038 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.035+0000 7f39713df640 1 -- 172.21.15.27:0/1213385339 shutdown_connections 2024-09-12T03:22:03.038 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.035+0000 7f39713df640 1 --2- 172.21.15.27:0/1213385339 >> v2:172.21.15.45:3300/0 conn(0x7f396c0701c0 0x7f396c0705e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:03.038 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.035+0000 7f39713df640 1 --2- 172.21.15.27:0/1213385339 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f396c06f990 0x7f396c06e820 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:03.038 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.035+0000 7f39713df640 1 -- 172.21.15.27:0/1213385339 >> 172.21.15.27:0/1213385339 conn(0x7f396c06d060 msgr2=0x7f396c06d470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:03.038 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.035+0000 7f39713df640 1 -- 172.21.15.27:0/1213385339 shutdown_connections 2024-09-12T03:22:03.038 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.035+0000 7f39713df640 1 -- 172.21.15.27:0/1213385339 wait complete. 2024-09-12T03:22:03.038 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.036+0000 7f39713df640 1 Processor -- start 2024-09-12T03:22:03.038 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.037+0000 7f39713df640 1 -- start start 2024-09-12T03:22:03.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.037+0000 7f39713df640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f396c06f990 0x7f396c157910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:03.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.037+0000 7f39713df640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f396c0701c0 0x7f396c150aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:03.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.037+0000 7f39713df640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f396c0725b0 0x7f396c155dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:03.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.037+0000 7f39713df640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f396c15fb10 con 0x7f396c0701c0 2024-09-12T03:22:03.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.037+0000 7f39713df640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f396c15f830 con 0x7f396c0725b0 2024-09-12T03:22:03.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.037+0000 7f39713df640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f396c15eb50 con 0x7f396c06f990 2024-09-12T03:22:03.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.037+0000 7f396a7fc640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f396c0701c0 0x7f396c150aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:03.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.037+0000 7f396a7fc640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f396c0701c0 0x7f396c150aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42416/0 (socket says 172.21.15.27:42416) 2024-09-12T03:22:03.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.037+0000 7f396a7fc640 1 -- 172.21.15.27:0/3195869463 learned_addr learned my addr 172.21.15.27:0/3195869463 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:22:03.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.038+0000 7f396b7fe640 1 --2- 172.21.15.27:0/3195869463 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f396c0725b0 0x7f396c155dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:03.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.038+0000 7f396affd640 1 --2- 172.21.15.27:0/3195869463 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f396c06f990 0x7f396c157910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:03.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.038+0000 7f396a7fc640 1 -- 172.21.15.27:0/3195869463 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f396c06f990 msgr2=0x7f396c157910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:03.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.038+0000 7f396a7fc640 1 --2- 172.21.15.27:0/3195869463 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f396c06f990 0x7f396c157910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:03.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.038+0000 7f396a7fc640 1 -- 172.21.15.27:0/3195869463 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f396c0725b0 msgr2=0x7f396c155dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:03.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.038+0000 7f396a7fc640 1 --2- 172.21.15.27:0/3195869463 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f396c0725b0 0x7f396c155dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:03.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.038+0000 7f396a7fc640 1 -- 172.21.15.27:0/3195869463 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f396c156870 con 0x7f396c0701c0 2024-09-12T03:22:03.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.038+0000 7f396a7fc640 1 --2- 172.21.15.27:0/3195869463 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f396c0701c0 0x7f396c150aa0 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f395400d7d0 tx=0x7f395400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:03.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.038+0000 7f39637fe640 1 -- 172.21.15.27:0/3195869463 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3954004030 con 0x7f396c0701c0 2024-09-12T03:22:03.043 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.038+0000 7f39637fe640 1 -- 172.21.15.27:0/3195869463 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39540041d0 con 0x7f396c0701c0 2024-09-12T03:22:03.043 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.038+0000 7f39713df640 1 -- 172.21.15.27:0/3195869463 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f396c156b50 con 0x7f396c0701c0 2024-09-12T03:22:03.043 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.038+0000 7f39637fe640 1 -- 172.21.15.27:0/3195869463 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39540096a0 con 0x7f396c0701c0 2024-09-12T03:22:03.043 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.038+0000 7f39713df640 1 -- 172.21.15.27:0/3195869463 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f396c157110 con 0x7f396c0701c0 2024-09-12T03:22:03.043 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.039+0000 7f39713df640 1 -- 172.21.15.27:0/3195869463 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f396c06e820 con 0x7f396c0701c0 2024-09-12T03:22:03.043 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.040+0000 7f39637fe640 1 -- 172.21.15.27:0/3195869463 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 20) ==== 90906+0+0 (secure 0 0 0) 0x7f3954009840 con 0x7f396c0701c0 2024-09-12T03:22:03.043 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.040+0000 7f39637fe640 1 --2- 172.21.15.27:0/3195869463 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f394c06c2e0 0x7f394c06e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:03.043 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.040+0000 7f39637fe640 1 -- 172.21.15.27:0/3195869463 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5737+0+0 (secure 0 0 0) 0x7f395408b680 con 0x7f396c0701c0 2024-09-12T03:22:03.043 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.040+0000 7f396affd640 1 --2- 172.21.15.27:0/3195869463 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f394c06c2e0 0x7f394c06e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:03.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.041+0000 7f396affd640 1 --2- 172.21.15.27:0/3195869463 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f394c06c2e0 0x7f394c06e7a0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f395c033000 tx=0x7f395c009000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:03.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.045+0000 7f39637fe640 1 -- 172.21.15.27:0/3195869463 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f39540594c0 con 0x7f396c0701c0 2024-09-12T03:22:03.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.245+0000 7f39713df640 1 -- 172.21.15.27:0/3195869463 --> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol43450", "target": ["mon-mgr", ""]}) -- 0x7f396c0b0020 con 0x7f394c06c2e0 2024-09-12T03:22:03.277 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.276+0000 7f39637fe640 1 -- 172.21.15.27:0/3195869463 <== mgr.4113 v2:172.21.15.27:6836/1226920526 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f396c0b0020 con 0x7f394c06c2e0 2024-09-12T03:22:03.280 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.279+0000 7f39713df640 1 -- 172.21.15.27:0/3195869463 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f394c06c2e0 msgr2=0x7f394c06e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:03.280 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.279+0000 7f39713df640 1 --2- 172.21.15.27:0/3195869463 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f394c06c2e0 0x7f394c06e7a0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f395c033000 tx=0x7f395c009000 comp rx=0 tx=0).stop 2024-09-12T03:22:03.280 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.279+0000 7f39713df640 1 -- 172.21.15.27:0/3195869463 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f396c0701c0 msgr2=0x7f396c150aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:03.280 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.279+0000 7f39713df640 1 --2- 172.21.15.27:0/3195869463 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f396c0701c0 0x7f396c150aa0 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f395400d7d0 tx=0x7f395400dc90 comp rx=0 tx=0).stop 2024-09-12T03:22:03.280 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.279+0000 7f39713df640 1 -- 172.21.15.27:0/3195869463 shutdown_connections 2024-09-12T03:22:03.280 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.279+0000 7f39713df640 1 --2- 172.21.15.27:0/3195869463 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f396c0725b0 0x7f396c155dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:03.281 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.279+0000 7f39713df640 1 --2- 172.21.15.27:0/3195869463 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f394c06c2e0 0x7f394c06e7a0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:03.281 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.279+0000 7f39713df640 1 --2- 172.21.15.27:0/3195869463 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f396c0701c0 0x7f396c150aa0 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:03.281 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.279+0000 7f39713df640 1 --2- 172.21.15.27:0/3195869463 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f396c06f990 0x7f396c157910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:03.281 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.279+0000 7f39713df640 1 -- 172.21.15.27:0/3195869463 >> 172.21.15.27:0/3195869463 conn(0x7f396c06d060 msgr2=0x7f396c06d440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:03.281 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.279+0000 7f39713df640 1 -- 172.21.15.27:0/3195869463 shutdown_connections 2024-09-12T03:22:03.281 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.279+0000 7f39713df640 1 -- 172.21.15.27:0/3195869463 wait complete. 2024-09-12T03:22:03.283 INFO:tasks.ceph.mgr.y.smithi027.stderr:2024-09-12T03:22:03.282+0000 7f32600e8640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:22:03.283 INFO:tasks.ceph.mgr.y.smithi027.stderr:2024-09-12T03:22:03.282+0000 7f32600e8640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:22:03.283 INFO:tasks.ceph.mgr.y.smithi027.stderr:2024-09-12T03:22:03.282+0000 7f32600e8640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:22:03.283 INFO:tasks.ceph.mgr.y.smithi027.stderr:2024-09-12T03:22:03.282+0000 7f32600e8640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:22:03.283 INFO:tasks.ceph.mgr.y.smithi027.stderr:2024-09-12T03:22:03.282+0000 7f32600e8640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:22:03.293 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol35811 subvol_grp35010 2024-09-12T03:22:03.297 INFO:tasks.ceph.mgr.y.smithi027.stderr:2024-09-12T03:22:03.296+0000 7f325f0e6640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:22:03.297 INFO:tasks.ceph.mgr.y.smithi027.stderr:2024-09-12T03:22:03.296+0000 7f325f0e6640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:22:03.297 INFO:tasks.ceph.mgr.y.smithi027.stderr:2024-09-12T03:22:03.296+0000 7f325f0e6640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:22:03.297 INFO:tasks.ceph.mgr.y.smithi027.stderr:2024-09-12T03:22:03.296+0000 7f325f0e6640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:22:03.297 INFO:tasks.ceph.mgr.y.smithi027.stderr:2024-09-12T03:22:03.296+0000 7f325f0e6640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:22:03.401 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.400+0000 7fa362465640 1 Processor -- start 2024-09-12T03:22:03.401 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.400+0000 7fa362465640 1 -- start start 2024-09-12T03:22:03.402 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.401+0000 7fa362465640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa35c0af9d0 0x7fa35c0b4010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:03.402 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.401+0000 7fa362465640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa35c0ce5e0 con 0x7fa35c06da00 2024-09-12T03:22:03.402 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.401+0000 7fa362465640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa35c0ceac0 con 0x7fa35c15ec30 2024-09-12T03:22:03.402 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.401+0000 7fa362465640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fa35c0d02d0 con 0x7fa35c0af9d0 2024-09-12T03:22:03.402 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.401+0000 7fa360a02640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fa35c06da00 0x7fa35c06de90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:51228/0 (socket says 172.21.15.27:51228) 2024-09-12T03:22:03.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.401+0000 7fa360a02640 1 -- 172.21.15.27:0/564874660 learned_addr learned my addr 172.21.15.27:0/564874660 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:22:03.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.401+0000 7fa353fff640 1 --2- 172.21.15.27:0/564874660 >> v2:172.21.15.45:3300/0 conn(0x7fa35c0af9d0 0x7fa35c0b4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:03.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.402+0000 7fa35b7fe640 1 -- 172.21.15.27:0/564874660 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3695301911 0 0) 0x7fa35c0ce5e0 con 0x7fa35c06da00 2024-09-12T03:22:03.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.402+0000 7fa35b7fe640 1 -- 172.21.15.27:0/564874660 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa348005730 con 0x7fa35c06da00 2024-09-12T03:22:03.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.402+0000 7fa35b7fe640 1 -- 172.21.15.27:0/564874660 <== mon.1 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2384061922 0 0) 0x7fa35c0ceac0 con 0x7fa35c15ec30 2024-09-12T03:22:03.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.402+0000 7fa35b7fe640 1 -- 172.21.15.27:0/564874660 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa35c0ce5e0 con 0x7fa35c15ec30 2024-09-12T03:22:03.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.402+0000 7fa353fff640 1 -- 172.21.15.27:0/564874660 >> v1:172.21.15.45:6789/0 conn(0x7fa35c15ec30 legacy=0x7fa35c0b3900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:03.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.402+0000 7fa353fff640 1 -- 172.21.15.27:0/564874660 >> v1:172.21.15.27:6789/0 conn(0x7fa35c06da00 legacy=0x7fa35c06de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:03.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.402+0000 7fa353fff640 1 -- 172.21.15.27:0/564874660 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa35c15f010 con 0x7fa35c0af9d0 2024-09-12T03:22:03.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.403+0000 7fa353fff640 1 --2- 172.21.15.27:0/564874660 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa35c0af9d0 0x7fa35c0b4010 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fa3440097a0 tx=0x7fa34402a230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=32e144a67b3fc31e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:03.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.403+0000 7fa35b7fe640 1 -- 172.21.15.27:0/564874660 <== mon.1 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa344037070 con 0x7fa35c0af9d0 2024-09-12T03:22:03.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.403+0000 7fa35b7fe640 1 -- 172.21.15.27:0/564874660 <== mon.1 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa344032440 con 0x7fa35c0af9d0 2024-09-12T03:22:03.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.403+0000 7fa35b7fe640 1 -- 172.21.15.27:0/564874660 <== mon.1 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa344032810 con 0x7fa35c0af9d0 2024-09-12T03:22:03.405 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.403+0000 7fa362465640 1 -- 172.21.15.27:0/564874660 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa35c0af9d0 msgr2=0x7fa35c0b4010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:03.406 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.403+0000 7fa362465640 1 --2- 172.21.15.27:0/564874660 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa35c0af9d0 0x7fa35c0b4010 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fa3440097a0 tx=0x7fa34402a230 comp rx=0 tx=0).stop 2024-09-12T03:22:03.406 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.404+0000 7fa362465640 1 -- 172.21.15.27:0/564874660 shutdown_connections 2024-09-12T03:22:03.406 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.404+0000 7fa362465640 1 --2- 172.21.15.27:0/564874660 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa35c0af9d0 0x7fa35c0b4010 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:03.406 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.404+0000 7fa362465640 1 -- 172.21.15.27:0/564874660 >> 172.21.15.27:0/564874660 conn(0x7fa35c06bb30 msgr2=0x7fa35c06bf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:03.406 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.404+0000 7fa362465640 1 -- 172.21.15.27:0/564874660 shutdown_connections 2024-09-12T03:22:03.406 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.404+0000 7fa362465640 1 -- 172.21.15.27:0/564874660 wait complete. 2024-09-12T03:22:03.406 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.405+0000 7fa362465640 1 Processor -- start 2024-09-12T03:22:03.406 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.405+0000 7fa362465640 1 -- start start 2024-09-12T03:22:03.409 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.405+0000 7fa362465640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa35c06da00 0x7fa35c14e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:03.409 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.406+0000 7fa362465640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa35c0af9d0 0x7fa35c15f450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:03.409 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.406+0000 7fa362465640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa35c15ec30 0x7fa35c150270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:03.409 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.406+0000 7fa362465640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fa35c0cfff0 con 0x7fa35c15ec30 2024-09-12T03:22:03.409 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.406+0000 7fa362465640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa35c06c650 con 0x7fa35c06da00 2024-09-12T03:22:03.409 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.406+0000 7fa362465640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fa35c0715f0 con 0x7fa35c0af9d0 2024-09-12T03:22:03.409 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.406+0000 7fa360a02640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa35c15ec30 0x7fa35c150270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:03.409 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.406+0000 7fa360a02640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa35c15ec30 0x7fa35c150270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42466/0 (socket says 172.21.15.27:42466) 2024-09-12T03:22:03.409 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.406+0000 7fa360a02640 1 -- 172.21.15.27:0/4108257832 learned_addr learned my addr 172.21.15.27:0/4108257832 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:22:03.409 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.406+0000 7fa35bfff640 1 --2- 172.21.15.27:0/4108257832 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa35c06da00 0x7fa35c14e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:03.409 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.406+0000 7fa353fff640 1 --2- 172.21.15.27:0/4108257832 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa35c0af9d0 0x7fa35c15f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:03.410 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.406+0000 7fa360a02640 1 -- 172.21.15.27:0/4108257832 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa35c0af9d0 msgr2=0x7fa35c15f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:03.410 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.406+0000 7fa360a02640 1 --2- 172.21.15.27:0/4108257832 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa35c0af9d0 0x7fa35c15f450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:03.410 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.406+0000 7fa360a02640 1 -- 172.21.15.27:0/4108257832 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa35c06da00 msgr2=0x7fa35c14e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:03.410 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.406+0000 7fa360a02640 1 --2- 172.21.15.27:0/4108257832 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa35c06da00 0x7fa35c14e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:03.410 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.406+0000 7fa360a02640 1 -- 172.21.15.27:0/4108257832 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa35c14f1d0 con 0x7fa35c15ec30 2024-09-12T03:22:03.410 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.406+0000 7fa360a02640 1 --2- 172.21.15.27:0/4108257832 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa35c15ec30 0x7fa35c150270 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7fa35400caf0 tx=0x7fa35400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:03.410 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.406+0000 7fa3597fa640 1 -- 172.21.15.27:0/4108257832 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa354017070 con 0x7fa35c15ec30 2024-09-12T03:22:03.410 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.406+0000 7fa3597fa640 1 -- 172.21.15.27:0/4108257832 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa35400aca0 con 0x7fa35c15ec30 2024-09-12T03:22:03.410 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.406+0000 7fa362465640 1 -- 172.21.15.27:0/4108257832 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa35c14f4b0 con 0x7fa35c15ec30 2024-09-12T03:22:03.410 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.406+0000 7fa3597fa640 1 -- 172.21.15.27:0/4108257832 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa354012620 con 0x7fa35c15ec30 2024-09-12T03:22:03.410 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.406+0000 7fa362465640 1 -- 172.21.15.27:0/4108257832 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa35c14fa70 con 0x7fa35c15ec30 2024-09-12T03:22:03.410 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.407+0000 7fa362465640 1 -- 172.21.15.27:0/4108257832 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa35c14ff60 con 0x7fa35c15ec30 2024-09-12T03:22:03.410 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.408+0000 7fa3597fa640 1 -- 172.21.15.27:0/4108257832 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 20) ==== 90906+0+0 (secure 0 0 0) 0x7fa35401a070 con 0x7fa35c15ec30 2024-09-12T03:22:03.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.408+0000 7fa3597fa640 1 --2- 172.21.15.27:0/4108257832 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fa33c06c2b0 0x7fa33c06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:03.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.408+0000 7fa3597fa640 1 -- 172.21.15.27:0/4108257832 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5737+0+0 (secure 0 0 0) 0x7fa35408b3e0 con 0x7fa35c15ec30 2024-09-12T03:22:03.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.408+0000 7fa35bfff640 1 --2- 172.21.15.27:0/4108257832 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fa33c06c2b0 0x7fa33c06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:03.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.409+0000 7fa35bfff640 1 --2- 172.21.15.27:0/4108257832 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fa33c06c2b0 0x7fa33c06e770 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fa35c15e100 tx=0x7fa34c009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:03.413 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.413+0000 7fa3597fa640 1 -- 172.21.15.27:0/4108257832 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fa354059740 con 0x7fa35c15ec30 2024-09-12T03:22:03.615 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.614+0000 7fa362465640 1 -- 172.21.15.27:0/4108257832 --> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol35811", "group_name": "subvol_grp35010", "target": ["mon-mgr", ""]}) -- 0x7fa35c14ef80 con 0x7fa33c06c2b0 2024-09-12T03:22:03.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.648+0000 7fa3597fa640 1 -- 172.21.15.27:0/4108257832 <== mgr.4113 v2:172.21.15.27:6836/1226920526 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fa35c14ef80 con 0x7fa33c06c2b0 2024-09-12T03:22:03.651 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.650+0000 7fa362465640 1 -- 172.21.15.27:0/4108257832 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fa33c06c2b0 msgr2=0x7fa33c06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:03.651 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.650+0000 7fa362465640 1 --2- 172.21.15.27:0/4108257832 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fa33c06c2b0 0x7fa33c06e770 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fa35c15e100 tx=0x7fa34c009040 comp rx=0 tx=0).stop 2024-09-12T03:22:03.651 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.650+0000 7fa362465640 1 -- 172.21.15.27:0/4108257832 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa35c15ec30 msgr2=0x7fa35c150270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:03.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.650+0000 7fa362465640 1 --2- 172.21.15.27:0/4108257832 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa35c15ec30 0x7fa35c150270 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7fa35400caf0 tx=0x7fa35400a060 comp rx=0 tx=0).stop 2024-09-12T03:22:03.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.650+0000 7fa362465640 1 -- 172.21.15.27:0/4108257832 shutdown_connections 2024-09-12T03:22:03.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.650+0000 7fa362465640 1 --2- 172.21.15.27:0/4108257832 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fa33c06c2b0 0x7fa33c06e770 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:03.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.650+0000 7fa362465640 1 --2- 172.21.15.27:0/4108257832 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa35c15ec30 0x7fa35c150270 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:03.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.650+0000 7fa362465640 1 --2- 172.21.15.27:0/4108257832 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa35c0af9d0 0x7fa35c15f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:03.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.650+0000 7fa362465640 1 --2- 172.21.15.27:0/4108257832 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa35c06da00 0x7fa35c14e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:03.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.650+0000 7fa362465640 1 -- 172.21.15.27:0/4108257832 >> 172.21.15.27:0/4108257832 conn(0x7fa35c06bb30 msgr2=0x7fa35c153070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:03.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.650+0000 7fa362465640 1 -- 172.21.15.27:0/4108257832 shutdown_connections 2024-09-12T03:22:03.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.650+0000 7fa362465640 1 -- 172.21.15.27:0/4108257832 wait complete. 2024-09-12T03:22:03.663 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol7372 2024-09-12T03:22:03.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.808+0000 7f878f21e640 1 Processor -- start 2024-09-12T03:22:03.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.808+0000 7f878f21e640 1 -- start start 2024-09-12T03:22:03.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.809+0000 7f878f21e640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f8788162040 0x7f87880d1da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:03.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.809+0000 7f878f21e640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f87880d2e30 con 0x7f87880d3790 2024-09-12T03:22:03.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.809+0000 7f878f21e640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8788162420 con 0x7f878806dfd0 2024-09-12T03:22:03.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.809+0000 7f878f21e640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f87880d30d0 con 0x7f8788162040 2024-09-12T03:22:03.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.809+0000 7f8787fff640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f87880d3790 0x7f878806d8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:51232/0 (socket says 172.21.15.27:51232) 2024-09-12T03:22:03.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.809+0000 7f8787fff640 1 -- 172.21.15.27:0/1666025156 learned_addr learned my addr 172.21.15.27:0/1666025156 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:22:03.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.810+0000 7f878cfba640 1 --2- 172.21.15.27:0/1666025156 >> v2:172.21.15.45:3300/0 conn(0x7f8788162040 0x7f87880d1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:03.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.810+0000 7f87877fe640 1 -- 172.21.15.27:0/1666025156 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 399986122 0 0) 0x7f87880d2e30 con 0x7f87880d3790 2024-09-12T03:22:03.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.810+0000 7f87877fe640 1 -- 172.21.15.27:0/1666025156 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8774003520 con 0x7f87880d3790 2024-09-12T03:22:03.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.810+0000 7f878cfba640 1 -- 172.21.15.27:0/1666025156 >> v1:172.21.15.45:6789/0 conn(0x7f878806dfd0 legacy=0x7f87880cf320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:03.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.810+0000 7f878cfba640 1 -- 172.21.15.27:0/1666025156 >> v1:172.21.15.27:6789/0 conn(0x7f87880d3790 legacy=0x7f878806d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:03.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.810+0000 7f878cfba640 1 -- 172.21.15.27:0/1666025156 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87880d33b0 con 0x7f8788162040 2024-09-12T03:22:03.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.811+0000 7f878cfba640 1 --2- 172.21.15.27:0/1666025156 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8788162040 0x7f87880d1da0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f8780009d90 tx=0x7f878002a6f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2a76c64c8dcf2678 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:03.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.811+0000 7f87877fe640 1 -- 172.21.15.27:0/1666025156 <== mon.1 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8780007740 con 0x7f8788162040 2024-09-12T03:22:03.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.811+0000 7f87877fe640 1 -- 172.21.15.27:0/1666025156 <== mon.1 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87800078e0 con 0x7f8788162040 2024-09-12T03:22:03.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.811+0000 7f87877fe640 1 -- 172.21.15.27:0/1666025156 <== mon.1 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8780007cb0 con 0x7f8788162040 2024-09-12T03:22:03.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.811+0000 7f878f21e640 1 -- 172.21.15.27:0/1666025156 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8788162040 msgr2=0x7f87880d1da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:03.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.811+0000 7f878f21e640 1 --2- 172.21.15.27:0/1666025156 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8788162040 0x7f87880d1da0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f8780009d90 tx=0x7f878002a6f0 comp rx=0 tx=0).stop 2024-09-12T03:22:03.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.812+0000 7f878f21e640 1 -- 172.21.15.27:0/1666025156 shutdown_connections 2024-09-12T03:22:03.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.812+0000 7f878f21e640 1 --2- 172.21.15.27:0/1666025156 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8788162040 0x7f87880d1da0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:03.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.812+0000 7f878f21e640 1 -- 172.21.15.27:0/1666025156 >> 172.21.15.27:0/1666025156 conn(0x7f87880b6700 msgr2=0x7f87880b6b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:03.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.812+0000 7f878f21e640 1 -- 172.21.15.27:0/1666025156 shutdown_connections 2024-09-12T03:22:03.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.812+0000 7f878f21e640 1 -- 172.21.15.27:0/1666025156 wait complete. 2024-09-12T03:22:03.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.813+0000 7f878f21e640 1 Processor -- start 2024-09-12T03:22:03.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.813+0000 7f878f21e640 1 -- start start 2024-09-12T03:22:03.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.814+0000 7f878f21e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f878806dfd0 0x7f8788157d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:03.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.814+0000 7f878f21e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f87880d3790 0x7f87880b50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:03.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.814+0000 7f878f21e640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f8788162040 0x7f87880b5600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:03.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.814+0000 7f878f21e640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f87880d4150 con 0x7f878806dfd0 2024-09-12T03:22:03.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.814+0000 7f878f21e640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f87880d4530 con 0x7f87880d3790 2024-09-12T03:22:03.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.814+0000 7f878f21e640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f87880d42d0 con 0x7f8788162040 2024-09-12T03:22:03.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.814+0000 7f878cfba640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f878806dfd0 0x7f8788157d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:03.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.814+0000 7f878cfba640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f878806dfd0 0x7f8788157d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42472/0 (socket says 172.21.15.27:42472) 2024-09-12T03:22:03.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.814+0000 7f878cfba640 1 -- 172.21.15.27:0/2209342298 learned_addr learned my addr 172.21.15.27:0/2209342298 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:22:03.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.814+0000 7f8787fff640 1 --2- 172.21.15.27:0/2209342298 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f87880d3790 0x7f87880b50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:03.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.814+0000 7f878d7bb640 1 --2- 172.21.15.27:0/2209342298 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f8788162040 0x7f87880b5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:03.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.814+0000 7f878cfba640 1 -- 172.21.15.27:0/2209342298 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f8788162040 msgr2=0x7f87880b5600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:03.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.814+0000 7f878cfba640 1 --2- 172.21.15.27:0/2209342298 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f8788162040 0x7f87880b5600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:03.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.814+0000 7f878cfba640 1 -- 172.21.15.27:0/2209342298 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f87880d3790 msgr2=0x7f87880b50c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:03.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.814+0000 7f878cfba640 1 --2- 172.21.15.27:0/2209342298 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f87880d3790 0x7f87880b50c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:03.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.814+0000 7f878cfba640 1 -- 172.21.15.27:0/2209342298 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f878815a5a0 con 0x7f878806dfd0 2024-09-12T03:22:03.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.814+0000 7f8787fff640 1 --2- 172.21.15.27:0/2209342298 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f87880d3790 0x7f87880b50c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:22:03.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.814+0000 7f878cfba640 1 --2- 172.21.15.27:0/2209342298 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f878806dfd0 0x7f8788157d80 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f8780009d90 tx=0x7f8780004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:03.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.814+0000 7f87857fa640 1 -- 172.21.15.27:0/2209342298 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f878003f070 con 0x7f878806dfd0 2024-09-12T03:22:03.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.814+0000 7f87857fa640 1 -- 172.21.15.27:0/2209342298 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8780031c40 con 0x7f878806dfd0 2024-09-12T03:22:03.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.814+0000 7f87857fa640 1 -- 172.21.15.27:0/2209342298 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8780044460 con 0x7f878806dfd0 2024-09-12T03:22:03.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.815+0000 7f878f21e640 1 -- 172.21.15.27:0/2209342298 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87881f5660 con 0x7f878806dfd0 2024-09-12T03:22:03.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.815+0000 7f878f21e640 1 -- 172.21.15.27:0/2209342298 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f878815ab40 con 0x7f878806dfd0 2024-09-12T03:22:03.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.815+0000 7f878f21e640 1 -- 172.21.15.27:0/2209342298 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87880ce8b0 con 0x7f878806dfd0 2024-09-12T03:22:03.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.816+0000 7f87857fa640 1 -- 172.21.15.27:0/2209342298 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 20) ==== 90906+0+0 (secure 0 0 0) 0x7f878003b410 con 0x7f878806dfd0 2024-09-12T03:22:03.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.816+0000 7f87857fa640 1 --2- 172.21.15.27:0/2209342298 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f876806c2b0 0x7f876806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:03.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.816+0000 7f8787fff640 1 --2- 172.21.15.27:0/2209342298 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f876806c2b0 0x7f876806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:03.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.816+0000 7f87857fa640 1 -- 172.21.15.27:0/2209342298 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5737+0+0 (secure 0 0 0) 0x7f87800acfa0 con 0x7f878806dfd0 2024-09-12T03:22:03.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.817+0000 7f8787fff640 1 --2- 172.21.15.27:0/2209342298 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f876806c2b0 0x7f876806e770 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f87780011e0 tx=0x7f8778009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:03.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:03.821+0000 7f87857fa640 1 -- 172.21.15.27:0/2209342298 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f878007b1f0 con 0x7f878806dfd0 2024-09-12T03:22:04.023 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:04.022+0000 7f878f21e640 1 -- 172.21.15.27:0/2209342298 --> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol7372", "target": ["mon-mgr", ""]}) -- 0x7f878806e6e0 con 0x7f876806c2b0 2024-09-12T03:22:04.039 INFO:tasks.ceph.mgr.y.smithi027.stderr:2024-09-12T03:22:04.039+0000 7f325d8e3640 -1 mgr.server reply reply (11) Resource temporarily unavailable subvolume 'subvol7372' is not ready for operation rm (use --force to override) 2024-09-12T03:22:04.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:04.039+0000 7f87857fa640 1 -- 172.21.15.27:0/2209342298 <== mgr.4113 v2:172.21.15.27:6836/1226920526 1 ==== mgr_command_reply(tid 0: -11 subvolume 'subvol7372' is not ready for operation rm (use --force to override)) ==== 86+0+0 (secure 0 0 0) 0x7f878806e6e0 con 0x7f876806c2b0 2024-09-12T03:22:04.040 INFO:teuthology.orchestra.run.smithi027.stderr:Error EAGAIN: subvolume 'subvol7372' is not ready for operation rm (use --force to override) 2024-09-12T03:22:04.045 DEBUG:teuthology.orchestra.run:got remote process result: 11 2024-09-12T03:22:04.045 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:22:04.046 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cat ./volumes/_nogroup/subvol7372/.meta) 2024-09-12T03:22:04.072 INFO:teuthology.orchestra.run.smithi027.stdout:[GLOBAL] 2024-09-12T03:22:04.072 INFO:teuthology.orchestra.run.smithi027.stdout:version = 1 2024-09-12T03:22:04.072 INFO:teuthology.orchestra.run.smithi027.stdout:type = clone 2024-09-12T03:22:04.072 INFO:teuthology.orchestra.run.smithi027.stdout:path = /volumes/_nogroup/subvol7372/7f4a6d24-0de7-4198-a773-301b654f504b 2024-09-12T03:22:04.072 INFO:teuthology.orchestra.run.smithi027.stdout:state = pending 2024-09-12T03:22:04.073 INFO:teuthology.orchestra.run.smithi027.stdout:[source] 2024-09-12T03:22:04.073 INFO:teuthology.orchestra.run.smithi027.stdout:volume = fake 2024-09-12T03:22:04.073 INFO:teuthology.orchestra.run.smithi027.stdout:subvolume = fake 2024-09-12T03:22:04.073 INFO:teuthology.orchestra.run.smithi027.stdout:snapshot = fake 2024-09-12T03:22:04.075 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol7372 --force 2024-09-12T03:22:04.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:04.220+0000 7fca04a27640 1 Processor -- start 2024-09-12T03:22:04.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:04.220+0000 7fca04a27640 1 -- start start 2024-09-12T03:22:04.223 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:04.221+0000 7fca04a27640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fca0015a270 0x7fca0006cff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:04.223 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:04.221+0000 7fca04a27640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fca000cfb20 0x7fca000726a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:04.223 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:04.221+0000 7fca04a27640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fca000ce650 con 0x7fca0006d530 2024-09-12T03:22:04.223 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:04.221+0000 7fca04a27640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7fca000cef80 con 0x7fca000cfb20 2024-09-12T03:22:04.223 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:04.221+0000 7fca04a27640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fca000ce930 con 0x7fca0015a270 2024-09-12T03:22:04.223 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:04.221+0000 7fc9fed9d640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fca000cfb20 0x7fca000726a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:04.223 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:04.221+0000 7fc9fdd9b640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fca0006d530 0x7fca000cf410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:51240/0 (socket says 172.21.15.27:51240) 2024-09-12T03:22:04.223 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:04.221+0000 7fc9fed9d640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fca000cfb20 0x7fca000726a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42488/0 (socket says 172.21.15.27:42488) 2024-09-12T03:22:04.223 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:04.221+0000 7fc9fdd9b640 1 -- 172.21.15.27:0/3192032561 learned_addr learned my addr 172.21.15.27:0/3192032561 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:22:04.223 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:04.222+0000 7fc9fe59c640 1 --2- 172.21.15.27:0/3192032561 >> v2:172.21.15.45:3300/0 conn(0x7fca0015a270 0x7fca0006cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:04.223 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:04.222+0000 7fc9fed9d640 1 -- 172.21.15.27:0/3192032561 >> v2:172.21.15.45:3300/0 conn(0x7fca0015a270 msgr2=0x7fca0006cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:04.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:04.222+0000 7fc9fed9d640 1 --2- 172.21.15.27:0/3192032561 >> v2:172.21.15.45:3300/0 conn(0x7fca0015a270 0x7fca0006cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:04.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:04.222+0000 7fc9fed9d640 1 -- 172.21.15.27:0/3192032561 >> v1:172.21.15.27:6789/0 conn(0x7fca0006d530 legacy=0x7fca000cf410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:04.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:04.222+0000 7fc9fd59a640 1 -- 172.21.15.27:0/3192032561 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 582574874 0 0) 0x7fca000ce650 con 0x7fca0006d530 2024-09-12T03:22:04.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:04.222+0000 7fc9fed9d640 1 -- 172.21.15.27:0/3192032561 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca000cec10 con 0x7fca000cfb20 2024-09-12T03:22:04.225 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:04.222+0000 7fc9fed9d640 1 --2- 172.21.15.27:0/3192032561 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fca000cfb20 0x7fca000726a0 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7fca00050600 tx=0x7fc9f002a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad075b5fc7e506c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:04.225 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:04.223+0000 7fc9fd59a640 1 -- 172.21.15.27:0/3192032561 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9f0007740 con 0x7fca000cfb20 2024-09-12T03:22:04.225 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:04.223+0000 7fc9fd59a640 1 -- 172.21.15.27:0/3192032561 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc9f00078e0 con 0x7fca000cfb20 2024-09-12T03:22:04.225 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:04.223+0000 7fc9fd59a640 1 -- 172.21.15.27:0/3192032561 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9f0007cb0 con 0x7fca000cfb20 2024-09-12T03:22:04.225 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:04.223+0000 7fca04a27640 1 -- 172.21.15.27:0/3192032561 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fca000cfb20 msgr2=0x7fca000726a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:04.225 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:04.223+0000 7fca04a27640 1 --2- 172.21.15.27:0/3192032561 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fca000cfb20 0x7fca000726a0 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7fca00050600 tx=0x7fc9f002a680 comp rx=0 tx=0).stop 2024-09-12T03:22:04.225 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:04.223+0000 7fca04a27640 1 -- 172.21.15.27:0/3192032561 shutdown_connections 2024-09-12T03:22:04.225 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:04.223+0000 7fca04a27640 1 --2- 172.21.15.27:0/3192032561 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fca000cfb20 0x7fca000726a0 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:04.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:04.223+0000 7fca04a27640 1 --2- 172.21.15.27:0/3192032561 >> v2:172.21.15.45:3300/0 conn(0x7fca0015a270 0x7fca0006cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:04.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:04.223+0000 7fca04a27640 1 -- 172.21.15.27:0/3192032561 >> 172.21.15.27:0/3192032561 conn(0x7fca0006bb30 msgr2=0x7fca0006bf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:04.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:04.223+0000 7fca04a27640 1 -- 172.21.15.27:0/3192032561 shutdown_connections 2024-09-12T03:22:04.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:04.223+0000 7fca04a27640 1 -- 172.21.15.27:0/3192032561 wait complete. 2024-09-12T03:22:04.228 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:04.225+0000 7fca04a27640 1 Processor -- start 2024-09-12T03:22:04.228 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:04.225+0000 7fca04a27640 1 -- start start 2024-09-12T03:22:04.228 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:04.225+0000 7fca04a27640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fca0006d530 0x7fca00154010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:04.228 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:04.225+0000 7fca04a27640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fca000cfb20 0x7fca0014d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:04.228 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:04.225+0000 7fca04a27640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fca0015a270 0x7fca0015db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:04.228 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:04.225+0000 7fca04a27640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fca0006c7d0 con 0x7fca0006d530 2024-09-12T03:22:04.228 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:04.225+0000 7fca04a27640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fca0006c650 con 0x7fca0015a270 2024-09-12T03:22:04.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:04.225+0000 7fca04a27640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fca0006c950 con 0x7fca000cfb20 2024-09-12T03:22:04.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:04.226+0000 7fc9fe59c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fca0006d530 0x7fca00154010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:04.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:04.226+0000 7fc9fe59c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fca0006d530 0x7fca00154010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42490/0 (socket says 172.21.15.27:42490) 2024-09-12T03:22:04.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:04.226+0000 7fc9fe59c640 1 -- 172.21.15.27:0/2957862852 learned_addr learned my addr 172.21.15.27:0/2957862852 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:22:04.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:04.226+0000 7fc9fed9d640 1 --2- 172.21.15.27:0/2957862852 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fca0015a270 0x7fca0015db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:04.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:04.226+0000 7fc9fdd9b640 1 --2- 172.21.15.27:0/2957862852 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fca000cfb20 0x7fca0014d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:04.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:04.226+0000 7fc9fe59c640 1 -- 172.21.15.27:0/2957862852 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fca000cfb20 msgr2=0x7fca0014d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:04.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:04.226+0000 7fc9fe59c640 1 --2- 172.21.15.27:0/2957862852 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fca000cfb20 0x7fca0014d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:04.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:04.226+0000 7fc9fe59c640 1 -- 172.21.15.27:0/2957862852 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fca0015a270 msgr2=0x7fca0015db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:04.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:04.226+0000 7fc9fe59c640 1 --2- 172.21.15.27:0/2957862852 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fca0015a270 0x7fca0015db20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:04.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:04.226+0000 7fc9fe59c640 1 -- 172.21.15.27:0/2957862852 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca0015e5c0 con 0x7fca0006d530 2024-09-12T03:22:04.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:04.226+0000 7fc9fe59c640 1 --2- 172.21.15.27:0/2957862852 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fca0006d530 0x7fca00154010 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7fc9f800d5f0 tx=0x7fc9f800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:04.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:04.226+0000 7fc9f6ffd640 1 -- 172.21.15.27:0/2957862852 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9f80125f0 con 0x7fca0006d530 2024-09-12T03:22:04.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:04.226+0000 7fc9f6ffd640 1 -- 172.21.15.27:0/2957862852 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc9f8019070 con 0x7fca0006d530 2024-09-12T03:22:04.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:04.226+0000 7fca04a27640 1 -- 172.21.15.27:0/2957862852 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca0015e8a0 con 0x7fca0006d530 2024-09-12T03:22:04.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:04.226+0000 7fc9f6ffd640 1 -- 172.21.15.27:0/2957862852 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9f80129d0 con 0x7fca0006d530 2024-09-12T03:22:04.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:04.226+0000 7fca04a27640 1 -- 172.21.15.27:0/2957862852 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca00153810 con 0x7fca0006d530 2024-09-12T03:22:04.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:04.227+0000 7fca04a27640 1 -- 172.21.15.27:0/2957862852 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca000b5900 con 0x7fca0006d530 2024-09-12T03:22:04.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:04.228+0000 7fc9f6ffd640 1 -- 172.21.15.27:0/2957862852 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 20) ==== 90906+0+0 (secure 0 0 0) 0x7fc9f8012b70 con 0x7fca0006d530 2024-09-12T03:22:04.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:04.228+0000 7fc9f6ffd640 1 --2- 172.21.15.27:0/2957862852 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fc9e006c2b0 0x7fc9e006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:04.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:04.228+0000 7fc9f6ffd640 1 -- 172.21.15.27:0/2957862852 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5737+0+0 (secure 0 0 0) 0x7fc9f808b8c0 con 0x7fca0006d530 2024-09-12T03:22:04.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:04.228+0000 7fc9fdd9b640 1 --2- 172.21.15.27:0/2957862852 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fc9e006c2b0 0x7fc9e006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:04.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:04.229+0000 7fc9fdd9b640 1 --2- 172.21.15.27:0/2957862852 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fc9e006c2b0 0x7fc9e006e770 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fc9e8000f60 tx=0x7fc9e8007b50 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:04.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:04.233+0000 7fc9f6ffd640 1 -- 172.21.15.27:0/2957862852 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fca000b5900 con 0x7fca0006d530 2024-09-12T03:22:04.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:04.335+0000 7fc9f6ffd640 1 -- 172.21.15.27:0/2957862852 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mgrmap(e 21) ==== 91008+0+0 (secure 0 0 0) 0x7fc9f8055cc0 con 0x7fca0006d530 2024-09-12T03:22:04.437 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:04.436+0000 7fca04a27640 1 -- 172.21.15.27:0/2957862852 --> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol7372", "force": true, "target": ["mon-mgr", ""]}) -- 0x7fca00063320 con 0x7fc9e006c2b0 2024-09-12T03:22:04.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:04.459+0000 7fc9f6ffd640 1 -- 172.21.15.27:0/2957862852 <== mgr.4113 v2:172.21.15.27:6836/1226920526 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fc9e8007170 con 0x7fc9e006c2b0 2024-09-12T03:22:04.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:04.462+0000 7fca04a27640 1 -- 172.21.15.27:0/2957862852 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fc9e006c2b0 msgr2=0x7fc9e006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:04.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:04.462+0000 7fca04a27640 1 --2- 172.21.15.27:0/2957862852 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fc9e006c2b0 0x7fc9e006e770 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fc9e8000f60 tx=0x7fc9e8007b50 comp rx=0 tx=0).stop 2024-09-12T03:22:04.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:04.462+0000 7fca04a27640 1 -- 172.21.15.27:0/2957862852 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fca0006d530 msgr2=0x7fca00154010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:04.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:04.462+0000 7fca04a27640 1 --2- 172.21.15.27:0/2957862852 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fca0006d530 0x7fca00154010 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7fc9f800d5f0 tx=0x7fc9f800dab0 comp rx=0 tx=0).stop 2024-09-12T03:22:04.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:04.462+0000 7fca04a27640 1 -- 172.21.15.27:0/2957862852 shutdown_connections 2024-09-12T03:22:04.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:04.462+0000 7fca04a27640 1 --2- 172.21.15.27:0/2957862852 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fca0015a270 0x7fca0015db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:04.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:04.462+0000 7fca04a27640 1 --2- 172.21.15.27:0/2957862852 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fca000cfb20 0x7fca0014d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:04.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:04.462+0000 7fca04a27640 1 --2- 172.21.15.27:0/2957862852 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fc9e006c2b0 0x7fc9e006e770 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:04.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:04.462+0000 7fca04a27640 1 --2- 172.21.15.27:0/2957862852 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fca0006d530 0x7fca00154010 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:04.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:04.462+0000 7fca04a27640 1 -- 172.21.15.27:0/2957862852 >> 172.21.15.27:0/2957862852 conn(0x7fca0006bb30 msgr2=0x7fca000b0c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:04.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:04.462+0000 7fca04a27640 1 -- 172.21.15.27:0/2957862852 shutdown_connections 2024-09-12T03:22:04.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:04.462+0000 7fca04a27640 1 -- 172.21.15.27:0/2957862852 wait complete. 2024-09-12T03:22:04.476 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume ls cephfs 2024-09-12T03:22:04.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:04.579+0000 7f7357f4b640 1 Processor -- start 2024-09-12T03:22:04.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:04.579+0000 7f7357f4b640 1 -- start start 2024-09-12T03:22:04.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:04.580+0000 7f7357f4b640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f735015e650 0x7f73500d1d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:04.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:04.580+0000 7f7357f4b640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f735006cc40 0x7f735006d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:04.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:04.580+0000 7f7357f4b640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f735006db30 0x7f735006df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:04.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:04.580+0000 7f7357f4b640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f735006d5d0 con 0x7f735015e650 2024-09-12T03:22:04.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:04.580+0000 7f7357f4b640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f73500d2280 con 0x7f735006cc40 2024-09-12T03:22:04.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:04.580+0000 7f7357f4b640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f735006d850 con 0x7f735006db30 2024-09-12T03:22:04.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:04.580+0000 7f7355ce7640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f735015e650 0x7f73500d1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:04.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:04.580+0000 7f7355ce7640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f735015e650 0x7f73500d1d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42498/0 (socket says 172.21.15.27:42498) 2024-09-12T03:22:04.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:04.580+0000 7f7355ce7640 1 -- 172.21.15.27:0/3251224801 learned_addr learned my addr 172.21.15.27:0/3251224801 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:22:04.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:04.581+0000 7f73554e6640 1 --2- 172.21.15.27:0/3251224801 >> v2:172.21.15.45:3300/0 conn(0x7f735006cc40 0x7f735006d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:04.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:04.581+0000 7f73564e8640 1 --2- 172.21.15.27:0/3251224801 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f735006db30 0x7f735006df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:04.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:04.581+0000 7f7355ce7640 1 -- 172.21.15.27:0/3251224801 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f735006db30 msgr2=0x7f735006df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:04.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:04.581+0000 7f7355ce7640 1 --2- 172.21.15.27:0/3251224801 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f735006db30 0x7f735006df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:04.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:04.581+0000 7f7355ce7640 1 -- 172.21.15.27:0/3251224801 >> v2:172.21.15.45:3300/0 conn(0x7f735006cc40 msgr2=0x7f735006d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:04.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:04.581+0000 7f7355ce7640 1 --2- 172.21.15.27:0/3251224801 >> v2:172.21.15.45:3300/0 conn(0x7f735006cc40 0x7f735006d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:04.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:04.581+0000 7f7355ce7640 1 -- 172.21.15.27:0/3251224801 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73500d1260 con 0x7f735015e650 2024-09-12T03:22:04.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:04.581+0000 7f73564e8640 1 --2- 172.21.15.27:0/3251224801 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f735006db30 0x7f735006df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:22:04.584 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:04.581+0000 7f7355ce7640 1 --2- 172.21.15.27:0/3251224801 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f735015e650 0x7f73500d1d40 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f73500b21c0 tx=0x7f734802a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d662bbe27eb2e51 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:04.584 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:04.581+0000 7f7354ce5640 1 -- 172.21.15.27:0/3251224801 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7348037070 con 0x7f735015e650 2024-09-12T03:22:04.584 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:04.582+0000 7f7354ce5640 1 -- 172.21.15.27:0/3251224801 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7348032440 con 0x7f735015e650 2024-09-12T03:22:04.584 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:04.582+0000 7f7354ce5640 1 -- 172.21.15.27:0/3251224801 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7348032810 con 0x7f735015e650 2024-09-12T03:22:04.584 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:04.582+0000 7f7357f4b640 1 -- 172.21.15.27:0/3251224801 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f735015e650 msgr2=0x7f73500d1d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:04.585 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:04.582+0000 7f7357f4b640 1 --2- 172.21.15.27:0/3251224801 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f735015e650 0x7f73500d1d40 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f73500b21c0 tx=0x7f734802a140 comp rx=0 tx=0).stop 2024-09-12T03:22:04.585 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:04.582+0000 7f7357f4b640 1 -- 172.21.15.27:0/3251224801 shutdown_connections 2024-09-12T03:22:04.585 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:04.582+0000 7f7357f4b640 1 --2- 172.21.15.27:0/3251224801 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f735006db30 0x7f735006df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:04.585 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:04.582+0000 7f7357f4b640 1 --2- 172.21.15.27:0/3251224801 >> v2:172.21.15.45:3300/0 conn(0x7f735006cc40 0x7f735006d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:04.585 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:04.582+0000 7f7357f4b640 1 --2- 172.21.15.27:0/3251224801 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f735015e650 0x7f73500d1d40 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:04.585 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:04.582+0000 7f7357f4b640 1 -- 172.21.15.27:0/3251224801 >> 172.21.15.27:0/3251224801 conn(0x7f735006f320 msgr2=0x7f735006f700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:04.585 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:04.582+0000 7f7357f4b640 1 -- 172.21.15.27:0/3251224801 shutdown_connections 2024-09-12T03:22:04.585 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:04.582+0000 7f7357f4b640 1 -- 172.21.15.27:0/3251224801 wait complete. 2024-09-12T03:22:04.585 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:04.583+0000 7f7357f4b640 1 Processor -- start 2024-09-12T03:22:04.585 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:04.583+0000 7f7357f4b640 1 -- start start 2024-09-12T03:22:04.587 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:04.584+0000 7f7357f4b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f735006cc40 0x7f7350158100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:04.587 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:04.584+0000 7f7357f4b640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f735006db30 0x7f7350156fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:04.587 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:04.584+0000 7f7357f4b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f735015e650 0x7f73500b3760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:04.587 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:04.584+0000 7f7357f4b640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f735015ebb0 con 0x7f735015e650 2024-09-12T03:22:04.587 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:04.584+0000 7f7357f4b640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f735015ea30 con 0x7f735006cc40 2024-09-12T03:22:04.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:04.584+0000 7f7357f4b640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f735006e470 con 0x7f735006db30 2024-09-12T03:22:04.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:04.584+0000 7f73564e8640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f735015e650 0x7f73500b3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:04.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:04.584+0000 7f73564e8640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f735015e650 0x7f73500b3760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42514/0 (socket says 172.21.15.27:42514) 2024-09-12T03:22:04.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:04.584+0000 7f7355ce7640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f735006cc40 0x7f7350158100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:04.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:04.584+0000 7f73564e8640 1 -- 172.21.15.27:0/45680888 learned_addr learned my addr 172.21.15.27:0/45680888 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:22:04.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:04.584+0000 7f73554e6640 1 --2- 172.21.15.27:0/45680888 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f735006db30 0x7f7350156fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:04.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:04.585+0000 7f73564e8640 1 -- 172.21.15.27:0/45680888 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f735006db30 msgr2=0x7f7350156fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:04.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:04.585+0000 7f73564e8640 1 --2- 172.21.15.27:0/45680888 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f735006db30 0x7f7350156fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:04.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:04.585+0000 7f73564e8640 1 -- 172.21.15.27:0/45680888 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f735006cc40 msgr2=0x7f7350158100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:04.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:04.585+0000 7f73564e8640 1 --2- 172.21.15.27:0/45680888 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f735006cc40 0x7f7350158100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:04.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:04.585+0000 7f73564e8640 1 -- 172.21.15.27:0/45680888 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73500ce010 con 0x7f735015e650 2024-09-12T03:22:04.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:04.585+0000 7f73564e8640 1 --2- 172.21.15.27:0/45680888 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f735015e650 0x7f73500b3760 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f734000cca0 tx=0x7f734000a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:04.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:04.585+0000 7f734e7fc640 1 -- 172.21.15.27:0/45680888 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f734000f680 con 0x7f735015e650 2024-09-12T03:22:04.589 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:04.585+0000 7f734e7fc640 1 -- 172.21.15.27:0/45680888 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f734000f840 con 0x7f735015e650 2024-09-12T03:22:04.589 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:04.585+0000 7f7357f4b640 1 -- 172.21.15.27:0/45680888 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73500ce2f0 con 0x7f735015e650 2024-09-12T03:22:04.589 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:04.585+0000 7f734e7fc640 1 -- 172.21.15.27:0/45680888 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f734000fc30 con 0x7f735015e650 2024-09-12T03:22:04.589 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:04.585+0000 7f7357f4b640 1 -- 172.21.15.27:0/45680888 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73500cc6d0 con 0x7f735015e650 2024-09-12T03:22:04.589 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:04.586+0000 7f7357f4b640 1 -- 172.21.15.27:0/45680888 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73500d36b0 con 0x7f735015e650 2024-09-12T03:22:04.589 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:04.586+0000 7f734e7fc640 1 -- 172.21.15.27:0/45680888 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 21) ==== 91008+0+0 (secure 0 0 0) 0x7f73400213f0 con 0x7f735015e650 2024-09-12T03:22:04.590 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:04.587+0000 7f734e7fc640 1 --2- 172.21.15.27:0/45680888 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f733c06c3d0 0x7f733c06e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:04.590 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:04.587+0000 7f7355ce7640 1 --2- 172.21.15.27:0/45680888 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f733c06c3d0 0x7f733c06e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:04.590 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:04.587+0000 7f734e7fc640 1 -- 172.21.15.27:0/45680888 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5737+0+0 (secure 0 0 0) 0x7f734008c5c0 con 0x7f735015e650 2024-09-12T03:22:04.590 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:04.588+0000 7f7355ce7640 1 --2- 172.21.15.27:0/45680888 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f733c06c3d0 0x7f733c06e890 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f7348034000 tx=0x7f734802e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:04.592 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:04.591+0000 7f734e7fc640 1 -- 172.21.15.27:0/45680888 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f73500d36b0 con 0x7f735015e650 2024-09-12T03:22:04.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:04.795+0000 7f7357f4b640 1 -- 172.21.15.27:0/45680888 --> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] -- mgr_command(tid 0: {"prefix": "fs subvolume ls", "vol_name": "cephfs", "target": ["mon-mgr", ""]}) -- 0x7f735015c6f0 con 0x7f733c06c3d0 2024-09-12T03:22:04.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:04.798+0000 7f734e7fc640 1 -- 172.21.15.27:0/45680888 <== mgr.4113 v2:172.21.15.27:6836/1226920526 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3 (secure 0 0 0) 0x7f735015c6f0 con 0x7f733c06c3d0 2024-09-12T03:22:04.799 INFO:teuthology.orchestra.run.smithi027.stdout:[] 2024-09-12T03:22:04.801 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:04.800+0000 7f7357f4b640 1 -- 172.21.15.27:0/45680888 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f733c06c3d0 msgr2=0x7f733c06e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:04.801 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:04.800+0000 7f7357f4b640 1 --2- 172.21.15.27:0/45680888 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f733c06c3d0 0x7f733c06e890 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f7348034000 tx=0x7f734802e000 comp rx=0 tx=0).stop 2024-09-12T03:22:04.801 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:04.800+0000 7f7357f4b640 1 -- 172.21.15.27:0/45680888 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f735015e650 msgr2=0x7f73500b3760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:04.801 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:04.800+0000 7f7357f4b640 1 --2- 172.21.15.27:0/45680888 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f735015e650 0x7f73500b3760 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f734000cca0 tx=0x7f734000a650 comp rx=0 tx=0).stop 2024-09-12T03:22:04.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:04.800+0000 7f7357f4b640 1 -- 172.21.15.27:0/45680888 shutdown_connections 2024-09-12T03:22:04.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:04.800+0000 7f7357f4b640 1 --2- 172.21.15.27:0/45680888 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f733c06c3d0 0x7f733c06e890 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:04.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:04.800+0000 7f7357f4b640 1 --2- 172.21.15.27:0/45680888 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f735015e650 0x7f73500b3760 unknown :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:04.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:04.800+0000 7f7357f4b640 1 --2- 172.21.15.27:0/45680888 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f735006db30 0x7f7350156fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:04.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:04.800+0000 7f7357f4b640 1 --2- 172.21.15.27:0/45680888 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f735006cc40 0x7f7350158100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:04.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:04.800+0000 7f7357f4b640 1 -- 172.21.15.27:0/45680888 >> 172.21.15.27:0/45680888 conn(0x7f735006f320 msgr2=0x7f735015b680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:04.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:04.800+0000 7f7357f4b640 1 -- 172.21.15.27:0/45680888 shutdown_connections 2024-09-12T03:22:04.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:04.800+0000 7f7357f4b640 1 -- 172.21.15.27:0/45680888 wait complete. 2024-09-12T03:22:04.811 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-09-12T03:22:04.862 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:04 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-12T03:22:04.864 INFO:teuthology.orchestra.run.smithi027.stdout:2 2024-09-12T03:22:04.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T03:22:04.870 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-12T03:22:04.870 DEBUG:teuthology.orchestra.run.smithi027:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-12T03:22:04.908 DEBUG:teuthology.orchestra.run.smithi045:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-12T03:22:04.957 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-12T03:22:04.957 INFO:tasks.cephfs.mount:Terminating background process 2024-09-12T03:22:04.957 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-12T03:22:04.957 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T03:22:04.974 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-12T03:22:04.974 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T03:22:05.030 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-12T03:22:05.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:05.172+0000 7fad7a21e640 1 Processor -- start 2024-09-12T03:22:05.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:05.172+0000 7fad7a21e640 1 -- start start 2024-09-12T03:22:05.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:05.173+0000 7fad7a21e640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fad7415b2b0 0x7fad740b3560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:05.175 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:05.173+0000 7fad7a21e640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fad74072d70 0x7fad74072430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:05.175 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:05.173+0000 7fad7a21e640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fad740ce410 con 0x7fad7406e9b0 2024-09-12T03:22:05.175 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:05.173+0000 7fad7a21e640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7fad740720a0 con 0x7fad74072d70 2024-09-12T03:22:05.175 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:05.173+0000 7fad7a21e640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fad74072970 con 0x7fad7415b2b0 2024-09-12T03:22:05.175 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:05.173+0000 7fad72ffd640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fad74072d70 0x7fad74072430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:05.175 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:05.173+0000 7fad73fff640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fad7406e9b0 0x7fad740cebb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:51248/0 (socket says 172.21.15.27:51248) 2024-09-12T03:22:05.175 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:05.173+0000 7fad72ffd640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fad74072d70 0x7fad74072430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42520/0 (socket says 172.21.15.27:42520) 2024-09-12T03:22:05.175 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:05.173+0000 7fad73fff640 1 -- 172.21.15.27:0/1909231544 learned_addr learned my addr 172.21.15.27:0/1909231544 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:22:05.175 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:05.174+0000 7fad737fe640 1 --2- 172.21.15.27:0/1909231544 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fad7415b2b0 0x7fad740b3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:05.175 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:05.174+0000 7fad72ffd640 1 -- 172.21.15.27:0/1909231544 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fad7415b2b0 msgr2=0x7fad740b3560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:05.175 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:05.174+0000 7fad72ffd640 1 --2- 172.21.15.27:0/1909231544 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fad7415b2b0 0x7fad740b3560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:05.175 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:05.174+0000 7fad72ffd640 1 -- 172.21.15.27:0/1909231544 >> v1:172.21.15.27:6789/0 conn(0x7fad7406e9b0 legacy=0x7fad740cebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:05.176 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:05.174+0000 7fad72ffd640 1 -- 172.21.15.27:0/1909231544 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad740732c0 con 0x7fad74072d70 2024-09-12T03:22:05.176 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:05.174+0000 7fad727fc640 1 -- 172.21.15.27:0/1909231544 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2331966471 0 0) 0x7fad6c001490 con 0x7fad7406e9b0 2024-09-12T03:22:05.177 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:05.174+0000 7fad72ffd640 1 --2- 172.21.15.27:0/1909231544 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fad74072d70 0x7fad74072430 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7fad680097a0 tx=0x7fad6802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7292e58149e959b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:05.177 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:05.175+0000 7fad727fc640 1 -- 172.21.15.27:0/1909231544 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad68037070 con 0x7fad74072d70 2024-09-12T03:22:05.177 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:05.175+0000 7fad727fc640 1 -- 172.21.15.27:0/1909231544 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad68032440 con 0x7fad74072d70 2024-09-12T03:22:05.177 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:05.175+0000 7fad727fc640 1 -- 172.21.15.27:0/1909231544 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad68032810 con 0x7fad74072d70 2024-09-12T03:22:05.177 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:05.175+0000 7fad7a21e640 1 -- 172.21.15.27:0/1909231544 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fad74072d70 msgr2=0x7fad74072430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:05.177 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:05.175+0000 7fad7a21e640 1 --2- 172.21.15.27:0/1909231544 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fad74072d70 0x7fad74072430 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7fad680097a0 tx=0x7fad6802a180 comp rx=0 tx=0).stop 2024-09-12T03:22:05.177 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:05.175+0000 7fad7a21e640 1 -- 172.21.15.27:0/1909231544 shutdown_connections 2024-09-12T03:22:05.177 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:05.175+0000 7fad7a21e640 1 --2- 172.21.15.27:0/1909231544 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fad74072d70 0x7fad74072430 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:05.177 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:05.175+0000 7fad7a21e640 1 --2- 172.21.15.27:0/1909231544 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fad7415b2b0 0x7fad740b3560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:05.177 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:05.175+0000 7fad7a21e640 1 -- 172.21.15.27:0/1909231544 >> 172.21.15.27:0/1909231544 conn(0x7fad74062c60 msgr2=0x7fad7406bb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:05.178 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:05.175+0000 7fad7a21e640 1 -- 172.21.15.27:0/1909231544 shutdown_connections 2024-09-12T03:22:05.178 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:05.175+0000 7fad7a21e640 1 -- 172.21.15.27:0/1909231544 wait complete. 2024-09-12T03:22:05.180 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:05.176+0000 7fad7a21e640 1 Processor -- start 2024-09-12T03:22:05.180 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:05.177+0000 7fad7a21e640 1 -- start start 2024-09-12T03:22:05.180 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:05.177+0000 7fad7a21e640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fad7406e9b0 0x7fad7415d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:05.180 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:05.177+0000 7fad7a21e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fad74072d70 0x7fad7415c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:05.180 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:05.177+0000 7fad7a21e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fad7415b2b0 0x7fad741575a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:05.180 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:05.177+0000 7fad7a21e640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fad740ce690 con 0x7fad7415b2b0 2024-09-12T03:22:05.180 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:05.177+0000 7fad7a21e640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fad74071dc0 con 0x7fad74072d70 2024-09-12T03:22:05.180 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:05.177+0000 7fad7a21e640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fad7415b690 con 0x7fad7406e9b0 2024-09-12T03:22:05.180 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:05.178+0000 7fad73fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fad7415b2b0 0x7fad741575a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:05.180 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:05.178+0000 7fad73fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fad7415b2b0 0x7fad741575a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42528/0 (socket says 172.21.15.27:42528) 2024-09-12T03:22:05.180 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:05.178+0000 7fad73fff640 1 -- 172.21.15.27:0/2320242234 learned_addr learned my addr 172.21.15.27:0/2320242234 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:22:05.180 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:05.178+0000 7fad737fe640 1 --2- 172.21.15.27:0/2320242234 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fad7406e9b0 0x7fad7415d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:05.181 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:05.178+0000 7fad72ffd640 1 --2- 172.21.15.27:0/2320242234 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fad74072d70 0x7fad7415c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:05.181 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:05.178+0000 7fad73fff640 1 -- 172.21.15.27:0/2320242234 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fad7406e9b0 msgr2=0x7fad7415d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:05.181 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:05.178+0000 7fad73fff640 1 --2- 172.21.15.27:0/2320242234 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fad7406e9b0 0x7fad7415d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:05.181 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:05.178+0000 7fad73fff640 1 -- 172.21.15.27:0/2320242234 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fad74072d70 msgr2=0x7fad7415c460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:05.181 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:05.178+0000 7fad73fff640 1 --2- 172.21.15.27:0/2320242234 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fad74072d70 0x7fad7415c460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:05.181 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:05.178+0000 7fad73fff640 1 -- 172.21.15.27:0/2320242234 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad7415c9a0 con 0x7fad7415b2b0 2024-09-12T03:22:05.181 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:05.178+0000 7fad73fff640 1 --2- 172.21.15.27:0/2320242234 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fad7415b2b0 0x7fad741575a0 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7fad6c00c7f0 tx=0x7fad6c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:05.181 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:05.178+0000 7fad5bfff640 1 -- 172.21.15.27:0/2320242234 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad6c018070 con 0x7fad7415b2b0 2024-09-12T03:22:05.181 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:05.178+0000 7fad5bfff640 1 -- 172.21.15.27:0/2320242234 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad6c00a5e0 con 0x7fad7415b2b0 2024-09-12T03:22:05.181 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:05.178+0000 7fad7a21e640 1 -- 172.21.15.27:0/2320242234 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad74157ea0 con 0x7fad7415b2b0 2024-09-12T03:22:05.181 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:05.178+0000 7fad5bfff640 1 -- 172.21.15.27:0/2320242234 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad6c00aa10 con 0x7fad7415b2b0 2024-09-12T03:22:05.181 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:05.178+0000 7fad7a21e640 1 -- 172.21.15.27:0/2320242234 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad7415b980 con 0x7fad7415b2b0 2024-09-12T03:22:05.182 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:05.179+0000 7fad7a21e640 1 -- 172.21.15.27:0/2320242234 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad74072430 con 0x7fad7415b2b0 2024-09-12T03:22:05.182 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:05.180+0000 7fad5bfff640 1 -- 172.21.15.27:0/2320242234 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 21) ==== 91008+0+0 (secure 0 0 0) 0x7fad6c01b070 con 0x7fad7415b2b0 2024-09-12T03:22:05.182 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:05.180+0000 7fad5bfff640 1 --2- 172.21.15.27:0/2320242234 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fad6006c3d0 0x7fad6006e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:05.182 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:05.180+0000 7fad737fe640 1 --2- 172.21.15.27:0/2320242234 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fad6006c3d0 0x7fad6006e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:05.182 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:05.180+0000 7fad5bfff640 1 -- 172.21.15.27:0/2320242234 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5737+0+0 (secure 0 0 0) 0x7fad6c08af10 con 0x7fad7415b2b0 2024-09-12T03:22:05.182 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:05.181+0000 7fad737fe640 1 --2- 172.21.15.27:0/2320242234 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fad6006c3d0 0x7fad6006e890 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fad64006fd0 tx=0x7fad64008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:05.185 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:05.185+0000 7fad5bfff640 1 -- 172.21.15.27:0/2320242234 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fad740ce410 con 0x7fad7415b2b0 2024-09-12T03:22:05.379 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:05.378+0000 7fad7a21e640 1 -- 172.21.15.27:0/2320242234 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fad74075440 con 0x7fad7415b2b0 2024-09-12T03:22:05.379 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:05.378+0000 7fad5bfff640 1 -- 172.21.15.27:0/2320242234 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v114) ==== 80+0+0 (secure 0 0 0) 0x7fad6c059030 con 0x7fad7415b2b0 2024-09-12T03:22:05.379 INFO:teuthology.orchestra.run.smithi027.stderr:listed 0 entries 2024-09-12T03:22:05.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:05.380+0000 7fad7a21e640 1 -- 172.21.15.27:0/2320242234 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fad6006c3d0 msgr2=0x7fad6006e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:05.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:05.380+0000 7fad7a21e640 1 --2- 172.21.15.27:0/2320242234 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fad6006c3d0 0x7fad6006e890 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fad64006fd0 tx=0x7fad64008040 comp rx=0 tx=0).stop 2024-09-12T03:22:05.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:05.380+0000 7fad7a21e640 1 -- 172.21.15.27:0/2320242234 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fad7415b2b0 msgr2=0x7fad741575a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:05.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:05.380+0000 7fad7a21e640 1 --2- 172.21.15.27:0/2320242234 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fad7415b2b0 0x7fad741575a0 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7fad6c00c7f0 tx=0x7fad6c00ccb0 comp rx=0 tx=0).stop 2024-09-12T03:22:05.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:05.380+0000 7fad7a21e640 1 -- 172.21.15.27:0/2320242234 shutdown_connections 2024-09-12T03:22:05.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:05.380+0000 7fad7a21e640 1 --2- 172.21.15.27:0/2320242234 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fad6006c3d0 0x7fad6006e890 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:05.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:05.380+0000 7fad7a21e640 1 --2- 172.21.15.27:0/2320242234 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fad7415b2b0 0x7fad741575a0 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:05.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:05.380+0000 7fad7a21e640 1 --2- 172.21.15.27:0/2320242234 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fad74072d70 0x7fad7415c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:05.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:05.380+0000 7fad7a21e640 1 --2- 172.21.15.27:0/2320242234 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fad7406e9b0 0x7fad7415d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:05.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:05.380+0000 7fad7a21e640 1 -- 172.21.15.27:0/2320242234 >> 172.21.15.27:0/2320242234 conn(0x7fad74062c60 msgr2=0x7fad7415eec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:05.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:05.381+0000 7fad7a21e640 1 -- 172.21.15.27:0/2320242234 shutdown_connections 2024-09-12T03:22:05.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:05.381+0000 7fad7a21e640 1 -- 172.21.15.27:0/2320242234 wait complete. 2024-09-12T03:22:05.392 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-12T03:22:05.392 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:22:05.392 DEBUG:teuthology.orchestra.run.smithi027:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-12T03:22:05.432 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi027.front.sepia.ceph.com 2024-09-12T03:22:05.432 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:22:05.433 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-12T03:22:05.489 INFO:tasks.cephfs.mount:Terminating background process 2024-09-12T03:22:05.489 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-12T03:22:05.490 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T03:22:05.509 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-12T03:22:05.509 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T03:22:05.567 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-12T03:22:05.670 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:05.669+0000 7f4c4e942640 1 Processor -- start 2024-09-12T03:22:05.670 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:05.669+0000 7f4c4e942640 1 -- start start 2024-09-12T03:22:05.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:05.670+0000 7f4c4e942640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4c481590d0 0x7f4c480b55d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:05.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:05.670+0000 7f4c4e942640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f4c4815f600 0x7f4c4806bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:05.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:05.670+0000 7f4c4e942640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4c48162c10 con 0x7f4c48158cf0 2024-09-12T03:22:05.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:05.670+0000 7f4c4e942640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f4c48162ef0 con 0x7f4c4815f600 2024-09-12T03:22:05.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:05.670+0000 7f4c4e942640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f4c481631d0 con 0x7f4c481590d0 2024-09-12T03:22:05.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:05.670+0000 7f4c47fff640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f4c48158cf0 0x7f4c48162430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:51264/0 (socket says 172.21.15.27:51264) 2024-09-12T03:22:05.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:05.670+0000 7f4c47fff640 1 -- 172.21.15.27:0/2942489413 learned_addr learned my addr 172.21.15.27:0/2942489413 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:22:05.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:05.670+0000 7f4c4cedf640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f4c4815f600 0x7f4c4806bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:05.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:05.670+0000 7f4c477fe640 1 --2- 172.21.15.27:0/2942489413 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4c481590d0 0x7f4c480b55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:05.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:05.670+0000 7f4c46ffd640 1 -- 172.21.15.27:0/2942489413 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1536887230 0 0) 0x7f4c48162c10 con 0x7f4c48158cf0 2024-09-12T03:22:05.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:05.670+0000 7f4c46ffd640 1 -- 172.21.15.27:0/2942489413 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4c30003520 con 0x7f4c48158cf0 2024-09-12T03:22:05.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:05.670+0000 7f4c4cedf640 1 -- 172.21.15.27:0/2942489413 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4c481590d0 msgr2=0x7f4c480b55d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:05.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:05.670+0000 7f4c4cedf640 1 --2- 172.21.15.27:0/2942489413 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4c481590d0 0x7f4c480b55d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:05.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:05.670+0000 7f4c4cedf640 1 -- 172.21.15.27:0/2942489413 >> v1:172.21.15.27:6789/0 conn(0x7f4c48158cf0 legacy=0x7f4c48162430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:05.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:05.671+0000 7f4c4cedf640 1 -- 172.21.15.27:0/2942489413 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c4815edc0 con 0x7f4c4815f600 2024-09-12T03:22:05.674 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:05.671+0000 7f4c4cedf640 1 --2- 172.21.15.27:0/2942489413 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4c4815f600 0x7f4c4806bb30 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f4c40009d90 tx=0x7f4c4002a530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8436cb8241fcacbd server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:05.674 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:05.671+0000 7f4c46ffd640 1 -- 172.21.15.27:0/2942489413 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c40007740 con 0x7f4c4815f600 2024-09-12T03:22:05.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:05.671+0000 7f4c46ffd640 1 -- 172.21.15.27:0/2942489413 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c400078e0 con 0x7f4c4815f600 2024-09-12T03:22:05.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:05.671+0000 7f4c46ffd640 1 -- 172.21.15.27:0/2942489413 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c40007cb0 con 0x7f4c4815f600 2024-09-12T03:22:05.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:05.672+0000 7f4c4e942640 1 -- 172.21.15.27:0/2942489413 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4c4815f600 msgr2=0x7f4c4806bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:05.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:05.672+0000 7f4c4e942640 1 --2- 172.21.15.27:0/2942489413 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4c4815f600 0x7f4c4806bb30 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f4c40009d90 tx=0x7f4c4002a530 comp rx=0 tx=0).stop 2024-09-12T03:22:05.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:05.672+0000 7f4c4e942640 1 -- 172.21.15.27:0/2942489413 shutdown_connections 2024-09-12T03:22:05.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:05.672+0000 7f4c4e942640 1 --2- 172.21.15.27:0/2942489413 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4c4815f600 0x7f4c4806bb30 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:05.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:05.672+0000 7f4c4e942640 1 --2- 172.21.15.27:0/2942489413 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4c481590d0 0x7f4c480b55d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:05.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:05.672+0000 7f4c4e942640 1 -- 172.21.15.27:0/2942489413 >> 172.21.15.27:0/2942489413 conn(0x7f4c4806cc40 msgr2=0x7f4c4806d050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:05.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:05.672+0000 7f4c4e942640 1 -- 172.21.15.27:0/2942489413 shutdown_connections 2024-09-12T03:22:05.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:05.672+0000 7f4c4e942640 1 -- 172.21.15.27:0/2942489413 wait complete. 2024-09-12T03:22:05.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:05.673+0000 7f4c4e942640 1 Processor -- start 2024-09-12T03:22:05.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:05.673+0000 7f4c4e942640 1 -- start start 2024-09-12T03:22:05.678 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:05.674+0000 7f4c4e942640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4c48158cf0 0x7f4c4815e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:05.679 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:05.674+0000 7f4c4e942640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4c481590d0 0x7f4c4814c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:05.679 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:05.674+0000 7f4c4e942640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4c4815f600 0x7f4c48161030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:05.679 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:05.674+0000 7f4c4e942640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f4c4806c070 con 0x7f4c48158cf0 2024-09-12T03:22:05.679 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:05.674+0000 7f4c4e942640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f4c4815fc80 con 0x7f4c481590d0 2024-09-12T03:22:05.679 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:05.674+0000 7f4c4e942640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f4c4806c1f0 con 0x7f4c4815f600 2024-09-12T03:22:05.679 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:05.674+0000 7f4c47fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4c48158cf0 0x7f4c4815e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:05.679 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:05.674+0000 7f4c47fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4c48158cf0 0x7f4c4815e260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42556/0 (socket says 172.21.15.27:42556) 2024-09-12T03:22:05.679 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:05.674+0000 7f4c47fff640 1 -- 172.21.15.27:0/4076085727 learned_addr learned my addr 172.21.15.27:0/4076085727 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:22:05.679 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:05.674+0000 7f4c477fe640 1 --2- 172.21.15.27:0/4076085727 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4c481590d0 0x7f4c4814c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:05.679 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:05.674+0000 7f4c4cedf640 1 --2- 172.21.15.27:0/4076085727 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4c4815f600 0x7f4c48161030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:05.679 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:05.674+0000 7f4c47fff640 1 -- 172.21.15.27:0/4076085727 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4c4815f600 msgr2=0x7f4c48161030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:05.679 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:05.674+0000 7f4c47fff640 1 --2- 172.21.15.27:0/4076085727 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4c4815f600 0x7f4c48161030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:05.680 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:05.675+0000 7f4c47fff640 1 -- 172.21.15.27:0/4076085727 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4c481590d0 msgr2=0x7f4c4814c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:05.680 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:05.675+0000 7f4c47fff640 1 --2- 172.21.15.27:0/4076085727 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4c481590d0 0x7f4c4814c5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:05.680 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:05.675+0000 7f4c47fff640 1 -- 172.21.15.27:0/4076085727 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c4815e9b0 con 0x7f4c48158cf0 2024-09-12T03:22:05.680 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:05.675+0000 7f4c477fe640 1 --2- 172.21.15.27:0/4076085727 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4c481590d0 0x7f4c4814c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:22:05.680 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:05.675+0000 7f4c4cedf640 1 --2- 172.21.15.27:0/4076085727 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4c4815f600 0x7f4c48161030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:22:05.680 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:05.675+0000 7f4c47fff640 1 --2- 172.21.15.27:0/4076085727 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4c48158cf0 0x7f4c4815e260 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f4c3800c390 tx=0x7f4c3800c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:05.680 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:05.675+0000 7f4c44ff9640 1 -- 172.21.15.27:0/4076085727 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c38017070 con 0x7f4c48158cf0 2024-09-12T03:22:05.680 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:05.675+0000 7f4c44ff9640 1 -- 172.21.15.27:0/4076085727 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c38005ce0 con 0x7f4c48158cf0 2024-09-12T03:22:05.680 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:05.675+0000 7f4c4e942640 1 -- 172.21.15.27:0/4076085727 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c481e82f0 con 0x7f4c48158cf0 2024-09-12T03:22:05.680 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:05.675+0000 7f4c44ff9640 1 -- 172.21.15.27:0/4076085727 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c380072f0 con 0x7f4c48158cf0 2024-09-12T03:22:05.680 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:05.675+0000 7f4c4e942640 1 -- 172.21.15.27:0/4076085727 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c481e8870 con 0x7f4c48158cf0 2024-09-12T03:22:05.680 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:05.676+0000 7f4c4e942640 1 -- 172.21.15.27:0/4076085727 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c4806bb30 con 0x7f4c48158cf0 2024-09-12T03:22:05.680 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:05.676+0000 7f4c44ff9640 1 -- 172.21.15.27:0/4076085727 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 21) ==== 91008+0+0 (secure 0 0 0) 0x7f4c3801a070 con 0x7f4c48158cf0 2024-09-12T03:22:05.681 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:05.677+0000 7f4c44ff9640 1 --2- 172.21.15.27:0/4076085727 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f4c3406c3d0 0x7f4c3406e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:05.681 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:05.677+0000 7f4c44ff9640 1 -- 172.21.15.27:0/4076085727 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5737+0+0 (secure 0 0 0) 0x7f4c3808ac50 con 0x7f4c48158cf0 2024-09-12T03:22:05.681 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:05.677+0000 7f4c477fe640 1 --2- 172.21.15.27:0/4076085727 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f4c3406c3d0 0x7f4c3406e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:05.681 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:05.677+0000 7f4c477fe640 1 --2- 172.21.15.27:0/4076085727 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f4c3406c3d0 0x7f4c3406e890 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f4c3c006fd0 tx=0x7f4c3c008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:05.682 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:05.681+0000 7f4c44ff9640 1 -- 172.21.15.27:0/4076085727 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f4c38058e00 con 0x7f4c48158cf0 2024-09-12T03:22:05.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:05.867+0000 7f4c4e942640 1 -- 172.21.15.27:0/4076085727 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f4c48157b10 con 0x7f4c48158cf0 2024-09-12T03:22:05.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:05.867+0000 7f4c44ff9640 1 -- 172.21.15.27:0/4076085727 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v114) ==== 80+0+0 (secure 0 0 0) 0x7f4c38058700 con 0x7f4c48158cf0 2024-09-12T03:22:05.868 INFO:teuthology.orchestra.run.smithi027.stderr:listed 0 entries 2024-09-12T03:22:05.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:05.869+0000 7f4c4e942640 1 -- 172.21.15.27:0/4076085727 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f4c3406c3d0 msgr2=0x7f4c3406e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:05.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:05.869+0000 7f4c4e942640 1 --2- 172.21.15.27:0/4076085727 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f4c3406c3d0 0x7f4c3406e890 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f4c3c006fd0 tx=0x7f4c3c008040 comp rx=0 tx=0).stop 2024-09-12T03:22:05.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:05.869+0000 7f4c4e942640 1 -- 172.21.15.27:0/4076085727 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4c48158cf0 msgr2=0x7f4c4815e260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:05.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:05.869+0000 7f4c4e942640 1 --2- 172.21.15.27:0/4076085727 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4c48158cf0 0x7f4c4815e260 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f4c3800c390 tx=0x7f4c3800c850 comp rx=0 tx=0).stop 2024-09-12T03:22:05.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:05.869+0000 7f4c4e942640 1 -- 172.21.15.27:0/4076085727 shutdown_connections 2024-09-12T03:22:05.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:05.869+0000 7f4c4e942640 1 --2- 172.21.15.27:0/4076085727 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4c4815f600 0x7f4c48161030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:05.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:05.869+0000 7f4c4e942640 1 --2- 172.21.15.27:0/4076085727 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4c481590d0 0x7f4c4814c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:05.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:05.869+0000 7f4c4e942640 1 --2- 172.21.15.27:0/4076085727 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f4c3406c3d0 0x7f4c3406e890 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:05.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:05.869+0000 7f4c4e942640 1 --2- 172.21.15.27:0/4076085727 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4c48158cf0 0x7f4c4815e260 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:05.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:05.869+0000 7f4c4e942640 1 -- 172.21.15.27:0/4076085727 >> 172.21.15.27:0/4076085727 conn(0x7f4c4806cc40 msgr2=0x7f4c480af8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:05.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:05.869+0000 7f4c4e942640 1 -- 172.21.15.27:0/4076085727 shutdown_connections 2024-09-12T03:22:05.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:05.869+0000 7f4c4e942640 1 -- 172.21.15.27:0/4076085727 wait complete. 2024-09-12T03:22:05.881 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-09-12T03:22:05.881 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:22:05.881 DEBUG:teuthology.orchestra.run.smithi045:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-09-12T03:22:05.917 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi045.front.sepia.ceph.com 2024-09-12T03:22:05.917 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:22:05.917 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-12T03:22:05.976 INFO:tasks.cephfs.mount:Terminating background process 2024-09-12T03:22:05.976 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-12T03:22:05.976 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T03:22:05.993 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056004,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497432k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16324 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248716k,nr_inodes=812179,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497432k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-12T03:22:05.993 INFO:tasks.cephfs.mount:Terminating background process 2024-09-12T03:22:05.993 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-12T03:22:05.993 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T03:22:06.034 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056795,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16602 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249348k,nr_inodes=812337,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-12T03:22:06.035 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-12T03:22:06.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.137+0000 7fac292e4640 1 Processor -- start 2024-09-12T03:22:06.138 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.137+0000 7fac292e4640 1 -- start start 2024-09-12T03:22:06.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.137+0000 7fac292e4640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fac2415b240 0x7fac240d0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:06.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.137+0000 7fac292e4640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fac240b45f0 0x7fac2406f450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:06.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.137+0000 7fac292e4640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fac240ced50 con 0x7fac2415eeb0 2024-09-12T03:22:06.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.137+0000 7fac292e4640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7fac2406f000 con 0x7fac240b45f0 2024-09-12T03:22:06.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.137+0000 7fac292e4640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fac240ceff0 con 0x7fac2415b240 2024-09-12T03:22:06.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.138+0000 7fac237fe640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fac240b45f0 0x7fac2406f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:06.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.138+0000 7fac227fc640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fac2415eeb0 0x7fac240b3ee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:51266/0 (socket says 172.21.15.27:51266) 2024-09-12T03:22:06.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.138+0000 7fac227fc640 1 -- 172.21.15.27:0/2812839303 learned_addr learned my addr 172.21.15.27:0/2812839303 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:22:06.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.138+0000 7fac237fe640 1 -- 172.21.15.27:0/2812839303 >> v2:172.21.15.45:3300/0 conn(0x7fac2415b240 msgr2=0x7fac240d0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:06.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.138+0000 7fac22ffd640 1 --2- 172.21.15.27:0/2812839303 >> v2:172.21.15.45:3300/0 conn(0x7fac2415b240 0x7fac240d0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:06.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.138+0000 7fac237fe640 1 --2- 172.21.15.27:0/2812839303 >> v2:172.21.15.45:3300/0 conn(0x7fac2415b240 0x7fac240d0090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:06.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.138+0000 7fac237fe640 1 -- 172.21.15.27:0/2812839303 >> v1:172.21.15.27:6789/0 conn(0x7fac2415eeb0 legacy=0x7fac240b3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:06.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.138+0000 7fac237fe640 1 -- 172.21.15.27:0/2812839303 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac240cf2d0 con 0x7fac240b45f0 2024-09-12T03:22:06.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.138+0000 7fac21ffb640 1 -- 172.21.15.27:0/2812839303 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1649372314 0 0) 0x7fac240ced50 con 0x7fac2415eeb0 2024-09-12T03:22:06.142 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.139+0000 7fac237fe640 1 --2- 172.21.15.27:0/2812839303 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fac240b45f0 0x7fac2406f450 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7fac2406f1e0 tx=0x7fac1402a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d196358672f6dc5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:06.142 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.139+0000 7fac21ffb640 1 -- 172.21.15.27:0/2812839303 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac14037070 con 0x7fac240b45f0 2024-09-12T03:22:06.142 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.139+0000 7fac21ffb640 1 -- 172.21.15.27:0/2812839303 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac14032440 con 0x7fac240b45f0 2024-09-12T03:22:06.142 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.139+0000 7fac21ffb640 1 -- 172.21.15.27:0/2812839303 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac14032810 con 0x7fac240b45f0 2024-09-12T03:22:06.142 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.139+0000 7fac292e4640 1 -- 172.21.15.27:0/2812839303 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fac240b45f0 msgr2=0x7fac2406f450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:06.142 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.139+0000 7fac292e4640 1 --2- 172.21.15.27:0/2812839303 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fac240b45f0 0x7fac2406f450 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7fac2406f1e0 tx=0x7fac1402a1c0 comp rx=0 tx=0).stop 2024-09-12T03:22:06.142 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.139+0000 7fac292e4640 1 -- 172.21.15.27:0/2812839303 shutdown_connections 2024-09-12T03:22:06.142 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.139+0000 7fac292e4640 1 --2- 172.21.15.27:0/2812839303 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fac240b45f0 0x7fac2406f450 unknown :-1 s=CLOSED pgs=1159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:06.143 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.139+0000 7fac292e4640 1 --2- 172.21.15.27:0/2812839303 >> v2:172.21.15.45:3300/0 conn(0x7fac2415b240 0x7fac240d0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:06.143 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.139+0000 7fac292e4640 1 -- 172.21.15.27:0/2812839303 >> 172.21.15.27:0/2812839303 conn(0x7fac2406ca30 msgr2=0x7fac2406ce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:06.143 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.139+0000 7fac292e4640 1 -- 172.21.15.27:0/2812839303 shutdown_connections 2024-09-12T03:22:06.143 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.139+0000 7fac292e4640 1 -- 172.21.15.27:0/2812839303 wait complete. 2024-09-12T03:22:06.143 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.141+0000 7fac292e4640 1 Processor -- start 2024-09-12T03:22:06.143 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.141+0000 7fac292e4640 1 -- start start 2024-09-12T03:22:06.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.141+0000 7fac292e4640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fac240b45f0 0x7fac24144750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:06.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.141+0000 7fac292e4640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fac2415b240 0x7fac24157ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:06.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.141+0000 7fac292e4640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fac2415eeb0 0x7fac240b17d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:06.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.141+0000 7fac292e4640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fac240cfb70 con 0x7fac240b45f0 2024-09-12T03:22:06.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.141+0000 7fac292e4640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fac240cf890 con 0x7fac2415eeb0 2024-09-12T03:22:06.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.141+0000 7fac292e4640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fac2415b620 con 0x7fac2415b240 2024-09-12T03:22:06.147 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.142+0000 7fac22ffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fac240b45f0 0x7fac24144750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:06.147 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.142+0000 7fac22ffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fac240b45f0 0x7fac24144750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42570/0 (socket says 172.21.15.27:42570) 2024-09-12T03:22:06.147 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.142+0000 7fac22ffd640 1 -- 172.21.15.27:0/3825366570 learned_addr learned my addr 172.21.15.27:0/3825366570 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:22:06.147 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.142+0000 7fac237fe640 1 --2- 172.21.15.27:0/3825366570 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fac2415eeb0 0x7fac240b17d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:06.147 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.142+0000 7fac22ffd640 1 -- 172.21.15.27:0/3825366570 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fac2415b240 msgr2=0x7fac24157ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:06.147 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.142+0000 7fac227fc640 1 --2- 172.21.15.27:0/3825366570 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fac2415b240 0x7fac24157ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:06.147 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.142+0000 7fac22ffd640 1 --2- 172.21.15.27:0/3825366570 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fac2415b240 0x7fac24157ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:06.148 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.142+0000 7fac22ffd640 1 -- 172.21.15.27:0/3825366570 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fac2415eeb0 msgr2=0x7fac240b17d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:06.148 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.142+0000 7fac22ffd640 1 --2- 172.21.15.27:0/3825366570 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fac2415eeb0 0x7fac240b17d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:06.148 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.142+0000 7fac22ffd640 1 -- 172.21.15.27:0/3825366570 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac24144c90 con 0x7fac240b45f0 2024-09-12T03:22:06.148 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.142+0000 7fac237fe640 1 --2- 172.21.15.27:0/3825366570 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fac2415eeb0 0x7fac240b17d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:22:06.148 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.142+0000 7fac227fc640 1 --2- 172.21.15.27:0/3825366570 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fac2415b240 0x7fac24157ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:22:06.148 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.142+0000 7fac22ffd640 1 --2- 172.21.15.27:0/3825366570 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fac240b45f0 0x7fac24144750 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7fac1c00ce10 tx=0x7fac1c00a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:06.148 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.142+0000 7fac1b7fe640 1 -- 172.21.15.27:0/3825366570 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac1c003b90 con 0x7fac240b45f0 2024-09-12T03:22:06.148 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.142+0000 7fac1b7fe640 1 -- 172.21.15.27:0/3825366570 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac1c003d30 con 0x7fac240b45f0 2024-09-12T03:22:06.149 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.142+0000 7fac292e4640 1 -- 172.21.15.27:0/3825366570 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac24150660 con 0x7fac240b45f0 2024-09-12T03:22:06.149 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.142+0000 7fac1b7fe640 1 -- 172.21.15.27:0/3825366570 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac1c004240 con 0x7fac240b45f0 2024-09-12T03:22:06.149 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.142+0000 7fac292e4640 1 -- 172.21.15.27:0/3825366570 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac24150c20 con 0x7fac240b45f0 2024-09-12T03:22:06.149 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.143+0000 7fac292e4640 1 -- 172.21.15.27:0/3825366570 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac240b4a50 con 0x7fac240b45f0 2024-09-12T03:22:06.149 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.144+0000 7fac1b7fe640 1 -- 172.21.15.27:0/3825366570 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 21) ==== 91008+0+0 (secure 0 0 0) 0x7fac1c0043e0 con 0x7fac240b45f0 2024-09-12T03:22:06.149 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.144+0000 7fac1b7fe640 1 --2- 172.21.15.27:0/3825366570 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fac0406c3d0 0x7fac0406e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:06.150 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.144+0000 7fac1b7fe640 1 -- 172.21.15.27:0/3825366570 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5737+0+0 (secure 0 0 0) 0x7fac1c08c4d0 con 0x7fac240b45f0 2024-09-12T03:22:06.150 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.144+0000 7fac227fc640 1 --2- 172.21.15.27:0/3825366570 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fac0406c3d0 0x7fac0406e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:06.150 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.145+0000 7fac227fc640 1 --2- 172.21.15.27:0/3825366570 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fac0406c3d0 0x7fac0406e890 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fac2415f600 tx=0x7fac0c009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:06.150 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.149+0000 7fac1b7fe640 1 -- 172.21.15.27:0/3825366570 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fac1c01d340 con 0x7fac240b45f0 2024-09-12T03:22:06.364 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.363+0000 7fac292e4640 1 -- 172.21.15.27:0/3825366570 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fac2415ec20 con 0x7fac240b45f0 2024-09-12T03:22:06.364 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.363+0000 7fac1b7fe640 1 -- 172.21.15.27:0/3825366570 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) ==== 94+0+5225 (secure 0 0 0) 0x7fac1c05a4e0 con 0x7fac240b45f0 2024-09-12T03:22:06.365 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:22:06.365 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":68,"btime":"2024-09-12T03:21:59:298296+0000","default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6206,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/2249506350","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2249506350},{"type":"v1","addr":"172.21.15.45:6834","nonce":2249506350}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63},{"gid":6209,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/2292759672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":2292759672},{"type":"v1","addr":"172.21.15.45:6835","nonce":2292759672}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63},{"gid":7299,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6834/4021825986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":4021825986},{"type":"v1","addr":"172.21.15.27:6834","nonce":4021825986}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":67,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-12T03:21:46.603883+0000","modified":"2024-09-12T03:21:48.956285+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7302},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7302":{"gid":7302,"name":"a","rank":0,"incarnation":64,"state":"up:active","state_seq":3,"addr":"172.21.15.27:6835/1760654641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":1760654641},{"type":"v1","addr":"172.21.15.27:6835","nonce":1760654641}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7302,"qdb_cluster":[7302]},"id":9}]} 2024-09-12T03:22:06.365 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 68 2024-09-12T03:22:06.367 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.366+0000 7fac292e4640 1 -- 172.21.15.27:0/3825366570 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fac0406c3d0 msgr2=0x7fac0406e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:06.367 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.366+0000 7fac292e4640 1 --2- 172.21.15.27:0/3825366570 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fac0406c3d0 0x7fac0406e890 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fac2415f600 tx=0x7fac0c009040 comp rx=0 tx=0).stop 2024-09-12T03:22:06.367 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.366+0000 7fac292e4640 1 -- 172.21.15.27:0/3825366570 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fac240b45f0 msgr2=0x7fac24144750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:06.367 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.366+0000 7fac292e4640 1 --2- 172.21.15.27:0/3825366570 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fac240b45f0 0x7fac24144750 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7fac1c00ce10 tx=0x7fac1c00a660 comp rx=0 tx=0).stop 2024-09-12T03:22:06.368 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.366+0000 7fac292e4640 1 -- 172.21.15.27:0/3825366570 shutdown_connections 2024-09-12T03:22:06.368 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.366+0000 7fac292e4640 1 --2- 172.21.15.27:0/3825366570 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fac2415eeb0 0x7fac240b17d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:06.368 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.366+0000 7fac292e4640 1 --2- 172.21.15.27:0/3825366570 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fac2415b240 0x7fac24157ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:06.368 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.366+0000 7fac292e4640 1 --2- 172.21.15.27:0/3825366570 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fac0406c3d0 0x7fac0406e890 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:06.368 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.366+0000 7fac292e4640 1 --2- 172.21.15.27:0/3825366570 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fac240b45f0 0x7fac24144750 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:06.368 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.366+0000 7fac292e4640 1 -- 172.21.15.27:0/3825366570 >> 172.21.15.27:0/3825366570 conn(0x7fac2406ca30 msgr2=0x7fac240b0b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:06.368 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.366+0000 7fac292e4640 1 -- 172.21.15.27:0/3825366570 shutdown_connections 2024-09-12T03:22:06.368 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.366+0000 7fac292e4640 1 -- 172.21.15.27:0/3825366570 wait complete. 2024-09-12T03:22:06.378 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-12T03:22:06.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.480+0000 7f5dfdaa6640 1 Processor -- start 2024-09-12T03:22:06.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.480+0000 7f5dfdaa6640 1 -- start start 2024-09-12T03:22:06.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.481+0000 7f5dfdaa6640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f5df80cf660 0x7f5df80d2780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:06.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.481+0000 7f5dfdaa6640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5df806cc40 con 0x7f5df806f360 2024-09-12T03:22:06.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.481+0000 7f5dfdaa6640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5df80d1ce0 con 0x7f5df80b1ca0 2024-09-12T03:22:06.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.481+0000 7f5dfdaa6640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f5df80d22f0 con 0x7f5df80cf660 2024-09-12T03:22:06.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.481+0000 7f5df6ffd640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f5df806f360 0x7f5df80cef50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:51276/0 (socket says 172.21.15.27:51276) 2024-09-12T03:22:06.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.481+0000 7f5df6ffd640 1 -- 172.21.15.27:0/163811878 learned_addr learned my addr 172.21.15.27:0/163811878 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:22:06.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.481+0000 7f5df7fff640 1 --2- 172.21.15.27:0/163811878 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f5df80cf660 0x7f5df80d2780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:06.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.481+0000 7f5df67fc640 1 -- 172.21.15.27:0/163811878 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1439740618 0 0) 0x7f5df806cc40 con 0x7f5df806f360 2024-09-12T03:22:06.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.481+0000 7f5df67fc640 1 -- 172.21.15.27:0/163811878 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5de0003520 con 0x7f5df806f360 2024-09-12T03:22:06.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.481+0000 7f5df67fc640 1 -- 172.21.15.27:0/163811878 <== mon.1 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 159263064 0 0) 0x7f5df80d1ce0 con 0x7f5df80b1ca0 2024-09-12T03:22:06.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.481+0000 7f5df67fc640 1 -- 172.21.15.27:0/163811878 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5df806cc40 con 0x7f5df80b1ca0 2024-09-12T03:22:06.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.481+0000 7f5df67fc640 1 -- 172.21.15.27:0/163811878 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 119275789 0 0) 0x7f5de0003520 con 0x7f5df806f360 2024-09-12T03:22:06.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.481+0000 7f5df7fff640 1 -- 172.21.15.27:0/163811878 >> v1:172.21.15.45:6789/0 conn(0x7f5df80b1ca0 legacy=0x7f5df815f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:06.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.481+0000 7f5df7fff640 1 -- 172.21.15.27:0/163811878 >> v1:172.21.15.27:6789/0 conn(0x7f5df806f360 legacy=0x7f5df80cef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:06.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.482+0000 7f5df7fff640 1 -- 172.21.15.27:0/163811878 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5df806cee0 con 0x7f5df80cf660 2024-09-12T03:22:06.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.482+0000 7f5df7fff640 1 --2- 172.21.15.27:0/163811878 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f5df80cf660 0x7f5df80d2780 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f5df0009dc0 tx=0x7f5df002a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2809792bebbf3716 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:06.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.482+0000 7f5df67fc640 1 -- 172.21.15.27:0/163811878 <== mon.2 v2:172.21.15.45:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dec001670 con 0x7f5df80cf660 2024-09-12T03:22:06.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.483+0000 7f5df67fc640 1 -- 172.21.15.27:0/163811878 <== mon.2 v2:172.21.15.45:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5df0007540 con 0x7f5df80cf660 2024-09-12T03:22:06.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.483+0000 7f5df67fc640 1 -- 172.21.15.27:0/163811878 <== mon.2 v2:172.21.15.45:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5df00078f0 con 0x7f5df80cf660 2024-09-12T03:22:06.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.483+0000 7f5dfdaa6640 1 -- 172.21.15.27:0/163811878 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f5df80cf660 msgr2=0x7f5df80d2780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:06.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.483+0000 7f5dfdaa6640 1 --2- 172.21.15.27:0/163811878 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f5df80cf660 0x7f5df80d2780 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f5df0009dc0 tx=0x7f5df002a660 comp rx=0 tx=0).stop 2024-09-12T03:22:06.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.483+0000 7f5dfdaa6640 1 -- 172.21.15.27:0/163811878 shutdown_connections 2024-09-12T03:22:06.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.483+0000 7f5dfdaa6640 1 --2- 172.21.15.27:0/163811878 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f5df80cf660 0x7f5df80d2780 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:06.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.483+0000 7f5dfdaa6640 1 -- 172.21.15.27:0/163811878 >> 172.21.15.27:0/163811878 conn(0x7f5df8073800 msgr2=0x7f5df809ba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:06.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.483+0000 7f5dfdaa6640 1 -- 172.21.15.27:0/163811878 shutdown_connections 2024-09-12T03:22:06.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.483+0000 7f5dfdaa6640 1 -- 172.21.15.27:0/163811878 wait complete. 2024-09-12T03:22:06.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.484+0000 7f5dfdaa6640 1 Processor -- start 2024-09-12T03:22:06.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.484+0000 7f5dfdaa6640 1 -- start start 2024-09-12T03:22:06.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.485+0000 7f5dfdaa6640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f5df806f360 0x7f5df81530f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:06.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.485+0000 7f5dfdaa6640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5df80b1ca0 0x7f5df80b40f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:06.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.485+0000 7f5dfdaa6640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5df80cf660 0x7f5df80b4630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:06.490 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.485+0000 7f5dfdaa6640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f5df806d340 con 0x7f5df80b1ca0 2024-09-12T03:22:06.490 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.485+0000 7f5dfdaa6640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f5df806d1c0 con 0x7f5df80cf660 2024-09-12T03:22:06.490 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.485+0000 7f5dfdaa6640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f5df80cfc40 con 0x7f5df806f360 2024-09-12T03:22:06.490 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.485+0000 7f5df6ffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5df80b1ca0 0x7f5df80b40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:06.490 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.485+0000 7f5df6ffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5df80b1ca0 0x7f5df80b40f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42586/0 (socket says 172.21.15.27:42586) 2024-09-12T03:22:06.490 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.485+0000 7f5df6ffd640 1 -- 172.21.15.27:0/118359806 learned_addr learned my addr 172.21.15.27:0/118359806 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:22:06.490 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.485+0000 7f5df77fe640 1 --2- 172.21.15.27:0/118359806 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f5df806f360 0x7f5df81530f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:06.490 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.485+0000 7f5df7fff640 1 --2- 172.21.15.27:0/118359806 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5df80cf660 0x7f5df80b4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:06.490 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.486+0000 7f5df6ffd640 1 -- 172.21.15.27:0/118359806 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f5df806f360 msgr2=0x7f5df81530f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:06.490 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.486+0000 7f5df6ffd640 1 --2- 172.21.15.27:0/118359806 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f5df806f360 0x7f5df81530f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:06.490 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.486+0000 7f5df6ffd640 1 -- 172.21.15.27:0/118359806 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5df80cf660 msgr2=0x7f5df80b4630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:06.490 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.486+0000 7f5df6ffd640 1 --2- 172.21.15.27:0/118359806 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5df80cf660 0x7f5df80b4630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:06.490 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.486+0000 7f5df6ffd640 1 -- 172.21.15.27:0/118359806 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5df80cb850 con 0x7f5df80b1ca0 2024-09-12T03:22:06.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.486+0000 7f5df7fff640 1 --2- 172.21.15.27:0/118359806 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5df80cf660 0x7f5df80b4630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:22:06.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.486+0000 7f5df6ffd640 1 --2- 172.21.15.27:0/118359806 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5df80b1ca0 0x7f5df80b40f0 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f5dec00c5a0 tx=0x7f5dec00ca60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:06.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.486+0000 7f5ddffff640 1 -- 172.21.15.27:0/118359806 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dec017070 con 0x7f5df80b1ca0 2024-09-12T03:22:06.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.486+0000 7f5ddffff640 1 -- 172.21.15.27:0/118359806 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5dec005ce0 con 0x7f5df80b1ca0 2024-09-12T03:22:06.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.486+0000 7f5dfdaa6640 1 -- 172.21.15.27:0/118359806 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5df80ccb90 con 0x7f5df80b1ca0 2024-09-12T03:22:06.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.486+0000 7f5ddffff640 1 -- 172.21.15.27:0/118359806 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dec0072b0 con 0x7f5df80b1ca0 2024-09-12T03:22:06.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.486+0000 7f5dfdaa6640 1 -- 172.21.15.27:0/118359806 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5df80cbdd0 con 0x7f5df80b1ca0 2024-09-12T03:22:06.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.487+0000 7f5dfdaa6640 1 -- 172.21.15.27:0/118359806 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5df806f7d0 con 0x7f5df80b1ca0 2024-09-12T03:22:06.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.488+0000 7f5ddffff640 1 -- 172.21.15.27:0/118359806 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 21) ==== 91008+0+0 (secure 0 0 0) 0x7f5dec01a070 con 0x7f5df80b1ca0 2024-09-12T03:22:06.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.488+0000 7f5ddffff640 1 --2- 172.21.15.27:0/118359806 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f5de406c3d0 0x7f5de406e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:06.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.488+0000 7f5ddffff640 1 -- 172.21.15.27:0/118359806 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5737+0+0 (secure 0 0 0) 0x7f5dec08adb0 con 0x7f5df80b1ca0 2024-09-12T03:22:06.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.488+0000 7f5df77fe640 1 --2- 172.21.15.27:0/118359806 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f5de406c3d0 0x7f5de406e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:06.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.489+0000 7f5df77fe640 1 --2- 172.21.15.27:0/118359806 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f5de406c3d0 0x7f5de406e890 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f5de8000f60 tx=0x7f5de8009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:06.493 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.493+0000 7f5ddffff640 1 -- 172.21.15.27:0/118359806 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f5dec058800 con 0x7f5df80b1ca0 2024-09-12T03:22:06.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.704+0000 7f5dfdaa6640 1 -- 172.21.15.27:0/118359806 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5df806eac0 con 0x7f5df80b1ca0 2024-09-12T03:22:06.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.705+0000 7f5ddffff640 1 -- 172.21.15.27:0/118359806 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) ==== 94+0+5225 (secure 0 0 0) 0x7f5dec05cb00 con 0x7f5df80b1ca0 2024-09-12T03:22:06.706 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:22:06.706 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":68,"btime":"2024-09-12T03:21:59:298296+0000","default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6206,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/2249506350","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2249506350},{"type":"v1","addr":"172.21.15.45:6834","nonce":2249506350}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63},{"gid":6209,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/2292759672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":2292759672},{"type":"v1","addr":"172.21.15.45:6835","nonce":2292759672}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63},{"gid":7299,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6834/4021825986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":4021825986},{"type":"v1","addr":"172.21.15.27:6834","nonce":4021825986}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":67,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-12T03:21:46.603883+0000","modified":"2024-09-12T03:21:48.956285+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7302},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7302":{"gid":7302,"name":"a","rank":0,"incarnation":64,"state":"up:active","state_seq":3,"addr":"172.21.15.27:6835/1760654641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":1760654641},{"type":"v1","addr":"172.21.15.27:6835","nonce":1760654641}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7302,"qdb_cluster":[7302]},"id":9}]} 2024-09-12T03:22:06.706 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 68 2024-09-12T03:22:06.708 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.707+0000 7f5dfdaa6640 1 -- 172.21.15.27:0/118359806 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f5de406c3d0 msgr2=0x7f5de406e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:06.708 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.707+0000 7f5dfdaa6640 1 --2- 172.21.15.27:0/118359806 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f5de406c3d0 0x7f5de406e890 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f5de8000f60 tx=0x7f5de8009040 comp rx=0 tx=0).stop 2024-09-12T03:22:06.708 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.707+0000 7f5dfdaa6640 1 -- 172.21.15.27:0/118359806 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5df80b1ca0 msgr2=0x7f5df80b40f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:06.708 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.707+0000 7f5dfdaa6640 1 --2- 172.21.15.27:0/118359806 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5df80b1ca0 0x7f5df80b40f0 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f5dec00c5a0 tx=0x7f5dec00ca60 comp rx=0 tx=0).stop 2024-09-12T03:22:06.708 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.707+0000 7f5dfdaa6640 1 -- 172.21.15.27:0/118359806 shutdown_connections 2024-09-12T03:22:06.708 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.707+0000 7f5dfdaa6640 1 --2- 172.21.15.27:0/118359806 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5df80cf660 0x7f5df80b4630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:06.708 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.707+0000 7f5dfdaa6640 1 --2- 172.21.15.27:0/118359806 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f5de406c3d0 0x7f5de406e890 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:06.708 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.707+0000 7f5dfdaa6640 1 --2- 172.21.15.27:0/118359806 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5df80b1ca0 0x7f5df80b40f0 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:06.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.707+0000 7f5dfdaa6640 1 --2- 172.21.15.27:0/118359806 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f5df806f360 0x7f5df81530f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:06.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.707+0000 7f5dfdaa6640 1 -- 172.21.15.27:0/118359806 >> 172.21.15.27:0/118359806 conn(0x7f5df8073800 msgr2=0x7f5df80b7450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:06.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.707+0000 7f5dfdaa6640 1 -- 172.21.15.27:0/118359806 shutdown_connections 2024-09-12T03:22:06.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.707+0000 7f5dfdaa6640 1 -- 172.21.15.27:0/118359806 wait complete. 2024-09-12T03:22:06.719 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-12T03:22:06.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.819+0000 7feac8bf6640 1 Processor -- start 2024-09-12T03:22:06.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.819+0000 7feac8bf6640 1 -- start start 2024-09-12T03:22:06.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.820+0000 7feac8bf6640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7feac415eb90 0x7feac40d2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:06.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.820+0000 7feac8bf6640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7feac40d5200 0x7feac40d1010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:06.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.820+0000 7feac8bf6640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feac40d19e0 con 0x7feac40d1550 2024-09-12T03:22:06.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.820+0000 7feac8bf6640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7feac40b7d20 con 0x7feac40d5200 2024-09-12T03:22:06.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.820+0000 7feac8bf6640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7feac40d42e0 con 0x7feac415eb90 2024-09-12T03:22:06.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.820+0000 7feac2d9d640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7feac40d1550 0x7feac40cf200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:51278/0 (socket says 172.21.15.27:51278) 2024-09-12T03:22:06.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.820+0000 7feac2d9d640 1 -- 172.21.15.27:0/3366141373 learned_addr learned my addr 172.21.15.27:0/3366141373 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:22:06.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.821+0000 7feac1d9b640 1 --2- 172.21.15.27:0/3366141373 >> v2:172.21.15.45:3300/0 conn(0x7feac40d5200 0x7feac40d1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:06.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.821+0000 7feac259c640 1 --2- 172.21.15.27:0/3366141373 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7feac415eb90 0x7feac40d2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:06.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.821+0000 7feac159a640 1 -- 172.21.15.27:0/3366141373 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3518809072 0 0) 0x7feac40d19e0 con 0x7feac40d1550 2024-09-12T03:22:06.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.821+0000 7feac159a640 1 -- 172.21.15.27:0/3366141373 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7feab0003520 con 0x7feac40d1550 2024-09-12T03:22:06.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.821+0000 7feac159a640 1 -- 172.21.15.27:0/3366141373 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4287684497 0 0) 0x7feab0003520 con 0x7feac40d1550 2024-09-12T03:22:06.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.821+0000 7feac159a640 1 -- 172.21.15.27:0/3366141373 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7feac415eb90 msgr2=0x7feac40d2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:06.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.821+0000 7feac159a640 1 --2- 172.21.15.27:0/3366141373 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7feac415eb90 0x7feac40d2df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:06.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.821+0000 7feac159a640 1 -- 172.21.15.27:0/3366141373 >> v2:172.21.15.45:3300/0 conn(0x7feac40d5200 msgr2=0x7feac40d1010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:06.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.821+0000 7feac159a640 1 --2- 172.21.15.27:0/3366141373 >> v2:172.21.15.45:3300/0 conn(0x7feac40d5200 0x7feac40d1010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:06.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.821+0000 7feac159a640 1 -- 172.21.15.27:0/3366141373 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feac40d3760 con 0x7feac40d1550 2024-09-12T03:22:06.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.821+0000 7feac159a640 1 -- 172.21.15.27:0/3366141373 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 847755213 0 0) 0x7feab4001670 con 0x7feac40d1550 2024-09-12T03:22:06.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.821+0000 7feac1d9b640 1 --2- 172.21.15.27:0/3366141373 >> v2:172.21.15.45:3300/0 conn(0x7feac40d5200 0x7feac40d1010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:22:06.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.821+0000 7feac259c640 1 --2- 172.21.15.27:0/3366141373 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7feac415eb90 0x7feac40d2df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:22:06.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.821+0000 7feac159a640 1 -- 172.21.15.27:0/3366141373 >> v1:172.21.15.27:6789/0 conn(0x7feac40d1550 legacy=0x7feac40cf200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:06.826 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.821+0000 7feac159a640 1 --2- 172.21.15.27:0/3366141373 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feab00039b0 0x7feab0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:06.826 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.821+0000 7feac159a640 1 --2- 172.21.15.27:0/3366141373 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7feac415eb90 0x7feab0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:06.826 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.821+0000 7feac159a640 1 --2- 172.21.15.27:0/3366141373 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feab0004d40 0x7feab00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:06.826 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.821+0000 7feac159a640 1 -- 172.21.15.27:0/3366141373 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7feac40d42e0 con 0x7feab0004d40 2024-09-12T03:22:06.826 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.821+0000 7feac159a640 1 -- 172.21.15.27:0/3366141373 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7feab000b9c0 con 0x7feab00039b0 2024-09-12T03:22:06.826 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.821+0000 7feac159a640 1 -- 172.21.15.27:0/3366141373 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7feab000bb40 con 0x7feac415eb90 2024-09-12T03:22:06.826 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.821+0000 7feac2d9d640 1 --2- 172.21.15.27:0/3366141373 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feab0004d40 0x7feab00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:06.826 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.821+0000 7feac259c640 1 --2- 172.21.15.27:0/3366141373 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feab00039b0 0x7feab0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:06.826 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.821+0000 7feac1d9b640 1 --2- 172.21.15.27:0/3366141373 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7feac415eb90 0x7feab0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:06.826 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.821+0000 7feac2d9d640 1 -- 172.21.15.27:0/3366141373 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7feac415eb90 msgr2=0x7feab0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:06.826 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.821+0000 7feac2d9d640 1 --2- 172.21.15.27:0/3366141373 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7feac415eb90 0x7feab0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:06.826 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.822+0000 7feac2d9d640 1 -- 172.21.15.27:0/3366141373 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feab00039b0 msgr2=0x7feab0003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:06.826 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.822+0000 7feac2d9d640 1 --2- 172.21.15.27:0/3366141373 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feab00039b0 0x7feab0003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:06.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.822+0000 7feac2d9d640 1 -- 172.21.15.27:0/3366141373 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feab0003520 con 0x7feab0004d40 2024-09-12T03:22:06.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.822+0000 7feac1d9b640 1 --2- 172.21.15.27:0/3366141373 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7feac415eb90 0x7feab0004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:22:06.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.822+0000 7feac2d9d640 1 --2- 172.21.15.27:0/3366141373 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feab0004d40 0x7feab00051c0 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7feab400c290 tx=0x7feab0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8985606a80e6b6e4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:06.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.822+0000 7feac159a640 1 -- 172.21.15.27:0/3366141373 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feab4007d50 con 0x7feab0004d40 2024-09-12T03:22:06.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.822+0000 7feac159a640 1 -- 172.21.15.27:0/3366141373 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feab4005af0 con 0x7feab0004d40 2024-09-12T03:22:06.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.822+0000 7feac159a640 1 -- 172.21.15.27:0/3366141373 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feab4007070 con 0x7feab0004d40 2024-09-12T03:22:06.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.823+0000 7feac8bf6640 1 -- 172.21.15.27:0/3366141373 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feab0004d40 msgr2=0x7feab00051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:06.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.823+0000 7feac8bf6640 1 --2- 172.21.15.27:0/3366141373 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feab0004d40 0x7feab00051c0 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7feab400c290 tx=0x7feab0005830 comp rx=0 tx=0).stop 2024-09-12T03:22:06.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.823+0000 7feac2d9d640 1 -- 172.21.15.27:0/3366141373 reap_dead start 2024-09-12T03:22:06.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.823+0000 7feac8bf6640 1 -- 172.21.15.27:0/3366141373 shutdown_connections 2024-09-12T03:22:06.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.823+0000 7feac8bf6640 1 -- 172.21.15.27:0/3366141373 >> 172.21.15.27:0/3366141373 conn(0x7feac406dce0 msgr2=0x7feac406e0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:06.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.823+0000 7feac8bf6640 1 -- 172.21.15.27:0/3366141373 shutdown_connections 2024-09-12T03:22:06.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.823+0000 7feac8bf6640 1 -- 172.21.15.27:0/3366141373 wait complete. 2024-09-12T03:22:06.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.824+0000 7feac8bf6640 1 Processor -- start 2024-09-12T03:22:06.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.824+0000 7feac8bf6640 1 -- start start 2024-09-12T03:22:06.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.825+0000 7feac8bf6640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feab0004d40 0x7feac415ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:06.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.825+0000 7feac8bf6640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7feac40c9c00 0x7feac40b8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:06.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.825+0000 7feac8bf6640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feac40b9070 0x7feac40abee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:06.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.825+0000 7feac8bf6640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7feac40d4000 con 0x7feab0004d40 2024-09-12T03:22:06.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.825+0000 7feac8bf6640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7feac40d3d20 con 0x7feac40b9070 2024-09-12T03:22:06.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.825+0000 7feac8bf6640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7feac415ef70 con 0x7feac40c9c00 2024-09-12T03:22:06.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.825+0000 7feac259c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feab0004d40 0x7feac415ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:06.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.825+0000 7feac259c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feab0004d40 0x7feac415ab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42598/0 (socket says 172.21.15.27:42598) 2024-09-12T03:22:06.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.825+0000 7feac259c640 1 -- 172.21.15.27:0/4128320770 learned_addr learned my addr 172.21.15.27:0/4128320770 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:22:06.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.826+0000 7feac2d9d640 1 --2- 172.21.15.27:0/4128320770 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feac40b9070 0x7feac40abee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:06.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.826+0000 7feac1d9b640 1 --2- 172.21.15.27:0/4128320770 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7feac40c9c00 0x7feac40b8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:06.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.826+0000 7feac259c640 1 -- 172.21.15.27:0/4128320770 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7feac40c9c00 msgr2=0x7feac40b8b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:06.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.826+0000 7feac259c640 1 --2- 172.21.15.27:0/4128320770 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7feac40c9c00 0x7feac40b8b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:06.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.826+0000 7feac259c640 1 -- 172.21.15.27:0/4128320770 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feac40b9070 msgr2=0x7feac40abee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:06.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.826+0000 7feac259c640 1 --2- 172.21.15.27:0/4128320770 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feac40b9070 0x7feac40abee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:06.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.826+0000 7feac259c640 1 -- 172.21.15.27:0/4128320770 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feac41f4120 con 0x7feab0004d40 2024-09-12T03:22:06.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.826+0000 7feac259c640 1 --2- 172.21.15.27:0/4128320770 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feab0004d40 0x7feac415ab50 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7feabc009fb0 tx=0x7feabc00f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:06.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.826+0000 7feabaffd640 1 -- 172.21.15.27:0/4128320770 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feabc00fea0 con 0x7feab0004d40 2024-09-12T03:22:06.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.826+0000 7feabaffd640 1 -- 172.21.15.27:0/4128320770 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feabc003b90 con 0x7feab0004d40 2024-09-12T03:22:06.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.826+0000 7feac8bf6640 1 -- 172.21.15.27:0/4128320770 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feac41f4400 con 0x7feab0004d40 2024-09-12T03:22:06.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.826+0000 7feabaffd640 1 -- 172.21.15.27:0/4128320770 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feabc004020 con 0x7feab0004d40 2024-09-12T03:22:06.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.826+0000 7feac8bf6640 1 -- 172.21.15.27:0/4128320770 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feac41f49c0 con 0x7feab0004d40 2024-09-12T03:22:06.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.827+0000 7feac8bf6640 1 -- 172.21.15.27:0/4128320770 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feac40d1010 con 0x7feab0004d40 2024-09-12T03:22:06.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.827+0000 7feabaffd640 1 -- 172.21.15.27:0/4128320770 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 21) ==== 91008+0+0 (secure 0 0 0) 0x7feabc0041c0 con 0x7feab0004d40 2024-09-12T03:22:06.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.828+0000 7feabaffd640 1 --2- 172.21.15.27:0/4128320770 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7feaa406c3d0 0x7feaa406e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:06.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.828+0000 7feabaffd640 1 -- 172.21.15.27:0/4128320770 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5737+0+0 (secure 0 0 0) 0x7feabc08c680 con 0x7feab0004d40 2024-09-12T03:22:06.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.828+0000 7feac1d9b640 1 --2- 172.21.15.27:0/4128320770 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7feaa406c3d0 0x7feaa406e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:06.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.828+0000 7feac1d9b640 1 --2- 172.21.15.27:0/4128320770 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7feaa406c3d0 0x7feaa406e890 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7feac40ca020 tx=0x7feaac008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:06.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:06.832+0000 7feabaffd640 1 -- 172.21.15.27:0/4128320770 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7feabc019030 con 0x7feab0004d40 2024-09-12T03:22:07.008 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.007+0000 7feac8bf6640 1 -- 172.21.15.27:0/4128320770 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7feac4069ea0 con 0x7feab0004d40 2024-09-12T03:22:07.009 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.008+0000 7feabaffd640 1 -- 172.21.15.27:0/4128320770 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v114) ==== 74+0+16735 (secure 0 0 0) 0x7feabc010040 con 0x7feab0004d40 2024-09-12T03:22:07.010 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:22:07.010 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":114,"fsid":"d2df504e-2647-4385-a5db-9fdf69f487a0","created":"2024-09-12T03:16:07.778937+0000","modified":"2024-09-12T03:21:48.262478+0000","last_up_change":"2024-09-12T03:16:19.314778+0000","last_in_change":"2024-09-12T03:16:09.893086+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T03:16:20.097646+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-09-12T03:21:44.263535+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-09-12T03:21:45.593648+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"114","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1851116c-1fb3-4dc9-9772-f90e2ef4c355","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6817","nonce":3002321488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6819","nonce":3002321488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6823","nonce":3002321488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6821","nonce":3002321488}]},"public_addr":"172.21.15.27:6817/3002321488","cluster_addr":"172.21.15.27:6819/3002321488","heartbeat_back_addr":"172.21.15.27:6823/3002321488","heartbeat_front_addr":"172.21.15.27:6821/3002321488","state":["exists","up"]},{"osd":1,"uuid":"f38ae17d-73eb-4204-b99d-975fa806fb87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6809","nonce":1613102886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6811","nonce":1613102886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6815","nonce":1613102886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6813","nonce":1613102886}]},"public_addr":"172.21.15.27:6809/1613102886","cluster_addr":"172.21.15.27:6811/1613102886","heartbeat_back_addr":"172.21.15.27:6815/1613102886","heartbeat_front_addr":"172.21.15.27:6813/1613102886","state":["exists","up"]},{"osd":2,"uuid":"fb12dcc8-bdd0-42b8-b46d-8b10106224bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6825","nonce":3534723355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6827","nonce":3534723355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6831","nonce":3534723355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6829","nonce":3534723355}]},"public_addr":"172.21.15.27:6825/3534723355","cluster_addr":"172.21.15.27:6827/3534723355","heartbeat_back_addr":"172.21.15.27:6831/3534723355","heartbeat_front_addr":"172.21.15.27:6829/3534723355","state":["exists","up"]},{"osd":3,"uuid":"d74d3f49-9c24-4b9f-98e2-3ecd7b5ef3b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6801","nonce":80661598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6803","nonce":80661598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6807","nonce":80661598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6805","nonce":80661598}]},"public_addr":"172.21.15.27:6801/80661598","cluster_addr":"172.21.15.27:6803/80661598","heartbeat_back_addr":"172.21.15.27:6807/80661598","heartbeat_front_addr":"172.21.15.27:6805/80661598","state":["exists","up"]},{"osd":4,"uuid":"2bf123f2-1b0d-41c4-aa3b-805bab267e59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6817","nonce":3943189827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6819","nonce":3943189827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6823","nonce":3943189827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6821","nonce":3943189827}]},"public_addr":"172.21.15.45:6817/3943189827","cluster_addr":"172.21.15.45:6819/3943189827","heartbeat_back_addr":"172.21.15.45:6823/3943189827","heartbeat_front_addr":"172.21.15.45:6821/3943189827","state":["exists","up"]},{"osd":5,"uuid":"c49ba5da-91c6-4a86-a97b-4aba89c253ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6809","nonce":3620760330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6811","nonce":3620760330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6815","nonce":3620760330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6813","nonce":3620760330}]},"public_addr":"172.21.15.45:6809/3620760330","cluster_addr":"172.21.15.45:6811/3620760330","heartbeat_back_addr":"172.21.15.45:6815/3620760330","heartbeat_front_addr":"172.21.15.45:6813/3620760330","state":["exists","up"]},{"osd":6,"uuid":"b0123475-89f7-4a34-9e33-d797b99dd9dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6801","nonce":1989083510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6803","nonce":1989083510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6807","nonce":1989083510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6805","nonce":1989083510}]},"public_addr":"172.21.15.45:6801/1989083510","cluster_addr":"172.21.15.45:6803/1989083510","heartbeat_back_addr":"172.21.15.45:6807/1989083510","heartbeat_front_addr":"172.21.15.45:6805/1989083510","state":["exists","up"]},{"osd":7,"uuid":"4dad773d-b7d8-46fc-8a10-a972aad27b29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6825","nonce":1513300294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6827","nonce":1513300294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6831","nonce":1513300294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6829","nonce":1513300294}]},"public_addr":"172.21.15.45:6825/1513300294","cluster_addr":"172.21.15.45:6827/1513300294","heartbeat_back_addr":"172.21.15.45:6831/1513300294","heartbeat_front_addr":"172.21.15.45:6829/1513300294","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.881855+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:17.133252+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.735339+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.500742+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.477571+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:17.167667+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.470291+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.873743+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T03:22:07.012 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.010+0000 7feac8bf6640 1 -- 172.21.15.27:0/4128320770 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7feaa406c3d0 msgr2=0x7feaa406e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:07.012 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.010+0000 7feac8bf6640 1 --2- 172.21.15.27:0/4128320770 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7feaa406c3d0 0x7feaa406e890 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7feac40ca020 tx=0x7feaac008040 comp rx=0 tx=0).stop 2024-09-12T03:22:07.012 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.010+0000 7feac8bf6640 1 -- 172.21.15.27:0/4128320770 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feab0004d40 msgr2=0x7feac415ab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:07.012 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.010+0000 7feac8bf6640 1 --2- 172.21.15.27:0/4128320770 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feab0004d40 0x7feac415ab50 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7feabc009fb0 tx=0x7feabc00f580 comp rx=0 tx=0).stop 2024-09-12T03:22:07.012 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.010+0000 7feac8bf6640 1 -- 172.21.15.27:0/4128320770 shutdown_connections 2024-09-12T03:22:07.012 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.010+0000 7feac8bf6640 1 --2- 172.21.15.27:0/4128320770 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feac40b9070 0x7feac40abee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:07.012 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.010+0000 7feac8bf6640 1 --2- 172.21.15.27:0/4128320770 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7feac40c9c00 0x7feac40b8b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:07.012 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.010+0000 7feac8bf6640 1 --2- 172.21.15.27:0/4128320770 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7feaa406c3d0 0x7feaa406e890 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:07.012 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.010+0000 7feac8bf6640 1 --2- 172.21.15.27:0/4128320770 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feab0004d40 0x7feac415ab50 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:07.012 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.010+0000 7feac8bf6640 1 -- 172.21.15.27:0/4128320770 >> 172.21.15.27:0/4128320770 conn(0x7feac406dce0 msgr2=0x7feac40ce310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:07.013 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.011+0000 7feac8bf6640 1 -- 172.21.15.27:0/4128320770 shutdown_connections 2024-09-12T03:22:07.013 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.011+0000 7feac8bf6640 1 -- 172.21.15.27:0/4128320770 wait complete. 2024-09-12T03:22:07.023 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-12T03:22:07.023 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-12T03:22:07.125 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.124+0000 7fcae858d640 1 Processor -- start 2024-09-12T03:22:07.126 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.124+0000 7fcae858d640 1 -- start start 2024-09-12T03:22:07.127 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.125+0000 7fcae858d640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fcae00d1050 0x7fcae00cf810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:07.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.125+0000 7fcae858d640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fcae00d3120 0x7fcae00d35d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:07.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.125+0000 7fcae858d640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcae00d1e60 con 0x7fcae00d52e0 2024-09-12T03:22:07.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.125+0000 7fcae858d640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fcae00d2380 con 0x7fcae00d3120 2024-09-12T03:22:07.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.125+0000 7fcae858d640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fcae00cfef0 con 0x7fcae00d1050 2024-09-12T03:22:07.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.125+0000 7fcae5b28640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fcae00d52e0 0x7fcae00d1430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:51294/0 (socket says 172.21.15.27:51294) 2024-09-12T03:22:07.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.125+0000 7fcae5b28640 1 -- 172.21.15.27:0/293562525 learned_addr learned my addr 172.21.15.27:0/293562525 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:22:07.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.125+0000 7fcae6b2a640 1 --2- 172.21.15.27:0/293562525 >> v2:172.21.15.45:3300/0 conn(0x7fcae00d3120 0x7fcae00d35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:07.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.125+0000 7fcae6329640 1 --2- 172.21.15.27:0/293562525 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fcae00d1050 0x7fcae00cf810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:07.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.126+0000 7fcae5327640 1 -- 172.21.15.27:0/293562525 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3215306747 0 0) 0x7fcae00d1e60 con 0x7fcae00d52e0 2024-09-12T03:22:07.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.126+0000 7fcae5327640 1 -- 172.21.15.27:0/293562525 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcacc003520 con 0x7fcae00d52e0 2024-09-12T03:22:07.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.126+0000 7fcae6329640 1 -- 172.21.15.27:0/293562525 >> v2:172.21.15.45:3300/0 conn(0x7fcae00d3120 msgr2=0x7fcae00d35d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:07.129 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.126+0000 7fcae6329640 1 --2- 172.21.15.27:0/293562525 >> v2:172.21.15.45:3300/0 conn(0x7fcae00d3120 0x7fcae00d35d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:07.129 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.126+0000 7fcae6329640 1 -- 172.21.15.27:0/293562525 >> v1:172.21.15.27:6789/0 conn(0x7fcae00d52e0 legacy=0x7fcae00d1430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:07.129 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.126+0000 7fcae5327640 1 -- 172.21.15.27:0/293562525 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 909163124 0 0) 0x7fcacc003520 con 0x7fcae00d52e0 2024-09-12T03:22:07.129 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.126+0000 7fcae6329640 1 -- 172.21.15.27:0/293562525 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcae00d2640 con 0x7fcae00d1050 2024-09-12T03:22:07.129 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.127+0000 7fcae6329640 1 --2- 172.21.15.27:0/293562525 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fcae00d1050 0x7fcae00cf810 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fcadc0097a0 tx=0x7fcadc02a110 comp rx=0 tx=0).ready entity=mon.2 client_cookie=10fb2253eb293b5a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:07.129 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.127+0000 7fcae5327640 1 -- 172.21.15.27:0/293562525 <== mon.2 v2:172.21.15.45:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcadc037070 con 0x7fcae00d1050 2024-09-12T03:22:07.129 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.127+0000 7fcae5327640 1 -- 172.21.15.27:0/293562525 <== mon.2 v2:172.21.15.45:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcadc02aba0 con 0x7fcae00d1050 2024-09-12T03:22:07.129 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.127+0000 7fcae5327640 1 -- 172.21.15.27:0/293562525 <== mon.2 v2:172.21.15.45:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcad0001670 con 0x7fcae00d1050 2024-09-12T03:22:07.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.127+0000 7fcae858d640 1 -- 172.21.15.27:0/293562525 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fcae00d1050 msgr2=0x7fcae00cf810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:07.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.127+0000 7fcae858d640 1 --2- 172.21.15.27:0/293562525 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fcae00d1050 0x7fcae00cf810 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fcadc0097a0 tx=0x7fcadc02a110 comp rx=0 tx=0).stop 2024-09-12T03:22:07.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.127+0000 7fcae858d640 1 -- 172.21.15.27:0/293562525 shutdown_connections 2024-09-12T03:22:07.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.127+0000 7fcae858d640 1 --2- 172.21.15.27:0/293562525 >> v2:172.21.15.45:3300/0 conn(0x7fcae00d3120 0x7fcae00d35d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:07.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.127+0000 7fcae858d640 1 --2- 172.21.15.27:0/293562525 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fcae00d1050 0x7fcae00cf810 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:07.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.127+0000 7fcae858d640 1 -- 172.21.15.27:0/293562525 >> 172.21.15.27:0/293562525 conn(0x7fcae006ddf0 msgr2=0x7fcae006e200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:07.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.127+0000 7fcae858d640 1 -- 172.21.15.27:0/293562525 shutdown_connections 2024-09-12T03:22:07.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.127+0000 7fcae858d640 1 -- 172.21.15.27:0/293562525 wait complete. 2024-09-12T03:22:07.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.129+0000 7fcae858d640 1 Processor -- start 2024-09-12T03:22:07.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.129+0000 7fcae858d640 1 -- start start 2024-09-12T03:22:07.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.129+0000 7fcae858d640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fcae00d1050 0x7fcae015a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:07.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.129+0000 7fcae858d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcae00d3120 0x7fcae01569b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:07.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.129+0000 7fcae858d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcae00d52e0 0x7fcae00c8b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:07.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.129+0000 7fcae858d640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fcae00b7a50 con 0x7fcae00d52e0 2024-09-12T03:22:07.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.129+0000 7fcae858d640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fcae00d2c00 con 0x7fcae00d3120 2024-09-12T03:22:07.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.129+0000 7fcae858d640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fcae00b7bd0 con 0x7fcae00d1050 2024-09-12T03:22:07.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.129+0000 7fcae6b2a640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcae00d52e0 0x7fcae00c8b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:07.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.129+0000 7fcae6b2a640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcae00d52e0 0x7fcae00c8b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42612/0 (socket says 172.21.15.27:42612) 2024-09-12T03:22:07.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.129+0000 7fcae6b2a640 1 -- 172.21.15.27:0/2142706029 learned_addr learned my addr 172.21.15.27:0/2142706029 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:22:07.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.130+0000 7fcae6329640 1 --2- 172.21.15.27:0/2142706029 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fcae00d1050 0x7fcae015a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:07.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.130+0000 7fcae5b28640 1 --2- 172.21.15.27:0/2142706029 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcae00d3120 0x7fcae01569b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:07.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.130+0000 7fcae6b2a640 1 -- 172.21.15.27:0/2142706029 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fcae00d1050 msgr2=0x7fcae015a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:07.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.130+0000 7fcae6b2a640 1 --2- 172.21.15.27:0/2142706029 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fcae00d1050 0x7fcae015a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:07.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.130+0000 7fcae6b2a640 1 -- 172.21.15.27:0/2142706029 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcae00d3120 msgr2=0x7fcae01569b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:07.138 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.130+0000 7fcae6b2a640 1 --2- 172.21.15.27:0/2142706029 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcae00d3120 0x7fcae01569b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:07.138 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.130+0000 7fcae6b2a640 1 -- 172.21.15.27:0/2142706029 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcae01584f0 con 0x7fcae00d52e0 2024-09-12T03:22:07.138 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.130+0000 7fcae6b2a640 1 --2- 172.21.15.27:0/2142706029 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcae00d52e0 0x7fcae00c8b40 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7fcad800efc0 tx=0x7fcad800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:07.138 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.130+0000 7fcad6ffd640 1 -- 172.21.15.27:0/2142706029 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcad8018020 con 0x7fcae00d52e0 2024-09-12T03:22:07.138 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.130+0000 7fcad6ffd640 1 -- 172.21.15.27:0/2142706029 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcad8014480 con 0x7fcae00d52e0 2024-09-12T03:22:07.138 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.130+0000 7fcae858d640 1 -- 172.21.15.27:0/2142706029 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcae00c8340 con 0x7fcae00d52e0 2024-09-12T03:22:07.138 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.130+0000 7fcad6ffd640 1 -- 172.21.15.27:0/2142706029 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcad8014870 con 0x7fcae00d52e0 2024-09-12T03:22:07.138 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.130+0000 7fcae858d640 1 -- 172.21.15.27:0/2142706029 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcae015a820 con 0x7fcae00d52e0 2024-09-12T03:22:07.138 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.131+0000 7fcae858d640 1 -- 172.21.15.27:0/2142706029 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcae015ec60 con 0x7fcae00d52e0 2024-09-12T03:22:07.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.132+0000 7fcad6ffd640 1 -- 172.21.15.27:0/2142706029 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 21) ==== 91008+0+0 (secure 0 0 0) 0x7fcad8005d10 con 0x7fcae00d52e0 2024-09-12T03:22:07.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.132+0000 7fcad6ffd640 1 --2- 172.21.15.27:0/2142706029 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fcac406c3d0 0x7fcac406e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:07.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.133+0000 7fcae6329640 1 --2- 172.21.15.27:0/2142706029 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fcac406c3d0 0x7fcac406e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:07.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.133+0000 7fcad6ffd640 1 -- 172.21.15.27:0/2142706029 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5737+0+0 (secure 0 0 0) 0x7fcad808c840 con 0x7fcae00d52e0 2024-09-12T03:22:07.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.133+0000 7fcae6329640 1 --2- 172.21.15.27:0/2142706029 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fcac406c3d0 0x7fcac406e890 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fcadc034000 tx=0x7fcadc02e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:07.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.137+0000 7fcad6ffd640 1 -- 172.21.15.27:0/2142706029 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fcad8022030 con 0x7fcae00d52e0 2024-09-12T03:22:07.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.358+0000 7fcae858d640 1 -- 172.21.15.27:0/2142706029 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcae00d20a0 con 0x7fcae00d52e0 2024-09-12T03:22:07.359 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.358+0000 7fcad6ffd640 1 -- 172.21.15.27:0/2142706029 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) ==== 94+0+5225 (secure 0 0 0) 0x7fcad8010040 con 0x7fcae00d52e0 2024-09-12T03:22:07.359 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:22:07.359 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":68,"btime":"2024-09-12T03:21:59:298296+0000","default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6206,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/2249506350","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2249506350},{"type":"v1","addr":"172.21.15.45:6834","nonce":2249506350}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63},{"gid":6209,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/2292759672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":2292759672},{"type":"v1","addr":"172.21.15.45:6835","nonce":2292759672}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63},{"gid":7299,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6834/4021825986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":4021825986},{"type":"v1","addr":"172.21.15.27:6834","nonce":4021825986}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":67,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-12T03:21:46.603883+0000","modified":"2024-09-12T03:21:48.956285+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7302},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7302":{"gid":7302,"name":"a","rank":0,"incarnation":64,"state":"up:active","state_seq":3,"addr":"172.21.15.27:6835/1760654641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":1760654641},{"type":"v1","addr":"172.21.15.27:6835","nonce":1760654641}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7302,"qdb_cluster":[7302]},"id":9}]} 2024-09-12T03:22:07.360 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 68 2024-09-12T03:22:07.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.360+0000 7fcae858d640 1 -- 172.21.15.27:0/2142706029 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fcac406c3d0 msgr2=0x7fcac406e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:07.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.360+0000 7fcae858d640 1 --2- 172.21.15.27:0/2142706029 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fcac406c3d0 0x7fcac406e890 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fcadc034000 tx=0x7fcadc02e000 comp rx=0 tx=0).stop 2024-09-12T03:22:07.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.360+0000 7fcae858d640 1 -- 172.21.15.27:0/2142706029 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcae00d52e0 msgr2=0x7fcae00c8b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:07.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.360+0000 7fcae858d640 1 --2- 172.21.15.27:0/2142706029 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcae00d52e0 0x7fcae00c8b40 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7fcad800efc0 tx=0x7fcad800c490 comp rx=0 tx=0).stop 2024-09-12T03:22:07.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.360+0000 7fcae858d640 1 -- 172.21.15.27:0/2142706029 shutdown_connections 2024-09-12T03:22:07.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.361+0000 7fcae858d640 1 --2- 172.21.15.27:0/2142706029 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fcac406c3d0 0x7fcac406e890 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:07.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.361+0000 7fcae858d640 1 --2- 172.21.15.27:0/2142706029 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcae00d52e0 0x7fcae00c8b40 unknown :-1 s=CLOSED pgs=1167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:07.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.361+0000 7fcae858d640 1 --2- 172.21.15.27:0/2142706029 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcae00d3120 0x7fcae01569b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:07.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.361+0000 7fcae858d640 1 --2- 172.21.15.27:0/2142706029 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fcae00d1050 0x7fcae015a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:07.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.361+0000 7fcae858d640 1 -- 172.21.15.27:0/2142706029 >> 172.21.15.27:0/2142706029 conn(0x7fcae006ddf0 msgr2=0x7fcae00d1430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:07.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.361+0000 7fcae858d640 1 -- 172.21.15.27:0/2142706029 shutdown_connections 2024-09-12T03:22:07.363 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.361+0000 7fcae858d640 1 -- 172.21.15.27:0/2142706029 wait complete. 2024-09-12T03:22:07.372 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-12T03:22:07.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.473+0000 7fd9d55a6640 1 Processor -- start 2024-09-12T03:22:07.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.473+0000 7fd9d55a6640 1 -- start start 2024-09-12T03:22:07.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.474+0000 7fd9d55a6640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fd9d0072d70 0x7fd9d00b3560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:07.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.474+0000 7fd9d55a6640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fd9d006e9b0 0x7fd9d006ee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:07.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.474+0000 7fd9d55a6640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd9d00ce410 con 0x7fd9d015b2b0 2024-09-12T03:22:07.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.474+0000 7fd9d55a6640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7fd9d00720a0 con 0x7fd9d0072d70 2024-09-12T03:22:07.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.474+0000 7fd9d55a6640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fd9d006f340 con 0x7fd9d006e9b0 2024-09-12T03:22:07.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.474+0000 7fd9ce7fc640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fd9d0072d70 0x7fd9d00b3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:07.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.474+0000 7fd9ce7fc640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fd9d0072d70 0x7fd9d00b3560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42618/0 (socket says 172.21.15.27:42618) 2024-09-12T03:22:07.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.474+0000 7fd9ce7fc640 1 -- 172.21.15.27:0/2213580315 learned_addr learned my addr 172.21.15.27:0/2213580315 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:22:07.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.475+0000 7fd9cf7fe640 1 --2- 172.21.15.27:0/2213580315 >> v2:172.21.15.45:3300/0 conn(0x7fd9d006e9b0 0x7fd9d006ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:07.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.475+0000 7fd9ce7fc640 1 -- 172.21.15.27:0/2213580315 >> v2:172.21.15.45:3300/0 conn(0x7fd9d006e9b0 msgr2=0x7fd9d006ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:07.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.475+0000 7fd9ce7fc640 1 --2- 172.21.15.27:0/2213580315 >> v2:172.21.15.45:3300/0 conn(0x7fd9d006e9b0 0x7fd9d006ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:07.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.475+0000 7fd9ce7fc640 1 -- 172.21.15.27:0/2213580315 >> v1:172.21.15.45:6789/0 conn(0x7fd9d015b2b0 legacy=0x7fd9d0072430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:07.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.475+0000 7fd9ce7fc640 1 -- 172.21.15.27:0/2213580315 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9d00732c0 con 0x7fd9d0072d70 2024-09-12T03:22:07.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.475+0000 7fd9ce7fc640 1 --2- 172.21.15.27:0/2213580315 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd9d0072d70 0x7fd9d00b3560 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7fd9d00ce1e0 tx=0x7fd9c402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea22bfbbf62c62a5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:07.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.476+0000 7fd9cdffb640 1 -- 172.21.15.27:0/2213580315 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9c4037070 con 0x7fd9d0072d70 2024-09-12T03:22:07.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.476+0000 7fd9cdffb640 1 -- 172.21.15.27:0/2213580315 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9c4032440 con 0x7fd9d0072d70 2024-09-12T03:22:07.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.476+0000 7fd9cdffb640 1 -- 172.21.15.27:0/2213580315 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9c4032810 con 0x7fd9d0072d70 2024-09-12T03:22:07.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.476+0000 7fd9d55a6640 1 -- 172.21.15.27:0/2213580315 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd9d0072d70 msgr2=0x7fd9d00b3560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:07.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.476+0000 7fd9d55a6640 1 --2- 172.21.15.27:0/2213580315 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd9d0072d70 0x7fd9d00b3560 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7fd9d00ce1e0 tx=0x7fd9c402a230 comp rx=0 tx=0).stop 2024-09-12T03:22:07.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.476+0000 7fd9d55a6640 1 -- 172.21.15.27:0/2213580315 shutdown_connections 2024-09-12T03:22:07.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.476+0000 7fd9d55a6640 1 --2- 172.21.15.27:0/2213580315 >> v2:172.21.15.45:3300/0 conn(0x7fd9d006e9b0 0x7fd9d006ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:07.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.476+0000 7fd9d55a6640 1 --2- 172.21.15.27:0/2213580315 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd9d0072d70 0x7fd9d00b3560 unknown :-1 s=CLOSED pgs=1168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:07.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.476+0000 7fd9d55a6640 1 -- 172.21.15.27:0/2213580315 >> 172.21.15.27:0/2213580315 conn(0x7fd9d0062c60 msgr2=0x7fd9d006bb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:07.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.476+0000 7fd9d55a6640 1 -- 172.21.15.27:0/2213580315 shutdown_connections 2024-09-12T03:22:07.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.476+0000 7fd9d55a6640 1 -- 172.21.15.27:0/2213580315 wait complete. 2024-09-12T03:22:07.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.477+0000 7fd9d55a6640 1 Processor -- start 2024-09-12T03:22:07.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.478+0000 7fd9d55a6640 1 -- start start 2024-09-12T03:22:07.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.478+0000 7fd9d55a6640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd9d006e9b0 0x7fd9d0147ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:07.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.478+0000 7fd9d55a6640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fd9d0072d70 0x7fd9d01524a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:07.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.478+0000 7fd9d55a6640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd9d015b2b0 0x7fd9d01529e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:07.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.478+0000 7fd9d55a6640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fd9d00ce690 con 0x7fd9d015b2b0 2024-09-12T03:22:07.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.478+0000 7fd9d55a6640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fd9d0071dc0 con 0x7fd9d006e9b0 2024-09-12T03:22:07.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.478+0000 7fd9d55a6640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fd9d015b690 con 0x7fd9d0072d70 2024-09-12T03:22:07.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.478+0000 7fd9cf7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd9d015b2b0 0x7fd9d01529e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:07.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.478+0000 7fd9cf7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd9d015b2b0 0x7fd9d01529e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42620/0 (socket says 172.21.15.27:42620) 2024-09-12T03:22:07.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.478+0000 7fd9cf7fe640 1 -- 172.21.15.27:0/1307704808 learned_addr learned my addr 172.21.15.27:0/1307704808 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:22:07.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.479+0000 7fd9ceffd640 1 --2- 172.21.15.27:0/1307704808 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd9d006e9b0 0x7fd9d0147ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:07.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.479+0000 7fd9cf7fe640 1 -- 172.21.15.27:0/1307704808 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fd9d0072d70 msgr2=0x7fd9d01524a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:07.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.479+0000 7fd9cf7fe640 1 --2- 172.21.15.27:0/1307704808 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fd9d0072d70 0x7fd9d01524a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:07.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.479+0000 7fd9cf7fe640 1 -- 172.21.15.27:0/1307704808 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd9d006e9b0 msgr2=0x7fd9d0147ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:07.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.479+0000 7fd9cf7fe640 1 --2- 172.21.15.27:0/1307704808 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd9d006e9b0 0x7fd9d0147ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:07.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.479+0000 7fd9cf7fe640 1 -- 172.21.15.27:0/1307704808 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9d0148020 con 0x7fd9d015b2b0 2024-09-12T03:22:07.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.479+0000 7fd9ceffd640 1 --2- 172.21.15.27:0/1307704808 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd9d006e9b0 0x7fd9d0147ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:22:07.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.479+0000 7fd9cf7fe640 1 --2- 172.21.15.27:0/1307704808 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd9d015b2b0 0x7fd9d01529e0 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7fd9c8007e40 tx=0x7fd9c800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:07.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.479+0000 7fd9b77fe640 1 -- 172.21.15.27:0/1307704808 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9c8014620 con 0x7fd9d015b2b0 2024-09-12T03:22:07.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.479+0000 7fd9b77fe640 1 -- 172.21.15.27:0/1307704808 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9c8019070 con 0x7fd9d015b2b0 2024-09-12T03:22:07.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.479+0000 7fd9b77fe640 1 -- 172.21.15.27:0/1307704808 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9c8014a20 con 0x7fd9d015b2b0 2024-09-12T03:22:07.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.479+0000 7fd9d55a6640 1 -- 172.21.15.27:0/1307704808 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9d015dbc0 con 0x7fd9d015b2b0 2024-09-12T03:22:07.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.479+0000 7fd9d55a6640 1 -- 172.21.15.27:0/1307704808 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9d015e180 con 0x7fd9d015b2b0 2024-09-12T03:22:07.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.480+0000 7fd9d55a6640 1 -- 172.21.15.27:0/1307704808 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9d006ee00 con 0x7fd9d015b2b0 2024-09-12T03:22:07.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.481+0000 7fd9b77fe640 1 -- 172.21.15.27:0/1307704808 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 21) ==== 91008+0+0 (secure 0 0 0) 0x7fd9c8014bc0 con 0x7fd9d015b2b0 2024-09-12T03:22:07.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.481+0000 7fd9b77fe640 1 --2- 172.21.15.27:0/1307704808 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fd9bc06c3d0 0x7fd9bc06e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:07.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.481+0000 7fd9b77fe640 1 -- 172.21.15.27:0/1307704808 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 5793+0+0 (secure 0 0 0) 0x7fd9c808c1f0 con 0x7fd9d015b2b0 2024-09-12T03:22:07.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.481+0000 7fd9ceffd640 1 --2- 172.21.15.27:0/1307704808 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fd9bc06c3d0 0x7fd9bc06e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:07.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.481+0000 7fd9ceffd640 1 --2- 172.21.15.27:0/1307704808 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fd9bc06c3d0 0x7fd9bc06e890 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fd9d014db90 tx=0x7fd9c0008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:07.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.485+0000 7fd9b77fe640 1 -- 172.21.15.27:0/1307704808 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fd9c805a2d0 con 0x7fd9d015b2b0 2024-09-12T03:22:07.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.699+0000 7fd9d55a6640 1 -- 172.21.15.27:0/1307704808 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd9d00710d0 con 0x7fd9d015b2b0 2024-09-12T03:22:07.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.700+0000 7fd9b77fe640 1 -- 172.21.15.27:0/1307704808 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) ==== 94+0+5225 (secure 0 0 0) 0x7fd9c8059bd0 con 0x7fd9d015b2b0 2024-09-12T03:22:07.701 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:22:07.701 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":68,"btime":"2024-09-12T03:21:59:298296+0000","default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6206,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/2249506350","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2249506350},{"type":"v1","addr":"172.21.15.45:6834","nonce":2249506350}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63},{"gid":6209,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/2292759672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":2292759672},{"type":"v1","addr":"172.21.15.45:6835","nonce":2292759672}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63},{"gid":7299,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6834/4021825986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":4021825986},{"type":"v1","addr":"172.21.15.27:6834","nonce":4021825986}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":67,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-12T03:21:46.603883+0000","modified":"2024-09-12T03:21:48.956285+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7302},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7302":{"gid":7302,"name":"a","rank":0,"incarnation":64,"state":"up:active","state_seq":3,"addr":"172.21.15.27:6835/1760654641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":1760654641},{"type":"v1","addr":"172.21.15.27:6835","nonce":1760654641}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7302,"qdb_cluster":[7302]},"id":9}]} 2024-09-12T03:22:07.702 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 68 2024-09-12T03:22:07.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.702+0000 7fd9d55a6640 1 -- 172.21.15.27:0/1307704808 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fd9bc06c3d0 msgr2=0x7fd9bc06e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:07.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.702+0000 7fd9d55a6640 1 --2- 172.21.15.27:0/1307704808 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fd9bc06c3d0 0x7fd9bc06e890 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fd9d014db90 tx=0x7fd9c0008040 comp rx=0 tx=0).stop 2024-09-12T03:22:07.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.702+0000 7fd9d55a6640 1 -- 172.21.15.27:0/1307704808 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd9d015b2b0 msgr2=0x7fd9d01529e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:07.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.702+0000 7fd9d55a6640 1 --2- 172.21.15.27:0/1307704808 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd9d015b2b0 0x7fd9d01529e0 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7fd9c8007e40 tx=0x7fd9c800e4b0 comp rx=0 tx=0).stop 2024-09-12T03:22:07.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.702+0000 7fd9d55a6640 1 -- 172.21.15.27:0/1307704808 shutdown_connections 2024-09-12T03:22:07.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.702+0000 7fd9d55a6640 1 --2- 172.21.15.27:0/1307704808 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fd9bc06c3d0 0x7fd9bc06e890 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:07.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.702+0000 7fd9d55a6640 1 --2- 172.21.15.27:0/1307704808 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd9d015b2b0 0x7fd9d01529e0 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:07.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.702+0000 7fd9d55a6640 1 --2- 172.21.15.27:0/1307704808 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fd9d0072d70 0x7fd9d01524a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:07.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.702+0000 7fd9d55a6640 1 --2- 172.21.15.27:0/1307704808 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd9d006e9b0 0x7fd9d0147ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:07.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.702+0000 7fd9d55a6640 1 -- 172.21.15.27:0/1307704808 >> 172.21.15.27:0/1307704808 conn(0x7fd9d0062c60 msgr2=0x7fd9d006d8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:07.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.702+0000 7fd9d55a6640 1 -- 172.21.15.27:0/1307704808 shutdown_connections 2024-09-12T03:22:07.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.702+0000 7fd9d55a6640 1 -- 172.21.15.27:0/1307704808 wait complete. 2024-09-12T03:22:07.715 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-12T03:22:07.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.819+0000 7fad7c265640 1 Processor -- start 2024-09-12T03:22:07.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.819+0000 7fad7c265640 1 -- start start 2024-09-12T03:22:07.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.820+0000 7fad7c265640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fad7415aa40 0x7fad7406e660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:07.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.820+0000 7fad7c265640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fad7406f6e0 con 0x7fad7406ef30 2024-09-12T03:22:07.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.820+0000 7fad7c265640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fad7406f9c0 con 0x7fad740728b0 2024-09-12T03:22:07.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.820+0000 7fad7c265640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fad740b3ee0 con 0x7fad7415aa40 2024-09-12T03:22:07.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.820+0000 7fad79800640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fad7406ef30 0x7fad740ce560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:51310/0 (socket says 172.21.15.27:51310) 2024-09-12T03:22:07.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.820+0000 7fad79800640 1 -- 172.21.15.27:0/3311670202 learned_addr learned my addr 172.21.15.27:0/3311670202 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:22:07.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.821+0000 7fad7a001640 1 --2- 172.21.15.27:0/3311670202 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fad7415aa40 0x7fad7406e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:07.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.821+0000 7fad78fff640 1 -- 172.21.15.27:0/3311670202 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2767336355 0 0) 0x7fad7406f6e0 con 0x7fad7406ef30 2024-09-12T03:22:07.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.821+0000 7fad78fff640 1 -- 172.21.15.27:0/3311670202 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fad60003520 con 0x7fad7406ef30 2024-09-12T03:22:07.826 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.821+0000 7fad78fff640 1 -- 172.21.15.27:0/3311670202 <== mon.1 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2067722967 0 0) 0x7fad7406f9c0 con 0x7fad740728b0 2024-09-12T03:22:07.826 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.821+0000 7fad78fff640 1 -- 172.21.15.27:0/3311670202 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fad7406f6e0 con 0x7fad740728b0 2024-09-12T03:22:07.826 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.821+0000 7fad78fff640 1 -- 172.21.15.27:0/3311670202 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 848021613 0 0) 0x7fad60003520 con 0x7fad7406ef30 2024-09-12T03:22:07.826 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.821+0000 7fad78fff640 1 -- 172.21.15.27:0/3311670202 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fad7415aa40 msgr2=0x7fad7406e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:07.826 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.821+0000 7fad78fff640 1 --2- 172.21.15.27:0/3311670202 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fad7415aa40 0x7fad7406e660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:07.826 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.821+0000 7fad78fff640 1 -- 172.21.15.27:0/3311670202 >> v1:172.21.15.45:6789/0 conn(0x7fad740728b0 legacy=0x7fad740b4930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:07.826 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.821+0000 7fad78fff640 1 -- 172.21.15.27:0/3311670202 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad740b4130 con 0x7fad7406ef30 2024-09-12T03:22:07.826 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.821+0000 7fad78fff640 1 -- 172.21.15.27:0/3311670202 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 847755213 0 0) 0x7fad64001670 con 0x7fad7406ef30 2024-09-12T03:22:07.826 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.821+0000 7fad78fff640 1 -- 172.21.15.27:0/3311670202 >> v1:172.21.15.27:6789/0 conn(0x7fad7406ef30 legacy=0x7fad740ce560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:07.826 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.821+0000 7fad78fff640 1 --2- 172.21.15.27:0/3311670202 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fad7415aa40 0x7fad60003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:07.826 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.821+0000 7fad78fff640 1 --2- 172.21.15.27:0/3311670202 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fad600040d0 0x7fad60004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:07.826 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.821+0000 7fad78fff640 1 --2- 172.21.15.27:0/3311670202 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fad60004d40 0x7fad600051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:07.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.821+0000 7fad78fff640 1 -- 172.21.15.27:0/3311670202 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fad740b3ee0 con 0x7fad60004d40 2024-09-12T03:22:07.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.821+0000 7fad78fff640 1 -- 172.21.15.27:0/3311670202 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fad6000bb50 con 0x7fad600040d0 2024-09-12T03:22:07.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.821+0000 7fad78fff640 1 -- 172.21.15.27:0/3311670202 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fad6000df30 con 0x7fad7415aa40 2024-09-12T03:22:07.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.821+0000 7fad7a802640 1 --2- 172.21.15.27:0/3311670202 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fad60004d40 0x7fad600051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:07.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.821+0000 7fad7a001640 1 --2- 172.21.15.27:0/3311670202 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fad7415aa40 0x7fad60003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:07.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.821+0000 7fad79800640 1 --2- 172.21.15.27:0/3311670202 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fad600040d0 0x7fad60004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:07.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.821+0000 7fad7a802640 1 -- 172.21.15.27:0/3311670202 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fad7415aa40 msgr2=0x7fad60003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:07.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.821+0000 7fad7a802640 1 --2- 172.21.15.27:0/3311670202 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fad7415aa40 0x7fad60003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:07.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.821+0000 7fad7a802640 1 -- 172.21.15.27:0/3311670202 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fad600040d0 msgr2=0x7fad60004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:07.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.821+0000 7fad7a802640 1 --2- 172.21.15.27:0/3311670202 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fad600040d0 0x7fad60004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:07.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.821+0000 7fad7a802640 1 -- 172.21.15.27:0/3311670202 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad60003520 con 0x7fad60004d40 2024-09-12T03:22:07.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.821+0000 7fad79800640 1 --2- 172.21.15.27:0/3311670202 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fad600040d0 0x7fad60004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:22:07.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.822+0000 7fad7a802640 1 --2- 172.21.15.27:0/3311670202 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fad60004d40 0x7fad600051c0 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7fad6c001fd0 tx=0x7fad60005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27cddb90d8dd1565 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:07.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.822+0000 7fad78fff640 1 -- 172.21.15.27:0/3311670202 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad6c001bf0 con 0x7fad60004d40 2024-09-12T03:22:07.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.822+0000 7fad78fff640 1 -- 172.21.15.27:0/3311670202 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad6c003b70 con 0x7fad60004d40 2024-09-12T03:22:07.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.822+0000 7fad78fff640 1 -- 172.21.15.27:0/3311670202 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad6c0050d0 con 0x7fad60004d40 2024-09-12T03:22:07.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.823+0000 7fad7c265640 1 -- 172.21.15.27:0/3311670202 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fad60004d40 msgr2=0x7fad600051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:07.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.823+0000 7fad7c265640 1 --2- 172.21.15.27:0/3311670202 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fad60004d40 0x7fad600051c0 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7fad6c001fd0 tx=0x7fad60005830 comp rx=0 tx=0).stop 2024-09-12T03:22:07.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.823+0000 7fad7a802640 1 -- 172.21.15.27:0/3311670202 reap_dead start 2024-09-12T03:22:07.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.823+0000 7fad7c265640 1 -- 172.21.15.27:0/3311670202 shutdown_connections 2024-09-12T03:22:07.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.823+0000 7fad7c265640 1 -- 172.21.15.27:0/3311670202 >> 172.21.15.27:0/3311670202 conn(0x7fad7406ca30 msgr2=0x7fad7406ce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:07.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.823+0000 7fad7c265640 1 -- 172.21.15.27:0/3311670202 shutdown_connections 2024-09-12T03:22:07.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.823+0000 7fad7c265640 1 -- 172.21.15.27:0/3311670202 wait complete. 2024-09-12T03:22:07.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.824+0000 7fad7c265640 1 Processor -- start 2024-09-12T03:22:07.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.825+0000 7fad7c265640 1 -- start start 2024-09-12T03:22:07.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.825+0000 7fad7c265640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fad60004d40 0x7fad740c7da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:07.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.825+0000 7fad7c265640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fad74154900 0x7fad740c5e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:07.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.825+0000 7fad7c265640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fad740c9650 0x7fad740cec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:07.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.825+0000 7fad7c265640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fad7406f330 con 0x7fad740c9650 2024-09-12T03:22:07.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.825+0000 7fad7c265640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fad740b44a0 con 0x7fad74154900 2024-09-12T03:22:07.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.825+0000 7fad7c265640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fad740cb4a0 con 0x7fad60004d40 2024-09-12T03:22:07.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.825+0000 7fad7a802640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fad740c9650 0x7fad740cec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:07.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.825+0000 7fad7a802640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fad740c9650 0x7fad740cec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42642/0 (socket says 172.21.15.27:42642) 2024-09-12T03:22:07.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.825+0000 7fad7a802640 1 -- 172.21.15.27:0/1496073520 learned_addr learned my addr 172.21.15.27:0/1496073520 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:22:07.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.826+0000 7fad79800640 1 --2- 172.21.15.27:0/1496073520 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fad74154900 0x7fad740c5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:07.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.826+0000 7fad7a001640 1 --2- 172.21.15.27:0/1496073520 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fad60004d40 0x7fad740c7da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:07.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.826+0000 7fad7a802640 1 -- 172.21.15.27:0/1496073520 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fad60004d40 msgr2=0x7fad740c7da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:07.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.826+0000 7fad7a802640 1 --2- 172.21.15.27:0/1496073520 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fad60004d40 0x7fad740c7da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:07.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.826+0000 7fad7a802640 1 -- 172.21.15.27:0/1496073520 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fad74154900 msgr2=0x7fad740c5e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:07.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.826+0000 7fad7a802640 1 --2- 172.21.15.27:0/1496073520 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fad74154900 0x7fad740c5e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:07.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.826+0000 7fad7a802640 1 -- 172.21.15.27:0/1496073520 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad740c82e0 con 0x7fad740c9650 2024-09-12T03:22:07.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.826+0000 7fad7a802640 1 --2- 172.21.15.27:0/1496073520 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fad740c9650 0x7fad740cec20 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7fad6c005040 tx=0x7fad6c001650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:07.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.826+0000 7fad6a7fc640 1 -- 172.21.15.27:0/1496073520 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad6c001bf0 con 0x7fad740c9650 2024-09-12T03:22:07.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.826+0000 7fad6a7fc640 1 -- 172.21.15.27:0/1496073520 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad6c043070 con 0x7fad740c9650 2024-09-12T03:22:07.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.826+0000 7fad7c265640 1 -- 172.21.15.27:0/1496073520 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad740cb6d0 con 0x7fad740c9650 2024-09-12T03:22:07.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.826+0000 7fad6a7fc640 1 -- 172.21.15.27:0/1496073520 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad6c005610 con 0x7fad740c9650 2024-09-12T03:22:07.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.826+0000 7fad7c265640 1 -- 172.21.15.27:0/1496073520 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad740cbc90 con 0x7fad740c9650 2024-09-12T03:22:07.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.827+0000 7fad7c265640 1 -- 172.21.15.27:0/1496073520 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad7406e660 con 0x7fad740c9650 2024-09-12T03:22:07.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.828+0000 7fad6a7fc640 1 -- 172.21.15.27:0/1496073520 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 21) ==== 91008+0+0 (secure 0 0 0) 0x7fad6c03a040 con 0x7fad740c9650 2024-09-12T03:22:07.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.828+0000 7fad6a7fc640 1 --2- 172.21.15.27:0/1496073520 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fad5806c3d0 0x7fad5806e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:07.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.828+0000 7fad6a7fc640 1 -- 172.21.15.27:0/1496073520 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 5793+0+0 (secure 0 0 0) 0x7fad6c0abd90 con 0x7fad740c9650 2024-09-12T03:22:07.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.828+0000 7fad7a001640 1 --2- 172.21.15.27:0/1496073520 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fad5806c3d0 0x7fad5806e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:07.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.829+0000 7fad7a001640 1 --2- 172.21.15.27:0/1496073520 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fad5806c3d0 0x7fad5806e890 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fad70009bc0 tx=0x7fad70009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:07.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:07.833+0000 7fad6a7fc640 1 -- 172.21.15.27:0/1496073520 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fad6c079f00 con 0x7fad740c9650 2024-09-12T03:22:08.011 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:08.010+0000 7fad7c265640 1 -- 172.21.15.27:0/1496073520 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fad741582d0 con 0x7fad740c9650 2024-09-12T03:22:08.012 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:08.011+0000 7fad6a7fc640 1 -- 172.21.15.27:0/1496073520 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v115) ==== 74+0+16847 (secure 0 0 0) 0x7fad6c079800 con 0x7fad740c9650 2024-09-12T03:22:08.012 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:22:08.013 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":115,"fsid":"d2df504e-2647-4385-a5db-9fdf69f487a0","created":"2024-09-12T03:16:07.778937+0000","modified":"2024-09-12T03:22:07.338033+0000","last_up_change":"2024-09-12T03:16:19.314778+0000","last_in_change":"2024-09-12T03:16:09.893086+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T03:16:20.097646+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-09-12T03:21:44.263535+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-09-12T03:21:45.593648+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"115","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":5,"snap_epoch":115,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1851116c-1fb3-4dc9-9772-f90e2ef4c355","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6817","nonce":3002321488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6819","nonce":3002321488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6823","nonce":3002321488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6821","nonce":3002321488}]},"public_addr":"172.21.15.27:6817/3002321488","cluster_addr":"172.21.15.27:6819/3002321488","heartbeat_back_addr":"172.21.15.27:6823/3002321488","heartbeat_front_addr":"172.21.15.27:6821/3002321488","state":["exists","up"]},{"osd":1,"uuid":"f38ae17d-73eb-4204-b99d-975fa806fb87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6809","nonce":1613102886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6811","nonce":1613102886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6815","nonce":1613102886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6813","nonce":1613102886}]},"public_addr":"172.21.15.27:6809/1613102886","cluster_addr":"172.21.15.27:6811/1613102886","heartbeat_back_addr":"172.21.15.27:6815/1613102886","heartbeat_front_addr":"172.21.15.27:6813/1613102886","state":["exists","up"]},{"osd":2,"uuid":"fb12dcc8-bdd0-42b8-b46d-8b10106224bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6825","nonce":3534723355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6827","nonce":3534723355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6831","nonce":3534723355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6829","nonce":3534723355}]},"public_addr":"172.21.15.27:6825/3534723355","cluster_addr":"172.21.15.27:6827/3534723355","heartbeat_back_addr":"172.21.15.27:6831/3534723355","heartbeat_front_addr":"172.21.15.27:6829/3534723355","state":["exists","up"]},{"osd":3,"uuid":"d74d3f49-9c24-4b9f-98e2-3ecd7b5ef3b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6801","nonce":80661598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6803","nonce":80661598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6807","nonce":80661598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6805","nonce":80661598}]},"public_addr":"172.21.15.27:6801/80661598","cluster_addr":"172.21.15.27:6803/80661598","heartbeat_back_addr":"172.21.15.27:6807/80661598","heartbeat_front_addr":"172.21.15.27:6805/80661598","state":["exists","up"]},{"osd":4,"uuid":"2bf123f2-1b0d-41c4-aa3b-805bab267e59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6817","nonce":3943189827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6819","nonce":3943189827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6823","nonce":3943189827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6821","nonce":3943189827}]},"public_addr":"172.21.15.45:6817/3943189827","cluster_addr":"172.21.15.45:6819/3943189827","heartbeat_back_addr":"172.21.15.45:6823/3943189827","heartbeat_front_addr":"172.21.15.45:6821/3943189827","state":["exists","up"]},{"osd":5,"uuid":"c49ba5da-91c6-4a86-a97b-4aba89c253ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6809","nonce":3620760330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6811","nonce":3620760330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6815","nonce":3620760330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6813","nonce":3620760330}]},"public_addr":"172.21.15.45:6809/3620760330","cluster_addr":"172.21.15.45:6811/3620760330","heartbeat_back_addr":"172.21.15.45:6815/3620760330","heartbeat_front_addr":"172.21.15.45:6813/3620760330","state":["exists","up"]},{"osd":6,"uuid":"b0123475-89f7-4a34-9e33-d797b99dd9dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6801","nonce":1989083510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6803","nonce":1989083510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6807","nonce":1989083510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6805","nonce":1989083510}]},"public_addr":"172.21.15.45:6801/1989083510","cluster_addr":"172.21.15.45:6803/1989083510","heartbeat_back_addr":"172.21.15.45:6807/1989083510","heartbeat_front_addr":"172.21.15.45:6805/1989083510","state":["exists","up"]},{"osd":7,"uuid":"4dad773d-b7d8-46fc-8a10-a972aad27b29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6825","nonce":1513300294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6827","nonce":1513300294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6831","nonce":1513300294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6829","nonce":1513300294}]},"public_addr":"172.21.15.45:6825/1513300294","cluster_addr":"172.21.15.45:6827/1513300294","heartbeat_back_addr":"172.21.15.45:6831/1513300294","heartbeat_front_addr":"172.21.15.45:6829/1513300294","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.881855+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:17.133252+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.735339+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.500742+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.477571+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:17.167667+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.470291+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.873743+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]},{"pool":19,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[{"pool":19,"snaps":[{"begin":2,"length":4}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T03:22:08.015 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:08.013+0000 7fad7c265640 1 -- 172.21.15.27:0/1496073520 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fad5806c3d0 msgr2=0x7fad5806e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:08.015 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:08.013+0000 7fad7c265640 1 --2- 172.21.15.27:0/1496073520 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fad5806c3d0 0x7fad5806e890 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fad70009bc0 tx=0x7fad70009190 comp rx=0 tx=0).stop 2024-09-12T03:22:08.015 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:08.013+0000 7fad7c265640 1 -- 172.21.15.27:0/1496073520 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fad740c9650 msgr2=0x7fad740cec20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:08.015 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:08.013+0000 7fad7c265640 1 --2- 172.21.15.27:0/1496073520 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fad740c9650 0x7fad740cec20 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7fad6c005040 tx=0x7fad6c001650 comp rx=0 tx=0).stop 2024-09-12T03:22:08.015 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:08.013+0000 7fad7c265640 1 -- 172.21.15.27:0/1496073520 shutdown_connections 2024-09-12T03:22:08.015 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:08.013+0000 7fad7c265640 1 --2- 172.21.15.27:0/1496073520 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fad5806c3d0 0x7fad5806e890 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:08.015 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:08.013+0000 7fad7c265640 1 --2- 172.21.15.27:0/1496073520 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fad740c9650 0x7fad740cec20 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:08.015 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:08.013+0000 7fad7c265640 1 --2- 172.21.15.27:0/1496073520 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fad74154900 0x7fad740c5e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:08.015 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:08.014+0000 7fad7c265640 1 --2- 172.21.15.27:0/1496073520 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fad60004d40 0x7fad740c7da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:08.015 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:08.014+0000 7fad7c265640 1 -- 172.21.15.27:0/1496073520 >> 172.21.15.27:0/1496073520 conn(0x7fad7406ca30 msgr2=0x7fad741557e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:08.016 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:08.014+0000 7fad7c265640 1 -- 172.21.15.27:0/1496073520 shutdown_connections 2024-09-12T03:22:08.016 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:08.014+0000 7fad7c265640 1 -- 172.21.15.27:0/1496073520 wait complete. 2024-09-12T03:22:08.026 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-12T03:22:08.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:08.131+0000 7fafba998640 1 Processor -- start 2024-09-12T03:22:08.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:08.131+0000 7fafba998640 1 -- start start 2024-09-12T03:22:08.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:08.132+0000 7fafba998640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fafb4145c30 0x7fafb415ca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:08.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:08.132+0000 7fafba998640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fafb40cda40 0x7fafb40b4040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:08.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:08.132+0000 7fafba998640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fafb40b4d70 con 0x7fafb4146010 2024-09-12T03:22:08.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:08.132+0000 7fafba998640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7fafb40b5050 con 0x7fafb4145c30 2024-09-12T03:22:08.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:08.132+0000 7fafba998640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fafb406d140 con 0x7fafb40cda40 2024-09-12T03:22:08.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:08.132+0000 7fafb3fff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fafb4145c30 0x7fafb415ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:08.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:08.132+0000 7fafb37fe640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fafb4146010 0x7fafb40cd330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:51314/0 (socket says 172.21.15.27:51314) 2024-09-12T03:22:08.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:08.132+0000 7fafb37fe640 1 -- 172.21.15.27:0/1960061783 learned_addr learned my addr 172.21.15.27:0/1960061783 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:22:08.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:08.133+0000 7fafb2ffd640 1 -- 172.21.15.27:0/1960061783 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 20382903 0 0) 0x7fafb40b4d70 con 0x7fafb4146010 2024-09-12T03:22:08.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:08.133+0000 7fafb2ffd640 1 -- 172.21.15.27:0/1960061783 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7faf9c003520 con 0x7fafb4146010 2024-09-12T03:22:08.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:08.133+0000 7fafb8f35640 1 --2- 172.21.15.27:0/1960061783 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fafb40cda40 0x7fafb40b4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:08.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:08.133+0000 7fafb3fff640 1 -- 172.21.15.27:0/1960061783 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fafb40cda40 msgr2=0x7fafb40b4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:08.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:08.133+0000 7fafb3fff640 1 --2- 172.21.15.27:0/1960061783 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fafb40cda40 0x7fafb40b4040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:08.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:08.133+0000 7fafb3fff640 1 -- 172.21.15.27:0/1960061783 >> v1:172.21.15.27:6789/0 conn(0x7fafb4146010 legacy=0x7fafb40cd330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:08.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:08.133+0000 7fafb3fff640 1 -- 172.21.15.27:0/1960061783 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafb406d420 con 0x7fafb4145c30 2024-09-12T03:22:08.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:08.133+0000 7fafb8f35640 1 --2- 172.21.15.27:0/1960061783 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fafb40cda40 0x7fafb40b4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:22:08.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:08.133+0000 7fafb3fff640 1 --2- 172.21.15.27:0/1960061783 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fafb4145c30 0x7fafb415ca00 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7fafa40097a0 tx=0x7fafa402a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f4bb2ee48722632 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:08.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:08.134+0000 7fafb2ffd640 1 -- 172.21.15.27:0/1960061783 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafa4037070 con 0x7fafb4145c30 2024-09-12T03:22:08.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:08.134+0000 7fafb2ffd640 1 -- 172.21.15.27:0/1960061783 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fafa402ae50 con 0x7fafb4145c30 2024-09-12T03:22:08.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:08.134+0000 7fafb2ffd640 1 -- 172.21.15.27:0/1960061783 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafa4032660 con 0x7fafb4145c30 2024-09-12T03:22:08.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:08.134+0000 7fafba998640 1 -- 172.21.15.27:0/1960061783 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fafb4145c30 msgr2=0x7fafb415ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:08.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:08.134+0000 7fafba998640 1 --2- 172.21.15.27:0/1960061783 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fafb4145c30 0x7fafb415ca00 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7fafa40097a0 tx=0x7fafa402a100 comp rx=0 tx=0).stop 2024-09-12T03:22:08.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:08.134+0000 7fafba998640 1 -- 172.21.15.27:0/1960061783 shutdown_connections 2024-09-12T03:22:08.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:08.134+0000 7fafba998640 1 --2- 172.21.15.27:0/1960061783 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fafb40cda40 0x7fafb40b4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:08.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:08.134+0000 7fafba998640 1 --2- 172.21.15.27:0/1960061783 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fafb4145c30 0x7fafb415ca00 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:08.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:08.134+0000 7fafba998640 1 -- 172.21.15.27:0/1960061783 >> 172.21.15.27:0/1960061783 conn(0x7fafb4062c60 msgr2=0x7fafb406bb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:08.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:08.134+0000 7fafba998640 1 -- 172.21.15.27:0/1960061783 shutdown_connections 2024-09-12T03:22:08.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:08.134+0000 7fafba998640 1 -- 172.21.15.27:0/1960061783 wait complete. 2024-09-12T03:22:08.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:08.135+0000 7fafba998640 1 Processor -- start 2024-09-12T03:22:08.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:08.136+0000 7fafba998640 1 -- start start 2024-09-12T03:22:08.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:08.136+0000 7fafba998640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fafb40cda40 0x7fafb40acf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:08.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:08.136+0000 7fafba998640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fafb4145c30 0x7fafb40aef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:08.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:08.136+0000 7fafba998640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fafb4146010 0x7fafb41e4a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:08.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:08.136+0000 7fafba998640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fafb40b4700 con 0x7fafb4145c30 2024-09-12T03:22:08.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:08.136+0000 7fafba998640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fafb40b4580 con 0x7fafb4146010 2024-09-12T03:22:08.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:08.136+0000 7fafba998640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fafb40b4880 con 0x7fafb40cda40 2024-09-12T03:22:08.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:08.137+0000 7fafb37fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fafb4145c30 0x7fafb40aef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:08.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:08.137+0000 7fafb37fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fafb4145c30 0x7fafb40aef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42662/0 (socket says 172.21.15.27:42662) 2024-09-12T03:22:08.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:08.137+0000 7fafb37fe640 1 -- 172.21.15.27:0/3815324776 learned_addr learned my addr 172.21.15.27:0/3815324776 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:22:08.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:08.137+0000 7fafb3fff640 1 --2- 172.21.15.27:0/3815324776 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fafb40cda40 0x7fafb40acf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:08.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:08.137+0000 7fafb8f35640 1 --2- 172.21.15.27:0/3815324776 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fafb4146010 0x7fafb41e4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:08.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:08.137+0000 7fafb37fe640 1 -- 172.21.15.27:0/3815324776 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fafb40cda40 msgr2=0x7fafb40acf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:08.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:08.137+0000 7fafb37fe640 1 --2- 172.21.15.27:0/3815324776 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fafb40cda40 0x7fafb40acf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:08.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:08.137+0000 7fafb37fe640 1 -- 172.21.15.27:0/3815324776 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fafb4146010 msgr2=0x7fafb41e4a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:08.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:08.137+0000 7fafb37fe640 1 --2- 172.21.15.27:0/3815324776 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fafb4146010 0x7fafb41e4a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:08.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:08.137+0000 7fafb37fe640 1 -- 172.21.15.27:0/3815324776 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafb40ad870 con 0x7fafb4145c30 2024-09-12T03:22:08.142 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:08.137+0000 7fafb8f35640 1 --2- 172.21.15.27:0/3815324776 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fafb4146010 0x7fafb41e4a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:22:08.142 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:08.137+0000 7fafb37fe640 1 --2- 172.21.15.27:0/3815324776 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fafb4145c30 0x7fafb40aef00 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7fafa800c390 tx=0x7fafa800c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:08.142 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:08.137+0000 7fafb0ff9640 1 -- 172.21.15.27:0/3815324776 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafa8017070 con 0x7fafb4145c30 2024-09-12T03:22:08.142 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:08.137+0000 7fafb0ff9640 1 -- 172.21.15.27:0/3815324776 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fafa8005ce0 con 0x7fafb4145c30 2024-09-12T03:22:08.142 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:08.137+0000 7fafba998640 1 -- 172.21.15.27:0/3815324776 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafb40adb50 con 0x7fafb4145c30 2024-09-12T03:22:08.142 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:08.137+0000 7fafb0ff9640 1 -- 172.21.15.27:0/3815324776 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafa80072f0 con 0x7fafb4145c30 2024-09-12T03:22:08.142 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:08.137+0000 7fafba998640 1 -- 172.21.15.27:0/3815324776 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafb40ae110 con 0x7fafb4145c30 2024-09-12T03:22:08.142 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:08.138+0000 7fafba998640 1 -- 172.21.15.27:0/3815324776 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fafb40b4040 con 0x7fafb4145c30 2024-09-12T03:22:08.142 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:08.139+0000 7fafb0ff9640 1 -- 172.21.15.27:0/3815324776 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 21) ==== 91008+0+0 (secure 0 0 0) 0x7fafa801a070 con 0x7fafb4145c30 2024-09-12T03:22:08.143 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:08.139+0000 7fafb0ff9640 1 --2- 172.21.15.27:0/3815324776 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fafa006c3d0 0x7fafa006e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:08.143 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:08.140+0000 7fafb3fff640 1 --2- 172.21.15.27:0/3815324776 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fafa006c3d0 0x7fafa006e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:08.143 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:08.140+0000 7fafb0ff9640 1 -- 172.21.15.27:0/3815324776 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 5793+0+0 (secure 0 0 0) 0x7fafa808ad20 con 0x7fafb4145c30 2024-09-12T03:22:08.143 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:08.140+0000 7fafb3fff640 1 --2- 172.21.15.27:0/3815324776 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fafa006c3d0 0x7fafa006e890 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fafa4034000 tx=0x7fafa402e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:08.144 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:08.144+0000 7fafb0ff9640 1 -- 172.21.15.27:0/3815324776 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fafa8058e00 con 0x7fafb4145c30 2024-09-12T03:22:08.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:08.356+0000 7fafba998640 1 -- 172.21.15.27:0/3815324776 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fafb40b1e40 con 0x7fafb4145c30 2024-09-12T03:22:08.358 INFO:tasks.ceph.mon.a.smithi027.stderr:2024-09-12T03:22:08.357+0000 7f959bfd9640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-12T03:22:08.359 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:08.358+0000 7fafb0ff9640 1 -- 172.21.15.27:0/3815324776 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v69) ==== 161+0+0 (secure 0 0 0) 0x7fafa8058700 con 0x7fafb4145c30 2024-09-12T03:22:08.359 INFO:teuthology.orchestra.run.smithi027.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-12T03:22:08.363 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:08.361+0000 7fafba998640 1 -- 172.21.15.27:0/3815324776 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fafa006c3d0 msgr2=0x7fafa006e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:08.363 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:08.361+0000 7fafba998640 1 --2- 172.21.15.27:0/3815324776 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fafa006c3d0 0x7fafa006e890 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fafa4034000 tx=0x7fafa402e000 comp rx=0 tx=0).stop 2024-09-12T03:22:08.363 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:08.361+0000 7fafba998640 1 -- 172.21.15.27:0/3815324776 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fafb4145c30 msgr2=0x7fafb40aef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:08.363 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:08.361+0000 7fafba998640 1 --2- 172.21.15.27:0/3815324776 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fafb4145c30 0x7fafb40aef00 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7fafa800c390 tx=0x7fafa800c850 comp rx=0 tx=0).stop 2024-09-12T03:22:08.363 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:08.361+0000 7fafba998640 1 -- 172.21.15.27:0/3815324776 shutdown_connections 2024-09-12T03:22:08.363 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:08.361+0000 7fafba998640 1 --2- 172.21.15.27:0/3815324776 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fafb4146010 0x7fafb41e4a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:08.363 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:08.361+0000 7fafba998640 1 --2- 172.21.15.27:0/3815324776 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fafa006c3d0 0x7fafa006e890 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:08.363 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:08.361+0000 7fafba998640 1 --2- 172.21.15.27:0/3815324776 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fafb4145c30 0x7fafb40aef00 unknown :-1 s=CLOSED pgs=1175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:08.363 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:08.361+0000 7fafba998640 1 --2- 172.21.15.27:0/3815324776 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fafb40cda40 0x7fafb40acf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:08.364 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:08.361+0000 7fafba998640 1 -- 172.21.15.27:0/3815324776 >> 172.21.15.27:0/3815324776 conn(0x7fafb4062c60 msgr2=0x7fafb413e8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:08.364 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:08.362+0000 7fafba998640 1 -- 172.21.15.27:0/3815324776 shutdown_connections 2024-09-12T03:22:08.364 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:08.362+0000 7fafba998640 1 -- 172.21.15.27:0/3815324776 wait complete. 2024-09-12T03:22:08.375 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-12T03:22:08.401 INFO:tasks.ceph.mds.a.smithi027.stdout:starting mds.a at 2024-09-12T03:22:08.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:08.481+0000 7ffb2aaa6640 1 Processor -- start 2024-09-12T03:22:08.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:08.481+0000 7ffb2aaa6640 1 -- start start 2024-09-12T03:22:08.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:08.482+0000 7ffb2aaa6640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7ffb2415e530 0x7ffb240aff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:08.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:08.482+0000 7ffb2aaa6640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ffb2415e910 0x7ffb240b4710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:08.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:08.482+0000 7ffb2aaa6640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffb2406dc20 con 0x7ffb240aeb80 2024-09-12T03:22:08.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:08.482+0000 7ffb2aaa6640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7ffb240b07e0 con 0x7ffb2415e530 2024-09-12T03:22:08.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:08.482+0000 7ffb2aaa6640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7ffb240b4c50 con 0x7ffb2415e910 2024-09-12T03:22:08.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:08.482+0000 7ffb28842640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7ffb2415e530 0x7ffb240aff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:08.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:08.482+0000 7ffb29043640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7ffb240aeb80 0x7ffb240722f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:51318/0 (socket says 172.21.15.27:51318) 2024-09-12T03:22:08.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:08.482+0000 7ffb28842640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7ffb2415e530 0x7ffb240aff20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42698/0 (socket says 172.21.15.27:42698) 2024-09-12T03:22:08.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:08.482+0000 7ffb29043640 1 -- 172.21.15.27:0/2247480488 learned_addr learned my addr 172.21.15.27:0/2247480488 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:22:08.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:08.483+0000 7ffb28842640 1 -- 172.21.15.27:0/2247480488 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ffb2415e910 msgr2=0x7ffb240b4710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:08.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:08.483+0000 7ffb28842640 1 --2- 172.21.15.27:0/2247480488 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ffb2415e910 0x7ffb240b4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:08.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:08.483+0000 7ffb28842640 1 -- 172.21.15.27:0/2247480488 >> v1:172.21.15.27:6789/0 conn(0x7ffb240aeb80 legacy=0x7ffb240722f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:08.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:08.483+0000 7ffb237fe640 1 -- 172.21.15.27:0/2247480488 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 758070886 0 0) 0x7ffb2406dc20 con 0x7ffb240aeb80 2024-09-12T03:22:08.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:08.483+0000 7ffb28842640 1 -- 172.21.15.27:0/2247480488 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb240b0500 con 0x7ffb2415e530 2024-09-12T03:22:08.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:08.483+0000 7ffb28842640 1 --2- 172.21.15.27:0/2247480488 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffb2415e530 0x7ffb240aff20 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7ffb140097a0 tx=0x7ffb1402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1077250bba65ea5d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:08.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:08.483+0000 7ffb237fe640 1 -- 172.21.15.27:0/2247480488 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb14037070 con 0x7ffb2415e530 2024-09-12T03:22:08.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:08.483+0000 7ffb237fe640 1 -- 172.21.15.27:0/2247480488 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb14032440 con 0x7ffb2415e530 2024-09-12T03:22:08.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:08.483+0000 7ffb237fe640 1 -- 172.21.15.27:0/2247480488 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb14032810 con 0x7ffb2415e530 2024-09-12T03:22:08.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:08.484+0000 7ffb2aaa6640 1 -- 172.21.15.27:0/2247480488 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffb2415e530 msgr2=0x7ffb240aff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:08.490 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:08.484+0000 7ffb2aaa6640 1 --2- 172.21.15.27:0/2247480488 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffb2415e530 0x7ffb240aff20 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7ffb140097a0 tx=0x7ffb1402a180 comp rx=0 tx=0).stop 2024-09-12T03:22:08.490 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:08.484+0000 7ffb2aaa6640 1 -- 172.21.15.27:0/2247480488 shutdown_connections 2024-09-12T03:22:08.490 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:08.484+0000 7ffb2aaa6640 1 --2- 172.21.15.27:0/2247480488 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ffb2415e910 0x7ffb240b4710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:08.490 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:08.484+0000 7ffb2aaa6640 1 --2- 172.21.15.27:0/2247480488 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffb2415e530 0x7ffb240aff20 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:08.490 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:08.484+0000 7ffb2aaa6640 1 -- 172.21.15.27:0/2247480488 >> 172.21.15.27:0/2247480488 conn(0x7ffb2406bb30 msgr2=0x7ffb2406bf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:08.490 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:08.484+0000 7ffb2aaa6640 1 -- 172.21.15.27:0/2247480488 shutdown_connections 2024-09-12T03:22:08.490 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:08.484+0000 7ffb2aaa6640 1 -- 172.21.15.27:0/2247480488 wait complete. 2024-09-12T03:22:08.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:08.485+0000 7ffb2aaa6640 1 Processor -- start 2024-09-12T03:22:08.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:08.485+0000 7ffb2aaa6640 1 -- start start 2024-09-12T03:22:08.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:08.486+0000 7ffb2aaa6640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffb240aeb80 0x7ffb2415b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:08.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:08.486+0000 7ffb2aaa6640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffb2415e530 0x7ffb241e8e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:08.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:08.486+0000 7ffb2aaa6640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ffb2415e910 0x7ffb241e9340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:08.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:08.486+0000 7ffb2aaa6640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7ffb24071ff0 con 0x7ffb2415e530 2024-09-12T03:22:08.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:08.486+0000 7ffb2aaa6640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ffb2406e1a0 con 0x7ffb240aeb80 2024-09-12T03:22:08.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:08.486+0000 7ffb2aaa6640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7ffb24072a00 con 0x7ffb2415e910 2024-09-12T03:22:08.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:08.486+0000 7ffb23fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffb2415e530 0x7ffb241e8e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:08.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:08.486+0000 7ffb23fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffb2415e530 0x7ffb241e8e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42700/0 (socket says 172.21.15.27:42700) 2024-09-12T03:22:08.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:08.486+0000 7ffb23fff640 1 -- 172.21.15.27:0/1516633035 learned_addr learned my addr 172.21.15.27:0/1516633035 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:22:08.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:08.487+0000 7ffb29043640 1 --2- 172.21.15.27:0/1516633035 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ffb2415e910 0x7ffb241e9340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:08.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:08.487+0000 7ffb28842640 1 --2- 172.21.15.27:0/1516633035 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffb240aeb80 0x7ffb2415b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:08.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:08.487+0000 7ffb23fff640 1 -- 172.21.15.27:0/1516633035 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ffb2415e910 msgr2=0x7ffb241e9340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:08.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:08.487+0000 7ffb23fff640 1 --2- 172.21.15.27:0/1516633035 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ffb2415e910 0x7ffb241e9340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:08.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:08.487+0000 7ffb23fff640 1 -- 172.21.15.27:0/1516633035 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffb240aeb80 msgr2=0x7ffb2415b640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:08.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:08.487+0000 7ffb23fff640 1 --2- 172.21.15.27:0/1516633035 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffb240aeb80 0x7ffb2415b640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:08.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:08.487+0000 7ffb23fff640 1 -- 172.21.15.27:0/1516633035 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb241509b0 con 0x7ffb2415e530 2024-09-12T03:22:08.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:08.487+0000 7ffb23fff640 1 --2- 172.21.15.27:0/1516633035 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffb2415e530 0x7ffb241e8e00 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7ffb1800b440 tx=0x7ffb1800b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:08.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:08.487+0000 7ffb217fa640 1 -- 172.21.15.27:0/1516633035 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb18015020 con 0x7ffb2415e530 2024-09-12T03:22:08.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:08.487+0000 7ffb217fa640 1 -- 172.21.15.27:0/1516633035 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb18011890 con 0x7ffb2415e530 2024-09-12T03:22:08.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:08.487+0000 7ffb217fa640 1 -- 172.21.15.27:0/1516633035 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb18011c80 con 0x7ffb2415e530 2024-09-12T03:22:08.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:08.487+0000 7ffb2aaa6640 1 -- 172.21.15.27:0/1516633035 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb241e8320 con 0x7ffb2415e530 2024-09-12T03:22:08.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:08.487+0000 7ffb2aaa6640 1 -- 172.21.15.27:0/1516633035 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb2415bb80 con 0x7ffb2415e530 2024-09-12T03:22:08.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:08.488+0000 7ffb2aaa6640 1 -- 172.21.15.27:0/1516633035 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb24073040 con 0x7ffb2415e530 2024-09-12T03:22:08.493 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:08.489+0000 7ffb217fa640 1 -- 172.21.15.27:0/1516633035 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 21) ==== 91008+0+0 (secure 0 0 0) 0x7ffb18018020 con 0x7ffb2415e530 2024-09-12T03:22:08.493 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:08.489+0000 7ffb217fa640 1 --2- 172.21.15.27:0/1516633035 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7ffb1006c400 0x7ffb1006e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:08.493 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:08.489+0000 7ffb217fa640 1 -- 172.21.15.27:0/1516633035 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5851+0+0 (secure 0 0 0) 0x7ffb1808ba90 con 0x7ffb2415e530 2024-09-12T03:22:08.493 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:08.489+0000 7ffb28842640 1 --2- 172.21.15.27:0/1516633035 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7ffb1006c400 0x7ffb1006e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:08.493 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:08.490+0000 7ffb28842640 1 --2- 172.21.15.27:0/1516633035 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7ffb1006c400 0x7ffb1006e8c0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ffb14034000 tx=0x7ffb1402e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:08.494 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:08.493+0000 7ffb217fa640 1 -- 172.21.15.27:0/1516633035 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7ffb24073040 con 0x7ffb2415e530 2024-09-12T03:22:08.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:08.704+0000 7ffb2aaa6640 1 -- 172.21.15.27:0/1516633035 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7ffb2406d030 con 0x7ffb2415e530 2024-09-12T03:22:09.364 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:09.363+0000 7ffb217fa640 1 -- 172.21.15.27:0/1516633035 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v70) ==== 104+0+0 (secure 0 0 0) 0x7ffb1801f020 con 0x7ffb2415e530 2024-09-12T03:22:09.367 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:09.366+0000 7ffb2aaa6640 1 -- 172.21.15.27:0/1516633035 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7ffb1006c400 msgr2=0x7ffb1006e8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:09.367 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:09.366+0000 7ffb2aaa6640 1 --2- 172.21.15.27:0/1516633035 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7ffb1006c400 0x7ffb1006e8c0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ffb14034000 tx=0x7ffb1402e000 comp rx=0 tx=0).stop 2024-09-12T03:22:09.367 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:09.366+0000 7ffb2aaa6640 1 -- 172.21.15.27:0/1516633035 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffb2415e530 msgr2=0x7ffb241e8e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:09.367 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:09.366+0000 7ffb2aaa6640 1 --2- 172.21.15.27:0/1516633035 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffb2415e530 0x7ffb241e8e00 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7ffb1800b440 tx=0x7ffb1800b900 comp rx=0 tx=0).stop 2024-09-12T03:22:09.368 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:09.366+0000 7ffb2aaa6640 1 -- 172.21.15.27:0/1516633035 shutdown_connections 2024-09-12T03:22:09.368 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:09.366+0000 7ffb2aaa6640 1 --2- 172.21.15.27:0/1516633035 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ffb2415e910 0x7ffb241e9340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:09.368 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:09.366+0000 7ffb2aaa6640 1 --2- 172.21.15.27:0/1516633035 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7ffb1006c400 0x7ffb1006e8c0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:09.368 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:09.366+0000 7ffb2aaa6640 1 --2- 172.21.15.27:0/1516633035 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffb2415e530 0x7ffb241e8e00 unknown :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:09.368 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:09.366+0000 7ffb2aaa6640 1 --2- 172.21.15.27:0/1516633035 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffb240aeb80 0x7ffb2415b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:09.368 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:09.366+0000 7ffb2aaa6640 1 -- 172.21.15.27:0/1516633035 >> 172.21.15.27:0/1516633035 conn(0x7ffb2406bb30 msgr2=0x7ffb240cd7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:09.368 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:09.366+0000 7ffb2aaa6640 1 -- 172.21.15.27:0/1516633035 shutdown_connections 2024-09-12T03:22:09.369 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:09.366+0000 7ffb2aaa6640 1 -- 172.21.15.27:0/1516633035 wait complete. 2024-09-12T03:22:09.380 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-12T03:22:09.532 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:09.531+0000 7f47e6e42640 1 Processor -- start 2024-09-12T03:22:09.532 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:09.531+0000 7f47e6e42640 1 -- start start 2024-09-12T03:22:09.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:09.531+0000 7f47e6e42640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f47e0162040 0x7f47e00d1da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:09.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:09.531+0000 7f47e6e42640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f47e00d3790 0x7f47e00b7a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:09.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:09.531+0000 7f47e6e42640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f47e006d8c0 0x7f47e006dd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:09.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:09.531+0000 7f47e6e42640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f47e00b7f90 con 0x7f47e006d8c0 2024-09-12T03:22:09.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:09.531+0000 7f47e6e42640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f47e00d2e30 con 0x7f47e0162040 2024-09-12T03:22:09.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:09.531+0000 7f47e6e42640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f47e0162420 con 0x7f47e00d3790 2024-09-12T03:22:09.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:09.532+0000 7f47e53df640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f47e006d8c0 0x7f47e006dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:09.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:09.532+0000 7f47e53df640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f47e006d8c0 0x7f47e006dd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:40890/0 (socket says 172.21.15.27:40890) 2024-09-12T03:22:09.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:09.532+0000 7f47e53df640 1 -- 172.21.15.27:0/2229501565 learned_addr learned my addr 172.21.15.27:0/2229501565 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:22:09.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:09.532+0000 7f47e4bde640 1 --2- 172.21.15.27:0/2229501565 >> v2:172.21.15.45:3300/0 conn(0x7f47e0162040 0x7f47e00d1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:09.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:09.532+0000 7f47dffff640 1 --2- 172.21.15.27:0/2229501565 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f47e00d3790 0x7f47e00b7a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:09.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:09.532+0000 7f47e53df640 1 -- 172.21.15.27:0/2229501565 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f47e00d3790 msgr2=0x7f47e00b7a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:09.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:09.532+0000 7f47e53df640 1 --2- 172.21.15.27:0/2229501565 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f47e00d3790 0x7f47e00b7a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:09.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:09.532+0000 7f47e53df640 1 -- 172.21.15.27:0/2229501565 >> v2:172.21.15.45:3300/0 conn(0x7f47e0162040 msgr2=0x7f47e00d1da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:09.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:09.532+0000 7f47e53df640 1 --2- 172.21.15.27:0/2229501565 >> v2:172.21.15.45:3300/0 conn(0x7f47e0162040 0x7f47e00d1da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:09.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:09.532+0000 7f47e53df640 1 -- 172.21.15.27:0/2229501565 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47e00d30d0 con 0x7f47e006d8c0 2024-09-12T03:22:09.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:09.533+0000 7f47e53df640 1 --2- 172.21.15.27:0/2229501565 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f47e006d8c0 0x7f47e006dd50 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f47e00d17b0 tx=0x7f47d802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea4656e5b6d05148 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:09.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:09.533+0000 7f47df7fe640 1 -- 172.21.15.27:0/2229501565 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47d8036070 con 0x7f47e006d8c0 2024-09-12T03:22:09.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:09.533+0000 7f47df7fe640 1 -- 172.21.15.27:0/2229501565 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47d8031600 con 0x7f47e006d8c0 2024-09-12T03:22:09.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:09.533+0000 7f47df7fe640 1 -- 172.21.15.27:0/2229501565 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47d80319d0 con 0x7f47e006d8c0 2024-09-12T03:22:09.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:09.533+0000 7f47e6e42640 1 -- 172.21.15.27:0/2229501565 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f47e006d8c0 msgr2=0x7f47e006dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:09.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:09.533+0000 7f47e6e42640 1 --2- 172.21.15.27:0/2229501565 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f47e006d8c0 0x7f47e006dd50 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f47e00d17b0 tx=0x7f47d802a6a0 comp rx=0 tx=0).stop 2024-09-12T03:22:09.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:09.533+0000 7f47e6e42640 1 -- 172.21.15.27:0/2229501565 shutdown_connections 2024-09-12T03:22:09.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:09.533+0000 7f47e6e42640 1 --2- 172.21.15.27:0/2229501565 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f47e006d8c0 0x7f47e006dd50 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:09.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:09.533+0000 7f47e6e42640 1 --2- 172.21.15.27:0/2229501565 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f47e00d3790 0x7f47e00b7a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:09.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:09.533+0000 7f47e6e42640 1 --2- 172.21.15.27:0/2229501565 >> v2:172.21.15.45:3300/0 conn(0x7f47e0162040 0x7f47e00d1da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:09.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:09.533+0000 7f47e6e42640 1 -- 172.21.15.27:0/2229501565 >> 172.21.15.27:0/2229501565 conn(0x7f47e00b6700 msgr2=0x7f47e00b6b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:09.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:09.533+0000 7f47e6e42640 1 -- 172.21.15.27:0/2229501565 shutdown_connections 2024-09-12T03:22:09.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:09.533+0000 7f47e6e42640 1 -- 172.21.15.27:0/2229501565 wait complete. 2024-09-12T03:22:09.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:09.535+0000 7f47e6e42640 1 Processor -- start 2024-09-12T03:22:09.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:09.535+0000 7f47e6e42640 1 -- start start 2024-09-12T03:22:09.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:09.535+0000 7f47e6e42640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f47e006d8c0 0x7f47e01506c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:09.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:09.535+0000 7f47e6e42640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f47e00d3790 0x7f47e01611e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:09.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:09.535+0000 7f47e6e42640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f47e0162040 0x7f47e0152080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:09.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:09.535+0000 7f47e6e42640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f47e00d4530 con 0x7f47e006d8c0 2024-09-12T03:22:09.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:09.535+0000 7f47e6e42640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f47e00cee00 con 0x7f47e0162040 2024-09-12T03:22:09.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:09.535+0000 7f47e6e42640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f47e00d4150 con 0x7f47e00d3790 2024-09-12T03:22:09.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:09.536+0000 7f47e4bde640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f47e006d8c0 0x7f47e01506c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:09.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:09.536+0000 7f47e4bde640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f47e006d8c0 0x7f47e01506c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:40898/0 (socket says 172.21.15.27:40898) 2024-09-12T03:22:09.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:09.536+0000 7f47e4bde640 1 -- 172.21.15.27:0/191855870 learned_addr learned my addr 172.21.15.27:0/191855870 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:22:09.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:09.536+0000 7f47dffff640 1 --2- 172.21.15.27:0/191855870 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f47e00d3790 0x7f47e01611e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:09.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:09.536+0000 7f47e53df640 1 --2- 172.21.15.27:0/191855870 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f47e0162040 0x7f47e0152080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:09.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:09.536+0000 7f47e4bde640 1 -- 172.21.15.27:0/191855870 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f47e00d3790 msgr2=0x7f47e01611e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:09.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:09.536+0000 7f47e4bde640 1 --2- 172.21.15.27:0/191855870 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f47e00d3790 0x7f47e01611e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:09.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:09.536+0000 7f47e4bde640 1 -- 172.21.15.27:0/191855870 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f47e0162040 msgr2=0x7f47e0152080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:09.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:09.536+0000 7f47e4bde640 1 --2- 172.21.15.27:0/191855870 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f47e0162040 0x7f47e0152080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:09.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:09.536+0000 7f47e4bde640 1 -- 172.21.15.27:0/191855870 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47e015eb40 con 0x7f47e006d8c0 2024-09-12T03:22:09.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:09.536+0000 7f47e4bde640 1 --2- 172.21.15.27:0/191855870 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f47e006d8c0 0x7f47e01506c0 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f47d400d800 tx=0x7f47d400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:09.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:09.536+0000 7f47dd7fa640 1 -- 172.21.15.27:0/191855870 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47d400b6c0 con 0x7f47e006d8c0 2024-09-12T03:22:09.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:09.536+0000 7f47dd7fa640 1 -- 172.21.15.27:0/191855870 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47d4019070 con 0x7f47e006d8c0 2024-09-12T03:22:09.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:09.536+0000 7f47dd7fa640 1 -- 172.21.15.27:0/191855870 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47d400ba80 con 0x7f47e006d8c0 2024-09-12T03:22:09.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:09.536+0000 7f47e6e42640 1 -- 172.21.15.27:0/191855870 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47e015ee20 con 0x7f47e006d8c0 2024-09-12T03:22:09.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:09.536+0000 7f47e6e42640 1 -- 172.21.15.27:0/191855870 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47e015f3e0 con 0x7f47e006d8c0 2024-09-12T03:22:09.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:09.537+0000 7f47e6e42640 1 -- 172.21.15.27:0/191855870 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47e00b7a50 con 0x7f47e006d8c0 2024-09-12T03:22:09.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:09.538+0000 7f47dd7fa640 1 -- 172.21.15.27:0/191855870 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 21) ==== 91008+0+0 (secure 0 0 0) 0x7f47d400bc20 con 0x7f47e006d8c0 2024-09-12T03:22:09.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:09.538+0000 7f47dd7fa640 1 --2- 172.21.15.27:0/191855870 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f47c406c3d0 0x7f47c406e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:09.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:09.538+0000 7f47dd7fa640 1 -- 172.21.15.27:0/191855870 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5851+0+0 (secure 0 0 0) 0x7f47d408bd70 con 0x7f47e006d8c0 2024-09-12T03:22:09.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:09.538+0000 7f47dffff640 1 --2- 172.21.15.27:0/191855870 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f47c406c3d0 0x7f47c406e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:09.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:09.539+0000 7f47dffff640 1 --2- 172.21.15.27:0/191855870 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f47c406c3d0 0x7f47c406e890 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f47e015b3f0 tx=0x7f47cc008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:09.543 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:09.543+0000 7f47dd7fa640 1 -- 172.21.15.27:0/191855870 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f47d401c360 con 0x7f47e006d8c0 2024-09-12T03:22:09.718 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:09.718+0000 7f47e6e42640 1 -- 172.21.15.27:0/191855870 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f47e006e6e0 con 0x7f47e006d8c0 2024-09-12T03:22:10.367 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:10.365+0000 7f47dd7fa640 1 -- 172.21.15.27:0/191855870 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v117) ==== 181+0+0 (secure 0 0 0) 0x7f47d4059eb0 con 0x7f47e006d8c0 2024-09-12T03:22:10.367 INFO:teuthology.orchestra.run.smithi027.stderr:pool 'cephfs_metadata' removed 2024-09-12T03:22:10.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:10.368+0000 7f47e6e42640 1 -- 172.21.15.27:0/191855870 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f47c406c3d0 msgr2=0x7f47c406e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:10.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:10.368+0000 7f47e6e42640 1 --2- 172.21.15.27:0/191855870 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f47c406c3d0 0x7f47c406e890 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f47e015b3f0 tx=0x7f47cc008040 comp rx=0 tx=0).stop 2024-09-12T03:22:10.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:10.368+0000 7f47e6e42640 1 -- 172.21.15.27:0/191855870 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f47e006d8c0 msgr2=0x7f47e01506c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:10.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:10.368+0000 7f47e6e42640 1 --2- 172.21.15.27:0/191855870 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f47e006d8c0 0x7f47e01506c0 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f47d400d800 tx=0x7f47d400dcc0 comp rx=0 tx=0).stop 2024-09-12T03:22:10.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:10.369+0000 7f47e6e42640 1 -- 172.21.15.27:0/191855870 shutdown_connections 2024-09-12T03:22:10.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:10.369+0000 7f47e6e42640 1 --2- 172.21.15.27:0/191855870 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f47e0162040 0x7f47e0152080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:10.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:10.369+0000 7f47e6e42640 1 --2- 172.21.15.27:0/191855870 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f47e00d3790 0x7f47e01611e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:10.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:10.369+0000 7f47e6e42640 1 --2- 172.21.15.27:0/191855870 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f47c406c3d0 0x7f47c406e890 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:10.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:10.369+0000 7f47e6e42640 1 --2- 172.21.15.27:0/191855870 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f47e006d8c0 0x7f47e01506c0 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:10.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:10.369+0000 7f47e6e42640 1 -- 172.21.15.27:0/191855870 >> 172.21.15.27:0/191855870 conn(0x7f47e00b6700 msgr2=0x7f47e00cf320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:10.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:10.369+0000 7f47e6e42640 1 -- 172.21.15.27:0/191855870 shutdown_connections 2024-09-12T03:22:10.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:10.369+0000 7f47e6e42640 1 -- 172.21.15.27:0/191855870 wait complete. 2024-09-12T03:22:10.385 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-12T03:22:10.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:10.497+0000 7fd508e77640 1 Processor -- start 2024-09-12T03:22:10.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:10.497+0000 7fd508e77640 1 -- start start 2024-09-12T03:22:10.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:10.498+0000 7fd508e77640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fd5040d3790 0x7fd5040d1da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:10.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:10.498+0000 7fd508e77640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd5040d2e30 con 0x7fd50406dfd0 2024-09-12T03:22:10.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:10.498+0000 7fd508e77640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd504162420 con 0x7fd504162040 2024-09-12T03:22:10.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:10.498+0000 7fd508e77640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7fd5040d30d0 con 0x7fd5040d3790 2024-09-12T03:22:10.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:10.499+0000 7fd502d9d640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fd5040d3790 0x7fd5040d1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:10.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:10.499+0000 7fd501d9b640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fd50406dfd0 0x7fd5040cf320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:35060/0 (socket says 172.21.15.27:35060) 2024-09-12T03:22:10.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:10.499+0000 7fd502d9d640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fd5040d3790 0x7fd5040d1da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:40908/0 (socket says 172.21.15.27:40908) 2024-09-12T03:22:10.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:10.499+0000 7fd501d9b640 1 -- 172.21.15.27:0/3242315705 learned_addr learned my addr 172.21.15.27:0/3242315705 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:22:10.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:10.499+0000 7fd502d9d640 1 -- 172.21.15.27:0/3242315705 >> v1:172.21.15.45:6789/0 conn(0x7fd504162040 legacy=0x7fd50406d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:10.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:10.499+0000 7fd502d9d640 1 -- 172.21.15.27:0/3242315705 >> v1:172.21.15.27:6789/0 conn(0x7fd50406dfd0 legacy=0x7fd5040cf320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:10.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:10.499+0000 7fd502d9d640 1 -- 172.21.15.27:0/3242315705 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5040d33b0 con 0x7fd5040d3790 2024-09-12T03:22:10.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:10.499+0000 7fd50159a640 1 -- 172.21.15.27:0/3242315705 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2000121409 0 0) 0x7fd5040d2e30 con 0x7fd50406dfd0 2024-09-12T03:22:10.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:10.500+0000 7fd502d9d640 1 --2- 172.21.15.27:0/3242315705 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd5040d3790 0x7fd5040d1da0 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7fd4f40097a0 tx=0x7fd4f402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acc074a2eb53de82 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:10.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:10.500+0000 7fd50159a640 1 -- 172.21.15.27:0/3242315705 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4f4037070 con 0x7fd5040d3790 2024-09-12T03:22:10.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:10.500+0000 7fd50159a640 1 -- 172.21.15.27:0/3242315705 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4f4032440 con 0x7fd5040d3790 2024-09-12T03:22:10.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:10.500+0000 7fd50159a640 1 -- 172.21.15.27:0/3242315705 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4f4032810 con 0x7fd5040d3790 2024-09-12T03:22:10.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:10.500+0000 7fd508e77640 1 -- 172.21.15.27:0/3242315705 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd5040d3790 msgr2=0x7fd5040d1da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:10.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:10.500+0000 7fd508e77640 1 --2- 172.21.15.27:0/3242315705 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd5040d3790 0x7fd5040d1da0 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7fd4f40097a0 tx=0x7fd4f402a230 comp rx=0 tx=0).stop 2024-09-12T03:22:10.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:10.500+0000 7fd508e77640 1 -- 172.21.15.27:0/3242315705 shutdown_connections 2024-09-12T03:22:10.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:10.500+0000 7fd508e77640 1 --2- 172.21.15.27:0/3242315705 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd5040d3790 0x7fd5040d1da0 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:10.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:10.500+0000 7fd508e77640 1 -- 172.21.15.27:0/3242315705 >> 172.21.15.27:0/3242315705 conn(0x7fd5040b6700 msgr2=0x7fd5040b6b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:10.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:10.500+0000 7fd508e77640 1 -- 172.21.15.27:0/3242315705 shutdown_connections 2024-09-12T03:22:10.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:10.500+0000 7fd508e77640 1 -- 172.21.15.27:0/3242315705 wait complete. 2024-09-12T03:22:10.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:10.502+0000 7fd508e77640 1 Processor -- start 2024-09-12T03:22:10.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:10.502+0000 7fd508e77640 1 -- start start 2024-09-12T03:22:10.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:10.502+0000 7fd508e77640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fd50406dfd0 0x7fd5040b8da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:10.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:10.503+0000 7fd508e77640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd5040d3790 0x7fd504158c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:10.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:10.503+0000 7fd508e77640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd504162040 0x7fd504159160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:10.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:10.503+0000 7fd508e77640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fd5040d4150 con 0x7fd5040d3790 2024-09-12T03:22:10.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:10.503+0000 7fd508e77640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fd5040d4530 con 0x7fd504162040 2024-09-12T03:22:10.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:10.503+0000 7fd508e77640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fd5040d42d0 con 0x7fd50406dfd0 2024-09-12T03:22:10.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:10.503+0000 7fd501d9b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd5040d3790 0x7fd504158c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:10.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:10.503+0000 7fd501d9b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd5040d3790 0x7fd504158c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:40910/0 (socket says 172.21.15.27:40910) 2024-09-12T03:22:10.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:10.503+0000 7fd501d9b640 1 -- 172.21.15.27:0/2666249475 learned_addr learned my addr 172.21.15.27:0/2666249475 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:22:10.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:10.503+0000 7fd502d9d640 1 --2- 172.21.15.27:0/2666249475 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd504162040 0x7fd504159160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:10.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:10.503+0000 7fd50259c640 1 --2- 172.21.15.27:0/2666249475 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fd50406dfd0 0x7fd5040b8da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:10.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:10.503+0000 7fd501d9b640 1 -- 172.21.15.27:0/2666249475 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fd50406dfd0 msgr2=0x7fd5040b8da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:10.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:10.503+0000 7fd501d9b640 1 --2- 172.21.15.27:0/2666249475 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fd50406dfd0 0x7fd5040b8da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:10.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:10.503+0000 7fd501d9b640 1 -- 172.21.15.27:0/2666249475 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd504162040 msgr2=0x7fd504159160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:10.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:10.503+0000 7fd501d9b640 1 --2- 172.21.15.27:0/2666249475 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd504162040 0x7fd504159160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:10.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:10.503+0000 7fd501d9b640 1 -- 172.21.15.27:0/2666249475 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd504157e80 con 0x7fd5040d3790 2024-09-12T03:22:10.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:10.503+0000 7fd501d9b640 1 --2- 172.21.15.27:0/2666249475 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd5040d3790 0x7fd504158c20 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7fd4ec00c4d0 tx=0x7fd4ec00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:10.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:10.503+0000 7fd4faffd640 1 -- 172.21.15.27:0/2666249475 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4ec018070 con 0x7fd5040d3790 2024-09-12T03:22:10.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:10.503+0000 7fd4faffd640 1 -- 172.21.15.27:0/2666249475 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4ec0074e0 con 0x7fd5040d3790 2024-09-12T03:22:10.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:10.503+0000 7fd508e77640 1 -- 172.21.15.27:0/2666249475 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd504159800 con 0x7fd5040d3790 2024-09-12T03:22:10.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:10.503+0000 7fd4faffd640 1 -- 172.21.15.27:0/2666249475 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4ec004a20 con 0x7fd5040d3790 2024-09-12T03:22:10.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:10.503+0000 7fd508e77640 1 -- 172.21.15.27:0/2666249475 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5040cd430 con 0x7fd5040d3790 2024-09-12T03:22:10.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:10.504+0000 7fd508e77640 1 -- 172.21.15.27:0/2666249475 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd50406e6e0 con 0x7fd5040d3790 2024-09-12T03:22:10.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:10.505+0000 7fd4faffd640 1 -- 172.21.15.27:0/2666249475 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 21) ==== 91008+0+0 (secure 0 0 0) 0x7fd4ec01b070 con 0x7fd5040d3790 2024-09-12T03:22:10.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:10.505+0000 7fd4faffd640 1 --2- 172.21.15.27:0/2666249475 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fd4e406c3d0 0x7fd4e406e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:10.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:10.505+0000 7fd4faffd640 1 -- 172.21.15.27:0/2666249475 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 5413+0+0 (secure 0 0 0) 0x7fd4ec08a940 con 0x7fd5040d3790 2024-09-12T03:22:10.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:10.506+0000 7fd50259c640 1 --2- 172.21.15.27:0/2666249475 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fd4e406c3d0 0x7fd4e406e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:10.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:10.506+0000 7fd50259c640 1 --2- 172.21.15.27:0/2666249475 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fd4e406c3d0 0x7fd4e406e890 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fd4fc0012e0 tx=0x7fd4fc009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:10.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:10.510+0000 7fd4faffd640 1 -- 172.21.15.27:0/2666249475 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fd4ec0585e0 con 0x7fd5040d3790 2024-09-12T03:22:10.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:10.684+0000 7fd508e77640 1 -- 172.21.15.27:0/2666249475 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fd50415ee20 con 0x7fd5040d3790 2024-09-12T03:22:11.371 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:11.370+0000 7fd4faffd640 1 -- 172.21.15.27:0/2666249475 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v118) ==== 169+0+0 (secure 0 0 0) 0x7fd4ec05c8e0 con 0x7fd5040d3790 2024-09-12T03:22:11.371 INFO:teuthology.orchestra.run.smithi027.stderr:pool 'cephfs_data' removed 2024-09-12T03:22:11.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:11.372+0000 7fd508e77640 1 -- 172.21.15.27:0/2666249475 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fd4e406c3d0 msgr2=0x7fd4e406e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:11.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:11.372+0000 7fd508e77640 1 --2- 172.21.15.27:0/2666249475 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fd4e406c3d0 0x7fd4e406e890 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fd4fc0012e0 tx=0x7fd4fc009040 comp rx=0 tx=0).stop 2024-09-12T03:22:11.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:11.372+0000 7fd508e77640 1 -- 172.21.15.27:0/2666249475 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd5040d3790 msgr2=0x7fd504158c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:11.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:11.372+0000 7fd508e77640 1 --2- 172.21.15.27:0/2666249475 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd5040d3790 0x7fd504158c20 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7fd4ec00c4d0 tx=0x7fd4ec00c990 comp rx=0 tx=0).stop 2024-09-12T03:22:11.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:11.372+0000 7fd508e77640 1 -- 172.21.15.27:0/2666249475 shutdown_connections 2024-09-12T03:22:11.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:11.372+0000 7fd508e77640 1 --2- 172.21.15.27:0/2666249475 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd504162040 0x7fd504159160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:11.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:11.372+0000 7fd508e77640 1 --2- 172.21.15.27:0/2666249475 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fd4e406c3d0 0x7fd4e406e890 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:11.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:11.372+0000 7fd508e77640 1 --2- 172.21.15.27:0/2666249475 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd5040d3790 0x7fd504158c20 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:11.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:11.372+0000 7fd508e77640 1 --2- 172.21.15.27:0/2666249475 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fd50406dfd0 0x7fd5040b8da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:11.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:11.372+0000 7fd508e77640 1 -- 172.21.15.27:0/2666249475 >> 172.21.15.27:0/2666249475 conn(0x7fd5040b6700 msgr2=0x7fd50406d8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:11.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:11.372+0000 7fd508e77640 1 -- 172.21.15.27:0/2666249475 shutdown_connections 2024-09-12T03:22:11.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:11.372+0000 7fd508e77640 1 -- 172.21.15.27:0/2666249475 wait complete. 2024-09-12T03:22:11.387 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-09-12T03:22:11.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:11.492+0000 7f079b186640 1 Processor -- start 2024-09-12T03:22:11.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:11.492+0000 7f079b186640 1 -- start start 2024-09-12T03:22:11.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:11.493+0000 7f079b186640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f079415a270 0x7f079406cff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:11.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:11.493+0000 7f079b186640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f079406d530 0x7f07940cf410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:11.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:11.493+0000 7f079b186640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f07940ce650 con 0x7f079406c650 2024-09-12T03:22:11.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:11.493+0000 7f079b186640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f07940cef80 con 0x7f079406d530 2024-09-12T03:22:11.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:11.493+0000 7f079b186640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f07940ce930 con 0x7f079415a270 2024-09-12T03:22:11.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:11.493+0000 7f0793fff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f079406d530 0x7f07940cf410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:11.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:11.494+0000 7f0793fff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f079406d530 0x7f07940cf410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:40912/0 (socket says 172.21.15.27:40912) 2024-09-12T03:22:11.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:11.494+0000 7f0793fff640 1 -- 172.21.15.27:0/965063347 learned_addr learned my addr 172.21.15.27:0/965063347 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:22:11.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:11.494+0000 7f0798f22640 1 --2- 172.21.15.27:0/965063347 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f079415a270 0x7f079406cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:11.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:11.494+0000 7f0793fff640 1 -- 172.21.15.27:0/965063347 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f079415a270 msgr2=0x7f079406cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:11.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:11.494+0000 7f0793fff640 1 --2- 172.21.15.27:0/965063347 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f079415a270 0x7f079406cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:11.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:11.494+0000 7f0793fff640 1 -- 172.21.15.27:0/965063347 >> v1:172.21.15.45:6789/0 conn(0x7f079406c650 legacy=0x7f07940726a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:11.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:11.494+0000 7f0793fff640 1 -- 172.21.15.27:0/965063347 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07940cec10 con 0x7f079406d530 2024-09-12T03:22:11.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:11.495+0000 7f0793fff640 1 --2- 172.21.15.27:0/965063347 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f079406d530 0x7f07940cf410 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f0784009e50 tx=0x7f078402a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e2151aa426c3b58 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:11.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:11.495+0000 7f07937fe640 1 -- 172.21.15.27:0/965063347 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0784036070 con 0x7f079406d530 2024-09-12T03:22:11.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:11.495+0000 7f07937fe640 1 -- 172.21.15.27:0/965063347 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0784031600 con 0x7f079406d530 2024-09-12T03:22:11.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:11.495+0000 7f07937fe640 1 -- 172.21.15.27:0/965063347 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07840319d0 con 0x7f079406d530 2024-09-12T03:22:11.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:11.495+0000 7f079b186640 1 -- 172.21.15.27:0/965063347 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f079406d530 msgr2=0x7f07940cf410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:11.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:11.495+0000 7f079b186640 1 --2- 172.21.15.27:0/965063347 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f079406d530 0x7f07940cf410 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f0784009e50 tx=0x7f078402a710 comp rx=0 tx=0).stop 2024-09-12T03:22:11.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:11.495+0000 7f079b186640 1 -- 172.21.15.27:0/965063347 shutdown_connections 2024-09-12T03:22:11.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:11.495+0000 7f079b186640 1 --2- 172.21.15.27:0/965063347 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f079406d530 0x7f07940cf410 unknown :-1 s=CLOSED pgs=1186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:11.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:11.495+0000 7f079b186640 1 --2- 172.21.15.27:0/965063347 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f079415a270 0x7f079406cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:11.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:11.495+0000 7f079b186640 1 -- 172.21.15.27:0/965063347 >> 172.21.15.27:0/965063347 conn(0x7f079406bb30 msgr2=0x7f079406bf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:11.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:11.495+0000 7f079b186640 1 -- 172.21.15.27:0/965063347 shutdown_connections 2024-09-12T03:22:11.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:11.495+0000 7f079b186640 1 -- 172.21.15.27:0/965063347 wait complete. 2024-09-12T03:22:11.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:11.497+0000 7f079b186640 1 Processor -- start 2024-09-12T03:22:11.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:11.497+0000 7f079b186640 1 -- start start 2024-09-12T03:22:11.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:11.498+0000 7f079b186640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f079406c650 0x7f0794148b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:11.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:11.498+0000 7f079b186640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f079406d530 0x7f0794144de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:11.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:11.498+0000 7f079b186640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f079415a270 0x7f07940b1020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:11.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:11.498+0000 7f079b186640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f0794072f30 con 0x7f079406d530 2024-09-12T03:22:11.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:11.498+0000 7f079b186640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f0794072db0 con 0x7f079406c650 2024-09-12T03:22:11.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:11.498+0000 7f079b186640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f07940730b0 con 0x7f079415a270 2024-09-12T03:22:11.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:11.498+0000 7f0793fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f079406d530 0x7f0794144de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:11.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:11.498+0000 7f0793fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f079406d530 0x7f0794144de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:40924/0 (socket says 172.21.15.27:40924) 2024-09-12T03:22:11.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:11.498+0000 7f0793fff640 1 -- 172.21.15.27:0/1487130136 learned_addr learned my addr 172.21.15.27:0/1487130136 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:22:11.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:11.498+0000 7f0798f22640 1 --2- 172.21.15.27:0/1487130136 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f079406c650 0x7f0794148b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:11.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:11.498+0000 7f0799723640 1 --2- 172.21.15.27:0/1487130136 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f079415a270 0x7f07940b1020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:11.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:11.498+0000 7f0793fff640 1 -- 172.21.15.27:0/1487130136 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f079415a270 msgr2=0x7f07940b1020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:11.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:11.498+0000 7f0793fff640 1 --2- 172.21.15.27:0/1487130136 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f079415a270 0x7f07940b1020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:11.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:11.498+0000 7f0793fff640 1 -- 172.21.15.27:0/1487130136 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f079406c650 msgr2=0x7f0794148b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:11.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:11.498+0000 7f0793fff640 1 --2- 172.21.15.27:0/1487130136 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f079406c650 0x7f0794148b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:11.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:11.498+0000 7f0793fff640 1 -- 172.21.15.27:0/1487130136 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07940b3a60 con 0x7f079406d530 2024-09-12T03:22:11.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:11.498+0000 7f0793fff640 1 --2- 172.21.15.27:0/1487130136 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f079406d530 0x7f0794144de0 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f0784000f80 tx=0x7f0784004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:11.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:11.498+0000 7f07917fa640 1 -- 172.21.15.27:0/1487130136 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0784036050 con 0x7f079406d530 2024-09-12T03:22:11.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:11.499+0000 7f07917fa640 1 -- 172.21.15.27:0/1487130136 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0784042070 con 0x7f079406d530 2024-09-12T03:22:11.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:11.499+0000 7f079b186640 1 -- 172.21.15.27:0/1487130136 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07940b3d40 con 0x7f079406d530 2024-09-12T03:22:11.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:11.499+0000 7f07917fa640 1 -- 172.21.15.27:0/1487130136 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0784009510 con 0x7f079406d530 2024-09-12T03:22:11.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:11.499+0000 7f079b186640 1 -- 172.21.15.27:0/1487130136 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07940b4300 con 0x7f079406d530 2024-09-12T03:22:11.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:11.499+0000 7f079b186640 1 -- 172.21.15.27:0/1487130136 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07940ccdb0 con 0x7f079406d530 2024-09-12T03:22:11.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:11.500+0000 7f07917fa640 1 -- 172.21.15.27:0/1487130136 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 21) ==== 91008+0+0 (secure 0 0 0) 0x7f0784007c70 con 0x7f079406d530 2024-09-12T03:22:11.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:11.500+0000 7f07917fa640 1 --2- 172.21.15.27:0/1487130136 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f077406c3d0 0x7f077406e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:11.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:11.501+0000 7f07917fa640 1 -- 172.21.15.27:0/1487130136 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 4999+0+0 (secure 0 0 0) 0x7f07840ac3c0 con 0x7f079406d530 2024-09-12T03:22:11.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:11.501+0000 7f0798f22640 1 --2- 172.21.15.27:0/1487130136 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f077406c3d0 0x7f077406e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:11.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:11.501+0000 7f0798f22640 1 --2- 172.21.15.27:0/1487130136 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f077406c3d0 0x7f077406e890 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f078c006fd0 tx=0x7f078c008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:11.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:11.505+0000 7f07917fa640 1 -- 172.21.15.27:0/1487130136 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f078407a0d0 con 0x7f079406d530 2024-09-12T03:22:11.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:11.683+0000 7f079b186640 1 -- 172.21.15.27:0/1487130136 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f0794149050 con 0x7f079406d530 2024-09-12T03:22:11.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:11.686+0000 7f07917fa640 1 -- 172.21.15.27:0/1487130136 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v23) ==== 104+0+0 (secure 0 0 0) 0x7f078407e3d0 con 0x7f079406d530 2024-09-12T03:22:11.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:11.688+0000 7f079b186640 1 -- 172.21.15.27:0/1487130136 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f077406c3d0 msgr2=0x7f077406e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:11.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:11.688+0000 7f079b186640 1 --2- 172.21.15.27:0/1487130136 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f077406c3d0 0x7f077406e890 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f078c006fd0 tx=0x7f078c008040 comp rx=0 tx=0).stop 2024-09-12T03:22:11.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:11.689+0000 7f079b186640 1 -- 172.21.15.27:0/1487130136 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f079406d530 msgr2=0x7f0794144de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:11.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:11.689+0000 7f079b186640 1 --2- 172.21.15.27:0/1487130136 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f079406d530 0x7f0794144de0 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f0784000f80 tx=0x7f0784004a40 comp rx=0 tx=0).stop 2024-09-12T03:22:11.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:11.689+0000 7f079b186640 1 -- 172.21.15.27:0/1487130136 shutdown_connections 2024-09-12T03:22:11.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:11.689+0000 7f079b186640 1 --2- 172.21.15.27:0/1487130136 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f079415a270 0x7f07940b1020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:11.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:11.689+0000 7f079b186640 1 --2- 172.21.15.27:0/1487130136 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f077406c3d0 0x7f077406e890 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:11.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:11.689+0000 7f079b186640 1 --2- 172.21.15.27:0/1487130136 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f079406d530 0x7f0794144de0 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:11.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:11.689+0000 7f079b186640 1 --2- 172.21.15.27:0/1487130136 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f079406c650 0x7f0794148b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:11.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:11.689+0000 7f079b186640 1 -- 172.21.15.27:0/1487130136 >> 172.21.15.27:0/1487130136 conn(0x7f079406bb30 msgr2=0x7f079415af10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:11.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:11.689+0000 7f079b186640 1 -- 172.21.15.27:0/1487130136 shutdown_connections 2024-09-12T03:22:11.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:11.689+0000 7f079b186640 1 -- 172.21.15.27:0/1487130136 wait complete. 2024-09-12T03:22:11.702 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_no_upgrade_v1_to_v2' 2024-09-12T03:22:11.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:11.803+0000 7f1d5cb86640 1 Processor -- start 2024-09-12T03:22:11.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:11.804+0000 7f1d5cb86640 1 -- start start 2024-09-12T03:22:11.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:11.804+0000 7f1d5cb86640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f1d5815eb90 0x7f1d580d2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:11.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:11.804+0000 7f1d5cb86640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1d580d3760 con 0x7f1d580d1720 2024-09-12T03:22:11.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:11.804+0000 7f1d5cb86640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1d580d3a40 con 0x7f1d580d5200 2024-09-12T03:22:11.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:11.804+0000 7f1d5cb86640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f1d580b7d20 con 0x7f1d5815eb90 2024-09-12T03:22:11.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:11.805+0000 7f1d56d9d640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f1d580d1720 0x7f1d580b8190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:35072/0 (socket says 172.21.15.27:35072) 2024-09-12T03:22:11.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:11.805+0000 7f1d5659c640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f1d5815eb90 0x7f1d580d2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:11.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:11.805+0000 7f1d56d9d640 1 -- 172.21.15.27:0/1535922564 learned_addr learned my addr 172.21.15.27:0/1535922564 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:22:11.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:11.805+0000 7f1d5559a640 1 -- 172.21.15.27:0/1535922564 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 788326846 0 0) 0x7f1d580d3760 con 0x7f1d580d1720 2024-09-12T03:22:11.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:11.805+0000 7f1d5559a640 1 -- 172.21.15.27:0/1535922564 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1d44003520 con 0x7f1d580d1720 2024-09-12T03:22:11.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:11.805+0000 7f1d5659c640 1 -- 172.21.15.27:0/1535922564 >> v1:172.21.15.45:6789/0 conn(0x7f1d580d5200 legacy=0x7f1d580d1010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:11.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:11.805+0000 7f1d5659c640 1 -- 172.21.15.27:0/1535922564 >> v1:172.21.15.27:6789/0 conn(0x7f1d580d1720 legacy=0x7f1d580b8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:11.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:11.805+0000 7f1d5659c640 1 -- 172.21.15.27:0/1535922564 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d580d3d20 con 0x7f1d5815eb90 2024-09-12T03:22:11.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:11.806+0000 7f1d5659c640 1 --2- 172.21.15.27:0/1535922564 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1d5815eb90 0x7f1d580d2df0 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f1d50009d90 tx=0x7f1d5002a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d158907ed5913cc server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:11.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:11.806+0000 7f1d5559a640 1 -- 172.21.15.27:0/1535922564 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d50007740 con 0x7f1d5815eb90 2024-09-12T03:22:11.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:11.806+0000 7f1d5559a640 1 -- 172.21.15.27:0/1535922564 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d500078e0 con 0x7f1d5815eb90 2024-09-12T03:22:11.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:11.806+0000 7f1d5559a640 1 -- 172.21.15.27:0/1535922564 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d50007cb0 con 0x7f1d5815eb90 2024-09-12T03:22:11.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:11.806+0000 7f1d5cb86640 1 -- 172.21.15.27:0/1535922564 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1d5815eb90 msgr2=0x7f1d580d2df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:11.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:11.806+0000 7f1d5cb86640 1 --2- 172.21.15.27:0/1535922564 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1d5815eb90 0x7f1d580d2df0 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f1d50009d90 tx=0x7f1d5002a6f0 comp rx=0 tx=0).stop 2024-09-12T03:22:11.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:11.807+0000 7f1d5cb86640 1 -- 172.21.15.27:0/1535922564 shutdown_connections 2024-09-12T03:22:11.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:11.807+0000 7f1d5cb86640 1 --2- 172.21.15.27:0/1535922564 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1d5815eb90 0x7f1d580d2df0 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:11.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:11.807+0000 7f1d5cb86640 1 -- 172.21.15.27:0/1535922564 >> 172.21.15.27:0/1535922564 conn(0x7f1d5806dce0 msgr2=0x7f1d5806e0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:11.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:11.807+0000 7f1d5cb86640 1 -- 172.21.15.27:0/1535922564 shutdown_connections 2024-09-12T03:22:11.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:11.807+0000 7f1d5cb86640 1 -- 172.21.15.27:0/1535922564 wait complete. 2024-09-12T03:22:11.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:11.808+0000 7f1d5cb86640 1 Processor -- start 2024-09-12T03:22:11.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:11.808+0000 7f1d5cb86640 1 -- start start 2024-09-12T03:22:11.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:11.809+0000 7f1d5cb86640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1d580d1720 0x7f1d5814da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:11.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:11.809+0000 7f1d5cb86640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1d580d5200 0x7f1d581595d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:11.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:11.809+0000 7f1d5cb86640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1d5815eb90 0x7f1d580c9f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:11.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:11.809+0000 7f1d5cb86640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f1d580d3330 con 0x7f1d580d1720 2024-09-12T03:22:11.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:11.809+0000 7f1d5cb86640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f1d580d42e0 con 0x7f1d580d5200 2024-09-12T03:22:11.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:11.809+0000 7f1d5cb86640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f1d580d34b0 con 0x7f1d5815eb90 2024-09-12T03:22:11.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:11.809+0000 7f1d5659c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1d580d1720 0x7f1d5814da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:11.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:11.809+0000 7f1d5659c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1d580d1720 0x7f1d5814da60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:40944/0 (socket says 172.21.15.27:40944) 2024-09-12T03:22:11.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:11.809+0000 7f1d5659c640 1 -- 172.21.15.27:0/896393304 learned_addr learned my addr 172.21.15.27:0/896393304 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:22:11.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:11.809+0000 7f1d55d9b640 1 --2- 172.21.15.27:0/896393304 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1d580d5200 0x7f1d581595d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:11.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:11.809+0000 7f1d56d9d640 1 --2- 172.21.15.27:0/896393304 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1d5815eb90 0x7f1d580c9f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:11.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:11.809+0000 7f1d5659c640 1 -- 172.21.15.27:0/896393304 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1d5815eb90 msgr2=0x7f1d580c9f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:11.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:11.809+0000 7f1d5659c640 1 --2- 172.21.15.27:0/896393304 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1d5815eb90 0x7f1d580c9f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:11.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:11.809+0000 7f1d5659c640 1 -- 172.21.15.27:0/896393304 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1d580d5200 msgr2=0x7f1d581595d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:11.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:11.809+0000 7f1d5659c640 1 --2- 172.21.15.27:0/896393304 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1d580d5200 0x7f1d581595d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:11.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:11.809+0000 7f1d5659c640 1 -- 172.21.15.27:0/896393304 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d5814dfa0 con 0x7f1d580d1720 2024-09-12T03:22:11.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:11.809+0000 7f1d55d9b640 1 --2- 172.21.15.27:0/896393304 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1d580d5200 0x7f1d581595d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:22:11.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:11.809+0000 7f1d5659c640 1 --2- 172.21.15.27:0/896393304 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1d580d1720 0x7f1d5814da60 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f1d50009d90 tx=0x7f1d50004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:11.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:11.810+0000 7f1d4effd640 1 -- 172.21.15.27:0/896393304 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d50040070 con 0x7f1d580d1720 2024-09-12T03:22:11.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:11.810+0000 7f1d4effd640 1 -- 172.21.15.27:0/896393304 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d50031c40 con 0x7f1d580d1720 2024-09-12T03:22:11.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:11.810+0000 7f1d5cb86640 1 -- 172.21.15.27:0/896393304 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d580caae0 con 0x7f1d580d1720 2024-09-12T03:22:11.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:11.810+0000 7f1d4effd640 1 -- 172.21.15.27:0/896393304 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d50009450 con 0x7f1d580d1720 2024-09-12T03:22:11.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:11.810+0000 7f1d5cb86640 1 -- 172.21.15.27:0/896393304 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d58158dd0 con 0x7f1d580d1720 2024-09-12T03:22:11.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:11.811+0000 7f1d5cb86640 1 -- 172.21.15.27:0/896393304 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d580693d0 con 0x7f1d580d1720 2024-09-12T03:22:11.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:11.811+0000 7f1d4effd640 1 -- 172.21.15.27:0/896393304 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 21) ==== 91008+0+0 (secure 0 0 0) 0x7f1d50039040 con 0x7f1d580d1720 2024-09-12T03:22:11.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:11.812+0000 7f1d4effd640 1 --2- 172.21.15.27:0/896393304 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f1d3806c3d0 0x7f1d3806e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:11.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:11.812+0000 7f1d55d9b640 1 --2- 172.21.15.27:0/896393304 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f1d3806c3d0 0x7f1d3806e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:11.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:11.812+0000 7f1d4effd640 1 -- 172.21.15.27:0/896393304 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 4999+0+0 (secure 0 0 0) 0x7f1d500acb40 con 0x7f1d580d1720 2024-09-12T03:22:11.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:11.812+0000 7f1d55d9b640 1 --2- 172.21.15.27:0/896393304 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f1d3806c3d0 0x7f1d3806e890 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f1d580c9440 tx=0x7f1d40009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:11.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:11.816+0000 7f1d4effd640 1 -- 172.21.15.27:0/896393304 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f1d5814dfa0 con 0x7f1d580d1720 2024-09-12T03:22:12.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:12.068+0000 7f1d5cb86640 1 -- 172.21.15.27:0/896393304 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_no_upgrade_v1_to_v2"]} v 0) -- 0x7f1d58161970 con 0x7f1d580d1720 2024-09-12T03:22:12.372 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:12.372+0000 7f1d4effd640 1 -- 172.21.15.27:0/896393304 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_no_upgrade_v1_to_v2"]}]=0 v312) ==== 148+0+0 (secure 0 0 0) 0x7f1d5007a870 con 0x7f1d580d1720 2024-09-12T03:22:12.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:12.374+0000 7f1d5cb86640 1 -- 172.21.15.27:0/896393304 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f1d3806c3d0 msgr2=0x7f1d3806e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:12.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:12.374+0000 7f1d5cb86640 1 --2- 172.21.15.27:0/896393304 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f1d3806c3d0 0x7f1d3806e890 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f1d580c9440 tx=0x7f1d40009040 comp rx=0 tx=0).stop 2024-09-12T03:22:12.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:12.374+0000 7f1d5cb86640 1 -- 172.21.15.27:0/896393304 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1d580d1720 msgr2=0x7f1d5814da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:12.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:12.374+0000 7f1d5cb86640 1 --2- 172.21.15.27:0/896393304 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1d580d1720 0x7f1d5814da60 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f1d50009d90 tx=0x7f1d50004a40 comp rx=0 tx=0).stop 2024-09-12T03:22:12.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:12.374+0000 7f1d5cb86640 1 -- 172.21.15.27:0/896393304 shutdown_connections 2024-09-12T03:22:12.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:12.374+0000 7f1d5cb86640 1 --2- 172.21.15.27:0/896393304 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1d5815eb90 0x7f1d580c9f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:12.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:12.374+0000 7f1d5cb86640 1 --2- 172.21.15.27:0/896393304 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1d580d5200 0x7f1d581595d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:12.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:12.374+0000 7f1d5cb86640 1 --2- 172.21.15.27:0/896393304 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f1d3806c3d0 0x7f1d3806e890 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:12.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:12.374+0000 7f1d5cb86640 1 --2- 172.21.15.27:0/896393304 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1d580d1720 0x7f1d5814da60 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:12.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:12.374+0000 7f1d5cb86640 1 -- 172.21.15.27:0/896393304 >> 172.21.15.27:0/896393304 conn(0x7f1d5806dce0 msgr2=0x7f1d58160c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:12.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:12.374+0000 7f1d5cb86640 1 -- 172.21.15.27:0/896393304 shutdown_connections 2024-09-12T03:22:12.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:12.374+0000 7f1d5cb86640 1 -- 172.21.15.27:0/896393304 wait complete. 2024-09-12T03:22:12.387 INFO:tasks.cephfs_test_runner:poor man's upgrade test -- theme continues... ... ok 2024-09-12T03:22:12.387 INFO:tasks.cephfs_test_runner:Starting test: test_subvolume_ops_on_nonexistent_vol (tasks.cephfs.test_volumes.TestMisc) 2024-09-12T03:22:12.388 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-12T03:22:12.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:12.490+0000 7f0f749be640 1 Processor -- start 2024-09-12T03:22:12.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:12.490+0000 7f0f749be640 1 -- start start 2024-09-12T03:22:12.495 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:12.491+0000 7f0f749be640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f0f7015eb90 0x7f0f700d2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:12.495 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:12.491+0000 7f0f749be640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f0f700d1720 0x7f0f700b8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:12.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:12.491+0000 7f0f749be640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0f700d3760 con 0x7f0f700d5200 2024-09-12T03:22:12.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:12.491+0000 7f0f749be640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f0f700b7d20 con 0x7f0f700d1720 2024-09-12T03:22:12.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:12.491+0000 7f0f749be640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f0f700d42e0 con 0x7f0f7015eb90 2024-09-12T03:22:12.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:12.492+0000 7f0f6dd9b640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f0f700d5200 0x7f0f700d1010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:35082/0 (socket says 172.21.15.27:35082) 2024-09-12T03:22:12.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:12.492+0000 7f0f6dd9b640 1 -- 172.21.15.27:0/3654594326 learned_addr learned my addr 172.21.15.27:0/3654594326 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:22:12.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:12.492+0000 7f0f6e59c640 1 --2- 172.21.15.27:0/3654594326 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f0f7015eb90 0x7f0f700d2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:12.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:12.492+0000 7f0f6ed9d640 1 --2- 172.21.15.27:0/3654594326 >> v2:172.21.15.45:3300/0 conn(0x7f0f700d1720 0x7f0f700b8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:12.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:12.492+0000 7f0f6d59a640 1 -- 172.21.15.27:0/3654594326 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2921447780 0 0) 0x7f0f700d3760 con 0x7f0f700d5200 2024-09-12T03:22:12.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:12.492+0000 7f0f6d59a640 1 -- 172.21.15.27:0/3654594326 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0f5c003520 con 0x7f0f700d5200 2024-09-12T03:22:12.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:12.492+0000 7f0f6d59a640 1 -- 172.21.15.27:0/3654594326 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2938160542 0 0) 0x7f0f5c003520 con 0x7f0f700d5200 2024-09-12T03:22:12.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:12.492+0000 7f0f6d59a640 1 -- 172.21.15.27:0/3654594326 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f0f7015eb90 msgr2=0x7f0f700d2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:12.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:12.492+0000 7f0f6d59a640 1 --2- 172.21.15.27:0/3654594326 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f0f7015eb90 0x7f0f700d2df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:12.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:12.492+0000 7f0f6d59a640 1 -- 172.21.15.27:0/3654594326 >> v2:172.21.15.45:3300/0 conn(0x7f0f700d1720 msgr2=0x7f0f700b8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:12.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:12.492+0000 7f0f6d59a640 1 --2- 172.21.15.27:0/3654594326 >> v2:172.21.15.45:3300/0 conn(0x7f0f700d1720 0x7f0f700b8190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:12.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:12.492+0000 7f0f6d59a640 1 -- 172.21.15.27:0/3654594326 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f700d3a40 con 0x7f0f700d5200 2024-09-12T03:22:12.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:12.492+0000 7f0f6d59a640 1 -- 172.21.15.27:0/3654594326 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 847755213 0 0) 0x7f0f58001670 con 0x7f0f700d5200 2024-09-12T03:22:12.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:12.492+0000 7f0f6d59a640 1 -- 172.21.15.27:0/3654594326 >> v1:172.21.15.27:6789/0 conn(0x7f0f700d5200 legacy=0x7f0f700d1010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:12.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:12.492+0000 7f0f6d59a640 1 --2- 172.21.15.27:0/3654594326 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0f5c0039b0 0x7f0f5c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:12.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:12.492+0000 7f0f6d59a640 1 --2- 172.21.15.27:0/3654594326 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f0f7015eb90 0x7f0f5c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:12.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:12.492+0000 7f0f6d59a640 1 --2- 172.21.15.27:0/3654594326 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0f5c004d40 0x7f0f5c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:12.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:12.492+0000 7f0f6d59a640 1 -- 172.21.15.27:0/3654594326 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f0f700d42e0 con 0x7f0f5c004d40 2024-09-12T03:22:12.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:12.492+0000 7f0f6d59a640 1 -- 172.21.15.27:0/3654594326 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f0f5c00b9c0 con 0x7f0f5c0039b0 2024-09-12T03:22:12.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:12.492+0000 7f0f6d59a640 1 -- 172.21.15.27:0/3654594326 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f0f5c00bb40 con 0x7f0f7015eb90 2024-09-12T03:22:12.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:12.492+0000 7f0f6ed9d640 1 --2- 172.21.15.27:0/3654594326 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0f5c004d40 0x7f0f5c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:12.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:12.492+0000 7f0f6e59c640 1 --2- 172.21.15.27:0/3654594326 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0f5c0039b0 0x7f0f5c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:12.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:12.493+0000 7f0f6dd9b640 1 --2- 172.21.15.27:0/3654594326 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f0f7015eb90 0x7f0f5c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:12.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:12.493+0000 7f0f6ed9d640 1 -- 172.21.15.27:0/3654594326 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f0f7015eb90 msgr2=0x7f0f5c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:12.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:12.493+0000 7f0f6ed9d640 1 --2- 172.21.15.27:0/3654594326 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f0f7015eb90 0x7f0f5c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:12.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:12.493+0000 7f0f6ed9d640 1 -- 172.21.15.27:0/3654594326 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0f5c0039b0 msgr2=0x7f0f5c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:12.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:12.493+0000 7f0f6ed9d640 1 --2- 172.21.15.27:0/3654594326 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0f5c0039b0 0x7f0f5c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:12.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:12.493+0000 7f0f6ed9d640 1 -- 172.21.15.27:0/3654594326 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f5c003520 con 0x7f0f5c004d40 2024-09-12T03:22:12.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:12.493+0000 7f0f6ed9d640 1 --2- 172.21.15.27:0/3654594326 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0f5c004d40 0x7f0f5c0051c0 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f0f68009f90 tx=0x7f0f5c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d290bc04ee23ce4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:12.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:12.494+0000 7f0f6d59a640 1 -- 172.21.15.27:0/3654594326 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f68009c10 con 0x7f0f5c004d40 2024-09-12T03:22:12.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:12.494+0000 7f0f6d59a640 1 -- 172.21.15.27:0/3654594326 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f68036070 con 0x7f0f5c004d40 2024-09-12T03:22:12.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:12.494+0000 7f0f6d59a640 1 -- 172.21.15.27:0/3654594326 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f68031840 con 0x7f0f5c004d40 2024-09-12T03:22:12.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:12.494+0000 7f0f749be640 1 -- 172.21.15.27:0/3654594326 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0f5c004d40 msgr2=0x7f0f5c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:12.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:12.494+0000 7f0f749be640 1 --2- 172.21.15.27:0/3654594326 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0f5c004d40 0x7f0f5c0051c0 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f0f68009f90 tx=0x7f0f5c005830 comp rx=0 tx=0).stop 2024-09-12T03:22:12.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:12.494+0000 7f0f6ed9d640 1 -- 172.21.15.27:0/3654594326 reap_dead start 2024-09-12T03:22:12.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:12.494+0000 7f0f749be640 1 -- 172.21.15.27:0/3654594326 shutdown_connections 2024-09-12T03:22:12.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:12.494+0000 7f0f749be640 1 -- 172.21.15.27:0/3654594326 >> 172.21.15.27:0/3654594326 conn(0x7f0f7006dce0 msgr2=0x7f0f7006e0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:12.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:12.494+0000 7f0f749be640 1 -- 172.21.15.27:0/3654594326 shutdown_connections 2024-09-12T03:22:12.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:12.494+0000 7f0f749be640 1 -- 172.21.15.27:0/3654594326 wait complete. 2024-09-12T03:22:12.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:12.496+0000 7f0f749be640 1 Processor -- start 2024-09-12T03:22:12.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:12.496+0000 7f0f749be640 1 -- start start 2024-09-12T03:22:12.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:12.496+0000 7f0f749be640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0f5c004d40 0x7f0f700caa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:12.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:12.496+0000 7f0f749be640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f0f701f3710 0x7f0f701f3b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:12.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:12.496+0000 7f0f749be640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0f700c8dc0 0x7f0f700c9240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:12.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:12.496+0000 7f0f749be640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f0f7015ef70 con 0x7f0f5c004d40 2024-09-12T03:22:12.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:12.496+0000 7f0f749be640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f0f700d4000 con 0x7f0f700c8dc0 2024-09-12T03:22:12.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:12.496+0000 7f0f749be640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f0f7015f0f0 con 0x7f0f701f3710 2024-09-12T03:22:12.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:12.496+0000 7f0f6e59c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0f5c004d40 0x7f0f700caa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:12.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:12.497+0000 7f0f6e59c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0f5c004d40 0x7f0f700caa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:40958/0 (socket says 172.21.15.27:40958) 2024-09-12T03:22:12.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:12.497+0000 7f0f6e59c640 1 -- 172.21.15.27:0/4080504646 learned_addr learned my addr 172.21.15.27:0/4080504646 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:22:12.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:12.497+0000 7f0f6ed9d640 1 --2- 172.21.15.27:0/4080504646 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0f700c8dc0 0x7f0f700c9240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:12.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:12.497+0000 7f0f6dd9b640 1 --2- 172.21.15.27:0/4080504646 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f0f701f3710 0x7f0f701f3b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:12.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:12.497+0000 7f0f6e59c640 1 -- 172.21.15.27:0/4080504646 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f0f701f3710 msgr2=0x7f0f701f3b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:12.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:12.497+0000 7f0f6e59c640 1 --2- 172.21.15.27:0/4080504646 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f0f701f3710 0x7f0f701f3b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:12.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:12.497+0000 7f0f6e59c640 1 -- 172.21.15.27:0/4080504646 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0f700c8dc0 msgr2=0x7f0f700c9240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:12.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:12.497+0000 7f0f6e59c640 1 --2- 172.21.15.27:0/4080504646 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0f700c8dc0 0x7f0f700c9240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:12.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:12.497+0000 7f0f6e59c640 1 -- 172.21.15.27:0/4080504646 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f701f40d0 con 0x7f0f5c004d40 2024-09-12T03:22:12.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:12.497+0000 7f0f6e59c640 1 --2- 172.21.15.27:0/4080504646 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0f5c004d40 0x7f0f700caa50 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f0f600118e0 tx=0x7f0f60011da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:12.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:12.497+0000 7f0f66ffd640 1 -- 172.21.15.27:0/4080504646 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f6000faf0 con 0x7f0f5c004d40 2024-09-12T03:22:12.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:12.497+0000 7f0f66ffd640 1 -- 172.21.15.27:0/4080504646 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f6000fc90 con 0x7f0f5c004d40 2024-09-12T03:22:12.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:12.497+0000 7f0f749be640 1 -- 172.21.15.27:0/4080504646 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f7014ba90 con 0x7f0f5c004d40 2024-09-12T03:22:12.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:12.497+0000 7f0f66ffd640 1 -- 172.21.15.27:0/4080504646 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f60014440 con 0x7f0f5c004d40 2024-09-12T03:22:12.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:12.497+0000 7f0f749be640 1 -- 172.21.15.27:0/4080504646 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f7014bd50 con 0x7f0f5c004d40 2024-09-12T03:22:12.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:12.498+0000 7f0f749be640 1 -- 172.21.15.27:0/4080504646 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f700d1010 con 0x7f0f5c004d40 2024-09-12T03:22:12.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:12.499+0000 7f0f66ffd640 1 -- 172.21.15.27:0/4080504646 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 21) ==== 91008+0+0 (secure 0 0 0) 0x7f0f60012040 con 0x7f0f5c004d40 2024-09-12T03:22:12.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:12.499+0000 7f0f66ffd640 1 --2- 172.21.15.27:0/4080504646 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f0f5006c3d0 0x7f0f5006e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:12.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:12.499+0000 7f0f66ffd640 1 -- 172.21.15.27:0/4080504646 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 4999+0+0 (secure 0 0 0) 0x7f0f6008bd50 con 0x7f0f5c004d40 2024-09-12T03:22:12.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:12.499+0000 7f0f6dd9b640 1 --2- 172.21.15.27:0/4080504646 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f0f5006c3d0 0x7f0f5006e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:12.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:12.500+0000 7f0f6dd9b640 1 --2- 172.21.15.27:0/4080504646 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f0f5006c3d0 0x7f0f5006e890 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f0f70160650 tx=0x7f0f58009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:12.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:12.504+0000 7f0f66ffd640 1 -- 172.21.15.27:0/4080504646 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f0f60092080 con 0x7f0f5c004d40 2024-09-12T03:22:12.682 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:12.681+0000 7f0f749be640 1 -- 172.21.15.27:0/4080504646 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0f700cfcd0 con 0x7f0f5c004d40 2024-09-12T03:22:12.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:12.682+0000 7f0f66ffd640 1 -- 172.21.15.27:0/4080504646 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v118) ==== 74+0+13083 (secure 0 0 0) 0x7f0f60019030 con 0x7f0f5c004d40 2024-09-12T03:22:12.683 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:22:12.683 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":118,"fsid":"d2df504e-2647-4385-a5db-9fdf69f487a0","created":"2024-09-12T03:16:07.778937+0000","modified":"2024-09-12T03:22:11.366953+0000","last_up_change":"2024-09-12T03:16:19.314778+0000","last_in_change":"2024-09-12T03:16:09.893086+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T03:16:20.097646+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1851116c-1fb3-4dc9-9772-f90e2ef4c355","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6817","nonce":3002321488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6819","nonce":3002321488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6823","nonce":3002321488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6821","nonce":3002321488}]},"public_addr":"172.21.15.27:6817/3002321488","cluster_addr":"172.21.15.27:6819/3002321488","heartbeat_back_addr":"172.21.15.27:6823/3002321488","heartbeat_front_addr":"172.21.15.27:6821/3002321488","state":["exists","up"]},{"osd":1,"uuid":"f38ae17d-73eb-4204-b99d-975fa806fb87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6809","nonce":1613102886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6811","nonce":1613102886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6815","nonce":1613102886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6813","nonce":1613102886}]},"public_addr":"172.21.15.27:6809/1613102886","cluster_addr":"172.21.15.27:6811/1613102886","heartbeat_back_addr":"172.21.15.27:6815/1613102886","heartbeat_front_addr":"172.21.15.27:6813/1613102886","state":["exists","up"]},{"osd":2,"uuid":"fb12dcc8-bdd0-42b8-b46d-8b10106224bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6825","nonce":3534723355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6827","nonce":3534723355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6831","nonce":3534723355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6829","nonce":3534723355}]},"public_addr":"172.21.15.27:6825/3534723355","cluster_addr":"172.21.15.27:6827/3534723355","heartbeat_back_addr":"172.21.15.27:6831/3534723355","heartbeat_front_addr":"172.21.15.27:6829/3534723355","state":["exists","up"]},{"osd":3,"uuid":"d74d3f49-9c24-4b9f-98e2-3ecd7b5ef3b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6801","nonce":80661598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6803","nonce":80661598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6807","nonce":80661598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6805","nonce":80661598}]},"public_addr":"172.21.15.27:6801/80661598","cluster_addr":"172.21.15.27:6803/80661598","heartbeat_back_addr":"172.21.15.27:6807/80661598","heartbeat_front_addr":"172.21.15.27:6805/80661598","state":["exists","up"]},{"osd":4,"uuid":"2bf123f2-1b0d-41c4-aa3b-805bab267e59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6817","nonce":3943189827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6819","nonce":3943189827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6823","nonce":3943189827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6821","nonce":3943189827}]},"public_addr":"172.21.15.45:6817/3943189827","cluster_addr":"172.21.15.45:6819/3943189827","heartbeat_back_addr":"172.21.15.45:6823/3943189827","heartbeat_front_addr":"172.21.15.45:6821/3943189827","state":["exists","up"]},{"osd":5,"uuid":"c49ba5da-91c6-4a86-a97b-4aba89c253ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6809","nonce":3620760330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6811","nonce":3620760330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6815","nonce":3620760330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6813","nonce":3620760330}]},"public_addr":"172.21.15.45:6809/3620760330","cluster_addr":"172.21.15.45:6811/3620760330","heartbeat_back_addr":"172.21.15.45:6815/3620760330","heartbeat_front_addr":"172.21.15.45:6813/3620760330","state":["exists","up"]},{"osd":6,"uuid":"b0123475-89f7-4a34-9e33-d797b99dd9dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6801","nonce":1989083510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6803","nonce":1989083510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6807","nonce":1989083510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6805","nonce":1989083510}]},"public_addr":"172.21.15.45:6801/1989083510","cluster_addr":"172.21.15.45:6803/1989083510","heartbeat_back_addr":"172.21.15.45:6807/1989083510","heartbeat_front_addr":"172.21.15.45:6805/1989083510","state":["exists","up"]},{"osd":7,"uuid":"4dad773d-b7d8-46fc-8a10-a972aad27b29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6825","nonce":1513300294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6827","nonce":1513300294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6831","nonce":1513300294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6829","nonce":1513300294}]},"public_addr":"172.21.15.45:6825/1513300294","cluster_addr":"172.21.15.45:6827/1513300294","heartbeat_back_addr":"172.21.15.45:6831/1513300294","heartbeat_front_addr":"172.21.15.45:6829/1513300294","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.881855+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:17.133252+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.735339+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.500742+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.477571+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:17.167667+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.470291+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.873743+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.27:6835/1760654641":"2024-09-13T03:22:08.358819+0000","172.21.15.27:6833/1760654641":"2024-09-13T03:22:08.358819+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]},{"pool":19,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T03:22:12.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:12.684+0000 7f0f749be640 1 -- 172.21.15.27:0/4080504646 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f0f5006c3d0 msgr2=0x7f0f5006e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:12.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:12.684+0000 7f0f749be640 1 --2- 172.21.15.27:0/4080504646 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f0f5006c3d0 0x7f0f5006e890 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f0f70160650 tx=0x7f0f58009040 comp rx=0 tx=0).stop 2024-09-12T03:22:12.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:12.684+0000 7f0f749be640 1 -- 172.21.15.27:0/4080504646 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0f5c004d40 msgr2=0x7f0f700caa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:12.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:12.684+0000 7f0f749be640 1 --2- 172.21.15.27:0/4080504646 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0f5c004d40 0x7f0f700caa50 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f0f600118e0 tx=0x7f0f60011da0 comp rx=0 tx=0).stop 2024-09-12T03:22:12.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:12.684+0000 7f0f749be640 1 -- 172.21.15.27:0/4080504646 shutdown_connections 2024-09-12T03:22:12.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:12.684+0000 7f0f749be640 1 --2- 172.21.15.27:0/4080504646 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0f700c8dc0 0x7f0f700c9240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:12.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:12.684+0000 7f0f749be640 1 --2- 172.21.15.27:0/4080504646 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f0f701f3710 0x7f0f701f3b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:12.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:12.684+0000 7f0f749be640 1 --2- 172.21.15.27:0/4080504646 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f0f5006c3d0 0x7f0f5006e890 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:12.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:12.684+0000 7f0f749be640 1 --2- 172.21.15.27:0/4080504646 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0f5c004d40 0x7f0f700caa50 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:12.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:12.684+0000 7f0f749be640 1 -- 172.21.15.27:0/4080504646 >> 172.21.15.27:0/4080504646 conn(0x7f0f7006dce0 msgr2=0x7f0f7015f2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:12.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:12.684+0000 7f0f749be640 1 -- 172.21.15.27:0/4080504646 shutdown_connections 2024-09-12T03:22:12.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:12.684+0000 7f0f749be640 1 -- 172.21.15.27:0/4080504646 wait complete. 2024-09-12T03:22:12.696 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-12T03:16:20.097646+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-12T03:22:12.697 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-12T03:22:12.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:12.801+0000 7fbda9749640 1 Processor -- start 2024-09-12T03:22:12.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:12.801+0000 7fbda9749640 1 -- start start 2024-09-12T03:22:12.803 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:12.802+0000 7fbda9749640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fbda40b6230 0x7fbda40b66c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:12.803 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:12.802+0000 7fbda9749640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbda415ed70 con 0x7fbda40b3600 2024-09-12T03:22:12.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:12.802+0000 7fbda9749640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbda40b7d10 con 0x7fbda415e990 2024-09-12T03:22:12.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:12.802+0000 7fbda9749640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7fbda40d2a30 con 0x7fbda40b6230 2024-09-12T03:22:12.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:12.802+0000 7fbda27fc640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fbda40b6230 0x7fbda40b66c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:12.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:12.802+0000 7fbda37fe640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fbda40b3600 0x7fbda40b3a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:35092/0 (socket says 172.21.15.27:35092) 2024-09-12T03:22:12.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:12.802+0000 7fbda27fc640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fbda40b6230 0x7fbda40b66c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:40966/0 (socket says 172.21.15.27:40966) 2024-09-12T03:22:12.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:12.802+0000 7fbda37fe640 1 -- 172.21.15.27:0/1134912014 learned_addr learned my addr 172.21.15.27:0/1134912014 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:22:12.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:12.802+0000 7fbda27fc640 1 -- 172.21.15.27:0/1134912014 >> v1:172.21.15.45:6789/0 conn(0x7fbda415e990 legacy=0x7fbda40b7600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:12.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:12.802+0000 7fbda27fc640 1 -- 172.21.15.27:0/1134912014 >> v1:172.21.15.27:6789/0 conn(0x7fbda40b3600 legacy=0x7fbda40b3a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:12.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:12.802+0000 7fbda27fc640 1 -- 172.21.15.27:0/1134912014 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbda40d2d10 con 0x7fbda40b6230 2024-09-12T03:22:12.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:12.803+0000 7fbda27fc640 1 --2- 172.21.15.27:0/1134912014 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbda40b6230 0x7fbda40b66c0 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7fbd8c0097a0 tx=0x7fbd8c02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=213dfa39d2f66684 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:12.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:12.803+0000 7fbda1ffb640 1 -- 172.21.15.27:0/1134912014 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd8c037070 con 0x7fbda40b6230 2024-09-12T03:22:12.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:12.803+0000 7fbda1ffb640 1 -- 172.21.15.27:0/1134912014 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd8c032440 con 0x7fbda40b6230 2024-09-12T03:22:12.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:12.803+0000 7fbda1ffb640 1 -- 172.21.15.27:0/1134912014 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd8c032810 con 0x7fbda40b6230 2024-09-12T03:22:12.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:12.804+0000 7fbda9749640 1 -- 172.21.15.27:0/1134912014 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbda40b6230 msgr2=0x7fbda40b66c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:12.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:12.804+0000 7fbda9749640 1 --2- 172.21.15.27:0/1134912014 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbda40b6230 0x7fbda40b66c0 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7fbd8c0097a0 tx=0x7fbd8c02a230 comp rx=0 tx=0).stop 2024-09-12T03:22:12.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:12.804+0000 7fbda9749640 1 -- 172.21.15.27:0/1134912014 shutdown_connections 2024-09-12T03:22:12.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:12.804+0000 7fbda9749640 1 --2- 172.21.15.27:0/1134912014 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbda40b6230 0x7fbda40b66c0 unknown :-1 s=CLOSED pgs=1195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:12.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:12.804+0000 7fbda9749640 1 -- 172.21.15.27:0/1134912014 >> 172.21.15.27:0/1134912014 conn(0x7fbda40d1230 msgr2=0x7fbda40d1640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:12.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:12.804+0000 7fbda9749640 1 -- 172.21.15.27:0/1134912014 shutdown_connections 2024-09-12T03:22:12.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:12.804+0000 7fbda9749640 1 -- 172.21.15.27:0/1134912014 wait complete. 2024-09-12T03:22:12.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:12.805+0000 7fbda9749640 1 Processor -- start 2024-09-12T03:22:12.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:12.805+0000 7fbda9749640 1 -- start start 2024-09-12T03:22:12.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:12.806+0000 7fbda9749640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbda40b3600 0x7fbda40c9350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:12.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:12.806+0000 7fbda9749640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbda40b6230 0x7fbda414ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:12.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:12.806+0000 7fbda9749640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fbda415e990 0x7fbda40ce160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:12.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:12.806+0000 7fbda9749640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fbda40d35b0 con 0x7fbda40b3600 2024-09-12T03:22:12.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:12.806+0000 7fbda9749640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fbda40d32d0 con 0x7fbda40b6230 2024-09-12T03:22:12.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:12.806+0000 7fbda9749640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fbda40d3ad0 con 0x7fbda415e990 2024-09-12T03:22:12.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:12.806+0000 7fbda2ffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbda40b3600 0x7fbda40c9350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:12.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:12.806+0000 7fbda2ffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbda40b3600 0x7fbda40c9350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:40980/0 (socket says 172.21.15.27:40980) 2024-09-12T03:22:12.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:12.806+0000 7fbda2ffd640 1 -- 172.21.15.27:0/3200337803 learned_addr learned my addr 172.21.15.27:0/3200337803 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:22:12.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:12.806+0000 7fbda27fc640 1 --2- 172.21.15.27:0/3200337803 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbda40b6230 0x7fbda414ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:12.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:12.806+0000 7fbda37fe640 1 --2- 172.21.15.27:0/3200337803 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fbda415e990 0x7fbda40ce160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:12.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:12.807+0000 7fbda2ffd640 1 -- 172.21.15.27:0/3200337803 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fbda415e990 msgr2=0x7fbda40ce160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:12.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:12.807+0000 7fbda2ffd640 1 --2- 172.21.15.27:0/3200337803 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fbda415e990 0x7fbda40ce160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:12.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:12.807+0000 7fbda2ffd640 1 -- 172.21.15.27:0/3200337803 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbda40b6230 msgr2=0x7fbda414ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:12.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:12.807+0000 7fbda2ffd640 1 --2- 172.21.15.27:0/3200337803 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbda40b6230 0x7fbda414ee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:12.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:12.807+0000 7fbda2ffd640 1 -- 172.21.15.27:0/3200337803 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbda40cd0c0 con 0x7fbda40b3600 2024-09-12T03:22:12.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:12.807+0000 7fbda37fe640 1 --2- 172.21.15.27:0/3200337803 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fbda415e990 0x7fbda40ce160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:22:12.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:12.807+0000 7fbda2ffd640 1 --2- 172.21.15.27:0/3200337803 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbda40b3600 0x7fbda40c9350 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7fbd9c00c3f0 tx=0x7fbd9c00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:12.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:12.807+0000 7fbd9b7fe640 1 -- 172.21.15.27:0/3200337803 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd9c0052d0 con 0x7fbda40b3600 2024-09-12T03:22:12.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:12.807+0000 7fbd9b7fe640 1 -- 172.21.15.27:0/3200337803 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd9c005490 con 0x7fbda40b3600 2024-09-12T03:22:12.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:12.807+0000 7fbd9b7fe640 1 -- 172.21.15.27:0/3200337803 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd9c007260 con 0x7fbda40b3600 2024-09-12T03:22:12.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:12.807+0000 7fbda9749640 1 -- 172.21.15.27:0/3200337803 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbda40cd3a0 con 0x7fbda40b3600 2024-09-12T03:22:12.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:12.807+0000 7fbda9749640 1 -- 172.21.15.27:0/3200337803 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbda40cd960 con 0x7fbda40b3600 2024-09-12T03:22:12.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:12.808+0000 7fbda9749640 1 -- 172.21.15.27:0/3200337803 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbda40b7780 con 0x7fbda40b3600 2024-09-12T03:22:12.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:12.809+0000 7fbd9b7fe640 1 -- 172.21.15.27:0/3200337803 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 21) ==== 91008+0+0 (secure 0 0 0) 0x7fbd9c007400 con 0x7fbda40b3600 2024-09-12T03:22:12.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:12.809+0000 7fbd9b7fe640 1 --2- 172.21.15.27:0/3200337803 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fbd8406c3d0 0x7fbd8406e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:12.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:12.809+0000 7fbd9b7fe640 1 -- 172.21.15.27:0/3200337803 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 4999+0+0 (secure 0 0 0) 0x7fbd9c08aa90 con 0x7fbda40b3600 2024-09-12T03:22:12.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:12.809+0000 7fbda27fc640 1 --2- 172.21.15.27:0/3200337803 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fbd8406c3d0 0x7fbd8406e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:12.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:12.810+0000 7fbda27fc640 1 --2- 172.21.15.27:0/3200337803 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fbd8406c3d0 0x7fbd8406e890 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fbd8c034000 tx=0x7fbd8c02e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:12.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:12.813+0000 7fbd9b7fe640 1 -- 172.21.15.27:0/3200337803 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fbd9c0587c0 con 0x7fbda40b3600 2024-09-12T03:22:12.993 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:12.992+0000 7fbda9749640 1 -- 172.21.15.27:0/3200337803 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fbda4069f10 con 0x7fbda40b3600 2024-09-12T03:22:12.993 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:12.992+0000 7fbd9b7fe640 1 -- 172.21.15.27:0/3200337803 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v118) ==== 93+0+10 (secure 0 0 0) 0x7fbd9c05cac0 con 0x7fbda40b3600 2024-09-12T03:22:12.993 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 1 2024-09-12T03:22:12.996 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:12.994+0000 7fbda9749640 1 -- 172.21.15.27:0/3200337803 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fbd8406c3d0 msgr2=0x7fbd8406e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:12.996 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:12.994+0000 7fbda9749640 1 --2- 172.21.15.27:0/3200337803 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fbd8406c3d0 0x7fbd8406e890 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fbd8c034000 tx=0x7fbd8c02e000 comp rx=0 tx=0).stop 2024-09-12T03:22:12.996 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:12.994+0000 7fbda9749640 1 -- 172.21.15.27:0/3200337803 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbda40b3600 msgr2=0x7fbda40c9350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:12.996 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:12.995+0000 7fbda9749640 1 --2- 172.21.15.27:0/3200337803 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbda40b3600 0x7fbda40c9350 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7fbd9c00c3f0 tx=0x7fbd9c00c8b0 comp rx=0 tx=0).stop 2024-09-12T03:22:12.996 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:12.995+0000 7fbda9749640 1 -- 172.21.15.27:0/3200337803 shutdown_connections 2024-09-12T03:22:12.996 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:12.995+0000 7fbda9749640 1 --2- 172.21.15.27:0/3200337803 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fbda415e990 0x7fbda40ce160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:12.996 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:12.995+0000 7fbda9749640 1 --2- 172.21.15.27:0/3200337803 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbda40b6230 0x7fbda414ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:12.996 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:12.995+0000 7fbda9749640 1 --2- 172.21.15.27:0/3200337803 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fbd8406c3d0 0x7fbd8406e890 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:12.996 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:12.995+0000 7fbda9749640 1 --2- 172.21.15.27:0/3200337803 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbda40b3600 0x7fbda40c9350 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:12.996 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:12.995+0000 7fbda9749640 1 -- 172.21.15.27:0/3200337803 >> 172.21.15.27:0/3200337803 conn(0x7fbda40d1230 msgr2=0x7fbda40d1610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:12.997 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:12.995+0000 7fbda9749640 1 -- 172.21.15.27:0/3200337803 shutdown_connections 2024-09-12T03:22:12.997 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:12.995+0000 7fbda9749640 1 -- 172.21.15.27:0/3200337803 wait complete. 2024-09-12T03:22:13.007 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_ops_on_nonexistent_vol' 2024-09-12T03:22:13.109 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:13.108+0000 7f4a6fc27640 1 Processor -- start 2024-09-12T03:22:13.109 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:13.108+0000 7f4a6fc27640 1 -- start start 2024-09-12T03:22:13.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:13.109+0000 7f4a6fc27640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f4a68158a50 0x7f4a6815f840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:13.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:13.109+0000 7f4a6fc27640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4a680b5f80 con 0x7f4a68158e30 2024-09-12T03:22:13.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:13.109+0000 7f4a6fc27640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4a680b6260 con 0x7f4a68073250 2024-09-12T03:22:13.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:13.109+0000 7f4a6fc27640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f4a680b6540 con 0x7f4a68158a50 2024-09-12T03:22:13.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:13.109+0000 7f4a6d1c2640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f4a68158e30 0x7f4a680b5770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:35098/0 (socket says 172.21.15.27:35098) 2024-09-12T03:22:13.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:13.109+0000 7f4a6d1c2640 1 -- 172.21.15.27:0/2663595037 learned_addr learned my addr 172.21.15.27:0/2663595037 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:22:13.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:13.109+0000 7f4a6d9c3640 1 --2- 172.21.15.27:0/2663595037 >> v2:172.21.15.45:3300/0 conn(0x7f4a68158a50 0x7f4a6815f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:13.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:13.110+0000 7f4a6c9c1640 1 -- 172.21.15.27:0/2663595037 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1204971318 0 0) 0x7f4a680b5f80 con 0x7f4a68158e30 2024-09-12T03:22:13.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:13.110+0000 7f4a6c9c1640 1 -- 172.21.15.27:0/2663595037 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4a50003520 con 0x7f4a68158e30 2024-09-12T03:22:13.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:13.110+0000 7f4a6d9c3640 1 -- 172.21.15.27:0/2663595037 >> v1:172.21.15.45:6789/0 conn(0x7f4a68073250 legacy=0x7f4a6806d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:13.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:13.110+0000 7f4a6c9c1640 1 -- 172.21.15.27:0/2663595037 <== mon.1 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 28817763 0 0) 0x7f4a680b6260 con 0x7f4a68073250 2024-09-12T03:22:13.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:13.110+0000 7f4a6d9c3640 1 -- 172.21.15.27:0/2663595037 >> v1:172.21.15.27:6789/0 conn(0x7f4a68158e30 legacy=0x7f4a680b5770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:13.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:13.110+0000 7f4a6d9c3640 1 -- 172.21.15.27:0/2663595037 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a680b6820 con 0x7f4a68158a50 2024-09-12T03:22:13.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:13.111+0000 7f4a6d9c3640 1 --2- 172.21.15.27:0/2663595037 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4a68158a50 0x7f4a6815f840 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f4a58009d90 tx=0x7f4a5802a6f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cf4b0c8cd3212ce1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:13.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:13.111+0000 7f4a6c9c1640 1 -- 172.21.15.27:0/2663595037 <== mon.1 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a58007740 con 0x7f4a68158a50 2024-09-12T03:22:13.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:13.111+0000 7f4a6c9c1640 1 -- 172.21.15.27:0/2663595037 <== mon.1 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a580078e0 con 0x7f4a68158a50 2024-09-12T03:22:13.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:13.111+0000 7f4a6c9c1640 1 -- 172.21.15.27:0/2663595037 <== mon.1 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a58007c90 con 0x7f4a68158a50 2024-09-12T03:22:13.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:13.111+0000 7f4a6fc27640 1 -- 172.21.15.27:0/2663595037 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4a68158a50 msgr2=0x7f4a6815f840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:13.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:13.111+0000 7f4a6fc27640 1 --2- 172.21.15.27:0/2663595037 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4a68158a50 0x7f4a6815f840 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f4a58009d90 tx=0x7f4a5802a6f0 comp rx=0 tx=0).stop 2024-09-12T03:22:13.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:13.111+0000 7f4a6fc27640 1 -- 172.21.15.27:0/2663595037 shutdown_connections 2024-09-12T03:22:13.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:13.111+0000 7f4a6fc27640 1 --2- 172.21.15.27:0/2663595037 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4a68158a50 0x7f4a6815f840 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:13.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:13.111+0000 7f4a6fc27640 1 -- 172.21.15.27:0/2663595037 >> 172.21.15.27:0/2663595037 conn(0x7f4a680b4470 msgr2=0x7f4a680b4850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:13.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:13.111+0000 7f4a6fc27640 1 -- 172.21.15.27:0/2663595037 shutdown_connections 2024-09-12T03:22:13.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:13.111+0000 7f4a6fc27640 1 -- 172.21.15.27:0/2663595037 wait complete. 2024-09-12T03:22:13.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:13.112+0000 7f4a6fc27640 1 Processor -- start 2024-09-12T03:22:13.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:13.113+0000 7f4a6fc27640 1 -- start start 2024-09-12T03:22:13.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:13.113+0000 7f4a6fc27640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4a68073250 0x7f4a681f3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:13.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:13.113+0000 7f4a6fc27640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4a68158a50 0x7f4a68153140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:13.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:13.113+0000 7f4a6fc27640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4a68158e30 0x7f4a68154300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:13.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:13.113+0000 7f4a6fc27640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f4a68073a70 con 0x7f4a68158e30 2024-09-12T03:22:13.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:13.113+0000 7f4a6fc27640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f4a680738f0 con 0x7f4a68158a50 2024-09-12T03:22:13.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:13.113+0000 7f4a6fc27640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f4a680cbaf0 con 0x7f4a68073250 2024-09-12T03:22:13.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:13.113+0000 7f4a6e1c4640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4a68158e30 0x7f4a68154300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:13.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:13.113+0000 7f4a6e1c4640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4a68158e30 0x7f4a68154300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:40992/0 (socket says 172.21.15.27:40992) 2024-09-12T03:22:13.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:13.113+0000 7f4a6e1c4640 1 -- 172.21.15.27:0/229899217 learned_addr learned my addr 172.21.15.27:0/229899217 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:22:13.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:13.114+0000 7f4a6d9c3640 1 --2- 172.21.15.27:0/229899217 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4a68073250 0x7f4a681f3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:13.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:13.114+0000 7f4a6d1c2640 1 --2- 172.21.15.27:0/229899217 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4a68158a50 0x7f4a68153140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:13.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:13.114+0000 7f4a6e1c4640 1 -- 172.21.15.27:0/229899217 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4a68073250 msgr2=0x7f4a681f3fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:13.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:13.114+0000 7f4a6e1c4640 1 --2- 172.21.15.27:0/229899217 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4a68073250 0x7f4a681f3fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:13.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:13.114+0000 7f4a6e1c4640 1 -- 172.21.15.27:0/229899217 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4a68158a50 msgr2=0x7f4a68153140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:13.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:13.114+0000 7f4a6e1c4640 1 --2- 172.21.15.27:0/229899217 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4a68158a50 0x7f4a68153140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:13.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:13.114+0000 7f4a6e1c4640 1 -- 172.21.15.27:0/229899217 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a68153680 con 0x7f4a68158e30 2024-09-12T03:22:13.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:13.114+0000 7f4a6d1c2640 1 --2- 172.21.15.27:0/229899217 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4a68158a50 0x7f4a68153140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:22:13.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:13.114+0000 7f4a6e1c4640 1 --2- 172.21.15.27:0/229899217 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4a68158e30 0x7f4a68154300 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f4a6000c3e0 tx=0x7f4a6000c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:13.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:13.114+0000 7f4a667fc640 1 -- 172.21.15.27:0/229899217 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a60017070 con 0x7f4a68158e30 2024-09-12T03:22:13.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:13.114+0000 7f4a667fc640 1 -- 172.21.15.27:0/229899217 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a600035f0 con 0x7f4a68158e30 2024-09-12T03:22:13.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:13.114+0000 7f4a6fc27640 1 -- 172.21.15.27:0/229899217 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a68154cc0 con 0x7f4a68158e30 2024-09-12T03:22:13.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:13.114+0000 7f4a667fc640 1 -- 172.21.15.27:0/229899217 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a600039e0 con 0x7f4a68158e30 2024-09-12T03:22:13.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:13.114+0000 7f4a6fc27640 1 -- 172.21.15.27:0/229899217 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a680cc020 con 0x7f4a68158e30 2024-09-12T03:22:13.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:13.115+0000 7f4a6fc27640 1 -- 172.21.15.27:0/229899217 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a6815f840 con 0x7f4a68158e30 2024-09-12T03:22:13.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:13.116+0000 7f4a667fc640 1 -- 172.21.15.27:0/229899217 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 21) ==== 91008+0+0 (secure 0 0 0) 0x7f4a6001a070 con 0x7f4a68158e30 2024-09-12T03:22:13.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:13.116+0000 7f4a667fc640 1 --2- 172.21.15.27:0/229899217 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f4a5406c400 0x7f4a5406e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:13.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:13.116+0000 7f4a667fc640 1 -- 172.21.15.27:0/229899217 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 4999+0+0 (secure 0 0 0) 0x7f4a6008a930 con 0x7f4a68158e30 2024-09-12T03:22:13.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:13.116+0000 7f4a6d9c3640 1 --2- 172.21.15.27:0/229899217 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f4a5406c400 0x7f4a5406e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:13.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:13.117+0000 7f4a6d9c3640 1 --2- 172.21.15.27:0/229899217 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f4a5406c400 0x7f4a5406e8c0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f4a58034000 tx=0x7f4a5802f000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:13.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:13.120+0000 7f4a667fc640 1 -- 172.21.15.27:0/229899217 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f4a60058d30 con 0x7f4a68158e30 2024-09-12T03:22:13.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:13.361+0000 7f4a6fc27640 1 -- 172.21.15.27:0/229899217 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_ops_on_nonexistent_vol"]} v 0) -- 0x7f4a6806dbd0 con 0x7f4a68158e30 2024-09-12T03:22:13.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:13.377+0000 7f4a667fc640 1 -- 172.21.15.27:0/229899217 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_ops_on_nonexistent_vol"]}]=0 v313) ==== 154+0+0 (secure 0 0 0) 0x7f4a60058630 con 0x7f4a68158e30 2024-09-12T03:22:13.379 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:13.379+0000 7f4a6fc27640 1 -- 172.21.15.27:0/229899217 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f4a5406c400 msgr2=0x7f4a5406e8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:13.379 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:13.379+0000 7f4a6fc27640 1 --2- 172.21.15.27:0/229899217 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f4a5406c400 0x7f4a5406e8c0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f4a58034000 tx=0x7f4a5802f000 comp rx=0 tx=0).stop 2024-09-12T03:22:13.379 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:13.379+0000 7f4a6fc27640 1 -- 172.21.15.27:0/229899217 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4a68158e30 msgr2=0x7f4a68154300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:13.379 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:13.379+0000 7f4a6fc27640 1 --2- 172.21.15.27:0/229899217 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4a68158e30 0x7f4a68154300 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f4a6000c3e0 tx=0x7f4a6000c8a0 comp rx=0 tx=0).stop 2024-09-12T03:22:13.379 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:13.379+0000 7f4a6fc27640 1 -- 172.21.15.27:0/229899217 shutdown_connections 2024-09-12T03:22:13.379 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:13.379+0000 7f4a6fc27640 1 --2- 172.21.15.27:0/229899217 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f4a5406c400 0x7f4a5406e8c0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:13.379 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:13.379+0000 7f4a6fc27640 1 --2- 172.21.15.27:0/229899217 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4a68158e30 0x7f4a68154300 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:13.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:13.379+0000 7f4a6fc27640 1 --2- 172.21.15.27:0/229899217 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4a68158a50 0x7f4a68153140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:13.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:13.379+0000 7f4a6fc27640 1 --2- 172.21.15.27:0/229899217 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4a68073250 0x7f4a681f3fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:13.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:13.379+0000 7f4a6fc27640 1 -- 172.21.15.27:0/229899217 >> 172.21.15.27:0/229899217 conn(0x7f4a680b4470 msgr2=0x7f4a680d1540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:13.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:13.379+0000 7f4a6fc27640 1 -- 172.21.15.27:0/229899217 shutdown_connections 2024-09-12T03:22:13.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:13.379+0000 7f4a6fc27640 1 -- 172.21.15.27:0/229899217 wait complete. 2024-09-12T03:22:13.390 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-12T03:22:13.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:13.496+0000 7f834198d640 1 Processor -- start 2024-09-12T03:22:13.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:13.496+0000 7f834198d640 1 -- start start 2024-09-12T03:22:13.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:13.497+0000 7f834198d640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f833c15e650 0x7f833c0d1d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:13.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:13.497+0000 7f834198d640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f833c06cc40 0x7f833c06d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:13.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:13.497+0000 7f834198d640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f833c06d5d0 con 0x7f833c06db30 2024-09-12T03:22:13.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:13.497+0000 7f834198d640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f833c0d2280 con 0x7f833c06cc40 2024-09-12T03:22:13.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:13.497+0000 7f834198d640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f833c06e670 con 0x7f833c15e650 2024-09-12T03:22:13.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:13.497+0000 7f833a7fc640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f833c06cc40 0x7f833c06d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:13.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:13.497+0000 7f833a7fc640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f833c06cc40 0x7f833c06d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41004/0 (socket says 172.21.15.27:41004) 2024-09-12T03:22:13.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:13.497+0000 7f833a7fc640 1 -- 172.21.15.27:0/2966705513 learned_addr learned my addr 172.21.15.27:0/2966705513 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:22:13.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:13.497+0000 7f833affd640 1 --2- 172.21.15.27:0/2966705513 >> v2:172.21.15.45:3300/0 conn(0x7f833c15e650 0x7f833c0d1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:13.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:13.497+0000 7f833a7fc640 1 -- 172.21.15.27:0/2966705513 >> v2:172.21.15.45:3300/0 conn(0x7f833c15e650 msgr2=0x7f833c0d1d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:13.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:13.497+0000 7f833a7fc640 1 --2- 172.21.15.27:0/2966705513 >> v2:172.21.15.45:3300/0 conn(0x7f833c15e650 0x7f833c0d1d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:13.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:13.497+0000 7f833a7fc640 1 -- 172.21.15.27:0/2966705513 >> v1:172.21.15.45:6789/0 conn(0x7f833c06db30 legacy=0x7f833c06df60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:13.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:13.497+0000 7f833a7fc640 1 -- 172.21.15.27:0/2966705513 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f833c06d850 con 0x7f833c06cc40 2024-09-12T03:22:13.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:13.498+0000 7f833a7fc640 1 --2- 172.21.15.27:0/2966705513 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f833c06cc40 0x7f833c06d090 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f833c075c50 tx=0x7f832402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc3c87e8e30e52a0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:13.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:13.498+0000 7f8339ffb640 1 -- 172.21.15.27:0/2966705513 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8324037070 con 0x7f833c06cc40 2024-09-12T03:22:13.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:13.498+0000 7f8339ffb640 1 -- 172.21.15.27:0/2966705513 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8324032440 con 0x7f833c06cc40 2024-09-12T03:22:13.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:13.498+0000 7f8339ffb640 1 -- 172.21.15.27:0/2966705513 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8324032810 con 0x7f833c06cc40 2024-09-12T03:22:13.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:13.499+0000 7f834198d640 1 -- 172.21.15.27:0/2966705513 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f833c06cc40 msgr2=0x7f833c06d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:13.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:13.499+0000 7f834198d640 1 --2- 172.21.15.27:0/2966705513 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f833c06cc40 0x7f833c06d090 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f833c075c50 tx=0x7f832402a1a0 comp rx=0 tx=0).stop 2024-09-12T03:22:13.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:13.499+0000 7f834198d640 1 -- 172.21.15.27:0/2966705513 shutdown_connections 2024-09-12T03:22:13.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:13.499+0000 7f834198d640 1 --2- 172.21.15.27:0/2966705513 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f833c06cc40 0x7f833c06d090 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:13.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:13.499+0000 7f834198d640 1 --2- 172.21.15.27:0/2966705513 >> v2:172.21.15.45:3300/0 conn(0x7f833c15e650 0x7f833c0d1d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:13.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:13.499+0000 7f834198d640 1 -- 172.21.15.27:0/2966705513 >> 172.21.15.27:0/2966705513 conn(0x7f833c06f320 msgr2=0x7f833c06f700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:13.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:13.499+0000 7f834198d640 1 -- 172.21.15.27:0/2966705513 shutdown_connections 2024-09-12T03:22:13.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:13.499+0000 7f834198d640 1 -- 172.21.15.27:0/2966705513 wait complete. 2024-09-12T03:22:13.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:13.500+0000 7f834198d640 1 Processor -- start 2024-09-12T03:22:13.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:13.500+0000 7f834198d640 1 -- start start 2024-09-12T03:22:13.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:13.501+0000 7f834198d640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f833c06cc40 0x7f833c0acb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:13.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:13.501+0000 7f834198d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f833c06db30 0x7f833c0ce990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:13.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:13.501+0000 7f834198d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f833c15e650 0x7f833c15b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:13.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:13.501+0000 7f834198d640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f833c15ea30 con 0x7f833c15e650 2024-09-12T03:22:13.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:13.501+0000 7f834198d640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f833c0d1540 con 0x7f833c06db30 2024-09-12T03:22:13.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:13.501+0000 7f834198d640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f833c15ebb0 con 0x7f833c06cc40 2024-09-12T03:22:13.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:13.501+0000 7f833b7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f833c15e650 0x7f833c15b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:13.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:13.501+0000 7f833b7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f833c15e650 0x7f833c15b490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41006/0 (socket says 172.21.15.27:41006) 2024-09-12T03:22:13.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:13.501+0000 7f833b7fe640 1 -- 172.21.15.27:0/3118346224 learned_addr learned my addr 172.21.15.27:0/3118346224 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:22:13.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:13.501+0000 7f833affd640 1 --2- 172.21.15.27:0/3118346224 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f833c06cc40 0x7f833c0acb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:13.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:13.501+0000 7f833a7fc640 1 --2- 172.21.15.27:0/3118346224 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f833c06db30 0x7f833c0ce990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:13.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:13.501+0000 7f833b7fe640 1 -- 172.21.15.27:0/3118346224 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f833c06cc40 msgr2=0x7f833c0acb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:13.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:13.501+0000 7f833b7fe640 1 --2- 172.21.15.27:0/3118346224 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f833c06cc40 0x7f833c0acb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:13.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:13.501+0000 7f833b7fe640 1 -- 172.21.15.27:0/3118346224 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f833c06db30 msgr2=0x7f833c0ce990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:13.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:13.501+0000 7f833b7fe640 1 --2- 172.21.15.27:0/3118346224 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f833c06db30 0x7f833c0ce990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:13.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:13.501+0000 7f833b7fe640 1 -- 172.21.15.27:0/3118346224 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f833c15b9d0 con 0x7f833c15e650 2024-09-12T03:22:13.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:13.502+0000 7f833b7fe640 1 --2- 172.21.15.27:0/3118346224 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f833c15e650 0x7f833c15b490 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f832c00b9e0 tx=0x7f832c00bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:13.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:13.502+0000 7f8333fff640 1 -- 172.21.15.27:0/3118346224 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f832c003b70 con 0x7f833c15e650 2024-09-12T03:22:13.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:13.502+0000 7f8333fff640 1 -- 172.21.15.27:0/3118346224 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f832c003d10 con 0x7f833c15e650 2024-09-12T03:22:13.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:13.502+0000 7f834198d640 1 -- 172.21.15.27:0/3118346224 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f833c0cf090 con 0x7f833c15e650 2024-09-12T03:22:13.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:13.502+0000 7f8333fff640 1 -- 172.21.15.27:0/3118346224 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f832c00cae0 con 0x7f833c15e650 2024-09-12T03:22:13.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:13.502+0000 7f834198d640 1 -- 172.21.15.27:0/3118346224 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f833c0cf650 con 0x7f833c15e650 2024-09-12T03:22:13.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:13.503+0000 7f834198d640 1 -- 172.21.15.27:0/3118346224 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f833c06d090 con 0x7f833c15e650 2024-09-12T03:22:13.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:13.503+0000 7f8333fff640 1 -- 172.21.15.27:0/3118346224 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 21) ==== 91008+0+0 (secure 0 0 0) 0x7f832c00cc80 con 0x7f833c15e650 2024-09-12T03:22:13.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:13.504+0000 7f8333fff640 1 --2- 172.21.15.27:0/3118346224 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f831c06c3d0 0x7f831c06e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:13.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:13.504+0000 7f8333fff640 1 -- 172.21.15.27:0/3118346224 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 4999+0+0 (secure 0 0 0) 0x7f832c08b390 con 0x7f833c15e650 2024-09-12T03:22:13.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:13.504+0000 7f833affd640 1 --2- 172.21.15.27:0/3118346224 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f831c06c3d0 0x7f831c06e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:13.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:13.504+0000 7f833affd640 1 --2- 172.21.15.27:0/3118346224 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f831c06c3d0 0x7f831c06e890 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f8334006fd0 tx=0x7f8334008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:13.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:13.508+0000 7f8333fff640 1 -- 172.21.15.27:0/3118346224 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f833c15b9d0 con 0x7f833c15e650 2024-09-12T03:22:13.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:13.683+0000 7f834198d640 1 -- 172.21.15.27:0/3118346224 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f833c0b6a60 con 0x7f833c15e650 2024-09-12T03:22:13.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:13.686+0000 7f8333fff640 1 -- 172.21.15.27:0/3118346224 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v24)=0 v24) ==== 122+0+0 (secure 0 0 0) 0x7f832c059790 con 0x7f833c15e650 2024-09-12T03:22:13.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:13.689+0000 7f834198d640 1 -- 172.21.15.27:0/3118346224 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f831c06c3d0 msgr2=0x7f831c06e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:13.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:13.689+0000 7f834198d640 1 --2- 172.21.15.27:0/3118346224 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f831c06c3d0 0x7f831c06e890 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f8334006fd0 tx=0x7f8334008040 comp rx=0 tx=0).stop 2024-09-12T03:22:13.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:13.689+0000 7f834198d640 1 -- 172.21.15.27:0/3118346224 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f833c15e650 msgr2=0x7f833c15b490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:13.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:13.689+0000 7f834198d640 1 --2- 172.21.15.27:0/3118346224 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f833c15e650 0x7f833c15b490 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f832c00b9e0 tx=0x7f832c00bea0 comp rx=0 tx=0).stop 2024-09-12T03:22:13.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:13.689+0000 7f834198d640 1 -- 172.21.15.27:0/3118346224 shutdown_connections 2024-09-12T03:22:13.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:13.689+0000 7f834198d640 1 --2- 172.21.15.27:0/3118346224 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f831c06c3d0 0x7f831c06e890 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:13.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:13.689+0000 7f834198d640 1 --2- 172.21.15.27:0/3118346224 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f833c15e650 0x7f833c15b490 unknown :-1 s=CLOSED pgs=1200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:13.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:13.689+0000 7f834198d640 1 --2- 172.21.15.27:0/3118346224 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f833c06db30 0x7f833c0ce990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:13.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:13.689+0000 7f834198d640 1 --2- 172.21.15.27:0/3118346224 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f833c06cc40 0x7f833c0acb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:13.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:13.689+0000 7f834198d640 1 -- 172.21.15.27:0/3118346224 >> 172.21.15.27:0/3118346224 conn(0x7f833c06f320 msgr2=0x7f833c0736c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:13.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:13.689+0000 7f834198d640 1 -- 172.21.15.27:0/3118346224 shutdown_connections 2024-09-12T03:22:13.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:13.689+0000 7f834198d640 1 -- 172.21.15.27:0/3118346224 wait complete. 2024-09-12T03:22:13.700 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-12T03:22:13.700 DEBUG:teuthology.orchestra.run.smithi027:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-12T03:22:13.745 DEBUG:teuthology.orchestra.run.smithi045:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-12T03:22:13.793 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-12T03:22:13.794 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-12T03:22:13.794 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T03:22:13.810 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056004,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497432k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16324 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248716k,nr_inodes=812179,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497432k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-12T03:22:13.811 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-12T03:22:13.811 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T03:22:13.831 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056795,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16602 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249348k,nr_inodes=812337,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-12T03:22:13.832 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-12T03:22:13.832 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T03:22:13.867 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056004,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497432k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16324 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248716k,nr_inodes=812179,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497432k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-12T03:22:13.867 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-12T03:22:13.867 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T03:22:13.892 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056795,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16602 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249348k,nr_inodes=812337,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-12T03:22:13.892 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-12T03:22:13.892 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-12T03:22:14.010 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:14.010+0000 7ff0dc677640 1 Processor -- start 2024-09-12T03:22:14.011 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:14.010+0000 7ff0dc677640 1 -- start start 2024-09-12T03:22:14.012 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:14.011+0000 7ff0dc677640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7ff0d415eb60 0x7ff0d40d1280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:14.012 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:14.011+0000 7ff0dc677640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ff0d40d17c0 0x7ff0d40b3390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:14.013 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:14.011+0000 7ff0dc677640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff0d40b6690 con 0x7ff0d415e780 2024-09-12T03:22:14.013 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:14.011+0000 7ff0dc677640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7ff0d40d1c30 con 0x7ff0d415eb60 2024-09-12T03:22:14.013 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:14.011+0000 7ff0dc677640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7ff0d40b7950 con 0x7ff0d40d17c0 2024-09-12T03:22:14.013 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:14.011+0000 7ff0d9c12640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7ff0d415eb60 0x7ff0d40d1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:14.013 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:14.011+0000 7ff0da413640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7ff0d415e780 0x7ff0d40d3e70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:35102/0 (socket says 172.21.15.27:35102) 2024-09-12T03:22:14.013 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:14.011+0000 7ff0da413640 1 -- 172.21.15.27:0/4218167395 learned_addr learned my addr 172.21.15.27:0/4218167395 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:22:14.013 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:14.011+0000 7ff0dac14640 1 --2- 172.21.15.27:0/4218167395 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ff0d40d17c0 0x7ff0d40b3390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:14.013 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:14.012+0000 7ff0d9c12640 1 -- 172.21.15.27:0/4218167395 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ff0d40d17c0 msgr2=0x7ff0d40b3390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:14.013 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:14.012+0000 7ff0d9c12640 1 --2- 172.21.15.27:0/4218167395 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ff0d40d17c0 0x7ff0d40b3390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:14.013 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:14.012+0000 7ff0d9c12640 1 -- 172.21.15.27:0/4218167395 >> v1:172.21.15.27:6789/0 conn(0x7ff0d415e780 legacy=0x7ff0d40d3e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:14.013 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:14.012+0000 7ff0d9c12640 1 -- 172.21.15.27:0/4218167395 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0d40b6970 con 0x7ff0d415eb60 2024-09-12T03:22:14.013 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:14.012+0000 7ff0d9c12640 1 --2- 172.21.15.27:0/4218167395 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff0d415eb60 0x7ff0d40d1280 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7ff0c40097d0 tx=0x7ff0c402a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e0b5668fc7f15d2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:14.014 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:14.012+0000 7ff0d9411640 1 -- 172.21.15.27:0/4218167395 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0c4037070 con 0x7ff0d415eb60 2024-09-12T03:22:14.014 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:14.012+0000 7ff0d9411640 1 -- 172.21.15.27:0/4218167395 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0c4032440 con 0x7ff0d415eb60 2024-09-12T03:22:14.014 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:14.013+0000 7ff0d9411640 1 -- 172.21.15.27:0/4218167395 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0c4032810 con 0x7ff0d415eb60 2024-09-12T03:22:14.016 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:14.013+0000 7ff0dc677640 1 -- 172.21.15.27:0/4218167395 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff0d415eb60 msgr2=0x7ff0d40d1280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:14.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:14.013+0000 7ff0dc677640 1 --2- 172.21.15.27:0/4218167395 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff0d415eb60 0x7ff0d40d1280 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7ff0c40097d0 tx=0x7ff0c402a1c0 comp rx=0 tx=0).stop 2024-09-12T03:22:14.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:14.013+0000 7ff0dc677640 1 -- 172.21.15.27:0/4218167395 shutdown_connections 2024-09-12T03:22:14.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:14.013+0000 7ff0dc677640 1 --2- 172.21.15.27:0/4218167395 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ff0d40d17c0 0x7ff0d40b3390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:14.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:14.013+0000 7ff0dc677640 1 --2- 172.21.15.27:0/4218167395 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff0d415eb60 0x7ff0d40d1280 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:14.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:14.013+0000 7ff0dc677640 1 -- 172.21.15.27:0/4218167395 >> 172.21.15.27:0/4218167395 conn(0x7ff0d406da60 msgr2=0x7ff0d406de70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:14.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:14.013+0000 7ff0dc677640 1 -- 172.21.15.27:0/4218167395 shutdown_connections 2024-09-12T03:22:14.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:14.013+0000 7ff0dc677640 1 -- 172.21.15.27:0/4218167395 wait complete. 2024-09-12T03:22:14.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:14.014+0000 7ff0dc677640 1 Processor -- start 2024-09-12T03:22:14.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:14.014+0000 7ff0dc677640 1 -- start start 2024-09-12T03:22:14.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:14.015+0000 7ff0dc677640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ff0d40d17c0 0x7ff0d415b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:14.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:14.015+0000 7ff0dc677640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff0d415e780 0x7ff0d4158d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:14.018 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:14.015+0000 7ff0dc677640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff0d415eb60 0x7ff0d40caa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:14.018 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:14.015+0000 7ff0dc677640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7ff0d40b6f30 con 0x7ff0d415eb60 2024-09-12T03:22:14.018 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:14.015+0000 7ff0dc677640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ff0d40b7210 con 0x7ff0d415e780 2024-09-12T03:22:14.018 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:14.015+0000 7ff0dc677640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7ff0d40b6340 con 0x7ff0d40d17c0 2024-09-12T03:22:14.018 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:14.015+0000 7ff0dac14640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff0d415eb60 0x7ff0d40caa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:14.018 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:14.015+0000 7ff0dac14640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff0d415eb60 0x7ff0d40caa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41022/0 (socket says 172.21.15.27:41022) 2024-09-12T03:22:14.018 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:14.015+0000 7ff0dac14640 1 -- 172.21.15.27:0/3230449182 learned_addr learned my addr 172.21.15.27:0/3230449182 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:22:14.018 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:14.016+0000 7ff0da413640 1 --2- 172.21.15.27:0/3230449182 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ff0d40d17c0 0x7ff0d415b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:14.018 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:14.016+0000 7ff0d9c12640 1 --2- 172.21.15.27:0/3230449182 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff0d415e780 0x7ff0d4158d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:14.018 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:14.016+0000 7ff0dac14640 1 -- 172.21.15.27:0/3230449182 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ff0d40d17c0 msgr2=0x7ff0d415b560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:14.018 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:14.016+0000 7ff0dac14640 1 --2- 172.21.15.27:0/3230449182 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ff0d40d17c0 0x7ff0d415b560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:14.018 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:14.016+0000 7ff0dac14640 1 -- 172.21.15.27:0/3230449182 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff0d415e780 msgr2=0x7ff0d4158d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:14.018 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:14.016+0000 7ff0dac14640 1 --2- 172.21.15.27:0/3230449182 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff0d415e780 0x7ff0d4158d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:14.019 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:14.016+0000 7ff0dac14640 1 -- 172.21.15.27:0/3230449182 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0d40cbbe0 con 0x7ff0d415eb60 2024-09-12T03:22:14.019 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:14.016+0000 7ff0da413640 1 --2- 172.21.15.27:0/3230449182 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ff0d40d17c0 0x7ff0d415b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:22:14.019 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:14.016+0000 7ff0d9c12640 1 --2- 172.21.15.27:0/3230449182 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff0d415e780 0x7ff0d4158d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:22:14.019 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:14.016+0000 7ff0dac14640 1 --2- 172.21.15.27:0/3230449182 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff0d415eb60 0x7ff0d40caa10 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7ff0cc007e40 tx=0x7ff0cc00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:14.019 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:14.016+0000 7ff0caffd640 1 -- 172.21.15.27:0/3230449182 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0cc014620 con 0x7ff0d415eb60 2024-09-12T03:22:14.019 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:14.016+0000 7ff0caffd640 1 -- 172.21.15.27:0/3230449182 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0cc019070 con 0x7ff0d415eb60 2024-09-12T03:22:14.019 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:14.016+0000 7ff0dc677640 1 -- 172.21.15.27:0/3230449182 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0d40cbec0 con 0x7ff0d415eb60 2024-09-12T03:22:14.019 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:14.016+0000 7ff0caffd640 1 -- 172.21.15.27:0/3230449182 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0cc014a20 con 0x7ff0d415eb60 2024-09-12T03:22:14.019 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:14.016+0000 7ff0dc677640 1 -- 172.21.15.27:0/3230449182 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0d40cc480 con 0x7ff0d415eb60 2024-09-12T03:22:14.020 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:14.017+0000 7ff0dc677640 1 -- 172.21.15.27:0/3230449182 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0d40d1280 con 0x7ff0d415eb60 2024-09-12T03:22:14.020 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:14.018+0000 7ff0caffd640 1 -- 172.21.15.27:0/3230449182 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 21) ==== 91008+0+0 (secure 0 0 0) 0x7ff0cc014bc0 con 0x7ff0d415eb60 2024-09-12T03:22:14.020 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:14.018+0000 7ff0caffd640 1 --2- 172.21.15.27:0/3230449182 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7ff0b806c400 0x7ff0b806e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:14.020 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:14.018+0000 7ff0caffd640 1 -- 172.21.15.27:0/3230449182 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 4999+0+0 (secure 0 0 0) 0x7ff0cc08be40 con 0x7ff0d415eb60 2024-09-12T03:22:14.020 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:14.018+0000 7ff0da413640 1 --2- 172.21.15.27:0/3230449182 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7ff0b806c400 0x7ff0b806e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:14.021 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:14.019+0000 7ff0da413640 1 --2- 172.21.15.27:0/3230449182 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7ff0b806c400 0x7ff0b806e8c0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7ff0d0000f60 tx=0x7ff0d0009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:14.022 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:14.023+0000 7ff0caffd640 1 -- 172.21.15.27:0/3230449182 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7ff0cc090070 con 0x7ff0d415eb60 2024-09-12T03:22:14.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:14.235+0000 7ff0dc677640 1 -- 172.21.15.27:0/3230449182 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff0d4161df0 con 0x7ff0d415eb60 2024-09-12T03:22:14.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:14.235+0000 7ff0caffd640 1 -- 172.21.15.27:0/3230449182 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 70 v70) ==== 94+0+3911 (secure 0 0 0) 0x7ff0cc05a2d0 con 0x7ff0d415eb60 2024-09-12T03:22:14.235 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:22:14.235 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":70,"btime":"2024-09-12T03:22:09:360726+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6206,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/2249506350","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2249506350},{"type":"v1","addr":"172.21.15.45:6834","nonce":2249506350}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63},{"gid":6209,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/2292759672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":2292759672},{"type":"v1","addr":"172.21.15.45:6835","nonce":2292759672}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63},{"gid":7299,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6834/4021825986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":4021825986},{"type":"v1","addr":"172.21.15.27:6834","nonce":4021825986}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":62},{"gid":7638,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/1094150439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":1094150439},{"type":"v1","addr":"172.21.15.27:6835","nonce":1094150439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70}],"filesystems":[]} 2024-09-12T03:22:14.236 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 70 2024-09-12T03:22:14.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:14.237+0000 7ff0dc677640 1 -- 172.21.15.27:0/3230449182 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7ff0b806c400 msgr2=0x7ff0b806e8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:14.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:14.237+0000 7ff0dc677640 1 --2- 172.21.15.27:0/3230449182 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7ff0b806c400 0x7ff0b806e8c0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7ff0d0000f60 tx=0x7ff0d0009040 comp rx=0 tx=0).stop 2024-09-12T03:22:14.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:14.237+0000 7ff0dc677640 1 -- 172.21.15.27:0/3230449182 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff0d415eb60 msgr2=0x7ff0d40caa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:14.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:14.237+0000 7ff0dc677640 1 --2- 172.21.15.27:0/3230449182 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff0d415eb60 0x7ff0d40caa10 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7ff0cc007e40 tx=0x7ff0cc00e4b0 comp rx=0 tx=0).stop 2024-09-12T03:22:14.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:14.237+0000 7ff0dc677640 1 -- 172.21.15.27:0/3230449182 shutdown_connections 2024-09-12T03:22:14.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:14.237+0000 7ff0dc677640 1 --2- 172.21.15.27:0/3230449182 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7ff0b806c400 0x7ff0b806e8c0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:14.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:14.237+0000 7ff0dc677640 1 --2- 172.21.15.27:0/3230449182 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff0d415eb60 0x7ff0d40caa10 unknown :-1 s=CLOSED pgs=1203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:14.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:14.237+0000 7ff0dc677640 1 --2- 172.21.15.27:0/3230449182 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff0d415e780 0x7ff0d4158d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:14.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:14.237+0000 7ff0dc677640 1 --2- 172.21.15.27:0/3230449182 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ff0d40d17c0 0x7ff0d415b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:14.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:14.237+0000 7ff0dc677640 1 -- 172.21.15.27:0/3230449182 >> 172.21.15.27:0/3230449182 conn(0x7ff0d406da60 msgr2=0x7ff0d415f5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:14.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:14.238+0000 7ff0dc677640 1 -- 172.21.15.27:0/3230449182 shutdown_connections 2024-09-12T03:22:14.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:14.238+0000 7ff0dc677640 1 -- 172.21.15.27:0/3230449182 wait complete. 2024-09-12T03:22:14.249 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-12T03:22:14.249 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-12T03:22:14.250 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-12T03:22:14.250 INFO:teuthology.orchestra.run:waiting for 300 2024-09-12T03:22:14.250 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-12T03:22:14.250 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-12T03:22:14.250 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-12T03:22:14.250 INFO:teuthology.orchestra.run:waiting for 300 2024-09-12T03:22:14.251 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-12T03:22:14.251 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-12T03:22:14.251 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-12T03:22:14.251 INFO:teuthology.orchestra.run:waiting for 300 2024-09-12T03:22:14.251 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-12T03:22:14.252 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-12T03:22:14.252 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-12T03:22:14.252 INFO:teuthology.orchestra.run:waiting for 300 2024-09-12T03:22:14.259 INFO:tasks.ceph.mds.a:Stopped 2024-09-12T03:22:14.259 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-12T03:22:14.260 INFO:tasks.ceph.mds.c:Stopped 2024-09-12T03:22:14.260 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-12T03:22:14.263 INFO:tasks.ceph.mds.b:Stopped 2024-09-12T03:22:14.263 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-12T03:22:14.264 INFO:tasks.ceph.mds.d:Stopped 2024-09-12T03:22:14.264 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-12T03:22:14.301 INFO:tasks.ceph.mds.a:Started 2024-09-12T03:22:14.302 DEBUG:teuthology.parallel:result is None 2024-09-12T03:22:14.302 INFO:tasks.ceph.mds.c:Started 2024-09-12T03:22:14.302 DEBUG:teuthology.parallel:result is None 2024-09-12T03:22:14.305 INFO:tasks.ceph.mds.b:Started 2024-09-12T03:22:14.305 DEBUG:teuthology.parallel:result is None 2024-09-12T03:22:14.306 INFO:tasks.ceph.mds.d:Started 2024-09-12T03:22:14.306 DEBUG:teuthology.parallel:result is None 2024-09-12T03:22:14.306 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-12T03:22:14.382 INFO:tasks.ceph.mds.c.smithi027.stdout:starting mds.c at 2024-09-12T03:22:14.382 INFO:tasks.ceph.mds.a.smithi027.stdout:starting mds.a at 2024-09-12T03:22:14.390 INFO:tasks.ceph.mds.b.smithi045.stdout:starting mds.b at 2024-09-12T03:22:14.390 INFO:tasks.ceph.mds.d.smithi045.stdout:starting mds.d at 2024-09-12T03:22:14.415 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:14.415+0000 7f26e13d6640 1 Processor -- start 2024-09-12T03:22:14.415 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:14.415+0000 7f26e13d6640 1 -- start start 2024-09-12T03:22:14.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:14.416+0000 7f26e13d6640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f26dc0d5200 0x7f26dc0d1010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:14.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:14.416+0000 7f26e13d6640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f26dc0d1550 0x7f26dc0b8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:14.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:14.416+0000 7f26e13d6640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f26dc0d19c0 con 0x7f26dc15eb90 2024-09-12T03:22:14.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:14.416+0000 7f26e13d6640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f26dc0b7d20 con 0x7f26dc0d1550 2024-09-12T03:22:14.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:14.416+0000 7f26e13d6640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f26dc0d42e0 con 0x7f26dc0d5200 2024-09-12T03:22:14.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:14.416+0000 7f26db7fe640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f26dc0d1550 0x7f26dc0b8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:14.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:14.416+0000 7f26db7fe640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f26dc0d1550 0x7f26dc0b8190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41080/0 (socket says 172.21.15.27:41080) 2024-09-12T03:22:14.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:14.416+0000 7f26db7fe640 1 -- 172.21.15.27:0/2237792629 learned_addr learned my addr 172.21.15.27:0/2237792629 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:22:14.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:14.417+0000 7f26da7fc640 1 --2- 172.21.15.27:0/2237792629 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f26dc0d5200 0x7f26dc0d1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:14.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:14.417+0000 7f26db7fe640 1 -- 172.21.15.27:0/2237792629 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f26dc0d5200 msgr2=0x7f26dc0d1010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:14.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:14.417+0000 7f26db7fe640 1 --2- 172.21.15.27:0/2237792629 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f26dc0d5200 0x7f26dc0d1010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:14.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:14.417+0000 7f26db7fe640 1 -- 172.21.15.27:0/2237792629 >> v1:172.21.15.45:6789/0 conn(0x7f26dc15eb90 legacy=0x7f26dc0d2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:14.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:14.417+0000 7f26db7fe640 1 -- 172.21.15.27:0/2237792629 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26dc0d3760 con 0x7f26dc0d1550 2024-09-12T03:22:14.419 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:14.417+0000 7f26db7fe640 1 --2- 172.21.15.27:0/2237792629 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f26dc0d1550 0x7f26dc0b8190 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f26d4009870 tx=0x7f26d402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b134a03327332ab3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:14.419 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:14.418+0000 7f26d9ffb640 1 -- 172.21.15.27:0/2237792629 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26d4037070 con 0x7f26dc0d1550 2024-09-12T03:22:14.420 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:14.418+0000 7f26d9ffb640 1 -- 172.21.15.27:0/2237792629 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26d4032440 con 0x7f26dc0d1550 2024-09-12T03:22:14.420 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:14.418+0000 7f26d9ffb640 1 -- 172.21.15.27:0/2237792629 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26d4032810 con 0x7f26dc0d1550 2024-09-12T03:22:14.420 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:14.418+0000 7f26e13d6640 1 -- 172.21.15.27:0/2237792629 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f26dc0d1550 msgr2=0x7f26dc0b8190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:14.420 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:14.418+0000 7f26e13d6640 1 --2- 172.21.15.27:0/2237792629 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f26dc0d1550 0x7f26dc0b8190 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f26d4009870 tx=0x7f26d402a210 comp rx=0 tx=0).stop 2024-09-12T03:22:14.420 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:14.418+0000 7f26e13d6640 1 -- 172.21.15.27:0/2237792629 shutdown_connections 2024-09-12T03:22:14.420 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:14.418+0000 7f26e13d6640 1 --2- 172.21.15.27:0/2237792629 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f26dc0d1550 0x7f26dc0b8190 unknown :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:14.420 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:14.418+0000 7f26e13d6640 1 --2- 172.21.15.27:0/2237792629 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f26dc0d5200 0x7f26dc0d1010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:14.420 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:14.418+0000 7f26e13d6640 1 -- 172.21.15.27:0/2237792629 >> 172.21.15.27:0/2237792629 conn(0x7f26dc06dce0 msgr2=0x7f26dc06e0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:14.421 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:14.418+0000 7f26e13d6640 1 -- 172.21.15.27:0/2237792629 shutdown_connections 2024-09-12T03:22:14.421 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:14.418+0000 7f26e13d6640 1 -- 172.21.15.27:0/2237792629 wait complete. 2024-09-12T03:22:14.421 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:14.419+0000 7f26e13d6640 1 Processor -- start 2024-09-12T03:22:14.421 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:14.420+0000 7f26e13d6640 1 -- start start 2024-09-12T03:22:14.423 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:14.420+0000 7f26e13d6640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f26dc0d1550 0x7f26dc15aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:14.423 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:14.420+0000 7f26e13d6640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f26dc0d5200 0x7f26dc14ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:14.424 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:14.420+0000 7f26e13d6640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f26dc15eb90 0x7f26dc1595e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:14.424 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:14.420+0000 7f26e13d6640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f26dc0d3d20 con 0x7f26dc0d5200 2024-09-12T03:22:14.424 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:14.420+0000 7f26e13d6640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f26dc0d3500 con 0x7f26dc15eb90 2024-09-12T03:22:14.424 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:14.420+0000 7f26e13d6640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f26dc0d4000 con 0x7f26dc0d1550 2024-09-12T03:22:14.424 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:14.420+0000 7f26da7fc640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f26dc0d5200 0x7f26dc14ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:14.424 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:14.420+0000 7f26daffd640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f26dc0d1550 0x7f26dc15aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:14.424 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:14.420+0000 7f26da7fc640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f26dc0d5200 0x7f26dc14ebe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41082/0 (socket says 172.21.15.27:41082) 2024-09-12T03:22:14.424 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:14.420+0000 7f26da7fc640 1 -- 172.21.15.27:0/4096504424 learned_addr learned my addr 172.21.15.27:0/4096504424 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:22:14.424 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:14.420+0000 7f26db7fe640 1 --2- 172.21.15.27:0/4096504424 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f26dc15eb90 0x7f26dc1595e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:14.425 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:14.421+0000 7f26da7fc640 1 -- 172.21.15.27:0/4096504424 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f26dc0d1550 msgr2=0x7f26dc15aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:14.425 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:14.421+0000 7f26da7fc640 1 --2- 172.21.15.27:0/4096504424 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f26dc0d1550 0x7f26dc15aeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:14.425 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:14.421+0000 7f26da7fc640 1 -- 172.21.15.27:0/4096504424 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f26dc15eb90 msgr2=0x7f26dc1595e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:14.425 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:14.421+0000 7f26da7fc640 1 --2- 172.21.15.27:0/4096504424 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f26dc15eb90 0x7f26dc1595e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:14.425 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:14.421+0000 7f26da7fc640 1 -- 172.21.15.27:0/4096504424 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26dc159e10 con 0x7f26dc0d5200 2024-09-12T03:22:14.425 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:14.421+0000 7f26daffd640 1 --2- 172.21.15.27:0/4096504424 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f26dc0d1550 0x7f26dc15aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:22:14.425 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:14.421+0000 7f26da7fc640 1 --2- 172.21.15.27:0/4096504424 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f26dc0d5200 0x7f26dc14ebe0 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f26c400c7f0 tx=0x7f26c400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:14.425 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:14.421+0000 7f26d37fe640 1 -- 172.21.15.27:0/4096504424 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26c4015070 con 0x7f26dc0d5200 2024-09-12T03:22:14.425 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:14.421+0000 7f26d37fe640 1 -- 172.21.15.27:0/4096504424 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26c4004220 con 0x7f26dc0d5200 2024-09-12T03:22:14.425 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:14.421+0000 7f26e13d6640 1 -- 172.21.15.27:0/4096504424 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26dc15a0f0 con 0x7f26dc0d5200 2024-09-12T03:22:14.425 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:14.421+0000 7f26d37fe640 1 -- 172.21.15.27:0/4096504424 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26c4007620 con 0x7f26dc0d5200 2024-09-12T03:22:14.425 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:14.421+0000 7f26e13d6640 1 -- 172.21.15.27:0/4096504424 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26dc15a6b0 con 0x7f26dc0d5200 2024-09-12T03:22:14.425 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:14.423+0000 7f26e13d6640 1 -- 172.21.15.27:0/4096504424 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26dc15f0e0 con 0x7f26dc0d5200 2024-09-12T03:22:14.426 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:14.423+0000 7f26d37fe640 1 -- 172.21.15.27:0/4096504424 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 21) ==== 91008+0+0 (secure 0 0 0) 0x7f26c40077c0 con 0x7f26dc0d5200 2024-09-12T03:22:14.426 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:14.423+0000 7f26d37fe640 1 --2- 172.21.15.27:0/4096504424 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f26bc06c3d0 0x7f26bc06e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:14.426 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:14.423+0000 7f26d37fe640 1 -- 172.21.15.27:0/4096504424 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 4999+0+0 (secure 0 0 0) 0x7f26c408ba80 con 0x7f26dc0d5200 2024-09-12T03:22:14.426 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:14.423+0000 7f26daffd640 1 --2- 172.21.15.27:0/4096504424 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f26bc06c3d0 0x7f26bc06e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:14.426 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:14.424+0000 7f26daffd640 1 --2- 172.21.15.27:0/4096504424 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f26bc06c3d0 0x7f26bc06e890 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f26cc00b820 tx=0x7f26cc00d040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:14.427 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:14.428+0000 7f26d37fe640 1 -- 172.21.15.27:0/4096504424 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f26c4059510 con 0x7f26dc0d5200 2024-09-12T03:22:14.617 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:14.617+0000 7f26e13d6640 1 -- 172.21.15.27:0/4096504424 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f26dc1620a0 con 0x7f26dc0d5200 2024-09-12T03:22:15.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:15.388+0000 7f26d37fe640 1 -- 172.21.15.27:0/4096504424 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v120) ==== 97+0+0 (secure 0 0 0) 0x7f26c405d810 con 0x7f26dc0d5200 2024-09-12T03:22:15.389 INFO:teuthology.orchestra.run.smithi027.stderr: removed all blocklist entries 2024-09-12T03:22:15.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:15.391+0000 7f26e13d6640 1 -- 172.21.15.27:0/4096504424 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f26bc06c3d0 msgr2=0x7f26bc06e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:15.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:15.391+0000 7f26e13d6640 1 --2- 172.21.15.27:0/4096504424 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f26bc06c3d0 0x7f26bc06e890 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f26cc00b820 tx=0x7f26cc00d040 comp rx=0 tx=0).stop 2024-09-12T03:22:15.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:15.391+0000 7f26e13d6640 1 -- 172.21.15.27:0/4096504424 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f26dc0d5200 msgr2=0x7f26dc14ebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:15.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:15.391+0000 7f26e13d6640 1 --2- 172.21.15.27:0/4096504424 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f26dc0d5200 0x7f26dc14ebe0 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f26c400c7f0 tx=0x7f26c400ccb0 comp rx=0 tx=0).stop 2024-09-12T03:22:15.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:15.392+0000 7f26e13d6640 1 -- 172.21.15.27:0/4096504424 shutdown_connections 2024-09-12T03:22:15.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:15.392+0000 7f26e13d6640 1 --2- 172.21.15.27:0/4096504424 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f26dc15eb90 0x7f26dc1595e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:15.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:15.392+0000 7f26e13d6640 1 --2- 172.21.15.27:0/4096504424 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f26bc06c3d0 0x7f26bc06e890 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:15.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:15.392+0000 7f26e13d6640 1 --2- 172.21.15.27:0/4096504424 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f26dc0d5200 0x7f26dc14ebe0 unknown :-1 s=CLOSED pgs=1212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:15.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:15.392+0000 7f26e13d6640 1 --2- 172.21.15.27:0/4096504424 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f26dc0d1550 0x7f26dc15aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:15.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:15.392+0000 7f26e13d6640 1 -- 172.21.15.27:0/4096504424 >> 172.21.15.27:0/4096504424 conn(0x7f26dc06dce0 msgr2=0x7f26dc06e150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:15.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:15.392+0000 7f26e13d6640 1 -- 172.21.15.27:0/4096504424 shutdown_connections 2024-09-12T03:22:15.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:15.392+0000 7f26e13d6640 1 -- 172.21.15.27:0/4096504424 wait complete. 2024-09-12T03:22:15.407 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-12T03:22:15.519 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:15.516+0000 7f7d0e8ac640 1 Processor -- start 2024-09-12T03:22:15.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:15.516+0000 7f7d0e8ac640 1 -- start start 2024-09-12T03:22:15.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:15.517+0000 7f7d0e8ac640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f7d0815b2b0 0x7f7d080b3560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:15.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:15.517+0000 7f7d0e8ac640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7d080ce410 con 0x7f7d0806e9b0 2024-09-12T03:22:15.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:15.517+0000 7f7d0e8ac640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7d080732c0 con 0x7f7d08072d70 2024-09-12T03:22:15.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:15.517+0000 7f7d0e8ac640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f7d080720a0 con 0x7f7d0815b2b0 2024-09-12T03:22:15.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:15.517+0000 7f7d0ce49640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f7d0806e9b0 0x7f7d080cebb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:35116/0 (socket says 172.21.15.27:35116) 2024-09-12T03:22:15.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:15.517+0000 7f7d0ce49640 1 -- 172.21.15.27:0/1178136778 learned_addr learned my addr 172.21.15.27:0/1178136778 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:22:15.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:15.517+0000 7f7d06ffd640 1 -- 172.21.15.27:0/1178136778 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2775315057 0 0) 0x7f7d00001490 con 0x7f7d0806e9b0 2024-09-12T03:22:15.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:15.517+0000 7f7d07fff640 1 --2- 172.21.15.27:0/1178136778 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f7d0815b2b0 0x7f7d080b3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:15.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:15.517+0000 7f7d06ffd640 1 -- 172.21.15.27:0/1178136778 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7cf0005730 con 0x7f7d0806e9b0 2024-09-12T03:22:15.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:15.517+0000 7f7d06ffd640 1 -- 172.21.15.27:0/1178136778 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1802028934 0 0) 0x7f7cf0005730 con 0x7f7d0806e9b0 2024-09-12T03:22:15.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:15.517+0000 7f7d06ffd640 1 -- 172.21.15.27:0/1178136778 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f7d0815b2b0 msgr2=0x7f7d080b3560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:15.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:15.517+0000 7f7d06ffd640 1 --2- 172.21.15.27:0/1178136778 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f7d0815b2b0 0x7f7d080b3560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:15.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:15.517+0000 7f7d06ffd640 1 -- 172.21.15.27:0/1178136778 >> v1:172.21.15.45:6789/0 conn(0x7f7d08072d70 legacy=0x7f7d08072430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:15.522 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:15.517+0000 7f7d06ffd640 1 -- 172.21.15.27:0/1178136778 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d08071ae0 con 0x7f7d0806e9b0 2024-09-12T03:22:15.522 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:15.517+0000 7f7d06ffd640 1 -- 172.21.15.27:0/1178136778 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 847755213 0 0) 0x7f7d00001bc0 con 0x7f7d0806e9b0 2024-09-12T03:22:15.522 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:15.517+0000 7f7d06ffd640 1 -- 172.21.15.27:0/1178136778 >> v1:172.21.15.27:6789/0 conn(0x7f7d0806e9b0 legacy=0x7f7d080cebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:15.522 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:15.517+0000 7f7d06ffd640 1 --2- 172.21.15.27:0/1178136778 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f7d0815b2b0 0x7f7cf0005bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:15.522 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:15.517+0000 7f7d06ffd640 1 --2- 172.21.15.27:0/1178136778 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7cf0006170 0x7f7cf0006a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:15.522 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:15.517+0000 7f7d06ffd640 1 --2- 172.21.15.27:0/1178136778 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7cf0006f50 0x7f7cf00073d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:15.522 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:15.517+0000 7f7d06ffd640 1 -- 172.21.15.27:0/1178136778 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f7d080720a0 con 0x7f7cf0006f50 2024-09-12T03:22:15.522 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:15.517+0000 7f7d06ffd640 1 -- 172.21.15.27:0/1178136778 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f7cf000dcb0 con 0x7f7cf0006170 2024-09-12T03:22:15.522 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:15.517+0000 7f7d06ffd640 1 -- 172.21.15.27:0/1178136778 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f7cf000deb0 con 0x7f7d0815b2b0 2024-09-12T03:22:15.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:15.518+0000 7f7d0ce49640 1 --2- 172.21.15.27:0/1178136778 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7cf0006f50 0x7f7cf00073d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:15.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:15.518+0000 7f7d07fff640 1 --2- 172.21.15.27:0/1178136778 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f7d0815b2b0 0x7f7cf0005bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:15.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:15.518+0000 7f7d077fe640 1 --2- 172.21.15.27:0/1178136778 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7cf0006170 0x7f7cf0006a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:15.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:15.518+0000 7f7d0ce49640 1 -- 172.21.15.27:0/1178136778 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f7d0815b2b0 msgr2=0x7f7cf0005bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:15.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:15.518+0000 7f7d0ce49640 1 --2- 172.21.15.27:0/1178136778 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f7d0815b2b0 0x7f7cf0005bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:15.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:15.518+0000 7f7d0ce49640 1 -- 172.21.15.27:0/1178136778 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7cf0006170 msgr2=0x7f7cf0006a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:15.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:15.518+0000 7f7d0ce49640 1 --2- 172.21.15.27:0/1178136778 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7cf0006170 0x7f7cf0006a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:15.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:15.518+0000 7f7d0ce49640 1 -- 172.21.15.27:0/1178136778 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cf0005730 con 0x7f7cf0006f50 2024-09-12T03:22:15.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:15.519+0000 7f7d0ce49640 1 --2- 172.21.15.27:0/1178136778 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7cf0006f50 0x7f7cf00073d0 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f7d0000aa30 tx=0x7f7cf0007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afaee75948f5b61 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:15.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:15.519+0000 7f7d06ffd640 1 -- 172.21.15.27:0/1178136778 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d0000a650 con 0x7f7cf0006f50 2024-09-12T03:22:15.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:15.519+0000 7f7d06ffd640 1 -- 172.21.15.27:0/1178136778 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d0002f070 con 0x7f7cf0006f50 2024-09-12T03:22:15.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:15.519+0000 7f7d06ffd640 1 -- 172.21.15.27:0/1178136778 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d00034470 con 0x7f7cf0006f50 2024-09-12T03:22:15.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:15.519+0000 7f7d0e8ac640 1 -- 172.21.15.27:0/1178136778 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7cf0006f50 msgr2=0x7f7cf00073d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:15.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:15.519+0000 7f7d0e8ac640 1 --2- 172.21.15.27:0/1178136778 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7cf0006f50 0x7f7cf00073d0 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f7d0000aa30 tx=0x7f7cf0007a40 comp rx=0 tx=0).stop 2024-09-12T03:22:15.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:15.519+0000 7f7d0ce49640 1 -- 172.21.15.27:0/1178136778 reap_dead start 2024-09-12T03:22:15.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:15.519+0000 7f7d0e8ac640 1 -- 172.21.15.27:0/1178136778 shutdown_connections 2024-09-12T03:22:15.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:15.519+0000 7f7d0e8ac640 1 -- 172.21.15.27:0/1178136778 >> 172.21.15.27:0/1178136778 conn(0x7f7d08062c60 msgr2=0x7f7d0806bb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:15.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:15.519+0000 7f7d0e8ac640 1 -- 172.21.15.27:0/1178136778 shutdown_connections 2024-09-12T03:22:15.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:15.519+0000 7f7d0e8ac640 1 -- 172.21.15.27:0/1178136778 wait complete. 2024-09-12T03:22:15.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:15.521+0000 7f7d0e8ac640 1 Processor -- start 2024-09-12T03:22:15.528 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:15.521+0000 7f7d0e8ac640 1 -- start start 2024-09-12T03:22:15.528 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:15.521+0000 7f7d0e8ac640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f7cf0006f50 0x7f7d081f0420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:15.528 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:15.522+0000 7f7d0e8ac640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7d08151390 0x7f7d08143b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:15.528 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:15.522+0000 7f7d0e8ac640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7d08150e40 0x7f7d080b0a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:15.528 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:15.522+0000 7f7d0e8ac640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f7d0815b690 con 0x7f7d08150e40 2024-09-12T03:22:15.528 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:15.522+0000 7f7d0e8ac640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f7d080ce690 con 0x7f7d08151390 2024-09-12T03:22:15.528 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:15.522+0000 7f7d0e8ac640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f7d0815b810 con 0x7f7cf0006f50 2024-09-12T03:22:15.528 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:15.522+0000 7f7d0ce49640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7d08150e40 0x7f7d080b0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:15.528 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:15.522+0000 7f7d0ce49640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7d08150e40 0x7f7d080b0a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41106/0 (socket says 172.21.15.27:41106) 2024-09-12T03:22:15.529 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:15.522+0000 7f7d0ce49640 1 -- 172.21.15.27:0/2522123161 learned_addr learned my addr 172.21.15.27:0/2522123161 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:22:15.529 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:15.522+0000 7f7d07fff640 1 --2- 172.21.15.27:0/2522123161 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f7cf0006f50 0x7f7d081f0420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:15.529 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:15.522+0000 7f7d077fe640 1 --2- 172.21.15.27:0/2522123161 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7d08151390 0x7f7d08143b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:15.529 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:15.522+0000 7f7d0ce49640 1 -- 172.21.15.27:0/2522123161 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f7cf0006f50 msgr2=0x7f7d081f0420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:15.529 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:15.522+0000 7f7d0ce49640 1 --2- 172.21.15.27:0/2522123161 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f7cf0006f50 0x7f7d081f0420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:15.529 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:15.522+0000 7f7d0ce49640 1 -- 172.21.15.27:0/2522123161 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7d08151390 msgr2=0x7f7d08143b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:15.529 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:15.522+0000 7f7d0ce49640 1 --2- 172.21.15.27:0/2522123161 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7d08151390 0x7f7d08143b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:15.529 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:15.522+0000 7f7d0ce49640 1 -- 172.21.15.27:0/2522123161 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d0815cb20 con 0x7f7d08150e40 2024-09-12T03:22:15.529 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:15.522+0000 7f7d0ce49640 1 --2- 172.21.15.27:0/2522123161 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7d08150e40 0x7f7d080b0a50 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f7d00000f80 tx=0x7f7d0002d070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:15.529 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:15.522+0000 7f7d04ff9640 1 -- 172.21.15.27:0/2522123161 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d0000a650 con 0x7f7d08150e40 2024-09-12T03:22:15.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:15.522+0000 7f7d04ff9640 1 -- 172.21.15.27:0/2522123161 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d00004d60 con 0x7f7d08150e40 2024-09-12T03:22:15.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:15.522+0000 7f7d04ff9640 1 -- 172.21.15.27:0/2522123161 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d00034670 con 0x7f7d08150e40 2024-09-12T03:22:15.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:15.522+0000 7f7d0e8ac640 1 -- 172.21.15.27:0/2522123161 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d0815ce00 con 0x7f7d08150e40 2024-09-12T03:22:15.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:15.523+0000 7f7d0e8ac640 1 -- 172.21.15.27:0/2522123161 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d0815d3c0 con 0x7f7d08150e40 2024-09-12T03:22:15.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:15.524+0000 7f7d0e8ac640 1 -- 172.21.15.27:0/2522123161 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d080b3560 con 0x7f7d08150e40 2024-09-12T03:22:15.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:15.524+0000 7f7d04ff9640 1 -- 172.21.15.27:0/2522123161 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 21) ==== 91008+0+0 (secure 0 0 0) 0x7f7d00034810 con 0x7f7d08150e40 2024-09-12T03:22:15.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:15.524+0000 7f7d04ff9640 1 --2- 172.21.15.27:0/2522123161 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f7cf406c3d0 0x7f7cf406e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:15.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:15.525+0000 7f7d04ff9640 1 -- 172.21.15.27:0/2522123161 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 4913+0+0 (secure 0 0 0) 0x7f7d000aafe0 con 0x7f7d08150e40 2024-09-12T03:22:15.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:15.525+0000 7f7d07fff640 1 --2- 172.21.15.27:0/2522123161 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f7cf406c3d0 0x7f7cf406e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:15.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:15.525+0000 7f7d07fff640 1 --2- 172.21.15.27:0/2522123161 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f7cf406c3d0 0x7f7cf406e890 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f7cf8009bc0 tx=0x7f7cf8009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:15.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:15.529+0000 7f7d04ff9640 1 -- 172.21.15.27:0/2522123161 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f7d080ce410 con 0x7f7d08150e40 2024-09-12T03:22:15.770 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:15.770+0000 7f7d0e8ac640 1 -- 172.21.15.27:0/2522123161 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f7d081551f0 con 0x7f7d08150e40 2024-09-12T03:22:15.771 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:15.771+0000 7f7d04ff9640 1 -- 172.21.15.27:0/2522123161 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v19) ==== 80+0+5304 (secure 0 0 0) 0x7f7d00078f40 con 0x7f7d08150e40 2024-09-12T03:22:15.771 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:22:15.771 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-12T03:22:15.771 INFO:teuthology.orchestra.run.smithi027.stdout: "auth_dump": [ 2024-09-12T03:22:15.771 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:22:15.771 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "mds.a", 2024-09-12T03:22:15.771 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDUXOJmIhxjNxAAaGg8SwtW6NRe/lIgNCV20w==", 2024-09-12T03:22:15.771 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:22:15.771 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow", 2024-09-12T03:22:15.771 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow *", 2024-09-12T03:22:15.772 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow *", 2024-09-12T03:22:15.772 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2024-09-12T03:22:15.772 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:22:15.772 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:22:15.772 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:22:15.772 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "mds.b", 2024-09-12T03:22:15.772 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDVXOJm63kWCBAAzjfOsBsWUI+IAsuZ49yEkw==", 2024-09-12T03:22:15.772 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:22:15.772 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow", 2024-09-12T03:22:15.772 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow *", 2024-09-12T03:22:15.772 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow *", 2024-09-12T03:22:15.772 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2024-09-12T03:22:15.773 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:22:15.773 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:22:15.773 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:22:15.773 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "mds.c", 2024-09-12T03:22:15.773 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDVXOJm0sX7AhAAdynCcho2a/AgFF9Cw3+vAw==", 2024-09-12T03:22:15.773 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:22:15.773 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow", 2024-09-12T03:22:15.773 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow *", 2024-09-12T03:22:15.773 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow *", 2024-09-12T03:22:15.773 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2024-09-12T03:22:15.773 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:22:15.773 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:22:15.773 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:22:15.774 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "mds.d", 2024-09-12T03:22:15.774 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDVXOJm7Dd8DxAANHYmvKgpRZ6sVfXDSgPHWw==", 2024-09-12T03:22:15.774 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:22:15.774 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow", 2024-09-12T03:22:15.774 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow *", 2024-09-12T03:22:15.774 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow *", 2024-09-12T03:22:15.774 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2024-09-12T03:22:15.774 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:22:15.774 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:22:15.774 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:22:15.774 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "osd.0", 2024-09-12T03:22:15.774 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDYXOJmJTVYLxAAmo8kxRjaiwTT44dZLZlq4A==", 2024-09-12T03:22:15.774 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:22:15.775 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow profile osd", 2024-09-12T03:22:15.775 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile osd", 2024-09-12T03:22:15.775 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2024-09-12T03:22:15.775 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:22:15.775 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:22:15.775 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:22:15.775 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "osd.1", 2024-09-12T03:22:15.775 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDcXOJmYV0BFBAAmoGX9gf9yLoE/qkWFnv9tg==", 2024-09-12T03:22:15.775 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:22:15.775 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow profile osd", 2024-09-12T03:22:15.775 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile osd", 2024-09-12T03:22:15.775 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2024-09-12T03:22:15.775 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:22:15.775 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:22:15.776 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:22:15.776 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "osd.2", 2024-09-12T03:22:15.776 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDfXOJmEetiJRAAXk+AvfTfhjsB6T2UB4Z2/w==", 2024-09-12T03:22:15.776 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:22:15.776 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow profile osd", 2024-09-12T03:22:15.776 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile osd", 2024-09-12T03:22:15.776 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2024-09-12T03:22:15.776 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:22:15.776 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:22:15.776 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:22:15.776 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "osd.3", 2024-09-12T03:22:15.776 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDiXOJmrQbPJxAAQsi2B5a1LnPTPGZVdo7j7w==", 2024-09-12T03:22:15.776 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:22:15.777 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow profile osd", 2024-09-12T03:22:15.777 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile osd", 2024-09-12T03:22:15.777 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2024-09-12T03:22:15.777 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:22:15.777 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:22:15.777 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:22:15.777 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "osd.4", 2024-09-12T03:22:15.777 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDnXOJmoMr1NxAAv3u2q8hYt9w//9gz+N7UAQ==", 2024-09-12T03:22:15.777 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:22:15.777 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow profile osd", 2024-09-12T03:22:15.777 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile osd", 2024-09-12T03:22:15.777 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2024-09-12T03:22:15.777 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:22:15.777 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:22:15.778 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:22:15.778 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "osd.5", 2024-09-12T03:22:15.778 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDqXOJmxpUPOxAAxLH6RJsyLOZ2XMfzM0fQ1g==", 2024-09-12T03:22:15.778 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:22:15.778 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow profile osd", 2024-09-12T03:22:15.778 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile osd", 2024-09-12T03:22:15.778 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2024-09-12T03:22:15.778 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:22:15.778 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:22:15.778 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:22:15.778 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "osd.6", 2024-09-12T03:22:15.778 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDuXOJmuP0+LxAAMHoz5eSO81hPtkdBIS2cig==", 2024-09-12T03:22:15.778 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:22:15.778 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow profile osd", 2024-09-12T03:22:15.779 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile osd", 2024-09-12T03:22:15.779 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2024-09-12T03:22:15.779 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:22:15.779 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:22:15.779 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:22:15.779 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "osd.7", 2024-09-12T03:22:15.779 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDxXOJmuIC/MRAAPDM+lAHqdoQyGVODwjFp+Q==", 2024-09-12T03:22:15.779 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:22:15.779 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow profile osd", 2024-09-12T03:22:15.779 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile osd", 2024-09-12T03:22:15.779 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2024-09-12T03:22:15.779 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:22:15.779 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:22:15.780 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:22:15.780 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "client.0", 2024-09-12T03:22:15.780 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDVXOJmLm+5ExAAECM4CKfuVElmp/n0b02mXg==", 2024-09-12T03:22:15.780 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:22:15.780 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow", 2024-09-12T03:22:15.780 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow r", 2024-09-12T03:22:15.780 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-12T03:22:15.780 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:22:15.780 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:22:15.780 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:22:15.780 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "client.1", 2024-09-12T03:22:15.780 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDVXOJm61/FHBAALCQouXySfQeZ5DESGmqkzA==", 2024-09-12T03:22:15.781 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:22:15.781 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow", 2024-09-12T03:22:15.781 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow r", 2024-09-12T03:22:15.781 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow rw", 2024-09-12T03:22:15.781 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow rwx" 2024-09-12T03:22:15.781 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:22:15.781 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:22:15.781 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:22:15.781 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "client.2", 2024-09-12T03:22:15.781 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDVXOJmq/ZkGRAAOzmQzEIhb4b/YTxayc34Bw==", 2024-09-12T03:22:15.781 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:22:15.781 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow", 2024-09-12T03:22:15.781 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow r", 2024-09-12T03:22:15.781 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow rw", 2024-09-12T03:22:15.782 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow rwx" 2024-09-12T03:22:15.782 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:22:15.782 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:22:15.782 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:22:15.782 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "client.3", 2024-09-12T03:22:15.782 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDVXOJm+DlzIhAATQKZe+Zyk0MM8efTa6g9PA==", 2024-09-12T03:22:15.782 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:22:15.782 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow", 2024-09-12T03:22:15.782 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow r", 2024-09-12T03:22:15.782 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow rw", 2024-09-12T03:22:15.782 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow rwx" 2024-09-12T03:22:15.782 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:22:15.782 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:22:15.783 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:22:15.783 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "client.admin", 2024-09-12T03:22:15.783 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDUXOJm+JGWGhAApTQw9RtJWd/J5QzQaUFXvg==", 2024-09-12T03:22:15.783 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:22:15.783 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow *", 2024-09-12T03:22:15.783 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow *", 2024-09-12T03:22:15.783 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow *", 2024-09-12T03:22:15.783 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2024-09-12T03:22:15.783 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:22:15.783 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:22:15.783 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:22:15.784 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "mgr.x", 2024-09-12T03:22:15.784 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDUXOJmZcsONBAAMTY2KykuHaEkv1t6Vz5KjA==", 2024-09-12T03:22:15.784 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:22:15.784 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow *", 2024-09-12T03:22:15.784 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile mgr", 2024-09-12T03:22:15.784 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2024-09-12T03:22:15.784 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:22:15.784 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:22:15.784 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:22:15.784 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "mgr.y", 2024-09-12T03:22:15.785 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDUXOJmJOgZMBAAQ/JTxfhox7HgxBMtsvUCBg==", 2024-09-12T03:22:15.785 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:22:15.785 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow *", 2024-09-12T03:22:15.785 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile mgr", 2024-09-12T03:22:15.785 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2024-09-12T03:22:15.785 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:22:15.785 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:22:15.785 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-12T03:22:15.785 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-12T03:22:15.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:15.773+0000 7f7d0e8ac640 1 -- 172.21.15.27:0/2522123161 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f7cf406c3d0 msgr2=0x7f7cf406e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:15.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:15.773+0000 7f7d0e8ac640 1 --2- 172.21.15.27:0/2522123161 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f7cf406c3d0 0x7f7cf406e890 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f7cf8009bc0 tx=0x7f7cf8009190 comp rx=0 tx=0).stop 2024-09-12T03:22:15.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:15.773+0000 7f7d0e8ac640 1 -- 172.21.15.27:0/2522123161 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7d08150e40 msgr2=0x7f7d080b0a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:15.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:15.773+0000 7f7d0e8ac640 1 --2- 172.21.15.27:0/2522123161 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7d08150e40 0x7f7d080b0a50 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f7d00000f80 tx=0x7f7d0002d070 comp rx=0 tx=0).stop 2024-09-12T03:22:15.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:15.773+0000 7f7d0e8ac640 1 -- 172.21.15.27:0/2522123161 shutdown_connections 2024-09-12T03:22:15.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:15.773+0000 7f7d0e8ac640 1 --2- 172.21.15.27:0/2522123161 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f7cf406c3d0 0x7f7cf406e890 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:15.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:15.773+0000 7f7d0e8ac640 1 --2- 172.21.15.27:0/2522123161 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7d08150e40 0x7f7d080b0a50 unknown :-1 s=CLOSED pgs=1215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:15.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:15.773+0000 7f7d0e8ac640 1 --2- 172.21.15.27:0/2522123161 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7d08151390 0x7f7d08143b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:15.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:15.773+0000 7f7d0e8ac640 1 --2- 172.21.15.27:0/2522123161 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f7cf0006f50 0x7f7d081f0420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:15.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:15.773+0000 7f7d0e8ac640 1 -- 172.21.15.27:0/2522123161 >> 172.21.15.27:0/2522123161 conn(0x7f7d08062c60 msgr2=0x7f7d0815eec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:15.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:15.773+0000 7f7d0e8ac640 1 -- 172.21.15.27:0/2522123161 shutdown_connections 2024-09-12T03:22:15.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:15.773+0000 7f7d0e8ac640 1 -- 172.21.15.27:0/2522123161 wait complete. 2024-09-12T03:22:15.789 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-12T03:22:15.892 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:15.892+0000 7fddaa1e4640 1 Processor -- start 2024-09-12T03:22:15.892 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:15.892+0000 7fddaa1e4640 1 -- start start 2024-09-12T03:22:15.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:15.893+0000 7fddaa1e4640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fdda40cf660 0x7fdda40d2780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:15.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:15.893+0000 7fddaa1e4640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdda406cc40 con 0x7fdda40b1ca0 2024-09-12T03:22:15.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:15.893+0000 7fddaa1e4640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdda40d1ce0 con 0x7fdda406f360 2024-09-12T03:22:15.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:15.893+0000 7fddaa1e4640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fdda40d22f0 con 0x7fdda40cf660 2024-09-12T03:22:15.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:15.893+0000 7fdda37fe640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fdda40b1ca0 0x7fdda415f830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:35130/0 (socket says 172.21.15.27:35130) 2024-09-12T03:22:15.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:15.893+0000 7fdda37fe640 1 -- 172.21.15.27:0/1530618863 learned_addr learned my addr 172.21.15.27:0/1530618863 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:22:15.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:15.893+0000 7fdda3fff640 1 --2- 172.21.15.27:0/1530618863 >> v2:172.21.15.45:3300/0 conn(0x7fdda40cf660 0x7fdda40d2780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:15.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:15.893+0000 7fdda27fc640 1 -- 172.21.15.27:0/1530618863 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2618975298 0 0) 0x7fdda406cc40 con 0x7fdda40b1ca0 2024-09-12T03:22:15.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:15.893+0000 7fdda27fc640 1 -- 172.21.15.27:0/1530618863 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdd8c003520 con 0x7fdda40b1ca0 2024-09-12T03:22:15.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:15.893+0000 7fdda27fc640 1 -- 172.21.15.27:0/1530618863 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 577335439 0 0) 0x7fdd8c003520 con 0x7fdda40b1ca0 2024-09-12T03:22:15.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:15.893+0000 7fdda27fc640 1 -- 172.21.15.27:0/1530618863 >> v2:172.21.15.45:3300/0 conn(0x7fdda40cf660 msgr2=0x7fdda40d2780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:15.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:15.893+0000 7fdda27fc640 1 --2- 172.21.15.27:0/1530618863 >> v2:172.21.15.45:3300/0 conn(0x7fdda40cf660 0x7fdda40d2780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:15.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:15.893+0000 7fdda27fc640 1 -- 172.21.15.27:0/1530618863 >> v1:172.21.15.45:6789/0 conn(0x7fdda406f360 legacy=0x7fdda40cef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:15.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:15.893+0000 7fdda27fc640 1 -- 172.21.15.27:0/1530618863 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdda406cee0 con 0x7fdda40b1ca0 2024-09-12T03:22:15.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:15.893+0000 7fdda27fc640 1 -- 172.21.15.27:0/1530618863 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 847755213 0 0) 0x7fdd94001670 con 0x7fdda40b1ca0 2024-09-12T03:22:15.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:15.893+0000 7fdda27fc640 1 -- 172.21.15.27:0/1530618863 >> v1:172.21.15.27:6789/0 conn(0x7fdda40b1ca0 legacy=0x7fdda415f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:15.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:15.894+0000 7fdda27fc640 1 --2- 172.21.15.27:0/1530618863 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdd8c0039b0 0x7fdd8c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:15.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:15.894+0000 7fdda27fc640 1 --2- 172.21.15.27:0/1530618863 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdd8c004800 0x7fdd8c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:15.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:15.894+0000 7fdda27fc640 1 --2- 172.21.15.27:0/1530618863 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fdd8c0051c0 0x7fdd8c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:15.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:15.894+0000 7fdda27fc640 1 -- 172.21.15.27:0/1530618863 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fdda40d22f0 con 0x7fdd8c004800 2024-09-12T03:22:15.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:15.894+0000 7fdda27fc640 1 -- 172.21.15.27:0/1530618863 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fdd8c00c060 con 0x7fdd8c0039b0 2024-09-12T03:22:15.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:15.894+0000 7fdda27fc640 1 -- 172.21.15.27:0/1530618863 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fdd8c00c260 con 0x7fdd8c0051c0 2024-09-12T03:22:15.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:15.894+0000 7fdda2ffd640 1 --2- 172.21.15.27:0/1530618863 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdd8c004800 0x7fdd8c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:15.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:15.894+0000 7fdda37fe640 1 --2- 172.21.15.27:0/1530618863 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdd8c0039b0 0x7fdd8c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:15.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:15.894+0000 7fdda3fff640 1 --2- 172.21.15.27:0/1530618863 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fdd8c0051c0 0x7fdd8c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:15.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:15.894+0000 7fdda2ffd640 1 -- 172.21.15.27:0/1530618863 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fdd8c0051c0 msgr2=0x7fdd8c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:15.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:15.894+0000 7fdda2ffd640 1 --2- 172.21.15.27:0/1530618863 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fdd8c0051c0 0x7fdd8c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:15.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:15.894+0000 7fdda2ffd640 1 -- 172.21.15.27:0/1530618863 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdd8c0039b0 msgr2=0x7fdd8c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:15.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:15.894+0000 7fdda2ffd640 1 --2- 172.21.15.27:0/1530618863 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdd8c0039b0 0x7fdd8c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:15.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:15.894+0000 7fdda2ffd640 1 -- 172.21.15.27:0/1530618863 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd8c003520 con 0x7fdd8c004800 2024-09-12T03:22:15.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:15.894+0000 7fdda3fff640 1 -- 172.21.15.27:0/1530618863 reap_dead start 2024-09-12T03:22:15.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:15.895+0000 7fdda2ffd640 1 --2- 172.21.15.27:0/1530618863 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdd8c004800 0x7fdd8c004c80 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7fdd98001720 tx=0x7fdd8c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22374cbcb0414a62 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:15.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:15.895+0000 7fdda27fc640 1 -- 172.21.15.27:0/1530618863 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd98001370 con 0x7fdd8c004800 2024-09-12T03:22:15.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:15.895+0000 7fdda27fc640 1 -- 172.21.15.27:0/1530618863 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd980052d0 con 0x7fdd8c004800 2024-09-12T03:22:15.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:15.895+0000 7fdda27fc640 1 -- 172.21.15.27:0/1530618863 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd9802ab90 con 0x7fdd8c004800 2024-09-12T03:22:15.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:15.895+0000 7fddaa1e4640 1 -- 172.21.15.27:0/1530618863 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdd8c004800 msgr2=0x7fdd8c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:15.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:15.895+0000 7fddaa1e4640 1 --2- 172.21.15.27:0/1530618863 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdd8c004800 0x7fdd8c004c80 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7fdd98001720 tx=0x7fdd8c005830 comp rx=0 tx=0).stop 2024-09-12T03:22:15.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:15.895+0000 7fddaa1e4640 1 -- 172.21.15.27:0/1530618863 shutdown_connections 2024-09-12T03:22:15.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:15.895+0000 7fddaa1e4640 1 --2- 172.21.15.27:0/1530618863 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdd8c004800 0x7fdd8c004c80 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:15.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:15.895+0000 7fddaa1e4640 1 -- 172.21.15.27:0/1530618863 >> 172.21.15.27:0/1530618863 conn(0x7fdda4073800 msgr2=0x7fdda409ba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:15.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:15.895+0000 7fddaa1e4640 1 -- 172.21.15.27:0/1530618863 shutdown_connections 2024-09-12T03:22:15.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:15.895+0000 7fddaa1e4640 1 -- 172.21.15.27:0/1530618863 wait complete. 2024-09-12T03:22:15.902 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:15.897+0000 7fddaa1e4640 1 Processor -- start 2024-09-12T03:22:15.902 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:15.897+0000 7fddaa1e4640 1 -- start start 2024-09-12T03:22:15.902 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:15.897+0000 7fddaa1e4640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdd8c004800 0x7fdda41f4440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:15.902 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:15.898+0000 7fddaa1e4640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdda414e940 0x7fdda41607f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:15.902 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:15.898+0000 7fddaa1e4640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fdda4160d30 0x7fdda40b57e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:15.902 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:15.898+0000 7fddaa1e4640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fdda406d340 con 0x7fdda414e940 2024-09-12T03:22:15.902 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:15.898+0000 7fddaa1e4640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fdda406d1c0 con 0x7fdd8c004800 2024-09-12T03:22:15.902 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:15.898+0000 7fddaa1e4640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fdda40cfc40 con 0x7fdda4160d30 2024-09-12T03:22:15.902 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:15.898+0000 7fdda2ffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdda414e940 0x7fdda41607f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:15.902 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:15.898+0000 7fdda2ffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdda414e940 0x7fdda41607f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41122/0 (socket says 172.21.15.27:41122) 2024-09-12T03:22:15.902 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:15.898+0000 7fdda2ffd640 1 -- 172.21.15.27:0/3342420675 learned_addr learned my addr 172.21.15.27:0/3342420675 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:22:15.902 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:15.898+0000 7fdda37fe640 1 --2- 172.21.15.27:0/3342420675 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdd8c004800 0x7fdda41f4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:15.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:15.898+0000 7fdda3fff640 1 --2- 172.21.15.27:0/3342420675 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fdda4160d30 0x7fdda40b57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:15.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:15.898+0000 7fdda2ffd640 1 -- 172.21.15.27:0/3342420675 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fdda4160d30 msgr2=0x7fdda40b57e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:15.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:15.898+0000 7fdda2ffd640 1 --2- 172.21.15.27:0/3342420675 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fdda4160d30 0x7fdda40b57e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:15.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:15.898+0000 7fdda2ffd640 1 -- 172.21.15.27:0/3342420675 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdd8c004800 msgr2=0x7fdda41f4440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:15.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:15.898+0000 7fdda2ffd640 1 --2- 172.21.15.27:0/3342420675 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdd8c004800 0x7fdda41f4440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:15.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:15.898+0000 7fdda2ffd640 1 -- 172.21.15.27:0/3342420675 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdda414edc0 con 0x7fdda414e940 2024-09-12T03:22:15.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:15.898+0000 7fdda2ffd640 1 --2- 172.21.15.27:0/3342420675 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdda414e940 0x7fdda41607f0 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7fdd98000c00 tx=0x7fdd98001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:15.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:15.898+0000 7fdd8bfff640 1 -- 172.21.15.27:0/3342420675 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd98040020 con 0x7fdda414e940 2024-09-12T03:22:15.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:15.898+0000 7fdd8bfff640 1 -- 172.21.15.27:0/3342420675 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd98005480 con 0x7fdda414e940 2024-09-12T03:22:15.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:15.898+0000 7fddaa1e4640 1 -- 172.21.15.27:0/3342420675 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdda40b6510 con 0x7fdda414e940 2024-09-12T03:22:15.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:15.898+0000 7fdd8bfff640 1 -- 172.21.15.27:0/3342420675 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd98001370 con 0x7fdda414e940 2024-09-12T03:22:15.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:15.898+0000 7fddaa1e4640 1 -- 172.21.15.27:0/3342420675 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdda40cbc60 con 0x7fdda414e940 2024-09-12T03:22:15.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:15.899+0000 7fddaa1e4640 1 -- 172.21.15.27:0/3342420675 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdda406c660 con 0x7fdda414e940 2024-09-12T03:22:15.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:15.900+0000 7fdd8bfff640 1 -- 172.21.15.27:0/3342420675 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 21) ==== 91008+0+0 (secure 0 0 0) 0x7fdd98043020 con 0x7fdda414e940 2024-09-12T03:22:15.904 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:15.900+0000 7fdd8bfff640 1 --2- 172.21.15.27:0/3342420675 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fdd9006c3d0 0x7fdd9006e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:15.904 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:15.901+0000 7fdda37fe640 1 --2- 172.21.15.27:0/3342420675 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fdd9006c3d0 0x7fdd9006e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:15.904 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:15.901+0000 7fdd8bfff640 1 -- 172.21.15.27:0/3342420675 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 4913+0+0 (secure 0 0 0) 0x7fdd980aafd0 con 0x7fdda414e940 2024-09-12T03:22:15.904 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:15.901+0000 7fdda37fe640 1 --2- 172.21.15.27:0/3342420675 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fdd9006c3d0 0x7fdd9006e890 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fdd940011e0 tx=0x7fdd94001170 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:15.904 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:15.905+0000 7fdd8bfff640 1 -- 172.21.15.27:0/3342420675 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fdda406c660 con 0x7fdda414e940 2024-09-12T03:22:16.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:16.083+0000 7fddaa1e4640 1 -- 172.21.15.27:0/3342420675 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fdda415db60 con 0x7fdda414e940 2024-09-12T03:22:16.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:16.083+0000 7fdd8bfff640 1 -- 172.21.15.27:0/3342420675 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v122) ==== 84+0+65 (secure 0 0 0) 0x7fdda415db60 con 0x7fdda414e940 2024-09-12T03:22:16.083 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:22:16.084 INFO:teuthology.orchestra.run.smithi027.stdout:[ 2024-09-12T03:22:16.084 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:22:16.084 INFO:teuthology.orchestra.run.smithi027.stdout: "poolnum": 1, 2024-09-12T03:22:16.084 INFO:teuthology.orchestra.run.smithi027.stdout: "poolname": ".mgr" 2024-09-12T03:22:16.084 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:22:16.084 INFO:teuthology.orchestra.run.smithi027.stdout:] 2024-09-12T03:22:16.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:16.085+0000 7fddaa1e4640 1 -- 172.21.15.27:0/3342420675 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fdd9006c3d0 msgr2=0x7fdd9006e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:16.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:16.085+0000 7fddaa1e4640 1 --2- 172.21.15.27:0/3342420675 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fdd9006c3d0 0x7fdd9006e890 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fdd940011e0 tx=0x7fdd94001170 comp rx=0 tx=0).stop 2024-09-12T03:22:16.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:16.085+0000 7fddaa1e4640 1 -- 172.21.15.27:0/3342420675 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdda414e940 msgr2=0x7fdda41607f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:16.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:16.085+0000 7fddaa1e4640 1 --2- 172.21.15.27:0/3342420675 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdda414e940 0x7fdda41607f0 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7fdd98000c00 tx=0x7fdd98001120 comp rx=0 tx=0).stop 2024-09-12T03:22:16.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:16.085+0000 7fddaa1e4640 1 -- 172.21.15.27:0/3342420675 shutdown_connections 2024-09-12T03:22:16.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:16.085+0000 7fddaa1e4640 1 --2- 172.21.15.27:0/3342420675 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fdda4160d30 0x7fdda40b57e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:16.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:16.085+0000 7fddaa1e4640 1 --2- 172.21.15.27:0/3342420675 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fdd9006c3d0 0x7fdd9006e890 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:16.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:16.085+0000 7fddaa1e4640 1 --2- 172.21.15.27:0/3342420675 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdda414e940 0x7fdda41607f0 unknown :-1 s=CLOSED pgs=1218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:16.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:16.085+0000 7fddaa1e4640 1 --2- 172.21.15.27:0/3342420675 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdd8c004800 0x7fdda41f4440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:16.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:16.085+0000 7fddaa1e4640 1 -- 172.21.15.27:0/3342420675 >> 172.21.15.27:0/3342420675 conn(0x7fdda4073800 msgr2=0x7fdda415e040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:16.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:16.085+0000 7fddaa1e4640 1 -- 172.21.15.27:0/3342420675 shutdown_connections 2024-09-12T03:22:16.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:16.085+0000 7fddaa1e4640 1 -- 172.21.15.27:0/3342420675 wait complete. 2024-09-12T03:22:16.096 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-12T03:22:16.096 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-12T03:22:16.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:16.203+0000 7f9737796640 1 Processor -- start 2024-09-12T03:22:16.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:16.203+0000 7f9737796640 1 -- start start 2024-09-12T03:22:16.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:16.204+0000 7f9737796640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f97300b7f80 0x7f9730167170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:16.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:16.204+0000 7f9737796640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f97300d07b0 con 0x7f97300d1040 2024-09-12T03:22:16.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:16.204+0000 7f9737796640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f97300d0a80 con 0x7f973015c6c0 2024-09-12T03:22:16.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:16.204+0000 7f9737796640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f973015caa0 con 0x7f97300b7f80 2024-09-12T03:22:16.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:16.204+0000 7f9734d31640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f97300d1040 0x7f97300d14b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:35138/0 (socket says 172.21.15.27:35138) 2024-09-12T03:22:16.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:16.204+0000 7f9734d31640 1 -- 172.21.15.27:0/427851304 learned_addr learned my addr 172.21.15.27:0/427851304 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:22:16.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:16.204+0000 7f9735d33640 1 --2- 172.21.15.27:0/427851304 >> v2:172.21.15.27:3300/0 conn(0x7f97300b7f80 0x7f9730167170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:16.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:16.205+0000 7f9735d33640 1 -- 172.21.15.27:0/427851304 >> v1:172.21.15.45:6789/0 conn(0x7f973015c6c0 legacy=0x7f97300d00a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:16.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:16.205+0000 7f9735d33640 1 -- 172.21.15.27:0/427851304 >> v1:172.21.15.27:6789/0 conn(0x7f97300d1040 legacy=0x7f97300d14b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:16.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:16.205+0000 7f9735d33640 1 -- 172.21.15.27:0/427851304 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97300d0d60 con 0x7f97300b7f80 2024-09-12T03:22:16.206 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:16.205+0000 7f9735d33640 1 --2- 172.21.15.27:0/427851304 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f97300b7f80 0x7f9730167170 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f9720009d90 tx=0x7f972002a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d290aacce54f2c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:16.206 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:16.206+0000 7f972ffff640 1 -- 172.21.15.27:0/427851304 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9720007740 con 0x7f97300b7f80 2024-09-12T03:22:16.206 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:16.206+0000 7f972ffff640 1 -- 172.21.15.27:0/427851304 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97200078e0 con 0x7f97300b7f80 2024-09-12T03:22:16.206 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:16.206+0000 7f972ffff640 1 -- 172.21.15.27:0/427851304 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9720007cb0 con 0x7f97300b7f80 2024-09-12T03:22:16.206 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:16.206+0000 7f9737796640 1 -- 172.21.15.27:0/427851304 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f97300b7f80 msgr2=0x7f9730167170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:16.207 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:16.206+0000 7f9737796640 1 --2- 172.21.15.27:0/427851304 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f97300b7f80 0x7f9730167170 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f9720009d90 tx=0x7f972002a660 comp rx=0 tx=0).stop 2024-09-12T03:22:16.207 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:16.206+0000 7f9737796640 1 -- 172.21.15.27:0/427851304 shutdown_connections 2024-09-12T03:22:16.207 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:16.206+0000 7f9737796640 1 --2- 172.21.15.27:0/427851304 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f97300b7f80 0x7f9730167170 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:16.207 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:16.206+0000 7f9737796640 1 -- 172.21.15.27:0/427851304 >> 172.21.15.27:0/427851304 conn(0x7f973006f320 msgr2=0x7f973006f700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:16.207 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:16.206+0000 7f9737796640 1 -- 172.21.15.27:0/427851304 shutdown_connections 2024-09-12T03:22:16.207 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:16.206+0000 7f9737796640 1 -- 172.21.15.27:0/427851304 wait complete. 2024-09-12T03:22:16.210 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:16.207+0000 7f9737796640 1 Processor -- start 2024-09-12T03:22:16.210 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:16.207+0000 7f9737796640 1 -- start start 2024-09-12T03:22:16.210 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:16.208+0000 7f9737796640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f97300b7f80 0x7f973015cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:16.211 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:16.208+0000 7f9737796640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f97300d1040 0x7f973015f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:16.211 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:16.208+0000 7f9737796640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f973015c6c0 0x7f9730150ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:16.211 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:16.208+0000 7f9737796640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f97300b6a50 con 0x7f97300b7f80 2024-09-12T03:22:16.211 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:16.208+0000 7f9737796640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f97300b6510 con 0x7f973015c6c0 2024-09-12T03:22:16.211 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:16.208+0000 7f9737796640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f97300b4400 con 0x7f97300d1040 2024-09-12T03:22:16.211 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:16.208+0000 7f9735532640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f97300b7f80 0x7f973015cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:16.211 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:16.208+0000 7f9735532640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f97300b7f80 0x7f973015cc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41132/0 (socket says 172.21.15.27:41132) 2024-09-12T03:22:16.211 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:16.208+0000 7f9735532640 1 -- 172.21.15.27:0/2713916029 learned_addr learned my addr 172.21.15.27:0/2713916029 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:22:16.211 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:16.208+0000 7f9735d33640 1 --2- 172.21.15.27:0/2713916029 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f973015c6c0 0x7f9730150ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:16.211 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:16.208+0000 7f9734d31640 1 --2- 172.21.15.27:0/2713916029 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f97300d1040 0x7f973015f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:16.211 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:16.209+0000 7f9735532640 1 -- 172.21.15.27:0/2713916029 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f97300d1040 msgr2=0x7f973015f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:16.211 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:16.209+0000 7f9735532640 1 --2- 172.21.15.27:0/2713916029 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f97300d1040 0x7f973015f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:16.211 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:16.209+0000 7f9735532640 1 -- 172.21.15.27:0/2713916029 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f973015c6c0 msgr2=0x7f9730150ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:16.211 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:16.209+0000 7f9735532640 1 --2- 172.21.15.27:0/2713916029 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f973015c6c0 0x7f9730150ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:16.212 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:16.209+0000 7f9735532640 1 -- 172.21.15.27:0/2713916029 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f973006ec00 con 0x7f97300b7f80 2024-09-12T03:22:16.212 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:16.209+0000 7f9735d33640 1 --2- 172.21.15.27:0/2713916029 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f973015c6c0 0x7f9730150ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:22:16.212 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:16.209+0000 7f9735532640 1 --2- 172.21.15.27:0/2713916029 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f97300b7f80 0x7f973015cc30 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f972800c430 tx=0x7f972800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:16.212 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:16.209+0000 7f972dffb640 1 -- 172.21.15.27:0/2713916029 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97280052d0 con 0x7f97300b7f80 2024-09-12T03:22:16.212 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:16.209+0000 7f972dffb640 1 -- 172.21.15.27:0/2713916029 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9728005490 con 0x7f97300b7f80 2024-09-12T03:22:16.212 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:16.209+0000 7f972dffb640 1 -- 172.21.15.27:0/2713916029 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9728007260 con 0x7f97300b7f80 2024-09-12T03:22:16.212 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:16.209+0000 7f9737796640 1 -- 172.21.15.27:0/2713916029 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f973015e6c0 con 0x7f97300b7f80 2024-09-12T03:22:16.212 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:16.209+0000 7f9737796640 1 -- 172.21.15.27:0/2713916029 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f973015ec80 con 0x7f97300b7f80 2024-09-12T03:22:16.212 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:16.210+0000 7f9737796640 1 -- 172.21.15.27:0/2713916029 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f973006fbf0 con 0x7f97300b7f80 2024-09-12T03:22:16.212 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:16.211+0000 7f972dffb640 1 -- 172.21.15.27:0/2713916029 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 21) ==== 91008+0+0 (secure 0 0 0) 0x7f9728007400 con 0x7f97300b7f80 2024-09-12T03:22:16.213 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:16.211+0000 7f972dffb640 1 --2- 172.21.15.27:0/2713916029 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f971c06c3d0 0x7f971c06e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:16.213 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:16.211+0000 7f972dffb640 1 -- 172.21.15.27:0/2713916029 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 4913+0+0 (secure 0 0 0) 0x7f972808aad0 con 0x7f97300b7f80 2024-09-12T03:22:16.213 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:16.211+0000 7f9734d31640 1 --2- 172.21.15.27:0/2713916029 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f971c06c3d0 0x7f971c06e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:16.213 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:16.212+0000 7f9734d31640 1 --2- 172.21.15.27:0/2713916029 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f971c06c3d0 0x7f971c06e890 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f97300b15b0 tx=0x7f9724009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:16.215 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:16.216+0000 7f972dffb640 1 -- 172.21.15.27:0/2713916029 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f9728058f20 con 0x7f97300b7f80 2024-09-12T03:22:16.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:16.394+0000 7f9737796640 1 -- 172.21.15.27:0/2713916029 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f97300d0080 con 0x7f97300b7f80 2024-09-12T03:22:17.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:17.392+0000 7f972dffb640 1 -- 172.21.15.27:0/2713916029 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v123) ==== 138+0+0 (secure 0 0 0) 0x7f9728058820 con 0x7f97300b7f80 2024-09-12T03:22:17.392 INFO:teuthology.orchestra.run.smithi027.stderr:pool 'cephfs_metadata' created 2024-09-12T03:22:17.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:17.395+0000 7f9737796640 1 -- 172.21.15.27:0/2713916029 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f971c06c3d0 msgr2=0x7f971c06e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:17.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:17.395+0000 7f9737796640 1 --2- 172.21.15.27:0/2713916029 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f971c06c3d0 0x7f971c06e890 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f97300b15b0 tx=0x7f9724009040 comp rx=0 tx=0).stop 2024-09-12T03:22:17.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:17.395+0000 7f9737796640 1 -- 172.21.15.27:0/2713916029 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f97300b7f80 msgr2=0x7f973015cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:17.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:17.395+0000 7f9737796640 1 --2- 172.21.15.27:0/2713916029 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f97300b7f80 0x7f973015cc30 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f972800c430 tx=0x7f972800c8f0 comp rx=0 tx=0).stop 2024-09-12T03:22:17.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:17.395+0000 7f9737796640 1 -- 172.21.15.27:0/2713916029 shutdown_connections 2024-09-12T03:22:17.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:17.395+0000 7f9737796640 1 --2- 172.21.15.27:0/2713916029 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f973015c6c0 0x7f9730150ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:17.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:17.395+0000 7f9737796640 1 --2- 172.21.15.27:0/2713916029 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f97300d1040 0x7f973015f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:17.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:17.395+0000 7f9737796640 1 --2- 172.21.15.27:0/2713916029 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f971c06c3d0 0x7f971c06e890 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:17.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:17.395+0000 7f9737796640 1 --2- 172.21.15.27:0/2713916029 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f97300b7f80 0x7f973015cc30 unknown :-1 s=CLOSED pgs=1221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:17.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:17.395+0000 7f9737796640 1 -- 172.21.15.27:0/2713916029 >> 172.21.15.27:0/2713916029 conn(0x7f973006f320 msgr2=0x7f97300b2590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:17.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:17.395+0000 7f9737796640 1 -- 172.21.15.27:0/2713916029 shutdown_connections 2024-09-12T03:22:17.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:17.395+0000 7f9737796640 1 -- 172.21.15.27:0/2713916029 wait complete. 2024-09-12T03:22:17.412 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-12T03:22:17.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:17.526+0000 7fa489a77640 1 Processor -- start 2024-09-12T03:22:17.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:17.526+0000 7fa489a77640 1 -- start start 2024-09-12T03:22:17.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:17.527+0000 7fa489a77640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa48415e900 0x7fa4840af370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:17.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:17.527+0000 7fa489a77640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa48406d730 0x7fa4840b5e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:17.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:17.527+0000 7fa489a77640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa4840b5940 con 0x7fa4840b6370 2024-09-12T03:22:17.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:17.527+0000 7fa489a77640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fa48415ece0 con 0x7fa48415e900 2024-09-12T03:22:17.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:17.527+0000 7fa489a77640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fa4840af950 con 0x7fa48406d730 2024-09-12T03:22:17.528 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:17.527+0000 7fa482ffd640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa48406d730 0x7fa4840b5e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:17.528 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:17.527+0000 7fa4837fe640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa48415e900 0x7fa4840af370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:17.528 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:17.527+0000 7fa482ffd640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa48406d730 0x7fa4840b5e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3301/0 says I am v2:172.21.15.27:53752/0 (socket says 172.21.15.27:53752) 2024-09-12T03:22:17.528 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:17.527+0000 7fa483fff640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fa4840b6370 0x7fa4840b4600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.27:57968/0 (socket says 172.21.15.27:57968) 2024-09-12T03:22:17.528 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:17.527+0000 7fa4837fe640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa48415e900 0x7fa4840af370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.27:55586/0 (socket says 172.21.15.27:55586) 2024-09-12T03:22:17.528 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:17.527+0000 7fa482ffd640 1 -- 172.21.15.27:0/4276777909 learned_addr learned my addr 172.21.15.27:0/4276777909 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:22:17.528 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:17.528+0000 7fa4837fe640 1 -- 172.21.15.27:0/4276777909 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa48406d730 msgr2=0x7fa4840b5e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:17.528 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:17.528+0000 7fa4837fe640 1 --2- 172.21.15.27:0/4276777909 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa48406d730 0x7fa4840b5e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:17.529 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:17.528+0000 7fa4837fe640 1 -- 172.21.15.27:0/4276777909 >> v1:172.21.15.45:6789/0 conn(0x7fa4840b6370 legacy=0x7fa4840b4600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:17.529 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:17.528+0000 7fa4837fe640 1 -- 172.21.15.27:0/4276777909 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4840afc30 con 0x7fa48415e900 2024-09-12T03:22:17.529 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:17.528+0000 7fa482ffd640 1 --2- 172.21.15.27:0/4276777909 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa48406d730 0x7fa4840b5e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:22:17.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:17.528+0000 7fa4837fe640 1 --2- 172.21.15.27:0/4276777909 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa48415e900 0x7fa4840af370 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fa4740097a0 tx=0x7fa47402a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f7f83ba8344c48b3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:17.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:17.529+0000 7fa4827fc640 1 -- 172.21.15.27:0/4276777909 <== mon.1 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa474037070 con 0x7fa48415e900 2024-09-12T03:22:17.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:17.529+0000 7fa4827fc640 1 -- 172.21.15.27:0/4276777909 <== mon.1 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa474032440 con 0x7fa48415e900 2024-09-12T03:22:17.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:17.529+0000 7fa4827fc640 1 -- 172.21.15.27:0/4276777909 <== mon.1 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa474032810 con 0x7fa48415e900 2024-09-12T03:22:17.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:17.529+0000 7fa489a77640 1 -- 172.21.15.27:0/4276777909 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa48415e900 msgr2=0x7fa4840af370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:17.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:17.529+0000 7fa489a77640 1 --2- 172.21.15.27:0/4276777909 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa48415e900 0x7fa4840af370 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fa4740097a0 tx=0x7fa47402a180 comp rx=0 tx=0).stop 2024-09-12T03:22:17.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:17.529+0000 7fa489a77640 1 -- 172.21.15.27:0/4276777909 shutdown_connections 2024-09-12T03:22:17.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:17.529+0000 7fa489a77640 1 --2- 172.21.15.27:0/4276777909 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa48406d730 0x7fa4840b5e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:17.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:17.529+0000 7fa489a77640 1 --2- 172.21.15.27:0/4276777909 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa48415e900 0x7fa4840af370 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:17.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:17.529+0000 7fa489a77640 1 -- 172.21.15.27:0/4276777909 >> 172.21.15.27:0/4276777909 conn(0x7fa48406cc40 msgr2=0x7fa48406d020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:17.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:17.529+0000 7fa489a77640 1 -- 172.21.15.27:0/4276777909 shutdown_connections 2024-09-12T03:22:17.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:17.529+0000 7fa489a77640 1 -- 172.21.15.27:0/4276777909 wait complete. 2024-09-12T03:22:17.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:17.530+0000 7fa489a77640 1 Processor -- start 2024-09-12T03:22:17.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:17.531+0000 7fa489a77640 1 -- start start 2024-09-12T03:22:17.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:17.531+0000 7fa489a77640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa48406d730 0x7fa4841609a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:17.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:17.531+0000 7fa489a77640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa4840b6370 0x7fa48415f970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:17.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:17.531+0000 7fa489a77640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa48415e900 0x7fa48414c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:17.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:17.531+0000 7fa489a77640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fa48406e880 con 0x7fa4840b6370 2024-09-12T03:22:17.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:17.531+0000 7fa489a77640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa4840b01f0 con 0x7fa48406d730 2024-09-12T03:22:17.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:17.531+0000 7fa489a77640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fa48406ea00 con 0x7fa48415e900 2024-09-12T03:22:17.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:17.531+0000 7fa482ffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa4840b6370 0x7fa48415f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:17.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:17.532+0000 7fa482ffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa4840b6370 0x7fa48415f970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41146/0 (socket says 172.21.15.27:41146) 2024-09-12T03:22:17.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:17.532+0000 7fa482ffd640 1 -- 172.21.15.27:0/3333894630 learned_addr learned my addr 172.21.15.27:0/3333894630 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:22:17.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:17.532+0000 7fa4837fe640 1 --2- 172.21.15.27:0/3333894630 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa48406d730 0x7fa4841609a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:17.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:17.532+0000 7fa483fff640 1 --2- 172.21.15.27:0/3333894630 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa48415e900 0x7fa48414c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:17.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:17.532+0000 7fa482ffd640 1 -- 172.21.15.27:0/3333894630 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa48415e900 msgr2=0x7fa48414c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:17.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:17.532+0000 7fa482ffd640 1 --2- 172.21.15.27:0/3333894630 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa48415e900 0x7fa48414c440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:17.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:17.532+0000 7fa482ffd640 1 -- 172.21.15.27:0/3333894630 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa48406d730 msgr2=0x7fa4841609a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:17.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:17.532+0000 7fa482ffd640 1 --2- 172.21.15.27:0/3333894630 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa48406d730 0x7fa4841609a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:17.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:17.532+0000 7fa482ffd640 1 -- 172.21.15.27:0/3333894630 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa484153b30 con 0x7fa4840b6370 2024-09-12T03:22:17.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:17.532+0000 7fa482ffd640 1 --2- 172.21.15.27:0/3333894630 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa4840b6370 0x7fa48415f970 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7fa47800e9b0 tx=0x7fa47800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:17.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:17.532+0000 7fa46bfff640 1 -- 172.21.15.27:0/3333894630 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa478017070 con 0x7fa4840b6370 2024-09-12T03:22:17.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:17.532+0000 7fa46bfff640 1 -- 172.21.15.27:0/3333894630 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa47800c9a0 con 0x7fa4840b6370 2024-09-12T03:22:17.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:17.532+0000 7fa489a77640 1 -- 172.21.15.27:0/3333894630 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa48414c980 con 0x7fa4840b6370 2024-09-12T03:22:17.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:17.532+0000 7fa46bfff640 1 -- 172.21.15.27:0/3333894630 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa47800cd50 con 0x7fa4840b6370 2024-09-12T03:22:17.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:17.532+0000 7fa489a77640 1 -- 172.21.15.27:0/3333894630 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4841540b0 con 0x7fa4840b6370 2024-09-12T03:22:17.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:17.533+0000 7fa489a77640 1 -- 172.21.15.27:0/3333894630 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4840b5e30 con 0x7fa4840b6370 2024-09-12T03:22:17.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:17.534+0000 7fa46bfff640 1 -- 172.21.15.27:0/3333894630 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 21) ==== 91008+0+0 (secure 0 0 0) 0x7fa47801a070 con 0x7fa4840b6370 2024-09-12T03:22:17.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:17.535+0000 7fa46bfff640 1 --2- 172.21.15.27:0/3333894630 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fa47006c3d0 0x7fa47006e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:17.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:17.535+0000 7fa4837fe640 1 --2- 172.21.15.27:0/3333894630 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fa47006c3d0 0x7fa47006e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:17.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:17.535+0000 7fa46bfff640 1 -- 172.21.15.27:0/3333894630 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 5283+0+0 (secure 0 0 0) 0x7fa47808b490 con 0x7fa4840b6370 2024-09-12T03:22:17.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:17.535+0000 7fa4837fe640 1 --2- 172.21.15.27:0/3333894630 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fa47006c3d0 0x7fa47006e890 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fa474034000 tx=0x7fa47402e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:17.539 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:17.540+0000 7fa46bfff640 1 -- 172.21.15.27:0/3333894630 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fa4780591a0 con 0x7fa4840b6370 2024-09-12T03:22:17.714 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:17.715+0000 7fa489a77640 1 -- 172.21.15.27:0/3333894630 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fa484161100 con 0x7fa4840b6370 2024-09-12T03:22:18.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:18.394+0000 7fa46bfff640 1 -- 172.21.15.27:0/3333894630 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v124) ==== 170+0+0 (secure 0 0 0) 0x7fa47805d4a0 con 0x7fa4840b6370 2024-09-12T03:22:18.394 INFO:teuthology.orchestra.run.smithi027.stderr:pool 'cephfs_data' created 2024-09-12T03:22:18.397 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:18.397+0000 7fa489a77640 1 -- 172.21.15.27:0/3333894630 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fa47006c3d0 msgr2=0x7fa47006e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:18.397 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:18.397+0000 7fa489a77640 1 --2- 172.21.15.27:0/3333894630 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fa47006c3d0 0x7fa47006e890 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fa474034000 tx=0x7fa47402e000 comp rx=0 tx=0).stop 2024-09-12T03:22:18.397 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:18.397+0000 7fa489a77640 1 -- 172.21.15.27:0/3333894630 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa4840b6370 msgr2=0x7fa48415f970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:18.397 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:18.397+0000 7fa489a77640 1 --2- 172.21.15.27:0/3333894630 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa4840b6370 0x7fa48415f970 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7fa47800e9b0 tx=0x7fa47800ee70 comp rx=0 tx=0).stop 2024-09-12T03:22:18.397 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:18.397+0000 7fa489a77640 1 -- 172.21.15.27:0/3333894630 shutdown_connections 2024-09-12T03:22:18.397 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:18.397+0000 7fa489a77640 1 --2- 172.21.15.27:0/3333894630 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa48415e900 0x7fa48414c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:18.397 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:18.397+0000 7fa489a77640 1 --2- 172.21.15.27:0/3333894630 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fa47006c3d0 0x7fa47006e890 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:18.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:18.397+0000 7fa489a77640 1 --2- 172.21.15.27:0/3333894630 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa4840b6370 0x7fa48415f970 unknown :-1 s=CLOSED pgs=1222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:18.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:18.397+0000 7fa489a77640 1 --2- 172.21.15.27:0/3333894630 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa48406d730 0x7fa4841609a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:18.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:18.397+0000 7fa489a77640 1 -- 172.21.15.27:0/3333894630 >> 172.21.15.27:0/3333894630 conn(0x7fa48406cc40 msgr2=0x7fa4840b26b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:18.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:18.397+0000 7fa489a77640 1 -- 172.21.15.27:0/3333894630 shutdown_connections 2024-09-12T03:22:18.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:18.397+0000 7fa489a77640 1 -- 172.21.15.27:0/3333894630 wait complete. 2024-09-12T03:22:18.420 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-12T03:22:18.528 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:18.528+0000 7f4d96e9b640 1 Processor -- start 2024-09-12T03:22:18.528 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:18.528+0000 7f4d96e9b640 1 -- start start 2024-09-12T03:22:18.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:18.529+0000 7f4d96e9b640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f4d9015e650 0x7f4d900d1d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:18.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:18.529+0000 7f4d96e9b640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4d9006cc40 0x7f4d9006d060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:18.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:18.529+0000 7f4d96e9b640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f4d9006db30 0x7f4d9006df60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:18.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:18.529+0000 7f4d96e9b640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f4d900d2280 con 0x7f4d9015e650 2024-09-12T03:22:18.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:18.529+0000 7f4d96e9b640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f4d9006d5a0 con 0x7f4d9006db30 2024-09-12T03:22:18.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:18.529+0000 7f4d96e9b640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f4d9006d850 con 0x7f4d9006cc40 2024-09-12T03:22:18.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:18.529+0000 7f4d94c37640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f4d9015e650 0x7f4d900d1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:18.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:18.529+0000 7f4d94c37640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f4d9015e650 0x7f4d900d1d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41156/0 (socket says 172.21.15.27:41156) 2024-09-12T03:22:18.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:18.529+0000 7f4d94c37640 1 -- 172.21.15.27:0/3051686410 learned_addr learned my addr 172.21.15.27:0/3051686410 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:22:18.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:18.529+0000 7f4d95438640 1 --2- 172.21.15.27:0/3051686410 >> v2:172.21.15.45:3300/0 conn(0x7f4d9006db30 0x7f4d9006df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:18.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:18.529+0000 7f4d87fff640 1 --2- 172.21.15.27:0/3051686410 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4d9006cc40 0x7f4d9006d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:18.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:18.529+0000 7f4d94c37640 1 -- 172.21.15.27:0/3051686410 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4d9006cc40 msgr2=0x7f4d9006d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:18.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:18.529+0000 7f4d94c37640 1 --2- 172.21.15.27:0/3051686410 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4d9006cc40 0x7f4d9006d060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:18.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:18.529+0000 7f4d94c37640 1 -- 172.21.15.27:0/3051686410 >> v2:172.21.15.45:3300/0 conn(0x7f4d9006db30 msgr2=0x7f4d9006df60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:18.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:18.529+0000 7f4d94c37640 1 --2- 172.21.15.27:0/3051686410 >> v2:172.21.15.45:3300/0 conn(0x7f4d9006db30 0x7f4d9006df60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:18.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:18.529+0000 7f4d94c37640 1 -- 172.21.15.27:0/3051686410 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d900d1260 con 0x7f4d9015e650 2024-09-12T03:22:18.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:18.529+0000 7f4d87fff640 1 --2- 172.21.15.27:0/3051686410 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4d9006cc40 0x7f4d9006d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:22:18.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:18.530+0000 7f4d94c37640 1 --2- 172.21.15.27:0/3051686410 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4d9015e650 0x7f4d900d1d40 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f4d90050f60 tx=0x7f4d8c02a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30244de2fe5a614c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:18.532 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:18.530+0000 7f4d877fe640 1 -- 172.21.15.27:0/3051686410 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d8c036070 con 0x7f4d9015e650 2024-09-12T03:22:18.532 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:18.530+0000 7f4d877fe640 1 -- 172.21.15.27:0/3051686410 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d8c031600 con 0x7f4d9015e650 2024-09-12T03:22:18.532 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:18.530+0000 7f4d877fe640 1 -- 172.21.15.27:0/3051686410 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d8c0319d0 con 0x7f4d9015e650 2024-09-12T03:22:18.532 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:18.531+0000 7f4d96e9b640 1 -- 172.21.15.27:0/3051686410 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4d9015e650 msgr2=0x7f4d900d1d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:18.532 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:18.531+0000 7f4d96e9b640 1 --2- 172.21.15.27:0/3051686410 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4d9015e650 0x7f4d900d1d40 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f4d90050f60 tx=0x7f4d8c02a610 comp rx=0 tx=0).stop 2024-09-12T03:22:18.532 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:18.531+0000 7f4d96e9b640 1 -- 172.21.15.27:0/3051686410 shutdown_connections 2024-09-12T03:22:18.532 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:18.531+0000 7f4d96e9b640 1 --2- 172.21.15.27:0/3051686410 >> v2:172.21.15.45:3300/0 conn(0x7f4d9006db30 0x7f4d9006df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:18.532 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:18.531+0000 7f4d96e9b640 1 --2- 172.21.15.27:0/3051686410 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4d9006cc40 0x7f4d9006d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:18.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:18.531+0000 7f4d96e9b640 1 --2- 172.21.15.27:0/3051686410 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4d9015e650 0x7f4d900d1d40 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:18.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:18.531+0000 7f4d96e9b640 1 -- 172.21.15.27:0/3051686410 >> 172.21.15.27:0/3051686410 conn(0x7f4d9006f320 msgr2=0x7f4d9006f700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:18.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:18.531+0000 7f4d96e9b640 1 -- 172.21.15.27:0/3051686410 shutdown_connections 2024-09-12T03:22:18.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:18.531+0000 7f4d96e9b640 1 -- 172.21.15.27:0/3051686410 wait complete. 2024-09-12T03:22:18.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:18.532+0000 7f4d96e9b640 1 Processor -- start 2024-09-12T03:22:18.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:18.532+0000 7f4d96e9b640 1 -- start start 2024-09-12T03:22:18.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:18.533+0000 7f4d96e9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4d9006cc40 0x7f4d9015b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:18.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:18.533+0000 7f4d96e9b640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4d9006db30 0x7f4d901f4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:18.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:18.533+0000 7f4d96e9b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4d9015e650 0x7f4d901570c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:18.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:18.533+0000 7f4d96e9b640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f4d9015ebb0 con 0x7f4d9015e650 2024-09-12T03:22:18.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:18.533+0000 7f4d96e9b640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f4d9015ea30 con 0x7f4d9006cc40 2024-09-12T03:22:18.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:18.533+0000 7f4d96e9b640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f4d9006e4a0 con 0x7f4d9006db30 2024-09-12T03:22:18.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:18.533+0000 7f4d95438640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4d9015e650 0x7f4d901570c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:18.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:18.533+0000 7f4d95438640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4d9015e650 0x7f4d901570c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41166/0 (socket says 172.21.15.27:41166) 2024-09-12T03:22:18.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:18.533+0000 7f4d95438640 1 -- 172.21.15.27:0/4292799207 learned_addr learned my addr 172.21.15.27:0/4292799207 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:22:18.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:18.533+0000 7f4d94c37640 1 --2- 172.21.15.27:0/4292799207 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4d9006cc40 0x7f4d9015b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:18.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:18.533+0000 7f4d87fff640 1 --2- 172.21.15.27:0/4292799207 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4d9006db30 0x7f4d901f4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:18.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:18.533+0000 7f4d95438640 1 -- 172.21.15.27:0/4292799207 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4d9006db30 msgr2=0x7f4d901f4fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:18.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:18.533+0000 7f4d95438640 1 --2- 172.21.15.27:0/4292799207 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4d9006db30 0x7f4d901f4fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:18.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:18.533+0000 7f4d95438640 1 -- 172.21.15.27:0/4292799207 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4d9006cc40 msgr2=0x7f4d9015b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:18.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:18.533+0000 7f4d95438640 1 --2- 172.21.15.27:0/4292799207 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4d9006cc40 0x7f4d9015b1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:18.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:18.533+0000 7f4d95438640 1 -- 172.21.15.27:0/4292799207 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d901f5510 con 0x7f4d9015e650 2024-09-12T03:22:18.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:18.533+0000 7f4d94c37640 1 --2- 172.21.15.27:0/4292799207 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4d9006cc40 0x7f4d9015b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:22:18.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:18.534+0000 7f4d95438640 1 --2- 172.21.15.27:0/4292799207 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4d9015e650 0x7f4d901570c0 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f4d8800d4c0 tx=0x7f4d8800d980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:18.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:18.534+0000 7f4d857fa640 1 -- 172.21.15.27:0/4292799207 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d880125f0 con 0x7f4d9015e650 2024-09-12T03:22:18.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:18.534+0000 7f4d857fa640 1 -- 172.21.15.27:0/4292799207 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d88019070 con 0x7f4d9015e650 2024-09-12T03:22:18.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:18.534+0000 7f4d96e9b640 1 -- 172.21.15.27:0/4292799207 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d900b7f60 con 0x7f4d9015e650 2024-09-12T03:22:18.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:18.534+0000 7f4d857fa640 1 -- 172.21.15.27:0/4292799207 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d880129f0 con 0x7f4d9015e650 2024-09-12T03:22:18.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:18.534+0000 7f4d96e9b640 1 -- 172.21.15.27:0/4292799207 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d900cbb80 con 0x7f4d9015e650 2024-09-12T03:22:18.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:18.535+0000 7f4d96e9b640 1 -- 172.21.15.27:0/4292799207 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d9006df60 con 0x7f4d9015e650 2024-09-12T03:22:18.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:18.536+0000 7f4d857fa640 1 -- 172.21.15.27:0/4292799207 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 21) ==== 91008+0+0 (secure 0 0 0) 0x7f4d88012b90 con 0x7f4d9015e650 2024-09-12T03:22:18.538 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:18.536+0000 7f4d857fa640 1 --2- 172.21.15.27:0/4292799207 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f4d7c06c3d0 0x7f4d7c06e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:18.538 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:18.536+0000 7f4d857fa640 1 -- 172.21.15.27:0/4292799207 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 5665+0+0 (secure 0 0 0) 0x7f4d8808baf0 con 0x7f4d9015e650 2024-09-12T03:22:18.538 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:18.536+0000 7f4d94c37640 1 --2- 172.21.15.27:0/4292799207 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f4d7c06c3d0 0x7f4d7c06e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:18.538 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:18.537+0000 7f4d94c37640 1 --2- 172.21.15.27:0/4292799207 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f4d7c06c3d0 0x7f4d7c06e890 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f4d8c033000 tx=0x7f4d8c009000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:18.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:18.541+0000 7f4d857fa640 1 -- 172.21.15.27:0/4292799207 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f4d901f5510 con 0x7f4d9015e650 2024-09-12T03:22:18.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:18.754+0000 7f4d96e9b640 1 -- 172.21.15.27:0/4292799207 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f4d9015cb60 con 0x7f4d9015e650 2024-09-12T03:22:19.415 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:19.415+0000 7f4d857fa640 1 -- 172.21.15.27:0/4292799207 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 20 and data pool 21 v73) ==== 174+0+0 (secure 0 0 0) 0x7f4d88059ce0 con 0x7f4d9015e650 2024-09-12T03:22:19.415 INFO:teuthology.orchestra.run.smithi027.stderr:new fs with metadata pool 20 and data pool 21 2024-09-12T03:22:19.420 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:19.420+0000 7f4d96e9b640 1 -- 172.21.15.27:0/4292799207 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f4d7c06c3d0 msgr2=0x7f4d7c06e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:19.421 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:19.420+0000 7f4d96e9b640 1 --2- 172.21.15.27:0/4292799207 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f4d7c06c3d0 0x7f4d7c06e890 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f4d8c033000 tx=0x7f4d8c009000 comp rx=0 tx=0).stop 2024-09-12T03:22:19.421 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:19.420+0000 7f4d96e9b640 1 -- 172.21.15.27:0/4292799207 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4d9015e650 msgr2=0x7f4d901570c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:19.421 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:19.420+0000 7f4d96e9b640 1 --2- 172.21.15.27:0/4292799207 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4d9015e650 0x7f4d901570c0 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f4d8800d4c0 tx=0x7f4d8800d980 comp rx=0 tx=0).stop 2024-09-12T03:22:19.422 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:19.421+0000 7f4d96e9b640 1 -- 172.21.15.27:0/4292799207 shutdown_connections 2024-09-12T03:22:19.422 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:19.421+0000 7f4d96e9b640 1 --2- 172.21.15.27:0/4292799207 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f4d7c06c3d0 0x7f4d7c06e890 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:19.422 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:19.422+0000 7f4d96e9b640 1 --2- 172.21.15.27:0/4292799207 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4d9015e650 0x7f4d901570c0 unknown :-1 s=CLOSED pgs=1224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:19.422 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:19.422+0000 7f4d96e9b640 1 --2- 172.21.15.27:0/4292799207 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4d9006db30 0x7f4d901f4fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:19.422 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:19.422+0000 7f4d96e9b640 1 --2- 172.21.15.27:0/4292799207 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4d9006cc40 0x7f4d9015b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:19.422 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:19.422+0000 7f4d96e9b640 1 -- 172.21.15.27:0/4292799207 >> 172.21.15.27:0/4292799207 conn(0x7f4d9006f320 msgr2=0x7f4d9015d080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:19.425 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:19.425+0000 7f4d96e9b640 1 -- 172.21.15.27:0/4292799207 shutdown_connections 2024-09-12T03:22:19.425 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:19.425+0000 7f4d96e9b640 1 -- 172.21.15.27:0/4292799207 wait complete. 2024-09-12T03:22:19.448 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-12T03:22:19.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:19.555+0000 7f69bf1a6640 1 Processor -- start 2024-09-12T03:22:19.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:19.555+0000 7f69bf1a6640 1 -- start start 2024-09-12T03:22:19.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:19.556+0000 7f69bf1a6640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f69b80ce6d0 0x7f69b80ceab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:19.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:19.556+0000 7f69bf1a6640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f69b8160d80 0x7f69b806bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:19.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:19.556+0000 7f69bf1a6640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f69b806c070 0x7f69b806e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:19.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:19.556+0000 7f69bf1a6640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f69b81463d0 con 0x7f69b806c070 2024-09-12T03:22:19.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:19.556+0000 7f69bf1a6640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f69b80ceff0 con 0x7f69b80ce6d0 2024-09-12T03:22:19.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:19.556+0000 7f69bf1a6640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f69b806c7f0 con 0x7f69b8160d80 2024-09-12T03:22:19.558 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:19.556+0000 7f69bd743640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f69b806c070 0x7f69b806e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:19.558 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:19.556+0000 7f69bd743640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f69b806c070 0x7f69b806e5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49140/0 (socket says 172.21.15.27:49140) 2024-09-12T03:22:19.558 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:19.556+0000 7f69bd743640 1 -- 172.21.15.27:0/3724335676 learned_addr learned my addr 172.21.15.27:0/3724335676 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:22:19.558 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:19.556+0000 7f69bcf42640 1 --2- 172.21.15.27:0/3724335676 >> v2:172.21.15.45:3300/0 conn(0x7f69b80ce6d0 0x7f69b80ceab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:19.558 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:19.556+0000 7f69b7fff640 1 --2- 172.21.15.27:0/3724335676 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f69b8160d80 0x7f69b806bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:19.558 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:19.556+0000 7f69bd743640 1 -- 172.21.15.27:0/3724335676 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f69b8160d80 msgr2=0x7f69b806bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:19.558 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:19.556+0000 7f69bd743640 1 --2- 172.21.15.27:0/3724335676 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f69b8160d80 0x7f69b806bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:19.558 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:19.556+0000 7f69bd743640 1 -- 172.21.15.27:0/3724335676 >> v2:172.21.15.45:3300/0 conn(0x7f69b80ce6d0 msgr2=0x7f69b80ceab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:19.558 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:19.556+0000 7f69bd743640 1 --2- 172.21.15.27:0/3724335676 >> v2:172.21.15.45:3300/0 conn(0x7f69b80ce6d0 0x7f69b80ceab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:19.558 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:19.556+0000 7f69bd743640 1 -- 172.21.15.27:0/3724335676 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69b81602a0 con 0x7f69b806c070 2024-09-12T03:22:19.558 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:19.557+0000 7f69bd743640 1 --2- 172.21.15.27:0/3724335676 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f69b806c070 0x7f69b806e5b0 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f69b80733f0 tx=0x7f69b002a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18ff49a9dc7da46f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:19.558 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:19.557+0000 7f69b77fe640 1 -- 172.21.15.27:0/3724335676 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69b0036070 con 0x7f69b806c070 2024-09-12T03:22:19.558 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:19.557+0000 7f69b77fe640 1 -- 172.21.15.27:0/3724335676 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69b0031600 con 0x7f69b806c070 2024-09-12T03:22:19.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:19.557+0000 7f69b77fe640 1 -- 172.21.15.27:0/3724335676 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69b00319d0 con 0x7f69b806c070 2024-09-12T03:22:19.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:19.558+0000 7f69bf1a6640 1 -- 172.21.15.27:0/3724335676 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f69b806c070 msgr2=0x7f69b806e5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:19.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:19.558+0000 7f69bf1a6640 1 --2- 172.21.15.27:0/3724335676 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f69b806c070 0x7f69b806e5b0 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f69b80733f0 tx=0x7f69b002a610 comp rx=0 tx=0).stop 2024-09-12T03:22:19.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:19.558+0000 7f69bf1a6640 1 -- 172.21.15.27:0/3724335676 shutdown_connections 2024-09-12T03:22:19.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:19.558+0000 7f69bf1a6640 1 --2- 172.21.15.27:0/3724335676 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f69b806c070 0x7f69b806e5b0 unknown :-1 s=CLOSED pgs=1225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:19.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:19.558+0000 7f69bf1a6640 1 --2- 172.21.15.27:0/3724335676 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f69b8160d80 0x7f69b806bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:19.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:19.558+0000 7f69bf1a6640 1 --2- 172.21.15.27:0/3724335676 >> v2:172.21.15.45:3300/0 conn(0x7f69b80ce6d0 0x7f69b80ceab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:19.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:19.558+0000 7f69bf1a6640 1 -- 172.21.15.27:0/3724335676 >> 172.21.15.27:0/3724335676 conn(0x7f69b806cc40 msgr2=0x7f69b806d050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:19.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:19.558+0000 7f69bf1a6640 1 -- 172.21.15.27:0/3724335676 shutdown_connections 2024-09-12T03:22:19.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:19.558+0000 7f69bf1a6640 1 -- 172.21.15.27:0/3724335676 wait complete. 2024-09-12T03:22:19.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:19.559+0000 7f69bf1a6640 1 Processor -- start 2024-09-12T03:22:19.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:19.559+0000 7f69bf1a6640 1 -- start start 2024-09-12T03:22:19.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:19.560+0000 7f69bf1a6640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f69b806c070 0x7f69b80aede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:19.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:19.560+0000 7f69bf1a6640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f69b80ce6d0 0x7f69b80b3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:19.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:19.560+0000 7f69bf1a6640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f69b8160d80 0x7f69b8136be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:19.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:19.560+0000 7f69bf1a6640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f69b806c4d0 con 0x7f69b80ce6d0 2024-09-12T03:22:19.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:19.560+0000 7f69bf1a6640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f69b8160860 con 0x7f69b806c070 2024-09-12T03:22:19.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:19.560+0000 7f69bf1a6640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f69b806eaf0 con 0x7f69b8160d80 2024-09-12T03:22:19.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:19.560+0000 7f69b7fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f69b80ce6d0 0x7f69b80b3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:19.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:19.560+0000 7f69b7fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f69b80ce6d0 0x7f69b80b3a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49154/0 (socket says 172.21.15.27:49154) 2024-09-12T03:22:19.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:19.560+0000 7f69b7fff640 1 -- 172.21.15.27:0/723168829 learned_addr learned my addr 172.21.15.27:0/723168829 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:22:19.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:19.560+0000 7f69bcf42640 1 --2- 172.21.15.27:0/723168829 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f69b806c070 0x7f69b80aede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:19.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:19.560+0000 7f69bd743640 1 --2- 172.21.15.27:0/723168829 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f69b8160d80 0x7f69b8136be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:19.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:19.560+0000 7f69b7fff640 1 -- 172.21.15.27:0/723168829 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f69b8160d80 msgr2=0x7f69b8136be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:19.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:19.560+0000 7f69b7fff640 1 --2- 172.21.15.27:0/723168829 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f69b8160d80 0x7f69b8136be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:19.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:19.560+0000 7f69b7fff640 1 -- 172.21.15.27:0/723168829 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f69b806c070 msgr2=0x7f69b80aede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:19.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:19.560+0000 7f69b7fff640 1 --2- 172.21.15.27:0/723168829 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f69b806c070 0x7f69b80aede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:19.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:19.560+0000 7f69b7fff640 1 -- 172.21.15.27:0/723168829 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69b80af3c0 con 0x7f69b80ce6d0 2024-09-12T03:22:19.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:19.560+0000 7f69bcf42640 1 --2- 172.21.15.27:0/723168829 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f69b806c070 0x7f69b80aede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:22:19.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:19.560+0000 7f69bd743640 1 --2- 172.21.15.27:0/723168829 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f69b8160d80 0x7f69b8136be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:22:19.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:19.560+0000 7f69b7fff640 1 --2- 172.21.15.27:0/723168829 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f69b80ce6d0 0x7f69b80b3a20 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f69a400d5f0 tx=0x7f69a400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:19.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:19.561+0000 7f69b57fa640 1 -- 172.21.15.27:0/723168829 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69a40125f0 con 0x7f69b80ce6d0 2024-09-12T03:22:19.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:19.561+0000 7f69b57fa640 1 -- 172.21.15.27:0/723168829 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69a4019070 con 0x7f69b80ce6d0 2024-09-12T03:22:19.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:19.561+0000 7f69bf1a6640 1 -- 172.21.15.27:0/723168829 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69b80af6a0 con 0x7f69b80ce6d0 2024-09-12T03:22:19.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:19.561+0000 7f69b57fa640 1 -- 172.21.15.27:0/723168829 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69a40129d0 con 0x7f69b80ce6d0 2024-09-12T03:22:19.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:19.561+0000 7f69bf1a6640 1 -- 172.21.15.27:0/723168829 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69b80aff40 con 0x7f69b80ce6d0 2024-09-12T03:22:19.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:19.562+0000 7f69bf1a6640 1 -- 172.21.15.27:0/723168829 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69b80ceab0 con 0x7f69b80ce6d0 2024-09-12T03:22:19.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:19.562+0000 7f69b57fa640 1 -- 172.21.15.27:0/723168829 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 21) ==== 91008+0+0 (secure 0 0 0) 0x7f69a4012b70 con 0x7f69b80ce6d0 2024-09-12T03:22:19.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:19.563+0000 7f69b57fa640 1 --2- 172.21.15.27:0/723168829 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f699c06c3d0 0x7f699c06e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:19.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:19.563+0000 7f69bcf42640 1 --2- 172.21.15.27:0/723168829 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f699c06c3d0 0x7f699c06e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:19.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:19.563+0000 7f69b57fa640 1 -- 172.21.15.27:0/723168829 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 5765+0+0 (secure 0 0 0) 0x7f69a408bd10 con 0x7f69b80ce6d0 2024-09-12T03:22:19.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:19.563+0000 7f69bcf42640 1 --2- 172.21.15.27:0/723168829 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f699c06c3d0 0x7f699c06e890 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f69ac006fd0 tx=0x7f69ac008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:19.567 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:19.567+0000 7f69b57fa640 1 -- 172.21.15.27:0/723168829 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f69a4059e10 con 0x7f69b80ce6d0 2024-09-12T03:22:19.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:19.744+0000 7f69bf1a6640 1 -- 172.21.15.27:0/723168829 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f69b806e390 con 0x7f69b80ce6d0 2024-09-12T03:22:19.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:19.745+0000 7f69b57fa640 1 -- 172.21.15.27:0/723168829 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v125) ==== 74+0+16793 (secure 0 0 0) 0x7f69a4059710 con 0x7f69b80ce6d0 2024-09-12T03:22:19.745 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:22:19.745 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":125,"fsid":"d2df504e-2647-4385-a5db-9fdf69f487a0","created":"2024-09-12T03:16:07.778937+0000","modified":"2024-09-12T03:22:18.754958+0000","last_up_change":"2024-09-12T03:16:19.314778+0000","last_in_change":"2024-09-12T03:16:09.893086+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T03:16:20.097646+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-09-12T03:22:16.425965+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"125","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-09-12T03:22:17.746336+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"125","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1851116c-1fb3-4dc9-9772-f90e2ef4c355","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6817","nonce":3002321488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6819","nonce":3002321488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6823","nonce":3002321488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6821","nonce":3002321488}]},"public_addr":"172.21.15.27:6817/3002321488","cluster_addr":"172.21.15.27:6819/3002321488","heartbeat_back_addr":"172.21.15.27:6823/3002321488","heartbeat_front_addr":"172.21.15.27:6821/3002321488","state":["exists","up"]},{"osd":1,"uuid":"f38ae17d-73eb-4204-b99d-975fa806fb87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6809","nonce":1613102886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6811","nonce":1613102886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6815","nonce":1613102886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6813","nonce":1613102886}]},"public_addr":"172.21.15.27:6809/1613102886","cluster_addr":"172.21.15.27:6811/1613102886","heartbeat_back_addr":"172.21.15.27:6815/1613102886","heartbeat_front_addr":"172.21.15.27:6813/1613102886","state":["exists","up"]},{"osd":2,"uuid":"fb12dcc8-bdd0-42b8-b46d-8b10106224bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6825","nonce":3534723355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6827","nonce":3534723355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6831","nonce":3534723355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6829","nonce":3534723355}]},"public_addr":"172.21.15.27:6825/3534723355","cluster_addr":"172.21.15.27:6827/3534723355","heartbeat_back_addr":"172.21.15.27:6831/3534723355","heartbeat_front_addr":"172.21.15.27:6829/3534723355","state":["exists","up"]},{"osd":3,"uuid":"d74d3f49-9c24-4b9f-98e2-3ecd7b5ef3b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6801","nonce":80661598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6803","nonce":80661598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6807","nonce":80661598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6805","nonce":80661598}]},"public_addr":"172.21.15.27:6801/80661598","cluster_addr":"172.21.15.27:6803/80661598","heartbeat_back_addr":"172.21.15.27:6807/80661598","heartbeat_front_addr":"172.21.15.27:6805/80661598","state":["exists","up"]},{"osd":4,"uuid":"2bf123f2-1b0d-41c4-aa3b-805bab267e59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6817","nonce":3943189827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6819","nonce":3943189827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6823","nonce":3943189827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6821","nonce":3943189827}]},"public_addr":"172.21.15.45:6817/3943189827","cluster_addr":"172.21.15.45:6819/3943189827","heartbeat_back_addr":"172.21.15.45:6823/3943189827","heartbeat_front_addr":"172.21.15.45:6821/3943189827","state":["exists","up"]},{"osd":5,"uuid":"c49ba5da-91c6-4a86-a97b-4aba89c253ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6809","nonce":3620760330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6811","nonce":3620760330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6815","nonce":3620760330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6813","nonce":3620760330}]},"public_addr":"172.21.15.45:6809/3620760330","cluster_addr":"172.21.15.45:6811/3620760330","heartbeat_back_addr":"172.21.15.45:6815/3620760330","heartbeat_front_addr":"172.21.15.45:6813/3620760330","state":["exists","up"]},{"osd":6,"uuid":"b0123475-89f7-4a34-9e33-d797b99dd9dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6801","nonce":1989083510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6803","nonce":1989083510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6807","nonce":1989083510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6805","nonce":1989083510}]},"public_addr":"172.21.15.45:6801/1989083510","cluster_addr":"172.21.15.45:6803/1989083510","heartbeat_back_addr":"172.21.15.45:6807/1989083510","heartbeat_front_addr":"172.21.15.45:6805/1989083510","state":["exists","up"]},{"osd":7,"uuid":"4dad773d-b7d8-46fc-8a10-a972aad27b29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6825","nonce":1513300294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6827","nonce":1513300294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6831","nonce":1513300294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6829","nonce":1513300294}]},"public_addr":"172.21.15.45:6825/1513300294","cluster_addr":"172.21.15.45:6827/1513300294","heartbeat_back_addr":"172.21.15.45:6831/1513300294","heartbeat_front_addr":"172.21.15.45:6829/1513300294","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.881855+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:17.133252+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.735339+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.500742+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.477571+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:17.167667+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.470291+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.873743+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]},{"pool":19,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T03:22:19.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:19.747+0000 7f69bf1a6640 1 -- 172.21.15.27:0/723168829 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f699c06c3d0 msgr2=0x7f699c06e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:19.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:19.747+0000 7f69bf1a6640 1 --2- 172.21.15.27:0/723168829 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f699c06c3d0 0x7f699c06e890 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f69ac006fd0 tx=0x7f69ac008040 comp rx=0 tx=0).stop 2024-09-12T03:22:19.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:19.747+0000 7f69bf1a6640 1 -- 172.21.15.27:0/723168829 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f69b80ce6d0 msgr2=0x7f69b80b3a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:19.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:19.747+0000 7f69bf1a6640 1 --2- 172.21.15.27:0/723168829 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f69b80ce6d0 0x7f69b80b3a20 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f69a400d5f0 tx=0x7f69a400dab0 comp rx=0 tx=0).stop 2024-09-12T03:22:19.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:19.747+0000 7f69bf1a6640 1 -- 172.21.15.27:0/723168829 shutdown_connections 2024-09-12T03:22:19.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:19.747+0000 7f69bf1a6640 1 --2- 172.21.15.27:0/723168829 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f69b8160d80 0x7f69b8136be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:19.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:19.747+0000 7f69bf1a6640 1 --2- 172.21.15.27:0/723168829 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f699c06c3d0 0x7f699c06e890 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:19.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:19.747+0000 7f69bf1a6640 1 --2- 172.21.15.27:0/723168829 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f69b80ce6d0 0x7f69b80b3a20 unknown :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:19.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:19.747+0000 7f69bf1a6640 1 --2- 172.21.15.27:0/723168829 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f69b806c070 0x7f69b80aede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:19.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:19.747+0000 7f69bf1a6640 1 -- 172.21.15.27:0/723168829 >> 172.21.15.27:0/723168829 conn(0x7f69b806cc40 msgr2=0x7f69b815f370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:19.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:19.747+0000 7f69bf1a6640 1 -- 172.21.15.27:0/723168829 shutdown_connections 2024-09-12T03:22:19.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:19.747+0000 7f69bf1a6640 1 -- 172.21.15.27:0/723168829 wait complete. 2024-09-12T03:22:19.758 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-12T03:22:19.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:19.861+0000 7f50d095b640 1 Processor -- start 2024-09-12T03:22:19.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:19.861+0000 7f50d095b640 1 -- start start 2024-09-12T03:22:19.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:19.862+0000 7f50d095b640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f50cc0d5200 0x7f50cc0d1010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:19.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:19.862+0000 7f50d095b640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f50cc0d1550 0x7f50cc0b8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:19.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:19.862+0000 7f50d095b640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f50cc0d3760 con 0x7f50cc15eb90 2024-09-12T03:22:19.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:19.862+0000 7f50d095b640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f50cc0b7d20 con 0x7f50cc0d5200 2024-09-12T03:22:19.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:19.862+0000 7f50d095b640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f50cc0d42e0 con 0x7f50cc0d1550 2024-09-12T03:22:19.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:19.862+0000 7f50c9d9b640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f50cc0d5200 0x7f50cc0d1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:19.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:19.862+0000 7f50c9d9b640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f50cc0d5200 0x7f50cc0d1010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49170/0 (socket says 172.21.15.27:49170) 2024-09-12T03:22:19.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:19.862+0000 7f50c9d9b640 1 -- 172.21.15.27:0/2258666518 learned_addr learned my addr 172.21.15.27:0/2258666518 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:22:19.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:19.862+0000 7f50cad9d640 1 --2- 172.21.15.27:0/2258666518 >> v2:172.21.15.45:3300/0 conn(0x7f50cc0d1550 0x7f50cc0b8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:19.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:19.862+0000 7f50c9d9b640 1 -- 172.21.15.27:0/2258666518 >> v2:172.21.15.45:3300/0 conn(0x7f50cc0d1550 msgr2=0x7f50cc0b8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:19.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:19.862+0000 7f50c9d9b640 1 --2- 172.21.15.27:0/2258666518 >> v2:172.21.15.45:3300/0 conn(0x7f50cc0d1550 0x7f50cc0b8190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:19.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:19.862+0000 7f50c9d9b640 1 -- 172.21.15.27:0/2258666518 >> v1:172.21.15.27:6789/0 conn(0x7f50cc15eb90 legacy=0x7f50cc0d2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:19.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:19.862+0000 7f50c9d9b640 1 -- 172.21.15.27:0/2258666518 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50cc0d3a40 con 0x7f50cc0d5200 2024-09-12T03:22:19.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:19.862+0000 7f50c959a640 1 -- 172.21.15.27:0/2258666518 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2376675916 0 0) 0x7f50cc0d3760 con 0x7f50cc15eb90 2024-09-12T03:22:19.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:19.862+0000 7f50cad9d640 1 --2- 172.21.15.27:0/2258666518 >> v2:172.21.15.45:3300/0 conn(0x7f50cc0d1550 0x7f50cc0b8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:22:19.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:19.863+0000 7f50c9d9b640 1 --2- 172.21.15.27:0/2258666518 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f50cc0d5200 0x7f50cc0d1010 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f50cc0c8270 tx=0x7f50c002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6543a5916348bf07 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:19.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:19.863+0000 7f50c959a640 1 -- 172.21.15.27:0/2258666518 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50c0037070 con 0x7f50cc0d5200 2024-09-12T03:22:19.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:19.863+0000 7f50c959a640 1 -- 172.21.15.27:0/2258666518 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50c0032440 con 0x7f50cc0d5200 2024-09-12T03:22:19.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:19.863+0000 7f50c959a640 1 -- 172.21.15.27:0/2258666518 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50c0032810 con 0x7f50cc0d5200 2024-09-12T03:22:19.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:19.863+0000 7f50d095b640 1 -- 172.21.15.27:0/2258666518 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f50cc0d5200 msgr2=0x7f50cc0d1010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:19.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:19.863+0000 7f50d095b640 1 --2- 172.21.15.27:0/2258666518 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f50cc0d5200 0x7f50cc0d1010 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f50cc0c8270 tx=0x7f50c002a270 comp rx=0 tx=0).stop 2024-09-12T03:22:19.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:19.864+0000 7f50d095b640 1 -- 172.21.15.27:0/2258666518 shutdown_connections 2024-09-12T03:22:19.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:19.864+0000 7f50d095b640 1 --2- 172.21.15.27:0/2258666518 >> v2:172.21.15.45:3300/0 conn(0x7f50cc0d1550 0x7f50cc0b8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:19.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:19.864+0000 7f50d095b640 1 --2- 172.21.15.27:0/2258666518 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f50cc0d5200 0x7f50cc0d1010 unknown :-1 s=CLOSED pgs=1228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:19.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:19.864+0000 7f50d095b640 1 -- 172.21.15.27:0/2258666518 >> 172.21.15.27:0/2258666518 conn(0x7f50cc06dce0 msgr2=0x7f50cc06e0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:19.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:19.864+0000 7f50d095b640 1 -- 172.21.15.27:0/2258666518 shutdown_connections 2024-09-12T03:22:19.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:19.864+0000 7f50d095b640 1 -- 172.21.15.27:0/2258666518 wait complete. 2024-09-12T03:22:19.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:19.865+0000 7f50d095b640 1 Processor -- start 2024-09-12T03:22:19.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:19.865+0000 7f50d095b640 1 -- start start 2024-09-12T03:22:19.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:19.866+0000 7f50d095b640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f50cc0d1550 0x7f50cc158c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:19.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:19.866+0000 7f50d095b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f50cc0d5200 0x7f50cc15aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:19.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:19.866+0000 7f50d095b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f50cc15eb90 0x7f50cc14ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:19.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:19.866+0000 7f50d095b640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f50cc0d4000 con 0x7f50cc0d5200 2024-09-12T03:22:19.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:19.866+0000 7f50d095b640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f50cc0d3500 con 0x7f50cc15eb90 2024-09-12T03:22:19.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:19.866+0000 7f50d095b640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f50cc0d1960 con 0x7f50cc0d1550 2024-09-12T03:22:19.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:19.866+0000 7f50c9d9b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f50cc0d5200 0x7f50cc15aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:19.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:19.866+0000 7f50c9d9b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f50cc0d5200 0x7f50cc15aa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49178/0 (socket says 172.21.15.27:49178) 2024-09-12T03:22:19.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:19.866+0000 7f50c9d9b640 1 -- 172.21.15.27:0/3716100163 learned_addr learned my addr 172.21.15.27:0/3716100163 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:22:19.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:19.866+0000 7f50cad9d640 1 --2- 172.21.15.27:0/3716100163 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f50cc15eb90 0x7f50cc14ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:19.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:19.866+0000 7f50ca59c640 1 --2- 172.21.15.27:0/3716100163 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f50cc0d1550 0x7f50cc158c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:19.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:19.866+0000 7f50c9d9b640 1 -- 172.21.15.27:0/3716100163 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f50cc0d1550 msgr2=0x7f50cc158c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:19.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:19.866+0000 7f50c9d9b640 1 --2- 172.21.15.27:0/3716100163 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f50cc0d1550 0x7f50cc158c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:19.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:19.866+0000 7f50c9d9b640 1 -- 172.21.15.27:0/3716100163 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f50cc15eb90 msgr2=0x7f50cc14ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:19.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:19.866+0000 7f50c9d9b640 1 --2- 172.21.15.27:0/3716100163 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f50cc15eb90 0x7f50cc14ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:19.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:19.866+0000 7f50c9d9b640 1 -- 172.21.15.27:0/3716100163 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50cc1591b0 con 0x7f50cc0d5200 2024-09-12T03:22:19.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:19.866+0000 7f50ca59c640 1 --2- 172.21.15.27:0/3716100163 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f50cc0d1550 0x7f50cc158c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:22:19.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:19.866+0000 7f50c9d9b640 1 --2- 172.21.15.27:0/3716100163 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f50cc0d5200 0x7f50cc15aa10 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f50c002a2b0 tx=0x7f50c00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:19.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:19.867+0000 7f50baffd640 1 -- 172.21.15.27:0/3716100163 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50c0041020 con 0x7f50cc0d5200 2024-09-12T03:22:19.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:19.867+0000 7f50baffd640 1 -- 172.21.15.27:0/3716100163 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50c00306c0 con 0x7f50cc0d5200 2024-09-12T03:22:19.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:19.867+0000 7f50d095b640 1 -- 172.21.15.27:0/3716100163 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50cc159450 con 0x7f50cc0d5200 2024-09-12T03:22:19.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:19.867+0000 7f50baffd640 1 -- 172.21.15.27:0/3716100163 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50c0037040 con 0x7f50cc0d5200 2024-09-12T03:22:19.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:19.867+0000 7f50d095b640 1 -- 172.21.15.27:0/3716100163 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50cc159a10 con 0x7f50cc0d5200 2024-09-12T03:22:19.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:19.867+0000 7f50d095b640 1 -- 172.21.15.27:0/3716100163 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50cc15f090 con 0x7f50cc0d5200 2024-09-12T03:22:19.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:19.868+0000 7f50baffd640 1 -- 172.21.15.27:0/3716100163 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 21) ==== 91008+0+0 (secure 0 0 0) 0x7f50c0044020 con 0x7f50cc0d5200 2024-09-12T03:22:19.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:19.869+0000 7f50baffd640 1 --2- 172.21.15.27:0/3716100163 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f50ac06c3d0 0x7f50ac06e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:19.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:19.869+0000 7f50baffd640 1 -- 172.21.15.27:0/3716100163 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 5765+0+0 (secure 0 0 0) 0x7f50c00abfe0 con 0x7f50cc0d5200 2024-09-12T03:22:19.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:19.869+0000 7f50ca59c640 1 --2- 172.21.15.27:0/3716100163 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f50ac06c3d0 0x7f50ac06e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:19.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:19.869+0000 7f50ca59c640 1 --2- 172.21.15.27:0/3716100163 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f50ac06c3d0 0x7f50ac06e890 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f50bc000f60 tx=0x7f50bc009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:19.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:19.873+0000 7f50baffd640 1 -- 172.21.15.27:0/3716100163 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f50c004b030 con 0x7f50cc0d5200 2024-09-12T03:22:20.050 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:20.050+0000 7f50d095b640 1 -- 172.21.15.27:0/3716100163 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f50cc0cff00 con 0x7f50cc0d5200 2024-09-12T03:22:20.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:20.051+0000 7f50baffd640 1 -- 172.21.15.27:0/3716100163 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v125) ==== 74+0+16793 (secure 0 0 0) 0x7f50c007a170 con 0x7f50cc0d5200 2024-09-12T03:22:20.051 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:22:20.052 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":125,"fsid":"d2df504e-2647-4385-a5db-9fdf69f487a0","created":"2024-09-12T03:16:07.778937+0000","modified":"2024-09-12T03:22:18.754958+0000","last_up_change":"2024-09-12T03:16:19.314778+0000","last_in_change":"2024-09-12T03:16:09.893086+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T03:16:20.097646+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-09-12T03:22:16.425965+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"125","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-09-12T03:22:17.746336+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"125","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1851116c-1fb3-4dc9-9772-f90e2ef4c355","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6817","nonce":3002321488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6819","nonce":3002321488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6823","nonce":3002321488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6821","nonce":3002321488}]},"public_addr":"172.21.15.27:6817/3002321488","cluster_addr":"172.21.15.27:6819/3002321488","heartbeat_back_addr":"172.21.15.27:6823/3002321488","heartbeat_front_addr":"172.21.15.27:6821/3002321488","state":["exists","up"]},{"osd":1,"uuid":"f38ae17d-73eb-4204-b99d-975fa806fb87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6809","nonce":1613102886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6811","nonce":1613102886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6815","nonce":1613102886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6813","nonce":1613102886}]},"public_addr":"172.21.15.27:6809/1613102886","cluster_addr":"172.21.15.27:6811/1613102886","heartbeat_back_addr":"172.21.15.27:6815/1613102886","heartbeat_front_addr":"172.21.15.27:6813/1613102886","state":["exists","up"]},{"osd":2,"uuid":"fb12dcc8-bdd0-42b8-b46d-8b10106224bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6825","nonce":3534723355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6827","nonce":3534723355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6831","nonce":3534723355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6829","nonce":3534723355}]},"public_addr":"172.21.15.27:6825/3534723355","cluster_addr":"172.21.15.27:6827/3534723355","heartbeat_back_addr":"172.21.15.27:6831/3534723355","heartbeat_front_addr":"172.21.15.27:6829/3534723355","state":["exists","up"]},{"osd":3,"uuid":"d74d3f49-9c24-4b9f-98e2-3ecd7b5ef3b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6801","nonce":80661598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6803","nonce":80661598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6807","nonce":80661598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6805","nonce":80661598}]},"public_addr":"172.21.15.27:6801/80661598","cluster_addr":"172.21.15.27:6803/80661598","heartbeat_back_addr":"172.21.15.27:6807/80661598","heartbeat_front_addr":"172.21.15.27:6805/80661598","state":["exists","up"]},{"osd":4,"uuid":"2bf123f2-1b0d-41c4-aa3b-805bab267e59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6817","nonce":3943189827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6819","nonce":3943189827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6823","nonce":3943189827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6821","nonce":3943189827}]},"public_addr":"172.21.15.45:6817/3943189827","cluster_addr":"172.21.15.45:6819/3943189827","heartbeat_back_addr":"172.21.15.45:6823/3943189827","heartbeat_front_addr":"172.21.15.45:6821/3943189827","state":["exists","up"]},{"osd":5,"uuid":"c49ba5da-91c6-4a86-a97b-4aba89c253ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6809","nonce":3620760330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6811","nonce":3620760330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6815","nonce":3620760330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6813","nonce":3620760330}]},"public_addr":"172.21.15.45:6809/3620760330","cluster_addr":"172.21.15.45:6811/3620760330","heartbeat_back_addr":"172.21.15.45:6815/3620760330","heartbeat_front_addr":"172.21.15.45:6813/3620760330","state":["exists","up"]},{"osd":6,"uuid":"b0123475-89f7-4a34-9e33-d797b99dd9dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6801","nonce":1989083510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6803","nonce":1989083510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6807","nonce":1989083510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6805","nonce":1989083510}]},"public_addr":"172.21.15.45:6801/1989083510","cluster_addr":"172.21.15.45:6803/1989083510","heartbeat_back_addr":"172.21.15.45:6807/1989083510","heartbeat_front_addr":"172.21.15.45:6805/1989083510","state":["exists","up"]},{"osd":7,"uuid":"4dad773d-b7d8-46fc-8a10-a972aad27b29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6825","nonce":1513300294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6827","nonce":1513300294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6831","nonce":1513300294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6829","nonce":1513300294}]},"public_addr":"172.21.15.45:6825/1513300294","cluster_addr":"172.21.15.45:6827/1513300294","heartbeat_back_addr":"172.21.15.45:6831/1513300294","heartbeat_front_addr":"172.21.15.45:6829/1513300294","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.881855+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:17.133252+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.735339+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.500742+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.477571+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:17.167667+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.470291+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.873743+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]},{"pool":19,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T03:22:20.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:20.053+0000 7f50d095b640 1 -- 172.21.15.27:0/3716100163 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f50ac06c3d0 msgr2=0x7f50ac06e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:20.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:20.053+0000 7f50d095b640 1 --2- 172.21.15.27:0/3716100163 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f50ac06c3d0 0x7f50ac06e890 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f50bc000f60 tx=0x7f50bc009040 comp rx=0 tx=0).stop 2024-09-12T03:22:20.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:20.053+0000 7f50d095b640 1 -- 172.21.15.27:0/3716100163 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f50cc0d5200 msgr2=0x7f50cc15aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:20.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:20.053+0000 7f50d095b640 1 --2- 172.21.15.27:0/3716100163 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f50cc0d5200 0x7f50cc15aa10 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f50c002a2b0 tx=0x7f50c00046c0 comp rx=0 tx=0).stop 2024-09-12T03:22:20.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:20.053+0000 7f50d095b640 1 -- 172.21.15.27:0/3716100163 shutdown_connections 2024-09-12T03:22:20.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:20.053+0000 7f50d095b640 1 --2- 172.21.15.27:0/3716100163 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f50cc15eb90 0x7f50cc14ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:20.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:20.053+0000 7f50d095b640 1 --2- 172.21.15.27:0/3716100163 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f50ac06c3d0 0x7f50ac06e890 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:20.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:20.053+0000 7f50d095b640 1 --2- 172.21.15.27:0/3716100163 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f50cc0d5200 0x7f50cc15aa10 unknown :-1 s=CLOSED pgs=1229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:20.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:20.053+0000 7f50d095b640 1 --2- 172.21.15.27:0/3716100163 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f50cc0d1550 0x7f50cc158c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:20.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:20.053+0000 7f50d095b640 1 -- 172.21.15.27:0/3716100163 >> 172.21.15.27:0/3716100163 conn(0x7f50cc06dce0 msgr2=0x7f50cc160280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:20.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:20.053+0000 7f50d095b640 1 -- 172.21.15.27:0/3716100163 shutdown_connections 2024-09-12T03:22:20.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:20.053+0000 7f50d095b640 1 -- 172.21.15.27:0/3716100163 wait complete. 2024-09-12T03:22:20.064 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-12T03:22:20.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:20.170+0000 7fdb9d34b640 1 Processor -- start 2024-09-12T03:22:20.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:20.170+0000 7fdb9d34b640 1 -- start start 2024-09-12T03:22:20.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:20.171+0000 7fdb9d34b640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fdb9815e990 0x7fdb980b7600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:20.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:20.171+0000 7fdb9d34b640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdb9815ed70 con 0x7fdb980b3d10 2024-09-12T03:22:20.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:20.171+0000 7fdb9d34b640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdb980d2a30 con 0x7fdb980b6230 2024-09-12T03:22:20.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:20.171+0000 7fdb9d34b640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fdb980d2d10 con 0x7fdb9815e990 2024-09-12T03:22:20.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:20.171+0000 7fdb977fe640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fdb980b3d10 0x7fdb980d3ad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:42752/0 (socket says 172.21.15.27:42752) 2024-09-12T03:22:20.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:20.171+0000 7fdb977fe640 1 -- 172.21.15.27:0/1274760939 learned_addr learned my addr 172.21.15.27:0/1274760939 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:22:20.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:20.171+0000 7fdb95ffb640 1 -- 172.21.15.27:0/1274760939 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1870406075 0 0) 0x7fdb9815ed70 con 0x7fdb980b3d10 2024-09-12T03:22:20.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:20.171+0000 7fdb95ffb640 1 -- 172.21.15.27:0/1274760939 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdb80003520 con 0x7fdb980b3d10 2024-09-12T03:22:20.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:20.171+0000 7fdb96ffd640 1 --2- 172.21.15.27:0/1274760939 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fdb9815e990 0x7fdb980b7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:20.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:20.171+0000 7fdb95ffb640 1 -- 172.21.15.27:0/1274760939 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1518012778 0 0) 0x7fdb80003520 con 0x7fdb980b3d10 2024-09-12T03:22:20.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:20.171+0000 7fdb95ffb640 1 -- 172.21.15.27:0/1274760939 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fdb9815e990 msgr2=0x7fdb980b7600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:20.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:20.171+0000 7fdb95ffb640 1 --2- 172.21.15.27:0/1274760939 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fdb9815e990 0x7fdb980b7600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:20.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:20.171+0000 7fdb95ffb640 1 -- 172.21.15.27:0/1274760939 >> v1:172.21.15.45:6789/0 conn(0x7fdb980b6230 legacy=0x7fdb980b3600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:20.175 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:20.171+0000 7fdb95ffb640 1 -- 172.21.15.27:0/1274760939 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb980d2ff0 con 0x7fdb980b3d10 2024-09-12T03:22:20.175 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:20.171+0000 7fdb95ffb640 1 -- 172.21.15.27:0/1274760939 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 847755213 0 0) 0x7fdb900019f0 con 0x7fdb980b3d10 2024-09-12T03:22:20.175 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:20.171+0000 7fdb95ffb640 1 -- 172.21.15.27:0/1274760939 >> v1:172.21.15.27:6789/0 conn(0x7fdb980b3d10 legacy=0x7fdb980d3ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:20.175 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:20.171+0000 7fdb95ffb640 1 --2- 172.21.15.27:0/1274760939 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fdb9815e990 0x7fdb800039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:20.175 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:20.171+0000 7fdb95ffb640 1 --2- 172.21.15.27:0/1274760939 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdb80003f60 0x7fdb80004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:20.175 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:20.171+0000 7fdb95ffb640 1 --2- 172.21.15.27:0/1274760939 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdb80004d40 0x7fdb800051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:20.175 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:20.172+0000 7fdb95ffb640 1 -- 172.21.15.27:0/1274760939 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fdb980d2d10 con 0x7fdb80003f60 2024-09-12T03:22:20.175 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:20.172+0000 7fdb95ffb640 1 -- 172.21.15.27:0/1274760939 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fdb8000baa0 con 0x7fdb80004d40 2024-09-12T03:22:20.175 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:20.172+0000 7fdb95ffb640 1 -- 172.21.15.27:0/1274760939 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fdb8000bca0 con 0x7fdb9815e990 2024-09-12T03:22:20.175 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:20.172+0000 7fdb967fc640 1 --2- 172.21.15.27:0/1274760939 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdb80003f60 0x7fdb80004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:20.175 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:20.172+0000 7fdb96ffd640 1 --2- 172.21.15.27:0/1274760939 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fdb9815e990 0x7fdb800039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:20.175 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:20.172+0000 7fdb977fe640 1 --2- 172.21.15.27:0/1274760939 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdb80004d40 0x7fdb800051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:20.175 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:20.172+0000 7fdb967fc640 1 -- 172.21.15.27:0/1274760939 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fdb9815e990 msgr2=0x7fdb800039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:20.175 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:20.172+0000 7fdb967fc640 1 --2- 172.21.15.27:0/1274760939 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fdb9815e990 0x7fdb800039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:20.176 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:20.172+0000 7fdb967fc640 1 -- 172.21.15.27:0/1274760939 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdb80004d40 msgr2=0x7fdb800051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:20.176 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:20.172+0000 7fdb967fc640 1 --2- 172.21.15.27:0/1274760939 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdb80004d40 0x7fdb800051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:20.176 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:20.172+0000 7fdb967fc640 1 -- 172.21.15.27:0/1274760939 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb80003520 con 0x7fdb80003f60 2024-09-12T03:22:20.176 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:20.172+0000 7fdb977fe640 1 --2- 172.21.15.27:0/1274760939 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdb80004d40 0x7fdb800051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:22:20.176 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:20.173+0000 7fdb967fc640 1 --2- 172.21.15.27:0/1274760939 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdb80003f60 0x7fdb80004800 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7fdb8c001720 tx=0x7fdb80005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe345f5447efa2d0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:20.176 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:20.173+0000 7fdb95ffb640 1 -- 172.21.15.27:0/1274760939 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb8c001370 con 0x7fdb80003f60 2024-09-12T03:22:20.176 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:20.173+0000 7fdb95ffb640 1 -- 172.21.15.27:0/1274760939 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb8c0052d0 con 0x7fdb80003f60 2024-09-12T03:22:20.176 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:20.173+0000 7fdb95ffb640 1 -- 172.21.15.27:0/1274760939 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb8c02abc0 con 0x7fdb80003f60 2024-09-12T03:22:20.176 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:20.173+0000 7fdb9d34b640 1 -- 172.21.15.27:0/1274760939 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdb80003f60 msgr2=0x7fdb80004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:20.176 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:20.173+0000 7fdb9d34b640 1 --2- 172.21.15.27:0/1274760939 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdb80003f60 0x7fdb80004800 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7fdb8c001720 tx=0x7fdb80005830 comp rx=0 tx=0).stop 2024-09-12T03:22:20.176 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:20.173+0000 7fdb977fe640 1 -- 172.21.15.27:0/1274760939 reap_dead start 2024-09-12T03:22:20.176 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:20.173+0000 7fdb9d34b640 1 -- 172.21.15.27:0/1274760939 shutdown_connections 2024-09-12T03:22:20.176 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:20.173+0000 7fdb9d34b640 1 -- 172.21.15.27:0/1274760939 >> 172.21.15.27:0/1274760939 conn(0x7fdb980d1230 msgr2=0x7fdb980d1640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:20.177 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:20.173+0000 7fdb9d34b640 1 -- 172.21.15.27:0/1274760939 shutdown_connections 2024-09-12T03:22:20.177 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:20.173+0000 7fdb9d34b640 1 -- 172.21.15.27:0/1274760939 wait complete. 2024-09-12T03:22:20.179 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:20.175+0000 7fdb9d34b640 1 Processor -- start 2024-09-12T03:22:20.180 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:20.175+0000 7fdb9d34b640 1 -- start start 2024-09-12T03:22:20.180 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:20.175+0000 7fdb9d34b640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fdb80003f60 0x7fdb981f5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:20.180 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:20.175+0000 7fdb9d34b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdb981f6e90 0x7fdb98160f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:20.180 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:20.175+0000 7fdb9d34b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdb981f72f0 0x7fdb98161480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:20.180 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:20.175+0000 7fdb9d34b640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fdb980d41e0 con 0x7fdb981f6e90 2024-09-12T03:22:20.180 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:20.175+0000 7fdb9d34b640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fdb980d35b0 con 0x7fdb981f72f0 2024-09-12T03:22:20.180 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:20.176+0000 7fdb9d34b640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fdb980d4360 con 0x7fdb80003f60 2024-09-12T03:22:20.180 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:20.176+0000 7fdb967fc640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdb981f6e90 0x7fdb98160f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:20.180 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:20.176+0000 7fdb967fc640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdb981f6e90 0x7fdb98160f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49202/0 (socket says 172.21.15.27:49202) 2024-09-12T03:22:20.180 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:20.176+0000 7fdb967fc640 1 -- 172.21.15.27:0/2698767369 learned_addr learned my addr 172.21.15.27:0/2698767369 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:22:20.180 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:20.176+0000 7fdb96ffd640 1 --2- 172.21.15.27:0/2698767369 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fdb80003f60 0x7fdb981f5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:20.181 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:20.176+0000 7fdb977fe640 1 --2- 172.21.15.27:0/2698767369 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdb981f72f0 0x7fdb98161480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:20.181 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:20.176+0000 7fdb967fc640 1 -- 172.21.15.27:0/2698767369 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fdb80003f60 msgr2=0x7fdb981f5380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:20.181 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:20.176+0000 7fdb967fc640 1 --2- 172.21.15.27:0/2698767369 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fdb80003f60 0x7fdb981f5380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:20.181 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:20.176+0000 7fdb967fc640 1 -- 172.21.15.27:0/2698767369 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdb981f72f0 msgr2=0x7fdb98161480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:20.181 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:20.176+0000 7fdb967fc640 1 --2- 172.21.15.27:0/2698767369 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdb981f72f0 0x7fdb98161480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:20.181 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:20.176+0000 7fdb967fc640 1 -- 172.21.15.27:0/2698767369 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb980cc570 con 0x7fdb981f6e90 2024-09-12T03:22:20.181 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:20.176+0000 7fdb977fe640 1 --2- 172.21.15.27:0/2698767369 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdb981f72f0 0x7fdb98161480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:22:20.181 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:20.176+0000 7fdb967fc640 1 --2- 172.21.15.27:0/2698767369 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdb981f6e90 0x7fdb98160f40 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7fdb8c008fd0 tx=0x7fdb8c001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:20.181 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:20.176+0000 7fdb7f7fe640 1 -- 172.21.15.27:0/2698767369 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb8c042070 con 0x7fdb981f6e90 2024-09-12T03:22:20.181 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:20.176+0000 7fdb7f7fe640 1 -- 172.21.15.27:0/2698767369 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb8c005480 con 0x7fdb981f6e90 2024-09-12T03:22:20.181 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:20.176+0000 7fdb9d34b640 1 -- 172.21.15.27:0/2698767369 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb980cc850 con 0x7fdb981f6e90 2024-09-12T03:22:20.181 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:20.176+0000 7fdb7f7fe640 1 -- 172.21.15.27:0/2698767369 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb8c037040 con 0x7fdb981f6e90 2024-09-12T03:22:20.181 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:20.176+0000 7fdb9d34b640 1 -- 172.21.15.27:0/2698767369 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb980cd4d0 con 0x7fdb981f6e90 2024-09-12T03:22:20.182 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:20.177+0000 7fdb9d34b640 1 -- 172.21.15.27:0/2698767369 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb980b6690 con 0x7fdb981f6e90 2024-09-12T03:22:20.182 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:20.178+0000 7fdb7f7fe640 1 -- 172.21.15.27:0/2698767369 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 21) ==== 91008+0+0 (secure 0 0 0) 0x7fdb8c045070 con 0x7fdb981f6e90 2024-09-12T03:22:20.182 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:20.178+0000 7fdb7f7fe640 1 --2- 172.21.15.27:0/2698767369 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fdb8406c400 0x7fdb8406e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:20.182 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:20.178+0000 7fdb7f7fe640 1 -- 172.21.15.27:0/2698767369 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 5765+0+0 (secure 0 0 0) 0x7fdb8c0ab000 con 0x7fdb981f6e90 2024-09-12T03:22:20.182 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:20.178+0000 7fdb96ffd640 1 --2- 172.21.15.27:0/2698767369 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fdb8406c400 0x7fdb8406e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:20.182 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:20.179+0000 7fdb96ffd640 1 --2- 172.21.15.27:0/2698767369 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fdb8406c400 0x7fdb8406e8c0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fdb88009b00 tx=0x7fdb88008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:20.183 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:20.183+0000 7fdb7f7fe640 1 -- 172.21.15.27:0/2698767369 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fdb8c079100 con 0x7fdb981f6e90 2024-09-12T03:22:20.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:20.393+0000 7fdb9d34b640 1 -- 172.21.15.27:0/2698767369 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fdb980b3750 con 0x7fdb981f6e90 2024-09-12T03:22:20.415 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:20.415+0000 7fdb7f7fe640 1 -- 172.21.15.27:0/2698767369 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v74) ==== 118+0+0 (secure 0 0 0) 0x7fdb8c078a00 con 0x7fdb981f6e90 2024-09-12T03:22:20.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:20.417+0000 7fdb9d34b640 1 -- 172.21.15.27:0/2698767369 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fdb8406c400 msgr2=0x7fdb8406e8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:20.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:20.417+0000 7fdb9d34b640 1 --2- 172.21.15.27:0/2698767369 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fdb8406c400 0x7fdb8406e8c0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fdb88009b00 tx=0x7fdb88008040 comp rx=0 tx=0).stop 2024-09-12T03:22:20.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:20.418+0000 7fdb9d34b640 1 -- 172.21.15.27:0/2698767369 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdb981f6e90 msgr2=0x7fdb98160f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:20.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:20.418+0000 7fdb9d34b640 1 --2- 172.21.15.27:0/2698767369 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdb981f6e90 0x7fdb98160f40 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7fdb8c008fd0 tx=0x7fdb8c001120 comp rx=0 tx=0).stop 2024-09-12T03:22:20.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:20.418+0000 7fdb9d34b640 1 -- 172.21.15.27:0/2698767369 shutdown_connections 2024-09-12T03:22:20.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:20.418+0000 7fdb9d34b640 1 --2- 172.21.15.27:0/2698767369 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdb981f72f0 0x7fdb98161480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:20.419 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:20.418+0000 7fdb9d34b640 1 --2- 172.21.15.27:0/2698767369 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fdb8406c400 0x7fdb8406e8c0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:20.419 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:20.418+0000 7fdb9d34b640 1 --2- 172.21.15.27:0/2698767369 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdb981f6e90 0x7fdb98160f40 unknown :-1 s=CLOSED pgs=1232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:20.419 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:20.418+0000 7fdb9d34b640 1 --2- 172.21.15.27:0/2698767369 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fdb80003f60 0x7fdb981f5380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:20.419 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:20.418+0000 7fdb9d34b640 1 -- 172.21.15.27:0/2698767369 >> 172.21.15.27:0/2698767369 conn(0x7fdb980d1230 msgr2=0x7fdb980b95f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:20.419 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:20.418+0000 7fdb9d34b640 1 -- 172.21.15.27:0/2698767369 shutdown_connections 2024-09-12T03:22:20.419 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:20.418+0000 7fdb9d34b640 1 -- 172.21.15.27:0/2698767369 wait complete. 2024-09-12T03:22:20.430 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-12T03:22:20.430 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-12T03:22:20.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:20.536+0000 7f761c73b640 1 Processor -- start 2024-09-12T03:22:20.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:20.536+0000 7f761c73b640 1 -- start start 2024-09-12T03:22:20.539 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:20.537+0000 7f761c73b640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f761415e990 0x7f76140b7600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:20.539 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:20.537+0000 7f761c73b640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f76140b6230 0x7f76140b6690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:20.539 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:20.537+0000 7f761c73b640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f761415ed70 con 0x7f76140b3600 2024-09-12T03:22:20.539 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:20.537+0000 7f761c73b640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f76140b6bd0 con 0x7f76140b6230 2024-09-12T03:22:20.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:20.537+0000 7f761c73b640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f76140d2a30 con 0x7f761415e990 2024-09-12T03:22:20.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:20.537+0000 7f7619cd6640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f76140b6230 0x7f76140b6690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:20.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:20.537+0000 7f7619cd6640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f76140b6230 0x7f76140b6690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49214/0 (socket says 172.21.15.27:49214) 2024-09-12T03:22:20.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:20.537+0000 7f7619cd6640 1 -- 172.21.15.27:0/1861032141 learned_addr learned my addr 172.21.15.27:0/1861032141 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:22:20.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:20.537+0000 7f761a4d7640 1 --2- 172.21.15.27:0/1861032141 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f761415e990 0x7f76140b7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:20.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:20.537+0000 7f7619cd6640 1 -- 172.21.15.27:0/1861032141 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f761415e990 msgr2=0x7f76140b7600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:20.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:20.537+0000 7f7619cd6640 1 --2- 172.21.15.27:0/1861032141 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f761415e990 0x7f76140b7600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:20.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:20.537+0000 7f7619cd6640 1 -- 172.21.15.27:0/1861032141 >> v1:172.21.15.27:6789/0 conn(0x7f76140b3600 legacy=0x7f76140b3a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:20.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:20.537+0000 7f76194d5640 1 -- 172.21.15.27:0/1861032141 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1296434155 0 0) 0x7f761415ed70 con 0x7f76140b3600 2024-09-12T03:22:20.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:20.537+0000 7f7619cd6640 1 -- 172.21.15.27:0/1861032141 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76140d2d10 con 0x7f76140b6230 2024-09-12T03:22:20.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:20.538+0000 7f7619cd6640 1 --2- 172.21.15.27:0/1861032141 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f76140b6230 0x7f76140b6690 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f7608009870 tx=0x7f760802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0036abee09ffe12 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:20.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:20.538+0000 7f76194d5640 1 -- 172.21.15.27:0/1861032141 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7608037070 con 0x7f76140b6230 2024-09-12T03:22:20.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:20.538+0000 7f76194d5640 1 -- 172.21.15.27:0/1861032141 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7608032440 con 0x7f76140b6230 2024-09-12T03:22:20.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:20.538+0000 7f76194d5640 1 -- 172.21.15.27:0/1861032141 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7608032810 con 0x7f76140b6230 2024-09-12T03:22:20.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:20.539+0000 7f761c73b640 1 -- 172.21.15.27:0/1861032141 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f76140b6230 msgr2=0x7f76140b6690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:20.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:20.539+0000 7f761c73b640 1 --2- 172.21.15.27:0/1861032141 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f76140b6230 0x7f76140b6690 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f7608009870 tx=0x7f760802a270 comp rx=0 tx=0).stop 2024-09-12T03:22:20.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:20.539+0000 7f761c73b640 1 -- 172.21.15.27:0/1861032141 shutdown_connections 2024-09-12T03:22:20.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:20.539+0000 7f761c73b640 1 --2- 172.21.15.27:0/1861032141 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f76140b6230 0x7f76140b6690 unknown :-1 s=CLOSED pgs=1234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:20.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:20.539+0000 7f761c73b640 1 --2- 172.21.15.27:0/1861032141 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f761415e990 0x7f76140b7600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:20.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:20.539+0000 7f761c73b640 1 -- 172.21.15.27:0/1861032141 >> 172.21.15.27:0/1861032141 conn(0x7f76140d1230 msgr2=0x7f76140d1640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:20.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:20.539+0000 7f761c73b640 1 -- 172.21.15.27:0/1861032141 shutdown_connections 2024-09-12T03:22:20.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:20.539+0000 7f761c73b640 1 -- 172.21.15.27:0/1861032141 wait complete. 2024-09-12T03:22:20.545 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:20.540+0000 7f761c73b640 1 Processor -- start 2024-09-12T03:22:20.545 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:20.540+0000 7f761c73b640 1 -- start start 2024-09-12T03:22:20.545 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:20.541+0000 7f761c73b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f76140b3600 0x7f7614146a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:20.545 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:20.541+0000 7f761c73b640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f76140b6230 0x7f76140ca960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:20.545 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:20.541+0000 7f761c73b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f761415e990 0x7f7614148130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:20.545 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:20.541+0000 7f761c73b640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f76140d35b0 con 0x7f76140b3600 2024-09-12T03:22:20.545 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:20.541+0000 7f761c73b640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f76140d32d0 con 0x7f761415e990 2024-09-12T03:22:20.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:20.541+0000 7f761c73b640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f76140b7b40 con 0x7f76140b6230 2024-09-12T03:22:20.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:20.541+0000 7f761a4d7640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f76140b3600 0x7f7614146a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:20.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:20.541+0000 7f761a4d7640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f76140b3600 0x7f7614146a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49224/0 (socket says 172.21.15.27:49224) 2024-09-12T03:22:20.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:20.541+0000 7f761a4d7640 1 -- 172.21.15.27:0/2108797966 learned_addr learned my addr 172.21.15.27:0/2108797966 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:22:20.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:20.541+0000 7f761a4d7640 1 -- 172.21.15.27:0/2108797966 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f76140b6230 msgr2=0x7f76140ca960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:20.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:20.541+0000 7f761a4d7640 1 --2- 172.21.15.27:0/2108797966 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f76140b6230 0x7f76140ca960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:20.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:20.541+0000 7f761a4d7640 1 -- 172.21.15.27:0/2108797966 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f761415e990 msgr2=0x7f7614148130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:20.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:20.541+0000 7f761a4d7640 1 --2- 172.21.15.27:0/2108797966 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f761415e990 0x7f7614148130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:20.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:20.541+0000 7f761a4d7640 1 -- 172.21.15.27:0/2108797966 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7614147090 con 0x7f76140b3600 2024-09-12T03:22:20.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:20.541+0000 7f761a4d7640 1 --2- 172.21.15.27:0/2108797966 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f76140b3600 0x7f7614146a70 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f761000d800 tx=0x7f761000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:20.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:20.541+0000 7f7606ffd640 1 -- 172.21.15.27:0/2108797966 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f761000b6c0 con 0x7f76140b3600 2024-09-12T03:22:20.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:20.541+0000 7f7606ffd640 1 -- 172.21.15.27:0/2108797966 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7610019070 con 0x7f76140b3600 2024-09-12T03:22:20.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:20.541+0000 7f761c73b640 1 -- 172.21.15.27:0/2108797966 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7614147370 con 0x7f76140b3600 2024-09-12T03:22:20.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:20.542+0000 7f7606ffd640 1 -- 172.21.15.27:0/2108797966 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f761000ba80 con 0x7f76140b3600 2024-09-12T03:22:20.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:20.542+0000 7f761c73b640 1 -- 172.21.15.27:0/2108797966 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7614147930 con 0x7f76140b3600 2024-09-12T03:22:20.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:20.543+0000 7f761c73b640 1 -- 172.21.15.27:0/2108797966 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76140b6690 con 0x7f76140b3600 2024-09-12T03:22:20.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:20.543+0000 7f7606ffd640 1 -- 172.21.15.27:0/2108797966 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 21) ==== 91008+0+0 (secure 0 0 0) 0x7f761000bc20 con 0x7f76140b3600 2024-09-12T03:22:20.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:20.544+0000 7f7606ffd640 1 --2- 172.21.15.27:0/2108797966 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f75f806c3d0 0x7f75f806e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:20.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:20.544+0000 7f7606ffd640 1 -- 172.21.15.27:0/2108797966 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 5765+0+0 (secure 0 0 0) 0x7f761008bdf0 con 0x7f76140b3600 2024-09-12T03:22:20.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:20.544+0000 7f7619cd6640 1 --2- 172.21.15.27:0/2108797966 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f75f806c3d0 0x7f75f806e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:20.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:20.544+0000 7f7619cd6640 1 --2- 172.21.15.27:0/2108797966 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f75f806c3d0 0x7f75f806e890 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f7608034000 tx=0x7f760802c000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:20.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:20.548+0000 7f7606ffd640 1 -- 172.21.15.27:0/2108797966 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f7610059ef0 con 0x7f76140b3600 2024-09-12T03:22:20.767 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:20.760+0000 7f761c73b640 1 -- 172.21.15.27:0/2108797966 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f76140cff00 con 0x7f76140b3600 2024-09-12T03:22:21.424 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:21.424+0000 7f7606ffd640 1 -- 172.21.15.27:0/2108797966 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v75) ==== 122+0+0 (secure 0 0 0) 0x7f76100597f0 con 0x7f76140b3600 2024-09-12T03:22:21.427 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:21.427+0000 7f761c73b640 1 -- 172.21.15.27:0/2108797966 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f75f806c3d0 msgr2=0x7f75f806e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:21.427 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:21.427+0000 7f761c73b640 1 --2- 172.21.15.27:0/2108797966 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f75f806c3d0 0x7f75f806e890 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f7608034000 tx=0x7f760802c000 comp rx=0 tx=0).stop 2024-09-12T03:22:21.427 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:21.427+0000 7f761c73b640 1 -- 172.21.15.27:0/2108797966 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f76140b3600 msgr2=0x7f7614146a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:21.427 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:21.427+0000 7f761c73b640 1 --2- 172.21.15.27:0/2108797966 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f76140b3600 0x7f7614146a70 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f761000d800 tx=0x7f761000dcc0 comp rx=0 tx=0).stop 2024-09-12T03:22:21.427 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:21.427+0000 7f761c73b640 1 -- 172.21.15.27:0/2108797966 shutdown_connections 2024-09-12T03:22:21.427 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:21.427+0000 7f761c73b640 1 --2- 172.21.15.27:0/2108797966 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f761415e990 0x7f7614148130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:21.427 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:21.427+0000 7f761c73b640 1 --2- 172.21.15.27:0/2108797966 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f76140b6230 0x7f76140ca960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:21.427 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:21.427+0000 7f761c73b640 1 --2- 172.21.15.27:0/2108797966 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f75f806c3d0 0x7f75f806e890 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:21.427 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:21.427+0000 7f761c73b640 1 --2- 172.21.15.27:0/2108797966 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f76140b3600 0x7f7614146a70 unknown :-1 s=CLOSED pgs=1235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:21.427 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:21.427+0000 7f761c73b640 1 -- 172.21.15.27:0/2108797966 >> 172.21.15.27:0/2108797966 conn(0x7f76140d1230 msgr2=0x7f761415dcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:21.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:21.427+0000 7f761c73b640 1 -- 172.21.15.27:0/2108797966 shutdown_connections 2024-09-12T03:22:21.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:21.427+0000 7f761c73b640 1 -- 172.21.15.27:0/2108797966 wait complete. 2024-09-12T03:22:21.441 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-12T03:22:21.441 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-12T03:22:21.551 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:21.551+0000 7f1602f23640 1 Processor -- start 2024-09-12T03:22:21.551 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:21.551+0000 7f1602f23640 1 -- start start 2024-09-12T03:22:21.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:21.552+0000 7f1602f23640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f15fc0cf660 0x7f15fc0d2780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:21.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:21.552+0000 7f1602f23640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f15fc06cc40 con 0x7f15fc06f360 2024-09-12T03:22:21.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:21.552+0000 7f1602f23640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f15fc0d1ce0 con 0x7f15fc0b1ca0 2024-09-12T03:22:21.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:21.552+0000 7f1602f23640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f15fc0d22f0 con 0x7f15fc0cf660 2024-09-12T03:22:21.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:21.552+0000 7f15fbfff640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f15fc06f360 0x7f15fc0cef50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:42778/0 (socket says 172.21.15.27:42778) 2024-09-12T03:22:21.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:21.552+0000 7f15fbfff640 1 -- 172.21.15.27:0/3228405170 learned_addr learned my addr 172.21.15.27:0/3228405170 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:22:21.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:21.552+0000 7f15fb7fe640 1 -- 172.21.15.27:0/3228405170 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 84851372 0 0) 0x7f15fc06cc40 con 0x7f15fc06f360 2024-09-12T03:22:21.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:21.552+0000 7f15fb7fe640 1 -- 172.21.15.27:0/3228405170 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f15e8003520 con 0x7f15fc06f360 2024-09-12T03:22:21.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:21.552+0000 7f16014c0640 1 --2- 172.21.15.27:0/3228405170 >> v2:172.21.15.45:3300/0 conn(0x7f15fc0cf660 0x7f15fc0d2780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:21.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:21.552+0000 7f15fb7fe640 1 -- 172.21.15.27:0/3228405170 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1415542247 0 0) 0x7f15e8003520 con 0x7f15fc06f360 2024-09-12T03:22:21.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:21.552+0000 7f15fb7fe640 1 -- 172.21.15.27:0/3228405170 >> v2:172.21.15.45:3300/0 conn(0x7f15fc0cf660 msgr2=0x7f15fc0d2780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:21.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:21.552+0000 7f15fb7fe640 1 --2- 172.21.15.27:0/3228405170 >> v2:172.21.15.45:3300/0 conn(0x7f15fc0cf660 0x7f15fc0d2780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:21.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:21.552+0000 7f15fb7fe640 1 -- 172.21.15.27:0/3228405170 >> v1:172.21.15.45:6789/0 conn(0x7f15fc0b1ca0 legacy=0x7f15fc15f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:21.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:21.552+0000 7f15fb7fe640 1 -- 172.21.15.27:0/3228405170 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15fc06cee0 con 0x7f15fc06f360 2024-09-12T03:22:21.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:21.552+0000 7f15fb7fe640 1 -- 172.21.15.27:0/3228405170 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 847755213 0 0) 0x7f15e40019f0 con 0x7f15fc06f360 2024-09-12T03:22:21.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:21.552+0000 7f15fb7fe640 1 -- 172.21.15.27:0/3228405170 >> v1:172.21.15.27:6789/0 conn(0x7f15fc06f360 legacy=0x7f15fc0cef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:21.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:21.552+0000 7f15fb7fe640 1 --2- 172.21.15.27:0/3228405170 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f15e80039b0 0x7f15e8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:21.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:21.552+0000 7f15fb7fe640 1 --2- 172.21.15.27:0/3228405170 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f15e8004800 0x7f15e8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:21.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:21.552+0000 7f15fb7fe640 1 --2- 172.21.15.27:0/3228405170 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f15e80051c0 0x7f15e800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:21.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:21.552+0000 7f15fb7fe640 1 -- 172.21.15.27:0/3228405170 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f15fc0d22f0 con 0x7f15e8004800 2024-09-12T03:22:21.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:21.552+0000 7f15fb7fe640 1 -- 172.21.15.27:0/3228405170 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f15e800c060 con 0x7f15e80051c0 2024-09-12T03:22:21.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:21.552+0000 7f15fb7fe640 1 -- 172.21.15.27:0/3228405170 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f15e800c260 con 0x7f15e80039b0 2024-09-12T03:22:21.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:21.553+0000 7f15fbfff640 1 --2- 172.21.15.27:0/3228405170 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f15e8004800 0x7f15e8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:21.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:21.553+0000 7f1600cbf640 1 --2- 172.21.15.27:0/3228405170 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f15e80039b0 0x7f15e8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:21.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:21.553+0000 7f16014c0640 1 --2- 172.21.15.27:0/3228405170 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f15e80051c0 0x7f15e800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:21.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:21.553+0000 7f15fbfff640 1 -- 172.21.15.27:0/3228405170 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f15e80039b0 msgr2=0x7f15e8003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:21.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:21.553+0000 7f15fbfff640 1 --2- 172.21.15.27:0/3228405170 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f15e80039b0 0x7f15e8003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:21.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:21.553+0000 7f15fbfff640 1 -- 172.21.15.27:0/3228405170 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f15e80051c0 msgr2=0x7f15e800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:21.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:21.553+0000 7f15fbfff640 1 --2- 172.21.15.27:0/3228405170 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f15e80051c0 0x7f15e800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:21.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:21.553+0000 7f16014c0640 1 -- 172.21.15.27:0/3228405170 reap_dead start 2024-09-12T03:22:21.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:21.553+0000 7f15fbfff640 1 -- 172.21.15.27:0/3228405170 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15e8003520 con 0x7f15e8004800 2024-09-12T03:22:21.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:21.553+0000 7f1600cbf640 1 --2- 172.21.15.27:0/3228405170 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f15e80039b0 0x7f15e8003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:22:21.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:21.554+0000 7f15fbfff640 1 --2- 172.21.15.27:0/3228405170 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f15e8004800 0x7f15e8004c80 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f15e400a860 tx=0x7f15e8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c1da1792a357d3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:21.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:21.554+0000 7f15fb7fe640 1 -- 172.21.15.27:0/3228405170 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15e400a480 con 0x7f15e8004800 2024-09-12T03:22:21.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:21.554+0000 7f15fb7fe640 1 -- 172.21.15.27:0/3228405170 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f15e402f070 con 0x7f15e8004800 2024-09-12T03:22:21.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:21.554+0000 7f15fb7fe640 1 -- 172.21.15.27:0/3228405170 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15e402bd10 con 0x7f15e8004800 2024-09-12T03:22:21.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:21.554+0000 7f1602f23640 1 -- 172.21.15.27:0/3228405170 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f15e8004800 msgr2=0x7f15e8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:21.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:21.554+0000 7f1602f23640 1 --2- 172.21.15.27:0/3228405170 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f15e8004800 0x7f15e8004c80 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f15e400a860 tx=0x7f15e8005830 comp rx=0 tx=0).stop 2024-09-12T03:22:21.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:21.554+0000 7f1602f23640 1 -- 172.21.15.27:0/3228405170 shutdown_connections 2024-09-12T03:22:21.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:21.554+0000 7f1602f23640 1 --2- 172.21.15.27:0/3228405170 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f15e8004800 0x7f15e8004c80 unknown :-1 s=CLOSED pgs=1237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:21.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:21.554+0000 7f1602f23640 1 -- 172.21.15.27:0/3228405170 >> 172.21.15.27:0/3228405170 conn(0x7f15fc073800 msgr2=0x7f15fc09ba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:21.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:21.554+0000 7f1602f23640 1 -- 172.21.15.27:0/3228405170 shutdown_connections 2024-09-12T03:22:21.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:21.554+0000 7f1602f23640 1 -- 172.21.15.27:0/3228405170 wait complete. 2024-09-12T03:22:21.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:21.556+0000 7f1602f23640 1 Processor -- start 2024-09-12T03:22:21.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:21.556+0000 7f1602f23640 1 -- start start 2024-09-12T03:22:21.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:21.557+0000 7f1602f23640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f15e8004800 0x7f15fc155cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:21.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:21.557+0000 7f1602f23640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f15fc15fef0 0x7f15fc0b6f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:21.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:21.557+0000 7f1602f23640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f15fc0b0780 0x7f15fc0b0c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:21.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:21.557+0000 7f1602f23640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f15fc06d340 con 0x7f15fc15fef0 2024-09-12T03:22:21.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:21.557+0000 7f1602f23640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f15fc06d1c0 con 0x7f15fc0b0780 2024-09-12T03:22:21.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:21.557+0000 7f1602f23640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f15fc0cfc40 con 0x7f15e8004800 2024-09-12T03:22:21.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:21.557+0000 7f15fbfff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f15fc15fef0 0x7f15fc0b6f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:21.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:21.557+0000 7f15fbfff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f15fc15fef0 0x7f15fc0b6f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49246/0 (socket says 172.21.15.27:49246) 2024-09-12T03:22:21.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:21.557+0000 7f15fbfff640 1 -- 172.21.15.27:0/1452892887 learned_addr learned my addr 172.21.15.27:0/1452892887 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:22:21.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:21.557+0000 7f1600cbf640 1 --2- 172.21.15.27:0/1452892887 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f15e8004800 0x7f15fc155cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:21.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:21.557+0000 7f16014c0640 1 --2- 172.21.15.27:0/1452892887 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f15fc0b0780 0x7f15fc0b0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:21.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:21.557+0000 7f15fbfff640 1 -- 172.21.15.27:0/1452892887 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f15e8004800 msgr2=0x7f15fc155cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:21.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:21.557+0000 7f15fbfff640 1 --2- 172.21.15.27:0/1452892887 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f15e8004800 0x7f15fc155cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:21.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:21.557+0000 7f15fbfff640 1 -- 172.21.15.27:0/1452892887 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f15fc0b0780 msgr2=0x7f15fc0b0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:21.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:21.557+0000 7f15fbfff640 1 --2- 172.21.15.27:0/1452892887 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f15fc0b0780 0x7f15fc0b0c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:21.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:21.557+0000 7f15fbfff640 1 -- 172.21.15.27:0/1452892887 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15fc0b78b0 con 0x7f15fc15fef0 2024-09-12T03:22:21.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:21.557+0000 7f15fbfff640 1 --2- 172.21.15.27:0/1452892887 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f15fc15fef0 0x7f15fc0b6f40 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f15e400a980 tx=0x7f15fc06d0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:21.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:21.557+0000 7f15f97fa640 1 -- 172.21.15.27:0/1452892887 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15e4044070 con 0x7f15fc15fef0 2024-09-12T03:22:21.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:21.557+0000 7f15f97fa640 1 -- 172.21.15.27:0/1452892887 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f15e402be10 con 0x7f15fc15fef0 2024-09-12T03:22:21.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:21.557+0000 7f1602f23640 1 -- 172.21.15.27:0/1452892887 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15fc1f81f0 con 0x7f15fc15fef0 2024-09-12T03:22:21.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:21.557+0000 7f15f97fa640 1 -- 172.21.15.27:0/1452892887 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15e400a480 con 0x7f15fc15fef0 2024-09-12T03:22:21.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:21.557+0000 7f1602f23640 1 -- 172.21.15.27:0/1452892887 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15fc1f87b0 con 0x7f15fc15fef0 2024-09-12T03:22:21.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:21.559+0000 7f1602f23640 1 -- 172.21.15.27:0/1452892887 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15fc06f7d0 con 0x7f15fc15fef0 2024-09-12T03:22:21.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:21.559+0000 7f15f97fa640 1 -- 172.21.15.27:0/1452892887 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 21) ==== 91008+0+0 (secure 0 0 0) 0x7f15e4047070 con 0x7f15fc15fef0 2024-09-12T03:22:21.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:21.560+0000 7f15f97fa640 1 --2- 172.21.15.27:0/1452892887 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f15dc06c3d0 0x7f15dc06e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:21.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:21.560+0000 7f15f97fa640 1 -- 172.21.15.27:0/1452892887 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 5765+0+0 (secure 0 0 0) 0x7f15e40ab630 con 0x7f15fc15fef0 2024-09-12T03:22:21.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:21.560+0000 7f1600cbf640 1 --2- 172.21.15.27:0/1452892887 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f15dc06c3d0 0x7f15dc06e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:21.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:21.560+0000 7f1600cbf640 1 --2- 172.21.15.27:0/1452892887 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f15dc06c3d0 0x7f15dc06e890 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f15fc06af40 tx=0x7f15ec009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:21.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:21.564+0000 7f15f97fa640 1 -- 172.21.15.27:0/1452892887 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f15e4079890 con 0x7f15fc15fef0 2024-09-12T03:22:21.774 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:21.774+0000 7f1602f23640 1 -- 172.21.15.27:0/1452892887 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f15fc0710a0 con 0x7f15fc15fef0 2024-09-12T03:22:21.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:21.775+0000 7f15f97fa640 1 -- 172.21.15.27:0/1452892887 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 75 v75) ==== 94+0+5227 (secure 0 0 0) 0x7f15e4079190 con 0x7f15fc15fef0 2024-09-12T03:22:21.775 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:22:21.775 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":75,"btime":"2024-09-12T03:22:21:420722+0000","default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6437,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/2244842464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2244842464},{"type":"v1","addr":"172.21.15.45:6834","nonce":2244842464}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":72},{"gid":7048,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/1976058956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":1976058956},{"type":"v1","addr":"172.21.15.27:6835","nonce":1976058956}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":72},{"gid":7060,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/1084669125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":1084669125},{"type":"v1","addr":"172.21.15.45:6835","nonce":1084669125}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":72}],"filesystems":[{"mdsmap":{"epoch":75,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-12T03:22:18.755267+0000","modified":"2024-09-12T03:22:20.761064+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7725},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7725":{"gid":7725,"name":"c","rank":0,"incarnation":73,"state":"up:active","state_seq":3,"addr":"172.21.15.27:6834/1711260376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":1711260376},{"type":"v1","addr":"172.21.15.27:6834","nonce":1711260376}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7725,"qdb_cluster":[7725]},"id":10}]} 2024-09-12T03:22:21.775 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 75 2024-09-12T03:22:21.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:21.777+0000 7f1602f23640 1 -- 172.21.15.27:0/1452892887 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f15dc06c3d0 msgr2=0x7f15dc06e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:21.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:21.777+0000 7f1602f23640 1 --2- 172.21.15.27:0/1452892887 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f15dc06c3d0 0x7f15dc06e890 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f15fc06af40 tx=0x7f15ec009040 comp rx=0 tx=0).stop 2024-09-12T03:22:21.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:21.777+0000 7f1602f23640 1 -- 172.21.15.27:0/1452892887 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f15fc15fef0 msgr2=0x7f15fc0b6f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:21.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:21.777+0000 7f1602f23640 1 --2- 172.21.15.27:0/1452892887 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f15fc15fef0 0x7f15fc0b6f40 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f15e400a980 tx=0x7f15fc06d0b0 comp rx=0 tx=0).stop 2024-09-12T03:22:21.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:21.777+0000 7f1602f23640 1 -- 172.21.15.27:0/1452892887 shutdown_connections 2024-09-12T03:22:21.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:21.777+0000 7f1602f23640 1 --2- 172.21.15.27:0/1452892887 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f15fc0b0780 0x7f15fc0b0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:21.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:21.777+0000 7f1602f23640 1 --2- 172.21.15.27:0/1452892887 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f15dc06c3d0 0x7f15dc06e890 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:21.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:21.777+0000 7f1602f23640 1 --2- 172.21.15.27:0/1452892887 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f15fc15fef0 0x7f15fc0b6f40 unknown :-1 s=CLOSED pgs=1238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:21.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:21.777+0000 7f1602f23640 1 --2- 172.21.15.27:0/1452892887 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f15e8004800 0x7f15fc155cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:21.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:21.777+0000 7f1602f23640 1 -- 172.21.15.27:0/1452892887 >> 172.21.15.27:0/1452892887 conn(0x7f15fc073800 msgr2=0x7f15fc0b36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:21.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:21.777+0000 7f1602f23640 1 -- 172.21.15.27:0/1452892887 shutdown_connections 2024-09-12T03:22:21.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:21.777+0000 7f1602f23640 1 -- 172.21.15.27:0/1452892887 wait complete. 2024-09-12T03:22:21.789 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-12T03:22:21.891 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:21.891+0000 7f255dff6640 1 Processor -- start 2024-09-12T03:22:21.891 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:21.891+0000 7f255dff6640 1 -- start start 2024-09-12T03:22:21.892 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:21.892+0000 7f255dff6640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f25581590d0 0x7f25580b55d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:21.892 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:21.892+0000 7f255dff6640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f255815f600 0x7f255806bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:21.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:21.892+0000 7f255dff6640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2558162c10 con 0x7f2558158cf0 2024-09-12T03:22:21.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:21.892+0000 7f255dff6640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f2558162ef0 con 0x7f25581590d0 2024-09-12T03:22:21.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:21.892+0000 7f255dff6640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f25581631d0 con 0x7f255815f600 2024-09-12T03:22:21.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:21.892+0000 7f2556ffd640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f25581590d0 0x7f25580b55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:21.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:21.892+0000 7f2556ffd640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f25581590d0 0x7f25580b55d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49248/0 (socket says 172.21.15.27:49248) 2024-09-12T03:22:21.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:21.892+0000 7f2556ffd640 1 -- 172.21.15.27:0/2652757636 learned_addr learned my addr 172.21.15.27:0/2652757636 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:22:21.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:21.892+0000 7f2557fff640 1 --2- 172.21.15.27:0/2652757636 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f255815f600 0x7f255806bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:21.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:21.893+0000 7f2556ffd640 1 -- 172.21.15.27:0/2652757636 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f255815f600 msgr2=0x7f255806bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:21.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:21.893+0000 7f2556ffd640 1 --2- 172.21.15.27:0/2652757636 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f255815f600 0x7f255806bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:21.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:21.893+0000 7f2556ffd640 1 -- 172.21.15.27:0/2652757636 >> v1:172.21.15.45:6789/0 conn(0x7f2558158cf0 legacy=0x7f2558162430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:21.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:21.893+0000 7f2556ffd640 1 -- 172.21.15.27:0/2652757636 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f255815edc0 con 0x7f25581590d0 2024-09-12T03:22:21.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:21.893+0000 7f2557fff640 1 --2- 172.21.15.27:0/2652757636 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f255815f600 0x7f255806bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:22:21.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:21.893+0000 7f2556ffd640 1 --2- 172.21.15.27:0/2652757636 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f25581590d0 0x7f25580b55d0 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f254c009870 tx=0x7f254c02a130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76adb440b1751fc5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:21.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:21.894+0000 7f25567fc640 1 -- 172.21.15.27:0/2652757636 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f254c037070 con 0x7f25581590d0 2024-09-12T03:22:21.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:21.894+0000 7f25567fc640 1 -- 172.21.15.27:0/2652757636 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f254c032440 con 0x7f25581590d0 2024-09-12T03:22:21.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:21.894+0000 7f25567fc640 1 -- 172.21.15.27:0/2652757636 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f254c032810 con 0x7f25581590d0 2024-09-12T03:22:21.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:21.894+0000 7f255dff6640 1 -- 172.21.15.27:0/2652757636 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f25581590d0 msgr2=0x7f25580b55d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:21.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:21.894+0000 7f255dff6640 1 --2- 172.21.15.27:0/2652757636 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f25581590d0 0x7f25580b55d0 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f254c009870 tx=0x7f254c02a130 comp rx=0 tx=0).stop 2024-09-12T03:22:21.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:21.894+0000 7f255dff6640 1 -- 172.21.15.27:0/2652757636 shutdown_connections 2024-09-12T03:22:21.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:21.894+0000 7f255dff6640 1 --2- 172.21.15.27:0/2652757636 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f255815f600 0x7f255806bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:21.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:21.894+0000 7f255dff6640 1 --2- 172.21.15.27:0/2652757636 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f25581590d0 0x7f25580b55d0 unknown :-1 s=CLOSED pgs=1239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:21.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:21.894+0000 7f255dff6640 1 -- 172.21.15.27:0/2652757636 >> 172.21.15.27:0/2652757636 conn(0x7f255806cc40 msgr2=0x7f255806d050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:21.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:21.894+0000 7f255dff6640 1 -- 172.21.15.27:0/2652757636 shutdown_connections 2024-09-12T03:22:21.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:21.894+0000 7f255dff6640 1 -- 172.21.15.27:0/2652757636 wait complete. 2024-09-12T03:22:21.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:21.895+0000 7f255dff6640 1 Processor -- start 2024-09-12T03:22:21.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:21.896+0000 7f255dff6640 1 -- start start 2024-09-12T03:22:21.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:21.896+0000 7f255dff6640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2558158cf0 0x7f25581527d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:21.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:21.896+0000 7f255dff6640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f25581590d0 0x7f2558150f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:21.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:21.896+0000 7f255dff6640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f255815f600 0x7f2558153090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:21.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:21.896+0000 7f255dff6640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f255806c070 con 0x7f255815f600 2024-09-12T03:22:21.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:21.896+0000 7f255dff6640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f255815fc60 con 0x7f25581590d0 2024-09-12T03:22:21.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:21.896+0000 7f255dff6640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f255806c1f0 con 0x7f2558158cf0 2024-09-12T03:22:21.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:21.896+0000 7f2557fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f255815f600 0x7f2558153090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:21.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:21.896+0000 7f2557fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f255815f600 0x7f2558153090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49260/0 (socket says 172.21.15.27:49260) 2024-09-12T03:22:21.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:21.896+0000 7f2557fff640 1 -- 172.21.15.27:0/2122355329 learned_addr learned my addr 172.21.15.27:0/2122355329 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:22:21.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:21.897+0000 7f25577fe640 1 --2- 172.21.15.27:0/2122355329 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2558158cf0 0x7f25581527d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:21.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:21.897+0000 7f2556ffd640 1 --2- 172.21.15.27:0/2122355329 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f25581590d0 0x7f2558150f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:21.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:21.897+0000 7f2557fff640 1 -- 172.21.15.27:0/2122355329 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2558158cf0 msgr2=0x7f25581527d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:21.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:21.897+0000 7f2557fff640 1 --2- 172.21.15.27:0/2122355329 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2558158cf0 0x7f25581527d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:21.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:21.897+0000 7f2557fff640 1 -- 172.21.15.27:0/2122355329 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f25581590d0 msgr2=0x7f2558150f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:21.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:21.897+0000 7f2557fff640 1 --2- 172.21.15.27:0/2122355329 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f25581590d0 0x7f2558150f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:21.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:21.897+0000 7f2557fff640 1 -- 172.21.15.27:0/2122355329 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2558153c70 con 0x7f255815f600 2024-09-12T03:22:21.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:21.897+0000 7f2556ffd640 1 --2- 172.21.15.27:0/2122355329 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f25581590d0 0x7f2558150f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:22:21.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:21.897+0000 7f2557fff640 1 --2- 172.21.15.27:0/2122355329 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f255815f600 0x7f2558153090 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f2550007ef0 tx=0x7f255000d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:21.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:21.897+0000 7f253ffff640 1 -- 172.21.15.27:0/2122355329 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f255001a070 con 0x7f255815f600 2024-09-12T03:22:21.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:21.897+0000 7f253ffff640 1 -- 172.21.15.27:0/2122355329 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2550003b90 con 0x7f255815f600 2024-09-12T03:22:21.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:21.897+0000 7f255dff6640 1 -- 172.21.15.27:0/2122355329 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2558151a10 con 0x7f255815f600 2024-09-12T03:22:21.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:21.897+0000 7f253ffff640 1 -- 172.21.15.27:0/2122355329 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2550004030 con 0x7f255815f600 2024-09-12T03:22:21.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:21.897+0000 7f255dff6640 1 -- 172.21.15.27:0/2122355329 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2558151fd0 con 0x7f255815f600 2024-09-12T03:22:21.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:21.898+0000 7f255dff6640 1 -- 172.21.15.27:0/2122355329 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25581524c0 con 0x7f255815f600 2024-09-12T03:22:21.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:21.899+0000 7f253ffff640 1 -- 172.21.15.27:0/2122355329 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 21) ==== 91008+0+0 (secure 0 0 0) 0x7f255001d070 con 0x7f255815f600 2024-09-12T03:22:21.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:21.899+0000 7f253ffff640 1 --2- 172.21.15.27:0/2122355329 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f254406c3d0 0x7f254406e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:21.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:21.899+0000 7f253ffff640 1 -- 172.21.15.27:0/2122355329 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 5765+0+0 (secure 0 0 0) 0x7f255008bb30 con 0x7f255815f600 2024-09-12T03:22:21.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:21.899+0000 7f25577fe640 1 --2- 172.21.15.27:0/2122355329 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f254406c3d0 0x7f254406e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:21.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:21.900+0000 7f25577fe640 1 --2- 172.21.15.27:0/2122355329 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f254406c3d0 0x7f254406e890 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f2548007fd0 tx=0x7f2548009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:21.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:21.904+0000 7f253ffff640 1 -- 172.21.15.27:0/2122355329 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f2558153c70 con 0x7f255815f600 2024-09-12T03:22:22.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.078+0000 7f255dff6640 1 -- 172.21.15.27:0/2122355329 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f255806bb30 con 0x7f255815f600 2024-09-12T03:22:22.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.079+0000 7f253ffff640 1 -- 172.21.15.27:0/2122355329 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v126) ==== 74+0+16780 (secure 0 0 0) 0x7f2550010070 con 0x7f255815f600 2024-09-12T03:22:22.079 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:22:22.079 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":126,"fsid":"d2df504e-2647-4385-a5db-9fdf69f487a0","created":"2024-09-12T03:16:07.778937+0000","modified":"2024-09-12T03:22:20.406881+0000","last_up_change":"2024-09-12T03:16:19.314778+0000","last_in_change":"2024-09-12T03:16:09.893086+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T03:16:20.097646+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-09-12T03:22:16.425965+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"125","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-09-12T03:22:17.746336+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"126","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1851116c-1fb3-4dc9-9772-f90e2ef4c355","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6817","nonce":3002321488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6819","nonce":3002321488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6823","nonce":3002321488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6821","nonce":3002321488}]},"public_addr":"172.21.15.27:6817/3002321488","cluster_addr":"172.21.15.27:6819/3002321488","heartbeat_back_addr":"172.21.15.27:6823/3002321488","heartbeat_front_addr":"172.21.15.27:6821/3002321488","state":["exists","up"]},{"osd":1,"uuid":"f38ae17d-73eb-4204-b99d-975fa806fb87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6809","nonce":1613102886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6811","nonce":1613102886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6815","nonce":1613102886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6813","nonce":1613102886}]},"public_addr":"172.21.15.27:6809/1613102886","cluster_addr":"172.21.15.27:6811/1613102886","heartbeat_back_addr":"172.21.15.27:6815/1613102886","heartbeat_front_addr":"172.21.15.27:6813/1613102886","state":["exists","up"]},{"osd":2,"uuid":"fb12dcc8-bdd0-42b8-b46d-8b10106224bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6825","nonce":3534723355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6827","nonce":3534723355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6831","nonce":3534723355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6829","nonce":3534723355}]},"public_addr":"172.21.15.27:6825/3534723355","cluster_addr":"172.21.15.27:6827/3534723355","heartbeat_back_addr":"172.21.15.27:6831/3534723355","heartbeat_front_addr":"172.21.15.27:6829/3534723355","state":["exists","up"]},{"osd":3,"uuid":"d74d3f49-9c24-4b9f-98e2-3ecd7b5ef3b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6801","nonce":80661598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6803","nonce":80661598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6807","nonce":80661598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6805","nonce":80661598}]},"public_addr":"172.21.15.27:6801/80661598","cluster_addr":"172.21.15.27:6803/80661598","heartbeat_back_addr":"172.21.15.27:6807/80661598","heartbeat_front_addr":"172.21.15.27:6805/80661598","state":["exists","up"]},{"osd":4,"uuid":"2bf123f2-1b0d-41c4-aa3b-805bab267e59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6817","nonce":3943189827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6819","nonce":3943189827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6823","nonce":3943189827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6821","nonce":3943189827}]},"public_addr":"172.21.15.45:6817/3943189827","cluster_addr":"172.21.15.45:6819/3943189827","heartbeat_back_addr":"172.21.15.45:6823/3943189827","heartbeat_front_addr":"172.21.15.45:6821/3943189827","state":["exists","up"]},{"osd":5,"uuid":"c49ba5da-91c6-4a86-a97b-4aba89c253ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6809","nonce":3620760330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6811","nonce":3620760330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6815","nonce":3620760330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6813","nonce":3620760330}]},"public_addr":"172.21.15.45:6809/3620760330","cluster_addr":"172.21.15.45:6811/3620760330","heartbeat_back_addr":"172.21.15.45:6815/3620760330","heartbeat_front_addr":"172.21.15.45:6813/3620760330","state":["exists","up"]},{"osd":6,"uuid":"b0123475-89f7-4a34-9e33-d797b99dd9dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6801","nonce":1989083510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6803","nonce":1989083510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6807","nonce":1989083510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6805","nonce":1989083510}]},"public_addr":"172.21.15.45:6801/1989083510","cluster_addr":"172.21.15.45:6803/1989083510","heartbeat_back_addr":"172.21.15.45:6807/1989083510","heartbeat_front_addr":"172.21.15.45:6805/1989083510","state":["exists","up"]},{"osd":7,"uuid":"4dad773d-b7d8-46fc-8a10-a972aad27b29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6825","nonce":1513300294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6827","nonce":1513300294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6831","nonce":1513300294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6829","nonce":1513300294}]},"public_addr":"172.21.15.45:6825/1513300294","cluster_addr":"172.21.15.45:6827/1513300294","heartbeat_back_addr":"172.21.15.45:6831/1513300294","heartbeat_front_addr":"172.21.15.45:6829/1513300294","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.881855+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:17.133252+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.735339+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.500742+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.477571+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:17.167667+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.470291+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.873743+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]},{"pool":19,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T03:22:22.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.081+0000 7f255dff6640 1 -- 172.21.15.27:0/2122355329 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f254406c3d0 msgr2=0x7f254406e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:22.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.081+0000 7f255dff6640 1 --2- 172.21.15.27:0/2122355329 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f254406c3d0 0x7f254406e890 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f2548007fd0 tx=0x7f2548009040 comp rx=0 tx=0).stop 2024-09-12T03:22:22.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.081+0000 7f255dff6640 1 -- 172.21.15.27:0/2122355329 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f255815f600 msgr2=0x7f2558153090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:22.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.081+0000 7f255dff6640 1 --2- 172.21.15.27:0/2122355329 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f255815f600 0x7f2558153090 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f2550007ef0 tx=0x7f255000d4a0 comp rx=0 tx=0).stop 2024-09-12T03:22:22.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.081+0000 7f255dff6640 1 -- 172.21.15.27:0/2122355329 shutdown_connections 2024-09-12T03:22:22.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.081+0000 7f255dff6640 1 --2- 172.21.15.27:0/2122355329 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f254406c3d0 0x7f254406e890 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:22.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.081+0000 7f255dff6640 1 --2- 172.21.15.27:0/2122355329 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f255815f600 0x7f2558153090 unknown :-1 s=CLOSED pgs=1240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:22.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.081+0000 7f255dff6640 1 --2- 172.21.15.27:0/2122355329 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f25581590d0 0x7f2558150f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:22.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.081+0000 7f255dff6640 1 --2- 172.21.15.27:0/2122355329 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2558158cf0 0x7f25581527d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:22.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.081+0000 7f255dff6640 1 -- 172.21.15.27:0/2122355329 >> 172.21.15.27:0/2122355329 conn(0x7f255806cc40 msgr2=0x7f25580afad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:22.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.081+0000 7f255dff6640 1 -- 172.21.15.27:0/2122355329 shutdown_connections 2024-09-12T03:22:22.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.081+0000 7f255dff6640 1 -- 172.21.15.27:0/2122355329 wait complete. 2024-09-12T03:22:22.092 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-12T03:22:22.093 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-12T03:22:22.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.195+0000 7f2b254e6640 1 Processor -- start 2024-09-12T03:22:22.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.195+0000 7f2b254e6640 1 -- start start 2024-09-12T03:22:22.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.196+0000 7f2b254e6640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f2b200b5a70 0x7f2b2015f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:22.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.196+0000 7f2b254e6640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f2b200ae8d0 0x7f2b200728b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:22.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.196+0000 7f2b254e6640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2b2015fb40 con 0x7f2b20072df0 2024-09-12T03:22:22.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.196+0000 7f2b254e6640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f2b20073240 con 0x7f2b200ae8d0 2024-09-12T03:22:22.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.196+0000 7f2b254e6640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f2b200aee90 con 0x7f2b200b5a70 2024-09-12T03:22:22.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.196+0000 7f2b1e7fc640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f2b200ae8d0 0x7f2b200728b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:22.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.196+0000 7f2b1e7fc640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f2b200ae8d0 0x7f2b200728b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49270/0 (socket says 172.21.15.27:49270) 2024-09-12T03:22:22.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.196+0000 7f2b1e7fc640 1 -- 172.21.15.27:0/701408702 learned_addr learned my addr 172.21.15.27:0/701408702 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:22:22.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.196+0000 7f2b1f7fe640 1 --1- 172.21.15.27:0/701408702 >> v1:172.21.15.27:6789/0 conn(0x7f2b20072df0 0x7f2b201621c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:42790/0 (socket says 172.21.15.27:42790) 2024-09-12T03:22:22.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.196+0000 7f2b1effd640 1 --2- 172.21.15.27:0/701408702 >> v2:172.21.15.45:3300/0 conn(0x7f2b200b5a70 0x7f2b2015f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:22.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.196+0000 7f2b1e7fc640 1 -- 172.21.15.27:0/701408702 >> v2:172.21.15.45:3300/0 conn(0x7f2b200b5a70 msgr2=0x7f2b2015f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:22.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.196+0000 7f2b1e7fc640 1 --2- 172.21.15.27:0/701408702 >> v2:172.21.15.45:3300/0 conn(0x7f2b200b5a70 0x7f2b2015f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:22.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.196+0000 7f2b1e7fc640 1 -- 172.21.15.27:0/701408702 >> v1:172.21.15.27:6789/0 conn(0x7f2b20072df0 legacy=0x7f2b201621c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:22.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.196+0000 7f2b1dffb640 1 -- 172.21.15.27:0/701408702 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1322066170 0 0) 0x7f2b2015fb40 con 0x7f2b20072df0 2024-09-12T03:22:22.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.196+0000 7f2b1e7fc640 1 -- 172.21.15.27:0/701408702 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b200af170 con 0x7f2b200ae8d0 2024-09-12T03:22:22.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.197+0000 7f2b1e7fc640 1 --2- 172.21.15.27:0/701408702 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2b200ae8d0 0x7f2b200728b0 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7f2b20062870 tx=0x7f2b1402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c33c8719e8a4729f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:22.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.197+0000 7f2b1dffb640 1 -- 172.21.15.27:0/701408702 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b14037070 con 0x7f2b200ae8d0 2024-09-12T03:22:22.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.197+0000 7f2b1dffb640 1 -- 172.21.15.27:0/701408702 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b14032440 con 0x7f2b200ae8d0 2024-09-12T03:22:22.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.197+0000 7f2b1dffb640 1 -- 172.21.15.27:0/701408702 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b14032810 con 0x7f2b200ae8d0 2024-09-12T03:22:22.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.198+0000 7f2b254e6640 1 -- 172.21.15.27:0/701408702 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2b200ae8d0 msgr2=0x7f2b200728b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:22.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.198+0000 7f2b254e6640 1 --2- 172.21.15.27:0/701408702 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2b200ae8d0 0x7f2b200728b0 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7f2b20062870 tx=0x7f2b1402a1e0 comp rx=0 tx=0).stop 2024-09-12T03:22:22.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.198+0000 7f2b254e6640 1 -- 172.21.15.27:0/701408702 shutdown_connections 2024-09-12T03:22:22.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.198+0000 7f2b254e6640 1 --2- 172.21.15.27:0/701408702 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2b200ae8d0 0x7f2b200728b0 unknown :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:22.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.198+0000 7f2b254e6640 1 --2- 172.21.15.27:0/701408702 >> v2:172.21.15.45:3300/0 conn(0x7f2b200b5a70 0x7f2b2015f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:22.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.198+0000 7f2b254e6640 1 -- 172.21.15.27:0/701408702 >> 172.21.15.27:0/701408702 conn(0x7f2b2006ca30 msgr2=0x7f2b2006ce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:22.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.198+0000 7f2b254e6640 1 -- 172.21.15.27:0/701408702 shutdown_connections 2024-09-12T03:22:22.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.198+0000 7f2b254e6640 1 -- 172.21.15.27:0/701408702 wait complete. 2024-09-12T03:22:22.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.199+0000 7f2b254e6640 1 Processor -- start 2024-09-12T03:22:22.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.199+0000 7f2b254e6640 1 -- start start 2024-09-12T03:22:22.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.200+0000 7f2b254e6640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2b20072df0 0x7f2b200cd8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:22.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.200+0000 7f2b254e6640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2b200ae8d0 0x7f2b2014d750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:22.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.200+0000 7f2b254e6640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2b200b5a70 0x7f2b2015a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:22.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.200+0000 7f2b254e6640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f2b2006f940 con 0x7f2b200b5a70 2024-09-12T03:22:22.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.200+0000 7f2b254e6640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f2b200af730 con 0x7f2b20072df0 2024-09-12T03:22:22.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.200+0000 7f2b254e6640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f2b2006fac0 con 0x7f2b200ae8d0 2024-09-12T03:22:22.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.200+0000 7f2b1f7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2b200b5a70 0x7f2b2015a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:22.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.200+0000 7f2b1f7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2b200b5a70 0x7f2b2015a340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49286/0 (socket says 172.21.15.27:49286) 2024-09-12T03:22:22.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.200+0000 7f2b1f7fe640 1 -- 172.21.15.27:0/997854363 learned_addr learned my addr 172.21.15.27:0/997854363 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:22:22.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.200+0000 7f2b1effd640 1 --2- 172.21.15.27:0/997854363 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2b20072df0 0x7f2b200cd8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:22.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.200+0000 7f2b1e7fc640 1 --2- 172.21.15.27:0/997854363 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2b200ae8d0 0x7f2b2014d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:22.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.200+0000 7f2b1f7fe640 1 -- 172.21.15.27:0/997854363 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2b200ae8d0 msgr2=0x7f2b2014d750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:22.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.200+0000 7f2b1f7fe640 1 --2- 172.21.15.27:0/997854363 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2b200ae8d0 0x7f2b2014d750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:22.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.200+0000 7f2b1f7fe640 1 -- 172.21.15.27:0/997854363 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2b20072df0 msgr2=0x7f2b200cd8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:22.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.200+0000 7f2b1f7fe640 1 --2- 172.21.15.27:0/997854363 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2b20072df0 0x7f2b200cd8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:22.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.200+0000 7f2b1f7fe640 1 -- 172.21.15.27:0/997854363 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b201f5430 con 0x7f2b200b5a70 2024-09-12T03:22:22.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.200+0000 7f2b1f7fe640 1 --2- 172.21.15.27:0/997854363 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2b200b5a70 0x7f2b2015a340 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7f2b1800c7a0 tx=0x7f2b1800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:22.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.201+0000 7f2b077fe640 1 -- 172.21.15.27:0/997854363 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b18018070 con 0x7f2b200b5a70 2024-09-12T03:22:22.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.201+0000 7f2b077fe640 1 -- 172.21.15.27:0/997854363 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b1800a640 con 0x7f2b200b5a70 2024-09-12T03:22:22.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.201+0000 7f2b254e6640 1 -- 172.21.15.27:0/997854363 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b201f5710 con 0x7f2b200b5a70 2024-09-12T03:22:22.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.201+0000 7f2b077fe640 1 -- 172.21.15.27:0/997854363 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b1800aab0 con 0x7f2b200b5a70 2024-09-12T03:22:22.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.201+0000 7f2b254e6640 1 -- 172.21.15.27:0/997854363 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b201f5cd0 con 0x7f2b200b5a70 2024-09-12T03:22:22.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.202+0000 7f2b254e6640 1 -- 172.21.15.27:0/997854363 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b200728b0 con 0x7f2b200b5a70 2024-09-12T03:22:22.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.202+0000 7f2b077fe640 1 -- 172.21.15.27:0/997854363 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 21) ==== 91008+0+0 (secure 0 0 0) 0x7f2b1801b070 con 0x7f2b200b5a70 2024-09-12T03:22:22.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.203+0000 7f2b077fe640 1 --2- 172.21.15.27:0/997854363 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f2b0c06c3d0 0x7f2b0c06e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:22.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.203+0000 7f2b077fe640 1 -- 172.21.15.27:0/997854363 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 5765+0+0 (secure 0 0 0) 0x7f2b1808af30 con 0x7f2b200b5a70 2024-09-12T03:22:22.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.203+0000 7f2b1effd640 1 --2- 172.21.15.27:0/997854363 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f2b0c06c3d0 0x7f2b0c06e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:22.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.203+0000 7f2b1effd640 1 --2- 172.21.15.27:0/997854363 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f2b0c06c3d0 0x7f2b0c06e890 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f2b10009a30 tx=0x7f2b10008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:22.207 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.207+0000 7f2b077fe640 1 -- 172.21.15.27:0/997854363 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f2b18059030 con 0x7f2b200b5a70 2024-09-12T03:22:22.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.381+0000 7f2b254e6640 1 -- 172.21.15.27:0/997854363 --> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2b200b3ff0 con 0x7f2b0c06c3d0 2024-09-12T03:22:22.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.387+0000 7f2b077fe640 1 -- 172.21.15.27:0/997854363 <== mgr.4113 v2:172.21.15.27:6836/1226920526 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252482 (secure 0 0 0) 0x7f2b200b3ff0 con 0x7f2b0c06c3d0 2024-09-12T03:22:22.390 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:22:22.391 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2024-09-12T03:22:22.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.390+0000 7f2b254e6640 1 -- 172.21.15.27:0/997854363 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f2b0c06c3d0 msgr2=0x7f2b0c06e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:22.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.390+0000 7f2b254e6640 1 --2- 172.21.15.27:0/997854363 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f2b0c06c3d0 0x7f2b0c06e890 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f2b10009a30 tx=0x7f2b10008040 comp rx=0 tx=0).stop 2024-09-12T03:22:22.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.390+0000 7f2b254e6640 1 -- 172.21.15.27:0/997854363 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2b200b5a70 msgr2=0x7f2b2015a340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:22.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.390+0000 7f2b254e6640 1 --2- 172.21.15.27:0/997854363 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2b200b5a70 0x7f2b2015a340 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7f2b1800c7a0 tx=0x7f2b1800cc60 comp rx=0 tx=0).stop 2024-09-12T03:22:22.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.390+0000 7f2b254e6640 1 -- 172.21.15.27:0/997854363 shutdown_connections 2024-09-12T03:22:22.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.390+0000 7f2b254e6640 1 --2- 172.21.15.27:0/997854363 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f2b0c06c3d0 0x7f2b0c06e890 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:22.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.390+0000 7f2b254e6640 1 --2- 172.21.15.27:0/997854363 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2b200b5a70 0x7f2b2015a340 unknown :-1 s=CLOSED pgs=1243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:22.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.390+0000 7f2b254e6640 1 --2- 172.21.15.27:0/997854363 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2b200ae8d0 0x7f2b2014d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:22.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.390+0000 7f2b254e6640 1 --2- 172.21.15.27:0/997854363 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2b20072df0 0x7f2b200cd8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:22.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.391+0000 7f2b254e6640 1 -- 172.21.15.27:0/997854363 >> 172.21.15.27:0/997854363 conn(0x7f2b2006ca30 msgr2=0x7f2b20157780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:22.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.391+0000 7f2b254e6640 1 -- 172.21.15.27:0/997854363 shutdown_connections 2024-09-12T03:22:22.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.391+0000 7f2b254e6640 1 -- 172.21.15.27:0/997854363 wait complete. 2024-09-12T03:22:22.401 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":123,"stamp":"2024-09-12T03:22:21.121894+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":224,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":167,"ondisk_log_size":167,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":80,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":279680,"kb_used_data":13496,"kb_used_omap":12,"kb_used_meta":265971,"kb_avail":754695040,"statfs":{"total":773094113280,"available":772807720960,"internally_reserved":0,"allocated":13819904,"data_stored":6740780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":272354864},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":23,"apply_latency_ms":23,"commit_latency_ns":23000000,"apply_latency_ns":23000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000693"},"pg_stats":[{"pgid":"21.3f","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T03:22:20.427934+0000","last_change":"2024-09-12T03:22:19.425643+0000","last_active":"2024-09-12T03:22:20.427934+0000","last_peered":"2024-09-12T03:22:20.427934+0000","last_clean":"2024-09-12T03:22:20.427934+0000","last_became_active":"2024-09-12T03:22:19.425322+0000","last_became_peered":"2024-09-12T03:22:19.425322+0000","last_unstale":"2024-09-12T03:22:20.427934+0000","last_undegraded":"2024-09-12T03:22:20.427934+0000","last_fullsized":"2024-09-12T03:22:20.427934+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_clean_scrub_stamp":"2024-09-12T03:22:18.392443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:45:31.104107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.3e","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T03:22:20.565814+0000","last_change":"2024-09-12T03:22:19.430562+0000","last_active":"2024-09-12T03:22:20.565814+0000","last_peered":"2024-09-12T03:22:20.565814+0000","last_clean":"2024-09-12T03:22:20.565814+0000","last_became_active":"2024-09-12T03:22:19.430254+0000","last_became_peered":"2024-09-12T03:22:19.430254+0000","last_unstale":"2024-09-12T03:22:20.565814+0000","last_undegraded":"2024-09-12T03:22:20.565814+0000","last_fullsized":"2024-09-12T03:22:20.565814+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_clean_scrub_stamp":"2024-09-12T03:22:18.392443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:50:24.103190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.3d","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T03:22:20.566564+0000","last_change":"2024-09-12T03:22:19.439778+0000","last_active":"2024-09-12T03:22:20.566564+0000","last_peered":"2024-09-12T03:22:20.566564+0000","last_clean":"2024-09-12T03:22:20.566564+0000","last_became_active":"2024-09-12T03:22:19.439263+0000","last_became_peered":"2024-09-12T03:22:19.439263+0000","last_unstale":"2024-09-12T03:22:20.566564+0000","last_undegraded":"2024-09-12T03:22:20.566564+0000","last_fullsized":"2024-09-12T03:22:20.566564+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_clean_scrub_stamp":"2024-09-12T03:22:18.392443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:54:05.437535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.3c","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T03:22:20.411897+0000","last_change":"2024-09-12T03:22:19.432003+0000","last_active":"2024-09-12T03:22:20.411897+0000","last_peered":"2024-09-12T03:22:20.411897+0000","last_clean":"2024-09-12T03:22:20.411897+0000","last_became_active":"2024-09-12T03:22:19.431621+0000","last_became_peered":"2024-09-12T03:22:19.431621+0000","last_unstale":"2024-09-12T03:22:20.411897+0000","last_undegraded":"2024-09-12T03:22:20.411897+0000","last_fullsized":"2024-09-12T03:22:20.411897+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_clean_scrub_stamp":"2024-09-12T03:22:18.392443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:56:49.416621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.3b","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T03:22:20.411612+0000","last_change":"2024-09-12T03:22:19.433231+0000","last_active":"2024-09-12T03:22:20.411612+0000","last_peered":"2024-09-12T03:22:20.411612+0000","last_clean":"2024-09-12T03:22:20.411612+0000","last_became_active":"2024-09-12T03:22:19.432900+0000","last_became_peered":"2024-09-12T03:22:19.432900+0000","last_unstale":"2024-09-12T03:22:20.411612+0000","last_undegraded":"2024-09-12T03:22:20.411612+0000","last_fullsized":"2024-09-12T03:22:20.411612+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_clean_scrub_stamp":"2024-09-12T03:22:18.392443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:15:00.096228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.3a","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T03:22:20.427533+0000","last_change":"2024-09-12T03:22:19.436158+0000","last_active":"2024-09-12T03:22:20.427533+0000","last_peered":"2024-09-12T03:22:20.427533+0000","last_clean":"2024-09-12T03:22:20.427533+0000","last_became_active":"2024-09-12T03:22:19.435866+0000","last_became_peered":"2024-09-12T03:22:19.435866+0000","last_unstale":"2024-09-12T03:22:20.427533+0000","last_undegraded":"2024-09-12T03:22:20.427533+0000","last_fullsized":"2024-09-12T03:22:20.427533+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_clean_scrub_stamp":"2024-09-12T03:22:18.392443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:40:55.731312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.39","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T03:22:20.410136+0000","last_change":"2024-09-12T03:22:19.418842+0000","last_active":"2024-09-12T03:22:20.410136+0000","last_peered":"2024-09-12T03:22:20.410136+0000","last_clean":"2024-09-12T03:22:20.410136+0000","last_became_active":"2024-09-12T03:22:19.418467+0000","last_became_peered":"2024-09-12T03:22:19.418467+0000","last_unstale":"2024-09-12T03:22:20.410136+0000","last_undegraded":"2024-09-12T03:22:20.410136+0000","last_fullsized":"2024-09-12T03:22:20.410136+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_clean_scrub_stamp":"2024-09-12T03:22:18.392443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:38:46.337249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.38","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T03:22:20.427327+0000","last_change":"2024-09-12T03:22:19.413757+0000","last_active":"2024-09-12T03:22:20.427327+0000","last_peered":"2024-09-12T03:22:20.427327+0000","last_clean":"2024-09-12T03:22:20.427327+0000","last_became_active":"2024-09-12T03:22:19.413424+0000","last_became_peered":"2024-09-12T03:22:19.413424+0000","last_unstale":"2024-09-12T03:22:20.427327+0000","last_undegraded":"2024-09-12T03:22:20.427327+0000","last_fullsized":"2024-09-12T03:22:20.427327+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_clean_scrub_stamp":"2024-09-12T03:22:18.392443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:47:26.560867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.37","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T03:22:20.565846+0000","last_change":"2024-09-12T03:22:19.428309+0000","last_active":"2024-09-12T03:22:20.565846+0000","last_peered":"2024-09-12T03:22:20.565846+0000","last_clean":"2024-09-12T03:22:20.565846+0000","last_became_active":"2024-09-12T03:22:19.427896+0000","last_became_peered":"2024-09-12T03:22:19.427896+0000","last_unstale":"2024-09-12T03:22:20.565846+0000","last_undegraded":"2024-09-12T03:22:20.565846+0000","last_fullsized":"2024-09-12T03:22:20.565846+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_clean_scrub_stamp":"2024-09-12T03:22:18.392443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:27:40.916612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.36","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T03:22:20.430842+0000","last_change":"2024-09-12T03:22:19.434045+0000","last_active":"2024-09-12T03:22:20.430842+0000","last_peered":"2024-09-12T03:22:20.430842+0000","last_clean":"2024-09-12T03:22:20.430842+0000","last_became_active":"2024-09-12T03:22:19.431316+0000","last_became_peered":"2024-09-12T03:22:19.431316+0000","last_unstale":"2024-09-12T03:22:20.430842+0000","last_undegraded":"2024-09-12T03:22:20.430842+0000","last_fullsized":"2024-09-12T03:22:20.430842+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_clean_scrub_stamp":"2024-09-12T03:22:18.392443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:31:57.234454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.35","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T03:22:20.427449+0000","last_change":"2024-09-12T03:22:19.413798+0000","last_active":"2024-09-12T03:22:20.427449+0000","last_peered":"2024-09-12T03:22:20.427449+0000","last_clean":"2024-09-12T03:22:20.427449+0000","last_became_active":"2024-09-12T03:22:19.413485+0000","last_became_peered":"2024-09-12T03:22:19.413485+0000","last_unstale":"2024-09-12T03:22:20.427449+0000","last_undegraded":"2024-09-12T03:22:20.427449+0000","last_fullsized":"2024-09-12T03:22:20.427449+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_clean_scrub_stamp":"2024-09-12T03:22:18.392443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:50:41.758774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.34","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T03:22:20.430725+0000","last_change":"2024-09-12T03:22:19.435530+0000","last_active":"2024-09-12T03:22:20.430725+0000","last_peered":"2024-09-12T03:22:20.430725+0000","last_clean":"2024-09-12T03:22:20.430725+0000","last_became_active":"2024-09-12T03:22:19.434535+0000","last_became_peered":"2024-09-12T03:22:19.434535+0000","last_unstale":"2024-09-12T03:22:20.430725+0000","last_undegraded":"2024-09-12T03:22:20.430725+0000","last_fullsized":"2024-09-12T03:22:20.430725+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_clean_scrub_stamp":"2024-09-12T03:22:18.392443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:39:01.904225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.33","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T03:22:20.411817+0000","last_change":"2024-09-12T03:22:19.432235+0000","last_active":"2024-09-12T03:22:20.411817+0000","last_peered":"2024-09-12T03:22:20.411817+0000","last_clean":"2024-09-12T03:22:20.411817+0000","last_became_active":"2024-09-12T03:22:19.431932+0000","last_became_peered":"2024-09-12T03:22:19.431932+0000","last_unstale":"2024-09-12T03:22:20.411817+0000","last_undegraded":"2024-09-12T03:22:20.411817+0000","last_fullsized":"2024-09-12T03:22:20.411817+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_clean_scrub_stamp":"2024-09-12T03:22:18.392443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:10:59.293591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.32","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T03:22:20.412125+0000","last_change":"2024-09-12T03:22:19.427849+0000","last_active":"2024-09-12T03:22:20.412125+0000","last_peered":"2024-09-12T03:22:20.412125+0000","last_clean":"2024-09-12T03:22:20.412125+0000","last_became_active":"2024-09-12T03:22:19.427556+0000","last_became_peered":"2024-09-12T03:22:19.427556+0000","last_unstale":"2024-09-12T03:22:20.412125+0000","last_undegraded":"2024-09-12T03:22:20.412125+0000","last_fullsized":"2024-09-12T03:22:20.412125+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_clean_scrub_stamp":"2024-09-12T03:22:18.392443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:06:53.485657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.31","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T03:22:20.430265+0000","last_change":"2024-09-12T03:22:19.426135+0000","last_active":"2024-09-12T03:22:20.430265+0000","last_peered":"2024-09-12T03:22:20.430265+0000","last_clean":"2024-09-12T03:22:20.430265+0000","last_became_active":"2024-09-12T03:22:19.425841+0000","last_became_peered":"2024-09-12T03:22:19.425841+0000","last_unstale":"2024-09-12T03:22:20.430265+0000","last_undegraded":"2024-09-12T03:22:20.430265+0000","last_fullsized":"2024-09-12T03:22:20.430265+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_clean_scrub_stamp":"2024-09-12T03:22:18.392443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:34:26.552409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.30","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T03:22:20.566448+0000","last_change":"2024-09-12T03:22:19.430530+0000","last_active":"2024-09-12T03:22:20.566448+0000","last_peered":"2024-09-12T03:22:20.566448+0000","last_clean":"2024-09-12T03:22:20.566448+0000","last_became_active":"2024-09-12T03:22:19.430245+0000","last_became_peered":"2024-09-12T03:22:19.430245+0000","last_unstale":"2024-09-12T03:22:20.566448+0000","last_undegraded":"2024-09-12T03:22:20.566448+0000","last_fullsized":"2024-09-12T03:22:20.566448+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_clean_scrub_stamp":"2024-09-12T03:22:18.392443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:08:54.698701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.2f","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T03:22:20.430943+0000","last_change":"2024-09-12T03:22:19.436725+0000","last_active":"2024-09-12T03:22:20.430943+0000","last_peered":"2024-09-12T03:22:20.430943+0000","last_clean":"2024-09-12T03:22:20.430943+0000","last_became_active":"2024-09-12T03:22:19.431721+0000","last_became_peered":"2024-09-12T03:22:19.431721+0000","last_unstale":"2024-09-12T03:22:20.430943+0000","last_undegraded":"2024-09-12T03:22:20.430943+0000","last_fullsized":"2024-09-12T03:22:20.430943+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_clean_scrub_stamp":"2024-09-12T03:22:18.392443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:36:43.352083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.2e","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T03:22:20.566617+0000","last_change":"2024-09-12T03:22:19.439648+0000","last_active":"2024-09-12T03:22:20.566617+0000","last_peered":"2024-09-12T03:22:20.566617+0000","last_clean":"2024-09-12T03:22:20.566617+0000","last_became_active":"2024-09-12T03:22:19.439341+0000","last_became_peered":"2024-09-12T03:22:19.439341+0000","last_unstale":"2024-09-12T03:22:20.566617+0000","last_undegraded":"2024-09-12T03:22:20.566617+0000","last_fullsized":"2024-09-12T03:22:20.566617+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_clean_scrub_stamp":"2024-09-12T03:22:18.392443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:00:45.005368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.2d","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T03:22:20.427625+0000","last_change":"2024-09-12T03:22:19.430547+0000","last_active":"2024-09-12T03:22:20.427625+0000","last_peered":"2024-09-12T03:22:20.427625+0000","last_clean":"2024-09-12T03:22:20.427625+0000","last_became_active":"2024-09-12T03:22:19.430148+0000","last_became_peered":"2024-09-12T03:22:19.430148+0000","last_unstale":"2024-09-12T03:22:20.427625+0000","last_undegraded":"2024-09-12T03:22:20.427625+0000","last_fullsized":"2024-09-12T03:22:20.427625+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_clean_scrub_stamp":"2024-09-12T03:22:18.392443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:04:26.613191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2c","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T03:22:20.566114+0000","last_change":"2024-09-12T03:22:19.428110+0000","last_active":"2024-09-12T03:22:20.566114+0000","last_peered":"2024-09-12T03:22:20.566114+0000","last_clean":"2024-09-12T03:22:20.566114+0000","last_became_active":"2024-09-12T03:22:19.427817+0000","last_became_peered":"2024-09-12T03:22:19.427817+0000","last_unstale":"2024-09-12T03:22:20.566114+0000","last_undegraded":"2024-09-12T03:22:20.566114+0000","last_fullsized":"2024-09-12T03:22:20.566114+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_clean_scrub_stamp":"2024-09-12T03:22:18.392443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:21:14.749329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.2b","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T03:22:20.430651+0000","last_change":"2024-09-12T03:22:19.432107+0000","last_active":"2024-09-12T03:22:20.430651+0000","last_peered":"2024-09-12T03:22:20.430651+0000","last_clean":"2024-09-12T03:22:20.430651+0000","last_became_active":"2024-09-12T03:22:19.430805+0000","last_became_peered":"2024-09-12T03:22:19.430805+0000","last_unstale":"2024-09-12T03:22:20.430651+0000","last_undegraded":"2024-09-12T03:22:20.430651+0000","last_fullsized":"2024-09-12T03:22:20.430651+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_clean_scrub_stamp":"2024-09-12T03:22:18.392443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:02:41.802998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.2a","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T03:22:20.412208+0000","last_change":"2024-09-12T03:22:19.426007+0000","last_active":"2024-09-12T03:22:20.412208+0000","last_peered":"2024-09-12T03:22:20.412208+0000","last_clean":"2024-09-12T03:22:20.412208+0000","last_became_active":"2024-09-12T03:22:19.424984+0000","last_became_peered":"2024-09-12T03:22:19.424984+0000","last_unstale":"2024-09-12T03:22:20.412208+0000","last_undegraded":"2024-09-12T03:22:20.412208+0000","last_fullsized":"2024-09-12T03:22:20.412208+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_clean_scrub_stamp":"2024-09-12T03:22:18.392443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:07:58.025130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.29","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T03:22:20.411721+0000","last_change":"2024-09-12T03:22:19.422131+0000","last_active":"2024-09-12T03:22:20.411721+0000","last_peered":"2024-09-12T03:22:20.411721+0000","last_clean":"2024-09-12T03:22:20.411721+0000","last_became_active":"2024-09-12T03:22:19.421483+0000","last_became_peered":"2024-09-12T03:22:19.421483+0000","last_unstale":"2024-09-12T03:22:20.411721+0000","last_undegraded":"2024-09-12T03:22:20.411721+0000","last_fullsized":"2024-09-12T03:22:20.411721+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_clean_scrub_stamp":"2024-09-12T03:22:18.392443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:52:23.195824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.28","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T03:22:20.430244+0000","last_change":"2024-09-12T03:22:19.430260+0000","last_active":"2024-09-12T03:22:20.430244+0000","last_peered":"2024-09-12T03:22:20.430244+0000","last_clean":"2024-09-12T03:22:20.430244+0000","last_became_active":"2024-09-12T03:22:19.429972+0000","last_became_peered":"2024-09-12T03:22:19.429972+0000","last_unstale":"2024-09-12T03:22:20.430244+0000","last_undegraded":"2024-09-12T03:22:20.430244+0000","last_fullsized":"2024-09-12T03:22:20.430244+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_clean_scrub_stamp":"2024-09-12T03:22:18.392443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:31:58.401020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.27","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T03:22:20.566124+0000","last_change":"2024-09-12T03:22:19.435157+0000","last_active":"2024-09-12T03:22:20.566124+0000","last_peered":"2024-09-12T03:22:20.566124+0000","last_clean":"2024-09-12T03:22:20.566124+0000","last_became_active":"2024-09-12T03:22:19.434780+0000","last_became_peered":"2024-09-12T03:22:19.434780+0000","last_unstale":"2024-09-12T03:22:20.566124+0000","last_undegraded":"2024-09-12T03:22:20.566124+0000","last_fullsized":"2024-09-12T03:22:20.566124+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_clean_scrub_stamp":"2024-09-12T03:22:18.392443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:49:39.702721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.26","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T03:22:20.427668+0000","last_change":"2024-09-12T03:22:19.424459+0000","last_active":"2024-09-12T03:22:20.427668+0000","last_peered":"2024-09-12T03:22:20.427668+0000","last_clean":"2024-09-12T03:22:20.427668+0000","last_became_active":"2024-09-12T03:22:19.424175+0000","last_became_peered":"2024-09-12T03:22:19.424175+0000","last_unstale":"2024-09-12T03:22:20.427668+0000","last_undegraded":"2024-09-12T03:22:20.427668+0000","last_fullsized":"2024-09-12T03:22:20.427668+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_clean_scrub_stamp":"2024-09-12T03:22:18.392443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:26:03.322839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.25","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T03:22:20.565865+0000","last_change":"2024-09-12T03:22:19.429027+0000","last_active":"2024-09-12T03:22:20.565865+0000","last_peered":"2024-09-12T03:22:20.565865+0000","last_clean":"2024-09-12T03:22:20.565865+0000","last_became_active":"2024-09-12T03:22:19.428729+0000","last_became_peered":"2024-09-12T03:22:19.428729+0000","last_unstale":"2024-09-12T03:22:20.565865+0000","last_undegraded":"2024-09-12T03:22:20.565865+0000","last_fullsized":"2024-09-12T03:22:20.565865+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_clean_scrub_stamp":"2024-09-12T03:22:18.392443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:55:06.811462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.24","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T03:22:20.410334+0000","last_change":"2024-09-12T03:22:19.430768+0000","last_active":"2024-09-12T03:22:20.410334+0000","last_peered":"2024-09-12T03:22:20.410334+0000","last_clean":"2024-09-12T03:22:20.410334+0000","last_became_active":"2024-09-12T03:22:19.430490+0000","last_became_peered":"2024-09-12T03:22:19.430490+0000","last_unstale":"2024-09-12T03:22:20.410334+0000","last_undegraded":"2024-09-12T03:22:20.410334+0000","last_fullsized":"2024-09-12T03:22:20.410334+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_clean_scrub_stamp":"2024-09-12T03:22:18.392443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:40:56.406906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.23","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T03:22:20.427422+0000","last_change":"2024-09-12T03:22:19.435437+0000","last_active":"2024-09-12T03:22:20.427422+0000","last_peered":"2024-09-12T03:22:20.427422+0000","last_clean":"2024-09-12T03:22:20.427422+0000","last_became_active":"2024-09-12T03:22:19.435151+0000","last_became_peered":"2024-09-12T03:22:19.435151+0000","last_unstale":"2024-09-12T03:22:20.427422+0000","last_undegraded":"2024-09-12T03:22:20.427422+0000","last_fullsized":"2024-09-12T03:22:20.427422+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_clean_scrub_stamp":"2024-09-12T03:22:18.392443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:04:59.936109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.22","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T03:22:20.412289+0000","last_change":"2024-09-12T03:22:19.426401+0000","last_active":"2024-09-12T03:22:20.412289+0000","last_peered":"2024-09-12T03:22:20.412289+0000","last_clean":"2024-09-12T03:22:20.412289+0000","last_became_active":"2024-09-12T03:22:19.425770+0000","last_became_peered":"2024-09-12T03:22:19.425770+0000","last_unstale":"2024-09-12T03:22:20.412289+0000","last_undegraded":"2024-09-12T03:22:20.412289+0000","last_fullsized":"2024-09-12T03:22:20.412289+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_clean_scrub_stamp":"2024-09-12T03:22:18.392443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:30:36.215990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.21","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T03:22:20.565584+0000","last_change":"2024-09-12T03:22:19.431671+0000","last_active":"2024-09-12T03:22:20.565584+0000","last_peered":"2024-09-12T03:22:20.565584+0000","last_clean":"2024-09-12T03:22:20.565584+0000","last_became_active":"2024-09-12T03:22:19.431316+0000","last_became_peered":"2024-09-12T03:22:19.431316+0000","last_unstale":"2024-09-12T03:22:20.565584+0000","last_undegraded":"2024-09-12T03:22:20.565584+0000","last_fullsized":"2024-09-12T03:22:20.565584+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_clean_scrub_stamp":"2024-09-12T03:22:18.392443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:11:09.928417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.20","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T03:22:20.566481+0000","last_change":"2024-09-12T03:22:19.420313+0000","last_active":"2024-09-12T03:22:20.566481+0000","last_peered":"2024-09-12T03:22:20.566481+0000","last_clean":"2024-09-12T03:22:20.566481+0000","last_became_active":"2024-09-12T03:22:19.419472+0000","last_became_peered":"2024-09-12T03:22:19.419472+0000","last_unstale":"2024-09-12T03:22:20.566481+0000","last_undegraded":"2024-09-12T03:22:20.566481+0000","last_fullsized":"2024-09-12T03:22:20.566481+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_clean_scrub_stamp":"2024-09-12T03:22:18.392443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:08:13.228080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.1e","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T03:22:20.430835+0000","last_change":"2024-09-12T03:22:19.437533+0000","last_active":"2024-09-12T03:22:20.430835+0000","last_peered":"2024-09-12T03:22:20.430835+0000","last_clean":"2024-09-12T03:22:20.430835+0000","last_became_active":"2024-09-12T03:22:19.436643+0000","last_became_peered":"2024-09-12T03:22:19.436643+0000","last_unstale":"2024-09-12T03:22:20.430835+0000","last_undegraded":"2024-09-12T03:22:20.430835+0000","last_fullsized":"2024-09-12T03:22:20.430835+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_clean_scrub_stamp":"2024-09-12T03:22:18.392443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:27:51.778796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1f","version":"125'4","reported_seq":18,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T03:22:20.430787+0000","last_change":"2024-09-12T03:22:18.415035+0000","last_active":"2024-09-12T03:22:20.430787+0000","last_peered":"2024-09-12T03:22:20.430787+0000","last_clean":"2024-09-12T03:22:20.430787+0000","last_became_active":"2024-09-12T03:22:18.414211+0000","last_became_peered":"2024-09-12T03:22:18.414211+0000","last_unstale":"2024-09-12T03:22:20.430787+0000","last_undegraded":"2024-09-12T03:22:20.430787+0000","last_fullsized":"2024-09-12T03:22:20.430787+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:17.387205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:17.387205+0000","last_clean_scrub_stamp":"2024-09-12T03:22:17.387205+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:19:14.731819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1f","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T03:22:20.565997+0000","last_change":"2024-09-12T03:22:19.427606+0000","last_active":"2024-09-12T03:22:20.565997+0000","last_peered":"2024-09-12T03:22:20.565997+0000","last_clean":"2024-09-12T03:22:20.565997+0000","last_became_active":"2024-09-12T03:22:19.427164+0000","last_became_peered":"2024-09-12T03:22:19.427164+0000","last_unstale":"2024-09-12T03:22:20.565997+0000","last_undegraded":"2024-09-12T03:22:20.565997+0000","last_fullsized":"2024-09-12T03:22:20.565997+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_clean_scrub_stamp":"2024-09-12T03:22:18.392443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:24:28.462100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1e","version":"125'1","reported_seq":15,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T03:22:20.565763+0000","last_change":"2024-09-12T03:22:18.412937+0000","last_active":"2024-09-12T03:22:20.565763+0000","last_peered":"2024-09-12T03:22:20.565763+0000","last_clean":"2024-09-12T03:22:20.565763+0000","last_became_active":"2024-09-12T03:22:18.412665+0000","last_became_peered":"2024-09-12T03:22:18.412665+0000","last_unstale":"2024-09-12T03:22:20.565763+0000","last_undegraded":"2024-09-12T03:22:20.565763+0000","last_fullsized":"2024-09-12T03:22:20.565763+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:17.387205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:17.387205+0000","last_clean_scrub_stamp":"2024-09-12T03:22:17.387205+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:26:03.137156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1c","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T03:22:20.427504+0000","last_change":"2024-09-12T03:22:19.420006+0000","last_active":"2024-09-12T03:22:20.427504+0000","last_peered":"2024-09-12T03:22:20.427504+0000","last_clean":"2024-09-12T03:22:20.427504+0000","last_became_active":"2024-09-12T03:22:19.419715+0000","last_became_peered":"2024-09-12T03:22:19.419715+0000","last_unstale":"2024-09-12T03:22:20.427504+0000","last_undegraded":"2024-09-12T03:22:20.427504+0000","last_fullsized":"2024-09-12T03:22:20.427504+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_clean_scrub_stamp":"2024-09-12T03:22:18.392443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:28:53.157392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1d","version":"125'2","reported_seq":16,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T03:22:20.411504+0000","last_change":"2024-09-12T03:22:18.411480+0000","last_active":"2024-09-12T03:22:20.411504+0000","last_peered":"2024-09-12T03:22:20.411504+0000","last_clean":"2024-09-12T03:22:20.411504+0000","last_became_active":"2024-09-12T03:22:18.411210+0000","last_became_peered":"2024-09-12T03:22:18.411210+0000","last_unstale":"2024-09-12T03:22:20.411504+0000","last_undegraded":"2024-09-12T03:22:20.411504+0000","last_fullsized":"2024-09-12T03:22:20.411504+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:17.387205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:17.387205+0000","last_clean_scrub_stamp":"2024-09-12T03:22:17.387205+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:27:55.581131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1d","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T03:22:20.430656+0000","last_change":"2024-09-12T03:22:19.437278+0000","last_active":"2024-09-12T03:22:20.430656+0000","last_peered":"2024-09-12T03:22:20.430656+0000","last_clean":"2024-09-12T03:22:20.430656+0000","last_became_active":"2024-09-12T03:22:19.436958+0000","last_became_peered":"2024-09-12T03:22:19.436958+0000","last_unstale":"2024-09-12T03:22:20.430656+0000","last_undegraded":"2024-09-12T03:22:20.430656+0000","last_fullsized":"2024-09-12T03:22:20.430656+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_clean_scrub_stamp":"2024-09-12T03:22:18.392443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:40:14.475613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1c","version":"125'2","reported_seq":16,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T03:22:20.411930+0000","last_change":"2024-09-12T03:22:18.413232+0000","last_active":"2024-09-12T03:22:20.411930+0000","last_peered":"2024-09-12T03:22:20.411930+0000","last_clean":"2024-09-12T03:22:20.411930+0000","last_became_active":"2024-09-12T03:22:18.412774+0000","last_became_peered":"2024-09-12T03:22:18.412774+0000","last_unstale":"2024-09-12T03:22:20.411930+0000","last_undegraded":"2024-09-12T03:22:20.411930+0000","last_fullsized":"2024-09-12T03:22:20.411930+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:17.387205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:17.387205+0000","last_clean_scrub_stamp":"2024-09-12T03:22:17.387205+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:06:23.253785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1a","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T03:22:20.565737+0000","last_change":"2024-09-12T03:22:19.429572+0000","last_active":"2024-09-12T03:22:20.565737+0000","last_peered":"2024-09-12T03:22:20.565737+0000","last_clean":"2024-09-12T03:22:20.565737+0000","last_became_active":"2024-09-12T03:22:19.428433+0000","last_became_peered":"2024-09-12T03:22:19.428433+0000","last_unstale":"2024-09-12T03:22:20.565737+0000","last_undegraded":"2024-09-12T03:22:20.565737+0000","last_fullsized":"2024-09-12T03:22:20.565737+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_clean_scrub_stamp":"2024-09-12T03:22:18.392443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:19:15.737056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1b","version":"125'1","reported_seq":15,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T03:22:20.566532+0000","last_change":"2024-09-12T03:22:18.417349+0000","last_active":"2024-09-12T03:22:20.566532+0000","last_peered":"2024-09-12T03:22:20.566532+0000","last_clean":"2024-09-12T03:22:20.566532+0000","last_became_active":"2024-09-12T03:22:18.416955+0000","last_became_peered":"2024-09-12T03:22:18.416955+0000","last_unstale":"2024-09-12T03:22:20.566532+0000","last_undegraded":"2024-09-12T03:22:20.566532+0000","last_fullsized":"2024-09-12T03:22:20.566532+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:17.387205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:17.387205+0000","last_clean_scrub_stamp":"2024-09-12T03:22:17.387205+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:29:56.097726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.1b","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T03:22:20.411472+0000","last_change":"2024-09-12T03:22:19.429046+0000","last_active":"2024-09-12T03:22:20.411472+0000","last_peered":"2024-09-12T03:22:20.411472+0000","last_clean":"2024-09-12T03:22:20.411472+0000","last_became_active":"2024-09-12T03:22:19.428763+0000","last_became_peered":"2024-09-12T03:22:19.428763+0000","last_unstale":"2024-09-12T03:22:20.411472+0000","last_undegraded":"2024-09-12T03:22:20.411472+0000","last_fullsized":"2024-09-12T03:22:20.411472+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_clean_scrub_stamp":"2024-09-12T03:22:18.392443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:39:00.233271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.1a","version":"125'1","reported_seq":15,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T03:22:20.411280+0000","last_change":"2024-09-12T03:22:18.411867+0000","last_active":"2024-09-12T03:22:20.411280+0000","last_peered":"2024-09-12T03:22:20.411280+0000","last_clean":"2024-09-12T03:22:20.411280+0000","last_became_active":"2024-09-12T03:22:18.411526+0000","last_became_peered":"2024-09-12T03:22:18.411526+0000","last_unstale":"2024-09-12T03:22:20.411280+0000","last_undegraded":"2024-09-12T03:22:20.411280+0000","last_fullsized":"2024-09-12T03:22:20.411280+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:17.387205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:17.387205+0000","last_clean_scrub_stamp":"2024-09-12T03:22:17.387205+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:57:50.843023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T03:22:20.565525+0000","last_change":"2024-09-12T03:22:19.431554+0000","last_active":"2024-09-12T03:22:20.565525+0000","last_peered":"2024-09-12T03:22:20.565525+0000","last_clean":"2024-09-12T03:22:20.565525+0000","last_became_active":"2024-09-12T03:22:19.431259+0000","last_became_peered":"2024-09-12T03:22:19.431259+0000","last_unstale":"2024-09-12T03:22:20.565525+0000","last_undegraded":"2024-09-12T03:22:20.565525+0000","last_fullsized":"2024-09-12T03:22:20.565525+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_clean_scrub_stamp":"2024-09-12T03:22:18.392443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:30:36.215990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":14,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T03:22:20.566342+0000","last_change":"2024-09-12T03:22:18.416398+0000","last_active":"2024-09-12T03:22:20.566342+0000","last_peered":"2024-09-12T03:22:20.566342+0000","last_clean":"2024-09-12T03:22:20.566342+0000","last_became_active":"2024-09-12T03:22:18.416123+0000","last_became_peered":"2024-09-12T03:22:18.416123+0000","last_unstale":"2024-09-12T03:22:20.566342+0000","last_undegraded":"2024-09-12T03:22:20.566342+0000","last_fullsized":"2024-09-12T03:22:20.566342+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:17.387205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:17.387205+0000","last_clean_scrub_stamp":"2024-09-12T03:22:17.387205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:36:33.858935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T03:22:20.411901+0000","last_change":"2024-09-12T03:22:19.418041+0000","last_active":"2024-09-12T03:22:20.411901+0000","last_peered":"2024-09-12T03:22:20.411901+0000","last_clean":"2024-09-12T03:22:20.411901+0000","last_became_active":"2024-09-12T03:22:19.417688+0000","last_became_peered":"2024-09-12T03:22:19.417688+0000","last_unstale":"2024-09-12T03:22:20.411901+0000","last_undegraded":"2024-09-12T03:22:20.411901+0000","last_fullsized":"2024-09-12T03:22:20.411901+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_clean_scrub_stamp":"2024-09-12T03:22:18.392443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:04:34.555718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":14,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T03:22:20.411091+0000","last_change":"2024-09-12T03:22:18.409718+0000","last_active":"2024-09-12T03:22:20.411091+0000","last_peered":"2024-09-12T03:22:20.411091+0000","last_clean":"2024-09-12T03:22:20.411091+0000","last_became_active":"2024-09-12T03:22:18.409325+0000","last_became_peered":"2024-09-12T03:22:18.409325+0000","last_unstale":"2024-09-12T03:22:20.411091+0000","last_undegraded":"2024-09-12T03:22:20.411091+0000","last_fullsized":"2024-09-12T03:22:20.411091+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:17.387205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:17.387205+0000","last_clean_scrub_stamp":"2024-09-12T03:22:17.387205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:43:02.304826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T03:22:20.411628+0000","last_change":"2024-09-12T03:22:19.425698+0000","last_active":"2024-09-12T03:22:20.411628+0000","last_peered":"2024-09-12T03:22:20.411628+0000","last_clean":"2024-09-12T03:22:20.411628+0000","last_became_active":"2024-09-12T03:22:19.425355+0000","last_became_peered":"2024-09-12T03:22:19.425355+0000","last_unstale":"2024-09-12T03:22:20.411628+0000","last_undegraded":"2024-09-12T03:22:20.411628+0000","last_fullsized":"2024-09-12T03:22:20.411628+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_clean_scrub_stamp":"2024-09-12T03:22:18.392443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:05:38.055036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":14,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T03:22:20.565973+0000","last_change":"2024-09-12T03:22:18.412282+0000","last_active":"2024-09-12T03:22:20.565973+0000","last_peered":"2024-09-12T03:22:20.565973+0000","last_clean":"2024-09-12T03:22:20.565973+0000","last_became_active":"2024-09-12T03:22:18.411973+0000","last_became_peered":"2024-09-12T03:22:18.411973+0000","last_unstale":"2024-09-12T03:22:20.565973+0000","last_undegraded":"2024-09-12T03:22:20.565973+0000","last_fullsized":"2024-09-12T03:22:20.565973+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:17.387205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:17.387205+0000","last_clean_scrub_stamp":"2024-09-12T03:22:17.387205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:00:44.000131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T03:22:20.430800+0000","last_change":"2024-09-12T03:22:19.432103+0000","last_active":"2024-09-12T03:22:20.430800+0000","last_peered":"2024-09-12T03:22:20.430800+0000","last_clean":"2024-09-12T03:22:20.430800+0000","last_became_active":"2024-09-12T03:22:19.431762+0000","last_became_peered":"2024-09-12T03:22:19.431762+0000","last_unstale":"2024-09-12T03:22:20.430800+0000","last_undegraded":"2024-09-12T03:22:20.430800+0000","last_fullsized":"2024-09-12T03:22:20.430800+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_clean_scrub_stamp":"2024-09-12T03:22:18.392443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:20:01.511235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":14,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T03:22:20.430964+0000","last_change":"2024-09-12T03:22:18.411651+0000","last_active":"2024-09-12T03:22:20.430964+0000","last_peered":"2024-09-12T03:22:20.430964+0000","last_clean":"2024-09-12T03:22:20.430964+0000","last_became_active":"2024-09-12T03:22:18.411347+0000","last_became_peered":"2024-09-12T03:22:18.411347+0000","last_unstale":"2024-09-12T03:22:20.430964+0000","last_undegraded":"2024-09-12T03:22:20.430964+0000","last_fullsized":"2024-09-12T03:22:20.430964+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:17.387205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:17.387205+0000","last_clean_scrub_stamp":"2024-09-12T03:22:17.387205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:27:39.911375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.15","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T03:22:20.566013+0000","last_change":"2024-09-12T03:22:19.427469+0000","last_active":"2024-09-12T03:22:20.566013+0000","last_peered":"2024-09-12T03:22:20.566013+0000","last_clean":"2024-09-12T03:22:20.566013+0000","last_became_active":"2024-09-12T03:22:19.427194+0000","last_became_peered":"2024-09-12T03:22:19.427194+0000","last_unstale":"2024-09-12T03:22:20.566013+0000","last_undegraded":"2024-09-12T03:22:20.566013+0000","last_fullsized":"2024-09-12T03:22:20.566013+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_clean_scrub_stamp":"2024-09-12T03:22:18.392443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:41:58.275814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.14","version":"125'6","reported_seq":20,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T03:22:20.410414+0000","last_change":"2024-09-12T03:22:18.411529+0000","last_active":"2024-09-12T03:22:20.410414+0000","last_peered":"2024-09-12T03:22:20.410414+0000","last_clean":"2024-09-12T03:22:20.410414+0000","last_became_active":"2024-09-12T03:22:18.411203+0000","last_became_peered":"2024-09-12T03:22:18.411203+0000","last_unstale":"2024-09-12T03:22:20.410414+0000","last_undegraded":"2024-09-12T03:22:20.410414+0000","last_fullsized":"2024-09-12T03:22:20.410414+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:17.387205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:17.387205+0000","last_clean_scrub_stamp":"2024-09-12T03:22:17.387205+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:02:01.032597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T03:22:20.566388+0000","last_change":"2024-09-12T03:22:19.427359+0000","last_active":"2024-09-12T03:22:20.566388+0000","last_peered":"2024-09-12T03:22:20.566388+0000","last_clean":"2024-09-12T03:22:20.566388+0000","last_became_active":"2024-09-12T03:22:19.426927+0000","last_became_peered":"2024-09-12T03:22:19.426927+0000","last_unstale":"2024-09-12T03:22:20.566388+0000","last_undegraded":"2024-09-12T03:22:20.566388+0000","last_fullsized":"2024-09-12T03:22:20.566388+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_clean_scrub_stamp":"2024-09-12T03:22:18.392443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:06:24.259022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.13","version":"125'2","reported_seq":16,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T03:22:20.566660+0000","last_change":"2024-09-12T03:22:18.412269+0000","last_active":"2024-09-12T03:22:20.566660+0000","last_peered":"2024-09-12T03:22:20.566660+0000","last_clean":"2024-09-12T03:22:20.566660+0000","last_became_active":"2024-09-12T03:22:18.411932+0000","last_became_peered":"2024-09-12T03:22:18.411932+0000","last_unstale":"2024-09-12T03:22:20.566660+0000","last_undegraded":"2024-09-12T03:22:20.566660+0000","last_fullsized":"2024-09-12T03:22:20.566660+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:17.387205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:17.387205+0000","last_clean_scrub_stamp":"2024-09-12T03:22:17.387205+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:56:21.746135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.13","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T03:22:20.565906+0000","last_change":"2024-09-12T03:22:19.432506+0000","last_active":"2024-09-12T03:22:20.565906+0000","last_peered":"2024-09-12T03:22:20.565906+0000","last_clean":"2024-09-12T03:22:20.565906+0000","last_became_active":"2024-09-12T03:22:19.432203+0000","last_became_peered":"2024-09-12T03:22:19.432203+0000","last_unstale":"2024-09-12T03:22:20.565906+0000","last_undegraded":"2024-09-12T03:22:20.565906+0000","last_fullsized":"2024-09-12T03:22:20.565906+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_clean_scrub_stamp":"2024-09-12T03:22:18.392443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:38:12.159504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":14,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T03:22:20.566470+0000","last_change":"2024-09-12T03:22:18.410561+0000","last_active":"2024-09-12T03:22:20.566470+0000","last_peered":"2024-09-12T03:22:20.566470+0000","last_clean":"2024-09-12T03:22:20.566470+0000","last_became_active":"2024-09-12T03:22:18.410222+0000","last_became_peered":"2024-09-12T03:22:18.410222+0000","last_unstale":"2024-09-12T03:22:20.566470+0000","last_undegraded":"2024-09-12T03:22:20.566470+0000","last_fullsized":"2024-09-12T03:22:20.566470+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:17.387205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:17.387205+0000","last_clean_scrub_stamp":"2024-09-12T03:22:17.387205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:55:25.079169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.10","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T03:22:20.411085+0000","last_change":"2024-09-12T03:22:19.432651+0000","last_active":"2024-09-12T03:22:20.411085+0000","last_peered":"2024-09-12T03:22:20.411085+0000","last_clean":"2024-09-12T03:22:20.411085+0000","last_became_active":"2024-09-12T03:22:19.431932+0000","last_became_peered":"2024-09-12T03:22:19.431932+0000","last_unstale":"2024-09-12T03:22:20.411085+0000","last_undegraded":"2024-09-12T03:22:20.411085+0000","last_fullsized":"2024-09-12T03:22:20.411085+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_clean_scrub_stamp":"2024-09-12T03:22:18.392443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:24:46.851580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":14,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T03:22:20.427301+0000","last_change":"2024-09-12T03:22:18.412405+0000","last_active":"2024-09-12T03:22:20.427301+0000","last_peered":"2024-09-12T03:22:20.427301+0000","last_clean":"2024-09-12T03:22:20.427301+0000","last_became_active":"2024-09-12T03:22:18.412125+0000","last_became_peered":"2024-09-12T03:22:18.412125+0000","last_unstale":"2024-09-12T03:22:20.427301+0000","last_undegraded":"2024-09-12T03:22:20.427301+0000","last_fullsized":"2024-09-12T03:22:20.427301+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:17.387205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:17.387205+0000","last_clean_scrub_stamp":"2024-09-12T03:22:17.387205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:57:17.314093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T03:22:20.566132+0000","last_change":"2024-09-12T03:22:19.428116+0000","last_active":"2024-09-12T03:22:20.566132+0000","last_peered":"2024-09-12T03:22:20.566132+0000","last_clean":"2024-09-12T03:22:20.566132+0000","last_became_active":"2024-09-12T03:22:19.427821+0000","last_became_peered":"2024-09-12T03:22:19.427821+0000","last_unstale":"2024-09-12T03:22:20.566132+0000","last_undegraded":"2024-09-12T03:22:20.566132+0000","last_fullsized":"2024-09-12T03:22:20.566132+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_clean_scrub_stamp":"2024-09-12T03:22:18.392443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:27:56.586369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.10","version":"125'8","reported_seq":22,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T03:22:20.566537+0000","last_change":"2024-09-12T03:22:18.409429+0000","last_active":"2024-09-12T03:22:20.566537+0000","last_peered":"2024-09-12T03:22:20.566537+0000","last_clean":"2024-09-12T03:22:20.566537+0000","last_became_active":"2024-09-12T03:22:18.408740+0000","last_became_peered":"2024-09-12T03:22:18.408740+0000","last_unstale":"2024-09-12T03:22:20.566537+0000","last_undegraded":"2024-09-12T03:22:20.566537+0000","last_fullsized":"2024-09-12T03:22:20.566537+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:17.387205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:17.387205+0000","last_clean_scrub_stamp":"2024-09-12T03:22:17.387205+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:28:19.216947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.e","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T03:22:20.430972+0000","last_change":"2024-09-12T03:22:19.436685+0000","last_active":"2024-09-12T03:22:20.430972+0000","last_peered":"2024-09-12T03:22:20.430972+0000","last_clean":"2024-09-12T03:22:20.430972+0000","last_became_active":"2024-09-12T03:22:19.436237+0000","last_became_peered":"2024-09-12T03:22:19.436237+0000","last_unstale":"2024-09-12T03:22:20.430972+0000","last_undegraded":"2024-09-12T03:22:20.430972+0000","last_fullsized":"2024-09-12T03:22:20.430972+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_clean_scrub_stamp":"2024-09-12T03:22:18.392443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:02:02.037835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.f","version":"125'1","reported_seq":15,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T03:22:20.411242+0000","last_change":"2024-09-12T03:22:18.406274+0000","last_active":"2024-09-12T03:22:20.411242+0000","last_peered":"2024-09-12T03:22:20.411242+0000","last_clean":"2024-09-12T03:22:20.411242+0000","last_became_active":"2024-09-12T03:22:18.405977+0000","last_became_peered":"2024-09-12T03:22:18.405977+0000","last_unstale":"2024-09-12T03:22:20.411242+0000","last_undegraded":"2024-09-12T03:22:20.411242+0000","last_fullsized":"2024-09-12T03:22:20.411242+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:17.387205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:17.387205+0000","last_clean_scrub_stamp":"2024-09-12T03:22:17.387205+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:07:24.978733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.f","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T03:22:20.566160+0000","last_change":"2024-09-12T03:22:19.425138+0000","last_active":"2024-09-12T03:22:20.566160+0000","last_peered":"2024-09-12T03:22:20.566160+0000","last_clean":"2024-09-12T03:22:20.566160+0000","last_became_active":"2024-09-12T03:22:19.424853+0000","last_became_peered":"2024-09-12T03:22:19.424853+0000","last_unstale":"2024-09-12T03:22:20.566160+0000","last_undegraded":"2024-09-12T03:22:20.566160+0000","last_fullsized":"2024-09-12T03:22:20.566160+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_clean_scrub_stamp":"2024-09-12T03:22:18.392443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:43:16.888355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.e","version":"125'2","reported_seq":16,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T03:22:20.565867+0000","last_change":"2024-09-12T03:22:18.415917+0000","last_active":"2024-09-12T03:22:20.565867+0000","last_peered":"2024-09-12T03:22:20.565867+0000","last_clean":"2024-09-12T03:22:20.565867+0000","last_became_active":"2024-09-12T03:22:18.415574+0000","last_became_peered":"2024-09-12T03:22:18.415574+0000","last_unstale":"2024-09-12T03:22:20.565867+0000","last_undegraded":"2024-09-12T03:22:20.565867+0000","last_fullsized":"2024-09-12T03:22:20.565867+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:17.387205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:17.387205+0000","last_clean_scrub_stamp":"2024-09-12T03:22:17.387205+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:53:00.789204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T03:22:20.566638+0000","last_change":"2024-09-12T03:22:19.429196+0000","last_active":"2024-09-12T03:22:20.566638+0000","last_peered":"2024-09-12T03:22:20.566638+0000","last_clean":"2024-09-12T03:22:20.566638+0000","last_became_active":"2024-09-12T03:22:19.428379+0000","last_became_peered":"2024-09-12T03:22:19.428379+0000","last_unstale":"2024-09-12T03:22:20.566638+0000","last_undegraded":"2024-09-12T03:22:20.566638+0000","last_fullsized":"2024-09-12T03:22:20.566638+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_clean_scrub_stamp":"2024-09-12T03:22:18.392443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:26:04.142393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.d","version":"125'1","reported_seq":15,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T03:22:20.566123+0000","last_change":"2024-09-12T03:22:18.417315+0000","last_active":"2024-09-12T03:22:20.566123+0000","last_peered":"2024-09-12T03:22:20.566123+0000","last_clean":"2024-09-12T03:22:20.566123+0000","last_became_active":"2024-09-12T03:22:18.416924+0000","last_became_peered":"2024-09-12T03:22:18.416924+0000","last_unstale":"2024-09-12T03:22:20.566123+0000","last_undegraded":"2024-09-12T03:22:20.566123+0000","last_fullsized":"2024-09-12T03:22:20.566123+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:17.387205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:17.387205+0000","last_clean_scrub_stamp":"2024-09-12T03:22:17.387205+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:36:28.656742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.d","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T03:22:20.427479+0000","last_change":"2024-09-12T03:22:19.436256+0000","last_active":"2024-09-12T03:22:20.427479+0000","last_peered":"2024-09-12T03:22:20.427479+0000","last_clean":"2024-09-12T03:22:20.427479+0000","last_became_active":"2024-09-12T03:22:19.435970+0000","last_became_peered":"2024-09-12T03:22:19.435970+0000","last_unstale":"2024-09-12T03:22:20.427479+0000","last_undegraded":"2024-09-12T03:22:20.427479+0000","last_fullsized":"2024-09-12T03:22:20.427479+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_clean_scrub_stamp":"2024-09-12T03:22:18.392443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:56:57.586482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":14,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T03:22:20.566249+0000","last_change":"2024-09-12T03:22:18.414031+0000","last_active":"2024-09-12T03:22:20.566249+0000","last_peered":"2024-09-12T03:22:20.566249+0000","last_clean":"2024-09-12T03:22:20.566249+0000","last_became_active":"2024-09-12T03:22:18.413712+0000","last_became_peered":"2024-09-12T03:22:18.413712+0000","last_unstale":"2024-09-12T03:22:20.566249+0000","last_undegraded":"2024-09-12T03:22:20.566249+0000","last_fullsized":"2024-09-12T03:22:20.566249+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:17.387205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:17.387205+0000","last_clean_scrub_stamp":"2024-09-12T03:22:17.387205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:47:01.633048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T03:22:20.412366+0000","last_change":"2024-09-12T03:22:19.427210+0000","last_active":"2024-09-12T03:22:20.412366+0000","last_peered":"2024-09-12T03:22:20.412366+0000","last_clean":"2024-09-12T03:22:20.412366+0000","last_became_active":"2024-09-12T03:22:19.425379+0000","last_became_peered":"2024-09-12T03:22:19.425379+0000","last_unstale":"2024-09-12T03:22:20.412366+0000","last_undegraded":"2024-09-12T03:22:20.412366+0000","last_fullsized":"2024-09-12T03:22:20.412366+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_clean_scrub_stamp":"2024-09-12T03:22:18.392443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:11:09.928417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.b","version":"125'2","reported_seq":16,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T03:22:20.410150+0000","last_change":"2024-09-12T03:22:18.407704+0000","last_active":"2024-09-12T03:22:20.410150+0000","last_peered":"2024-09-12T03:22:20.410150+0000","last_clean":"2024-09-12T03:22:20.410150+0000","last_became_active":"2024-09-12T03:22:18.407392+0000","last_became_peered":"2024-09-12T03:22:18.407392+0000","last_unstale":"2024-09-12T03:22:20.410150+0000","last_undegraded":"2024-09-12T03:22:20.410150+0000","last_fullsized":"2024-09-12T03:22:20.410150+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:17.387205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:17.387205+0000","last_clean_scrub_stamp":"2024-09-12T03:22:17.387205+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:27:50.773558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.b","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T03:22:20.410173+0000","last_change":"2024-09-12T03:22:19.425182+0000","last_active":"2024-09-12T03:22:20.410173+0000","last_peered":"2024-09-12T03:22:20.410173+0000","last_clean":"2024-09-12T03:22:20.410173+0000","last_became_active":"2024-09-12T03:22:19.423095+0000","last_became_peered":"2024-09-12T03:22:19.423095+0000","last_unstale":"2024-09-12T03:22:20.410173+0000","last_undegraded":"2024-09-12T03:22:20.410173+0000","last_fullsized":"2024-09-12T03:22:20.410173+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_clean_scrub_stamp":"2024-09-12T03:22:18.392443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:15:32.505422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":14,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T03:22:20.430357+0000","last_change":"2024-09-12T03:22:18.414506+0000","last_active":"2024-09-12T03:22:20.430357+0000","last_peered":"2024-09-12T03:22:20.430357+0000","last_clean":"2024-09-12T03:22:20.430357+0000","last_became_active":"2024-09-12T03:22:18.414138+0000","last_became_peered":"2024-09-12T03:22:18.414138+0000","last_unstale":"2024-09-12T03:22:20.430357+0000","last_undegraded":"2024-09-12T03:22:20.430357+0000","last_fullsized":"2024-09-12T03:22:20.430357+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:17.387205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:17.387205+0000","last_clean_scrub_stamp":"2024-09-12T03:22:17.387205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:07:57.019892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T03:22:20.411113+0000","last_change":"2024-09-12T03:22:19.421627+0000","last_active":"2024-09-12T03:22:20.411113+0000","last_peered":"2024-09-12T03:22:20.411113+0000","last_clean":"2024-09-12T03:22:20.411113+0000","last_became_active":"2024-09-12T03:22:19.421280+0000","last_became_peered":"2024-09-12T03:22:19.421280+0000","last_unstale":"2024-09-12T03:22:20.411113+0000","last_undegraded":"2024-09-12T03:22:20.411113+0000","last_fullsized":"2024-09-12T03:22:20.411113+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_clean_scrub_stamp":"2024-09-12T03:22:18.392443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:11:08.266994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":14,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T03:22:20.430191+0000","last_change":"2024-09-12T03:22:18.413128+0000","last_active":"2024-09-12T03:22:20.430191+0000","last_peered":"2024-09-12T03:22:20.430191+0000","last_clean":"2024-09-12T03:22:20.430191+0000","last_became_active":"2024-09-12T03:22:18.412815+0000","last_became_peered":"2024-09-12T03:22:18.412815+0000","last_unstale":"2024-09-12T03:22:20.430191+0000","last_undegraded":"2024-09-12T03:22:20.430191+0000","last_fullsized":"2024-09-12T03:22:20.430191+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:17.387205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:17.387205+0000","last_clean_scrub_stamp":"2024-09-12T03:22:17.387205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:30:35.210752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T03:22:20.410182+0000","last_change":"2024-09-12T03:22:19.420982+0000","last_active":"2024-09-12T03:22:20.410182+0000","last_peered":"2024-09-12T03:22:20.410182+0000","last_clean":"2024-09-12T03:22:20.410182+0000","last_became_active":"2024-09-12T03:22:19.420532+0000","last_became_peered":"2024-09-12T03:22:19.420532+0000","last_unstale":"2024-09-12T03:22:20.410182+0000","last_undegraded":"2024-09-12T03:22:20.410182+0000","last_fullsized":"2024-09-12T03:22:20.410182+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_clean_scrub_stamp":"2024-09-12T03:22:18.392443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:00:32.782156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":14,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T03:22:20.566036+0000","last_change":"2024-09-12T03:22:18.415874+0000","last_active":"2024-09-12T03:22:20.566036+0000","last_peered":"2024-09-12T03:22:20.566036+0000","last_clean":"2024-09-12T03:22:20.566036+0000","last_became_active":"2024-09-12T03:22:18.415465+0000","last_became_peered":"2024-09-12T03:22:18.415465+0000","last_unstale":"2024-09-12T03:22:20.566036+0000","last_undegraded":"2024-09-12T03:22:20.566036+0000","last_fullsized":"2024-09-12T03:22:20.566036+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:17.387205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:17.387205+0000","last_clean_scrub_stamp":"2024-09-12T03:22:17.387205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:38:48.375420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T03:22:20.411791+0000","last_change":"2024-09-12T03:22:19.418987+0000","last_active":"2024-09-12T03:22:20.411791+0000","last_peered":"2024-09-12T03:22:20.411791+0000","last_clean":"2024-09-12T03:22:20.411791+0000","last_became_active":"2024-09-12T03:22:19.418708+0000","last_became_peered":"2024-09-12T03:22:19.418708+0000","last_unstale":"2024-09-12T03:22:20.411791+0000","last_undegraded":"2024-09-12T03:22:20.411791+0000","last_fullsized":"2024-09-12T03:22:20.411791+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_clean_scrub_stamp":"2024-09-12T03:22:18.392443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:17:25.507271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.7","version":"125'2","reported_seq":16,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T03:22:20.430942+0000","last_change":"2024-09-12T03:22:18.415733+0000","last_active":"2024-09-12T03:22:20.430942+0000","last_peered":"2024-09-12T03:22:20.430942+0000","last_clean":"2024-09-12T03:22:20.430942+0000","last_became_active":"2024-09-12T03:22:18.415408+0000","last_became_peered":"2024-09-12T03:22:18.415408+0000","last_unstale":"2024-09-12T03:22:20.430942+0000","last_undegraded":"2024-09-12T03:22:20.430942+0000","last_fullsized":"2024-09-12T03:22:20.430942+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:17.387205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:17.387205+0000","last_clean_scrub_stamp":"2024-09-12T03:22:17.387205+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:50:23.097953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.7","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T03:22:20.566695+0000","last_change":"2024-09-12T03:22:19.427214+0000","last_active":"2024-09-12T03:22:20.566695+0000","last_peered":"2024-09-12T03:22:20.566695+0000","last_clean":"2024-09-12T03:22:20.566695+0000","last_became_active":"2024-09-12T03:22:19.426845+0000","last_became_peered":"2024-09-12T03:22:19.426845+0000","last_unstale":"2024-09-12T03:22:20.566695+0000","last_undegraded":"2024-09-12T03:22:20.566695+0000","last_fullsized":"2024-09-12T03:22:20.566695+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_clean_scrub_stamp":"2024-09-12T03:22:18.392443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:53:01.794441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.6","version":"126'1","reported_seq":15,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T03:22:20.429416+0000","last_change":"2024-09-12T03:22:18.414645+0000","last_active":"2024-09-12T03:22:20.429416+0000","last_peered":"2024-09-12T03:22:20.429416+0000","last_clean":"2024-09-12T03:22:20.429416+0000","last_became_active":"2024-09-12T03:22:18.414345+0000","last_became_peered":"2024-09-12T03:22:18.414345+0000","last_unstale":"2024-09-12T03:22:20.429416+0000","last_undegraded":"2024-09-12T03:22:20.429416+0000","last_fullsized":"2024-09-12T03:22:20.429416+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:17.387205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:17.387205+0000","last_clean_scrub_stamp":"2024-09-12T03:22:17.387205+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:56:18.878188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T03:22:20.566159+0000","last_change":"2024-09-12T03:22:19.425316+0000","last_active":"2024-09-12T03:22:20.566159+0000","last_peered":"2024-09-12T03:22:20.566159+0000","last_clean":"2024-09-12T03:22:20.566159+0000","last_became_active":"2024-09-12T03:22:19.425041+0000","last_became_peered":"2024-09-12T03:22:19.425041+0000","last_unstale":"2024-09-12T03:22:20.566159+0000","last_undegraded":"2024-09-12T03:22:20.566159+0000","last_fullsized":"2024-09-12T03:22:20.566159+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_clean_scrub_stamp":"2024-09-12T03:22:18.392443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:50:13.734588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":14,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T03:22:20.427702+0000","last_change":"2024-09-12T03:22:18.416225+0000","last_active":"2024-09-12T03:22:20.427702+0000","last_peered":"2024-09-12T03:22:20.427702+0000","last_clean":"2024-09-12T03:22:20.427702+0000","last_became_active":"2024-09-12T03:22:18.415966+0000","last_became_peered":"2024-09-12T03:22:18.415966+0000","last_unstale":"2024-09-12T03:22:20.427702+0000","last_undegraded":"2024-09-12T03:22:20.427702+0000","last_fullsized":"2024-09-12T03:22:20.427702+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:17.387205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:17.387205+0000","last_clean_scrub_stamp":"2024-09-12T03:22:17.387205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:14:59.090991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T03:22:20.427611+0000","last_change":"2024-09-12T03:22:19.419166+0000","last_active":"2024-09-12T03:22:20.427611+0000","last_peered":"2024-09-12T03:22:20.427611+0000","last_clean":"2024-09-12T03:22:20.427611+0000","last_became_active":"2024-09-12T03:22:19.418870+0000","last_became_peered":"2024-09-12T03:22:19.418870+0000","last_unstale":"2024-09-12T03:22:20.427611+0000","last_undegraded":"2024-09-12T03:22:20.427611+0000","last_fullsized":"2024-09-12T03:22:20.427611+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_clean_scrub_stamp":"2024-09-12T03:22:18.392443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:31:36.296210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":14,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T03:22:20.427518+0000","last_change":"2024-09-12T03:22:18.416919+0000","last_active":"2024-09-12T03:22:20.427518+0000","last_peered":"2024-09-12T03:22:20.427518+0000","last_clean":"2024-09-12T03:22:20.427518+0000","last_became_active":"2024-09-12T03:22:18.416646+0000","last_became_peered":"2024-09-12T03:22:18.416646+0000","last_unstale":"2024-09-12T03:22:20.427518+0000","last_undegraded":"2024-09-12T03:22:20.427518+0000","last_fullsized":"2024-09-12T03:22:20.427518+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:17.387205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:17.387205+0000","last_clean_scrub_stamp":"2024-09-12T03:22:17.387205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:24:45.846343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T03:22:20.565690+0000","last_change":"2024-09-12T03:22:19.422916+0000","last_active":"2024-09-12T03:22:20.565690+0000","last_peered":"2024-09-12T03:22:20.565690+0000","last_clean":"2024-09-12T03:22:20.565690+0000","last_became_active":"2024-09-12T03:22:19.422632+0000","last_became_peered":"2024-09-12T03:22:19.422632+0000","last_unstale":"2024-09-12T03:22:20.565690+0000","last_undegraded":"2024-09-12T03:22:20.565690+0000","last_fullsized":"2024-09-12T03:22:20.565690+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_clean_scrub_stamp":"2024-09-12T03:22:18.392443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:33:20.519934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":14,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T03:22:20.410250+0000","last_change":"2024-09-12T03:22:18.405214+0000","last_active":"2024-09-12T03:22:20.410250+0000","last_peered":"2024-09-12T03:22:20.410250+0000","last_clean":"2024-09-12T03:22:20.410250+0000","last_became_active":"2024-09-12T03:22:18.404954+0000","last_became_peered":"2024-09-12T03:22:18.404954+0000","last_unstale":"2024-09-12T03:22:20.410250+0000","last_undegraded":"2024-09-12T03:22:20.410250+0000","last_fullsized":"2024-09-12T03:22:20.410250+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:17.387205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:17.387205+0000","last_clean_scrub_stamp":"2024-09-12T03:22:17.387205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:36:42.346845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T03:22:20.411557+0000","last_change":"2024-09-12T03:22:19.429146+0000","last_active":"2024-09-12T03:22:20.411557+0000","last_peered":"2024-09-12T03:22:20.411557+0000","last_clean":"2024-09-12T03:22:20.411557+0000","last_became_active":"2024-09-12T03:22:19.428834+0000","last_became_peered":"2024-09-12T03:22:19.428834+0000","last_unstale":"2024-09-12T03:22:20.411557+0000","last_undegraded":"2024-09-12T03:22:20.411557+0000","last_fullsized":"2024-09-12T03:22:20.411557+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_clean_scrub_stamp":"2024-09-12T03:22:18.392443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:56:19.883426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":14,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T03:22:20.430353+0000","last_change":"2024-09-12T03:22:18.410879+0000","last_active":"2024-09-12T03:22:20.430353+0000","last_peered":"2024-09-12T03:22:20.430353+0000","last_clean":"2024-09-12T03:22:20.430353+0000","last_became_active":"2024-09-12T03:22:18.410557+0000","last_became_peered":"2024-09-12T03:22:18.410557+0000","last_unstale":"2024-09-12T03:22:20.430353+0000","last_undegraded":"2024-09-12T03:22:20.430353+0000","last_fullsized":"2024-09-12T03:22:20.430353+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:17.387205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:17.387205+0000","last_clean_scrub_stamp":"2024-09-12T03:22:17.387205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:11:08.923180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T03:22:20.430287+0000","last_change":"2024-09-12T03:22:19.427002+0000","last_active":"2024-09-12T03:22:20.430287+0000","last_peered":"2024-09-12T03:22:20.430287+0000","last_clean":"2024-09-12T03:22:20.430287+0000","last_became_active":"2024-09-12T03:22:19.426679+0000","last_became_peered":"2024-09-12T03:22:19.426679+0000","last_unstale":"2024-09-12T03:22:20.430287+0000","last_undegraded":"2024-09-12T03:22:20.430287+0000","last_fullsized":"2024-09-12T03:22:20.430287+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_clean_scrub_stamp":"2024-09-12T03:22:18.392443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:14:17.658382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":14,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T03:22:20.566091+0000","last_change":"2024-09-12T03:22:18.413673+0000","last_active":"2024-09-12T03:22:20.566091+0000","last_peered":"2024-09-12T03:22:20.566091+0000","last_clean":"2024-09-12T03:22:20.566091+0000","last_became_active":"2024-09-12T03:22:18.413272+0000","last_became_peered":"2024-09-12T03:22:18.413272+0000","last_unstale":"2024-09-12T03:22:20.566091+0000","last_undegraded":"2024-09-12T03:22:20.566091+0000","last_fullsized":"2024-09-12T03:22:20.566091+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:17.387205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:17.387205+0000","last_clean_scrub_stamp":"2024-09-12T03:22:17.387205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:41:13.809653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T03:22:20.411898+0000","last_change":"2024-09-12T03:22:19.424919+0000","last_active":"2024-09-12T03:22:20.411898+0000","last_peered":"2024-09-12T03:22:20.411898+0000","last_clean":"2024-09-12T03:22:20.411898+0000","last_became_active":"2024-09-12T03:22:19.424539+0000","last_became_peered":"2024-09-12T03:22:19.424539+0000","last_unstale":"2024-09-12T03:22:20.411898+0000","last_undegraded":"2024-09-12T03:22:20.411898+0000","last_fullsized":"2024-09-12T03:22:20.411898+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_clean_scrub_stamp":"2024-09-12T03:22:18.392443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:06:28.637876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":14,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T03:22:20.565482+0000","last_change":"2024-09-12T03:22:18.412627+0000","last_active":"2024-09-12T03:22:20.565482+0000","last_peered":"2024-09-12T03:22:20.565482+0000","last_clean":"2024-09-12T03:22:20.565482+0000","last_became_active":"2024-09-12T03:22:18.412316+0000","last_became_peered":"2024-09-12T03:22:18.412316+0000","last_unstale":"2024-09-12T03:22:20.565482+0000","last_undegraded":"2024-09-12T03:22:20.565482+0000","last_fullsized":"2024-09-12T03:22:20.565482+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:17.387205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:17.387205+0000","last_clean_scrub_stamp":"2024-09-12T03:22:17.387205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:28:05.310409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.14","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T03:22:20.410363+0000","last_change":"2024-09-12T03:22:19.420036+0000","last_active":"2024-09-12T03:22:20.410363+0000","last_peered":"2024-09-12T03:22:20.410363+0000","last_clean":"2024-09-12T03:22:20.410363+0000","last_became_active":"2024-09-12T03:22:19.419765+0000","last_became_peered":"2024-09-12T03:22:19.419765+0000","last_unstale":"2024-09-12T03:22:20.410363+0000","last_undegraded":"2024-09-12T03:22:20.410363+0000","last_fullsized":"2024-09-12T03:22:20.410363+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_clean_scrub_stamp":"2024-09-12T03:22:18.392443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:43:27.847547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.15","version":"125'3","reported_seq":17,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T03:22:20.566574+0000","last_change":"2024-09-12T03:22:18.412881+0000","last_active":"2024-09-12T03:22:20.566574+0000","last_peered":"2024-09-12T03:22:20.566574+0000","last_clean":"2024-09-12T03:22:20.566574+0000","last_became_active":"2024-09-12T03:22:18.412605+0000","last_became_peered":"2024-09-12T03:22:18.412605+0000","last_unstale":"2024-09-12T03:22:20.566574+0000","last_undegraded":"2024-09-12T03:22:20.566574+0000","last_fullsized":"2024-09-12T03:22:20.566574+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:17.387205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:17.387205+0000","last_clean_scrub_stamp":"2024-09-12T03:22:17.387205+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:44:50.205161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"70'128","reported_seq":402,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T03:22:20.430346+0000","last_change":"2024-09-12T03:16:21.883446+0000","last_active":"2024-09-12T03:22:20.430346+0000","last_peered":"2024-09-12T03:22:20.430346+0000","last_clean":"2024-09-12T03:22:20.430346+0000","last_became_active":"2024-09-12T03:16:21.882207+0000","last_became_peered":"2024-09-12T03:16:21.882207+0000","last_unstale":"2024-09-12T03:22:20.430346+0000","last_undegraded":"2024-09-12T03:22:20.430346+0000","last_fullsized":"2024-09-12T03:22:20.430346+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:20.316426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:20.316426+0000","last_clean_scrub_stamp":"2024-09-12T03:16:20.316426+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:12:15.424320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":21,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":20,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542218,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37476,"kb_used_data":2128,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334364,"statfs":{"total":96636764160,"available":96598388736,"internally_reserved":0,"allocated":2179072,"data_stored":1288601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542220,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36896,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334944,"statfs":{"total":96636764160,"available":96598982656,"internally_reserved":0,"allocated":1576960,"data_stored":691371,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542219,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36892,"kb_used_data":1544,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334948,"statfs":{"total":96636764160,"available":96598986752,"internally_reserved":0,"allocated":1581056,"data_stored":697603,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542218,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36896,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334944,"statfs":{"total":96636764160,"available":96598982656,"internally_reserved":0,"allocated":1576960,"data_stored":697581,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542219,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32728,"kb_used_data":1532,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339112,"statfs":{"total":96636764160,"available":96603250688,"internally_reserved":0,"allocated":1568768,"data_stored":691315,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542219,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32732,"kb_used_data":1544,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339108,"statfs":{"total":96636764160,"available":96603246592,"internally_reserved":0,"allocated":1581056,"data_stored":692907,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542220,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32728,"kb_used_data":1532,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339112,"statfs":{"total":96636764160,"available":96603250688,"internally_reserved":0,"allocated":1568768,"data_stored":691807,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542220,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33332,"kb_used_data":2136,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338508,"statfs":{"total":96636764160,"available":96602632192,"internally_reserved":0,"allocated":2187264,"data_stored":1289595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T03:22:22.405 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-12T03:22:22.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.544+0000 7f0a9f7e4640 1 Processor -- start 2024-09-12T03:22:22.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.544+0000 7f0a9f7e4640 1 -- start start 2024-09-12T03:22:22.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.545+0000 7f0a9f7e4640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f0a9815e5f0 0x7f0a98073250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:22.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.545+0000 7f0a9f7e4640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0a980d2240 con 0x7f0a98073790 2024-09-12T03:22:22.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.545+0000 7f0a9f7e4640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0a98163090 con 0x7f0a98162a30 2024-09-12T03:22:22.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.545+0000 7f0a9f7e4640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f0a98162e60 con 0x7f0a9815e5f0 2024-09-12T03:22:22.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.545+0000 7f0a9cd7f640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f0a98073790 0x7f0a98162320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:42806/0 (socket says 172.21.15.27:42806) 2024-09-12T03:22:22.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.545+0000 7f0a9cd7f640 1 -- 172.21.15.27:0/994553022 learned_addr learned my addr 172.21.15.27:0/994553022 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:22:22.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.545+0000 7f0a8ffff640 1 -- 172.21.15.27:0/994553022 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1242439177 0 0) 0x7f0a980d2240 con 0x7f0a98073790 2024-09-12T03:22:22.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.545+0000 7f0a8ffff640 1 -- 172.21.15.27:0/994553022 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0a84003520 con 0x7f0a98073790 2024-09-12T03:22:22.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.545+0000 7f0a9d580640 1 --2- 172.21.15.27:0/994553022 >> v2:172.21.15.45:3300/0 conn(0x7f0a9815e5f0 0x7f0a98073250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:22.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.546+0000 7f0a8ffff640 1 -- 172.21.15.27:0/994553022 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3855375298 0 0) 0x7f0a84003520 con 0x7f0a98073790 2024-09-12T03:22:22.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.546+0000 7f0a8ffff640 1 -- 172.21.15.27:0/994553022 >> v2:172.21.15.45:3300/0 conn(0x7f0a9815e5f0 msgr2=0x7f0a98073250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:22.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.546+0000 7f0a8ffff640 1 --2- 172.21.15.27:0/994553022 >> v2:172.21.15.45:3300/0 conn(0x7f0a9815e5f0 0x7f0a98073250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:22.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.546+0000 7f0a8ffff640 1 -- 172.21.15.27:0/994553022 >> v1:172.21.15.45:6789/0 conn(0x7f0a98162a30 legacy=0x7f0a9815e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:22.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.546+0000 7f0a8ffff640 1 -- 172.21.15.27:0/994553022 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a980d0470 con 0x7f0a98073790 2024-09-12T03:22:22.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.546+0000 7f0a8ffff640 1 -- 172.21.15.27:0/994553022 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 847755213 0 0) 0x7f0a88001670 con 0x7f0a98073790 2024-09-12T03:22:22.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.546+0000 7f0a9d580640 1 --2- 172.21.15.27:0/994553022 >> v2:172.21.15.45:3300/0 conn(0x7f0a9815e5f0 0x7f0a98073250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:22:22.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.546+0000 7f0a8ffff640 1 -- 172.21.15.27:0/994553022 >> v1:172.21.15.27:6789/0 conn(0x7f0a98073790 legacy=0x7f0a98162320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:22.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.546+0000 7f0a8ffff640 1 --2- 172.21.15.27:0/994553022 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0a840039b0 0x7f0a84003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:22.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.546+0000 7f0a8ffff640 1 --2- 172.21.15.27:0/994553022 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0a84004800 0x7f0a84004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:22.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.546+0000 7f0a8ffff640 1 --2- 172.21.15.27:0/994553022 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f0a840051c0 0x7f0a8400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:22.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.546+0000 7f0a8ffff640 1 -- 172.21.15.27:0/994553022 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f0a8400c0e0 con 0x7f0a84004800 2024-09-12T03:22:22.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.546+0000 7f0a8ffff640 1 -- 172.21.15.27:0/994553022 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f0a8400c290 con 0x7f0a840039b0 2024-09-12T03:22:22.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.546+0000 7f0a8ffff640 1 -- 172.21.15.27:0/994553022 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f0a8400c490 con 0x7f0a840051c0 2024-09-12T03:22:22.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.546+0000 7f0a9cd7f640 1 --2- 172.21.15.27:0/994553022 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0a84004800 0x7f0a84004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:22.551 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.546+0000 7f0a9d580640 1 --2- 172.21.15.27:0/994553022 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0a840039b0 0x7f0a84003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:22.551 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.546+0000 7f0a9dd81640 1 --2- 172.21.15.27:0/994553022 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f0a840051c0 0x7f0a8400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:22.551 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.546+0000 7f0a9cd7f640 1 -- 172.21.15.27:0/994553022 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f0a840051c0 msgr2=0x7f0a8400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:22.551 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.546+0000 7f0a9cd7f640 1 --2- 172.21.15.27:0/994553022 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f0a840051c0 0x7f0a8400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:22.551 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.546+0000 7f0a9cd7f640 1 -- 172.21.15.27:0/994553022 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0a840039b0 msgr2=0x7f0a84003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:22.551 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.546+0000 7f0a9cd7f640 1 --2- 172.21.15.27:0/994553022 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0a840039b0 0x7f0a84003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:22.551 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.546+0000 7f0a9cd7f640 1 -- 172.21.15.27:0/994553022 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a84003520 con 0x7f0a84004800 2024-09-12T03:22:22.551 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.546+0000 7f0a9dd81640 1 -- 172.21.15.27:0/994553022 reap_dead start 2024-09-12T03:22:22.551 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.547+0000 7f0a9cd7f640 1 --2- 172.21.15.27:0/994553022 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0a84004800 0x7f0a84004c80 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7f0a8800c240 tx=0x7f0a84005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1fbcc716c6a4321 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:22.551 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.547+0000 7f0a8ffff640 1 -- 172.21.15.27:0/994553022 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a88007cf0 con 0x7f0a84004800 2024-09-12T03:22:22.551 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.547+0000 7f0a8ffff640 1 -- 172.21.15.27:0/994553022 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a880058d0 con 0x7f0a84004800 2024-09-12T03:22:22.551 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.547+0000 7f0a8ffff640 1 -- 172.21.15.27:0/994553022 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a88005d20 con 0x7f0a84004800 2024-09-12T03:22:22.551 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.548+0000 7f0a9f7e4640 1 -- 172.21.15.27:0/994553022 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0a84004800 msgr2=0x7f0a84004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:22.552 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.548+0000 7f0a9f7e4640 1 --2- 172.21.15.27:0/994553022 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0a84004800 0x7f0a84004c80 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7f0a8800c240 tx=0x7f0a84005830 comp rx=0 tx=0).stop 2024-09-12T03:22:22.552 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.548+0000 7f0a9f7e4640 1 -- 172.21.15.27:0/994553022 shutdown_connections 2024-09-12T03:22:22.552 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.548+0000 7f0a9f7e4640 1 --2- 172.21.15.27:0/994553022 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0a84004800 0x7f0a84004c80 unknown :-1 s=CLOSED pgs=1245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:22.552 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.548+0000 7f0a9f7e4640 1 -- 172.21.15.27:0/994553022 >> 172.21.15.27:0/994553022 conn(0x7f0a9806f300 msgr2=0x7f0a9806f710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:22.552 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.548+0000 7f0a9f7e4640 1 -- 172.21.15.27:0/994553022 shutdown_connections 2024-09-12T03:22:22.552 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.548+0000 7f0a9f7e4640 1 -- 172.21.15.27:0/994553022 wait complete. 2024-09-12T03:22:22.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.549+0000 7f0a9f7e4640 1 Processor -- start 2024-09-12T03:22:22.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.549+0000 7f0a9f7e4640 1 -- start start 2024-09-12T03:22:22.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.550+0000 7f0a9f7e4640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f0a84004800 0x7f0a981490e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:22.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.550+0000 7f0a9f7e4640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0a98154760 0x7f0a9815c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:22.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.550+0000 7f0a9f7e4640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0a98154be0 0x7f0a9815cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:22.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.550+0000 7f0a9f7e4640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f0a980d0da0 con 0x7f0a98154be0 2024-09-12T03:22:22.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.550+0000 7f0a9f7e4640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f0a98073be0 con 0x7f0a98154760 2024-09-12T03:22:22.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.550+0000 7f0a9f7e4640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f0a980d0a30 con 0x7f0a84004800 2024-09-12T03:22:22.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.550+0000 7f0a9dd81640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0a98154be0 0x7f0a9815cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:22.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.550+0000 7f0a9dd81640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0a98154be0 0x7f0a9815cba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49304/0 (socket says 172.21.15.27:49304) 2024-09-12T03:22:22.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.550+0000 7f0a9dd81640 1 -- 172.21.15.27:0/2779432546 learned_addr learned my addr 172.21.15.27:0/2779432546 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:22:22.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.550+0000 7f0a9d580640 1 --2- 172.21.15.27:0/2779432546 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f0a84004800 0x7f0a981490e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:22.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.550+0000 7f0a9cd7f640 1 --2- 172.21.15.27:0/2779432546 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0a98154760 0x7f0a9815c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:22.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.550+0000 7f0a9dd81640 1 -- 172.21.15.27:0/2779432546 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f0a84004800 msgr2=0x7f0a981490e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:22.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.550+0000 7f0a9dd81640 1 --2- 172.21.15.27:0/2779432546 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f0a84004800 0x7f0a981490e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:22.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.550+0000 7f0a9dd81640 1 -- 172.21.15.27:0/2779432546 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0a98154760 msgr2=0x7f0a9815c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:22.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.550+0000 7f0a9dd81640 1 --2- 172.21.15.27:0/2779432546 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0a98154760 0x7f0a9815c660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:22.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.550+0000 7f0a9dd81640 1 -- 172.21.15.27:0/2779432546 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a981561e0 con 0x7f0a98154be0 2024-09-12T03:22:22.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.550+0000 7f0a9dd81640 1 --2- 172.21.15.27:0/2779432546 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0a98154be0 0x7f0a9815cba0 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7f0a98072d30 tx=0x7f0a9000e820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:22.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.550+0000 7f0a8dffb640 1 -- 172.21.15.27:0/2779432546 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a90019070 con 0x7f0a98154be0 2024-09-12T03:22:22.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.550+0000 7f0a8dffb640 1 -- 172.21.15.27:0/2779432546 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a900050d0 con 0x7f0a98154be0 2024-09-12T03:22:22.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.550+0000 7f0a8dffb640 1 -- 172.21.15.27:0/2779432546 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a900035f0 con 0x7f0a98154be0 2024-09-12T03:22:22.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.550+0000 7f0a9f7e4640 1 -- 172.21.15.27:0/2779432546 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a98155140 con 0x7f0a98154be0 2024-09-12T03:22:22.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.551+0000 7f0a9f7e4640 1 -- 172.21.15.27:0/2779432546 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a98155700 con 0x7f0a98154be0 2024-09-12T03:22:22.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.552+0000 7f0a9f7e4640 1 -- 172.21.15.27:0/2779432546 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a98068e60 con 0x7f0a98154be0 2024-09-12T03:22:22.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.552+0000 7f0a8dffb640 1 -- 172.21.15.27:0/2779432546 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 21) ==== 91008+0+0 (secure 0 0 0) 0x7f0a90003790 con 0x7f0a98154be0 2024-09-12T03:22:22.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.552+0000 7f0a8dffb640 1 --2- 172.21.15.27:0/2779432546 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f0a7c06c400 0x7f0a7c06e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:22.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.553+0000 7f0a8dffb640 1 -- 172.21.15.27:0/2779432546 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 5765+0+0 (secure 0 0 0) 0x7f0a9008aa80 con 0x7f0a98154be0 2024-09-12T03:22:22.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.553+0000 7f0a9d580640 1 --2- 172.21.15.27:0/2779432546 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f0a7c06c400 0x7f0a7c06e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:22.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.553+0000 7f0a9d580640 1 --2- 172.21.15.27:0/2779432546 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f0a7c06c400 0x7f0a7c06e8c0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f0a94003c40 tx=0x7f0a94009820 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:22.558 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.557+0000 7f0a8dffb640 1 -- 172.21.15.27:0/2779432546 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f0a90058530 con 0x7f0a98154be0 2024-09-12T03:22:22.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.729+0000 7f0a9f7e4640 1 -- 172.21.15.27:0/2779432546 --> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f0a98160ec0 con 0x7f0a7c06c400 2024-09-12T03:22:22.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.735+0000 7f0a8dffb640 1 -- 172.21.15.27:0/2779432546 <== mgr.4113 v2:172.21.15.27:6836/1226920526 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252482 (secure 0 0 0) 0x7f0a98160ec0 con 0x7f0a7c06c400 2024-09-12T03:22:22.737 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:22:22.738 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2024-09-12T03:22:22.740 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.738+0000 7f0a9f7e4640 1 -- 172.21.15.27:0/2779432546 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f0a7c06c400 msgr2=0x7f0a7c06e8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:22.740 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.738+0000 7f0a9f7e4640 1 --2- 172.21.15.27:0/2779432546 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f0a7c06c400 0x7f0a7c06e8c0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f0a94003c40 tx=0x7f0a94009820 comp rx=0 tx=0).stop 2024-09-12T03:22:22.740 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.738+0000 7f0a9f7e4640 1 -- 172.21.15.27:0/2779432546 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0a98154be0 msgr2=0x7f0a9815cba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:22.740 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.738+0000 7f0a9f7e4640 1 --2- 172.21.15.27:0/2779432546 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0a98154be0 0x7f0a9815cba0 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7f0a98072d30 tx=0x7f0a9000e820 comp rx=0 tx=0).stop 2024-09-12T03:22:22.740 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.738+0000 7f0a9f7e4640 1 -- 172.21.15.27:0/2779432546 shutdown_connections 2024-09-12T03:22:22.740 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.738+0000 7f0a9f7e4640 1 --2- 172.21.15.27:0/2779432546 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f0a7c06c400 0x7f0a7c06e8c0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:22.740 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.738+0000 7f0a9f7e4640 1 --2- 172.21.15.27:0/2779432546 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0a98154be0 0x7f0a9815cba0 unknown :-1 s=CLOSED pgs=1246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:22.740 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.738+0000 7f0a9f7e4640 1 --2- 172.21.15.27:0/2779432546 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0a98154760 0x7f0a9815c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:22.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.738+0000 7f0a9f7e4640 1 --2- 172.21.15.27:0/2779432546 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f0a84004800 0x7f0a981490e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:22.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.738+0000 7f0a9f7e4640 1 -- 172.21.15.27:0/2779432546 >> 172.21.15.27:0/2779432546 conn(0x7f0a9806f300 msgr2=0x7f0a9815edb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:22.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.738+0000 7f0a9f7e4640 1 -- 172.21.15.27:0/2779432546 shutdown_connections 2024-09-12T03:22:22.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.738+0000 7f0a9f7e4640 1 -- 172.21.15.27:0/2779432546 wait complete. 2024-09-12T03:22:22.749 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":123,"stamp":"2024-09-12T03:22:21.121894+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":224,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":167,"ondisk_log_size":167,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":80,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":279680,"kb_used_data":13496,"kb_used_omap":12,"kb_used_meta":265971,"kb_avail":754695040,"statfs":{"total":773094113280,"available":772807720960,"internally_reserved":0,"allocated":13819904,"data_stored":6740780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":272354864},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":23,"apply_latency_ms":23,"commit_latency_ns":23000000,"apply_latency_ns":23000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000693"},"pg_stats":[{"pgid":"21.3f","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T03:22:20.427934+0000","last_change":"2024-09-12T03:22:19.425643+0000","last_active":"2024-09-12T03:22:20.427934+0000","last_peered":"2024-09-12T03:22:20.427934+0000","last_clean":"2024-09-12T03:22:20.427934+0000","last_became_active":"2024-09-12T03:22:19.425322+0000","last_became_peered":"2024-09-12T03:22:19.425322+0000","last_unstale":"2024-09-12T03:22:20.427934+0000","last_undegraded":"2024-09-12T03:22:20.427934+0000","last_fullsized":"2024-09-12T03:22:20.427934+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_clean_scrub_stamp":"2024-09-12T03:22:18.392443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:45:31.104107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.3e","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T03:22:20.565814+0000","last_change":"2024-09-12T03:22:19.430562+0000","last_active":"2024-09-12T03:22:20.565814+0000","last_peered":"2024-09-12T03:22:20.565814+0000","last_clean":"2024-09-12T03:22:20.565814+0000","last_became_active":"2024-09-12T03:22:19.430254+0000","last_became_peered":"2024-09-12T03:22:19.430254+0000","last_unstale":"2024-09-12T03:22:20.565814+0000","last_undegraded":"2024-09-12T03:22:20.565814+0000","last_fullsized":"2024-09-12T03:22:20.565814+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_clean_scrub_stamp":"2024-09-12T03:22:18.392443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:50:24.103190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.3d","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T03:22:20.566564+0000","last_change":"2024-09-12T03:22:19.439778+0000","last_active":"2024-09-12T03:22:20.566564+0000","last_peered":"2024-09-12T03:22:20.566564+0000","last_clean":"2024-09-12T03:22:20.566564+0000","last_became_active":"2024-09-12T03:22:19.439263+0000","last_became_peered":"2024-09-12T03:22:19.439263+0000","last_unstale":"2024-09-12T03:22:20.566564+0000","last_undegraded":"2024-09-12T03:22:20.566564+0000","last_fullsized":"2024-09-12T03:22:20.566564+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_clean_scrub_stamp":"2024-09-12T03:22:18.392443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:54:05.437535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.3c","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T03:22:20.411897+0000","last_change":"2024-09-12T03:22:19.432003+0000","last_active":"2024-09-12T03:22:20.411897+0000","last_peered":"2024-09-12T03:22:20.411897+0000","last_clean":"2024-09-12T03:22:20.411897+0000","last_became_active":"2024-09-12T03:22:19.431621+0000","last_became_peered":"2024-09-12T03:22:19.431621+0000","last_unstale":"2024-09-12T03:22:20.411897+0000","last_undegraded":"2024-09-12T03:22:20.411897+0000","last_fullsized":"2024-09-12T03:22:20.411897+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_clean_scrub_stamp":"2024-09-12T03:22:18.392443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:56:49.416621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.3b","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T03:22:20.411612+0000","last_change":"2024-09-12T03:22:19.433231+0000","last_active":"2024-09-12T03:22:20.411612+0000","last_peered":"2024-09-12T03:22:20.411612+0000","last_clean":"2024-09-12T03:22:20.411612+0000","last_became_active":"2024-09-12T03:22:19.432900+0000","last_became_peered":"2024-09-12T03:22:19.432900+0000","last_unstale":"2024-09-12T03:22:20.411612+0000","last_undegraded":"2024-09-12T03:22:20.411612+0000","last_fullsized":"2024-09-12T03:22:20.411612+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_clean_scrub_stamp":"2024-09-12T03:22:18.392443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:15:00.096228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.3a","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T03:22:20.427533+0000","last_change":"2024-09-12T03:22:19.436158+0000","last_active":"2024-09-12T03:22:20.427533+0000","last_peered":"2024-09-12T03:22:20.427533+0000","last_clean":"2024-09-12T03:22:20.427533+0000","last_became_active":"2024-09-12T03:22:19.435866+0000","last_became_peered":"2024-09-12T03:22:19.435866+0000","last_unstale":"2024-09-12T03:22:20.427533+0000","last_undegraded":"2024-09-12T03:22:20.427533+0000","last_fullsized":"2024-09-12T03:22:20.427533+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_clean_scrub_stamp":"2024-09-12T03:22:18.392443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:40:55.731312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.39","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T03:22:20.410136+0000","last_change":"2024-09-12T03:22:19.418842+0000","last_active":"2024-09-12T03:22:20.410136+0000","last_peered":"2024-09-12T03:22:20.410136+0000","last_clean":"2024-09-12T03:22:20.410136+0000","last_became_active":"2024-09-12T03:22:19.418467+0000","last_became_peered":"2024-09-12T03:22:19.418467+0000","last_unstale":"2024-09-12T03:22:20.410136+0000","last_undegraded":"2024-09-12T03:22:20.410136+0000","last_fullsized":"2024-09-12T03:22:20.410136+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_clean_scrub_stamp":"2024-09-12T03:22:18.392443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:38:46.337249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.38","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T03:22:20.427327+0000","last_change":"2024-09-12T03:22:19.413757+0000","last_active":"2024-09-12T03:22:20.427327+0000","last_peered":"2024-09-12T03:22:20.427327+0000","last_clean":"2024-09-12T03:22:20.427327+0000","last_became_active":"2024-09-12T03:22:19.413424+0000","last_became_peered":"2024-09-12T03:22:19.413424+0000","last_unstale":"2024-09-12T03:22:20.427327+0000","last_undegraded":"2024-09-12T03:22:20.427327+0000","last_fullsized":"2024-09-12T03:22:20.427327+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_clean_scrub_stamp":"2024-09-12T03:22:18.392443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:47:26.560867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.37","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T03:22:20.565846+0000","last_change":"2024-09-12T03:22:19.428309+0000","last_active":"2024-09-12T03:22:20.565846+0000","last_peered":"2024-09-12T03:22:20.565846+0000","last_clean":"2024-09-12T03:22:20.565846+0000","last_became_active":"2024-09-12T03:22:19.427896+0000","last_became_peered":"2024-09-12T03:22:19.427896+0000","last_unstale":"2024-09-12T03:22:20.565846+0000","last_undegraded":"2024-09-12T03:22:20.565846+0000","last_fullsized":"2024-09-12T03:22:20.565846+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_clean_scrub_stamp":"2024-09-12T03:22:18.392443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:27:40.916612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.36","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T03:22:20.430842+0000","last_change":"2024-09-12T03:22:19.434045+0000","last_active":"2024-09-12T03:22:20.430842+0000","last_peered":"2024-09-12T03:22:20.430842+0000","last_clean":"2024-09-12T03:22:20.430842+0000","last_became_active":"2024-09-12T03:22:19.431316+0000","last_became_peered":"2024-09-12T03:22:19.431316+0000","last_unstale":"2024-09-12T03:22:20.430842+0000","last_undegraded":"2024-09-12T03:22:20.430842+0000","last_fullsized":"2024-09-12T03:22:20.430842+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_clean_scrub_stamp":"2024-09-12T03:22:18.392443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:31:57.234454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.35","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T03:22:20.427449+0000","last_change":"2024-09-12T03:22:19.413798+0000","last_active":"2024-09-12T03:22:20.427449+0000","last_peered":"2024-09-12T03:22:20.427449+0000","last_clean":"2024-09-12T03:22:20.427449+0000","last_became_active":"2024-09-12T03:22:19.413485+0000","last_became_peered":"2024-09-12T03:22:19.413485+0000","last_unstale":"2024-09-12T03:22:20.427449+0000","last_undegraded":"2024-09-12T03:22:20.427449+0000","last_fullsized":"2024-09-12T03:22:20.427449+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_clean_scrub_stamp":"2024-09-12T03:22:18.392443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:50:41.758774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.34","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T03:22:20.430725+0000","last_change":"2024-09-12T03:22:19.435530+0000","last_active":"2024-09-12T03:22:20.430725+0000","last_peered":"2024-09-12T03:22:20.430725+0000","last_clean":"2024-09-12T03:22:20.430725+0000","last_became_active":"2024-09-12T03:22:19.434535+0000","last_became_peered":"2024-09-12T03:22:19.434535+0000","last_unstale":"2024-09-12T03:22:20.430725+0000","last_undegraded":"2024-09-12T03:22:20.430725+0000","last_fullsized":"2024-09-12T03:22:20.430725+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_clean_scrub_stamp":"2024-09-12T03:22:18.392443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:39:01.904225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.33","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T03:22:20.411817+0000","last_change":"2024-09-12T03:22:19.432235+0000","last_active":"2024-09-12T03:22:20.411817+0000","last_peered":"2024-09-12T03:22:20.411817+0000","last_clean":"2024-09-12T03:22:20.411817+0000","last_became_active":"2024-09-12T03:22:19.431932+0000","last_became_peered":"2024-09-12T03:22:19.431932+0000","last_unstale":"2024-09-12T03:22:20.411817+0000","last_undegraded":"2024-09-12T03:22:20.411817+0000","last_fullsized":"2024-09-12T03:22:20.411817+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_clean_scrub_stamp":"2024-09-12T03:22:18.392443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:10:59.293591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.32","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T03:22:20.412125+0000","last_change":"2024-09-12T03:22:19.427849+0000","last_active":"2024-09-12T03:22:20.412125+0000","last_peered":"2024-09-12T03:22:20.412125+0000","last_clean":"2024-09-12T03:22:20.412125+0000","last_became_active":"2024-09-12T03:22:19.427556+0000","last_became_peered":"2024-09-12T03:22:19.427556+0000","last_unstale":"2024-09-12T03:22:20.412125+0000","last_undegraded":"2024-09-12T03:22:20.412125+0000","last_fullsized":"2024-09-12T03:22:20.412125+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_clean_scrub_stamp":"2024-09-12T03:22:18.392443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:06:53.485657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.31","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T03:22:20.430265+0000","last_change":"2024-09-12T03:22:19.426135+0000","last_active":"2024-09-12T03:22:20.430265+0000","last_peered":"2024-09-12T03:22:20.430265+0000","last_clean":"2024-09-12T03:22:20.430265+0000","last_became_active":"2024-09-12T03:22:19.425841+0000","last_became_peered":"2024-09-12T03:22:19.425841+0000","last_unstale":"2024-09-12T03:22:20.430265+0000","last_undegraded":"2024-09-12T03:22:20.430265+0000","last_fullsized":"2024-09-12T03:22:20.430265+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_clean_scrub_stamp":"2024-09-12T03:22:18.392443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:34:26.552409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.30","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T03:22:20.566448+0000","last_change":"2024-09-12T03:22:19.430530+0000","last_active":"2024-09-12T03:22:20.566448+0000","last_peered":"2024-09-12T03:22:20.566448+0000","last_clean":"2024-09-12T03:22:20.566448+0000","last_became_active":"2024-09-12T03:22:19.430245+0000","last_became_peered":"2024-09-12T03:22:19.430245+0000","last_unstale":"2024-09-12T03:22:20.566448+0000","last_undegraded":"2024-09-12T03:22:20.566448+0000","last_fullsized":"2024-09-12T03:22:20.566448+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_clean_scrub_stamp":"2024-09-12T03:22:18.392443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:08:54.698701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.2f","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T03:22:20.430943+0000","last_change":"2024-09-12T03:22:19.436725+0000","last_active":"2024-09-12T03:22:20.430943+0000","last_peered":"2024-09-12T03:22:20.430943+0000","last_clean":"2024-09-12T03:22:20.430943+0000","last_became_active":"2024-09-12T03:22:19.431721+0000","last_became_peered":"2024-09-12T03:22:19.431721+0000","last_unstale":"2024-09-12T03:22:20.430943+0000","last_undegraded":"2024-09-12T03:22:20.430943+0000","last_fullsized":"2024-09-12T03:22:20.430943+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_clean_scrub_stamp":"2024-09-12T03:22:18.392443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:36:43.352083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.2e","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T03:22:20.566617+0000","last_change":"2024-09-12T03:22:19.439648+0000","last_active":"2024-09-12T03:22:20.566617+0000","last_peered":"2024-09-12T03:22:20.566617+0000","last_clean":"2024-09-12T03:22:20.566617+0000","last_became_active":"2024-09-12T03:22:19.439341+0000","last_became_peered":"2024-09-12T03:22:19.439341+0000","last_unstale":"2024-09-12T03:22:20.566617+0000","last_undegraded":"2024-09-12T03:22:20.566617+0000","last_fullsized":"2024-09-12T03:22:20.566617+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_clean_scrub_stamp":"2024-09-12T03:22:18.392443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:00:45.005368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.2d","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T03:22:20.427625+0000","last_change":"2024-09-12T03:22:19.430547+0000","last_active":"2024-09-12T03:22:20.427625+0000","last_peered":"2024-09-12T03:22:20.427625+0000","last_clean":"2024-09-12T03:22:20.427625+0000","last_became_active":"2024-09-12T03:22:19.430148+0000","last_became_peered":"2024-09-12T03:22:19.430148+0000","last_unstale":"2024-09-12T03:22:20.427625+0000","last_undegraded":"2024-09-12T03:22:20.427625+0000","last_fullsized":"2024-09-12T03:22:20.427625+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_clean_scrub_stamp":"2024-09-12T03:22:18.392443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:04:26.613191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2c","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T03:22:20.566114+0000","last_change":"2024-09-12T03:22:19.428110+0000","last_active":"2024-09-12T03:22:20.566114+0000","last_peered":"2024-09-12T03:22:20.566114+0000","last_clean":"2024-09-12T03:22:20.566114+0000","last_became_active":"2024-09-12T03:22:19.427817+0000","last_became_peered":"2024-09-12T03:22:19.427817+0000","last_unstale":"2024-09-12T03:22:20.566114+0000","last_undegraded":"2024-09-12T03:22:20.566114+0000","last_fullsized":"2024-09-12T03:22:20.566114+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_clean_scrub_stamp":"2024-09-12T03:22:18.392443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:21:14.749329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.2b","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T03:22:20.430651+0000","last_change":"2024-09-12T03:22:19.432107+0000","last_active":"2024-09-12T03:22:20.430651+0000","last_peered":"2024-09-12T03:22:20.430651+0000","last_clean":"2024-09-12T03:22:20.430651+0000","last_became_active":"2024-09-12T03:22:19.430805+0000","last_became_peered":"2024-09-12T03:22:19.430805+0000","last_unstale":"2024-09-12T03:22:20.430651+0000","last_undegraded":"2024-09-12T03:22:20.430651+0000","last_fullsized":"2024-09-12T03:22:20.430651+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_clean_scrub_stamp":"2024-09-12T03:22:18.392443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:02:41.802998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.2a","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T03:22:20.412208+0000","last_change":"2024-09-12T03:22:19.426007+0000","last_active":"2024-09-12T03:22:20.412208+0000","last_peered":"2024-09-12T03:22:20.412208+0000","last_clean":"2024-09-12T03:22:20.412208+0000","last_became_active":"2024-09-12T03:22:19.424984+0000","last_became_peered":"2024-09-12T03:22:19.424984+0000","last_unstale":"2024-09-12T03:22:20.412208+0000","last_undegraded":"2024-09-12T03:22:20.412208+0000","last_fullsized":"2024-09-12T03:22:20.412208+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_clean_scrub_stamp":"2024-09-12T03:22:18.392443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:07:58.025130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.29","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T03:22:20.411721+0000","last_change":"2024-09-12T03:22:19.422131+0000","last_active":"2024-09-12T03:22:20.411721+0000","last_peered":"2024-09-12T03:22:20.411721+0000","last_clean":"2024-09-12T03:22:20.411721+0000","last_became_active":"2024-09-12T03:22:19.421483+0000","last_became_peered":"2024-09-12T03:22:19.421483+0000","last_unstale":"2024-09-12T03:22:20.411721+0000","last_undegraded":"2024-09-12T03:22:20.411721+0000","last_fullsized":"2024-09-12T03:22:20.411721+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_clean_scrub_stamp":"2024-09-12T03:22:18.392443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:52:23.195824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.28","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T03:22:20.430244+0000","last_change":"2024-09-12T03:22:19.430260+0000","last_active":"2024-09-12T03:22:20.430244+0000","last_peered":"2024-09-12T03:22:20.430244+0000","last_clean":"2024-09-12T03:22:20.430244+0000","last_became_active":"2024-09-12T03:22:19.429972+0000","last_became_peered":"2024-09-12T03:22:19.429972+0000","last_unstale":"2024-09-12T03:22:20.430244+0000","last_undegraded":"2024-09-12T03:22:20.430244+0000","last_fullsized":"2024-09-12T03:22:20.430244+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_clean_scrub_stamp":"2024-09-12T03:22:18.392443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:31:58.401020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.27","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T03:22:20.566124+0000","last_change":"2024-09-12T03:22:19.435157+0000","last_active":"2024-09-12T03:22:20.566124+0000","last_peered":"2024-09-12T03:22:20.566124+0000","last_clean":"2024-09-12T03:22:20.566124+0000","last_became_active":"2024-09-12T03:22:19.434780+0000","last_became_peered":"2024-09-12T03:22:19.434780+0000","last_unstale":"2024-09-12T03:22:20.566124+0000","last_undegraded":"2024-09-12T03:22:20.566124+0000","last_fullsized":"2024-09-12T03:22:20.566124+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_clean_scrub_stamp":"2024-09-12T03:22:18.392443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:49:39.702721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.26","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T03:22:20.427668+0000","last_change":"2024-09-12T03:22:19.424459+0000","last_active":"2024-09-12T03:22:20.427668+0000","last_peered":"2024-09-12T03:22:20.427668+0000","last_clean":"2024-09-12T03:22:20.427668+0000","last_became_active":"2024-09-12T03:22:19.424175+0000","last_became_peered":"2024-09-12T03:22:19.424175+0000","last_unstale":"2024-09-12T03:22:20.427668+0000","last_undegraded":"2024-09-12T03:22:20.427668+0000","last_fullsized":"2024-09-12T03:22:20.427668+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_clean_scrub_stamp":"2024-09-12T03:22:18.392443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:26:03.322839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.25","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T03:22:20.565865+0000","last_change":"2024-09-12T03:22:19.429027+0000","last_active":"2024-09-12T03:22:20.565865+0000","last_peered":"2024-09-12T03:22:20.565865+0000","last_clean":"2024-09-12T03:22:20.565865+0000","last_became_active":"2024-09-12T03:22:19.428729+0000","last_became_peered":"2024-09-12T03:22:19.428729+0000","last_unstale":"2024-09-12T03:22:20.565865+0000","last_undegraded":"2024-09-12T03:22:20.565865+0000","last_fullsized":"2024-09-12T03:22:20.565865+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_clean_scrub_stamp":"2024-09-12T03:22:18.392443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:55:06.811462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.24","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T03:22:20.410334+0000","last_change":"2024-09-12T03:22:19.430768+0000","last_active":"2024-09-12T03:22:20.410334+0000","last_peered":"2024-09-12T03:22:20.410334+0000","last_clean":"2024-09-12T03:22:20.410334+0000","last_became_active":"2024-09-12T03:22:19.430490+0000","last_became_peered":"2024-09-12T03:22:19.430490+0000","last_unstale":"2024-09-12T03:22:20.410334+0000","last_undegraded":"2024-09-12T03:22:20.410334+0000","last_fullsized":"2024-09-12T03:22:20.410334+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_clean_scrub_stamp":"2024-09-12T03:22:18.392443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:40:56.406906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.23","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T03:22:20.427422+0000","last_change":"2024-09-12T03:22:19.435437+0000","last_active":"2024-09-12T03:22:20.427422+0000","last_peered":"2024-09-12T03:22:20.427422+0000","last_clean":"2024-09-12T03:22:20.427422+0000","last_became_active":"2024-09-12T03:22:19.435151+0000","last_became_peered":"2024-09-12T03:22:19.435151+0000","last_unstale":"2024-09-12T03:22:20.427422+0000","last_undegraded":"2024-09-12T03:22:20.427422+0000","last_fullsized":"2024-09-12T03:22:20.427422+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_clean_scrub_stamp":"2024-09-12T03:22:18.392443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:04:59.936109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.22","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T03:22:20.412289+0000","last_change":"2024-09-12T03:22:19.426401+0000","last_active":"2024-09-12T03:22:20.412289+0000","last_peered":"2024-09-12T03:22:20.412289+0000","last_clean":"2024-09-12T03:22:20.412289+0000","last_became_active":"2024-09-12T03:22:19.425770+0000","last_became_peered":"2024-09-12T03:22:19.425770+0000","last_unstale":"2024-09-12T03:22:20.412289+0000","last_undegraded":"2024-09-12T03:22:20.412289+0000","last_fullsized":"2024-09-12T03:22:20.412289+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_clean_scrub_stamp":"2024-09-12T03:22:18.392443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:30:36.215990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.21","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T03:22:20.565584+0000","last_change":"2024-09-12T03:22:19.431671+0000","last_active":"2024-09-12T03:22:20.565584+0000","last_peered":"2024-09-12T03:22:20.565584+0000","last_clean":"2024-09-12T03:22:20.565584+0000","last_became_active":"2024-09-12T03:22:19.431316+0000","last_became_peered":"2024-09-12T03:22:19.431316+0000","last_unstale":"2024-09-12T03:22:20.565584+0000","last_undegraded":"2024-09-12T03:22:20.565584+0000","last_fullsized":"2024-09-12T03:22:20.565584+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_clean_scrub_stamp":"2024-09-12T03:22:18.392443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:11:09.928417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.20","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T03:22:20.566481+0000","last_change":"2024-09-12T03:22:19.420313+0000","last_active":"2024-09-12T03:22:20.566481+0000","last_peered":"2024-09-12T03:22:20.566481+0000","last_clean":"2024-09-12T03:22:20.566481+0000","last_became_active":"2024-09-12T03:22:19.419472+0000","last_became_peered":"2024-09-12T03:22:19.419472+0000","last_unstale":"2024-09-12T03:22:20.566481+0000","last_undegraded":"2024-09-12T03:22:20.566481+0000","last_fullsized":"2024-09-12T03:22:20.566481+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_clean_scrub_stamp":"2024-09-12T03:22:18.392443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:08:13.228080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.1e","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T03:22:20.430835+0000","last_change":"2024-09-12T03:22:19.437533+0000","last_active":"2024-09-12T03:22:20.430835+0000","last_peered":"2024-09-12T03:22:20.430835+0000","last_clean":"2024-09-12T03:22:20.430835+0000","last_became_active":"2024-09-12T03:22:19.436643+0000","last_became_peered":"2024-09-12T03:22:19.436643+0000","last_unstale":"2024-09-12T03:22:20.430835+0000","last_undegraded":"2024-09-12T03:22:20.430835+0000","last_fullsized":"2024-09-12T03:22:20.430835+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_clean_scrub_stamp":"2024-09-12T03:22:18.392443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:27:51.778796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1f","version":"125'4","reported_seq":18,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T03:22:20.430787+0000","last_change":"2024-09-12T03:22:18.415035+0000","last_active":"2024-09-12T03:22:20.430787+0000","last_peered":"2024-09-12T03:22:20.430787+0000","last_clean":"2024-09-12T03:22:20.430787+0000","last_became_active":"2024-09-12T03:22:18.414211+0000","last_became_peered":"2024-09-12T03:22:18.414211+0000","last_unstale":"2024-09-12T03:22:20.430787+0000","last_undegraded":"2024-09-12T03:22:20.430787+0000","last_fullsized":"2024-09-12T03:22:20.430787+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:17.387205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:17.387205+0000","last_clean_scrub_stamp":"2024-09-12T03:22:17.387205+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:19:14.731819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1f","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T03:22:20.565997+0000","last_change":"2024-09-12T03:22:19.427606+0000","last_active":"2024-09-12T03:22:20.565997+0000","last_peered":"2024-09-12T03:22:20.565997+0000","last_clean":"2024-09-12T03:22:20.565997+0000","last_became_active":"2024-09-12T03:22:19.427164+0000","last_became_peered":"2024-09-12T03:22:19.427164+0000","last_unstale":"2024-09-12T03:22:20.565997+0000","last_undegraded":"2024-09-12T03:22:20.565997+0000","last_fullsized":"2024-09-12T03:22:20.565997+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_clean_scrub_stamp":"2024-09-12T03:22:18.392443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:24:28.462100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1e","version":"125'1","reported_seq":15,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T03:22:20.565763+0000","last_change":"2024-09-12T03:22:18.412937+0000","last_active":"2024-09-12T03:22:20.565763+0000","last_peered":"2024-09-12T03:22:20.565763+0000","last_clean":"2024-09-12T03:22:20.565763+0000","last_became_active":"2024-09-12T03:22:18.412665+0000","last_became_peered":"2024-09-12T03:22:18.412665+0000","last_unstale":"2024-09-12T03:22:20.565763+0000","last_undegraded":"2024-09-12T03:22:20.565763+0000","last_fullsized":"2024-09-12T03:22:20.565763+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:17.387205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:17.387205+0000","last_clean_scrub_stamp":"2024-09-12T03:22:17.387205+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:26:03.137156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1c","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T03:22:20.427504+0000","last_change":"2024-09-12T03:22:19.420006+0000","last_active":"2024-09-12T03:22:20.427504+0000","last_peered":"2024-09-12T03:22:20.427504+0000","last_clean":"2024-09-12T03:22:20.427504+0000","last_became_active":"2024-09-12T03:22:19.419715+0000","last_became_peered":"2024-09-12T03:22:19.419715+0000","last_unstale":"2024-09-12T03:22:20.427504+0000","last_undegraded":"2024-09-12T03:22:20.427504+0000","last_fullsized":"2024-09-12T03:22:20.427504+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_clean_scrub_stamp":"2024-09-12T03:22:18.392443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:28:53.157392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1d","version":"125'2","reported_seq":16,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T03:22:20.411504+0000","last_change":"2024-09-12T03:22:18.411480+0000","last_active":"2024-09-12T03:22:20.411504+0000","last_peered":"2024-09-12T03:22:20.411504+0000","last_clean":"2024-09-12T03:22:20.411504+0000","last_became_active":"2024-09-12T03:22:18.411210+0000","last_became_peered":"2024-09-12T03:22:18.411210+0000","last_unstale":"2024-09-12T03:22:20.411504+0000","last_undegraded":"2024-09-12T03:22:20.411504+0000","last_fullsized":"2024-09-12T03:22:20.411504+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:17.387205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:17.387205+0000","last_clean_scrub_stamp":"2024-09-12T03:22:17.387205+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:27:55.581131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1d","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T03:22:20.430656+0000","last_change":"2024-09-12T03:22:19.437278+0000","last_active":"2024-09-12T03:22:20.430656+0000","last_peered":"2024-09-12T03:22:20.430656+0000","last_clean":"2024-09-12T03:22:20.430656+0000","last_became_active":"2024-09-12T03:22:19.436958+0000","last_became_peered":"2024-09-12T03:22:19.436958+0000","last_unstale":"2024-09-12T03:22:20.430656+0000","last_undegraded":"2024-09-12T03:22:20.430656+0000","last_fullsized":"2024-09-12T03:22:20.430656+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_clean_scrub_stamp":"2024-09-12T03:22:18.392443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:40:14.475613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1c","version":"125'2","reported_seq":16,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T03:22:20.411930+0000","last_change":"2024-09-12T03:22:18.413232+0000","last_active":"2024-09-12T03:22:20.411930+0000","last_peered":"2024-09-12T03:22:20.411930+0000","last_clean":"2024-09-12T03:22:20.411930+0000","last_became_active":"2024-09-12T03:22:18.412774+0000","last_became_peered":"2024-09-12T03:22:18.412774+0000","last_unstale":"2024-09-12T03:22:20.411930+0000","last_undegraded":"2024-09-12T03:22:20.411930+0000","last_fullsized":"2024-09-12T03:22:20.411930+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:17.387205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:17.387205+0000","last_clean_scrub_stamp":"2024-09-12T03:22:17.387205+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:06:23.253785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1a","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T03:22:20.565737+0000","last_change":"2024-09-12T03:22:19.429572+0000","last_active":"2024-09-12T03:22:20.565737+0000","last_peered":"2024-09-12T03:22:20.565737+0000","last_clean":"2024-09-12T03:22:20.565737+0000","last_became_active":"2024-09-12T03:22:19.428433+0000","last_became_peered":"2024-09-12T03:22:19.428433+0000","last_unstale":"2024-09-12T03:22:20.565737+0000","last_undegraded":"2024-09-12T03:22:20.565737+0000","last_fullsized":"2024-09-12T03:22:20.565737+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_clean_scrub_stamp":"2024-09-12T03:22:18.392443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:19:15.737056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1b","version":"125'1","reported_seq":15,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T03:22:20.566532+0000","last_change":"2024-09-12T03:22:18.417349+0000","last_active":"2024-09-12T03:22:20.566532+0000","last_peered":"2024-09-12T03:22:20.566532+0000","last_clean":"2024-09-12T03:22:20.566532+0000","last_became_active":"2024-09-12T03:22:18.416955+0000","last_became_peered":"2024-09-12T03:22:18.416955+0000","last_unstale":"2024-09-12T03:22:20.566532+0000","last_undegraded":"2024-09-12T03:22:20.566532+0000","last_fullsized":"2024-09-12T03:22:20.566532+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:17.387205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:17.387205+0000","last_clean_scrub_stamp":"2024-09-12T03:22:17.387205+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:29:56.097726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.1b","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T03:22:20.411472+0000","last_change":"2024-09-12T03:22:19.429046+0000","last_active":"2024-09-12T03:22:20.411472+0000","last_peered":"2024-09-12T03:22:20.411472+0000","last_clean":"2024-09-12T03:22:20.411472+0000","last_became_active":"2024-09-12T03:22:19.428763+0000","last_became_peered":"2024-09-12T03:22:19.428763+0000","last_unstale":"2024-09-12T03:22:20.411472+0000","last_undegraded":"2024-09-12T03:22:20.411472+0000","last_fullsized":"2024-09-12T03:22:20.411472+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_clean_scrub_stamp":"2024-09-12T03:22:18.392443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:39:00.233271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.1a","version":"125'1","reported_seq":15,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T03:22:20.411280+0000","last_change":"2024-09-12T03:22:18.411867+0000","last_active":"2024-09-12T03:22:20.411280+0000","last_peered":"2024-09-12T03:22:20.411280+0000","last_clean":"2024-09-12T03:22:20.411280+0000","last_became_active":"2024-09-12T03:22:18.411526+0000","last_became_peered":"2024-09-12T03:22:18.411526+0000","last_unstale":"2024-09-12T03:22:20.411280+0000","last_undegraded":"2024-09-12T03:22:20.411280+0000","last_fullsized":"2024-09-12T03:22:20.411280+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:17.387205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:17.387205+0000","last_clean_scrub_stamp":"2024-09-12T03:22:17.387205+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:57:50.843023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T03:22:20.565525+0000","last_change":"2024-09-12T03:22:19.431554+0000","last_active":"2024-09-12T03:22:20.565525+0000","last_peered":"2024-09-12T03:22:20.565525+0000","last_clean":"2024-09-12T03:22:20.565525+0000","last_became_active":"2024-09-12T03:22:19.431259+0000","last_became_peered":"2024-09-12T03:22:19.431259+0000","last_unstale":"2024-09-12T03:22:20.565525+0000","last_undegraded":"2024-09-12T03:22:20.565525+0000","last_fullsized":"2024-09-12T03:22:20.565525+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_clean_scrub_stamp":"2024-09-12T03:22:18.392443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:30:36.215990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":14,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T03:22:20.566342+0000","last_change":"2024-09-12T03:22:18.416398+0000","last_active":"2024-09-12T03:22:20.566342+0000","last_peered":"2024-09-12T03:22:20.566342+0000","last_clean":"2024-09-12T03:22:20.566342+0000","last_became_active":"2024-09-12T03:22:18.416123+0000","last_became_peered":"2024-09-12T03:22:18.416123+0000","last_unstale":"2024-09-12T03:22:20.566342+0000","last_undegraded":"2024-09-12T03:22:20.566342+0000","last_fullsized":"2024-09-12T03:22:20.566342+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:17.387205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:17.387205+0000","last_clean_scrub_stamp":"2024-09-12T03:22:17.387205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:36:33.858935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T03:22:20.411901+0000","last_change":"2024-09-12T03:22:19.418041+0000","last_active":"2024-09-12T03:22:20.411901+0000","last_peered":"2024-09-12T03:22:20.411901+0000","last_clean":"2024-09-12T03:22:20.411901+0000","last_became_active":"2024-09-12T03:22:19.417688+0000","last_became_peered":"2024-09-12T03:22:19.417688+0000","last_unstale":"2024-09-12T03:22:20.411901+0000","last_undegraded":"2024-09-12T03:22:20.411901+0000","last_fullsized":"2024-09-12T03:22:20.411901+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_clean_scrub_stamp":"2024-09-12T03:22:18.392443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:04:34.555718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":14,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T03:22:20.411091+0000","last_change":"2024-09-12T03:22:18.409718+0000","last_active":"2024-09-12T03:22:20.411091+0000","last_peered":"2024-09-12T03:22:20.411091+0000","last_clean":"2024-09-12T03:22:20.411091+0000","last_became_active":"2024-09-12T03:22:18.409325+0000","last_became_peered":"2024-09-12T03:22:18.409325+0000","last_unstale":"2024-09-12T03:22:20.411091+0000","last_undegraded":"2024-09-12T03:22:20.411091+0000","last_fullsized":"2024-09-12T03:22:20.411091+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:17.387205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:17.387205+0000","last_clean_scrub_stamp":"2024-09-12T03:22:17.387205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:43:02.304826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T03:22:20.411628+0000","last_change":"2024-09-12T03:22:19.425698+0000","last_active":"2024-09-12T03:22:20.411628+0000","last_peered":"2024-09-12T03:22:20.411628+0000","last_clean":"2024-09-12T03:22:20.411628+0000","last_became_active":"2024-09-12T03:22:19.425355+0000","last_became_peered":"2024-09-12T03:22:19.425355+0000","last_unstale":"2024-09-12T03:22:20.411628+0000","last_undegraded":"2024-09-12T03:22:20.411628+0000","last_fullsized":"2024-09-12T03:22:20.411628+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_clean_scrub_stamp":"2024-09-12T03:22:18.392443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:05:38.055036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":14,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T03:22:20.565973+0000","last_change":"2024-09-12T03:22:18.412282+0000","last_active":"2024-09-12T03:22:20.565973+0000","last_peered":"2024-09-12T03:22:20.565973+0000","last_clean":"2024-09-12T03:22:20.565973+0000","last_became_active":"2024-09-12T03:22:18.411973+0000","last_became_peered":"2024-09-12T03:22:18.411973+0000","last_unstale":"2024-09-12T03:22:20.565973+0000","last_undegraded":"2024-09-12T03:22:20.565973+0000","last_fullsized":"2024-09-12T03:22:20.565973+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:17.387205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:17.387205+0000","last_clean_scrub_stamp":"2024-09-12T03:22:17.387205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:00:44.000131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T03:22:20.430800+0000","last_change":"2024-09-12T03:22:19.432103+0000","last_active":"2024-09-12T03:22:20.430800+0000","last_peered":"2024-09-12T03:22:20.430800+0000","last_clean":"2024-09-12T03:22:20.430800+0000","last_became_active":"2024-09-12T03:22:19.431762+0000","last_became_peered":"2024-09-12T03:22:19.431762+0000","last_unstale":"2024-09-12T03:22:20.430800+0000","last_undegraded":"2024-09-12T03:22:20.430800+0000","last_fullsized":"2024-09-12T03:22:20.430800+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_clean_scrub_stamp":"2024-09-12T03:22:18.392443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:20:01.511235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":14,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T03:22:20.430964+0000","last_change":"2024-09-12T03:22:18.411651+0000","last_active":"2024-09-12T03:22:20.430964+0000","last_peered":"2024-09-12T03:22:20.430964+0000","last_clean":"2024-09-12T03:22:20.430964+0000","last_became_active":"2024-09-12T03:22:18.411347+0000","last_became_peered":"2024-09-12T03:22:18.411347+0000","last_unstale":"2024-09-12T03:22:20.430964+0000","last_undegraded":"2024-09-12T03:22:20.430964+0000","last_fullsized":"2024-09-12T03:22:20.430964+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:17.387205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:17.387205+0000","last_clean_scrub_stamp":"2024-09-12T03:22:17.387205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:27:39.911375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.15","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T03:22:20.566013+0000","last_change":"2024-09-12T03:22:19.427469+0000","last_active":"2024-09-12T03:22:20.566013+0000","last_peered":"2024-09-12T03:22:20.566013+0000","last_clean":"2024-09-12T03:22:20.566013+0000","last_became_active":"2024-09-12T03:22:19.427194+0000","last_became_peered":"2024-09-12T03:22:19.427194+0000","last_unstale":"2024-09-12T03:22:20.566013+0000","last_undegraded":"2024-09-12T03:22:20.566013+0000","last_fullsized":"2024-09-12T03:22:20.566013+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_clean_scrub_stamp":"2024-09-12T03:22:18.392443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:41:58.275814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.14","version":"125'6","reported_seq":20,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T03:22:20.410414+0000","last_change":"2024-09-12T03:22:18.411529+0000","last_active":"2024-09-12T03:22:20.410414+0000","last_peered":"2024-09-12T03:22:20.410414+0000","last_clean":"2024-09-12T03:22:20.410414+0000","last_became_active":"2024-09-12T03:22:18.411203+0000","last_became_peered":"2024-09-12T03:22:18.411203+0000","last_unstale":"2024-09-12T03:22:20.410414+0000","last_undegraded":"2024-09-12T03:22:20.410414+0000","last_fullsized":"2024-09-12T03:22:20.410414+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:17.387205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:17.387205+0000","last_clean_scrub_stamp":"2024-09-12T03:22:17.387205+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:02:01.032597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T03:22:20.566388+0000","last_change":"2024-09-12T03:22:19.427359+0000","last_active":"2024-09-12T03:22:20.566388+0000","last_peered":"2024-09-12T03:22:20.566388+0000","last_clean":"2024-09-12T03:22:20.566388+0000","last_became_active":"2024-09-12T03:22:19.426927+0000","last_became_peered":"2024-09-12T03:22:19.426927+0000","last_unstale":"2024-09-12T03:22:20.566388+0000","last_undegraded":"2024-09-12T03:22:20.566388+0000","last_fullsized":"2024-09-12T03:22:20.566388+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_clean_scrub_stamp":"2024-09-12T03:22:18.392443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:06:24.259022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.13","version":"125'2","reported_seq":16,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T03:22:20.566660+0000","last_change":"2024-09-12T03:22:18.412269+0000","last_active":"2024-09-12T03:22:20.566660+0000","last_peered":"2024-09-12T03:22:20.566660+0000","last_clean":"2024-09-12T03:22:20.566660+0000","last_became_active":"2024-09-12T03:22:18.411932+0000","last_became_peered":"2024-09-12T03:22:18.411932+0000","last_unstale":"2024-09-12T03:22:20.566660+0000","last_undegraded":"2024-09-12T03:22:20.566660+0000","last_fullsized":"2024-09-12T03:22:20.566660+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:17.387205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:17.387205+0000","last_clean_scrub_stamp":"2024-09-12T03:22:17.387205+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:56:21.746135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.13","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T03:22:20.565906+0000","last_change":"2024-09-12T03:22:19.432506+0000","last_active":"2024-09-12T03:22:20.565906+0000","last_peered":"2024-09-12T03:22:20.565906+0000","last_clean":"2024-09-12T03:22:20.565906+0000","last_became_active":"2024-09-12T03:22:19.432203+0000","last_became_peered":"2024-09-12T03:22:19.432203+0000","last_unstale":"2024-09-12T03:22:20.565906+0000","last_undegraded":"2024-09-12T03:22:20.565906+0000","last_fullsized":"2024-09-12T03:22:20.565906+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_clean_scrub_stamp":"2024-09-12T03:22:18.392443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:38:12.159504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":14,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T03:22:20.566470+0000","last_change":"2024-09-12T03:22:18.410561+0000","last_active":"2024-09-12T03:22:20.566470+0000","last_peered":"2024-09-12T03:22:20.566470+0000","last_clean":"2024-09-12T03:22:20.566470+0000","last_became_active":"2024-09-12T03:22:18.410222+0000","last_became_peered":"2024-09-12T03:22:18.410222+0000","last_unstale":"2024-09-12T03:22:20.566470+0000","last_undegraded":"2024-09-12T03:22:20.566470+0000","last_fullsized":"2024-09-12T03:22:20.566470+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:17.387205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:17.387205+0000","last_clean_scrub_stamp":"2024-09-12T03:22:17.387205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:55:25.079169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.10","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T03:22:20.411085+0000","last_change":"2024-09-12T03:22:19.432651+0000","last_active":"2024-09-12T03:22:20.411085+0000","last_peered":"2024-09-12T03:22:20.411085+0000","last_clean":"2024-09-12T03:22:20.411085+0000","last_became_active":"2024-09-12T03:22:19.431932+0000","last_became_peered":"2024-09-12T03:22:19.431932+0000","last_unstale":"2024-09-12T03:22:20.411085+0000","last_undegraded":"2024-09-12T03:22:20.411085+0000","last_fullsized":"2024-09-12T03:22:20.411085+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_clean_scrub_stamp":"2024-09-12T03:22:18.392443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:24:46.851580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":14,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T03:22:20.427301+0000","last_change":"2024-09-12T03:22:18.412405+0000","last_active":"2024-09-12T03:22:20.427301+0000","last_peered":"2024-09-12T03:22:20.427301+0000","last_clean":"2024-09-12T03:22:20.427301+0000","last_became_active":"2024-09-12T03:22:18.412125+0000","last_became_peered":"2024-09-12T03:22:18.412125+0000","last_unstale":"2024-09-12T03:22:20.427301+0000","last_undegraded":"2024-09-12T03:22:20.427301+0000","last_fullsized":"2024-09-12T03:22:20.427301+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:17.387205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:17.387205+0000","last_clean_scrub_stamp":"2024-09-12T03:22:17.387205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:57:17.314093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T03:22:20.566132+0000","last_change":"2024-09-12T03:22:19.428116+0000","last_active":"2024-09-12T03:22:20.566132+0000","last_peered":"2024-09-12T03:22:20.566132+0000","last_clean":"2024-09-12T03:22:20.566132+0000","last_became_active":"2024-09-12T03:22:19.427821+0000","last_became_peered":"2024-09-12T03:22:19.427821+0000","last_unstale":"2024-09-12T03:22:20.566132+0000","last_undegraded":"2024-09-12T03:22:20.566132+0000","last_fullsized":"2024-09-12T03:22:20.566132+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_clean_scrub_stamp":"2024-09-12T03:22:18.392443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:27:56.586369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.10","version":"125'8","reported_seq":22,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T03:22:20.566537+0000","last_change":"2024-09-12T03:22:18.409429+0000","last_active":"2024-09-12T03:22:20.566537+0000","last_peered":"2024-09-12T03:22:20.566537+0000","last_clean":"2024-09-12T03:22:20.566537+0000","last_became_active":"2024-09-12T03:22:18.408740+0000","last_became_peered":"2024-09-12T03:22:18.408740+0000","last_unstale":"2024-09-12T03:22:20.566537+0000","last_undegraded":"2024-09-12T03:22:20.566537+0000","last_fullsized":"2024-09-12T03:22:20.566537+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:17.387205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:17.387205+0000","last_clean_scrub_stamp":"2024-09-12T03:22:17.387205+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:28:19.216947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.e","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T03:22:20.430972+0000","last_change":"2024-09-12T03:22:19.436685+0000","last_active":"2024-09-12T03:22:20.430972+0000","last_peered":"2024-09-12T03:22:20.430972+0000","last_clean":"2024-09-12T03:22:20.430972+0000","last_became_active":"2024-09-12T03:22:19.436237+0000","last_became_peered":"2024-09-12T03:22:19.436237+0000","last_unstale":"2024-09-12T03:22:20.430972+0000","last_undegraded":"2024-09-12T03:22:20.430972+0000","last_fullsized":"2024-09-12T03:22:20.430972+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_clean_scrub_stamp":"2024-09-12T03:22:18.392443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:02:02.037835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.f","version":"125'1","reported_seq":15,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T03:22:20.411242+0000","last_change":"2024-09-12T03:22:18.406274+0000","last_active":"2024-09-12T03:22:20.411242+0000","last_peered":"2024-09-12T03:22:20.411242+0000","last_clean":"2024-09-12T03:22:20.411242+0000","last_became_active":"2024-09-12T03:22:18.405977+0000","last_became_peered":"2024-09-12T03:22:18.405977+0000","last_unstale":"2024-09-12T03:22:20.411242+0000","last_undegraded":"2024-09-12T03:22:20.411242+0000","last_fullsized":"2024-09-12T03:22:20.411242+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:17.387205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:17.387205+0000","last_clean_scrub_stamp":"2024-09-12T03:22:17.387205+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:07:24.978733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.f","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T03:22:20.566160+0000","last_change":"2024-09-12T03:22:19.425138+0000","last_active":"2024-09-12T03:22:20.566160+0000","last_peered":"2024-09-12T03:22:20.566160+0000","last_clean":"2024-09-12T03:22:20.566160+0000","last_became_active":"2024-09-12T03:22:19.424853+0000","last_became_peered":"2024-09-12T03:22:19.424853+0000","last_unstale":"2024-09-12T03:22:20.566160+0000","last_undegraded":"2024-09-12T03:22:20.566160+0000","last_fullsized":"2024-09-12T03:22:20.566160+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_clean_scrub_stamp":"2024-09-12T03:22:18.392443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:43:16.888355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.e","version":"125'2","reported_seq":16,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T03:22:20.565867+0000","last_change":"2024-09-12T03:22:18.415917+0000","last_active":"2024-09-12T03:22:20.565867+0000","last_peered":"2024-09-12T03:22:20.565867+0000","last_clean":"2024-09-12T03:22:20.565867+0000","last_became_active":"2024-09-12T03:22:18.415574+0000","last_became_peered":"2024-09-12T03:22:18.415574+0000","last_unstale":"2024-09-12T03:22:20.565867+0000","last_undegraded":"2024-09-12T03:22:20.565867+0000","last_fullsized":"2024-09-12T03:22:20.565867+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:17.387205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:17.387205+0000","last_clean_scrub_stamp":"2024-09-12T03:22:17.387205+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:53:00.789204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T03:22:20.566638+0000","last_change":"2024-09-12T03:22:19.429196+0000","last_active":"2024-09-12T03:22:20.566638+0000","last_peered":"2024-09-12T03:22:20.566638+0000","last_clean":"2024-09-12T03:22:20.566638+0000","last_became_active":"2024-09-12T03:22:19.428379+0000","last_became_peered":"2024-09-12T03:22:19.428379+0000","last_unstale":"2024-09-12T03:22:20.566638+0000","last_undegraded":"2024-09-12T03:22:20.566638+0000","last_fullsized":"2024-09-12T03:22:20.566638+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_clean_scrub_stamp":"2024-09-12T03:22:18.392443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:26:04.142393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.d","version":"125'1","reported_seq":15,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T03:22:20.566123+0000","last_change":"2024-09-12T03:22:18.417315+0000","last_active":"2024-09-12T03:22:20.566123+0000","last_peered":"2024-09-12T03:22:20.566123+0000","last_clean":"2024-09-12T03:22:20.566123+0000","last_became_active":"2024-09-12T03:22:18.416924+0000","last_became_peered":"2024-09-12T03:22:18.416924+0000","last_unstale":"2024-09-12T03:22:20.566123+0000","last_undegraded":"2024-09-12T03:22:20.566123+0000","last_fullsized":"2024-09-12T03:22:20.566123+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:17.387205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:17.387205+0000","last_clean_scrub_stamp":"2024-09-12T03:22:17.387205+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:36:28.656742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.d","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T03:22:20.427479+0000","last_change":"2024-09-12T03:22:19.436256+0000","last_active":"2024-09-12T03:22:20.427479+0000","last_peered":"2024-09-12T03:22:20.427479+0000","last_clean":"2024-09-12T03:22:20.427479+0000","last_became_active":"2024-09-12T03:22:19.435970+0000","last_became_peered":"2024-09-12T03:22:19.435970+0000","last_unstale":"2024-09-12T03:22:20.427479+0000","last_undegraded":"2024-09-12T03:22:20.427479+0000","last_fullsized":"2024-09-12T03:22:20.427479+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_clean_scrub_stamp":"2024-09-12T03:22:18.392443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:56:57.586482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":14,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T03:22:20.566249+0000","last_change":"2024-09-12T03:22:18.414031+0000","last_active":"2024-09-12T03:22:20.566249+0000","last_peered":"2024-09-12T03:22:20.566249+0000","last_clean":"2024-09-12T03:22:20.566249+0000","last_became_active":"2024-09-12T03:22:18.413712+0000","last_became_peered":"2024-09-12T03:22:18.413712+0000","last_unstale":"2024-09-12T03:22:20.566249+0000","last_undegraded":"2024-09-12T03:22:20.566249+0000","last_fullsized":"2024-09-12T03:22:20.566249+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:17.387205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:17.387205+0000","last_clean_scrub_stamp":"2024-09-12T03:22:17.387205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:47:01.633048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T03:22:20.412366+0000","last_change":"2024-09-12T03:22:19.427210+0000","last_active":"2024-09-12T03:22:20.412366+0000","last_peered":"2024-09-12T03:22:20.412366+0000","last_clean":"2024-09-12T03:22:20.412366+0000","last_became_active":"2024-09-12T03:22:19.425379+0000","last_became_peered":"2024-09-12T03:22:19.425379+0000","last_unstale":"2024-09-12T03:22:20.412366+0000","last_undegraded":"2024-09-12T03:22:20.412366+0000","last_fullsized":"2024-09-12T03:22:20.412366+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_clean_scrub_stamp":"2024-09-12T03:22:18.392443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:11:09.928417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.b","version":"125'2","reported_seq":16,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T03:22:20.410150+0000","last_change":"2024-09-12T03:22:18.407704+0000","last_active":"2024-09-12T03:22:20.410150+0000","last_peered":"2024-09-12T03:22:20.410150+0000","last_clean":"2024-09-12T03:22:20.410150+0000","last_became_active":"2024-09-12T03:22:18.407392+0000","last_became_peered":"2024-09-12T03:22:18.407392+0000","last_unstale":"2024-09-12T03:22:20.410150+0000","last_undegraded":"2024-09-12T03:22:20.410150+0000","last_fullsized":"2024-09-12T03:22:20.410150+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:17.387205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:17.387205+0000","last_clean_scrub_stamp":"2024-09-12T03:22:17.387205+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:27:50.773558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.b","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T03:22:20.410173+0000","last_change":"2024-09-12T03:22:19.425182+0000","last_active":"2024-09-12T03:22:20.410173+0000","last_peered":"2024-09-12T03:22:20.410173+0000","last_clean":"2024-09-12T03:22:20.410173+0000","last_became_active":"2024-09-12T03:22:19.423095+0000","last_became_peered":"2024-09-12T03:22:19.423095+0000","last_unstale":"2024-09-12T03:22:20.410173+0000","last_undegraded":"2024-09-12T03:22:20.410173+0000","last_fullsized":"2024-09-12T03:22:20.410173+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_clean_scrub_stamp":"2024-09-12T03:22:18.392443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:15:32.505422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":14,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T03:22:20.430357+0000","last_change":"2024-09-12T03:22:18.414506+0000","last_active":"2024-09-12T03:22:20.430357+0000","last_peered":"2024-09-12T03:22:20.430357+0000","last_clean":"2024-09-12T03:22:20.430357+0000","last_became_active":"2024-09-12T03:22:18.414138+0000","last_became_peered":"2024-09-12T03:22:18.414138+0000","last_unstale":"2024-09-12T03:22:20.430357+0000","last_undegraded":"2024-09-12T03:22:20.430357+0000","last_fullsized":"2024-09-12T03:22:20.430357+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:17.387205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:17.387205+0000","last_clean_scrub_stamp":"2024-09-12T03:22:17.387205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:07:57.019892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T03:22:20.411113+0000","last_change":"2024-09-12T03:22:19.421627+0000","last_active":"2024-09-12T03:22:20.411113+0000","last_peered":"2024-09-12T03:22:20.411113+0000","last_clean":"2024-09-12T03:22:20.411113+0000","last_became_active":"2024-09-12T03:22:19.421280+0000","last_became_peered":"2024-09-12T03:22:19.421280+0000","last_unstale":"2024-09-12T03:22:20.411113+0000","last_undegraded":"2024-09-12T03:22:20.411113+0000","last_fullsized":"2024-09-12T03:22:20.411113+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_clean_scrub_stamp":"2024-09-12T03:22:18.392443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:11:08.266994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":14,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T03:22:20.430191+0000","last_change":"2024-09-12T03:22:18.413128+0000","last_active":"2024-09-12T03:22:20.430191+0000","last_peered":"2024-09-12T03:22:20.430191+0000","last_clean":"2024-09-12T03:22:20.430191+0000","last_became_active":"2024-09-12T03:22:18.412815+0000","last_became_peered":"2024-09-12T03:22:18.412815+0000","last_unstale":"2024-09-12T03:22:20.430191+0000","last_undegraded":"2024-09-12T03:22:20.430191+0000","last_fullsized":"2024-09-12T03:22:20.430191+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:17.387205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:17.387205+0000","last_clean_scrub_stamp":"2024-09-12T03:22:17.387205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:30:35.210752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T03:22:20.410182+0000","last_change":"2024-09-12T03:22:19.420982+0000","last_active":"2024-09-12T03:22:20.410182+0000","last_peered":"2024-09-12T03:22:20.410182+0000","last_clean":"2024-09-12T03:22:20.410182+0000","last_became_active":"2024-09-12T03:22:19.420532+0000","last_became_peered":"2024-09-12T03:22:19.420532+0000","last_unstale":"2024-09-12T03:22:20.410182+0000","last_undegraded":"2024-09-12T03:22:20.410182+0000","last_fullsized":"2024-09-12T03:22:20.410182+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_clean_scrub_stamp":"2024-09-12T03:22:18.392443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:00:32.782156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":14,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T03:22:20.566036+0000","last_change":"2024-09-12T03:22:18.415874+0000","last_active":"2024-09-12T03:22:20.566036+0000","last_peered":"2024-09-12T03:22:20.566036+0000","last_clean":"2024-09-12T03:22:20.566036+0000","last_became_active":"2024-09-12T03:22:18.415465+0000","last_became_peered":"2024-09-12T03:22:18.415465+0000","last_unstale":"2024-09-12T03:22:20.566036+0000","last_undegraded":"2024-09-12T03:22:20.566036+0000","last_fullsized":"2024-09-12T03:22:20.566036+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:17.387205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:17.387205+0000","last_clean_scrub_stamp":"2024-09-12T03:22:17.387205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:38:48.375420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T03:22:20.411791+0000","last_change":"2024-09-12T03:22:19.418987+0000","last_active":"2024-09-12T03:22:20.411791+0000","last_peered":"2024-09-12T03:22:20.411791+0000","last_clean":"2024-09-12T03:22:20.411791+0000","last_became_active":"2024-09-12T03:22:19.418708+0000","last_became_peered":"2024-09-12T03:22:19.418708+0000","last_unstale":"2024-09-12T03:22:20.411791+0000","last_undegraded":"2024-09-12T03:22:20.411791+0000","last_fullsized":"2024-09-12T03:22:20.411791+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_clean_scrub_stamp":"2024-09-12T03:22:18.392443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:17:25.507271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.7","version":"125'2","reported_seq":16,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T03:22:20.430942+0000","last_change":"2024-09-12T03:22:18.415733+0000","last_active":"2024-09-12T03:22:20.430942+0000","last_peered":"2024-09-12T03:22:20.430942+0000","last_clean":"2024-09-12T03:22:20.430942+0000","last_became_active":"2024-09-12T03:22:18.415408+0000","last_became_peered":"2024-09-12T03:22:18.415408+0000","last_unstale":"2024-09-12T03:22:20.430942+0000","last_undegraded":"2024-09-12T03:22:20.430942+0000","last_fullsized":"2024-09-12T03:22:20.430942+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:17.387205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:17.387205+0000","last_clean_scrub_stamp":"2024-09-12T03:22:17.387205+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:50:23.097953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.7","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T03:22:20.566695+0000","last_change":"2024-09-12T03:22:19.427214+0000","last_active":"2024-09-12T03:22:20.566695+0000","last_peered":"2024-09-12T03:22:20.566695+0000","last_clean":"2024-09-12T03:22:20.566695+0000","last_became_active":"2024-09-12T03:22:19.426845+0000","last_became_peered":"2024-09-12T03:22:19.426845+0000","last_unstale":"2024-09-12T03:22:20.566695+0000","last_undegraded":"2024-09-12T03:22:20.566695+0000","last_fullsized":"2024-09-12T03:22:20.566695+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_clean_scrub_stamp":"2024-09-12T03:22:18.392443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:53:01.794441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.6","version":"126'1","reported_seq":15,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T03:22:20.429416+0000","last_change":"2024-09-12T03:22:18.414645+0000","last_active":"2024-09-12T03:22:20.429416+0000","last_peered":"2024-09-12T03:22:20.429416+0000","last_clean":"2024-09-12T03:22:20.429416+0000","last_became_active":"2024-09-12T03:22:18.414345+0000","last_became_peered":"2024-09-12T03:22:18.414345+0000","last_unstale":"2024-09-12T03:22:20.429416+0000","last_undegraded":"2024-09-12T03:22:20.429416+0000","last_fullsized":"2024-09-12T03:22:20.429416+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:17.387205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:17.387205+0000","last_clean_scrub_stamp":"2024-09-12T03:22:17.387205+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:56:18.878188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T03:22:20.566159+0000","last_change":"2024-09-12T03:22:19.425316+0000","last_active":"2024-09-12T03:22:20.566159+0000","last_peered":"2024-09-12T03:22:20.566159+0000","last_clean":"2024-09-12T03:22:20.566159+0000","last_became_active":"2024-09-12T03:22:19.425041+0000","last_became_peered":"2024-09-12T03:22:19.425041+0000","last_unstale":"2024-09-12T03:22:20.566159+0000","last_undegraded":"2024-09-12T03:22:20.566159+0000","last_fullsized":"2024-09-12T03:22:20.566159+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_clean_scrub_stamp":"2024-09-12T03:22:18.392443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:50:13.734588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":14,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T03:22:20.427702+0000","last_change":"2024-09-12T03:22:18.416225+0000","last_active":"2024-09-12T03:22:20.427702+0000","last_peered":"2024-09-12T03:22:20.427702+0000","last_clean":"2024-09-12T03:22:20.427702+0000","last_became_active":"2024-09-12T03:22:18.415966+0000","last_became_peered":"2024-09-12T03:22:18.415966+0000","last_unstale":"2024-09-12T03:22:20.427702+0000","last_undegraded":"2024-09-12T03:22:20.427702+0000","last_fullsized":"2024-09-12T03:22:20.427702+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:17.387205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:17.387205+0000","last_clean_scrub_stamp":"2024-09-12T03:22:17.387205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:14:59.090991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T03:22:20.427611+0000","last_change":"2024-09-12T03:22:19.419166+0000","last_active":"2024-09-12T03:22:20.427611+0000","last_peered":"2024-09-12T03:22:20.427611+0000","last_clean":"2024-09-12T03:22:20.427611+0000","last_became_active":"2024-09-12T03:22:19.418870+0000","last_became_peered":"2024-09-12T03:22:19.418870+0000","last_unstale":"2024-09-12T03:22:20.427611+0000","last_undegraded":"2024-09-12T03:22:20.427611+0000","last_fullsized":"2024-09-12T03:22:20.427611+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_clean_scrub_stamp":"2024-09-12T03:22:18.392443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:31:36.296210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":14,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T03:22:20.427518+0000","last_change":"2024-09-12T03:22:18.416919+0000","last_active":"2024-09-12T03:22:20.427518+0000","last_peered":"2024-09-12T03:22:20.427518+0000","last_clean":"2024-09-12T03:22:20.427518+0000","last_became_active":"2024-09-12T03:22:18.416646+0000","last_became_peered":"2024-09-12T03:22:18.416646+0000","last_unstale":"2024-09-12T03:22:20.427518+0000","last_undegraded":"2024-09-12T03:22:20.427518+0000","last_fullsized":"2024-09-12T03:22:20.427518+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:17.387205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:17.387205+0000","last_clean_scrub_stamp":"2024-09-12T03:22:17.387205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:24:45.846343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T03:22:20.565690+0000","last_change":"2024-09-12T03:22:19.422916+0000","last_active":"2024-09-12T03:22:20.565690+0000","last_peered":"2024-09-12T03:22:20.565690+0000","last_clean":"2024-09-12T03:22:20.565690+0000","last_became_active":"2024-09-12T03:22:19.422632+0000","last_became_peered":"2024-09-12T03:22:19.422632+0000","last_unstale":"2024-09-12T03:22:20.565690+0000","last_undegraded":"2024-09-12T03:22:20.565690+0000","last_fullsized":"2024-09-12T03:22:20.565690+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_clean_scrub_stamp":"2024-09-12T03:22:18.392443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:33:20.519934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":14,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T03:22:20.410250+0000","last_change":"2024-09-12T03:22:18.405214+0000","last_active":"2024-09-12T03:22:20.410250+0000","last_peered":"2024-09-12T03:22:20.410250+0000","last_clean":"2024-09-12T03:22:20.410250+0000","last_became_active":"2024-09-12T03:22:18.404954+0000","last_became_peered":"2024-09-12T03:22:18.404954+0000","last_unstale":"2024-09-12T03:22:20.410250+0000","last_undegraded":"2024-09-12T03:22:20.410250+0000","last_fullsized":"2024-09-12T03:22:20.410250+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:17.387205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:17.387205+0000","last_clean_scrub_stamp":"2024-09-12T03:22:17.387205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:36:42.346845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T03:22:20.411557+0000","last_change":"2024-09-12T03:22:19.429146+0000","last_active":"2024-09-12T03:22:20.411557+0000","last_peered":"2024-09-12T03:22:20.411557+0000","last_clean":"2024-09-12T03:22:20.411557+0000","last_became_active":"2024-09-12T03:22:19.428834+0000","last_became_peered":"2024-09-12T03:22:19.428834+0000","last_unstale":"2024-09-12T03:22:20.411557+0000","last_undegraded":"2024-09-12T03:22:20.411557+0000","last_fullsized":"2024-09-12T03:22:20.411557+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_clean_scrub_stamp":"2024-09-12T03:22:18.392443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:56:19.883426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":14,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T03:22:20.430353+0000","last_change":"2024-09-12T03:22:18.410879+0000","last_active":"2024-09-12T03:22:20.430353+0000","last_peered":"2024-09-12T03:22:20.430353+0000","last_clean":"2024-09-12T03:22:20.430353+0000","last_became_active":"2024-09-12T03:22:18.410557+0000","last_became_peered":"2024-09-12T03:22:18.410557+0000","last_unstale":"2024-09-12T03:22:20.430353+0000","last_undegraded":"2024-09-12T03:22:20.430353+0000","last_fullsized":"2024-09-12T03:22:20.430353+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:17.387205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:17.387205+0000","last_clean_scrub_stamp":"2024-09-12T03:22:17.387205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:11:08.923180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T03:22:20.430287+0000","last_change":"2024-09-12T03:22:19.427002+0000","last_active":"2024-09-12T03:22:20.430287+0000","last_peered":"2024-09-12T03:22:20.430287+0000","last_clean":"2024-09-12T03:22:20.430287+0000","last_became_active":"2024-09-12T03:22:19.426679+0000","last_became_peered":"2024-09-12T03:22:19.426679+0000","last_unstale":"2024-09-12T03:22:20.430287+0000","last_undegraded":"2024-09-12T03:22:20.430287+0000","last_fullsized":"2024-09-12T03:22:20.430287+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_clean_scrub_stamp":"2024-09-12T03:22:18.392443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:14:17.658382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":14,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T03:22:20.566091+0000","last_change":"2024-09-12T03:22:18.413673+0000","last_active":"2024-09-12T03:22:20.566091+0000","last_peered":"2024-09-12T03:22:20.566091+0000","last_clean":"2024-09-12T03:22:20.566091+0000","last_became_active":"2024-09-12T03:22:18.413272+0000","last_became_peered":"2024-09-12T03:22:18.413272+0000","last_unstale":"2024-09-12T03:22:20.566091+0000","last_undegraded":"2024-09-12T03:22:20.566091+0000","last_fullsized":"2024-09-12T03:22:20.566091+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:17.387205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:17.387205+0000","last_clean_scrub_stamp":"2024-09-12T03:22:17.387205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:41:13.809653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T03:22:20.411898+0000","last_change":"2024-09-12T03:22:19.424919+0000","last_active":"2024-09-12T03:22:20.411898+0000","last_peered":"2024-09-12T03:22:20.411898+0000","last_clean":"2024-09-12T03:22:20.411898+0000","last_became_active":"2024-09-12T03:22:19.424539+0000","last_became_peered":"2024-09-12T03:22:19.424539+0000","last_unstale":"2024-09-12T03:22:20.411898+0000","last_undegraded":"2024-09-12T03:22:20.411898+0000","last_fullsized":"2024-09-12T03:22:20.411898+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_clean_scrub_stamp":"2024-09-12T03:22:18.392443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:06:28.637876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":14,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T03:22:20.565482+0000","last_change":"2024-09-12T03:22:18.412627+0000","last_active":"2024-09-12T03:22:20.565482+0000","last_peered":"2024-09-12T03:22:20.565482+0000","last_clean":"2024-09-12T03:22:20.565482+0000","last_became_active":"2024-09-12T03:22:18.412316+0000","last_became_peered":"2024-09-12T03:22:18.412316+0000","last_unstale":"2024-09-12T03:22:20.565482+0000","last_undegraded":"2024-09-12T03:22:20.565482+0000","last_fullsized":"2024-09-12T03:22:20.565482+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:17.387205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:17.387205+0000","last_clean_scrub_stamp":"2024-09-12T03:22:17.387205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:28:05.310409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.14","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T03:22:20.410363+0000","last_change":"2024-09-12T03:22:19.420036+0000","last_active":"2024-09-12T03:22:20.410363+0000","last_peered":"2024-09-12T03:22:20.410363+0000","last_clean":"2024-09-12T03:22:20.410363+0000","last_became_active":"2024-09-12T03:22:19.419765+0000","last_became_peered":"2024-09-12T03:22:19.419765+0000","last_unstale":"2024-09-12T03:22:20.410363+0000","last_undegraded":"2024-09-12T03:22:20.410363+0000","last_fullsized":"2024-09-12T03:22:20.410363+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:18.392443+0000","last_clean_scrub_stamp":"2024-09-12T03:22:18.392443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:43:27.847547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.15","version":"125'3","reported_seq":17,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T03:22:20.566574+0000","last_change":"2024-09-12T03:22:18.412881+0000","last_active":"2024-09-12T03:22:20.566574+0000","last_peered":"2024-09-12T03:22:20.566574+0000","last_clean":"2024-09-12T03:22:20.566574+0000","last_became_active":"2024-09-12T03:22:18.412605+0000","last_became_peered":"2024-09-12T03:22:18.412605+0000","last_unstale":"2024-09-12T03:22:20.566574+0000","last_undegraded":"2024-09-12T03:22:20.566574+0000","last_fullsized":"2024-09-12T03:22:20.566574+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:17.387205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:17.387205+0000","last_clean_scrub_stamp":"2024-09-12T03:22:17.387205+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:44:50.205161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"70'128","reported_seq":402,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T03:22:20.430346+0000","last_change":"2024-09-12T03:16:21.883446+0000","last_active":"2024-09-12T03:22:20.430346+0000","last_peered":"2024-09-12T03:22:20.430346+0000","last_clean":"2024-09-12T03:22:20.430346+0000","last_became_active":"2024-09-12T03:16:21.882207+0000","last_became_peered":"2024-09-12T03:16:21.882207+0000","last_unstale":"2024-09-12T03:22:20.430346+0000","last_undegraded":"2024-09-12T03:22:20.430346+0000","last_fullsized":"2024-09-12T03:22:20.430346+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:20.316426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:20.316426+0000","last_clean_scrub_stamp":"2024-09-12T03:16:20.316426+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:12:15.424320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":21,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":20,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542218,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37476,"kb_used_data":2128,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334364,"statfs":{"total":96636764160,"available":96598388736,"internally_reserved":0,"allocated":2179072,"data_stored":1288601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542220,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36896,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334944,"statfs":{"total":96636764160,"available":96598982656,"internally_reserved":0,"allocated":1576960,"data_stored":691371,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542219,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36892,"kb_used_data":1544,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334948,"statfs":{"total":96636764160,"available":96598986752,"internally_reserved":0,"allocated":1581056,"data_stored":697603,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542218,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36896,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334944,"statfs":{"total":96636764160,"available":96598982656,"internally_reserved":0,"allocated":1576960,"data_stored":697581,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542219,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32728,"kb_used_data":1532,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339112,"statfs":{"total":96636764160,"available":96603250688,"internally_reserved":0,"allocated":1568768,"data_stored":691315,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542219,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32732,"kb_used_data":1544,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339108,"statfs":{"total":96636764160,"available":96603246592,"internally_reserved":0,"allocated":1581056,"data_stored":692907,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542220,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32728,"kb_used_data":1532,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339112,"statfs":{"total":96636764160,"available":96603250688,"internally_reserved":0,"allocated":1568768,"data_stored":691807,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542220,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33332,"kb_used_data":2136,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338508,"statfs":{"total":96636764160,"available":96602632192,"internally_reserved":0,"allocated":2187264,"data_stored":1289595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T03:22:22.753 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-12T03:22:22.753 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-12T03:22:22.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.899+0000 7f1270742640 1 Processor -- start 2024-09-12T03:22:22.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.899+0000 7f1270742640 1 -- start start 2024-09-12T03:22:22.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.900+0000 7f1270742640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f1268070d30 0x7f126806cc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:22.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.900+0000 7f1270742640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f126806e960 0x7f12680d0440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:22.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.900+0000 7f1270742640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f12680cf9f0 con 0x7f12680d0980 2024-09-12T03:22:22.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.900+0000 7f1270742640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f12680d1a40 con 0x7f1268070d30 2024-09-12T03:22:22.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.900+0000 7f1270742640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f12680cfc90 con 0x7f126806e960 2024-09-12T03:22:22.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.900+0000 7f126dcdd640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f126806e960 0x7f12680d0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:22.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.900+0000 7f126e4de640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f1268070d30 0x7f126806cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:22.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.901+0000 7f126e4de640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f1268070d30 0x7f126806cc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.27:38002/0 (socket says 172.21.15.27:38002) 2024-09-12T03:22:22.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.901+0000 7f126e4de640 1 -- 172.21.15.27:0/837834618 learned_addr learned my addr 172.21.15.27:0/837834618 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:22:22.902 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.901+0000 7f126dcdd640 1 -- 172.21.15.27:0/837834618 >> v2:172.21.15.45:3300/0 conn(0x7f1268070d30 msgr2=0x7f126806cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:22.902 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.901+0000 7f126dcdd640 1 --2- 172.21.15.27:0/837834618 >> v2:172.21.15.45:3300/0 conn(0x7f1268070d30 0x7f126806cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:22.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.901+0000 7f126dcdd640 1 -- 172.21.15.27:0/837834618 >> v1:172.21.15.45:6789/0 conn(0x7f12680d0980 legacy=0x7f12680d21b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:22.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.901+0000 7f126dcdd640 1 -- 172.21.15.27:0/837834618 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12680cff70 con 0x7f126806e960 2024-09-12T03:22:22.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.901+0000 7f126e4de640 1 --2- 172.21.15.27:0/837834618 >> v2:172.21.15.45:3300/0 conn(0x7f1268070d30 0x7f126806cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:22:22.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.902+0000 7f126dcdd640 1 --2- 172.21.15.27:0/837834618 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f126806e960 0x7f12680d0440 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f12580097a0 tx=0x7f125802a060 comp rx=0 tx=0).ready entity=mon.2 client_cookie=32c99a80d85763af server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:22.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.902+0000 7f126d4dc640 1 -- 172.21.15.27:0/837834618 <== mon.2 v2:172.21.15.45:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1258037070 con 0x7f126806e960 2024-09-12T03:22:22.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.902+0000 7f126d4dc640 1 -- 172.21.15.27:0/837834618 <== mon.2 v2:172.21.15.45:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f125802adb0 con 0x7f126806e960 2024-09-12T03:22:22.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.902+0000 7f126d4dc640 1 -- 172.21.15.27:0/837834618 <== mon.2 v2:172.21.15.45:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1258032660 con 0x7f126806e960 2024-09-12T03:22:22.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.902+0000 7f1270742640 1 -- 172.21.15.27:0/837834618 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f126806e960 msgr2=0x7f12680d0440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:22.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.902+0000 7f1270742640 1 --2- 172.21.15.27:0/837834618 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f126806e960 0x7f12680d0440 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f12580097a0 tx=0x7f125802a060 comp rx=0 tx=0).stop 2024-09-12T03:22:22.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.902+0000 7f1270742640 1 -- 172.21.15.27:0/837834618 shutdown_connections 2024-09-12T03:22:22.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.902+0000 7f1270742640 1 --2- 172.21.15.27:0/837834618 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f126806e960 0x7f12680d0440 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:22.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.902+0000 7f1270742640 1 --2- 172.21.15.27:0/837834618 >> v2:172.21.15.45:3300/0 conn(0x7f1268070d30 0x7f126806cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:22.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.902+0000 7f1270742640 1 -- 172.21.15.27:0/837834618 >> 172.21.15.27:0/837834618 conn(0x7f12680b7c10 msgr2=0x7f126809ba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:22.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.902+0000 7f1270742640 1 -- 172.21.15.27:0/837834618 shutdown_connections 2024-09-12T03:22:22.904 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.902+0000 7f1270742640 1 -- 172.21.15.27:0/837834618 wait complete. 2024-09-12T03:22:22.906 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.904+0000 7f1270742640 1 Processor -- start 2024-09-12T03:22:22.906 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.904+0000 7f1270742640 1 -- start start 2024-09-12T03:22:22.906 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.904+0000 7f1270742640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f126806e960 0x7f126814c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:22.906 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.904+0000 7f1270742640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1268070d30 0x7f12680b3b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:22.906 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.904+0000 7f1270742640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f12680d0980 0x7f12680b40d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:22.906 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.904+0000 7f1270742640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f12680d28c0 con 0x7f126806e960 2024-09-12T03:22:22.906 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.904+0000 7f1270742640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f126806d180 con 0x7f12680d0980 2024-09-12T03:22:22.906 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.904+0000 7f1270742640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f12680d2a40 con 0x7f1268070d30 2024-09-12T03:22:22.906 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.904+0000 7f126e4de640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f126806e960 0x7f126814c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:22.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.904+0000 7f126e4de640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f126806e960 0x7f126814c0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49310/0 (socket says 172.21.15.27:49310) 2024-09-12T03:22:22.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.904+0000 7f126e4de640 1 -- 172.21.15.27:0/3703200829 learned_addr learned my addr 172.21.15.27:0/3703200829 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:22:22.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.905+0000 7f126dcdd640 1 --2- 172.21.15.27:0/3703200829 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1268070d30 0x7f12680b3b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:22.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.905+0000 7f126ecdf640 1 --2- 172.21.15.27:0/3703200829 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f12680d0980 0x7f12680b40d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:22.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.905+0000 7f126e4de640 1 -- 172.21.15.27:0/3703200829 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1268070d30 msgr2=0x7f12680b3b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:22.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.905+0000 7f126e4de640 1 --2- 172.21.15.27:0/3703200829 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1268070d30 0x7f12680b3b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:22.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.905+0000 7f126e4de640 1 -- 172.21.15.27:0/3703200829 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f12680d0980 msgr2=0x7f12680b40d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:22.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.905+0000 7f126e4de640 1 --2- 172.21.15.27:0/3703200829 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f12680d0980 0x7f12680b40d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:22.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.905+0000 7f126e4de640 1 -- 172.21.15.27:0/3703200829 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12680cde20 con 0x7f126806e960 2024-09-12T03:22:22.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.905+0000 7f126dcdd640 1 --2- 172.21.15.27:0/3703200829 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1268070d30 0x7f12680b3b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:22:22.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.905+0000 7f126ecdf640 1 --2- 172.21.15.27:0/3703200829 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f12680d0980 0x7f12680b40d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:22:22.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.905+0000 7f126e4de640 1 --2- 172.21.15.27:0/3703200829 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f126806e960 0x7f126814c0f0 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7f126400ea00 tx=0x7f126400eec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:22.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.905+0000 7f125effd640 1 -- 172.21.15.27:0/3703200829 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1264017070 con 0x7f126806e960 2024-09-12T03:22:22.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.905+0000 7f125effd640 1 -- 172.21.15.27:0/3703200829 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f126400ca60 con 0x7f126806e960 2024-09-12T03:22:22.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.905+0000 7f1270742640 1 -- 172.21.15.27:0/3703200829 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12680cbaa0 con 0x7f126806e960 2024-09-12T03:22:22.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.905+0000 7f125effd640 1 -- 172.21.15.27:0/3703200829 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f126400ce10 con 0x7f126806e960 2024-09-12T03:22:22.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.905+0000 7f1270742640 1 -- 172.21.15.27:0/3703200829 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12680cc060 con 0x7f126806e960 2024-09-12T03:22:22.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.906+0000 7f1270742640 1 -- 172.21.15.27:0/3703200829 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f126809ba60 con 0x7f126806e960 2024-09-12T03:22:22.909 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.907+0000 7f125effd640 1 -- 172.21.15.27:0/3703200829 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 21) ==== 91008+0+0 (secure 0 0 0) 0x7f126401a070 con 0x7f126806e960 2024-09-12T03:22:22.909 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.907+0000 7f125effd640 1 --2- 172.21.15.27:0/3703200829 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f124c06c3d0 0x7f124c06e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:22.909 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.908+0000 7f126dcdd640 1 --2- 172.21.15.27:0/3703200829 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f124c06c3d0 0x7f124c06e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:22.909 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.908+0000 7f125effd640 1 -- 172.21.15.27:0/3703200829 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 5765+0+0 (secure 0 0 0) 0x7f126408b8c0 con 0x7f126806e960 2024-09-12T03:22:22.909 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.908+0000 7f126dcdd640 1 --2- 172.21.15.27:0/3703200829 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f124c06c3d0 0x7f124c06e890 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f1258034000 tx=0x7f125802e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:22.912 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:22.912+0000 7f125effd640 1 -- 172.21.15.27:0/3703200829 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f126809ba60 con 0x7f126806e960 2024-09-12T03:22:23.151 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:23.152+0000 7f1270742640 1 -- 172.21.15.27:0/3703200829 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f12680d0440 con 0x7f126806e960 2024-09-12T03:22:23.153 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:23.154+0000 7f125effd640 1 -- 172.21.15.27:0/3703200829 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v20) ==== 190+0+147 (secure 0 0 0) 0x7f12640599c0 con 0x7f126806e960 2024-09-12T03:22:23.154 INFO:teuthology.orchestra.run.smithi027.stdout:[client.0] 2024-09-12T03:22:23.154 INFO:teuthology.orchestra.run.smithi027.stdout: key = AQDVXOJmLm+5ExAAECM4CKfuVElmp/n0b02mXg== 2024-09-12T03:22:23.154 INFO:teuthology.orchestra.run.smithi027.stdout: caps mds = "allow" 2024-09-12T03:22:23.154 INFO:teuthology.orchestra.run.smithi027.stdout: caps mon = "allow r" 2024-09-12T03:22:23.154 INFO:teuthology.orchestra.run.smithi027.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-12T03:22:23.154 INFO:teuthology.orchestra.run.smithi027.stderr:updated caps for client.0 2024-09-12T03:22:23.156 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:23.156+0000 7f1270742640 1 -- 172.21.15.27:0/3703200829 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f124c06c3d0 msgr2=0x7f124c06e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:23.157 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:23.156+0000 7f1270742640 1 --2- 172.21.15.27:0/3703200829 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f124c06c3d0 0x7f124c06e890 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f1258034000 tx=0x7f125802e000 comp rx=0 tx=0).stop 2024-09-12T03:22:23.157 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:23.156+0000 7f1270742640 1 -- 172.21.15.27:0/3703200829 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f126806e960 msgr2=0x7f126814c0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:23.157 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:23.156+0000 7f1270742640 1 --2- 172.21.15.27:0/3703200829 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f126806e960 0x7f126814c0f0 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7f126400ea00 tx=0x7f126400eec0 comp rx=0 tx=0).stop 2024-09-12T03:22:23.157 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:23.156+0000 7f1270742640 1 -- 172.21.15.27:0/3703200829 shutdown_connections 2024-09-12T03:22:23.157 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:23.156+0000 7f1270742640 1 --2- 172.21.15.27:0/3703200829 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f12680d0980 0x7f12680b40d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:23.157 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:23.156+0000 7f1270742640 1 --2- 172.21.15.27:0/3703200829 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1268070d30 0x7f12680b3b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:23.157 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:23.156+0000 7f1270742640 1 --2- 172.21.15.27:0/3703200829 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f124c06c3d0 0x7f124c06e890 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:23.157 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:23.156+0000 7f1270742640 1 --2- 172.21.15.27:0/3703200829 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f126806e960 0x7f126814c0f0 unknown :-1 s=CLOSED pgs=1247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:23.157 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:23.156+0000 7f1270742640 1 -- 172.21.15.27:0/3703200829 >> 172.21.15.27:0/3703200829 conn(0x7f12680b7c10 msgr2=0x7f12680b15f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:23.157 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:23.156+0000 7f1270742640 1 -- 172.21.15.27:0/3703200829 shutdown_connections 2024-09-12T03:22:23.157 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:23.156+0000 7f1270742640 1 -- 172.21.15.27:0/3703200829 wait complete. 2024-09-12T03:22:23.167 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-12T03:22:23.269 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:23.269+0000 7feea3c5e640 1 Processor -- start 2024-09-12T03:22:23.270 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:23.270+0000 7feea3c5e640 1 -- start start 2024-09-12T03:22:23.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:23.270+0000 7feea3c5e640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fee9c0cda40 0x7fee9c15ca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:23.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:23.270+0000 7feea3c5e640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fee9c146010 0x7fee9c0b4040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:23.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:23.270+0000 7feea3c5e640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fee9c0b4d70 con 0x7fee9c145c30 2024-09-12T03:22:23.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:23.271+0000 7feea3c5e640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7fee9c0b5050 con 0x7fee9c146010 2024-09-12T03:22:23.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:23.271+0000 7feea3c5e640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fee9c06d140 con 0x7fee9c0cda40 2024-09-12T03:22:23.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:23.271+0000 7feea21fb640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fee9c146010 0x7fee9c0b4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:23.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:23.271+0000 7feea19fa640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fee9c145c30 0x7fee9c0cd330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:42812/0 (socket says 172.21.15.27:42812) 2024-09-12T03:22:23.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:23.271+0000 7feea21fb640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fee9c146010 0x7fee9c0b4040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49326/0 (socket says 172.21.15.27:49326) 2024-09-12T03:22:23.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:23.271+0000 7feea19fa640 1 -- 172.21.15.27:0/3367218943 learned_addr learned my addr 172.21.15.27:0/3367218943 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:22:23.272 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:23.271+0000 7feea11f9640 1 --2- 172.21.15.27:0/3367218943 >> v2:172.21.15.45:3300/0 conn(0x7fee9c0cda40 0x7fee9c15ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:23.272 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:23.271+0000 7feea21fb640 1 -- 172.21.15.27:0/3367218943 >> v2:172.21.15.45:3300/0 conn(0x7fee9c0cda40 msgr2=0x7fee9c15ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:23.272 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:23.271+0000 7feea21fb640 1 --2- 172.21.15.27:0/3367218943 >> v2:172.21.15.45:3300/0 conn(0x7fee9c0cda40 0x7fee9c15ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:23.272 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:23.271+0000 7feea21fb640 1 -- 172.21.15.27:0/3367218943 >> v1:172.21.15.27:6789/0 conn(0x7fee9c145c30 legacy=0x7fee9c0cd330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:23.272 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:23.271+0000 7feea09f8640 1 -- 172.21.15.27:0/3367218943 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4037687683 0 0) 0x7fee9c0b4d70 con 0x7fee9c145c30 2024-09-12T03:22:23.272 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:23.271+0000 7feea21fb640 1 -- 172.21.15.27:0/3367218943 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee9c06d420 con 0x7fee9c146010 2024-09-12T03:22:23.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:23.272+0000 7feea21fb640 1 --2- 172.21.15.27:0/3367218943 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fee9c146010 0x7fee9c0b4040 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7fee9c06c6f0 tx=0x7fee9402a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b20e9a89714fcc14 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:23.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:23.272+0000 7feea09f8640 1 -- 172.21.15.27:0/3367218943 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee94007740 con 0x7fee9c146010 2024-09-12T03:22:23.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:23.272+0000 7feea09f8640 1 -- 172.21.15.27:0/3367218943 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee940078e0 con 0x7fee9c146010 2024-09-12T03:22:23.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:23.272+0000 7feea09f8640 1 -- 172.21.15.27:0/3367218943 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee94007cb0 con 0x7fee9c146010 2024-09-12T03:22:23.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:23.272+0000 7feea3c5e640 1 -- 172.21.15.27:0/3367218943 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fee9c146010 msgr2=0x7fee9c0b4040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:23.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:23.272+0000 7feea3c5e640 1 --2- 172.21.15.27:0/3367218943 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fee9c146010 0x7fee9c0b4040 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7fee9c06c6f0 tx=0x7fee9402a5f0 comp rx=0 tx=0).stop 2024-09-12T03:22:23.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:23.273+0000 7feea3c5e640 1 -- 172.21.15.27:0/3367218943 shutdown_connections 2024-09-12T03:22:23.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:23.273+0000 7feea3c5e640 1 --2- 172.21.15.27:0/3367218943 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fee9c146010 0x7fee9c0b4040 unknown :-1 s=CLOSED pgs=1249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:23.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:23.273+0000 7feea3c5e640 1 --2- 172.21.15.27:0/3367218943 >> v2:172.21.15.45:3300/0 conn(0x7fee9c0cda40 0x7fee9c15ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:23.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:23.273+0000 7feea3c5e640 1 -- 172.21.15.27:0/3367218943 >> 172.21.15.27:0/3367218943 conn(0x7fee9c062c60 msgr2=0x7fee9c06bb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:23.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:23.273+0000 7feea3c5e640 1 -- 172.21.15.27:0/3367218943 shutdown_connections 2024-09-12T03:22:23.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:23.273+0000 7feea3c5e640 1 -- 172.21.15.27:0/3367218943 wait complete. 2024-09-12T03:22:23.278 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:23.274+0000 7feea3c5e640 1 Processor -- start 2024-09-12T03:22:23.278 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:23.274+0000 7feea3c5e640 1 -- start start 2024-09-12T03:22:23.278 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:23.275+0000 7feea3c5e640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fee9c0cda40 0x7fee9c0c7540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:23.278 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:23.275+0000 7feea3c5e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fee9c145c30 0x7fee9c0c9560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:23.278 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:23.275+0000 7feea3c5e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fee9c146010 0x7fee9c140e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:23.278 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:23.275+0000 7feea3c5e640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fee9c0b4700 con 0x7fee9c146010 2024-09-12T03:22:23.278 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:23.275+0000 7feea3c5e640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fee9c0b4580 con 0x7fee9c145c30 2024-09-12T03:22:23.278 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:23.275+0000 7feea3c5e640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fee9c0b4880 con 0x7fee9c0cda40 2024-09-12T03:22:23.278 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:23.275+0000 7feea21fb640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fee9c146010 0x7fee9c140e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:23.278 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:23.275+0000 7feea21fb640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fee9c146010 0x7fee9c140e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49336/0 (socket says 172.21.15.27:49336) 2024-09-12T03:22:23.278 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:23.275+0000 7feea21fb640 1 -- 172.21.15.27:0/2276157126 learned_addr learned my addr 172.21.15.27:0/2276157126 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:22:23.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:23.275+0000 7feea11f9640 1 --2- 172.21.15.27:0/2276157126 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fee9c145c30 0x7fee9c0c9560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:23.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:23.275+0000 7feea19fa640 1 --2- 172.21.15.27:0/2276157126 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fee9c0cda40 0x7fee9c0c7540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:23.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:23.275+0000 7feea21fb640 1 -- 172.21.15.27:0/2276157126 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fee9c0cda40 msgr2=0x7fee9c0c7540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:23.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:23.275+0000 7feea21fb640 1 --2- 172.21.15.27:0/2276157126 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fee9c0cda40 0x7fee9c0c7540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:23.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:23.275+0000 7feea21fb640 1 -- 172.21.15.27:0/2276157126 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fee9c145c30 msgr2=0x7fee9c0c9560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:23.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:23.275+0000 7feea21fb640 1 --2- 172.21.15.27:0/2276157126 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fee9c145c30 0x7fee9c0c9560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:23.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:23.275+0000 7feea21fb640 1 -- 172.21.15.27:0/2276157126 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee9c141350 con 0x7fee9c146010 2024-09-12T03:22:23.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:23.275+0000 7feea21fb640 1 --2- 172.21.15.27:0/2276157126 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fee9c146010 0x7fee9c140e10 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7fee9402abe0 tx=0x7fee94004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:23.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:23.276+0000 7fee8e7fc640 1 -- 172.21.15.27:0/2276157126 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee9403f070 con 0x7fee9c146010 2024-09-12T03:22:23.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:23.276+0000 7fee8e7fc640 1 -- 172.21.15.27:0/2276157126 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee94031c40 con 0x7fee9c146010 2024-09-12T03:22:23.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:23.276+0000 7feea3c5e640 1 -- 172.21.15.27:0/2276157126 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee9c0c80e0 con 0x7fee9c146010 2024-09-12T03:22:23.280 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:23.276+0000 7fee8e7fc640 1 -- 172.21.15.27:0/2276157126 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee94009450 con 0x7fee9c146010 2024-09-12T03:22:23.280 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:23.276+0000 7feea3c5e640 1 -- 172.21.15.27:0/2276157126 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee9c0c7e00 con 0x7fee9c146010 2024-09-12T03:22:23.280 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:23.277+0000 7feea3c5e640 1 -- 172.21.15.27:0/2276157126 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee9c15ca00 con 0x7fee9c146010 2024-09-12T03:22:23.280 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:23.277+0000 7fee8e7fc640 1 -- 172.21.15.27:0/2276157126 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 21) ==== 91008+0+0 (secure 0 0 0) 0x7fee94003990 con 0x7fee9c146010 2024-09-12T03:22:23.280 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:23.278+0000 7fee8e7fc640 1 --2- 172.21.15.27:0/2276157126 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fee8006c3d0 0x7fee8006e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:23.280 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:23.278+0000 7fee8e7fc640 1 -- 172.21.15.27:0/2276157126 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 5765+0+0 (secure 0 0 0) 0x7fee940aca30 con 0x7fee9c146010 2024-09-12T03:22:23.280 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:23.278+0000 7feea19fa640 1 --2- 172.21.15.27:0/2276157126 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fee8006c3d0 0x7fee8006e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:23.281 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:23.278+0000 7feea19fa640 1 --2- 172.21.15.27:0/2276157126 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fee8006c3d0 0x7fee8006e890 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fee98000f60 tx=0x7fee98007d50 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:23.282 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:23.282+0000 7fee8e7fc640 1 -- 172.21.15.27:0/2276157126 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fee9c141350 con 0x7fee9c146010 2024-09-12T03:22:23.494 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:23.494+0000 7feea3c5e640 1 -- 172.21.15.27:0/2276157126 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fee9c075440 con 0x7fee9c146010 2024-09-12T03:22:23.494 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:23.494+0000 7fee8e7fc640 1 -- 172.21.15.27:0/2276157126 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 75 v75) ==== 94+0+5227 (secure 0 0 0) 0x7fee9407a3d0 con 0x7fee9c146010 2024-09-12T03:22:23.494 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:22:23.495 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":75,"btime":"2024-09-12T03:22:21:420722+0000","default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6437,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/2244842464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2244842464},{"type":"v1","addr":"172.21.15.45:6834","nonce":2244842464}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":72},{"gid":7048,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/1976058956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":1976058956},{"type":"v1","addr":"172.21.15.27:6835","nonce":1976058956}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":72},{"gid":7060,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/1084669125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":1084669125},{"type":"v1","addr":"172.21.15.45:6835","nonce":1084669125}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":72}],"filesystems":[{"mdsmap":{"epoch":75,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-12T03:22:18.755267+0000","modified":"2024-09-12T03:22:20.761064+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7725},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7725":{"gid":7725,"name":"c","rank":0,"incarnation":73,"state":"up:active","state_seq":3,"addr":"172.21.15.27:6834/1711260376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":1711260376},{"type":"v1","addr":"172.21.15.27:6834","nonce":1711260376}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7725,"qdb_cluster":[7725]},"id":10}]} 2024-09-12T03:22:23.495 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 75 2024-09-12T03:22:23.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:23.496+0000 7feea3c5e640 1 -- 172.21.15.27:0/2276157126 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fee8006c3d0 msgr2=0x7fee8006e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:23.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:23.496+0000 7feea3c5e640 1 --2- 172.21.15.27:0/2276157126 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fee8006c3d0 0x7fee8006e890 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fee98000f60 tx=0x7fee98007d50 comp rx=0 tx=0).stop 2024-09-12T03:22:23.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:23.496+0000 7feea3c5e640 1 -- 172.21.15.27:0/2276157126 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fee9c146010 msgr2=0x7fee9c140e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:23.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:23.496+0000 7feea3c5e640 1 --2- 172.21.15.27:0/2276157126 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fee9c146010 0x7fee9c140e10 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7fee9402abe0 tx=0x7fee94004a40 comp rx=0 tx=0).stop 2024-09-12T03:22:23.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:23.496+0000 7feea3c5e640 1 -- 172.21.15.27:0/2276157126 shutdown_connections 2024-09-12T03:22:23.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:23.496+0000 7feea3c5e640 1 --2- 172.21.15.27:0/2276157126 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fee8006c3d0 0x7fee8006e890 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:23.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:23.496+0000 7feea3c5e640 1 --2- 172.21.15.27:0/2276157126 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fee9c146010 0x7fee9c140e10 unknown :-1 s=CLOSED pgs=1250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:23.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:23.496+0000 7feea3c5e640 1 --2- 172.21.15.27:0/2276157126 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fee9c145c30 0x7fee9c0c9560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:23.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:23.496+0000 7feea3c5e640 1 --2- 172.21.15.27:0/2276157126 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fee9c0cda40 0x7fee9c0c7540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:23.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:23.496+0000 7feea3c5e640 1 -- 172.21.15.27:0/2276157126 >> 172.21.15.27:0/2276157126 conn(0x7fee9c062c60 msgr2=0x7fee9c0b19e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:23.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:23.496+0000 7feea3c5e640 1 -- 172.21.15.27:0/2276157126 shutdown_connections 2024-09-12T03:22:23.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:23.496+0000 7feea3c5e640 1 -- 172.21.15.27:0/2276157126 wait complete. 2024-09-12T03:22:23.909 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 75, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-12T03:22:18.755267+0000', 'modified': '2024-09-12T03:22:20.761064+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7725}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7725': {'gid': 7725, 'name': 'c', 'rank': 0, 'incarnation': 73, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.27:6834/1711260376', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6832', 'nonce': 1711260376}, {'type': 'v1', 'addr': '172.21.15.27:6834', 'nonce': 1711260376}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [21], 'metadata_pool': 20, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 7725, 'qdb_cluster': [7725]} 2024-09-12T03:22:23.910 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-12T03:22:23.910 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-12T03:22:24.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.017+0000 7f064219f640 1 Processor -- start 2024-09-12T03:22:24.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.017+0000 7f064219f640 1 -- start start 2024-09-12T03:22:24.018 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.018+0000 7f064219f640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f063c0d4090 0x7f063c0d21b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:24.018 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.018+0000 7f064219f640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f063c0d26f0 0x7f063c0d2ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:24.018 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.018+0000 7f064219f640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f063c0d3920 con 0x7f063c06c160 2024-09-12T03:22:24.018 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.018+0000 7f064219f640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f063c06cc50 con 0x7f063c0d4090 2024-09-12T03:22:24.019 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.018+0000 7f064219f640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f063c0d54d0 con 0x7f063c0d26f0 2024-09-12T03:22:24.019 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.019+0000 7f064199e640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f063c0d26f0 0x7f063c0d2ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:24.019 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.019+0000 7f064099c640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f063c0d4090 0x7f063c0d21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:24.019 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.019+0000 7f064119d640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f063c06c160 0x7f063c06c540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.27:35990/0 (socket says 172.21.15.27:35990) 2024-09-12T03:22:24.019 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.019+0000 7f064119d640 1 -- 172.21.15.27:0/4219398157 learned_addr learned my addr 172.21.15.27:0/4219398157 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:22:24.020 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.019+0000 7f064199e640 1 -- 172.21.15.27:0/4219398157 >> v2:172.21.15.45:3300/0 conn(0x7f063c0d4090 msgr2=0x7f063c0d21b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:24.020 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.019+0000 7f064199e640 1 --2- 172.21.15.27:0/4219398157 >> v2:172.21.15.45:3300/0 conn(0x7f063c0d4090 0x7f063c0d21b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:24.021 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.019+0000 7f064199e640 1 -- 172.21.15.27:0/4219398157 >> v1:172.21.15.45:6789/0 conn(0x7f063c06c160 legacy=0x7f063c06c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:24.021 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.019+0000 7f064199e640 1 -- 172.21.15.27:0/4219398157 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f063c0d3360 con 0x7f063c0d26f0 2024-09-12T03:22:24.021 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.019+0000 7f064099c640 1 --2- 172.21.15.27:0/4219398157 >> v2:172.21.15.45:3300/0 conn(0x7f063c0d4090 0x7f063c0d21b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:22:24.021 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.020+0000 7f064199e640 1 --2- 172.21.15.27:0/4219398157 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f063c0d26f0 0x7f063c0d2ad0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f06340097a0 tx=0x7f063402a060 comp rx=0 tx=0).ready entity=mon.2 client_cookie=67b7870b65299e75 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:24.021 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.020+0000 7f063bfff640 1 -- 172.21.15.27:0/4219398157 <== mon.2 v2:172.21.15.45:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0634037070 con 0x7f063c0d26f0 2024-09-12T03:22:24.021 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.020+0000 7f063bfff640 1 -- 172.21.15.27:0/4219398157 <== mon.2 v2:172.21.15.45:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f063402adb0 con 0x7f063c0d26f0 2024-09-12T03:22:24.021 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.020+0000 7f063bfff640 1 -- 172.21.15.27:0/4219398157 <== mon.2 v2:172.21.15.45:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0634032660 con 0x7f063c0d26f0 2024-09-12T03:22:24.021 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.020+0000 7f064219f640 1 -- 172.21.15.27:0/4219398157 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f063c0d26f0 msgr2=0x7f063c0d2ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:24.021 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.020+0000 7f064219f640 1 --2- 172.21.15.27:0/4219398157 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f063c0d26f0 0x7f063c0d2ad0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f06340097a0 tx=0x7f063402a060 comp rx=0 tx=0).stop 2024-09-12T03:22:24.021 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.020+0000 7f064219f640 1 -- 172.21.15.27:0/4219398157 shutdown_connections 2024-09-12T03:22:24.022 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.020+0000 7f064219f640 1 --2- 172.21.15.27:0/4219398157 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f063c0d26f0 0x7f063c0d2ad0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:24.022 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.020+0000 7f064219f640 1 --2- 172.21.15.27:0/4219398157 >> v2:172.21.15.45:3300/0 conn(0x7f063c0d4090 0x7f063c0d21b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:24.022 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.020+0000 7f064219f640 1 -- 172.21.15.27:0/4219398157 >> 172.21.15.27:0/4219398157 conn(0x7f063c0d45e0 msgr2=0x7f063c0d49f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:24.022 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.021+0000 7f064219f640 1 -- 172.21.15.27:0/4219398157 shutdown_connections 2024-09-12T03:22:24.022 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.021+0000 7f064219f640 1 -- 172.21.15.27:0/4219398157 wait complete. 2024-09-12T03:22:24.024 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.022+0000 7f064219f640 1 Processor -- start 2024-09-12T03:22:24.024 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.022+0000 7f064219f640 1 -- start start 2024-09-12T03:22:24.024 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.022+0000 7f064219f640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f063c06c160 0x7f063c0aebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:24.024 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.022+0000 7f064219f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f063c0d26f0 0x7f063c0ccc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:24.024 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.022+0000 7f064219f640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f063c0d4090 0x7f063c0cd180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:24.024 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.022+0000 7f064219f640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f063c0cf830 con 0x7f063c0d4090 2024-09-12T03:22:24.024 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.022+0000 7f064219f640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f063c0cf550 con 0x7f063c0d26f0 2024-09-12T03:22:24.024 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.022+0000 7f064219f640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f063c0d3010 con 0x7f063c06c160 2024-09-12T03:22:24.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.023+0000 7f064199e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f063c0d4090 0x7f063c0cd180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:24.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.023+0000 7f064199e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f063c0d4090 0x7f063c0cd180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49348/0 (socket says 172.21.15.27:49348) 2024-09-12T03:22:24.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.023+0000 7f064199e640 1 -- 172.21.15.27:0/1336375710 learned_addr learned my addr 172.21.15.27:0/1336375710 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:22:24.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.023+0000 7f064099c640 1 --2- 172.21.15.27:0/1336375710 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f063c0d26f0 0x7f063c0ccc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:24.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.023+0000 7f064119d640 1 --2- 172.21.15.27:0/1336375710 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f063c06c160 0x7f063c0aebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:24.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.023+0000 7f064199e640 1 -- 172.21.15.27:0/1336375710 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f063c06c160 msgr2=0x7f063c0aebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:24.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.023+0000 7f064199e640 1 --2- 172.21.15.27:0/1336375710 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f063c06c160 0x7f063c0aebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:24.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.023+0000 7f064199e640 1 -- 172.21.15.27:0/1336375710 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f063c0d26f0 msgr2=0x7f063c0ccc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:24.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.023+0000 7f064199e640 1 --2- 172.21.15.27:0/1336375710 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f063c0d26f0 0x7f063c0ccc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:24.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.023+0000 7f064199e640 1 -- 172.21.15.27:0/1336375710 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f063c1f46a0 con 0x7f063c0d4090 2024-09-12T03:22:24.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.023+0000 7f064199e640 1 --2- 172.21.15.27:0/1336375710 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f063c0d4090 0x7f063c0cd180 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f0634035040 tx=0x7f06340046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:24.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.023+0000 7f0639ffb640 1 -- 172.21.15.27:0/1336375710 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0634041070 con 0x7f063c0d4090 2024-09-12T03:22:24.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.023+0000 7f0639ffb640 1 -- 172.21.15.27:0/1336375710 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f063403c670 con 0x7f063c0d4090 2024-09-12T03:22:24.026 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.023+0000 7f0639ffb640 1 -- 172.21.15.27:0/1336375710 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0634037040 con 0x7f063c0d4090 2024-09-12T03:22:24.026 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.023+0000 7f064219f640 1 -- 172.21.15.27:0/1336375710 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f063c0cd7b0 con 0x7f063c0d4090 2024-09-12T03:22:24.026 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.024+0000 7f064219f640 1 -- 172.21.15.27:0/1336375710 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f063c0cdd70 con 0x7f063c0d4090 2024-09-12T03:22:24.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.026+0000 7f0639ffb640 1 -- 172.21.15.27:0/1336375710 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 21) ==== 91008+0+0 (secure 0 0 0) 0x7f063403a070 con 0x7f063c0d4090 2024-09-12T03:22:24.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.026+0000 7f0639ffb640 1 --2- 172.21.15.27:0/1336375710 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f062806c3d0 0x7f062806e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:24.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.026+0000 7f0639ffb640 1 -- 172.21.15.27:0/1336375710 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 5765+0+0 (secure 0 0 0) 0x7f06340abb00 con 0x7f063c0d4090 2024-09-12T03:22:24.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.026+0000 7f064119d640 1 --2- 172.21.15.27:0/1336375710 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f062806c3d0 0x7f062806e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:24.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.027+0000 7f064119d640 1 --2- 172.21.15.27:0/1336375710 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f062806c3d0 0x7f062806e890 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f062c0012e0 tx=0x7f062c001990 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:24.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.027+0000 7f0646275740 1 Processor -- start 2024-09-12T03:22:24.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.027+0000 7f0646275740 1 -- start start 2024-09-12T03:22:24.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.028+0000 7f0646275740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fdab4fe4b0 0x55fdab543c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:24.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.028+0000 7f0646275740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fdab4fbc70 0x55fdab4fe890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:24.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.028+0000 7f0646275740 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x55fdab4ffc40 0x55fdab4f8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:24.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.028+0000 7f0646275740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55fdab460520 con 0x55fdab4fe4b0 2024-09-12T03:22:24.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.028+0000 7f0646275740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55fdab4083e0 con 0x55fdab4fbc70 2024-09-12T03:22:24.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.028+0000 7f0646275740 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x55fdab45c4e0 con 0x55fdab4ffc40 2024-09-12T03:22:24.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.028+0000 7f064119d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fdab4fe4b0 0x55fdab543c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:24.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.028+0000 7f064119d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fdab4fe4b0 0x55fdab543c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49358/0 (socket says 172.21.15.27:49358) 2024-09-12T03:22:24.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.028+0000 7f064119d640 1 -- 172.21.15.27:0/3614608491 learned_addr learned my addr 172.21.15.27:0/3614608491 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:22:24.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.028+0000 7f064099c640 1 --2- 172.21.15.27:0/3614608491 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fdab4fbc70 0x55fdab4fe890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:24.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.028+0000 7f064119d640 1 -- 172.21.15.27:0/3614608491 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x55fdab4ffc40 msgr2=0x55fdab4f8aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:24.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.028+0000 7f064119d640 1 --2- 172.21.15.27:0/3614608491 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x55fdab4ffc40 0x55fdab4f8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:24.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.028+0000 7f064119d640 1 -- 172.21.15.27:0/3614608491 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fdab4fbc70 msgr2=0x55fdab4fe890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:24.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.028+0000 7f064119d640 1 --2- 172.21.15.27:0/3614608491 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fdab4fbc70 0x55fdab4fe890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:24.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.028+0000 7f064119d640 1 -- 172.21.15.27:0/3614608491 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fdab4232f0 con 0x55fdab4fe4b0 2024-09-12T03:22:24.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.028+0000 7f064099c640 1 --2- 172.21.15.27:0/3614608491 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fdab4fbc70 0x55fdab4fe890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:22:24.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.028+0000 7f064119d640 1 --2- 172.21.15.27:0/3614608491 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fdab4fe4b0 0x55fdab543c50 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7f062c013420 tx=0x7f062c013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=491fb4365c634cdc server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:24.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.028+0000 7f0622ffd640 1 -- 172.21.15.27:0/3614608491 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f062c026070 con 0x55fdab4fe4b0 2024-09-12T03:22:24.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.029+0000 7f0622ffd640 1 -- 172.21.15.27:0/3614608491 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f062c012560 con 0x55fdab4fe4b0 2024-09-12T03:22:24.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.029+0000 7f0622ffd640 1 -- 172.21.15.27:0/3614608491 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f062c012910 con 0x55fdab4fe4b0 2024-09-12T03:22:24.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.029+0000 7f0646275740 1 -- 172.21.15.27:0/3614608491 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fdab4fe4b0 msgr2=0x55fdab543c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:24.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.029+0000 7f0646275740 1 --2- 172.21.15.27:0/3614608491 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fdab4fe4b0 0x55fdab543c50 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7f062c013420 tx=0x7f062c013000 comp rx=0 tx=0).stop 2024-09-12T03:22:24.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.029+0000 7f0646275740 1 -- 172.21.15.27:0/3614608491 shutdown_connections 2024-09-12T03:22:24.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.029+0000 7f0646275740 1 --2- 172.21.15.27:0/3614608491 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x55fdab4ffc40 0x55fdab4f8aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:24.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.029+0000 7f0646275740 1 --2- 172.21.15.27:0/3614608491 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fdab4fbc70 0x55fdab4fe890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:24.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.029+0000 7f0646275740 1 --2- 172.21.15.27:0/3614608491 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fdab4fe4b0 0x55fdab543c50 unknown :-1 s=CLOSED pgs=1252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:24.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.029+0000 7f0646275740 1 -- 172.21.15.27:0/3614608491 >> 172.21.15.27:0/3614608491 conn(0x55fdab4fac70 msgr2=0x55fdab543120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:24.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.029+0000 7f0646275740 1 -- 172.21.15.27:0/3614608491 shutdown_connections 2024-09-12T03:22:24.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.029+0000 7f0646275740 1 -- 172.21.15.27:0/3614608491 wait complete. 2024-09-12T03:22:24.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.030+0000 7f0646275740 1 -- start start 2024-09-12T03:22:24.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.030+0000 7f0646275740 1 Processor -- start 2024-09-12T03:22:24.039 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.031+0000 7f0646275740 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x55fdab4fe4b0 0x55fdab53cf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:24.039 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.031+0000 7f0646275740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fdab4ffc40 0x55fdab524a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:24.039 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.031+0000 7f0646275740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fdab51f930 0x55fdab524f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:24.039 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.031+0000 7f0646275740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55fdab4fa110 con 0x55fdab4ffc40 2024-09-12T03:22:24.039 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.031+0000 7f0646275740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55fdab3e9450 con 0x55fdab51f930 2024-09-12T03:22:24.039 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.031+0000 7f0646275740 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x55fdab544480 con 0x55fdab4fe4b0 2024-09-12T03:22:24.039 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.031+0000 7f0646275740 10 client.0 _ll_register_callbacks cb 0x55fdab172ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:22:24.039 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.031+0000 7f064099c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fdab4ffc40 0x55fdab524a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:24.039 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.031+0000 7f064099c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fdab4ffc40 0x55fdab524a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49374/0 (socket says 172.21.15.27:49374) 2024-09-12T03:22:24.039 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.031+0000 7f064099c640 1 -- 172.21.15.27:0/4165575431 learned_addr learned my addr 172.21.15.27:0/4165575431 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:22:24.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.031+0000 7f064199e640 1 --2- 172.21.15.27:0/4165575431 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fdab51f930 0x55fdab524f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:24.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.031+0000 7f064119d640 1 --2- 172.21.15.27:0/4165575431 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x55fdab4fe4b0 0x55fdab53cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:24.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.031+0000 7f064099c640 1 -- 172.21.15.27:0/4165575431 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x55fdab4fe4b0 msgr2=0x55fdab53cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:24.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.031+0000 7f064099c640 1 --2- 172.21.15.27:0/4165575431 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x55fdab4fe4b0 0x55fdab53cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:24.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.031+0000 7f064099c640 1 -- 172.21.15.27:0/4165575431 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fdab51f930 msgr2=0x55fdab524f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:24.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.031+0000 7f064099c640 1 --2- 172.21.15.27:0/4165575431 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fdab51f930 0x55fdab524f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:24.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.031+0000 7f064099c640 1 -- 172.21.15.27:0/4165575431 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fdab1732d0 con 0x55fdab4ffc40 2024-09-12T03:22:24.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.031+0000 7f064199e640 1 --2- 172.21.15.27:0/4165575431 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fdab51f930 0x55fdab524f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:22:24.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.031+0000 7f064099c640 1 --2- 172.21.15.27:0/4165575431 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fdab4ffc40 0x55fdab524a00 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f0630013310 tx=0x7f06300137d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb21727d5544b1a1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:24.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.031+0000 7f0617fff640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-12T03:22:24.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.032+0000 7f0617fff640 1 -- 172.21.15.27:0/4165575431 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f063001b020 con 0x55fdab4ffc40 2024-09-12T03:22:24.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.032+0000 7f0617fff640 1 -- 172.21.15.27:0/4165575431 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(126..126 src has 1..126) ==== 5765+0+0 (secure 0 0 0) 0x7f0630004240 con 0x55fdab4ffc40 2024-09-12T03:22:24.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.032+0000 7f0646275740 1 -- 172.21.15.27:0/4165575431 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f062c012560 con 0x55fdab4ffc40 2024-09-12T03:22:24.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.032+0000 7f0617fff640 1 -- 172.21.15.27:0/4165575431 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=75) ==== 24+0+0 (secure 0 0 0) 0x7f0630025050 con 0x55fdab4ffc40 2024-09-12T03:22:24.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.032+0000 7f0646275740 10 client.7857 fetch_fsmap learned FSMap version 75 2024-09-12T03:22:24.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.032+0000 7f0646275740 1 -- 172.21.15.27:0/4165575431 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=75,monmap=2+}) -- 0x55fdab23d6b0 con 0x55fdab4ffc40 2024-09-12T03:22:24.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.032+0000 7f0617fff640 1 -- 172.21.15.27:0/4165575431 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f062c012560 con 0x55fdab4ffc40 2024-09-12T03:22:24.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.032+0000 7f0617fff640 1 -- 172.21.15.27:0/4165575431 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 75) ==== 3502+0+0 (secure 0 0 0) 0x7f0630028020 con 0x55fdab4ffc40 2024-09-12T03:22:24.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.032+0000 7f0646275740 10 client.7857 fetch_fsmap finished waiting for FSMap version 75 2024-09-12T03:22:24.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.032+0000 7f0646275740 10 client.7857 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-12T03:22:24.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.032+0000 7f0646275740 20 client.7857 populate_metadata read hostname 'smithi027' 2024-09-12T03:22:24.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.032+0000 7f0646275740 1 --2- 172.21.15.27:0/4165575431 >> [v2:172.21.15.27:6832/1711260376,v1:172.21.15.27:6834/1711260376] conn(0x55fdab51d9e0 0x55fdab51ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:24.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.032+0000 7f0646275740 4 client.7857 mds_command: new command op to 7725 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:22:24.043 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.032+0000 7f0646275740 1 -- 172.21.15.27:0/4165575431 --> [v2:172.21.15.27:6832/1711260376,v1:172.21.15.27:6834/1711260376] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fdab46dc50 con 0x55fdab51d9e0 2024-09-12T03:22:24.043 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.032+0000 7f064119d640 1 --2- 172.21.15.27:0/4165575431 >> [v2:172.21.15.27:6832/1711260376,v1:172.21.15.27:6834/1711260376] conn(0x55fdab51d9e0 0x55fdab51ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:24.043 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.032+0000 7f0617fff640 10 client.7857 ms_handle_connect on v2:172.21.15.27:6832/1711260376 2024-09-12T03:22:24.043 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.032+0000 7f064119d640 1 --2- 172.21.15.27:0/4165575431 >> [v2:172.21.15.27:6832/1711260376,v1:172.21.15.27:6834/1711260376] conn(0x55fdab51d9e0 0x55fdab51ddc0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=73b3a6e46056ea7a server_cookie=8506abe8dde7feb3 in_seq=0 out_seq=0 2024-09-12T03:22:24.043 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.033+0000 7f0617fff640 1 -- 172.21.15.27:0/4165575431 <== mds.0 v2:172.21.15.27:6832/1711260376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55fdab46dc50 con 0x55fdab51d9e0 2024-09-12T03:22:24.043 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.033+0000 7f0617fff640 10 client.7857 handle_command_reply: tid=0 2024-09-12T03:22:24.043 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.033+0000 7f0646275740 1 client.7857 shutdown 2024-09-12T03:22:24.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.033+0000 7f0646275740 1 -- 172.21.15.27:0/4165575431 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fdab4ffc40 msgr2=0x55fdab524a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:24.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.033+0000 7f0646275740 1 --2- 172.21.15.27:0/4165575431 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fdab4ffc40 0x55fdab524a00 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f0630013310 tx=0x7f06300137d0 comp rx=0 tx=0).stop 2024-09-12T03:22:24.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.033+0000 7f0646275740 1 -- 172.21.15.27:0/4165575431 shutdown_connections 2024-09-12T03:22:24.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.033+0000 7f0646275740 1 --2- 172.21.15.27:0/4165575431 >> [v2:172.21.15.27:6832/1711260376,v1:172.21.15.27:6834/1711260376] conn(0x55fdab51d9e0 0x55fdab51ddc0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:24.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.033+0000 7f0646275740 1 --2- 172.21.15.27:0/4165575431 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fdab51f930 0x55fdab524f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:24.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.033+0000 7f0646275740 1 --2- 172.21.15.27:0/4165575431 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fdab4ffc40 0x55fdab524a00 unknown :-1 s=CLOSED pgs=1253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:24.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.033+0000 7f0646275740 1 --2- 172.21.15.27:0/4165575431 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x55fdab4fe4b0 0x55fdab53cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:24.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.033+0000 7f0617fff640 0 client.7857 ms_handle_reset on v2:172.21.15.27:6832/1711260376 2024-09-12T03:22:24.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.033+0000 7f0646275740 1 -- 172.21.15.27:0/4165575431 >> 172.21.15.27:0/4165575431 conn(0x55fdab4fbc70 msgr2=0x55fdab5411a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:24.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.033+0000 7f0646275740 1 -- 172.21.15.27:0/4165575431 shutdown_connections 2024-09-12T03:22:24.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.034+0000 7f0646275740 1 -- 172.21.15.27:0/4165575431 wait complete. 2024-09-12T03:22:24.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.034+0000 7f0646275740 20 client.7857 trim_cache size 0 max 16384 2024-09-12T03:22:24.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.052+0000 7f0646275740 1 Processor -- start 2024-09-12T03:22:24.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.052+0000 7f0646275740 1 -- start start 2024-09-12T03:22:24.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.052+0000 7f0646275740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fdab4fe4b0 0x55fdab5433f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:24.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.052+0000 7f0646275740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fdab4ffc40 0x55fdab5bc6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:24.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.052+0000 7f0646275740 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x55fdab51f930 0x55fdab5bcbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:24.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.052+0000 7f0646275740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f063c1ffc90 con 0x55fdab4ffc40 2024-09-12T03:22:24.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.052+0000 7f0646275740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f063c1f8760 con 0x55fdab4fe4b0 2024-09-12T03:22:24.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.052+0000 7f0646275740 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f063c0acd90 con 0x55fdab51f930 2024-09-12T03:22:24.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.052+0000 7f064099c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fdab4ffc40 0x55fdab5bc6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:24.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.052+0000 7f064099c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fdab4ffc40 0x55fdab5bc6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49390/0 (socket says 172.21.15.27:49390) 2024-09-12T03:22:24.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.052+0000 7f064099c640 1 -- 172.21.15.27:0/1387377771 learned_addr learned my addr 172.21.15.27:0/1387377771 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:22:24.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.052+0000 7f064119d640 1 --2- 172.21.15.27:0/1387377771 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fdab4fe4b0 0x55fdab5433f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:24.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.053+0000 7f064199e640 1 --2- 172.21.15.27:0/1387377771 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x55fdab51f930 0x55fdab5bcbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:24.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.053+0000 7f064099c640 1 -- 172.21.15.27:0/1387377771 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x55fdab51f930 msgr2=0x55fdab5bcbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:24.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.053+0000 7f064099c640 1 --2- 172.21.15.27:0/1387377771 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x55fdab51f930 0x55fdab5bcbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:24.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.053+0000 7f064099c640 1 -- 172.21.15.27:0/1387377771 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fdab4fe4b0 msgr2=0x55fdab5433f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:24.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.053+0000 7f064099c640 1 --2- 172.21.15.27:0/1387377771 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fdab4fe4b0 0x55fdab5433f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:24.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.053+0000 7f064099c640 1 -- 172.21.15.27:0/1387377771 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fdab51f760 con 0x55fdab4ffc40 2024-09-12T03:22:24.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.053+0000 7f064099c640 1 --2- 172.21.15.27:0/1387377771 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fdab4ffc40 0x55fdab5bc6a0 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f0630000c00 tx=0x7f0630018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f2dc8c1d0f8a5e5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:24.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.053+0000 7f06227fc640 1 -- 172.21.15.27:0/1387377771 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f063002e070 con 0x55fdab4ffc40 2024-09-12T03:22:24.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.053+0000 7f06227fc640 1 -- 172.21.15.27:0/1387377771 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0630003960 con 0x55fdab4ffc40 2024-09-12T03:22:24.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.053+0000 7f06227fc640 1 -- 172.21.15.27:0/1387377771 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f063001b040 con 0x55fdab4ffc40 2024-09-12T03:22:24.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.054+0000 7f0646275740 1 -- 172.21.15.27:0/1387377771 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fdab4ffc40 msgr2=0x55fdab5bc6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:24.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.054+0000 7f0646275740 1 --2- 172.21.15.27:0/1387377771 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fdab4ffc40 0x55fdab5bc6a0 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f0630000c00 tx=0x7f0630018000 comp rx=0 tx=0).stop 2024-09-12T03:22:24.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.054+0000 7f0646275740 1 -- 172.21.15.27:0/1387377771 shutdown_connections 2024-09-12T03:22:24.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.054+0000 7f0646275740 1 --2- 172.21.15.27:0/1387377771 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x55fdab51f930 0x55fdab5bcbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:24.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.054+0000 7f0646275740 1 --2- 172.21.15.27:0/1387377771 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fdab4ffc40 0x55fdab5bc6a0 unknown :-1 s=CLOSED pgs=1254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:24.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.054+0000 7f0646275740 1 --2- 172.21.15.27:0/1387377771 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fdab4fe4b0 0x55fdab5433f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:24.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.054+0000 7f0646275740 1 -- 172.21.15.27:0/1387377771 >> 172.21.15.27:0/1387377771 conn(0x55fdab4fbc70 msgr2=0x55fdab51ddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:24.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.054+0000 7f0646275740 1 -- 172.21.15.27:0/1387377771 shutdown_connections 2024-09-12T03:22:24.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.054+0000 7f0646275740 1 -- 172.21.15.27:0/1387377771 wait complete. 2024-09-12T03:22:24.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.054+0000 7f0646275740 1 -- start start 2024-09-12T03:22:24.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.054+0000 7f0646275740 1 Processor -- start 2024-09-12T03:22:24.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.055+0000 7f0646275740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fdab4fe4b0 0x55fdab5aa1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:24.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.055+0000 7f0646275740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fdab4ffc40 0x55fdab51ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:24.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.055+0000 7f0646275740 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x55fdab51f930 0x55fdab51e300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:24.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.055+0000 7f0646275740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55fdab53d440 con 0x55fdab4ffc40 2024-09-12T03:22:24.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.055+0000 7f0646275740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55fdab528a60 con 0x55fdab4fe4b0 2024-09-12T03:22:24.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.055+0000 7f0646275740 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x55fdab4f7e60 con 0x55fdab51f930 2024-09-12T03:22:24.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.055+0000 7f0646275740 10 client.0 _ll_register_callbacks cb 0x7f0630009400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:22:24.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.055+0000 7f064099c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fdab4ffc40 0x55fdab51ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:24.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.055+0000 7f064099c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fdab4ffc40 0x55fdab51ddc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49404/0 (socket says 172.21.15.27:49404) 2024-09-12T03:22:24.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.055+0000 7f064099c640 1 -- 172.21.15.27:0/180834264 learned_addr learned my addr 172.21.15.27:0/180834264 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:22:24.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.055+0000 7f064119d640 1 --2- 172.21.15.27:0/180834264 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fdab4fe4b0 0x55fdab5aa1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:24.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.055+0000 7f064199e640 1 --2- 172.21.15.27:0/180834264 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x55fdab51f930 0x55fdab51e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:24.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.056+0000 7f064099c640 1 -- 172.21.15.27:0/180834264 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x55fdab51f930 msgr2=0x55fdab51e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:24.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.056+0000 7f064099c640 1 --2- 172.21.15.27:0/180834264 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x55fdab51f930 0x55fdab51e300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:24.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.056+0000 7f064099c640 1 -- 172.21.15.27:0/180834264 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fdab4fe4b0 msgr2=0x55fdab5aa1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:24.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.056+0000 7f064099c640 1 --2- 172.21.15.27:0/180834264 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fdab4fe4b0 0x55fdab5aa1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:24.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.056+0000 7f064099c640 1 -- 172.21.15.27:0/180834264 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fdab5253d0 con 0x55fdab4ffc40 2024-09-12T03:22:24.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.056+0000 7f064099c640 1 --2- 172.21.15.27:0/180834264 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fdab4ffc40 0x55fdab51ddc0 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f0630004500 tx=0x7f06300046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4eaa0283bc6f5bcc server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:24.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.056+0000 7f0617fff640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-12T03:22:24.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.056+0000 7f0617fff640 1 -- 172.21.15.27:0/180834264 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f063001b070 con 0x55fdab4ffc40 2024-09-12T03:22:24.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.056+0000 7f0617fff640 1 -- 172.21.15.27:0/180834264 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(126..126 src has 1..126) ==== 5765+0+0 (secure 0 0 0) 0x7f0630034b30 con 0x55fdab4ffc40 2024-09-12T03:22:24.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.056+0000 7f0646275740 1 -- 172.21.15.27:0/180834264 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0630003960 con 0x55fdab4ffc40 2024-09-12T03:22:24.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.056+0000 7f0617fff640 1 -- 172.21.15.27:0/180834264 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=75) ==== 24+0+0 (secure 0 0 0) 0x7f063003b050 con 0x55fdab4ffc40 2024-09-12T03:22:24.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.056+0000 7f0646275740 10 client.7863 fetch_fsmap learned FSMap version 75 2024-09-12T03:22:24.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.056+0000 7f0646275740 1 -- 172.21.15.27:0/180834264 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=75,monmap=2+}) -- 0x55fdab5aa780 con 0x55fdab4ffc40 2024-09-12T03:22:24.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.056+0000 7f0617fff640 1 -- 172.21.15.27:0/180834264 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0630003960 con 0x55fdab4ffc40 2024-09-12T03:22:24.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.056+0000 7f0617fff640 1 -- 172.21.15.27:0/180834264 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 75) ==== 3502+0+0 (secure 0 0 0) 0x7f063003e020 con 0x55fdab4ffc40 2024-09-12T03:22:24.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.057+0000 7f0646275740 10 client.7863 fetch_fsmap finished waiting for FSMap version 75 2024-09-12T03:22:24.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.057+0000 7f0646275740 10 client.7863 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-12T03:22:24.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.057+0000 7f0646275740 20 client.7863 populate_metadata read hostname 'smithi027' 2024-09-12T03:22:24.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.057+0000 7f0646275740 1 --2- 172.21.15.27:0/180834264 >> [v2:172.21.15.27:6832/1711260376,v1:172.21.15.27:6834/1711260376] conn(0x55fdab552f80 0x55fdab5232d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:24.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.057+0000 7f0646275740 4 client.7863 mds_command: new command op to 7725 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-12T03:22:24.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.057+0000 7f0646275740 1 -- 172.21.15.27:0/180834264 --> [v2:172.21.15.27:6832/1711260376,v1:172.21.15.27:6834/1711260376] -- command(tid 0: {"prefix": "status"}) -- 0x55fdab467fa0 con 0x55fdab552f80 2024-09-12T03:22:24.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.057+0000 7f064119d640 1 --2- 172.21.15.27:0/180834264 >> [v2:172.21.15.27:6832/1711260376,v1:172.21.15.27:6834/1711260376] conn(0x55fdab552f80 0x55fdab5232d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:24.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.057+0000 7f064119d640 1 --2- 172.21.15.27:0/180834264 >> [v2:172.21.15.27:6832/1711260376,v1:172.21.15.27:6834/1711260376] conn(0x55fdab552f80 0x55fdab5232d0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d00b6bd07b7c41a9 server_cookie=8622f29aa8428bf in_seq=0 out_seq=0 2024-09-12T03:22:24.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.057+0000 7f0617fff640 10 client.7863 ms_handle_connect on v2:172.21.15.27:6832/1711260376 2024-09-12T03:22:24.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.057+0000 7f0617fff640 1 -- 172.21.15.27:0/180834264 <== mds.0 v2:172.21.15.27:6832/1711260376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+333 (crc 0 0 0) 0x55fdab467fa0 con 0x55fdab552f80 2024-09-12T03:22:24.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.057+0000 7f0617fff640 10 client.7863 handle_command_reply: tid=0 2024-09-12T03:22:24.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.057+0000 7f0646275740 1 client.7863 shutdown 2024-09-12T03:22:24.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.057+0000 7f0646275740 1 -- 172.21.15.27:0/180834264 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fdab4ffc40 msgr2=0x55fdab51ddc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:24.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.057+0000 7f0646275740 1 --2- 172.21.15.27:0/180834264 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fdab4ffc40 0x55fdab51ddc0 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f0630004500 tx=0x7f06300046c0 comp rx=0 tx=0).stop 2024-09-12T03:22:24.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.057+0000 7f0646275740 1 -- 172.21.15.27:0/180834264 shutdown_connections 2024-09-12T03:22:24.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.057+0000 7f0646275740 1 --2- 172.21.15.27:0/180834264 >> [v2:172.21.15.27:6832/1711260376,v1:172.21.15.27:6834/1711260376] conn(0x55fdab552f80 0x55fdab5232d0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:24.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.057+0000 7f0646275740 1 --2- 172.21.15.27:0/180834264 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x55fdab51f930 0x55fdab51e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:24.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.057+0000 7f0617fff640 0 client.7863 ms_handle_reset on v2:172.21.15.27:6832/1711260376 2024-09-12T03:22:24.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.057+0000 7f0646275740 1 --2- 172.21.15.27:0/180834264 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fdab4ffc40 0x55fdab51ddc0 unknown :-1 s=CLOSED pgs=1255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:24.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.057+0000 7f0646275740 1 --2- 172.21.15.27:0/180834264 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fdab4fe4b0 0x55fdab5aa1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:24.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.057+0000 7f0646275740 1 -- 172.21.15.27:0/180834264 >> 172.21.15.27:0/180834264 conn(0x55fdab4fbc70 msgr2=0x55fdab543480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:24.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.057+0000 7f0646275740 1 -- 172.21.15.27:0/180834264 shutdown_connections 2024-09-12T03:22:24.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.057+0000 7f0646275740 1 -- 172.21.15.27:0/180834264 wait complete. 2024-09-12T03:22:24.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.058+0000 7f0646275740 20 client.7863 trim_cache size 0 max 16384 2024-09-12T03:22:24.068 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-12T03:22:24.068 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_fsid": "d2df504e-2647-4385-a5db-9fdf69f487a0", 2024-09-12T03:22:24.068 INFO:teuthology.orchestra.run.smithi027.stdout: "whoami": 0, 2024-09-12T03:22:24.068 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 7725, 2024-09-12T03:22:24.068 INFO:teuthology.orchestra.run.smithi027.stdout: "want_state": "up:active", 2024-09-12T03:22:24.068 INFO:teuthology.orchestra.run.smithi027.stdout: "state": "up:active", 2024-09-12T03:22:24.069 INFO:teuthology.orchestra.run.smithi027.stdout: "fs_name": "cephfs", 2024-09-12T03:22:24.069 INFO:teuthology.orchestra.run.smithi027.stdout: "rank_uptime": 4.6416840290000003, 2024-09-12T03:22:24.069 INFO:teuthology.orchestra.run.smithi027.stdout: "mdsmap_epoch": 75, 2024-09-12T03:22:24.069 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch": 126, 2024-09-12T03:22:24.069 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch_barrier": 126, 2024-09-12T03:22:24.069 INFO:teuthology.orchestra.run.smithi027.stdout: "uptime": 9.6736650700000002 2024-09-12T03:22:24.069 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-12T03:22:24.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.058+0000 7f064219f640 1 -- 172.21.15.27:0/1336375710 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f062806c3d0 msgr2=0x7f062806e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:24.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.058+0000 7f064219f640 1 --2- 172.21.15.27:0/1336375710 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f062806c3d0 0x7f062806e890 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f062c0012e0 tx=0x7f062c001990 comp rx=0 tx=0).stop 2024-09-12T03:22:24.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.058+0000 7f064219f640 1 -- 172.21.15.27:0/1336375710 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f063c0d4090 msgr2=0x7f063c0cd180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:24.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.058+0000 7f064219f640 1 --2- 172.21.15.27:0/1336375710 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f063c0d4090 0x7f063c0cd180 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f0634035040 tx=0x7f06340046c0 comp rx=0 tx=0).stop 2024-09-12T03:22:24.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.058+0000 7f064219f640 1 -- 172.21.15.27:0/1336375710 shutdown_connections 2024-09-12T03:22:24.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.058+0000 7f064219f640 1 --2- 172.21.15.27:0/1336375710 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f062806c3d0 0x7f062806e890 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:24.072 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.058+0000 7f064219f640 1 --2- 172.21.15.27:0/1336375710 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f063c0d4090 0x7f063c0cd180 unknown :-1 s=CLOSED pgs=1251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:24.072 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.058+0000 7f064219f640 1 --2- 172.21.15.27:0/1336375710 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f063c0d26f0 0x7f063c0ccc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:24.072 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.058+0000 7f064219f640 1 --2- 172.21.15.27:0/1336375710 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f063c06c160 0x7f063c0aebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:24.072 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.058+0000 7f064219f640 1 -- 172.21.15.27:0/1336375710 >> 172.21.15.27:0/1336375710 conn(0x7f063c0d45e0 msgr2=0x7f063c0b9e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:24.072 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.058+0000 7f064219f640 1 -- 172.21.15.27:0/1336375710 shutdown_connections 2024-09-12T03:22:24.072 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.058+0000 7f064219f640 1 -- 172.21.15.27:0/1336375710 wait complete. 2024-09-12T03:22:24.072 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-12T03:22:24.175 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.175+0000 7f32de318640 1 Processor -- start 2024-09-12T03:22:24.176 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.175+0000 7f32de318640 1 -- start start 2024-09-12T03:22:24.183 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.176+0000 7f32de318640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f32d815e5f0 0x7f32d8073250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:24.184 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.176+0000 7f32de318640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f32d8162a30 0x7f32d815e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:24.184 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.176+0000 7f32de318640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f32d80d2240 con 0x7f32d8073790 2024-09-12T03:22:24.184 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.176+0000 7f32de318640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f32d8073be0 con 0x7f32d815e5f0 2024-09-12T03:22:24.184 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.176+0000 7f32de318640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f32d80d0da0 con 0x7f32d8162a30 2024-09-12T03:22:24.184 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.176+0000 7f32d77fe640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f32d8073790 0x7f32d8162320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:42828/0 (socket says 172.21.15.27:42828) 2024-09-12T03:22:24.184 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.176+0000 7f32d77fe640 1 -- 172.21.15.27:0/1876735991 learned_addr learned my addr 172.21.15.27:0/1876735991 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:22:24.184 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.176+0000 7f32d7fff640 1 --2- 172.21.15.27:0/1876735991 >> v2:172.21.15.45:3300/0 conn(0x7f32d815e5f0 0x7f32d8073250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:24.184 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.177+0000 7f32dc8b5640 1 --2- 172.21.15.27:0/1876735991 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f32d8162a30 0x7f32d815e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:24.184 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.177+0000 7f32d6ffd640 1 -- 172.21.15.27:0/1876735991 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 552766475 0 0) 0x7f32d80d2240 con 0x7f32d8073790 2024-09-12T03:22:24.185 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.177+0000 7f32d6ffd640 1 -- 172.21.15.27:0/1876735991 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f32c0003520 con 0x7f32d8073790 2024-09-12T03:22:24.185 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.177+0000 7f32d6ffd640 1 -- 172.21.15.27:0/1876735991 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1944965159 0 0) 0x7f32c0003520 con 0x7f32d8073790 2024-09-12T03:22:24.185 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.177+0000 7f32d6ffd640 1 -- 172.21.15.27:0/1876735991 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f32d8162a30 msgr2=0x7f32d815e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:24.185 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.177+0000 7f32d6ffd640 1 --2- 172.21.15.27:0/1876735991 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f32d8162a30 0x7f32d815e9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:24.185 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.177+0000 7f32d6ffd640 1 -- 172.21.15.27:0/1876735991 >> v2:172.21.15.45:3300/0 conn(0x7f32d815e5f0 msgr2=0x7f32d8073250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:24.185 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.177+0000 7f32d6ffd640 1 --2- 172.21.15.27:0/1876735991 >> v2:172.21.15.45:3300/0 conn(0x7f32d815e5f0 0x7f32d8073250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:24.185 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.177+0000 7f32d6ffd640 1 -- 172.21.15.27:0/1876735991 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32d8163090 con 0x7f32d8073790 2024-09-12T03:22:24.185 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.177+0000 7f32d6ffd640 1 -- 172.21.15.27:0/1876735991 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 847755213 0 0) 0x7f32cc001670 con 0x7f32d8073790 2024-09-12T03:22:24.185 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.177+0000 7f32d6ffd640 1 -- 172.21.15.27:0/1876735991 >> v1:172.21.15.27:6789/0 conn(0x7f32d8073790 legacy=0x7f32d8162320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:24.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.177+0000 7f32d6ffd640 1 --2- 172.21.15.27:0/1876735991 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f32c00039b0 0x7f32c0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:24.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.177+0000 7f32d6ffd640 1 --2- 172.21.15.27:0/1876735991 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f32c0004800 0x7f32c0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:24.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.177+0000 7f32d6ffd640 1 --2- 172.21.15.27:0/1876735991 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f32d8162a30 0x7f32c00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:24.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.177+0000 7f32d6ffd640 1 -- 172.21.15.27:0/1876735991 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f32d8073be0 con 0x7f32c00039b0 2024-09-12T03:22:24.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.177+0000 7f32d6ffd640 1 -- 172.21.15.27:0/1876735991 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f32d80d0da0 con 0x7f32c0004800 2024-09-12T03:22:24.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.177+0000 7f32d6ffd640 1 -- 172.21.15.27:0/1876735991 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f32c000b9c0 con 0x7f32d8162a30 2024-09-12T03:22:24.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.177+0000 7f32d7fff640 1 --2- 172.21.15.27:0/1876735991 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f32c00039b0 0x7f32c0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:24.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.177+0000 7f32dc8b5640 1 --2- 172.21.15.27:0/1876735991 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f32d8162a30 0x7f32c00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:24.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.177+0000 7f32d77fe640 1 --2- 172.21.15.27:0/1876735991 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f32c0004800 0x7f32c0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:24.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.177+0000 7f32d7fff640 1 -- 172.21.15.27:0/1876735991 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f32d8162a30 msgr2=0x7f32c00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:24.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.177+0000 7f32d7fff640 1 --2- 172.21.15.27:0/1876735991 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f32d8162a30 0x7f32c00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:24.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.177+0000 7f32d7fff640 1 -- 172.21.15.27:0/1876735991 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f32c0004800 msgr2=0x7f32c0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:24.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.177+0000 7f32d7fff640 1 --2- 172.21.15.27:0/1876735991 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f32c0004800 0x7f32c0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:24.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.177+0000 7f32d7fff640 1 -- 172.21.15.27:0/1876735991 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32c0003520 con 0x7f32c00039b0 2024-09-12T03:22:24.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.177+0000 7f32d77fe640 1 --2- 172.21.15.27:0/1876735991 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f32c0004800 0x7f32c0004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:22:24.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.177+0000 7f32dc8b5640 1 --2- 172.21.15.27:0/1876735991 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f32d8162a30 0x7f32c00051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:22:24.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.178+0000 7f32d7fff640 1 --2- 172.21.15.27:0/1876735991 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f32c00039b0 0x7f32c0003dd0 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f32c8009ac0 tx=0x7f32c0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f2e19411a424042 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:24.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.178+0000 7f32d6ffd640 1 -- 172.21.15.27:0/1876735991 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32c8036070 con 0x7f32c00039b0 2024-09-12T03:22:24.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.178+0000 7f32d6ffd640 1 -- 172.21.15.27:0/1876735991 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32c8031400 con 0x7f32c00039b0 2024-09-12T03:22:24.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.178+0000 7f32d6ffd640 1 -- 172.21.15.27:0/1876735991 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32c8009690 con 0x7f32c00039b0 2024-09-12T03:22:24.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.179+0000 7f32de318640 1 -- 172.21.15.27:0/1876735991 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f32c00039b0 msgr2=0x7f32c0003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:24.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.179+0000 7f32de318640 1 --2- 172.21.15.27:0/1876735991 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f32c00039b0 0x7f32c0003dd0 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f32c8009ac0 tx=0x7f32c0005830 comp rx=0 tx=0).stop 2024-09-12T03:22:24.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.179+0000 7f32dc8b5640 1 -- 172.21.15.27:0/1876735991 reap_dead start 2024-09-12T03:22:24.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.179+0000 7f32de318640 1 -- 172.21.15.27:0/1876735991 shutdown_connections 2024-09-12T03:22:24.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.179+0000 7f32de318640 1 -- 172.21.15.27:0/1876735991 >> 172.21.15.27:0/1876735991 conn(0x7f32d806f300 msgr2=0x7f32d806f710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:24.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.179+0000 7f32de318640 1 -- 172.21.15.27:0/1876735991 shutdown_connections 2024-09-12T03:22:24.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.179+0000 7f32de318640 1 -- 172.21.15.27:0/1876735991 wait complete. 2024-09-12T03:22:24.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.180+0000 7f32de318640 1 Processor -- start 2024-09-12T03:22:24.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.180+0000 7f32de318640 1 -- start start 2024-09-12T03:22:24.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.181+0000 7f32de318640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f32c00039b0 0x7f32d8158d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:24.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.181+0000 7f32de318640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f32d80cd710 0x7f32d80cdb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:24.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.181+0000 7f32de318640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f32d80b6e00 0x7f32d80b7280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:24.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.181+0000 7f32de318640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f32d80d0a30 con 0x7f32d80b6e00 2024-09-12T03:22:24.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.181+0000 7f32de318640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f32d80d0750 con 0x7f32c00039b0 2024-09-12T03:22:24.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.181+0000 7f32de318640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f32d80b5620 con 0x7f32d80cd710 2024-09-12T03:22:24.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.181+0000 7f32dc8b5640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f32d80b6e00 0x7f32d80b7280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:24.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.181+0000 7f32dc8b5640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f32d80b6e00 0x7f32d80b7280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49432/0 (socket says 172.21.15.27:49432) 2024-09-12T03:22:24.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.181+0000 7f32dc8b5640 1 -- 172.21.15.27:0/3909472821 learned_addr learned my addr 172.21.15.27:0/3909472821 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:22:24.190 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.181+0000 7f32d7fff640 1 --2- 172.21.15.27:0/3909472821 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f32c00039b0 0x7f32d8158d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:24.190 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.181+0000 7f32d77fe640 1 --2- 172.21.15.27:0/3909472821 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f32d80cd710 0x7f32d80cdb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:24.190 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.181+0000 7f32dc8b5640 1 -- 172.21.15.27:0/3909472821 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f32d80cd710 msgr2=0x7f32d80cdb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:24.190 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.181+0000 7f32dc8b5640 1 --2- 172.21.15.27:0/3909472821 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f32d80cd710 0x7f32d80cdb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:24.190 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.181+0000 7f32dc8b5640 1 -- 172.21.15.27:0/3909472821 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f32c00039b0 msgr2=0x7f32d8158d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:24.190 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.181+0000 7f32dc8b5640 1 --2- 172.21.15.27:0/3909472821 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f32c00039b0 0x7f32d8158d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:24.190 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.181+0000 7f32dc8b5640 1 -- 172.21.15.27:0/3909472821 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32d81f5210 con 0x7f32d80b6e00 2024-09-12T03:22:24.190 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.181+0000 7f32d7fff640 1 --2- 172.21.15.27:0/3909472821 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f32c00039b0 0x7f32d8158d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:22:24.190 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.181+0000 7f32d77fe640 1 --2- 172.21.15.27:0/3909472821 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f32d80cd710 0x7f32d80cdb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:22:24.190 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.182+0000 7f32dc8b5640 1 --2- 172.21.15.27:0/3909472821 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f32d80b6e00 0x7f32d80b7280 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f32d80b42d0 tx=0x7f32d0009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:24.190 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.182+0000 7f32d4ff9640 1 -- 172.21.15.27:0/3909472821 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32d001a030 con 0x7f32d80b6e00 2024-09-12T03:22:24.191 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.182+0000 7f32d4ff9640 1 -- 172.21.15.27:0/3909472821 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32d0010df0 con 0x7f32d80b6e00 2024-09-12T03:22:24.191 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.182+0000 7f32d4ff9640 1 -- 172.21.15.27:0/3909472821 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32d0003bd0 con 0x7f32d80b6e00 2024-09-12T03:22:24.191 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.182+0000 7f32de318640 1 -- 172.21.15.27:0/3909472821 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32d80b16a0 con 0x7f32d80b6e00 2024-09-12T03:22:24.191 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.182+0000 7f32de318640 1 -- 172.21.15.27:0/3909472821 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32d81f4450 con 0x7f32d80b6e00 2024-09-12T03:22:24.191 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.183+0000 7f32de318640 1 -- 172.21.15.27:0/3909472821 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32d8073250 con 0x7f32d80b6e00 2024-09-12T03:22:24.191 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.183+0000 7f32d4ff9640 1 -- 172.21.15.27:0/3909472821 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 21) ==== 91008+0+0 (secure 0 0 0) 0x7f32d001d070 con 0x7f32d80b6e00 2024-09-12T03:22:24.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.184+0000 7f32d4ff9640 1 --2- 172.21.15.27:0/3909472821 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f32c406c3d0 0x7f32c406e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:24.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.184+0000 7f32d4ff9640 1 -- 172.21.15.27:0/3909472821 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 5765+0+0 (secure 0 0 0) 0x7f32d008bee0 con 0x7f32d80b6e00 2024-09-12T03:22:24.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.184+0000 7f32d7fff640 1 --2- 172.21.15.27:0/3909472821 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f32c406c3d0 0x7f32c406e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:24.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.184+0000 7f32d7fff640 1 --2- 172.21.15.27:0/3909472821 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f32c406c3d0 0x7f32c406e890 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f32c8033000 tx=0x7f32c8008030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:24.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.188+0000 7f32d4ff9640 1 -- 172.21.15.27:0/3909472821 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f32d005a0c0 con 0x7f32d80b6e00 2024-09-12T03:22:24.397 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.397+0000 7f32de318640 1 -- 172.21.15.27:0/3909472821 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f32d8159f20 con 0x7f32d80b6e00 2024-09-12T03:22:24.397 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.398+0000 7f32d4ff9640 1 -- 172.21.15.27:0/3909472821 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v75) ==== 53+0+73 (secure 0 0 0) 0x7f32d00599c0 con 0x7f32d80b6e00 2024-09-12T03:22:24.398 INFO:teuthology.orchestra.run.smithi027.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-12T03:22:24.400 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.399+0000 7f32de318640 1 -- 172.21.15.27:0/3909472821 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f32c406c3d0 msgr2=0x7f32c406e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:24.401 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.399+0000 7f32de318640 1 --2- 172.21.15.27:0/3909472821 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f32c406c3d0 0x7f32c406e890 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f32c8033000 tx=0x7f32c8008030 comp rx=0 tx=0).stop 2024-09-12T03:22:24.401 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.400+0000 7f32de318640 1 -- 172.21.15.27:0/3909472821 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f32d80b6e00 msgr2=0x7f32d80b7280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:24.401 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.400+0000 7f32de318640 1 --2- 172.21.15.27:0/3909472821 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f32d80b6e00 0x7f32d80b7280 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f32d80b42d0 tx=0x7f32d0009f90 comp rx=0 tx=0).stop 2024-09-12T03:22:24.401 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.400+0000 7f32de318640 1 -- 172.21.15.27:0/3909472821 shutdown_connections 2024-09-12T03:22:24.401 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.400+0000 7f32de318640 1 --2- 172.21.15.27:0/3909472821 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f32c406c3d0 0x7f32c406e890 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:24.401 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.400+0000 7f32de318640 1 --2- 172.21.15.27:0/3909472821 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f32d80b6e00 0x7f32d80b7280 unknown :-1 s=CLOSED pgs=1258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:24.401 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.400+0000 7f32de318640 1 --2- 172.21.15.27:0/3909472821 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f32d80cd710 0x7f32d80cdb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:24.401 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.400+0000 7f32de318640 1 --2- 172.21.15.27:0/3909472821 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f32c00039b0 0x7f32d8158d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:24.401 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.400+0000 7f32de318640 1 -- 172.21.15.27:0/3909472821 >> 172.21.15.27:0/3909472821 conn(0x7f32d806f300 msgr2=0x7f32d8159370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:24.401 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.400+0000 7f32de318640 1 -- 172.21.15.27:0/3909472821 shutdown_connections 2024-09-12T03:22:24.401 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24.400+0000 7f32de318640 1 -- 172.21.15.27:0/3909472821 wait complete. 2024-09-12T03:22:24.411 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-12T03:22:24.411 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-12T03:22:24.411 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi027.front.sepia.ceph.com 2024-09-12T03:22:24.411 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-12T03:22:24.411 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-12T03:22:24.412 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-12T03:22:24.412 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-12T03:22:24.412 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-12T03:22:24.412 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-12T03:22:24.412 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:22:24.412 DEBUG:teuthology.orchestra.run.smithi027:> ip addr 2024-09-12T03:22:24.429 INFO:teuthology.orchestra.run.smithi027.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-12T03:22:24.429 INFO:teuthology.orchestra.run.smithi027.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-12T03:22:24.429 INFO:teuthology.orchestra.run.smithi027.stdout: inet 127.0.0.1/8 scope host lo 2024-09-12T03:22:24.430 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:22:24.430 INFO:teuthology.orchestra.run.smithi027.stdout: inet6 ::1/128 scope host 2024-09-12T03:22:24.430 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:22:24.430 INFO:teuthology.orchestra.run.smithi027.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-12T03:22:24.430 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether 0c:c4:7a:6b:fe:c8 brd ff:ff:ff:ff:ff:ff 2024-09-12T03:22:24.430 INFO:teuthology.orchestra.run.smithi027.stdout: altname enp7s0f0 2024-09-12T03:22:24.430 INFO:teuthology.orchestra.run.smithi027.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-12T03:22:24.430 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether 0c:c4:7a:6b:fe:c9 brd ff:ff:ff:ff:ff:ff 2024-09-12T03:22:24.430 INFO:teuthology.orchestra.run.smithi027.stdout: altname enp7s0f1 2024-09-12T03:22:24.431 INFO:teuthology.orchestra.run.smithi027.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-12T03:22:24.431 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether ac:1f:6b:f8:39:5a brd ff:ff:ff:ff:ff:ff 2024-09-12T03:22:24.431 INFO:teuthology.orchestra.run.smithi027.stdout: altname enp1s0f0 2024-09-12T03:22:24.431 INFO:teuthology.orchestra.run.smithi027.stdout: inet 172.21.15.27/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-12T03:22:24.431 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft 42444sec preferred_lft 42444sec 2024-09-12T03:22:24.431 INFO:teuthology.orchestra.run.smithi027.stdout: inet6 fe80::ae1f:6bff:fef8:395a/64 scope link 2024-09-12T03:22:24.431 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:22:24.431 INFO:teuthology.orchestra.run.smithi027.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-12T03:22:24.432 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether ac:1f:6b:f8:39:5b brd ff:ff:ff:ff:ff:ff 2024-09-12T03:22:24.432 INFO:teuthology.orchestra.run.smithi027.stdout: altname enp1s0f1 2024-09-12T03:22:24.432 INFO:teuthology.orchestra.run.smithi027.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-12T03:22:24.432 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether 2e:7f:89:6c:33:a0 brd ff:ff:ff:ff:ff:ff 2024-09-12T03:22:24.432 INFO:teuthology.orchestra.run.smithi027.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-12T03:22:24.432 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:22:24.432 INFO:teuthology.orchestra.run.smithi027.stdout: inet6 fe80::c84b:19ff:fef0:ca74/64 scope link 2024-09-12T03:22:24.433 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:22:24.433 INFO:teuthology.orchestra.run.smithi027.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-12T03:22:24.433 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether 42:40:d7:5c:7a:8e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-12T03:22:24.433 INFO:teuthology.orchestra.run.smithi027.stdout: inet6 fe80::4040:d7ff:fe5c:7a8e/64 scope link 2024-09-12T03:22:24.433 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:22:24.433 INFO:teuthology.orchestra.run.smithi027.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-12T03:22:24.433 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether 2e:7f:89:6c:33:a0 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-12T03:22:24.433 INFO:teuthology.orchestra.run.smithi027.stdout: inet6 fe80::2c7f:89ff:fe6c:33a0/64 scope link 2024-09-12T03:22:24.434 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:22:24.434 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:22:24.434 DEBUG:teuthology.orchestra.run.smithi027:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-12T03:22:24.494 INFO:teuthology.orchestra.run.smithi027.stdout:1 2024-09-12T03:22:24.496 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:22:24.496 DEBUG:teuthology.orchestra.run.smithi027:> route 2024-09-12T03:22:24.553 INFO:teuthology.orchestra.run.smithi027.stdout:Kernel IP routing table 2024-09-12T03:22:24.553 INFO:teuthology.orchestra.run.smithi027.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-12T03:22:24.553 INFO:teuthology.orchestra.run.smithi027.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-12T03:22:24.553 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-12T03:22:24.553 INFO:teuthology.orchestra.run.smithi027.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-12T03:22:24.553 DEBUG:teuthology.orchestra.run.smithi027:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-12T03:22:24.553 DEBUG:teuthology.orchestra.run.smithi027:> set -e 2024-09-12T03:22:24.553 DEBUG:teuthology.orchestra.run.smithi027:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-12T03:22:24.554 DEBUG:teuthology.orchestra.run.smithi027:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-12T03:22:24.554 DEBUG:teuthology.orchestra.run.smithi027:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-12T03:22:24.554 DEBUG:teuthology.orchestra.run.smithi027:> ') 2024-09-12T03:22:24.642 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T03:22:24.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T03:22:24.681 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:22:24.682 DEBUG:teuthology.orchestra.run.smithi027:> ip netns list 2024-09-12T03:22:24.698 INFO:teuthology.orchestra.run.smithi027.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-12T03:22:24.698 INFO:teuthology.orchestra.run.smithi027.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-12T03:22:24.698 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-12T03:22:24.698 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:22:24.698 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip link set brx.0 up 2024-09-12T03:22:24.764 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-12T03:22:24.764 DEBUG:teuthology.orchestra.run.smithi027:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-12T03:22:24.820 INFO:teuthology.orchestra.run.smithi027.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-12T03:22:24.820 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-12T03:22:24.820 DEBUG:teuthology.orchestra.run.smithi027:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-12T03:22:24.877 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-09-12T03:22:24.877 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:22:24.877 DEBUG:teuthology.orchestra.run.smithi027:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-12T03:22:24.962 INFO:teuthology.orchestra.run.smithi027.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-12T03:22:24.962 INFO:teuthology.orchestra.run.smithi027.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-12T03:22:24.970 INFO:teuthology.orchestra.run.smithi027.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-12T03:22:24.970 INFO:teuthology.orchestra.run.smithi027.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-12T03:22:24.970 INFO:teuthology.orchestra.run.smithi027.stdout:mount.ceph: resolved to: "172.21.15.27:3300,172.21.15.45:3300,172.21.15.45:3301" 2024-09-12T03:22:24.970 INFO:teuthology.orchestra.run.smithi027.stdout:mount.ceph: trying mount with new device syntax: 0@d2df504e-2647-4385-a5db-9fdf69f487a0.cephfs=/ 2024-09-12T03:22:24.970 INFO:teuthology.orchestra.run.smithi027.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.27:3300/172.21.15.45:3300/172.21.15.45:3301" will pass to kernel 2024-09-12T03:22:24.971 INFO:teuthology.orchestra.run.smithi027.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-12T03:22:24.971 INFO:teuthology.orchestra.run.smithi027.stdout: You just mounted a file system that supports labels which does not 2024-09-12T03:22:24.971 INFO:teuthology.orchestra.run.smithi027.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-12T03:22:24.971 INFO:teuthology.orchestra.run.smithi027.stdout: applications will generate AVC messages and not be allowed access to 2024-09-12T03:22:24.971 INFO:teuthology.orchestra.run.smithi027.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-12T03:22:24.973 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-12T03:22:24.973 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:22:24.973 DEBUG:teuthology.orchestra.run.smithi027:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-12T03:22:24.999 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-12T03:22:25.090 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:25 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-12T03:22:25.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T03:22:25.101 INFO:teuthology.orchestra.run.smithi027.stdout:client7180 2024-09-12T03:22:25.101 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-12T03:22:25.102 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T03:22:25.118 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-12T03:22:25.260 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:25.260+0000 7fa91700e640 1 Processor -- start 2024-09-12T03:22:25.261 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:25.260+0000 7fa91700e640 1 -- start start 2024-09-12T03:22:25.262 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:25.261+0000 7fa91700e640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa91015c460 0x7fa9100b55b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:25.262 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:25.261+0000 7fa91700e640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa9100b69b0 0x7fa9100d2360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:25.262 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:25.261+0000 7fa91700e640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa9100b5f00 con 0x7fa9100d28a0 2024-09-12T03:22:25.262 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:25.261+0000 7fa91700e640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fa9100d2d10 con 0x7fa9100b69b0 2024-09-12T03:22:25.262 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:25.261+0000 7fa91700e640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fa9100d00d0 con 0x7fa91015c460 2024-09-12T03:22:25.262 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:25.262+0000 7fa914daa640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa91015c460 0x7fa9100b55b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:25.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:25.262+0000 7fa90ffff640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa9100b69b0 0x7fa9100d2360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:25.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:25.262+0000 7fa914daa640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa91015c460 0x7fa9100b55b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3301/0 says I am v2:172.21.15.27:54438/0 (socket says 172.21.15.27:54438) 2024-09-12T03:22:25.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:25.262+0000 7fa90ffff640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa9100b69b0 0x7fa9100d2360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.27:38126/0 (socket says 172.21.15.27:38126) 2024-09-12T03:22:25.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:25.262+0000 7fa914daa640 1 -- 172.21.15.27:0/3880556977 learned_addr learned my addr 172.21.15.27:0/3880556977 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:22:25.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:25.262+0000 7fa9155ab640 1 --1- 172.21.15.27:0/3880556977 >> v1:172.21.15.45:6789/0 conn(0x7fa9100d28a0 0x7fa9100731c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.27:36002/0 (socket says 172.21.15.27:36002) 2024-09-12T03:22:25.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:25.262+0000 7fa914daa640 1 -- 172.21.15.27:0/3880556977 >> v2:172.21.15.45:3300/0 conn(0x7fa9100b69b0 msgr2=0x7fa9100d2360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:25.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:25.262+0000 7fa914daa640 1 --2- 172.21.15.27:0/3880556977 >> v2:172.21.15.45:3300/0 conn(0x7fa9100b69b0 0x7fa9100d2360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:25.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:25.262+0000 7fa914daa640 1 -- 172.21.15.27:0/3880556977 >> v1:172.21.15.45:6789/0 conn(0x7fa9100d28a0 legacy=0x7fa9100731c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:25.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:25.262+0000 7fa914daa640 1 -- 172.21.15.27:0/3880556977 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9100b61e0 con 0x7fa91015c460 2024-09-12T03:22:25.264 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:25.263+0000 7fa914daa640 1 --2- 172.21.15.27:0/3880556977 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa91015c460 0x7fa9100b55b0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fa908009e50 tx=0x7fa90802a710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c131285fffe22268 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:25.265 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:25.263+0000 7fa90f7fe640 1 -- 172.21.15.27:0/3880556977 <== mon.2 v2:172.21.15.45:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa908036070 con 0x7fa91015c460 2024-09-12T03:22:25.265 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:25.263+0000 7fa90f7fe640 1 -- 172.21.15.27:0/3880556977 <== mon.2 v2:172.21.15.45:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa908031600 con 0x7fa91015c460 2024-09-12T03:22:25.265 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:25.263+0000 7fa90f7fe640 1 -- 172.21.15.27:0/3880556977 <== mon.2 v2:172.21.15.45:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9080319d0 con 0x7fa91015c460 2024-09-12T03:22:25.265 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:25.263+0000 7fa91700e640 1 -- 172.21.15.27:0/3880556977 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa91015c460 msgr2=0x7fa9100b55b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:25.265 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:25.263+0000 7fa91700e640 1 --2- 172.21.15.27:0/3880556977 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa91015c460 0x7fa9100b55b0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fa908009e50 tx=0x7fa90802a710 comp rx=0 tx=0).stop 2024-09-12T03:22:25.265 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:25.264+0000 7fa91700e640 1 -- 172.21.15.27:0/3880556977 shutdown_connections 2024-09-12T03:22:25.265 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:25.264+0000 7fa91700e640 1 --2- 172.21.15.27:0/3880556977 >> v2:172.21.15.45:3300/0 conn(0x7fa9100b69b0 0x7fa9100d2360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:25.265 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:25.264+0000 7fa91700e640 1 --2- 172.21.15.27:0/3880556977 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa91015c460 0x7fa9100b55b0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:25.265 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:25.264+0000 7fa91700e640 1 -- 172.21.15.27:0/3880556977 >> 172.21.15.27:0/3880556977 conn(0x7fa91006f320 msgr2=0x7fa91006f700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:25.265 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:25.264+0000 7fa91700e640 1 -- 172.21.15.27:0/3880556977 shutdown_connections 2024-09-12T03:22:25.266 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:25.264+0000 7fa91700e640 1 -- 172.21.15.27:0/3880556977 wait complete. 2024-09-12T03:22:25.266 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:25.265+0000 7fa91700e640 1 Processor -- start 2024-09-12T03:22:25.266 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:25.265+0000 7fa91700e640 1 -- start start 2024-09-12T03:22:25.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:25.266+0000 7fa91700e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa9100b69b0 0x7fa910159900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:25.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:25.266+0000 7fa91700e640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa9100d28a0 0x7fa9100aa4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:25.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:25.266+0000 7fa91700e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa91015c460 0x7fa91014ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:25.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:25.266+0000 7fa91700e640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fa9100d0670 con 0x7fa9100b69b0 2024-09-12T03:22:25.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:25.266+0000 7fa91700e640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa9100b64c0 con 0x7fa91015c460 2024-09-12T03:22:25.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:25.266+0000 7fa91700e640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fa9100b5af0 con 0x7fa9100d28a0 2024-09-12T03:22:25.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:25.266+0000 7fa914daa640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa9100b69b0 0x7fa910159900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:25.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:25.266+0000 7fa914daa640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa9100b69b0 0x7fa910159900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49444/0 (socket says 172.21.15.27:49444) 2024-09-12T03:22:25.269 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:25.266+0000 7fa914daa640 1 -- 172.21.15.27:0/3988793181 learned_addr learned my addr 172.21.15.27:0/3988793181 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:22:25.269 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:25.266+0000 7fa9155ab640 1 --2- 172.21.15.27:0/3988793181 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa91015c460 0x7fa91014ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:25.269 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:25.266+0000 7fa90ffff640 1 --2- 172.21.15.27:0/3988793181 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa9100d28a0 0x7fa9100aa4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:25.269 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:25.266+0000 7fa914daa640 1 -- 172.21.15.27:0/3988793181 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa9100d28a0 msgr2=0x7fa9100aa4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:25.269 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:25.266+0000 7fa914daa640 1 --2- 172.21.15.27:0/3988793181 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa9100d28a0 0x7fa9100aa4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:25.269 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:25.266+0000 7fa914daa640 1 -- 172.21.15.27:0/3988793181 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa91015c460 msgr2=0x7fa91014ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:25.269 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:25.266+0000 7fa914daa640 1 --2- 172.21.15.27:0/3988793181 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa91015c460 0x7fa91014ddc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:25.269 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:25.266+0000 7fa914daa640 1 -- 172.21.15.27:0/3988793181 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa910158c50 con 0x7fa9100b69b0 2024-09-12T03:22:25.269 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:25.266+0000 7fa90ffff640 1 --2- 172.21.15.27:0/3988793181 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa9100d28a0 0x7fa9100aa4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:22:25.269 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:25.266+0000 7fa914daa640 1 --2- 172.21.15.27:0/3988793181 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa9100b69b0 0x7fa910159900 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7fa908009f70 tx=0x7fa908004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:25.270 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:25.266+0000 7fa90d7fa640 1 -- 172.21.15.27:0/3988793181 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa90803f020 con 0x7fa9100b69b0 2024-09-12T03:22:25.270 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:25.266+0000 7fa90d7fa640 1 -- 172.21.15.27:0/3988793181 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa908037040 con 0x7fa9100b69b0 2024-09-12T03:22:25.270 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:25.266+0000 7fa91700e640 1 -- 172.21.15.27:0/3988793181 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa910159f70 con 0x7fa9100b69b0 2024-09-12T03:22:25.270 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:25.267+0000 7fa90d7fa640 1 -- 172.21.15.27:0/3988793181 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa908036070 con 0x7fa9100b69b0 2024-09-12T03:22:25.270 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:25.267+0000 7fa91700e640 1 -- 172.21.15.27:0/3988793181 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa91006d1a0 con 0x7fa9100b69b0 2024-09-12T03:22:25.270 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:25.267+0000 7fa91700e640 1 -- 172.21.15.27:0/3988793181 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa91015c960 con 0x7fa9100b69b0 2024-09-12T03:22:25.270 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:25.268+0000 7fa90d7fa640 1 -- 172.21.15.27:0/3988793181 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 21) ==== 91008+0+0 (secure 0 0 0) 0x7fa908042020 con 0x7fa9100b69b0 2024-09-12T03:22:25.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:25.269+0000 7fa90d7fa640 1 --2- 172.21.15.27:0/3988793181 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fa8f006c3d0 0x7fa8f006e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:25.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:25.269+0000 7fa90d7fa640 1 -- 172.21.15.27:0/3988793181 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 5765+0+0 (secure 0 0 0) 0x7fa9080abd90 con 0x7fa9100b69b0 2024-09-12T03:22:25.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:25.269+0000 7fa90ffff640 1 --2- 172.21.15.27:0/3988793181 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fa8f006c3d0 0x7fa8f006e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:25.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:25.269+0000 7fa90ffff640 1 --2- 172.21.15.27:0/3988793181 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fa8f006c3d0 0x7fa8f006e890 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fa910158ea0 tx=0x7fa900008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:25.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:25.273+0000 7fa90d7fa640 1 -- 172.21.15.27:0/3988793181 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fa908049030 con 0x7fa9100b69b0 2024-09-12T03:22:25.393 INFO:tasks.ceph.mds.c.smithi027.stderr:2024-09-12T03:22:25.393+0000 7fd2fb742640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-12T03:22:25.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:25.483+0000 7fa91700e640 1 -- 172.21.15.27:0/3988793181 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fa91015f9f0 con 0x7fa9100b69b0 2024-09-12T03:22:25.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:25.484+0000 7fa90d7fa640 1 -- 172.21.15.27:0/3988793181 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v75) ==== 53+0+73 (secure 0 0 0) 0x7fa908007c70 con 0x7fa9100b69b0 2024-09-12T03:22:25.484 INFO:teuthology.orchestra.run.smithi027.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-12T03:22:25.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:25.486+0000 7fa91700e640 1 -- 172.21.15.27:0/3988793181 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fa8f006c3d0 msgr2=0x7fa8f006e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:25.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:25.486+0000 7fa91700e640 1 --2- 172.21.15.27:0/3988793181 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fa8f006c3d0 0x7fa8f006e890 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fa910158ea0 tx=0x7fa900008040 comp rx=0 tx=0).stop 2024-09-12T03:22:25.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:25.486+0000 7fa91700e640 1 -- 172.21.15.27:0/3988793181 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa9100b69b0 msgr2=0x7fa910159900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:25.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:25.486+0000 7fa91700e640 1 --2- 172.21.15.27:0/3988793181 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa9100b69b0 0x7fa910159900 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7fa908009f70 tx=0x7fa908004a40 comp rx=0 tx=0).stop 2024-09-12T03:22:25.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:25.486+0000 7fa91700e640 1 -- 172.21.15.27:0/3988793181 shutdown_connections 2024-09-12T03:22:25.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:25.486+0000 7fa91700e640 1 --2- 172.21.15.27:0/3988793181 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa91015c460 0x7fa91014ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:25.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:25.486+0000 7fa91700e640 1 --2- 172.21.15.27:0/3988793181 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa9100d28a0 0x7fa9100aa4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:25.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:25.486+0000 7fa91700e640 1 --2- 172.21.15.27:0/3988793181 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fa8f006c3d0 0x7fa8f006e890 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:25.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:25.486+0000 7fa91700e640 1 --2- 172.21.15.27:0/3988793181 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa9100b69b0 0x7fa910159900 unknown :-1 s=CLOSED pgs=1259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:25.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:25.486+0000 7fa91700e640 1 -- 172.21.15.27:0/3988793181 >> 172.21.15.27:0/3988793181 conn(0x7fa91006f320 msgr2=0x7fa9100d0b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:25.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:25.486+0000 7fa91700e640 1 -- 172.21.15.27:0/3988793181 shutdown_connections 2024-09-12T03:22:25.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:25.486+0000 7fa91700e640 1 -- 172.21.15.27:0/3988793181 wait complete. 2024-09-12T03:22:25.497 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-12T03:22:25.497 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-12T03:22:25.497 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi045.front.sepia.ceph.com 2024-09-12T03:22:25.497 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-12T03:22:25.497 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-12T03:22:25.497 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-12T03:22:25.497 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-12T03:22:25.497 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-12T03:22:25.497 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-12T03:22:25.497 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:22:25.498 DEBUG:teuthology.orchestra.run.smithi045:> ip addr 2024-09-12T03:22:25.519 INFO:teuthology.orchestra.run.smithi045.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-12T03:22:25.519 INFO:teuthology.orchestra.run.smithi045.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-12T03:22:25.519 INFO:teuthology.orchestra.run.smithi045.stdout: inet 127.0.0.1/8 scope host lo 2024-09-12T03:22:25.519 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:22:25.520 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 ::1/128 scope host 2024-09-12T03:22:25.520 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:22:25.520 INFO:teuthology.orchestra.run.smithi045.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-12T03:22:25.520 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:6b:fd:a6 brd ff:ff:ff:ff:ff:ff 2024-09-12T03:22:25.520 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp7s0f0 2024-09-12T03:22:25.521 INFO:teuthology.orchestra.run.smithi045.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-12T03:22:25.521 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:6b:fd:a7 brd ff:ff:ff:ff:ff:ff 2024-09-12T03:22:25.521 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp7s0f1 2024-09-12T03:22:25.521 INFO:teuthology.orchestra.run.smithi045.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-12T03:22:25.521 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:bd:12:be brd ff:ff:ff:ff:ff:ff 2024-09-12T03:22:25.522 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp1s0f0 2024-09-12T03:22:25.522 INFO:teuthology.orchestra.run.smithi045.stdout: inet 172.21.15.45/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-12T03:22:25.522 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft 42388sec preferred_lft 42388sec 2024-09-12T03:22:25.522 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::ec4:7aff:febd:12be/64 scope link 2024-09-12T03:22:25.522 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:22:25.522 INFO:teuthology.orchestra.run.smithi045.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-12T03:22:25.523 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:bd:12:bf brd ff:ff:ff:ff:ff:ff 2024-09-12T03:22:25.523 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp1s0f1 2024-09-12T03:22:25.523 INFO:teuthology.orchestra.run.smithi045.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-12T03:22:25.523 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 52:97:33:5c:60:1b brd ff:ff:ff:ff:ff:ff 2024-09-12T03:22:25.523 INFO:teuthology.orchestra.run.smithi045.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-12T03:22:25.524 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:22:25.524 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::d8c2:5dff:fe26:78c8/64 scope link 2024-09-12T03:22:25.524 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:22:25.524 INFO:teuthology.orchestra.run.smithi045.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-12T03:22:25.524 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 5a:ec:d7:40:ef:75 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-12T03:22:25.524 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::58ec:d7ff:fe40:ef75/64 scope link 2024-09-12T03:22:25.525 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:22:25.525 INFO:teuthology.orchestra.run.smithi045.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-12T03:22:25.525 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 52:97:33:5c:60:1b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-12T03:22:25.525 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::5097:33ff:fe5c:601b/64 scope link 2024-09-12T03:22:25.525 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:22:25.526 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:22:25.526 DEBUG:teuthology.orchestra.run.smithi045:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-12T03:22:25.589 INFO:teuthology.orchestra.run.smithi045.stdout:1 2024-09-12T03:22:25.590 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:22:25.590 DEBUG:teuthology.orchestra.run.smithi045:> route 2024-09-12T03:22:25.650 INFO:teuthology.orchestra.run.smithi045.stdout:Kernel IP routing table 2024-09-12T03:22:25.650 INFO:teuthology.orchestra.run.smithi045.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-12T03:22:25.650 INFO:teuthology.orchestra.run.smithi045.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-12T03:22:25.650 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-12T03:22:25.650 INFO:teuthology.orchestra.run.smithi045.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-12T03:22:25.651 DEBUG:teuthology.orchestra.run.smithi045:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-12T03:22:25.651 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-09-12T03:22:25.651 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-12T03:22:25.651 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-12T03:22:25.651 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-12T03:22:25.651 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-09-12T03:22:25.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:22:25 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T03:22:25.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:22:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T03:22:25.788 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:22:25.788 DEBUG:teuthology.orchestra.run.smithi045:> ip netns list 2024-09-12T03:22:25.805 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-09-12T03:22:25.805 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-12T03:22:25.805 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-12T03:22:25.805 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:22:25.806 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link set brx.0 up 2024-09-12T03:22:25.872 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-12T03:22:25.873 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-12T03:22:25.929 INFO:teuthology.orchestra.run.smithi045.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-12T03:22:25.929 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-12T03:22:25.929 DEBUG:teuthology.orchestra.run.smithi045:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-12T03:22:25.985 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2024-09-12T03:22:25.985 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:22:25.986 DEBUG:teuthology.orchestra.run.smithi045:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-12T03:22:26.070 INFO:teuthology.orchestra.run.smithi045.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-12T03:22:26.071 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-12T03:22:26.078 INFO:teuthology.orchestra.run.smithi045.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-12T03:22:26.078 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-12T03:22:26.078 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: resolved to: "172.21.15.27:3300,172.21.15.45:3300,172.21.15.45:3301" 2024-09-12T03:22:26.078 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: trying mount with new device syntax: 1@d2df504e-2647-4385-a5db-9fdf69f487a0.cephfs=/ 2024-09-12T03:22:26.078 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.27:3300/172.21.15.45:3300/172.21.15.45:3301" will pass to kernel 2024-09-12T03:22:26.078 INFO:teuthology.orchestra.run.smithi045.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-09-12T03:22:26.079 INFO:teuthology.orchestra.run.smithi045.stdout: You just mounted a file system that supports labels which does not 2024-09-12T03:22:26.079 INFO:teuthology.orchestra.run.smithi045.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-12T03:22:26.079 INFO:teuthology.orchestra.run.smithi045.stdout: applications will generate AVC messages and not be allowed access to 2024-09-12T03:22:26.079 INFO:teuthology.orchestra.run.smithi045.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-12T03:22:26.080 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-12T03:22:26.080 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:22:26.080 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-12T03:22:26.105 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-12T03:22:26.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:22:26 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-12T03:22:26.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:22:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T03:22:26.203 INFO:teuthology.orchestra.run.smithi045.stdout:client7189 2024-09-12T03:22:26.204 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-12T03:22:26.204 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T03:22:26.220 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-09-12T03:22:26.337 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:26.337+0000 7f4493a7b640 1 Processor -- start 2024-09-12T03:22:26.337 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:26.337+0000 7f4493a7b640 1 -- start start 2024-09-12T03:22:26.338 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:26.338+0000 7f4493a7b640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f448c06cc40 0x7f448c06d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:26.338 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:26.338+0000 7f4493a7b640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f448c06db30 0x7f448c06df60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:26.338 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:26.338+0000 7f4493a7b640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f448c06d5d0 con 0x7f448c15e650 2024-09-12T03:22:26.338 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:26.338+0000 7f4493a7b640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f448c06d850 con 0x7f448c06cc40 2024-09-12T03:22:26.338 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:26.338+0000 7f4493a7b640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f448c0d1260 con 0x7f448c06db30 2024-09-12T03:22:26.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:26.338+0000 7f4491016640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f448c06cc40 0x7f448c06d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:26.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:26.338+0000 7f4491817640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f448c15e650 0x7f448c0d1d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:42834/0 (socket says 172.21.15.27:42834) 2024-09-12T03:22:26.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:26.338+0000 7f4491817640 1 -- 172.21.15.27:0/215097026 learned_addr learned my addr 172.21.15.27:0/215097026 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:22:26.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:26.338+0000 7f4492018640 1 --2- 172.21.15.27:0/215097026 >> v2:172.21.15.45:3300/0 conn(0x7f448c06db30 0x7f448c06df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:26.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:26.338+0000 7f4491016640 1 -- 172.21.15.27:0/215097026 >> v2:172.21.15.45:3300/0 conn(0x7f448c06db30 msgr2=0x7f448c06df60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:26.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:26.338+0000 7f4491016640 1 --2- 172.21.15.27:0/215097026 >> v2:172.21.15.45:3300/0 conn(0x7f448c06db30 0x7f448c06df60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:26.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:26.338+0000 7f4491016640 1 -- 172.21.15.27:0/215097026 >> v1:172.21.15.27:6789/0 conn(0x7f448c15e650 legacy=0x7f448c0d1d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:26.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:26.338+0000 7f4491016640 1 -- 172.21.15.27:0/215097026 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f448c0d1540 con 0x7f448c06cc40 2024-09-12T03:22:26.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:26.338+0000 7f4490815640 1 -- 172.21.15.27:0/215097026 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 765475198 0 0) 0x7f4488001110 con 0x7f448c15e650 2024-09-12T03:22:26.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:26.339+0000 7f4491016640 1 --2- 172.21.15.27:0/215097026 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f448c06cc40 0x7f448c06d090 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f448c0b21f0 tx=0x7f447c02a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f07ca93acea5e4df server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:26.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:26.339+0000 7f4490815640 1 -- 172.21.15.27:0/215097026 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f447c037070 con 0x7f448c06cc40 2024-09-12T03:22:26.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:26.339+0000 7f4490815640 1 -- 172.21.15.27:0/215097026 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f447c032440 con 0x7f448c06cc40 2024-09-12T03:22:26.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:26.339+0000 7f4490815640 1 -- 172.21.15.27:0/215097026 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f447c02ae60 con 0x7f448c06cc40 2024-09-12T03:22:26.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:26.340+0000 7f4493a7b640 1 -- 172.21.15.27:0/215097026 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f448c06cc40 msgr2=0x7f448c06d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:26.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:26.340+0000 7f4493a7b640 1 --2- 172.21.15.27:0/215097026 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f448c06cc40 0x7f448c06d090 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f448c0b21f0 tx=0x7f447c02a110 comp rx=0 tx=0).stop 2024-09-12T03:22:26.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:26.340+0000 7f4493a7b640 1 -- 172.21.15.27:0/215097026 shutdown_connections 2024-09-12T03:22:26.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:26.340+0000 7f4493a7b640 1 --2- 172.21.15.27:0/215097026 >> v2:172.21.15.45:3300/0 conn(0x7f448c06db30 0x7f448c06df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:26.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:26.340+0000 7f4493a7b640 1 --2- 172.21.15.27:0/215097026 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f448c06cc40 0x7f448c06d090 unknown :-1 s=CLOSED pgs=1261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:26.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:26.340+0000 7f4493a7b640 1 -- 172.21.15.27:0/215097026 >> 172.21.15.27:0/215097026 conn(0x7f448c06f320 msgr2=0x7f448c06f700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:26.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:26.340+0000 7f4493a7b640 1 -- 172.21.15.27:0/215097026 shutdown_connections 2024-09-12T03:22:26.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:26.340+0000 7f4493a7b640 1 -- 172.21.15.27:0/215097026 wait complete. 2024-09-12T03:22:26.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:26.341+0000 7f4493a7b640 1 Processor -- start 2024-09-12T03:22:26.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:26.341+0000 7f4493a7b640 1 -- start start 2024-09-12T03:22:26.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:26.342+0000 7f4493a7b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f448c06cc40 0x7f448c14b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:26.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:26.342+0000 7f4493a7b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f448c06db30 0x7f448c14be80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:26.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:26.342+0000 7f4493a7b640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f448c15e650 0x7f448c14e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:26.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:26.342+0000 7f4493a7b640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f448c06e620 con 0x7f448c06cc40 2024-09-12T03:22:26.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:26.342+0000 7f4493a7b640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f448c06e4a0 con 0x7f448c06db30 2024-09-12T03:22:26.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:26.342+0000 7f4493a7b640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f448c0b55b0 con 0x7f448c15e650 2024-09-12T03:22:26.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:26.342+0000 7f4491817640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f448c06cc40 0x7f448c14b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:26.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:26.342+0000 7f4491817640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f448c06cc40 0x7f448c14b940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49462/0 (socket says 172.21.15.27:49462) 2024-09-12T03:22:26.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:26.342+0000 7f4491817640 1 -- 172.21.15.27:0/2411007378 learned_addr learned my addr 172.21.15.27:0/2411007378 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:22:26.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:26.342+0000 7f4492018640 1 --2- 172.21.15.27:0/2411007378 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f448c15e650 0x7f448c14e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:26.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:26.342+0000 7f4491016640 1 --2- 172.21.15.27:0/2411007378 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f448c06db30 0x7f448c14be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:26.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:26.342+0000 7f4491817640 1 -- 172.21.15.27:0/2411007378 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f448c15e650 msgr2=0x7f448c14e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:26.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:26.342+0000 7f4491817640 1 --2- 172.21.15.27:0/2411007378 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f448c15e650 0x7f448c14e5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:26.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:26.342+0000 7f4491817640 1 -- 172.21.15.27:0/2411007378 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f448c06db30 msgr2=0x7f448c14be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:26.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:26.342+0000 7f4491817640 1 --2- 172.21.15.27:0/2411007378 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f448c06db30 0x7f448c14be80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:26.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:26.342+0000 7f4491817640 1 -- 172.21.15.27:0/2411007378 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f448c14d4c0 con 0x7f448c06cc40 2024-09-12T03:22:26.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:26.342+0000 7f4491817640 1 --2- 172.21.15.27:0/2411007378 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f448c06cc40 0x7f448c14b940 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f448800c460 tx=0x7f448800c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:26.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:26.343+0000 7f44827fc640 1 -- 172.21.15.27:0/2411007378 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4488018070 con 0x7f448c06cc40 2024-09-12T03:22:26.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:26.343+0000 7f44827fc640 1 -- 172.21.15.27:0/2411007378 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4488007500 con 0x7f448c06cc40 2024-09-12T03:22:26.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:26.343+0000 7f4493a7b640 1 -- 172.21.15.27:0/2411007378 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f448c14d7a0 con 0x7f448c06cc40 2024-09-12T03:22:26.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:26.343+0000 7f44827fc640 1 -- 172.21.15.27:0/2411007378 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4488004a20 con 0x7f448c06cc40 2024-09-12T03:22:26.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:26.343+0000 7f4493a7b640 1 -- 172.21.15.27:0/2411007378 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f448c14dd60 con 0x7f448c06cc40 2024-09-12T03:22:26.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:26.344+0000 7f4493a7b640 1 -- 172.21.15.27:0/2411007378 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f448c0b8350 con 0x7f448c06cc40 2024-09-12T03:22:26.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:26.344+0000 7f44827fc640 1 -- 172.21.15.27:0/2411007378 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 21) ==== 91008+0+0 (secure 0 0 0) 0x7f448801b070 con 0x7f448c06cc40 2024-09-12T03:22:26.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:26.345+0000 7f44827fc640 1 --2- 172.21.15.27:0/2411007378 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f447006c3d0 0x7f447006e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:26.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:26.345+0000 7f4491016640 1 --2- 172.21.15.27:0/2411007378 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f447006c3d0 0x7f447006e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:26.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:26.345+0000 7f44827fc640 1 -- 172.21.15.27:0/2411007378 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 5765+0+0 (secure 0 0 0) 0x7f448808aa50 con 0x7f448c06cc40 2024-09-12T03:22:26.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:26.346+0000 7f4491016640 1 --2- 172.21.15.27:0/2411007378 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f447006c3d0 0x7f447006e890 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f447c034000 tx=0x7f447c02e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:26.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:26.349+0000 7f44827fc640 1 -- 172.21.15.27:0/2411007378 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f448c0b8350 con 0x7f448c06cc40 2024-09-12T03:22:26.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:26.564+0000 7f4493a7b640 1 -- 172.21.15.27:0/2411007378 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7f448c15dd30 con 0x7f448c06cc40 2024-09-12T03:22:27.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:27.442+0000 7f44827fc640 1 -- 172.21.15.27:0/2411007378 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v76) ==== 136+0+0 (secure 0 0 0) 0x7f448c06d5d0 con 0x7f448c06cc40 2024-09-12T03:22:27.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:27.445+0000 7f4493a7b640 1 -- 172.21.15.27:0/2411007378 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f447006c3d0 msgr2=0x7f447006e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:27.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:27.445+0000 7f4493a7b640 1 --2- 172.21.15.27:0/2411007378 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f447006c3d0 0x7f447006e890 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f447c034000 tx=0x7f447c02e000 comp rx=0 tx=0).stop 2024-09-12T03:22:27.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:27.445+0000 7f4493a7b640 1 -- 172.21.15.27:0/2411007378 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f448c06cc40 msgr2=0x7f448c14b940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:27.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:27.445+0000 7f4493a7b640 1 --2- 172.21.15.27:0/2411007378 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f448c06cc40 0x7f448c14b940 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f448800c460 tx=0x7f448800c920 comp rx=0 tx=0).stop 2024-09-12T03:22:27.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:27.445+0000 7f4493a7b640 1 -- 172.21.15.27:0/2411007378 shutdown_connections 2024-09-12T03:22:27.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:27.445+0000 7f4493a7b640 1 --2- 172.21.15.27:0/2411007378 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f448c15e650 0x7f448c14e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:27.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:27.445+0000 7f4493a7b640 1 --2- 172.21.15.27:0/2411007378 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f448c06db30 0x7f448c14be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:27.446 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:27.445+0000 7f4493a7b640 1 --2- 172.21.15.27:0/2411007378 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f447006c3d0 0x7f447006e890 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:27.446 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:27.445+0000 7f4493a7b640 1 --2- 172.21.15.27:0/2411007378 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f448c06cc40 0x7f448c14b940 unknown :-1 s=CLOSED pgs=1262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:27.446 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:27.445+0000 7f4493a7b640 1 -- 172.21.15.27:0/2411007378 >> 172.21.15.27:0/2411007378 conn(0x7f448c06f320 msgr2=0x7f448c15f000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:27.446 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:27.445+0000 7f4493a7b640 1 -- 172.21.15.27:0/2411007378 shutdown_connections 2024-09-12T03:22:27.446 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:27.445+0000 7f4493a7b640 1 -- 172.21.15.27:0/2411007378 wait complete. 2024-09-12T03:22:27.458 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-12T03:22:27.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:27.569+0000 7f54de913640 1 Processor -- start 2024-09-12T03:22:27.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:27.569+0000 7f54de913640 1 -- start start 2024-09-12T03:22:27.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:27.570+0000 7f54de913640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f54d8072d70 0x7f54d80b3560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:27.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:27.570+0000 7f54de913640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f54d80ce410 con 0x7f54d815b2b0 2024-09-12T03:22:27.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:27.570+0000 7f54de913640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f54d80732c0 con 0x7f54d806e9b0 2024-09-12T03:22:27.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:27.570+0000 7f54de913640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f54d80720a0 con 0x7f54d8072d70 2024-09-12T03:22:27.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:27.570+0000 7f54d77fe640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f54d8072d70 0x7f54d80b3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:27.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:27.570+0000 7f54d7fff640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f54d815b2b0 0x7f54d8072430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:42850/0 (socket says 172.21.15.27:42850) 2024-09-12T03:22:27.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:27.570+0000 7f54d7fff640 1 -- 172.21.15.27:0/3660938252 learned_addr learned my addr 172.21.15.27:0/3660938252 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:22:27.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:27.571+0000 7f54d77fe640 1 -- 172.21.15.27:0/3660938252 >> v1:172.21.15.45:6789/0 conn(0x7f54d806e9b0 legacy=0x7f54d80cebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:27.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:27.571+0000 7f54d77fe640 1 -- 172.21.15.27:0/3660938252 >> v1:172.21.15.27:6789/0 conn(0x7f54d815b2b0 legacy=0x7f54d8072430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:27.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:27.571+0000 7f54d6ffd640 1 -- 172.21.15.27:0/3660938252 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3634627464 0 0) 0x7f54c8001110 con 0x7f54d815b2b0 2024-09-12T03:22:27.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:27.571+0000 7f54d77fe640 1 -- 172.21.15.27:0/3660938252 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54d8071ae0 con 0x7f54d8072d70 2024-09-12T03:22:27.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:27.571+0000 7f54d77fe640 1 --2- 172.21.15.27:0/3660938252 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f54d8072d70 0x7f54d80b3560 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f54cc0097a0 tx=0x7f54cc02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86fc6442ac4558ba server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:27.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:27.572+0000 7f54d6ffd640 1 -- 172.21.15.27:0/3660938252 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54cc037070 con 0x7f54d8072d70 2024-09-12T03:22:27.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:27.572+0000 7f54d6ffd640 1 -- 172.21.15.27:0/3660938252 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54cc032440 con 0x7f54d8072d70 2024-09-12T03:22:27.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:27.572+0000 7f54d6ffd640 1 -- 172.21.15.27:0/3660938252 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54cc032810 con 0x7f54d8072d70 2024-09-12T03:22:27.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:27.572+0000 7f54de913640 1 -- 172.21.15.27:0/3660938252 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f54d8072d70 msgr2=0x7f54d80b3560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:27.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:27.572+0000 7f54de913640 1 --2- 172.21.15.27:0/3660938252 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f54d8072d70 0x7f54d80b3560 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f54cc0097a0 tx=0x7f54cc02a1e0 comp rx=0 tx=0).stop 2024-09-12T03:22:27.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:27.572+0000 7f54de913640 1 -- 172.21.15.27:0/3660938252 shutdown_connections 2024-09-12T03:22:27.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:27.572+0000 7f54de913640 1 --2- 172.21.15.27:0/3660938252 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f54d8072d70 0x7f54d80b3560 unknown :-1 s=CLOSED pgs=1264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:27.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:27.572+0000 7f54de913640 1 -- 172.21.15.27:0/3660938252 >> 172.21.15.27:0/3660938252 conn(0x7f54d8062c60 msgr2=0x7f54d806bb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:27.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:27.572+0000 7f54de913640 1 -- 172.21.15.27:0/3660938252 shutdown_connections 2024-09-12T03:22:27.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:27.572+0000 7f54de913640 1 -- 172.21.15.27:0/3660938252 wait complete. 2024-09-12T03:22:27.574 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:27.573+0000 7f54de913640 1 Processor -- start 2024-09-12T03:22:27.574 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:27.574+0000 7f54de913640 1 -- start start 2024-09-12T03:22:27.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:27.574+0000 7f54de913640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f54d806e9b0 0x7f54d814dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:27.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:27.574+0000 7f54de913640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f54d8072d70 0x7f54d81592b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:27.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:27.574+0000 7f54de913640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f54d815b2b0 0x7f54d8146080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:27.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:27.574+0000 7f54de913640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f54d815b690 con 0x7f54d815b2b0 2024-09-12T03:22:27.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:27.574+0000 7f54de913640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f54d80ce690 con 0x7f54d806e9b0 2024-09-12T03:22:27.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:27.574+0000 7f54de913640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f54d815b810 con 0x7f54d8072d70 2024-09-12T03:22:27.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:27.574+0000 7f54dceb0640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f54d815b2b0 0x7f54d8146080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:27.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:27.575+0000 7f54dceb0640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f54d815b2b0 0x7f54d8146080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49480/0 (socket says 172.21.15.27:49480) 2024-09-12T03:22:27.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:27.575+0000 7f54dceb0640 1 -- 172.21.15.27:0/2516347860 learned_addr learned my addr 172.21.15.27:0/2516347860 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:22:27.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:27.575+0000 7f54d7fff640 1 --2- 172.21.15.27:0/2516347860 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f54d806e9b0 0x7f54d814dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:27.577 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:27.575+0000 7f54d77fe640 1 --2- 172.21.15.27:0/2516347860 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f54d8072d70 0x7f54d81592b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:27.577 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:27.575+0000 7f54dceb0640 1 -- 172.21.15.27:0/2516347860 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f54d8072d70 msgr2=0x7f54d81592b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:27.577 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:27.575+0000 7f54dceb0640 1 --2- 172.21.15.27:0/2516347860 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f54d8072d70 0x7f54d81592b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:27.577 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:27.575+0000 7f54dceb0640 1 -- 172.21.15.27:0/2516347860 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f54d806e9b0 msgr2=0x7f54d814dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:27.577 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:27.575+0000 7f54dceb0640 1 --2- 172.21.15.27:0/2516347860 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f54d806e9b0 0x7f54d814dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:27.577 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:27.575+0000 7f54dceb0640 1 -- 172.21.15.27:0/2516347860 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54d815a680 con 0x7f54d815b2b0 2024-09-12T03:22:27.577 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:27.575+0000 7f54dceb0640 1 --2- 172.21.15.27:0/2516347860 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f54d815b2b0 0x7f54d8146080 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f54d000c780 tx=0x7f54d000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:27.577 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:27.575+0000 7f54d4ff9640 1 -- 172.21.15.27:0/2516347860 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54d0004b60 con 0x7f54d815b2b0 2024-09-12T03:22:27.577 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:27.575+0000 7f54d4ff9640 1 -- 172.21.15.27:0/2516347860 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54d0004d00 con 0x7f54d815b2b0 2024-09-12T03:22:27.577 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:27.575+0000 7f54d4ff9640 1 -- 172.21.15.27:0/2516347860 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54d000aa90 con 0x7f54d815b2b0 2024-09-12T03:22:27.577 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:27.575+0000 7f54de913640 1 -- 172.21.15.27:0/2516347860 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54d80b4e70 con 0x7f54d815b2b0 2024-09-12T03:22:27.577 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:27.575+0000 7f54de913640 1 -- 172.21.15.27:0/2516347860 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54d80b5430 con 0x7f54d815b2b0 2024-09-12T03:22:27.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:27.576+0000 7f54de913640 1 -- 172.21.15.27:0/2516347860 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54d80b3560 con 0x7f54d815b2b0 2024-09-12T03:22:27.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:27.577+0000 7f54d4ff9640 1 -- 172.21.15.27:0/2516347860 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 21) ==== 91008+0+0 (secure 0 0 0) 0x7f54d000ac30 con 0x7f54d815b2b0 2024-09-12T03:22:27.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:27.577+0000 7f54d4ff9640 1 --2- 172.21.15.27:0/2516347860 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f54c406c400 0x7f54c406e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:27.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:27.578+0000 7f54d4ff9640 1 -- 172.21.15.27:0/2516347860 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 5765+0+0 (secure 0 0 0) 0x7f54d008ba10 con 0x7f54d815b2b0 2024-09-12T03:22:27.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:27.578+0000 7f54d7fff640 1 --2- 172.21.15.27:0/2516347860 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f54c406c400 0x7f54c406e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:27.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:27.578+0000 7f54d7fff640 1 --2- 172.21.15.27:0/2516347860 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f54c406c400 0x7f54c406e8c0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f54c8007fd0 tx=0x7f54c8007de0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:27.581 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:27.582+0000 7f54d4ff9640 1 -- 172.21.15.27:0/2516347860 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f54d0020070 con 0x7f54d815b2b0 2024-09-12T03:22:27.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:27.778+0000 7f54de913640 1 -- 172.21.15.27:0/2516347860 --> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f54d8074f00 con 0x7f54c406c400 2024-09-12T03:22:27.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:27.780+0000 7f54d4ff9640 1 -- 172.21.15.27:0/2516347860 <== mgr.4113 v2:172.21.15.27:6836/1226920526 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f54d8074f00 con 0x7f54c406c400 2024-09-12T03:22:27.780 INFO:teuthology.orchestra.run.smithi027.stdout:[ 2024-09-12T03:22:27.780 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:22:27.780 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "cephfs" 2024-09-12T03:22:27.780 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:22:27.780 INFO:teuthology.orchestra.run.smithi027.stdout:] 2024-09-12T03:22:27.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:27.782+0000 7f54de913640 1 -- 172.21.15.27:0/2516347860 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f54c406c400 msgr2=0x7f54c406e8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:27.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:27.782+0000 7f54de913640 1 --2- 172.21.15.27:0/2516347860 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f54c406c400 0x7f54c406e8c0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f54c8007fd0 tx=0x7f54c8007de0 comp rx=0 tx=0).stop 2024-09-12T03:22:27.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:27.782+0000 7f54de913640 1 -- 172.21.15.27:0/2516347860 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f54d815b2b0 msgr2=0x7f54d8146080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:27.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:27.782+0000 7f54de913640 1 --2- 172.21.15.27:0/2516347860 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f54d815b2b0 0x7f54d8146080 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f54d000c780 tx=0x7f54d000cc40 comp rx=0 tx=0).stop 2024-09-12T03:22:27.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:27.782+0000 7f54de913640 1 -- 172.21.15.27:0/2516347860 shutdown_connections 2024-09-12T03:22:27.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:27.782+0000 7f54de913640 1 --2- 172.21.15.27:0/2516347860 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f54c406c400 0x7f54c406e8c0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:27.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:27.782+0000 7f54de913640 1 --2- 172.21.15.27:0/2516347860 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f54d815b2b0 0x7f54d8146080 unknown :-1 s=CLOSED pgs=1265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:27.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:27.782+0000 7f54de913640 1 --2- 172.21.15.27:0/2516347860 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f54d8072d70 0x7f54d81592b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:27.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:27.782+0000 7f54de913640 1 --2- 172.21.15.27:0/2516347860 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f54d806e9b0 0x7f54d814dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:27.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:27.782+0000 7f54de913640 1 -- 172.21.15.27:0/2516347860 >> 172.21.15.27:0/2516347860 conn(0x7f54d8062c60 msgr2=0x7f54d815f800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:27.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:27.782+0000 7f54de913640 1 -- 172.21.15.27:0/2516347860 shutdown_connections 2024-09-12T03:22:27.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:27.782+0000 7f54de913640 1 -- 172.21.15.27:0/2516347860 wait complete. 2024-09-12T03:22:27.793 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-12T03:22:27.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:27.898+0000 7f1477ced640 1 Processor -- start 2024-09-12T03:22:27.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:27.899+0000 7f1477ced640 1 -- start start 2024-09-12T03:22:27.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:27.900+0000 7f1477ced640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f14700d3790 0x7f14700d1da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:27.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:27.900+0000 7f1477ced640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f14700d2e30 con 0x7f1470162040 2024-09-12T03:22:27.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:27.900+0000 7f1477ced640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1470162420 con 0x7f147006dfd0 2024-09-12T03:22:27.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:27.900+0000 7f1477ced640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f14700d30d0 con 0x7f14700d3790 2024-09-12T03:22:27.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:27.900+0000 7f147628a640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f14700d3790 0x7f14700d1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:27.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:27.900+0000 7f147628a640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f14700d3790 0x7f14700d1da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49490/0 (socket says 172.21.15.27:49490) 2024-09-12T03:22:27.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:27.900+0000 7f147628a640 1 -- 172.21.15.27:0/2802031283 learned_addr learned my addr 172.21.15.27:0/2802031283 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:22:27.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:27.900+0000 7f1475a89640 1 --1- 172.21.15.27:0/2802031283 >> v1:172.21.15.27:6789/0 conn(0x7f1470162040 0x7f147006d8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:42858/0 (socket says 172.21.15.27:42858) 2024-09-12T03:22:27.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:27.900+0000 7f147628a640 1 -- 172.21.15.27:0/2802031283 >> v1:172.21.15.45:6789/0 conn(0x7f147006dfd0 legacy=0x7f14700cf320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:27.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:27.900+0000 7f147628a640 1 -- 172.21.15.27:0/2802031283 >> v1:172.21.15.27:6789/0 conn(0x7f1470162040 legacy=0x7f147006d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:27.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:27.900+0000 7f147628a640 1 -- 172.21.15.27:0/2802031283 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14700d33b0 con 0x7f14700d3790 2024-09-12T03:22:27.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:27.900+0000 7f1474a87640 1 -- 172.21.15.27:0/2802031283 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1664077623 0 0) 0x7f14700d2e30 con 0x7f1470162040 2024-09-12T03:22:27.902 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:27.901+0000 7f147628a640 1 --2- 172.21.15.27:0/2802031283 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f14700d3790 0x7f14700d1da0 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7f1460009e50 tx=0x7f146002a7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91168d60fadfe168 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:27.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:27.901+0000 7f1474a87640 1 -- 172.21.15.27:0/2802031283 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1460036070 con 0x7f14700d3790 2024-09-12T03:22:27.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:27.901+0000 7f1474a87640 1 -- 172.21.15.27:0/2802031283 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1460031600 con 0x7f14700d3790 2024-09-12T03:22:27.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:27.901+0000 7f1474a87640 1 -- 172.21.15.27:0/2802031283 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14600319f0 con 0x7f14700d3790 2024-09-12T03:22:27.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:27.901+0000 7f1477ced640 1 -- 172.21.15.27:0/2802031283 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f14700d3790 msgr2=0x7f14700d1da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:27.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:27.901+0000 7f1477ced640 1 --2- 172.21.15.27:0/2802031283 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f14700d3790 0x7f14700d1da0 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7f1460009e50 tx=0x7f146002a7c0 comp rx=0 tx=0).stop 2024-09-12T03:22:27.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:27.901+0000 7f1477ced640 1 -- 172.21.15.27:0/2802031283 shutdown_connections 2024-09-12T03:22:27.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:27.901+0000 7f1477ced640 1 --2- 172.21.15.27:0/2802031283 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f14700d3790 0x7f14700d1da0 unknown :-1 s=CLOSED pgs=1267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:27.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:27.902+0000 7f1477ced640 1 -- 172.21.15.27:0/2802031283 >> 172.21.15.27:0/2802031283 conn(0x7f14700b6700 msgr2=0x7f14700b6b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:27.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:27.902+0000 7f1477ced640 1 -- 172.21.15.27:0/2802031283 shutdown_connections 2024-09-12T03:22:27.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:27.902+0000 7f1477ced640 1 -- 172.21.15.27:0/2802031283 wait complete. 2024-09-12T03:22:27.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:27.903+0000 7f1477ced640 1 Processor -- start 2024-09-12T03:22:27.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:27.903+0000 7f1477ced640 1 -- start start 2024-09-12T03:22:27.905 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:27.903+0000 7f1477ced640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f147006dfd0 0x7f14701585e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:27.905 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:27.903+0000 7f1477ced640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f14700d3790 0x7f147015b160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:27.906 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:27.903+0000 7f1477ced640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1470162040 0x7f147014b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:27.906 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:27.903+0000 7f1477ced640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f14700d4150 con 0x7f14700d3790 2024-09-12T03:22:27.906 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:27.903+0000 7f1477ced640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f14700d4530 con 0x7f147006dfd0 2024-09-12T03:22:27.906 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:27.903+0000 7f1477ced640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f14700d42d0 con 0x7f1470162040 2024-09-12T03:22:27.906 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:27.904+0000 7f1475288640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f14700d3790 0x7f147015b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:27.906 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:27.904+0000 7f1475288640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f14700d3790 0x7f147015b160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49492/0 (socket says 172.21.15.27:49492) 2024-09-12T03:22:27.906 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:27.904+0000 7f1475a89640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f147006dfd0 0x7f14701585e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:27.906 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:27.904+0000 7f1475288640 1 -- 172.21.15.27:0/122439768 learned_addr learned my addr 172.21.15.27:0/122439768 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:22:27.906 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:27.904+0000 7f147628a640 1 --2- 172.21.15.27:0/122439768 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1470162040 0x7f147014b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:27.906 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:27.904+0000 7f1475288640 1 -- 172.21.15.27:0/122439768 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1470162040 msgr2=0x7f147014b3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:27.906 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:27.904+0000 7f1475288640 1 --2- 172.21.15.27:0/122439768 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1470162040 0x7f147014b3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:27.906 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:27.904+0000 7f1475288640 1 -- 172.21.15.27:0/122439768 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f147006dfd0 msgr2=0x7f14701585e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:27.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:27.904+0000 7f1475288640 1 --2- 172.21.15.27:0/122439768 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f147006dfd0 0x7f14701585e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:27.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:27.904+0000 7f1475288640 1 -- 172.21.15.27:0/122439768 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14700cd700 con 0x7f14700d3790 2024-09-12T03:22:27.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:27.904+0000 7f1475288640 1 --2- 172.21.15.27:0/122439768 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f14700d3790 0x7f147015b160 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f146400c450 tx=0x7f146400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:27.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:27.904+0000 7f146e7fc640 1 -- 172.21.15.27:0/122439768 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14640052d0 con 0x7f14700d3790 2024-09-12T03:22:27.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:27.904+0000 7f146e7fc640 1 -- 172.21.15.27:0/122439768 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1464005470 con 0x7f14700d3790 2024-09-12T03:22:27.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:27.904+0000 7f1477ced640 1 -- 172.21.15.27:0/122439768 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f147015b6a0 con 0x7f14700d3790 2024-09-12T03:22:27.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:27.904+0000 7f146e7fc640 1 -- 172.21.15.27:0/122439768 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1464012870 con 0x7f14700d3790 2024-09-12T03:22:27.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:27.904+0000 7f1477ced640 1 -- 172.21.15.27:0/122439768 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14700cd420 con 0x7f14700d3790 2024-09-12T03:22:27.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:27.905+0000 7f1477ced640 1 -- 172.21.15.27:0/122439768 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f147006e6e0 con 0x7f14700d3790 2024-09-12T03:22:27.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:27.906+0000 7f146e7fc640 1 -- 172.21.15.27:0/122439768 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 21) ==== 91008+0+0 (secure 0 0 0) 0x7f1464012a10 con 0x7f14700d3790 2024-09-12T03:22:27.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:27.906+0000 7f146e7fc640 1 --2- 172.21.15.27:0/122439768 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f145c06c3d0 0x7f145c06e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:27.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:27.907+0000 7f1475a89640 1 --2- 172.21.15.27:0/122439768 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f145c06c3d0 0x7f145c06e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:27.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:27.907+0000 7f146e7fc640 1 -- 172.21.15.27:0/122439768 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 5765+0+0 (secure 0 0 0) 0x7f146408b470 con 0x7f14700d3790 2024-09-12T03:22:27.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:27.907+0000 7f1475a89640 1 --2- 172.21.15.27:0/122439768 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f145c06c3d0 0x7f145c06e890 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f1468000f60 tx=0x7f1468007e40 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:27.910 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:27.911+0000 7f146e7fc640 1 -- 172.21.15.27:0/122439768 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f1464058e90 con 0x7f14700d3790 2024-09-12T03:22:28.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.089+0000 7f1477ced640 1 -- 172.21.15.27:0/122439768 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f14701616a0 con 0x7f14700d3790 2024-09-12T03:22:28.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.089+0000 7f146e7fc640 1 -- 172.21.15.27:0/122439768 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v24)=0 v24) ==== 122+0+0 (secure 0 0 0) 0x7f146405d190 con 0x7f14700d3790 2024-09-12T03:22:28.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.092+0000 7f1477ced640 1 -- 172.21.15.27:0/122439768 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f145c06c3d0 msgr2=0x7f145c06e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:28.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.092+0000 7f1477ced640 1 --2- 172.21.15.27:0/122439768 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f145c06c3d0 0x7f145c06e890 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f1468000f60 tx=0x7f1468007e40 comp rx=0 tx=0).stop 2024-09-12T03:22:28.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.092+0000 7f1477ced640 1 -- 172.21.15.27:0/122439768 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f14700d3790 msgr2=0x7f147015b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:28.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.092+0000 7f1477ced640 1 --2- 172.21.15.27:0/122439768 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f14700d3790 0x7f147015b160 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f146400c450 tx=0x7f146400c910 comp rx=0 tx=0).stop 2024-09-12T03:22:28.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.092+0000 7f1477ced640 1 -- 172.21.15.27:0/122439768 shutdown_connections 2024-09-12T03:22:28.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.092+0000 7f1477ced640 1 --2- 172.21.15.27:0/122439768 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1470162040 0x7f147014b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:28.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.092+0000 7f1477ced640 1 --2- 172.21.15.27:0/122439768 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f145c06c3d0 0x7f145c06e890 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:28.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.092+0000 7f1477ced640 1 --2- 172.21.15.27:0/122439768 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f14700d3790 0x7f147015b160 unknown :-1 s=CLOSED pgs=1268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:28.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.092+0000 7f1477ced640 1 --2- 172.21.15.27:0/122439768 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f147006dfd0 0x7f14701585e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:28.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.092+0000 7f1477ced640 1 -- 172.21.15.27:0/122439768 >> 172.21.15.27:0/122439768 conn(0x7f14700b6700 msgr2=0x7f14701604e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:28.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.092+0000 7f1477ced640 1 -- 172.21.15.27:0/122439768 shutdown_connections 2024-09-12T03:22:28.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.092+0000 7f1477ced640 1 -- 172.21.15.27:0/122439768 wait complete. 2024-09-12T03:22:28.103 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create non_existent_subvolume subvolume_1 2024-09-12T03:22:28.209 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.209+0000 7f5c1acd7640 1 Processor -- start 2024-09-12T03:22:28.209 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.209+0000 7f5c1acd7640 1 -- start start 2024-09-12T03:22:28.212 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.210+0000 7f5c1acd7640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f5c141645c0 0x7f5c141669b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:28.213 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.210+0000 7f5c1acd7640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5c140b4720 con 0x7f5c140af9d0 2024-09-12T03:22:28.213 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.210+0000 7f5c1acd7640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5c140ce5e0 con 0x7f5c1415ec30 2024-09-12T03:22:28.213 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.210+0000 7f5c1acd7640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f5c140d02d0 con 0x7f5c141645c0 2024-09-12T03:22:28.213 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.210+0000 7f5c13fff640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f5c140af9d0 0x7f5c140b4010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:42874/0 (socket says 172.21.15.27:42874) 2024-09-12T03:22:28.213 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.210+0000 7f5c13fff640 1 -- 172.21.15.27:0/1141971684 learned_addr learned my addr 172.21.15.27:0/1141971684 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:22:28.213 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.210+0000 7f5c137fe640 1 -- 172.21.15.27:0/1141971684 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1852868226 0 0) 0x7f5c140b4720 con 0x7f5c140af9d0 2024-09-12T03:22:28.213 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.210+0000 7f5c137fe640 1 -- 172.21.15.27:0/1141971684 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5c00003520 con 0x7f5c140af9d0 2024-09-12T03:22:28.213 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.210+0000 7f5c19274640 1 --2- 172.21.15.27:0/1141971684 >> v2:172.21.15.45:3300/0 conn(0x7f5c141645c0 0x7f5c141669b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:28.213 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.211+0000 7f5c137fe640 1 -- 172.21.15.27:0/1141971684 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1125104250 0 0) 0x7f5c00003520 con 0x7f5c140af9d0 2024-09-12T03:22:28.213 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.211+0000 7f5c137fe640 1 -- 172.21.15.27:0/1141971684 >> v2:172.21.15.45:3300/0 conn(0x7f5c141645c0 msgr2=0x7f5c141669b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:28.213 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.211+0000 7f5c137fe640 1 --2- 172.21.15.27:0/1141971684 >> v2:172.21.15.45:3300/0 conn(0x7f5c141645c0 0x7f5c141669b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:28.213 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.211+0000 7f5c137fe640 1 -- 172.21.15.27:0/1141971684 >> v1:172.21.15.45:6789/0 conn(0x7f5c1415ec30 legacy=0x7f5c140b3900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:28.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.211+0000 7f5c137fe640 1 -- 172.21.15.27:0/1141971684 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c140ceac0 con 0x7f5c140af9d0 2024-09-12T03:22:28.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.211+0000 7f5c137fe640 1 -- 172.21.15.27:0/1141971684 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 847755213 0 0) 0x7f5bfc0019f0 con 0x7f5c140af9d0 2024-09-12T03:22:28.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.211+0000 7f5c137fe640 1 -- 172.21.15.27:0/1141971684 >> v1:172.21.15.27:6789/0 conn(0x7f5c140af9d0 legacy=0x7f5c140b4010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:28.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.211+0000 7f5c137fe640 1 --2- 172.21.15.27:0/1141971684 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f5c000039b0 0x7f5c00003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:28.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.211+0000 7f5c137fe640 1 --2- 172.21.15.27:0/1141971684 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5c00004800 0x7f5c00004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:28.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.211+0000 7f5c137fe640 1 --2- 172.21.15.27:0/1141971684 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c000051c0 0x7f5c0000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:28.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.211+0000 7f5c137fe640 1 -- 172.21.15.27:0/1141971684 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f5c140d02d0 con 0x7f5c000051c0 2024-09-12T03:22:28.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.211+0000 7f5c137fe640 1 -- 172.21.15.27:0/1141971684 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f5c0000c060 con 0x7f5c00004800 2024-09-12T03:22:28.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.211+0000 7f5c137fe640 1 -- 172.21.15.27:0/1141971684 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f5c0000c260 con 0x7f5c000039b0 2024-09-12T03:22:28.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.211+0000 7f5c19274640 1 --2- 172.21.15.27:0/1141971684 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c000051c0 0x7f5c0000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:28.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.211+0000 7f5c18a73640 1 --2- 172.21.15.27:0/1141971684 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f5c000039b0 0x7f5c00003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:28.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.211+0000 7f5c13fff640 1 --2- 172.21.15.27:0/1141971684 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5c00004800 0x7f5c00004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:28.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.211+0000 7f5c19274640 1 -- 172.21.15.27:0/1141971684 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f5c000039b0 msgr2=0x7f5c00003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:28.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.211+0000 7f5c19274640 1 --2- 172.21.15.27:0/1141971684 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f5c000039b0 0x7f5c00003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:28.215 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.211+0000 7f5c19274640 1 -- 172.21.15.27:0/1141971684 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5c00004800 msgr2=0x7f5c00004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:28.215 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.211+0000 7f5c19274640 1 --2- 172.21.15.27:0/1141971684 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5c00004800 0x7f5c00004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:28.215 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.211+0000 7f5c19274640 1 -- 172.21.15.27:0/1141971684 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c00003520 con 0x7f5c000051c0 2024-09-12T03:22:28.215 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.211+0000 7f5c13fff640 1 --2- 172.21.15.27:0/1141971684 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5c00004800 0x7f5c00004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:22:28.215 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.212+0000 7f5c19274640 1 -- 172.21.15.27:0/1141971684 reap_dead start 2024-09-12T03:22:28.215 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.212+0000 7f5c19274640 1 --2- 172.21.15.27:0/1141971684 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c000051c0 0x7f5c0000b9c0 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f5c14051070 tx=0x7f5c00005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a5da85fcc3d9183 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:28.215 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.212+0000 7f5c137fe640 1 -- 172.21.15.27:0/1141971684 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c0c009690 con 0x7f5c000051c0 2024-09-12T03:22:28.215 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.212+0000 7f5c137fe640 1 -- 172.21.15.27:0/1141971684 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c0c004020 con 0x7f5c000051c0 2024-09-12T03:22:28.215 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.212+0000 7f5c137fe640 1 -- 172.21.15.27:0/1141971684 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c0c0043d0 con 0x7f5c000051c0 2024-09-12T03:22:28.215 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.213+0000 7f5c1acd7640 1 -- 172.21.15.27:0/1141971684 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c000051c0 msgr2=0x7f5c0000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:28.215 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.213+0000 7f5c1acd7640 1 --2- 172.21.15.27:0/1141971684 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c000051c0 0x7f5c0000b9c0 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f5c14051070 tx=0x7f5c00005830 comp rx=0 tx=0).stop 2024-09-12T03:22:28.215 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.213+0000 7f5c1acd7640 1 -- 172.21.15.27:0/1141971684 shutdown_connections 2024-09-12T03:22:28.215 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.213+0000 7f5c1acd7640 1 --2- 172.21.15.27:0/1141971684 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c000051c0 0x7f5c0000b9c0 unknown :-1 s=CLOSED pgs=1270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:28.215 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.213+0000 7f5c1acd7640 1 -- 172.21.15.27:0/1141971684 >> 172.21.15.27:0/1141971684 conn(0x7f5c1406bb30 msgr2=0x7f5c1406bf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:28.216 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.213+0000 7f5c1acd7640 1 -- 172.21.15.27:0/1141971684 shutdown_connections 2024-09-12T03:22:28.216 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.213+0000 7f5c1acd7640 1 -- 172.21.15.27:0/1141971684 wait complete. 2024-09-12T03:22:28.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.214+0000 7f5c1acd7640 1 Processor -- start 2024-09-12T03:22:28.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.214+0000 7f5c1acd7640 1 -- start start 2024-09-12T03:22:28.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.215+0000 7f5c1acd7640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f5c000051c0 0x7f5c140ae830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:28.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.215+0000 7f5c1acd7640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c1406d240 0x7f5c140ac840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:28.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.215+0000 7f5c1acd7640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5c1406d6a0 0x7f5c140acd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:28.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.215+0000 7f5c1acd7640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f5c1406c8a0 con 0x7f5c1406d240 2024-09-12T03:22:28.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.215+0000 7f5c1acd7640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f5c1406c650 con 0x7f5c1406d6a0 2024-09-12T03:22:28.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.215+0000 7f5c1acd7640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f5c140cfff0 con 0x7f5c000051c0 2024-09-12T03:22:28.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.215+0000 7f5c13fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c1406d240 0x7f5c140ac840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:28.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.215+0000 7f5c13fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c1406d240 0x7f5c140ac840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49504/0 (socket says 172.21.15.27:49504) 2024-09-12T03:22:28.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.215+0000 7f5c13fff640 1 -- 172.21.15.27:0/4063779605 learned_addr learned my addr 172.21.15.27:0/4063779605 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:22:28.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.215+0000 7f5c18a73640 1 --2- 172.21.15.27:0/4063779605 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f5c000051c0 0x7f5c140ae830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:28.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.215+0000 7f5c19274640 1 --2- 172.21.15.27:0/4063779605 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5c1406d6a0 0x7f5c140acd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:28.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.215+0000 7f5c13fff640 1 -- 172.21.15.27:0/4063779605 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f5c000051c0 msgr2=0x7f5c140ae830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:28.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.215+0000 7f5c13fff640 1 --2- 172.21.15.27:0/4063779605 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f5c000051c0 0x7f5c140ae830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:28.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.215+0000 7f5c13fff640 1 -- 172.21.15.27:0/4063779605 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5c1406d6a0 msgr2=0x7f5c140acd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:28.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.215+0000 7f5c13fff640 1 --2- 172.21.15.27:0/4063779605 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5c1406d6a0 0x7f5c140acd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:28.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.215+0000 7f5c13fff640 1 -- 172.21.15.27:0/4063779605 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c140ad790 con 0x7f5c1406d240 2024-09-12T03:22:28.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.216+0000 7f5c13fff640 1 --2- 172.21.15.27:0/4063779605 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c1406d240 0x7f5c140ac840 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f5bfc0079d0 tx=0x7f5bfc00e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:28.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.216+0000 7f5c117fa640 1 -- 172.21.15.27:0/4063779605 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bfc00ee50 con 0x7f5c1406d240 2024-09-12T03:22:28.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.216+0000 7f5c117fa640 1 -- 172.21.15.27:0/4063779605 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5bfc018070 con 0x7f5c1406d240 2024-09-12T03:22:28.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.216+0000 7f5c1acd7640 1 -- 172.21.15.27:0/4063779605 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c140ada70 con 0x7f5c1406d240 2024-09-12T03:22:28.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.216+0000 7f5c117fa640 1 -- 172.21.15.27:0/4063779605 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bfc013660 con 0x7f5c1406d240 2024-09-12T03:22:28.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.216+0000 7f5c1acd7640 1 -- 172.21.15.27:0/4063779605 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c140ae030 con 0x7f5c1406d240 2024-09-12T03:22:28.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.217+0000 7f5c1acd7640 1 -- 172.21.15.27:0/4063779605 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c140715f0 con 0x7f5c1406d240 2024-09-12T03:22:28.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.217+0000 7f5c117fa640 1 -- 172.21.15.27:0/4063779605 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 21) ==== 91008+0+0 (secure 0 0 0) 0x7f5bfc013800 con 0x7f5c1406d240 2024-09-12T03:22:28.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.218+0000 7f5c117fa640 1 --2- 172.21.15.27:0/4063779605 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f5bf406c3d0 0x7f5bf406e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:28.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.218+0000 7f5c117fa640 1 -- 172.21.15.27:0/4063779605 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 5765+0+0 (secure 0 0 0) 0x7f5bfc08bec0 con 0x7f5c1406d240 2024-09-12T03:22:28.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.218+0000 7f5c18a73640 1 --2- 172.21.15.27:0/4063779605 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f5bf406c3d0 0x7f5bf406e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:28.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.218+0000 7f5c18a73640 1 --2- 172.21.15.27:0/4063779605 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f5bf406c3d0 0x7f5bf406e890 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f5c04000c00 tx=0x7f5c04009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:28.222 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.222+0000 7f5c117fa640 1 -- 172.21.15.27:0/4063779605 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f5bfc0598e0 con 0x7f5c1406d240 2024-09-12T03:22:28.426 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.426+0000 7f5c1acd7640 1 -- 172.21.15.27:0/4063779605 --> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "non_existent_subvolume", "sub_name": "subvolume_1", "target": ["mon-mgr", ""]}) -- 0x7f5c1406f240 con 0x7f5bf406c3d0 2024-09-12T03:22:28.427 INFO:tasks.ceph.mgr.y.smithi027.stderr:2024-09-12T03:22:28.427+0000 7f325d8e3640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-12T03:22:28.427 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.427+0000 7f5c117fa640 1 -- 172.21.15.27:0/4063779605 <== mgr.4113 v2:172.21.15.27:6836/1226920526 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7f5c1406f240 con 0x7f5bf406c3d0 2024-09-12T03:22:28.427 INFO:teuthology.orchestra.run.smithi027.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-12T03:22:28.433 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-12T03:22:28.434 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm non_existent_subvolume subvolume_1 2024-09-12T03:22:28.539 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.539+0000 7f386eaac640 1 Processor -- start 2024-09-12T03:22:28.539 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.539+0000 7f386eaac640 1 -- start start 2024-09-12T03:22:28.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.540+0000 7f386eaac640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f386815ec30 0x7f38680b3900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:28.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.540+0000 7f386eaac640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f38680af9d0 0x7f38680b3e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:28.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.540+0000 7f386eaac640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f38680ce5e0 con 0x7f38680b4380 2024-09-12T03:22:28.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.540+0000 7f386eaac640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f38680d02d0 con 0x7f386815ec30 2024-09-12T03:22:28.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.540+0000 7f386eaac640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f386806c650 con 0x7f38680af9d0 2024-09-12T03:22:28.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.540+0000 7f386c848640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f386815ec30 0x7f38680b3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:28.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.540+0000 7f386d049640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f38680b4380 0x7f386815f2a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:42882/0 (socket says 172.21.15.27:42882) 2024-09-12T03:22:28.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.540+0000 7f386d049640 1 -- 172.21.15.27:0/4292258505 learned_addr learned my addr 172.21.15.27:0/4292258505 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:22:28.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.540+0000 7f3867fff640 1 --2- 172.21.15.27:0/4292258505 >> v2:172.21.15.45:3300/0 conn(0x7f38680af9d0 0x7f38680b3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:28.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.540+0000 7f386c848640 1 -- 172.21.15.27:0/4292258505 >> v2:172.21.15.45:3300/0 conn(0x7f38680af9d0 msgr2=0x7f38680b3e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:28.543 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.540+0000 7f386c848640 1 --2- 172.21.15.27:0/4292258505 >> v2:172.21.15.45:3300/0 conn(0x7f38680af9d0 0x7f38680b3e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:28.543 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.540+0000 7f386c848640 1 -- 172.21.15.27:0/4292258505 >> v1:172.21.15.27:6789/0 conn(0x7f38680b4380 legacy=0x7f386815f2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:28.543 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.540+0000 7f386c848640 1 -- 172.21.15.27:0/4292258505 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38680ceac0 con 0x7f386815ec30 2024-09-12T03:22:28.543 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.540+0000 7f38677fe640 1 -- 172.21.15.27:0/4292258505 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3437107779 0 0) 0x7f38680ce5e0 con 0x7f38680b4380 2024-09-12T03:22:28.543 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.540+0000 7f3867fff640 1 --2- 172.21.15.27:0/4292258505 >> v2:172.21.15.45:3300/0 conn(0x7f38680af9d0 0x7f38680b3e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:22:28.543 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.541+0000 7f386c848640 1 --2- 172.21.15.27:0/4292258505 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f386815ec30 0x7f38680b3900 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f38680cc7d0 tx=0x7f385802a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee293ad071077833 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:28.543 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.541+0000 7f38677fe640 1 -- 172.21.15.27:0/4292258505 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3858007740 con 0x7f386815ec30 2024-09-12T03:22:28.543 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.541+0000 7f38677fe640 1 -- 172.21.15.27:0/4292258505 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38580078e0 con 0x7f386815ec30 2024-09-12T03:22:28.543 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.541+0000 7f38677fe640 1 -- 172.21.15.27:0/4292258505 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3858007cb0 con 0x7f386815ec30 2024-09-12T03:22:28.543 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.541+0000 7f386eaac640 1 -- 172.21.15.27:0/4292258505 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f386815ec30 msgr2=0x7f38680b3900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:28.543 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.541+0000 7f386eaac640 1 --2- 172.21.15.27:0/4292258505 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f386815ec30 0x7f38680b3900 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f38680cc7d0 tx=0x7f385802a680 comp rx=0 tx=0).stop 2024-09-12T03:22:28.543 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.542+0000 7f386eaac640 1 -- 172.21.15.27:0/4292258505 shutdown_connections 2024-09-12T03:22:28.544 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.542+0000 7f386eaac640 1 --2- 172.21.15.27:0/4292258505 >> v2:172.21.15.45:3300/0 conn(0x7f38680af9d0 0x7f38680b3e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:28.544 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.542+0000 7f386eaac640 1 --2- 172.21.15.27:0/4292258505 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f386815ec30 0x7f38680b3900 unknown :-1 s=CLOSED pgs=1273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:28.544 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.542+0000 7f386eaac640 1 -- 172.21.15.27:0/4292258505 >> 172.21.15.27:0/4292258505 conn(0x7f386806bb30 msgr2=0x7f386806bf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:28.544 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.542+0000 7f386eaac640 1 -- 172.21.15.27:0/4292258505 shutdown_connections 2024-09-12T03:22:28.544 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.542+0000 7f386eaac640 1 -- 172.21.15.27:0/4292258505 wait complete. 2024-09-12T03:22:28.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.543+0000 7f386eaac640 1 Processor -- start 2024-09-12T03:22:28.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.543+0000 7f386eaac640 1 -- start start 2024-09-12T03:22:28.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.544+0000 7f386eaac640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f38680af9d0 0x7f38681f2300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:28.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.544+0000 7f386eaac640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f38680b4380 0x7f38680cecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:28.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.544+0000 7f386eaac640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f386815ec30 0x7f38680cf210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:28.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.544+0000 7f386eaac640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f38680cfff0 con 0x7f38680b4380 2024-09-12T03:22:28.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.544+0000 7f386eaac640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f386806c8a0 con 0x7f386815ec30 2024-09-12T03:22:28.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.544+0000 7f386eaac640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f38680715f0 con 0x7f38680af9d0 2024-09-12T03:22:28.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.544+0000 7f3867fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f38680b4380 0x7f38680cecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:28.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.544+0000 7f3867fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f38680b4380 0x7f38680cecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49532/0 (socket says 172.21.15.27:49532) 2024-09-12T03:22:28.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.544+0000 7f3867fff640 1 -- 172.21.15.27:0/747497564 learned_addr learned my addr 172.21.15.27:0/747497564 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:22:28.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.544+0000 7f386d049640 1 --2- 172.21.15.27:0/747497564 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f386815ec30 0x7f38680cf210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:28.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.544+0000 7f386c848640 1 --2- 172.21.15.27:0/747497564 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f38680af9d0 0x7f38681f2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:28.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.544+0000 7f3867fff640 1 -- 172.21.15.27:0/747497564 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f38680af9d0 msgr2=0x7f38681f2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:28.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.544+0000 7f3867fff640 1 --2- 172.21.15.27:0/747497564 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f38680af9d0 0x7f38681f2300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:28.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.544+0000 7f3867fff640 1 -- 172.21.15.27:0/747497564 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f386815ec30 msgr2=0x7f38680cf210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:28.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.544+0000 7f3867fff640 1 --2- 172.21.15.27:0/747497564 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f386815ec30 0x7f38680cf210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:28.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.544+0000 7f3867fff640 1 -- 172.21.15.27:0/747497564 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38681f32a0 con 0x7f38680b4380 2024-09-12T03:22:28.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.544+0000 7f386d049640 1 --2- 172.21.15.27:0/747497564 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f386815ec30 0x7f38680cf210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:22:28.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.544+0000 7f386c848640 1 --2- 172.21.15.27:0/747497564 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f38680af9d0 0x7f38681f2300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:22:28.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.544+0000 7f3867fff640 1 --2- 172.21.15.27:0/747497564 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f38680b4380 0x7f38680cecd0 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f385000c7f0 tx=0x7f385000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:28.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.544+0000 7f38657fa640 1 -- 172.21.15.27:0/747497564 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3850015070 con 0x7f38680b4380 2024-09-12T03:22:28.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.545+0000 7f38657fa640 1 -- 172.21.15.27:0/747497564 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3850007820 con 0x7f38680b4380 2024-09-12T03:22:28.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.545+0000 7f38657fa640 1 -- 172.21.15.27:0/747497564 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3850007bd0 con 0x7f38680b4380 2024-09-12T03:22:28.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.545+0000 7f386eaac640 1 -- 172.21.15.27:0/747497564 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38681f2ce0 con 0x7f38680b4380 2024-09-12T03:22:28.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.545+0000 7f386eaac640 1 -- 172.21.15.27:0/747497564 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3868155850 con 0x7f38680b4380 2024-09-12T03:22:28.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.545+0000 7f386eaac640 1 -- 172.21.15.27:0/747497564 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38680b3900 con 0x7f38680b4380 2024-09-12T03:22:28.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.546+0000 7f38657fa640 1 -- 172.21.15.27:0/747497564 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 21) ==== 91008+0+0 (secure 0 0 0) 0x7f3850018070 con 0x7f38680b4380 2024-09-12T03:22:28.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.546+0000 7f38657fa640 1 --2- 172.21.15.27:0/747497564 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f384806c3d0 0x7f384806e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:28.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.547+0000 7f38657fa640 1 -- 172.21.15.27:0/747497564 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 5765+0+0 (secure 0 0 0) 0x7f385008ba80 con 0x7f38680b4380 2024-09-12T03:22:28.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.547+0000 7f386c848640 1 --2- 172.21.15.27:0/747497564 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f384806c3d0 0x7f384806e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:28.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.547+0000 7f386c848640 1 --2- 172.21.15.27:0/747497564 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f384806c3d0 0x7f384806e890 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f3858034000 tx=0x7f385802f000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:28.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.551+0000 7f38657fa640 1 -- 172.21.15.27:0/747497564 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f3850059b00 con 0x7f38680b4380 2024-09-12T03:22:28.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.750+0000 7f386eaac640 1 -- 172.21.15.27:0/747497564 --> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "non_existent_subvolume", "sub_name": "subvolume_1", "target": ["mon-mgr", ""]}) -- 0x7f386806f240 con 0x7f384806c3d0 2024-09-12T03:22:28.751 INFO:tasks.ceph.mgr.y.smithi027.stderr:2024-09-12T03:22:28.751+0000 7f325d8e3640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-12T03:22:28.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.751+0000 7f38657fa640 1 -- 172.21.15.27:0/747497564 <== mgr.4113 v2:172.21.15.27:6836/1226920526 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7f386806f240 con 0x7f384806c3d0 2024-09-12T03:22:28.751 INFO:teuthology.orchestra.run.smithi027.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-12T03:22:28.756 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-12T03:22:28.757 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath non_existent_subvolume subvolume_1 2024-09-12T03:22:28.862 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.862+0000 7efe08aeb640 1 Processor -- start 2024-09-12T03:22:28.862 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.862+0000 7efe08aeb640 1 -- start start 2024-09-12T03:22:28.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.863+0000 7efe08aeb640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7efe040b6240 0x7efe040b6620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:28.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.863+0000 7efe08aeb640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7efe040d3c20 0x7efe040d4040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:28.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.863+0000 7efe08aeb640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efe040b6b60 con 0x7efe0406d960 2024-09-12T03:22:28.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.863+0000 7efe08aeb640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7efe040d2930 con 0x7efe040d3c20 2024-09-12T03:22:28.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.863+0000 7efe08aeb640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7efe040d2c10 con 0x7efe040b6240 2024-09-12T03:22:28.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.863+0000 7efe01d9b640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7efe040d3c20 0x7efe040d4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:28.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.863+0000 7efe01d9b640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7efe040d3c20 0x7efe040d4040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49546/0 (socket says 172.21.15.27:49546) 2024-09-12T03:22:28.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.863+0000 7efe01d9b640 1 -- 172.21.15.27:0/2172351559 learned_addr learned my addr 172.21.15.27:0/2172351559 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:22:28.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.864+0000 7efe0259c640 1 --2- 172.21.15.27:0/2172351559 >> v2:172.21.15.45:3300/0 conn(0x7efe040b6240 0x7efe040b6620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:28.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.864+0000 7efe01d9b640 1 -- 172.21.15.27:0/2172351559 >> v2:172.21.15.45:3300/0 conn(0x7efe040b6240 msgr2=0x7efe040b6620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:28.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.864+0000 7efe01d9b640 1 --2- 172.21.15.27:0/2172351559 >> v2:172.21.15.45:3300/0 conn(0x7efe040b6240 0x7efe040b6620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:28.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.864+0000 7efe01d9b640 1 -- 172.21.15.27:0/2172351559 >> v1:172.21.15.45:6789/0 conn(0x7efe0406d960 legacy=0x7efe0406ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:28.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.864+0000 7efe01d9b640 1 -- 172.21.15.27:0/2172351559 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe040d2ef0 con 0x7efe040d3c20 2024-09-12T03:22:28.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.864+0000 7efe01d9b640 1 --2- 172.21.15.27:0/2172351559 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efe040d3c20 0x7efe040d4040 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7efe040d58c0 tx=0x7efdec02a2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a3f00c4139c4a4b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:28.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.865+0000 7efe0159a640 1 -- 172.21.15.27:0/2172351559 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdec037070 con 0x7efe040d3c20 2024-09-12T03:22:28.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.865+0000 7efe0159a640 1 -- 172.21.15.27:0/2172351559 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efdec032440 con 0x7efe040d3c20 2024-09-12T03:22:28.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.865+0000 7efe0159a640 1 -- 172.21.15.27:0/2172351559 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdec032810 con 0x7efe040d3c20 2024-09-12T03:22:28.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.865+0000 7efe08aeb640 1 -- 172.21.15.27:0/2172351559 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efe040d3c20 msgr2=0x7efe040d4040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:28.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.865+0000 7efe08aeb640 1 --2- 172.21.15.27:0/2172351559 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efe040d3c20 0x7efe040d4040 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7efe040d58c0 tx=0x7efdec02a2c0 comp rx=0 tx=0).stop 2024-09-12T03:22:28.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.865+0000 7efe08aeb640 1 -- 172.21.15.27:0/2172351559 shutdown_connections 2024-09-12T03:22:28.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.865+0000 7efe08aeb640 1 --2- 172.21.15.27:0/2172351559 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efe040d3c20 0x7efe040d4040 unknown :-1 s=CLOSED pgs=1275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:28.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.865+0000 7efe08aeb640 1 --2- 172.21.15.27:0/2172351559 >> v2:172.21.15.45:3300/0 conn(0x7efe040b6240 0x7efe040b6620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:28.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.865+0000 7efe08aeb640 1 -- 172.21.15.27:0/2172351559 >> 172.21.15.27:0/2172351559 conn(0x7efe040d1420 msgr2=0x7efe040d1830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:28.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.865+0000 7efe08aeb640 1 -- 172.21.15.27:0/2172351559 shutdown_connections 2024-09-12T03:22:28.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.865+0000 7efe08aeb640 1 -- 172.21.15.27:0/2172351559 wait complete. 2024-09-12T03:22:28.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.867+0000 7efe08aeb640 1 Processor -- start 2024-09-12T03:22:28.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.867+0000 7efe08aeb640 1 -- start start 2024-09-12T03:22:28.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.867+0000 7efe08aeb640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efe0406d960 0x7efe04157f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:28.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.867+0000 7efe08aeb640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7efe040b6240 0x7efe0415d200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:28.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.867+0000 7efe08aeb640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efe040d3c20 0x7efe0415d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:28.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.867+0000 7efe08aeb640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7efe0415ece0 con 0x7efe040d3c20 2024-09-12T03:22:28.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.867+0000 7efe08aeb640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7efe0415eb60 con 0x7efe0406d960 2024-09-12T03:22:28.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.867+0000 7efe08aeb640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7efe040cfc70 con 0x7efe040b6240 2024-09-12T03:22:28.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.867+0000 7efe02d9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efe040d3c20 0x7efe0415d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:28.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.868+0000 7efe02d9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efe040d3c20 0x7efe0415d740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49550/0 (socket says 172.21.15.27:49550) 2024-09-12T03:22:28.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.868+0000 7efe02d9d640 1 -- 172.21.15.27:0/317542164 learned_addr learned my addr 172.21.15.27:0/317542164 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:22:28.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.868+0000 7efe0259c640 1 --2- 172.21.15.27:0/317542164 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efe0406d960 0x7efe04157f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:28.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.868+0000 7efe01d9b640 1 --2- 172.21.15.27:0/317542164 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7efe040b6240 0x7efe0415d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:28.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.868+0000 7efe02d9d640 1 -- 172.21.15.27:0/317542164 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7efe040b6240 msgr2=0x7efe0415d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:28.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.868+0000 7efe02d9d640 1 --2- 172.21.15.27:0/317542164 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7efe040b6240 0x7efe0415d200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:28.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.868+0000 7efe02d9d640 1 -- 172.21.15.27:0/317542164 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efe0406d960 msgr2=0x7efe04157f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:28.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.868+0000 7efe02d9d640 1 --2- 172.21.15.27:0/317542164 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efe0406d960 0x7efe04157f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:28.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.868+0000 7efe02d9d640 1 -- 172.21.15.27:0/317542164 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe041584b0 con 0x7efe040d3c20 2024-09-12T03:22:28.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.868+0000 7efe02d9d640 1 --2- 172.21.15.27:0/317542164 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efe040d3c20 0x7efe0415d740 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7efdfc00c410 tx=0x7efdfc00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:28.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.868+0000 7efdfaffd640 1 -- 172.21.15.27:0/317542164 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdfc011790 con 0x7efe040d3c20 2024-09-12T03:22:28.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.868+0000 7efdfaffd640 1 -- 172.21.15.27:0/317542164 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efdfc017070 con 0x7efe040d3c20 2024-09-12T03:22:28.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.868+0000 7efdfaffd640 1 -- 172.21.15.27:0/317542164 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdfc011b70 con 0x7efe040d3c20 2024-09-12T03:22:28.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.868+0000 7efe08aeb640 1 -- 172.21.15.27:0/317542164 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe040ce580 con 0x7efe040d3c20 2024-09-12T03:22:28.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.868+0000 7efe08aeb640 1 -- 172.21.15.27:0/317542164 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe040c8490 con 0x7efe040d3c20 2024-09-12T03:22:28.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.869+0000 7efe08aeb640 1 -- 172.21.15.27:0/317542164 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe040b6620 con 0x7efe040d3c20 2024-09-12T03:22:28.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.870+0000 7efdfaffd640 1 -- 172.21.15.27:0/317542164 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 21) ==== 91008+0+0 (secure 0 0 0) 0x7efdfc011d10 con 0x7efe040d3c20 2024-09-12T03:22:28.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.870+0000 7efdfaffd640 1 --2- 172.21.15.27:0/317542164 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7efde406c3d0 0x7efde406e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:28.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.870+0000 7efdfaffd640 1 -- 172.21.15.27:0/317542164 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 5765+0+0 (secure 0 0 0) 0x7efdfc08b3d0 con 0x7efe040d3c20 2024-09-12T03:22:28.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.871+0000 7efe0259c640 1 --2- 172.21.15.27:0/317542164 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7efde406c3d0 0x7efde406e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:28.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.871+0000 7efe0259c640 1 --2- 172.21.15.27:0/317542164 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7efde406c3d0 0x7efde406e890 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7efdf400f510 tx=0x7efdf40077b0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:28.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:28.875+0000 7efdfaffd640 1 -- 172.21.15.27:0/317542164 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7efdfc021020 con 0x7efe040d3c20 2024-09-12T03:22:29.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.079+0000 7efe08aeb640 1 -- 172.21.15.27:0/317542164 --> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "non_existent_subvolume", "sub_name": "subvolume_1", "target": ["mon-mgr", ""]}) -- 0x7efe040d1b40 con 0x7efde406c3d0 2024-09-12T03:22:29.080 INFO:tasks.ceph.mgr.y.smithi027.stderr:2024-09-12T03:22:29.081+0000 7f325d8e3640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-12T03:22:29.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.081+0000 7efdfaffd640 1 -- 172.21.15.27:0/317542164 <== mgr.4113 v2:172.21.15.27:6836/1226920526 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7efe040d1b40 con 0x7efde406c3d0 2024-09-12T03:22:29.081 INFO:teuthology.orchestra.run.smithi027.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-12T03:22:29.086 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-12T03:22:29.087 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume info non_existent_subvolume subvolume_1 2024-09-12T03:22:29.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.192+0000 7f5852fb7640 1 Processor -- start 2024-09-12T03:22:29.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.192+0000 7f5852fb7640 1 -- start start 2024-09-12T03:22:29.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.193+0000 7f5852fb7640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f584c162040 0x7f584c0d1da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:29.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.193+0000 7f5852fb7640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f584c0d2e30 con 0x7f584c0d3790 2024-09-12T03:22:29.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.193+0000 7f5852fb7640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f584c162420 con 0x7f584c06dfd0 2024-09-12T03:22:29.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.193+0000 7f5852fb7640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f584c0d30d0 con 0x7f584c162040 2024-09-12T03:22:29.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.193+0000 7f584bfff640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f584c0d3790 0x7f584c06d8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:42892/0 (socket says 172.21.15.27:42892) 2024-09-12T03:22:29.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.193+0000 7f584bfff640 1 -- 172.21.15.27:0/1596209653 learned_addr learned my addr 172.21.15.27:0/1596209653 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:22:29.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.194+0000 7f5850d53640 1 --2- 172.21.15.27:0/1596209653 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f584c162040 0x7f584c0d1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:29.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.194+0000 7f584b7fe640 1 -- 172.21.15.27:0/1596209653 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 141008307 0 0) 0x7f584c0d2e30 con 0x7f584c0d3790 2024-09-12T03:22:29.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.194+0000 7f584b7fe640 1 -- 172.21.15.27:0/1596209653 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5838003520 con 0x7f584c0d3790 2024-09-12T03:22:29.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.194+0000 7f584b7fe640 1 -- 172.21.15.27:0/1596209653 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3575958708 0 0) 0x7f5838003520 con 0x7f584c0d3790 2024-09-12T03:22:29.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.194+0000 7f584b7fe640 1 -- 172.21.15.27:0/1596209653 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f584c162040 msgr2=0x7f584c0d1da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:29.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.194+0000 7f584b7fe640 1 --2- 172.21.15.27:0/1596209653 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f584c162040 0x7f584c0d1da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:29.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.194+0000 7f584b7fe640 1 -- 172.21.15.27:0/1596209653 >> v1:172.21.15.45:6789/0 conn(0x7f584c06dfd0 legacy=0x7f584c0cf320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:29.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.194+0000 7f584b7fe640 1 -- 172.21.15.27:0/1596209653 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f584c0d33b0 con 0x7f584c0d3790 2024-09-12T03:22:29.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.194+0000 7f584b7fe640 1 -- 172.21.15.27:0/1596209653 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 847755213 0 0) 0x7f583c0019f0 con 0x7f584c0d3790 2024-09-12T03:22:29.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.194+0000 7f584b7fe640 1 -- 172.21.15.27:0/1596209653 >> v1:172.21.15.27:6789/0 conn(0x7f584c0d3790 legacy=0x7f584c06d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:29.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.194+0000 7f584b7fe640 1 --2- 172.21.15.27:0/1596209653 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f58380039e0 0x7f5838003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:29.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.194+0000 7f584b7fe640 1 --2- 172.21.15.27:0/1596209653 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f584c162040 0x7f5838004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:29.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.194+0000 7f584b7fe640 1 --2- 172.21.15.27:0/1596209653 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5838004d40 0x7f58380051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:29.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.194+0000 7f584b7fe640 1 -- 172.21.15.27:0/1596209653 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f584c0d30d0 con 0x7f58380039e0 2024-09-12T03:22:29.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.194+0000 7f584b7fe640 1 -- 172.21.15.27:0/1596209653 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f583800b9c0 con 0x7f5838004d40 2024-09-12T03:22:29.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.194+0000 7f584b7fe640 1 -- 172.21.15.27:0/1596209653 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f583800bbc0 con 0x7f584c162040 2024-09-12T03:22:29.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.194+0000 7f5850d53640 1 --2- 172.21.15.27:0/1596209653 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f58380039e0 0x7f5838003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:29.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.194+0000 7f584bfff640 1 --2- 172.21.15.27:0/1596209653 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f584c162040 0x7f5838004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:29.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.194+0000 7f5851554640 1 --2- 172.21.15.27:0/1596209653 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5838004d40 0x7f58380051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:29.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.195+0000 7f5850d53640 1 -- 172.21.15.27:0/1596209653 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f584c162040 msgr2=0x7f5838004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:29.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.195+0000 7f5850d53640 1 --2- 172.21.15.27:0/1596209653 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f584c162040 0x7f5838004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:29.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.195+0000 7f5850d53640 1 -- 172.21.15.27:0/1596209653 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5838004d40 msgr2=0x7f58380051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:29.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.195+0000 7f5850d53640 1 --2- 172.21.15.27:0/1596209653 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5838004d40 0x7f58380051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:29.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.195+0000 7f5850d53640 1 -- 172.21.15.27:0/1596209653 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5838003520 con 0x7f58380039e0 2024-09-12T03:22:29.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.195+0000 7f5850d53640 1 --2- 172.21.15.27:0/1596209653 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f58380039e0 0x7f5838003dc0 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f5844009ac0 tx=0x7f5838005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1b3f95716b367 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:29.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.195+0000 7f584b7fe640 1 -- 172.21.15.27:0/1596209653 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5844036070 con 0x7f58380039e0 2024-09-12T03:22:29.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.195+0000 7f584b7fe640 1 -- 172.21.15.27:0/1596209653 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5844031400 con 0x7f58380039e0 2024-09-12T03:22:29.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.195+0000 7f584b7fe640 1 -- 172.21.15.27:0/1596209653 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5844009690 con 0x7f58380039e0 2024-09-12T03:22:29.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.196+0000 7f5852fb7640 1 -- 172.21.15.27:0/1596209653 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f58380039e0 msgr2=0x7f5838003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:29.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.196+0000 7f5852fb7640 1 --2- 172.21.15.27:0/1596209653 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f58380039e0 0x7f5838003dc0 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f5844009ac0 tx=0x7f5838005830 comp rx=0 tx=0).stop 2024-09-12T03:22:29.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.196+0000 7f5851554640 1 -- 172.21.15.27:0/1596209653 reap_dead start 2024-09-12T03:22:29.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.196+0000 7f5852fb7640 1 -- 172.21.15.27:0/1596209653 shutdown_connections 2024-09-12T03:22:29.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.196+0000 7f5852fb7640 1 -- 172.21.15.27:0/1596209653 >> 172.21.15.27:0/1596209653 conn(0x7f584c0b6700 msgr2=0x7f584c0b6b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:29.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.196+0000 7f5852fb7640 1 -- 172.21.15.27:0/1596209653 shutdown_connections 2024-09-12T03:22:29.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.196+0000 7f5852fb7640 1 -- 172.21.15.27:0/1596209653 wait complete. 2024-09-12T03:22:29.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.197+0000 7f5852fb7640 1 Processor -- start 2024-09-12T03:22:29.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.197+0000 7f5852fb7640 1 -- start start 2024-09-12T03:22:29.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.198+0000 7f5852fb7640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f58380039e0 0x7f584c15fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:29.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.198+0000 7f5852fb7640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f584c15ee10 0x7f584c1f5620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:29.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.198+0000 7f5852fb7640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f584c0cd350 0x7f584c0cd7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:29.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.198+0000 7f5852fb7640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f584c0d4150 con 0x7f58380039e0 2024-09-12T03:22:29.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.198+0000 7f5852fb7640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f584c0d4530 con 0x7f584c15ee10 2024-09-12T03:22:29.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.198+0000 7f5852fb7640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f584c0d42d0 con 0x7f584c0cd350 2024-09-12T03:22:29.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.198+0000 7f5850d53640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f58380039e0 0x7f584c15fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:29.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.198+0000 7f5850d53640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f58380039e0 0x7f584c15fc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49572/0 (socket says 172.21.15.27:49572) 2024-09-12T03:22:29.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.198+0000 7f5850d53640 1 -- 172.21.15.27:0/642114027 learned_addr learned my addr 172.21.15.27:0/642114027 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:22:29.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.199+0000 7f584bfff640 1 --2- 172.21.15.27:0/642114027 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f584c15ee10 0x7f584c1f5620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:29.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.199+0000 7f5851554640 1 --2- 172.21.15.27:0/642114027 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f584c0cd350 0x7f584c0cd7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:29.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.199+0000 7f5850d53640 1 -- 172.21.15.27:0/642114027 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f584c0cd350 msgr2=0x7f584c0cd7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:29.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.199+0000 7f5850d53640 1 --2- 172.21.15.27:0/642114027 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f584c0cd350 0x7f584c0cd7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:29.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.199+0000 7f5850d53640 1 -- 172.21.15.27:0/642114027 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f584c15ee10 msgr2=0x7f584c1f5620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:29.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.199+0000 7f5850d53640 1 --2- 172.21.15.27:0/642114027 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f584c15ee10 0x7f584c1f5620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:29.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.199+0000 7f5850d53640 1 -- 172.21.15.27:0/642114027 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f584c1f5b60 con 0x7f58380039e0 2024-09-12T03:22:29.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.199+0000 7f5850d53640 1 --2- 172.21.15.27:0/642114027 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f58380039e0 0x7f584c15fc50 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f584402aae0 tx=0x7f5844004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:29.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.199+0000 7f58497fa640 1 -- 172.21.15.27:0/642114027 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5844040020 con 0x7f58380039e0 2024-09-12T03:22:29.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.199+0000 7f58497fa640 1 -- 172.21.15.27:0/642114027 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5844003940 con 0x7f58380039e0 2024-09-12T03:22:29.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.199+0000 7f58497fa640 1 -- 172.21.15.27:0/642114027 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5844036040 con 0x7f58380039e0 2024-09-12T03:22:29.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.199+0000 7f5852fb7640 1 -- 172.21.15.27:0/642114027 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f584c0c8320 con 0x7f58380039e0 2024-09-12T03:22:29.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.199+0000 7f5852fb7640 1 -- 172.21.15.27:0/642114027 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f584c0c88e0 con 0x7f58380039e0 2024-09-12T03:22:29.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.200+0000 7f5852fb7640 1 -- 172.21.15.27:0/642114027 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f584c0d1e30 con 0x7f58380039e0 2024-09-12T03:22:29.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.201+0000 7f58497fa640 1 -- 172.21.15.27:0/642114027 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 21) ==== 91008+0+0 (secure 0 0 0) 0x7f5844043020 con 0x7f58380039e0 2024-09-12T03:22:29.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.201+0000 7f58497fa640 1 --2- 172.21.15.27:0/642114027 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f582c06c3d0 0x7f582c06e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:29.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.201+0000 7f58497fa640 1 -- 172.21.15.27:0/642114027 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 5765+0+0 (secure 0 0 0) 0x7f58440ab4a0 con 0x7f58380039e0 2024-09-12T03:22:29.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.201+0000 7f584bfff640 1 --2- 172.21.15.27:0/642114027 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f582c06c3d0 0x7f582c06e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:29.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.202+0000 7f584bfff640 1 --2- 172.21.15.27:0/642114027 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f582c06c3d0 0x7f582c06e890 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f583c0013d0 tx=0x7f583c006040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:29.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.205+0000 7f58497fa640 1 -- 172.21.15.27:0/642114027 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f584403a370 con 0x7f58380039e0 2024-09-12T03:22:29.407 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.407+0000 7f5852fb7640 1 -- 172.21.15.27:0/642114027 --> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] -- mgr_command(tid 0: {"prefix": "fs subvolume info", "vol_name": "non_existent_subvolume", "sub_name": "subvolume_1", "target": ["mon-mgr", ""]}) -- 0x7f584c06e6e0 con 0x7f582c06c3d0 2024-09-12T03:22:29.408 INFO:tasks.ceph.mgr.y.smithi027.stderr:2024-09-12T03:22:29.409+0000 7f325d8e3640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-12T03:22:29.409 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.409+0000 7f58497fa640 1 -- 172.21.15.27:0/642114027 <== mgr.4113 v2:172.21.15.27:6836/1226920526 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7f584c06e6e0 con 0x7f582c06c3d0 2024-09-12T03:22:29.409 INFO:teuthology.orchestra.run.smithi027.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-12T03:22:29.414 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-12T03:22:29.415 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume resize non_existent_subvolume subvolname_1 inf 2024-09-12T03:22:29.522 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.522+0000 7f9578311640 1 Processor -- start 2024-09-12T03:22:29.522 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.522+0000 7f9578311640 1 -- start start 2024-09-12T03:22:29.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.523+0000 7f9578311640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f95700d1040 0x7f95700d1480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:29.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.523+0000 7f9578311640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f95700b6f70 0x7f95700b7400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:29.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.523+0000 7f9578311640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f95700d07b0 con 0x7f957015c6c0 2024-09-12T03:22:29.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.523+0000 7f9578311640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f957015caa0 con 0x7f95700b6f70 2024-09-12T03:22:29.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.523+0000 7f9578311640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f95700b6510 con 0x7f95700d1040 2024-09-12T03:22:29.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.523+0000 7f95760ad640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f957015c6c0 0x7f95700d00a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:38524/0 (socket says 172.21.15.27:38524) 2024-09-12T03:22:29.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.523+0000 7f95760ad640 1 -- 172.21.15.27:0/1819690115 learned_addr learned my addr 172.21.15.27:0/1819690115 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:22:29.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.523+0000 7f95758ac640 1 --2- 172.21.15.27:0/1819690115 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f95700d1040 0x7f95700d1480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:29.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.523+0000 7f95768ae640 1 --2- 172.21.15.27:0/1819690115 >> v2:172.21.15.45:3300/0 conn(0x7f95700b6f70 0x7f95700b7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:29.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.523+0000 7f95750ab640 1 -- 172.21.15.27:0/1819690115 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1476907317 0 0) 0x7f95700d07b0 con 0x7f957015c6c0 2024-09-12T03:22:29.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.523+0000 7f95750ab640 1 -- 172.21.15.27:0/1819690115 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f955c003520 con 0x7f957015c6c0 2024-09-12T03:22:29.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.523+0000 7f95768ae640 1 -- 172.21.15.27:0/1819690115 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f95700d1040 msgr2=0x7f95700d1480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:29.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.523+0000 7f95768ae640 1 --2- 172.21.15.27:0/1819690115 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f95700d1040 0x7f95700d1480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:29.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.523+0000 7f95768ae640 1 -- 172.21.15.27:0/1819690115 >> v1:172.21.15.27:6789/0 conn(0x7f957015c6c0 legacy=0x7f95700d00a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:29.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.523+0000 7f95750ab640 1 -- 172.21.15.27:0/1819690115 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1592935900 0 0) 0x7f955c003520 con 0x7f957015c6c0 2024-09-12T03:22:29.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.523+0000 7f95768ae640 1 -- 172.21.15.27:0/1819690115 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95700d0a80 con 0x7f95700b6f70 2024-09-12T03:22:29.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.524+0000 7f95768ae640 1 --2- 172.21.15.27:0/1819690115 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f95700b6f70 0x7f95700b7400 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f9568009dc0 tx=0x7f956802a580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=96975335f755159f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:29.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.524+0000 7f95750ab640 1 -- 172.21.15.27:0/1819690115 <== mon.1 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f956c001670 con 0x7f95700b6f70 2024-09-12T03:22:29.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.524+0000 7f95750ab640 1 -- 172.21.15.27:0/1819690115 <== mon.1 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9568007540 con 0x7f95700b6f70 2024-09-12T03:22:29.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.525+0000 7f95750ab640 1 -- 172.21.15.27:0/1819690115 <== mon.1 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9568007910 con 0x7f95700b6f70 2024-09-12T03:22:29.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.525+0000 7f9578311640 1 -- 172.21.15.27:0/1819690115 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f95700b6f70 msgr2=0x7f95700b7400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:29.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.525+0000 7f9578311640 1 --2- 172.21.15.27:0/1819690115 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f95700b6f70 0x7f95700b7400 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f9568009dc0 tx=0x7f956802a580 comp rx=0 tx=0).stop 2024-09-12T03:22:29.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.525+0000 7f9578311640 1 -- 172.21.15.27:0/1819690115 shutdown_connections 2024-09-12T03:22:29.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.525+0000 7f9578311640 1 --2- 172.21.15.27:0/1819690115 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f95700b6f70 0x7f95700b7400 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:29.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.525+0000 7f9578311640 1 --2- 172.21.15.27:0/1819690115 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f95700d1040 0x7f95700d1480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:29.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.525+0000 7f9578311640 1 -- 172.21.15.27:0/1819690115 >> 172.21.15.27:0/1819690115 conn(0x7f957006f320 msgr2=0x7f957006f700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:29.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.525+0000 7f9578311640 1 -- 172.21.15.27:0/1819690115 shutdown_connections 2024-09-12T03:22:29.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.525+0000 7f9578311640 1 -- 172.21.15.27:0/1819690115 wait complete. 2024-09-12T03:22:29.532 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.526+0000 7f9578311640 1 Processor -- start 2024-09-12T03:22:29.532 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.526+0000 7f9578311640 1 -- start start 2024-09-12T03:22:29.532 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.527+0000 7f9578311640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f95700b6f70 0x7f95701f5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:29.532 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.527+0000 7f9578311640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f95700d1040 0x7f95700aac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:29.532 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.527+0000 7f9578311640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f957015c6c0 0x7f95700ab180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:29.532 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.527+0000 7f9578311640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f95700b6a50 con 0x7f95700b6f70 2024-09-12T03:22:29.532 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.527+0000 7f9578311640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f95700b6770 con 0x7f957015c6c0 2024-09-12T03:22:29.532 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.527+0000 7f9578311640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f95700d19c0 con 0x7f95700d1040 2024-09-12T03:22:29.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.527+0000 7f95760ad640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f95700b6f70 0x7f95701f5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:29.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.527+0000 7f95760ad640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f95700b6f70 0x7f95701f5050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54790/0 (socket says 172.21.15.27:54790) 2024-09-12T03:22:29.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.527+0000 7f95760ad640 1 -- 172.21.15.27:0/2830768044 learned_addr learned my addr 172.21.15.27:0/2830768044 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:22:29.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.527+0000 7f95768ae640 1 --2- 172.21.15.27:0/2830768044 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f957015c6c0 0x7f95700ab180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:29.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.527+0000 7f95758ac640 1 --2- 172.21.15.27:0/2830768044 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f95700d1040 0x7f95700aac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:29.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.527+0000 7f95760ad640 1 -- 172.21.15.27:0/2830768044 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f95700d1040 msgr2=0x7f95700aac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:29.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.527+0000 7f95760ad640 1 --2- 172.21.15.27:0/2830768044 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f95700d1040 0x7f95700aac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:29.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.527+0000 7f95760ad640 1 -- 172.21.15.27:0/2830768044 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f957015c6c0 msgr2=0x7f95700ab180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:29.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.527+0000 7f95760ad640 1 --2- 172.21.15.27:0/2830768044 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f957015c6c0 0x7f95700ab180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:29.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.527+0000 7f95760ad640 1 -- 172.21.15.27:0/2830768044 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95700ca3e0 con 0x7f95700b6f70 2024-09-12T03:22:29.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.527+0000 7f95758ac640 1 --2- 172.21.15.27:0/2830768044 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f95700d1040 0x7f95700aac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:22:29.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.527+0000 7f95768ae640 1 --2- 172.21.15.27:0/2830768044 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f957015c6c0 0x7f95700ab180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:22:29.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.528+0000 7f95760ad640 1 --2- 172.21.15.27:0/2830768044 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f95700b6f70 0x7f95701f5050 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7f956c00c780 tx=0x7f956c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:29.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.528+0000 7f9566ffd640 1 -- 172.21.15.27:0/2830768044 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f956c017070 con 0x7f95700b6f70 2024-09-12T03:22:29.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.528+0000 7f9566ffd640 1 -- 172.21.15.27:0/2830768044 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f956c005ad0 con 0x7f95700b6f70 2024-09-12T03:22:29.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.528+0000 7f9578311640 1 -- 172.21.15.27:0/2830768044 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95700ca6c0 con 0x7f95700b6f70 2024-09-12T03:22:29.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.528+0000 7f9566ffd640 1 -- 172.21.15.27:0/2830768044 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f956c007070 con 0x7f95700b6f70 2024-09-12T03:22:29.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.528+0000 7f9578311640 1 -- 172.21.15.27:0/2830768044 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95700cac80 con 0x7f95700b6f70 2024-09-12T03:22:29.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.529+0000 7f9578311640 1 -- 172.21.15.27:0/2830768044 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95700cb1a0 con 0x7f95700b6f70 2024-09-12T03:22:29.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.529+0000 7f9566ffd640 1 -- 172.21.15.27:0/2830768044 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 21) ==== 91008+0+0 (secure 0 0 0) 0x7f956c01a070 con 0x7f95700b6f70 2024-09-12T03:22:29.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.530+0000 7f9566ffd640 1 --2- 172.21.15.27:0/2830768044 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f955406c3d0 0x7f955406e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:29.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.530+0000 7f9566ffd640 1 -- 172.21.15.27:0/2830768044 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 5765+0+0 (secure 0 0 0) 0x7f956c08ad40 con 0x7f95700b6f70 2024-09-12T03:22:29.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.530+0000 7f95758ac640 1 --2- 172.21.15.27:0/2830768044 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f955406c3d0 0x7f955406e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:29.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.530+0000 7f95758ac640 1 --2- 172.21.15.27:0/2830768044 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f955406c3d0 0x7f955406e890 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f95701599f0 tx=0x7f9560008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:29.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.534+0000 7f9566ffd640 1 -- 172.21.15.27:0/2830768044 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f956c058ed0 con 0x7f95700b6f70 2024-09-12T03:22:29.740 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.740+0000 7f9578311640 1 -- 172.21.15.27:0/2830768044 --> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] -- mgr_command(tid 0: {"prefix": "fs subvolume resize", "vol_name": "non_existent_subvolume", "sub_name": "subvolname_1", "new_size": "inf", "target": ["mon-mgr", ""]}) -- 0x7f95700b4400 con 0x7f955406c3d0 2024-09-12T03:22:29.741 INFO:tasks.ceph.mgr.y.smithi027.stderr:2024-09-12T03:22:29.742+0000 7f325d8e3640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-12T03:22:29.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.742+0000 7f9566ffd640 1 -- 172.21.15.27:0/2830768044 <== mgr.4113 v2:172.21.15.27:6836/1226920526 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7f95700b4400 con 0x7f955406c3d0 2024-09-12T03:22:29.742 INFO:teuthology.orchestra.run.smithi027.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-12T03:22:29.746 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-12T03:22:29.747 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume pin non_existent_subvolume subvolname_1 export 1 2024-09-12T03:22:29.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.854+0000 7f8ed64a5640 1 Processor -- start 2024-09-12T03:22:29.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.854+0000 7f8ed64a5640 1 -- start start 2024-09-12T03:22:29.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.855+0000 7f8ed64a5640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f8ed015e840 0x7f8ed006bf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:29.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.855+0000 7f8ed64a5640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8ed00d3210 con 0x7f8ed006cc50 2024-09-12T03:22:29.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.855+0000 7f8ed64a5640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8ed00cfbc0 con 0x7f8ed00b88c0 2024-09-12T03:22:29.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.855+0000 7f8ed64a5640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f8ed00d3490 con 0x7f8ed015e840 2024-09-12T03:22:29.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.855+0000 7f8ecf7fe640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f8ed006cc50 0x7f8ed00ceda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:38540/0 (socket says 172.21.15.27:38540) 2024-09-12T03:22:29.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.855+0000 7f8ecf7fe640 1 -- 172.21.15.27:0/2273793956 learned_addr learned my addr 172.21.15.27:0/2273793956 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:22:29.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.856+0000 7f8ecffff640 1 --2- 172.21.15.27:0/2273793956 >> v2:172.21.15.45:3300/0 conn(0x7f8ed015e840 0x7f8ed006bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:29.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.856+0000 7f8eceffd640 1 -- 172.21.15.27:0/2273793956 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1377342240 0 0) 0x7f8ec4001110 con 0x7f8ed006cc50 2024-09-12T03:22:29.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.856+0000 7f8eceffd640 1 -- 172.21.15.27:0/2273793956 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8eb8003520 con 0x7f8ed006cc50 2024-09-12T03:22:29.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.856+0000 7f8eceffd640 1 -- 172.21.15.27:0/2273793956 <== mon.1 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1069636767 0 0) 0x7f8ed00cfbc0 con 0x7f8ed00b88c0 2024-09-12T03:22:29.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.856+0000 7f8eceffd640 1 -- 172.21.15.27:0/2273793956 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8ec4001110 con 0x7f8ed00b88c0 2024-09-12T03:22:29.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.856+0000 7f8eceffd640 1 -- 172.21.15.27:0/2273793956 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3683658209 0 0) 0x7f8eb8003520 con 0x7f8ed006cc50 2024-09-12T03:22:29.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.856+0000 7f8ecffff640 1 -- 172.21.15.27:0/2273793956 >> v1:172.21.15.45:6789/0 conn(0x7f8ed00b88c0 legacy=0x7f8ed00cf4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:29.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.856+0000 7f8ecffff640 1 -- 172.21.15.27:0/2273793956 >> v1:172.21.15.27:6789/0 conn(0x7f8ed006cc50 legacy=0x7f8ed00ceda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:29.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.856+0000 7f8ecffff640 1 -- 172.21.15.27:0/2273793956 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ed00d3770 con 0x7f8ed015e840 2024-09-12T03:22:29.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.857+0000 7f8ecffff640 1 --2- 172.21.15.27:0/2273793956 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ed015e840 0x7f8ed006bf10 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f8ec00097a0 tx=0x7f8ec002a130 comp rx=0 tx=0).ready entity=mon.1 client_cookie=304d2d0b745c9cab server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:29.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.857+0000 7f8eceffd640 1 -- 172.21.15.27:0/2273793956 <== mon.1 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ec0037070 con 0x7f8ed015e840 2024-09-12T03:22:29.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.857+0000 7f8eceffd640 1 -- 172.21.15.27:0/2273793956 <== mon.1 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ec002abc0 con 0x7f8ed015e840 2024-09-12T03:22:29.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.857+0000 7f8eceffd640 1 -- 172.21.15.27:0/2273793956 <== mon.1 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ec40017c0 con 0x7f8ed015e840 2024-09-12T03:22:29.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.858+0000 7f8ed64a5640 1 -- 172.21.15.27:0/2273793956 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ed015e840 msgr2=0x7f8ed006bf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:29.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.858+0000 7f8ed64a5640 1 --2- 172.21.15.27:0/2273793956 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ed015e840 0x7f8ed006bf10 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f8ec00097a0 tx=0x7f8ec002a130 comp rx=0 tx=0).stop 2024-09-12T03:22:29.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.858+0000 7f8ed64a5640 1 -- 172.21.15.27:0/2273793956 shutdown_connections 2024-09-12T03:22:29.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.858+0000 7f8ed64a5640 1 --2- 172.21.15.27:0/2273793956 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ed015e840 0x7f8ed006bf10 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:29.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.858+0000 7f8ed64a5640 1 -- 172.21.15.27:0/2273793956 >> 172.21.15.27:0/2273793956 conn(0x7f8ed00d4750 msgr2=0x7f8ed00d4b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:29.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.858+0000 7f8ed64a5640 1 -- 172.21.15.27:0/2273793956 shutdown_connections 2024-09-12T03:22:29.860 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.858+0000 7f8ed64a5640 1 -- 172.21.15.27:0/2273793956 wait complete. 2024-09-12T03:22:29.860 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.859+0000 7f8ed64a5640 1 Processor -- start 2024-09-12T03:22:29.860 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.859+0000 7f8ed64a5640 1 -- start start 2024-09-12T03:22:29.862 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.860+0000 7f8ed64a5640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ed006cc50 0x7f8ed00cb020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:29.862 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.860+0000 7f8ed64a5640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f8ed00b88c0 0x7f8ed01f4940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:29.862 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.860+0000 7f8ed64a5640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8ed015e840 0x7f8ed01f7630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:29.862 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.860+0000 7f8ed64a5640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f8ed00cff00 con 0x7f8ed015e840 2024-09-12T03:22:29.862 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.860+0000 7f8ed64a5640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f8ed00d3d30 con 0x7f8ed006cc50 2024-09-12T03:22:29.862 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.860+0000 7f8ed64a5640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f8ed015ec20 con 0x7f8ed00b88c0 2024-09-12T03:22:29.862 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.860+0000 7f8ed4a42640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8ed015e840 0x7f8ed01f7630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:29.862 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.860+0000 7f8ed4a42640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8ed015e840 0x7f8ed01f7630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54804/0 (socket says 172.21.15.27:54804) 2024-09-12T03:22:29.862 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.860+0000 7f8ed4a42640 1 -- 172.21.15.27:0/370324934 learned_addr learned my addr 172.21.15.27:0/370324934 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:22:29.862 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.860+0000 7f8ecffff640 1 --2- 172.21.15.27:0/370324934 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ed006cc50 0x7f8ed00cb020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:29.862 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.860+0000 7f8ecf7fe640 1 --2- 172.21.15.27:0/370324934 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f8ed00b88c0 0x7f8ed01f4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:29.862 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.860+0000 7f8ed4a42640 1 -- 172.21.15.27:0/370324934 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f8ed00b88c0 msgr2=0x7f8ed01f4940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:29.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.860+0000 7f8ed4a42640 1 --2- 172.21.15.27:0/370324934 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f8ed00b88c0 0x7f8ed01f4940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:29.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.860+0000 7f8ed4a42640 1 -- 172.21.15.27:0/370324934 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ed006cc50 msgr2=0x7f8ed00cb020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:29.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.860+0000 7f8ed4a42640 1 --2- 172.21.15.27:0/370324934 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ed006cc50 0x7f8ed00cb020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:29.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.860+0000 7f8ed4a42640 1 -- 172.21.15.27:0/370324934 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ed01f4e80 con 0x7f8ed015e840 2024-09-12T03:22:29.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.860+0000 7f8ecffff640 1 --2- 172.21.15.27:0/370324934 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ed006cc50 0x7f8ed00cb020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:22:29.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.860+0000 7f8ed4a42640 1 --2- 172.21.15.27:0/370324934 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8ed015e840 0x7f8ed01f7630 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f8ec800c830 tx=0x7f8ec800ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:29.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.861+0000 7f8eccff9640 1 -- 172.21.15.27:0/370324934 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ec8003b70 con 0x7f8ed015e840 2024-09-12T03:22:29.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.861+0000 7f8eccff9640 1 -- 172.21.15.27:0/370324934 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ec8003d10 con 0x7f8ed015e840 2024-09-12T03:22:29.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.861+0000 7f8eccff9640 1 -- 172.21.15.27:0/370324934 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ec800aa90 con 0x7f8ed015e840 2024-09-12T03:22:29.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.861+0000 7f8ed64a5640 1 -- 172.21.15.27:0/370324934 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ed01e8d80 con 0x7f8ed015e840 2024-09-12T03:22:29.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.861+0000 7f8ed64a5640 1 -- 172.21.15.27:0/370324934 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ed015fed0 con 0x7f8ed015e840 2024-09-12T03:22:29.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.862+0000 7f8ed64a5640 1 -- 172.21.15.27:0/370324934 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ed00b3e40 con 0x7f8ed015e840 2024-09-12T03:22:29.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.862+0000 7f8eccff9640 1 -- 172.21.15.27:0/370324934 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 21) ==== 91008+0+0 (secure 0 0 0) 0x7f8ec800ac30 con 0x7f8ed015e840 2024-09-12T03:22:29.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.863+0000 7f8eccff9640 1 --2- 172.21.15.27:0/370324934 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f8ebc06c3d0 0x7f8ebc06e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:29.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.863+0000 7f8ecffff640 1 --2- 172.21.15.27:0/370324934 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f8ebc06c3d0 0x7f8ebc06e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:29.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.863+0000 7f8eccff9640 1 -- 172.21.15.27:0/370324934 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 5765+0+0 (secure 0 0 0) 0x7f8ec808b930 con 0x7f8ed015e840 2024-09-12T03:22:29.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.863+0000 7f8ecffff640 1 --2- 172.21.15.27:0/370324934 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f8ebc06c3d0 0x7f8ebc06e890 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f8ec0034000 tx=0x7f8ec002e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:29.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:29.867+0000 7f8eccff9640 1 -- 172.21.15.27:0/370324934 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f8ec8059b00 con 0x7f8ed015e840 2024-09-12T03:22:30.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.074+0000 7f8ed64a5640 1 -- 172.21.15.27:0/370324934 --> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] -- mgr_command(tid 0: {"prefix": "fs subvolume pin", "vol_name": "non_existent_subvolume", "sub_name": "subvolname_1", "pin_type": "export", "pin_setting": "1", "target": ["mon-mgr", ""]}) -- 0x7f8ed00d0d00 con 0x7f8ebc06c3d0 2024-09-12T03:22:30.075 INFO:tasks.ceph.mgr.y.smithi027.stderr:2024-09-12T03:22:30.075+0000 7f325d8e3640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-12T03:22:30.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.075+0000 7f8eccff9640 1 -- 172.21.15.27:0/370324934 <== mgr.4113 v2:172.21.15.27:6836/1226920526 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7f8ed00d0d00 con 0x7f8ebc06c3d0 2024-09-12T03:22:30.075 INFO:teuthology.orchestra.run.smithi027.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-12T03:22:30.081 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-12T03:22:30.081 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume ls non_existent_subvolume 2024-09-12T03:22:30.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.186+0000 7fb1d561e640 1 Processor -- start 2024-09-12T03:22:30.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.186+0000 7fb1d561e640 1 -- start start 2024-09-12T03:22:30.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.187+0000 7fb1d561e640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fb1d006f990 0x7fb1d006e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:30.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.187+0000 7fb1d561e640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb1d00701c0 0x7fb1d00705e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:30.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.187+0000 7fb1d561e640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb1d006ed60 con 0x7fb1d00725b0 2024-09-12T03:22:30.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.187+0000 7fb1d561e640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7fb1d0070b20 con 0x7fb1d006f990 2024-09-12T03:22:30.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.187+0000 7fb1d561e640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fb1d015ef90 con 0x7fb1d00701c0 2024-09-12T03:22:30.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.187+0000 7fb1ceffd640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fb1d006f990 0x7fb1d006e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:30.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.187+0000 7fb1ceffd640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fb1d006f990 0x7fb1d006e820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54814/0 (socket says 172.21.15.27:54814) 2024-09-12T03:22:30.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.187+0000 7fb1cf7fe640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fb1d00725b0 0x7fb1d0162510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:38556/0 (socket says 172.21.15.27:38556) 2024-09-12T03:22:30.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.187+0000 7fb1ceffd640 1 -- 172.21.15.27:0/436741629 learned_addr learned my addr 172.21.15.27:0/436741629 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:22:30.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.188+0000 7fb1ce7fc640 1 --2- 172.21.15.27:0/436741629 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb1d00701c0 0x7fb1d00705e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:30.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.188+0000 7fb1ceffd640 1 -- 172.21.15.27:0/436741629 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb1d00701c0 msgr2=0x7fb1d00705e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:30.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.188+0000 7fb1ceffd640 1 --2- 172.21.15.27:0/436741629 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb1d00701c0 0x7fb1d00705e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:30.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.188+0000 7fb1ceffd640 1 -- 172.21.15.27:0/436741629 >> v1:172.21.15.27:6789/0 conn(0x7fb1d00725b0 legacy=0x7fb1d0162510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:30.190 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.188+0000 7fb1ceffd640 1 -- 172.21.15.27:0/436741629 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1d015f270 con 0x7fb1d006f990 2024-09-12T03:22:30.190 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.188+0000 7fb1ce7fc640 1 --2- 172.21.15.27:0/436741629 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb1d00701c0 0x7fb1d00705e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:22:30.190 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.188+0000 7fb1ceffd640 1 --2- 172.21.15.27:0/436741629 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb1d006f990 0x7fb1d006e820 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7fb1c0009870 tx=0x7fb1c002a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a98c60e3d82b275d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:30.190 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.189+0000 7fb1cdffb640 1 -- 172.21.15.27:0/436741629 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1c0037070 con 0x7fb1d006f990 2024-09-12T03:22:30.190 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.189+0000 7fb1cdffb640 1 -- 172.21.15.27:0/436741629 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb1c0032440 con 0x7fb1d006f990 2024-09-12T03:22:30.190 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.189+0000 7fb1cdffb640 1 -- 172.21.15.27:0/436741629 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1c0032810 con 0x7fb1d006f990 2024-09-12T03:22:30.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.189+0000 7fb1d561e640 1 -- 172.21.15.27:0/436741629 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb1d006f990 msgr2=0x7fb1d006e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:30.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.189+0000 7fb1d561e640 1 --2- 172.21.15.27:0/436741629 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb1d006f990 0x7fb1d006e820 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7fb1c0009870 tx=0x7fb1c002a1c0 comp rx=0 tx=0).stop 2024-09-12T03:22:30.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.189+0000 7fb1d561e640 1 -- 172.21.15.27:0/436741629 shutdown_connections 2024-09-12T03:22:30.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.189+0000 7fb1d561e640 1 --2- 172.21.15.27:0/436741629 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb1d00701c0 0x7fb1d00705e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:30.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.189+0000 7fb1d561e640 1 --2- 172.21.15.27:0/436741629 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb1d006f990 0x7fb1d006e820 unknown :-1 s=CLOSED pgs=1285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:30.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.189+0000 7fb1d561e640 1 -- 172.21.15.27:0/436741629 >> 172.21.15.27:0/436741629 conn(0x7fb1d006d060 msgr2=0x7fb1d006d470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:30.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.189+0000 7fb1d561e640 1 -- 172.21.15.27:0/436741629 shutdown_connections 2024-09-12T03:22:30.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.189+0000 7fb1d561e640 1 -- 172.21.15.27:0/436741629 wait complete. 2024-09-12T03:22:30.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.191+0000 7fb1d561e640 1 Processor -- start 2024-09-12T03:22:30.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.191+0000 7fb1d561e640 1 -- start start 2024-09-12T03:22:30.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.191+0000 7fb1d561e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb1d006f990 0x7fb1d00c9960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:30.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.191+0000 7fb1d561e640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb1d00701c0 0x7fb1d00c76e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:30.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.191+0000 7fb1d561e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb1d00725b0 0x7fb1d014bde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:30.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.191+0000 7fb1d561e640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fb1d015fb10 con 0x7fb1d006f990 2024-09-12T03:22:30.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.191+0000 7fb1d561e640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb1d015f830 con 0x7fb1d00725b0 2024-09-12T03:22:30.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.191+0000 7fb1d561e640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fb1d015eb50 con 0x7fb1d00701c0 2024-09-12T03:22:30.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.192+0000 7fb1ceffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb1d006f990 0x7fb1d00c9960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:30.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.192+0000 7fb1ceffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb1d006f990 0x7fb1d00c9960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54824/0 (socket says 172.21.15.27:54824) 2024-09-12T03:22:30.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.192+0000 7fb1ceffd640 1 -- 172.21.15.27:0/1542009573 learned_addr learned my addr 172.21.15.27:0/1542009573 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:22:30.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.192+0000 7fb1ce7fc640 1 --2- 172.21.15.27:0/1542009573 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb1d00701c0 0x7fb1d00c76e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:30.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.192+0000 7fb1ceffd640 1 -- 172.21.15.27:0/1542009573 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb1d00701c0 msgr2=0x7fb1d00c76e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:30.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.192+0000 7fb1ceffd640 1 --2- 172.21.15.27:0/1542009573 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb1d00701c0 0x7fb1d00c76e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:30.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.192+0000 7fb1ceffd640 1 -- 172.21.15.27:0/1542009573 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb1d00725b0 msgr2=0x7fb1d014bde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:30.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.192+0000 7fb1cf7fe640 1 --2- 172.21.15.27:0/1542009573 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb1d00725b0 0x7fb1d014bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:30.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.192+0000 7fb1ceffd640 1 --2- 172.21.15.27:0/1542009573 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb1d00725b0 0x7fb1d014bde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:30.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.192+0000 7fb1ceffd640 1 -- 172.21.15.27:0/1542009573 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1d00c81a0 con 0x7fb1d006f990 2024-09-12T03:22:30.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.192+0000 7fb1ceffd640 1 --2- 172.21.15.27:0/1542009573 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb1d006f990 0x7fb1d00c9960 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7fb1c0000c00 tx=0x7fb1c00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:30.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.192+0000 7fb1b77fe640 1 -- 172.21.15.27:0/1542009573 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1c0037040 con 0x7fb1d006f990 2024-09-12T03:22:30.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.192+0000 7fb1b77fe640 1 -- 172.21.15.27:0/1542009573 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb1c0042070 con 0x7fb1d006f990 2024-09-12T03:22:30.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.192+0000 7fb1b77fe640 1 -- 172.21.15.27:0/1542009573 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1c0030bd0 con 0x7fb1d006f990 2024-09-12T03:22:30.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.192+0000 7fb1d561e640 1 -- 172.21.15.27:0/1542009573 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1d00c8480 con 0x7fb1d006f990 2024-09-12T03:22:30.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.192+0000 7fb1d561e640 1 -- 172.21.15.27:0/1542009573 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1d00c8a40 con 0x7fb1d006f990 2024-09-12T03:22:30.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.194+0000 7fb1d561e640 1 -- 172.21.15.27:0/1542009573 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1d006e820 con 0x7fb1d006f990 2024-09-12T03:22:30.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.194+0000 7fb1b77fe640 1 -- 172.21.15.27:0/1542009573 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 21) ==== 91008+0+0 (secure 0 0 0) 0x7fb1c002ad10 con 0x7fb1d006f990 2024-09-12T03:22:30.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.195+0000 7fb1b77fe640 1 --2- 172.21.15.27:0/1542009573 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fb1bc06c3d0 0x7fb1bc06e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:30.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.195+0000 7fb1ce7fc640 1 --2- 172.21.15.27:0/1542009573 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fb1bc06c3d0 0x7fb1bc06e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:30.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.195+0000 7fb1b77fe640 1 -- 172.21.15.27:0/1542009573 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 5765+0+0 (secure 0 0 0) 0x7fb1c00ac200 con 0x7fb1d006f990 2024-09-12T03:22:30.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.195+0000 7fb1ce7fc640 1 --2- 172.21.15.27:0/1542009573 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fb1bc06c3d0 0x7fb1bc06e890 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fb1d00c6c60 tx=0x7fb1c4008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:30.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.199+0000 7fb1b77fe640 1 -- 172.21.15.27:0/1542009573 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fb1c0045360 con 0x7fb1d006f990 2024-09-12T03:22:30.400 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.400+0000 7fb1d561e640 1 -- 172.21.15.27:0/1542009573 --> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] -- mgr_command(tid 0: {"prefix": "fs subvolume ls", "vol_name": "non_existent_subvolume", "target": ["mon-mgr", ""]}) -- 0x7fb1d00c8e60 con 0x7fb1bc06c3d0 2024-09-12T03:22:30.401 INFO:tasks.ceph.mgr.y.smithi027.stderr:2024-09-12T03:22:30.401+0000 7f325d8e3640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-12T03:22:30.401 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.401+0000 7fb1b77fe640 1 -- 172.21.15.27:0/1542009573 <== mgr.4113 v2:172.21.15.27:6836/1226920526 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7fb1d00c8e60 con 0x7fb1bc06c3d0 2024-09-12T03:22:30.401 INFO:teuthology.orchestra.run.smithi027.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-12T03:22:30.406 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-12T03:22:30.407 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot create non_existent_subvolume subvolume_1 snapshot_1 2024-09-12T03:22:30.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.517+0000 7fb08b152640 1 Processor -- start 2024-09-12T03:22:30.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.517+0000 7fb08b152640 1 -- start start 2024-09-12T03:22:30.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.517+0000 7fb08b152640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fb084162040 0x7fb0840d1da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:30.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.517+0000 7fb08b152640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb08406dfd0 0x7fb0840b7a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:30.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.517+0000 7fb08b152640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb0840d2e30 con 0x7fb0840d3790 2024-09-12T03:22:30.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.517+0000 7fb08b152640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fb0840b7f90 con 0x7fb084162040 2024-09-12T03:22:30.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.517+0000 7fb08b152640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fb084162420 con 0x7fb08406dfd0 2024-09-12T03:22:30.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.518+0000 7fb083fff640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fb0840d3790 0x7fb08406d8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:38570/0 (socket says 172.21.15.27:38570) 2024-09-12T03:22:30.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.518+0000 7fb083fff640 1 -- 172.21.15.27:0/621051220 learned_addr learned my addr 172.21.15.27:0/621051220 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:22:30.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.518+0000 7fb0837fe640 1 -- 172.21.15.27:0/621051220 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3375841450 0 0) 0x7fb0840d2e30 con 0x7fb0840d3790 2024-09-12T03:22:30.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.518+0000 7fb0896ef640 1 --2- 172.21.15.27:0/621051220 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb08406dfd0 0x7fb0840b7a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:30.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.518+0000 7fb0837fe640 1 -- 172.21.15.27:0/621051220 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb074003520 con 0x7fb0840d3790 2024-09-12T03:22:30.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.518+0000 7fb088eee640 1 --2- 172.21.15.27:0/621051220 >> v2:172.21.15.45:3300/0 conn(0x7fb084162040 0x7fb0840d1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:30.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.518+0000 7fb0837fe640 1 -- 172.21.15.27:0/621051220 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 340286036 0 0) 0x7fb074003520 con 0x7fb0840d3790 2024-09-12T03:22:30.522 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.518+0000 7fb0837fe640 1 -- 172.21.15.27:0/621051220 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb08406dfd0 msgr2=0x7fb0840b7a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:30.522 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.518+0000 7fb0837fe640 1 --2- 172.21.15.27:0/621051220 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb08406dfd0 0x7fb0840b7a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:30.522 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.518+0000 7fb0837fe640 1 -- 172.21.15.27:0/621051220 >> v2:172.21.15.45:3300/0 conn(0x7fb084162040 msgr2=0x7fb0840d1da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:30.522 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.518+0000 7fb0837fe640 1 --2- 172.21.15.27:0/621051220 >> v2:172.21.15.45:3300/0 conn(0x7fb084162040 0x7fb0840d1da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:30.522 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.518+0000 7fb0837fe640 1 -- 172.21.15.27:0/621051220 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0840d30d0 con 0x7fb0840d3790 2024-09-12T03:22:30.522 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.518+0000 7fb0837fe640 1 -- 172.21.15.27:0/621051220 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 847755213 0 0) 0x7fb070001670 con 0x7fb0840d3790 2024-09-12T03:22:30.522 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.518+0000 7fb0837fe640 1 -- 172.21.15.27:0/621051220 >> v1:172.21.15.27:6789/0 conn(0x7fb0840d3790 legacy=0x7fb08406d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:30.522 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.518+0000 7fb0837fe640 1 --2- 172.21.15.27:0/621051220 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb08406dfd0 0x7fb0740039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:30.522 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.518+0000 7fb0837fe640 1 --2- 172.21.15.27:0/621051220 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb074003f30 0x7fb074004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:30.522 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.518+0000 7fb0837fe640 1 --2- 172.21.15.27:0/621051220 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb074004d40 0x7fb0740051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:30.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.518+0000 7fb0837fe640 1 -- 172.21.15.27:0/621051220 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fb0840b7f90 con 0x7fb074003f30 2024-09-12T03:22:30.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.518+0000 7fb0837fe640 1 -- 172.21.15.27:0/621051220 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb084162420 con 0x7fb074004d40 2024-09-12T03:22:30.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.518+0000 7fb0837fe640 1 -- 172.21.15.27:0/621051220 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fb07400b9c0 con 0x7fb08406dfd0 2024-09-12T03:22:30.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.518+0000 7fb083fff640 1 --2- 172.21.15.27:0/621051220 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb074003f30 0x7fb074004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:30.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.518+0000 7fb088eee640 1 --2- 172.21.15.27:0/621051220 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb08406dfd0 0x7fb0740039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:30.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.518+0000 7fb0896ef640 1 --2- 172.21.15.27:0/621051220 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb074004d40 0x7fb0740051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:30.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.519+0000 7fb083fff640 1 -- 172.21.15.27:0/621051220 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb08406dfd0 msgr2=0x7fb0740039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:30.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.519+0000 7fb083fff640 1 --2- 172.21.15.27:0/621051220 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb08406dfd0 0x7fb0740039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:30.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.519+0000 7fb083fff640 1 -- 172.21.15.27:0/621051220 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb074004d40 msgr2=0x7fb0740051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:30.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.519+0000 7fb083fff640 1 --2- 172.21.15.27:0/621051220 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb074004d40 0x7fb0740051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:30.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.519+0000 7fb083fff640 1 -- 172.21.15.27:0/621051220 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb074003520 con 0x7fb074003f30 2024-09-12T03:22:30.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.519+0000 7fb083fff640 1 --2- 172.21.15.27:0/621051220 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb074003f30 0x7fb074004800 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7fb070001050 tx=0x7fb074005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c11a9375eb271707 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:30.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.520+0000 7fb0837fe640 1 -- 172.21.15.27:0/621051220 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb070007d70 con 0x7fb074003f30 2024-09-12T03:22:30.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.520+0000 7fb0837fe640 1 -- 172.21.15.27:0/621051220 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0700058d0 con 0x7fb074003f30 2024-09-12T03:22:30.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.520+0000 7fb0837fe640 1 -- 172.21.15.27:0/621051220 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb070005ce0 con 0x7fb074003f30 2024-09-12T03:22:30.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.520+0000 7fb08b152640 1 -- 172.21.15.27:0/621051220 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb074003f30 msgr2=0x7fb074004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:30.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.520+0000 7fb08b152640 1 --2- 172.21.15.27:0/621051220 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb074003f30 0x7fb074004800 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7fb070001050 tx=0x7fb074005830 comp rx=0 tx=0).stop 2024-09-12T03:22:30.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.520+0000 7fb0896ef640 1 -- 172.21.15.27:0/621051220 reap_dead start 2024-09-12T03:22:30.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.520+0000 7fb08b152640 1 -- 172.21.15.27:0/621051220 shutdown_connections 2024-09-12T03:22:30.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.520+0000 7fb08b152640 1 -- 172.21.15.27:0/621051220 >> 172.21.15.27:0/621051220 conn(0x7fb0840b6700 msgr2=0x7fb0840b6b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:30.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.520+0000 7fb08b152640 1 -- 172.21.15.27:0/621051220 shutdown_connections 2024-09-12T03:22:30.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.520+0000 7fb08b152640 1 -- 172.21.15.27:0/621051220 wait complete. 2024-09-12T03:22:30.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.521+0000 7fb08b152640 1 Processor -- start 2024-09-12T03:22:30.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.522+0000 7fb08b152640 1 -- start start 2024-09-12T03:22:30.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.522+0000 7fb08b152640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb074003f30 0x7fb084154d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:30.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.522+0000 7fb08b152640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb0840b0cb0 0x7fb0840af0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:30.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.522+0000 7fb08b152640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb0840af630 0x7fb084160640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:30.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.522+0000 7fb08b152640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fb0840d4530 con 0x7fb0840af630 2024-09-12T03:22:30.528 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.522+0000 7fb08b152640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb0840cee00 con 0x7fb0840b0cb0 2024-09-12T03:22:30.528 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.522+0000 7fb08b152640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fb0840d4150 con 0x7fb074003f30 2024-09-12T03:22:30.528 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.523+0000 7fb0896ef640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb0840af630 0x7fb084160640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:30.528 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.523+0000 7fb0896ef640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb0840af630 0x7fb084160640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54838/0 (socket says 172.21.15.27:54838) 2024-09-12T03:22:30.528 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.523+0000 7fb0896ef640 1 -- 172.21.15.27:0/3600273129 learned_addr learned my addr 172.21.15.27:0/3600273129 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:22:30.528 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.523+0000 7fb083fff640 1 --2- 172.21.15.27:0/3600273129 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb0840b0cb0 0x7fb0840af0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:30.528 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.523+0000 7fb088eee640 1 --2- 172.21.15.27:0/3600273129 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb074003f30 0x7fb084154d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:30.528 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.523+0000 7fb0896ef640 1 -- 172.21.15.27:0/3600273129 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb074003f30 msgr2=0x7fb084154d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:30.528 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.523+0000 7fb0896ef640 1 --2- 172.21.15.27:0/3600273129 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb074003f30 0x7fb084154d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:30.528 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.523+0000 7fb0896ef640 1 -- 172.21.15.27:0/3600273129 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb0840b0cb0 msgr2=0x7fb0840af0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:30.528 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.523+0000 7fb0896ef640 1 --2- 172.21.15.27:0/3600273129 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb0840b0cb0 0x7fb0840af0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:30.528 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.523+0000 7fb0896ef640 1 -- 172.21.15.27:0/3600273129 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0841f5e70 con 0x7fb0840af630 2024-09-12T03:22:30.529 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.523+0000 7fb0896ef640 1 --2- 172.21.15.27:0/3600273129 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb0840af630 0x7fb084160640 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7fb0840d1cb0 tx=0x7fb07c00f1b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:30.529 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.523+0000 7fb0817fa640 1 -- 172.21.15.27:0/3600273129 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb07c00fd10 con 0x7fb0840af630 2024-09-12T03:22:30.529 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.523+0000 7fb0817fa640 1 -- 172.21.15.27:0/3600273129 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb07c015430 con 0x7fb0840af630 2024-09-12T03:22:30.529 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.523+0000 7fb08b152640 1 -- 172.21.15.27:0/3600273129 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0841f6150 con 0x7fb0840af630 2024-09-12T03:22:30.529 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.523+0000 7fb0817fa640 1 -- 172.21.15.27:0/3600273129 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb07c015820 con 0x7fb0840af630 2024-09-12T03:22:30.529 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.523+0000 7fb08b152640 1 -- 172.21.15.27:0/3600273129 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0841f6710 con 0x7fb0840af630 2024-09-12T03:22:30.529 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.524+0000 7fb08b152640 1 -- 172.21.15.27:0/3600273129 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb08406e6e0 con 0x7fb0840af630 2024-09-12T03:22:30.529 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.525+0000 7fb0817fa640 1 -- 172.21.15.27:0/3600273129 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 21) ==== 91008+0+0 (secure 0 0 0) 0x7fb07c0159c0 con 0x7fb0840af630 2024-09-12T03:22:30.529 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.525+0000 7fb0817fa640 1 --2- 172.21.15.27:0/3600273129 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fb06806c3d0 0x7fb06806e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:30.529 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.526+0000 7fb0817fa640 1 -- 172.21.15.27:0/3600273129 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 5765+0+0 (secure 0 0 0) 0x7fb07c08c440 con 0x7fb0840af630 2024-09-12T03:22:30.529 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.526+0000 7fb088eee640 1 --2- 172.21.15.27:0/3600273129 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fb06806c3d0 0x7fb06806e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:30.529 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.526+0000 7fb088eee640 1 --2- 172.21.15.27:0/3600273129 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fb06806c3d0 0x7fb06806e890 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fb084161940 tx=0x7fb078009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:30.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.530+0000 7fb0817fa640 1 -- 172.21.15.27:0/3600273129 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fb07c019030 con 0x7fb0840af630 2024-09-12T03:22:30.733 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.733+0000 7fb08b152640 1 -- 172.21.15.27:0/3600273129 --> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot create", "vol_name": "non_existent_subvolume", "sub_name": "subvolume_1", "snap_name": "snapshot_1", "target": ["mon-mgr", ""]}) -- 0x7fb0840d2b50 con 0x7fb06806c3d0 2024-09-12T03:22:30.734 INFO:tasks.ceph.mgr.y.smithi027.stderr:2024-09-12T03:22:30.734+0000 7f325d8e3640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-12T03:22:30.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.734+0000 7fb0817fa640 1 -- 172.21.15.27:0/3600273129 <== mgr.4113 v2:172.21.15.27:6836/1226920526 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7fb0840d2b50 con 0x7fb06806c3d0 2024-09-12T03:22:30.734 INFO:teuthology.orchestra.run.smithi027.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-12T03:22:30.739 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-12T03:22:30.739 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm non_existent_subvolume subvolume_1 snapshot_1 2024-09-12T03:22:30.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.848+0000 7f3f5d7a6640 1 Processor -- start 2024-09-12T03:22:30.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.848+0000 7f3f5d7a6640 1 -- start start 2024-09-12T03:22:30.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.849+0000 7f3f5d7a6640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f3f580b6230 0x7f3f580b6690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:30.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.849+0000 7f3f5d7a6640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f3f580b3600 0x7f3f580b3a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:30.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.849+0000 7f3f5d7a6640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f5815ed70 con 0x7f3f5815e990 2024-09-12T03:22:30.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.849+0000 7f3f5d7a6640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f3f580b6bd0 con 0x7f3f580b3600 2024-09-12T03:22:30.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.849+0000 7f3f5d7a6640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f3f580b7d10 con 0x7f3f580b6230 2024-09-12T03:22:30.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.849+0000 7f3f577fe640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f3f580b3600 0x7f3f580b3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:30.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.849+0000 7f3f567fc640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f3f580b6230 0x7f3f580b6690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:30.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.849+0000 7f3f577fe640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f3f580b3600 0x7f3f580b3a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54846/0 (socket says 172.21.15.27:54846) 2024-09-12T03:22:30.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.849+0000 7f3f577fe640 1 -- 172.21.15.27:0/3653330733 learned_addr learned my addr 172.21.15.27:0/3653330733 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:22:30.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.849+0000 7f3f577fe640 1 -- 172.21.15.27:0/3653330733 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f3f580b6230 msgr2=0x7f3f580b6690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:30.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.849+0000 7f3f577fe640 1 --2- 172.21.15.27:0/3653330733 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f3f580b6230 0x7f3f580b6690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:30.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.849+0000 7f3f577fe640 1 -- 172.21.15.27:0/3653330733 >> v1:172.21.15.45:6789/0 conn(0x7f3f5815e990 legacy=0x7f3f580b7600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:30.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.849+0000 7f3f577fe640 1 -- 172.21.15.27:0/3653330733 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f580b3fb0 con 0x7f3f580b3600 2024-09-12T03:22:30.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.850+0000 7f3f577fe640 1 --2- 172.21.15.27:0/3653330733 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3f580b3600 0x7f3f580b3a70 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7f3f50009e50 tx=0x7f3f5002a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=583015b119013938 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:30.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.850+0000 7f3f55ffb640 1 -- 172.21.15.27:0/3653330733 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f50036070 con 0x7f3f580b3600 2024-09-12T03:22:30.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.850+0000 7f3f55ffb640 1 -- 172.21.15.27:0/3653330733 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f50031600 con 0x7f3f580b3600 2024-09-12T03:22:30.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.850+0000 7f3f55ffb640 1 -- 172.21.15.27:0/3653330733 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f500319d0 con 0x7f3f580b3600 2024-09-12T03:22:30.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.851+0000 7f3f5d7a6640 1 -- 172.21.15.27:0/3653330733 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3f580b3600 msgr2=0x7f3f580b3a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:30.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.851+0000 7f3f5d7a6640 1 --2- 172.21.15.27:0/3653330733 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3f580b3600 0x7f3f580b3a70 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7f3f50009e50 tx=0x7f3f5002a710 comp rx=0 tx=0).stop 2024-09-12T03:22:30.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.851+0000 7f3f5d7a6640 1 -- 172.21.15.27:0/3653330733 shutdown_connections 2024-09-12T03:22:30.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.851+0000 7f3f5d7a6640 1 --2- 172.21.15.27:0/3653330733 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3f580b3600 0x7f3f580b3a70 unknown :-1 s=CLOSED pgs=1290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:30.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.851+0000 7f3f5d7a6640 1 --2- 172.21.15.27:0/3653330733 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f3f580b6230 0x7f3f580b6690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:30.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.851+0000 7f3f5d7a6640 1 -- 172.21.15.27:0/3653330733 >> 172.21.15.27:0/3653330733 conn(0x7f3f580d1230 msgr2=0x7f3f580d1640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:30.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.851+0000 7f3f5d7a6640 1 -- 172.21.15.27:0/3653330733 shutdown_connections 2024-09-12T03:22:30.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.851+0000 7f3f5d7a6640 1 -- 172.21.15.27:0/3653330733 wait complete. 2024-09-12T03:22:30.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.852+0000 7f3f5d7a6640 1 Processor -- start 2024-09-12T03:22:30.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.852+0000 7f3f5d7a6640 1 -- start start 2024-09-12T03:22:30.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.853+0000 7f3f5d7a6640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f3f580b3600 0x7f3f581555f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:30.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.853+0000 7f3f5d7a6640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3f580b6230 0x7f3f58161770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:30.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.853+0000 7f3f5d7a6640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3f5815e990 0x7f3f58150ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:30.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.853+0000 7f3f5d7a6640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f3f580d2ff0 con 0x7f3f580b6230 2024-09-12T03:22:30.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.853+0000 7f3f5d7a6640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f3f580d2d10 con 0x7f3f5815e990 2024-09-12T03:22:30.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.853+0000 7f3f5d7a6640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f3f580d32d0 con 0x7f3f580b3600 2024-09-12T03:22:30.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.853+0000 7f3f567fc640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3f580b6230 0x7f3f58161770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:30.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.853+0000 7f3f567fc640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3f580b6230 0x7f3f58161770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54852/0 (socket says 172.21.15.27:54852) 2024-09-12T03:22:30.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.853+0000 7f3f567fc640 1 -- 172.21.15.27:0/3373827001 learned_addr learned my addr 172.21.15.27:0/3373827001 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:22:30.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.853+0000 7f3f56ffd640 1 --2- 172.21.15.27:0/3373827001 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f3f580b3600 0x7f3f581555f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:30.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.853+0000 7f3f577fe640 1 --2- 172.21.15.27:0/3373827001 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3f5815e990 0x7f3f58150ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:30.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.853+0000 7f3f567fc640 1 -- 172.21.15.27:0/3373827001 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f3f580b3600 msgr2=0x7f3f581555f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:30.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.853+0000 7f3f567fc640 1 --2- 172.21.15.27:0/3373827001 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f3f580b3600 0x7f3f581555f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:30.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.853+0000 7f3f567fc640 1 -- 172.21.15.27:0/3373827001 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3f5815e990 msgr2=0x7f3f58150ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:30.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.853+0000 7f3f567fc640 1 --2- 172.21.15.27:0/3373827001 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3f5815e990 0x7f3f58150ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:30.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.853+0000 7f3f567fc640 1 -- 172.21.15.27:0/3373827001 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f58155b30 con 0x7f3f580b6230 2024-09-12T03:22:30.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.853+0000 7f3f577fe640 1 --2- 172.21.15.27:0/3373827001 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3f5815e990 0x7f3f58150ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:22:30.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.853+0000 7f3f56ffd640 1 --2- 172.21.15.27:0/3373827001 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f3f580b3600 0x7f3f581555f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:22:30.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.854+0000 7f3f567fc640 1 --2- 172.21.15.27:0/3373827001 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3f580b6230 0x7f3f58161770 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7f3f4000e9b0 tx=0x7f3f4000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:30.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.854+0000 7f3f4f7fe640 1 -- 172.21.15.27:0/3373827001 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f40017070 con 0x7f3f580b6230 2024-09-12T03:22:30.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.854+0000 7f3f4f7fe640 1 -- 172.21.15.27:0/3373827001 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f4000c9a0 con 0x7f3f580b6230 2024-09-12T03:22:30.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.854+0000 7f3f5d7a6640 1 -- 172.21.15.27:0/3373827001 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f5815f2b0 con 0x7f3f580b6230 2024-09-12T03:22:30.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.854+0000 7f3f4f7fe640 1 -- 172.21.15.27:0/3373827001 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f4000cd90 con 0x7f3f580b6230 2024-09-12T03:22:30.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.854+0000 7f3f5d7a6640 1 -- 172.21.15.27:0/3373827001 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f58160f70 con 0x7f3f580b6230 2024-09-12T03:22:30.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.855+0000 7f3f5d7a6640 1 -- 172.21.15.27:0/3373827001 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f5815f7d0 con 0x7f3f580b6230 2024-09-12T03:22:30.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.855+0000 7f3f4f7fe640 1 -- 172.21.15.27:0/3373827001 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 21) ==== 91008+0+0 (secure 0 0 0) 0x7f3f4001a070 con 0x7f3f580b6230 2024-09-12T03:22:30.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.856+0000 7f3f4f7fe640 1 --2- 172.21.15.27:0/3373827001 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f3f3806c3d0 0x7f3f3806e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:30.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.856+0000 7f3f4f7fe640 1 -- 172.21.15.27:0/3373827001 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 5765+0+0 (secure 0 0 0) 0x7f3f4008b8a0 con 0x7f3f580b6230 2024-09-12T03:22:30.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.856+0000 7f3f56ffd640 1 --2- 172.21.15.27:0/3373827001 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f3f3806c3d0 0x7f3f3806e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:30.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.856+0000 7f3f56ffd640 1 --2- 172.21.15.27:0/3373827001 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f3f3806c3d0 0x7f3f3806e890 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f3f48001660 tx=0x7f3f48009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:30.860 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:30.860+0000 7f3f4f7fe640 1 -- 172.21.15.27:0/3373827001 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f3f40059a30 con 0x7f3f580b6230 2024-09-12T03:22:31.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:31.066+0000 7f3f5d7a6640 1 -- 172.21.15.27:0/3373827001 --> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "vol_name": "non_existent_subvolume", "sub_name": "subvolume_1", "snap_name": "snapshot_1", "target": ["mon-mgr", ""]}) -- 0x7f3f580d35b0 con 0x7f3f3806c3d0 2024-09-12T03:22:31.077 INFO:tasks.ceph.mgr.y.smithi027.stderr:2024-09-12T03:22:31.067+0000 7f325d8e3640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-12T03:22:31.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:31.067+0000 7f3f4f7fe640 1 -- 172.21.15.27:0/3373827001 <== mgr.4113 v2:172.21.15.27:6836/1226920526 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7f3f580d35b0 con 0x7f3f3806c3d0 2024-09-12T03:22:31.078 INFO:teuthology.orchestra.run.smithi027.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-12T03:22:31.081 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-12T03:22:31.082 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot info non_existent_subvolume subvolume_1 snapshot_1 2024-09-12T03:22:31.196 DEBUG:teuthology.orchestra.run.smithi027:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-12T03:22:31.206 DEBUG:teuthology.orchestra.run.smithi045:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-12T03:22:31.208 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:31.208+0000 7f924049b640 1 Processor -- start 2024-09-12T03:22:31.208 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:31.208+0000 7f924049b640 1 -- start start 2024-09-12T03:22:31.210 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:31.209+0000 7f924049b640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f923806f990 0x7f923806e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:31.210 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:31.209+0000 7f924049b640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f9238072cc0 0x7f92380705e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:31.210 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:31.209+0000 7f924049b640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f923806ed60 con 0x7f92380701c0 2024-09-12T03:22:31.210 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:31.209+0000 7f924049b640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f9238070b20 con 0x7f9238072cc0 2024-09-12T03:22:31.210 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:31.209+0000 7f924049b640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f923815ef90 con 0x7f923806f990 2024-09-12T03:22:31.210 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:31.209+0000 7f923ea38640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f9238072cc0 0x7f92380705e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:31.210 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:31.209+0000 7f923da36640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f92380701c0 0x7f92380725b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:38586/0 (socket says 172.21.15.27:38586) 2024-09-12T03:22:31.210 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:31.210+0000 7f923da36640 1 -- 172.21.15.27:0/1373911869 learned_addr learned my addr 172.21.15.27:0/1373911869 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:22:31.210 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:31.210+0000 7f923e237640 1 --2- 172.21.15.27:0/1373911869 >> v2:172.21.15.45:3300/0 conn(0x7f923806f990 0x7f923806e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:31.211 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:31.210+0000 7f923ea38640 1 -- 172.21.15.27:0/1373911869 >> v2:172.21.15.45:3300/0 conn(0x7f923806f990 msgr2=0x7f923806e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:31.211 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:31.210+0000 7f923ea38640 1 --2- 172.21.15.27:0/1373911869 >> v2:172.21.15.45:3300/0 conn(0x7f923806f990 0x7f923806e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:31.211 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:31.210+0000 7f923ea38640 1 -- 172.21.15.27:0/1373911869 >> v1:172.21.15.27:6789/0 conn(0x7f92380701c0 legacy=0x7f92380725b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:31.211 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:31.210+0000 7f923ea38640 1 -- 172.21.15.27:0/1373911869 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f923815f270 con 0x7f9238072cc0 2024-09-12T03:22:31.212 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:31.211+0000 7f923ea38640 1 --2- 172.21.15.27:0/1373911869 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9238072cc0 0x7f92380705e0 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7f923806e020 tx=0x7f923002a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81d36a36b678e7f7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:31.213 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:31.211+0000 7f923d235640 1 -- 172.21.15.27:0/1373911869 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9230007740 con 0x7f9238072cc0 2024-09-12T03:22:31.213 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:31.211+0000 7f923d235640 1 -- 172.21.15.27:0/1373911869 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92300078e0 con 0x7f9238072cc0 2024-09-12T03:22:31.213 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:31.211+0000 7f923d235640 1 -- 172.21.15.27:0/1373911869 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9230007cb0 con 0x7f9238072cc0 2024-09-12T03:22:31.213 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:31.211+0000 7f924049b640 1 -- 172.21.15.27:0/1373911869 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9238072cc0 msgr2=0x7f92380705e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:31.213 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:31.211+0000 7f924049b640 1 --2- 172.21.15.27:0/1373911869 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9238072cc0 0x7f92380705e0 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7f923806e020 tx=0x7f923002a5f0 comp rx=0 tx=0).stop 2024-09-12T03:22:31.213 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:31.212+0000 7f924049b640 1 -- 172.21.15.27:0/1373911869 shutdown_connections 2024-09-12T03:22:31.213 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:31.212+0000 7f924049b640 1 --2- 172.21.15.27:0/1373911869 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9238072cc0 0x7f92380705e0 unknown :-1 s=CLOSED pgs=1293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:31.213 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:31.212+0000 7f924049b640 1 --2- 172.21.15.27:0/1373911869 >> v2:172.21.15.45:3300/0 conn(0x7f923806f990 0x7f923806e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:31.213 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:31.212+0000 7f924049b640 1 -- 172.21.15.27:0/1373911869 >> 172.21.15.27:0/1373911869 conn(0x7f923806d060 msgr2=0x7f923806d470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:31.213 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:31.212+0000 7f924049b640 1 -- 172.21.15.27:0/1373911869 shutdown_connections 2024-09-12T03:22:31.213 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:31.212+0000 7f924049b640 1 -- 172.21.15.27:0/1373911869 wait complete. 2024-09-12T03:22:31.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:31.213+0000 7f924049b640 1 Processor -- start 2024-09-12T03:22:31.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:31.213+0000 7f924049b640 1 -- start start 2024-09-12T03:22:31.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:31.214+0000 7f924049b640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f923806f990 0x7f92381f42a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:31.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:31.214+0000 7f924049b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f92380701c0 0x7f92380b25a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:31.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:31.214+0000 7f924049b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9238072cc0 0x7f9238159b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:31.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:31.214+0000 7f924049b640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f923815fb10 con 0x7f92380701c0 2024-09-12T03:22:31.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:31.214+0000 7f924049b640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f923815f830 con 0x7f9238072cc0 2024-09-12T03:22:31.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:31.214+0000 7f924049b640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f923815eb50 con 0x7f923806f990 2024-09-12T03:22:31.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:31.214+0000 7f923da36640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f92380701c0 0x7f92380b25a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:31.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:31.214+0000 7f923da36640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f92380701c0 0x7f92380b25a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54874/0 (socket says 172.21.15.27:54874) 2024-09-12T03:22:31.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:31.214+0000 7f923da36640 1 -- 172.21.15.27:0/282752181 learned_addr learned my addr 172.21.15.27:0/282752181 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:22:31.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:31.214+0000 7f923e237640 1 --2- 172.21.15.27:0/282752181 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f923806f990 0x7f92381f42a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:31.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:31.214+0000 7f923ea38640 1 --2- 172.21.15.27:0/282752181 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9238072cc0 0x7f9238159b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:31.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:31.214+0000 7f923da36640 1 -- 172.21.15.27:0/282752181 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f923806f990 msgr2=0x7f92381f42a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:31.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:31.214+0000 7f923da36640 1 --2- 172.21.15.27:0/282752181 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f923806f990 0x7f92381f42a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:31.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:31.214+0000 7f923da36640 1 -- 172.21.15.27:0/282752181 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9238072cc0 msgr2=0x7f9238159b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:31.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:31.214+0000 7f923da36640 1 --2- 172.21.15.27:0/282752181 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9238072cc0 0x7f9238159b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:31.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:31.214+0000 7f923da36640 1 -- 172.21.15.27:0/282752181 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92380c7960 con 0x7f92380701c0 2024-09-12T03:22:31.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:31.214+0000 7f923ea38640 1 --2- 172.21.15.27:0/282752181 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9238072cc0 0x7f9238159b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:22:31.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:31.214+0000 7f923e237640 1 --2- 172.21.15.27:0/282752181 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f923806f990 0x7f92381f42a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:22:31.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:31.214+0000 7f923da36640 1 --2- 172.21.15.27:0/282752181 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f92380701c0 0x7f92380b25a0 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7f922c00c3c0 tx=0x7f922c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:31.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:31.214+0000 7f922affd640 1 -- 172.21.15.27:0/282752181 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f922c017070 con 0x7f92380701c0 2024-09-12T03:22:31.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:31.214+0000 7f922affd640 1 -- 172.21.15.27:0/282752181 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f922c005ce0 con 0x7f92380701c0 2024-09-12T03:22:31.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:31.215+0000 7f924049b640 1 -- 172.21.15.27:0/282752181 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92380c7c40 con 0x7f92380701c0 2024-09-12T03:22:31.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:31.215+0000 7f922affd640 1 -- 172.21.15.27:0/282752181 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f922c0072f0 con 0x7f92380701c0 2024-09-12T03:22:31.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:31.215+0000 7f924049b640 1 -- 172.21.15.27:0/282752181 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92380c8200 con 0x7f92380701c0 2024-09-12T03:22:31.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:31.215+0000 7f924049b640 1 -- 172.21.15.27:0/282752181 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f923806e820 con 0x7f92380701c0 2024-09-12T03:22:31.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:31.216+0000 7f922affd640 1 -- 172.21.15.27:0/282752181 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 21) ==== 91008+0+0 (secure 0 0 0) 0x7f922c01a070 con 0x7f92380701c0 2024-09-12T03:22:31.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:31.217+0000 7f922affd640 1 --2- 172.21.15.27:0/282752181 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f921c06c400 0x7f921c06e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:31.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:31.217+0000 7f922affd640 1 -- 172.21.15.27:0/282752181 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 5765+0+0 (secure 0 0 0) 0x7f922c08abd0 con 0x7f92380701c0 2024-09-12T03:22:31.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:31.217+0000 7f923e237640 1 --2- 172.21.15.27:0/282752181 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f921c06c400 0x7f921c06e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:31.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:31.217+0000 7f923e237640 1 --2- 172.21.15.27:0/282752181 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f921c06c400 0x7f921c06e8c0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f9234006fd0 tx=0x7f9234008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:31.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:31.221+0000 7f922affd640 1 -- 172.21.15.27:0/282752181 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f922c058d60 con 0x7f92380701c0 2024-09-12T03:22:31.426 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:31.426+0000 7f924049b640 1 -- 172.21.15.27:0/282752181 --> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot info", "vol_name": "non_existent_subvolume", "sub_name": "subvolume_1", "snap_name": "snapshot_1", "target": ["mon-mgr", ""]}) -- 0x7f92381f47e0 con 0x7f921c06c400 2024-09-12T03:22:31.427 INFO:tasks.ceph.mgr.y.smithi027.stderr:2024-09-12T03:22:31.427+0000 7f325d8e3640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-12T03:22:31.427 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:31.427+0000 7f922affd640 1 -- 172.21.15.27:0/282752181 <== mgr.4113 v2:172.21.15.27:6836/1226920526 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7f92381f47e0 con 0x7f921c06c400 2024-09-12T03:22:31.427 INFO:teuthology.orchestra.run.smithi027.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-12T03:22:31.432 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-12T03:22:31.433 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot protect non_existent_subvolume subvolume_1 snapshot_1 2024-09-12T03:22:31.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:31.548+0000 7f9e5f04c640 1 Processor -- start 2024-09-12T03:22:31.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:31.548+0000 7f9e5f04c640 1 -- start start 2024-09-12T03:22:31.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:31.548+0000 7f9e5f04c640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f9e58162040 0x7f9e580d1da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:31.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:31.549+0000 7f9e5f04c640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f9e5806dfd0 0x7f9e580b7a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:31.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:31.549+0000 7f9e5f04c640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9e580d2e30 con 0x7f9e580d3790 2024-09-12T03:22:31.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:31.549+0000 7f9e5f04c640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f9e580b7f90 con 0x7f9e58162040 2024-09-12T03:22:31.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:31.549+0000 7f9e5f04c640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f9e58162420 con 0x7f9e5806dfd0 2024-09-12T03:22:31.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:31.549+0000 7f9e5cde8640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f9e58162040 0x7f9e580d1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:31.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:31.549+0000 7f9e5cde8640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f9e58162040 0x7f9e580d1da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54876/0 (socket says 172.21.15.27:54876) 2024-09-12T03:22:31.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:31.549+0000 7f9e5cde8640 1 -- 172.21.15.27:0/2625314086 learned_addr learned my addr 172.21.15.27:0/2625314086 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:22:31.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:31.549+0000 7f9e5d5e9640 1 --2- 172.21.15.27:0/2625314086 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f9e5806dfd0 0x7f9e580b7a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:31.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:31.549+0000 7f9e5cde8640 1 -- 172.21.15.27:0/2625314086 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f9e5806dfd0 msgr2=0x7f9e580b7a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:31.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:31.549+0000 7f9e5cde8640 1 --2- 172.21.15.27:0/2625314086 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f9e5806dfd0 0x7f9e580b7a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:31.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:31.549+0000 7f9e5cde8640 1 -- 172.21.15.27:0/2625314086 >> v1:172.21.15.45:6789/0 conn(0x7f9e580d3790 legacy=0x7f9e5806d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:31.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:31.549+0000 7f9e5cde8640 1 -- 172.21.15.27:0/2625314086 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e580d30d0 con 0x7f9e58162040 2024-09-12T03:22:31.551 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:31.550+0000 7f9e5cde8640 1 --2- 172.21.15.27:0/2625314086 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9e58162040 0x7f9e580d1da0 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7f9e4c009870 tx=0x7f9e4c02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d374e41cf08830c6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:31.551 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:31.550+0000 7f9e577fe640 1 -- 172.21.15.27:0/2625314086 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e4c037070 con 0x7f9e58162040 2024-09-12T03:22:31.551 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:31.550+0000 7f9e577fe640 1 -- 172.21.15.27:0/2625314086 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e4c032440 con 0x7f9e58162040 2024-09-12T03:22:31.551 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:31.550+0000 7f9e577fe640 1 -- 172.21.15.27:0/2625314086 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e4c032810 con 0x7f9e58162040 2024-09-12T03:22:31.551 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:31.550+0000 7f9e5f04c640 1 -- 172.21.15.27:0/2625314086 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9e58162040 msgr2=0x7f9e580d1da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:31.551 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:31.550+0000 7f9e5f04c640 1 --2- 172.21.15.27:0/2625314086 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9e58162040 0x7f9e580d1da0 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7f9e4c009870 tx=0x7f9e4c02a210 comp rx=0 tx=0).stop 2024-09-12T03:22:31.552 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:31.551+0000 7f9e5f04c640 1 -- 172.21.15.27:0/2625314086 shutdown_connections 2024-09-12T03:22:31.552 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:31.551+0000 7f9e5f04c640 1 --2- 172.21.15.27:0/2625314086 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f9e5806dfd0 0x7f9e580b7a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:31.552 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:31.551+0000 7f9e5f04c640 1 --2- 172.21.15.27:0/2625314086 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9e58162040 0x7f9e580d1da0 unknown :-1 s=CLOSED pgs=1295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:31.552 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:31.551+0000 7f9e5f04c640 1 -- 172.21.15.27:0/2625314086 >> 172.21.15.27:0/2625314086 conn(0x7f9e580b6700 msgr2=0x7f9e580b6b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:31.552 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:31.551+0000 7f9e5f04c640 1 -- 172.21.15.27:0/2625314086 shutdown_connections 2024-09-12T03:22:31.552 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:31.551+0000 7f9e5f04c640 1 -- 172.21.15.27:0/2625314086 wait complete. 2024-09-12T03:22:31.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:31.552+0000 7f9e5f04c640 1 Processor -- start 2024-09-12T03:22:31.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:31.552+0000 7f9e5f04c640 1 -- start start 2024-09-12T03:22:31.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:31.553+0000 7f9e5f04c640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f9e5806dfd0 0x7f9e581f7cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:31.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:31.553+0000 7f9e5f04c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9e580d3790 0x7f9e580cf320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:31.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:31.553+0000 7f9e5f04c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9e58162040 0x7f9e580cf860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:31.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:31.553+0000 7f9e5f04c640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f9e580d4530 con 0x7f9e580d3790 2024-09-12T03:22:31.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:31.553+0000 7f9e5f04c640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f9e580cee00 con 0x7f9e58162040 2024-09-12T03:22:31.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:31.553+0000 7f9e5f04c640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f9e580d4150 con 0x7f9e5806dfd0 2024-09-12T03:22:31.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:31.553+0000 7f9e57fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9e580d3790 0x7f9e580cf320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:31.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:31.553+0000 7f9e57fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9e580d3790 0x7f9e580cf320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54882/0 (socket says 172.21.15.27:54882) 2024-09-12T03:22:31.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:31.553+0000 7f9e57fff640 1 -- 172.21.15.27:0/1309539121 learned_addr learned my addr 172.21.15.27:0/1309539121 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:22:31.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:31.553+0000 7f9e5cde8640 1 --2- 172.21.15.27:0/1309539121 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f9e5806dfd0 0x7f9e581f7cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:31.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:31.553+0000 7f9e5d5e9640 1 --2- 172.21.15.27:0/1309539121 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9e58162040 0x7f9e580cf860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:31.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:31.553+0000 7f9e57fff640 1 -- 172.21.15.27:0/1309539121 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f9e5806dfd0 msgr2=0x7f9e581f7cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:31.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:31.553+0000 7f9e57fff640 1 --2- 172.21.15.27:0/1309539121 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f9e5806dfd0 0x7f9e581f7cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:31.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:31.553+0000 7f9e57fff640 1 -- 172.21.15.27:0/1309539121 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9e58162040 msgr2=0x7f9e580cf860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:31.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:31.553+0000 7f9e57fff640 1 --2- 172.21.15.27:0/1309539121 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9e58162040 0x7f9e580cf860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:31.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:31.553+0000 7f9e57fff640 1 -- 172.21.15.27:0/1309539121 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e5815e4a0 con 0x7f9e580d3790 2024-09-12T03:22:31.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:31.553+0000 7f9e5d5e9640 1 --2- 172.21.15.27:0/1309539121 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9e58162040 0x7f9e580cf860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:22:31.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:31.553+0000 7f9e57fff640 1 --2- 172.21.15.27:0/1309539121 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9e580d3790 0x7f9e580cf320 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7f9e4400c400 tx=0x7f9e4400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:31.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:31.553+0000 7f9e557fa640 1 -- 172.21.15.27:0/1309539121 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e44011790 con 0x7f9e580d3790 2024-09-12T03:22:31.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:31.553+0000 7f9e557fa640 1 -- 172.21.15.27:0/1309539121 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e44017070 con 0x7f9e580d3790 2024-09-12T03:22:31.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:31.553+0000 7f9e5f04c640 1 -- 172.21.15.27:0/1309539121 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e5815e780 con 0x7f9e580d3790 2024-09-12T03:22:31.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:31.554+0000 7f9e557fa640 1 -- 172.21.15.27:0/1309539121 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e44011bb0 con 0x7f9e580d3790 2024-09-12T03:22:31.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:31.554+0000 7f9e5f04c640 1 -- 172.21.15.27:0/1309539121 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e5815ed40 con 0x7f9e580d3790 2024-09-12T03:22:31.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:31.554+0000 7f9e5f04c640 1 -- 172.21.15.27:0/1309539121 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e580b7a50 con 0x7f9e580d3790 2024-09-12T03:22:31.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:31.555+0000 7f9e557fa640 1 -- 172.21.15.27:0/1309539121 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 21) ==== 91008+0+0 (secure 0 0 0) 0x7f9e4401c460 con 0x7f9e580d3790 2024-09-12T03:22:31.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:31.555+0000 7f9e557fa640 1 --2- 172.21.15.27:0/1309539121 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f9e3c06c400 0x7f9e3c06e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:31.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:31.556+0000 7f9e557fa640 1 -- 172.21.15.27:0/1309539121 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 5765+0+0 (secure 0 0 0) 0x7f9e4408b630 con 0x7f9e580d3790 2024-09-12T03:22:31.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:31.556+0000 7f9e5cde8640 1 --2- 172.21.15.27:0/1309539121 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f9e3c06c400 0x7f9e3c06e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:31.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:31.556+0000 7f9e5cde8640 1 --2- 172.21.15.27:0/1309539121 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f9e3c06c400 0x7f9e3c06e8c0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f9e4c034000 tx=0x7f9e4c02e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:31.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:31.560+0000 7f9e557fa640 1 -- 172.21.15.27:0/1309539121 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f9e44059830 con 0x7f9e580d3790 2024-09-12T03:22:31.765 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:31.765+0000 7f9e5f04c640 1 -- 172.21.15.27:0/1309539121 --> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot protect", "vol_name": "non_existent_subvolume", "sub_name": "subvolume_1", "snap_name": "snapshot_1", "target": ["mon-mgr", ""]}) -- 0x7f9e5806e6e0 con 0x7f9e3c06c400 2024-09-12T03:22:31.766 INFO:tasks.ceph.mgr.y.smithi027.stderr:2024-09-12T03:22:31.766+0000 7f325d8e3640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-12T03:22:31.766 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:31.766+0000 7f9e557fa640 1 -- 172.21.15.27:0/1309539121 <== mgr.4113 v2:172.21.15.27:6836/1226920526 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7f9e5806e6e0 con 0x7f9e3c06c400 2024-09-12T03:22:31.766 INFO:teuthology.orchestra.run.smithi027.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-12T03:22:31.771 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-12T03:22:31.772 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot unprotect non_existent_subvolume subvolume_1 snapshot_1 2024-09-12T03:22:31.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:31.880+0000 7fbeff27b640 1 Processor -- start 2024-09-12T03:22:31.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:31.880+0000 7fbeff27b640 1 -- start start 2024-09-12T03:22:31.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:31.881+0000 7fbeff27b640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fbef80b3d10 0x7fbef80b66c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:31.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:31.881+0000 7fbeff27b640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbef815ed70 con 0x7fbef80b6230 2024-09-12T03:22:31.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:31.881+0000 7fbeff27b640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbef80b7d10 con 0x7fbef815e990 2024-09-12T03:22:31.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:31.881+0000 7fbeff27b640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fbef80d2a30 con 0x7fbef80b3d10 2024-09-12T03:22:31.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:31.881+0000 7fbefc816640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fbef80b6230 0x7fbef80b3600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:38598/0 (socket says 172.21.15.27:38598) 2024-09-12T03:22:31.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:31.881+0000 7fbefc816640 1 -- 172.21.15.27:0/1188500819 learned_addr learned my addr 172.21.15.27:0/1188500819 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:22:31.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:31.881+0000 7fbefd818640 1 --2- 172.21.15.27:0/1188500819 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fbef80b3d10 0x7fbef80b66c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:31.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:31.881+0000 7fbeeffff640 1 -- 172.21.15.27:0/1188500819 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 749029082 0 0) 0x7fbef815ed70 con 0x7fbef80b6230 2024-09-12T03:22:31.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:31.881+0000 7fbeeffff640 1 -- 172.21.15.27:0/1188500819 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbee4003520 con 0x7fbef80b6230 2024-09-12T03:22:31.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:31.881+0000 7fbeeffff640 1 -- 172.21.15.27:0/1188500819 <== mon.1 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2665078632 0 0) 0x7fbef80b7d10 con 0x7fbef815e990 2024-09-12T03:22:31.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:31.881+0000 7fbeeffff640 1 -- 172.21.15.27:0/1188500819 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbef815ed70 con 0x7fbef815e990 2024-09-12T03:22:31.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:31.881+0000 7fbefd818640 1 -- 172.21.15.27:0/1188500819 >> v1:172.21.15.45:6789/0 conn(0x7fbef815e990 legacy=0x7fbef80b7600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:31.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:31.881+0000 7fbeeffff640 1 -- 172.21.15.27:0/1188500819 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3234579380 0 0) 0x7fbee4003520 con 0x7fbef80b6230 2024-09-12T03:22:31.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:31.881+0000 7fbefd818640 1 -- 172.21.15.27:0/1188500819 >> v1:172.21.15.27:6789/0 conn(0x7fbef80b6230 legacy=0x7fbef80b3600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:31.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:31.882+0000 7fbefd818640 1 -- 172.21.15.27:0/1188500819 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbef80d2d10 con 0x7fbef80b3d10 2024-09-12T03:22:31.886 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:31.882+0000 7fbefd818640 1 --2- 172.21.15.27:0/1188500819 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fbef80b3d10 0x7fbef80b66c0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fbef0009dc0 tx=0x7fbef002a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7e5a502e3d24c2cb server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:31.886 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:31.882+0000 7fbeeffff640 1 -- 172.21.15.27:0/1188500819 <== mon.2 v2:172.21.15.45:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbee8001670 con 0x7fbef80b3d10 2024-09-12T03:22:31.886 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:31.883+0000 7fbeeffff640 1 -- 172.21.15.27:0/1188500819 <== mon.2 v2:172.21.15.45:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbef0007540 con 0x7fbef80b3d10 2024-09-12T03:22:31.886 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:31.883+0000 7fbeeffff640 1 -- 172.21.15.27:0/1188500819 <== mon.2 v2:172.21.15.45:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbef00078f0 con 0x7fbef80b3d10 2024-09-12T03:22:31.886 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:31.883+0000 7fbeff27b640 1 -- 172.21.15.27:0/1188500819 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fbef80b3d10 msgr2=0x7fbef80b66c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:31.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:31.883+0000 7fbeff27b640 1 --2- 172.21.15.27:0/1188500819 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fbef80b3d10 0x7fbef80b66c0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fbef0009dc0 tx=0x7fbef002a660 comp rx=0 tx=0).stop 2024-09-12T03:22:31.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:31.883+0000 7fbeff27b640 1 -- 172.21.15.27:0/1188500819 shutdown_connections 2024-09-12T03:22:31.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:31.883+0000 7fbeff27b640 1 --2- 172.21.15.27:0/1188500819 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fbef80b3d10 0x7fbef80b66c0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:31.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:31.883+0000 7fbeff27b640 1 -- 172.21.15.27:0/1188500819 >> 172.21.15.27:0/1188500819 conn(0x7fbef80d1230 msgr2=0x7fbef80d1640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:31.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:31.883+0000 7fbeff27b640 1 -- 172.21.15.27:0/1188500819 shutdown_connections 2024-09-12T03:22:31.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:31.883+0000 7fbeff27b640 1 -- 172.21.15.27:0/1188500819 wait complete. 2024-09-12T03:22:31.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:31.884+0000 7fbeff27b640 1 Processor -- start 2024-09-12T03:22:31.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:31.884+0000 7fbeff27b640 1 -- start start 2024-09-12T03:22:31.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:31.885+0000 7fbeff27b640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fbef80b3d10 0x7fbef8158aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:31.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:31.885+0000 7fbeff27b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbef80b6230 0x7fbef80b5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:31.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:31.885+0000 7fbeff27b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbef815e990 0x7fbef80b5a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:31.888 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:31.885+0000 7fbeff27b640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fbef80d35b0 con 0x7fbef80b6230 2024-09-12T03:22:31.888 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:31.885+0000 7fbeff27b640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fbef80d32d0 con 0x7fbef815e990 2024-09-12T03:22:31.888 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:31.885+0000 7fbeff27b640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fbef80d3ad0 con 0x7fbef80b3d10 2024-09-12T03:22:31.888 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:31.885+0000 7fbefc816640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbef80b6230 0x7fbef80b5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:31.888 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:31.885+0000 7fbefc816640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbef80b6230 0x7fbef80b5550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54890/0 (socket says 172.21.15.27:54890) 2024-09-12T03:22:31.888 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:31.885+0000 7fbefc816640 1 -- 172.21.15.27:0/2057068842 learned_addr learned my addr 172.21.15.27:0/2057068842 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:22:31.888 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:31.885+0000 7fbefd818640 1 --2- 172.21.15.27:0/2057068842 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbef815e990 0x7fbef80b5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:31.888 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:31.885+0000 7fbefd017640 1 --2- 172.21.15.27:0/2057068842 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fbef80b3d10 0x7fbef8158aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:31.888 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:31.885+0000 7fbefc816640 1 -- 172.21.15.27:0/2057068842 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fbef80b3d10 msgr2=0x7fbef8158aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:31.888 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:31.886+0000 7fbefc816640 1 --2- 172.21.15.27:0/2057068842 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fbef80b3d10 0x7fbef8158aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:31.888 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:31.886+0000 7fbefc816640 1 -- 172.21.15.27:0/2057068842 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbef815e990 msgr2=0x7fbef80b5a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:31.888 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:31.886+0000 7fbefc816640 1 --2- 172.21.15.27:0/2057068842 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbef815e990 0x7fbef80b5a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:31.888 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:31.886+0000 7fbefc816640 1 -- 172.21.15.27:0/2057068842 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbef80ca4c0 con 0x7fbef80b6230 2024-09-12T03:22:31.889 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:31.886+0000 7fbefd017640 1 --2- 172.21.15.27:0/2057068842 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fbef80b3d10 0x7fbef8158aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:22:31.889 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:31.886+0000 7fbefc816640 1 --2- 172.21.15.27:0/2057068842 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbef80b6230 0x7fbef80b5550 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7fbee800c5d0 tx=0x7fbee800ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:31.889 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:31.886+0000 7fbeedffb640 1 -- 172.21.15.27:0/2057068842 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbee8017070 con 0x7fbef80b6230 2024-09-12T03:22:31.889 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:31.886+0000 7fbeedffb640 1 -- 172.21.15.27:0/2057068842 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbee8005ce0 con 0x7fbef80b6230 2024-09-12T03:22:31.889 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:31.886+0000 7fbeff27b640 1 -- 172.21.15.27:0/2057068842 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbef80ca7a0 con 0x7fbef80b6230 2024-09-12T03:22:31.889 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:31.886+0000 7fbeedffb640 1 -- 172.21.15.27:0/2057068842 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbee80072b0 con 0x7fbef80b6230 2024-09-12T03:22:31.889 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:31.886+0000 7fbeff27b640 1 -- 172.21.15.27:0/2057068842 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbef80cad60 con 0x7fbef80b6230 2024-09-12T03:22:31.890 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:31.887+0000 7fbeff27b640 1 -- 172.21.15.27:0/2057068842 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbef80b66c0 con 0x7fbef80b6230 2024-09-12T03:22:31.890 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:31.888+0000 7fbeedffb640 1 -- 172.21.15.27:0/2057068842 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 21) ==== 91008+0+0 (secure 0 0 0) 0x7fbee801a070 con 0x7fbef80b6230 2024-09-12T03:22:31.890 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:31.888+0000 7fbeedffb640 1 --2- 172.21.15.27:0/2057068842 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fbedc06c3d0 0x7fbedc06e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:31.890 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:31.888+0000 7fbeedffb640 1 -- 172.21.15.27:0/2057068842 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 5765+0+0 (secure 0 0 0) 0x7fbee808ad60 con 0x7fbef80b6230 2024-09-12T03:22:31.890 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:31.888+0000 7fbefd017640 1 --2- 172.21.15.27:0/2057068842 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fbedc06c3d0 0x7fbedc06e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:31.890 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:31.889+0000 7fbefd017640 1 --2- 172.21.15.27:0/2057068842 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fbedc06c3d0 0x7fbedc06e890 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fbef40011e0 tx=0x7fbef4009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:31.892 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:31.893+0000 7fbeedffb640 1 -- 172.21.15.27:0/2057068842 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fbee8058810 con 0x7fbef80b6230 2024-09-12T03:22:32.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:32.102+0000 7fbeff27b640 1 -- 172.21.15.27:0/2057068842 --> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot unprotect", "vol_name": "non_existent_subvolume", "sub_name": "subvolume_1", "snap_name": "snapshot_1", "target": ["mon-mgr", ""]}) -- 0x7fbef815dd10 con 0x7fbedc06c3d0 2024-09-12T03:22:32.102 INFO:tasks.ceph.mgr.y.smithi027.stderr:2024-09-12T03:22:32.103+0000 7f325d8e3640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-12T03:22:32.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:32.103+0000 7fbeedffb640 1 -- 172.21.15.27:0/2057068842 <== mgr.4113 v2:172.21.15.27:6836/1226920526 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7fbef815dd10 con 0x7fbedc06c3d0 2024-09-12T03:22:32.103 INFO:teuthology.orchestra.run.smithi027.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-12T03:22:32.108 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-12T03:22:32.109 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot ls non_existent_subvolume subvolume_1 2024-09-12T03:22:32.215 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:32.216+0000 7faf85bcf640 1 Processor -- start 2024-09-12T03:22:32.216 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:32.216+0000 7faf85bcf640 1 -- start start 2024-09-12T03:22:32.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:32.216+0000 7faf85bcf640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7faf80162040 0x7faf800d1da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:32.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:32.216+0000 7faf85bcf640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7faf800d3790 0x7faf800b7a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:32.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:32.216+0000 7faf85bcf640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faf800d2e30 con 0x7faf8006d8c0 2024-09-12T03:22:32.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:32.216+0000 7faf85bcf640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7faf800b7f90 con 0x7faf80162040 2024-09-12T03:22:32.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:32.216+0000 7faf85bcf640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7faf80162420 con 0x7faf800d3790 2024-09-12T03:22:32.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:32.217+0000 7faf7f7fe640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7faf80162040 0x7faf800d1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:32.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:32.217+0000 7faf7effd640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7faf800d3790 0x7faf800b7a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:32.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:32.217+0000 7faf7ffff640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7faf8006d8c0 0x7faf800cf320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.27:41122/0 (socket says 172.21.15.27:41122) 2024-09-12T03:22:32.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:32.217+0000 7faf7ffff640 1 -- 172.21.15.27:0/1793343135 learned_addr learned my addr 172.21.15.27:0/1793343135 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:22:32.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:32.217+0000 7faf7e7fc640 1 -- 172.21.15.27:0/1793343135 <== mon.1 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3486499103 0 0) 0x7faf800d2e30 con 0x7faf8006d8c0 2024-09-12T03:22:32.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:32.217+0000 7faf7e7fc640 1 -- 172.21.15.27:0/1793343135 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7faf68003520 con 0x7faf8006d8c0 2024-09-12T03:22:32.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:32.217+0000 7faf7effd640 1 -- 172.21.15.27:0/1793343135 >> v2:172.21.15.45:3300/0 conn(0x7faf80162040 msgr2=0x7faf800d1da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:32.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:32.217+0000 7faf7effd640 1 --2- 172.21.15.27:0/1793343135 >> v2:172.21.15.45:3300/0 conn(0x7faf80162040 0x7faf800d1da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:32.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:32.217+0000 7faf7effd640 1 -- 172.21.15.27:0/1793343135 >> v1:172.21.15.45:6789/0 conn(0x7faf8006d8c0 legacy=0x7faf800cf320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:32.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:32.217+0000 7faf7effd640 1 -- 172.21.15.27:0/1793343135 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf800d30d0 con 0x7faf800d3790 2024-09-12T03:22:32.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:32.217+0000 7faf7f7fe640 1 --2- 172.21.15.27:0/1793343135 >> v2:172.21.15.45:3300/0 conn(0x7faf80162040 0x7faf800d1da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:22:32.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:32.218+0000 7faf7effd640 1 --2- 172.21.15.27:0/1793343135 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7faf800d3790 0x7faf800b7a50 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7faf740097a0 tx=0x7faf7402a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bb5ace859ff425f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:32.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:32.218+0000 7faf7e7fc640 1 -- 172.21.15.27:0/1793343135 <== mon.2 v2:172.21.15.45:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf74037070 con 0x7faf800d3790 2024-09-12T03:22:32.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:32.218+0000 7faf7e7fc640 1 -- 172.21.15.27:0/1793343135 <== mon.2 v2:172.21.15.45:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf74032440 con 0x7faf800d3790 2024-09-12T03:22:32.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:32.218+0000 7faf7e7fc640 1 -- 172.21.15.27:0/1793343135 <== mon.2 v2:172.21.15.45:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf740327f0 con 0x7faf800d3790 2024-09-12T03:22:32.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:32.219+0000 7faf85bcf640 1 -- 172.21.15.27:0/1793343135 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7faf800d3790 msgr2=0x7faf800b7a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:32.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:32.219+0000 7faf85bcf640 1 --2- 172.21.15.27:0/1793343135 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7faf800d3790 0x7faf800b7a50 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7faf740097a0 tx=0x7faf7402a180 comp rx=0 tx=0).stop 2024-09-12T03:22:32.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:32.219+0000 7faf85bcf640 1 -- 172.21.15.27:0/1793343135 shutdown_connections 2024-09-12T03:22:32.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:32.219+0000 7faf85bcf640 1 --2- 172.21.15.27:0/1793343135 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7faf800d3790 0x7faf800b7a50 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:32.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:32.219+0000 7faf85bcf640 1 --2- 172.21.15.27:0/1793343135 >> v2:172.21.15.45:3300/0 conn(0x7faf80162040 0x7faf800d1da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:32.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:32.219+0000 7faf85bcf640 1 -- 172.21.15.27:0/1793343135 >> 172.21.15.27:0/1793343135 conn(0x7faf800b6700 msgr2=0x7faf800b6b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:32.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:32.219+0000 7faf85bcf640 1 -- 172.21.15.27:0/1793343135 shutdown_connections 2024-09-12T03:22:32.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:32.219+0000 7faf85bcf640 1 -- 172.21.15.27:0/1793343135 wait complete. 2024-09-12T03:22:32.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:32.220+0000 7faf85bcf640 1 Processor -- start 2024-09-12T03:22:32.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:32.220+0000 7faf85bcf640 1 -- start start 2024-09-12T03:22:32.223 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:32.221+0000 7faf85bcf640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faf8006d8c0 0x7faf801567b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:32.223 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:32.221+0000 7faf85bcf640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faf800d3790 0x7faf801599e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:32.223 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:32.221+0000 7faf85bcf640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7faf80162040 0x7faf80159f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:32.223 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:32.221+0000 7faf85bcf640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7faf800d4530 con 0x7faf8006d8c0 2024-09-12T03:22:32.223 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:32.221+0000 7faf85bcf640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7faf800cee00 con 0x7faf800d3790 2024-09-12T03:22:32.223 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:32.221+0000 7faf85bcf640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7faf800d4150 con 0x7faf80162040 2024-09-12T03:22:32.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:32.221+0000 7faf7f7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faf8006d8c0 0x7faf801567b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:32.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:32.221+0000 7faf7f7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faf8006d8c0 0x7faf801567b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54896/0 (socket says 172.21.15.27:54896) 2024-09-12T03:22:32.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:32.221+0000 7faf7f7fe640 1 -- 172.21.15.27:0/1609434766 learned_addr learned my addr 172.21.15.27:0/1609434766 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:22:32.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:32.221+0000 7faf7ffff640 1 --2- 172.21.15.27:0/1609434766 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7faf80162040 0x7faf80159f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:32.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:32.221+0000 7faf7effd640 1 --2- 172.21.15.27:0/1609434766 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faf800d3790 0x7faf801599e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:32.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:32.222+0000 7faf7f7fe640 1 -- 172.21.15.27:0/1609434766 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7faf80162040 msgr2=0x7faf80159f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:32.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:32.222+0000 7faf7f7fe640 1 --2- 172.21.15.27:0/1609434766 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7faf80162040 0x7faf80159f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:32.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:32.222+0000 7faf7f7fe640 1 -- 172.21.15.27:0/1609434766 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faf800d3790 msgr2=0x7faf801599e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:32.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:32.222+0000 7faf7f7fe640 1 --2- 172.21.15.27:0/1609434766 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faf800d3790 0x7faf801599e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:32.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:32.222+0000 7faf7f7fe640 1 -- 172.21.15.27:0/1609434766 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf80157340 con 0x7faf8006d8c0 2024-09-12T03:22:32.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:32.222+0000 7faf7f7fe640 1 --2- 172.21.15.27:0/1609434766 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faf8006d8c0 0x7faf801567b0 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7faf7000ec10 tx=0x7faf7000c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:32.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:32.222+0000 7faf67fff640 1 -- 172.21.15.27:0/1609434766 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf70017070 con 0x7faf8006d8c0 2024-09-12T03:22:32.225 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:32.222+0000 7faf67fff640 1 -- 172.21.15.27:0/1609434766 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf7000cc30 con 0x7faf8006d8c0 2024-09-12T03:22:32.225 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:32.222+0000 7faf67fff640 1 -- 172.21.15.27:0/1609434766 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf70013470 con 0x7faf8006d8c0 2024-09-12T03:22:32.225 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:32.222+0000 7faf85bcf640 1 -- 172.21.15.27:0/1609434766 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf801575b0 con 0x7faf8006d8c0 2024-09-12T03:22:32.225 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:32.222+0000 7faf85bcf640 1 -- 172.21.15.27:0/1609434766 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf801f9090 con 0x7faf8006d8c0 2024-09-12T03:22:32.225 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:32.223+0000 7faf85bcf640 1 -- 172.21.15.27:0/1609434766 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf800b7a50 con 0x7faf8006d8c0 2024-09-12T03:22:32.225 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:32.223+0000 7faf67fff640 1 -- 172.21.15.27:0/1609434766 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 21) ==== 91008+0+0 (secure 0 0 0) 0x7faf7001a070 con 0x7faf8006d8c0 2024-09-12T03:22:32.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:32.224+0000 7faf67fff640 1 --2- 172.21.15.27:0/1609434766 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7faf6c06c3d0 0x7faf6c06e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:32.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:32.224+0000 7faf67fff640 1 -- 172.21.15.27:0/1609434766 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 5765+0+0 (secure 0 0 0) 0x7faf7008bc60 con 0x7faf8006d8c0 2024-09-12T03:22:32.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:32.224+0000 7faf7effd640 1 --2- 172.21.15.27:0/1609434766 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7faf6c06c3d0 0x7faf6c06e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:32.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:32.224+0000 7faf7effd640 1 --2- 172.21.15.27:0/1609434766 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7faf6c06c3d0 0x7faf6c06e890 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7faf74034000 tx=0x7faf7402e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:32.228 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:32.228+0000 7faf67fff640 1 -- 172.21.15.27:0/1609434766 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7faf70059ce0 con 0x7faf8006d8c0 2024-09-12T03:22:32.430 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:32.430+0000 7faf85bcf640 1 -- 172.21.15.27:0/1609434766 --> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot ls", "vol_name": "non_existent_subvolume", "sub_name": "subvolume_1", "target": ["mon-mgr", ""]}) -- 0x7faf8006e6e0 con 0x7faf6c06c3d0 2024-09-12T03:22:32.432 INFO:tasks.ceph.mgr.y.smithi027.stderr:2024-09-12T03:22:32.432+0000 7f325d8e3640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-12T03:22:32.432 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:32.432+0000 7faf67fff640 1 -- 172.21.15.27:0/1609434766 <== mgr.4113 v2:172.21.15.27:6836/1226920526 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7faf8006e6e0 con 0x7faf6c06c3d0 2024-09-12T03:22:32.432 INFO:teuthology.orchestra.run.smithi027.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-12T03:22:32.437 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-12T03:22:32.438 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone non_existent_subvolume subvolume_1 snapshot_1 clone_1 2024-09-12T03:22:32.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:32.548+0000 7f2920ee6640 1 Processor -- start 2024-09-12T03:22:32.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:32.548+0000 7f2920ee6640 1 -- start start 2024-09-12T03:22:32.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:32.548+0000 7f2920ee6640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f291c070d30 0x7f291c06cc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:32.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:32.549+0000 7f2920ee6640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f291c0d0b50 0x7f291c168300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:32.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:32.549+0000 7f2920ee6640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f291c0cf9f0 con 0x7f291c06e960 2024-09-12T03:22:32.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:32.549+0000 7f2920ee6640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f291c0d1a40 con 0x7f291c070d30 2024-09-12T03:22:32.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:32.549+0000 7f2920ee6640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f291c0cfc90 con 0x7f291c0d0b50 2024-09-12T03:22:32.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:32.549+0000 7f291a59c640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f291c070d30 0x7f291c06cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:32.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:32.549+0000 7f291a59c640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f291c070d30 0x7f291c06cc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54904/0 (socket says 172.21.15.27:54904) 2024-09-12T03:22:32.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:32.549+0000 7f291a59c640 1 -- 172.21.15.27:0/3517191833 learned_addr learned my addr 172.21.15.27:0/3517191833 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:22:32.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:32.549+0000 7f291ad9d640 1 --2- 172.21.15.27:0/3517191833 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f291c0d0b50 0x7f291c168300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:32.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:32.549+0000 7f291a59c640 1 -- 172.21.15.27:0/3517191833 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f291c0d0b50 msgr2=0x7f291c168300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:32.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:32.549+0000 7f291a59c640 1 --2- 172.21.15.27:0/3517191833 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f291c0d0b50 0x7f291c168300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:32.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:32.549+0000 7f291a59c640 1 -- 172.21.15.27:0/3517191833 >> v1:172.21.15.45:6789/0 conn(0x7f291c06e960 legacy=0x7f291c0d0440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:32.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:32.549+0000 7f291a59c640 1 -- 172.21.15.27:0/3517191833 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f291c0cff70 con 0x7f291c070d30 2024-09-12T03:22:32.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:32.549+0000 7f291ad9d640 1 --2- 172.21.15.27:0/3517191833 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f291c0d0b50 0x7f291c168300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:22:32.551 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:32.550+0000 7f291a59c640 1 --2- 172.21.15.27:0/3517191833 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f291c070d30 0x7f291c06cc40 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f291c073770 tx=0x7f290c02a0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f25a99fa6aff7cb server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:32.551 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:32.550+0000 7f291959a640 1 -- 172.21.15.27:0/3517191833 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f290c037070 con 0x7f291c070d30 2024-09-12T03:22:32.551 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:32.550+0000 7f291959a640 1 -- 172.21.15.27:0/3517191833 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f290c02adc0 con 0x7f291c070d30 2024-09-12T03:22:32.551 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:32.550+0000 7f291959a640 1 -- 172.21.15.27:0/3517191833 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f290c032660 con 0x7f291c070d30 2024-09-12T03:22:32.551 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:32.550+0000 7f2920ee6640 1 -- 172.21.15.27:0/3517191833 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f291c070d30 msgr2=0x7f291c06cc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:32.552 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:32.550+0000 7f2920ee6640 1 --2- 172.21.15.27:0/3517191833 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f291c070d30 0x7f291c06cc40 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f291c073770 tx=0x7f290c02a0b0 comp rx=0 tx=0).stop 2024-09-12T03:22:32.552 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:32.551+0000 7f2920ee6640 1 -- 172.21.15.27:0/3517191833 shutdown_connections 2024-09-12T03:22:32.552 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:32.551+0000 7f2920ee6640 1 --2- 172.21.15.27:0/3517191833 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f291c0d0b50 0x7f291c168300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:32.552 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:32.551+0000 7f2920ee6640 1 --2- 172.21.15.27:0/3517191833 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f291c070d30 0x7f291c06cc40 unknown :-1 s=CLOSED pgs=1300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:32.552 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:32.551+0000 7f2920ee6640 1 -- 172.21.15.27:0/3517191833 >> 172.21.15.27:0/3517191833 conn(0x7f291c0b7c10 msgr2=0x7f291c09ba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:32.552 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:32.551+0000 7f2920ee6640 1 -- 172.21.15.27:0/3517191833 shutdown_connections 2024-09-12T03:22:32.552 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:32.551+0000 7f2920ee6640 1 -- 172.21.15.27:0/3517191833 wait complete. 2024-09-12T03:22:32.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:32.552+0000 7f2920ee6640 1 Processor -- start 2024-09-12T03:22:32.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:32.552+0000 7f2920ee6640 1 -- start start 2024-09-12T03:22:32.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:32.553+0000 7f2920ee6640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f291c06e960 0x7f291c15c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:32.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:32.553+0000 7f2920ee6640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f291c070d30 0x7f291c0b4630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:32.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:32.553+0000 7f2920ee6640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f291c0d0b50 0x7f291c0b4b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:32.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:32.553+0000 7f2920ee6640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f291c0d31c0 con 0x7f291c0d0b50 2024-09-12T03:22:32.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:32.553+0000 7f2920ee6640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f291c06d180 con 0x7f291c070d30 2024-09-12T03:22:32.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:32.553+0000 7f2920ee6640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f291c0d3340 con 0x7f291c06e960 2024-09-12T03:22:32.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:32.553+0000 7f291ad9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f291c0d0b50 0x7f291c0b4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:32.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:32.553+0000 7f291ad9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f291c0d0b50 0x7f291c0b4b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54914/0 (socket says 172.21.15.27:54914) 2024-09-12T03:22:32.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:32.553+0000 7f291ad9d640 1 -- 172.21.15.27:0/1994772941 learned_addr learned my addr 172.21.15.27:0/1994772941 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:22:32.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:32.553+0000 7f291a59c640 1 --2- 172.21.15.27:0/1994772941 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f291c06e960 0x7f291c15c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:32.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:32.553+0000 7f2919d9b640 1 --2- 172.21.15.27:0/1994772941 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f291c070d30 0x7f291c0b4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:32.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:32.553+0000 7f291ad9d640 1 -- 172.21.15.27:0/1994772941 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f291c06e960 msgr2=0x7f291c15c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:32.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:32.553+0000 7f291ad9d640 1 --2- 172.21.15.27:0/1994772941 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f291c06e960 0x7f291c15c5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:32.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:32.553+0000 7f291ad9d640 1 -- 172.21.15.27:0/1994772941 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f291c070d30 msgr2=0x7f291c0b4630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:32.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:32.553+0000 7f291ad9d640 1 --2- 172.21.15.27:0/1994772941 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f291c070d30 0x7f291c0b4630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:32.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:32.553+0000 7f291ad9d640 1 -- 172.21.15.27:0/1994772941 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f291c0b2dc0 con 0x7f291c0d0b50 2024-09-12T03:22:32.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:32.553+0000 7f2919d9b640 1 --2- 172.21.15.27:0/1994772941 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f291c070d30 0x7f291c0b4630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:22:32.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:32.553+0000 7f291ad9d640 1 --2- 172.21.15.27:0/1994772941 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f291c0d0b50 0x7f291c0b4b70 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f291400cdb0 tx=0x7f291400a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:32.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:32.553+0000 7f2912ffd640 1 -- 172.21.15.27:0/1994772941 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2914003b90 con 0x7f291c0d0b50 2024-09-12T03:22:32.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:32.553+0000 7f2912ffd640 1 -- 172.21.15.27:0/1994772941 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2914003d50 con 0x7f291c0d0b50 2024-09-12T03:22:32.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:32.554+0000 7f2912ffd640 1 -- 172.21.15.27:0/1994772941 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2914004260 con 0x7f291c0d0b50 2024-09-12T03:22:32.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:32.554+0000 7f2920ee6640 1 -- 172.21.15.27:0/1994772941 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f291c0b30a0 con 0x7f291c0d0b50 2024-09-12T03:22:32.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:32.554+0000 7f2920ee6640 1 -- 172.21.15.27:0/1994772941 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f291c0b3660 con 0x7f291c0d0b50 2024-09-12T03:22:32.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:32.555+0000 7f2920ee6640 1 -- 172.21.15.27:0/1994772941 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f291c06cc40 con 0x7f291c0d0b50 2024-09-12T03:22:32.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:32.555+0000 7f2912ffd640 1 -- 172.21.15.27:0/1994772941 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 21) ==== 91008+0+0 (secure 0 0 0) 0x7f2914004400 con 0x7f291c0d0b50 2024-09-12T03:22:32.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:32.556+0000 7f2912ffd640 1 --2- 172.21.15.27:0/1994772941 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f28fc06c3d0 0x7f28fc06e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:32.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:32.556+0000 7f2912ffd640 1 -- 172.21.15.27:0/1994772941 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 5765+0+0 (secure 0 0 0) 0x7f291408c480 con 0x7f291c0d0b50 2024-09-12T03:22:32.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:32.556+0000 7f291a59c640 1 --2- 172.21.15.27:0/1994772941 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f28fc06c3d0 0x7f28fc06e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:32.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:32.556+0000 7f291a59c640 1 --2- 172.21.15.27:0/1994772941 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f28fc06c3d0 0x7f28fc06e890 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f290c034000 tx=0x7f290c02e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:32.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:32.560+0000 7f2912ffd640 1 -- 172.21.15.27:0/1994772941 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f2914014030 con 0x7f291c0d0b50 2024-09-12T03:22:32.769 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:32.769+0000 7f2920ee6640 1 -- 172.21.15.27:0/1994772941 --> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "non_existent_subvolume", "sub_name": "subvolume_1", "snap_name": "snapshot_1", "target_sub_name": "clone_1", "target": ["mon-mgr", ""]}) -- 0x7f291c0d1230 con 0x7f28fc06c3d0 2024-09-12T03:22:32.775 INFO:tasks.ceph.mgr.y.smithi027.stderr:2024-09-12T03:22:32.775+0000 7f325d8e3640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:22:32.775 INFO:tasks.ceph.mgr.y.smithi027.stderr:2024-09-12T03:22:32.775+0000 7f325d8e3640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:22:32.775 INFO:tasks.ceph.mgr.y.smithi027.stderr:2024-09-12T03:22:32.775+0000 7f325d8e3640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:22:32.775 INFO:tasks.ceph.mgr.y.smithi027.stderr:2024-09-12T03:22:32.775+0000 7f325d8e3640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:22:32.775 INFO:tasks.ceph.mgr.y.smithi027.stderr:2024-09-12T03:22:32.775+0000 7f325d8e3640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:22:32.779 INFO:tasks.ceph.mgr.y.smithi027.stderr:2024-09-12T03:22:32.779+0000 7f325d8e3640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-12T03:22:32.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:32.779+0000 7f2912ffd640 1 -- 172.21.15.27:0/1994772941 <== mgr.4113 v2:172.21.15.27:6836/1226920526 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7f291c0d1230 con 0x7f28fc06c3d0 2024-09-12T03:22:32.779 INFO:teuthology.orchestra.run.smithi027.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-12T03:22:32.785 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-12T03:22:32.785 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status non_existent_subvolume clone_1 2024-09-12T03:22:32.891 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:32.891+0000 7f130d82e640 1 Processor -- start 2024-09-12T03:22:32.892 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:32.892+0000 7f130d82e640 1 -- start start 2024-09-12T03:22:32.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:32.892+0000 7f130d82e640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f13080730e0 0x7f13080b6d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:32.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:32.892+0000 7f130d82e640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f13080d0380 0x7f13080b72d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:32.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:32.892+0000 7f130d82e640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f13080736a0 con 0x7f130815a1a0 2024-09-12T03:22:32.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:32.892+0000 7f130d82e640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f1308074220 con 0x7f13080d0380 2024-09-12T03:22:32.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:32.892+0000 7f130d82e640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f1308073980 con 0x7f13080730e0 2024-09-12T03:22:32.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:32.893+0000 7f13077fe640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f13080d0380 0x7f13080b72d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:32.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:32.893+0000 7f13077fe640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f13080d0380 0x7f13080b72d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54940/0 (socket says 172.21.15.27:54940) 2024-09-12T03:22:32.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:32.893+0000 7f13077fe640 1 -- 172.21.15.27:0/100814555 learned_addr learned my addr 172.21.15.27:0/100814555 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:22:32.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:32.893+0000 7f13067fc640 1 --2- 172.21.15.27:0/100814555 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f13080730e0 0x7f13080b6d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:32.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:32.893+0000 7f13077fe640 1 -- 172.21.15.27:0/100814555 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f13080730e0 msgr2=0x7f13080b6d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:32.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:32.893+0000 7f13077fe640 1 --2- 172.21.15.27:0/100814555 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f13080730e0 0x7f13080b6d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:32.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:32.893+0000 7f13077fe640 1 -- 172.21.15.27:0/100814555 >> v1:172.21.15.45:6789/0 conn(0x7f130815a1a0 legacy=0x7f130815d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:32.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:32.893+0000 7f13077fe640 1 -- 172.21.15.27:0/100814555 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1308073c60 con 0x7f13080d0380 2024-09-12T03:22:32.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:32.894+0000 7f13077fe640 1 --2- 172.21.15.27:0/100814555 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f13080d0380 0x7f13080b72d0 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f12f8009e50 tx=0x7f12f802a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94ac98254bc18f63 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:32.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:32.894+0000 7f1305ffb640 1 -- 172.21.15.27:0/100814555 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12f8036070 con 0x7f13080d0380 2024-09-12T03:22:32.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:32.894+0000 7f1305ffb640 1 -- 172.21.15.27:0/100814555 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f12f8031600 con 0x7f13080d0380 2024-09-12T03:22:32.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:32.894+0000 7f1305ffb640 1 -- 172.21.15.27:0/100814555 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12f80319d0 con 0x7f13080d0380 2024-09-12T03:22:32.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:32.894+0000 7f130d82e640 1 -- 172.21.15.27:0/100814555 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f13080d0380 msgr2=0x7f13080b72d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:32.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:32.894+0000 7f130d82e640 1 --2- 172.21.15.27:0/100814555 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f13080d0380 0x7f13080b72d0 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f12f8009e50 tx=0x7f12f802a680 comp rx=0 tx=0).stop 2024-09-12T03:22:32.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:32.894+0000 7f130d82e640 1 -- 172.21.15.27:0/100814555 shutdown_connections 2024-09-12T03:22:32.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:32.894+0000 7f130d82e640 1 --2- 172.21.15.27:0/100814555 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f13080d0380 0x7f13080b72d0 unknown :-1 s=CLOSED pgs=1304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:32.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:32.894+0000 7f130d82e640 1 --2- 172.21.15.27:0/100814555 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f13080730e0 0x7f13080b6d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:32.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:32.894+0000 7f130d82e640 1 -- 172.21.15.27:0/100814555 >> 172.21.15.27:0/100814555 conn(0x7f13080b4280 msgr2=0x7f13080b4690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:32.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:32.894+0000 7f130d82e640 1 -- 172.21.15.27:0/100814555 shutdown_connections 2024-09-12T03:22:32.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:32.894+0000 7f130d82e640 1 -- 172.21.15.27:0/100814555 wait complete. 2024-09-12T03:22:32.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:32.895+0000 7f130d82e640 1 Processor -- start 2024-09-12T03:22:32.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:32.896+0000 7f130d82e640 1 -- start start 2024-09-12T03:22:32.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:32.896+0000 7f130d82e640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f13080730e0 0x7f130814c250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:32.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:32.896+0000 7f130d82e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f13080d0380 0x7f13080b5910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:32.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:32.896+0000 7f130d82e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f130815a1a0 0x7f130815ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:32.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:32.896+0000 7f130d82e640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f1308070d90 con 0x7f130815a1a0 2024-09-12T03:22:32.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:32.896+0000 7f130d82e640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f1308070c10 con 0x7f13080d0380 2024-09-12T03:22:32.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:32.896+0000 7f130d82e640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f1308070f10 con 0x7f13080730e0 2024-09-12T03:22:32.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:32.896+0000 7f13077fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f130815a1a0 0x7f130815ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:32.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:32.896+0000 7f13077fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f130815a1a0 0x7f130815ea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54952/0 (socket says 172.21.15.27:54952) 2024-09-12T03:22:32.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:32.896+0000 7f13077fe640 1 -- 172.21.15.27:0/1196403041 learned_addr learned my addr 172.21.15.27:0/1196403041 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:22:32.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:32.897+0000 7f1306ffd640 1 --2- 172.21.15.27:0/1196403041 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f13080730e0 0x7f130814c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:32.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:32.897+0000 7f13067fc640 1 --2- 172.21.15.27:0/1196403041 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f13080d0380 0x7f13080b5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:32.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:32.897+0000 7f13077fe640 1 -- 172.21.15.27:0/1196403041 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f13080730e0 msgr2=0x7f130814c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:32.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:32.897+0000 7f13077fe640 1 --2- 172.21.15.27:0/1196403041 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f13080730e0 0x7f130814c250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:32.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:32.897+0000 7f13077fe640 1 -- 172.21.15.27:0/1196403041 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f13080d0380 msgr2=0x7f13080b5910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:32.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:32.897+0000 7f13077fe640 1 --2- 172.21.15.27:0/1196403041 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f13080d0380 0x7f13080b5910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:32.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:32.897+0000 7f13077fe640 1 -- 172.21.15.27:0/1196403041 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f130806cce0 con 0x7f130815a1a0 2024-09-12T03:22:32.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:32.897+0000 7f13077fe640 1 --2- 172.21.15.27:0/1196403041 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f130815a1a0 0x7f130815ea60 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7f12f8034040 tx=0x7f12f8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:32.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:32.897+0000 7f12fffff640 1 -- 172.21.15.27:0/1196403041 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12f8036050 con 0x7f130815a1a0 2024-09-12T03:22:32.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:32.897+0000 7f12fffff640 1 -- 172.21.15.27:0/1196403041 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f12f8042070 con 0x7f130815a1a0 2024-09-12T03:22:32.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:32.897+0000 7f130d82e640 1 -- 172.21.15.27:0/1196403041 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13080b5e50 con 0x7f130815a1a0 2024-09-12T03:22:32.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:32.897+0000 7f12fffff640 1 -- 172.21.15.27:0/1196403041 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12f8009510 con 0x7f130815a1a0 2024-09-12T03:22:32.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:32.897+0000 7f130d82e640 1 -- 172.21.15.27:0/1196403041 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f130806d280 con 0x7f130815a1a0 2024-09-12T03:22:32.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:32.898+0000 7f130d82e640 1 -- 172.21.15.27:0/1196403041 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f130815d410 con 0x7f130815a1a0 2024-09-12T03:22:32.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:32.899+0000 7f12fffff640 1 -- 172.21.15.27:0/1196403041 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 21) ==== 91008+0+0 (secure 0 0 0) 0x7f12f8007bf0 con 0x7f130815a1a0 2024-09-12T03:22:32.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:32.899+0000 7f12fffff640 1 --2- 172.21.15.27:0/1196403041 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f12e806c3d0 0x7f12e806e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:32.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:32.899+0000 7f12fffff640 1 -- 172.21.15.27:0/1196403041 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 5765+0+0 (secure 0 0 0) 0x7f12f80ac660 con 0x7f130815a1a0 2024-09-12T03:22:32.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:32.899+0000 7f1306ffd640 1 --2- 172.21.15.27:0/1196403041 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f12e806c3d0 0x7f12e806e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:32.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:32.900+0000 7f1306ffd640 1 --2- 172.21.15.27:0/1196403041 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f12e806c3d0 0x7f12e806e890 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f1300007fd0 tx=0x7f1300009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:32.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:32.904+0000 7f12fffff640 1 -- 172.21.15.27:0/1196403041 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f12f8045360 con 0x7f130815a1a0 2024-09-12T03:22:33.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.114+0000 7f130d82e640 1 -- 172.21.15.27:0/1196403041 --> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "non_existent_subvolume", "clone_name": "clone_1", "target": ["mon-mgr", ""]}) -- 0x7f13080ce820 con 0x7f12e806c3d0 2024-09-12T03:22:33.115 INFO:tasks.ceph.mgr.y.smithi027.stderr:2024-09-12T03:22:33.115+0000 7f325d8e3640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-12T03:22:33.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.115+0000 7f12fffff640 1 -- 172.21.15.27:0/1196403041 <== mgr.4113 v2:172.21.15.27:6836/1226920526 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7f13080ce820 con 0x7f12e806c3d0 2024-09-12T03:22:33.116 INFO:teuthology.orchestra.run.smithi027.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-12T03:22:33.121 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-12T03:22:33.121 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create non_existent_subvolume group_1 2024-09-12T03:22:33.227 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.227+0000 7fa25119f640 1 Processor -- start 2024-09-12T03:22:33.228 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.228+0000 7fa25119f640 1 -- start start 2024-09-12T03:22:33.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.228+0000 7fa25119f640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa24c0cee10 0x7fa24c0cf5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:33.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.228+0000 7fa25119f640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa24c0d3480 con 0x7fa24c0d0ef0 2024-09-12T03:22:33.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.228+0000 7fa25119f640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa24c0d3760 con 0x7fa24c15eb80 2024-09-12T03:22:33.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.228+0000 7fa25119f640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fa24c0d3a40 con 0x7fa24c0cee10 2024-09-12T03:22:33.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.228+0000 7fa24bfff640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fa24c0d0ef0 0x7fa24c0d12d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:38600/0 (socket says 172.21.15.27:38600) 2024-09-12T03:22:33.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.229+0000 7fa24bfff640 1 -- 172.21.15.27:0/3681961014 learned_addr learned my addr 172.21.15.27:0/3681961014 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:22:33.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.229+0000 7fa25099e640 1 --2- 172.21.15.27:0/3681961014 >> v2:172.21.15.45:3300/0 conn(0x7fa24c0cee10 0x7fa24c0cf5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:33.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.229+0000 7fa24affd640 1 -- 172.21.15.27:0/3681961014 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2345642702 0 0) 0x7fa24c0d3480 con 0x7fa24c0d0ef0 2024-09-12T03:22:33.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.229+0000 7fa24affd640 1 -- 172.21.15.27:0/3681961014 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa234003520 con 0x7fa24c0d0ef0 2024-09-12T03:22:33.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.229+0000 7fa24affd640 1 -- 172.21.15.27:0/3681961014 <== mon.1 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1138663951 0 0) 0x7fa24c0d3760 con 0x7fa24c15eb80 2024-09-12T03:22:33.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.229+0000 7fa24affd640 1 -- 172.21.15.27:0/3681961014 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa24c0d3480 con 0x7fa24c15eb80 2024-09-12T03:22:33.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.229+0000 7fa24affd640 1 -- 172.21.15.27:0/3681961014 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1500202962 0 0) 0x7fa234003520 con 0x7fa24c0d0ef0 2024-09-12T03:22:33.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.229+0000 7fa24affd640 1 -- 172.21.15.27:0/3681961014 >> v2:172.21.15.45:3300/0 conn(0x7fa24c0cee10 msgr2=0x7fa24c0cf5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:33.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.229+0000 7fa24affd640 1 --2- 172.21.15.27:0/3681961014 >> v2:172.21.15.45:3300/0 conn(0x7fa24c0cee10 0x7fa24c0cf5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:33.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.229+0000 7fa24affd640 1 -- 172.21.15.27:0/3681961014 >> v1:172.21.15.45:6789/0 conn(0x7fa24c15eb80 legacy=0x7fa24c0b77e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:33.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.229+0000 7fa24affd640 1 -- 172.21.15.27:0/3681961014 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa24c0d3d20 con 0x7fa24c0d0ef0 2024-09-12T03:22:33.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.229+0000 7fa24affd640 1 -- 172.21.15.27:0/3681961014 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 847755213 0 0) 0x7fa23c0015f0 con 0x7fa24c0d0ef0 2024-09-12T03:22:33.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.229+0000 7fa24affd640 1 -- 172.21.15.27:0/3681961014 >> v1:172.21.15.27:6789/0 conn(0x7fa24c0d0ef0 legacy=0x7fa24c0d12d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:33.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.229+0000 7fa24affd640 1 --2- 172.21.15.27:0/3681961014 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa234003c00 0x7fa234004000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:33.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.229+0000 7fa24affd640 1 --2- 172.21.15.27:0/3681961014 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa234004800 0x7fa234004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:33.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.229+0000 7fa24affd640 1 --2- 172.21.15.27:0/3681961014 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa2340051c0 0x7fa23400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:33.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.229+0000 7fa24affd640 1 -- 172.21.15.27:0/3681961014 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fa24c0d3a40 con 0x7fa2340051c0 2024-09-12T03:22:33.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.229+0000 7fa24affd640 1 -- 172.21.15.27:0/3681961014 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa23400c160 con 0x7fa234003c00 2024-09-12T03:22:33.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.229+0000 7fa24affd640 1 -- 172.21.15.27:0/3681961014 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fa23400c360 con 0x7fa234004800 2024-09-12T03:22:33.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.229+0000 7fa25099e640 1 --2- 172.21.15.27:0/3681961014 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa2340051c0 0x7fa23400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:33.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.229+0000 7fa24bfff640 1 --2- 172.21.15.27:0/3681961014 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa234003c00 0x7fa234004000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:33.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.229+0000 7fa24b7fe640 1 --2- 172.21.15.27:0/3681961014 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa234004800 0x7fa234004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:33.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.230+0000 7fa25099e640 1 -- 172.21.15.27:0/3681961014 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa234004800 msgr2=0x7fa234004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:33.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.230+0000 7fa25099e640 1 --2- 172.21.15.27:0/3681961014 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa234004800 0x7fa234004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:33.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.230+0000 7fa25099e640 1 -- 172.21.15.27:0/3681961014 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa234003c00 msgr2=0x7fa234004000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:33.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.230+0000 7fa25099e640 1 --2- 172.21.15.27:0/3681961014 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa234003c00 0x7fa234004000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:33.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.230+0000 7fa25099e640 1 -- 172.21.15.27:0/3681961014 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa234003520 con 0x7fa2340051c0 2024-09-12T03:22:33.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.230+0000 7fa25099e640 1 -- 172.21.15.27:0/3681961014 reap_dead start 2024-09-12T03:22:33.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.230+0000 7fa25099e640 1 --2- 172.21.15.27:0/3681961014 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa2340051c0 0x7fa23400b9c0 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7fa244009fc0 tx=0x7fa234005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c75ba1a9043992d5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:33.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.230+0000 7fa24affd640 1 -- 172.21.15.27:0/3681961014 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa244035020 con 0x7fa2340051c0 2024-09-12T03:22:33.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.230+0000 7fa24affd640 1 -- 172.21.15.27:0/3681961014 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa244003bd0 con 0x7fa2340051c0 2024-09-12T03:22:33.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.230+0000 7fa24affd640 1 -- 172.21.15.27:0/3681961014 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa244009c10 con 0x7fa2340051c0 2024-09-12T03:22:33.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.231+0000 7fa25119f640 1 -- 172.21.15.27:0/3681961014 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa2340051c0 msgr2=0x7fa23400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:33.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.231+0000 7fa25119f640 1 --2- 172.21.15.27:0/3681961014 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa2340051c0 0x7fa23400b9c0 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7fa244009fc0 tx=0x7fa234005830 comp rx=0 tx=0).stop 2024-09-12T03:22:33.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.231+0000 7fa25119f640 1 -- 172.21.15.27:0/3681961014 shutdown_connections 2024-09-12T03:22:33.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.231+0000 7fa25119f640 1 --2- 172.21.15.27:0/3681961014 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa2340051c0 0x7fa23400b9c0 unknown :-1 s=CLOSED pgs=1307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:33.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.231+0000 7fa25119f640 1 -- 172.21.15.27:0/3681961014 >> 172.21.15.27:0/3681961014 conn(0x7fa24c06c1e0 msgr2=0x7fa24c06c5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:33.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.231+0000 7fa25119f640 1 -- 172.21.15.27:0/3681961014 shutdown_connections 2024-09-12T03:22:33.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.231+0000 7fa25119f640 1 -- 172.21.15.27:0/3681961014 wait complete. 2024-09-12T03:22:33.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.232+0000 7fa25119f640 1 Processor -- start 2024-09-12T03:22:33.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.232+0000 7fa25119f640 1 -- start start 2024-09-12T03:22:33.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.233+0000 7fa25119f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa2340051c0 0x7fa24c1f4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:33.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.233+0000 7fa25119f640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa24c158f30 0x7fa24c0cd690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:33.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.233+0000 7fa25119f640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa24c0cc970 0x7fa24c155380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:33.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.233+0000 7fa25119f640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fa24c0cf5f0 con 0x7fa24c158f30 2024-09-12T03:22:33.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.233+0000 7fa25119f640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa24c0cfd40 con 0x7fa2340051c0 2024-09-12T03:22:33.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.233+0000 7fa25119f640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fa24c0cf770 con 0x7fa24c0cc970 2024-09-12T03:22:33.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.233+0000 7fa24b7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa24c158f30 0x7fa24c0cd690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:33.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.233+0000 7fa24b7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa24c158f30 0x7fa24c0cd690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54982/0 (socket says 172.21.15.27:54982) 2024-09-12T03:22:33.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.233+0000 7fa24b7fe640 1 -- 172.21.15.27:0/2059311019 learned_addr learned my addr 172.21.15.27:0/2059311019 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:22:33.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.233+0000 7fa24bfff640 1 --2- 172.21.15.27:0/2059311019 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa2340051c0 0x7fa24c1f4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:33.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.233+0000 7fa25099e640 1 --2- 172.21.15.27:0/2059311019 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa24c0cc970 0x7fa24c155380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:33.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.234+0000 7fa24b7fe640 1 -- 172.21.15.27:0/2059311019 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa24c0cc970 msgr2=0x7fa24c155380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:33.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.234+0000 7fa24b7fe640 1 --2- 172.21.15.27:0/2059311019 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa24c0cc970 0x7fa24c155380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:33.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.234+0000 7fa24b7fe640 1 -- 172.21.15.27:0/2059311019 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa2340051c0 msgr2=0x7fa24c1f4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:33.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.234+0000 7fa24b7fe640 1 --2- 172.21.15.27:0/2059311019 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa2340051c0 0x7fa24c1f4cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:33.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.234+0000 7fa24b7fe640 1 -- 172.21.15.27:0/2059311019 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa24c1f5230 con 0x7fa24c158f30 2024-09-12T03:22:33.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.234+0000 7fa24b7fe640 1 --2- 172.21.15.27:0/2059311019 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa24c158f30 0x7fa24c0cd690 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7fa24000f700 tx=0x7fa24000fb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:33.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.234+0000 7fa248ff9640 1 -- 172.21.15.27:0/2059311019 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2400052e0 con 0x7fa24c158f30 2024-09-12T03:22:33.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.234+0000 7fa248ff9640 1 -- 172.21.15.27:0/2059311019 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa240018070 con 0x7fa24c158f30 2024-09-12T03:22:33.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.234+0000 7fa25119f640 1 -- 172.21.15.27:0/2059311019 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa24c0cb8d0 con 0x7fa24c158f30 2024-09-12T03:22:33.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.234+0000 7fa248ff9640 1 -- 172.21.15.27:0/2059311019 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa240003830 con 0x7fa24c158f30 2024-09-12T03:22:33.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.234+0000 7fa25119f640 1 -- 172.21.15.27:0/2059311019 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa24c0cbe90 con 0x7fa24c158f30 2024-09-12T03:22:33.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.235+0000 7fa25119f640 1 -- 172.21.15.27:0/2059311019 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa24c0cb660 con 0x7fa24c158f30 2024-09-12T03:22:33.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.236+0000 7fa248ff9640 1 -- 172.21.15.27:0/2059311019 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 21) ==== 91008+0+0 (secure 0 0 0) 0x7fa2400039d0 con 0x7fa24c158f30 2024-09-12T03:22:33.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.236+0000 7fa248ff9640 1 --2- 172.21.15.27:0/2059311019 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fa23806c400 0x7fa23806e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:33.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.236+0000 7fa248ff9640 1 -- 172.21.15.27:0/2059311019 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 5765+0+0 (secure 0 0 0) 0x7fa24008b600 con 0x7fa24c158f30 2024-09-12T03:22:33.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.236+0000 7fa24bfff640 1 --2- 172.21.15.27:0/2059311019 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fa23806c400 0x7fa23806e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:33.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.237+0000 7fa24bfff640 1 --2- 172.21.15.27:0/2059311019 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fa23806c400 0x7fa23806e8c0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fa23c000c00 tx=0x7fa23c001cf0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:33.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.240+0000 7fa248ff9640 1 -- 172.21.15.27:0/2059311019 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fa24c1f5230 con 0x7fa24c158f30 2024-09-12T03:22:33.439 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.439+0000 7fa25119f640 1 -- 172.21.15.27:0/2059311019 --> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "non_existent_subvolume", "group_name": "group_1", "target": ["mon-mgr", ""]}) -- 0x7fa24c0cfa70 con 0x7fa23806c400 2024-09-12T03:22:33.440 INFO:tasks.ceph.mgr.y.smithi027.stderr:2024-09-12T03:22:33.440+0000 7f325d8e3640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-12T03:22:33.440 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.440+0000 7fa248ff9640 1 -- 172.21.15.27:0/2059311019 <== mgr.4113 v2:172.21.15.27:6836/1226920526 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7fa24c0cfa70 con 0x7fa23806c400 2024-09-12T03:22:33.441 INFO:teuthology.orchestra.run.smithi027.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-12T03:22:33.445 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-12T03:22:33.446 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup rm non_existent_subvolume group_1 2024-09-12T03:22:33.551 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.551+0000 7f348ccd0640 1 Processor -- start 2024-09-12T03:22:33.551 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.551+0000 7f348ccd0640 1 -- start start 2024-09-12T03:22:33.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.552+0000 7f348ccd0640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f348815a270 0x7f348806cff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:33.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.552+0000 7f348ccd0640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f348806d530 0x7f34880cf410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:33.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.552+0000 7f348ccd0640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f34880ce650 con 0x7f348806c650 2024-09-12T03:22:33.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.552+0000 7f348ccd0640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f34880cef80 con 0x7f348815a270 2024-09-12T03:22:33.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.552+0000 7f348ccd0640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f34880ce930 con 0x7f348806d530 2024-09-12T03:22:33.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.552+0000 7f348659c640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f348815a270 0x7f348806cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:33.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.552+0000 7f3486d9d640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f348806c650 0x7f34880726a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:38610/0 (socket says 172.21.15.27:38610) 2024-09-12T03:22:33.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.552+0000 7f348659c640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f348815a270 0x7f348806cff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54994/0 (socket says 172.21.15.27:54994) 2024-09-12T03:22:33.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.552+0000 7f3486d9d640 1 -- 172.21.15.27:0/816762653 learned_addr learned my addr 172.21.15.27:0/816762653 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:22:33.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.553+0000 7f3485d9b640 1 --2- 172.21.15.27:0/816762653 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f348806d530 0x7f34880cf410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:33.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.553+0000 7f348659c640 1 -- 172.21.15.27:0/816762653 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f348806d530 msgr2=0x7f34880cf410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:33.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.553+0000 7f348659c640 1 --2- 172.21.15.27:0/816762653 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f348806d530 0x7f34880cf410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:33.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.553+0000 7f348659c640 1 -- 172.21.15.27:0/816762653 >> v1:172.21.15.27:6789/0 conn(0x7f348806c650 legacy=0x7f34880726a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:33.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.553+0000 7f348559a640 1 -- 172.21.15.27:0/816762653 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1791448840 0 0) 0x7f34880ce650 con 0x7f348806c650 2024-09-12T03:22:33.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.553+0000 7f348659c640 1 -- 172.21.15.27:0/816762653 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34880cec10 con 0x7f348815a270 2024-09-12T03:22:33.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.553+0000 7f348659c640 1 --2- 172.21.15.27:0/816762653 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f348815a270 0x7f348806cff0 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7f34800097a0 tx=0x7f348002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9010d83c43be33d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:33.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.554+0000 7f348559a640 1 -- 172.21.15.27:0/816762653 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3480037070 con 0x7f348815a270 2024-09-12T03:22:33.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.554+0000 7f348559a640 1 -- 172.21.15.27:0/816762653 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3480032440 con 0x7f348815a270 2024-09-12T03:22:33.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.554+0000 7f348559a640 1 -- 172.21.15.27:0/816762653 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3480032810 con 0x7f348815a270 2024-09-12T03:22:33.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.554+0000 7f348ccd0640 1 -- 172.21.15.27:0/816762653 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f348815a270 msgr2=0x7f348806cff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:33.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.554+0000 7f348ccd0640 1 --2- 172.21.15.27:0/816762653 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f348815a270 0x7f348806cff0 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7f34800097a0 tx=0x7f348002a180 comp rx=0 tx=0).stop 2024-09-12T03:22:33.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.554+0000 7f348ccd0640 1 -- 172.21.15.27:0/816762653 shutdown_connections 2024-09-12T03:22:33.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.554+0000 7f348ccd0640 1 --2- 172.21.15.27:0/816762653 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f348806d530 0x7f34880cf410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:33.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.554+0000 7f348ccd0640 1 --2- 172.21.15.27:0/816762653 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f348815a270 0x7f348806cff0 unknown :-1 s=CLOSED pgs=1310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:33.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.554+0000 7f348ccd0640 1 -- 172.21.15.27:0/816762653 >> 172.21.15.27:0/816762653 conn(0x7f348806bb30 msgr2=0x7f348806bf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:33.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.554+0000 7f348ccd0640 1 -- 172.21.15.27:0/816762653 shutdown_connections 2024-09-12T03:22:33.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.554+0000 7f348ccd0640 1 -- 172.21.15.27:0/816762653 wait complete. 2024-09-12T03:22:33.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.556+0000 7f348ccd0640 1 Processor -- start 2024-09-12T03:22:33.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.556+0000 7f348ccd0640 1 -- start start 2024-09-12T03:22:33.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.556+0000 7f348ccd0640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f348806c650 0x7f3488156860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:33.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.557+0000 7f348ccd0640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f348806d530 0x7f34880ad970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:33.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.557+0000 7f348ccd0640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f348815a270 0x7f34880adeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:33.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.557+0000 7f348ccd0640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f3488072f30 con 0x7f348815a270 2024-09-12T03:22:33.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.557+0000 7f348ccd0640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f3488072db0 con 0x7f348806d530 2024-09-12T03:22:33.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.557+0000 7f348ccd0640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f34880730b0 con 0x7f348806c650 2024-09-12T03:22:33.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.557+0000 7f3486d9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f348815a270 0x7f34880adeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:33.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.557+0000 7f3486d9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f348815a270 0x7f34880adeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55010/0 (socket says 172.21.15.27:55010) 2024-09-12T03:22:33.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.557+0000 7f3486d9d640 1 -- 172.21.15.27:0/1715642972 learned_addr learned my addr 172.21.15.27:0/1715642972 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:22:33.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.557+0000 7f348659c640 1 --2- 172.21.15.27:0/1715642972 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f348806c650 0x7f3488156860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:33.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.557+0000 7f3485d9b640 1 --2- 172.21.15.27:0/1715642972 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f348806d530 0x7f34880ad970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:33.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.557+0000 7f3486d9d640 1 -- 172.21.15.27:0/1715642972 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f348806c650 msgr2=0x7f3488156860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:33.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.557+0000 7f3486d9d640 1 --2- 172.21.15.27:0/1715642972 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f348806c650 0x7f3488156860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:33.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.557+0000 7f3486d9d640 1 -- 172.21.15.27:0/1715642972 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f348806d530 msgr2=0x7f34880ad970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:33.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.557+0000 7f3486d9d640 1 --2- 172.21.15.27:0/1715642972 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f348806d530 0x7f34880ad970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:33.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.557+0000 7f3486d9d640 1 -- 172.21.15.27:0/1715642972 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34880abc00 con 0x7f348815a270 2024-09-12T03:22:33.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.557+0000 7f3486d9d640 1 --2- 172.21.15.27:0/1715642972 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f348815a270 0x7f34880adeb0 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7f347800c6d0 tx=0x7f347800cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:33.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.557+0000 7f347effd640 1 -- 172.21.15.27:0/1715642972 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3478018070 con 0x7f348815a270 2024-09-12T03:22:33.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.557+0000 7f347effd640 1 -- 172.21.15.27:0/1715642972 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3478004d60 con 0x7f348815a270 2024-09-12T03:22:33.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.557+0000 7f347effd640 1 -- 172.21.15.27:0/1715642972 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f347800a880 con 0x7f348815a270 2024-09-12T03:22:33.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.557+0000 7f348ccd0640 1 -- 172.21.15.27:0/1715642972 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34880abee0 con 0x7f348815a270 2024-09-12T03:22:33.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.558+0000 7f348ccd0640 1 -- 172.21.15.27:0/1715642972 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34880ac4a0 con 0x7f348815a270 2024-09-12T03:22:33.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.558+0000 7f348ccd0640 1 -- 172.21.15.27:0/1715642972 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f348815d560 con 0x7f348815a270 2024-09-12T03:22:33.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.559+0000 7f347effd640 1 -- 172.21.15.27:0/1715642972 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 21) ==== 91008+0+0 (secure 0 0 0) 0x7f347800ac30 con 0x7f348815a270 2024-09-12T03:22:33.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.559+0000 7f347effd640 1 --2- 172.21.15.27:0/1715642972 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f346806c3d0 0x7f346806e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:33.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.560+0000 7f347effd640 1 -- 172.21.15.27:0/1715642972 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 5765+0+0 (secure 0 0 0) 0x7f347808b090 con 0x7f348815a270 2024-09-12T03:22:33.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.560+0000 7f348659c640 1 --2- 172.21.15.27:0/1715642972 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f346806c3d0 0x7f346806e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:33.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.560+0000 7f348659c640 1 --2- 172.21.15.27:0/1715642972 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f346806c3d0 0x7f346806e890 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f3480034000 tx=0x7f348002e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:33.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.564+0000 7f347effd640 1 -- 172.21.15.27:0/1715642972 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f3478059190 con 0x7f348815a270 2024-09-12T03:22:33.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.762+0000 7f348ccd0640 1 -- 172.21.15.27:0/1715642972 --> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup rm", "vol_name": "non_existent_subvolume", "group_name": "group_1", "target": ["mon-mgr", ""]}) -- 0x7f3488063320 con 0x7f346806c3d0 2024-09-12T03:22:33.763 INFO:tasks.ceph.mgr.y.smithi027.stderr:2024-09-12T03:22:33.763+0000 7f325d8e3640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-12T03:22:33.764 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.764+0000 7f347effd640 1 -- 172.21.15.27:0/1715642972 <== mgr.4113 v2:172.21.15.27:6836/1226920526 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7f3480038040 con 0x7f346806c3d0 2024-09-12T03:22:33.764 INFO:teuthology.orchestra.run.smithi027.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-12T03:22:33.769 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-12T03:22:33.769 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup getpath non_existent_subvolume group_1 2024-09-12T03:22:33.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.875+0000 7fd7b8eba640 1 Processor -- start 2024-09-12T03:22:33.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.875+0000 7fd7b8eba640 1 -- start start 2024-09-12T03:22:33.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.876+0000 7fd7b8eba640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fd7b415ebb0 0x7fd7b4072d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:33.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.876+0000 7fd7b8eba640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd7b40aedd0 con 0x7fd7b40ae8d0 2024-09-12T03:22:33.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.876+0000 7fd7b8eba640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd7b40af0b0 con 0x7fd7b406e550 2024-09-12T03:22:33.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.876+0000 7fd7b8eba640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fd7b40af390 con 0x7fd7b415ebb0 2024-09-12T03:22:33.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.876+0000 7fd7b1d9b640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fd7b40ae8d0 0x7fd7b4072610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:38612/0 (socket says 172.21.15.27:38612) 2024-09-12T03:22:33.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.876+0000 7fd7b1d9b640 1 -- 172.21.15.27:0/3149323254 learned_addr learned my addr 172.21.15.27:0/3149323254 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:22:33.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.876+0000 7fd7b2d9d640 1 --2- 172.21.15.27:0/3149323254 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fd7b415ebb0 0x7fd7b4072d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:33.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.876+0000 7fd7b159a640 1 -- 172.21.15.27:0/3149323254 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1117471187 0 0) 0x7fd7b40aedd0 con 0x7fd7b40ae8d0 2024-09-12T03:22:33.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.876+0000 7fd7b159a640 1 -- 172.21.15.27:0/3149323254 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd7a0003520 con 0x7fd7b40ae8d0 2024-09-12T03:22:33.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.877+0000 7fd7b159a640 1 -- 172.21.15.27:0/3149323254 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2301824258 0 0) 0x7fd7a0003520 con 0x7fd7b40ae8d0 2024-09-12T03:22:33.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.877+0000 7fd7b159a640 1 -- 172.21.15.27:0/3149323254 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fd7b415ebb0 msgr2=0x7fd7b4072d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:33.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.877+0000 7fd7b159a640 1 --2- 172.21.15.27:0/3149323254 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fd7b415ebb0 0x7fd7b4072d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:33.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.877+0000 7fd7b159a640 1 -- 172.21.15.27:0/3149323254 >> v1:172.21.15.45:6789/0 conn(0x7fd7b406e550 legacy=0x7fd7b406e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:33.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.877+0000 7fd7b159a640 1 -- 172.21.15.27:0/3149323254 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7b40af670 con 0x7fd7b40ae8d0 2024-09-12T03:22:33.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.877+0000 7fd7b159a640 1 -- 172.21.15.27:0/3149323254 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 847755213 0 0) 0x7fd79c001670 con 0x7fd7b40ae8d0 2024-09-12T03:22:33.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.877+0000 7fd7b159a640 1 -- 172.21.15.27:0/3149323254 >> v1:172.21.15.27:6789/0 conn(0x7fd7b40ae8d0 legacy=0x7fd7b4072610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:33.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.877+0000 7fd7b159a640 1 --2- 172.21.15.27:0/3149323254 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fd7b415ebb0 0x7fd7a00039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:33.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.877+0000 7fd7b159a640 1 --2- 172.21.15.27:0/3149323254 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd7a0003f60 0x7fd7a0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:33.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.877+0000 7fd7b159a640 1 --2- 172.21.15.27:0/3149323254 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd7a0004d40 0x7fd7a00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:33.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.877+0000 7fd7b159a640 1 -- 172.21.15.27:0/3149323254 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fd7b40af390 con 0x7fd7a0003f60 2024-09-12T03:22:33.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.877+0000 7fd7b159a640 1 -- 172.21.15.27:0/3149323254 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fd7a000bb30 con 0x7fd7a0004d40 2024-09-12T03:22:33.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.877+0000 7fd7b159a640 1 -- 172.21.15.27:0/3149323254 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fd7a000bd30 con 0x7fd7b415ebb0 2024-09-12T03:22:33.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.877+0000 7fd7b1d9b640 1 --2- 172.21.15.27:0/3149323254 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd7a0003f60 0x7fd7a0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:33.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.877+0000 7fd7b259c640 1 --2- 172.21.15.27:0/3149323254 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fd7b415ebb0 0x7fd7a00039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:33.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.877+0000 7fd7b2d9d640 1 --2- 172.21.15.27:0/3149323254 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd7a0004d40 0x7fd7a00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:33.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.877+0000 7fd7b1d9b640 1 -- 172.21.15.27:0/3149323254 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fd7b415ebb0 msgr2=0x7fd7a00039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:33.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.877+0000 7fd7b1d9b640 1 --2- 172.21.15.27:0/3149323254 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fd7b415ebb0 0x7fd7a00039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:33.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.877+0000 7fd7b1d9b640 1 -- 172.21.15.27:0/3149323254 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd7a0004d40 msgr2=0x7fd7a00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:33.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.877+0000 7fd7b1d9b640 1 --2- 172.21.15.27:0/3149323254 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd7a0004d40 0x7fd7a00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:33.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.877+0000 7fd7b1d9b640 1 -- 172.21.15.27:0/3149323254 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7a0003520 con 0x7fd7a0003f60 2024-09-12T03:22:33.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.877+0000 7fd7b2d9d640 1 --2- 172.21.15.27:0/3149323254 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd7a0004d40 0x7fd7a00051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:22:33.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.878+0000 7fd7b1d9b640 1 --2- 172.21.15.27:0/3149323254 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd7a0003f60 0x7fd7a0004800 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7fd79c00c290 tx=0x7fd7a0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e20af83ef44af4e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:33.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.878+0000 7fd7b159a640 1 -- 172.21.15.27:0/3149323254 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd79c007d30 con 0x7fd7a0003f60 2024-09-12T03:22:33.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.878+0000 7fd7b159a640 1 -- 172.21.15.27:0/3149323254 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd79c0058d0 con 0x7fd7a0003f60 2024-09-12T03:22:33.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.878+0000 7fd7b159a640 1 -- 172.21.15.27:0/3149323254 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd79c005ce0 con 0x7fd7a0003f60 2024-09-12T03:22:33.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.879+0000 7fd7b8eba640 1 -- 172.21.15.27:0/3149323254 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd7a0003f60 msgr2=0x7fd7a0004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:33.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.879+0000 7fd7b8eba640 1 --2- 172.21.15.27:0/3149323254 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd7a0003f60 0x7fd7a0004800 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7fd79c00c290 tx=0x7fd7a0005830 comp rx=0 tx=0).stop 2024-09-12T03:22:33.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.879+0000 7fd7b2d9d640 1 -- 172.21.15.27:0/3149323254 reap_dead start 2024-09-12T03:22:33.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.879+0000 7fd7b8eba640 1 -- 172.21.15.27:0/3149323254 shutdown_connections 2024-09-12T03:22:33.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.879+0000 7fd7b8eba640 1 -- 172.21.15.27:0/3149323254 >> 172.21.15.27:0/3149323254 conn(0x7fd7b40b26c0 msgr2=0x7fd7b40b2aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:33.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.879+0000 7fd7b8eba640 1 -- 172.21.15.27:0/3149323254 shutdown_connections 2024-09-12T03:22:33.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.879+0000 7fd7b8eba640 1 -- 172.21.15.27:0/3149323254 wait complete. 2024-09-12T03:22:33.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.880+0000 7fd7b8eba640 1 Processor -- start 2024-09-12T03:22:33.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.880+0000 7fd7b8eba640 1 -- start start 2024-09-12T03:22:33.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.881+0000 7fd7b8eba640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fd7a0003f60 0x7fd7b4145240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:33.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.881+0000 7fd7b8eba640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd7b4158390 0x7fd7b4158810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:33.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.881+0000 7fd7b8eba640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd7b4145b70 0x7fd7b4146000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:33.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.881+0000 7fd7b8eba640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fd7b40b3480 con 0x7fd7b4145b70 2024-09-12T03:22:33.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.881+0000 7fd7b8eba640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fd7b40b3300 con 0x7fd7b4158390 2024-09-12T03:22:33.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.881+0000 7fd7b8eba640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fd7b40b39c0 con 0x7fd7a0003f60 2024-09-12T03:22:33.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.881+0000 7fd7b2d9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd7b4145b70 0x7fd7b4146000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:33.886 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.881+0000 7fd7b2d9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd7b4145b70 0x7fd7b4146000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55034/0 (socket says 172.21.15.27:55034) 2024-09-12T03:22:33.886 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.881+0000 7fd7b2d9d640 1 -- 172.21.15.27:0/980339087 learned_addr learned my addr 172.21.15.27:0/980339087 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:22:33.886 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.881+0000 7fd7b259c640 1 --2- 172.21.15.27:0/980339087 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fd7a0003f60 0x7fd7b4145240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:33.886 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.881+0000 7fd7b1d9b640 1 --2- 172.21.15.27:0/980339087 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd7b4158390 0x7fd7b4158810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:33.886 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.881+0000 7fd7b2d9d640 1 -- 172.21.15.27:0/980339087 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fd7a0003f60 msgr2=0x7fd7b4145240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:33.886 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.881+0000 7fd7b2d9d640 1 --2- 172.21.15.27:0/980339087 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fd7a0003f60 0x7fd7b4145240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:33.886 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.881+0000 7fd7b2d9d640 1 -- 172.21.15.27:0/980339087 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd7b4158390 msgr2=0x7fd7b4158810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:33.886 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.881+0000 7fd7b2d9d640 1 --2- 172.21.15.27:0/980339087 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd7b4158390 0x7fd7b4158810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:33.886 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.881+0000 7fd7b2d9d640 1 -- 172.21.15.27:0/980339087 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7b4145780 con 0x7fd7b4145b70 2024-09-12T03:22:33.886 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.881+0000 7fd7b1d9b640 1 --2- 172.21.15.27:0/980339087 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd7b4158390 0x7fd7b4158810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:22:33.886 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.882+0000 7fd7b2d9d640 1 --2- 172.21.15.27:0/980339087 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd7b4145b70 0x7fd7b4146000 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7fd7b406d350 tx=0x7fd7a4010590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:33.886 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.882+0000 7fd7aaffd640 1 -- 172.21.15.27:0/980339087 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7a4010e60 con 0x7fd7b4145b70 2024-09-12T03:22:33.886 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.882+0000 7fd7aaffd640 1 -- 172.21.15.27:0/980339087 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7a4003990 con 0x7fd7b4145b70 2024-09-12T03:22:33.886 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.882+0000 7fd7b8eba640 1 -- 172.21.15.27:0/980339087 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7b4157050 con 0x7fd7b4145b70 2024-09-12T03:22:33.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.882+0000 7fd7aaffd640 1 -- 172.21.15.27:0/980339087 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7a4003da0 con 0x7fd7b4145b70 2024-09-12T03:22:33.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.882+0000 7fd7b8eba640 1 -- 172.21.15.27:0/980339087 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7b41575d0 con 0x7fd7b4145b70 2024-09-12T03:22:33.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.883+0000 7fd7b8eba640 1 -- 172.21.15.27:0/980339087 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7b40aedd0 con 0x7fd7b4145b70 2024-09-12T03:22:33.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.884+0000 7fd7aaffd640 1 -- 172.21.15.27:0/980339087 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 21) ==== 91008+0+0 (secure 0 0 0) 0x7fd7a401e030 con 0x7fd7b4145b70 2024-09-12T03:22:33.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.884+0000 7fd7aaffd640 1 --2- 172.21.15.27:0/980339087 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fd79406c3d0 0x7fd79406e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:33.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.884+0000 7fd7aaffd640 1 -- 172.21.15.27:0/980339087 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 5765+0+0 (secure 0 0 0) 0x7fd7a408bf80 con 0x7fd7b4145b70 2024-09-12T03:22:33.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.884+0000 7fd7b259c640 1 --2- 172.21.15.27:0/980339087 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fd79406c3d0 0x7fd79406e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:33.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.885+0000 7fd7b259c640 1 --2- 172.21.15.27:0/980339087 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fd79406c3d0 0x7fd79406e890 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fd7ac0017e0 tx=0x7fd7ac009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:33.888 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:33.888+0000 7fd7aaffd640 1 -- 172.21.15.27:0/980339087 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fd7a4025020 con 0x7fd7b4145b70 2024-09-12T03:22:34.090 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.090+0000 7fd7b8eba640 1 -- 172.21.15.27:0/980339087 --> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup getpath", "vol_name": "non_existent_subvolume", "group_name": "group_1", "target": ["mon-mgr", ""]}) -- 0x7fd7b40afcb0 con 0x7fd79406c3d0 2024-09-12T03:22:34.091 INFO:tasks.ceph.mgr.y.smithi027.stderr:2024-09-12T03:22:34.091+0000 7f325d8e3640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-12T03:22:34.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.091+0000 7fd7aaffd640 1 -- 172.21.15.27:0/980339087 <== mgr.4113 v2:172.21.15.27:6836/1226920526 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7fd7b40afcb0 con 0x7fd79406c3d0 2024-09-12T03:22:34.091 INFO:teuthology.orchestra.run.smithi027.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-12T03:22:34.097 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-12T03:22:34.097 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup pin non_existent_subvolume group_1 export 0 2024-09-12T03:22:34.206 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.205+0000 7f81b605a640 1 Processor -- start 2024-09-12T03:22:34.206 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.206+0000 7f81b605a640 1 -- start start 2024-09-12T03:22:34.207 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.206+0000 7f81b605a640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f81b006dfd0 0x7f81b00d1da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:34.207 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.206+0000 7f81b605a640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f81b00d3790 0x7f81b00b7a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:34.207 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.206+0000 7f81b605a640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f81b00d2e30 con 0x7f81b0162040 2024-09-12T03:22:34.207 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.206+0000 7f81b605a640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f81b00b7f90 con 0x7f81b00d3790 2024-09-12T03:22:34.208 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.206+0000 7f81b605a640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f81b0162420 con 0x7f81b006dfd0 2024-09-12T03:22:34.208 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.207+0000 7f81af7fe640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f81b0162040 0x7f81b006d8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:38616/0 (socket says 172.21.15.27:38616) 2024-09-12T03:22:34.208 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.207+0000 7f81af7fe640 1 -- 172.21.15.27:0/1805442945 learned_addr learned my addr 172.21.15.27:0/1805442945 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:22:34.208 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.207+0000 7f81aeffd640 1 --2- 172.21.15.27:0/1805442945 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f81b006dfd0 0x7f81b00d1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:34.208 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.207+0000 7f81affff640 1 --2- 172.21.15.27:0/1805442945 >> v2:172.21.15.45:3300/0 conn(0x7f81b00d3790 0x7f81b00b7a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:34.208 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.207+0000 7f81ae7fc640 1 -- 172.21.15.27:0/1805442945 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4211343854 0 0) 0x7f81b00d2e30 con 0x7f81b0162040 2024-09-12T03:22:34.208 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.207+0000 7f81ae7fc640 1 -- 172.21.15.27:0/1805442945 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f819c003520 con 0x7f81b0162040 2024-09-12T03:22:34.208 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.207+0000 7f81ae7fc640 1 -- 172.21.15.27:0/1805442945 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3546112738 0 0) 0x7f819c003520 con 0x7f81b0162040 2024-09-12T03:22:34.208 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.207+0000 7f81aeffd640 1 -- 172.21.15.27:0/1805442945 >> v2:172.21.15.45:3300/0 conn(0x7f81b00d3790 msgr2=0x7f81b00b7a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:34.208 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.207+0000 7f81aeffd640 1 --2- 172.21.15.27:0/1805442945 >> v2:172.21.15.45:3300/0 conn(0x7f81b00d3790 0x7f81b00b7a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:34.208 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.207+0000 7f81aeffd640 1 -- 172.21.15.27:0/1805442945 >> v1:172.21.15.27:6789/0 conn(0x7f81b0162040 legacy=0x7f81b006d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:34.208 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.207+0000 7f81aeffd640 1 -- 172.21.15.27:0/1805442945 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81b00d30d0 con 0x7f81b006dfd0 2024-09-12T03:22:34.210 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.208+0000 7f81aeffd640 1 --2- 172.21.15.27:0/1805442945 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f81b006dfd0 0x7f81b00d1da0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f81980097a0 tx=0x7f819802a110 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7cc685cb27db978f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:34.210 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.208+0000 7f81ae7fc640 1 -- 172.21.15.27:0/1805442945 <== mon.2 v2:172.21.15.45:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8198037070 con 0x7f81b006dfd0 2024-09-12T03:22:34.212 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.208+0000 7f81ae7fc640 1 -- 172.21.15.27:0/1805442945 <== mon.2 v2:172.21.15.45:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f819802aba0 con 0x7f81b006dfd0 2024-09-12T03:22:34.212 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.208+0000 7f81ae7fc640 1 -- 172.21.15.27:0/1805442945 <== mon.2 v2:172.21.15.45:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81a00019f0 con 0x7f81b006dfd0 2024-09-12T03:22:34.212 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.209+0000 7f81b605a640 1 -- 172.21.15.27:0/1805442945 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f81b006dfd0 msgr2=0x7f81b00d1da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:34.212 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.209+0000 7f81b605a640 1 --2- 172.21.15.27:0/1805442945 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f81b006dfd0 0x7f81b00d1da0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f81980097a0 tx=0x7f819802a110 comp rx=0 tx=0).stop 2024-09-12T03:22:34.212 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.209+0000 7f81b605a640 1 -- 172.21.15.27:0/1805442945 shutdown_connections 2024-09-12T03:22:34.212 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.209+0000 7f81b605a640 1 --2- 172.21.15.27:0/1805442945 >> v2:172.21.15.45:3300/0 conn(0x7f81b00d3790 0x7f81b00b7a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:34.213 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.209+0000 7f81b605a640 1 --2- 172.21.15.27:0/1805442945 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f81b006dfd0 0x7f81b00d1da0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:34.213 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.209+0000 7f81b605a640 1 -- 172.21.15.27:0/1805442945 >> 172.21.15.27:0/1805442945 conn(0x7f81b00b6700 msgr2=0x7f81b00b6b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:34.213 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.209+0000 7f81b605a640 1 -- 172.21.15.27:0/1805442945 shutdown_connections 2024-09-12T03:22:34.213 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.209+0000 7f81b605a640 1 -- 172.21.15.27:0/1805442945 wait complete. 2024-09-12T03:22:34.216 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.210+0000 7f81b605a640 1 Processor -- start 2024-09-12T03:22:34.216 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.210+0000 7f81b605a640 1 -- start start 2024-09-12T03:22:34.216 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.211+0000 7f81b605a640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f81b006dfd0 0x7f81b00cd820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:34.216 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.211+0000 7f81b605a640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f81b00d3790 0x7f81b00ac2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:34.216 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.211+0000 7f81b605a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f81b0162040 0x7f81b0151b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:34.216 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.211+0000 7f81b605a640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f81b00d4530 con 0x7f81b006dfd0 2024-09-12T03:22:34.216 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.211+0000 7f81b605a640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f81b00cee00 con 0x7f81b0162040 2024-09-12T03:22:34.216 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.211+0000 7f81b605a640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f81b00d4150 con 0x7f81b00d3790 2024-09-12T03:22:34.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.211+0000 7f81af7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f81b006dfd0 0x7f81b00cd820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:34.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.211+0000 7f81af7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f81b006dfd0 0x7f81b00cd820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55036/0 (socket says 172.21.15.27:55036) 2024-09-12T03:22:34.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.211+0000 7f81af7fe640 1 -- 172.21.15.27:0/775299066 learned_addr learned my addr 172.21.15.27:0/775299066 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:22:34.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.211+0000 7f81aeffd640 1 --2- 172.21.15.27:0/775299066 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f81b00d3790 0x7f81b00ac2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:34.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.211+0000 7f81af7fe640 1 -- 172.21.15.27:0/775299066 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f81b00d3790 msgr2=0x7f81b00ac2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:34.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.211+0000 7f81af7fe640 1 --2- 172.21.15.27:0/775299066 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f81b00d3790 0x7f81b00ac2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:34.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.211+0000 7f81af7fe640 1 -- 172.21.15.27:0/775299066 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f81b0162040 msgr2=0x7f81b0151b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:34.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.211+0000 7f81af7fe640 1 --2- 172.21.15.27:0/775299066 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f81b0162040 0x7f81b0151b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:34.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.211+0000 7f81af7fe640 1 -- 172.21.15.27:0/775299066 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81b00cdf20 con 0x7f81b006dfd0 2024-09-12T03:22:34.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.212+0000 7f81af7fe640 1 --2- 172.21.15.27:0/775299066 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f81b006dfd0 0x7f81b00cd820 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7f81a00018d0 tx=0x7f81a000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:34.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.212+0000 7f81a7fff640 1 -- 172.21.15.27:0/775299066 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81a0017070 con 0x7f81b006dfd0 2024-09-12T03:22:34.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.212+0000 7f81a7fff640 1 -- 172.21.15.27:0/775299066 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81a0012400 con 0x7f81b006dfd0 2024-09-12T03:22:34.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.212+0000 7f81b605a640 1 -- 172.21.15.27:0/775299066 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81b00ce200 con 0x7f81b006dfd0 2024-09-12T03:22:34.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.212+0000 7f81a7fff640 1 -- 172.21.15.27:0/775299066 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81a00127f0 con 0x7f81b006dfd0 2024-09-12T03:22:34.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.212+0000 7f81b605a640 1 -- 172.21.15.27:0/775299066 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81b00ce7c0 con 0x7f81b006dfd0 2024-09-12T03:22:34.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.213+0000 7f81b605a640 1 -- 172.21.15.27:0/775299066 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81b006e6e0 con 0x7f81b006dfd0 2024-09-12T03:22:34.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.213+0000 7f81a7fff640 1 -- 172.21.15.27:0/775299066 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 21) ==== 91008+0+0 (secure 0 0 0) 0x7f81a001a070 con 0x7f81b006dfd0 2024-09-12T03:22:34.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.214+0000 7f81a7fff640 1 --2- 172.21.15.27:0/775299066 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f819006c3d0 0x7f819006e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:34.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.214+0000 7f81aeffd640 1 --2- 172.21.15.27:0/775299066 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f819006c3d0 0x7f819006e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:34.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.214+0000 7f81a7fff640 1 -- 172.21.15.27:0/775299066 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 5765+0+0 (secure 0 0 0) 0x7f81a008b910 con 0x7f81b006dfd0 2024-09-12T03:22:34.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.214+0000 7f81aeffd640 1 --2- 172.21.15.27:0/775299066 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f819006c3d0 0x7f819006e890 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f8198034000 tx=0x7f819802e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:34.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.218+0000 7f81a7fff640 1 -- 172.21.15.27:0/775299066 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f81a0059aa0 con 0x7f81b006dfd0 2024-09-12T03:22:34.420 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.420+0000 7f81b605a640 1 -- 172.21.15.27:0/775299066 --> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "non_existent_subvolume", "group_name": "group_1", "pin_type": "export", "pin_setting": "0", "target": ["mon-mgr", ""]}) -- 0x7f81b01f56f0 con 0x7f819006c3d0 2024-09-12T03:22:34.421 INFO:tasks.ceph.mgr.y.smithi027.stderr:2024-09-12T03:22:34.421+0000 7f325d8e3640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-12T03:22:34.421 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.421+0000 7f81a7fff640 1 -- 172.21.15.27:0/775299066 <== mgr.4113 v2:172.21.15.27:6836/1226920526 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7f819803d440 con 0x7f819006c3d0 2024-09-12T03:22:34.421 INFO:teuthology.orchestra.run.smithi027.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-12T03:22:34.426 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-12T03:22:34.427 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup ls non_existent_subvolume 2024-09-12T03:22:34.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.534+0000 7fca4362e640 1 Processor -- start 2024-09-12T03:22:34.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.534+0000 7fca4362e640 1 -- start start 2024-09-12T03:22:34.539 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.535+0000 7fca4362e640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fca3c15a270 0x7fca3c06cff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:34.539 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.535+0000 7fca4362e640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fca3c0ce650 con 0x7fca3c06d530 2024-09-12T03:22:34.539 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.535+0000 7fca4362e640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fca3c0ce930 con 0x7fca3c0cfb20 2024-09-12T03:22:34.539 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.535+0000 7fca4362e640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fca3c0cef80 con 0x7fca3c15a270 2024-09-12T03:22:34.539 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.535+0000 7fca40bc9640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fca3c06d530 0x7fca3c0cf410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:38620/0 (socket says 172.21.15.27:38620) 2024-09-12T03:22:34.539 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.535+0000 7fca40bc9640 1 -- 172.21.15.27:0/1780182636 learned_addr learned my addr 172.21.15.27:0/1780182636 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:22:34.539 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.535+0000 7fca33fff640 1 -- 172.21.15.27:0/1780182636 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3754000405 0 0) 0x7fca3c0ce650 con 0x7fca3c06d530 2024-09-12T03:22:34.539 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.535+0000 7fca33fff640 1 -- 172.21.15.27:0/1780182636 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fca28003520 con 0x7fca3c06d530 2024-09-12T03:22:34.539 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.535+0000 7fca413ca640 1 --2- 172.21.15.27:0/1780182636 >> v2:172.21.15.45:3300/0 conn(0x7fca3c15a270 0x7fca3c06cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:34.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.535+0000 7fca33fff640 1 -- 172.21.15.27:0/1780182636 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2399290626 0 0) 0x7fca28003520 con 0x7fca3c06d530 2024-09-12T03:22:34.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.535+0000 7fca33fff640 1 -- 172.21.15.27:0/1780182636 >> v2:172.21.15.45:3300/0 conn(0x7fca3c15a270 msgr2=0x7fca3c06cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:34.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.535+0000 7fca33fff640 1 --2- 172.21.15.27:0/1780182636 >> v2:172.21.15.45:3300/0 conn(0x7fca3c15a270 0x7fca3c06cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:34.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.535+0000 7fca33fff640 1 -- 172.21.15.27:0/1780182636 >> v1:172.21.15.45:6789/0 conn(0x7fca3c0cfb20 legacy=0x7fca3c0726a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:34.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.535+0000 7fca33fff640 1 -- 172.21.15.27:0/1780182636 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca3c0cec10 con 0x7fca3c06d530 2024-09-12T03:22:34.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.535+0000 7fca33fff640 1 -- 172.21.15.27:0/1780182636 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 847755213 0 0) 0x7fca2c001670 con 0x7fca3c06d530 2024-09-12T03:22:34.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.535+0000 7fca33fff640 1 -- 172.21.15.27:0/1780182636 >> v1:172.21.15.27:6789/0 conn(0x7fca3c06d530 legacy=0x7fca3c0cf410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:34.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.535+0000 7fca33fff640 1 --2- 172.21.15.27:0/1780182636 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fca28003a00 0x7fca28003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:34.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.535+0000 7fca33fff640 1 --2- 172.21.15.27:0/1780182636 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fca28004800 0x7fca28004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:34.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.535+0000 7fca33fff640 1 --2- 172.21.15.27:0/1780182636 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fca280051c0 0x7fca2800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:34.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.535+0000 7fca33fff640 1 -- 172.21.15.27:0/1780182636 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fca3c0cef80 con 0x7fca28004800 2024-09-12T03:22:34.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.535+0000 7fca33fff640 1 -- 172.21.15.27:0/1780182636 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fca2800c0c0 con 0x7fca280051c0 2024-09-12T03:22:34.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.535+0000 7fca33fff640 1 -- 172.21.15.27:0/1780182636 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fca2800c2c0 con 0x7fca28003a00 2024-09-12T03:22:34.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.536+0000 7fca40bc9640 1 --2- 172.21.15.27:0/1780182636 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fca28004800 0x7fca28004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:34.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.536+0000 7fca413ca640 1 --2- 172.21.15.27:0/1780182636 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fca28003a00 0x7fca28003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:34.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.536+0000 7fca41bcb640 1 --2- 172.21.15.27:0/1780182636 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fca280051c0 0x7fca2800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:34.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.536+0000 7fca40bc9640 1 -- 172.21.15.27:0/1780182636 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fca28003a00 msgr2=0x7fca28003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:34.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.536+0000 7fca40bc9640 1 --2- 172.21.15.27:0/1780182636 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fca28003a00 0x7fca28003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:34.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.536+0000 7fca40bc9640 1 -- 172.21.15.27:0/1780182636 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fca280051c0 msgr2=0x7fca2800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:34.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.536+0000 7fca40bc9640 1 --2- 172.21.15.27:0/1780182636 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fca280051c0 0x7fca2800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:34.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.536+0000 7fca40bc9640 1 -- 172.21.15.27:0/1780182636 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca28003520 con 0x7fca28004800 2024-09-12T03:22:34.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.536+0000 7fca41bcb640 1 -- 172.21.15.27:0/1780182636 reap_dead start 2024-09-12T03:22:34.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.537+0000 7fca40bc9640 1 --2- 172.21.15.27:0/1780182636 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fca28004800 0x7fca28004c80 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7fca2c00c0e0 tx=0x7fca28005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b81164cd9b9f15e6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:34.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.537+0000 7fca33fff640 1 -- 172.21.15.27:0/1780182636 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca2c007c40 con 0x7fca28004800 2024-09-12T03:22:34.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.537+0000 7fca33fff640 1 -- 172.21.15.27:0/1780182636 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca2c005af0 con 0x7fca28004800 2024-09-12T03:22:34.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.537+0000 7fca33fff640 1 -- 172.21.15.27:0/1780182636 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca2c007070 con 0x7fca28004800 2024-09-12T03:22:34.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.537+0000 7fca4362e640 1 -- 172.21.15.27:0/1780182636 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fca28004800 msgr2=0x7fca28004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:34.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.537+0000 7fca4362e640 1 --2- 172.21.15.27:0/1780182636 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fca28004800 0x7fca28004c80 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7fca2c00c0e0 tx=0x7fca28005830 comp rx=0 tx=0).stop 2024-09-12T03:22:34.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.537+0000 7fca4362e640 1 -- 172.21.15.27:0/1780182636 shutdown_connections 2024-09-12T03:22:34.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.537+0000 7fca4362e640 1 --2- 172.21.15.27:0/1780182636 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fca28004800 0x7fca28004c80 unknown :-1 s=CLOSED pgs=1318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:34.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.537+0000 7fca4362e640 1 -- 172.21.15.27:0/1780182636 >> 172.21.15.27:0/1780182636 conn(0x7fca3c06bb30 msgr2=0x7fca3c06bf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:34.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.538+0000 7fca4362e640 1 -- 172.21.15.27:0/1780182636 shutdown_connections 2024-09-12T03:22:34.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.538+0000 7fca4362e640 1 -- 172.21.15.27:0/1780182636 wait complete. 2024-09-12T03:22:34.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.539+0000 7fca4362e640 1 Processor -- start 2024-09-12T03:22:34.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.539+0000 7fca4362e640 1 -- start start 2024-09-12T03:22:34.545 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.540+0000 7fca4362e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fca28004800 0x7fca3c146c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:34.545 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.540+0000 7fca4362e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fca3c159410 0x7fca3c159890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:34.545 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.540+0000 7fca4362e640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fca3c0b4ac0 0x7fca3c0b4f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:34.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.540+0000 7fca4362e640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fca3c06c7d0 con 0x7fca28004800 2024-09-12T03:22:34.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.540+0000 7fca4362e640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fca3c06c650 con 0x7fca3c159410 2024-09-12T03:22:34.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.540+0000 7fca4362e640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fca3c06c950 con 0x7fca3c0b4ac0 2024-09-12T03:22:34.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.540+0000 7fca413ca640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fca28004800 0x7fca3c146c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:34.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.540+0000 7fca413ca640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fca28004800 0x7fca3c146c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55054/0 (socket says 172.21.15.27:55054) 2024-09-12T03:22:34.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.540+0000 7fca413ca640 1 -- 172.21.15.27:0/3381389083 learned_addr learned my addr 172.21.15.27:0/3381389083 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:22:34.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.540+0000 7fca40bc9640 1 --2- 172.21.15.27:0/3381389083 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fca3c159410 0x7fca3c159890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:34.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.540+0000 7fca413ca640 1 -- 172.21.15.27:0/3381389083 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fca3c0b4ac0 msgr2=0x7fca3c0b4f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:34.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.540+0000 7fca41bcb640 1 --2- 172.21.15.27:0/3381389083 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fca3c0b4ac0 0x7fca3c0b4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:34.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.540+0000 7fca413ca640 1 --2- 172.21.15.27:0/3381389083 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fca3c0b4ac0 0x7fca3c0b4f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:34.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.540+0000 7fca413ca640 1 -- 172.21.15.27:0/3381389083 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fca3c159410 msgr2=0x7fca3c159890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:34.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.540+0000 7fca413ca640 1 --2- 172.21.15.27:0/3381389083 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fca3c159410 0x7fca3c159890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:34.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.540+0000 7fca413ca640 1 -- 172.21.15.27:0/3381389083 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca3c1497e0 con 0x7fca28004800 2024-09-12T03:22:34.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.540+0000 7fca40bc9640 1 --2- 172.21.15.27:0/3381389083 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fca3c159410 0x7fca3c159890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:22:34.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.540+0000 7fca413ca640 1 --2- 172.21.15.27:0/3381389083 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fca28004800 0x7fca3c146c40 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7fca38011a70 tx=0x7fca38011f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:34.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.540+0000 7fca31ffb640 1 -- 172.21.15.27:0/3381389083 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca3800fb50 con 0x7fca28004800 2024-09-12T03:22:34.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.540+0000 7fca31ffb640 1 -- 172.21.15.27:0/3381389083 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca3800fcf0 con 0x7fca28004800 2024-09-12T03:22:34.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.540+0000 7fca31ffb640 1 -- 172.21.15.27:0/3381389083 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca3800fea0 con 0x7fca28004800 2024-09-12T03:22:34.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.540+0000 7fca4362e640 1 -- 172.21.15.27:0/3381389083 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca3c149ac0 con 0x7fca28004800 2024-09-12T03:22:34.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.541+0000 7fca4362e640 1 -- 172.21.15.27:0/3381389083 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca3c14a080 con 0x7fca28004800 2024-09-12T03:22:34.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.542+0000 7fca4362e640 1 -- 172.21.15.27:0/3381389083 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca3c159dd0 con 0x7fca28004800 2024-09-12T03:22:34.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.542+0000 7fca31ffb640 1 -- 172.21.15.27:0/3381389083 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 22) ==== 91059+0+0 (secure 0 0 0) 0x7fca38012040 con 0x7fca28004800 2024-09-12T03:22:34.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.542+0000 7fca31ffb640 1 --2- 172.21.15.27:0/3381389083 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fca2006c490 0x7fca2006e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:34.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.543+0000 7fca31ffb640 1 -- 172.21.15.27:0/3381389083 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 5765+0+0 (secure 0 0 0) 0x7fca3808c120 con 0x7fca28004800 2024-09-12T03:22:34.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.543+0000 7fca40bc9640 1 --2- 172.21.15.27:0/3381389083 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fca2006c490 0x7fca2006e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:34.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.543+0000 7fca40bc9640 1 --2- 172.21.15.27:0/3381389083 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fca2006c490 0x7fca2006e950 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fca2c032000 tx=0x7fca2c02d000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:34.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.547+0000 7fca31ffb640 1 -- 172.21.15.27:0/3381389083 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fca38019030 con 0x7fca28004800 2024-09-12T03:22:34.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.745+0000 7fca4362e640 1 -- 172.21.15.27:0/3381389083 --> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup ls", "vol_name": "non_existent_subvolume", "target": ["mon-mgr", ""]}) -- 0x7fca3c063320 con 0x7fca2006c490 2024-09-12T03:22:34.746 INFO:tasks.ceph.mgr.y.smithi027.stderr:2024-09-12T03:22:34.746+0000 7f325d8e3640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-12T03:22:34.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.746+0000 7fca31ffb640 1 -- 172.21.15.27:0/3381389083 <== mgr.4113 v2:172.21.15.27:6836/1226920526 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7fca2c038040 con 0x7fca2006c490 2024-09-12T03:22:34.746 INFO:teuthology.orchestra.run.smithi027.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-12T03:22:34.751 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-12T03:22:34.752 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup snapshot create non_existent_subvolume group_1 snapshot_1 2024-09-12T03:22:34.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.860+0000 7f248ab9d640 1 Processor -- start 2024-09-12T03:22:34.860 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.860+0000 7f248ab9d640 1 -- start start 2024-09-12T03:22:34.862 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.861+0000 7f248ab9d640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f2484161c10 0x7f24840cf690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:34.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.861+0000 7f248ab9d640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f24840cfd70 con 0x7f24840d1660 2024-09-12T03:22:34.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.861+0000 7f248ab9d640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f24840d3550 con 0x7f24840d23d0 2024-09-12T03:22:34.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.861+0000 7f248ab9d640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f24840d3e80 con 0x7f2484161c10 2024-09-12T03:22:34.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.861+0000 7f248a39c640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f24840d1660 0x7f24840b7860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:38632/0 (socket says 172.21.15.27:38632) 2024-09-12T03:22:34.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.861+0000 7f248a39c640 1 -- 172.21.15.27:0/3064324315 learned_addr learned my addr 172.21.15.27:0/3064324315 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:22:34.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.861+0000 7f2489b9b640 1 --2- 172.21.15.27:0/3064324315 >> v2:172.21.15.45:3300/0 conn(0x7f2484161c10 0x7f24840cf690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:34.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.861+0000 7f2488b99640 1 -- 172.21.15.27:0/3064324315 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1767893149 0 0) 0x7f24840cfd70 con 0x7f24840d1660 2024-09-12T03:22:34.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.861+0000 7f2488b99640 1 -- 172.21.15.27:0/3064324315 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2470003520 con 0x7f24840d1660 2024-09-12T03:22:34.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.861+0000 7f2488b99640 1 -- 172.21.15.27:0/3064324315 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3337864866 0 0) 0x7f2470003520 con 0x7f24840d1660 2024-09-12T03:22:34.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.861+0000 7f2488b99640 1 -- 172.21.15.27:0/3064324315 >> v2:172.21.15.45:3300/0 conn(0x7f2484161c10 msgr2=0x7f24840cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:34.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.861+0000 7f2488b99640 1 --2- 172.21.15.27:0/3064324315 >> v2:172.21.15.45:3300/0 conn(0x7f2484161c10 0x7f24840cf690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:34.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.861+0000 7f2488b99640 1 -- 172.21.15.27:0/3064324315 >> v1:172.21.15.45:6789/0 conn(0x7f24840d23d0 legacy=0x7f24840d0f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:34.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.861+0000 7f2488b99640 1 -- 172.21.15.27:0/3064324315 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24840d3830 con 0x7f24840d1660 2024-09-12T03:22:34.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.861+0000 7f2488b99640 1 -- 172.21.15.27:0/3064324315 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 847755213 0 0) 0x7f247c0019f0 con 0x7f24840d1660 2024-09-12T03:22:34.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.861+0000 7f2488b99640 1 -- 172.21.15.27:0/3064324315 >> v1:172.21.15.27:6789/0 conn(0x7f24840d1660 legacy=0x7f24840b7860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:34.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.861+0000 7f2488b99640 1 --2- 172.21.15.27:0/3064324315 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2470003a20 0x7f2470003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:34.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.861+0000 7f2488b99640 1 --2- 172.21.15.27:0/3064324315 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2470004800 0x7f2470004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:34.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.861+0000 7f2488b99640 1 --2- 172.21.15.27:0/3064324315 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f24700051c0 0x7f247000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:34.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.861+0000 7f2488b99640 1 -- 172.21.15.27:0/3064324315 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f24840d3e80 con 0x7f24700051c0 2024-09-12T03:22:34.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.861+0000 7f2488b99640 1 -- 172.21.15.27:0/3064324315 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f247000c0d0 con 0x7f2470004800 2024-09-12T03:22:34.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.861+0000 7f2488b99640 1 -- 172.21.15.27:0/3064324315 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f247000c2d0 con 0x7f2470003a20 2024-09-12T03:22:34.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.861+0000 7f248a39c640 1 --2- 172.21.15.27:0/3064324315 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f24700051c0 0x7f247000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:34.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.862+0000 7f2489b9b640 1 --2- 172.21.15.27:0/3064324315 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2470003a20 0x7f2470003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:34.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.862+0000 7f248939a640 1 --2- 172.21.15.27:0/3064324315 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2470004800 0x7f2470004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:34.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.862+0000 7f248a39c640 1 -- 172.21.15.27:0/3064324315 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2470003a20 msgr2=0x7f2470003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:34.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.862+0000 7f248a39c640 1 --2- 172.21.15.27:0/3064324315 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2470003a20 0x7f2470003e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:34.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.862+0000 7f248a39c640 1 -- 172.21.15.27:0/3064324315 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2470004800 msgr2=0x7f2470004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:34.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.862+0000 7f248a39c640 1 --2- 172.21.15.27:0/3064324315 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2470004800 0x7f2470004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:34.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.862+0000 7f248a39c640 1 -- 172.21.15.27:0/3064324315 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2470003520 con 0x7f24700051c0 2024-09-12T03:22:34.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.862+0000 7f248a39c640 1 -- 172.21.15.27:0/3064324315 reap_dead start 2024-09-12T03:22:34.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.863+0000 7f248a39c640 1 --2- 172.21.15.27:0/3064324315 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f24700051c0 0x7f247000b9c0 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f24840d1a70 tx=0x7f2470005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b863fbf076e7e47 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:34.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.863+0000 7f2488b99640 1 -- 172.21.15.27:0/3064324315 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f247c00a4a0 con 0x7f24700051c0 2024-09-12T03:22:34.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.863+0000 7f2488b99640 1 -- 172.21.15.27:0/3064324315 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f247c02f070 con 0x7f24700051c0 2024-09-12T03:22:34.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.863+0000 7f2488b99640 1 -- 172.21.15.27:0/3064324315 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f247c02bcb0 con 0x7f24700051c0 2024-09-12T03:22:34.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.863+0000 7f248ab9d640 1 -- 172.21.15.27:0/3064324315 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f24700051c0 msgr2=0x7f247000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:34.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.863+0000 7f248ab9d640 1 --2- 172.21.15.27:0/3064324315 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f24700051c0 0x7f247000b9c0 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f24840d1a70 tx=0x7f2470005830 comp rx=0 tx=0).stop 2024-09-12T03:22:34.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.863+0000 7f248ab9d640 1 -- 172.21.15.27:0/3064324315 shutdown_connections 2024-09-12T03:22:34.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.863+0000 7f248ab9d640 1 --2- 172.21.15.27:0/3064324315 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f24700051c0 0x7f247000b9c0 unknown :-1 s=CLOSED pgs=1321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:34.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.863+0000 7f248ab9d640 1 -- 172.21.15.27:0/3064324315 >> 172.21.15.27:0/3064324315 conn(0x7f24840b6320 msgr2=0x7f24840b6730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:34.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.863+0000 7f248ab9d640 1 -- 172.21.15.27:0/3064324315 shutdown_connections 2024-09-12T03:22:34.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.863+0000 7f248ab9d640 1 -- 172.21.15.27:0/3064324315 wait complete. 2024-09-12T03:22:34.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.865+0000 7f248ab9d640 1 Processor -- start 2024-09-12T03:22:34.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.865+0000 7f248ab9d640 1 -- start start 2024-09-12T03:22:34.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.865+0000 7f248ab9d640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f24700051c0 0x7f2484155b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:34.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.865+0000 7f248ab9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f24840c7a70 0x7f2484159350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:34.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.865+0000 7f248ab9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2484159890 0x7f2484159d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:34.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.865+0000 7f248ab9d640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f2484161ff0 con 0x7f2484159890 2024-09-12T03:22:34.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.865+0000 7f248ab9d640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f24840d2840 con 0x7f24840c7a70 2024-09-12T03:22:34.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.865+0000 7f248ab9d640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f24840d4310 con 0x7f24700051c0 2024-09-12T03:22:34.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.865+0000 7f248a39c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2484159890 0x7f2484159d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:34.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.865+0000 7f248a39c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2484159890 0x7f2484159d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55066/0 (socket says 172.21.15.27:55066) 2024-09-12T03:22:34.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.865+0000 7f248a39c640 1 -- 172.21.15.27:0/1791943786 learned_addr learned my addr 172.21.15.27:0/1791943786 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:22:34.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.866+0000 7f2489b9b640 1 --2- 172.21.15.27:0/1791943786 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f24700051c0 0x7f2484155b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:34.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.866+0000 7f248939a640 1 --2- 172.21.15.27:0/1791943786 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f24840c7a70 0x7f2484159350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:34.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.866+0000 7f248a39c640 1 -- 172.21.15.27:0/1791943786 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f24700051c0 msgr2=0x7f2484155b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:34.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.866+0000 7f248a39c640 1 --2- 172.21.15.27:0/1791943786 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f24700051c0 0x7f2484155b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:34.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.866+0000 7f248a39c640 1 -- 172.21.15.27:0/1791943786 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f24840c7a70 msgr2=0x7f2484159350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:34.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.866+0000 7f248a39c640 1 --2- 172.21.15.27:0/1791943786 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f24840c7a70 0x7f2484159350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:34.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.866+0000 7f248a39c640 1 -- 172.21.15.27:0/1791943786 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2484158e30 con 0x7f2484159890 2024-09-12T03:22:34.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.866+0000 7f248939a640 1 --2- 172.21.15.27:0/1791943786 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f24840c7a70 0x7f2484159350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:22:34.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.866+0000 7f248a39c640 1 --2- 172.21.15.27:0/1791943786 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2484159890 0x7f2484159d10 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f247c02b500 tx=0x7f247c02d070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:34.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.866+0000 7f247a7fc640 1 -- 172.21.15.27:0/1791943786 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f247c00a480 con 0x7f2484159890 2024-09-12T03:22:34.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.866+0000 7f247a7fc640 1 -- 172.21.15.27:0/1791943786 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f247c034470 con 0x7f2484159890 2024-09-12T03:22:34.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.866+0000 7f247a7fc640 1 -- 172.21.15.27:0/1791943786 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f247c034820 con 0x7f2484159890 2024-09-12T03:22:34.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.866+0000 7f248ab9d640 1 -- 172.21.15.27:0/1791943786 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2484158590 con 0x7f2484159890 2024-09-12T03:22:34.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.866+0000 7f248ab9d640 1 -- 172.21.15.27:0/1791943786 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2484158b50 con 0x7f2484159890 2024-09-12T03:22:34.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.867+0000 7f248ab9d640 1 -- 172.21.15.27:0/1791943786 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24840cb4c0 con 0x7f2484159890 2024-09-12T03:22:34.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.868+0000 7f247a7fc640 1 -- 172.21.15.27:0/1791943786 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 22) ==== 91059+0+0 (secure 0 0 0) 0x7f247c0349c0 con 0x7f2484159890 2024-09-12T03:22:34.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.868+0000 7f247a7fc640 1 --2- 172.21.15.27:0/1791943786 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f246806c490 0x7f246806e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:34.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.868+0000 7f247a7fc640 1 -- 172.21.15.27:0/1791943786 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 5765+0+0 (secure 0 0 0) 0x7f247c0abc30 con 0x7f2484159890 2024-09-12T03:22:34.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.868+0000 7f2489b9b640 1 --2- 172.21.15.27:0/1791943786 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f246806c490 0x7f246806e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:34.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.869+0000 7f2489b9b640 1 --2- 172.21.15.27:0/1791943786 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f246806c490 0x7f246806e950 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f2480009b40 tx=0x7f2480009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:34.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:34.872+0000 7f247a7fc640 1 -- 172.21.15.27:0/1791943786 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f247c079cf0 con 0x7f2484159890 2024-09-12T03:22:35.072 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:35.072+0000 7f248ab9d640 1 -- 172.21.15.27:0/1791943786 --> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup snapshot create", "vol_name": "non_existent_subvolume", "group_name": "group_1", "snap_name": "snapshot_1", "target": ["mon-mgr", ""]}) -- 0x7f24840b97a0 con 0x7f246806c490 2024-09-12T03:22:35.073 INFO:tasks.ceph.mgr.y.smithi027.stderr:2024-09-12T03:22:35.074+0000 7f325d8e3640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-12T03:22:35.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:35.074+0000 7f247a7fc640 1 -- 172.21.15.27:0/1791943786 <== mgr.4113 v2:172.21.15.27:6836/1226920526 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7f24840b97a0 con 0x7f246806c490 2024-09-12T03:22:35.074 INFO:teuthology.orchestra.run.smithi027.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-12T03:22:35.079 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-12T03:22:35.079 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup snapshot rm non_existent_subvolume group_1 snapshot_1 2024-09-12T03:22:35.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:35.187+0000 7fc8438c9640 1 Processor -- start 2024-09-12T03:22:35.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:35.187+0000 7fc8438c9640 1 -- start start 2024-09-12T03:22:35.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:35.188+0000 7fc8438c9640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fc83c1645c0 0x7fc83c1669b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:35.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:35.188+0000 7fc8438c9640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc83c0b4720 con 0x7fc83c15ec30 2024-09-12T03:22:35.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:35.188+0000 7fc8438c9640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc83c0ce5e0 con 0x7fc83c0af9d0 2024-09-12T03:22:35.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:35.188+0000 7fc8438c9640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fc83c0d02d0 con 0x7fc83c1645c0 2024-09-12T03:22:35.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:35.188+0000 7fc841665640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fc83c15ec30 0x7fc83c0b3900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:38634/0 (socket says 172.21.15.27:38634) 2024-09-12T03:22:35.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:35.188+0000 7fc841665640 1 -- 172.21.15.27:0/2947913807 learned_addr learned my addr 172.21.15.27:0/2947913807 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:22:35.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:35.188+0000 7fc83bfff640 1 -- 172.21.15.27:0/2947913807 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 784761473 0 0) 0x7fc83c0b4720 con 0x7fc83c15ec30 2024-09-12T03:22:35.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:35.188+0000 7fc841e66640 1 --2- 172.21.15.27:0/2947913807 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fc83c1645c0 0x7fc83c1669b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:35.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:35.188+0000 7fc83bfff640 1 -- 172.21.15.27:0/2947913807 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc824003520 con 0x7fc83c15ec30 2024-09-12T03:22:35.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:35.189+0000 7fc83bfff640 1 -- 172.21.15.27:0/2947913807 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1697766294 0 0) 0x7fc824003520 con 0x7fc83c15ec30 2024-09-12T03:22:35.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:35.189+0000 7fc83bfff640 1 -- 172.21.15.27:0/2947913807 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fc83c1645c0 msgr2=0x7fc83c1669b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:35.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:35.189+0000 7fc83bfff640 1 --2- 172.21.15.27:0/2947913807 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fc83c1645c0 0x7fc83c1669b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:35.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:35.189+0000 7fc83bfff640 1 -- 172.21.15.27:0/2947913807 >> v1:172.21.15.45:6789/0 conn(0x7fc83c0af9d0 legacy=0x7fc83c0b4010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:35.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:35.189+0000 7fc83bfff640 1 -- 172.21.15.27:0/2947913807 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc83c0ceac0 con 0x7fc83c15ec30 2024-09-12T03:22:35.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:35.189+0000 7fc83bfff640 1 -- 172.21.15.27:0/2947913807 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 847755213 0 0) 0x7fc82c0019f0 con 0x7fc83c15ec30 2024-09-12T03:22:35.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:35.189+0000 7fc83bfff640 1 -- 172.21.15.27:0/2947913807 >> v1:172.21.15.27:6789/0 conn(0x7fc83c15ec30 legacy=0x7fc83c0b3900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:35.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:35.189+0000 7fc83bfff640 1 --2- 172.21.15.27:0/2947913807 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fc83c1645c0 0x7fc8240039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:35.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:35.189+0000 7fc83bfff640 1 --2- 172.21.15.27:0/2947913807 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc824003f20 0x7fc824004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:35.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:35.189+0000 7fc83bfff640 1 --2- 172.21.15.27:0/2947913807 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc824004d40 0x7fc8240051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:35.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:35.189+0000 7fc83bfff640 1 -- 172.21.15.27:0/2947913807 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fc83c0d02d0 con 0x7fc824004d40 2024-09-12T03:22:35.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:35.189+0000 7fc83bfff640 1 -- 172.21.15.27:0/2947913807 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc82400ba20 con 0x7fc824003f20 2024-09-12T03:22:35.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:35.189+0000 7fc83bfff640 1 -- 172.21.15.27:0/2947913807 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fc82400de30 con 0x7fc83c1645c0 2024-09-12T03:22:35.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:35.189+0000 7fc841e66640 1 --2- 172.21.15.27:0/2947913807 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc824004d40 0x7fc8240051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:35.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:35.189+0000 7fc841665640 1 --2- 172.21.15.27:0/2947913807 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fc83c1645c0 0x7fc8240039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:35.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:35.189+0000 7fc840e64640 1 --2- 172.21.15.27:0/2947913807 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc824003f20 0x7fc824004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:35.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:35.189+0000 7fc841e66640 1 -- 172.21.15.27:0/2947913807 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fc83c1645c0 msgr2=0x7fc8240039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:35.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:35.189+0000 7fc841e66640 1 --2- 172.21.15.27:0/2947913807 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fc83c1645c0 0x7fc8240039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:35.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:35.189+0000 7fc841e66640 1 -- 172.21.15.27:0/2947913807 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc824003f20 msgr2=0x7fc824004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:35.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:35.189+0000 7fc841e66640 1 --2- 172.21.15.27:0/2947913807 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc824003f20 0x7fc824004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:35.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:35.189+0000 7fc841e66640 1 -- 172.21.15.27:0/2947913807 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc824003520 con 0x7fc824004d40 2024-09-12T03:22:35.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:35.190+0000 7fc841e66640 1 --2- 172.21.15.27:0/2947913807 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc824004d40 0x7fc8240051c0 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7fc834009ac0 tx=0x7fc824005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68d2cf852d22bf36 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:35.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:35.190+0000 7fc83bfff640 1 -- 172.21.15.27:0/2947913807 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc834036070 con 0x7fc824004d40 2024-09-12T03:22:35.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:35.190+0000 7fc83bfff640 1 -- 172.21.15.27:0/2947913807 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc834031400 con 0x7fc824004d40 2024-09-12T03:22:35.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:35.190+0000 7fc83bfff640 1 -- 172.21.15.27:0/2947913807 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc834009690 con 0x7fc824004d40 2024-09-12T03:22:35.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:35.191+0000 7fc8438c9640 1 -- 172.21.15.27:0/2947913807 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc824004d40 msgr2=0x7fc8240051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:35.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:35.191+0000 7fc8438c9640 1 --2- 172.21.15.27:0/2947913807 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc824004d40 0x7fc8240051c0 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7fc834009ac0 tx=0x7fc824005830 comp rx=0 tx=0).stop 2024-09-12T03:22:35.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:35.191+0000 7fc841e66640 1 -- 172.21.15.27:0/2947913807 reap_dead start 2024-09-12T03:22:35.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:35.191+0000 7fc8438c9640 1 -- 172.21.15.27:0/2947913807 shutdown_connections 2024-09-12T03:22:35.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:35.191+0000 7fc8438c9640 1 -- 172.21.15.27:0/2947913807 >> 172.21.15.27:0/2947913807 conn(0x7fc83c06bb30 msgr2=0x7fc83c06bf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:35.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:35.191+0000 7fc8438c9640 1 -- 172.21.15.27:0/2947913807 shutdown_connections 2024-09-12T03:22:35.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:35.191+0000 7fc8438c9640 1 -- 172.21.15.27:0/2947913807 wait complete. 2024-09-12T03:22:35.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:35.192+0000 7fc8438c9640 1 Processor -- start 2024-09-12T03:22:35.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:35.192+0000 7fc8438c9640 1 -- start start 2024-09-12T03:22:35.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:35.193+0000 7fc8438c9640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc83c1f3cd0 0x7fc83c153800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:35.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:35.193+0000 7fc8438c9640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fc83c153d40 0x7fc83c151c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:35.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:35.193+0000 7fc8438c9640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc83c152150 0x7fc83c1f40b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:35.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:35.193+0000 7fc8438c9640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fc83c0cf130 con 0x7fc83c1f3cd0 2024-09-12T03:22:35.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:35.193+0000 7fc8438c9640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc83c0cefb0 con 0x7fc83c152150 2024-09-12T03:22:35.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:35.193+0000 7fc8438c9640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fc83c0cf2b0 con 0x7fc83c153d40 2024-09-12T03:22:35.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:35.193+0000 7fc841665640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc83c1f3cd0 0x7fc83c153800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:35.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:35.193+0000 7fc841665640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc83c1f3cd0 0x7fc83c153800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55074/0 (socket says 172.21.15.27:55074) 2024-09-12T03:22:35.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:35.193+0000 7fc841665640 1 -- 172.21.15.27:0/1400556082 learned_addr learned my addr 172.21.15.27:0/1400556082 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:22:35.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:35.193+0000 7fc840e64640 1 --2- 172.21.15.27:0/1400556082 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fc83c153d40 0x7fc83c151c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:35.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:35.193+0000 7fc841665640 1 -- 172.21.15.27:0/1400556082 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fc83c153d40 msgr2=0x7fc83c151c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:35.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:35.193+0000 7fc841665640 1 --2- 172.21.15.27:0/1400556082 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fc83c153d40 0x7fc83c151c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:35.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:35.193+0000 7fc841e66640 1 --2- 172.21.15.27:0/1400556082 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc83c152150 0x7fc83c1f40b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:35.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:35.193+0000 7fc841665640 1 -- 172.21.15.27:0/1400556082 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc83c152150 msgr2=0x7fc83c1f40b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:35.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:35.193+0000 7fc841665640 1 --2- 172.21.15.27:0/1400556082 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc83c152150 0x7fc83c1f40b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:35.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:35.193+0000 7fc841665640 1 -- 172.21.15.27:0/1400556082 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc83c1527f0 con 0x7fc83c1f3cd0 2024-09-12T03:22:35.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:35.193+0000 7fc841e66640 1 --2- 172.21.15.27:0/1400556082 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc83c152150 0x7fc83c1f40b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:22:35.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:35.193+0000 7fc841665640 1 --2- 172.21.15.27:0/1400556082 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc83c1f3cd0 0x7fc83c153800 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7fc82c00a460 tx=0x7fc82c00a920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:35.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:35.194+0000 7fc839ffb640 1 -- 172.21.15.27:0/1400556082 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc82c017020 con 0x7fc83c1f3cd0 2024-09-12T03:22:35.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:35.194+0000 7fc839ffb640 1 -- 172.21.15.27:0/1400556082 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc82c013600 con 0x7fc83c1f3cd0 2024-09-12T03:22:35.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:35.194+0000 7fc8438c9640 1 -- 172.21.15.27:0/1400556082 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc83c152ad0 con 0x7fc83c1f3cd0 2024-09-12T03:22:35.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:35.194+0000 7fc839ffb640 1 -- 172.21.15.27:0/1400556082 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc82c013a10 con 0x7fc83c1f3cd0 2024-09-12T03:22:35.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:35.194+0000 7fc8438c9640 1 -- 172.21.15.27:0/1400556082 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc83c153090 con 0x7fc83c1f3cd0 2024-09-12T03:22:35.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:35.195+0000 7fc8438c9640 1 -- 172.21.15.27:0/1400556082 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc83c152560 con 0x7fc83c1f3cd0 2024-09-12T03:22:35.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:35.195+0000 7fc839ffb640 1 -- 172.21.15.27:0/1400556082 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 22) ==== 91059+0+0 (secure 0 0 0) 0x7fc82c01a020 con 0x7fc83c1f3cd0 2024-09-12T03:22:35.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:35.196+0000 7fc839ffb640 1 --2- 172.21.15.27:0/1400556082 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fc82806c460 0x7fc82806e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:35.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:35.196+0000 7fc839ffb640 1 -- 172.21.15.27:0/1400556082 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 5765+0+0 (secure 0 0 0) 0x7fc82c08cab0 con 0x7fc83c1f3cd0 2024-09-12T03:22:35.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:35.196+0000 7fc840e64640 1 --2- 172.21.15.27:0/1400556082 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fc82806c460 0x7fc82806e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:35.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:35.196+0000 7fc840e64640 1 --2- 172.21.15.27:0/1400556082 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fc82806c460 0x7fc82806e920 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fc830005620 tx=0x7fc830009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:35.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:35.200+0000 7fc839ffb640 1 -- 172.21.15.27:0/1400556082 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fc82c021020 con 0x7fc83c1f3cd0 2024-09-12T03:22:35.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:35.399+0000 7fc8438c9640 1 -- 172.21.15.27:0/1400556082 --> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup snapshot rm", "vol_name": "non_existent_subvolume", "group_name": "group_1", "snap_name": "snapshot_1", "target": ["mon-mgr", ""]}) -- 0x7fc83c162b60 con 0x7fc82806c460 2024-09-12T03:22:35.399 INFO:tasks.ceph.mgr.y.smithi027.stderr:2024-09-12T03:22:35.400+0000 7f325d8e3640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-12T03:22:35.400 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:35.400+0000 7fc839ffb640 1 -- 172.21.15.27:0/1400556082 <== mgr.4113 v2:172.21.15.27:6836/1226920526 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7fc830005270 con 0x7fc82806c460 2024-09-12T03:22:35.400 INFO:teuthology.orchestra.run.smithi027.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-12T03:22:35.405 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-12T03:22:35.406 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup snapshot ls non_existent_subvolume group_1 2024-09-12T03:22:35.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:35.515+0000 7f3c52ee4640 1 Processor -- start 2024-09-12T03:22:35.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:35.515+0000 7f3c52ee4640 1 -- start start 2024-09-12T03:22:35.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:35.516+0000 7f3c52ee4640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f3c4c15e650 0x7f3c4c0d1d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:35.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:35.516+0000 7f3c52ee4640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f3c4c06cc40 0x7f3c4c06d060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:35.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:35.516+0000 7f3c52ee4640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3c4c0d2280 con 0x7f3c4c06db30 2024-09-12T03:22:35.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:35.516+0000 7f3c52ee4640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f3c4c06d5a0 con 0x7f3c4c15e650 2024-09-12T03:22:35.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:35.516+0000 7f3c52ee4640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f3c4c06e670 con 0x7f3c4c06cc40 2024-09-12T03:22:35.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:35.516+0000 7f3c50c80640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f3c4c15e650 0x7f3c4c0d1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:35.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:35.516+0000 7f3c4bfff640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f3c4c06cc40 0x7f3c4c06d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:35.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:35.516+0000 7f3c51481640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f3c4c06db30 0x7f3c4c06df60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.27:41194/0 (socket says 172.21.15.27:41194) 2024-09-12T03:22:35.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:35.516+0000 7f3c4bfff640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f3c4c06cc40 0x7f3c4c06d060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3301/0 says I am v2:172.21.15.27:36968/0 (socket says 172.21.15.27:36968) 2024-09-12T03:22:35.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:35.516+0000 7f3c50c80640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f3c4c15e650 0x7f3c4c0d1d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.27:57114/0 (socket says 172.21.15.27:57114) 2024-09-12T03:22:35.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:35.516+0000 7f3c51481640 1 -- 172.21.15.27:0/2126131153 learned_addr learned my addr 172.21.15.27:0/2126131153 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:22:35.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:35.517+0000 7f3c4bfff640 1 -- 172.21.15.27:0/2126131153 >> v2:172.21.15.45:3300/0 conn(0x7f3c4c15e650 msgr2=0x7f3c4c0d1d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:35.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:35.517+0000 7f3c4bfff640 1 --2- 172.21.15.27:0/2126131153 >> v2:172.21.15.45:3300/0 conn(0x7f3c4c15e650 0x7f3c4c0d1d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:35.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:35.517+0000 7f3c4bfff640 1 -- 172.21.15.27:0/2126131153 >> v1:172.21.15.45:6789/0 conn(0x7f3c4c06db30 legacy=0x7f3c4c06df60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:35.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:35.517+0000 7f3c4bfff640 1 -- 172.21.15.27:0/2126131153 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c4c06d850 con 0x7f3c4c06cc40 2024-09-12T03:22:35.519 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:35.517+0000 7f3c4bfff640 1 --2- 172.21.15.27:0/2126131153 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f3c4c06cc40 0x7f3c4c06d060 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f3c3c009d90 tx=0x7f3c3c02a5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c914b723b9b77982 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:35.519 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:35.518+0000 7f3c4b7fe640 1 -- 172.21.15.27:0/2126131153 <== mon.2 v2:172.21.15.45:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c3c031600 con 0x7f3c4c06cc40 2024-09-12T03:22:35.519 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:35.518+0000 7f3c4b7fe640 1 -- 172.21.15.27:0/2126131153 <== mon.2 v2:172.21.15.45:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c3c0317a0 con 0x7f3c4c06cc40 2024-09-12T03:22:35.519 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:35.518+0000 7f3c4b7fe640 1 -- 172.21.15.27:0/2126131153 <== mon.2 v2:172.21.15.45:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c3c031b70 con 0x7f3c4c06cc40 2024-09-12T03:22:35.519 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:35.518+0000 7f3c52ee4640 1 -- 172.21.15.27:0/2126131153 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f3c4c06cc40 msgr2=0x7f3c4c06d060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:35.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:35.518+0000 7f3c52ee4640 1 --2- 172.21.15.27:0/2126131153 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f3c4c06cc40 0x7f3c4c06d060 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f3c3c009d90 tx=0x7f3c3c02a5d0 comp rx=0 tx=0).stop 2024-09-12T03:22:35.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:35.518+0000 7f3c52ee4640 1 -- 172.21.15.27:0/2126131153 shutdown_connections 2024-09-12T03:22:35.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:35.518+0000 7f3c52ee4640 1 --2- 172.21.15.27:0/2126131153 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f3c4c06cc40 0x7f3c4c06d060 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:35.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:35.518+0000 7f3c52ee4640 1 --2- 172.21.15.27:0/2126131153 >> v2:172.21.15.45:3300/0 conn(0x7f3c4c15e650 0x7f3c4c0d1d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:35.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:35.518+0000 7f3c52ee4640 1 -- 172.21.15.27:0/2126131153 >> 172.21.15.27:0/2126131153 conn(0x7f3c4c06f320 msgr2=0x7f3c4c06f700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:35.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:35.518+0000 7f3c52ee4640 1 -- 172.21.15.27:0/2126131153 shutdown_connections 2024-09-12T03:22:35.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:35.518+0000 7f3c52ee4640 1 -- 172.21.15.27:0/2126131153 wait complete. 2024-09-12T03:22:35.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:35.519+0000 7f3c52ee4640 1 Processor -- start 2024-09-12T03:22:35.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:35.520+0000 7f3c52ee4640 1 -- start start 2024-09-12T03:22:35.522 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:35.520+0000 7f3c52ee4640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f3c4c06cc40 0x7f3c4c0ce720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:35.522 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:35.520+0000 7f3c52ee4640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3c4c06db30 0x7f3c4c1f44d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:35.522 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:35.520+0000 7f3c52ee4640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3c4c15e650 0x7f3c4c1f4a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:35.522 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:35.520+0000 7f3c52ee4640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f3c4c15ea30 con 0x7f3c4c15e650 2024-09-12T03:22:35.522 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:35.520+0000 7f3c52ee4640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f3c4c0d1540 con 0x7f3c4c06db30 2024-09-12T03:22:35.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:35.520+0000 7f3c52ee4640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f3c4c15ebb0 con 0x7f3c4c06cc40 2024-09-12T03:22:35.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:35.520+0000 7f3c51481640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3c4c15e650 0x7f3c4c1f4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:35.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:35.521+0000 7f3c51481640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3c4c15e650 0x7f3c4c1f4a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55090/0 (socket says 172.21.15.27:55090) 2024-09-12T03:22:35.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:35.521+0000 7f3c51481640 1 -- 172.21.15.27:0/3440752493 learned_addr learned my addr 172.21.15.27:0/3440752493 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:22:35.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:35.521+0000 7f3c4bfff640 1 --2- 172.21.15.27:0/3440752493 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3c4c06db30 0x7f3c4c1f44d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:35.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:35.521+0000 7f3c50c80640 1 --2- 172.21.15.27:0/3440752493 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f3c4c06cc40 0x7f3c4c0ce720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:35.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:35.521+0000 7f3c51481640 1 -- 172.21.15.27:0/3440752493 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f3c4c06cc40 msgr2=0x7f3c4c0ce720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:35.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:35.521+0000 7f3c51481640 1 --2- 172.21.15.27:0/3440752493 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f3c4c06cc40 0x7f3c4c0ce720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:35.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:35.521+0000 7f3c51481640 1 -- 172.21.15.27:0/3440752493 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3c4c06db30 msgr2=0x7f3c4c1f44d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:35.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:35.521+0000 7f3c51481640 1 --2- 172.21.15.27:0/3440752493 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3c4c06db30 0x7f3c4c1f44d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:35.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:35.521+0000 7f3c51481640 1 -- 172.21.15.27:0/3440752493 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c4c0cf210 con 0x7f3c4c15e650 2024-09-12T03:22:35.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:35.521+0000 7f3c51481640 1 --2- 172.21.15.27:0/3440752493 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3c4c15e650 0x7f3c4c1f4a10 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7f3c3400c3c0 tx=0x7f3c3400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:35.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:35.521+0000 7f3c497fa640 1 -- 172.21.15.27:0/3440752493 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c34017070 con 0x7f3c4c15e650 2024-09-12T03:22:35.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:35.521+0000 7f3c497fa640 1 -- 172.21.15.27:0/3440752493 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c34005ce0 con 0x7f3c4c15e650 2024-09-12T03:22:35.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:35.521+0000 7f3c52ee4640 1 -- 172.21.15.27:0/3440752493 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c4c0cf4f0 con 0x7f3c4c15e650 2024-09-12T03:22:35.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:35.521+0000 7f3c497fa640 1 -- 172.21.15.27:0/3440752493 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c340072f0 con 0x7f3c4c15e650 2024-09-12T03:22:35.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:35.521+0000 7f3c52ee4640 1 -- 172.21.15.27:0/3440752493 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c4c1e8300 con 0x7f3c4c15e650 2024-09-12T03:22:35.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:35.522+0000 7f3c52ee4640 1 -- 172.21.15.27:0/3440752493 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c4c06d060 con 0x7f3c4c15e650 2024-09-12T03:22:35.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:35.523+0000 7f3c497fa640 1 -- 172.21.15.27:0/3440752493 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 22) ==== 91059+0+0 (secure 0 0 0) 0x7f3c3401a070 con 0x7f3c4c15e650 2024-09-12T03:22:35.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:35.523+0000 7f3c497fa640 1 --2- 172.21.15.27:0/3440752493 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f3c2c06c460 0x7f3c2c06e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:35.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:35.523+0000 7f3c497fa640 1 -- 172.21.15.27:0/3440752493 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 5765+0+0 (secure 0 0 0) 0x7f3c3408aaf0 con 0x7f3c4c15e650 2024-09-12T03:22:35.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:35.523+0000 7f3c50c80640 1 --2- 172.21.15.27:0/3440752493 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f3c2c06c460 0x7f3c2c06e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:35.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:35.524+0000 7f3c50c80640 1 --2- 172.21.15.27:0/3440752493 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f3c2c06c460 0x7f3c2c06e920 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f3c4c15ca20 tx=0x7f3c44009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:35.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:35.528+0000 7f3c497fa640 1 -- 172.21.15.27:0/3440752493 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f3c34058560 con 0x7f3c4c15e650 2024-09-12T03:22:35.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:35.730+0000 7f3c52ee4640 1 -- 172.21.15.27:0/3440752493 --> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup snapshot ls", "vol_name": "non_existent_subvolume", "group_name": "group_1", "target": ["mon-mgr", ""]}) -- 0x7f3c4c15d870 con 0x7f3c2c06c460 2024-09-12T03:22:35.730 INFO:tasks.ceph.mgr.y.smithi027.stderr:2024-09-12T03:22:35.731+0000 7f325d8e3640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-12T03:22:35.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:35.731+0000 7f3c497fa640 1 -- 172.21.15.27:0/3440752493 <== mgr.4113 v2:172.21.15.27:6836/1226920526 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7f3c4c15d870 con 0x7f3c2c06c460 2024-09-12T03:22:35.731 INFO:teuthology.orchestra.run.smithi027.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-12T03:22:35.736 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-12T03:22:35.737 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-12T03:22:35.737 DEBUG:teuthology.orchestra.run.smithi027:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-12T03:22:35.774 DEBUG:teuthology.orchestra.run.smithi045:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-12T03:22:35.826 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-12T03:22:35.826 INFO:tasks.cephfs.mount:Terminating background process 2024-09-12T03:22:35.826 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-12T03:22:35.826 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T03:22:35.843 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-12T03:22:35.843 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T03:22:35.900 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-12T03:22:36.043 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:36.042+0000 7fecb5ca5640 1 Processor -- start 2024-09-12T03:22:36.043 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:36.042+0000 7fecb5ca5640 1 -- start start 2024-09-12T03:22:36.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:36.042+0000 7fecb5ca5640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fecb0145c30 0x7fecb015ca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:36.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:36.043+0000 7fecb5ca5640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fecb0146010 0x7fecb00cd330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:36.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:36.043+0000 7fecb5ca5640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fecb00cd870 0x7fecb00b4040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:36.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:36.043+0000 7fecb5ca5640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7fecb00b5050 con 0x7fecb0145c30 2024-09-12T03:22:36.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:36.043+0000 7fecb5ca5640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fecb00b4d70 con 0x7fecb00cd870 2024-09-12T03:22:36.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:36.043+0000 7fecb5ca5640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fecb006d140 con 0x7fecb0146010 2024-09-12T03:22:36.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:36.043+0000 7fecaf7fe640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fecb0145c30 0x7fecb015ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:36.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:36.043+0000 7fecaf7fe640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fecb0145c30 0x7fecb015ca00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55106/0 (socket says 172.21.15.27:55106) 2024-09-12T03:22:36.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:36.043+0000 7fecaf7fe640 1 -- 172.21.15.27:0/3888587064 learned_addr learned my addr 172.21.15.27:0/3888587064 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:22:36.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:36.043+0000 7fecaeffd640 1 --2- 172.21.15.27:0/3888587064 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fecb0146010 0x7fecb00cd330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:36.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:36.043+0000 7fecaf7fe640 1 -- 172.21.15.27:0/3888587064 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fecb0146010 msgr2=0x7fecb00cd330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:36.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:36.043+0000 7fecaffff640 1 --2- 172.21.15.27:0/3888587064 >> v2:172.21.15.45:3300/0 conn(0x7fecb00cd870 0x7fecb00b4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:36.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:36.043+0000 7fecaf7fe640 1 --2- 172.21.15.27:0/3888587064 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fecb0146010 0x7fecb00cd330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:36.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:36.043+0000 7fecaf7fe640 1 -- 172.21.15.27:0/3888587064 >> v2:172.21.15.45:3300/0 conn(0x7fecb00cd870 msgr2=0x7fecb00b4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:36.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:36.043+0000 7fecaf7fe640 1 --2- 172.21.15.27:0/3888587064 >> v2:172.21.15.45:3300/0 conn(0x7fecb00cd870 0x7fecb00b4040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:36.046 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:36.043+0000 7fecaf7fe640 1 -- 172.21.15.27:0/3888587064 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecb006d420 con 0x7fecb0145c30 2024-09-12T03:22:36.046 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:36.044+0000 7fecaf7fe640 1 --2- 172.21.15.27:0/3888587064 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fecb0145c30 0x7fecb015ca00 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7fecb00b3f20 tx=0x7feca002a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6130ab0bffd26a7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:36.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:36.044+0000 7fecae7fc640 1 -- 172.21.15.27:0/3888587064 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feca0037070 con 0x7fecb0145c30 2024-09-12T03:22:36.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:36.044+0000 7fecae7fc640 1 -- 172.21.15.27:0/3888587064 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feca0032440 con 0x7fecb0145c30 2024-09-12T03:22:36.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:36.044+0000 7fecae7fc640 1 -- 172.21.15.27:0/3888587064 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feca0032810 con 0x7fecb0145c30 2024-09-12T03:22:36.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:36.044+0000 7fecb5ca5640 1 -- 172.21.15.27:0/3888587064 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fecb0145c30 msgr2=0x7fecb015ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:36.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:36.044+0000 7fecb5ca5640 1 --2- 172.21.15.27:0/3888587064 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fecb0145c30 0x7fecb015ca00 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7fecb00b3f20 tx=0x7feca002a140 comp rx=0 tx=0).stop 2024-09-12T03:22:36.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:36.045+0000 7fecb5ca5640 1 -- 172.21.15.27:0/3888587064 shutdown_connections 2024-09-12T03:22:36.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:36.045+0000 7fecb5ca5640 1 --2- 172.21.15.27:0/3888587064 >> v2:172.21.15.45:3300/0 conn(0x7fecb00cd870 0x7fecb00b4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:36.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:36.045+0000 7fecb5ca5640 1 --2- 172.21.15.27:0/3888587064 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fecb0146010 0x7fecb00cd330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:36.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:36.045+0000 7fecb5ca5640 1 --2- 172.21.15.27:0/3888587064 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fecb0145c30 0x7fecb015ca00 unknown :-1 s=CLOSED pgs=1327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:36.048 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:36.045+0000 7fecb5ca5640 1 -- 172.21.15.27:0/3888587064 >> 172.21.15.27:0/3888587064 conn(0x7fecb0062c60 msgr2=0x7fecb006bb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:36.048 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:36.045+0000 7fecb5ca5640 1 -- 172.21.15.27:0/3888587064 shutdown_connections 2024-09-12T03:22:36.048 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:36.045+0000 7fecb5ca5640 1 -- 172.21.15.27:0/3888587064 wait complete. 2024-09-12T03:22:36.048 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:36.046+0000 7fecb5ca5640 1 Processor -- start 2024-09-12T03:22:36.048 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:36.046+0000 7fecb5ca5640 1 -- start start 2024-09-12T03:22:36.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:36.047+0000 7fecb5ca5640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fecb00cd870 0x7fecb00ae890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:36.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:36.047+0000 7fecb5ca5640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fecb0145c30 0x7fecb0141ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:36.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:36.047+0000 7fecb5ca5640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fecb0146010 0x7fecb0141ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:36.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:36.047+0000 7fecb5ca5640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fecb00b4700 con 0x7fecb0146010 2024-09-12T03:22:36.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:36.047+0000 7fecb5ca5640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fecb00b4580 con 0x7fecb00cd870 2024-09-12T03:22:36.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:36.047+0000 7fecb5ca5640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fecb00b4880 con 0x7fecb0145c30 2024-09-12T03:22:36.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:36.047+0000 7fecaffff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fecb0146010 0x7fecb0141ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:36.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:36.047+0000 7fecaffff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fecb0146010 0x7fecb0141ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55120/0 (socket says 172.21.15.27:55120) 2024-09-12T03:22:36.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:36.047+0000 7fecaffff640 1 -- 172.21.15.27:0/713642578 learned_addr learned my addr 172.21.15.27:0/713642578 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:22:36.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:36.047+0000 7fecaeffd640 1 --2- 172.21.15.27:0/713642578 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fecb0145c30 0x7fecb0141ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:36.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:36.047+0000 7fecaf7fe640 1 --2- 172.21.15.27:0/713642578 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fecb00cd870 0x7fecb00ae890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:36.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:36.047+0000 7fecaffff640 1 -- 172.21.15.27:0/713642578 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fecb0145c30 msgr2=0x7fecb0141ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:36.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:36.047+0000 7fecaffff640 1 --2- 172.21.15.27:0/713642578 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fecb0145c30 0x7fecb0141ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:36.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:36.047+0000 7fecaffff640 1 -- 172.21.15.27:0/713642578 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fecb00cd870 msgr2=0x7fecb00ae890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:36.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:36.047+0000 7fecaffff640 1 --2- 172.21.15.27:0/713642578 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fecb00cd870 0x7fecb00ae890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:36.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:36.047+0000 7fecaffff640 1 -- 172.21.15.27:0/713642578 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecb00c5570 con 0x7fecb0146010 2024-09-12T03:22:36.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:36.047+0000 7fecaf7fe640 1 --2- 172.21.15.27:0/713642578 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fecb00cd870 0x7fecb00ae890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:22:36.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:36.047+0000 7fecaffff640 1 --2- 172.21.15.27:0/713642578 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fecb0146010 0x7fecb0141ff0 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7feca800cd70 tx=0x7feca800a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:36.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:36.047+0000 7fec97fff640 1 -- 172.21.15.27:0/713642578 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feca8003b90 con 0x7fecb0146010 2024-09-12T03:22:36.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:36.047+0000 7fec97fff640 1 -- 172.21.15.27:0/713642578 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feca8003d30 con 0x7fecb0146010 2024-09-12T03:22:36.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:36.047+0000 7fecb5ca5640 1 -- 172.21.15.27:0/713642578 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fecb00c5810 con 0x7fecb0146010 2024-09-12T03:22:36.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:36.047+0000 7fec97fff640 1 -- 172.21.15.27:0/713642578 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feca8004240 con 0x7fecb0146010 2024-09-12T03:22:36.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:36.047+0000 7fecb5ca5640 1 -- 172.21.15.27:0/713642578 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fecb00c60b0 con 0x7fecb0146010 2024-09-12T03:22:36.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:36.049+0000 7fecb5ca5640 1 -- 172.21.15.27:0/713642578 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fecb00c59e0 con 0x7fecb0146010 2024-09-12T03:22:36.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:36.049+0000 7fec97fff640 1 -- 172.21.15.27:0/713642578 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 22) ==== 91059+0+0 (secure 0 0 0) 0x7feca80043e0 con 0x7fecb0146010 2024-09-12T03:22:36.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:36.049+0000 7fec97fff640 1 --2- 172.21.15.27:0/713642578 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fec9c06c460 0x7fec9c06e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:36.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:36.050+0000 7fecaf7fe640 1 --2- 172.21.15.27:0/713642578 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fec9c06c460 0x7fec9c06e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:36.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:36.050+0000 7fec97fff640 1 -- 172.21.15.27:0/713642578 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 5765+0+0 (secure 0 0 0) 0x7feca808c560 con 0x7fecb0146010 2024-09-12T03:22:36.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:36.050+0000 7fecaf7fe640 1 --2- 172.21.15.27:0/713642578 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fec9c06c460 0x7fec9c06e920 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7feca0034000 tx=0x7feca002e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:36.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:36.054+0000 7fec97fff640 1 -- 172.21.15.27:0/713642578 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7feca805a510 con 0x7fecb0146010 2024-09-12T03:22:36.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:36.244+0000 7fecb5ca5640 1 -- 172.21.15.27:0/713642578 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fecb015ca00 con 0x7fecb0146010 2024-09-12T03:22:36.311 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:36.244+0000 7fec97fff640 1 -- 172.21.15.27:0/713642578 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v126) ==== 80+0+0 (secure 0 0 0) 0x7feca8059e10 con 0x7fecb0146010 2024-09-12T03:22:36.312 INFO:teuthology.orchestra.run.smithi027.stderr:listed 0 entries 2024-09-12T03:22:36.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:36.246+0000 7fecb5ca5640 1 -- 172.21.15.27:0/713642578 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fec9c06c460 msgr2=0x7fec9c06e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:36.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:36.246+0000 7fecb5ca5640 1 --2- 172.21.15.27:0/713642578 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fec9c06c460 0x7fec9c06e920 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7feca0034000 tx=0x7feca002e000 comp rx=0 tx=0).stop 2024-09-12T03:22:36.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:36.246+0000 7fecb5ca5640 1 -- 172.21.15.27:0/713642578 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fecb0146010 msgr2=0x7fecb0141ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:36.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:36.246+0000 7fecb5ca5640 1 --2- 172.21.15.27:0/713642578 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fecb0146010 0x7fecb0141ff0 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7feca800cd70 tx=0x7feca800a660 comp rx=0 tx=0).stop 2024-09-12T03:22:36.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:36.246+0000 7fecb5ca5640 1 -- 172.21.15.27:0/713642578 shutdown_connections 2024-09-12T03:22:36.314 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:36.246+0000 7fecb5ca5640 1 --2- 172.21.15.27:0/713642578 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fec9c06c460 0x7fec9c06e920 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:36.314 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:36.246+0000 7fecb5ca5640 1 --2- 172.21.15.27:0/713642578 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fecb0146010 0x7fecb0141ff0 unknown :-1 s=CLOSED pgs=1328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:36.314 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:36.246+0000 7fecb5ca5640 1 --2- 172.21.15.27:0/713642578 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fecb0145c30 0x7fecb0141ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:36.314 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:36.246+0000 7fecb5ca5640 1 --2- 172.21.15.27:0/713642578 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fecb00cd870 0x7fecb00ae890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:36.314 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:36.246+0000 7fecb5ca5640 1 -- 172.21.15.27:0/713642578 >> 172.21.15.27:0/713642578 conn(0x7fecb0062c60 msgr2=0x7fecb00aaf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:36.314 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:36.247+0000 7fecb5ca5640 1 -- 172.21.15.27:0/713642578 shutdown_connections 2024-09-12T03:22:36.314 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:36.247+0000 7fecb5ca5640 1 -- 172.21.15.27:0/713642578 wait complete. 2024-09-12T03:22:36.314 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-12T03:22:36.314 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:22:36.314 DEBUG:teuthology.orchestra.run.smithi027:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-12T03:22:36.345 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi027.front.sepia.ceph.com 2024-09-12T03:22:36.345 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:22:36.345 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-12T03:22:36.402 INFO:tasks.cephfs.mount:Terminating background process 2024-09-12T03:22:36.402 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-12T03:22:36.402 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T03:22:36.422 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-12T03:22:36.422 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T03:22:36.483 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-12T03:22:36.586 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:36.585+0000 7f1e49f03640 1 Processor -- start 2024-09-12T03:22:36.586 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:36.585+0000 7f1e49f03640 1 -- start start 2024-09-12T03:22:36.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:36.586+0000 7f1e49f03640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f1e440b69b0 0x7f1e440d2360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:36.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:36.586+0000 7f1e49f03640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1e440b5f00 con 0x7f1e440d28a0 2024-09-12T03:22:36.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:36.586+0000 7f1e49f03640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1e440b61e0 con 0x7f1e4415c460 2024-09-12T03:22:36.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:36.586+0000 7f1e49f03640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f1e440d2d10 con 0x7f1e440b69b0 2024-09-12T03:22:36.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:36.586+0000 7f1e43fff640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f1e440d28a0 0x7f1e440d0b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:38646/0 (socket says 172.21.15.27:38646) 2024-09-12T03:22:36.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:36.586+0000 7f1e43fff640 1 -- 172.21.15.27:0/2330620996 learned_addr learned my addr 172.21.15.27:0/2330620996 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:22:36.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:36.587+0000 7f1e42ffd640 1 --2- 172.21.15.27:0/2330620996 >> v2:172.21.15.45:3300/0 conn(0x7f1e440b69b0 0x7f1e440d2360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:36.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:36.587+0000 7f1e427fc640 1 -- 172.21.15.27:0/2330620996 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 127828066 0 0) 0x7f1e440b5f00 con 0x7f1e440d28a0 2024-09-12T03:22:36.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:36.587+0000 7f1e427fc640 1 -- 172.21.15.27:0/2330620996 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1e30003520 con 0x7f1e440d28a0 2024-09-12T03:22:36.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:36.587+0000 7f1e42ffd640 1 -- 172.21.15.27:0/2330620996 >> v1:172.21.15.45:6789/0 conn(0x7f1e4415c460 legacy=0x7f1e440b55b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:36.589 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:36.587+0000 7f1e42ffd640 1 -- 172.21.15.27:0/2330620996 >> v1:172.21.15.27:6789/0 conn(0x7f1e440d28a0 legacy=0x7f1e440d0b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:36.589 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:36.587+0000 7f1e42ffd640 1 -- 172.21.15.27:0/2330620996 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e440d0300 con 0x7f1e440b69b0 2024-09-12T03:22:36.590 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:36.588+0000 7f1e42ffd640 1 --2- 172.21.15.27:0/2330620996 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1e440b69b0 0x7f1e440d2360 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f1e2c0097a0 tx=0x7f1e2c02a230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4c66b479a40bac96 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:36.590 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:36.588+0000 7f1e427fc640 1 -- 172.21.15.27:0/2330620996 <== mon.1 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e2c037070 con 0x7f1e440b69b0 2024-09-12T03:22:36.590 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:36.588+0000 7f1e427fc640 1 -- 172.21.15.27:0/2330620996 <== mon.1 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e2c032440 con 0x7f1e440b69b0 2024-09-12T03:22:36.590 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:36.588+0000 7f1e427fc640 1 -- 172.21.15.27:0/2330620996 <== mon.1 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e2c0327f0 con 0x7f1e440b69b0 2024-09-12T03:22:36.590 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:36.588+0000 7f1e49f03640 1 -- 172.21.15.27:0/2330620996 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1e440b69b0 msgr2=0x7f1e440d2360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:36.590 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:36.588+0000 7f1e49f03640 1 --2- 172.21.15.27:0/2330620996 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1e440b69b0 0x7f1e440d2360 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f1e2c0097a0 tx=0x7f1e2c02a230 comp rx=0 tx=0).stop 2024-09-12T03:22:36.590 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:36.588+0000 7f1e49f03640 1 -- 172.21.15.27:0/2330620996 shutdown_connections 2024-09-12T03:22:36.590 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:36.588+0000 7f1e49f03640 1 --2- 172.21.15.27:0/2330620996 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1e440b69b0 0x7f1e440d2360 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:36.590 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:36.589+0000 7f1e49f03640 1 -- 172.21.15.27:0/2330620996 >> 172.21.15.27:0/2330620996 conn(0x7f1e4406f320 msgr2=0x7f1e4406f700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:36.590 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:36.589+0000 7f1e49f03640 1 -- 172.21.15.27:0/2330620996 shutdown_connections 2024-09-12T03:22:36.590 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:36.589+0000 7f1e49f03640 1 -- 172.21.15.27:0/2330620996 wait complete. 2024-09-12T03:22:36.592 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:36.590+0000 7f1e49f03640 1 Processor -- start 2024-09-12T03:22:36.593 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:36.590+0000 7f1e49f03640 1 -- start start 2024-09-12T03:22:36.593 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:36.591+0000 7f1e49f03640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1e440b69b0 0x7f1e44140aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:36.593 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:36.591+0000 7f1e49f03640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1e440d28a0 0x7f1e44158130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:36.593 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:36.591+0000 7f1e49f03640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1e4415c460 0x7f1e44158670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:36.593 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:36.591+0000 7f1e49f03640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f1e440b5cc0 con 0x7f1e440d28a0 2024-09-12T03:22:36.593 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:36.591+0000 7f1e49f03640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f1e440b64c0 con 0x7f1e4415c460 2024-09-12T03:22:36.593 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:36.591+0000 7f1e49f03640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f1e440d0670 con 0x7f1e440b69b0 2024-09-12T03:22:36.593 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:36.591+0000 7f1e42ffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1e440d28a0 0x7f1e44158130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:36.593 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:36.591+0000 7f1e42ffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1e440d28a0 0x7f1e44158130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55124/0 (socket says 172.21.15.27:55124) 2024-09-12T03:22:36.593 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:36.591+0000 7f1e42ffd640 1 -- 172.21.15.27:0/709256474 learned_addr learned my addr 172.21.15.27:0/709256474 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:22:36.593 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:36.591+0000 7f1e43fff640 1 --2- 172.21.15.27:0/709256474 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1e4415c460 0x7f1e44158670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:36.593 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:36.591+0000 7f1e437fe640 1 --2- 172.21.15.27:0/709256474 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1e440b69b0 0x7f1e44140aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:36.593 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:36.591+0000 7f1e42ffd640 1 -- 172.21.15.27:0/709256474 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1e440b69b0 msgr2=0x7f1e44140aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:36.594 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:36.591+0000 7f1e42ffd640 1 --2- 172.21.15.27:0/709256474 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1e440b69b0 0x7f1e44140aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:36.594 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:36.591+0000 7f1e42ffd640 1 -- 172.21.15.27:0/709256474 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1e4415c460 msgr2=0x7f1e44158670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:36.594 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:36.591+0000 7f1e42ffd640 1 --2- 172.21.15.27:0/709256474 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1e4415c460 0x7f1e44158670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:36.594 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:36.591+0000 7f1e42ffd640 1 -- 172.21.15.27:0/709256474 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e441e4f80 con 0x7f1e440d28a0 2024-09-12T03:22:36.594 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:36.591+0000 7f1e437fe640 1 --2- 172.21.15.27:0/709256474 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1e440b69b0 0x7f1e44140aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:22:36.594 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:36.591+0000 7f1e43fff640 1 --2- 172.21.15.27:0/709256474 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1e4415c460 0x7f1e44158670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:22:36.594 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:36.591+0000 7f1e42ffd640 1 --2- 172.21.15.27:0/709256474 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1e440d28a0 0x7f1e44158130 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7f1e2c0097a0 tx=0x7f1e2c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:36.594 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:36.591+0000 7f1e3bfff640 1 -- 172.21.15.27:0/709256474 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e2c037050 con 0x7f1e440d28a0 2024-09-12T03:22:36.594 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:36.591+0000 7f1e3bfff640 1 -- 172.21.15.27:0/709256474 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e2c038040 con 0x7f1e440d28a0 2024-09-12T03:22:36.594 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:36.592+0000 7f1e3bfff640 1 -- 172.21.15.27:0/709256474 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e2c004470 con 0x7f1e440d28a0 2024-09-12T03:22:36.594 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:36.592+0000 7f1e49f03640 1 -- 172.21.15.27:0/709256474 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e441411b0 con 0x7f1e440d28a0 2024-09-12T03:22:36.594 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:36.592+0000 7f1e49f03640 1 -- 172.21.15.27:0/709256474 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e44141770 con 0x7f1e440d28a0 2024-09-12T03:22:36.595 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:36.593+0000 7f1e49f03640 1 -- 172.21.15.27:0/709256474 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e440b5ab0 con 0x7f1e440d28a0 2024-09-12T03:22:36.595 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:36.593+0000 7f1e3bfff640 1 -- 172.21.15.27:0/709256474 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 22) ==== 91059+0+0 (secure 0 0 0) 0x7f1e2c03c6d0 con 0x7f1e440d28a0 2024-09-12T03:22:36.595 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:36.593+0000 7f1e3bfff640 1 --2- 172.21.15.27:0/709256474 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f1e2406c460 0x7f1e2406e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:36.595 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:36.594+0000 7f1e3bfff640 1 -- 172.21.15.27:0/709256474 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 5765+0+0 (secure 0 0 0) 0x7f1e2c0ac430 con 0x7f1e440d28a0 2024-09-12T03:22:36.596 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:36.594+0000 7f1e437fe640 1 --2- 172.21.15.27:0/709256474 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f1e2406c460 0x7f1e2406e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:36.596 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:36.594+0000 7f1e437fe640 1 --2- 172.21.15.27:0/709256474 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f1e2406c460 0x7f1e2406e920 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f1e440cea40 tx=0x7f1e34009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:36.598 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:36.598+0000 7f1e3bfff640 1 -- 172.21.15.27:0/709256474 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f1e440b5ab0 con 0x7f1e440d28a0 2024-09-12T03:22:36.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:36.784+0000 7f1e49f03640 1 -- 172.21.15.27:0/709256474 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f1e440d2360 con 0x7f1e440d28a0 2024-09-12T03:22:36.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:36.785+0000 7f1e3bfff640 1 -- 172.21.15.27:0/709256474 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v126) ==== 80+0+0 (secure 0 0 0) 0x7f1e2c07a460 con 0x7f1e440d28a0 2024-09-12T03:22:36.786 INFO:teuthology.orchestra.run.smithi027.stderr:listed 0 entries 2024-09-12T03:22:36.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:36.787+0000 7f1e49f03640 1 -- 172.21.15.27:0/709256474 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f1e2406c460 msgr2=0x7f1e2406e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:36.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:36.787+0000 7f1e49f03640 1 --2- 172.21.15.27:0/709256474 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f1e2406c460 0x7f1e2406e920 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f1e440cea40 tx=0x7f1e34009040 comp rx=0 tx=0).stop 2024-09-12T03:22:36.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:36.787+0000 7f1e49f03640 1 -- 172.21.15.27:0/709256474 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1e440d28a0 msgr2=0x7f1e44158130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:36.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:36.787+0000 7f1e49f03640 1 --2- 172.21.15.27:0/709256474 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1e440d28a0 0x7f1e44158130 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7f1e2c0097a0 tx=0x7f1e2c0046c0 comp rx=0 tx=0).stop 2024-09-12T03:22:36.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:36.787+0000 7f1e49f03640 1 -- 172.21.15.27:0/709256474 shutdown_connections 2024-09-12T03:22:36.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:36.787+0000 7f1e49f03640 1 --2- 172.21.15.27:0/709256474 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1e4415c460 0x7f1e44158670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:36.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:36.787+0000 7f1e49f03640 1 --2- 172.21.15.27:0/709256474 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f1e2406c460 0x7f1e2406e920 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:36.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:36.787+0000 7f1e49f03640 1 --2- 172.21.15.27:0/709256474 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1e440d28a0 0x7f1e44158130 unknown :-1 s=CLOSED pgs=1330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:36.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:36.787+0000 7f1e49f03640 1 --2- 172.21.15.27:0/709256474 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1e440b69b0 0x7f1e44140aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:36.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:36.787+0000 7f1e49f03640 1 -- 172.21.15.27:0/709256474 >> 172.21.15.27:0/709256474 conn(0x7f1e4406f320 msgr2=0x7f1e4415f130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:36.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:36.787+0000 7f1e49f03640 1 -- 172.21.15.27:0/709256474 shutdown_connections 2024-09-12T03:22:36.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:36.787+0000 7f1e49f03640 1 -- 172.21.15.27:0/709256474 wait complete. 2024-09-12T03:22:36.800 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-09-12T03:22:36.800 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:22:36.800 DEBUG:teuthology.orchestra.run.smithi045:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-09-12T03:22:36.835 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi045.front.sepia.ceph.com 2024-09-12T03:22:36.835 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:22:36.835 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-12T03:22:36.894 INFO:tasks.cephfs.mount:Terminating background process 2024-09-12T03:22:36.894 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-12T03:22:36.894 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T03:22:36.911 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056004,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497432k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16324 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248716k,nr_inodes=812179,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497432k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-12T03:22:36.911 INFO:tasks.cephfs.mount:Terminating background process 2024-09-12T03:22:36.911 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-12T03:22:36.911 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T03:22:36.953 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056795,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16602 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249348k,nr_inodes=812337,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-12T03:22:36.953 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-12T03:22:37.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.054+0000 7fad40168640 1 Processor -- start 2024-09-12T03:22:37.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.054+0000 7fad40168640 1 -- start start 2024-09-12T03:22:37.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.055+0000 7fad40168640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fad381590d0 0x7fad380b55d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:37.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.055+0000 7fad40168640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fad38162c10 con 0x7fad3815f600 2024-09-12T03:22:37.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.055+0000 7fad40168640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fad38162ef0 con 0x7fad38158cf0 2024-09-12T03:22:37.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.055+0000 7fad40168640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fad381631d0 con 0x7fad381590d0 2024-09-12T03:22:37.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.055+0000 7fad3e705640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fad3815f600 0x7fad3806bb30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:38654/0 (socket says 172.21.15.27:38654) 2024-09-12T03:22:37.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.055+0000 7fad3e705640 1 -- 172.21.15.27:0/658079747 learned_addr learned my addr 172.21.15.27:0/658079747 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:22:37.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.055+0000 7fad3d703640 1 --2- 172.21.15.27:0/658079747 >> v2:172.21.15.45:3300/0 conn(0x7fad381590d0 0x7fad380b55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:37.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.055+0000 7fad3cf02640 1 -- 172.21.15.27:0/658079747 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1372458095 0 0) 0x7fad38162c10 con 0x7fad3815f600 2024-09-12T03:22:37.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.055+0000 7fad3cf02640 1 -- 172.21.15.27:0/658079747 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fad20003520 con 0x7fad3815f600 2024-09-12T03:22:37.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.056+0000 7fad3cf02640 1 -- 172.21.15.27:0/658079747 <== mon.1 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 37961023 0 0) 0x7fad38162ef0 con 0x7fad38158cf0 2024-09-12T03:22:37.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.056+0000 7fad3cf02640 1 -- 172.21.15.27:0/658079747 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fad38162c10 con 0x7fad38158cf0 2024-09-12T03:22:37.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.056+0000 7fad3cf02640 1 -- 172.21.15.27:0/658079747 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1865399669 0 0) 0x7fad20003520 con 0x7fad3815f600 2024-09-12T03:22:37.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.056+0000 7fad3d703640 1 -- 172.21.15.27:0/658079747 >> v1:172.21.15.45:6789/0 conn(0x7fad38158cf0 legacy=0x7fad38162430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:37.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.056+0000 7fad3d703640 1 -- 172.21.15.27:0/658079747 >> v1:172.21.15.27:6789/0 conn(0x7fad3815f600 legacy=0x7fad3806bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:37.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.056+0000 7fad3d703640 1 -- 172.21.15.27:0/658079747 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad3815edc0 con 0x7fad381590d0 2024-09-12T03:22:37.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.057+0000 7fad3d703640 1 --2- 172.21.15.27:0/658079747 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fad381590d0 0x7fad380b55d0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fad2c0097a0 tx=0x7fad2c02a0a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5f97fece6954bbd8 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:37.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.057+0000 7fad3cf02640 1 -- 172.21.15.27:0/658079747 <== mon.1 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad2c037070 con 0x7fad381590d0 2024-09-12T03:22:37.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.057+0000 7fad3cf02640 1 -- 172.21.15.27:0/658079747 <== mon.1 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad2c02ab30 con 0x7fad381590d0 2024-09-12T03:22:37.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.057+0000 7fad3cf02640 1 -- 172.21.15.27:0/658079747 <== mon.1 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad30001670 con 0x7fad381590d0 2024-09-12T03:22:37.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.057+0000 7fad40168640 1 -- 172.21.15.27:0/658079747 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fad381590d0 msgr2=0x7fad380b55d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:37.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.057+0000 7fad40168640 1 --2- 172.21.15.27:0/658079747 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fad381590d0 0x7fad380b55d0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fad2c0097a0 tx=0x7fad2c02a0a0 comp rx=0 tx=0).stop 2024-09-12T03:22:37.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.057+0000 7fad40168640 1 -- 172.21.15.27:0/658079747 shutdown_connections 2024-09-12T03:22:37.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.057+0000 7fad40168640 1 --2- 172.21.15.27:0/658079747 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fad381590d0 0x7fad380b55d0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:37.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.057+0000 7fad40168640 1 -- 172.21.15.27:0/658079747 >> 172.21.15.27:0/658079747 conn(0x7fad3806cc40 msgr2=0x7fad3806d050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:37.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.057+0000 7fad40168640 1 -- 172.21.15.27:0/658079747 shutdown_connections 2024-09-12T03:22:37.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.057+0000 7fad40168640 1 -- 172.21.15.27:0/658079747 wait complete. 2024-09-12T03:22:37.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.058+0000 7fad40168640 1 Processor -- start 2024-09-12T03:22:37.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.059+0000 7fad40168640 1 -- start start 2024-09-12T03:22:37.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.059+0000 7fad40168640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fad38158cf0 0x7fad38157860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:37.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.059+0000 7fad40168640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fad381590d0 0x7fad3815c090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:37.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.059+0000 7fad40168640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fad3815f600 0x7fad38151810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:37.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.059+0000 7fad40168640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fad3815fa50 con 0x7fad38158cf0 2024-09-12T03:22:37.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.059+0000 7fad40168640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fad3806c490 con 0x7fad381590d0 2024-09-12T03:22:37.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.059+0000 7fad40168640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fad3815fbd0 con 0x7fad3815f600 2024-09-12T03:22:37.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.059+0000 7fad3df04640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fad38158cf0 0x7fad38157860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:37.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.059+0000 7fad3df04640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fad38158cf0 0x7fad38157860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55132/0 (socket says 172.21.15.27:55132) 2024-09-12T03:22:37.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.059+0000 7fad3df04640 1 -- 172.21.15.27:0/1749298729 learned_addr learned my addr 172.21.15.27:0/1749298729 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:22:37.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.060+0000 7fad3d703640 1 --2- 172.21.15.27:0/1749298729 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fad381590d0 0x7fad3815c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:37.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.060+0000 7fad3e705640 1 --2- 172.21.15.27:0/1749298729 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fad3815f600 0x7fad38151810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:37.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.060+0000 7fad3df04640 1 -- 172.21.15.27:0/1749298729 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fad3815f600 msgr2=0x7fad38151810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:37.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.060+0000 7fad3df04640 1 --2- 172.21.15.27:0/1749298729 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fad3815f600 0x7fad38151810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:37.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.060+0000 7fad3df04640 1 -- 172.21.15.27:0/1749298729 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fad381590d0 msgr2=0x7fad3815c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:37.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.060+0000 7fad3df04640 1 --2- 172.21.15.27:0/1749298729 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fad381590d0 0x7fad3815c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:37.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.060+0000 7fad3df04640 1 -- 172.21.15.27:0/1749298729 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad3815c5d0 con 0x7fad38158cf0 2024-09-12T03:22:37.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.060+0000 7fad3df04640 1 --2- 172.21.15.27:0/1749298729 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fad38158cf0 0x7fad38157860 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7fad2800c7a0 tx=0x7fad2800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:37.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.060+0000 7fad367fc640 1 -- 172.21.15.27:0/1749298729 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad28003b70 con 0x7fad38158cf0 2024-09-12T03:22:37.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.060+0000 7fad367fc640 1 -- 172.21.15.27:0/1749298729 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad28003d10 con 0x7fad38158cf0 2024-09-12T03:22:37.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.060+0000 7fad40168640 1 -- 172.21.15.27:0/1749298729 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad38151d50 con 0x7fad38158cf0 2024-09-12T03:22:37.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.060+0000 7fad367fc640 1 -- 172.21.15.27:0/1749298729 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad2800aa90 con 0x7fad38158cf0 2024-09-12T03:22:37.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.060+0000 7fad40168640 1 -- 172.21.15.27:0/1749298729 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad3815b1f0 con 0x7fad38158cf0 2024-09-12T03:22:37.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.061+0000 7fad40168640 1 -- 172.21.15.27:0/1749298729 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad381e82e0 con 0x7fad38158cf0 2024-09-12T03:22:37.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.061+0000 7fad367fc640 1 -- 172.21.15.27:0/1749298729 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 22) ==== 91059+0+0 (secure 0 0 0) 0x7fad2800ac30 con 0x7fad38158cf0 2024-09-12T03:22:37.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.062+0000 7fad367fc640 1 --2- 172.21.15.27:0/1749298729 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fad2406c490 0x7fad2406e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:37.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.062+0000 7fad367fc640 1 -- 172.21.15.27:0/1749298729 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 5765+0+0 (secure 0 0 0) 0x7fad2808b8a0 con 0x7fad38158cf0 2024-09-12T03:22:37.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.062+0000 7fad3d703640 1 --2- 172.21.15.27:0/1749298729 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fad2406c490 0x7fad2406e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:37.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.063+0000 7fad3d703640 1 --2- 172.21.15.27:0/1749298729 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fad2406c490 0x7fad2406e950 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fad2c034000 tx=0x7fad2c02e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:37.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.066+0000 7fad367fc640 1 -- 172.21.15.27:0/1749298729 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fad28059960 con 0x7fad38158cf0 2024-09-12T03:22:37.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.278+0000 7fad40168640 1 -- 172.21.15.27:0/1749298729 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fad380b5550 con 0x7fad38158cf0 2024-09-12T03:22:37.280 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.279+0000 7fad367fc640 1 -- 172.21.15.27:0/1749298729 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) ==== 94+0+5227 (secure 0 0 0) 0x7fad2808b070 con 0x7fad38158cf0 2024-09-12T03:22:37.280 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:22:37.280 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":76,"btime":"2024-09-12T03:22:27:434130+0000","default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6437,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/2244842464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2244842464},{"type":"v1","addr":"172.21.15.45:6834","nonce":2244842464}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":72},{"gid":7048,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/1976058956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":1976058956},{"type":"v1","addr":"172.21.15.27:6835","nonce":1976058956}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":72},{"gid":7060,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/1084669125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":1084669125},{"type":"v1","addr":"172.21.15.45:6835","nonce":1084669125}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":72}],"filesystems":[{"mdsmap":{"epoch":75,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-12T03:22:18.755267+0000","modified":"2024-09-12T03:22:20.761064+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7725},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7725":{"gid":7725,"name":"c","rank":0,"incarnation":73,"state":"up:active","state_seq":3,"addr":"172.21.15.27:6834/1711260376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":1711260376},{"type":"v1","addr":"172.21.15.27:6834","nonce":1711260376}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7725,"qdb_cluster":[7725]},"id":10}]} 2024-09-12T03:22:37.281 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 76 2024-09-12T03:22:37.282 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.281+0000 7fad40168640 1 -- 172.21.15.27:0/1749298729 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fad2406c490 msgr2=0x7fad2406e950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:37.283 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.281+0000 7fad40168640 1 --2- 172.21.15.27:0/1749298729 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fad2406c490 0x7fad2406e950 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fad2c034000 tx=0x7fad2c02e000 comp rx=0 tx=0).stop 2024-09-12T03:22:37.283 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.281+0000 7fad40168640 1 -- 172.21.15.27:0/1749298729 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fad38158cf0 msgr2=0x7fad38157860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:37.283 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.281+0000 7fad40168640 1 --2- 172.21.15.27:0/1749298729 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fad38158cf0 0x7fad38157860 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7fad2800c7a0 tx=0x7fad2800cc60 comp rx=0 tx=0).stop 2024-09-12T03:22:37.283 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.281+0000 7fad40168640 1 -- 172.21.15.27:0/1749298729 shutdown_connections 2024-09-12T03:22:37.283 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.281+0000 7fad40168640 1 --2- 172.21.15.27:0/1749298729 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fad3815f600 0x7fad38151810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:37.283 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.281+0000 7fad40168640 1 --2- 172.21.15.27:0/1749298729 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fad381590d0 0x7fad3815c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:37.283 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.281+0000 7fad40168640 1 --2- 172.21.15.27:0/1749298729 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fad2406c490 0x7fad2406e950 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:37.283 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.281+0000 7fad40168640 1 --2- 172.21.15.27:0/1749298729 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fad38158cf0 0x7fad38157860 unknown :-1 s=CLOSED pgs=1332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:37.283 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.281+0000 7fad40168640 1 -- 172.21.15.27:0/1749298729 >> 172.21.15.27:0/1749298729 conn(0x7fad3806cc40 msgr2=0x7fad380aec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:37.283 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.281+0000 7fad40168640 1 -- 172.21.15.27:0/1749298729 shutdown_connections 2024-09-12T03:22:37.283 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.281+0000 7fad40168640 1 -- 172.21.15.27:0/1749298729 wait complete. 2024-09-12T03:22:37.294 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-12T03:22:37.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.395+0000 7f2d802f1640 1 Processor -- start 2024-09-12T03:22:37.397 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.395+0000 7f2d802f1640 1 -- start start 2024-09-12T03:22:37.408 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.396+0000 7f2d802f1640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f2d7806f0c0 0x7f2d780b3560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:37.408 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.396+0000 7f2d802f1640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d780ce410 con 0x7f2d7815b2b0 2024-09-12T03:22:37.408 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.396+0000 7f2d802f1640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d780732c0 con 0x7f2d78072d70 2024-09-12T03:22:37.408 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.396+0000 7f2d802f1640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f2d780720a0 con 0x7f2d7806f0c0 2024-09-12T03:22:37.409 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.396+0000 7f2d7e08d640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f2d7815b2b0 0x7f2d78072430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:38662/0 (socket says 172.21.15.27:38662) 2024-09-12T03:22:37.409 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.396+0000 7f2d7e08d640 1 -- 172.21.15.27:0/1382978886 learned_addr learned my addr 172.21.15.27:0/1382978886 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:22:37.409 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.397+0000 7f2d7d08b640 1 -- 172.21.15.27:0/1382978886 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 903237500 0 0) 0x7f2d70001110 con 0x7f2d7815b2b0 2024-09-12T03:22:37.409 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.397+0000 7f2d7e88e640 1 --2- 172.21.15.27:0/1382978886 >> v2:172.21.15.45:3300/0 conn(0x7f2d7806f0c0 0x7f2d780b3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:37.409 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.397+0000 7f2d7d08b640 1 -- 172.21.15.27:0/1382978886 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2d60003520 con 0x7f2d7815b2b0 2024-09-12T03:22:37.410 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.397+0000 7f2d7d08b640 1 -- 172.21.15.27:0/1382978886 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3006087095 0 0) 0x7f2d60003520 con 0x7f2d7815b2b0 2024-09-12T03:22:37.410 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.397+0000 7f2d7d08b640 1 -- 172.21.15.27:0/1382978886 >> v2:172.21.15.45:3300/0 conn(0x7f2d7806f0c0 msgr2=0x7f2d780b3560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:37.410 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.397+0000 7f2d7d08b640 1 --2- 172.21.15.27:0/1382978886 >> v2:172.21.15.45:3300/0 conn(0x7f2d7806f0c0 0x7f2d780b3560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:37.410 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.397+0000 7f2d7d08b640 1 -- 172.21.15.27:0/1382978886 >> v1:172.21.15.45:6789/0 conn(0x7f2d78072d70 legacy=0x7f2d7806e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:37.410 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.397+0000 7f2d7d08b640 1 -- 172.21.15.27:0/1382978886 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d78071ae0 con 0x7f2d7815b2b0 2024-09-12T03:22:37.410 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.397+0000 7f2d7d08b640 1 -- 172.21.15.27:0/1382978886 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 847755213 0 0) 0x7f2d70001840 con 0x7f2d7815b2b0 2024-09-12T03:22:37.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.397+0000 7f2d7d08b640 1 -- 172.21.15.27:0/1382978886 >> v1:172.21.15.27:6789/0 conn(0x7f2d7815b2b0 legacy=0x7f2d78072430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:37.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.397+0000 7f2d7d08b640 1 --2- 172.21.15.27:0/1382978886 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2d600039b0 0x7f2d60003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:37.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.397+0000 7f2d7d08b640 1 --2- 172.21.15.27:0/1382978886 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2d60004800 0x7f2d60004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:37.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.397+0000 7f2d7d08b640 1 --2- 172.21.15.27:0/1382978886 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2d600051c0 0x7f2d6000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:37.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.397+0000 7f2d7d08b640 1 -- 172.21.15.27:0/1382978886 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f2d780720a0 con 0x7f2d60004800 2024-09-12T03:22:37.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.397+0000 7f2d7d08b640 1 -- 172.21.15.27:0/1382978886 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f2d6000c060 con 0x7f2d600039b0 2024-09-12T03:22:37.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.397+0000 7f2d7d08b640 1 -- 172.21.15.27:0/1382978886 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f2d6000c260 con 0x7f2d600051c0 2024-09-12T03:22:37.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.397+0000 7f2d7d88c640 1 --2- 172.21.15.27:0/1382978886 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2d60004800 0x7f2d60004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:37.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.397+0000 7f2d7e08d640 1 --2- 172.21.15.27:0/1382978886 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2d600039b0 0x7f2d60003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:37.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.397+0000 7f2d7e88e640 1 --2- 172.21.15.27:0/1382978886 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2d600051c0 0x7f2d6000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:37.413 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.397+0000 7f2d7d88c640 1 -- 172.21.15.27:0/1382978886 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2d600051c0 msgr2=0x7f2d6000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:37.413 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.397+0000 7f2d7d88c640 1 --2- 172.21.15.27:0/1382978886 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2d600051c0 0x7f2d6000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:37.413 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.397+0000 7f2d7d88c640 1 -- 172.21.15.27:0/1382978886 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2d600039b0 msgr2=0x7f2d60003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:37.413 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.397+0000 7f2d7d88c640 1 --2- 172.21.15.27:0/1382978886 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2d600039b0 0x7f2d60003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:37.413 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.397+0000 7f2d7d88c640 1 -- 172.21.15.27:0/1382978886 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d60003520 con 0x7f2d60004800 2024-09-12T03:22:37.413 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.397+0000 7f2d7e88e640 1 -- 172.21.15.27:0/1382978886 reap_dead start 2024-09-12T03:22:37.414 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.398+0000 7f2d7d88c640 1 --2- 172.21.15.27:0/1382978886 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2d60004800 0x7f2d60004c80 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f2d6c001750 tx=0x7f2d60005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=901780ac2da59510 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:37.414 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.398+0000 7f2d7d08b640 1 -- 172.21.15.27:0/1382978886 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d6c001370 con 0x7f2d60004800 2024-09-12T03:22:37.414 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.398+0000 7f2d7d08b640 1 -- 172.21.15.27:0/1382978886 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d6c0052d0 con 0x7f2d60004800 2024-09-12T03:22:37.414 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.398+0000 7f2d7d08b640 1 -- 172.21.15.27:0/1382978886 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d6c02abc0 con 0x7f2d60004800 2024-09-12T03:22:37.414 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.399+0000 7f2d802f1640 1 -- 172.21.15.27:0/1382978886 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2d60004800 msgr2=0x7f2d60004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:37.414 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.399+0000 7f2d802f1640 1 --2- 172.21.15.27:0/1382978886 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2d60004800 0x7f2d60004c80 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f2d6c001750 tx=0x7f2d60005830 comp rx=0 tx=0).stop 2024-09-12T03:22:37.414 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.399+0000 7f2d802f1640 1 -- 172.21.15.27:0/1382978886 shutdown_connections 2024-09-12T03:22:37.414 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.399+0000 7f2d802f1640 1 --2- 172.21.15.27:0/1382978886 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2d60004800 0x7f2d60004c80 unknown :-1 s=CLOSED pgs=1334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:37.414 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.399+0000 7f2d802f1640 1 -- 172.21.15.27:0/1382978886 >> 172.21.15.27:0/1382978886 conn(0x7f2d78062c60 msgr2=0x7f2d7806bb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:37.414 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.399+0000 7f2d802f1640 1 -- 172.21.15.27:0/1382978886 shutdown_connections 2024-09-12T03:22:37.415 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.399+0000 7f2d802f1640 1 -- 172.21.15.27:0/1382978886 wait complete. 2024-09-12T03:22:37.415 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.400+0000 7f2d802f1640 1 Processor -- start 2024-09-12T03:22:37.415 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.400+0000 7f2d802f1640 1 -- start start 2024-09-12T03:22:37.415 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.401+0000 7f2d802f1640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2d60004800 0x7f2d78142e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:37.415 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.401+0000 7f2d802f1640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2d780b53b0 0x7f2d7815eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:37.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.401+0000 7f2d802f1640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2d780b5810 0x7f2d7815f400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:37.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.401+0000 7f2d802f1640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f2d7815b690 con 0x7f2d780b5810 2024-09-12T03:22:37.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.401+0000 7f2d802f1640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f2d780ce690 con 0x7f2d60004800 2024-09-12T03:22:37.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.401+0000 7f2d802f1640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f2d7815b810 con 0x7f2d780b53b0 2024-09-12T03:22:37.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.401+0000 7f2d7e88e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2d780b5810 0x7f2d7815f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:37.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.401+0000 7f2d7e88e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2d780b5810 0x7f2d7815f400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55144/0 (socket says 172.21.15.27:55144) 2024-09-12T03:22:37.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.401+0000 7f2d7e88e640 1 -- 172.21.15.27:0/1078188739 learned_addr learned my addr 172.21.15.27:0/1078188739 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:22:37.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.401+0000 7f2d7e08d640 1 --2- 172.21.15.27:0/1078188739 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2d60004800 0x7f2d78142e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:37.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.401+0000 7f2d7d88c640 1 --2- 172.21.15.27:0/1078188739 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2d780b53b0 0x7f2d7815eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:37.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.401+0000 7f2d7e88e640 1 -- 172.21.15.27:0/1078188739 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2d780b53b0 msgr2=0x7f2d7815eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:37.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.401+0000 7f2d7e88e640 1 --2- 172.21.15.27:0/1078188739 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2d780b53b0 0x7f2d7815eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:37.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.401+0000 7f2d7e88e640 1 -- 172.21.15.27:0/1078188739 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2d60004800 msgr2=0x7f2d78142e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:37.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.401+0000 7f2d7e88e640 1 --2- 172.21.15.27:0/1078188739 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2d60004800 0x7f2d78142e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:37.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.401+0000 7f2d7e88e640 1 -- 172.21.15.27:0/1078188739 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d7815bb90 con 0x7f2d780b5810 2024-09-12T03:22:37.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.402+0000 7f2d7e88e640 1 --2- 172.21.15.27:0/1078188739 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2d780b5810 0x7f2d7815f400 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7f2d68007cf0 tx=0x7f2d680115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:37.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.402+0000 7f2d76ffd640 1 -- 172.21.15.27:0/1078188739 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d68011e40 con 0x7f2d780b5810 2024-09-12T03:22:37.419 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.402+0000 7f2d76ffd640 1 -- 172.21.15.27:0/1078188739 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d6801b070 con 0x7f2d780b5810 2024-09-12T03:22:37.419 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.402+0000 7f2d802f1640 1 -- 172.21.15.27:0/1078188739 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d7815be70 con 0x7f2d780b5810 2024-09-12T03:22:37.419 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.402+0000 7f2d76ffd640 1 -- 172.21.15.27:0/1078188739 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d68003990 con 0x7f2d780b5810 2024-09-12T03:22:37.419 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.402+0000 7f2d802f1640 1 -- 172.21.15.27:0/1078188739 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d7815c430 con 0x7f2d780b5810 2024-09-12T03:22:37.419 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.403+0000 7f2d802f1640 1 -- 172.21.15.27:0/1078188739 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d780b46e0 con 0x7f2d780b5810 2024-09-12T03:22:37.420 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.403+0000 7f2d76ffd640 1 -- 172.21.15.27:0/1078188739 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 22) ==== 91059+0+0 (secure 0 0 0) 0x7f2d68003b30 con 0x7f2d780b5810 2024-09-12T03:22:37.420 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.404+0000 7f2d76ffd640 1 --2- 172.21.15.27:0/1078188739 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f2d6406c460 0x7f2d6406e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:37.420 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.404+0000 7f2d76ffd640 1 -- 172.21.15.27:0/1078188739 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 5765+0+0 (secure 0 0 0) 0x7f2d6808c2c0 con 0x7f2d780b5810 2024-09-12T03:22:37.420 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.404+0000 7f2d7e08d640 1 --2- 172.21.15.27:0/1078188739 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f2d6406c460 0x7f2d6406e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:37.420 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.404+0000 7f2d7e08d640 1 --2- 172.21.15.27:0/1078188739 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f2d6406c460 0x7f2d6406e920 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f2d70001050 tx=0x7f2d70000f80 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:37.421 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.408+0000 7f2d76ffd640 1 -- 172.21.15.27:0/1078188739 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f2d68003e20 con 0x7f2d780b5810 2024-09-12T03:22:37.619 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.618+0000 7f2d802f1640 1 -- 172.21.15.27:0/1078188739 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2d780cf8b0 con 0x7f2d780b5810 2024-09-12T03:22:37.620 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.618+0000 7f2d76ffd640 1 -- 172.21.15.27:0/1078188739 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) ==== 94+0+5227 (secure 0 0 0) 0x7f2d68059b00 con 0x7f2d780b5810 2024-09-12T03:22:37.620 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:22:37.620 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":76,"btime":"2024-09-12T03:22:27:434130+0000","default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6437,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/2244842464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2244842464},{"type":"v1","addr":"172.21.15.45:6834","nonce":2244842464}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":72},{"gid":7048,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/1976058956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":1976058956},{"type":"v1","addr":"172.21.15.27:6835","nonce":1976058956}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":72},{"gid":7060,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/1084669125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":1084669125},{"type":"v1","addr":"172.21.15.45:6835","nonce":1084669125}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":72}],"filesystems":[{"mdsmap":{"epoch":75,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-12T03:22:18.755267+0000","modified":"2024-09-12T03:22:20.761064+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7725},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7725":{"gid":7725,"name":"c","rank":0,"incarnation":73,"state":"up:active","state_seq":3,"addr":"172.21.15.27:6834/1711260376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":1711260376},{"type":"v1","addr":"172.21.15.27:6834","nonce":1711260376}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7725,"qdb_cluster":[7725]},"id":10}]} 2024-09-12T03:22:37.620 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 76 2024-09-12T03:22:37.622 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.620+0000 7f2d802f1640 1 -- 172.21.15.27:0/1078188739 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f2d6406c460 msgr2=0x7f2d6406e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:37.622 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.620+0000 7f2d802f1640 1 --2- 172.21.15.27:0/1078188739 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f2d6406c460 0x7f2d6406e920 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f2d70001050 tx=0x7f2d70000f80 comp rx=0 tx=0).stop 2024-09-12T03:22:37.622 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.620+0000 7f2d802f1640 1 -- 172.21.15.27:0/1078188739 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2d780b5810 msgr2=0x7f2d7815f400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:37.622 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.620+0000 7f2d802f1640 1 --2- 172.21.15.27:0/1078188739 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2d780b5810 0x7f2d7815f400 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7f2d68007cf0 tx=0x7f2d680115a0 comp rx=0 tx=0).stop 2024-09-12T03:22:37.622 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.621+0000 7f2d802f1640 1 -- 172.21.15.27:0/1078188739 shutdown_connections 2024-09-12T03:22:37.622 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.621+0000 7f2d802f1640 1 --2- 172.21.15.27:0/1078188739 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f2d6406c460 0x7f2d6406e920 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:37.622 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.621+0000 7f2d802f1640 1 --2- 172.21.15.27:0/1078188739 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2d780b5810 0x7f2d7815f400 unknown :-1 s=CLOSED pgs=1335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:37.622 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.621+0000 7f2d802f1640 1 --2- 172.21.15.27:0/1078188739 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2d780b53b0 0x7f2d7815eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:37.622 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.621+0000 7f2d802f1640 1 --2- 172.21.15.27:0/1078188739 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2d60004800 0x7f2d78142e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:37.622 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.621+0000 7f2d802f1640 1 -- 172.21.15.27:0/1078188739 >> 172.21.15.27:0/1078188739 conn(0x7f2d78062c60 msgr2=0x7f2d78157690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:37.622 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.621+0000 7f2d802f1640 1 -- 172.21.15.27:0/1078188739 shutdown_connections 2024-09-12T03:22:37.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.621+0000 7f2d802f1640 1 -- 172.21.15.27:0/1078188739 wait complete. 2024-09-12T03:22:37.632 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-12T03:22:37.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.734+0000 7fab1e4d0640 1 Processor -- start 2024-09-12T03:22:37.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.734+0000 7fab1e4d0640 1 -- start start 2024-09-12T03:22:37.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.735+0000 7fab1e4d0640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fab1815e780 0x7fab180d3e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:37.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.735+0000 7fab1e4d0640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fab180d43b0 0x7fab180d1280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:37.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.735+0000 7fab1e4d0640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fab1815eb60 0x7fab180d17c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:37.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.735+0000 7fab1e4d0640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7fab180b7950 con 0x7fab1815e780 2024-09-12T03:22:37.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.735+0000 7fab1e4d0640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fab180b7210 con 0x7fab1815eb60 2024-09-12T03:22:37.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.735+0000 7fab1e4d0640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fab180b6690 con 0x7fab180d43b0 2024-09-12T03:22:37.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.735+0000 7fab17fff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fab1815e780 0x7fab180d3e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:37.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.735+0000 7fab17fff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fab1815e780 0x7fab180d3e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55156/0 (socket says 172.21.15.27:55156) 2024-09-12T03:22:37.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.735+0000 7fab17fff640 1 -- 172.21.15.27:0/3775381300 learned_addr learned my addr 172.21.15.27:0/3775381300 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:22:37.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.735+0000 7fab1ca6d640 1 --2- 172.21.15.27:0/3775381300 >> v2:172.21.15.45:3300/0 conn(0x7fab1815eb60 0x7fab180d17c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:37.738 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.735+0000 7fab177fe640 1 --2- 172.21.15.27:0/3775381300 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fab180d43b0 0x7fab180d1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:37.738 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.735+0000 7fab17fff640 1 -- 172.21.15.27:0/3775381300 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fab180d43b0 msgr2=0x7fab180d1280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:37.738 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.735+0000 7fab17fff640 1 --2- 172.21.15.27:0/3775381300 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fab180d43b0 0x7fab180d1280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:37.738 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.735+0000 7fab17fff640 1 -- 172.21.15.27:0/3775381300 >> v2:172.21.15.45:3300/0 conn(0x7fab1815eb60 msgr2=0x7fab180d17c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:37.738 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.735+0000 7fab17fff640 1 --2- 172.21.15.27:0/3775381300 >> v2:172.21.15.45:3300/0 conn(0x7fab1815eb60 0x7fab180d17c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:37.738 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.735+0000 7fab17fff640 1 -- 172.21.15.27:0/3775381300 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab180b6970 con 0x7fab1815e780 2024-09-12T03:22:37.738 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.736+0000 7fab17fff640 1 --2- 172.21.15.27:0/3775381300 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab1815e780 0x7fab180d3e70 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7fab18069cd0 tx=0x7fab0802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=854b2965fdeb7dc1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:37.738 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.736+0000 7fab16ffd640 1 -- 172.21.15.27:0/3775381300 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab08037070 con 0x7fab1815e780 2024-09-12T03:22:37.738 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.736+0000 7fab16ffd640 1 -- 172.21.15.27:0/3775381300 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab08032440 con 0x7fab1815e780 2024-09-12T03:22:37.738 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.736+0000 7fab16ffd640 1 -- 172.21.15.27:0/3775381300 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab08032810 con 0x7fab1815e780 2024-09-12T03:22:37.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.737+0000 7fab1e4d0640 1 -- 172.21.15.27:0/3775381300 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab1815e780 msgr2=0x7fab180d3e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:37.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.737+0000 7fab1e4d0640 1 --2- 172.21.15.27:0/3775381300 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab1815e780 0x7fab180d3e70 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7fab18069cd0 tx=0x7fab0802a1e0 comp rx=0 tx=0).stop 2024-09-12T03:22:37.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.737+0000 7fab1e4d0640 1 -- 172.21.15.27:0/3775381300 shutdown_connections 2024-09-12T03:22:37.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.737+0000 7fab1e4d0640 1 --2- 172.21.15.27:0/3775381300 >> v2:172.21.15.45:3300/0 conn(0x7fab1815eb60 0x7fab180d17c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:37.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.737+0000 7fab1e4d0640 1 --2- 172.21.15.27:0/3775381300 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fab180d43b0 0x7fab180d1280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:37.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.737+0000 7fab1e4d0640 1 --2- 172.21.15.27:0/3775381300 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab1815e780 0x7fab180d3e70 unknown :-1 s=CLOSED pgs=1336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:37.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.737+0000 7fab1e4d0640 1 -- 172.21.15.27:0/3775381300 >> 172.21.15.27:0/3775381300 conn(0x7fab1806da60 msgr2=0x7fab1806de70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:37.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.737+0000 7fab1e4d0640 1 -- 172.21.15.27:0/3775381300 shutdown_connections 2024-09-12T03:22:37.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.737+0000 7fab1e4d0640 1 -- 172.21.15.27:0/3775381300 wait complete. 2024-09-12T03:22:37.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.738+0000 7fab1e4d0640 1 Processor -- start 2024-09-12T03:22:37.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.738+0000 7fab1e4d0640 1 -- start start 2024-09-12T03:22:37.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.739+0000 7fab1e4d0640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab180d43b0 0x7fab18153e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:37.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.739+0000 7fab1e4d0640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab1815e780 0x7fab180c8d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:37.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.739+0000 7fab1e4d0640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fab1815eb60 0x7fab181f43f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:37.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.739+0000 7fab1e4d0640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fab180b6340 con 0x7fab1815e780 2024-09-12T03:22:37.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.739+0000 7fab1e4d0640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fab180b6f30 con 0x7fab180d43b0 2024-09-12T03:22:37.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.739+0000 7fab1e4d0640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fab180b3390 con 0x7fab1815eb60 2024-09-12T03:22:37.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.739+0000 7fab177fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab1815e780 0x7fab180c8d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:37.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.739+0000 7fab177fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab1815e780 0x7fab180c8d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55166/0 (socket says 172.21.15.27:55166) 2024-09-12T03:22:37.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.739+0000 7fab177fe640 1 -- 172.21.15.27:0/1230899215 learned_addr learned my addr 172.21.15.27:0/1230899215 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:22:37.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.739+0000 7fab17fff640 1 --2- 172.21.15.27:0/1230899215 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab180d43b0 0x7fab18153e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:37.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.739+0000 7fab1ca6d640 1 --2- 172.21.15.27:0/1230899215 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fab1815eb60 0x7fab181f43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:37.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.739+0000 7fab177fe640 1 -- 172.21.15.27:0/1230899215 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fab1815eb60 msgr2=0x7fab181f43f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:37.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.739+0000 7fab177fe640 1 --2- 172.21.15.27:0/1230899215 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fab1815eb60 0x7fab181f43f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:37.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.739+0000 7fab177fe640 1 -- 172.21.15.27:0/1230899215 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab180d43b0 msgr2=0x7fab18153e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:37.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.739+0000 7fab177fe640 1 --2- 172.21.15.27:0/1230899215 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab180d43b0 0x7fab18153e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:37.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.739+0000 7fab177fe640 1 -- 172.21.15.27:0/1230899215 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab18154390 con 0x7fab1815e780 2024-09-12T03:22:37.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.739+0000 7fab177fe640 1 --2- 172.21.15.27:0/1230899215 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab1815e780 0x7fab180c8d20 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7fab0c00d800 tx=0x7fab0c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:37.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.739+0000 7fab14ff9640 1 -- 172.21.15.27:0/1230899215 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab0c018020 con 0x7fab1815e780 2024-09-12T03:22:37.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.739+0000 7fab14ff9640 1 -- 172.21.15.27:0/1230899215 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab0c0096a0 con 0x7fab1815e780 2024-09-12T03:22:37.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.739+0000 7fab14ff9640 1 -- 172.21.15.27:0/1230899215 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab0c009a70 con 0x7fab1815e780 2024-09-12T03:22:37.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.740+0000 7fab1e4d0640 1 -- 172.21.15.27:0/1230899215 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab181f4dd0 con 0x7fab1815e780 2024-09-12T03:22:37.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.740+0000 7fab1e4d0640 1 -- 172.21.15.27:0/1230899215 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab180c8520 con 0x7fab1815e780 2024-09-12T03:22:37.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.740+0000 7fab1e4d0640 1 -- 172.21.15.27:0/1230899215 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab180d3e70 con 0x7fab1815e780 2024-09-12T03:22:37.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.741+0000 7fab14ff9640 1 -- 172.21.15.27:0/1230899215 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 22) ==== 91059+0+0 (secure 0 0 0) 0x7fab0c01b020 con 0x7fab1815e780 2024-09-12T03:22:37.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.742+0000 7fab14ff9640 1 --2- 172.21.15.27:0/1230899215 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fab0406c460 0x7fab0406e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:37.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.742+0000 7fab17fff640 1 --2- 172.21.15.27:0/1230899215 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fab0406c460 0x7fab0406e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:37.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.742+0000 7fab14ff9640 1 -- 172.21.15.27:0/1230899215 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 5765+0+0 (secure 0 0 0) 0x7fab0c08bb30 con 0x7fab1815e780 2024-09-12T03:22:37.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.742+0000 7fab17fff640 1 --2- 172.21.15.27:0/1230899215 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fab0406c460 0x7fab0406e920 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fab08034000 tx=0x7fab0802e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:37.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.746+0000 7fab14ff9640 1 -- 172.21.15.27:0/1230899215 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fab0c022030 con 0x7fab1815e780 2024-09-12T03:22:37.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.924+0000 7fab1e4d0640 1 -- 172.21.15.27:0/1230899215 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fab1806e310 con 0x7fab1815e780 2024-09-12T03:22:37.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.925+0000 7fab14ff9640 1 -- 172.21.15.27:0/1230899215 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v126) ==== 74+0+16780 (secure 0 0 0) 0x7fab18154390 con 0x7fab1815e780 2024-09-12T03:22:37.926 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:22:37.926 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":126,"fsid":"d2df504e-2647-4385-a5db-9fdf69f487a0","created":"2024-09-12T03:16:07.778937+0000","modified":"2024-09-12T03:22:20.406881+0000","last_up_change":"2024-09-12T03:16:19.314778+0000","last_in_change":"2024-09-12T03:16:09.893086+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T03:16:20.097646+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-09-12T03:22:16.425965+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"125","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-09-12T03:22:17.746336+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"126","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1851116c-1fb3-4dc9-9772-f90e2ef4c355","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6817","nonce":3002321488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6819","nonce":3002321488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6823","nonce":3002321488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6821","nonce":3002321488}]},"public_addr":"172.21.15.27:6817/3002321488","cluster_addr":"172.21.15.27:6819/3002321488","heartbeat_back_addr":"172.21.15.27:6823/3002321488","heartbeat_front_addr":"172.21.15.27:6821/3002321488","state":["exists","up"]},{"osd":1,"uuid":"f38ae17d-73eb-4204-b99d-975fa806fb87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6809","nonce":1613102886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6811","nonce":1613102886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6815","nonce":1613102886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6813","nonce":1613102886}]},"public_addr":"172.21.15.27:6809/1613102886","cluster_addr":"172.21.15.27:6811/1613102886","heartbeat_back_addr":"172.21.15.27:6815/1613102886","heartbeat_front_addr":"172.21.15.27:6813/1613102886","state":["exists","up"]},{"osd":2,"uuid":"fb12dcc8-bdd0-42b8-b46d-8b10106224bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6825","nonce":3534723355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6827","nonce":3534723355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6831","nonce":3534723355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6829","nonce":3534723355}]},"public_addr":"172.21.15.27:6825/3534723355","cluster_addr":"172.21.15.27:6827/3534723355","heartbeat_back_addr":"172.21.15.27:6831/3534723355","heartbeat_front_addr":"172.21.15.27:6829/3534723355","state":["exists","up"]},{"osd":3,"uuid":"d74d3f49-9c24-4b9f-98e2-3ecd7b5ef3b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6801","nonce":80661598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6803","nonce":80661598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6807","nonce":80661598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6805","nonce":80661598}]},"public_addr":"172.21.15.27:6801/80661598","cluster_addr":"172.21.15.27:6803/80661598","heartbeat_back_addr":"172.21.15.27:6807/80661598","heartbeat_front_addr":"172.21.15.27:6805/80661598","state":["exists","up"]},{"osd":4,"uuid":"2bf123f2-1b0d-41c4-aa3b-805bab267e59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6817","nonce":3943189827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6819","nonce":3943189827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6823","nonce":3943189827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6821","nonce":3943189827}]},"public_addr":"172.21.15.45:6817/3943189827","cluster_addr":"172.21.15.45:6819/3943189827","heartbeat_back_addr":"172.21.15.45:6823/3943189827","heartbeat_front_addr":"172.21.15.45:6821/3943189827","state":["exists","up"]},{"osd":5,"uuid":"c49ba5da-91c6-4a86-a97b-4aba89c253ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6809","nonce":3620760330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6811","nonce":3620760330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6815","nonce":3620760330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6813","nonce":3620760330}]},"public_addr":"172.21.15.45:6809/3620760330","cluster_addr":"172.21.15.45:6811/3620760330","heartbeat_back_addr":"172.21.15.45:6815/3620760330","heartbeat_front_addr":"172.21.15.45:6813/3620760330","state":["exists","up"]},{"osd":6,"uuid":"b0123475-89f7-4a34-9e33-d797b99dd9dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6801","nonce":1989083510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6803","nonce":1989083510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6807","nonce":1989083510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6805","nonce":1989083510}]},"public_addr":"172.21.15.45:6801/1989083510","cluster_addr":"172.21.15.45:6803/1989083510","heartbeat_back_addr":"172.21.15.45:6807/1989083510","heartbeat_front_addr":"172.21.15.45:6805/1989083510","state":["exists","up"]},{"osd":7,"uuid":"4dad773d-b7d8-46fc-8a10-a972aad27b29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6825","nonce":1513300294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6827","nonce":1513300294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6831","nonce":1513300294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6829","nonce":1513300294}]},"public_addr":"172.21.15.45:6825/1513300294","cluster_addr":"172.21.15.45:6827/1513300294","heartbeat_back_addr":"172.21.15.45:6831/1513300294","heartbeat_front_addr":"172.21.15.45:6829/1513300294","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.881855+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:17.133252+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.735339+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.500742+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.477571+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:17.167667+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.470291+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.873743+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]},{"pool":19,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T03:22:37.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.927+0000 7fab1e4d0640 1 -- 172.21.15.27:0/1230899215 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fab0406c460 msgr2=0x7fab0406e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:37.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.927+0000 7fab1e4d0640 1 --2- 172.21.15.27:0/1230899215 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fab0406c460 0x7fab0406e920 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fab08034000 tx=0x7fab0802e000 comp rx=0 tx=0).stop 2024-09-12T03:22:37.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.927+0000 7fab1e4d0640 1 -- 172.21.15.27:0/1230899215 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab1815e780 msgr2=0x7fab180c8d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:37.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.927+0000 7fab1e4d0640 1 --2- 172.21.15.27:0/1230899215 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab1815e780 0x7fab180c8d20 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7fab0c00d800 tx=0x7fab0c00dcc0 comp rx=0 tx=0).stop 2024-09-12T03:22:37.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.927+0000 7fab1e4d0640 1 -- 172.21.15.27:0/1230899215 shutdown_connections 2024-09-12T03:22:37.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.927+0000 7fab1e4d0640 1 --2- 172.21.15.27:0/1230899215 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fab1815eb60 0x7fab181f43f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:37.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.927+0000 7fab1e4d0640 1 --2- 172.21.15.27:0/1230899215 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fab0406c460 0x7fab0406e920 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:37.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.927+0000 7fab1e4d0640 1 --2- 172.21.15.27:0/1230899215 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab1815e780 0x7fab180c8d20 unknown :-1 s=CLOSED pgs=1337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:37.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.927+0000 7fab1e4d0640 1 --2- 172.21.15.27:0/1230899215 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab180d43b0 0x7fab18153e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:37.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.927+0000 7fab1e4d0640 1 -- 172.21.15.27:0/1230899215 >> 172.21.15.27:0/1230899215 conn(0x7fab1806da60 msgr2=0x7fab180b7d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:37.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.927+0000 7fab1e4d0640 1 -- 172.21.15.27:0/1230899215 shutdown_connections 2024-09-12T03:22:37.930 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:37.927+0000 7fab1e4d0640 1 -- 172.21.15.27:0/1230899215 wait complete. 2024-09-12T03:22:37.940 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-12T03:22:37.940 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-12T03:22:38.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.043+0000 7f01fe798640 1 Processor -- start 2024-09-12T03:22:38.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.043+0000 7f01fe798640 1 -- start start 2024-09-12T03:22:38.048 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.044+0000 7f01fe798640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f01f815e5f0 0x7f01f8073250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:38.048 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.044+0000 7f01fe798640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f01f8073bb0 con 0x7f01f8162a30 2024-09-12T03:22:38.048 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.044+0000 7f01fe798640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f01f80d2240 con 0x7f01f8073790 2024-09-12T03:22:38.048 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.044+0000 7f01fe798640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f01f80d0da0 con 0x7f01f815e5f0 2024-09-12T03:22:38.048 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.044+0000 7f01fcd35640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f01f8162a30 0x7f01f815e9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:38676/0 (socket says 172.21.15.27:38676) 2024-09-12T03:22:38.048 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.044+0000 7f01fcd35640 1 -- 172.21.15.27:0/1038923986 learned_addr learned my addr 172.21.15.27:0/1038923986 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:22:38.048 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.044+0000 7f01f7fff640 1 --2- 172.21.15.27:0/1038923986 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f01f815e5f0 0x7f01f8073250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:38.048 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.044+0000 7f01f6ffd640 1 -- 172.21.15.27:0/1038923986 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 644600383 0 0) 0x7f01f8073bb0 con 0x7f01f8162a30 2024-09-12T03:22:38.048 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.044+0000 7f01f6ffd640 1 -- 172.21.15.27:0/1038923986 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f01e0003520 con 0x7f01f8162a30 2024-09-12T03:22:38.048 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.045+0000 7f01f6ffd640 1 -- 172.21.15.27:0/1038923986 <== mon.1 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1930203749 0 0) 0x7f01f80d2240 con 0x7f01f8073790 2024-09-12T03:22:38.048 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.045+0000 7f01f6ffd640 1 -- 172.21.15.27:0/1038923986 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f01f8073bb0 con 0x7f01f8073790 2024-09-12T03:22:38.048 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.045+0000 7f01f6ffd640 1 -- 172.21.15.27:0/1038923986 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3002136645 0 0) 0x7f01e0003520 con 0x7f01f8162a30 2024-09-12T03:22:38.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.045+0000 7f01f6ffd640 1 -- 172.21.15.27:0/1038923986 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f01f815e5f0 msgr2=0x7f01f8073250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:38.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.045+0000 7f01f6ffd640 1 --2- 172.21.15.27:0/1038923986 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f01f815e5f0 0x7f01f8073250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:38.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.045+0000 7f01f6ffd640 1 -- 172.21.15.27:0/1038923986 >> v1:172.21.15.45:6789/0 conn(0x7f01f8073790 legacy=0x7f01f8162320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:38.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.045+0000 7f01f6ffd640 1 -- 172.21.15.27:0/1038923986 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01f8163090 con 0x7f01f8162a30 2024-09-12T03:22:38.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.045+0000 7f01f6ffd640 1 -- 172.21.15.27:0/1038923986 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 847755213 0 0) 0x7f01f00019f0 con 0x7f01f8162a30 2024-09-12T03:22:38.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.045+0000 7f01f6ffd640 1 -- 172.21.15.27:0/1038923986 >> v1:172.21.15.27:6789/0 conn(0x7f01f8162a30 legacy=0x7f01f815e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:38.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.045+0000 7f01f6ffd640 1 --2- 172.21.15.27:0/1038923986 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f01e0003b70 0x7f01e0003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:38.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.045+0000 7f01f6ffd640 1 --2- 172.21.15.27:0/1038923986 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f01e0004800 0x7f01e0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:38.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.045+0000 7f01f6ffd640 1 --2- 172.21.15.27:0/1038923986 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f01f815e5f0 0x7f01e00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:38.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.045+0000 7f01f6ffd640 1 -- 172.21.15.27:0/1038923986 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f01f80d0da0 con 0x7f01e0003b70 2024-09-12T03:22:38.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.045+0000 7f01f6ffd640 1 -- 172.21.15.27:0/1038923986 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f01e000ba70 con 0x7f01e0004800 2024-09-12T03:22:38.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.045+0000 7f01f6ffd640 1 -- 172.21.15.27:0/1038923986 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f01e000bc40 con 0x7f01f815e5f0 2024-09-12T03:22:38.050 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.045+0000 7f01f7fff640 1 --2- 172.21.15.27:0/1038923986 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f01e0003b70 0x7f01e0003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:38.050 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.045+0000 7f01f77fe640 1 --2- 172.21.15.27:0/1038923986 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f01e0004800 0x7f01e0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:38.050 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.045+0000 7f01fcd35640 1 --2- 172.21.15.27:0/1038923986 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f01f815e5f0 0x7f01e00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:38.050 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.045+0000 7f01f7fff640 1 -- 172.21.15.27:0/1038923986 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f01f815e5f0 msgr2=0x7f01e00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:38.050 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.045+0000 7f01f7fff640 1 --2- 172.21.15.27:0/1038923986 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f01f815e5f0 0x7f01e00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:38.050 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.045+0000 7f01f7fff640 1 -- 172.21.15.27:0/1038923986 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f01e0004800 msgr2=0x7f01e0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:38.050 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.045+0000 7f01f7fff640 1 --2- 172.21.15.27:0/1038923986 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f01e0004800 0x7f01e0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:38.050 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.045+0000 7f01f7fff640 1 -- 172.21.15.27:0/1038923986 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01e0003520 con 0x7f01e0003b70 2024-09-12T03:22:38.050 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.045+0000 7f01f77fe640 1 --2- 172.21.15.27:0/1038923986 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f01e0004800 0x7f01e0004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:22:38.050 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.046+0000 7f01f7fff640 1 --2- 172.21.15.27:0/1038923986 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f01e0003b70 0x7f01e0003f70 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7f01e8009ac0 tx=0x7f01e0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ec1bc8f877e92d4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:38.050 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.046+0000 7f01f6ffd640 1 -- 172.21.15.27:0/1038923986 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01e8036070 con 0x7f01e0003b70 2024-09-12T03:22:38.050 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.046+0000 7f01f6ffd640 1 -- 172.21.15.27:0/1038923986 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01e8031400 con 0x7f01e0003b70 2024-09-12T03:22:38.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.046+0000 7f01f6ffd640 1 -- 172.21.15.27:0/1038923986 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01e8009690 con 0x7f01e0003b70 2024-09-12T03:22:38.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.046+0000 7f01fe798640 1 -- 172.21.15.27:0/1038923986 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f01e0003b70 msgr2=0x7f01e0003f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:38.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.046+0000 7f01fe798640 1 --2- 172.21.15.27:0/1038923986 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f01e0003b70 0x7f01e0003f70 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7f01e8009ac0 tx=0x7f01e0005830 comp rx=0 tx=0).stop 2024-09-12T03:22:38.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.046+0000 7f01fcd35640 1 -- 172.21.15.27:0/1038923986 reap_dead start 2024-09-12T03:22:38.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.047+0000 7f01fe798640 1 -- 172.21.15.27:0/1038923986 shutdown_connections 2024-09-12T03:22:38.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.047+0000 7f01fe798640 1 -- 172.21.15.27:0/1038923986 >> 172.21.15.27:0/1038923986 conn(0x7f01f806f300 msgr2=0x7f01f806f710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:38.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.047+0000 7f01fe798640 1 -- 172.21.15.27:0/1038923986 shutdown_connections 2024-09-12T03:22:38.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.047+0000 7f01fe798640 1 -- 172.21.15.27:0/1038923986 wait complete. 2024-09-12T03:22:38.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.048+0000 7f01fe798640 1 Processor -- start 2024-09-12T03:22:38.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.048+0000 7f01fe798640 1 -- start start 2024-09-12T03:22:38.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.049+0000 7f01fe798640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f01e0003b70 0x7f01f8153b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:38.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.049+0000 7f01fe798640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f01f8160f30 0x7f01f81613b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:38.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.049+0000 7f01fe798640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f01f81554e0 0x7f01f8155960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:38.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.049+0000 7f01fe798640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f01f80d0a30 con 0x7f01e0003b70 2024-09-12T03:22:38.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.049+0000 7f01fe798640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f01f80d0750 con 0x7f01f8160f30 2024-09-12T03:22:38.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.049+0000 7f01fe798640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f01f80b5620 con 0x7f01f81554e0 2024-09-12T03:22:38.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.049+0000 7f01f7fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f01e0003b70 0x7f01f8153b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:38.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.049+0000 7f01f7fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f01e0003b70 0x7f01f8153b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55180/0 (socket says 172.21.15.27:55180) 2024-09-12T03:22:38.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.049+0000 7f01f7fff640 1 -- 172.21.15.27:0/1431228156 learned_addr learned my addr 172.21.15.27:0/1431228156 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:22:38.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.049+0000 7f01fcd35640 1 --2- 172.21.15.27:0/1431228156 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f01f81554e0 0x7f01f8155960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:38.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.049+0000 7f01f77fe640 1 --2- 172.21.15.27:0/1431228156 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f01f8160f30 0x7f01f81613b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:38.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.049+0000 7f01f7fff640 1 -- 172.21.15.27:0/1431228156 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f01f81554e0 msgr2=0x7f01f8155960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:38.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.049+0000 7f01f7fff640 1 --2- 172.21.15.27:0/1431228156 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f01f81554e0 0x7f01f8155960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:38.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.049+0000 7f01f7fff640 1 -- 172.21.15.27:0/1431228156 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f01f8160f30 msgr2=0x7f01f81613b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:38.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.049+0000 7f01f7fff640 1 --2- 172.21.15.27:0/1431228156 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f01f8160f30 0x7f01f81613b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:38.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.049+0000 7f01f7fff640 1 -- 172.21.15.27:0/1431228156 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01f81562d0 con 0x7f01e0003b70 2024-09-12T03:22:38.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.049+0000 7f01f7fff640 1 --2- 172.21.15.27:0/1431228156 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f01e0003b70 0x7f01f8153b20 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7f01e8034040 tx=0x7f01e8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:38.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.049+0000 7f01f4ff9640 1 -- 172.21.15.27:0/1431228156 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01e8041020 con 0x7f01e0003b70 2024-09-12T03:22:38.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.049+0000 7f01f4ff9640 1 -- 172.21.15.27:0/1431228156 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01e803c470 con 0x7f01e0003b70 2024-09-12T03:22:38.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.050+0000 7f01fe798640 1 -- 172.21.15.27:0/1431228156 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01f81565b0 con 0x7f01e0003b70 2024-09-12T03:22:38.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.050+0000 7f01f4ff9640 1 -- 172.21.15.27:0/1431228156 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01e8036040 con 0x7f01e0003b70 2024-09-12T03:22:38.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.050+0000 7f01fe798640 1 -- 172.21.15.27:0/1431228156 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01f8154060 con 0x7f01e0003b70 2024-09-12T03:22:38.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.051+0000 7f01fe798640 1 -- 172.21.15.27:0/1431228156 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01f8073250 con 0x7f01e0003b70 2024-09-12T03:22:38.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.051+0000 7f01f4ff9640 1 -- 172.21.15.27:0/1431228156 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 22) ==== 91059+0+0 (secure 0 0 0) 0x7f01e8044020 con 0x7f01e0003b70 2024-09-12T03:22:38.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.052+0000 7f01f4ff9640 1 --2- 172.21.15.27:0/1431228156 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f01e406c460 0x7f01e406e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:38.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.052+0000 7f01f4ff9640 1 -- 172.21.15.27:0/1431228156 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 5765+0+0 (secure 0 0 0) 0x7f01e80abf20 con 0x7f01e0003b70 2024-09-12T03:22:38.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.052+0000 7f01f77fe640 1 --2- 172.21.15.27:0/1431228156 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f01e406c460 0x7f01e406e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:38.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.052+0000 7f01f77fe640 1 --2- 172.21.15.27:0/1431228156 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f01e406c460 0x7f01e406e920 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f01f80cf570 tx=0x7f01ec009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:38.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.056+0000 7f01f4ff9640 1 -- 172.21.15.27:0/1431228156 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f01e804b030 con 0x7f01e0003b70 2024-09-12T03:22:38.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.268+0000 7f01fe798640 1 -- 172.21.15.27:0/1431228156 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f01f815f6c0 con 0x7f01e0003b70 2024-09-12T03:22:38.269 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.268+0000 7f01f4ff9640 1 -- 172.21.15.27:0/1431228156 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) ==== 94+0+5227 (secure 0 0 0) 0x7f01e80ad090 con 0x7f01e0003b70 2024-09-12T03:22:38.269 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:22:38.269 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":76,"btime":"2024-09-12T03:22:27:434130+0000","default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6437,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/2244842464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2244842464},{"type":"v1","addr":"172.21.15.45:6834","nonce":2244842464}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":72},{"gid":7048,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/1976058956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":1976058956},{"type":"v1","addr":"172.21.15.27:6835","nonce":1976058956}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":72},{"gid":7060,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/1084669125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":1084669125},{"type":"v1","addr":"172.21.15.45:6835","nonce":1084669125}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":72}],"filesystems":[{"mdsmap":{"epoch":75,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-12T03:22:18.755267+0000","modified":"2024-09-12T03:22:20.761064+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7725},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7725":{"gid":7725,"name":"c","rank":0,"incarnation":73,"state":"up:active","state_seq":3,"addr":"172.21.15.27:6834/1711260376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":1711260376},{"type":"v1","addr":"172.21.15.27:6834","nonce":1711260376}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7725,"qdb_cluster":[7725]},"id":10}]} 2024-09-12T03:22:38.269 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 76 2024-09-12T03:22:38.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.270+0000 7f01fe798640 1 -- 172.21.15.27:0/1431228156 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f01e406c460 msgr2=0x7f01e406e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:38.272 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.270+0000 7f01fe798640 1 --2- 172.21.15.27:0/1431228156 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f01e406c460 0x7f01e406e920 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f01f80cf570 tx=0x7f01ec009040 comp rx=0 tx=0).stop 2024-09-12T03:22:38.272 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.270+0000 7f01fe798640 1 -- 172.21.15.27:0/1431228156 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f01e0003b70 msgr2=0x7f01f8153b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:38.272 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.270+0000 7f01fe798640 1 --2- 172.21.15.27:0/1431228156 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f01e0003b70 0x7f01f8153b20 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7f01e8034040 tx=0x7f01e8004730 comp rx=0 tx=0).stop 2024-09-12T03:22:38.272 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.270+0000 7f01fe798640 1 -- 172.21.15.27:0/1431228156 shutdown_connections 2024-09-12T03:22:38.272 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.270+0000 7f01fe798640 1 --2- 172.21.15.27:0/1431228156 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f01f81554e0 0x7f01f8155960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:38.272 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.270+0000 7f01fe798640 1 --2- 172.21.15.27:0/1431228156 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f01f8160f30 0x7f01f81613b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:38.272 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.270+0000 7f01fe798640 1 --2- 172.21.15.27:0/1431228156 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f01e406c460 0x7f01e406e920 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:38.272 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.270+0000 7f01fe798640 1 --2- 172.21.15.27:0/1431228156 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f01e0003b70 0x7f01f8153b20 unknown :-1 s=CLOSED pgs=1340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:38.272 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.270+0000 7f01fe798640 1 -- 172.21.15.27:0/1431228156 >> 172.21.15.27:0/1431228156 conn(0x7f01f806f300 msgr2=0x7f01f815f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:38.272 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.270+0000 7f01fe798640 1 -- 172.21.15.27:0/1431228156 shutdown_connections 2024-09-12T03:22:38.272 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.270+0000 7f01fe798640 1 -- 172.21.15.27:0/1431228156 wait complete. 2024-09-12T03:22:38.282 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-12T03:22:38.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.384+0000 7f6672578640 1 Processor -- start 2024-09-12T03:22:38.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.384+0000 7f6672578640 1 -- start start 2024-09-12T03:22:38.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.384+0000 7f6672578640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f666c15eb90 0x7f666c0d2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:38.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.384+0000 7f6672578640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f666c0d5200 0x7f666c0d1010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:38.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.385+0000 7f6672578640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f666c0d3760 con 0x7f666c0d1550 2024-09-12T03:22:38.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.385+0000 7f6672578640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f666c0b7d20 con 0x7f666c15eb90 2024-09-12T03:22:38.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.385+0000 7f6672578640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f666c0d42e0 con 0x7f666c0d5200 2024-09-12T03:22:38.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.385+0000 7f666bfff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f666c15eb90 0x7f666c0d2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:38.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.385+0000 7f666bfff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f666c15eb90 0x7f666c0d2df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55186/0 (socket says 172.21.15.27:55186) 2024-09-12T03:22:38.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.385+0000 7f666bfff640 1 -- 172.21.15.27:0/2381047641 learned_addr learned my addr 172.21.15.27:0/2381047641 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:22:38.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.385+0000 7f666b7fe640 1 --2- 172.21.15.27:0/2381047641 >> v2:172.21.15.45:3300/0 conn(0x7f666c0d5200 0x7f666c0d1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:38.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.385+0000 7f666bfff640 1 -- 172.21.15.27:0/2381047641 >> v2:172.21.15.45:3300/0 conn(0x7f666c0d5200 msgr2=0x7f666c0d1010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:38.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.385+0000 7f666bfff640 1 --2- 172.21.15.27:0/2381047641 >> v2:172.21.15.45:3300/0 conn(0x7f666c0d5200 0x7f666c0d1010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:38.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.385+0000 7f666bfff640 1 -- 172.21.15.27:0/2381047641 >> v1:172.21.15.45:6789/0 conn(0x7f666c0d1550 legacy=0x7f666c0cf200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:38.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.385+0000 7f666bfff640 1 -- 172.21.15.27:0/2381047641 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f666c0d3a40 con 0x7f666c15eb90 2024-09-12T03:22:38.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.385+0000 7f666b7fe640 1 --2- 172.21.15.27:0/2381047641 >> v2:172.21.15.45:3300/0 conn(0x7f666c0d5200 0x7f666c0d1010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:22:38.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.386+0000 7f666bfff640 1 --2- 172.21.15.27:0/2381047641 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f666c15eb90 0x7f666c0d2df0 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7f666c0d4750 tx=0x7f665c02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c218907b2d0c0c23 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:38.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.386+0000 7f666affd640 1 -- 172.21.15.27:0/2381047641 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f665c037070 con 0x7f666c15eb90 2024-09-12T03:22:38.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.386+0000 7f666affd640 1 -- 172.21.15.27:0/2381047641 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f665c032440 con 0x7f666c15eb90 2024-09-12T03:22:38.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.386+0000 7f666affd640 1 -- 172.21.15.27:0/2381047641 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f665c032810 con 0x7f666c15eb90 2024-09-12T03:22:38.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.386+0000 7f6672578640 1 -- 172.21.15.27:0/2381047641 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f666c15eb90 msgr2=0x7f666c0d2df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:38.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.386+0000 7f6672578640 1 --2- 172.21.15.27:0/2381047641 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f666c15eb90 0x7f666c0d2df0 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7f666c0d4750 tx=0x7f665c02a230 comp rx=0 tx=0).stop 2024-09-12T03:22:38.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.387+0000 7f6672578640 1 -- 172.21.15.27:0/2381047641 shutdown_connections 2024-09-12T03:22:38.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.387+0000 7f6672578640 1 --2- 172.21.15.27:0/2381047641 >> v2:172.21.15.45:3300/0 conn(0x7f666c0d5200 0x7f666c0d1010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:38.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.387+0000 7f6672578640 1 --2- 172.21.15.27:0/2381047641 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f666c15eb90 0x7f666c0d2df0 unknown :-1 s=CLOSED pgs=1341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:38.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.387+0000 7f6672578640 1 -- 172.21.15.27:0/2381047641 >> 172.21.15.27:0/2381047641 conn(0x7f666c06dce0 msgr2=0x7f666c06e0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:38.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.387+0000 7f6672578640 1 -- 172.21.15.27:0/2381047641 shutdown_connections 2024-09-12T03:22:38.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.387+0000 7f6672578640 1 -- 172.21.15.27:0/2381047641 wait complete. 2024-09-12T03:22:38.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.388+0000 7f6672578640 1 Processor -- start 2024-09-12T03:22:38.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.388+0000 7f6672578640 1 -- start start 2024-09-12T03:22:38.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.389+0000 7f6672578640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f666c0d1550 0x7f666c15bb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:38.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.389+0000 7f6672578640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f666c0d5200 0x7f666c1e97e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:38.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.389+0000 7f6672578640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f666c15eb90 0x7f666c1e9d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:38.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.389+0000 7f6672578640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f666c0d1960 con 0x7f666c15eb90 2024-09-12T03:22:38.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.389+0000 7f6672578640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f666c0d4000 con 0x7f666c0d5200 2024-09-12T03:22:38.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.389+0000 7f6672578640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f666c0d3330 con 0x7f666c0d1550 2024-09-12T03:22:38.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.389+0000 7f6670b15640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f666c15eb90 0x7f666c1e9d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:38.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.389+0000 7f6670b15640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f666c15eb90 0x7f666c1e9d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55192/0 (socket says 172.21.15.27:55192) 2024-09-12T03:22:38.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.389+0000 7f6670b15640 1 -- 172.21.15.27:0/1719335358 learned_addr learned my addr 172.21.15.27:0/1719335358 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:22:38.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.389+0000 7f666b7fe640 1 --2- 172.21.15.27:0/1719335358 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f666c0d5200 0x7f666c1e97e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:38.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.389+0000 7f666bfff640 1 --2- 172.21.15.27:0/1719335358 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f666c0d1550 0x7f666c15bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:38.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.389+0000 7f6670b15640 1 -- 172.21.15.27:0/1719335358 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f666c0d1550 msgr2=0x7f666c15bb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:38.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.389+0000 7f6670b15640 1 --2- 172.21.15.27:0/1719335358 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f666c0d1550 0x7f666c15bb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:38.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.389+0000 7f6670b15640 1 -- 172.21.15.27:0/1719335358 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f666c0d5200 msgr2=0x7f666c1e97e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:38.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.389+0000 7f6670b15640 1 --2- 172.21.15.27:0/1719335358 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f666c0d5200 0x7f666c1e97e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:38.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.389+0000 7f6670b15640 1 -- 172.21.15.27:0/1719335358 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f666c0c7e40 con 0x7f666c15eb90 2024-09-12T03:22:38.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.389+0000 7f666b7fe640 1 --2- 172.21.15.27:0/1719335358 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f666c0d5200 0x7f666c1e97e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:22:38.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.389+0000 7f6670b15640 1 --2- 172.21.15.27:0/1719335358 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f666c15eb90 0x7f666c1e9d20 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7f666400c780 tx=0x7f666400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:38.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.389+0000 7f6668ff9640 1 -- 172.21.15.27:0/1719335358 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6664004b10 con 0x7f666c15eb90 2024-09-12T03:22:38.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.389+0000 7f6668ff9640 1 -- 172.21.15.27:0/1719335358 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6664004cb0 con 0x7f666c15eb90 2024-09-12T03:22:38.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.389+0000 7f6668ff9640 1 -- 172.21.15.27:0/1719335358 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f666400aa90 con 0x7f666c15eb90 2024-09-12T03:22:38.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.389+0000 7f6672578640 1 -- 172.21.15.27:0/1719335358 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f666c0cd000 con 0x7f666c15eb90 2024-09-12T03:22:38.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.390+0000 7f6672578640 1 -- 172.21.15.27:0/1719335358 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f666c148f00 con 0x7f666c15eb90 2024-09-12T03:22:38.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.391+0000 7f6672578640 1 -- 172.21.15.27:0/1719335358 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f666c15ef70 con 0x7f666c15eb90 2024-09-12T03:22:38.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.391+0000 7f6668ff9640 1 -- 172.21.15.27:0/1719335358 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 22) ==== 91059+0+0 (secure 0 0 0) 0x7f666400ac30 con 0x7f666c15eb90 2024-09-12T03:22:38.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.392+0000 7f6668ff9640 1 --2- 172.21.15.27:0/1719335358 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f665806c490 0x7f665806e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:38.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.392+0000 7f6668ff9640 1 -- 172.21.15.27:0/1719335358 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 5765+0+0 (secure 0 0 0) 0x7f666408bb70 con 0x7f666c15eb90 2024-09-12T03:22:38.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.392+0000 7f666bfff640 1 --2- 172.21.15.27:0/1719335358 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f665806c490 0x7f665806e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:38.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.392+0000 7f666bfff640 1 --2- 172.21.15.27:0/1719335358 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f665806c490 0x7f665806e950 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f665c034000 tx=0x7f665c02e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:38.397 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.396+0000 7f6668ff9640 1 -- 172.21.15.27:0/1719335358 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f6664020070 con 0x7f666c15eb90 2024-09-12T03:22:38.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.608+0000 7f6672578640 1 -- 172.21.15.27:0/1719335358 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f666c0b99b0 con 0x7f666c15eb90 2024-09-12T03:22:38.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.609+0000 7f6668ff9640 1 -- 172.21.15.27:0/1719335358 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) ==== 94+0+5227 (secure 0 0 0) 0x7f6664059c30 con 0x7f666c15eb90 2024-09-12T03:22:38.610 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:22:38.610 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":76,"btime":"2024-09-12T03:22:27:434130+0000","default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6437,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/2244842464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2244842464},{"type":"v1","addr":"172.21.15.45:6834","nonce":2244842464}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":72},{"gid":7048,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/1976058956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":1976058956},{"type":"v1","addr":"172.21.15.27:6835","nonce":1976058956}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":72},{"gid":7060,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/1084669125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":1084669125},{"type":"v1","addr":"172.21.15.45:6835","nonce":1084669125}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":72}],"filesystems":[{"mdsmap":{"epoch":75,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-12T03:22:18.755267+0000","modified":"2024-09-12T03:22:20.761064+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7725},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7725":{"gid":7725,"name":"c","rank":0,"incarnation":73,"state":"up:active","state_seq":3,"addr":"172.21.15.27:6834/1711260376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":1711260376},{"type":"v1","addr":"172.21.15.27:6834","nonce":1711260376}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7725,"qdb_cluster":[7725]},"id":10}]} 2024-09-12T03:22:38.610 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 76 2024-09-12T03:22:38.612 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.611+0000 7f6672578640 1 -- 172.21.15.27:0/1719335358 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f665806c490 msgr2=0x7f665806e950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:38.612 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.611+0000 7f6672578640 1 --2- 172.21.15.27:0/1719335358 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f665806c490 0x7f665806e950 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f665c034000 tx=0x7f665c02e000 comp rx=0 tx=0).stop 2024-09-12T03:22:38.613 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.611+0000 7f6672578640 1 -- 172.21.15.27:0/1719335358 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f666c15eb90 msgr2=0x7f666c1e9d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:38.613 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.611+0000 7f6672578640 1 --2- 172.21.15.27:0/1719335358 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f666c15eb90 0x7f666c1e9d20 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7f666400c780 tx=0x7f666400cc40 comp rx=0 tx=0).stop 2024-09-12T03:22:38.613 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.611+0000 7f6672578640 1 -- 172.21.15.27:0/1719335358 shutdown_connections 2024-09-12T03:22:38.613 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.611+0000 7f6672578640 1 --2- 172.21.15.27:0/1719335358 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f665806c490 0x7f665806e950 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:38.613 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.611+0000 7f6672578640 1 --2- 172.21.15.27:0/1719335358 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f666c15eb90 0x7f666c1e9d20 unknown :-1 s=CLOSED pgs=1342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:38.613 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.611+0000 7f6672578640 1 --2- 172.21.15.27:0/1719335358 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f666c0d5200 0x7f666c1e97e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:38.613 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.611+0000 7f6672578640 1 --2- 172.21.15.27:0/1719335358 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f666c0d1550 0x7f666c15bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:38.613 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.611+0000 7f6672578640 1 -- 172.21.15.27:0/1719335358 >> 172.21.15.27:0/1719335358 conn(0x7f666c06dce0 msgr2=0x7f666c0ca350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:38.613 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.611+0000 7f6672578640 1 -- 172.21.15.27:0/1719335358 shutdown_connections 2024-09-12T03:22:38.614 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.611+0000 7f6672578640 1 -- 172.21.15.27:0/1719335358 wait complete. 2024-09-12T03:22:38.623 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-12T03:22:38.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.724+0000 7ffa689bd640 1 Processor -- start 2024-09-12T03:22:38.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.724+0000 7ffa689bd640 1 -- start start 2024-09-12T03:22:38.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.725+0000 7ffa689bd640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ffa64158cf0 0x7ffa640b55d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:38.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.725+0000 7ffa689bd640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ffa6415f600 0x7ffa6406bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:38.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.725+0000 7ffa689bd640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffa64162c10 con 0x7ffa641590d0 2024-09-12T03:22:38.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.725+0000 7ffa689bd640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7ffa64162ef0 con 0x7ffa64158cf0 2024-09-12T03:22:38.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.725+0000 7ffa689bd640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7ffa641631d0 con 0x7ffa6415f600 2024-09-12T03:22:38.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.725+0000 7ffa61d9b640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7ffa641590d0 0x7ffa64162430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:38686/0 (socket says 172.21.15.27:38686) 2024-09-12T03:22:38.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.725+0000 7ffa61d9b640 1 -- 172.21.15.27:0/4273051612 learned_addr learned my addr 172.21.15.27:0/4273051612 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:22:38.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.725+0000 7ffa6159a640 1 -- 172.21.15.27:0/4273051612 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2731736927 0 0) 0x7ffa64162c10 con 0x7ffa641590d0 2024-09-12T03:22:38.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.725+0000 7ffa6159a640 1 -- 172.21.15.27:0/4273051612 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ffa50003520 con 0x7ffa641590d0 2024-09-12T03:22:38.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.725+0000 7ffa62d9d640 1 --2- 172.21.15.27:0/4273051612 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ffa6415f600 0x7ffa6406bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:38.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.726+0000 7ffa6259c640 1 --2- 172.21.15.27:0/4273051612 >> v2:172.21.15.45:3300/0 conn(0x7ffa64158cf0 0x7ffa640b55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:38.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.726+0000 7ffa6159a640 1 -- 172.21.15.27:0/4273051612 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1755105926 0 0) 0x7ffa50003520 con 0x7ffa641590d0 2024-09-12T03:22:38.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.726+0000 7ffa6159a640 1 -- 172.21.15.27:0/4273051612 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ffa6415f600 msgr2=0x7ffa6406bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:38.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.726+0000 7ffa6159a640 1 --2- 172.21.15.27:0/4273051612 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ffa6415f600 0x7ffa6406bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:38.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.726+0000 7ffa6159a640 1 -- 172.21.15.27:0/4273051612 >> v2:172.21.15.45:3300/0 conn(0x7ffa64158cf0 msgr2=0x7ffa640b55d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:38.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.726+0000 7ffa6159a640 1 --2- 172.21.15.27:0/4273051612 >> v2:172.21.15.45:3300/0 conn(0x7ffa64158cf0 0x7ffa640b55d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:38.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.726+0000 7ffa6159a640 1 -- 172.21.15.27:0/4273051612 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa6415edc0 con 0x7ffa641590d0 2024-09-12T03:22:38.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.726+0000 7ffa6159a640 1 -- 172.21.15.27:0/4273051612 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 847755213 0 0) 0x7ffa4c001670 con 0x7ffa641590d0 2024-09-12T03:22:38.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.726+0000 7ffa6159a640 1 -- 172.21.15.27:0/4273051612 >> v1:172.21.15.27:6789/0 conn(0x7ffa641590d0 legacy=0x7ffa64162430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:38.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.726+0000 7ffa6159a640 1 --2- 172.21.15.27:0/4273051612 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ffa6415f600 0x7ffa500039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:38.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.726+0000 7ffa6159a640 1 --2- 172.21.15.27:0/4273051612 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffa50003fe0 0x7ffa50004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:38.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.726+0000 7ffa6159a640 1 --2- 172.21.15.27:0/4273051612 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffa50004d40 0x7ffa500051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:38.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.726+0000 7ffa6159a640 1 -- 172.21.15.27:0/4273051612 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7ffa64162ef0 con 0x7ffa50004d40 2024-09-12T03:22:38.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.726+0000 7ffa6159a640 1 -- 172.21.15.27:0/4273051612 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ffa641631d0 con 0x7ffa50003fe0 2024-09-12T03:22:38.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.726+0000 7ffa6159a640 1 -- 172.21.15.27:0/4273051612 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7ffa5000bc30 con 0x7ffa6415f600 2024-09-12T03:22:38.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.726+0000 7ffa62d9d640 1 --2- 172.21.15.27:0/4273051612 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffa50004d40 0x7ffa500051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:38.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.726+0000 7ffa61d9b640 1 --2- 172.21.15.27:0/4273051612 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffa50003fe0 0x7ffa50004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:38.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.726+0000 7ffa6259c640 1 --2- 172.21.15.27:0/4273051612 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ffa6415f600 0x7ffa500039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:38.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.726+0000 7ffa62d9d640 1 -- 172.21.15.27:0/4273051612 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ffa6415f600 msgr2=0x7ffa500039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:38.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.726+0000 7ffa62d9d640 1 --2- 172.21.15.27:0/4273051612 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ffa6415f600 0x7ffa500039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:38.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.726+0000 7ffa62d9d640 1 -- 172.21.15.27:0/4273051612 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffa50003fe0 msgr2=0x7ffa50004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:38.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.726+0000 7ffa62d9d640 1 --2- 172.21.15.27:0/4273051612 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffa50003fe0 0x7ffa50004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:38.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.726+0000 7ffa62d9d640 1 -- 172.21.15.27:0/4273051612 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa50003520 con 0x7ffa50004d40 2024-09-12T03:22:38.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.727+0000 7ffa62d9d640 1 --2- 172.21.15.27:0/4273051612 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffa50004d40 0x7ffa500051c0 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7ffa5c00ba10 tx=0x7ffa50005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d2f8dca654f5447 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:38.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.727+0000 7ffa6159a640 1 -- 172.21.15.27:0/4273051612 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa5c030070 con 0x7ffa50004d40 2024-09-12T03:22:38.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.727+0000 7ffa6159a640 1 -- 172.21.15.27:0/4273051612 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa5c02add0 con 0x7ffa50004d40 2024-09-12T03:22:38.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.727+0000 7ffa6159a640 1 -- 172.21.15.27:0/4273051612 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa5c00b5e0 con 0x7ffa50004d40 2024-09-12T03:22:38.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.728+0000 7ffa689bd640 1 -- 172.21.15.27:0/4273051612 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffa50004d40 msgr2=0x7ffa500051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:38.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.728+0000 7ffa689bd640 1 --2- 172.21.15.27:0/4273051612 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffa50004d40 0x7ffa500051c0 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7ffa5c00ba10 tx=0x7ffa50005830 comp rx=0 tx=0).stop 2024-09-12T03:22:38.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.728+0000 7ffa62d9d640 1 -- 172.21.15.27:0/4273051612 reap_dead start 2024-09-12T03:22:38.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.728+0000 7ffa689bd640 1 -- 172.21.15.27:0/4273051612 shutdown_connections 2024-09-12T03:22:38.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.728+0000 7ffa689bd640 1 -- 172.21.15.27:0/4273051612 >> 172.21.15.27:0/4273051612 conn(0x7ffa6406cc40 msgr2=0x7ffa6406d050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:38.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.728+0000 7ffa689bd640 1 -- 172.21.15.27:0/4273051612 shutdown_connections 2024-09-12T03:22:38.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.728+0000 7ffa689bd640 1 -- 172.21.15.27:0/4273051612 wait complete. 2024-09-12T03:22:38.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.729+0000 7ffa689bd640 1 Processor -- start 2024-09-12T03:22:38.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.729+0000 7ffa689bd640 1 -- start start 2024-09-12T03:22:38.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.730+0000 7ffa689bd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffa50004d40 0x7ffa641f4c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:38.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.730+0000 7ffa689bd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffa64159ec0 0x7ffa641542f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:38.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.730+0000 7ffa689bd640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ffa6415b9b0 0x7ffa6415be30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:38.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.730+0000 7ffa689bd640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7ffa6406bb30 con 0x7ffa50004d40 2024-09-12T03:22:38.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.730+0000 7ffa689bd640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ffa6415fc60 con 0x7ffa64159ec0 2024-09-12T03:22:38.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.730+0000 7ffa689bd640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7ffa6406bcb0 con 0x7ffa6415b9b0 2024-09-12T03:22:38.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.730+0000 7ffa6259c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffa50004d40 0x7ffa641f4c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:38.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.730+0000 7ffa6259c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffa50004d40 0x7ffa641f4c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55212/0 (socket says 172.21.15.27:55212) 2024-09-12T03:22:38.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.730+0000 7ffa6259c640 1 -- 172.21.15.27:0/1566820323 learned_addr learned my addr 172.21.15.27:0/1566820323 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:22:38.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.730+0000 7ffa6259c640 1 -- 172.21.15.27:0/1566820323 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ffa6415b9b0 msgr2=0x7ffa6415be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:38.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.730+0000 7ffa61d9b640 1 --2- 172.21.15.27:0/1566820323 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffa64159ec0 0x7ffa641542f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:38.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.730+0000 7ffa6259c640 1 --2- 172.21.15.27:0/1566820323 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ffa6415b9b0 0x7ffa6415be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:38.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.730+0000 7ffa6259c640 1 -- 172.21.15.27:0/1566820323 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffa64159ec0 msgr2=0x7ffa641542f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:38.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.730+0000 7ffa6259c640 1 --2- 172.21.15.27:0/1566820323 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffa64159ec0 0x7ffa641542f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:38.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.730+0000 7ffa6259c640 1 -- 172.21.15.27:0/1566820323 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa64154830 con 0x7ffa50004d40 2024-09-12T03:22:38.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.731+0000 7ffa6259c640 1 --2- 172.21.15.27:0/1566820323 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffa50004d40 0x7ffa641f4c20 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7ffa540049a0 tx=0x7ffa54007700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:38.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.731+0000 7ffa5affd640 1 -- 172.21.15.27:0/1566820323 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa54015420 con 0x7ffa50004d40 2024-09-12T03:22:38.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.731+0000 7ffa5affd640 1 -- 172.21.15.27:0/1566820323 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa5401a070 con 0x7ffa50004d40 2024-09-12T03:22:38.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.731+0000 7ffa689bd640 1 -- 172.21.15.27:0/1566820323 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa641594b0 con 0x7ffa50004d40 2024-09-12T03:22:38.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.731+0000 7ffa5affd640 1 -- 172.21.15.27:0/1566820323 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa54015800 con 0x7ffa50004d40 2024-09-12T03:22:38.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.731+0000 7ffa689bd640 1 -- 172.21.15.27:0/1566820323 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa64159a70 con 0x7ffa50004d40 2024-09-12T03:22:38.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.732+0000 7ffa689bd640 1 -- 172.21.15.27:0/1566820323 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa6406c040 con 0x7ffa50004d40 2024-09-12T03:22:38.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.732+0000 7ffa5affd640 1 -- 172.21.15.27:0/1566820323 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 22) ==== 91059+0+0 (secure 0 0 0) 0x7ffa540159a0 con 0x7ffa50004d40 2024-09-12T03:22:38.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.733+0000 7ffa5affd640 1 --2- 172.21.15.27:0/1566820323 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7ffa4406c460 0x7ffa4406e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:38.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.733+0000 7ffa61d9b640 1 --2- 172.21.15.27:0/1566820323 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7ffa4406c460 0x7ffa4406e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:38.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.733+0000 7ffa5affd640 1 -- 172.21.15.27:0/1566820323 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 5765+0+0 (secure 0 0 0) 0x7ffa5408c660 con 0x7ffa50004d40 2024-09-12T03:22:38.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.733+0000 7ffa61d9b640 1 --2- 172.21.15.27:0/1566820323 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7ffa4406c460 0x7ffa4406e920 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7ffa4c0010e0 tx=0x7ffa4c009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:38.738 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.737+0000 7ffa5affd640 1 -- 172.21.15.27:0/1566820323 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7ffa5405a510 con 0x7ffa50004d40 2024-09-12T03:22:38.915 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.914+0000 7ffa689bd640 1 -- 172.21.15.27:0/1566820323 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ffa640b4fb0 con 0x7ffa50004d40 2024-09-12T03:22:38.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.915+0000 7ffa5affd640 1 -- 172.21.15.27:0/1566820323 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v126) ==== 74+0+16780 (secure 0 0 0) 0x7ffa54059e10 con 0x7ffa50004d40 2024-09-12T03:22:38.916 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:22:38.916 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":126,"fsid":"d2df504e-2647-4385-a5db-9fdf69f487a0","created":"2024-09-12T03:16:07.778937+0000","modified":"2024-09-12T03:22:20.406881+0000","last_up_change":"2024-09-12T03:16:19.314778+0000","last_in_change":"2024-09-12T03:16:09.893086+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T03:16:20.097646+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-09-12T03:22:16.425965+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"125","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-09-12T03:22:17.746336+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"126","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1851116c-1fb3-4dc9-9772-f90e2ef4c355","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6817","nonce":3002321488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6819","nonce":3002321488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6823","nonce":3002321488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6821","nonce":3002321488}]},"public_addr":"172.21.15.27:6817/3002321488","cluster_addr":"172.21.15.27:6819/3002321488","heartbeat_back_addr":"172.21.15.27:6823/3002321488","heartbeat_front_addr":"172.21.15.27:6821/3002321488","state":["exists","up"]},{"osd":1,"uuid":"f38ae17d-73eb-4204-b99d-975fa806fb87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6809","nonce":1613102886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6811","nonce":1613102886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6815","nonce":1613102886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6813","nonce":1613102886}]},"public_addr":"172.21.15.27:6809/1613102886","cluster_addr":"172.21.15.27:6811/1613102886","heartbeat_back_addr":"172.21.15.27:6815/1613102886","heartbeat_front_addr":"172.21.15.27:6813/1613102886","state":["exists","up"]},{"osd":2,"uuid":"fb12dcc8-bdd0-42b8-b46d-8b10106224bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6825","nonce":3534723355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6827","nonce":3534723355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6831","nonce":3534723355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6829","nonce":3534723355}]},"public_addr":"172.21.15.27:6825/3534723355","cluster_addr":"172.21.15.27:6827/3534723355","heartbeat_back_addr":"172.21.15.27:6831/3534723355","heartbeat_front_addr":"172.21.15.27:6829/3534723355","state":["exists","up"]},{"osd":3,"uuid":"d74d3f49-9c24-4b9f-98e2-3ecd7b5ef3b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6801","nonce":80661598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6803","nonce":80661598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6807","nonce":80661598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6805","nonce":80661598}]},"public_addr":"172.21.15.27:6801/80661598","cluster_addr":"172.21.15.27:6803/80661598","heartbeat_back_addr":"172.21.15.27:6807/80661598","heartbeat_front_addr":"172.21.15.27:6805/80661598","state":["exists","up"]},{"osd":4,"uuid":"2bf123f2-1b0d-41c4-aa3b-805bab267e59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6817","nonce":3943189827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6819","nonce":3943189827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6823","nonce":3943189827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6821","nonce":3943189827}]},"public_addr":"172.21.15.45:6817/3943189827","cluster_addr":"172.21.15.45:6819/3943189827","heartbeat_back_addr":"172.21.15.45:6823/3943189827","heartbeat_front_addr":"172.21.15.45:6821/3943189827","state":["exists","up"]},{"osd":5,"uuid":"c49ba5da-91c6-4a86-a97b-4aba89c253ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6809","nonce":3620760330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6811","nonce":3620760330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6815","nonce":3620760330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6813","nonce":3620760330}]},"public_addr":"172.21.15.45:6809/3620760330","cluster_addr":"172.21.15.45:6811/3620760330","heartbeat_back_addr":"172.21.15.45:6815/3620760330","heartbeat_front_addr":"172.21.15.45:6813/3620760330","state":["exists","up"]},{"osd":6,"uuid":"b0123475-89f7-4a34-9e33-d797b99dd9dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6801","nonce":1989083510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6803","nonce":1989083510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6807","nonce":1989083510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6805","nonce":1989083510}]},"public_addr":"172.21.15.45:6801/1989083510","cluster_addr":"172.21.15.45:6803/1989083510","heartbeat_back_addr":"172.21.15.45:6807/1989083510","heartbeat_front_addr":"172.21.15.45:6805/1989083510","state":["exists","up"]},{"osd":7,"uuid":"4dad773d-b7d8-46fc-8a10-a972aad27b29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6825","nonce":1513300294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6827","nonce":1513300294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6831","nonce":1513300294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6829","nonce":1513300294}]},"public_addr":"172.21.15.45:6825/1513300294","cluster_addr":"172.21.15.45:6827/1513300294","heartbeat_back_addr":"172.21.15.45:6831/1513300294","heartbeat_front_addr":"172.21.15.45:6829/1513300294","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.881855+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:17.133252+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.735339+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.500742+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.477571+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:17.167667+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.470291+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.873743+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]},{"pool":19,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T03:22:38.918 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.917+0000 7ffa689bd640 1 -- 172.21.15.27:0/1566820323 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7ffa4406c460 msgr2=0x7ffa4406e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:38.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.917+0000 7ffa689bd640 1 --2- 172.21.15.27:0/1566820323 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7ffa4406c460 0x7ffa4406e920 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7ffa4c0010e0 tx=0x7ffa4c009040 comp rx=0 tx=0).stop 2024-09-12T03:22:38.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.917+0000 7ffa689bd640 1 -- 172.21.15.27:0/1566820323 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffa50004d40 msgr2=0x7ffa641f4c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:38.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.917+0000 7ffa689bd640 1 --2- 172.21.15.27:0/1566820323 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffa50004d40 0x7ffa641f4c20 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7ffa540049a0 tx=0x7ffa54007700 comp rx=0 tx=0).stop 2024-09-12T03:22:38.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.917+0000 7ffa689bd640 1 -- 172.21.15.27:0/1566820323 shutdown_connections 2024-09-12T03:22:38.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.917+0000 7ffa689bd640 1 --2- 172.21.15.27:0/1566820323 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ffa6415b9b0 0x7ffa6415be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:38.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.917+0000 7ffa689bd640 1 --2- 172.21.15.27:0/1566820323 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffa64159ec0 0x7ffa641542f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:38.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.917+0000 7ffa689bd640 1 --2- 172.21.15.27:0/1566820323 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7ffa4406c460 0x7ffa4406e920 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:38.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.917+0000 7ffa689bd640 1 --2- 172.21.15.27:0/1566820323 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffa50004d40 0x7ffa641f4c20 unknown :-1 s=CLOSED pgs=1345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:38.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.917+0000 7ffa689bd640 1 -- 172.21.15.27:0/1566820323 >> 172.21.15.27:0/1566820323 conn(0x7ffa6406cc40 msgr2=0x7ffa640b21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:38.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.917+0000 7ffa689bd640 1 -- 172.21.15.27:0/1566820323 shutdown_connections 2024-09-12T03:22:38.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:38.917+0000 7ffa689bd640 1 -- 172.21.15.27:0/1566820323 wait complete. 2024-09-12T03:22:38.929 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-12T03:22:39.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:39.032+0000 7f35d04b7640 1 Processor -- start 2024-09-12T03:22:39.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:39.032+0000 7f35d04b7640 1 -- start start 2024-09-12T03:22:39.034 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:39.033+0000 7f35d04b7640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f35c80b5a70 0x7f35c815f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:39.034 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:39.033+0000 7f35d04b7640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f35c815fb40 con 0x7f35c80ae8d0 2024-09-12T03:22:39.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:39.033+0000 7f35d04b7640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f35c80aee90 con 0x7f35c8072fc0 2024-09-12T03:22:39.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:39.033+0000 7f35d04b7640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f35c80af170 con 0x7f35c80b5a70 2024-09-12T03:22:39.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:39.033+0000 7f35cda52640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f35c80ae8d0 0x7f35c80728b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:38694/0 (socket says 172.21.15.27:38694) 2024-09-12T03:22:39.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:39.033+0000 7f35cda52640 1 -- 172.21.15.27:0/2024755229 learned_addr learned my addr 172.21.15.27:0/2024755229 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:22:39.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:39.033+0000 7f35ce253640 1 --2- 172.21.15.27:0/2024755229 >> v2:172.21.15.45:3300/0 conn(0x7f35c80b5a70 0x7f35c815f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:39.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:39.033+0000 7f35cd251640 1 -- 172.21.15.27:0/2024755229 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2466982725 0 0) 0x7f35c815fb40 con 0x7f35c80ae8d0 2024-09-12T03:22:39.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:39.033+0000 7f35cd251640 1 -- 172.21.15.27:0/2024755229 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f35b4003520 con 0x7f35c80ae8d0 2024-09-12T03:22:39.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:39.033+0000 7f35ce253640 1 -- 172.21.15.27:0/2024755229 >> v1:172.21.15.45:6789/0 conn(0x7f35c8072fc0 legacy=0x7f35c806e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:39.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:39.033+0000 7f35ce253640 1 -- 172.21.15.27:0/2024755229 >> v1:172.21.15.27:6789/0 conn(0x7f35c80ae8d0 legacy=0x7f35c80728b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:39.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:39.033+0000 7f35ce253640 1 -- 172.21.15.27:0/2024755229 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35c80af450 con 0x7f35c80b5a70 2024-09-12T03:22:39.037 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:39.034+0000 7f35ce253640 1 --2- 172.21.15.27:0/2024755229 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35c80b5a70 0x7f35c815f600 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f35c40097a0 tx=0x7f35c402a1a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ac6166a5abb11a59 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:39.037 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:39.034+0000 7f35cd251640 1 -- 172.21.15.27:0/2024755229 <== mon.1 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35c4037070 con 0x7f35c80b5a70 2024-09-12T03:22:39.037 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:39.034+0000 7f35cd251640 1 -- 172.21.15.27:0/2024755229 <== mon.1 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35c4032440 con 0x7f35c80b5a70 2024-09-12T03:22:39.037 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:39.034+0000 7f35cd251640 1 -- 172.21.15.27:0/2024755229 <== mon.1 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35c40327f0 con 0x7f35c80b5a70 2024-09-12T03:22:39.037 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:39.035+0000 7f35d04b7640 1 -- 172.21.15.27:0/2024755229 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35c80b5a70 msgr2=0x7f35c815f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:39.037 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:39.035+0000 7f35d04b7640 1 --2- 172.21.15.27:0/2024755229 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35c80b5a70 0x7f35c815f600 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f35c40097a0 tx=0x7f35c402a1a0 comp rx=0 tx=0).stop 2024-09-12T03:22:39.038 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:39.035+0000 7f35d04b7640 1 -- 172.21.15.27:0/2024755229 shutdown_connections 2024-09-12T03:22:39.038 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:39.035+0000 7f35d04b7640 1 --2- 172.21.15.27:0/2024755229 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35c80b5a70 0x7f35c815f600 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:39.038 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:39.035+0000 7f35d04b7640 1 -- 172.21.15.27:0/2024755229 >> 172.21.15.27:0/2024755229 conn(0x7f35c806ca30 msgr2=0x7f35c806ce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:39.038 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:39.035+0000 7f35d04b7640 1 -- 172.21.15.27:0/2024755229 shutdown_connections 2024-09-12T03:22:39.038 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:39.035+0000 7f35d04b7640 1 -- 172.21.15.27:0/2024755229 wait complete. 2024-09-12T03:22:39.038 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:39.036+0000 7f35d04b7640 1 Processor -- start 2024-09-12T03:22:39.038 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:39.036+0000 7f35d04b7640 1 -- start start 2024-09-12T03:22:39.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:39.037+0000 7f35d04b7640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f35c8072fc0 0x7f35c814d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:39.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:39.037+0000 7f35d04b7640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35c80ae8d0 0x7f35c80ce470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:39.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:39.037+0000 7f35d04b7640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f35c80b5a70 0x7f35c80ce9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:39.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:39.037+0000 7f35d04b7640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f35c806f1c0 con 0x7f35c8072fc0 2024-09-12T03:22:39.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:39.037+0000 7f35d04b7640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f35c806f040 con 0x7f35c80ae8d0 2024-09-12T03:22:39.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:39.037+0000 7f35d04b7640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f35c806f340 con 0x7f35c80b5a70 2024-09-12T03:22:39.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:39.037+0000 7f35ce253640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f35c8072fc0 0x7f35c814d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:39.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:39.037+0000 7f35ce253640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f35c8072fc0 0x7f35c814d870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55222/0 (socket says 172.21.15.27:55222) 2024-09-12T03:22:39.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:39.037+0000 7f35ce253640 1 -- 172.21.15.27:0/1901352022 learned_addr learned my addr 172.21.15.27:0/1901352022 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:22:39.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:39.037+0000 7f35ce253640 1 -- 172.21.15.27:0/1901352022 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f35c80b5a70 msgr2=0x7f35c80ce9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:39.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:39.037+0000 7f35ce253640 1 --2- 172.21.15.27:0/1901352022 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f35c80b5a70 0x7f35c80ce9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:39.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:39.037+0000 7f35ce253640 1 -- 172.21.15.27:0/1901352022 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35c80ae8d0 msgr2=0x7f35c80ce470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:39.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:39.037+0000 7f35ce253640 1 --2- 172.21.15.27:0/1901352022 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35c80ae8d0 0x7f35c80ce470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:39.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:39.037+0000 7f35ce253640 1 -- 172.21.15.27:0/1901352022 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35c81f5360 con 0x7f35c8072fc0 2024-09-12T03:22:39.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:39.038+0000 7f35ce253640 1 --2- 172.21.15.27:0/1901352022 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f35c8072fc0 0x7f35c814d870 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7f35c40097a0 tx=0x7f35c40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:39.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:39.038+0000 7f35beffd640 1 -- 172.21.15.27:0/1901352022 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35c4037050 con 0x7f35c8072fc0 2024-09-12T03:22:39.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:39.038+0000 7f35beffd640 1 -- 172.21.15.27:0/1901352022 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35c4038040 con 0x7f35c8072fc0 2024-09-12T03:22:39.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:39.038+0000 7f35d04b7640 1 -- 172.21.15.27:0/1901352022 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35c81f5640 con 0x7f35c8072fc0 2024-09-12T03:22:39.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:39.038+0000 7f35beffd640 1 -- 172.21.15.27:0/1901352022 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35c4004470 con 0x7f35c8072fc0 2024-09-12T03:22:39.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:39.038+0000 7f35d04b7640 1 -- 172.21.15.27:0/1901352022 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35c81f5c00 con 0x7f35c8072fc0 2024-09-12T03:22:39.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:39.039+0000 7f35d04b7640 1 -- 172.21.15.27:0/1901352022 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35c8158580 con 0x7f35c8072fc0 2024-09-12T03:22:39.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:39.039+0000 7f35beffd640 1 -- 172.21.15.27:0/1901352022 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 22) ==== 91059+0+0 (secure 0 0 0) 0x7f35c403c6d0 con 0x7f35c8072fc0 2024-09-12T03:22:39.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:39.040+0000 7f35beffd640 1 --2- 172.21.15.27:0/1901352022 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f35ac06c460 0x7f35ac06e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:39.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:39.040+0000 7f35beffd640 1 -- 172.21.15.27:0/1901352022 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 5765+0+0 (secure 0 0 0) 0x7f35c40ac420 con 0x7f35c8072fc0 2024-09-12T03:22:39.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:39.040+0000 7f35cda52640 1 --2- 172.21.15.27:0/1901352022 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f35ac06c460 0x7f35ac06e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:39.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:39.040+0000 7f35cda52640 1 --2- 172.21.15.27:0/1901352022 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f35ac06c460 0x7f35ac06e920 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f35c815c530 tx=0x7f35b8007930 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:39.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:39.044+0000 7f35beffd640 1 -- 172.21.15.27:0/1901352022 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f35c407a460 con 0x7f35c8072fc0 2024-09-12T03:22:39.256 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:39.255+0000 7f35d04b7640 1 -- 172.21.15.27:0/1901352022 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f35c806e930 con 0x7f35c8072fc0 2024-09-12T03:22:39.257 INFO:tasks.ceph.mon.a.smithi027.stderr:2024-09-12T03:22:39.256+0000 7f959bfd9640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-12T03:22:39.258 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:39.257+0000 7f35beffd640 1 -- 172.21.15.27:0/1901352022 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v77) ==== 161+0+0 (secure 0 0 0) 0x7f35c4079d60 con 0x7f35c8072fc0 2024-09-12T03:22:39.258 INFO:teuthology.orchestra.run.smithi027.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-12T03:22:39.261 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:39.260+0000 7f35d04b7640 1 -- 172.21.15.27:0/1901352022 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f35ac06c460 msgr2=0x7f35ac06e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:39.262 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:39.260+0000 7f35d04b7640 1 --2- 172.21.15.27:0/1901352022 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f35ac06c460 0x7f35ac06e920 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f35c815c530 tx=0x7f35b8007930 comp rx=0 tx=0).stop 2024-09-12T03:22:39.262 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:39.260+0000 7f35d04b7640 1 -- 172.21.15.27:0/1901352022 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f35c8072fc0 msgr2=0x7f35c814d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:39.262 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:39.260+0000 7f35d04b7640 1 --2- 172.21.15.27:0/1901352022 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f35c8072fc0 0x7f35c814d870 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7f35c40097a0 tx=0x7f35c40046c0 comp rx=0 tx=0).stop 2024-09-12T03:22:39.262 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:39.260+0000 7f35d04b7640 1 -- 172.21.15.27:0/1901352022 shutdown_connections 2024-09-12T03:22:39.262 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:39.260+0000 7f35d04b7640 1 --2- 172.21.15.27:0/1901352022 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f35c80b5a70 0x7f35c80ce9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:39.262 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:39.260+0000 7f35d04b7640 1 --2- 172.21.15.27:0/1901352022 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35c80ae8d0 0x7f35c80ce470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:39.262 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:39.260+0000 7f35d04b7640 1 --2- 172.21.15.27:0/1901352022 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f35ac06c460 0x7f35ac06e920 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:39.262 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:39.260+0000 7f35d04b7640 1 --2- 172.21.15.27:0/1901352022 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f35c8072fc0 0x7f35c814d870 unknown :-1 s=CLOSED pgs=1347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:39.262 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:39.260+0000 7f35d04b7640 1 -- 172.21.15.27:0/1901352022 >> 172.21.15.27:0/1901352022 conn(0x7f35c806ca30 msgr2=0x7f35c81577f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:39.262 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:39.260+0000 7f35d04b7640 1 -- 172.21.15.27:0/1901352022 shutdown_connections 2024-09-12T03:22:39.262 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:39.261+0000 7f35d04b7640 1 -- 172.21.15.27:0/1901352022 wait complete. 2024-09-12T03:22:39.264 INFO:tasks.ceph.mds.c.smithi027.stderr: -1257> 2024-09-12T03:22:25.393+0000 7fd2fb742640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-12T03:22:39.273 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-12T03:22:39.291 INFO:tasks.ceph.mds.c.smithi027.stdout:starting mds.c at 2024-09-12T03:22:39.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:39.376+0000 7fb2dd131640 1 Processor -- start 2024-09-12T03:22:39.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:39.376+0000 7fb2dd131640 1 -- start start 2024-09-12T03:22:39.379 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:39.377+0000 7fb2dd131640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb2d8162040 0x7fb2d80d1da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:39.379 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:39.377+0000 7fb2dd131640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb2d80d2e30 con 0x7fb2d806dfd0 2024-09-12T03:22:39.379 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:39.377+0000 7fb2dd131640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb2d8162420 con 0x7fb2d80d3790 2024-09-12T03:22:39.379 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:39.377+0000 7fb2dd131640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fb2d80d30d0 con 0x7fb2d8162040 2024-09-12T03:22:39.379 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:39.377+0000 7fb2d759e640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fb2d806dfd0 0x7fb2d80cf320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:55352/0 (socket says 172.21.15.27:55352) 2024-09-12T03:22:39.379 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:39.377+0000 7fb2d759e640 1 -- 172.21.15.27:0/3651398589 learned_addr learned my addr 172.21.15.27:0/3651398589 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:22:39.379 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:39.378+0000 7fb2d6d9d640 1 --2- 172.21.15.27:0/3651398589 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb2d8162040 0x7fb2d80d1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:39.379 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:39.378+0000 7fb2d5d9b640 1 -- 172.21.15.27:0/3651398589 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1263890274 0 0) 0x7fb2d80d2e30 con 0x7fb2d806dfd0 2024-09-12T03:22:39.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:39.378+0000 7fb2d5d9b640 1 -- 172.21.15.27:0/3651398589 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb2c4003520 con 0x7fb2d806dfd0 2024-09-12T03:22:39.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:39.378+0000 7fb2d6d9d640 1 -- 172.21.15.27:0/3651398589 >> v1:172.21.15.45:6789/0 conn(0x7fb2d80d3790 legacy=0x7fb2d806d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:39.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:39.378+0000 7fb2d5d9b640 1 -- 172.21.15.27:0/3651398589 <== mon.1 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1259246234 0 0) 0x7fb2d8162420 con 0x7fb2d80d3790 2024-09-12T03:22:39.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:39.378+0000 7fb2d6d9d640 1 -- 172.21.15.27:0/3651398589 >> v1:172.21.15.27:6789/0 conn(0x7fb2d806dfd0 legacy=0x7fb2d80cf320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:39.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:39.378+0000 7fb2d6d9d640 1 -- 172.21.15.27:0/3651398589 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2d80d33b0 con 0x7fb2d8162040 2024-09-12T03:22:39.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:39.379+0000 7fb2d6d9d640 1 --2- 172.21.15.27:0/3651398589 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb2d8162040 0x7fb2d80d1da0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fb2d0009dc0 tx=0x7fb2d002a640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c6c7a05b1cc647b1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:39.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:39.379+0000 7fb2d5d9b640 1 -- 172.21.15.27:0/3651398589 <== mon.2 v2:172.21.15.45:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2d0035020 con 0x7fb2d8162040 2024-09-12T03:22:39.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:39.379+0000 7fb2d5d9b640 1 -- 172.21.15.27:0/3651398589 <== mon.2 v2:172.21.15.45:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2d0007540 con 0x7fb2d8162040 2024-09-12T03:22:39.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:39.379+0000 7fb2d5d9b640 1 -- 172.21.15.27:0/3651398589 <== mon.2 v2:172.21.15.45:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2c8001670 con 0x7fb2d8162040 2024-09-12T03:22:39.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:39.379+0000 7fb2dd131640 1 -- 172.21.15.27:0/3651398589 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb2d8162040 msgr2=0x7fb2d80d1da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:39.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:39.379+0000 7fb2dd131640 1 --2- 172.21.15.27:0/3651398589 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb2d8162040 0x7fb2d80d1da0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fb2d0009dc0 tx=0x7fb2d002a640 comp rx=0 tx=0).stop 2024-09-12T03:22:39.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:39.379+0000 7fb2dd131640 1 -- 172.21.15.27:0/3651398589 shutdown_connections 2024-09-12T03:22:39.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:39.379+0000 7fb2dd131640 1 --2- 172.21.15.27:0/3651398589 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb2d8162040 0x7fb2d80d1da0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:39.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:39.379+0000 7fb2dd131640 1 -- 172.21.15.27:0/3651398589 >> 172.21.15.27:0/3651398589 conn(0x7fb2d80b6700 msgr2=0x7fb2d80b6b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:39.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:39.379+0000 7fb2dd131640 1 -- 172.21.15.27:0/3651398589 shutdown_connections 2024-09-12T03:22:39.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:39.379+0000 7fb2dd131640 1 -- 172.21.15.27:0/3651398589 wait complete. 2024-09-12T03:22:39.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:39.381+0000 7fb2dd131640 1 Processor -- start 2024-09-12T03:22:39.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:39.381+0000 7fb2dd131640 1 -- start start 2024-09-12T03:22:39.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:39.382+0000 7fb2dd131640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb2d806dfd0 0x7fb2d815fe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:39.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:39.382+0000 7fb2dd131640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb2d80d3790 0x7fb2d815ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:39.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:39.382+0000 7fb2dd131640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb2d8162040 0x7fb2d81f8560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:39.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:39.382+0000 7fb2dd131640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fb2d80d4150 con 0x7fb2d806dfd0 2024-09-12T03:22:39.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:39.382+0000 7fb2dd131640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb2d80d4530 con 0x7fb2d80d3790 2024-09-12T03:22:39.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:39.382+0000 7fb2dd131640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fb2d80d42d0 con 0x7fb2d8162040 2024-09-12T03:22:39.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:39.382+0000 7fb2d6d9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb2d806dfd0 0x7fb2d815fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:39.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:39.382+0000 7fb2d6d9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb2d806dfd0 0x7fb2d815fe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35298/0 (socket says 172.21.15.27:35298) 2024-09-12T03:22:39.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:39.382+0000 7fb2d6d9d640 1 -- 172.21.15.27:0/1605848517 learned_addr learned my addr 172.21.15.27:0/1605848517 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:22:39.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:39.382+0000 7fb2d659c640 1 --2- 172.21.15.27:0/1605848517 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb2d80d3790 0x7fb2d815ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:39.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:39.382+0000 7fb2d759e640 1 --2- 172.21.15.27:0/1605848517 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb2d8162040 0x7fb2d81f8560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:39.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:39.382+0000 7fb2d6d9d640 1 -- 172.21.15.27:0/1605848517 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb2d8162040 msgr2=0x7fb2d81f8560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:39.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:39.382+0000 7fb2d6d9d640 1 --2- 172.21.15.27:0/1605848517 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb2d8162040 0x7fb2d81f8560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:39.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:39.382+0000 7fb2d6d9d640 1 -- 172.21.15.27:0/1605848517 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb2d80d3790 msgr2=0x7fb2d815ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:39.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:39.382+0000 7fb2d6d9d640 1 --2- 172.21.15.27:0/1605848517 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb2d80d3790 0x7fb2d815ae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:39.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:39.382+0000 7fb2d6d9d640 1 -- 172.21.15.27:0/1605848517 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2d8160470 con 0x7fb2d806dfd0 2024-09-12T03:22:39.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:39.382+0000 7fb2d6d9d640 1 --2- 172.21.15.27:0/1605848517 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb2d806dfd0 0x7fb2d815fe50 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7fb2d0000f80 tx=0x7fb2d0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:39.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:39.382+0000 7fb2cf7fe640 1 -- 172.21.15.27:0/1605848517 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2d0035040 con 0x7fb2d806dfd0 2024-09-12T03:22:39.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:39.382+0000 7fb2cf7fe640 1 -- 172.21.15.27:0/1605848517 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2d0031c60 con 0x7fb2d806dfd0 2024-09-12T03:22:39.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:39.383+0000 7fb2dd131640 1 -- 172.21.15.27:0/1605848517 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2d8160750 con 0x7fb2d806dfd0 2024-09-12T03:22:39.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:39.383+0000 7fb2cf7fe640 1 -- 172.21.15.27:0/1605848517 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2d0009450 con 0x7fb2d806dfd0 2024-09-12T03:22:39.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:39.383+0000 7fb2dd131640 1 -- 172.21.15.27:0/1605848517 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2d81f8040 con 0x7fb2d806dfd0 2024-09-12T03:22:39.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:39.383+0000 7fb2dd131640 1 -- 172.21.15.27:0/1605848517 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2d81f7e30 con 0x7fb2d806dfd0 2024-09-12T03:22:39.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:39.384+0000 7fb2cf7fe640 1 -- 172.21.15.27:0/1605848517 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 22) ==== 91059+0+0 (secure 0 0 0) 0x7fb2d0003bd0 con 0x7fb2d806dfd0 2024-09-12T03:22:39.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:39.384+0000 7fb2cf7fe640 1 --2- 172.21.15.27:0/1605848517 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fb2b806c460 0x7fb2b806e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:39.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:39.385+0000 7fb2cf7fe640 1 -- 172.21.15.27:0/1605848517 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5851+0+0 (secure 0 0 0) 0x7fb2d00ac670 con 0x7fb2d806dfd0 2024-09-12T03:22:39.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:39.385+0000 7fb2d659c640 1 --2- 172.21.15.27:0/1605848517 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fb2b806c460 0x7fb2b806e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:39.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:39.385+0000 7fb2d659c640 1 --2- 172.21.15.27:0/1605848517 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fb2b806c460 0x7fb2b806e920 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fb2d80cddc0 tx=0x7fb2c0009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:39.390 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:39.389+0000 7fb2cf7fe640 1 -- 172.21.15.27:0/1605848517 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fb2d0040020 con 0x7fb2d806dfd0 2024-09-12T03:22:39.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:39.601+0000 7fb2dd131640 1 -- 172.21.15.27:0/1605848517 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fb2d80b44a0 con 0x7fb2d806dfd0 2024-09-12T03:22:40.259 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:40.258+0000 7fb2cf7fe640 1 -- 172.21.15.27:0/1605848517 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v78) ==== 104+0+0 (secure 0 0 0) 0x7fb2d007a0f0 con 0x7fb2d806dfd0 2024-09-12T03:22:40.262 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:40.261+0000 7fb2dd131640 1 -- 172.21.15.27:0/1605848517 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fb2b806c460 msgr2=0x7fb2b806e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:40.262 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:40.261+0000 7fb2dd131640 1 --2- 172.21.15.27:0/1605848517 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fb2b806c460 0x7fb2b806e920 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fb2d80cddc0 tx=0x7fb2c0009040 comp rx=0 tx=0).stop 2024-09-12T03:22:40.262 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:40.261+0000 7fb2dd131640 1 -- 172.21.15.27:0/1605848517 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb2d806dfd0 msgr2=0x7fb2d815fe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:40.262 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:40.261+0000 7fb2dd131640 1 --2- 172.21.15.27:0/1605848517 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb2d806dfd0 0x7fb2d815fe50 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7fb2d0000f80 tx=0x7fb2d0004a40 comp rx=0 tx=0).stop 2024-09-12T03:22:40.262 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:40.261+0000 7fb2dd131640 1 -- 172.21.15.27:0/1605848517 shutdown_connections 2024-09-12T03:22:40.262 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:40.261+0000 7fb2dd131640 1 --2- 172.21.15.27:0/1605848517 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb2d8162040 0x7fb2d81f8560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:40.262 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:40.261+0000 7fb2dd131640 1 --2- 172.21.15.27:0/1605848517 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb2d80d3790 0x7fb2d815ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:40.262 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:40.261+0000 7fb2dd131640 1 --2- 172.21.15.27:0/1605848517 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fb2b806c460 0x7fb2b806e920 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:40.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:40.261+0000 7fb2dd131640 1 --2- 172.21.15.27:0/1605848517 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb2d806dfd0 0x7fb2d815fe50 unknown :-1 s=CLOSED pgs=1351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:40.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:40.261+0000 7fb2dd131640 1 -- 172.21.15.27:0/1605848517 >> 172.21.15.27:0/1605848517 conn(0x7fb2d80b6700 msgr2=0x7fb2d80b86f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:40.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:40.261+0000 7fb2dd131640 1 -- 172.21.15.27:0/1605848517 shutdown_connections 2024-09-12T03:22:40.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:40.261+0000 7fb2dd131640 1 -- 172.21.15.27:0/1605848517 wait complete. 2024-09-12T03:22:40.275 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-12T03:22:40.425 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:40.424+0000 7f39df9c6640 1 Processor -- start 2024-09-12T03:22:40.426 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:40.425+0000 7f39df9c6640 1 -- start start 2024-09-12T03:22:40.427 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:40.425+0000 7f39df9c6640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f39d806dfd0 0x7f39d80d1da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:40.427 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:40.425+0000 7f39df9c6640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f39d80d2e30 con 0x7f39d8162040 2024-09-12T03:22:40.427 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:40.426+0000 7f39df9c6640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f39d8162420 con 0x7f39d80d3790 2024-09-12T03:22:40.427 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:40.426+0000 7f39df9c6640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f39d80d30d0 con 0x7f39d806dfd0 2024-09-12T03:22:40.427 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:40.426+0000 7f39dcf61640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f39d806dfd0 0x7f39d80d1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:40.427 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:40.426+0000 7f39dcf61640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f39d806dfd0 0x7f39d80d1da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35308/0 (socket says 172.21.15.27:35308) 2024-09-12T03:22:40.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:40.426+0000 7f39dd762640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f39d8162040 0x7f39d806d8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:55354/0 (socket says 172.21.15.27:55354) 2024-09-12T03:22:40.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:40.426+0000 7f39dcf61640 1 -- 172.21.15.27:0/3147445232 learned_addr learned my addr 172.21.15.27:0/3147445232 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:22:40.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:40.426+0000 7f39dcf61640 1 -- 172.21.15.27:0/3147445232 >> v1:172.21.15.45:6789/0 conn(0x7f39d80d3790 legacy=0x7f39d80cf320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:40.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:40.426+0000 7f39dcf61640 1 -- 172.21.15.27:0/3147445232 >> v1:172.21.15.27:6789/0 conn(0x7f39d8162040 legacy=0x7f39d806d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:40.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:40.426+0000 7f39d7fff640 1 -- 172.21.15.27:0/3147445232 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 712739022 0 0) 0x7f39d80d2e30 con 0x7f39d8162040 2024-09-12T03:22:40.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:40.426+0000 7f39dcf61640 1 -- 172.21.15.27:0/3147445232 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39d80d33b0 con 0x7f39d806dfd0 2024-09-12T03:22:40.434 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:40.427+0000 7f39dcf61640 1 --2- 172.21.15.27:0/3147445232 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f39d806dfd0 0x7f39d80d1da0 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7f39cc009870 tx=0x7f39cc02a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dc3139f0d6fc1cb server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:40.434 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:40.427+0000 7f39d7fff640 1 -- 172.21.15.27:0/3147445232 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39cc037070 con 0x7f39d806dfd0 2024-09-12T03:22:40.434 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:40.427+0000 7f39d7fff640 1 -- 172.21.15.27:0/3147445232 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39cc02ae50 con 0x7f39d806dfd0 2024-09-12T03:22:40.434 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:40.427+0000 7f39d7fff640 1 -- 172.21.15.27:0/3147445232 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39cc032860 con 0x7f39d806dfd0 2024-09-12T03:22:40.434 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:40.427+0000 7f39df9c6640 1 -- 172.21.15.27:0/3147445232 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f39d806dfd0 msgr2=0x7f39d80d1da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:40.434 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:40.427+0000 7f39df9c6640 1 --2- 172.21.15.27:0/3147445232 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f39d806dfd0 0x7f39d80d1da0 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7f39cc009870 tx=0x7f39cc02a300 comp rx=0 tx=0).stop 2024-09-12T03:22:40.434 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:40.427+0000 7f39df9c6640 1 -- 172.21.15.27:0/3147445232 shutdown_connections 2024-09-12T03:22:40.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:40.427+0000 7f39df9c6640 1 --2- 172.21.15.27:0/3147445232 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f39d806dfd0 0x7f39d80d1da0 unknown :-1 s=CLOSED pgs=1353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:40.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:40.427+0000 7f39df9c6640 1 -- 172.21.15.27:0/3147445232 >> 172.21.15.27:0/3147445232 conn(0x7f39d80b6700 msgr2=0x7f39d80b6b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:40.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:40.428+0000 7f39df9c6640 1 -- 172.21.15.27:0/3147445232 shutdown_connections 2024-09-12T03:22:40.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:40.428+0000 7f39df9c6640 1 -- 172.21.15.27:0/3147445232 wait complete. 2024-09-12T03:22:40.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:40.429+0000 7f39df9c6640 1 Processor -- start 2024-09-12T03:22:40.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:40.429+0000 7f39df9c6640 1 -- start start 2024-09-12T03:22:40.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:40.429+0000 7f39df9c6640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f39d806dfd0 0x7f39d814ea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:40.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:40.429+0000 7f39df9c6640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f39d80d3790 0x7f39d81f4780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:40.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:40.429+0000 7f39df9c6640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f39d8162040 0x7f39d815b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:40.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:40.429+0000 7f39df9c6640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f39d80d4150 con 0x7f39d8162040 2024-09-12T03:22:40.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:40.429+0000 7f39df9c6640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f39d80d4530 con 0x7f39d80d3790 2024-09-12T03:22:40.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:40.429+0000 7f39df9c6640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f39d80d42d0 con 0x7f39d806dfd0 2024-09-12T03:22:40.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:40.430+0000 7f39ddf63640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f39d8162040 0x7f39d815b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:40.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:40.430+0000 7f39ddf63640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f39d8162040 0x7f39d815b080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35320/0 (socket says 172.21.15.27:35320) 2024-09-12T03:22:40.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:40.430+0000 7f39ddf63640 1 -- 172.21.15.27:0/178131991 learned_addr learned my addr 172.21.15.27:0/178131991 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:22:40.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:40.430+0000 7f39dcf61640 1 --2- 172.21.15.27:0/178131991 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f39d80d3790 0x7f39d81f4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:40.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:40.430+0000 7f39dd762640 1 --2- 172.21.15.27:0/178131991 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f39d806dfd0 0x7f39d814ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:40.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:40.430+0000 7f39ddf63640 1 -- 172.21.15.27:0/178131991 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f39d806dfd0 msgr2=0x7f39d814ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:40.437 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:40.430+0000 7f39ddf63640 1 --2- 172.21.15.27:0/178131991 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f39d806dfd0 0x7f39d814ea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:40.437 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:40.430+0000 7f39ddf63640 1 -- 172.21.15.27:0/178131991 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f39d80d3790 msgr2=0x7f39d81f4780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:40.437 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:40.430+0000 7f39ddf63640 1 --2- 172.21.15.27:0/178131991 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f39d80d3790 0x7f39d81f4780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:40.437 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:40.430+0000 7f39ddf63640 1 -- 172.21.15.27:0/178131991 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39d80cd020 con 0x7f39d8162040 2024-09-12T03:22:40.437 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:40.430+0000 7f39dd762640 1 --2- 172.21.15.27:0/178131991 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f39d806dfd0 0x7f39d814ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:22:40.437 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:40.430+0000 7f39ddf63640 1 --2- 172.21.15.27:0/178131991 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f39d8162040 0x7f39d815b080 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7f39c800c730 tx=0x7f39c800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:40.437 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:40.430+0000 7f39d5ffb640 1 -- 172.21.15.27:0/178131991 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39c8003b70 con 0x7f39d8162040 2024-09-12T03:22:40.437 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:40.430+0000 7f39d5ffb640 1 -- 172.21.15.27:0/178131991 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39c8003d10 con 0x7f39d8162040 2024-09-12T03:22:40.438 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:40.430+0000 7f39df9c6640 1 -- 172.21.15.27:0/178131991 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39d80cd300 con 0x7f39d8162040 2024-09-12T03:22:40.438 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:40.430+0000 7f39d5ffb640 1 -- 172.21.15.27:0/178131991 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39c800aa90 con 0x7f39d8162040 2024-09-12T03:22:40.438 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:40.431+0000 7f39df9c6640 1 -- 172.21.15.27:0/178131991 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39d80cd8c0 con 0x7f39d8162040 2024-09-12T03:22:40.438 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:40.432+0000 7f39df9c6640 1 -- 172.21.15.27:0/178131991 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39d806e6e0 con 0x7f39d8162040 2024-09-12T03:22:40.438 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:40.432+0000 7f39d5ffb640 1 -- 172.21.15.27:0/178131991 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 22) ==== 91059+0+0 (secure 0 0 0) 0x7f39c800ac30 con 0x7f39d8162040 2024-09-12T03:22:40.438 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:40.432+0000 7f39d5ffb640 1 --2- 172.21.15.27:0/178131991 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f39c406c460 0x7f39c406e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:40.438 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:40.432+0000 7f39d5ffb640 1 -- 172.21.15.27:0/178131991 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5851+0+0 (secure 0 0 0) 0x7f39c808b850 con 0x7f39d8162040 2024-09-12T03:22:40.438 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:40.433+0000 7f39dd762640 1 --2- 172.21.15.27:0/178131991 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f39c406c460 0x7f39c406e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:40.439 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:40.433+0000 7f39dd762640 1 --2- 172.21.15.27:0/178131991 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f39c406c460 0x7f39c406e920 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f39d0007fd0 tx=0x7f39d0007dc0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:40.439 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:40.437+0000 7f39d5ffb640 1 -- 172.21.15.27:0/178131991 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f39c80598c0 con 0x7f39d8162040 2024-09-12T03:22:40.612 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:40.611+0000 7f39df9c6640 1 -- 172.21.15.27:0/178131991 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f39d81612e0 con 0x7f39d8162040 2024-09-12T03:22:41.490 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:41.489+0000 7f39d5ffb640 1 -- 172.21.15.27:0/178131991 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v128) ==== 181+0+0 (secure 0 0 0) 0x7f39c80591c0 con 0x7f39d8162040 2024-09-12T03:22:41.491 INFO:teuthology.orchestra.run.smithi027.stderr:pool 'cephfs_metadata' removed 2024-09-12T03:22:41.495 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:41.493+0000 7f39df9c6640 1 -- 172.21.15.27:0/178131991 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f39c406c460 msgr2=0x7f39c406e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:41.495 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:41.493+0000 7f39df9c6640 1 --2- 172.21.15.27:0/178131991 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f39c406c460 0x7f39c406e920 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f39d0007fd0 tx=0x7f39d0007dc0 comp rx=0 tx=0).stop 2024-09-12T03:22:41.495 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:41.493+0000 7f39df9c6640 1 -- 172.21.15.27:0/178131991 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f39d8162040 msgr2=0x7f39d815b080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:41.495 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:41.493+0000 7f39df9c6640 1 --2- 172.21.15.27:0/178131991 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f39d8162040 0x7f39d815b080 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7f39c800c730 tx=0x7f39c800cbf0 comp rx=0 tx=0).stop 2024-09-12T03:22:41.495 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:41.494+0000 7f39df9c6640 1 -- 172.21.15.27:0/178131991 shutdown_connections 2024-09-12T03:22:41.495 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:41.494+0000 7f39df9c6640 1 --2- 172.21.15.27:0/178131991 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f39c406c460 0x7f39c406e920 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:41.495 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:41.494+0000 7f39df9c6640 1 --2- 172.21.15.27:0/178131991 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f39d8162040 0x7f39d815b080 unknown :-1 s=CLOSED pgs=1354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:41.495 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:41.494+0000 7f39df9c6640 1 --2- 172.21.15.27:0/178131991 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f39d80d3790 0x7f39d81f4780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:41.495 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:41.494+0000 7f39df9c6640 1 --2- 172.21.15.27:0/178131991 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f39d806dfd0 0x7f39d814ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:41.495 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:41.494+0000 7f39df9c6640 1 -- 172.21.15.27:0/178131991 >> 172.21.15.27:0/178131991 conn(0x7f39d80b6700 msgr2=0x7f39d80cf790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:41.495 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:41.494+0000 7f39df9c6640 1 -- 172.21.15.27:0/178131991 shutdown_connections 2024-09-12T03:22:41.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:41.494+0000 7f39df9c6640 1 -- 172.21.15.27:0/178131991 wait complete. 2024-09-12T03:22:41.509 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-12T03:22:41.622 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:41.621+0000 7fea0e8ac640 1 Processor -- start 2024-09-12T03:22:41.622 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:41.621+0000 7fea0e8ac640 1 -- start start 2024-09-12T03:22:41.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:41.622+0000 7fea0e8ac640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fea080d3c20 0x7fea080d4040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:41.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:41.622+0000 7fea0e8ac640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fea080b77c0 0x7fea080b7c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:41.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:41.622+0000 7fea0e8ac640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fea080d2930 con 0x7fea080b6240 2024-09-12T03:22:41.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:41.622+0000 7fea0e8ac640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7fea080d2c10 con 0x7fea080b77c0 2024-09-12T03:22:41.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:41.622+0000 7fea0e8ac640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fea080d2ef0 con 0x7fea080d3c20 2024-09-12T03:22:41.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:41.622+0000 7fea0ce49640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fea080b77c0 0x7fea080b7c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:41.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:41.622+0000 7fea0ce49640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fea080b77c0 0x7fea080b7c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35334/0 (socket says 172.21.15.27:35334) 2024-09-12T03:22:41.628 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:41.622+0000 7fea0ce49640 1 -- 172.21.15.27:0/281553237 learned_addr learned my addr 172.21.15.27:0/281553237 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:22:41.628 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:41.622+0000 7fea077fe640 1 --2- 172.21.15.27:0/281553237 >> v2:172.21.15.45:3300/0 conn(0x7fea080d3c20 0x7fea080d4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:41.628 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:41.622+0000 7fea0ce49640 1 -- 172.21.15.27:0/281553237 >> v2:172.21.15.45:3300/0 conn(0x7fea080d3c20 msgr2=0x7fea080d4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:41.628 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:41.622+0000 7fea0ce49640 1 --2- 172.21.15.27:0/281553237 >> v2:172.21.15.45:3300/0 conn(0x7fea080d3c20 0x7fea080d4040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:41.628 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:41.622+0000 7fea0ce49640 1 -- 172.21.15.27:0/281553237 >> v1:172.21.15.45:6789/0 conn(0x7fea080b6240 legacy=0x7fea080b6620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:41.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:41.622+0000 7fea0ce49640 1 -- 172.21.15.27:0/281553237 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea080d31d0 con 0x7fea080b77c0 2024-09-12T03:22:41.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:41.622+0000 7fea077fe640 1 --2- 172.21.15.27:0/281553237 >> v2:172.21.15.45:3300/0 conn(0x7fea080d3c20 0x7fea080d4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:22:41.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:41.623+0000 7fea0ce49640 1 --2- 172.21.15.27:0/281553237 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fea080b77c0 0x7fea080b7c50 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7fea0808d410 tx=0x7fea0002a780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b97a59c73970de server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:41.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:41.623+0000 7fea06ffd640 1 -- 172.21.15.27:0/281553237 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea00036070 con 0x7fea080b77c0 2024-09-12T03:22:41.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:41.623+0000 7fea06ffd640 1 -- 172.21.15.27:0/281553237 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea00031600 con 0x7fea080b77c0 2024-09-12T03:22:41.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:41.623+0000 7fea06ffd640 1 -- 172.21.15.27:0/281553237 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea000319d0 con 0x7fea080b77c0 2024-09-12T03:22:41.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:41.624+0000 7fea0e8ac640 1 -- 172.21.15.27:0/281553237 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fea080b77c0 msgr2=0x7fea080b7c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:41.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:41.624+0000 7fea0e8ac640 1 --2- 172.21.15.27:0/281553237 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fea080b77c0 0x7fea080b7c50 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7fea0808d410 tx=0x7fea0002a780 comp rx=0 tx=0).stop 2024-09-12T03:22:41.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:41.624+0000 7fea0e8ac640 1 -- 172.21.15.27:0/281553237 shutdown_connections 2024-09-12T03:22:41.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:41.624+0000 7fea0e8ac640 1 --2- 172.21.15.27:0/281553237 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fea080b77c0 0x7fea080b7c50 unknown :-1 s=CLOSED pgs=1355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:41.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:41.624+0000 7fea0e8ac640 1 --2- 172.21.15.27:0/281553237 >> v2:172.21.15.45:3300/0 conn(0x7fea080d3c20 0x7fea080d4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:41.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:41.624+0000 7fea0e8ac640 1 -- 172.21.15.27:0/281553237 >> 172.21.15.27:0/281553237 conn(0x7fea080d1420 msgr2=0x7fea080d1830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:41.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:41.624+0000 7fea0e8ac640 1 -- 172.21.15.27:0/281553237 shutdown_connections 2024-09-12T03:22:41.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:41.624+0000 7fea0e8ac640 1 -- 172.21.15.27:0/281553237 wait complete. 2024-09-12T03:22:41.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:41.625+0000 7fea0e8ac640 1 Processor -- start 2024-09-12T03:22:41.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:41.625+0000 7fea0e8ac640 1 -- start start 2024-09-12T03:22:41.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:41.626+0000 7fea0e8ac640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fea080b6240 0x7fea08157fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:41.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:41.626+0000 7fea0e8ac640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fea080b77c0 0x7fea080cb1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:41.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:41.626+0000 7fea0e8ac640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fea080d3c20 0x7fea080ce660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:41.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:41.626+0000 7fea0e8ac640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fea080cfdf0 con 0x7fea080b77c0 2024-09-12T03:22:41.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:41.626+0000 7fea0e8ac640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fea080cfc70 con 0x7fea080d3c20 2024-09-12T03:22:41.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:41.626+0000 7fea0e8ac640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fea080b8190 con 0x7fea080b6240 2024-09-12T03:22:41.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:41.626+0000 7fea077fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fea080b77c0 0x7fea080cb1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:41.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:41.626+0000 7fea077fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fea080b77c0 0x7fea080cb1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35350/0 (socket says 172.21.15.27:35350) 2024-09-12T03:22:41.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:41.626+0000 7fea077fe640 1 -- 172.21.15.27:0/1819410222 learned_addr learned my addr 172.21.15.27:0/1819410222 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:22:41.638 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:41.626+0000 7fea0ce49640 1 --2- 172.21.15.27:0/1819410222 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fea080d3c20 0x7fea080ce660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:41.638 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:41.626+0000 7fea07fff640 1 --2- 172.21.15.27:0/1819410222 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fea080b6240 0x7fea08157fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:41.638 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:41.626+0000 7fea077fe640 1 -- 172.21.15.27:0/1819410222 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fea080b6240 msgr2=0x7fea08157fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:41.638 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:41.626+0000 7fea077fe640 1 --2- 172.21.15.27:0/1819410222 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fea080b6240 0x7fea08157fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:41.638 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:41.626+0000 7fea077fe640 1 -- 172.21.15.27:0/1819410222 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fea080d3c20 msgr2=0x7fea080ce660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:41.638 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:41.626+0000 7fea077fe640 1 --2- 172.21.15.27:0/1819410222 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fea080d3c20 0x7fea080ce660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:41.638 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:41.626+0000 7fea077fe640 1 -- 172.21.15.27:0/1819410222 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea080ceba0 con 0x7fea080b77c0 2024-09-12T03:22:41.638 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:41.627+0000 7fea077fe640 1 --2- 172.21.15.27:0/1819410222 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fea080b77c0 0x7fea080cb1d0 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7fe9fc00c7f0 tx=0x7fe9fc00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:41.638 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:41.627+0000 7fea04ff9640 1 -- 172.21.15.27:0/1819410222 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9fc015070 con 0x7fea080b77c0 2024-09-12T03:22:41.638 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:41.627+0000 7fea04ff9640 1 -- 172.21.15.27:0/1819410222 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9fc007820 con 0x7fea080b77c0 2024-09-12T03:22:41.638 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:41.627+0000 7fea0e8ac640 1 -- 172.21.15.27:0/1819410222 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea080cd2e0 con 0x7fea080b77c0 2024-09-12T03:22:41.639 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:41.627+0000 7fea04ff9640 1 -- 172.21.15.27:0/1819410222 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9fc007bd0 con 0x7fea080b77c0 2024-09-12T03:22:41.639 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:41.627+0000 7fea0e8ac640 1 -- 172.21.15.27:0/1819410222 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea080cd8a0 con 0x7fea080b77c0 2024-09-12T03:22:41.639 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:41.628+0000 7fea0e8ac640 1 -- 172.21.15.27:0/1819410222 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea080ce2a0 con 0x7fea080b77c0 2024-09-12T03:22:41.639 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:41.628+0000 7fea04ff9640 1 -- 172.21.15.27:0/1819410222 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 22) ==== 91059+0+0 (secure 0 0 0) 0x7fe9fc018070 con 0x7fea080b77c0 2024-09-12T03:22:41.639 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:41.629+0000 7fea04ff9640 1 --2- 172.21.15.27:0/1819410222 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fe9f406c460 0x7fe9f406e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:41.639 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:41.629+0000 7fea07fff640 1 --2- 172.21.15.27:0/1819410222 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fe9f406c460 0x7fe9f406e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:41.639 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:41.629+0000 7fea04ff9640 1 -- 172.21.15.27:0/1819410222 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 5413+0+0 (secure 0 0 0) 0x7fe9fc08ba10 con 0x7fea080b77c0 2024-09-12T03:22:41.639 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:41.629+0000 7fea07fff640 1 --2- 172.21.15.27:0/1819410222 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fe9f406c460 0x7fe9f406e920 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fe9f8004f60 tx=0x7fe9f8007e30 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:41.639 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:41.633+0000 7fea04ff9640 1 -- 172.21.15.27:0/1819410222 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fe9fc059440 con 0x7fea080b77c0 2024-09-12T03:22:41.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:41.812+0000 7fea0e8ac640 1 -- 172.21.15.27:0/1819410222 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fea080b5e60 con 0x7fea080b77c0 2024-09-12T03:22:42.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:42.491+0000 7fea04ff9640 1 -- 172.21.15.27:0/1819410222 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v129) ==== 169+0+0 (secure 0 0 0) 0x7fe9fc05d740 con 0x7fea080b77c0 2024-09-12T03:22:42.492 INFO:teuthology.orchestra.run.smithi027.stderr:pool 'cephfs_data' removed 2024-09-12T03:22:42.495 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:42.494+0000 7fea0e8ac640 1 -- 172.21.15.27:0/1819410222 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fe9f406c460 msgr2=0x7fe9f406e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:42.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:42.494+0000 7fea0e8ac640 1 --2- 172.21.15.27:0/1819410222 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fe9f406c460 0x7fe9f406e920 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fe9f8004f60 tx=0x7fe9f8007e30 comp rx=0 tx=0).stop 2024-09-12T03:22:42.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:42.494+0000 7fea0e8ac640 1 -- 172.21.15.27:0/1819410222 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fea080b77c0 msgr2=0x7fea080cb1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:42.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:42.494+0000 7fea0e8ac640 1 --2- 172.21.15.27:0/1819410222 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fea080b77c0 0x7fea080cb1d0 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7fe9fc00c7f0 tx=0x7fe9fc00ccb0 comp rx=0 tx=0).stop 2024-09-12T03:22:42.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:42.494+0000 7fea0e8ac640 1 -- 172.21.15.27:0/1819410222 shutdown_connections 2024-09-12T03:22:42.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:42.494+0000 7fea0e8ac640 1 --2- 172.21.15.27:0/1819410222 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fea080d3c20 0x7fea080ce660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:42.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:42.494+0000 7fea0e8ac640 1 --2- 172.21.15.27:0/1819410222 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fe9f406c460 0x7fe9f406e920 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:42.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:42.494+0000 7fea0e8ac640 1 --2- 172.21.15.27:0/1819410222 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fea080b77c0 0x7fea080cb1d0 unknown :-1 s=CLOSED pgs=1356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:42.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:42.494+0000 7fea0e8ac640 1 --2- 172.21.15.27:0/1819410222 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fea080b6240 0x7fea08157fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:42.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:42.494+0000 7fea0e8ac640 1 -- 172.21.15.27:0/1819410222 >> 172.21.15.27:0/1819410222 conn(0x7fea080d1420 msgr2=0x7fea080b5270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:42.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:42.494+0000 7fea0e8ac640 1 -- 172.21.15.27:0/1819410222 shutdown_connections 2024-09-12T03:22:42.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:42.495+0000 7fea0e8ac640 1 -- 172.21.15.27:0/1819410222 wait complete. 2024-09-12T03:22:42.513 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-09-12T03:22:42.624 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:42.623+0000 7f08c538d640 1 Processor -- start 2024-09-12T03:22:42.624 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:42.623+0000 7f08c538d640 1 -- start start 2024-09-12T03:22:42.624 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:42.624+0000 7f08c538d640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f08c00d31c0 0x7f08c00d0440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:42.624 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:42.624+0000 7f08c538d640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f08c006e960 0x7f08c00d0980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:42.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:42.624+0000 7f08c538d640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08c00cf9f0 con 0x7f08c0070d30 2024-09-12T03:22:42.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:42.624+0000 7f08c538d640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f08c00d1a40 con 0x7f08c006e960 2024-09-12T03:22:42.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:42.624+0000 7f08c538d640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f08c00cfc90 con 0x7f08c00d31c0 2024-09-12T03:22:42.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:42.624+0000 7f08bf7fe640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f08c006e960 0x7f08c00d0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:42.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:42.624+0000 7f08b67fc640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f08c00d31c0 0x7f08c00d0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:42.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:42.624+0000 7f08bf7fe640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f08c006e960 0x7f08c00d0980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.27:35624/0 (socket says 172.21.15.27:35624) 2024-09-12T03:22:42.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:42.624+0000 7f08bf7fe640 1 -- 172.21.15.27:0/3839660345 learned_addr learned my addr 172.21.15.27:0/3839660345 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:22:42.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:42.624+0000 7f08beffd640 1 --1- 172.21.15.27:0/3839660345 >> v1:172.21.15.45:6789/0 conn(0x7f08c0070d30 0x7f08c006cc40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.27:35768/0 (socket says 172.21.15.27:35768) 2024-09-12T03:22:42.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:42.624+0000 7f08b67fc640 1 -- 172.21.15.27:0/3839660345 >> v2:172.21.15.45:3300/0 conn(0x7f08c006e960 msgr2=0x7f08c00d0980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:42.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:42.624+0000 7f08b67fc640 1 --2- 172.21.15.27:0/3839660345 >> v2:172.21.15.45:3300/0 conn(0x7f08c006e960 0x7f08c00d0980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:42.628 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:42.624+0000 7f08b67fc640 1 -- 172.21.15.27:0/3839660345 >> v1:172.21.15.45:6789/0 conn(0x7f08c0070d30 legacy=0x7f08c006cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:42.628 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:42.624+0000 7f08b67fc640 1 -- 172.21.15.27:0/3839660345 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08c00cff70 con 0x7f08c00d31c0 2024-09-12T03:22:42.628 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:42.625+0000 7f08bf7fe640 1 --2- 172.21.15.27:0/3839660345 >> v2:172.21.15.45:3300/0 conn(0x7f08c006e960 0x7f08c00d0980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:22:42.628 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:42.625+0000 7f08b67fc640 1 --2- 172.21.15.27:0/3839660345 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f08c00d31c0 0x7f08c00d0440 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f08a80097d0 tx=0x7f08a802a0a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a496411276bdfcdd server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:42.628 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:42.626+0000 7f08be7fc640 1 -- 172.21.15.27:0/3839660345 <== mon.2 v2:172.21.15.45:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08a8037070 con 0x7f08c00d31c0 2024-09-12T03:22:42.628 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:42.626+0000 7f08be7fc640 1 -- 172.21.15.27:0/3839660345 <== mon.2 v2:172.21.15.45:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08a802adb0 con 0x7f08c00d31c0 2024-09-12T03:22:42.628 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:42.626+0000 7f08be7fc640 1 -- 172.21.15.27:0/3839660345 <== mon.2 v2:172.21.15.45:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08a8032660 con 0x7f08c00d31c0 2024-09-12T03:22:42.628 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:42.626+0000 7f08c538d640 1 -- 172.21.15.27:0/3839660345 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f08c00d31c0 msgr2=0x7f08c00d0440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:42.628 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:42.626+0000 7f08c538d640 1 --2- 172.21.15.27:0/3839660345 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f08c00d31c0 0x7f08c00d0440 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f08a80097d0 tx=0x7f08a802a0a0 comp rx=0 tx=0).stop 2024-09-12T03:22:42.628 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:42.626+0000 7f08c538d640 1 -- 172.21.15.27:0/3839660345 shutdown_connections 2024-09-12T03:22:42.628 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:42.626+0000 7f08c538d640 1 --2- 172.21.15.27:0/3839660345 >> v2:172.21.15.45:3300/0 conn(0x7f08c006e960 0x7f08c00d0980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:42.628 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:42.626+0000 7f08c538d640 1 --2- 172.21.15.27:0/3839660345 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f08c00d31c0 0x7f08c00d0440 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:42.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:42.626+0000 7f08c538d640 1 -- 172.21.15.27:0/3839660345 >> 172.21.15.27:0/3839660345 conn(0x7f08c00b7c10 msgr2=0x7f08c009ba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:42.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:42.626+0000 7f08c538d640 1 -- 172.21.15.27:0/3839660345 shutdown_connections 2024-09-12T03:22:42.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:42.626+0000 7f08c538d640 1 -- 172.21.15.27:0/3839660345 wait complete. 2024-09-12T03:22:42.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:42.627+0000 7f08c538d640 1 Processor -- start 2024-09-12T03:22:42.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:42.628+0000 7f08c538d640 1 -- start start 2024-09-12T03:22:42.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:42.628+0000 7f08c538d640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f08c006e960 0x7f08c01f57a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:42.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:42.628+0000 7f08c538d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f08c0070d30 0x7f08c0150e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:42.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:42.628+0000 7f08c538d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f08c00d31c0 0x7f08c00b5e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:42.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:42.628+0000 7f08c538d640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f08c0168480 con 0x7f08c00d31c0 2024-09-12T03:22:42.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:42.628+0000 7f08c538d640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f08c0168300 con 0x7f08c0070d30 2024-09-12T03:22:42.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:42.628+0000 7f08c538d640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f08c0168600 con 0x7f08c006e960 2024-09-12T03:22:42.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:42.628+0000 7f08bf7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f08c00d31c0 0x7f08c00b5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:42.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:42.628+0000 7f08bf7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f08c00d31c0 0x7f08c00b5e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35358/0 (socket says 172.21.15.27:35358) 2024-09-12T03:22:42.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:42.628+0000 7f08bf7fe640 1 -- 172.21.15.27:0/2320992060 learned_addr learned my addr 172.21.15.27:0/2320992060 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:22:42.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:42.629+0000 7f08b67fc640 1 --2- 172.21.15.27:0/2320992060 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f08c0070d30 0x7f08c0150e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:42.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:42.629+0000 7f08beffd640 1 --2- 172.21.15.27:0/2320992060 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f08c006e960 0x7f08c01f57a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:42.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:42.629+0000 7f08bf7fe640 1 -- 172.21.15.27:0/2320992060 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f08c006e960 msgr2=0x7f08c01f57a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:42.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:42.629+0000 7f08bf7fe640 1 --2- 172.21.15.27:0/2320992060 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f08c006e960 0x7f08c01f57a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:42.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:42.629+0000 7f08bf7fe640 1 -- 172.21.15.27:0/2320992060 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f08c0070d30 msgr2=0x7f08c0150e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:42.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:42.629+0000 7f08bf7fe640 1 --2- 172.21.15.27:0/2320992060 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f08c0070d30 0x7f08c0150e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:42.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:42.629+0000 7f08bf7fe640 1 -- 172.21.15.27:0/2320992060 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08c01f5ce0 con 0x7f08c00d31c0 2024-09-12T03:22:42.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:42.629+0000 7f08beffd640 1 --2- 172.21.15.27:0/2320992060 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f08c006e960 0x7f08c01f57a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:22:42.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:42.629+0000 7f08bf7fe640 1 --2- 172.21.15.27:0/2320992060 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f08c00d31c0 0x7f08c00b5e70 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7f08b800efc0 tx=0x7f08b800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:42.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:42.629+0000 7f08b7fff640 1 -- 172.21.15.27:0/2320992060 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08b800cd60 con 0x7f08c00d31c0 2024-09-12T03:22:42.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:42.629+0000 7f08b7fff640 1 -- 172.21.15.27:0/2320992060 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08b8019070 con 0x7f08c00d31c0 2024-09-12T03:22:42.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:42.629+0000 7f08c538d640 1 -- 172.21.15.27:0/2320992060 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08c0159cf0 con 0x7f08c00d31c0 2024-09-12T03:22:42.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:42.629+0000 7f08b7fff640 1 -- 172.21.15.27:0/2320992060 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08b80146c0 con 0x7f08c00d31c0 2024-09-12T03:22:42.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:42.629+0000 7f08c538d640 1 -- 172.21.15.27:0/2320992060 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08c015a2b0 con 0x7f08c00d31c0 2024-09-12T03:22:42.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:42.630+0000 7f08c538d640 1 -- 172.21.15.27:0/2320992060 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08c006cc40 con 0x7f08c00d31c0 2024-09-12T03:22:42.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:42.631+0000 7f08b7fff640 1 -- 172.21.15.27:0/2320992060 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 22) ==== 91059+0+0 (secure 0 0 0) 0x7f08b8005d10 con 0x7f08c00d31c0 2024-09-12T03:22:42.634 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:42.631+0000 7f08b7fff640 1 --2- 172.21.15.27:0/2320992060 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f08a006c460 0x7f08a006e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:42.634 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:42.631+0000 7f08b7fff640 1 -- 172.21.15.27:0/2320992060 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 4999+0+0 (secure 0 0 0) 0x7f08b808bb80 con 0x7f08c00d31c0 2024-09-12T03:22:42.634 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:42.631+0000 7f08beffd640 1 --2- 172.21.15.27:0/2320992060 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f08a006c460 0x7f08a006e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:42.634 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:42.632+0000 7f08beffd640 1 --2- 172.21.15.27:0/2320992060 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f08a006c460 0x7f08a006e920 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f08b0000f60 tx=0x7f08b0009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:42.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:42.635+0000 7f08b7fff640 1 -- 172.21.15.27:0/2320992060 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f08c01f5ce0 con 0x7f08c00d31c0 2024-09-12T03:22:42.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:42.816+0000 7f08c538d640 1 -- 172.21.15.27:0/2320992060 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f08c00b1e00 con 0x7f08c00d31c0 2024-09-12T03:22:42.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:42.819+0000 7f08b7fff640 1 -- 172.21.15.27:0/2320992060 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v25) ==== 104+0+0 (secure 0 0 0) 0x7f08b8010040 con 0x7f08c00d31c0 2024-09-12T03:22:42.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:42.822+0000 7f08c538d640 1 -- 172.21.15.27:0/2320992060 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f08a006c460 msgr2=0x7f08a006e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:42.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:42.822+0000 7f08c538d640 1 --2- 172.21.15.27:0/2320992060 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f08a006c460 0x7f08a006e920 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f08b0000f60 tx=0x7f08b0009040 comp rx=0 tx=0).stop 2024-09-12T03:22:42.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:42.822+0000 7f08c538d640 1 -- 172.21.15.27:0/2320992060 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f08c00d31c0 msgr2=0x7f08c00b5e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:42.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:42.822+0000 7f08c538d640 1 --2- 172.21.15.27:0/2320992060 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f08c00d31c0 0x7f08c00b5e70 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7f08b800efc0 tx=0x7f08b800c490 comp rx=0 tx=0).stop 2024-09-12T03:22:42.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:42.822+0000 7f08c538d640 1 -- 172.21.15.27:0/2320992060 shutdown_connections 2024-09-12T03:22:42.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:42.822+0000 7f08c538d640 1 --2- 172.21.15.27:0/2320992060 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f08a006c460 0x7f08a006e920 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:42.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:42.822+0000 7f08c538d640 1 --2- 172.21.15.27:0/2320992060 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f08c00d31c0 0x7f08c00b5e70 unknown :-1 s=CLOSED pgs=1357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:42.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:42.822+0000 7f08c538d640 1 --2- 172.21.15.27:0/2320992060 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f08c0070d30 0x7f08c0150e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:42.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:42.822+0000 7f08c538d640 1 --2- 172.21.15.27:0/2320992060 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f08c006e960 0x7f08c01f57a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:42.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:42.822+0000 7f08c538d640 1 -- 172.21.15.27:0/2320992060 >> 172.21.15.27:0/2320992060 conn(0x7f08c00b7c10 msgr2=0x7f08c00af4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:42.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:42.822+0000 7f08c538d640 1 -- 172.21.15.27:0/2320992060 shutdown_connections 2024-09-12T03:22:42.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:42.822+0000 7f08c538d640 1 -- 172.21.15.27:0/2320992060 wait complete. 2024-09-12T03:22:42.834 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_ops_on_nonexistent_vol' 2024-09-12T03:22:42.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:42.936+0000 7f9bb3d2c640 1 Processor -- start 2024-09-12T03:22:42.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:42.936+0000 7f9bb3d2c640 1 -- start start 2024-09-12T03:22:42.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:42.937+0000 7f9bb3d2c640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f9bac15b2b0 0x7f9bac0b3560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:42.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:42.937+0000 7f9bb3d2c640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f9bac06e9b0 0x7f9bac06ee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:42.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:42.937+0000 7f9bb3d2c640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9bac0ce410 con 0x7f9bac072d70 2024-09-12T03:22:42.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:42.937+0000 7f9bb3d2c640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f9bac0720a0 con 0x7f9bac15b2b0 2024-09-12T03:22:42.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:42.937+0000 7f9bb3d2c640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f9bac06f340 con 0x7f9bac06e9b0 2024-09-12T03:22:42.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:42.937+0000 7f9bb1ac8640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f9bac15b2b0 0x7f9bac0b3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:42.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:42.937+0000 7f9bb1ac8640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f9bac15b2b0 0x7f9bac0b3560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35366/0 (socket says 172.21.15.27:35366) 2024-09-12T03:22:42.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:42.937+0000 7f9bb1ac8640 1 -- 172.21.15.27:0/455801636 learned_addr learned my addr 172.21.15.27:0/455801636 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:22:42.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:42.937+0000 7f9bb22c9640 1 --2- 172.21.15.27:0/455801636 >> v2:172.21.15.45:3300/0 conn(0x7f9bac06e9b0 0x7f9bac06ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:42.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:42.937+0000 7f9bb1ac8640 1 -- 172.21.15.27:0/455801636 >> v2:172.21.15.45:3300/0 conn(0x7f9bac06e9b0 msgr2=0x7f9bac06ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:42.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:42.937+0000 7f9bb1ac8640 1 --2- 172.21.15.27:0/455801636 >> v2:172.21.15.45:3300/0 conn(0x7f9bac06e9b0 0x7f9bac06ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:42.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:42.937+0000 7f9bb1ac8640 1 -- 172.21.15.27:0/455801636 >> v1:172.21.15.45:6789/0 conn(0x7f9bac072d70 legacy=0x7f9bac072430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:42.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:42.937+0000 7f9bb1ac8640 1 -- 172.21.15.27:0/455801636 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bac0732c0 con 0x7f9bac15b2b0 2024-09-12T03:22:42.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:42.937+0000 7f9bb22c9640 1 --2- 172.21.15.27:0/455801636 >> v2:172.21.15.45:3300/0 conn(0x7f9bac06e9b0 0x7f9bac06ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:22:42.942 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:42.938+0000 7f9bb1ac8640 1 --2- 172.21.15.27:0/455801636 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9bac15b2b0 0x7f9bac0b3560 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7f9bac0ce1e0 tx=0x7f9ba802c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4eeb4786359d9648 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:42.942 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:42.938+0000 7f9bb0ac6640 1 -- 172.21.15.27:0/455801636 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ba80325a0 con 0x7f9bac15b2b0 2024-09-12T03:22:42.942 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:42.938+0000 7f9bb0ac6640 1 -- 172.21.15.27:0/455801636 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ba8038070 con 0x7f9bac15b2b0 2024-09-12T03:22:42.942 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:42.938+0000 7f9bb0ac6640 1 -- 172.21.15.27:0/455801636 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ba8032960 con 0x7f9bac15b2b0 2024-09-12T03:22:42.942 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:42.939+0000 7f9bb3d2c640 1 -- 172.21.15.27:0/455801636 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9bac15b2b0 msgr2=0x7f9bac0b3560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:42.942 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:42.939+0000 7f9bb3d2c640 1 --2- 172.21.15.27:0/455801636 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9bac15b2b0 0x7f9bac0b3560 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7f9bac0ce1e0 tx=0x7f9ba802c8e0 comp rx=0 tx=0).stop 2024-09-12T03:22:42.943 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:42.939+0000 7f9bb3d2c640 1 -- 172.21.15.27:0/455801636 shutdown_connections 2024-09-12T03:22:42.943 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:42.939+0000 7f9bb3d2c640 1 --2- 172.21.15.27:0/455801636 >> v2:172.21.15.45:3300/0 conn(0x7f9bac06e9b0 0x7f9bac06ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:42.943 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:42.939+0000 7f9bb3d2c640 1 --2- 172.21.15.27:0/455801636 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9bac15b2b0 0x7f9bac0b3560 unknown :-1 s=CLOSED pgs=1358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:42.943 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:42.939+0000 7f9bb3d2c640 1 -- 172.21.15.27:0/455801636 >> 172.21.15.27:0/455801636 conn(0x7f9bac062c60 msgr2=0x7f9bac06bb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:42.943 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:42.939+0000 7f9bb3d2c640 1 -- 172.21.15.27:0/455801636 shutdown_connections 2024-09-12T03:22:42.943 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:42.939+0000 7f9bb3d2c640 1 -- 172.21.15.27:0/455801636 wait complete. 2024-09-12T03:22:42.947 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:42.940+0000 7f9bb3d2c640 1 Processor -- start 2024-09-12T03:22:42.947 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:42.940+0000 7f9bb3d2c640 1 -- start start 2024-09-12T03:22:42.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:42.941+0000 7f9bb3d2c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9bac06e9b0 0x7f9bac1f7450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:42.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:42.941+0000 7f9bb3d2c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9bac072d70 0x7f9bac1561b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:42.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:42.941+0000 7f9bb3d2c640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f9bac15b2b0 0x7f9bac1566f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:42.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:42.941+0000 7f9bb3d2c640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f9bac0ce690 con 0x7f9bac072d70 2024-09-12T03:22:42.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:42.941+0000 7f9bb3d2c640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f9bac071dc0 con 0x7f9bac06e9b0 2024-09-12T03:22:42.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:42.941+0000 7f9bb3d2c640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f9bac15b690 con 0x7f9bac15b2b0 2024-09-12T03:22:42.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:42.941+0000 7f9bb12c7640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9bac072d70 0x7f9bac1561b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:42.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:42.941+0000 7f9bb12c7640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9bac072d70 0x7f9bac1561b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35368/0 (socket says 172.21.15.27:35368) 2024-09-12T03:22:42.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:42.941+0000 7f9bb12c7640 1 -- 172.21.15.27:0/486094427 learned_addr learned my addr 172.21.15.27:0/486094427 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:22:42.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:42.941+0000 7f9bb1ac8640 1 --2- 172.21.15.27:0/486094427 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9bac06e9b0 0x7f9bac1f7450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:42.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:42.941+0000 7f9bb22c9640 1 --2- 172.21.15.27:0/486094427 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f9bac15b2b0 0x7f9bac1566f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:42.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:42.941+0000 7f9bb12c7640 1 -- 172.21.15.27:0/486094427 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f9bac15b2b0 msgr2=0x7f9bac1566f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:42.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:42.941+0000 7f9bb12c7640 1 --2- 172.21.15.27:0/486094427 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f9bac15b2b0 0x7f9bac1566f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:42.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:42.941+0000 7f9bb12c7640 1 -- 172.21.15.27:0/486094427 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9bac06e9b0 msgr2=0x7f9bac1f7450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:42.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:42.941+0000 7f9bb12c7640 1 --2- 172.21.15.27:0/486094427 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9bac06e9b0 0x7f9bac1f7450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:42.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:42.941+0000 7f9bb12c7640 1 -- 172.21.15.27:0/486094427 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bac159bf0 con 0x7f9bac072d70 2024-09-12T03:22:42.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:42.941+0000 7f9bb22c9640 1 --2- 172.21.15.27:0/486094427 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f9bac15b2b0 0x7f9bac1566f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:22:42.950 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:42.941+0000 7f9bb12c7640 1 --2- 172.21.15.27:0/486094427 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9bac072d70 0x7f9bac1561b0 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7f9b9c00c460 tx=0x7f9b9c00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:42.950 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:42.941+0000 7f9ba27fc640 1 -- 172.21.15.27:0/486094427 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b9c016020 con 0x7f9bac072d70 2024-09-12T03:22:42.950 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:42.941+0000 7f9ba27fc640 1 -- 172.21.15.27:0/486094427 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b9c012870 con 0x7f9bac072d70 2024-09-12T03:22:42.950 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:42.941+0000 7f9bb3d2c640 1 -- 172.21.15.27:0/486094427 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9bac159ed0 con 0x7f9bac072d70 2024-09-12T03:22:42.950 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:42.941+0000 7f9ba27fc640 1 -- 172.21.15.27:0/486094427 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b9c012c60 con 0x7f9bac072d70 2024-09-12T03:22:42.950 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:42.942+0000 7f9bb3d2c640 1 -- 172.21.15.27:0/486094427 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9bac15a490 con 0x7f9bac072d70 2024-09-12T03:22:42.950 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:42.943+0000 7f9bb3d2c640 1 -- 172.21.15.27:0/486094427 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9bac157460 con 0x7f9bac072d70 2024-09-12T03:22:42.950 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:42.943+0000 7f9ba27fc640 1 -- 172.21.15.27:0/486094427 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 22) ==== 91059+0+0 (secure 0 0 0) 0x7f9b9c019020 con 0x7f9bac072d70 2024-09-12T03:22:42.950 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:42.943+0000 7f9ba27fc640 1 --2- 172.21.15.27:0/486094427 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f9b9006c490 0x7f9b9006e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:42.950 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:42.943+0000 7f9ba27fc640 1 -- 172.21.15.27:0/486094427 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 4999+0+0 (secure 0 0 0) 0x7f9b9c08b2e0 con 0x7f9bac072d70 2024-09-12T03:22:42.950 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:42.944+0000 7f9bb1ac8640 1 --2- 172.21.15.27:0/486094427 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f9b9006c490 0x7f9b9006e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:42.951 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:42.944+0000 7f9bb1ac8640 1 --2- 172.21.15.27:0/486094427 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f9b9006c490 0x7f9b9006e950 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f9ba8037000 tx=0x7f9ba8032000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:42.951 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:42.948+0000 7f9ba27fc640 1 -- 172.21.15.27:0/486094427 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f9b9c020030 con 0x7f9bac072d70 2024-09-12T03:22:43.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:43.194+0000 7f9bb3d2c640 1 -- 172.21.15.27:0/486094427 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_ops_on_nonexistent_vol"]} v 0) -- 0x7f9bac1f7990 con 0x7f9bac072d70 2024-09-12T03:22:43.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:43.501+0000 7f9ba27fc640 1 -- 172.21.15.27:0/486094427 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_ops_on_nonexistent_vol"]}]=0 v342) ==== 151+0+0 (secure 0 0 0) 0x7f9b9c059610 con 0x7f9bac072d70 2024-09-12T03:22:43.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:43.503+0000 7f9bb3d2c640 1 -- 172.21.15.27:0/486094427 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f9b9006c490 msgr2=0x7f9b9006e950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:43.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:43.503+0000 7f9bb3d2c640 1 --2- 172.21.15.27:0/486094427 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f9b9006c490 0x7f9b9006e950 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f9ba8037000 tx=0x7f9ba8032000 comp rx=0 tx=0).stop 2024-09-12T03:22:43.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:43.503+0000 7f9bb3d2c640 1 -- 172.21.15.27:0/486094427 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9bac072d70 msgr2=0x7f9bac1561b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:43.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:43.503+0000 7f9bb3d2c640 1 --2- 172.21.15.27:0/486094427 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9bac072d70 0x7f9bac1561b0 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7f9b9c00c460 tx=0x7f9b9c00c920 comp rx=0 tx=0).stop 2024-09-12T03:22:43.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:43.503+0000 7f9bb3d2c640 1 -- 172.21.15.27:0/486094427 shutdown_connections 2024-09-12T03:22:43.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:43.503+0000 7f9bb3d2c640 1 --2- 172.21.15.27:0/486094427 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f9bac15b2b0 0x7f9bac1566f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:43.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:43.503+0000 7f9bb3d2c640 1 --2- 172.21.15.27:0/486094427 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f9b9006c490 0x7f9b9006e950 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:43.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:43.503+0000 7f9bb3d2c640 1 --2- 172.21.15.27:0/486094427 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9bac072d70 0x7f9bac1561b0 unknown :-1 s=CLOSED pgs=1359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:43.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:43.503+0000 7f9bb3d2c640 1 --2- 172.21.15.27:0/486094427 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9bac06e9b0 0x7f9bac1f7450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:43.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:43.503+0000 7f9bb3d2c640 1 -- 172.21.15.27:0/486094427 >> 172.21.15.27:0/486094427 conn(0x7f9bac062c60 msgr2=0x7f9bac14fcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:43.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:43.504+0000 7f9bb3d2c640 1 -- 172.21.15.27:0/486094427 shutdown_connections 2024-09-12T03:22:43.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:43.504+0000 7f9bb3d2c640 1 -- 172.21.15.27:0/486094427 wait complete. 2024-09-12T03:22:43.516 INFO:tasks.cephfs_test_runner:test_subvolume_ops_on_nonexistent_vol (tasks.cephfs.test_volumes.TestMisc) ... ok 2024-09-12T03:22:43.516 INFO:tasks.cephfs_test_runner:Starting test: test_subvolume_upgrade_legacy_to_v1 (tasks.cephfs.test_volumes.TestMisc) poor man's upgrade test -- rather than going through a full upgrade cycle, 2024-09-12T03:22:43.516 INFO:tasks.cephfs_test_runner:test_subvolume_upgrade_legacy_to_v1 (tasks.cephfs.test_volumes.TestMisc) 2024-09-12T03:22:43.517 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-12T03:22:43.620 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:43.619+0000 7f4b5d354640 1 Processor -- start 2024-09-12T03:22:43.621 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:43.619+0000 7f4b5d354640 1 -- start start 2024-09-12T03:22:43.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:43.620+0000 7f4b5d354640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4b580d1050 0x7f4b580cf810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:43.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:43.620+0000 7f4b5d354640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f4b580d52e0 0x7f4b580d1430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:43.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:43.620+0000 7f4b5d354640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4b580d1e60 con 0x7f4b58162520 2024-09-12T03:22:43.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:43.620+0000 7f4b5d354640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f4b580d2380 con 0x7f4b580d52e0 2024-09-12T03:22:43.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:43.620+0000 7f4b5d354640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f4b580cfef0 con 0x7f4b580d1050 2024-09-12T03:22:43.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:43.620+0000 7f4b567fc640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f4b580d52e0 0x7f4b580d1430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:43.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:43.620+0000 7f4b577fe640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f4b58162520 0x7f4b581629d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:55370/0 (socket says 172.21.15.27:55370) 2024-09-12T03:22:43.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:43.620+0000 7f4b577fe640 1 -- 172.21.15.27:0/1062169414 learned_addr learned my addr 172.21.15.27:0/1062169414 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:22:43.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:43.620+0000 7f4b56ffd640 1 --2- 172.21.15.27:0/1062169414 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4b580d1050 0x7f4b580cf810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:43.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:43.620+0000 7f4b567fc640 1 -- 172.21.15.27:0/1062169414 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4b580d1050 msgr2=0x7f4b580cf810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:43.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:43.620+0000 7f4b567fc640 1 --2- 172.21.15.27:0/1062169414 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4b580d1050 0x7f4b580cf810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:43.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:43.620+0000 7f4b567fc640 1 -- 172.21.15.27:0/1062169414 >> v1:172.21.15.27:6789/0 conn(0x7f4b58162520 legacy=0x7f4b581629d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:43.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:43.620+0000 7f4b567fc640 1 -- 172.21.15.27:0/1062169414 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b580d2640 con 0x7f4b580d52e0 2024-09-12T03:22:43.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:43.620+0000 7f4b55ffb640 1 -- 172.21.15.27:0/1062169414 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3188451692 0 0) 0x7f4b580d1e60 con 0x7f4b58162520 2024-09-12T03:22:43.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:43.621+0000 7f4b567fc640 1 --2- 172.21.15.27:0/1062169414 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4b580d52e0 0x7f4b580d1430 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f4b4c0097a0 tx=0x7f4b4c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b958f0c6170133e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:43.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:43.621+0000 7f4b55ffb640 1 -- 172.21.15.27:0/1062169414 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b4c037070 con 0x7f4b580d52e0 2024-09-12T03:22:43.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:43.621+0000 7f4b55ffb640 1 -- 172.21.15.27:0/1062169414 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b4c032440 con 0x7f4b580d52e0 2024-09-12T03:22:43.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:43.621+0000 7f4b55ffb640 1 -- 172.21.15.27:0/1062169414 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b4c032810 con 0x7f4b580d52e0 2024-09-12T03:22:43.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:43.622+0000 7f4b5d354640 1 -- 172.21.15.27:0/1062169414 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4b580d52e0 msgr2=0x7f4b580d1430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:43.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:43.622+0000 7f4b5d354640 1 --2- 172.21.15.27:0/1062169414 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4b580d52e0 0x7f4b580d1430 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f4b4c0097a0 tx=0x7f4b4c02a180 comp rx=0 tx=0).stop 2024-09-12T03:22:43.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:43.622+0000 7f4b5d354640 1 -- 172.21.15.27:0/1062169414 shutdown_connections 2024-09-12T03:22:43.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:43.622+0000 7f4b5d354640 1 --2- 172.21.15.27:0/1062169414 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4b580d52e0 0x7f4b580d1430 unknown :-1 s=CLOSED pgs=1361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:43.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:43.622+0000 7f4b5d354640 1 --2- 172.21.15.27:0/1062169414 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4b580d1050 0x7f4b580cf810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:43.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:43.622+0000 7f4b5d354640 1 -- 172.21.15.27:0/1062169414 >> 172.21.15.27:0/1062169414 conn(0x7f4b5806ddf0 msgr2=0x7f4b5806e200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:43.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:43.622+0000 7f4b5d354640 1 -- 172.21.15.27:0/1062169414 shutdown_connections 2024-09-12T03:22:43.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:43.622+0000 7f4b5d354640 1 -- 172.21.15.27:0/1062169414 wait complete. 2024-09-12T03:22:43.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:43.623+0000 7f4b5d354640 1 Processor -- start 2024-09-12T03:22:43.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:43.623+0000 7f4b5d354640 1 -- start start 2024-09-12T03:22:43.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:43.624+0000 7f4b5d354640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4b580d1050 0x7f4b5814a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:43.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:43.624+0000 7f4b5d354640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4b580d52e0 0x7f4b580cb880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:43.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:43.624+0000 7f4b5d354640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b58162520 0x7f4b580cbdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:43.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:43.624+0000 7f4b5d354640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f4b580d1970 con 0x7f4b580d1050 2024-09-12T03:22:43.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:43.624+0000 7f4b5d354640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f4b580d2c00 con 0x7f4b58162520 2024-09-12T03:22:43.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:43.624+0000 7f4b5d354640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f4b580b7a50 con 0x7f4b580d52e0 2024-09-12T03:22:43.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:43.624+0000 7f4b56ffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4b580d1050 0x7f4b5814a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:43.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:43.624+0000 7f4b56ffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4b580d1050 0x7f4b5814a830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35372/0 (socket says 172.21.15.27:35372) 2024-09-12T03:22:43.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:43.624+0000 7f4b56ffd640 1 -- 172.21.15.27:0/3249471384 learned_addr learned my addr 172.21.15.27:0/3249471384 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:22:43.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:43.624+0000 7f4b577fe640 1 --2- 172.21.15.27:0/3249471384 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b58162520 0x7f4b580cbdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:43.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:43.624+0000 7f4b567fc640 1 --2- 172.21.15.27:0/3249471384 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4b580d52e0 0x7f4b580cb880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:43.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:43.625+0000 7f4b56ffd640 1 -- 172.21.15.27:0/3249471384 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4b580d52e0 msgr2=0x7f4b580cb880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:43.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:43.625+0000 7f4b56ffd640 1 --2- 172.21.15.27:0/3249471384 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4b580d52e0 0x7f4b580cb880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:43.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:43.625+0000 7f4b56ffd640 1 -- 172.21.15.27:0/3249471384 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b58162520 msgr2=0x7f4b580cbdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:43.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:43.625+0000 7f4b56ffd640 1 --2- 172.21.15.27:0/3249471384 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b58162520 0x7f4b580cbdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:43.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:43.625+0000 7f4b56ffd640 1 -- 172.21.15.27:0/3249471384 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b580b47b0 con 0x7f4b580d1050 2024-09-12T03:22:43.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:43.625+0000 7f4b56ffd640 1 --2- 172.21.15.27:0/3249471384 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4b580d1050 0x7f4b5814a830 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7f4b48007e70 tx=0x7f4b4800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:43.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:43.625+0000 7f4b3f7fe640 1 -- 172.21.15.27:0/3249471384 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b48014620 con 0x7f4b580d1050 2024-09-12T03:22:43.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:43.625+0000 7f4b3f7fe640 1 -- 172.21.15.27:0/3249471384 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b48019070 con 0x7f4b580d1050 2024-09-12T03:22:43.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:43.625+0000 7f4b5d354640 1 -- 172.21.15.27:0/3249471384 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b580b4a90 con 0x7f4b580d1050 2024-09-12T03:22:43.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:43.625+0000 7f4b3f7fe640 1 -- 172.21.15.27:0/3249471384 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b48014a00 con 0x7f4b580d1050 2024-09-12T03:22:43.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:43.625+0000 7f4b5d354640 1 -- 172.21.15.27:0/3249471384 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b580b5050 con 0x7f4b580d1050 2024-09-12T03:22:43.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:43.626+0000 7f4b5d354640 1 -- 172.21.15.27:0/3249471384 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b5815ec60 con 0x7f4b580d1050 2024-09-12T03:22:43.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:43.626+0000 7f4b3f7fe640 1 -- 172.21.15.27:0/3249471384 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 22) ==== 91059+0+0 (secure 0 0 0) 0x7f4b48014ba0 con 0x7f4b580d1050 2024-09-12T03:22:43.634 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:43.627+0000 7f4b3f7fe640 1 --2- 172.21.15.27:0/3249471384 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f4b4406c460 0x7f4b4406e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:43.634 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:43.627+0000 7f4b3f7fe640 1 -- 172.21.15.27:0/3249471384 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 4999+0+0 (secure 0 0 0) 0x7f4b4808be50 con 0x7f4b580d1050 2024-09-12T03:22:43.634 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:43.627+0000 7f4b567fc640 1 --2- 172.21.15.27:0/3249471384 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f4b4406c460 0x7f4b4406e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:43.634 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:43.627+0000 7f4b567fc640 1 --2- 172.21.15.27:0/3249471384 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f4b4406c460 0x7f4b4406e920 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f4b4c034000 tx=0x7f4b4c02e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:43.634 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:43.631+0000 7f4b3f7fe640 1 -- 172.21.15.27:0/3249471384 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f4b48059bc0 con 0x7f4b580d1050 2024-09-12T03:22:43.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:43.807+0000 7f4b5d354640 1 -- 172.21.15.27:0/3249471384 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4b580d20a0 con 0x7f4b580d1050 2024-09-12T03:22:43.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:43.808+0000 7f4b3f7fe640 1 -- 172.21.15.27:0/3249471384 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v129) ==== 74+0+13083 (secure 0 0 0) 0x7f4b4805dec0 con 0x7f4b580d1050 2024-09-12T03:22:43.809 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:22:43.809 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":129,"fsid":"d2df504e-2647-4385-a5db-9fdf69f487a0","created":"2024-09-12T03:16:07.778937+0000","modified":"2024-09-12T03:22:42.491412+0000","last_up_change":"2024-09-12T03:16:19.314778+0000","last_in_change":"2024-09-12T03:16:09.893086+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T03:16:20.097646+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1851116c-1fb3-4dc9-9772-f90e2ef4c355","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6817","nonce":3002321488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6819","nonce":3002321488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6823","nonce":3002321488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6821","nonce":3002321488}]},"public_addr":"172.21.15.27:6817/3002321488","cluster_addr":"172.21.15.27:6819/3002321488","heartbeat_back_addr":"172.21.15.27:6823/3002321488","heartbeat_front_addr":"172.21.15.27:6821/3002321488","state":["exists","up"]},{"osd":1,"uuid":"f38ae17d-73eb-4204-b99d-975fa806fb87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6809","nonce":1613102886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6811","nonce":1613102886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6815","nonce":1613102886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6813","nonce":1613102886}]},"public_addr":"172.21.15.27:6809/1613102886","cluster_addr":"172.21.15.27:6811/1613102886","heartbeat_back_addr":"172.21.15.27:6815/1613102886","heartbeat_front_addr":"172.21.15.27:6813/1613102886","state":["exists","up"]},{"osd":2,"uuid":"fb12dcc8-bdd0-42b8-b46d-8b10106224bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6825","nonce":3534723355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6827","nonce":3534723355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6831","nonce":3534723355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6829","nonce":3534723355}]},"public_addr":"172.21.15.27:6825/3534723355","cluster_addr":"172.21.15.27:6827/3534723355","heartbeat_back_addr":"172.21.15.27:6831/3534723355","heartbeat_front_addr":"172.21.15.27:6829/3534723355","state":["exists","up"]},{"osd":3,"uuid":"d74d3f49-9c24-4b9f-98e2-3ecd7b5ef3b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6801","nonce":80661598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6803","nonce":80661598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6807","nonce":80661598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6805","nonce":80661598}]},"public_addr":"172.21.15.27:6801/80661598","cluster_addr":"172.21.15.27:6803/80661598","heartbeat_back_addr":"172.21.15.27:6807/80661598","heartbeat_front_addr":"172.21.15.27:6805/80661598","state":["exists","up"]},{"osd":4,"uuid":"2bf123f2-1b0d-41c4-aa3b-805bab267e59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6817","nonce":3943189827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6819","nonce":3943189827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6823","nonce":3943189827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6821","nonce":3943189827}]},"public_addr":"172.21.15.45:6817/3943189827","cluster_addr":"172.21.15.45:6819/3943189827","heartbeat_back_addr":"172.21.15.45:6823/3943189827","heartbeat_front_addr":"172.21.15.45:6821/3943189827","state":["exists","up"]},{"osd":5,"uuid":"c49ba5da-91c6-4a86-a97b-4aba89c253ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6809","nonce":3620760330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6811","nonce":3620760330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6815","nonce":3620760330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6813","nonce":3620760330}]},"public_addr":"172.21.15.45:6809/3620760330","cluster_addr":"172.21.15.45:6811/3620760330","heartbeat_back_addr":"172.21.15.45:6815/3620760330","heartbeat_front_addr":"172.21.15.45:6813/3620760330","state":["exists","up"]},{"osd":6,"uuid":"b0123475-89f7-4a34-9e33-d797b99dd9dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6801","nonce":1989083510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6803","nonce":1989083510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6807","nonce":1989083510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6805","nonce":1989083510}]},"public_addr":"172.21.15.45:6801/1989083510","cluster_addr":"172.21.15.45:6803/1989083510","heartbeat_back_addr":"172.21.15.45:6807/1989083510","heartbeat_front_addr":"172.21.15.45:6805/1989083510","state":["exists","up"]},{"osd":7,"uuid":"4dad773d-b7d8-46fc-8a10-a972aad27b29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6825","nonce":1513300294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6827","nonce":1513300294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6831","nonce":1513300294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6829","nonce":1513300294}]},"public_addr":"172.21.15.45:6825/1513300294","cluster_addr":"172.21.15.45:6827/1513300294","heartbeat_back_addr":"172.21.15.45:6831/1513300294","heartbeat_front_addr":"172.21.15.45:6829/1513300294","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.881855+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:17.133252+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.735339+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.500742+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.477571+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:17.167667+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.470291+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.873743+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.27:6834/1711260376":"2024-09-13T03:22:39.257551+0000","172.21.15.27:6832/1711260376":"2024-09-13T03:22:39.257551+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]},{"pool":19,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T03:22:43.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:43.810+0000 7f4b5d354640 1 -- 172.21.15.27:0/3249471384 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f4b4406c460 msgr2=0x7f4b4406e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:43.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:43.810+0000 7f4b5d354640 1 --2- 172.21.15.27:0/3249471384 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f4b4406c460 0x7f4b4406e920 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f4b4c034000 tx=0x7f4b4c02e000 comp rx=0 tx=0).stop 2024-09-12T03:22:43.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:43.810+0000 7f4b5d354640 1 -- 172.21.15.27:0/3249471384 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4b580d1050 msgr2=0x7f4b5814a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:43.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:43.810+0000 7f4b5d354640 1 --2- 172.21.15.27:0/3249471384 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4b580d1050 0x7f4b5814a830 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7f4b48007e70 tx=0x7f4b4800e4b0 comp rx=0 tx=0).stop 2024-09-12T03:22:43.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:43.810+0000 7f4b5d354640 1 -- 172.21.15.27:0/3249471384 shutdown_connections 2024-09-12T03:22:43.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:43.810+0000 7f4b5d354640 1 --2- 172.21.15.27:0/3249471384 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b58162520 0x7f4b580cbdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:43.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:43.810+0000 7f4b5d354640 1 --2- 172.21.15.27:0/3249471384 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4b580d52e0 0x7f4b580cb880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:43.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:43.810+0000 7f4b5d354640 1 --2- 172.21.15.27:0/3249471384 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f4b4406c460 0x7f4b4406e920 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:43.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:43.810+0000 7f4b5d354640 1 --2- 172.21.15.27:0/3249471384 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4b580d1050 0x7f4b5814a830 unknown :-1 s=CLOSED pgs=1362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:43.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:43.810+0000 7f4b5d354640 1 -- 172.21.15.27:0/3249471384 >> 172.21.15.27:0/3249471384 conn(0x7f4b5806ddf0 msgr2=0x7f4b58162bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:43.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:43.810+0000 7f4b5d354640 1 -- 172.21.15.27:0/3249471384 shutdown_connections 2024-09-12T03:22:43.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:43.810+0000 7f4b5d354640 1 -- 172.21.15.27:0/3249471384 wait complete. 2024-09-12T03:22:43.822 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-12T03:16:20.097646+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-12T03:22:43.822 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-12T03:22:43.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:43.927+0000 7f6c96c5e640 1 Processor -- start 2024-09-12T03:22:43.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:43.928+0000 7f6c96c5e640 1 -- start start 2024-09-12T03:22:43.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:43.928+0000 7f6c96c5e640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f6c900b64a0 0x7f6c900d0830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:43.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:43.928+0000 7f6c96c5e640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6c900d3390 con 0x7f6c901621c0 2024-09-12T03:22:43.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:43.928+0000 7f6c96c5e640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6c900d3640 con 0x7f6c900d40e0 2024-09-12T03:22:43.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:43.928+0000 7f6c96c5e640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f6c900cff80 con 0x7f6c900b64a0 2024-09-12T03:22:43.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:43.928+0000 7f6c951fb640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f6c901621c0 0x7f6c900b79e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:55386/0 (socket says 172.21.15.27:55386) 2024-09-12T03:22:43.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:43.928+0000 7f6c951fb640 1 -- 172.21.15.27:0/4200718340 learned_addr learned my addr 172.21.15.27:0/4200718340 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:22:43.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:43.929+0000 7f6c8f7fe640 1 -- 172.21.15.27:0/4200718340 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1857889458 0 0) 0x7f6c900d3390 con 0x7f6c901621c0 2024-09-12T03:22:43.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:43.929+0000 7f6c8f7fe640 1 -- 172.21.15.27:0/4200718340 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6c7c003520 con 0x7f6c901621c0 2024-09-12T03:22:43.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:43.929+0000 7f6c949fa640 1 --2- 172.21.15.27:0/4200718340 >> v2:172.21.15.45:3300/0 conn(0x7f6c900b64a0 0x7f6c900d0830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:43.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:43.929+0000 7f6c8f7fe640 1 -- 172.21.15.27:0/4200718340 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 902156362 0 0) 0x7f6c7c003520 con 0x7f6c901621c0 2024-09-12T03:22:43.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:43.929+0000 7f6c8f7fe640 1 -- 172.21.15.27:0/4200718340 >> v2:172.21.15.45:3300/0 conn(0x7f6c900b64a0 msgr2=0x7f6c900d0830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:43.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:43.929+0000 7f6c8f7fe640 1 --2- 172.21.15.27:0/4200718340 >> v2:172.21.15.45:3300/0 conn(0x7f6c900b64a0 0x7f6c900d0830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:43.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:43.929+0000 7f6c8f7fe640 1 -- 172.21.15.27:0/4200718340 >> v1:172.21.15.45:6789/0 conn(0x7f6c900d40e0 legacy=0x7f6c90161ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:43.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:43.929+0000 7f6c8f7fe640 1 -- 172.21.15.27:0/4200718340 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c900d3920 con 0x7f6c901621c0 2024-09-12T03:22:43.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:43.929+0000 7f6c8f7fe640 1 -- 172.21.15.27:0/4200718340 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 847755213 0 0) 0x7f6c78001670 con 0x7f6c901621c0 2024-09-12T03:22:43.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:43.929+0000 7f6c8f7fe640 1 -- 172.21.15.27:0/4200718340 >> v1:172.21.15.27:6789/0 conn(0x7f6c901621c0 legacy=0x7f6c900b79e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:43.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:43.929+0000 7f6c8f7fe640 1 --2- 172.21.15.27:0/4200718340 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6c7c0039b0 0x7f6c7c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:43.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:43.929+0000 7f6c8f7fe640 1 --2- 172.21.15.27:0/4200718340 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c7c004800 0x7f6c7c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:43.934 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:43.929+0000 7f6c8f7fe640 1 --2- 172.21.15.27:0/4200718340 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f6c7c0051c0 0x7f6c7c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:43.934 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:43.929+0000 7f6c8f7fe640 1 -- 172.21.15.27:0/4200718340 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f6c900cff80 con 0x7f6c7c004800 2024-09-12T03:22:43.934 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:43.929+0000 7f6c8f7fe640 1 -- 172.21.15.27:0/4200718340 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f6c7c00c160 con 0x7f6c7c0039b0 2024-09-12T03:22:43.934 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:43.929+0000 7f6c8f7fe640 1 -- 172.21.15.27:0/4200718340 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f6c7c00c360 con 0x7f6c7c0051c0 2024-09-12T03:22:43.934 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:43.929+0000 7f6c8ffff640 1 --2- 172.21.15.27:0/4200718340 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c7c004800 0x7f6c7c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:43.934 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:43.929+0000 7f6c949fa640 1 --2- 172.21.15.27:0/4200718340 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6c7c0039b0 0x7f6c7c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:43.934 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:43.929+0000 7f6c951fb640 1 --2- 172.21.15.27:0/4200718340 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f6c7c0051c0 0x7f6c7c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:43.934 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:43.930+0000 7f6c8ffff640 1 -- 172.21.15.27:0/4200718340 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f6c7c0051c0 msgr2=0x7f6c7c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:43.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:43.930+0000 7f6c8ffff640 1 --2- 172.21.15.27:0/4200718340 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f6c7c0051c0 0x7f6c7c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:43.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:43.930+0000 7f6c8ffff640 1 -- 172.21.15.27:0/4200718340 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6c7c0039b0 msgr2=0x7f6c7c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:43.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:43.930+0000 7f6c8ffff640 1 --2- 172.21.15.27:0/4200718340 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6c7c0039b0 0x7f6c7c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:43.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:43.930+0000 7f6c8ffff640 1 -- 172.21.15.27:0/4200718340 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c7c003520 con 0x7f6c7c004800 2024-09-12T03:22:43.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:43.930+0000 7f6c951fb640 1 -- 172.21.15.27:0/4200718340 reap_dead start 2024-09-12T03:22:43.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:43.930+0000 7f6c8ffff640 1 --2- 172.21.15.27:0/4200718340 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c7c004800 0x7f6c7c004c80 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f6c80001750 tx=0x7f6c7c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91cbf7d604ca86f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:43.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:43.930+0000 7f6c8f7fe640 1 -- 172.21.15.27:0/4200718340 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c80001370 con 0x7f6c7c004800 2024-09-12T03:22:43.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:43.931+0000 7f6c8f7fe640 1 -- 172.21.15.27:0/4200718340 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c800052d0 con 0x7f6c7c004800 2024-09-12T03:22:43.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:43.931+0000 7f6c8f7fe640 1 -- 172.21.15.27:0/4200718340 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c8002abc0 con 0x7f6c7c004800 2024-09-12T03:22:43.936 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:43.931+0000 7f6c96c5e640 1 -- 172.21.15.27:0/4200718340 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c7c004800 msgr2=0x7f6c7c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:43.936 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:43.931+0000 7f6c96c5e640 1 --2- 172.21.15.27:0/4200718340 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c7c004800 0x7f6c7c004c80 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f6c80001750 tx=0x7f6c7c005830 comp rx=0 tx=0).stop 2024-09-12T03:22:43.936 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:43.931+0000 7f6c96c5e640 1 -- 172.21.15.27:0/4200718340 shutdown_connections 2024-09-12T03:22:43.936 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:43.931+0000 7f6c96c5e640 1 --2- 172.21.15.27:0/4200718340 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c7c004800 0x7f6c7c004c80 unknown :-1 s=CLOSED pgs=1364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:43.936 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:43.931+0000 7f6c96c5e640 1 -- 172.21.15.27:0/4200718340 >> 172.21.15.27:0/4200718340 conn(0x7f6c900d1230 msgr2=0x7f6c900d1640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:43.936 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:43.931+0000 7f6c96c5e640 1 -- 172.21.15.27:0/4200718340 shutdown_connections 2024-09-12T03:22:43.936 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:43.931+0000 7f6c96c5e640 1 -- 172.21.15.27:0/4200718340 wait complete. 2024-09-12T03:22:43.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:43.932+0000 7f6c96c5e640 1 Processor -- start 2024-09-12T03:22:43.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:43.932+0000 7f6c96c5e640 1 -- start start 2024-09-12T03:22:43.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:43.933+0000 7f6c96c5e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c7c004800 0x7f6c9015e070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:43.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:43.933+0000 7f6c96c5e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6c9015f2d0 0x7f6c900ac880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:43.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:43.933+0000 7f6c96c5e640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f6c9015b870 0x7f6c900b94a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:43.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:43.933+0000 7f6c96c5e640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f6c900cfaf0 con 0x7f6c7c004800 2024-09-12T03:22:43.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:43.933+0000 7f6c96c5e640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f6c900cf810 con 0x7f6c9015f2d0 2024-09-12T03:22:43.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:43.933+0000 7f6c96c5e640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f6c900b6880 con 0x7f6c9015b870 2024-09-12T03:22:43.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:43.933+0000 7f6c949fa640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c7c004800 0x7f6c9015e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:43.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:43.933+0000 7f6c949fa640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c7c004800 0x7f6c9015e070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35400/0 (socket says 172.21.15.27:35400) 2024-09-12T03:22:43.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:43.933+0000 7f6c949fa640 1 -- 172.21.15.27:0/1340368211 learned_addr learned my addr 172.21.15.27:0/1340368211 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:22:43.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:43.934+0000 7f6c949fa640 1 -- 172.21.15.27:0/1340368211 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f6c9015b870 msgr2=0x7f6c900b94a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:43.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:43.934+0000 7f6c951fb640 1 --2- 172.21.15.27:0/1340368211 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f6c9015b870 0x7f6c900b94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:43.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:43.934+0000 7f6c949fa640 1 --2- 172.21.15.27:0/1340368211 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f6c9015b870 0x7f6c900b94a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:43.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:43.934+0000 7f6c949fa640 1 -- 172.21.15.27:0/1340368211 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6c9015f2d0 msgr2=0x7f6c900ac880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:43.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:43.934+0000 7f6c8ffff640 1 --2- 172.21.15.27:0/1340368211 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6c9015f2d0 0x7f6c900ac880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:43.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:43.934+0000 7f6c949fa640 1 --2- 172.21.15.27:0/1340368211 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6c9015f2d0 0x7f6c900ac880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:43.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:43.934+0000 7f6c949fa640 1 -- 172.21.15.27:0/1340368211 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c901f4170 con 0x7f6c7c004800 2024-09-12T03:22:43.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:43.934+0000 7f6c949fa640 1 --2- 172.21.15.27:0/1340368211 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c7c004800 0x7f6c9015e070 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7f6c88009f90 tx=0x7f6c8800f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:43.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:43.934+0000 7f6c8d7fa640 1 -- 172.21.15.27:0/1340368211 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c8800fea0 con 0x7f6c7c004800 2024-09-12T03:22:43.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:43.934+0000 7f6c8d7fa640 1 -- 172.21.15.27:0/1340368211 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c88015630 con 0x7f6c7c004800 2024-09-12T03:22:43.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:43.934+0000 7f6c96c5e640 1 -- 172.21.15.27:0/1340368211 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c900ad0d0 con 0x7f6c7c004800 2024-09-12T03:22:43.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:43.934+0000 7f6c8d7fa640 1 -- 172.21.15.27:0/1340368211 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c88015a20 con 0x7f6c7c004800 2024-09-12T03:22:43.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:43.934+0000 7f6c96c5e640 1 -- 172.21.15.27:0/1340368211 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c9015eaf0 con 0x7f6c7c004800 2024-09-12T03:22:43.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:43.935+0000 7f6c96c5e640 1 -- 172.21.15.27:0/1340368211 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c900ad4c0 con 0x7f6c7c004800 2024-09-12T03:22:43.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:43.936+0000 7f6c8d7fa640 1 -- 172.21.15.27:0/1340368211 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 22) ==== 91059+0+0 (secure 0 0 0) 0x7f6c88015bc0 con 0x7f6c7c004800 2024-09-12T03:22:43.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:43.936+0000 7f6c8d7fa640 1 --2- 172.21.15.27:0/1340368211 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f6c7006c460 0x7f6c7006e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:43.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:43.936+0000 7f6c8d7fa640 1 -- 172.21.15.27:0/1340368211 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 4999+0+0 (secure 0 0 0) 0x7f6c8808c890 con 0x7f6c7c004800 2024-09-12T03:22:43.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:43.936+0000 7f6c8ffff640 1 --2- 172.21.15.27:0/1340368211 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f6c7006c460 0x7f6c7006e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:43.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:43.937+0000 7f6c8ffff640 1 --2- 172.21.15.27:0/1340368211 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f6c7006c460 0x7f6c7006e920 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f6c80032000 tx=0x7f6c8002e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:43.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:43.941+0000 7f6c8d7fa640 1 -- 172.21.15.27:0/1340368211 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f6c900ad4c0 con 0x7f6c7c004800 2024-09-12T03:22:44.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:44.116+0000 7f6c96c5e640 1 -- 172.21.15.27:0/1340368211 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f6c90161ab0 con 0x7f6c7c004800 2024-09-12T03:22:44.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:44.117+0000 7f6c8d7fa640 1 -- 172.21.15.27:0/1340368211 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v129) ==== 93+0+10 (secure 0 0 0) 0x7f6c88019030 con 0x7f6c7c004800 2024-09-12T03:22:44.118 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 1 2024-09-12T03:22:44.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:44.119+0000 7f6c96c5e640 1 -- 172.21.15.27:0/1340368211 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f6c7006c460 msgr2=0x7f6c7006e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:44.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:44.119+0000 7f6c96c5e640 1 --2- 172.21.15.27:0/1340368211 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f6c7006c460 0x7f6c7006e920 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f6c80032000 tx=0x7f6c8002e000 comp rx=0 tx=0).stop 2024-09-12T03:22:44.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:44.119+0000 7f6c96c5e640 1 -- 172.21.15.27:0/1340368211 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c7c004800 msgr2=0x7f6c9015e070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:44.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:44.119+0000 7f6c96c5e640 1 --2- 172.21.15.27:0/1340368211 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c7c004800 0x7f6c9015e070 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7f6c88009f90 tx=0x7f6c8800f580 comp rx=0 tx=0).stop 2024-09-12T03:22:44.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:44.119+0000 7f6c96c5e640 1 -- 172.21.15.27:0/1340368211 shutdown_connections 2024-09-12T03:22:44.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:44.119+0000 7f6c96c5e640 1 --2- 172.21.15.27:0/1340368211 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f6c9015b870 0x7f6c900b94a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:44.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:44.119+0000 7f6c96c5e640 1 --2- 172.21.15.27:0/1340368211 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6c9015f2d0 0x7f6c900ac880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:44.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:44.119+0000 7f6c96c5e640 1 --2- 172.21.15.27:0/1340368211 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f6c7006c460 0x7f6c7006e920 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:44.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:44.119+0000 7f6c96c5e640 1 --2- 172.21.15.27:0/1340368211 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c7c004800 0x7f6c9015e070 unknown :-1 s=CLOSED pgs=1365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:44.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:44.119+0000 7f6c96c5e640 1 -- 172.21.15.27:0/1340368211 >> 172.21.15.27:0/1340368211 conn(0x7f6c900d1230 msgr2=0x7f6c900b7de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:44.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:44.119+0000 7f6c96c5e640 1 -- 172.21.15.27:0/1340368211 shutdown_connections 2024-09-12T03:22:44.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:44.119+0000 7f6c96c5e640 1 -- 172.21.15.27:0/1340368211 wait complete. 2024-09-12T03:22:44.131 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_upgrade_legacy_to_v1' 2024-09-12T03:22:44.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:44.233+0000 7fa8f6333640 1 Processor -- start 2024-09-12T03:22:44.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:44.233+0000 7fa8f6333640 1 -- start start 2024-09-12T03:22:44.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:44.234+0000 7fa8f6333640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa8f0158cf0 0x7fa8f00b55d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:44.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:44.234+0000 7fa8f6333640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa8f01590d0 0x7fa8f0162430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:44.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:44.234+0000 7fa8f6333640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fa8f015f600 0x7fa8f006bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:44.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:44.234+0000 7fa8f6333640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7fa8f0162970 con 0x7fa8f015f600 2024-09-12T03:22:44.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:44.234+0000 7fa8f6333640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fa8f0162c10 con 0x7fa8f0158cf0 2024-09-12T03:22:44.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:44.234+0000 7fa8f6333640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fa8f0162ef0 con 0x7fa8f01590d0 2024-09-12T03:22:44.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:44.234+0000 7fa8f48d0640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fa8f015f600 0x7fa8f006bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:44.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:44.235+0000 7fa8f48d0640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fa8f015f600 0x7fa8f006bb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35404/0 (socket says 172.21.15.27:35404) 2024-09-12T03:22:44.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:44.235+0000 7fa8f48d0640 1 -- 172.21.15.27:0/3753050050 learned_addr learned my addr 172.21.15.27:0/3753050050 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:22:44.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:44.235+0000 7fa8effff640 1 --2- 172.21.15.27:0/3753050050 >> v2:172.21.15.45:3300/0 conn(0x7fa8f0158cf0 0x7fa8f00b55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:44.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:44.235+0000 7fa8e7fff640 1 --2- 172.21.15.27:0/3753050050 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa8f01590d0 0x7fa8f0162430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:44.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:44.235+0000 7fa8f48d0640 1 -- 172.21.15.27:0/3753050050 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa8f01590d0 msgr2=0x7fa8f0162430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:44.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:44.235+0000 7fa8f48d0640 1 --2- 172.21.15.27:0/3753050050 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa8f01590d0 0x7fa8f0162430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:44.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:44.235+0000 7fa8f48d0640 1 -- 172.21.15.27:0/3753050050 >> v2:172.21.15.45:3300/0 conn(0x7fa8f0158cf0 msgr2=0x7fa8f00b55d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:44.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:44.235+0000 7fa8f48d0640 1 --2- 172.21.15.27:0/3753050050 >> v2:172.21.15.45:3300/0 conn(0x7fa8f0158cf0 0x7fa8f00b55d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:44.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:44.235+0000 7fa8f48d0640 1 -- 172.21.15.27:0/3753050050 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8f01631d0 con 0x7fa8f015f600 2024-09-12T03:22:44.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:44.235+0000 7fa8f48d0640 1 --2- 172.21.15.27:0/3753050050 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa8f015f600 0x7fa8f006bb30 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7fa8f00733c0 tx=0x7fa8e802a570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=540f81013b806bc2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:44.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:44.236+0000 7fa8ef7fe640 1 -- 172.21.15.27:0/3753050050 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8e8036070 con 0x7fa8f015f600 2024-09-12T03:22:44.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:44.236+0000 7fa8ef7fe640 1 -- 172.21.15.27:0/3753050050 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa8e8031600 con 0x7fa8f015f600 2024-09-12T03:22:44.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:44.236+0000 7fa8ef7fe640 1 -- 172.21.15.27:0/3753050050 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8e80319d0 con 0x7fa8f015f600 2024-09-12T03:22:44.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:44.236+0000 7fa8f6333640 1 -- 172.21.15.27:0/3753050050 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa8f015f600 msgr2=0x7fa8f006bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:44.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:44.236+0000 7fa8f6333640 1 --2- 172.21.15.27:0/3753050050 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa8f015f600 0x7fa8f006bb30 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7fa8f00733c0 tx=0x7fa8e802a570 comp rx=0 tx=0).stop 2024-09-12T03:22:44.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:44.236+0000 7fa8f6333640 1 -- 172.21.15.27:0/3753050050 shutdown_connections 2024-09-12T03:22:44.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:44.236+0000 7fa8f6333640 1 --2- 172.21.15.27:0/3753050050 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa8f015f600 0x7fa8f006bb30 unknown :-1 s=CLOSED pgs=1366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:44.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:44.236+0000 7fa8f6333640 1 --2- 172.21.15.27:0/3753050050 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa8f01590d0 0x7fa8f0162430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:44.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:44.236+0000 7fa8f6333640 1 --2- 172.21.15.27:0/3753050050 >> v2:172.21.15.45:3300/0 conn(0x7fa8f0158cf0 0x7fa8f00b55d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:44.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:44.236+0000 7fa8f6333640 1 -- 172.21.15.27:0/3753050050 >> 172.21.15.27:0/3753050050 conn(0x7fa8f006cc40 msgr2=0x7fa8f006d050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:44.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:44.236+0000 7fa8f6333640 1 -- 172.21.15.27:0/3753050050 shutdown_connections 2024-09-12T03:22:44.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:44.236+0000 7fa8f6333640 1 -- 172.21.15.27:0/3753050050 wait complete. 2024-09-12T03:22:44.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:44.237+0000 7fa8f6333640 1 Processor -- start 2024-09-12T03:22:44.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:44.238+0000 7fa8f6333640 1 -- start start 2024-09-12T03:22:44.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:44.238+0000 7fa8f6333640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa8f0158cf0 0x7fa8f01508b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:44.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:44.238+0000 7fa8f6333640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa8f01590d0 0x7fa8f015d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:44.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:44.238+0000 7fa8f6333640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa8f015f600 0x7fa8f01e98c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:44.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:44.238+0000 7fa8f6333640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fa8f015fbd0 con 0x7fa8f0158cf0 2024-09-12T03:22:44.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:44.238+0000 7fa8f6333640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa8f015fa50 con 0x7fa8f01590d0 2024-09-12T03:22:44.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:44.238+0000 7fa8f6333640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fa8f015fd50 con 0x7fa8f015f600 2024-09-12T03:22:44.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:44.238+0000 7fa8effff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa8f0158cf0 0x7fa8f01508b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:44.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:44.238+0000 7fa8effff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa8f0158cf0 0x7fa8f01508b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35414/0 (socket says 172.21.15.27:35414) 2024-09-12T03:22:44.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:44.238+0000 7fa8effff640 1 -- 172.21.15.27:0/4150392308 learned_addr learned my addr 172.21.15.27:0/4150392308 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:22:44.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:44.239+0000 7fa8e7fff640 1 --2- 172.21.15.27:0/4150392308 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa8f01590d0 0x7fa8f015d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:44.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:44.239+0000 7fa8f48d0640 1 --2- 172.21.15.27:0/4150392308 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa8f015f600 0x7fa8f01e98c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:44.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:44.239+0000 7fa8effff640 1 -- 172.21.15.27:0/4150392308 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa8f015f600 msgr2=0x7fa8f01e98c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:44.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:44.239+0000 7fa8effff640 1 --2- 172.21.15.27:0/4150392308 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa8f015f600 0x7fa8f01e98c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:44.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:44.239+0000 7fa8effff640 1 -- 172.21.15.27:0/4150392308 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa8f01590d0 msgr2=0x7fa8f015d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:44.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:44.239+0000 7fa8effff640 1 --2- 172.21.15.27:0/4150392308 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa8f01590d0 0x7fa8f015d830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:44.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:44.239+0000 7fa8effff640 1 -- 172.21.15.27:0/4150392308 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8f01e8e70 con 0x7fa8f0158cf0 2024-09-12T03:22:44.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:44.239+0000 7fa8effff640 1 --2- 172.21.15.27:0/4150392308 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa8f0158cf0 0x7fa8f01508b0 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7fa8e000d800 tx=0x7fa8e000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:44.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:44.239+0000 7fa8ed7fa640 1 -- 172.21.15.27:0/4150392308 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8e0018030 con 0x7fa8f0158cf0 2024-09-12T03:22:44.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:44.239+0000 7fa8ed7fa640 1 -- 172.21.15.27:0/4150392308 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa8e00096a0 con 0x7fa8f0158cf0 2024-09-12T03:22:44.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:44.239+0000 7fa8f6333640 1 -- 172.21.15.27:0/4150392308 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8f01e9150 con 0x7fa8f0158cf0 2024-09-12T03:22:44.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:44.239+0000 7fa8ed7fa640 1 -- 172.21.15.27:0/4150392308 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8e0009870 con 0x7fa8f0158cf0 2024-09-12T03:22:44.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:44.239+0000 7fa8f6333640 1 -- 172.21.15.27:0/4150392308 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8f01e8af0 con 0x7fa8f0158cf0 2024-09-12T03:22:44.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:44.240+0000 7fa8f6333640 1 -- 172.21.15.27:0/4150392308 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8f0162430 con 0x7fa8f0158cf0 2024-09-12T03:22:44.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:44.241+0000 7fa8ed7fa640 1 -- 172.21.15.27:0/4150392308 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 22) ==== 91059+0+0 (secure 0 0 0) 0x7fa8e001b070 con 0x7fa8f0158cf0 2024-09-12T03:22:44.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:44.241+0000 7fa8ed7fa640 1 --2- 172.21.15.27:0/4150392308 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fa8d006c460 0x7fa8d006e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:44.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:44.241+0000 7fa8ed7fa640 1 -- 172.21.15.27:0/4150392308 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 4999+0+0 (secure 0 0 0) 0x7fa8e008ba00 con 0x7fa8f0158cf0 2024-09-12T03:22:44.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:44.241+0000 7fa8e7fff640 1 --2- 172.21.15.27:0/4150392308 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fa8d006c460 0x7fa8d006e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:44.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:44.242+0000 7fa8e7fff640 1 --2- 172.21.15.27:0/4150392308 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fa8d006c460 0x7fa8d006e920 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fa8d8006fd0 tx=0x7fa8d8008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:44.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:44.246+0000 7fa8ed7fa640 1 -- 172.21.15.27:0/4150392308 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fa8e0059d30 con 0x7fa8f0158cf0 2024-09-12T03:22:44.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:44.486+0000 7fa8f6333640 1 -- 172.21.15.27:0/4150392308 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_upgrade_legacy_to_v1"]} v 0) -- 0x7fa8f0068ee0 con 0x7fa8f0158cf0 2024-09-12T03:22:44.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:44.502+0000 7fa8ed7fa640 1 -- 172.21.15.27:0/4150392308 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_upgrade_legacy_to_v1"]}]=0 v343) ==== 152+0+0 (secure 0 0 0) 0x7fa8e0059630 con 0x7fa8f0158cf0 2024-09-12T03:22:44.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:44.505+0000 7fa8f6333640 1 -- 172.21.15.27:0/4150392308 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fa8d006c460 msgr2=0x7fa8d006e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:44.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:44.505+0000 7fa8f6333640 1 --2- 172.21.15.27:0/4150392308 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fa8d006c460 0x7fa8d006e920 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fa8d8006fd0 tx=0x7fa8d8008040 comp rx=0 tx=0).stop 2024-09-12T03:22:44.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:44.505+0000 7fa8f6333640 1 -- 172.21.15.27:0/4150392308 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa8f0158cf0 msgr2=0x7fa8f01508b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:44.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:44.505+0000 7fa8f6333640 1 --2- 172.21.15.27:0/4150392308 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa8f0158cf0 0x7fa8f01508b0 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7fa8e000d800 tx=0x7fa8e000dcc0 comp rx=0 tx=0).stop 2024-09-12T03:22:44.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:44.505+0000 7fa8f6333640 1 -- 172.21.15.27:0/4150392308 shutdown_connections 2024-09-12T03:22:44.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:44.505+0000 7fa8f6333640 1 --2- 172.21.15.27:0/4150392308 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa8f015f600 0x7fa8f01e98c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:44.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:44.505+0000 7fa8f6333640 1 --2- 172.21.15.27:0/4150392308 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa8f01590d0 0x7fa8f015d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:44.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:44.505+0000 7fa8f6333640 1 --2- 172.21.15.27:0/4150392308 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fa8d006c460 0x7fa8d006e920 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:44.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:44.505+0000 7fa8f6333640 1 --2- 172.21.15.27:0/4150392308 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa8f0158cf0 0x7fa8f01508b0 unknown :-1 s=CLOSED pgs=1367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:44.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:44.505+0000 7fa8f6333640 1 -- 172.21.15.27:0/4150392308 >> 172.21.15.27:0/4150392308 conn(0x7fa8f006cc40 msgr2=0x7fa8f0157a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:44.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:44.505+0000 7fa8f6333640 1 -- 172.21.15.27:0/4150392308 shutdown_connections 2024-09-12T03:22:44.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:44.505+0000 7fa8f6333640 1 -- 172.21.15.27:0/4150392308 wait complete. 2024-09-12T03:22:44.517 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-12T03:22:44.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:44.622+0000 7fb86db5f640 1 Processor -- start 2024-09-12T03:22:44.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:44.622+0000 7fb86db5f640 1 -- start start 2024-09-12T03:22:44.624 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:44.623+0000 7fb86db5f640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fb86806e550 0x7fb86815ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:44.624 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:44.623+0000 7fb86db5f640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb8680aedd0 con 0x7fb8680ae8d0 2024-09-12T03:22:44.624 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:44.623+0000 7fb86db5f640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb8680af0b0 con 0x7fb868072610 2024-09-12T03:22:44.624 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:44.623+0000 7fb86db5f640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7fb8680af390 con 0x7fb86806e550 2024-09-12T03:22:44.624 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:44.623+0000 7fb8677fe640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fb86806e550 0x7fb86815ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:44.624 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:44.623+0000 7fb866ffd640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fb8680ae8d0 0x7fb86806e930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:55388/0 (socket says 172.21.15.27:55388) 2024-09-12T03:22:44.624 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:44.623+0000 7fb866ffd640 1 -- 172.21.15.27:0/3221388866 learned_addr learned my addr 172.21.15.27:0/3221388866 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:22:44.624 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:44.623+0000 7fb8677fe640 1 -- 172.21.15.27:0/3221388866 >> v1:172.21.15.45:6789/0 conn(0x7fb868072610 legacy=0x7fb868072ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:44.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:44.623+0000 7fb8677fe640 1 -- 172.21.15.27:0/3221388866 >> v1:172.21.15.27:6789/0 conn(0x7fb8680ae8d0 legacy=0x7fb86806e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:44.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:44.623+0000 7fb8677fe640 1 -- 172.21.15.27:0/3221388866 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8680af670 con 0x7fb86806e550 2024-09-12T03:22:44.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:44.623+0000 7fb8667fc640 1 -- 172.21.15.27:0/3221388866 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 434441781 0 0) 0x7fb8680aedd0 con 0x7fb8680ae8d0 2024-09-12T03:22:44.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:44.624+0000 7fb8677fe640 1 --2- 172.21.15.27:0/3221388866 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb86806e550 0x7fb86815ebb0 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7fb8580097a0 tx=0x7fb85802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bb70f54afc4523f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:44.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:44.624+0000 7fb8667fc640 1 -- 172.21.15.27:0/3221388866 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb858037070 con 0x7fb86806e550 2024-09-12T03:22:44.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:44.624+0000 7fb8667fc640 1 -- 172.21.15.27:0/3221388866 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb858032440 con 0x7fb86806e550 2024-09-12T03:22:44.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:44.624+0000 7fb8667fc640 1 -- 172.21.15.27:0/3221388866 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb858032810 con 0x7fb86806e550 2024-09-12T03:22:44.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:44.624+0000 7fb86db5f640 1 -- 172.21.15.27:0/3221388866 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb86806e550 msgr2=0x7fb86815ebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:44.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:44.624+0000 7fb86db5f640 1 --2- 172.21.15.27:0/3221388866 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb86806e550 0x7fb86815ebb0 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7fb8580097a0 tx=0x7fb85802a1e0 comp rx=0 tx=0).stop 2024-09-12T03:22:44.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:44.624+0000 7fb86db5f640 1 -- 172.21.15.27:0/3221388866 shutdown_connections 2024-09-12T03:22:44.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:44.624+0000 7fb86db5f640 1 --2- 172.21.15.27:0/3221388866 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb86806e550 0x7fb86815ebb0 unknown :-1 s=CLOSED pgs=1369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:44.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:44.625+0000 7fb86db5f640 1 -- 172.21.15.27:0/3221388866 >> 172.21.15.27:0/3221388866 conn(0x7fb8680b26c0 msgr2=0x7fb8680b2aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:44.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:44.625+0000 7fb86db5f640 1 -- 172.21.15.27:0/3221388866 shutdown_connections 2024-09-12T03:22:44.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:44.625+0000 7fb86db5f640 1 -- 172.21.15.27:0/3221388866 wait complete. 2024-09-12T03:22:44.634 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:44.626+0000 7fb86db5f640 1 Processor -- start 2024-09-12T03:22:44.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:44.626+0000 7fb86db5f640 1 -- start start 2024-09-12T03:22:44.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:44.627+0000 7fb86db5f640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb86806e550 0x7fb8681f5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:44.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:44.627+0000 7fb86db5f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb868072610 0x7fb86815a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:44.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:44.627+0000 7fb86db5f640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb8680ae8d0 0x7fb8680c9780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:44.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:44.627+0000 7fb86db5f640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fb8680b3480 con 0x7fb8680ae8d0 2024-09-12T03:22:44.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:44.627+0000 7fb86db5f640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb8680b3300 con 0x7fb868072610 2024-09-12T03:22:44.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:44.627+0000 7fb86db5f640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fb8680b39c0 con 0x7fb86806e550 2024-09-12T03:22:44.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:44.627+0000 7fb867fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb8680ae8d0 0x7fb8680c9780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:44.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:44.627+0000 7fb867fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb8680ae8d0 0x7fb8680c9780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35438/0 (socket says 172.21.15.27:35438) 2024-09-12T03:22:44.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:44.627+0000 7fb867fff640 1 -- 172.21.15.27:0/925454234 learned_addr learned my addr 172.21.15.27:0/925454234 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:22:44.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:44.627+0000 7fb8677fe640 1 --2- 172.21.15.27:0/925454234 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb86806e550 0x7fb8681f5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:44.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:44.627+0000 7fb866ffd640 1 --2- 172.21.15.27:0/925454234 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb868072610 0x7fb86815a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:44.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:44.627+0000 7fb867fff640 1 -- 172.21.15.27:0/925454234 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb86806e550 msgr2=0x7fb8681f5cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:44.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:44.627+0000 7fb867fff640 1 --2- 172.21.15.27:0/925454234 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb86806e550 0x7fb8681f5cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:44.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:44.627+0000 7fb867fff640 1 -- 172.21.15.27:0/925454234 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb868072610 msgr2=0x7fb86815a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:44.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:44.627+0000 7fb867fff640 1 --2- 172.21.15.27:0/925454234 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb868072610 0x7fb86815a370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:44.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:44.627+0000 7fb867fff640 1 -- 172.21.15.27:0/925454234 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8681f5ac0 con 0x7fb8680ae8d0 2024-09-12T03:22:44.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:44.627+0000 7fb867fff640 1 --2- 172.21.15.27:0/925454234 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb8680ae8d0 0x7fb8680c9780 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7fb86000b970 tx=0x7fb86000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:44.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:44.627+0000 7fb84ffff640 1 -- 172.21.15.27:0/925454234 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb860019070 con 0x7fb8680ae8d0 2024-09-12T03:22:44.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:44.627+0000 7fb84ffff640 1 -- 172.21.15.27:0/925454234 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb86000cb40 con 0x7fb8680ae8d0 2024-09-12T03:22:44.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:44.627+0000 7fb86db5f640 1 -- 172.21.15.27:0/925454234 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8681f4f40 con 0x7fb8680ae8d0 2024-09-12T03:22:44.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:44.627+0000 7fb84ffff640 1 -- 172.21.15.27:0/925454234 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb860014440 con 0x7fb8680ae8d0 2024-09-12T03:22:44.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:44.627+0000 7fb86db5f640 1 -- 172.21.15.27:0/925454234 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8681f57e0 con 0x7fb8680ae8d0 2024-09-12T03:22:44.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:44.628+0000 7fb86db5f640 1 -- 172.21.15.27:0/925454234 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8680aedd0 con 0x7fb8680ae8d0 2024-09-12T03:22:44.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:44.629+0000 7fb84ffff640 1 -- 172.21.15.27:0/925454234 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 22) ==== 91059+0+0 (secure 0 0 0) 0x7fb86001c070 con 0x7fb8680ae8d0 2024-09-12T03:22:44.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:44.629+0000 7fb84ffff640 1 --2- 172.21.15.27:0/925454234 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fb85406c460 0x7fb85406e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:44.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:44.630+0000 7fb84ffff640 1 -- 172.21.15.27:0/925454234 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 4999+0+0 (secure 0 0 0) 0x7fb86008b320 con 0x7fb8680ae8d0 2024-09-12T03:22:44.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:44.630+0000 7fb8677fe640 1 --2- 172.21.15.27:0/925454234 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fb85406c460 0x7fb85406e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:44.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:44.630+0000 7fb8677fe640 1 --2- 172.21.15.27:0/925454234 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fb85406c460 0x7fb85406e920 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fb858034000 tx=0x7fb85802e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:44.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:44.634+0000 7fb84ffff640 1 -- 172.21.15.27:0/925454234 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fb860059750 con 0x7fb8680ae8d0 2024-09-12T03:22:44.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:44.809+0000 7fb86db5f640 1 -- 172.21.15.27:0/925454234 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fb86815bbc0 con 0x7fb8680ae8d0 2024-09-12T03:22:44.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:44.812+0000 7fb84ffff640 1 -- 172.21.15.27:0/925454234 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v26)=0 v26) ==== 122+0+0 (secure 0 0 0) 0x7fb860059050 con 0x7fb8680ae8d0 2024-09-12T03:22:44.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:44.815+0000 7fb86db5f640 1 -- 172.21.15.27:0/925454234 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fb85406c460 msgr2=0x7fb85406e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:44.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:44.815+0000 7fb86db5f640 1 --2- 172.21.15.27:0/925454234 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fb85406c460 0x7fb85406e920 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fb858034000 tx=0x7fb85802e000 comp rx=0 tx=0).stop 2024-09-12T03:22:44.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:44.815+0000 7fb86db5f640 1 -- 172.21.15.27:0/925454234 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb8680ae8d0 msgr2=0x7fb8680c9780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:44.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:44.815+0000 7fb86db5f640 1 --2- 172.21.15.27:0/925454234 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb8680ae8d0 0x7fb8680c9780 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7fb86000b970 tx=0x7fb86000be30 comp rx=0 tx=0).stop 2024-09-12T03:22:44.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:44.815+0000 7fb86db5f640 1 -- 172.21.15.27:0/925454234 shutdown_connections 2024-09-12T03:22:44.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:44.815+0000 7fb86db5f640 1 --2- 172.21.15.27:0/925454234 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fb85406c460 0x7fb85406e920 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:44.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:44.815+0000 7fb86db5f640 1 --2- 172.21.15.27:0/925454234 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb8680ae8d0 0x7fb8680c9780 unknown :-1 s=CLOSED pgs=1370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:44.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:44.815+0000 7fb86db5f640 1 --2- 172.21.15.27:0/925454234 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb868072610 0x7fb86815a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:44.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:44.815+0000 7fb86db5f640 1 --2- 172.21.15.27:0/925454234 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb86806e550 0x7fb8681f5cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:44.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:44.815+0000 7fb86db5f640 1 -- 172.21.15.27:0/925454234 >> 172.21.15.27:0/925454234 conn(0x7fb8680b26c0 msgr2=0x7fb8680b2aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:44.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:44.815+0000 7fb86db5f640 1 -- 172.21.15.27:0/925454234 shutdown_connections 2024-09-12T03:22:44.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:44.815+0000 7fb86db5f640 1 -- 172.21.15.27:0/925454234 wait complete. 2024-09-12T03:22:44.827 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-12T03:22:44.827 DEBUG:teuthology.orchestra.run.smithi027:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-12T03:22:44.865 DEBUG:teuthology.orchestra.run.smithi045:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-12T03:22:44.921 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-12T03:22:44.922 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-12T03:22:44.922 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T03:22:44.937 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056004,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497432k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16324 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248716k,nr_inodes=812179,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497432k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-12T03:22:44.938 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-12T03:22:44.938 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T03:22:44.957 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056795,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16602 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249348k,nr_inodes=812337,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-12T03:22:44.957 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-12T03:22:44.957 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T03:22:44.994 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056004,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497432k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16324 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248716k,nr_inodes=812179,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497432k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-12T03:22:44.994 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-12T03:22:44.994 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T03:22:45.017 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056795,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16602 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249348k,nr_inodes=812337,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-12T03:22:45.017 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-12T03:22:45.017 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-12T03:22:45.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:45.135+0000 7efe25731640 1 Processor -- start 2024-09-12T03:22:45.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:45.135+0000 7efe25731640 1 -- start start 2024-09-12T03:22:45.138 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:45.136+0000 7efe25731640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7efe2015e5f0 0x7efe20073250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:45.138 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:45.136+0000 7efe25731640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7efe20073790 0x7efe20162320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:45.138 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:45.136+0000 7efe25731640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efe20163090 con 0x7efe20162860 2024-09-12T03:22:45.138 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:45.136+0000 7efe25731640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7efe20073be0 con 0x7efe2015e5f0 2024-09-12T03:22:45.138 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:45.136+0000 7efe25731640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7efe200d0da0 con 0x7efe20073790 2024-09-12T03:22:45.138 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:45.136+0000 7efe1effd640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7efe2015e5f0 0x7efe20073250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:45.138 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:45.136+0000 7efe1f7fe640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7efe20162860 0x7efe200d1230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:55402/0 (socket says 172.21.15.27:55402) 2024-09-12T03:22:45.138 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:45.136+0000 7efe1f7fe640 1 -- 172.21.15.27:0/660756250 learned_addr learned my addr 172.21.15.27:0/660756250 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:22:45.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:45.136+0000 7efe17fff640 1 --2- 172.21.15.27:0/660756250 >> v2:172.21.15.45:3300/0 conn(0x7efe20073790 0x7efe20162320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:45.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:45.136+0000 7efe1effd640 1 -- 172.21.15.27:0/660756250 >> v2:172.21.15.45:3300/0 conn(0x7efe20073790 msgr2=0x7efe20162320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:45.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:45.136+0000 7efe1effd640 1 --2- 172.21.15.27:0/660756250 >> v2:172.21.15.45:3300/0 conn(0x7efe20073790 0x7efe20162320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:45.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:45.136+0000 7efe1effd640 1 -- 172.21.15.27:0/660756250 >> v1:172.21.15.27:6789/0 conn(0x7efe20162860 legacy=0x7efe200d1230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:45.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:45.136+0000 7efe1effd640 1 -- 172.21.15.27:0/660756250 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe200d0470 con 0x7efe2015e5f0 2024-09-12T03:22:45.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:45.136+0000 7efe1e7fc640 1 -- 172.21.15.27:0/660756250 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2878194211 0 0) 0x7efe20163090 con 0x7efe20162860 2024-09-12T03:22:45.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:45.137+0000 7efe1effd640 1 --2- 172.21.15.27:0/660756250 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efe2015e5f0 0x7efe20073250 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7efe200741c0 tx=0x7efe1002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a36ab5f8d7fc4a5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:45.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:45.137+0000 7efe1e7fc640 1 -- 172.21.15.27:0/660756250 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe10037070 con 0x7efe2015e5f0 2024-09-12T03:22:45.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:45.137+0000 7efe1e7fc640 1 -- 172.21.15.27:0/660756250 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe10032440 con 0x7efe2015e5f0 2024-09-12T03:22:45.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:45.137+0000 7efe1e7fc640 1 -- 172.21.15.27:0/660756250 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe10032810 con 0x7efe2015e5f0 2024-09-12T03:22:45.142 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:45.138+0000 7efe25731640 1 -- 172.21.15.27:0/660756250 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efe2015e5f0 msgr2=0x7efe20073250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:45.142 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:45.138+0000 7efe25731640 1 --2- 172.21.15.27:0/660756250 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efe2015e5f0 0x7efe20073250 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7efe200741c0 tx=0x7efe1002a1a0 comp rx=0 tx=0).stop 2024-09-12T03:22:45.142 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:45.138+0000 7efe25731640 1 -- 172.21.15.27:0/660756250 shutdown_connections 2024-09-12T03:22:45.142 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:45.138+0000 7efe25731640 1 --2- 172.21.15.27:0/660756250 >> v2:172.21.15.45:3300/0 conn(0x7efe20073790 0x7efe20162320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:45.142 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:45.138+0000 7efe25731640 1 --2- 172.21.15.27:0/660756250 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efe2015e5f0 0x7efe20073250 unknown :-1 s=CLOSED pgs=1372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:45.142 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:45.138+0000 7efe25731640 1 -- 172.21.15.27:0/660756250 >> 172.21.15.27:0/660756250 conn(0x7efe2006f300 msgr2=0x7efe2006f710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:45.142 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:45.138+0000 7efe25731640 1 -- 172.21.15.27:0/660756250 shutdown_connections 2024-09-12T03:22:45.142 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:45.138+0000 7efe25731640 1 -- 172.21.15.27:0/660756250 wait complete. 2024-09-12T03:22:45.142 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:45.139+0000 7efe25731640 1 Processor -- start 2024-09-12T03:22:45.142 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:45.139+0000 7efe25731640 1 -- start start 2024-09-12T03:22:45.142 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:45.140+0000 7efe25731640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7efe20073790 0x7efe20156370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:45.143 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:45.140+0000 7efe25731640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efe2015e5f0 0x7efe2015c570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:45.143 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:45.140+0000 7efe25731640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efe20162860 0x7efe20152a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:45.143 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:45.140+0000 7efe25731640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7efe200b5620 con 0x7efe20162860 2024-09-12T03:22:45.143 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:45.140+0000 7efe25731640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7efe200d0a30 con 0x7efe2015e5f0 2024-09-12T03:22:45.143 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:45.140+0000 7efe25731640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7efe20162c90 con 0x7efe20073790 2024-09-12T03:22:45.143 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:45.140+0000 7efe1f7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efe20162860 0x7efe20152a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:45.143 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:45.140+0000 7efe1f7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efe20162860 0x7efe20152a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35448/0 (socket says 172.21.15.27:35448) 2024-09-12T03:22:45.143 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:45.140+0000 7efe1f7fe640 1 -- 172.21.15.27:0/3425574471 learned_addr learned my addr 172.21.15.27:0/3425574471 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:22:45.143 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:45.140+0000 7efe17fff640 1 --2- 172.21.15.27:0/3425574471 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efe2015e5f0 0x7efe2015c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:45.143 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:45.140+0000 7efe1effd640 1 --2- 172.21.15.27:0/3425574471 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7efe20073790 0x7efe20156370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:45.143 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:45.140+0000 7efe1f7fe640 1 -- 172.21.15.27:0/3425574471 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7efe20073790 msgr2=0x7efe20156370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:45.143 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:45.140+0000 7efe1f7fe640 1 --2- 172.21.15.27:0/3425574471 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7efe20073790 0x7efe20156370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:45.143 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:45.140+0000 7efe1f7fe640 1 -- 172.21.15.27:0/3425574471 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efe2015e5f0 msgr2=0x7efe2015c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:45.143 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:45.140+0000 7efe1f7fe640 1 --2- 172.21.15.27:0/3425574471 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efe2015e5f0 0x7efe2015c570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:45.144 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:45.140+0000 7efe1f7fe640 1 -- 172.21.15.27:0/3425574471 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe2015cfc0 con 0x7efe20162860 2024-09-12T03:22:45.144 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:45.140+0000 7efe1effd640 1 --2- 172.21.15.27:0/3425574471 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7efe20073790 0x7efe20156370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:22:45.144 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:45.140+0000 7efe1f7fe640 1 --2- 172.21.15.27:0/3425574471 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efe20162860 0x7efe20152a50 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7efe1800c7a0 tx=0x7efe1800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:45.144 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:45.140+0000 7efe177fe640 1 -- 172.21.15.27:0/3425574471 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe18018070 con 0x7efe20162860 2024-09-12T03:22:45.144 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:45.140+0000 7efe177fe640 1 -- 172.21.15.27:0/3425574471 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe1800a5e0 con 0x7efe20162860 2024-09-12T03:22:45.144 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:45.141+0000 7efe25731640 1 -- 172.21.15.27:0/3425574471 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe2015a7f0 con 0x7efe20162860 2024-09-12T03:22:45.144 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:45.141+0000 7efe177fe640 1 -- 172.21.15.27:0/3425574471 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe1800aa50 con 0x7efe20162860 2024-09-12T03:22:45.144 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:45.141+0000 7efe25731640 1 -- 172.21.15.27:0/3425574471 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe2015b090 con 0x7efe20162860 2024-09-12T03:22:45.145 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:45.142+0000 7efe25731640 1 -- 172.21.15.27:0/3425574471 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe20162320 con 0x7efe20162860 2024-09-12T03:22:45.145 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:45.142+0000 7efe177fe640 1 -- 172.21.15.27:0/3425574471 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 22) ==== 91059+0+0 (secure 0 0 0) 0x7efe1801b070 con 0x7efe20162860 2024-09-12T03:22:45.145 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:45.143+0000 7efe177fe640 1 --2- 172.21.15.27:0/3425574471 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7efe0006c460 0x7efe0006e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:45.145 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:45.143+0000 7efe1effd640 1 --2- 172.21.15.27:0/3425574471 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7efe0006c460 0x7efe0006e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:45.145 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:45.143+0000 7efe177fe640 1 -- 172.21.15.27:0/3425574471 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 4999+0+0 (secure 0 0 0) 0x7efe1808aba0 con 0x7efe20162860 2024-09-12T03:22:45.145 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:45.143+0000 7efe1effd640 1 --2- 172.21.15.27:0/3425574471 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7efe0006c460 0x7efe0006e920 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7efe10034000 tx=0x7efe1002e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:45.147 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:45.147+0000 7efe177fe640 1 -- 172.21.15.27:0/3425574471 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7efe18058910 con 0x7efe20162860 2024-09-12T03:22:45.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:45.360+0000 7efe25731640 1 -- 172.21.15.27:0/3425574471 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efe2015e9d0 con 0x7efe20162860 2024-09-12T03:22:45.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:45.361+0000 7efe177fe640 1 -- 172.21.15.27:0/3425574471 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 78 v78) ==== 94+0+3911 (secure 0 0 0) 0x7efe1805cc10 con 0x7efe20162860 2024-09-12T03:22:45.362 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:22:45.362 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":78,"btime":"2024-09-12T03:22:40:259080+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6437,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/2244842464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2244842464},{"type":"v1","addr":"172.21.15.45:6834","nonce":2244842464}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":72},{"gid":7048,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/1976058956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":1976058956},{"type":"v1","addr":"172.21.15.27:6835","nonce":1976058956}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":72},{"gid":7060,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/1084669125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":1084669125},{"type":"v1","addr":"172.21.15.45:6835","nonce":1084669125}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":72},{"gid":8109,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6834/1928848266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":1928848266},{"type":"v1","addr":"172.21.15.27:6834","nonce":1928848266}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78}],"filesystems":[]} 2024-09-12T03:22:45.362 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 78 2024-09-12T03:22:45.364 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:45.363+0000 7efe25731640 1 -- 172.21.15.27:0/3425574471 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7efe0006c460 msgr2=0x7efe0006e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:45.364 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:45.363+0000 7efe25731640 1 --2- 172.21.15.27:0/3425574471 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7efe0006c460 0x7efe0006e920 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7efe10034000 tx=0x7efe1002e000 comp rx=0 tx=0).stop 2024-09-12T03:22:45.364 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:45.363+0000 7efe25731640 1 -- 172.21.15.27:0/3425574471 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efe20162860 msgr2=0x7efe20152a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:45.364 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:45.363+0000 7efe25731640 1 --2- 172.21.15.27:0/3425574471 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efe20162860 0x7efe20152a50 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7efe1800c7a0 tx=0x7efe1800cc60 comp rx=0 tx=0).stop 2024-09-12T03:22:45.364 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:45.363+0000 7efe25731640 1 -- 172.21.15.27:0/3425574471 shutdown_connections 2024-09-12T03:22:45.364 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:45.363+0000 7efe25731640 1 --2- 172.21.15.27:0/3425574471 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7efe0006c460 0x7efe0006e920 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:45.364 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:45.363+0000 7efe25731640 1 --2- 172.21.15.27:0/3425574471 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efe20162860 0x7efe20152a50 unknown :-1 s=CLOSED pgs=1373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:45.364 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:45.363+0000 7efe25731640 1 --2- 172.21.15.27:0/3425574471 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efe2015e5f0 0x7efe2015c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:45.364 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:45.363+0000 7efe25731640 1 --2- 172.21.15.27:0/3425574471 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7efe20073790 0x7efe20156370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:45.364 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:45.363+0000 7efe25731640 1 -- 172.21.15.27:0/3425574471 >> 172.21.15.27:0/3425574471 conn(0x7efe2006f300 msgr2=0x7efe20159170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:45.365 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:45.363+0000 7efe25731640 1 -- 172.21.15.27:0/3425574471 shutdown_connections 2024-09-12T03:22:45.365 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:45.363+0000 7efe25731640 1 -- 172.21.15.27:0/3425574471 wait complete. 2024-09-12T03:22:45.375 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-12T03:22:45.375 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-12T03:22:45.375 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-12T03:22:45.375 INFO:teuthology.orchestra.run:waiting for 300 2024-09-12T03:22:45.376 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-12T03:22:45.376 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-12T03:22:45.376 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-12T03:22:45.376 INFO:teuthology.orchestra.run:waiting for 300 2024-09-12T03:22:45.376 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-12T03:22:45.376 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-12T03:22:45.377 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-12T03:22:45.377 INFO:teuthology.orchestra.run:waiting for 300 2024-09-12T03:22:45.377 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-12T03:22:45.377 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-12T03:22:45.377 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-12T03:22:45.377 INFO:teuthology.orchestra.run:waiting for 300 2024-09-12T03:22:45.385 INFO:tasks.ceph.mds.a:Stopped 2024-09-12T03:22:45.385 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-12T03:22:45.386 INFO:tasks.ceph.mds.c:Stopped 2024-09-12T03:22:45.386 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-12T03:22:45.389 INFO:tasks.ceph.mds.b:Stopped 2024-09-12T03:22:45.389 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-12T03:22:45.389 INFO:tasks.ceph.mds.d:Stopped 2024-09-12T03:22:45.390 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-12T03:22:45.427 INFO:tasks.ceph.mds.a:Started 2024-09-12T03:22:45.427 DEBUG:teuthology.parallel:result is None 2024-09-12T03:22:45.428 INFO:tasks.ceph.mds.c:Started 2024-09-12T03:22:45.428 DEBUG:teuthology.parallel:result is None 2024-09-12T03:22:45.431 INFO:tasks.ceph.mds.b:Started 2024-09-12T03:22:45.431 DEBUG:teuthology.parallel:result is None 2024-09-12T03:22:45.432 INFO:tasks.ceph.mds.d:Started 2024-09-12T03:22:45.432 DEBUG:teuthology.parallel:result is None 2024-09-12T03:22:45.432 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-12T03:22:45.505 INFO:tasks.ceph.mds.c.smithi027.stdout:starting mds.c at 2024-09-12T03:22:45.505 INFO:tasks.ceph.mds.a.smithi027.stdout:starting mds.a at 2024-09-12T03:22:45.516 INFO:tasks.ceph.mds.b.smithi045.stdout:starting mds.b at 2024-09-12T03:22:45.517 INFO:tasks.ceph.mds.d.smithi045.stdout:starting mds.d at 2024-09-12T03:22:45.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:45.541+0000 7efeb3366640 1 Processor -- start 2024-09-12T03:22:45.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:45.541+0000 7efeb3366640 1 -- start start 2024-09-12T03:22:45.543 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:45.542+0000 7efeb3366640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7efeac15eb90 0x7efeac0d2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:45.543 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:45.542+0000 7efeb3366640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7efeac0d5200 0x7efeac0d1010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:45.543 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:45.542+0000 7efeb3366640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7efeac0d1550 0x7efeac0cf200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:45.544 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:45.542+0000 7efeb3366640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7efeac0b7d20 con 0x7efeac0d5200 2024-09-12T03:22:45.544 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:45.542+0000 7efeb3366640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7efeac0d42e0 con 0x7efeac0d1550 2024-09-12T03:22:45.544 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:45.542+0000 7efeb3366640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7efeac0d19e0 con 0x7efeac15eb90 2024-09-12T03:22:45.544 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:45.542+0000 7efeb0901640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7efeac0d5200 0x7efeac0d1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:45.544 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:45.542+0000 7efeb0901640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7efeac0d5200 0x7efeac0d1010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35506/0 (socket says 172.21.15.27:35506) 2024-09-12T03:22:45.544 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:45.542+0000 7efeb0901640 1 -- 172.21.15.27:0/132772435 learned_addr learned my addr 172.21.15.27:0/132772435 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:22:45.544 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:45.542+0000 7efeb1903640 1 --2- 172.21.15.27:0/132772435 >> v2:172.21.15.45:3300/0 conn(0x7efeac0d1550 0x7efeac0cf200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:45.544 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:45.542+0000 7efeb1102640 1 --2- 172.21.15.27:0/132772435 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7efeac15eb90 0x7efeac0d2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:45.544 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:45.542+0000 7efeb0901640 1 -- 172.21.15.27:0/132772435 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7efeac15eb90 msgr2=0x7efeac0d2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:45.544 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:45.542+0000 7efeb0901640 1 --2- 172.21.15.27:0/132772435 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7efeac15eb90 0x7efeac0d2df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:45.544 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:45.542+0000 7efeb0901640 1 -- 172.21.15.27:0/132772435 >> v2:172.21.15.45:3300/0 conn(0x7efeac0d1550 msgr2=0x7efeac0cf200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:45.544 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:45.542+0000 7efeb0901640 1 --2- 172.21.15.27:0/132772435 >> v2:172.21.15.45:3300/0 conn(0x7efeac0d1550 0x7efeac0cf200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:45.544 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:45.542+0000 7efeb0901640 1 -- 172.21.15.27:0/132772435 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efeac0d3760 con 0x7efeac0d5200 2024-09-12T03:22:45.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:45.543+0000 7efeb0901640 1 --2- 172.21.15.27:0/132772435 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efeac0d5200 0x7efeac0d1010 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7efeac08d510 tx=0x7efe9c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a5bb99b2e7fad25 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:45.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:45.543+0000 7efea3fff640 1 -- 172.21.15.27:0/132772435 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe9c037070 con 0x7efeac0d5200 2024-09-12T03:22:45.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:45.543+0000 7efea3fff640 1 -- 172.21.15.27:0/132772435 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe9c032440 con 0x7efeac0d5200 2024-09-12T03:22:45.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:45.543+0000 7efea3fff640 1 -- 172.21.15.27:0/132772435 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe9c032810 con 0x7efeac0d5200 2024-09-12T03:22:45.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:45.544+0000 7efeb3366640 1 -- 172.21.15.27:0/132772435 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efeac0d5200 msgr2=0x7efeac0d1010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:45.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:45.544+0000 7efeb3366640 1 --2- 172.21.15.27:0/132772435 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efeac0d5200 0x7efeac0d1010 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7efeac08d510 tx=0x7efe9c02a1e0 comp rx=0 tx=0).stop 2024-09-12T03:22:45.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:45.544+0000 7efeb3366640 1 -- 172.21.15.27:0/132772435 shutdown_connections 2024-09-12T03:22:45.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:45.544+0000 7efeb3366640 1 --2- 172.21.15.27:0/132772435 >> v2:172.21.15.45:3300/0 conn(0x7efeac0d1550 0x7efeac0cf200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:45.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:45.544+0000 7efeb3366640 1 --2- 172.21.15.27:0/132772435 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efeac0d5200 0x7efeac0d1010 unknown :-1 s=CLOSED pgs=1381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:45.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:45.544+0000 7efeb3366640 1 --2- 172.21.15.27:0/132772435 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7efeac15eb90 0x7efeac0d2df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:45.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:45.544+0000 7efeb3366640 1 -- 172.21.15.27:0/132772435 >> 172.21.15.27:0/132772435 conn(0x7efeac06dce0 msgr2=0x7efeac06e0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:45.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:45.544+0000 7efeb3366640 1 -- 172.21.15.27:0/132772435 shutdown_connections 2024-09-12T03:22:45.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:45.544+0000 7efeb3366640 1 -- 172.21.15.27:0/132772435 wait complete. 2024-09-12T03:22:45.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:45.545+0000 7efeb3366640 1 Processor -- start 2024-09-12T03:22:45.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:45.545+0000 7efeb3366640 1 -- start start 2024-09-12T03:22:45.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:45.546+0000 7efeb3366640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efeac0d1550 0x7efeac14ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:45.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:45.546+0000 7efeb3366640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7efeac0d5200 0x7efeac15b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:45.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:45.546+0000 7efeb3366640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efeac15eb90 0x7efeac0c9830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:45.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:45.546+0000 7efeb3366640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7efeac0d4000 con 0x7efeac15eb90 2024-09-12T03:22:45.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:45.546+0000 7efeb3366640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7efeac0d3d20 con 0x7efeac0d1550 2024-09-12T03:22:45.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:45.546+0000 7efeb3366640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7efeac0d3330 con 0x7efeac0d5200 2024-09-12T03:22:45.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:45.546+0000 7efeb1903640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efeac15eb90 0x7efeac0c9830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:45.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:45.546+0000 7efeb1903640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efeac15eb90 0x7efeac0c9830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35516/0 (socket says 172.21.15.27:35516) 2024-09-12T03:22:45.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:45.546+0000 7efeb1903640 1 -- 172.21.15.27:0/33465231 learned_addr learned my addr 172.21.15.27:0/33465231 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:22:45.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:45.546+0000 7efeb1102640 1 --2- 172.21.15.27:0/33465231 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efeac0d1550 0x7efeac14ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:45.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:45.546+0000 7efeb0901640 1 --2- 172.21.15.27:0/33465231 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7efeac0d5200 0x7efeac15b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:45.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:45.546+0000 7efeb1903640 1 -- 172.21.15.27:0/33465231 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7efeac0d5200 msgr2=0x7efeac15b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:45.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:45.546+0000 7efeb1903640 1 --2- 172.21.15.27:0/33465231 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7efeac0d5200 0x7efeac15b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:45.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:45.546+0000 7efeb1903640 1 -- 172.21.15.27:0/33465231 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efeac0d1550 msgr2=0x7efeac14ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:45.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:45.546+0000 7efeb1903640 1 --2- 172.21.15.27:0/33465231 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efeac0d1550 0x7efeac14ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:45.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:45.546+0000 7efeb1903640 1 -- 172.21.15.27:0/33465231 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efeac1593a0 con 0x7efeac15eb90 2024-09-12T03:22:45.551 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:45.547+0000 7efeb1903640 1 --2- 172.21.15.27:0/33465231 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efeac15eb90 0x7efeac0c9830 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7efea4009d20 tx=0x7efea400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:45.551 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:45.547+0000 7efea1ffb640 1 -- 172.21.15.27:0/33465231 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efea4018020 con 0x7efeac15eb90 2024-09-12T03:22:45.551 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:45.547+0000 7efea1ffb640 1 -- 172.21.15.27:0/33465231 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efea4009b10 con 0x7efeac15eb90 2024-09-12T03:22:45.551 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:45.547+0000 7efeb3366640 1 -- 172.21.15.27:0/33465231 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efeac159680 con 0x7efeac15eb90 2024-09-12T03:22:45.551 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:45.547+0000 7efea1ffb640 1 -- 172.21.15.27:0/33465231 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efea40146c0 con 0x7efeac15eb90 2024-09-12T03:22:45.551 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:45.547+0000 7efeb3366640 1 -- 172.21.15.27:0/33465231 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efeac159c40 con 0x7efeac15eb90 2024-09-12T03:22:45.551 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:45.548+0000 7efeb3366640 1 -- 172.21.15.27:0/33465231 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efeac15ef70 con 0x7efeac15eb90 2024-09-12T03:22:45.551 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:45.548+0000 7efea1ffb640 1 -- 172.21.15.27:0/33465231 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 22) ==== 91059+0+0 (secure 0 0 0) 0x7efea4005d10 con 0x7efeac15eb90 2024-09-12T03:22:45.551 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:45.549+0000 7efea1ffb640 1 --2- 172.21.15.27:0/33465231 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7efe9006c460 0x7efe9006e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:45.552 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:45.549+0000 7efea1ffb640 1 -- 172.21.15.27:0/33465231 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 4999+0+0 (secure 0 0 0) 0x7efea408c130 con 0x7efeac15eb90 2024-09-12T03:22:45.552 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:45.549+0000 7efeb1102640 1 --2- 172.21.15.27:0/33465231 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7efe9006c460 0x7efe9006e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:45.552 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:45.549+0000 7efeb1102640 1 --2- 172.21.15.27:0/33465231 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7efe9006c460 0x7efe9006e920 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7efea80099b0 tx=0x7efea8008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:45.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:45.553+0000 7efea1ffb640 1 -- 172.21.15.27:0/33465231 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7efea4022070 con 0x7efeac15eb90 2024-09-12T03:22:45.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:45.739+0000 7efeb3366640 1 -- 172.21.15.27:0/33465231 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7efeac15fd00 con 0x7efeac15eb90 2024-09-12T03:22:46.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:46.823+0000 7efea1ffb640 1 -- 172.21.15.27:0/33465231 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v134) ==== 97+0+0 (secure 0 0 0) 0x7efea4010040 con 0x7efeac15eb90 2024-09-12T03:22:46.825 INFO:teuthology.orchestra.run.smithi027.stderr: removed all blocklist entries 2024-09-12T03:22:46.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:46.826+0000 7efeb3366640 1 -- 172.21.15.27:0/33465231 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7efe9006c460 msgr2=0x7efe9006e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:46.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:46.826+0000 7efeb3366640 1 --2- 172.21.15.27:0/33465231 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7efe9006c460 0x7efe9006e920 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7efea80099b0 tx=0x7efea8008040 comp rx=0 tx=0).stop 2024-09-12T03:22:46.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:46.826+0000 7efeb3366640 1 -- 172.21.15.27:0/33465231 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efeac15eb90 msgr2=0x7efeac0c9830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:46.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:46.826+0000 7efeb3366640 1 --2- 172.21.15.27:0/33465231 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efeac15eb90 0x7efeac0c9830 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7efea4009d20 tx=0x7efea400c490 comp rx=0 tx=0).stop 2024-09-12T03:22:46.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:46.826+0000 7efeb3366640 1 -- 172.21.15.27:0/33465231 shutdown_connections 2024-09-12T03:22:46.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:46.826+0000 7efeb3366640 1 --2- 172.21.15.27:0/33465231 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7efe9006c460 0x7efe9006e920 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:46.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:46.826+0000 7efeb3366640 1 --2- 172.21.15.27:0/33465231 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efeac15eb90 0x7efeac0c9830 unknown :-1 s=CLOSED pgs=1382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:46.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:46.826+0000 7efeb3366640 1 --2- 172.21.15.27:0/33465231 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7efeac0d5200 0x7efeac15b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:46.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:46.826+0000 7efeb3366640 1 --2- 172.21.15.27:0/33465231 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efeac0d1550 0x7efeac14ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:46.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:46.826+0000 7efeb3366640 1 -- 172.21.15.27:0/33465231 >> 172.21.15.27:0/33465231 conn(0x7efeac06dce0 msgr2=0x7efeac0d27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:46.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:46.826+0000 7efeb3366640 1 -- 172.21.15.27:0/33465231 shutdown_connections 2024-09-12T03:22:46.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:46.826+0000 7efeb3366640 1 -- 172.21.15.27:0/33465231 wait complete. 2024-09-12T03:22:46.842 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-12T03:22:46.951 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:46.950+0000 7f8ce106e640 1 Processor -- start 2024-09-12T03:22:46.951 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:46.950+0000 7f8ce106e640 1 -- start start 2024-09-12T03:22:46.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:46.951+0000 7f8ce106e640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f8cdc0b91a0 0x7f8cdc169880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:46.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:46.951+0000 7f8ce106e640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8cdc0d3760 con 0x7f8cdc15eb90 2024-09-12T03:22:46.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:46.951+0000 7f8ce106e640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8cdc0d3a40 con 0x7f8cdc0d5200 2024-09-12T03:22:46.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:46.951+0000 7f8ce106e640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f8cdc0b7d20 con 0x7f8cdc0b91a0 2024-09-12T03:22:46.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:46.951+0000 7f8cdad9d640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f8cdc15eb90 0x7f8cdc0d2df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:55432/0 (socket says 172.21.15.27:55432) 2024-09-12T03:22:46.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:46.951+0000 7f8cdad9d640 1 -- 172.21.15.27:0/636808637 learned_addr learned my addr 172.21.15.27:0/636808637 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:22:46.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:46.952+0000 7f8cdb59e640 1 --2- 172.21.15.27:0/636808637 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f8cdc0b91a0 0x7f8cdc169880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:46.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:46.952+0000 7f8cd9d9b640 1 -- 172.21.15.27:0/636808637 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2017308210 0 0) 0x7f8cdc0d3760 con 0x7f8cdc15eb90 2024-09-12T03:22:46.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:46.952+0000 7f8cd9d9b640 1 -- 172.21.15.27:0/636808637 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8cc8003520 con 0x7f8cdc15eb90 2024-09-12T03:22:46.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:46.952+0000 7f8cd9d9b640 1 -- 172.21.15.27:0/636808637 <== mon.1 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3016328150 0 0) 0x7f8cdc0d3a40 con 0x7f8cdc0d5200 2024-09-12T03:22:46.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:46.952+0000 7f8cd9d9b640 1 -- 172.21.15.27:0/636808637 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8cdc0d3760 con 0x7f8cdc0d5200 2024-09-12T03:22:46.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:46.952+0000 7f8cd9d9b640 1 -- 172.21.15.27:0/636808637 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1725203953 0 0) 0x7f8cc8003520 con 0x7f8cdc15eb90 2024-09-12T03:22:46.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:46.952+0000 7f8cd9d9b640 1 -- 172.21.15.27:0/636808637 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f8cdc0b91a0 msgr2=0x7f8cdc169880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:46.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:46.952+0000 7f8cd9d9b640 1 --2- 172.21.15.27:0/636808637 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f8cdc0b91a0 0x7f8cdc169880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:46.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:46.952+0000 7f8cd9d9b640 1 -- 172.21.15.27:0/636808637 >> v1:172.21.15.45:6789/0 conn(0x7f8cdc0d5200 legacy=0x7f8cdc0d1010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:46.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:46.952+0000 7f8cd9d9b640 1 -- 172.21.15.27:0/636808637 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cdc0d3d20 con 0x7f8cdc15eb90 2024-09-12T03:22:46.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:46.952+0000 7f8cd9d9b640 1 -- 172.21.15.27:0/636808637 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 847755213 0 0) 0x7f8ccc0019f0 con 0x7f8cdc15eb90 2024-09-12T03:22:46.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:46.952+0000 7f8cd9d9b640 1 -- 172.21.15.27:0/636808637 >> v1:172.21.15.27:6789/0 conn(0x7f8cdc15eb90 legacy=0x7f8cdc0d2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:46.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:46.952+0000 7f8cd9d9b640 1 --2- 172.21.15.27:0/636808637 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8cc8003b70 0x7f8cc8003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:46.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:46.952+0000 7f8cd9d9b640 1 --2- 172.21.15.27:0/636808637 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8cc8004800 0x7f8cc8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:46.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:46.952+0000 7f8cd9d9b640 1 --2- 172.21.15.27:0/636808637 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f8cdc0b91a0 0x7f8cc80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:46.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:46.952+0000 7f8cd9d9b640 1 -- 172.21.15.27:0/636808637 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f8cc800ba70 con 0x7f8cc8004800 2024-09-12T03:22:46.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:46.952+0000 7f8cd9d9b640 1 -- 172.21.15.27:0/636808637 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f8cc800bbf0 con 0x7f8cc8003b70 2024-09-12T03:22:46.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:46.952+0000 7f8cd9d9b640 1 -- 172.21.15.27:0/636808637 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f8cc800bdc0 con 0x7f8cdc0b91a0 2024-09-12T03:22:46.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:46.952+0000 7f8cda59c640 1 --2- 172.21.15.27:0/636808637 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8cc8004800 0x7f8cc8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:46.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:46.952+0000 7f8cdad9d640 1 --2- 172.21.15.27:0/636808637 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8cc8003b70 0x7f8cc8003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:46.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:46.952+0000 7f8cdb59e640 1 --2- 172.21.15.27:0/636808637 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f8cdc0b91a0 0x7f8cc80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:46.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:46.952+0000 7f8cda59c640 1 -- 172.21.15.27:0/636808637 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f8cdc0b91a0 msgr2=0x7f8cc80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:46.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:46.952+0000 7f8cda59c640 1 --2- 172.21.15.27:0/636808637 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f8cdc0b91a0 0x7f8cc80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:46.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:46.952+0000 7f8cda59c640 1 -- 172.21.15.27:0/636808637 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8cc8003b70 msgr2=0x7f8cc8003f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:46.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:46.952+0000 7f8cda59c640 1 --2- 172.21.15.27:0/636808637 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8cc8003b70 0x7f8cc8003f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:46.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:46.952+0000 7f8cda59c640 1 -- 172.21.15.27:0/636808637 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cc8003520 con 0x7f8cc8004800 2024-09-12T03:22:46.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:46.952+0000 7f8cdad9d640 1 --2- 172.21.15.27:0/636808637 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8cc8003b70 0x7f8cc8003f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:22:46.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:46.953+0000 7f8cda59c640 1 --2- 172.21.15.27:0/636808637 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8cc8004800 0x7f8cc8004c80 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f8cc40018a0 tx=0x7f8cc8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b47fd010146d1af6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:46.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:46.953+0000 7f8cd9d9b640 1 -- 172.21.15.27:0/636808637 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cc40014f0 con 0x7f8cc8004800 2024-09-12T03:22:46.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:46.953+0000 7f8cd9d9b640 1 -- 172.21.15.27:0/636808637 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8cc40052d0 con 0x7f8cc8004800 2024-09-12T03:22:46.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:46.953+0000 7f8cd9d9b640 1 -- 172.21.15.27:0/636808637 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cc402ac00 con 0x7f8cc8004800 2024-09-12T03:22:46.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:46.954+0000 7f8ce106e640 1 -- 172.21.15.27:0/636808637 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8cc8004800 msgr2=0x7f8cc8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:46.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:46.954+0000 7f8ce106e640 1 --2- 172.21.15.27:0/636808637 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8cc8004800 0x7f8cc8004c80 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f8cc40018a0 tx=0x7f8cc8005830 comp rx=0 tx=0).stop 2024-09-12T03:22:46.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:46.954+0000 7f8cdb59e640 1 -- 172.21.15.27:0/636808637 reap_dead start 2024-09-12T03:22:46.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:46.954+0000 7f8ce106e640 1 -- 172.21.15.27:0/636808637 shutdown_connections 2024-09-12T03:22:46.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:46.954+0000 7f8ce106e640 1 -- 172.21.15.27:0/636808637 >> 172.21.15.27:0/636808637 conn(0x7f8cdc06dce0 msgr2=0x7f8cdc06e0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:46.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:46.954+0000 7f8ce106e640 1 -- 172.21.15.27:0/636808637 shutdown_connections 2024-09-12T03:22:46.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:46.954+0000 7f8ce106e640 1 -- 172.21.15.27:0/636808637 wait complete. 2024-09-12T03:22:46.961 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:46.955+0000 7f8ce106e640 1 Processor -- start 2024-09-12T03:22:46.961 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:46.956+0000 7f8ce106e640 1 -- start start 2024-09-12T03:22:46.961 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:46.956+0000 7f8ce106e640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f8cc8004800 0x7f8cdc149140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:46.961 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:46.956+0000 7f8ce106e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8cdc0c9cf0 0x7f8cdc0ca170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:46.961 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:46.956+0000 7f8ce106e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8cdc0c8600 0x7f8cdc0c8a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:46.961 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:46.956+0000 7f8ce106e640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f8cdc0d3500 con 0x7f8cdc0c8600 2024-09-12T03:22:46.961 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:46.956+0000 7f8ce106e640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f8cdc0d42e0 con 0x7f8cdc0c9cf0 2024-09-12T03:22:46.961 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:46.956+0000 7f8ce106e640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f8cdc15ef70 con 0x7f8cc8004800 2024-09-12T03:22:46.961 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:46.956+0000 7f8cdb59e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8cdc0c8600 0x7f8cdc0c8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:46.962 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:46.956+0000 7f8cdb59e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8cdc0c8600 0x7f8cdc0c8a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35522/0 (socket says 172.21.15.27:35522) 2024-09-12T03:22:46.962 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:46.956+0000 7f8cdb59e640 1 -- 172.21.15.27:0/492126836 learned_addr learned my addr 172.21.15.27:0/492126836 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:22:46.962 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:46.957+0000 7f8cdad9d640 1 --2- 172.21.15.27:0/492126836 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f8cc8004800 0x7f8cdc149140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:46.962 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:46.957+0000 7f8cda59c640 1 --2- 172.21.15.27:0/492126836 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8cdc0c9cf0 0x7f8cdc0ca170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:46.962 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:46.957+0000 7f8cdb59e640 1 -- 172.21.15.27:0/492126836 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f8cc8004800 msgr2=0x7f8cdc149140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:46.962 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:46.957+0000 7f8cdb59e640 1 --2- 172.21.15.27:0/492126836 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f8cc8004800 0x7f8cdc149140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:46.962 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:46.957+0000 7f8cdb59e640 1 -- 172.21.15.27:0/492126836 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8cdc0c9cf0 msgr2=0x7f8cdc0ca170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:46.962 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:46.957+0000 7f8cdb59e640 1 --2- 172.21.15.27:0/492126836 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8cdc0c9cf0 0x7f8cdc0ca170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:46.962 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:46.957+0000 7f8cdb59e640 1 -- 172.21.15.27:0/492126836 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cdc158530 con 0x7f8cdc0c8600 2024-09-12T03:22:46.962 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:46.957+0000 7f8cdb59e640 1 --2- 172.21.15.27:0/492126836 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8cdc0c8600 0x7f8cdc0c8a80 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7f8cdc0b7400 tx=0x7f8cd4011860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:46.962 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:46.957+0000 7f8cd37fe640 1 -- 172.21.15.27:0/492126836 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cd4019020 con 0x7f8cdc0c8600 2024-09-12T03:22:46.962 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:46.957+0000 7f8cd37fe640 1 -- 172.21.15.27:0/492126836 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8cd4004440 con 0x7f8cdc0c8600 2024-09-12T03:22:46.962 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:46.957+0000 7f8ce106e640 1 -- 172.21.15.27:0/492126836 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8cdc149680 con 0x7f8cdc0c8600 2024-09-12T03:22:46.963 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:46.957+0000 7f8cd37fe640 1 -- 172.21.15.27:0/492126836 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cd4003850 con 0x7f8cdc0c8600 2024-09-12T03:22:46.963 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:46.957+0000 7f8ce106e640 1 -- 172.21.15.27:0/492126836 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8cdc0c9660 con 0x7f8cdc0c8600 2024-09-12T03:22:46.963 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:46.958+0000 7f8ce106e640 1 -- 172.21.15.27:0/492126836 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8cdc0d2df0 con 0x7f8cdc0c8600 2024-09-12T03:22:46.963 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:46.959+0000 7f8cd37fe640 1 -- 172.21.15.27:0/492126836 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 22) ==== 91059+0+0 (secure 0 0 0) 0x7f8cd401c020 con 0x7f8cdc0c8600 2024-09-12T03:22:46.963 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:46.959+0000 7f8cd37fe640 1 --2- 172.21.15.27:0/492126836 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f8cbc06c460 0x7f8cbc06e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:46.963 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:46.960+0000 7f8cd37fe640 1 -- 172.21.15.27:0/492126836 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 4913+0+0 (secure 0 0 0) 0x7f8cd408b5f0 con 0x7f8cdc0c8600 2024-09-12T03:22:46.963 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:46.960+0000 7f8cdad9d640 1 --2- 172.21.15.27:0/492126836 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f8cbc06c460 0x7f8cbc06e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:46.963 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:46.960+0000 7f8cdad9d640 1 --2- 172.21.15.27:0/492126836 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f8cbc06c460 0x7f8cbc06e920 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f8ccc002230 tx=0x7f8ccc002180 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:46.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:46.964+0000 7f8cd37fe640 1 -- 172.21.15.27:0/492126836 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f8cd4023030 con 0x7f8cdc0c8600 2024-09-12T03:22:47.207 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:47.206+0000 7f8ce106e640 1 -- 172.21.15.27:0/492126836 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f8cdc0c7ea0 con 0x7f8cdc0c8600 2024-09-12T03:22:47.207 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:47.206+0000 7f8cd37fe640 1 -- 172.21.15.27:0/492126836 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v20) ==== 80+0+5304 (secure 0 0 0) 0x7f8cd4059a00 con 0x7f8cdc0c8600 2024-09-12T03:22:47.208 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:22:47.208 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-12T03:22:47.208 INFO:teuthology.orchestra.run.smithi027.stdout: "auth_dump": [ 2024-09-12T03:22:47.208 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:22:47.208 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "mds.a", 2024-09-12T03:22:47.208 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDUXOJmIhxjNxAAaGg8SwtW6NRe/lIgNCV20w==", 2024-09-12T03:22:47.208 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:22:47.208 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow", 2024-09-12T03:22:47.208 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow *", 2024-09-12T03:22:47.208 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow *", 2024-09-12T03:22:47.208 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2024-09-12T03:22:47.208 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:22:47.209 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:22:47.209 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:22:47.209 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "mds.b", 2024-09-12T03:22:47.209 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDVXOJm63kWCBAAzjfOsBsWUI+IAsuZ49yEkw==", 2024-09-12T03:22:47.209 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:22:47.209 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow", 2024-09-12T03:22:47.209 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow *", 2024-09-12T03:22:47.209 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow *", 2024-09-12T03:22:47.209 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2024-09-12T03:22:47.209 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:22:47.209 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:22:47.209 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:22:47.209 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "mds.c", 2024-09-12T03:22:47.210 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDVXOJm0sX7AhAAdynCcho2a/AgFF9Cw3+vAw==", 2024-09-12T03:22:47.210 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:22:47.210 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow", 2024-09-12T03:22:47.210 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow *", 2024-09-12T03:22:47.210 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow *", 2024-09-12T03:22:47.210 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2024-09-12T03:22:47.210 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:22:47.210 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:22:47.210 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:22:47.210 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "mds.d", 2024-09-12T03:22:47.210 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDVXOJm7Dd8DxAANHYmvKgpRZ6sVfXDSgPHWw==", 2024-09-12T03:22:47.210 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:22:47.211 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow", 2024-09-12T03:22:47.211 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow *", 2024-09-12T03:22:47.211 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow *", 2024-09-12T03:22:47.211 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2024-09-12T03:22:47.211 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:22:47.211 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:22:47.211 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:22:47.211 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "osd.0", 2024-09-12T03:22:47.211 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDYXOJmJTVYLxAAmo8kxRjaiwTT44dZLZlq4A==", 2024-09-12T03:22:47.211 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:22:47.211 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow profile osd", 2024-09-12T03:22:47.211 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile osd", 2024-09-12T03:22:47.211 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2024-09-12T03:22:47.211 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:22:47.212 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:22:47.212 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:22:47.212 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "osd.1", 2024-09-12T03:22:47.212 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDcXOJmYV0BFBAAmoGX9gf9yLoE/qkWFnv9tg==", 2024-09-12T03:22:47.212 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:22:47.212 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow profile osd", 2024-09-12T03:22:47.212 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile osd", 2024-09-12T03:22:47.212 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2024-09-12T03:22:47.212 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:22:47.212 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:22:47.212 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:22:47.212 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "osd.2", 2024-09-12T03:22:47.212 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDfXOJmEetiJRAAXk+AvfTfhjsB6T2UB4Z2/w==", 2024-09-12T03:22:47.213 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:22:47.213 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow profile osd", 2024-09-12T03:22:47.213 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile osd", 2024-09-12T03:22:47.213 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2024-09-12T03:22:47.213 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:22:47.213 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:22:47.213 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:22:47.213 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "osd.3", 2024-09-12T03:22:47.213 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDiXOJmrQbPJxAAQsi2B5a1LnPTPGZVdo7j7w==", 2024-09-12T03:22:47.213 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:22:47.213 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow profile osd", 2024-09-12T03:22:47.213 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile osd", 2024-09-12T03:22:47.213 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2024-09-12T03:22:47.214 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:22:47.214 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:22:47.214 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:22:47.214 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "osd.4", 2024-09-12T03:22:47.214 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDnXOJmoMr1NxAAv3u2q8hYt9w//9gz+N7UAQ==", 2024-09-12T03:22:47.214 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:22:47.214 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow profile osd", 2024-09-12T03:22:47.214 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile osd", 2024-09-12T03:22:47.214 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2024-09-12T03:22:47.214 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:22:47.214 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:22:47.214 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:22:47.215 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "osd.5", 2024-09-12T03:22:47.215 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDqXOJmxpUPOxAAxLH6RJsyLOZ2XMfzM0fQ1g==", 2024-09-12T03:22:47.215 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:22:47.215 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow profile osd", 2024-09-12T03:22:47.215 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile osd", 2024-09-12T03:22:47.215 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2024-09-12T03:22:47.215 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:22:47.215 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:22:47.215 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:22:47.215 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "osd.6", 2024-09-12T03:22:47.215 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDuXOJmuP0+LxAAMHoz5eSO81hPtkdBIS2cig==", 2024-09-12T03:22:47.215 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:22:47.215 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow profile osd", 2024-09-12T03:22:47.216 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile osd", 2024-09-12T03:22:47.216 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2024-09-12T03:22:47.216 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:22:47.216 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:22:47.216 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:22:47.216 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "osd.7", 2024-09-12T03:22:47.216 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDxXOJmuIC/MRAAPDM+lAHqdoQyGVODwjFp+Q==", 2024-09-12T03:22:47.216 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:22:47.216 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow profile osd", 2024-09-12T03:22:47.216 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile osd", 2024-09-12T03:22:47.216 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2024-09-12T03:22:47.216 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:22:47.216 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:22:47.217 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:22:47.217 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "client.0", 2024-09-12T03:22:47.217 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDVXOJmLm+5ExAAECM4CKfuVElmp/n0b02mXg==", 2024-09-12T03:22:47.217 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:22:47.217 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow", 2024-09-12T03:22:47.217 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow r", 2024-09-12T03:22:47.217 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-12T03:22:47.217 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:22:47.217 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:22:47.217 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:22:47.217 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "client.1", 2024-09-12T03:22:47.217 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDVXOJm61/FHBAALCQouXySfQeZ5DESGmqkzA==", 2024-09-12T03:22:47.217 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:22:47.217 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow", 2024-09-12T03:22:47.218 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow r", 2024-09-12T03:22:47.218 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow rw", 2024-09-12T03:22:47.218 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow rwx" 2024-09-12T03:22:47.218 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:22:47.218 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:22:47.218 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:22:47.218 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "client.2", 2024-09-12T03:22:47.218 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDVXOJmq/ZkGRAAOzmQzEIhb4b/YTxayc34Bw==", 2024-09-12T03:22:47.218 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:22:47.218 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow", 2024-09-12T03:22:47.218 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow r", 2024-09-12T03:22:47.218 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow rw", 2024-09-12T03:22:47.218 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow rwx" 2024-09-12T03:22:47.219 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:22:47.219 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:22:47.219 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:22:47.219 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "client.3", 2024-09-12T03:22:47.219 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDVXOJm+DlzIhAATQKZe+Zyk0MM8efTa6g9PA==", 2024-09-12T03:22:47.219 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:22:47.219 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow", 2024-09-12T03:22:47.219 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow r", 2024-09-12T03:22:47.219 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow rw", 2024-09-12T03:22:47.219 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow rwx" 2024-09-12T03:22:47.219 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:22:47.220 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:22:47.220 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:22:47.220 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "client.admin", 2024-09-12T03:22:47.220 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDUXOJm+JGWGhAApTQw9RtJWd/J5QzQaUFXvg==", 2024-09-12T03:22:47.220 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:22:47.220 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow *", 2024-09-12T03:22:47.220 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow *", 2024-09-12T03:22:47.220 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow *", 2024-09-12T03:22:47.220 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2024-09-12T03:22:47.220 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:22:47.220 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:22:47.221 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:22:47.221 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "mgr.x", 2024-09-12T03:22:47.221 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDUXOJmZcsONBAAMTY2KykuHaEkv1t6Vz5KjA==", 2024-09-12T03:22:47.221 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:22:47.221 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow *", 2024-09-12T03:22:47.221 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile mgr", 2024-09-12T03:22:47.221 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2024-09-12T03:22:47.221 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:22:47.221 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:22:47.221 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:22:47.221 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "mgr.y", 2024-09-12T03:22:47.221 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDUXOJmJOgZMBAAQ/JTxfhox7HgxBMtsvUCBg==", 2024-09-12T03:22:47.221 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:22:47.222 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow *", 2024-09-12T03:22:47.222 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile mgr", 2024-09-12T03:22:47.222 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2024-09-12T03:22:47.222 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:22:47.222 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:22:47.222 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-12T03:22:47.222 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-12T03:22:47.223 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:47.208+0000 7f8ce106e640 1 -- 172.21.15.27:0/492126836 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f8cbc06c460 msgr2=0x7f8cbc06e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:47.223 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:47.208+0000 7f8ce106e640 1 --2- 172.21.15.27:0/492126836 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f8cbc06c460 0x7f8cbc06e920 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f8ccc002230 tx=0x7f8ccc002180 comp rx=0 tx=0).stop 2024-09-12T03:22:47.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:47.208+0000 7f8ce106e640 1 -- 172.21.15.27:0/492126836 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8cdc0c8600 msgr2=0x7f8cdc0c8a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:47.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:47.208+0000 7f8ce106e640 1 --2- 172.21.15.27:0/492126836 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8cdc0c8600 0x7f8cdc0c8a80 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7f8cdc0b7400 tx=0x7f8cd4011860 comp rx=0 tx=0).stop 2024-09-12T03:22:47.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:47.208+0000 7f8ce106e640 1 -- 172.21.15.27:0/492126836 shutdown_connections 2024-09-12T03:22:47.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:47.208+0000 7f8ce106e640 1 --2- 172.21.15.27:0/492126836 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f8cbc06c460 0x7f8cbc06e920 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:47.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:47.208+0000 7f8ce106e640 1 --2- 172.21.15.27:0/492126836 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8cdc0c8600 0x7f8cdc0c8a80 unknown :-1 s=CLOSED pgs=1385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:47.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:47.208+0000 7f8ce106e640 1 --2- 172.21.15.27:0/492126836 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8cdc0c9cf0 0x7f8cdc0ca170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:47.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:47.208+0000 7f8ce106e640 1 --2- 172.21.15.27:0/492126836 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f8cc8004800 0x7f8cdc149140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:47.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:47.208+0000 7f8ce106e640 1 -- 172.21.15.27:0/492126836 >> 172.21.15.27:0/492126836 conn(0x7f8cdc06dce0 msgr2=0x7f8cdc161010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:47.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:47.209+0000 7f8ce106e640 1 -- 172.21.15.27:0/492126836 shutdown_connections 2024-09-12T03:22:47.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:47.209+0000 7f8ce106e640 1 -- 172.21.15.27:0/492126836 wait complete. 2024-09-12T03:22:47.225 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-12T03:22:47.327 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:47.326+0000 7f24cd533640 1 Processor -- start 2024-09-12T03:22:47.327 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:47.326+0000 7f24cd533640 1 -- start start 2024-09-12T03:22:47.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:47.327+0000 7f24cd533640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f24c8145090 0x7f24c80affb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:47.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:47.327+0000 7f24cd533640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f24c806bb30 0x7f24c806bf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:47.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:47.327+0000 7f24cd533640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f24c815f260 con 0x7f24c80ae8d0 2024-09-12T03:22:47.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:47.327+0000 7f24cd533640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f24c806c4b0 con 0x7f24c806bb30 2024-09-12T03:22:47.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:47.327+0000 7f24cd533640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f24c815f540 con 0x7f24c8145090 2024-09-12T03:22:47.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:47.328+0000 7f24c67fc640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f24c806bb30 0x7f24c806bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:47.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:47.328+0000 7f24c6ffd640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f24c8145090 0x7f24c80affb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:47.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:47.328+0000 7f24c77fe640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f24c80ae8d0 0x7f24c80aed20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.27:35804/0 (socket says 172.21.15.27:35804) 2024-09-12T03:22:47.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:47.328+0000 7f24c77fe640 1 -- 172.21.15.27:0/1342994601 learned_addr learned my addr 172.21.15.27:0/1342994601 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:22:47.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:47.328+0000 7f24c67fc640 1 -- 172.21.15.27:0/1342994601 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f24c8145090 msgr2=0x7f24c80affb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:47.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:47.328+0000 7f24c67fc640 1 --2- 172.21.15.27:0/1342994601 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f24c8145090 0x7f24c80affb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:47.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:47.328+0000 7f24c67fc640 1 -- 172.21.15.27:0/1342994601 >> v1:172.21.15.45:6789/0 conn(0x7f24c80ae8d0 legacy=0x7f24c80aed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:47.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:47.328+0000 7f24c67fc640 1 -- 172.21.15.27:0/1342994601 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24c815f820 con 0x7f24c806bb30 2024-09-12T03:22:47.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:47.329+0000 7f24c67fc640 1 --2- 172.21.15.27:0/1342994601 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f24c806bb30 0x7f24c806bf70 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f24bc0097a0 tx=0x7f24bc02a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dbf08bd3ab18d62 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:47.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:47.329+0000 7f24c5ffb640 1 -- 172.21.15.27:0/1342994601 <== mon.1 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24bc037070 con 0x7f24c806bb30 2024-09-12T03:22:47.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:47.329+0000 7f24c5ffb640 1 -- 172.21.15.27:0/1342994601 <== mon.1 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24bc02ae50 con 0x7f24c806bb30 2024-09-12T03:22:47.331 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:47.329+0000 7f24c5ffb640 1 -- 172.21.15.27:0/1342994601 <== mon.1 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24bc032660 con 0x7f24c806bb30 2024-09-12T03:22:47.332 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:47.330+0000 7f24cd533640 1 -- 172.21.15.27:0/1342994601 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f24c806bb30 msgr2=0x7f24c806bf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:47.332 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:47.330+0000 7f24cd533640 1 --2- 172.21.15.27:0/1342994601 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f24c806bb30 0x7f24c806bf70 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f24bc0097a0 tx=0x7f24bc02a100 comp rx=0 tx=0).stop 2024-09-12T03:22:47.332 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:47.330+0000 7f24cd533640 1 -- 172.21.15.27:0/1342994601 shutdown_connections 2024-09-12T03:22:47.332 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:47.330+0000 7f24cd533640 1 --2- 172.21.15.27:0/1342994601 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f24c806bb30 0x7f24c806bf70 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:47.332 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:47.330+0000 7f24cd533640 1 --2- 172.21.15.27:0/1342994601 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f24c8145090 0x7f24c80affb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:47.332 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:47.330+0000 7f24cd533640 1 -- 172.21.15.27:0/1342994601 >> 172.21.15.27:0/1342994601 conn(0x7f24c806d270 msgr2=0x7f24c80b3270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:47.332 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:47.330+0000 7f24cd533640 1 -- 172.21.15.27:0/1342994601 shutdown_connections 2024-09-12T03:22:47.332 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:47.330+0000 7f24cd533640 1 -- 172.21.15.27:0/1342994601 wait complete. 2024-09-12T03:22:47.332 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:47.331+0000 7f24cd533640 1 Processor -- start 2024-09-12T03:22:47.332 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:47.331+0000 7f24cd533640 1 -- start start 2024-09-12T03:22:47.334 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:47.332+0000 7f24cd533640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f24c806bb30 0x7f24c8140b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:47.334 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:47.332+0000 7f24cd533640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f24c80ae8d0 0x7f24c81f75c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:47.334 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:47.332+0000 7f24cd533640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f24c8145090 0x7f24c806ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:47.334 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:47.332+0000 7f24cd533640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f24c815fde0 con 0x7f24c80ae8d0 2024-09-12T03:22:47.334 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:47.332+0000 7f24cd533640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f24c815ef50 con 0x7f24c806bb30 2024-09-12T03:22:47.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:47.332+0000 7f24cd533640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f24c80b04f0 con 0x7f24c8145090 2024-09-12T03:22:47.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:47.332+0000 7f24c67fc640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f24c80ae8d0 0x7f24c81f75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:47.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:47.332+0000 7f24c67fc640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f24c80ae8d0 0x7f24c81f75c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35534/0 (socket says 172.21.15.27:35534) 2024-09-12T03:22:47.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:47.332+0000 7f24c67fc640 1 -- 172.21.15.27:0/4092154302 learned_addr learned my addr 172.21.15.27:0/4092154302 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:22:47.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:47.332+0000 7f24c77fe640 1 --2- 172.21.15.27:0/4092154302 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f24c8145090 0x7f24c806ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:47.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:47.332+0000 7f24c6ffd640 1 --2- 172.21.15.27:0/4092154302 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f24c806bb30 0x7f24c8140b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:47.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:47.332+0000 7f24c67fc640 1 -- 172.21.15.27:0/4092154302 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f24c8145090 msgr2=0x7f24c806ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:47.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:47.332+0000 7f24c67fc640 1 --2- 172.21.15.27:0/4092154302 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f24c8145090 0x7f24c806ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:47.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:47.332+0000 7f24c67fc640 1 -- 172.21.15.27:0/4092154302 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f24c806bb30 msgr2=0x7f24c8140b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:47.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:47.332+0000 7f24c67fc640 1 --2- 172.21.15.27:0/4092154302 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f24c806bb30 0x7f24c8140b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:47.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:47.332+0000 7f24c67fc640 1 -- 172.21.15.27:0/4092154302 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24c8141090 con 0x7f24c80ae8d0 2024-09-12T03:22:47.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:47.332+0000 7f24c6ffd640 1 --2- 172.21.15.27:0/4092154302 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f24c806bb30 0x7f24c8140b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:22:47.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:47.332+0000 7f24c77fe640 1 --2- 172.21.15.27:0/4092154302 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f24c8145090 0x7f24c806ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:22:47.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:47.332+0000 7f24c67fc640 1 --2- 172.21.15.27:0/4092154302 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f24c80ae8d0 0x7f24c81f75c0 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7f24bc02a140 tx=0x7f24bc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:47.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:47.333+0000 7f24af7fe640 1 -- 172.21.15.27:0/4092154302 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24bc041070 con 0x7f24c80ae8d0 2024-09-12T03:22:47.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:47.333+0000 7f24af7fe640 1 -- 172.21.15.27:0/4092154302 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24bc03c670 con 0x7f24c80ae8d0 2024-09-12T03:22:47.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:47.333+0000 7f24cd533640 1 -- 172.21.15.27:0/4092154302 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24c81f6520 con 0x7f24c80ae8d0 2024-09-12T03:22:47.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:47.333+0000 7f24af7fe640 1 -- 172.21.15.27:0/4092154302 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24bc037040 con 0x7f24c80ae8d0 2024-09-12T03:22:47.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:47.333+0000 7f24cd533640 1 -- 172.21.15.27:0/4092154302 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24c81f6ae0 con 0x7f24c80ae8d0 2024-09-12T03:22:47.337 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:47.334+0000 7f24cd533640 1 -- 172.21.15.27:0/4092154302 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24c80afad0 con 0x7f24c80ae8d0 2024-09-12T03:22:47.337 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:47.334+0000 7f24af7fe640 1 -- 172.21.15.27:0/4092154302 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 22) ==== 91059+0+0 (secure 0 0 0) 0x7f24bc044070 con 0x7f24c80ae8d0 2024-09-12T03:22:47.337 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:47.335+0000 7f24af7fe640 1 --2- 172.21.15.27:0/4092154302 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f24b406c460 0x7f24b406e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:47.337 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:47.335+0000 7f24c6ffd640 1 --2- 172.21.15.27:0/4092154302 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f24b406c460 0x7f24b406e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:47.337 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:47.335+0000 7f24af7fe640 1 -- 172.21.15.27:0/4092154302 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 4913+0+0 (secure 0 0 0) 0x7f24bc0abd70 con 0x7f24c80ae8d0 2024-09-12T03:22:47.337 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:47.335+0000 7f24c6ffd640 1 --2- 172.21.15.27:0/4092154302 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f24b406c460 0x7f24b406e920 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f24c80b3b10 tx=0x7f24b8008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:47.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:47.339+0000 7f24af7fe640 1 -- 172.21.15.27:0/4092154302 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f24bc07a0f0 con 0x7f24c80ae8d0 2024-09-12T03:22:47.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:47.516+0000 7f24cd533640 1 -- 172.21.15.27:0/4092154302 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f24c806fad0 con 0x7f24c80ae8d0 2024-09-12T03:22:47.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:47.516+0000 7f24af7fe640 1 -- 172.21.15.27:0/4092154302 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v134) ==== 84+0+65 (secure 0 0 0) 0x7f24bc0799f0 con 0x7f24c80ae8d0 2024-09-12T03:22:47.518 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:22:47.518 INFO:teuthology.orchestra.run.smithi027.stdout:[ 2024-09-12T03:22:47.518 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:22:47.518 INFO:teuthology.orchestra.run.smithi027.stdout: "poolnum": 1, 2024-09-12T03:22:47.518 INFO:teuthology.orchestra.run.smithi027.stdout: "poolname": ".mgr" 2024-09-12T03:22:47.518 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:22:47.518 INFO:teuthology.orchestra.run.smithi027.stdout:] 2024-09-12T03:22:47.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:47.518+0000 7f24cd533640 1 -- 172.21.15.27:0/4092154302 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f24b406c460 msgr2=0x7f24b406e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:47.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:47.518+0000 7f24cd533640 1 --2- 172.21.15.27:0/4092154302 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f24b406c460 0x7f24b406e920 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f24c80b3b10 tx=0x7f24b8008040 comp rx=0 tx=0).stop 2024-09-12T03:22:47.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:47.518+0000 7f24cd533640 1 -- 172.21.15.27:0/4092154302 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f24c80ae8d0 msgr2=0x7f24c81f75c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:47.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:47.518+0000 7f24cd533640 1 --2- 172.21.15.27:0/4092154302 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f24c80ae8d0 0x7f24c81f75c0 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7f24bc02a140 tx=0x7f24bc0046c0 comp rx=0 tx=0).stop 2024-09-12T03:22:47.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:47.518+0000 7f24cd533640 1 -- 172.21.15.27:0/4092154302 shutdown_connections 2024-09-12T03:22:47.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:47.518+0000 7f24cd533640 1 --2- 172.21.15.27:0/4092154302 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f24c8145090 0x7f24c806ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:47.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:47.518+0000 7f24cd533640 1 --2- 172.21.15.27:0/4092154302 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f24b406c460 0x7f24b406e920 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:47.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:47.518+0000 7f24cd533640 1 --2- 172.21.15.27:0/4092154302 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f24c80ae8d0 0x7f24c81f75c0 unknown :-1 s=CLOSED pgs=1386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:47.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:47.518+0000 7f24cd533640 1 --2- 172.21.15.27:0/4092154302 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f24c806bb30 0x7f24c8140b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:47.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:47.519+0000 7f24cd533640 1 -- 172.21.15.27:0/4092154302 >> 172.21.15.27:0/4092154302 conn(0x7f24c806d270 msgr2=0x7f24c81611d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:47.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:47.519+0000 7f24cd533640 1 -- 172.21.15.27:0/4092154302 shutdown_connections 2024-09-12T03:22:47.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:47.519+0000 7f24cd533640 1 -- 172.21.15.27:0/4092154302 wait complete. 2024-09-12T03:22:47.531 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-12T03:22:47.531 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-12T03:22:47.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:47.636+0000 7fbf0b94b640 1 Processor -- start 2024-09-12T03:22:47.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:47.636+0000 7fbf0b94b640 1 -- start start 2024-09-12T03:22:47.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:47.637+0000 7fbf0b94b640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fbf04162040 0x7fbf040d1da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:47.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:47.637+0000 7fbf0b94b640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fbf0406dfd0 0x7fbf040b7a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:47.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:47.637+0000 7fbf0b94b640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbf040d2e30 con 0x7fbf040d3790 2024-09-12T03:22:47.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:47.637+0000 7fbf0b94b640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fbf040b7f90 con 0x7fbf04162040 2024-09-12T03:22:47.641 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:47.637+0000 7fbf0b94b640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fbf04162420 con 0x7fbf0406dfd0 2024-09-12T03:22:47.641 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:47.637+0000 7fbf08ee6640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fbf040d3790 0x7fbf0406d8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:55446/0 (socket says 172.21.15.27:55446) 2024-09-12T03:22:47.641 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:47.637+0000 7fbf08ee6640 1 -- 172.21.15.27:0/3828263435 learned_addr learned my addr 172.21.15.27:0/3828263435 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:22:47.641 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:47.638+0000 7fbf096e7640 1 --2- 172.21.15.27:0/3828263435 >> v2:172.21.15.45:3300/0 conn(0x7fbf04162040 0x7fbf040d1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:47.641 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:47.638+0000 7fbf03fff640 1 -- 172.21.15.27:0/3828263435 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 560052057 0 0) 0x7fbf040d2e30 con 0x7fbf040d3790 2024-09-12T03:22:47.641 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:47.638+0000 7fbf09ee8640 1 --2- 172.21.15.27:0/3828263435 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fbf0406dfd0 0x7fbf040b7a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:47.641 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:47.638+0000 7fbf03fff640 1 -- 172.21.15.27:0/3828263435 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbeec003520 con 0x7fbf040d3790 2024-09-12T03:22:47.641 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:47.638+0000 7fbf03fff640 1 -- 172.21.15.27:0/3828263435 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1331239255 0 0) 0x7fbeec003520 con 0x7fbf040d3790 2024-09-12T03:22:47.641 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:47.638+0000 7fbf03fff640 1 -- 172.21.15.27:0/3828263435 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fbf0406dfd0 msgr2=0x7fbf040b7a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:47.641 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:47.638+0000 7fbf03fff640 1 --2- 172.21.15.27:0/3828263435 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fbf0406dfd0 0x7fbf040b7a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:47.641 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:47.638+0000 7fbf03fff640 1 -- 172.21.15.27:0/3828263435 >> v2:172.21.15.45:3300/0 conn(0x7fbf04162040 msgr2=0x7fbf040d1da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:47.641 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:47.638+0000 7fbf03fff640 1 --2- 172.21.15.27:0/3828263435 >> v2:172.21.15.45:3300/0 conn(0x7fbf04162040 0x7fbf040d1da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:47.641 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:47.638+0000 7fbf03fff640 1 -- 172.21.15.27:0/3828263435 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf040d30d0 con 0x7fbf040d3790 2024-09-12T03:22:47.642 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:47.638+0000 7fbf03fff640 1 -- 172.21.15.27:0/3828263435 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 847755213 0 0) 0x7fbef8001670 con 0x7fbf040d3790 2024-09-12T03:22:47.642 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:47.638+0000 7fbf03fff640 1 -- 172.21.15.27:0/3828263435 >> v1:172.21.15.27:6789/0 conn(0x7fbf040d3790 legacy=0x7fbf0406d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:47.642 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:47.638+0000 7fbf03fff640 1 --2- 172.21.15.27:0/3828263435 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbeec0039b0 0x7fbeec003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:47.642 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:47.638+0000 7fbf03fff640 1 --2- 172.21.15.27:0/3828263435 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbeec004800 0x7fbeec004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:47.642 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:47.638+0000 7fbf03fff640 1 --2- 172.21.15.27:0/3828263435 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fbf0406dfd0 0x7fbeec0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:47.642 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:47.638+0000 7fbf03fff640 1 -- 172.21.15.27:0/3828263435 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fbf040b7f90 con 0x7fbeec0039b0 2024-09-12T03:22:47.642 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:47.638+0000 7fbf03fff640 1 -- 172.21.15.27:0/3828263435 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fbf04162420 con 0x7fbeec004800 2024-09-12T03:22:47.642 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:47.638+0000 7fbf03fff640 1 -- 172.21.15.27:0/3828263435 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fbeec00b9c0 con 0x7fbf0406dfd0 2024-09-12T03:22:47.642 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:47.638+0000 7fbf096e7640 1 --2- 172.21.15.27:0/3828263435 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbeec0039b0 0x7fbeec003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:47.642 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:47.638+0000 7fbf08ee6640 1 --2- 172.21.15.27:0/3828263435 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbeec004800 0x7fbeec004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:47.642 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:47.638+0000 7fbf09ee8640 1 --2- 172.21.15.27:0/3828263435 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fbf0406dfd0 0x7fbeec0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:47.642 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:47.639+0000 7fbf096e7640 1 -- 172.21.15.27:0/3828263435 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fbf0406dfd0 msgr2=0x7fbeec0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:47.642 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:47.639+0000 7fbf096e7640 1 --2- 172.21.15.27:0/3828263435 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fbf0406dfd0 0x7fbeec0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:47.643 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:47.639+0000 7fbf096e7640 1 -- 172.21.15.27:0/3828263435 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbeec004800 msgr2=0x7fbeec004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:47.643 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:47.639+0000 7fbf096e7640 1 --2- 172.21.15.27:0/3828263435 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbeec004800 0x7fbeec004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:47.643 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:47.639+0000 7fbf096e7640 1 -- 172.21.15.27:0/3828263435 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbeec003520 con 0x7fbeec0039b0 2024-09-12T03:22:47.643 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:47.639+0000 7fbf096e7640 1 --2- 172.21.15.27:0/3828263435 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbeec0039b0 0x7fbeec003dd0 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7fbefc009ac0 tx=0x7fbeec005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcfb7ada509221b3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:47.643 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:47.639+0000 7fbf03fff640 1 -- 172.21.15.27:0/3828263435 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbefc036070 con 0x7fbeec0039b0 2024-09-12T03:22:47.643 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:47.639+0000 7fbf03fff640 1 -- 172.21.15.27:0/3828263435 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbefc031400 con 0x7fbeec0039b0 2024-09-12T03:22:47.643 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:47.639+0000 7fbf03fff640 1 -- 172.21.15.27:0/3828263435 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbefc009690 con 0x7fbeec0039b0 2024-09-12T03:22:47.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:47.640+0000 7fbf0b94b640 1 -- 172.21.15.27:0/3828263435 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbeec0039b0 msgr2=0x7fbeec003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:47.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:47.640+0000 7fbf0b94b640 1 --2- 172.21.15.27:0/3828263435 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbeec0039b0 0x7fbeec003dd0 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7fbefc009ac0 tx=0x7fbeec005830 comp rx=0 tx=0).stop 2024-09-12T03:22:47.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:47.640+0000 7fbf09ee8640 1 -- 172.21.15.27:0/3828263435 reap_dead start 2024-09-12T03:22:47.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:47.640+0000 7fbf0b94b640 1 -- 172.21.15.27:0/3828263435 shutdown_connections 2024-09-12T03:22:47.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:47.640+0000 7fbf0b94b640 1 -- 172.21.15.27:0/3828263435 >> 172.21.15.27:0/3828263435 conn(0x7fbf040b6700 msgr2=0x7fbf040b6b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:47.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:47.640+0000 7fbf0b94b640 1 -- 172.21.15.27:0/3828263435 shutdown_connections 2024-09-12T03:22:47.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:47.640+0000 7fbf0b94b640 1 -- 172.21.15.27:0/3828263435 wait complete. 2024-09-12T03:22:47.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:47.641+0000 7fbf0b94b640 1 Processor -- start 2024-09-12T03:22:47.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:47.641+0000 7fbf0b94b640 1 -- start start 2024-09-12T03:22:47.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:47.642+0000 7fbf0b94b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbeec0039b0 0x7fbf04150b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:47.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:47.642+0000 7fbf0b94b640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fbf040b5260 0x7fbf040b56e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:47.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:47.642+0000 7fbf0b94b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf040ada80 0x7fbf040adf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:47.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:47.642+0000 7fbf0b94b640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fbf040d4530 con 0x7fbeec0039b0 2024-09-12T03:22:47.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:47.642+0000 7fbf0b94b640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fbf040cee00 con 0x7fbf040ada80 2024-09-12T03:22:47.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:47.642+0000 7fbf0b94b640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fbf040d4150 con 0x7fbf040b5260 2024-09-12T03:22:47.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:47.642+0000 7fbf096e7640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbeec0039b0 0x7fbf04150b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:47.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:47.642+0000 7fbf096e7640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbeec0039b0 0x7fbf04150b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35560/0 (socket says 172.21.15.27:35560) 2024-09-12T03:22:47.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:47.642+0000 7fbf096e7640 1 -- 172.21.15.27:0/472237036 learned_addr learned my addr 172.21.15.27:0/472237036 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:22:47.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:47.642+0000 7fbf09ee8640 1 --2- 172.21.15.27:0/472237036 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf040ada80 0x7fbf040adf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:47.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:47.642+0000 7fbf08ee6640 1 --2- 172.21.15.27:0/472237036 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fbf040b5260 0x7fbf040b56e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:47.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:47.642+0000 7fbf096e7640 1 -- 172.21.15.27:0/472237036 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fbf040b5260 msgr2=0x7fbf040b56e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:47.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:47.642+0000 7fbf096e7640 1 --2- 172.21.15.27:0/472237036 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fbf040b5260 0x7fbf040b56e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:47.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:47.642+0000 7fbf096e7640 1 -- 172.21.15.27:0/472237036 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf040ada80 msgr2=0x7fbf040adf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:47.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:47.642+0000 7fbf096e7640 1 --2- 172.21.15.27:0/472237036 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf040ada80 0x7fbf040adf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:47.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:47.642+0000 7fbf096e7640 1 -- 172.21.15.27:0/472237036 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf040ab350 con 0x7fbeec0039b0 2024-09-12T03:22:47.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:47.642+0000 7fbf08ee6640 1 --2- 172.21.15.27:0/472237036 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fbf040b5260 0x7fbf040b56e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:22:47.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:47.643+0000 7fbf096e7640 1 --2- 172.21.15.27:0/472237036 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbeec0039b0 0x7fbf04150b60 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7fbefc009ac0 tx=0x7fbefc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:47.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:47.643+0000 7fbf01ffb640 1 -- 172.21.15.27:0/472237036 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbefc041020 con 0x7fbeec0039b0 2024-09-12T03:22:47.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:47.643+0000 7fbf01ffb640 1 -- 172.21.15.27:0/472237036 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbefc03c470 con 0x7fbeec0039b0 2024-09-12T03:22:47.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:47.643+0000 7fbf0b94b640 1 -- 172.21.15.27:0/472237036 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf040ab630 con 0x7fbeec0039b0 2024-09-12T03:22:47.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:47.643+0000 7fbf01ffb640 1 -- 172.21.15.27:0/472237036 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbefc036040 con 0x7fbeec0039b0 2024-09-12T03:22:47.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:47.643+0000 7fbf0b94b640 1 -- 172.21.15.27:0/472237036 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf040abbf0 con 0x7fbeec0039b0 2024-09-12T03:22:47.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:47.644+0000 7fbf0b94b640 1 -- 172.21.15.27:0/472237036 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf0406e6e0 con 0x7fbeec0039b0 2024-09-12T03:22:47.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:47.645+0000 7fbf01ffb640 1 -- 172.21.15.27:0/472237036 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 22) ==== 91059+0+0 (secure 0 0 0) 0x7fbefc044020 con 0x7fbeec0039b0 2024-09-12T03:22:47.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:47.645+0000 7fbf01ffb640 1 --2- 172.21.15.27:0/472237036 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fbef006c490 0x7fbef006e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:47.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:47.645+0000 7fbf01ffb640 1 -- 172.21.15.27:0/472237036 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 4913+0+0 (secure 0 0 0) 0x7fbefc0ab2d0 con 0x7fbeec0039b0 2024-09-12T03:22:47.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:47.645+0000 7fbf08ee6640 1 --2- 172.21.15.27:0/472237036 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fbef006c490 0x7fbef006e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:47.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:47.646+0000 7fbf08ee6640 1 --2- 172.21.15.27:0/472237036 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fbef006c490 0x7fbef006e950 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fbef8007fd0 tx=0x7fbef8009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:47.650 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:47.649+0000 7fbf01ffb640 1 -- 172.21.15.27:0/472237036 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fbefc079200 con 0x7fbeec0039b0 2024-09-12T03:22:47.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:47.826+0000 7fbf0b94b640 1 -- 172.21.15.27:0/472237036 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fbf040008d0 con 0x7fbeec0039b0 2024-09-12T03:22:48.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:48.826+0000 7fbf01ffb640 1 -- 172.21.15.27:0/472237036 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v135) ==== 138+0+0 (secure 0 0 0) 0x7fbefc07d500 con 0x7fbeec0039b0 2024-09-12T03:22:48.828 INFO:teuthology.orchestra.run.smithi027.stderr:pool 'cephfs_metadata' created 2024-09-12T03:22:48.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:48.829+0000 7fbf0b94b640 1 -- 172.21.15.27:0/472237036 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fbef006c490 msgr2=0x7fbef006e950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:48.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:48.829+0000 7fbf0b94b640 1 --2- 172.21.15.27:0/472237036 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fbef006c490 0x7fbef006e950 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fbef8007fd0 tx=0x7fbef8009040 comp rx=0 tx=0).stop 2024-09-12T03:22:48.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:48.829+0000 7fbf0b94b640 1 -- 172.21.15.27:0/472237036 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbeec0039b0 msgr2=0x7fbf04150b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:48.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:48.829+0000 7fbf0b94b640 1 --2- 172.21.15.27:0/472237036 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbeec0039b0 0x7fbf04150b60 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7fbefc009ac0 tx=0x7fbefc004730 comp rx=0 tx=0).stop 2024-09-12T03:22:48.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:48.829+0000 7fbf0b94b640 1 -- 172.21.15.27:0/472237036 shutdown_connections 2024-09-12T03:22:48.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:48.829+0000 7fbf0b94b640 1 --2- 172.21.15.27:0/472237036 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf040ada80 0x7fbf040adf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:48.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:48.829+0000 7fbf0b94b640 1 --2- 172.21.15.27:0/472237036 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fbf040b5260 0x7fbf040b56e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:48.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:48.829+0000 7fbf0b94b640 1 --2- 172.21.15.27:0/472237036 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fbef006c490 0x7fbef006e950 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:48.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:48.829+0000 7fbf0b94b640 1 --2- 172.21.15.27:0/472237036 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbeec0039b0 0x7fbf04150b60 unknown :-1 s=CLOSED pgs=1389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:48.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:48.829+0000 7fbf0b94b640 1 -- 172.21.15.27:0/472237036 >> 172.21.15.27:0/472237036 conn(0x7fbf040b6700 msgr2=0x7fbf040d2650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:48.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:48.830+0000 7fbf0b94b640 1 -- 172.21.15.27:0/472237036 shutdown_connections 2024-09-12T03:22:48.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:48.830+0000 7fbf0b94b640 1 -- 172.21.15.27:0/472237036 wait complete. 2024-09-12T03:22:48.848 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-12T03:22:48.964 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:48.963+0000 7f212c21e640 1 Processor -- start 2024-09-12T03:22:48.964 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:48.963+0000 7f212c21e640 1 -- start start 2024-09-12T03:22:48.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:48.964+0000 7f212c21e640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f21240d0b50 0x7f2124169310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:48.967 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:48.964+0000 7f212c21e640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f21240cf9f0 con 0x7f2124070d30 2024-09-12T03:22:48.967 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:48.964+0000 7f212c21e640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f21240cfc90 con 0x7f21240d31c0 2024-09-12T03:22:48.967 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:48.964+0000 7f212c21e640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f21240d1a40 con 0x7f21240d0b50 2024-09-12T03:22:48.967 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:48.964+0000 7f2129fba640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f2124070d30 0x7f212406cc40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:55454/0 (socket says 172.21.15.27:55454) 2024-09-12T03:22:48.967 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:48.964+0000 7f2129fba640 1 -- 172.21.15.27:0/1950340326 learned_addr learned my addr 172.21.15.27:0/1950340326 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:22:48.967 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:48.964+0000 7f212a7bb640 1 --2- 172.21.15.27:0/1950340326 >> v2:172.21.15.45:3300/0 conn(0x7f21240d0b50 0x7f2124169310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:48.967 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:48.965+0000 7f2128fb8640 1 -- 172.21.15.27:0/1950340326 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2468409564 0 0) 0x7f21240cf9f0 con 0x7f2124070d30 2024-09-12T03:22:48.967 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:48.965+0000 7f2128fb8640 1 -- 172.21.15.27:0/1950340326 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2110003520 con 0x7f2124070d30 2024-09-12T03:22:48.967 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:48.965+0000 7f2128fb8640 1 -- 172.21.15.27:0/1950340326 <== mon.1 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 104230554 0 0) 0x7f21240cfc90 con 0x7f21240d31c0 2024-09-12T03:22:48.967 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:48.965+0000 7f2128fb8640 1 -- 172.21.15.27:0/1950340326 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f21240cf9f0 con 0x7f21240d31c0 2024-09-12T03:22:48.967 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:48.965+0000 7f2128fb8640 1 -- 172.21.15.27:0/1950340326 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3986613897 0 0) 0x7f2110003520 con 0x7f2124070d30 2024-09-12T03:22:48.967 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:48.965+0000 7f212a7bb640 1 -- 172.21.15.27:0/1950340326 >> v1:172.21.15.45:6789/0 conn(0x7f21240d31c0 legacy=0x7f21240d0440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:48.968 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:48.965+0000 7f212a7bb640 1 -- 172.21.15.27:0/1950340326 >> v1:172.21.15.27:6789/0 conn(0x7f2124070d30 legacy=0x7f212406cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:48.968 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:48.965+0000 7f212a7bb640 1 -- 172.21.15.27:0/1950340326 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21240cff70 con 0x7f21240d0b50 2024-09-12T03:22:48.968 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:48.966+0000 7f212a7bb640 1 --2- 172.21.15.27:0/1950340326 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f21240d0b50 0x7f2124169310 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f211c009d90 tx=0x7f211c02a560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bda97d5e0e28688e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:48.968 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:48.966+0000 7f2128fb8640 1 -- 172.21.15.27:0/1950340326 <== mon.1 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2120001670 con 0x7f21240d0b50 2024-09-12T03:22:48.968 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:48.966+0000 7f2128fb8640 1 -- 172.21.15.27:0/1950340326 <== mon.1 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f211c007540 con 0x7f21240d0b50 2024-09-12T03:22:48.968 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:48.966+0000 7f2128fb8640 1 -- 172.21.15.27:0/1950340326 <== mon.1 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f211c0078f0 con 0x7f21240d0b50 2024-09-12T03:22:48.971 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:48.966+0000 7f212c21e640 1 -- 172.21.15.27:0/1950340326 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f21240d0b50 msgr2=0x7f2124169310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:48.971 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:48.966+0000 7f212c21e640 1 --2- 172.21.15.27:0/1950340326 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f21240d0b50 0x7f2124169310 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f211c009d90 tx=0x7f211c02a560 comp rx=0 tx=0).stop 2024-09-12T03:22:48.971 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:48.966+0000 7f212c21e640 1 -- 172.21.15.27:0/1950340326 shutdown_connections 2024-09-12T03:22:48.971 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:48.966+0000 7f212c21e640 1 --2- 172.21.15.27:0/1950340326 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f21240d0b50 0x7f2124169310 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:48.971 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:48.966+0000 7f212c21e640 1 -- 172.21.15.27:0/1950340326 >> 172.21.15.27:0/1950340326 conn(0x7f21240b7c10 msgr2=0x7f212409ba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:48.971 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:48.967+0000 7f212c21e640 1 -- 172.21.15.27:0/1950340326 shutdown_connections 2024-09-12T03:22:48.971 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:48.967+0000 7f212c21e640 1 -- 172.21.15.27:0/1950340326 wait complete. 2024-09-12T03:22:48.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:48.968+0000 7f212c21e640 1 Processor -- start 2024-09-12T03:22:48.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:48.968+0000 7f212c21e640 1 -- start start 2024-09-12T03:22:48.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:48.968+0000 7f212c21e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2124070d30 0x7f2124151a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:48.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:48.968+0000 7f212c21e640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f21240d0b50 0x7f21241fc250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:48.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:48.968+0000 7f212c21e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f21240d31c0 0x7f21241fc790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:48.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:48.968+0000 7f212c21e640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f212406eae0 con 0x7f21240d31c0 2024-09-12T03:22:48.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:48.968+0000 7f212c21e640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f212406e960 con 0x7f2124070d30 2024-09-12T03:22:48.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:48.968+0000 7f212c21e640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f2124169850 con 0x7f21240d0b50 2024-09-12T03:22:48.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:48.969+0000 7f212a7bb640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f21240d31c0 0x7f21241fc790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:48.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:48.969+0000 7f212a7bb640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f21240d31c0 0x7f21241fc790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35562/0 (socket says 172.21.15.27:35562) 2024-09-12T03:22:48.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:48.969+0000 7f212a7bb640 1 -- 172.21.15.27:0/1225344356 learned_addr learned my addr 172.21.15.27:0/1225344356 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:22:48.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:48.969+0000 7f21297b9640 1 --2- 172.21.15.27:0/1225344356 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f21240d0b50 0x7f21241fc250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:48.973 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:48.969+0000 7f2129fba640 1 --2- 172.21.15.27:0/1225344356 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2124070d30 0x7f2124151a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:48.973 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:48.969+0000 7f212a7bb640 1 -- 172.21.15.27:0/1225344356 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f21240d0b50 msgr2=0x7f21241fc250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:48.973 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:48.969+0000 7f212a7bb640 1 --2- 172.21.15.27:0/1225344356 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f21240d0b50 0x7f21241fc250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:48.973 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:48.969+0000 7f212a7bb640 1 -- 172.21.15.27:0/1225344356 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2124070d30 msgr2=0x7f2124151a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:48.973 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:48.969+0000 7f212a7bb640 1 --2- 172.21.15.27:0/1225344356 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2124070d30 0x7f2124151a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:48.973 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:48.969+0000 7f212a7bb640 1 -- 172.21.15.27:0/1225344356 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f212415d760 con 0x7f21240d31c0 2024-09-12T03:22:48.973 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:48.969+0000 7f212a7bb640 1 --2- 172.21.15.27:0/1225344356 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f21240d31c0 0x7f21241fc790 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7f211c02ab50 tx=0x7f211c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:48.973 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:48.969+0000 7f211a7fc640 1 -- 172.21.15.27:0/1225344356 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f211c009450 con 0x7f21240d31c0 2024-09-12T03:22:48.973 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:48.969+0000 7f211a7fc640 1 -- 172.21.15.27:0/1225344356 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f211c004240 con 0x7f21240d31c0 2024-09-12T03:22:48.973 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:48.969+0000 7f212c21e640 1 -- 172.21.15.27:0/1225344356 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21241523f0 con 0x7f21240d31c0 2024-09-12T03:22:48.973 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:48.969+0000 7f211a7fc640 1 -- 172.21.15.27:0/1225344356 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f211c0045f0 con 0x7f21240d31c0 2024-09-12T03:22:48.973 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:48.969+0000 7f212c21e640 1 -- 172.21.15.27:0/1225344356 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21241529b0 con 0x7f21240d31c0 2024-09-12T03:22:48.973 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:48.970+0000 7f212c21e640 1 -- 172.21.15.27:0/1225344356 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f212406cc40 con 0x7f21240d31c0 2024-09-12T03:22:48.974 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:48.971+0000 7f211a7fc640 1 -- 172.21.15.27:0/1225344356 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 22) ==== 91059+0+0 (secure 0 0 0) 0x7f211c039040 con 0x7f21240d31c0 2024-09-12T03:22:48.974 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:48.972+0000 7f211a7fc640 1 --2- 172.21.15.27:0/1225344356 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f210806c460 0x7f210806e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:48.974 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:48.972+0000 7f211a7fc640 1 -- 172.21.15.27:0/1225344356 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(135..135 src has 1..135) ==== 5283+0+0 (secure 0 0 0) 0x7f211c0aca30 con 0x7f21240d31c0 2024-09-12T03:22:48.975 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:48.972+0000 7f2129fba640 1 --2- 172.21.15.27:0/1225344356 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f210806c460 0x7f210806e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:48.975 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:48.972+0000 7f2129fba640 1 --2- 172.21.15.27:0/1225344356 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f210806c460 0x7f210806e920 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f21200017e0 tx=0x7f2120001610 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:48.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:48.976+0000 7f211a7fc640 1 -- 172.21.15.27:0/1225344356 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f211c040030 con 0x7f21240d31c0 2024-09-12T03:22:49.151 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:49.150+0000 7f212c21e640 1 -- 172.21.15.27:0/1225344356 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f21240b0040 con 0x7f21240d31c0 2024-09-12T03:22:49.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:49.834+0000 7f211a7fc640 1 -- 172.21.15.27:0/1225344356 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v136) ==== 170+0+0 (secure 0 0 0) 0x7f211c07a5d0 con 0x7f21240d31c0 2024-09-12T03:22:49.835 INFO:teuthology.orchestra.run.smithi027.stderr:pool 'cephfs_data' created 2024-09-12T03:22:49.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:49.838+0000 7f212c21e640 1 -- 172.21.15.27:0/1225344356 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f210806c460 msgr2=0x7f210806e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:49.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:49.838+0000 7f212c21e640 1 --2- 172.21.15.27:0/1225344356 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f210806c460 0x7f210806e920 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f21200017e0 tx=0x7f2120001610 comp rx=0 tx=0).stop 2024-09-12T03:22:49.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:49.838+0000 7f212c21e640 1 -- 172.21.15.27:0/1225344356 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f21240d31c0 msgr2=0x7f21241fc790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:49.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:49.838+0000 7f212c21e640 1 --2- 172.21.15.27:0/1225344356 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f21240d31c0 0x7f21241fc790 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7f211c02ab50 tx=0x7f211c004a40 comp rx=0 tx=0).stop 2024-09-12T03:22:49.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:49.838+0000 7f212c21e640 1 -- 172.21.15.27:0/1225344356 shutdown_connections 2024-09-12T03:22:49.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:49.838+0000 7f212c21e640 1 --2- 172.21.15.27:0/1225344356 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f210806c460 0x7f210806e920 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:49.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:49.838+0000 7f212c21e640 1 --2- 172.21.15.27:0/1225344356 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f21240d31c0 0x7f21241fc790 unknown :-1 s=CLOSED pgs=1391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:49.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:49.838+0000 7f212c21e640 1 --2- 172.21.15.27:0/1225344356 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f21240d0b50 0x7f21241fc250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:49.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:49.838+0000 7f212c21e640 1 --2- 172.21.15.27:0/1225344356 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2124070d30 0x7f2124151a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:49.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:49.838+0000 7f212c21e640 1 -- 172.21.15.27:0/1225344356 >> 172.21.15.27:0/1225344356 conn(0x7f21240b7c10 msgr2=0x7f21240b0dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:49.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:49.838+0000 7f212c21e640 1 -- 172.21.15.27:0/1225344356 shutdown_connections 2024-09-12T03:22:49.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:49.838+0000 7f212c21e640 1 -- 172.21.15.27:0/1225344356 wait complete. 2024-09-12T03:22:49.865 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-12T03:22:49.976 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:49.975+0000 7f3feefe4640 1 Processor -- start 2024-09-12T03:22:49.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:49.975+0000 7f3feefe4640 1 -- start start 2024-09-12T03:22:49.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:49.976+0000 7f3feefe4640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f3fe806dfd0 0x7f3fe80d1da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:49.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:49.976+0000 7f3feefe4640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f3fe80d3790 0x7f3fe80b7a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:49.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:49.976+0000 7f3feefe4640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3fe80d2e30 con 0x7f3fe8162040 2024-09-12T03:22:49.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:49.976+0000 7f3feefe4640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f3fe80b7f90 con 0x7f3fe80d3790 2024-09-12T03:22:49.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:49.976+0000 7f3feefe4640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f3fe8162420 con 0x7f3fe806dfd0 2024-09-12T03:22:49.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:49.976+0000 7f3fecd80640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f3fe8162040 0x7f3fe806d8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:52314/0 (socket says 172.21.15.27:52314) 2024-09-12T03:22:49.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:49.976+0000 7f3fecd80640 1 -- 172.21.15.27:0/15938660 learned_addr learned my addr 172.21.15.27:0/15938660 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:22:49.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:49.976+0000 7f3fed581640 1 --2- 172.21.15.27:0/15938660 >> v2:172.21.15.27:3300/0 conn(0x7f3fe80d3790 0x7f3fe80b7a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:49.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:49.977+0000 7f3fe7fff640 1 --2- 172.21.15.27:0/15938660 >> v2:172.21.15.45:3300/0 conn(0x7f3fe806dfd0 0x7f3fe80d1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:49.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:49.977+0000 7f3fe77fe640 1 -- 172.21.15.27:0/15938660 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 728338589 0 0) 0x7f3fe80d2e30 con 0x7f3fe8162040 2024-09-12T03:22:49.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:49.977+0000 7f3fe77fe640 1 -- 172.21.15.27:0/15938660 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3fd0003520 con 0x7f3fe8162040 2024-09-12T03:22:49.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:49.977+0000 7f3fed581640 1 -- 172.21.15.27:0/15938660 >> v2:172.21.15.45:3300/0 conn(0x7f3fe806dfd0 msgr2=0x7f3fe80d1da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:49.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:49.977+0000 7f3fed581640 1 --2- 172.21.15.27:0/15938660 >> v2:172.21.15.45:3300/0 conn(0x7f3fe806dfd0 0x7f3fe80d1da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:49.980 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:49.977+0000 7f3fed581640 1 -- 172.21.15.27:0/15938660 >> v1:172.21.15.27:6789/0 conn(0x7f3fe8162040 legacy=0x7f3fe806d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:49.980 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:49.977+0000 7f3fed581640 1 -- 172.21.15.27:0/15938660 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fe80d30d0 con 0x7f3fe80d3790 2024-09-12T03:22:49.980 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:49.978+0000 7f3fed581640 1 --2- 172.21.15.27:0/15938660 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3fe80d3790 0x7f3fe80b7a50 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7f3fe806e630 tx=0x7f3fd802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2809d73a9c3064c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:49.980 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:49.978+0000 7f3fe77fe640 1 -- 172.21.15.27:0/15938660 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fd8037070 con 0x7f3fe80d3790 2024-09-12T03:22:49.980 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:49.978+0000 7f3fe77fe640 1 -- 172.21.15.27:0/15938660 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3fd8032440 con 0x7f3fe80d3790 2024-09-12T03:22:49.980 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:49.978+0000 7f3fe77fe640 1 -- 172.21.15.27:0/15938660 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fd8032810 con 0x7f3fe80d3790 2024-09-12T03:22:49.984 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:49.978+0000 7f3feefe4640 1 -- 172.21.15.27:0/15938660 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3fe80d3790 msgr2=0x7f3fe80b7a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:49.984 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:49.978+0000 7f3feefe4640 1 --2- 172.21.15.27:0/15938660 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3fe80d3790 0x7f3fe80b7a50 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7f3fe806e630 tx=0x7f3fd802a1a0 comp rx=0 tx=0).stop 2024-09-12T03:22:49.984 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:49.978+0000 7f3feefe4640 1 -- 172.21.15.27:0/15938660 shutdown_connections 2024-09-12T03:22:49.984 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:49.978+0000 7f3feefe4640 1 --2- 172.21.15.27:0/15938660 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3fe80d3790 0x7f3fe80b7a50 unknown :-1 s=CLOSED pgs=1393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:49.984 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:49.978+0000 7f3feefe4640 1 --2- 172.21.15.27:0/15938660 >> v2:172.21.15.45:3300/0 conn(0x7f3fe806dfd0 0x7f3fe80d1da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:49.984 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:49.978+0000 7f3feefe4640 1 -- 172.21.15.27:0/15938660 >> 172.21.15.27:0/15938660 conn(0x7f3fe80b6700 msgr2=0x7f3fe80b6b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:49.984 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:49.978+0000 7f3feefe4640 1 -- 172.21.15.27:0/15938660 shutdown_connections 2024-09-12T03:22:49.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:49.978+0000 7f3feefe4640 1 -- 172.21.15.27:0/15938660 wait complete. 2024-09-12T03:22:49.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:49.980+0000 7f3feefe4640 1 Processor -- start 2024-09-12T03:22:49.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:49.980+0000 7f3feefe4640 1 -- start start 2024-09-12T03:22:49.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:49.980+0000 7f3feefe4640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3fe806dfd0 0x7f3fe815af10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:49.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:49.980+0000 7f3feefe4640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3fe80d3790 0x7f3fe814ea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:49.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:49.980+0000 7f3feefe4640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f3fe8162040 0x7f3fe80b9190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:49.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:49.981+0000 7f3feefe4640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f3fe80d4530 con 0x7f3fe806dfd0 2024-09-12T03:22:49.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:49.981+0000 7f3feefe4640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f3fe80cee00 con 0x7f3fe80d3790 2024-09-12T03:22:49.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:49.981+0000 7f3feefe4640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f3fe80d4150 con 0x7f3fe8162040 2024-09-12T03:22:49.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:49.981+0000 7f3fecd80640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3fe806dfd0 0x7f3fe815af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:49.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:49.981+0000 7f3fecd80640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3fe806dfd0 0x7f3fe815af10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56168/0 (socket says 172.21.15.27:56168) 2024-09-12T03:22:49.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:49.981+0000 7f3fecd80640 1 -- 172.21.15.27:0/1602414885 learned_addr learned my addr 172.21.15.27:0/1602414885 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:22:49.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:49.981+0000 7f3fe7fff640 1 --2- 172.21.15.27:0/1602414885 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3fe80d3790 0x7f3fe814ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:49.986 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:49.981+0000 7f3fecd80640 1 -- 172.21.15.27:0/1602414885 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f3fe8162040 msgr2=0x7f3fe80b9190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:49.986 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:49.981+0000 7f3fed581640 1 --2- 172.21.15.27:0/1602414885 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f3fe8162040 0x7f3fe80b9190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:49.986 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:49.981+0000 7f3fecd80640 1 --2- 172.21.15.27:0/1602414885 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f3fe8162040 0x7f3fe80b9190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:49.986 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:49.981+0000 7f3fecd80640 1 -- 172.21.15.27:0/1602414885 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3fe80d3790 msgr2=0x7f3fe814ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:49.986 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:49.981+0000 7f3fecd80640 1 --2- 172.21.15.27:0/1602414885 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3fe80d3790 0x7f3fe814ea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:49.986 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:49.981+0000 7f3fecd80640 1 -- 172.21.15.27:0/1602414885 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fe814ef70 con 0x7f3fe806dfd0 2024-09-12T03:22:49.986 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:49.981+0000 7f3fe7fff640 1 --2- 172.21.15.27:0/1602414885 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3fe80d3790 0x7f3fe814ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:22:49.986 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:49.981+0000 7f3fed581640 1 --2- 172.21.15.27:0/1602414885 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f3fe8162040 0x7f3fe80b9190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:22:49.986 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:49.981+0000 7f3fecd80640 1 --2- 172.21.15.27:0/1602414885 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3fe806dfd0 0x7f3fe815af10 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7f3fe000caa0 tx=0x7f3fe000cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:49.986 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:49.981+0000 7f3fe57fa640 1 -- 172.21.15.27:0/1602414885 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fe0017070 con 0x7f3fe806dfd0 2024-09-12T03:22:49.986 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:49.981+0000 7f3fe57fa640 1 -- 172.21.15.27:0/1602414885 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3fe000ac00 con 0x7f3fe806dfd0 2024-09-12T03:22:49.986 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:49.981+0000 7f3feefe4640 1 -- 172.21.15.27:0/1602414885 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3fe815b6a0 con 0x7f3fe806dfd0 2024-09-12T03:22:49.986 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:49.981+0000 7f3fe57fa640 1 -- 172.21.15.27:0/1602414885 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fe0012400 con 0x7f3fe806dfd0 2024-09-12T03:22:49.986 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:49.981+0000 7f3feefe4640 1 -- 172.21.15.27:0/1602414885 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3fe80cd0f0 con 0x7f3fe806dfd0 2024-09-12T03:22:49.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:49.982+0000 7f3feefe4640 1 -- 172.21.15.27:0/1602414885 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3fe806e6e0 con 0x7f3fe806dfd0 2024-09-12T03:22:49.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:49.983+0000 7f3fe57fa640 1 -- 172.21.15.27:0/1602414885 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 22) ==== 91059+0+0 (secure 0 0 0) 0x7f3fe001a070 con 0x7f3fe806dfd0 2024-09-12T03:22:49.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:49.983+0000 7f3fe57fa640 1 --2- 172.21.15.27:0/1602414885 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f3fd406c460 0x7f3fd406e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:49.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:49.984+0000 7f3fe57fa640 1 -- 172.21.15.27:0/1602414885 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(136..136 src has 1..136) ==== 5665+0+0 (secure 0 0 0) 0x7f3fe008b660 con 0x7f3fe806dfd0 2024-09-12T03:22:49.988 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:49.984+0000 7f3fe7fff640 1 --2- 172.21.15.27:0/1602414885 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f3fd406c460 0x7f3fd406e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:49.988 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:49.984+0000 7f3fe7fff640 1 --2- 172.21.15.27:0/1602414885 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f3fd406c460 0x7f3fd406e920 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f3fe80b9e60 tx=0x7f3fdc008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:49.989 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:49.988+0000 7f3fe57fa640 1 -- 172.21.15.27:0/1602414885 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f3fe0059800 con 0x7f3fe806dfd0 2024-09-12T03:22:50.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:50.198+0000 7f3feefe4640 1 -- 172.21.15.27:0/1602414885 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f3fe81581d0 con 0x7f3fe806dfd0 2024-09-12T03:22:50.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:50.836+0000 7f3fe57fa640 1 -- 172.21.15.27:0/1602414885 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 22 and data pool 23 v80) ==== 174+0+0 (secure 0 0 0) 0x7f3fe0059100 con 0x7f3fe806dfd0 2024-09-12T03:22:50.837 INFO:teuthology.orchestra.run.smithi027.stderr:new fs with metadata pool 22 and data pool 23 2024-09-12T03:22:50.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:50.841+0000 7f3feefe4640 1 -- 172.21.15.27:0/1602414885 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f3fd406c460 msgr2=0x7f3fd406e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:50.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:50.841+0000 7f3feefe4640 1 --2- 172.21.15.27:0/1602414885 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f3fd406c460 0x7f3fd406e920 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f3fe80b9e60 tx=0x7f3fdc008040 comp rx=0 tx=0).stop 2024-09-12T03:22:50.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:50.841+0000 7f3feefe4640 1 -- 172.21.15.27:0/1602414885 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3fe806dfd0 msgr2=0x7f3fe815af10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:50.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:50.841+0000 7f3feefe4640 1 --2- 172.21.15.27:0/1602414885 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3fe806dfd0 0x7f3fe815af10 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7f3fe000caa0 tx=0x7f3fe000cf60 comp rx=0 tx=0).stop 2024-09-12T03:22:50.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:50.842+0000 7f3feefe4640 1 -- 172.21.15.27:0/1602414885 shutdown_connections 2024-09-12T03:22:50.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:50.842+0000 7f3feefe4640 1 --2- 172.21.15.27:0/1602414885 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f3fe8162040 0x7f3fe80b9190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:50.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:50.842+0000 7f3feefe4640 1 --2- 172.21.15.27:0/1602414885 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3fe80d3790 0x7f3fe814ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:50.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:50.842+0000 7f3feefe4640 1 --2- 172.21.15.27:0/1602414885 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f3fd406c460 0x7f3fd406e920 secure :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f3fe80b9e60 tx=0x7f3fdc008040 comp rx=0 tx=0).stop 2024-09-12T03:22:50.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:50.842+0000 7f3feefe4640 1 --2- 172.21.15.27:0/1602414885 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3fe806dfd0 0x7f3fe815af10 unknown :-1 s=CLOSED pgs=1394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:50.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:50.842+0000 7f3feefe4640 1 -- 172.21.15.27:0/1602414885 >> 172.21.15.27:0/1602414885 conn(0x7f3fe80b6700 msgr2=0x7f3fe80d4d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:50.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:50.842+0000 7f3feefe4640 1 -- 172.21.15.27:0/1602414885 shutdown_connections 2024-09-12T03:22:50.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:50.842+0000 7f3feefe4640 1 -- 172.21.15.27:0/1602414885 wait complete. 2024-09-12T03:22:50.873 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-12T03:22:50.980 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:50.979+0000 7f5f267d8640 1 Processor -- start 2024-09-12T03:22:50.980 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:50.979+0000 7f5f267d8640 1 -- start start 2024-09-12T03:22:50.983 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:50.980+0000 7f5f267d8640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f5f2006f360 0x7f5f200cef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:50.983 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:50.980+0000 7f5f267d8640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5f2006cc40 con 0x7f5f200b1ca0 2024-09-12T03:22:50.983 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:50.980+0000 7f5f267d8640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5f200d1ce0 con 0x7f5f200cf490 2024-09-12T03:22:50.983 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:50.980+0000 7f5f267d8640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f5f200d22f0 con 0x7f5f2006f360 2024-09-12T03:22:50.983 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:50.980+0000 7f5f1ffff640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f5f200b1ca0 0x7f5f2015f830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:52320/0 (socket says 172.21.15.27:52320) 2024-09-12T03:22:50.983 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:50.980+0000 7f5f1ffff640 1 -- 172.21.15.27:0/3064277607 learned_addr learned my addr 172.21.15.27:0/3064277607 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:22:50.983 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:50.980+0000 7f5f1f7fe640 1 --2- 172.21.15.27:0/3064277607 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f5f2006f360 0x7f5f200cef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:50.984 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:50.980+0000 7f5f1effd640 1 -- 172.21.15.27:0/3064277607 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3594431016 0 0) 0x7f5f2006cc40 con 0x7f5f200b1ca0 2024-09-12T03:22:50.984 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:50.980+0000 7f5f1effd640 1 -- 172.21.15.27:0/3064277607 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5f08003520 con 0x7f5f200b1ca0 2024-09-12T03:22:50.984 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:50.981+0000 7f5f1effd640 1 -- 172.21.15.27:0/3064277607 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2600034787 0 0) 0x7f5f08003520 con 0x7f5f200b1ca0 2024-09-12T03:22:50.984 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:50.981+0000 7f5f1effd640 1 -- 172.21.15.27:0/3064277607 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f5f2006f360 msgr2=0x7f5f200cef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:50.984 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:50.981+0000 7f5f1effd640 1 --2- 172.21.15.27:0/3064277607 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f5f2006f360 0x7f5f200cef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:50.984 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:50.981+0000 7f5f1effd640 1 -- 172.21.15.27:0/3064277607 >> v1:172.21.15.45:6789/0 conn(0x7f5f200cf490 legacy=0x7f5f200d2780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:50.984 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:50.981+0000 7f5f1effd640 1 -- 172.21.15.27:0/3064277607 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f2006cee0 con 0x7f5f200b1ca0 2024-09-12T03:22:50.984 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:50.981+0000 7f5f1effd640 1 -- 172.21.15.27:0/3064277607 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 847755213 0 0) 0x7f5f10001670 con 0x7f5f200b1ca0 2024-09-12T03:22:50.984 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:50.981+0000 7f5f1effd640 1 -- 172.21.15.27:0/3064277607 >> v1:172.21.15.27:6789/0 conn(0x7f5f200b1ca0 legacy=0x7f5f2015f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:50.984 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:50.981+0000 7f5f1effd640 1 --2- 172.21.15.27:0/3064277607 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5f080039e0 0x7f5f08003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:50.984 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:50.981+0000 7f5f1effd640 1 --2- 172.21.15.27:0/3064277607 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f5f2006f360 0x7f5f08004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:50.984 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:50.981+0000 7f5f1effd640 1 --2- 172.21.15.27:0/3064277607 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5f08004d40 0x7f5f080051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:50.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:50.981+0000 7f5f1effd640 1 -- 172.21.15.27:0/3064277607 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f5f200d22f0 con 0x7f5f080039e0 2024-09-12T03:22:50.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:50.981+0000 7f5f1effd640 1 -- 172.21.15.27:0/3064277607 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f5f0800baa0 con 0x7f5f08004d40 2024-09-12T03:22:50.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:50.981+0000 7f5f1effd640 1 -- 172.21.15.27:0/3064277607 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f5f0800bca0 con 0x7f5f2006f360 2024-09-12T03:22:50.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:50.981+0000 7f5f1ffff640 1 --2- 172.21.15.27:0/3064277607 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5f080039e0 0x7f5f08003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:50.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:50.981+0000 7f5f1f7fe640 1 --2- 172.21.15.27:0/3064277607 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f5f2006f360 0x7f5f08004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:50.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:50.981+0000 7f5f24d75640 1 --2- 172.21.15.27:0/3064277607 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5f08004d40 0x7f5f080051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:50.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:50.981+0000 7f5f1ffff640 1 -- 172.21.15.27:0/3064277607 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f5f2006f360 msgr2=0x7f5f08004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:50.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:50.981+0000 7f5f1ffff640 1 --2- 172.21.15.27:0/3064277607 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f5f2006f360 0x7f5f08004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:50.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:50.981+0000 7f5f1ffff640 1 -- 172.21.15.27:0/3064277607 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5f08004d40 msgr2=0x7f5f080051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:50.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:50.981+0000 7f5f1ffff640 1 --2- 172.21.15.27:0/3064277607 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5f08004d40 0x7f5f080051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:50.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:50.981+0000 7f5f1ffff640 1 -- 172.21.15.27:0/3064277607 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f08003520 con 0x7f5f080039e0 2024-09-12T03:22:50.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:50.982+0000 7f5f1ffff640 1 --2- 172.21.15.27:0/3064277607 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5f080039e0 0x7f5f08003e00 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7f5f1000c210 tx=0x7f5f08005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d6521bebf81e49c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:50.986 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:50.982+0000 7f5f1effd640 1 -- 172.21.15.27:0/3064277607 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f10007d10 con 0x7f5f080039e0 2024-09-12T03:22:50.986 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:50.982+0000 7f5f1effd640 1 -- 172.21.15.27:0/3064277607 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f100058d0 con 0x7f5f080039e0 2024-09-12T03:22:50.986 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:50.982+0000 7f5f1effd640 1 -- 172.21.15.27:0/3064277607 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f10005ce0 con 0x7f5f080039e0 2024-09-12T03:22:50.989 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:50.983+0000 7f5f267d8640 1 -- 172.21.15.27:0/3064277607 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5f080039e0 msgr2=0x7f5f08003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:50.989 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:50.983+0000 7f5f267d8640 1 --2- 172.21.15.27:0/3064277607 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5f080039e0 0x7f5f08003e00 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7f5f1000c210 tx=0x7f5f08005830 comp rx=0 tx=0).stop 2024-09-12T03:22:50.989 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:50.983+0000 7f5f24d75640 1 -- 172.21.15.27:0/3064277607 reap_dead start 2024-09-12T03:22:50.989 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:50.983+0000 7f5f267d8640 1 -- 172.21.15.27:0/3064277607 shutdown_connections 2024-09-12T03:22:50.989 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:50.983+0000 7f5f267d8640 1 -- 172.21.15.27:0/3064277607 >> 172.21.15.27:0/3064277607 conn(0x7f5f20073800 msgr2=0x7f5f2009ba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:50.989 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:50.983+0000 7f5f267d8640 1 -- 172.21.15.27:0/3064277607 shutdown_connections 2024-09-12T03:22:50.989 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:50.983+0000 7f5f267d8640 1 -- 172.21.15.27:0/3064277607 wait complete. 2024-09-12T03:22:50.989 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:50.984+0000 7f5f267d8640 1 Processor -- start 2024-09-12T03:22:50.989 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:50.984+0000 7f5f267d8640 1 -- start start 2024-09-12T03:22:50.989 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:50.985+0000 7f5f267d8640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5f080039e0 0x7f5f20155dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:50.990 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:50.985+0000 7f5f267d8640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5f201e6fd0 0x7f5f200b6ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:50.990 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:50.985+0000 7f5f267d8640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f5f2015b240 0x7f5f2015b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:50.990 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:50.985+0000 7f5f267d8640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f5f2006d340 con 0x7f5f201e6fd0 2024-09-12T03:22:50.990 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:50.985+0000 7f5f267d8640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f5f2006d1c0 con 0x7f5f080039e0 2024-09-12T03:22:50.990 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:50.985+0000 7f5f267d8640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f5f200cfc40 con 0x7f5f2015b240 2024-09-12T03:22:50.990 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:50.985+0000 7f5f1f7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5f201e6fd0 0x7f5f200b6ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:50.990 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:50.985+0000 7f5f1f7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5f201e6fd0 0x7f5f200b6ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56182/0 (socket says 172.21.15.27:56182) 2024-09-12T03:22:50.990 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:50.985+0000 7f5f1f7fe640 1 -- 172.21.15.27:0/3188848210 learned_addr learned my addr 172.21.15.27:0/3188848210 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:22:50.990 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:50.985+0000 7f5f24d75640 1 --2- 172.21.15.27:0/3188848210 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f5f2015b240 0x7f5f2015b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:50.990 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:50.985+0000 7f5f1f7fe640 1 -- 172.21.15.27:0/3188848210 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f5f2015b240 msgr2=0x7f5f2015b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:50.990 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:50.985+0000 7f5f1ffff640 1 --2- 172.21.15.27:0/3188848210 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5f080039e0 0x7f5f20155dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:50.990 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:50.985+0000 7f5f1f7fe640 1 --2- 172.21.15.27:0/3188848210 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f5f2015b240 0x7f5f2015b6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:50.990 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:50.985+0000 7f5f1f7fe640 1 -- 172.21.15.27:0/3188848210 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5f080039e0 msgr2=0x7f5f20155dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:50.991 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:50.985+0000 7f5f1f7fe640 1 --2- 172.21.15.27:0/3188848210 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5f080039e0 0x7f5f20155dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:50.991 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:50.985+0000 7f5f1f7fe640 1 -- 172.21.15.27:0/3188848210 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f200b2e00 con 0x7f5f201e6fd0 2024-09-12T03:22:50.991 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:50.985+0000 7f5f1f7fe640 1 --2- 172.21.15.27:0/3188848210 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5f201e6fd0 0x7f5f200b6ff0 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7f5f14011940 tx=0x7f5f14011e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:50.991 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:50.986+0000 7f5f1cff9640 1 -- 172.21.15.27:0/3188848210 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f1400fa90 con 0x7f5f201e6fd0 2024-09-12T03:22:50.991 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:50.986+0000 7f5f1cff9640 1 -- 172.21.15.27:0/3188848210 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f1400fc30 con 0x7f5f201e6fd0 2024-09-12T03:22:50.991 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:50.986+0000 7f5f267d8640 1 -- 172.21.15.27:0/3188848210 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f20156310 con 0x7f5f201e6fd0 2024-09-12T03:22:50.991 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:50.986+0000 7f5f1cff9640 1 -- 172.21.15.27:0/3188848210 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f14014440 con 0x7f5f201e6fd0 2024-09-12T03:22:50.991 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:50.986+0000 7f5f267d8640 1 -- 172.21.15.27:0/3188848210 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f201e7820 con 0x7f5f201e6fd0 2024-09-12T03:22:50.991 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:50.987+0000 7f5f267d8640 1 -- 172.21.15.27:0/3188848210 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f2006f7a0 con 0x7f5f201e6fd0 2024-09-12T03:22:50.991 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:50.987+0000 7f5f1cff9640 1 -- 172.21.15.27:0/3188848210 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 22) ==== 91059+0+0 (secure 0 0 0) 0x7f5f14012040 con 0x7f5f201e6fd0 2024-09-12T03:22:50.991 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:50.988+0000 7f5f1cff9640 1 --2- 172.21.15.27:0/3188848210 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f5f0c06c460 0x7f5f0c06e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:50.991 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:50.988+0000 7f5f1cff9640 1 -- 172.21.15.27:0/3188848210 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 5765+0+0 (secure 0 0 0) 0x7f5f1408c030 con 0x7f5f201e6fd0 2024-09-12T03:22:50.991 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:50.988+0000 7f5f1ffff640 1 --2- 172.21.15.27:0/3188848210 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f5f0c06c460 0x7f5f0c06e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:50.992 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:50.988+0000 7f5f1ffff640 1 --2- 172.21.15.27:0/3188848210 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f5f0c06c460 0x7f5f0c06e920 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f5f10033000 tx=0x7f5f1002e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:50.993 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:50.992+0000 7f5f1cff9640 1 -- 172.21.15.27:0/3188848210 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f5f14019030 con 0x7f5f201e6fd0 2024-09-12T03:22:51.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.170+0000 7f5f267d8640 1 -- 172.21.15.27:0/3188848210 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5f20070af0 con 0x7f5f201e6fd0 2024-09-12T03:22:51.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.171+0000 7f5f1cff9640 1 -- 172.21.15.27:0/3188848210 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v137) ==== 74+0+16785 (secure 0 0 0) 0x7f5f14059fd0 con 0x7f5f201e6fd0 2024-09-12T03:22:51.172 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:22:51.173 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":137,"fsid":"d2df504e-2647-4385-a5db-9fdf69f487a0","created":"2024-09-12T03:16:07.778937+0000","modified":"2024-09-12T03:22:50.200037+0000","last_up_change":"2024-09-12T03:16:19.314778+0000","last_in_change":"2024-09-12T03:16:09.893086+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T03:16:20.097646+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2024-09-12T03:22:47.859008+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"137","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2024-09-12T03:22:49.183109+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"137","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1851116c-1fb3-4dc9-9772-f90e2ef4c355","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6817","nonce":3002321488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6819","nonce":3002321488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6823","nonce":3002321488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6821","nonce":3002321488}]},"public_addr":"172.21.15.27:6817/3002321488","cluster_addr":"172.21.15.27:6819/3002321488","heartbeat_back_addr":"172.21.15.27:6823/3002321488","heartbeat_front_addr":"172.21.15.27:6821/3002321488","state":["exists","up"]},{"osd":1,"uuid":"f38ae17d-73eb-4204-b99d-975fa806fb87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6809","nonce":1613102886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6811","nonce":1613102886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6815","nonce":1613102886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6813","nonce":1613102886}]},"public_addr":"172.21.15.27:6809/1613102886","cluster_addr":"172.21.15.27:6811/1613102886","heartbeat_back_addr":"172.21.15.27:6815/1613102886","heartbeat_front_addr":"172.21.15.27:6813/1613102886","state":["exists","up"]},{"osd":2,"uuid":"fb12dcc8-bdd0-42b8-b46d-8b10106224bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6825","nonce":3534723355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6827","nonce":3534723355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6831","nonce":3534723355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6829","nonce":3534723355}]},"public_addr":"172.21.15.27:6825/3534723355","cluster_addr":"172.21.15.27:6827/3534723355","heartbeat_back_addr":"172.21.15.27:6831/3534723355","heartbeat_front_addr":"172.21.15.27:6829/3534723355","state":["exists","up"]},{"osd":3,"uuid":"d74d3f49-9c24-4b9f-98e2-3ecd7b5ef3b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6801","nonce":80661598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6803","nonce":80661598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6807","nonce":80661598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6805","nonce":80661598}]},"public_addr":"172.21.15.27:6801/80661598","cluster_addr":"172.21.15.27:6803/80661598","heartbeat_back_addr":"172.21.15.27:6807/80661598","heartbeat_front_addr":"172.21.15.27:6805/80661598","state":["exists","up"]},{"osd":4,"uuid":"2bf123f2-1b0d-41c4-aa3b-805bab267e59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6817","nonce":3943189827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6819","nonce":3943189827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6823","nonce":3943189827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6821","nonce":3943189827}]},"public_addr":"172.21.15.45:6817/3943189827","cluster_addr":"172.21.15.45:6819/3943189827","heartbeat_back_addr":"172.21.15.45:6823/3943189827","heartbeat_front_addr":"172.21.15.45:6821/3943189827","state":["exists","up"]},{"osd":5,"uuid":"c49ba5da-91c6-4a86-a97b-4aba89c253ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6809","nonce":3620760330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6811","nonce":3620760330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6815","nonce":3620760330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6813","nonce":3620760330}]},"public_addr":"172.21.15.45:6809/3620760330","cluster_addr":"172.21.15.45:6811/3620760330","heartbeat_back_addr":"172.21.15.45:6815/3620760330","heartbeat_front_addr":"172.21.15.45:6813/3620760330","state":["exists","up"]},{"osd":6,"uuid":"b0123475-89f7-4a34-9e33-d797b99dd9dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6801","nonce":1989083510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6803","nonce":1989083510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6807","nonce":1989083510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6805","nonce":1989083510}]},"public_addr":"172.21.15.45:6801/1989083510","cluster_addr":"172.21.15.45:6803/1989083510","heartbeat_back_addr":"172.21.15.45:6807/1989083510","heartbeat_front_addr":"172.21.15.45:6805/1989083510","state":["exists","up"]},{"osd":7,"uuid":"4dad773d-b7d8-46fc-8a10-a972aad27b29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6825","nonce":1513300294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6827","nonce":1513300294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6831","nonce":1513300294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6829","nonce":1513300294}]},"public_addr":"172.21.15.45:6825/1513300294","cluster_addr":"172.21.15.45:6827/1513300294","heartbeat_back_addr":"172.21.15.45:6831/1513300294","heartbeat_front_addr":"172.21.15.45:6829/1513300294","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.881855+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:17.133252+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.735339+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.500742+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.477571+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:17.167667+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.470291+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.873743+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]},{"pool":19,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T03:22:51.175 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.173+0000 7f5f267d8640 1 -- 172.21.15.27:0/3188848210 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f5f0c06c460 msgr2=0x7f5f0c06e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:51.175 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.173+0000 7f5f267d8640 1 --2- 172.21.15.27:0/3188848210 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f5f0c06c460 0x7f5f0c06e920 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f5f10033000 tx=0x7f5f1002e000 comp rx=0 tx=0).stop 2024-09-12T03:22:51.175 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.173+0000 7f5f267d8640 1 -- 172.21.15.27:0/3188848210 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5f201e6fd0 msgr2=0x7f5f200b6ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:51.175 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.173+0000 7f5f267d8640 1 --2- 172.21.15.27:0/3188848210 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5f201e6fd0 0x7f5f200b6ff0 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7f5f14011940 tx=0x7f5f14011e00 comp rx=0 tx=0).stop 2024-09-12T03:22:51.175 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.173+0000 7f5f267d8640 1 -- 172.21.15.27:0/3188848210 shutdown_connections 2024-09-12T03:22:51.175 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.173+0000 7f5f267d8640 1 --2- 172.21.15.27:0/3188848210 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f5f2015b240 0x7f5f2015b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:51.175 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.173+0000 7f5f267d8640 1 --2- 172.21.15.27:0/3188848210 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f5f0c06c460 0x7f5f0c06e920 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:51.175 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.173+0000 7f5f267d8640 1 --2- 172.21.15.27:0/3188848210 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5f201e6fd0 0x7f5f200b6ff0 unknown :-1 s=CLOSED pgs=1397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:51.175 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.173+0000 7f5f267d8640 1 --2- 172.21.15.27:0/3188848210 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5f080039e0 0x7f5f20155dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:51.175 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.173+0000 7f5f267d8640 1 -- 172.21.15.27:0/3188848210 >> 172.21.15.27:0/3188848210 conn(0x7f5f20073800 msgr2=0x7f5f200af540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:51.175 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.173+0000 7f5f267d8640 1 -- 172.21.15.27:0/3188848210 shutdown_connections 2024-09-12T03:22:51.175 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.173+0000 7f5f267d8640 1 -- 172.21.15.27:0/3188848210 wait complete. 2024-09-12T03:22:51.186 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-12T03:22:51.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.287+0000 7fcf12847640 1 Processor -- start 2024-09-12T03:22:51.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.287+0000 7fcf12847640 1 -- start start 2024-09-12T03:22:51.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.288+0000 7fcf12847640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fcf0c0b1ca0 0x7fcf0c15f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:51.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.288+0000 7fcf12847640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fcf0c0cf660 0x7fcf0c0d2780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:51.290 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.288+0000 7fcf12847640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcf0c15fd70 con 0x7fcf0c06f360 2024-09-12T03:22:51.290 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.288+0000 7fcf12847640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7fcf0c0d22f0 con 0x7fcf0c0cf660 2024-09-12T03:22:51.290 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.288+0000 7fcf12847640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fcf0c06cc40 con 0x7fcf0c0b1ca0 2024-09-12T03:22:51.290 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.288+0000 7fcf10de4640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fcf0c0cf660 0x7fcf0c0d2780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:51.290 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.288+0000 7fcf0b7fe640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fcf0c06f360 0x7fcf0c0cef50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:52328/0 (socket says 172.21.15.27:52328) 2024-09-12T03:22:51.290 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.288+0000 7fcf0b7fe640 1 -- 172.21.15.27:0/514624263 learned_addr learned my addr 172.21.15.27:0/514624263 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:22:51.290 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.288+0000 7fcf0bfff640 1 --2- 172.21.15.27:0/514624263 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fcf0c0b1ca0 0x7fcf0c15f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:51.290 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.289+0000 7fcf10de4640 1 -- 172.21.15.27:0/514624263 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fcf0c0b1ca0 msgr2=0x7fcf0c15f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:51.290 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.289+0000 7fcf10de4640 1 --2- 172.21.15.27:0/514624263 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fcf0c0b1ca0 0x7fcf0c15f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:51.290 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.289+0000 7fcf10de4640 1 -- 172.21.15.27:0/514624263 >> v1:172.21.15.27:6789/0 conn(0x7fcf0c06f360 legacy=0x7fcf0c0cef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:51.290 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.289+0000 7fcf10de4640 1 -- 172.21.15.27:0/514624263 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf0c0d1ce0 con 0x7fcf0c0cf660 2024-09-12T03:22:51.291 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.289+0000 7fcf10de4640 1 --2- 172.21.15.27:0/514624263 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcf0c0cf660 0x7fcf0c0d2780 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7fcf04009dc0 tx=0x7fcf0402a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58ccb1e8ea370a09 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:51.291 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.290+0000 7fcf0affd640 1 -- 172.21.15.27:0/514624263 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf0402ae60 con 0x7fcf0c0cf660 2024-09-12T03:22:51.291 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.290+0000 7fcf0affd640 1 -- 172.21.15.27:0/514624263 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf04007740 con 0x7fcf0c0cf660 2024-09-12T03:22:51.291 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.290+0000 7fcf0affd640 1 -- 172.21.15.27:0/514624263 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf04007b10 con 0x7fcf0c0cf660 2024-09-12T03:22:51.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.290+0000 7fcf12847640 1 -- 172.21.15.27:0/514624263 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcf0c0cf660 msgr2=0x7fcf0c0d2780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:51.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.290+0000 7fcf12847640 1 --2- 172.21.15.27:0/514624263 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcf0c0cf660 0x7fcf0c0d2780 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7fcf04009dc0 tx=0x7fcf0402a680 comp rx=0 tx=0).stop 2024-09-12T03:22:51.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.290+0000 7fcf12847640 1 -- 172.21.15.27:0/514624263 shutdown_connections 2024-09-12T03:22:51.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.290+0000 7fcf12847640 1 --2- 172.21.15.27:0/514624263 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcf0c0cf660 0x7fcf0c0d2780 unknown :-1 s=CLOSED pgs=1399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:51.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.290+0000 7fcf12847640 1 --2- 172.21.15.27:0/514624263 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fcf0c0b1ca0 0x7fcf0c15f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:51.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.290+0000 7fcf12847640 1 -- 172.21.15.27:0/514624263 >> 172.21.15.27:0/514624263 conn(0x7fcf0c073800 msgr2=0x7fcf0c09ba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:51.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.290+0000 7fcf12847640 1 -- 172.21.15.27:0/514624263 shutdown_connections 2024-09-12T03:22:51.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.290+0000 7fcf12847640 1 -- 172.21.15.27:0/514624263 wait complete. 2024-09-12T03:22:51.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.291+0000 7fcf12847640 1 Processor -- start 2024-09-12T03:22:51.293 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.292+0000 7fcf12847640 1 -- start start 2024-09-12T03:22:51.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.292+0000 7fcf12847640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcf0c06f360 0x7fcf0c0d06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:51.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.292+0000 7fcf12847640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fcf0c0b1ca0 0x7fcf0c152b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:51.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.292+0000 7fcf12847640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcf0c0cf660 0x7fcf0c153070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:51.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.292+0000 7fcf12847640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fcf0c06d1c0 con 0x7fcf0c0cf660 2024-09-12T03:22:51.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.292+0000 7fcf12847640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fcf0c0d1f80 con 0x7fcf0c06f360 2024-09-12T03:22:51.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.292+0000 7fcf12847640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fcf0c06d340 con 0x7fcf0c0b1ca0 2024-09-12T03:22:51.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.293+0000 7fcf10de4640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcf0c0cf660 0x7fcf0c153070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:51.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.293+0000 7fcf10de4640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcf0c0cf660 0x7fcf0c153070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56198/0 (socket says 172.21.15.27:56198) 2024-09-12T03:22:51.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.293+0000 7fcf10de4640 1 -- 172.21.15.27:0/3167585648 learned_addr learned my addr 172.21.15.27:0/3167585648 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:22:51.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.293+0000 7fcf0bfff640 1 --2- 172.21.15.27:0/3167585648 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcf0c06f360 0x7fcf0c0d06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:51.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.293+0000 7fcf0b7fe640 1 --2- 172.21.15.27:0/3167585648 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fcf0c0b1ca0 0x7fcf0c152b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:51.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.293+0000 7fcf10de4640 1 -- 172.21.15.27:0/3167585648 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fcf0c0b1ca0 msgr2=0x7fcf0c152b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:51.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.293+0000 7fcf10de4640 1 --2- 172.21.15.27:0/3167585648 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fcf0c0b1ca0 0x7fcf0c152b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:51.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.293+0000 7fcf10de4640 1 -- 172.21.15.27:0/3167585648 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcf0c06f360 msgr2=0x7fcf0c0d06b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:51.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.293+0000 7fcf10de4640 1 --2- 172.21.15.27:0/3167585648 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcf0c06f360 0x7fcf0c0d06b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:51.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.293+0000 7fcf10de4640 1 -- 172.21.15.27:0/3167585648 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf0c0b78a0 con 0x7fcf0c0cf660 2024-09-12T03:22:51.296 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.293+0000 7fcf10de4640 1 --2- 172.21.15.27:0/3167585648 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcf0c0cf660 0x7fcf0c153070 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7fcf0402ac70 tx=0x7fcf04004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:51.296 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.293+0000 7fcf08ff9640 1 -- 172.21.15.27:0/3167585648 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf0403f070 con 0x7fcf0c0cf660 2024-09-12T03:22:51.296 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.293+0000 7fcf08ff9640 1 -- 172.21.15.27:0/3167585648 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf04007e20 con 0x7fcf0c0cf660 2024-09-12T03:22:51.296 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.293+0000 7fcf12847640 1 -- 172.21.15.27:0/3167585648 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf0c0b54d0 con 0x7fcf0c0cf660 2024-09-12T03:22:51.296 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.293+0000 7fcf08ff9640 1 -- 172.21.15.27:0/3167585648 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf04009450 con 0x7fcf0c0cf660 2024-09-12T03:22:51.296 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.293+0000 7fcf12847640 1 -- 172.21.15.27:0/3167585648 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf0c0b5a90 con 0x7fcf0c0cf660 2024-09-12T03:22:51.297 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.294+0000 7fcf12847640 1 -- 172.21.15.27:0/3167585648 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf0c0709d0 con 0x7fcf0c0cf660 2024-09-12T03:22:51.297 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.295+0000 7fcf08ff9640 1 -- 172.21.15.27:0/3167585648 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 22) ==== 91059+0+0 (secure 0 0 0) 0x7fcf0403b410 con 0x7fcf0c0cf660 2024-09-12T03:22:51.297 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.295+0000 7fcf08ff9640 1 --2- 172.21.15.27:0/3167585648 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fcef806c460 0x7fcef806e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:51.297 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.295+0000 7fcf08ff9640 1 -- 172.21.15.27:0/3167585648 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 5765+0+0 (secure 0 0 0) 0x7fcf040acf20 con 0x7fcf0c0cf660 2024-09-12T03:22:51.297 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.295+0000 7fcf0bfff640 1 --2- 172.21.15.27:0/3167585648 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fcef806c460 0x7fcef806e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:51.297 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.296+0000 7fcf0bfff640 1 --2- 172.21.15.27:0/3167585648 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fcef806c460 0x7fcef806e920 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fcefc009b40 tx=0x7fcefc009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:51.300 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.300+0000 7fcf08ff9640 1 -- 172.21.15.27:0/3167585648 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fcf0c0709d0 con 0x7fcf0c0cf660 2024-09-12T03:22:51.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.477+0000 7fcf12847640 1 -- 172.21.15.27:0/3167585648 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcf0c06f9d0 con 0x7fcf0c0cf660 2024-09-12T03:22:51.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.478+0000 7fcf08ff9640 1 -- 172.21.15.27:0/3167585648 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v137) ==== 74+0+16785 (secure 0 0 0) 0x7fcf0407a960 con 0x7fcf0c0cf660 2024-09-12T03:22:51.479 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:22:51.479 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":137,"fsid":"d2df504e-2647-4385-a5db-9fdf69f487a0","created":"2024-09-12T03:16:07.778937+0000","modified":"2024-09-12T03:22:50.200037+0000","last_up_change":"2024-09-12T03:16:19.314778+0000","last_in_change":"2024-09-12T03:16:09.893086+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T03:16:20.097646+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2024-09-12T03:22:47.859008+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"137","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2024-09-12T03:22:49.183109+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"137","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1851116c-1fb3-4dc9-9772-f90e2ef4c355","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6817","nonce":3002321488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6819","nonce":3002321488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6823","nonce":3002321488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6821","nonce":3002321488}]},"public_addr":"172.21.15.27:6817/3002321488","cluster_addr":"172.21.15.27:6819/3002321488","heartbeat_back_addr":"172.21.15.27:6823/3002321488","heartbeat_front_addr":"172.21.15.27:6821/3002321488","state":["exists","up"]},{"osd":1,"uuid":"f38ae17d-73eb-4204-b99d-975fa806fb87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6809","nonce":1613102886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6811","nonce":1613102886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6815","nonce":1613102886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6813","nonce":1613102886}]},"public_addr":"172.21.15.27:6809/1613102886","cluster_addr":"172.21.15.27:6811/1613102886","heartbeat_back_addr":"172.21.15.27:6815/1613102886","heartbeat_front_addr":"172.21.15.27:6813/1613102886","state":["exists","up"]},{"osd":2,"uuid":"fb12dcc8-bdd0-42b8-b46d-8b10106224bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6825","nonce":3534723355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6827","nonce":3534723355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6831","nonce":3534723355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6829","nonce":3534723355}]},"public_addr":"172.21.15.27:6825/3534723355","cluster_addr":"172.21.15.27:6827/3534723355","heartbeat_back_addr":"172.21.15.27:6831/3534723355","heartbeat_front_addr":"172.21.15.27:6829/3534723355","state":["exists","up"]},{"osd":3,"uuid":"d74d3f49-9c24-4b9f-98e2-3ecd7b5ef3b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6801","nonce":80661598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6803","nonce":80661598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6807","nonce":80661598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6805","nonce":80661598}]},"public_addr":"172.21.15.27:6801/80661598","cluster_addr":"172.21.15.27:6803/80661598","heartbeat_back_addr":"172.21.15.27:6807/80661598","heartbeat_front_addr":"172.21.15.27:6805/80661598","state":["exists","up"]},{"osd":4,"uuid":"2bf123f2-1b0d-41c4-aa3b-805bab267e59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6817","nonce":3943189827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6819","nonce":3943189827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6823","nonce":3943189827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6821","nonce":3943189827}]},"public_addr":"172.21.15.45:6817/3943189827","cluster_addr":"172.21.15.45:6819/3943189827","heartbeat_back_addr":"172.21.15.45:6823/3943189827","heartbeat_front_addr":"172.21.15.45:6821/3943189827","state":["exists","up"]},{"osd":5,"uuid":"c49ba5da-91c6-4a86-a97b-4aba89c253ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6809","nonce":3620760330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6811","nonce":3620760330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6815","nonce":3620760330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6813","nonce":3620760330}]},"public_addr":"172.21.15.45:6809/3620760330","cluster_addr":"172.21.15.45:6811/3620760330","heartbeat_back_addr":"172.21.15.45:6815/3620760330","heartbeat_front_addr":"172.21.15.45:6813/3620760330","state":["exists","up"]},{"osd":6,"uuid":"b0123475-89f7-4a34-9e33-d797b99dd9dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6801","nonce":1989083510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6803","nonce":1989083510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6807","nonce":1989083510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6805","nonce":1989083510}]},"public_addr":"172.21.15.45:6801/1989083510","cluster_addr":"172.21.15.45:6803/1989083510","heartbeat_back_addr":"172.21.15.45:6807/1989083510","heartbeat_front_addr":"172.21.15.45:6805/1989083510","state":["exists","up"]},{"osd":7,"uuid":"4dad773d-b7d8-46fc-8a10-a972aad27b29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6825","nonce":1513300294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6827","nonce":1513300294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6831","nonce":1513300294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6829","nonce":1513300294}]},"public_addr":"172.21.15.45:6825/1513300294","cluster_addr":"172.21.15.45:6827/1513300294","heartbeat_back_addr":"172.21.15.45:6831/1513300294","heartbeat_front_addr":"172.21.15.45:6829/1513300294","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.881855+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:17.133252+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.735339+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.500742+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.477571+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:17.167667+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.470291+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.873743+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]},{"pool":19,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T03:22:51.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.480+0000 7fcf12847640 1 -- 172.21.15.27:0/3167585648 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fcef806c460 msgr2=0x7fcef806e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:51.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.480+0000 7fcf12847640 1 --2- 172.21.15.27:0/3167585648 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fcef806c460 0x7fcef806e920 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fcefc009b40 tx=0x7fcefc009190 comp rx=0 tx=0).stop 2024-09-12T03:22:51.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.480+0000 7fcf12847640 1 -- 172.21.15.27:0/3167585648 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcf0c0cf660 msgr2=0x7fcf0c153070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:51.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.480+0000 7fcf12847640 1 --2- 172.21.15.27:0/3167585648 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcf0c0cf660 0x7fcf0c153070 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7fcf0402ac70 tx=0x7fcf04004a40 comp rx=0 tx=0).stop 2024-09-12T03:22:51.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.480+0000 7fcf12847640 1 -- 172.21.15.27:0/3167585648 shutdown_connections 2024-09-12T03:22:51.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.480+0000 7fcf12847640 1 --2- 172.21.15.27:0/3167585648 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fcef806c460 0x7fcef806e920 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:51.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.480+0000 7fcf12847640 1 --2- 172.21.15.27:0/3167585648 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcf0c0cf660 0x7fcf0c153070 unknown :-1 s=CLOSED pgs=1400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:51.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.480+0000 7fcf12847640 1 --2- 172.21.15.27:0/3167585648 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fcf0c0b1ca0 0x7fcf0c152b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:51.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.480+0000 7fcf12847640 1 --2- 172.21.15.27:0/3167585648 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcf0c06f360 0x7fcf0c0d06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:51.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.480+0000 7fcf12847640 1 -- 172.21.15.27:0/3167585648 >> 172.21.15.27:0/3167585648 conn(0x7fcf0c073800 msgr2=0x7fcf0c160150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:51.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.480+0000 7fcf12847640 1 -- 172.21.15.27:0/3167585648 shutdown_connections 2024-09-12T03:22:51.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.480+0000 7fcf12847640 1 -- 172.21.15.27:0/3167585648 wait complete. 2024-09-12T03:22:51.492 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-12T03:22:51.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.597+0000 7fb098d59640 1 Processor -- start 2024-09-12T03:22:51.603 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.597+0000 7fb098d59640 1 -- start start 2024-09-12T03:22:51.603 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.598+0000 7fb098d59640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb09415e530 0x7fb0940aff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:51.603 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.598+0000 7fb098d59640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fb09415e910 0x7fb0940b4710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:51.603 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.598+0000 7fb098d59640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb09406dc20 con 0x7fb0940aeb80 2024-09-12T03:22:51.603 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.598+0000 7fb098d59640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fb0940b07e0 con 0x7fb09415e910 2024-09-12T03:22:51.603 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.598+0000 7fb098d59640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fb0940b4c50 con 0x7fb09415e530 2024-09-12T03:22:51.603 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.598+0000 7fb08bfff640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fb09415e910 0x7fb0940b4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:51.603 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.598+0000 7fb09259c640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb09415e530 0x7fb0940aff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:51.603 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.599+0000 7fb092d9d640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fb0940aeb80 0x7fb0940722f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.27:49126/0 (socket says 172.21.15.27:49126) 2024-09-12T03:22:51.603 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.599+0000 7fb092d9d640 1 -- 172.21.15.27:0/2298301671 learned_addr learned my addr 172.21.15.27:0/2298301671 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:22:51.603 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.599+0000 7fb09259c640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb09415e530 0x7fb0940aff20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3301/0 says I am v2:172.21.15.27:59926/0 (socket says 172.21.15.27:59926) 2024-09-12T03:22:51.604 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.599+0000 7fb08bfff640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fb09415e910 0x7fb0940b4710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.27:36446/0 (socket says 172.21.15.27:36446) 2024-09-12T03:22:51.604 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.599+0000 7fb09259c640 1 -- 172.21.15.27:0/2298301671 >> v2:172.21.15.45:3300/0 conn(0x7fb09415e910 msgr2=0x7fb0940b4710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:51.604 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.599+0000 7fb09259c640 1 --2- 172.21.15.27:0/2298301671 >> v2:172.21.15.45:3300/0 conn(0x7fb09415e910 0x7fb0940b4710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:51.604 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.599+0000 7fb09259c640 1 -- 172.21.15.27:0/2298301671 >> v1:172.21.15.45:6789/0 conn(0x7fb0940aeb80 legacy=0x7fb0940722f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:51.604 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.599+0000 7fb09259c640 1 -- 172.21.15.27:0/2298301671 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0940b0500 con 0x7fb09415e530 2024-09-12T03:22:51.604 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.600+0000 7fb09259c640 1 --2- 172.21.15.27:0/2298301671 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb09415e530 0x7fb0940aff20 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fb0840097a0 tx=0x7fb08402a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1ab2128556f40820 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:51.604 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.600+0000 7fb091d9b640 1 -- 172.21.15.27:0/2298301671 <== mon.2 v2:172.21.15.45:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb084037070 con 0x7fb09415e530 2024-09-12T03:22:51.604 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.600+0000 7fb091d9b640 1 -- 172.21.15.27:0/2298301671 <== mon.2 v2:172.21.15.45:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb084032440 con 0x7fb09415e530 2024-09-12T03:22:51.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.600+0000 7fb091d9b640 1 -- 172.21.15.27:0/2298301671 <== mon.2 v2:172.21.15.45:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb084032810 con 0x7fb09415e530 2024-09-12T03:22:51.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.600+0000 7fb098d59640 1 -- 172.21.15.27:0/2298301671 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb09415e530 msgr2=0x7fb0940aff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:51.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.600+0000 7fb098d59640 1 --2- 172.21.15.27:0/2298301671 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb09415e530 0x7fb0940aff20 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fb0840097a0 tx=0x7fb08402a180 comp rx=0 tx=0).stop 2024-09-12T03:22:51.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.600+0000 7fb098d59640 1 -- 172.21.15.27:0/2298301671 shutdown_connections 2024-09-12T03:22:51.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.600+0000 7fb098d59640 1 --2- 172.21.15.27:0/2298301671 >> v2:172.21.15.45:3300/0 conn(0x7fb09415e910 0x7fb0940b4710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:51.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.600+0000 7fb098d59640 1 --2- 172.21.15.27:0/2298301671 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb09415e530 0x7fb0940aff20 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:51.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.600+0000 7fb098d59640 1 -- 172.21.15.27:0/2298301671 >> 172.21.15.27:0/2298301671 conn(0x7fb09406bb30 msgr2=0x7fb09406bf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:51.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.600+0000 7fb098d59640 1 -- 172.21.15.27:0/2298301671 shutdown_connections 2024-09-12T03:22:51.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.600+0000 7fb098d59640 1 -- 172.21.15.27:0/2298301671 wait complete. 2024-09-12T03:22:51.612 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.602+0000 7fb098d59640 1 Processor -- start 2024-09-12T03:22:51.612 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.602+0000 7fb098d59640 1 -- start start 2024-09-12T03:22:51.612 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.602+0000 7fb098d59640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb0940aeb80 0x7fb0941603d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:51.612 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.602+0000 7fb098d59640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb09415e530 0x7fb0940c7680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:51.613 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.603+0000 7fb098d59640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb09415e910 0x7fb0940c5de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:51.613 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.603+0000 7fb098d59640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fb094071ff0 con 0x7fb09415e530 2024-09-12T03:22:51.613 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.603+0000 7fb098d59640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb09406e1a0 con 0x7fb09415e910 2024-09-12T03:22:51.613 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.603+0000 7fb098d59640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fb094072a00 con 0x7fb0940aeb80 2024-09-12T03:22:51.613 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.603+0000 7fb08bfff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb09415e530 0x7fb0940c7680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:51.613 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.603+0000 7fb08bfff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb09415e530 0x7fb0940c7680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56206/0 (socket says 172.21.15.27:56206) 2024-09-12T03:22:51.613 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.603+0000 7fb08bfff640 1 -- 172.21.15.27:0/656442179 learned_addr learned my addr 172.21.15.27:0/656442179 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:22:51.614 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.603+0000 7fb092d9d640 1 --2- 172.21.15.27:0/656442179 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb09415e910 0x7fb0940c5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:51.614 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.603+0000 7fb09259c640 1 --2- 172.21.15.27:0/656442179 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb0940aeb80 0x7fb0941603d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:51.614 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.603+0000 7fb08bfff640 1 -- 172.21.15.27:0/656442179 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb0940aeb80 msgr2=0x7fb0941603d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:51.614 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.603+0000 7fb08bfff640 1 --2- 172.21.15.27:0/656442179 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb0940aeb80 0x7fb0941603d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:51.614 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.603+0000 7fb08bfff640 1 -- 172.21.15.27:0/656442179 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb09415e910 msgr2=0x7fb0940c5de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:51.615 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.603+0000 7fb08bfff640 1 --2- 172.21.15.27:0/656442179 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb09415e910 0x7fb0940c5de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:51.615 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.603+0000 7fb08bfff640 1 -- 172.21.15.27:0/656442179 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb094162f80 con 0x7fb09415e530 2024-09-12T03:22:51.615 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.603+0000 7fb08bfff640 1 --2- 172.21.15.27:0/656442179 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb09415e530 0x7fb0940c7680 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7fb07c00e840 tx=0x7fb07c00ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:51.615 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.603+0000 7fb08affd640 1 -- 172.21.15.27:0/656442179 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb07c017070 con 0x7fb09415e530 2024-09-12T03:22:51.615 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.603+0000 7fb08affd640 1 -- 172.21.15.27:0/656442179 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb07c00c8e0 con 0x7fb09415e530 2024-09-12T03:22:51.615 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.603+0000 7fb098d59640 1 -- 172.21.15.27:0/656442179 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb09415f8f0 con 0x7fb09415e530 2024-09-12T03:22:51.615 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.603+0000 7fb08affd640 1 -- 172.21.15.27:0/656442179 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb07c00ccb0 con 0x7fb09415e530 2024-09-12T03:22:51.615 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.604+0000 7fb098d59640 1 -- 172.21.15.27:0/656442179 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb09415feb0 con 0x7fb09415e530 2024-09-12T03:22:51.615 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.604+0000 7fb098d59640 1 -- 172.21.15.27:0/656442179 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0940aff20 con 0x7fb09415e530 2024-09-12T03:22:51.615 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.605+0000 7fb08affd640 1 -- 172.21.15.27:0/656442179 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 22) ==== 91059+0+0 (secure 0 0 0) 0x7fb07c01a070 con 0x7fb09415e530 2024-09-12T03:22:51.615 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.606+0000 7fb08affd640 1 --2- 172.21.15.27:0/656442179 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fb07406c460 0x7fb07406e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:51.615 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.606+0000 7fb09259c640 1 --2- 172.21.15.27:0/656442179 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fb07406c460 0x7fb07406e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:51.615 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.606+0000 7fb08affd640 1 -- 172.21.15.27:0/656442179 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 5765+0+0 (secure 0 0 0) 0x7fb07c08b600 con 0x7fb09415e530 2024-09-12T03:22:51.616 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.606+0000 7fb09259c640 1 --2- 172.21.15.27:0/656442179 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fb07406c460 0x7fb07406e920 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fb084034000 tx=0x7fb08402e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:51.616 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.610+0000 7fb08affd640 1 -- 172.21.15.27:0/656442179 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fb07c059170 con 0x7fb09415e530 2024-09-12T03:22:51.826 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.824+0000 7fb098d59640 1 -- 172.21.15.27:0/656442179 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fb094071d80 con 0x7fb09415e530 2024-09-12T03:22:51.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.851+0000 7fb08affd640 1 -- 172.21.15.27:0/656442179 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v81) ==== 118+0+0 (secure 0 0 0) 0x7fb07c05d470 con 0x7fb09415e530 2024-09-12T03:22:51.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.854+0000 7fb098d59640 1 -- 172.21.15.27:0/656442179 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fb07406c460 msgr2=0x7fb07406e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:51.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.854+0000 7fb098d59640 1 --2- 172.21.15.27:0/656442179 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fb07406c460 0x7fb07406e920 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fb084034000 tx=0x7fb08402e000 comp rx=0 tx=0).stop 2024-09-12T03:22:51.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.854+0000 7fb098d59640 1 -- 172.21.15.27:0/656442179 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb09415e530 msgr2=0x7fb0940c7680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:51.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.854+0000 7fb098d59640 1 --2- 172.21.15.27:0/656442179 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb09415e530 0x7fb0940c7680 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7fb07c00e840 tx=0x7fb07c00ed00 comp rx=0 tx=0).stop 2024-09-12T03:22:51.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.854+0000 7fb098d59640 1 -- 172.21.15.27:0/656442179 shutdown_connections 2024-09-12T03:22:51.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.854+0000 7fb098d59640 1 --2- 172.21.15.27:0/656442179 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb09415e910 0x7fb0940c5de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:51.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.854+0000 7fb098d59640 1 --2- 172.21.15.27:0/656442179 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fb07406c460 0x7fb07406e920 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:51.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.854+0000 7fb098d59640 1 --2- 172.21.15.27:0/656442179 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb09415e530 0x7fb0940c7680 unknown :-1 s=CLOSED pgs=1401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:51.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.854+0000 7fb098d59640 1 --2- 172.21.15.27:0/656442179 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb0940aeb80 0x7fb0941603d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:51.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.854+0000 7fb098d59640 1 -- 172.21.15.27:0/656442179 >> 172.21.15.27:0/656442179 conn(0x7fb09406bb30 msgr2=0x7fb0940b1970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:51.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.855+0000 7fb098d59640 1 -- 172.21.15.27:0/656442179 shutdown_connections 2024-09-12T03:22:51.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.855+0000 7fb098d59640 1 -- 172.21.15.27:0/656442179 wait complete. 2024-09-12T03:22:51.867 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-12T03:22:51.867 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-12T03:22:51.975 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.974+0000 7f71d0a7b640 1 Processor -- start 2024-09-12T03:22:51.975 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.974+0000 7f71d0a7b640 1 -- start start 2024-09-12T03:22:51.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.975+0000 7f71d0a7b640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f71cc0b3d10 0x7f71cc0b66c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:51.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.975+0000 7f71d0a7b640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f71cc15ed70 con 0x7f71cc0b6230 2024-09-12T03:22:51.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.975+0000 7f71d0a7b640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f71cc0b7d10 con 0x7f71cc15e990 2024-09-12T03:22:51.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.975+0000 7f71d0a7b640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f71cc0d2a30 con 0x7f71cc0b3d10 2024-09-12T03:22:51.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.975+0000 7f71c9d9b640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f71cc0b6230 0x7f71cc0b3600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:52342/0 (socket says 172.21.15.27:52342) 2024-09-12T03:22:51.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.975+0000 7f71c9d9b640 1 -- 172.21.15.27:0/3409818662 learned_addr learned my addr 172.21.15.27:0/3409818662 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:22:51.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.976+0000 7f71cad9d640 1 --2- 172.21.15.27:0/3409818662 >> v2:172.21.15.45:3300/0 conn(0x7f71cc0b3d10 0x7f71cc0b66c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:51.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.976+0000 7f71c959a640 1 -- 172.21.15.27:0/3409818662 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2671719744 0 0) 0x7f71cc15ed70 con 0x7f71cc0b6230 2024-09-12T03:22:51.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.976+0000 7f71c959a640 1 -- 172.21.15.27:0/3409818662 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f71b8003520 con 0x7f71cc0b6230 2024-09-12T03:22:51.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.976+0000 7f71c959a640 1 -- 172.21.15.27:0/3409818662 <== mon.1 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3866693998 0 0) 0x7f71cc0b7d10 con 0x7f71cc15e990 2024-09-12T03:22:51.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.976+0000 7f71c959a640 1 -- 172.21.15.27:0/3409818662 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f71cc15ed70 con 0x7f71cc15e990 2024-09-12T03:22:51.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.976+0000 7f71cad9d640 1 -- 172.21.15.27:0/3409818662 >> v1:172.21.15.45:6789/0 conn(0x7f71cc15e990 legacy=0x7f71cc0b7600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:51.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.976+0000 7f71cad9d640 1 -- 172.21.15.27:0/3409818662 >> v1:172.21.15.27:6789/0 conn(0x7f71cc0b6230 legacy=0x7f71cc0b3600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:51.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.976+0000 7f71c959a640 1 -- 172.21.15.27:0/3409818662 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3087880907 0 0) 0x7f71b8003520 con 0x7f71cc0b6230 2024-09-12T03:22:51.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.976+0000 7f71cad9d640 1 -- 172.21.15.27:0/3409818662 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71cc0d2d10 con 0x7f71cc0b3d10 2024-09-12T03:22:51.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.977+0000 7f71cad9d640 1 --2- 172.21.15.27:0/3409818662 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f71cc0b3d10 0x7f71cc0b66c0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f71bc009d90 tx=0x7f71bc02a680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fd889399ad064eb7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:51.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.977+0000 7f71c959a640 1 -- 172.21.15.27:0/3409818662 <== mon.1 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71b4001670 con 0x7f71cc0b3d10 2024-09-12T03:22:51.980 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.977+0000 7f71c959a640 1 -- 172.21.15.27:0/3409818662 <== mon.1 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71bc007540 con 0x7f71cc0b3d10 2024-09-12T03:22:51.980 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.977+0000 7f71c959a640 1 -- 172.21.15.27:0/3409818662 <== mon.1 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71bc007910 con 0x7f71cc0b3d10 2024-09-12T03:22:51.980 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.978+0000 7f71d0a7b640 1 -- 172.21.15.27:0/3409818662 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f71cc0b3d10 msgr2=0x7f71cc0b66c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:51.980 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.978+0000 7f71d0a7b640 1 --2- 172.21.15.27:0/3409818662 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f71cc0b3d10 0x7f71cc0b66c0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f71bc009d90 tx=0x7f71bc02a680 comp rx=0 tx=0).stop 2024-09-12T03:22:51.980 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.978+0000 7f71d0a7b640 1 -- 172.21.15.27:0/3409818662 shutdown_connections 2024-09-12T03:22:51.980 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.978+0000 7f71d0a7b640 1 --2- 172.21.15.27:0/3409818662 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f71cc0b3d10 0x7f71cc0b66c0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:51.980 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.978+0000 7f71d0a7b640 1 -- 172.21.15.27:0/3409818662 >> 172.21.15.27:0/3409818662 conn(0x7f71cc0d1230 msgr2=0x7f71cc0d1640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:51.980 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.978+0000 7f71d0a7b640 1 -- 172.21.15.27:0/3409818662 shutdown_connections 2024-09-12T03:22:51.980 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.978+0000 7f71d0a7b640 1 -- 172.21.15.27:0/3409818662 wait complete. 2024-09-12T03:22:51.983 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.979+0000 7f71d0a7b640 1 Processor -- start 2024-09-12T03:22:51.983 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.979+0000 7f71d0a7b640 1 -- start start 2024-09-12T03:22:51.983 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.980+0000 7f71d0a7b640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f71cc0b3d10 0x7f71cc158a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:51.983 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.980+0000 7f71d0a7b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f71cc0b6230 0x7f71cc15ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:51.983 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.980+0000 7f71d0a7b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f71cc15e990 0x7f71cc1609b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:51.983 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.980+0000 7f71d0a7b640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f71cc0d35b0 con 0x7f71cc15e990 2024-09-12T03:22:51.983 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.980+0000 7f71d0a7b640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f71cc0d32d0 con 0x7f71cc0b6230 2024-09-12T03:22:51.983 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.980+0000 7f71d0a7b640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f71cc0d3ad0 con 0x7f71cc0b3d10 2024-09-12T03:22:51.983 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.980+0000 7f71cad9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f71cc15e990 0x7f71cc1609b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:51.984 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.980+0000 7f71cad9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f71cc15e990 0x7f71cc1609b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56214/0 (socket says 172.21.15.27:56214) 2024-09-12T03:22:51.984 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.980+0000 7f71cad9d640 1 -- 172.21.15.27:0/892254892 learned_addr learned my addr 172.21.15.27:0/892254892 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:22:51.984 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.980+0000 7f71c9d9b640 1 --2- 172.21.15.27:0/892254892 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f71cc0b6230 0x7f71cc15ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:51.984 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.980+0000 7f71ca59c640 1 --2- 172.21.15.27:0/892254892 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f71cc0b3d10 0x7f71cc158a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:51.984 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.980+0000 7f71cad9d640 1 -- 172.21.15.27:0/892254892 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f71cc0b3d10 msgr2=0x7f71cc158a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:51.984 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.980+0000 7f71cad9d640 1 --2- 172.21.15.27:0/892254892 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f71cc0b3d10 0x7f71cc158a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:51.984 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.980+0000 7f71cad9d640 1 -- 172.21.15.27:0/892254892 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f71cc0b6230 msgr2=0x7f71cc15ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:51.984 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.980+0000 7f71cad9d640 1 --2- 172.21.15.27:0/892254892 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f71cc0b6230 0x7f71cc15ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:51.984 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.980+0000 7f71cad9d640 1 -- 172.21.15.27:0/892254892 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71cc1f47b0 con 0x7f71cc15e990 2024-09-12T03:22:51.984 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.980+0000 7f71c9d9b640 1 --2- 172.21.15.27:0/892254892 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f71cc0b6230 0x7f71cc15ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:22:51.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.980+0000 7f71cad9d640 1 --2- 172.21.15.27:0/892254892 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f71cc15e990 0x7f71cc1609b0 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7f71bc009eb0 tx=0x7f71bc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:51.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.981+0000 7f71c2ffd640 1 -- 172.21.15.27:0/892254892 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71bc040070 con 0x7f71cc15e990 2024-09-12T03:22:51.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.981+0000 7f71c2ffd640 1 -- 172.21.15.27:0/892254892 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71bc03c830 con 0x7f71cc15e990 2024-09-12T03:22:51.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.981+0000 7f71d0a7b640 1 -- 172.21.15.27:0/892254892 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71cc1f4a90 con 0x7f71cc15e990 2024-09-12T03:22:51.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.981+0000 7f71c2ffd640 1 -- 172.21.15.27:0/892254892 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71bc03cbe0 con 0x7f71cc15e990 2024-09-12T03:22:51.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.981+0000 7f71d0a7b640 1 -- 172.21.15.27:0/892254892 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71cc1f5050 con 0x7f71cc15e990 2024-09-12T03:22:51.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.981+0000 7f71d0a7b640 1 -- 172.21.15.27:0/892254892 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71cc1f1d60 con 0x7f71cc15e990 2024-09-12T03:22:51.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.982+0000 7f71c2ffd640 1 -- 172.21.15.27:0/892254892 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 22) ==== 91059+0+0 (secure 0 0 0) 0x7f71bc039040 con 0x7f71cc15e990 2024-09-12T03:22:51.986 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.982+0000 7f71c2ffd640 1 --2- 172.21.15.27:0/892254892 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f71ac06c460 0x7f71ac06e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:51.986 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.983+0000 7f71c2ffd640 1 -- 172.21.15.27:0/892254892 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 5765+0+0 (secure 0 0 0) 0x7f71bc0acd00 con 0x7f71cc15e990 2024-09-12T03:22:51.986 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.983+0000 7f71ca59c640 1 --2- 172.21.15.27:0/892254892 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f71ac06c460 0x7f71ac06e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:51.986 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.983+0000 7f71ca59c640 1 --2- 172.21.15.27:0/892254892 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f71ac06c460 0x7f71ac06e920 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f71c40011e0 tx=0x7f71c4009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:51.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:51.987+0000 7f71c2ffd640 1 -- 172.21.15.27:0/892254892 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f71bc07ad10 con 0x7f71cc15e990 2024-09-12T03:22:52.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:52.202+0000 7f71d0a7b640 1 -- 172.21.15.27:0/892254892 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f71cc0d5350 con 0x7f71cc15e990 2024-09-12T03:22:52.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:52.857+0000 7f71c2ffd640 1 -- 172.21.15.27:0/892254892 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v82) ==== 122+0+0 (secure 0 0 0) 0x7f71bc04ce10 con 0x7f71cc15e990 2024-09-12T03:22:52.860 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:52.859+0000 7f71d0a7b640 1 -- 172.21.15.27:0/892254892 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f71ac06c460 msgr2=0x7f71ac06e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:52.860 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:52.859+0000 7f71d0a7b640 1 --2- 172.21.15.27:0/892254892 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f71ac06c460 0x7f71ac06e920 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f71c40011e0 tx=0x7f71c4009040 comp rx=0 tx=0).stop 2024-09-12T03:22:52.860 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:52.860+0000 7f71d0a7b640 1 -- 172.21.15.27:0/892254892 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f71cc15e990 msgr2=0x7f71cc1609b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:52.860 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:52.860+0000 7f71d0a7b640 1 --2- 172.21.15.27:0/892254892 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f71cc15e990 0x7f71cc1609b0 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7f71bc009eb0 tx=0x7f71bc004a40 comp rx=0 tx=0).stop 2024-09-12T03:22:52.860 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:52.860+0000 7f71d0a7b640 1 -- 172.21.15.27:0/892254892 shutdown_connections 2024-09-12T03:22:52.860 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:52.860+0000 7f71d0a7b640 1 --2- 172.21.15.27:0/892254892 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f71ac06c460 0x7f71ac06e920 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:52.860 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:52.860+0000 7f71d0a7b640 1 --2- 172.21.15.27:0/892254892 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f71cc15e990 0x7f71cc1609b0 unknown :-1 s=CLOSED pgs=1403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:52.860 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:52.860+0000 7f71d0a7b640 1 --2- 172.21.15.27:0/892254892 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f71cc0b6230 0x7f71cc15ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:52.860 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:52.860+0000 7f71d0a7b640 1 --2- 172.21.15.27:0/892254892 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f71cc0b3d10 0x7f71cc158a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:52.860 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:52.860+0000 7f71d0a7b640 1 -- 172.21.15.27:0/892254892 >> 172.21.15.27:0/892254892 conn(0x7f71cc0d1230 msgr2=0x7f71cc0d45b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:52.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:52.860+0000 7f71d0a7b640 1 -- 172.21.15.27:0/892254892 shutdown_connections 2024-09-12T03:22:52.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:52.860+0000 7f71d0a7b640 1 -- 172.21.15.27:0/892254892 wait complete. 2024-09-12T03:22:52.872 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-12T03:22:52.872 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-12T03:22:52.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:52.977+0000 7fde05d59640 1 Processor -- start 2024-09-12T03:22:52.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:52.977+0000 7fde05d59640 1 -- start start 2024-09-12T03:22:52.980 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:52.978+0000 7fde05d59640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fde00158cf0 0x7fde000b55d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:52.980 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:52.978+0000 7fde05d59640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fde001590d0 0x7fde00162430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:52.981 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:52.978+0000 7fde05d59640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fde00162970 con 0x7fde0015f600 2024-09-12T03:22:52.981 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:52.978+0000 7fde05d59640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7fde00162c10 con 0x7fde001590d0 2024-09-12T03:22:52.981 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:52.978+0000 7fde05d59640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fde00162ef0 con 0x7fde00158cf0 2024-09-12T03:22:52.981 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:52.978+0000 7fddfeffd640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fde001590d0 0x7fde00162430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:52.981 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:52.978+0000 7fddfffff640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fde0015f600 0x7fde0006bb30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:52344/0 (socket says 172.21.15.27:52344) 2024-09-12T03:22:52.981 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:52.978+0000 7fddfffff640 1 -- 172.21.15.27:0/183451345 learned_addr learned my addr 172.21.15.27:0/183451345 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:22:52.981 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:52.978+0000 7fddff7fe640 1 --2- 172.21.15.27:0/183451345 >> v2:172.21.15.45:3300/0 conn(0x7fde00158cf0 0x7fde000b55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:52.981 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:52.978+0000 7fddfeffd640 1 -- 172.21.15.27:0/183451345 >> v2:172.21.15.45:3300/0 conn(0x7fde00158cf0 msgr2=0x7fde000b55d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:52.981 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:52.978+0000 7fddfeffd640 1 --2- 172.21.15.27:0/183451345 >> v2:172.21.15.45:3300/0 conn(0x7fde00158cf0 0x7fde000b55d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:52.981 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:52.979+0000 7fddfeffd640 1 -- 172.21.15.27:0/183451345 >> v1:172.21.15.27:6789/0 conn(0x7fde0015f600 legacy=0x7fde0006bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:52.981 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:52.978+0000 7fddfe7fc640 1 -- 172.21.15.27:0/183451345 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1800039353 0 0) 0x7fde00162970 con 0x7fde0015f600 2024-09-12T03:22:52.981 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:52.979+0000 7fddfeffd640 1 -- 172.21.15.27:0/183451345 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde001631d0 con 0x7fde001590d0 2024-09-12T03:22:52.981 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:52.979+0000 7fddfeffd640 1 --2- 172.21.15.27:0/183451345 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fde001590d0 0x7fde00162430 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7fde000733c0 tx=0x7fddf402a080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3b20ef98f25ccba server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:52.982 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:52.979+0000 7fddfe7fc640 1 -- 172.21.15.27:0/183451345 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddf4037070 con 0x7fde001590d0 2024-09-12T03:22:52.982 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:52.979+0000 7fddfe7fc640 1 -- 172.21.15.27:0/183451345 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fddf402add0 con 0x7fde001590d0 2024-09-12T03:22:52.982 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:52.979+0000 7fddfe7fc640 1 -- 172.21.15.27:0/183451345 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddf4032660 con 0x7fde001590d0 2024-09-12T03:22:52.982 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:52.980+0000 7fde05d59640 1 -- 172.21.15.27:0/183451345 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fde001590d0 msgr2=0x7fde00162430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:52.982 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:52.980+0000 7fde05d59640 1 --2- 172.21.15.27:0/183451345 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fde001590d0 0x7fde00162430 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7fde000733c0 tx=0x7fddf402a080 comp rx=0 tx=0).stop 2024-09-12T03:22:52.982 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:52.980+0000 7fde05d59640 1 -- 172.21.15.27:0/183451345 shutdown_connections 2024-09-12T03:22:52.982 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:52.980+0000 7fde05d59640 1 --2- 172.21.15.27:0/183451345 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fde001590d0 0x7fde00162430 unknown :-1 s=CLOSED pgs=1405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:52.982 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:52.980+0000 7fde05d59640 1 --2- 172.21.15.27:0/183451345 >> v2:172.21.15.45:3300/0 conn(0x7fde00158cf0 0x7fde000b55d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:52.982 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:52.980+0000 7fde05d59640 1 -- 172.21.15.27:0/183451345 >> 172.21.15.27:0/183451345 conn(0x7fde0006cc40 msgr2=0x7fde0006d050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:52.982 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:52.980+0000 7fde05d59640 1 -- 172.21.15.27:0/183451345 shutdown_connections 2024-09-12T03:22:52.982 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:52.980+0000 7fde05d59640 1 -- 172.21.15.27:0/183451345 wait complete. 2024-09-12T03:22:52.984 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:52.981+0000 7fde05d59640 1 Processor -- start 2024-09-12T03:22:52.984 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:52.981+0000 7fde05d59640 1 -- start start 2024-09-12T03:22:52.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:52.982+0000 7fde05d59640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fde00158cf0 0x7fde0015cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:52.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:52.982+0000 7fde05d59640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fde001590d0 0x7fde0015e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:52.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:52.982+0000 7fde05d59640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fde0015f600 0x7fde00153370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:52.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:52.982+0000 7fde05d59640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fde0006c3c0 con 0x7fde001590d0 2024-09-12T03:22:52.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:52.982+0000 7fde05d59640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fde0006c240 con 0x7fde0015f600 2024-09-12T03:22:52.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:52.982+0000 7fde05d59640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fde0015fa70 con 0x7fde00158cf0 2024-09-12T03:22:52.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:52.982+0000 7fddfeffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fde001590d0 0x7fde0015e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:52.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:52.982+0000 7fddfeffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fde001590d0 0x7fde0015e3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56232/0 (socket says 172.21.15.27:56232) 2024-09-12T03:22:52.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:52.982+0000 7fddfeffd640 1 -- 172.21.15.27:0/2388442633 learned_addr learned my addr 172.21.15.27:0/2388442633 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:22:52.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:52.983+0000 7fddff7fe640 1 --2- 172.21.15.27:0/2388442633 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fde00158cf0 0x7fde0015cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:52.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:52.983+0000 7fddfffff640 1 --2- 172.21.15.27:0/2388442633 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fde0015f600 0x7fde00153370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:52.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:52.983+0000 7fddfeffd640 1 -- 172.21.15.27:0/2388442633 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fde00158cf0 msgr2=0x7fde0015cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:52.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:52.983+0000 7fddfeffd640 1 --2- 172.21.15.27:0/2388442633 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fde00158cf0 0x7fde0015cd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:52.986 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:52.983+0000 7fddfeffd640 1 -- 172.21.15.27:0/2388442633 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fde0015f600 msgr2=0x7fde00153370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:52.986 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:52.983+0000 7fddfeffd640 1 --2- 172.21.15.27:0/2388442633 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fde0015f600 0x7fde00153370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:52.986 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:52.983+0000 7fddfeffd640 1 -- 172.21.15.27:0/2388442633 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde0015d330 con 0x7fde001590d0 2024-09-12T03:22:52.986 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:52.983+0000 7fddfeffd640 1 --2- 172.21.15.27:0/2388442633 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fde001590d0 0x7fde0015e3d0 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7fddf4035040 tx=0x7fddf40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:52.986 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:52.983+0000 7fdde7fff640 1 -- 172.21.15.27:0/2388442633 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddf4037040 con 0x7fde001590d0 2024-09-12T03:22:52.986 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:52.983+0000 7fdde7fff640 1 -- 172.21.15.27:0/2388442633 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fddf403b430 con 0x7fde001590d0 2024-09-12T03:22:52.986 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:52.983+0000 7fdde7fff640 1 -- 172.21.15.27:0/2388442633 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddf403b820 con 0x7fde001590d0 2024-09-12T03:22:52.986 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:52.983+0000 7fde05d59640 1 -- 172.21.15.27:0/2388442633 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde0015d610 con 0x7fde001590d0 2024-09-12T03:22:52.986 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:52.983+0000 7fde05d59640 1 -- 172.21.15.27:0/2388442633 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde0015dbd0 con 0x7fde001590d0 2024-09-12T03:22:52.986 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:52.984+0000 7fde05d59640 1 -- 172.21.15.27:0/2388442633 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde000b55d0 con 0x7fde001590d0 2024-09-12T03:22:52.986 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:52.985+0000 7fdde7fff640 1 -- 172.21.15.27:0/2388442633 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 22) ==== 91059+0+0 (secure 0 0 0) 0x7fddf4044020 con 0x7fde001590d0 2024-09-12T03:22:52.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:52.985+0000 7fdde7fff640 1 --2- 172.21.15.27:0/2388442633 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fddec06c460 0x7fddec06e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:52.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:52.985+0000 7fdde7fff640 1 -- 172.21.15.27:0/2388442633 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 5765+0+0 (secure 0 0 0) 0x7fddf40abab0 con 0x7fde001590d0 2024-09-12T03:22:52.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:52.985+0000 7fddff7fe640 1 --2- 172.21.15.27:0/2388442633 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fddec06c460 0x7fddec06e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:52.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:52.986+0000 7fddff7fe640 1 --2- 172.21.15.27:0/2388442633 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fddec06c460 0x7fddec06e920 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fddf0006fd0 tx=0x7fddf0008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:52.990 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:52.990+0000 7fdde7fff640 1 -- 172.21.15.27:0/2388442633 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fddf4079b70 con 0x7fde001590d0 2024-09-12T03:22:53.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.200+0000 7fde05d59640 1 -- 172.21.15.27:0/2388442633 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fde000d0f40 con 0x7fde001590d0 2024-09-12T03:22:53.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.201+0000 7fdde7fff640 1 -- 172.21.15.27:0/2388442633 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 82 v82) ==== 94+0+5224 (secure 0 0 0) 0x7fddf4079470 con 0x7fde001590d0 2024-09-12T03:22:53.201 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:22:53.201 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":82,"btime":"2024-09-12T03:22:52:853366+0000","default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6689,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/2285501348","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2285501348},{"type":"v1","addr":"172.21.15.45:6833","nonce":2285501348}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79},{"gid":7453,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/1872420883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":1872420883},{"type":"v1","addr":"172.21.15.45:6835","nonce":1872420883}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79},{"gid":8193,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6834/3968488955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":3968488955},{"type":"v1","addr":"172.21.15.27:6834","nonce":3968488955}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79}],"filesystems":[{"mdsmap":{"epoch":82,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-12T03:22:50.200339+0000","modified":"2024-09-12T03:22:52.204820+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8196},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8196":{"gid":8196,"name":"a","rank":0,"incarnation":80,"state":"up:active","state_seq":3,"addr":"172.21.15.27:6835/389118996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":389118996},{"type":"v1","addr":"172.21.15.27:6835","nonce":389118996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8196,"qdb_cluster":[8196]},"id":11}]} 2024-09-12T03:22:53.201 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 82 2024-09-12T03:22:53.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.202+0000 7fde05d59640 1 -- 172.21.15.27:0/2388442633 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fddec06c460 msgr2=0x7fddec06e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:53.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.202+0000 7fde05d59640 1 --2- 172.21.15.27:0/2388442633 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fddec06c460 0x7fddec06e920 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fddf0006fd0 tx=0x7fddf0008040 comp rx=0 tx=0).stop 2024-09-12T03:22:53.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.202+0000 7fde05d59640 1 -- 172.21.15.27:0/2388442633 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fde001590d0 msgr2=0x7fde0015e3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:53.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.202+0000 7fde05d59640 1 --2- 172.21.15.27:0/2388442633 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fde001590d0 0x7fde0015e3d0 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7fddf4035040 tx=0x7fddf40046c0 comp rx=0 tx=0).stop 2024-09-12T03:22:53.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.203+0000 7fde05d59640 1 -- 172.21.15.27:0/2388442633 shutdown_connections 2024-09-12T03:22:53.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.203+0000 7fde05d59640 1 --2- 172.21.15.27:0/2388442633 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fde0015f600 0x7fde00153370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:53.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.203+0000 7fde05d59640 1 --2- 172.21.15.27:0/2388442633 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fddec06c460 0x7fddec06e920 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:53.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.203+0000 7fde05d59640 1 --2- 172.21.15.27:0/2388442633 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fde001590d0 0x7fde0015e3d0 unknown :-1 s=CLOSED pgs=1406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:53.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.203+0000 7fde05d59640 1 --2- 172.21.15.27:0/2388442633 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fde00158cf0 0x7fde0015cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:53.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.203+0000 7fde05d59640 1 -- 172.21.15.27:0/2388442633 >> 172.21.15.27:0/2388442633 conn(0x7fde0006cc40 msgr2=0x7fde0006bb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:53.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.203+0000 7fde05d59640 1 -- 172.21.15.27:0/2388442633 shutdown_connections 2024-09-12T03:22:53.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.203+0000 7fde05d59640 1 -- 172.21.15.27:0/2388442633 wait complete. 2024-09-12T03:22:53.213 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-12T03:22:53.316 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.316+0000 7f01e974b640 1 Processor -- start 2024-09-12T03:22:53.316 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.316+0000 7f01e974b640 1 -- start start 2024-09-12T03:22:53.317 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.317+0000 7f01e974b640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f01e415e5f0 0x7f01e4073250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:53.318 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.317+0000 7f01e974b640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f01e40d2240 con 0x7f01e4162a30 2024-09-12T03:22:53.318 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.317+0000 7f01e974b640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f01e4163090 con 0x7f01e4073790 2024-09-12T03:22:53.318 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.317+0000 7f01e974b640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f01e4162e60 con 0x7f01e415e5f0 2024-09-12T03:22:53.318 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.317+0000 7f01e37fe640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f01e4162a30 0x7f01e415e9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:52354/0 (socket says 172.21.15.27:52354) 2024-09-12T03:22:53.318 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.317+0000 7f01e2ffd640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f01e415e5f0 0x7f01e4073250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:53.318 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.317+0000 7f01e37fe640 1 -- 172.21.15.27:0/1781051590 learned_addr learned my addr 172.21.15.27:0/1781051590 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:22:53.318 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.318+0000 7f01e2ffd640 1 -- 172.21.15.27:0/1781051590 >> v1:172.21.15.45:6789/0 conn(0x7f01e4073790 legacy=0x7f01e4162320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:53.318 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.318+0000 7f01e2ffd640 1 -- 172.21.15.27:0/1781051590 >> v1:172.21.15.27:6789/0 conn(0x7f01e4162a30 legacy=0x7f01e415e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:53.318 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.318+0000 7f01e1ffb640 1 -- 172.21.15.27:0/1781051590 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 139734251 0 0) 0x7f01e40d2240 con 0x7f01e4162a30 2024-09-12T03:22:53.318 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.318+0000 7f01e2ffd640 1 -- 172.21.15.27:0/1781051590 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01e40d0470 con 0x7f01e415e5f0 2024-09-12T03:22:53.319 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.318+0000 7f01e2ffd640 1 --2- 172.21.15.27:0/1781051590 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f01e415e5f0 0x7f01e4073250 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7f01d40097a0 tx=0x7f01d402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd4b3ec0f8e05a45 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:53.320 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.319+0000 7f01e1ffb640 1 -- 172.21.15.27:0/1781051590 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01d4037070 con 0x7f01e415e5f0 2024-09-12T03:22:53.320 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.319+0000 7f01e1ffb640 1 -- 172.21.15.27:0/1781051590 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01d4032440 con 0x7f01e415e5f0 2024-09-12T03:22:53.320 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.319+0000 7f01e1ffb640 1 -- 172.21.15.27:0/1781051590 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01d4032810 con 0x7f01e415e5f0 2024-09-12T03:22:53.320 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.319+0000 7f01e974b640 1 -- 172.21.15.27:0/1781051590 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f01e415e5f0 msgr2=0x7f01e4073250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:53.320 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.319+0000 7f01e974b640 1 --2- 172.21.15.27:0/1781051590 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f01e415e5f0 0x7f01e4073250 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7f01d40097a0 tx=0x7f01d402a230 comp rx=0 tx=0).stop 2024-09-12T03:22:53.320 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.319+0000 7f01e974b640 1 -- 172.21.15.27:0/1781051590 shutdown_connections 2024-09-12T03:22:53.320 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.319+0000 7f01e974b640 1 --2- 172.21.15.27:0/1781051590 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f01e415e5f0 0x7f01e4073250 unknown :-1 s=CLOSED pgs=1408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:53.320 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.319+0000 7f01e974b640 1 -- 172.21.15.27:0/1781051590 >> 172.21.15.27:0/1781051590 conn(0x7f01e406f300 msgr2=0x7f01e406f710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:53.320 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.319+0000 7f01e974b640 1 -- 172.21.15.27:0/1781051590 shutdown_connections 2024-09-12T03:22:53.320 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.319+0000 7f01e974b640 1 -- 172.21.15.27:0/1781051590 wait complete. 2024-09-12T03:22:53.322 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.320+0000 7f01e974b640 1 Processor -- start 2024-09-12T03:22:53.323 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.321+0000 7f01e974b640 1 -- start start 2024-09-12T03:22:53.323 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.321+0000 7f01e974b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f01e4073790 0x7f01e41611e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:53.323 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.321+0000 7f01e974b640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f01e415e5f0 0x7f01e4141560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:53.323 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.321+0000 7f01e974b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f01e4162a30 0x7f01e4141aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:53.323 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.321+0000 7f01e974b640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f01e40d0da0 con 0x7f01e4162a30 2024-09-12T03:22:53.323 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.321+0000 7f01e974b640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f01e4073be0 con 0x7f01e4073790 2024-09-12T03:22:53.323 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.321+0000 7f01e974b640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f01e40d0a30 con 0x7f01e415e5f0 2024-09-12T03:22:53.323 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.321+0000 7f01e37fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f01e4162a30 0x7f01e4141aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:53.323 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.321+0000 7f01e37fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f01e4162a30 0x7f01e4141aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56260/0 (socket says 172.21.15.27:56260) 2024-09-12T03:22:53.323 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.321+0000 7f01e37fe640 1 -- 172.21.15.27:0/2044131706 learned_addr learned my addr 172.21.15.27:0/2044131706 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:22:53.323 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.321+0000 7f01e2ffd640 1 --2- 172.21.15.27:0/2044131706 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f01e4073790 0x7f01e41611e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:53.323 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.322+0000 7f01e27fc640 1 --2- 172.21.15.27:0/2044131706 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f01e415e5f0 0x7f01e4141560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:53.324 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.322+0000 7f01e37fe640 1 -- 172.21.15.27:0/2044131706 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f01e415e5f0 msgr2=0x7f01e4141560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:53.324 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.322+0000 7f01e37fe640 1 --2- 172.21.15.27:0/2044131706 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f01e415e5f0 0x7f01e4141560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:53.324 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.322+0000 7f01e37fe640 1 -- 172.21.15.27:0/2044131706 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f01e4073790 msgr2=0x7f01e41611e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:53.324 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.322+0000 7f01e37fe640 1 --2- 172.21.15.27:0/2044131706 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f01e4073790 0x7f01e41611e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:53.324 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.322+0000 7f01e37fe640 1 -- 172.21.15.27:0/2044131706 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01e41f8980 con 0x7f01e4162a30 2024-09-12T03:22:53.324 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.322+0000 7f01e37fe640 1 --2- 172.21.15.27:0/2044131706 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f01e4162a30 0x7f01e4141aa0 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7f01dc00c700 tx=0x7f01dc00cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:53.324 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.322+0000 7f01db7fe640 1 -- 172.21.15.27:0/2044131706 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01dc018070 con 0x7f01e4162a30 2024-09-12T03:22:53.324 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.322+0000 7f01db7fe640 1 -- 172.21.15.27:0/2044131706 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01dc00ce10 con 0x7f01e4162a30 2024-09-12T03:22:53.324 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.322+0000 7f01e974b640 1 -- 172.21.15.27:0/2044131706 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01e4144480 con 0x7f01e4162a30 2024-09-12T03:22:53.324 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.322+0000 7f01db7fe640 1 -- 172.21.15.27:0/2044131706 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01dc00a800 con 0x7f01e4162a30 2024-09-12T03:22:53.324 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.322+0000 7f01e974b640 1 -- 172.21.15.27:0/2044131706 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01e4144a40 con 0x7f01e4162a30 2024-09-12T03:22:53.325 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.323+0000 7f01e974b640 1 -- 172.21.15.27:0/2044131706 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01e40b56b0 con 0x7f01e4162a30 2024-09-12T03:22:53.325 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.324+0000 7f01db7fe640 1 -- 172.21.15.27:0/2044131706 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 22) ==== 91059+0+0 (secure 0 0 0) 0x7f01dc01b070 con 0x7f01e4162a30 2024-09-12T03:22:53.325 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.324+0000 7f01db7fe640 1 --2- 172.21.15.27:0/2044131706 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f01c406c460 0x7f01c406e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:53.325 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.324+0000 7f01db7fe640 1 -- 172.21.15.27:0/2044131706 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 5765+0+0 (secure 0 0 0) 0x7f01dc08ae20 con 0x7f01e4162a30 2024-09-12T03:22:53.325 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.324+0000 7f01e2ffd640 1 --2- 172.21.15.27:0/2044131706 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f01c406c460 0x7f01c406e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:53.326 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.325+0000 7f01e2ffd640 1 --2- 172.21.15.27:0/2044131706 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f01c406c460 0x7f01c406e920 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f01d4034000 tx=0x7f01d402e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:53.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.329+0000 7f01db7fe640 1 -- 172.21.15.27:0/2044131706 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f01dc058890 con 0x7f01e4162a30 2024-09-12T03:22:53.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.509+0000 7f01e974b640 1 -- 172.21.15.27:0/2044131706 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f01e4161c80 con 0x7f01e4162a30 2024-09-12T03:22:53.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.510+0000 7f01db7fe640 1 -- 172.21.15.27:0/2044131706 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v138) ==== 74+0+16772 (secure 0 0 0) 0x7f01dc05cb90 con 0x7f01e4162a30 2024-09-12T03:22:53.510 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:22:53.510 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":138,"fsid":"d2df504e-2647-4385-a5db-9fdf69f487a0","created":"2024-09-12T03:16:07.778937+0000","modified":"2024-09-12T03:22:51.848998+0000","last_up_change":"2024-09-12T03:16:19.314778+0000","last_in_change":"2024-09-12T03:16:09.893086+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T03:16:20.097646+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2024-09-12T03:22:47.859008+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"137","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2024-09-12T03:22:49.183109+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"138","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1851116c-1fb3-4dc9-9772-f90e2ef4c355","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6817","nonce":3002321488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6819","nonce":3002321488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6823","nonce":3002321488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6821","nonce":3002321488}]},"public_addr":"172.21.15.27:6817/3002321488","cluster_addr":"172.21.15.27:6819/3002321488","heartbeat_back_addr":"172.21.15.27:6823/3002321488","heartbeat_front_addr":"172.21.15.27:6821/3002321488","state":["exists","up"]},{"osd":1,"uuid":"f38ae17d-73eb-4204-b99d-975fa806fb87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6809","nonce":1613102886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6811","nonce":1613102886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6815","nonce":1613102886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6813","nonce":1613102886}]},"public_addr":"172.21.15.27:6809/1613102886","cluster_addr":"172.21.15.27:6811/1613102886","heartbeat_back_addr":"172.21.15.27:6815/1613102886","heartbeat_front_addr":"172.21.15.27:6813/1613102886","state":["exists","up"]},{"osd":2,"uuid":"fb12dcc8-bdd0-42b8-b46d-8b10106224bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6825","nonce":3534723355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6827","nonce":3534723355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6831","nonce":3534723355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6829","nonce":3534723355}]},"public_addr":"172.21.15.27:6825/3534723355","cluster_addr":"172.21.15.27:6827/3534723355","heartbeat_back_addr":"172.21.15.27:6831/3534723355","heartbeat_front_addr":"172.21.15.27:6829/3534723355","state":["exists","up"]},{"osd":3,"uuid":"d74d3f49-9c24-4b9f-98e2-3ecd7b5ef3b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6801","nonce":80661598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6803","nonce":80661598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6807","nonce":80661598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6805","nonce":80661598}]},"public_addr":"172.21.15.27:6801/80661598","cluster_addr":"172.21.15.27:6803/80661598","heartbeat_back_addr":"172.21.15.27:6807/80661598","heartbeat_front_addr":"172.21.15.27:6805/80661598","state":["exists","up"]},{"osd":4,"uuid":"2bf123f2-1b0d-41c4-aa3b-805bab267e59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6817","nonce":3943189827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6819","nonce":3943189827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6823","nonce":3943189827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6821","nonce":3943189827}]},"public_addr":"172.21.15.45:6817/3943189827","cluster_addr":"172.21.15.45:6819/3943189827","heartbeat_back_addr":"172.21.15.45:6823/3943189827","heartbeat_front_addr":"172.21.15.45:6821/3943189827","state":["exists","up"]},{"osd":5,"uuid":"c49ba5da-91c6-4a86-a97b-4aba89c253ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6809","nonce":3620760330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6811","nonce":3620760330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6815","nonce":3620760330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6813","nonce":3620760330}]},"public_addr":"172.21.15.45:6809/3620760330","cluster_addr":"172.21.15.45:6811/3620760330","heartbeat_back_addr":"172.21.15.45:6815/3620760330","heartbeat_front_addr":"172.21.15.45:6813/3620760330","state":["exists","up"]},{"osd":6,"uuid":"b0123475-89f7-4a34-9e33-d797b99dd9dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6801","nonce":1989083510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6803","nonce":1989083510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6807","nonce":1989083510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6805","nonce":1989083510}]},"public_addr":"172.21.15.45:6801/1989083510","cluster_addr":"172.21.15.45:6803/1989083510","heartbeat_back_addr":"172.21.15.45:6807/1989083510","heartbeat_front_addr":"172.21.15.45:6805/1989083510","state":["exists","up"]},{"osd":7,"uuid":"4dad773d-b7d8-46fc-8a10-a972aad27b29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6825","nonce":1513300294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6827","nonce":1513300294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6831","nonce":1513300294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6829","nonce":1513300294}]},"public_addr":"172.21.15.45:6825/1513300294","cluster_addr":"172.21.15.45:6827/1513300294","heartbeat_back_addr":"172.21.15.45:6831/1513300294","heartbeat_front_addr":"172.21.15.45:6829/1513300294","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.881855+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:17.133252+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.735339+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.500742+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.477571+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:17.167667+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.470291+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.873743+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]},{"pool":19,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T03:22:53.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.512+0000 7f01e974b640 1 -- 172.21.15.27:0/2044131706 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f01c406c460 msgr2=0x7f01c406e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:53.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.512+0000 7f01e974b640 1 --2- 172.21.15.27:0/2044131706 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f01c406c460 0x7f01c406e920 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f01d4034000 tx=0x7f01d402e000 comp rx=0 tx=0).stop 2024-09-12T03:22:53.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.512+0000 7f01e974b640 1 -- 172.21.15.27:0/2044131706 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f01e4162a30 msgr2=0x7f01e4141aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:53.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.512+0000 7f01e974b640 1 --2- 172.21.15.27:0/2044131706 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f01e4162a30 0x7f01e4141aa0 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7f01dc00c700 tx=0x7f01dc00cbc0 comp rx=0 tx=0).stop 2024-09-12T03:22:53.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.512+0000 7f01e974b640 1 -- 172.21.15.27:0/2044131706 shutdown_connections 2024-09-12T03:22:53.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.512+0000 7f01e974b640 1 --2- 172.21.15.27:0/2044131706 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f01c406c460 0x7f01c406e920 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:53.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.512+0000 7f01e974b640 1 --2- 172.21.15.27:0/2044131706 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f01e4162a30 0x7f01e4141aa0 unknown :-1 s=CLOSED pgs=1409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:53.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.512+0000 7f01e974b640 1 --2- 172.21.15.27:0/2044131706 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f01e415e5f0 0x7f01e4141560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:53.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.512+0000 7f01e974b640 1 --2- 172.21.15.27:0/2044131706 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f01e4073790 0x7f01e41611e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:53.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.512+0000 7f01e974b640 1 -- 172.21.15.27:0/2044131706 >> 172.21.15.27:0/2044131706 conn(0x7f01e406f300 msgr2=0x7f01e40b33b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:53.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.512+0000 7f01e974b640 1 -- 172.21.15.27:0/2044131706 shutdown_connections 2024-09-12T03:22:53.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.512+0000 7f01e974b640 1 -- 172.21.15.27:0/2044131706 wait complete. 2024-09-12T03:22:53.523 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-12T03:22:53.524 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-12T03:22:53.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.626+0000 7f947fb2a640 1 Processor -- start 2024-09-12T03:22:53.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.626+0000 7f947fb2a640 1 -- start start 2024-09-12T03:22:53.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.627+0000 7f947fb2a640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f9478158cf0 0x7f94780b55d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:53.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.627+0000 7f947fb2a640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f94781590d0 0x7f9478162430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:53.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.627+0000 7f947fb2a640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9478162970 con 0x7f947815f600 2024-09-12T03:22:53.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.627+0000 7f947fb2a640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f9478162c10 con 0x7f94781590d0 2024-09-12T03:22:53.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.627+0000 7f947fb2a640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f9478162ef0 con 0x7f9478158cf0 2024-09-12T03:22:53.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.627+0000 7f947d8c6640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f9478158cf0 0x7f94780b55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:53.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.627+0000 7f947d0c5640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f94781590d0 0x7f9478162430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:53.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.627+0000 7f947d0c5640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f94781590d0 0x7f9478162430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.27:36508/0 (socket says 172.21.15.27:36508) 2024-09-12T03:22:53.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.627+0000 7f947d0c5640 1 -- 172.21.15.27:0/4061418234 learned_addr learned my addr 172.21.15.27:0/4061418234 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:22:53.628 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.627+0000 7f947e0c7640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f947815f600 0x7f947806bb30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.27:49156/0 (socket says 172.21.15.27:49156) 2024-09-12T03:22:53.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.627+0000 7f947d0c5640 1 -- 172.21.15.27:0/4061418234 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f9478158cf0 msgr2=0x7f94780b55d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:53.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.627+0000 7f947d0c5640 1 --2- 172.21.15.27:0/4061418234 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f9478158cf0 0x7f94780b55d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:53.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.627+0000 7f947d0c5640 1 -- 172.21.15.27:0/4061418234 >> v1:172.21.15.45:6789/0 conn(0x7f947815f600 legacy=0x7f947806bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:53.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.627+0000 7f947d0c5640 1 -- 172.21.15.27:0/4061418234 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94781631d0 con 0x7f94781590d0 2024-09-12T03:22:53.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.628+0000 7f947d8c6640 1 --2- 172.21.15.27:0/4061418234 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f9478158cf0 0x7f94780b55d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:22:53.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.628+0000 7f947d0c5640 1 --2- 172.21.15.27:0/4061418234 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f94781590d0 0x7f9478162430 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f9468009870 tx=0x7f946802a130 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c2845dd276492e5d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:53.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.628+0000 7f947c8c4640 1 -- 172.21.15.27:0/4061418234 <== mon.1 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9468037070 con 0x7f94781590d0 2024-09-12T03:22:53.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.628+0000 7f947c8c4640 1 -- 172.21.15.27:0/4061418234 <== mon.1 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9468032440 con 0x7f94781590d0 2024-09-12T03:22:53.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.629+0000 7f947c8c4640 1 -- 172.21.15.27:0/4061418234 <== mon.1 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9468032810 con 0x7f94781590d0 2024-09-12T03:22:53.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.629+0000 7f947fb2a640 1 -- 172.21.15.27:0/4061418234 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f94781590d0 msgr2=0x7f9478162430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:53.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.629+0000 7f947fb2a640 1 --2- 172.21.15.27:0/4061418234 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f94781590d0 0x7f9478162430 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f9468009870 tx=0x7f946802a130 comp rx=0 tx=0).stop 2024-09-12T03:22:53.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.629+0000 7f947fb2a640 1 -- 172.21.15.27:0/4061418234 shutdown_connections 2024-09-12T03:22:53.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.629+0000 7f947fb2a640 1 --2- 172.21.15.27:0/4061418234 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f94781590d0 0x7f9478162430 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:53.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.629+0000 7f947fb2a640 1 --2- 172.21.15.27:0/4061418234 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f9478158cf0 0x7f94780b55d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:53.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.629+0000 7f947fb2a640 1 -- 172.21.15.27:0/4061418234 >> 172.21.15.27:0/4061418234 conn(0x7f947806cc40 msgr2=0x7f947806d050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:53.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.629+0000 7f947fb2a640 1 -- 172.21.15.27:0/4061418234 shutdown_connections 2024-09-12T03:22:53.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.629+0000 7f947fb2a640 1 -- 172.21.15.27:0/4061418234 wait complete. 2024-09-12T03:22:53.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.630+0000 7f947fb2a640 1 Processor -- start 2024-09-12T03:22:53.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.630+0000 7f947fb2a640 1 -- start start 2024-09-12T03:22:53.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.631+0000 7f947fb2a640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f9478158cf0 0x7f9478151870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:53.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.631+0000 7f947fb2a640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f94781590d0 0x7f94781511c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:53.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.631+0000 7f947fb2a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f947815f600 0x7f947815be30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:53.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.631+0000 7f947fb2a640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f947806c3c0 con 0x7f94781590d0 2024-09-12T03:22:53.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.631+0000 7f947fb2a640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f947806c240 con 0x7f947815f600 2024-09-12T03:22:53.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.631+0000 7f947fb2a640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f947815fa10 con 0x7f9478158cf0 2024-09-12T03:22:53.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.631+0000 7f947d0c5640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f94781590d0 0x7f94781511c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:53.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.631+0000 7f947d0c5640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f94781590d0 0x7f94781511c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56270/0 (socket says 172.21.15.27:56270) 2024-09-12T03:22:53.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.631+0000 7f947d0c5640 1 -- 172.21.15.27:0/484279638 learned_addr learned my addr 172.21.15.27:0/484279638 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:22:53.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.631+0000 7f947e0c7640 1 --2- 172.21.15.27:0/484279638 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f947815f600 0x7f947815be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:53.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.631+0000 7f947d8c6640 1 --2- 172.21.15.27:0/484279638 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f9478158cf0 0x7f9478151870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:53.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.632+0000 7f947d0c5640 1 -- 172.21.15.27:0/484279638 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f9478158cf0 msgr2=0x7f9478151870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:53.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.632+0000 7f947d0c5640 1 --2- 172.21.15.27:0/484279638 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f9478158cf0 0x7f9478151870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:53.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.632+0000 7f947d0c5640 1 -- 172.21.15.27:0/484279638 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f947815f600 msgr2=0x7f947815be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:53.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.632+0000 7f947d0c5640 1 --2- 172.21.15.27:0/484279638 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f947815f600 0x7f947815be30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:53.634 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.632+0000 7f947d0c5640 1 -- 172.21.15.27:0/484279638 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9478151f00 con 0x7f94781590d0 2024-09-12T03:22:53.634 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.632+0000 7f947d8c6640 1 --2- 172.21.15.27:0/484279638 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f9478158cf0 0x7f9478151870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:22:53.634 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.632+0000 7f947d0c5640 1 --2- 172.21.15.27:0/484279638 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f94781590d0 0x7f94781511c0 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f9468000c00 tx=0x7f94680046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:53.634 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.632+0000 7f946e7fc640 1 -- 172.21.15.27:0/484279638 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9468041020 con 0x7f94781590d0 2024-09-12T03:22:53.634 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.632+0000 7f946e7fc640 1 -- 172.21.15.27:0/484279638 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9468030640 con 0x7f94781590d0 2024-09-12T03:22:53.634 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.632+0000 7f947fb2a640 1 -- 172.21.15.27:0/484279638 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94781521e0 con 0x7f94781590d0 2024-09-12T03:22:53.634 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.632+0000 7f946e7fc640 1 -- 172.21.15.27:0/484279638 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9468037040 con 0x7f94781590d0 2024-09-12T03:22:53.634 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.632+0000 7f947fb2a640 1 -- 172.21.15.27:0/484279638 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94781527a0 con 0x7f94781590d0 2024-09-12T03:22:53.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.633+0000 7f947fb2a640 1 -- 172.21.15.27:0/484279638 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9478162430 con 0x7f94781590d0 2024-09-12T03:22:53.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.634+0000 7f946e7fc640 1 -- 172.21.15.27:0/484279638 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 22) ==== 91059+0+0 (secure 0 0 0) 0x7f9468044020 con 0x7f94781590d0 2024-09-12T03:22:53.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.634+0000 7f946e7fc640 1 --2- 172.21.15.27:0/484279638 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f945c06c460 0x7f945c06e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:53.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.635+0000 7f947d8c6640 1 --2- 172.21.15.27:0/484279638 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f945c06c460 0x7f945c06e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:53.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.635+0000 7f946e7fc640 1 -- 172.21.15.27:0/484279638 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 5765+0+0 (secure 0 0 0) 0x7f94680ac150 con 0x7f94781590d0 2024-09-12T03:22:53.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.635+0000 7f947d8c6640 1 --2- 172.21.15.27:0/484279638 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f945c06c460 0x7f945c06e920 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f9478160670 tx=0x7f9474008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:53.638 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.639+0000 7f946e7fc640 1 -- 172.21.15.27:0/484279638 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f946804b030 con 0x7f94781590d0 2024-09-12T03:22:53.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.810+0000 7f947fb2a640 1 -- 172.21.15.27:0/484279638 --> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f94781606a0 con 0x7f945c06c460 2024-09-12T03:22:53.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.816+0000 7f946e7fc640 1 -- 172.21.15.27:0/484279638 <== mgr.4113 v2:172.21.15.27:6836/1226920526 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252413 (secure 0 0 0) 0x7f94781606a0 con 0x7f945c06c460 2024-09-12T03:22:53.819 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:22:53.820 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2024-09-12T03:22:53.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.819+0000 7f947fb2a640 1 -- 172.21.15.27:0/484279638 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f945c06c460 msgr2=0x7f945c06e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:53.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.819+0000 7f947fb2a640 1 --2- 172.21.15.27:0/484279638 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f945c06c460 0x7f945c06e920 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f9478160670 tx=0x7f9474008040 comp rx=0 tx=0).stop 2024-09-12T03:22:53.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.819+0000 7f947fb2a640 1 -- 172.21.15.27:0/484279638 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f94781590d0 msgr2=0x7f94781511c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:53.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.819+0000 7f947fb2a640 1 --2- 172.21.15.27:0/484279638 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f94781590d0 0x7f94781511c0 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f9468000c00 tx=0x7f94680046c0 comp rx=0 tx=0).stop 2024-09-12T03:22:53.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.819+0000 7f947fb2a640 1 -- 172.21.15.27:0/484279638 shutdown_connections 2024-09-12T03:22:53.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.819+0000 7f947fb2a640 1 --2- 172.21.15.27:0/484279638 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f947815f600 0x7f947815be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:53.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.819+0000 7f947fb2a640 1 --2- 172.21.15.27:0/484279638 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f945c06c460 0x7f945c06e920 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:53.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.819+0000 7f947fb2a640 1 --2- 172.21.15.27:0/484279638 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f94781590d0 0x7f94781511c0 unknown :-1 s=CLOSED pgs=1410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:53.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.819+0000 7f947fb2a640 1 --2- 172.21.15.27:0/484279638 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f9478158cf0 0x7f9478151870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:53.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.819+0000 7f947fb2a640 1 -- 172.21.15.27:0/484279638 >> 172.21.15.27:0/484279638 conn(0x7f947806cc40 msgr2=0x7f94780b5050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:53.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.819+0000 7f947fb2a640 1 -- 172.21.15.27:0/484279638 shutdown_connections 2024-09-12T03:22:53.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.819+0000 7f947fb2a640 1 -- 172.21.15.27:0/484279638 wait complete. 2024-09-12T03:22:53.830 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":151,"stamp":"2024-09-12T03:22:53.127399+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":223,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":166,"ondisk_log_size":166,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":280724,"kb_used_data":14564,"kb_used_omap":12,"kb_used_meta":265971,"kb_avail":754693996,"statfs":{"total":773094113280,"available":772806651904,"internally_reserved":0,"allocated":14913536,"data_stored":7236798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":272354864},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.298416"},"pg_stats":[{"pgid":"23.3f","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-12T03:22:50.860209+0000","last_change":"2024-09-12T03:22:50.855893+0000","last_active":"2024-09-12T03:22:50.860209+0000","last_peered":"2024-09-12T03:22:50.860209+0000","last_clean":"2024-09-12T03:22:50.860209+0000","last_became_active":"2024-09-12T03:22:50.855595+0000","last_became_peered":"2024-09-12T03:22:50.855595+0000","last_unstale":"2024-09-12T03:22:50.860209+0000","last_undegraded":"2024-09-12T03:22:50.860209+0000","last_fullsized":"2024-09-12T03:22:50.860209+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_clean_scrub_stamp":"2024-09-12T03:22:49.828625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:34:27.627909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.3e","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-12T03:22:50.860575+0000","last_change":"2024-09-12T03:22:50.855396+0000","last_active":"2024-09-12T03:22:50.860575+0000","last_peered":"2024-09-12T03:22:50.860575+0000","last_clean":"2024-09-12T03:22:50.860575+0000","last_became_active":"2024-09-12T03:22:50.854895+0000","last_became_peered":"2024-09-12T03:22:50.854895+0000","last_unstale":"2024-09-12T03:22:50.860575+0000","last_undegraded":"2024-09-12T03:22:50.860575+0000","last_fullsized":"2024-09-12T03:22:50.860575+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_clean_scrub_stamp":"2024-09-12T03:22:49.828625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:01:56.553283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.3d","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-12T03:22:50.852260+0000","last_change":"2024-09-12T03:22:50.851101+0000","last_active":"2024-09-12T03:22:50.852260+0000","last_peered":"2024-09-12T03:22:50.852260+0000","last_clean":"2024-09-12T03:22:50.852260+0000","last_became_active":"2024-09-12T03:22:50.850787+0000","last_became_peered":"2024-09-12T03:22:50.850787+0000","last_unstale":"2024-09-12T03:22:50.852260+0000","last_undegraded":"2024-09-12T03:22:50.852260+0000","last_fullsized":"2024-09-12T03:22:50.852260+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_clean_scrub_stamp":"2024-09-12T03:22:49.828625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:19:04.989627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.3c","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-12T03:22:50.868519+0000","last_change":"2024-09-12T03:22:50.860649+0000","last_active":"2024-09-12T03:22:50.868519+0000","last_peered":"2024-09-12T03:22:50.868519+0000","last_clean":"2024-09-12T03:22:50.868519+0000","last_became_active":"2024-09-12T03:22:50.860355+0000","last_became_peered":"2024-09-12T03:22:50.860355+0000","last_unstale":"2024-09-12T03:22:50.868519+0000","last_undegraded":"2024-09-12T03:22:50.868519+0000","last_fullsized":"2024-09-12T03:22:50.868519+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_clean_scrub_stamp":"2024-09-12T03:22:49.828625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:56:01.904455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.3b","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-12T03:22:50.868263+0000","last_change":"2024-09-12T03:22:50.863316+0000","last_active":"2024-09-12T03:22:50.868263+0000","last_peered":"2024-09-12T03:22:50.868263+0000","last_clean":"2024-09-12T03:22:50.868263+0000","last_became_active":"2024-09-12T03:22:50.862971+0000","last_became_peered":"2024-09-12T03:22:50.862971+0000","last_unstale":"2024-09-12T03:22:50.868263+0000","last_undegraded":"2024-09-12T03:22:50.868263+0000","last_fullsized":"2024-09-12T03:22:50.868263+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_clean_scrub_stamp":"2024-09-12T03:22:49.828625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:50:53.181760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.3a","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-12T03:22:50.862655+0000","last_change":"2024-09-12T03:22:50.859146+0000","last_active":"2024-09-12T03:22:50.862655+0000","last_peered":"2024-09-12T03:22:50.862655+0000","last_clean":"2024-09-12T03:22:50.862655+0000","last_became_active":"2024-09-12T03:22:50.858862+0000","last_became_peered":"2024-09-12T03:22:50.858862+0000","last_unstale":"2024-09-12T03:22:50.862655+0000","last_undegraded":"2024-09-12T03:22:50.862655+0000","last_fullsized":"2024-09-12T03:22:50.862655+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_clean_scrub_stamp":"2024-09-12T03:22:49.828625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:00:22.288239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.39","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-12T03:22:50.864993+0000","last_change":"2024-09-12T03:22:50.859715+0000","last_active":"2024-09-12T03:22:50.864993+0000","last_peered":"2024-09-12T03:22:50.864993+0000","last_clean":"2024-09-12T03:22:50.864993+0000","last_became_active":"2024-09-12T03:22:50.859443+0000","last_became_peered":"2024-09-12T03:22:50.859443+0000","last_unstale":"2024-09-12T03:22:50.864993+0000","last_undegraded":"2024-09-12T03:22:50.864993+0000","last_fullsized":"2024-09-12T03:22:50.864993+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_clean_scrub_stamp":"2024-09-12T03:22:49.828625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:56:04.414983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.38","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-12T03:22:50.853525+0000","last_change":"2024-09-12T03:22:50.851401+0000","last_active":"2024-09-12T03:22:50.853525+0000","last_peered":"2024-09-12T03:22:50.853525+0000","last_clean":"2024-09-12T03:22:50.853525+0000","last_became_active":"2024-09-12T03:22:50.851137+0000","last_became_peered":"2024-09-12T03:22:50.851137+0000","last_unstale":"2024-09-12T03:22:50.853525+0000","last_undegraded":"2024-09-12T03:22:50.853525+0000","last_fullsized":"2024-09-12T03:22:50.853525+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_clean_scrub_stamp":"2024-09-12T03:22:49.828625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:50:45.170771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.37","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-12T03:22:50.870402+0000","last_change":"2024-09-12T03:22:50.868300+0000","last_active":"2024-09-12T03:22:50.870402+0000","last_peered":"2024-09-12T03:22:50.870402+0000","last_clean":"2024-09-12T03:22:50.870402+0000","last_became_active":"2024-09-12T03:22:50.867981+0000","last_became_peered":"2024-09-12T03:22:50.867981+0000","last_unstale":"2024-09-12T03:22:50.870402+0000","last_undegraded":"2024-09-12T03:22:50.870402+0000","last_fullsized":"2024-09-12T03:22:50.870402+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_clean_scrub_stamp":"2024-09-12T03:22:49.828625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:56:31.375666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.36","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-12T03:22:50.870085+0000","last_change":"2024-09-12T03:22:50.866608+0000","last_active":"2024-09-12T03:22:50.870085+0000","last_peered":"2024-09-12T03:22:50.870085+0000","last_clean":"2024-09-12T03:22:50.870085+0000","last_became_active":"2024-09-12T03:22:50.866323+0000","last_became_peered":"2024-09-12T03:22:50.866323+0000","last_unstale":"2024-09-12T03:22:50.870085+0000","last_undegraded":"2024-09-12T03:22:50.870085+0000","last_fullsized":"2024-09-12T03:22:50.870085+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_clean_scrub_stamp":"2024-09-12T03:22:49.828625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:00:42.792111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.35","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-12T03:22:50.862244+0000","last_change":"2024-09-12T03:22:50.858452+0000","last_active":"2024-09-12T03:22:50.862244+0000","last_peered":"2024-09-12T03:22:50.862244+0000","last_clean":"2024-09-12T03:22:50.862244+0000","last_became_active":"2024-09-12T03:22:50.858186+0000","last_became_peered":"2024-09-12T03:22:50.858186+0000","last_unstale":"2024-09-12T03:22:50.862244+0000","last_undegraded":"2024-09-12T03:22:50.862244+0000","last_fullsized":"2024-09-12T03:22:50.862244+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_clean_scrub_stamp":"2024-09-12T03:22:49.828625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:39:57.874510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.34","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-12T03:22:50.860615+0000","last_change":"2024-09-12T03:22:50.854108+0000","last_active":"2024-09-12T03:22:50.860615+0000","last_peered":"2024-09-12T03:22:50.860615+0000","last_clean":"2024-09-12T03:22:50.860615+0000","last_became_active":"2024-09-12T03:22:50.853827+0000","last_became_peered":"2024-09-12T03:22:50.853827+0000","last_unstale":"2024-09-12T03:22:50.860615+0000","last_undegraded":"2024-09-12T03:22:50.860615+0000","last_fullsized":"2024-09-12T03:22:50.860615+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_clean_scrub_stamp":"2024-09-12T03:22:49.828625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:11:51.287650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.33","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-12T03:22:50.864236+0000","last_change":"2024-09-12T03:22:50.862995+0000","last_active":"2024-09-12T03:22:50.864236+0000","last_peered":"2024-09-12T03:22:50.864236+0000","last_clean":"2024-09-12T03:22:50.864236+0000","last_became_active":"2024-09-12T03:22:50.861804+0000","last_became_peered":"2024-09-12T03:22:50.861804+0000","last_unstale":"2024-09-12T03:22:50.864236+0000","last_undegraded":"2024-09-12T03:22:50.864236+0000","last_fullsized":"2024-09-12T03:22:50.864236+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_clean_scrub_stamp":"2024-09-12T03:22:49.828625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:08:11.122706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.32","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-12T03:22:50.864101+0000","last_change":"2024-09-12T03:22:50.855025+0000","last_active":"2024-09-12T03:22:50.864101+0000","last_peered":"2024-09-12T03:22:50.864101+0000","last_clean":"2024-09-12T03:22:50.864101+0000","last_became_active":"2024-09-12T03:22:50.854738+0000","last_became_peered":"2024-09-12T03:22:50.854738+0000","last_unstale":"2024-09-12T03:22:50.864101+0000","last_undegraded":"2024-09-12T03:22:50.864101+0000","last_fullsized":"2024-09-12T03:22:50.864101+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_clean_scrub_stamp":"2024-09-12T03:22:49.828625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:54:04.916494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.31","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-12T03:22:50.868807+0000","last_change":"2024-09-12T03:22:50.867245+0000","last_active":"2024-09-12T03:22:50.868807+0000","last_peered":"2024-09-12T03:22:50.868807+0000","last_clean":"2024-09-12T03:22:50.868807+0000","last_became_active":"2024-09-12T03:22:50.866963+0000","last_became_peered":"2024-09-12T03:22:50.866963+0000","last_unstale":"2024-09-12T03:22:50.868807+0000","last_undegraded":"2024-09-12T03:22:50.868807+0000","last_fullsized":"2024-09-12T03:22:50.868807+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_clean_scrub_stamp":"2024-09-12T03:22:49.828625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:35:10.105847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.30","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-12T03:22:50.856766+0000","last_change":"2024-09-12T03:22:50.855923+0000","last_active":"2024-09-12T03:22:50.856766+0000","last_peered":"2024-09-12T03:22:50.856766+0000","last_clean":"2024-09-12T03:22:50.856766+0000","last_became_active":"2024-09-12T03:22:50.855655+0000","last_became_peered":"2024-09-12T03:22:50.855655+0000","last_unstale":"2024-09-12T03:22:50.856766+0000","last_undegraded":"2024-09-12T03:22:50.856766+0000","last_fullsized":"2024-09-12T03:22:50.856766+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_clean_scrub_stamp":"2024-09-12T03:22:49.828625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:32:53.224639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.2f","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-12T03:22:50.862408+0000","last_change":"2024-09-12T03:22:50.856698+0000","last_active":"2024-09-12T03:22:50.862408+0000","last_peered":"2024-09-12T03:22:50.862408+0000","last_clean":"2024-09-12T03:22:50.862408+0000","last_became_active":"2024-09-12T03:22:50.855891+0000","last_became_peered":"2024-09-12T03:22:50.855891+0000","last_unstale":"2024-09-12T03:22:50.862408+0000","last_undegraded":"2024-09-12T03:22:50.862408+0000","last_fullsized":"2024-09-12T03:22:50.862408+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_clean_scrub_stamp":"2024-09-12T03:22:49.828625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:45:22.646582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.2e","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-12T03:22:50.864644+0000","last_change":"2024-09-12T03:22:50.863716+0000","last_active":"2024-09-12T03:22:50.864644+0000","last_peered":"2024-09-12T03:22:50.864644+0000","last_clean":"2024-09-12T03:22:50.864644+0000","last_became_active":"2024-09-12T03:22:50.863404+0000","last_became_peered":"2024-09-12T03:22:50.863404+0000","last_unstale":"2024-09-12T03:22:50.864644+0000","last_undegraded":"2024-09-12T03:22:50.864644+0000","last_fullsized":"2024-09-12T03:22:50.864644+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_clean_scrub_stamp":"2024-09-12T03:22:49.828625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:02:54.239157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.2d","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-12T03:22:50.869383+0000","last_change":"2024-09-12T03:22:50.867116+0000","last_active":"2024-09-12T03:22:50.869383+0000","last_peered":"2024-09-12T03:22:50.869383+0000","last_clean":"2024-09-12T03:22:50.869383+0000","last_became_active":"2024-09-12T03:22:50.866777+0000","last_became_peered":"2024-09-12T03:22:50.866777+0000","last_unstale":"2024-09-12T03:22:50.869383+0000","last_undegraded":"2024-09-12T03:22:50.869383+0000","last_fullsized":"2024-09-12T03:22:50.869383+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_clean_scrub_stamp":"2024-09-12T03:22:49.828625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:21:29.676316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.2c","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-12T03:22:50.863911+0000","last_change":"2024-09-12T03:22:50.855007+0000","last_active":"2024-09-12T03:22:50.863911+0000","last_peered":"2024-09-12T03:22:50.863911+0000","last_clean":"2024-09-12T03:22:50.863911+0000","last_became_active":"2024-09-12T03:22:50.854562+0000","last_became_peered":"2024-09-12T03:22:50.854562+0000","last_unstale":"2024-09-12T03:22:50.863911+0000","last_undegraded":"2024-09-12T03:22:50.863911+0000","last_fullsized":"2024-09-12T03:22:50.863911+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_clean_scrub_stamp":"2024-09-12T03:22:49.828625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:22:23.000995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.2b","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-12T03:22:50.868519+0000","last_change":"2024-09-12T03:22:50.865234+0000","last_active":"2024-09-12T03:22:50.868519+0000","last_peered":"2024-09-12T03:22:50.868519+0000","last_clean":"2024-09-12T03:22:50.868519+0000","last_became_active":"2024-09-12T03:22:50.864974+0000","last_became_peered":"2024-09-12T03:22:50.864974+0000","last_unstale":"2024-09-12T03:22:50.868519+0000","last_undegraded":"2024-09-12T03:22:50.868519+0000","last_fullsized":"2024-09-12T03:22:50.868519+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_clean_scrub_stamp":"2024-09-12T03:22:49.828625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:11:46.457103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.2a","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-12T03:22:50.861940+0000","last_change":"2024-09-12T03:22:50.858104+0000","last_active":"2024-09-12T03:22:50.861940+0000","last_peered":"2024-09-12T03:22:50.861940+0000","last_clean":"2024-09-12T03:22:50.861940+0000","last_became_active":"2024-09-12T03:22:50.857769+0000","last_became_peered":"2024-09-12T03:22:50.857769+0000","last_unstale":"2024-09-12T03:22:50.861940+0000","last_undegraded":"2024-09-12T03:22:50.861940+0000","last_fullsized":"2024-09-12T03:22:50.861940+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_clean_scrub_stamp":"2024-09-12T03:22:49.828625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:28:17.630365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.29","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-12T03:22:50.862626+0000","last_change":"2024-09-12T03:22:50.859217+0000","last_active":"2024-09-12T03:22:50.862626+0000","last_peered":"2024-09-12T03:22:50.862626+0000","last_clean":"2024-09-12T03:22:50.862626+0000","last_became_active":"2024-09-12T03:22:50.858944+0000","last_became_peered":"2024-09-12T03:22:50.858944+0000","last_unstale":"2024-09-12T03:22:50.862626+0000","last_undegraded":"2024-09-12T03:22:50.862626+0000","last_fullsized":"2024-09-12T03:22:50.862626+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_clean_scrub_stamp":"2024-09-12T03:22:49.828625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:50:56.415485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.28","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-12T03:22:50.868135+0000","last_change":"2024-09-12T03:22:50.863333+0000","last_active":"2024-09-12T03:22:50.868135+0000","last_peered":"2024-09-12T03:22:50.868135+0000","last_clean":"2024-09-12T03:22:50.868135+0000","last_became_active":"2024-09-12T03:22:50.862644+0000","last_became_peered":"2024-09-12T03:22:50.862644+0000","last_unstale":"2024-09-12T03:22:50.868135+0000","last_undegraded":"2024-09-12T03:22:50.868135+0000","last_fullsized":"2024-09-12T03:22:50.868135+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_clean_scrub_stamp":"2024-09-12T03:22:49.828625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:26:57.478055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.27","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-12T03:22:50.851965+0000","last_change":"2024-09-12T03:22:50.850375+0000","last_active":"2024-09-12T03:22:50.851965+0000","last_peered":"2024-09-12T03:22:50.851965+0000","last_clean":"2024-09-12T03:22:50.851965+0000","last_became_active":"2024-09-12T03:22:50.849587+0000","last_became_peered":"2024-09-12T03:22:50.849587+0000","last_unstale":"2024-09-12T03:22:50.851965+0000","last_undegraded":"2024-09-12T03:22:50.851965+0000","last_fullsized":"2024-09-12T03:22:50.851965+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_clean_scrub_stamp":"2024-09-12T03:22:49.828625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:39:31.969229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.26","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-12T03:22:50.874381+0000","last_change":"2024-09-12T03:22:50.871659+0000","last_active":"2024-09-12T03:22:50.874381+0000","last_peered":"2024-09-12T03:22:50.874381+0000","last_clean":"2024-09-12T03:22:50.874381+0000","last_became_active":"2024-09-12T03:22:50.871361+0000","last_became_peered":"2024-09-12T03:22:50.871361+0000","last_unstale":"2024-09-12T03:22:50.874381+0000","last_undegraded":"2024-09-12T03:22:50.874381+0000","last_fullsized":"2024-09-12T03:22:50.874381+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_clean_scrub_stamp":"2024-09-12T03:22:49.828625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:36:08.825514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.25","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-12T03:22:50.869823+0000","last_change":"2024-09-12T03:22:50.867563+0000","last_active":"2024-09-12T03:22:50.869823+0000","last_peered":"2024-09-12T03:22:50.869823+0000","last_clean":"2024-09-12T03:22:50.869823+0000","last_became_active":"2024-09-12T03:22:50.867225+0000","last_became_peered":"2024-09-12T03:22:50.867225+0000","last_unstale":"2024-09-12T03:22:50.869823+0000","last_undegraded":"2024-09-12T03:22:50.869823+0000","last_fullsized":"2024-09-12T03:22:50.869823+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_clean_scrub_stamp":"2024-09-12T03:22:49.828625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:35:27.521408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.24","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-12T03:22:50.865985+0000","last_change":"2024-09-12T03:22:50.850968+0000","last_active":"2024-09-12T03:22:50.865985+0000","last_peered":"2024-09-12T03:22:50.865985+0000","last_clean":"2024-09-12T03:22:50.865985+0000","last_became_active":"2024-09-12T03:22:50.850655+0000","last_became_peered":"2024-09-12T03:22:50.850655+0000","last_unstale":"2024-09-12T03:22:50.865985+0000","last_undegraded":"2024-09-12T03:22:50.865985+0000","last_fullsized":"2024-09-12T03:22:50.865985+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_clean_scrub_stamp":"2024-09-12T03:22:49.828625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:05:39.784163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.23","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-12T03:22:50.869640+0000","last_change":"2024-09-12T03:22:50.867343+0000","last_active":"2024-09-12T03:22:50.869640+0000","last_peered":"2024-09-12T03:22:50.869640+0000","last_clean":"2024-09-12T03:22:50.869640+0000","last_became_active":"2024-09-12T03:22:50.867098+0000","last_became_peered":"2024-09-12T03:22:50.867098+0000","last_unstale":"2024-09-12T03:22:50.869640+0000","last_undegraded":"2024-09-12T03:22:50.869640+0000","last_fullsized":"2024-09-12T03:22:50.869640+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_clean_scrub_stamp":"2024-09-12T03:22:49.828625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:20:44.992719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.22","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-12T03:22:50.866835+0000","last_change":"2024-09-12T03:22:50.859328+0000","last_active":"2024-09-12T03:22:50.866835+0000","last_peered":"2024-09-12T03:22:50.866835+0000","last_clean":"2024-09-12T03:22:50.866835+0000","last_became_active":"2024-09-12T03:22:50.859032+0000","last_became_peered":"2024-09-12T03:22:50.859032+0000","last_unstale":"2024-09-12T03:22:50.866835+0000","last_undegraded":"2024-09-12T03:22:50.866835+0000","last_fullsized":"2024-09-12T03:22:50.866835+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_clean_scrub_stamp":"2024-09-12T03:22:49.828625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:02:33.474017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.21","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-12T03:22:50.867018+0000","last_change":"2024-09-12T03:22:50.860225+0000","last_active":"2024-09-12T03:22:50.867018+0000","last_peered":"2024-09-12T03:22:50.867018+0000","last_clean":"2024-09-12T03:22:50.867018+0000","last_became_active":"2024-09-12T03:22:50.859920+0000","last_became_peered":"2024-09-12T03:22:50.859920+0000","last_unstale":"2024-09-12T03:22:50.867018+0000","last_undegraded":"2024-09-12T03:22:50.867018+0000","last_fullsized":"2024-09-12T03:22:50.867018+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_clean_scrub_stamp":"2024-09-12T03:22:49.828625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:21:49.881460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.20","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-12T03:22:50.864664+0000","last_change":"2024-09-12T03:22:50.862949+0000","last_active":"2024-09-12T03:22:50.864664+0000","last_peered":"2024-09-12T03:22:50.864664+0000","last_clean":"2024-09-12T03:22:50.864664+0000","last_became_active":"2024-09-12T03:22:50.862688+0000","last_became_peered":"2024-09-12T03:22:50.862688+0000","last_unstale":"2024-09-12T03:22:50.864664+0000","last_undegraded":"2024-09-12T03:22:50.864664+0000","last_fullsized":"2024-09-12T03:22:50.864664+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_clean_scrub_stamp":"2024-09-12T03:22:49.828625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:02:33.474017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.1e","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-12T03:22:50.874358+0000","last_change":"2024-09-12T03:22:50.870927+0000","last_active":"2024-09-12T03:22:50.874358+0000","last_peered":"2024-09-12T03:22:50.874358+0000","last_clean":"2024-09-12T03:22:50.874358+0000","last_became_active":"2024-09-12T03:22:50.870624+0000","last_became_peered":"2024-09-12T03:22:50.870624+0000","last_unstale":"2024-09-12T03:22:50.874358+0000","last_undegraded":"2024-09-12T03:22:50.874358+0000","last_fullsized":"2024-09-12T03:22:50.874358+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_clean_scrub_stamp":"2024-09-12T03:22:49.828625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:41:46.198481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1f","version":"137'4","reported_seq":16,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-12T03:22:50.870583+0000","last_change":"2024-09-12T03:22:49.855893+0000","last_active":"2024-09-12T03:22:50.870583+0000","last_peered":"2024-09-12T03:22:50.870583+0000","last_clean":"2024-09-12T03:22:50.870583+0000","last_became_active":"2024-09-12T03:22:49.855580+0000","last_became_peered":"2024-09-12T03:22:49.855580+0000","last_unstale":"2024-09-12T03:22:50.870583+0000","last_undegraded":"2024-09-12T03:22:50.870583+0000","last_fullsized":"2024-09-12T03:22:50.870583+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:48.826338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:48.826338+0000","last_clean_scrub_stamp":"2024-09-12T03:22:48.826338+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:22:18.465634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1f","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-12T03:22:50.866922+0000","last_change":"2024-09-12T03:22:50.860093+0000","last_active":"2024-09-12T03:22:50.866922+0000","last_peered":"2024-09-12T03:22:50.866922+0000","last_clean":"2024-09-12T03:22:50.866922+0000","last_became_active":"2024-09-12T03:22:50.859792+0000","last_became_peered":"2024-09-12T03:22:50.859792+0000","last_unstale":"2024-09-12T03:22:50.866922+0000","last_undegraded":"2024-09-12T03:22:50.866922+0000","last_fullsized":"2024-09-12T03:22:50.866922+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_clean_scrub_stamp":"2024-09-12T03:22:49.828625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:02:54.239157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.1e","version":"137'1","reported_seq":13,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-12T03:22:50.867566+0000","last_change":"2024-09-12T03:22:49.852179+0000","last_active":"2024-09-12T03:22:50.867566+0000","last_peered":"2024-09-12T03:22:50.867566+0000","last_clean":"2024-09-12T03:22:50.867566+0000","last_became_active":"2024-09-12T03:22:49.851780+0000","last_became_peered":"2024-09-12T03:22:49.851780+0000","last_unstale":"2024-09-12T03:22:50.867566+0000","last_undegraded":"2024-09-12T03:22:50.867566+0000","last_fullsized":"2024-09-12T03:22:50.867566+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:48.826338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:48.826338+0000","last_clean_scrub_stamp":"2024-09-12T03:22:48.826338+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:20:31.945130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.1c","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-12T03:22:50.862539+0000","last_change":"2024-09-12T03:22:50.855355+0000","last_active":"2024-09-12T03:22:50.862539+0000","last_peered":"2024-09-12T03:22:50.862539+0000","last_clean":"2024-09-12T03:22:50.862539+0000","last_became_active":"2024-09-12T03:22:50.855084+0000","last_became_peered":"2024-09-12T03:22:50.855084+0000","last_unstale":"2024-09-12T03:22:50.862539+0000","last_undegraded":"2024-09-12T03:22:50.862539+0000","last_fullsized":"2024-09-12T03:22:50.862539+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_clean_scrub_stamp":"2024-09-12T03:22:49.828625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:56:54.187555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.1d","version":"137'2","reported_seq":14,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-12T03:22:50.871473+0000","last_change":"2024-09-12T03:22:49.852968+0000","last_active":"2024-09-12T03:22:50.871473+0000","last_peered":"2024-09-12T03:22:50.871473+0000","last_clean":"2024-09-12T03:22:50.871473+0000","last_became_active":"2024-09-12T03:22:49.852628+0000","last_became_peered":"2024-09-12T03:22:49.852628+0000","last_unstale":"2024-09-12T03:22:50.871473+0000","last_undegraded":"2024-09-12T03:22:50.871473+0000","last_fullsized":"2024-09-12T03:22:50.871473+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:48.826338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:48.826338+0000","last_clean_scrub_stamp":"2024-09-12T03:22:48.826338+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:16:22.473280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.1d","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-12T03:22:50.864170+0000","last_change":"2024-09-12T03:22:50.858651+0000","last_active":"2024-09-12T03:22:50.864170+0000","last_peered":"2024-09-12T03:22:50.864170+0000","last_clean":"2024-09-12T03:22:50.864170+0000","last_became_active":"2024-09-12T03:22:50.858374+0000","last_became_peered":"2024-09-12T03:22:50.858374+0000","last_unstale":"2024-09-12T03:22:50.864170+0000","last_undegraded":"2024-09-12T03:22:50.864170+0000","last_fullsized":"2024-09-12T03:22:50.864170+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_clean_scrub_stamp":"2024-09-12T03:22:49.828625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:25:37.877910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1c","version":"137'2","reported_seq":14,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-12T03:22:50.866147+0000","last_change":"2024-09-12T03:22:49.850674+0000","last_active":"2024-09-12T03:22:50.866147+0000","last_peered":"2024-09-12T03:22:50.866147+0000","last_clean":"2024-09-12T03:22:50.866147+0000","last_became_active":"2024-09-12T03:22:49.850328+0000","last_became_peered":"2024-09-12T03:22:49.850328+0000","last_unstale":"2024-09-12T03:22:50.866147+0000","last_undegraded":"2024-09-12T03:22:50.866147+0000","last_fullsized":"2024-09-12T03:22:50.866147+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:48.826338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:48.826338+0000","last_clean_scrub_stamp":"2024-09-12T03:22:48.826338+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:53:10.816034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.1a","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-12T03:22:50.865023+0000","last_change":"2024-09-12T03:22:50.858876+0000","last_active":"2024-09-12T03:22:50.865023+0000","last_peered":"2024-09-12T03:22:50.865023+0000","last_clean":"2024-09-12T03:22:50.865023+0000","last_became_active":"2024-09-12T03:22:50.858560+0000","last_became_peered":"2024-09-12T03:22:50.858560+0000","last_unstale":"2024-09-12T03:22:50.865023+0000","last_undegraded":"2024-09-12T03:22:50.865023+0000","last_fullsized":"2024-09-12T03:22:50.865023+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_clean_scrub_stamp":"2024-09-12T03:22:49.828625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:02:02.553069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1b","version":"137'1","reported_seq":13,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-12T03:22:50.869825+0000","last_change":"2024-09-12T03:22:49.848462+0000","last_active":"2024-09-12T03:22:50.869825+0000","last_peered":"2024-09-12T03:22:50.869825+0000","last_clean":"2024-09-12T03:22:50.869825+0000","last_became_active":"2024-09-12T03:22:49.848180+0000","last_became_peered":"2024-09-12T03:22:49.848180+0000","last_unstale":"2024-09-12T03:22:50.869825+0000","last_undegraded":"2024-09-12T03:22:50.869825+0000","last_fullsized":"2024-09-12T03:22:50.869825+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:48.826338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:48.826338+0000","last_clean_scrub_stamp":"2024-09-12T03:22:48.826338+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:27:52.204757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.1b","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-12T03:22:50.868526+0000","last_change":"2024-09-12T03:22:50.863673+0000","last_active":"2024-09-12T03:22:50.868526+0000","last_peered":"2024-09-12T03:22:50.868526+0000","last_clean":"2024-09-12T03:22:50.868526+0000","last_became_active":"2024-09-12T03:22:50.863369+0000","last_became_peered":"2024-09-12T03:22:50.863369+0000","last_unstale":"2024-09-12T03:22:50.868526+0000","last_undegraded":"2024-09-12T03:22:50.868526+0000","last_fullsized":"2024-09-12T03:22:50.868526+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_clean_scrub_stamp":"2024-09-12T03:22:49.828625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:25:42.433733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.1a","version":"137'1","reported_seq":13,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-12T03:22:50.870008+0000","last_change":"2024-09-12T03:22:49.855130+0000","last_active":"2024-09-12T03:22:50.870008+0000","last_peered":"2024-09-12T03:22:50.870008+0000","last_clean":"2024-09-12T03:22:50.870008+0000","last_became_active":"2024-09-12T03:22:49.854800+0000","last_became_peered":"2024-09-12T03:22:49.854800+0000","last_unstale":"2024-09-12T03:22:50.870008+0000","last_undegraded":"2024-09-12T03:22:50.870008+0000","last_fullsized":"2024-09-12T03:22:50.870008+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:48.826338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:48.826338+0000","last_clean_scrub_stamp":"2024-09-12T03:22:48.826338+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:59:04.016946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.18","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-12T03:22:50.865424+0000","last_change":"2024-09-12T03:22:50.857379+0000","last_active":"2024-09-12T03:22:50.865424+0000","last_peered":"2024-09-12T03:22:50.865424+0000","last_clean":"2024-09-12T03:22:50.865424+0000","last_became_active":"2024-09-12T03:22:50.856075+0000","last_became_peered":"2024-09-12T03:22:50.856075+0000","last_unstale":"2024-09-12T03:22:50.865424+0000","last_undegraded":"2024-09-12T03:22:50.865424+0000","last_fullsized":"2024-09-12T03:22:50.865424+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_clean_scrub_stamp":"2024-09-12T03:22:49.828625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:02:29.722088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.19","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-12T03:22:50.847213+0000","last_change":"2024-09-12T03:22:49.851694+0000","last_active":"2024-09-12T03:22:50.847213+0000","last_peered":"2024-09-12T03:22:50.847213+0000","last_clean":"2024-09-12T03:22:50.847213+0000","last_became_active":"2024-09-12T03:22:49.851405+0000","last_became_peered":"2024-09-12T03:22:49.851405+0000","last_unstale":"2024-09-12T03:22:50.847213+0000","last_undegraded":"2024-09-12T03:22:50.847213+0000","last_fullsized":"2024-09-12T03:22:50.847213+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:48.826338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:48.826338+0000","last_clean_scrub_stamp":"2024-09-12T03:22:48.826338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:28:11.350507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.19","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-12T03:22:50.862608+0000","last_change":"2024-09-12T03:22:50.858426+0000","last_active":"2024-09-12T03:22:50.862608+0000","last_peered":"2024-09-12T03:22:50.862608+0000","last_clean":"2024-09-12T03:22:50.862608+0000","last_became_active":"2024-09-12T03:22:50.858101+0000","last_became_peered":"2024-09-12T03:22:50.858101+0000","last_unstale":"2024-09-12T03:22:50.862608+0000","last_undegraded":"2024-09-12T03:22:50.862608+0000","last_fullsized":"2024-09-12T03:22:50.862608+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_clean_scrub_stamp":"2024-09-12T03:22:49.828625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:59:46.483645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.18","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-12T03:22:50.848699+0000","last_change":"2024-09-12T03:22:49.858710+0000","last_active":"2024-09-12T03:22:50.848699+0000","last_peered":"2024-09-12T03:22:50.848699+0000","last_clean":"2024-09-12T03:22:50.848699+0000","last_became_active":"2024-09-12T03:22:49.857931+0000","last_became_peered":"2024-09-12T03:22:49.857931+0000","last_unstale":"2024-09-12T03:22:50.848699+0000","last_undegraded":"2024-09-12T03:22:50.848699+0000","last_fullsized":"2024-09-12T03:22:50.848699+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:48.826338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:48.826338+0000","last_clean_scrub_stamp":"2024-09-12T03:22:48.826338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:34:26.625621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.17","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-12T03:22:50.872182+0000","last_change":"2024-09-12T03:22:50.869769+0000","last_active":"2024-09-12T03:22:50.872182+0000","last_peered":"2024-09-12T03:22:50.872182+0000","last_clean":"2024-09-12T03:22:50.872182+0000","last_became_active":"2024-09-12T03:22:50.868700+0000","last_became_peered":"2024-09-12T03:22:50.868700+0000","last_unstale":"2024-09-12T03:22:50.872182+0000","last_undegraded":"2024-09-12T03:22:50.872182+0000","last_fullsized":"2024-09-12T03:22:50.872182+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_clean_scrub_stamp":"2024-09-12T03:22:49.828625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:11:12.062704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.16","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-12T03:22:50.848906+0000","last_change":"2024-09-12T03:22:49.854389+0000","last_active":"2024-09-12T03:22:50.848906+0000","last_peered":"2024-09-12T03:22:50.848906+0000","last_clean":"2024-09-12T03:22:50.848906+0000","last_became_active":"2024-09-12T03:22:49.854101+0000","last_became_peered":"2024-09-12T03:22:49.854101+0000","last_unstale":"2024-09-12T03:22:50.848906+0000","last_undegraded":"2024-09-12T03:22:50.848906+0000","last_fullsized":"2024-09-12T03:22:50.848906+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:48.826338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:48.826338+0000","last_clean_scrub_stamp":"2024-09-12T03:22:48.826338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:39:19.814553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.14","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-12T03:22:50.872172+0000","last_change":"2024-09-12T03:22:50.869496+0000","last_active":"2024-09-12T03:22:50.872172+0000","last_peered":"2024-09-12T03:22:50.872172+0000","last_clean":"2024-09-12T03:22:50.872172+0000","last_became_active":"2024-09-12T03:22:50.868978+0000","last_became_peered":"2024-09-12T03:22:50.868978+0000","last_unstale":"2024-09-12T03:22:50.872172+0000","last_undegraded":"2024-09-12T03:22:50.872172+0000","last_fullsized":"2024-09-12T03:22:50.872172+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_clean_scrub_stamp":"2024-09-12T03:22:49.828625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:48:13.106684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.15","version":"137'3","reported_seq":15,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-12T03:22:50.867600+0000","last_change":"2024-09-12T03:22:49.859597+0000","last_active":"2024-09-12T03:22:50.867600+0000","last_peered":"2024-09-12T03:22:50.867600+0000","last_clean":"2024-09-12T03:22:50.867600+0000","last_became_active":"2024-09-12T03:22:49.859260+0000","last_became_peered":"2024-09-12T03:22:49.859260+0000","last_unstale":"2024-09-12T03:22:50.867600+0000","last_undegraded":"2024-09-12T03:22:50.867600+0000","last_fullsized":"2024-09-12T03:22:50.867600+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:48.826338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:48.826338+0000","last_clean_scrub_stamp":"2024-09-12T03:22:48.826338+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:41:45.248785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.15","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-12T03:22:50.850510+0000","last_change":"2024-09-12T03:22:50.848161+0000","last_active":"2024-09-12T03:22:50.850510+0000","last_peered":"2024-09-12T03:22:50.850510+0000","last_clean":"2024-09-12T03:22:50.850510+0000","last_became_active":"2024-09-12T03:22:50.847855+0000","last_became_peered":"2024-09-12T03:22:50.847855+0000","last_unstale":"2024-09-12T03:22:50.850510+0000","last_undegraded":"2024-09-12T03:22:50.850510+0000","last_fullsized":"2024-09-12T03:22:50.850510+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_clean_scrub_stamp":"2024-09-12T03:22:49.828625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:37:14.788266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.14","version":"137'6","reported_seq":18,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-12T03:22:50.871686+0000","last_change":"2024-09-12T03:22:49.853821+0000","last_active":"2024-09-12T03:22:50.871686+0000","last_peered":"2024-09-12T03:22:50.871686+0000","last_clean":"2024-09-12T03:22:50.871686+0000","last_became_active":"2024-09-12T03:22:49.853533+0000","last_became_peered":"2024-09-12T03:22:49.853533+0000","last_unstale":"2024-09-12T03:22:50.871686+0000","last_undegraded":"2024-09-12T03:22:50.871686+0000","last_fullsized":"2024-09-12T03:22:50.871686+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:48.826338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:48.826338+0000","last_clean_scrub_stamp":"2024-09-12T03:22:48.826338+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:39:32.338120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.12","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-12T03:22:50.862355+0000","last_change":"2024-09-12T03:22:50.858038+0000","last_active":"2024-09-12T03:22:50.862355+0000","last_peered":"2024-09-12T03:22:50.862355+0000","last_clean":"2024-09-12T03:22:50.862355+0000","last_became_active":"2024-09-12T03:22:50.857623+0000","last_became_peered":"2024-09-12T03:22:50.857623+0000","last_unstale":"2024-09-12T03:22:50.862355+0000","last_undegraded":"2024-09-12T03:22:50.862355+0000","last_fullsized":"2024-09-12T03:22:50.862355+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_clean_scrub_stamp":"2024-09-12T03:22:49.828625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:28:23.214979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.13","version":"137'2","reported_seq":14,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-12T03:22:50.868447+0000","last_change":"2024-09-12T03:22:49.854043+0000","last_active":"2024-09-12T03:22:50.868447+0000","last_peered":"2024-09-12T03:22:50.868447+0000","last_clean":"2024-09-12T03:22:50.868447+0000","last_became_active":"2024-09-12T03:22:49.853742+0000","last_became_peered":"2024-09-12T03:22:49.853742+0000","last_unstale":"2024-09-12T03:22:50.868447+0000","last_undegraded":"2024-09-12T03:22:50.868447+0000","last_fullsized":"2024-09-12T03:22:50.868447+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:48.826338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:48.826338+0000","last_clean_scrub_stamp":"2024-09-12T03:22:48.826338+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:18:31.622015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.13","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-12T03:22:50.856734+0000","last_change":"2024-09-12T03:22:50.855073+0000","last_active":"2024-09-12T03:22:50.856734+0000","last_peered":"2024-09-12T03:22:50.856734+0000","last_clean":"2024-09-12T03:22:50.856734+0000","last_became_active":"2024-09-12T03:22:50.854787+0000","last_became_peered":"2024-09-12T03:22:50.854787+0000","last_unstale":"2024-09-12T03:22:50.856734+0000","last_undegraded":"2024-09-12T03:22:50.856734+0000","last_fullsized":"2024-09-12T03:22:50.856734+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_clean_scrub_stamp":"2024-09-12T03:22:49.828625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:52:23.711059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.12","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-12T03:22:50.854241+0000","last_change":"2024-09-12T03:22:49.857897+0000","last_active":"2024-09-12T03:22:50.854241+0000","last_peered":"2024-09-12T03:22:50.854241+0000","last_clean":"2024-09-12T03:22:50.854241+0000","last_became_active":"2024-09-12T03:22:49.857602+0000","last_became_peered":"2024-09-12T03:22:49.857602+0000","last_unstale":"2024-09-12T03:22:50.854241+0000","last_undegraded":"2024-09-12T03:22:50.854241+0000","last_fullsized":"2024-09-12T03:22:50.854241+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:48.826338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:48.826338+0000","last_clean_scrub_stamp":"2024-09-12T03:22:48.826338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:20:31.945130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.10","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-12T03:22:50.865399+0000","last_change":"2024-09-12T03:22:50.856364+0000","last_active":"2024-09-12T03:22:50.865399+0000","last_peered":"2024-09-12T03:22:50.865399+0000","last_clean":"2024-09-12T03:22:50.865399+0000","last_became_active":"2024-09-12T03:22:50.855657+0000","last_became_peered":"2024-09-12T03:22:50.855657+0000","last_unstale":"2024-09-12T03:22:50.865399+0000","last_undegraded":"2024-09-12T03:22:50.865399+0000","last_fullsized":"2024-09-12T03:22:50.865399+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_clean_scrub_stamp":"2024-09-12T03:22:49.828625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:52:16.974059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.11","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-12T03:22:50.840846+0000","last_change":"2024-09-12T03:22:49.849723+0000","last_active":"2024-09-12T03:22:50.840846+0000","last_peered":"2024-09-12T03:22:50.840846+0000","last_clean":"2024-09-12T03:22:50.840846+0000","last_became_active":"2024-09-12T03:22:49.849423+0000","last_became_peered":"2024-09-12T03:22:49.849423+0000","last_unstale":"2024-09-12T03:22:50.840846+0000","last_undegraded":"2024-09-12T03:22:50.840846+0000","last_fullsized":"2024-09-12T03:22:50.840846+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:48.826338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:48.826338+0000","last_clean_scrub_stamp":"2024-09-12T03:22:48.826338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:19:43.037310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.11","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-12T03:22:50.864443+0000","last_change":"2024-09-12T03:22:50.855109+0000","last_active":"2024-09-12T03:22:50.864443+0000","last_peered":"2024-09-12T03:22:50.864443+0000","last_clean":"2024-09-12T03:22:50.864443+0000","last_became_active":"2024-09-12T03:22:50.854795+0000","last_became_peered":"2024-09-12T03:22:50.854795+0000","last_unstale":"2024-09-12T03:22:50.864443+0000","last_undegraded":"2024-09-12T03:22:50.864443+0000","last_fullsized":"2024-09-12T03:22:50.864443+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_clean_scrub_stamp":"2024-09-12T03:22:49.828625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:51:36.386947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.10","version":"137'8","reported_seq":20,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-12T03:22:50.869819+0000","last_change":"2024-09-12T03:22:49.854880+0000","last_active":"2024-09-12T03:22:50.869819+0000","last_peered":"2024-09-12T03:22:50.869819+0000","last_clean":"2024-09-12T03:22:50.869819+0000","last_became_active":"2024-09-12T03:22:49.854167+0000","last_became_peered":"2024-09-12T03:22:49.854167+0000","last_unstale":"2024-09-12T03:22:50.869819+0000","last_undegraded":"2024-09-12T03:22:50.869819+0000","last_fullsized":"2024-09-12T03:22:50.869819+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:48.826338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:48.826338+0000","last_clean_scrub_stamp":"2024-09-12T03:22:48.826338+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:03:12.236893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.e","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-12T03:22:50.865674+0000","last_change":"2024-09-12T03:22:50.857848+0000","last_active":"2024-09-12T03:22:50.865674+0000","last_peered":"2024-09-12T03:22:50.865674+0000","last_clean":"2024-09-12T03:22:50.865674+0000","last_became_active":"2024-09-12T03:22:50.857348+0000","last_became_peered":"2024-09-12T03:22:50.857348+0000","last_unstale":"2024-09-12T03:22:50.865674+0000","last_undegraded":"2024-09-12T03:22:50.865674+0000","last_fullsized":"2024-09-12T03:22:50.865674+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_clean_scrub_stamp":"2024-09-12T03:22:49.828625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:47:48.004911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.f","version":"137'1","reported_seq":13,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-12T03:22:50.870317+0000","last_change":"2024-09-12T03:22:49.851789+0000","last_active":"2024-09-12T03:22:50.870317+0000","last_peered":"2024-09-12T03:22:50.870317+0000","last_clean":"2024-09-12T03:22:50.870317+0000","last_became_active":"2024-09-12T03:22:49.851432+0000","last_became_peered":"2024-09-12T03:22:49.851432+0000","last_unstale":"2024-09-12T03:22:50.870317+0000","last_undegraded":"2024-09-12T03:22:50.870317+0000","last_fullsized":"2024-09-12T03:22:50.870317+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:48.826338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:48.826338+0000","last_clean_scrub_stamp":"2024-09-12T03:22:48.826338+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:46:01.538002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.f","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-12T03:22:50.874371+0000","last_change":"2024-09-12T03:22:50.870855+0000","last_active":"2024-09-12T03:22:50.874371+0000","last_peered":"2024-09-12T03:22:50.874371+0000","last_clean":"2024-09-12T03:22:50.874371+0000","last_became_active":"2024-09-12T03:22:50.868491+0000","last_became_peered":"2024-09-12T03:22:50.868491+0000","last_unstale":"2024-09-12T03:22:50.874371+0000","last_undegraded":"2024-09-12T03:22:50.874371+0000","last_fullsized":"2024-09-12T03:22:50.874371+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_clean_scrub_stamp":"2024-09-12T03:22:49.828625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:45:10.451298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.e","version":"137'2","reported_seq":14,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-12T03:22:50.870727+0000","last_change":"2024-09-12T03:22:49.852706+0000","last_active":"2024-09-12T03:22:50.870727+0000","last_peered":"2024-09-12T03:22:50.870727+0000","last_clean":"2024-09-12T03:22:50.870727+0000","last_became_active":"2024-09-12T03:22:49.852423+0000","last_became_peered":"2024-09-12T03:22:49.852423+0000","last_unstale":"2024-09-12T03:22:50.870727+0000","last_undegraded":"2024-09-12T03:22:50.870727+0000","last_fullsized":"2024-09-12T03:22:50.870727+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:48.826338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:48.826338+0000","last_clean_scrub_stamp":"2024-09-12T03:22:48.826338+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:26:33.756735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.c","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-12T03:22:50.868541+0000","last_change":"2024-09-12T03:22:50.864312+0000","last_active":"2024-09-12T03:22:50.868541+0000","last_peered":"2024-09-12T03:22:50.868541+0000","last_clean":"2024-09-12T03:22:50.868541+0000","last_became_active":"2024-09-12T03:22:50.864053+0000","last_became_peered":"2024-09-12T03:22:50.864053+0000","last_unstale":"2024-09-12T03:22:50.868541+0000","last_undegraded":"2024-09-12T03:22:50.868541+0000","last_fullsized":"2024-09-12T03:22:50.868541+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_clean_scrub_stamp":"2024-09-12T03:22:49.828625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:52:51.383350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.d","version":"137'1","reported_seq":13,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-12T03:22:50.870339+0000","last_change":"2024-09-12T03:22:49.853673+0000","last_active":"2024-09-12T03:22:50.870339+0000","last_peered":"2024-09-12T03:22:50.870339+0000","last_clean":"2024-09-12T03:22:50.870339+0000","last_became_active":"2024-09-12T03:22:49.853374+0000","last_became_peered":"2024-09-12T03:22:49.853374+0000","last_unstale":"2024-09-12T03:22:50.870339+0000","last_undegraded":"2024-09-12T03:22:50.870339+0000","last_fullsized":"2024-09-12T03:22:50.870339+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:48.826338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:48.826338+0000","last_clean_scrub_stamp":"2024-09-12T03:22:48.826338+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:45:05.684426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.d","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-12T03:22:50.864945+0000","last_change":"2024-09-12T03:22:50.862005+0000","last_active":"2024-09-12T03:22:50.864945+0000","last_peered":"2024-09-12T03:22:50.864945+0000","last_clean":"2024-09-12T03:22:50.864945+0000","last_became_active":"2024-09-12T03:22:50.861682+0000","last_became_peered":"2024-09-12T03:22:50.861682+0000","last_unstale":"2024-09-12T03:22:50.864945+0000","last_undegraded":"2024-09-12T03:22:50.864945+0000","last_fullsized":"2024-09-12T03:22:50.864945+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_clean_scrub_stamp":"2024-09-12T03:22:49.828625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:47:18.184259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.c","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-12T03:22:50.847289+0000","last_change":"2024-09-12T03:22:49.857153+0000","last_active":"2024-09-12T03:22:50.847289+0000","last_peered":"2024-09-12T03:22:50.847289+0000","last_clean":"2024-09-12T03:22:50.847289+0000","last_became_active":"2024-09-12T03:22:49.856827+0000","last_became_peered":"2024-09-12T03:22:49.856827+0000","last_unstale":"2024-09-12T03:22:50.847289+0000","last_undegraded":"2024-09-12T03:22:50.847289+0000","last_fullsized":"2024-09-12T03:22:50.847289+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:48.826338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:48.826338+0000","last_clean_scrub_stamp":"2024-09-12T03:22:48.826338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:54:45.693003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.a","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-12T03:22:50.864273+0000","last_change":"2024-09-12T03:22:50.860128+0000","last_active":"2024-09-12T03:22:50.864273+0000","last_peered":"2024-09-12T03:22:50.864273+0000","last_clean":"2024-09-12T03:22:50.864273+0000","last_became_active":"2024-09-12T03:22:50.859864+0000","last_became_peered":"2024-09-12T03:22:50.859864+0000","last_unstale":"2024-09-12T03:22:50.864273+0000","last_undegraded":"2024-09-12T03:22:50.864273+0000","last_fullsized":"2024-09-12T03:22:50.864273+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_clean_scrub_stamp":"2024-09-12T03:22:49.828625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:46:46.620260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.b","version":"137'2","reported_seq":14,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-12T03:22:50.871609+0000","last_change":"2024-09-12T03:22:49.855989+0000","last_active":"2024-09-12T03:22:50.871609+0000","last_peered":"2024-09-12T03:22:50.871609+0000","last_clean":"2024-09-12T03:22:50.871609+0000","last_became_active":"2024-09-12T03:22:49.854000+0000","last_became_peered":"2024-09-12T03:22:49.854000+0000","last_unstale":"2024-09-12T03:22:50.871609+0000","last_undegraded":"2024-09-12T03:22:50.871609+0000","last_fullsized":"2024-09-12T03:22:50.871609+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:48.826338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:48.826338+0000","last_clean_scrub_stamp":"2024-09-12T03:22:48.826338+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:33:38.632902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.b","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-12T03:22:50.855474+0000","last_change":"2024-09-12T03:22:50.853807+0000","last_active":"2024-09-12T03:22:50.855474+0000","last_peered":"2024-09-12T03:22:50.855474+0000","last_clean":"2024-09-12T03:22:50.855474+0000","last_became_active":"2024-09-12T03:22:50.853488+0000","last_became_peered":"2024-09-12T03:22:50.853488+0000","last_unstale":"2024-09-12T03:22:50.855474+0000","last_undegraded":"2024-09-12T03:22:50.855474+0000","last_fullsized":"2024-09-12T03:22:50.855474+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_clean_scrub_stamp":"2024-09-12T03:22:49.828625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:35:10.105847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.a","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-12T03:22:50.845726+0000","last_change":"2024-09-12T03:22:49.858817+0000","last_active":"2024-09-12T03:22:50.845726+0000","last_peered":"2024-09-12T03:22:50.845726+0000","last_clean":"2024-09-12T03:22:50.845726+0000","last_became_active":"2024-09-12T03:22:49.858312+0000","last_became_peered":"2024-09-12T03:22:49.858312+0000","last_unstale":"2024-09-12T03:22:50.845726+0000","last_undegraded":"2024-09-12T03:22:50.845726+0000","last_fullsized":"2024-09-12T03:22:50.845726+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:48.826338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:48.826338+0000","last_clean_scrub_stamp":"2024-09-12T03:22:48.826338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:53:33.732611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.8","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-12T03:22:50.858300+0000","last_change":"2024-09-12T03:22:50.856665+0000","last_active":"2024-09-12T03:22:50.858300+0000","last_peered":"2024-09-12T03:22:50.858300+0000","last_clean":"2024-09-12T03:22:50.858300+0000","last_became_active":"2024-09-12T03:22:50.856366+0000","last_became_peered":"2024-09-12T03:22:50.856366+0000","last_unstale":"2024-09-12T03:22:50.858300+0000","last_undegraded":"2024-09-12T03:22:50.858300+0000","last_fullsized":"2024-09-12T03:22:50.858300+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_clean_scrub_stamp":"2024-09-12T03:22:49.828625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:33:47.516847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.9","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-12T03:22:50.840896+0000","last_change":"2024-09-12T03:22:49.846978+0000","last_active":"2024-09-12T03:22:50.840896+0000","last_peered":"2024-09-12T03:22:50.840896+0000","last_clean":"2024-09-12T03:22:50.840896+0000","last_became_active":"2024-09-12T03:22:49.846663+0000","last_became_peered":"2024-09-12T03:22:49.846663+0000","last_unstale":"2024-09-12T03:22:50.840896+0000","last_undegraded":"2024-09-12T03:22:50.840896+0000","last_fullsized":"2024-09-12T03:22:50.840896+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:48.826338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:48.826338+0000","last_clean_scrub_stamp":"2024-09-12T03:22:48.826338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:53:23.453570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.9","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-12T03:22:50.862586+0000","last_change":"2024-09-12T03:22:50.857545+0000","last_active":"2024-09-12T03:22:50.862586+0000","last_peered":"2024-09-12T03:22:50.862586+0000","last_clean":"2024-09-12T03:22:50.862586+0000","last_became_active":"2024-09-12T03:22:50.857249+0000","last_became_peered":"2024-09-12T03:22:50.857249+0000","last_unstale":"2024-09-12T03:22:50.862586+0000","last_undegraded":"2024-09-12T03:22:50.862586+0000","last_fullsized":"2024-09-12T03:22:50.862586+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_clean_scrub_stamp":"2024-09-12T03:22:49.828625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:55:57.520589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.8","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-12T03:22:50.847002+0000","last_change":"2024-09-12T03:22:49.853736+0000","last_active":"2024-09-12T03:22:50.847002+0000","last_peered":"2024-09-12T03:22:50.847002+0000","last_clean":"2024-09-12T03:22:50.847002+0000","last_became_active":"2024-09-12T03:22:49.853428+0000","last_became_peered":"2024-09-12T03:22:49.853428+0000","last_unstale":"2024-09-12T03:22:50.847002+0000","last_undegraded":"2024-09-12T03:22:50.847002+0000","last_fullsized":"2024-09-12T03:22:50.847002+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:48.826338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:48.826338+0000","last_clean_scrub_stamp":"2024-09-12T03:22:48.826338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:41:47.248620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.6","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-12T03:22:50.872201+0000","last_change":"2024-09-12T03:22:50.869488+0000","last_active":"2024-09-12T03:22:50.872201+0000","last_peered":"2024-09-12T03:22:50.872201+0000","last_clean":"2024-09-12T03:22:50.872201+0000","last_became_active":"2024-09-12T03:22:50.869036+0000","last_became_peered":"2024-09-12T03:22:50.869036+0000","last_unstale":"2024-09-12T03:22:50.872201+0000","last_undegraded":"2024-09-12T03:22:50.872201+0000","last_fullsized":"2024-09-12T03:22:50.872201+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_clean_scrub_stamp":"2024-09-12T03:22:49.828625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:21:49.881460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.7","version":"137'2","reported_seq":14,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-12T03:22:50.865954+0000","last_change":"2024-09-12T03:22:49.855553+0000","last_active":"2024-09-12T03:22:50.865954+0000","last_peered":"2024-09-12T03:22:50.865954+0000","last_clean":"2024-09-12T03:22:50.865954+0000","last_became_active":"2024-09-12T03:22:49.855277+0000","last_became_peered":"2024-09-12T03:22:49.855277+0000","last_unstale":"2024-09-12T03:22:50.865954+0000","last_undegraded":"2024-09-12T03:22:50.865954+0000","last_fullsized":"2024-09-12T03:22:50.865954+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:48.826338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:48.826338+0000","last_clean_scrub_stamp":"2024-09-12T03:22:48.826338+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:39:32.338120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.7","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-12T03:22:50.856662+0000","last_change":"2024-09-12T03:22:50.855109+0000","last_active":"2024-09-12T03:22:50.856662+0000","last_peered":"2024-09-12T03:22:50.856662+0000","last_clean":"2024-09-12T03:22:50.856662+0000","last_became_active":"2024-09-12T03:22:50.854826+0000","last_became_peered":"2024-09-12T03:22:50.854826+0000","last_unstale":"2024-09-12T03:22:50.856662+0000","last_undegraded":"2024-09-12T03:22:50.856662+0000","last_fullsized":"2024-09-12T03:22:50.856662+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_clean_scrub_stamp":"2024-09-12T03:22:49.828625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:50:31.028630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.6","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-12T03:22:50.842903+0000","last_change":"2024-09-12T03:22:49.849266+0000","last_active":"2024-09-12T03:22:50.842903+0000","last_peered":"2024-09-12T03:22:50.842903+0000","last_clean":"2024-09-12T03:22:50.842903+0000","last_became_active":"2024-09-12T03:22:49.848896+0000","last_became_peered":"2024-09-12T03:22:49.848896+0000","last_unstale":"2024-09-12T03:22:50.842903+0000","last_undegraded":"2024-09-12T03:22:50.842903+0000","last_fullsized":"2024-09-12T03:22:50.842903+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:48.826338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:48.826338+0000","last_clean_scrub_stamp":"2024-09-12T03:22:48.826338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:19:38.538075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.4","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-12T03:22:50.870596+0000","last_change":"2024-09-12T03:22:50.869131+0000","last_active":"2024-09-12T03:22:50.870596+0000","last_peered":"2024-09-12T03:22:50.870596+0000","last_clean":"2024-09-12T03:22:50.870596+0000","last_became_active":"2024-09-12T03:22:50.866882+0000","last_became_peered":"2024-09-12T03:22:50.866882+0000","last_unstale":"2024-09-12T03:22:50.870596+0000","last_undegraded":"2024-09-12T03:22:50.870596+0000","last_fullsized":"2024-09-12T03:22:50.870596+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_clean_scrub_stamp":"2024-09-12T03:22:49.828625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:43:34.746247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.5","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-12T03:22:50.849598+0000","last_change":"2024-09-12T03:22:49.858237+0000","last_active":"2024-09-12T03:22:50.849598+0000","last_peered":"2024-09-12T03:22:50.849598+0000","last_clean":"2024-09-12T03:22:50.849598+0000","last_became_active":"2024-09-12T03:22:49.857571+0000","last_became_peered":"2024-09-12T03:22:49.857571+0000","last_unstale":"2024-09-12T03:22:50.849598+0000","last_undegraded":"2024-09-12T03:22:50.849598+0000","last_fullsized":"2024-09-12T03:22:50.849598+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:48.826338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:48.826338+0000","last_clean_scrub_stamp":"2024-09-12T03:22:48.826338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:24:58.895995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.5","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-12T03:22:50.861212+0000","last_change":"2024-09-12T03:22:50.856590+0000","last_active":"2024-09-12T03:22:50.861212+0000","last_peered":"2024-09-12T03:22:50.861212+0000","last_clean":"2024-09-12T03:22:50.861212+0000","last_became_active":"2024-09-12T03:22:50.856285+0000","last_became_peered":"2024-09-12T03:22:50.856285+0000","last_unstale":"2024-09-12T03:22:50.861212+0000","last_undegraded":"2024-09-12T03:22:50.861212+0000","last_fullsized":"2024-09-12T03:22:50.861212+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_clean_scrub_stamp":"2024-09-12T03:22:49.828625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:32:28.670636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.4","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-12T03:22:50.846769+0000","last_change":"2024-09-12T03:22:49.852512+0000","last_active":"2024-09-12T03:22:50.846769+0000","last_peered":"2024-09-12T03:22:50.846769+0000","last_clean":"2024-09-12T03:22:50.846769+0000","last_became_active":"2024-09-12T03:22:49.852237+0000","last_became_peered":"2024-09-12T03:22:49.852237+0000","last_unstale":"2024-09-12T03:22:50.846769+0000","last_undegraded":"2024-09-12T03:22:50.846769+0000","last_fullsized":"2024-09-12T03:22:50.846769+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:48.826338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:48.826338+0000","last_clean_scrub_stamp":"2024-09-12T03:22:48.826338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:40:44.909508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.2","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-12T03:22:50.864509+0000","last_change":"2024-09-12T03:22:50.862870+0000","last_active":"2024-09-12T03:22:50.864509+0000","last_peered":"2024-09-12T03:22:50.864509+0000","last_clean":"2024-09-12T03:22:50.864509+0000","last_became_active":"2024-09-12T03:22:50.862581+0000","last_became_peered":"2024-09-12T03:22:50.862581+0000","last_unstale":"2024-09-12T03:22:50.864509+0000","last_undegraded":"2024-09-12T03:22:50.864509+0000","last_fullsized":"2024-09-12T03:22:50.864509+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_clean_scrub_stamp":"2024-09-12T03:22:49.828625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:37:14.788266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.3","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-12T03:22:50.844592+0000","last_change":"2024-09-12T03:22:49.857358+0000","last_active":"2024-09-12T03:22:50.844592+0000","last_peered":"2024-09-12T03:22:50.844592+0000","last_clean":"2024-09-12T03:22:50.844592+0000","last_became_active":"2024-09-12T03:22:49.857043+0000","last_became_peered":"2024-09-12T03:22:49.857043+0000","last_unstale":"2024-09-12T03:22:50.844592+0000","last_undegraded":"2024-09-12T03:22:50.844592+0000","last_fullsized":"2024-09-12T03:22:50.844592+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:48.826338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:48.826338+0000","last_clean_scrub_stamp":"2024-09-12T03:22:48.826338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:28:22.212691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.3","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-12T03:22:50.867025+0000","last_change":"2024-09-12T03:22:50.861882+0000","last_active":"2024-09-12T03:22:50.867025+0000","last_peered":"2024-09-12T03:22:50.867025+0000","last_clean":"2024-09-12T03:22:50.867025+0000","last_became_active":"2024-09-12T03:22:50.860858+0000","last_became_peered":"2024-09-12T03:22:50.860858+0000","last_unstale":"2024-09-12T03:22:50.867025+0000","last_undegraded":"2024-09-12T03:22:50.867025+0000","last_fullsized":"2024-09-12T03:22:50.867025+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_clean_scrub_stamp":"2024-09-12T03:22:49.828625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:48:13.106684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.2","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-12T03:22:50.841451+0000","last_change":"2024-09-12T03:22:49.850743+0000","last_active":"2024-09-12T03:22:50.841451+0000","last_peered":"2024-09-12T03:22:50.841451+0000","last_clean":"2024-09-12T03:22:50.841451+0000","last_became_active":"2024-09-12T03:22:49.850370+0000","last_became_peered":"2024-09-12T03:22:49.850370+0000","last_unstale":"2024-09-12T03:22:50.841451+0000","last_undegraded":"2024-09-12T03:22:50.841451+0000","last_fullsized":"2024-09-12T03:22:50.841451+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:48.826338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:48.826338+0000","last_clean_scrub_stamp":"2024-09-12T03:22:48.826338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:05:15.266590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.0","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-12T03:22:50.855513+0000","last_change":"2024-09-12T03:22:50.853289+0000","last_active":"2024-09-12T03:22:50.855513+0000","last_peered":"2024-09-12T03:22:50.855513+0000","last_clean":"2024-09-12T03:22:50.855513+0000","last_became_active":"2024-09-12T03:22:50.852934+0000","last_became_peered":"2024-09-12T03:22:50.852934+0000","last_unstale":"2024-09-12T03:22:50.855513+0000","last_undegraded":"2024-09-12T03:22:50.855513+0000","last_fullsized":"2024-09-12T03:22:50.855513+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_clean_scrub_stamp":"2024-09-12T03:22:49.828625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:33:47.516847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-12T03:22:50.847299+0000","last_change":"2024-09-12T03:22:49.856653+0000","last_active":"2024-09-12T03:22:50.847299+0000","last_peered":"2024-09-12T03:22:50.847299+0000","last_clean":"2024-09-12T03:22:50.847299+0000","last_became_active":"2024-09-12T03:22:49.856359+0000","last_became_peered":"2024-09-12T03:22:49.856359+0000","last_unstale":"2024-09-12T03:22:50.847299+0000","last_undegraded":"2024-09-12T03:22:50.847299+0000","last_fullsized":"2024-09-12T03:22:50.847299+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:48.826338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:48.826338+0000","last_clean_scrub_stamp":"2024-09-12T03:22:48.826338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:38:42.593399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.1","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-12T03:22:50.864471+0000","last_change":"2024-09-12T03:22:50.857986+0000","last_active":"2024-09-12T03:22:50.864471+0000","last_peered":"2024-09-12T03:22:50.864471+0000","last_clean":"2024-09-12T03:22:50.864471+0000","last_became_active":"2024-09-12T03:22:50.857674+0000","last_became_peered":"2024-09-12T03:22:50.857674+0000","last_unstale":"2024-09-12T03:22:50.864471+0000","last_undegraded":"2024-09-12T03:22:50.864471+0000","last_fullsized":"2024-09-12T03:22:50.864471+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_clean_scrub_stamp":"2024-09-12T03:22:49.828625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:24:51.460112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.0","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-12T03:22:50.843982+0000","last_change":"2024-09-12T03:22:49.855102+0000","last_active":"2024-09-12T03:22:50.843982+0000","last_peered":"2024-09-12T03:22:50.843982+0000","last_clean":"2024-09-12T03:22:50.843982+0000","last_became_active":"2024-09-12T03:22:49.854616+0000","last_became_peered":"2024-09-12T03:22:49.854616+0000","last_unstale":"2024-09-12T03:22:50.843982+0000","last_undegraded":"2024-09-12T03:22:50.843982+0000","last_fullsized":"2024-09-12T03:22:50.843982+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:48.826338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:48.826338+0000","last_clean_scrub_stamp":"2024-09-12T03:22:48.826338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:40:44.909508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.16","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-12T03:22:50.872223+0000","last_change":"2024-09-12T03:22:50.869792+0000","last_active":"2024-09-12T03:22:50.872223+0000","last_peered":"2024-09-12T03:22:50.872223+0000","last_clean":"2024-09-12T03:22:50.872223+0000","last_became_active":"2024-09-12T03:22:50.869286+0000","last_became_peered":"2024-09-12T03:22:50.869286+0000","last_unstale":"2024-09-12T03:22:50.872223+0000","last_undegraded":"2024-09-12T03:22:50.872223+0000","last_fullsized":"2024-09-12T03:22:50.872223+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_clean_scrub_stamp":"2024-09-12T03:22:49.828625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:10:41.417435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.17","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-12T03:22:50.844442+0000","last_change":"2024-09-12T03:22:49.856157+0000","last_active":"2024-09-12T03:22:50.844442+0000","last_peered":"2024-09-12T03:22:50.844442+0000","last_clean":"2024-09-12T03:22:50.844442+0000","last_became_active":"2024-09-12T03:22:49.855792+0000","last_became_peered":"2024-09-12T03:22:49.855792+0000","last_unstale":"2024-09-12T03:22:50.844442+0000","last_undegraded":"2024-09-12T03:22:50.844442+0000","last_fullsized":"2024-09-12T03:22:50.844442+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:48.826338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:48.826338+0000","last_clean_scrub_stamp":"2024-09-12T03:22:48.826338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:32:27.668349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"70'128","reported_seq":422,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-12T03:22:50.847095+0000","last_change":"2024-09-12T03:16:21.883446+0000","last_active":"2024-09-12T03:22:50.847095+0000","last_peered":"2024-09-12T03:22:50.847095+0000","last_clean":"2024-09-12T03:22:50.847095+0000","last_became_active":"2024-09-12T03:16:21.882207+0000","last_became_peered":"2024-09-12T03:16:21.882207+0000","last_unstale":"2024-09-12T03:22:50.847095+0000","last_undegraded":"2024-09-12T03:22:50.847095+0000","last_fullsized":"2024-09-12T03:22:50.847095+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:20.316426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:20.316426+0000","last_clean_scrub_stamp":"2024-09-12T03:16:20.316426+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:12:15.424320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":23,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":22,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542224,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37600,"kb_used_data":2248,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334240,"statfs":{"total":96636764160,"available":96598261760,"internally_reserved":0,"allocated":2301952,"data_stored":1343949,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542226,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37036,"kb_used_data":1684,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334804,"statfs":{"total":96636764160,"available":96598839296,"internally_reserved":0,"allocated":1724416,"data_stored":760434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542225,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37020,"kb_used_data":1668,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334820,"statfs":{"total":96636764160,"available":96598855680,"internally_reserved":0,"allocated":1708032,"data_stored":753581,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542224,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37020,"kb_used_data":1668,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334820,"statfs":{"total":96636764160,"available":96598855680,"internally_reserved":0,"allocated":1708032,"data_stored":753581,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542225,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32872,"kb_used_data":1684,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338968,"statfs":{"total":96636764160,"available":96603103232,"internally_reserved":0,"allocated":1724416,"data_stored":760446,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542225,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32860,"kb_used_data":1668,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338980,"statfs":{"total":96636764160,"available":96603115520,"internally_reserved":0,"allocated":1708032,"data_stored":753581,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542226,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32872,"kb_used_data":1684,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338968,"statfs":{"total":96636764160,"available":96603103232,"internally_reserved":0,"allocated":1724416,"data_stored":760446,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542226,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33444,"kb_used_data":2260,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338396,"statfs":{"total":96636764160,"available":96602517504,"internally_reserved":0,"allocated":2314240,"data_stored":1350780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T03:22:53.834 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-12T03:22:53.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.972+0000 7f822d1c4640 1 Processor -- start 2024-09-12T03:22:53.973 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.972+0000 7f822d1c4640 1 -- start start 2024-09-12T03:22:53.974 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.973+0000 7f822d1c4640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f822815c2e0 0x7f82280cf070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:53.974 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.973+0000 7f822d1c4640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f8228073920 0x7f82280b5bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:53.974 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.973+0000 7f822d1c4640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f82280cf8a0 con 0x7f822815c6c0 2024-09-12T03:22:53.974 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.973+0000 7f822d1c4640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f8228160eb0 con 0x7f8228073920 2024-09-12T03:22:53.974 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.973+0000 7f822d1c4640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f82281600f0 con 0x7f822815c2e0 2024-09-12T03:22:53.974 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.974+0000 7f822759e640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f8228073920 0x7f82280b5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:53.974 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.974+0000 7f8226d9d640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f822815c2e0 0x7f82280cf070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:53.974 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.974+0000 7f822659c640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f822815c6c0 0x7f8228073210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.27:49160/0 (socket says 172.21.15.27:49160) 2024-09-12T03:22:53.974 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.974+0000 7f8226d9d640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f822815c2e0 0x7f82280cf070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3301/0 says I am v2:172.21.15.27:59972/0 (socket says 172.21.15.27:59972) 2024-09-12T03:22:53.974 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.974+0000 7f822759e640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f8228073920 0x7f82280b5bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.27:36532/0 (socket says 172.21.15.27:36532) 2024-09-12T03:22:53.975 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.974+0000 7f822659c640 1 -- 172.21.15.27:0/747735881 learned_addr learned my addr 172.21.15.27:0/747735881 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:22:53.975 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.974+0000 7f822759e640 1 -- 172.21.15.27:0/747735881 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f822815c2e0 msgr2=0x7f82280cf070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:53.975 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.974+0000 7f822759e640 1 --2- 172.21.15.27:0/747735881 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f822815c2e0 0x7f82280cf070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:53.975 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.974+0000 7f822759e640 1 -- 172.21.15.27:0/747735881 >> v1:172.21.15.45:6789/0 conn(0x7f822815c6c0 legacy=0x7f8228073210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:53.975 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.974+0000 7f822759e640 1 -- 172.21.15.27:0/747735881 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82281603d0 con 0x7f8228073920 2024-09-12T03:22:53.975 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.974+0000 7f8226d9d640 1 --2- 172.21.15.27:0/747735881 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f822815c2e0 0x7f82280cf070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:22:53.976 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.975+0000 7f822759e640 1 --2- 172.21.15.27:0/747735881 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8228073920 0x7f82280b5bc0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f82180097a0 tx=0x7f821802a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d970d52817b48517 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:53.976 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.975+0000 7f8225d9b640 1 -- 172.21.15.27:0/747735881 <== mon.1 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8218037070 con 0x7f8228073920 2024-09-12T03:22:53.976 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.975+0000 7f8225d9b640 1 -- 172.21.15.27:0/747735881 <== mon.1 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f821802ae50 con 0x7f8228073920 2024-09-12T03:22:53.976 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.975+0000 7f8225d9b640 1 -- 172.21.15.27:0/747735881 <== mon.1 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8218032660 con 0x7f8228073920 2024-09-12T03:22:53.976 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.975+0000 7f822d1c4640 1 -- 172.21.15.27:0/747735881 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8228073920 msgr2=0x7f82280b5bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:53.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.975+0000 7f822d1c4640 1 --2- 172.21.15.27:0/747735881 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8228073920 0x7f82280b5bc0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f82180097a0 tx=0x7f821802a100 comp rx=0 tx=0).stop 2024-09-12T03:22:53.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.976+0000 7f822d1c4640 1 -- 172.21.15.27:0/747735881 shutdown_connections 2024-09-12T03:22:53.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.976+0000 7f822d1c4640 1 --2- 172.21.15.27:0/747735881 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8228073920 0x7f82280b5bc0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:53.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.976+0000 7f822d1c4640 1 --2- 172.21.15.27:0/747735881 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f822815c2e0 0x7f82280cf070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:53.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.976+0000 7f822d1c4640 1 -- 172.21.15.27:0/747735881 >> 172.21.15.27:0/747735881 conn(0x7f822806f2c0 msgr2=0x7f822806f6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:53.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.976+0000 7f822d1c4640 1 -- 172.21.15.27:0/747735881 shutdown_connections 2024-09-12T03:22:53.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.976+0000 7f822d1c4640 1 -- 172.21.15.27:0/747735881 wait complete. 2024-09-12T03:22:53.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.977+0000 7f822d1c4640 1 Processor -- start 2024-09-12T03:22:53.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.977+0000 7f822d1c4640 1 -- start start 2024-09-12T03:22:53.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.978+0000 7f822d1c4640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8228073920 0x7f82281557f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:53.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.978+0000 7f822d1c4640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f822815c2e0 0x7f8228151940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:53.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.978+0000 7f822d1c4640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f822815c6c0 0x7f82281e60a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:53.980 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.978+0000 7f822d1c4640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f82280cfaf0 con 0x7f822815c6c0 2024-09-12T03:22:53.980 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.978+0000 7f822d1c4640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f82280cf5b0 con 0x7f8228073920 2024-09-12T03:22:53.980 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.978+0000 7f822d1c4640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f82280cfc70 con 0x7f822815c2e0 2024-09-12T03:22:53.980 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.978+0000 7f822759e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f822815c6c0 0x7f82281e60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:53.980 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.978+0000 7f822759e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f822815c6c0 0x7f82281e60a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56284/0 (socket says 172.21.15.27:56284) 2024-09-12T03:22:53.980 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.978+0000 7f822759e640 1 -- 172.21.15.27:0/3230676670 learned_addr learned my addr 172.21.15.27:0/3230676670 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:22:53.980 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.978+0000 7f822659c640 1 --2- 172.21.15.27:0/3230676670 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f822815c2e0 0x7f8228151940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:53.980 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.978+0000 7f8226d9d640 1 --2- 172.21.15.27:0/3230676670 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8228073920 0x7f82281557f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:53.980 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.978+0000 7f822759e640 1 -- 172.21.15.27:0/3230676670 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f822815c2e0 msgr2=0x7f8228151940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:53.980 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.978+0000 7f822759e640 1 --2- 172.21.15.27:0/3230676670 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f822815c2e0 0x7f8228151940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:53.980 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.978+0000 7f822759e640 1 -- 172.21.15.27:0/3230676670 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8228073920 msgr2=0x7f82281557f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:53.980 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.978+0000 7f822759e640 1 --2- 172.21.15.27:0/3230676670 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8228073920 0x7f82281557f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:53.981 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.978+0000 7f822759e640 1 -- 172.21.15.27:0/3230676670 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82280af560 con 0x7f822815c6c0 2024-09-12T03:22:53.981 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.979+0000 7f822759e640 1 --2- 172.21.15.27:0/3230676670 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f822815c6c0 0x7f82281e60a0 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f82180098c0 tx=0x7f82180046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:53.981 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.979+0000 7f821f7fe640 1 -- 172.21.15.27:0/3230676670 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8218041020 con 0x7f822815c6c0 2024-09-12T03:22:53.981 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.979+0000 7f821f7fe640 1 -- 172.21.15.27:0/3230676670 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f821803c670 con 0x7f822815c6c0 2024-09-12T03:22:53.981 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.979+0000 7f822d1c4640 1 -- 172.21.15.27:0/3230676670 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82280af840 con 0x7f822815c6c0 2024-09-12T03:22:53.981 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.979+0000 7f821f7fe640 1 -- 172.21.15.27:0/3230676670 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8218037040 con 0x7f822815c6c0 2024-09-12T03:22:53.981 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.979+0000 7f822d1c4640 1 -- 172.21.15.27:0/3230676670 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82280afe00 con 0x7f822815c6c0 2024-09-12T03:22:53.982 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.980+0000 7f822d1c4640 1 -- 172.21.15.27:0/3230676670 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82280712c0 con 0x7f822815c6c0 2024-09-12T03:22:53.982 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.980+0000 7f821f7fe640 1 -- 172.21.15.27:0/3230676670 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 22) ==== 91059+0+0 (secure 0 0 0) 0x7f8218044020 con 0x7f822815c6c0 2024-09-12T03:22:53.982 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.981+0000 7f821f7fe640 1 --2- 172.21.15.27:0/3230676670 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f820806c460 0x7f820806e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:53.982 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.981+0000 7f821f7fe640 1 -- 172.21.15.27:0/3230676670 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 5765+0+0 (secure 0 0 0) 0x7f82180ac010 con 0x7f822815c6c0 2024-09-12T03:22:53.982 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.981+0000 7f8226d9d640 1 --2- 172.21.15.27:0/3230676670 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f820806c460 0x7f820806e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:53.982 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.981+0000 7f8226d9d640 1 --2- 172.21.15.27:0/3230676670 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f820806c460 0x7f820806e920 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f82280cefc0 tx=0x7f822000b040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:53.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:53.985+0000 7f821f7fe640 1 -- 172.21.15.27:0/3230676670 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f821804b030 con 0x7f822815c6c0 2024-09-12T03:22:54.156 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:54.155+0000 7f822d1c4640 1 -- 172.21.15.27:0/3230676670 --> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8228152350 con 0x7f820806c460 2024-09-12T03:22:54.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:54.161+0000 7f821f7fe640 1 -- 172.21.15.27:0/3230676670 <== mgr.4113 v2:172.21.15.27:6836/1226920526 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252413 (secure 0 0 0) 0x7f8228152350 con 0x7f820806c460 2024-09-12T03:22:54.164 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:22:54.165 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2024-09-12T03:22:54.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:54.165+0000 7f822d1c4640 1 -- 172.21.15.27:0/3230676670 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f820806c460 msgr2=0x7f820806e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:54.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:54.165+0000 7f822d1c4640 1 --2- 172.21.15.27:0/3230676670 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f820806c460 0x7f820806e920 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f82280cefc0 tx=0x7f822000b040 comp rx=0 tx=0).stop 2024-09-12T03:22:54.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:54.165+0000 7f822d1c4640 1 -- 172.21.15.27:0/3230676670 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f822815c6c0 msgr2=0x7f82281e60a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:54.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:54.165+0000 7f822d1c4640 1 --2- 172.21.15.27:0/3230676670 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f822815c6c0 0x7f82281e60a0 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f82180098c0 tx=0x7f82180046c0 comp rx=0 tx=0).stop 2024-09-12T03:22:54.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:54.165+0000 7f822d1c4640 1 -- 172.21.15.27:0/3230676670 shutdown_connections 2024-09-12T03:22:54.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:54.165+0000 7f822d1c4640 1 --2- 172.21.15.27:0/3230676670 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f820806c460 0x7f820806e920 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:54.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:54.165+0000 7f822d1c4640 1 --2- 172.21.15.27:0/3230676670 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f822815c6c0 0x7f82281e60a0 unknown :-1 s=CLOSED pgs=1411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:54.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:54.165+0000 7f822d1c4640 1 --2- 172.21.15.27:0/3230676670 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f822815c2e0 0x7f8228151940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:54.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:54.165+0000 7f822d1c4640 1 --2- 172.21.15.27:0/3230676670 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8228073920 0x7f82281557f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:54.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:54.165+0000 7f822d1c4640 1 -- 172.21.15.27:0/3230676670 >> 172.21.15.27:0/3230676670 conn(0x7f822806f2c0 msgr2=0x7f82281581d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:54.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:54.165+0000 7f822d1c4640 1 -- 172.21.15.27:0/3230676670 shutdown_connections 2024-09-12T03:22:54.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:54.165+0000 7f822d1c4640 1 -- 172.21.15.27:0/3230676670 wait complete. 2024-09-12T03:22:54.175 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":151,"stamp":"2024-09-12T03:22:53.127399+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":223,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":166,"ondisk_log_size":166,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":280724,"kb_used_data":14564,"kb_used_omap":12,"kb_used_meta":265971,"kb_avail":754693996,"statfs":{"total":773094113280,"available":772806651904,"internally_reserved":0,"allocated":14913536,"data_stored":7236798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":272354864},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.298416"},"pg_stats":[{"pgid":"23.3f","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-12T03:22:50.860209+0000","last_change":"2024-09-12T03:22:50.855893+0000","last_active":"2024-09-12T03:22:50.860209+0000","last_peered":"2024-09-12T03:22:50.860209+0000","last_clean":"2024-09-12T03:22:50.860209+0000","last_became_active":"2024-09-12T03:22:50.855595+0000","last_became_peered":"2024-09-12T03:22:50.855595+0000","last_unstale":"2024-09-12T03:22:50.860209+0000","last_undegraded":"2024-09-12T03:22:50.860209+0000","last_fullsized":"2024-09-12T03:22:50.860209+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_clean_scrub_stamp":"2024-09-12T03:22:49.828625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:34:27.627909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.3e","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-12T03:22:50.860575+0000","last_change":"2024-09-12T03:22:50.855396+0000","last_active":"2024-09-12T03:22:50.860575+0000","last_peered":"2024-09-12T03:22:50.860575+0000","last_clean":"2024-09-12T03:22:50.860575+0000","last_became_active":"2024-09-12T03:22:50.854895+0000","last_became_peered":"2024-09-12T03:22:50.854895+0000","last_unstale":"2024-09-12T03:22:50.860575+0000","last_undegraded":"2024-09-12T03:22:50.860575+0000","last_fullsized":"2024-09-12T03:22:50.860575+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_clean_scrub_stamp":"2024-09-12T03:22:49.828625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:01:56.553283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.3d","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-12T03:22:50.852260+0000","last_change":"2024-09-12T03:22:50.851101+0000","last_active":"2024-09-12T03:22:50.852260+0000","last_peered":"2024-09-12T03:22:50.852260+0000","last_clean":"2024-09-12T03:22:50.852260+0000","last_became_active":"2024-09-12T03:22:50.850787+0000","last_became_peered":"2024-09-12T03:22:50.850787+0000","last_unstale":"2024-09-12T03:22:50.852260+0000","last_undegraded":"2024-09-12T03:22:50.852260+0000","last_fullsized":"2024-09-12T03:22:50.852260+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_clean_scrub_stamp":"2024-09-12T03:22:49.828625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:19:04.989627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.3c","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-12T03:22:50.868519+0000","last_change":"2024-09-12T03:22:50.860649+0000","last_active":"2024-09-12T03:22:50.868519+0000","last_peered":"2024-09-12T03:22:50.868519+0000","last_clean":"2024-09-12T03:22:50.868519+0000","last_became_active":"2024-09-12T03:22:50.860355+0000","last_became_peered":"2024-09-12T03:22:50.860355+0000","last_unstale":"2024-09-12T03:22:50.868519+0000","last_undegraded":"2024-09-12T03:22:50.868519+0000","last_fullsized":"2024-09-12T03:22:50.868519+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_clean_scrub_stamp":"2024-09-12T03:22:49.828625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:56:01.904455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.3b","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-12T03:22:50.868263+0000","last_change":"2024-09-12T03:22:50.863316+0000","last_active":"2024-09-12T03:22:50.868263+0000","last_peered":"2024-09-12T03:22:50.868263+0000","last_clean":"2024-09-12T03:22:50.868263+0000","last_became_active":"2024-09-12T03:22:50.862971+0000","last_became_peered":"2024-09-12T03:22:50.862971+0000","last_unstale":"2024-09-12T03:22:50.868263+0000","last_undegraded":"2024-09-12T03:22:50.868263+0000","last_fullsized":"2024-09-12T03:22:50.868263+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_clean_scrub_stamp":"2024-09-12T03:22:49.828625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:50:53.181760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.3a","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-12T03:22:50.862655+0000","last_change":"2024-09-12T03:22:50.859146+0000","last_active":"2024-09-12T03:22:50.862655+0000","last_peered":"2024-09-12T03:22:50.862655+0000","last_clean":"2024-09-12T03:22:50.862655+0000","last_became_active":"2024-09-12T03:22:50.858862+0000","last_became_peered":"2024-09-12T03:22:50.858862+0000","last_unstale":"2024-09-12T03:22:50.862655+0000","last_undegraded":"2024-09-12T03:22:50.862655+0000","last_fullsized":"2024-09-12T03:22:50.862655+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_clean_scrub_stamp":"2024-09-12T03:22:49.828625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:00:22.288239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.39","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-12T03:22:50.864993+0000","last_change":"2024-09-12T03:22:50.859715+0000","last_active":"2024-09-12T03:22:50.864993+0000","last_peered":"2024-09-12T03:22:50.864993+0000","last_clean":"2024-09-12T03:22:50.864993+0000","last_became_active":"2024-09-12T03:22:50.859443+0000","last_became_peered":"2024-09-12T03:22:50.859443+0000","last_unstale":"2024-09-12T03:22:50.864993+0000","last_undegraded":"2024-09-12T03:22:50.864993+0000","last_fullsized":"2024-09-12T03:22:50.864993+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_clean_scrub_stamp":"2024-09-12T03:22:49.828625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:56:04.414983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.38","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-12T03:22:50.853525+0000","last_change":"2024-09-12T03:22:50.851401+0000","last_active":"2024-09-12T03:22:50.853525+0000","last_peered":"2024-09-12T03:22:50.853525+0000","last_clean":"2024-09-12T03:22:50.853525+0000","last_became_active":"2024-09-12T03:22:50.851137+0000","last_became_peered":"2024-09-12T03:22:50.851137+0000","last_unstale":"2024-09-12T03:22:50.853525+0000","last_undegraded":"2024-09-12T03:22:50.853525+0000","last_fullsized":"2024-09-12T03:22:50.853525+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_clean_scrub_stamp":"2024-09-12T03:22:49.828625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:50:45.170771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.37","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-12T03:22:50.870402+0000","last_change":"2024-09-12T03:22:50.868300+0000","last_active":"2024-09-12T03:22:50.870402+0000","last_peered":"2024-09-12T03:22:50.870402+0000","last_clean":"2024-09-12T03:22:50.870402+0000","last_became_active":"2024-09-12T03:22:50.867981+0000","last_became_peered":"2024-09-12T03:22:50.867981+0000","last_unstale":"2024-09-12T03:22:50.870402+0000","last_undegraded":"2024-09-12T03:22:50.870402+0000","last_fullsized":"2024-09-12T03:22:50.870402+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_clean_scrub_stamp":"2024-09-12T03:22:49.828625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:56:31.375666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.36","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-12T03:22:50.870085+0000","last_change":"2024-09-12T03:22:50.866608+0000","last_active":"2024-09-12T03:22:50.870085+0000","last_peered":"2024-09-12T03:22:50.870085+0000","last_clean":"2024-09-12T03:22:50.870085+0000","last_became_active":"2024-09-12T03:22:50.866323+0000","last_became_peered":"2024-09-12T03:22:50.866323+0000","last_unstale":"2024-09-12T03:22:50.870085+0000","last_undegraded":"2024-09-12T03:22:50.870085+0000","last_fullsized":"2024-09-12T03:22:50.870085+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_clean_scrub_stamp":"2024-09-12T03:22:49.828625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:00:42.792111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.35","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-12T03:22:50.862244+0000","last_change":"2024-09-12T03:22:50.858452+0000","last_active":"2024-09-12T03:22:50.862244+0000","last_peered":"2024-09-12T03:22:50.862244+0000","last_clean":"2024-09-12T03:22:50.862244+0000","last_became_active":"2024-09-12T03:22:50.858186+0000","last_became_peered":"2024-09-12T03:22:50.858186+0000","last_unstale":"2024-09-12T03:22:50.862244+0000","last_undegraded":"2024-09-12T03:22:50.862244+0000","last_fullsized":"2024-09-12T03:22:50.862244+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_clean_scrub_stamp":"2024-09-12T03:22:49.828625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:39:57.874510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.34","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-12T03:22:50.860615+0000","last_change":"2024-09-12T03:22:50.854108+0000","last_active":"2024-09-12T03:22:50.860615+0000","last_peered":"2024-09-12T03:22:50.860615+0000","last_clean":"2024-09-12T03:22:50.860615+0000","last_became_active":"2024-09-12T03:22:50.853827+0000","last_became_peered":"2024-09-12T03:22:50.853827+0000","last_unstale":"2024-09-12T03:22:50.860615+0000","last_undegraded":"2024-09-12T03:22:50.860615+0000","last_fullsized":"2024-09-12T03:22:50.860615+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_clean_scrub_stamp":"2024-09-12T03:22:49.828625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:11:51.287650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.33","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-12T03:22:50.864236+0000","last_change":"2024-09-12T03:22:50.862995+0000","last_active":"2024-09-12T03:22:50.864236+0000","last_peered":"2024-09-12T03:22:50.864236+0000","last_clean":"2024-09-12T03:22:50.864236+0000","last_became_active":"2024-09-12T03:22:50.861804+0000","last_became_peered":"2024-09-12T03:22:50.861804+0000","last_unstale":"2024-09-12T03:22:50.864236+0000","last_undegraded":"2024-09-12T03:22:50.864236+0000","last_fullsized":"2024-09-12T03:22:50.864236+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_clean_scrub_stamp":"2024-09-12T03:22:49.828625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:08:11.122706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.32","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-12T03:22:50.864101+0000","last_change":"2024-09-12T03:22:50.855025+0000","last_active":"2024-09-12T03:22:50.864101+0000","last_peered":"2024-09-12T03:22:50.864101+0000","last_clean":"2024-09-12T03:22:50.864101+0000","last_became_active":"2024-09-12T03:22:50.854738+0000","last_became_peered":"2024-09-12T03:22:50.854738+0000","last_unstale":"2024-09-12T03:22:50.864101+0000","last_undegraded":"2024-09-12T03:22:50.864101+0000","last_fullsized":"2024-09-12T03:22:50.864101+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_clean_scrub_stamp":"2024-09-12T03:22:49.828625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:54:04.916494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.31","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-12T03:22:50.868807+0000","last_change":"2024-09-12T03:22:50.867245+0000","last_active":"2024-09-12T03:22:50.868807+0000","last_peered":"2024-09-12T03:22:50.868807+0000","last_clean":"2024-09-12T03:22:50.868807+0000","last_became_active":"2024-09-12T03:22:50.866963+0000","last_became_peered":"2024-09-12T03:22:50.866963+0000","last_unstale":"2024-09-12T03:22:50.868807+0000","last_undegraded":"2024-09-12T03:22:50.868807+0000","last_fullsized":"2024-09-12T03:22:50.868807+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_clean_scrub_stamp":"2024-09-12T03:22:49.828625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:35:10.105847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.30","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-12T03:22:50.856766+0000","last_change":"2024-09-12T03:22:50.855923+0000","last_active":"2024-09-12T03:22:50.856766+0000","last_peered":"2024-09-12T03:22:50.856766+0000","last_clean":"2024-09-12T03:22:50.856766+0000","last_became_active":"2024-09-12T03:22:50.855655+0000","last_became_peered":"2024-09-12T03:22:50.855655+0000","last_unstale":"2024-09-12T03:22:50.856766+0000","last_undegraded":"2024-09-12T03:22:50.856766+0000","last_fullsized":"2024-09-12T03:22:50.856766+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_clean_scrub_stamp":"2024-09-12T03:22:49.828625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:32:53.224639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.2f","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-12T03:22:50.862408+0000","last_change":"2024-09-12T03:22:50.856698+0000","last_active":"2024-09-12T03:22:50.862408+0000","last_peered":"2024-09-12T03:22:50.862408+0000","last_clean":"2024-09-12T03:22:50.862408+0000","last_became_active":"2024-09-12T03:22:50.855891+0000","last_became_peered":"2024-09-12T03:22:50.855891+0000","last_unstale":"2024-09-12T03:22:50.862408+0000","last_undegraded":"2024-09-12T03:22:50.862408+0000","last_fullsized":"2024-09-12T03:22:50.862408+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_clean_scrub_stamp":"2024-09-12T03:22:49.828625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:45:22.646582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.2e","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-12T03:22:50.864644+0000","last_change":"2024-09-12T03:22:50.863716+0000","last_active":"2024-09-12T03:22:50.864644+0000","last_peered":"2024-09-12T03:22:50.864644+0000","last_clean":"2024-09-12T03:22:50.864644+0000","last_became_active":"2024-09-12T03:22:50.863404+0000","last_became_peered":"2024-09-12T03:22:50.863404+0000","last_unstale":"2024-09-12T03:22:50.864644+0000","last_undegraded":"2024-09-12T03:22:50.864644+0000","last_fullsized":"2024-09-12T03:22:50.864644+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_clean_scrub_stamp":"2024-09-12T03:22:49.828625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:02:54.239157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.2d","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-12T03:22:50.869383+0000","last_change":"2024-09-12T03:22:50.867116+0000","last_active":"2024-09-12T03:22:50.869383+0000","last_peered":"2024-09-12T03:22:50.869383+0000","last_clean":"2024-09-12T03:22:50.869383+0000","last_became_active":"2024-09-12T03:22:50.866777+0000","last_became_peered":"2024-09-12T03:22:50.866777+0000","last_unstale":"2024-09-12T03:22:50.869383+0000","last_undegraded":"2024-09-12T03:22:50.869383+0000","last_fullsized":"2024-09-12T03:22:50.869383+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_clean_scrub_stamp":"2024-09-12T03:22:49.828625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:21:29.676316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.2c","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-12T03:22:50.863911+0000","last_change":"2024-09-12T03:22:50.855007+0000","last_active":"2024-09-12T03:22:50.863911+0000","last_peered":"2024-09-12T03:22:50.863911+0000","last_clean":"2024-09-12T03:22:50.863911+0000","last_became_active":"2024-09-12T03:22:50.854562+0000","last_became_peered":"2024-09-12T03:22:50.854562+0000","last_unstale":"2024-09-12T03:22:50.863911+0000","last_undegraded":"2024-09-12T03:22:50.863911+0000","last_fullsized":"2024-09-12T03:22:50.863911+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_clean_scrub_stamp":"2024-09-12T03:22:49.828625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:22:23.000995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.2b","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-12T03:22:50.868519+0000","last_change":"2024-09-12T03:22:50.865234+0000","last_active":"2024-09-12T03:22:50.868519+0000","last_peered":"2024-09-12T03:22:50.868519+0000","last_clean":"2024-09-12T03:22:50.868519+0000","last_became_active":"2024-09-12T03:22:50.864974+0000","last_became_peered":"2024-09-12T03:22:50.864974+0000","last_unstale":"2024-09-12T03:22:50.868519+0000","last_undegraded":"2024-09-12T03:22:50.868519+0000","last_fullsized":"2024-09-12T03:22:50.868519+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_clean_scrub_stamp":"2024-09-12T03:22:49.828625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:11:46.457103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.2a","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-12T03:22:50.861940+0000","last_change":"2024-09-12T03:22:50.858104+0000","last_active":"2024-09-12T03:22:50.861940+0000","last_peered":"2024-09-12T03:22:50.861940+0000","last_clean":"2024-09-12T03:22:50.861940+0000","last_became_active":"2024-09-12T03:22:50.857769+0000","last_became_peered":"2024-09-12T03:22:50.857769+0000","last_unstale":"2024-09-12T03:22:50.861940+0000","last_undegraded":"2024-09-12T03:22:50.861940+0000","last_fullsized":"2024-09-12T03:22:50.861940+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_clean_scrub_stamp":"2024-09-12T03:22:49.828625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:28:17.630365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.29","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-12T03:22:50.862626+0000","last_change":"2024-09-12T03:22:50.859217+0000","last_active":"2024-09-12T03:22:50.862626+0000","last_peered":"2024-09-12T03:22:50.862626+0000","last_clean":"2024-09-12T03:22:50.862626+0000","last_became_active":"2024-09-12T03:22:50.858944+0000","last_became_peered":"2024-09-12T03:22:50.858944+0000","last_unstale":"2024-09-12T03:22:50.862626+0000","last_undegraded":"2024-09-12T03:22:50.862626+0000","last_fullsized":"2024-09-12T03:22:50.862626+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_clean_scrub_stamp":"2024-09-12T03:22:49.828625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:50:56.415485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.28","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-12T03:22:50.868135+0000","last_change":"2024-09-12T03:22:50.863333+0000","last_active":"2024-09-12T03:22:50.868135+0000","last_peered":"2024-09-12T03:22:50.868135+0000","last_clean":"2024-09-12T03:22:50.868135+0000","last_became_active":"2024-09-12T03:22:50.862644+0000","last_became_peered":"2024-09-12T03:22:50.862644+0000","last_unstale":"2024-09-12T03:22:50.868135+0000","last_undegraded":"2024-09-12T03:22:50.868135+0000","last_fullsized":"2024-09-12T03:22:50.868135+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_clean_scrub_stamp":"2024-09-12T03:22:49.828625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:26:57.478055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.27","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-12T03:22:50.851965+0000","last_change":"2024-09-12T03:22:50.850375+0000","last_active":"2024-09-12T03:22:50.851965+0000","last_peered":"2024-09-12T03:22:50.851965+0000","last_clean":"2024-09-12T03:22:50.851965+0000","last_became_active":"2024-09-12T03:22:50.849587+0000","last_became_peered":"2024-09-12T03:22:50.849587+0000","last_unstale":"2024-09-12T03:22:50.851965+0000","last_undegraded":"2024-09-12T03:22:50.851965+0000","last_fullsized":"2024-09-12T03:22:50.851965+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_clean_scrub_stamp":"2024-09-12T03:22:49.828625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:39:31.969229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.26","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-12T03:22:50.874381+0000","last_change":"2024-09-12T03:22:50.871659+0000","last_active":"2024-09-12T03:22:50.874381+0000","last_peered":"2024-09-12T03:22:50.874381+0000","last_clean":"2024-09-12T03:22:50.874381+0000","last_became_active":"2024-09-12T03:22:50.871361+0000","last_became_peered":"2024-09-12T03:22:50.871361+0000","last_unstale":"2024-09-12T03:22:50.874381+0000","last_undegraded":"2024-09-12T03:22:50.874381+0000","last_fullsized":"2024-09-12T03:22:50.874381+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_clean_scrub_stamp":"2024-09-12T03:22:49.828625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:36:08.825514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.25","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-12T03:22:50.869823+0000","last_change":"2024-09-12T03:22:50.867563+0000","last_active":"2024-09-12T03:22:50.869823+0000","last_peered":"2024-09-12T03:22:50.869823+0000","last_clean":"2024-09-12T03:22:50.869823+0000","last_became_active":"2024-09-12T03:22:50.867225+0000","last_became_peered":"2024-09-12T03:22:50.867225+0000","last_unstale":"2024-09-12T03:22:50.869823+0000","last_undegraded":"2024-09-12T03:22:50.869823+0000","last_fullsized":"2024-09-12T03:22:50.869823+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_clean_scrub_stamp":"2024-09-12T03:22:49.828625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:35:27.521408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.24","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-12T03:22:50.865985+0000","last_change":"2024-09-12T03:22:50.850968+0000","last_active":"2024-09-12T03:22:50.865985+0000","last_peered":"2024-09-12T03:22:50.865985+0000","last_clean":"2024-09-12T03:22:50.865985+0000","last_became_active":"2024-09-12T03:22:50.850655+0000","last_became_peered":"2024-09-12T03:22:50.850655+0000","last_unstale":"2024-09-12T03:22:50.865985+0000","last_undegraded":"2024-09-12T03:22:50.865985+0000","last_fullsized":"2024-09-12T03:22:50.865985+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_clean_scrub_stamp":"2024-09-12T03:22:49.828625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:05:39.784163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.23","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-12T03:22:50.869640+0000","last_change":"2024-09-12T03:22:50.867343+0000","last_active":"2024-09-12T03:22:50.869640+0000","last_peered":"2024-09-12T03:22:50.869640+0000","last_clean":"2024-09-12T03:22:50.869640+0000","last_became_active":"2024-09-12T03:22:50.867098+0000","last_became_peered":"2024-09-12T03:22:50.867098+0000","last_unstale":"2024-09-12T03:22:50.869640+0000","last_undegraded":"2024-09-12T03:22:50.869640+0000","last_fullsized":"2024-09-12T03:22:50.869640+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_clean_scrub_stamp":"2024-09-12T03:22:49.828625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:20:44.992719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.22","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-12T03:22:50.866835+0000","last_change":"2024-09-12T03:22:50.859328+0000","last_active":"2024-09-12T03:22:50.866835+0000","last_peered":"2024-09-12T03:22:50.866835+0000","last_clean":"2024-09-12T03:22:50.866835+0000","last_became_active":"2024-09-12T03:22:50.859032+0000","last_became_peered":"2024-09-12T03:22:50.859032+0000","last_unstale":"2024-09-12T03:22:50.866835+0000","last_undegraded":"2024-09-12T03:22:50.866835+0000","last_fullsized":"2024-09-12T03:22:50.866835+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_clean_scrub_stamp":"2024-09-12T03:22:49.828625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:02:33.474017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.21","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-12T03:22:50.867018+0000","last_change":"2024-09-12T03:22:50.860225+0000","last_active":"2024-09-12T03:22:50.867018+0000","last_peered":"2024-09-12T03:22:50.867018+0000","last_clean":"2024-09-12T03:22:50.867018+0000","last_became_active":"2024-09-12T03:22:50.859920+0000","last_became_peered":"2024-09-12T03:22:50.859920+0000","last_unstale":"2024-09-12T03:22:50.867018+0000","last_undegraded":"2024-09-12T03:22:50.867018+0000","last_fullsized":"2024-09-12T03:22:50.867018+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_clean_scrub_stamp":"2024-09-12T03:22:49.828625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:21:49.881460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.20","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-12T03:22:50.864664+0000","last_change":"2024-09-12T03:22:50.862949+0000","last_active":"2024-09-12T03:22:50.864664+0000","last_peered":"2024-09-12T03:22:50.864664+0000","last_clean":"2024-09-12T03:22:50.864664+0000","last_became_active":"2024-09-12T03:22:50.862688+0000","last_became_peered":"2024-09-12T03:22:50.862688+0000","last_unstale":"2024-09-12T03:22:50.864664+0000","last_undegraded":"2024-09-12T03:22:50.864664+0000","last_fullsized":"2024-09-12T03:22:50.864664+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_clean_scrub_stamp":"2024-09-12T03:22:49.828625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:02:33.474017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.1e","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-12T03:22:50.874358+0000","last_change":"2024-09-12T03:22:50.870927+0000","last_active":"2024-09-12T03:22:50.874358+0000","last_peered":"2024-09-12T03:22:50.874358+0000","last_clean":"2024-09-12T03:22:50.874358+0000","last_became_active":"2024-09-12T03:22:50.870624+0000","last_became_peered":"2024-09-12T03:22:50.870624+0000","last_unstale":"2024-09-12T03:22:50.874358+0000","last_undegraded":"2024-09-12T03:22:50.874358+0000","last_fullsized":"2024-09-12T03:22:50.874358+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_clean_scrub_stamp":"2024-09-12T03:22:49.828625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:41:46.198481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1f","version":"137'4","reported_seq":16,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-12T03:22:50.870583+0000","last_change":"2024-09-12T03:22:49.855893+0000","last_active":"2024-09-12T03:22:50.870583+0000","last_peered":"2024-09-12T03:22:50.870583+0000","last_clean":"2024-09-12T03:22:50.870583+0000","last_became_active":"2024-09-12T03:22:49.855580+0000","last_became_peered":"2024-09-12T03:22:49.855580+0000","last_unstale":"2024-09-12T03:22:50.870583+0000","last_undegraded":"2024-09-12T03:22:50.870583+0000","last_fullsized":"2024-09-12T03:22:50.870583+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:48.826338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:48.826338+0000","last_clean_scrub_stamp":"2024-09-12T03:22:48.826338+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:22:18.465634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1f","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-12T03:22:50.866922+0000","last_change":"2024-09-12T03:22:50.860093+0000","last_active":"2024-09-12T03:22:50.866922+0000","last_peered":"2024-09-12T03:22:50.866922+0000","last_clean":"2024-09-12T03:22:50.866922+0000","last_became_active":"2024-09-12T03:22:50.859792+0000","last_became_peered":"2024-09-12T03:22:50.859792+0000","last_unstale":"2024-09-12T03:22:50.866922+0000","last_undegraded":"2024-09-12T03:22:50.866922+0000","last_fullsized":"2024-09-12T03:22:50.866922+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_clean_scrub_stamp":"2024-09-12T03:22:49.828625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:02:54.239157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.1e","version":"137'1","reported_seq":13,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-12T03:22:50.867566+0000","last_change":"2024-09-12T03:22:49.852179+0000","last_active":"2024-09-12T03:22:50.867566+0000","last_peered":"2024-09-12T03:22:50.867566+0000","last_clean":"2024-09-12T03:22:50.867566+0000","last_became_active":"2024-09-12T03:22:49.851780+0000","last_became_peered":"2024-09-12T03:22:49.851780+0000","last_unstale":"2024-09-12T03:22:50.867566+0000","last_undegraded":"2024-09-12T03:22:50.867566+0000","last_fullsized":"2024-09-12T03:22:50.867566+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:48.826338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:48.826338+0000","last_clean_scrub_stamp":"2024-09-12T03:22:48.826338+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:20:31.945130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.1c","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-12T03:22:50.862539+0000","last_change":"2024-09-12T03:22:50.855355+0000","last_active":"2024-09-12T03:22:50.862539+0000","last_peered":"2024-09-12T03:22:50.862539+0000","last_clean":"2024-09-12T03:22:50.862539+0000","last_became_active":"2024-09-12T03:22:50.855084+0000","last_became_peered":"2024-09-12T03:22:50.855084+0000","last_unstale":"2024-09-12T03:22:50.862539+0000","last_undegraded":"2024-09-12T03:22:50.862539+0000","last_fullsized":"2024-09-12T03:22:50.862539+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_clean_scrub_stamp":"2024-09-12T03:22:49.828625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:56:54.187555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.1d","version":"137'2","reported_seq":14,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-12T03:22:50.871473+0000","last_change":"2024-09-12T03:22:49.852968+0000","last_active":"2024-09-12T03:22:50.871473+0000","last_peered":"2024-09-12T03:22:50.871473+0000","last_clean":"2024-09-12T03:22:50.871473+0000","last_became_active":"2024-09-12T03:22:49.852628+0000","last_became_peered":"2024-09-12T03:22:49.852628+0000","last_unstale":"2024-09-12T03:22:50.871473+0000","last_undegraded":"2024-09-12T03:22:50.871473+0000","last_fullsized":"2024-09-12T03:22:50.871473+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:48.826338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:48.826338+0000","last_clean_scrub_stamp":"2024-09-12T03:22:48.826338+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:16:22.473280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.1d","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-12T03:22:50.864170+0000","last_change":"2024-09-12T03:22:50.858651+0000","last_active":"2024-09-12T03:22:50.864170+0000","last_peered":"2024-09-12T03:22:50.864170+0000","last_clean":"2024-09-12T03:22:50.864170+0000","last_became_active":"2024-09-12T03:22:50.858374+0000","last_became_peered":"2024-09-12T03:22:50.858374+0000","last_unstale":"2024-09-12T03:22:50.864170+0000","last_undegraded":"2024-09-12T03:22:50.864170+0000","last_fullsized":"2024-09-12T03:22:50.864170+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_clean_scrub_stamp":"2024-09-12T03:22:49.828625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:25:37.877910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1c","version":"137'2","reported_seq":14,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-12T03:22:50.866147+0000","last_change":"2024-09-12T03:22:49.850674+0000","last_active":"2024-09-12T03:22:50.866147+0000","last_peered":"2024-09-12T03:22:50.866147+0000","last_clean":"2024-09-12T03:22:50.866147+0000","last_became_active":"2024-09-12T03:22:49.850328+0000","last_became_peered":"2024-09-12T03:22:49.850328+0000","last_unstale":"2024-09-12T03:22:50.866147+0000","last_undegraded":"2024-09-12T03:22:50.866147+0000","last_fullsized":"2024-09-12T03:22:50.866147+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:48.826338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:48.826338+0000","last_clean_scrub_stamp":"2024-09-12T03:22:48.826338+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:53:10.816034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.1a","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-12T03:22:50.865023+0000","last_change":"2024-09-12T03:22:50.858876+0000","last_active":"2024-09-12T03:22:50.865023+0000","last_peered":"2024-09-12T03:22:50.865023+0000","last_clean":"2024-09-12T03:22:50.865023+0000","last_became_active":"2024-09-12T03:22:50.858560+0000","last_became_peered":"2024-09-12T03:22:50.858560+0000","last_unstale":"2024-09-12T03:22:50.865023+0000","last_undegraded":"2024-09-12T03:22:50.865023+0000","last_fullsized":"2024-09-12T03:22:50.865023+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_clean_scrub_stamp":"2024-09-12T03:22:49.828625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:02:02.553069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1b","version":"137'1","reported_seq":13,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-12T03:22:50.869825+0000","last_change":"2024-09-12T03:22:49.848462+0000","last_active":"2024-09-12T03:22:50.869825+0000","last_peered":"2024-09-12T03:22:50.869825+0000","last_clean":"2024-09-12T03:22:50.869825+0000","last_became_active":"2024-09-12T03:22:49.848180+0000","last_became_peered":"2024-09-12T03:22:49.848180+0000","last_unstale":"2024-09-12T03:22:50.869825+0000","last_undegraded":"2024-09-12T03:22:50.869825+0000","last_fullsized":"2024-09-12T03:22:50.869825+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:48.826338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:48.826338+0000","last_clean_scrub_stamp":"2024-09-12T03:22:48.826338+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:27:52.204757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.1b","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-12T03:22:50.868526+0000","last_change":"2024-09-12T03:22:50.863673+0000","last_active":"2024-09-12T03:22:50.868526+0000","last_peered":"2024-09-12T03:22:50.868526+0000","last_clean":"2024-09-12T03:22:50.868526+0000","last_became_active":"2024-09-12T03:22:50.863369+0000","last_became_peered":"2024-09-12T03:22:50.863369+0000","last_unstale":"2024-09-12T03:22:50.868526+0000","last_undegraded":"2024-09-12T03:22:50.868526+0000","last_fullsized":"2024-09-12T03:22:50.868526+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_clean_scrub_stamp":"2024-09-12T03:22:49.828625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:25:42.433733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.1a","version":"137'1","reported_seq":13,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-12T03:22:50.870008+0000","last_change":"2024-09-12T03:22:49.855130+0000","last_active":"2024-09-12T03:22:50.870008+0000","last_peered":"2024-09-12T03:22:50.870008+0000","last_clean":"2024-09-12T03:22:50.870008+0000","last_became_active":"2024-09-12T03:22:49.854800+0000","last_became_peered":"2024-09-12T03:22:49.854800+0000","last_unstale":"2024-09-12T03:22:50.870008+0000","last_undegraded":"2024-09-12T03:22:50.870008+0000","last_fullsized":"2024-09-12T03:22:50.870008+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:48.826338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:48.826338+0000","last_clean_scrub_stamp":"2024-09-12T03:22:48.826338+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:59:04.016946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.18","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-12T03:22:50.865424+0000","last_change":"2024-09-12T03:22:50.857379+0000","last_active":"2024-09-12T03:22:50.865424+0000","last_peered":"2024-09-12T03:22:50.865424+0000","last_clean":"2024-09-12T03:22:50.865424+0000","last_became_active":"2024-09-12T03:22:50.856075+0000","last_became_peered":"2024-09-12T03:22:50.856075+0000","last_unstale":"2024-09-12T03:22:50.865424+0000","last_undegraded":"2024-09-12T03:22:50.865424+0000","last_fullsized":"2024-09-12T03:22:50.865424+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_clean_scrub_stamp":"2024-09-12T03:22:49.828625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:02:29.722088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.19","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-12T03:22:50.847213+0000","last_change":"2024-09-12T03:22:49.851694+0000","last_active":"2024-09-12T03:22:50.847213+0000","last_peered":"2024-09-12T03:22:50.847213+0000","last_clean":"2024-09-12T03:22:50.847213+0000","last_became_active":"2024-09-12T03:22:49.851405+0000","last_became_peered":"2024-09-12T03:22:49.851405+0000","last_unstale":"2024-09-12T03:22:50.847213+0000","last_undegraded":"2024-09-12T03:22:50.847213+0000","last_fullsized":"2024-09-12T03:22:50.847213+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:48.826338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:48.826338+0000","last_clean_scrub_stamp":"2024-09-12T03:22:48.826338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:28:11.350507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.19","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-12T03:22:50.862608+0000","last_change":"2024-09-12T03:22:50.858426+0000","last_active":"2024-09-12T03:22:50.862608+0000","last_peered":"2024-09-12T03:22:50.862608+0000","last_clean":"2024-09-12T03:22:50.862608+0000","last_became_active":"2024-09-12T03:22:50.858101+0000","last_became_peered":"2024-09-12T03:22:50.858101+0000","last_unstale":"2024-09-12T03:22:50.862608+0000","last_undegraded":"2024-09-12T03:22:50.862608+0000","last_fullsized":"2024-09-12T03:22:50.862608+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_clean_scrub_stamp":"2024-09-12T03:22:49.828625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:59:46.483645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.18","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-12T03:22:50.848699+0000","last_change":"2024-09-12T03:22:49.858710+0000","last_active":"2024-09-12T03:22:50.848699+0000","last_peered":"2024-09-12T03:22:50.848699+0000","last_clean":"2024-09-12T03:22:50.848699+0000","last_became_active":"2024-09-12T03:22:49.857931+0000","last_became_peered":"2024-09-12T03:22:49.857931+0000","last_unstale":"2024-09-12T03:22:50.848699+0000","last_undegraded":"2024-09-12T03:22:50.848699+0000","last_fullsized":"2024-09-12T03:22:50.848699+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:48.826338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:48.826338+0000","last_clean_scrub_stamp":"2024-09-12T03:22:48.826338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:34:26.625621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.17","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-12T03:22:50.872182+0000","last_change":"2024-09-12T03:22:50.869769+0000","last_active":"2024-09-12T03:22:50.872182+0000","last_peered":"2024-09-12T03:22:50.872182+0000","last_clean":"2024-09-12T03:22:50.872182+0000","last_became_active":"2024-09-12T03:22:50.868700+0000","last_became_peered":"2024-09-12T03:22:50.868700+0000","last_unstale":"2024-09-12T03:22:50.872182+0000","last_undegraded":"2024-09-12T03:22:50.872182+0000","last_fullsized":"2024-09-12T03:22:50.872182+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_clean_scrub_stamp":"2024-09-12T03:22:49.828625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:11:12.062704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.16","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-12T03:22:50.848906+0000","last_change":"2024-09-12T03:22:49.854389+0000","last_active":"2024-09-12T03:22:50.848906+0000","last_peered":"2024-09-12T03:22:50.848906+0000","last_clean":"2024-09-12T03:22:50.848906+0000","last_became_active":"2024-09-12T03:22:49.854101+0000","last_became_peered":"2024-09-12T03:22:49.854101+0000","last_unstale":"2024-09-12T03:22:50.848906+0000","last_undegraded":"2024-09-12T03:22:50.848906+0000","last_fullsized":"2024-09-12T03:22:50.848906+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:48.826338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:48.826338+0000","last_clean_scrub_stamp":"2024-09-12T03:22:48.826338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:39:19.814553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.14","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-12T03:22:50.872172+0000","last_change":"2024-09-12T03:22:50.869496+0000","last_active":"2024-09-12T03:22:50.872172+0000","last_peered":"2024-09-12T03:22:50.872172+0000","last_clean":"2024-09-12T03:22:50.872172+0000","last_became_active":"2024-09-12T03:22:50.868978+0000","last_became_peered":"2024-09-12T03:22:50.868978+0000","last_unstale":"2024-09-12T03:22:50.872172+0000","last_undegraded":"2024-09-12T03:22:50.872172+0000","last_fullsized":"2024-09-12T03:22:50.872172+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_clean_scrub_stamp":"2024-09-12T03:22:49.828625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:48:13.106684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.15","version":"137'3","reported_seq":15,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-12T03:22:50.867600+0000","last_change":"2024-09-12T03:22:49.859597+0000","last_active":"2024-09-12T03:22:50.867600+0000","last_peered":"2024-09-12T03:22:50.867600+0000","last_clean":"2024-09-12T03:22:50.867600+0000","last_became_active":"2024-09-12T03:22:49.859260+0000","last_became_peered":"2024-09-12T03:22:49.859260+0000","last_unstale":"2024-09-12T03:22:50.867600+0000","last_undegraded":"2024-09-12T03:22:50.867600+0000","last_fullsized":"2024-09-12T03:22:50.867600+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:48.826338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:48.826338+0000","last_clean_scrub_stamp":"2024-09-12T03:22:48.826338+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:41:45.248785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.15","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-12T03:22:50.850510+0000","last_change":"2024-09-12T03:22:50.848161+0000","last_active":"2024-09-12T03:22:50.850510+0000","last_peered":"2024-09-12T03:22:50.850510+0000","last_clean":"2024-09-12T03:22:50.850510+0000","last_became_active":"2024-09-12T03:22:50.847855+0000","last_became_peered":"2024-09-12T03:22:50.847855+0000","last_unstale":"2024-09-12T03:22:50.850510+0000","last_undegraded":"2024-09-12T03:22:50.850510+0000","last_fullsized":"2024-09-12T03:22:50.850510+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_clean_scrub_stamp":"2024-09-12T03:22:49.828625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:37:14.788266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.14","version":"137'6","reported_seq":18,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-12T03:22:50.871686+0000","last_change":"2024-09-12T03:22:49.853821+0000","last_active":"2024-09-12T03:22:50.871686+0000","last_peered":"2024-09-12T03:22:50.871686+0000","last_clean":"2024-09-12T03:22:50.871686+0000","last_became_active":"2024-09-12T03:22:49.853533+0000","last_became_peered":"2024-09-12T03:22:49.853533+0000","last_unstale":"2024-09-12T03:22:50.871686+0000","last_undegraded":"2024-09-12T03:22:50.871686+0000","last_fullsized":"2024-09-12T03:22:50.871686+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:48.826338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:48.826338+0000","last_clean_scrub_stamp":"2024-09-12T03:22:48.826338+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:39:32.338120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.12","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-12T03:22:50.862355+0000","last_change":"2024-09-12T03:22:50.858038+0000","last_active":"2024-09-12T03:22:50.862355+0000","last_peered":"2024-09-12T03:22:50.862355+0000","last_clean":"2024-09-12T03:22:50.862355+0000","last_became_active":"2024-09-12T03:22:50.857623+0000","last_became_peered":"2024-09-12T03:22:50.857623+0000","last_unstale":"2024-09-12T03:22:50.862355+0000","last_undegraded":"2024-09-12T03:22:50.862355+0000","last_fullsized":"2024-09-12T03:22:50.862355+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_clean_scrub_stamp":"2024-09-12T03:22:49.828625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:28:23.214979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.13","version":"137'2","reported_seq":14,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-12T03:22:50.868447+0000","last_change":"2024-09-12T03:22:49.854043+0000","last_active":"2024-09-12T03:22:50.868447+0000","last_peered":"2024-09-12T03:22:50.868447+0000","last_clean":"2024-09-12T03:22:50.868447+0000","last_became_active":"2024-09-12T03:22:49.853742+0000","last_became_peered":"2024-09-12T03:22:49.853742+0000","last_unstale":"2024-09-12T03:22:50.868447+0000","last_undegraded":"2024-09-12T03:22:50.868447+0000","last_fullsized":"2024-09-12T03:22:50.868447+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:48.826338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:48.826338+0000","last_clean_scrub_stamp":"2024-09-12T03:22:48.826338+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:18:31.622015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.13","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-12T03:22:50.856734+0000","last_change":"2024-09-12T03:22:50.855073+0000","last_active":"2024-09-12T03:22:50.856734+0000","last_peered":"2024-09-12T03:22:50.856734+0000","last_clean":"2024-09-12T03:22:50.856734+0000","last_became_active":"2024-09-12T03:22:50.854787+0000","last_became_peered":"2024-09-12T03:22:50.854787+0000","last_unstale":"2024-09-12T03:22:50.856734+0000","last_undegraded":"2024-09-12T03:22:50.856734+0000","last_fullsized":"2024-09-12T03:22:50.856734+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_clean_scrub_stamp":"2024-09-12T03:22:49.828625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:52:23.711059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.12","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-12T03:22:50.854241+0000","last_change":"2024-09-12T03:22:49.857897+0000","last_active":"2024-09-12T03:22:50.854241+0000","last_peered":"2024-09-12T03:22:50.854241+0000","last_clean":"2024-09-12T03:22:50.854241+0000","last_became_active":"2024-09-12T03:22:49.857602+0000","last_became_peered":"2024-09-12T03:22:49.857602+0000","last_unstale":"2024-09-12T03:22:50.854241+0000","last_undegraded":"2024-09-12T03:22:50.854241+0000","last_fullsized":"2024-09-12T03:22:50.854241+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:48.826338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:48.826338+0000","last_clean_scrub_stamp":"2024-09-12T03:22:48.826338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:20:31.945130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.10","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-12T03:22:50.865399+0000","last_change":"2024-09-12T03:22:50.856364+0000","last_active":"2024-09-12T03:22:50.865399+0000","last_peered":"2024-09-12T03:22:50.865399+0000","last_clean":"2024-09-12T03:22:50.865399+0000","last_became_active":"2024-09-12T03:22:50.855657+0000","last_became_peered":"2024-09-12T03:22:50.855657+0000","last_unstale":"2024-09-12T03:22:50.865399+0000","last_undegraded":"2024-09-12T03:22:50.865399+0000","last_fullsized":"2024-09-12T03:22:50.865399+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_clean_scrub_stamp":"2024-09-12T03:22:49.828625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:52:16.974059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.11","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-12T03:22:50.840846+0000","last_change":"2024-09-12T03:22:49.849723+0000","last_active":"2024-09-12T03:22:50.840846+0000","last_peered":"2024-09-12T03:22:50.840846+0000","last_clean":"2024-09-12T03:22:50.840846+0000","last_became_active":"2024-09-12T03:22:49.849423+0000","last_became_peered":"2024-09-12T03:22:49.849423+0000","last_unstale":"2024-09-12T03:22:50.840846+0000","last_undegraded":"2024-09-12T03:22:50.840846+0000","last_fullsized":"2024-09-12T03:22:50.840846+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:48.826338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:48.826338+0000","last_clean_scrub_stamp":"2024-09-12T03:22:48.826338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:19:43.037310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.11","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-12T03:22:50.864443+0000","last_change":"2024-09-12T03:22:50.855109+0000","last_active":"2024-09-12T03:22:50.864443+0000","last_peered":"2024-09-12T03:22:50.864443+0000","last_clean":"2024-09-12T03:22:50.864443+0000","last_became_active":"2024-09-12T03:22:50.854795+0000","last_became_peered":"2024-09-12T03:22:50.854795+0000","last_unstale":"2024-09-12T03:22:50.864443+0000","last_undegraded":"2024-09-12T03:22:50.864443+0000","last_fullsized":"2024-09-12T03:22:50.864443+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_clean_scrub_stamp":"2024-09-12T03:22:49.828625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:51:36.386947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.10","version":"137'8","reported_seq":20,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-12T03:22:50.869819+0000","last_change":"2024-09-12T03:22:49.854880+0000","last_active":"2024-09-12T03:22:50.869819+0000","last_peered":"2024-09-12T03:22:50.869819+0000","last_clean":"2024-09-12T03:22:50.869819+0000","last_became_active":"2024-09-12T03:22:49.854167+0000","last_became_peered":"2024-09-12T03:22:49.854167+0000","last_unstale":"2024-09-12T03:22:50.869819+0000","last_undegraded":"2024-09-12T03:22:50.869819+0000","last_fullsized":"2024-09-12T03:22:50.869819+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:48.826338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:48.826338+0000","last_clean_scrub_stamp":"2024-09-12T03:22:48.826338+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:03:12.236893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.e","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-12T03:22:50.865674+0000","last_change":"2024-09-12T03:22:50.857848+0000","last_active":"2024-09-12T03:22:50.865674+0000","last_peered":"2024-09-12T03:22:50.865674+0000","last_clean":"2024-09-12T03:22:50.865674+0000","last_became_active":"2024-09-12T03:22:50.857348+0000","last_became_peered":"2024-09-12T03:22:50.857348+0000","last_unstale":"2024-09-12T03:22:50.865674+0000","last_undegraded":"2024-09-12T03:22:50.865674+0000","last_fullsized":"2024-09-12T03:22:50.865674+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_clean_scrub_stamp":"2024-09-12T03:22:49.828625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:47:48.004911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.f","version":"137'1","reported_seq":13,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-12T03:22:50.870317+0000","last_change":"2024-09-12T03:22:49.851789+0000","last_active":"2024-09-12T03:22:50.870317+0000","last_peered":"2024-09-12T03:22:50.870317+0000","last_clean":"2024-09-12T03:22:50.870317+0000","last_became_active":"2024-09-12T03:22:49.851432+0000","last_became_peered":"2024-09-12T03:22:49.851432+0000","last_unstale":"2024-09-12T03:22:50.870317+0000","last_undegraded":"2024-09-12T03:22:50.870317+0000","last_fullsized":"2024-09-12T03:22:50.870317+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:48.826338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:48.826338+0000","last_clean_scrub_stamp":"2024-09-12T03:22:48.826338+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:46:01.538002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.f","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-12T03:22:50.874371+0000","last_change":"2024-09-12T03:22:50.870855+0000","last_active":"2024-09-12T03:22:50.874371+0000","last_peered":"2024-09-12T03:22:50.874371+0000","last_clean":"2024-09-12T03:22:50.874371+0000","last_became_active":"2024-09-12T03:22:50.868491+0000","last_became_peered":"2024-09-12T03:22:50.868491+0000","last_unstale":"2024-09-12T03:22:50.874371+0000","last_undegraded":"2024-09-12T03:22:50.874371+0000","last_fullsized":"2024-09-12T03:22:50.874371+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_clean_scrub_stamp":"2024-09-12T03:22:49.828625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:45:10.451298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.e","version":"137'2","reported_seq":14,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-12T03:22:50.870727+0000","last_change":"2024-09-12T03:22:49.852706+0000","last_active":"2024-09-12T03:22:50.870727+0000","last_peered":"2024-09-12T03:22:50.870727+0000","last_clean":"2024-09-12T03:22:50.870727+0000","last_became_active":"2024-09-12T03:22:49.852423+0000","last_became_peered":"2024-09-12T03:22:49.852423+0000","last_unstale":"2024-09-12T03:22:50.870727+0000","last_undegraded":"2024-09-12T03:22:50.870727+0000","last_fullsized":"2024-09-12T03:22:50.870727+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:48.826338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:48.826338+0000","last_clean_scrub_stamp":"2024-09-12T03:22:48.826338+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:26:33.756735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.c","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-12T03:22:50.868541+0000","last_change":"2024-09-12T03:22:50.864312+0000","last_active":"2024-09-12T03:22:50.868541+0000","last_peered":"2024-09-12T03:22:50.868541+0000","last_clean":"2024-09-12T03:22:50.868541+0000","last_became_active":"2024-09-12T03:22:50.864053+0000","last_became_peered":"2024-09-12T03:22:50.864053+0000","last_unstale":"2024-09-12T03:22:50.868541+0000","last_undegraded":"2024-09-12T03:22:50.868541+0000","last_fullsized":"2024-09-12T03:22:50.868541+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_clean_scrub_stamp":"2024-09-12T03:22:49.828625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:52:51.383350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.d","version":"137'1","reported_seq":13,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-12T03:22:50.870339+0000","last_change":"2024-09-12T03:22:49.853673+0000","last_active":"2024-09-12T03:22:50.870339+0000","last_peered":"2024-09-12T03:22:50.870339+0000","last_clean":"2024-09-12T03:22:50.870339+0000","last_became_active":"2024-09-12T03:22:49.853374+0000","last_became_peered":"2024-09-12T03:22:49.853374+0000","last_unstale":"2024-09-12T03:22:50.870339+0000","last_undegraded":"2024-09-12T03:22:50.870339+0000","last_fullsized":"2024-09-12T03:22:50.870339+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:48.826338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:48.826338+0000","last_clean_scrub_stamp":"2024-09-12T03:22:48.826338+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:45:05.684426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.d","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-12T03:22:50.864945+0000","last_change":"2024-09-12T03:22:50.862005+0000","last_active":"2024-09-12T03:22:50.864945+0000","last_peered":"2024-09-12T03:22:50.864945+0000","last_clean":"2024-09-12T03:22:50.864945+0000","last_became_active":"2024-09-12T03:22:50.861682+0000","last_became_peered":"2024-09-12T03:22:50.861682+0000","last_unstale":"2024-09-12T03:22:50.864945+0000","last_undegraded":"2024-09-12T03:22:50.864945+0000","last_fullsized":"2024-09-12T03:22:50.864945+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_clean_scrub_stamp":"2024-09-12T03:22:49.828625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:47:18.184259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.c","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-12T03:22:50.847289+0000","last_change":"2024-09-12T03:22:49.857153+0000","last_active":"2024-09-12T03:22:50.847289+0000","last_peered":"2024-09-12T03:22:50.847289+0000","last_clean":"2024-09-12T03:22:50.847289+0000","last_became_active":"2024-09-12T03:22:49.856827+0000","last_became_peered":"2024-09-12T03:22:49.856827+0000","last_unstale":"2024-09-12T03:22:50.847289+0000","last_undegraded":"2024-09-12T03:22:50.847289+0000","last_fullsized":"2024-09-12T03:22:50.847289+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:48.826338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:48.826338+0000","last_clean_scrub_stamp":"2024-09-12T03:22:48.826338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:54:45.693003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.a","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-12T03:22:50.864273+0000","last_change":"2024-09-12T03:22:50.860128+0000","last_active":"2024-09-12T03:22:50.864273+0000","last_peered":"2024-09-12T03:22:50.864273+0000","last_clean":"2024-09-12T03:22:50.864273+0000","last_became_active":"2024-09-12T03:22:50.859864+0000","last_became_peered":"2024-09-12T03:22:50.859864+0000","last_unstale":"2024-09-12T03:22:50.864273+0000","last_undegraded":"2024-09-12T03:22:50.864273+0000","last_fullsized":"2024-09-12T03:22:50.864273+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_clean_scrub_stamp":"2024-09-12T03:22:49.828625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:46:46.620260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.b","version":"137'2","reported_seq":14,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-12T03:22:50.871609+0000","last_change":"2024-09-12T03:22:49.855989+0000","last_active":"2024-09-12T03:22:50.871609+0000","last_peered":"2024-09-12T03:22:50.871609+0000","last_clean":"2024-09-12T03:22:50.871609+0000","last_became_active":"2024-09-12T03:22:49.854000+0000","last_became_peered":"2024-09-12T03:22:49.854000+0000","last_unstale":"2024-09-12T03:22:50.871609+0000","last_undegraded":"2024-09-12T03:22:50.871609+0000","last_fullsized":"2024-09-12T03:22:50.871609+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:48.826338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:48.826338+0000","last_clean_scrub_stamp":"2024-09-12T03:22:48.826338+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:33:38.632902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.b","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-12T03:22:50.855474+0000","last_change":"2024-09-12T03:22:50.853807+0000","last_active":"2024-09-12T03:22:50.855474+0000","last_peered":"2024-09-12T03:22:50.855474+0000","last_clean":"2024-09-12T03:22:50.855474+0000","last_became_active":"2024-09-12T03:22:50.853488+0000","last_became_peered":"2024-09-12T03:22:50.853488+0000","last_unstale":"2024-09-12T03:22:50.855474+0000","last_undegraded":"2024-09-12T03:22:50.855474+0000","last_fullsized":"2024-09-12T03:22:50.855474+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_clean_scrub_stamp":"2024-09-12T03:22:49.828625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:35:10.105847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.a","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-12T03:22:50.845726+0000","last_change":"2024-09-12T03:22:49.858817+0000","last_active":"2024-09-12T03:22:50.845726+0000","last_peered":"2024-09-12T03:22:50.845726+0000","last_clean":"2024-09-12T03:22:50.845726+0000","last_became_active":"2024-09-12T03:22:49.858312+0000","last_became_peered":"2024-09-12T03:22:49.858312+0000","last_unstale":"2024-09-12T03:22:50.845726+0000","last_undegraded":"2024-09-12T03:22:50.845726+0000","last_fullsized":"2024-09-12T03:22:50.845726+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:48.826338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:48.826338+0000","last_clean_scrub_stamp":"2024-09-12T03:22:48.826338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:53:33.732611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.8","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-12T03:22:50.858300+0000","last_change":"2024-09-12T03:22:50.856665+0000","last_active":"2024-09-12T03:22:50.858300+0000","last_peered":"2024-09-12T03:22:50.858300+0000","last_clean":"2024-09-12T03:22:50.858300+0000","last_became_active":"2024-09-12T03:22:50.856366+0000","last_became_peered":"2024-09-12T03:22:50.856366+0000","last_unstale":"2024-09-12T03:22:50.858300+0000","last_undegraded":"2024-09-12T03:22:50.858300+0000","last_fullsized":"2024-09-12T03:22:50.858300+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_clean_scrub_stamp":"2024-09-12T03:22:49.828625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:33:47.516847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.9","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-12T03:22:50.840896+0000","last_change":"2024-09-12T03:22:49.846978+0000","last_active":"2024-09-12T03:22:50.840896+0000","last_peered":"2024-09-12T03:22:50.840896+0000","last_clean":"2024-09-12T03:22:50.840896+0000","last_became_active":"2024-09-12T03:22:49.846663+0000","last_became_peered":"2024-09-12T03:22:49.846663+0000","last_unstale":"2024-09-12T03:22:50.840896+0000","last_undegraded":"2024-09-12T03:22:50.840896+0000","last_fullsized":"2024-09-12T03:22:50.840896+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:48.826338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:48.826338+0000","last_clean_scrub_stamp":"2024-09-12T03:22:48.826338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:53:23.453570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.9","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-12T03:22:50.862586+0000","last_change":"2024-09-12T03:22:50.857545+0000","last_active":"2024-09-12T03:22:50.862586+0000","last_peered":"2024-09-12T03:22:50.862586+0000","last_clean":"2024-09-12T03:22:50.862586+0000","last_became_active":"2024-09-12T03:22:50.857249+0000","last_became_peered":"2024-09-12T03:22:50.857249+0000","last_unstale":"2024-09-12T03:22:50.862586+0000","last_undegraded":"2024-09-12T03:22:50.862586+0000","last_fullsized":"2024-09-12T03:22:50.862586+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_clean_scrub_stamp":"2024-09-12T03:22:49.828625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:55:57.520589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.8","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-12T03:22:50.847002+0000","last_change":"2024-09-12T03:22:49.853736+0000","last_active":"2024-09-12T03:22:50.847002+0000","last_peered":"2024-09-12T03:22:50.847002+0000","last_clean":"2024-09-12T03:22:50.847002+0000","last_became_active":"2024-09-12T03:22:49.853428+0000","last_became_peered":"2024-09-12T03:22:49.853428+0000","last_unstale":"2024-09-12T03:22:50.847002+0000","last_undegraded":"2024-09-12T03:22:50.847002+0000","last_fullsized":"2024-09-12T03:22:50.847002+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:48.826338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:48.826338+0000","last_clean_scrub_stamp":"2024-09-12T03:22:48.826338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:41:47.248620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.6","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-12T03:22:50.872201+0000","last_change":"2024-09-12T03:22:50.869488+0000","last_active":"2024-09-12T03:22:50.872201+0000","last_peered":"2024-09-12T03:22:50.872201+0000","last_clean":"2024-09-12T03:22:50.872201+0000","last_became_active":"2024-09-12T03:22:50.869036+0000","last_became_peered":"2024-09-12T03:22:50.869036+0000","last_unstale":"2024-09-12T03:22:50.872201+0000","last_undegraded":"2024-09-12T03:22:50.872201+0000","last_fullsized":"2024-09-12T03:22:50.872201+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_clean_scrub_stamp":"2024-09-12T03:22:49.828625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:21:49.881460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.7","version":"137'2","reported_seq":14,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-12T03:22:50.865954+0000","last_change":"2024-09-12T03:22:49.855553+0000","last_active":"2024-09-12T03:22:50.865954+0000","last_peered":"2024-09-12T03:22:50.865954+0000","last_clean":"2024-09-12T03:22:50.865954+0000","last_became_active":"2024-09-12T03:22:49.855277+0000","last_became_peered":"2024-09-12T03:22:49.855277+0000","last_unstale":"2024-09-12T03:22:50.865954+0000","last_undegraded":"2024-09-12T03:22:50.865954+0000","last_fullsized":"2024-09-12T03:22:50.865954+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:48.826338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:48.826338+0000","last_clean_scrub_stamp":"2024-09-12T03:22:48.826338+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:39:32.338120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.7","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-12T03:22:50.856662+0000","last_change":"2024-09-12T03:22:50.855109+0000","last_active":"2024-09-12T03:22:50.856662+0000","last_peered":"2024-09-12T03:22:50.856662+0000","last_clean":"2024-09-12T03:22:50.856662+0000","last_became_active":"2024-09-12T03:22:50.854826+0000","last_became_peered":"2024-09-12T03:22:50.854826+0000","last_unstale":"2024-09-12T03:22:50.856662+0000","last_undegraded":"2024-09-12T03:22:50.856662+0000","last_fullsized":"2024-09-12T03:22:50.856662+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_clean_scrub_stamp":"2024-09-12T03:22:49.828625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:50:31.028630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.6","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-12T03:22:50.842903+0000","last_change":"2024-09-12T03:22:49.849266+0000","last_active":"2024-09-12T03:22:50.842903+0000","last_peered":"2024-09-12T03:22:50.842903+0000","last_clean":"2024-09-12T03:22:50.842903+0000","last_became_active":"2024-09-12T03:22:49.848896+0000","last_became_peered":"2024-09-12T03:22:49.848896+0000","last_unstale":"2024-09-12T03:22:50.842903+0000","last_undegraded":"2024-09-12T03:22:50.842903+0000","last_fullsized":"2024-09-12T03:22:50.842903+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:48.826338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:48.826338+0000","last_clean_scrub_stamp":"2024-09-12T03:22:48.826338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:19:38.538075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.4","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-12T03:22:50.870596+0000","last_change":"2024-09-12T03:22:50.869131+0000","last_active":"2024-09-12T03:22:50.870596+0000","last_peered":"2024-09-12T03:22:50.870596+0000","last_clean":"2024-09-12T03:22:50.870596+0000","last_became_active":"2024-09-12T03:22:50.866882+0000","last_became_peered":"2024-09-12T03:22:50.866882+0000","last_unstale":"2024-09-12T03:22:50.870596+0000","last_undegraded":"2024-09-12T03:22:50.870596+0000","last_fullsized":"2024-09-12T03:22:50.870596+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_clean_scrub_stamp":"2024-09-12T03:22:49.828625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:43:34.746247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.5","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-12T03:22:50.849598+0000","last_change":"2024-09-12T03:22:49.858237+0000","last_active":"2024-09-12T03:22:50.849598+0000","last_peered":"2024-09-12T03:22:50.849598+0000","last_clean":"2024-09-12T03:22:50.849598+0000","last_became_active":"2024-09-12T03:22:49.857571+0000","last_became_peered":"2024-09-12T03:22:49.857571+0000","last_unstale":"2024-09-12T03:22:50.849598+0000","last_undegraded":"2024-09-12T03:22:50.849598+0000","last_fullsized":"2024-09-12T03:22:50.849598+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:48.826338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:48.826338+0000","last_clean_scrub_stamp":"2024-09-12T03:22:48.826338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:24:58.895995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.5","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-12T03:22:50.861212+0000","last_change":"2024-09-12T03:22:50.856590+0000","last_active":"2024-09-12T03:22:50.861212+0000","last_peered":"2024-09-12T03:22:50.861212+0000","last_clean":"2024-09-12T03:22:50.861212+0000","last_became_active":"2024-09-12T03:22:50.856285+0000","last_became_peered":"2024-09-12T03:22:50.856285+0000","last_unstale":"2024-09-12T03:22:50.861212+0000","last_undegraded":"2024-09-12T03:22:50.861212+0000","last_fullsized":"2024-09-12T03:22:50.861212+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_clean_scrub_stamp":"2024-09-12T03:22:49.828625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:32:28.670636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.4","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-12T03:22:50.846769+0000","last_change":"2024-09-12T03:22:49.852512+0000","last_active":"2024-09-12T03:22:50.846769+0000","last_peered":"2024-09-12T03:22:50.846769+0000","last_clean":"2024-09-12T03:22:50.846769+0000","last_became_active":"2024-09-12T03:22:49.852237+0000","last_became_peered":"2024-09-12T03:22:49.852237+0000","last_unstale":"2024-09-12T03:22:50.846769+0000","last_undegraded":"2024-09-12T03:22:50.846769+0000","last_fullsized":"2024-09-12T03:22:50.846769+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:48.826338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:48.826338+0000","last_clean_scrub_stamp":"2024-09-12T03:22:48.826338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:40:44.909508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.2","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-12T03:22:50.864509+0000","last_change":"2024-09-12T03:22:50.862870+0000","last_active":"2024-09-12T03:22:50.864509+0000","last_peered":"2024-09-12T03:22:50.864509+0000","last_clean":"2024-09-12T03:22:50.864509+0000","last_became_active":"2024-09-12T03:22:50.862581+0000","last_became_peered":"2024-09-12T03:22:50.862581+0000","last_unstale":"2024-09-12T03:22:50.864509+0000","last_undegraded":"2024-09-12T03:22:50.864509+0000","last_fullsized":"2024-09-12T03:22:50.864509+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_clean_scrub_stamp":"2024-09-12T03:22:49.828625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:37:14.788266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.3","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-12T03:22:50.844592+0000","last_change":"2024-09-12T03:22:49.857358+0000","last_active":"2024-09-12T03:22:50.844592+0000","last_peered":"2024-09-12T03:22:50.844592+0000","last_clean":"2024-09-12T03:22:50.844592+0000","last_became_active":"2024-09-12T03:22:49.857043+0000","last_became_peered":"2024-09-12T03:22:49.857043+0000","last_unstale":"2024-09-12T03:22:50.844592+0000","last_undegraded":"2024-09-12T03:22:50.844592+0000","last_fullsized":"2024-09-12T03:22:50.844592+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:48.826338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:48.826338+0000","last_clean_scrub_stamp":"2024-09-12T03:22:48.826338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:28:22.212691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.3","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-12T03:22:50.867025+0000","last_change":"2024-09-12T03:22:50.861882+0000","last_active":"2024-09-12T03:22:50.867025+0000","last_peered":"2024-09-12T03:22:50.867025+0000","last_clean":"2024-09-12T03:22:50.867025+0000","last_became_active":"2024-09-12T03:22:50.860858+0000","last_became_peered":"2024-09-12T03:22:50.860858+0000","last_unstale":"2024-09-12T03:22:50.867025+0000","last_undegraded":"2024-09-12T03:22:50.867025+0000","last_fullsized":"2024-09-12T03:22:50.867025+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_clean_scrub_stamp":"2024-09-12T03:22:49.828625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:48:13.106684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.2","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-12T03:22:50.841451+0000","last_change":"2024-09-12T03:22:49.850743+0000","last_active":"2024-09-12T03:22:50.841451+0000","last_peered":"2024-09-12T03:22:50.841451+0000","last_clean":"2024-09-12T03:22:50.841451+0000","last_became_active":"2024-09-12T03:22:49.850370+0000","last_became_peered":"2024-09-12T03:22:49.850370+0000","last_unstale":"2024-09-12T03:22:50.841451+0000","last_undegraded":"2024-09-12T03:22:50.841451+0000","last_fullsized":"2024-09-12T03:22:50.841451+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:48.826338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:48.826338+0000","last_clean_scrub_stamp":"2024-09-12T03:22:48.826338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:05:15.266590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.0","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-12T03:22:50.855513+0000","last_change":"2024-09-12T03:22:50.853289+0000","last_active":"2024-09-12T03:22:50.855513+0000","last_peered":"2024-09-12T03:22:50.855513+0000","last_clean":"2024-09-12T03:22:50.855513+0000","last_became_active":"2024-09-12T03:22:50.852934+0000","last_became_peered":"2024-09-12T03:22:50.852934+0000","last_unstale":"2024-09-12T03:22:50.855513+0000","last_undegraded":"2024-09-12T03:22:50.855513+0000","last_fullsized":"2024-09-12T03:22:50.855513+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_clean_scrub_stamp":"2024-09-12T03:22:49.828625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:33:47.516847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-12T03:22:50.847299+0000","last_change":"2024-09-12T03:22:49.856653+0000","last_active":"2024-09-12T03:22:50.847299+0000","last_peered":"2024-09-12T03:22:50.847299+0000","last_clean":"2024-09-12T03:22:50.847299+0000","last_became_active":"2024-09-12T03:22:49.856359+0000","last_became_peered":"2024-09-12T03:22:49.856359+0000","last_unstale":"2024-09-12T03:22:50.847299+0000","last_undegraded":"2024-09-12T03:22:50.847299+0000","last_fullsized":"2024-09-12T03:22:50.847299+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:48.826338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:48.826338+0000","last_clean_scrub_stamp":"2024-09-12T03:22:48.826338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:38:42.593399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.1","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-12T03:22:50.864471+0000","last_change":"2024-09-12T03:22:50.857986+0000","last_active":"2024-09-12T03:22:50.864471+0000","last_peered":"2024-09-12T03:22:50.864471+0000","last_clean":"2024-09-12T03:22:50.864471+0000","last_became_active":"2024-09-12T03:22:50.857674+0000","last_became_peered":"2024-09-12T03:22:50.857674+0000","last_unstale":"2024-09-12T03:22:50.864471+0000","last_undegraded":"2024-09-12T03:22:50.864471+0000","last_fullsized":"2024-09-12T03:22:50.864471+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_clean_scrub_stamp":"2024-09-12T03:22:49.828625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:24:51.460112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.0","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-12T03:22:50.843982+0000","last_change":"2024-09-12T03:22:49.855102+0000","last_active":"2024-09-12T03:22:50.843982+0000","last_peered":"2024-09-12T03:22:50.843982+0000","last_clean":"2024-09-12T03:22:50.843982+0000","last_became_active":"2024-09-12T03:22:49.854616+0000","last_became_peered":"2024-09-12T03:22:49.854616+0000","last_unstale":"2024-09-12T03:22:50.843982+0000","last_undegraded":"2024-09-12T03:22:50.843982+0000","last_fullsized":"2024-09-12T03:22:50.843982+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:48.826338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:48.826338+0000","last_clean_scrub_stamp":"2024-09-12T03:22:48.826338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:40:44.909508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.16","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-12T03:22:50.872223+0000","last_change":"2024-09-12T03:22:50.869792+0000","last_active":"2024-09-12T03:22:50.872223+0000","last_peered":"2024-09-12T03:22:50.872223+0000","last_clean":"2024-09-12T03:22:50.872223+0000","last_became_active":"2024-09-12T03:22:50.869286+0000","last_became_peered":"2024-09-12T03:22:50.869286+0000","last_unstale":"2024-09-12T03:22:50.872223+0000","last_undegraded":"2024-09-12T03:22:50.872223+0000","last_fullsized":"2024-09-12T03:22:50.872223+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:49.828625+0000","last_clean_scrub_stamp":"2024-09-12T03:22:49.828625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:10:41.417435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.17","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-12T03:22:50.844442+0000","last_change":"2024-09-12T03:22:49.856157+0000","last_active":"2024-09-12T03:22:50.844442+0000","last_peered":"2024-09-12T03:22:50.844442+0000","last_clean":"2024-09-12T03:22:50.844442+0000","last_became_active":"2024-09-12T03:22:49.855792+0000","last_became_peered":"2024-09-12T03:22:49.855792+0000","last_unstale":"2024-09-12T03:22:50.844442+0000","last_undegraded":"2024-09-12T03:22:50.844442+0000","last_fullsized":"2024-09-12T03:22:50.844442+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:22:48.826338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:22:48.826338+0000","last_clean_scrub_stamp":"2024-09-12T03:22:48.826338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:32:27.668349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"70'128","reported_seq":422,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-12T03:22:50.847095+0000","last_change":"2024-09-12T03:16:21.883446+0000","last_active":"2024-09-12T03:22:50.847095+0000","last_peered":"2024-09-12T03:22:50.847095+0000","last_clean":"2024-09-12T03:22:50.847095+0000","last_became_active":"2024-09-12T03:16:21.882207+0000","last_became_peered":"2024-09-12T03:16:21.882207+0000","last_unstale":"2024-09-12T03:22:50.847095+0000","last_undegraded":"2024-09-12T03:22:50.847095+0000","last_fullsized":"2024-09-12T03:22:50.847095+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:20.316426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:20.316426+0000","last_clean_scrub_stamp":"2024-09-12T03:16:20.316426+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:12:15.424320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":23,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":22,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542224,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37600,"kb_used_data":2248,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334240,"statfs":{"total":96636764160,"available":96598261760,"internally_reserved":0,"allocated":2301952,"data_stored":1343949,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542226,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37036,"kb_used_data":1684,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334804,"statfs":{"total":96636764160,"available":96598839296,"internally_reserved":0,"allocated":1724416,"data_stored":760434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542225,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37020,"kb_used_data":1668,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334820,"statfs":{"total":96636764160,"available":96598855680,"internally_reserved":0,"allocated":1708032,"data_stored":753581,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542224,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37020,"kb_used_data":1668,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334820,"statfs":{"total":96636764160,"available":96598855680,"internally_reserved":0,"allocated":1708032,"data_stored":753581,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542225,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32872,"kb_used_data":1684,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338968,"statfs":{"total":96636764160,"available":96603103232,"internally_reserved":0,"allocated":1724416,"data_stored":760446,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542225,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32860,"kb_used_data":1668,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338980,"statfs":{"total":96636764160,"available":96603115520,"internally_reserved":0,"allocated":1708032,"data_stored":753581,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542226,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32872,"kb_used_data":1684,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338968,"statfs":{"total":96636764160,"available":96603103232,"internally_reserved":0,"allocated":1724416,"data_stored":760446,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542226,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33444,"kb_used_data":2260,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338396,"statfs":{"total":96636764160,"available":96602517504,"internally_reserved":0,"allocated":2314240,"data_stored":1350780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T03:22:54.179 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-12T03:22:54.180 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-12T03:22:54.326 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:54.326+0000 7f7277dd8640 1 Processor -- start 2024-09-12T03:22:54.326 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:54.326+0000 7f7277dd8640 1 -- start start 2024-09-12T03:22:54.327 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:54.327+0000 7f7277dd8640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f727015e900 0x7f72700af370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:54.327 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:54.327+0000 7f7277dd8640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f72700b6540 0x7f727006ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:54.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:54.327+0000 7f7277dd8640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f72700b5940 con 0x7f727006d730 2024-09-12T03:22:54.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:54.327+0000 7f7277dd8640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f727015ece0 con 0x7f727015e900 2024-09-12T03:22:54.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:54.327+0000 7f7277dd8640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f72700af950 con 0x7f72700b6540 2024-09-12T03:22:54.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:54.327+0000 7f7275373640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f727006d730 0x7f72700b5e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:52370/0 (socket says 172.21.15.27:52370) 2024-09-12T03:22:54.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:54.327+0000 7f7275373640 1 -- 172.21.15.27:0/4109754327 learned_addr learned my addr 172.21.15.27:0/4109754327 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:22:54.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:54.327+0000 7f7275b74640 1 --2- 172.21.15.27:0/4109754327 >> v2:172.21.15.45:3300/0 conn(0x7f727015e900 0x7f72700af370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:54.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:54.327+0000 7f7276375640 1 --2- 172.21.15.27:0/4109754327 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f72700b6540 0x7f727006ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:54.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:54.327+0000 7f7274b72640 1 -- 172.21.15.27:0/4109754327 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 181517710 0 0) 0x7f7260001110 con 0x7f727006d730 2024-09-12T03:22:54.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:54.327+0000 7f7274b72640 1 -- 172.21.15.27:0/4109754327 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f725c003520 con 0x7f727006d730 2024-09-12T03:22:54.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:54.327+0000 7f7275b74640 1 -- 172.21.15.27:0/4109754327 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f72700b6540 msgr2=0x7f727006ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:54.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:54.327+0000 7f7275b74640 1 --2- 172.21.15.27:0/4109754327 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f72700b6540 0x7f727006ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:54.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:54.327+0000 7f7275b74640 1 -- 172.21.15.27:0/4109754327 >> v1:172.21.15.27:6789/0 conn(0x7f727006d730 legacy=0x7f72700b5e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:54.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:54.327+0000 7f7275b74640 1 -- 172.21.15.27:0/4109754327 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72700afc30 con 0x7f727015e900 2024-09-12T03:22:54.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:54.328+0000 7f7275b74640 1 --2- 172.21.15.27:0/4109754327 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f727015e900 0x7f72700af370 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f726c0097a0 tx=0x7f726c02a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1af0e798902658de server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:54.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:54.328+0000 7f7274b72640 1 -- 172.21.15.27:0/4109754327 <== mon.1 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f726c037070 con 0x7f727015e900 2024-09-12T03:22:54.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:54.328+0000 7f7274b72640 1 -- 172.21.15.27:0/4109754327 <== mon.1 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f726c032440 con 0x7f727015e900 2024-09-12T03:22:54.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:54.328+0000 7f7274b72640 1 -- 172.21.15.27:0/4109754327 <== mon.1 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f726c032810 con 0x7f727015e900 2024-09-12T03:22:54.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:54.329+0000 7f7277dd8640 1 -- 172.21.15.27:0/4109754327 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f727015e900 msgr2=0x7f72700af370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:54.331 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:54.329+0000 7f7277dd8640 1 --2- 172.21.15.27:0/4109754327 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f727015e900 0x7f72700af370 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f726c0097a0 tx=0x7f726c02a180 comp rx=0 tx=0).stop 2024-09-12T03:22:54.331 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:54.329+0000 7f7277dd8640 1 -- 172.21.15.27:0/4109754327 shutdown_connections 2024-09-12T03:22:54.331 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:54.329+0000 7f7277dd8640 1 --2- 172.21.15.27:0/4109754327 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f72700b6540 0x7f727006ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:54.331 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:54.329+0000 7f7277dd8640 1 --2- 172.21.15.27:0/4109754327 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f727015e900 0x7f72700af370 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:54.331 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:54.329+0000 7f7277dd8640 1 -- 172.21.15.27:0/4109754327 >> 172.21.15.27:0/4109754327 conn(0x7f727006cc40 msgr2=0x7f727006d020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:54.331 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:54.329+0000 7f7277dd8640 1 -- 172.21.15.27:0/4109754327 shutdown_connections 2024-09-12T03:22:54.331 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:54.329+0000 7f7277dd8640 1 -- 172.21.15.27:0/4109754327 wait complete. 2024-09-12T03:22:54.331 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:54.330+0000 7f7277dd8640 1 Processor -- start 2024-09-12T03:22:54.331 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:54.330+0000 7f7277dd8640 1 -- start start 2024-09-12T03:22:54.333 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:54.331+0000 7f7277dd8640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f727006d730 0x7f7270142390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:54.333 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:54.331+0000 7f7277dd8640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f72700b6540 0x7f72701539d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:54.333 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:54.331+0000 7f7277dd8640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f727015e900 0x7f72700b4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:54.333 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:54.331+0000 7f7277dd8640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f727006e880 con 0x7f727015e900 2024-09-12T03:22:54.334 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:54.331+0000 7f7277dd8640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f72700b01f0 con 0x7f727006d730 2024-09-12T03:22:54.334 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:54.331+0000 7f7277dd8640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f727006ea00 con 0x7f72700b6540 2024-09-12T03:22:54.334 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:54.331+0000 7f7276375640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f727015e900 0x7f72700b4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:54.334 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:54.331+0000 7f7276375640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f727015e900 0x7f72700b4600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56300/0 (socket says 172.21.15.27:56300) 2024-09-12T03:22:54.334 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:54.331+0000 7f7276375640 1 -- 172.21.15.27:0/1850347103 learned_addr learned my addr 172.21.15.27:0/1850347103 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:22:54.334 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:54.332+0000 7f7275b74640 1 --2- 172.21.15.27:0/1850347103 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f727006d730 0x7f7270142390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:54.334 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:54.332+0000 7f7275373640 1 --2- 172.21.15.27:0/1850347103 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f72700b6540 0x7f72701539d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:54.334 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:54.332+0000 7f7276375640 1 -- 172.21.15.27:0/1850347103 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f72700b6540 msgr2=0x7f72701539d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:54.334 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:54.332+0000 7f7276375640 1 --2- 172.21.15.27:0/1850347103 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f72700b6540 0x7f72701539d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:54.334 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:54.332+0000 7f7276375640 1 -- 172.21.15.27:0/1850347103 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f727006d730 msgr2=0x7f7270142390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:54.334 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:54.332+0000 7f7276375640 1 --2- 172.21.15.27:0/1850347103 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f727006d730 0x7f7270142390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:54.334 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:54.332+0000 7f7276375640 1 -- 172.21.15.27:0/1850347103 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72701543b0 con 0x7f727015e900 2024-09-12T03:22:54.334 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:54.332+0000 7f7275373640 1 --2- 172.21.15.27:0/1850347103 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f72700b6540 0x7f72701539d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:22:54.334 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:54.332+0000 7f7276375640 1 --2- 172.21.15.27:0/1850347103 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f727015e900 0x7f72700b4600 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f726800ef90 tx=0x7f726800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:54.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:54.332+0000 7f72667fc640 1 -- 172.21.15.27:0/1850347103 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7268018020 con 0x7f727015e900 2024-09-12T03:22:54.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:54.332+0000 7f72667fc640 1 -- 172.21.15.27:0/1850347103 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7268014480 con 0x7f727015e900 2024-09-12T03:22:54.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:54.332+0000 7f7277dd8640 1 -- 172.21.15.27:0/1850347103 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7270154690 con 0x7f727015e900 2024-09-12T03:22:54.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:54.332+0000 7f72667fc640 1 -- 172.21.15.27:0/1850347103 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7268014870 con 0x7f727015e900 2024-09-12T03:22:54.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:54.332+0000 7f7277dd8640 1 -- 172.21.15.27:0/1850347103 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72701428d0 con 0x7f727015e900 2024-09-12T03:22:54.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:54.333+0000 7f7277dd8640 1 -- 172.21.15.27:0/1850347103 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72700af370 con 0x7f727015e900 2024-09-12T03:22:54.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:54.334+0000 7f72667fc640 1 -- 172.21.15.27:0/1850347103 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 22) ==== 91059+0+0 (secure 0 0 0) 0x7f7268005d10 con 0x7f727015e900 2024-09-12T03:22:54.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:54.334+0000 7f72667fc640 1 --2- 172.21.15.27:0/1850347103 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f725406c490 0x7f725406e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:54.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:54.334+0000 7f7275b74640 1 --2- 172.21.15.27:0/1850347103 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f725406c490 0x7f725406e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:54.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:54.334+0000 7f72667fc640 1 -- 172.21.15.27:0/1850347103 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 5765+0+0 (secure 0 0 0) 0x7f726808c8f0 con 0x7f727015e900 2024-09-12T03:22:54.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:54.335+0000 7f7275b74640 1 --2- 172.21.15.27:0/1850347103 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f725406c490 0x7f725406e950 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f726c034000 tx=0x7f726c02e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:54.338 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:54.339+0000 7f72667fc640 1 -- 172.21.15.27:0/1850347103 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f7268022030 con 0x7f727015e900 2024-09-12T03:22:54.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:54.579+0000 7f7277dd8640 1 -- 172.21.15.27:0/1850347103 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f7270159d80 con 0x7f727015e900 2024-09-12T03:22:54.581 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:54.581+0000 7f72667fc640 1 -- 172.21.15.27:0/1850347103 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v21) ==== 190+0+147 (secure 0 0 0) 0x7f7268010040 con 0x7f727015e900 2024-09-12T03:22:54.581 INFO:teuthology.orchestra.run.smithi027.stdout:[client.0] 2024-09-12T03:22:54.581 INFO:teuthology.orchestra.run.smithi027.stdout: key = AQDVXOJmLm+5ExAAECM4CKfuVElmp/n0b02mXg== 2024-09-12T03:22:54.581 INFO:teuthology.orchestra.run.smithi027.stdout: caps mds = "allow" 2024-09-12T03:22:54.581 INFO:teuthology.orchestra.run.smithi027.stdout: caps mon = "allow r" 2024-09-12T03:22:54.582 INFO:teuthology.orchestra.run.smithi027.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-12T03:22:54.582 INFO:teuthology.orchestra.run.smithi027.stderr:updated caps for client.0 2024-09-12T03:22:54.584 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:54.583+0000 7f7277dd8640 1 -- 172.21.15.27:0/1850347103 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f725406c490 msgr2=0x7f725406e950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:54.584 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:54.583+0000 7f7277dd8640 1 --2- 172.21.15.27:0/1850347103 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f725406c490 0x7f725406e950 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f726c034000 tx=0x7f726c02e000 comp rx=0 tx=0).stop 2024-09-12T03:22:54.584 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:54.583+0000 7f7277dd8640 1 -- 172.21.15.27:0/1850347103 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f727015e900 msgr2=0x7f72700b4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:54.584 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:54.583+0000 7f7277dd8640 1 --2- 172.21.15.27:0/1850347103 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f727015e900 0x7f72700b4600 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f726800ef90 tx=0x7f726800c490 comp rx=0 tx=0).stop 2024-09-12T03:22:54.584 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:54.583+0000 7f7277dd8640 1 -- 172.21.15.27:0/1850347103 shutdown_connections 2024-09-12T03:22:54.584 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:54.583+0000 7f7277dd8640 1 --2- 172.21.15.27:0/1850347103 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f725406c490 0x7f725406e950 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:54.584 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:54.583+0000 7f7277dd8640 1 --2- 172.21.15.27:0/1850347103 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f727015e900 0x7f72700b4600 unknown :-1 s=CLOSED pgs=1413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:54.584 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:54.583+0000 7f7277dd8640 1 --2- 172.21.15.27:0/1850347103 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f72700b6540 0x7f72701539d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:54.584 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:54.583+0000 7f7277dd8640 1 --2- 172.21.15.27:0/1850347103 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f727006d730 0x7f7270142390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:54.585 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:54.583+0000 7f7277dd8640 1 -- 172.21.15.27:0/1850347103 >> 172.21.15.27:0/1850347103 conn(0x7f727006cc40 msgr2=0x7f727015da00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:54.585 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:54.584+0000 7f7277dd8640 1 -- 172.21.15.27:0/1850347103 shutdown_connections 2024-09-12T03:22:54.585 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:54.584+0000 7f7277dd8640 1 -- 172.21.15.27:0/1850347103 wait complete. 2024-09-12T03:22:54.594 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-12T03:22:54.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:54.696+0000 7f7b56ced640 1 Processor -- start 2024-09-12T03:22:54.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:54.696+0000 7f7b56ced640 1 -- start start 2024-09-12T03:22:54.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:54.697+0000 7f7b56ced640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f7b500cda40 0x7f7b5015ca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:54.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:54.697+0000 7f7b56ced640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f7b50146010 0x7f7b500b4040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:54.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:54.697+0000 7f7b56ced640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7b500b4d70 con 0x7f7b50145c30 2024-09-12T03:22:54.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:54.697+0000 7f7b56ced640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f7b500b5050 con 0x7f7b50146010 2024-09-12T03:22:54.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:54.697+0000 7f7b56ced640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f7b5006d140 con 0x7f7b500cda40 2024-09-12T03:22:54.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:54.698+0000 7f7b4ffff640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f7b500cda40 0x7f7b5015ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:54.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:54.698+0000 7f7b5528a640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f7b50146010 0x7f7b500b4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:54.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:54.698+0000 7f7b54a89640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f7b50145c30 0x7f7b500cd330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.27:49176/0 (socket says 172.21.15.27:49176) 2024-09-12T03:22:54.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:54.698+0000 7f7b54a89640 1 -- 172.21.15.27:0/1977347474 learned_addr learned my addr 172.21.15.27:0/1977347474 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:22:54.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:54.698+0000 7f7b5528a640 1 -- 172.21.15.27:0/1977347474 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f7b500cda40 msgr2=0x7f7b5015ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:54.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:54.698+0000 7f7b5528a640 1 --2- 172.21.15.27:0/1977347474 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f7b500cda40 0x7f7b5015ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:54.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:54.698+0000 7f7b5528a640 1 -- 172.21.15.27:0/1977347474 >> v1:172.21.15.45:6789/0 conn(0x7f7b50145c30 legacy=0x7f7b500cd330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:54.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:54.698+0000 7f7b5528a640 1 -- 172.21.15.27:0/1977347474 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b5006d420 con 0x7f7b50146010 2024-09-12T03:22:54.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:54.698+0000 7f7b4ffff640 1 --2- 172.21.15.27:0/1977347474 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f7b500cda40 0x7f7b5015ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:22:54.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:54.698+0000 7f7b4f7fe640 1 -- 172.21.15.27:0/1977347474 <== mon.1 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3716806083 0 0) 0x7f7b500b4d70 con 0x7f7b50145c30 2024-09-12T03:22:54.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:54.699+0000 7f7b5528a640 1 --2- 172.21.15.27:0/1977347474 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7b50146010 0x7f7b500b4040 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f7b380097d0 tx=0x7f7b3802a130 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a4eac247cd497585 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:54.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:54.699+0000 7f7b4f7fe640 1 -- 172.21.15.27:0/1977347474 <== mon.1 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b38037070 con 0x7f7b50146010 2024-09-12T03:22:54.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:54.699+0000 7f7b4f7fe640 1 -- 172.21.15.27:0/1977347474 <== mon.1 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b3802ae40 con 0x7f7b50146010 2024-09-12T03:22:54.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:54.699+0000 7f7b4f7fe640 1 -- 172.21.15.27:0/1977347474 <== mon.1 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b38032660 con 0x7f7b50146010 2024-09-12T03:22:54.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:54.700+0000 7f7b56ced640 1 -- 172.21.15.27:0/1977347474 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7b50146010 msgr2=0x7f7b500b4040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:54.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:54.700+0000 7f7b56ced640 1 --2- 172.21.15.27:0/1977347474 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7b50146010 0x7f7b500b4040 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f7b380097d0 tx=0x7f7b3802a130 comp rx=0 tx=0).stop 2024-09-12T03:22:54.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:54.700+0000 7f7b56ced640 1 -- 172.21.15.27:0/1977347474 shutdown_connections 2024-09-12T03:22:54.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:54.700+0000 7f7b56ced640 1 --2- 172.21.15.27:0/1977347474 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7b50146010 0x7f7b500b4040 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:54.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:54.700+0000 7f7b56ced640 1 --2- 172.21.15.27:0/1977347474 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f7b500cda40 0x7f7b5015ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:54.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:54.700+0000 7f7b56ced640 1 -- 172.21.15.27:0/1977347474 >> 172.21.15.27:0/1977347474 conn(0x7f7b50062c60 msgr2=0x7f7b5006bb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:54.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:54.700+0000 7f7b56ced640 1 -- 172.21.15.27:0/1977347474 shutdown_connections 2024-09-12T03:22:54.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:54.700+0000 7f7b56ced640 1 -- 172.21.15.27:0/1977347474 wait complete. 2024-09-12T03:22:54.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:54.701+0000 7f7b56ced640 1 Processor -- start 2024-09-12T03:22:54.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:54.701+0000 7f7b56ced640 1 -- start start 2024-09-12T03:22:54.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:54.702+0000 7f7b56ced640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7b500cda40 0x7f7b5015be80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:54.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:54.702+0000 7f7b56ced640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f7b50145c30 0x7f7b500c98e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:54.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:54.702+0000 7f7b56ced640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7b50146010 0x7f7b50141980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:54.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:54.702+0000 7f7b56ced640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f7b500b4700 con 0x7f7b50146010 2024-09-12T03:22:54.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:54.702+0000 7f7b56ced640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f7b500b4580 con 0x7f7b500cda40 2024-09-12T03:22:54.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:54.702+0000 7f7b56ced640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f7b500b4880 con 0x7f7b50145c30 2024-09-12T03:22:54.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:54.702+0000 7f7b5528a640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7b50146010 0x7f7b50141980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:54.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:54.702+0000 7f7b5528a640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7b50146010 0x7f7b50141980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56312/0 (socket says 172.21.15.27:56312) 2024-09-12T03:22:54.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:54.702+0000 7f7b5528a640 1 -- 172.21.15.27:0/2719726833 learned_addr learned my addr 172.21.15.27:0/2719726833 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:22:54.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:54.702+0000 7f7b54a89640 1 --2- 172.21.15.27:0/2719726833 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7b500cda40 0x7f7b5015be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:54.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:54.702+0000 7f7b4ffff640 1 --2- 172.21.15.27:0/2719726833 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f7b50145c30 0x7f7b500c98e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:54.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:54.702+0000 7f7b5528a640 1 -- 172.21.15.27:0/2719726833 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f7b50145c30 msgr2=0x7f7b500c98e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:54.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:54.702+0000 7f7b5528a640 1 --2- 172.21.15.27:0/2719726833 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f7b50145c30 0x7f7b500c98e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:54.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:54.702+0000 7f7b5528a640 1 -- 172.21.15.27:0/2719726833 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7b500cda40 msgr2=0x7f7b5015be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:54.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:54.702+0000 7f7b5528a640 1 --2- 172.21.15.27:0/2719726833 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7b500cda40 0x7f7b5015be80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:54.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:54.702+0000 7f7b5528a640 1 -- 172.21.15.27:0/2719726833 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b5015c3c0 con 0x7f7b50146010 2024-09-12T03:22:54.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:54.703+0000 7f7b5528a640 1 --2- 172.21.15.27:0/2719726833 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7b50146010 0x7f7b50141980 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7f7b3802a720 tx=0x7f7b380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:54.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:54.703+0000 7f7b4d7fa640 1 -- 172.21.15.27:0/2719726833 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b38041030 con 0x7f7b50146010 2024-09-12T03:22:54.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:54.703+0000 7f7b56ced640 1 -- 172.21.15.27:0/2719726833 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b500ca1e0 con 0x7f7b50146010 2024-09-12T03:22:54.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:54.703+0000 7f7b4d7fa640 1 -- 172.21.15.27:0/2719726833 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b3802ac40 con 0x7f7b50146010 2024-09-12T03:22:54.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:54.703+0000 7f7b4d7fa640 1 -- 172.21.15.27:0/2719726833 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b38037040 con 0x7f7b50146010 2024-09-12T03:22:54.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:54.703+0000 7f7b56ced640 1 -- 172.21.15.27:0/2719726833 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b500711b0 con 0x7f7b50146010 2024-09-12T03:22:54.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:54.704+0000 7f7b56ced640 1 -- 172.21.15.27:0/2719726833 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b5015ca00 con 0x7f7b50146010 2024-09-12T03:22:54.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:54.704+0000 7f7b4d7fa640 1 -- 172.21.15.27:0/2719726833 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 22) ==== 91059+0+0 (secure 0 0 0) 0x7f7b38044070 con 0x7f7b50146010 2024-09-12T03:22:54.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:54.705+0000 7f7b4d7fa640 1 --2- 172.21.15.27:0/2719726833 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f7b3006c460 0x7f7b3006e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:54.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:54.705+0000 7f7b4d7fa640 1 -- 172.21.15.27:0/2719726833 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 5765+0+0 (secure 0 0 0) 0x7f7b380abec0 con 0x7f7b50146010 2024-09-12T03:22:54.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:54.705+0000 7f7b54a89640 1 --2- 172.21.15.27:0/2719726833 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f7b3006c460 0x7f7b3006e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:54.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:54.705+0000 7f7b54a89640 1 --2- 172.21.15.27:0/2719726833 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f7b3006c460 0x7f7b3006e920 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f7b480012e0 tx=0x7f7b48007ed0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:54.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:54.709+0000 7f7b4d7fa640 1 -- 172.21.15.27:0/2719726833 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f7b3804b070 con 0x7f7b50146010 2024-09-12T03:22:54.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:54.922+0000 7f7b56ced640 1 -- 172.21.15.27:0/2719726833 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7b5006e6f0 con 0x7f7b50146010 2024-09-12T03:22:54.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:54.922+0000 7f7b4d7fa640 1 -- 172.21.15.27:0/2719726833 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 82 v82) ==== 94+0+5224 (secure 0 0 0) 0x7f7b38044e40 con 0x7f7b50146010 2024-09-12T03:22:54.922 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:22:54.923 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":82,"btime":"2024-09-12T03:22:52:853366+0000","default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6689,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/2285501348","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2285501348},{"type":"v1","addr":"172.21.15.45:6833","nonce":2285501348}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79},{"gid":7453,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/1872420883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":1872420883},{"type":"v1","addr":"172.21.15.45:6835","nonce":1872420883}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79},{"gid":8193,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6834/3968488955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":3968488955},{"type":"v1","addr":"172.21.15.27:6834","nonce":3968488955}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79}],"filesystems":[{"mdsmap":{"epoch":82,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-12T03:22:50.200339+0000","modified":"2024-09-12T03:22:52.204820+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8196},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8196":{"gid":8196,"name":"a","rank":0,"incarnation":80,"state":"up:active","state_seq":3,"addr":"172.21.15.27:6835/389118996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":389118996},{"type":"v1","addr":"172.21.15.27:6835","nonce":389118996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8196,"qdb_cluster":[8196]},"id":11}]} 2024-09-12T03:22:54.923 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 82 2024-09-12T03:22:54.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:54.924+0000 7f7b56ced640 1 -- 172.21.15.27:0/2719726833 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f7b3006c460 msgr2=0x7f7b3006e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:54.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:54.924+0000 7f7b56ced640 1 --2- 172.21.15.27:0/2719726833 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f7b3006c460 0x7f7b3006e920 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f7b480012e0 tx=0x7f7b48007ed0 comp rx=0 tx=0).stop 2024-09-12T03:22:54.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:54.924+0000 7f7b56ced640 1 -- 172.21.15.27:0/2719726833 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7b50146010 msgr2=0x7f7b50141980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:54.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:54.924+0000 7f7b56ced640 1 --2- 172.21.15.27:0/2719726833 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7b50146010 0x7f7b50141980 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7f7b3802a720 tx=0x7f7b380046c0 comp rx=0 tx=0).stop 2024-09-12T03:22:54.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:54.924+0000 7f7b56ced640 1 -- 172.21.15.27:0/2719726833 shutdown_connections 2024-09-12T03:22:54.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:54.924+0000 7f7b56ced640 1 --2- 172.21.15.27:0/2719726833 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f7b3006c460 0x7f7b3006e920 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:54.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:54.924+0000 7f7b56ced640 1 --2- 172.21.15.27:0/2719726833 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7b50146010 0x7f7b50141980 unknown :-1 s=CLOSED pgs=1414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:54.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:54.924+0000 7f7b56ced640 1 --2- 172.21.15.27:0/2719726833 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f7b50145c30 0x7f7b500c98e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:54.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:54.924+0000 7f7b56ced640 1 --2- 172.21.15.27:0/2719726833 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7b500cda40 0x7f7b5015be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:54.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:54.924+0000 7f7b56ced640 1 -- 172.21.15.27:0/2719726833 >> 172.21.15.27:0/2719726833 conn(0x7f7b50062c60 msgr2=0x7f7b5006bb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:54.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:54.924+0000 7f7b56ced640 1 -- 172.21.15.27:0/2719726833 shutdown_connections 2024-09-12T03:22:54.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:54.924+0000 7f7b56ced640 1 -- 172.21.15.27:0/2719726833 wait complete. 2024-09-12T03:22:55.338 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 82, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-12T03:22:50.200339+0000', 'modified': '2024-09-12T03:22:52.204820+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8196}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8196': {'gid': 8196, 'name': 'a', 'rank': 0, 'incarnation': 80, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.27:6835/389118996', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6833', 'nonce': 389118996}, {'type': 'v1', 'addr': '172.21.15.27:6835', 'nonce': 389118996}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [23], 'metadata_pool': 22, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 8196, 'qdb_cluster': [8196]} 2024-09-12T03:22:55.338 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-12T03:22:55.338 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-09-12T03:22:55.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.444+0000 7fdafa30c640 1 Processor -- start 2024-09-12T03:22:55.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.444+0000 7fdafa30c640 1 -- start start 2024-09-12T03:22:55.446 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.445+0000 7fdafa30c640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fdaf415a8c0 0x7fdaf40cdb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:55.446 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.445+0000 7fdafa30c640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fdaf4071be0 0x7fdaf406e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:55.446 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.445+0000 7fdafa30c640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdaf406daf0 con 0x7fdaf415aca0 2024-09-12T03:22:55.446 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.445+0000 7fdafa30c640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7fdaf406d8a0 con 0x7fdaf415a8c0 2024-09-12T03:22:55.446 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.445+0000 7fdafa30c640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fdaf406ddd0 con 0x7fdaf4071be0 2024-09-12T03:22:55.446 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.445+0000 7fdaf3fff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fdaf415a8c0 0x7fdaf40cdb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:55.446 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.445+0000 7fdaf3fff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fdaf415a8c0 0x7fdaf40cdb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56316/0 (socket says 172.21.15.27:56316) 2024-09-12T03:22:55.446 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.445+0000 7fdaf3fff640 1 -- 172.21.15.27:0/3457410724 learned_addr learned my addr 172.21.15.27:0/3457410724 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:22:55.447 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.446+0000 7fdaf37fe640 1 --2- 172.21.15.27:0/3457410724 >> v2:172.21.15.45:3300/0 conn(0x7fdaf4071be0 0x7fdaf406e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:55.447 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.446+0000 7fdaf3fff640 1 -- 172.21.15.27:0/3457410724 >> v2:172.21.15.45:3300/0 conn(0x7fdaf4071be0 msgr2=0x7fdaf406e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:55.447 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.446+0000 7fdaf3fff640 1 --2- 172.21.15.27:0/3457410724 >> v2:172.21.15.45:3300/0 conn(0x7fdaf4071be0 0x7fdaf406e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:55.447 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.446+0000 7fdaf3fff640 1 -- 172.21.15.27:0/3457410724 >> v1:172.21.15.45:6789/0 conn(0x7fdaf415aca0 legacy=0x7fdaf40b4710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:55.447 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.446+0000 7fdaf3fff640 1 -- 172.21.15.27:0/3457410724 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdaf406e0b0 con 0x7fdaf415a8c0 2024-09-12T03:22:55.447 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.446+0000 7fdaf37fe640 1 --2- 172.21.15.27:0/3457410724 >> v2:172.21.15.45:3300/0 conn(0x7fdaf4071be0 0x7fdaf406e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:22:55.447 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.446+0000 7fdaf3fff640 1 --2- 172.21.15.27:0/3457410724 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdaf415a8c0 0x7fdaf40cdb30 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7fdaf40d0480 tx=0x7fdae402c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f23164004876f436 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:55.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.447+0000 7fdaf2ffd640 1 -- 172.21.15.27:0/3457410724 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdae40325a0 con 0x7fdaf415a8c0 2024-09-12T03:22:55.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.447+0000 7fdaf2ffd640 1 -- 172.21.15.27:0/3457410724 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdae4038070 con 0x7fdaf415a8c0 2024-09-12T03:22:55.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.447+0000 7fdaf2ffd640 1 -- 172.21.15.27:0/3457410724 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdae4032960 con 0x7fdaf415a8c0 2024-09-12T03:22:55.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.447+0000 7fdafa30c640 1 -- 172.21.15.27:0/3457410724 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdaf415a8c0 msgr2=0x7fdaf40cdb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:55.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.447+0000 7fdafa30c640 1 --2- 172.21.15.27:0/3457410724 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdaf415a8c0 0x7fdaf40cdb30 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7fdaf40d0480 tx=0x7fdae402c8e0 comp rx=0 tx=0).stop 2024-09-12T03:22:55.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.447+0000 7fdafa30c640 1 -- 172.21.15.27:0/3457410724 shutdown_connections 2024-09-12T03:22:55.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.447+0000 7fdafa30c640 1 --2- 172.21.15.27:0/3457410724 >> v2:172.21.15.45:3300/0 conn(0x7fdaf4071be0 0x7fdaf406e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:55.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.447+0000 7fdafa30c640 1 --2- 172.21.15.27:0/3457410724 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdaf415a8c0 0x7fdaf40cdb30 unknown :-1 s=CLOSED pgs=1415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:55.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.447+0000 7fdafa30c640 1 -- 172.21.15.27:0/3457410724 >> 172.21.15.27:0/3457410724 conn(0x7fdaf4062c60 msgr2=0x7fdaf406bb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:55.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.447+0000 7fdafa30c640 1 -- 172.21.15.27:0/3457410724 shutdown_connections 2024-09-12T03:22:55.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.447+0000 7fdafa30c640 1 -- 172.21.15.27:0/3457410724 wait complete. 2024-09-12T03:22:55.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.448+0000 7fdafa30c640 1 Processor -- start 2024-09-12T03:22:55.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.449+0000 7fdafa30c640 1 -- start start 2024-09-12T03:22:55.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.449+0000 7fdafa30c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdaf4071be0 0x7fdaf40c92c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:55.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.449+0000 7fdafa30c640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fdaf415a8c0 0x7fdaf4157000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:55.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.449+0000 7fdafa30c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdaf415aca0 0x7fdaf415e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:55.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.449+0000 7fdafa30c640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf40b36e0 con 0x7fdaf415aca0 2024-09-12T03:22:55.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.449+0000 7fdafa30c640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf40b3560 con 0x7fdaf4071be0 2024-09-12T03:22:55.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.449+0000 7fdafa30c640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fdaf40b4e20 con 0x7fdaf415a8c0 2024-09-12T03:22:55.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.450+0000 7fdaf88a9640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdaf415aca0 0x7fdaf415e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:55.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.450+0000 7fdaf88a9640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdaf415aca0 0x7fdaf415e130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56326/0 (socket says 172.21.15.27:56326) 2024-09-12T03:22:55.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.450+0000 7fdaf88a9640 1 -- 172.21.15.27:0/1673632465 learned_addr learned my addr 172.21.15.27:0/1673632465 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:22:55.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.450+0000 7fdaf3fff640 1 --2- 172.21.15.27:0/1673632465 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdaf4071be0 0x7fdaf40c92c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:55.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.450+0000 7fdaf37fe640 1 --2- 172.21.15.27:0/1673632465 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fdaf415a8c0 0x7fdaf4157000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:55.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.450+0000 7fdaf88a9640 1 -- 172.21.15.27:0/1673632465 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fdaf415a8c0 msgr2=0x7fdaf4157000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:55.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.450+0000 7fdaf88a9640 1 --2- 172.21.15.27:0/1673632465 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fdaf415a8c0 0x7fdaf4157000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:55.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.450+0000 7fdaf88a9640 1 -- 172.21.15.27:0/1673632465 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdaf4071be0 msgr2=0x7fdaf40c92c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:55.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.450+0000 7fdaf88a9640 1 --2- 172.21.15.27:0/1673632465 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdaf4071be0 0x7fdaf40c92c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:55.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.450+0000 7fdaf88a9640 1 -- 172.21.15.27:0/1673632465 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdaf40afb00 con 0x7fdaf415aca0 2024-09-12T03:22:55.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.450+0000 7fdaf37fe640 1 --2- 172.21.15.27:0/1673632465 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fdaf415a8c0 0x7fdaf4157000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:22:55.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.450+0000 7fdaf88a9640 1 --2- 172.21.15.27:0/1673632465 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdaf415aca0 0x7fdaf415e130 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7fdaec00b9e0 tx=0x7fdaec00bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:55.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.450+0000 7fdaf0ff9640 1 -- 172.21.15.27:0/1673632465 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaec003b70 con 0x7fdaf415aca0 2024-09-12T03:22:55.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.450+0000 7fdaf0ff9640 1 -- 172.21.15.27:0/1673632465 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdaec003d10 con 0x7fdaf415aca0 2024-09-12T03:22:55.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.450+0000 7fdafa30c640 1 -- 172.21.15.27:0/1673632465 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdaf40ca480 con 0x7fdaf415aca0 2024-09-12T03:22:55.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.450+0000 7fdaf0ff9640 1 -- 172.21.15.27:0/1673632465 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaec00cb60 con 0x7fdaf415aca0 2024-09-12T03:22:55.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.450+0000 7fdafa30c640 1 -- 172.21.15.27:0/1673632465 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdaf40caa40 con 0x7fdaf415aca0 2024-09-12T03:22:55.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.452+0000 7fdaf0ff9640 1 -- 172.21.15.27:0/1673632465 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 22) ==== 91059+0+0 (secure 0 0 0) 0x7fdaec00cd00 con 0x7fdaf415aca0 2024-09-12T03:22:55.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.452+0000 7fdaf0ff9640 1 --2- 172.21.15.27:0/1673632465 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fdae006c490 0x7fdae006e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:55.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.452+0000 7fdaf0ff9640 1 -- 172.21.15.27:0/1673632465 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 5765+0+0 (secure 0 0 0) 0x7fdaec08b890 con 0x7fdaf415aca0 2024-09-12T03:22:55.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.452+0000 7fdaf3fff640 1 --2- 172.21.15.27:0/1673632465 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fdae006c490 0x7fdae006e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:55.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.453+0000 7fdaf3fff640 1 --2- 172.21.15.27:0/1673632465 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fdae006c490 0x7fdae006e950 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fdae4037000 tx=0x7fdae4032000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:55.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.453+0000 7fdafd09d740 1 Processor -- start 2024-09-12T03:22:55.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.453+0000 7fdafd09d740 1 -- start start 2024-09-12T03:22:55.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.454+0000 7fdafd09d740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f0e14862c0 0x55f0e14cba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:55.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.454+0000 7fdafd09d740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f0e1483a80 0x55f0e14866a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:55.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.454+0000 7fdafd09d740 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x55f0e1487a50 0x55f0e14808b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:55.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.454+0000 7fdafd09d740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55f0e13e8520 con 0x55f0e1483a80 2024-09-12T03:22:55.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.454+0000 7fdafd09d740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55f0e13903e0 con 0x55f0e14862c0 2024-09-12T03:22:55.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.454+0000 7fdafd09d740 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x55f0e13e44e0 con 0x55f0e1487a50 2024-09-12T03:22:55.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.454+0000 7fdaf37fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f0e1483a80 0x55f0e14866a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:55.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.454+0000 7fdaf37fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f0e1483a80 0x55f0e14866a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56340/0 (socket says 172.21.15.27:56340) 2024-09-12T03:22:55.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.454+0000 7fdaf37fe640 1 -- 172.21.15.27:0/2111385406 learned_addr learned my addr 172.21.15.27:0/2111385406 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:22:55.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.454+0000 7fdaf3fff640 1 --2- 172.21.15.27:0/2111385406 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f0e14862c0 0x55f0e14cba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:55.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.454+0000 7fdaf37fe640 1 -- 172.21.15.27:0/2111385406 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x55f0e1487a50 msgr2=0x55f0e14808b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:55.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.454+0000 7fdaf37fe640 1 --2- 172.21.15.27:0/2111385406 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x55f0e1487a50 0x55f0e14808b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:55.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.454+0000 7fdaf37fe640 1 -- 172.21.15.27:0/2111385406 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f0e14862c0 msgr2=0x55f0e14cba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:55.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.454+0000 7fdaf37fe640 1 --2- 172.21.15.27:0/2111385406 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f0e14862c0 0x55f0e14cba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:55.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.455+0000 7fdaf37fe640 1 -- 172.21.15.27:0/2111385406 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f0e13ab2f0 con 0x55f0e1483a80 2024-09-12T03:22:55.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.455+0000 7fdaf37fe640 1 --2- 172.21.15.27:0/2111385406 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f0e1483a80 0x55f0e14866a0 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7fdae800bad0 tx=0x7fdae800bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ab8c495c601f26c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:55.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.455+0000 7fdad9ffb640 1 -- 172.21.15.27:0/2111385406 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdae8016070 con 0x55f0e1483a80 2024-09-12T03:22:55.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.455+0000 7fdad9ffb640 1 -- 172.21.15.27:0/2111385406 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdae800dd90 con 0x55f0e1483a80 2024-09-12T03:22:55.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.455+0000 7fdad9ffb640 1 -- 172.21.15.27:0/2111385406 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdae8012680 con 0x55f0e1483a80 2024-09-12T03:22:55.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.455+0000 7fdafd09d740 1 -- 172.21.15.27:0/2111385406 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f0e1483a80 msgr2=0x55f0e14866a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:55.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.455+0000 7fdafd09d740 1 --2- 172.21.15.27:0/2111385406 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f0e1483a80 0x55f0e14866a0 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7fdae800bad0 tx=0x7fdae800bf90 comp rx=0 tx=0).stop 2024-09-12T03:22:55.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.455+0000 7fdafd09d740 1 -- 172.21.15.27:0/2111385406 shutdown_connections 2024-09-12T03:22:55.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.455+0000 7fdafd09d740 1 --2- 172.21.15.27:0/2111385406 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x55f0e1487a50 0x55f0e14808b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:55.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.455+0000 7fdafd09d740 1 --2- 172.21.15.27:0/2111385406 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f0e1483a80 0x55f0e14866a0 unknown :-1 s=CLOSED pgs=1417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:55.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.455+0000 7fdafd09d740 1 --2- 172.21.15.27:0/2111385406 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f0e14862c0 0x55f0e14cba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:55.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.455+0000 7fdafd09d740 1 -- 172.21.15.27:0/2111385406 >> 172.21.15.27:0/2111385406 conn(0x55f0e1482a80 msgr2=0x55f0e14caf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:55.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.456+0000 7fdafd09d740 1 -- 172.21.15.27:0/2111385406 shutdown_connections 2024-09-12T03:22:55.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.456+0000 7fdafd09d740 1 -- 172.21.15.27:0/2111385406 wait complete. 2024-09-12T03:22:55.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.456+0000 7fdafd09d740 1 -- start start 2024-09-12T03:22:55.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.456+0000 7fdafd09d740 1 Processor -- start 2024-09-12T03:22:55.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.457+0000 7fdafd09d740 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x55f0e14862c0 0x55f0e14addc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:55.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.457+0000 7fdafd09d740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f0e1487a50 0x55f0e149a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:55.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.457+0000 7fdafd09d740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f0e14ac2f0 0x55f0e14ac770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:55.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.457+0000 7fdafd09d740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55f0e1481f20 con 0x55f0e1487a50 2024-09-12T03:22:55.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.457+0000 7fdafd09d740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55f0e1371450 con 0x55f0e14ac2f0 2024-09-12T03:22:55.468 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.457+0000 7fdafd09d740 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x55f0e14cc290 con 0x55f0e14862c0 2024-09-12T03:22:55.468 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.457+0000 7fdafd09d740 10 client.0 _ll_register_callbacks cb 0x55f0e10faac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:22:55.468 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.457+0000 7fdaf37fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f0e1487a50 0x55f0e149a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:55.468 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.457+0000 7fdaf37fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f0e1487a50 0x55f0e149a9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56342/0 (socket says 172.21.15.27:56342) 2024-09-12T03:22:55.468 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.457+0000 7fdaf37fe640 1 -- 172.21.15.27:0/1383926964 learned_addr learned my addr 172.21.15.27:0/1383926964 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:22:55.468 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.457+0000 7fdaf3fff640 1 --2- 172.21.15.27:0/1383926964 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x55f0e14862c0 0x55f0e14addc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:55.468 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.457+0000 7fdaf88a9640 1 --2- 172.21.15.27:0/1383926964 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f0e14ac2f0 0x55f0e14ac770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:55.468 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.458+0000 7fdaf37fe640 1 -- 172.21.15.27:0/1383926964 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x55f0e14862c0 msgr2=0x55f0e14addc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:55.468 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.458+0000 7fdaf37fe640 1 --2- 172.21.15.27:0/1383926964 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x55f0e14862c0 0x55f0e14addc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:55.468 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.458+0000 7fdaf37fe640 1 -- 172.21.15.27:0/1383926964 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f0e14ac2f0 msgr2=0x55f0e14ac770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:55.468 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.458+0000 7fdaf37fe640 1 --2- 172.21.15.27:0/1383926964 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f0e14ac2f0 0x55f0e14ac770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:55.468 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.458+0000 7fdaf37fe640 1 -- 172.21.15.27:0/1383926964 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f0e10fb2d0 con 0x55f0e1487a50 2024-09-12T03:22:55.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.458+0000 7fdaf37fe640 1 --2- 172.21.15.27:0/1383926964 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f0e1487a50 0x55f0e149a9d0 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7fdae8006fd0 tx=0x7fdae8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f2297e813bcf8d5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:55.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.458+0000 7fdad2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-12T03:22:55.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.458+0000 7fdad2ffd640 1 -- 172.21.15.27:0/1383926964 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdae8016040 con 0x55f0e1487a50 2024-09-12T03:22:55.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.458+0000 7fdad2ffd640 1 -- 172.21.15.27:0/1383926964 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(138..138 src has 1..138) ==== 5765+0+0 (secure 0 0 0) 0x7fdae8025b30 con 0x55f0e1487a50 2024-09-12T03:22:55.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.458+0000 7fdafd09d740 1 -- 172.21.15.27:0/1383926964 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdae800dd90 con 0x55f0e1487a50 2024-09-12T03:22:55.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.458+0000 7fdad2ffd640 1 -- 172.21.15.27:0/1383926964 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7fdae802c050 con 0x55f0e1487a50 2024-09-12T03:22:55.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.458+0000 7fdafd09d740 10 client.8316 fetch_fsmap learned FSMap version 82 2024-09-12T03:22:55.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.458+0000 7fdafd09d740 1 -- 172.21.15.27:0/1383926964 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=82,monmap=2+}) -- 0x55f0e11c56b0 con 0x55f0e1487a50 2024-09-12T03:22:55.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.458+0000 7fdad2ffd640 1 -- 172.21.15.27:0/1383926964 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdae800dd90 con 0x55f0e1487a50 2024-09-12T03:22:55.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.458+0000 7fdad2ffd640 1 -- 172.21.15.27:0/1383926964 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 82) ==== 3502+0+0 (secure 0 0 0) 0x7fdae802f020 con 0x55f0e1487a50 2024-09-12T03:22:55.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.458+0000 7fdafd09d740 10 client.8316 fetch_fsmap finished waiting for FSMap version 82 2024-09-12T03:22:55.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.458+0000 7fdafd09d740 10 client.8316 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-12T03:22:55.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.458+0000 7fdafd09d740 20 client.8316 populate_metadata read hostname 'smithi027' 2024-09-12T03:22:55.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.458+0000 7fdafd09d740 1 --2- 172.21.15.27:0/1383926964 >> [v2:172.21.15.27:6833/389118996,v1:172.21.15.27:6835/389118996] conn(0x55f0e14afc90 0x55f0e14b0070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:55.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.458+0000 7fdafd09d740 4 client.8316 mds_command: new command op to 8196 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:22:55.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.458+0000 7fdafd09d740 1 -- 172.21.15.27:0/1383926964 --> [v2:172.21.15.27:6833/389118996,v1:172.21.15.27:6835/389118996] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f0e13f5c50 con 0x55f0e14afc90 2024-09-12T03:22:55.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.459+0000 7fdaf3fff640 1 --2- 172.21.15.27:0/1383926964 >> [v2:172.21.15.27:6833/389118996,v1:172.21.15.27:6835/389118996] conn(0x55f0e14afc90 0x55f0e14b0070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:55.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.459+0000 7fdad2ffd640 10 client.8316 ms_handle_connect on v2:172.21.15.27:6833/389118996 2024-09-12T03:22:55.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.459+0000 7fdaf3fff640 1 --2- 172.21.15.27:0/1383926964 >> [v2:172.21.15.27:6833/389118996,v1:172.21.15.27:6835/389118996] conn(0x55f0e14afc90 0x55f0e14b0070 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3aef06d75999008b server_cookie=3a442ff73c42431c in_seq=0 out_seq=0 2024-09-12T03:22:55.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.460+0000 7fdad2ffd640 1 -- 172.21.15.27:0/1383926964 <== mds.0 v2:172.21.15.27:6833/389118996 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f0e13f5c50 con 0x55f0e14afc90 2024-09-12T03:22:55.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.460+0000 7fdad2ffd640 10 client.8316 handle_command_reply: tid=0 2024-09-12T03:22:55.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.460+0000 7fdafd09d740 1 client.8316 shutdown 2024-09-12T03:22:55.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.460+0000 7fdafd09d740 1 -- 172.21.15.27:0/1383926964 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f0e1487a50 msgr2=0x55f0e149a9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:55.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.460+0000 7fdafd09d740 1 --2- 172.21.15.27:0/1383926964 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f0e1487a50 0x55f0e149a9d0 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7fdae8006fd0 tx=0x7fdae8004730 comp rx=0 tx=0).stop 2024-09-12T03:22:55.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.460+0000 7fdafd09d740 1 -- 172.21.15.27:0/1383926964 shutdown_connections 2024-09-12T03:22:55.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.460+0000 7fdafd09d740 1 --2- 172.21.15.27:0/1383926964 >> [v2:172.21.15.27:6833/389118996,v1:172.21.15.27:6835/389118996] conn(0x55f0e14afc90 0x55f0e14b0070 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:55.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.460+0000 7fdafd09d740 1 --2- 172.21.15.27:0/1383926964 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f0e14ac2f0 0x55f0e14ac770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:55.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.460+0000 7fdafd09d740 1 --2- 172.21.15.27:0/1383926964 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f0e1487a50 0x55f0e149a9d0 unknown :-1 s=CLOSED pgs=1418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:55.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.460+0000 7fdafd09d740 1 --2- 172.21.15.27:0/1383926964 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x55f0e14862c0 0x55f0e14addc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:55.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.460+0000 7fdad2ffd640 0 client.8316 ms_handle_reset on v2:172.21.15.27:6833/389118996 2024-09-12T03:22:55.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.460+0000 7fdafd09d740 1 -- 172.21.15.27:0/1383926964 >> 172.21.15.27:0/1383926964 conn(0x55f0e1483a80 msgr2=0x55f0e14c8ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:55.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.460+0000 7fdafd09d740 1 -- 172.21.15.27:0/1383926964 shutdown_connections 2024-09-12T03:22:55.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.460+0000 7fdafd09d740 1 -- 172.21.15.27:0/1383926964 wait complete. 2024-09-12T03:22:55.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.460+0000 7fdafd09d740 20 client.8316 trim_cache size 0 max 16384 2024-09-12T03:22:55.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.478+0000 7fdafd09d740 1 Processor -- start 2024-09-12T03:22:55.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.478+0000 7fdafd09d740 1 -- start start 2024-09-12T03:22:55.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.479+0000 7fdafd09d740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f0e14862c0 0x55f0e14a6fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:55.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.479+0000 7fdafd09d740 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x55f0e1487a50 0x55f0e14c9210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:55.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.479+0000 7fdafd09d740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f0e14ac2f0 0x55f0e14c9750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:55.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.479+0000 7fdafd09d740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf41ffd00 con 0x55f0e14862c0 2024-09-12T03:22:55.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.479+0000 7fdafd09d740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf40af5c0 con 0x55f0e14ac2f0 2024-09-12T03:22:55.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.479+0000 7fdafd09d740 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fdaf40c6110 con 0x55f0e1487a50 2024-09-12T03:22:55.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.479+0000 7fdaf3fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f0e14862c0 0x55f0e14a6fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:55.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.479+0000 7fdaf3fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f0e14862c0 0x55f0e14a6fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56348/0 (socket says 172.21.15.27:56348) 2024-09-12T03:22:55.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.479+0000 7fdaf3fff640 1 -- 172.21.15.27:0/1419744189 learned_addr learned my addr 172.21.15.27:0/1419744189 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:22:55.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.479+0000 7fdaf3fff640 1 -- 172.21.15.27:0/1419744189 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x55f0e1487a50 msgr2=0x55f0e14c9210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:55.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.479+0000 7fdaf3fff640 1 --2- 172.21.15.27:0/1419744189 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x55f0e1487a50 0x55f0e14c9210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:55.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.479+0000 7fdaf3fff640 1 -- 172.21.15.27:0/1419744189 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f0e14ac2f0 msgr2=0x55f0e14c9750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:55.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.479+0000 7fdaf3fff640 1 --2- 172.21.15.27:0/1419744189 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f0e14ac2f0 0x55f0e14c9750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:55.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.479+0000 7fdaf3fff640 1 -- 172.21.15.27:0/1419744189 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f0e1481aa0 con 0x55f0e14862c0 2024-09-12T03:22:55.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.479+0000 7fdaf3fff640 1 --2- 172.21.15.27:0/1419744189 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f0e14862c0 0x55f0e14a6fb0 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7fdae400bc00 tx=0x7fdae400b450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2312f423d6696409 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:55.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.479+0000 7fdad97fa640 1 -- 172.21.15.27:0/1419744189 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdae4032d70 con 0x55f0e14862c0 2024-09-12T03:22:55.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.479+0000 7fdad97fa640 1 -- 172.21.15.27:0/1419744189 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdae4038040 con 0x55f0e14862c0 2024-09-12T03:22:55.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.479+0000 7fdad97fa640 1 -- 172.21.15.27:0/1419744189 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdae40535c0 con 0x55f0e14862c0 2024-09-12T03:22:55.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.480+0000 7fdafd09d740 1 -- 172.21.15.27:0/1419744189 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f0e14862c0 msgr2=0x55f0e14a6fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:55.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.480+0000 7fdafd09d740 1 --2- 172.21.15.27:0/1419744189 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f0e14862c0 0x55f0e14a6fb0 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7fdae400bc00 tx=0x7fdae400b450 comp rx=0 tx=0).stop 2024-09-12T03:22:55.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.480+0000 7fdafd09d740 1 -- 172.21.15.27:0/1419744189 shutdown_connections 2024-09-12T03:22:55.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.480+0000 7fdafd09d740 1 --2- 172.21.15.27:0/1419744189 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f0e14ac2f0 0x55f0e14c9750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:55.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.480+0000 7fdafd09d740 1 --2- 172.21.15.27:0/1419744189 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x55f0e1487a50 0x55f0e14c9210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:55.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.480+0000 7fdafd09d740 1 --2- 172.21.15.27:0/1419744189 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f0e14862c0 0x55f0e14a6fb0 unknown :-1 s=CLOSED pgs=1419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:55.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.480+0000 7fdafd09d740 1 -- 172.21.15.27:0/1419744189 >> 172.21.15.27:0/1419744189 conn(0x55f0e1483a80 msgr2=0x55f0e14da4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:55.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.480+0000 7fdafd09d740 1 -- 172.21.15.27:0/1419744189 shutdown_connections 2024-09-12T03:22:55.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.480+0000 7fdafd09d740 1 -- 172.21.15.27:0/1419744189 wait complete. 2024-09-12T03:22:55.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.480+0000 7fdafd09d740 1 -- start start 2024-09-12T03:22:55.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.481+0000 7fdafd09d740 1 Processor -- start 2024-09-12T03:22:55.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.481+0000 7fdafd09d740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f0e14862c0 0x55f0e15491f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:55.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.481+0000 7fdafd09d740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f0e1487a50 0x55f0e1549730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:55.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.481+0000 7fdafd09d740 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x55f0e14ac2f0 0x55f0e149a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:55.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.481+0000 7fdafd09d740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55f0e14b0db0 con 0x55f0e1487a50 2024-09-12T03:22:55.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.481+0000 7fdafd09d740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55f0e1480650 con 0x55f0e14862c0 2024-09-12T03:22:55.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.481+0000 7fdafd09d740 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x55f0e149b080 con 0x55f0e14ac2f0 2024-09-12T03:22:55.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.482+0000 7fdafd09d740 10 client.0 _ll_register_callbacks cb 0x7fdae800b8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:22:55.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.482+0000 7fdaf37fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f0e1487a50 0x55f0e1549730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:55.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.482+0000 7fdaf37fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f0e1487a50 0x55f0e1549730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56358/0 (socket says 172.21.15.27:56358) 2024-09-12T03:22:55.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.482+0000 7fdaf37fe640 1 -- 172.21.15.27:0/3930408797 learned_addr learned my addr 172.21.15.27:0/3930408797 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:22:55.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.482+0000 7fdaf88a9640 1 --2- 172.21.15.27:0/3930408797 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x55f0e14ac2f0 0x55f0e149a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:55.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.482+0000 7fdaf3fff640 1 --2- 172.21.15.27:0/3930408797 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f0e14862c0 0x55f0e15491f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:55.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.482+0000 7fdaf37fe640 1 -- 172.21.15.27:0/3930408797 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x55f0e14ac2f0 msgr2=0x55f0e149a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:55.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.482+0000 7fdaf37fe640 1 --2- 172.21.15.27:0/3930408797 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x55f0e14ac2f0 0x55f0e149a1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:55.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.482+0000 7fdaf37fe640 1 -- 172.21.15.27:0/3930408797 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f0e14862c0 msgr2=0x55f0e15491f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:55.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.482+0000 7fdaf37fe640 1 --2- 172.21.15.27:0/3930408797 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f0e14862c0 0x55f0e15491f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:55.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.482+0000 7fdaf37fe640 1 -- 172.21.15.27:0/3930408797 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f0e147c4a0 con 0x55f0e1487a50 2024-09-12T03:22:55.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.482+0000 7fdaf37fe640 1 --2- 172.21.15.27:0/3930408797 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f0e1487a50 0x55f0e1549730 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7fdae8008010 tx=0x7fdae80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1b4cc4762cf6d3c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:55.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.482+0000 7fdad2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-12T03:22:55.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.482+0000 7fdad2ffd640 1 -- 172.21.15.27:0/3930408797 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdae8034020 con 0x55f0e1487a50 2024-09-12T03:22:55.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.482+0000 7fdad2ffd640 1 -- 172.21.15.27:0/3930408797 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(138..138 src has 1..138) ==== 5765+0+0 (secure 0 0 0) 0x7fdae801ccd0 con 0x55f0e1487a50 2024-09-12T03:22:55.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.482+0000 7fdafd09d740 1 -- 172.21.15.27:0/3930408797 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdae4038040 con 0x55f0e1487a50 2024-09-12T03:22:55.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.482+0000 7fdad2ffd640 1 -- 172.21.15.27:0/3930408797 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7fdae803e050 con 0x55f0e1487a50 2024-09-12T03:22:55.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.482+0000 7fdafd09d740 10 client.8322 fetch_fsmap learned FSMap version 82 2024-09-12T03:22:55.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.482+0000 7fdafd09d740 1 -- 172.21.15.27:0/3930408797 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=82,monmap=2+}) -- 0x55f0e14cd0e0 con 0x55f0e1487a50 2024-09-12T03:22:55.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.483+0000 7fdad2ffd640 1 -- 172.21.15.27:0/3930408797 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdae4038040 con 0x55f0e1487a50 2024-09-12T03:22:55.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.483+0000 7fdad2ffd640 1 -- 172.21.15.27:0/3930408797 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 82) ==== 3502+0+0 (secure 0 0 0) 0x7fdae8041020 con 0x55f0e1487a50 2024-09-12T03:22:55.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.483+0000 7fdafd09d740 10 client.8322 fetch_fsmap finished waiting for FSMap version 82 2024-09-12T03:22:55.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.483+0000 7fdafd09d740 10 client.8322 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-12T03:22:55.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.483+0000 7fdafd09d740 20 client.8322 populate_metadata read hostname 'smithi027' 2024-09-12T03:22:55.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.483+0000 7fdafd09d740 1 --2- 172.21.15.27:0/3930408797 >> [v2:172.21.15.27:6833/389118996,v1:172.21.15.27:6835/389118996] conn(0x55f0e14df8c0 0x55f0e14dd680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:55.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.483+0000 7fdafd09d740 4 client.8322 mds_command: new command op to 8196 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-12T03:22:55.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.483+0000 7fdafd09d740 1 -- 172.21.15.27:0/3930408797 --> [v2:172.21.15.27:6833/389118996,v1:172.21.15.27:6835/389118996] -- command(tid 0: {"prefix": "status"}) -- 0x55f0e13effa0 con 0x55f0e14df8c0 2024-09-12T03:22:55.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.483+0000 7fdaf3fff640 1 --2- 172.21.15.27:0/3930408797 >> [v2:172.21.15.27:6833/389118996,v1:172.21.15.27:6835/389118996] conn(0x55f0e14df8c0 0x55f0e14dd680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:55.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.483+0000 7fdad2ffd640 10 client.8322 ms_handle_connect on v2:172.21.15.27:6833/389118996 2024-09-12T03:22:55.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.483+0000 7fdaf3fff640 1 --2- 172.21.15.27:0/3930408797 >> [v2:172.21.15.27:6833/389118996,v1:172.21.15.27:6835/389118996] conn(0x55f0e14df8c0 0x55f0e14dd680 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5bd24343a334a271 server_cookie=64865acd4f4ed59d in_seq=0 out_seq=0 2024-09-12T03:22:55.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.483+0000 7fdad2ffd640 1 -- 172.21.15.27:0/3930408797 <== mds.0 v2:172.21.15.27:6833/389118996 1 ==== command_reply(tid 0: 0 ) ==== 8+0+333 (crc 0 0 0) 0x55f0e13effa0 con 0x55f0e14df8c0 2024-09-12T03:22:55.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.483+0000 7fdad2ffd640 10 client.8322 handle_command_reply: tid=0 2024-09-12T03:22:55.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.483+0000 7fdafd09d740 1 client.8322 shutdown 2024-09-12T03:22:55.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.484+0000 7fdafd09d740 1 -- 172.21.15.27:0/3930408797 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f0e1487a50 msgr2=0x55f0e1549730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:55.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.484+0000 7fdafd09d740 1 --2- 172.21.15.27:0/3930408797 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f0e1487a50 0x55f0e1549730 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7fdae8008010 tx=0x7fdae80046c0 comp rx=0 tx=0).stop 2024-09-12T03:22:55.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.484+0000 7fdafd09d740 1 -- 172.21.15.27:0/3930408797 shutdown_connections 2024-09-12T03:22:55.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.484+0000 7fdafd09d740 1 --2- 172.21.15.27:0/3930408797 >> [v2:172.21.15.27:6833/389118996,v1:172.21.15.27:6835/389118996] conn(0x55f0e14df8c0 0x55f0e14dd680 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:55.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.484+0000 7fdafd09d740 1 --2- 172.21.15.27:0/3930408797 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x55f0e14ac2f0 0x55f0e149a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:55.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.484+0000 7fdafd09d740 1 --2- 172.21.15.27:0/3930408797 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f0e1487a50 0x55f0e1549730 unknown :-1 s=CLOSED pgs=1420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:55.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.484+0000 7fdad2ffd640 0 client.8322 ms_handle_reset on v2:172.21.15.27:6833/389118996 2024-09-12T03:22:55.490 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.484+0000 7fdafd09d740 1 --2- 172.21.15.27:0/3930408797 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f0e14862c0 0x55f0e15491f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:55.490 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.484+0000 7fdafd09d740 1 -- 172.21.15.27:0/3930408797 >> 172.21.15.27:0/3930408797 conn(0x55f0e1483a80 msgr2=0x55f0e14abbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:55.490 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.484+0000 7fdafd09d740 1 -- 172.21.15.27:0/3930408797 shutdown_connections 2024-09-12T03:22:55.490 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.484+0000 7fdafd09d740 1 -- 172.21.15.27:0/3930408797 wait complete. 2024-09-12T03:22:55.490 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.484+0000 7fdafd09d740 20 client.8322 trim_cache size 0 max 16384 2024-09-12T03:22:55.490 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-12T03:22:55.490 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_fsid": "d2df504e-2647-4385-a5db-9fdf69f487a0", 2024-09-12T03:22:55.490 INFO:teuthology.orchestra.run.smithi027.stdout: "whoami": 0, 2024-09-12T03:22:55.490 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 8196, 2024-09-12T03:22:55.490 INFO:teuthology.orchestra.run.smithi027.stdout: "want_state": "up:active", 2024-09-12T03:22:55.490 INFO:teuthology.orchestra.run.smithi027.stdout: "state": "up:active", 2024-09-12T03:22:55.491 INFO:teuthology.orchestra.run.smithi027.stdout: "fs_name": "cephfs", 2024-09-12T03:22:55.491 INFO:teuthology.orchestra.run.smithi027.stdout: "rank_uptime": 4.6457797860000003, 2024-09-12T03:22:55.491 INFO:teuthology.orchestra.run.smithi027.stdout: "mdsmap_epoch": 82, 2024-09-12T03:22:55.491 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch": 138, 2024-09-12T03:22:55.491 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch_barrier": 138, 2024-09-12T03:22:55.491 INFO:teuthology.orchestra.run.smithi027.stdout: "uptime": 9.9764306840000003 2024-09-12T03:22:55.491 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-12T03:22:55.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.484+0000 7fdafa30c640 1 -- 172.21.15.27:0/1673632465 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fdae006c490 msgr2=0x7fdae006e950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:55.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.484+0000 7fdafa30c640 1 --2- 172.21.15.27:0/1673632465 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fdae006c490 0x7fdae006e950 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fdae4037000 tx=0x7fdae4032000 comp rx=0 tx=0).stop 2024-09-12T03:22:55.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.484+0000 7fdafa30c640 1 -- 172.21.15.27:0/1673632465 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdaf415aca0 msgr2=0x7fdaf415e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:55.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.484+0000 7fdafa30c640 1 --2- 172.21.15.27:0/1673632465 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdaf415aca0 0x7fdaf415e130 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7fdaec00b9e0 tx=0x7fdaec00bea0 comp rx=0 tx=0).stop 2024-09-12T03:22:55.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.484+0000 7fdafa30c640 1 -- 172.21.15.27:0/1673632465 shutdown_connections 2024-09-12T03:22:55.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.484+0000 7fdafa30c640 1 --2- 172.21.15.27:0/1673632465 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fdae006c490 0x7fdae006e950 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:55.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.484+0000 7fdafa30c640 1 --2- 172.21.15.27:0/1673632465 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdaf415aca0 0x7fdaf415e130 unknown :-1 s=CLOSED pgs=1416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:55.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.484+0000 7fdafa30c640 1 --2- 172.21.15.27:0/1673632465 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fdaf415a8c0 0x7fdaf4157000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:55.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.484+0000 7fdafa30c640 1 --2- 172.21.15.27:0/1673632465 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdaf4071be0 0x7fdaf40c92c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:55.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.484+0000 7fdafa30c640 1 -- 172.21.15.27:0/1673632465 >> 172.21.15.27:0/1673632465 conn(0x7fdaf4062c60 msgr2=0x7fdaf406e8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:55.493 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.484+0000 7fdafa30c640 1 -- 172.21.15.27:0/1673632465 shutdown_connections 2024-09-12T03:22:55.493 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.485+0000 7fdafa30c640 1 -- 172.21.15.27:0/1673632465 wait complete. 2024-09-12T03:22:55.495 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-12T03:22:55.597 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.597+0000 7ff2887a6640 1 Processor -- start 2024-09-12T03:22:55.597 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.597+0000 7ff2887a6640 1 -- start start 2024-09-12T03:22:55.598 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.598+0000 7ff2887a6640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7ff2800b91a0 0x7ff280169880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:55.598 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.598+0000 7ff2887a6640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff2800d3760 con 0x7ff2800d5200 2024-09-12T03:22:55.598 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.598+0000 7ff2887a6640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff2800d3a40 con 0x7ff28015eb90 2024-09-12T03:22:55.598 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.598+0000 7ff2887a6640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7ff2800b7d20 con 0x7ff2800b91a0 2024-09-12T03:22:55.598 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.598+0000 7ff286d43640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7ff2800b91a0 0x7ff280169880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:55.598 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.598+0000 7ff285d41640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7ff2800d5200 0x7ff2800d1010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:52376/0 (socket says 172.21.15.27:52376) 2024-09-12T03:22:55.599 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.598+0000 7ff285d41640 1 -- 172.21.15.27:0/3779346871 learned_addr learned my addr 172.21.15.27:0/3779346871 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:22:55.599 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.599+0000 7ff286d43640 1 -- 172.21.15.27:0/3779346871 >> v1:172.21.15.45:6789/0 conn(0x7ff28015eb90 legacy=0x7ff2800d2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:55.599 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.599+0000 7ff285540640 1 -- 172.21.15.27:0/3779346871 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2468924810 0 0) 0x7ff2800d3760 con 0x7ff2800d5200 2024-09-12T03:22:55.599 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.599+0000 7ff286d43640 1 -- 172.21.15.27:0/3779346871 >> v1:172.21.15.27:6789/0 conn(0x7ff2800d5200 legacy=0x7ff2800d1010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:55.599 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.599+0000 7ff286d43640 1 -- 172.21.15.27:0/3779346871 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2800d3d20 con 0x7ff2800b91a0 2024-09-12T03:22:55.599 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.600+0000 7ff286d43640 1 --2- 172.21.15.27:0/3779346871 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff2800b91a0 0x7ff280169880 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7ff270009d90 tx=0x7ff27002a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5ea276a87de3fc9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:55.601 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.600+0000 7ff285540640 1 -- 172.21.15.27:0/3779346871 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff270007740 con 0x7ff2800b91a0 2024-09-12T03:22:55.601 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.600+0000 7ff285540640 1 -- 172.21.15.27:0/3779346871 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2700078e0 con 0x7ff2800b91a0 2024-09-12T03:22:55.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.600+0000 7ff285540640 1 -- 172.21.15.27:0/3779346871 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff270007cb0 con 0x7ff2800b91a0 2024-09-12T03:22:55.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.600+0000 7ff2887a6640 1 -- 172.21.15.27:0/3779346871 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff2800b91a0 msgr2=0x7ff280169880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:55.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.600+0000 7ff2887a6640 1 --2- 172.21.15.27:0/3779346871 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff2800b91a0 0x7ff280169880 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7ff270009d90 tx=0x7ff27002a6f0 comp rx=0 tx=0).stop 2024-09-12T03:22:55.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.600+0000 7ff2887a6640 1 -- 172.21.15.27:0/3779346871 shutdown_connections 2024-09-12T03:22:55.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.600+0000 7ff2887a6640 1 --2- 172.21.15.27:0/3779346871 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff2800b91a0 0x7ff280169880 unknown :-1 s=CLOSED pgs=1422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:55.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.600+0000 7ff2887a6640 1 -- 172.21.15.27:0/3779346871 >> 172.21.15.27:0/3779346871 conn(0x7ff28006dce0 msgr2=0x7ff28006e0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:55.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.600+0000 7ff2887a6640 1 -- 172.21.15.27:0/3779346871 shutdown_connections 2024-09-12T03:22:55.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.600+0000 7ff2887a6640 1 -- 172.21.15.27:0/3779346871 wait complete. 2024-09-12T03:22:55.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.602+0000 7ff2887a6640 1 Processor -- start 2024-09-12T03:22:55.603 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.602+0000 7ff2887a6640 1 -- start start 2024-09-12T03:22:55.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.602+0000 7ff2887a6640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff2800b91a0 0x7ff2800cdab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:55.607 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.602+0000 7ff2887a6640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ff2800d5200 0x7ff2800cdff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:55.607 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.602+0000 7ff2887a6640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff28015eb90 0x7ff2800c8340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:55.607 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.602+0000 7ff2887a6640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7ff2800d3500 con 0x7ff2800b91a0 2024-09-12T03:22:55.607 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.602+0000 7ff2887a6640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ff2800d42e0 con 0x7ff28015eb90 2024-09-12T03:22:55.607 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.602+0000 7ff2887a6640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7ff28015ef70 con 0x7ff2800d5200 2024-09-12T03:22:55.607 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.602+0000 7ff286542640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff2800b91a0 0x7ff2800cdab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:55.607 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.602+0000 7ff286542640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff2800b91a0 0x7ff2800cdab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56376/0 (socket says 172.21.15.27:56376) 2024-09-12T03:22:55.607 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.602+0000 7ff286542640 1 -- 172.21.15.27:0/2274315170 learned_addr learned my addr 172.21.15.27:0/2274315170 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:22:55.608 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.603+0000 7ff286542640 1 -- 172.21.15.27:0/2274315170 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ff2800d5200 msgr2=0x7ff2800cdff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:55.608 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.603+0000 7ff286542640 1 --2- 172.21.15.27:0/2274315170 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ff2800d5200 0x7ff2800cdff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:55.608 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.603+0000 7ff286542640 1 -- 172.21.15.27:0/2274315170 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff28015eb90 msgr2=0x7ff2800c8340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:55.608 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.603+0000 7ff286d43640 1 --2- 172.21.15.27:0/2274315170 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff28015eb90 0x7ff2800c8340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:55.608 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.603+0000 7ff286542640 1 --2- 172.21.15.27:0/2274315170 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff28015eb90 0x7ff2800c8340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:55.608 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.603+0000 7ff286542640 1 -- 172.21.15.27:0/2274315170 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2801e7330 con 0x7ff2800b91a0 2024-09-12T03:22:55.608 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.603+0000 7ff286542640 1 --2- 172.21.15.27:0/2274315170 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff2800b91a0 0x7ff2800cdab0 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7ff27800c3c0 tx=0x7ff27800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:55.608 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.603+0000 7ff27effd640 1 -- 172.21.15.27:0/2274315170 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2780052d0 con 0x7ff2800b91a0 2024-09-12T03:22:55.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.603+0000 7ff27effd640 1 -- 172.21.15.27:0/2274315170 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff278005490 con 0x7ff2800b91a0 2024-09-12T03:22:55.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.603+0000 7ff2887a6640 1 -- 172.21.15.27:0/2274315170 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2800ca2d0 con 0x7ff2800b91a0 2024-09-12T03:22:55.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.603+0000 7ff27effd640 1 -- 172.21.15.27:0/2274315170 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff278007260 con 0x7ff2800b91a0 2024-09-12T03:22:55.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.603+0000 7ff2887a6640 1 -- 172.21.15.27:0/2274315170 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2800ca850 con 0x7ff2800b91a0 2024-09-12T03:22:55.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.604+0000 7ff2887a6640 1 -- 172.21.15.27:0/2274315170 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2800d2df0 con 0x7ff2800b91a0 2024-09-12T03:22:55.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.605+0000 7ff27effd640 1 -- 172.21.15.27:0/2274315170 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 22) ==== 91059+0+0 (secure 0 0 0) 0x7ff278007400 con 0x7ff2800b91a0 2024-09-12T03:22:55.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.605+0000 7ff27effd640 1 --2- 172.21.15.27:0/2274315170 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7ff26c06c460 0x7ff26c06e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:55.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.605+0000 7ff285d41640 1 --2- 172.21.15.27:0/2274315170 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7ff26c06c460 0x7ff26c06e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:55.610 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.605+0000 7ff27effd640 1 -- 172.21.15.27:0/2274315170 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 5765+0+0 (secure 0 0 0) 0x7ff27808ae60 con 0x7ff2800b91a0 2024-09-12T03:22:55.610 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.606+0000 7ff285d41640 1 --2- 172.21.15.27:0/2274315170 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7ff26c06c460 0x7ff26c06e920 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7ff274007fd0 tx=0x7ff274007900 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:55.610 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.610+0000 7ff27effd640 1 -- 172.21.15.27:0/2274315170 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7ff278058f20 con 0x7ff2800b91a0 2024-09-12T03:22:55.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.820+0000 7ff2887a6640 1 -- 172.21.15.27:0/2274315170 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7ff280161d00 con 0x7ff2800b91a0 2024-09-12T03:22:55.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.821+0000 7ff27effd640 1 -- 172.21.15.27:0/2274315170 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v82) ==== 53+0+73 (secure 0 0 0) 0x7ff278058820 con 0x7ff2800b91a0 2024-09-12T03:22:55.821 INFO:teuthology.orchestra.run.smithi027.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-12T03:22:55.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.823+0000 7ff2887a6640 1 -- 172.21.15.27:0/2274315170 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7ff26c06c460 msgr2=0x7ff26c06e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:55.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.823+0000 7ff2887a6640 1 --2- 172.21.15.27:0/2274315170 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7ff26c06c460 0x7ff26c06e920 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7ff274007fd0 tx=0x7ff274007900 comp rx=0 tx=0).stop 2024-09-12T03:22:55.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.823+0000 7ff2887a6640 1 -- 172.21.15.27:0/2274315170 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff2800b91a0 msgr2=0x7ff2800cdab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:55.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.823+0000 7ff2887a6640 1 --2- 172.21.15.27:0/2274315170 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff2800b91a0 0x7ff2800cdab0 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7ff27800c3c0 tx=0x7ff27800c880 comp rx=0 tx=0).stop 2024-09-12T03:22:55.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.823+0000 7ff2887a6640 1 -- 172.21.15.27:0/2274315170 shutdown_connections 2024-09-12T03:22:55.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.823+0000 7ff2887a6640 1 --2- 172.21.15.27:0/2274315170 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff28015eb90 0x7ff2800c8340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:55.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.823+0000 7ff2887a6640 1 --2- 172.21.15.27:0/2274315170 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ff2800d5200 0x7ff2800cdff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:55.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.823+0000 7ff2887a6640 1 --2- 172.21.15.27:0/2274315170 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7ff26c06c460 0x7ff26c06e920 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:55.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.823+0000 7ff2887a6640 1 --2- 172.21.15.27:0/2274315170 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff2800b91a0 0x7ff2800cdab0 unknown :-1 s=CLOSED pgs=1423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:55.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.823+0000 7ff2887a6640 1 -- 172.21.15.27:0/2274315170 >> 172.21.15.27:0/2274315170 conn(0x7ff28006dce0 msgr2=0x7ff2800d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:55.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.823+0000 7ff2887a6640 1 -- 172.21.15.27:0/2274315170 shutdown_connections 2024-09-12T03:22:55.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:55.823+0000 7ff2887a6640 1 -- 172.21.15.27:0/2274315170 wait complete. 2024-09-12T03:22:55.833 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-12T03:22:55.833 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-12T03:22:55.834 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi027.front.sepia.ceph.com 2024-09-12T03:22:55.834 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-12T03:22:55.834 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-12T03:22:55.834 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-12T03:22:55.834 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-12T03:22:55.834 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-12T03:22:55.834 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-12T03:22:55.834 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:22:55.834 DEBUG:teuthology.orchestra.run.smithi027:> ip addr 2024-09-12T03:22:55.851 INFO:teuthology.orchestra.run.smithi027.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-12T03:22:55.851 INFO:teuthology.orchestra.run.smithi027.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-12T03:22:55.851 INFO:teuthology.orchestra.run.smithi027.stdout: inet 127.0.0.1/8 scope host lo 2024-09-12T03:22:55.851 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:22:55.852 INFO:teuthology.orchestra.run.smithi027.stdout: inet6 ::1/128 scope host 2024-09-12T03:22:55.852 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:22:55.852 INFO:teuthology.orchestra.run.smithi027.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-12T03:22:55.852 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether 0c:c4:7a:6b:fe:c8 brd ff:ff:ff:ff:ff:ff 2024-09-12T03:22:55.852 INFO:teuthology.orchestra.run.smithi027.stdout: altname enp7s0f0 2024-09-12T03:22:55.852 INFO:teuthology.orchestra.run.smithi027.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-12T03:22:55.852 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether 0c:c4:7a:6b:fe:c9 brd ff:ff:ff:ff:ff:ff 2024-09-12T03:22:55.852 INFO:teuthology.orchestra.run.smithi027.stdout: altname enp7s0f1 2024-09-12T03:22:55.852 INFO:teuthology.orchestra.run.smithi027.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-12T03:22:55.852 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether ac:1f:6b:f8:39:5a brd ff:ff:ff:ff:ff:ff 2024-09-12T03:22:55.853 INFO:teuthology.orchestra.run.smithi027.stdout: altname enp1s0f0 2024-09-12T03:22:55.853 INFO:teuthology.orchestra.run.smithi027.stdout: inet 172.21.15.27/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-12T03:22:55.853 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft 42413sec preferred_lft 42413sec 2024-09-12T03:22:55.853 INFO:teuthology.orchestra.run.smithi027.stdout: inet6 fe80::ae1f:6bff:fef8:395a/64 scope link 2024-09-12T03:22:55.853 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:22:55.853 INFO:teuthology.orchestra.run.smithi027.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-12T03:22:55.853 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether ac:1f:6b:f8:39:5b brd ff:ff:ff:ff:ff:ff 2024-09-12T03:22:55.853 INFO:teuthology.orchestra.run.smithi027.stdout: altname enp1s0f1 2024-09-12T03:22:55.853 INFO:teuthology.orchestra.run.smithi027.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-12T03:22:55.853 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether 2e:7f:89:6c:33:a0 brd ff:ff:ff:ff:ff:ff 2024-09-12T03:22:55.853 INFO:teuthology.orchestra.run.smithi027.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-12T03:22:55.853 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:22:55.853 INFO:teuthology.orchestra.run.smithi027.stdout: inet6 fe80::c84b:19ff:fef0:ca74/64 scope link 2024-09-12T03:22:55.853 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:22:55.854 INFO:teuthology.orchestra.run.smithi027.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-12T03:22:55.854 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether 42:40:d7:5c:7a:8e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-12T03:22:55.854 INFO:teuthology.orchestra.run.smithi027.stdout: inet6 fe80::4040:d7ff:fe5c:7a8e/64 scope link 2024-09-12T03:22:55.854 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:22:55.854 INFO:teuthology.orchestra.run.smithi027.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-12T03:22:55.854 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether 2e:7f:89:6c:33:a0 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-12T03:22:55.854 INFO:teuthology.orchestra.run.smithi027.stdout: inet6 fe80::2c7f:89ff:fe6c:33a0/64 scope link 2024-09-12T03:22:55.854 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:22:55.854 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:22:55.854 DEBUG:teuthology.orchestra.run.smithi027:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-12T03:22:55.916 INFO:teuthology.orchestra.run.smithi027.stdout:1 2024-09-12T03:22:55.918 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:22:55.918 DEBUG:teuthology.orchestra.run.smithi027:> route 2024-09-12T03:22:55.974 INFO:teuthology.orchestra.run.smithi027.stdout:Kernel IP routing table 2024-09-12T03:22:55.974 INFO:teuthology.orchestra.run.smithi027.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-12T03:22:55.975 INFO:teuthology.orchestra.run.smithi027.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-12T03:22:55.975 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-12T03:22:55.975 INFO:teuthology.orchestra.run.smithi027.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-12T03:22:55.975 DEBUG:teuthology.orchestra.run.smithi027:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-12T03:22:55.975 DEBUG:teuthology.orchestra.run.smithi027:> set -e 2024-09-12T03:22:55.975 DEBUG:teuthology.orchestra.run.smithi027:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-12T03:22:55.975 DEBUG:teuthology.orchestra.run.smithi027:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-12T03:22:55.975 DEBUG:teuthology.orchestra.run.smithi027:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-12T03:22:55.975 DEBUG:teuthology.orchestra.run.smithi027:> ') 2024-09-12T03:22:56.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:56 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T03:22:56.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T03:22:56.103 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:22:56.103 DEBUG:teuthology.orchestra.run.smithi027:> ip netns list 2024-09-12T03:22:56.120 INFO:teuthology.orchestra.run.smithi027.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-12T03:22:56.120 INFO:teuthology.orchestra.run.smithi027.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-12T03:22:56.120 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-12T03:22:56.120 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:22:56.120 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip link set brx.0 up 2024-09-12T03:22:56.185 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-12T03:22:56.186 DEBUG:teuthology.orchestra.run.smithi027:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-12T03:22:56.242 INFO:teuthology.orchestra.run.smithi027.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-12T03:22:56.242 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-12T03:22:56.242 DEBUG:teuthology.orchestra.run.smithi027:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-12T03:22:56.297 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-09-12T03:22:56.297 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:22:56.297 DEBUG:teuthology.orchestra.run.smithi027:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-12T03:22:56.382 INFO:teuthology.orchestra.run.smithi027.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-12T03:22:56.382 INFO:teuthology.orchestra.run.smithi027.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-12T03:22:56.390 INFO:teuthology.orchestra.run.smithi027.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-12T03:22:56.390 INFO:teuthology.orchestra.run.smithi027.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-12T03:22:56.390 INFO:teuthology.orchestra.run.smithi027.stdout:mount.ceph: resolved to: "172.21.15.27:3300,172.21.15.45:3300,172.21.15.45:3301" 2024-09-12T03:22:56.390 INFO:teuthology.orchestra.run.smithi027.stdout:mount.ceph: trying mount with new device syntax: 0@d2df504e-2647-4385-a5db-9fdf69f487a0.cephfs=/ 2024-09-12T03:22:56.390 INFO:teuthology.orchestra.run.smithi027.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.27:3300/172.21.15.45:3300/172.21.15.45:3301" will pass to kernel 2024-09-12T03:22:56.390 INFO:teuthology.orchestra.run.smithi027.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-12T03:22:56.390 INFO:teuthology.orchestra.run.smithi027.stdout: You just mounted a file system that supports labels which does not 2024-09-12T03:22:56.391 INFO:teuthology.orchestra.run.smithi027.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-12T03:22:56.391 INFO:teuthology.orchestra.run.smithi027.stdout: applications will generate AVC messages and not be allowed access to 2024-09-12T03:22:56.391 INFO:teuthology.orchestra.run.smithi027.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-12T03:22:56.392 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-12T03:22:56.392 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:22:56.392 DEBUG:teuthology.orchestra.run.smithi027:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-12T03:22:56.418 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-12T03:22:56.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:56 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-12T03:22:56.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T03:22:56.517 INFO:tasks.ceph.mds.a.smithi027.stderr:2024-09-12T03:22:56.517+0000 7ff61291b640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-12T03:22:56.521 INFO:teuthology.orchestra.run.smithi027.stdout:client6767 2024-09-12T03:22:56.524 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-12T03:22:56.524 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T03:22:56.541 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-12T03:22:56.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:56.683+0000 7f69adb9f640 1 Processor -- start 2024-09-12T03:22:56.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:56.683+0000 7f69adb9f640 1 -- start start 2024-09-12T03:22:56.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:56.684+0000 7f69adb9f640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f69a806f360 0x7f69a80cef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:56.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:56.684+0000 7f69adb9f640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f69a80cf490 0x7f69a80d2780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:56.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:56.684+0000 7f69adb9f640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f69a806cc40 con 0x7f69a80b1ca0 2024-09-12T03:22:56.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:56.684+0000 7f69adb9f640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f69a80d22f0 con 0x7f69a80cf490 2024-09-12T03:22:56.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:56.684+0000 7f69adb9f640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f69a80d1ce0 con 0x7f69a806f360 2024-09-12T03:22:56.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:56.684+0000 7f69a77fe640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f69a80b1ca0 0x7f69a815f830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:52380/0 (socket says 172.21.15.27:52380) 2024-09-12T03:22:56.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:56.684+0000 7f69a77fe640 1 -- 172.21.15.27:0/1887561779 learned_addr learned my addr 172.21.15.27:0/1887561779 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:22:56.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:56.684+0000 7f69a7fff640 1 --2- 172.21.15.27:0/1887561779 >> v2:172.21.15.45:3300/0 conn(0x7f69a80cf490 0x7f69a80d2780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:56.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:56.684+0000 7f69a6ffd640 1 --2- 172.21.15.27:0/1887561779 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f69a806f360 0x7f69a80cef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:56.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:56.684+0000 7f69a67fc640 1 -- 172.21.15.27:0/1887561779 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2995838195 0 0) 0x7f69a806cc40 con 0x7f69a80b1ca0 2024-09-12T03:22:56.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:56.684+0000 7f69a67fc640 1 -- 172.21.15.27:0/1887561779 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6990003520 con 0x7f69a80b1ca0 2024-09-12T03:22:56.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:56.684+0000 7f69a67fc640 1 -- 172.21.15.27:0/1887561779 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 646720804 0 0) 0x7f6990003520 con 0x7f69a80b1ca0 2024-09-12T03:22:56.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:56.684+0000 7f69a67fc640 1 -- 172.21.15.27:0/1887561779 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f69a806f360 msgr2=0x7f69a80cef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:56.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:56.684+0000 7f69a67fc640 1 --2- 172.21.15.27:0/1887561779 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f69a806f360 0x7f69a80cef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:56.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:56.684+0000 7f69a67fc640 1 -- 172.21.15.27:0/1887561779 >> v2:172.21.15.45:3300/0 conn(0x7f69a80cf490 msgr2=0x7f69a80d2780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:56.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:56.684+0000 7f69a67fc640 1 --2- 172.21.15.27:0/1887561779 >> v2:172.21.15.45:3300/0 conn(0x7f69a80cf490 0x7f69a80d2780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:56.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:56.684+0000 7f69a67fc640 1 -- 172.21.15.27:0/1887561779 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69a806cee0 con 0x7f69a80b1ca0 2024-09-12T03:22:56.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:56.684+0000 7f69a67fc640 1 -- 172.21.15.27:0/1887561779 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 847755213 0 0) 0x7f6998001670 con 0x7f69a80b1ca0 2024-09-12T03:22:56.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:56.684+0000 7f69a67fc640 1 -- 172.21.15.27:0/1887561779 >> v1:172.21.15.27:6789/0 conn(0x7f69a80b1ca0 legacy=0x7f69a815f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:56.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:56.684+0000 7f69a67fc640 1 --2- 172.21.15.27:0/1887561779 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f69a806f360 0x7f69900039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:56.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:56.685+0000 7f69a67fc640 1 --2- 172.21.15.27:0/1887561779 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6990003f30 0x7f6990004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:56.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:56.685+0000 7f69a67fc640 1 --2- 172.21.15.27:0/1887561779 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6990004d40 0x7f69900051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:56.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:56.685+0000 7f69a67fc640 1 -- 172.21.15.27:0/1887561779 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f69a80d22f0 con 0x7f6990004d40 2024-09-12T03:22:56.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:56.685+0000 7f69a67fc640 1 -- 172.21.15.27:0/1887561779 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f69a80d1ce0 con 0x7f6990003f30 2024-09-12T03:22:56.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:56.685+0000 7f69a67fc640 1 -- 172.21.15.27:0/1887561779 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f699000b9c0 con 0x7f69a806f360 2024-09-12T03:22:56.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:56.685+0000 7f69a7fff640 1 --2- 172.21.15.27:0/1887561779 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6990004d40 0x7f69900051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:56.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:56.685+0000 7f69a77fe640 1 --2- 172.21.15.27:0/1887561779 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f69a806f360 0x7f69900039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:56.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:56.685+0000 7f69a6ffd640 1 --2- 172.21.15.27:0/1887561779 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6990003f30 0x7f6990004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:56.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:56.685+0000 7f69a7fff640 1 -- 172.21.15.27:0/1887561779 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f69a806f360 msgr2=0x7f69900039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:56.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:56.685+0000 7f69a7fff640 1 --2- 172.21.15.27:0/1887561779 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f69a806f360 0x7f69900039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:56.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:56.685+0000 7f69a7fff640 1 -- 172.21.15.27:0/1887561779 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6990003f30 msgr2=0x7f6990004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:56.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:56.685+0000 7f69a7fff640 1 --2- 172.21.15.27:0/1887561779 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6990003f30 0x7f6990004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:56.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:56.685+0000 7f69a7fff640 1 -- 172.21.15.27:0/1887561779 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6990003520 con 0x7f6990004d40 2024-09-12T03:22:56.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:56.686+0000 7f69a7fff640 1 --2- 172.21.15.27:0/1887561779 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6990004d40 0x7f69900051c0 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7f69a0009fc0 tx=0x7f6990005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2955216a82b824e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:56.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:56.686+0000 7f69a67fc640 1 -- 172.21.15.27:0/1887561779 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69a0009c10 con 0x7f6990004d40 2024-09-12T03:22:56.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:56.686+0000 7f69a67fc640 1 -- 172.21.15.27:0/1887561779 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69a0036070 con 0x7f6990004d40 2024-09-12T03:22:56.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:56.686+0000 7f69a67fc640 1 -- 172.21.15.27:0/1887561779 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69a002d070 con 0x7f6990004d40 2024-09-12T03:22:56.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:56.686+0000 7f69adb9f640 1 -- 172.21.15.27:0/1887561779 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6990004d40 msgr2=0x7f69900051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:56.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:56.686+0000 7f69adb9f640 1 --2- 172.21.15.27:0/1887561779 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6990004d40 0x7f69900051c0 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7f69a0009fc0 tx=0x7f6990005830 comp rx=0 tx=0).stop 2024-09-12T03:22:56.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:56.686+0000 7f69a7fff640 1 -- 172.21.15.27:0/1887561779 reap_dead start 2024-09-12T03:22:56.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:56.686+0000 7f69adb9f640 1 -- 172.21.15.27:0/1887561779 shutdown_connections 2024-09-12T03:22:56.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:56.686+0000 7f69adb9f640 1 -- 172.21.15.27:0/1887561779 >> 172.21.15.27:0/1887561779 conn(0x7f69a8073800 msgr2=0x7f69a809ba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:56.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:56.686+0000 7f69adb9f640 1 -- 172.21.15.27:0/1887561779 shutdown_connections 2024-09-12T03:22:56.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:56.686+0000 7f69adb9f640 1 -- 172.21.15.27:0/1887561779 wait complete. 2024-09-12T03:22:56.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:56.688+0000 7f69adb9f640 1 Processor -- start 2024-09-12T03:22:56.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:56.688+0000 7f69adb9f640 1 -- start start 2024-09-12T03:22:56.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:56.688+0000 7f69adb9f640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6990004d40 0x7f69a814f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:56.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:56.688+0000 7f69adb9f640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f69a815b380 0x7f69a814f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:56.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:56.688+0000 7f69adb9f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f69a80ce300 0x7f69a80b0500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:56.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:56.688+0000 7f69adb9f640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f69a806d340 con 0x7f6990004d40 2024-09-12T03:22:56.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:56.689+0000 7f69adb9f640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f69a806d1c0 con 0x7f69a80ce300 2024-09-12T03:22:56.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:56.689+0000 7f69adb9f640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f69a80cfc40 con 0x7f69a815b380 2024-09-12T03:22:56.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:56.689+0000 7f69a77fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6990004d40 0x7f69a814f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:56.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:56.689+0000 7f69a77fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6990004d40 0x7f69a814f840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56394/0 (socket says 172.21.15.27:56394) 2024-09-12T03:22:56.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:56.689+0000 7f69a77fe640 1 -- 172.21.15.27:0/3544836595 learned_addr learned my addr 172.21.15.27:0/3544836595 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:22:56.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:56.689+0000 7f69a6ffd640 1 --2- 172.21.15.27:0/3544836595 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f69a815b380 0x7f69a814f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:56.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:56.689+0000 7f69a7fff640 1 --2- 172.21.15.27:0/3544836595 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f69a80ce300 0x7f69a80b0500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:56.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:56.689+0000 7f69a77fe640 1 -- 172.21.15.27:0/3544836595 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f69a815b380 msgr2=0x7f69a814f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:56.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:56.689+0000 7f69a77fe640 1 --2- 172.21.15.27:0/3544836595 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f69a815b380 0x7f69a814f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:56.693 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:56.689+0000 7f69a77fe640 1 -- 172.21.15.27:0/3544836595 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f69a80ce300 msgr2=0x7f69a80b0500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:56.693 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:56.689+0000 7f69a77fe640 1 --2- 172.21.15.27:0/3544836595 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f69a80ce300 0x7f69a80b0500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:56.693 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:56.689+0000 7f69a77fe640 1 -- 172.21.15.27:0/3544836595 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69a8160270 con 0x7f6990004d40 2024-09-12T03:22:56.693 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:56.689+0000 7f69a7fff640 1 --2- 172.21.15.27:0/3544836595 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f69a80ce300 0x7f69a80b0500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:22:56.693 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:56.689+0000 7f69a77fe640 1 --2- 172.21.15.27:0/3544836595 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6990004d40 0x7f69a814f840 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7f6998007310 tx=0x7f6998007710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:56.693 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:56.689+0000 7f698ffff640 1 -- 172.21.15.27:0/3544836595 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f699800f960 con 0x7f6990004d40 2024-09-12T03:22:56.693 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:56.689+0000 7f698ffff640 1 -- 172.21.15.27:0/3544836595 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6998018070 con 0x7f6990004d40 2024-09-12T03:22:56.693 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:56.689+0000 7f69adb9f640 1 -- 172.21.15.27:0/3544836595 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69a8160550 con 0x7f6990004d40 2024-09-12T03:22:56.693 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:56.689+0000 7f698ffff640 1 -- 172.21.15.27:0/3544836595 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f699800fd40 con 0x7f6990004d40 2024-09-12T03:22:56.693 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:56.689+0000 7f69adb9f640 1 -- 172.21.15.27:0/3544836595 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69a8160b10 con 0x7f6990004d40 2024-09-12T03:22:56.693 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:56.690+0000 7f69adb9f640 1 -- 172.21.15.27:0/3544836595 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69a8160d10 con 0x7f6990004d40 2024-09-12T03:22:56.693 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:56.691+0000 7f698ffff640 1 -- 172.21.15.27:0/3544836595 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 22) ==== 91059+0+0 (secure 0 0 0) 0x7f699801d460 con 0x7f6990004d40 2024-09-12T03:22:56.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:56.691+0000 7f698ffff640 1 --2- 172.21.15.27:0/3544836595 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f699406c490 0x7f699406e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:56.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:56.691+0000 7f698ffff640 1 -- 172.21.15.27:0/3544836595 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 5765+0+0 (secure 0 0 0) 0x7f699808ba70 con 0x7f6990004d40 2024-09-12T03:22:56.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:56.691+0000 7f69a6ffd640 1 --2- 172.21.15.27:0/3544836595 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f699406c490 0x7f699406e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:56.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:56.692+0000 7f69a6ffd640 1 --2- 172.21.15.27:0/3544836595 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f699406c490 0x7f699406e950 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f69a80cbf70 tx=0x7f699c009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:56.695 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:56.695+0000 7f698ffff640 1 -- 172.21.15.27:0/3544836595 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f69a8160d10 con 0x7f6990004d40 2024-09-12T03:22:56.906 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:56.906+0000 7f69adb9f640 1 -- 172.21.15.27:0/3544836595 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f69a806f7a0 con 0x7f6990004d40 2024-09-12T03:22:56.906 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:56.906+0000 7f698ffff640 1 -- 172.21.15.27:0/3544836595 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v82) ==== 53+0+73 (secure 0 0 0) 0x7f6998059b30 con 0x7f6990004d40 2024-09-12T03:22:56.907 INFO:teuthology.orchestra.run.smithi027.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-12T03:22:56.909 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:56.908+0000 7f69adb9f640 1 -- 172.21.15.27:0/3544836595 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f699406c490 msgr2=0x7f699406e950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:56.909 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:56.908+0000 7f69adb9f640 1 --2- 172.21.15.27:0/3544836595 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f699406c490 0x7f699406e950 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f69a80cbf70 tx=0x7f699c009190 comp rx=0 tx=0).stop 2024-09-12T03:22:56.909 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:56.908+0000 7f69adb9f640 1 -- 172.21.15.27:0/3544836595 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6990004d40 msgr2=0x7f69a814f840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:56.909 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:56.908+0000 7f69adb9f640 1 --2- 172.21.15.27:0/3544836595 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6990004d40 0x7f69a814f840 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7f6998007310 tx=0x7f6998007710 comp rx=0 tx=0).stop 2024-09-12T03:22:56.909 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:56.908+0000 7f69adb9f640 1 -- 172.21.15.27:0/3544836595 shutdown_connections 2024-09-12T03:22:56.909 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:56.908+0000 7f69adb9f640 1 --2- 172.21.15.27:0/3544836595 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f69a80ce300 0x7f69a80b0500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:56.909 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:56.908+0000 7f69adb9f640 1 --2- 172.21.15.27:0/3544836595 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f69a815b380 0x7f69a814f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:56.909 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:56.908+0000 7f69adb9f640 1 --2- 172.21.15.27:0/3544836595 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f699406c490 0x7f699406e950 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:56.909 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:56.909+0000 7f69adb9f640 1 --2- 172.21.15.27:0/3544836595 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6990004d40 0x7f69a814f840 unknown :-1 s=CLOSED pgs=1426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:56.909 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:56.909+0000 7f69adb9f640 1 -- 172.21.15.27:0/3544836595 >> 172.21.15.27:0/3544836595 conn(0x7f69a8073800 msgr2=0x7f69a80b3020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:56.910 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:56.909+0000 7f69adb9f640 1 -- 172.21.15.27:0/3544836595 shutdown_connections 2024-09-12T03:22:56.910 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:56.909+0000 7f69adb9f640 1 -- 172.21.15.27:0/3544836595 wait complete. 2024-09-12T03:22:56.919 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-12T03:22:56.919 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-12T03:22:56.919 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi045.front.sepia.ceph.com 2024-09-12T03:22:56.919 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-12T03:22:56.920 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-12T03:22:56.920 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-12T03:22:56.920 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-12T03:22:56.920 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-12T03:22:56.920 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-12T03:22:56.920 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:22:56.920 DEBUG:teuthology.orchestra.run.smithi045:> ip addr 2024-09-12T03:22:56.941 INFO:teuthology.orchestra.run.smithi045.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-12T03:22:56.941 INFO:teuthology.orchestra.run.smithi045.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-12T03:22:56.941 INFO:teuthology.orchestra.run.smithi045.stdout: inet 127.0.0.1/8 scope host lo 2024-09-12T03:22:56.941 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:22:56.941 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 ::1/128 scope host 2024-09-12T03:22:56.941 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:22:56.941 INFO:teuthology.orchestra.run.smithi045.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-12T03:22:56.941 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:6b:fd:a6 brd ff:ff:ff:ff:ff:ff 2024-09-12T03:22:56.941 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp7s0f0 2024-09-12T03:22:56.942 INFO:teuthology.orchestra.run.smithi045.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-12T03:22:56.942 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:6b:fd:a7 brd ff:ff:ff:ff:ff:ff 2024-09-12T03:22:56.942 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp7s0f1 2024-09-12T03:22:56.942 INFO:teuthology.orchestra.run.smithi045.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-12T03:22:56.942 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:bd:12:be brd ff:ff:ff:ff:ff:ff 2024-09-12T03:22:56.942 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp1s0f0 2024-09-12T03:22:56.942 INFO:teuthology.orchestra.run.smithi045.stdout: inet 172.21.15.45/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-12T03:22:56.942 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft 42357sec preferred_lft 42357sec 2024-09-12T03:22:56.942 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::ec4:7aff:febd:12be/64 scope link 2024-09-12T03:22:56.942 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:22:56.942 INFO:teuthology.orchestra.run.smithi045.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-12T03:22:56.942 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:bd:12:bf brd ff:ff:ff:ff:ff:ff 2024-09-12T03:22:56.943 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp1s0f1 2024-09-12T03:22:56.943 INFO:teuthology.orchestra.run.smithi045.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-12T03:22:56.943 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 52:97:33:5c:60:1b brd ff:ff:ff:ff:ff:ff 2024-09-12T03:22:56.943 INFO:teuthology.orchestra.run.smithi045.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-12T03:22:56.943 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:22:56.943 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::d8c2:5dff:fe26:78c8/64 scope link 2024-09-12T03:22:56.943 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:22:56.943 INFO:teuthology.orchestra.run.smithi045.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-12T03:22:56.943 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 5a:ec:d7:40:ef:75 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-12T03:22:56.943 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::58ec:d7ff:fe40:ef75/64 scope link 2024-09-12T03:22:56.943 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:22:56.943 INFO:teuthology.orchestra.run.smithi045.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-12T03:22:56.943 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 52:97:33:5c:60:1b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-12T03:22:56.944 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::5097:33ff:fe5c:601b/64 scope link 2024-09-12T03:22:56.944 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:22:56.944 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:22:56.944 DEBUG:teuthology.orchestra.run.smithi045:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-12T03:22:57.012 INFO:teuthology.orchestra.run.smithi045.stdout:1 2024-09-12T03:22:57.013 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:22:57.013 DEBUG:teuthology.orchestra.run.smithi045:> route 2024-09-12T03:22:57.073 INFO:teuthology.orchestra.run.smithi045.stdout:Kernel IP routing table 2024-09-12T03:22:57.073 INFO:teuthology.orchestra.run.smithi045.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-12T03:22:57.073 INFO:teuthology.orchestra.run.smithi045.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-12T03:22:57.074 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-12T03:22:57.074 INFO:teuthology.orchestra.run.smithi045.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-12T03:22:57.074 DEBUG:teuthology.orchestra.run.smithi045:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-12T03:22:57.074 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-09-12T03:22:57.074 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-12T03:22:57.074 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-12T03:22:57.074 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-12T03:22:57.074 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-09-12T03:22:57.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:22:57 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T03:22:57.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:22:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T03:22:57.210 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:22:57.210 DEBUG:teuthology.orchestra.run.smithi045:> ip netns list 2024-09-12T03:22:57.226 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-09-12T03:22:57.227 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-12T03:22:57.227 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-12T03:22:57.227 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:22:57.227 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link set brx.0 up 2024-09-12T03:22:57.293 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-12T03:22:57.293 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-12T03:22:57.349 INFO:teuthology.orchestra.run.smithi045.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-12T03:22:57.350 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-12T03:22:57.350 DEBUG:teuthology.orchestra.run.smithi045:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-12T03:22:57.405 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2024-09-12T03:22:57.406 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:22:57.406 DEBUG:teuthology.orchestra.run.smithi045:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-12T03:22:57.490 INFO:teuthology.orchestra.run.smithi045.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-12T03:22:57.490 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-12T03:22:57.501 INFO:teuthology.orchestra.run.smithi045.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-12T03:22:57.501 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-12T03:22:57.501 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: resolved to: "172.21.15.27:3300,172.21.15.45:3300,172.21.15.45:3301" 2024-09-12T03:22:57.501 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: trying mount with new device syntax: 1@d2df504e-2647-4385-a5db-9fdf69f487a0.cephfs=/ 2024-09-12T03:22:57.501 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.27:3300/172.21.15.45:3300/172.21.15.45:3301" will pass to kernel 2024-09-12T03:22:57.501 INFO:teuthology.orchestra.run.smithi045.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-09-12T03:22:57.502 INFO:teuthology.orchestra.run.smithi045.stdout: You just mounted a file system that supports labels which does not 2024-09-12T03:22:57.502 INFO:teuthology.orchestra.run.smithi045.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-12T03:22:57.502 INFO:teuthology.orchestra.run.smithi045.stdout: applications will generate AVC messages and not be allowed access to 2024-09-12T03:22:57.502 INFO:teuthology.orchestra.run.smithi045.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-12T03:22:57.503 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-12T03:22:57.503 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:22:57.504 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-12T03:22:57.529 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-12T03:22:57.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:22:57 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-12T03:22:57.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:22:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T03:22:57.627 INFO:teuthology.orchestra.run.smithi045.stdout:client7570 2024-09-12T03:22:57.627 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-12T03:22:57.628 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T03:22:57.644 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-09-12T03:22:57.764 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:57.764+0000 7fe63e58d640 1 Processor -- start 2024-09-12T03:22:57.764 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:57.764+0000 7fe63e58d640 1 -- start start 2024-09-12T03:22:57.765 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:57.765+0000 7fe63e58d640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fe63815e650 0x7fe6380d1d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:57.765 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:57.765+0000 7fe63e58d640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe63806cc40 0x7fe63806d060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:57.765 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:57.765+0000 7fe63e58d640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe6380d2280 con 0x7fe63806db30 2024-09-12T03:22:57.766 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:57.765+0000 7fe63e58d640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7fe63806d5a0 con 0x7fe63815e650 2024-09-12T03:22:57.766 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:57.765+0000 7fe63e58d640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fe63806e670 con 0x7fe63806cc40 2024-09-12T03:22:57.766 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:57.765+0000 7fe637fff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fe63815e650 0x7fe6380d1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:57.766 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:57.765+0000 7fe63cb2a640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fe63806db30 0x7fe63806df60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:52390/0 (socket says 172.21.15.27:52390) 2024-09-12T03:22:57.766 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:57.765+0000 7fe63cb2a640 1 -- 172.21.15.27:0/2455404384 learned_addr learned my addr 172.21.15.27:0/2455404384 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:22:57.766 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:57.765+0000 7fe6377fe640 1 --2- 172.21.15.27:0/2455404384 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe63806cc40 0x7fe63806d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:57.766 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:57.766+0000 7fe637fff640 1 -- 172.21.15.27:0/2455404384 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe63806cc40 msgr2=0x7fe63806d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:57.766 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:57.766+0000 7fe637fff640 1 --2- 172.21.15.27:0/2455404384 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe63806cc40 0x7fe63806d060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:57.766 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:57.766+0000 7fe637fff640 1 -- 172.21.15.27:0/2455404384 >> v1:172.21.15.27:6789/0 conn(0x7fe63806db30 legacy=0x7fe63806df60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:57.766 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:57.766+0000 7fe636ffd640 1 -- 172.21.15.27:0/2455404384 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 84286294 0 0) 0x7fe6380d2280 con 0x7fe63806db30 2024-09-12T03:22:57.766 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:57.766+0000 7fe637fff640 1 -- 172.21.15.27:0/2455404384 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe63806d850 con 0x7fe63815e650 2024-09-12T03:22:57.767 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:57.766+0000 7fe637fff640 1 --2- 172.21.15.27:0/2455404384 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe63815e650 0x7fe6380d1d40 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7fe6280097a0 tx=0x7fe62802a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efbbbd9e5e50b35c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:57.768 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:57.767+0000 7fe636ffd640 1 -- 172.21.15.27:0/2455404384 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe628037070 con 0x7fe63815e650 2024-09-12T03:22:57.768 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:57.767+0000 7fe636ffd640 1 -- 172.21.15.27:0/2455404384 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe62802ae50 con 0x7fe63815e650 2024-09-12T03:22:57.768 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:57.767+0000 7fe636ffd640 1 -- 172.21.15.27:0/2455404384 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe628032660 con 0x7fe63815e650 2024-09-12T03:22:57.768 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:57.767+0000 7fe63e58d640 1 -- 172.21.15.27:0/2455404384 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe63815e650 msgr2=0x7fe6380d1d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:57.768 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:57.767+0000 7fe63e58d640 1 --2- 172.21.15.27:0/2455404384 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe63815e650 0x7fe6380d1d40 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7fe6280097a0 tx=0x7fe62802a100 comp rx=0 tx=0).stop 2024-09-12T03:22:57.768 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:57.767+0000 7fe63e58d640 1 -- 172.21.15.27:0/2455404384 shutdown_connections 2024-09-12T03:22:57.768 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:57.767+0000 7fe63e58d640 1 --2- 172.21.15.27:0/2455404384 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe63806cc40 0x7fe63806d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:57.768 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:57.767+0000 7fe63e58d640 1 --2- 172.21.15.27:0/2455404384 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe63815e650 0x7fe6380d1d40 unknown :-1 s=CLOSED pgs=1428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:57.768 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:57.767+0000 7fe63e58d640 1 -- 172.21.15.27:0/2455404384 >> 172.21.15.27:0/2455404384 conn(0x7fe63806f320 msgr2=0x7fe63806f700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:57.768 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:57.767+0000 7fe63e58d640 1 -- 172.21.15.27:0/2455404384 shutdown_connections 2024-09-12T03:22:57.768 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:57.767+0000 7fe63e58d640 1 -- 172.21.15.27:0/2455404384 wait complete. 2024-09-12T03:22:57.770 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:57.768+0000 7fe63e58d640 1 Processor -- start 2024-09-12T03:22:57.770 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:57.769+0000 7fe63e58d640 1 -- start start 2024-09-12T03:22:57.770 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:57.769+0000 7fe63e58d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe63806cc40 0x7fe6380cf5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:57.771 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:57.769+0000 7fe63e58d640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe63806db30 0x7fe6381f48e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:57.771 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:57.769+0000 7fe63e58d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe63815e650 0x7fe6380cc5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:57.771 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:57.769+0000 7fe63e58d640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fe63815ea30 con 0x7fe63815e650 2024-09-12T03:22:57.771 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:57.769+0000 7fe63e58d640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe6380d1540 con 0x7fe63806cc40 2024-09-12T03:22:57.771 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:57.769+0000 7fe63e58d640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fe63815ebb0 con 0x7fe63806db30 2024-09-12T03:22:57.771 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:57.770+0000 7fe63cb2a640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe63815e650 0x7fe6380cc5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:57.771 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:57.770+0000 7fe63cb2a640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe63815e650 0x7fe6380cc5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56420/0 (socket says 172.21.15.27:56420) 2024-09-12T03:22:57.771 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:57.770+0000 7fe63cb2a640 1 -- 172.21.15.27:0/767375919 learned_addr learned my addr 172.21.15.27:0/767375919 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:22:57.771 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:57.770+0000 7fe6377fe640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe63806db30 0x7fe6381f48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:57.772 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:57.770+0000 7fe637fff640 1 --2- 172.21.15.27:0/767375919 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe63806cc40 0x7fe6380cf5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:57.772 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:57.770+0000 7fe63cb2a640 1 -- 172.21.15.27:0/767375919 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe63806db30 msgr2=0x7fe6381f48e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:57.772 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:57.770+0000 7fe63cb2a640 1 --2- 172.21.15.27:0/767375919 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe63806db30 0x7fe6381f48e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:57.772 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:57.770+0000 7fe63cb2a640 1 -- 172.21.15.27:0/767375919 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe63806cc40 msgr2=0x7fe6380cf5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:57.772 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:57.770+0000 7fe63cb2a640 1 --2- 172.21.15.27:0/767375919 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe63806cc40 0x7fe6380cf5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:57.773 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:57.770+0000 7fe63cb2a640 1 -- 172.21.15.27:0/767375919 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6380cd0f0 con 0x7fe63815e650 2024-09-12T03:22:57.773 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:57.770+0000 7fe63cb2a640 1 --2- 172.21.15.27:0/767375919 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe63815e650 0x7fe6380cc5e0 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7fe63000c780 tx=0x7fe63000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:57.773 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:57.770+0000 7fe634ff9640 1 -- 172.21.15.27:0/767375919 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe630018070 con 0x7fe63815e650 2024-09-12T03:22:57.773 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:57.770+0000 7fe634ff9640 1 -- 172.21.15.27:0/767375919 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe63000ce30 con 0x7fe63815e650 2024-09-12T03:22:57.773 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:57.770+0000 7fe634ff9640 1 -- 172.21.15.27:0/767375919 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe63000a840 con 0x7fe63815e650 2024-09-12T03:22:57.773 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:57.770+0000 7fe63e58d640 1 -- 172.21.15.27:0/767375919 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6380cead0 con 0x7fe63815e650 2024-09-12T03:22:57.773 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:57.771+0000 7fe63e58d640 1 -- 172.21.15.27:0/767375919 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6380cf090 con 0x7fe63815e650 2024-09-12T03:22:57.774 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:57.772+0000 7fe63e58d640 1 -- 172.21.15.27:0/767375919 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe63806d060 con 0x7fe63815e650 2024-09-12T03:22:57.774 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:57.772+0000 7fe634ff9640 1 -- 172.21.15.27:0/767375919 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 22) ==== 91059+0+0 (secure 0 0 0) 0x7fe63001b070 con 0x7fe63815e650 2024-09-12T03:22:57.774 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:57.772+0000 7fe634ff9640 1 --2- 172.21.15.27:0/767375919 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fe62406c460 0x7fe62406e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:57.774 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:57.772+0000 7fe634ff9640 1 -- 172.21.15.27:0/767375919 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 5765+0+0 (secure 0 0 0) 0x7fe63008af60 con 0x7fe63815e650 2024-09-12T03:22:57.774 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:57.772+0000 7fe637fff640 1 --2- 172.21.15.27:0/767375919 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fe62406c460 0x7fe62406e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:57.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:57.773+0000 7fe637fff640 1 --2- 172.21.15.27:0/767375919 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fe62406c460 0x7fe62406e920 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fe628034000 tx=0x7fe62802e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:57.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:57.777+0000 7fe634ff9640 1 -- 172.21.15.27:0/767375919 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fe630058940 con 0x7fe63815e650 2024-09-12T03:22:58.009 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:58.009+0000 7fe63e58d640 1 -- 172.21.15.27:0/767375919 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7fe6380d1fc0 con 0x7fe63815e650 2024-09-12T03:22:58.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:58.972+0000 7fe634ff9640 1 -- 172.21.15.27:0/767375919 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v83) ==== 136+0+0 (secure 0 0 0) 0x7fe63005cc40 con 0x7fe63815e650 2024-09-12T03:22:58.975 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:58.975+0000 7fe63e58d640 1 -- 172.21.15.27:0/767375919 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fe62406c460 msgr2=0x7fe62406e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:58.975 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:58.975+0000 7fe63e58d640 1 --2- 172.21.15.27:0/767375919 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fe62406c460 0x7fe62406e920 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fe628034000 tx=0x7fe62802e000 comp rx=0 tx=0).stop 2024-09-12T03:22:58.975 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:58.975+0000 7fe63e58d640 1 -- 172.21.15.27:0/767375919 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe63815e650 msgr2=0x7fe6380cc5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:58.975 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:58.975+0000 7fe63e58d640 1 --2- 172.21.15.27:0/767375919 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe63815e650 0x7fe6380cc5e0 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7fe63000c780 tx=0x7fe63000cc40 comp rx=0 tx=0).stop 2024-09-12T03:22:58.975 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:58.975+0000 7fe63e58d640 1 -- 172.21.15.27:0/767375919 shutdown_connections 2024-09-12T03:22:58.976 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:58.975+0000 7fe63e58d640 1 --2- 172.21.15.27:0/767375919 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fe62406c460 0x7fe62406e920 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:58.976 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:58.975+0000 7fe63e58d640 1 --2- 172.21.15.27:0/767375919 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe63815e650 0x7fe6380cc5e0 unknown :-1 s=CLOSED pgs=1429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:58.976 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:58.975+0000 7fe63e58d640 1 --2- 172.21.15.27:0/767375919 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe63806db30 0x7fe6381f48e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:58.976 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:58.975+0000 7fe63e58d640 1 --2- 172.21.15.27:0/767375919 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe63806cc40 0x7fe6380cf5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:58.976 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:58.975+0000 7fe63e58d640 1 -- 172.21.15.27:0/767375919 >> 172.21.15.27:0/767375919 conn(0x7fe63806f320 msgr2=0x7fe6380b3820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:58.976 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:58.975+0000 7fe63e58d640 1 -- 172.21.15.27:0/767375919 shutdown_connections 2024-09-12T03:22:58.976 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:58.975+0000 7fe63e58d640 1 -- 172.21.15.27:0/767375919 wait complete. 2024-09-12T03:22:58.989 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-12T03:22:59.099 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.099+0000 7febb3e27640 1 Processor -- start 2024-09-12T03:22:59.099 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.099+0000 7febb3e27640 1 -- start start 2024-09-12T03:22:59.100 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.100+0000 7febb3e27640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7febac0b1ca0 0x7febac15f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:59.100 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.100+0000 7febb3e27640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7febac0cf660 0x7febac0d2780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:59.100 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.100+0000 7febb3e27640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7febac15fd70 con 0x7febac06f360 2024-09-12T03:22:59.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.100+0000 7febb3e27640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7febac0d22f0 con 0x7febac0cf660 2024-09-12T03:22:59.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.100+0000 7febb3e27640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7febac06cc40 con 0x7febac0b1ca0 2024-09-12T03:22:59.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.100+0000 7febb23c4640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7febac0cf660 0x7febac0d2780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:59.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.100+0000 7febb23c4640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7febac0cf660 0x7febac0d2780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56422/0 (socket says 172.21.15.27:56422) 2024-09-12T03:22:59.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.100+0000 7febb23c4640 1 -- 172.21.15.27:0/219595790 learned_addr learned my addr 172.21.15.27:0/219595790 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:22:59.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.100+0000 7febb23c4640 1 -- 172.21.15.27:0/219595790 >> v2:172.21.15.45:3300/0 conn(0x7febac0b1ca0 msgr2=0x7febac15f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:59.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.100+0000 7febb23c4640 1 --2- 172.21.15.27:0/219595790 >> v2:172.21.15.45:3300/0 conn(0x7febac0b1ca0 0x7febac15f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:59.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.100+0000 7febb23c4640 1 -- 172.21.15.27:0/219595790 >> v1:172.21.15.45:6789/0 conn(0x7febac06f360 legacy=0x7febac0cef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:59.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.100+0000 7febb23c4640 1 -- 172.21.15.27:0/219595790 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febac0d1ce0 con 0x7febac0cf660 2024-09-12T03:22:59.102 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.101+0000 7febb23c4640 1 --2- 172.21.15.27:0/219595790 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7febac0cf660 0x7febac0d2780 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7febac073ed0 tx=0x7feba402a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=701f5d70f5e390eb server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:59.102 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.101+0000 7febb0bc1640 1 -- 172.21.15.27:0/219595790 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feba4036070 con 0x7febac0cf660 2024-09-12T03:22:59.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.101+0000 7febb0bc1640 1 -- 172.21.15.27:0/219595790 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feba4031600 con 0x7febac0cf660 2024-09-12T03:22:59.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.101+0000 7febb0bc1640 1 -- 172.21.15.27:0/219595790 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feba40319d0 con 0x7febac0cf660 2024-09-12T03:22:59.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.102+0000 7febb3e27640 1 -- 172.21.15.27:0/219595790 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7febac0cf660 msgr2=0x7febac0d2780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:59.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.102+0000 7febb3e27640 1 --2- 172.21.15.27:0/219595790 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7febac0cf660 0x7febac0d2780 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7febac073ed0 tx=0x7feba402a6f0 comp rx=0 tx=0).stop 2024-09-12T03:22:59.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.102+0000 7febb3e27640 1 -- 172.21.15.27:0/219595790 shutdown_connections 2024-09-12T03:22:59.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.102+0000 7febb3e27640 1 --2- 172.21.15.27:0/219595790 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7febac0cf660 0x7febac0d2780 unknown :-1 s=CLOSED pgs=1430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:59.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.102+0000 7febb3e27640 1 --2- 172.21.15.27:0/219595790 >> v2:172.21.15.45:3300/0 conn(0x7febac0b1ca0 0x7febac15f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:59.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.102+0000 7febb3e27640 1 -- 172.21.15.27:0/219595790 >> 172.21.15.27:0/219595790 conn(0x7febac073800 msgr2=0x7febac09ba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:59.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.102+0000 7febb3e27640 1 -- 172.21.15.27:0/219595790 shutdown_connections 2024-09-12T03:22:59.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.102+0000 7febb3e27640 1 -- 172.21.15.27:0/219595790 wait complete. 2024-09-12T03:22:59.105 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.103+0000 7febb3e27640 1 Processor -- start 2024-09-12T03:22:59.105 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.103+0000 7febb3e27640 1 -- start start 2024-09-12T03:22:59.105 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.104+0000 7febb3e27640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7febac06f360 0x7febac151fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:59.105 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.104+0000 7febb3e27640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7febac0b1ca0 0x7febac09d040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:59.105 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.104+0000 7febb3e27640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7febac0cf660 0x7febac0b6520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:59.106 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.104+0000 7febb3e27640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7febac06d1c0 con 0x7febac06f360 2024-09-12T03:22:59.106 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.104+0000 7febb3e27640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7febac0d1f80 con 0x7febac0b1ca0 2024-09-12T03:22:59.106 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.104+0000 7febb3e27640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7febac06d340 con 0x7febac0cf660 2024-09-12T03:22:59.106 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.104+0000 7febb1bc3640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7febac06f360 0x7febac151fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:59.106 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.104+0000 7febb1bc3640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7febac06f360 0x7febac151fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56436/0 (socket says 172.21.15.27:56436) 2024-09-12T03:22:59.106 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.104+0000 7febb1bc3640 1 -- 172.21.15.27:0/2361307814 learned_addr learned my addr 172.21.15.27:0/2361307814 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:22:59.106 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.104+0000 7febb1bc3640 1 -- 172.21.15.27:0/2361307814 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7febac0cf660 msgr2=0x7febac0b6520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:59.106 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.104+0000 7febb1bc3640 1 --2- 172.21.15.27:0/2361307814 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7febac0cf660 0x7febac0b6520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:59.106 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.104+0000 7febb1bc3640 1 -- 172.21.15.27:0/2361307814 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7febac0b1ca0 msgr2=0x7febac09d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:59.106 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.104+0000 7febb1bc3640 1 --2- 172.21.15.27:0/2361307814 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7febac0b1ca0 0x7febac09d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:59.106 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.105+0000 7febb1bc3640 1 -- 172.21.15.27:0/2361307814 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febac0cb8f0 con 0x7febac06f360 2024-09-12T03:22:59.106 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.105+0000 7febb1bc3640 1 --2- 172.21.15.27:0/2361307814 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7febac06f360 0x7febac151fe0 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7feba800b4a0 tx=0x7feba800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:59.106 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.105+0000 7feba27fc640 1 -- 172.21.15.27:0/2361307814 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feba8014070 con 0x7febac06f360 2024-09-12T03:22:59.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.105+0000 7feba27fc640 1 -- 172.21.15.27:0/2361307814 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feba8010610 con 0x7febac06f360 2024-09-12T03:22:59.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.105+0000 7feba27fc640 1 -- 172.21.15.27:0/2361307814 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feba8010a00 con 0x7febac06f360 2024-09-12T03:22:59.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.105+0000 7febb3e27640 1 -- 172.21.15.27:0/2361307814 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febac0cbbd0 con 0x7febac06f360 2024-09-12T03:22:59.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.105+0000 7febb3e27640 1 -- 172.21.15.27:0/2361307814 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febac0cc190 con 0x7febac06f360 2024-09-12T03:22:59.108 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.106+0000 7febb3e27640 1 -- 172.21.15.27:0/2361307814 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7febac0cc680 con 0x7febac06f360 2024-09-12T03:22:59.108 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.107+0000 7feba27fc640 1 -- 172.21.15.27:0/2361307814 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 22) ==== 91059+0+0 (secure 0 0 0) 0x7feba8017070 con 0x7febac06f360 2024-09-12T03:22:59.108 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.107+0000 7feba27fc640 1 --2- 172.21.15.27:0/2361307814 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7feb9006c460 0x7feb9006e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:59.108 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.107+0000 7febb13c2640 1 --2- 172.21.15.27:0/2361307814 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7feb9006c460 0x7feb9006e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:59.108 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.107+0000 7feba27fc640 1 -- 172.21.15.27:0/2361307814 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 5765+0+0 (secure 0 0 0) 0x7feba808b410 con 0x7febac06f360 2024-09-12T03:22:59.108 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.108+0000 7febb13c2640 1 --2- 172.21.15.27:0/2361307814 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7feb9006c460 0x7feb9006e920 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7febac160750 tx=0x7feb9c007990 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:59.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.111+0000 7feba27fc640 1 -- 172.21.15.27:0/2361307814 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7feba80594d0 con 0x7febac06f360 2024-09-12T03:22:59.311 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.311+0000 7febb3e27640 1 -- 172.21.15.27:0/2361307814 --> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7febac0d1810 con 0x7feb9006c460 2024-09-12T03:22:59.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.313+0000 7feba27fc640 1 -- 172.21.15.27:0/2361307814 <== mgr.4113 v2:172.21.15.27:6836/1226920526 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7febac0d1810 con 0x7feb9006c460 2024-09-12T03:22:59.313 INFO:teuthology.orchestra.run.smithi027.stdout:[ 2024-09-12T03:22:59.313 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:22:59.313 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "cephfs" 2024-09-12T03:22:59.313 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:22:59.313 INFO:teuthology.orchestra.run.smithi027.stdout:] 2024-09-12T03:22:59.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.315+0000 7febb3e27640 1 -- 172.21.15.27:0/2361307814 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7feb9006c460 msgr2=0x7feb9006e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:59.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.315+0000 7febb3e27640 1 --2- 172.21.15.27:0/2361307814 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7feb9006c460 0x7feb9006e920 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7febac160750 tx=0x7feb9c007990 comp rx=0 tx=0).stop 2024-09-12T03:22:59.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.315+0000 7febb3e27640 1 -- 172.21.15.27:0/2361307814 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7febac06f360 msgr2=0x7febac151fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:59.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.315+0000 7febb3e27640 1 --2- 172.21.15.27:0/2361307814 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7febac06f360 0x7febac151fe0 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7feba800b4a0 tx=0x7feba800b960 comp rx=0 tx=0).stop 2024-09-12T03:22:59.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.315+0000 7febb3e27640 1 -- 172.21.15.27:0/2361307814 shutdown_connections 2024-09-12T03:22:59.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.315+0000 7febb3e27640 1 --2- 172.21.15.27:0/2361307814 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7febac0cf660 0x7febac0b6520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:59.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.315+0000 7febb3e27640 1 --2- 172.21.15.27:0/2361307814 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7febac0b1ca0 0x7febac09d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:59.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.315+0000 7febb3e27640 1 --2- 172.21.15.27:0/2361307814 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7feb9006c460 0x7feb9006e920 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:59.316 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.315+0000 7febb3e27640 1 --2- 172.21.15.27:0/2361307814 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7febac06f360 0x7febac151fe0 unknown :-1 s=CLOSED pgs=1431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:59.316 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.315+0000 7febb3e27640 1 -- 172.21.15.27:0/2361307814 >> 172.21.15.27:0/2361307814 conn(0x7febac073800 msgr2=0x7febac0cef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:59.316 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.315+0000 7febb3e27640 1 -- 172.21.15.27:0/2361307814 shutdown_connections 2024-09-12T03:22:59.316 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.315+0000 7febb3e27640 1 -- 172.21.15.27:0/2361307814 wait complete. 2024-09-12T03:22:59.326 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-12T03:22:59.431 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.431+0000 7f95284dd640 1 Processor -- start 2024-09-12T03:22:59.431 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.431+0000 7f95284dd640 1 -- start start 2024-09-12T03:22:59.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.432+0000 7f95284dd640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f952015ec30 0x7f95200b3900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:59.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.432+0000 7f95284dd640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f95200af9d0 0x7f95200b3e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:59.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.432+0000 7f95284dd640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f95200b4380 0x7f952015f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:59.434 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.432+0000 7f95284dd640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f95200d02d0 con 0x7f95200b4380 2024-09-12T03:22:59.434 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.432+0000 7f95284dd640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f952006c650 con 0x7f952015ec30 2024-09-12T03:22:59.434 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.432+0000 7f95284dd640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f95200ce5e0 con 0x7f95200af9d0 2024-09-12T03:22:59.434 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.432+0000 7f9526a7a640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f95200b4380 0x7f952015f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:59.434 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.432+0000 7f9526a7a640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f95200b4380 0x7f952015f2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59656/0 (socket says 172.21.15.27:59656) 2024-09-12T03:22:59.434 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.432+0000 7f9526a7a640 1 -- 172.21.15.27:0/2474685176 learned_addr learned my addr 172.21.15.27:0/2474685176 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:22:59.434 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.432+0000 7f9526279640 1 --2- 172.21.15.27:0/2474685176 >> v2:172.21.15.45:3300/0 conn(0x7f952015ec30 0x7f95200b3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:59.434 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.432+0000 7f9525a78640 1 --2- 172.21.15.27:0/2474685176 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f95200af9d0 0x7f95200b3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:59.434 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.433+0000 7f9526a7a640 1 -- 172.21.15.27:0/2474685176 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f95200af9d0 msgr2=0x7f95200b3e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:59.434 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.433+0000 7f9526a7a640 1 --2- 172.21.15.27:0/2474685176 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f95200af9d0 0x7f95200b3e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:59.434 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.433+0000 7f9526a7a640 1 -- 172.21.15.27:0/2474685176 >> v2:172.21.15.45:3300/0 conn(0x7f952015ec30 msgr2=0x7f95200b3900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:59.434 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.433+0000 7f9526a7a640 1 --2- 172.21.15.27:0/2474685176 >> v2:172.21.15.45:3300/0 conn(0x7f952015ec30 0x7f95200b3900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:59.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.433+0000 7f9526a7a640 1 -- 172.21.15.27:0/2474685176 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95200ceac0 con 0x7f95200b4380 2024-09-12T03:22:59.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.433+0000 7f9526a7a640 1 --2- 172.21.15.27:0/2474685176 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f95200b4380 0x7f952015f2a0 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7f952006fb80 tx=0x7f951002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad876c77f5a981e0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:59.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.433+0000 7f9525277640 1 -- 172.21.15.27:0/2474685176 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9510036070 con 0x7f95200b4380 2024-09-12T03:22:59.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.433+0000 7f9525277640 1 -- 172.21.15.27:0/2474685176 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9510031600 con 0x7f95200b4380 2024-09-12T03:22:59.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.434+0000 7f9525277640 1 -- 172.21.15.27:0/2474685176 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95100319d0 con 0x7f95200b4380 2024-09-12T03:22:59.437 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.434+0000 7f95284dd640 1 -- 172.21.15.27:0/2474685176 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f95200b4380 msgr2=0x7f952015f2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:59.438 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.434+0000 7f95284dd640 1 --2- 172.21.15.27:0/2474685176 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f95200b4380 0x7f952015f2a0 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7f952006fb80 tx=0x7f951002a6a0 comp rx=0 tx=0).stop 2024-09-12T03:22:59.438 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.434+0000 7f95284dd640 1 -- 172.21.15.27:0/2474685176 shutdown_connections 2024-09-12T03:22:59.438 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.434+0000 7f95284dd640 1 --2- 172.21.15.27:0/2474685176 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f95200b4380 0x7f952015f2a0 unknown :-1 s=CLOSED pgs=1432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:59.438 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.434+0000 7f95284dd640 1 --2- 172.21.15.27:0/2474685176 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f95200af9d0 0x7f95200b3e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:59.438 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.434+0000 7f95284dd640 1 --2- 172.21.15.27:0/2474685176 >> v2:172.21.15.45:3300/0 conn(0x7f952015ec30 0x7f95200b3900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:59.438 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.434+0000 7f95284dd640 1 -- 172.21.15.27:0/2474685176 >> 172.21.15.27:0/2474685176 conn(0x7f952006bb30 msgr2=0x7f952006bf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:59.438 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.434+0000 7f95284dd640 1 -- 172.21.15.27:0/2474685176 shutdown_connections 2024-09-12T03:22:59.438 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.434+0000 7f95284dd640 1 -- 172.21.15.27:0/2474685176 wait complete. 2024-09-12T03:22:59.438 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.435+0000 7f95284dd640 1 Processor -- start 2024-09-12T03:22:59.438 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.435+0000 7f95284dd640 1 -- start start 2024-09-12T03:22:59.439 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.436+0000 7f95284dd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f95200af9d0 0x7f9520152a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:59.439 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.436+0000 7f95284dd640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f95200b4380 0x7f9520151000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:59.439 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.436+0000 7f95284dd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f952015ec30 0x7f952015b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:59.439 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.436+0000 7f95284dd640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f95200cfff0 con 0x7f95200af9d0 2024-09-12T03:22:59.439 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.436+0000 7f95284dd640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f952006c8a0 con 0x7f952015ec30 2024-09-12T03:22:59.439 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.436+0000 7f95284dd640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f95200715f0 con 0x7f95200b4380 2024-09-12T03:22:59.439 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.436+0000 7f9526279640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f95200af9d0 0x7f9520152a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:59.439 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.436+0000 7f9526279640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f95200af9d0 0x7f9520152a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59660/0 (socket says 172.21.15.27:59660) 2024-09-12T03:22:59.439 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.436+0000 7f9526279640 1 -- 172.21.15.27:0/2500267964 learned_addr learned my addr 172.21.15.27:0/2500267964 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:22:59.439 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.436+0000 7f9525a78640 1 --2- 172.21.15.27:0/2500267964 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f95200b4380 0x7f9520151000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:59.439 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.436+0000 7f9526a7a640 1 --2- 172.21.15.27:0/2500267964 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f952015ec30 0x7f952015b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:59.439 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.436+0000 7f9526279640 1 -- 172.21.15.27:0/2500267964 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f95200b4380 msgr2=0x7f9520151000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:59.439 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.436+0000 7f9526279640 1 --2- 172.21.15.27:0/2500267964 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f95200b4380 0x7f9520151000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:59.440 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.436+0000 7f9526279640 1 -- 172.21.15.27:0/2500267964 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f952015ec30 msgr2=0x7f952015b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:59.440 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.436+0000 7f9526279640 1 --2- 172.21.15.27:0/2500267964 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f952015ec30 0x7f952015b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:59.440 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.436+0000 7f9526279640 1 -- 172.21.15.27:0/2500267964 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f952014e470 con 0x7f95200af9d0 2024-09-12T03:22:59.440 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.437+0000 7f9526279640 1 --2- 172.21.15.27:0/2500267964 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f95200af9d0 0x7f9520152a30 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7f951800e8a0 tx=0x7f951800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:59.440 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.437+0000 7f951effd640 1 -- 172.21.15.27:0/2500267964 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9518017070 con 0x7f95200af9d0 2024-09-12T03:22:59.440 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.437+0000 7f951effd640 1 -- 172.21.15.27:0/2500267964 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f951800c940 con 0x7f95200af9d0 2024-09-12T03:22:59.440 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.437+0000 7f951effd640 1 -- 172.21.15.27:0/2500267964 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f951800cd30 con 0x7f95200af9d0 2024-09-12T03:22:59.440 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.437+0000 7f95284dd640 1 -- 172.21.15.27:0/2500267964 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9520152f70 con 0x7f95200af9d0 2024-09-12T03:22:59.440 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.437+0000 7f95284dd640 1 -- 172.21.15.27:0/2500267964 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9520153510 con 0x7f95200af9d0 2024-09-12T03:22:59.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.438+0000 7f95284dd640 1 -- 172.21.15.27:0/2500267964 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95200b3900 con 0x7f95200af9d0 2024-09-12T03:22:59.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.438+0000 7f951effd640 1 -- 172.21.15.27:0/2500267964 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 22) ==== 91059+0+0 (secure 0 0 0) 0x7f951801a070 con 0x7f95200af9d0 2024-09-12T03:22:59.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.439+0000 7f951effd640 1 --2- 172.21.15.27:0/2500267964 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f950c06c460 0x7f950c06e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:59.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.439+0000 7f951effd640 1 -- 172.21.15.27:0/2500267964 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 5765+0+0 (secure 0 0 0) 0x7f951808b8d0 con 0x7f95200af9d0 2024-09-12T03:22:59.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.439+0000 7f9525a78640 1 --2- 172.21.15.27:0/2500267964 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f950c06c460 0x7f950c06e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:59.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.439+0000 7f9525a78640 1 --2- 172.21.15.27:0/2500267964 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f950c06c460 0x7f950c06e920 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f95200cc7a0 tx=0x7f9514008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:59.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.443+0000 7f951effd640 1 -- 172.21.15.27:0/2500267964 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f9518059900 con 0x7f95200af9d0 2024-09-12T03:22:59.624 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.624+0000 7f95284dd640 1 -- 172.21.15.27:0/2500267964 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f952006f240 con 0x7f95200af9d0 2024-09-12T03:22:59.624 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.624+0000 7f951effd640 1 -- 172.21.15.27:0/2500267964 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v26)=0 v26) ==== 122+0+0 (secure 0 0 0) 0x7f9518059200 con 0x7f95200af9d0 2024-09-12T03:22:59.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.626+0000 7f95284dd640 1 -- 172.21.15.27:0/2500267964 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f950c06c460 msgr2=0x7f950c06e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:59.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.626+0000 7f95284dd640 1 --2- 172.21.15.27:0/2500267964 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f950c06c460 0x7f950c06e920 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f95200cc7a0 tx=0x7f9514008040 comp rx=0 tx=0).stop 2024-09-12T03:22:59.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.626+0000 7f95284dd640 1 -- 172.21.15.27:0/2500267964 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f95200af9d0 msgr2=0x7f9520152a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:59.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.626+0000 7f95284dd640 1 --2- 172.21.15.27:0/2500267964 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f95200af9d0 0x7f9520152a30 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7f951800e8a0 tx=0x7f951800ed60 comp rx=0 tx=0).stop 2024-09-12T03:22:59.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.626+0000 7f95284dd640 1 -- 172.21.15.27:0/2500267964 shutdown_connections 2024-09-12T03:22:59.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.626+0000 7f95284dd640 1 --2- 172.21.15.27:0/2500267964 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f952015ec30 0x7f952015b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:59.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.626+0000 7f95284dd640 1 --2- 172.21.15.27:0/2500267964 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f95200b4380 0x7f9520151000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:59.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.626+0000 7f95284dd640 1 --2- 172.21.15.27:0/2500267964 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f950c06c460 0x7f950c06e920 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:59.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.626+0000 7f95284dd640 1 --2- 172.21.15.27:0/2500267964 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f95200af9d0 0x7f9520152a30 unknown :-1 s=CLOSED pgs=1433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:59.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.626+0000 7f95284dd640 1 -- 172.21.15.27:0/2500267964 >> 172.21.15.27:0/2500267964 conn(0x7f952006bb30 msgr2=0x7f9520072470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:59.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.627+0000 7f95284dd640 1 -- 172.21.15.27:0/2500267964 shutdown_connections 2024-09-12T03:22:59.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.627+0000 7f95284dd640 1 -- 172.21.15.27:0/2500267964 wait complete. 2024-09-12T03:22:59.637 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:22:59.637 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p ./volumes/_nogroup/subvol530) 2024-09-12T03:22:59.669 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:22:59.669 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p ./volumes/subvol_grp18820/subvol98201) 2024-09-12T03:22:59.737 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol530 2024-09-12T03:22:59.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.884+0000 7f5e8ec1e640 1 Processor -- start 2024-09-12T03:22:59.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.884+0000 7f5e8ec1e640 1 -- start start 2024-09-12T03:22:59.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.885+0000 7f5e8ec1e640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f5e880b6230 0x7f5e880b6690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:59.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.885+0000 7f5e8ec1e640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f5e880b3600 0x7f5e880b3a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:59.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.885+0000 7f5e8ec1e640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5e8815ed70 con 0x7f5e8815e990 2024-09-12T03:22:59.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.885+0000 7f5e8ec1e640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f5e880b6bd0 con 0x7f5e880b3600 2024-09-12T03:22:59.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.885+0000 7f5e8ec1e640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f5e880b7d10 con 0x7f5e880b6230 2024-09-12T03:22:59.886 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.885+0000 7f5e87fff640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f5e880b6230 0x7f5e880b6690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:59.886 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.886+0000 7f5e87fff640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f5e880b6230 0x7f5e880b6690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3301/0 says I am v2:172.21.15.27:39158/0 (socket says 172.21.15.27:39158) 2024-09-12T03:22:59.886 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.886+0000 7f5e8d1bb640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f5e880b3600 0x7f5e880b3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:59.886 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.886+0000 7f5e87fff640 1 -- 172.21.15.27:0/1678530037 learned_addr learned my addr 172.21.15.27:0/1678530037 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:22:59.886 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.886+0000 7f5e87fff640 1 -- 172.21.15.27:0/1678530037 >> v2:172.21.15.45:3300/0 conn(0x7f5e880b3600 msgr2=0x7f5e880b3a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:59.886 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.886+0000 7f5e87fff640 1 --2- 172.21.15.27:0/1678530037 >> v2:172.21.15.45:3300/0 conn(0x7f5e880b3600 0x7f5e880b3a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:59.886 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.886+0000 7f5e87fff640 1 -- 172.21.15.27:0/1678530037 >> v1:172.21.15.45:6789/0 conn(0x7f5e8815e990 legacy=0x7f5e880b7600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:59.886 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.886+0000 7f5e87fff640 1 -- 172.21.15.27:0/1678530037 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e880b3fb0 con 0x7f5e880b6230 2024-09-12T03:22:59.888 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.887+0000 7f5e87fff640 1 --2- 172.21.15.27:0/1678530037 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f5e880b6230 0x7f5e880b6690 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f5e7c009870 tx=0x7f5e7c02a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bec5fbaf7508601b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:59.888 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.887+0000 7f5e877fe640 1 -- 172.21.15.27:0/1678530037 <== mon.2 v2:172.21.15.45:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e7c037070 con 0x7f5e880b6230 2024-09-12T03:22:59.888 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.887+0000 7f5e877fe640 1 -- 172.21.15.27:0/1678530037 <== mon.2 v2:172.21.15.45:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e7c032440 con 0x7f5e880b6230 2024-09-12T03:22:59.888 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.887+0000 7f5e877fe640 1 -- 172.21.15.27:0/1678530037 <== mon.2 v2:172.21.15.45:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e7c032810 con 0x7f5e880b6230 2024-09-12T03:22:59.888 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.887+0000 7f5e8ec1e640 1 -- 172.21.15.27:0/1678530037 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f5e880b6230 msgr2=0x7f5e880b6690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:59.888 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.887+0000 7f5e8ec1e640 1 --2- 172.21.15.27:0/1678530037 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f5e880b6230 0x7f5e880b6690 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f5e7c009870 tx=0x7f5e7c02a270 comp rx=0 tx=0).stop 2024-09-12T03:22:59.888 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.887+0000 7f5e8ec1e640 1 -- 172.21.15.27:0/1678530037 shutdown_connections 2024-09-12T03:22:59.888 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.887+0000 7f5e8ec1e640 1 --2- 172.21.15.27:0/1678530037 >> v2:172.21.15.45:3300/0 conn(0x7f5e880b3600 0x7f5e880b3a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:59.888 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.887+0000 7f5e8ec1e640 1 --2- 172.21.15.27:0/1678530037 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f5e880b6230 0x7f5e880b6690 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:59.888 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.887+0000 7f5e8ec1e640 1 -- 172.21.15.27:0/1678530037 >> 172.21.15.27:0/1678530037 conn(0x7f5e880d1230 msgr2=0x7f5e880d1640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:59.888 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.888+0000 7f5e8ec1e640 1 -- 172.21.15.27:0/1678530037 shutdown_connections 2024-09-12T03:22:59.889 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.888+0000 7f5e8ec1e640 1 -- 172.21.15.27:0/1678530037 wait complete. 2024-09-12T03:22:59.891 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.889+0000 7f5e8ec1e640 1 Processor -- start 2024-09-12T03:22:59.891 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.889+0000 7f5e8ec1e640 1 -- start start 2024-09-12T03:22:59.891 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.890+0000 7f5e8ec1e640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f5e880b3600 0x7f5e88147c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:59.891 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.890+0000 7f5e8ec1e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5e880b6230 0x7f5e88148160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:59.891 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.890+0000 7f5e8ec1e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5e8815e990 0x7f5e881585a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:59.891 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.890+0000 7f5e8ec1e640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f5e880d2ff0 con 0x7f5e880b6230 2024-09-12T03:22:59.891 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.890+0000 7f5e8ec1e640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f5e880d2d10 con 0x7f5e8815e990 2024-09-12T03:22:59.891 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.890+0000 7f5e8ec1e640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f5e880d32d0 con 0x7f5e880b3600 2024-09-12T03:22:59.891 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.890+0000 7f5e87fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5e880b6230 0x7f5e88148160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:59.891 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.890+0000 7f5e87fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5e880b6230 0x7f5e88148160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59664/0 (socket says 172.21.15.27:59664) 2024-09-12T03:22:59.892 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.890+0000 7f5e87fff640 1 -- 172.21.15.27:0/332022619 learned_addr learned my addr 172.21.15.27:0/332022619 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:22:59.892 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.890+0000 7f5e8d1bb640 1 --2- 172.21.15.27:0/332022619 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5e8815e990 0x7f5e881585a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:59.892 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.890+0000 7f5e8c9ba640 1 --2- 172.21.15.27:0/332022619 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f5e880b3600 0x7f5e88147c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:59.892 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.890+0000 7f5e87fff640 1 -- 172.21.15.27:0/332022619 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f5e880b3600 msgr2=0x7f5e88147c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:59.892 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.890+0000 7f5e87fff640 1 --2- 172.21.15.27:0/332022619 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f5e880b3600 0x7f5e88147c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:59.892 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.890+0000 7f5e87fff640 1 -- 172.21.15.27:0/332022619 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5e8815e990 msgr2=0x7f5e881585a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:59.892 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.890+0000 7f5e87fff640 1 --2- 172.21.15.27:0/332022619 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5e8815e990 0x7f5e881585a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:59.892 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.890+0000 7f5e87fff640 1 -- 172.21.15.27:0/332022619 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e88146c10 con 0x7f5e880b6230 2024-09-12T03:22:59.892 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.890+0000 7f5e87fff640 1 --2- 172.21.15.27:0/332022619 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5e880b6230 0x7f5e88148160 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7f5e7c000c00 tx=0x7f5e7c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:59.892 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.890+0000 7f5e857fa640 1 -- 172.21.15.27:0/332022619 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e7c037040 con 0x7f5e880b6230 2024-09-12T03:22:59.892 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.891+0000 7f5e857fa640 1 -- 172.21.15.27:0/332022619 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e7c042070 con 0x7f5e880b6230 2024-09-12T03:22:59.892 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.891+0000 7f5e8ec1e640 1 -- 172.21.15.27:0/332022619 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e88146ef0 con 0x7f5e880b6230 2024-09-12T03:22:59.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.891+0000 7f5e857fa640 1 -- 172.21.15.27:0/332022619 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e7c030880 con 0x7f5e880b6230 2024-09-12T03:22:59.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.891+0000 7f5e8ec1e640 1 -- 172.21.15.27:0/332022619 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e881474b0 con 0x7f5e880b6230 2024-09-12T03:22:59.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.891+0000 7f5e8ec1e640 1 -- 172.21.15.27:0/332022619 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e880b6020 con 0x7f5e880b6230 2024-09-12T03:22:59.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.892+0000 7f5e857fa640 1 -- 172.21.15.27:0/332022619 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 22) ==== 91059+0+0 (secure 0 0 0) 0x7f5e7c030a20 con 0x7f5e880b6230 2024-09-12T03:22:59.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.893+0000 7f5e857fa640 1 --2- 172.21.15.27:0/332022619 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f5e7406c460 0x7f5e7406e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:59.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.893+0000 7f5e857fa640 1 -- 172.21.15.27:0/332022619 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 5765+0+0 (secure 0 0 0) 0x7f5e7c0ac0f0 con 0x7f5e880b6230 2024-09-12T03:22:59.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.893+0000 7f5e8c9ba640 1 --2- 172.21.15.27:0/332022619 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f5e7406c460 0x7f5e7406e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:59.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.893+0000 7f5e8c9ba640 1 --2- 172.21.15.27:0/332022619 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f5e7406c460 0x7f5e7406e920 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f5e880ce610 tx=0x7f5e78009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:59.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:22:59.897+0000 7f5e857fa640 1 -- 172.21.15.27:0/332022619 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f5e880b6020 con 0x7f5e880b6230 2024-09-12T03:23:00.100 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:00.100+0000 7f5e8ec1e640 1 -- 172.21.15.27:0/332022619 --> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol530", "target": ["mon-mgr", ""]}) -- 0x7f5e88062a40 con 0x7f5e7406c460 2024-09-12T03:23:00.104 INFO:tasks.ceph.mgr.y.smithi027.stderr:2024-09-12T03:23:00.104+0000 7f325d8e3640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:23:00.104 INFO:tasks.ceph.mgr.y.smithi027.stderr:2024-09-12T03:23:00.104+0000 7f325d8e3640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:23:00.105 INFO:tasks.ceph.mgr.y.smithi027.stderr:2024-09-12T03:23:00.104+0000 7f325d8e3640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:23:00.105 INFO:tasks.ceph.mgr.y.smithi027.stderr:2024-09-12T03:23:00.105+0000 7f325d8e3640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:23:00.105 INFO:tasks.ceph.mgr.y.smithi027.stderr:2024-09-12T03:23:00.105+0000 7f325d8e3640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:23:00.130 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:00.130+0000 7f5e857fa640 1 -- 172.21.15.27:0/332022619 <== mgr.4113 v2:172.21.15.27:6836/1226920526 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7f5e88062a40 con 0x7f5e7406c460 2024-09-12T03:23:00.130 INFO:teuthology.orchestra.run.smithi027.stdout:/volumes/_nogroup/subvol530 2024-09-12T03:23:00.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:00.133+0000 7f5e8ec1e640 1 -- 172.21.15.27:0/332022619 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f5e7406c460 msgr2=0x7f5e7406e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:00.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:00.133+0000 7f5e8ec1e640 1 --2- 172.21.15.27:0/332022619 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f5e7406c460 0x7f5e7406e920 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f5e880ce610 tx=0x7f5e78009040 comp rx=0 tx=0).stop 2024-09-12T03:23:00.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:00.133+0000 7f5e8ec1e640 1 -- 172.21.15.27:0/332022619 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5e880b6230 msgr2=0x7f5e88148160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:00.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:00.133+0000 7f5e8ec1e640 1 --2- 172.21.15.27:0/332022619 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5e880b6230 0x7f5e88148160 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7f5e7c000c00 tx=0x7f5e7c0046c0 comp rx=0 tx=0).stop 2024-09-12T03:23:00.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:00.133+0000 7f5e8ec1e640 1 -- 172.21.15.27:0/332022619 shutdown_connections 2024-09-12T03:23:00.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:00.133+0000 7f5e8ec1e640 1 --2- 172.21.15.27:0/332022619 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5e8815e990 0x7f5e881585a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:00.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:00.133+0000 7f5e8ec1e640 1 --2- 172.21.15.27:0/332022619 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f5e7406c460 0x7f5e7406e920 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:00.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:00.133+0000 7f5e8ec1e640 1 --2- 172.21.15.27:0/332022619 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5e880b6230 0x7f5e88148160 unknown :-1 s=CLOSED pgs=1434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:00.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:00.133+0000 7f5e8ec1e640 1 --2- 172.21.15.27:0/332022619 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f5e880b3600 0x7f5e88147c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:00.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:00.133+0000 7f5e8ec1e640 1 -- 172.21.15.27:0/332022619 >> 172.21.15.27:0/332022619 conn(0x7f5e880d1230 msgr2=0x7f5e880b5910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:00.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:00.133+0000 7f5e8ec1e640 1 -- 172.21.15.27:0/332022619 shutdown_connections 2024-09-12T03:23:00.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:00.133+0000 7f5e8ec1e640 1 -- 172.21.15.27:0/332022619 wait complete. 2024-09-12T03:23:00.144 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol98201 subvol_grp18820 2024-09-12T03:23:00.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:00.252+0000 7fdaab76f640 1 Processor -- start 2024-09-12T03:23:00.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:00.252+0000 7fdaab76f640 1 -- start start 2024-09-12T03:23:00.253 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:00.253+0000 7fdaab76f640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fdaa40af9d0 0x7fdaa40b4010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:00.253 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:00.253+0000 7fdaab76f640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fdaa406da00 0x7fdaa406de90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:00.253 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:00.253+0000 7fdaab76f640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdaa40ce5e0 con 0x7fdaa415ec30 2024-09-12T03:23:00.253 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:00.253+0000 7fdaab76f640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7fdaa40d02d0 con 0x7fdaa40af9d0 2024-09-12T03:23:00.254 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:00.253+0000 7fdaab76f640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fdaa406c650 con 0x7fdaa406da00 2024-09-12T03:23:00.254 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:00.253+0000 7fdaa8d0a640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fdaa40af9d0 0x7fdaa40b4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:00.254 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:00.253+0000 7fdaa950b640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fdaa415ec30 0x7fdaa40b3900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:54096/0 (socket says 172.21.15.27:54096) 2024-09-12T03:23:00.254 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:00.253+0000 7fdaa950b640 1 -- 172.21.15.27:0/2823274740 learned_addr learned my addr 172.21.15.27:0/2823274740 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:23:00.254 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:00.253+0000 7fdaa9d0c640 1 --2- 172.21.15.27:0/2823274740 >> v2:172.21.15.45:3300/0 conn(0x7fdaa406da00 0x7fdaa406de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:00.254 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:00.253+0000 7fdaa8d0a640 1 -- 172.21.15.27:0/2823274740 >> v2:172.21.15.45:3300/0 conn(0x7fdaa406da00 msgr2=0x7fdaa406de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:00.254 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:00.253+0000 7fdaa8d0a640 1 --2- 172.21.15.27:0/2823274740 >> v2:172.21.15.45:3300/0 conn(0x7fdaa406da00 0x7fdaa406de90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:00.254 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:00.253+0000 7fdaa8d0a640 1 -- 172.21.15.27:0/2823274740 >> v1:172.21.15.27:6789/0 conn(0x7fdaa415ec30 legacy=0x7fdaa40b3900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:00.254 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:00.253+0000 7fdaa8d0a640 1 -- 172.21.15.27:0/2823274740 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdaa40ceac0 con 0x7fdaa40af9d0 2024-09-12T03:23:00.255 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:00.254+0000 7fdaa8d0a640 1 --2- 172.21.15.27:0/2823274740 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdaa40af9d0 0x7fdaa40b4010 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7fdaa40633b0 tx=0x7fda9402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c686662029419e5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:00.255 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:00.254+0000 7fda9bfff640 1 -- 172.21.15.27:0/2823274740 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda94037070 con 0x7fdaa40af9d0 2024-09-12T03:23:00.255 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:00.254+0000 7fda9bfff640 1 -- 172.21.15.27:0/2823274740 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda94032440 con 0x7fdaa40af9d0 2024-09-12T03:23:00.256 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:00.254+0000 7fda9bfff640 1 -- 172.21.15.27:0/2823274740 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda94032810 con 0x7fdaa40af9d0 2024-09-12T03:23:00.256 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:00.255+0000 7fdaab76f640 1 -- 172.21.15.27:0/2823274740 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdaa40af9d0 msgr2=0x7fdaa40b4010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:00.256 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:00.255+0000 7fdaab76f640 1 --2- 172.21.15.27:0/2823274740 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdaa40af9d0 0x7fdaa40b4010 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7fdaa40633b0 tx=0x7fda9402a1a0 comp rx=0 tx=0).stop 2024-09-12T03:23:00.256 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:00.255+0000 7fdaab76f640 1 -- 172.21.15.27:0/2823274740 shutdown_connections 2024-09-12T03:23:00.256 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:00.255+0000 7fdaab76f640 1 --2- 172.21.15.27:0/2823274740 >> v2:172.21.15.45:3300/0 conn(0x7fdaa406da00 0x7fdaa406de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:00.256 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:00.255+0000 7fdaab76f640 1 --2- 172.21.15.27:0/2823274740 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdaa40af9d0 0x7fdaa40b4010 unknown :-1 s=CLOSED pgs=1438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:00.256 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:00.255+0000 7fdaab76f640 1 -- 172.21.15.27:0/2823274740 >> 172.21.15.27:0/2823274740 conn(0x7fdaa406bb30 msgr2=0x7fdaa406bf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:00.256 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:00.255+0000 7fdaab76f640 1 -- 172.21.15.27:0/2823274740 shutdown_connections 2024-09-12T03:23:00.256 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:00.255+0000 7fdaab76f640 1 -- 172.21.15.27:0/2823274740 wait complete. 2024-09-12T03:23:00.258 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:00.256+0000 7fdaab76f640 1 Processor -- start 2024-09-12T03:23:00.258 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:00.256+0000 7fdaab76f640 1 -- start start 2024-09-12T03:23:00.258 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:00.257+0000 7fdaab76f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdaa406da00 0x7fdaa40af160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:00.258 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:00.257+0000 7fdaab76f640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdaa40af9d0 0x7fdaa40b3b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:00.259 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:00.257+0000 7fdaab76f640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fdaa415ec30 0x7fdaa40b40a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:00.259 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:00.257+0000 7fdaab76f640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fdaa40cfff0 con 0x7fdaa40af9d0 2024-09-12T03:23:00.259 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:00.257+0000 7fdaab76f640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fdaa406c8a0 con 0x7fdaa406da00 2024-09-12T03:23:00.259 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:00.257+0000 7fdaab76f640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fdaa40715f0 con 0x7fdaa415ec30 2024-09-12T03:23:00.259 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:00.257+0000 7fdaa8d0a640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdaa40af9d0 0x7fdaa40b3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:00.259 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:00.257+0000 7fdaa8d0a640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdaa40af9d0 0x7fdaa40b3b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59694/0 (socket says 172.21.15.27:59694) 2024-09-12T03:23:00.259 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:00.257+0000 7fdaa8d0a640 1 -- 172.21.15.27:0/14485740 learned_addr learned my addr 172.21.15.27:0/14485740 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:23:00.259 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:00.257+0000 7fdaa950b640 1 --2- 172.21.15.27:0/14485740 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdaa406da00 0x7fdaa40af160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:00.259 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:00.257+0000 7fdaa8d0a640 1 -- 172.21.15.27:0/14485740 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fdaa415ec30 msgr2=0x7fdaa40b40a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:00.259 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:00.257+0000 7fdaa9d0c640 1 --2- 172.21.15.27:0/14485740 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fdaa415ec30 0x7fdaa40b40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:00.259 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:00.257+0000 7fdaa8d0a640 1 --2- 172.21.15.27:0/14485740 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fdaa415ec30 0x7fdaa40b40a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:00.259 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:00.257+0000 7fdaa8d0a640 1 -- 172.21.15.27:0/14485740 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdaa406da00 msgr2=0x7fdaa40af160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:00.259 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:00.257+0000 7fdaa8d0a640 1 --2- 172.21.15.27:0/14485740 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdaa406da00 0x7fdaa40af160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:00.260 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:00.257+0000 7fdaa8d0a640 1 -- 172.21.15.27:0/14485740 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdaa40af6a0 con 0x7fdaa40af9d0 2024-09-12T03:23:00.260 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:00.258+0000 7fdaa8d0a640 1 --2- 172.21.15.27:0/14485740 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdaa40af9d0 0x7fdaa40b3b60 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7fda94009890 tx=0x7fda940046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:00.260 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:00.258+0000 7fda99ffb640 1 -- 172.21.15.27:0/14485740 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda94041070 con 0x7fdaa40af9d0 2024-09-12T03:23:00.260 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:00.258+0000 7fda99ffb640 1 -- 172.21.15.27:0/14485740 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda9403c890 con 0x7fdaa40af9d0 2024-09-12T03:23:00.260 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:00.258+0000 7fdaab76f640 1 -- 172.21.15.27:0/14485740 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdaa40ad8a0 con 0x7fdaa40af9d0 2024-09-12T03:23:00.260 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:00.258+0000 7fda99ffb640 1 -- 172.21.15.27:0/14485740 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda94037040 con 0x7fdaa40af9d0 2024-09-12T03:23:00.260 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:00.258+0000 7fdaab76f640 1 -- 172.21.15.27:0/14485740 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdaa40ade60 con 0x7fdaa40af9d0 2024-09-12T03:23:00.261 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:00.259+0000 7fdaab76f640 1 -- 172.21.15.27:0/14485740 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdaa406de90 con 0x7fdaa40af9d0 2024-09-12T03:23:00.261 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:00.259+0000 7fda99ffb640 1 -- 172.21.15.27:0/14485740 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 22) ==== 91059+0+0 (secure 0 0 0) 0x7fda94044070 con 0x7fdaa40af9d0 2024-09-12T03:23:00.261 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:00.260+0000 7fda99ffb640 1 --2- 172.21.15.27:0/14485740 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fda8806c490 0x7fda8806e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:00.261 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:00.260+0000 7fda99ffb640 1 -- 172.21.15.27:0/14485740 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 5765+0+0 (secure 0 0 0) 0x7fda940ac540 con 0x7fdaa40af9d0 2024-09-12T03:23:00.261 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:00.260+0000 7fdaa950b640 1 --2- 172.21.15.27:0/14485740 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fda8806c490 0x7fda8806e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:00.261 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:00.260+0000 7fdaa950b640 1 --2- 172.21.15.27:0/14485740 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fda8806c490 0x7fda8806e950 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fdaa0000f60 tx=0x7fdaa0009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:00.264 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:00.264+0000 7fda99ffb640 1 -- 172.21.15.27:0/14485740 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fda9407a4e0 con 0x7fdaa40af9d0 2024-09-12T03:23:00.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:00.466+0000 7fdaab76f640 1 -- 172.21.15.27:0/14485740 --> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol98201", "group_name": "subvol_grp18820", "target": ["mon-mgr", ""]}) -- 0x7fdaa406f240 con 0x7fda8806c490 2024-09-12T03:23:00.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:00.482+0000 7fda99ffb640 1 -- 172.21.15.27:0/14485740 <== mgr.4113 v2:172.21.15.27:6836/1226920526 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fdaa406f240 con 0x7fda8806c490 2024-09-12T03:23:00.483 INFO:teuthology.orchestra.run.smithi027.stdout:/volumes/subvol_grp18820/subvol98201 2024-09-12T03:23:00.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:00.484+0000 7fdaab76f640 1 -- 172.21.15.27:0/14485740 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fda8806c490 msgr2=0x7fda8806e950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:00.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:00.484+0000 7fdaab76f640 1 --2- 172.21.15.27:0/14485740 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fda8806c490 0x7fda8806e950 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fdaa0000f60 tx=0x7fdaa0009040 comp rx=0 tx=0).stop 2024-09-12T03:23:00.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:00.484+0000 7fdaab76f640 1 -- 172.21.15.27:0/14485740 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdaa40af9d0 msgr2=0x7fdaa40b3b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:00.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:00.484+0000 7fdaab76f640 1 --2- 172.21.15.27:0/14485740 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdaa40af9d0 0x7fdaa40b3b60 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7fda94009890 tx=0x7fda940046c0 comp rx=0 tx=0).stop 2024-09-12T03:23:00.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:00.485+0000 7fdaab76f640 1 -- 172.21.15.27:0/14485740 shutdown_connections 2024-09-12T03:23:00.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:00.485+0000 7fdaab76f640 1 --2- 172.21.15.27:0/14485740 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fdaa415ec30 0x7fdaa40b40a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:00.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:00.485+0000 7fdaab76f640 1 --2- 172.21.15.27:0/14485740 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fda8806c490 0x7fda8806e950 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:00.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:00.485+0000 7fdaab76f640 1 --2- 172.21.15.27:0/14485740 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdaa40af9d0 0x7fdaa40b3b60 unknown :-1 s=CLOSED pgs=1439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:00.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:00.485+0000 7fdaab76f640 1 --2- 172.21.15.27:0/14485740 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdaa406da00 0x7fdaa40af160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:00.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:00.485+0000 7fdaab76f640 1 -- 172.21.15.27:0/14485740 >> 172.21.15.27:0/14485740 conn(0x7fdaa406bb30 msgr2=0x7fdaa40719a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:00.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:00.485+0000 7fdaab76f640 1 -- 172.21.15.27:0/14485740 shutdown_connections 2024-09-12T03:23:00.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:00.485+0000 7fdaab76f640 1 -- 172.21.15.27:0/14485740 wait complete. 2024-09-12T03:23:00.495 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol530 2024-09-12T03:23:00.601 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:00.601+0000 7fc86f986640 1 Processor -- start 2024-09-12T03:23:00.601 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:00.601+0000 7fc86f986640 1 -- start start 2024-09-12T03:23:00.604 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:00.602+0000 7fc86f986640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc868070d30 0x7fc86806cc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:00.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:00.602+0000 7fc86f986640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc8680cf9f0 con 0x7fc86806e960 2024-09-12T03:23:00.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:00.602+0000 7fc86f986640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc8680cfc90 con 0x7fc8680d0b50 2024-09-12T03:23:00.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:00.602+0000 7fc86f986640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fc8680d1a40 con 0x7fc868070d30 2024-09-12T03:23:00.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:00.602+0000 7fc86cf21640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fc86806e960 0x7fc8680d0440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:54110/0 (socket says 172.21.15.27:54110) 2024-09-12T03:23:00.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:00.602+0000 7fc86cf21640 1 -- 172.21.15.27:0/3822224806 learned_addr learned my addr 172.21.15.27:0/3822224806 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:23:00.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:00.602+0000 7fc85ffff640 1 -- 172.21.15.27:0/3822224806 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3827705613 0 0) 0x7fc8680cf9f0 con 0x7fc86806e960 2024-09-12T03:23:00.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:00.602+0000 7fc86d722640 1 --2- 172.21.15.27:0/3822224806 >> v2:172.21.15.45:3300/0 conn(0x7fc868070d30 0x7fc86806cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:00.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:00.602+0000 7fc85ffff640 1 -- 172.21.15.27:0/3822224806 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc854003520 con 0x7fc86806e960 2024-09-12T03:23:00.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:00.602+0000 7fc85ffff640 1 -- 172.21.15.27:0/3822224806 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2869267664 0 0) 0x7fc854003520 con 0x7fc86806e960 2024-09-12T03:23:00.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:00.602+0000 7fc85ffff640 1 -- 172.21.15.27:0/3822224806 >> v2:172.21.15.45:3300/0 conn(0x7fc868070d30 msgr2=0x7fc86806cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:00.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:00.602+0000 7fc85ffff640 1 --2- 172.21.15.27:0/3822224806 >> v2:172.21.15.45:3300/0 conn(0x7fc868070d30 0x7fc86806cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:00.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:00.602+0000 7fc85ffff640 1 -- 172.21.15.27:0/3822224806 >> v1:172.21.15.45:6789/0 conn(0x7fc8680d0b50 legacy=0x7fc868168300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:00.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:00.602+0000 7fc85ffff640 1 -- 172.21.15.27:0/3822224806 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8680cff70 con 0x7fc86806e960 2024-09-12T03:23:00.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:00.603+0000 7fc85ffff640 1 -- 172.21.15.27:0/3822224806 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 847755213 0 0) 0x7fc858001690 con 0x7fc86806e960 2024-09-12T03:23:00.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:00.603+0000 7fc85ffff640 1 -- 172.21.15.27:0/3822224806 >> v1:172.21.15.27:6789/0 conn(0x7fc86806e960 legacy=0x7fc8680d0440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:00.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:00.603+0000 7fc85ffff640 1 --2- 172.21.15.27:0/3822224806 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fc854003a00 0x7fc854003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:00.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:00.603+0000 7fc85ffff640 1 --2- 172.21.15.27:0/3822224806 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc854004800 0x7fc854004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:00.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:00.603+0000 7fc85ffff640 1 --2- 172.21.15.27:0/3822224806 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc8540052e0 0x7fc85400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:00.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:00.603+0000 7fc85ffff640 1 -- 172.21.15.27:0/3822224806 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fc8680d1a40 con 0x7fc8540052e0 2024-09-12T03:23:00.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:00.603+0000 7fc85ffff640 1 -- 172.21.15.27:0/3822224806 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc85400c1e0 con 0x7fc854004800 2024-09-12T03:23:00.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:00.603+0000 7fc85ffff640 1 -- 172.21.15.27:0/3822224806 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fc85400c3e0 con 0x7fc854003a00 2024-09-12T03:23:00.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:00.603+0000 7fc86df23640 1 --2- 172.21.15.27:0/3822224806 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc8540052e0 0x7fc85400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:00.607 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:00.603+0000 7fc86cf21640 1 --2- 172.21.15.27:0/3822224806 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc854004800 0x7fc854004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:00.607 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:00.603+0000 7fc86d722640 1 --2- 172.21.15.27:0/3822224806 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fc854003a00 0x7fc854003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:00.607 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:00.603+0000 7fc86df23640 1 -- 172.21.15.27:0/3822224806 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fc854003a00 msgr2=0x7fc854003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:00.607 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:00.603+0000 7fc86df23640 1 --2- 172.21.15.27:0/3822224806 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fc854003a00 0x7fc854003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:00.607 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:00.603+0000 7fc86df23640 1 -- 172.21.15.27:0/3822224806 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc854004800 msgr2=0x7fc854004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:00.607 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:00.603+0000 7fc86df23640 1 --2- 172.21.15.27:0/3822224806 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc854004800 0x7fc854004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:00.607 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:00.603+0000 7fc86df23640 1 -- 172.21.15.27:0/3822224806 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc854003520 con 0x7fc8540052e0 2024-09-12T03:23:00.607 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:00.603+0000 7fc86d722640 1 --2- 172.21.15.27:0/3822224806 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fc854003a00 0x7fc854003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:23:00.607 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:00.604+0000 7fc86df23640 1 -- 172.21.15.27:0/3822224806 reap_dead start 2024-09-12T03:23:00.607 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:00.604+0000 7fc86df23640 1 --2- 172.21.15.27:0/3822224806 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc8540052e0 0x7fc85400b9c0 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7fc860001e50 tx=0x7fc854005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f36bc7a6848900 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:00.607 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:00.604+0000 7fc85ffff640 1 -- 172.21.15.27:0/3822224806 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc860001a70 con 0x7fc8540052e0 2024-09-12T03:23:00.607 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:00.604+0000 7fc85ffff640 1 -- 172.21.15.27:0/3822224806 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc860003970 con 0x7fc8540052e0 2024-09-12T03:23:00.608 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:00.604+0000 7fc85ffff640 1 -- 172.21.15.27:0/3822224806 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc860003d60 con 0x7fc8540052e0 2024-09-12T03:23:00.608 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:00.604+0000 7fc86f986640 1 -- 172.21.15.27:0/3822224806 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc8540052e0 msgr2=0x7fc85400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:00.608 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:00.604+0000 7fc86f986640 1 --2- 172.21.15.27:0/3822224806 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc8540052e0 0x7fc85400b9c0 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7fc860001e50 tx=0x7fc854005830 comp rx=0 tx=0).stop 2024-09-12T03:23:00.608 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:00.604+0000 7fc86f986640 1 -- 172.21.15.27:0/3822224806 shutdown_connections 2024-09-12T03:23:00.608 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:00.604+0000 7fc86f986640 1 --2- 172.21.15.27:0/3822224806 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc8540052e0 0x7fc85400b9c0 unknown :-1 s=CLOSED pgs=1441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:00.608 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:00.604+0000 7fc86f986640 1 -- 172.21.15.27:0/3822224806 >> 172.21.15.27:0/3822224806 conn(0x7fc8680b7c10 msgr2=0x7fc86809ba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:00.608 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:00.605+0000 7fc86f986640 1 -- 172.21.15.27:0/3822224806 shutdown_connections 2024-09-12T03:23:00.608 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:00.605+0000 7fc86f986640 1 -- 172.21.15.27:0/3822224806 wait complete. 2024-09-12T03:23:00.611 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:00.606+0000 7fc86f986640 1 Processor -- start 2024-09-12T03:23:00.611 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:00.606+0000 7fc86f986640 1 -- start start 2024-09-12T03:23:00.611 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:00.606+0000 7fc86f986640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc8540052e0 0x7fc868153860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:00.611 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:00.606+0000 7fc86f986640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fc868161100 0x7fc8680b4930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:00.611 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:00.607+0000 7fc86f986640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc8680cd8a0 0x7fc8680cdd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:00.611 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:00.607+0000 7fc86f986640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fc8680d05c0 con 0x7fc8540052e0 2024-09-12T03:23:00.611 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:00.607+0000 7fc86f986640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc8680d0440 con 0x7fc8680cd8a0 2024-09-12T03:23:00.611 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:00.607+0000 7fc86f986640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fc8680d0740 con 0x7fc868161100 2024-09-12T03:23:00.612 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:00.607+0000 7fc86d722640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc8540052e0 0x7fc868153860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:00.612 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:00.607+0000 7fc86d722640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc8540052e0 0x7fc868153860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59716/0 (socket says 172.21.15.27:59716) 2024-09-12T03:23:00.612 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:00.607+0000 7fc86d722640 1 -- 172.21.15.27:0/3262507191 learned_addr learned my addr 172.21.15.27:0/3262507191 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:23:00.612 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:00.607+0000 7fc86df23640 1 --2- 172.21.15.27:0/3262507191 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc8680cd8a0 0x7fc8680cdd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:00.612 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:00.607+0000 7fc86cf21640 1 --2- 172.21.15.27:0/3262507191 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fc868161100 0x7fc8680b4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:00.612 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:00.607+0000 7fc86d722640 1 -- 172.21.15.27:0/3262507191 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fc868161100 msgr2=0x7fc8680b4930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:00.612 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:00.607+0000 7fc86d722640 1 --2- 172.21.15.27:0/3262507191 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fc868161100 0x7fc8680b4930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:00.612 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:00.607+0000 7fc86d722640 1 -- 172.21.15.27:0/3262507191 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc8680cd8a0 msgr2=0x7fc8680cdd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:00.612 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:00.607+0000 7fc86d722640 1 --2- 172.21.15.27:0/3262507191 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc8680cd8a0 0x7fc8680cdd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:00.612 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:00.607+0000 7fc86d722640 1 -- 172.21.15.27:0/3262507191 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8680b3b50 con 0x7fc8540052e0 2024-09-12T03:23:00.612 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:00.607+0000 7fc86cf21640 1 --2- 172.21.15.27:0/3262507191 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fc868161100 0x7fc8680b4930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:23:00.612 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:00.607+0000 7fc86d722640 1 --2- 172.21.15.27:0/3262507191 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc8540052e0 0x7fc868153860 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7fc864011540 tx=0x7fc864011a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:00.612 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:00.607+0000 7fc85dffb640 1 -- 172.21.15.27:0/3262507191 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc864015870 con 0x7fc8540052e0 2024-09-12T03:23:00.613 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:00.607+0000 7fc85dffb640 1 -- 172.21.15.27:0/3262507191 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc864015a10 con 0x7fc8540052e0 2024-09-12T03:23:00.613 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:00.607+0000 7fc86f986640 1 -- 172.21.15.27:0/3262507191 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc868153da0 con 0x7fc8540052e0 2024-09-12T03:23:00.613 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:00.607+0000 7fc85dffb640 1 -- 172.21.15.27:0/3262507191 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc864015dc0 con 0x7fc8540052e0 2024-09-12T03:23:00.613 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:00.607+0000 7fc86f986640 1 -- 172.21.15.27:0/3262507191 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8680b3df0 con 0x7fc8540052e0 2024-09-12T03:23:00.613 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:00.608+0000 7fc86f986640 1 -- 172.21.15.27:0/3262507191 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc86806cc40 con 0x7fc8540052e0 2024-09-12T03:23:00.613 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:00.609+0000 7fc85dffb640 1 -- 172.21.15.27:0/3262507191 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 22) ==== 91059+0+0 (secure 0 0 0) 0x7fc86400f460 con 0x7fc8540052e0 2024-09-12T03:23:00.613 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:00.609+0000 7fc85dffb640 1 --2- 172.21.15.27:0/3262507191 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fc84c06c460 0x7fc84c06e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:00.613 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:00.610+0000 7fc85dffb640 1 -- 172.21.15.27:0/3262507191 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 5765+0+0 (secure 0 0 0) 0x7fc86408c2f0 con 0x7fc8540052e0 2024-09-12T03:23:00.613 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:00.610+0000 7fc86cf21640 1 --2- 172.21.15.27:0/3262507191 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fc84c06c460 0x7fc84c06e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:00.613 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:00.610+0000 7fc86cf21640 1 --2- 172.21.15.27:0/3262507191 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fc84c06c460 0x7fc84c06e920 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fc868160c20 tx=0x7fc858009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:00.614 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:00.614+0000 7fc85dffb640 1 -- 172.21.15.27:0/3262507191 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fc864019030 con 0x7fc8540052e0 2024-09-12T03:23:00.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:00.814+0000 7fc86f986640 1 -- 172.21.15.27:0/3262507191 --> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol530", "target": ["mon-mgr", ""]}) -- 0x7fc8680d1230 con 0x7fc84c06c460 2024-09-12T03:23:00.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:00.819+0000 7fc85dffb640 1 -- 172.21.15.27:0/3262507191 <== mgr.4113 v2:172.21.15.27:6836/1226920526 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7fc8680d1230 con 0x7fc84c06c460 2024-09-12T03:23:00.818 INFO:teuthology.orchestra.run.smithi027.stdout:/volumes/_nogroup/subvol530 2024-09-12T03:23:00.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:00.820+0000 7fc86f986640 1 -- 172.21.15.27:0/3262507191 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fc84c06c460 msgr2=0x7fc84c06e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:00.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:00.820+0000 7fc86f986640 1 --2- 172.21.15.27:0/3262507191 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fc84c06c460 0x7fc84c06e920 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fc868160c20 tx=0x7fc858009040 comp rx=0 tx=0).stop 2024-09-12T03:23:00.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:00.820+0000 7fc86f986640 1 -- 172.21.15.27:0/3262507191 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc8540052e0 msgr2=0x7fc868153860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:00.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:00.820+0000 7fc86f986640 1 --2- 172.21.15.27:0/3262507191 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc8540052e0 0x7fc868153860 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7fc864011540 tx=0x7fc864011a00 comp rx=0 tx=0).stop 2024-09-12T03:23:00.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:00.821+0000 7fc86f986640 1 -- 172.21.15.27:0/3262507191 shutdown_connections 2024-09-12T03:23:00.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:00.821+0000 7fc86f986640 1 --2- 172.21.15.27:0/3262507191 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc8680cd8a0 0x7fc8680cdd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:00.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:00.821+0000 7fc86f986640 1 --2- 172.21.15.27:0/3262507191 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fc868161100 0x7fc8680b4930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:00.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:00.821+0000 7fc86f986640 1 --2- 172.21.15.27:0/3262507191 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fc84c06c460 0x7fc84c06e920 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:00.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:00.821+0000 7fc86f986640 1 --2- 172.21.15.27:0/3262507191 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc8540052e0 0x7fc868153860 unknown :-1 s=CLOSED pgs=1442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:00.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:00.821+0000 7fc86f986640 1 -- 172.21.15.27:0/3262507191 >> 172.21.15.27:0/3262507191 conn(0x7fc8680b7c10 msgr2=0x7fc8680af4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:00.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:00.821+0000 7fc86f986640 1 -- 172.21.15.27:0/3262507191 shutdown_connections 2024-09-12T03:23:00.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:00.821+0000 7fc86f986640 1 -- 172.21.15.27:0/3262507191 wait complete. 2024-09-12T03:23:00.831 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:23:00.831 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cat ./volumes/_legacy/00725ea440c873bac1e197b887cb5f9b.meta) 2024-09-12T03:23:01.281 DEBUG:teuthology.orchestra.run.smithi027:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-12T03:23:01.284 DEBUG:teuthology.orchestra.run.smithi045:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-12T03:23:05.516 INFO:teuthology.orchestra.run.smithi027.stdout:[GLOBAL] 2024-09-12T03:23:05.517 INFO:teuthology.orchestra.run.smithi027.stdout:version = 1 2024-09-12T03:23:05.517 INFO:teuthology.orchestra.run.smithi027.stdout:type = subvolume 2024-09-12T03:23:05.517 INFO:teuthology.orchestra.run.smithi027.stdout:path = /volumes/_nogroup/subvol530 2024-09-12T03:23:05.517 INFO:teuthology.orchestra.run.smithi027.stdout:state = complete 2024-09-12T03:23:05.517 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:23:05.519 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol98201 subvol_grp18820 2024-09-12T03:23:05.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:05.637+0000 7f4dcd847640 1 Processor -- start 2024-09-12T03:23:05.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:05.637+0000 7f4dcd847640 1 -- start start 2024-09-12T03:23:05.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:05.638+0000 7f4dcd847640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4dc80b6230 0x7f4dc80b6690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:05.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:05.638+0000 7f4dcd847640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4dc815ed70 con 0x7f4dc80b3600 2024-09-12T03:23:05.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:05.638+0000 7f4dcd847640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4dc80b7d10 con 0x7f4dc815e990 2024-09-12T03:23:05.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:05.638+0000 7f4dcd847640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f4dc80b6bd0 con 0x7f4dc80b6230 2024-09-12T03:23:05.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:05.638+0000 7f4dc77fe640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f4dc80b3600 0x7f4dc80b3a70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:54112/0 (socket says 172.21.15.27:54112) 2024-09-12T03:23:05.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:05.638+0000 7f4dc77fe640 1 -- 172.21.15.27:0/4057911852 learned_addr learned my addr 172.21.15.27:0/4057911852 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:23:05.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:05.638+0000 7f4dc67fc640 1 --2- 172.21.15.27:0/4057911852 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4dc80b6230 0x7f4dc80b6690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:05.641 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:05.638+0000 7f4dc5ffb640 1 -- 172.21.15.27:0/4057911852 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2354671327 0 0) 0x7f4dc815ed70 con 0x7f4dc80b3600 2024-09-12T03:23:05.641 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:05.638+0000 7f4dc5ffb640 1 -- 172.21.15.27:0/4057911852 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4db4003520 con 0x7f4dc80b3600 2024-09-12T03:23:05.641 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:05.639+0000 7f4dc5ffb640 1 -- 172.21.15.27:0/4057911852 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 235132288 0 0) 0x7f4db4003520 con 0x7f4dc80b3600 2024-09-12T03:23:05.641 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:05.639+0000 7f4dc5ffb640 1 -- 172.21.15.27:0/4057911852 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4dc80b6230 msgr2=0x7f4dc80b6690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:05.641 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:05.639+0000 7f4dc5ffb640 1 --2- 172.21.15.27:0/4057911852 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4dc80b6230 0x7f4dc80b6690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:05.641 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:05.639+0000 7f4dc5ffb640 1 -- 172.21.15.27:0/4057911852 >> v1:172.21.15.45:6789/0 conn(0x7f4dc815e990 legacy=0x7f4dc80b7600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:05.641 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:05.639+0000 7f4dc5ffb640 1 -- 172.21.15.27:0/4057911852 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4dc80d2a30 con 0x7f4dc80b3600 2024-09-12T03:23:05.641 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:05.639+0000 7f4dc5ffb640 1 -- 172.21.15.27:0/4057911852 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 847755213 0 0) 0x7f4dc0001670 con 0x7f4dc80b3600 2024-09-12T03:23:05.641 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:05.639+0000 7f4dc5ffb640 1 -- 172.21.15.27:0/4057911852 >> v1:172.21.15.27:6789/0 conn(0x7f4dc80b3600 legacy=0x7f4dc80b3a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:05.641 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:05.639+0000 7f4dc5ffb640 1 --2- 172.21.15.27:0/4057911852 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4db40039e0 0x7f4db4003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:05.641 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:05.639+0000 7f4dc5ffb640 1 --2- 172.21.15.27:0/4057911852 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4db4004390 0x7f4db4004860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:05.641 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:05.639+0000 7f4dc5ffb640 1 --2- 172.21.15.27:0/4057911852 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4dc80b6230 0x7f4db40050c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:05.641 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:05.639+0000 7f4dc5ffb640 1 -- 172.21.15.27:0/4057911852 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f4db400b9c0 con 0x7f4db40039e0 2024-09-12T03:23:05.642 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:05.639+0000 7f4dc5ffb640 1 -- 172.21.15.27:0/4057911852 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f4db400bb70 con 0x7f4db4004390 2024-09-12T03:23:05.642 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:05.639+0000 7f4dc5ffb640 1 -- 172.21.15.27:0/4057911852 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f4db400bd70 con 0x7f4dc80b6230 2024-09-12T03:23:05.642 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:05.639+0000 7f4dc6ffd640 1 --2- 172.21.15.27:0/4057911852 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4db40039e0 0x7f4db4003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:05.642 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:05.639+0000 7f4dc67fc640 1 --2- 172.21.15.27:0/4057911852 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4db4004390 0x7f4db4004860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:05.642 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:05.639+0000 7f4dc77fe640 1 --2- 172.21.15.27:0/4057911852 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4dc80b6230 0x7f4db40050c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:05.642 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:05.639+0000 7f4dc6ffd640 1 -- 172.21.15.27:0/4057911852 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4dc80b6230 msgr2=0x7f4db40050c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:05.642 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:05.639+0000 7f4dc6ffd640 1 --2- 172.21.15.27:0/4057911852 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4dc80b6230 0x7f4db40050c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:05.642 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:05.639+0000 7f4dc6ffd640 1 -- 172.21.15.27:0/4057911852 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4db4004390 msgr2=0x7f4db4004860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:05.642 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:05.639+0000 7f4dc6ffd640 1 --2- 172.21.15.27:0/4057911852 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4db4004390 0x7f4db4004860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:05.642 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:05.639+0000 7f4dc6ffd640 1 -- 172.21.15.27:0/4057911852 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4db4003520 con 0x7f4db40039e0 2024-09-12T03:23:05.642 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:05.640+0000 7f4dc6ffd640 1 --2- 172.21.15.27:0/4057911852 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4db40039e0 0x7f4db4003dc0 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7f4db8001f60 tx=0x7f4db4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14e63e9e5c998c68 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:05.642 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:05.640+0000 7f4dc5ffb640 1 -- 172.21.15.27:0/4057911852 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4db8001b30 con 0x7f4db40039e0 2024-09-12T03:23:05.642 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:05.640+0000 7f4dc5ffb640 1 -- 172.21.15.27:0/4057911852 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4db8031620 con 0x7f4db40039e0 2024-09-12T03:23:05.642 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:05.640+0000 7f4dc5ffb640 1 -- 172.21.15.27:0/4057911852 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4db8031a10 con 0x7f4db40039e0 2024-09-12T03:23:05.645 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:05.641+0000 7f4dcd847640 1 -- 172.21.15.27:0/4057911852 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4db40039e0 msgr2=0x7f4db4003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:05.645 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:05.641+0000 7f4dcd847640 1 --2- 172.21.15.27:0/4057911852 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4db40039e0 0x7f4db4003dc0 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7f4db8001f60 tx=0x7f4db4005830 comp rx=0 tx=0).stop 2024-09-12T03:23:05.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:05.641+0000 7f4dc77fe640 1 -- 172.21.15.27:0/4057911852 reap_dead start 2024-09-12T03:23:05.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:05.641+0000 7f4dcd847640 1 -- 172.21.15.27:0/4057911852 shutdown_connections 2024-09-12T03:23:05.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:05.641+0000 7f4dcd847640 1 -- 172.21.15.27:0/4057911852 >> 172.21.15.27:0/4057911852 conn(0x7f4dc80d1230 msgr2=0x7f4dc80d1640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:05.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:05.641+0000 7f4dcd847640 1 -- 172.21.15.27:0/4057911852 shutdown_connections 2024-09-12T03:23:05.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:05.641+0000 7f4dcd847640 1 -- 172.21.15.27:0/4057911852 wait complete. 2024-09-12T03:23:05.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:05.642+0000 7f4dcd847640 1 Processor -- start 2024-09-12T03:23:05.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:05.642+0000 7f4dcd847640 1 -- start start 2024-09-12T03:23:05.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:05.643+0000 7f4dcd847640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4db40039e0 0x7f4dc80ce050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:05.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:05.643+0000 7f4dcd847640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4dc80b3090 0x7f4dc81f3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:05.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:05.643+0000 7f4dcd847640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4dc806d3e0 0x7f4dc806d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:05.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:05.643+0000 7f4dcd847640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f4dc80d32d0 con 0x7f4dc80b3090 2024-09-12T03:23:05.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:05.643+0000 7f4dcd847640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f4dc80d2ff0 con 0x7f4dc806d3e0 2024-09-12T03:23:05.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:05.643+0000 7f4dcd847640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f4dc80d35b0 con 0x7f4db40039e0 2024-09-12T03:23:05.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:05.643+0000 7f4dc67fc640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4dc80b3090 0x7f4dc81f3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:05.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:05.643+0000 7f4dc67fc640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4dc80b3090 0x7f4dc81f3680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59736/0 (socket says 172.21.15.27:59736) 2024-09-12T03:23:05.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:05.643+0000 7f4dc67fc640 1 -- 172.21.15.27:0/2319855146 learned_addr learned my addr 172.21.15.27:0/2319855146 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:23:05.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:05.643+0000 7f4dc6ffd640 1 --2- 172.21.15.27:0/2319855146 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4db40039e0 0x7f4dc80ce050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:05.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:05.643+0000 7f4dc77fe640 1 --2- 172.21.15.27:0/2319855146 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4dc806d3e0 0x7f4dc806d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:05.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:05.644+0000 7f4dc67fc640 1 -- 172.21.15.27:0/2319855146 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4db40039e0 msgr2=0x7f4dc80ce050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:05.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:05.644+0000 7f4dc67fc640 1 --2- 172.21.15.27:0/2319855146 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4db40039e0 0x7f4dc80ce050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:05.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:05.644+0000 7f4dc67fc640 1 -- 172.21.15.27:0/2319855146 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4dc806d3e0 msgr2=0x7f4dc806d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:05.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:05.644+0000 7f4dc67fc640 1 --2- 172.21.15.27:0/2319855146 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4dc806d3e0 0x7f4dc806d870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:05.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:05.644+0000 7f4dc67fc640 1 -- 172.21.15.27:0/2319855146 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4dc81f73d0 con 0x7f4dc80b3090 2024-09-12T03:23:05.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:05.644+0000 7f4dc67fc640 1 --2- 172.21.15.27:0/2319855146 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4dc80b3090 0x7f4dc81f3680 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7f4db00119d0 tx=0x7f4db0011e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:05.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:05.644+0000 7f4dbffff640 1 -- 172.21.15.27:0/2319855146 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4db000fb50 con 0x7f4dc80b3090 2024-09-12T03:23:05.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:05.644+0000 7f4dbffff640 1 -- 172.21.15.27:0/2319855146 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4db000fcf0 con 0x7f4dc80b3090 2024-09-12T03:23:05.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:05.644+0000 7f4dcd847640 1 -- 172.21.15.27:0/2319855146 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4dc81f7670 con 0x7f4dc80b3090 2024-09-12T03:23:05.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:05.644+0000 7f4dbffff640 1 -- 172.21.15.27:0/2319855146 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4db000fea0 con 0x7f4dc80b3090 2024-09-12T03:23:05.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:05.644+0000 7f4dcd847640 1 -- 172.21.15.27:0/2319855146 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4dc81f7c30 con 0x7f4dc80b3090 2024-09-12T03:23:05.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:05.645+0000 7f4dcd847640 1 -- 172.21.15.27:0/2319855146 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4dc80b6690 con 0x7f4dc80b3090 2024-09-12T03:23:05.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:05.646+0000 7f4dbffff640 1 -- 172.21.15.27:0/2319855146 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 23) ==== 91110+0+0 (secure 0 0 0) 0x7f4db0012040 con 0x7f4dc80b3090 2024-09-12T03:23:05.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:05.646+0000 7f4dbffff640 1 --2- 172.21.15.27:0/2319855146 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f4da806c4f0 0x7f4da806e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:05.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:05.646+0000 7f4dbffff640 1 -- 172.21.15.27:0/2319855146 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 5765+0+0 (secure 0 0 0) 0x7f4db008bf70 con 0x7f4dc80b3090 2024-09-12T03:23:05.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:05.647+0000 7f4dc6ffd640 1 --2- 172.21.15.27:0/2319855146 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f4da806c4f0 0x7f4da806e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:05.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:05.647+0000 7f4dc6ffd640 1 --2- 172.21.15.27:0/2319855146 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f4da806c4f0 0x7f4da806e9b0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f4db8034000 tx=0x7f4db802f000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:05.650 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:05.651+0000 7f4dbffff640 1 -- 172.21.15.27:0/2319855146 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f4db0019030 con 0x7f4dc80b3090 2024-09-12T03:23:05.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:05.851+0000 7f4dcd847640 1 -- 172.21.15.27:0/2319855146 --> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol98201", "group_name": "subvol_grp18820", "target": ["mon-mgr", ""]}) -- 0x7f4dc806ddb0 con 0x7f4da806c4f0 2024-09-12T03:23:05.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:05.856+0000 7f4dbffff640 1 -- 172.21.15.27:0/2319855146 <== mgr.4113 v2:172.21.15.27:6836/1226920526 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f4dc806ddb0 con 0x7f4da806c4f0 2024-09-12T03:23:05.856 INFO:teuthology.orchestra.run.smithi027.stdout:/volumes/subvol_grp18820/subvol98201 2024-09-12T03:23:05.860 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:05.858+0000 7f4dcd847640 1 -- 172.21.15.27:0/2319855146 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f4da806c4f0 msgr2=0x7f4da806e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:05.860 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:05.858+0000 7f4dcd847640 1 --2- 172.21.15.27:0/2319855146 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f4da806c4f0 0x7f4da806e9b0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f4db8034000 tx=0x7f4db802f000 comp rx=0 tx=0).stop 2024-09-12T03:23:05.860 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:05.858+0000 7f4dcd847640 1 -- 172.21.15.27:0/2319855146 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4dc80b3090 msgr2=0x7f4dc81f3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:05.860 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:05.858+0000 7f4dcd847640 1 --2- 172.21.15.27:0/2319855146 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4dc80b3090 0x7f4dc81f3680 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7f4db00119d0 tx=0x7f4db0011e90 comp rx=0 tx=0).stop 2024-09-12T03:23:05.860 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:05.858+0000 7f4dcd847640 1 -- 172.21.15.27:0/2319855146 shutdown_connections 2024-09-12T03:23:05.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:05.858+0000 7f4dcd847640 1 --2- 172.21.15.27:0/2319855146 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4dc806d3e0 0x7f4dc806d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:05.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:05.858+0000 7f4dcd847640 1 --2- 172.21.15.27:0/2319855146 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f4da806c4f0 0x7f4da806e9b0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:05.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:05.858+0000 7f4dcd847640 1 --2- 172.21.15.27:0/2319855146 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4dc80b3090 0x7f4dc81f3680 unknown :-1 s=CLOSED pgs=1445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:05.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:05.858+0000 7f4dcd847640 1 --2- 172.21.15.27:0/2319855146 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4db40039e0 0x7f4dc80ce050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:05.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:05.859+0000 7f4dcd847640 1 -- 172.21.15.27:0/2319855146 >> 172.21.15.27:0/2319855146 conn(0x7f4dc80d1230 msgr2=0x7f4dc80b9830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:05.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:05.859+0000 7f4dcd847640 1 -- 172.21.15.27:0/2319855146 shutdown_connections 2024-09-12T03:23:05.862 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:05.859+0000 7f4dcd847640 1 -- 172.21.15.27:0/2319855146 wait complete. 2024-09-12T03:23:05.870 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:23:05.870 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cat ./volumes/_legacy/00e874e973209d21b1eb678107e3a5f4.meta) 2024-09-12T03:23:05.899 INFO:teuthology.orchestra.run.smithi027.stdout:[GLOBAL] 2024-09-12T03:23:05.900 INFO:teuthology.orchestra.run.smithi027.stdout:version = 1 2024-09-12T03:23:05.900 INFO:teuthology.orchestra.run.smithi027.stdout:type = subvolume 2024-09-12T03:23:05.900 INFO:teuthology.orchestra.run.smithi027.stdout:path = /volumes/subvol_grp18820/subvol98201 2024-09-12T03:23:05.900 INFO:teuthology.orchestra.run.smithi027.stdout:state = complete 2024-09-12T03:23:05.900 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:23:05.901 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol530 2024-09-12T03:23:06.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.047+0000 7efef84d0640 1 Processor -- start 2024-09-12T03:23:06.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.047+0000 7efef84d0640 1 -- start start 2024-09-12T03:23:06.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.048+0000 7efef84d0640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7efef00d0b50 0x7efef0169310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:06.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.048+0000 7efef84d0640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efef00cf9f0 con 0x7efef00d31c0 2024-09-12T03:23:06.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.048+0000 7efef84d0640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efef00cfc90 con 0x7efef0070d30 2024-09-12T03:23:06.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.048+0000 7efef84d0640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7efef00d1a40 con 0x7efef00d0b50 2024-09-12T03:23:06.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.048+0000 7efef5a6b640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7efef00d31c0 0x7efef00d0440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:54126/0 (socket says 172.21.15.27:54126) 2024-09-12T03:23:06.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.048+0000 7efef5a6b640 1 -- 172.21.15.27:0/1857331099 learned_addr learned my addr 172.21.15.27:0/1857331099 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:23:06.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.048+0000 7efef6a6d640 1 --2- 172.21.15.27:0/1857331099 >> v2:172.21.15.27:3300/0 conn(0x7efef00d0b50 0x7efef0169310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:06.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.049+0000 7efef526a640 1 -- 172.21.15.27:0/1857331099 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3517713169 0 0) 0x7efef00cf9f0 con 0x7efef00d31c0 2024-09-12T03:23:06.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.049+0000 7efef526a640 1 -- 172.21.15.27:0/1857331099 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efedc003520 con 0x7efef00d31c0 2024-09-12T03:23:06.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.049+0000 7efef6a6d640 1 -- 172.21.15.27:0/1857331099 >> v1:172.21.15.45:6789/0 conn(0x7efef0070d30 legacy=0x7efef006cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:06.050 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.049+0000 7efef6a6d640 1 -- 172.21.15.27:0/1857331099 >> v1:172.21.15.27:6789/0 conn(0x7efef00d31c0 legacy=0x7efef00d0440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:06.050 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.049+0000 7efef6a6d640 1 -- 172.21.15.27:0/1857331099 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efef00cff70 con 0x7efef00d0b50 2024-09-12T03:23:06.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.049+0000 7efef6a6d640 1 --2- 172.21.15.27:0/1857331099 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efef00d0b50 0x7efef0169310 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7efee8009d90 tx=0x7efee802a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6efb50c1aefa1684 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:06.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.050+0000 7efef526a640 1 -- 172.21.15.27:0/1857331099 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efee8031600 con 0x7efef00d0b50 2024-09-12T03:23:06.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.050+0000 7efef526a640 1 -- 172.21.15.27:0/1857331099 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efee80317a0 con 0x7efef00d0b50 2024-09-12T03:23:06.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.050+0000 7efef526a640 1 -- 172.21.15.27:0/1857331099 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efee8031b70 con 0x7efef00d0b50 2024-09-12T03:23:06.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.050+0000 7efef84d0640 1 -- 172.21.15.27:0/1857331099 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efef00d0b50 msgr2=0x7efef0169310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:06.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.050+0000 7efef84d0640 1 --2- 172.21.15.27:0/1857331099 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efef00d0b50 0x7efef0169310 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7efee8009d90 tx=0x7efee802a5d0 comp rx=0 tx=0).stop 2024-09-12T03:23:06.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.050+0000 7efef84d0640 1 -- 172.21.15.27:0/1857331099 shutdown_connections 2024-09-12T03:23:06.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.050+0000 7efef84d0640 1 --2- 172.21.15.27:0/1857331099 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efef00d0b50 0x7efef0169310 unknown :-1 s=CLOSED pgs=1447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:06.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.050+0000 7efef84d0640 1 -- 172.21.15.27:0/1857331099 >> 172.21.15.27:0/1857331099 conn(0x7efef00b7c10 msgr2=0x7efef009ba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:06.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.050+0000 7efef84d0640 1 -- 172.21.15.27:0/1857331099 shutdown_connections 2024-09-12T03:23:06.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.050+0000 7efef84d0640 1 -- 172.21.15.27:0/1857331099 wait complete. 2024-09-12T03:23:06.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.051+0000 7efef84d0640 1 Processor -- start 2024-09-12T03:23:06.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.052+0000 7efef84d0640 1 -- start start 2024-09-12T03:23:06.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.052+0000 7efef84d0640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7efef0070d30 0x7efef015a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:06.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.052+0000 7efef84d0640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efef00d0b50 0x7efef015ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:06.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.052+0000 7efef84d0640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efef00d31c0 0x7efef0150830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:06.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.052+0000 7efef84d0640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7efef006eae0 con 0x7efef00d31c0 2024-09-12T03:23:06.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.052+0000 7efef84d0640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7efef006e960 con 0x7efef00d0b50 2024-09-12T03:23:06.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.052+0000 7efef84d0640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7efef0169850 con 0x7efef0070d30 2024-09-12T03:23:06.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.052+0000 7efef6a6d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efef00d31c0 0x7efef0150830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:06.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.052+0000 7efef6a6d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efef00d31c0 0x7efef0150830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59758/0 (socket says 172.21.15.27:59758) 2024-09-12T03:23:06.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.052+0000 7efef6a6d640 1 -- 172.21.15.27:0/2211941437 learned_addr learned my addr 172.21.15.27:0/2211941437 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:23:06.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.052+0000 7efef5a6b640 1 --2- 172.21.15.27:0/2211941437 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efef00d0b50 0x7efef015ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:06.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.053+0000 7efef626c640 1 --2- 172.21.15.27:0/2211941437 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7efef0070d30 0x7efef015a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:06.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.053+0000 7efef6a6d640 1 -- 172.21.15.27:0/2211941437 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7efef0070d30 msgr2=0x7efef015a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:06.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.053+0000 7efef6a6d640 1 --2- 172.21.15.27:0/2211941437 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7efef0070d30 0x7efef015a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:06.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.053+0000 7efef6a6d640 1 -- 172.21.15.27:0/2211941437 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efef00d0b50 msgr2=0x7efef015ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:06.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.053+0000 7efef6a6d640 1 --2- 172.21.15.27:0/2211941437 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efef00d0b50 0x7efef015ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:06.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.053+0000 7efef6a6d640 1 -- 172.21.15.27:0/2211941437 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efef0158590 con 0x7efef00d31c0 2024-09-12T03:23:06.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.053+0000 7efef6a6d640 1 --2- 172.21.15.27:0/2211941437 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efef00d31c0 0x7efef0150830 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7efee802abc0 tx=0x7efee8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:06.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.053+0000 7efee6ffd640 1 -- 172.21.15.27:0/2211941437 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efee8040070 con 0x7efef00d31c0 2024-09-12T03:23:06.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.053+0000 7efee6ffd640 1 -- 172.21.15.27:0/2211941437 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efee8030b60 con 0x7efef00d31c0 2024-09-12T03:23:06.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.053+0000 7efef84d0640 1 -- 172.21.15.27:0/2211941437 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efef0160fd0 con 0x7efef00d31c0 2024-09-12T03:23:06.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.053+0000 7efee6ffd640 1 -- 172.21.15.27:0/2211941437 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efee8009450 con 0x7efef00d31c0 2024-09-12T03:23:06.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.053+0000 7efef84d0640 1 -- 172.21.15.27:0/2211941437 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efef0161590 con 0x7efef00d31c0 2024-09-12T03:23:06.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.054+0000 7efef84d0640 1 -- 172.21.15.27:0/2211941437 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efef006cc40 con 0x7efef00d31c0 2024-09-12T03:23:06.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.055+0000 7efee6ffd640 1 -- 172.21.15.27:0/2211941437 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 23) ==== 91110+0+0 (secure 0 0 0) 0x7efee8039040 con 0x7efef00d31c0 2024-09-12T03:23:06.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.055+0000 7efee6ffd640 1 --2- 172.21.15.27:0/2211941437 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7efed406c520 0x7efed406e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:06.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.055+0000 7efef626c640 1 --2- 172.21.15.27:0/2211941437 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7efed406c520 0x7efed406e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:06.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.056+0000 7efee6ffd640 1 -- 172.21.15.27:0/2211941437 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 5765+0+0 (secure 0 0 0) 0x7efee80ad040 con 0x7efef00d31c0 2024-09-12T03:23:06.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.056+0000 7efef626c640 1 --2- 172.21.15.27:0/2211941437 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7efed406c520 0x7efed406e9e0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7efeec001780 tx=0x7efeec009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:06.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.060+0000 7efee6ffd640 1 -- 172.21.15.27:0/2211941437 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7efee807a8c0 con 0x7efef00d31c0 2024-09-12T03:23:06.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.263+0000 7efef84d0640 1 -- 172.21.15.27:0/2211941437 --> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol530", "target": ["mon-mgr", ""]}) -- 0x7efef0161760 con 0x7efed406c520 2024-09-12T03:23:06.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.279+0000 7efee6ffd640 1 -- 172.21.15.27:0/2211941437 <== mgr.4113 v2:172.21.15.27:6836/1226920526 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7efef0161760 con 0x7efed406c520 2024-09-12T03:23:06.282 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.282+0000 7efef84d0640 1 -- 172.21.15.27:0/2211941437 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7efed406c520 msgr2=0x7efed406e9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:06.283 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.282+0000 7efef84d0640 1 --2- 172.21.15.27:0/2211941437 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7efed406c520 0x7efed406e9e0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7efeec001780 tx=0x7efeec009040 comp rx=0 tx=0).stop 2024-09-12T03:23:06.283 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.282+0000 7efef84d0640 1 -- 172.21.15.27:0/2211941437 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efef00d31c0 msgr2=0x7efef0150830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:06.283 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.282+0000 7efef84d0640 1 --2- 172.21.15.27:0/2211941437 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efef00d31c0 0x7efef0150830 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7efee802abc0 tx=0x7efee8004a40 comp rx=0 tx=0).stop 2024-09-12T03:23:06.283 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.282+0000 7efef84d0640 1 -- 172.21.15.27:0/2211941437 shutdown_connections 2024-09-12T03:23:06.283 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.282+0000 7efef84d0640 1 --2- 172.21.15.27:0/2211941437 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7efed406c520 0x7efed406e9e0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:06.283 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.282+0000 7efef84d0640 1 --2- 172.21.15.27:0/2211941437 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efef00d31c0 0x7efef0150830 unknown :-1 s=CLOSED pgs=1448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:06.283 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.282+0000 7efef84d0640 1 --2- 172.21.15.27:0/2211941437 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efef00d0b50 0x7efef015ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:06.283 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.282+0000 7efef84d0640 1 --2- 172.21.15.27:0/2211941437 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7efef0070d30 0x7efef015a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:06.283 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.282+0000 7efef84d0640 1 -- 172.21.15.27:0/2211941437 >> 172.21.15.27:0/2211941437 conn(0x7efef00b7c10 msgr2=0x7efef00d0440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:06.283 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.282+0000 7efef84d0640 1 -- 172.21.15.27:0/2211941437 shutdown_connections 2024-09-12T03:23:06.283 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.282+0000 7efef84d0640 1 -- 172.21.15.27:0/2211941437 wait complete. 2024-09-12T03:23:06.285 INFO:tasks.ceph.mgr.y.smithi027.stderr:2024-09-12T03:23:06.285+0000 7f32608e9640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:23:06.285 INFO:tasks.ceph.mgr.y.smithi027.stderr:2024-09-12T03:23:06.285+0000 7f32608e9640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:23:06.285 INFO:tasks.ceph.mgr.y.smithi027.stderr:2024-09-12T03:23:06.285+0000 7f32608e9640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:23:06.285 INFO:tasks.ceph.mgr.y.smithi027.stderr:2024-09-12T03:23:06.285+0000 7f32608e9640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:23:06.285 INFO:tasks.ceph.mgr.y.smithi027.stderr:2024-09-12T03:23:06.285+0000 7f32608e9640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:23:06.295 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol98201 subvol_grp18820 2024-09-12T03:23:06.299 INFO:tasks.ceph.mgr.y.smithi027.stderr:2024-09-12T03:23:06.299+0000 7f325f8e7640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:23:06.299 INFO:tasks.ceph.mgr.y.smithi027.stderr:2024-09-12T03:23:06.299+0000 7f325f8e7640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:23:06.299 INFO:tasks.ceph.mgr.y.smithi027.stderr:2024-09-12T03:23:06.299+0000 7f325f8e7640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:23:06.299 INFO:tasks.ceph.mgr.y.smithi027.stderr:2024-09-12T03:23:06.299+0000 7f325f8e7640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:23:06.299 INFO:tasks.ceph.mgr.y.smithi027.stderr:2024-09-12T03:23:06.299+0000 7f325f8e7640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:23:06.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.403+0000 7f8984713640 1 Processor -- start 2024-09-12T03:23:06.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.403+0000 7f8984713640 1 -- start start 2024-09-12T03:23:06.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.404+0000 7f8984713640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f897c06dfd0 0x7f897c0d1da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:06.405 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.404+0000 7f8984713640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f897c0d3790 0x7f897c0b7a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:06.405 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.404+0000 7f8984713640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f897c0d2e30 con 0x7f897c162040 2024-09-12T03:23:06.405 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.404+0000 7f8984713640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f897c0b7f90 con 0x7f897c0d3790 2024-09-12T03:23:06.405 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.404+0000 7f8984713640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f897c162420 con 0x7f897c06dfd0 2024-09-12T03:23:06.405 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.404+0000 7f8982cb0640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f897c0d3790 0x7f897c0b7a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:06.405 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.404+0000 7f8982cb0640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f897c0d3790 0x7f897c0b7a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59802/0 (socket says 172.21.15.27:59802) 2024-09-12T03:23:06.405 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.404+0000 7f89824af640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f897c162040 0x7f897c06d8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:54128/0 (socket says 172.21.15.27:54128) 2024-09-12T03:23:06.405 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.404+0000 7f8982cb0640 1 -- 172.21.15.27:0/3044865205 learned_addr learned my addr 172.21.15.27:0/3044865205 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:23:06.405 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.404+0000 7f8981cae640 1 --2- 172.21.15.27:0/3044865205 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f897c06dfd0 0x7f897c0d1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:06.405 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.404+0000 7f8982cb0640 1 -- 172.21.15.27:0/3044865205 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f897c06dfd0 msgr2=0x7f897c0d1da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:06.405 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.404+0000 7f89814ad640 1 -- 172.21.15.27:0/3044865205 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2356644045 0 0) 0x7f897c0d2e30 con 0x7f897c162040 2024-09-12T03:23:06.406 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.404+0000 7f8982cb0640 1 --2- 172.21.15.27:0/3044865205 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f897c06dfd0 0x7f897c0d1da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:06.406 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.405+0000 7f8982cb0640 1 -- 172.21.15.27:0/3044865205 >> v1:172.21.15.27:6789/0 conn(0x7f897c162040 legacy=0x7f897c06d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:06.406 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.405+0000 7f8982cb0640 1 -- 172.21.15.27:0/3044865205 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f897c0d30d0 con 0x7f897c0d3790 2024-09-12T03:23:06.407 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.405+0000 7f8982cb0640 1 --2- 172.21.15.27:0/3044865205 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f897c0d3790 0x7f897c0b7a50 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7f8974009e50 tx=0x7f897402a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d95b73b0a0b7e3c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:06.407 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.405+0000 7f89814ad640 1 -- 172.21.15.27:0/3044865205 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8974036070 con 0x7f897c0d3790 2024-09-12T03:23:06.407 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.405+0000 7f89814ad640 1 -- 172.21.15.27:0/3044865205 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8974031600 con 0x7f897c0d3790 2024-09-12T03:23:06.407 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.405+0000 7f89814ad640 1 -- 172.21.15.27:0/3044865205 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89740319d0 con 0x7f897c0d3790 2024-09-12T03:23:06.408 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.406+0000 7f8984713640 1 -- 172.21.15.27:0/3044865205 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f897c0d3790 msgr2=0x7f897c0b7a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:06.408 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.406+0000 7f8984713640 1 --2- 172.21.15.27:0/3044865205 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f897c0d3790 0x7f897c0b7a50 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7f8974009e50 tx=0x7f897402a710 comp rx=0 tx=0).stop 2024-09-12T03:23:06.408 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.406+0000 7f8984713640 1 -- 172.21.15.27:0/3044865205 shutdown_connections 2024-09-12T03:23:06.408 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.406+0000 7f8984713640 1 --2- 172.21.15.27:0/3044865205 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f897c0d3790 0x7f897c0b7a50 unknown :-1 s=CLOSED pgs=1454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:06.408 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.406+0000 7f8984713640 1 --2- 172.21.15.27:0/3044865205 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f897c06dfd0 0x7f897c0d1da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:06.408 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.406+0000 7f8984713640 1 -- 172.21.15.27:0/3044865205 >> 172.21.15.27:0/3044865205 conn(0x7f897c0b6700 msgr2=0x7f897c0b6b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:06.408 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.406+0000 7f8984713640 1 -- 172.21.15.27:0/3044865205 shutdown_connections 2024-09-12T03:23:06.408 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.406+0000 7f8984713640 1 -- 172.21.15.27:0/3044865205 wait complete. 2024-09-12T03:23:06.408 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.407+0000 7f8984713640 1 Processor -- start 2024-09-12T03:23:06.408 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.407+0000 7f8984713640 1 -- start start 2024-09-12T03:23:06.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.408+0000 7f8984713640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f897c06dfd0 0x7f897c155dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:06.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.408+0000 7f8984713640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f897c0d3790 0x7f897c0b9ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:06.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.408+0000 7f8984713640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f897c162040 0x7f897c15a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:06.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.408+0000 7f8984713640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f897c0d4530 con 0x7f897c0d3790 2024-09-12T03:23:06.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.408+0000 7f8984713640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f897c0cee00 con 0x7f897c06dfd0 2024-09-12T03:23:06.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.408+0000 7f8984713640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f897c0d4150 con 0x7f897c162040 2024-09-12T03:23:06.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.408+0000 7f8981cae640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f897c0d3790 0x7f897c0b9ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:06.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.408+0000 7f8981cae640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f897c0d3790 0x7f897c0b9ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59816/0 (socket says 172.21.15.27:59816) 2024-09-12T03:23:06.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.408+0000 7f8981cae640 1 -- 172.21.15.27:0/2142447655 learned_addr learned my addr 172.21.15.27:0/2142447655 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:23:06.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.408+0000 7f89824af640 1 --2- 172.21.15.27:0/2142447655 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f897c06dfd0 0x7f897c155dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:06.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.408+0000 7f8981cae640 1 -- 172.21.15.27:0/2142447655 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f897c162040 msgr2=0x7f897c15a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:06.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.408+0000 7f8981cae640 1 --2- 172.21.15.27:0/2142447655 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f897c162040 0x7f897c15a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:06.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.408+0000 7f8981cae640 1 -- 172.21.15.27:0/2142447655 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f897c06dfd0 msgr2=0x7f897c155dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:06.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.408+0000 7f8981cae640 1 --2- 172.21.15.27:0/2142447655 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f897c06dfd0 0x7f897c155dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:06.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.408+0000 7f8981cae640 1 -- 172.21.15.27:0/2142447655 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f897c156310 con 0x7f897c0d3790 2024-09-12T03:23:06.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.409+0000 7f8981cae640 1 --2- 172.21.15.27:0/2142447655 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f897c0d3790 0x7f897c0b9ee0 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7f897000c820 tx=0x7f897000cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:06.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.409+0000 7f897affd640 1 -- 172.21.15.27:0/2142447655 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8970015070 con 0x7f897c0d3790 2024-09-12T03:23:06.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.409+0000 7f897affd640 1 -- 172.21.15.27:0/2142447655 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8970004240 con 0x7f897c0d3790 2024-09-12T03:23:06.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.409+0000 7f8984713640 1 -- 172.21.15.27:0/2142447655 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f897c0b8520 con 0x7f897c0d3790 2024-09-12T03:23:06.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.409+0000 7f897affd640 1 -- 172.21.15.27:0/2142447655 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f897000ce10 con 0x7f897c0d3790 2024-09-12T03:23:06.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.409+0000 7f8984713640 1 -- 172.21.15.27:0/2142447655 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f897c0b8ae0 con 0x7f897c0d3790 2024-09-12T03:23:06.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.410+0000 7f8984713640 1 -- 172.21.15.27:0/2142447655 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f897c06e6e0 con 0x7f897c0d3790 2024-09-12T03:23:06.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.410+0000 7f897affd640 1 -- 172.21.15.27:0/2142447655 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 23) ==== 91110+0+0 (secure 0 0 0) 0x7f8970007830 con 0x7f897c0d3790 2024-09-12T03:23:06.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.411+0000 7f897affd640 1 --2- 172.21.15.27:0/2142447655 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f896806c4f0 0x7f896806e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:06.413 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.411+0000 7f897affd640 1 -- 172.21.15.27:0/2142447655 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 5765+0+0 (secure 0 0 0) 0x7f897008bb20 con 0x7f897c0d3790 2024-09-12T03:23:06.413 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.411+0000 7f89824af640 1 --2- 172.21.15.27:0/2142447655 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f896806c4f0 0x7f896806e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:06.413 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.411+0000 7f89824af640 1 --2- 172.21.15.27:0/2142447655 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f896806c4f0 0x7f896806e9b0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f896c000f60 tx=0x7f896c009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:06.415 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.415+0000 7f897affd640 1 -- 172.21.15.27:0/2142447655 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f8970059960 con 0x7f897c0d3790 2024-09-12T03:23:06.615 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.615+0000 7f8984713640 1 -- 172.21.15.27:0/2142447655 --> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol98201", "group_name": "subvol_grp18820", "target": ["mon-mgr", ""]}) -- 0x7f897c0b44a0 con 0x7f896806c4f0 2024-09-12T03:23:06.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.627+0000 7f897affd640 1 -- 172.21.15.27:0/2142447655 <== mgr.4113 v2:172.21.15.27:6836/1226920526 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f897c0b44a0 con 0x7f896806c4f0 2024-09-12T03:23:06.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.629+0000 7f8984713640 1 -- 172.21.15.27:0/2142447655 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f896806c4f0 msgr2=0x7f896806e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:06.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.629+0000 7f8984713640 1 --2- 172.21.15.27:0/2142447655 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f896806c4f0 0x7f896806e9b0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f896c000f60 tx=0x7f896c009040 comp rx=0 tx=0).stop 2024-09-12T03:23:06.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.629+0000 7f8984713640 1 -- 172.21.15.27:0/2142447655 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f897c0d3790 msgr2=0x7f897c0b9ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:06.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.629+0000 7f8984713640 1 --2- 172.21.15.27:0/2142447655 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f897c0d3790 0x7f897c0b9ee0 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7f897000c820 tx=0x7f897000cce0 comp rx=0 tx=0).stop 2024-09-12T03:23:06.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.629+0000 7f8984713640 1 -- 172.21.15.27:0/2142447655 shutdown_connections 2024-09-12T03:23:06.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.629+0000 7f8984713640 1 --2- 172.21.15.27:0/2142447655 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f897c162040 0x7f897c15a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:06.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.629+0000 7f8984713640 1 --2- 172.21.15.27:0/2142447655 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f896806c4f0 0x7f896806e9b0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:06.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.629+0000 7f8984713640 1 --2- 172.21.15.27:0/2142447655 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f897c0d3790 0x7f897c0b9ee0 unknown :-1 s=CLOSED pgs=1455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:06.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.629+0000 7f8984713640 1 --2- 172.21.15.27:0/2142447655 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f897c06dfd0 0x7f897c155dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:06.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.629+0000 7f8984713640 1 -- 172.21.15.27:0/2142447655 >> 172.21.15.27:0/2142447655 conn(0x7f897c0b6700 msgr2=0x7f897c0d2330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:06.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.629+0000 7f8984713640 1 -- 172.21.15.27:0/2142447655 shutdown_connections 2024-09-12T03:23:06.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.629+0000 7f8984713640 1 -- 172.21.15.27:0/2142447655 wait complete. 2024-09-12T03:23:06.641 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-09-12T03:23:06.732 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-12T03:23:06.734 INFO:teuthology.orchestra.run.smithi027.stdout:2 2024-09-12T03:23:06.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T03:23:06.741 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup rm cephfs subvol_grp18820 2024-09-12T03:23:06.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.846+0000 7fb02520a640 1 Processor -- start 2024-09-12T03:23:06.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.846+0000 7fb02520a640 1 -- start start 2024-09-12T03:23:06.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.847+0000 7fb02520a640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fb02006cc40 0x7fb02006d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:06.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.847+0000 7fb02520a640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb02006db30 0x7fb02006df60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:06.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.847+0000 7fb02520a640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb02006d5d0 con 0x7fb02015e650 2024-09-12T03:23:06.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.847+0000 7fb02520a640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fb02006d850 con 0x7fb02006cc40 2024-09-12T03:23:06.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.847+0000 7fb02520a640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fb0200d1260 con 0x7fb02006db30 2024-09-12T03:23:06.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.847+0000 7fb01ed9d640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fb02015e650 0x7fb0200d1d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:54140/0 (socket says 172.21.15.27:54140) 2024-09-12T03:23:06.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.847+0000 7fb01ed9d640 1 -- 172.21.15.27:0/1958463777 learned_addr learned my addr 172.21.15.27:0/1958463777 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:23:06.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.847+0000 7fb01e59c640 1 --2- 172.21.15.27:0/1958463777 >> v2:172.21.15.45:3300/0 conn(0x7fb02006cc40 0x7fb02006d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:06.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.847+0000 7fb01dd9b640 1 -- 172.21.15.27:0/1958463777 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4253522374 0 0) 0x7fb018001110 con 0x7fb02015e650 2024-09-12T03:23:06.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.847+0000 7fb01f59e640 1 --2- 172.21.15.27:0/1958463777 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb02006db30 0x7fb02006df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:06.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.847+0000 7fb01dd9b640 1 -- 172.21.15.27:0/1958463777 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb00c003520 con 0x7fb02015e650 2024-09-12T03:23:06.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.848+0000 7fb01dd9b640 1 -- 172.21.15.27:0/1958463777 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1711522605 0 0) 0x7fb00c003520 con 0x7fb02015e650 2024-09-12T03:23:06.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.848+0000 7fb01dd9b640 1 -- 172.21.15.27:0/1958463777 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb02006db30 msgr2=0x7fb02006df60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:06.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.848+0000 7fb01dd9b640 1 --2- 172.21.15.27:0/1958463777 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb02006db30 0x7fb02006df60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:06.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.848+0000 7fb01dd9b640 1 -- 172.21.15.27:0/1958463777 >> v2:172.21.15.45:3300/0 conn(0x7fb02006cc40 msgr2=0x7fb02006d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:06.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.848+0000 7fb01dd9b640 1 --2- 172.21.15.27:0/1958463777 >> v2:172.21.15.45:3300/0 conn(0x7fb02006cc40 0x7fb02006d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:06.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.848+0000 7fb01dd9b640 1 -- 172.21.15.27:0/1958463777 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0200d1540 con 0x7fb02015e650 2024-09-12T03:23:06.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.848+0000 7fb01dd9b640 1 -- 172.21.15.27:0/1958463777 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 847755213 0 0) 0x7fb018001840 con 0x7fb02015e650 2024-09-12T03:23:06.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.848+0000 7fb01dd9b640 1 -- 172.21.15.27:0/1958463777 >> v1:172.21.15.27:6789/0 conn(0x7fb02015e650 legacy=0x7fb0200d1d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:06.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.848+0000 7fb01dd9b640 1 --2- 172.21.15.27:0/1958463777 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb02006db30 0x7fb00c0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:06.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.848+0000 7fb01dd9b640 1 --2- 172.21.15.27:0/1958463777 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb00c003f30 0x7fb00c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:06.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.848+0000 7fb01dd9b640 1 --2- 172.21.15.27:0/1958463777 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb00c004d40 0x7fb00c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:06.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.848+0000 7fb01dd9b640 1 -- 172.21.15.27:0/1958463777 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fb02006d850 con 0x7fb00c003f30 2024-09-12T03:23:06.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.848+0000 7fb01dd9b640 1 -- 172.21.15.27:0/1958463777 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb0200d1260 con 0x7fb00c004d40 2024-09-12T03:23:06.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.848+0000 7fb01dd9b640 1 -- 172.21.15.27:0/1958463777 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fb00c00ba10 con 0x7fb02006db30 2024-09-12T03:23:06.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.848+0000 7fb01e59c640 1 --2- 172.21.15.27:0/1958463777 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb00c003f30 0x7fb00c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:06.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.848+0000 7fb01ed9d640 1 --2- 172.21.15.27:0/1958463777 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb02006db30 0x7fb00c0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:06.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.848+0000 7fb01f59e640 1 --2- 172.21.15.27:0/1958463777 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb00c004d40 0x7fb00c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:06.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.848+0000 7fb01e59c640 1 -- 172.21.15.27:0/1958463777 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb02006db30 msgr2=0x7fb00c0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:06.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.848+0000 7fb01e59c640 1 --2- 172.21.15.27:0/1958463777 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb02006db30 0x7fb00c0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:06.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.848+0000 7fb01e59c640 1 -- 172.21.15.27:0/1958463777 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb00c004d40 msgr2=0x7fb00c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:06.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.848+0000 7fb01e59c640 1 --2- 172.21.15.27:0/1958463777 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb00c004d40 0x7fb00c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:06.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.848+0000 7fb01e59c640 1 -- 172.21.15.27:0/1958463777 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb00c003520 con 0x7fb00c003f30 2024-09-12T03:23:06.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.849+0000 7fb01e59c640 1 --2- 172.21.15.27:0/1958463777 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb00c003f30 0x7fb00c004800 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7fb008009af0 tx=0x7fb00c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5297afaafebea3bf server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:06.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.849+0000 7fb01dd9b640 1 -- 172.21.15.27:0/1958463777 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb008036070 con 0x7fb00c003f30 2024-09-12T03:23:06.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.849+0000 7fb01dd9b640 1 -- 172.21.15.27:0/1958463777 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb008031400 con 0x7fb00c003f30 2024-09-12T03:23:06.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.849+0000 7fb01dd9b640 1 -- 172.21.15.27:0/1958463777 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb008009690 con 0x7fb00c003f30 2024-09-12T03:23:06.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.850+0000 7fb02520a640 1 -- 172.21.15.27:0/1958463777 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb00c003f30 msgr2=0x7fb00c004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:06.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.850+0000 7fb02520a640 1 --2- 172.21.15.27:0/1958463777 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb00c003f30 0x7fb00c004800 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7fb008009af0 tx=0x7fb00c005830 comp rx=0 tx=0).stop 2024-09-12T03:23:06.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.850+0000 7fb01f59e640 1 -- 172.21.15.27:0/1958463777 reap_dead start 2024-09-12T03:23:06.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.850+0000 7fb02520a640 1 -- 172.21.15.27:0/1958463777 shutdown_connections 2024-09-12T03:23:06.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.850+0000 7fb02520a640 1 -- 172.21.15.27:0/1958463777 >> 172.21.15.27:0/1958463777 conn(0x7fb02006f320 msgr2=0x7fb02006f700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:06.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.850+0000 7fb02520a640 1 -- 172.21.15.27:0/1958463777 shutdown_connections 2024-09-12T03:23:06.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.850+0000 7fb02520a640 1 -- 172.21.15.27:0/1958463777 wait complete. 2024-09-12T03:23:06.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.851+0000 7fb02520a640 1 Processor -- start 2024-09-12T03:23:06.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.851+0000 7fb02520a640 1 -- start start 2024-09-12T03:23:06.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.852+0000 7fb02520a640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb00c003f30 0x7fb0200b1a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:06.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.852+0000 7fb02520a640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb0200aa4c0 0x7fb0200aa940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:06.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.852+0000 7fb02520a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb0200cbb60 0x7fb0200cbfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:06.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.852+0000 7fb02520a640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fb02006e0e0 con 0x7fb00c003f30 2024-09-12T03:23:06.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.852+0000 7fb02520a640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb02006df60 con 0x7fb0200cbb60 2024-09-12T03:23:06.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.852+0000 7fb02520a640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fb02006e260 con 0x7fb0200aa4c0 2024-09-12T03:23:06.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.852+0000 7fb01ed9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb00c003f30 0x7fb0200b1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:06.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.852+0000 7fb01ed9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb00c003f30 0x7fb0200b1a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59846/0 (socket says 172.21.15.27:59846) 2024-09-12T03:23:06.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.852+0000 7fb01ed9d640 1 -- 172.21.15.27:0/1966511256 learned_addr learned my addr 172.21.15.27:0/1966511256 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:23:06.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.852+0000 7fb01f59e640 1 --2- 172.21.15.27:0/1966511256 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb0200cbb60 0x7fb0200cbfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:06.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.852+0000 7fb01e59c640 1 --2- 172.21.15.27:0/1966511256 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb0200aa4c0 0x7fb0200aa940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:06.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.853+0000 7fb01ed9d640 1 -- 172.21.15.27:0/1966511256 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb0200aa4c0 msgr2=0x7fb0200aa940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:06.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.853+0000 7fb01ed9d640 1 --2- 172.21.15.27:0/1966511256 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb0200aa4c0 0x7fb0200aa940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:06.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.853+0000 7fb01ed9d640 1 -- 172.21.15.27:0/1966511256 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb0200cbb60 msgr2=0x7fb0200cbfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:06.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.853+0000 7fb01ed9d640 1 --2- 172.21.15.27:0/1966511256 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb0200cbb60 0x7fb0200cbfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:06.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.853+0000 7fb01ed9d640 1 -- 172.21.15.27:0/1966511256 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0200b1fc0 con 0x7fb00c003f30 2024-09-12T03:23:06.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.853+0000 7fb01ed9d640 1 --2- 172.21.15.27:0/1966511256 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb00c003f30 0x7fb0200b1a80 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7fb018007310 tx=0x7fb0180077d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:06.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.853+0000 7fb0177fe640 1 -- 172.21.15.27:0/1966511256 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb01800f990 con 0x7fb00c003f30 2024-09-12T03:23:06.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.853+0000 7fb0177fe640 1 -- 172.21.15.27:0/1966511256 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb018018070 con 0x7fb00c003f30 2024-09-12T03:23:06.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.853+0000 7fb02520a640 1 -- 172.21.15.27:0/1966511256 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0200cc8d0 con 0x7fb00c003f30 2024-09-12T03:23:06.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.853+0000 7fb0177fe640 1 -- 172.21.15.27:0/1966511256 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb01800fd70 con 0x7fb00c003f30 2024-09-12T03:23:06.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.853+0000 7fb02520a640 1 -- 172.21.15.27:0/1966511256 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb02014b4a0 con 0x7fb00c003f30 2024-09-12T03:23:06.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.854+0000 7fb02520a640 1 -- 172.21.15.27:0/1966511256 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0200d36b0 con 0x7fb00c003f30 2024-09-12T03:23:06.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.855+0000 7fb0177fe640 1 -- 172.21.15.27:0/1966511256 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 23) ==== 91110+0+0 (secure 0 0 0) 0x7fb01801d460 con 0x7fb00c003f30 2024-09-12T03:23:06.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.855+0000 7fb0177fe640 1 --2- 172.21.15.27:0/1966511256 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fb00006c520 0x7fb00006e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:06.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.855+0000 7fb0177fe640 1 -- 172.21.15.27:0/1966511256 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 5765+0+0 (secure 0 0 0) 0x7fb01808b990 con 0x7fb00c003f30 2024-09-12T03:23:06.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.855+0000 7fb01e59c640 1 --2- 172.21.15.27:0/1966511256 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fb00006c520 0x7fb00006e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:06.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.856+0000 7fb01e59c640 1 --2- 172.21.15.27:0/1966511256 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fb00006c520 0x7fb00006e9e0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fb008033000 tx=0x7fb008008030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:06.860 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:06.860+0000 7fb0177fe640 1 -- 172.21.15.27:0/1966511256 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fb02006d5d0 con 0x7fb00c003f30 2024-09-12T03:23:07.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:07.062+0000 7fb02520a640 1 -- 172.21.15.27:0/1966511256 --> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup rm", "vol_name": "cephfs", "group_name": "subvol_grp18820", "target": ["mon-mgr", ""]}) -- 0x7fb0200d02c0 con 0x7fb00006c520 2024-09-12T03:23:07.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:07.066+0000 7fb0177fe640 1 -- 172.21.15.27:0/1966511256 <== mgr.4113 v2:172.21.15.27:6836/1226920526 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fb0200d02c0 con 0x7fb00006c520 2024-09-12T03:23:07.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:07.068+0000 7fb02520a640 1 -- 172.21.15.27:0/1966511256 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fb00006c520 msgr2=0x7fb00006e9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:07.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:07.068+0000 7fb02520a640 1 --2- 172.21.15.27:0/1966511256 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fb00006c520 0x7fb00006e9e0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fb008033000 tx=0x7fb008008030 comp rx=0 tx=0).stop 2024-09-12T03:23:07.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:07.068+0000 7fb02520a640 1 -- 172.21.15.27:0/1966511256 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb00c003f30 msgr2=0x7fb0200b1a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:07.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:07.068+0000 7fb02520a640 1 --2- 172.21.15.27:0/1966511256 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb00c003f30 0x7fb0200b1a80 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7fb018007310 tx=0x7fb0180077d0 comp rx=0 tx=0).stop 2024-09-12T03:23:07.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:07.068+0000 7fb02520a640 1 -- 172.21.15.27:0/1966511256 shutdown_connections 2024-09-12T03:23:07.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:07.068+0000 7fb02520a640 1 --2- 172.21.15.27:0/1966511256 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb0200cbb60 0x7fb0200cbfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:07.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:07.068+0000 7fb02520a640 1 --2- 172.21.15.27:0/1966511256 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb0200aa4c0 0x7fb0200aa940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:07.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:07.068+0000 7fb02520a640 1 --2- 172.21.15.27:0/1966511256 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fb00006c520 0x7fb00006e9e0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:07.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:07.068+0000 7fb02520a640 1 --2- 172.21.15.27:0/1966511256 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb00c003f30 0x7fb0200b1a80 unknown :-1 s=CLOSED pgs=1458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:07.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:07.068+0000 7fb02520a640 1 -- 172.21.15.27:0/1966511256 >> 172.21.15.27:0/1966511256 conn(0x7fb02006f320 msgr2=0x7fb0200b5690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:07.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:07.068+0000 7fb02520a640 1 -- 172.21.15.27:0/1966511256 shutdown_connections 2024-09-12T03:23:07.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:07.068+0000 7fb02520a640 1 -- 172.21.15.27:0/1966511256 wait complete. 2024-09-12T03:23:07.079 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-12T03:23:07.079 DEBUG:teuthology.orchestra.run.smithi027:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-12T03:23:07.116 DEBUG:teuthology.orchestra.run.smithi045:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-12T03:23:07.159 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-12T03:23:07.159 INFO:tasks.cephfs.mount:Terminating background process 2024-09-12T03:23:07.159 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-12T03:23:07.159 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T03:23:07.175 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-12T03:23:07.175 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T03:23:07.232 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-12T03:23:07.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:07.461+0000 7f2d6539f640 1 Processor -- start 2024-09-12T03:23:07.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:07.461+0000 7f2d6539f640 1 -- start start 2024-09-12T03:23:07.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:07.462+0000 7f2d6539f640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2d600cf810 0x7f2d600b8000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:07.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:07.462+0000 7f2d6539f640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d6006c100 con 0x7f2d60161d80 2024-09-12T03:23:07.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:07.462+0000 7f2d6539f640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d6006c3e0 con 0x7f2d600d5320 2024-09-12T03:23:07.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:07.462+0000 7f2d6539f640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f2d600d4460 con 0x7f2d600cf810 2024-09-12T03:23:07.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:07.462+0000 7f2d5ffff640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f2d60161d80 0x7f2d600cf100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:54152/0 (socket says 172.21.15.27:54152) 2024-09-12T03:23:07.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:07.462+0000 7f2d5ffff640 1 -- 172.21.15.27:0/3768601072 learned_addr learned my addr 172.21.15.27:0/3768601072 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:23:07.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:07.462+0000 7f2d5f7fe640 1 -- 172.21.15.27:0/3768601072 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3755126547 0 0) 0x7f2d6006c100 con 0x7f2d60161d80 2024-09-12T03:23:07.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:07.462+0000 7f2d5f7fe640 1 -- 172.21.15.27:0/3768601072 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2d4c003520 con 0x7f2d60161d80 2024-09-12T03:23:07.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:07.463+0000 7f2d64b9e640 1 --2- 172.21.15.27:0/3768601072 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2d600cf810 0x7f2d600b8000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:07.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:07.463+0000 7f2d5f7fe640 1 -- 172.21.15.27:0/3768601072 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2551039138 0 0) 0x7f2d4c003520 con 0x7f2d60161d80 2024-09-12T03:23:07.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:07.463+0000 7f2d5f7fe640 1 -- 172.21.15.27:0/3768601072 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2d600cf810 msgr2=0x7f2d600b8000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:07.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:07.463+0000 7f2d5f7fe640 1 --2- 172.21.15.27:0/3768601072 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2d600cf810 0x7f2d600b8000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:07.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:07.463+0000 7f2d5f7fe640 1 -- 172.21.15.27:0/3768601072 >> v1:172.21.15.45:6789/0 conn(0x7f2d600d5320 legacy=0x7f2d600b78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:07.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:07.463+0000 7f2d5f7fe640 1 -- 172.21.15.27:0/3768601072 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d6006c6c0 con 0x7f2d60161d80 2024-09-12T03:23:07.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:07.463+0000 7f2d5f7fe640 1 -- 172.21.15.27:0/3768601072 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 847755213 0 0) 0x7f2d50001670 con 0x7f2d60161d80 2024-09-12T03:23:07.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:07.463+0000 7f2d5f7fe640 1 -- 172.21.15.27:0/3768601072 >> v1:172.21.15.27:6789/0 conn(0x7f2d60161d80 legacy=0x7f2d600cf100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:07.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:07.463+0000 7f2d5f7fe640 1 --2- 172.21.15.27:0/3768601072 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2d4c003ac0 0x7f2d4c003ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:07.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:07.463+0000 7f2d5f7fe640 1 --2- 172.21.15.27:0/3768601072 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2d600cf810 0x7f2d4c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:07.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:07.463+0000 7f2d5f7fe640 1 --2- 172.21.15.27:0/3768601072 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2d4c004e60 0x7f2d4c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:07.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:07.463+0000 7f2d5f7fe640 1 -- 172.21.15.27:0/3768601072 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f2d4c005510 con 0x7f2d4c003ac0 2024-09-12T03:23:07.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:07.463+0000 7f2d5f7fe640 1 -- 172.21.15.27:0/3768601072 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f2d4c00bf00 con 0x7f2d4c004e60 2024-09-12T03:23:07.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:07.463+0000 7f2d5f7fe640 1 -- 172.21.15.27:0/3768601072 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f2d4c00c080 con 0x7f2d600cf810 2024-09-12T03:23:07.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:07.463+0000 7f2d5ffff640 1 --2- 172.21.15.27:0/3768601072 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2d4c003ac0 0x7f2d4c003ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:07.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:07.463+0000 7f2d57fff640 1 --2- 172.21.15.27:0/3768601072 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2d600cf810 0x7f2d4c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:07.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:07.463+0000 7f2d64b9e640 1 --2- 172.21.15.27:0/3768601072 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2d4c004e60 0x7f2d4c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:07.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:07.463+0000 7f2d5ffff640 1 -- 172.21.15.27:0/3768601072 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2d600cf810 msgr2=0x7f2d4c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:07.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:07.463+0000 7f2d5ffff640 1 --2- 172.21.15.27:0/3768601072 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2d600cf810 0x7f2d4c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:07.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:07.463+0000 7f2d5ffff640 1 -- 172.21.15.27:0/3768601072 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2d4c004e60 msgr2=0x7f2d4c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:07.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:07.463+0000 7f2d5ffff640 1 --2- 172.21.15.27:0/3768601072 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2d4c004e60 0x7f2d4c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:07.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:07.463+0000 7f2d5ffff640 1 -- 172.21.15.27:0/3768601072 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d4c003520 con 0x7f2d4c003ac0 2024-09-12T03:23:07.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:07.463+0000 7f2d57fff640 1 --2- 172.21.15.27:0/3768601072 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2d600cf810 0x7f2d4c004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:23:07.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:07.464+0000 7f2d5ffff640 1 --2- 172.21.15.27:0/3768601072 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2d4c003ac0 0x7f2d4c003ee0 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7f2d5000c240 tx=0x7f2d4c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c549c63bc7e28af server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:07.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:07.464+0000 7f2d5f7fe640 1 -- 172.21.15.27:0/3768601072 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d50007d10 con 0x7f2d4c003ac0 2024-09-12T03:23:07.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:07.464+0000 7f2d5f7fe640 1 -- 172.21.15.27:0/3768601072 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d50005ad0 con 0x7f2d4c003ac0 2024-09-12T03:23:07.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:07.464+0000 7f2d5f7fe640 1 -- 172.21.15.27:0/3768601072 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d50007070 con 0x7f2d4c003ac0 2024-09-12T03:23:07.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:07.465+0000 7f2d6539f640 1 -- 172.21.15.27:0/3768601072 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2d4c003ac0 msgr2=0x7f2d4c003ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:07.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:07.465+0000 7f2d6539f640 1 --2- 172.21.15.27:0/3768601072 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2d4c003ac0 0x7f2d4c003ee0 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7f2d5000c240 tx=0x7f2d4c005830 comp rx=0 tx=0).stop 2024-09-12T03:23:07.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:07.465+0000 7f2d64b9e640 1 -- 172.21.15.27:0/3768601072 reap_dead start 2024-09-12T03:23:07.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:07.465+0000 7f2d6539f640 1 -- 172.21.15.27:0/3768601072 shutdown_connections 2024-09-12T03:23:07.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:07.465+0000 7f2d6539f640 1 -- 172.21.15.27:0/3768601072 >> 172.21.15.27:0/3768601072 conn(0x7f2d600d45f0 msgr2=0x7f2d600d4a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:07.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:07.465+0000 7f2d6539f640 1 -- 172.21.15.27:0/3768601072 shutdown_connections 2024-09-12T03:23:07.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:07.465+0000 7f2d6539f640 1 -- 172.21.15.27:0/3768601072 wait complete. 2024-09-12T03:23:07.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:07.466+0000 7f2d6539f640 1 Processor -- start 2024-09-12T03:23:07.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:07.466+0000 7f2d6539f640 1 -- start start 2024-09-12T03:23:07.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:07.467+0000 7f2d6539f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2d4c003ac0 0x7f2d600cd130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:07.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:07.467+0000 7f2d6539f640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2d601f4e90 0x7f2d6015a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:07.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:07.467+0000 7f2d6539f640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2d6015ade0 0x7f2d6015b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:07.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:07.467+0000 7f2d6539f640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f2d600d4220 con 0x7f2d601f4e90 2024-09-12T03:23:07.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:07.467+0000 7f2d6539f640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f2d600d40a0 con 0x7f2d4c003ac0 2024-09-12T03:23:07.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:07.467+0000 7f2d6539f640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f2d600b3b40 con 0x7f2d6015ade0 2024-09-12T03:23:07.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:07.467+0000 7f2d57fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2d601f4e90 0x7f2d6015a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:07.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:07.467+0000 7f2d57fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2d601f4e90 0x7f2d6015a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59872/0 (socket says 172.21.15.27:59872) 2024-09-12T03:23:07.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:07.467+0000 7f2d57fff640 1 -- 172.21.15.27:0/4222579164 learned_addr learned my addr 172.21.15.27:0/4222579164 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:23:07.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:07.467+0000 7f2d5ffff640 1 --2- 172.21.15.27:0/4222579164 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2d4c003ac0 0x7f2d600cd130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:07.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:07.467+0000 7f2d64b9e640 1 --2- 172.21.15.27:0/4222579164 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2d6015ade0 0x7f2d6015b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:07.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:07.467+0000 7f2d57fff640 1 -- 172.21.15.27:0/4222579164 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2d6015ade0 msgr2=0x7f2d6015b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:07.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:07.467+0000 7f2d57fff640 1 --2- 172.21.15.27:0/4222579164 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2d6015ade0 0x7f2d6015b7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:07.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:07.467+0000 7f2d57fff640 1 -- 172.21.15.27:0/4222579164 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2d4c003ac0 msgr2=0x7f2d600cd130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:07.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:07.467+0000 7f2d57fff640 1 --2- 172.21.15.27:0/4222579164 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2d4c003ac0 0x7f2d600cd130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:07.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:07.467+0000 7f2d57fff640 1 -- 172.21.15.27:0/4222579164 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d6014ac60 con 0x7f2d601f4e90 2024-09-12T03:23:07.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:07.468+0000 7f2d57fff640 1 --2- 172.21.15.27:0/4222579164 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2d601f4e90 0x7f2d6015a8a0 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7f2d4800e630 tx=0x7f2d4800eaf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:07.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:07.468+0000 7f2d5d7fa640 1 -- 172.21.15.27:0/4222579164 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d480052e0 con 0x7f2d601f4e90 2024-09-12T03:23:07.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:07.468+0000 7f2d5d7fa640 1 -- 172.21.15.27:0/4222579164 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d48018070 con 0x7f2d601f4e90 2024-09-12T03:23:07.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:07.468+0000 7f2d6539f640 1 -- 172.21.15.27:0/4222579164 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d6014af40 con 0x7f2d601f4e90 2024-09-12T03:23:07.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:07.468+0000 7f2d5d7fa640 1 -- 172.21.15.27:0/4222579164 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d48003810 con 0x7f2d601f4e90 2024-09-12T03:23:07.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:07.468+0000 7f2d6539f640 1 -- 172.21.15.27:0/4222579164 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d6014b500 con 0x7f2d601f4e90 2024-09-12T03:23:07.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:07.469+0000 7f2d6539f640 1 -- 172.21.15.27:0/4222579164 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d60063330 con 0x7f2d601f4e90 2024-09-12T03:23:07.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:07.469+0000 7f2d5d7fa640 1 -- 172.21.15.27:0/4222579164 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 23) ==== 91110+0+0 (secure 0 0 0) 0x7f2d480039b0 con 0x7f2d601f4e90 2024-09-12T03:23:07.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:07.470+0000 7f2d5d7fa640 1 --2- 172.21.15.27:0/4222579164 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f2d4006c520 0x7f2d4006e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:07.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:07.470+0000 7f2d5d7fa640 1 -- 172.21.15.27:0/4222579164 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 5765+0+0 (secure 0 0 0) 0x7f2d4808b520 con 0x7f2d601f4e90 2024-09-12T03:23:07.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:07.470+0000 7f2d5ffff640 1 --2- 172.21.15.27:0/4222579164 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f2d4006c520 0x7f2d4006e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:07.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:07.470+0000 7f2d5ffff640 1 --2- 172.21.15.27:0/4222579164 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f2d4006c520 0x7f2d4006e9e0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f2d50033000 tx=0x7f2d5002e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:07.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:07.474+0000 7f2d5d7fa640 1 -- 172.21.15.27:0/4222579164 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f2d48058fb0 con 0x7f2d601f4e90 2024-09-12T03:23:07.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:07.660+0000 7f2d6539f640 1 -- 172.21.15.27:0/4222579164 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f2d600cf100 con 0x7f2d601f4e90 2024-09-12T03:23:07.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:07.660+0000 7f2d5d7fa640 1 -- 172.21.15.27:0/4222579164 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v138) ==== 80+0+0 (secure 0 0 0) 0x7f2d4805d2b0 con 0x7f2d601f4e90 2024-09-12T03:23:07.661 INFO:teuthology.orchestra.run.smithi027.stderr:listed 0 entries 2024-09-12T03:23:07.662 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:07.662+0000 7f2d6539f640 1 -- 172.21.15.27:0/4222579164 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f2d4006c520 msgr2=0x7f2d4006e9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:07.662 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:07.662+0000 7f2d6539f640 1 --2- 172.21.15.27:0/4222579164 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f2d4006c520 0x7f2d4006e9e0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f2d50033000 tx=0x7f2d5002e000 comp rx=0 tx=0).stop 2024-09-12T03:23:07.663 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:07.662+0000 7f2d6539f640 1 -- 172.21.15.27:0/4222579164 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2d601f4e90 msgr2=0x7f2d6015a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:07.663 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:07.662+0000 7f2d6539f640 1 --2- 172.21.15.27:0/4222579164 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2d601f4e90 0x7f2d6015a8a0 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7f2d4800e630 tx=0x7f2d4800eaf0 comp rx=0 tx=0).stop 2024-09-12T03:23:07.663 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:07.662+0000 7f2d6539f640 1 -- 172.21.15.27:0/4222579164 shutdown_connections 2024-09-12T03:23:07.663 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:07.662+0000 7f2d6539f640 1 --2- 172.21.15.27:0/4222579164 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2d6015ade0 0x7f2d6015b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:07.663 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:07.662+0000 7f2d6539f640 1 --2- 172.21.15.27:0/4222579164 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f2d4006c520 0x7f2d4006e9e0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:07.663 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:07.662+0000 7f2d6539f640 1 --2- 172.21.15.27:0/4222579164 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2d601f4e90 0x7f2d6015a8a0 unknown :-1 s=CLOSED pgs=1461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:07.663 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:07.662+0000 7f2d6539f640 1 --2- 172.21.15.27:0/4222579164 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2d4c003ac0 0x7f2d600cd130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:07.663 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:07.662+0000 7f2d6539f640 1 -- 172.21.15.27:0/4222579164 >> 172.21.15.27:0/4222579164 conn(0x7f2d600d45f0 msgr2=0x7f2d600b5790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:07.663 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:07.662+0000 7f2d6539f640 1 -- 172.21.15.27:0/4222579164 shutdown_connections 2024-09-12T03:23:07.664 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:07.662+0000 7f2d6539f640 1 -- 172.21.15.27:0/4222579164 wait complete. 2024-09-12T03:23:07.674 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-12T03:23:07.674 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:23:07.674 DEBUG:teuthology.orchestra.run.smithi027:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-12T03:23:07.704 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi027.front.sepia.ceph.com 2024-09-12T03:23:07.705 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:23:07.705 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-12T03:23:07.761 INFO:tasks.cephfs.mount:Terminating background process 2024-09-12T03:23:07.761 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-12T03:23:07.761 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T03:23:07.782 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-12T03:23:07.782 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T03:23:07.842 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-12T03:23:07.945 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:07.945+0000 7f441c7bd640 1 Processor -- start 2024-09-12T03:23:07.945 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:07.945+0000 7f441c7bd640 1 -- start start 2024-09-12T03:23:07.947 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:07.946+0000 7f441c7bd640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f44140b91a0 0x7f4414169880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:07.947 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:07.946+0000 7f441c7bd640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f44140d3760 con 0x7f441415eb90 2024-09-12T03:23:07.947 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:07.946+0000 7f441c7bd640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f44140d3a40 con 0x7f44140d5200 2024-09-12T03:23:07.947 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:07.946+0000 7f441c7bd640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f44140b7d20 con 0x7f44140b91a0 2024-09-12T03:23:07.947 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:07.947+0000 7f441ad5a640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f44140b91a0 0x7f4414169880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:07.947 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:07.947+0000 7f441a559640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f441415eb90 0x7f44140d2df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:54166/0 (socket says 172.21.15.27:54166) 2024-09-12T03:23:07.947 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:07.947+0000 7f441a559640 1 -- 172.21.15.27:0/3893245470 learned_addr learned my addr 172.21.15.27:0/3893245470 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:23:07.947 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:07.947+0000 7f441ad5a640 1 -- 172.21.15.27:0/3893245470 >> v1:172.21.15.45:6789/0 conn(0x7f44140d5200 legacy=0x7f44140d1010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:07.947 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:07.947+0000 7f441ad5a640 1 -- 172.21.15.27:0/3893245470 >> v1:172.21.15.27:6789/0 conn(0x7f441415eb90 legacy=0x7f44140d2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:07.947 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:07.947+0000 7f441ad5a640 1 -- 172.21.15.27:0/3893245470 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44140d3d20 con 0x7f44140b91a0 2024-09-12T03:23:07.947 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:07.947+0000 7f4419557640 1 -- 172.21.15.27:0/3893245470 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1990725292 0 0) 0x7f44140d3760 con 0x7f441415eb90 2024-09-12T03:23:07.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:07.948+0000 7f441ad5a640 1 --2- 172.21.15.27:0/3893245470 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f44140b91a0 0x7f4414169880 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7f4404009d90 tx=0x7f440402a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbc6b58ff65bf96d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:07.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:07.948+0000 7f4419557640 1 -- 172.21.15.27:0/3893245470 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4404007740 con 0x7f44140b91a0 2024-09-12T03:23:07.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:07.948+0000 7f4419557640 1 -- 172.21.15.27:0/3893245470 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44040078e0 con 0x7f44140b91a0 2024-09-12T03:23:07.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:07.948+0000 7f4419557640 1 -- 172.21.15.27:0/3893245470 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4404007cb0 con 0x7f44140b91a0 2024-09-12T03:23:07.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:07.948+0000 7f441c7bd640 1 -- 172.21.15.27:0/3893245470 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f44140b91a0 msgr2=0x7f4414169880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:07.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:07.948+0000 7f441c7bd640 1 --2- 172.21.15.27:0/3893245470 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f44140b91a0 0x7f4414169880 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7f4404009d90 tx=0x7f440402a6f0 comp rx=0 tx=0).stop 2024-09-12T03:23:07.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:07.948+0000 7f441c7bd640 1 -- 172.21.15.27:0/3893245470 shutdown_connections 2024-09-12T03:23:07.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:07.948+0000 7f441c7bd640 1 --2- 172.21.15.27:0/3893245470 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f44140b91a0 0x7f4414169880 unknown :-1 s=CLOSED pgs=1463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:07.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:07.949+0000 7f441c7bd640 1 -- 172.21.15.27:0/3893245470 >> 172.21.15.27:0/3893245470 conn(0x7f441406dce0 msgr2=0x7f441406e0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:07.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:07.949+0000 7f441c7bd640 1 -- 172.21.15.27:0/3893245470 shutdown_connections 2024-09-12T03:23:07.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:07.949+0000 7f441c7bd640 1 -- 172.21.15.27:0/3893245470 wait complete. 2024-09-12T03:23:07.952 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:07.950+0000 7f441c7bd640 1 Processor -- start 2024-09-12T03:23:07.952 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:07.950+0000 7f441c7bd640 1 -- start start 2024-09-12T03:23:07.952 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:07.950+0000 7f441c7bd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f44140b91a0 0x7f441415a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:07.952 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:07.950+0000 7f441c7bd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f44140d5200 0x7f44140ab210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:07.952 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:07.950+0000 7f441c7bd640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f441415eb90 0x7f44141e8910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:07.952 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:07.950+0000 7f441c7bd640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f44140d3500 con 0x7f44140b91a0 2024-09-12T03:23:07.952 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:07.950+0000 7f441c7bd640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f44140d42e0 con 0x7f44140d5200 2024-09-12T03:23:07.952 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:07.950+0000 7f441c7bd640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f441415ef70 con 0x7f441415eb90 2024-09-12T03:23:07.952 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:07.951+0000 7f441a559640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f44140b91a0 0x7f441415a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:07.952 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:07.951+0000 7f441a559640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f44140b91a0 0x7f441415a1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59900/0 (socket says 172.21.15.27:59900) 2024-09-12T03:23:07.952 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:07.951+0000 7f441a559640 1 -- 172.21.15.27:0/1500792560 learned_addr learned my addr 172.21.15.27:0/1500792560 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:23:07.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:07.951+0000 7f441ad5a640 1 --2- 172.21.15.27:0/1500792560 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f441415eb90 0x7f44141e8910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:07.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:07.951+0000 7f4419d58640 1 --2- 172.21.15.27:0/1500792560 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f44140d5200 0x7f44140ab210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:07.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:07.951+0000 7f441a559640 1 -- 172.21.15.27:0/1500792560 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f441415eb90 msgr2=0x7f44141e8910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:07.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:07.951+0000 7f441a559640 1 --2- 172.21.15.27:0/1500792560 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f441415eb90 0x7f44141e8910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:07.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:07.951+0000 7f441a559640 1 -- 172.21.15.27:0/1500792560 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f44140d5200 msgr2=0x7f44140ab210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:07.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:07.951+0000 7f441a559640 1 --2- 172.21.15.27:0/1500792560 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f44140d5200 0x7f44140ab210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:07.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:07.951+0000 7f441a559640 1 -- 172.21.15.27:0/1500792560 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44140cdd40 con 0x7f44140b91a0 2024-09-12T03:23:07.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:07.951+0000 7f441ad5a640 1 --2- 172.21.15.27:0/1500792560 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f441415eb90 0x7f44141e8910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:23:07.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:07.951+0000 7f441a559640 1 --2- 172.21.15.27:0/1500792560 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f44140b91a0 0x7f441415a1e0 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7f440c00c3f0 tx=0x7f440c00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:07.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:07.951+0000 7f4412ffd640 1 -- 172.21.15.27:0/1500792560 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f440c0074e0 con 0x7f44140b91a0 2024-09-12T03:23:07.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:07.951+0000 7f4412ffd640 1 -- 172.21.15.27:0/1500792560 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f440c0047e0 con 0x7f44140b91a0 2024-09-12T03:23:07.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:07.951+0000 7f441c7bd640 1 -- 172.21.15.27:0/1500792560 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44140ce020 con 0x7f44140b91a0 2024-09-12T03:23:07.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:07.951+0000 7f4412ffd640 1 -- 172.21.15.27:0/1500792560 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f440c004bd0 con 0x7f44140b91a0 2024-09-12T03:23:07.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:07.951+0000 7f441c7bd640 1 -- 172.21.15.27:0/1500792560 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f441415a720 con 0x7f44140b91a0 2024-09-12T03:23:07.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:07.952+0000 7f441c7bd640 1 -- 172.21.15.27:0/1500792560 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44141e8410 con 0x7f44140b91a0 2024-09-12T03:23:07.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:07.953+0000 7f4412ffd640 1 -- 172.21.15.27:0/1500792560 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 23) ==== 91110+0+0 (secure 0 0 0) 0x7f440c011070 con 0x7f44140b91a0 2024-09-12T03:23:07.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:07.953+0000 7f4412ffd640 1 --2- 172.21.15.27:0/1500792560 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f440006c4f0 0x7f440006e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:07.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:07.953+0000 7f4412ffd640 1 -- 172.21.15.27:0/1500792560 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 5765+0+0 (secure 0 0 0) 0x7f440c08b530 con 0x7f44140b91a0 2024-09-12T03:23:07.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:07.953+0000 7f4419d58640 1 --2- 172.21.15.27:0/1500792560 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f440006c4f0 0x7f440006e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:07.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:07.954+0000 7f4419d58640 1 --2- 172.21.15.27:0/1500792560 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f440006c4f0 0x7f440006e9b0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f44140cab30 tx=0x7f4408009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:07.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:07.958+0000 7f4412ffd640 1 -- 172.21.15.27:0/1500792560 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f440c01a2b0 con 0x7f44140b91a0 2024-09-12T03:23:08.145 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:08.145+0000 7f441c7bd640 1 -- 172.21.15.27:0/1500792560 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f44140d2df0 con 0x7f44140b91a0 2024-09-12T03:23:08.145 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:08.145+0000 7f4412ffd640 1 -- 172.21.15.27:0/1500792560 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v138) ==== 80+0+0 (secure 0 0 0) 0x7f440c0596a0 con 0x7f44140b91a0 2024-09-12T03:23:08.145 INFO:teuthology.orchestra.run.smithi027.stderr:listed 0 entries 2024-09-12T03:23:08.148 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:08.147+0000 7f441c7bd640 1 -- 172.21.15.27:0/1500792560 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f440006c4f0 msgr2=0x7f440006e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:08.148 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:08.147+0000 7f441c7bd640 1 --2- 172.21.15.27:0/1500792560 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f440006c4f0 0x7f440006e9b0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f44140cab30 tx=0x7f4408009040 comp rx=0 tx=0).stop 2024-09-12T03:23:08.148 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:08.147+0000 7f441c7bd640 1 -- 172.21.15.27:0/1500792560 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f44140b91a0 msgr2=0x7f441415a1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:08.148 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:08.147+0000 7f441c7bd640 1 --2- 172.21.15.27:0/1500792560 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f44140b91a0 0x7f441415a1e0 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7f440c00c3f0 tx=0x7f440c00c8b0 comp rx=0 tx=0).stop 2024-09-12T03:23:08.148 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:08.147+0000 7f441c7bd640 1 -- 172.21.15.27:0/1500792560 shutdown_connections 2024-09-12T03:23:08.148 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:08.147+0000 7f441c7bd640 1 --2- 172.21.15.27:0/1500792560 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f441415eb90 0x7f44141e8910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:08.148 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:08.147+0000 7f441c7bd640 1 --2- 172.21.15.27:0/1500792560 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f44140d5200 0x7f44140ab210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:08.148 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:08.147+0000 7f441c7bd640 1 --2- 172.21.15.27:0/1500792560 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f440006c4f0 0x7f440006e9b0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:08.148 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:08.147+0000 7f441c7bd640 1 --2- 172.21.15.27:0/1500792560 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f44140b91a0 0x7f441415a1e0 unknown :-1 s=CLOSED pgs=1464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:08.148 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:08.147+0000 7f441c7bd640 1 -- 172.21.15.27:0/1500792560 >> 172.21.15.27:0/1500792560 conn(0x7f441406dce0 msgr2=0x7f44140d1010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:08.148 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:08.147+0000 7f441c7bd640 1 -- 172.21.15.27:0/1500792560 shutdown_connections 2024-09-12T03:23:08.148 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:08.147+0000 7f441c7bd640 1 -- 172.21.15.27:0/1500792560 wait complete. 2024-09-12T03:23:08.158 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-09-12T03:23:08.158 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:23:08.158 DEBUG:teuthology.orchestra.run.smithi045:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-09-12T03:23:08.193 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi045.front.sepia.ceph.com 2024-09-12T03:23:08.194 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:23:08.194 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-12T03:23:08.252 INFO:tasks.cephfs.mount:Terminating background process 2024-09-12T03:23:08.252 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-12T03:23:08.252 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T03:23:08.269 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056004,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497432k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16324 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248716k,nr_inodes=812179,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497432k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-12T03:23:08.269 INFO:tasks.cephfs.mount:Terminating background process 2024-09-12T03:23:08.269 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-12T03:23:08.269 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T03:23:08.310 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056795,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16602 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249348k,nr_inodes=812337,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-12T03:23:08.311 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-12T03:23:08.413 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:08.413+0000 7fd1d771a640 1 Processor -- start 2024-09-12T03:23:08.413 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:08.413+0000 7fd1d771a640 1 -- start start 2024-09-12T03:23:08.414 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:08.414+0000 7fd1d771a640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fd1d0072d70 0x7fd1d00b3560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:08.414 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:08.414+0000 7fd1d771a640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd1d00ce410 con 0x7fd1d015b2b0 2024-09-12T03:23:08.414 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:08.414+0000 7fd1d771a640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd1d00732c0 con 0x7fd1d006e9b0 2024-09-12T03:23:08.414 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:08.414+0000 7fd1d771a640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7fd1d00720a0 con 0x7fd1d0072d70 2024-09-12T03:23:08.415 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:08.414+0000 7fd1d54b6640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fd1d015b2b0 0x7fd1d0072430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:54168/0 (socket says 172.21.15.27:54168) 2024-09-12T03:23:08.415 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:08.414+0000 7fd1d54b6640 1 -- 172.21.15.27:0/3578598174 learned_addr learned my addr 172.21.15.27:0/3578598174 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:23:08.415 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:08.414+0000 7fd1d4cb5640 1 --2- 172.21.15.27:0/3578598174 >> v2:172.21.15.27:3300/0 conn(0x7fd1d0072d70 0x7fd1d00b3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:08.415 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:08.414+0000 7fd1c7fff640 1 -- 172.21.15.27:0/3578598174 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3320013242 0 0) 0x7fd1cc001110 con 0x7fd1d015b2b0 2024-09-12T03:23:08.415 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:08.414+0000 7fd1c7fff640 1 -- 172.21.15.27:0/3578598174 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd1bc003520 con 0x7fd1d015b2b0 2024-09-12T03:23:08.415 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:08.414+0000 7fd1d4cb5640 1 -- 172.21.15.27:0/3578598174 >> v1:172.21.15.45:6789/0 conn(0x7fd1d006e9b0 legacy=0x7fd1d00cebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:08.415 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:08.414+0000 7fd1d4cb5640 1 -- 172.21.15.27:0/3578598174 >> v1:172.21.15.27:6789/0 conn(0x7fd1d015b2b0 legacy=0x7fd1d0072430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:08.415 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:08.414+0000 7fd1d4cb5640 1 -- 172.21.15.27:0/3578598174 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1d0071ae0 con 0x7fd1d0072d70 2024-09-12T03:23:08.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:08.415+0000 7fd1d4cb5640 1 --2- 172.21.15.27:0/3578598174 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd1d0072d70 0x7fd1d00b3560 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7fd1c00097a0 tx=0x7fd1c002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47600847c6004afa server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:08.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:08.415+0000 7fd1c7fff640 1 -- 172.21.15.27:0/3578598174 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1c0037070 con 0x7fd1d0072d70 2024-09-12T03:23:08.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:08.415+0000 7fd1c7fff640 1 -- 172.21.15.27:0/3578598174 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1c0032440 con 0x7fd1d0072d70 2024-09-12T03:23:08.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:08.415+0000 7fd1c7fff640 1 -- 172.21.15.27:0/3578598174 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1c00327f0 con 0x7fd1d0072d70 2024-09-12T03:23:08.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:08.416+0000 7fd1d771a640 1 -- 172.21.15.27:0/3578598174 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd1d0072d70 msgr2=0x7fd1d00b3560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:08.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:08.416+0000 7fd1d771a640 1 --2- 172.21.15.27:0/3578598174 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd1d0072d70 0x7fd1d00b3560 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7fd1c00097a0 tx=0x7fd1c002a1e0 comp rx=0 tx=0).stop 2024-09-12T03:23:08.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:08.416+0000 7fd1d771a640 1 -- 172.21.15.27:0/3578598174 shutdown_connections 2024-09-12T03:23:08.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:08.416+0000 7fd1d771a640 1 --2- 172.21.15.27:0/3578598174 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd1d0072d70 0x7fd1d00b3560 unknown :-1 s=CLOSED pgs=1466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:08.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:08.416+0000 7fd1d771a640 1 -- 172.21.15.27:0/3578598174 >> 172.21.15.27:0/3578598174 conn(0x7fd1d0062c60 msgr2=0x7fd1d006bb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:08.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:08.416+0000 7fd1d771a640 1 -- 172.21.15.27:0/3578598174 shutdown_connections 2024-09-12T03:23:08.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:08.416+0000 7fd1d771a640 1 -- 172.21.15.27:0/3578598174 wait complete. 2024-09-12T03:23:08.419 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:08.417+0000 7fd1d771a640 1 Processor -- start 2024-09-12T03:23:08.419 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:08.417+0000 7fd1d771a640 1 -- start start 2024-09-12T03:23:08.419 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:08.418+0000 7fd1d771a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd1d006e9b0 0x7fd1d014da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:08.419 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:08.418+0000 7fd1d771a640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd1d0072d70 0x7fd1d014d280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:08.419 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:08.418+0000 7fd1d771a640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fd1d015b2b0 0x7fd1d015c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:08.420 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:08.418+0000 7fd1d771a640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fd1d015b690 con 0x7fd1d0072d70 2024-09-12T03:23:08.420 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:08.418+0000 7fd1d771a640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fd1d00ce690 con 0x7fd1d006e9b0 2024-09-12T03:23:08.420 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:08.418+0000 7fd1d771a640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fd1d015b810 con 0x7fd1d015b2b0 2024-09-12T03:23:08.420 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:08.418+0000 7fd1d4cb5640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd1d0072d70 0x7fd1d014d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:08.420 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:08.418+0000 7fd1d4cb5640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd1d0072d70 0x7fd1d014d280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59904/0 (socket says 172.21.15.27:59904) 2024-09-12T03:23:08.420 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:08.418+0000 7fd1d4cb5640 1 -- 172.21.15.27:0/2013797955 learned_addr learned my addr 172.21.15.27:0/2013797955 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:23:08.420 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:08.418+0000 7fd1d54b6640 1 --2- 172.21.15.27:0/2013797955 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd1d006e9b0 0x7fd1d014da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:08.420 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:08.418+0000 7fd1d5cb7640 1 --2- 172.21.15.27:0/2013797955 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fd1d015b2b0 0x7fd1d015c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:08.420 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:08.418+0000 7fd1d4cb5640 1 -- 172.21.15.27:0/2013797955 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fd1d015b2b0 msgr2=0x7fd1d015c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:08.420 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:08.418+0000 7fd1d4cb5640 1 --2- 172.21.15.27:0/2013797955 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fd1d015b2b0 0x7fd1d015c8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:08.420 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:08.418+0000 7fd1d4cb5640 1 -- 172.21.15.27:0/2013797955 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd1d006e9b0 msgr2=0x7fd1d014da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:08.420 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:08.418+0000 7fd1d4cb5640 1 --2- 172.21.15.27:0/2013797955 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd1d006e9b0 0x7fd1d014da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:08.421 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:08.418+0000 7fd1d4cb5640 1 -- 172.21.15.27:0/2013797955 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1d014d7c0 con 0x7fd1d0072d70 2024-09-12T03:23:08.421 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:08.418+0000 7fd1d5cb7640 1 --2- 172.21.15.27:0/2013797955 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fd1d015b2b0 0x7fd1d015c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:23:08.422 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:08.418+0000 7fd1d54b6640 1 --2- 172.21.15.27:0/2013797955 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd1d006e9b0 0x7fd1d014da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:23:08.422 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:08.418+0000 7fd1d4cb5640 1 --2- 172.21.15.27:0/2013797955 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd1d0072d70 0x7fd1d014d280 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7fd1c002a220 tx=0x7fd1c00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:08.422 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:08.418+0000 7fd1c5ffb640 1 -- 172.21.15.27:0/2013797955 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1c0037040 con 0x7fd1d0072d70 2024-09-12T03:23:08.422 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:08.419+0000 7fd1c5ffb640 1 -- 172.21.15.27:0/2013797955 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1c0042070 con 0x7fd1d0072d70 2024-09-12T03:23:08.422 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:08.419+0000 7fd1d771a640 1 -- 172.21.15.27:0/2013797955 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1d015bb20 con 0x7fd1d0072d70 2024-09-12T03:23:08.422 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:08.419+0000 7fd1c5ffb640 1 -- 172.21.15.27:0/2013797955 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1c0006e10 con 0x7fd1d0072d70 2024-09-12T03:23:08.422 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:08.419+0000 7fd1d771a640 1 -- 172.21.15.27:0/2013797955 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1d015c0e0 con 0x7fd1d0072d70 2024-09-12T03:23:08.423 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:08.420+0000 7fd1d771a640 1 -- 172.21.15.27:0/2013797955 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1d00b3560 con 0x7fd1d0072d70 2024-09-12T03:23:08.423 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:08.420+0000 7fd1c5ffb640 1 -- 172.21.15.27:0/2013797955 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 24) ==== 91212+0+0 (secure 0 0 0) 0x7fd1c0032830 con 0x7fd1d0072d70 2024-09-12T03:23:08.423 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:08.421+0000 7fd1c5ffb640 1 --2- 172.21.15.27:0/2013797955 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fd1b406c610 0x7fd1b406ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:08.423 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:08.421+0000 7fd1d54b6640 1 --2- 172.21.15.27:0/2013797955 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fd1b406c610 0x7fd1b406ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:08.423 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:08.421+0000 7fd1c5ffb640 1 -- 172.21.15.27:0/2013797955 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 5765+0+0 (secure 0 0 0) 0x7fd1c00aa180 con 0x7fd1d0072d70 2024-09-12T03:23:08.423 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:08.421+0000 7fd1d54b6640 1 --2- 172.21.15.27:0/2013797955 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fd1b406c610 0x7fd1b406ead0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fd1cc0012b0 tx=0x7fd1cc009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:08.425 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:08.425+0000 7fd1c5ffb640 1 -- 172.21.15.27:0/2013797955 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fd1c007a420 con 0x7fd1d0072d70 2024-09-12T03:23:08.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:08.636+0000 7fd1d771a640 1 -- 172.21.15.27:0/2013797955 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd1d00cca90 con 0x7fd1d0072d70 2024-09-12T03:23:08.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:08.636+0000 7fd1c5ffb640 1 -- 172.21.15.27:0/2013797955 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 83 v83) ==== 94+0+5224 (secure 0 0 0) 0x7fd1c0079d20 con 0x7fd1d0072d70 2024-09-12T03:23:08.636 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:23:08.637 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":83,"btime":"2024-09-12T03:22:58:967052+0000","default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6689,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/2285501348","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2285501348},{"type":"v1","addr":"172.21.15.45:6833","nonce":2285501348}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79},{"gid":7453,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/1872420883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":1872420883},{"type":"v1","addr":"172.21.15.45:6835","nonce":1872420883}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79},{"gid":8193,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6834/3968488955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":3968488955},{"type":"v1","addr":"172.21.15.27:6834","nonce":3968488955}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79}],"filesystems":[{"mdsmap":{"epoch":82,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-12T03:22:50.200339+0000","modified":"2024-09-12T03:22:52.204820+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8196},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8196":{"gid":8196,"name":"a","rank":0,"incarnation":80,"state":"up:active","state_seq":3,"addr":"172.21.15.27:6835/389118996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":389118996},{"type":"v1","addr":"172.21.15.27:6835","nonce":389118996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8196,"qdb_cluster":[8196]},"id":11}]} 2024-09-12T03:23:08.637 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 83 2024-09-12T03:23:08.639 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:08.639+0000 7fd1d771a640 1 -- 172.21.15.27:0/2013797955 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fd1b406c610 msgr2=0x7fd1b406ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:08.639 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:08.639+0000 7fd1d771a640 1 --2- 172.21.15.27:0/2013797955 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fd1b406c610 0x7fd1b406ead0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fd1cc0012b0 tx=0x7fd1cc009040 comp rx=0 tx=0).stop 2024-09-12T03:23:08.639 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:08.639+0000 7fd1d771a640 1 -- 172.21.15.27:0/2013797955 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd1d0072d70 msgr2=0x7fd1d014d280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:08.639 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:08.639+0000 7fd1d771a640 1 --2- 172.21.15.27:0/2013797955 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd1d0072d70 0x7fd1d014d280 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7fd1c002a220 tx=0x7fd1c00046c0 comp rx=0 tx=0).stop 2024-09-12T03:23:08.639 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:08.639+0000 7fd1d771a640 1 -- 172.21.15.27:0/2013797955 shutdown_connections 2024-09-12T03:23:08.639 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:08.639+0000 7fd1d771a640 1 --2- 172.21.15.27:0/2013797955 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fd1d015b2b0 0x7fd1d015c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:08.639 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:08.639+0000 7fd1d771a640 1 --2- 172.21.15.27:0/2013797955 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fd1b406c610 0x7fd1b406ead0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:08.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:08.639+0000 7fd1d771a640 1 --2- 172.21.15.27:0/2013797955 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd1d0072d70 0x7fd1d014d280 unknown :-1 s=CLOSED pgs=1467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:08.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:08.639+0000 7fd1d771a640 1 --2- 172.21.15.27:0/2013797955 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd1d006e9b0 0x7fd1d014da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:08.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:08.639+0000 7fd1d771a640 1 -- 172.21.15.27:0/2013797955 >> 172.21.15.27:0/2013797955 conn(0x7fd1d0062c60 msgr2=0x7fd1d01574b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:08.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:08.639+0000 7fd1d771a640 1 -- 172.21.15.27:0/2013797955 shutdown_connections 2024-09-12T03:23:08.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:08.639+0000 7fd1d771a640 1 -- 172.21.15.27:0/2013797955 wait complete. 2024-09-12T03:23:08.650 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-12T03:23:08.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:08.752+0000 7f6b230a4640 1 Processor -- start 2024-09-12T03:23:08.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:08.752+0000 7f6b230a4640 1 -- start start 2024-09-12T03:23:08.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:08.753+0000 7f6b230a4640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f6b1c0cef50 0x7f6b1c0cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:08.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:08.753+0000 7f6b230a4640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f6b1c0d0b50 0x7f6b1c0d0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:08.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:08.753+0000 7f6b230a4640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6b1c06fa00 con 0x7f6b1c070e80 2024-09-12T03:23:08.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:08.753+0000 7f6b230a4640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f6b1c15ca90 con 0x7f6b1c0cef50 2024-09-12T03:23:08.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:08.753+0000 7f6b230a4640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f6b1c0cf870 con 0x7f6b1c0d0b50 2024-09-12T03:23:08.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:08.753+0000 7f6b20e40640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f6b1c0cef50 0x7f6b1c0cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:08.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:08.753+0000 7f6b21641640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f6b1c0d0b50 0x7f6b1c0d0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:08.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:08.753+0000 7f6b1bfff640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f6b1c070e80 0x7f6b1c0d0440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.27:51578/0 (socket says 172.21.15.27:51578) 2024-09-12T03:23:08.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:08.753+0000 7f6b1bfff640 1 -- 172.21.15.27:0/1054685441 learned_addr learned my addr 172.21.15.27:0/1054685441 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:23:08.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:08.754+0000 7f6b1b7fe640 1 -- 172.21.15.27:0/1054685441 <== mon.1 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 780913725 0 0) 0x7f6b1c06fa00 con 0x7f6b1c070e80 2024-09-12T03:23:08.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:08.754+0000 7f6b1b7fe640 1 -- 172.21.15.27:0/1054685441 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6b0c003520 con 0x7f6b1c070e80 2024-09-12T03:23:08.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:08.754+0000 7f6b20e40640 1 -- 172.21.15.27:0/1054685441 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f6b1c0d0b50 msgr2=0x7f6b1c0d0fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:08.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:08.754+0000 7f6b20e40640 1 --2- 172.21.15.27:0/1054685441 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f6b1c0d0b50 0x7f6b1c0d0fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:08.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:08.754+0000 7f6b20e40640 1 -- 172.21.15.27:0/1054685441 >> v1:172.21.15.45:6789/0 conn(0x7f6b1c070e80 legacy=0x7f6b1c0d0440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:08.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:08.754+0000 7f6b20e40640 1 -- 172.21.15.27:0/1054685441 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b1c06f720 con 0x7f6b1c0cef50 2024-09-12T03:23:08.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:08.754+0000 7f6b21641640 1 --2- 172.21.15.27:0/1054685441 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f6b1c0d0b50 0x7f6b1c0d0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:23:08.756 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:08.754+0000 7f6b20e40640 1 --2- 172.21.15.27:0/1054685441 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6b1c0cef50 0x7f6b1c0cf330 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f6b100097a0 tx=0x7f6b1002a060 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f8525b066c66743b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:08.756 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:08.755+0000 7f6b1b7fe640 1 -- 172.21.15.27:0/1054685441 <== mon.1 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b10037070 con 0x7f6b1c0cef50 2024-09-12T03:23:08.756 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:08.755+0000 7f6b1b7fe640 1 -- 172.21.15.27:0/1054685441 <== mon.1 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b1002adb0 con 0x7f6b1c0cef50 2024-09-12T03:23:08.756 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:08.755+0000 7f6b1b7fe640 1 -- 172.21.15.27:0/1054685441 <== mon.1 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b10032660 con 0x7f6b1c0cef50 2024-09-12T03:23:08.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:08.755+0000 7f6b230a4640 1 -- 172.21.15.27:0/1054685441 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6b1c0cef50 msgr2=0x7f6b1c0cf330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:08.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:08.755+0000 7f6b230a4640 1 --2- 172.21.15.27:0/1054685441 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6b1c0cef50 0x7f6b1c0cf330 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f6b100097a0 tx=0x7f6b1002a060 comp rx=0 tx=0).stop 2024-09-12T03:23:08.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:08.755+0000 7f6b230a4640 1 -- 172.21.15.27:0/1054685441 shutdown_connections 2024-09-12T03:23:08.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:08.755+0000 7f6b230a4640 1 --2- 172.21.15.27:0/1054685441 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f6b1c0d0b50 0x7f6b1c0d0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:08.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:08.755+0000 7f6b230a4640 1 --2- 172.21.15.27:0/1054685441 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6b1c0cef50 0x7f6b1c0cf330 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:08.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:08.755+0000 7f6b230a4640 1 -- 172.21.15.27:0/1054685441 >> 172.21.15.27:0/1054685441 conn(0x7f6b1c073800 msgr2=0x7f6b1c09ba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:08.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:08.755+0000 7f6b230a4640 1 -- 172.21.15.27:0/1054685441 shutdown_connections 2024-09-12T03:23:08.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:08.755+0000 7f6b230a4640 1 -- 172.21.15.27:0/1054685441 wait complete. 2024-09-12T03:23:08.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:08.756+0000 7f6b230a4640 1 Processor -- start 2024-09-12T03:23:08.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:08.757+0000 7f6b230a4640 1 -- start start 2024-09-12T03:23:08.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:08.757+0000 7f6b230a4640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6b1c070e80 0x7f6b1c14fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:08.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:08.757+0000 7f6b230a4640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f6b1c0cef50 0x7f6b1c1f5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:08.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:08.757+0000 7f6b230a4640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6b1c0d0b50 0x7f6b1c1f56c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:08.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:08.757+0000 7f6b230a4640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f6b1c06f360 con 0x7f6b1c070e80 2024-09-12T03:23:08.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:08.757+0000 7f6b230a4640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f6b1c0d2d10 con 0x7f6b1c0d0b50 2024-09-12T03:23:08.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:08.757+0000 7f6b230a4640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f6b1c06f4e0 con 0x7f6b1c0cef50 2024-09-12T03:23:08.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:08.758+0000 7f6b20e40640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6b1c070e80 0x7f6b1c14fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:08.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:08.758+0000 7f6b20e40640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6b1c070e80 0x7f6b1c14fd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59906/0 (socket says 172.21.15.27:59906) 2024-09-12T03:23:08.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:08.758+0000 7f6b20e40640 1 -- 172.21.15.27:0/2752352645 learned_addr learned my addr 172.21.15.27:0/2752352645 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:23:08.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:08.758+0000 7f6b21641640 1 --2- 172.21.15.27:0/2752352645 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6b1c0d0b50 0x7f6b1c1f56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:08.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:08.758+0000 7f6b1bfff640 1 --2- 172.21.15.27:0/2752352645 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f6b1c0cef50 0x7f6b1c1f5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:08.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:08.758+0000 7f6b20e40640 1 -- 172.21.15.27:0/2752352645 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f6b1c0cef50 msgr2=0x7f6b1c1f5180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:08.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:08.758+0000 7f6b20e40640 1 --2- 172.21.15.27:0/2752352645 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f6b1c0cef50 0x7f6b1c1f5180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:08.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:08.758+0000 7f6b20e40640 1 -- 172.21.15.27:0/2752352645 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6b1c0d0b50 msgr2=0x7f6b1c1f56c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:08.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:08.758+0000 7f6b20e40640 1 --2- 172.21.15.27:0/2752352645 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6b1c0d0b50 0x7f6b1c1f56c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:08.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:08.758+0000 7f6b20e40640 1 -- 172.21.15.27:0/2752352645 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b1c155e70 con 0x7f6b1c070e80 2024-09-12T03:23:08.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:08.758+0000 7f6b1bfff640 1 --2- 172.21.15.27:0/2752352645 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f6b1c0cef50 0x7f6b1c1f5180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:23:08.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:08.758+0000 7f6b21641640 1 --2- 172.21.15.27:0/2752352645 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6b1c0d0b50 0x7f6b1c1f56c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:23:08.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:08.758+0000 7f6b20e40640 1 --2- 172.21.15.27:0/2752352645 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6b1c070e80 0x7f6b1c14fd40 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7f6b10035040 tx=0x7f6b100046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:08.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:08.758+0000 7f6b197fa640 1 -- 172.21.15.27:0/2752352645 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b10041070 con 0x7f6b1c070e80 2024-09-12T03:23:08.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:08.758+0000 7f6b197fa640 1 -- 172.21.15.27:0/2752352645 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b1003c460 con 0x7f6b1c070e80 2024-09-12T03:23:08.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:08.758+0000 7f6b230a4640 1 -- 172.21.15.27:0/2752352645 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b1c15d070 con 0x7f6b1c070e80 2024-09-12T03:23:08.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:08.758+0000 7f6b197fa640 1 -- 172.21.15.27:0/2752352645 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b10037040 con 0x7f6b1c070e80 2024-09-12T03:23:08.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:08.758+0000 7f6b230a4640 1 -- 172.21.15.27:0/2752352645 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b1c15d630 con 0x7f6b1c070e80 2024-09-12T03:23:08.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:08.759+0000 7f6b230a4640 1 -- 172.21.15.27:0/2752352645 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b1c0b6980 con 0x7f6b1c070e80 2024-09-12T03:23:08.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:08.760+0000 7f6b197fa640 1 -- 172.21.15.27:0/2752352645 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 24) ==== 91212+0+0 (secure 0 0 0) 0x7f6b1003a070 con 0x7f6b1c070e80 2024-09-12T03:23:08.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:08.760+0000 7f6b197fa640 1 --2- 172.21.15.27:0/2752352645 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f6b0006c610 0x7f6b0006ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:08.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:08.760+0000 7f6b197fa640 1 -- 172.21.15.27:0/2752352645 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 5765+0+0 (secure 0 0 0) 0x7f6b100a9a40 con 0x7f6b1c070e80 2024-09-12T03:23:08.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:08.760+0000 7f6b1bfff640 1 --2- 172.21.15.27:0/2752352645 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f6b0006c610 0x7f6b0006ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:08.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:08.761+0000 7f6b1bfff640 1 --2- 172.21.15.27:0/2752352645 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f6b0006c610 0x7f6b0006ead0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f6b08007fd0 tx=0x7f6b08009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:08.764 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:08.765+0000 7f6b197fa640 1 -- 172.21.15.27:0/2752352645 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f6b1c0b6980 con 0x7f6b1c070e80 2024-09-12T03:23:08.981 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:08.980+0000 7f6b230a4640 1 -- 172.21.15.27:0/2752352645 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6b1c0712c0 con 0x7f6b1c070e80 2024-09-12T03:23:08.981 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:08.981+0000 7f6b197fa640 1 -- 172.21.15.27:0/2752352645 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 83 v83) ==== 94+0+5224 (secure 0 0 0) 0x7f6b10079ce0 con 0x7f6b1c070e80 2024-09-12T03:23:08.981 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:23:08.981 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":83,"btime":"2024-09-12T03:22:58:967052+0000","default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6689,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/2285501348","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2285501348},{"type":"v1","addr":"172.21.15.45:6833","nonce":2285501348}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79},{"gid":7453,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/1872420883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":1872420883},{"type":"v1","addr":"172.21.15.45:6835","nonce":1872420883}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79},{"gid":8193,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6834/3968488955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":3968488955},{"type":"v1","addr":"172.21.15.27:6834","nonce":3968488955}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79}],"filesystems":[{"mdsmap":{"epoch":82,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-12T03:22:50.200339+0000","modified":"2024-09-12T03:22:52.204820+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8196},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8196":{"gid":8196,"name":"a","rank":0,"incarnation":80,"state":"up:active","state_seq":3,"addr":"172.21.15.27:6835/389118996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":389118996},{"type":"v1","addr":"172.21.15.27:6835","nonce":389118996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8196,"qdb_cluster":[8196]},"id":11}]} 2024-09-12T03:23:08.982 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 83 2024-09-12T03:23:08.983 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:08.983+0000 7f6b230a4640 1 -- 172.21.15.27:0/2752352645 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f6b0006c610 msgr2=0x7f6b0006ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:08.983 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:08.983+0000 7f6b230a4640 1 --2- 172.21.15.27:0/2752352645 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f6b0006c610 0x7f6b0006ead0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f6b08007fd0 tx=0x7f6b08009040 comp rx=0 tx=0).stop 2024-09-12T03:23:08.984 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:08.983+0000 7f6b230a4640 1 -- 172.21.15.27:0/2752352645 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6b1c070e80 msgr2=0x7f6b1c14fd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:08.984 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:08.983+0000 7f6b230a4640 1 --2- 172.21.15.27:0/2752352645 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6b1c070e80 0x7f6b1c14fd40 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7f6b10035040 tx=0x7f6b100046c0 comp rx=0 tx=0).stop 2024-09-12T03:23:08.984 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:08.983+0000 7f6b230a4640 1 -- 172.21.15.27:0/2752352645 shutdown_connections 2024-09-12T03:23:08.984 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:08.983+0000 7f6b230a4640 1 --2- 172.21.15.27:0/2752352645 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6b1c0d0b50 0x7f6b1c1f56c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:08.984 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:08.983+0000 7f6b230a4640 1 --2- 172.21.15.27:0/2752352645 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f6b1c0cef50 0x7f6b1c1f5180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:08.984 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:08.983+0000 7f6b230a4640 1 --2- 172.21.15.27:0/2752352645 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f6b0006c610 0x7f6b0006ead0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:08.984 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:08.983+0000 7f6b230a4640 1 --2- 172.21.15.27:0/2752352645 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6b1c070e80 0x7f6b1c14fd40 unknown :-1 s=CLOSED pgs=1468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:08.984 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:08.983+0000 7f6b230a4640 1 -- 172.21.15.27:0/2752352645 >> 172.21.15.27:0/2752352645 conn(0x7f6b1c073800 msgr2=0x7f6b1c0b5cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:08.984 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:08.984+0000 7f6b230a4640 1 -- 172.21.15.27:0/2752352645 shutdown_connections 2024-09-12T03:23:08.984 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:08.984+0000 7f6b230a4640 1 -- 172.21.15.27:0/2752352645 wait complete. 2024-09-12T03:23:08.994 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-12T03:23:09.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.097+0000 7f52d8df1640 1 Processor -- start 2024-09-12T03:23:09.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.097+0000 7f52d8df1640 1 -- start start 2024-09-12T03:23:09.098 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.098+0000 7f52d8df1640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f52d4071be0 0x7f52d406e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:09.098 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.098+0000 7f52d8df1640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52d406daf0 con 0x7f52d415aca0 2024-09-12T03:23:09.098 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.098+0000 7f52d8df1640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52d406ddd0 con 0x7f52d415a8c0 2024-09-12T03:23:09.098 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.098+0000 7f52d8df1640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f52d406d8a0 con 0x7f52d4071be0 2024-09-12T03:23:09.099 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.098+0000 7f52d2d9d640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f52d415aca0 0x7f52d40b4710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:54172/0 (socket says 172.21.15.27:54172) 2024-09-12T03:23:09.099 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.098+0000 7f52d2d9d640 1 -- 172.21.15.27:0/2181372207 learned_addr learned my addr 172.21.15.27:0/2181372207 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:23:09.099 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.098+0000 7f52d1d9b640 1 --2- 172.21.15.27:0/2181372207 >> v2:172.21.15.45:3300/0 conn(0x7f52d4071be0 0x7f52d406e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:09.099 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.098+0000 7f52d159a640 1 -- 172.21.15.27:0/2181372207 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3751353459 0 0) 0x7f52d406daf0 con 0x7f52d415aca0 2024-09-12T03:23:09.099 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.098+0000 7f52d159a640 1 -- 172.21.15.27:0/2181372207 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f52bc003520 con 0x7f52d415aca0 2024-09-12T03:23:09.099 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.098+0000 7f52d159a640 1 -- 172.21.15.27:0/2181372207 <== mon.1 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2283877129 0 0) 0x7f52d406ddd0 con 0x7f52d415a8c0 2024-09-12T03:23:09.099 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.098+0000 7f52d159a640 1 -- 172.21.15.27:0/2181372207 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f52d406daf0 con 0x7f52d415a8c0 2024-09-12T03:23:09.099 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.098+0000 7f52d1d9b640 1 -- 172.21.15.27:0/2181372207 >> v1:172.21.15.45:6789/0 conn(0x7f52d415a8c0 legacy=0x7f52d40cdb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:09.099 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.098+0000 7f52d1d9b640 1 -- 172.21.15.27:0/2181372207 >> v1:172.21.15.27:6789/0 conn(0x7f52d415aca0 legacy=0x7f52d40b4710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:09.100 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.098+0000 7f52d1d9b640 1 -- 172.21.15.27:0/2181372207 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52d406e0b0 con 0x7f52d4071be0 2024-09-12T03:23:09.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.099+0000 7f52d1d9b640 1 --2- 172.21.15.27:0/2181372207 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f52d4071be0 0x7f52d406e8b0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f52c80097a0 tx=0x7f52c802a230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2b124aa54f7cb22c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:09.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.099+0000 7f52d159a640 1 -- 172.21.15.27:0/2181372207 <== mon.1 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52c8037070 con 0x7f52d4071be0 2024-09-12T03:23:09.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.099+0000 7f52d159a640 1 -- 172.21.15.27:0/2181372207 <== mon.1 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52c8032440 con 0x7f52d4071be0 2024-09-12T03:23:09.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.099+0000 7f52d159a640 1 -- 172.21.15.27:0/2181372207 <== mon.1 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52c80327f0 con 0x7f52d4071be0 2024-09-12T03:23:09.102 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.100+0000 7f52d8df1640 1 -- 172.21.15.27:0/2181372207 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f52d4071be0 msgr2=0x7f52d406e8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:09.102 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.100+0000 7f52d8df1640 1 --2- 172.21.15.27:0/2181372207 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f52d4071be0 0x7f52d406e8b0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f52c80097a0 tx=0x7f52c802a230 comp rx=0 tx=0).stop 2024-09-12T03:23:09.102 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.100+0000 7f52d8df1640 1 -- 172.21.15.27:0/2181372207 shutdown_connections 2024-09-12T03:23:09.102 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.100+0000 7f52d8df1640 1 --2- 172.21.15.27:0/2181372207 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f52d4071be0 0x7f52d406e8b0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:09.102 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.100+0000 7f52d8df1640 1 -- 172.21.15.27:0/2181372207 >> 172.21.15.27:0/2181372207 conn(0x7f52d4062c60 msgr2=0x7f52d406bb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:09.102 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.100+0000 7f52d8df1640 1 -- 172.21.15.27:0/2181372207 shutdown_connections 2024-09-12T03:23:09.102 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.100+0000 7f52d8df1640 1 -- 172.21.15.27:0/2181372207 wait complete. 2024-09-12T03:23:09.102 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.101+0000 7f52d8df1640 1 Processor -- start 2024-09-12T03:23:09.102 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.101+0000 7f52d8df1640 1 -- start start 2024-09-12T03:23:09.105 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.102+0000 7f52d8df1640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f52d4071be0 0x7f52d41443b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:09.105 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.102+0000 7f52d8df1640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f52d415a8c0 0x7f52d415bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:09.105 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.102+0000 7f52d8df1640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f52d415aca0 0x7f52d4146c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:09.105 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.102+0000 7f52d8df1640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f52d40ce3c0 con 0x7f52d415aca0 2024-09-12T03:23:09.105 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.102+0000 7f52d8df1640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f52d40ce240 con 0x7f52d4071be0 2024-09-12T03:23:09.105 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.102+0000 7f52d8df1640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f52d40b3560 con 0x7f52d415a8c0 2024-09-12T03:23:09.105 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.102+0000 7f52d2d9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f52d415aca0 0x7f52d4146c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:09.105 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.102+0000 7f52d2d9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f52d415aca0 0x7f52d4146c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59908/0 (socket says 172.21.15.27:59908) 2024-09-12T03:23:09.106 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.102+0000 7f52d2d9d640 1 -- 172.21.15.27:0/4216467927 learned_addr learned my addr 172.21.15.27:0/4216467927 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:23:09.106 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.102+0000 7f52d259c640 1 --2- 172.21.15.27:0/4216467927 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f52d4071be0 0x7f52d41443b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:09.106 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.102+0000 7f52d1d9b640 1 --2- 172.21.15.27:0/4216467927 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f52d415a8c0 0x7f52d415bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:09.106 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.102+0000 7f52d2d9d640 1 -- 172.21.15.27:0/4216467927 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f52d415a8c0 msgr2=0x7f52d415bae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:09.106 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.102+0000 7f52d2d9d640 1 --2- 172.21.15.27:0/4216467927 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f52d415a8c0 0x7f52d415bae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:09.106 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.102+0000 7f52d2d9d640 1 -- 172.21.15.27:0/4216467927 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f52d4071be0 msgr2=0x7f52d41443b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:09.106 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.102+0000 7f52d2d9d640 1 --2- 172.21.15.27:0/4216467927 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f52d4071be0 0x7f52d41443b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:09.106 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.102+0000 7f52d2d9d640 1 -- 172.21.15.27:0/4216467927 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52d40ad9b0 con 0x7f52d415aca0 2024-09-12T03:23:09.106 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.103+0000 7f52d2d9d640 1 --2- 172.21.15.27:0/4216467927 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f52d415aca0 0x7f52d4146c40 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7f52cc00cac0 tx=0x7f52cc00cf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:09.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.103+0000 7f52c2ffd640 1 -- 172.21.15.27:0/4216467927 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52cc017070 con 0x7f52d415aca0 2024-09-12T03:23:09.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.103+0000 7f52c2ffd640 1 -- 172.21.15.27:0/4216467927 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52cc00ac30 con 0x7f52d415aca0 2024-09-12T03:23:09.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.103+0000 7f52d8df1640 1 -- 172.21.15.27:0/4216467927 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52d415c530 con 0x7f52d415aca0 2024-09-12T03:23:09.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.103+0000 7f52c2ffd640 1 -- 172.21.15.27:0/4216467927 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52cc012400 con 0x7f52d415aca0 2024-09-12T03:23:09.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.103+0000 7f52d8df1640 1 -- 172.21.15.27:0/4216467927 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52d40adc20 con 0x7f52d415aca0 2024-09-12T03:23:09.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.104+0000 7f52d8df1640 1 -- 172.21.15.27:0/4216467927 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52d40ade20 con 0x7f52d415aca0 2024-09-12T03:23:09.108 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.105+0000 7f52c2ffd640 1 -- 172.21.15.27:0/4216467927 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 24) ==== 91212+0+0 (secure 0 0 0) 0x7f52cc01a070 con 0x7f52d415aca0 2024-09-12T03:23:09.108 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.105+0000 7f52c2ffd640 1 --2- 172.21.15.27:0/4216467927 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f52b406c610 0x7f52b406ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:09.108 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.105+0000 7f52c2ffd640 1 -- 172.21.15.27:0/4216467927 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 5765+0+0 (secure 0 0 0) 0x7f52cc0895a0 con 0x7f52d415aca0 2024-09-12T03:23:09.108 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.105+0000 7f52d259c640 1 --2- 172.21.15.27:0/4216467927 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f52b406c610 0x7f52b406ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:09.108 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.106+0000 7f52d259c640 1 --2- 172.21.15.27:0/4216467927 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f52b406c610 0x7f52b406ead0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f52c4001650 tx=0x7f52c4009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:09.109 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.109+0000 7f52c2ffd640 1 -- 172.21.15.27:0/4216467927 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f52d40ade20 con 0x7f52d415aca0 2024-09-12T03:23:09.285 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.285+0000 7f52d8df1640 1 -- 172.21.15.27:0/4216467927 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f52d4072ba0 con 0x7f52d415aca0 2024-09-12T03:23:09.286 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.287+0000 7f52c2ffd640 1 -- 172.21.15.27:0/4216467927 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v138) ==== 74+0+16772 (secure 0 0 0) 0x7f52cc059840 con 0x7f52d415aca0 2024-09-12T03:23:09.287 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:23:09.287 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":138,"fsid":"d2df504e-2647-4385-a5db-9fdf69f487a0","created":"2024-09-12T03:16:07.778937+0000","modified":"2024-09-12T03:22:51.848998+0000","last_up_change":"2024-09-12T03:16:19.314778+0000","last_in_change":"2024-09-12T03:16:09.893086+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T03:16:20.097646+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2024-09-12T03:22:47.859008+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"137","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2024-09-12T03:22:49.183109+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"138","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1851116c-1fb3-4dc9-9772-f90e2ef4c355","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6817","nonce":3002321488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6819","nonce":3002321488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6823","nonce":3002321488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6821","nonce":3002321488}]},"public_addr":"172.21.15.27:6817/3002321488","cluster_addr":"172.21.15.27:6819/3002321488","heartbeat_back_addr":"172.21.15.27:6823/3002321488","heartbeat_front_addr":"172.21.15.27:6821/3002321488","state":["exists","up"]},{"osd":1,"uuid":"f38ae17d-73eb-4204-b99d-975fa806fb87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6809","nonce":1613102886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6811","nonce":1613102886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6815","nonce":1613102886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6813","nonce":1613102886}]},"public_addr":"172.21.15.27:6809/1613102886","cluster_addr":"172.21.15.27:6811/1613102886","heartbeat_back_addr":"172.21.15.27:6815/1613102886","heartbeat_front_addr":"172.21.15.27:6813/1613102886","state":["exists","up"]},{"osd":2,"uuid":"fb12dcc8-bdd0-42b8-b46d-8b10106224bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6825","nonce":3534723355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6827","nonce":3534723355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6831","nonce":3534723355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6829","nonce":3534723355}]},"public_addr":"172.21.15.27:6825/3534723355","cluster_addr":"172.21.15.27:6827/3534723355","heartbeat_back_addr":"172.21.15.27:6831/3534723355","heartbeat_front_addr":"172.21.15.27:6829/3534723355","state":["exists","up"]},{"osd":3,"uuid":"d74d3f49-9c24-4b9f-98e2-3ecd7b5ef3b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6801","nonce":80661598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6803","nonce":80661598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6807","nonce":80661598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6805","nonce":80661598}]},"public_addr":"172.21.15.27:6801/80661598","cluster_addr":"172.21.15.27:6803/80661598","heartbeat_back_addr":"172.21.15.27:6807/80661598","heartbeat_front_addr":"172.21.15.27:6805/80661598","state":["exists","up"]},{"osd":4,"uuid":"2bf123f2-1b0d-41c4-aa3b-805bab267e59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6817","nonce":3943189827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6819","nonce":3943189827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6823","nonce":3943189827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6821","nonce":3943189827}]},"public_addr":"172.21.15.45:6817/3943189827","cluster_addr":"172.21.15.45:6819/3943189827","heartbeat_back_addr":"172.21.15.45:6823/3943189827","heartbeat_front_addr":"172.21.15.45:6821/3943189827","state":["exists","up"]},{"osd":5,"uuid":"c49ba5da-91c6-4a86-a97b-4aba89c253ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6809","nonce":3620760330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6811","nonce":3620760330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6815","nonce":3620760330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6813","nonce":3620760330}]},"public_addr":"172.21.15.45:6809/3620760330","cluster_addr":"172.21.15.45:6811/3620760330","heartbeat_back_addr":"172.21.15.45:6815/3620760330","heartbeat_front_addr":"172.21.15.45:6813/3620760330","state":["exists","up"]},{"osd":6,"uuid":"b0123475-89f7-4a34-9e33-d797b99dd9dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6801","nonce":1989083510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6803","nonce":1989083510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6807","nonce":1989083510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6805","nonce":1989083510}]},"public_addr":"172.21.15.45:6801/1989083510","cluster_addr":"172.21.15.45:6803/1989083510","heartbeat_back_addr":"172.21.15.45:6807/1989083510","heartbeat_front_addr":"172.21.15.45:6805/1989083510","state":["exists","up"]},{"osd":7,"uuid":"4dad773d-b7d8-46fc-8a10-a972aad27b29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6825","nonce":1513300294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6827","nonce":1513300294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6831","nonce":1513300294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6829","nonce":1513300294}]},"public_addr":"172.21.15.45:6825/1513300294","cluster_addr":"172.21.15.45:6827/1513300294","heartbeat_back_addr":"172.21.15.45:6831/1513300294","heartbeat_front_addr":"172.21.15.45:6829/1513300294","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.881855+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:17.133252+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.735339+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.500742+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.477571+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:17.167667+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.470291+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.873743+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]},{"pool":19,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T03:23:09.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.289+0000 7f52d8df1640 1 -- 172.21.15.27:0/4216467927 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f52b406c610 msgr2=0x7f52b406ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:09.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.289+0000 7f52d8df1640 1 --2- 172.21.15.27:0/4216467927 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f52b406c610 0x7f52b406ead0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f52c4001650 tx=0x7f52c4009040 comp rx=0 tx=0).stop 2024-09-12T03:23:09.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.289+0000 7f52d8df1640 1 -- 172.21.15.27:0/4216467927 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f52d415aca0 msgr2=0x7f52d4146c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:09.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.289+0000 7f52d8df1640 1 --2- 172.21.15.27:0/4216467927 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f52d415aca0 0x7f52d4146c40 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7f52cc00cac0 tx=0x7f52cc00cf80 comp rx=0 tx=0).stop 2024-09-12T03:23:09.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.289+0000 7f52d8df1640 1 -- 172.21.15.27:0/4216467927 shutdown_connections 2024-09-12T03:23:09.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.289+0000 7f52d8df1640 1 --2- 172.21.15.27:0/4216467927 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f52b406c610 0x7f52b406ead0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:09.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.289+0000 7f52d8df1640 1 --2- 172.21.15.27:0/4216467927 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f52d415aca0 0x7f52d4146c40 unknown :-1 s=CLOSED pgs=1470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:09.290 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.289+0000 7f52d8df1640 1 --2- 172.21.15.27:0/4216467927 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f52d415a8c0 0x7f52d415bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:09.290 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.289+0000 7f52d8df1640 1 --2- 172.21.15.27:0/4216467927 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f52d4071be0 0x7f52d41443b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:09.290 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.289+0000 7f52d8df1640 1 -- 172.21.15.27:0/4216467927 >> 172.21.15.27:0/4216467927 conn(0x7f52d4062c60 msgr2=0x7f52d415ee90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:09.290 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.289+0000 7f52d8df1640 1 -- 172.21.15.27:0/4216467927 shutdown_connections 2024-09-12T03:23:09.290 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.289+0000 7f52d8df1640 1 -- 172.21.15.27:0/4216467927 wait complete. 2024-09-12T03:23:09.300 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-12T03:23:09.301 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-12T03:23:09.402 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.402+0000 7f42dced0640 1 Processor -- start 2024-09-12T03:23:09.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.402+0000 7f42dced0640 1 -- start start 2024-09-12T03:23:09.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.403+0000 7f42dced0640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f42d815eb90 0x7f42d80d2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:09.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.403+0000 7f42dced0640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f42d80d5200 0x7f42d80d1010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:09.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.403+0000 7f42dced0640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f42d80d3760 con 0x7f42d80d1550 2024-09-12T03:23:09.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.403+0000 7f42dced0640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f42d80b7d20 con 0x7f42d80d5200 2024-09-12T03:23:09.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.403+0000 7f42dced0640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f42d80d42e0 con 0x7f42d815eb90 2024-09-12T03:23:09.405 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.403+0000 7f42d5d9b640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f42d80d5200 0x7f42d80d1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:09.405 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.403+0000 7f42d6d9d640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f42d80d1550 0x7f42d80cf200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:48574/0 (socket says 172.21.15.27:48574) 2024-09-12T03:23:09.405 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.403+0000 7f42d6d9d640 1 -- 172.21.15.27:0/396981718 learned_addr learned my addr 172.21.15.27:0/396981718 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:23:09.405 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.404+0000 7f42d659c640 1 --2- 172.21.15.27:0/396981718 >> v2:172.21.15.45:3300/0 conn(0x7f42d815eb90 0x7f42d80d2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:09.405 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.404+0000 7f42d5d9b640 1 -- 172.21.15.27:0/396981718 >> v2:172.21.15.45:3300/0 conn(0x7f42d815eb90 msgr2=0x7f42d80d2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:09.405 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.404+0000 7f42d5d9b640 1 --2- 172.21.15.27:0/396981718 >> v2:172.21.15.45:3300/0 conn(0x7f42d815eb90 0x7f42d80d2df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:09.405 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.404+0000 7f42d5d9b640 1 -- 172.21.15.27:0/396981718 >> v1:172.21.15.27:6789/0 conn(0x7f42d80d1550 legacy=0x7f42d80cf200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:09.405 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.404+0000 7f42d5d9b640 1 -- 172.21.15.27:0/396981718 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42d80d3a40 con 0x7f42d80d5200 2024-09-12T03:23:09.405 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.404+0000 7f42d559a640 1 -- 172.21.15.27:0/396981718 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1005694984 0 0) 0x7f42d80d3760 con 0x7f42d80d1550 2024-09-12T03:23:09.405 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.404+0000 7f42d659c640 1 --2- 172.21.15.27:0/396981718 >> v2:172.21.15.45:3300/0 conn(0x7f42d815eb90 0x7f42d80d2df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:23:09.407 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.404+0000 7f42d5d9b640 1 --2- 172.21.15.27:0/396981718 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f42d80d5200 0x7f42d80d1010 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7f42d8075560 tx=0x7f42c002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=733f53b838275b4b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:09.407 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.405+0000 7f42d559a640 1 -- 172.21.15.27:0/396981718 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42c0037070 con 0x7f42d80d5200 2024-09-12T03:23:09.407 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.405+0000 7f42d559a640 1 -- 172.21.15.27:0/396981718 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f42c0032440 con 0x7f42d80d5200 2024-09-12T03:23:09.407 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.405+0000 7f42d559a640 1 -- 172.21.15.27:0/396981718 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42c0032810 con 0x7f42d80d5200 2024-09-12T03:23:09.407 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.405+0000 7f42dced0640 1 -- 172.21.15.27:0/396981718 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f42d80d5200 msgr2=0x7f42d80d1010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:09.407 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.405+0000 7f42dced0640 1 --2- 172.21.15.27:0/396981718 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f42d80d5200 0x7f42d80d1010 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7f42d8075560 tx=0x7f42c002a1a0 comp rx=0 tx=0).stop 2024-09-12T03:23:09.407 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.405+0000 7f42dced0640 1 -- 172.21.15.27:0/396981718 shutdown_connections 2024-09-12T03:23:09.407 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.405+0000 7f42dced0640 1 --2- 172.21.15.27:0/396981718 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f42d80d5200 0x7f42d80d1010 unknown :-1 s=CLOSED pgs=1472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:09.407 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.405+0000 7f42dced0640 1 --2- 172.21.15.27:0/396981718 >> v2:172.21.15.45:3300/0 conn(0x7f42d815eb90 0x7f42d80d2df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:09.408 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.405+0000 7f42dced0640 1 -- 172.21.15.27:0/396981718 >> 172.21.15.27:0/396981718 conn(0x7f42d806dce0 msgr2=0x7f42d806e0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:09.408 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.405+0000 7f42dced0640 1 -- 172.21.15.27:0/396981718 shutdown_connections 2024-09-12T03:23:09.408 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.405+0000 7f42dced0640 1 -- 172.21.15.27:0/396981718 wait complete. 2024-09-12T03:23:09.408 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.407+0000 7f42dced0640 1 Processor -- start 2024-09-12T03:23:09.408 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.407+0000 7f42dced0640 1 -- start start 2024-09-12T03:23:09.410 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.407+0000 7f42dced0640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f42d80d1550 0x7f42d81f3f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:09.410 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.407+0000 7f42dced0640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f42d80d5200 0x7f42d815a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:09.410 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.408+0000 7f42dced0640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f42d815eb90 0x7f42d80ce050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:09.410 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.408+0000 7f42dced0640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f42d80d1960 con 0x7f42d80d1550 2024-09-12T03:23:09.410 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.408+0000 7f42dced0640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f42d80d4000 con 0x7f42d815eb90 2024-09-12T03:23:09.410 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.408+0000 7f42dced0640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f42d80d3330 con 0x7f42d80d5200 2024-09-12T03:23:09.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.408+0000 7f42d659c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f42d80d1550 0x7f42d81f3f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:09.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.408+0000 7f42d659c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f42d80d1550 0x7f42d81f3f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58994/0 (socket says 172.21.15.27:58994) 2024-09-12T03:23:09.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.408+0000 7f42d659c640 1 -- 172.21.15.27:0/861776373 learned_addr learned my addr 172.21.15.27:0/861776373 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:23:09.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.408+0000 7f42d6d9d640 1 --2- 172.21.15.27:0/861776373 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f42d815eb90 0x7f42d80ce050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:09.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.408+0000 7f42d5d9b640 1 --2- 172.21.15.27:0/861776373 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f42d80d5200 0x7f42d815a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:09.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.408+0000 7f42d659c640 1 -- 172.21.15.27:0/861776373 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f42d80d5200 msgr2=0x7f42d815a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:09.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.408+0000 7f42d659c640 1 --2- 172.21.15.27:0/861776373 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f42d80d5200 0x7f42d815a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:09.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.408+0000 7f42d659c640 1 -- 172.21.15.27:0/861776373 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f42d815eb90 msgr2=0x7f42d80ce050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:09.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.408+0000 7f42d659c640 1 --2- 172.21.15.27:0/861776373 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f42d815eb90 0x7f42d80ce050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:09.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.408+0000 7f42d659c640 1 -- 172.21.15.27:0/861776373 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42d80c9ae0 con 0x7f42d80d1550 2024-09-12T03:23:09.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.408+0000 7f42d659c640 1 --2- 172.21.15.27:0/861776373 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f42d80d1550 0x7f42d81f3f00 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7f42d0009fd0 tx=0x7f42d000d560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:09.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.408+0000 7f42ceffd640 1 -- 172.21.15.27:0/861776373 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42d001a070 con 0x7f42d80d1550 2024-09-12T03:23:09.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.408+0000 7f42ceffd640 1 -- 172.21.15.27:0/861776373 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f42d0003bb0 con 0x7f42d80d1550 2024-09-12T03:23:09.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.408+0000 7f42dced0640 1 -- 172.21.15.27:0/861776373 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42d80c9dc0 con 0x7f42d80d1550 2024-09-12T03:23:09.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.408+0000 7f42ceffd640 1 -- 172.21.15.27:0/861776373 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42d0004030 con 0x7f42d80d1550 2024-09-12T03:23:09.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.408+0000 7f42dced0640 1 -- 172.21.15.27:0/861776373 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42d80ca380 con 0x7f42d80d1550 2024-09-12T03:23:09.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.409+0000 7f42dced0640 1 -- 172.21.15.27:0/861776373 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42d815ef70 con 0x7f42d80d1550 2024-09-12T03:23:09.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.410+0000 7f42ceffd640 1 -- 172.21.15.27:0/861776373 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 24) ==== 91212+0+0 (secure 0 0 0) 0x7f42d001d070 con 0x7f42d80d1550 2024-09-12T03:23:09.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.410+0000 7f42ceffd640 1 --2- 172.21.15.27:0/861776373 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f42b806c610 0x7f42b806ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:09.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.410+0000 7f42d5d9b640 1 --2- 172.21.15.27:0/861776373 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f42b806c610 0x7f42b806ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:09.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.410+0000 7f42ceffd640 1 -- 172.21.15.27:0/861776373 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 5765+0+0 (secure 0 0 0) 0x7f42d00899f0 con 0x7f42d80d1550 2024-09-12T03:23:09.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.411+0000 7f42d5d9b640 1 --2- 172.21.15.27:0/861776373 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f42b806c610 0x7f42b806ead0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f42c0034000 tx=0x7f42c002e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:09.414 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.414+0000 7f42ceffd640 1 -- 172.21.15.27:0/861776373 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f42d0010070 con 0x7f42d80d1550 2024-09-12T03:23:09.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.629+0000 7f42dced0640 1 -- 172.21.15.27:0/861776373 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f42d8069ea0 con 0x7f42d80d1550 2024-09-12T03:23:09.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.630+0000 7f42ceffd640 1 -- 172.21.15.27:0/861776373 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 83 v83) ==== 94+0+5224 (secure 0 0 0) 0x7f42d0059640 con 0x7f42d80d1550 2024-09-12T03:23:09.630 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:23:09.630 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":83,"btime":"2024-09-12T03:22:58:967052+0000","default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6689,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/2285501348","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2285501348},{"type":"v1","addr":"172.21.15.45:6833","nonce":2285501348}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79},{"gid":7453,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/1872420883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":1872420883},{"type":"v1","addr":"172.21.15.45:6835","nonce":1872420883}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79},{"gid":8193,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6834/3968488955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":3968488955},{"type":"v1","addr":"172.21.15.27:6834","nonce":3968488955}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79}],"filesystems":[{"mdsmap":{"epoch":82,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-12T03:22:50.200339+0000","modified":"2024-09-12T03:22:52.204820+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8196},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8196":{"gid":8196,"name":"a","rank":0,"incarnation":80,"state":"up:active","state_seq":3,"addr":"172.21.15.27:6835/389118996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":389118996},{"type":"v1","addr":"172.21.15.27:6835","nonce":389118996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8196,"qdb_cluster":[8196]},"id":11}]} 2024-09-12T03:23:09.630 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 83 2024-09-12T03:23:09.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.632+0000 7f42dced0640 1 -- 172.21.15.27:0/861776373 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f42b806c610 msgr2=0x7f42b806ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:09.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.632+0000 7f42dced0640 1 --2- 172.21.15.27:0/861776373 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f42b806c610 0x7f42b806ead0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f42c0034000 tx=0x7f42c002e000 comp rx=0 tx=0).stop 2024-09-12T03:23:09.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.632+0000 7f42dced0640 1 -- 172.21.15.27:0/861776373 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f42d80d1550 msgr2=0x7f42d81f3f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:09.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.632+0000 7f42dced0640 1 --2- 172.21.15.27:0/861776373 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f42d80d1550 0x7f42d81f3f00 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7f42d0009fd0 tx=0x7f42d000d560 comp rx=0 tx=0).stop 2024-09-12T03:23:09.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.632+0000 7f42dced0640 1 -- 172.21.15.27:0/861776373 shutdown_connections 2024-09-12T03:23:09.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.632+0000 7f42dced0640 1 --2- 172.21.15.27:0/861776373 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f42d815eb90 0x7f42d80ce050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:09.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.632+0000 7f42dced0640 1 --2- 172.21.15.27:0/861776373 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f42d80d5200 0x7f42d815a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:09.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.632+0000 7f42dced0640 1 --2- 172.21.15.27:0/861776373 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f42b806c610 0x7f42b806ead0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:09.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.632+0000 7f42dced0640 1 --2- 172.21.15.27:0/861776373 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f42d80d1550 0x7f42d81f3f00 unknown :-1 s=CLOSED pgs=1473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:09.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.632+0000 7f42dced0640 1 -- 172.21.15.27:0/861776373 >> 172.21.15.27:0/861776373 conn(0x7f42d806dce0 msgr2=0x7f42d80d1f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:09.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.632+0000 7f42dced0640 1 -- 172.21.15.27:0/861776373 shutdown_connections 2024-09-12T03:23:09.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.632+0000 7f42dced0640 1 -- 172.21.15.27:0/861776373 wait complete. 2024-09-12T03:23:09.644 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-12T03:23:09.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.745+0000 7fc017711640 1 Processor -- start 2024-09-12T03:23:09.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.746+0000 7fc017711640 1 -- start start 2024-09-12T03:23:09.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.746+0000 7fc017711640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc01015eb90 0x7fc0100d2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:09.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.746+0000 7fc017711640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fc0100d1720 0x7fc0100b8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:09.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.746+0000 7fc017711640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc0100d3760 con 0x7fc0100d5200 2024-09-12T03:23:09.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.746+0000 7fc017711640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fc0100b7d20 con 0x7fc01015eb90 2024-09-12T03:23:09.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.746+0000 7fc017711640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fc0100d42e0 con 0x7fc0100d1720 2024-09-12T03:23:09.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.747+0000 7fc0154ad640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc01015eb90 0x7fc0100d2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:09.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.747+0000 7fc015cae640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fc0100d1720 0x7fc0100b8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:09.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.747+0000 7fc015cae640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fc0100d1720 0x7fc0100b8190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3301/0 says I am v2:172.21.15.27:36736/0 (socket says 172.21.15.27:36736) 2024-09-12T03:23:09.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.747+0000 7fc0154ad640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc01015eb90 0x7fc0100d2df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.27:52644/0 (socket says 172.21.15.27:52644) 2024-09-12T03:23:09.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.747+0000 7fc015cae640 1 -- 172.21.15.27:0/4248945584 learned_addr learned my addr 172.21.15.27:0/4248945584 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:23:09.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.747+0000 7fc0154ad640 1 -- 172.21.15.27:0/4248945584 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fc0100d1720 msgr2=0x7fc0100b8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:09.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.747+0000 7fc0154ad640 1 --2- 172.21.15.27:0/4248945584 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fc0100d1720 0x7fc0100b8190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:09.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.747+0000 7fc0154ad640 1 -- 172.21.15.27:0/4248945584 >> v1:172.21.15.45:6789/0 conn(0x7fc0100d5200 legacy=0x7fc0100d1010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:09.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.747+0000 7fc0154ad640 1 -- 172.21.15.27:0/4248945584 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0100d3a40 con 0x7fc01015eb90 2024-09-12T03:23:09.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.748+0000 7fc0154ad640 1 --2- 172.21.15.27:0/4248945584 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc01015eb90 0x7fc0100d2df0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fc00c0097a0 tx=0x7fc00c02a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=52be831ee82c1f6f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:09.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.748+0000 7fc007fff640 1 -- 172.21.15.27:0/4248945584 <== mon.1 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc00c037070 con 0x7fc01015eb90 2024-09-12T03:23:09.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.748+0000 7fc007fff640 1 -- 172.21.15.27:0/4248945584 <== mon.1 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc00c032440 con 0x7fc01015eb90 2024-09-12T03:23:09.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.748+0000 7fc007fff640 1 -- 172.21.15.27:0/4248945584 <== mon.1 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc00c032810 con 0x7fc01015eb90 2024-09-12T03:23:09.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.749+0000 7fc017711640 1 -- 172.21.15.27:0/4248945584 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc01015eb90 msgr2=0x7fc0100d2df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:09.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.749+0000 7fc017711640 1 --2- 172.21.15.27:0/4248945584 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc01015eb90 0x7fc0100d2df0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fc00c0097a0 tx=0x7fc00c02a180 comp rx=0 tx=0).stop 2024-09-12T03:23:09.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.749+0000 7fc017711640 1 -- 172.21.15.27:0/4248945584 shutdown_connections 2024-09-12T03:23:09.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.749+0000 7fc017711640 1 --2- 172.21.15.27:0/4248945584 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fc0100d1720 0x7fc0100b8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:09.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.749+0000 7fc017711640 1 --2- 172.21.15.27:0/4248945584 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc01015eb90 0x7fc0100d2df0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:09.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.749+0000 7fc017711640 1 -- 172.21.15.27:0/4248945584 >> 172.21.15.27:0/4248945584 conn(0x7fc01006dce0 msgr2=0x7fc01006e0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:09.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.749+0000 7fc017711640 1 -- 172.21.15.27:0/4248945584 shutdown_connections 2024-09-12T03:23:09.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.749+0000 7fc017711640 1 -- 172.21.15.27:0/4248945584 wait complete. 2024-09-12T03:23:09.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.750+0000 7fc017711640 1 Processor -- start 2024-09-12T03:23:09.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.750+0000 7fc017711640 1 -- start start 2024-09-12T03:23:09.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.751+0000 7fc017711640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc0100d1720 0x7fc0101e8350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:09.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.751+0000 7fc017711640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc0100d5200 0x7fc0101f52b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:09.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.751+0000 7fc017711640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fc01015eb90 0x7fc0101e9520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:09.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.751+0000 7fc017711640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fc0100d3330 con 0x7fc0100d5200 2024-09-12T03:23:09.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.751+0000 7fc017711640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc0100d4000 con 0x7fc0100d1720 2024-09-12T03:23:09.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.751+0000 7fc017711640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fc0100d34b0 con 0x7fc01015eb90 2024-09-12T03:23:09.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.751+0000 7fc014cac640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc0100d5200 0x7fc0101f52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:09.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.751+0000 7fc014cac640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc0100d5200 0x7fc0101f52b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58996/0 (socket says 172.21.15.27:58996) 2024-09-12T03:23:09.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.751+0000 7fc014cac640 1 -- 172.21.15.27:0/3598806942 learned_addr learned my addr 172.21.15.27:0/3598806942 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:23:09.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.751+0000 7fc015cae640 1 --2- 172.21.15.27:0/3598806942 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fc01015eb90 0x7fc0101e9520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:09.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.751+0000 7fc0154ad640 1 --2- 172.21.15.27:0/3598806942 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc0100d1720 0x7fc0101e8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:09.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.751+0000 7fc014cac640 1 -- 172.21.15.27:0/3598806942 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fc01015eb90 msgr2=0x7fc0101e9520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:09.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.751+0000 7fc014cac640 1 --2- 172.21.15.27:0/3598806942 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fc01015eb90 0x7fc0101e9520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:09.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.751+0000 7fc014cac640 1 -- 172.21.15.27:0/3598806942 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc0100d1720 msgr2=0x7fc0101e8350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:09.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.751+0000 7fc014cac640 1 --2- 172.21.15.27:0/3598806942 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc0100d1720 0x7fc0101e8350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:09.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.751+0000 7fc014cac640 1 -- 172.21.15.27:0/3598806942 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0100ce600 con 0x7fc0100d5200 2024-09-12T03:23:09.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.751+0000 7fc0154ad640 1 --2- 172.21.15.27:0/3598806942 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc0100d1720 0x7fc0101e8350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:23:09.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.751+0000 7fc014cac640 1 --2- 172.21.15.27:0/3598806942 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc0100d5200 0x7fc0101f52b0 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7fc00000c470 tx=0x7fc000001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:09.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.752+0000 7fc005ffb640 1 -- 172.21.15.27:0/3598806942 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0000052d0 con 0x7fc0100d5200 2024-09-12T03:23:09.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.752+0000 7fc017711640 1 -- 172.21.15.27:0/3598806942 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0100ce8e0 con 0x7fc0100d5200 2024-09-12T03:23:09.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.752+0000 7fc005ffb640 1 -- 172.21.15.27:0/3598806942 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc000005470 con 0x7fc0100d5200 2024-09-12T03:23:09.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.752+0000 7fc005ffb640 1 -- 172.21.15.27:0/3598806942 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc000007260 con 0x7fc0100d5200 2024-09-12T03:23:09.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.752+0000 7fc017711640 1 -- 172.21.15.27:0/3598806942 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0101e8d20 con 0x7fc0100d5200 2024-09-12T03:23:09.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.753+0000 7fc017711640 1 -- 172.21.15.27:0/3598806942 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc01015f090 con 0x7fc0100d5200 2024-09-12T03:23:09.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.753+0000 7fc005ffb640 1 -- 172.21.15.27:0/3598806942 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 24) ==== 91212+0+0 (secure 0 0 0) 0x7fc000007400 con 0x7fc0100d5200 2024-09-12T03:23:09.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.754+0000 7fc005ffb640 1 --2- 172.21.15.27:0/3598806942 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fbff406c610 0x7fbff406ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:09.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.754+0000 7fc005ffb640 1 -- 172.21.15.27:0/3598806942 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 5765+0+0 (secure 0 0 0) 0x7fc000088de0 con 0x7fc0100d5200 2024-09-12T03:23:09.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.754+0000 7fc0154ad640 1 --2- 172.21.15.27:0/3598806942 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fbff406c610 0x7fbff406ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:09.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.754+0000 7fc0154ad640 1 --2- 172.21.15.27:0/3598806942 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fbff406c610 0x7fbff406ead0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fc00c034000 tx=0x7fc00c02e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:09.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.758+0000 7fc005ffb640 1 -- 172.21.15.27:0/3598806942 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fc000059000 con 0x7fc0100d5200 2024-09-12T03:23:09.974 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.974+0000 7fc017711640 1 -- 172.21.15.27:0/3598806942 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc0100b91d0 con 0x7fc0100d5200 2024-09-12T03:23:09.974 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.974+0000 7fc005ffb640 1 -- 172.21.15.27:0/3598806942 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 83 v83) ==== 94+0+5224 (secure 0 0 0) 0x7fc000058900 con 0x7fc0100d5200 2024-09-12T03:23:09.975 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:23:09.975 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":83,"btime":"2024-09-12T03:22:58:967052+0000","default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6689,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/2285501348","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2285501348},{"type":"v1","addr":"172.21.15.45:6833","nonce":2285501348}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79},{"gid":7453,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/1872420883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":1872420883},{"type":"v1","addr":"172.21.15.45:6835","nonce":1872420883}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79},{"gid":8193,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6834/3968488955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":3968488955},{"type":"v1","addr":"172.21.15.27:6834","nonce":3968488955}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79}],"filesystems":[{"mdsmap":{"epoch":82,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-12T03:22:50.200339+0000","modified":"2024-09-12T03:22:52.204820+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8196},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8196":{"gid":8196,"name":"a","rank":0,"incarnation":80,"state":"up:active","state_seq":3,"addr":"172.21.15.27:6835/389118996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":389118996},{"type":"v1","addr":"172.21.15.27:6835","nonce":389118996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8196,"qdb_cluster":[8196]},"id":11}]} 2024-09-12T03:23:09.975 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 83 2024-09-12T03:23:09.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.977+0000 7fc017711640 1 -- 172.21.15.27:0/3598806942 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fbff406c610 msgr2=0x7fbff406ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:09.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.977+0000 7fc017711640 1 --2- 172.21.15.27:0/3598806942 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fbff406c610 0x7fbff406ead0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fc00c034000 tx=0x7fc00c02e000 comp rx=0 tx=0).stop 2024-09-12T03:23:09.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.977+0000 7fc017711640 1 -- 172.21.15.27:0/3598806942 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc0100d5200 msgr2=0x7fc0101f52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:09.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.977+0000 7fc017711640 1 --2- 172.21.15.27:0/3598806942 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc0100d5200 0x7fc0101f52b0 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7fc00000c470 tx=0x7fc000001e70 comp rx=0 tx=0).stop 2024-09-12T03:23:09.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.977+0000 7fc017711640 1 -- 172.21.15.27:0/3598806942 shutdown_connections 2024-09-12T03:23:09.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.977+0000 7fc017711640 1 --2- 172.21.15.27:0/3598806942 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fc01015eb90 0x7fc0101e9520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:09.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.977+0000 7fc017711640 1 --2- 172.21.15.27:0/3598806942 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fbff406c610 0x7fbff406ead0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:09.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.977+0000 7fc017711640 1 --2- 172.21.15.27:0/3598806942 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc0100d5200 0x7fc0101f52b0 unknown :-1 s=CLOSED pgs=1474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:09.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.977+0000 7fc017711640 1 --2- 172.21.15.27:0/3598806942 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc0100d1720 0x7fc0101e8350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:09.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.977+0000 7fc017711640 1 -- 172.21.15.27:0/3598806942 >> 172.21.15.27:0/3598806942 conn(0x7fc01006dce0 msgr2=0x7fc0101612e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:09.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.977+0000 7fc017711640 1 -- 172.21.15.27:0/3598806942 shutdown_connections 2024-09-12T03:23:09.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:09.977+0000 7fc017711640 1 -- 172.21.15.27:0/3598806942 wait complete. 2024-09-12T03:23:09.987 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-12T03:23:10.090 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.090+0000 7fb28d2d0640 1 Processor -- start 2024-09-12T03:23:10.090 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.090+0000 7fb28d2d0640 1 -- start start 2024-09-12T03:23:10.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.091+0000 7fb28d2d0640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fb28815a8c0 0x7fb2880cdb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:10.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.091+0000 7fb28d2d0640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb28806daf0 con 0x7fb28815aca0 2024-09-12T03:23:10.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.091+0000 7fb28d2d0640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb28806ddd0 con 0x7fb288071be0 2024-09-12T03:23:10.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.091+0000 7fb28d2d0640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fb28806d8a0 con 0x7fb28815a8c0 2024-09-12T03:23:10.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.091+0000 7fb2877fe640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fb28815aca0 0x7fb2880b4710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:48588/0 (socket says 172.21.15.27:48588) 2024-09-12T03:23:10.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.091+0000 7fb2877fe640 1 -- 172.21.15.27:0/1995438734 learned_addr learned my addr 172.21.15.27:0/1995438734 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:23:10.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.091+0000 7fb286ffd640 1 --2- 172.21.15.27:0/1995438734 >> v2:172.21.15.45:3300/0 conn(0x7fb28815a8c0 0x7fb2880cdb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:10.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.091+0000 7fb285ffb640 1 -- 172.21.15.27:0/1995438734 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1609811088 0 0) 0x7fb28806daf0 con 0x7fb28815aca0 2024-09-12T03:23:10.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.091+0000 7fb285ffb640 1 -- 172.21.15.27:0/1995438734 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb270003520 con 0x7fb28815aca0 2024-09-12T03:23:10.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.091+0000 7fb285ffb640 1 -- 172.21.15.27:0/1995438734 <== mon.1 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3520395922 0 0) 0x7fb28806ddd0 con 0x7fb288071be0 2024-09-12T03:23:10.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.091+0000 7fb285ffb640 1 -- 172.21.15.27:0/1995438734 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb28806daf0 con 0x7fb288071be0 2024-09-12T03:23:10.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.091+0000 7fb285ffb640 1 -- 172.21.15.27:0/1995438734 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 11600909 0 0) 0x7fb270003520 con 0x7fb28815aca0 2024-09-12T03:23:10.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.091+0000 7fb285ffb640 1 -- 172.21.15.27:0/1995438734 >> v2:172.21.15.45:3300/0 conn(0x7fb28815a8c0 msgr2=0x7fb2880cdb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:10.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.091+0000 7fb285ffb640 1 --2- 172.21.15.27:0/1995438734 >> v2:172.21.15.45:3300/0 conn(0x7fb28815a8c0 0x7fb2880cdb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:10.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.091+0000 7fb285ffb640 1 -- 172.21.15.27:0/1995438734 >> v1:172.21.15.45:6789/0 conn(0x7fb288071be0 legacy=0x7fb28806e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:10.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.091+0000 7fb285ffb640 1 -- 172.21.15.27:0/1995438734 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb28806e0b0 con 0x7fb28815aca0 2024-09-12T03:23:10.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.091+0000 7fb285ffb640 1 -- 172.21.15.27:0/1995438734 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 847755213 0 0) 0x7fb2800019f0 con 0x7fb28815aca0 2024-09-12T03:23:10.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.091+0000 7fb285ffb640 1 -- 172.21.15.27:0/1995438734 >> v1:172.21.15.27:6789/0 conn(0x7fb28815aca0 legacy=0x7fb2880b4710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:10.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.091+0000 7fb285ffb640 1 --2- 172.21.15.27:0/1995438734 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb270003b40 0x7fb270003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:10.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.091+0000 7fb285ffb640 1 --2- 172.21.15.27:0/1995438734 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb270004800 0x7fb270004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:10.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.091+0000 7fb285ffb640 1 --2- 172.21.15.27:0/1995438734 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb2700051c0 0x7fb27000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:10.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.091+0000 7fb285ffb640 1 -- 172.21.15.27:0/1995438734 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fb28806d8a0 con 0x7fb2700051c0 2024-09-12T03:23:10.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.091+0000 7fb285ffb640 1 -- 172.21.15.27:0/1995438734 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb27000c040 con 0x7fb270003b40 2024-09-12T03:23:10.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.091+0000 7fb285ffb640 1 -- 172.21.15.27:0/1995438734 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fb27000e450 con 0x7fb270004800 2024-09-12T03:23:10.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.092+0000 7fb2877fe640 1 --2- 172.21.15.27:0/1995438734 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb2700051c0 0x7fb27000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:10.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.092+0000 7fb286ffd640 1 --2- 172.21.15.27:0/1995438734 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb270003b40 0x7fb270003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:10.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.092+0000 7fb2867fc640 1 --2- 172.21.15.27:0/1995438734 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb270004800 0x7fb270004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:10.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.092+0000 7fb2877fe640 1 -- 172.21.15.27:0/1995438734 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb270004800 msgr2=0x7fb270004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:10.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.092+0000 7fb2877fe640 1 --2- 172.21.15.27:0/1995438734 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb270004800 0x7fb270004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:10.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.092+0000 7fb2877fe640 1 -- 172.21.15.27:0/1995438734 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb270003b40 msgr2=0x7fb270003f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:10.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.092+0000 7fb2877fe640 1 --2- 172.21.15.27:0/1995438734 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb270003b40 0x7fb270003f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:10.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.092+0000 7fb2877fe640 1 -- 172.21.15.27:0/1995438734 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb270003520 con 0x7fb2700051c0 2024-09-12T03:23:10.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.092+0000 7fb2877fe640 1 -- 172.21.15.27:0/1995438734 reap_dead start 2024-09-12T03:23:10.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.093+0000 7fb2877fe640 1 --2- 172.21.15.27:0/1995438734 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb2700051c0 0x7fb27000b9c0 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7fb2880b3a10 tx=0x7fb270005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51f5dfc00293482b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:10.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.093+0000 7fb285ffb640 1 -- 172.21.15.27:0/1995438734 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb28000a450 con 0x7fb2700051c0 2024-09-12T03:23:10.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.093+0000 7fb285ffb640 1 -- 172.21.15.27:0/1995438734 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb28002f070 con 0x7fb2700051c0 2024-09-12T03:23:10.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.093+0000 7fb285ffb640 1 -- 172.21.15.27:0/1995438734 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb28002bc40 con 0x7fb2700051c0 2024-09-12T03:23:10.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.093+0000 7fb28d2d0640 1 -- 172.21.15.27:0/1995438734 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb2700051c0 msgr2=0x7fb27000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:10.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.093+0000 7fb28d2d0640 1 --2- 172.21.15.27:0/1995438734 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb2700051c0 0x7fb27000b9c0 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7fb2880b3a10 tx=0x7fb270005830 comp rx=0 tx=0).stop 2024-09-12T03:23:10.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.093+0000 7fb28d2d0640 1 -- 172.21.15.27:0/1995438734 shutdown_connections 2024-09-12T03:23:10.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.093+0000 7fb28d2d0640 1 --2- 172.21.15.27:0/1995438734 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb2700051c0 0x7fb27000b9c0 unknown :-1 s=CLOSED pgs=1476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:10.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.093+0000 7fb28d2d0640 1 -- 172.21.15.27:0/1995438734 >> 172.21.15.27:0/1995438734 conn(0x7fb288062c60 msgr2=0x7fb28806bb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:10.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.093+0000 7fb28d2d0640 1 -- 172.21.15.27:0/1995438734 shutdown_connections 2024-09-12T03:23:10.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.093+0000 7fb28d2d0640 1 -- 172.21.15.27:0/1995438734 wait complete. 2024-09-12T03:23:10.100 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.095+0000 7fb28d2d0640 1 Processor -- start 2024-09-12T03:23:10.100 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.095+0000 7fb28d2d0640 1 -- start start 2024-09-12T03:23:10.100 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.095+0000 7fb28d2d0640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb2700051c0 0x7fb288156ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:10.100 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.095+0000 7fb28d2d0640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb2880caff0 0x7fb28815d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:10.100 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.095+0000 7fb28d2d0640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb28815d8e0 0x7fb2880ae8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:10.100 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.095+0000 7fb28d2d0640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fb2880b36e0 con 0x7fb28815d8e0 2024-09-12T03:23:10.100 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.095+0000 7fb28d2d0640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb2880b3560 con 0x7fb2880caff0 2024-09-12T03:23:10.100 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.095+0000 7fb28d2d0640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fb288072ba0 con 0x7fb2700051c0 2024-09-12T03:23:10.100 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.096+0000 7fb2877fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb28815d8e0 0x7fb2880ae8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:10.100 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.096+0000 7fb2877fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb28815d8e0 0x7fb2880ae8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59008/0 (socket says 172.21.15.27:59008) 2024-09-12T03:23:10.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.096+0000 7fb2877fe640 1 -- 172.21.15.27:0/1445002913 learned_addr learned my addr 172.21.15.27:0/1445002913 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:23:10.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.096+0000 7fb286ffd640 1 --2- 172.21.15.27:0/1445002913 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb2700051c0 0x7fb288156ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:10.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.096+0000 7fb2867fc640 1 --2- 172.21.15.27:0/1445002913 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb2880caff0 0x7fb28815d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:10.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.096+0000 7fb2877fe640 1 -- 172.21.15.27:0/1445002913 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb2700051c0 msgr2=0x7fb288156ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:10.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.096+0000 7fb2877fe640 1 --2- 172.21.15.27:0/1445002913 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb2700051c0 0x7fb288156ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:10.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.096+0000 7fb2877fe640 1 -- 172.21.15.27:0/1445002913 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb2880caff0 msgr2=0x7fb28815d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:10.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.096+0000 7fb2877fe640 1 --2- 172.21.15.27:0/1445002913 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb2880caff0 0x7fb28815d3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:10.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.096+0000 7fb2877fe640 1 -- 172.21.15.27:0/1445002913 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb28815de40 con 0x7fb28815d8e0 2024-09-12T03:23:10.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.096+0000 7fb2877fe640 1 --2- 172.21.15.27:0/1445002913 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb28815d8e0 0x7fb2880ae8d0 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7fb28000a8f0 tx=0x7fb28806dfa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:10.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.096+0000 7fb26f7fe640 1 -- 172.21.15.27:0/1445002913 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb28002bdf0 con 0x7fb28815d8e0 2024-09-12T03:23:10.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.096+0000 7fb26f7fe640 1 -- 172.21.15.27:0/1445002913 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb280004b60 con 0x7fb28815d8e0 2024-09-12T03:23:10.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.096+0000 7fb28d2d0640 1 -- 172.21.15.27:0/1445002913 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2880ca510 con 0x7fb28815d8e0 2024-09-12T03:23:10.102 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.096+0000 7fb26f7fe640 1 -- 172.21.15.27:0/1445002913 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb280040480 con 0x7fb28815d8e0 2024-09-12T03:23:10.102 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.096+0000 7fb28d2d0640 1 -- 172.21.15.27:0/1445002913 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2880caad0 con 0x7fb28815d8e0 2024-09-12T03:23:10.102 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.097+0000 7fb28d2d0640 1 -- 172.21.15.27:0/1445002913 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb28806d430 con 0x7fb28815d8e0 2024-09-12T03:23:10.102 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.098+0000 7fb26f7fe640 1 -- 172.21.15.27:0/1445002913 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 24) ==== 91212+0+0 (secure 0 0 0) 0x7fb280040620 con 0x7fb28815d8e0 2024-09-12T03:23:10.102 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.098+0000 7fb26f7fe640 1 --2- 172.21.15.27:0/1445002913 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fb27406c610 0x7fb27406ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:10.102 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.099+0000 7fb286ffd640 1 --2- 172.21.15.27:0/1445002913 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fb27406c610 0x7fb27406ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:10.102 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.099+0000 7fb26f7fe640 1 -- 172.21.15.27:0/1445002913 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 5765+0+0 (secure 0 0 0) 0x7fb2800a9a70 con 0x7fb28815d8e0 2024-09-12T03:23:10.102 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.099+0000 7fb286ffd640 1 --2- 172.21.15.27:0/1445002913 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fb27406c610 0x7fb27406ead0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fb278009a80 tx=0x7fb278008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:10.102 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.103+0000 7fb26f7fe640 1 -- 172.21.15.27:0/1445002913 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fb280079d10 con 0x7fb28815d8e0 2024-09-12T03:23:10.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.279+0000 7fb28d2d0640 1 -- 172.21.15.27:0/1445002913 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb28806d430 con 0x7fb28815d8e0 2024-09-12T03:23:10.280 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.280+0000 7fb26f7fe640 1 -- 172.21.15.27:0/1445002913 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v138) ==== 74+0+16772 (secure 0 0 0) 0x7fb280079610 con 0x7fb28815d8e0 2024-09-12T03:23:10.280 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:23:10.280 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":138,"fsid":"d2df504e-2647-4385-a5db-9fdf69f487a0","created":"2024-09-12T03:16:07.778937+0000","modified":"2024-09-12T03:22:51.848998+0000","last_up_change":"2024-09-12T03:16:19.314778+0000","last_in_change":"2024-09-12T03:16:09.893086+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T03:16:20.097646+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2024-09-12T03:22:47.859008+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"137","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2024-09-12T03:22:49.183109+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"138","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1851116c-1fb3-4dc9-9772-f90e2ef4c355","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6817","nonce":3002321488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6819","nonce":3002321488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6823","nonce":3002321488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6821","nonce":3002321488}]},"public_addr":"172.21.15.27:6817/3002321488","cluster_addr":"172.21.15.27:6819/3002321488","heartbeat_back_addr":"172.21.15.27:6823/3002321488","heartbeat_front_addr":"172.21.15.27:6821/3002321488","state":["exists","up"]},{"osd":1,"uuid":"f38ae17d-73eb-4204-b99d-975fa806fb87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6809","nonce":1613102886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6811","nonce":1613102886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6815","nonce":1613102886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6813","nonce":1613102886}]},"public_addr":"172.21.15.27:6809/1613102886","cluster_addr":"172.21.15.27:6811/1613102886","heartbeat_back_addr":"172.21.15.27:6815/1613102886","heartbeat_front_addr":"172.21.15.27:6813/1613102886","state":["exists","up"]},{"osd":2,"uuid":"fb12dcc8-bdd0-42b8-b46d-8b10106224bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6825","nonce":3534723355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6827","nonce":3534723355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6831","nonce":3534723355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6829","nonce":3534723355}]},"public_addr":"172.21.15.27:6825/3534723355","cluster_addr":"172.21.15.27:6827/3534723355","heartbeat_back_addr":"172.21.15.27:6831/3534723355","heartbeat_front_addr":"172.21.15.27:6829/3534723355","state":["exists","up"]},{"osd":3,"uuid":"d74d3f49-9c24-4b9f-98e2-3ecd7b5ef3b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6801","nonce":80661598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6803","nonce":80661598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6807","nonce":80661598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6805","nonce":80661598}]},"public_addr":"172.21.15.27:6801/80661598","cluster_addr":"172.21.15.27:6803/80661598","heartbeat_back_addr":"172.21.15.27:6807/80661598","heartbeat_front_addr":"172.21.15.27:6805/80661598","state":["exists","up"]},{"osd":4,"uuid":"2bf123f2-1b0d-41c4-aa3b-805bab267e59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6817","nonce":3943189827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6819","nonce":3943189827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6823","nonce":3943189827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6821","nonce":3943189827}]},"public_addr":"172.21.15.45:6817/3943189827","cluster_addr":"172.21.15.45:6819/3943189827","heartbeat_back_addr":"172.21.15.45:6823/3943189827","heartbeat_front_addr":"172.21.15.45:6821/3943189827","state":["exists","up"]},{"osd":5,"uuid":"c49ba5da-91c6-4a86-a97b-4aba89c253ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6809","nonce":3620760330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6811","nonce":3620760330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6815","nonce":3620760330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6813","nonce":3620760330}]},"public_addr":"172.21.15.45:6809/3620760330","cluster_addr":"172.21.15.45:6811/3620760330","heartbeat_back_addr":"172.21.15.45:6815/3620760330","heartbeat_front_addr":"172.21.15.45:6813/3620760330","state":["exists","up"]},{"osd":6,"uuid":"b0123475-89f7-4a34-9e33-d797b99dd9dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6801","nonce":1989083510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6803","nonce":1989083510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6807","nonce":1989083510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6805","nonce":1989083510}]},"public_addr":"172.21.15.45:6801/1989083510","cluster_addr":"172.21.15.45:6803/1989083510","heartbeat_back_addr":"172.21.15.45:6807/1989083510","heartbeat_front_addr":"172.21.15.45:6805/1989083510","state":["exists","up"]},{"osd":7,"uuid":"4dad773d-b7d8-46fc-8a10-a972aad27b29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6825","nonce":1513300294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6827","nonce":1513300294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6831","nonce":1513300294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6829","nonce":1513300294}]},"public_addr":"172.21.15.45:6825/1513300294","cluster_addr":"172.21.15.45:6827/1513300294","heartbeat_back_addr":"172.21.15.45:6831/1513300294","heartbeat_front_addr":"172.21.15.45:6829/1513300294","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.881855+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:17.133252+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.735339+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.500742+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.477571+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:17.167667+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.470291+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.873743+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]},{"pool":19,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T03:23:10.282 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.282+0000 7fb28d2d0640 1 -- 172.21.15.27:0/1445002913 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fb27406c610 msgr2=0x7fb27406ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:10.282 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.282+0000 7fb28d2d0640 1 --2- 172.21.15.27:0/1445002913 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fb27406c610 0x7fb27406ead0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fb278009a80 tx=0x7fb278008040 comp rx=0 tx=0).stop 2024-09-12T03:23:10.283 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.282+0000 7fb28d2d0640 1 -- 172.21.15.27:0/1445002913 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb28815d8e0 msgr2=0x7fb2880ae8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:10.283 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.282+0000 7fb28d2d0640 1 --2- 172.21.15.27:0/1445002913 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb28815d8e0 0x7fb2880ae8d0 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7fb28000a8f0 tx=0x7fb28806dfa0 comp rx=0 tx=0).stop 2024-09-12T03:23:10.283 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.282+0000 7fb28d2d0640 1 -- 172.21.15.27:0/1445002913 shutdown_connections 2024-09-12T03:23:10.283 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.282+0000 7fb28d2d0640 1 --2- 172.21.15.27:0/1445002913 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fb27406c610 0x7fb27406ead0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:10.283 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.282+0000 7fb28d2d0640 1 --2- 172.21.15.27:0/1445002913 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb28815d8e0 0x7fb2880ae8d0 unknown :-1 s=CLOSED pgs=1477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:10.283 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.282+0000 7fb28d2d0640 1 --2- 172.21.15.27:0/1445002913 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb2880caff0 0x7fb28815d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:10.283 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.282+0000 7fb28d2d0640 1 --2- 172.21.15.27:0/1445002913 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb2700051c0 0x7fb288156ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:10.283 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.282+0000 7fb28d2d0640 1 -- 172.21.15.27:0/1445002913 >> 172.21.15.27:0/1445002913 conn(0x7fb288062c60 msgr2=0x7fb2880b0cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:10.283 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.282+0000 7fb28d2d0640 1 -- 172.21.15.27:0/1445002913 shutdown_connections 2024-09-12T03:23:10.283 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.282+0000 7fb28d2d0640 1 -- 172.21.15.27:0/1445002913 wait complete. 2024-09-12T03:23:10.294 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-12T03:23:10.397 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.397+0000 7fce3e69b640 1 Processor -- start 2024-09-12T03:23:10.397 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.397+0000 7fce3e69b640 1 -- start start 2024-09-12T03:23:10.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.398+0000 7fce3e69b640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fce380b1ca0 0x7fce3815f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:10.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.398+0000 7fce3e69b640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fce3806f360 0x7fce380cef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:10.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.398+0000 7fce3e69b640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fce3815fd70 con 0x7fce380cf490 2024-09-12T03:23:10.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.398+0000 7fce3e69b640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fce380d22f0 con 0x7fce3806f360 2024-09-12T03:23:10.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.398+0000 7fce3e69b640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fce3806cc40 con 0x7fce380b1ca0 2024-09-12T03:23:10.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.398+0000 7fce37fff640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fce380b1ca0 0x7fce3815f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:10.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.398+0000 7fce377fe640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fce3806f360 0x7fce380cef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:10.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.398+0000 7fce3cc38640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fce380cf490 0x7fce380d2780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.27:50860/0 (socket says 172.21.15.27:50860) 2024-09-12T03:23:10.400 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.398+0000 7fce377fe640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fce3806f360 0x7fce380cef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.27:52704/0 (socket says 172.21.15.27:52704) 2024-09-12T03:23:10.400 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.398+0000 7fce37fff640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fce380b1ca0 0x7fce3815f830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3301/0 says I am v2:172.21.15.27:36788/0 (socket says 172.21.15.27:36788) 2024-09-12T03:23:10.400 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.399+0000 7fce3cc38640 1 -- 172.21.15.27:0/4150003402 learned_addr learned my addr 172.21.15.27:0/4150003402 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:23:10.400 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.399+0000 7fce36ffd640 1 -- 172.21.15.27:0/4150003402 <== mon.1 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 868233122 0 0) 0x7fce30001490 con 0x7fce380cf490 2024-09-12T03:23:10.400 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.399+0000 7fce36ffd640 1 -- 172.21.15.27:0/4150003402 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fce20003520 con 0x7fce380cf490 2024-09-12T03:23:10.400 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.399+0000 7fce377fe640 1 -- 172.21.15.27:0/4150003402 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fce380b1ca0 msgr2=0x7fce3815f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:10.400 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.399+0000 7fce377fe640 1 --2- 172.21.15.27:0/4150003402 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fce380b1ca0 0x7fce3815f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:10.400 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.399+0000 7fce377fe640 1 -- 172.21.15.27:0/4150003402 >> v1:172.21.15.45:6789/0 conn(0x7fce380cf490 legacy=0x7fce380d2780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:10.400 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.399+0000 7fce377fe640 1 -- 172.21.15.27:0/4150003402 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce380d1ce0 con 0x7fce3806f360 2024-09-12T03:23:10.400 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.399+0000 7fce37fff640 1 --2- 172.21.15.27:0/4150003402 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fce380b1ca0 0x7fce3815f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:23:10.402 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.400+0000 7fce377fe640 1 --2- 172.21.15.27:0/4150003402 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fce3806f360 0x7fce380cef50 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fce2c0097a0 tx=0x7fce2c02a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8e91beee4ad79a3a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:10.402 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.400+0000 7fce36ffd640 1 -- 172.21.15.27:0/4150003402 <== mon.1 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce2c037070 con 0x7fce3806f360 2024-09-12T03:23:10.402 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.400+0000 7fce36ffd640 1 -- 172.21.15.27:0/4150003402 <== mon.1 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce2c032440 con 0x7fce3806f360 2024-09-12T03:23:10.402 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.400+0000 7fce36ffd640 1 -- 172.21.15.27:0/4150003402 <== mon.1 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce2c0327f0 con 0x7fce3806f360 2024-09-12T03:23:10.402 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.400+0000 7fce3e69b640 1 -- 172.21.15.27:0/4150003402 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fce3806f360 msgr2=0x7fce380cef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:10.402 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.400+0000 7fce3e69b640 1 --2- 172.21.15.27:0/4150003402 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fce3806f360 0x7fce380cef50 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fce2c0097a0 tx=0x7fce2c02a180 comp rx=0 tx=0).stop 2024-09-12T03:23:10.402 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.400+0000 7fce3e69b640 1 -- 172.21.15.27:0/4150003402 shutdown_connections 2024-09-12T03:23:10.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.400+0000 7fce3e69b640 1 --2- 172.21.15.27:0/4150003402 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fce3806f360 0x7fce380cef50 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:10.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.400+0000 7fce3e69b640 1 --2- 172.21.15.27:0/4150003402 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fce380b1ca0 0x7fce3815f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:10.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.400+0000 7fce3e69b640 1 -- 172.21.15.27:0/4150003402 >> 172.21.15.27:0/4150003402 conn(0x7fce38073800 msgr2=0x7fce3809ba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:10.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.400+0000 7fce3e69b640 1 -- 172.21.15.27:0/4150003402 shutdown_connections 2024-09-12T03:23:10.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.401+0000 7fce3e69b640 1 -- 172.21.15.27:0/4150003402 wait complete. 2024-09-12T03:23:10.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.402+0000 7fce3e69b640 1 Processor -- start 2024-09-12T03:23:10.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.402+0000 7fce3e69b640 1 -- start start 2024-09-12T03:23:10.406 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.403+0000 7fce3e69b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fce3806f360 0x7fce381e7da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:10.406 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.403+0000 7fce3e69b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fce380b1ca0 0x7fce38150360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:10.406 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.403+0000 7fce3e69b640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fce380cf490 0x7fce380ce2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:10.406 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.403+0000 7fce3e69b640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fce3806d1c0 con 0x7fce3806f360 2024-09-12T03:23:10.406 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.403+0000 7fce3e69b640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fce380d1f80 con 0x7fce380b1ca0 2024-09-12T03:23:10.406 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.403+0000 7fce3e69b640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fce3806d340 con 0x7fce380cf490 2024-09-12T03:23:10.406 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.403+0000 7fce37fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fce3806f360 0x7fce381e7da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:10.406 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.403+0000 7fce37fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fce3806f360 0x7fce381e7da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59020/0 (socket says 172.21.15.27:59020) 2024-09-12T03:23:10.406 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.403+0000 7fce37fff640 1 -- 172.21.15.27:0/1313036219 learned_addr learned my addr 172.21.15.27:0/1313036219 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:23:10.406 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.403+0000 7fce377fe640 1 --2- 172.21.15.27:0/1313036219 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fce380b1ca0 0x7fce38150360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:10.406 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.403+0000 7fce3cc38640 1 --2- 172.21.15.27:0/1313036219 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fce380cf490 0x7fce380ce2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:10.406 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.403+0000 7fce37fff640 1 -- 172.21.15.27:0/1313036219 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fce380cf490 msgr2=0x7fce380ce2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:10.406 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.403+0000 7fce37fff640 1 --2- 172.21.15.27:0/1313036219 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fce380cf490 0x7fce380ce2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:10.407 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.403+0000 7fce37fff640 1 -- 172.21.15.27:0/1313036219 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fce380b1ca0 msgr2=0x7fce38150360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:10.407 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.403+0000 7fce37fff640 1 --2- 172.21.15.27:0/1313036219 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fce380b1ca0 0x7fce38150360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:10.407 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.403+0000 7fce37fff640 1 -- 172.21.15.27:0/1313036219 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce381508a0 con 0x7fce3806f360 2024-09-12T03:23:10.407 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.403+0000 7fce37fff640 1 --2- 172.21.15.27:0/1313036219 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fce3806f360 0x7fce381e7da0 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7fce2800ec10 tx=0x7fce2800c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:10.407 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.403+0000 7fce34ff9640 1 -- 172.21.15.27:0/1313036219 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce28017070 con 0x7fce3806f360 2024-09-12T03:23:10.407 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.403+0000 7fce34ff9640 1 -- 172.21.15.27:0/1313036219 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce2800cc30 con 0x7fce3806f360 2024-09-12T03:23:10.407 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.404+0000 7fce34ff9640 1 -- 172.21.15.27:0/1313036219 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce28013470 con 0x7fce3806f360 2024-09-12T03:23:10.407 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.404+0000 7fce3e69b640 1 -- 172.21.15.27:0/1313036219 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce38150b40 con 0x7fce3806f360 2024-09-12T03:23:10.407 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.404+0000 7fce3e69b640 1 -- 172.21.15.27:0/1313036219 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce38151100 con 0x7fce3806f360 2024-09-12T03:23:10.407 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.405+0000 7fce3e69b640 1 -- 172.21.15.27:0/1313036219 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce380cfc40 con 0x7fce3806f360 2024-09-12T03:23:10.407 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.405+0000 7fce34ff9640 1 -- 172.21.15.27:0/1313036219 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 24) ==== 91212+0+0 (secure 0 0 0) 0x7fce2801a070 con 0x7fce3806f360 2024-09-12T03:23:10.407 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.406+0000 7fce34ff9640 1 --2- 172.21.15.27:0/1313036219 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fce2406c610 0x7fce2406ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:10.407 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.406+0000 7fce377fe640 1 --2- 172.21.15.27:0/1313036219 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fce2406c610 0x7fce2406ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:10.407 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.406+0000 7fce34ff9640 1 -- 172.21.15.27:0/1313036219 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 5765+0+0 (secure 0 0 0) 0x7fce28089a80 con 0x7fce3806f360 2024-09-12T03:23:10.408 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.406+0000 7fce377fe640 1 --2- 172.21.15.27:0/1313036219 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fce2406c610 0x7fce2406ead0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fce2c034000 tx=0x7fce2c02e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:10.410 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.410+0000 7fce34ff9640 1 -- 172.21.15.27:0/1313036219 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fce280596d0 con 0x7fce3806f360 2024-09-12T03:23:10.620 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.620+0000 7fce3e69b640 1 -- 172.21.15.27:0/1313036219 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fce380b34a0 con 0x7fce3806f360 2024-09-12T03:23:10.621 INFO:tasks.ceph.mon.a.smithi027.stderr:2024-09-12T03:23:10.621+0000 7f959bfd9640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-12T03:23:10.624 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.624+0000 7fce34ff9640 1 -- 172.21.15.27:0/1313036219 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v84) ==== 161+0+0 (secure 0 0 0) 0x7fce2805d9d0 con 0x7fce3806f360 2024-09-12T03:23:10.624 INFO:teuthology.orchestra.run.smithi027.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-12T03:23:10.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.626+0000 7fce3e69b640 1 -- 172.21.15.27:0/1313036219 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fce2406c610 msgr2=0x7fce2406ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:10.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.626+0000 7fce3e69b640 1 --2- 172.21.15.27:0/1313036219 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fce2406c610 0x7fce2406ead0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fce2c034000 tx=0x7fce2c02e000 comp rx=0 tx=0).stop 2024-09-12T03:23:10.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.626+0000 7fce3e69b640 1 -- 172.21.15.27:0/1313036219 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fce3806f360 msgr2=0x7fce381e7da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:10.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.626+0000 7fce3e69b640 1 --2- 172.21.15.27:0/1313036219 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fce3806f360 0x7fce381e7da0 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7fce2800ec10 tx=0x7fce2800c520 comp rx=0 tx=0).stop 2024-09-12T03:23:10.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.626+0000 7fce3e69b640 1 -- 172.21.15.27:0/1313036219 shutdown_connections 2024-09-12T03:23:10.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.626+0000 7fce3e69b640 1 --2- 172.21.15.27:0/1313036219 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fce380cf490 0x7fce380ce2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:10.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.626+0000 7fce3e69b640 1 --2- 172.21.15.27:0/1313036219 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fce380b1ca0 0x7fce38150360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:10.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.626+0000 7fce3e69b640 1 --2- 172.21.15.27:0/1313036219 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fce2406c610 0x7fce2406ead0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:10.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.626+0000 7fce3e69b640 1 --2- 172.21.15.27:0/1313036219 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fce3806f360 0x7fce381e7da0 unknown :-1 s=CLOSED pgs=1478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:10.628 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.626+0000 7fce3e69b640 1 -- 172.21.15.27:0/1313036219 >> 172.21.15.27:0/1313036219 conn(0x7fce38073800 msgr2=0x7fce38070b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:10.628 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.627+0000 7fce3e69b640 1 -- 172.21.15.27:0/1313036219 shutdown_connections 2024-09-12T03:23:10.628 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.627+0000 7fce3e69b640 1 -- 172.21.15.27:0/1313036219 wait complete. 2024-09-12T03:23:10.640 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-12T03:23:10.666 INFO:tasks.ceph.mds.a.smithi027.stdout:starting mds.a at 2024-09-12T03:23:10.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.745+0000 7fee80cc9640 1 Processor -- start 2024-09-12T03:23:10.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.745+0000 7fee80cc9640 1 -- start start 2024-09-12T03:23:10.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.746+0000 7fee80cc9640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fee7c0b6540 0x7fee7c0af370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:10.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.746+0000 7fee80cc9640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fee7c0b5940 con 0x7fee7c06d730 2024-09-12T03:23:10.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.746+0000 7fee80cc9640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fee7c0af950 con 0x7fee7c15e900 2024-09-12T03:23:10.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.746+0000 7fee80cc9640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fee7c15ece0 con 0x7fee7c0b6540 2024-09-12T03:23:10.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.746+0000 7fee7ad9d640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fee7c06d730 0x7fee7c06ee80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:48600/0 (socket says 172.21.15.27:48600) 2024-09-12T03:23:10.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.746+0000 7fee7ad9d640 1 -- 172.21.15.27:0/2325727744 learned_addr learned my addr 172.21.15.27:0/2325727744 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:23:10.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.746+0000 7fee79d9b640 1 --2- 172.21.15.27:0/2325727744 >> v2:172.21.15.45:3300/0 conn(0x7fee7c0b6540 0x7fee7c0af370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:10.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.746+0000 7fee7959a640 1 -- 172.21.15.27:0/2325727744 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3991783050 0 0) 0x7fee6c001490 con 0x7fee7c06d730 2024-09-12T03:23:10.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.746+0000 7fee7959a640 1 -- 172.21.15.27:0/2325727744 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fee64003520 con 0x7fee7c06d730 2024-09-12T03:23:10.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.747+0000 7fee7959a640 1 -- 172.21.15.27:0/2325727744 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1645309033 0 0) 0x7fee64003520 con 0x7fee7c06d730 2024-09-12T03:23:10.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.747+0000 7fee7959a640 1 -- 172.21.15.27:0/2325727744 >> v2:172.21.15.45:3300/0 conn(0x7fee7c0b6540 msgr2=0x7fee7c0af370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:10.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.747+0000 7fee7959a640 1 --2- 172.21.15.27:0/2325727744 >> v2:172.21.15.45:3300/0 conn(0x7fee7c0b6540 0x7fee7c0af370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:10.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.747+0000 7fee7959a640 1 -- 172.21.15.27:0/2325727744 >> v1:172.21.15.45:6789/0 conn(0x7fee7c15e900 legacy=0x7fee7c0b5e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:10.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.747+0000 7fee7959a640 1 -- 172.21.15.27:0/2325727744 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee7c0afc30 con 0x7fee7c06d730 2024-09-12T03:23:10.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.747+0000 7fee7959a640 1 -- 172.21.15.27:0/2325727744 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 847755213 0 0) 0x7fee6c001bc0 con 0x7fee7c06d730 2024-09-12T03:23:10.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.747+0000 7fee7959a640 1 -- 172.21.15.27:0/2325727744 >> v1:172.21.15.27:6789/0 conn(0x7fee7c06d730 legacy=0x7fee7c06ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:10.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.747+0000 7fee7959a640 1 --2- 172.21.15.27:0/2325727744 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fee640039b0 0x7fee64003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:10.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.747+0000 7fee7959a640 1 --2- 172.21.15.27:0/2325727744 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fee64004800 0x7fee64004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:10.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.747+0000 7fee7959a640 1 --2- 172.21.15.27:0/2325727744 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fee640051c0 0x7fee6400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:10.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.747+0000 7fee7959a640 1 -- 172.21.15.27:0/2325727744 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fee6400c030 con 0x7fee64004800 2024-09-12T03:23:10.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.747+0000 7fee7959a640 1 -- 172.21.15.27:0/2325727744 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fee6400c1e0 con 0x7fee640051c0 2024-09-12T03:23:10.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.747+0000 7fee7959a640 1 -- 172.21.15.27:0/2325727744 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fee6400c3e0 con 0x7fee640039b0 2024-09-12T03:23:10.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.747+0000 7fee79d9b640 1 --2- 172.21.15.27:0/2325727744 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fee64004800 0x7fee64004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:10.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.747+0000 7fee7ad9d640 1 --2- 172.21.15.27:0/2325727744 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fee640051c0 0x7fee6400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:10.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.747+0000 7fee7a59c640 1 --2- 172.21.15.27:0/2325727744 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fee640039b0 0x7fee64003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:10.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.747+0000 7fee79d9b640 1 -- 172.21.15.27:0/2325727744 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fee640039b0 msgr2=0x7fee64003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:10.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.747+0000 7fee79d9b640 1 --2- 172.21.15.27:0/2325727744 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fee640039b0 0x7fee64003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:10.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.747+0000 7fee79d9b640 1 -- 172.21.15.27:0/2325727744 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fee640051c0 msgr2=0x7fee6400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:10.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.747+0000 7fee79d9b640 1 --2- 172.21.15.27:0/2325727744 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fee640051c0 0x7fee6400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:10.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.747+0000 7fee79d9b640 1 -- 172.21.15.27:0/2325727744 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee64003520 con 0x7fee64004800 2024-09-12T03:23:10.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.747+0000 7fee7a59c640 1 --2- 172.21.15.27:0/2325727744 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fee640039b0 0x7fee64003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:23:10.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.747+0000 7fee7ad9d640 1 --2- 172.21.15.27:0/2325727744 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fee640051c0 0x7fee6400b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:23:10.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.747+0000 7fee7ad9d640 1 -- 172.21.15.27:0/2325727744 reap_dead start 2024-09-12T03:23:10.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.748+0000 7fee79d9b640 1 --2- 172.21.15.27:0/2325727744 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fee64004800 0x7fee64004c80 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7fee70009ac0 tx=0x7fee64005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46950f14adcf286c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:10.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.748+0000 7fee7959a640 1 -- 172.21.15.27:0/2325727744 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee70036070 con 0x7fee64004800 2024-09-12T03:23:10.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.748+0000 7fee7959a640 1 -- 172.21.15.27:0/2325727744 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee70031400 con 0x7fee64004800 2024-09-12T03:23:10.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.748+0000 7fee7959a640 1 -- 172.21.15.27:0/2325727744 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee70009690 con 0x7fee64004800 2024-09-12T03:23:10.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.748+0000 7fee80cc9640 1 -- 172.21.15.27:0/2325727744 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fee64004800 msgr2=0x7fee64004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:10.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.748+0000 7fee80cc9640 1 --2- 172.21.15.27:0/2325727744 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fee64004800 0x7fee64004c80 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7fee70009ac0 tx=0x7fee64005830 comp rx=0 tx=0).stop 2024-09-12T03:23:10.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.749+0000 7fee80cc9640 1 -- 172.21.15.27:0/2325727744 shutdown_connections 2024-09-12T03:23:10.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.749+0000 7fee80cc9640 1 --2- 172.21.15.27:0/2325727744 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fee64004800 0x7fee64004c80 unknown :-1 s=CLOSED pgs=1483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:10.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.749+0000 7fee80cc9640 1 -- 172.21.15.27:0/2325727744 >> 172.21.15.27:0/2325727744 conn(0x7fee7c06cc40 msgr2=0x7fee7c06d020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:10.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.749+0000 7fee80cc9640 1 -- 172.21.15.27:0/2325727744 shutdown_connections 2024-09-12T03:23:10.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.749+0000 7fee80cc9640 1 -- 172.21.15.27:0/2325727744 wait complete. 2024-09-12T03:23:10.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.750+0000 7fee80cc9640 1 Processor -- start 2024-09-12T03:23:10.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.750+0000 7fee80cc9640 1 -- start start 2024-09-12T03:23:10.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.751+0000 7fee80cc9640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fee64004800 0x7fee7c0c6f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:10.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.751+0000 7fee80cc9640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fee7c1e8570 0x7fee7c1e7010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:10.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.751+0000 7fee80cc9640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fee7c1e7550 0x7fee7c1e79d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:10.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.751+0000 7fee80cc9640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fee7c0b01f0 con 0x7fee7c1e8570 2024-09-12T03:23:10.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.751+0000 7fee80cc9640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fee7c06db40 con 0x7fee7c1e7550 2024-09-12T03:23:10.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.751+0000 7fee80cc9640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fee7c06e880 con 0x7fee64004800 2024-09-12T03:23:10.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.751+0000 7fee79d9b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fee7c1e8570 0x7fee7c1e7010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:10.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.751+0000 7fee79d9b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fee7c1e8570 0x7fee7c1e7010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59060/0 (socket says 172.21.15.27:59060) 2024-09-12T03:23:10.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.751+0000 7fee79d9b640 1 -- 172.21.15.27:0/3885690896 learned_addr learned my addr 172.21.15.27:0/3885690896 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:23:10.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.751+0000 7fee79d9b640 1 -- 172.21.15.27:0/3885690896 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fee64004800 msgr2=0x7fee7c0c6f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:10.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.751+0000 7fee79d9b640 1 --2- 172.21.15.27:0/3885690896 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fee64004800 0x7fee7c0c6f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:10.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.751+0000 7fee79d9b640 1 -- 172.21.15.27:0/3885690896 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fee7c1e7550 msgr2=0x7fee7c1e79d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:10.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.751+0000 7fee79d9b640 1 --2- 172.21.15.27:0/3885690896 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fee7c1e7550 0x7fee7c1e79d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:10.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.751+0000 7fee79d9b640 1 -- 172.21.15.27:0/3885690896 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee7c0c7640 con 0x7fee7c1e8570 2024-09-12T03:23:10.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.751+0000 7fee79d9b640 1 --2- 172.21.15.27:0/3885690896 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fee7c1e8570 0x7fee7c1e7010 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7fee70034040 tx=0x7fee70004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:10.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.751+0000 7fee6affd640 1 -- 172.21.15.27:0/3885690896 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee7003f020 con 0x7fee7c1e8570 2024-09-12T03:23:10.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.751+0000 7fee6affd640 1 -- 172.21.15.27:0/3885690896 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee70037040 con 0x7fee7c1e8570 2024-09-12T03:23:10.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.751+0000 7fee80cc9640 1 -- 172.21.15.27:0/3885690896 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee7c0c7920 con 0x7fee7c1e8570 2024-09-12T03:23:10.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.751+0000 7fee6affd640 1 -- 172.21.15.27:0/3885690896 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee70036070 con 0x7fee7c1e8570 2024-09-12T03:23:10.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.751+0000 7fee80cc9640 1 -- 172.21.15.27:0/3885690896 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee7c0c7ee0 con 0x7fee7c1e8570 2024-09-12T03:23:10.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.752+0000 7fee80cc9640 1 -- 172.21.15.27:0/3885690896 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee7c06beb0 con 0x7fee7c1e8570 2024-09-12T03:23:10.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.753+0000 7fee6affd640 1 -- 172.21.15.27:0/3885690896 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 24) ==== 91212+0+0 (secure 0 0 0) 0x7fee70042020 con 0x7fee7c1e8570 2024-09-12T03:23:10.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.753+0000 7fee6affd640 1 --2- 172.21.15.27:0/3885690896 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fee5c06c610 0x7fee5c06ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:10.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.754+0000 7fee6affd640 1 -- 172.21.15.27:0/3885690896 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(139..139 src has 1..139) ==== 5851+0+0 (secure 0 0 0) 0x7fee700a98d0 con 0x7fee7c1e8570 2024-09-12T03:23:10.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.754+0000 7fee7a59c640 1 --2- 172.21.15.27:0/3885690896 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fee5c06c610 0x7fee5c06ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:10.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.754+0000 7fee7a59c640 1 --2- 172.21.15.27:0/3885690896 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fee5c06c610 0x7fee5c06ead0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fee74007fd0 tx=0x7fee74009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:10.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.758+0000 7fee6affd640 1 -- 172.21.15.27:0/3885690896 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fee70049030 con 0x7fee7c1e8570 2024-09-12T03:23:10.969 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:10.969+0000 7fee80cc9640 1 -- 172.21.15.27:0/3885690896 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fee7c0008d0 con 0x7fee7c1e8570 2024-09-12T03:23:11.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:11.627+0000 7fee6affd640 1 -- 172.21.15.27:0/3885690896 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v85) ==== 104+0+0 (secure 0 0 0) 0x7fee70079c30 con 0x7fee7c1e8570 2024-09-12T03:23:11.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:11.630+0000 7fee80cc9640 1 -- 172.21.15.27:0/3885690896 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fee5c06c610 msgr2=0x7fee5c06ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:11.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:11.630+0000 7fee80cc9640 1 --2- 172.21.15.27:0/3885690896 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fee5c06c610 0x7fee5c06ead0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fee74007fd0 tx=0x7fee74009040 comp rx=0 tx=0).stop 2024-09-12T03:23:11.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:11.630+0000 7fee80cc9640 1 -- 172.21.15.27:0/3885690896 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fee7c1e8570 msgr2=0x7fee7c1e7010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:11.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:11.630+0000 7fee80cc9640 1 --2- 172.21.15.27:0/3885690896 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fee7c1e8570 0x7fee7c1e7010 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7fee70034040 tx=0x7fee70004730 comp rx=0 tx=0).stop 2024-09-12T03:23:11.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:11.630+0000 7fee80cc9640 1 -- 172.21.15.27:0/3885690896 shutdown_connections 2024-09-12T03:23:11.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:11.630+0000 7fee80cc9640 1 --2- 172.21.15.27:0/3885690896 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fee7c1e7550 0x7fee7c1e79d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:11.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:11.630+0000 7fee80cc9640 1 --2- 172.21.15.27:0/3885690896 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fee5c06c610 0x7fee5c06ead0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:11.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:11.630+0000 7fee80cc9640 1 --2- 172.21.15.27:0/3885690896 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fee7c1e8570 0x7fee7c1e7010 unknown :-1 s=CLOSED pgs=1484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:11.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:11.630+0000 7fee80cc9640 1 --2- 172.21.15.27:0/3885690896 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fee64004800 0x7fee7c0c6f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:11.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:11.630+0000 7fee80cc9640 1 -- 172.21.15.27:0/3885690896 >> 172.21.15.27:0/3885690896 conn(0x7fee7c06cc40 msgr2=0x7fee7c15be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:11.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:11.630+0000 7fee80cc9640 1 -- 172.21.15.27:0/3885690896 shutdown_connections 2024-09-12T03:23:11.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:11.630+0000 7fee80cc9640 1 -- 172.21.15.27:0/3885690896 wait complete. 2024-09-12T03:23:11.643 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-12T03:23:11.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:11.792+0000 7f11134c9640 1 Processor -- start 2024-09-12T03:23:11.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:11.793+0000 7f11134c9640 1 -- start start 2024-09-12T03:23:11.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:11.794+0000 7f11134c9640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f110c15e530 0x7f110c0aff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:11.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:11.794+0000 7f11134c9640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f110c06dc20 con 0x7f110c15e910 2024-09-12T03:23:11.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:11.794+0000 7f11134c9640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f110c0b0500 con 0x7f110c072a00 2024-09-12T03:23:11.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:11.794+0000 7f11134c9640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f110c0b07e0 con 0x7f110c15e530 2024-09-12T03:23:11.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:11.794+0000 7f1111265640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f110c15e530 0x7f110c0aff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:11.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:11.794+0000 7f1110a64640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f110c15e910 0x7f110c0722f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:48610/0 (socket says 172.21.15.27:48610) 2024-09-12T03:23:11.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:11.794+0000 7f1110a64640 1 -- 172.21.15.27:0/479841787 learned_addr learned my addr 172.21.15.27:0/479841787 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:23:11.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:11.794+0000 7f1111265640 1 -- 172.21.15.27:0/479841787 >> v1:172.21.15.45:6789/0 conn(0x7f110c072a00 legacy=0x7f110c06e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:11.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:11.794+0000 7f1111265640 1 -- 172.21.15.27:0/479841787 >> v1:172.21.15.27:6789/0 conn(0x7f110c15e910 legacy=0x7f110c0722f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:11.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:11.794+0000 7f1111265640 1 -- 172.21.15.27:0/479841787 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f110c06dec0 con 0x7f110c15e530 2024-09-12T03:23:11.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:11.795+0000 7f1111265640 1 --2- 172.21.15.27:0/479841787 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f110c15e530 0x7f110c0aff20 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7f11080097a0 tx=0x7f110802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=532cd73ffd20684 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:11.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:11.795+0000 7f10fffff640 1 -- 172.21.15.27:0/479841787 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1108037070 con 0x7f110c15e530 2024-09-12T03:23:11.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:11.795+0000 7f10fffff640 1 -- 172.21.15.27:0/479841787 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1108032440 con 0x7f110c15e530 2024-09-12T03:23:11.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:11.795+0000 7f10fffff640 1 -- 172.21.15.27:0/479841787 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1108032810 con 0x7f110c15e530 2024-09-12T03:23:11.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:11.795+0000 7f11134c9640 1 -- 172.21.15.27:0/479841787 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f110c15e530 msgr2=0x7f110c0aff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:11.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:11.795+0000 7f11134c9640 1 --2- 172.21.15.27:0/479841787 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f110c15e530 0x7f110c0aff20 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7f11080097a0 tx=0x7f110802a1e0 comp rx=0 tx=0).stop 2024-09-12T03:23:11.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:11.796+0000 7f11134c9640 1 -- 172.21.15.27:0/479841787 shutdown_connections 2024-09-12T03:23:11.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:11.796+0000 7f11134c9640 1 --2- 172.21.15.27:0/479841787 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f110c15e530 0x7f110c0aff20 unknown :-1 s=CLOSED pgs=1486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:11.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:11.796+0000 7f11134c9640 1 -- 172.21.15.27:0/479841787 >> 172.21.15.27:0/479841787 conn(0x7f110c06bb30 msgr2=0x7f110c06bf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:11.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:11.796+0000 7f11134c9640 1 -- 172.21.15.27:0/479841787 shutdown_connections 2024-09-12T03:23:11.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:11.796+0000 7f11134c9640 1 -- 172.21.15.27:0/479841787 wait complete. 2024-09-12T03:23:11.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:11.797+0000 7f11134c9640 1 Processor -- start 2024-09-12T03:23:11.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:11.797+0000 7f11134c9640 1 -- start start 2024-09-12T03:23:11.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:11.798+0000 7f11134c9640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f110c072a00 0x7f110c153b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:11.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:11.798+0000 7f11134c9640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f110c15e530 0x7f110c1541f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:11.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:11.798+0000 7f11134c9640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f110c15e910 0x7f110c154730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:11.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:11.798+0000 7f11134c9640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f110c0aeb80 con 0x7f110c15e530 2024-09-12T03:23:11.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:11.798+0000 7f11134c9640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f110c071ff0 con 0x7f110c072a00 2024-09-12T03:23:11.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:11.798+0000 7f11134c9640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f110c0aed00 con 0x7f110c15e910 2024-09-12T03:23:11.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:11.798+0000 7f1110a64640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f110c15e530 0x7f110c1541f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:11.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:11.798+0000 7f1110a64640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f110c15e530 0x7f110c1541f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59070/0 (socket says 172.21.15.27:59070) 2024-09-12T03:23:11.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:11.798+0000 7f1110a64640 1 -- 172.21.15.27:0/406677702 learned_addr learned my addr 172.21.15.27:0/406677702 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:23:11.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:11.798+0000 7f1111265640 1 --2- 172.21.15.27:0/406677702 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f110c072a00 0x7f110c153b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:11.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:11.798+0000 7f1111a66640 1 --2- 172.21.15.27:0/406677702 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f110c15e910 0x7f110c154730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:11.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:11.798+0000 7f1110a64640 1 -- 172.21.15.27:0/406677702 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f110c15e910 msgr2=0x7f110c154730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:11.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:11.798+0000 7f1110a64640 1 --2- 172.21.15.27:0/406677702 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f110c15e910 0x7f110c154730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:11.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:11.798+0000 7f1110a64640 1 -- 172.21.15.27:0/406677702 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f110c072a00 msgr2=0x7f110c153b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:11.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:11.798+0000 7f1110a64640 1 --2- 172.21.15.27:0/406677702 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f110c072a00 0x7f110c153b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:11.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:11.798+0000 7f1110a64640 1 -- 172.21.15.27:0/406677702 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f110c0ce590 con 0x7f110c15e530 2024-09-12T03:23:11.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:11.798+0000 7f1111265640 1 --2- 172.21.15.27:0/406677702 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f110c072a00 0x7f110c153b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:23:11.801 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:11.798+0000 7f1110a64640 1 --2- 172.21.15.27:0/406677702 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f110c15e530 0x7f110c1541f0 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7f110000c3c0 tx=0x7f110000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:11.801 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:11.798+0000 7f10fdffb640 1 -- 172.21.15.27:0/406677702 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1100017070 con 0x7f110c15e530 2024-09-12T03:23:11.801 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:11.798+0000 7f10fdffb640 1 -- 172.21.15.27:0/406677702 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1100005ce0 con 0x7f110c15e530 2024-09-12T03:23:11.801 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:11.798+0000 7f10fdffb640 1 -- 172.21.15.27:0/406677702 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11000072f0 con 0x7f110c15e530 2024-09-12T03:23:11.801 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:11.798+0000 7f11134c9640 1 -- 172.21.15.27:0/406677702 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f110c0ce870 con 0x7f110c15e530 2024-09-12T03:23:11.801 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:11.799+0000 7f11134c9640 1 -- 172.21.15.27:0/406677702 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f110c0cf660 con 0x7f110c15e530 2024-09-12T03:23:11.801 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:11.799+0000 7f11134c9640 1 -- 172.21.15.27:0/406677702 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f110c15a3b0 con 0x7f110c15e530 2024-09-12T03:23:11.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:11.800+0000 7f10fdffb640 1 -- 172.21.15.27:0/406677702 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 24) ==== 91212+0+0 (secure 0 0 0) 0x7f110001a070 con 0x7f110c15e530 2024-09-12T03:23:11.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:11.800+0000 7f10fdffb640 1 --2- 172.21.15.27:0/406677702 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f10f006c610 0x7f10f006ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:11.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:11.801+0000 7f10fdffb640 1 -- 172.21.15.27:0/406677702 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(139..139 src has 1..139) ==== 5851+0+0 (secure 0 0 0) 0x7f1100088b90 con 0x7f110c15e530 2024-09-12T03:23:11.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:11.801+0000 7f1111265640 1 --2- 172.21.15.27:0/406677702 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f10f006c610 0x7f10f006ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:11.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:11.801+0000 7f1111265640 1 --2- 172.21.15.27:0/406677702 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f10f006c610 0x7f10f006ead0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f1108034000 tx=0x7f110802e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:11.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:11.805+0000 7f10fdffb640 1 -- 172.21.15.27:0/406677702 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f1100058e30 con 0x7f110c15e530 2024-09-12T03:23:11.982 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:11.982+0000 7f11134c9640 1 -- 172.21.15.27:0/406677702 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f110c0aff20 con 0x7f110c15e530 2024-09-12T03:23:12.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:12.637+0000 7f10fdffb640 1 -- 172.21.15.27:0/406677702 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v140) ==== 181+0+0 (secure 0 0 0) 0x7f1100058730 con 0x7f110c15e530 2024-09-12T03:23:12.637 INFO:teuthology.orchestra.run.smithi027.stderr:pool 'cephfs_metadata' removed 2024-09-12T03:23:12.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:12.640+0000 7f11134c9640 1 -- 172.21.15.27:0/406677702 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f10f006c610 msgr2=0x7f10f006ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:12.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:12.640+0000 7f11134c9640 1 --2- 172.21.15.27:0/406677702 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f10f006c610 0x7f10f006ead0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f1108034000 tx=0x7f110802e000 comp rx=0 tx=0).stop 2024-09-12T03:23:12.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:12.640+0000 7f11134c9640 1 -- 172.21.15.27:0/406677702 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f110c15e530 msgr2=0x7f110c1541f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:12.641 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:12.640+0000 7f11134c9640 1 --2- 172.21.15.27:0/406677702 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f110c15e530 0x7f110c1541f0 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7f110000c3c0 tx=0x7f110000c880 comp rx=0 tx=0).stop 2024-09-12T03:23:12.641 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:12.640+0000 7f11134c9640 1 -- 172.21.15.27:0/406677702 shutdown_connections 2024-09-12T03:23:12.641 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:12.640+0000 7f11134c9640 1 --2- 172.21.15.27:0/406677702 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f110c15e910 0x7f110c154730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:12.641 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:12.640+0000 7f11134c9640 1 --2- 172.21.15.27:0/406677702 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f10f006c610 0x7f10f006ead0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:12.641 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:12.640+0000 7f11134c9640 1 --2- 172.21.15.27:0/406677702 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f110c15e530 0x7f110c1541f0 unknown :-1 s=CLOSED pgs=1487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:12.641 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:12.640+0000 7f11134c9640 1 --2- 172.21.15.27:0/406677702 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f110c072a00 0x7f110c153b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:12.641 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:12.640+0000 7f11134c9640 1 -- 172.21.15.27:0/406677702 >> 172.21.15.27:0/406677702 conn(0x7f110c06bb30 msgr2=0x7f110c15a5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:12.641 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:12.640+0000 7f11134c9640 1 -- 172.21.15.27:0/406677702 shutdown_connections 2024-09-12T03:23:12.641 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:12.640+0000 7f11134c9640 1 -- 172.21.15.27:0/406677702 wait complete. 2024-09-12T03:23:12.656 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-12T03:23:12.772 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:12.772+0000 7fb4be52a640 1 Processor -- start 2024-09-12T03:23:12.773 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:12.772+0000 7fb4be52a640 1 -- start start 2024-09-12T03:23:12.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:12.773+0000 7fb4be52a640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb4b806e550 0x7fb4b815ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:12.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:12.773+0000 7fb4be52a640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fb4b8072610 0x7fb4b8072ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:12.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:12.773+0000 7fb4be52a640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb4b80aedd0 con 0x7fb4b80ae8d0 2024-09-12T03:23:12.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:12.773+0000 7fb4be52a640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7fb4b80af0b0 con 0x7fb4b8072610 2024-09-12T03:23:12.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:12.773+0000 7fb4be52a640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fb4b80af390 con 0x7fb4b806e550 2024-09-12T03:23:12.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:12.773+0000 7fb4bcac7640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fb4b8072610 0x7fb4b8072ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:12.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:12.773+0000 7fb4bcac7640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fb4b8072610 0x7fb4b8072ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59074/0 (socket says 172.21.15.27:59074) 2024-09-12T03:23:12.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:12.773+0000 7fb4bcac7640 1 -- 172.21.15.27:0/3672509636 learned_addr learned my addr 172.21.15.27:0/3672509636 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:23:12.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:12.774+0000 7fb4b7fff640 1 --2- 172.21.15.27:0/3672509636 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb4b806e550 0x7fb4b815ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:12.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:12.774+0000 7fb4bcac7640 1 -- 172.21.15.27:0/3672509636 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb4b806e550 msgr2=0x7fb4b815ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:12.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:12.774+0000 7fb4bcac7640 1 --2- 172.21.15.27:0/3672509636 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb4b806e550 0x7fb4b815ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:12.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:12.774+0000 7fb4bcac7640 1 -- 172.21.15.27:0/3672509636 >> v1:172.21.15.45:6789/0 conn(0x7fb4b80ae8d0 legacy=0x7fb4b806e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:12.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:12.774+0000 7fb4bcac7640 1 -- 172.21.15.27:0/3672509636 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4b80af670 con 0x7fb4b8072610 2024-09-12T03:23:12.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:12.774+0000 7fb4b7fff640 1 --2- 172.21.15.27:0/3672509636 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb4b806e550 0x7fb4b815ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:23:12.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:12.774+0000 7fb4bcac7640 1 --2- 172.21.15.27:0/3672509636 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb4b8072610 0x7fb4b8072ac0 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7fb4b0009e50 tx=0x7fb4b002a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5f9041dc719f6a5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:12.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:12.775+0000 7fb4b77fe640 1 -- 172.21.15.27:0/3672509636 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4b0036070 con 0x7fb4b8072610 2024-09-12T03:23:12.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:12.775+0000 7fb4b77fe640 1 -- 172.21.15.27:0/3672509636 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4b0031600 con 0x7fb4b8072610 2024-09-12T03:23:12.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:12.775+0000 7fb4b77fe640 1 -- 172.21.15.27:0/3672509636 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4b00319d0 con 0x7fb4b8072610 2024-09-12T03:23:12.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:12.775+0000 7fb4be52a640 1 -- 172.21.15.27:0/3672509636 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb4b8072610 msgr2=0x7fb4b8072ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:12.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:12.775+0000 7fb4be52a640 1 --2- 172.21.15.27:0/3672509636 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb4b8072610 0x7fb4b8072ac0 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7fb4b0009e50 tx=0x7fb4b002a6c0 comp rx=0 tx=0).stop 2024-09-12T03:23:12.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:12.775+0000 7fb4be52a640 1 -- 172.21.15.27:0/3672509636 shutdown_connections 2024-09-12T03:23:12.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:12.775+0000 7fb4be52a640 1 --2- 172.21.15.27:0/3672509636 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb4b8072610 0x7fb4b8072ac0 unknown :-1 s=CLOSED pgs=1488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:12.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:12.775+0000 7fb4be52a640 1 --2- 172.21.15.27:0/3672509636 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb4b806e550 0x7fb4b815ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:12.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:12.775+0000 7fb4be52a640 1 -- 172.21.15.27:0/3672509636 >> 172.21.15.27:0/3672509636 conn(0x7fb4b80b26c0 msgr2=0x7fb4b80b2aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:12.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:12.775+0000 7fb4be52a640 1 -- 172.21.15.27:0/3672509636 shutdown_connections 2024-09-12T03:23:12.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:12.775+0000 7fb4be52a640 1 -- 172.21.15.27:0/3672509636 wait complete. 2024-09-12T03:23:12.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:12.777+0000 7fb4be52a640 1 Processor -- start 2024-09-12T03:23:12.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:12.777+0000 7fb4be52a640 1 -- start start 2024-09-12T03:23:12.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:12.777+0000 7fb4be52a640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb4b806e550 0x7fb4b81503d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:12.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:12.777+0000 7fb4be52a640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb4b8072610 0x7fb4b80b4830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:12.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:12.777+0000 7fb4be52a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb4b80ae8d0 0x7fb4b80b4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:12.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:12.777+0000 7fb4be52a640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fb4b80b3480 con 0x7fb4b8072610 2024-09-12T03:23:12.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:12.777+0000 7fb4be52a640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb4b80b3300 con 0x7fb4b80ae8d0 2024-09-12T03:23:12.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:12.777+0000 7fb4be52a640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fb4b80b39c0 con 0x7fb4b806e550 2024-09-12T03:23:12.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:12.777+0000 7fb4affff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb4b8072610 0x7fb4b80b4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:12.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:12.777+0000 7fb4affff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb4b8072610 0x7fb4b80b4830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59082/0 (socket says 172.21.15.27:59082) 2024-09-12T03:23:12.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:12.777+0000 7fb4affff640 1 -- 172.21.15.27:0/1089479092 learned_addr learned my addr 172.21.15.27:0/1089479092 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:23:12.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:12.777+0000 7fb4b7fff640 1 --2- 172.21.15.27:0/1089479092 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb4b806e550 0x7fb4b81503d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:12.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:12.778+0000 7fb4bcac7640 1 --2- 172.21.15.27:0/1089479092 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb4b80ae8d0 0x7fb4b80b4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:12.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:12.778+0000 7fb4affff640 1 -- 172.21.15.27:0/1089479092 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb4b806e550 msgr2=0x7fb4b81503d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:12.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:12.778+0000 7fb4affff640 1 --2- 172.21.15.27:0/1089479092 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb4b806e550 0x7fb4b81503d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:12.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:12.778+0000 7fb4affff640 1 -- 172.21.15.27:0/1089479092 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb4b80ae8d0 msgr2=0x7fb4b80b4d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:12.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:12.778+0000 7fb4affff640 1 --2- 172.21.15.27:0/1089479092 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb4b80ae8d0 0x7fb4b80b4d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:12.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:12.778+0000 7fb4affff640 1 -- 172.21.15.27:0/1089479092 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4b8157630 con 0x7fb4b8072610 2024-09-12T03:23:12.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:12.778+0000 7fb4affff640 1 --2- 172.21.15.27:0/1089479092 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb4b8072610 0x7fb4b80b4830 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7fb4a000c3f0 tx=0x7fb4a000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:12.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:12.778+0000 7fb4b57fa640 1 -- 172.21.15.27:0/1089479092 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4a0007020 con 0x7fb4b8072610 2024-09-12T03:23:12.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:12.778+0000 7fb4b57fa640 1 -- 172.21.15.27:0/1089479092 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4a00071c0 con 0x7fb4b8072610 2024-09-12T03:23:12.784 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:12.778+0000 7fb4b57fa640 1 -- 172.21.15.27:0/1089479092 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4a0007590 con 0x7fb4b8072610 2024-09-12T03:23:12.784 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:12.778+0000 7fb4be52a640 1 -- 172.21.15.27:0/1089479092 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4b81578f0 con 0x7fb4b8072610 2024-09-12T03:23:12.784 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:12.778+0000 7fb4be52a640 1 -- 172.21.15.27:0/1089479092 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4b8157eb0 con 0x7fb4b8072610 2024-09-12T03:23:12.784 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:12.779+0000 7fb4be52a640 1 -- 172.21.15.27:0/1089479092 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4b815ebb0 con 0x7fb4b8072610 2024-09-12T03:23:12.784 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:12.780+0000 7fb4b57fa640 1 -- 172.21.15.27:0/1089479092 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 24) ==== 91212+0+0 (secure 0 0 0) 0x7fb4a0019020 con 0x7fb4b8072610 2024-09-12T03:23:12.784 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:12.780+0000 7fb4b57fa640 1 --2- 172.21.15.27:0/1089479092 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fb49806c610 0x7fb49806ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:12.784 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:12.780+0000 7fb4b57fa640 1 -- 172.21.15.27:0/1089479092 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 5413+0+0 (secure 0 0 0) 0x7fb4a0088ef0 con 0x7fb4b8072610 2024-09-12T03:23:12.784 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:12.780+0000 7fb4b7fff640 1 --2- 172.21.15.27:0/1089479092 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fb49806c610 0x7fb49806ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:12.784 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:12.781+0000 7fb4b7fff640 1 --2- 172.21.15.27:0/1089479092 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fb49806c610 0x7fb49806ead0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fb4a800aac0 tx=0x7fb4a8008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:12.784 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:12.785+0000 7fb4b57fa640 1 -- 172.21.15.27:0/1089479092 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fb4a0058b40 con 0x7fb4b8072610 2024-09-12T03:23:12.961 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:12.961+0000 7fb4be52a640 1 -- 172.21.15.27:0/1089479092 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fb4b8075640 con 0x7fb4b8072610 2024-09-12T03:23:13.641 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:13.641+0000 7fb4b57fa640 1 -- 172.21.15.27:0/1089479092 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v141) ==== 169+0+0 (secure 0 0 0) 0x7fb4a005ce40 con 0x7fb4b8072610 2024-09-12T03:23:13.642 INFO:teuthology.orchestra.run.smithi027.stderr:pool 'cephfs_data' removed 2024-09-12T03:23:13.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:13.645+0000 7fb4be52a640 1 -- 172.21.15.27:0/1089479092 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fb49806c610 msgr2=0x7fb49806ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:13.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:13.646+0000 7fb4be52a640 1 --2- 172.21.15.27:0/1089479092 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fb49806c610 0x7fb49806ead0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fb4a800aac0 tx=0x7fb4a8008040 comp rx=0 tx=0).stop 2024-09-12T03:23:13.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:13.646+0000 7fb4be52a640 1 -- 172.21.15.27:0/1089479092 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb4b8072610 msgr2=0x7fb4b80b4830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:13.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:13.646+0000 7fb4be52a640 1 --2- 172.21.15.27:0/1089479092 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb4b8072610 0x7fb4b80b4830 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7fb4a000c3f0 tx=0x7fb4a000c8b0 comp rx=0 tx=0).stop 2024-09-12T03:23:13.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:13.646+0000 7fb4be52a640 1 -- 172.21.15.27:0/1089479092 shutdown_connections 2024-09-12T03:23:13.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:13.646+0000 7fb4be52a640 1 --2- 172.21.15.27:0/1089479092 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb4b80ae8d0 0x7fb4b80b4d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:13.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:13.646+0000 7fb4be52a640 1 --2- 172.21.15.27:0/1089479092 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fb49806c610 0x7fb49806ead0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:13.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:13.646+0000 7fb4be52a640 1 --2- 172.21.15.27:0/1089479092 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb4b8072610 0x7fb4b80b4830 unknown :-1 s=CLOSED pgs=1489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:13.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:13.646+0000 7fb4be52a640 1 --2- 172.21.15.27:0/1089479092 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb4b806e550 0x7fb4b81503d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:13.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:13.646+0000 7fb4be52a640 1 -- 172.21.15.27:0/1089479092 >> 172.21.15.27:0/1089479092 conn(0x7fb4b80b26c0 msgr2=0x7fb4b815b100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:13.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:13.646+0000 7fb4be52a640 1 -- 172.21.15.27:0/1089479092 shutdown_connections 2024-09-12T03:23:13.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:13.646+0000 7fb4be52a640 1 -- 172.21.15.27:0/1089479092 wait complete. 2024-09-12T03:23:13.660 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-09-12T03:23:13.768 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:13.768+0000 7f28ae68d640 1 Processor -- start 2024-09-12T03:23:13.768 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:13.768+0000 7f28ae68d640 1 -- start start 2024-09-12T03:23:13.772 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:13.769+0000 7f28ae68d640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f28a81645c0 0x7f28a81669b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:13.772 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:13.769+0000 7f28ae68d640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f28a80b4720 con 0x7f28a80af9d0 2024-09-12T03:23:13.772 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:13.769+0000 7f28ae68d640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f28a80ce5e0 con 0x7f28a815ec30 2024-09-12T03:23:13.772 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:13.769+0000 7f28ae68d640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f28a80d02d0 con 0x7f28a81645c0 2024-09-12T03:23:13.772 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:13.769+0000 7f28a77fe640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f28a80af9d0 0x7f28a80b4010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:48614/0 (socket says 172.21.15.27:48614) 2024-09-12T03:23:13.772 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:13.769+0000 7f28a77fe640 1 -- 172.21.15.27:0/2290582552 learned_addr learned my addr 172.21.15.27:0/2290582552 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:23:13.772 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:13.770+0000 7f28acc2a640 1 --2- 172.21.15.27:0/2290582552 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f28a81645c0 0x7f28a81669b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:13.773 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:13.770+0000 7f28a6ffd640 1 -- 172.21.15.27:0/2290582552 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2376771838 0 0) 0x7f28a80b4720 con 0x7f28a80af9d0 2024-09-12T03:23:13.773 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:13.770+0000 7f28a6ffd640 1 -- 172.21.15.27:0/2290582552 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2894003520 con 0x7f28a80af9d0 2024-09-12T03:23:13.773 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:13.770+0000 7f28a6ffd640 1 -- 172.21.15.27:0/2290582552 <== mon.1 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3690709529 0 0) 0x7f28a80ce5e0 con 0x7f28a815ec30 2024-09-12T03:23:13.773 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:13.770+0000 7f28a6ffd640 1 -- 172.21.15.27:0/2290582552 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f28a80b4720 con 0x7f28a815ec30 2024-09-12T03:23:13.773 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:13.770+0000 7f28a6ffd640 1 -- 172.21.15.27:0/2290582552 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1815816417 0 0) 0x7f2894003520 con 0x7f28a80af9d0 2024-09-12T03:23:13.773 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:13.770+0000 7f28a6ffd640 1 -- 172.21.15.27:0/2290582552 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f28a81645c0 msgr2=0x7f28a81669b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:13.773 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:13.770+0000 7f28a6ffd640 1 --2- 172.21.15.27:0/2290582552 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f28a81645c0 0x7f28a81669b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:13.773 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:13.770+0000 7f28a6ffd640 1 -- 172.21.15.27:0/2290582552 >> v1:172.21.15.45:6789/0 conn(0x7f28a815ec30 legacy=0x7f28a80b3900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:13.773 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:13.770+0000 7f28a6ffd640 1 -- 172.21.15.27:0/2290582552 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28a80ceac0 con 0x7f28a80af9d0 2024-09-12T03:23:13.773 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:13.770+0000 7f28a6ffd640 1 -- 172.21.15.27:0/2290582552 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 847755213 0 0) 0x7f2890001670 con 0x7f28a80af9d0 2024-09-12T03:23:13.773 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:13.770+0000 7f28a6ffd640 1 -- 172.21.15.27:0/2290582552 >> v1:172.21.15.27:6789/0 conn(0x7f28a80af9d0 legacy=0x7f28a80b4010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:13.774 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:13.770+0000 7f28a6ffd640 1 --2- 172.21.15.27:0/2290582552 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f28a81645c0 0x7f2894003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:13.774 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:13.770+0000 7f28a6ffd640 1 --2- 172.21.15.27:0/2290582552 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f28940040d0 0x7f2894004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:13.774 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:13.770+0000 7f28a6ffd640 1 --2- 172.21.15.27:0/2290582552 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2894004d40 0x7f28940051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:13.774 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:13.770+0000 7f28a6ffd640 1 -- 172.21.15.27:0/2290582552 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f28a80d02d0 con 0x7f2894004d40 2024-09-12T03:23:13.774 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:13.770+0000 7f28a6ffd640 1 -- 172.21.15.27:0/2290582552 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f289400baa0 con 0x7f28940040d0 2024-09-12T03:23:13.774 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:13.770+0000 7f28a6ffd640 1 -- 172.21.15.27:0/2290582552 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f289400bca0 con 0x7f28a81645c0 2024-09-12T03:23:13.774 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:13.770+0000 7f28acc2a640 1 --2- 172.21.15.27:0/2290582552 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2894004d40 0x7f28940051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:13.774 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:13.770+0000 7f28a7fff640 1 --2- 172.21.15.27:0/2290582552 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f28a81645c0 0x7f2894003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:13.774 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:13.770+0000 7f28a77fe640 1 --2- 172.21.15.27:0/2290582552 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f28940040d0 0x7f2894004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:13.774 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:13.770+0000 7f28acc2a640 1 -- 172.21.15.27:0/2290582552 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f28a81645c0 msgr2=0x7f2894003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:13.774 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:13.770+0000 7f28acc2a640 1 --2- 172.21.15.27:0/2290582552 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f28a81645c0 0x7f2894003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:13.774 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:13.770+0000 7f28acc2a640 1 -- 172.21.15.27:0/2290582552 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f28940040d0 msgr2=0x7f2894004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:13.774 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:13.770+0000 7f28acc2a640 1 --2- 172.21.15.27:0/2290582552 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f28940040d0 0x7f2894004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:13.774 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:13.770+0000 7f28acc2a640 1 -- 172.21.15.27:0/2290582552 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2894003520 con 0x7f2894004d40 2024-09-12T03:23:13.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:13.771+0000 7f28acc2a640 1 --2- 172.21.15.27:0/2290582552 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2894004d40 0x7f28940051c0 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7f28a0009fc0 tx=0x7f2894005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cc9b0f907deff58 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:13.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:13.771+0000 7f28a6ffd640 1 -- 172.21.15.27:0/2290582552 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28a0009c10 con 0x7f2894004d40 2024-09-12T03:23:13.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:13.771+0000 7f28a6ffd640 1 -- 172.21.15.27:0/2290582552 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28a002e610 con 0x7f2894004d40 2024-09-12T03:23:13.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:13.771+0000 7f28a6ffd640 1 -- 172.21.15.27:0/2290582552 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28a002ea00 con 0x7f2894004d40 2024-09-12T03:23:13.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:13.772+0000 7f28ae68d640 1 -- 172.21.15.27:0/2290582552 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2894004d40 msgr2=0x7f28940051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:13.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:13.772+0000 7f28ae68d640 1 --2- 172.21.15.27:0/2290582552 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2894004d40 0x7f28940051c0 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7f28a0009fc0 tx=0x7f2894005830 comp rx=0 tx=0).stop 2024-09-12T03:23:13.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:13.772+0000 7f28acc2a640 1 -- 172.21.15.27:0/2290582552 reap_dead start 2024-09-12T03:23:13.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:13.772+0000 7f28ae68d640 1 -- 172.21.15.27:0/2290582552 shutdown_connections 2024-09-12T03:23:13.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:13.772+0000 7f28ae68d640 1 -- 172.21.15.27:0/2290582552 >> 172.21.15.27:0/2290582552 conn(0x7f28a806bb30 msgr2=0x7f28a806bf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:13.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:13.772+0000 7f28ae68d640 1 -- 172.21.15.27:0/2290582552 shutdown_connections 2024-09-12T03:23:13.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:13.772+0000 7f28ae68d640 1 -- 172.21.15.27:0/2290582552 wait complete. 2024-09-12T03:23:13.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:13.773+0000 7f28ae68d640 1 Processor -- start 2024-09-12T03:23:13.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:13.773+0000 7f28ae68d640 1 -- start start 2024-09-12T03:23:13.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:13.774+0000 7f28ae68d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2894004d40 0x7f28a815e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:13.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:13.774+0000 7f28ae68d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f28a815b420 0x7f28a81605a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:13.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:13.774+0000 7f28ae68d640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f28a814c6e0 0x7f28a81541a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:13.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:13.774+0000 7f28ae68d640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f28a80cf170 con 0x7f28a815b420 2024-09-12T03:23:13.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:13.774+0000 7f28ae68d640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f28a80ceff0 con 0x7f2894004d40 2024-09-12T03:23:13.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:13.774+0000 7f28ae68d640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f28a80cf2f0 con 0x7f28a814c6e0 2024-09-12T03:23:13.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:13.774+0000 7f28a77fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f28a815b420 0x7f28a81605a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:13.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:13.774+0000 7f28a77fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f28a815b420 0x7f28a81605a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59106/0 (socket says 172.21.15.27:59106) 2024-09-12T03:23:13.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:13.774+0000 7f28a77fe640 1 -- 172.21.15.27:0/276158445 learned_addr learned my addr 172.21.15.27:0/276158445 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:23:13.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:13.774+0000 7f28a7fff640 1 --2- 172.21.15.27:0/276158445 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2894004d40 0x7f28a815e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:13.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:13.774+0000 7f28acc2a640 1 --2- 172.21.15.27:0/276158445 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f28a814c6e0 0x7f28a81541a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:13.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:13.774+0000 7f28a77fe640 1 -- 172.21.15.27:0/276158445 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f28a814c6e0 msgr2=0x7f28a81541a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:13.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:13.774+0000 7f28a77fe640 1 --2- 172.21.15.27:0/276158445 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f28a814c6e0 0x7f28a81541a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:13.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:13.774+0000 7f28a77fe640 1 -- 172.21.15.27:0/276158445 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2894004d40 msgr2=0x7f28a815e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:13.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:13.774+0000 7f28a77fe640 1 --2- 172.21.15.27:0/276158445 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2894004d40 0x7f28a815e0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:13.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:13.775+0000 7f28a77fe640 1 -- 172.21.15.27:0/276158445 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28a815bb00 con 0x7f28a815b420 2024-09-12T03:23:13.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:13.775+0000 7f28a77fe640 1 --2- 172.21.15.27:0/276158445 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f28a815b420 0x7f28a81605a0 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7f2890007370 tx=0x7f2890007770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:13.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:13.775+0000 7f28a4ff9640 1 -- 172.21.15.27:0/276158445 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f289001a070 con 0x7f28a815b420 2024-09-12T03:23:13.780 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:13.775+0000 7f28a4ff9640 1 -- 172.21.15.27:0/276158445 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f289000f8e0 con 0x7f28a815b420 2024-09-12T03:23:13.780 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:13.775+0000 7f28ae68d640 1 -- 172.21.15.27:0/276158445 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28a815bde0 con 0x7f28a815b420 2024-09-12T03:23:13.780 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:13.775+0000 7f28a4ff9640 1 -- 172.21.15.27:0/276158445 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f289000fcd0 con 0x7f28a815b420 2024-09-12T03:23:13.780 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:13.775+0000 7f28ae68d640 1 -- 172.21.15.27:0/276158445 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28a815c3a0 con 0x7f28a815b420 2024-09-12T03:23:13.780 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:13.776+0000 7f28ae68d640 1 -- 172.21.15.27:0/276158445 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28a80695a0 con 0x7f28a815b420 2024-09-12T03:23:13.780 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:13.776+0000 7f28a4ff9640 1 -- 172.21.15.27:0/276158445 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 24) ==== 91212+0+0 (secure 0 0 0) 0x7f289001d070 con 0x7f28a815b420 2024-09-12T03:23:13.780 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:13.777+0000 7f28a4ff9640 1 --2- 172.21.15.27:0/276158445 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f288806c640 0x7f288806eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:13.780 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:13.777+0000 7f28a4ff9640 1 -- 172.21.15.27:0/276158445 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(141..141 src has 1..141) ==== 4999+0+0 (secure 0 0 0) 0x7f28900890e0 con 0x7f28a815b420 2024-09-12T03:23:13.780 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:13.777+0000 7f28a7fff640 1 --2- 172.21.15.27:0/276158445 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f288806c640 0x7f288806eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:13.780 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:13.777+0000 7f28a7fff640 1 --2- 172.21.15.27:0/276158445 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f288806c640 0x7f288806eb00 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f28a814d5f0 tx=0x7f2898009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:13.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:13.781+0000 7f28a4ff9640 1 -- 172.21.15.27:0/276158445 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f2890058d30 con 0x7f28a815b420 2024-09-12T03:23:13.964 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:13.964+0000 7f28ae68d640 1 -- 172.21.15.27:0/276158445 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f28a815f010 con 0x7f28a815b420 2024-09-12T03:23:13.967 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:13.967+0000 7f28a4ff9640 1 -- 172.21.15.27:0/276158445 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v27) ==== 104+0+0 (secure 0 0 0) 0x7f289005d030 con 0x7f28a815b420 2024-09-12T03:23:13.969 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:13.969+0000 7f28ae68d640 1 -- 172.21.15.27:0/276158445 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f288806c640 msgr2=0x7f288806eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:13.969 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:13.969+0000 7f28ae68d640 1 --2- 172.21.15.27:0/276158445 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f288806c640 0x7f288806eb00 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f28a814d5f0 tx=0x7f2898009040 comp rx=0 tx=0).stop 2024-09-12T03:23:13.969 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:13.969+0000 7f28ae68d640 1 -- 172.21.15.27:0/276158445 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f28a815b420 msgr2=0x7f28a81605a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:13.970 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:13.969+0000 7f28ae68d640 1 --2- 172.21.15.27:0/276158445 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f28a815b420 0x7f28a81605a0 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7f2890007370 tx=0x7f2890007770 comp rx=0 tx=0).stop 2024-09-12T03:23:13.970 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:13.969+0000 7f28ae68d640 1 -- 172.21.15.27:0/276158445 shutdown_connections 2024-09-12T03:23:13.970 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:13.969+0000 7f28ae68d640 1 --2- 172.21.15.27:0/276158445 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f28a814c6e0 0x7f28a81541a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:13.970 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:13.969+0000 7f28ae68d640 1 --2- 172.21.15.27:0/276158445 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f288806c640 0x7f288806eb00 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:13.970 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:13.969+0000 7f28ae68d640 1 --2- 172.21.15.27:0/276158445 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f28a815b420 0x7f28a81605a0 unknown :-1 s=CLOSED pgs=1492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:13.970 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:13.969+0000 7f28ae68d640 1 --2- 172.21.15.27:0/276158445 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2894004d40 0x7f28a815e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:13.970 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:13.969+0000 7f28ae68d640 1 -- 172.21.15.27:0/276158445 >> 172.21.15.27:0/276158445 conn(0x7f28a806bb30 msgr2=0x7f28a8071da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:13.970 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:13.969+0000 7f28ae68d640 1 -- 172.21.15.27:0/276158445 shutdown_connections 2024-09-12T03:23:13.970 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:13.969+0000 7f28ae68d640 1 -- 172.21.15.27:0/276158445 wait complete. 2024-09-12T03:23:13.980 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_upgrade_legacy_to_v1' 2024-09-12T03:23:14.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:14.082+0000 7f977eaf8640 1 Processor -- start 2024-09-12T03:23:14.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:14.082+0000 7f977eaf8640 1 -- start start 2024-09-12T03:23:14.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:14.083+0000 7f977eaf8640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f977815eb60 0x7f97780d1280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:14.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:14.083+0000 7f977eaf8640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f97780d17c0 0x7f97780b3390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:14.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:14.083+0000 7f977eaf8640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f97780b6690 con 0x7f977815e780 2024-09-12T03:23:14.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:14.083+0000 7f977eaf8640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f97780d1c30 con 0x7f977815eb60 2024-09-12T03:23:14.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:14.083+0000 7f977eaf8640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f97780b7950 con 0x7f97780d17c0 2024-09-12T03:23:14.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:14.083+0000 7f9777fff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f977815eb60 0x7f97780d1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:14.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:14.083+0000 7f977c894640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f977815e780 0x7f97780d3e70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:48624/0 (socket says 172.21.15.27:48624) 2024-09-12T03:23:14.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:14.083+0000 7f977c894640 1 -- 172.21.15.27:0/1641350713 learned_addr learned my addr 172.21.15.27:0/1641350713 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:23:14.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:14.083+0000 7f9777fff640 1 --2- 172.21.15.27:0/1641350713 >> v2:172.21.15.27:3300/0 conn(0x7f977815eb60 0x7f97780d1280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59108/0 (socket says 172.21.15.27:59108) 2024-09-12T03:23:14.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:14.084+0000 7f977d095640 1 --2- 172.21.15.27:0/1641350713 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f97780d17c0 0x7f97780b3390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:14.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:14.084+0000 7f9777fff640 1 -- 172.21.15.27:0/1641350713 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f97780d17c0 msgr2=0x7f97780b3390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:14.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:14.084+0000 7f9777fff640 1 --2- 172.21.15.27:0/1641350713 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f97780d17c0 0x7f97780b3390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:14.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:14.084+0000 7f9777fff640 1 -- 172.21.15.27:0/1641350713 >> v1:172.21.15.27:6789/0 conn(0x7f977815e780 legacy=0x7f97780d3e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:14.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:14.084+0000 7f97777fe640 1 -- 172.21.15.27:0/1641350713 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2395463879 0 0) 0x7f97780b6690 con 0x7f977815e780 2024-09-12T03:23:14.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:14.084+0000 7f9777fff640 1 -- 172.21.15.27:0/1641350713 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97780b6970 con 0x7f977815eb60 2024-09-12T03:23:14.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:14.084+0000 7f9777fff640 1 --2- 172.21.15.27:0/1641350713 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f977815eb60 0x7f97780d1280 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7f976c0097d0 tx=0x7f976c02a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f41517af593e044 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:14.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:14.085+0000 7f97777fe640 1 -- 172.21.15.27:0/1641350713 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f976c037070 con 0x7f977815eb60 2024-09-12T03:23:14.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:14.085+0000 7f97777fe640 1 -- 172.21.15.27:0/1641350713 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f976c032440 con 0x7f977815eb60 2024-09-12T03:23:14.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:14.085+0000 7f97777fe640 1 -- 172.21.15.27:0/1641350713 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f976c032810 con 0x7f977815eb60 2024-09-12T03:23:14.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:14.085+0000 7f977eaf8640 1 -- 172.21.15.27:0/1641350713 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f977815eb60 msgr2=0x7f97780d1280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:14.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:14.085+0000 7f977eaf8640 1 --2- 172.21.15.27:0/1641350713 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f977815eb60 0x7f97780d1280 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7f976c0097d0 tx=0x7f976c02a1c0 comp rx=0 tx=0).stop 2024-09-12T03:23:14.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:14.085+0000 7f977eaf8640 1 -- 172.21.15.27:0/1641350713 shutdown_connections 2024-09-12T03:23:14.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:14.085+0000 7f977eaf8640 1 --2- 172.21.15.27:0/1641350713 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f97780d17c0 0x7f97780b3390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:14.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:14.085+0000 7f977eaf8640 1 --2- 172.21.15.27:0/1641350713 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f977815eb60 0x7f97780d1280 unknown :-1 s=CLOSED pgs=1494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:14.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:14.085+0000 7f977eaf8640 1 -- 172.21.15.27:0/1641350713 >> 172.21.15.27:0/1641350713 conn(0x7f977806da60 msgr2=0x7f977806de70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:14.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:14.085+0000 7f977eaf8640 1 -- 172.21.15.27:0/1641350713 shutdown_connections 2024-09-12T03:23:14.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:14.085+0000 7f977eaf8640 1 -- 172.21.15.27:0/1641350713 wait complete. 2024-09-12T03:23:14.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:14.086+0000 7f977eaf8640 1 Processor -- start 2024-09-12T03:23:14.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:14.086+0000 7f977eaf8640 1 -- start start 2024-09-12T03:23:14.090 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:14.087+0000 7f977eaf8640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f97780d17c0 0x7f97780cb2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:14.090 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:14.087+0000 7f977eaf8640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f977815e780 0x7f977814c600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:14.090 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:14.087+0000 7f977eaf8640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f977815eb60 0x7f97780c8f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:14.090 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:14.087+0000 7f977eaf8640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f97780b6f30 con 0x7f977815e780 2024-09-12T03:23:14.090 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:14.087+0000 7f977eaf8640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f97780b7210 con 0x7f97780d17c0 2024-09-12T03:23:14.090 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:14.087+0000 7f977eaf8640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f97780b6340 con 0x7f977815eb60 2024-09-12T03:23:14.090 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:14.087+0000 7f9777fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f977815e780 0x7f977814c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:14.090 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:14.087+0000 7f9777fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f977815e780 0x7f977814c600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59112/0 (socket says 172.21.15.27:59112) 2024-09-12T03:23:14.090 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:14.087+0000 7f9777fff640 1 -- 172.21.15.27:0/1761370109 learned_addr learned my addr 172.21.15.27:0/1761370109 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:23:14.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:14.088+0000 7f977d095640 1 --2- 172.21.15.27:0/1761370109 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f977815eb60 0x7f97780c8f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:14.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:14.088+0000 7f977c894640 1 --2- 172.21.15.27:0/1761370109 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f97780d17c0 0x7f97780cb2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:14.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:14.088+0000 7f9777fff640 1 -- 172.21.15.27:0/1761370109 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f977815eb60 msgr2=0x7f97780c8f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:14.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:14.088+0000 7f9777fff640 1 --2- 172.21.15.27:0/1761370109 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f977815eb60 0x7f97780c8f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:14.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:14.088+0000 7f9777fff640 1 -- 172.21.15.27:0/1761370109 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f97780d17c0 msgr2=0x7f97780cb2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:14.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:14.088+0000 7f9777fff640 1 --2- 172.21.15.27:0/1761370109 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f97780d17c0 0x7f97780cb2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:14.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:14.088+0000 7f9777fff640 1 -- 172.21.15.27:0/1761370109 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f977814cb40 con 0x7f977815e780 2024-09-12T03:23:14.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:14.088+0000 7f977d095640 1 --2- 172.21.15.27:0/1761370109 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f977815eb60 0x7f97780c8f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:23:14.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:14.088+0000 7f9777fff640 1 --2- 172.21.15.27:0/1761370109 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f977815e780 0x7f977814c600 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7f976c0098f0 tx=0x7f976c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:14.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:14.088+0000 7f97757fa640 1 -- 172.21.15.27:0/1761370109 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f976c037040 con 0x7f977815e780 2024-09-12T03:23:14.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:14.088+0000 7f97757fa640 1 -- 172.21.15.27:0/1761370109 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f976c042070 con 0x7f977815e780 2024-09-12T03:23:14.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:14.088+0000 7f977eaf8640 1 -- 172.21.15.27:0/1761370109 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f977815af40 con 0x7f977815e780 2024-09-12T03:23:14.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:14.088+0000 7f97757fa640 1 -- 172.21.15.27:0/1761370109 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f976c030bb0 con 0x7f977815e780 2024-09-12T03:23:14.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:14.088+0000 7f977eaf8640 1 -- 172.21.15.27:0/1761370109 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f977815b500 con 0x7f977815e780 2024-09-12T03:23:14.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:14.089+0000 7f977eaf8640 1 -- 172.21.15.27:0/1761370109 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97780d1280 con 0x7f977815e780 2024-09-12T03:23:14.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:14.090+0000 7f97757fa640 1 -- 172.21.15.27:0/1761370109 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 24) ==== 91212+0+0 (secure 0 0 0) 0x7f976c047460 con 0x7f977815e780 2024-09-12T03:23:14.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:14.090+0000 7f97757fa640 1 --2- 172.21.15.27:0/1761370109 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f976406c610 0x7f976406ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:14.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:14.090+0000 7f97757fa640 1 -- 172.21.15.27:0/1761370109 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(141..141 src has 1..141) ==== 4999+0+0 (secure 0 0 0) 0x7f976c0aa4a0 con 0x7f977815e780 2024-09-12T03:23:14.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:14.090+0000 7f977c894640 1 --2- 172.21.15.27:0/1761370109 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f976406c610 0x7f976406ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:14.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:14.091+0000 7f977c894640 1 --2- 172.21.15.27:0/1761370109 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f976406c610 0x7f976406ead0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f9768007fd0 tx=0x7f9768009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:14.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:14.094+0000 7f97757fa640 1 -- 172.21.15.27:0/1761370109 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f976c032e20 con 0x7f977815e780 2024-09-12T03:23:14.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:14.335+0000 7f977eaf8640 1 -- 172.21.15.27:0/1761370109 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_upgrade_legacy_to_v1"]} v 0) -- 0x7f9778069ff0 con 0x7f977815e780 2024-09-12T03:23:14.643 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:14.643+0000 7f97757fa640 1 -- 172.21.15.27:0/1761370109 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_upgrade_legacy_to_v1"]}]=0 v370) ==== 149+0+0 (secure 0 0 0) 0x7f976c07a740 con 0x7f977815e780 2024-09-12T03:23:14.645 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:14.645+0000 7f977eaf8640 1 -- 172.21.15.27:0/1761370109 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f976406c610 msgr2=0x7f976406ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:14.645 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:14.645+0000 7f977eaf8640 1 --2- 172.21.15.27:0/1761370109 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f976406c610 0x7f976406ead0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f9768007fd0 tx=0x7f9768009040 comp rx=0 tx=0).stop 2024-09-12T03:23:14.645 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:14.645+0000 7f977eaf8640 1 -- 172.21.15.27:0/1761370109 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f977815e780 msgr2=0x7f977814c600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:14.645 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:14.645+0000 7f977eaf8640 1 --2- 172.21.15.27:0/1761370109 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f977815e780 0x7f977814c600 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7f976c0098f0 tx=0x7f976c0046c0 comp rx=0 tx=0).stop 2024-09-12T03:23:14.645 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:14.645+0000 7f977eaf8640 1 -- 172.21.15.27:0/1761370109 shutdown_connections 2024-09-12T03:23:14.645 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:14.645+0000 7f977eaf8640 1 --2- 172.21.15.27:0/1761370109 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f977815eb60 0x7f97780c8f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:14.645 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:14.645+0000 7f977eaf8640 1 --2- 172.21.15.27:0/1761370109 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f976406c610 0x7f976406ead0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:14.645 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:14.645+0000 7f977eaf8640 1 --2- 172.21.15.27:0/1761370109 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f977815e780 0x7f977814c600 unknown :-1 s=CLOSED pgs=1495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:14.645 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:14.645+0000 7f977eaf8640 1 --2- 172.21.15.27:0/1761370109 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f97780d17c0 0x7f97780cb2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:14.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:14.645+0000 7f977eaf8640 1 -- 172.21.15.27:0/1761370109 >> 172.21.15.27:0/1761370109 conn(0x7f977806da60 msgr2=0x7f97780b9fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:14.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:14.645+0000 7f977eaf8640 1 -- 172.21.15.27:0/1761370109 shutdown_connections 2024-09-12T03:23:14.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:14.645+0000 7f977eaf8640 1 -- 172.21.15.27:0/1761370109 wait complete. 2024-09-12T03:23:14.657 INFO:tasks.cephfs_test_runner:poor man's upgrade test -- rather than going through a full upgrade cycle, ... ok 2024-09-12T03:23:14.657 INFO:tasks.cephfs_test_runner:Starting test: test_subvolume_upgrade_v1_to_v2 (tasks.cephfs.test_volumes.TestMisc) poor man's upgrade test -- theme continues... 2024-09-12T03:23:14.657 INFO:tasks.cephfs_test_runner:test_subvolume_upgrade_v1_to_v2 (tasks.cephfs.test_volumes.TestMisc) 2024-09-12T03:23:14.658 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-12T03:23:14.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:14.761+0000 7f56715c0640 1 Processor -- start 2024-09-12T03:23:14.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:14.761+0000 7f56715c0640 1 -- start start 2024-09-12T03:23:14.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:14.762+0000 7f56715c0640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f566c158cf0 0x7f566c0b55d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:14.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:14.762+0000 7f56715c0640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f566c1590d0 0x7f566c162430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:14.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:14.762+0000 7f56715c0640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f566c162970 con 0x7f566c15f600 2024-09-12T03:23:14.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:14.762+0000 7f56715c0640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f566c162c10 con 0x7f566c1590d0 2024-09-12T03:23:14.764 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:14.762+0000 7f56715c0640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f566c162ef0 con 0x7f566c158cf0 2024-09-12T03:23:14.764 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:14.762+0000 7f566a7fc640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f566c1590d0 0x7f566c162430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:14.764 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:14.762+0000 7f566a7fc640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f566c1590d0 0x7f566c162430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59128/0 (socket says 172.21.15.27:59128) 2024-09-12T03:23:14.764 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:14.762+0000 7f566a7fc640 1 -- 172.21.15.27:0/4073486374 learned_addr learned my addr 172.21.15.27:0/4073486374 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:23:14.764 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:14.762+0000 7f566affd640 1 --2- 172.21.15.27:0/4073486374 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f566c158cf0 0x7f566c0b55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:14.764 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:14.763+0000 7f566a7fc640 1 -- 172.21.15.27:0/4073486374 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f566c158cf0 msgr2=0x7f566c0b55d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:14.764 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:14.763+0000 7f566a7fc640 1 --2- 172.21.15.27:0/4073486374 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f566c158cf0 0x7f566c0b55d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:14.764 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:14.763+0000 7f566a7fc640 1 -- 172.21.15.27:0/4073486374 >> v1:172.21.15.45:6789/0 conn(0x7f566c15f600 legacy=0x7f566c06bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:14.764 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:14.763+0000 7f566a7fc640 1 -- 172.21.15.27:0/4073486374 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f566c1631d0 con 0x7f566c1590d0 2024-09-12T03:23:14.764 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:14.763+0000 7f566a7fc640 1 --2- 172.21.15.27:0/4073486374 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f566c1590d0 0x7f566c162430 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7f5660009870 tx=0x7f566002a130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d87022cf723dacfb server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:14.764 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:14.764+0000 7f5669ffb640 1 -- 172.21.15.27:0/4073486374 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5660037070 con 0x7f566c1590d0 2024-09-12T03:23:14.764 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:14.764+0000 7f5669ffb640 1 -- 172.21.15.27:0/4073486374 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5660032440 con 0x7f566c1590d0 2024-09-12T03:23:14.764 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:14.764+0000 7f5669ffb640 1 -- 172.21.15.27:0/4073486374 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5660032810 con 0x7f566c1590d0 2024-09-12T03:23:14.765 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:14.764+0000 7f56715c0640 1 -- 172.21.15.27:0/4073486374 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f566c1590d0 msgr2=0x7f566c162430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:14.766 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:14.764+0000 7f56715c0640 1 --2- 172.21.15.27:0/4073486374 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f566c1590d0 0x7f566c162430 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7f5660009870 tx=0x7f566002a130 comp rx=0 tx=0).stop 2024-09-12T03:23:14.766 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:14.764+0000 7f56715c0640 1 -- 172.21.15.27:0/4073486374 shutdown_connections 2024-09-12T03:23:14.766 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:14.764+0000 7f56715c0640 1 --2- 172.21.15.27:0/4073486374 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f566c1590d0 0x7f566c162430 unknown :-1 s=CLOSED pgs=1496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:14.766 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:14.764+0000 7f56715c0640 1 --2- 172.21.15.27:0/4073486374 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f566c158cf0 0x7f566c0b55d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:14.766 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:14.764+0000 7f56715c0640 1 -- 172.21.15.27:0/4073486374 >> 172.21.15.27:0/4073486374 conn(0x7f566c06cc40 msgr2=0x7f566c06d050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:14.766 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:14.764+0000 7f56715c0640 1 -- 172.21.15.27:0/4073486374 shutdown_connections 2024-09-12T03:23:14.766 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:14.764+0000 7f56715c0640 1 -- 172.21.15.27:0/4073486374 wait complete. 2024-09-12T03:23:14.766 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:14.765+0000 7f56715c0640 1 Processor -- start 2024-09-12T03:23:14.766 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:14.766+0000 7f56715c0640 1 -- start start 2024-09-12T03:23:14.768 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:14.766+0000 7f56715c0640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f566c158cf0 0x7f566c1e90e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:14.768 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:14.766+0000 7f56715c0640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f566c1590d0 0x7f566c15ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:14.768 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:14.766+0000 7f56715c0640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f566c15f600 0x7f566c15bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:14.768 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:14.766+0000 7f56715c0640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f566c06c3c0 con 0x7f566c158cf0 2024-09-12T03:23:14.768 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:14.766+0000 7f56715c0640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f566c06c240 con 0x7f566c1590d0 2024-09-12T03:23:14.768 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:14.766+0000 7f56715c0640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f566c15fa10 con 0x7f566c15f600 2024-09-12T03:23:14.769 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:14.766+0000 7f566affd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f566c158cf0 0x7f566c1e90e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:14.769 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:14.766+0000 7f566affd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f566c158cf0 0x7f566c1e90e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59142/0 (socket says 172.21.15.27:59142) 2024-09-12T03:23:14.769 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:14.766+0000 7f566affd640 1 -- 172.21.15.27:0/1657189684 learned_addr learned my addr 172.21.15.27:0/1657189684 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:23:14.769 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:14.767+0000 7f566a7fc640 1 --2- 172.21.15.27:0/1657189684 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f566c1590d0 0x7f566c15ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:14.769 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:14.767+0000 7f566b7fe640 1 --2- 172.21.15.27:0/1657189684 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f566c15f600 0x7f566c15bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:14.769 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:14.767+0000 7f566affd640 1 -- 172.21.15.27:0/1657189684 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f566c15f600 msgr2=0x7f566c15bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:14.769 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:14.767+0000 7f566affd640 1 --2- 172.21.15.27:0/1657189684 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f566c15f600 0x7f566c15bc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:14.769 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:14.767+0000 7f566affd640 1 -- 172.21.15.27:0/1657189684 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f566c1590d0 msgr2=0x7f566c15ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:14.769 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:14.767+0000 7f566affd640 1 --2- 172.21.15.27:0/1657189684 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f566c1590d0 0x7f566c15ca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:14.770 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:14.767+0000 7f566affd640 1 -- 172.21.15.27:0/1657189684 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f566c15d520 con 0x7f566c158cf0 2024-09-12T03:23:14.770 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:14.767+0000 7f566affd640 1 --2- 172.21.15.27:0/1657189684 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f566c158cf0 0x7f566c1e90e0 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7f565c00d800 tx=0x7f565c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:14.770 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:14.767+0000 7f56537fe640 1 -- 172.21.15.27:0/1657189684 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f565c00b6c0 con 0x7f566c158cf0 2024-09-12T03:23:14.770 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:14.767+0000 7f56537fe640 1 -- 172.21.15.27:0/1657189684 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f565c019070 con 0x7f566c158cf0 2024-09-12T03:23:14.770 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:14.767+0000 7f56715c0640 1 -- 172.21.15.27:0/1657189684 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f566c15d800 con 0x7f566c158cf0 2024-09-12T03:23:14.770 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:14.767+0000 7f56537fe640 1 -- 172.21.15.27:0/1657189684 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f565c00bac0 con 0x7f566c158cf0 2024-09-12T03:23:14.770 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:14.767+0000 7f56715c0640 1 -- 172.21.15.27:0/1657189684 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f566c15ddc0 con 0x7f566c158cf0 2024-09-12T03:23:14.770 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:14.768+0000 7f56715c0640 1 -- 172.21.15.27:0/1657189684 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f566c162430 con 0x7f566c158cf0 2024-09-12T03:23:14.771 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:14.769+0000 7f56537fe640 1 -- 172.21.15.27:0/1657189684 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 24) ==== 91212+0+0 (secure 0 0 0) 0x7f565c00bc60 con 0x7f566c158cf0 2024-09-12T03:23:14.771 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:14.769+0000 7f56537fe640 1 --2- 172.21.15.27:0/1657189684 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f565806c610 0x7f565806ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:14.771 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:14.769+0000 7f56537fe640 1 -- 172.21.15.27:0/1657189684 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(141..141 src has 1..141) ==== 4999+0+0 (secure 0 0 0) 0x7f565c089c30 con 0x7f566c158cf0 2024-09-12T03:23:14.771 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:14.769+0000 7f566a7fc640 1 --2- 172.21.15.27:0/1657189684 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f565806c610 0x7f565806ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:14.771 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:14.770+0000 7f566a7fc640 1 --2- 172.21.15.27:0/1657189684 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f565806c610 0x7f565806ead0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f5660034000 tx=0x7f566002e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:14.773 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:14.773+0000 7f56537fe640 1 -- 172.21.15.27:0/1657189684 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f565c0597f0 con 0x7f566c158cf0 2024-09-12T03:23:14.952 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:14.952+0000 7f56715c0640 1 -- 172.21.15.27:0/1657189684 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f566c0b5220 con 0x7f566c158cf0 2024-09-12T03:23:14.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:14.953+0000 7f56537fe640 1 -- 172.21.15.27:0/1657189684 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v141) ==== 74+0+13081 (secure 0 0 0) 0x7f565c05daf0 con 0x7f566c158cf0 2024-09-12T03:23:14.953 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:23:14.953 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":141,"fsid":"d2df504e-2647-4385-a5db-9fdf69f487a0","created":"2024-09-12T03:16:07.778937+0000","modified":"2024-09-12T03:23:13.637921+0000","last_up_change":"2024-09-12T03:16:19.314778+0000","last_in_change":"2024-09-12T03:16:09.893086+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T03:16:20.097646+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1851116c-1fb3-4dc9-9772-f90e2ef4c355","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6817","nonce":3002321488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6819","nonce":3002321488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6823","nonce":3002321488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6821","nonce":3002321488}]},"public_addr":"172.21.15.27:6817/3002321488","cluster_addr":"172.21.15.27:6819/3002321488","heartbeat_back_addr":"172.21.15.27:6823/3002321488","heartbeat_front_addr":"172.21.15.27:6821/3002321488","state":["exists","up"]},{"osd":1,"uuid":"f38ae17d-73eb-4204-b99d-975fa806fb87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6809","nonce":1613102886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6811","nonce":1613102886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6815","nonce":1613102886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6813","nonce":1613102886}]},"public_addr":"172.21.15.27:6809/1613102886","cluster_addr":"172.21.15.27:6811/1613102886","heartbeat_back_addr":"172.21.15.27:6815/1613102886","heartbeat_front_addr":"172.21.15.27:6813/1613102886","state":["exists","up"]},{"osd":2,"uuid":"fb12dcc8-bdd0-42b8-b46d-8b10106224bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6825","nonce":3534723355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6827","nonce":3534723355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6831","nonce":3534723355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6829","nonce":3534723355}]},"public_addr":"172.21.15.27:6825/3534723355","cluster_addr":"172.21.15.27:6827/3534723355","heartbeat_back_addr":"172.21.15.27:6831/3534723355","heartbeat_front_addr":"172.21.15.27:6829/3534723355","state":["exists","up"]},{"osd":3,"uuid":"d74d3f49-9c24-4b9f-98e2-3ecd7b5ef3b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6801","nonce":80661598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6803","nonce":80661598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6807","nonce":80661598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6805","nonce":80661598}]},"public_addr":"172.21.15.27:6801/80661598","cluster_addr":"172.21.15.27:6803/80661598","heartbeat_back_addr":"172.21.15.27:6807/80661598","heartbeat_front_addr":"172.21.15.27:6805/80661598","state":["exists","up"]},{"osd":4,"uuid":"2bf123f2-1b0d-41c4-aa3b-805bab267e59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6817","nonce":3943189827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6819","nonce":3943189827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6823","nonce":3943189827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6821","nonce":3943189827}]},"public_addr":"172.21.15.45:6817/3943189827","cluster_addr":"172.21.15.45:6819/3943189827","heartbeat_back_addr":"172.21.15.45:6823/3943189827","heartbeat_front_addr":"172.21.15.45:6821/3943189827","state":["exists","up"]},{"osd":5,"uuid":"c49ba5da-91c6-4a86-a97b-4aba89c253ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6809","nonce":3620760330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6811","nonce":3620760330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6815","nonce":3620760330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6813","nonce":3620760330}]},"public_addr":"172.21.15.45:6809/3620760330","cluster_addr":"172.21.15.45:6811/3620760330","heartbeat_back_addr":"172.21.15.45:6815/3620760330","heartbeat_front_addr":"172.21.15.45:6813/3620760330","state":["exists","up"]},{"osd":6,"uuid":"b0123475-89f7-4a34-9e33-d797b99dd9dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6801","nonce":1989083510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6803","nonce":1989083510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6807","nonce":1989083510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6805","nonce":1989083510}]},"public_addr":"172.21.15.45:6801/1989083510","cluster_addr":"172.21.15.45:6803/1989083510","heartbeat_back_addr":"172.21.15.45:6807/1989083510","heartbeat_front_addr":"172.21.15.45:6805/1989083510","state":["exists","up"]},{"osd":7,"uuid":"4dad773d-b7d8-46fc-8a10-a972aad27b29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6825","nonce":1513300294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6827","nonce":1513300294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6831","nonce":1513300294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6829","nonce":1513300294}]},"public_addr":"172.21.15.45:6825/1513300294","cluster_addr":"172.21.15.45:6827/1513300294","heartbeat_back_addr":"172.21.15.45:6831/1513300294","heartbeat_front_addr":"172.21.15.45:6829/1513300294","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.881855+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:17.133252+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.735339+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.500742+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.477571+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:17.167667+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.470291+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.873743+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.27:6835/389118996":"2024-09-13T03:23:10.621729+0000","172.21.15.27:6833/389118996":"2024-09-13T03:23:10.621729+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]},{"pool":19,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T03:23:14.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:14.955+0000 7f56715c0640 1 -- 172.21.15.27:0/1657189684 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f565806c610 msgr2=0x7f565806ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:14.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:14.955+0000 7f56715c0640 1 --2- 172.21.15.27:0/1657189684 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f565806c610 0x7f565806ead0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f5660034000 tx=0x7f566002e000 comp rx=0 tx=0).stop 2024-09-12T03:23:14.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:14.955+0000 7f56715c0640 1 -- 172.21.15.27:0/1657189684 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f566c158cf0 msgr2=0x7f566c1e90e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:14.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:14.955+0000 7f56715c0640 1 --2- 172.21.15.27:0/1657189684 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f566c158cf0 0x7f566c1e90e0 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7f565c00d800 tx=0x7f565c00dcc0 comp rx=0 tx=0).stop 2024-09-12T03:23:14.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:14.955+0000 7f56715c0640 1 -- 172.21.15.27:0/1657189684 shutdown_connections 2024-09-12T03:23:14.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:14.955+0000 7f56715c0640 1 --2- 172.21.15.27:0/1657189684 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f566c15f600 0x7f566c15bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:14.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:14.955+0000 7f56715c0640 1 --2- 172.21.15.27:0/1657189684 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f566c1590d0 0x7f566c15ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:14.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:14.955+0000 7f56715c0640 1 --2- 172.21.15.27:0/1657189684 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f565806c610 0x7f565806ead0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:14.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:14.955+0000 7f56715c0640 1 --2- 172.21.15.27:0/1657189684 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f566c158cf0 0x7f566c1e90e0 unknown :-1 s=CLOSED pgs=1497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:14.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:14.955+0000 7f56715c0640 1 -- 172.21.15.27:0/1657189684 >> 172.21.15.27:0/1657189684 conn(0x7f566c06cc40 msgr2=0x7f566c0b5a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:14.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:14.955+0000 7f56715c0640 1 -- 172.21.15.27:0/1657189684 shutdown_connections 2024-09-12T03:23:14.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:14.955+0000 7f56715c0640 1 -- 172.21.15.27:0/1657189684 wait complete. 2024-09-12T03:23:14.966 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-12T03:16:20.097646+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-12T03:23:14.966 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-12T03:23:15.072 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.072+0000 7f563696f640 1 Processor -- start 2024-09-12T03:23:15.072 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.072+0000 7f563696f640 1 -- start start 2024-09-12T03:23:15.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.073+0000 7f563696f640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f563015e990 0x7f56300b7600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:15.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.073+0000 7f563696f640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f56300b3d10 0x7f56300b66c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:15.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.073+0000 7f563696f640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f563015ed70 con 0x7f56300b6230 2024-09-12T03:23:15.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.073+0000 7f563696f640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f56300d2a30 con 0x7f563015e990 2024-09-12T03:23:15.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.073+0000 7f563696f640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f56300d2d10 con 0x7f56300b3d10 2024-09-12T03:23:15.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.073+0000 7f562ffff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f563015e990 0x7f56300b7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:15.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.073+0000 7f562ffff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f563015e990 0x7f56300b7600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59146/0 (socket says 172.21.15.27:59146) 2024-09-12T03:23:15.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.073+0000 7f562ffff640 1 -- 172.21.15.27:0/1873830959 learned_addr learned my addr 172.21.15.27:0/1873830959 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:23:15.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.073+0000 7f5634f0c640 1 --2- 172.21.15.27:0/1873830959 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f56300b3d10 0x7f56300b66c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:15.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.073+0000 7f562ffff640 1 -- 172.21.15.27:0/1873830959 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f56300b3d10 msgr2=0x7f56300b66c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:15.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.073+0000 7f562ffff640 1 --2- 172.21.15.27:0/1873830959 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f56300b3d10 0x7f56300b66c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:15.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.073+0000 7f562ffff640 1 -- 172.21.15.27:0/1873830959 >> v1:172.21.15.45:6789/0 conn(0x7f56300b6230 legacy=0x7f56300b3600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:15.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.073+0000 7f562ffff640 1 -- 172.21.15.27:0/1873830959 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56300d2ff0 con 0x7f563015e990 2024-09-12T03:23:15.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.073+0000 7f5634f0c640 1 --2- 172.21.15.27:0/1873830959 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f56300b3d10 0x7f56300b66c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:23:15.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.074+0000 7f562ffff640 1 --2- 172.21.15.27:0/1873830959 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f563015e990 0x7f56300b7600 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7f5620009870 tx=0x7f562002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ed7a9f766517b64 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:15.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.074+0000 7f562effd640 1 -- 172.21.15.27:0/1873830959 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5620037070 con 0x7f563015e990 2024-09-12T03:23:15.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.074+0000 7f562effd640 1 -- 172.21.15.27:0/1873830959 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5620032440 con 0x7f563015e990 2024-09-12T03:23:15.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.074+0000 7f562effd640 1 -- 172.21.15.27:0/1873830959 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5620032810 con 0x7f563015e990 2024-09-12T03:23:15.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.075+0000 7f563696f640 1 -- 172.21.15.27:0/1873830959 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f563015e990 msgr2=0x7f56300b7600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:15.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.075+0000 7f563696f640 1 --2- 172.21.15.27:0/1873830959 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f563015e990 0x7f56300b7600 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7f5620009870 tx=0x7f562002a270 comp rx=0 tx=0).stop 2024-09-12T03:23:15.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.075+0000 7f563696f640 1 -- 172.21.15.27:0/1873830959 shutdown_connections 2024-09-12T03:23:15.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.075+0000 7f563696f640 1 --2- 172.21.15.27:0/1873830959 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f56300b3d10 0x7f56300b66c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:15.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.075+0000 7f563696f640 1 --2- 172.21.15.27:0/1873830959 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f563015e990 0x7f56300b7600 unknown :-1 s=CLOSED pgs=1498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:15.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.075+0000 7f563696f640 1 -- 172.21.15.27:0/1873830959 >> 172.21.15.27:0/1873830959 conn(0x7f56300d1230 msgr2=0x7f56300d1640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:15.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.075+0000 7f563696f640 1 -- 172.21.15.27:0/1873830959 shutdown_connections 2024-09-12T03:23:15.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.075+0000 7f563696f640 1 -- 172.21.15.27:0/1873830959 wait complete. 2024-09-12T03:23:15.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.076+0000 7f563696f640 1 Processor -- start 2024-09-12T03:23:15.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.076+0000 7f563696f640 1 -- start start 2024-09-12T03:23:15.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.077+0000 7f563696f640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f56300b3d10 0x7f56301510d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:15.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.077+0000 7f563696f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f56300b6230 0x7f56300c82e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:15.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.077+0000 7f563696f640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f563015e990 0x7f56300c8820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:15.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.077+0000 7f563696f640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f56300b7b40 con 0x7f563015e990 2024-09-12T03:23:15.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.077+0000 7f563696f640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f56300d35b0 con 0x7f56300b6230 2024-09-12T03:23:15.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.077+0000 7f563696f640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f56300b7cc0 con 0x7f56300b3d10 2024-09-12T03:23:15.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.077+0000 7f5634f0c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f563015e990 0x7f56300c8820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:15.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.077+0000 7f5634f0c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f563015e990 0x7f56300c8820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59158/0 (socket says 172.21.15.27:59158) 2024-09-12T03:23:15.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.077+0000 7f5634f0c640 1 -- 172.21.15.27:0/4205137788 learned_addr learned my addr 172.21.15.27:0/4205137788 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:23:15.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.077+0000 7f562f7fe640 1 --2- 172.21.15.27:0/4205137788 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f56300b6230 0x7f56300c82e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:15.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.077+0000 7f562ffff640 1 --2- 172.21.15.27:0/4205137788 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f56300b3d10 0x7f56301510d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:15.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.077+0000 7f5634f0c640 1 -- 172.21.15.27:0/4205137788 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f56300b3d10 msgr2=0x7f56301510d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:15.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.077+0000 7f5634f0c640 1 --2- 172.21.15.27:0/4205137788 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f56300b3d10 0x7f56301510d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:15.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.077+0000 7f5634f0c640 1 -- 172.21.15.27:0/4205137788 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f56300b6230 msgr2=0x7f56300c82e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:15.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.077+0000 7f5634f0c640 1 --2- 172.21.15.27:0/4205137788 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f56300b6230 0x7f56300c82e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:15.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.077+0000 7f5634f0c640 1 -- 172.21.15.27:0/4205137788 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56300cbd80 con 0x7f563015e990 2024-09-12T03:23:15.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.078+0000 7f5634f0c640 1 --2- 172.21.15.27:0/4205137788 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f563015e990 0x7f56300c8820 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7f5628007d40 tx=0x7f562800d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:15.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.078+0000 7f562cff9640 1 -- 172.21.15.27:0/4205137788 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f562801a070 con 0x7f563015e990 2024-09-12T03:23:15.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.078+0000 7f562cff9640 1 -- 172.21.15.27:0/4205137788 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5628003990 con 0x7f563015e990 2024-09-12T03:23:15.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.078+0000 7f563696f640 1 -- 172.21.15.27:0/4205137788 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56300cc060 con 0x7f563015e990 2024-09-12T03:23:15.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.078+0000 7f562cff9640 1 -- 172.21.15.27:0/4205137788 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5628003d80 con 0x7f563015e990 2024-09-12T03:23:15.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.078+0000 7f563696f640 1 -- 172.21.15.27:0/4205137788 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56300cc620 con 0x7f563015e990 2024-09-12T03:23:15.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.079+0000 7f563696f640 1 -- 172.21.15.27:0/4205137788 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56300b7600 con 0x7f563015e990 2024-09-12T03:23:15.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.079+0000 7f562cff9640 1 -- 172.21.15.27:0/4205137788 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 24) ==== 91212+0+0 (secure 0 0 0) 0x7f562801d070 con 0x7f563015e990 2024-09-12T03:23:15.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.080+0000 7f562cff9640 1 --2- 172.21.15.27:0/4205137788 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f561c06c610 0x7f561c06ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:15.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.080+0000 7f562cff9640 1 -- 172.21.15.27:0/4205137788 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(141..141 src has 1..141) ==== 4999+0+0 (secure 0 0 0) 0x7f5628089920 con 0x7f563015e990 2024-09-12T03:23:15.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.080+0000 7f562ffff640 1 --2- 172.21.15.27:0/4205137788 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f561c06c610 0x7f561c06ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:15.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.080+0000 7f562ffff640 1 --2- 172.21.15.27:0/4205137788 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f561c06c610 0x7f561c06ead0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f5620034000 tx=0x7f562002e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:15.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.084+0000 7f562cff9640 1 -- 172.21.15.27:0/4205137788 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f5628010070 con 0x7f563015e990 2024-09-12T03:23:15.260 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.260+0000 7f563696f640 1 -- 172.21.15.27:0/4205137788 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f563006e500 con 0x7f563015e990 2024-09-12T03:23:15.260 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.260+0000 7f562cff9640 1 -- 172.21.15.27:0/4205137788 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v141) ==== 93+0+10 (secure 0 0 0) 0x7f5628059570 con 0x7f563015e990 2024-09-12T03:23:15.260 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 1 2024-09-12T03:23:15.262 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.262+0000 7f563696f640 1 -- 172.21.15.27:0/4205137788 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f561c06c610 msgr2=0x7f561c06ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:15.262 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.262+0000 7f563696f640 1 --2- 172.21.15.27:0/4205137788 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f561c06c610 0x7f561c06ead0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f5620034000 tx=0x7f562002e000 comp rx=0 tx=0).stop 2024-09-12T03:23:15.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.262+0000 7f563696f640 1 -- 172.21.15.27:0/4205137788 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f563015e990 msgr2=0x7f56300c8820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:15.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.262+0000 7f563696f640 1 --2- 172.21.15.27:0/4205137788 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f563015e990 0x7f56300c8820 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7f5628007d40 tx=0x7f562800d4a0 comp rx=0 tx=0).stop 2024-09-12T03:23:15.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.262+0000 7f563696f640 1 -- 172.21.15.27:0/4205137788 shutdown_connections 2024-09-12T03:23:15.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.262+0000 7f563696f640 1 --2- 172.21.15.27:0/4205137788 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f561c06c610 0x7f561c06ead0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:15.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.262+0000 7f563696f640 1 --2- 172.21.15.27:0/4205137788 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f563015e990 0x7f56300c8820 unknown :-1 s=CLOSED pgs=1499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:15.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.262+0000 7f563696f640 1 --2- 172.21.15.27:0/4205137788 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f56300b6230 0x7f56300c82e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:15.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.262+0000 7f563696f640 1 --2- 172.21.15.27:0/4205137788 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f56300b3d10 0x7f56301510d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:15.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.262+0000 7f563696f640 1 -- 172.21.15.27:0/4205137788 >> 172.21.15.27:0/4205137788 conn(0x7f56300d1230 msgr2=0x7f56300cede0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:15.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.262+0000 7f563696f640 1 -- 172.21.15.27:0/4205137788 shutdown_connections 2024-09-12T03:23:15.264 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.262+0000 7f563696f640 1 -- 172.21.15.27:0/4205137788 wait complete. 2024-09-12T03:23:15.273 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_upgrade_v1_to_v2' 2024-09-12T03:23:15.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.375+0000 7f6ce5d31640 1 Processor -- start 2024-09-12T03:23:15.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.375+0000 7f6ce5d31640 1 -- start start 2024-09-12T03:23:15.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.376+0000 7f6ce5d31640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f6ce00b1ca0 0x7f6ce015f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:15.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.376+0000 7f6ce5d31640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f6ce006f360 0x7f6ce00cef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:15.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.376+0000 7f6ce5d31640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6ce015fd70 con 0x7f6ce00cf490 2024-09-12T03:23:15.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.376+0000 7f6ce5d31640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f6ce00cf900 con 0x7f6ce00b1ca0 2024-09-12T03:23:15.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.376+0000 7f6ce5d31640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f6ce00d22f0 con 0x7f6ce006f360 2024-09-12T03:23:15.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.376+0000 7f6cdf7fe640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f6ce00b1ca0 0x7f6ce015f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:15.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.376+0000 7f6cdf7fe640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f6ce00b1ca0 0x7f6ce015f830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59166/0 (socket says 172.21.15.27:59166) 2024-09-12T03:23:15.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.376+0000 7f6cdf7fe640 1 -- 172.21.15.27:0/831575779 learned_addr learned my addr 172.21.15.27:0/831575779 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:23:15.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.377+0000 7f6cdeffd640 1 --2- 172.21.15.27:0/831575779 >> v2:172.21.15.45:3300/0 conn(0x7f6ce006f360 0x7f6ce00cef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:15.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.377+0000 7f6cdf7fe640 1 -- 172.21.15.27:0/831575779 >> v2:172.21.15.45:3300/0 conn(0x7f6ce006f360 msgr2=0x7f6ce00cef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:15.379 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.377+0000 7f6cdf7fe640 1 --2- 172.21.15.27:0/831575779 >> v2:172.21.15.45:3300/0 conn(0x7f6ce006f360 0x7f6ce00cef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:15.379 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.377+0000 7f6cdf7fe640 1 -- 172.21.15.27:0/831575779 >> v1:172.21.15.45:6789/0 conn(0x7f6ce00cf490 legacy=0x7f6ce00d2780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:15.379 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.377+0000 7f6cdf7fe640 1 -- 172.21.15.27:0/831575779 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ce006cc40 con 0x7f6ce00b1ca0 2024-09-12T03:23:15.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.377+0000 7f6cdf7fe640 1 --2- 172.21.15.27:0/831575779 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ce00b1ca0 0x7f6ce015f830 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7f6ce00b5220 tx=0x7f6cd002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fafe7fd6c908c67 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:15.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.378+0000 7f6cde7fc640 1 -- 172.21.15.27:0/831575779 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cd0037070 con 0x7f6ce00b1ca0 2024-09-12T03:23:15.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.378+0000 7f6cde7fc640 1 -- 172.21.15.27:0/831575779 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6cd0032440 con 0x7f6ce00b1ca0 2024-09-12T03:23:15.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.378+0000 7f6cde7fc640 1 -- 172.21.15.27:0/831575779 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cd0032810 con 0x7f6ce00b1ca0 2024-09-12T03:23:15.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.378+0000 7f6ce5d31640 1 -- 172.21.15.27:0/831575779 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ce00b1ca0 msgr2=0x7f6ce015f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:15.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.378+0000 7f6ce5d31640 1 --2- 172.21.15.27:0/831575779 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ce00b1ca0 0x7f6ce015f830 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7f6ce00b5220 tx=0x7f6cd002a270 comp rx=0 tx=0).stop 2024-09-12T03:23:15.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.378+0000 7f6ce5d31640 1 -- 172.21.15.27:0/831575779 shutdown_connections 2024-09-12T03:23:15.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.378+0000 7f6ce5d31640 1 --2- 172.21.15.27:0/831575779 >> v2:172.21.15.45:3300/0 conn(0x7f6ce006f360 0x7f6ce00cef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:15.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.378+0000 7f6ce5d31640 1 --2- 172.21.15.27:0/831575779 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ce00b1ca0 0x7f6ce015f830 unknown :-1 s=CLOSED pgs=1500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:15.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.378+0000 7f6ce5d31640 1 -- 172.21.15.27:0/831575779 >> 172.21.15.27:0/831575779 conn(0x7f6ce0073800 msgr2=0x7f6ce009ba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:15.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.378+0000 7f6ce5d31640 1 -- 172.21.15.27:0/831575779 shutdown_connections 2024-09-12T03:23:15.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.378+0000 7f6ce5d31640 1 -- 172.21.15.27:0/831575779 wait complete. 2024-09-12T03:23:15.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.379+0000 7f6ce5d31640 1 Processor -- start 2024-09-12T03:23:15.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.380+0000 7f6ce5d31640 1 -- start start 2024-09-12T03:23:15.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.380+0000 7f6ce5d31640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ce006f360 0x7f6ce00b0b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:15.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.380+0000 7f6ce5d31640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6ce00b1ca0 0x7f6ce0153160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:15.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.380+0000 7f6ce5d31640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f6ce00cf490 0x7f6ce00cd8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:15.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.380+0000 7f6ce5d31640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f6ce00d1f80 con 0x7f6ce006f360 2024-09-12T03:23:15.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.380+0000 7f6ce5d31640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f6ce00d1ce0 con 0x7f6ce00b1ca0 2024-09-12T03:23:15.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.380+0000 7f6ce5d31640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f6ce006d1c0 con 0x7f6ce00cf490 2024-09-12T03:23:15.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.380+0000 7f6cdf7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ce006f360 0x7f6ce00b0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:15.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.380+0000 7f6cdf7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ce006f360 0x7f6ce00b0b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59170/0 (socket says 172.21.15.27:59170) 2024-09-12T03:23:15.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.380+0000 7f6cdf7fe640 1 -- 172.21.15.27:0/3956537276 learned_addr learned my addr 172.21.15.27:0/3956537276 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:23:15.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.381+0000 7f6cdeffd640 1 --2- 172.21.15.27:0/3956537276 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6ce00b1ca0 0x7f6ce0153160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:15.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.381+0000 7f6cdffff640 1 --2- 172.21.15.27:0/3956537276 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f6ce00cf490 0x7f6ce00cd8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:15.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.381+0000 7f6cdf7fe640 1 -- 172.21.15.27:0/3956537276 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f6ce00cf490 msgr2=0x7f6ce00cd8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:15.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.381+0000 7f6cdf7fe640 1 --2- 172.21.15.27:0/3956537276 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f6ce00cf490 0x7f6ce00cd8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:15.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.381+0000 7f6cdf7fe640 1 -- 172.21.15.27:0/3956537276 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6ce00b1ca0 msgr2=0x7f6ce0153160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:15.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.381+0000 7f6cdf7fe640 1 --2- 172.21.15.27:0/3956537276 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6ce00b1ca0 0x7f6ce0153160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:15.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.381+0000 7f6cdf7fe640 1 -- 172.21.15.27:0/3956537276 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ce00b1380 con 0x7f6ce006f360 2024-09-12T03:23:15.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.381+0000 7f6cdeffd640 1 --2- 172.21.15.27:0/3956537276 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6ce00b1ca0 0x7f6ce0153160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:23:15.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.381+0000 7f6cdf7fe640 1 --2- 172.21.15.27:0/3956537276 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ce006f360 0x7f6ce00b0b50 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7f6cd0000c00 tx=0x7f6cd00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:15.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.381+0000 7f6cc7fff640 1 -- 172.21.15.27:0/3956537276 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cd0037040 con 0x7f6ce006f360 2024-09-12T03:23:15.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.381+0000 7f6cc7fff640 1 -- 172.21.15.27:0/3956537276 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6cd0042070 con 0x7f6ce006f360 2024-09-12T03:23:15.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.381+0000 7f6cc7fff640 1 -- 172.21.15.27:0/3956537276 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cd0030880 con 0x7f6ce006f360 2024-09-12T03:23:15.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.381+0000 7f6ce5d31640 1 -- 172.21.15.27:0/3956537276 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ce00cce10 con 0x7f6ce006f360 2024-09-12T03:23:15.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.381+0000 7f6ce5d31640 1 -- 172.21.15.27:0/3956537276 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ce00b1090 con 0x7f6ce006f360 2024-09-12T03:23:15.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.382+0000 7f6ce5d31640 1 -- 172.21.15.27:0/3956537276 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ce00cfc40 con 0x7f6ce006f360 2024-09-12T03:23:15.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.382+0000 7f6cc7fff640 1 -- 172.21.15.27:0/3956537276 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 24) ==== 91212+0+0 (secure 0 0 0) 0x7f6cd0030a20 con 0x7f6ce006f360 2024-09-12T03:23:15.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.383+0000 7f6cc7fff640 1 --2- 172.21.15.27:0/3956537276 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f6ccc06c640 0x7f6ccc06eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:15.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.383+0000 7f6cc7fff640 1 -- 172.21.15.27:0/3956537276 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(141..141 src has 1..141) ==== 4999+0+0 (secure 0 0 0) 0x7f6cd00aa0d0 con 0x7f6ce006f360 2024-09-12T03:23:15.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.383+0000 7f6cdeffd640 1 --2- 172.21.15.27:0/3956537276 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f6ccc06c640 0x7f6ccc06eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:15.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.383+0000 7f6cdeffd640 1 --2- 172.21.15.27:0/3956537276 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f6ccc06c640 0x7f6ccc06eb00 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f6ce00a0a10 tx=0x7f6cd4008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:15.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.387+0000 7f6cc7fff640 1 -- 172.21.15.27:0/3956537276 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f6cd007a370 con 0x7f6ce006f360 2024-09-12T03:23:15.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.632+0000 7f6ce5d31640 1 -- 172.21.15.27:0/3956537276 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_upgrade_v1_to_v2"]} v 0) -- 0x7f6ce0161b10 con 0x7f6ce006f360 2024-09-12T03:23:15.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.649+0000 7f6cc7fff640 1 -- 172.21.15.27:0/3956537276 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_upgrade_v1_to_v2"]}]=0 v371) ==== 148+0+0 (secure 0 0 0) 0x7f6cd0080030 con 0x7f6ce006f360 2024-09-12T03:23:15.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.651+0000 7f6ce5d31640 1 -- 172.21.15.27:0/3956537276 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f6ccc06c640 msgr2=0x7f6ccc06eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:15.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.651+0000 7f6ce5d31640 1 --2- 172.21.15.27:0/3956537276 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f6ccc06c640 0x7f6ccc06eb00 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f6ce00a0a10 tx=0x7f6cd4008040 comp rx=0 tx=0).stop 2024-09-12T03:23:15.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.651+0000 7f6ce5d31640 1 -- 172.21.15.27:0/3956537276 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ce006f360 msgr2=0x7f6ce00b0b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:15.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.651+0000 7f6ce5d31640 1 --2- 172.21.15.27:0/3956537276 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ce006f360 0x7f6ce00b0b50 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7f6cd0000c00 tx=0x7f6cd00046c0 comp rx=0 tx=0).stop 2024-09-12T03:23:15.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.651+0000 7f6ce5d31640 1 -- 172.21.15.27:0/3956537276 shutdown_connections 2024-09-12T03:23:15.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.651+0000 7f6ce5d31640 1 --2- 172.21.15.27:0/3956537276 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f6ce00cf490 0x7f6ce00cd8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:15.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.651+0000 7f6ce5d31640 1 --2- 172.21.15.27:0/3956537276 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6ce00b1ca0 0x7f6ce0153160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:15.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.651+0000 7f6ce5d31640 1 --2- 172.21.15.27:0/3956537276 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f6ccc06c640 0x7f6ccc06eb00 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:15.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.651+0000 7f6ce5d31640 1 --2- 172.21.15.27:0/3956537276 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ce006f360 0x7f6ce00b0b50 unknown :-1 s=CLOSED pgs=1501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:15.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.651+0000 7f6ce5d31640 1 -- 172.21.15.27:0/3956537276 >> 172.21.15.27:0/3956537276 conn(0x7f6ce0073800 msgr2=0x7f6ce00d2780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:15.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.651+0000 7f6ce5d31640 1 -- 172.21.15.27:0/3956537276 shutdown_connections 2024-09-12T03:23:15.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.651+0000 7f6ce5d31640 1 -- 172.21.15.27:0/3956537276 wait complete. 2024-09-12T03:23:15.663 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-12T03:23:15.768 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.767+0000 7fad640d0640 1 Processor -- start 2024-09-12T03:23:15.768 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.767+0000 7fad640d0640 1 -- start start 2024-09-12T03:23:15.770 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.768+0000 7fad640d0640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fad5c15c6c0 0x7fad5c0d00a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:15.770 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.768+0000 7fad640d0640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fad5c0d05e0 0x7fad5c0d1040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:15.770 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.768+0000 7fad640d0640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fad5c0d1580 0x7fad5c0d24f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:15.770 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.768+0000 7fad640d0640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7fad5c15caa0 con 0x7fad5c15c6c0 2024-09-12T03:23:15.771 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.768+0000 7fad640d0640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fad5c0b6510 con 0x7fad5c0d1580 2024-09-12T03:23:15.771 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.768+0000 7fad640d0640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fad5c0d0a80 con 0x7fad5c0d05e0 2024-09-12T03:23:15.771 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.768+0000 7fad61e6c640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fad5c15c6c0 0x7fad5c0d00a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:15.771 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.768+0000 7fad61e6c640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fad5c15c6c0 0x7fad5c0d00a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59176/0 (socket says 172.21.15.27:59176) 2024-09-12T03:23:15.771 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.768+0000 7fad61e6c640 1 -- 172.21.15.27:0/2589056439 learned_addr learned my addr 172.21.15.27:0/2589056439 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:23:15.771 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.768+0000 7fad6266d640 1 --2- 172.21.15.27:0/2589056439 >> v2:172.21.15.45:3300/0 conn(0x7fad5c0d1580 0x7fad5c0d24f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:15.771 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.768+0000 7fad6166b640 1 --2- 172.21.15.27:0/2589056439 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fad5c0d05e0 0x7fad5c0d1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:15.771 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.768+0000 7fad61e6c640 1 -- 172.21.15.27:0/2589056439 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fad5c0d05e0 msgr2=0x7fad5c0d1040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:15.771 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.768+0000 7fad61e6c640 1 --2- 172.21.15.27:0/2589056439 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fad5c0d05e0 0x7fad5c0d1040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:15.771 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.768+0000 7fad61e6c640 1 -- 172.21.15.27:0/2589056439 >> v2:172.21.15.45:3300/0 conn(0x7fad5c0d1580 msgr2=0x7fad5c0d24f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:15.771 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.768+0000 7fad61e6c640 1 --2- 172.21.15.27:0/2589056439 >> v2:172.21.15.45:3300/0 conn(0x7fad5c0d1580 0x7fad5c0d24f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:15.772 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.768+0000 7fad61e6c640 1 -- 172.21.15.27:0/2589056439 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad5c0d0d60 con 0x7fad5c15c6c0 2024-09-12T03:23:15.772 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.769+0000 7fad61e6c640 1 --2- 172.21.15.27:0/2589056439 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fad5c15c6c0 0x7fad5c0d00a0 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7fad5c073230 tx=0x7fad5802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c41399a5354d324f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:15.772 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.769+0000 7fad60e6a640 1 -- 172.21.15.27:0/2589056439 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad58037070 con 0x7fad5c15c6c0 2024-09-12T03:23:15.772 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.769+0000 7fad60e6a640 1 -- 172.21.15.27:0/2589056439 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad58032440 con 0x7fad5c15c6c0 2024-09-12T03:23:15.772 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.769+0000 7fad60e6a640 1 -- 172.21.15.27:0/2589056439 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad58032810 con 0x7fad5c15c6c0 2024-09-12T03:23:15.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.770+0000 7fad640d0640 1 -- 172.21.15.27:0/2589056439 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fad5c15c6c0 msgr2=0x7fad5c0d00a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:15.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.770+0000 7fad640d0640 1 --2- 172.21.15.27:0/2589056439 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fad5c15c6c0 0x7fad5c0d00a0 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7fad5c073230 tx=0x7fad5802a180 comp rx=0 tx=0).stop 2024-09-12T03:23:15.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.770+0000 7fad640d0640 1 -- 172.21.15.27:0/2589056439 shutdown_connections 2024-09-12T03:23:15.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.770+0000 7fad640d0640 1 --2- 172.21.15.27:0/2589056439 >> v2:172.21.15.45:3300/0 conn(0x7fad5c0d1580 0x7fad5c0d24f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:15.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.770+0000 7fad640d0640 1 --2- 172.21.15.27:0/2589056439 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fad5c0d05e0 0x7fad5c0d1040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:15.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.770+0000 7fad640d0640 1 --2- 172.21.15.27:0/2589056439 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fad5c15c6c0 0x7fad5c0d00a0 unknown :-1 s=CLOSED pgs=1502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:15.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.770+0000 7fad640d0640 1 -- 172.21.15.27:0/2589056439 >> 172.21.15.27:0/2589056439 conn(0x7fad5c06f320 msgr2=0x7fad5c06f700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:15.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.770+0000 7fad640d0640 1 -- 172.21.15.27:0/2589056439 shutdown_connections 2024-09-12T03:23:15.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.770+0000 7fad640d0640 1 -- 172.21.15.27:0/2589056439 wait complete. 2024-09-12T03:23:15.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.771+0000 7fad640d0640 1 Processor -- start 2024-09-12T03:23:15.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.771+0000 7fad640d0640 1 -- start start 2024-09-12T03:23:15.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.772+0000 7fad640d0640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fad5c0d05e0 0x7fad5c06d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:15.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.772+0000 7fad640d0640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fad5c0d1580 0x7fad5c14b420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:15.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.772+0000 7fad640d0640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fad5c15c6c0 0x7fad5c14dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:15.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.772+0000 7fad640d0640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fad5c0d1a10 con 0x7fad5c0d05e0 2024-09-12T03:23:15.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.772+0000 7fad640d0640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fad5c0b6a50 con 0x7fad5c15c6c0 2024-09-12T03:23:15.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.772+0000 7fad640d0640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fad5c0b4400 con 0x7fad5c0d1580 2024-09-12T03:23:15.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.772+0000 7fad61e6c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fad5c0d05e0 0x7fad5c06d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:15.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.772+0000 7fad61e6c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fad5c0d05e0 0x7fad5c06d720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59190/0 (socket says 172.21.15.27:59190) 2024-09-12T03:23:15.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.772+0000 7fad61e6c640 1 -- 172.21.15.27:0/1609107571 learned_addr learned my addr 172.21.15.27:0/1609107571 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:23:15.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.772+0000 7fad6166b640 1 --2- 172.21.15.27:0/1609107571 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fad5c0d1580 0x7fad5c14b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:15.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.772+0000 7fad6266d640 1 --2- 172.21.15.27:0/1609107571 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fad5c15c6c0 0x7fad5c14dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:15.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.772+0000 7fad61e6c640 1 -- 172.21.15.27:0/1609107571 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fad5c0d1580 msgr2=0x7fad5c14b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:15.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.772+0000 7fad61e6c640 1 --2- 172.21.15.27:0/1609107571 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fad5c0d1580 0x7fad5c14b420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:15.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.772+0000 7fad61e6c640 1 -- 172.21.15.27:0/1609107571 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fad5c15c6c0 msgr2=0x7fad5c14dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:15.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.772+0000 7fad61e6c640 1 --2- 172.21.15.27:0/1609107571 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fad5c15c6c0 0x7fad5c14dd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:15.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.772+0000 7fad61e6c640 1 -- 172.21.15.27:0/1609107571 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad5c14e900 con 0x7fad5c0d05e0 2024-09-12T03:23:15.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.772+0000 7fad6166b640 1 --2- 172.21.15.27:0/1609107571 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fad5c0d1580 0x7fad5c14b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:23:15.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.772+0000 7fad61e6c640 1 --2- 172.21.15.27:0/1609107571 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fad5c0d05e0 0x7fad5c06d720 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7fad58009870 tx=0x7fad580046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:15.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.773+0000 7fad527fc640 1 -- 172.21.15.27:0/1609107571 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad58041070 con 0x7fad5c0d05e0 2024-09-12T03:23:15.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.773+0000 7fad527fc640 1 -- 172.21.15.27:0/1609107571 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad5803c870 con 0x7fad5c0d05e0 2024-09-12T03:23:15.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.773+0000 7fad640d0640 1 -- 172.21.15.27:0/1609107571 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad5c14ebe0 con 0x7fad5c0d05e0 2024-09-12T03:23:15.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.773+0000 7fad527fc640 1 -- 172.21.15.27:0/1609107571 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad58037040 con 0x7fad5c0d05e0 2024-09-12T03:23:15.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.773+0000 7fad640d0640 1 -- 172.21.15.27:0/1609107571 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad5c06cf20 con 0x7fad5c0d05e0 2024-09-12T03:23:15.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.774+0000 7fad640d0640 1 -- 172.21.15.27:0/1609107571 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad5c0d1040 con 0x7fad5c0d05e0 2024-09-12T03:23:15.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.774+0000 7fad527fc640 1 -- 172.21.15.27:0/1609107571 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 24) ==== 91212+0+0 (secure 0 0 0) 0x7fad58044070 con 0x7fad5c0d05e0 2024-09-12T03:23:15.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.775+0000 7fad527fc640 1 --2- 172.21.15.27:0/1609107571 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fad4006c610 0x7fad4006ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:15.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.775+0000 7fad527fc640 1 -- 172.21.15.27:0/1609107571 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(141..141 src has 1..141) ==== 4999+0+0 (secure 0 0 0) 0x7fad580aa580 con 0x7fad5c0d05e0 2024-09-12T03:23:15.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.775+0000 7fad6166b640 1 --2- 172.21.15.27:0/1609107571 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fad4006c610 0x7fad4006ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:15.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.775+0000 7fad6166b640 1 --2- 172.21.15.27:0/1609107571 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fad4006c610 0x7fad4006ead0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fad5c15df50 tx=0x7fad4c008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:15.780 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.779+0000 7fad527fc640 1 -- 172.21.15.27:0/1609107571 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fad5807a940 con 0x7fad5c0d05e0 2024-09-12T03:23:15.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.953+0000 7fad640d0640 1 -- 172.21.15.27:0/1609107571 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fad5c069f80 con 0x7fad5c0d05e0 2024-09-12T03:23:15.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.958+0000 7fad527fc640 1 -- 172.21.15.27:0/1609107571 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v28)=0 v28) ==== 122+0+0 (secure 0 0 0) 0x7fad5807a240 con 0x7fad5c0d05e0 2024-09-12T03:23:15.961 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.960+0000 7fad640d0640 1 -- 172.21.15.27:0/1609107571 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fad4006c610 msgr2=0x7fad4006ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:15.961 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.960+0000 7fad640d0640 1 --2- 172.21.15.27:0/1609107571 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fad4006c610 0x7fad4006ead0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fad5c15df50 tx=0x7fad4c008040 comp rx=0 tx=0).stop 2024-09-12T03:23:15.961 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.960+0000 7fad640d0640 1 -- 172.21.15.27:0/1609107571 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fad5c0d05e0 msgr2=0x7fad5c06d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:15.962 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.960+0000 7fad640d0640 1 --2- 172.21.15.27:0/1609107571 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fad5c0d05e0 0x7fad5c06d720 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7fad58009870 tx=0x7fad580046c0 comp rx=0 tx=0).stop 2024-09-12T03:23:15.962 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.960+0000 7fad640d0640 1 -- 172.21.15.27:0/1609107571 shutdown_connections 2024-09-12T03:23:15.962 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.960+0000 7fad640d0640 1 --2- 172.21.15.27:0/1609107571 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fad5c15c6c0 0x7fad5c14dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:15.962 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.960+0000 7fad640d0640 1 --2- 172.21.15.27:0/1609107571 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fad5c0d1580 0x7fad5c14b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:15.962 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.960+0000 7fad640d0640 1 --2- 172.21.15.27:0/1609107571 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fad4006c610 0x7fad4006ead0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:15.962 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.960+0000 7fad640d0640 1 --2- 172.21.15.27:0/1609107571 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fad5c0d05e0 0x7fad5c06d720 unknown :-1 s=CLOSED pgs=1503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:15.962 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.960+0000 7fad640d0640 1 -- 172.21.15.27:0/1609107571 >> 172.21.15.27:0/1609107571 conn(0x7fad5c06f320 msgr2=0x7fad5c0b6f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:15.962 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.960+0000 7fad640d0640 1 -- 172.21.15.27:0/1609107571 shutdown_connections 2024-09-12T03:23:15.962 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:15.960+0000 7fad640d0640 1 -- 172.21.15.27:0/1609107571 wait complete. 2024-09-12T03:23:15.972 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-12T03:23:15.973 DEBUG:teuthology.orchestra.run.smithi027:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-12T03:23:16.014 DEBUG:teuthology.orchestra.run.smithi045:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-12T03:23:16.068 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-12T03:23:16.068 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-12T03:23:16.069 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T03:23:16.085 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056004,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497432k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16324 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248716k,nr_inodes=812179,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497432k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-12T03:23:16.085 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-12T03:23:16.085 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T03:23:16.104 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056795,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16602 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249348k,nr_inodes=812337,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-12T03:23:16.105 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-12T03:23:16.105 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T03:23:16.142 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056004,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497432k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16324 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248716k,nr_inodes=812179,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497432k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-12T03:23:16.142 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-12T03:23:16.142 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T03:23:16.163 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056795,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16602 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249348k,nr_inodes=812337,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-12T03:23:16.163 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-12T03:23:16.164 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-12T03:23:16.285 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:16.284+0000 7fbb3eb81640 1 Processor -- start 2024-09-12T03:23:16.285 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:16.284+0000 7fbb3eb81640 1 -- start start 2024-09-12T03:23:16.287 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:16.285+0000 7fbb3eb81640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fbb3815b2b0 0x7fbb380b3560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:16.287 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:16.285+0000 7fbb3eb81640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fbb38072d70 0x7fbb38072430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:16.287 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:16.285+0000 7fbb3eb81640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbb380ce410 con 0x7fbb3806e9b0 2024-09-12T03:23:16.287 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:16.285+0000 7fbb3eb81640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fbb380720a0 con 0x7fbb3815b2b0 2024-09-12T03:23:16.287 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:16.285+0000 7fbb3eb81640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fbb38072970 con 0x7fbb38072d70 2024-09-12T03:23:16.287 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:16.285+0000 7fbb3c91d640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fbb3815b2b0 0x7fbb380b3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:16.287 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:16.285+0000 7fbb37fff640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fbb38072d70 0x7fbb38072430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:16.287 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:16.285+0000 7fbb3d11e640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fbb3806e9b0 0x7fbb380cebb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.27:50944/0 (socket says 172.21.15.27:50944) 2024-09-12T03:23:16.287 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:16.285+0000 7fbb3d11e640 1 -- 172.21.15.27:0/3993756060 learned_addr learned my addr 172.21.15.27:0/3993756060 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:23:16.287 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:16.286+0000 7fbb377fe640 1 -- 172.21.15.27:0/3993756060 <== mon.1 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4185385057 0 0) 0x7fbb30001490 con 0x7fbb3806e9b0 2024-09-12T03:23:16.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:16.286+0000 7fbb377fe640 1 -- 172.21.15.27:0/3993756060 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbb28003520 con 0x7fbb3806e9b0 2024-09-12T03:23:16.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:16.286+0000 7fbb37fff640 1 -- 172.21.15.27:0/3993756060 >> v2:172.21.15.45:3300/0 conn(0x7fbb3815b2b0 msgr2=0x7fbb380b3560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:16.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:16.286+0000 7fbb37fff640 1 --2- 172.21.15.27:0/3993756060 >> v2:172.21.15.45:3300/0 conn(0x7fbb3815b2b0 0x7fbb380b3560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:16.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:16.286+0000 7fbb37fff640 1 -- 172.21.15.27:0/3993756060 >> v1:172.21.15.45:6789/0 conn(0x7fbb3806e9b0 legacy=0x7fbb380cebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:16.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:16.286+0000 7fbb37fff640 1 -- 172.21.15.27:0/3993756060 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb380732c0 con 0x7fbb38072d70 2024-09-12T03:23:16.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:16.286+0000 7fbb3c91d640 1 --2- 172.21.15.27:0/3993756060 >> v2:172.21.15.45:3300/0 conn(0x7fbb3815b2b0 0x7fbb380b3560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:23:16.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:16.286+0000 7fbb37fff640 1 --2- 172.21.15.27:0/3993756060 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fbb38072d70 0x7fbb38072430 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fbb240097a0 tx=0x7fbb2402a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=31907275be542e40 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:16.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:16.287+0000 7fbb377fe640 1 -- 172.21.15.27:0/3993756060 <== mon.2 v2:172.21.15.45:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb24037070 con 0x7fbb38072d70 2024-09-12T03:23:16.290 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:16.287+0000 7fbb377fe640 1 -- 172.21.15.27:0/3993756060 <== mon.2 v2:172.21.15.45:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb24032440 con 0x7fbb38072d70 2024-09-12T03:23:16.290 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:16.287+0000 7fbb377fe640 1 -- 172.21.15.27:0/3993756060 <== mon.2 v2:172.21.15.45:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb240327f0 con 0x7fbb38072d70 2024-09-12T03:23:16.290 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:16.287+0000 7fbb3eb81640 1 -- 172.21.15.27:0/3993756060 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fbb38072d70 msgr2=0x7fbb38072430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:16.290 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:16.287+0000 7fbb3eb81640 1 --2- 172.21.15.27:0/3993756060 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fbb38072d70 0x7fbb38072430 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fbb240097a0 tx=0x7fbb2402a180 comp rx=0 tx=0).stop 2024-09-12T03:23:16.290 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:16.287+0000 7fbb3eb81640 1 -- 172.21.15.27:0/3993756060 shutdown_connections 2024-09-12T03:23:16.290 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:16.287+0000 7fbb3eb81640 1 --2- 172.21.15.27:0/3993756060 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fbb38072d70 0x7fbb38072430 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:16.290 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:16.287+0000 7fbb3eb81640 1 --2- 172.21.15.27:0/3993756060 >> v2:172.21.15.45:3300/0 conn(0x7fbb3815b2b0 0x7fbb380b3560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:16.290 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:16.287+0000 7fbb3eb81640 1 -- 172.21.15.27:0/3993756060 >> 172.21.15.27:0/3993756060 conn(0x7fbb38062c60 msgr2=0x7fbb3806bb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:16.290 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:16.287+0000 7fbb3eb81640 1 -- 172.21.15.27:0/3993756060 shutdown_connections 2024-09-12T03:23:16.290 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:16.287+0000 7fbb3eb81640 1 -- 172.21.15.27:0/3993756060 wait complete. 2024-09-12T03:23:16.290 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:16.288+0000 7fbb3eb81640 1 Processor -- start 2024-09-12T03:23:16.290 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:16.289+0000 7fbb3eb81640 1 -- start start 2024-09-12T03:23:16.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:16.289+0000 7fbb3eb81640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbb3806e9b0 0x7fbb3806db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:16.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:16.289+0000 7fbb3eb81640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbb38072d70 0x7fbb38147a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:16.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:16.289+0000 7fbb3eb81640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fbb3815b2b0 0x7fbb38147f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:16.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:16.289+0000 7fbb3eb81640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fbb380ce690 con 0x7fbb38072d70 2024-09-12T03:23:16.293 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:16.289+0000 7fbb3eb81640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fbb38071dc0 con 0x7fbb3806e9b0 2024-09-12T03:23:16.293 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:16.289+0000 7fbb3eb81640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fbb3815b690 con 0x7fbb3815b2b0 2024-09-12T03:23:16.293 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:16.290+0000 7fbb37fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbb38072d70 0x7fbb38147a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:16.293 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:16.290+0000 7fbb37fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbb38072d70 0x7fbb38147a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59204/0 (socket says 172.21.15.27:59204) 2024-09-12T03:23:16.293 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:16.290+0000 7fbb37fff640 1 -- 172.21.15.27:0/896365230 learned_addr learned my addr 172.21.15.27:0/896365230 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:23:16.293 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:16.290+0000 7fbb3c91d640 1 --2- 172.21.15.27:0/896365230 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbb3806e9b0 0x7fbb3806db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:16.293 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:16.290+0000 7fbb3d11e640 1 --2- 172.21.15.27:0/896365230 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fbb3815b2b0 0x7fbb38147f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:16.293 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:16.290+0000 7fbb37fff640 1 -- 172.21.15.27:0/896365230 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fbb3815b2b0 msgr2=0x7fbb38147f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:16.293 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:16.290+0000 7fbb37fff640 1 --2- 172.21.15.27:0/896365230 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fbb3815b2b0 0x7fbb38147f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:16.293 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:16.290+0000 7fbb37fff640 1 -- 172.21.15.27:0/896365230 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbb3806e9b0 msgr2=0x7fbb3806db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:16.293 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:16.290+0000 7fbb37fff640 1 --2- 172.21.15.27:0/896365230 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbb3806e9b0 0x7fbb3806db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:16.293 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:16.290+0000 7fbb37fff640 1 -- 172.21.15.27:0/896365230 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb3815dc50 con 0x7fbb38072d70 2024-09-12T03:23:16.293 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:16.290+0000 7fbb37fff640 1 --2- 172.21.15.27:0/896365230 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbb38072d70 0x7fbb38147a50 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7fbb2402a1c0 tx=0x7fbb240046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:16.293 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:16.290+0000 7fbb357fa640 1 -- 172.21.15.27:0/896365230 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb24037040 con 0x7fbb38072d70 2024-09-12T03:23:16.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:16.290+0000 7fbb357fa640 1 -- 172.21.15.27:0/896365230 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb24042070 con 0x7fbb38072d70 2024-09-12T03:23:16.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:16.290+0000 7fbb3eb81640 1 -- 172.21.15.27:0/896365230 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb3815df30 con 0x7fbb38072d70 2024-09-12T03:23:16.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:16.290+0000 7fbb357fa640 1 -- 172.21.15.27:0/896365230 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb2403b930 con 0x7fbb38072d70 2024-09-12T03:23:16.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:16.290+0000 7fbb3eb81640 1 -- 172.21.15.27:0/896365230 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb38156370 con 0x7fbb38072d70 2024-09-12T03:23:16.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:16.291+0000 7fbb3eb81640 1 -- 172.21.15.27:0/896365230 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb38072430 con 0x7fbb38072d70 2024-09-12T03:23:16.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:16.292+0000 7fbb357fa640 1 -- 172.21.15.27:0/896365230 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 24) ==== 91212+0+0 (secure 0 0 0) 0x7fbb2403bad0 con 0x7fbb38072d70 2024-09-12T03:23:16.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:16.292+0000 7fbb357fa640 1 --2- 172.21.15.27:0/896365230 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fbb1c06c610 0x7fbb1c06ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:16.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:16.292+0000 7fbb357fa640 1 -- 172.21.15.27:0/896365230 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(141..141 src has 1..141) ==== 4999+0+0 (secure 0 0 0) 0x7fbb240a9f90 con 0x7fbb38072d70 2024-09-12T03:23:16.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:16.292+0000 7fbb3c91d640 1 --2- 172.21.15.27:0/896365230 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fbb1c06c610 0x7fbb1c06ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:16.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:16.293+0000 7fbb3c91d640 1 --2- 172.21.15.27:0/896365230 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fbb1c06c610 0x7fbb1c06ead0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fbb2c009560 tx=0x7fbb2c009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:16.297 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:16.297+0000 7fbb357fa640 1 -- 172.21.15.27:0/896365230 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fbb24079be0 con 0x7fbb38072d70 2024-09-12T03:23:16.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:16.507+0000 7fbb3eb81640 1 -- 172.21.15.27:0/896365230 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbb380b3600 con 0x7fbb38072d70 2024-09-12T03:23:16.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:16.508+0000 7fbb357fa640 1 -- 172.21.15.27:0/896365230 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 85 v85) ==== 94+0+3911 (secure 0 0 0) 0x7fbb2407dee0 con 0x7fbb38072d70 2024-09-12T03:23:16.509 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:23:16.509 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":85,"btime":"2024-09-12T03:23:11:625132+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6689,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/2285501348","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2285501348},{"type":"v1","addr":"172.21.15.45:6833","nonce":2285501348}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79},{"gid":7453,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/1872420883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":1872420883},{"type":"v1","addr":"172.21.15.45:6835","nonce":1872420883}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79},{"gid":8193,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6834/3968488955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":3968488955},{"type":"v1","addr":"172.21.15.27:6834","nonce":3968488955}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79},{"gid":8484,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/4091527414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":4091527414},{"type":"v1","addr":"172.21.15.27:6835","nonce":4091527414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85}],"filesystems":[]} 2024-09-12T03:23:16.509 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 85 2024-09-12T03:23:16.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:16.510+0000 7fbb3eb81640 1 -- 172.21.15.27:0/896365230 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fbb1c06c610 msgr2=0x7fbb1c06ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:16.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:16.510+0000 7fbb3eb81640 1 --2- 172.21.15.27:0/896365230 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fbb1c06c610 0x7fbb1c06ead0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fbb2c009560 tx=0x7fbb2c009190 comp rx=0 tx=0).stop 2024-09-12T03:23:16.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:16.510+0000 7fbb3eb81640 1 -- 172.21.15.27:0/896365230 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbb38072d70 msgr2=0x7fbb38147a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:16.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:16.510+0000 7fbb3eb81640 1 --2- 172.21.15.27:0/896365230 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbb38072d70 0x7fbb38147a50 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7fbb2402a1c0 tx=0x7fbb240046c0 comp rx=0 tx=0).stop 2024-09-12T03:23:16.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:16.510+0000 7fbb3eb81640 1 -- 172.21.15.27:0/896365230 shutdown_connections 2024-09-12T03:23:16.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:16.510+0000 7fbb3eb81640 1 --2- 172.21.15.27:0/896365230 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fbb3815b2b0 0x7fbb38147f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:16.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:16.510+0000 7fbb3eb81640 1 --2- 172.21.15.27:0/896365230 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fbb1c06c610 0x7fbb1c06ead0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:16.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:16.510+0000 7fbb3eb81640 1 --2- 172.21.15.27:0/896365230 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbb38072d70 0x7fbb38147a50 unknown :-1 s=CLOSED pgs=1504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:16.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:16.510+0000 7fbb3eb81640 1 --2- 172.21.15.27:0/896365230 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbb3806e9b0 0x7fbb3806db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:16.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:16.510+0000 7fbb3eb81640 1 -- 172.21.15.27:0/896365230 >> 172.21.15.27:0/896365230 conn(0x7fbb38062c60 msgr2=0x7fbb380b5310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:16.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:16.510+0000 7fbb3eb81640 1 -- 172.21.15.27:0/896365230 shutdown_connections 2024-09-12T03:23:16.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:16.510+0000 7fbb3eb81640 1 -- 172.21.15.27:0/896365230 wait complete. 2024-09-12T03:23:16.522 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-12T03:23:16.522 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-12T03:23:16.523 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-12T03:23:16.523 INFO:teuthology.orchestra.run:waiting for 300 2024-09-12T03:23:16.523 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-12T03:23:16.523 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-12T03:23:16.523 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-12T03:23:16.523 INFO:teuthology.orchestra.run:waiting for 300 2024-09-12T03:23:16.523 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-12T03:23:16.523 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-12T03:23:16.524 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-12T03:23:16.524 INFO:teuthology.orchestra.run:waiting for 300 2024-09-12T03:23:16.524 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-12T03:23:16.524 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-12T03:23:16.524 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-12T03:23:16.524 INFO:teuthology.orchestra.run:waiting for 300 2024-09-12T03:23:16.532 INFO:tasks.ceph.mds.a:Stopped 2024-09-12T03:23:16.532 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-12T03:23:16.533 INFO:tasks.ceph.mds.c:Stopped 2024-09-12T03:23:16.533 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-12T03:23:16.536 INFO:tasks.ceph.mds.b:Stopped 2024-09-12T03:23:16.536 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-12T03:23:16.537 INFO:tasks.ceph.mds.d:Stopped 2024-09-12T03:23:16.537 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-12T03:23:16.575 INFO:tasks.ceph.mds.a:Started 2024-09-12T03:23:16.576 DEBUG:teuthology.parallel:result is None 2024-09-12T03:23:16.576 INFO:tasks.ceph.mds.c:Started 2024-09-12T03:23:16.576 DEBUG:teuthology.parallel:result is None 2024-09-12T03:23:16.579 INFO:tasks.ceph.mds.b:Started 2024-09-12T03:23:16.579 DEBUG:teuthology.parallel:result is None 2024-09-12T03:23:16.580 INFO:tasks.ceph.mds.d:Started 2024-09-12T03:23:16.580 DEBUG:teuthology.parallel:result is None 2024-09-12T03:23:16.580 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-12T03:23:16.656 INFO:tasks.ceph.mds.c.smithi027.stdout:starting mds.c at 2024-09-12T03:23:16.657 INFO:tasks.ceph.mds.a.smithi027.stdout:starting mds.a at 2024-09-12T03:23:16.666 INFO:tasks.ceph.mds.d.smithi045.stdout:starting mds.d at 2024-09-12T03:23:16.667 INFO:tasks.ceph.mds.b.smithi045.stdout:starting mds.b at 2024-09-12T03:23:16.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:16.689+0000 7fbea5427640 1 Processor -- start 2024-09-12T03:23:16.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:16.689+0000 7fbea5427640 1 -- start start 2024-09-12T03:23:16.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:16.690+0000 7fbea5427640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fbea015eb90 0x7fbea00d2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:16.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:16.690+0000 7fbea5427640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fbea00d5200 0x7fbea00d1010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:16.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:16.690+0000 7fbea5427640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbea00d3760 con 0x7fbea00d1550 2024-09-12T03:23:16.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:16.690+0000 7fbea5427640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7fbea00b7d20 con 0x7fbea00d5200 2024-09-12T03:23:16.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:16.690+0000 7fbea5427640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fbea00d42e0 con 0x7fbea015eb90 2024-09-12T03:23:16.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:16.690+0000 7fbe97fff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fbea00d5200 0x7fbea00d1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:16.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:16.690+0000 7fbe97fff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fbea00d5200 0x7fbea00d1010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59276/0 (socket says 172.21.15.27:59276) 2024-09-12T03:23:16.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:16.690+0000 7fbe97fff640 1 -- 172.21.15.27:0/527487099 learned_addr learned my addr 172.21.15.27:0/527487099 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:23:16.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:16.690+0000 7fbe9effd640 1 --2- 172.21.15.27:0/527487099 >> v2:172.21.15.45:3300/0 conn(0x7fbea015eb90 0x7fbea00d2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:16.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:16.690+0000 7fbe97fff640 1 -- 172.21.15.27:0/527487099 >> v2:172.21.15.45:3300/0 conn(0x7fbea015eb90 msgr2=0x7fbea00d2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:16.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:16.690+0000 7fbe97fff640 1 --2- 172.21.15.27:0/527487099 >> v2:172.21.15.45:3300/0 conn(0x7fbea015eb90 0x7fbea00d2df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:16.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:16.690+0000 7fbe97fff640 1 -- 172.21.15.27:0/527487099 >> v1:172.21.15.45:6789/0 conn(0x7fbea00d1550 legacy=0x7fbea00cf200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:16.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:16.690+0000 7fbe97fff640 1 -- 172.21.15.27:0/527487099 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbea00d3a40 con 0x7fbea00d5200 2024-09-12T03:23:16.693 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:16.691+0000 7fbe97fff640 1 --2- 172.21.15.27:0/527487099 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbea00d5200 0x7fbea00d1010 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7fbea0075560 tx=0x7fbe8802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd241e59a7fba338 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:16.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:16.691+0000 7fbe9e7fc640 1 -- 172.21.15.27:0/527487099 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe88037070 con 0x7fbea00d5200 2024-09-12T03:23:16.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:16.691+0000 7fbe9e7fc640 1 -- 172.21.15.27:0/527487099 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe88032440 con 0x7fbea00d5200 2024-09-12T03:23:16.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:16.691+0000 7fbe9e7fc640 1 -- 172.21.15.27:0/527487099 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe88032810 con 0x7fbea00d5200 2024-09-12T03:23:16.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:16.692+0000 7fbea5427640 1 -- 172.21.15.27:0/527487099 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbea00d5200 msgr2=0x7fbea00d1010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:16.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:16.692+0000 7fbea5427640 1 --2- 172.21.15.27:0/527487099 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbea00d5200 0x7fbea00d1010 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7fbea0075560 tx=0x7fbe8802a270 comp rx=0 tx=0).stop 2024-09-12T03:23:16.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:16.692+0000 7fbea5427640 1 -- 172.21.15.27:0/527487099 shutdown_connections 2024-09-12T03:23:16.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:16.692+0000 7fbea5427640 1 --2- 172.21.15.27:0/527487099 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbea00d5200 0x7fbea00d1010 unknown :-1 s=CLOSED pgs=1513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:16.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:16.692+0000 7fbea5427640 1 --2- 172.21.15.27:0/527487099 >> v2:172.21.15.45:3300/0 conn(0x7fbea015eb90 0x7fbea00d2df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:16.695 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:16.692+0000 7fbea5427640 1 -- 172.21.15.27:0/527487099 >> 172.21.15.27:0/527487099 conn(0x7fbea006dce0 msgr2=0x7fbea006e0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:16.695 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:16.692+0000 7fbea5427640 1 -- 172.21.15.27:0/527487099 shutdown_connections 2024-09-12T03:23:16.695 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:16.692+0000 7fbea5427640 1 -- 172.21.15.27:0/527487099 wait complete. 2024-09-12T03:23:16.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:16.693+0000 7fbea5427640 1 Processor -- start 2024-09-12T03:23:16.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:16.694+0000 7fbea5427640 1 -- start start 2024-09-12T03:23:16.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:16.694+0000 7fbea5427640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fbea00d1550 0x7fbea015a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:16.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:16.694+0000 7fbea5427640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbea00d5200 0x7fbea015a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:16.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:16.694+0000 7fbea5427640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbea015eb90 0x7fbea01f7210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:16.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:16.694+0000 7fbea5427640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fbea00d1960 con 0x7fbea015eb90 2024-09-12T03:23:16.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:16.694+0000 7fbea5427640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fbea00d4000 con 0x7fbea00d5200 2024-09-12T03:23:16.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:16.694+0000 7fbea5427640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fbea00d3330 con 0x7fbea00d1550 2024-09-12T03:23:16.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:16.694+0000 7fbe9f7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbea015eb90 0x7fbea01f7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:16.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:16.694+0000 7fbe97fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbea00d5200 0x7fbea015a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:16.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:16.694+0000 7fbe9f7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbea015eb90 0x7fbea01f7210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59282/0 (socket says 172.21.15.27:59282) 2024-09-12T03:23:16.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:16.694+0000 7fbe9effd640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fbea00d1550 0x7fbea015a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:16.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:16.694+0000 7fbe9f7fe640 1 -- 172.21.15.27:0/2275341738 learned_addr learned my addr 172.21.15.27:0/2275341738 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:23:16.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:16.695+0000 7fbe9f7fe640 1 -- 172.21.15.27:0/2275341738 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fbea00d1550 msgr2=0x7fbea015a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:16.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:16.695+0000 7fbe9f7fe640 1 --2- 172.21.15.27:0/2275341738 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fbea00d1550 0x7fbea015a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:16.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:16.695+0000 7fbe9f7fe640 1 -- 172.21.15.27:0/2275341738 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbea00d5200 msgr2=0x7fbea015a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:16.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:16.695+0000 7fbe9f7fe640 1 --2- 172.21.15.27:0/2275341738 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbea00d5200 0x7fbea015a070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:16.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:16.695+0000 7fbe9f7fe640 1 -- 172.21.15.27:0/2275341738 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbea015afa0 con 0x7fbea015eb90 2024-09-12T03:23:16.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:16.695+0000 7fbe9f7fe640 1 --2- 172.21.15.27:0/2275341738 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbea015eb90 0x7fbea01f7210 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7fbe9800c730 tx=0x7fbe9800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:16.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:16.695+0000 7fbe977fe640 1 -- 172.21.15.27:0/2275341738 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe98003b70 con 0x7fbea015eb90 2024-09-12T03:23:16.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:16.695+0000 7fbe977fe640 1 -- 172.21.15.27:0/2275341738 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe98003d30 con 0x7fbea015eb90 2024-09-12T03:23:16.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:16.695+0000 7fbe977fe640 1 -- 172.21.15.27:0/2275341738 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe98005310 con 0x7fbea015eb90 2024-09-12T03:23:16.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:16.695+0000 7fbea5427640 1 -- 172.21.15.27:0/2275341738 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbea01f6170 con 0x7fbea015eb90 2024-09-12T03:23:16.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:16.695+0000 7fbea5427640 1 -- 172.21.15.27:0/2275341738 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbea01f6730 con 0x7fbea015eb90 2024-09-12T03:23:16.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:16.696+0000 7fbea5427640 1 -- 172.21.15.27:0/2275341738 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbea015ef70 con 0x7fbea015eb90 2024-09-12T03:23:16.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:16.697+0000 7fbe977fe640 1 -- 172.21.15.27:0/2275341738 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 24) ==== 91212+0+0 (secure 0 0 0) 0x7fbe980056c0 con 0x7fbea015eb90 2024-09-12T03:23:16.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:16.697+0000 7fbe977fe640 1 --2- 172.21.15.27:0/2275341738 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fbe8006c640 0x7fbe8006eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:16.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:16.697+0000 7fbe977fe640 1 -- 172.21.15.27:0/2275341738 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(141..141 src has 1..141) ==== 4999+0+0 (secure 0 0 0) 0x7fbe98089730 con 0x7fbea015eb90 2024-09-12T03:23:16.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:16.697+0000 7fbe9effd640 1 --2- 172.21.15.27:0/2275341738 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fbe8006c640 0x7fbe8006eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:16.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:16.698+0000 7fbe9effd640 1 --2- 172.21.15.27:0/2275341738 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fbe8006c640 0x7fbe8006eb00 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fbea00b9950 tx=0x7fbe90009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:16.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:16.701+0000 7fbe977fe640 1 -- 172.21.15.27:0/2275341738 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fbea015afa0 con 0x7fbea015eb90 2024-09-12T03:23:16.890 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:16.889+0000 7fbea5427640 1 -- 172.21.15.27:0/2275341738 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7fbea0069ea0 con 0x7fbea015eb90 2024-09-12T03:23:17.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:17.660+0000 7fbe977fe640 1 -- 172.21.15.27:0/2275341738 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v143) ==== 97+0+0 (secure 0 0 0) 0x7fbe98059380 con 0x7fbea015eb90 2024-09-12T03:23:17.661 INFO:teuthology.orchestra.run.smithi027.stderr: removed all blocklist entries 2024-09-12T03:23:17.664 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:17.663+0000 7fbea5427640 1 -- 172.21.15.27:0/2275341738 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fbe8006c640 msgr2=0x7fbe8006eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:17.664 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:17.663+0000 7fbea5427640 1 --2- 172.21.15.27:0/2275341738 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fbe8006c640 0x7fbe8006eb00 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fbea00b9950 tx=0x7fbe90009190 comp rx=0 tx=0).stop 2024-09-12T03:23:17.664 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:17.663+0000 7fbea5427640 1 -- 172.21.15.27:0/2275341738 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbea015eb90 msgr2=0x7fbea01f7210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:17.665 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:17.663+0000 7fbea5427640 1 --2- 172.21.15.27:0/2275341738 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbea015eb90 0x7fbea01f7210 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7fbe9800c730 tx=0x7fbe9800cbf0 comp rx=0 tx=0).stop 2024-09-12T03:23:17.665 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:17.663+0000 7fbea5427640 1 -- 172.21.15.27:0/2275341738 shutdown_connections 2024-09-12T03:23:17.665 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:17.663+0000 7fbea5427640 1 --2- 172.21.15.27:0/2275341738 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fbe8006c640 0x7fbe8006eb00 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:17.665 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:17.663+0000 7fbea5427640 1 --2- 172.21.15.27:0/2275341738 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbea015eb90 0x7fbea01f7210 unknown :-1 s=CLOSED pgs=1514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:17.665 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:17.663+0000 7fbea5427640 1 --2- 172.21.15.27:0/2275341738 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbea00d5200 0x7fbea015a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:17.665 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:17.663+0000 7fbea5427640 1 --2- 172.21.15.27:0/2275341738 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fbea00d1550 0x7fbea015a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:17.665 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:17.663+0000 7fbea5427640 1 -- 172.21.15.27:0/2275341738 >> 172.21.15.27:0/2275341738 conn(0x7fbea006dce0 msgr2=0x7fbea0160e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:17.665 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:17.663+0000 7fbea5427640 1 -- 172.21.15.27:0/2275341738 shutdown_connections 2024-09-12T03:23:17.665 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:17.664+0000 7fbea5427640 1 -- 172.21.15.27:0/2275341738 wait complete. 2024-09-12T03:23:17.680 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-12T03:23:17.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:17.788+0000 7f7f1325c640 1 Processor -- start 2024-09-12T03:23:17.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:17.788+0000 7f7f1325c640 1 -- start start 2024-09-12T03:23:17.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:17.789+0000 7f7f1325c640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f7f0c0ce6d0 0x7f7f0c0ceab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:17.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:17.789+0000 7f7f1325c640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f7f0c160d80 0x7f7f0c06bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:17.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:17.789+0000 7f7f1325c640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f7f0c06c070 0x7f7f0c06e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:17.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:17.789+0000 7f7f1325c640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f7f0c1463d0 con 0x7f7f0c0ce6d0 2024-09-12T03:23:17.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:17.789+0000 7f7f1325c640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f7f0c0ceff0 con 0x7f7f0c160d80 2024-09-12T03:23:17.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:17.789+0000 7f7f1325c640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f7f0c06c7f0 con 0x7f7f0c06c070 2024-09-12T03:23:17.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:17.789+0000 7f7f10ff8640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f7f0c0ce6d0 0x7f7f0c0ceab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:17.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:17.789+0000 7f7f10ff8640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f7f0c0ce6d0 0x7f7f0c0ceab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59288/0 (socket says 172.21.15.27:59288) 2024-09-12T03:23:17.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:17.789+0000 7f7f10ff8640 1 -- 172.21.15.27:0/2799592339 learned_addr learned my addr 172.21.15.27:0/2799592339 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:23:17.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:17.789+0000 7f7f117f9640 1 --2- 172.21.15.27:0/2799592339 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f7f0c06c070 0x7f7f0c06e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:17.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:17.789+0000 7f7f0bfff640 1 --2- 172.21.15.27:0/2799592339 >> v2:172.21.15.45:3300/0 conn(0x7f7f0c160d80 0x7f7f0c06bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:17.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:17.789+0000 7f7f10ff8640 1 -- 172.21.15.27:0/2799592339 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f7f0c06c070 msgr2=0x7f7f0c06e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:17.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:17.789+0000 7f7f10ff8640 1 --2- 172.21.15.27:0/2799592339 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f7f0c06c070 0x7f7f0c06e5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:17.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:17.789+0000 7f7f10ff8640 1 -- 172.21.15.27:0/2799592339 >> v2:172.21.15.45:3300/0 conn(0x7f7f0c160d80 msgr2=0x7f7f0c06bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:17.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:17.789+0000 7f7f10ff8640 1 --2- 172.21.15.27:0/2799592339 >> v2:172.21.15.45:3300/0 conn(0x7f7f0c160d80 0x7f7f0c06bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:17.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:17.789+0000 7f7f10ff8640 1 -- 172.21.15.27:0/2799592339 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f0c1602a0 con 0x7f7f0c0ce6d0 2024-09-12T03:23:17.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:17.790+0000 7f7f10ff8640 1 --2- 172.21.15.27:0/2799592339 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7f0c0ce6d0 0x7f7f0c0ceab0 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7f7f0c0ce350 tx=0x7f7f0002a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8bfc8f00b46b6aa server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:17.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:17.790+0000 7f7f0b7fe640 1 -- 172.21.15.27:0/2799592339 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f00037070 con 0x7f7f0c0ce6d0 2024-09-12T03:23:17.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:17.790+0000 7f7f0b7fe640 1 -- 172.21.15.27:0/2799592339 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f00032440 con 0x7f7f0c0ce6d0 2024-09-12T03:23:17.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:17.790+0000 7f7f0b7fe640 1 -- 172.21.15.27:0/2799592339 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f00032810 con 0x7f7f0c0ce6d0 2024-09-12T03:23:17.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:17.791+0000 7f7f1325c640 1 -- 172.21.15.27:0/2799592339 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7f0c0ce6d0 msgr2=0x7f7f0c0ceab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:17.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:17.791+0000 7f7f1325c640 1 --2- 172.21.15.27:0/2799592339 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7f0c0ce6d0 0x7f7f0c0ceab0 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7f7f0c0ce350 tx=0x7f7f0002a140 comp rx=0 tx=0).stop 2024-09-12T03:23:17.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:17.791+0000 7f7f1325c640 1 -- 172.21.15.27:0/2799592339 shutdown_connections 2024-09-12T03:23:17.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:17.791+0000 7f7f1325c640 1 --2- 172.21.15.27:0/2799592339 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f7f0c06c070 0x7f7f0c06e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:17.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:17.791+0000 7f7f1325c640 1 --2- 172.21.15.27:0/2799592339 >> v2:172.21.15.45:3300/0 conn(0x7f7f0c160d80 0x7f7f0c06bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:17.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:17.791+0000 7f7f1325c640 1 --2- 172.21.15.27:0/2799592339 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7f0c0ce6d0 0x7f7f0c0ceab0 unknown :-1 s=CLOSED pgs=1515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:17.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:17.791+0000 7f7f1325c640 1 -- 172.21.15.27:0/2799592339 >> 172.21.15.27:0/2799592339 conn(0x7f7f0c06cc40 msgr2=0x7f7f0c06d050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:17.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:17.791+0000 7f7f1325c640 1 -- 172.21.15.27:0/2799592339 shutdown_connections 2024-09-12T03:23:17.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:17.791+0000 7f7f1325c640 1 -- 172.21.15.27:0/2799592339 wait complete. 2024-09-12T03:23:17.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:17.792+0000 7f7f1325c640 1 Processor -- start 2024-09-12T03:23:17.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:17.792+0000 7f7f1325c640 1 -- start start 2024-09-12T03:23:17.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:17.793+0000 7f7f1325c640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f7f0c06c070 0x7f7f0c13ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:17.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:17.793+0000 7f7f1325c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7f0c0ce6d0 0x7f7f0c0b3900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:17.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:17.793+0000 7f7f1325c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7f0c160d80 0x7f7f0c15b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:17.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:17.793+0000 7f7f1325c640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f7f0c06c500 con 0x7f7f0c0ce6d0 2024-09-12T03:23:17.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:17.793+0000 7f7f1325c640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f7f0c160860 con 0x7f7f0c160d80 2024-09-12T03:23:17.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:17.793+0000 7f7f1325c640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f7f0c06eaf0 con 0x7f7f0c06c070 2024-09-12T03:23:17.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:17.793+0000 7f7f0bfff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7f0c0ce6d0 0x7f7f0c0b3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:17.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:17.793+0000 7f7f0bfff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7f0c0ce6d0 0x7f7f0c0b3900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59294/0 (socket says 172.21.15.27:59294) 2024-09-12T03:23:17.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:17.793+0000 7f7f0bfff640 1 -- 172.21.15.27:0/124609299 learned_addr learned my addr 172.21.15.27:0/124609299 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:23:17.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:17.793+0000 7f7f10ff8640 1 --2- 172.21.15.27:0/124609299 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f7f0c06c070 0x7f7f0c13ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:17.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:17.793+0000 7f7f117f9640 1 --2- 172.21.15.27:0/124609299 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7f0c160d80 0x7f7f0c15b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:17.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:17.794+0000 7f7f0bfff640 1 -- 172.21.15.27:0/124609299 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f7f0c06c070 msgr2=0x7f7f0c13ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:17.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:17.794+0000 7f7f0bfff640 1 --2- 172.21.15.27:0/124609299 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f7f0c06c070 0x7f7f0c13ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:17.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:17.794+0000 7f7f0bfff640 1 -- 172.21.15.27:0/124609299 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7f0c160d80 msgr2=0x7f7f0c15b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:17.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:17.794+0000 7f7f0bfff640 1 --2- 172.21.15.27:0/124609299 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7f0c160d80 0x7f7f0c15b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:17.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:17.794+0000 7f7f0bfff640 1 -- 172.21.15.27:0/124609299 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f0c13f290 con 0x7f7f0c0ce6d0 2024-09-12T03:23:17.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:17.794+0000 7f7f0bfff640 1 --2- 172.21.15.27:0/124609299 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7f0c0ce6d0 0x7f7f0c0b3900 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7f7ef800d5f0 tx=0x7f7ef800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:17.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:17.794+0000 7f7f097fa640 1 -- 172.21.15.27:0/124609299 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ef80125f0 con 0x7f7f0c0ce6d0 2024-09-12T03:23:17.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:17.794+0000 7f7f097fa640 1 -- 172.21.15.27:0/124609299 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ef8019070 con 0x7f7f0c0ce6d0 2024-09-12T03:23:17.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:17.794+0000 7f7f1325c640 1 -- 172.21.15.27:0/124609299 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f0c13df90 con 0x7f7f0c0ce6d0 2024-09-12T03:23:17.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:17.794+0000 7f7f097fa640 1 -- 172.21.15.27:0/124609299 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ef80129d0 con 0x7f7f0c0ce6d0 2024-09-12T03:23:17.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:17.794+0000 7f7f1325c640 1 -- 172.21.15.27:0/124609299 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f0c13e550 con 0x7f7f0c0ce6d0 2024-09-12T03:23:17.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:17.795+0000 7f7f1325c640 1 -- 172.21.15.27:0/124609299 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f0c0ceab0 con 0x7f7f0c0ce6d0 2024-09-12T03:23:17.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:17.795+0000 7f7f097fa640 1 -- 172.21.15.27:0/124609299 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 24) ==== 91212+0+0 (secure 0 0 0) 0x7f7ef8012b70 con 0x7f7f0c0ce6d0 2024-09-12T03:23:17.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:17.796+0000 7f7f097fa640 1 --2- 172.21.15.27:0/124609299 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f7ef006c610 0x7f7ef006ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:17.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:17.796+0000 7f7f10ff8640 1 --2- 172.21.15.27:0/124609299 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f7ef006c610 0x7f7ef006ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:17.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:17.796+0000 7f7f097fa640 1 -- 172.21.15.27:0/124609299 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 4913+0+0 (secure 0 0 0) 0x7f7ef8089ac0 con 0x7f7f0c0ce6d0 2024-09-12T03:23:17.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:17.797+0000 7f7f10ff8640 1 --2- 172.21.15.27:0/124609299 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f7ef006c610 0x7f7ef006ead0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f7f00034000 tx=0x7f7f0002e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:17.801 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:17.800+0000 7f7f097fa640 1 -- 172.21.15.27:0/124609299 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f7ef8059710 con 0x7f7f0c0ce6d0 2024-09-12T03:23:18.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:18.044+0000 7f7f1325c640 1 -- 172.21.15.27:0/124609299 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f7f0c06e390 con 0x7f7f0c0ce6d0 2024-09-12T03:23:18.046 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:18.045+0000 7f7f097fa640 1 -- 172.21.15.27:0/124609299 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v21) ==== 80+0+5304 (secure 0 0 0) 0x7f7ef805da10 con 0x7f7f0c0ce6d0 2024-09-12T03:23:18.046 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:23:18.046 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-12T03:23:18.046 INFO:teuthology.orchestra.run.smithi027.stdout: "auth_dump": [ 2024-09-12T03:23:18.046 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:23:18.046 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "mds.a", 2024-09-12T03:23:18.046 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDUXOJmIhxjNxAAaGg8SwtW6NRe/lIgNCV20w==", 2024-09-12T03:23:18.046 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:23:18.046 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow", 2024-09-12T03:23:18.046 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow *", 2024-09-12T03:23:18.047 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow *", 2024-09-12T03:23:18.047 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2024-09-12T03:23:18.047 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:23:18.047 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:23:18.047 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:23:18.047 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "mds.b", 2024-09-12T03:23:18.047 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDVXOJm63kWCBAAzjfOsBsWUI+IAsuZ49yEkw==", 2024-09-12T03:23:18.047 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:23:18.047 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow", 2024-09-12T03:23:18.047 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow *", 2024-09-12T03:23:18.047 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow *", 2024-09-12T03:23:18.047 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2024-09-12T03:23:18.047 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:23:18.048 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:23:18.048 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:23:18.048 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "mds.c", 2024-09-12T03:23:18.048 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDVXOJm0sX7AhAAdynCcho2a/AgFF9Cw3+vAw==", 2024-09-12T03:23:18.048 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:23:18.048 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow", 2024-09-12T03:23:18.048 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow *", 2024-09-12T03:23:18.048 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow *", 2024-09-12T03:23:18.048 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2024-09-12T03:23:18.048 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:23:18.048 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:23:18.048 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:23:18.048 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "mds.d", 2024-09-12T03:23:18.049 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDVXOJm7Dd8DxAANHYmvKgpRZ6sVfXDSgPHWw==", 2024-09-12T03:23:18.049 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:23:18.049 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow", 2024-09-12T03:23:18.049 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow *", 2024-09-12T03:23:18.049 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow *", 2024-09-12T03:23:18.049 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2024-09-12T03:23:18.049 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:23:18.049 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:23:18.049 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:23:18.049 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "osd.0", 2024-09-12T03:23:18.049 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDYXOJmJTVYLxAAmo8kxRjaiwTT44dZLZlq4A==", 2024-09-12T03:23:18.049 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:23:18.049 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow profile osd", 2024-09-12T03:23:18.049 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile osd", 2024-09-12T03:23:18.050 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2024-09-12T03:23:18.050 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:23:18.050 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:23:18.050 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:23:18.050 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "osd.1", 2024-09-12T03:23:18.050 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDcXOJmYV0BFBAAmoGX9gf9yLoE/qkWFnv9tg==", 2024-09-12T03:23:18.050 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:23:18.050 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow profile osd", 2024-09-12T03:23:18.050 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile osd", 2024-09-12T03:23:18.050 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2024-09-12T03:23:18.050 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:23:18.050 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:23:18.050 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:23:18.050 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "osd.2", 2024-09-12T03:23:18.051 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDfXOJmEetiJRAAXk+AvfTfhjsB6T2UB4Z2/w==", 2024-09-12T03:23:18.051 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:23:18.051 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow profile osd", 2024-09-12T03:23:18.051 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile osd", 2024-09-12T03:23:18.051 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2024-09-12T03:23:18.051 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:23:18.051 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:23:18.051 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:23:18.051 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "osd.3", 2024-09-12T03:23:18.051 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDiXOJmrQbPJxAAQsi2B5a1LnPTPGZVdo7j7w==", 2024-09-12T03:23:18.051 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:23:18.051 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow profile osd", 2024-09-12T03:23:18.051 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile osd", 2024-09-12T03:23:18.051 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2024-09-12T03:23:18.052 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:23:18.052 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:23:18.052 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:23:18.052 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "osd.4", 2024-09-12T03:23:18.052 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDnXOJmoMr1NxAAv3u2q8hYt9w//9gz+N7UAQ==", 2024-09-12T03:23:18.052 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:23:18.052 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow profile osd", 2024-09-12T03:23:18.052 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile osd", 2024-09-12T03:23:18.052 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2024-09-12T03:23:18.052 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:23:18.052 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:23:18.052 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:23:18.052 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "osd.5", 2024-09-12T03:23:18.053 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDqXOJmxpUPOxAAxLH6RJsyLOZ2XMfzM0fQ1g==", 2024-09-12T03:23:18.053 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:23:18.053 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow profile osd", 2024-09-12T03:23:18.053 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile osd", 2024-09-12T03:23:18.053 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2024-09-12T03:23:18.053 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:23:18.053 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:23:18.053 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:23:18.053 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "osd.6", 2024-09-12T03:23:18.053 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDuXOJmuP0+LxAAMHoz5eSO81hPtkdBIS2cig==", 2024-09-12T03:23:18.053 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:23:18.053 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow profile osd", 2024-09-12T03:23:18.053 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile osd", 2024-09-12T03:23:18.054 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2024-09-12T03:23:18.054 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:23:18.054 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:23:18.054 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:23:18.054 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "osd.7", 2024-09-12T03:23:18.054 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDxXOJmuIC/MRAAPDM+lAHqdoQyGVODwjFp+Q==", 2024-09-12T03:23:18.054 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:23:18.054 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow profile osd", 2024-09-12T03:23:18.054 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile osd", 2024-09-12T03:23:18.054 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2024-09-12T03:23:18.054 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:23:18.054 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:23:18.054 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:23:18.054 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "client.0", 2024-09-12T03:23:18.055 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDVXOJmLm+5ExAAECM4CKfuVElmp/n0b02mXg==", 2024-09-12T03:23:18.055 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:23:18.055 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow", 2024-09-12T03:23:18.055 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow r", 2024-09-12T03:23:18.055 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-12T03:23:18.055 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:23:18.055 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:23:18.055 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:23:18.055 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "client.1", 2024-09-12T03:23:18.055 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDVXOJm61/FHBAALCQouXySfQeZ5DESGmqkzA==", 2024-09-12T03:23:18.055 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:23:18.055 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow", 2024-09-12T03:23:18.055 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow r", 2024-09-12T03:23:18.056 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow rw", 2024-09-12T03:23:18.056 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow rwx" 2024-09-12T03:23:18.056 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:23:18.056 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:23:18.056 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:23:18.056 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "client.2", 2024-09-12T03:23:18.056 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDVXOJmq/ZkGRAAOzmQzEIhb4b/YTxayc34Bw==", 2024-09-12T03:23:18.056 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:23:18.056 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow", 2024-09-12T03:23:18.056 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow r", 2024-09-12T03:23:18.056 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow rw", 2024-09-12T03:23:18.056 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow rwx" 2024-09-12T03:23:18.056 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:23:18.056 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:23:18.057 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:23:18.057 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "client.3", 2024-09-12T03:23:18.057 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDVXOJm+DlzIhAATQKZe+Zyk0MM8efTa6g9PA==", 2024-09-12T03:23:18.057 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:23:18.057 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow", 2024-09-12T03:23:18.057 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow r", 2024-09-12T03:23:18.057 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow rw", 2024-09-12T03:23:18.057 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow rwx" 2024-09-12T03:23:18.057 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:23:18.057 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:23:18.057 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:23:18.057 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "client.admin", 2024-09-12T03:23:18.057 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDUXOJm+JGWGhAApTQw9RtJWd/J5QzQaUFXvg==", 2024-09-12T03:23:18.058 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:23:18.058 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow *", 2024-09-12T03:23:18.058 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow *", 2024-09-12T03:23:18.058 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow *", 2024-09-12T03:23:18.058 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2024-09-12T03:23:18.058 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:23:18.058 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:23:18.058 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:23:18.058 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "mgr.x", 2024-09-12T03:23:18.058 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDUXOJmZcsONBAAMTY2KykuHaEkv1t6Vz5KjA==", 2024-09-12T03:23:18.058 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:23:18.058 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow *", 2024-09-12T03:23:18.058 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile mgr", 2024-09-12T03:23:18.058 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2024-09-12T03:23:18.058 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:23:18.059 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:23:18.059 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:23:18.059 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "mgr.y", 2024-09-12T03:23:18.059 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDUXOJmJOgZMBAAQ/JTxfhox7HgxBMtsvUCBg==", 2024-09-12T03:23:18.059 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:23:18.059 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow *", 2024-09-12T03:23:18.059 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile mgr", 2024-09-12T03:23:18.059 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2024-09-12T03:23:18.059 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:23:18.059 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:23:18.059 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-12T03:23:18.059 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-12T03:23:18.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:18.047+0000 7f7f1325c640 1 -- 172.21.15.27:0/124609299 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f7ef006c610 msgr2=0x7f7ef006ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:18.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:18.047+0000 7f7f1325c640 1 --2- 172.21.15.27:0/124609299 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f7ef006c610 0x7f7ef006ead0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f7f00034000 tx=0x7f7f0002e000 comp rx=0 tx=0).stop 2024-09-12T03:23:18.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:18.047+0000 7f7f1325c640 1 -- 172.21.15.27:0/124609299 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7f0c0ce6d0 msgr2=0x7f7f0c0b3900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:18.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:18.047+0000 7f7f1325c640 1 --2- 172.21.15.27:0/124609299 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7f0c0ce6d0 0x7f7f0c0b3900 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7f7ef800d5f0 tx=0x7f7ef800dab0 comp rx=0 tx=0).stop 2024-09-12T03:23:18.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:18.047+0000 7f7f1325c640 1 -- 172.21.15.27:0/124609299 shutdown_connections 2024-09-12T03:23:18.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:18.047+0000 7f7f1325c640 1 --2- 172.21.15.27:0/124609299 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7f0c160d80 0x7f7f0c15b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:18.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:18.047+0000 7f7f1325c640 1 --2- 172.21.15.27:0/124609299 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f7ef006c610 0x7f7ef006ead0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:18.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:18.047+0000 7f7f1325c640 1 --2- 172.21.15.27:0/124609299 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7f0c0ce6d0 0x7f7f0c0b3900 unknown :-1 s=CLOSED pgs=1516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:18.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:18.047+0000 7f7f1325c640 1 --2- 172.21.15.27:0/124609299 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f7f0c06c070 0x7f7f0c13ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:18.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:18.047+0000 7f7f1325c640 1 -- 172.21.15.27:0/124609299 >> 172.21.15.27:0/124609299 conn(0x7f7f0c06cc40 msgr2=0x7f7f0c0b1230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:18.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:18.047+0000 7f7f1325c640 1 -- 172.21.15.27:0/124609299 shutdown_connections 2024-09-12T03:23:18.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:18.047+0000 7f7f1325c640 1 -- 172.21.15.27:0/124609299 wait complete. 2024-09-12T03:23:18.062 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-12T03:23:18.164 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:18.163+0000 7f69dcf0a640 1 Processor -- start 2024-09-12T03:23:18.164 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:18.163+0000 7f69dcf0a640 1 -- start start 2024-09-12T03:23:18.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:18.164+0000 7f69dcf0a640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f69d80b1ca0 0x7f69d815f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:18.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:18.164+0000 7f69dcf0a640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f69d80cf660 0x7f69d80d2780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:18.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:18.164+0000 7f69dcf0a640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f69d815fd70 con 0x7f69d806f360 2024-09-12T03:23:18.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:18.164+0000 7f69dcf0a640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f69d80d22f0 con 0x7f69d80b1ca0 2024-09-12T03:23:18.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:18.164+0000 7f69dcf0a640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f69d806cc40 con 0x7f69d80cf660 2024-09-12T03:23:18.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:18.164+0000 7f69d5d9b640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f69d806f360 0x7f69d80cef50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:48642/0 (socket says 172.21.15.27:48642) 2024-09-12T03:23:18.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:18.164+0000 7f69d5d9b640 1 -- 172.21.15.27:0/1420213977 learned_addr learned my addr 172.21.15.27:0/1420213977 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:23:18.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:18.164+0000 7f69d559a640 1 -- 172.21.15.27:0/1420213977 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1787679380 0 0) 0x7f69cc001110 con 0x7f69d806f360 2024-09-12T03:23:18.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:18.164+0000 7f69d659c640 1 --2- 172.21.15.27:0/1420213977 >> v2:172.21.15.45:3300/0 conn(0x7f69d80b1ca0 0x7f69d815f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:18.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:18.165+0000 7f69d559a640 1 -- 172.21.15.27:0/1420213977 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f69c0003520 con 0x7f69d806f360 2024-09-12T03:23:18.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:18.165+0000 7f69d6d9d640 1 --2- 172.21.15.27:0/1420213977 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f69d80cf660 0x7f69d80d2780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:18.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:18.165+0000 7f69d559a640 1 -- 172.21.15.27:0/1420213977 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1425796381 0 0) 0x7f69c0003520 con 0x7f69d806f360 2024-09-12T03:23:18.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:18.165+0000 7f69d559a640 1 -- 172.21.15.27:0/1420213977 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f69d80cf660 msgr2=0x7f69d80d2780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:18.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:18.165+0000 7f69d559a640 1 --2- 172.21.15.27:0/1420213977 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f69d80cf660 0x7f69d80d2780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:18.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:18.165+0000 7f69d559a640 1 -- 172.21.15.27:0/1420213977 >> v2:172.21.15.45:3300/0 conn(0x7f69d80b1ca0 msgr2=0x7f69d815f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:18.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:18.165+0000 7f69d559a640 1 --2- 172.21.15.27:0/1420213977 >> v2:172.21.15.45:3300/0 conn(0x7f69d80b1ca0 0x7f69d815f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:18.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:18.165+0000 7f69d559a640 1 -- 172.21.15.27:0/1420213977 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69d80d1ce0 con 0x7f69d806f360 2024-09-12T03:23:18.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:18.165+0000 7f69d559a640 1 -- 172.21.15.27:0/1420213977 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 847755213 0 0) 0x7f69cc001840 con 0x7f69d806f360 2024-09-12T03:23:18.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:18.165+0000 7f69d559a640 1 -- 172.21.15.27:0/1420213977 >> v1:172.21.15.27:6789/0 conn(0x7f69d806f360 legacy=0x7f69d80cef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:18.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:18.165+0000 7f69d559a640 1 --2- 172.21.15.27:0/1420213977 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f69c00039b0 0x7f69c0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:18.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:18.165+0000 7f69d559a640 1 --2- 172.21.15.27:0/1420213977 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f69d80cf660 0x7f69c0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:18.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:18.165+0000 7f69d559a640 1 --2- 172.21.15.27:0/1420213977 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f69c0004d40 0x7f69c00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:18.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:18.165+0000 7f69d559a640 1 -- 172.21.15.27:0/1420213977 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f69d80d22f0 con 0x7f69c00039b0 2024-09-12T03:23:18.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:18.165+0000 7f69d559a640 1 -- 172.21.15.27:0/1420213977 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f69d806cc40 con 0x7f69c0004d40 2024-09-12T03:23:18.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:18.165+0000 7f69d559a640 1 -- 172.21.15.27:0/1420213977 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f69c000b9c0 con 0x7f69d80cf660 2024-09-12T03:23:18.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:18.165+0000 7f69d659c640 1 --2- 172.21.15.27:0/1420213977 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f69c00039b0 0x7f69c0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:18.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:18.165+0000 7f69d5d9b640 1 --2- 172.21.15.27:0/1420213977 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f69d80cf660 0x7f69c0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:18.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:18.165+0000 7f69d6d9d640 1 --2- 172.21.15.27:0/1420213977 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f69c0004d40 0x7f69c00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:18.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:18.165+0000 7f69d659c640 1 -- 172.21.15.27:0/1420213977 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f69d80cf660 msgr2=0x7f69c0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:18.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:18.165+0000 7f69d659c640 1 --2- 172.21.15.27:0/1420213977 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f69d80cf660 0x7f69c0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:18.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:18.165+0000 7f69d659c640 1 -- 172.21.15.27:0/1420213977 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f69c0004d40 msgr2=0x7f69c00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:18.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:18.165+0000 7f69d659c640 1 --2- 172.21.15.27:0/1420213977 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f69c0004d40 0x7f69c00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:18.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:18.165+0000 7f69d659c640 1 -- 172.21.15.27:0/1420213977 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69c0003520 con 0x7f69c00039b0 2024-09-12T03:23:18.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:18.166+0000 7f69d659c640 1 --2- 172.21.15.27:0/1420213977 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f69c00039b0 0x7f69c0003dd0 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7f69c8009af0 tx=0x7f69c0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbff7a4fb3736b38 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:18.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:18.166+0000 7f69d559a640 1 -- 172.21.15.27:0/1420213977 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69c8036070 con 0x7f69c00039b0 2024-09-12T03:23:18.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:18.166+0000 7f69d559a640 1 -- 172.21.15.27:0/1420213977 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69c8031400 con 0x7f69c00039b0 2024-09-12T03:23:18.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:18.166+0000 7f69d559a640 1 -- 172.21.15.27:0/1420213977 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69c8009690 con 0x7f69c00039b0 2024-09-12T03:23:18.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:18.167+0000 7f69dcf0a640 1 -- 172.21.15.27:0/1420213977 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f69c00039b0 msgr2=0x7f69c0003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:18.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:18.167+0000 7f69dcf0a640 1 --2- 172.21.15.27:0/1420213977 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f69c00039b0 0x7f69c0003dd0 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7f69c8009af0 tx=0x7f69c0005830 comp rx=0 tx=0).stop 2024-09-12T03:23:18.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:18.167+0000 7f69d6d9d640 1 -- 172.21.15.27:0/1420213977 reap_dead start 2024-09-12T03:23:18.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:18.167+0000 7f69dcf0a640 1 -- 172.21.15.27:0/1420213977 shutdown_connections 2024-09-12T03:23:18.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:18.167+0000 7f69dcf0a640 1 -- 172.21.15.27:0/1420213977 >> 172.21.15.27:0/1420213977 conn(0x7f69d8073800 msgr2=0x7f69d809ba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:18.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:18.167+0000 7f69dcf0a640 1 -- 172.21.15.27:0/1420213977 shutdown_connections 2024-09-12T03:23:18.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:18.167+0000 7f69dcf0a640 1 -- 172.21.15.27:0/1420213977 wait complete. 2024-09-12T03:23:18.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:18.168+0000 7f69dcf0a640 1 Processor -- start 2024-09-12T03:23:18.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:18.168+0000 7f69dcf0a640 1 -- start start 2024-09-12T03:23:18.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:18.169+0000 7f69dcf0a640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f69c00039b0 0x7f69d8156d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:18.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:18.169+0000 7f69dcf0a640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f69d81574e0 0x7f69d8158d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:18.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:18.169+0000 7f69dcf0a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f69d815ce30 0x7f69d80b0030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:18.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:18.169+0000 7f69dcf0a640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f69d806d1c0 con 0x7f69c00039b0 2024-09-12T03:23:18.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:18.169+0000 7f69dcf0a640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f69d80d1f80 con 0x7f69d815ce30 2024-09-12T03:23:18.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:18.169+0000 7f69dcf0a640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f69d806d340 con 0x7f69d81574e0 2024-09-12T03:23:18.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:18.169+0000 7f69d659c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f69c00039b0 0x7f69d8156d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:18.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:18.169+0000 7f69d659c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f69c00039b0 0x7f69d8156d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59300/0 (socket says 172.21.15.27:59300) 2024-09-12T03:23:18.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:18.169+0000 7f69d659c640 1 -- 172.21.15.27:0/4126108912 learned_addr learned my addr 172.21.15.27:0/4126108912 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:23:18.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:18.169+0000 7f69d659c640 1 -- 172.21.15.27:0/4126108912 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f69d81574e0 msgr2=0x7f69d8158d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:18.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:18.169+0000 7f69d659c640 1 --2- 172.21.15.27:0/4126108912 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f69d81574e0 0x7f69d8158d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:18.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:18.169+0000 7f69d659c640 1 -- 172.21.15.27:0/4126108912 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f69d815ce30 msgr2=0x7f69d80b0030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:18.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:18.169+0000 7f69d659c640 1 --2- 172.21.15.27:0/4126108912 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f69d815ce30 0x7f69d80b0030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:18.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:18.169+0000 7f69d659c640 1 -- 172.21.15.27:0/4126108912 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69d8157290 con 0x7f69c00039b0 2024-09-12T03:23:18.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:18.169+0000 7f69d659c640 1 --2- 172.21.15.27:0/4126108912 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f69c00039b0 0x7f69d8156d50 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7f69c8034040 tx=0x7f69c8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:18.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:18.170+0000 7f69c6ffd640 1 -- 172.21.15.27:0/4126108912 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69c8041020 con 0x7f69c00039b0 2024-09-12T03:23:18.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:18.170+0000 7f69c6ffd640 1 -- 172.21.15.27:0/4126108912 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69c802ade0 con 0x7f69c00039b0 2024-09-12T03:23:18.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:18.170+0000 7f69c6ffd640 1 -- 172.21.15.27:0/4126108912 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69c8036040 con 0x7f69c00039b0 2024-09-12T03:23:18.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:18.170+0000 7f69dcf0a640 1 -- 172.21.15.27:0/4126108912 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69d81595c0 con 0x7f69c00039b0 2024-09-12T03:23:18.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:18.170+0000 7f69dcf0a640 1 -- 172.21.15.27:0/4126108912 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69d80af830 con 0x7f69c00039b0 2024-09-12T03:23:18.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:18.171+0000 7f69dcf0a640 1 -- 172.21.15.27:0/4126108912 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69d80cfc40 con 0x7f69c00039b0 2024-09-12T03:23:18.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:18.171+0000 7f69c6ffd640 1 -- 172.21.15.27:0/4126108912 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 24) ==== 91212+0+0 (secure 0 0 0) 0x7f69c8044020 con 0x7f69c00039b0 2024-09-12T03:23:18.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:18.172+0000 7f69c6ffd640 1 --2- 172.21.15.27:0/4126108912 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f69b806c640 0x7f69b806eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:18.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:18.172+0000 7f69c6ffd640 1 -- 172.21.15.27:0/4126108912 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 4913+0+0 (secure 0 0 0) 0x7f69c80a9420 con 0x7f69c00039b0 2024-09-12T03:23:18.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:18.172+0000 7f69d5d9b640 1 --2- 172.21.15.27:0/4126108912 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f69b806c640 0x7f69b806eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:18.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:18.172+0000 7f69d5d9b640 1 --2- 172.21.15.27:0/4126108912 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f69b806c640 0x7f69b806eb00 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f69cc000f60 tx=0x7f69cc000ed0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:18.177 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:18.176+0000 7f69c6ffd640 1 -- 172.21.15.27:0/4126108912 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f69c80796c0 con 0x7f69c00039b0 2024-09-12T03:23:18.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:18.356+0000 7f69dcf0a640 1 -- 172.21.15.27:0/4126108912 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f69d815f530 con 0x7f69c00039b0 2024-09-12T03:23:18.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:18.357+0000 7f69c6ffd640 1 -- 172.21.15.27:0/4126108912 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v145) ==== 84+0+65 (secure 0 0 0) 0x7f69c8078fc0 con 0x7f69c00039b0 2024-09-12T03:23:18.358 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:23:18.358 INFO:teuthology.orchestra.run.smithi027.stdout:[ 2024-09-12T03:23:18.358 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:23:18.358 INFO:teuthology.orchestra.run.smithi027.stdout: "poolnum": 1, 2024-09-12T03:23:18.358 INFO:teuthology.orchestra.run.smithi027.stdout: "poolname": ".mgr" 2024-09-12T03:23:18.358 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:23:18.358 INFO:teuthology.orchestra.run.smithi027.stdout:] 2024-09-12T03:23:18.360 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:18.358+0000 7f69dcf0a640 1 -- 172.21.15.27:0/4126108912 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f69b806c640 msgr2=0x7f69b806eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:18.360 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:18.358+0000 7f69dcf0a640 1 --2- 172.21.15.27:0/4126108912 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f69b806c640 0x7f69b806eb00 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f69cc000f60 tx=0x7f69cc000ed0 comp rx=0 tx=0).stop 2024-09-12T03:23:18.360 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:18.358+0000 7f69dcf0a640 1 -- 172.21.15.27:0/4126108912 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f69c00039b0 msgr2=0x7f69d8156d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:18.360 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:18.358+0000 7f69dcf0a640 1 --2- 172.21.15.27:0/4126108912 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f69c00039b0 0x7f69d8156d50 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7f69c8034040 tx=0x7f69c8004730 comp rx=0 tx=0).stop 2024-09-12T03:23:18.360 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:18.358+0000 7f69dcf0a640 1 -- 172.21.15.27:0/4126108912 shutdown_connections 2024-09-12T03:23:18.360 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:18.358+0000 7f69dcf0a640 1 --2- 172.21.15.27:0/4126108912 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f69d815ce30 0x7f69d80b0030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:18.360 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:18.358+0000 7f69dcf0a640 1 --2- 172.21.15.27:0/4126108912 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f69d81574e0 0x7f69d8158d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:18.360 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:18.358+0000 7f69dcf0a640 1 --2- 172.21.15.27:0/4126108912 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f69b806c640 0x7f69b806eb00 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:18.360 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:18.358+0000 7f69dcf0a640 1 --2- 172.21.15.27:0/4126108912 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f69c00039b0 0x7f69d8156d50 unknown :-1 s=CLOSED pgs=1519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:18.360 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:18.358+0000 7f69dcf0a640 1 -- 172.21.15.27:0/4126108912 >> 172.21.15.27:0/4126108912 conn(0x7f69d8073800 msgr2=0x7f69d80d1550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:18.360 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:18.359+0000 7f69dcf0a640 1 -- 172.21.15.27:0/4126108912 shutdown_connections 2024-09-12T03:23:18.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:18.359+0000 7f69dcf0a640 1 -- 172.21.15.27:0/4126108912 wait complete. 2024-09-12T03:23:18.370 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-12T03:23:18.370 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-12T03:23:18.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:18.476+0000 7f88a2474640 1 Processor -- start 2024-09-12T03:23:18.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:18.476+0000 7f88a2474640 1 -- start start 2024-09-12T03:23:18.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:18.477+0000 7f88a2474640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f889c162040 0x7f889c0d1da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:18.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:18.477+0000 7f88a2474640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f889c0d3790 0x7f889c0b7a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:18.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:18.477+0000 7f88a2474640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f889c0d2e30 con 0x7f889c06d8c0 2024-09-12T03:23:18.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:18.477+0000 7f88a2474640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f889c0b7f90 con 0x7f889c162040 2024-09-12T03:23:18.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:18.477+0000 7f88a2474640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f889c162420 con 0x7f889c0d3790 2024-09-12T03:23:18.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:18.477+0000 7f889bfff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f889c162040 0x7f889c0d1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:18.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:18.477+0000 7f88a0a11640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f889c06d8c0 0x7f889c0cf320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:48656/0 (socket says 172.21.15.27:48656) 2024-09-12T03:23:18.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:18.477+0000 7f88a0a11640 1 -- 172.21.15.27:0/2179906807 learned_addr learned my addr 172.21.15.27:0/2179906807 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:23:18.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:18.477+0000 7f889b7fe640 1 --2- 172.21.15.27:0/2179906807 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f889c0d3790 0x7f889c0b7a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:18.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:18.477+0000 7f889bfff640 1 -- 172.21.15.27:0/2179906807 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f889c0d3790 msgr2=0x7f889c0b7a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:18.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:18.477+0000 7f889bfff640 1 --2- 172.21.15.27:0/2179906807 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f889c0d3790 0x7f889c0b7a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:18.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:18.477+0000 7f889bfff640 1 -- 172.21.15.27:0/2179906807 >> v1:172.21.15.27:6789/0 conn(0x7f889c06d8c0 legacy=0x7f889c0cf320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:18.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:18.477+0000 7f889bfff640 1 -- 172.21.15.27:0/2179906807 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f889c0d30d0 con 0x7f889c162040 2024-09-12T03:23:18.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:18.477+0000 7f889affd640 1 -- 172.21.15.27:0/2179906807 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3314160387 0 0) 0x7f889c0d2e30 con 0x7f889c06d8c0 2024-09-12T03:23:18.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:18.477+0000 7f889b7fe640 1 --2- 172.21.15.27:0/2179906807 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f889c0d3790 0x7f889c0b7a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:23:18.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:18.478+0000 7f889bfff640 1 --2- 172.21.15.27:0/2179906807 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f889c162040 0x7f889c0d1da0 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7f888c0097a0 tx=0x7f888c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5fed4806269a475 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:18.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:18.478+0000 7f889affd640 1 -- 172.21.15.27:0/2179906807 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f888c037070 con 0x7f889c162040 2024-09-12T03:23:18.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:18.478+0000 7f889affd640 1 -- 172.21.15.27:0/2179906807 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f888c032440 con 0x7f889c162040 2024-09-12T03:23:18.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:18.478+0000 7f889affd640 1 -- 172.21.15.27:0/2179906807 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f888c032810 con 0x7f889c162040 2024-09-12T03:23:18.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:18.479+0000 7f88a2474640 1 -- 172.21.15.27:0/2179906807 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f889c162040 msgr2=0x7f889c0d1da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:18.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:18.479+0000 7f88a2474640 1 --2- 172.21.15.27:0/2179906807 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f889c162040 0x7f889c0d1da0 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7f888c0097a0 tx=0x7f888c02a180 comp rx=0 tx=0).stop 2024-09-12T03:23:18.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:18.479+0000 7f88a2474640 1 -- 172.21.15.27:0/2179906807 shutdown_connections 2024-09-12T03:23:18.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:18.479+0000 7f88a2474640 1 --2- 172.21.15.27:0/2179906807 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f889c0d3790 0x7f889c0b7a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:18.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:18.479+0000 7f88a2474640 1 --2- 172.21.15.27:0/2179906807 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f889c162040 0x7f889c0d1da0 unknown :-1 s=CLOSED pgs=1521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:18.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:18.479+0000 7f88a2474640 1 -- 172.21.15.27:0/2179906807 >> 172.21.15.27:0/2179906807 conn(0x7f889c0b6700 msgr2=0x7f889c0b6b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:18.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:18.479+0000 7f88a2474640 1 -- 172.21.15.27:0/2179906807 shutdown_connections 2024-09-12T03:23:18.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:18.479+0000 7f88a2474640 1 -- 172.21.15.27:0/2179906807 wait complete. 2024-09-12T03:23:18.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:18.480+0000 7f88a2474640 1 Processor -- start 2024-09-12T03:23:18.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:18.480+0000 7f88a2474640 1 -- start start 2024-09-12T03:23:18.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:18.481+0000 7f88a2474640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f889c06d8c0 0x7f889c0ae430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:18.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:18.481+0000 7f88a2474640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f889c0d3790 0x7f889c0ab7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:18.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:18.481+0000 7f88a2474640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f889c162040 0x7f889c0b01b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:18.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:18.481+0000 7f88a2474640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f889c0d4530 con 0x7f889c06d8c0 2024-09-12T03:23:18.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:18.481+0000 7f88a2474640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f889c0cee00 con 0x7f889c0d3790 2024-09-12T03:23:18.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:18.481+0000 7f88a2474640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f889c0d4150 con 0x7f889c162040 2024-09-12T03:23:18.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:18.481+0000 7f889bfff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f889c06d8c0 0x7f889c0ae430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:18.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:18.481+0000 7f889bfff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f889c06d8c0 0x7f889c0ae430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59320/0 (socket says 172.21.15.27:59320) 2024-09-12T03:23:18.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:18.481+0000 7f889bfff640 1 -- 172.21.15.27:0/3752020019 learned_addr learned my addr 172.21.15.27:0/3752020019 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:23:18.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:18.481+0000 7f889b7fe640 1 --2- 172.21.15.27:0/3752020019 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f889c0d3790 0x7f889c0ab7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:18.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:18.481+0000 7f88a0a11640 1 --2- 172.21.15.27:0/3752020019 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f889c162040 0x7f889c0b01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:18.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:18.482+0000 7f889bfff640 1 -- 172.21.15.27:0/3752020019 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f889c162040 msgr2=0x7f889c0b01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:18.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:18.482+0000 7f889bfff640 1 --2- 172.21.15.27:0/3752020019 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f889c162040 0x7f889c0b01b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:18.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:18.482+0000 7f889bfff640 1 -- 172.21.15.27:0/3752020019 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f889c0d3790 msgr2=0x7f889c0ab7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:18.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:18.482+0000 7f889bfff640 1 --2- 172.21.15.27:0/3752020019 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f889c0d3790 0x7f889c0ab7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:18.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:18.482+0000 7f889bfff640 1 -- 172.21.15.27:0/3752020019 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f889c0b0e20 con 0x7f889c06d8c0 2024-09-12T03:23:18.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:18.482+0000 7f889bfff640 1 --2- 172.21.15.27:0/3752020019 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f889c06d8c0 0x7f889c0ae430 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7f888c0097a0 tx=0x7f888c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:18.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:18.482+0000 7f8898ff9640 1 -- 172.21.15.27:0/3752020019 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f888c041070 con 0x7f889c06d8c0 2024-09-12T03:23:18.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:18.482+0000 7f8898ff9640 1 -- 172.21.15.27:0/3752020019 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f888c03c870 con 0x7f889c06d8c0 2024-09-12T03:23:18.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:18.482+0000 7f88a2474640 1 -- 172.21.15.27:0/3752020019 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f889c0b1100 con 0x7f889c06d8c0 2024-09-12T03:23:18.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:18.482+0000 7f8898ff9640 1 -- 172.21.15.27:0/3752020019 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f888c037040 con 0x7f889c06d8c0 2024-09-12T03:23:18.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:18.482+0000 7f88a2474640 1 -- 172.21.15.27:0/3752020019 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f889c15d450 con 0x7f889c06d8c0 2024-09-12T03:23:18.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:18.483+0000 7f88a2474640 1 -- 172.21.15.27:0/3752020019 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f889c0b14f0 con 0x7f889c06d8c0 2024-09-12T03:23:18.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:18.484+0000 7f8898ff9640 1 -- 172.21.15.27:0/3752020019 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 24) ==== 91212+0+0 (secure 0 0 0) 0x7f888c044070 con 0x7f889c06d8c0 2024-09-12T03:23:18.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:18.484+0000 7f8898ff9640 1 --2- 172.21.15.27:0/3752020019 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f887c06c610 0x7f887c06ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:18.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:18.484+0000 7f8898ff9640 1 -- 172.21.15.27:0/3752020019 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 4913+0+0 (secure 0 0 0) 0x7f888c0aa530 con 0x7f889c06d8c0 2024-09-12T03:23:18.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:18.484+0000 7f889b7fe640 1 --2- 172.21.15.27:0/3752020019 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f887c06c610 0x7f887c06ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:18.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:18.485+0000 7f889b7fe640 1 --2- 172.21.15.27:0/3752020019 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f887c06c610 0x7f887c06ead0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f889c160bc0 tx=0x7f8884008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:18.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:18.488+0000 7f8898ff9640 1 -- 172.21.15.27:0/3752020019 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f888c07a8f0 con 0x7f889c06d8c0 2024-09-12T03:23:18.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:18.667+0000 7f88a2474640 1 -- 172.21.15.27:0/3752020019 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f889c0b12d0 con 0x7f889c06d8c0 2024-09-12T03:23:19.665 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:19.663+0000 7f8898ff9640 1 -- 172.21.15.27:0/3752020019 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v146) ==== 138+0+0 (secure 0 0 0) 0x7f888c07a1f0 con 0x7f889c06d8c0 2024-09-12T03:23:19.665 INFO:teuthology.orchestra.run.smithi027.stderr:pool 'cephfs_metadata' created 2024-09-12T03:23:19.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:19.667+0000 7f88a2474640 1 -- 172.21.15.27:0/3752020019 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f887c06c610 msgr2=0x7f887c06ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:19.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:19.667+0000 7f88a2474640 1 --2- 172.21.15.27:0/3752020019 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f887c06c610 0x7f887c06ead0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f889c160bc0 tx=0x7f8884008040 comp rx=0 tx=0).stop 2024-09-12T03:23:19.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:19.667+0000 7f88a2474640 1 -- 172.21.15.27:0/3752020019 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f889c06d8c0 msgr2=0x7f889c0ae430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:19.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:19.667+0000 7f88a2474640 1 --2- 172.21.15.27:0/3752020019 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f889c06d8c0 0x7f889c0ae430 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7f888c0097a0 tx=0x7f888c0046c0 comp rx=0 tx=0).stop 2024-09-12T03:23:19.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:19.667+0000 7f88a2474640 1 -- 172.21.15.27:0/3752020019 shutdown_connections 2024-09-12T03:23:19.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:19.667+0000 7f88a2474640 1 --2- 172.21.15.27:0/3752020019 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f889c162040 0x7f889c0b01b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:19.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:19.667+0000 7f88a2474640 1 --2- 172.21.15.27:0/3752020019 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f889c0d3790 0x7f889c0ab7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:19.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:19.667+0000 7f88a2474640 1 --2- 172.21.15.27:0/3752020019 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f887c06c610 0x7f887c06ead0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:19.669 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:19.667+0000 7f88a2474640 1 --2- 172.21.15.27:0/3752020019 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f889c06d8c0 0x7f889c0ae430 unknown :-1 s=CLOSED pgs=1522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:19.669 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:19.667+0000 7f88a2474640 1 -- 172.21.15.27:0/3752020019 >> 172.21.15.27:0/3752020019 conn(0x7f889c0b6700 msgr2=0x7f889c0b8f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:19.669 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:19.667+0000 7f88a2474640 1 -- 172.21.15.27:0/3752020019 shutdown_connections 2024-09-12T03:23:19.669 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:19.667+0000 7f88a2474640 1 -- 172.21.15.27:0/3752020019 wait complete. 2024-09-12T03:23:19.686 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-12T03:23:19.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:19.801+0000 7f2261b86640 1 Processor -- start 2024-09-12T03:23:19.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:19.801+0000 7f2261b86640 1 -- start start 2024-09-12T03:23:19.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:19.802+0000 7f2261b86640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f225c06cc40 0x7f225c06d060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:19.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:19.802+0000 7f2261b86640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f225c06db30 0x7f225c06df80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:19.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:19.802+0000 7f2261b86640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f225c06d5a0 con 0x7f225c15e650 2024-09-12T03:23:19.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:19.802+0000 7f2261b86640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f225c06d850 con 0x7f225c06db30 2024-09-12T03:23:19.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:19.802+0000 7f2261b86640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f225c0d1260 con 0x7f225c06cc40 2024-09-12T03:23:19.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:19.802+0000 7f225b7fe640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f225c15e650 0x7f225c0d1d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:56052/0 (socket says 172.21.15.27:56052) 2024-09-12T03:23:19.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:19.802+0000 7f225b7fe640 1 -- 172.21.15.27:0/1520258038 learned_addr learned my addr 172.21.15.27:0/1520258038 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:23:19.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:19.802+0000 7f225a7fc640 1 -- 172.21.15.27:0/1520258038 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3592982863 0 0) 0x7f225c06d5a0 con 0x7f225c15e650 2024-09-12T03:23:19.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:19.802+0000 7f225a7fc640 1 -- 172.21.15.27:0/1520258038 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2244003520 con 0x7f225c15e650 2024-09-12T03:23:19.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:19.802+0000 7f225bfff640 1 --2- 172.21.15.27:0/1520258038 >> v2:172.21.15.45:3300/0 conn(0x7f225c06db30 0x7f225c06df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:19.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:19.802+0000 7f225affd640 1 --2- 172.21.15.27:0/1520258038 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f225c06cc40 0x7f225c06d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:19.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:19.802+0000 7f225a7fc640 1 -- 172.21.15.27:0/1520258038 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 370166438 0 0) 0x7f2244003520 con 0x7f225c15e650 2024-09-12T03:23:19.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:19.802+0000 7f225a7fc640 1 -- 172.21.15.27:0/1520258038 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f225c06cc40 msgr2=0x7f225c06d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:19.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:19.803+0000 7f225a7fc640 1 --2- 172.21.15.27:0/1520258038 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f225c06cc40 0x7f225c06d060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:19.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:19.803+0000 7f225a7fc640 1 -- 172.21.15.27:0/1520258038 >> v2:172.21.15.45:3300/0 conn(0x7f225c06db30 msgr2=0x7f225c06df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:19.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:19.803+0000 7f225a7fc640 1 --2- 172.21.15.27:0/1520258038 >> v2:172.21.15.45:3300/0 conn(0x7f225c06db30 0x7f225c06df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:19.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:19.803+0000 7f225a7fc640 1 -- 172.21.15.27:0/1520258038 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f225c0d1540 con 0x7f225c15e650 2024-09-12T03:23:19.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:19.803+0000 7f225a7fc640 1 -- 172.21.15.27:0/1520258038 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 847755213 0 0) 0x7f224c001670 con 0x7f225c15e650 2024-09-12T03:23:19.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:19.803+0000 7f225a7fc640 1 -- 172.21.15.27:0/1520258038 >> v1:172.21.15.27:6789/0 conn(0x7f225c15e650 legacy=0x7f225c0d1d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:19.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:19.803+0000 7f225a7fc640 1 --2- 172.21.15.27:0/1520258038 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f22440039b0 0x7f2244003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:19.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:19.803+0000 7f225a7fc640 1 --2- 172.21.15.27:0/1520258038 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f225c06cc40 0x7f2244004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:19.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:19.803+0000 7f225a7fc640 1 --2- 172.21.15.27:0/1520258038 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2244004d40 0x7f22440051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:19.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:19.803+0000 7f225a7fc640 1 -- 172.21.15.27:0/1520258038 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f225c06d850 con 0x7f22440039b0 2024-09-12T03:23:19.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:19.803+0000 7f225a7fc640 1 -- 172.21.15.27:0/1520258038 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f225c0d1260 con 0x7f2244004d40 2024-09-12T03:23:19.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:19.803+0000 7f225a7fc640 1 -- 172.21.15.27:0/1520258038 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f224400bb50 con 0x7f225c06cc40 2024-09-12T03:23:19.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:19.803+0000 7f225b7fe640 1 --2- 172.21.15.27:0/1520258038 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f22440039b0 0x7f2244003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:19.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:19.803+0000 7f225affd640 1 --2- 172.21.15.27:0/1520258038 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f225c06cc40 0x7f2244004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:19.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:19.803+0000 7f225bfff640 1 --2- 172.21.15.27:0/1520258038 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2244004d40 0x7f22440051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:19.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:19.803+0000 7f225b7fe640 1 -- 172.21.15.27:0/1520258038 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f225c06cc40 msgr2=0x7f2244004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:19.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:19.803+0000 7f225b7fe640 1 --2- 172.21.15.27:0/1520258038 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f225c06cc40 0x7f2244004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:19.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:19.803+0000 7f225b7fe640 1 -- 172.21.15.27:0/1520258038 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2244004d40 msgr2=0x7f22440051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:19.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:19.803+0000 7f225b7fe640 1 --2- 172.21.15.27:0/1520258038 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2244004d40 0x7f22440051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:19.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:19.803+0000 7f225b7fe640 1 -- 172.21.15.27:0/1520258038 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2244003520 con 0x7f22440039b0 2024-09-12T03:23:19.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:19.803+0000 7f225affd640 1 --2- 172.21.15.27:0/1520258038 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f225c06cc40 0x7f2244004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:23:19.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:19.804+0000 7f225b7fe640 1 --2- 172.21.15.27:0/1520258038 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f22440039b0 0x7f2244003dd0 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7f224c00c290 tx=0x7f2244005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=359eaaf9b15c1491 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:19.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:19.804+0000 7f225a7fc640 1 -- 172.21.15.27:0/1520258038 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f224c007d50 con 0x7f22440039b0 2024-09-12T03:23:19.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:19.804+0000 7f225a7fc640 1 -- 172.21.15.27:0/1520258038 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f224c005af0 con 0x7f22440039b0 2024-09-12T03:23:19.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:19.804+0000 7f225a7fc640 1 -- 172.21.15.27:0/1520258038 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f224c007070 con 0x7f22440039b0 2024-09-12T03:23:19.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:19.804+0000 7f2261b86640 1 -- 172.21.15.27:0/1520258038 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f22440039b0 msgr2=0x7f2244003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:19.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:19.804+0000 7f2261b86640 1 --2- 172.21.15.27:0/1520258038 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f22440039b0 0x7f2244003dd0 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7f224c00c290 tx=0x7f2244005830 comp rx=0 tx=0).stop 2024-09-12T03:23:19.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:19.804+0000 7f225bfff640 1 -- 172.21.15.27:0/1520258038 reap_dead start 2024-09-12T03:23:19.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:19.804+0000 7f2261b86640 1 -- 172.21.15.27:0/1520258038 shutdown_connections 2024-09-12T03:23:19.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:19.805+0000 7f2261b86640 1 -- 172.21.15.27:0/1520258038 >> 172.21.15.27:0/1520258038 conn(0x7f225c06f320 msgr2=0x7f225c06f700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:19.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:19.805+0000 7f2261b86640 1 -- 172.21.15.27:0/1520258038 shutdown_connections 2024-09-12T03:23:19.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:19.805+0000 7f2261b86640 1 -- 172.21.15.27:0/1520258038 wait complete. 2024-09-12T03:23:19.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:19.806+0000 7f2261b86640 1 Processor -- start 2024-09-12T03:23:19.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:19.806+0000 7f2261b86640 1 -- start start 2024-09-12T03:23:19.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:19.807+0000 7f2261b86640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f22440039b0 0x7f225c158f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:19.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:19.807+0000 7f2261b86640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f225c14b190 0x7f225c155250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:19.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:19.807+0000 7f2261b86640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f225c155790 0x7f225c1f4590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:19.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:19.807+0000 7f2261b86640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f225c06e640 con 0x7f225c14b190 2024-09-12T03:23:19.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:19.807+0000 7f2261b86640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f225c06e4c0 con 0x7f22440039b0 2024-09-12T03:23:19.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:19.807+0000 7f2261b86640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f225c06d060 con 0x7f225c155790 2024-09-12T03:23:19.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:19.807+0000 7f225affd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f225c14b190 0x7f225c155250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:19.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:19.807+0000 7f225affd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f225c14b190 0x7f225c155250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42336/0 (socket says 172.21.15.27:42336) 2024-09-12T03:23:19.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:19.807+0000 7f225affd640 1 -- 172.21.15.27:0/3436324434 learned_addr learned my addr 172.21.15.27:0/3436324434 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:23:19.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:19.807+0000 7f225b7fe640 1 --2- 172.21.15.27:0/3436324434 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f22440039b0 0x7f225c158f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:19.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:19.807+0000 7f225bfff640 1 --2- 172.21.15.27:0/3436324434 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f225c155790 0x7f225c1f4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:19.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:19.807+0000 7f225affd640 1 -- 172.21.15.27:0/3436324434 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f225c155790 msgr2=0x7f225c1f4590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:19.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:19.807+0000 7f225affd640 1 --2- 172.21.15.27:0/3436324434 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f225c155790 0x7f225c1f4590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:19.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:19.807+0000 7f225affd640 1 -- 172.21.15.27:0/3436324434 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f22440039b0 msgr2=0x7f225c158f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:19.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:19.807+0000 7f225affd640 1 --2- 172.21.15.27:0/3436324434 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f22440039b0 0x7f225c158f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:19.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:19.807+0000 7f225affd640 1 -- 172.21.15.27:0/3436324434 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f225c1594c0 con 0x7f225c14b190 2024-09-12T03:23:19.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:19.808+0000 7f225affd640 1 --2- 172.21.15.27:0/3436324434 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f225c14b190 0x7f225c155250 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7f2250006a90 tx=0x7f2250004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:19.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:19.808+0000 7f2243fff640 1 -- 172.21.15.27:0/3436324434 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2250019020 con 0x7f225c14b190 2024-09-12T03:23:19.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:19.808+0000 7f2243fff640 1 -- 172.21.15.27:0/3436324434 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2250014870 con 0x7f225c14b190 2024-09-12T03:23:19.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:19.808+0000 7f2261b86640 1 -- 172.21.15.27:0/3436324434 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f225c15ac00 con 0x7f225c14b190 2024-09-12T03:23:19.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:19.808+0000 7f2243fff640 1 -- 172.21.15.27:0/3436324434 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2250014c40 con 0x7f225c14b190 2024-09-12T03:23:19.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:19.808+0000 7f2261b86640 1 -- 172.21.15.27:0/3436324434 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f225c160bd0 con 0x7f225c14b190 2024-09-12T03:23:19.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:19.809+0000 7f2261b86640 1 -- 172.21.15.27:0/3436324434 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f225c06d4a0 con 0x7f225c14b190 2024-09-12T03:23:19.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:19.809+0000 7f2243fff640 1 -- 172.21.15.27:0/3436324434 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 24) ==== 91212+0+0 (secure 0 0 0) 0x7f225001c020 con 0x7f225c14b190 2024-09-12T03:23:19.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:19.810+0000 7f2243fff640 1 --2- 172.21.15.27:0/3436324434 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f224806c640 0x7f224806eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:19.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:19.810+0000 7f225b7fe640 1 --2- 172.21.15.27:0/3436324434 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f224806c640 0x7f224806eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:19.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:19.810+0000 7f2243fff640 1 -- 172.21.15.27:0/3436324434 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 5283+0+0 (secure 0 0 0) 0x7f2250089be0 con 0x7f225c14b190 2024-09-12T03:23:19.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:19.811+0000 7f225b7fe640 1 --2- 172.21.15.27:0/3436324434 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f224806c640 0x7f224806eb00 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f224c033000 tx=0x7f224c009030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:19.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:19.815+0000 7f2243fff640 1 -- 172.21.15.27:0/3436324434 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f2250023030 con 0x7f225c14b190 2024-09-12T03:23:19.991 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:19.990+0000 7f2261b86640 1 -- 172.21.15.27:0/3436324434 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f225c155b70 con 0x7f225c14b190 2024-09-12T03:23:20.667 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:20.666+0000 7f2243fff640 1 -- 172.21.15.27:0/3436324434 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v147) ==== 170+0+0 (secure 0 0 0) 0x7f225008c120 con 0x7f225c14b190 2024-09-12T03:23:20.667 INFO:teuthology.orchestra.run.smithi027.stderr:pool 'cephfs_data' created 2024-09-12T03:23:20.670 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:20.669+0000 7f2261b86640 1 -- 172.21.15.27:0/3436324434 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f224806c640 msgr2=0x7f224806eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:20.670 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:20.669+0000 7f2261b86640 1 --2- 172.21.15.27:0/3436324434 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f224806c640 0x7f224806eb00 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f224c033000 tx=0x7f224c009030 comp rx=0 tx=0).stop 2024-09-12T03:23:20.670 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:20.669+0000 7f2261b86640 1 -- 172.21.15.27:0/3436324434 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f225c14b190 msgr2=0x7f225c155250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:20.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:20.669+0000 7f2261b86640 1 --2- 172.21.15.27:0/3436324434 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f225c14b190 0x7f225c155250 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7f2250006a90 tx=0x7f2250004610 comp rx=0 tx=0).stop 2024-09-12T03:23:20.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:20.669+0000 7f2261b86640 1 -- 172.21.15.27:0/3436324434 shutdown_connections 2024-09-12T03:23:20.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:20.669+0000 7f2261b86640 1 --2- 172.21.15.27:0/3436324434 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f225c155790 0x7f225c1f4590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:20.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:20.669+0000 7f2261b86640 1 --2- 172.21.15.27:0/3436324434 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f224806c640 0x7f224806eb00 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:20.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:20.669+0000 7f2261b86640 1 --2- 172.21.15.27:0/3436324434 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f225c14b190 0x7f225c155250 unknown :-1 s=CLOSED pgs=1525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:20.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:20.669+0000 7f2261b86640 1 --2- 172.21.15.27:0/3436324434 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f22440039b0 0x7f225c158f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:20.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:20.669+0000 7f2261b86640 1 -- 172.21.15.27:0/3436324434 >> 172.21.15.27:0/3436324434 conn(0x7f225c06f320 msgr2=0x7f225c0b69f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:20.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:20.669+0000 7f2261b86640 1 -- 172.21.15.27:0/3436324434 shutdown_connections 2024-09-12T03:23:20.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:20.669+0000 7f2261b86640 1 -- 172.21.15.27:0/3436324434 wait complete. 2024-09-12T03:23:20.695 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-12T03:23:20.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:20.805+0000 7f555d9ff640 1 Processor -- start 2024-09-12T03:23:20.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:20.806+0000 7f555d9ff640 1 -- start start 2024-09-12T03:23:20.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:20.806+0000 7f555d9ff640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f555815e8f0 0x7f555806ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:20.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:20.806+0000 7f555d9ff640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f55580cf6b0 0x7f55580d3b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:20.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:20.806+0000 7f555d9ff640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f55580d2ae0 con 0x7f55580d40c0 2024-09-12T03:23:20.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:20.806+0000 7f555d9ff640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f55580d36f0 con 0x7f55580cf6b0 2024-09-12T03:23:20.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:20.806+0000 7f555d9ff640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f55580d2dc0 con 0x7f555815e8f0 2024-09-12T03:23:20.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:20.806+0000 7f55577fe640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f55580d40c0 0x7f55580b6160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:56064/0 (socket says 172.21.15.27:56064) 2024-09-12T03:23:20.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:20.807+0000 7f55577fe640 1 -- 172.21.15.27:0/2089620543 learned_addr learned my addr 172.21.15.27:0/2089620543 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:23:20.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:20.807+0000 7f5556ffd640 1 --2- 172.21.15.27:0/2089620543 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f555815e8f0 0x7f555806ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:20.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:20.807+0000 7f55567fc640 1 --2- 172.21.15.27:0/2089620543 >> v2:172.21.15.45:3300/0 conn(0x7f55580cf6b0 0x7f55580d3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:20.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:20.807+0000 7f5555ffb640 1 -- 172.21.15.27:0/2089620543 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1469728513 0 0) 0x7f55580d2ae0 con 0x7f55580d40c0 2024-09-12T03:23:20.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:20.807+0000 7f5555ffb640 1 -- 172.21.15.27:0/2089620543 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5544003520 con 0x7f55580d40c0 2024-09-12T03:23:20.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:20.807+0000 7f55567fc640 1 -- 172.21.15.27:0/2089620543 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f555815e8f0 msgr2=0x7f555806ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:20.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:20.807+0000 7f55567fc640 1 --2- 172.21.15.27:0/2089620543 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f555815e8f0 0x7f555806ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:20.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:20.807+0000 7f55567fc640 1 -- 172.21.15.27:0/2089620543 >> v1:172.21.15.27:6789/0 conn(0x7f55580d40c0 legacy=0x7f55580b6160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:20.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:20.807+0000 7f55567fc640 1 -- 172.21.15.27:0/2089620543 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55580d30a0 con 0x7f55580cf6b0 2024-09-12T03:23:20.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:20.807+0000 7f5556ffd640 1 --2- 172.21.15.27:0/2089620543 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f555815e8f0 0x7f555806ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:23:20.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:20.808+0000 7f55567fc640 1 --2- 172.21.15.27:0/2089620543 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f55580cf6b0 0x7f55580d3b80 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f55400097a0 tx=0x7f554002a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=496660739af44dc6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:20.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:20.808+0000 7f5555ffb640 1 -- 172.21.15.27:0/2089620543 <== mon.1 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5540037070 con 0x7f55580cf6b0 2024-09-12T03:23:20.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:20.808+0000 7f5555ffb640 1 -- 172.21.15.27:0/2089620543 <== mon.1 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f554002ae50 con 0x7f55580cf6b0 2024-09-12T03:23:20.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:20.808+0000 7f5555ffb640 1 -- 172.21.15.27:0/2089620543 <== mon.1 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5540032640 con 0x7f55580cf6b0 2024-09-12T03:23:20.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:20.809+0000 7f555d9ff640 1 -- 172.21.15.27:0/2089620543 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f55580cf6b0 msgr2=0x7f55580d3b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:20.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:20.809+0000 7f555d9ff640 1 --2- 172.21.15.27:0/2089620543 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f55580cf6b0 0x7f55580d3b80 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f55400097a0 tx=0x7f554002a100 comp rx=0 tx=0).stop 2024-09-12T03:23:20.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:20.809+0000 7f555d9ff640 1 -- 172.21.15.27:0/2089620543 shutdown_connections 2024-09-12T03:23:20.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:20.809+0000 7f555d9ff640 1 --2- 172.21.15.27:0/2089620543 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f55580cf6b0 0x7f55580d3b80 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:20.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:20.809+0000 7f555d9ff640 1 --2- 172.21.15.27:0/2089620543 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f555815e8f0 0x7f555806ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:20.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:20.809+0000 7f555d9ff640 1 -- 172.21.15.27:0/2089620543 >> 172.21.15.27:0/2089620543 conn(0x7f55580d1160 msgr2=0x7f55580d1540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:20.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:20.809+0000 7f555d9ff640 1 -- 172.21.15.27:0/2089620543 shutdown_connections 2024-09-12T03:23:20.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:20.809+0000 7f555d9ff640 1 -- 172.21.15.27:0/2089620543 wait complete. 2024-09-12T03:23:20.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:20.810+0000 7f555d9ff640 1 Processor -- start 2024-09-12T03:23:20.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:20.810+0000 7f555d9ff640 1 -- start start 2024-09-12T03:23:20.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:20.811+0000 7f555d9ff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f55580cf6b0 0x7f555814a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:20.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:20.811+0000 7f555d9ff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f55580d40c0 0x7f55580cd980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:20.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:20.811+0000 7f555d9ff640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f555815e8f0 0x7f55580cdec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:20.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:20.811+0000 7f555d9ff640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f55580b6870 con 0x7f55580cf6b0 2024-09-12T03:23:20.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:20.811+0000 7f555d9ff640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f555806cf50 con 0x7f55580d40c0 2024-09-12T03:23:20.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:20.811+0000 7f555d9ff640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f55580b69f0 con 0x7f555815e8f0 2024-09-12T03:23:20.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:20.811+0000 7f5556ffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f55580cf6b0 0x7f555814a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:20.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:20.811+0000 7f5556ffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f55580cf6b0 0x7f555814a540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42340/0 (socket says 172.21.15.27:42340) 2024-09-12T03:23:20.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:20.811+0000 7f5556ffd640 1 -- 172.21.15.27:0/793791632 learned_addr learned my addr 172.21.15.27:0/793791632 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:23:20.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:20.811+0000 7f55567fc640 1 --2- 172.21.15.27:0/793791632 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f55580d40c0 0x7f55580cd980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:20.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:20.811+0000 7f55577fe640 1 --2- 172.21.15.27:0/793791632 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f555815e8f0 0x7f55580cdec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:20.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:20.811+0000 7f5556ffd640 1 -- 172.21.15.27:0/793791632 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f555815e8f0 msgr2=0x7f55580cdec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:20.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:20.811+0000 7f5556ffd640 1 --2- 172.21.15.27:0/793791632 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f555815e8f0 0x7f55580cdec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:20.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:20.811+0000 7f5556ffd640 1 -- 172.21.15.27:0/793791632 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f55580d40c0 msgr2=0x7f55580cd980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:20.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:20.811+0000 7f5556ffd640 1 --2- 172.21.15.27:0/793791632 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f55580d40c0 0x7f55580cd980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:20.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:20.811+0000 7f5556ffd640 1 -- 172.21.15.27:0/793791632 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5558161690 con 0x7f55580cf6b0 2024-09-12T03:23:20.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:20.811+0000 7f5556ffd640 1 --2- 172.21.15.27:0/793791632 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f55580cf6b0 0x7f555814a540 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7f554800ec10 tx=0x7f554800c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:20.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:20.812+0000 7f554ffff640 1 -- 172.21.15.27:0/793791632 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5548017070 con 0x7f55580cf6b0 2024-09-12T03:23:20.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:20.812+0000 7f554ffff640 1 -- 172.21.15.27:0/793791632 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f554800cc30 con 0x7f55580cf6b0 2024-09-12T03:23:20.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:20.812+0000 7f555d9ff640 1 -- 172.21.15.27:0/793791632 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5558161970 con 0x7f55580cf6b0 2024-09-12T03:23:20.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:20.812+0000 7f554ffff640 1 -- 172.21.15.27:0/793791632 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5548013470 con 0x7f55580cf6b0 2024-09-12T03:23:20.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:20.812+0000 7f555d9ff640 1 -- 172.21.15.27:0/793791632 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5558161f30 con 0x7f55580cf6b0 2024-09-12T03:23:20.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:20.813+0000 7f555d9ff640 1 -- 172.21.15.27:0/793791632 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55580d2ae0 con 0x7f55580cf6b0 2024-09-12T03:23:20.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:20.813+0000 7f554ffff640 1 -- 172.21.15.27:0/793791632 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 24) ==== 91212+0+0 (secure 0 0 0) 0x7f554801a070 con 0x7f55580cf6b0 2024-09-12T03:23:20.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:20.814+0000 7f554ffff640 1 --2- 172.21.15.27:0/793791632 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f553806c610 0x7f553806ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:20.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:20.814+0000 7f554ffff640 1 -- 172.21.15.27:0/793791632 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(147..147 src has 1..147) ==== 5665+0+0 (secure 0 0 0) 0x7f5548089bd0 con 0x7f55580cf6b0 2024-09-12T03:23:20.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:20.814+0000 7f55567fc640 1 --2- 172.21.15.27:0/793791632 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f553806c610 0x7f553806ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:20.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:20.814+0000 7f55567fc640 1 --2- 172.21.15.27:0/793791632 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f553806c610 0x7f553806ead0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f5540034000 tx=0x7f554002e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:20.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:20.819+0000 7f554ffff640 1 -- 172.21.15.27:0/793791632 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f5548059df0 con 0x7f55580cf6b0 2024-09-12T03:23:21.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:21.029+0000 7f555d9ff640 1 -- 172.21.15.27:0/793791632 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f555815d090 con 0x7f55580cf6b0 2024-09-12T03:23:21.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:21.689+0000 7f554ffff640 1 -- 172.21.15.27:0/793791632 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 24 and data pool 25 v88) ==== 174+0+0 (secure 0 0 0) 0x7f55480596f0 con 0x7f55580cf6b0 2024-09-12T03:23:21.696 INFO:teuthology.orchestra.run.smithi027.stderr:new fs with metadata pool 24 and data pool 25 2024-09-12T03:23:21.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:21.707+0000 7f555d9ff640 1 -- 172.21.15.27:0/793791632 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f553806c610 msgr2=0x7f553806ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:21.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:21.707+0000 7f555d9ff640 1 --2- 172.21.15.27:0/793791632 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f553806c610 0x7f553806ead0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f5540034000 tx=0x7f554002e000 comp rx=0 tx=0).stop 2024-09-12T03:23:21.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:21.708+0000 7f555d9ff640 1 -- 172.21.15.27:0/793791632 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f55580cf6b0 msgr2=0x7f555814a540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:21.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:21.708+0000 7f555d9ff640 1 --2- 172.21.15.27:0/793791632 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f55580cf6b0 0x7f555814a540 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7f554800ec10 tx=0x7f554800c520 comp rx=0 tx=0).stop 2024-09-12T03:23:21.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:21.708+0000 7f555d9ff640 1 -- 172.21.15.27:0/793791632 shutdown_connections 2024-09-12T03:23:21.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:21.708+0000 7f555d9ff640 1 --2- 172.21.15.27:0/793791632 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f555815e8f0 0x7f55580cdec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:21.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:21.708+0000 7f555d9ff640 1 --2- 172.21.15.27:0/793791632 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f55580d40c0 0x7f55580cd980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:21.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:21.708+0000 7f555d9ff640 1 --2- 172.21.15.27:0/793791632 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f553806c610 0x7f553806ead0 secure :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f5540034000 tx=0x7f554002e000 comp rx=0 tx=0).stop 2024-09-12T03:23:21.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:21.708+0000 7f555d9ff640 1 --2- 172.21.15.27:0/793791632 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f55580cf6b0 0x7f555814a540 unknown :-1 s=CLOSED pgs=1527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:21.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:21.708+0000 7f555d9ff640 1 -- 172.21.15.27:0/793791632 >> 172.21.15.27:0/793791632 conn(0x7f55580d1160 msgr2=0x7f55580ced10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:21.710 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:21.710+0000 7f555d9ff640 1 -- 172.21.15.27:0/793791632 shutdown_connections 2024-09-12T03:23:21.712 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:21.711+0000 7f555d9ff640 1 -- 172.21.15.27:0/793791632 wait complete. 2024-09-12T03:23:21.726 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-12T03:23:21.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:21.830+0000 7fa0e6549640 1 Processor -- start 2024-09-12T03:23:21.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:21.831+0000 7fa0e6549640 1 -- start start 2024-09-12T03:23:21.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:21.831+0000 7fa0e6549640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa0e00d5200 0x7fa0e00d1010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:21.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:21.831+0000 7fa0e6549640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa0e00d3760 con 0x7fa0e00d1550 2024-09-12T03:23:21.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:21.831+0000 7fa0e6549640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa0e00d3a40 con 0x7fa0e015eb90 2024-09-12T03:23:21.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:21.831+0000 7fa0e6549640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fa0e00b7d20 con 0x7fa0e00d5200 2024-09-12T03:23:21.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:21.831+0000 7fa0e4ae6640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fa0e00d1550 0x7fa0e00b8190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:56080/0 (socket says 172.21.15.27:56080) 2024-09-12T03:23:21.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:21.832+0000 7fa0e4ae6640 1 -- 172.21.15.27:0/3842445547 learned_addr learned my addr 172.21.15.27:0/3842445547 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:23:21.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:21.832+0000 7fa0df7fe640 1 --2- 172.21.15.27:0/3842445547 >> v2:172.21.15.45:3300/0 conn(0x7fa0e00d5200 0x7fa0e00d1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:21.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:21.832+0000 7fa0deffd640 1 -- 172.21.15.27:0/3842445547 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2222975659 0 0) 0x7fa0e00d3760 con 0x7fa0e00d1550 2024-09-12T03:23:21.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:21.832+0000 7fa0deffd640 1 -- 172.21.15.27:0/3842445547 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa0c8003520 con 0x7fa0e00d1550 2024-09-12T03:23:21.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:21.832+0000 7fa0df7fe640 1 -- 172.21.15.27:0/3842445547 >> v1:172.21.15.45:6789/0 conn(0x7fa0e015eb90 legacy=0x7fa0e00d2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:21.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:21.832+0000 7fa0df7fe640 1 -- 172.21.15.27:0/3842445547 >> v1:172.21.15.27:6789/0 conn(0x7fa0e00d1550 legacy=0x7fa0e00b8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:21.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:21.832+0000 7fa0df7fe640 1 -- 172.21.15.27:0/3842445547 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0e00d3d20 con 0x7fa0e00d5200 2024-09-12T03:23:21.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:21.833+0000 7fa0df7fe640 1 --2- 172.21.15.27:0/3842445547 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa0e00d5200 0x7fa0e00d1010 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fa0d40097a0 tx=0x7fa0d402a230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8c99a477c3950cce server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:21.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:21.833+0000 7fa0deffd640 1 -- 172.21.15.27:0/3842445547 <== mon.1 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0d4037070 con 0x7fa0e00d5200 2024-09-12T03:23:21.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:21.833+0000 7fa0deffd640 1 -- 172.21.15.27:0/3842445547 <== mon.1 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0d4032440 con 0x7fa0e00d5200 2024-09-12T03:23:21.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:21.833+0000 7fa0deffd640 1 -- 172.21.15.27:0/3842445547 <== mon.1 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0d40327f0 con 0x7fa0e00d5200 2024-09-12T03:23:21.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:21.833+0000 7fa0e6549640 1 -- 172.21.15.27:0/3842445547 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa0e00d5200 msgr2=0x7fa0e00d1010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:21.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:21.833+0000 7fa0e6549640 1 --2- 172.21.15.27:0/3842445547 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa0e00d5200 0x7fa0e00d1010 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fa0d40097a0 tx=0x7fa0d402a230 comp rx=0 tx=0).stop 2024-09-12T03:23:21.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:21.834+0000 7fa0e6549640 1 -- 172.21.15.27:0/3842445547 shutdown_connections 2024-09-12T03:23:21.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:21.834+0000 7fa0e6549640 1 --2- 172.21.15.27:0/3842445547 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa0e00d5200 0x7fa0e00d1010 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:21.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:21.834+0000 7fa0e6549640 1 -- 172.21.15.27:0/3842445547 >> 172.21.15.27:0/3842445547 conn(0x7fa0e006dce0 msgr2=0x7fa0e006e0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:21.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:21.834+0000 7fa0e6549640 1 -- 172.21.15.27:0/3842445547 shutdown_connections 2024-09-12T03:23:21.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:21.834+0000 7fa0e6549640 1 -- 172.21.15.27:0/3842445547 wait complete. 2024-09-12T03:23:21.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:21.835+0000 7fa0e6549640 1 Processor -- start 2024-09-12T03:23:21.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:21.835+0000 7fa0e6549640 1 -- start start 2024-09-12T03:23:21.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:21.836+0000 7fa0e6549640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa0e00d1550 0x7fa0e00cabd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:21.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:21.836+0000 7fa0e6549640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa0e00d5200 0x7fa0e01e7290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:21.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:21.836+0000 7fa0e6549640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa0e015eb90 0x7fa0e0159c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:21.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:21.836+0000 7fa0e6549640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fa0e00d3500 con 0x7fa0e015eb90 2024-09-12T03:23:21.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:21.836+0000 7fa0e6549640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa0e00d42e0 con 0x7fa0e00d1550 2024-09-12T03:23:21.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:21.836+0000 7fa0e6549640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fa0e00d1960 con 0x7fa0e00d5200 2024-09-12T03:23:21.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:21.836+0000 7fa0e4ae6640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa0e015eb90 0x7fa0e0159c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:21.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:21.836+0000 7fa0e4ae6640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa0e015eb90 0x7fa0e0159c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42344/0 (socket says 172.21.15.27:42344) 2024-09-12T03:23:21.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:21.836+0000 7fa0e4ae6640 1 -- 172.21.15.27:0/576746327 learned_addr learned my addr 172.21.15.27:0/576746327 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:23:21.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:21.836+0000 7fa0df7fe640 1 --2- 172.21.15.27:0/576746327 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa0e00d5200 0x7fa0e01e7290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:21.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:21.836+0000 7fa0dffff640 1 --2- 172.21.15.27:0/576746327 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa0e00d1550 0x7fa0e00cabd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:21.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:21.836+0000 7fa0e4ae6640 1 -- 172.21.15.27:0/576746327 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa0e00d5200 msgr2=0x7fa0e01e7290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:21.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:21.836+0000 7fa0e4ae6640 1 --2- 172.21.15.27:0/576746327 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa0e00d5200 0x7fa0e01e7290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:21.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:21.836+0000 7fa0e4ae6640 1 -- 172.21.15.27:0/576746327 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa0e00d1550 msgr2=0x7fa0e00cabd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:21.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:21.836+0000 7fa0e4ae6640 1 --2- 172.21.15.27:0/576746327 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa0e00d1550 0x7fa0e00cabd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:21.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:21.836+0000 7fa0e4ae6640 1 -- 172.21.15.27:0/576746327 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0e0146dc0 con 0x7fa0e015eb90 2024-09-12T03:23:21.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:21.836+0000 7fa0dffff640 1 --2- 172.21.15.27:0/576746327 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa0e00d1550 0x7fa0e00cabd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:23:21.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:21.836+0000 7fa0e4ae6640 1 --2- 172.21.15.27:0/576746327 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa0e015eb90 0x7fa0e0159c70 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7fa0d800ca20 tx=0x7fa0d800cee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:21.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:21.836+0000 7fa0dcff9640 1 -- 172.21.15.27:0/576746327 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0d8017070 con 0x7fa0e015eb90 2024-09-12T03:23:21.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:21.836+0000 7fa0dcff9640 1 -- 172.21.15.27:0/576746327 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0d800abd0 con 0x7fa0e015eb90 2024-09-12T03:23:21.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:21.836+0000 7fa0dcff9640 1 -- 172.21.15.27:0/576746327 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0d8012400 con 0x7fa0e015eb90 2024-09-12T03:23:21.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:21.836+0000 7fa0e6549640 1 -- 172.21.15.27:0/576746327 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0e01470a0 con 0x7fa0e015eb90 2024-09-12T03:23:21.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:21.837+0000 7fa0e6549640 1 -- 172.21.15.27:0/576746327 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0e0147660 con 0x7fa0e015eb90 2024-09-12T03:23:21.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:21.838+0000 7fa0e6549640 1 -- 172.21.15.27:0/576746327 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0e00693d0 con 0x7fa0e015eb90 2024-09-12T03:23:21.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:21.838+0000 7fa0dcff9640 1 -- 172.21.15.27:0/576746327 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 24) ==== 91212+0+0 (secure 0 0 0) 0x7fa0d801a070 con 0x7fa0e015eb90 2024-09-12T03:23:21.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:21.839+0000 7fa0dcff9640 1 --2- 172.21.15.27:0/576746327 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fa0cc06c610 0x7fa0cc06ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:21.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:21.839+0000 7fa0dcff9640 1 -- 172.21.15.27:0/576746327 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 5765+0+0 (secure 0 0 0) 0x7fa0d8089560 con 0x7fa0e015eb90 2024-09-12T03:23:21.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:21.839+0000 7fa0dffff640 1 --2- 172.21.15.27:0/576746327 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fa0cc06c610 0x7fa0cc06ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:21.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:21.839+0000 7fa0dffff640 1 --2- 172.21.15.27:0/576746327 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fa0cc06c610 0x7fa0cc06ead0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fa0e0149f50 tx=0x7fa0d0009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:21.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:21.843+0000 7fa0dcff9640 1 -- 172.21.15.27:0/576746327 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fa0d8059800 con 0x7fa0e015eb90 2024-09-12T03:23:22.019 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:22.018+0000 7fa0e6549640 1 -- 172.21.15.27:0/576746327 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa0e01602a0 con 0x7fa0e015eb90 2024-09-12T03:23:22.020 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:22.019+0000 7fa0dcff9640 1 -- 172.21.15.27:0/576746327 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v148) ==== 74+0+16793 (secure 0 0 0) 0x7fa0d8059100 con 0x7fa0e015eb90 2024-09-12T03:23:22.021 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:23:22.021 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":148,"fsid":"d2df504e-2647-4385-a5db-9fdf69f487a0","created":"2024-09-12T03:16:07.778937+0000","modified":"2024-09-12T03:23:21.031279+0000","last_up_change":"2024-09-12T03:16:19.314778+0000","last_in_change":"2024-09-12T03:16:09.893086+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T03:16:20.097646+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_metadata","create_time":"2024-09-12T03:23:18.699292+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"148","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_data","create_time":"2024-09-12T03:23:20.022169+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"148","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1851116c-1fb3-4dc9-9772-f90e2ef4c355","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6817","nonce":3002321488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6819","nonce":3002321488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6823","nonce":3002321488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6821","nonce":3002321488}]},"public_addr":"172.21.15.27:6817/3002321488","cluster_addr":"172.21.15.27:6819/3002321488","heartbeat_back_addr":"172.21.15.27:6823/3002321488","heartbeat_front_addr":"172.21.15.27:6821/3002321488","state":["exists","up"]},{"osd":1,"uuid":"f38ae17d-73eb-4204-b99d-975fa806fb87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6809","nonce":1613102886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6811","nonce":1613102886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6815","nonce":1613102886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6813","nonce":1613102886}]},"public_addr":"172.21.15.27:6809/1613102886","cluster_addr":"172.21.15.27:6811/1613102886","heartbeat_back_addr":"172.21.15.27:6815/1613102886","heartbeat_front_addr":"172.21.15.27:6813/1613102886","state":["exists","up"]},{"osd":2,"uuid":"fb12dcc8-bdd0-42b8-b46d-8b10106224bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6825","nonce":3534723355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6827","nonce":3534723355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6831","nonce":3534723355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6829","nonce":3534723355}]},"public_addr":"172.21.15.27:6825/3534723355","cluster_addr":"172.21.15.27:6827/3534723355","heartbeat_back_addr":"172.21.15.27:6831/3534723355","heartbeat_front_addr":"172.21.15.27:6829/3534723355","state":["exists","up"]},{"osd":3,"uuid":"d74d3f49-9c24-4b9f-98e2-3ecd7b5ef3b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6801","nonce":80661598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6803","nonce":80661598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6807","nonce":80661598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6805","nonce":80661598}]},"public_addr":"172.21.15.27:6801/80661598","cluster_addr":"172.21.15.27:6803/80661598","heartbeat_back_addr":"172.21.15.27:6807/80661598","heartbeat_front_addr":"172.21.15.27:6805/80661598","state":["exists","up"]},{"osd":4,"uuid":"2bf123f2-1b0d-41c4-aa3b-805bab267e59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6817","nonce":3943189827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6819","nonce":3943189827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6823","nonce":3943189827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6821","nonce":3943189827}]},"public_addr":"172.21.15.45:6817/3943189827","cluster_addr":"172.21.15.45:6819/3943189827","heartbeat_back_addr":"172.21.15.45:6823/3943189827","heartbeat_front_addr":"172.21.15.45:6821/3943189827","state":["exists","up"]},{"osd":5,"uuid":"c49ba5da-91c6-4a86-a97b-4aba89c253ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6809","nonce":3620760330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6811","nonce":3620760330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6815","nonce":3620760330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6813","nonce":3620760330}]},"public_addr":"172.21.15.45:6809/3620760330","cluster_addr":"172.21.15.45:6811/3620760330","heartbeat_back_addr":"172.21.15.45:6815/3620760330","heartbeat_front_addr":"172.21.15.45:6813/3620760330","state":["exists","up"]},{"osd":6,"uuid":"b0123475-89f7-4a34-9e33-d797b99dd9dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6801","nonce":1989083510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6803","nonce":1989083510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6807","nonce":1989083510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6805","nonce":1989083510}]},"public_addr":"172.21.15.45:6801/1989083510","cluster_addr":"172.21.15.45:6803/1989083510","heartbeat_back_addr":"172.21.15.45:6807/1989083510","heartbeat_front_addr":"172.21.15.45:6805/1989083510","state":["exists","up"]},{"osd":7,"uuid":"4dad773d-b7d8-46fc-8a10-a972aad27b29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6825","nonce":1513300294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6827","nonce":1513300294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6831","nonce":1513300294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6829","nonce":1513300294}]},"public_addr":"172.21.15.45:6825/1513300294","cluster_addr":"172.21.15.45:6827/1513300294","heartbeat_back_addr":"172.21.15.45:6831/1513300294","heartbeat_front_addr":"172.21.15.45:6829/1513300294","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.881855+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:17.133252+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.735339+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.500742+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.477571+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:17.167667+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.470291+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.873743+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]},{"pool":19,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T03:23:22.022 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:22.021+0000 7fa0e6549640 1 -- 172.21.15.27:0/576746327 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fa0cc06c610 msgr2=0x7fa0cc06ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:22.023 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:22.021+0000 7fa0e6549640 1 --2- 172.21.15.27:0/576746327 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fa0cc06c610 0x7fa0cc06ead0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fa0e0149f50 tx=0x7fa0d0009040 comp rx=0 tx=0).stop 2024-09-12T03:23:22.023 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:22.021+0000 7fa0e6549640 1 -- 172.21.15.27:0/576746327 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa0e015eb90 msgr2=0x7fa0e0159c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:22.023 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:22.021+0000 7fa0e6549640 1 --2- 172.21.15.27:0/576746327 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa0e015eb90 0x7fa0e0159c70 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7fa0d800ca20 tx=0x7fa0d800cee0 comp rx=0 tx=0).stop 2024-09-12T03:23:22.023 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:22.021+0000 7fa0e6549640 1 -- 172.21.15.27:0/576746327 shutdown_connections 2024-09-12T03:23:22.023 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:22.021+0000 7fa0e6549640 1 --2- 172.21.15.27:0/576746327 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fa0cc06c610 0x7fa0cc06ead0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:22.023 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:22.021+0000 7fa0e6549640 1 --2- 172.21.15.27:0/576746327 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa0e015eb90 0x7fa0e0159c70 unknown :-1 s=CLOSED pgs=1529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:22.023 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:22.021+0000 7fa0e6549640 1 --2- 172.21.15.27:0/576746327 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa0e00d5200 0x7fa0e01e7290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:22.023 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:22.021+0000 7fa0e6549640 1 --2- 172.21.15.27:0/576746327 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa0e00d1550 0x7fa0e00cabd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:22.023 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:22.021+0000 7fa0e6549640 1 -- 172.21.15.27:0/576746327 >> 172.21.15.27:0/576746327 conn(0x7fa0e006dce0 msgr2=0x7fa0e0161200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:22.023 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:22.021+0000 7fa0e6549640 1 -- 172.21.15.27:0/576746327 shutdown_connections 2024-09-12T03:23:22.023 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:22.021+0000 7fa0e6549640 1 -- 172.21.15.27:0/576746327 wait complete. 2024-09-12T03:23:22.034 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-12T03:23:22.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:22.135+0000 7f2e22798640 1 Processor -- start 2024-09-12T03:23:22.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:22.135+0000 7f2e22798640 1 -- start start 2024-09-12T03:23:22.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:22.136+0000 7f2e22798640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2e1c0d1990 0x7f2e1c0b3390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:22.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:22.136+0000 7f2e22798640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2e1c0b6690 con 0x7f2e1c15e780 2024-09-12T03:23:22.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:22.136+0000 7f2e22798640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2e1c0b6970 con 0x7f2e1c15eb60 2024-09-12T03:23:22.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:22.136+0000 7f2e22798640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f2e1c0b7950 con 0x7f2e1c0d1990 2024-09-12T03:23:22.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:22.136+0000 7f2e1bfff640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f2e1c15e780 0x7f2e1c0d3e70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:56088/0 (socket says 172.21.15.27:56088) 2024-09-12T03:23:22.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:22.136+0000 7f2e1bfff640 1 -- 172.21.15.27:0/1304689848 learned_addr learned my addr 172.21.15.27:0/1304689848 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:23:22.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:22.136+0000 7f2e20d35640 1 --2- 172.21.15.27:0/1304689848 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2e1c0d1990 0x7f2e1c0b3390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:22.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:22.136+0000 7f2e1affd640 1 -- 172.21.15.27:0/1304689848 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3406834937 0 0) 0x7f2e1c0b6690 con 0x7f2e1c15e780 2024-09-12T03:23:22.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:22.136+0000 7f2e1affd640 1 -- 172.21.15.27:0/1304689848 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2e08003520 con 0x7f2e1c15e780 2024-09-12T03:23:22.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:22.137+0000 7f2e1affd640 1 -- 172.21.15.27:0/1304689848 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 451147575 0 0) 0x7f2e08003520 con 0x7f2e1c15e780 2024-09-12T03:23:22.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:22.137+0000 7f2e1affd640 1 -- 172.21.15.27:0/1304689848 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2e1c0d1990 msgr2=0x7f2e1c0b3390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:22.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:22.137+0000 7f2e1affd640 1 --2- 172.21.15.27:0/1304689848 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2e1c0d1990 0x7f2e1c0b3390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:22.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:22.137+0000 7f2e1affd640 1 -- 172.21.15.27:0/1304689848 >> v1:172.21.15.45:6789/0 conn(0x7f2e1c15eb60 legacy=0x7f2e1c0d1280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:22.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:22.137+0000 7f2e1affd640 1 -- 172.21.15.27:0/1304689848 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e1c0b6c50 con 0x7f2e1c15e780 2024-09-12T03:23:22.142 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:22.137+0000 7f2e1affd640 1 -- 172.21.15.27:0/1304689848 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 847755213 0 0) 0x7f2e0c001670 con 0x7f2e1c15e780 2024-09-12T03:23:22.142 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:22.137+0000 7f2e1affd640 1 -- 172.21.15.27:0/1304689848 >> v1:172.21.15.27:6789/0 conn(0x7f2e1c15e780 legacy=0x7f2e1c0d3e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:22.142 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:22.137+0000 7f2e1affd640 1 --2- 172.21.15.27:0/1304689848 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2e1c0d1990 0x7f2e080039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:22.142 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:22.137+0000 7f2e1affd640 1 --2- 172.21.15.27:0/1304689848 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2e08003f20 0x7f2e08004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:22.142 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:22.137+0000 7f2e1affd640 1 --2- 172.21.15.27:0/1304689848 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2e08004d40 0x7f2e080051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:22.142 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:22.137+0000 7f2e1affd640 1 -- 172.21.15.27:0/1304689848 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f2e0800b9f0 con 0x7f2e08003f20 2024-09-12T03:23:22.142 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:22.137+0000 7f2e1affd640 1 -- 172.21.15.27:0/1304689848 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f2e0800bba0 con 0x7f2e08004d40 2024-09-12T03:23:22.142 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:22.137+0000 7f2e1affd640 1 -- 172.21.15.27:0/1304689848 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f2e0800bda0 con 0x7f2e1c0d1990 2024-09-12T03:23:22.142 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:22.137+0000 7f2e1b7fe640 1 --2- 172.21.15.27:0/1304689848 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2e08003f20 0x7f2e08004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:22.142 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:22.137+0000 7f2e1bfff640 1 --2- 172.21.15.27:0/1304689848 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2e1c0d1990 0x7f2e080039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:22.143 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:22.137+0000 7f2e20d35640 1 --2- 172.21.15.27:0/1304689848 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2e08004d40 0x7f2e080051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:22.143 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:22.137+0000 7f2e1b7fe640 1 -- 172.21.15.27:0/1304689848 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2e1c0d1990 msgr2=0x7f2e080039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:22.143 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:22.137+0000 7f2e1b7fe640 1 --2- 172.21.15.27:0/1304689848 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2e1c0d1990 0x7f2e080039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:22.143 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:22.137+0000 7f2e1b7fe640 1 -- 172.21.15.27:0/1304689848 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2e08004d40 msgr2=0x7f2e080051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:22.143 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:22.137+0000 7f2e1b7fe640 1 --2- 172.21.15.27:0/1304689848 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2e08004d40 0x7f2e080051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:22.143 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:22.137+0000 7f2e1b7fe640 1 -- 172.21.15.27:0/1304689848 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e08003520 con 0x7f2e08003f20 2024-09-12T03:23:22.143 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:22.138+0000 7f2e1b7fe640 1 --2- 172.21.15.27:0/1304689848 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2e08003f20 0x7f2e08004800 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7f2e04001860 tx=0x7f2e08005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7a24d4382779fbf server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:22.143 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:22.138+0000 7f2e1affd640 1 -- 172.21.15.27:0/1304689848 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e04037070 con 0x7f2e08003f20 2024-09-12T03:23:22.143 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:22.138+0000 7f2e1affd640 1 -- 172.21.15.27:0/1304689848 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e0402ab60 con 0x7f2e08003f20 2024-09-12T03:23:22.143 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:22.138+0000 7f2e1affd640 1 -- 172.21.15.27:0/1304689848 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e04001430 con 0x7f2e08003f20 2024-09-12T03:23:22.143 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:22.139+0000 7f2e22798640 1 -- 172.21.15.27:0/1304689848 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2e08003f20 msgr2=0x7f2e08004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:22.144 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:22.139+0000 7f2e22798640 1 --2- 172.21.15.27:0/1304689848 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2e08003f20 0x7f2e08004800 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7f2e04001860 tx=0x7f2e08005830 comp rx=0 tx=0).stop 2024-09-12T03:23:22.144 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:22.139+0000 7f2e20d35640 1 -- 172.21.15.27:0/1304689848 reap_dead start 2024-09-12T03:23:22.144 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:22.139+0000 7f2e22798640 1 -- 172.21.15.27:0/1304689848 shutdown_connections 2024-09-12T03:23:22.144 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:22.139+0000 7f2e22798640 1 -- 172.21.15.27:0/1304689848 >> 172.21.15.27:0/1304689848 conn(0x7f2e1c06da60 msgr2=0x7f2e1c06de70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:22.144 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:22.139+0000 7f2e22798640 1 -- 172.21.15.27:0/1304689848 shutdown_connections 2024-09-12T03:23:22.144 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:22.139+0000 7f2e22798640 1 -- 172.21.15.27:0/1304689848 wait complete. 2024-09-12T03:23:22.147 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:22.140+0000 7f2e22798640 1 Processor -- start 2024-09-12T03:23:22.147 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:22.140+0000 7f2e22798640 1 -- start start 2024-09-12T03:23:22.147 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:22.141+0000 7f2e22798640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2e08003f20 0x7f2e1c1f5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:22.147 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:22.141+0000 7f2e22798640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2e1c15d610 0x7f2e1c1f7530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:22.147 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:22.141+0000 7f2e22798640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2e1c1f7a70 0x7f2e1c1f3110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:22.147 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:22.141+0000 7f2e22798640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f2e1c0b6340 con 0x7f2e1c1f7a70 2024-09-12T03:23:22.147 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:22.141+0000 7f2e22798640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f2e1c0b7210 con 0x7f2e1c15d610 2024-09-12T03:23:22.148 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:22.141+0000 7f2e22798640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f2e1c0d4580 con 0x7f2e08003f20 2024-09-12T03:23:22.148 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:22.141+0000 7f2e20d35640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2e1c1f7a70 0x7f2e1c1f3110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:22.148 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:22.141+0000 7f2e20d35640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2e1c1f7a70 0x7f2e1c1f3110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42368/0 (socket says 172.21.15.27:42368) 2024-09-12T03:23:22.148 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:22.141+0000 7f2e20d35640 1 -- 172.21.15.27:0/824965329 learned_addr learned my addr 172.21.15.27:0/824965329 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:23:22.148 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:22.141+0000 7f2e1b7fe640 1 --2- 172.21.15.27:0/824965329 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2e1c15d610 0x7f2e1c1f7530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:22.148 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:22.141+0000 7f2e1bfff640 1 --2- 172.21.15.27:0/824965329 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2e08003f20 0x7f2e1c1f5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:22.148 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:22.142+0000 7f2e20d35640 1 -- 172.21.15.27:0/824965329 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2e08003f20 msgr2=0x7f2e1c1f5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:22.148 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:22.142+0000 7f2e20d35640 1 --2- 172.21.15.27:0/824965329 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2e08003f20 0x7f2e1c1f5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:22.148 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:22.142+0000 7f2e20d35640 1 -- 172.21.15.27:0/824965329 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2e1c15d610 msgr2=0x7f2e1c1f7530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:22.148 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:22.142+0000 7f2e20d35640 1 --2- 172.21.15.27:0/824965329 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2e1c15d610 0x7f2e1c1f7530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:22.148 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:22.142+0000 7f2e20d35640 1 -- 172.21.15.27:0/824965329 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e1c15da90 con 0x7f2e1c1f7a70 2024-09-12T03:23:22.148 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:22.142+0000 7f2e20d35640 1 --2- 172.21.15.27:0/824965329 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2e1c1f7a70 0x7f2e1c1f3110 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7f2e1c15eff0 tx=0x7f2e1400f1b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:22.149 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:22.142+0000 7f2e18ff9640 1 -- 172.21.15.27:0/824965329 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e14003e60 con 0x7f2e1c1f7a70 2024-09-12T03:23:22.149 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:22.142+0000 7f2e18ff9640 1 -- 172.21.15.27:0/824965329 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e1401c070 con 0x7f2e1c1f7a70 2024-09-12T03:23:22.149 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:22.142+0000 7f2e22798640 1 -- 172.21.15.27:0/824965329 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e1c0abf70 con 0x7f2e1c1f7a70 2024-09-12T03:23:22.149 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:22.142+0000 7f2e18ff9640 1 -- 172.21.15.27:0/824965329 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e1400fb10 con 0x7f2e1c1f7a70 2024-09-12T03:23:22.149 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:22.142+0000 7f2e22798640 1 -- 172.21.15.27:0/824965329 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e1c1f2910 con 0x7f2e1c1f7a70 2024-09-12T03:23:22.149 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:22.143+0000 7f2e22798640 1 -- 172.21.15.27:0/824965329 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e1c0d3e70 con 0x7f2e1c1f7a70 2024-09-12T03:23:22.149 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:22.144+0000 7f2e18ff9640 1 -- 172.21.15.27:0/824965329 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 24) ==== 91212+0+0 (secure 0 0 0) 0x7f2e1400fcb0 con 0x7f2e1c1f7a70 2024-09-12T03:23:22.150 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:22.144+0000 7f2e18ff9640 1 --2- 172.21.15.27:0/824965329 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f2dfc06c640 0x7f2dfc06eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:22.150 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:22.144+0000 7f2e18ff9640 1 -- 172.21.15.27:0/824965329 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 5765+0+0 (secure 0 0 0) 0x7f2e14089fb0 con 0x7f2e1c1f7a70 2024-09-12T03:23:22.150 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:22.144+0000 7f2e1bfff640 1 --2- 172.21.15.27:0/824965329 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f2dfc06c640 0x7f2dfc06eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:22.150 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:22.145+0000 7f2e1bfff640 1 --2- 172.21.15.27:0/824965329 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f2dfc06c640 0x7f2dfc06eb00 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f2e0c000c00 tx=0x7f2e0c0078f0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:22.150 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:22.148+0000 7f2e18ff9640 1 -- 172.21.15.27:0/824965329 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f2e14010040 con 0x7f2e1c1f7a70 2024-09-12T03:23:22.326 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:22.325+0000 7f2e22798640 1 -- 172.21.15.27:0/824965329 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2e1c0d0cc0 con 0x7f2e1c1f7a70 2024-09-12T03:23:22.327 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:22.326+0000 7f2e18ff9640 1 -- 172.21.15.27:0/824965329 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v148) ==== 74+0+16793 (secure 0 0 0) 0x7f2e1405a250 con 0x7f2e1c1f7a70 2024-09-12T03:23:22.327 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:23:22.327 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":148,"fsid":"d2df504e-2647-4385-a5db-9fdf69f487a0","created":"2024-09-12T03:16:07.778937+0000","modified":"2024-09-12T03:23:21.031279+0000","last_up_change":"2024-09-12T03:16:19.314778+0000","last_in_change":"2024-09-12T03:16:09.893086+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T03:16:20.097646+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_metadata","create_time":"2024-09-12T03:23:18.699292+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"148","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_data","create_time":"2024-09-12T03:23:20.022169+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"148","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1851116c-1fb3-4dc9-9772-f90e2ef4c355","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6817","nonce":3002321488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6819","nonce":3002321488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6823","nonce":3002321488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6821","nonce":3002321488}]},"public_addr":"172.21.15.27:6817/3002321488","cluster_addr":"172.21.15.27:6819/3002321488","heartbeat_back_addr":"172.21.15.27:6823/3002321488","heartbeat_front_addr":"172.21.15.27:6821/3002321488","state":["exists","up"]},{"osd":1,"uuid":"f38ae17d-73eb-4204-b99d-975fa806fb87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6809","nonce":1613102886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6811","nonce":1613102886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6815","nonce":1613102886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6813","nonce":1613102886}]},"public_addr":"172.21.15.27:6809/1613102886","cluster_addr":"172.21.15.27:6811/1613102886","heartbeat_back_addr":"172.21.15.27:6815/1613102886","heartbeat_front_addr":"172.21.15.27:6813/1613102886","state":["exists","up"]},{"osd":2,"uuid":"fb12dcc8-bdd0-42b8-b46d-8b10106224bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6825","nonce":3534723355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6827","nonce":3534723355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6831","nonce":3534723355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6829","nonce":3534723355}]},"public_addr":"172.21.15.27:6825/3534723355","cluster_addr":"172.21.15.27:6827/3534723355","heartbeat_back_addr":"172.21.15.27:6831/3534723355","heartbeat_front_addr":"172.21.15.27:6829/3534723355","state":["exists","up"]},{"osd":3,"uuid":"d74d3f49-9c24-4b9f-98e2-3ecd7b5ef3b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6801","nonce":80661598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6803","nonce":80661598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6807","nonce":80661598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6805","nonce":80661598}]},"public_addr":"172.21.15.27:6801/80661598","cluster_addr":"172.21.15.27:6803/80661598","heartbeat_back_addr":"172.21.15.27:6807/80661598","heartbeat_front_addr":"172.21.15.27:6805/80661598","state":["exists","up"]},{"osd":4,"uuid":"2bf123f2-1b0d-41c4-aa3b-805bab267e59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6817","nonce":3943189827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6819","nonce":3943189827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6823","nonce":3943189827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6821","nonce":3943189827}]},"public_addr":"172.21.15.45:6817/3943189827","cluster_addr":"172.21.15.45:6819/3943189827","heartbeat_back_addr":"172.21.15.45:6823/3943189827","heartbeat_front_addr":"172.21.15.45:6821/3943189827","state":["exists","up"]},{"osd":5,"uuid":"c49ba5da-91c6-4a86-a97b-4aba89c253ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6809","nonce":3620760330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6811","nonce":3620760330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6815","nonce":3620760330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6813","nonce":3620760330}]},"public_addr":"172.21.15.45:6809/3620760330","cluster_addr":"172.21.15.45:6811/3620760330","heartbeat_back_addr":"172.21.15.45:6815/3620760330","heartbeat_front_addr":"172.21.15.45:6813/3620760330","state":["exists","up"]},{"osd":6,"uuid":"b0123475-89f7-4a34-9e33-d797b99dd9dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6801","nonce":1989083510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6803","nonce":1989083510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6807","nonce":1989083510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6805","nonce":1989083510}]},"public_addr":"172.21.15.45:6801/1989083510","cluster_addr":"172.21.15.45:6803/1989083510","heartbeat_back_addr":"172.21.15.45:6807/1989083510","heartbeat_front_addr":"172.21.15.45:6805/1989083510","state":["exists","up"]},{"osd":7,"uuid":"4dad773d-b7d8-46fc-8a10-a972aad27b29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6825","nonce":1513300294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6827","nonce":1513300294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6831","nonce":1513300294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6829","nonce":1513300294}]},"public_addr":"172.21.15.45:6825/1513300294","cluster_addr":"172.21.15.45:6827/1513300294","heartbeat_back_addr":"172.21.15.45:6831/1513300294","heartbeat_front_addr":"172.21.15.45:6829/1513300294","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.881855+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:17.133252+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.735339+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.500742+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.477571+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:17.167667+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.470291+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.873743+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]},{"pool":19,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T03:23:22.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:22.328+0000 7f2e22798640 1 -- 172.21.15.27:0/824965329 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f2dfc06c640 msgr2=0x7f2dfc06eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:22.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:22.328+0000 7f2e22798640 1 --2- 172.21.15.27:0/824965329 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f2dfc06c640 0x7f2dfc06eb00 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f2e0c000c00 tx=0x7f2e0c0078f0 comp rx=0 tx=0).stop 2024-09-12T03:23:22.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:22.328+0000 7f2e22798640 1 -- 172.21.15.27:0/824965329 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2e1c1f7a70 msgr2=0x7f2e1c1f3110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:22.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:22.328+0000 7f2e22798640 1 --2- 172.21.15.27:0/824965329 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2e1c1f7a70 0x7f2e1c1f3110 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7f2e1c15eff0 tx=0x7f2e1400f1b0 comp rx=0 tx=0).stop 2024-09-12T03:23:22.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:22.328+0000 7f2e22798640 1 -- 172.21.15.27:0/824965329 shutdown_connections 2024-09-12T03:23:22.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:22.328+0000 7f2e22798640 1 --2- 172.21.15.27:0/824965329 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f2dfc06c640 0x7f2dfc06eb00 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:22.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:22.328+0000 7f2e22798640 1 --2- 172.21.15.27:0/824965329 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2e1c1f7a70 0x7f2e1c1f3110 unknown :-1 s=CLOSED pgs=1532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:22.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:22.328+0000 7f2e22798640 1 --2- 172.21.15.27:0/824965329 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2e1c15d610 0x7f2e1c1f7530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:22.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:22.328+0000 7f2e22798640 1 --2- 172.21.15.27:0/824965329 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2e08003f20 0x7f2e1c1f5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:22.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:22.328+0000 7f2e22798640 1 -- 172.21.15.27:0/824965329 >> 172.21.15.27:0/824965329 conn(0x7f2e1c06da60 msgr2=0x7f2e1c0d33c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:22.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:22.328+0000 7f2e22798640 1 -- 172.21.15.27:0/824965329 shutdown_connections 2024-09-12T03:23:22.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:22.328+0000 7f2e22798640 1 -- 172.21.15.27:0/824965329 wait complete. 2024-09-12T03:23:22.341 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-12T03:23:22.446 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:22.445+0000 7ff63531a640 1 Processor -- start 2024-09-12T03:23:22.446 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:22.445+0000 7ff63531a640 1 -- start start 2024-09-12T03:23:22.447 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:22.446+0000 7ff63531a640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ff63006dfd0 0x7ff6300d1da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:22.447 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:22.446+0000 7ff63531a640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ff6300d3790 0x7ff6300b7a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:22.447 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:22.446+0000 7ff63531a640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff6300d2e30 con 0x7ff630162040 2024-09-12T03:23:22.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:22.446+0000 7ff63531a640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7ff6300b7f90 con 0x7ff63006dfd0 2024-09-12T03:23:22.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:22.446+0000 7ff63531a640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7ff630162420 con 0x7ff6300d3790 2024-09-12T03:23:22.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:22.446+0000 7ff62effd640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7ff630162040 0x7ff63006d8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:56092/0 (socket says 172.21.15.27:56092) 2024-09-12T03:23:22.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:22.446+0000 7ff62effd640 1 -- 172.21.15.27:0/3806470265 learned_addr learned my addr 172.21.15.27:0/3806470265 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:23:22.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:22.446+0000 7ff62e7fc640 1 --2- 172.21.15.27:0/3806470265 >> v2:172.21.15.45:3300/0 conn(0x7ff63006dfd0 0x7ff6300d1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:22.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:22.446+0000 7ff62f7fe640 1 --2- 172.21.15.27:0/3806470265 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ff6300d3790 0x7ff6300b7a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:22.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:22.446+0000 7ff62dffb640 1 -- 172.21.15.27:0/3806470265 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3599143773 0 0) 0x7ff6300d2e30 con 0x7ff630162040 2024-09-12T03:23:22.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:22.446+0000 7ff62dffb640 1 -- 172.21.15.27:0/3806470265 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff618003520 con 0x7ff630162040 2024-09-12T03:23:22.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:22.447+0000 7ff62f7fe640 1 -- 172.21.15.27:0/3806470265 >> v2:172.21.15.45:3300/0 conn(0x7ff63006dfd0 msgr2=0x7ff6300d1da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:22.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:22.447+0000 7ff62dffb640 1 -- 172.21.15.27:0/3806470265 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1818429086 0 0) 0x7ff618003520 con 0x7ff630162040 2024-09-12T03:23:22.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:22.447+0000 7ff62f7fe640 1 --2- 172.21.15.27:0/3806470265 >> v2:172.21.15.45:3300/0 conn(0x7ff63006dfd0 0x7ff6300d1da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:22.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:22.447+0000 7ff62f7fe640 1 -- 172.21.15.27:0/3806470265 >> v1:172.21.15.27:6789/0 conn(0x7ff630162040 legacy=0x7ff63006d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:22.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:22.447+0000 7ff62f7fe640 1 -- 172.21.15.27:0/3806470265 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6300d30d0 con 0x7ff6300d3790 2024-09-12T03:23:22.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:22.447+0000 7ff62f7fe640 1 --2- 172.21.15.27:0/3806470265 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ff6300d3790 0x7ff6300b7a50 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7ff628009d90 tx=0x7ff62802a5f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ff94dc908de9cd37 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:22.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:22.448+0000 7ff62dffb640 1 -- 172.21.15.27:0/3806470265 <== mon.2 v2:172.21.15.45:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff620001670 con 0x7ff6300d3790 2024-09-12T03:23:22.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:22.448+0000 7ff62dffb640 1 -- 172.21.15.27:0/3806470265 <== mon.2 v2:172.21.15.45:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff628007540 con 0x7ff6300d3790 2024-09-12T03:23:22.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:22.448+0000 7ff62dffb640 1 -- 172.21.15.27:0/3806470265 <== mon.2 v2:172.21.15.45:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6280078f0 con 0x7ff6300d3790 2024-09-12T03:23:22.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:22.448+0000 7ff63531a640 1 -- 172.21.15.27:0/3806470265 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ff6300d3790 msgr2=0x7ff6300b7a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:22.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:22.448+0000 7ff63531a640 1 --2- 172.21.15.27:0/3806470265 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ff6300d3790 0x7ff6300b7a50 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7ff628009d90 tx=0x7ff62802a5f0 comp rx=0 tx=0).stop 2024-09-12T03:23:22.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:22.448+0000 7ff63531a640 1 -- 172.21.15.27:0/3806470265 shutdown_connections 2024-09-12T03:23:22.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:22.448+0000 7ff63531a640 1 --2- 172.21.15.27:0/3806470265 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ff6300d3790 0x7ff6300b7a50 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:22.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:22.448+0000 7ff63531a640 1 --2- 172.21.15.27:0/3806470265 >> v2:172.21.15.45:3300/0 conn(0x7ff63006dfd0 0x7ff6300d1da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:22.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:22.448+0000 7ff63531a640 1 -- 172.21.15.27:0/3806470265 >> 172.21.15.27:0/3806470265 conn(0x7ff6300b6700 msgr2=0x7ff6300b6b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:22.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:22.448+0000 7ff63531a640 1 -- 172.21.15.27:0/3806470265 shutdown_connections 2024-09-12T03:23:22.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:22.448+0000 7ff63531a640 1 -- 172.21.15.27:0/3806470265 wait complete. 2024-09-12T03:23:22.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:22.450+0000 7ff63531a640 1 Processor -- start 2024-09-12T03:23:22.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:22.450+0000 7ff63531a640 1 -- start start 2024-09-12T03:23:22.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:22.450+0000 7ff63531a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff63006dfd0 0x7ff6300cd2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:22.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:22.450+0000 7ff63531a640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff6300d3790 0x7ff63014dd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:22.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:22.450+0000 7ff63531a640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ff630162040 0x7ff6300cc290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:22.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:22.450+0000 7ff63531a640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7ff6300d4530 con 0x7ff6300d3790 2024-09-12T03:23:22.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:22.450+0000 7ff63531a640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ff6300cee00 con 0x7ff63006dfd0 2024-09-12T03:23:22.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:22.450+0000 7ff63531a640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7ff6300d4150 con 0x7ff630162040 2024-09-12T03:23:22.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:22.450+0000 7ff62e7fc640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff6300d3790 0x7ff63014dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:22.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:22.450+0000 7ff62e7fc640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff6300d3790 0x7ff63014dd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42372/0 (socket says 172.21.15.27:42372) 2024-09-12T03:23:22.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:22.451+0000 7ff62e7fc640 1 -- 172.21.15.27:0/2075104720 learned_addr learned my addr 172.21.15.27:0/2075104720 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:23:22.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:22.451+0000 7ff62f7fe640 1 --2- 172.21.15.27:0/2075104720 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ff630162040 0x7ff6300cc290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:22.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:22.451+0000 7ff62effd640 1 --2- 172.21.15.27:0/2075104720 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff63006dfd0 0x7ff6300cd2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:22.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:22.451+0000 7ff62e7fc640 1 -- 172.21.15.27:0/2075104720 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ff630162040 msgr2=0x7ff6300cc290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:22.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:22.451+0000 7ff62e7fc640 1 --2- 172.21.15.27:0/2075104720 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ff630162040 0x7ff6300cc290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:22.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:22.451+0000 7ff62e7fc640 1 -- 172.21.15.27:0/2075104720 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff63006dfd0 msgr2=0x7ff6300cd2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:22.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:22.451+0000 7ff62e7fc640 1 --2- 172.21.15.27:0/2075104720 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff63006dfd0 0x7ff6300cd2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:22.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:22.451+0000 7ff62e7fc640 1 -- 172.21.15.27:0/2075104720 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6300cd920 con 0x7ff6300d3790 2024-09-12T03:23:22.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:22.451+0000 7ff62effd640 1 --2- 172.21.15.27:0/2075104720 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff63006dfd0 0x7ff6300cd2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:23:22.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:22.451+0000 7ff62e7fc640 1 --2- 172.21.15.27:0/2075104720 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff6300d3790 0x7ff63014dd00 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7ff62400e9b0 tx=0x7ff62400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:22.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:22.451+0000 7ff6177fe640 1 -- 172.21.15.27:0/2075104720 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff624017070 con 0x7ff6300d3790 2024-09-12T03:23:22.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:22.451+0000 7ff6177fe640 1 -- 172.21.15.27:0/2075104720 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff62400c9a0 con 0x7ff6300d3790 2024-09-12T03:23:22.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:22.451+0000 7ff63531a640 1 -- 172.21.15.27:0/2075104720 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6300cdc00 con 0x7ff6300d3790 2024-09-12T03:23:22.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:22.451+0000 7ff6177fe640 1 -- 172.21.15.27:0/2075104720 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff62400cd90 con 0x7ff6300d3790 2024-09-12T03:23:22.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:22.451+0000 7ff63531a640 1 -- 172.21.15.27:0/2075104720 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6300ce1c0 con 0x7ff6300d3790 2024-09-12T03:23:22.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:22.452+0000 7ff63531a640 1 -- 172.21.15.27:0/2075104720 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff63006e6e0 con 0x7ff6300d3790 2024-09-12T03:23:22.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:22.453+0000 7ff6177fe640 1 -- 172.21.15.27:0/2075104720 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 24) ==== 91212+0+0 (secure 0 0 0) 0x7ff62401a070 con 0x7ff6300d3790 2024-09-12T03:23:22.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:22.453+0000 7ff6177fe640 1 --2- 172.21.15.27:0/2075104720 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7ff61c06c610 0x7ff61c06ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:22.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:22.453+0000 7ff6177fe640 1 -- 172.21.15.27:0/2075104720 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 5765+0+0 (secure 0 0 0) 0x7ff624089b90 con 0x7ff6300d3790 2024-09-12T03:23:22.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:22.454+0000 7ff62effd640 1 --2- 172.21.15.27:0/2075104720 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7ff61c06c610 0x7ff61c06ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:22.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:22.454+0000 7ff62effd640 1 --2- 172.21.15.27:0/2075104720 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7ff61c06c610 0x7ff61c06ead0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7ff620001eb0 tx=0x7ff620009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:22.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:22.458+0000 7ff6177fe640 1 -- 172.21.15.27:0/2075104720 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7ff624059e30 con 0x7ff6300d3790 2024-09-12T03:23:22.665 INFO:tasks.ceph.mds.c.smithi027.stderr:2024-09-12T03:23:22.664+0000 7f17536bd640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-12T03:23:22.670 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:22.669+0000 7ff63531a640 1 -- 172.21.15.27:0/2075104720 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7ff6300cfed0 con 0x7ff6300d3790 2024-09-12T03:23:23.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:23.626+0000 7ff6177fe640 1 -- 172.21.15.27:0/2075104720 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v90) ==== 118+0+0 (secure 0 0 0) 0x7ff624059730 con 0x7ff6300d3790 2024-09-12T03:23:23.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:23.629+0000 7ff63531a640 1 -- 172.21.15.27:0/2075104720 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7ff61c06c610 msgr2=0x7ff61c06ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:23.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:23.629+0000 7ff63531a640 1 --2- 172.21.15.27:0/2075104720 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7ff61c06c610 0x7ff61c06ead0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7ff620001eb0 tx=0x7ff620009040 comp rx=0 tx=0).stop 2024-09-12T03:23:23.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:23.629+0000 7ff63531a640 1 -- 172.21.15.27:0/2075104720 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff6300d3790 msgr2=0x7ff63014dd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:23.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:23.629+0000 7ff63531a640 1 --2- 172.21.15.27:0/2075104720 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff6300d3790 0x7ff63014dd00 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7ff62400e9b0 tx=0x7ff62400ee70 comp rx=0 tx=0).stop 2024-09-12T03:23:23.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:23.629+0000 7ff63531a640 1 -- 172.21.15.27:0/2075104720 shutdown_connections 2024-09-12T03:23:23.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:23.629+0000 7ff63531a640 1 --2- 172.21.15.27:0/2075104720 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ff630162040 0x7ff6300cc290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:23.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:23.629+0000 7ff63531a640 1 --2- 172.21.15.27:0/2075104720 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7ff61c06c610 0x7ff61c06ead0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:23.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:23.629+0000 7ff63531a640 1 --2- 172.21.15.27:0/2075104720 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff6300d3790 0x7ff63014dd00 unknown :-1 s=CLOSED pgs=1534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:23.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:23.629+0000 7ff63531a640 1 --2- 172.21.15.27:0/2075104720 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff63006dfd0 0x7ff6300cd2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:23.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:23.629+0000 7ff63531a640 1 -- 172.21.15.27:0/2075104720 >> 172.21.15.27:0/2075104720 conn(0x7ff6300b6700 msgr2=0x7ff6300d46b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:23.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:23.630+0000 7ff63531a640 1 -- 172.21.15.27:0/2075104720 shutdown_connections 2024-09-12T03:23:23.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:23.630+0000 7ff63531a640 1 -- 172.21.15.27:0/2075104720 wait complete. 2024-09-12T03:23:23.644 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-12T03:23:23.645 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-12T03:23:23.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:23.757+0000 7fb3a6d78640 1 Processor -- start 2024-09-12T03:23:23.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:23.757+0000 7fb3a6d78640 1 -- start start 2024-09-12T03:23:23.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:23.758+0000 7fb3a6d78640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb3a00cef40 0x7fb3a00b4000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:23.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:23.758+0000 7fb3a6d78640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fb3a00cf320 0x7fb3a015f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:23.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:23.758+0000 7fb3a6d78640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb3a015f600 con 0x7fb3a006c650 2024-09-12T03:23:23.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:23.758+0000 7fb3a6d78640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7fb3a006dee0 con 0x7fb3a00cf320 2024-09-12T03:23:23.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:23.758+0000 7fb3a6d78640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fb3a006cff0 con 0x7fb3a00cef40 2024-09-12T03:23:23.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:23.758+0000 7fb3a5315640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fb3a00cf320 0x7fb3a015f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:23.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:23.758+0000 7fb3a5315640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fb3a00cf320 0x7fb3a015f0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42378/0 (socket says 172.21.15.27:42378) 2024-09-12T03:23:23.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:23.758+0000 7fb3a5315640 1 -- 172.21.15.27:0/1420498694 learned_addr learned my addr 172.21.15.27:0/1420498694 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:23:23.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:23.758+0000 7fb3a4b14640 1 --2- 172.21.15.27:0/1420498694 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb3a00cef40 0x7fb3a00b4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:23.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:23.758+0000 7fb3a5315640 1 -- 172.21.15.27:0/1420498694 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb3a00cef40 msgr2=0x7fb3a00b4000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:23.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:23.758+0000 7fb3a5315640 1 --2- 172.21.15.27:0/1420498694 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb3a00cef40 0x7fb3a00b4000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:23.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:23.758+0000 7fb3a5315640 1 -- 172.21.15.27:0/1420498694 >> v1:172.21.15.45:6789/0 conn(0x7fb3a006c650 legacy=0x7fb3a00710c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:23.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:23.758+0000 7fb3a5315640 1 -- 172.21.15.27:0/1420498694 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3a006d2d0 con 0x7fb3a00cf320 2024-09-12T03:23:23.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:23.759+0000 7fb3a5315640 1 --2- 172.21.15.27:0/1420498694 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb3a00cf320 0x7fb3a015f0c0 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7fb398009870 tx=0x7fb39802a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8176ee2d48443059 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:23.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:23.759+0000 7fb39f7fe640 1 -- 172.21.15.27:0/1420498694 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb398037070 con 0x7fb3a00cf320 2024-09-12T03:23:23.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:23.759+0000 7fb39f7fe640 1 -- 172.21.15.27:0/1420498694 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb398032440 con 0x7fb3a00cf320 2024-09-12T03:23:23.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:23.759+0000 7fb39f7fe640 1 -- 172.21.15.27:0/1420498694 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb398032810 con 0x7fb3a00cf320 2024-09-12T03:23:23.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:23.760+0000 7fb3a6d78640 1 -- 172.21.15.27:0/1420498694 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb3a00cf320 msgr2=0x7fb3a015f0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:23.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:23.760+0000 7fb3a6d78640 1 --2- 172.21.15.27:0/1420498694 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb3a00cf320 0x7fb3a015f0c0 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7fb398009870 tx=0x7fb39802a1c0 comp rx=0 tx=0).stop 2024-09-12T03:23:23.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:23.760+0000 7fb3a6d78640 1 -- 172.21.15.27:0/1420498694 shutdown_connections 2024-09-12T03:23:23.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:23.760+0000 7fb3a6d78640 1 --2- 172.21.15.27:0/1420498694 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb3a00cf320 0x7fb3a015f0c0 unknown :-1 s=CLOSED pgs=1535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:23.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:23.760+0000 7fb3a6d78640 1 --2- 172.21.15.27:0/1420498694 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb3a00cef40 0x7fb3a00b4000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:23.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:23.760+0000 7fb3a6d78640 1 -- 172.21.15.27:0/1420498694 >> 172.21.15.27:0/1420498694 conn(0x7fb3a006bb30 msgr2=0x7fb3a006bf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:23.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:23.760+0000 7fb3a6d78640 1 -- 172.21.15.27:0/1420498694 shutdown_connections 2024-09-12T03:23:23.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:23.760+0000 7fb3a6d78640 1 -- 172.21.15.27:0/1420498694 wait complete. 2024-09-12T03:23:23.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:23.761+0000 7fb3a6d78640 1 Processor -- start 2024-09-12T03:23:23.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:23.761+0000 7fb3a6d78640 1 -- start start 2024-09-12T03:23:23.765 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:23.762+0000 7fb3a6d78640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb3a006c650 0x7fb3a00aef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:23.765 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:23.762+0000 7fb3a6d78640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb3a00cef40 0x7fb3a0157ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:23.765 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:23.762+0000 7fb3a6d78640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb3a00cf320 0x7fb3a015a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:23.765 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:23.762+0000 7fb3a6d78640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fb3a006db70 con 0x7fb3a006c650 2024-09-12T03:23:23.765 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:23.762+0000 7fb3a6d78640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb3a006d890 con 0x7fb3a00cef40 2024-09-12T03:23:23.765 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:23.762+0000 7fb3a6d78640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fb3a00ce440 con 0x7fb3a00cf320 2024-09-12T03:23:23.765 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:23.762+0000 7fb3a4b14640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb3a006c650 0x7fb3a00aef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:23.765 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:23.762+0000 7fb3a4b14640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb3a006c650 0x7fb3a00aef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42384/0 (socket says 172.21.15.27:42384) 2024-09-12T03:23:23.765 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:23.762+0000 7fb3a4b14640 1 -- 172.21.15.27:0/2294063717 learned_addr learned my addr 172.21.15.27:0/2294063717 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:23:23.766 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:23.762+0000 7fb39ffff640 1 --2- 172.21.15.27:0/2294063717 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb3a00cef40 0x7fb3a0157ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:23.766 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:23.762+0000 7fb3a5315640 1 --2- 172.21.15.27:0/2294063717 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb3a00cf320 0x7fb3a015a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:23.766 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:23.762+0000 7fb3a4b14640 1 -- 172.21.15.27:0/2294063717 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb3a00cf320 msgr2=0x7fb3a015a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:23.766 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:23.762+0000 7fb3a4b14640 1 --2- 172.21.15.27:0/2294063717 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb3a00cf320 0x7fb3a015a370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:23.766 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:23.762+0000 7fb3a4b14640 1 -- 172.21.15.27:0/2294063717 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb3a00cef40 msgr2=0x7fb3a0157ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:23.766 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:23.762+0000 7fb3a4b14640 1 --2- 172.21.15.27:0/2294063717 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb3a00cef40 0x7fb3a0157ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:23.766 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:23.762+0000 7fb3a4b14640 1 -- 172.21.15.27:0/2294063717 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3a00ae450 con 0x7fb3a006c650 2024-09-12T03:23:23.766 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:23.763+0000 7fb39ffff640 1 --2- 172.21.15.27:0/2294063717 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb3a00cef40 0x7fb3a0157ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:23:23.766 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:23.763+0000 7fb3a4b14640 1 --2- 172.21.15.27:0/2294063717 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb3a006c650 0x7fb3a00aef20 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7fb39000d800 tx=0x7fb39000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:23.766 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:23.763+0000 7fb39d7fa640 1 -- 172.21.15.27:0/2294063717 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb39000b6c0 con 0x7fb3a006c650 2024-09-12T03:23:23.766 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:23.763+0000 7fb39d7fa640 1 -- 172.21.15.27:0/2294063717 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb390019070 con 0x7fb3a006c650 2024-09-12T03:23:23.766 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:23.763+0000 7fb39d7fa640 1 -- 172.21.15.27:0/2294063717 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb39000ba80 con 0x7fb3a006c650 2024-09-12T03:23:23.767 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:23.763+0000 7fb3a6d78640 1 -- 172.21.15.27:0/2294063717 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3a0158a00 con 0x7fb3a006c650 2024-09-12T03:23:23.767 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:23.763+0000 7fb3a6d78640 1 -- 172.21.15.27:0/2294063717 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3a00ae720 con 0x7fb3a006c650 2024-09-12T03:23:23.767 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:23.764+0000 7fb3a6d78640 1 -- 172.21.15.27:0/2294063717 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3a00ce7c0 con 0x7fb3a006c650 2024-09-12T03:23:23.768 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:23.765+0000 7fb39d7fa640 1 -- 172.21.15.27:0/2294063717 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 24) ==== 91212+0+0 (secure 0 0 0) 0x7fb39000bc20 con 0x7fb3a006c650 2024-09-12T03:23:23.768 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:23.765+0000 7fb39d7fa640 1 --2- 172.21.15.27:0/2294063717 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fb38006c610 0x7fb38006ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:23.768 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:23.765+0000 7fb39d7fa640 1 -- 172.21.15.27:0/2294063717 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 5765+0+0 (secure 0 0 0) 0x7fb390089cc0 con 0x7fb3a006c650 2024-09-12T03:23:23.768 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:23.765+0000 7fb39ffff640 1 --2- 172.21.15.27:0/2294063717 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fb38006c610 0x7fb38006ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:23.768 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:23.765+0000 7fb39ffff640 1 --2- 172.21.15.27:0/2294063717 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fb38006c610 0x7fb38006ead0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fb3a00cfa80 tx=0x7fb388009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:23.770 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:23.769+0000 7fb39d7fa640 1 -- 172.21.15.27:0/2294063717 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fb390059f60 con 0x7fb3a006c650 2024-09-12T03:23:23.981 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:23.981+0000 7fb3a6d78640 1 -- 172.21.15.27:0/2294063717 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fb3a006f240 con 0x7fb3a006c650 2024-09-12T03:23:24.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:24.700+0000 7fb39d7fa640 1 -- 172.21.15.27:0/2294063717 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v91) ==== 122+0+0 (secure 0 0 0) 0x7fb390059860 con 0x7fb3a006c650 2024-09-12T03:23:24.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:24.704+0000 7fb3a6d78640 1 -- 172.21.15.27:0/2294063717 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fb38006c610 msgr2=0x7fb38006ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:24.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:24.704+0000 7fb3a6d78640 1 --2- 172.21.15.27:0/2294063717 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fb38006c610 0x7fb38006ead0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fb3a00cfa80 tx=0x7fb388009040 comp rx=0 tx=0).stop 2024-09-12T03:23:24.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:24.704+0000 7fb3a6d78640 1 -- 172.21.15.27:0/2294063717 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb3a006c650 msgr2=0x7fb3a00aef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:24.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:24.704+0000 7fb3a6d78640 1 --2- 172.21.15.27:0/2294063717 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb3a006c650 0x7fb3a00aef20 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7fb39000d800 tx=0x7fb39000dcc0 comp rx=0 tx=0).stop 2024-09-12T03:23:24.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:24.704+0000 7fb3a6d78640 1 -- 172.21.15.27:0/2294063717 shutdown_connections 2024-09-12T03:23:24.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:24.704+0000 7fb3a6d78640 1 --2- 172.21.15.27:0/2294063717 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb3a00cf320 0x7fb3a015a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:24.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:24.704+0000 7fb3a6d78640 1 --2- 172.21.15.27:0/2294063717 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb3a00cef40 0x7fb3a0157ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:24.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:24.704+0000 7fb3a6d78640 1 --2- 172.21.15.27:0/2294063717 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fb38006c610 0x7fb38006ead0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:24.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:24.704+0000 7fb3a6d78640 1 --2- 172.21.15.27:0/2294063717 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb3a006c650 0x7fb3a00aef20 unknown :-1 s=CLOSED pgs=1536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:24.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:24.704+0000 7fb3a6d78640 1 -- 172.21.15.27:0/2294063717 >> 172.21.15.27:0/2294063717 conn(0x7fb3a006bb30 msgr2=0x7fb3a01517c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:24.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:24.704+0000 7fb3a6d78640 1 -- 172.21.15.27:0/2294063717 shutdown_connections 2024-09-12T03:23:24.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:24.704+0000 7fb3a6d78640 1 -- 172.21.15.27:0/2294063717 wait complete. 2024-09-12T03:23:24.719 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-12T03:23:24.719 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-12T03:23:24.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:24.830+0000 7fa4e5111640 1 Processor -- start 2024-09-12T03:23:24.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:24.830+0000 7fa4e5111640 1 -- start start 2024-09-12T03:23:24.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:24.831+0000 7fa4e5111640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fa4e015e5f0 0x7fa4e0073250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:24.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:24.831+0000 7fa4e5111640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa4e0073790 0x7fa4e0162320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:24.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:24.831+0000 7fa4e5111640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa4e0162860 0x7fa4e00d1230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:24.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:24.831+0000 7fa4e5111640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7fa4e00d0da0 con 0x7fa4e015e5f0 2024-09-12T03:23:24.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:24.831+0000 7fa4e5111640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fa4e0073bb0 con 0x7fa4e0162860 2024-09-12T03:23:24.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:24.831+0000 7fa4e5111640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fa4e0163090 con 0x7fa4e0073790 2024-09-12T03:23:24.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:24.831+0000 7fa4ded9d640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fa4e015e5f0 0x7fa4e0073250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:24.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:24.831+0000 7fa4ded9d640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fa4e015e5f0 0x7fa4e0073250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42394/0 (socket says 172.21.15.27:42394) 2024-09-12T03:23:24.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:24.831+0000 7fa4ded9d640 1 -- 172.21.15.27:0/1340803091 learned_addr learned my addr 172.21.15.27:0/1340803091 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:23:24.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:24.831+0000 7fa4de59c640 1 --2- 172.21.15.27:0/1340803091 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa4e0073790 0x7fa4e0162320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:24.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:24.831+0000 7fa4df59e640 1 --2- 172.21.15.27:0/1340803091 >> v2:172.21.15.45:3300/0 conn(0x7fa4e0162860 0x7fa4e00d1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:24.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:24.832+0000 7fa4ded9d640 1 -- 172.21.15.27:0/1340803091 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa4e0073790 msgr2=0x7fa4e0162320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:24.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:24.832+0000 7fa4ded9d640 1 --2- 172.21.15.27:0/1340803091 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa4e0073790 0x7fa4e0162320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:24.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:24.832+0000 7fa4ded9d640 1 -- 172.21.15.27:0/1340803091 >> v2:172.21.15.45:3300/0 conn(0x7fa4e0162860 msgr2=0x7fa4e00d1230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:24.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:24.832+0000 7fa4ded9d640 1 --2- 172.21.15.27:0/1340803091 >> v2:172.21.15.45:3300/0 conn(0x7fa4e0162860 0x7fa4e00d1230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:24.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:24.832+0000 7fa4ded9d640 1 -- 172.21.15.27:0/1340803091 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4e00d0470 con 0x7fa4e015e5f0 2024-09-12T03:23:24.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:24.832+0000 7fa4de59c640 1 --2- 172.21.15.27:0/1340803091 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa4e0073790 0x7fa4e0162320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:23:24.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:24.832+0000 7fa4ded9d640 1 --2- 172.21.15.27:0/1340803091 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa4e015e5f0 0x7fa4e0073250 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7fa4e0072d30 tx=0x7fa4d802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85141accc188b8a9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:24.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:24.832+0000 7fa4ddd9b640 1 -- 172.21.15.27:0/1340803091 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4d8037070 con 0x7fa4e015e5f0 2024-09-12T03:23:24.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:24.833+0000 7fa4ddd9b640 1 -- 172.21.15.27:0/1340803091 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4d8032440 con 0x7fa4e015e5f0 2024-09-12T03:23:24.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:24.833+0000 7fa4ddd9b640 1 -- 172.21.15.27:0/1340803091 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4d8032810 con 0x7fa4e015e5f0 2024-09-12T03:23:24.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:24.833+0000 7fa4e5111640 1 -- 172.21.15.27:0/1340803091 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa4e015e5f0 msgr2=0x7fa4e0073250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:24.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:24.833+0000 7fa4e5111640 1 --2- 172.21.15.27:0/1340803091 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa4e015e5f0 0x7fa4e0073250 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7fa4e0072d30 tx=0x7fa4d802a1e0 comp rx=0 tx=0).stop 2024-09-12T03:23:24.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:24.833+0000 7fa4e5111640 1 -- 172.21.15.27:0/1340803091 shutdown_connections 2024-09-12T03:23:24.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:24.833+0000 7fa4e5111640 1 --2- 172.21.15.27:0/1340803091 >> v2:172.21.15.45:3300/0 conn(0x7fa4e0162860 0x7fa4e00d1230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:24.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:24.833+0000 7fa4e5111640 1 --2- 172.21.15.27:0/1340803091 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa4e0073790 0x7fa4e0162320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:24.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:24.833+0000 7fa4e5111640 1 --2- 172.21.15.27:0/1340803091 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa4e015e5f0 0x7fa4e0073250 unknown :-1 s=CLOSED pgs=1537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:24.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:24.833+0000 7fa4e5111640 1 -- 172.21.15.27:0/1340803091 >> 172.21.15.27:0/1340803091 conn(0x7fa4e006f300 msgr2=0x7fa4e006f710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:24.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:24.833+0000 7fa4e5111640 1 -- 172.21.15.27:0/1340803091 shutdown_connections 2024-09-12T03:23:24.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:24.833+0000 7fa4e5111640 1 -- 172.21.15.27:0/1340803091 wait complete. 2024-09-12T03:23:24.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:24.835+0000 7fa4e5111640 1 Processor -- start 2024-09-12T03:23:24.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:24.835+0000 7fa4e5111640 1 -- start start 2024-09-12T03:23:24.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:24.836+0000 7fa4e5111640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa4e0073790 0x7fa4e0148e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:24.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:24.836+0000 7fa4e5111640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa4e015e5f0 0x7fa4e01f4630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:24.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:24.836+0000 7fa4e5111640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa4e0162860 0x7fa4e015b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:24.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:24.836+0000 7fa4e5111640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fa4e00b5620 con 0x7fa4e015e5f0 2024-09-12T03:23:24.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:24.836+0000 7fa4e5111640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa4e00d0a30 con 0x7fa4e0162860 2024-09-12T03:23:24.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:24.836+0000 7fa4e5111640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fa4e0162cf0 con 0x7fa4e0073790 2024-09-12T03:23:24.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:24.836+0000 7fa4de59c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa4e015e5f0 0x7fa4e01f4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:24.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:24.836+0000 7fa4de59c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa4e015e5f0 0x7fa4e01f4630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42410/0 (socket says 172.21.15.27:42410) 2024-09-12T03:23:24.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:24.836+0000 7fa4de59c640 1 -- 172.21.15.27:0/662145426 learned_addr learned my addr 172.21.15.27:0/662145426 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:23:24.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:24.836+0000 7fa4ded9d640 1 --2- 172.21.15.27:0/662145426 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa4e0073790 0x7fa4e0148e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:24.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:24.836+0000 7fa4df59e640 1 --2- 172.21.15.27:0/662145426 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa4e0162860 0x7fa4e015b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:24.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:24.836+0000 7fa4de59c640 1 -- 172.21.15.27:0/662145426 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa4e0073790 msgr2=0x7fa4e0148e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:24.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:24.836+0000 7fa4de59c640 1 --2- 172.21.15.27:0/662145426 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa4e0073790 0x7fa4e0148e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:24.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:24.836+0000 7fa4de59c640 1 -- 172.21.15.27:0/662145426 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa4e0162860 msgr2=0x7fa4e015b3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:24.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:24.836+0000 7fa4de59c640 1 --2- 172.21.15.27:0/662145426 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa4e0162860 0x7fa4e015b3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:24.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:24.836+0000 7fa4de59c640 1 -- 172.21.15.27:0/662145426 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4e0153a60 con 0x7fa4e015e5f0 2024-09-12T03:23:24.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:24.836+0000 7fa4de59c640 1 --2- 172.21.15.27:0/662145426 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa4e015e5f0 0x7fa4e01f4630 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7fa4c800e7e0 tx=0x7fa4c800eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:24.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:24.836+0000 7fa4d77fe640 1 -- 172.21.15.27:0/662145426 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4c8017070 con 0x7fa4e015e5f0 2024-09-12T03:23:24.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:24.836+0000 7fa4d77fe640 1 -- 172.21.15.27:0/662145426 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4c800c8d0 con 0x7fa4e015e5f0 2024-09-12T03:23:24.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:24.837+0000 7fa4d77fe640 1 -- 172.21.15.27:0/662145426 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4c800f040 con 0x7fa4e015e5f0 2024-09-12T03:23:24.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:24.837+0000 7fa4e5111640 1 -- 172.21.15.27:0/662145426 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4e0153d40 con 0x7fa4e015e5f0 2024-09-12T03:23:24.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:24.837+0000 7fa4e5111640 1 -- 172.21.15.27:0/662145426 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4e0154300 con 0x7fa4e015e5f0 2024-09-12T03:23:24.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:24.838+0000 7fa4e5111640 1 -- 172.21.15.27:0/662145426 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4e0162320 con 0x7fa4e015e5f0 2024-09-12T03:23:24.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:24.838+0000 7fa4d77fe640 1 -- 172.21.15.27:0/662145426 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 24) ==== 91212+0+0 (secure 0 0 0) 0x7fa4c801a070 con 0x7fa4e015e5f0 2024-09-12T03:23:24.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:24.839+0000 7fa4d77fe640 1 --2- 172.21.15.27:0/662145426 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fa4c006c610 0x7fa4c006ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:24.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:24.839+0000 7fa4ded9d640 1 --2- 172.21.15.27:0/662145426 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fa4c006c610 0x7fa4c006ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:24.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:24.839+0000 7fa4d77fe640 1 -- 172.21.15.27:0/662145426 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 5765+0+0 (secure 0 0 0) 0x7fa4c8089370 con 0x7fa4e015e5f0 2024-09-12T03:23:24.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:24.839+0000 7fa4ded9d640 1 --2- 172.21.15.27:0/662145426 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fa4c006c610 0x7fa4c006ead0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fa4d8034000 tx=0x7fa4d802e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:24.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:24.843+0000 7fa4d77fe640 1 -- 172.21.15.27:0/662145426 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fa4c8058fc0 con 0x7fa4e015e5f0 2024-09-12T03:23:25.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.056+0000 7fa4e5111640 1 -- 172.21.15.27:0/662145426 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa4e015f030 con 0x7fa4e015e5f0 2024-09-12T03:23:25.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.057+0000 7fa4d77fe640 1 -- 172.21.15.27:0/662145426 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 91 v91) ==== 94+0+5227 (secure 0 0 0) 0x7fa4c805d2c0 con 0x7fa4e015e5f0 2024-09-12T03:23:25.058 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:23:25.058 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":91,"btime":"2024-09-12T03:23:24:700400+0000","default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6917,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/3682270902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":3682270902},{"type":"v1","addr":"172.21.15.45:6834","nonce":3682270902}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":87},{"gid":7771,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/2399103579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":2399103579},{"type":"v1","addr":"172.21.15.45:6835","nonce":2399103579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":87},{"gid":8559,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6834/1045432254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":1045432254},{"type":"v1","addr":"172.21.15.27:6834","nonce":1045432254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":87}],"filesystems":[{"mdsmap":{"epoch":91,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-12T03:23:21.031667+0000","modified":"2024-09-12T03:23:23.983036+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8562},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8562":{"gid":8562,"name":"c","rank":0,"incarnation":88,"state":"up:active","state_seq":3,"addr":"172.21.15.27:6835/1241148443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":1241148443},{"type":"v1","addr":"172.21.15.27:6835","nonce":1241148443}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[25],"metadata_pool":24,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8562,"qdb_cluster":[8562]},"id":12}]} 2024-09-12T03:23:25.059 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 91 2024-09-12T03:23:25.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.059+0000 7fa4e5111640 1 -- 172.21.15.27:0/662145426 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fa4c006c610 msgr2=0x7fa4c006ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:25.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.059+0000 7fa4e5111640 1 --2- 172.21.15.27:0/662145426 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fa4c006c610 0x7fa4c006ead0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fa4d8034000 tx=0x7fa4d802e000 comp rx=0 tx=0).stop 2024-09-12T03:23:25.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.059+0000 7fa4e5111640 1 -- 172.21.15.27:0/662145426 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa4e015e5f0 msgr2=0x7fa4e01f4630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:25.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.059+0000 7fa4e5111640 1 --2- 172.21.15.27:0/662145426 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa4e015e5f0 0x7fa4e01f4630 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7fa4c800e7e0 tx=0x7fa4c800eca0 comp rx=0 tx=0).stop 2024-09-12T03:23:25.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.059+0000 7fa4e5111640 1 -- 172.21.15.27:0/662145426 shutdown_connections 2024-09-12T03:23:25.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.059+0000 7fa4e5111640 1 --2- 172.21.15.27:0/662145426 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa4e0162860 0x7fa4e015b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:25.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.059+0000 7fa4e5111640 1 --2- 172.21.15.27:0/662145426 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fa4c006c610 0x7fa4c006ead0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:25.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.059+0000 7fa4e5111640 1 --2- 172.21.15.27:0/662145426 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa4e015e5f0 0x7fa4e01f4630 unknown :-1 s=CLOSED pgs=1538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:25.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.059+0000 7fa4e5111640 1 --2- 172.21.15.27:0/662145426 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa4e0073790 0x7fa4e0148e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:25.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.059+0000 7fa4e5111640 1 -- 172.21.15.27:0/662145426 >> 172.21.15.27:0/662145426 conn(0x7fa4e006f300 msgr2=0x7fa4e0159400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:25.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.059+0000 7fa4e5111640 1 -- 172.21.15.27:0/662145426 shutdown_connections 2024-09-12T03:23:25.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.059+0000 7fa4e5111640 1 -- 172.21.15.27:0/662145426 wait complete. 2024-09-12T03:23:25.071 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-12T03:23:25.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.173+0000 7f41d407b640 1 Processor -- start 2024-09-12T03:23:25.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.173+0000 7f41d407b640 1 -- start start 2024-09-12T03:23:25.175 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.174+0000 7f41d407b640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f41cc15e5f0 0x7f41cc073250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:25.175 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.174+0000 7f41d407b640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f41cc162a30 0x7f41cc15e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:25.175 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.174+0000 7f41d407b640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f41cc0d2240 con 0x7f41cc073790 2024-09-12T03:23:25.175 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.174+0000 7f41d407b640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f41cc162e60 con 0x7f41cc15e5f0 2024-09-12T03:23:25.175 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.174+0000 7f41d407b640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f41cc073be0 con 0x7f41cc162a30 2024-09-12T03:23:25.175 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.174+0000 7f41d1e17640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f41cc15e5f0 0x7f41cc073250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:25.175 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.174+0000 7f41d1616640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f41cc073790 0x7f41cc162320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:56106/0 (socket says 172.21.15.27:56106) 2024-09-12T03:23:25.176 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.174+0000 7f41d1616640 1 -- 172.21.15.27:0/1504472629 learned_addr learned my addr 172.21.15.27:0/1504472629 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:23:25.176 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.174+0000 7f41d2618640 1 --2- 172.21.15.27:0/1504472629 >> v2:172.21.15.45:3300/0 conn(0x7f41cc162a30 0x7f41cc15e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:25.176 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.174+0000 7f41d1e17640 1 -- 172.21.15.27:0/1504472629 >> v2:172.21.15.45:3300/0 conn(0x7f41cc162a30 msgr2=0x7f41cc15e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:25.176 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.174+0000 7f41d1e17640 1 --2- 172.21.15.27:0/1504472629 >> v2:172.21.15.45:3300/0 conn(0x7f41cc162a30 0x7f41cc15e9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:25.176 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.174+0000 7f41d1e17640 1 -- 172.21.15.27:0/1504472629 >> v1:172.21.15.27:6789/0 conn(0x7f41cc073790 legacy=0x7f41cc162320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:25.176 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.174+0000 7f41d0e15640 1 -- 172.21.15.27:0/1504472629 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1954776114 0 0) 0x7f41cc0d2240 con 0x7f41cc073790 2024-09-12T03:23:25.176 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.174+0000 7f41d1e17640 1 -- 172.21.15.27:0/1504472629 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41cc163090 con 0x7f41cc15e5f0 2024-09-12T03:23:25.177 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.175+0000 7f41d1e17640 1 --2- 172.21.15.27:0/1504472629 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f41cc15e5f0 0x7f41cc073250 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7f41cc0702a0 tx=0x7f41bc02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=655aadee3338c101 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:25.178 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.175+0000 7f41d0e15640 1 -- 172.21.15.27:0/1504472629 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41bc037070 con 0x7f41cc15e5f0 2024-09-12T03:23:25.178 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.175+0000 7f41d0e15640 1 -- 172.21.15.27:0/1504472629 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41bc032440 con 0x7f41cc15e5f0 2024-09-12T03:23:25.178 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.175+0000 7f41d0e15640 1 -- 172.21.15.27:0/1504472629 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41bc032810 con 0x7f41cc15e5f0 2024-09-12T03:23:25.178 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.176+0000 7f41d407b640 1 -- 172.21.15.27:0/1504472629 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f41cc15e5f0 msgr2=0x7f41cc073250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:25.178 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.176+0000 7f41d407b640 1 --2- 172.21.15.27:0/1504472629 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f41cc15e5f0 0x7f41cc073250 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7f41cc0702a0 tx=0x7f41bc02a1a0 comp rx=0 tx=0).stop 2024-09-12T03:23:25.178 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.176+0000 7f41d407b640 1 -- 172.21.15.27:0/1504472629 shutdown_connections 2024-09-12T03:23:25.178 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.176+0000 7f41d407b640 1 --2- 172.21.15.27:0/1504472629 >> v2:172.21.15.45:3300/0 conn(0x7f41cc162a30 0x7f41cc15e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:25.178 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.176+0000 7f41d407b640 1 --2- 172.21.15.27:0/1504472629 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f41cc15e5f0 0x7f41cc073250 unknown :-1 s=CLOSED pgs=1540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:25.178 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.176+0000 7f41d407b640 1 -- 172.21.15.27:0/1504472629 >> 172.21.15.27:0/1504472629 conn(0x7f41cc06f300 msgr2=0x7f41cc06f710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:25.178 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.176+0000 7f41d407b640 1 -- 172.21.15.27:0/1504472629 shutdown_connections 2024-09-12T03:23:25.178 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.176+0000 7f41d407b640 1 -- 172.21.15.27:0/1504472629 wait complete. 2024-09-12T03:23:25.178 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.177+0000 7f41d407b640 1 Processor -- start 2024-09-12T03:23:25.178 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.177+0000 7f41d407b640 1 -- start start 2024-09-12T03:23:25.180 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.178+0000 7f41d407b640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f41cc073790 0x7f41cc147d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:25.180 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.178+0000 7f41d407b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f41cc15e5f0 0x7f41cc149570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:25.181 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.178+0000 7f41d407b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f41cc162a30 0x7f41cc0b5830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:25.181 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.178+0000 7f41d407b640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f41cc0d0750 con 0x7f41cc15e5f0 2024-09-12T03:23:25.181 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.178+0000 7f41d407b640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f41cc0d0da0 con 0x7f41cc162a30 2024-09-12T03:23:25.181 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.178+0000 7f41d407b640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f41cc0d0a30 con 0x7f41cc073790 2024-09-12T03:23:25.181 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.178+0000 7f41d1616640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f41cc15e5f0 0x7f41cc149570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:25.181 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.178+0000 7f41d1616640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f41cc15e5f0 0x7f41cc149570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42442/0 (socket says 172.21.15.27:42442) 2024-09-12T03:23:25.181 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.178+0000 7f41d1616640 1 -- 172.21.15.27:0/3798266765 learned_addr learned my addr 172.21.15.27:0/3798266765 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:23:25.181 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.178+0000 7f41d1616640 1 -- 172.21.15.27:0/3798266765 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f41cc073790 msgr2=0x7f41cc147d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:25.181 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.178+0000 7f41d1616640 1 --2- 172.21.15.27:0/3798266765 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f41cc073790 0x7f41cc147d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:25.181 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.178+0000 7f41d1616640 1 -- 172.21.15.27:0/3798266765 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f41cc162a30 msgr2=0x7f41cc0b5830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:25.181 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.178+0000 7f41d1616640 1 --2- 172.21.15.27:0/3798266765 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f41cc162a30 0x7f41cc0b5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:25.181 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.178+0000 7f41d1616640 1 -- 172.21.15.27:0/3798266765 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41cc1484d0 con 0x7f41cc15e5f0 2024-09-12T03:23:25.181 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.178+0000 7f41d1616640 1 --2- 172.21.15.27:0/3798266765 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f41cc15e5f0 0x7f41cc149570 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7f41c000c890 tx=0x7f41c000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:25.182 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.178+0000 7f41ca7fc640 1 -- 172.21.15.27:0/3798266765 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41c0018070 con 0x7f41cc15e5f0 2024-09-12T03:23:25.182 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.178+0000 7f41ca7fc640 1 -- 172.21.15.27:0/3798266765 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41c000a640 con 0x7f41cc15e5f0 2024-09-12T03:23:25.182 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.178+0000 7f41d407b640 1 -- 172.21.15.27:0/3798266765 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41cc1487b0 con 0x7f41cc15e5f0 2024-09-12T03:23:25.182 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.178+0000 7f41ca7fc640 1 -- 172.21.15.27:0/3798266765 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41c000aa30 con 0x7f41cc15e5f0 2024-09-12T03:23:25.182 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.178+0000 7f41d407b640 1 -- 172.21.15.27:0/3798266765 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41cc148d70 con 0x7f41cc15e5f0 2024-09-12T03:23:25.182 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.179+0000 7f41d407b640 1 -- 172.21.15.27:0/3798266765 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41cc0b5620 con 0x7f41cc15e5f0 2024-09-12T03:23:25.182 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.180+0000 7f41ca7fc640 1 -- 172.21.15.27:0/3798266765 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 24) ==== 91212+0+0 (secure 0 0 0) 0x7f41c001b070 con 0x7f41cc15e5f0 2024-09-12T03:23:25.183 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.180+0000 7f41ca7fc640 1 --2- 172.21.15.27:0/3798266765 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f41b806c610 0x7f41b806ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:25.183 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.180+0000 7f41ca7fc640 1 -- 172.21.15.27:0/3798266765 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 5765+0+0 (secure 0 0 0) 0x7f41c0088f70 con 0x7f41cc15e5f0 2024-09-12T03:23:25.183 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.180+0000 7f41d1e17640 1 --2- 172.21.15.27:0/3798266765 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f41b806c610 0x7f41b806ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:25.183 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.181+0000 7f41d1e17640 1 --2- 172.21.15.27:0/3798266765 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f41b806c610 0x7f41b806ead0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f41bc034000 tx=0x7f41bc02c000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:25.185 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.185+0000 7f41ca7fc640 1 -- 172.21.15.27:0/3798266765 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f41c0059210 con 0x7f41cc15e5f0 2024-09-12T03:23:25.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.361+0000 7f41d407b640 1 -- 172.21.15.27:0/3798266765 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f41cc15dbe0 con 0x7f41cc15e5f0 2024-09-12T03:23:25.363 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.363+0000 7f41ca7fc640 1 -- 172.21.15.27:0/3798266765 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v149) ==== 74+0+16780 (secure 0 0 0) 0x7f41c0058b10 con 0x7f41cc15e5f0 2024-09-12T03:23:25.364 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:23:25.364 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":149,"fsid":"d2df504e-2647-4385-a5db-9fdf69f487a0","created":"2024-09-12T03:16:07.778937+0000","modified":"2024-09-12T03:23:22.688848+0000","last_up_change":"2024-09-12T03:16:19.314778+0000","last_in_change":"2024-09-12T03:16:09.893086+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T03:16:20.097646+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_metadata","create_time":"2024-09-12T03:23:18.699292+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"148","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_data","create_time":"2024-09-12T03:23:20.022169+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"149","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1851116c-1fb3-4dc9-9772-f90e2ef4c355","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6817","nonce":3002321488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6819","nonce":3002321488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6823","nonce":3002321488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6821","nonce":3002321488}]},"public_addr":"172.21.15.27:6817/3002321488","cluster_addr":"172.21.15.27:6819/3002321488","heartbeat_back_addr":"172.21.15.27:6823/3002321488","heartbeat_front_addr":"172.21.15.27:6821/3002321488","state":["exists","up"]},{"osd":1,"uuid":"f38ae17d-73eb-4204-b99d-975fa806fb87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6809","nonce":1613102886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6811","nonce":1613102886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6815","nonce":1613102886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6813","nonce":1613102886}]},"public_addr":"172.21.15.27:6809/1613102886","cluster_addr":"172.21.15.27:6811/1613102886","heartbeat_back_addr":"172.21.15.27:6815/1613102886","heartbeat_front_addr":"172.21.15.27:6813/1613102886","state":["exists","up"]},{"osd":2,"uuid":"fb12dcc8-bdd0-42b8-b46d-8b10106224bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6825","nonce":3534723355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6827","nonce":3534723355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6831","nonce":3534723355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6829","nonce":3534723355}]},"public_addr":"172.21.15.27:6825/3534723355","cluster_addr":"172.21.15.27:6827/3534723355","heartbeat_back_addr":"172.21.15.27:6831/3534723355","heartbeat_front_addr":"172.21.15.27:6829/3534723355","state":["exists","up"]},{"osd":3,"uuid":"d74d3f49-9c24-4b9f-98e2-3ecd7b5ef3b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6801","nonce":80661598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6803","nonce":80661598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6807","nonce":80661598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6805","nonce":80661598}]},"public_addr":"172.21.15.27:6801/80661598","cluster_addr":"172.21.15.27:6803/80661598","heartbeat_back_addr":"172.21.15.27:6807/80661598","heartbeat_front_addr":"172.21.15.27:6805/80661598","state":["exists","up"]},{"osd":4,"uuid":"2bf123f2-1b0d-41c4-aa3b-805bab267e59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6817","nonce":3943189827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6819","nonce":3943189827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6823","nonce":3943189827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6821","nonce":3943189827}]},"public_addr":"172.21.15.45:6817/3943189827","cluster_addr":"172.21.15.45:6819/3943189827","heartbeat_back_addr":"172.21.15.45:6823/3943189827","heartbeat_front_addr":"172.21.15.45:6821/3943189827","state":["exists","up"]},{"osd":5,"uuid":"c49ba5da-91c6-4a86-a97b-4aba89c253ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6809","nonce":3620760330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6811","nonce":3620760330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6815","nonce":3620760330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6813","nonce":3620760330}]},"public_addr":"172.21.15.45:6809/3620760330","cluster_addr":"172.21.15.45:6811/3620760330","heartbeat_back_addr":"172.21.15.45:6815/3620760330","heartbeat_front_addr":"172.21.15.45:6813/3620760330","state":["exists","up"]},{"osd":6,"uuid":"b0123475-89f7-4a34-9e33-d797b99dd9dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6801","nonce":1989083510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6803","nonce":1989083510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6807","nonce":1989083510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6805","nonce":1989083510}]},"public_addr":"172.21.15.45:6801/1989083510","cluster_addr":"172.21.15.45:6803/1989083510","heartbeat_back_addr":"172.21.15.45:6807/1989083510","heartbeat_front_addr":"172.21.15.45:6805/1989083510","state":["exists","up"]},{"osd":7,"uuid":"4dad773d-b7d8-46fc-8a10-a972aad27b29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6825","nonce":1513300294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6827","nonce":1513300294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6831","nonce":1513300294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6829","nonce":1513300294}]},"public_addr":"172.21.15.45:6825/1513300294","cluster_addr":"172.21.15.45:6827/1513300294","heartbeat_back_addr":"172.21.15.45:6831/1513300294","heartbeat_front_addr":"172.21.15.45:6829/1513300294","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.881855+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:17.133252+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.735339+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.500742+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.477571+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:17.167667+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.470291+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.873743+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]},{"pool":19,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T03:23:25.366 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.365+0000 7f41d407b640 1 -- 172.21.15.27:0/3798266765 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f41b806c610 msgr2=0x7f41b806ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:25.366 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.365+0000 7f41d407b640 1 --2- 172.21.15.27:0/3798266765 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f41b806c610 0x7f41b806ead0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f41bc034000 tx=0x7f41bc02c000 comp rx=0 tx=0).stop 2024-09-12T03:23:25.366 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.365+0000 7f41d407b640 1 -- 172.21.15.27:0/3798266765 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f41cc15e5f0 msgr2=0x7f41cc149570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:25.366 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.365+0000 7f41d407b640 1 --2- 172.21.15.27:0/3798266765 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f41cc15e5f0 0x7f41cc149570 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7f41c000c890 tx=0x7f41c000cd50 comp rx=0 tx=0).stop 2024-09-12T03:23:25.366 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.365+0000 7f41d407b640 1 -- 172.21.15.27:0/3798266765 shutdown_connections 2024-09-12T03:23:25.366 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.365+0000 7f41d407b640 1 --2- 172.21.15.27:0/3798266765 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f41cc162a30 0x7f41cc0b5830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:25.366 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.365+0000 7f41d407b640 1 --2- 172.21.15.27:0/3798266765 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f41b806c610 0x7f41b806ead0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:25.366 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.365+0000 7f41d407b640 1 --2- 172.21.15.27:0/3798266765 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f41cc15e5f0 0x7f41cc149570 unknown :-1 s=CLOSED pgs=1541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:25.367 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.365+0000 7f41d407b640 1 --2- 172.21.15.27:0/3798266765 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f41cc073790 0x7f41cc147d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:25.367 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.365+0000 7f41d407b640 1 -- 172.21.15.27:0/3798266765 >> 172.21.15.27:0/3798266765 conn(0x7f41cc06f300 msgr2=0x7f41cc15fdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:25.367 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.365+0000 7f41d407b640 1 -- 172.21.15.27:0/3798266765 shutdown_connections 2024-09-12T03:23:25.367 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.365+0000 7f41d407b640 1 -- 172.21.15.27:0/3798266765 wait complete. 2024-09-12T03:23:25.377 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-12T03:23:25.377 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-12T03:23:25.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.479+0000 7f0bdc694640 1 Processor -- start 2024-09-12T03:23:25.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.479+0000 7f0bdc694640 1 -- start start 2024-09-12T03:23:25.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.480+0000 7f0bdc694640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f0bd406f0c0 0x7f0bd40b3560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:25.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.480+0000 7f0bdc694640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0bd40ce410 con 0x7f0bd4072d70 2024-09-12T03:23:25.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.480+0000 7f0bdc694640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0bd40732c0 con 0x7f0bd415b2b0 2024-09-12T03:23:25.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.480+0000 7f0bdc694640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f0bd40720a0 con 0x7f0bd406f0c0 2024-09-12T03:23:25.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.480+0000 7f0bdac31640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f0bd406f0c0 0x7f0bd40b3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:25.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.480+0000 7f0bd9c2f640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f0bd4072d70 0x7f0bd406e9b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:56118/0 (socket says 172.21.15.27:56118) 2024-09-12T03:23:25.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.480+0000 7f0bd9c2f640 1 -- 172.21.15.27:0/1375442934 learned_addr learned my addr 172.21.15.27:0/1375442934 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:23:25.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.480+0000 7f0bdac31640 1 -- 172.21.15.27:0/1375442934 >> v1:172.21.15.45:6789/0 conn(0x7f0bd415b2b0 legacy=0x7f0bd4072430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:25.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.480+0000 7f0bd942e640 1 -- 172.21.15.27:0/1375442934 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1942785683 0 0) 0x7f0bc8001110 con 0x7f0bd4072d70 2024-09-12T03:23:25.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.480+0000 7f0bdac31640 1 -- 172.21.15.27:0/1375442934 >> v1:172.21.15.27:6789/0 conn(0x7f0bd4072d70 legacy=0x7f0bd406e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:25.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.480+0000 7f0bdac31640 1 -- 172.21.15.27:0/1375442934 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bd4071ae0 con 0x7f0bd406f0c0 2024-09-12T03:23:25.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.481+0000 7f0bdac31640 1 --2- 172.21.15.27:0/1375442934 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0bd406f0c0 0x7f0bd40b3560 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7f0bcc009d90 tx=0x7f0bcc02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f39e3920e2fc3291 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:25.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.481+0000 7f0bd942e640 1 -- 172.21.15.27:0/1375442934 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bcc007740 con 0x7f0bd406f0c0 2024-09-12T03:23:25.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.481+0000 7f0bd942e640 1 -- 172.21.15.27:0/1375442934 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0bcc0078e0 con 0x7f0bd406f0c0 2024-09-12T03:23:25.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.481+0000 7f0bd942e640 1 -- 172.21.15.27:0/1375442934 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bcc007cb0 con 0x7f0bd406f0c0 2024-09-12T03:23:25.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.482+0000 7f0bdc694640 1 -- 172.21.15.27:0/1375442934 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0bd406f0c0 msgr2=0x7f0bd40b3560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:25.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.482+0000 7f0bdc694640 1 --2- 172.21.15.27:0/1375442934 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0bd406f0c0 0x7f0bd40b3560 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7f0bcc009d90 tx=0x7f0bcc02a6a0 comp rx=0 tx=0).stop 2024-09-12T03:23:25.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.482+0000 7f0bdc694640 1 -- 172.21.15.27:0/1375442934 shutdown_connections 2024-09-12T03:23:25.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.482+0000 7f0bdc694640 1 --2- 172.21.15.27:0/1375442934 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0bd406f0c0 0x7f0bd40b3560 unknown :-1 s=CLOSED pgs=1543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:25.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.482+0000 7f0bdc694640 1 -- 172.21.15.27:0/1375442934 >> 172.21.15.27:0/1375442934 conn(0x7f0bd4062c60 msgr2=0x7f0bd406bb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:25.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.482+0000 7f0bdc694640 1 -- 172.21.15.27:0/1375442934 shutdown_connections 2024-09-12T03:23:25.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.482+0000 7f0bdc694640 1 -- 172.21.15.27:0/1375442934 wait complete. 2024-09-12T03:23:25.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.483+0000 7f0bdc694640 1 Processor -- start 2024-09-12T03:23:25.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.483+0000 7f0bdc694640 1 -- start start 2024-09-12T03:23:25.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.484+0000 7f0bdc694640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f0bd406f0c0 0x7f0bd415e390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:25.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.484+0000 7f0bdc694640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0bd4072d70 0x7f0bd41f5930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:25.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.484+0000 7f0bdc694640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0bd415b2b0 0x7f0bd41f5e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:25.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.484+0000 7f0bdc694640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f0bd415b690 con 0x7f0bd4072d70 2024-09-12T03:23:25.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.484+0000 7f0bdc694640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f0bd40ce690 con 0x7f0bd415b2b0 2024-09-12T03:23:25.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.484+0000 7f0bdc694640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f0bd415b810 con 0x7f0bd406f0c0 2024-09-12T03:23:25.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.484+0000 7f0bd9c2f640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0bd4072d70 0x7f0bd41f5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:25.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.484+0000 7f0bd9c2f640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0bd4072d70 0x7f0bd41f5930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42446/0 (socket says 172.21.15.27:42446) 2024-09-12T03:23:25.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.484+0000 7f0bd9c2f640 1 -- 172.21.15.27:0/373146844 learned_addr learned my addr 172.21.15.27:0/373146844 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:23:25.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.484+0000 7f0bda430640 1 --2- 172.21.15.27:0/373146844 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f0bd406f0c0 0x7f0bd415e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:25.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.484+0000 7f0bdac31640 1 --2- 172.21.15.27:0/373146844 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0bd415b2b0 0x7f0bd41f5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:25.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.484+0000 7f0bd9c2f640 1 -- 172.21.15.27:0/373146844 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f0bd406f0c0 msgr2=0x7f0bd415e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:25.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.484+0000 7f0bd9c2f640 1 --2- 172.21.15.27:0/373146844 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f0bd406f0c0 0x7f0bd415e390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:25.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.484+0000 7f0bd9c2f640 1 -- 172.21.15.27:0/373146844 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0bd415b2b0 msgr2=0x7f0bd41f5e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:25.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.484+0000 7f0bd9c2f640 1 --2- 172.21.15.27:0/373146844 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0bd415b2b0 0x7f0bd41f5e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:25.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.485+0000 7f0bd9c2f640 1 -- 172.21.15.27:0/373146844 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bd4155700 con 0x7f0bd4072d70 2024-09-12T03:23:25.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.485+0000 7f0bd9c2f640 1 --2- 172.21.15.27:0/373146844 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0bd4072d70 0x7f0bd41f5930 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7f0bc800c7b0 tx=0x7f0bc800cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:25.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.485+0000 7f0bc6ffd640 1 -- 172.21.15.27:0/373146844 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bc8018070 con 0x7f0bd4072d70 2024-09-12T03:23:25.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.485+0000 7f0bc6ffd640 1 -- 172.21.15.27:0/373146844 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0bc80072d0 con 0x7f0bd4072d70 2024-09-12T03:23:25.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.485+0000 7f0bc6ffd640 1 -- 172.21.15.27:0/373146844 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bc80047e0 con 0x7f0bd4072d70 2024-09-12T03:23:25.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.485+0000 7f0bdc694640 1 -- 172.21.15.27:0/373146844 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0bd41559e0 con 0x7f0bd4072d70 2024-09-12T03:23:25.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.485+0000 7f0bdc694640 1 -- 172.21.15.27:0/373146844 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0bd4155fa0 con 0x7f0bd4072d70 2024-09-12T03:23:25.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.486+0000 7f0bdc694640 1 -- 172.21.15.27:0/373146844 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0bd40b3560 con 0x7f0bd4072d70 2024-09-12T03:23:25.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.487+0000 7f0bc6ffd640 1 -- 172.21.15.27:0/373146844 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 24) ==== 91212+0+0 (secure 0 0 0) 0x7f0bc801b070 con 0x7f0bd4072d70 2024-09-12T03:23:25.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.487+0000 7f0bc6ffd640 1 --2- 172.21.15.27:0/373146844 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f0bb806c640 0x7f0bb806eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:25.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.487+0000 7f0bc6ffd640 1 -- 172.21.15.27:0/373146844 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 5765+0+0 (secure 0 0 0) 0x7f0bc8088b90 con 0x7f0bd4072d70 2024-09-12T03:23:25.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.487+0000 7f0bda430640 1 --2- 172.21.15.27:0/373146844 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f0bb806c640 0x7f0bb806eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:25.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.488+0000 7f0bda430640 1 --2- 172.21.15.27:0/373146844 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f0bb806c640 0x7f0bb806eb00 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f0bd0006fd0 tx=0x7f0bd0008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:25.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.491+0000 7f0bc6ffd640 1 -- 172.21.15.27:0/373146844 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f0bd40ce410 con 0x7f0bd4072d70 2024-09-12T03:23:25.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.665+0000 7f0bdc694640 1 -- 172.21.15.27:0/373146844 --> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f0bd414e2e0 con 0x7f0bb806c640 2024-09-12T03:23:25.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.671+0000 7f0bc6ffd640 1 -- 172.21.15.27:0/373146844 <== mgr.4113 v2:172.21.15.27:6836/1226920526 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247090 (secure 0 0 0) 0x7f0bd414e2e0 con 0x7f0bb806c640 2024-09-12T03:23:25.675 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:23:25.676 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2024-09-12T03:23:25.678 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.674+0000 7f0bdc694640 1 -- 172.21.15.27:0/373146844 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f0bb806c640 msgr2=0x7f0bb806eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:25.678 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.674+0000 7f0bdc694640 1 --2- 172.21.15.27:0/373146844 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f0bb806c640 0x7f0bb806eb00 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f0bd0006fd0 tx=0x7f0bd0008040 comp rx=0 tx=0).stop 2024-09-12T03:23:25.678 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.674+0000 7f0bdc694640 1 -- 172.21.15.27:0/373146844 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0bd4072d70 msgr2=0x7f0bd41f5930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:25.678 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.674+0000 7f0bdc694640 1 --2- 172.21.15.27:0/373146844 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0bd4072d70 0x7f0bd41f5930 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7f0bc800c7b0 tx=0x7f0bc800cc70 comp rx=0 tx=0).stop 2024-09-12T03:23:25.678 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.674+0000 7f0bdc694640 1 -- 172.21.15.27:0/373146844 shutdown_connections 2024-09-12T03:23:25.678 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.674+0000 7f0bdc694640 1 --2- 172.21.15.27:0/373146844 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0bd415b2b0 0x7f0bd41f5e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:25.678 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.674+0000 7f0bdc694640 1 --2- 172.21.15.27:0/373146844 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f0bb806c640 0x7f0bb806eb00 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:25.678 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.675+0000 7f0bdc694640 1 --2- 172.21.15.27:0/373146844 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0bd4072d70 0x7f0bd41f5930 unknown :-1 s=CLOSED pgs=1544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:25.678 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.675+0000 7f0bdc694640 1 --2- 172.21.15.27:0/373146844 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f0bd406f0c0 0x7f0bd415e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:25.678 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.675+0000 7f0bdc694640 1 -- 172.21.15.27:0/373146844 >> 172.21.15.27:0/373146844 conn(0x7f0bd4062c60 msgr2=0x7f0bd406da70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:25.678 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.675+0000 7f0bdc694640 1 -- 172.21.15.27:0/373146844 shutdown_connections 2024-09-12T03:23:25.679 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.675+0000 7f0bdc694640 1 -- 172.21.15.27:0/373146844 wait complete. 2024-09-12T03:23:25.686 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":178,"stamp":"2024-09-12T03:23:25.134647+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":41,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":281756,"kb_used_data":15588,"kb_used_omap":12,"kb_used_meta":265971,"kb_avail":754692964,"statfs":{"total":773094113280,"available":772805595136,"internally_reserved":0,"allocated":15962112,"data_stored":7700919,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":272354864},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":15,"apply_latency_ms":15,"commit_latency_ns":15000000,"apply_latency_ns":15000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.468635"},"pg_stats":[{"pgid":"25.3f","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.676683+0000","last_change":"2024-09-12T03:23:20.676683+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.676683+0000","last_undegraded":"2024-09-12T03:23:20.676683+0000","last_fullsized":"2024-09-12T03:23:20.676683+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.3e","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.677002+0000","last_change":"2024-09-12T03:23:20.677002+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.677002+0000","last_undegraded":"2024-09-12T03:23:20.677002+0000","last_fullsized":"2024-09-12T03:23:20.677002+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.3d","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.680808+0000","last_change":"2024-09-12T03:23:20.680808+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.680808+0000","last_undegraded":"2024-09-12T03:23:20.680808+0000","last_fullsized":"2024-09-12T03:23:20.680808+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.3c","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.677777+0000","last_change":"2024-09-12T03:23:20.677777+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.677777+0000","last_undegraded":"2024-09-12T03:23:20.677777+0000","last_fullsized":"2024-09-12T03:23:20.677777+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.3b","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.683306+0000","last_change":"2024-09-12T03:23:20.683306+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.683306+0000","last_undegraded":"2024-09-12T03:23:20.683306+0000","last_fullsized":"2024-09-12T03:23:20.683306+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.3a","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.680330+0000","last_change":"2024-09-12T03:23:20.680330+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.680330+0000","last_undegraded":"2024-09-12T03:23:20.680330+0000","last_fullsized":"2024-09-12T03:23:20.680330+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.39","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.672348+0000","last_change":"2024-09-12T03:23:20.672348+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.672348+0000","last_undegraded":"2024-09-12T03:23:20.672348+0000","last_fullsized":"2024-09-12T03:23:20.672348+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.38","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.676156+0000","last_change":"2024-09-12T03:23:20.676156+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.676156+0000","last_undegraded":"2024-09-12T03:23:20.676156+0000","last_fullsized":"2024-09-12T03:23:20.676156+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.37","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.674337+0000","last_change":"2024-09-12T03:23:20.674337+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.674337+0000","last_undegraded":"2024-09-12T03:23:20.674337+0000","last_fullsized":"2024-09-12T03:23:20.674337+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.36","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.679738+0000","last_change":"2024-09-12T03:23:20.679738+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.679738+0000","last_undegraded":"2024-09-12T03:23:20.679738+0000","last_fullsized":"2024-09-12T03:23:20.679738+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.35","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.671126+0000","last_change":"2024-09-12T03:23:20.671126+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.671126+0000","last_undegraded":"2024-09-12T03:23:20.671126+0000","last_fullsized":"2024-09-12T03:23:20.671126+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.34","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.677214+0000","last_change":"2024-09-12T03:23:20.677214+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.677214+0000","last_undegraded":"2024-09-12T03:23:20.677214+0000","last_fullsized":"2024-09-12T03:23:20.677214+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.33","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.677238+0000","last_change":"2024-09-12T03:23:20.677238+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.677238+0000","last_undegraded":"2024-09-12T03:23:20.677238+0000","last_fullsized":"2024-09-12T03:23:20.677238+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.32","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.675036+0000","last_change":"2024-09-12T03:23:20.675036+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.675036+0000","last_undegraded":"2024-09-12T03:23:20.675036+0000","last_fullsized":"2024-09-12T03:23:20.675036+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.31","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.675841+0000","last_change":"2024-09-12T03:23:20.675841+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.675841+0000","last_undegraded":"2024-09-12T03:23:20.675841+0000","last_fullsized":"2024-09-12T03:23:20.675841+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.30","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.675750+0000","last_change":"2024-09-12T03:23:20.675750+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.675750+0000","last_undegraded":"2024-09-12T03:23:20.675750+0000","last_fullsized":"2024-09-12T03:23:20.675750+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.2f","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.687993+0000","last_change":"2024-09-12T03:23:20.687993+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.687993+0000","last_undegraded":"2024-09-12T03:23:20.687993+0000","last_fullsized":"2024-09-12T03:23:20.687993+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.2e","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.676010+0000","last_change":"2024-09-12T03:23:20.676010+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.676010+0000","last_undegraded":"2024-09-12T03:23:20.676010+0000","last_fullsized":"2024-09-12T03:23:20.676010+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.2d","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.680215+0000","last_change":"2024-09-12T03:23:20.680215+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.680215+0000","last_undegraded":"2024-09-12T03:23:20.680215+0000","last_fullsized":"2024-09-12T03:23:20.680215+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.2c","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.685697+0000","last_change":"2024-09-12T03:23:20.685697+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.685697+0000","last_undegraded":"2024-09-12T03:23:20.685697+0000","last_fullsized":"2024-09-12T03:23:20.685697+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.2b","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.682781+0000","last_change":"2024-09-12T03:23:20.682781+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.682781+0000","last_undegraded":"2024-09-12T03:23:20.682781+0000","last_fullsized":"2024-09-12T03:23:20.682781+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.2a","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.679222+0000","last_change":"2024-09-12T03:23:20.679222+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.679222+0000","last_undegraded":"2024-09-12T03:23:20.679222+0000","last_fullsized":"2024-09-12T03:23:20.679222+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.29","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.679592+0000","last_change":"2024-09-12T03:23:20.679592+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.679592+0000","last_undegraded":"2024-09-12T03:23:20.679592+0000","last_fullsized":"2024-09-12T03:23:20.679592+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.28","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.675319+0000","last_change":"2024-09-12T03:23:20.675319+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.675319+0000","last_undegraded":"2024-09-12T03:23:20.675319+0000","last_fullsized":"2024-09-12T03:23:20.675319+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.27","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.676115+0000","last_change":"2024-09-12T03:23:20.676115+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.676115+0000","last_undegraded":"2024-09-12T03:23:20.676115+0000","last_fullsized":"2024-09-12T03:23:20.676115+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.26","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.671542+0000","last_change":"2024-09-12T03:23:20.671542+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.671542+0000","last_undegraded":"2024-09-12T03:23:20.671542+0000","last_fullsized":"2024-09-12T03:23:20.671542+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.25","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.676098+0000","last_change":"2024-09-12T03:23:20.676098+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.676098+0000","last_undegraded":"2024-09-12T03:23:20.676098+0000","last_fullsized":"2024-09-12T03:23:20.676098+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.24","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.685181+0000","last_change":"2024-09-12T03:23:20.685181+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.685181+0000","last_undegraded":"2024-09-12T03:23:20.685181+0000","last_fullsized":"2024-09-12T03:23:20.685181+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.23","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.676786+0000","last_change":"2024-09-12T03:23:20.676786+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.676786+0000","last_undegraded":"2024-09-12T03:23:20.676786+0000","last_fullsized":"2024-09-12T03:23:20.676786+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.22","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.674611+0000","last_change":"2024-09-12T03:23:20.674611+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.674611+0000","last_undegraded":"2024-09-12T03:23:20.674611+0000","last_fullsized":"2024-09-12T03:23:20.674611+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.21","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.676607+0000","last_change":"2024-09-12T03:23:20.676607+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.676607+0000","last_undegraded":"2024-09-12T03:23:20.676607+0000","last_fullsized":"2024-09-12T03:23:20.676607+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.20","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.682292+0000","last_change":"2024-09-12T03:23:20.682292+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.682292+0000","last_undegraded":"2024-09-12T03:23:20.682292+0000","last_fullsized":"2024-09-12T03:23:20.682292+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1e","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.678929+0000","last_change":"2024-09-12T03:23:20.678929+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.678929+0000","last_undegraded":"2024-09-12T03:23:20.678929+0000","last_fullsized":"2024-09-12T03:23:20.678929+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1f","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-12T03:23:20.688172+0000","last_change":"2024-09-12T03:23:20.682141+0000","last_active":"2024-09-12T03:23:20.688172+0000","last_peered":"2024-09-12T03:23:20.688172+0000","last_clean":"2024-09-12T03:23:20.688172+0000","last_became_active":"2024-09-12T03:23:20.681852+0000","last_became_peered":"2024-09-12T03:23:20.681852+0000","last_unstale":"2024-09-12T03:23:20.688172+0000","last_undegraded":"2024-09-12T03:23:20.688172+0000","last_fullsized":"2024-09-12T03:23:20.688172+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_clean_scrub_stamp":"2024-09-12T03:23:19.659665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:20:17.004278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1f","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.678598+0000","last_change":"2024-09-12T03:23:20.678598+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.678598+0000","last_undegraded":"2024-09-12T03:23:20.678598+0000","last_fullsized":"2024-09-12T03:23:20.678598+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.1e","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-12T03:23:20.687095+0000","last_change":"2024-09-12T03:23:20.686262+0000","last_active":"2024-09-12T03:23:20.687095+0000","last_peered":"2024-09-12T03:23:20.687095+0000","last_clean":"2024-09-12T03:23:20.687095+0000","last_became_active":"2024-09-12T03:23:20.685552+0000","last_became_peered":"2024-09-12T03:23:20.685552+0000","last_unstale":"2024-09-12T03:23:20.687095+0000","last_undegraded":"2024-09-12T03:23:20.687095+0000","last_fullsized":"2024-09-12T03:23:20.687095+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_clean_scrub_stamp":"2024-09-12T03:23:19.659665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:20:51.463260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.1c","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.672426+0000","last_change":"2024-09-12T03:23:20.672426+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.672426+0000","last_undegraded":"2024-09-12T03:23:20.672426+0000","last_fullsized":"2024-09-12T03:23:20.672426+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.1d","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-12T03:23:20.695515+0000","last_change":"2024-09-12T03:23:20.690566+0000","last_active":"2024-09-12T03:23:20.695515+0000","last_peered":"2024-09-12T03:23:20.695515+0000","last_clean":"2024-09-12T03:23:20.695515+0000","last_became_active":"2024-09-12T03:23:20.690296+0000","last_became_peered":"2024-09-12T03:23:20.690296+0000","last_unstale":"2024-09-12T03:23:20.695515+0000","last_undegraded":"2024-09-12T03:23:20.695515+0000","last_fullsized":"2024-09-12T03:23:20.695515+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_clean_scrub_stamp":"2024-09-12T03:23:19.659665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:13:40.613414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1d","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.672931+0000","last_change":"2024-09-12T03:23:20.672931+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.672931+0000","last_undegraded":"2024-09-12T03:23:20.672931+0000","last_fullsized":"2024-09-12T03:23:20.672931+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.1c","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-12T03:23:20.687327+0000","last_change":"2024-09-12T03:23:20.686667+0000","last_active":"2024-09-12T03:23:20.687327+0000","last_peered":"2024-09-12T03:23:20.687327+0000","last_clean":"2024-09-12T03:23:20.687327+0000","last_became_active":"2024-09-12T03:23:20.686355+0000","last_became_peered":"2024-09-12T03:23:20.686355+0000","last_unstale":"2024-09-12T03:23:20.687327+0000","last_undegraded":"2024-09-12T03:23:20.687327+0000","last_fullsized":"2024-09-12T03:23:20.687327+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_clean_scrub_stamp":"2024-09-12T03:23:19.659665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:01:34.049378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.1a","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.678590+0000","last_change":"2024-09-12T03:23:20.678590+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.678590+0000","last_undegraded":"2024-09-12T03:23:20.678590+0000","last_fullsized":"2024-09-12T03:23:20.678590+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.1b","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-12T03:23:20.686092+0000","last_change":"2024-09-12T03:23:20.684507+0000","last_active":"2024-09-12T03:23:20.686092+0000","last_peered":"2024-09-12T03:23:20.686092+0000","last_clean":"2024-09-12T03:23:20.686092+0000","last_became_active":"2024-09-12T03:23:20.684246+0000","last_became_peered":"2024-09-12T03:23:20.684246+0000","last_unstale":"2024-09-12T03:23:20.686092+0000","last_undegraded":"2024-09-12T03:23:20.686092+0000","last_fullsized":"2024-09-12T03:23:20.686092+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_clean_scrub_stamp":"2024-09-12T03:23:19.659665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:39:13.426726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1b","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.674380+0000","last_change":"2024-09-12T03:23:20.674380+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.674380+0000","last_undegraded":"2024-09-12T03:23:20.674380+0000","last_fullsized":"2024-09-12T03:23:20.674380+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1a","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-12T03:23:20.697734+0000","last_change":"2024-09-12T03:23:20.696117+0000","last_active":"2024-09-12T03:23:20.697734+0000","last_peered":"2024-09-12T03:23:20.697734+0000","last_clean":"2024-09-12T03:23:20.697734+0000","last_became_active":"2024-09-12T03:23:20.695795+0000","last_became_peered":"2024-09-12T03:23:20.695795+0000","last_unstale":"2024-09-12T03:23:20.697734+0000","last_undegraded":"2024-09-12T03:23:20.697734+0000","last_fullsized":"2024-09-12T03:23:20.697734+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_clean_scrub_stamp":"2024-09-12T03:23:19.659665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:36:38.656553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.19","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.671879+0000","last_change":"2024-09-12T03:23:20.671879+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.671879+0000","last_undegraded":"2024-09-12T03:23:20.671879+0000","last_fullsized":"2024-09-12T03:23:20.671879+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.18","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-12T03:23:20.690256+0000","last_change":"2024-09-12T03:23:20.689802+0000","last_active":"2024-09-12T03:23:20.690256+0000","last_peered":"2024-09-12T03:23:20.690256+0000","last_clean":"2024-09-12T03:23:20.690256+0000","last_became_active":"2024-09-12T03:23:20.689503+0000","last_became_peered":"2024-09-12T03:23:20.689503+0000","last_unstale":"2024-09-12T03:23:20.690256+0000","last_undegraded":"2024-09-12T03:23:20.690256+0000","last_fullsized":"2024-09-12T03:23:20.690256+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_clean_scrub_stamp":"2024-09-12T03:23:19.659665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:29:59.930252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.16","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.675332+0000","last_change":"2024-09-12T03:23:20.675332+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.675332+0000","last_undegraded":"2024-09-12T03:23:20.675332+0000","last_fullsized":"2024-09-12T03:23:20.675332+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.17","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-12T03:23:20.679242+0000","last_change":"2024-09-12T03:23:20.678720+0000","last_active":"2024-09-12T03:23:20.679242+0000","last_peered":"2024-09-12T03:23:20.679242+0000","last_clean":"2024-09-12T03:23:20.679242+0000","last_became_active":"2024-09-12T03:23:20.678415+0000","last_became_peered":"2024-09-12T03:23:20.678415+0000","last_unstale":"2024-09-12T03:23:20.679242+0000","last_undegraded":"2024-09-12T03:23:20.679242+0000","last_fullsized":"2024-09-12T03:23:20.679242+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_clean_scrub_stamp":"2024-09-12T03:23:19.659665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:22:49.298961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.17","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.674869+0000","last_change":"2024-09-12T03:23:20.674869+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.674869+0000","last_undegraded":"2024-09-12T03:23:20.674869+0000","last_fullsized":"2024-09-12T03:23:20.674869+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.16","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-12T03:23:20.686685+0000","last_change":"2024-09-12T03:23:20.683066+0000","last_active":"2024-09-12T03:23:20.686685+0000","last_peered":"2024-09-12T03:23:20.686685+0000","last_clean":"2024-09-12T03:23:20.686685+0000","last_became_active":"2024-09-12T03:23:20.682759+0000","last_became_peered":"2024-09-12T03:23:20.682759+0000","last_unstale":"2024-09-12T03:23:20.686685+0000","last_undegraded":"2024-09-12T03:23:20.686685+0000","last_fullsized":"2024-09-12T03:23:20.686685+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_clean_scrub_stamp":"2024-09-12T03:23:19.659665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:51:25.370412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.14","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.671943+0000","last_change":"2024-09-12T03:23:20.671943+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.671943+0000","last_undegraded":"2024-09-12T03:23:20.671943+0000","last_fullsized":"2024-09-12T03:23:20.671943+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.15","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-12T03:23:20.693567+0000","last_change":"2024-09-12T03:23:20.691649+0000","last_active":"2024-09-12T03:23:20.693567+0000","last_peered":"2024-09-12T03:23:20.693567+0000","last_clean":"2024-09-12T03:23:20.693567+0000","last_became_active":"2024-09-12T03:23:20.691385+0000","last_became_peered":"2024-09-12T03:23:20.691385+0000","last_unstale":"2024-09-12T03:23:20.693567+0000","last_undegraded":"2024-09-12T03:23:20.693567+0000","last_fullsized":"2024-09-12T03:23:20.693567+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_clean_scrub_stamp":"2024-09-12T03:23:19.659665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:51:15.001810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.15","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.677434+0000","last_change":"2024-09-12T03:23:20.677434+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.677434+0000","last_undegraded":"2024-09-12T03:23:20.677434+0000","last_fullsized":"2024-09-12T03:23:20.677434+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.14","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-12T03:23:20.684819+0000","last_change":"2024-09-12T03:23:20.683263+0000","last_active":"2024-09-12T03:23:20.684819+0000","last_peered":"2024-09-12T03:23:20.684819+0000","last_clean":"2024-09-12T03:23:20.684819+0000","last_became_active":"2024-09-12T03:23:20.682986+0000","last_became_peered":"2024-09-12T03:23:20.682986+0000","last_unstale":"2024-09-12T03:23:20.684819+0000","last_undegraded":"2024-09-12T03:23:20.684819+0000","last_fullsized":"2024-09-12T03:23:20.684819+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_clean_scrub_stamp":"2024-09-12T03:23:19.659665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:17:57.817757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.12","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.676361+0000","last_change":"2024-09-12T03:23:20.676361+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.676361+0000","last_undegraded":"2024-09-12T03:23:20.676361+0000","last_fullsized":"2024-09-12T03:23:20.676361+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.13","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-12T03:23:20.690520+0000","last_change":"2024-09-12T03:23:20.689845+0000","last_active":"2024-09-12T03:23:20.690520+0000","last_peered":"2024-09-12T03:23:20.690520+0000","last_clean":"2024-09-12T03:23:20.690520+0000","last_became_active":"2024-09-12T03:23:20.689485+0000","last_became_peered":"2024-09-12T03:23:20.689485+0000","last_unstale":"2024-09-12T03:23:20.690520+0000","last_undegraded":"2024-09-12T03:23:20.690520+0000","last_fullsized":"2024-09-12T03:23:20.690520+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_clean_scrub_stamp":"2024-09-12T03:23:19.659665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:41:28.835418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.13","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.672076+0000","last_change":"2024-09-12T03:23:20.672076+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.672076+0000","last_undegraded":"2024-09-12T03:23:20.672076+0000","last_fullsized":"2024-09-12T03:23:20.672076+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.12","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-12T03:23:20.692153+0000","last_change":"2024-09-12T03:23:20.689243+0000","last_active":"2024-09-12T03:23:20.692153+0000","last_peered":"2024-09-12T03:23:20.692153+0000","last_clean":"2024-09-12T03:23:20.692153+0000","last_became_active":"2024-09-12T03:23:20.688661+0000","last_became_peered":"2024-09-12T03:23:20.688661+0000","last_unstale":"2024-09-12T03:23:20.692153+0000","last_undegraded":"2024-09-12T03:23:20.692153+0000","last_fullsized":"2024-09-12T03:23:20.692153+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_clean_scrub_stamp":"2024-09-12T03:23:19.659665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:16:21.498607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.10","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.672680+0000","last_change":"2024-09-12T03:23:20.672680+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.672680+0000","last_undegraded":"2024-09-12T03:23:20.672680+0000","last_fullsized":"2024-09-12T03:23:20.672680+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.11","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-12T03:23:20.687225+0000","last_change":"2024-09-12T03:23:20.680790+0000","last_active":"2024-09-12T03:23:20.687225+0000","last_peered":"2024-09-12T03:23:20.687225+0000","last_clean":"2024-09-12T03:23:20.687225+0000","last_became_active":"2024-09-12T03:23:20.680508+0000","last_became_peered":"2024-09-12T03:23:20.680508+0000","last_unstale":"2024-09-12T03:23:20.687225+0000","last_undegraded":"2024-09-12T03:23:20.687225+0000","last_fullsized":"2024-09-12T03:23:20.687225+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_clean_scrub_stamp":"2024-09-12T03:23:19.659665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:56:34.246022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.11","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.676970+0000","last_change":"2024-09-12T03:23:20.676970+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.676970+0000","last_undegraded":"2024-09-12T03:23:20.676970+0000","last_fullsized":"2024-09-12T03:23:20.676970+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.10","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-12T03:23:20.690498+0000","last_change":"2024-09-12T03:23:20.689843+0000","last_active":"2024-09-12T03:23:20.690498+0000","last_peered":"2024-09-12T03:23:20.690498+0000","last_clean":"2024-09-12T03:23:20.690498+0000","last_became_active":"2024-09-12T03:23:20.689517+0000","last_became_peered":"2024-09-12T03:23:20.689517+0000","last_unstale":"2024-09-12T03:23:20.690498+0000","last_undegraded":"2024-09-12T03:23:20.690498+0000","last_fullsized":"2024-09-12T03:23:20.690498+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_clean_scrub_stamp":"2024-09-12T03:23:19.659665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:27:53.769921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.e","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.678489+0000","last_change":"2024-09-12T03:23:20.678489+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.678489+0000","last_undegraded":"2024-09-12T03:23:20.678489+0000","last_fullsized":"2024-09-12T03:23:20.678489+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.f","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-12T03:23:20.692368+0000","last_change":"2024-09-12T03:23:20.689669+0000","last_active":"2024-09-12T03:23:20.692368+0000","last_peered":"2024-09-12T03:23:20.692368+0000","last_clean":"2024-09-12T03:23:20.692368+0000","last_became_active":"2024-09-12T03:23:20.689291+0000","last_became_peered":"2024-09-12T03:23:20.689291+0000","last_unstale":"2024-09-12T03:23:20.692368+0000","last_undegraded":"2024-09-12T03:23:20.692368+0000","last_fullsized":"2024-09-12T03:23:20.692368+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_clean_scrub_stamp":"2024-09-12T03:23:19.659665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:25:29.729322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.f","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.673914+0000","last_change":"2024-09-12T03:23:20.673914+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.673914+0000","last_undegraded":"2024-09-12T03:23:20.673914+0000","last_fullsized":"2024-09-12T03:23:20.673914+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.e","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-12T03:23:20.692311+0000","last_change":"2024-09-12T03:23:20.691112+0000","last_active":"2024-09-12T03:23:20.692311+0000","last_peered":"2024-09-12T03:23:20.692311+0000","last_clean":"2024-09-12T03:23:20.692311+0000","last_became_active":"2024-09-12T03:23:20.690862+0000","last_became_peered":"2024-09-12T03:23:20.690862+0000","last_unstale":"2024-09-12T03:23:20.692311+0000","last_undegraded":"2024-09-12T03:23:20.692311+0000","last_fullsized":"2024-09-12T03:23:20.692311+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_clean_scrub_stamp":"2024-09-12T03:23:19.659665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:42:16.029521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.c","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.672469+0000","last_change":"2024-09-12T03:23:20.672469+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.672469+0000","last_undegraded":"2024-09-12T03:23:20.672469+0000","last_fullsized":"2024-09-12T03:23:20.672469+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.d","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-12T03:23:20.688563+0000","last_change":"2024-09-12T03:23:20.688103+0000","last_active":"2024-09-12T03:23:20.688563+0000","last_peered":"2024-09-12T03:23:20.688563+0000","last_clean":"2024-09-12T03:23:20.688563+0000","last_became_active":"2024-09-12T03:23:20.687797+0000","last_became_peered":"2024-09-12T03:23:20.687797+0000","last_unstale":"2024-09-12T03:23:20.688563+0000","last_undegraded":"2024-09-12T03:23:20.688563+0000","last_fullsized":"2024-09-12T03:23:20.688563+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_clean_scrub_stamp":"2024-09-12T03:23:19.659665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:34:39.656922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.d","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.677217+0000","last_change":"2024-09-12T03:23:20.677217+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.677217+0000","last_undegraded":"2024-09-12T03:23:20.677217+0000","last_fullsized":"2024-09-12T03:23:20.677217+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.c","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-12T03:23:20.684552+0000","last_change":"2024-09-12T03:23:20.682322+0000","last_active":"2024-09-12T03:23:20.684552+0000","last_peered":"2024-09-12T03:23:20.684552+0000","last_clean":"2024-09-12T03:23:20.684552+0000","last_became_active":"2024-09-12T03:23:20.682033+0000","last_became_peered":"2024-09-12T03:23:20.682033+0000","last_unstale":"2024-09-12T03:23:20.684552+0000","last_undegraded":"2024-09-12T03:23:20.684552+0000","last_fullsized":"2024-09-12T03:23:20.684552+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_clean_scrub_stamp":"2024-09-12T03:23:19.659665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:42:18.081947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.a","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.676951+0000","last_change":"2024-09-12T03:23:20.676951+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.676951+0000","last_undegraded":"2024-09-12T03:23:20.676951+0000","last_fullsized":"2024-09-12T03:23:20.676951+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.b","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-12T03:23:20.690808+0000","last_change":"2024-09-12T03:23:20.688832+0000","last_active":"2024-09-12T03:23:20.690808+0000","last_peered":"2024-09-12T03:23:20.690808+0000","last_clean":"2024-09-12T03:23:20.690808+0000","last_became_active":"2024-09-12T03:23:20.688021+0000","last_became_peered":"2024-09-12T03:23:20.688021+0000","last_unstale":"2024-09-12T03:23:20.690808+0000","last_undegraded":"2024-09-12T03:23:20.690808+0000","last_fullsized":"2024-09-12T03:23:20.690808+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_clean_scrub_stamp":"2024-09-12T03:23:19.659665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:11:32.173284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.b","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.671109+0000","last_change":"2024-09-12T03:23:20.671109+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.671109+0000","last_undegraded":"2024-09-12T03:23:20.671109+0000","last_fullsized":"2024-09-12T03:23:20.671109+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.a","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-12T03:23:20.686788+0000","last_change":"2024-09-12T03:23:20.683771+0000","last_active":"2024-09-12T03:23:20.686788+0000","last_peered":"2024-09-12T03:23:20.686788+0000","last_clean":"2024-09-12T03:23:20.686788+0000","last_became_active":"2024-09-12T03:23:20.683511+0000","last_became_peered":"2024-09-12T03:23:20.683511+0000","last_unstale":"2024-09-12T03:23:20.686788+0000","last_undegraded":"2024-09-12T03:23:20.686788+0000","last_fullsized":"2024-09-12T03:23:20.686788+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_clean_scrub_stamp":"2024-09-12T03:23:19.659665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:28:42.183834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.8","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.675851+0000","last_change":"2024-09-12T03:23:20.675851+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.675851+0000","last_undegraded":"2024-09-12T03:23:20.675851+0000","last_fullsized":"2024-09-12T03:23:20.675851+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.9","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-12T03:23:20.695616+0000","last_change":"2024-09-12T03:23:20.692711+0000","last_active":"2024-09-12T03:23:20.695616+0000","last_peered":"2024-09-12T03:23:20.695616+0000","last_clean":"2024-09-12T03:23:20.695616+0000","last_became_active":"2024-09-12T03:23:20.692455+0000","last_became_peered":"2024-09-12T03:23:20.692455+0000","last_unstale":"2024-09-12T03:23:20.695616+0000","last_undegraded":"2024-09-12T03:23:20.695616+0000","last_fullsized":"2024-09-12T03:23:20.695616+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_clean_scrub_stamp":"2024-09-12T03:23:19.659665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:47:47.363872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.9","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.681456+0000","last_change":"2024-09-12T03:23:20.681456+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.681456+0000","last_undegraded":"2024-09-12T03:23:20.681456+0000","last_fullsized":"2024-09-12T03:23:20.681456+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.8","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-12T03:23:20.684083+0000","last_change":"2024-09-12T03:23:20.679394+0000","last_active":"2024-09-12T03:23:20.684083+0000","last_peered":"2024-09-12T03:23:20.684083+0000","last_clean":"2024-09-12T03:23:20.684083+0000","last_became_active":"2024-09-12T03:23:20.679100+0000","last_became_peered":"2024-09-12T03:23:20.679100+0000","last_unstale":"2024-09-12T03:23:20.684083+0000","last_undegraded":"2024-09-12T03:23:20.684083+0000","last_fullsized":"2024-09-12T03:23:20.684083+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_clean_scrub_stamp":"2024-09-12T03:23:19.659665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:49:36.375371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.6","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.681144+0000","last_change":"2024-09-12T03:23:20.681144+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.681144+0000","last_undegraded":"2024-09-12T03:23:20.681144+0000","last_fullsized":"2024-09-12T03:23:20.681144+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.7","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-12T03:23:20.693562+0000","last_change":"2024-09-12T03:23:20.693222+0000","last_active":"2024-09-12T03:23:20.693562+0000","last_peered":"2024-09-12T03:23:20.693562+0000","last_clean":"2024-09-12T03:23:20.693562+0000","last_became_active":"2024-09-12T03:23:20.692889+0000","last_became_peered":"2024-09-12T03:23:20.692889+0000","last_unstale":"2024-09-12T03:23:20.693562+0000","last_undegraded":"2024-09-12T03:23:20.693562+0000","last_fullsized":"2024-09-12T03:23:20.693562+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_clean_scrub_stamp":"2024-09-12T03:23:19.659665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:08:14.319966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.7","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.676386+0000","last_change":"2024-09-12T03:23:20.676386+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.676386+0000","last_undegraded":"2024-09-12T03:23:20.676386+0000","last_fullsized":"2024-09-12T03:23:20.676386+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.6","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-12T03:23:20.688527+0000","last_change":"2024-09-12T03:23:20.687884+0000","last_active":"2024-09-12T03:23:20.688527+0000","last_peered":"2024-09-12T03:23:20.688527+0000","last_clean":"2024-09-12T03:23:20.688527+0000","last_became_active":"2024-09-12T03:23:20.687621+0000","last_became_peered":"2024-09-12T03:23:20.687621+0000","last_unstale":"2024-09-12T03:23:20.688527+0000","last_undegraded":"2024-09-12T03:23:20.688527+0000","last_fullsized":"2024-09-12T03:23:20.688527+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_clean_scrub_stamp":"2024-09-12T03:23:19.659665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:34:47.119838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.4","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.677605+0000","last_change":"2024-09-12T03:23:20.677605+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.677605+0000","last_undegraded":"2024-09-12T03:23:20.677605+0000","last_fullsized":"2024-09-12T03:23:20.677605+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.5","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-12T03:23:20.688498+0000","last_change":"2024-09-12T03:23:20.686431+0000","last_active":"2024-09-12T03:23:20.688498+0000","last_peered":"2024-09-12T03:23:20.688498+0000","last_clean":"2024-09-12T03:23:20.688498+0000","last_became_active":"2024-09-12T03:23:20.686145+0000","last_became_peered":"2024-09-12T03:23:20.686145+0000","last_unstale":"2024-09-12T03:23:20.688498+0000","last_undegraded":"2024-09-12T03:23:20.688498+0000","last_fullsized":"2024-09-12T03:23:20.688498+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_clean_scrub_stamp":"2024-09-12T03:23:19.659665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:14:07.788062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.5","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.677039+0000","last_change":"2024-09-12T03:23:20.677039+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.677039+0000","last_undegraded":"2024-09-12T03:23:20.677039+0000","last_fullsized":"2024-09-12T03:23:20.677039+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.4","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-12T03:23:20.691435+0000","last_change":"2024-09-12T03:23:20.689734+0000","last_active":"2024-09-12T03:23:20.691435+0000","last_peered":"2024-09-12T03:23:20.691435+0000","last_clean":"2024-09-12T03:23:20.691435+0000","last_became_active":"2024-09-12T03:23:20.689439+0000","last_became_peered":"2024-09-12T03:23:20.689439+0000","last_unstale":"2024-09-12T03:23:20.691435+0000","last_undegraded":"2024-09-12T03:23:20.691435+0000","last_fullsized":"2024-09-12T03:23:20.691435+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_clean_scrub_stamp":"2024-09-12T03:23:19.659665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:47:48.015298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.2","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.675165+0000","last_change":"2024-09-12T03:23:20.675165+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.675165+0000","last_undegraded":"2024-09-12T03:23:20.675165+0000","last_fullsized":"2024-09-12T03:23:20.675165+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.3","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-12T03:23:20.685867+0000","last_change":"2024-09-12T03:23:20.684832+0000","last_active":"2024-09-12T03:23:20.685867+0000","last_peered":"2024-09-12T03:23:20.685867+0000","last_clean":"2024-09-12T03:23:20.685867+0000","last_became_active":"2024-09-12T03:23:20.684531+0000","last_became_peered":"2024-09-12T03:23:20.684531+0000","last_unstale":"2024-09-12T03:23:20.685867+0000","last_undegraded":"2024-09-12T03:23:20.685867+0000","last_fullsized":"2024-09-12T03:23:20.685867+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_clean_scrub_stamp":"2024-09-12T03:23:19.659665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:11:40.963697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.3","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.675175+0000","last_change":"2024-09-12T03:23:20.675175+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.675175+0000","last_undegraded":"2024-09-12T03:23:20.675175+0000","last_fullsized":"2024-09-12T03:23:20.675175+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.2","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-12T03:23:20.685296+0000","last_change":"2024-09-12T03:23:20.681981+0000","last_active":"2024-09-12T03:23:20.685296+0000","last_peered":"2024-09-12T03:23:20.685296+0000","last_clean":"2024-09-12T03:23:20.685296+0000","last_became_active":"2024-09-12T03:23:20.681487+0000","last_became_peered":"2024-09-12T03:23:20.681487+0000","last_unstale":"2024-09-12T03:23:20.685296+0000","last_undegraded":"2024-09-12T03:23:20.685296+0000","last_fullsized":"2024-09-12T03:23:20.685296+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_clean_scrub_stamp":"2024-09-12T03:23:19.659665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:52:54.619581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.0","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.675295+0000","last_change":"2024-09-12T03:23:20.675295+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.675295+0000","last_undegraded":"2024-09-12T03:23:20.675295+0000","last_fullsized":"2024-09-12T03:23:20.675295+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.1","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-12T03:23:20.682875+0000","last_change":"2024-09-12T03:23:20.682237+0000","last_active":"2024-09-12T03:23:20.682875+0000","last_peered":"2024-09-12T03:23:20.682875+0000","last_clean":"2024-09-12T03:23:20.682875+0000","last_became_active":"2024-09-12T03:23:20.681397+0000","last_became_peered":"2024-09-12T03:23:20.681397+0000","last_unstale":"2024-09-12T03:23:20.682875+0000","last_undegraded":"2024-09-12T03:23:20.682875+0000","last_fullsized":"2024-09-12T03:23:20.682875+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_clean_scrub_stamp":"2024-09-12T03:23:19.659665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:16:33.772644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.1","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.680888+0000","last_change":"2024-09-12T03:23:20.680888+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.680888+0000","last_undegraded":"2024-09-12T03:23:20.680888+0000","last_fullsized":"2024-09-12T03:23:20.680888+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.0","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-12T03:23:20.695121+0000","last_change":"2024-09-12T03:23:20.688059+0000","last_active":"2024-09-12T03:23:20.695121+0000","last_peered":"2024-09-12T03:23:20.695121+0000","last_clean":"2024-09-12T03:23:20.695121+0000","last_became_active":"2024-09-12T03:23:20.687774+0000","last_became_peered":"2024-09-12T03:23:20.687774+0000","last_unstale":"2024-09-12T03:23:20.695121+0000","last_undegraded":"2024-09-12T03:23:20.695121+0000","last_fullsized":"2024-09-12T03:23:20.695121+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_clean_scrub_stamp":"2024-09-12T03:23:19.659665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:31:56.155648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.18","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.678051+0000","last_change":"2024-09-12T03:23:20.678051+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.678051+0000","last_undegraded":"2024-09-12T03:23:20.678051+0000","last_fullsized":"2024-09-12T03:23:20.678051+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.19","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-12T03:23:20.686693+0000","last_change":"2024-09-12T03:23:20.685565+0000","last_active":"2024-09-12T03:23:20.686693+0000","last_peered":"2024-09-12T03:23:20.686693+0000","last_clean":"2024-09-12T03:23:20.686693+0000","last_became_active":"2024-09-12T03:23:20.685230+0000","last_became_peered":"2024-09-12T03:23:20.685230+0000","last_unstale":"2024-09-12T03:23:20.686693+0000","last_undegraded":"2024-09-12T03:23:20.686693+0000","last_fullsized":"2024-09-12T03:23:20.686693+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_clean_scrub_stamp":"2024-09-12T03:23:19.659665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:25:00.890612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"70'128","reported_seq":441,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-12T03:23:20.680128+0000","last_change":"2024-09-12T03:16:21.883446+0000","last_active":"2024-09-12T03:23:20.680128+0000","last_peered":"2024-09-12T03:23:20.680128+0000","last_clean":"2024-09-12T03:23:20.680128+0000","last_became_active":"2024-09-12T03:16:21.882207+0000","last_became_peered":"2024-09-12T03:16:21.882207+0000","last_unstale":"2024-09-12T03:23:20.680128+0000","last_undegraded":"2024-09-12T03:23:20.680128+0000","last_fullsized":"2024-09-12T03:23:20.680128+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:20.316426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:20.316426+0000","last_clean_scrub_stamp":"2024-09-12T03:16:20.316426+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:12:15.424320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":25,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":24,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542230,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37736,"kb_used_data":2388,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334104,"statfs":{"total":96636764160,"available":96598122496,"internally_reserved":0,"allocated":2445312,"data_stored":1407765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542232,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37148,"kb_used_data":1796,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334692,"statfs":{"total":96636764160,"available":96598724608,"internally_reserved":0,"allocated":1839104,"data_stored":811066,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542231,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37156,"kb_used_data":1808,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334684,"statfs":{"total":96636764160,"available":96598716416,"internally_reserved":0,"allocated":1851392,"data_stored":817397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542230,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37160,"kb_used_data":1808,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334680,"statfs":{"total":96636764160,"available":96598712320,"internally_reserved":0,"allocated":1851392,"data_stored":817397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542231,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32996,"kb_used_data":1796,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338844,"statfs":{"total":96636764160,"available":96602976256,"internally_reserved":0,"allocated":1839104,"data_stored":811066,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542231,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32984,"kb_used_data":1796,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338856,"statfs":{"total":96636764160,"available":96602988544,"internally_reserved":0,"allocated":1839104,"data_stored":811066,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542232,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32996,"kb_used_data":1808,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338844,"statfs":{"total":96636764160,"available":96602976256,"internally_reserved":0,"allocated":1851392,"data_stored":817397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542232,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33580,"kb_used_data":2388,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338260,"statfs":{"total":96636764160,"available":96602378240,"internally_reserved":0,"allocated":2445312,"data_stored":1407765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T03:23:25.691 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-12T03:23:25.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.828+0000 7fb6e9667640 1 Processor -- start 2024-09-12T03:23:25.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.828+0000 7fb6e9667640 1 -- start start 2024-09-12T03:23:25.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.829+0000 7fb6e9667640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fb6e40ce6d0 0x7fb6e40ceab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:25.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.829+0000 7fb6e9667640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb6e4160d80 0x7fb6e406bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:25.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.829+0000 7fb6e9667640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb6e41463d0 con 0x7fb6e406c070 2024-09-12T03:23:25.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.829+0000 7fb6e9667640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7fb6e40ceff0 con 0x7fb6e40ce6d0 2024-09-12T03:23:25.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.829+0000 7fb6e9667640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fb6e406c7f0 con 0x7fb6e4160d80 2024-09-12T03:23:25.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.829+0000 7fb6e2ffd640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fb6e40ce6d0 0x7fb6e40ceab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:25.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.829+0000 7fb6e37fe640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fb6e406c070 0x7fb6e406e5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:56132/0 (socket says 172.21.15.27:56132) 2024-09-12T03:23:25.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.829+0000 7fb6e37fe640 1 -- 172.21.15.27:0/494728073 learned_addr learned my addr 172.21.15.27:0/494728073 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:23:25.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.829+0000 7fb6e27fc640 1 --2- 172.21.15.27:0/494728073 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb6e4160d80 0x7fb6e406bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:25.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.829+0000 7fb6e1ffb640 1 -- 172.21.15.27:0/494728073 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4076942176 0 0) 0x7fb6e41463d0 con 0x7fb6e406c070 2024-09-12T03:23:25.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.830+0000 7fb6e1ffb640 1 -- 172.21.15.27:0/494728073 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb6d0003520 con 0x7fb6e406c070 2024-09-12T03:23:25.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.830+0000 7fb6e2ffd640 1 -- 172.21.15.27:0/494728073 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb6e4160d80 msgr2=0x7fb6e406bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:25.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.830+0000 7fb6e2ffd640 1 --2- 172.21.15.27:0/494728073 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb6e4160d80 0x7fb6e406bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:25.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.830+0000 7fb6e2ffd640 1 -- 172.21.15.27:0/494728073 >> v1:172.21.15.27:6789/0 conn(0x7fb6e406c070 legacy=0x7fb6e406e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:25.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.830+0000 7fb6e2ffd640 1 -- 172.21.15.27:0/494728073 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6e41602a0 con 0x7fb6e40ce6d0 2024-09-12T03:23:25.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.830+0000 7fb6e27fc640 1 --2- 172.21.15.27:0/494728073 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb6e4160d80 0x7fb6e406bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:23:25.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.830+0000 7fb6e2ffd640 1 --2- 172.21.15.27:0/494728073 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb6e40ce6d0 0x7fb6e40ceab0 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7fb6dc0097a0 tx=0x7fb6dc02a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=234a2a1a26fc2103 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:25.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.830+0000 7fb6e1ffb640 1 -- 172.21.15.27:0/494728073 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6dc037070 con 0x7fb6e40ce6d0 2024-09-12T03:23:25.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.831+0000 7fb6e1ffb640 1 -- 172.21.15.27:0/494728073 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6dc02ae50 con 0x7fb6e40ce6d0 2024-09-12T03:23:25.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.831+0000 7fb6e1ffb640 1 -- 172.21.15.27:0/494728073 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6dc032660 con 0x7fb6e40ce6d0 2024-09-12T03:23:25.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.831+0000 7fb6e9667640 1 -- 172.21.15.27:0/494728073 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb6e40ce6d0 msgr2=0x7fb6e40ceab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:25.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.831+0000 7fb6e9667640 1 --2- 172.21.15.27:0/494728073 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb6e40ce6d0 0x7fb6e40ceab0 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7fb6dc0097a0 tx=0x7fb6dc02a100 comp rx=0 tx=0).stop 2024-09-12T03:23:25.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.831+0000 7fb6e9667640 1 -- 172.21.15.27:0/494728073 shutdown_connections 2024-09-12T03:23:25.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.831+0000 7fb6e9667640 1 --2- 172.21.15.27:0/494728073 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb6e4160d80 0x7fb6e406bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:25.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.831+0000 7fb6e9667640 1 --2- 172.21.15.27:0/494728073 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb6e40ce6d0 0x7fb6e40ceab0 unknown :-1 s=CLOSED pgs=1546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:25.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.831+0000 7fb6e9667640 1 -- 172.21.15.27:0/494728073 >> 172.21.15.27:0/494728073 conn(0x7fb6e406cc40 msgr2=0x7fb6e406d050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:25.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.831+0000 7fb6e9667640 1 -- 172.21.15.27:0/494728073 shutdown_connections 2024-09-12T03:23:25.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.831+0000 7fb6e9667640 1 -- 172.21.15.27:0/494728073 wait complete. 2024-09-12T03:23:25.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.832+0000 7fb6e9667640 1 Processor -- start 2024-09-12T03:23:25.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.832+0000 7fb6e9667640 1 -- start start 2024-09-12T03:23:25.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.833+0000 7fb6e9667640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb6e406c070 0x7fb6e40c6830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:25.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.833+0000 7fb6e9667640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb6e40ce6d0 0x7fb6e40aeff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:25.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.833+0000 7fb6e9667640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb6e4160d80 0x7fb6e4141a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:25.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.833+0000 7fb6e9667640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fb6e406c4d0 con 0x7fb6e4160d80 2024-09-12T03:23:25.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.833+0000 7fb6e9667640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb6e4160860 con 0x7fb6e40ce6d0 2024-09-12T03:23:25.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.833+0000 7fb6e9667640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fb6e406ecc0 con 0x7fb6e406c070 2024-09-12T03:23:25.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.833+0000 7fb6e37fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb6e4160d80 0x7fb6e4141a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:25.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.833+0000 7fb6e37fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb6e4160d80 0x7fb6e4141a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42474/0 (socket says 172.21.15.27:42474) 2024-09-12T03:23:25.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.833+0000 7fb6e37fe640 1 -- 172.21.15.27:0/3621590248 learned_addr learned my addr 172.21.15.27:0/3621590248 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:23:25.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.833+0000 7fb6e2ffd640 1 --2- 172.21.15.27:0/3621590248 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb6e406c070 0x7fb6e40c6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:25.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.833+0000 7fb6e27fc640 1 --2- 172.21.15.27:0/3621590248 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb6e40ce6d0 0x7fb6e40aeff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:25.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.834+0000 7fb6e37fe640 1 -- 172.21.15.27:0/3621590248 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb6e406c070 msgr2=0x7fb6e40c6830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:25.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.834+0000 7fb6e37fe640 1 --2- 172.21.15.27:0/3621590248 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb6e406c070 0x7fb6e40c6830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:25.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.834+0000 7fb6e37fe640 1 -- 172.21.15.27:0/3621590248 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb6e40ce6d0 msgr2=0x7fb6e40aeff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:25.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.834+0000 7fb6e37fe640 1 --2- 172.21.15.27:0/3621590248 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb6e40ce6d0 0x7fb6e40aeff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:25.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.834+0000 7fb6e37fe640 1 -- 172.21.15.27:0/3621590248 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6e40abe00 con 0x7fb6e4160d80 2024-09-12T03:23:25.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.834+0000 7fb6e27fc640 1 --2- 172.21.15.27:0/3621590248 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb6e40ce6d0 0x7fb6e40aeff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:23:25.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.834+0000 7fb6e2ffd640 1 --2- 172.21.15.27:0/3621590248 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb6e406c070 0x7fb6e40c6830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:23:25.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.834+0000 7fb6e37fe640 1 --2- 172.21.15.27:0/3621590248 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb6e4160d80 0x7fb6e4141a10 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7fb6d400cb70 tx=0x7fb6d400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:25.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.834+0000 7fb6db7fe640 1 -- 172.21.15.27:0/3621590248 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6d4017070 con 0x7fb6e4160d80 2024-09-12T03:23:25.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.834+0000 7fb6db7fe640 1 -- 172.21.15.27:0/3621590248 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6d400ad40 con 0x7fb6e4160d80 2024-09-12T03:23:25.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.834+0000 7fb6e9667640 1 -- 172.21.15.27:0/3621590248 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6e40ac820 con 0x7fb6e4160d80 2024-09-12T03:23:25.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.834+0000 7fb6db7fe640 1 -- 172.21.15.27:0/3621590248 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6d4012620 con 0x7fb6e4160d80 2024-09-12T03:23:25.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.834+0000 7fb6e9667640 1 -- 172.21.15.27:0/3621590248 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6e40ad0c0 con 0x7fb6e4160d80 2024-09-12T03:23:25.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.835+0000 7fb6e9667640 1 -- 172.21.15.27:0/3621590248 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6e40acc10 con 0x7fb6e4160d80 2024-09-12T03:23:25.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.835+0000 7fb6db7fe640 1 -- 172.21.15.27:0/3621590248 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 24) ==== 91212+0+0 (secure 0 0 0) 0x7fb6d401a070 con 0x7fb6e4160d80 2024-09-12T03:23:25.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.836+0000 7fb6db7fe640 1 --2- 172.21.15.27:0/3621590248 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fb6c406c610 0x7fb6c406ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:25.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.836+0000 7fb6db7fe640 1 -- 172.21.15.27:0/3621590248 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 5765+0+0 (secure 0 0 0) 0x7fb6d40896c0 con 0x7fb6e4160d80 2024-09-12T03:23:25.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.836+0000 7fb6e2ffd640 1 --2- 172.21.15.27:0/3621590248 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fb6c406c610 0x7fb6c406ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:25.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.836+0000 7fb6e2ffd640 1 --2- 172.21.15.27:0/3621590248 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fb6c406c610 0x7fb6c406ead0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fb6dc034000 tx=0x7fb6dc02e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:25.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:25.840+0000 7fb6db7fe640 1 -- 172.21.15.27:0/3621590248 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fb6d4059960 con 0x7fb6e4160d80 2024-09-12T03:23:26.016 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.016+0000 7fb6e9667640 1 -- 172.21.15.27:0/3621590248 --> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb6e406f0f0 con 0x7fb6c406c610 2024-09-12T03:23:26.022 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.021+0000 7fb6db7fe640 1 -- 172.21.15.27:0/3621590248 <== mgr.4113 v2:172.21.15.27:6836/1226920526 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247090 (secure 0 0 0) 0x7fb6dc07e550 con 0x7fb6c406c610 2024-09-12T03:23:26.025 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:23:26.026 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2024-09-12T03:23:26.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.024+0000 7fb6e9667640 1 -- 172.21.15.27:0/3621590248 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fb6c406c610 msgr2=0x7fb6c406ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:26.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.024+0000 7fb6e9667640 1 --2- 172.21.15.27:0/3621590248 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fb6c406c610 0x7fb6c406ead0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fb6dc034000 tx=0x7fb6dc02e000 comp rx=0 tx=0).stop 2024-09-12T03:23:26.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.024+0000 7fb6e9667640 1 -- 172.21.15.27:0/3621590248 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb6e4160d80 msgr2=0x7fb6e4141a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:26.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.024+0000 7fb6e9667640 1 --2- 172.21.15.27:0/3621590248 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb6e4160d80 0x7fb6e4141a10 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7fb6d400cb70 tx=0x7fb6d400a430 comp rx=0 tx=0).stop 2024-09-12T03:23:26.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.025+0000 7fb6e9667640 1 -- 172.21.15.27:0/3621590248 shutdown_connections 2024-09-12T03:23:26.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.025+0000 7fb6e9667640 1 --2- 172.21.15.27:0/3621590248 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fb6c406c610 0x7fb6c406ead0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:26.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.025+0000 7fb6e9667640 1 --2- 172.21.15.27:0/3621590248 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb6e4160d80 0x7fb6e4141a10 unknown :-1 s=CLOSED pgs=1547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:26.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.025+0000 7fb6e9667640 1 --2- 172.21.15.27:0/3621590248 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb6e40ce6d0 0x7fb6e40aeff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:26.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.025+0000 7fb6e9667640 1 --2- 172.21.15.27:0/3621590248 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb6e406c070 0x7fb6e40c6830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:26.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.025+0000 7fb6e9667640 1 -- 172.21.15.27:0/3621590248 >> 172.21.15.27:0/3621590248 conn(0x7fb6e406cc40 msgr2=0x7fb6e415e6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:26.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.025+0000 7fb6e9667640 1 -- 172.21.15.27:0/3621590248 shutdown_connections 2024-09-12T03:23:26.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.025+0000 7fb6e9667640 1 -- 172.21.15.27:0/3621590248 wait complete. 2024-09-12T03:23:26.036 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":178,"stamp":"2024-09-12T03:23:25.134647+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":41,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":281756,"kb_used_data":15588,"kb_used_omap":12,"kb_used_meta":265971,"kb_avail":754692964,"statfs":{"total":773094113280,"available":772805595136,"internally_reserved":0,"allocated":15962112,"data_stored":7700919,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":272354864},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":15,"apply_latency_ms":15,"commit_latency_ns":15000000,"apply_latency_ns":15000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.468635"},"pg_stats":[{"pgid":"25.3f","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.676683+0000","last_change":"2024-09-12T03:23:20.676683+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.676683+0000","last_undegraded":"2024-09-12T03:23:20.676683+0000","last_fullsized":"2024-09-12T03:23:20.676683+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.3e","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.677002+0000","last_change":"2024-09-12T03:23:20.677002+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.677002+0000","last_undegraded":"2024-09-12T03:23:20.677002+0000","last_fullsized":"2024-09-12T03:23:20.677002+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.3d","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.680808+0000","last_change":"2024-09-12T03:23:20.680808+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.680808+0000","last_undegraded":"2024-09-12T03:23:20.680808+0000","last_fullsized":"2024-09-12T03:23:20.680808+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.3c","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.677777+0000","last_change":"2024-09-12T03:23:20.677777+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.677777+0000","last_undegraded":"2024-09-12T03:23:20.677777+0000","last_fullsized":"2024-09-12T03:23:20.677777+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.3b","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.683306+0000","last_change":"2024-09-12T03:23:20.683306+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.683306+0000","last_undegraded":"2024-09-12T03:23:20.683306+0000","last_fullsized":"2024-09-12T03:23:20.683306+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.3a","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.680330+0000","last_change":"2024-09-12T03:23:20.680330+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.680330+0000","last_undegraded":"2024-09-12T03:23:20.680330+0000","last_fullsized":"2024-09-12T03:23:20.680330+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.39","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.672348+0000","last_change":"2024-09-12T03:23:20.672348+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.672348+0000","last_undegraded":"2024-09-12T03:23:20.672348+0000","last_fullsized":"2024-09-12T03:23:20.672348+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.38","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.676156+0000","last_change":"2024-09-12T03:23:20.676156+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.676156+0000","last_undegraded":"2024-09-12T03:23:20.676156+0000","last_fullsized":"2024-09-12T03:23:20.676156+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.37","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.674337+0000","last_change":"2024-09-12T03:23:20.674337+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.674337+0000","last_undegraded":"2024-09-12T03:23:20.674337+0000","last_fullsized":"2024-09-12T03:23:20.674337+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.36","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.679738+0000","last_change":"2024-09-12T03:23:20.679738+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.679738+0000","last_undegraded":"2024-09-12T03:23:20.679738+0000","last_fullsized":"2024-09-12T03:23:20.679738+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.35","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.671126+0000","last_change":"2024-09-12T03:23:20.671126+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.671126+0000","last_undegraded":"2024-09-12T03:23:20.671126+0000","last_fullsized":"2024-09-12T03:23:20.671126+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.34","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.677214+0000","last_change":"2024-09-12T03:23:20.677214+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.677214+0000","last_undegraded":"2024-09-12T03:23:20.677214+0000","last_fullsized":"2024-09-12T03:23:20.677214+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.33","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.677238+0000","last_change":"2024-09-12T03:23:20.677238+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.677238+0000","last_undegraded":"2024-09-12T03:23:20.677238+0000","last_fullsized":"2024-09-12T03:23:20.677238+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.32","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.675036+0000","last_change":"2024-09-12T03:23:20.675036+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.675036+0000","last_undegraded":"2024-09-12T03:23:20.675036+0000","last_fullsized":"2024-09-12T03:23:20.675036+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.31","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.675841+0000","last_change":"2024-09-12T03:23:20.675841+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.675841+0000","last_undegraded":"2024-09-12T03:23:20.675841+0000","last_fullsized":"2024-09-12T03:23:20.675841+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.30","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.675750+0000","last_change":"2024-09-12T03:23:20.675750+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.675750+0000","last_undegraded":"2024-09-12T03:23:20.675750+0000","last_fullsized":"2024-09-12T03:23:20.675750+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.2f","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.687993+0000","last_change":"2024-09-12T03:23:20.687993+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.687993+0000","last_undegraded":"2024-09-12T03:23:20.687993+0000","last_fullsized":"2024-09-12T03:23:20.687993+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.2e","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.676010+0000","last_change":"2024-09-12T03:23:20.676010+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.676010+0000","last_undegraded":"2024-09-12T03:23:20.676010+0000","last_fullsized":"2024-09-12T03:23:20.676010+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.2d","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.680215+0000","last_change":"2024-09-12T03:23:20.680215+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.680215+0000","last_undegraded":"2024-09-12T03:23:20.680215+0000","last_fullsized":"2024-09-12T03:23:20.680215+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.2c","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.685697+0000","last_change":"2024-09-12T03:23:20.685697+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.685697+0000","last_undegraded":"2024-09-12T03:23:20.685697+0000","last_fullsized":"2024-09-12T03:23:20.685697+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.2b","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.682781+0000","last_change":"2024-09-12T03:23:20.682781+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.682781+0000","last_undegraded":"2024-09-12T03:23:20.682781+0000","last_fullsized":"2024-09-12T03:23:20.682781+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.2a","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.679222+0000","last_change":"2024-09-12T03:23:20.679222+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.679222+0000","last_undegraded":"2024-09-12T03:23:20.679222+0000","last_fullsized":"2024-09-12T03:23:20.679222+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.29","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.679592+0000","last_change":"2024-09-12T03:23:20.679592+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.679592+0000","last_undegraded":"2024-09-12T03:23:20.679592+0000","last_fullsized":"2024-09-12T03:23:20.679592+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.28","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.675319+0000","last_change":"2024-09-12T03:23:20.675319+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.675319+0000","last_undegraded":"2024-09-12T03:23:20.675319+0000","last_fullsized":"2024-09-12T03:23:20.675319+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.27","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.676115+0000","last_change":"2024-09-12T03:23:20.676115+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.676115+0000","last_undegraded":"2024-09-12T03:23:20.676115+0000","last_fullsized":"2024-09-12T03:23:20.676115+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.26","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.671542+0000","last_change":"2024-09-12T03:23:20.671542+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.671542+0000","last_undegraded":"2024-09-12T03:23:20.671542+0000","last_fullsized":"2024-09-12T03:23:20.671542+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.25","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.676098+0000","last_change":"2024-09-12T03:23:20.676098+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.676098+0000","last_undegraded":"2024-09-12T03:23:20.676098+0000","last_fullsized":"2024-09-12T03:23:20.676098+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.24","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.685181+0000","last_change":"2024-09-12T03:23:20.685181+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.685181+0000","last_undegraded":"2024-09-12T03:23:20.685181+0000","last_fullsized":"2024-09-12T03:23:20.685181+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.23","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.676786+0000","last_change":"2024-09-12T03:23:20.676786+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.676786+0000","last_undegraded":"2024-09-12T03:23:20.676786+0000","last_fullsized":"2024-09-12T03:23:20.676786+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.22","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.674611+0000","last_change":"2024-09-12T03:23:20.674611+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.674611+0000","last_undegraded":"2024-09-12T03:23:20.674611+0000","last_fullsized":"2024-09-12T03:23:20.674611+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.21","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.676607+0000","last_change":"2024-09-12T03:23:20.676607+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.676607+0000","last_undegraded":"2024-09-12T03:23:20.676607+0000","last_fullsized":"2024-09-12T03:23:20.676607+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.20","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.682292+0000","last_change":"2024-09-12T03:23:20.682292+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.682292+0000","last_undegraded":"2024-09-12T03:23:20.682292+0000","last_fullsized":"2024-09-12T03:23:20.682292+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1e","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.678929+0000","last_change":"2024-09-12T03:23:20.678929+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.678929+0000","last_undegraded":"2024-09-12T03:23:20.678929+0000","last_fullsized":"2024-09-12T03:23:20.678929+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1f","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-12T03:23:20.688172+0000","last_change":"2024-09-12T03:23:20.682141+0000","last_active":"2024-09-12T03:23:20.688172+0000","last_peered":"2024-09-12T03:23:20.688172+0000","last_clean":"2024-09-12T03:23:20.688172+0000","last_became_active":"2024-09-12T03:23:20.681852+0000","last_became_peered":"2024-09-12T03:23:20.681852+0000","last_unstale":"2024-09-12T03:23:20.688172+0000","last_undegraded":"2024-09-12T03:23:20.688172+0000","last_fullsized":"2024-09-12T03:23:20.688172+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_clean_scrub_stamp":"2024-09-12T03:23:19.659665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:20:17.004278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1f","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.678598+0000","last_change":"2024-09-12T03:23:20.678598+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.678598+0000","last_undegraded":"2024-09-12T03:23:20.678598+0000","last_fullsized":"2024-09-12T03:23:20.678598+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.1e","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-12T03:23:20.687095+0000","last_change":"2024-09-12T03:23:20.686262+0000","last_active":"2024-09-12T03:23:20.687095+0000","last_peered":"2024-09-12T03:23:20.687095+0000","last_clean":"2024-09-12T03:23:20.687095+0000","last_became_active":"2024-09-12T03:23:20.685552+0000","last_became_peered":"2024-09-12T03:23:20.685552+0000","last_unstale":"2024-09-12T03:23:20.687095+0000","last_undegraded":"2024-09-12T03:23:20.687095+0000","last_fullsized":"2024-09-12T03:23:20.687095+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_clean_scrub_stamp":"2024-09-12T03:23:19.659665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:20:51.463260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.1c","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.672426+0000","last_change":"2024-09-12T03:23:20.672426+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.672426+0000","last_undegraded":"2024-09-12T03:23:20.672426+0000","last_fullsized":"2024-09-12T03:23:20.672426+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.1d","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-12T03:23:20.695515+0000","last_change":"2024-09-12T03:23:20.690566+0000","last_active":"2024-09-12T03:23:20.695515+0000","last_peered":"2024-09-12T03:23:20.695515+0000","last_clean":"2024-09-12T03:23:20.695515+0000","last_became_active":"2024-09-12T03:23:20.690296+0000","last_became_peered":"2024-09-12T03:23:20.690296+0000","last_unstale":"2024-09-12T03:23:20.695515+0000","last_undegraded":"2024-09-12T03:23:20.695515+0000","last_fullsized":"2024-09-12T03:23:20.695515+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_clean_scrub_stamp":"2024-09-12T03:23:19.659665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:13:40.613414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1d","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.672931+0000","last_change":"2024-09-12T03:23:20.672931+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.672931+0000","last_undegraded":"2024-09-12T03:23:20.672931+0000","last_fullsized":"2024-09-12T03:23:20.672931+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.1c","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-12T03:23:20.687327+0000","last_change":"2024-09-12T03:23:20.686667+0000","last_active":"2024-09-12T03:23:20.687327+0000","last_peered":"2024-09-12T03:23:20.687327+0000","last_clean":"2024-09-12T03:23:20.687327+0000","last_became_active":"2024-09-12T03:23:20.686355+0000","last_became_peered":"2024-09-12T03:23:20.686355+0000","last_unstale":"2024-09-12T03:23:20.687327+0000","last_undegraded":"2024-09-12T03:23:20.687327+0000","last_fullsized":"2024-09-12T03:23:20.687327+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_clean_scrub_stamp":"2024-09-12T03:23:19.659665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:01:34.049378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.1a","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.678590+0000","last_change":"2024-09-12T03:23:20.678590+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.678590+0000","last_undegraded":"2024-09-12T03:23:20.678590+0000","last_fullsized":"2024-09-12T03:23:20.678590+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.1b","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-12T03:23:20.686092+0000","last_change":"2024-09-12T03:23:20.684507+0000","last_active":"2024-09-12T03:23:20.686092+0000","last_peered":"2024-09-12T03:23:20.686092+0000","last_clean":"2024-09-12T03:23:20.686092+0000","last_became_active":"2024-09-12T03:23:20.684246+0000","last_became_peered":"2024-09-12T03:23:20.684246+0000","last_unstale":"2024-09-12T03:23:20.686092+0000","last_undegraded":"2024-09-12T03:23:20.686092+0000","last_fullsized":"2024-09-12T03:23:20.686092+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_clean_scrub_stamp":"2024-09-12T03:23:19.659665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:39:13.426726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1b","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.674380+0000","last_change":"2024-09-12T03:23:20.674380+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.674380+0000","last_undegraded":"2024-09-12T03:23:20.674380+0000","last_fullsized":"2024-09-12T03:23:20.674380+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1a","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-12T03:23:20.697734+0000","last_change":"2024-09-12T03:23:20.696117+0000","last_active":"2024-09-12T03:23:20.697734+0000","last_peered":"2024-09-12T03:23:20.697734+0000","last_clean":"2024-09-12T03:23:20.697734+0000","last_became_active":"2024-09-12T03:23:20.695795+0000","last_became_peered":"2024-09-12T03:23:20.695795+0000","last_unstale":"2024-09-12T03:23:20.697734+0000","last_undegraded":"2024-09-12T03:23:20.697734+0000","last_fullsized":"2024-09-12T03:23:20.697734+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_clean_scrub_stamp":"2024-09-12T03:23:19.659665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:36:38.656553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.19","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.671879+0000","last_change":"2024-09-12T03:23:20.671879+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.671879+0000","last_undegraded":"2024-09-12T03:23:20.671879+0000","last_fullsized":"2024-09-12T03:23:20.671879+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.18","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-12T03:23:20.690256+0000","last_change":"2024-09-12T03:23:20.689802+0000","last_active":"2024-09-12T03:23:20.690256+0000","last_peered":"2024-09-12T03:23:20.690256+0000","last_clean":"2024-09-12T03:23:20.690256+0000","last_became_active":"2024-09-12T03:23:20.689503+0000","last_became_peered":"2024-09-12T03:23:20.689503+0000","last_unstale":"2024-09-12T03:23:20.690256+0000","last_undegraded":"2024-09-12T03:23:20.690256+0000","last_fullsized":"2024-09-12T03:23:20.690256+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_clean_scrub_stamp":"2024-09-12T03:23:19.659665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:29:59.930252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.16","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.675332+0000","last_change":"2024-09-12T03:23:20.675332+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.675332+0000","last_undegraded":"2024-09-12T03:23:20.675332+0000","last_fullsized":"2024-09-12T03:23:20.675332+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.17","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-12T03:23:20.679242+0000","last_change":"2024-09-12T03:23:20.678720+0000","last_active":"2024-09-12T03:23:20.679242+0000","last_peered":"2024-09-12T03:23:20.679242+0000","last_clean":"2024-09-12T03:23:20.679242+0000","last_became_active":"2024-09-12T03:23:20.678415+0000","last_became_peered":"2024-09-12T03:23:20.678415+0000","last_unstale":"2024-09-12T03:23:20.679242+0000","last_undegraded":"2024-09-12T03:23:20.679242+0000","last_fullsized":"2024-09-12T03:23:20.679242+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_clean_scrub_stamp":"2024-09-12T03:23:19.659665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:22:49.298961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.17","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.674869+0000","last_change":"2024-09-12T03:23:20.674869+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.674869+0000","last_undegraded":"2024-09-12T03:23:20.674869+0000","last_fullsized":"2024-09-12T03:23:20.674869+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.16","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-12T03:23:20.686685+0000","last_change":"2024-09-12T03:23:20.683066+0000","last_active":"2024-09-12T03:23:20.686685+0000","last_peered":"2024-09-12T03:23:20.686685+0000","last_clean":"2024-09-12T03:23:20.686685+0000","last_became_active":"2024-09-12T03:23:20.682759+0000","last_became_peered":"2024-09-12T03:23:20.682759+0000","last_unstale":"2024-09-12T03:23:20.686685+0000","last_undegraded":"2024-09-12T03:23:20.686685+0000","last_fullsized":"2024-09-12T03:23:20.686685+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_clean_scrub_stamp":"2024-09-12T03:23:19.659665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:51:25.370412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.14","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.671943+0000","last_change":"2024-09-12T03:23:20.671943+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.671943+0000","last_undegraded":"2024-09-12T03:23:20.671943+0000","last_fullsized":"2024-09-12T03:23:20.671943+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.15","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-12T03:23:20.693567+0000","last_change":"2024-09-12T03:23:20.691649+0000","last_active":"2024-09-12T03:23:20.693567+0000","last_peered":"2024-09-12T03:23:20.693567+0000","last_clean":"2024-09-12T03:23:20.693567+0000","last_became_active":"2024-09-12T03:23:20.691385+0000","last_became_peered":"2024-09-12T03:23:20.691385+0000","last_unstale":"2024-09-12T03:23:20.693567+0000","last_undegraded":"2024-09-12T03:23:20.693567+0000","last_fullsized":"2024-09-12T03:23:20.693567+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_clean_scrub_stamp":"2024-09-12T03:23:19.659665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:51:15.001810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.15","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.677434+0000","last_change":"2024-09-12T03:23:20.677434+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.677434+0000","last_undegraded":"2024-09-12T03:23:20.677434+0000","last_fullsized":"2024-09-12T03:23:20.677434+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.14","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-12T03:23:20.684819+0000","last_change":"2024-09-12T03:23:20.683263+0000","last_active":"2024-09-12T03:23:20.684819+0000","last_peered":"2024-09-12T03:23:20.684819+0000","last_clean":"2024-09-12T03:23:20.684819+0000","last_became_active":"2024-09-12T03:23:20.682986+0000","last_became_peered":"2024-09-12T03:23:20.682986+0000","last_unstale":"2024-09-12T03:23:20.684819+0000","last_undegraded":"2024-09-12T03:23:20.684819+0000","last_fullsized":"2024-09-12T03:23:20.684819+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_clean_scrub_stamp":"2024-09-12T03:23:19.659665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:17:57.817757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.12","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.676361+0000","last_change":"2024-09-12T03:23:20.676361+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.676361+0000","last_undegraded":"2024-09-12T03:23:20.676361+0000","last_fullsized":"2024-09-12T03:23:20.676361+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.13","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-12T03:23:20.690520+0000","last_change":"2024-09-12T03:23:20.689845+0000","last_active":"2024-09-12T03:23:20.690520+0000","last_peered":"2024-09-12T03:23:20.690520+0000","last_clean":"2024-09-12T03:23:20.690520+0000","last_became_active":"2024-09-12T03:23:20.689485+0000","last_became_peered":"2024-09-12T03:23:20.689485+0000","last_unstale":"2024-09-12T03:23:20.690520+0000","last_undegraded":"2024-09-12T03:23:20.690520+0000","last_fullsized":"2024-09-12T03:23:20.690520+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_clean_scrub_stamp":"2024-09-12T03:23:19.659665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:41:28.835418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.13","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.672076+0000","last_change":"2024-09-12T03:23:20.672076+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.672076+0000","last_undegraded":"2024-09-12T03:23:20.672076+0000","last_fullsized":"2024-09-12T03:23:20.672076+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.12","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-12T03:23:20.692153+0000","last_change":"2024-09-12T03:23:20.689243+0000","last_active":"2024-09-12T03:23:20.692153+0000","last_peered":"2024-09-12T03:23:20.692153+0000","last_clean":"2024-09-12T03:23:20.692153+0000","last_became_active":"2024-09-12T03:23:20.688661+0000","last_became_peered":"2024-09-12T03:23:20.688661+0000","last_unstale":"2024-09-12T03:23:20.692153+0000","last_undegraded":"2024-09-12T03:23:20.692153+0000","last_fullsized":"2024-09-12T03:23:20.692153+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_clean_scrub_stamp":"2024-09-12T03:23:19.659665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:16:21.498607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.10","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.672680+0000","last_change":"2024-09-12T03:23:20.672680+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.672680+0000","last_undegraded":"2024-09-12T03:23:20.672680+0000","last_fullsized":"2024-09-12T03:23:20.672680+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.11","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-12T03:23:20.687225+0000","last_change":"2024-09-12T03:23:20.680790+0000","last_active":"2024-09-12T03:23:20.687225+0000","last_peered":"2024-09-12T03:23:20.687225+0000","last_clean":"2024-09-12T03:23:20.687225+0000","last_became_active":"2024-09-12T03:23:20.680508+0000","last_became_peered":"2024-09-12T03:23:20.680508+0000","last_unstale":"2024-09-12T03:23:20.687225+0000","last_undegraded":"2024-09-12T03:23:20.687225+0000","last_fullsized":"2024-09-12T03:23:20.687225+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_clean_scrub_stamp":"2024-09-12T03:23:19.659665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:56:34.246022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.11","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.676970+0000","last_change":"2024-09-12T03:23:20.676970+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.676970+0000","last_undegraded":"2024-09-12T03:23:20.676970+0000","last_fullsized":"2024-09-12T03:23:20.676970+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.10","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-12T03:23:20.690498+0000","last_change":"2024-09-12T03:23:20.689843+0000","last_active":"2024-09-12T03:23:20.690498+0000","last_peered":"2024-09-12T03:23:20.690498+0000","last_clean":"2024-09-12T03:23:20.690498+0000","last_became_active":"2024-09-12T03:23:20.689517+0000","last_became_peered":"2024-09-12T03:23:20.689517+0000","last_unstale":"2024-09-12T03:23:20.690498+0000","last_undegraded":"2024-09-12T03:23:20.690498+0000","last_fullsized":"2024-09-12T03:23:20.690498+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_clean_scrub_stamp":"2024-09-12T03:23:19.659665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:27:53.769921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.e","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.678489+0000","last_change":"2024-09-12T03:23:20.678489+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.678489+0000","last_undegraded":"2024-09-12T03:23:20.678489+0000","last_fullsized":"2024-09-12T03:23:20.678489+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.f","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-12T03:23:20.692368+0000","last_change":"2024-09-12T03:23:20.689669+0000","last_active":"2024-09-12T03:23:20.692368+0000","last_peered":"2024-09-12T03:23:20.692368+0000","last_clean":"2024-09-12T03:23:20.692368+0000","last_became_active":"2024-09-12T03:23:20.689291+0000","last_became_peered":"2024-09-12T03:23:20.689291+0000","last_unstale":"2024-09-12T03:23:20.692368+0000","last_undegraded":"2024-09-12T03:23:20.692368+0000","last_fullsized":"2024-09-12T03:23:20.692368+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_clean_scrub_stamp":"2024-09-12T03:23:19.659665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:25:29.729322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.f","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.673914+0000","last_change":"2024-09-12T03:23:20.673914+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.673914+0000","last_undegraded":"2024-09-12T03:23:20.673914+0000","last_fullsized":"2024-09-12T03:23:20.673914+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.e","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-12T03:23:20.692311+0000","last_change":"2024-09-12T03:23:20.691112+0000","last_active":"2024-09-12T03:23:20.692311+0000","last_peered":"2024-09-12T03:23:20.692311+0000","last_clean":"2024-09-12T03:23:20.692311+0000","last_became_active":"2024-09-12T03:23:20.690862+0000","last_became_peered":"2024-09-12T03:23:20.690862+0000","last_unstale":"2024-09-12T03:23:20.692311+0000","last_undegraded":"2024-09-12T03:23:20.692311+0000","last_fullsized":"2024-09-12T03:23:20.692311+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_clean_scrub_stamp":"2024-09-12T03:23:19.659665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:42:16.029521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.c","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.672469+0000","last_change":"2024-09-12T03:23:20.672469+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.672469+0000","last_undegraded":"2024-09-12T03:23:20.672469+0000","last_fullsized":"2024-09-12T03:23:20.672469+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.d","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-12T03:23:20.688563+0000","last_change":"2024-09-12T03:23:20.688103+0000","last_active":"2024-09-12T03:23:20.688563+0000","last_peered":"2024-09-12T03:23:20.688563+0000","last_clean":"2024-09-12T03:23:20.688563+0000","last_became_active":"2024-09-12T03:23:20.687797+0000","last_became_peered":"2024-09-12T03:23:20.687797+0000","last_unstale":"2024-09-12T03:23:20.688563+0000","last_undegraded":"2024-09-12T03:23:20.688563+0000","last_fullsized":"2024-09-12T03:23:20.688563+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_clean_scrub_stamp":"2024-09-12T03:23:19.659665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:34:39.656922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.d","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.677217+0000","last_change":"2024-09-12T03:23:20.677217+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.677217+0000","last_undegraded":"2024-09-12T03:23:20.677217+0000","last_fullsized":"2024-09-12T03:23:20.677217+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.c","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-12T03:23:20.684552+0000","last_change":"2024-09-12T03:23:20.682322+0000","last_active":"2024-09-12T03:23:20.684552+0000","last_peered":"2024-09-12T03:23:20.684552+0000","last_clean":"2024-09-12T03:23:20.684552+0000","last_became_active":"2024-09-12T03:23:20.682033+0000","last_became_peered":"2024-09-12T03:23:20.682033+0000","last_unstale":"2024-09-12T03:23:20.684552+0000","last_undegraded":"2024-09-12T03:23:20.684552+0000","last_fullsized":"2024-09-12T03:23:20.684552+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_clean_scrub_stamp":"2024-09-12T03:23:19.659665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:42:18.081947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.a","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.676951+0000","last_change":"2024-09-12T03:23:20.676951+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.676951+0000","last_undegraded":"2024-09-12T03:23:20.676951+0000","last_fullsized":"2024-09-12T03:23:20.676951+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.b","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-12T03:23:20.690808+0000","last_change":"2024-09-12T03:23:20.688832+0000","last_active":"2024-09-12T03:23:20.690808+0000","last_peered":"2024-09-12T03:23:20.690808+0000","last_clean":"2024-09-12T03:23:20.690808+0000","last_became_active":"2024-09-12T03:23:20.688021+0000","last_became_peered":"2024-09-12T03:23:20.688021+0000","last_unstale":"2024-09-12T03:23:20.690808+0000","last_undegraded":"2024-09-12T03:23:20.690808+0000","last_fullsized":"2024-09-12T03:23:20.690808+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_clean_scrub_stamp":"2024-09-12T03:23:19.659665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:11:32.173284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.b","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.671109+0000","last_change":"2024-09-12T03:23:20.671109+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.671109+0000","last_undegraded":"2024-09-12T03:23:20.671109+0000","last_fullsized":"2024-09-12T03:23:20.671109+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.a","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-12T03:23:20.686788+0000","last_change":"2024-09-12T03:23:20.683771+0000","last_active":"2024-09-12T03:23:20.686788+0000","last_peered":"2024-09-12T03:23:20.686788+0000","last_clean":"2024-09-12T03:23:20.686788+0000","last_became_active":"2024-09-12T03:23:20.683511+0000","last_became_peered":"2024-09-12T03:23:20.683511+0000","last_unstale":"2024-09-12T03:23:20.686788+0000","last_undegraded":"2024-09-12T03:23:20.686788+0000","last_fullsized":"2024-09-12T03:23:20.686788+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_clean_scrub_stamp":"2024-09-12T03:23:19.659665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:28:42.183834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.8","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.675851+0000","last_change":"2024-09-12T03:23:20.675851+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.675851+0000","last_undegraded":"2024-09-12T03:23:20.675851+0000","last_fullsized":"2024-09-12T03:23:20.675851+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.9","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-12T03:23:20.695616+0000","last_change":"2024-09-12T03:23:20.692711+0000","last_active":"2024-09-12T03:23:20.695616+0000","last_peered":"2024-09-12T03:23:20.695616+0000","last_clean":"2024-09-12T03:23:20.695616+0000","last_became_active":"2024-09-12T03:23:20.692455+0000","last_became_peered":"2024-09-12T03:23:20.692455+0000","last_unstale":"2024-09-12T03:23:20.695616+0000","last_undegraded":"2024-09-12T03:23:20.695616+0000","last_fullsized":"2024-09-12T03:23:20.695616+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_clean_scrub_stamp":"2024-09-12T03:23:19.659665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:47:47.363872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.9","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.681456+0000","last_change":"2024-09-12T03:23:20.681456+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.681456+0000","last_undegraded":"2024-09-12T03:23:20.681456+0000","last_fullsized":"2024-09-12T03:23:20.681456+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.8","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-12T03:23:20.684083+0000","last_change":"2024-09-12T03:23:20.679394+0000","last_active":"2024-09-12T03:23:20.684083+0000","last_peered":"2024-09-12T03:23:20.684083+0000","last_clean":"2024-09-12T03:23:20.684083+0000","last_became_active":"2024-09-12T03:23:20.679100+0000","last_became_peered":"2024-09-12T03:23:20.679100+0000","last_unstale":"2024-09-12T03:23:20.684083+0000","last_undegraded":"2024-09-12T03:23:20.684083+0000","last_fullsized":"2024-09-12T03:23:20.684083+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_clean_scrub_stamp":"2024-09-12T03:23:19.659665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:49:36.375371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.6","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.681144+0000","last_change":"2024-09-12T03:23:20.681144+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.681144+0000","last_undegraded":"2024-09-12T03:23:20.681144+0000","last_fullsized":"2024-09-12T03:23:20.681144+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.7","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-12T03:23:20.693562+0000","last_change":"2024-09-12T03:23:20.693222+0000","last_active":"2024-09-12T03:23:20.693562+0000","last_peered":"2024-09-12T03:23:20.693562+0000","last_clean":"2024-09-12T03:23:20.693562+0000","last_became_active":"2024-09-12T03:23:20.692889+0000","last_became_peered":"2024-09-12T03:23:20.692889+0000","last_unstale":"2024-09-12T03:23:20.693562+0000","last_undegraded":"2024-09-12T03:23:20.693562+0000","last_fullsized":"2024-09-12T03:23:20.693562+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_clean_scrub_stamp":"2024-09-12T03:23:19.659665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:08:14.319966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.7","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.676386+0000","last_change":"2024-09-12T03:23:20.676386+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.676386+0000","last_undegraded":"2024-09-12T03:23:20.676386+0000","last_fullsized":"2024-09-12T03:23:20.676386+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.6","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-12T03:23:20.688527+0000","last_change":"2024-09-12T03:23:20.687884+0000","last_active":"2024-09-12T03:23:20.688527+0000","last_peered":"2024-09-12T03:23:20.688527+0000","last_clean":"2024-09-12T03:23:20.688527+0000","last_became_active":"2024-09-12T03:23:20.687621+0000","last_became_peered":"2024-09-12T03:23:20.687621+0000","last_unstale":"2024-09-12T03:23:20.688527+0000","last_undegraded":"2024-09-12T03:23:20.688527+0000","last_fullsized":"2024-09-12T03:23:20.688527+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_clean_scrub_stamp":"2024-09-12T03:23:19.659665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:34:47.119838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.4","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.677605+0000","last_change":"2024-09-12T03:23:20.677605+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.677605+0000","last_undegraded":"2024-09-12T03:23:20.677605+0000","last_fullsized":"2024-09-12T03:23:20.677605+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.5","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-12T03:23:20.688498+0000","last_change":"2024-09-12T03:23:20.686431+0000","last_active":"2024-09-12T03:23:20.688498+0000","last_peered":"2024-09-12T03:23:20.688498+0000","last_clean":"2024-09-12T03:23:20.688498+0000","last_became_active":"2024-09-12T03:23:20.686145+0000","last_became_peered":"2024-09-12T03:23:20.686145+0000","last_unstale":"2024-09-12T03:23:20.688498+0000","last_undegraded":"2024-09-12T03:23:20.688498+0000","last_fullsized":"2024-09-12T03:23:20.688498+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_clean_scrub_stamp":"2024-09-12T03:23:19.659665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:14:07.788062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.5","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.677039+0000","last_change":"2024-09-12T03:23:20.677039+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.677039+0000","last_undegraded":"2024-09-12T03:23:20.677039+0000","last_fullsized":"2024-09-12T03:23:20.677039+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.4","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-12T03:23:20.691435+0000","last_change":"2024-09-12T03:23:20.689734+0000","last_active":"2024-09-12T03:23:20.691435+0000","last_peered":"2024-09-12T03:23:20.691435+0000","last_clean":"2024-09-12T03:23:20.691435+0000","last_became_active":"2024-09-12T03:23:20.689439+0000","last_became_peered":"2024-09-12T03:23:20.689439+0000","last_unstale":"2024-09-12T03:23:20.691435+0000","last_undegraded":"2024-09-12T03:23:20.691435+0000","last_fullsized":"2024-09-12T03:23:20.691435+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_clean_scrub_stamp":"2024-09-12T03:23:19.659665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:47:48.015298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.2","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.675165+0000","last_change":"2024-09-12T03:23:20.675165+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.675165+0000","last_undegraded":"2024-09-12T03:23:20.675165+0000","last_fullsized":"2024-09-12T03:23:20.675165+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.3","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-12T03:23:20.685867+0000","last_change":"2024-09-12T03:23:20.684832+0000","last_active":"2024-09-12T03:23:20.685867+0000","last_peered":"2024-09-12T03:23:20.685867+0000","last_clean":"2024-09-12T03:23:20.685867+0000","last_became_active":"2024-09-12T03:23:20.684531+0000","last_became_peered":"2024-09-12T03:23:20.684531+0000","last_unstale":"2024-09-12T03:23:20.685867+0000","last_undegraded":"2024-09-12T03:23:20.685867+0000","last_fullsized":"2024-09-12T03:23:20.685867+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_clean_scrub_stamp":"2024-09-12T03:23:19.659665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:11:40.963697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.3","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.675175+0000","last_change":"2024-09-12T03:23:20.675175+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.675175+0000","last_undegraded":"2024-09-12T03:23:20.675175+0000","last_fullsized":"2024-09-12T03:23:20.675175+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.2","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-12T03:23:20.685296+0000","last_change":"2024-09-12T03:23:20.681981+0000","last_active":"2024-09-12T03:23:20.685296+0000","last_peered":"2024-09-12T03:23:20.685296+0000","last_clean":"2024-09-12T03:23:20.685296+0000","last_became_active":"2024-09-12T03:23:20.681487+0000","last_became_peered":"2024-09-12T03:23:20.681487+0000","last_unstale":"2024-09-12T03:23:20.685296+0000","last_undegraded":"2024-09-12T03:23:20.685296+0000","last_fullsized":"2024-09-12T03:23:20.685296+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_clean_scrub_stamp":"2024-09-12T03:23:19.659665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:52:54.619581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.0","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.675295+0000","last_change":"2024-09-12T03:23:20.675295+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.675295+0000","last_undegraded":"2024-09-12T03:23:20.675295+0000","last_fullsized":"2024-09-12T03:23:20.675295+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.1","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-12T03:23:20.682875+0000","last_change":"2024-09-12T03:23:20.682237+0000","last_active":"2024-09-12T03:23:20.682875+0000","last_peered":"2024-09-12T03:23:20.682875+0000","last_clean":"2024-09-12T03:23:20.682875+0000","last_became_active":"2024-09-12T03:23:20.681397+0000","last_became_peered":"2024-09-12T03:23:20.681397+0000","last_unstale":"2024-09-12T03:23:20.682875+0000","last_undegraded":"2024-09-12T03:23:20.682875+0000","last_fullsized":"2024-09-12T03:23:20.682875+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_clean_scrub_stamp":"2024-09-12T03:23:19.659665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:16:33.772644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.1","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.680888+0000","last_change":"2024-09-12T03:23:20.680888+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.680888+0000","last_undegraded":"2024-09-12T03:23:20.680888+0000","last_fullsized":"2024-09-12T03:23:20.680888+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.0","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-12T03:23:20.695121+0000","last_change":"2024-09-12T03:23:20.688059+0000","last_active":"2024-09-12T03:23:20.695121+0000","last_peered":"2024-09-12T03:23:20.695121+0000","last_clean":"2024-09-12T03:23:20.695121+0000","last_became_active":"2024-09-12T03:23:20.687774+0000","last_became_peered":"2024-09-12T03:23:20.687774+0000","last_unstale":"2024-09-12T03:23:20.695121+0000","last_undegraded":"2024-09-12T03:23:20.695121+0000","last_fullsized":"2024-09-12T03:23:20.695121+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_clean_scrub_stamp":"2024-09-12T03:23:19.659665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:31:56.155648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.18","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.678051+0000","last_change":"2024-09-12T03:23:20.678051+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.678051+0000","last_undegraded":"2024-09-12T03:23:20.678051+0000","last_fullsized":"2024-09-12T03:23:20.678051+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.19","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-12T03:23:20.686693+0000","last_change":"2024-09-12T03:23:20.685565+0000","last_active":"2024-09-12T03:23:20.686693+0000","last_peered":"2024-09-12T03:23:20.686693+0000","last_clean":"2024-09-12T03:23:20.686693+0000","last_became_active":"2024-09-12T03:23:20.685230+0000","last_became_peered":"2024-09-12T03:23:20.685230+0000","last_unstale":"2024-09-12T03:23:20.686693+0000","last_undegraded":"2024-09-12T03:23:20.686693+0000","last_fullsized":"2024-09-12T03:23:20.686693+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_clean_scrub_stamp":"2024-09-12T03:23:19.659665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:25:00.890612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"70'128","reported_seq":441,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-12T03:23:20.680128+0000","last_change":"2024-09-12T03:16:21.883446+0000","last_active":"2024-09-12T03:23:20.680128+0000","last_peered":"2024-09-12T03:23:20.680128+0000","last_clean":"2024-09-12T03:23:20.680128+0000","last_became_active":"2024-09-12T03:16:21.882207+0000","last_became_peered":"2024-09-12T03:16:21.882207+0000","last_unstale":"2024-09-12T03:23:20.680128+0000","last_undegraded":"2024-09-12T03:23:20.680128+0000","last_fullsized":"2024-09-12T03:23:20.680128+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:20.316426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:20.316426+0000","last_clean_scrub_stamp":"2024-09-12T03:16:20.316426+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:12:15.424320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":25,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":24,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542230,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37736,"kb_used_data":2388,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334104,"statfs":{"total":96636764160,"available":96598122496,"internally_reserved":0,"allocated":2445312,"data_stored":1407765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542232,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37148,"kb_used_data":1796,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334692,"statfs":{"total":96636764160,"available":96598724608,"internally_reserved":0,"allocated":1839104,"data_stored":811066,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542231,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37156,"kb_used_data":1808,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334684,"statfs":{"total":96636764160,"available":96598716416,"internally_reserved":0,"allocated":1851392,"data_stored":817397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542230,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37160,"kb_used_data":1808,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334680,"statfs":{"total":96636764160,"available":96598712320,"internally_reserved":0,"allocated":1851392,"data_stored":817397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542231,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32996,"kb_used_data":1796,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338844,"statfs":{"total":96636764160,"available":96602976256,"internally_reserved":0,"allocated":1839104,"data_stored":811066,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542231,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32984,"kb_used_data":1796,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338856,"statfs":{"total":96636764160,"available":96602988544,"internally_reserved":0,"allocated":1839104,"data_stored":811066,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542232,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32996,"kb_used_data":1808,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338844,"statfs":{"total":96636764160,"available":96602976256,"internally_reserved":0,"allocated":1851392,"data_stored":817397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542232,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33580,"kb_used_data":2388,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338260,"statfs":{"total":96636764160,"available":96602378240,"internally_reserved":0,"allocated":2445312,"data_stored":1407765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T03:23:26.040 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-12T03:23:26.179 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.178+0000 7f79ce050640 1 Processor -- start 2024-09-12T03:23:26.180 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.179+0000 7f79ce050640 1 -- start start 2024-09-12T03:23:26.181 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.179+0000 7f79ce050640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f79c80d5200 0x7f79c80d1010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:26.182 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.179+0000 7f79ce050640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f79c80d1550 0x7f79c80b8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:26.182 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.179+0000 7f79ce050640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f79c80d19c0 con 0x7f79c815eb90 2024-09-12T03:23:26.182 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.179+0000 7f79ce050640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f79c80b7d20 con 0x7f79c80d1550 2024-09-12T03:23:26.182 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.179+0000 7f79ce050640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f79c80d42e0 con 0x7f79c80d5200 2024-09-12T03:23:26.182 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.179+0000 7f79c77fe640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f79c815eb90 0x7f79c80d2df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:56136/0 (socket says 172.21.15.27:56136) 2024-09-12T03:23:26.182 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.180+0000 7f79c77fe640 1 -- 172.21.15.27:0/3681661554 learned_addr learned my addr 172.21.15.27:0/3681661554 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:23:26.182 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.180+0000 7f79c7fff640 1 --2- 172.21.15.27:0/3681661554 >> v2:172.21.15.27:3300/0 conn(0x7f79c80d1550 0x7f79c80b8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:26.182 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.180+0000 7f79c6ffd640 1 --2- 172.21.15.27:0/3681661554 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f79c80d5200 0x7f79c80d1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:26.182 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.180+0000 7f79c67fc640 1 -- 172.21.15.27:0/3681661554 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 215142048 0 0) 0x7f79c80d19c0 con 0x7f79c815eb90 2024-09-12T03:23:26.182 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.180+0000 7f79c67fc640 1 -- 172.21.15.27:0/3681661554 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f79b0003520 con 0x7f79c815eb90 2024-09-12T03:23:26.182 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.180+0000 7f79c7fff640 1 -- 172.21.15.27:0/3681661554 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f79c80d5200 msgr2=0x7f79c80d1010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:26.182 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.180+0000 7f79c7fff640 1 --2- 172.21.15.27:0/3681661554 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f79c80d5200 0x7f79c80d1010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:26.182 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.180+0000 7f79c7fff640 1 -- 172.21.15.27:0/3681661554 >> v1:172.21.15.27:6789/0 conn(0x7f79c815eb90 legacy=0x7f79c80d2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:26.183 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.180+0000 7f79c7fff640 1 -- 172.21.15.27:0/3681661554 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79c80d3760 con 0x7f79c80d1550 2024-09-12T03:23:26.183 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.180+0000 7f79c6ffd640 1 --2- 172.21.15.27:0/3681661554 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f79c80d5200 0x7f79c80d1010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:23:26.183 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.181+0000 7f79c7fff640 1 --2- 172.21.15.27:0/3681661554 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f79c80d1550 0x7f79c80b8190 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7f79c00097a0 tx=0x7f79c002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc9c99f4681010f5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:26.183 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.181+0000 7f79c67fc640 1 -- 172.21.15.27:0/3681661554 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79c0037070 con 0x7f79c80d1550 2024-09-12T03:23:26.183 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.181+0000 7f79c67fc640 1 -- 172.21.15.27:0/3681661554 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79c0032440 con 0x7f79c80d1550 2024-09-12T03:23:26.183 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.181+0000 7f79c67fc640 1 -- 172.21.15.27:0/3681661554 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79c0032810 con 0x7f79c80d1550 2024-09-12T03:23:26.184 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.181+0000 7f79ce050640 1 -- 172.21.15.27:0/3681661554 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f79c80d1550 msgr2=0x7f79c80b8190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:26.184 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.181+0000 7f79ce050640 1 --2- 172.21.15.27:0/3681661554 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f79c80d1550 0x7f79c80b8190 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7f79c00097a0 tx=0x7f79c002a180 comp rx=0 tx=0).stop 2024-09-12T03:23:26.184 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.181+0000 7f79ce050640 1 -- 172.21.15.27:0/3681661554 shutdown_connections 2024-09-12T03:23:26.184 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.181+0000 7f79ce050640 1 --2- 172.21.15.27:0/3681661554 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f79c80d1550 0x7f79c80b8190 unknown :-1 s=CLOSED pgs=1549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:26.184 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.181+0000 7f79ce050640 1 --2- 172.21.15.27:0/3681661554 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f79c80d5200 0x7f79c80d1010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:26.184 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.181+0000 7f79ce050640 1 -- 172.21.15.27:0/3681661554 >> 172.21.15.27:0/3681661554 conn(0x7f79c806dce0 msgr2=0x7f79c806e0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:26.184 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.181+0000 7f79ce050640 1 -- 172.21.15.27:0/3681661554 shutdown_connections 2024-09-12T03:23:26.184 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.182+0000 7f79ce050640 1 -- 172.21.15.27:0/3681661554 wait complete. 2024-09-12T03:23:26.185 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.183+0000 7f79ce050640 1 Processor -- start 2024-09-12T03:23:26.185 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.183+0000 7f79ce050640 1 -- start start 2024-09-12T03:23:26.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.184+0000 7f79ce050640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f79c80d1550 0x7f79c80ca780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:26.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.184+0000 7f79ce050640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f79c80d5200 0x7f79c81f4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:26.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.184+0000 7f79ce050640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f79c815eb90 0x7f79c81f5600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:26.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.184+0000 7f79ce050640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f79c80d3d20 con 0x7f79c80d1550 2024-09-12T03:23:26.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.184+0000 7f79ce050640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f79c80d3500 con 0x7f79c80d5200 2024-09-12T03:23:26.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.184+0000 7f79ce050640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f79c80d4000 con 0x7f79c815eb90 2024-09-12T03:23:26.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.184+0000 7f79c77fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f79c80d1550 0x7f79c80ca780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:26.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.184+0000 7f79c77fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f79c80d1550 0x7f79c80ca780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42490/0 (socket says 172.21.15.27:42490) 2024-09-12T03:23:26.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.184+0000 7f79c77fe640 1 -- 172.21.15.27:0/1756570810 learned_addr learned my addr 172.21.15.27:0/1756570810 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:23:26.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.184+0000 7f79c7fff640 1 --2- 172.21.15.27:0/1756570810 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f79c815eb90 0x7f79c81f5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:26.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.184+0000 7f79c6ffd640 1 --2- 172.21.15.27:0/1756570810 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f79c80d5200 0x7f79c81f4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:26.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.184+0000 7f79c77fe640 1 -- 172.21.15.27:0/1756570810 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f79c815eb90 msgr2=0x7f79c81f5600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:26.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.184+0000 7f79c77fe640 1 --2- 172.21.15.27:0/1756570810 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f79c815eb90 0x7f79c81f5600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:26.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.184+0000 7f79c77fe640 1 -- 172.21.15.27:0/1756570810 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f79c80d5200 msgr2=0x7f79c81f4ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:26.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.184+0000 7f79c77fe640 1 --2- 172.21.15.27:0/1756570810 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f79c80d5200 0x7f79c81f4ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:26.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.184+0000 7f79c77fe640 1 -- 172.21.15.27:0/1756570810 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79c80c8520 con 0x7f79c80d1550 2024-09-12T03:23:26.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.184+0000 7f79c6ffd640 1 --2- 172.21.15.27:0/1756570810 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f79c80d5200 0x7f79c81f4ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:23:26.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.184+0000 7f79c7fff640 1 --2- 172.21.15.27:0/1756570810 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f79c815eb90 0x7f79c81f5600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:23:26.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.184+0000 7f79c77fe640 1 --2- 172.21.15.27:0/1756570810 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f79c80d1550 0x7f79c80ca780 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7f79b800caa0 tx=0x7f79b800cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:26.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.184+0000 7f79affff640 1 -- 172.21.15.27:0/1756570810 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79b8017070 con 0x7f79c80d1550 2024-09-12T03:23:26.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.184+0000 7f79affff640 1 -- 172.21.15.27:0/1756570810 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79b800ac00 con 0x7f79c80d1550 2024-09-12T03:23:26.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.184+0000 7f79ce050640 1 -- 172.21.15.27:0/1756570810 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79c80c8800 con 0x7f79c80d1550 2024-09-12T03:23:26.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.184+0000 7f79affff640 1 -- 172.21.15.27:0/1756570810 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79b8012400 con 0x7f79c80d1550 2024-09-12T03:23:26.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.184+0000 7f79ce050640 1 -- 172.21.15.27:0/1756570810 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79c80c8dc0 con 0x7f79c80d1550 2024-09-12T03:23:26.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.185+0000 7f79ce050640 1 -- 172.21.15.27:0/1756570810 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79c80d5630 con 0x7f79c80d1550 2024-09-12T03:23:26.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.186+0000 7f79affff640 1 -- 172.21.15.27:0/1756570810 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 24) ==== 91212+0+0 (secure 0 0 0) 0x7f79b801a070 con 0x7f79c80d1550 2024-09-12T03:23:26.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.186+0000 7f79affff640 1 --2- 172.21.15.27:0/1756570810 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f79b406c610 0x7f79b406ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:26.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.187+0000 7f79affff640 1 -- 172.21.15.27:0/1756570810 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 5765+0+0 (secure 0 0 0) 0x7f79b8089640 con 0x7f79c80d1550 2024-09-12T03:23:26.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.187+0000 7f79c6ffd640 1 --2- 172.21.15.27:0/1756570810 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f79b406c610 0x7f79b406ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:26.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.187+0000 7f79c6ffd640 1 --2- 172.21.15.27:0/1756570810 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f79b406c610 0x7f79b406ead0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f79c8160190 tx=0x7f79bc008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:26.191 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.191+0000 7f79affff640 1 -- 172.21.15.27:0/1756570810 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f79c80d5630 con 0x7f79c80d1550 2024-09-12T03:23:26.365 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.364+0000 7f79ce050640 1 -- 172.21.15.27:0/1756570810 --> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f79c815f0e0 con 0x7f79b406c610 2024-09-12T03:23:26.371 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.370+0000 7f79affff640 1 -- 172.21.15.27:0/1756570810 <== mgr.4113 v2:172.21.15.27:6836/1226920526 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247090 (secure 0 0 0) 0x7f79c815f0e0 con 0x7f79b406c610 2024-09-12T03:23:26.376 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:23:26.378 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2024-09-12T03:23:26.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.373+0000 7f79ce050640 1 -- 172.21.15.27:0/1756570810 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f79b406c610 msgr2=0x7f79b406ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:26.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.373+0000 7f79ce050640 1 --2- 172.21.15.27:0/1756570810 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f79b406c610 0x7f79b406ead0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f79c8160190 tx=0x7f79bc008040 comp rx=0 tx=0).stop 2024-09-12T03:23:26.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.373+0000 7f79ce050640 1 -- 172.21.15.27:0/1756570810 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f79c80d1550 msgr2=0x7f79c80ca780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:26.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.373+0000 7f79ce050640 1 --2- 172.21.15.27:0/1756570810 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f79c80d1550 0x7f79c80ca780 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7f79b800caa0 tx=0x7f79b800cf60 comp rx=0 tx=0).stop 2024-09-12T03:23:26.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.373+0000 7f79ce050640 1 -- 172.21.15.27:0/1756570810 shutdown_connections 2024-09-12T03:23:26.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.373+0000 7f79ce050640 1 --2- 172.21.15.27:0/1756570810 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f79c815eb90 0x7f79c81f5600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:26.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.373+0000 7f79ce050640 1 --2- 172.21.15.27:0/1756570810 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f79c80d5200 0x7f79c81f4ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:26.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.373+0000 7f79ce050640 1 --2- 172.21.15.27:0/1756570810 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f79b406c610 0x7f79b406ead0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:26.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.373+0000 7f79ce050640 1 --2- 172.21.15.27:0/1756570810 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f79c80d1550 0x7f79c80ca780 unknown :-1 s=CLOSED pgs=1550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:26.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.373+0000 7f79ce050640 1 -- 172.21.15.27:0/1756570810 >> 172.21.15.27:0/1756570810 conn(0x7f79c806dce0 msgr2=0x7f79c8161080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:26.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.374+0000 7f79ce050640 1 -- 172.21.15.27:0/1756570810 shutdown_connections 2024-09-12T03:23:26.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.374+0000 7f79ce050640 1 -- 172.21.15.27:0/1756570810 wait complete. 2024-09-12T03:23:26.385 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":178,"stamp":"2024-09-12T03:23:25.134647+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":41,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":281756,"kb_used_data":15588,"kb_used_omap":12,"kb_used_meta":265971,"kb_avail":754692964,"statfs":{"total":773094113280,"available":772805595136,"internally_reserved":0,"allocated":15962112,"data_stored":7700919,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":272354864},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":15,"apply_latency_ms":15,"commit_latency_ns":15000000,"apply_latency_ns":15000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.468635"},"pg_stats":[{"pgid":"25.3f","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.676683+0000","last_change":"2024-09-12T03:23:20.676683+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.676683+0000","last_undegraded":"2024-09-12T03:23:20.676683+0000","last_fullsized":"2024-09-12T03:23:20.676683+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.3e","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.677002+0000","last_change":"2024-09-12T03:23:20.677002+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.677002+0000","last_undegraded":"2024-09-12T03:23:20.677002+0000","last_fullsized":"2024-09-12T03:23:20.677002+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.3d","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.680808+0000","last_change":"2024-09-12T03:23:20.680808+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.680808+0000","last_undegraded":"2024-09-12T03:23:20.680808+0000","last_fullsized":"2024-09-12T03:23:20.680808+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.3c","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.677777+0000","last_change":"2024-09-12T03:23:20.677777+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.677777+0000","last_undegraded":"2024-09-12T03:23:20.677777+0000","last_fullsized":"2024-09-12T03:23:20.677777+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.3b","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.683306+0000","last_change":"2024-09-12T03:23:20.683306+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.683306+0000","last_undegraded":"2024-09-12T03:23:20.683306+0000","last_fullsized":"2024-09-12T03:23:20.683306+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.3a","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.680330+0000","last_change":"2024-09-12T03:23:20.680330+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.680330+0000","last_undegraded":"2024-09-12T03:23:20.680330+0000","last_fullsized":"2024-09-12T03:23:20.680330+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.39","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.672348+0000","last_change":"2024-09-12T03:23:20.672348+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.672348+0000","last_undegraded":"2024-09-12T03:23:20.672348+0000","last_fullsized":"2024-09-12T03:23:20.672348+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.38","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.676156+0000","last_change":"2024-09-12T03:23:20.676156+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.676156+0000","last_undegraded":"2024-09-12T03:23:20.676156+0000","last_fullsized":"2024-09-12T03:23:20.676156+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.37","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.674337+0000","last_change":"2024-09-12T03:23:20.674337+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.674337+0000","last_undegraded":"2024-09-12T03:23:20.674337+0000","last_fullsized":"2024-09-12T03:23:20.674337+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.36","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.679738+0000","last_change":"2024-09-12T03:23:20.679738+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.679738+0000","last_undegraded":"2024-09-12T03:23:20.679738+0000","last_fullsized":"2024-09-12T03:23:20.679738+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.35","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.671126+0000","last_change":"2024-09-12T03:23:20.671126+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.671126+0000","last_undegraded":"2024-09-12T03:23:20.671126+0000","last_fullsized":"2024-09-12T03:23:20.671126+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.34","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.677214+0000","last_change":"2024-09-12T03:23:20.677214+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.677214+0000","last_undegraded":"2024-09-12T03:23:20.677214+0000","last_fullsized":"2024-09-12T03:23:20.677214+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.33","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.677238+0000","last_change":"2024-09-12T03:23:20.677238+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.677238+0000","last_undegraded":"2024-09-12T03:23:20.677238+0000","last_fullsized":"2024-09-12T03:23:20.677238+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.32","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.675036+0000","last_change":"2024-09-12T03:23:20.675036+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.675036+0000","last_undegraded":"2024-09-12T03:23:20.675036+0000","last_fullsized":"2024-09-12T03:23:20.675036+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.31","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.675841+0000","last_change":"2024-09-12T03:23:20.675841+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.675841+0000","last_undegraded":"2024-09-12T03:23:20.675841+0000","last_fullsized":"2024-09-12T03:23:20.675841+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.30","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.675750+0000","last_change":"2024-09-12T03:23:20.675750+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.675750+0000","last_undegraded":"2024-09-12T03:23:20.675750+0000","last_fullsized":"2024-09-12T03:23:20.675750+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.2f","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.687993+0000","last_change":"2024-09-12T03:23:20.687993+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.687993+0000","last_undegraded":"2024-09-12T03:23:20.687993+0000","last_fullsized":"2024-09-12T03:23:20.687993+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.2e","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.676010+0000","last_change":"2024-09-12T03:23:20.676010+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.676010+0000","last_undegraded":"2024-09-12T03:23:20.676010+0000","last_fullsized":"2024-09-12T03:23:20.676010+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.2d","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.680215+0000","last_change":"2024-09-12T03:23:20.680215+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.680215+0000","last_undegraded":"2024-09-12T03:23:20.680215+0000","last_fullsized":"2024-09-12T03:23:20.680215+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.2c","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.685697+0000","last_change":"2024-09-12T03:23:20.685697+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.685697+0000","last_undegraded":"2024-09-12T03:23:20.685697+0000","last_fullsized":"2024-09-12T03:23:20.685697+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.2b","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.682781+0000","last_change":"2024-09-12T03:23:20.682781+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.682781+0000","last_undegraded":"2024-09-12T03:23:20.682781+0000","last_fullsized":"2024-09-12T03:23:20.682781+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.2a","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.679222+0000","last_change":"2024-09-12T03:23:20.679222+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.679222+0000","last_undegraded":"2024-09-12T03:23:20.679222+0000","last_fullsized":"2024-09-12T03:23:20.679222+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.29","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.679592+0000","last_change":"2024-09-12T03:23:20.679592+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.679592+0000","last_undegraded":"2024-09-12T03:23:20.679592+0000","last_fullsized":"2024-09-12T03:23:20.679592+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.28","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.675319+0000","last_change":"2024-09-12T03:23:20.675319+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.675319+0000","last_undegraded":"2024-09-12T03:23:20.675319+0000","last_fullsized":"2024-09-12T03:23:20.675319+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.27","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.676115+0000","last_change":"2024-09-12T03:23:20.676115+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.676115+0000","last_undegraded":"2024-09-12T03:23:20.676115+0000","last_fullsized":"2024-09-12T03:23:20.676115+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.26","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.671542+0000","last_change":"2024-09-12T03:23:20.671542+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.671542+0000","last_undegraded":"2024-09-12T03:23:20.671542+0000","last_fullsized":"2024-09-12T03:23:20.671542+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.25","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.676098+0000","last_change":"2024-09-12T03:23:20.676098+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.676098+0000","last_undegraded":"2024-09-12T03:23:20.676098+0000","last_fullsized":"2024-09-12T03:23:20.676098+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.24","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.685181+0000","last_change":"2024-09-12T03:23:20.685181+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.685181+0000","last_undegraded":"2024-09-12T03:23:20.685181+0000","last_fullsized":"2024-09-12T03:23:20.685181+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.23","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.676786+0000","last_change":"2024-09-12T03:23:20.676786+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.676786+0000","last_undegraded":"2024-09-12T03:23:20.676786+0000","last_fullsized":"2024-09-12T03:23:20.676786+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.22","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.674611+0000","last_change":"2024-09-12T03:23:20.674611+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.674611+0000","last_undegraded":"2024-09-12T03:23:20.674611+0000","last_fullsized":"2024-09-12T03:23:20.674611+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.21","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.676607+0000","last_change":"2024-09-12T03:23:20.676607+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.676607+0000","last_undegraded":"2024-09-12T03:23:20.676607+0000","last_fullsized":"2024-09-12T03:23:20.676607+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.20","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.682292+0000","last_change":"2024-09-12T03:23:20.682292+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.682292+0000","last_undegraded":"2024-09-12T03:23:20.682292+0000","last_fullsized":"2024-09-12T03:23:20.682292+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1e","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.678929+0000","last_change":"2024-09-12T03:23:20.678929+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.678929+0000","last_undegraded":"2024-09-12T03:23:20.678929+0000","last_fullsized":"2024-09-12T03:23:20.678929+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1f","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-12T03:23:20.688172+0000","last_change":"2024-09-12T03:23:20.682141+0000","last_active":"2024-09-12T03:23:20.688172+0000","last_peered":"2024-09-12T03:23:20.688172+0000","last_clean":"2024-09-12T03:23:20.688172+0000","last_became_active":"2024-09-12T03:23:20.681852+0000","last_became_peered":"2024-09-12T03:23:20.681852+0000","last_unstale":"2024-09-12T03:23:20.688172+0000","last_undegraded":"2024-09-12T03:23:20.688172+0000","last_fullsized":"2024-09-12T03:23:20.688172+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_clean_scrub_stamp":"2024-09-12T03:23:19.659665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:20:17.004278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1f","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.678598+0000","last_change":"2024-09-12T03:23:20.678598+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.678598+0000","last_undegraded":"2024-09-12T03:23:20.678598+0000","last_fullsized":"2024-09-12T03:23:20.678598+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.1e","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-12T03:23:20.687095+0000","last_change":"2024-09-12T03:23:20.686262+0000","last_active":"2024-09-12T03:23:20.687095+0000","last_peered":"2024-09-12T03:23:20.687095+0000","last_clean":"2024-09-12T03:23:20.687095+0000","last_became_active":"2024-09-12T03:23:20.685552+0000","last_became_peered":"2024-09-12T03:23:20.685552+0000","last_unstale":"2024-09-12T03:23:20.687095+0000","last_undegraded":"2024-09-12T03:23:20.687095+0000","last_fullsized":"2024-09-12T03:23:20.687095+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_clean_scrub_stamp":"2024-09-12T03:23:19.659665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:20:51.463260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.1c","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.672426+0000","last_change":"2024-09-12T03:23:20.672426+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.672426+0000","last_undegraded":"2024-09-12T03:23:20.672426+0000","last_fullsized":"2024-09-12T03:23:20.672426+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.1d","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-12T03:23:20.695515+0000","last_change":"2024-09-12T03:23:20.690566+0000","last_active":"2024-09-12T03:23:20.695515+0000","last_peered":"2024-09-12T03:23:20.695515+0000","last_clean":"2024-09-12T03:23:20.695515+0000","last_became_active":"2024-09-12T03:23:20.690296+0000","last_became_peered":"2024-09-12T03:23:20.690296+0000","last_unstale":"2024-09-12T03:23:20.695515+0000","last_undegraded":"2024-09-12T03:23:20.695515+0000","last_fullsized":"2024-09-12T03:23:20.695515+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_clean_scrub_stamp":"2024-09-12T03:23:19.659665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:13:40.613414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1d","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.672931+0000","last_change":"2024-09-12T03:23:20.672931+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.672931+0000","last_undegraded":"2024-09-12T03:23:20.672931+0000","last_fullsized":"2024-09-12T03:23:20.672931+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.1c","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-12T03:23:20.687327+0000","last_change":"2024-09-12T03:23:20.686667+0000","last_active":"2024-09-12T03:23:20.687327+0000","last_peered":"2024-09-12T03:23:20.687327+0000","last_clean":"2024-09-12T03:23:20.687327+0000","last_became_active":"2024-09-12T03:23:20.686355+0000","last_became_peered":"2024-09-12T03:23:20.686355+0000","last_unstale":"2024-09-12T03:23:20.687327+0000","last_undegraded":"2024-09-12T03:23:20.687327+0000","last_fullsized":"2024-09-12T03:23:20.687327+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_clean_scrub_stamp":"2024-09-12T03:23:19.659665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:01:34.049378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.1a","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.678590+0000","last_change":"2024-09-12T03:23:20.678590+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.678590+0000","last_undegraded":"2024-09-12T03:23:20.678590+0000","last_fullsized":"2024-09-12T03:23:20.678590+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.1b","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-12T03:23:20.686092+0000","last_change":"2024-09-12T03:23:20.684507+0000","last_active":"2024-09-12T03:23:20.686092+0000","last_peered":"2024-09-12T03:23:20.686092+0000","last_clean":"2024-09-12T03:23:20.686092+0000","last_became_active":"2024-09-12T03:23:20.684246+0000","last_became_peered":"2024-09-12T03:23:20.684246+0000","last_unstale":"2024-09-12T03:23:20.686092+0000","last_undegraded":"2024-09-12T03:23:20.686092+0000","last_fullsized":"2024-09-12T03:23:20.686092+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_clean_scrub_stamp":"2024-09-12T03:23:19.659665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:39:13.426726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1b","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.674380+0000","last_change":"2024-09-12T03:23:20.674380+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.674380+0000","last_undegraded":"2024-09-12T03:23:20.674380+0000","last_fullsized":"2024-09-12T03:23:20.674380+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1a","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-12T03:23:20.697734+0000","last_change":"2024-09-12T03:23:20.696117+0000","last_active":"2024-09-12T03:23:20.697734+0000","last_peered":"2024-09-12T03:23:20.697734+0000","last_clean":"2024-09-12T03:23:20.697734+0000","last_became_active":"2024-09-12T03:23:20.695795+0000","last_became_peered":"2024-09-12T03:23:20.695795+0000","last_unstale":"2024-09-12T03:23:20.697734+0000","last_undegraded":"2024-09-12T03:23:20.697734+0000","last_fullsized":"2024-09-12T03:23:20.697734+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_clean_scrub_stamp":"2024-09-12T03:23:19.659665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:36:38.656553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.19","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.671879+0000","last_change":"2024-09-12T03:23:20.671879+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.671879+0000","last_undegraded":"2024-09-12T03:23:20.671879+0000","last_fullsized":"2024-09-12T03:23:20.671879+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.18","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-12T03:23:20.690256+0000","last_change":"2024-09-12T03:23:20.689802+0000","last_active":"2024-09-12T03:23:20.690256+0000","last_peered":"2024-09-12T03:23:20.690256+0000","last_clean":"2024-09-12T03:23:20.690256+0000","last_became_active":"2024-09-12T03:23:20.689503+0000","last_became_peered":"2024-09-12T03:23:20.689503+0000","last_unstale":"2024-09-12T03:23:20.690256+0000","last_undegraded":"2024-09-12T03:23:20.690256+0000","last_fullsized":"2024-09-12T03:23:20.690256+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_clean_scrub_stamp":"2024-09-12T03:23:19.659665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:29:59.930252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.16","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.675332+0000","last_change":"2024-09-12T03:23:20.675332+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.675332+0000","last_undegraded":"2024-09-12T03:23:20.675332+0000","last_fullsized":"2024-09-12T03:23:20.675332+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.17","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-12T03:23:20.679242+0000","last_change":"2024-09-12T03:23:20.678720+0000","last_active":"2024-09-12T03:23:20.679242+0000","last_peered":"2024-09-12T03:23:20.679242+0000","last_clean":"2024-09-12T03:23:20.679242+0000","last_became_active":"2024-09-12T03:23:20.678415+0000","last_became_peered":"2024-09-12T03:23:20.678415+0000","last_unstale":"2024-09-12T03:23:20.679242+0000","last_undegraded":"2024-09-12T03:23:20.679242+0000","last_fullsized":"2024-09-12T03:23:20.679242+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_clean_scrub_stamp":"2024-09-12T03:23:19.659665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:22:49.298961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.17","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.674869+0000","last_change":"2024-09-12T03:23:20.674869+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.674869+0000","last_undegraded":"2024-09-12T03:23:20.674869+0000","last_fullsized":"2024-09-12T03:23:20.674869+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.16","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-12T03:23:20.686685+0000","last_change":"2024-09-12T03:23:20.683066+0000","last_active":"2024-09-12T03:23:20.686685+0000","last_peered":"2024-09-12T03:23:20.686685+0000","last_clean":"2024-09-12T03:23:20.686685+0000","last_became_active":"2024-09-12T03:23:20.682759+0000","last_became_peered":"2024-09-12T03:23:20.682759+0000","last_unstale":"2024-09-12T03:23:20.686685+0000","last_undegraded":"2024-09-12T03:23:20.686685+0000","last_fullsized":"2024-09-12T03:23:20.686685+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_clean_scrub_stamp":"2024-09-12T03:23:19.659665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:51:25.370412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.14","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.671943+0000","last_change":"2024-09-12T03:23:20.671943+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.671943+0000","last_undegraded":"2024-09-12T03:23:20.671943+0000","last_fullsized":"2024-09-12T03:23:20.671943+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.15","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-12T03:23:20.693567+0000","last_change":"2024-09-12T03:23:20.691649+0000","last_active":"2024-09-12T03:23:20.693567+0000","last_peered":"2024-09-12T03:23:20.693567+0000","last_clean":"2024-09-12T03:23:20.693567+0000","last_became_active":"2024-09-12T03:23:20.691385+0000","last_became_peered":"2024-09-12T03:23:20.691385+0000","last_unstale":"2024-09-12T03:23:20.693567+0000","last_undegraded":"2024-09-12T03:23:20.693567+0000","last_fullsized":"2024-09-12T03:23:20.693567+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_clean_scrub_stamp":"2024-09-12T03:23:19.659665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:51:15.001810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.15","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.677434+0000","last_change":"2024-09-12T03:23:20.677434+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.677434+0000","last_undegraded":"2024-09-12T03:23:20.677434+0000","last_fullsized":"2024-09-12T03:23:20.677434+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.14","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-12T03:23:20.684819+0000","last_change":"2024-09-12T03:23:20.683263+0000","last_active":"2024-09-12T03:23:20.684819+0000","last_peered":"2024-09-12T03:23:20.684819+0000","last_clean":"2024-09-12T03:23:20.684819+0000","last_became_active":"2024-09-12T03:23:20.682986+0000","last_became_peered":"2024-09-12T03:23:20.682986+0000","last_unstale":"2024-09-12T03:23:20.684819+0000","last_undegraded":"2024-09-12T03:23:20.684819+0000","last_fullsized":"2024-09-12T03:23:20.684819+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_clean_scrub_stamp":"2024-09-12T03:23:19.659665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:17:57.817757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.12","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.676361+0000","last_change":"2024-09-12T03:23:20.676361+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.676361+0000","last_undegraded":"2024-09-12T03:23:20.676361+0000","last_fullsized":"2024-09-12T03:23:20.676361+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.13","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-12T03:23:20.690520+0000","last_change":"2024-09-12T03:23:20.689845+0000","last_active":"2024-09-12T03:23:20.690520+0000","last_peered":"2024-09-12T03:23:20.690520+0000","last_clean":"2024-09-12T03:23:20.690520+0000","last_became_active":"2024-09-12T03:23:20.689485+0000","last_became_peered":"2024-09-12T03:23:20.689485+0000","last_unstale":"2024-09-12T03:23:20.690520+0000","last_undegraded":"2024-09-12T03:23:20.690520+0000","last_fullsized":"2024-09-12T03:23:20.690520+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_clean_scrub_stamp":"2024-09-12T03:23:19.659665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:41:28.835418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.13","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.672076+0000","last_change":"2024-09-12T03:23:20.672076+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.672076+0000","last_undegraded":"2024-09-12T03:23:20.672076+0000","last_fullsized":"2024-09-12T03:23:20.672076+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.12","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-12T03:23:20.692153+0000","last_change":"2024-09-12T03:23:20.689243+0000","last_active":"2024-09-12T03:23:20.692153+0000","last_peered":"2024-09-12T03:23:20.692153+0000","last_clean":"2024-09-12T03:23:20.692153+0000","last_became_active":"2024-09-12T03:23:20.688661+0000","last_became_peered":"2024-09-12T03:23:20.688661+0000","last_unstale":"2024-09-12T03:23:20.692153+0000","last_undegraded":"2024-09-12T03:23:20.692153+0000","last_fullsized":"2024-09-12T03:23:20.692153+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_clean_scrub_stamp":"2024-09-12T03:23:19.659665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:16:21.498607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.10","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.672680+0000","last_change":"2024-09-12T03:23:20.672680+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.672680+0000","last_undegraded":"2024-09-12T03:23:20.672680+0000","last_fullsized":"2024-09-12T03:23:20.672680+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.11","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-12T03:23:20.687225+0000","last_change":"2024-09-12T03:23:20.680790+0000","last_active":"2024-09-12T03:23:20.687225+0000","last_peered":"2024-09-12T03:23:20.687225+0000","last_clean":"2024-09-12T03:23:20.687225+0000","last_became_active":"2024-09-12T03:23:20.680508+0000","last_became_peered":"2024-09-12T03:23:20.680508+0000","last_unstale":"2024-09-12T03:23:20.687225+0000","last_undegraded":"2024-09-12T03:23:20.687225+0000","last_fullsized":"2024-09-12T03:23:20.687225+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_clean_scrub_stamp":"2024-09-12T03:23:19.659665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:56:34.246022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.11","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.676970+0000","last_change":"2024-09-12T03:23:20.676970+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.676970+0000","last_undegraded":"2024-09-12T03:23:20.676970+0000","last_fullsized":"2024-09-12T03:23:20.676970+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.10","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-12T03:23:20.690498+0000","last_change":"2024-09-12T03:23:20.689843+0000","last_active":"2024-09-12T03:23:20.690498+0000","last_peered":"2024-09-12T03:23:20.690498+0000","last_clean":"2024-09-12T03:23:20.690498+0000","last_became_active":"2024-09-12T03:23:20.689517+0000","last_became_peered":"2024-09-12T03:23:20.689517+0000","last_unstale":"2024-09-12T03:23:20.690498+0000","last_undegraded":"2024-09-12T03:23:20.690498+0000","last_fullsized":"2024-09-12T03:23:20.690498+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_clean_scrub_stamp":"2024-09-12T03:23:19.659665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:27:53.769921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.e","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.678489+0000","last_change":"2024-09-12T03:23:20.678489+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.678489+0000","last_undegraded":"2024-09-12T03:23:20.678489+0000","last_fullsized":"2024-09-12T03:23:20.678489+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.f","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-12T03:23:20.692368+0000","last_change":"2024-09-12T03:23:20.689669+0000","last_active":"2024-09-12T03:23:20.692368+0000","last_peered":"2024-09-12T03:23:20.692368+0000","last_clean":"2024-09-12T03:23:20.692368+0000","last_became_active":"2024-09-12T03:23:20.689291+0000","last_became_peered":"2024-09-12T03:23:20.689291+0000","last_unstale":"2024-09-12T03:23:20.692368+0000","last_undegraded":"2024-09-12T03:23:20.692368+0000","last_fullsized":"2024-09-12T03:23:20.692368+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_clean_scrub_stamp":"2024-09-12T03:23:19.659665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:25:29.729322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.f","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.673914+0000","last_change":"2024-09-12T03:23:20.673914+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.673914+0000","last_undegraded":"2024-09-12T03:23:20.673914+0000","last_fullsized":"2024-09-12T03:23:20.673914+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.e","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-12T03:23:20.692311+0000","last_change":"2024-09-12T03:23:20.691112+0000","last_active":"2024-09-12T03:23:20.692311+0000","last_peered":"2024-09-12T03:23:20.692311+0000","last_clean":"2024-09-12T03:23:20.692311+0000","last_became_active":"2024-09-12T03:23:20.690862+0000","last_became_peered":"2024-09-12T03:23:20.690862+0000","last_unstale":"2024-09-12T03:23:20.692311+0000","last_undegraded":"2024-09-12T03:23:20.692311+0000","last_fullsized":"2024-09-12T03:23:20.692311+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_clean_scrub_stamp":"2024-09-12T03:23:19.659665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:42:16.029521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.c","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.672469+0000","last_change":"2024-09-12T03:23:20.672469+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.672469+0000","last_undegraded":"2024-09-12T03:23:20.672469+0000","last_fullsized":"2024-09-12T03:23:20.672469+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.d","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-12T03:23:20.688563+0000","last_change":"2024-09-12T03:23:20.688103+0000","last_active":"2024-09-12T03:23:20.688563+0000","last_peered":"2024-09-12T03:23:20.688563+0000","last_clean":"2024-09-12T03:23:20.688563+0000","last_became_active":"2024-09-12T03:23:20.687797+0000","last_became_peered":"2024-09-12T03:23:20.687797+0000","last_unstale":"2024-09-12T03:23:20.688563+0000","last_undegraded":"2024-09-12T03:23:20.688563+0000","last_fullsized":"2024-09-12T03:23:20.688563+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_clean_scrub_stamp":"2024-09-12T03:23:19.659665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:34:39.656922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.d","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.677217+0000","last_change":"2024-09-12T03:23:20.677217+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.677217+0000","last_undegraded":"2024-09-12T03:23:20.677217+0000","last_fullsized":"2024-09-12T03:23:20.677217+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.c","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-12T03:23:20.684552+0000","last_change":"2024-09-12T03:23:20.682322+0000","last_active":"2024-09-12T03:23:20.684552+0000","last_peered":"2024-09-12T03:23:20.684552+0000","last_clean":"2024-09-12T03:23:20.684552+0000","last_became_active":"2024-09-12T03:23:20.682033+0000","last_became_peered":"2024-09-12T03:23:20.682033+0000","last_unstale":"2024-09-12T03:23:20.684552+0000","last_undegraded":"2024-09-12T03:23:20.684552+0000","last_fullsized":"2024-09-12T03:23:20.684552+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_clean_scrub_stamp":"2024-09-12T03:23:19.659665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:42:18.081947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.a","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.676951+0000","last_change":"2024-09-12T03:23:20.676951+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.676951+0000","last_undegraded":"2024-09-12T03:23:20.676951+0000","last_fullsized":"2024-09-12T03:23:20.676951+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.b","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-12T03:23:20.690808+0000","last_change":"2024-09-12T03:23:20.688832+0000","last_active":"2024-09-12T03:23:20.690808+0000","last_peered":"2024-09-12T03:23:20.690808+0000","last_clean":"2024-09-12T03:23:20.690808+0000","last_became_active":"2024-09-12T03:23:20.688021+0000","last_became_peered":"2024-09-12T03:23:20.688021+0000","last_unstale":"2024-09-12T03:23:20.690808+0000","last_undegraded":"2024-09-12T03:23:20.690808+0000","last_fullsized":"2024-09-12T03:23:20.690808+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_clean_scrub_stamp":"2024-09-12T03:23:19.659665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:11:32.173284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.b","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.671109+0000","last_change":"2024-09-12T03:23:20.671109+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.671109+0000","last_undegraded":"2024-09-12T03:23:20.671109+0000","last_fullsized":"2024-09-12T03:23:20.671109+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.a","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-12T03:23:20.686788+0000","last_change":"2024-09-12T03:23:20.683771+0000","last_active":"2024-09-12T03:23:20.686788+0000","last_peered":"2024-09-12T03:23:20.686788+0000","last_clean":"2024-09-12T03:23:20.686788+0000","last_became_active":"2024-09-12T03:23:20.683511+0000","last_became_peered":"2024-09-12T03:23:20.683511+0000","last_unstale":"2024-09-12T03:23:20.686788+0000","last_undegraded":"2024-09-12T03:23:20.686788+0000","last_fullsized":"2024-09-12T03:23:20.686788+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_clean_scrub_stamp":"2024-09-12T03:23:19.659665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:28:42.183834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.8","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.675851+0000","last_change":"2024-09-12T03:23:20.675851+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.675851+0000","last_undegraded":"2024-09-12T03:23:20.675851+0000","last_fullsized":"2024-09-12T03:23:20.675851+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.9","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-12T03:23:20.695616+0000","last_change":"2024-09-12T03:23:20.692711+0000","last_active":"2024-09-12T03:23:20.695616+0000","last_peered":"2024-09-12T03:23:20.695616+0000","last_clean":"2024-09-12T03:23:20.695616+0000","last_became_active":"2024-09-12T03:23:20.692455+0000","last_became_peered":"2024-09-12T03:23:20.692455+0000","last_unstale":"2024-09-12T03:23:20.695616+0000","last_undegraded":"2024-09-12T03:23:20.695616+0000","last_fullsized":"2024-09-12T03:23:20.695616+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_clean_scrub_stamp":"2024-09-12T03:23:19.659665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:47:47.363872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.9","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.681456+0000","last_change":"2024-09-12T03:23:20.681456+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.681456+0000","last_undegraded":"2024-09-12T03:23:20.681456+0000","last_fullsized":"2024-09-12T03:23:20.681456+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.8","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-12T03:23:20.684083+0000","last_change":"2024-09-12T03:23:20.679394+0000","last_active":"2024-09-12T03:23:20.684083+0000","last_peered":"2024-09-12T03:23:20.684083+0000","last_clean":"2024-09-12T03:23:20.684083+0000","last_became_active":"2024-09-12T03:23:20.679100+0000","last_became_peered":"2024-09-12T03:23:20.679100+0000","last_unstale":"2024-09-12T03:23:20.684083+0000","last_undegraded":"2024-09-12T03:23:20.684083+0000","last_fullsized":"2024-09-12T03:23:20.684083+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_clean_scrub_stamp":"2024-09-12T03:23:19.659665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:49:36.375371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.6","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.681144+0000","last_change":"2024-09-12T03:23:20.681144+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.681144+0000","last_undegraded":"2024-09-12T03:23:20.681144+0000","last_fullsized":"2024-09-12T03:23:20.681144+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.7","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-12T03:23:20.693562+0000","last_change":"2024-09-12T03:23:20.693222+0000","last_active":"2024-09-12T03:23:20.693562+0000","last_peered":"2024-09-12T03:23:20.693562+0000","last_clean":"2024-09-12T03:23:20.693562+0000","last_became_active":"2024-09-12T03:23:20.692889+0000","last_became_peered":"2024-09-12T03:23:20.692889+0000","last_unstale":"2024-09-12T03:23:20.693562+0000","last_undegraded":"2024-09-12T03:23:20.693562+0000","last_fullsized":"2024-09-12T03:23:20.693562+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_clean_scrub_stamp":"2024-09-12T03:23:19.659665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:08:14.319966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.7","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.676386+0000","last_change":"2024-09-12T03:23:20.676386+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.676386+0000","last_undegraded":"2024-09-12T03:23:20.676386+0000","last_fullsized":"2024-09-12T03:23:20.676386+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.6","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-12T03:23:20.688527+0000","last_change":"2024-09-12T03:23:20.687884+0000","last_active":"2024-09-12T03:23:20.688527+0000","last_peered":"2024-09-12T03:23:20.688527+0000","last_clean":"2024-09-12T03:23:20.688527+0000","last_became_active":"2024-09-12T03:23:20.687621+0000","last_became_peered":"2024-09-12T03:23:20.687621+0000","last_unstale":"2024-09-12T03:23:20.688527+0000","last_undegraded":"2024-09-12T03:23:20.688527+0000","last_fullsized":"2024-09-12T03:23:20.688527+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_clean_scrub_stamp":"2024-09-12T03:23:19.659665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:34:47.119838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.4","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.677605+0000","last_change":"2024-09-12T03:23:20.677605+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.677605+0000","last_undegraded":"2024-09-12T03:23:20.677605+0000","last_fullsized":"2024-09-12T03:23:20.677605+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.5","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-12T03:23:20.688498+0000","last_change":"2024-09-12T03:23:20.686431+0000","last_active":"2024-09-12T03:23:20.688498+0000","last_peered":"2024-09-12T03:23:20.688498+0000","last_clean":"2024-09-12T03:23:20.688498+0000","last_became_active":"2024-09-12T03:23:20.686145+0000","last_became_peered":"2024-09-12T03:23:20.686145+0000","last_unstale":"2024-09-12T03:23:20.688498+0000","last_undegraded":"2024-09-12T03:23:20.688498+0000","last_fullsized":"2024-09-12T03:23:20.688498+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_clean_scrub_stamp":"2024-09-12T03:23:19.659665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:14:07.788062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.5","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.677039+0000","last_change":"2024-09-12T03:23:20.677039+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.677039+0000","last_undegraded":"2024-09-12T03:23:20.677039+0000","last_fullsized":"2024-09-12T03:23:20.677039+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.4","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-12T03:23:20.691435+0000","last_change":"2024-09-12T03:23:20.689734+0000","last_active":"2024-09-12T03:23:20.691435+0000","last_peered":"2024-09-12T03:23:20.691435+0000","last_clean":"2024-09-12T03:23:20.691435+0000","last_became_active":"2024-09-12T03:23:20.689439+0000","last_became_peered":"2024-09-12T03:23:20.689439+0000","last_unstale":"2024-09-12T03:23:20.691435+0000","last_undegraded":"2024-09-12T03:23:20.691435+0000","last_fullsized":"2024-09-12T03:23:20.691435+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_clean_scrub_stamp":"2024-09-12T03:23:19.659665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:47:48.015298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.2","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.675165+0000","last_change":"2024-09-12T03:23:20.675165+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.675165+0000","last_undegraded":"2024-09-12T03:23:20.675165+0000","last_fullsized":"2024-09-12T03:23:20.675165+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.3","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-12T03:23:20.685867+0000","last_change":"2024-09-12T03:23:20.684832+0000","last_active":"2024-09-12T03:23:20.685867+0000","last_peered":"2024-09-12T03:23:20.685867+0000","last_clean":"2024-09-12T03:23:20.685867+0000","last_became_active":"2024-09-12T03:23:20.684531+0000","last_became_peered":"2024-09-12T03:23:20.684531+0000","last_unstale":"2024-09-12T03:23:20.685867+0000","last_undegraded":"2024-09-12T03:23:20.685867+0000","last_fullsized":"2024-09-12T03:23:20.685867+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_clean_scrub_stamp":"2024-09-12T03:23:19.659665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:11:40.963697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.3","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.675175+0000","last_change":"2024-09-12T03:23:20.675175+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.675175+0000","last_undegraded":"2024-09-12T03:23:20.675175+0000","last_fullsized":"2024-09-12T03:23:20.675175+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.2","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-12T03:23:20.685296+0000","last_change":"2024-09-12T03:23:20.681981+0000","last_active":"2024-09-12T03:23:20.685296+0000","last_peered":"2024-09-12T03:23:20.685296+0000","last_clean":"2024-09-12T03:23:20.685296+0000","last_became_active":"2024-09-12T03:23:20.681487+0000","last_became_peered":"2024-09-12T03:23:20.681487+0000","last_unstale":"2024-09-12T03:23:20.685296+0000","last_undegraded":"2024-09-12T03:23:20.685296+0000","last_fullsized":"2024-09-12T03:23:20.685296+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_clean_scrub_stamp":"2024-09-12T03:23:19.659665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:52:54.619581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.0","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.675295+0000","last_change":"2024-09-12T03:23:20.675295+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.675295+0000","last_undegraded":"2024-09-12T03:23:20.675295+0000","last_fullsized":"2024-09-12T03:23:20.675295+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.1","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-12T03:23:20.682875+0000","last_change":"2024-09-12T03:23:20.682237+0000","last_active":"2024-09-12T03:23:20.682875+0000","last_peered":"2024-09-12T03:23:20.682875+0000","last_clean":"2024-09-12T03:23:20.682875+0000","last_became_active":"2024-09-12T03:23:20.681397+0000","last_became_peered":"2024-09-12T03:23:20.681397+0000","last_unstale":"2024-09-12T03:23:20.682875+0000","last_undegraded":"2024-09-12T03:23:20.682875+0000","last_fullsized":"2024-09-12T03:23:20.682875+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_clean_scrub_stamp":"2024-09-12T03:23:19.659665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:16:33.772644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.1","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.680888+0000","last_change":"2024-09-12T03:23:20.680888+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.680888+0000","last_undegraded":"2024-09-12T03:23:20.680888+0000","last_fullsized":"2024-09-12T03:23:20.680888+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.0","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-12T03:23:20.695121+0000","last_change":"2024-09-12T03:23:20.688059+0000","last_active":"2024-09-12T03:23:20.695121+0000","last_peered":"2024-09-12T03:23:20.695121+0000","last_clean":"2024-09-12T03:23:20.695121+0000","last_became_active":"2024-09-12T03:23:20.687774+0000","last_became_peered":"2024-09-12T03:23:20.687774+0000","last_unstale":"2024-09-12T03:23:20.695121+0000","last_undegraded":"2024-09-12T03:23:20.695121+0000","last_fullsized":"2024-09-12T03:23:20.695121+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_clean_scrub_stamp":"2024-09-12T03:23:19.659665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:31:56.155648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.18","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.678051+0000","last_change":"2024-09-12T03:23:20.678051+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.678051+0000","last_undegraded":"2024-09-12T03:23:20.678051+0000","last_fullsized":"2024-09-12T03:23:20.678051+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.19","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-12T03:23:20.686693+0000","last_change":"2024-09-12T03:23:20.685565+0000","last_active":"2024-09-12T03:23:20.686693+0000","last_peered":"2024-09-12T03:23:20.686693+0000","last_clean":"2024-09-12T03:23:20.686693+0000","last_became_active":"2024-09-12T03:23:20.685230+0000","last_became_peered":"2024-09-12T03:23:20.685230+0000","last_unstale":"2024-09-12T03:23:20.686693+0000","last_undegraded":"2024-09-12T03:23:20.686693+0000","last_fullsized":"2024-09-12T03:23:20.686693+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_clean_scrub_stamp":"2024-09-12T03:23:19.659665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:25:00.890612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"70'128","reported_seq":441,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-12T03:23:20.680128+0000","last_change":"2024-09-12T03:16:21.883446+0000","last_active":"2024-09-12T03:23:20.680128+0000","last_peered":"2024-09-12T03:23:20.680128+0000","last_clean":"2024-09-12T03:23:20.680128+0000","last_became_active":"2024-09-12T03:16:21.882207+0000","last_became_peered":"2024-09-12T03:16:21.882207+0000","last_unstale":"2024-09-12T03:23:20.680128+0000","last_undegraded":"2024-09-12T03:23:20.680128+0000","last_fullsized":"2024-09-12T03:23:20.680128+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:20.316426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:20.316426+0000","last_clean_scrub_stamp":"2024-09-12T03:16:20.316426+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:12:15.424320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":25,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":24,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542230,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37736,"kb_used_data":2388,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334104,"statfs":{"total":96636764160,"available":96598122496,"internally_reserved":0,"allocated":2445312,"data_stored":1407765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542232,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37148,"kb_used_data":1796,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334692,"statfs":{"total":96636764160,"available":96598724608,"internally_reserved":0,"allocated":1839104,"data_stored":811066,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542231,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37156,"kb_used_data":1808,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334684,"statfs":{"total":96636764160,"available":96598716416,"internally_reserved":0,"allocated":1851392,"data_stored":817397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542230,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37160,"kb_used_data":1808,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334680,"statfs":{"total":96636764160,"available":96598712320,"internally_reserved":0,"allocated":1851392,"data_stored":817397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542231,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32996,"kb_used_data":1796,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338844,"statfs":{"total":96636764160,"available":96602976256,"internally_reserved":0,"allocated":1839104,"data_stored":811066,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542231,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32984,"kb_used_data":1796,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338856,"statfs":{"total":96636764160,"available":96602988544,"internally_reserved":0,"allocated":1839104,"data_stored":811066,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542232,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32996,"kb_used_data":1808,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338844,"statfs":{"total":96636764160,"available":96602976256,"internally_reserved":0,"allocated":1851392,"data_stored":817397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542232,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33580,"kb_used_data":2388,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338260,"statfs":{"total":96636764160,"available":96602378240,"internally_reserved":0,"allocated":2445312,"data_stored":1407765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T03:23:26.389 INFO:tasks.ceph.ceph_manager.ceph:PG 25.3f is not active+clean 2024-09-12T03:23:26.389 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 147, 'state': 'creating+peering', 'last_fresh': '2024-09-12T03:23:20.676683+0000', 'last_change': '2024-09-12T03:23:20.676683+0000', 'last_active': '2024-09-12T03:23:20.665225+0000', 'last_peered': '2024-09-12T03:23:20.665225+0000', 'last_clean': '2024-09-12T03:23:20.665225+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:23:20.676683+0000', 'last_undegraded': '2024-09-12T03:23:20.676683+0000', 'last_fullsized': '2024-09-12T03:23:20.676683+0000', 'mapping_epoch': 147, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 147, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:23:20.665225+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:23:20.665225+0000', 'last_clean_scrub_stamp': '2024-09-12T03:23:20.665225+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-12T03:23:26.390 INFO:tasks.ceph.ceph_manager.ceph:PG 25.3e is not active+clean 2024-09-12T03:23:26.390 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 147, 'state': 'creating+peering', 'last_fresh': '2024-09-12T03:23:20.677002+0000', 'last_change': '2024-09-12T03:23:20.677002+0000', 'last_active': '2024-09-12T03:23:20.665225+0000', 'last_peered': '2024-09-12T03:23:20.665225+0000', 'last_clean': '2024-09-12T03:23:20.665225+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:23:20.677002+0000', 'last_undegraded': '2024-09-12T03:23:20.677002+0000', 'last_fullsized': '2024-09-12T03:23:20.677002+0000', 'mapping_epoch': 147, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 147, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:23:20.665225+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:23:20.665225+0000', 'last_clean_scrub_stamp': '2024-09-12T03:23:20.665225+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-12T03:23:26.390 INFO:tasks.ceph.ceph_manager.ceph:PG 25.3d is not active+clean 2024-09-12T03:23:26.390 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 147, 'state': 'creating+peering', 'last_fresh': '2024-09-12T03:23:20.680808+0000', 'last_change': '2024-09-12T03:23:20.680808+0000', 'last_active': '2024-09-12T03:23:20.665225+0000', 'last_peered': '2024-09-12T03:23:20.665225+0000', 'last_clean': '2024-09-12T03:23:20.665225+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:23:20.680808+0000', 'last_undegraded': '2024-09-12T03:23:20.680808+0000', 'last_fullsized': '2024-09-12T03:23:20.680808+0000', 'mapping_epoch': 147, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 147, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:23:20.665225+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:23:20.665225+0000', 'last_clean_scrub_stamp': '2024-09-12T03:23:20.665225+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-12T03:23:26.390 INFO:tasks.ceph.ceph_manager.ceph:PG 25.3c is not active+clean 2024-09-12T03:23:26.390 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 147, 'state': 'creating+peering', 'last_fresh': '2024-09-12T03:23:20.677777+0000', 'last_change': '2024-09-12T03:23:20.677777+0000', 'last_active': '2024-09-12T03:23:20.665225+0000', 'last_peered': '2024-09-12T03:23:20.665225+0000', 'last_clean': '2024-09-12T03:23:20.665225+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:23:20.677777+0000', 'last_undegraded': '2024-09-12T03:23:20.677777+0000', 'last_fullsized': '2024-09-12T03:23:20.677777+0000', 'mapping_epoch': 147, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 147, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:23:20.665225+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:23:20.665225+0000', 'last_clean_scrub_stamp': '2024-09-12T03:23:20.665225+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-12T03:23:26.390 INFO:tasks.ceph.ceph_manager.ceph:PG 25.3b is not active+clean 2024-09-12T03:23:26.390 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 147, 'state': 'creating+peering', 'last_fresh': '2024-09-12T03:23:20.683306+0000', 'last_change': '2024-09-12T03:23:20.683306+0000', 'last_active': '2024-09-12T03:23:20.665225+0000', 'last_peered': '2024-09-12T03:23:20.665225+0000', 'last_clean': '2024-09-12T03:23:20.665225+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:23:20.683306+0000', 'last_undegraded': '2024-09-12T03:23:20.683306+0000', 'last_fullsized': '2024-09-12T03:23:20.683306+0000', 'mapping_epoch': 147, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 147, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:23:20.665225+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:23:20.665225+0000', 'last_clean_scrub_stamp': '2024-09-12T03:23:20.665225+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-12T03:23:26.390 INFO:tasks.ceph.ceph_manager.ceph:PG 25.3a is not active+clean 2024-09-12T03:23:26.390 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 147, 'state': 'creating+peering', 'last_fresh': '2024-09-12T03:23:20.680330+0000', 'last_change': '2024-09-12T03:23:20.680330+0000', 'last_active': '2024-09-12T03:23:20.665225+0000', 'last_peered': '2024-09-12T03:23:20.665225+0000', 'last_clean': '2024-09-12T03:23:20.665225+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:23:20.680330+0000', 'last_undegraded': '2024-09-12T03:23:20.680330+0000', 'last_fullsized': '2024-09-12T03:23:20.680330+0000', 'mapping_epoch': 147, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 147, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:23:20.665225+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:23:20.665225+0000', 'last_clean_scrub_stamp': '2024-09-12T03:23:20.665225+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-12T03:23:26.391 INFO:tasks.ceph.ceph_manager.ceph:PG 25.39 is not active+clean 2024-09-12T03:23:26.391 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 147, 'state': 'creating+peering', 'last_fresh': '2024-09-12T03:23:20.672348+0000', 'last_change': '2024-09-12T03:23:20.672348+0000', 'last_active': '2024-09-12T03:23:20.665225+0000', 'last_peered': '2024-09-12T03:23:20.665225+0000', 'last_clean': '2024-09-12T03:23:20.665225+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:23:20.672348+0000', 'last_undegraded': '2024-09-12T03:23:20.672348+0000', 'last_fullsized': '2024-09-12T03:23:20.672348+0000', 'mapping_epoch': 147, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 147, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:23:20.665225+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:23:20.665225+0000', 'last_clean_scrub_stamp': '2024-09-12T03:23:20.665225+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-12T03:23:26.391 INFO:tasks.ceph.ceph_manager.ceph:PG 25.38 is not active+clean 2024-09-12T03:23:26.391 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 147, 'state': 'creating+peering', 'last_fresh': '2024-09-12T03:23:20.676156+0000', 'last_change': '2024-09-12T03:23:20.676156+0000', 'last_active': '2024-09-12T03:23:20.665225+0000', 'last_peered': '2024-09-12T03:23:20.665225+0000', 'last_clean': '2024-09-12T03:23:20.665225+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:23:20.676156+0000', 'last_undegraded': '2024-09-12T03:23:20.676156+0000', 'last_fullsized': '2024-09-12T03:23:20.676156+0000', 'mapping_epoch': 147, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 147, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:23:20.665225+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:23:20.665225+0000', 'last_clean_scrub_stamp': '2024-09-12T03:23:20.665225+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-12T03:23:26.391 INFO:tasks.ceph.ceph_manager.ceph:PG 25.37 is not active+clean 2024-09-12T03:23:26.391 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 147, 'state': 'creating+peering', 'last_fresh': '2024-09-12T03:23:20.674337+0000', 'last_change': '2024-09-12T03:23:20.674337+0000', 'last_active': '2024-09-12T03:23:20.665225+0000', 'last_peered': '2024-09-12T03:23:20.665225+0000', 'last_clean': '2024-09-12T03:23:20.665225+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:23:20.674337+0000', 'last_undegraded': '2024-09-12T03:23:20.674337+0000', 'last_fullsized': '2024-09-12T03:23:20.674337+0000', 'mapping_epoch': 147, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 147, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:23:20.665225+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:23:20.665225+0000', 'last_clean_scrub_stamp': '2024-09-12T03:23:20.665225+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-12T03:23:26.391 INFO:tasks.ceph.ceph_manager.ceph:PG 25.36 is not active+clean 2024-09-12T03:23:26.391 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 147, 'state': 'creating+peering', 'last_fresh': '2024-09-12T03:23:20.679738+0000', 'last_change': '2024-09-12T03:23:20.679738+0000', 'last_active': '2024-09-12T03:23:20.665225+0000', 'last_peered': '2024-09-12T03:23:20.665225+0000', 'last_clean': '2024-09-12T03:23:20.665225+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:23:20.679738+0000', 'last_undegraded': '2024-09-12T03:23:20.679738+0000', 'last_fullsized': '2024-09-12T03:23:20.679738+0000', 'mapping_epoch': 147, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 147, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:23:20.665225+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:23:20.665225+0000', 'last_clean_scrub_stamp': '2024-09-12T03:23:20.665225+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-12T03:23:26.391 INFO:tasks.ceph.ceph_manager.ceph:PG 25.35 is not active+clean 2024-09-12T03:23:26.391 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 147, 'state': 'creating+peering', 'last_fresh': '2024-09-12T03:23:20.671126+0000', 'last_change': '2024-09-12T03:23:20.671126+0000', 'last_active': '2024-09-12T03:23:20.665225+0000', 'last_peered': '2024-09-12T03:23:20.665225+0000', 'last_clean': '2024-09-12T03:23:20.665225+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:23:20.671126+0000', 'last_undegraded': '2024-09-12T03:23:20.671126+0000', 'last_fullsized': '2024-09-12T03:23:20.671126+0000', 'mapping_epoch': 147, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 147, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:23:20.665225+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:23:20.665225+0000', 'last_clean_scrub_stamp': '2024-09-12T03:23:20.665225+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-12T03:23:26.392 INFO:tasks.ceph.ceph_manager.ceph:PG 25.34 is not active+clean 2024-09-12T03:23:26.392 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 147, 'state': 'creating+peering', 'last_fresh': '2024-09-12T03:23:20.677214+0000', 'last_change': '2024-09-12T03:23:20.677214+0000', 'last_active': '2024-09-12T03:23:20.665225+0000', 'last_peered': '2024-09-12T03:23:20.665225+0000', 'last_clean': '2024-09-12T03:23:20.665225+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:23:20.677214+0000', 'last_undegraded': '2024-09-12T03:23:20.677214+0000', 'last_fullsized': '2024-09-12T03:23:20.677214+0000', 'mapping_epoch': 147, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 147, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:23:20.665225+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:23:20.665225+0000', 'last_clean_scrub_stamp': '2024-09-12T03:23:20.665225+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-12T03:23:26.392 INFO:tasks.ceph.ceph_manager.ceph:PG 25.33 is not active+clean 2024-09-12T03:23:26.392 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 147, 'state': 'creating+peering', 'last_fresh': '2024-09-12T03:23:20.677238+0000', 'last_change': '2024-09-12T03:23:20.677238+0000', 'last_active': '2024-09-12T03:23:20.665225+0000', 'last_peered': '2024-09-12T03:23:20.665225+0000', 'last_clean': '2024-09-12T03:23:20.665225+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:23:20.677238+0000', 'last_undegraded': '2024-09-12T03:23:20.677238+0000', 'last_fullsized': '2024-09-12T03:23:20.677238+0000', 'mapping_epoch': 147, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 147, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:23:20.665225+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:23:20.665225+0000', 'last_clean_scrub_stamp': '2024-09-12T03:23:20.665225+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-12T03:23:26.392 INFO:tasks.ceph.ceph_manager.ceph:PG 25.32 is not active+clean 2024-09-12T03:23:26.392 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 147, 'state': 'creating+peering', 'last_fresh': '2024-09-12T03:23:20.675036+0000', 'last_change': '2024-09-12T03:23:20.675036+0000', 'last_active': '2024-09-12T03:23:20.665225+0000', 'last_peered': '2024-09-12T03:23:20.665225+0000', 'last_clean': '2024-09-12T03:23:20.665225+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:23:20.675036+0000', 'last_undegraded': '2024-09-12T03:23:20.675036+0000', 'last_fullsized': '2024-09-12T03:23:20.675036+0000', 'mapping_epoch': 147, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 147, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:23:20.665225+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:23:20.665225+0000', 'last_clean_scrub_stamp': '2024-09-12T03:23:20.665225+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-12T03:23:26.392 INFO:tasks.ceph.ceph_manager.ceph:PG 25.31 is not active+clean 2024-09-12T03:23:26.392 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 147, 'state': 'creating+peering', 'last_fresh': '2024-09-12T03:23:20.675841+0000', 'last_change': '2024-09-12T03:23:20.675841+0000', 'last_active': '2024-09-12T03:23:20.665225+0000', 'last_peered': '2024-09-12T03:23:20.665225+0000', 'last_clean': '2024-09-12T03:23:20.665225+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:23:20.675841+0000', 'last_undegraded': '2024-09-12T03:23:20.675841+0000', 'last_fullsized': '2024-09-12T03:23:20.675841+0000', 'mapping_epoch': 147, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 147, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:23:20.665225+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:23:20.665225+0000', 'last_clean_scrub_stamp': '2024-09-12T03:23:20.665225+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-12T03:23:26.392 INFO:tasks.ceph.ceph_manager.ceph:PG 25.30 is not active+clean 2024-09-12T03:23:26.392 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 147, 'state': 'creating+peering', 'last_fresh': '2024-09-12T03:23:20.675750+0000', 'last_change': '2024-09-12T03:23:20.675750+0000', 'last_active': '2024-09-12T03:23:20.665225+0000', 'last_peered': '2024-09-12T03:23:20.665225+0000', 'last_clean': '2024-09-12T03:23:20.665225+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:23:20.675750+0000', 'last_undegraded': '2024-09-12T03:23:20.675750+0000', 'last_fullsized': '2024-09-12T03:23:20.675750+0000', 'mapping_epoch': 147, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 147, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:23:20.665225+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:23:20.665225+0000', 'last_clean_scrub_stamp': '2024-09-12T03:23:20.665225+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-12T03:23:26.393 INFO:tasks.ceph.ceph_manager.ceph:PG 25.2f is not active+clean 2024-09-12T03:23:26.393 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 147, 'state': 'creating+peering', 'last_fresh': '2024-09-12T03:23:20.687993+0000', 'last_change': '2024-09-12T03:23:20.687993+0000', 'last_active': '2024-09-12T03:23:20.665225+0000', 'last_peered': '2024-09-12T03:23:20.665225+0000', 'last_clean': '2024-09-12T03:23:20.665225+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:23:20.687993+0000', 'last_undegraded': '2024-09-12T03:23:20.687993+0000', 'last_fullsized': '2024-09-12T03:23:20.687993+0000', 'mapping_epoch': 147, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 147, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:23:20.665225+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:23:20.665225+0000', 'last_clean_scrub_stamp': '2024-09-12T03:23:20.665225+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 6], 'acting': [2, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-12T03:23:26.393 INFO:tasks.ceph.ceph_manager.ceph:PG 25.2e is not active+clean 2024-09-12T03:23:26.393 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 147, 'state': 'creating+peering', 'last_fresh': '2024-09-12T03:23:20.676010+0000', 'last_change': '2024-09-12T03:23:20.676010+0000', 'last_active': '2024-09-12T03:23:20.665225+0000', 'last_peered': '2024-09-12T03:23:20.665225+0000', 'last_clean': '2024-09-12T03:23:20.665225+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:23:20.676010+0000', 'last_undegraded': '2024-09-12T03:23:20.676010+0000', 'last_fullsized': '2024-09-12T03:23:20.676010+0000', 'mapping_epoch': 147, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 147, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:23:20.665225+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:23:20.665225+0000', 'last_clean_scrub_stamp': '2024-09-12T03:23:20.665225+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-12T03:23:26.393 INFO:tasks.ceph.ceph_manager.ceph:PG 25.2d is not active+clean 2024-09-12T03:23:26.393 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 147, 'state': 'creating+peering', 'last_fresh': '2024-09-12T03:23:20.680215+0000', 'last_change': '2024-09-12T03:23:20.680215+0000', 'last_active': '2024-09-12T03:23:20.665225+0000', 'last_peered': '2024-09-12T03:23:20.665225+0000', 'last_clean': '2024-09-12T03:23:20.665225+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:23:20.680215+0000', 'last_undegraded': '2024-09-12T03:23:20.680215+0000', 'last_fullsized': '2024-09-12T03:23:20.680215+0000', 'mapping_epoch': 147, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 147, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:23:20.665225+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:23:20.665225+0000', 'last_clean_scrub_stamp': '2024-09-12T03:23:20.665225+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-12T03:23:26.393 INFO:tasks.ceph.ceph_manager.ceph:PG 25.2c is not active+clean 2024-09-12T03:23:26.393 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 147, 'state': 'creating+peering', 'last_fresh': '2024-09-12T03:23:20.685697+0000', 'last_change': '2024-09-12T03:23:20.685697+0000', 'last_active': '2024-09-12T03:23:20.665225+0000', 'last_peered': '2024-09-12T03:23:20.665225+0000', 'last_clean': '2024-09-12T03:23:20.665225+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:23:20.685697+0000', 'last_undegraded': '2024-09-12T03:23:20.685697+0000', 'last_fullsized': '2024-09-12T03:23:20.685697+0000', 'mapping_epoch': 147, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 147, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:23:20.665225+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:23:20.665225+0000', 'last_clean_scrub_stamp': '2024-09-12T03:23:20.665225+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 6], 'acting': [2, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-12T03:23:26.394 INFO:tasks.ceph.ceph_manager.ceph:PG 25.2b is not active+clean 2024-09-12T03:23:26.394 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 147, 'state': 'creating+peering', 'last_fresh': '2024-09-12T03:23:20.682781+0000', 'last_change': '2024-09-12T03:23:20.682781+0000', 'last_active': '2024-09-12T03:23:20.665225+0000', 'last_peered': '2024-09-12T03:23:20.665225+0000', 'last_clean': '2024-09-12T03:23:20.665225+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:23:20.682781+0000', 'last_undegraded': '2024-09-12T03:23:20.682781+0000', 'last_fullsized': '2024-09-12T03:23:20.682781+0000', 'mapping_epoch': 147, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 147, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:23:20.665225+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:23:20.665225+0000', 'last_clean_scrub_stamp': '2024-09-12T03:23:20.665225+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-12T03:23:26.394 INFO:tasks.ceph.ceph_manager.ceph:PG 25.2a is not active+clean 2024-09-12T03:23:26.394 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 147, 'state': 'creating+peering', 'last_fresh': '2024-09-12T03:23:20.679222+0000', 'last_change': '2024-09-12T03:23:20.679222+0000', 'last_active': '2024-09-12T03:23:20.665225+0000', 'last_peered': '2024-09-12T03:23:20.665225+0000', 'last_clean': '2024-09-12T03:23:20.665225+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:23:20.679222+0000', 'last_undegraded': '2024-09-12T03:23:20.679222+0000', 'last_fullsized': '2024-09-12T03:23:20.679222+0000', 'mapping_epoch': 147, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 147, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:23:20.665225+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:23:20.665225+0000', 'last_clean_scrub_stamp': '2024-09-12T03:23:20.665225+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-12T03:23:26.394 INFO:tasks.ceph.ceph_manager.ceph:PG 25.29 is not active+clean 2024-09-12T03:23:26.394 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 147, 'state': 'creating+peering', 'last_fresh': '2024-09-12T03:23:20.679592+0000', 'last_change': '2024-09-12T03:23:20.679592+0000', 'last_active': '2024-09-12T03:23:20.665225+0000', 'last_peered': '2024-09-12T03:23:20.665225+0000', 'last_clean': '2024-09-12T03:23:20.665225+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:23:20.679592+0000', 'last_undegraded': '2024-09-12T03:23:20.679592+0000', 'last_fullsized': '2024-09-12T03:23:20.679592+0000', 'mapping_epoch': 147, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 147, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:23:20.665225+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:23:20.665225+0000', 'last_clean_scrub_stamp': '2024-09-12T03:23:20.665225+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-12T03:23:26.394 INFO:tasks.ceph.ceph_manager.ceph:PG 25.28 is not active+clean 2024-09-12T03:23:26.394 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 147, 'state': 'creating+peering', 'last_fresh': '2024-09-12T03:23:20.675319+0000', 'last_change': '2024-09-12T03:23:20.675319+0000', 'last_active': '2024-09-12T03:23:20.665225+0000', 'last_peered': '2024-09-12T03:23:20.665225+0000', 'last_clean': '2024-09-12T03:23:20.665225+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:23:20.675319+0000', 'last_undegraded': '2024-09-12T03:23:20.675319+0000', 'last_fullsized': '2024-09-12T03:23:20.675319+0000', 'mapping_epoch': 147, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 147, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:23:20.665225+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:23:20.665225+0000', 'last_clean_scrub_stamp': '2024-09-12T03:23:20.665225+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-12T03:23:26.394 INFO:tasks.ceph.ceph_manager.ceph:PG 25.27 is not active+clean 2024-09-12T03:23:26.395 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 147, 'state': 'creating+peering', 'last_fresh': '2024-09-12T03:23:20.676115+0000', 'last_change': '2024-09-12T03:23:20.676115+0000', 'last_active': '2024-09-12T03:23:20.665225+0000', 'last_peered': '2024-09-12T03:23:20.665225+0000', 'last_clean': '2024-09-12T03:23:20.665225+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:23:20.676115+0000', 'last_undegraded': '2024-09-12T03:23:20.676115+0000', 'last_fullsized': '2024-09-12T03:23:20.676115+0000', 'mapping_epoch': 147, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 147, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:23:20.665225+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:23:20.665225+0000', 'last_clean_scrub_stamp': '2024-09-12T03:23:20.665225+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-12T03:23:26.395 INFO:tasks.ceph.ceph_manager.ceph:PG 25.26 is not active+clean 2024-09-12T03:23:26.395 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 147, 'state': 'creating+peering', 'last_fresh': '2024-09-12T03:23:20.671542+0000', 'last_change': '2024-09-12T03:23:20.671542+0000', 'last_active': '2024-09-12T03:23:20.665225+0000', 'last_peered': '2024-09-12T03:23:20.665225+0000', 'last_clean': '2024-09-12T03:23:20.665225+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:23:20.671542+0000', 'last_undegraded': '2024-09-12T03:23:20.671542+0000', 'last_fullsized': '2024-09-12T03:23:20.671542+0000', 'mapping_epoch': 147, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 147, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:23:20.665225+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:23:20.665225+0000', 'last_clean_scrub_stamp': '2024-09-12T03:23:20.665225+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-12T03:23:26.395 INFO:tasks.ceph.ceph_manager.ceph:PG 25.25 is not active+clean 2024-09-12T03:23:26.395 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 147, 'state': 'creating+peering', 'last_fresh': '2024-09-12T03:23:20.676098+0000', 'last_change': '2024-09-12T03:23:20.676098+0000', 'last_active': '2024-09-12T03:23:20.665225+0000', 'last_peered': '2024-09-12T03:23:20.665225+0000', 'last_clean': '2024-09-12T03:23:20.665225+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:23:20.676098+0000', 'last_undegraded': '2024-09-12T03:23:20.676098+0000', 'last_fullsized': '2024-09-12T03:23:20.676098+0000', 'mapping_epoch': 147, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 147, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:23:20.665225+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:23:20.665225+0000', 'last_clean_scrub_stamp': '2024-09-12T03:23:20.665225+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-12T03:23:26.395 INFO:tasks.ceph.ceph_manager.ceph:PG 25.24 is not active+clean 2024-09-12T03:23:26.395 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 147, 'state': 'creating+peering', 'last_fresh': '2024-09-12T03:23:20.685181+0000', 'last_change': '2024-09-12T03:23:20.685181+0000', 'last_active': '2024-09-12T03:23:20.665225+0000', 'last_peered': '2024-09-12T03:23:20.665225+0000', 'last_clean': '2024-09-12T03:23:20.665225+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:23:20.685181+0000', 'last_undegraded': '2024-09-12T03:23:20.685181+0000', 'last_fullsized': '2024-09-12T03:23:20.685181+0000', 'mapping_epoch': 147, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 147, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:23:20.665225+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:23:20.665225+0000', 'last_clean_scrub_stamp': '2024-09-12T03:23:20.665225+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-12T03:23:26.395 INFO:tasks.ceph.ceph_manager.ceph:PG 25.23 is not active+clean 2024-09-12T03:23:26.395 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 147, 'state': 'creating+peering', 'last_fresh': '2024-09-12T03:23:20.676786+0000', 'last_change': '2024-09-12T03:23:20.676786+0000', 'last_active': '2024-09-12T03:23:20.665225+0000', 'last_peered': '2024-09-12T03:23:20.665225+0000', 'last_clean': '2024-09-12T03:23:20.665225+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:23:20.676786+0000', 'last_undegraded': '2024-09-12T03:23:20.676786+0000', 'last_fullsized': '2024-09-12T03:23:20.676786+0000', 'mapping_epoch': 147, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 147, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:23:20.665225+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:23:20.665225+0000', 'last_clean_scrub_stamp': '2024-09-12T03:23:20.665225+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-12T03:23:26.395 INFO:tasks.ceph.ceph_manager.ceph:PG 25.22 is not active+clean 2024-09-12T03:23:26.395 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 147, 'state': 'creating+peering', 'last_fresh': '2024-09-12T03:23:20.674611+0000', 'last_change': '2024-09-12T03:23:20.674611+0000', 'last_active': '2024-09-12T03:23:20.665225+0000', 'last_peered': '2024-09-12T03:23:20.665225+0000', 'last_clean': '2024-09-12T03:23:20.665225+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:23:20.674611+0000', 'last_undegraded': '2024-09-12T03:23:20.674611+0000', 'last_fullsized': '2024-09-12T03:23:20.674611+0000', 'mapping_epoch': 147, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 147, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:23:20.665225+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:23:20.665225+0000', 'last_clean_scrub_stamp': '2024-09-12T03:23:20.665225+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-12T03:23:26.396 INFO:tasks.ceph.ceph_manager.ceph:PG 25.21 is not active+clean 2024-09-12T03:23:26.396 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 147, 'state': 'creating+peering', 'last_fresh': '2024-09-12T03:23:20.676607+0000', 'last_change': '2024-09-12T03:23:20.676607+0000', 'last_active': '2024-09-12T03:23:20.665225+0000', 'last_peered': '2024-09-12T03:23:20.665225+0000', 'last_clean': '2024-09-12T03:23:20.665225+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:23:20.676607+0000', 'last_undegraded': '2024-09-12T03:23:20.676607+0000', 'last_fullsized': '2024-09-12T03:23:20.676607+0000', 'mapping_epoch': 147, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 147, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:23:20.665225+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:23:20.665225+0000', 'last_clean_scrub_stamp': '2024-09-12T03:23:20.665225+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-12T03:23:26.396 INFO:tasks.ceph.ceph_manager.ceph:PG 25.20 is not active+clean 2024-09-12T03:23:26.396 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 147, 'state': 'creating+peering', 'last_fresh': '2024-09-12T03:23:20.682292+0000', 'last_change': '2024-09-12T03:23:20.682292+0000', 'last_active': '2024-09-12T03:23:20.665225+0000', 'last_peered': '2024-09-12T03:23:20.665225+0000', 'last_clean': '2024-09-12T03:23:20.665225+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:23:20.682292+0000', 'last_undegraded': '2024-09-12T03:23:20.682292+0000', 'last_fullsized': '2024-09-12T03:23:20.682292+0000', 'mapping_epoch': 147, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 147, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:23:20.665225+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:23:20.665225+0000', 'last_clean_scrub_stamp': '2024-09-12T03:23:20.665225+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-12T03:23:26.396 INFO:tasks.ceph.ceph_manager.ceph:PG 25.1e is not active+clean 2024-09-12T03:23:26.396 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 147, 'state': 'creating+peering', 'last_fresh': '2024-09-12T03:23:20.678929+0000', 'last_change': '2024-09-12T03:23:20.678929+0000', 'last_active': '2024-09-12T03:23:20.665225+0000', 'last_peered': '2024-09-12T03:23:20.665225+0000', 'last_clean': '2024-09-12T03:23:20.665225+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:23:20.678929+0000', 'last_undegraded': '2024-09-12T03:23:20.678929+0000', 'last_fullsized': '2024-09-12T03:23:20.678929+0000', 'mapping_epoch': 147, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 147, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:23:20.665225+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:23:20.665225+0000', 'last_clean_scrub_stamp': '2024-09-12T03:23:20.665225+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-12T03:23:26.396 INFO:tasks.ceph.ceph_manager.ceph:PG 25.1f is not active+clean 2024-09-12T03:23:26.396 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 147, 'state': 'creating+peering', 'last_fresh': '2024-09-12T03:23:20.678598+0000', 'last_change': '2024-09-12T03:23:20.678598+0000', 'last_active': '2024-09-12T03:23:20.665225+0000', 'last_peered': '2024-09-12T03:23:20.665225+0000', 'last_clean': '2024-09-12T03:23:20.665225+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:23:20.678598+0000', 'last_undegraded': '2024-09-12T03:23:20.678598+0000', 'last_fullsized': '2024-09-12T03:23:20.678598+0000', 'mapping_epoch': 147, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 147, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:23:20.665225+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:23:20.665225+0000', 'last_clean_scrub_stamp': '2024-09-12T03:23:20.665225+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-12T03:23:26.396 INFO:tasks.ceph.ceph_manager.ceph:PG 25.1c is not active+clean 2024-09-12T03:23:26.396 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 147, 'state': 'creating+peering', 'last_fresh': '2024-09-12T03:23:20.672426+0000', 'last_change': '2024-09-12T03:23:20.672426+0000', 'last_active': '2024-09-12T03:23:20.665225+0000', 'last_peered': '2024-09-12T03:23:20.665225+0000', 'last_clean': '2024-09-12T03:23:20.665225+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:23:20.672426+0000', 'last_undegraded': '2024-09-12T03:23:20.672426+0000', 'last_fullsized': '2024-09-12T03:23:20.672426+0000', 'mapping_epoch': 147, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 147, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:23:20.665225+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:23:20.665225+0000', 'last_clean_scrub_stamp': '2024-09-12T03:23:20.665225+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-12T03:23:26.396 INFO:tasks.ceph.ceph_manager.ceph:PG 25.1d is not active+clean 2024-09-12T03:23:26.397 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 147, 'state': 'creating+peering', 'last_fresh': '2024-09-12T03:23:20.672931+0000', 'last_change': '2024-09-12T03:23:20.672931+0000', 'last_active': '2024-09-12T03:23:20.665225+0000', 'last_peered': '2024-09-12T03:23:20.665225+0000', 'last_clean': '2024-09-12T03:23:20.665225+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:23:20.672931+0000', 'last_undegraded': '2024-09-12T03:23:20.672931+0000', 'last_fullsized': '2024-09-12T03:23:20.672931+0000', 'mapping_epoch': 147, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 147, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:23:20.665225+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:23:20.665225+0000', 'last_clean_scrub_stamp': '2024-09-12T03:23:20.665225+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-12T03:23:26.397 INFO:tasks.ceph.ceph_manager.ceph:PG 25.1a is not active+clean 2024-09-12T03:23:26.397 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 147, 'state': 'creating+peering', 'last_fresh': '2024-09-12T03:23:20.678590+0000', 'last_change': '2024-09-12T03:23:20.678590+0000', 'last_active': '2024-09-12T03:23:20.665225+0000', 'last_peered': '2024-09-12T03:23:20.665225+0000', 'last_clean': '2024-09-12T03:23:20.665225+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:23:20.678590+0000', 'last_undegraded': '2024-09-12T03:23:20.678590+0000', 'last_fullsized': '2024-09-12T03:23:20.678590+0000', 'mapping_epoch': 147, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 147, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:23:20.665225+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:23:20.665225+0000', 'last_clean_scrub_stamp': '2024-09-12T03:23:20.665225+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-12T03:23:26.397 INFO:tasks.ceph.ceph_manager.ceph:PG 25.1b is not active+clean 2024-09-12T03:23:26.397 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 147, 'state': 'creating+peering', 'last_fresh': '2024-09-12T03:23:20.674380+0000', 'last_change': '2024-09-12T03:23:20.674380+0000', 'last_active': '2024-09-12T03:23:20.665225+0000', 'last_peered': '2024-09-12T03:23:20.665225+0000', 'last_clean': '2024-09-12T03:23:20.665225+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:23:20.674380+0000', 'last_undegraded': '2024-09-12T03:23:20.674380+0000', 'last_fullsized': '2024-09-12T03:23:20.674380+0000', 'mapping_epoch': 147, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 147, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:23:20.665225+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:23:20.665225+0000', 'last_clean_scrub_stamp': '2024-09-12T03:23:20.665225+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-12T03:23:26.397 INFO:tasks.ceph.ceph_manager.ceph:PG 25.19 is not active+clean 2024-09-12T03:23:26.397 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 147, 'state': 'creating+peering', 'last_fresh': '2024-09-12T03:23:20.671879+0000', 'last_change': '2024-09-12T03:23:20.671879+0000', 'last_active': '2024-09-12T03:23:20.665225+0000', 'last_peered': '2024-09-12T03:23:20.665225+0000', 'last_clean': '2024-09-12T03:23:20.665225+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:23:20.671879+0000', 'last_undegraded': '2024-09-12T03:23:20.671879+0000', 'last_fullsized': '2024-09-12T03:23:20.671879+0000', 'mapping_epoch': 147, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 147, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:23:20.665225+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:23:20.665225+0000', 'last_clean_scrub_stamp': '2024-09-12T03:23:20.665225+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-12T03:23:26.397 INFO:tasks.ceph.ceph_manager.ceph:PG 25.16 is not active+clean 2024-09-12T03:23:26.397 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 147, 'state': 'creating+peering', 'last_fresh': '2024-09-12T03:23:20.675332+0000', 'last_change': '2024-09-12T03:23:20.675332+0000', 'last_active': '2024-09-12T03:23:20.665225+0000', 'last_peered': '2024-09-12T03:23:20.665225+0000', 'last_clean': '2024-09-12T03:23:20.665225+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:23:20.675332+0000', 'last_undegraded': '2024-09-12T03:23:20.675332+0000', 'last_fullsized': '2024-09-12T03:23:20.675332+0000', 'mapping_epoch': 147, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 147, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:23:20.665225+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:23:20.665225+0000', 'last_clean_scrub_stamp': '2024-09-12T03:23:20.665225+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-12T03:23:26.397 INFO:tasks.ceph.ceph_manager.ceph:PG 25.17 is not active+clean 2024-09-12T03:23:26.398 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 147, 'state': 'creating+peering', 'last_fresh': '2024-09-12T03:23:20.674869+0000', 'last_change': '2024-09-12T03:23:20.674869+0000', 'last_active': '2024-09-12T03:23:20.665225+0000', 'last_peered': '2024-09-12T03:23:20.665225+0000', 'last_clean': '2024-09-12T03:23:20.665225+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:23:20.674869+0000', 'last_undegraded': '2024-09-12T03:23:20.674869+0000', 'last_fullsized': '2024-09-12T03:23:20.674869+0000', 'mapping_epoch': 147, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 147, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:23:20.665225+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:23:20.665225+0000', 'last_clean_scrub_stamp': '2024-09-12T03:23:20.665225+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-12T03:23:26.398 INFO:tasks.ceph.ceph_manager.ceph:PG 25.14 is not active+clean 2024-09-12T03:23:26.398 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 147, 'state': 'creating+peering', 'last_fresh': '2024-09-12T03:23:20.671943+0000', 'last_change': '2024-09-12T03:23:20.671943+0000', 'last_active': '2024-09-12T03:23:20.665225+0000', 'last_peered': '2024-09-12T03:23:20.665225+0000', 'last_clean': '2024-09-12T03:23:20.665225+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:23:20.671943+0000', 'last_undegraded': '2024-09-12T03:23:20.671943+0000', 'last_fullsized': '2024-09-12T03:23:20.671943+0000', 'mapping_epoch': 147, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 147, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:23:20.665225+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:23:20.665225+0000', 'last_clean_scrub_stamp': '2024-09-12T03:23:20.665225+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-12T03:23:26.398 INFO:tasks.ceph.ceph_manager.ceph:PG 25.15 is not active+clean 2024-09-12T03:23:26.398 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 147, 'state': 'creating+peering', 'last_fresh': '2024-09-12T03:23:20.677434+0000', 'last_change': '2024-09-12T03:23:20.677434+0000', 'last_active': '2024-09-12T03:23:20.665225+0000', 'last_peered': '2024-09-12T03:23:20.665225+0000', 'last_clean': '2024-09-12T03:23:20.665225+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:23:20.677434+0000', 'last_undegraded': '2024-09-12T03:23:20.677434+0000', 'last_fullsized': '2024-09-12T03:23:20.677434+0000', 'mapping_epoch': 147, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 147, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:23:20.665225+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:23:20.665225+0000', 'last_clean_scrub_stamp': '2024-09-12T03:23:20.665225+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-12T03:23:26.398 INFO:tasks.ceph.ceph_manager.ceph:PG 25.12 is not active+clean 2024-09-12T03:23:26.398 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 147, 'state': 'creating+peering', 'last_fresh': '2024-09-12T03:23:20.676361+0000', 'last_change': '2024-09-12T03:23:20.676361+0000', 'last_active': '2024-09-12T03:23:20.665225+0000', 'last_peered': '2024-09-12T03:23:20.665225+0000', 'last_clean': '2024-09-12T03:23:20.665225+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:23:20.676361+0000', 'last_undegraded': '2024-09-12T03:23:20.676361+0000', 'last_fullsized': '2024-09-12T03:23:20.676361+0000', 'mapping_epoch': 147, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 147, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:23:20.665225+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:23:20.665225+0000', 'last_clean_scrub_stamp': '2024-09-12T03:23:20.665225+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-12T03:23:26.398 INFO:tasks.ceph.ceph_manager.ceph:PG 25.13 is not active+clean 2024-09-12T03:23:26.398 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 147, 'state': 'creating+peering', 'last_fresh': '2024-09-12T03:23:20.672076+0000', 'last_change': '2024-09-12T03:23:20.672076+0000', 'last_active': '2024-09-12T03:23:20.665225+0000', 'last_peered': '2024-09-12T03:23:20.665225+0000', 'last_clean': '2024-09-12T03:23:20.665225+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:23:20.672076+0000', 'last_undegraded': '2024-09-12T03:23:20.672076+0000', 'last_fullsized': '2024-09-12T03:23:20.672076+0000', 'mapping_epoch': 147, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 147, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:23:20.665225+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:23:20.665225+0000', 'last_clean_scrub_stamp': '2024-09-12T03:23:20.665225+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-12T03:23:26.398 INFO:tasks.ceph.ceph_manager.ceph:PG 25.10 is not active+clean 2024-09-12T03:23:26.398 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 147, 'state': 'creating+peering', 'last_fresh': '2024-09-12T03:23:20.672680+0000', 'last_change': '2024-09-12T03:23:20.672680+0000', 'last_active': '2024-09-12T03:23:20.665225+0000', 'last_peered': '2024-09-12T03:23:20.665225+0000', 'last_clean': '2024-09-12T03:23:20.665225+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:23:20.672680+0000', 'last_undegraded': '2024-09-12T03:23:20.672680+0000', 'last_fullsized': '2024-09-12T03:23:20.672680+0000', 'mapping_epoch': 147, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 147, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:23:20.665225+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:23:20.665225+0000', 'last_clean_scrub_stamp': '2024-09-12T03:23:20.665225+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-12T03:23:26.399 INFO:tasks.ceph.ceph_manager.ceph:PG 25.11 is not active+clean 2024-09-12T03:23:26.399 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 147, 'state': 'creating+peering', 'last_fresh': '2024-09-12T03:23:20.676970+0000', 'last_change': '2024-09-12T03:23:20.676970+0000', 'last_active': '2024-09-12T03:23:20.665225+0000', 'last_peered': '2024-09-12T03:23:20.665225+0000', 'last_clean': '2024-09-12T03:23:20.665225+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:23:20.676970+0000', 'last_undegraded': '2024-09-12T03:23:20.676970+0000', 'last_fullsized': '2024-09-12T03:23:20.676970+0000', 'mapping_epoch': 147, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 147, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:23:20.665225+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:23:20.665225+0000', 'last_clean_scrub_stamp': '2024-09-12T03:23:20.665225+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-12T03:23:26.399 INFO:tasks.ceph.ceph_manager.ceph:PG 25.e is not active+clean 2024-09-12T03:23:26.399 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 147, 'state': 'creating+peering', 'last_fresh': '2024-09-12T03:23:20.678489+0000', 'last_change': '2024-09-12T03:23:20.678489+0000', 'last_active': '2024-09-12T03:23:20.665225+0000', 'last_peered': '2024-09-12T03:23:20.665225+0000', 'last_clean': '2024-09-12T03:23:20.665225+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:23:20.678489+0000', 'last_undegraded': '2024-09-12T03:23:20.678489+0000', 'last_fullsized': '2024-09-12T03:23:20.678489+0000', 'mapping_epoch': 147, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 147, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:23:20.665225+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:23:20.665225+0000', 'last_clean_scrub_stamp': '2024-09-12T03:23:20.665225+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-12T03:23:26.399 INFO:tasks.ceph.ceph_manager.ceph:PG 25.f is not active+clean 2024-09-12T03:23:26.399 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 147, 'state': 'creating+peering', 'last_fresh': '2024-09-12T03:23:20.673914+0000', 'last_change': '2024-09-12T03:23:20.673914+0000', 'last_active': '2024-09-12T03:23:20.665225+0000', 'last_peered': '2024-09-12T03:23:20.665225+0000', 'last_clean': '2024-09-12T03:23:20.665225+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:23:20.673914+0000', 'last_undegraded': '2024-09-12T03:23:20.673914+0000', 'last_fullsized': '2024-09-12T03:23:20.673914+0000', 'mapping_epoch': 147, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 147, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:23:20.665225+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:23:20.665225+0000', 'last_clean_scrub_stamp': '2024-09-12T03:23:20.665225+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-12T03:23:26.399 INFO:tasks.ceph.ceph_manager.ceph:PG 25.c is not active+clean 2024-09-12T03:23:26.399 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 147, 'state': 'creating+peering', 'last_fresh': '2024-09-12T03:23:20.672469+0000', 'last_change': '2024-09-12T03:23:20.672469+0000', 'last_active': '2024-09-12T03:23:20.665225+0000', 'last_peered': '2024-09-12T03:23:20.665225+0000', 'last_clean': '2024-09-12T03:23:20.665225+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:23:20.672469+0000', 'last_undegraded': '2024-09-12T03:23:20.672469+0000', 'last_fullsized': '2024-09-12T03:23:20.672469+0000', 'mapping_epoch': 147, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 147, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:23:20.665225+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:23:20.665225+0000', 'last_clean_scrub_stamp': '2024-09-12T03:23:20.665225+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-12T03:23:26.400 INFO:tasks.ceph.ceph_manager.ceph:PG 25.d is not active+clean 2024-09-12T03:23:26.400 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 147, 'state': 'creating+peering', 'last_fresh': '2024-09-12T03:23:20.677217+0000', 'last_change': '2024-09-12T03:23:20.677217+0000', 'last_active': '2024-09-12T03:23:20.665225+0000', 'last_peered': '2024-09-12T03:23:20.665225+0000', 'last_clean': '2024-09-12T03:23:20.665225+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:23:20.677217+0000', 'last_undegraded': '2024-09-12T03:23:20.677217+0000', 'last_fullsized': '2024-09-12T03:23:20.677217+0000', 'mapping_epoch': 147, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 147, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:23:20.665225+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:23:20.665225+0000', 'last_clean_scrub_stamp': '2024-09-12T03:23:20.665225+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-12T03:23:26.400 INFO:tasks.ceph.ceph_manager.ceph:PG 25.a is not active+clean 2024-09-12T03:23:26.400 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 147, 'state': 'creating+peering', 'last_fresh': '2024-09-12T03:23:20.676951+0000', 'last_change': '2024-09-12T03:23:20.676951+0000', 'last_active': '2024-09-12T03:23:20.665225+0000', 'last_peered': '2024-09-12T03:23:20.665225+0000', 'last_clean': '2024-09-12T03:23:20.665225+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:23:20.676951+0000', 'last_undegraded': '2024-09-12T03:23:20.676951+0000', 'last_fullsized': '2024-09-12T03:23:20.676951+0000', 'mapping_epoch': 147, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 147, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:23:20.665225+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:23:20.665225+0000', 'last_clean_scrub_stamp': '2024-09-12T03:23:20.665225+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-12T03:23:26.400 INFO:tasks.ceph.ceph_manager.ceph:PG 25.b is not active+clean 2024-09-12T03:23:26.400 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 147, 'state': 'creating+peering', 'last_fresh': '2024-09-12T03:23:20.671109+0000', 'last_change': '2024-09-12T03:23:20.671109+0000', 'last_active': '2024-09-12T03:23:20.665225+0000', 'last_peered': '2024-09-12T03:23:20.665225+0000', 'last_clean': '2024-09-12T03:23:20.665225+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:23:20.671109+0000', 'last_undegraded': '2024-09-12T03:23:20.671109+0000', 'last_fullsized': '2024-09-12T03:23:20.671109+0000', 'mapping_epoch': 147, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 147, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:23:20.665225+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:23:20.665225+0000', 'last_clean_scrub_stamp': '2024-09-12T03:23:20.665225+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-12T03:23:26.400 INFO:tasks.ceph.ceph_manager.ceph:PG 25.8 is not active+clean 2024-09-12T03:23:26.400 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 147, 'state': 'creating+peering', 'last_fresh': '2024-09-12T03:23:20.675851+0000', 'last_change': '2024-09-12T03:23:20.675851+0000', 'last_active': '2024-09-12T03:23:20.665225+0000', 'last_peered': '2024-09-12T03:23:20.665225+0000', 'last_clean': '2024-09-12T03:23:20.665225+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:23:20.675851+0000', 'last_undegraded': '2024-09-12T03:23:20.675851+0000', 'last_fullsized': '2024-09-12T03:23:20.675851+0000', 'mapping_epoch': 147, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 147, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:23:20.665225+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:23:20.665225+0000', 'last_clean_scrub_stamp': '2024-09-12T03:23:20.665225+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-12T03:23:26.400 INFO:tasks.ceph.ceph_manager.ceph:PG 25.9 is not active+clean 2024-09-12T03:23:26.400 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 147, 'state': 'creating+peering', 'last_fresh': '2024-09-12T03:23:20.681456+0000', 'last_change': '2024-09-12T03:23:20.681456+0000', 'last_active': '2024-09-12T03:23:20.665225+0000', 'last_peered': '2024-09-12T03:23:20.665225+0000', 'last_clean': '2024-09-12T03:23:20.665225+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:23:20.681456+0000', 'last_undegraded': '2024-09-12T03:23:20.681456+0000', 'last_fullsized': '2024-09-12T03:23:20.681456+0000', 'mapping_epoch': 147, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 147, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:23:20.665225+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:23:20.665225+0000', 'last_clean_scrub_stamp': '2024-09-12T03:23:20.665225+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-12T03:23:26.401 INFO:tasks.ceph.ceph_manager.ceph:PG 25.6 is not active+clean 2024-09-12T03:23:26.401 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 147, 'state': 'creating+peering', 'last_fresh': '2024-09-12T03:23:20.681144+0000', 'last_change': '2024-09-12T03:23:20.681144+0000', 'last_active': '2024-09-12T03:23:20.665225+0000', 'last_peered': '2024-09-12T03:23:20.665225+0000', 'last_clean': '2024-09-12T03:23:20.665225+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:23:20.681144+0000', 'last_undegraded': '2024-09-12T03:23:20.681144+0000', 'last_fullsized': '2024-09-12T03:23:20.681144+0000', 'mapping_epoch': 147, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 147, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:23:20.665225+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:23:20.665225+0000', 'last_clean_scrub_stamp': '2024-09-12T03:23:20.665225+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-12T03:23:26.401 INFO:tasks.ceph.ceph_manager.ceph:PG 25.7 is not active+clean 2024-09-12T03:23:26.401 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 147, 'state': 'creating+peering', 'last_fresh': '2024-09-12T03:23:20.676386+0000', 'last_change': '2024-09-12T03:23:20.676386+0000', 'last_active': '2024-09-12T03:23:20.665225+0000', 'last_peered': '2024-09-12T03:23:20.665225+0000', 'last_clean': '2024-09-12T03:23:20.665225+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:23:20.676386+0000', 'last_undegraded': '2024-09-12T03:23:20.676386+0000', 'last_fullsized': '2024-09-12T03:23:20.676386+0000', 'mapping_epoch': 147, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 147, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:23:20.665225+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:23:20.665225+0000', 'last_clean_scrub_stamp': '2024-09-12T03:23:20.665225+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-12T03:23:26.401 INFO:tasks.ceph.ceph_manager.ceph:PG 25.4 is not active+clean 2024-09-12T03:23:26.401 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 147, 'state': 'creating+peering', 'last_fresh': '2024-09-12T03:23:20.677605+0000', 'last_change': '2024-09-12T03:23:20.677605+0000', 'last_active': '2024-09-12T03:23:20.665225+0000', 'last_peered': '2024-09-12T03:23:20.665225+0000', 'last_clean': '2024-09-12T03:23:20.665225+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:23:20.677605+0000', 'last_undegraded': '2024-09-12T03:23:20.677605+0000', 'last_fullsized': '2024-09-12T03:23:20.677605+0000', 'mapping_epoch': 147, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 147, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:23:20.665225+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:23:20.665225+0000', 'last_clean_scrub_stamp': '2024-09-12T03:23:20.665225+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-12T03:23:26.401 INFO:tasks.ceph.ceph_manager.ceph:PG 25.5 is not active+clean 2024-09-12T03:23:26.401 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 147, 'state': 'creating+peering', 'last_fresh': '2024-09-12T03:23:20.677039+0000', 'last_change': '2024-09-12T03:23:20.677039+0000', 'last_active': '2024-09-12T03:23:20.665225+0000', 'last_peered': '2024-09-12T03:23:20.665225+0000', 'last_clean': '2024-09-12T03:23:20.665225+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:23:20.677039+0000', 'last_undegraded': '2024-09-12T03:23:20.677039+0000', 'last_fullsized': '2024-09-12T03:23:20.677039+0000', 'mapping_epoch': 147, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 147, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:23:20.665225+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:23:20.665225+0000', 'last_clean_scrub_stamp': '2024-09-12T03:23:20.665225+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-12T03:23:26.401 INFO:tasks.ceph.ceph_manager.ceph:PG 25.2 is not active+clean 2024-09-12T03:23:26.401 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 147, 'state': 'creating+peering', 'last_fresh': '2024-09-12T03:23:20.675165+0000', 'last_change': '2024-09-12T03:23:20.675165+0000', 'last_active': '2024-09-12T03:23:20.665225+0000', 'last_peered': '2024-09-12T03:23:20.665225+0000', 'last_clean': '2024-09-12T03:23:20.665225+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:23:20.675165+0000', 'last_undegraded': '2024-09-12T03:23:20.675165+0000', 'last_fullsized': '2024-09-12T03:23:20.675165+0000', 'mapping_epoch': 147, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 147, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:23:20.665225+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:23:20.665225+0000', 'last_clean_scrub_stamp': '2024-09-12T03:23:20.665225+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-12T03:23:26.402 INFO:tasks.ceph.ceph_manager.ceph:PG 25.3 is not active+clean 2024-09-12T03:23:26.402 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 147, 'state': 'creating+peering', 'last_fresh': '2024-09-12T03:23:20.675175+0000', 'last_change': '2024-09-12T03:23:20.675175+0000', 'last_active': '2024-09-12T03:23:20.665225+0000', 'last_peered': '2024-09-12T03:23:20.665225+0000', 'last_clean': '2024-09-12T03:23:20.665225+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:23:20.675175+0000', 'last_undegraded': '2024-09-12T03:23:20.675175+0000', 'last_fullsized': '2024-09-12T03:23:20.675175+0000', 'mapping_epoch': 147, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 147, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:23:20.665225+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:23:20.665225+0000', 'last_clean_scrub_stamp': '2024-09-12T03:23:20.665225+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-12T03:23:26.402 INFO:tasks.ceph.ceph_manager.ceph:PG 25.0 is not active+clean 2024-09-12T03:23:26.402 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 147, 'state': 'creating+peering', 'last_fresh': '2024-09-12T03:23:20.675295+0000', 'last_change': '2024-09-12T03:23:20.675295+0000', 'last_active': '2024-09-12T03:23:20.665225+0000', 'last_peered': '2024-09-12T03:23:20.665225+0000', 'last_clean': '2024-09-12T03:23:20.665225+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:23:20.675295+0000', 'last_undegraded': '2024-09-12T03:23:20.675295+0000', 'last_fullsized': '2024-09-12T03:23:20.675295+0000', 'mapping_epoch': 147, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 147, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:23:20.665225+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:23:20.665225+0000', 'last_clean_scrub_stamp': '2024-09-12T03:23:20.665225+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-12T03:23:26.402 INFO:tasks.ceph.ceph_manager.ceph:PG 25.1 is not active+clean 2024-09-12T03:23:26.402 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 147, 'state': 'creating+peering', 'last_fresh': '2024-09-12T03:23:20.680888+0000', 'last_change': '2024-09-12T03:23:20.680888+0000', 'last_active': '2024-09-12T03:23:20.665225+0000', 'last_peered': '2024-09-12T03:23:20.665225+0000', 'last_clean': '2024-09-12T03:23:20.665225+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:23:20.680888+0000', 'last_undegraded': '2024-09-12T03:23:20.680888+0000', 'last_fullsized': '2024-09-12T03:23:20.680888+0000', 'mapping_epoch': 147, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 147, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:23:20.665225+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:23:20.665225+0000', 'last_clean_scrub_stamp': '2024-09-12T03:23:20.665225+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 6], 'acting': [2, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-12T03:23:26.402 INFO:tasks.ceph.ceph_manager.ceph:PG 25.18 is not active+clean 2024-09-12T03:23:26.402 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 147, 'state': 'creating+peering', 'last_fresh': '2024-09-12T03:23:20.678051+0000', 'last_change': '2024-09-12T03:23:20.678051+0000', 'last_active': '2024-09-12T03:23:20.665225+0000', 'last_peered': '2024-09-12T03:23:20.665225+0000', 'last_clean': '2024-09-12T03:23:20.665225+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T03:23:20.678051+0000', 'last_undegraded': '2024-09-12T03:23:20.678051+0000', 'last_fullsized': '2024-09-12T03:23:20.678051+0000', 'mapping_epoch': 147, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 147, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:23:20.665225+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:23:20.665225+0000', 'last_clean_scrub_stamp': '2024-09-12T03:23:20.665225+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-12T03:23:26.403 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-12T03:23:26.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.526+0000 7fc0692f8640 1 Processor -- start 2024-09-12T03:23:26.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.526+0000 7fc0692f8640 1 -- start start 2024-09-12T03:23:26.528 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.527+0000 7fc0692f8640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fc064072d70 0x7fc0640b3560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:26.528 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.527+0000 7fc0692f8640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fc06406e9b0 0x7fc06406ee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:26.529 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.527+0000 7fc0692f8640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc0640ce410 con 0x7fc06415b2b0 2024-09-12T03:23:26.529 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.527+0000 7fc0692f8640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7fc0640720a0 con 0x7fc064072d70 2024-09-12T03:23:26.529 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.527+0000 7fc0692f8640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fc0640732c0 con 0x7fc06406e9b0 2024-09-12T03:23:26.529 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.527+0000 7fc0627fc640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fc064072d70 0x7fc0640b3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:26.529 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.527+0000 7fc062ffd640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fc06415b2b0 0x7fc064072430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:56152/0 (socket says 172.21.15.27:56152) 2024-09-12T03:23:26.529 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.527+0000 7fc062ffd640 1 -- 172.21.15.27:0/3708651902 learned_addr learned my addr 172.21.15.27:0/3708651902 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:23:26.529 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.527+0000 7fc0637fe640 1 --2- 172.21.15.27:0/3708651902 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fc06406e9b0 0x7fc06406ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:26.529 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.527+0000 7fc0627fc640 1 -- 172.21.15.27:0/3708651902 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fc06406e9b0 msgr2=0x7fc06406ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:26.529 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.527+0000 7fc0627fc640 1 --2- 172.21.15.27:0/3708651902 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fc06406e9b0 0x7fc06406ee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:26.529 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.527+0000 7fc0627fc640 1 -- 172.21.15.27:0/3708651902 >> v1:172.21.15.27:6789/0 conn(0x7fc06415b2b0 legacy=0x7fc064072430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:26.529 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.527+0000 7fc0627fc640 1 -- 172.21.15.27:0/3708651902 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc064071ae0 con 0x7fc064072d70 2024-09-12T03:23:26.529 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.527+0000 7fc061ffb640 1 -- 172.21.15.27:0/3708651902 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2392286424 0 0) 0x7fc054001490 con 0x7fc06415b2b0 2024-09-12T03:23:26.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.528+0000 7fc0627fc640 1 --2- 172.21.15.27:0/3708651902 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc064072d70 0x7fc0640b3560 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7fc04c0097a0 tx=0x7fc04c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=349bcca2eec356a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:26.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.528+0000 7fc061ffb640 1 -- 172.21.15.27:0/3708651902 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc04c037070 con 0x7fc064072d70 2024-09-12T03:23:26.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.528+0000 7fc061ffb640 1 -- 172.21.15.27:0/3708651902 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc04c032440 con 0x7fc064072d70 2024-09-12T03:23:26.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.528+0000 7fc061ffb640 1 -- 172.21.15.27:0/3708651902 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc04c032810 con 0x7fc064072d70 2024-09-12T03:23:26.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.529+0000 7fc0692f8640 1 -- 172.21.15.27:0/3708651902 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc064072d70 msgr2=0x7fc0640b3560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:26.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.529+0000 7fc0692f8640 1 --2- 172.21.15.27:0/3708651902 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc064072d70 0x7fc0640b3560 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7fc04c0097a0 tx=0x7fc04c02a180 comp rx=0 tx=0).stop 2024-09-12T03:23:26.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.529+0000 7fc0692f8640 1 -- 172.21.15.27:0/3708651902 shutdown_connections 2024-09-12T03:23:26.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.529+0000 7fc0692f8640 1 --2- 172.21.15.27:0/3708651902 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fc06406e9b0 0x7fc06406ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:26.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.529+0000 7fc0692f8640 1 --2- 172.21.15.27:0/3708651902 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc064072d70 0x7fc0640b3560 unknown :-1 s=CLOSED pgs=1552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:26.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.529+0000 7fc0692f8640 1 -- 172.21.15.27:0/3708651902 >> 172.21.15.27:0/3708651902 conn(0x7fc064062c60 msgr2=0x7fc06406bb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:26.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.529+0000 7fc0692f8640 1 -- 172.21.15.27:0/3708651902 shutdown_connections 2024-09-12T03:23:26.532 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.529+0000 7fc0692f8640 1 -- 172.21.15.27:0/3708651902 wait complete. 2024-09-12T03:23:26.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.530+0000 7fc0692f8640 1 Processor -- start 2024-09-12T03:23:26.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.530+0000 7fc0692f8640 1 -- start start 2024-09-12T03:23:26.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.531+0000 7fc0692f8640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc06406e9b0 0x7fc06414f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:26.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.531+0000 7fc0692f8640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc064072d70 0x7fc064159b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:26.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.531+0000 7fc0692f8640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fc06415b2b0 0x7fc06415a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:26.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.531+0000 7fc0692f8640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fc06415b690 con 0x7fc06406e9b0 2024-09-12T03:23:26.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.531+0000 7fc0692f8640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc0640ce690 con 0x7fc064072d70 2024-09-12T03:23:26.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.531+0000 7fc0692f8640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fc06415b810 con 0x7fc06415b2b0 2024-09-12T03:23:26.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.531+0000 7fc062ffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc06406e9b0 0x7fc06414f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:26.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.531+0000 7fc062ffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc06406e9b0 0x7fc06414f8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42504/0 (socket says 172.21.15.27:42504) 2024-09-12T03:23:26.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.531+0000 7fc062ffd640 1 -- 172.21.15.27:0/65918174 learned_addr learned my addr 172.21.15.27:0/65918174 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:23:26.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.531+0000 7fc0637fe640 1 --2- 172.21.15.27:0/65918174 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fc06415b2b0 0x7fc06415a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:26.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.531+0000 7fc0627fc640 1 --2- 172.21.15.27:0/65918174 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc064072d70 0x7fc064159b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:26.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.532+0000 7fc062ffd640 1 -- 172.21.15.27:0/65918174 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fc06415b2b0 msgr2=0x7fc06415a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:26.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.532+0000 7fc062ffd640 1 --2- 172.21.15.27:0/65918174 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fc06415b2b0 0x7fc06415a040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:26.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.532+0000 7fc062ffd640 1 -- 172.21.15.27:0/65918174 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc064072d70 msgr2=0x7fc064159b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:26.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.532+0000 7fc062ffd640 1 --2- 172.21.15.27:0/65918174 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc064072d70 0x7fc064159b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:26.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.532+0000 7fc062ffd640 1 -- 172.21.15.27:0/65918174 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0641586b0 con 0x7fc06406e9b0 2024-09-12T03:23:26.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.532+0000 7fc062ffd640 1 --2- 172.21.15.27:0/65918174 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc06406e9b0 0x7fc06414f8c0 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7fc05400c7f0 tx=0x7fc05400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:26.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.532+0000 7fc05b7fe640 1 -- 172.21.15.27:0/65918174 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc054018070 con 0x7fc06406e9b0 2024-09-12T03:23:26.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.532+0000 7fc05b7fe640 1 -- 172.21.15.27:0/65918174 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc05400a640 con 0x7fc06406e9b0 2024-09-12T03:23:26.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.532+0000 7fc0692f8640 1 -- 172.21.15.27:0/65918174 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc06414e820 con 0x7fc06406e9b0 2024-09-12T03:23:26.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.532+0000 7fc05b7fe640 1 -- 172.21.15.27:0/65918174 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc05400aa30 con 0x7fc06406e9b0 2024-09-12T03:23:26.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.532+0000 7fc0692f8640 1 -- 172.21.15.27:0/65918174 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc06414ede0 con 0x7fc06406e9b0 2024-09-12T03:23:26.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.533+0000 7fc0692f8640 1 -- 172.21.15.27:0/65918174 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0640b3560 con 0x7fc06406e9b0 2024-09-12T03:23:26.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.533+0000 7fc05b7fe640 1 -- 172.21.15.27:0/65918174 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 24) ==== 91212+0+0 (secure 0 0 0) 0x7fc05400abd0 con 0x7fc06406e9b0 2024-09-12T03:23:26.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.534+0000 7fc05b7fe640 1 --2- 172.21.15.27:0/65918174 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fc04406c610 0x7fc04406ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:26.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.534+0000 7fc05b7fe640 1 -- 172.21.15.27:0/65918174 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 5765+0+0 (secure 0 0 0) 0x7fc054088d60 con 0x7fc06406e9b0 2024-09-12T03:23:26.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.534+0000 7fc0627fc640 1 --2- 172.21.15.27:0/65918174 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fc04406c610 0x7fc04406ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:26.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.534+0000 7fc0627fc640 1 --2- 172.21.15.27:0/65918174 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fc04406c610 0x7fc04406ead0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fc04c034000 tx=0x7fc04c02e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:26.539 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.538+0000 7fc05b7fe640 1 -- 172.21.15.27:0/65918174 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fc0640ce410 con 0x7fc06406e9b0 2024-09-12T03:23:26.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.781+0000 7fc0692f8640 1 -- 172.21.15.27:0/65918174 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7fc0640713b0 con 0x7fc06406e9b0 2024-09-12T03:23:26.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.781+0000 7fc05b7fe640 1 -- 172.21.15.27:0/65918174 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1499 (secure 0 0 0) 0x7fc054058930 con 0x7fc06406e9b0 2024-09-12T03:23:26.783 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:23:26.783 INFO:teuthology.orchestra.run.smithi027.stdout:{"fsid":"d2df504e-2647-4385-a5db-9fdf69f487a0","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":439,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":149,"num_osds":8,"num_up_osds":8,"osd_up_since":1726110979,"num_in_osds":8,"osd_in_since":1726110969,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"creating+peering","count":64},{"state_name":"active+clean","count":33}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":288518144,"bytes_avail":772805595136,"bytes_total":773094113280,"inactive_pgs_ratio":0.65979379415512085},"fsmap":{"epoch":91,"btime":"2024-09-12T03:23:24:700400+0000","id":12,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":12,"rank":0,"name":"c","status":"up:active","gid":8562}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":15,"modified":"2024-09-12T03:23:11.131984+0000","services":{"mds":{"daemons":{"summary":"","b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-12T03:23:26.784 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.783+0000 7fc0692f8640 1 -- 172.21.15.27:0/65918174 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fc04406c610 msgr2=0x7fc04406ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:26.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.783+0000 7fc0692f8640 1 --2- 172.21.15.27:0/65918174 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fc04406c610 0x7fc04406ead0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fc04c034000 tx=0x7fc04c02e000 comp rx=0 tx=0).stop 2024-09-12T03:23:26.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.783+0000 7fc0692f8640 1 -- 172.21.15.27:0/65918174 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc06406e9b0 msgr2=0x7fc06414f8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:26.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.783+0000 7fc0692f8640 1 --2- 172.21.15.27:0/65918174 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc06406e9b0 0x7fc06414f8c0 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7fc05400c7f0 tx=0x7fc05400ccb0 comp rx=0 tx=0).stop 2024-09-12T03:23:26.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.783+0000 7fc0692f8640 1 -- 172.21.15.27:0/65918174 shutdown_connections 2024-09-12T03:23:26.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.783+0000 7fc0692f8640 1 --2- 172.21.15.27:0/65918174 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fc06415b2b0 0x7fc06415a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:26.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.783+0000 7fc0692f8640 1 --2- 172.21.15.27:0/65918174 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc064072d70 0x7fc064159b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:26.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.783+0000 7fc0692f8640 1 --2- 172.21.15.27:0/65918174 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fc04406c610 0x7fc04406ead0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:26.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.783+0000 7fc0692f8640 1 --2- 172.21.15.27:0/65918174 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc06406e9b0 0x7fc06414f8c0 unknown :-1 s=CLOSED pgs=1553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:26.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.783+0000 7fc0692f8640 1 -- 172.21.15.27:0/65918174 >> 172.21.15.27:0/65918174 conn(0x7fc064062c60 msgr2=0x7fc06406bb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:26.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.784+0000 7fc0692f8640 1 -- 172.21.15.27:0/65918174 shutdown_connections 2024-09-12T03:23:26.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.784+0000 7fc0692f8640 1 -- 172.21.15.27:0/65918174 wait complete. 2024-09-12T03:23:26.796 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-09-12T03:23:26.797 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-12T03:23:26.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.898+0000 7f87e218d640 1 Processor -- start 2024-09-12T03:23:26.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.898+0000 7f87e218d640 1 -- start start 2024-09-12T03:23:26.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.899+0000 7f87e218d640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f87dc15eb90 0x7f87dc0d2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:26.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.899+0000 7f87e218d640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f87dc0d5200 0x7f87dc0d1010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:26.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.899+0000 7f87e218d640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f87dc0d19c0 con 0x7f87dc0d1550 2024-09-12T03:23:26.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.899+0000 7f87e218d640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f87dc0b7d20 con 0x7f87dc15eb90 2024-09-12T03:23:26.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.899+0000 7f87e218d640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f87dc0d42e0 con 0x7f87dc0d5200 2024-09-12T03:23:26.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.899+0000 7f87db7fe640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f87dc15eb90 0x7f87dc0d2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:26.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.900+0000 7f87db7fe640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f87dc15eb90 0x7f87dc0d2df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42512/0 (socket says 172.21.15.27:42512) 2024-09-12T03:23:26.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.900+0000 7f87db7fe640 1 -- 172.21.15.27:0/3840475627 learned_addr learned my addr 172.21.15.27:0/3840475627 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:23:26.902 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.900+0000 7f87daffd640 1 --2- 172.21.15.27:0/3840475627 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f87dc0d5200 0x7f87dc0d1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:26.902 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.900+0000 7f87db7fe640 1 -- 172.21.15.27:0/3840475627 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f87dc0d5200 msgr2=0x7f87dc0d1010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:26.902 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.900+0000 7f87db7fe640 1 --2- 172.21.15.27:0/3840475627 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f87dc0d5200 0x7f87dc0d1010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:26.902 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.900+0000 7f87db7fe640 1 -- 172.21.15.27:0/3840475627 >> v1:172.21.15.45:6789/0 conn(0x7f87dc0d1550 legacy=0x7f87dc0cf200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:26.902 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.900+0000 7f87db7fe640 1 -- 172.21.15.27:0/3840475627 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87dc0d3760 con 0x7f87dc15eb90 2024-09-12T03:23:26.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.901+0000 7f87db7fe640 1 --2- 172.21.15.27:0/3840475627 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f87dc15eb90 0x7f87dc0d2df0 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7f87cc009e50 tx=0x7f87cc02a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17f7574a7f2ab545 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:26.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.901+0000 7f87da7fc640 1 -- 172.21.15.27:0/3840475627 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87cc036070 con 0x7f87dc15eb90 2024-09-12T03:23:26.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.901+0000 7f87da7fc640 1 -- 172.21.15.27:0/3840475627 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87cc031600 con 0x7f87dc15eb90 2024-09-12T03:23:26.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.901+0000 7f87da7fc640 1 -- 172.21.15.27:0/3840475627 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87cc0319d0 con 0x7f87dc15eb90 2024-09-12T03:23:26.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.901+0000 7f87e218d640 1 -- 172.21.15.27:0/3840475627 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f87dc15eb90 msgr2=0x7f87dc0d2df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:26.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.901+0000 7f87e218d640 1 --2- 172.21.15.27:0/3840475627 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f87dc15eb90 0x7f87dc0d2df0 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7f87cc009e50 tx=0x7f87cc02a710 comp rx=0 tx=0).stop 2024-09-12T03:23:26.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.901+0000 7f87e218d640 1 -- 172.21.15.27:0/3840475627 shutdown_connections 2024-09-12T03:23:26.904 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.901+0000 7f87e218d640 1 --2- 172.21.15.27:0/3840475627 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f87dc0d5200 0x7f87dc0d1010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:26.904 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.901+0000 7f87e218d640 1 --2- 172.21.15.27:0/3840475627 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f87dc15eb90 0x7f87dc0d2df0 unknown :-1 s=CLOSED pgs=1554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:26.904 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.901+0000 7f87e218d640 1 -- 172.21.15.27:0/3840475627 >> 172.21.15.27:0/3840475627 conn(0x7f87dc06dce0 msgr2=0x7f87dc06e0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:26.904 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.901+0000 7f87e218d640 1 -- 172.21.15.27:0/3840475627 shutdown_connections 2024-09-12T03:23:26.904 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.901+0000 7f87e218d640 1 -- 172.21.15.27:0/3840475627 wait complete. 2024-09-12T03:23:26.904 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.903+0000 7f87e218d640 1 Processor -- start 2024-09-12T03:23:26.906 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.903+0000 7f87e218d640 1 -- start start 2024-09-12T03:23:26.906 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.903+0000 7f87e218d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f87dc0d1550 0x7f87dc1570f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:26.906 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.903+0000 7f87e218d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f87dc0d5200 0x7f87dc0ca040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:26.906 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.904+0000 7f87e218d640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f87dc15eb90 0x7f87dc0ca580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:26.906 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.904+0000 7f87e218d640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f87dc0d4000 con 0x7f87dc0d5200 2024-09-12T03:23:26.906 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.904+0000 7f87e218d640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f87dc0d3d20 con 0x7f87dc0d1550 2024-09-12T03:23:26.906 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.904+0000 7f87e218d640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f87dc0d3330 con 0x7f87dc15eb90 2024-09-12T03:23:26.906 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.904+0000 7f87daffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f87dc0d5200 0x7f87dc0ca040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:26.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.904+0000 7f87daffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f87dc0d5200 0x7f87dc0ca040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42524/0 (socket says 172.21.15.27:42524) 2024-09-12T03:23:26.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.904+0000 7f87daffd640 1 -- 172.21.15.27:0/498796016 learned_addr learned my addr 172.21.15.27:0/498796016 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:23:26.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.904+0000 7f87db7fe640 1 --2- 172.21.15.27:0/498796016 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f87dc0d1550 0x7f87dc1570f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:26.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.904+0000 7f87dbfff640 1 --2- 172.21.15.27:0/498796016 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f87dc15eb90 0x7f87dc0ca580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:26.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.904+0000 7f87daffd640 1 -- 172.21.15.27:0/498796016 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f87dc15eb90 msgr2=0x7f87dc0ca580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:26.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.904+0000 7f87daffd640 1 --2- 172.21.15.27:0/498796016 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f87dc15eb90 0x7f87dc0ca580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:26.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.904+0000 7f87daffd640 1 -- 172.21.15.27:0/498796016 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f87dc0d1550 msgr2=0x7f87dc1570f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:26.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.904+0000 7f87daffd640 1 --2- 172.21.15.27:0/498796016 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f87dc0d1550 0x7f87dc1570f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:26.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.904+0000 7f87daffd640 1 -- 172.21.15.27:0/498796016 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87dc1f6050 con 0x7f87dc0d5200 2024-09-12T03:23:26.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.904+0000 7f87dbfff640 1 --2- 172.21.15.27:0/498796016 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f87dc15eb90 0x7f87dc0ca580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:23:26.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.904+0000 7f87daffd640 1 --2- 172.21.15.27:0/498796016 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f87dc0d5200 0x7f87dc0ca040 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7f87c400e9b0 tx=0x7f87c400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:26.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.904+0000 7f87d3fff640 1 -- 172.21.15.27:0/498796016 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87c4017070 con 0x7f87dc0d5200 2024-09-12T03:23:26.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.904+0000 7f87d3fff640 1 -- 172.21.15.27:0/498796016 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87c400c9a0 con 0x7f87dc0d5200 2024-09-12T03:23:26.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.904+0000 7f87e218d640 1 -- 172.21.15.27:0/498796016 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87dc1f5950 con 0x7f87dc0d5200 2024-09-12T03:23:26.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.904+0000 7f87d3fff640 1 -- 172.21.15.27:0/498796016 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87c400cd70 con 0x7f87dc0d5200 2024-09-12T03:23:26.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.904+0000 7f87e218d640 1 -- 172.21.15.27:0/498796016 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87dc1f65d0 con 0x7f87dc0d5200 2024-09-12T03:23:26.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.905+0000 7f87e218d640 1 -- 172.21.15.27:0/498796016 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87dc15ef70 con 0x7f87dc0d5200 2024-09-12T03:23:26.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.906+0000 7f87d3fff640 1 -- 172.21.15.27:0/498796016 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 24) ==== 91212+0+0 (secure 0 0 0) 0x7f87c401a070 con 0x7f87dc0d5200 2024-09-12T03:23:26.909 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.906+0000 7f87d3fff640 1 --2- 172.21.15.27:0/498796016 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f87bc06c610 0x7f87bc06ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:26.909 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.906+0000 7f87d3fff640 1 -- 172.21.15.27:0/498796016 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 5765+0+0 (secure 0 0 0) 0x7f87c4089b40 con 0x7f87dc0d5200 2024-09-12T03:23:26.909 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.907+0000 7f87db7fe640 1 --2- 172.21.15.27:0/498796016 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f87bc06c610 0x7f87bc06ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:26.909 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.907+0000 7f87db7fe640 1 --2- 172.21.15.27:0/498796016 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f87bc06c610 0x7f87bc06ead0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f87cc033000 tx=0x7f87cc009000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:26.911 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:26.911+0000 7f87d3fff640 1 -- 172.21.15.27:0/498796016 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f87c4059790 con 0x7f87dc0d5200 2024-09-12T03:23:27.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:27.083+0000 7f87e218d640 1 -- 172.21.15.27:0/498796016 --> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f87dc0d2980 con 0x7f87bc06c610 2024-09-12T03:23:27.090 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:27.089+0000 7f87d3fff640 1 -- 172.21.15.27:0/498796016 <== mgr.4113 v2:172.21.15.27:6836/1226920526 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247090 (secure 0 0 0) 0x7f87dc0d2980 con 0x7f87bc06c610 2024-09-12T03:23:27.093 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:23:27.094 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2024-09-12T03:23:27.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:27.092+0000 7f87e218d640 1 -- 172.21.15.27:0/498796016 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f87bc06c610 msgr2=0x7f87bc06ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:27.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:27.092+0000 7f87e218d640 1 --2- 172.21.15.27:0/498796016 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f87bc06c610 0x7f87bc06ead0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f87cc033000 tx=0x7f87cc009000 comp rx=0 tx=0).stop 2024-09-12T03:23:27.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:27.092+0000 7f87e218d640 1 -- 172.21.15.27:0/498796016 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f87dc0d5200 msgr2=0x7f87dc0ca040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:27.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:27.092+0000 7f87e218d640 1 --2- 172.21.15.27:0/498796016 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f87dc0d5200 0x7f87dc0ca040 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7f87c400e9b0 tx=0x7f87c400ee70 comp rx=0 tx=0).stop 2024-09-12T03:23:27.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:27.092+0000 7f87e218d640 1 -- 172.21.15.27:0/498796016 shutdown_connections 2024-09-12T03:23:27.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:27.092+0000 7f87e218d640 1 --2- 172.21.15.27:0/498796016 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f87dc15eb90 0x7f87dc0ca580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:27.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:27.092+0000 7f87e218d640 1 --2- 172.21.15.27:0/498796016 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f87bc06c610 0x7f87bc06ead0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:27.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:27.092+0000 7f87e218d640 1 --2- 172.21.15.27:0/498796016 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f87dc0d5200 0x7f87dc0ca040 unknown :-1 s=CLOSED pgs=1555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:27.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:27.092+0000 7f87e218d640 1 --2- 172.21.15.27:0/498796016 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f87dc0d1550 0x7f87dc1570f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:27.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:27.092+0000 7f87e218d640 1 -- 172.21.15.27:0/498796016 >> 172.21.15.27:0/498796016 conn(0x7f87dc06dce0 msgr2=0x7f87dc0cfcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:27.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:27.093+0000 7f87e218d640 1 -- 172.21.15.27:0/498796016 shutdown_connections 2024-09-12T03:23:27.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:27.093+0000 7f87e218d640 1 -- 172.21.15.27:0/498796016 wait complete. 2024-09-12T03:23:27.104 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":178,"stamp":"2024-09-12T03:23:25.134647+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":41,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":281756,"kb_used_data":15588,"kb_used_omap":12,"kb_used_meta":265971,"kb_avail":754692964,"statfs":{"total":773094113280,"available":772805595136,"internally_reserved":0,"allocated":15962112,"data_stored":7700919,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":272354864},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":15,"apply_latency_ms":15,"commit_latency_ns":15000000,"apply_latency_ns":15000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.468635"},"pg_stats":[{"pgid":"25.3f","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.676683+0000","last_change":"2024-09-12T03:23:20.676683+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.676683+0000","last_undegraded":"2024-09-12T03:23:20.676683+0000","last_fullsized":"2024-09-12T03:23:20.676683+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.3e","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.677002+0000","last_change":"2024-09-12T03:23:20.677002+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.677002+0000","last_undegraded":"2024-09-12T03:23:20.677002+0000","last_fullsized":"2024-09-12T03:23:20.677002+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.3d","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.680808+0000","last_change":"2024-09-12T03:23:20.680808+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.680808+0000","last_undegraded":"2024-09-12T03:23:20.680808+0000","last_fullsized":"2024-09-12T03:23:20.680808+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.3c","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.677777+0000","last_change":"2024-09-12T03:23:20.677777+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.677777+0000","last_undegraded":"2024-09-12T03:23:20.677777+0000","last_fullsized":"2024-09-12T03:23:20.677777+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.3b","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.683306+0000","last_change":"2024-09-12T03:23:20.683306+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.683306+0000","last_undegraded":"2024-09-12T03:23:20.683306+0000","last_fullsized":"2024-09-12T03:23:20.683306+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.3a","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.680330+0000","last_change":"2024-09-12T03:23:20.680330+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.680330+0000","last_undegraded":"2024-09-12T03:23:20.680330+0000","last_fullsized":"2024-09-12T03:23:20.680330+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.39","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.672348+0000","last_change":"2024-09-12T03:23:20.672348+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.672348+0000","last_undegraded":"2024-09-12T03:23:20.672348+0000","last_fullsized":"2024-09-12T03:23:20.672348+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.38","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.676156+0000","last_change":"2024-09-12T03:23:20.676156+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.676156+0000","last_undegraded":"2024-09-12T03:23:20.676156+0000","last_fullsized":"2024-09-12T03:23:20.676156+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.37","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.674337+0000","last_change":"2024-09-12T03:23:20.674337+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.674337+0000","last_undegraded":"2024-09-12T03:23:20.674337+0000","last_fullsized":"2024-09-12T03:23:20.674337+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.36","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.679738+0000","last_change":"2024-09-12T03:23:20.679738+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.679738+0000","last_undegraded":"2024-09-12T03:23:20.679738+0000","last_fullsized":"2024-09-12T03:23:20.679738+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.35","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.671126+0000","last_change":"2024-09-12T03:23:20.671126+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.671126+0000","last_undegraded":"2024-09-12T03:23:20.671126+0000","last_fullsized":"2024-09-12T03:23:20.671126+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.34","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.677214+0000","last_change":"2024-09-12T03:23:20.677214+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.677214+0000","last_undegraded":"2024-09-12T03:23:20.677214+0000","last_fullsized":"2024-09-12T03:23:20.677214+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.33","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.677238+0000","last_change":"2024-09-12T03:23:20.677238+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.677238+0000","last_undegraded":"2024-09-12T03:23:20.677238+0000","last_fullsized":"2024-09-12T03:23:20.677238+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.32","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.675036+0000","last_change":"2024-09-12T03:23:20.675036+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.675036+0000","last_undegraded":"2024-09-12T03:23:20.675036+0000","last_fullsized":"2024-09-12T03:23:20.675036+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.31","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.675841+0000","last_change":"2024-09-12T03:23:20.675841+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.675841+0000","last_undegraded":"2024-09-12T03:23:20.675841+0000","last_fullsized":"2024-09-12T03:23:20.675841+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.30","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.675750+0000","last_change":"2024-09-12T03:23:20.675750+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.675750+0000","last_undegraded":"2024-09-12T03:23:20.675750+0000","last_fullsized":"2024-09-12T03:23:20.675750+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.2f","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.687993+0000","last_change":"2024-09-12T03:23:20.687993+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.687993+0000","last_undegraded":"2024-09-12T03:23:20.687993+0000","last_fullsized":"2024-09-12T03:23:20.687993+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.2e","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.676010+0000","last_change":"2024-09-12T03:23:20.676010+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.676010+0000","last_undegraded":"2024-09-12T03:23:20.676010+0000","last_fullsized":"2024-09-12T03:23:20.676010+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.2d","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.680215+0000","last_change":"2024-09-12T03:23:20.680215+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.680215+0000","last_undegraded":"2024-09-12T03:23:20.680215+0000","last_fullsized":"2024-09-12T03:23:20.680215+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.2c","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.685697+0000","last_change":"2024-09-12T03:23:20.685697+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.685697+0000","last_undegraded":"2024-09-12T03:23:20.685697+0000","last_fullsized":"2024-09-12T03:23:20.685697+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.2b","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.682781+0000","last_change":"2024-09-12T03:23:20.682781+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.682781+0000","last_undegraded":"2024-09-12T03:23:20.682781+0000","last_fullsized":"2024-09-12T03:23:20.682781+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.2a","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.679222+0000","last_change":"2024-09-12T03:23:20.679222+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.679222+0000","last_undegraded":"2024-09-12T03:23:20.679222+0000","last_fullsized":"2024-09-12T03:23:20.679222+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.29","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.679592+0000","last_change":"2024-09-12T03:23:20.679592+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.679592+0000","last_undegraded":"2024-09-12T03:23:20.679592+0000","last_fullsized":"2024-09-12T03:23:20.679592+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.28","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.675319+0000","last_change":"2024-09-12T03:23:20.675319+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.675319+0000","last_undegraded":"2024-09-12T03:23:20.675319+0000","last_fullsized":"2024-09-12T03:23:20.675319+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.27","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.676115+0000","last_change":"2024-09-12T03:23:20.676115+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.676115+0000","last_undegraded":"2024-09-12T03:23:20.676115+0000","last_fullsized":"2024-09-12T03:23:20.676115+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.26","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.671542+0000","last_change":"2024-09-12T03:23:20.671542+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.671542+0000","last_undegraded":"2024-09-12T03:23:20.671542+0000","last_fullsized":"2024-09-12T03:23:20.671542+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.25","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.676098+0000","last_change":"2024-09-12T03:23:20.676098+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.676098+0000","last_undegraded":"2024-09-12T03:23:20.676098+0000","last_fullsized":"2024-09-12T03:23:20.676098+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.24","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.685181+0000","last_change":"2024-09-12T03:23:20.685181+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.685181+0000","last_undegraded":"2024-09-12T03:23:20.685181+0000","last_fullsized":"2024-09-12T03:23:20.685181+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.23","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.676786+0000","last_change":"2024-09-12T03:23:20.676786+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.676786+0000","last_undegraded":"2024-09-12T03:23:20.676786+0000","last_fullsized":"2024-09-12T03:23:20.676786+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.22","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.674611+0000","last_change":"2024-09-12T03:23:20.674611+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.674611+0000","last_undegraded":"2024-09-12T03:23:20.674611+0000","last_fullsized":"2024-09-12T03:23:20.674611+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.21","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.676607+0000","last_change":"2024-09-12T03:23:20.676607+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.676607+0000","last_undegraded":"2024-09-12T03:23:20.676607+0000","last_fullsized":"2024-09-12T03:23:20.676607+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.20","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.682292+0000","last_change":"2024-09-12T03:23:20.682292+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.682292+0000","last_undegraded":"2024-09-12T03:23:20.682292+0000","last_fullsized":"2024-09-12T03:23:20.682292+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1e","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.678929+0000","last_change":"2024-09-12T03:23:20.678929+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.678929+0000","last_undegraded":"2024-09-12T03:23:20.678929+0000","last_fullsized":"2024-09-12T03:23:20.678929+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1f","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-12T03:23:20.688172+0000","last_change":"2024-09-12T03:23:20.682141+0000","last_active":"2024-09-12T03:23:20.688172+0000","last_peered":"2024-09-12T03:23:20.688172+0000","last_clean":"2024-09-12T03:23:20.688172+0000","last_became_active":"2024-09-12T03:23:20.681852+0000","last_became_peered":"2024-09-12T03:23:20.681852+0000","last_unstale":"2024-09-12T03:23:20.688172+0000","last_undegraded":"2024-09-12T03:23:20.688172+0000","last_fullsized":"2024-09-12T03:23:20.688172+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_clean_scrub_stamp":"2024-09-12T03:23:19.659665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:20:17.004278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1f","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.678598+0000","last_change":"2024-09-12T03:23:20.678598+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.678598+0000","last_undegraded":"2024-09-12T03:23:20.678598+0000","last_fullsized":"2024-09-12T03:23:20.678598+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.1e","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-12T03:23:20.687095+0000","last_change":"2024-09-12T03:23:20.686262+0000","last_active":"2024-09-12T03:23:20.687095+0000","last_peered":"2024-09-12T03:23:20.687095+0000","last_clean":"2024-09-12T03:23:20.687095+0000","last_became_active":"2024-09-12T03:23:20.685552+0000","last_became_peered":"2024-09-12T03:23:20.685552+0000","last_unstale":"2024-09-12T03:23:20.687095+0000","last_undegraded":"2024-09-12T03:23:20.687095+0000","last_fullsized":"2024-09-12T03:23:20.687095+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_clean_scrub_stamp":"2024-09-12T03:23:19.659665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:20:51.463260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.1c","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.672426+0000","last_change":"2024-09-12T03:23:20.672426+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.672426+0000","last_undegraded":"2024-09-12T03:23:20.672426+0000","last_fullsized":"2024-09-12T03:23:20.672426+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.1d","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-12T03:23:20.695515+0000","last_change":"2024-09-12T03:23:20.690566+0000","last_active":"2024-09-12T03:23:20.695515+0000","last_peered":"2024-09-12T03:23:20.695515+0000","last_clean":"2024-09-12T03:23:20.695515+0000","last_became_active":"2024-09-12T03:23:20.690296+0000","last_became_peered":"2024-09-12T03:23:20.690296+0000","last_unstale":"2024-09-12T03:23:20.695515+0000","last_undegraded":"2024-09-12T03:23:20.695515+0000","last_fullsized":"2024-09-12T03:23:20.695515+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_clean_scrub_stamp":"2024-09-12T03:23:19.659665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:13:40.613414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1d","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.672931+0000","last_change":"2024-09-12T03:23:20.672931+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.672931+0000","last_undegraded":"2024-09-12T03:23:20.672931+0000","last_fullsized":"2024-09-12T03:23:20.672931+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.1c","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-12T03:23:20.687327+0000","last_change":"2024-09-12T03:23:20.686667+0000","last_active":"2024-09-12T03:23:20.687327+0000","last_peered":"2024-09-12T03:23:20.687327+0000","last_clean":"2024-09-12T03:23:20.687327+0000","last_became_active":"2024-09-12T03:23:20.686355+0000","last_became_peered":"2024-09-12T03:23:20.686355+0000","last_unstale":"2024-09-12T03:23:20.687327+0000","last_undegraded":"2024-09-12T03:23:20.687327+0000","last_fullsized":"2024-09-12T03:23:20.687327+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_clean_scrub_stamp":"2024-09-12T03:23:19.659665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:01:34.049378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.1a","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.678590+0000","last_change":"2024-09-12T03:23:20.678590+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.678590+0000","last_undegraded":"2024-09-12T03:23:20.678590+0000","last_fullsized":"2024-09-12T03:23:20.678590+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.1b","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-12T03:23:20.686092+0000","last_change":"2024-09-12T03:23:20.684507+0000","last_active":"2024-09-12T03:23:20.686092+0000","last_peered":"2024-09-12T03:23:20.686092+0000","last_clean":"2024-09-12T03:23:20.686092+0000","last_became_active":"2024-09-12T03:23:20.684246+0000","last_became_peered":"2024-09-12T03:23:20.684246+0000","last_unstale":"2024-09-12T03:23:20.686092+0000","last_undegraded":"2024-09-12T03:23:20.686092+0000","last_fullsized":"2024-09-12T03:23:20.686092+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_clean_scrub_stamp":"2024-09-12T03:23:19.659665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:39:13.426726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1b","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.674380+0000","last_change":"2024-09-12T03:23:20.674380+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.674380+0000","last_undegraded":"2024-09-12T03:23:20.674380+0000","last_fullsized":"2024-09-12T03:23:20.674380+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1a","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-12T03:23:20.697734+0000","last_change":"2024-09-12T03:23:20.696117+0000","last_active":"2024-09-12T03:23:20.697734+0000","last_peered":"2024-09-12T03:23:20.697734+0000","last_clean":"2024-09-12T03:23:20.697734+0000","last_became_active":"2024-09-12T03:23:20.695795+0000","last_became_peered":"2024-09-12T03:23:20.695795+0000","last_unstale":"2024-09-12T03:23:20.697734+0000","last_undegraded":"2024-09-12T03:23:20.697734+0000","last_fullsized":"2024-09-12T03:23:20.697734+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_clean_scrub_stamp":"2024-09-12T03:23:19.659665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:36:38.656553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.19","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.671879+0000","last_change":"2024-09-12T03:23:20.671879+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.671879+0000","last_undegraded":"2024-09-12T03:23:20.671879+0000","last_fullsized":"2024-09-12T03:23:20.671879+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.18","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-12T03:23:20.690256+0000","last_change":"2024-09-12T03:23:20.689802+0000","last_active":"2024-09-12T03:23:20.690256+0000","last_peered":"2024-09-12T03:23:20.690256+0000","last_clean":"2024-09-12T03:23:20.690256+0000","last_became_active":"2024-09-12T03:23:20.689503+0000","last_became_peered":"2024-09-12T03:23:20.689503+0000","last_unstale":"2024-09-12T03:23:20.690256+0000","last_undegraded":"2024-09-12T03:23:20.690256+0000","last_fullsized":"2024-09-12T03:23:20.690256+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_clean_scrub_stamp":"2024-09-12T03:23:19.659665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:29:59.930252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.16","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.675332+0000","last_change":"2024-09-12T03:23:20.675332+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.675332+0000","last_undegraded":"2024-09-12T03:23:20.675332+0000","last_fullsized":"2024-09-12T03:23:20.675332+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.17","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-12T03:23:20.679242+0000","last_change":"2024-09-12T03:23:20.678720+0000","last_active":"2024-09-12T03:23:20.679242+0000","last_peered":"2024-09-12T03:23:20.679242+0000","last_clean":"2024-09-12T03:23:20.679242+0000","last_became_active":"2024-09-12T03:23:20.678415+0000","last_became_peered":"2024-09-12T03:23:20.678415+0000","last_unstale":"2024-09-12T03:23:20.679242+0000","last_undegraded":"2024-09-12T03:23:20.679242+0000","last_fullsized":"2024-09-12T03:23:20.679242+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_clean_scrub_stamp":"2024-09-12T03:23:19.659665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:22:49.298961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.17","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.674869+0000","last_change":"2024-09-12T03:23:20.674869+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.674869+0000","last_undegraded":"2024-09-12T03:23:20.674869+0000","last_fullsized":"2024-09-12T03:23:20.674869+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.16","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-12T03:23:20.686685+0000","last_change":"2024-09-12T03:23:20.683066+0000","last_active":"2024-09-12T03:23:20.686685+0000","last_peered":"2024-09-12T03:23:20.686685+0000","last_clean":"2024-09-12T03:23:20.686685+0000","last_became_active":"2024-09-12T03:23:20.682759+0000","last_became_peered":"2024-09-12T03:23:20.682759+0000","last_unstale":"2024-09-12T03:23:20.686685+0000","last_undegraded":"2024-09-12T03:23:20.686685+0000","last_fullsized":"2024-09-12T03:23:20.686685+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_clean_scrub_stamp":"2024-09-12T03:23:19.659665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:51:25.370412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.14","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.671943+0000","last_change":"2024-09-12T03:23:20.671943+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.671943+0000","last_undegraded":"2024-09-12T03:23:20.671943+0000","last_fullsized":"2024-09-12T03:23:20.671943+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.15","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-12T03:23:20.693567+0000","last_change":"2024-09-12T03:23:20.691649+0000","last_active":"2024-09-12T03:23:20.693567+0000","last_peered":"2024-09-12T03:23:20.693567+0000","last_clean":"2024-09-12T03:23:20.693567+0000","last_became_active":"2024-09-12T03:23:20.691385+0000","last_became_peered":"2024-09-12T03:23:20.691385+0000","last_unstale":"2024-09-12T03:23:20.693567+0000","last_undegraded":"2024-09-12T03:23:20.693567+0000","last_fullsized":"2024-09-12T03:23:20.693567+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_clean_scrub_stamp":"2024-09-12T03:23:19.659665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:51:15.001810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.15","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.677434+0000","last_change":"2024-09-12T03:23:20.677434+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.677434+0000","last_undegraded":"2024-09-12T03:23:20.677434+0000","last_fullsized":"2024-09-12T03:23:20.677434+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.14","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-12T03:23:20.684819+0000","last_change":"2024-09-12T03:23:20.683263+0000","last_active":"2024-09-12T03:23:20.684819+0000","last_peered":"2024-09-12T03:23:20.684819+0000","last_clean":"2024-09-12T03:23:20.684819+0000","last_became_active":"2024-09-12T03:23:20.682986+0000","last_became_peered":"2024-09-12T03:23:20.682986+0000","last_unstale":"2024-09-12T03:23:20.684819+0000","last_undegraded":"2024-09-12T03:23:20.684819+0000","last_fullsized":"2024-09-12T03:23:20.684819+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_clean_scrub_stamp":"2024-09-12T03:23:19.659665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:17:57.817757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.12","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.676361+0000","last_change":"2024-09-12T03:23:20.676361+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.676361+0000","last_undegraded":"2024-09-12T03:23:20.676361+0000","last_fullsized":"2024-09-12T03:23:20.676361+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.13","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-12T03:23:20.690520+0000","last_change":"2024-09-12T03:23:20.689845+0000","last_active":"2024-09-12T03:23:20.690520+0000","last_peered":"2024-09-12T03:23:20.690520+0000","last_clean":"2024-09-12T03:23:20.690520+0000","last_became_active":"2024-09-12T03:23:20.689485+0000","last_became_peered":"2024-09-12T03:23:20.689485+0000","last_unstale":"2024-09-12T03:23:20.690520+0000","last_undegraded":"2024-09-12T03:23:20.690520+0000","last_fullsized":"2024-09-12T03:23:20.690520+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_clean_scrub_stamp":"2024-09-12T03:23:19.659665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:41:28.835418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.13","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.672076+0000","last_change":"2024-09-12T03:23:20.672076+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.672076+0000","last_undegraded":"2024-09-12T03:23:20.672076+0000","last_fullsized":"2024-09-12T03:23:20.672076+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.12","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-12T03:23:20.692153+0000","last_change":"2024-09-12T03:23:20.689243+0000","last_active":"2024-09-12T03:23:20.692153+0000","last_peered":"2024-09-12T03:23:20.692153+0000","last_clean":"2024-09-12T03:23:20.692153+0000","last_became_active":"2024-09-12T03:23:20.688661+0000","last_became_peered":"2024-09-12T03:23:20.688661+0000","last_unstale":"2024-09-12T03:23:20.692153+0000","last_undegraded":"2024-09-12T03:23:20.692153+0000","last_fullsized":"2024-09-12T03:23:20.692153+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_clean_scrub_stamp":"2024-09-12T03:23:19.659665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:16:21.498607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.10","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.672680+0000","last_change":"2024-09-12T03:23:20.672680+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.672680+0000","last_undegraded":"2024-09-12T03:23:20.672680+0000","last_fullsized":"2024-09-12T03:23:20.672680+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.11","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-12T03:23:20.687225+0000","last_change":"2024-09-12T03:23:20.680790+0000","last_active":"2024-09-12T03:23:20.687225+0000","last_peered":"2024-09-12T03:23:20.687225+0000","last_clean":"2024-09-12T03:23:20.687225+0000","last_became_active":"2024-09-12T03:23:20.680508+0000","last_became_peered":"2024-09-12T03:23:20.680508+0000","last_unstale":"2024-09-12T03:23:20.687225+0000","last_undegraded":"2024-09-12T03:23:20.687225+0000","last_fullsized":"2024-09-12T03:23:20.687225+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_clean_scrub_stamp":"2024-09-12T03:23:19.659665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:56:34.246022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.11","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.676970+0000","last_change":"2024-09-12T03:23:20.676970+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.676970+0000","last_undegraded":"2024-09-12T03:23:20.676970+0000","last_fullsized":"2024-09-12T03:23:20.676970+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.10","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-12T03:23:20.690498+0000","last_change":"2024-09-12T03:23:20.689843+0000","last_active":"2024-09-12T03:23:20.690498+0000","last_peered":"2024-09-12T03:23:20.690498+0000","last_clean":"2024-09-12T03:23:20.690498+0000","last_became_active":"2024-09-12T03:23:20.689517+0000","last_became_peered":"2024-09-12T03:23:20.689517+0000","last_unstale":"2024-09-12T03:23:20.690498+0000","last_undegraded":"2024-09-12T03:23:20.690498+0000","last_fullsized":"2024-09-12T03:23:20.690498+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_clean_scrub_stamp":"2024-09-12T03:23:19.659665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:27:53.769921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.e","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.678489+0000","last_change":"2024-09-12T03:23:20.678489+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.678489+0000","last_undegraded":"2024-09-12T03:23:20.678489+0000","last_fullsized":"2024-09-12T03:23:20.678489+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.f","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-12T03:23:20.692368+0000","last_change":"2024-09-12T03:23:20.689669+0000","last_active":"2024-09-12T03:23:20.692368+0000","last_peered":"2024-09-12T03:23:20.692368+0000","last_clean":"2024-09-12T03:23:20.692368+0000","last_became_active":"2024-09-12T03:23:20.689291+0000","last_became_peered":"2024-09-12T03:23:20.689291+0000","last_unstale":"2024-09-12T03:23:20.692368+0000","last_undegraded":"2024-09-12T03:23:20.692368+0000","last_fullsized":"2024-09-12T03:23:20.692368+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_clean_scrub_stamp":"2024-09-12T03:23:19.659665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:25:29.729322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.f","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.673914+0000","last_change":"2024-09-12T03:23:20.673914+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.673914+0000","last_undegraded":"2024-09-12T03:23:20.673914+0000","last_fullsized":"2024-09-12T03:23:20.673914+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.e","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-12T03:23:20.692311+0000","last_change":"2024-09-12T03:23:20.691112+0000","last_active":"2024-09-12T03:23:20.692311+0000","last_peered":"2024-09-12T03:23:20.692311+0000","last_clean":"2024-09-12T03:23:20.692311+0000","last_became_active":"2024-09-12T03:23:20.690862+0000","last_became_peered":"2024-09-12T03:23:20.690862+0000","last_unstale":"2024-09-12T03:23:20.692311+0000","last_undegraded":"2024-09-12T03:23:20.692311+0000","last_fullsized":"2024-09-12T03:23:20.692311+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_clean_scrub_stamp":"2024-09-12T03:23:19.659665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:42:16.029521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.c","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.672469+0000","last_change":"2024-09-12T03:23:20.672469+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.672469+0000","last_undegraded":"2024-09-12T03:23:20.672469+0000","last_fullsized":"2024-09-12T03:23:20.672469+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.d","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-12T03:23:20.688563+0000","last_change":"2024-09-12T03:23:20.688103+0000","last_active":"2024-09-12T03:23:20.688563+0000","last_peered":"2024-09-12T03:23:20.688563+0000","last_clean":"2024-09-12T03:23:20.688563+0000","last_became_active":"2024-09-12T03:23:20.687797+0000","last_became_peered":"2024-09-12T03:23:20.687797+0000","last_unstale":"2024-09-12T03:23:20.688563+0000","last_undegraded":"2024-09-12T03:23:20.688563+0000","last_fullsized":"2024-09-12T03:23:20.688563+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_clean_scrub_stamp":"2024-09-12T03:23:19.659665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:34:39.656922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.d","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.677217+0000","last_change":"2024-09-12T03:23:20.677217+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.677217+0000","last_undegraded":"2024-09-12T03:23:20.677217+0000","last_fullsized":"2024-09-12T03:23:20.677217+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.c","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-12T03:23:20.684552+0000","last_change":"2024-09-12T03:23:20.682322+0000","last_active":"2024-09-12T03:23:20.684552+0000","last_peered":"2024-09-12T03:23:20.684552+0000","last_clean":"2024-09-12T03:23:20.684552+0000","last_became_active":"2024-09-12T03:23:20.682033+0000","last_became_peered":"2024-09-12T03:23:20.682033+0000","last_unstale":"2024-09-12T03:23:20.684552+0000","last_undegraded":"2024-09-12T03:23:20.684552+0000","last_fullsized":"2024-09-12T03:23:20.684552+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_clean_scrub_stamp":"2024-09-12T03:23:19.659665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:42:18.081947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.a","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.676951+0000","last_change":"2024-09-12T03:23:20.676951+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.676951+0000","last_undegraded":"2024-09-12T03:23:20.676951+0000","last_fullsized":"2024-09-12T03:23:20.676951+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.b","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-12T03:23:20.690808+0000","last_change":"2024-09-12T03:23:20.688832+0000","last_active":"2024-09-12T03:23:20.690808+0000","last_peered":"2024-09-12T03:23:20.690808+0000","last_clean":"2024-09-12T03:23:20.690808+0000","last_became_active":"2024-09-12T03:23:20.688021+0000","last_became_peered":"2024-09-12T03:23:20.688021+0000","last_unstale":"2024-09-12T03:23:20.690808+0000","last_undegraded":"2024-09-12T03:23:20.690808+0000","last_fullsized":"2024-09-12T03:23:20.690808+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_clean_scrub_stamp":"2024-09-12T03:23:19.659665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:11:32.173284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.b","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.671109+0000","last_change":"2024-09-12T03:23:20.671109+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.671109+0000","last_undegraded":"2024-09-12T03:23:20.671109+0000","last_fullsized":"2024-09-12T03:23:20.671109+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.a","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-12T03:23:20.686788+0000","last_change":"2024-09-12T03:23:20.683771+0000","last_active":"2024-09-12T03:23:20.686788+0000","last_peered":"2024-09-12T03:23:20.686788+0000","last_clean":"2024-09-12T03:23:20.686788+0000","last_became_active":"2024-09-12T03:23:20.683511+0000","last_became_peered":"2024-09-12T03:23:20.683511+0000","last_unstale":"2024-09-12T03:23:20.686788+0000","last_undegraded":"2024-09-12T03:23:20.686788+0000","last_fullsized":"2024-09-12T03:23:20.686788+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_clean_scrub_stamp":"2024-09-12T03:23:19.659665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:28:42.183834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.8","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.675851+0000","last_change":"2024-09-12T03:23:20.675851+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.675851+0000","last_undegraded":"2024-09-12T03:23:20.675851+0000","last_fullsized":"2024-09-12T03:23:20.675851+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.9","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-12T03:23:20.695616+0000","last_change":"2024-09-12T03:23:20.692711+0000","last_active":"2024-09-12T03:23:20.695616+0000","last_peered":"2024-09-12T03:23:20.695616+0000","last_clean":"2024-09-12T03:23:20.695616+0000","last_became_active":"2024-09-12T03:23:20.692455+0000","last_became_peered":"2024-09-12T03:23:20.692455+0000","last_unstale":"2024-09-12T03:23:20.695616+0000","last_undegraded":"2024-09-12T03:23:20.695616+0000","last_fullsized":"2024-09-12T03:23:20.695616+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_clean_scrub_stamp":"2024-09-12T03:23:19.659665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:47:47.363872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.9","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.681456+0000","last_change":"2024-09-12T03:23:20.681456+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.681456+0000","last_undegraded":"2024-09-12T03:23:20.681456+0000","last_fullsized":"2024-09-12T03:23:20.681456+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.8","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-12T03:23:20.684083+0000","last_change":"2024-09-12T03:23:20.679394+0000","last_active":"2024-09-12T03:23:20.684083+0000","last_peered":"2024-09-12T03:23:20.684083+0000","last_clean":"2024-09-12T03:23:20.684083+0000","last_became_active":"2024-09-12T03:23:20.679100+0000","last_became_peered":"2024-09-12T03:23:20.679100+0000","last_unstale":"2024-09-12T03:23:20.684083+0000","last_undegraded":"2024-09-12T03:23:20.684083+0000","last_fullsized":"2024-09-12T03:23:20.684083+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_clean_scrub_stamp":"2024-09-12T03:23:19.659665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:49:36.375371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.6","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.681144+0000","last_change":"2024-09-12T03:23:20.681144+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.681144+0000","last_undegraded":"2024-09-12T03:23:20.681144+0000","last_fullsized":"2024-09-12T03:23:20.681144+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.7","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-12T03:23:20.693562+0000","last_change":"2024-09-12T03:23:20.693222+0000","last_active":"2024-09-12T03:23:20.693562+0000","last_peered":"2024-09-12T03:23:20.693562+0000","last_clean":"2024-09-12T03:23:20.693562+0000","last_became_active":"2024-09-12T03:23:20.692889+0000","last_became_peered":"2024-09-12T03:23:20.692889+0000","last_unstale":"2024-09-12T03:23:20.693562+0000","last_undegraded":"2024-09-12T03:23:20.693562+0000","last_fullsized":"2024-09-12T03:23:20.693562+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_clean_scrub_stamp":"2024-09-12T03:23:19.659665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:08:14.319966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.7","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.676386+0000","last_change":"2024-09-12T03:23:20.676386+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.676386+0000","last_undegraded":"2024-09-12T03:23:20.676386+0000","last_fullsized":"2024-09-12T03:23:20.676386+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.6","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-12T03:23:20.688527+0000","last_change":"2024-09-12T03:23:20.687884+0000","last_active":"2024-09-12T03:23:20.688527+0000","last_peered":"2024-09-12T03:23:20.688527+0000","last_clean":"2024-09-12T03:23:20.688527+0000","last_became_active":"2024-09-12T03:23:20.687621+0000","last_became_peered":"2024-09-12T03:23:20.687621+0000","last_unstale":"2024-09-12T03:23:20.688527+0000","last_undegraded":"2024-09-12T03:23:20.688527+0000","last_fullsized":"2024-09-12T03:23:20.688527+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_clean_scrub_stamp":"2024-09-12T03:23:19.659665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:34:47.119838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.4","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.677605+0000","last_change":"2024-09-12T03:23:20.677605+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.677605+0000","last_undegraded":"2024-09-12T03:23:20.677605+0000","last_fullsized":"2024-09-12T03:23:20.677605+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.5","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-12T03:23:20.688498+0000","last_change":"2024-09-12T03:23:20.686431+0000","last_active":"2024-09-12T03:23:20.688498+0000","last_peered":"2024-09-12T03:23:20.688498+0000","last_clean":"2024-09-12T03:23:20.688498+0000","last_became_active":"2024-09-12T03:23:20.686145+0000","last_became_peered":"2024-09-12T03:23:20.686145+0000","last_unstale":"2024-09-12T03:23:20.688498+0000","last_undegraded":"2024-09-12T03:23:20.688498+0000","last_fullsized":"2024-09-12T03:23:20.688498+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_clean_scrub_stamp":"2024-09-12T03:23:19.659665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:14:07.788062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.5","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.677039+0000","last_change":"2024-09-12T03:23:20.677039+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.677039+0000","last_undegraded":"2024-09-12T03:23:20.677039+0000","last_fullsized":"2024-09-12T03:23:20.677039+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.4","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-12T03:23:20.691435+0000","last_change":"2024-09-12T03:23:20.689734+0000","last_active":"2024-09-12T03:23:20.691435+0000","last_peered":"2024-09-12T03:23:20.691435+0000","last_clean":"2024-09-12T03:23:20.691435+0000","last_became_active":"2024-09-12T03:23:20.689439+0000","last_became_peered":"2024-09-12T03:23:20.689439+0000","last_unstale":"2024-09-12T03:23:20.691435+0000","last_undegraded":"2024-09-12T03:23:20.691435+0000","last_fullsized":"2024-09-12T03:23:20.691435+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_clean_scrub_stamp":"2024-09-12T03:23:19.659665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:47:48.015298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.2","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.675165+0000","last_change":"2024-09-12T03:23:20.675165+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.675165+0000","last_undegraded":"2024-09-12T03:23:20.675165+0000","last_fullsized":"2024-09-12T03:23:20.675165+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.3","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-12T03:23:20.685867+0000","last_change":"2024-09-12T03:23:20.684832+0000","last_active":"2024-09-12T03:23:20.685867+0000","last_peered":"2024-09-12T03:23:20.685867+0000","last_clean":"2024-09-12T03:23:20.685867+0000","last_became_active":"2024-09-12T03:23:20.684531+0000","last_became_peered":"2024-09-12T03:23:20.684531+0000","last_unstale":"2024-09-12T03:23:20.685867+0000","last_undegraded":"2024-09-12T03:23:20.685867+0000","last_fullsized":"2024-09-12T03:23:20.685867+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_clean_scrub_stamp":"2024-09-12T03:23:19.659665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:11:40.963697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.3","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.675175+0000","last_change":"2024-09-12T03:23:20.675175+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.675175+0000","last_undegraded":"2024-09-12T03:23:20.675175+0000","last_fullsized":"2024-09-12T03:23:20.675175+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.2","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-12T03:23:20.685296+0000","last_change":"2024-09-12T03:23:20.681981+0000","last_active":"2024-09-12T03:23:20.685296+0000","last_peered":"2024-09-12T03:23:20.685296+0000","last_clean":"2024-09-12T03:23:20.685296+0000","last_became_active":"2024-09-12T03:23:20.681487+0000","last_became_peered":"2024-09-12T03:23:20.681487+0000","last_unstale":"2024-09-12T03:23:20.685296+0000","last_undegraded":"2024-09-12T03:23:20.685296+0000","last_fullsized":"2024-09-12T03:23:20.685296+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_clean_scrub_stamp":"2024-09-12T03:23:19.659665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:52:54.619581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.0","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.675295+0000","last_change":"2024-09-12T03:23:20.675295+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.675295+0000","last_undegraded":"2024-09-12T03:23:20.675295+0000","last_fullsized":"2024-09-12T03:23:20.675295+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.1","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-12T03:23:20.682875+0000","last_change":"2024-09-12T03:23:20.682237+0000","last_active":"2024-09-12T03:23:20.682875+0000","last_peered":"2024-09-12T03:23:20.682875+0000","last_clean":"2024-09-12T03:23:20.682875+0000","last_became_active":"2024-09-12T03:23:20.681397+0000","last_became_peered":"2024-09-12T03:23:20.681397+0000","last_unstale":"2024-09-12T03:23:20.682875+0000","last_undegraded":"2024-09-12T03:23:20.682875+0000","last_fullsized":"2024-09-12T03:23:20.682875+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_clean_scrub_stamp":"2024-09-12T03:23:19.659665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:16:33.772644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.1","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.680888+0000","last_change":"2024-09-12T03:23:20.680888+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.680888+0000","last_undegraded":"2024-09-12T03:23:20.680888+0000","last_fullsized":"2024-09-12T03:23:20.680888+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.0","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-12T03:23:20.695121+0000","last_change":"2024-09-12T03:23:20.688059+0000","last_active":"2024-09-12T03:23:20.695121+0000","last_peered":"2024-09-12T03:23:20.695121+0000","last_clean":"2024-09-12T03:23:20.695121+0000","last_became_active":"2024-09-12T03:23:20.687774+0000","last_became_peered":"2024-09-12T03:23:20.687774+0000","last_unstale":"2024-09-12T03:23:20.695121+0000","last_undegraded":"2024-09-12T03:23:20.695121+0000","last_fullsized":"2024-09-12T03:23:20.695121+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_clean_scrub_stamp":"2024-09-12T03:23:19.659665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:31:56.155648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.18","version":"0'0","reported_seq":1,"reported_epoch":147,"state":"creating+peering","last_fresh":"2024-09-12T03:23:20.678051+0000","last_change":"2024-09-12T03:23:20.678051+0000","last_active":"2024-09-12T03:23:20.665225+0000","last_peered":"2024-09-12T03:23:20.665225+0000","last_clean":"2024-09-12T03:23:20.665225+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T03:23:20.678051+0000","last_undegraded":"2024-09-12T03:23:20.678051+0000","last_fullsized":"2024-09-12T03:23:20.678051+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.19","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-12T03:23:20.686693+0000","last_change":"2024-09-12T03:23:20.685565+0000","last_active":"2024-09-12T03:23:20.686693+0000","last_peered":"2024-09-12T03:23:20.686693+0000","last_clean":"2024-09-12T03:23:20.686693+0000","last_became_active":"2024-09-12T03:23:20.685230+0000","last_became_peered":"2024-09-12T03:23:20.685230+0000","last_unstale":"2024-09-12T03:23:20.686693+0000","last_undegraded":"2024-09-12T03:23:20.686693+0000","last_fullsized":"2024-09-12T03:23:20.686693+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_clean_scrub_stamp":"2024-09-12T03:23:19.659665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:25:00.890612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"70'128","reported_seq":441,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-12T03:23:20.680128+0000","last_change":"2024-09-12T03:16:21.883446+0000","last_active":"2024-09-12T03:23:20.680128+0000","last_peered":"2024-09-12T03:23:20.680128+0000","last_clean":"2024-09-12T03:23:20.680128+0000","last_became_active":"2024-09-12T03:16:21.882207+0000","last_became_peered":"2024-09-12T03:16:21.882207+0000","last_unstale":"2024-09-12T03:23:20.680128+0000","last_undegraded":"2024-09-12T03:23:20.680128+0000","last_fullsized":"2024-09-12T03:23:20.680128+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:20.316426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:20.316426+0000","last_clean_scrub_stamp":"2024-09-12T03:16:20.316426+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:12:15.424320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":25,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":24,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542230,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37736,"kb_used_data":2388,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334104,"statfs":{"total":96636764160,"available":96598122496,"internally_reserved":0,"allocated":2445312,"data_stored":1407765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542232,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37148,"kb_used_data":1796,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334692,"statfs":{"total":96636764160,"available":96598724608,"internally_reserved":0,"allocated":1839104,"data_stored":811066,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542231,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37156,"kb_used_data":1808,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334684,"statfs":{"total":96636764160,"available":96598716416,"internally_reserved":0,"allocated":1851392,"data_stored":817397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542230,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37160,"kb_used_data":1808,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334680,"statfs":{"total":96636764160,"available":96598712320,"internally_reserved":0,"allocated":1851392,"data_stored":817397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542231,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32996,"kb_used_data":1796,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338844,"statfs":{"total":96636764160,"available":96602976256,"internally_reserved":0,"allocated":1839104,"data_stored":811066,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542231,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32984,"kb_used_data":1796,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338856,"statfs":{"total":96636764160,"available":96602988544,"internally_reserved":0,"allocated":1839104,"data_stored":811066,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542232,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32996,"kb_used_data":1808,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338844,"statfs":{"total":96636764160,"available":96602976256,"internally_reserved":0,"allocated":1851392,"data_stored":817397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542232,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33580,"kb_used_data":2388,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338260,"statfs":{"total":96636764160,"available":96602378240,"internally_reserved":0,"allocated":2445312,"data_stored":1407765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T03:23:30.109 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-12T03:23:30.225 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.224+0000 7fd420217640 1 Processor -- start 2024-09-12T03:23:30.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.225+0000 7fd420217640 1 -- start start 2024-09-12T03:23:30.227 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.225+0000 7fd420217640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fd41815eb90 0x7fd4180d2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:30.227 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.226+0000 7fd420217640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fd4180d1720 0x7fd4180b8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:30.227 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.226+0000 7fd420217640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd4180d3760 con 0x7fd4180d5200 2024-09-12T03:23:30.227 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.226+0000 7fd420217640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7fd4180b7d20 con 0x7fd41815eb90 2024-09-12T03:23:30.227 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.226+0000 7fd420217640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fd4180d42e0 con 0x7fd4180d1720 2024-09-12T03:23:30.227 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.226+0000 7fd41dfb3640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fd41815eb90 0x7fd4180d2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:30.227 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.226+0000 7fd41dfb3640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fd41815eb90 0x7fd4180d2df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34556/0 (socket says 172.21.15.27:34556) 2024-09-12T03:23:30.228 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.226+0000 7fd41dfb3640 1 -- 172.21.15.27:0/2912082003 learned_addr learned my addr 172.21.15.27:0/2912082003 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:23:30.228 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.226+0000 7fd41e7b4640 1 --2- 172.21.15.27:0/2912082003 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fd4180d1720 0x7fd4180b8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:30.228 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.226+0000 7fd41dfb3640 1 -- 172.21.15.27:0/2912082003 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fd4180d1720 msgr2=0x7fd4180b8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:30.228 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.226+0000 7fd41dfb3640 1 --2- 172.21.15.27:0/2912082003 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fd4180d1720 0x7fd4180b8190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:30.228 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.226+0000 7fd41dfb3640 1 -- 172.21.15.27:0/2912082003 >> v1:172.21.15.45:6789/0 conn(0x7fd4180d5200 legacy=0x7fd4180d1010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:30.228 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.226+0000 7fd41dfb3640 1 -- 172.21.15.27:0/2912082003 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4180d3a40 con 0x7fd41815eb90 2024-09-12T03:23:30.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.227+0000 7fd41dfb3640 1 --2- 172.21.15.27:0/2912082003 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd41815eb90 0x7fd4180d2df0 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7fd414009870 tx=0x7fd41402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39a5149023774f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:30.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.227+0000 7fd41cfb1640 1 -- 172.21.15.27:0/2912082003 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd414037070 con 0x7fd41815eb90 2024-09-12T03:23:30.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.227+0000 7fd41cfb1640 1 -- 172.21.15.27:0/2912082003 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd414032440 con 0x7fd41815eb90 2024-09-12T03:23:30.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.227+0000 7fd41cfb1640 1 -- 172.21.15.27:0/2912082003 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd414032810 con 0x7fd41815eb90 2024-09-12T03:23:30.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.227+0000 7fd420217640 1 -- 172.21.15.27:0/2912082003 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd41815eb90 msgr2=0x7fd4180d2df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:30.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.227+0000 7fd420217640 1 --2- 172.21.15.27:0/2912082003 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd41815eb90 0x7fd4180d2df0 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7fd414009870 tx=0x7fd41402a210 comp rx=0 tx=0).stop 2024-09-12T03:23:30.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.228+0000 7fd420217640 1 -- 172.21.15.27:0/2912082003 shutdown_connections 2024-09-12T03:23:30.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.228+0000 7fd420217640 1 --2- 172.21.15.27:0/2912082003 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fd4180d1720 0x7fd4180b8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:30.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.228+0000 7fd420217640 1 --2- 172.21.15.27:0/2912082003 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd41815eb90 0x7fd4180d2df0 unknown :-1 s=CLOSED pgs=1556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:30.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.228+0000 7fd420217640 1 -- 172.21.15.27:0/2912082003 >> 172.21.15.27:0/2912082003 conn(0x7fd41806dce0 msgr2=0x7fd41806e0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:30.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.228+0000 7fd420217640 1 -- 172.21.15.27:0/2912082003 shutdown_connections 2024-09-12T03:23:30.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.228+0000 7fd420217640 1 -- 172.21.15.27:0/2912082003 wait complete. 2024-09-12T03:23:30.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.229+0000 7fd420217640 1 Processor -- start 2024-09-12T03:23:30.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.229+0000 7fd420217640 1 -- start start 2024-09-12T03:23:30.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.230+0000 7fd420217640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fd4180d1720 0x7fd4180cab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:30.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.230+0000 7fd420217640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd4180d5200 0x7fd418158fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:30.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.230+0000 7fd420217640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd41815eb90 0x7fd4181f74c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:30.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.230+0000 7fd420217640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fd4180d3330 con 0x7fd41815eb90 2024-09-12T03:23:30.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.230+0000 7fd420217640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fd4180d4000 con 0x7fd4180d5200 2024-09-12T03:23:30.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.230+0000 7fd420217640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fd4180d34b0 con 0x7fd4180d1720 2024-09-12T03:23:30.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.230+0000 7fd41e7b4640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd41815eb90 0x7fd4181f74c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:30.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.230+0000 7fd41e7b4640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd41815eb90 0x7fd4181f74c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34558/0 (socket says 172.21.15.27:34558) 2024-09-12T03:23:30.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.230+0000 7fd41e7b4640 1 -- 172.21.15.27:0/4012265571 learned_addr learned my addr 172.21.15.27:0/4012265571 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:23:30.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.230+0000 7fd41dfb3640 1 --2- 172.21.15.27:0/4012265571 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fd4180d1720 0x7fd4180cab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:30.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.230+0000 7fd41d7b2640 1 --2- 172.21.15.27:0/4012265571 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd4180d5200 0x7fd418158fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:30.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.230+0000 7fd41e7b4640 1 -- 172.21.15.27:0/4012265571 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fd4180d1720 msgr2=0x7fd4180cab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:30.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.230+0000 7fd41e7b4640 1 --2- 172.21.15.27:0/4012265571 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fd4180d1720 0x7fd4180cab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:30.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.230+0000 7fd41e7b4640 1 -- 172.21.15.27:0/4012265571 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd4180d5200 msgr2=0x7fd418158fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:30.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.230+0000 7fd41e7b4640 1 --2- 172.21.15.27:0/4012265571 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd4180d5200 0x7fd418158fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:30.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.230+0000 7fd41e7b4640 1 -- 172.21.15.27:0/4012265571 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4181594e0 con 0x7fd41815eb90 2024-09-12T03:23:30.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.230+0000 7fd41e7b4640 1 --2- 172.21.15.27:0/4012265571 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd41815eb90 0x7fd4181f74c0 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7fd41000cd20 tx=0x7fd41000a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:30.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.231+0000 7fd40e7fc640 1 -- 172.21.15.27:0/4012265571 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd410003b90 con 0x7fd41815eb90 2024-09-12T03:23:30.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.231+0000 7fd40e7fc640 1 -- 172.21.15.27:0/4012265571 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd410003d30 con 0x7fd41815eb90 2024-09-12T03:23:30.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.231+0000 7fd420217640 1 -- 172.21.15.27:0/4012265571 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd418159780 con 0x7fd41815eb90 2024-09-12T03:23:30.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.231+0000 7fd40e7fc640 1 -- 172.21.15.27:0/4012265571 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd410004240 con 0x7fd41815eb90 2024-09-12T03:23:30.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.231+0000 7fd420217640 1 -- 172.21.15.27:0/4012265571 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd41815a020 con 0x7fd41815eb90 2024-09-12T03:23:30.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.232+0000 7fd420217640 1 -- 172.21.15.27:0/4012265571 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd41815f090 con 0x7fd41815eb90 2024-09-12T03:23:30.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.232+0000 7fd40e7fc640 1 -- 172.21.15.27:0/4012265571 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 24) ==== 91212+0+0 (secure 0 0 0) 0x7fd4100043e0 con 0x7fd41815eb90 2024-09-12T03:23:30.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.233+0000 7fd40e7fc640 1 --2- 172.21.15.27:0/4012265571 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fd3fc06c610 0x7fd3fc06ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:30.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.233+0000 7fd40e7fc640 1 -- 172.21.15.27:0/4012265571 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 5765+0+0 (secure 0 0 0) 0x7fd410089f70 con 0x7fd41815eb90 2024-09-12T03:23:30.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.233+0000 7fd41dfb3640 1 --2- 172.21.15.27:0/4012265571 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fd3fc06c610 0x7fd3fc06ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:30.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.233+0000 7fd41dfb3640 1 --2- 172.21.15.27:0/4012265571 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fd3fc06c610 0x7fd3fc06ead0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fd414034000 tx=0x7fd41402e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:30.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.237+0000 7fd40e7fc640 1 -- 172.21.15.27:0/4012265571 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fd410059d80 con 0x7fd41815eb90 2024-09-12T03:23:30.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.415+0000 7fd420217640 1 -- 172.21.15.27:0/4012265571 --> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd4180b9f30 con 0x7fd3fc06c610 2024-09-12T03:23:30.421 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.421+0000 7fd40e7fc640 1 -- 172.21.15.27:0/4012265571 <== mgr.4113 v2:172.21.15.27:6836/1226920526 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252485 (secure 0 0 0) 0x7fd4180b9f30 con 0x7fd3fc06c610 2024-09-12T03:23:30.424 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:23:30.426 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2024-09-12T03:23:30.427 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.424+0000 7fd420217640 1 -- 172.21.15.27:0/4012265571 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fd3fc06c610 msgr2=0x7fd3fc06ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:30.427 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.424+0000 7fd420217640 1 --2- 172.21.15.27:0/4012265571 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fd3fc06c610 0x7fd3fc06ead0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fd414034000 tx=0x7fd41402e000 comp rx=0 tx=0).stop 2024-09-12T03:23:30.427 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.424+0000 7fd420217640 1 -- 172.21.15.27:0/4012265571 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd41815eb90 msgr2=0x7fd4181f74c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:30.427 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.424+0000 7fd420217640 1 --2- 172.21.15.27:0/4012265571 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd41815eb90 0x7fd4181f74c0 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7fd41000cd20 tx=0x7fd41000a660 comp rx=0 tx=0).stop 2024-09-12T03:23:30.427 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.424+0000 7fd420217640 1 -- 172.21.15.27:0/4012265571 shutdown_connections 2024-09-12T03:23:30.427 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.424+0000 7fd420217640 1 --2- 172.21.15.27:0/4012265571 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fd3fc06c610 0x7fd3fc06ead0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:30.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.424+0000 7fd420217640 1 --2- 172.21.15.27:0/4012265571 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd41815eb90 0x7fd4181f74c0 unknown :-1 s=CLOSED pgs=1557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:30.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.424+0000 7fd420217640 1 --2- 172.21.15.27:0/4012265571 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd4180d5200 0x7fd418158fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:30.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.424+0000 7fd420217640 1 --2- 172.21.15.27:0/4012265571 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fd4180d1720 0x7fd4180cab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:30.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.424+0000 7fd420217640 1 -- 172.21.15.27:0/4012265571 >> 172.21.15.27:0/4012265571 conn(0x7fd41806dce0 msgr2=0x7fd4180b8190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:30.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.424+0000 7fd420217640 1 -- 172.21.15.27:0/4012265571 shutdown_connections 2024-09-12T03:23:30.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.424+0000 7fd420217640 1 -- 172.21.15.27:0/4012265571 wait complete. 2024-09-12T03:23:30.435 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":180,"stamp":"2024-09-12T03:23:29.135480+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":224,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":167,"ondisk_log_size":167,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":286224,"kb_used_data":15880,"kb_used_omap":12,"kb_used_meta":270131,"kb_avail":754688496,"statfs":{"total":773094113280,"available":772801019904,"internally_reserved":0,"allocated":16261120,"data_stored":7830228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":276614704},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001764"},"pg_stats":[{"pgid":"25.3f","version":"0'0","reported_seq":12,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-12T03:23:22.941990+0000","last_change":"2024-09-12T03:23:21.698380+0000","last_active":"2024-09-12T03:23:22.941990+0000","last_peered":"2024-09-12T03:23:22.941990+0000","last_clean":"2024-09-12T03:23:22.941990+0000","last_became_active":"2024-09-12T03:23:21.697839+0000","last_became_peered":"2024-09-12T03:23:21.697839+0000","last_unstale":"2024-09-12T03:23:22.941990+0000","last_undegraded":"2024-09-12T03:23:22.941990+0000","last_fullsized":"2024-09-12T03:23:22.941990+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:14:08.793622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.3e","version":"0'0","reported_seq":12,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-12T03:23:22.942357+0000","last_change":"2024-09-12T03:23:21.711223+0000","last_active":"2024-09-12T03:23:22.942357+0000","last_peered":"2024-09-12T03:23:22.942357+0000","last_clean":"2024-09-12T03:23:22.942357+0000","last_became_active":"2024-09-12T03:23:21.711013+0000","last_became_peered":"2024-09-12T03:23:21.711013+0000","last_unstale":"2024-09-12T03:23:22.942357+0000","last_undegraded":"2024-09-12T03:23:22.942357+0000","last_fullsized":"2024-09-12T03:23:22.942357+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:56:35.251583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.3d","version":"0'0","reported_seq":12,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-12T03:23:22.699977+0000","last_change":"2024-09-12T03:23:21.701585+0000","last_active":"2024-09-12T03:23:22.699977+0000","last_peered":"2024-09-12T03:23:22.699977+0000","last_clean":"2024-09-12T03:23:22.699977+0000","last_became_active":"2024-09-12T03:23:21.701288+0000","last_became_peered":"2024-09-12T03:23:21.701288+0000","last_unstale":"2024-09-12T03:23:22.699977+0000","last_undegraded":"2024-09-12T03:23:22.699977+0000","last_fullsized":"2024-09-12T03:23:22.699977+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:10:55.170491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.3c","version":"0'0","reported_seq":12,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-12T03:23:22.942407+0000","last_change":"2024-09-12T03:23:21.711867+0000","last_active":"2024-09-12T03:23:22.942407+0000","last_peered":"2024-09-12T03:23:22.942407+0000","last_clean":"2024-09-12T03:23:22.942407+0000","last_became_active":"2024-09-12T03:23:21.711511+0000","last_became_peered":"2024-09-12T03:23:21.711511+0000","last_unstale":"2024-09-12T03:23:22.942407+0000","last_undegraded":"2024-09-12T03:23:22.942407+0000","last_fullsized":"2024-09-12T03:23:22.942407+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:47:49.020859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.3b","version":"0'0","reported_seq":12,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-12T03:23:22.692334+0000","last_change":"2024-09-12T03:23:21.704004+0000","last_active":"2024-09-12T03:23:22.692334+0000","last_peered":"2024-09-12T03:23:22.692334+0000","last_clean":"2024-09-12T03:23:22.692334+0000","last_became_active":"2024-09-12T03:23:21.703370+0000","last_became_peered":"2024-09-12T03:23:21.703370+0000","last_unstale":"2024-09-12T03:23:22.692334+0000","last_undegraded":"2024-09-12T03:23:22.692334+0000","last_fullsized":"2024-09-12T03:23:22.692334+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:03:04.310617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.3a","version":"0'0","reported_seq":12,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-12T03:23:22.699840+0000","last_change":"2024-09-12T03:23:21.696928+0000","last_active":"2024-09-12T03:23:22.699840+0000","last_peered":"2024-09-12T03:23:22.699840+0000","last_clean":"2024-09-12T03:23:22.699840+0000","last_became_active":"2024-09-12T03:23:21.696661+0000","last_became_peered":"2024-09-12T03:23:21.696661+0000","last_unstale":"2024-09-12T03:23:22.699840+0000","last_undegraded":"2024-09-12T03:23:22.699840+0000","last_fullsized":"2024-09-12T03:23:22.699840+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:49:26.575282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.39","version":"0'0","reported_seq":12,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-12T03:23:22.692736+0000","last_change":"2024-09-12T03:23:21.695249+0000","last_active":"2024-09-12T03:23:22.692736+0000","last_peered":"2024-09-12T03:23:22.692736+0000","last_clean":"2024-09-12T03:23:22.692736+0000","last_became_active":"2024-09-12T03:23:21.694967+0000","last_became_peered":"2024-09-12T03:23:21.694967+0000","last_unstale":"2024-09-12T03:23:22.692736+0000","last_undegraded":"2024-09-12T03:23:22.692736+0000","last_fullsized":"2024-09-12T03:23:22.692736+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:36:39.662114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.38","version":"0'0","reported_seq":12,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-12T03:23:22.941231+0000","last_change":"2024-09-12T03:23:21.704553+0000","last_active":"2024-09-12T03:23:22.941231+0000","last_peered":"2024-09-12T03:23:22.941231+0000","last_clean":"2024-09-12T03:23:22.941231+0000","last_became_active":"2024-09-12T03:23:21.703955+0000","last_became_peered":"2024-09-12T03:23:21.703955+0000","last_unstale":"2024-09-12T03:23:22.941231+0000","last_undegraded":"2024-09-12T03:23:22.941231+0000","last_fullsized":"2024-09-12T03:23:22.941231+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:15:41.487430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.37","version":"0'0","reported_seq":12,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-12T03:23:22.693118+0000","last_change":"2024-09-12T03:23:21.698163+0000","last_active":"2024-09-12T03:23:22.693118+0000","last_peered":"2024-09-12T03:23:22.693118+0000","last_clean":"2024-09-12T03:23:22.693118+0000","last_became_active":"2024-09-12T03:23:21.695843+0000","last_became_peered":"2024-09-12T03:23:21.695843+0000","last_unstale":"2024-09-12T03:23:22.693118+0000","last_undegraded":"2024-09-12T03:23:22.693118+0000","last_fullsized":"2024-09-12T03:23:22.693118+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:31:57.161209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.36","version":"0'0","reported_seq":12,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-12T03:23:22.698015+0000","last_change":"2024-09-12T03:23:21.699938+0000","last_active":"2024-09-12T03:23:22.698015+0000","last_peered":"2024-09-12T03:23:22.698015+0000","last_clean":"2024-09-12T03:23:22.698015+0000","last_became_active":"2024-09-12T03:23:21.699659+0000","last_became_peered":"2024-09-12T03:23:21.699659+0000","last_unstale":"2024-09-12T03:23:22.698015+0000","last_undegraded":"2024-09-12T03:23:22.698015+0000","last_fullsized":"2024-09-12T03:23:22.698015+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:58:27.725998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.35","version":"0'0","reported_seq":12,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-12T03:23:22.693170+0000","last_change":"2024-09-12T03:23:21.697692+0000","last_active":"2024-09-12T03:23:22.693170+0000","last_peered":"2024-09-12T03:23:22.693170+0000","last_clean":"2024-09-12T03:23:22.693170+0000","last_became_active":"2024-09-12T03:23:21.697352+0000","last_became_peered":"2024-09-12T03:23:21.697352+0000","last_unstale":"2024-09-12T03:23:22.693170+0000","last_undegraded":"2024-09-12T03:23:22.693170+0000","last_fullsized":"2024-09-12T03:23:22.693170+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:07:40.313140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.34","version":"0'0","reported_seq":12,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-12T03:23:22.942672+0000","last_change":"2024-09-12T03:23:21.698026+0000","last_active":"2024-09-12T03:23:22.942672+0000","last_peered":"2024-09-12T03:23:22.942672+0000","last_clean":"2024-09-12T03:23:22.942672+0000","last_became_active":"2024-09-12T03:23:21.697437+0000","last_became_peered":"2024-09-12T03:23:21.697437+0000","last_unstale":"2024-09-12T03:23:22.942672+0000","last_undegraded":"2024-09-12T03:23:22.942672+0000","last_fullsized":"2024-09-12T03:23:22.942672+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:36:39.662114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.33","version":"0'0","reported_seq":12,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-12T03:23:22.941440+0000","last_change":"2024-09-12T03:23:21.699154+0000","last_active":"2024-09-12T03:23:22.941440+0000","last_peered":"2024-09-12T03:23:22.941440+0000","last_clean":"2024-09-12T03:23:22.941440+0000","last_became_active":"2024-09-12T03:23:21.698661+0000","last_became_peered":"2024-09-12T03:23:21.698661+0000","last_unstale":"2024-09-12T03:23:22.941440+0000","last_undegraded":"2024-09-12T03:23:22.941440+0000","last_fullsized":"2024-09-12T03:23:22.941440+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:35:40.942447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.32","version":"0'0","reported_seq":12,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-12T03:23:22.694232+0000","last_change":"2024-09-12T03:23:21.696953+0000","last_active":"2024-09-12T03:23:22.694232+0000","last_peered":"2024-09-12T03:23:22.694232+0000","last_clean":"2024-09-12T03:23:22.694232+0000","last_became_active":"2024-09-12T03:23:21.696665+0000","last_became_peered":"2024-09-12T03:23:21.696665+0000","last_unstale":"2024-09-12T03:23:22.694232+0000","last_undegraded":"2024-09-12T03:23:22.694232+0000","last_fullsized":"2024-09-12T03:23:22.694232+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:44:03.608670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.31","version":"0'0","reported_seq":12,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-12T03:23:22.941956+0000","last_change":"2024-09-12T03:23:21.697955+0000","last_active":"2024-09-12T03:23:22.941956+0000","last_peered":"2024-09-12T03:23:22.941956+0000","last_clean":"2024-09-12T03:23:22.941956+0000","last_became_active":"2024-09-12T03:23:21.697631+0000","last_became_peered":"2024-09-12T03:23:21.697631+0000","last_unstale":"2024-09-12T03:23:22.941956+0000","last_undegraded":"2024-09-12T03:23:22.941956+0000","last_fullsized":"2024-09-12T03:23:22.941956+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:33:24.061239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.30","version":"0'0","reported_seq":12,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-12T03:23:22.941240+0000","last_change":"2024-09-12T03:23:21.704109+0000","last_active":"2024-09-12T03:23:22.941240+0000","last_peered":"2024-09-12T03:23:22.941240+0000","last_clean":"2024-09-12T03:23:22.941240+0000","last_became_active":"2024-09-12T03:23:21.696031+0000","last_became_peered":"2024-09-12T03:23:21.696031+0000","last_unstale":"2024-09-12T03:23:22.941240+0000","last_undegraded":"2024-09-12T03:23:22.941240+0000","last_fullsized":"2024-09-12T03:23:22.941240+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:19:55.068868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.2f","version":"0'0","reported_seq":12,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-12T03:23:22.692603+0000","last_change":"2024-09-12T03:23:21.698992+0000","last_active":"2024-09-12T03:23:22.692603+0000","last_peered":"2024-09-12T03:23:22.692603+0000","last_clean":"2024-09-12T03:23:22.692603+0000","last_became_active":"2024-09-12T03:23:21.698713+0000","last_became_peered":"2024-09-12T03:23:21.698713+0000","last_unstale":"2024-09-12T03:23:22.692603+0000","last_undegraded":"2024-09-12T03:23:22.692603+0000","last_fullsized":"2024-09-12T03:23:22.692603+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:11:12.254035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.2e","version":"0'0","reported_seq":12,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-12T03:23:22.942376+0000","last_change":"2024-09-12T03:23:21.705459+0000","last_active":"2024-09-12T03:23:22.942376+0000","last_peered":"2024-09-12T03:23:22.942376+0000","last_clean":"2024-09-12T03:23:22.942376+0000","last_became_active":"2024-09-12T03:23:21.703753+0000","last_became_peered":"2024-09-12T03:23:21.703753+0000","last_unstale":"2024-09-12T03:23:22.942376+0000","last_undegraded":"2024-09-12T03:23:22.942376+0000","last_fullsized":"2024-09-12T03:23:22.942376+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:16:22.504168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.2d","version":"0'0","reported_seq":12,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-12T03:23:22.698009+0000","last_change":"2024-09-12T03:23:21.707060+0000","last_active":"2024-09-12T03:23:22.698009+0000","last_peered":"2024-09-12T03:23:22.698009+0000","last_clean":"2024-09-12T03:23:22.698009+0000","last_became_active":"2024-09-12T03:23:21.706750+0000","last_became_peered":"2024-09-12T03:23:21.706750+0000","last_unstale":"2024-09-12T03:23:22.698009+0000","last_undegraded":"2024-09-12T03:23:22.698009+0000","last_fullsized":"2024-09-12T03:23:22.698009+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:01:46.813633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.2c","version":"0'0","reported_seq":12,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-12T03:23:22.692384+0000","last_change":"2024-09-12T03:23:21.699009+0000","last_active":"2024-09-12T03:23:22.692384+0000","last_peered":"2024-09-12T03:23:22.692384+0000","last_clean":"2024-09-12T03:23:22.692384+0000","last_became_active":"2024-09-12T03:23:21.698726+0000","last_became_peered":"2024-09-12T03:23:21.698726+0000","last_unstale":"2024-09-12T03:23:22.692384+0000","last_undegraded":"2024-09-12T03:23:22.692384+0000","last_fullsized":"2024-09-12T03:23:22.692384+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:48:43.943284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.2b","version":"0'0","reported_seq":12,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-12T03:23:22.692340+0000","last_change":"2024-09-12T03:23:21.704236+0000","last_active":"2024-09-12T03:23:22.692340+0000","last_peered":"2024-09-12T03:23:22.692340+0000","last_clean":"2024-09-12T03:23:22.692340+0000","last_became_active":"2024-09-12T03:23:21.703739+0000","last_became_peered":"2024-09-12T03:23:21.703739+0000","last_unstale":"2024-09-12T03:23:22.692340+0000","last_undegraded":"2024-09-12T03:23:22.692340+0000","last_fullsized":"2024-09-12T03:23:22.692340+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:03:25.075757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.2a","version":"0'0","reported_seq":12,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-12T03:23:22.697829+0000","last_change":"2024-09-12T03:23:21.711794+0000","last_active":"2024-09-12T03:23:22.697829+0000","last_peered":"2024-09-12T03:23:22.697829+0000","last_clean":"2024-09-12T03:23:22.697829+0000","last_became_active":"2024-09-12T03:23:21.711500+0000","last_became_peered":"2024-09-12T03:23:21.711500+0000","last_unstale":"2024-09-12T03:23:22.697829+0000","last_undegraded":"2024-09-12T03:23:22.697829+0000","last_fullsized":"2024-09-12T03:23:22.697829+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:27:14.884295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.29","version":"0'0","reported_seq":12,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-12T03:23:22.699802+0000","last_change":"2024-09-12T03:23:21.704507+0000","last_active":"2024-09-12T03:23:22.699802+0000","last_peered":"2024-09-12T03:23:22.699802+0000","last_clean":"2024-09-12T03:23:22.699802+0000","last_became_active":"2024-09-12T03:23:21.704127+0000","last_became_peered":"2024-09-12T03:23:21.704127+0000","last_unstale":"2024-09-12T03:23:22.699802+0000","last_undegraded":"2024-09-12T03:23:22.699802+0000","last_fullsized":"2024-09-12T03:23:22.699802+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:34:32.156983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.28","version":"0'0","reported_seq":12,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-12T03:23:22.941401+0000","last_change":"2024-09-12T03:23:21.704272+0000","last_active":"2024-09-12T03:23:22.941401+0000","last_peered":"2024-09-12T03:23:22.941401+0000","last_clean":"2024-09-12T03:23:22.941401+0000","last_became_active":"2024-09-12T03:23:21.703806+0000","last_became_peered":"2024-09-12T03:23:21.703806+0000","last_unstale":"2024-09-12T03:23:22.941401+0000","last_undegraded":"2024-09-12T03:23:22.941401+0000","last_fullsized":"2024-09-12T03:23:22.941401+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:46:26.393683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.27","version":"0'0","reported_seq":12,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-12T03:23:22.942453+0000","last_change":"2024-09-12T03:23:21.709617+0000","last_active":"2024-09-12T03:23:22.942453+0000","last_peered":"2024-09-12T03:23:22.942453+0000","last_clean":"2024-09-12T03:23:22.942453+0000","last_became_active":"2024-09-12T03:23:21.709283+0000","last_became_peered":"2024-09-12T03:23:21.709283+0000","last_unstale":"2024-09-12T03:23:22.942453+0000","last_undegraded":"2024-09-12T03:23:22.942453+0000","last_fullsized":"2024-09-12T03:23:22.942453+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:42:17.035081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.26","version":"0'0","reported_seq":12,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-12T03:23:22.693543+0000","last_change":"2024-09-12T03:23:21.699904+0000","last_active":"2024-09-12T03:23:22.693543+0000","last_peered":"2024-09-12T03:23:22.693543+0000","last_clean":"2024-09-12T03:23:22.693543+0000","last_became_active":"2024-09-12T03:23:21.699349+0000","last_became_peered":"2024-09-12T03:23:21.699349+0000","last_unstale":"2024-09-12T03:23:22.693543+0000","last_undegraded":"2024-09-12T03:23:22.693543+0000","last_fullsized":"2024-09-12T03:23:22.693543+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:25:01.896172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.25","version":"0'0","reported_seq":12,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-12T03:23:22.700012+0000","last_change":"2024-09-12T03:23:21.700961+0000","last_active":"2024-09-12T03:23:22.700012+0000","last_peered":"2024-09-12T03:23:22.700012+0000","last_clean":"2024-09-12T03:23:22.700012+0000","last_became_active":"2024-09-12T03:23:21.700610+0000","last_became_peered":"2024-09-12T03:23:21.700610+0000","last_unstale":"2024-09-12T03:23:22.700012+0000","last_undegraded":"2024-09-12T03:23:22.700012+0000","last_fullsized":"2024-09-12T03:23:22.700012+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:33:37.413327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.24","version":"0'0","reported_seq":12,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-12T03:23:22.692378+0000","last_change":"2024-09-12T03:23:21.707225+0000","last_active":"2024-09-12T03:23:22.692378+0000","last_peered":"2024-09-12T03:23:22.692378+0000","last_clean":"2024-09-12T03:23:22.692378+0000","last_became_active":"2024-09-12T03:23:21.706917+0000","last_became_peered":"2024-09-12T03:23:21.706917+0000","last_unstale":"2024-09-12T03:23:22.692378+0000","last_undegraded":"2024-09-12T03:23:22.692378+0000","last_fullsized":"2024-09-12T03:23:22.692378+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:22:20.718060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.23","version":"0'0","reported_seq":12,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-12T03:23:22.941593+0000","last_change":"2024-09-12T03:23:21.697065+0000","last_active":"2024-09-12T03:23:22.941593+0000","last_peered":"2024-09-12T03:23:22.941593+0000","last_clean":"2024-09-12T03:23:22.941593+0000","last_became_active":"2024-09-12T03:23:21.696807+0000","last_became_peered":"2024-09-12T03:23:21.696807+0000","last_unstale":"2024-09-12T03:23:22.941593+0000","last_undegraded":"2024-09-12T03:23:22.941593+0000","last_fullsized":"2024-09-12T03:23:22.941593+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:51:01.865230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.22","version":"0'0","reported_seq":12,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-12T03:23:22.694041+0000","last_change":"2024-09-12T03:23:21.701634+0000","last_active":"2024-09-12T03:23:22.694041+0000","last_peered":"2024-09-12T03:23:22.694041+0000","last_clean":"2024-09-12T03:23:22.694041+0000","last_became_active":"2024-09-12T03:23:21.701015+0000","last_became_peered":"2024-09-12T03:23:21.701015+0000","last_unstale":"2024-09-12T03:23:22.694041+0000","last_undegraded":"2024-09-12T03:23:22.694041+0000","last_fullsized":"2024-09-12T03:23:22.694041+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:42:46.967753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.21","version":"0'0","reported_seq":12,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-12T03:23:22.699896+0000","last_change":"2024-09-12T03:23:21.696779+0000","last_active":"2024-09-12T03:23:22.699896+0000","last_peered":"2024-09-12T03:23:22.699896+0000","last_clean":"2024-09-12T03:23:22.699896+0000","last_became_active":"2024-09-12T03:23:21.696438+0000","last_became_peered":"2024-09-12T03:23:21.696438+0000","last_unstale":"2024-09-12T03:23:22.699896+0000","last_undegraded":"2024-09-12T03:23:22.699896+0000","last_fullsized":"2024-09-12T03:23:22.699896+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:16:39.527863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.20","version":"0'0","reported_seq":12,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-12T03:23:22.692141+0000","last_change":"2024-09-12T03:23:21.697909+0000","last_active":"2024-09-12T03:23:22.692141+0000","last_peered":"2024-09-12T03:23:22.692141+0000","last_clean":"2024-09-12T03:23:22.692141+0000","last_became_active":"2024-09-12T03:23:21.697644+0000","last_became_peered":"2024-09-12T03:23:21.697644+0000","last_unstale":"2024-09-12T03:23:22.692141+0000","last_undegraded":"2024-09-12T03:23:22.692141+0000","last_fullsized":"2024-09-12T03:23:22.692141+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:28:54.051579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1e","version":"0'0","reported_seq":12,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-12T03:23:22.941619+0000","last_change":"2024-09-12T03:23:21.705256+0000","last_active":"2024-09-12T03:23:22.941619+0000","last_peered":"2024-09-12T03:23:22.941619+0000","last_clean":"2024-09-12T03:23:22.941619+0000","last_became_active":"2024-09-12T03:23:21.704595+0000","last_became_peered":"2024-09-12T03:23:21.704595+0000","last_unstale":"2024-09-12T03:23:22.941619+0000","last_undegraded":"2024-09-12T03:23:22.941619+0000","last_fullsized":"2024-09-12T03:23:22.941619+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:14:08.793622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1f","version":"148'4","reported_seq":18,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-12T03:23:22.692632+0000","last_change":"2024-09-12T03:23:20.682141+0000","last_active":"2024-09-12T03:23:22.692632+0000","last_peered":"2024-09-12T03:23:22.692632+0000","last_clean":"2024-09-12T03:23:22.692632+0000","last_became_active":"2024-09-12T03:23:20.681852+0000","last_became_peered":"2024-09-12T03:23:20.681852+0000","last_unstale":"2024-09-12T03:23:22.692632+0000","last_undegraded":"2024-09-12T03:23:22.692632+0000","last_fullsized":"2024-09-12T03:23:22.692632+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_clean_scrub_stamp":"2024-09-12T03:23:19.659665+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:51:23.012799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1f","version":"0'0","reported_seq":12,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-12T03:23:22.698293+0000","last_change":"2024-09-12T03:23:21.707364+0000","last_active":"2024-09-12T03:23:22.698293+0000","last_peered":"2024-09-12T03:23:22.698293+0000","last_clean":"2024-09-12T03:23:22.698293+0000","last_became_active":"2024-09-12T03:23:21.706755+0000","last_became_peered":"2024-09-12T03:23:21.706755+0000","last_unstale":"2024-09-12T03:23:22.698293+0000","last_undegraded":"2024-09-12T03:23:22.698293+0000","last_fullsized":"2024-09-12T03:23:22.698293+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:21:33.454456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.1e","version":"148'1","reported_seq":15,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-12T03:23:22.693057+0000","last_change":"2024-09-12T03:23:20.686262+0000","last_active":"2024-09-12T03:23:22.693057+0000","last_peered":"2024-09-12T03:23:22.693057+0000","last_clean":"2024-09-12T03:23:22.693057+0000","last_became_active":"2024-09-12T03:23:20.685552+0000","last_became_peered":"2024-09-12T03:23:20.685552+0000","last_unstale":"2024-09-12T03:23:22.693057+0000","last_undegraded":"2024-09-12T03:23:22.693057+0000","last_fullsized":"2024-09-12T03:23:22.693057+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_clean_scrub_stamp":"2024-09-12T03:23:19.659665+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:14:29.012982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.1c","version":"0'0","reported_seq":12,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-12T03:23:22.694100+0000","last_change":"2024-09-12T03:23:21.699712+0000","last_active":"2024-09-12T03:23:22.694100+0000","last_peered":"2024-09-12T03:23:22.694100+0000","last_clean":"2024-09-12T03:23:22.694100+0000","last_became_active":"2024-09-12T03:23:21.699301+0000","last_became_peered":"2024-09-12T03:23:21.699301+0000","last_unstale":"2024-09-12T03:23:22.694100+0000","last_undegraded":"2024-09-12T03:23:22.694100+0000","last_fullsized":"2024-09-12T03:23:22.694100+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:41:29.840978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.1d","version":"148'2","reported_seq":16,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-12T03:23:22.700099+0000","last_change":"2024-09-12T03:23:20.690566+0000","last_active":"2024-09-12T03:23:22.700099+0000","last_peered":"2024-09-12T03:23:22.700099+0000","last_clean":"2024-09-12T03:23:22.700099+0000","last_became_active":"2024-09-12T03:23:20.690296+0000","last_became_peered":"2024-09-12T03:23:20.690296+0000","last_unstale":"2024-09-12T03:23:22.700099+0000","last_undegraded":"2024-09-12T03:23:22.700099+0000","last_fullsized":"2024-09-12T03:23:22.700099+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_clean_scrub_stamp":"2024-09-12T03:23:19.659665+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:55:35.495695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1d","version":"0'0","reported_seq":12,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-12T03:23:22.693889+0000","last_change":"2024-09-12T03:23:21.689147+0000","last_active":"2024-09-12T03:23:22.693889+0000","last_peered":"2024-09-12T03:23:22.693889+0000","last_clean":"2024-09-12T03:23:22.693889+0000","last_became_active":"2024-09-12T03:23:21.688828+0000","last_became_peered":"2024-09-12T03:23:21.688828+0000","last_unstale":"2024-09-12T03:23:22.693889+0000","last_undegraded":"2024-09-12T03:23:22.693889+0000","last_fullsized":"2024-09-12T03:23:22.693889+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:47:48.369432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.1c","version":"148'2","reported_seq":16,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-12T03:23:22.942541+0000","last_change":"2024-09-12T03:23:20.686667+0000","last_active":"2024-09-12T03:23:22.942541+0000","last_peered":"2024-09-12T03:23:22.942541+0000","last_clean":"2024-09-12T03:23:22.942541+0000","last_became_active":"2024-09-12T03:23:20.686355+0000","last_became_peered":"2024-09-12T03:23:20.686355+0000","last_unstale":"2024-09-12T03:23:22.942541+0000","last_undegraded":"2024-09-12T03:23:22.942541+0000","last_fullsized":"2024-09-12T03:23:22.942541+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_clean_scrub_stamp":"2024-09-12T03:23:19.659665+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:41:57.674128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.1a","version":"0'0","reported_seq":12,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-12T03:23:22.697889+0000","last_change":"2024-09-12T03:23:21.711495+0000","last_active":"2024-09-12T03:23:22.697889+0000","last_peered":"2024-09-12T03:23:22.697889+0000","last_clean":"2024-09-12T03:23:22.697889+0000","last_became_active":"2024-09-12T03:23:21.711065+0000","last_became_peered":"2024-09-12T03:23:21.711065+0000","last_unstale":"2024-09-12T03:23:22.697889+0000","last_undegraded":"2024-09-12T03:23:22.697889+0000","last_fullsized":"2024-09-12T03:23:22.697889+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:06:06.903813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.1b","version":"148'1","reported_seq":15,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-12T03:23:22.692463+0000","last_change":"2024-09-12T03:23:20.684507+0000","last_active":"2024-09-12T03:23:22.692463+0000","last_peered":"2024-09-12T03:23:22.692463+0000","last_clean":"2024-09-12T03:23:22.692463+0000","last_became_active":"2024-09-12T03:23:20.684246+0000","last_became_peered":"2024-09-12T03:23:20.684246+0000","last_unstale":"2024-09-12T03:23:22.692463+0000","last_undegraded":"2024-09-12T03:23:22.692463+0000","last_fullsized":"2024-09-12T03:23:22.692463+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_clean_scrub_stamp":"2024-09-12T03:23:19.659665+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:01:12.623150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1b","version":"0'0","reported_seq":12,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-12T03:23:22.941349+0000","last_change":"2024-09-12T03:23:21.704607+0000","last_active":"2024-09-12T03:23:22.941349+0000","last_peered":"2024-09-12T03:23:22.941349+0000","last_clean":"2024-09-12T03:23:22.941349+0000","last_became_active":"2024-09-12T03:23:21.704214+0000","last_became_peered":"2024-09-12T03:23:21.704214+0000","last_unstale":"2024-09-12T03:23:22.941349+0000","last_undegraded":"2024-09-12T03:23:22.941349+0000","last_fullsized":"2024-09-12T03:23:22.941349+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:23:40.686819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1a","version":"148'1","reported_seq":15,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-12T03:23:22.941297+0000","last_change":"2024-09-12T03:23:20.696117+0000","last_active":"2024-09-12T03:23:22.941297+0000","last_peered":"2024-09-12T03:23:22.941297+0000","last_clean":"2024-09-12T03:23:22.941297+0000","last_became_active":"2024-09-12T03:23:20.695795+0000","last_became_peered":"2024-09-12T03:23:20.695795+0000","last_unstale":"2024-09-12T03:23:22.941297+0000","last_undegraded":"2024-09-12T03:23:22.941297+0000","last_fullsized":"2024-09-12T03:23:22.941297+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_clean_scrub_stamp":"2024-09-12T03:23:19.659665+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:46:48.068133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.19","version":"0'0","reported_seq":12,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-12T03:23:22.692947+0000","last_change":"2024-09-12T03:23:21.699270+0000","last_active":"2024-09-12T03:23:22.692947+0000","last_peered":"2024-09-12T03:23:22.692947+0000","last_clean":"2024-09-12T03:23:22.692947+0000","last_became_active":"2024-09-12T03:23:21.698441+0000","last_became_peered":"2024-09-12T03:23:21.698441+0000","last_unstale":"2024-09-12T03:23:22.692947+0000","last_undegraded":"2024-09-12T03:23:22.692947+0000","last_fullsized":"2024-09-12T03:23:22.692947+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:46:49.073693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.18","version":"0'0","reported_seq":14,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-12T03:23:22.941285+0000","last_change":"2024-09-12T03:23:20.689802+0000","last_active":"2024-09-12T03:23:22.941285+0000","last_peered":"2024-09-12T03:23:22.941285+0000","last_clean":"2024-09-12T03:23:22.941285+0000","last_became_active":"2024-09-12T03:23:20.689503+0000","last_became_peered":"2024-09-12T03:23:20.689503+0000","last_unstale":"2024-09-12T03:23:22.941285+0000","last_undegraded":"2024-09-12T03:23:22.941285+0000","last_fullsized":"2024-09-12T03:23:22.941285+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_clean_scrub_stamp":"2024-09-12T03:23:19.659665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:00:06.849881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.16","version":"0'0","reported_seq":12,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-12T03:23:22.700155+0000","last_change":"2024-09-12T03:23:21.707564+0000","last_active":"2024-09-12T03:23:22.700155+0000","last_peered":"2024-09-12T03:23:22.700155+0000","last_clean":"2024-09-12T03:23:22.700155+0000","last_became_active":"2024-09-12T03:23:21.707266+0000","last_became_peered":"2024-09-12T03:23:21.707266+0000","last_unstale":"2024-09-12T03:23:22.700155+0000","last_undegraded":"2024-09-12T03:23:22.700155+0000","last_fullsized":"2024-09-12T03:23:22.700155+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:14:46.027671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.17","version":"0'0","reported_seq":14,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-12T03:23:22.693309+0000","last_change":"2024-09-12T03:23:20.678720+0000","last_active":"2024-09-12T03:23:22.693309+0000","last_peered":"2024-09-12T03:23:22.693309+0000","last_clean":"2024-09-12T03:23:22.693309+0000","last_became_active":"2024-09-12T03:23:20.678415+0000","last_became_peered":"2024-09-12T03:23:20.678415+0000","last_unstale":"2024-09-12T03:23:22.693309+0000","last_undegraded":"2024-09-12T03:23:22.693309+0000","last_fullsized":"2024-09-12T03:23:22.693309+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_clean_scrub_stamp":"2024-09-12T03:23:19.659665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:34:57.458948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.17","version":"0'0","reported_seq":12,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-12T03:23:22.941541+0000","last_change":"2024-09-12T03:23:21.704958+0000","last_active":"2024-09-12T03:23:22.941541+0000","last_peered":"2024-09-12T03:23:22.941541+0000","last_clean":"2024-09-12T03:23:22.941541+0000","last_became_active":"2024-09-12T03:23:21.704675+0000","last_became_peered":"2024-09-12T03:23:21.704675+0000","last_unstale":"2024-09-12T03:23:22.941541+0000","last_undegraded":"2024-09-12T03:23:22.941541+0000","last_fullsized":"2024-09-12T03:23:22.941541+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:23:10.326116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.16","version":"0'0","reported_seq":14,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-12T03:23:22.692453+0000","last_change":"2024-09-12T03:23:20.683066+0000","last_active":"2024-09-12T03:23:22.692453+0000","last_peered":"2024-09-12T03:23:22.692453+0000","last_clean":"2024-09-12T03:23:22.692453+0000","last_became_active":"2024-09-12T03:23:20.682759+0000","last_became_peered":"2024-09-12T03:23:20.682759+0000","last_unstale":"2024-09-12T03:23:22.692453+0000","last_undegraded":"2024-09-12T03:23:22.692453+0000","last_fullsized":"2024-09-12T03:23:22.692453+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_clean_scrub_stamp":"2024-09-12T03:23:19.659665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:28:47.461404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.14","version":"0'0","reported_seq":12,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-12T03:23:22.694315+0000","last_change":"2024-09-12T03:23:21.692482+0000","last_active":"2024-09-12T03:23:22.694315+0000","last_peered":"2024-09-12T03:23:22.694315+0000","last_clean":"2024-09-12T03:23:22.694315+0000","last_became_active":"2024-09-12T03:23:21.692162+0000","last_became_peered":"2024-09-12T03:23:21.692162+0000","last_unstale":"2024-09-12T03:23:22.694315+0000","last_undegraded":"2024-09-12T03:23:22.694315+0000","last_fullsized":"2024-09-12T03:23:22.694315+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:27:54.775482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.15","version":"148'3","reported_seq":17,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-12T03:23:22.692343+0000","last_change":"2024-09-12T03:23:20.691649+0000","last_active":"2024-09-12T03:23:22.692343+0000","last_peered":"2024-09-12T03:23:22.692343+0000","last_clean":"2024-09-12T03:23:22.692343+0000","last_became_active":"2024-09-12T03:23:20.691385+0000","last_became_peered":"2024-09-12T03:23:20.691385+0000","last_unstale":"2024-09-12T03:23:22.692343+0000","last_undegraded":"2024-09-12T03:23:22.692343+0000","last_fullsized":"2024-09-12T03:23:22.692343+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_clean_scrub_stamp":"2024-09-12T03:23:19.659665+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:40:27.705549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.15","version":"0'0","reported_seq":12,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-12T03:23:22.698092+0000","last_change":"2024-09-12T03:23:21.710608+0000","last_active":"2024-09-12T03:23:22.698092+0000","last_peered":"2024-09-12T03:23:22.698092+0000","last_clean":"2024-09-12T03:23:22.698092+0000","last_became_active":"2024-09-12T03:23:21.710313+0000","last_became_peered":"2024-09-12T03:23:21.710313+0000","last_unstale":"2024-09-12T03:23:22.698092+0000","last_undegraded":"2024-09-12T03:23:22.698092+0000","last_fullsized":"2024-09-12T03:23:22.698092+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:47:10.593600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.14","version":"148'6","reported_seq":20,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-12T03:23:22.699943+0000","last_change":"2024-09-12T03:23:20.683263+0000","last_active":"2024-09-12T03:23:22.699943+0000","last_peered":"2024-09-12T03:23:22.699943+0000","last_clean":"2024-09-12T03:23:22.699943+0000","last_became_active":"2024-09-12T03:23:20.682986+0000","last_became_peered":"2024-09-12T03:23:20.682986+0000","last_unstale":"2024-09-12T03:23:22.699943+0000","last_undegraded":"2024-09-12T03:23:22.699943+0000","last_fullsized":"2024-09-12T03:23:22.699943+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_clean_scrub_stamp":"2024-09-12T03:23:19.659665+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:30:37.319430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.12","version":"0'0","reported_seq":12,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-12T03:23:22.699937+0000","last_change":"2024-09-12T03:23:21.705266+0000","last_active":"2024-09-12T03:23:22.699937+0000","last_peered":"2024-09-12T03:23:22.699937+0000","last_clean":"2024-09-12T03:23:22.699937+0000","last_became_active":"2024-09-12T03:23:21.704938+0000","last_became_peered":"2024-09-12T03:23:21.704938+0000","last_unstale":"2024-09-12T03:23:22.699937+0000","last_undegraded":"2024-09-12T03:23:22.699937+0000","last_fullsized":"2024-09-12T03:23:22.699937+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:25:41.510281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.13","version":"148'2","reported_seq":16,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-12T03:23:22.941824+0000","last_change":"2024-09-12T03:23:20.689845+0000","last_active":"2024-09-12T03:23:22.941824+0000","last_peered":"2024-09-12T03:23:22.941824+0000","last_clean":"2024-09-12T03:23:22.941824+0000","last_became_active":"2024-09-12T03:23:20.689485+0000","last_became_peered":"2024-09-12T03:23:20.689485+0000","last_unstale":"2024-09-12T03:23:22.941824+0000","last_undegraded":"2024-09-12T03:23:22.941824+0000","last_fullsized":"2024-09-12T03:23:22.941824+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_clean_scrub_stamp":"2024-09-12T03:23:19.659665+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:57:51.589003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.13","version":"0'0","reported_seq":12,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-12T03:23:22.693695+0000","last_change":"2024-09-12T03:23:21.696933+0000","last_active":"2024-09-12T03:23:22.693695+0000","last_peered":"2024-09-12T03:23:22.693695+0000","last_clean":"2024-09-12T03:23:22.693695+0000","last_became_active":"2024-09-12T03:23:21.696390+0000","last_became_peered":"2024-09-12T03:23:21.696390+0000","last_unstale":"2024-09-12T03:23:22.693695+0000","last_undegraded":"2024-09-12T03:23:22.693695+0000","last_fullsized":"2024-09-12T03:23:22.693695+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:34:40.662483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.12","version":"0'0","reported_seq":14,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-12T03:23:22.698081+0000","last_change":"2024-09-12T03:23:20.689243+0000","last_active":"2024-09-12T03:23:22.698081+0000","last_peered":"2024-09-12T03:23:22.698081+0000","last_clean":"2024-09-12T03:23:22.698081+0000","last_became_active":"2024-09-12T03:23:20.688661+0000","last_became_peered":"2024-09-12T03:23:20.688661+0000","last_unstale":"2024-09-12T03:23:22.698081+0000","last_undegraded":"2024-09-12T03:23:22.698081+0000","last_fullsized":"2024-09-12T03:23:22.698081+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_clean_scrub_stamp":"2024-09-12T03:23:19.659665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:44:02.698631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.10","version":"0'0","reported_seq":12,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-12T03:23:22.692826+0000","last_change":"2024-09-12T03:23:21.697065+0000","last_active":"2024-09-12T03:23:22.692826+0000","last_peered":"2024-09-12T03:23:22.692826+0000","last_clean":"2024-09-12T03:23:22.692826+0000","last_became_active":"2024-09-12T03:23:21.696761+0000","last_became_peered":"2024-09-12T03:23:21.696761+0000","last_unstale":"2024-09-12T03:23:22.692826+0000","last_undegraded":"2024-09-12T03:23:22.692826+0000","last_fullsized":"2024-09-12T03:23:22.692826+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:17:58.823318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.11","version":"0'0","reported_seq":14,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-12T03:23:22.941137+0000","last_change":"2024-09-12T03:23:20.680790+0000","last_active":"2024-09-12T03:23:22.941137+0000","last_peered":"2024-09-12T03:23:22.941137+0000","last_clean":"2024-09-12T03:23:22.941137+0000","last_became_active":"2024-09-12T03:23:20.680508+0000","last_became_peered":"2024-09-12T03:23:20.680508+0000","last_unstale":"2024-09-12T03:23:22.941137+0000","last_undegraded":"2024-09-12T03:23:22.941137+0000","last_fullsized":"2024-09-12T03:23:22.941137+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_clean_scrub_stamp":"2024-09-12T03:23:19.659665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:07:39.307580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.11","version":"0'0","reported_seq":12,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-12T03:23:22.697822+0000","last_change":"2024-09-12T03:23:21.700466+0000","last_active":"2024-09-12T03:23:22.697822+0000","last_peered":"2024-09-12T03:23:22.697822+0000","last_clean":"2024-09-12T03:23:22.697822+0000","last_became_active":"2024-09-12T03:23:21.700115+0000","last_became_peered":"2024-09-12T03:23:21.700115+0000","last_unstale":"2024-09-12T03:23:22.697822+0000","last_undegraded":"2024-09-12T03:23:22.697822+0000","last_fullsized":"2024-09-12T03:23:22.697822+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:38:25.680916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.10","version":"148'8","reported_seq":22,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-12T03:23:22.941383+0000","last_change":"2024-09-12T03:23:20.689843+0000","last_active":"2024-09-12T03:23:22.941383+0000","last_peered":"2024-09-12T03:23:22.941383+0000","last_clean":"2024-09-12T03:23:22.941383+0000","last_became_active":"2024-09-12T03:23:20.689517+0000","last_became_peered":"2024-09-12T03:23:20.689517+0000","last_unstale":"2024-09-12T03:23:22.941383+0000","last_undegraded":"2024-09-12T03:23:22.941383+0000","last_fullsized":"2024-09-12T03:23:22.941383+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_clean_scrub_stamp":"2024-09-12T03:23:19.659665+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:35:30.168705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.e","version":"0'0","reported_seq":12,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-12T03:23:22.698136+0000","last_change":"2024-09-12T03:23:21.702241+0000","last_active":"2024-09-12T03:23:22.698136+0000","last_peered":"2024-09-12T03:23:22.698136+0000","last_clean":"2024-09-12T03:23:22.698136+0000","last_became_active":"2024-09-12T03:23:21.701933+0000","last_became_peered":"2024-09-12T03:23:21.701933+0000","last_unstale":"2024-09-12T03:23:22.698136+0000","last_undegraded":"2024-09-12T03:23:22.698136+0000","last_fullsized":"2024-09-12T03:23:22.698136+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:02:06.835227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.f","version":"148'1","reported_seq":15,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-12T03:23:22.692655+0000","last_change":"2024-09-12T03:23:20.689669+0000","last_active":"2024-09-12T03:23:22.692655+0000","last_peered":"2024-09-12T03:23:22.692655+0000","last_clean":"2024-09-12T03:23:22.692655+0000","last_became_active":"2024-09-12T03:23:20.689291+0000","last_became_peered":"2024-09-12T03:23:20.689291+0000","last_unstale":"2024-09-12T03:23:22.692655+0000","last_undegraded":"2024-09-12T03:23:22.692655+0000","last_fullsized":"2024-09-12T03:23:22.692655+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_clean_scrub_stamp":"2024-09-12T03:23:19.659665+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:27:27.309094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.f","version":"0'0","reported_seq":12,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-12T03:23:22.693276+0000","last_change":"2024-09-12T03:23:21.698799+0000","last_active":"2024-09-12T03:23:22.693276+0000","last_peered":"2024-09-12T03:23:22.693276+0000","last_clean":"2024-09-12T03:23:22.693276+0000","last_became_active":"2024-09-12T03:23:21.698487+0000","last_became_peered":"2024-09-12T03:23:21.698487+0000","last_unstale":"2024-09-12T03:23:22.693276+0000","last_undegraded":"2024-09-12T03:23:22.693276+0000","last_fullsized":"2024-09-12T03:23:22.693276+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:13:41.618974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.e","version":"148'2","reported_seq":16,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-12T03:23:22.941651+0000","last_change":"2024-09-12T03:23:20.691112+0000","last_active":"2024-09-12T03:23:22.941651+0000","last_peered":"2024-09-12T03:23:22.941651+0000","last_clean":"2024-09-12T03:23:22.941651+0000","last_became_active":"2024-09-12T03:23:20.690862+0000","last_became_peered":"2024-09-12T03:23:20.690862+0000","last_unstale":"2024-09-12T03:23:22.941651+0000","last_undegraded":"2024-09-12T03:23:22.941651+0000","last_fullsized":"2024-09-12T03:23:22.941651+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_clean_scrub_stamp":"2024-09-12T03:23:19.659665+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:34:39.656922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.c","version":"0'0","reported_seq":12,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-12T03:23:22.693074+0000","last_change":"2024-09-12T03:23:21.701072+0000","last_active":"2024-09-12T03:23:22.693074+0000","last_peered":"2024-09-12T03:23:22.693074+0000","last_clean":"2024-09-12T03:23:22.693074+0000","last_became_active":"2024-09-12T03:23:21.700749+0000","last_became_peered":"2024-09-12T03:23:21.700749+0000","last_unstale":"2024-09-12T03:23:22.693074+0000","last_undegraded":"2024-09-12T03:23:22.693074+0000","last_fullsized":"2024-09-12T03:23:22.693074+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:11:33.178844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.d","version":"148'1","reported_seq":15,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-12T03:23:22.942040+0000","last_change":"2024-09-12T03:23:20.688103+0000","last_active":"2024-09-12T03:23:22.942040+0000","last_peered":"2024-09-12T03:23:22.942040+0000","last_clean":"2024-09-12T03:23:22.942040+0000","last_became_active":"2024-09-12T03:23:20.687797+0000","last_became_peered":"2024-09-12T03:23:20.687797+0000","last_unstale":"2024-09-12T03:23:22.942040+0000","last_undegraded":"2024-09-12T03:23:22.942040+0000","last_fullsized":"2024-09-12T03:23:22.942040+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_clean_scrub_stamp":"2024-09-12T03:23:19.659665+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:19:54.063308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.d","version":"0'0","reported_seq":12,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-12T03:23:22.942179+0000","last_change":"2024-09-12T03:23:21.711940+0000","last_active":"2024-09-12T03:23:22.942179+0000","last_peered":"2024-09-12T03:23:22.942179+0000","last_clean":"2024-09-12T03:23:22.942179+0000","last_became_active":"2024-09-12T03:23:21.711576+0000","last_became_peered":"2024-09-12T03:23:21.711576+0000","last_unstale":"2024-09-12T03:23:22.942179+0000","last_undegraded":"2024-09-12T03:23:22.942179+0000","last_fullsized":"2024-09-12T03:23:22.942179+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:02:33.389669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.c","version":"0'0","reported_seq":14,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-12T03:23:22.694246+0000","last_change":"2024-09-12T03:23:20.682322+0000","last_active":"2024-09-12T03:23:22.694246+0000","last_peered":"2024-09-12T03:23:22.694246+0000","last_clean":"2024-09-12T03:23:22.694246+0000","last_became_active":"2024-09-12T03:23:20.682033+0000","last_became_peered":"2024-09-12T03:23:20.682033+0000","last_unstale":"2024-09-12T03:23:22.694246+0000","last_undegraded":"2024-09-12T03:23:22.694246+0000","last_fullsized":"2024-09-12T03:23:22.694246+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_clean_scrub_stamp":"2024-09-12T03:23:19.659665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:34:17.347887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.a","version":"0'0","reported_seq":12,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-12T03:23:22.697985+0000","last_change":"2024-09-12T03:23:21.702903+0000","last_active":"2024-09-12T03:23:22.697985+0000","last_peered":"2024-09-12T03:23:22.697985+0000","last_clean":"2024-09-12T03:23:22.697985+0000","last_became_active":"2024-09-12T03:23:21.702620+0000","last_became_peered":"2024-09-12T03:23:21.702620+0000","last_unstale":"2024-09-12T03:23:22.697985+0000","last_undegraded":"2024-09-12T03:23:22.697985+0000","last_fullsized":"2024-09-12T03:23:22.697985+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:50:36.189977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.b","version":"148'2","reported_seq":16,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-12T03:23:22.699792+0000","last_change":"2024-09-12T03:23:20.688832+0000","last_active":"2024-09-12T03:23:22.699792+0000","last_peered":"2024-09-12T03:23:22.699792+0000","last_clean":"2024-09-12T03:23:22.699792+0000","last_became_active":"2024-09-12T03:23:20.688021+0000","last_became_peered":"2024-09-12T03:23:20.688021+0000","last_unstale":"2024-09-12T03:23:22.699792+0000","last_undegraded":"2024-09-12T03:23:22.699792+0000","last_fullsized":"2024-09-12T03:23:22.699792+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_clean_scrub_stamp":"2024-09-12T03:23:19.659665+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:12:13.107605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.b","version":"0'0","reported_seq":12,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-12T03:23:22.693774+0000","last_change":"2024-09-12T03:23:21.707486+0000","last_active":"2024-09-12T03:23:22.693774+0000","last_peered":"2024-09-12T03:23:22.693774+0000","last_clean":"2024-09-12T03:23:22.693774+0000","last_became_active":"2024-09-12T03:23:21.707168+0000","last_became_peered":"2024-09-12T03:23:21.707168+0000","last_unstale":"2024-09-12T03:23:22.693774+0000","last_undegraded":"2024-09-12T03:23:22.693774+0000","last_fullsized":"2024-09-12T03:23:22.693774+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:30:00.935813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.a","version":"0'0","reported_seq":14,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-12T03:23:22.692229+0000","last_change":"2024-09-12T03:23:20.683771+0000","last_active":"2024-09-12T03:23:22.692229+0000","last_peered":"2024-09-12T03:23:22.692229+0000","last_clean":"2024-09-12T03:23:22.692229+0000","last_became_active":"2024-09-12T03:23:20.683511+0000","last_became_peered":"2024-09-12T03:23:20.683511+0000","last_unstale":"2024-09-12T03:23:22.692229+0000","last_undegraded":"2024-09-12T03:23:22.692229+0000","last_fullsized":"2024-09-12T03:23:22.692229+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_clean_scrub_stamp":"2024-09-12T03:23:19.659665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:02:26.384322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.8","version":"0'0","reported_seq":12,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-12T03:23:22.941362+0000","last_change":"2024-09-12T03:23:21.698113+0000","last_active":"2024-09-12T03:23:22.941362+0000","last_peered":"2024-09-12T03:23:22.941362+0000","last_clean":"2024-09-12T03:23:22.941362+0000","last_became_active":"2024-09-12T03:23:21.697616+0000","last_became_peered":"2024-09-12T03:23:21.697616+0000","last_unstale":"2024-09-12T03:23:22.941362+0000","last_undegraded":"2024-09-12T03:23:22.941362+0000","last_fullsized":"2024-09-12T03:23:22.941362+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:34:18.353447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.9","version":"0'0","reported_seq":14,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-12T03:23:22.699910+0000","last_change":"2024-09-12T03:23:20.692711+0000","last_active":"2024-09-12T03:23:22.699910+0000","last_peered":"2024-09-12T03:23:22.699910+0000","last_clean":"2024-09-12T03:23:22.699910+0000","last_became_active":"2024-09-12T03:23:20.692455+0000","last_became_peered":"2024-09-12T03:23:20.692455+0000","last_unstale":"2024-09-12T03:23:22.699910+0000","last_undegraded":"2024-09-12T03:23:22.699910+0000","last_fullsized":"2024-09-12T03:23:22.699910+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_clean_scrub_stamp":"2024-09-12T03:23:19.659665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:23:42.496572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.9","version":"0'0","reported_seq":12,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-12T03:23:22.692127+0000","last_change":"2024-09-12T03:23:21.695600+0000","last_active":"2024-09-12T03:23:22.692127+0000","last_peered":"2024-09-12T03:23:22.692127+0000","last_clean":"2024-09-12T03:23:22.692127+0000","last_became_active":"2024-09-12T03:23:21.695228+0000","last_became_peered":"2024-09-12T03:23:21.695228+0000","last_unstale":"2024-09-12T03:23:22.692127+0000","last_undegraded":"2024-09-12T03:23:22.692127+0000","last_fullsized":"2024-09-12T03:23:22.692127+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:32:59.507236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.8","version":"0'0","reported_seq":14,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-12T03:23:22.692928+0000","last_change":"2024-09-12T03:23:20.679394+0000","last_active":"2024-09-12T03:23:22.692928+0000","last_peered":"2024-09-12T03:23:22.692928+0000","last_clean":"2024-09-12T03:23:22.692928+0000","last_became_active":"2024-09-12T03:23:20.679100+0000","last_became_peered":"2024-09-12T03:23:20.679100+0000","last_unstale":"2024-09-12T03:23:22.692928+0000","last_undegraded":"2024-09-12T03:23:22.692928+0000","last_fullsized":"2024-09-12T03:23:22.692928+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_clean_scrub_stamp":"2024-09-12T03:23:19.659665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:40:17.545149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.6","version":"0'0","reported_seq":12,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-12T03:23:22.692485+0000","last_change":"2024-09-12T03:23:21.695817+0000","last_active":"2024-09-12T03:23:22.692485+0000","last_peered":"2024-09-12T03:23:22.692485+0000","last_clean":"2024-09-12T03:23:22.692485+0000","last_became_active":"2024-09-12T03:23:21.695514+0000","last_became_peered":"2024-09-12T03:23:21.695514+0000","last_unstale":"2024-09-12T03:23:22.692485+0000","last_undegraded":"2024-09-12T03:23:22.692485+0000","last_fullsized":"2024-09-12T03:23:22.692485+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:11:42.899304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.7","version":"148'2","reported_seq":16,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-12T03:23:22.941910+0000","last_change":"2024-09-12T03:23:20.693222+0000","last_active":"2024-09-12T03:23:22.941910+0000","last_peered":"2024-09-12T03:23:22.941910+0000","last_clean":"2024-09-12T03:23:22.941910+0000","last_became_active":"2024-09-12T03:23:20.692889+0000","last_became_peered":"2024-09-12T03:23:20.692889+0000","last_unstale":"2024-09-12T03:23:22.941910+0000","last_undegraded":"2024-09-12T03:23:22.941910+0000","last_fullsized":"2024-09-12T03:23:22.941910+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_clean_scrub_stamp":"2024-09-12T03:23:19.659665+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:15:40.481869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.7","version":"0'0","reported_seq":12,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-12T03:23:22.698325+0000","last_change":"2024-09-12T03:23:21.698212+0000","last_active":"2024-09-12T03:23:22.698325+0000","last_peered":"2024-09-12T03:23:22.698325+0000","last_clean":"2024-09-12T03:23:22.698325+0000","last_became_active":"2024-09-12T03:23:21.697813+0000","last_became_peered":"2024-09-12T03:23:21.697813+0000","last_unstale":"2024-09-12T03:23:22.698325+0000","last_undegraded":"2024-09-12T03:23:22.698325+0000","last_fullsized":"2024-09-12T03:23:22.698325+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:06:01.855351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.6","version":"149'1","reported_seq":15,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-12T03:23:22.698215+0000","last_change":"2024-09-12T03:23:20.687884+0000","last_active":"2024-09-12T03:23:22.698215+0000","last_peered":"2024-09-12T03:23:22.698215+0000","last_clean":"2024-09-12T03:23:22.698215+0000","last_became_active":"2024-09-12T03:23:20.687621+0000","last_became_peered":"2024-09-12T03:23:20.687621+0000","last_unstale":"2024-09-12T03:23:22.698215+0000","last_undegraded":"2024-09-12T03:23:22.698215+0000","last_fullsized":"2024-09-12T03:23:22.698215+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_clean_scrub_stamp":"2024-09-12T03:23:19.659665+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:36:31.461296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.4","version":"0'0","reported_seq":12,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-12T03:23:22.941490+0000","last_change":"2024-09-12T03:23:21.695725+0000","last_active":"2024-09-12T03:23:22.941490+0000","last_peered":"2024-09-12T03:23:22.941490+0000","last_clean":"2024-09-12T03:23:22.941490+0000","last_became_active":"2024-09-12T03:23:21.695437+0000","last_became_peered":"2024-09-12T03:23:21.695437+0000","last_unstale":"2024-09-12T03:23:22.941490+0000","last_undegraded":"2024-09-12T03:23:22.941490+0000","last_fullsized":"2024-09-12T03:23:22.941490+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:42:11.926957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.5","version":"0'0","reported_seq":14,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-12T03:23:22.698106+0000","last_change":"2024-09-12T03:23:20.686431+0000","last_active":"2024-09-12T03:23:22.698106+0000","last_peered":"2024-09-12T03:23:22.698106+0000","last_clean":"2024-09-12T03:23:22.698106+0000","last_became_active":"2024-09-12T03:23:20.686145+0000","last_became_peered":"2024-09-12T03:23:20.686145+0000","last_unstale":"2024-09-12T03:23:22.698106+0000","last_undegraded":"2024-09-12T03:23:22.698106+0000","last_fullsized":"2024-09-12T03:23:22.698106+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_clean_scrub_stamp":"2024-09-12T03:23:19.659665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:04:21.002292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.5","version":"0'0","reported_seq":12,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-12T03:23:22.941547+0000","last_change":"2024-09-12T03:23:21.695877+0000","last_active":"2024-09-12T03:23:22.941547+0000","last_peered":"2024-09-12T03:23:22.941547+0000","last_clean":"2024-09-12T03:23:22.941547+0000","last_became_active":"2024-09-12T03:23:21.695592+0000","last_became_peered":"2024-09-12T03:23:21.695592+0000","last_unstale":"2024-09-12T03:23:22.941547+0000","last_undegraded":"2024-09-12T03:23:22.941547+0000","last_fullsized":"2024-09-12T03:23:22.941547+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:05:24.815946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.4","version":"0'0","reported_seq":14,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-12T03:23:22.941448+0000","last_change":"2024-09-12T03:23:20.689734+0000","last_active":"2024-09-12T03:23:22.941448+0000","last_peered":"2024-09-12T03:23:22.941448+0000","last_clean":"2024-09-12T03:23:22.941448+0000","last_became_active":"2024-09-12T03:23:20.689439+0000","last_became_peered":"2024-09-12T03:23:20.689439+0000","last_unstale":"2024-09-12T03:23:22.941448+0000","last_undegraded":"2024-09-12T03:23:22.941448+0000","last_fullsized":"2024-09-12T03:23:22.941448+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_clean_scrub_stamp":"2024-09-12T03:23:19.659665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:25:00.890612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.2","version":"0'0","reported_seq":12,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-12T03:23:22.941996+0000","last_change":"2024-09-12T03:23:21.714219+0000","last_active":"2024-09-12T03:23:22.941996+0000","last_peered":"2024-09-12T03:23:22.941996+0000","last_clean":"2024-09-12T03:23:22.941996+0000","last_became_active":"2024-09-12T03:23:21.713906+0000","last_became_peered":"2024-09-12T03:23:21.713906+0000","last_unstale":"2024-09-12T03:23:22.941996+0000","last_undegraded":"2024-09-12T03:23:22.941996+0000","last_fullsized":"2024-09-12T03:23:22.941996+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:34:48.125399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.3","version":"0'0","reported_seq":14,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-12T03:23:22.694024+0000","last_change":"2024-09-12T03:23:20.684832+0000","last_active":"2024-09-12T03:23:22.694024+0000","last_peered":"2024-09-12T03:23:22.694024+0000","last_clean":"2024-09-12T03:23:22.694024+0000","last_became_active":"2024-09-12T03:23:20.684531+0000","last_became_peered":"2024-09-12T03:23:20.684531+0000","last_unstale":"2024-09-12T03:23:22.694024+0000","last_undegraded":"2024-09-12T03:23:22.694024+0000","last_fullsized":"2024-09-12T03:23:22.694024+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_clean_scrub_stamp":"2024-09-12T03:23:19.659665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:44:48.254874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.3","version":"0'0","reported_seq":12,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-12T03:23:22.941871+0000","last_change":"2024-09-12T03:23:21.703676+0000","last_active":"2024-09-12T03:23:22.941871+0000","last_peered":"2024-09-12T03:23:22.941871+0000","last_clean":"2024-09-12T03:23:22.941871+0000","last_became_active":"2024-09-12T03:23:21.703310+0000","last_became_peered":"2024-09-12T03:23:21.703310+0000","last_unstale":"2024-09-12T03:23:22.941871+0000","last_undegraded":"2024-09-12T03:23:22.941871+0000","last_fullsized":"2024-09-12T03:23:22.941871+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:05:24.815946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.2","version":"0'0","reported_seq":14,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-12T03:23:22.694146+0000","last_change":"2024-09-12T03:23:20.681981+0000","last_active":"2024-09-12T03:23:22.694146+0000","last_peered":"2024-09-12T03:23:22.694146+0000","last_clean":"2024-09-12T03:23:22.694146+0000","last_became_active":"2024-09-12T03:23:20.681487+0000","last_became_peered":"2024-09-12T03:23:20.681487+0000","last_unstale":"2024-09-12T03:23:22.694146+0000","last_undegraded":"2024-09-12T03:23:22.694146+0000","last_fullsized":"2024-09-12T03:23:22.694146+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_clean_scrub_stamp":"2024-09-12T03:23:19.659665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:54:56.251068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.0","version":"0'0","reported_seq":12,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-12T03:23:22.941463+0000","last_change":"2024-09-12T03:23:21.694252+0000","last_active":"2024-09-12T03:23:22.941463+0000","last_peered":"2024-09-12T03:23:22.941463+0000","last_clean":"2024-09-12T03:23:22.941463+0000","last_became_active":"2024-09-12T03:23:21.693506+0000","last_became_peered":"2024-09-12T03:23:21.693506+0000","last_unstale":"2024-09-12T03:23:22.941463+0000","last_undegraded":"2024-09-12T03:23:22.941463+0000","last_fullsized":"2024-09-12T03:23:22.941463+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:52:54.547658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.1","version":"0'0","reported_seq":14,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-12T03:23:22.942058+0000","last_change":"2024-09-12T03:23:20.682237+0000","last_active":"2024-09-12T03:23:22.942058+0000","last_peered":"2024-09-12T03:23:22.942058+0000","last_clean":"2024-09-12T03:23:22.942058+0000","last_became_active":"2024-09-12T03:23:20.681397+0000","last_became_peered":"2024-09-12T03:23:20.681397+0000","last_unstale":"2024-09-12T03:23:22.942058+0000","last_undegraded":"2024-09-12T03:23:22.942058+0000","last_fullsized":"2024-09-12T03:23:22.942058+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_clean_scrub_stamp":"2024-09-12T03:23:19.659665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:44:29.114769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.1","version":"0'0","reported_seq":12,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-12T03:23:22.692154+0000","last_change":"2024-09-12T03:23:21.696236+0000","last_active":"2024-09-12T03:23:22.692154+0000","last_peered":"2024-09-12T03:23:22.692154+0000","last_clean":"2024-09-12T03:23:22.692154+0000","last_became_active":"2024-09-12T03:23:21.695956+0000","last_became_peered":"2024-09-12T03:23:21.695956+0000","last_unstale":"2024-09-12T03:23:22.692154+0000","last_undegraded":"2024-09-12T03:23:22.692154+0000","last_fullsized":"2024-09-12T03:23:22.692154+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:37:45.624866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.0","version":"0'0","reported_seq":14,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-12T03:23:22.699785+0000","last_change":"2024-09-12T03:23:20.688059+0000","last_active":"2024-09-12T03:23:22.699785+0000","last_peered":"2024-09-12T03:23:22.699785+0000","last_clean":"2024-09-12T03:23:22.699785+0000","last_became_active":"2024-09-12T03:23:20.687774+0000","last_became_peered":"2024-09-12T03:23:20.687774+0000","last_unstale":"2024-09-12T03:23:22.699785+0000","last_undegraded":"2024-09-12T03:23:22.699785+0000","last_fullsized":"2024-09-12T03:23:22.699785+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_clean_scrub_stamp":"2024-09-12T03:23:19.659665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:44:02.603110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.18","version":"0'0","reported_seq":12,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-12T03:23:22.697805+0000","last_change":"2024-09-12T03:23:21.706201+0000","last_active":"2024-09-12T03:23:22.697805+0000","last_peered":"2024-09-12T03:23:22.697805+0000","last_clean":"2024-09-12T03:23:22.697805+0000","last_became_active":"2024-09-12T03:23:21.705906+0000","last_became_peered":"2024-09-12T03:23:21.705906+0000","last_unstale":"2024-09-12T03:23:22.697805+0000","last_undegraded":"2024-09-12T03:23:22.697805+0000","last_fullsized":"2024-09-12T03:23:22.697805+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:20.665225+0000","last_clean_scrub_stamp":"2024-09-12T03:23:20.665225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:40:23.338472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.19","version":"0'0","reported_seq":14,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-12T03:23:22.941654+0000","last_change":"2024-09-12T03:23:20.685565+0000","last_active":"2024-09-12T03:23:22.941654+0000","last_peered":"2024-09-12T03:23:22.941654+0000","last_clean":"2024-09-12T03:23:22.941654+0000","last_became_active":"2024-09-12T03:23:20.685230+0000","last_became_peered":"2024-09-12T03:23:20.685230+0000","last_unstale":"2024-09-12T03:23:22.941654+0000","last_undegraded":"2024-09-12T03:23:22.941654+0000","last_fullsized":"2024-09-12T03:23:22.941654+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:19.659665+0000","last_clean_scrub_stamp":"2024-09-12T03:23:19.659665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:49:03.225006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"70'128","reported_seq":445,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-12T03:23:22.941419+0000","last_change":"2024-09-12T03:16:21.883446+0000","last_active":"2024-09-12T03:23:22.941419+0000","last_peered":"2024-09-12T03:23:22.941419+0000","last_clean":"2024-09-12T03:23:22.941419+0000","last_became_active":"2024-09-12T03:16:21.882207+0000","last_became_peered":"2024-09-12T03:16:21.882207+0000","last_unstale":"2024-09-12T03:23:22.941419+0000","last_undegraded":"2024-09-12T03:23:22.941419+0000","last_fullsized":"2024-09-12T03:23:22.941419+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:20.316426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:20.316426+0000","last_clean_scrub_stamp":"2024-09-12T03:16:20.316426+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:12:15.424320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":25,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":24,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542231,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37776,"kb_used_data":2420,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334064,"statfs":{"total":96636764160,"available":96598081536,"internally_reserved":0,"allocated":2478080,"data_stored":1421056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542233,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37180,"kb_used_data":1832,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334660,"statfs":{"total":96636764160,"available":96598691840,"internally_reserved":0,"allocated":1875968,"data_stored":830584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542232,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41348,"kb_used_data":1832,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330492,"statfs":{"total":96636764160,"available":96594423808,"internally_reserved":0,"allocated":1875968,"data_stored":830584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542231,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37200,"kb_used_data":1848,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334640,"statfs":{"total":96636764160,"available":96598671360,"internally_reserved":0,"allocated":1892352,"data_stored":832778,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542232,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33040,"kb_used_data":1844,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338800,"statfs":{"total":96636764160,"available":96602931200,"internally_reserved":0,"allocated":1888256,"data_stored":830778,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542232,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33032,"kb_used_data":1844,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338808,"statfs":{"total":96636764160,"available":96602939392,"internally_reserved":0,"allocated":1888256,"data_stored":831278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542233,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33040,"kb_used_data":1844,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338800,"statfs":{"total":96636764160,"available":96602931200,"internally_reserved":0,"allocated":1888256,"data_stored":832196,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542233,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33608,"kb_used_data":2416,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338232,"statfs":{"total":96636764160,"available":96602349568,"internally_reserved":0,"allocated":2473984,"data_stored":1420974,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T03:23:30.439 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-12T03:23:30.439 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-12T03:23:30.586 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.585+0000 7f86cca06640 1 Processor -- start 2024-09-12T03:23:30.586 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.585+0000 7f86cca06640 1 -- start start 2024-09-12T03:23:30.587 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.586+0000 7f86cca06640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f86c815e530 0x7f86c80aff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:30.587 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.586+0000 7f86cca06640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f86c815e910 0x7f86c80b4710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:30.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.586+0000 7f86cca06640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f86c80aeb80 0x7f86c80722f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:30.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.586+0000 7f86cca06640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f86c80b07e0 con 0x7f86c815e530 2024-09-12T03:23:30.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.586+0000 7f86cca06640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f86c80b4c50 con 0x7f86c815e910 2024-09-12T03:23:30.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.586+0000 7f86cca06640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f86c806dc20 con 0x7f86c80aeb80 2024-09-12T03:23:30.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.586+0000 7f86c659c640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f86c815e530 0x7f86c80aff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:30.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.586+0000 7f86c659c640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f86c815e530 0x7f86c80aff20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34566/0 (socket says 172.21.15.27:34566) 2024-09-12T03:23:30.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.586+0000 7f86c659c640 1 -- 172.21.15.27:0/478367058 learned_addr learned my addr 172.21.15.27:0/478367058 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:23:30.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.586+0000 7f86c5d9b640 1 --2- 172.21.15.27:0/478367058 >> v2:172.21.15.45:3300/0 conn(0x7f86c815e910 0x7f86c80b4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:30.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.586+0000 7f86c6d9d640 1 --2- 172.21.15.27:0/478367058 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f86c80aeb80 0x7f86c80722f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:30.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.586+0000 7f86c659c640 1 -- 172.21.15.27:0/478367058 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f86c80aeb80 msgr2=0x7f86c80722f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:30.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.586+0000 7f86c659c640 1 --2- 172.21.15.27:0/478367058 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f86c80aeb80 0x7f86c80722f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:30.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.586+0000 7f86c659c640 1 -- 172.21.15.27:0/478367058 >> v2:172.21.15.45:3300/0 conn(0x7f86c815e910 msgr2=0x7f86c80b4710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:30.589 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.586+0000 7f86c659c640 1 --2- 172.21.15.27:0/478367058 >> v2:172.21.15.45:3300/0 conn(0x7f86c815e910 0x7f86c80b4710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:30.589 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.586+0000 7f86c659c640 1 -- 172.21.15.27:0/478367058 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86c80b0500 con 0x7f86c815e530 2024-09-12T03:23:30.589 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.586+0000 7f86c6d9d640 1 --2- 172.21.15.27:0/478367058 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f86c80aeb80 0x7f86c80722f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:23:30.589 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.586+0000 7f86c5d9b640 1 --2- 172.21.15.27:0/478367058 >> v2:172.21.15.45:3300/0 conn(0x7f86c815e910 0x7f86c80b4710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:23:30.590 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.587+0000 7f86c659c640 1 --2- 172.21.15.27:0/478367058 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f86c815e530 0x7f86c80aff20 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7f86c806f650 tx=0x7f86b802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=641054afc80ff81d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:30.590 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.587+0000 7f86c559a640 1 -- 172.21.15.27:0/478367058 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86b8036070 con 0x7f86c815e530 2024-09-12T03:23:30.590 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.587+0000 7f86c559a640 1 -- 172.21.15.27:0/478367058 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86b8031600 con 0x7f86c815e530 2024-09-12T03:23:30.590 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.587+0000 7f86c559a640 1 -- 172.21.15.27:0/478367058 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86b80319d0 con 0x7f86c815e530 2024-09-12T03:23:30.590 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.588+0000 7f86cca06640 1 -- 172.21.15.27:0/478367058 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f86c815e530 msgr2=0x7f86c80aff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:30.590 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.588+0000 7f86cca06640 1 --2- 172.21.15.27:0/478367058 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f86c815e530 0x7f86c80aff20 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7f86c806f650 tx=0x7f86b802a6a0 comp rx=0 tx=0).stop 2024-09-12T03:23:30.591 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.588+0000 7f86cca06640 1 -- 172.21.15.27:0/478367058 shutdown_connections 2024-09-12T03:23:30.591 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.588+0000 7f86cca06640 1 --2- 172.21.15.27:0/478367058 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f86c80aeb80 0x7f86c80722f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:30.591 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.588+0000 7f86cca06640 1 --2- 172.21.15.27:0/478367058 >> v2:172.21.15.45:3300/0 conn(0x7f86c815e910 0x7f86c80b4710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:30.591 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.588+0000 7f86cca06640 1 --2- 172.21.15.27:0/478367058 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f86c815e530 0x7f86c80aff20 unknown :-1 s=CLOSED pgs=1558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:30.591 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.588+0000 7f86cca06640 1 -- 172.21.15.27:0/478367058 >> 172.21.15.27:0/478367058 conn(0x7f86c806bb30 msgr2=0x7f86c806bf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:30.591 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.588+0000 7f86cca06640 1 -- 172.21.15.27:0/478367058 shutdown_connections 2024-09-12T03:23:30.591 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.588+0000 7f86cca06640 1 -- 172.21.15.27:0/478367058 wait complete. 2024-09-12T03:23:30.591 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.589+0000 7f86cca06640 1 Processor -- start 2024-09-12T03:23:30.591 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.589+0000 7f86cca06640 1 -- start start 2024-09-12T03:23:30.593 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.590+0000 7f86cca06640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f86c80aeb80 0x7f86c8162780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:30.593 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.590+0000 7f86cca06640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f86c815e530 0x7f86c81538b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:30.593 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.590+0000 7f86cca06640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f86c815e910 0x7f86c80cf8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:30.593 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.590+0000 7f86cca06640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f86c8071ff0 con 0x7f86c815e910 2024-09-12T03:23:30.593 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.590+0000 7f86cca06640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f86c806e1a0 con 0x7f86c80aeb80 2024-09-12T03:23:30.593 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.590+0000 7f86cca06640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f86c8072830 con 0x7f86c815e530 2024-09-12T03:23:30.593 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.590+0000 7f86c6d9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f86c815e910 0x7f86c80cf8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:30.594 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.590+0000 7f86c6d9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f86c815e910 0x7f86c80cf8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34582/0 (socket says 172.21.15.27:34582) 2024-09-12T03:23:30.594 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.590+0000 7f86c6d9d640 1 -- 172.21.15.27:0/2645645793 learned_addr learned my addr 172.21.15.27:0/2645645793 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:23:30.594 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.591+0000 7f86c5d9b640 1 --2- 172.21.15.27:0/2645645793 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f86c815e530 0x7f86c81538b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:30.594 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.591+0000 7f86c659c640 1 --2- 172.21.15.27:0/2645645793 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f86c80aeb80 0x7f86c8162780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:30.594 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.591+0000 7f86c6d9d640 1 -- 172.21.15.27:0/2645645793 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f86c815e530 msgr2=0x7f86c81538b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:30.594 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.591+0000 7f86c6d9d640 1 --2- 172.21.15.27:0/2645645793 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f86c815e530 0x7f86c81538b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:30.594 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.591+0000 7f86c6d9d640 1 -- 172.21.15.27:0/2645645793 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f86c80aeb80 msgr2=0x7f86c8162780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:30.594 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.591+0000 7f86c6d9d640 1 --2- 172.21.15.27:0/2645645793 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f86c80aeb80 0x7f86c8162780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:30.594 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.591+0000 7f86c6d9d640 1 -- 172.21.15.27:0/2645645793 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86c8154850 con 0x7f86c815e910 2024-09-12T03:23:30.594 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.591+0000 7f86c6d9d640 1 --2- 172.21.15.27:0/2645645793 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f86c815e910 0x7f86c80cf8b0 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7f86c000c820 tx=0x7f86c000cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:30.594 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.591+0000 7f86beffd640 1 -- 172.21.15.27:0/2645645793 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86c0015070 con 0x7f86c815e910 2024-09-12T03:23:30.594 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.591+0000 7f86beffd640 1 -- 172.21.15.27:0/2645645793 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86c0007a30 con 0x7f86c815e910 2024-09-12T03:23:30.594 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.591+0000 7f86cca06640 1 -- 172.21.15.27:0/2645645793 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86c8160e40 con 0x7f86c815e910 2024-09-12T03:23:30.595 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.591+0000 7f86beffd640 1 -- 172.21.15.27:0/2645645793 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86c0007e20 con 0x7f86c815e910 2024-09-12T03:23:30.595 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.591+0000 7f86cca06640 1 -- 172.21.15.27:0/2645645793 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86c80cd330 con 0x7f86c815e910 2024-09-12T03:23:30.595 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.592+0000 7f86cca06640 1 -- 172.21.15.27:0/2645645793 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86c8072c10 con 0x7f86c815e910 2024-09-12T03:23:30.595 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.593+0000 7f86beffd640 1 -- 172.21.15.27:0/2645645793 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 24) ==== 91212+0+0 (secure 0 0 0) 0x7f86c0018070 con 0x7f86c815e910 2024-09-12T03:23:30.595 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.593+0000 7f86beffd640 1 --2- 172.21.15.27:0/2645645793 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f86a806c640 0x7f86a806eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:30.596 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.593+0000 7f86beffd640 1 -- 172.21.15.27:0/2645645793 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 5765+0+0 (secure 0 0 0) 0x7f86c0089770 con 0x7f86c815e910 2024-09-12T03:23:30.596 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.593+0000 7f86c659c640 1 --2- 172.21.15.27:0/2645645793 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f86a806c640 0x7f86a806eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:30.596 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.594+0000 7f86c659c640 1 --2- 172.21.15.27:0/2645645793 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f86a806c640 0x7f86a806eb00 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f86b8033000 tx=0x7f86b8009000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:30.598 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.597+0000 7f86beffd640 1 -- 172.21.15.27:0/2645645793 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f86c0059a10 con 0x7f86c815e910 2024-09-12T03:23:30.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.839+0000 7f86cca06640 1 -- 172.21.15.27:0/2645645793 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f86c80b6040 con 0x7f86c815e910 2024-09-12T03:23:30.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.841+0000 7f86beffd640 1 -- 172.21.15.27:0/2645645793 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v22) ==== 190+0+147 (secure 0 0 0) 0x7f86c0059310 con 0x7f86c815e910 2024-09-12T03:23:30.842 INFO:teuthology.orchestra.run.smithi027.stdout:[client.0] 2024-09-12T03:23:30.842 INFO:teuthology.orchestra.run.smithi027.stdout: key = AQDVXOJmLm+5ExAAECM4CKfuVElmp/n0b02mXg== 2024-09-12T03:23:30.842 INFO:teuthology.orchestra.run.smithi027.stdout: caps mds = "allow" 2024-09-12T03:23:30.842 INFO:teuthology.orchestra.run.smithi027.stdout: caps mon = "allow r" 2024-09-12T03:23:30.842 INFO:teuthology.orchestra.run.smithi027.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-12T03:23:30.843 INFO:teuthology.orchestra.run.smithi027.stderr:updated caps for client.0 2024-09-12T03:23:30.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.843+0000 7f86cca06640 1 -- 172.21.15.27:0/2645645793 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f86a806c640 msgr2=0x7f86a806eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:30.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.843+0000 7f86cca06640 1 --2- 172.21.15.27:0/2645645793 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f86a806c640 0x7f86a806eb00 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f86b8033000 tx=0x7f86b8009000 comp rx=0 tx=0).stop 2024-09-12T03:23:30.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.843+0000 7f86cca06640 1 -- 172.21.15.27:0/2645645793 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f86c815e910 msgr2=0x7f86c80cf8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:30.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.843+0000 7f86cca06640 1 --2- 172.21.15.27:0/2645645793 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f86c815e910 0x7f86c80cf8b0 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7f86c000c820 tx=0x7f86c000cce0 comp rx=0 tx=0).stop 2024-09-12T03:23:30.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.843+0000 7f86cca06640 1 -- 172.21.15.27:0/2645645793 shutdown_connections 2024-09-12T03:23:30.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.843+0000 7f86cca06640 1 --2- 172.21.15.27:0/2645645793 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f86a806c640 0x7f86a806eb00 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:30.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.843+0000 7f86cca06640 1 --2- 172.21.15.27:0/2645645793 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f86c815e910 0x7f86c80cf8b0 unknown :-1 s=CLOSED pgs=1559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:30.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.843+0000 7f86cca06640 1 --2- 172.21.15.27:0/2645645793 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f86c815e530 0x7f86c81538b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:30.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.843+0000 7f86cca06640 1 --2- 172.21.15.27:0/2645645793 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f86c80aeb80 0x7f86c8162780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:30.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.843+0000 7f86cca06640 1 -- 172.21.15.27:0/2645645793 >> 172.21.15.27:0/2645645793 conn(0x7f86c806bb30 msgr2=0x7f86c815cfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:30.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.844+0000 7f86cca06640 1 -- 172.21.15.27:0/2645645793 shutdown_connections 2024-09-12T03:23:30.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.844+0000 7f86cca06640 1 -- 172.21.15.27:0/2645645793 wait complete. 2024-09-12T03:23:30.856 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-12T03:23:30.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.957+0000 7fb657a30640 1 Processor -- start 2024-09-12T03:23:30.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.957+0000 7fb657a30640 1 -- start start 2024-09-12T03:23:30.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.958+0000 7fb657a30640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fb65015eb90 0x7fb6500d2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:30.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.958+0000 7fb657a30640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb6500d3760 con 0x7fb6500d5200 2024-09-12T03:23:30.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.958+0000 7fb657a30640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb6500d3a40 con 0x7fb6500d1720 2024-09-12T03:23:30.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.958+0000 7fb657a30640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7fb6500b7d20 con 0x7fb65015eb90 2024-09-12T03:23:30.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.958+0000 7fb6557cc640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fb65015eb90 0x7fb6500d2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:30.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.958+0000 7fb654fcb640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fb6500d5200 0x7fb6500d1010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:51234/0 (socket says 172.21.15.27:51234) 2024-09-12T03:23:30.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.958+0000 7fb654fcb640 1 -- 172.21.15.27:0/3768212121 learned_addr learned my addr 172.21.15.27:0/3768212121 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:23:30.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.959+0000 7fb6557cc640 1 -- 172.21.15.27:0/3768212121 >> v1:172.21.15.45:6789/0 conn(0x7fb6500d1720 legacy=0x7fb6500b8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:30.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.959+0000 7fb6557cc640 1 -- 172.21.15.27:0/3768212121 >> v1:172.21.15.27:6789/0 conn(0x7fb6500d5200 legacy=0x7fb6500d1010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:30.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.959+0000 7fb6557cc640 1 -- 172.21.15.27:0/3768212121 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6500d3d20 con 0x7fb65015eb90 2024-09-12T03:23:30.961 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.959+0000 7fb6557cc640 1 --2- 172.21.15.27:0/3768212121 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb65015eb90 0x7fb6500d2df0 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7fb64c0097a0 tx=0x7fb64c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=caa4a2d6d2ee35da server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:30.962 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.960+0000 7fb647fff640 1 -- 172.21.15.27:0/3768212121 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb64c037070 con 0x7fb65015eb90 2024-09-12T03:23:30.962 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.960+0000 7fb647fff640 1 -- 172.21.15.27:0/3768212121 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb64c032440 con 0x7fb65015eb90 2024-09-12T03:23:30.962 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.960+0000 7fb647fff640 1 -- 172.21.15.27:0/3768212121 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb64c032810 con 0x7fb65015eb90 2024-09-12T03:23:30.962 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.960+0000 7fb657a30640 1 -- 172.21.15.27:0/3768212121 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb65015eb90 msgr2=0x7fb6500d2df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:30.962 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.960+0000 7fb657a30640 1 --2- 172.21.15.27:0/3768212121 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb65015eb90 0x7fb6500d2df0 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7fb64c0097a0 tx=0x7fb64c02a1e0 comp rx=0 tx=0).stop 2024-09-12T03:23:30.962 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.960+0000 7fb657a30640 1 -- 172.21.15.27:0/3768212121 shutdown_connections 2024-09-12T03:23:30.962 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.960+0000 7fb657a30640 1 --2- 172.21.15.27:0/3768212121 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb65015eb90 0x7fb6500d2df0 unknown :-1 s=CLOSED pgs=1561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:30.962 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.960+0000 7fb657a30640 1 -- 172.21.15.27:0/3768212121 >> 172.21.15.27:0/3768212121 conn(0x7fb65006dce0 msgr2=0x7fb65006e0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:30.962 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.960+0000 7fb657a30640 1 -- 172.21.15.27:0/3768212121 shutdown_connections 2024-09-12T03:23:30.962 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.960+0000 7fb657a30640 1 -- 172.21.15.27:0/3768212121 wait complete. 2024-09-12T03:23:30.964 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.961+0000 7fb657a30640 1 Processor -- start 2024-09-12T03:23:30.964 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.961+0000 7fb657a30640 1 -- start start 2024-09-12T03:23:30.964 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.962+0000 7fb657a30640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb6500d1720 0x7fb650158100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:30.964 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.962+0000 7fb657a30640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb6500d5200 0x7fb6501590f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:30.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.962+0000 7fb657a30640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb65015eb90 0x7fb65015b040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:30.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.962+0000 7fb657a30640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fb6500d3330 con 0x7fb6500d1720 2024-09-12T03:23:30.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.962+0000 7fb657a30640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb6500d42e0 con 0x7fb6500d5200 2024-09-12T03:23:30.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.962+0000 7fb657a30640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fb6500d34b0 con 0x7fb65015eb90 2024-09-12T03:23:30.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.962+0000 7fb6557cc640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb6500d1720 0x7fb650158100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:30.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.962+0000 7fb6557cc640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb6500d1720 0x7fb650158100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34610/0 (socket says 172.21.15.27:34610) 2024-09-12T03:23:30.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.962+0000 7fb6557cc640 1 -- 172.21.15.27:0/973571810 learned_addr learned my addr 172.21.15.27:0/973571810 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:23:30.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.962+0000 7fb654fcb640 1 --2- 172.21.15.27:0/973571810 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb6500d5200 0x7fb6501590f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:30.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.962+0000 7fb655fcd640 1 --2- 172.21.15.27:0/973571810 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb65015eb90 0x7fb65015b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:30.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.963+0000 7fb6557cc640 1 -- 172.21.15.27:0/973571810 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb65015eb90 msgr2=0x7fb65015b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:30.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.963+0000 7fb6557cc640 1 --2- 172.21.15.27:0/973571810 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb65015eb90 0x7fb65015b040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:30.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.963+0000 7fb6557cc640 1 -- 172.21.15.27:0/973571810 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb6500d5200 msgr2=0x7fb6501590f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:30.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.963+0000 7fb6557cc640 1 --2- 172.21.15.27:0/973571810 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb6500d5200 0x7fb6501590f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:30.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.963+0000 7fb6557cc640 1 -- 172.21.15.27:0/973571810 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb650159840 con 0x7fb6500d1720 2024-09-12T03:23:30.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.963+0000 7fb6557cc640 1 --2- 172.21.15.27:0/973571810 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb6500d1720 0x7fb650158100 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7fb64c0098c0 tx=0x7fb64c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:30.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.963+0000 7fb645ffb640 1 -- 172.21.15.27:0/973571810 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb64c041070 con 0x7fb6500d1720 2024-09-12T03:23:30.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.963+0000 7fb645ffb640 1 -- 172.21.15.27:0/973571810 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb64c03c930 con 0x7fb6500d1720 2024-09-12T03:23:30.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.963+0000 7fb645ffb640 1 -- 172.21.15.27:0/973571810 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb64c037040 con 0x7fb6500d1720 2024-09-12T03:23:30.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.963+0000 7fb657a30640 1 -- 172.21.15.27:0/973571810 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6501e8700 con 0x7fb6500d1720 2024-09-12T03:23:30.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.963+0000 7fb657a30640 1 -- 172.21.15.27:0/973571810 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6500c8340 con 0x7fb6500d1720 2024-09-12T03:23:30.967 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.964+0000 7fb657a30640 1 -- 172.21.15.27:0/973571810 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6500693d0 con 0x7fb6500d1720 2024-09-12T03:23:30.967 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.965+0000 7fb645ffb640 1 -- 172.21.15.27:0/973571810 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 24) ==== 91212+0+0 (secure 0 0 0) 0x7fb64c044070 con 0x7fb6500d1720 2024-09-12T03:23:30.967 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.965+0000 7fb645ffb640 1 --2- 172.21.15.27:0/973571810 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fb63406c610 0x7fb63406ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:30.967 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.965+0000 7fb645ffb640 1 -- 172.21.15.27:0/973571810 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 5765+0+0 (secure 0 0 0) 0x7fb64c0aa5d0 con 0x7fb6500d1720 2024-09-12T03:23:30.967 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.965+0000 7fb654fcb640 1 --2- 172.21.15.27:0/973571810 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fb63406c610 0x7fb63406ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:30.967 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.966+0000 7fb654fcb640 1 --2- 172.21.15.27:0/973571810 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fb63406c610 0x7fb63406ead0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fb650158b00 tx=0x7fb640007e80 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:30.970 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:30.970+0000 7fb645ffb640 1 -- 172.21.15.27:0/973571810 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fb64c0b2020 con 0x7fb6500d1720 2024-09-12T03:23:31.182 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.181+0000 7fb657a30640 1 -- 172.21.15.27:0/973571810 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb650162360 con 0x7fb6500d1720 2024-09-12T03:23:31.183 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.182+0000 7fb645ffb640 1 -- 172.21.15.27:0/973571810 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 91 v91) ==== 94+0+5227 (secure 0 0 0) 0x7fb64c07a340 con 0x7fb6500d1720 2024-09-12T03:23:31.183 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:23:31.183 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":91,"btime":"2024-09-12T03:23:24:700400+0000","default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6917,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/3682270902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":3682270902},{"type":"v1","addr":"172.21.15.45:6834","nonce":3682270902}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":87},{"gid":7771,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/2399103579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":2399103579},{"type":"v1","addr":"172.21.15.45:6835","nonce":2399103579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":87},{"gid":8559,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6834/1045432254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":1045432254},{"type":"v1","addr":"172.21.15.27:6834","nonce":1045432254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":87}],"filesystems":[{"mdsmap":{"epoch":91,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-12T03:23:21.031667+0000","modified":"2024-09-12T03:23:23.983036+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8562},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8562":{"gid":8562,"name":"c","rank":0,"incarnation":88,"state":"up:active","state_seq":3,"addr":"172.21.15.27:6835/1241148443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":1241148443},{"type":"v1","addr":"172.21.15.27:6835","nonce":1241148443}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[25],"metadata_pool":24,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8562,"qdb_cluster":[8562]},"id":12}]} 2024-09-12T03:23:31.183 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 91 2024-09-12T03:23:31.185 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.184+0000 7fb657a30640 1 -- 172.21.15.27:0/973571810 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fb63406c610 msgr2=0x7fb63406ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:31.185 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.184+0000 7fb657a30640 1 --2- 172.21.15.27:0/973571810 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fb63406c610 0x7fb63406ead0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fb650158b00 tx=0x7fb640007e80 comp rx=0 tx=0).stop 2024-09-12T03:23:31.185 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.184+0000 7fb657a30640 1 -- 172.21.15.27:0/973571810 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb6500d1720 msgr2=0x7fb650158100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:31.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.184+0000 7fb657a30640 1 --2- 172.21.15.27:0/973571810 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb6500d1720 0x7fb650158100 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7fb64c0098c0 tx=0x7fb64c0046c0 comp rx=0 tx=0).stop 2024-09-12T03:23:31.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.184+0000 7fb657a30640 1 -- 172.21.15.27:0/973571810 shutdown_connections 2024-09-12T03:23:31.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.184+0000 7fb657a30640 1 --2- 172.21.15.27:0/973571810 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb65015eb90 0x7fb65015b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:31.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.184+0000 7fb657a30640 1 --2- 172.21.15.27:0/973571810 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb6500d5200 0x7fb6501590f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:31.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.184+0000 7fb657a30640 1 --2- 172.21.15.27:0/973571810 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fb63406c610 0x7fb63406ead0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:31.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.184+0000 7fb657a30640 1 --2- 172.21.15.27:0/973571810 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb6500d1720 0x7fb650158100 unknown :-1 s=CLOSED pgs=1562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:31.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.184+0000 7fb657a30640 1 -- 172.21.15.27:0/973571810 >> 172.21.15.27:0/973571810 conn(0x7fb65006dce0 msgr2=0x7fb650160fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:31.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.184+0000 7fb657a30640 1 -- 172.21.15.27:0/973571810 shutdown_connections 2024-09-12T03:23:31.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.184+0000 7fb657a30640 1 -- 172.21.15.27:0/973571810 wait complete. 2024-09-12T03:23:31.344 DEBUG:teuthology.orchestra.run.smithi027:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-12T03:23:31.347 DEBUG:teuthology.orchestra.run.smithi045:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-12T03:23:31.600 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 91, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-12T03:23:21.031667+0000', 'modified': '2024-09-12T03:23:23.983036+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8562}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8562': {'gid': 8562, 'name': 'c', 'rank': 0, 'incarnation': 88, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.27:6835/1241148443', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6833', 'nonce': 1241148443}, {'type': 'v1', 'addr': '172.21.15.27:6835', 'nonce': 1241148443}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [25], 'metadata_pool': 24, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 8562, 'qdb_cluster': [8562]} 2024-09-12T03:23:31.600 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-12T03:23:31.600 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-12T03:23:31.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.705+0000 7f16ab752640 1 Processor -- start 2024-09-12T03:23:31.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.705+0000 7f16ab752640 1 -- start start 2024-09-12T03:23:31.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.706+0000 7f16ab752640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f16a406f360 0x7f16a40cef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:31.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.706+0000 7f16ab752640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f16a40cf490 0x7f16a40d2780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:31.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.706+0000 7f16ab752640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f16a406cc40 con 0x7f16a40b1ca0 2024-09-12T03:23:31.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.706+0000 7f16ab752640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f16a40d22f0 con 0x7f16a40cf490 2024-09-12T03:23:31.710 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.706+0000 7f16ab752640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f16a40d1ce0 con 0x7f16a406f360 2024-09-12T03:23:31.710 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.706+0000 7f16a94ee640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f16a40b1ca0 0x7f16a415f830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:51246/0 (socket says 172.21.15.27:51246) 2024-09-12T03:23:31.710 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.706+0000 7f16a94ee640 1 -- 172.21.15.27:0/2662410669 learned_addr learned my addr 172.21.15.27:0/2662410669 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:23:31.710 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.707+0000 7f16a9cef640 1 --2- 172.21.15.27:0/2662410669 >> v2:172.21.15.45:3300/0 conn(0x7f16a40cf490 0x7f16a40d2780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:31.710 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.707+0000 7f16a8ced640 1 --2- 172.21.15.27:0/2662410669 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f16a406f360 0x7f16a40cef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:31.710 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.707+0000 7f16a3fff640 1 -- 172.21.15.27:0/2662410669 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1180675125 0 0) 0x7f16a406cc40 con 0x7f16a40b1ca0 2024-09-12T03:23:31.710 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.707+0000 7f16a3fff640 1 -- 172.21.15.27:0/2662410669 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f168c003520 con 0x7f16a40b1ca0 2024-09-12T03:23:31.710 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.707+0000 7f16a3fff640 1 -- 172.21.15.27:0/2662410669 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2165218002 0 0) 0x7f168c003520 con 0x7f16a40b1ca0 2024-09-12T03:23:31.710 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.707+0000 7f16a3fff640 1 -- 172.21.15.27:0/2662410669 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f16a406f360 msgr2=0x7f16a40cef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:31.710 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.707+0000 7f16a3fff640 1 --2- 172.21.15.27:0/2662410669 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f16a406f360 0x7f16a40cef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:31.710 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.707+0000 7f16a3fff640 1 -- 172.21.15.27:0/2662410669 >> v2:172.21.15.45:3300/0 conn(0x7f16a40cf490 msgr2=0x7f16a40d2780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:31.710 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.707+0000 7f16a3fff640 1 --2- 172.21.15.27:0/2662410669 >> v2:172.21.15.45:3300/0 conn(0x7f16a40cf490 0x7f16a40d2780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:31.710 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.707+0000 7f16a3fff640 1 -- 172.21.15.27:0/2662410669 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16a406cee0 con 0x7f16a40b1ca0 2024-09-12T03:23:31.711 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.707+0000 7f16a3fff640 1 -- 172.21.15.27:0/2662410669 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 847755213 0 0) 0x7f16940019f0 con 0x7f16a40b1ca0 2024-09-12T03:23:31.711 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.707+0000 7f16a9cef640 1 --2- 172.21.15.27:0/2662410669 >> v2:172.21.15.45:3300/0 conn(0x7f16a40cf490 0x7f16a40d2780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:23:31.711 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.707+0000 7f16a3fff640 1 -- 172.21.15.27:0/2662410669 >> v1:172.21.15.27:6789/0 conn(0x7f16a40b1ca0 legacy=0x7f16a415f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:31.711 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.707+0000 7f16a3fff640 1 --2- 172.21.15.27:0/2662410669 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f168c0039b0 0x7f168c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:31.711 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.707+0000 7f16a3fff640 1 --2- 172.21.15.27:0/2662410669 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f16a406f360 0x7f168c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:31.711 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.707+0000 7f16a3fff640 1 --2- 172.21.15.27:0/2662410669 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f168c004d40 0x7f168c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:31.711 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.707+0000 7f16a3fff640 1 -- 172.21.15.27:0/2662410669 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f16a40d22f0 con 0x7f168c004d40 2024-09-12T03:23:31.711 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.707+0000 7f16a3fff640 1 -- 172.21.15.27:0/2662410669 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f16a40d1ce0 con 0x7f168c0039b0 2024-09-12T03:23:31.711 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.707+0000 7f16a3fff640 1 -- 172.21.15.27:0/2662410669 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f168c00b9c0 con 0x7f16a406f360 2024-09-12T03:23:31.711 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.707+0000 7f16a9cef640 1 --2- 172.21.15.27:0/2662410669 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f168c004d40 0x7f168c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:31.711 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.707+0000 7f16a94ee640 1 --2- 172.21.15.27:0/2662410669 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f168c0039b0 0x7f168c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:31.711 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.707+0000 7f16a8ced640 1 --2- 172.21.15.27:0/2662410669 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f16a406f360 0x7f168c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:31.712 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.708+0000 7f16a9cef640 1 -- 172.21.15.27:0/2662410669 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f16a406f360 msgr2=0x7f168c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:31.712 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.708+0000 7f16a9cef640 1 --2- 172.21.15.27:0/2662410669 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f16a406f360 0x7f168c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:31.712 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.708+0000 7f16a9cef640 1 -- 172.21.15.27:0/2662410669 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f168c0039b0 msgr2=0x7f168c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:31.712 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.708+0000 7f16a9cef640 1 --2- 172.21.15.27:0/2662410669 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f168c0039b0 0x7f168c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:31.712 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.708+0000 7f16a9cef640 1 -- 172.21.15.27:0/2662410669 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f168c003520 con 0x7f168c004d40 2024-09-12T03:23:31.712 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.708+0000 7f16a9cef640 1 --2- 172.21.15.27:0/2662410669 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f168c004d40 0x7f168c0051c0 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7f169c009cc0 tx=0x7f168c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c711a98194e1869 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:31.712 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.708+0000 7f16a3fff640 1 -- 172.21.15.27:0/2662410669 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f169c036070 con 0x7f168c004d40 2024-09-12T03:23:31.712 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.708+0000 7f16a3fff640 1 -- 172.21.15.27:0/2662410669 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f169c031400 con 0x7f168c004d40 2024-09-12T03:23:31.712 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.708+0000 7f16a3fff640 1 -- 172.21.15.27:0/2662410669 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f169c009860 con 0x7f168c004d40 2024-09-12T03:23:31.715 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.709+0000 7f16ab752640 1 -- 172.21.15.27:0/2662410669 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f168c004d40 msgr2=0x7f168c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:31.715 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.709+0000 7f16ab752640 1 --2- 172.21.15.27:0/2662410669 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f168c004d40 0x7f168c0051c0 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7f169c009cc0 tx=0x7f168c005830 comp rx=0 tx=0).stop 2024-09-12T03:23:31.715 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.709+0000 7f16a9cef640 1 -- 172.21.15.27:0/2662410669 reap_dead start 2024-09-12T03:23:31.715 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.709+0000 7f16ab752640 1 -- 172.21.15.27:0/2662410669 shutdown_connections 2024-09-12T03:23:31.715 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.709+0000 7f16ab752640 1 -- 172.21.15.27:0/2662410669 >> 172.21.15.27:0/2662410669 conn(0x7f16a4073800 msgr2=0x7f16a409ba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:31.715 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.709+0000 7f16ab752640 1 -- 172.21.15.27:0/2662410669 shutdown_connections 2024-09-12T03:23:31.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.709+0000 7f16ab752640 1 -- 172.21.15.27:0/2662410669 wait complete. 2024-09-12T03:23:31.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.710+0000 7f16ab752640 1 Processor -- start 2024-09-12T03:23:31.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.710+0000 7f16ab752640 1 -- start start 2024-09-12T03:23:31.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.711+0000 7f16ab752640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f168c004d40 0x7f16a40cb850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:31.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.711+0000 7f16ab752640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f16a4161930 0x7f16a40b6790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:31.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.711+0000 7f16ab752640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f16a415fe80 0x7f16a415af70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:31.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.711+0000 7f16ab752640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f16a406d340 con 0x7f16a4161930 2024-09-12T03:23:31.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.711+0000 7f16ab752640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f16a406d1c0 con 0x7f16a415fe80 2024-09-12T03:23:31.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.711+0000 7f16ab752640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f16a40cfc40 con 0x7f168c004d40 2024-09-12T03:23:31.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.711+0000 7f16a8ced640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f16a4161930 0x7f16a40b6790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:31.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.711+0000 7f16a8ced640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f16a4161930 0x7f16a40b6790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34626/0 (socket says 172.21.15.27:34626) 2024-09-12T03:23:31.717 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.711+0000 7f16a8ced640 1 -- 172.21.15.27:0/1834011858 learned_addr learned my addr 172.21.15.27:0/1834011858 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:23:31.717 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.711+0000 7f16a9cef640 1 --2- 172.21.15.27:0/1834011858 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f16a415fe80 0x7f16a415af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:31.717 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.711+0000 7f16a94ee640 1 --2- 172.21.15.27:0/1834011858 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f168c004d40 0x7f16a40cb850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:31.717 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.711+0000 7f16a8ced640 1 -- 172.21.15.27:0/1834011858 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f168c004d40 msgr2=0x7f16a40cb850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:31.717 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.711+0000 7f16a8ced640 1 --2- 172.21.15.27:0/1834011858 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f168c004d40 0x7f16a40cb850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:31.717 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.711+0000 7f16a8ced640 1 -- 172.21.15.27:0/1834011858 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f16a415fe80 msgr2=0x7f16a415af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:31.717 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.711+0000 7f16a8ced640 1 --2- 172.21.15.27:0/1834011858 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f16a415fe80 0x7f16a415af70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:31.717 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.711+0000 7f16a8ced640 1 -- 172.21.15.27:0/1834011858 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16a41f4040 con 0x7f16a4161930 2024-09-12T03:23:31.717 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.711+0000 7f16a94ee640 1 --2- 172.21.15.27:0/1834011858 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f168c004d40 0x7f16a40cb850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:23:31.717 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.712+0000 7f16a8ced640 1 --2- 172.21.15.27:0/1834011858 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f16a4161930 0x7f16a40b6790 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7f1698011940 tx=0x7f1698011e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:31.717 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.712+0000 7f16a1ffb640 1 -- 172.21.15.27:0/1834011858 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f169800faf0 con 0x7f16a4161930 2024-09-12T03:23:31.717 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.712+0000 7f16a1ffb640 1 -- 172.21.15.27:0/1834011858 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f169800fc90 con 0x7f16a4161930 2024-09-12T03:23:31.717 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.712+0000 7f16ab752640 1 -- 172.21.15.27:0/1834011858 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16a40cc300 con 0x7f16a4161930 2024-09-12T03:23:31.718 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.712+0000 7f16a1ffb640 1 -- 172.21.15.27:0/1834011858 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1698014440 con 0x7f16a4161930 2024-09-12T03:23:31.718 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.712+0000 7f16ab752640 1 -- 172.21.15.27:0/1834011858 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16a40ceb70 con 0x7f16a4161930 2024-09-12T03:23:31.718 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.714+0000 7f16a1ffb640 1 -- 172.21.15.27:0/1834011858 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 24) ==== 91212+0+0 (secure 0 0 0) 0x7f1698012040 con 0x7f16a4161930 2024-09-12T03:23:31.718 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.714+0000 7f16a1ffb640 1 --2- 172.21.15.27:0/1834011858 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f169006c610 0x7f169006ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:31.718 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.714+0000 7f16a1ffb640 1 -- 172.21.15.27:0/1834011858 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 5765+0+0 (secure 0 0 0) 0x7f169808c6e0 con 0x7f16a4161930 2024-09-12T03:23:31.718 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.714+0000 7f16a94ee640 1 --2- 172.21.15.27:0/1834011858 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f169006c610 0x7f169006ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:31.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.715+0000 7f16a94ee640 1 --2- 172.21.15.27:0/1834011858 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f169006c610 0x7f169006ead0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f16a40b3750 tx=0x7f1694006040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:31.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.715+0000 7f16ae5cd740 1 Processor -- start 2024-09-12T03:23:31.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.715+0000 7f16ae5cd740 1 -- start start 2024-09-12T03:23:31.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.716+0000 7f16ae5cd740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563ee8df32c0 0x563ee8e38a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:31.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.716+0000 7f16ae5cd740 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x563ee8df0a80 0x563ee8df36a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:31.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.716+0000 7f16ae5cd740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563ee8df4a50 0x563ee8ded8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:31.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.716+0000 7f16ae5cd740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x563ee8d55520 con 0x563ee8df4a50 2024-09-12T03:23:31.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.716+0000 7f16ae5cd740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x563ee8cfd3e0 con 0x563ee8df32c0 2024-09-12T03:23:31.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.716+0000 7f16ae5cd740 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x563ee8d514e0 con 0x563ee8df0a80 2024-09-12T03:23:31.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.716+0000 7f16a9cef640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563ee8df4a50 0x563ee8ded8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:31.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.716+0000 7f16a9cef640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563ee8df4a50 0x563ee8ded8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34634/0 (socket says 172.21.15.27:34634) 2024-09-12T03:23:31.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.716+0000 7f16a9cef640 1 -- 172.21.15.27:0/3512591314 learned_addr learned my addr 172.21.15.27:0/3512591314 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:23:31.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.716+0000 7f16a94ee640 1 --2- 172.21.15.27:0/3512591314 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563ee8df32c0 0x563ee8e38a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:31.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.716+0000 7f16a8ced640 1 --2- 172.21.15.27:0/3512591314 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x563ee8df0a80 0x563ee8df36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:31.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.716+0000 7f16a9cef640 1 -- 172.21.15.27:0/3512591314 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x563ee8df0a80 msgr2=0x563ee8df36a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:31.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.716+0000 7f16a9cef640 1 --2- 172.21.15.27:0/3512591314 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x563ee8df0a80 0x563ee8df36a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:31.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.716+0000 7f16a9cef640 1 -- 172.21.15.27:0/3512591314 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563ee8df32c0 msgr2=0x563ee8e38a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:31.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.716+0000 7f16a9cef640 1 --2- 172.21.15.27:0/3512591314 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563ee8df32c0 0x563ee8e38a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:31.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.716+0000 7f16a9cef640 1 -- 172.21.15.27:0/3512591314 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563ee8d182f0 con 0x563ee8df4a50 2024-09-12T03:23:31.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.716+0000 7f16a94ee640 1 --2- 172.21.15.27:0/3512591314 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563ee8df32c0 0x563ee8e38a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:23:31.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.716+0000 7f16a9cef640 1 --2- 172.21.15.27:0/3512591314 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563ee8df4a50 0x563ee8ded8b0 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7f169c009be0 tx=0x7f169c02e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cf0a5fc8281966b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:31.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.717+0000 7f168affd640 1 -- 172.21.15.27:0/3512591314 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f169c036040 con 0x563ee8df4a50 2024-09-12T03:23:31.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.717+0000 7f168affd640 1 -- 172.21.15.27:0/3512591314 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f169c004020 con 0x563ee8df4a50 2024-09-12T03:23:31.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.717+0000 7f168affd640 1 -- 172.21.15.27:0/3512591314 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f169c0043f0 con 0x563ee8df4a50 2024-09-12T03:23:31.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.718+0000 7f16ae5cd740 1 -- 172.21.15.27:0/3512591314 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563ee8df4a50 msgr2=0x563ee8ded8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:31.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.718+0000 7f16ae5cd740 1 --2- 172.21.15.27:0/3512591314 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563ee8df4a50 0x563ee8ded8b0 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7f169c009be0 tx=0x7f169c02e000 comp rx=0 tx=0).stop 2024-09-12T03:23:31.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.718+0000 7f16ae5cd740 1 -- 172.21.15.27:0/3512591314 shutdown_connections 2024-09-12T03:23:31.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.718+0000 7f16ae5cd740 1 --2- 172.21.15.27:0/3512591314 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563ee8df4a50 0x563ee8ded8b0 unknown :-1 s=CLOSED pgs=1566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:31.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.718+0000 7f16ae5cd740 1 --2- 172.21.15.27:0/3512591314 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x563ee8df0a80 0x563ee8df36a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:31.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.718+0000 7f16ae5cd740 1 --2- 172.21.15.27:0/3512591314 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563ee8df32c0 0x563ee8e38a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:31.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.718+0000 7f16ae5cd740 1 -- 172.21.15.27:0/3512591314 >> 172.21.15.27:0/3512591314 conn(0x563ee8defa80 msgr2=0x563ee8e37f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:31.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.718+0000 7f16ae5cd740 1 -- 172.21.15.27:0/3512591314 shutdown_connections 2024-09-12T03:23:31.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.718+0000 7f16ae5cd740 1 -- 172.21.15.27:0/3512591314 wait complete. 2024-09-12T03:23:31.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.718+0000 7f16ae5cd740 1 -- start start 2024-09-12T03:23:31.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.719+0000 7f16ae5cd740 1 Processor -- start 2024-09-12T03:23:31.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.719+0000 7f16ae5cd740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563ee8df32c0 0x563ee8e4b590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:31.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.719+0000 7f16ae5cd740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563ee8df4a50 0x563ee8e127f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:31.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.720+0000 7f16ae5cd740 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x563ee8ded050 0x563ee8e192f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:31.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.720+0000 7f16ae5cd740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x563ee8deef20 con 0x563ee8df32c0 2024-09-12T03:23:31.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.720+0000 7f16ae5cd740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x563ee8cde450 con 0x563ee8df4a50 2024-09-12T03:23:31.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.720+0000 7f16ae5cd740 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x563ee8e39290 con 0x563ee8ded050 2024-09-12T03:23:31.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.720+0000 7f16ae5cd740 10 client.0 _ll_register_callbacks cb 0x563ee8a67ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:23:31.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.720+0000 7f16a94ee640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563ee8df32c0 0x563ee8e4b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:31.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.720+0000 7f16a94ee640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563ee8df32c0 0x563ee8e4b590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34644/0 (socket says 172.21.15.27:34644) 2024-09-12T03:23:31.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.720+0000 7f16a94ee640 1 -- 172.21.15.27:0/267478958 learned_addr learned my addr 172.21.15.27:0/267478958 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:23:31.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.720+0000 7f16a8ced640 1 --2- 172.21.15.27:0/267478958 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563ee8df4a50 0x563ee8e127f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:31.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.720+0000 7f16a9cef640 1 --2- 172.21.15.27:0/267478958 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x563ee8ded050 0x563ee8e192f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:31.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.720+0000 7f16a94ee640 1 -- 172.21.15.27:0/267478958 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x563ee8ded050 msgr2=0x563ee8e192f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:31.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.720+0000 7f16a94ee640 1 --2- 172.21.15.27:0/267478958 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x563ee8ded050 0x563ee8e192f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:31.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.720+0000 7f16a94ee640 1 -- 172.21.15.27:0/267478958 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563ee8df4a50 msgr2=0x563ee8e127f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:31.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.720+0000 7f16a94ee640 1 --2- 172.21.15.27:0/267478958 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563ee8df4a50 0x563ee8e127f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:31.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.720+0000 7f16a94ee640 1 -- 172.21.15.27:0/267478958 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563ee8a682d0 con 0x563ee8df32c0 2024-09-12T03:23:31.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.720+0000 7f16a94ee640 1 --2- 172.21.15.27:0/267478958 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563ee8df32c0 0x563ee8e4b590 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7f1694015eb0 tx=0x7f16a40d2260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6243ead5a8cecf41 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:31.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.720+0000 7f1683fff640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-12T03:23:31.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.720+0000 7f1683fff640 1 -- 172.21.15.27:0/267478958 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1694028060 con 0x563ee8df32c0 2024-09-12T03:23:31.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.720+0000 7f1683fff640 1 -- 172.21.15.27:0/267478958 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(149..149 src has 1..149) ==== 5765+0+0 (secure 0 0 0) 0x7f1694023b20 con 0x563ee8df32c0 2024-09-12T03:23:31.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.720+0000 7f16ae5cd740 1 -- 172.21.15.27:0/267478958 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f169c004020 con 0x563ee8df32c0 2024-09-12T03:23:31.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.721+0000 7f1683fff640 1 -- 172.21.15.27:0/267478958 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f1694023d20 con 0x563ee8df32c0 2024-09-12T03:23:31.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.721+0000 7f16ae5cd740 10 client.8730 fetch_fsmap learned FSMap version 91 2024-09-12T03:23:31.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.721+0000 7f16ae5cd740 1 -- 172.21.15.27:0/267478958 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=91,monmap=2+}) -- 0x563ee8b326b0 con 0x563ee8df32c0 2024-09-12T03:23:31.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.721+0000 7f1683fff640 1 -- 172.21.15.27:0/267478958 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f169c004020 con 0x563ee8df32c0 2024-09-12T03:23:31.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.721+0000 7f1683fff640 1 -- 172.21.15.27:0/267478958 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 91) ==== 3502+0+0 (secure 0 0 0) 0x7f169402d5c0 con 0x563ee8df32c0 2024-09-12T03:23:31.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.721+0000 7f16ae5cd740 10 client.8730 fetch_fsmap finished waiting for FSMap version 91 2024-09-12T03:23:31.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.721+0000 7f16ae5cd740 10 client.8730 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-12T03:23:31.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.721+0000 7f16ae5cd740 20 client.8730 populate_metadata read hostname 'smithi027' 2024-09-12T03:23:31.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.721+0000 7f16ae5cd740 1 --2- 172.21.15.27:0/267478958 >> [v2:172.21.15.27:6833/1241148443,v1:172.21.15.27:6835/1241148443] conn(0x563ee8e13ca0 0x563ee8e14080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:31.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.721+0000 7f16ae5cd740 4 client.8730 mds_command: new command op to 8562 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:23:31.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.721+0000 7f16ae5cd740 1 -- 172.21.15.27:0/267478958 --> [v2:172.21.15.27:6833/1241148443,v1:172.21.15.27:6835/1241148443] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563ee8d62c50 con 0x563ee8e13ca0 2024-09-12T03:23:31.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.721+0000 7f16a8ced640 1 --2- 172.21.15.27:0/267478958 >> [v2:172.21.15.27:6833/1241148443,v1:172.21.15.27:6835/1241148443] conn(0x563ee8e13ca0 0x563ee8e14080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:31.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.721+0000 7f1683fff640 10 client.8730 ms_handle_connect on v2:172.21.15.27:6833/1241148443 2024-09-12T03:23:31.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.721+0000 7f16a8ced640 1 --2- 172.21.15.27:0/267478958 >> [v2:172.21.15.27:6833/1241148443,v1:172.21.15.27:6835/1241148443] conn(0x563ee8e13ca0 0x563ee8e14080 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a054ab83ac565c5 server_cookie=b1f6a792a90f2b95 in_seq=0 out_seq=0 2024-09-12T03:23:31.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.722+0000 7f1683fff640 1 -- 172.21.15.27:0/267478958 <== mds.0 v2:172.21.15.27:6833/1241148443 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563ee8d62c50 con 0x563ee8e13ca0 2024-09-12T03:23:31.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.722+0000 7f1683fff640 10 client.8730 handle_command_reply: tid=0 2024-09-12T03:23:31.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.722+0000 7f16ae5cd740 1 client.8730 shutdown 2024-09-12T03:23:31.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.722+0000 7f16ae5cd740 1 -- 172.21.15.27:0/267478958 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563ee8df32c0 msgr2=0x563ee8e4b590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:31.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.722+0000 7f16ae5cd740 1 --2- 172.21.15.27:0/267478958 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563ee8df32c0 0x563ee8e4b590 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7f1694015eb0 tx=0x7f16a40d2260 comp rx=0 tx=0).stop 2024-09-12T03:23:31.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.722+0000 7f16ae5cd740 1 -- 172.21.15.27:0/267478958 shutdown_connections 2024-09-12T03:23:31.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.722+0000 7f16ae5cd740 1 --2- 172.21.15.27:0/267478958 >> [v2:172.21.15.27:6833/1241148443,v1:172.21.15.27:6835/1241148443] conn(0x563ee8e13ca0 0x563ee8e14080 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:31.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.722+0000 7f16ae5cd740 1 --2- 172.21.15.27:0/267478958 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x563ee8ded050 0x563ee8e192f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:31.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.722+0000 7f16ae5cd740 1 --2- 172.21.15.27:0/267478958 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563ee8df4a50 0x563ee8e127f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:31.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.722+0000 7f1683fff640 0 client.8730 ms_handle_reset on v2:172.21.15.27:6833/1241148443 2024-09-12T03:23:31.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.722+0000 7f16ae5cd740 1 --2- 172.21.15.27:0/267478958 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563ee8df32c0 0x563ee8e4b590 unknown :-1 s=CLOSED pgs=1567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:31.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.722+0000 7f16ae5cd740 1 -- 172.21.15.27:0/267478958 >> 172.21.15.27:0/267478958 conn(0x563ee8df0a80 msgr2=0x563ee8e3b380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:31.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.722+0000 7f16ae5cd740 1 -- 172.21.15.27:0/267478958 shutdown_connections 2024-09-12T03:23:31.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.722+0000 7f16ae5cd740 1 -- 172.21.15.27:0/267478958 wait complete. 2024-09-12T03:23:31.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.723+0000 7f16ae5cd740 20 client.8730 trim_cache size 0 max 16384 2024-09-12T03:23:31.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.741+0000 7f16ae5cd740 1 Processor -- start 2024-09-12T03:23:31.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.741+0000 7f16ae5cd740 1 -- start start 2024-09-12T03:23:31.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.741+0000 7f16ae5cd740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563ee8ded050 0x563ee8e1d820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:31.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.741+0000 7f16ae5cd740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563ee8df32c0 0x563ee8e39a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:31.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.741+0000 7f16ae5cd740 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x563ee8df4a50 0x563ee8e39fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:31.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.741+0000 7f16ae5cd740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f16a42003c0 con 0x563ee8df32c0 2024-09-12T03:23:31.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.741+0000 7f16ae5cd740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f16a41ff770 con 0x563ee8ded050 2024-09-12T03:23:31.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.741+0000 7f16ae5cd740 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f16a41ffd70 con 0x563ee8df4a50 2024-09-12T03:23:31.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.742+0000 7f16a8ced640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563ee8df32c0 0x563ee8e39a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:31.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.742+0000 7f16a8ced640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563ee8df32c0 0x563ee8e39a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34658/0 (socket says 172.21.15.27:34658) 2024-09-12T03:23:31.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.742+0000 7f16a8ced640 1 -- 172.21.15.27:0/1516874716 learned_addr learned my addr 172.21.15.27:0/1516874716 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:23:31.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.742+0000 7f16a94ee640 1 --2- 172.21.15.27:0/1516874716 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563ee8ded050 0x563ee8e1d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:31.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.742+0000 7f16a9cef640 1 --2- 172.21.15.27:0/1516874716 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x563ee8df4a50 0x563ee8e39fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:31.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.742+0000 7f16a8ced640 1 -- 172.21.15.27:0/1516874716 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x563ee8df4a50 msgr2=0x563ee8e39fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:31.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.742+0000 7f16a8ced640 1 --2- 172.21.15.27:0/1516874716 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x563ee8df4a50 0x563ee8e39fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:31.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.742+0000 7f16a8ced640 1 -- 172.21.15.27:0/1516874716 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563ee8ded050 msgr2=0x563ee8e1d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:31.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.742+0000 7f16a8ced640 1 --2- 172.21.15.27:0/1516874716 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563ee8ded050 0x563ee8e1d820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:31.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.742+0000 7f16a8ced640 1 -- 172.21.15.27:0/1516874716 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563ee8e1c060 con 0x563ee8df32c0 2024-09-12T03:23:31.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.742+0000 7f16a8ced640 1 --2- 172.21.15.27:0/1516874716 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563ee8df32c0 0x563ee8e39a90 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7f1698000c00 tx=0x7f1698011f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fda2d29ba2d9c68 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:31.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.742+0000 7f168a7fc640 1 -- 172.21.15.27:0/1516874716 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1698088070 con 0x563ee8df32c0 2024-09-12T03:23:31.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.742+0000 7f168a7fc640 1 -- 172.21.15.27:0/1516874716 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1698057d70 con 0x563ee8df32c0 2024-09-12T03:23:31.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.742+0000 7f168a7fc640 1 -- 172.21.15.27:0/1516874716 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16980621c0 con 0x563ee8df32c0 2024-09-12T03:23:31.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.743+0000 7f16ae5cd740 1 -- 172.21.15.27:0/1516874716 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563ee8df32c0 msgr2=0x563ee8e39a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:31.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.743+0000 7f16ae5cd740 1 --2- 172.21.15.27:0/1516874716 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563ee8df32c0 0x563ee8e39a90 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7f1698000c00 tx=0x7f1698011f30 comp rx=0 tx=0).stop 2024-09-12T03:23:31.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.743+0000 7f16ae5cd740 1 -- 172.21.15.27:0/1516874716 shutdown_connections 2024-09-12T03:23:31.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.743+0000 7f16ae5cd740 1 --2- 172.21.15.27:0/1516874716 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x563ee8df4a50 0x563ee8e39fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:31.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.743+0000 7f16ae5cd740 1 --2- 172.21.15.27:0/1516874716 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563ee8df32c0 0x563ee8e39a90 unknown :-1 s=CLOSED pgs=1568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:31.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.743+0000 7f16ae5cd740 1 --2- 172.21.15.27:0/1516874716 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563ee8ded050 0x563ee8e1d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:31.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.743+0000 7f16ae5cd740 1 -- 172.21.15.27:0/1516874716 >> 172.21.15.27:0/1516874716 conn(0x563ee8df0a80 msgr2=0x563ee8e13400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:31.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.743+0000 7f16ae5cd740 1 -- 172.21.15.27:0/1516874716 shutdown_connections 2024-09-12T03:23:31.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.743+0000 7f16ae5cd740 1 -- 172.21.15.27:0/1516874716 wait complete. 2024-09-12T03:23:31.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.743+0000 7f16ae5cd740 1 -- start start 2024-09-12T03:23:31.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.743+0000 7f16ae5cd740 1 Processor -- start 2024-09-12T03:23:31.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.744+0000 7f16ae5cd740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563ee8ded050 0x563ee8e111b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:31.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.744+0000 7f16ae5cd740 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x563ee8df32c0 0x563ee8e14970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:31.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.744+0000 7f16ae5cd740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563ee8df4a50 0x563ee8e14eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:31.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.744+0000 7f16ae5cd740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x563ee8ded4d0 con 0x563ee8ded050 2024-09-12T03:23:31.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.744+0000 7f16ae5cd740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x563ee8e12d30 con 0x563ee8df4a50 2024-09-12T03:23:31.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.744+0000 7f16ae5cd740 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x563ee8dee000 con 0x563ee8df32c0 2024-09-12T03:23:31.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.744+0000 7f16ae5cd740 10 client.0 _ll_register_callbacks cb 0x7f1694001a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:23:31.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.744+0000 7f16a94ee640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563ee8ded050 0x563ee8e111b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:31.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.744+0000 7f16a94ee640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563ee8ded050 0x563ee8e111b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34668/0 (socket says 172.21.15.27:34668) 2024-09-12T03:23:31.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.744+0000 7f16a94ee640 1 -- 172.21.15.27:0/1481356418 learned_addr learned my addr 172.21.15.27:0/1481356418 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:23:31.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.745+0000 7f16a8ced640 1 --2- 172.21.15.27:0/1481356418 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x563ee8df32c0 0x563ee8e14970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:31.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.745+0000 7f16a9cef640 1 --2- 172.21.15.27:0/1481356418 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563ee8df4a50 0x563ee8e14eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:31.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.745+0000 7f16a94ee640 1 -- 172.21.15.27:0/1481356418 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x563ee8df32c0 msgr2=0x563ee8e14970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:31.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.745+0000 7f16a94ee640 1 --2- 172.21.15.27:0/1481356418 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x563ee8df32c0 0x563ee8e14970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:31.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.745+0000 7f16a94ee640 1 -- 172.21.15.27:0/1481356418 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563ee8df4a50 msgr2=0x563ee8e14eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:31.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.745+0000 7f16a94ee640 1 --2- 172.21.15.27:0/1481356418 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563ee8df4a50 0x563ee8e14eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:31.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.745+0000 7f16a94ee640 1 -- 172.21.15.27:0/1481356418 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563ee8e19660 con 0x563ee8ded050 2024-09-12T03:23:31.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.745+0000 7f16a94ee640 1 --2- 172.21.15.27:0/1481356418 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563ee8ded050 0x563ee8e111b0 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7f1694007c40 tx=0x7f1694016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=580308f5fe77d9fc server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:31.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.745+0000 7f1683fff640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-12T03:23:31.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.745+0000 7f1683fff640 1 -- 172.21.15.27:0/1481356418 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f169402dd20 con 0x563ee8ded050 2024-09-12T03:23:31.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.745+0000 7f1683fff640 1 -- 172.21.15.27:0/1481356418 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(149..149 src has 1..149) ==== 5765+0+0 (secure 0 0 0) 0x7f1694038b20 con 0x563ee8ded050 2024-09-12T03:23:31.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.745+0000 7f16ae5cd740 1 -- 172.21.15.27:0/1481356418 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1698057d70 con 0x563ee8ded050 2024-09-12T03:23:31.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.745+0000 7f1683fff640 1 -- 172.21.15.27:0/1481356418 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f1694028020 con 0x563ee8ded050 2024-09-12T03:23:31.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.745+0000 7f16ae5cd740 10 client.8736 fetch_fsmap learned FSMap version 91 2024-09-12T03:23:31.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.745+0000 7f16ae5cd740 1 -- 172.21.15.27:0/1481356418 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=91,monmap=2+}) -- 0x563ee8def820 con 0x563ee8ded050 2024-09-12T03:23:31.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.745+0000 7f1683fff640 1 -- 172.21.15.27:0/1481356418 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1698057d70 con 0x563ee8ded050 2024-09-12T03:23:31.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.746+0000 7f1683fff640 1 -- 172.21.15.27:0/1481356418 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 91) ==== 3502+0+0 (secure 0 0 0) 0x7f1694038d20 con 0x563ee8ded050 2024-09-12T03:23:31.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.746+0000 7f16ae5cd740 10 client.8736 fetch_fsmap finished waiting for FSMap version 91 2024-09-12T03:23:31.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.746+0000 7f16ae5cd740 10 client.8736 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-12T03:23:31.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.746+0000 7f16ae5cd740 20 client.8736 populate_metadata read hostname 'smithi027' 2024-09-12T03:23:31.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.746+0000 7f16ae5cd740 1 --2- 172.21.15.27:0/1481356418 >> [v2:172.21.15.27:6833/1241148443,v1:172.21.15.27:6835/1241148443] conn(0x563ee8eb6e00 0x563ee8e0d200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:31.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.746+0000 7f16ae5cd740 4 client.8736 mds_command: new command op to 8562 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-12T03:23:31.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.746+0000 7f16ae5cd740 1 -- 172.21.15.27:0/1481356418 --> [v2:172.21.15.27:6833/1241148443,v1:172.21.15.27:6835/1241148443] -- command(tid 0: {"prefix": "status"}) -- 0x563ee8d5cfa0 con 0x563ee8eb6e00 2024-09-12T03:23:31.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.746+0000 7f16a8ced640 1 --2- 172.21.15.27:0/1481356418 >> [v2:172.21.15.27:6833/1241148443,v1:172.21.15.27:6835/1241148443] conn(0x563ee8eb6e00 0x563ee8e0d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:31.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.746+0000 7f1683fff640 10 client.8736 ms_handle_connect on v2:172.21.15.27:6833/1241148443 2024-09-12T03:23:31.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.746+0000 7f16a8ced640 1 --2- 172.21.15.27:0/1481356418 >> [v2:172.21.15.27:6833/1241148443,v1:172.21.15.27:6835/1241148443] conn(0x563ee8eb6e00 0x563ee8e0d200 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=33643d94a67d8270 server_cookie=15bb09387dc8ebc6 in_seq=0 out_seq=0 2024-09-12T03:23:31.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.746+0000 7f1683fff640 1 -- 172.21.15.27:0/1481356418 <== mds.0 v2:172.21.15.27:6833/1241148443 1 ==== command_reply(tid 0: 0 ) ==== 8+0+321 (crc 0 0 0) 0x563ee8d5cfa0 con 0x563ee8eb6e00 2024-09-12T03:23:31.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.746+0000 7f1683fff640 10 client.8736 handle_command_reply: tid=0 2024-09-12T03:23:31.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.746+0000 7f16ae5cd740 1 client.8736 shutdown 2024-09-12T03:23:31.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.746+0000 7f16ae5cd740 1 -- 172.21.15.27:0/1481356418 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563ee8ded050 msgr2=0x563ee8e111b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:31.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.746+0000 7f16ae5cd740 1 --2- 172.21.15.27:0/1481356418 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563ee8ded050 0x563ee8e111b0 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7f1694007c40 tx=0x7f1694016000 comp rx=0 tx=0).stop 2024-09-12T03:23:31.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.746+0000 7f16ae5cd740 1 -- 172.21.15.27:0/1481356418 shutdown_connections 2024-09-12T03:23:31.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.746+0000 7f16ae5cd740 1 --2- 172.21.15.27:0/1481356418 >> [v2:172.21.15.27:6833/1241148443,v1:172.21.15.27:6835/1241148443] conn(0x563ee8eb6e00 0x563ee8e0d200 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:31.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.746+0000 7f16ae5cd740 1 --2- 172.21.15.27:0/1481356418 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563ee8df4a50 0x563ee8e14eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:31.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.746+0000 7f1683fff640 0 client.8736 ms_handle_reset on v2:172.21.15.27:6833/1241148443 2024-09-12T03:23:31.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.746+0000 7f16ae5cd740 1 --2- 172.21.15.27:0/1481356418 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x563ee8df32c0 0x563ee8e14970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:31.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.746+0000 7f16ae5cd740 1 --2- 172.21.15.27:0/1481356418 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563ee8ded050 0x563ee8e111b0 unknown :-1 s=CLOSED pgs=1569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:31.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.746+0000 7f16ae5cd740 1 -- 172.21.15.27:0/1481356418 >> 172.21.15.27:0/1481356418 conn(0x563ee8df0a80 msgr2=0x563ee8e125a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:31.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.747+0000 7f16ae5cd740 1 -- 172.21.15.27:0/1481356418 shutdown_connections 2024-09-12T03:23:31.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.747+0000 7f16ae5cd740 1 -- 172.21.15.27:0/1481356418 wait complete. 2024-09-12T03:23:31.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.747+0000 7f16ae5cd740 20 client.8736 trim_cache size 0 max 16384 2024-09-12T03:23:31.755 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-12T03:23:31.755 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_fsid": "d2df504e-2647-4385-a5db-9fdf69f487a0", 2024-09-12T03:23:31.755 INFO:teuthology.orchestra.run.smithi027.stdout: "whoami": 0, 2024-09-12T03:23:31.755 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 8562, 2024-09-12T03:23:31.755 INFO:teuthology.orchestra.run.smithi027.stdout: "want_state": "up:active", 2024-09-12T03:23:31.755 INFO:teuthology.orchestra.run.smithi027.stdout: "state": "up:active", 2024-09-12T03:23:31.755 INFO:teuthology.orchestra.run.smithi027.stdout: "fs_name": "cephfs", 2024-09-12T03:23:31.755 INFO:teuthology.orchestra.run.smithi027.stdout: "rank_uptime": 10.057326372, 2024-09-12T03:23:31.755 INFO:teuthology.orchestra.run.smithi027.stdout: "mdsmap_epoch": 91, 2024-09-12T03:23:31.755 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch": 149, 2024-09-12T03:23:31.755 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch_barrier": 149, 2024-09-12T03:23:31.755 INFO:teuthology.orchestra.run.smithi027.stdout: "uptime": 15.088917706 2024-09-12T03:23:31.755 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-12T03:23:31.756 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.747+0000 7f16ab752640 1 -- 172.21.15.27:0/1834011858 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f169006c610 msgr2=0x7f169006ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:31.756 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.747+0000 7f16ab752640 1 --2- 172.21.15.27:0/1834011858 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f169006c610 0x7f169006ead0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f16a40b3750 tx=0x7f1694006040 comp rx=0 tx=0).stop 2024-09-12T03:23:31.756 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.747+0000 7f16ab752640 1 -- 172.21.15.27:0/1834011858 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f16a4161930 msgr2=0x7f16a40b6790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:31.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.747+0000 7f16ab752640 1 --2- 172.21.15.27:0/1834011858 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f16a4161930 0x7f16a40b6790 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7f1698011940 tx=0x7f1698011e00 comp rx=0 tx=0).stop 2024-09-12T03:23:31.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.747+0000 7f16ab752640 1 -- 172.21.15.27:0/1834011858 shutdown_connections 2024-09-12T03:23:31.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.747+0000 7f16ab752640 1 --2- 172.21.15.27:0/1834011858 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f16a415fe80 0x7f16a415af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:31.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.747+0000 7f16ab752640 1 --2- 172.21.15.27:0/1834011858 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f169006c610 0x7f169006ead0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:31.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.747+0000 7f16ab752640 1 --2- 172.21.15.27:0/1834011858 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f16a4161930 0x7f16a40b6790 unknown :-1 s=CLOSED pgs=1565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:31.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.747+0000 7f16ab752640 1 --2- 172.21.15.27:0/1834011858 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f168c004d40 0x7f16a40cb850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:31.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.747+0000 7f16ab752640 1 -- 172.21.15.27:0/1834011858 >> 172.21.15.27:0/1834011858 conn(0x7f16a4073800 msgr2=0x7f16a415d720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:31.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.747+0000 7f16ab752640 1 -- 172.21.15.27:0/1834011858 shutdown_connections 2024-09-12T03:23:31.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.747+0000 7f16ab752640 1 -- 172.21.15.27:0/1834011858 wait complete. 2024-09-12T03:23:31.759 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-12T03:23:31.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.860+0000 7f0c1b6ba640 1 Processor -- start 2024-09-12T03:23:31.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.860+0000 7f0c1b6ba640 1 -- start start 2024-09-12T03:23:31.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.861+0000 7f0c1b6ba640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f0c1415eb90 0x7f0c140d2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:31.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.861+0000 7f0c1b6ba640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f0c140d1720 0x7f0c140b8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:31.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.861+0000 7f0c1b6ba640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0c140d3760 con 0x7f0c140d5200 2024-09-12T03:23:31.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.861+0000 7f0c1b6ba640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f0c140b7d20 con 0x7f0c1415eb90 2024-09-12T03:23:31.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.861+0000 7f0c1b6ba640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f0c140d42e0 con 0x7f0c140d1720 2024-09-12T03:23:31.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.861+0000 7f0c18c55640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f0c140d5200 0x7f0c140d1010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:51254/0 (socket says 172.21.15.27:51254) 2024-09-12T03:23:31.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.861+0000 7f0c18c55640 1 -- 172.21.15.27:0/1116620744 learned_addr learned my addr 172.21.15.27:0/1116620744 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:23:31.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.862+0000 7f0c19c57640 1 --2- 172.21.15.27:0/1116620744 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f0c140d1720 0x7f0c140b8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:31.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.862+0000 7f0c13fff640 1 -- 172.21.15.27:0/1116620744 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2403550564 0 0) 0x7f0c140d3760 con 0x7f0c140d5200 2024-09-12T03:23:31.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.862+0000 7f0c19456640 1 --2- 172.21.15.27:0/1116620744 >> v2:172.21.15.45:3300/0 conn(0x7f0c1415eb90 0x7f0c140d2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:31.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.862+0000 7f0c13fff640 1 -- 172.21.15.27:0/1116620744 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0bfc003520 con 0x7f0c140d5200 2024-09-12T03:23:31.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.862+0000 7f0c13fff640 1 -- 172.21.15.27:0/1116620744 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1632457486 0 0) 0x7f0bfc003520 con 0x7f0c140d5200 2024-09-12T03:23:31.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.862+0000 7f0c13fff640 1 -- 172.21.15.27:0/1116620744 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f0c140d1720 msgr2=0x7f0c140b8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:31.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.862+0000 7f0c13fff640 1 --2- 172.21.15.27:0/1116620744 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f0c140d1720 0x7f0c140b8190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:31.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.862+0000 7f0c13fff640 1 -- 172.21.15.27:0/1116620744 >> v2:172.21.15.45:3300/0 conn(0x7f0c1415eb90 msgr2=0x7f0c140d2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:31.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.862+0000 7f0c13fff640 1 --2- 172.21.15.27:0/1116620744 >> v2:172.21.15.45:3300/0 conn(0x7f0c1415eb90 0x7f0c140d2df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:31.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.862+0000 7f0c13fff640 1 -- 172.21.15.27:0/1116620744 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c140d3a40 con 0x7f0c140d5200 2024-09-12T03:23:31.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.862+0000 7f0c13fff640 1 -- 172.21.15.27:0/1116620744 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 847755213 0 0) 0x7f0c08001670 con 0x7f0c140d5200 2024-09-12T03:23:31.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.862+0000 7f0c13fff640 1 -- 172.21.15.27:0/1116620744 >> v1:172.21.15.27:6789/0 conn(0x7f0c140d5200 legacy=0x7f0c140d1010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:31.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.862+0000 7f0c13fff640 1 --2- 172.21.15.27:0/1116620744 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f0c140d1720 0x7f0bfc0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:31.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.862+0000 7f0c13fff640 1 --2- 172.21.15.27:0/1116620744 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0bfc003f30 0x7f0bfc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:31.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.862+0000 7f0c13fff640 1 --2- 172.21.15.27:0/1116620744 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0bfc004d40 0x7f0bfc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:31.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.862+0000 7f0c13fff640 1 -- 172.21.15.27:0/1116620744 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f0c140d42e0 con 0x7f0bfc004d40 2024-09-12T03:23:31.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.862+0000 7f0c13fff640 1 -- 172.21.15.27:0/1116620744 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f0bfc00b9c0 con 0x7f0bfc003f30 2024-09-12T03:23:31.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.862+0000 7f0c13fff640 1 -- 172.21.15.27:0/1116620744 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f0bfc00bb40 con 0x7f0c140d1720 2024-09-12T03:23:31.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.862+0000 7f0c19c57640 1 --2- 172.21.15.27:0/1116620744 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0bfc004d40 0x7f0bfc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:31.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.862+0000 7f0c19456640 1 --2- 172.21.15.27:0/1116620744 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f0c140d1720 0x7f0bfc0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:31.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.862+0000 7f0c18c55640 1 --2- 172.21.15.27:0/1116620744 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0bfc003f30 0x7f0bfc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:31.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.863+0000 7f0c19c57640 1 -- 172.21.15.27:0/1116620744 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f0c140d1720 msgr2=0x7f0bfc0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:31.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.863+0000 7f0c19c57640 1 --2- 172.21.15.27:0/1116620744 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f0c140d1720 0x7f0bfc0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:31.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.863+0000 7f0c19c57640 1 -- 172.21.15.27:0/1116620744 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0bfc003f30 msgr2=0x7f0bfc004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:31.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.863+0000 7f0c19c57640 1 --2- 172.21.15.27:0/1116620744 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0bfc003f30 0x7f0bfc004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:31.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.863+0000 7f0c19c57640 1 -- 172.21.15.27:0/1116620744 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bfc003520 con 0x7f0bfc004d40 2024-09-12T03:23:31.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.863+0000 7f0c18c55640 1 --2- 172.21.15.27:0/1116620744 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0bfc003f30 0x7f0bfc004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:23:31.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.863+0000 7f0c19c57640 1 --2- 172.21.15.27:0/1116620744 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0bfc004d40 0x7f0bfc0051c0 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7f0c04009fc0 tx=0x7f0bfc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f463714f87e19fd server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:31.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.863+0000 7f0c13fff640 1 -- 172.21.15.27:0/1116620744 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c04009c10 con 0x7f0bfc004d40 2024-09-12T03:23:31.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.863+0000 7f0c13fff640 1 -- 172.21.15.27:0/1116620744 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c04036070 con 0x7f0bfc004d40 2024-09-12T03:23:31.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.863+0000 7f0c13fff640 1 -- 172.21.15.27:0/1116620744 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c04031670 con 0x7f0bfc004d40 2024-09-12T03:23:31.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.864+0000 7f0c1b6ba640 1 -- 172.21.15.27:0/1116620744 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0bfc004d40 msgr2=0x7f0bfc0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:31.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.864+0000 7f0c1b6ba640 1 --2- 172.21.15.27:0/1116620744 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0bfc004d40 0x7f0bfc0051c0 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7f0c04009fc0 tx=0x7f0bfc005830 comp rx=0 tx=0).stop 2024-09-12T03:23:31.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.864+0000 7f0c19c57640 1 -- 172.21.15.27:0/1116620744 reap_dead start 2024-09-12T03:23:31.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.864+0000 7f0c1b6ba640 1 -- 172.21.15.27:0/1116620744 shutdown_connections 2024-09-12T03:23:31.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.864+0000 7f0c1b6ba640 1 -- 172.21.15.27:0/1116620744 >> 172.21.15.27:0/1116620744 conn(0x7f0c1406dce0 msgr2=0x7f0c1406e0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:31.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.864+0000 7f0c1b6ba640 1 -- 172.21.15.27:0/1116620744 shutdown_connections 2024-09-12T03:23:31.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.864+0000 7f0c1b6ba640 1 -- 172.21.15.27:0/1116620744 wait complete. 2024-09-12T03:23:31.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.865+0000 7f0c1b6ba640 1 Processor -- start 2024-09-12T03:23:31.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.865+0000 7f0c1b6ba640 1 -- start start 2024-09-12T03:23:31.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.866+0000 7f0c1b6ba640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0c140c91c0 0x7f0c140cb8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:31.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.866+0000 7f0c1b6ba640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0c140cb1e0 0x7f0c14158b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:31.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.866+0000 7f0c1b6ba640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f0c14159080 0x7f0c14159500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:31.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.866+0000 7f0c1b6ba640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f0c140d3330 con 0x7f0c140c91c0 2024-09-12T03:23:31.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.866+0000 7f0c1b6ba640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f0c140d4000 con 0x7f0c140cb1e0 2024-09-12T03:23:31.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.866+0000 7f0c1b6ba640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f0c140d34b0 con 0x7f0c14159080 2024-09-12T03:23:31.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.866+0000 7f0c19456640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0c140c91c0 0x7f0c140cb8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:31.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.866+0000 7f0c19456640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0c140c91c0 0x7f0c140cb8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34688/0 (socket says 172.21.15.27:34688) 2024-09-12T03:23:31.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.866+0000 7f0c19456640 1 -- 172.21.15.27:0/3759257018 learned_addr learned my addr 172.21.15.27:0/3759257018 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:23:31.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.866+0000 7f0c19c57640 1 --2- 172.21.15.27:0/3759257018 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f0c14159080 0x7f0c14159500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:31.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.866+0000 7f0c18c55640 1 --2- 172.21.15.27:0/3759257018 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0c140cb1e0 0x7f0c14158b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:31.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.867+0000 7f0c19456640 1 -- 172.21.15.27:0/3759257018 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f0c14159080 msgr2=0x7f0c14159500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:31.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.867+0000 7f0c19456640 1 --2- 172.21.15.27:0/3759257018 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f0c14159080 0x7f0c14159500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:31.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.867+0000 7f0c19456640 1 -- 172.21.15.27:0/3759257018 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0c140cb1e0 msgr2=0x7f0c14158b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:31.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.867+0000 7f0c19456640 1 --2- 172.21.15.27:0/3759257018 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0c140cb1e0 0x7f0c14158b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:31.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.867+0000 7f0c19456640 1 -- 172.21.15.27:0/3759257018 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c14156590 con 0x7f0c140c91c0 2024-09-12T03:23:31.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.867+0000 7f0c19456640 1 --2- 172.21.15.27:0/3759257018 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0c140c91c0 0x7f0c140cb8e0 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7f0c0c011ad0 tx=0x7f0c0c011f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:31.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.867+0000 7f0c11ffb640 1 -- 172.21.15.27:0/3759257018 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c0c00fb50 con 0x7f0c140c91c0 2024-09-12T03:23:31.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.867+0000 7f0c11ffb640 1 -- 172.21.15.27:0/3759257018 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c0c00fcf0 con 0x7f0c140c91c0 2024-09-12T03:23:31.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.867+0000 7f0c11ffb640 1 -- 172.21.15.27:0/3759257018 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c0c00fea0 con 0x7f0c140c91c0 2024-09-12T03:23:31.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.867+0000 7f0c1b6ba640 1 -- 172.21.15.27:0/3759257018 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c14156830 con 0x7f0c140c91c0 2024-09-12T03:23:31.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.867+0000 7f0c1b6ba640 1 -- 172.21.15.27:0/3759257018 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c140c8cb0 con 0x7f0c140c91c0 2024-09-12T03:23:31.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.868+0000 7f0c1b6ba640 1 -- 172.21.15.27:0/3759257018 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c140cb5c0 con 0x7f0c140c91c0 2024-09-12T03:23:31.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.869+0000 7f0c11ffb640 1 -- 172.21.15.27:0/3759257018 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 24) ==== 91212+0+0 (secure 0 0 0) 0x7f0c0c012040 con 0x7f0c140c91c0 2024-09-12T03:23:31.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.869+0000 7f0c11ffb640 1 --2- 172.21.15.27:0/3759257018 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f0c0006c640 0x7f0c0006eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:31.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.869+0000 7f0c11ffb640 1 -- 172.21.15.27:0/3759257018 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 5765+0+0 (secure 0 0 0) 0x7f0c0c08c640 con 0x7f0c140c91c0 2024-09-12T03:23:31.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.869+0000 7f0c18c55640 1 --2- 172.21.15.27:0/3759257018 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f0c0006c640 0x7f0c0006eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:31.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.869+0000 7f0c18c55640 1 --2- 172.21.15.27:0/3759257018 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f0c0006c640 0x7f0c0006eb00 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f0c08004840 tx=0x7f0c08009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:31.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:31.873+0000 7f0c11ffb640 1 -- 172.21.15.27:0/3759257018 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f0c0c019030 con 0x7f0c140c91c0 2024-09-12T03:23:32.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:32.084+0000 7f0c1b6ba640 1 -- 172.21.15.27:0/3759257018 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f0c1415f000 con 0x7f0c140c91c0 2024-09-12T03:23:32.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:32.084+0000 7f0c11ffb640 1 -- 172.21.15.27:0/3759257018 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v91) ==== 53+0+73 (secure 0 0 0) 0x7f0c0c05a0d0 con 0x7f0c140c91c0 2024-09-12T03:23:32.085 INFO:teuthology.orchestra.run.smithi027.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-12T03:23:32.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:32.087+0000 7f0c1b6ba640 1 -- 172.21.15.27:0/3759257018 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f0c0006c640 msgr2=0x7f0c0006eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:32.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:32.087+0000 7f0c1b6ba640 1 --2- 172.21.15.27:0/3759257018 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f0c0006c640 0x7f0c0006eb00 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f0c08004840 tx=0x7f0c08009040 comp rx=0 tx=0).stop 2024-09-12T03:23:32.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:32.087+0000 7f0c1b6ba640 1 -- 172.21.15.27:0/3759257018 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0c140c91c0 msgr2=0x7f0c140cb8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:32.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:32.087+0000 7f0c1b6ba640 1 --2- 172.21.15.27:0/3759257018 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0c140c91c0 0x7f0c140cb8e0 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7f0c0c011ad0 tx=0x7f0c0c011f90 comp rx=0 tx=0).stop 2024-09-12T03:23:32.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:32.087+0000 7f0c1b6ba640 1 -- 172.21.15.27:0/3759257018 shutdown_connections 2024-09-12T03:23:32.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:32.087+0000 7f0c1b6ba640 1 --2- 172.21.15.27:0/3759257018 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f0c14159080 0x7f0c14159500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:32.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:32.087+0000 7f0c1b6ba640 1 --2- 172.21.15.27:0/3759257018 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0c140cb1e0 0x7f0c14158b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:32.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:32.087+0000 7f0c1b6ba640 1 --2- 172.21.15.27:0/3759257018 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f0c0006c640 0x7f0c0006eb00 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:32.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:32.087+0000 7f0c1b6ba640 1 --2- 172.21.15.27:0/3759257018 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0c140c91c0 0x7f0c140cb8e0 unknown :-1 s=CLOSED pgs=1572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:32.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:32.087+0000 7f0c1b6ba640 1 -- 172.21.15.27:0/3759257018 >> 172.21.15.27:0/3759257018 conn(0x7f0c1406dce0 msgr2=0x7f0c140cee40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:32.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:32.087+0000 7f0c1b6ba640 1 -- 172.21.15.27:0/3759257018 shutdown_connections 2024-09-12T03:23:32.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:32.087+0000 7f0c1b6ba640 1 -- 172.21.15.27:0/3759257018 wait complete. 2024-09-12T03:23:32.098 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-12T03:23:32.098 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-12T03:23:32.098 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi027.front.sepia.ceph.com 2024-09-12T03:23:32.098 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-12T03:23:32.099 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-12T03:23:32.099 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-12T03:23:32.099 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-12T03:23:32.099 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-12T03:23:32.099 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-12T03:23:32.099 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:23:32.099 DEBUG:teuthology.orchestra.run.smithi027:> ip addr 2024-09-12T03:23:32.115 INFO:teuthology.orchestra.run.smithi027.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-12T03:23:32.115 INFO:teuthology.orchestra.run.smithi027.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-12T03:23:32.115 INFO:teuthology.orchestra.run.smithi027.stdout: inet 127.0.0.1/8 scope host lo 2024-09-12T03:23:32.115 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:23:32.115 INFO:teuthology.orchestra.run.smithi027.stdout: inet6 ::1/128 scope host 2024-09-12T03:23:32.116 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:23:32.116 INFO:teuthology.orchestra.run.smithi027.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-12T03:23:32.116 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether 0c:c4:7a:6b:fe:c8 brd ff:ff:ff:ff:ff:ff 2024-09-12T03:23:32.116 INFO:teuthology.orchestra.run.smithi027.stdout: altname enp7s0f0 2024-09-12T03:23:32.116 INFO:teuthology.orchestra.run.smithi027.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-12T03:23:32.116 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether 0c:c4:7a:6b:fe:c9 brd ff:ff:ff:ff:ff:ff 2024-09-12T03:23:32.116 INFO:teuthology.orchestra.run.smithi027.stdout: altname enp7s0f1 2024-09-12T03:23:32.116 INFO:teuthology.orchestra.run.smithi027.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-12T03:23:32.116 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether ac:1f:6b:f8:39:5a brd ff:ff:ff:ff:ff:ff 2024-09-12T03:23:32.116 INFO:teuthology.orchestra.run.smithi027.stdout: altname enp1s0f0 2024-09-12T03:23:32.116 INFO:teuthology.orchestra.run.smithi027.stdout: inet 172.21.15.27/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-12T03:23:32.116 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft 42377sec preferred_lft 42377sec 2024-09-12T03:23:32.117 INFO:teuthology.orchestra.run.smithi027.stdout: inet6 fe80::ae1f:6bff:fef8:395a/64 scope link 2024-09-12T03:23:32.117 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:23:32.117 INFO:teuthology.orchestra.run.smithi027.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-12T03:23:32.117 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether ac:1f:6b:f8:39:5b brd ff:ff:ff:ff:ff:ff 2024-09-12T03:23:32.117 INFO:teuthology.orchestra.run.smithi027.stdout: altname enp1s0f1 2024-09-12T03:23:32.117 INFO:teuthology.orchestra.run.smithi027.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-12T03:23:32.117 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether 2e:7f:89:6c:33:a0 brd ff:ff:ff:ff:ff:ff 2024-09-12T03:23:32.117 INFO:teuthology.orchestra.run.smithi027.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-12T03:23:32.117 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:23:32.117 INFO:teuthology.orchestra.run.smithi027.stdout: inet6 fe80::c84b:19ff:fef0:ca74/64 scope link 2024-09-12T03:23:32.117 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:23:32.117 INFO:teuthology.orchestra.run.smithi027.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-12T03:23:32.117 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether 42:40:d7:5c:7a:8e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-12T03:23:32.118 INFO:teuthology.orchestra.run.smithi027.stdout: inet6 fe80::4040:d7ff:fe5c:7a8e/64 scope link 2024-09-12T03:23:32.118 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:23:32.118 INFO:teuthology.orchestra.run.smithi027.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-12T03:23:32.118 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether 2e:7f:89:6c:33:a0 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-12T03:23:32.118 INFO:teuthology.orchestra.run.smithi027.stdout: inet6 fe80::2c7f:89ff:fe6c:33a0/64 scope link 2024-09-12T03:23:32.118 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:23:32.118 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:23:32.118 DEBUG:teuthology.orchestra.run.smithi027:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-12T03:23:32.180 INFO:teuthology.orchestra.run.smithi027.stdout:1 2024-09-12T03:23:32.181 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:23:32.181 DEBUG:teuthology.orchestra.run.smithi027:> route 2024-09-12T03:23:32.238 INFO:teuthology.orchestra.run.smithi027.stdout:Kernel IP routing table 2024-09-12T03:23:32.238 INFO:teuthology.orchestra.run.smithi027.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-12T03:23:32.238 INFO:teuthology.orchestra.run.smithi027.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-12T03:23:32.238 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-12T03:23:32.238 INFO:teuthology.orchestra.run.smithi027.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-12T03:23:32.239 DEBUG:teuthology.orchestra.run.smithi027:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-12T03:23:32.239 DEBUG:teuthology.orchestra.run.smithi027:> set -e 2024-09-12T03:23:32.239 DEBUG:teuthology.orchestra.run.smithi027:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-12T03:23:32.239 DEBUG:teuthology.orchestra.run.smithi027:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-12T03:23:32.239 DEBUG:teuthology.orchestra.run.smithi027:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-12T03:23:32.239 DEBUG:teuthology.orchestra.run.smithi027:> ') 2024-09-12T03:23:32.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:32 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T03:23:32.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T03:23:32.368 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:23:32.368 DEBUG:teuthology.orchestra.run.smithi027:> ip netns list 2024-09-12T03:23:32.383 INFO:teuthology.orchestra.run.smithi027.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-12T03:23:32.384 INFO:teuthology.orchestra.run.smithi027.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-12T03:23:32.384 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-12T03:23:32.384 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:23:32.384 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip link set brx.0 up 2024-09-12T03:23:32.449 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-12T03:23:32.450 DEBUG:teuthology.orchestra.run.smithi027:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-12T03:23:32.505 INFO:teuthology.orchestra.run.smithi027.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-12T03:23:32.506 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-12T03:23:32.506 DEBUG:teuthology.orchestra.run.smithi027:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-12T03:23:32.564 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-09-12T03:23:32.564 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:23:32.564 DEBUG:teuthology.orchestra.run.smithi027:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-12T03:23:32.645 INFO:teuthology.orchestra.run.smithi027.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-12T03:23:32.645 INFO:teuthology.orchestra.run.smithi027.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-12T03:23:32.653 INFO:teuthology.orchestra.run.smithi027.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-12T03:23:32.653 INFO:teuthology.orchestra.run.smithi027.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-12T03:23:32.653 INFO:teuthology.orchestra.run.smithi027.stdout:mount.ceph: resolved to: "172.21.15.27:3300,172.21.15.45:3300,172.21.15.45:3301" 2024-09-12T03:23:32.653 INFO:teuthology.orchestra.run.smithi027.stdout:mount.ceph: trying mount with new device syntax: 0@d2df504e-2647-4385-a5db-9fdf69f487a0.cephfs=/ 2024-09-12T03:23:32.654 INFO:teuthology.orchestra.run.smithi027.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.27:3300/172.21.15.45:3300/172.21.15.45:3301" will pass to kernel 2024-09-12T03:23:32.654 INFO:teuthology.orchestra.run.smithi027.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-12T03:23:32.654 INFO:teuthology.orchestra.run.smithi027.stdout: You just mounted a file system that supports labels which does not 2024-09-12T03:23:32.654 INFO:teuthology.orchestra.run.smithi027.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-12T03:23:32.654 INFO:teuthology.orchestra.run.smithi027.stdout: applications will generate AVC messages and not be allowed access to 2024-09-12T03:23:32.654 INFO:teuthology.orchestra.run.smithi027.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-12T03:23:32.656 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-12T03:23:32.656 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:23:32.656 DEBUG:teuthology.orchestra.run.smithi027:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-12T03:23:32.682 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-12T03:23:32.774 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:32 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-12T03:23:32.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T03:23:32.785 INFO:teuthology.orchestra.run.smithi027.stdout:client7885 2024-09-12T03:23:32.786 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-12T03:23:32.786 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T03:23:32.803 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-12T03:23:32.945 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:32.945+0000 7f93e7798640 1 Processor -- start 2024-09-12T03:23:32.945 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:32.945+0000 7f93e7798640 1 -- start start 2024-09-12T03:23:32.947 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:32.946+0000 7f93e7798640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f93e00d5200 0x7f93e00d1010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:32.947 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:32.946+0000 7f93e7798640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f93e00d3760 con 0x7f93e00d1550 2024-09-12T03:23:32.947 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:32.946+0000 7f93e7798640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f93e00d3a40 con 0x7f93e015eb90 2024-09-12T03:23:32.947 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:32.946+0000 7f93e7798640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f93e00b7d20 con 0x7f93e00d5200 2024-09-12T03:23:32.947 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:32.946+0000 7f93e5d35640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f93e00d1550 0x7f93e00b8190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:51270/0 (socket says 172.21.15.27:51270) 2024-09-12T03:23:32.947 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:32.946+0000 7f93e5d35640 1 -- 172.21.15.27:0/3476642561 learned_addr learned my addr 172.21.15.27:0/3476642561 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:23:32.947 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:32.947+0000 7f93e4d33640 1 --2- 172.21.15.27:0/3476642561 >> v2:172.21.15.45:3300/0 conn(0x7f93e00d5200 0x7f93e00d1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:32.947 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:32.947+0000 7f93dffff640 1 -- 172.21.15.27:0/3476642561 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1769410359 0 0) 0x7f93e00d3760 con 0x7f93e00d1550 2024-09-12T03:23:32.947 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:32.947+0000 7f93dffff640 1 -- 172.21.15.27:0/3476642561 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f93c8003520 con 0x7f93e00d1550 2024-09-12T03:23:32.947 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:32.947+0000 7f93dffff640 1 -- 172.21.15.27:0/3476642561 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3191565910 0 0) 0x7f93c8003520 con 0x7f93e00d1550 2024-09-12T03:23:32.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:32.947+0000 7f93e4d33640 1 -- 172.21.15.27:0/3476642561 >> v1:172.21.15.45:6789/0 conn(0x7f93e015eb90 legacy=0x7f93e00d2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:32.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:32.947+0000 7f93e4d33640 1 -- 172.21.15.27:0/3476642561 >> v1:172.21.15.27:6789/0 conn(0x7f93e00d1550 legacy=0x7f93e00b8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:32.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:32.947+0000 7f93e4d33640 1 -- 172.21.15.27:0/3476642561 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93e00d3d20 con 0x7f93e00d5200 2024-09-12T03:23:32.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:32.948+0000 7f93e4d33640 1 --2- 172.21.15.27:0/3476642561 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f93e00d5200 0x7f93e00d1010 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f93d40097a0 tx=0x7f93d402a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4cd32bf9c48cb805 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:32.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:32.948+0000 7f93dffff640 1 -- 172.21.15.27:0/3476642561 <== mon.1 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93d4037070 con 0x7f93e00d5200 2024-09-12T03:23:32.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:32.948+0000 7f93dffff640 1 -- 172.21.15.27:0/3476642561 <== mon.1 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93d402ac50 con 0x7f93e00d5200 2024-09-12T03:23:32.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:32.948+0000 7f93dffff640 1 -- 172.21.15.27:0/3476642561 <== mon.1 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93d00019f0 con 0x7f93e00d5200 2024-09-12T03:23:32.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:32.948+0000 7f93e7798640 1 -- 172.21.15.27:0/3476642561 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f93e00d5200 msgr2=0x7f93e00d1010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:32.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:32.948+0000 7f93e7798640 1 --2- 172.21.15.27:0/3476642561 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f93e00d5200 0x7f93e00d1010 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f93d40097a0 tx=0x7f93d402a1c0 comp rx=0 tx=0).stop 2024-09-12T03:23:32.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:32.949+0000 7f93e7798640 1 -- 172.21.15.27:0/3476642561 shutdown_connections 2024-09-12T03:23:32.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:32.949+0000 7f93e7798640 1 --2- 172.21.15.27:0/3476642561 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f93e00d5200 0x7f93e00d1010 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:32.950 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:32.949+0000 7f93e7798640 1 -- 172.21.15.27:0/3476642561 >> 172.21.15.27:0/3476642561 conn(0x7f93e006dce0 msgr2=0x7f93e006e0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:32.950 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:32.949+0000 7f93e7798640 1 -- 172.21.15.27:0/3476642561 shutdown_connections 2024-09-12T03:23:32.950 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:32.949+0000 7f93e7798640 1 -- 172.21.15.27:0/3476642561 wait complete. 2024-09-12T03:23:32.952 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:32.950+0000 7f93e7798640 1 Processor -- start 2024-09-12T03:23:32.952 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:32.950+0000 7f93e7798640 1 -- start start 2024-09-12T03:23:32.952 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:32.950+0000 7f93e7798640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f93e00d1550 0x7f93e014ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:32.952 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:32.951+0000 7f93e7798640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f93e00d5200 0x7f93e015b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:32.952 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:32.951+0000 7f93e7798640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f93e015eb90 0x7f93e00ca340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:32.952 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:32.951+0000 7f93e7798640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f93e00d3500 con 0x7f93e00d5200 2024-09-12T03:23:32.952 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:32.951+0000 7f93e7798640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f93e00d42e0 con 0x7f93e015eb90 2024-09-12T03:23:32.952 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:32.951+0000 7f93e7798640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f93e00d1960 con 0x7f93e00d1550 2024-09-12T03:23:32.952 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:32.951+0000 7f93e4d33640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f93e00d5200 0x7f93e015b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:32.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:32.951+0000 7f93e4d33640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f93e00d5200 0x7f93e015b250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34690/0 (socket says 172.21.15.27:34690) 2024-09-12T03:23:32.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:32.951+0000 7f93e4d33640 1 -- 172.21.15.27:0/4139786967 learned_addr learned my addr 172.21.15.27:0/4139786967 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:23:32.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:32.951+0000 7f93e5d35640 1 --2- 172.21.15.27:0/4139786967 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f93e015eb90 0x7f93e00ca340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:32.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:32.951+0000 7f93e5534640 1 --2- 172.21.15.27:0/4139786967 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f93e00d1550 0x7f93e014ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:32.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:32.951+0000 7f93e4d33640 1 -- 172.21.15.27:0/4139786967 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f93e00d1550 msgr2=0x7f93e014ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:32.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:32.951+0000 7f93e4d33640 1 --2- 172.21.15.27:0/4139786967 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f93e00d1550 0x7f93e014ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:32.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:32.951+0000 7f93e4d33640 1 -- 172.21.15.27:0/4139786967 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f93e015eb90 msgr2=0x7f93e00ca340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:32.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:32.951+0000 7f93e4d33640 1 --2- 172.21.15.27:0/4139786967 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f93e015eb90 0x7f93e00ca340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:32.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:32.951+0000 7f93e4d33640 1 -- 172.21.15.27:0/4139786967 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93e014d200 con 0x7f93e00d5200 2024-09-12T03:23:32.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:32.951+0000 7f93e5d35640 1 --2- 172.21.15.27:0/4139786967 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f93e015eb90 0x7f93e00ca340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:23:32.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:32.951+0000 7f93e4d33640 1 --2- 172.21.15.27:0/4139786967 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f93e00d5200 0x7f93e015b250 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7f93d4035040 tx=0x7f93d40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:32.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:32.951+0000 7f93ddffb640 1 -- 172.21.15.27:0/4139786967 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93d4041020 con 0x7f93e00d5200 2024-09-12T03:23:32.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:32.951+0000 7f93ddffb640 1 -- 172.21.15.27:0/4139786967 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93d40044c0 con 0x7f93e00d5200 2024-09-12T03:23:32.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:32.951+0000 7f93e7798640 1 -- 172.21.15.27:0/4139786967 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93e00c8ef0 con 0x7f93e00d5200 2024-09-12T03:23:32.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:32.951+0000 7f93ddffb640 1 -- 172.21.15.27:0/4139786967 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93d4037040 con 0x7f93e00d5200 2024-09-12T03:23:32.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:32.952+0000 7f93e7798640 1 -- 172.21.15.27:0/4139786967 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93e00c94b0 con 0x7f93e00d5200 2024-09-12T03:23:32.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:32.953+0000 7f93e7798640 1 -- 172.21.15.27:0/4139786967 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93e00693d0 con 0x7f93e00d5200 2024-09-12T03:23:32.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:32.953+0000 7f93ddffb640 1 -- 172.21.15.27:0/4139786967 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 24) ==== 91212+0+0 (secure 0 0 0) 0x7f93d4044020 con 0x7f93e00d5200 2024-09-12T03:23:32.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:32.954+0000 7f93ddffb640 1 --2- 172.21.15.27:0/4139786967 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f93cc06c610 0x7f93cc06ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:32.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:32.954+0000 7f93ddffb640 1 -- 172.21.15.27:0/4139786967 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 5765+0+0 (secure 0 0 0) 0x7f93d40a9970 con 0x7f93e00d5200 2024-09-12T03:23:32.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:32.954+0000 7f93e5534640 1 --2- 172.21.15.27:0/4139786967 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f93cc06c610 0x7f93cc06ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:32.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:32.954+0000 7f93e5534640 1 --2- 172.21.15.27:0/4139786967 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f93cc06c610 0x7f93cc06ead0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f93d8000f60 tx=0x7f93d8009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:32.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:32.958+0000 7f93ddffb640 1 -- 172.21.15.27:0/4139786967 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f93d404b070 con 0x7f93e00d5200 2024-09-12T03:23:33.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:33.168+0000 7f93e7798640 1 -- 172.21.15.27:0/4139786967 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f93e015be40 con 0x7f93e00d5200 2024-09-12T03:23:33.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:33.168+0000 7f93ddffb640 1 -- 172.21.15.27:0/4139786967 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v91) ==== 53+0+73 (secure 0 0 0) 0x7f93e014d200 con 0x7f93e00d5200 2024-09-12T03:23:33.169 INFO:teuthology.orchestra.run.smithi027.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-12T03:23:33.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:33.170+0000 7f93e7798640 1 -- 172.21.15.27:0/4139786967 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f93cc06c610 msgr2=0x7f93cc06ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:33.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:33.170+0000 7f93e7798640 1 --2- 172.21.15.27:0/4139786967 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f93cc06c610 0x7f93cc06ead0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f93d8000f60 tx=0x7f93d8009040 comp rx=0 tx=0).stop 2024-09-12T03:23:33.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:33.170+0000 7f93e7798640 1 -- 172.21.15.27:0/4139786967 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f93e00d5200 msgr2=0x7f93e015b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:33.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:33.170+0000 7f93e7798640 1 --2- 172.21.15.27:0/4139786967 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f93e00d5200 0x7f93e015b250 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7f93d4035040 tx=0x7f93d40046c0 comp rx=0 tx=0).stop 2024-09-12T03:23:33.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:33.170+0000 7f93e7798640 1 -- 172.21.15.27:0/4139786967 shutdown_connections 2024-09-12T03:23:33.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:33.170+0000 7f93e7798640 1 --2- 172.21.15.27:0/4139786967 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f93e015eb90 0x7f93e00ca340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:33.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:33.170+0000 7f93e7798640 1 --2- 172.21.15.27:0/4139786967 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f93cc06c610 0x7f93cc06ead0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:33.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:33.170+0000 7f93e7798640 1 --2- 172.21.15.27:0/4139786967 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f93e00d5200 0x7f93e015b250 unknown :-1 s=CLOSED pgs=1574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:33.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:33.170+0000 7f93e7798640 1 --2- 172.21.15.27:0/4139786967 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f93e00d1550 0x7f93e014ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:33.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:33.170+0000 7f93e7798640 1 -- 172.21.15.27:0/4139786967 >> 172.21.15.27:0/4139786967 conn(0x7f93e006dce0 msgr2=0x7f93e015f340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:33.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:33.170+0000 7f93e7798640 1 -- 172.21.15.27:0/4139786967 shutdown_connections 2024-09-12T03:23:33.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:33.171+0000 7f93e7798640 1 -- 172.21.15.27:0/4139786967 wait complete. 2024-09-12T03:23:33.181 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-12T03:23:33.181 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-12T03:23:33.181 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi045.front.sepia.ceph.com 2024-09-12T03:23:33.181 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-12T03:23:33.181 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-12T03:23:33.181 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-12T03:23:33.181 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-12T03:23:33.182 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-12T03:23:33.182 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-12T03:23:33.182 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:23:33.182 DEBUG:teuthology.orchestra.run.smithi045:> ip addr 2024-09-12T03:23:33.202 INFO:teuthology.orchestra.run.smithi045.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-12T03:23:33.202 INFO:teuthology.orchestra.run.smithi045.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-12T03:23:33.202 INFO:teuthology.orchestra.run.smithi045.stdout: inet 127.0.0.1/8 scope host lo 2024-09-12T03:23:33.202 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:23:33.202 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 ::1/128 scope host 2024-09-12T03:23:33.203 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:23:33.203 INFO:teuthology.orchestra.run.smithi045.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-12T03:23:33.203 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:6b:fd:a6 brd ff:ff:ff:ff:ff:ff 2024-09-12T03:23:33.203 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp7s0f0 2024-09-12T03:23:33.203 INFO:teuthology.orchestra.run.smithi045.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-12T03:23:33.203 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:6b:fd:a7 brd ff:ff:ff:ff:ff:ff 2024-09-12T03:23:33.203 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp7s0f1 2024-09-12T03:23:33.203 INFO:teuthology.orchestra.run.smithi045.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-12T03:23:33.203 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:bd:12:be brd ff:ff:ff:ff:ff:ff 2024-09-12T03:23:33.203 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp1s0f0 2024-09-12T03:23:33.203 INFO:teuthology.orchestra.run.smithi045.stdout: inet 172.21.15.45/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-12T03:23:33.203 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft 42320sec preferred_lft 42320sec 2024-09-12T03:23:33.203 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::ec4:7aff:febd:12be/64 scope link 2024-09-12T03:23:33.204 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:23:33.204 INFO:teuthology.orchestra.run.smithi045.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-12T03:23:33.204 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:bd:12:bf brd ff:ff:ff:ff:ff:ff 2024-09-12T03:23:33.204 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp1s0f1 2024-09-12T03:23:33.204 INFO:teuthology.orchestra.run.smithi045.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-12T03:23:33.204 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 52:97:33:5c:60:1b brd ff:ff:ff:ff:ff:ff 2024-09-12T03:23:33.204 INFO:teuthology.orchestra.run.smithi045.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-12T03:23:33.204 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:23:33.204 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::d8c2:5dff:fe26:78c8/64 scope link 2024-09-12T03:23:33.204 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:23:33.204 INFO:teuthology.orchestra.run.smithi045.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-12T03:23:33.204 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 5a:ec:d7:40:ef:75 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-12T03:23:33.204 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::58ec:d7ff:fe40:ef75/64 scope link 2024-09-12T03:23:33.205 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:23:33.205 INFO:teuthology.orchestra.run.smithi045.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-12T03:23:33.205 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 52:97:33:5c:60:1b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-12T03:23:33.205 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::5097:33ff:fe5c:601b/64 scope link 2024-09-12T03:23:33.205 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:23:33.205 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:23:33.205 DEBUG:teuthology.orchestra.run.smithi045:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-12T03:23:33.273 INFO:teuthology.orchestra.run.smithi045.stdout:1 2024-09-12T03:23:33.274 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:23:33.275 DEBUG:teuthology.orchestra.run.smithi045:> route 2024-09-12T03:23:33.334 INFO:teuthology.orchestra.run.smithi045.stdout:Kernel IP routing table 2024-09-12T03:23:33.335 INFO:teuthology.orchestra.run.smithi045.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-12T03:23:33.335 INFO:teuthology.orchestra.run.smithi045.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-12T03:23:33.335 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-12T03:23:33.335 INFO:teuthology.orchestra.run.smithi045.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-12T03:23:33.335 DEBUG:teuthology.orchestra.run.smithi045:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-12T03:23:33.335 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-09-12T03:23:33.335 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-12T03:23:33.335 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-12T03:23:33.335 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-12T03:23:33.336 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-09-12T03:23:33.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:23:33 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T03:23:33.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:23:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T03:23:33.473 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:23:33.473 DEBUG:teuthology.orchestra.run.smithi045:> ip netns list 2024-09-12T03:23:33.489 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-09-12T03:23:33.489 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-12T03:23:33.490 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-12T03:23:33.490 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:23:33.490 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link set brx.0 up 2024-09-12T03:23:33.556 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-12T03:23:33.557 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-12T03:23:33.613 INFO:teuthology.orchestra.run.smithi045.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-12T03:23:33.613 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-12T03:23:33.613 DEBUG:teuthology.orchestra.run.smithi045:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-12T03:23:33.669 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2024-09-12T03:23:33.669 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:23:33.670 DEBUG:teuthology.orchestra.run.smithi045:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-12T03:23:33.755 INFO:teuthology.orchestra.run.smithi045.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-12T03:23:33.755 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-12T03:23:33.765 INFO:teuthology.orchestra.run.smithi045.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-12T03:23:33.765 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-12T03:23:33.765 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: resolved to: "172.21.15.27:3300,172.21.15.45:3300,172.21.15.45:3301" 2024-09-12T03:23:33.765 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: trying mount with new device syntax: 1@d2df504e-2647-4385-a5db-9fdf69f487a0.cephfs=/ 2024-09-12T03:23:33.765 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.27:3300/172.21.15.45:3300/172.21.15.45:3301" will pass to kernel 2024-09-12T03:23:33.766 INFO:teuthology.orchestra.run.smithi045.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-09-12T03:23:33.766 INFO:teuthology.orchestra.run.smithi045.stdout: You just mounted a file system that supports labels which does not 2024-09-12T03:23:33.766 INFO:teuthology.orchestra.run.smithi045.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-12T03:23:33.766 INFO:teuthology.orchestra.run.smithi045.stdout: applications will generate AVC messages and not be allowed access to 2024-09-12T03:23:33.766 INFO:teuthology.orchestra.run.smithi045.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-12T03:23:33.767 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-12T03:23:33.768 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:23:33.768 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-12T03:23:33.794 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-12T03:23:33.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:23:33 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-12T03:23:33.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:23:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T03:23:33.892 INFO:teuthology.orchestra.run.smithi045.stdout:client8751 2024-09-12T03:23:33.892 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-12T03:23:33.892 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T03:23:33.909 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-09-12T03:23:34.024 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:34.024+0000 7f9805027640 1 Processor -- start 2024-09-12T03:23:34.024 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:34.024+0000 7f9805027640 1 -- start start 2024-09-12T03:23:34.026 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:34.025+0000 7f9805027640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f98000d4040 0x7f98000b7ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:34.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:34.025+0000 7f9805027640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f98000d2930 con 0x7f98000b6240 2024-09-12T03:23:34.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:34.025+0000 7f9805027640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f98000d2c10 con 0x7f98000d3c20 2024-09-12T03:23:34.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:34.025+0000 7f9805027640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f98000d2ef0 con 0x7f98000d4040 2024-09-12T03:23:34.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:34.025+0000 7f97fed9d640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f98000b6240 0x7f98000b6620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:51286/0 (socket says 172.21.15.27:51286) 2024-09-12T03:23:34.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:34.025+0000 7f97fed9d640 1 -- 172.21.15.27:0/1811550948 learned_addr learned my addr 172.21.15.27:0/1811550948 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:23:34.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:34.025+0000 7f97ff59e640 1 --2- 172.21.15.27:0/1811550948 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f98000d4040 0x7f98000b7ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:34.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:34.026+0000 7f97fdd9b640 1 -- 172.21.15.27:0/1811550948 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1564927193 0 0) 0x7f98000d2930 con 0x7f98000b6240 2024-09-12T03:23:34.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:34.026+0000 7f97fdd9b640 1 -- 172.21.15.27:0/1811550948 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f97e8003520 con 0x7f98000b6240 2024-09-12T03:23:34.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:34.026+0000 7f97fdd9b640 1 -- 172.21.15.27:0/1811550948 <== mon.1 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2187723319 0 0) 0x7f98000d2c10 con 0x7f98000d3c20 2024-09-12T03:23:34.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:34.026+0000 7f97fdd9b640 1 -- 172.21.15.27:0/1811550948 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f98000d2930 con 0x7f98000d3c20 2024-09-12T03:23:34.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:34.026+0000 7f97ff59e640 1 -- 172.21.15.27:0/1811550948 >> v1:172.21.15.45:6789/0 conn(0x7f98000d3c20 legacy=0x7f98000b77c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:34.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:34.026+0000 7f97fdd9b640 1 -- 172.21.15.27:0/1811550948 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2799282598 0 0) 0x7f97e8003520 con 0x7f98000b6240 2024-09-12T03:23:34.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:34.026+0000 7f97ff59e640 1 -- 172.21.15.27:0/1811550948 >> v1:172.21.15.27:6789/0 conn(0x7f98000b6240 legacy=0x7f98000b6620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:34.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:34.026+0000 7f97ff59e640 1 -- 172.21.15.27:0/1811550948 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98000d31d0 con 0x7f98000d4040 2024-09-12T03:23:34.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:34.027+0000 7f97ff59e640 1 --2- 172.21.15.27:0/1811550948 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f98000d4040 0x7f98000b7ed0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f97f8009dc0 tx=0x7f97f802a6f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=77ff3dcf4c6acfd0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:34.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:34.027+0000 7f97fdd9b640 1 -- 172.21.15.27:0/1811550948 <== mon.2 v2:172.21.15.45:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97f0001670 con 0x7f98000d4040 2024-09-12T03:23:34.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:34.027+0000 7f97fdd9b640 1 -- 172.21.15.27:0/1811550948 <== mon.2 v2:172.21.15.45:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97f802ae50 con 0x7f98000d4040 2024-09-12T03:23:34.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:34.027+0000 7f97fdd9b640 1 -- 172.21.15.27:0/1811550948 <== mon.2 v2:172.21.15.45:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97f8007740 con 0x7f98000d4040 2024-09-12T03:23:34.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:34.027+0000 7f9805027640 1 -- 172.21.15.27:0/1811550948 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f98000d4040 msgr2=0x7f98000b7ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:34.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:34.027+0000 7f9805027640 1 --2- 172.21.15.27:0/1811550948 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f98000d4040 0x7f98000b7ed0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f97f8009dc0 tx=0x7f97f802a6f0 comp rx=0 tx=0).stop 2024-09-12T03:23:34.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:34.027+0000 7f9805027640 1 -- 172.21.15.27:0/1811550948 shutdown_connections 2024-09-12T03:23:34.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:34.027+0000 7f9805027640 1 --2- 172.21.15.27:0/1811550948 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f98000d4040 0x7f98000b7ed0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:34.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:34.027+0000 7f9805027640 1 -- 172.21.15.27:0/1811550948 >> 172.21.15.27:0/1811550948 conn(0x7f98000d1420 msgr2=0x7f98000d1830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:34.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:34.027+0000 7f9805027640 1 -- 172.21.15.27:0/1811550948 shutdown_connections 2024-09-12T03:23:34.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:34.027+0000 7f9805027640 1 -- 172.21.15.27:0/1811550948 wait complete. 2024-09-12T03:23:34.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:34.029+0000 7f9805027640 1 Processor -- start 2024-09-12T03:23:34.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:34.029+0000 7f9805027640 1 -- start start 2024-09-12T03:23:34.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:34.030+0000 7f9805027640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f98000b6240 0x7f98000c82f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:34.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:34.030+0000 7f9805027640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f98000d3c20 0x7f980014daa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:34.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:34.030+0000 7f9805027640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f98000d4040 0x7f98000cda60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:34.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:34.030+0000 7f9805027640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f98000cfdf0 con 0x7f98000d4040 2024-09-12T03:23:34.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:34.030+0000 7f9805027640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f98000cfc70 con 0x7f98000b6240 2024-09-12T03:23:34.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:34.030+0000 7f9805027640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f98000b8410 con 0x7f98000d3c20 2024-09-12T03:23:34.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:34.030+0000 7f97ff59e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f98000d4040 0x7f98000cda60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:34.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:34.030+0000 7f97fed9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f98000b6240 0x7f98000c82f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:34.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:34.030+0000 7f97ff59e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f98000d4040 0x7f98000cda60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34706/0 (socket says 172.21.15.27:34706) 2024-09-12T03:23:34.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:34.030+0000 7f97ff59e640 1 -- 172.21.15.27:0/3107453139 learned_addr learned my addr 172.21.15.27:0/3107453139 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:23:34.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:34.030+0000 7f97fe59c640 1 --2- 172.21.15.27:0/3107453139 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f98000d3c20 0x7f980014daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:34.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:34.030+0000 7f97fed9d640 1 -- 172.21.15.27:0/3107453139 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f98000d3c20 msgr2=0x7f980014daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:34.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:34.030+0000 7f97fed9d640 1 --2- 172.21.15.27:0/3107453139 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f98000d3c20 0x7f980014daa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:34.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:34.030+0000 7f97fed9d640 1 -- 172.21.15.27:0/3107453139 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f98000d4040 msgr2=0x7f98000cda60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:34.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:34.030+0000 7f97fed9d640 1 --2- 172.21.15.27:0/3107453139 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f98000d4040 0x7f98000cda60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:34.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:34.030+0000 7f97fed9d640 1 -- 172.21.15.27:0/3107453139 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98000c89d0 con 0x7f98000b6240 2024-09-12T03:23:34.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:34.030+0000 7f97ff59e640 1 --2- 172.21.15.27:0/3107453139 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f98000d4040 0x7f98000cda60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:23:34.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:34.030+0000 7f97fed9d640 1 --2- 172.21.15.27:0/3107453139 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f98000b6240 0x7f98000c82f0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f97f0001380 tx=0x7f97f000c9c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:34.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:34.031+0000 7f97e77fe640 1 -- 172.21.15.27:0/3107453139 <== mon.1 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97f0017070 con 0x7f98000b6240 2024-09-12T03:23:34.034 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:34.031+0000 7f97e77fe640 1 -- 172.21.15.27:0/3107453139 <== mon.1 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97f0005ce0 con 0x7f98000b6240 2024-09-12T03:23:34.034 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:34.031+0000 7f9805027640 1 -- 172.21.15.27:0/3107453139 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98000c8cb0 con 0x7f98000b6240 2024-09-12T03:23:34.034 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:34.031+0000 7f97e77fe640 1 -- 172.21.15.27:0/3107453139 <== mon.1 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97f0007330 con 0x7f98000b6240 2024-09-12T03:23:34.034 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:34.031+0000 7f9805027640 1 -- 172.21.15.27:0/3107453139 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98000c9270 con 0x7f98000b6240 2024-09-12T03:23:34.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:34.032+0000 7f9805027640 1 -- 172.21.15.27:0/3107453139 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98000b8770 con 0x7f98000b6240 2024-09-12T03:23:34.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:34.032+0000 7f97e77fe640 1 -- 172.21.15.27:0/3107453139 <== mon.1 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 24) ==== 91212+0+0 (secure 0 0 0) 0x7f97f001a070 con 0x7f98000b6240 2024-09-12T03:23:34.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:34.033+0000 7f97e77fe640 1 --2- 172.21.15.27:0/3107453139 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f97ec06c610 0x7f97ec06ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:34.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:34.033+0000 7f97e77fe640 1 -- 172.21.15.27:0/3107453139 <== mon.1 v2:172.21.15.45:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 5765+0+0 (secure 0 0 0) 0x7f97f0088c60 con 0x7f98000b6240 2024-09-12T03:23:34.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:34.033+0000 7f97fe59c640 1 --2- 172.21.15.27:0/3107453139 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f97ec06c610 0x7f97ec06ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:34.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:34.033+0000 7f97fe59c640 1 --2- 172.21.15.27:0/3107453139 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f97ec06c610 0x7f97ec06ead0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f97f4007fd0 tx=0x7f97f4009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:34.037 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:34.037+0000 7f97e77fe640 1 -- 172.21.15.27:0/3107453139 <== mon.1 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f97f00588b0 con 0x7f98000b6240 2024-09-12T03:23:34.251 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:34.251+0000 7f9805027640 1 -- 172.21.15.27:0/3107453139 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7f98000ceaf0 con 0x7f98000b6240 2024-09-12T03:23:34.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:34.852+0000 7f97e77fe640 1 -- 172.21.15.27:0/3107453139 <== mon.1 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v92) ==== 136+0+0 (secure 0 0 0) 0x7f97f005cbb0 con 0x7f98000b6240 2024-09-12T03:23:34.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:34.854+0000 7f9805027640 1 -- 172.21.15.27:0/3107453139 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f97ec06c610 msgr2=0x7f97ec06ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:34.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:34.854+0000 7f9805027640 1 --2- 172.21.15.27:0/3107453139 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f97ec06c610 0x7f97ec06ead0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f97f4007fd0 tx=0x7f97f4009040 comp rx=0 tx=0).stop 2024-09-12T03:23:34.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:34.854+0000 7f9805027640 1 -- 172.21.15.27:0/3107453139 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f98000b6240 msgr2=0x7f98000c82f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:34.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:34.854+0000 7f9805027640 1 --2- 172.21.15.27:0/3107453139 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f98000b6240 0x7f98000c82f0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f97f0001380 tx=0x7f97f000c9c0 comp rx=0 tx=0).stop 2024-09-12T03:23:34.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:34.855+0000 7f9805027640 1 -- 172.21.15.27:0/3107453139 shutdown_connections 2024-09-12T03:23:34.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:34.855+0000 7f9805027640 1 --2- 172.21.15.27:0/3107453139 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f97ec06c610 0x7f97ec06ead0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:34.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:34.855+0000 7f9805027640 1 --2- 172.21.15.27:0/3107453139 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f98000d4040 0x7f98000cda60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:34.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:34.855+0000 7f9805027640 1 --2- 172.21.15.27:0/3107453139 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f98000d3c20 0x7f980014daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:34.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:34.855+0000 7f9805027640 1 --2- 172.21.15.27:0/3107453139 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f98000b6240 0x7f98000c82f0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:34.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:34.855+0000 7f9805027640 1 -- 172.21.15.27:0/3107453139 >> 172.21.15.27:0/3107453139 conn(0x7f98000d1420 msgr2=0x7f98000b6620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:34.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:34.855+0000 7f9805027640 1 -- 172.21.15.27:0/3107453139 shutdown_connections 2024-09-12T03:23:34.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:34.855+0000 7f9805027640 1 -- 172.21.15.27:0/3107453139 wait complete. 2024-09-12T03:23:34.868 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-12T03:23:34.974 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:34.974+0000 7f4f1c08d640 1 Processor -- start 2024-09-12T03:23:34.975 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:34.975+0000 7f4f1c08d640 1 -- start start 2024-09-12T03:23:34.976 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:34.976+0000 7f4f1c08d640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f4f14073960 0x7f4f14162320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:34.976 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:34.976+0000 7f4f1c08d640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4f140d2240 con 0x7f4f14162860 2024-09-12T03:23:34.976 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:34.976+0000 7f4f1c08d640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4f14163090 con 0x7f4f1415e5f0 2024-09-12T03:23:34.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:34.976+0000 7f4f1c08d640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f4f140d0da0 con 0x7f4f14073960 2024-09-12T03:23:34.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:34.976+0000 7f4f1a62a640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f4f14162860 0x7f4f1415e9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:51302/0 (socket says 172.21.15.27:51302) 2024-09-12T03:23:34.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:34.976+0000 7f4f1a62a640 1 -- 172.21.15.27:0/3952774717 learned_addr learned my addr 172.21.15.27:0/3952774717 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:23:34.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:34.976+0000 7f4f19628640 1 --2- 172.21.15.27:0/3952774717 >> v2:172.21.15.45:3300/0 conn(0x7f4f14073960 0x7f4f14162320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:34.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:34.976+0000 7f4f18e27640 1 -- 172.21.15.27:0/3952774717 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2265291079 0 0) 0x7f4f140d2240 con 0x7f4f14162860 2024-09-12T03:23:34.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:34.976+0000 7f4f18e27640 1 -- 172.21.15.27:0/3952774717 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4efc003520 con 0x7f4f14162860 2024-09-12T03:23:34.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:34.976+0000 7f4f19628640 1 -- 172.21.15.27:0/3952774717 >> v1:172.21.15.45:6789/0 conn(0x7f4f1415e5f0 legacy=0x7f4f14073250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:34.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:34.976+0000 7f4f19628640 1 -- 172.21.15.27:0/3952774717 >> v1:172.21.15.27:6789/0 conn(0x7f4f14162860 legacy=0x7f4f1415e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:34.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:34.976+0000 7f4f19628640 1 -- 172.21.15.27:0/3952774717 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f140d0470 con 0x7f4f14073960 2024-09-12T03:23:34.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:34.977+0000 7f4f19628640 1 --2- 172.21.15.27:0/3952774717 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4f14073960 0x7f4f14162320 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f4f080097a0 tx=0x7f4f0802a230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ca461a42d7b74057 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:34.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:34.977+0000 7f4f18e27640 1 -- 172.21.15.27:0/3952774717 <== mon.1 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f08037070 con 0x7f4f14073960 2024-09-12T03:23:34.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:34.977+0000 7f4f18e27640 1 -- 172.21.15.27:0/3952774717 <== mon.1 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f08032440 con 0x7f4f14073960 2024-09-12T03:23:34.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:34.977+0000 7f4f18e27640 1 -- 172.21.15.27:0/3952774717 <== mon.1 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f08032810 con 0x7f4f14073960 2024-09-12T03:23:34.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:34.978+0000 7f4f1c08d640 1 -- 172.21.15.27:0/3952774717 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4f14073960 msgr2=0x7f4f14162320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:34.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:34.978+0000 7f4f1c08d640 1 --2- 172.21.15.27:0/3952774717 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4f14073960 0x7f4f14162320 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f4f080097a0 tx=0x7f4f0802a230 comp rx=0 tx=0).stop 2024-09-12T03:23:34.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:34.978+0000 7f4f1c08d640 1 -- 172.21.15.27:0/3952774717 shutdown_connections 2024-09-12T03:23:34.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:34.978+0000 7f4f1c08d640 1 --2- 172.21.15.27:0/3952774717 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4f14073960 0x7f4f14162320 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:34.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:34.978+0000 7f4f1c08d640 1 -- 172.21.15.27:0/3952774717 >> 172.21.15.27:0/3952774717 conn(0x7f4f1406f300 msgr2=0x7f4f1406f710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:34.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:34.978+0000 7f4f1c08d640 1 -- 172.21.15.27:0/3952774717 shutdown_connections 2024-09-12T03:23:34.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:34.978+0000 7f4f1c08d640 1 -- 172.21.15.27:0/3952774717 wait complete. 2024-09-12T03:23:34.982 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:34.979+0000 7f4f1c08d640 1 Processor -- start 2024-09-12T03:23:34.982 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:34.979+0000 7f4f1c08d640 1 -- start start 2024-09-12T03:23:34.983 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:34.980+0000 7f4f1c08d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4f14073960 0x7f4f141560b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:34.983 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:34.980+0000 7f4f1c08d640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4f1415e5f0 0x7f4f140b6e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:34.983 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:34.980+0000 7f4f1c08d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4f14162860 0x7f4f140b7350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:34.983 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:34.980+0000 7f4f1c08d640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f4f140b5620 con 0x7f4f14073960 2024-09-12T03:23:34.983 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:34.980+0000 7f4f1c08d640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f4f140d0a30 con 0x7f4f14162860 2024-09-12T03:23:34.983 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:34.980+0000 7f4f1c08d640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f4f14162c70 con 0x7f4f1415e5f0 2024-09-12T03:23:34.983 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:34.980+0000 7f4f19e29640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4f14073960 0x7f4f141560b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:34.983 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:34.980+0000 7f4f19e29640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4f14073960 0x7f4f141560b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34720/0 (socket says 172.21.15.27:34720) 2024-09-12T03:23:34.983 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:34.980+0000 7f4f19e29640 1 -- 172.21.15.27:0/2725342409 learned_addr learned my addr 172.21.15.27:0/2725342409 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:23:34.983 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:34.980+0000 7f4f19628640 1 --2- 172.21.15.27:0/2725342409 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4f1415e5f0 0x7f4f140b6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:34.983 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:34.980+0000 7f4f1a62a640 1 --2- 172.21.15.27:0/2725342409 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4f14162860 0x7f4f140b7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:34.983 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:34.980+0000 7f4f19e29640 1 -- 172.21.15.27:0/2725342409 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4f1415e5f0 msgr2=0x7f4f140b6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:34.983 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:34.980+0000 7f4f19e29640 1 --2- 172.21.15.27:0/2725342409 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4f1415e5f0 0x7f4f140b6e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:34.984 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:34.980+0000 7f4f19e29640 1 -- 172.21.15.27:0/2725342409 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4f14162860 msgr2=0x7f4f140b7350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:34.984 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:34.980+0000 7f4f19e29640 1 --2- 172.21.15.27:0/2725342409 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4f14162860 0x7f4f140b7350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:34.984 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:34.980+0000 7f4f19e29640 1 -- 172.21.15.27:0/2725342409 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f141565f0 con 0x7f4f14073960 2024-09-12T03:23:34.984 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:34.980+0000 7f4f19e29640 1 --2- 172.21.15.27:0/2725342409 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4f14073960 0x7f4f141560b0 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7f4f0400c790 tx=0x7f4f0400cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:34.984 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:34.981+0000 7f4f127fc640 1 -- 172.21.15.27:0/2725342409 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f04016020 con 0x7f4f14073960 2024-09-12T03:23:34.984 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:34.981+0000 7f4f127fc640 1 -- 172.21.15.27:0/2725342409 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f04006d50 con 0x7f4f14073960 2024-09-12T03:23:34.984 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:34.981+0000 7f4f127fc640 1 -- 172.21.15.27:0/2725342409 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f04003bb0 con 0x7f4f14073960 2024-09-12T03:23:34.984 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:34.981+0000 7f4f1c08d640 1 -- 172.21.15.27:0/2725342409 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f141f7930 con 0x7f4f14073960 2024-09-12T03:23:34.984 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:34.981+0000 7f4f1c08d640 1 -- 172.21.15.27:0/2725342409 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f141f7ef0 con 0x7f4f14073960 2024-09-12T03:23:34.984 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:34.982+0000 7f4f1c08d640 1 -- 172.21.15.27:0/2725342409 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f14162320 con 0x7f4f14073960 2024-09-12T03:23:34.984 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:34.982+0000 7f4f127fc640 1 -- 172.21.15.27:0/2725342409 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 24) ==== 91212+0+0 (secure 0 0 0) 0x7f4f0400a850 con 0x7f4f14073960 2024-09-12T03:23:34.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:34.983+0000 7f4f127fc640 1 --2- 172.21.15.27:0/2725342409 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f4f0006c610 0x7f4f0006ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:34.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:34.983+0000 7f4f127fc640 1 -- 172.21.15.27:0/2725342409 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 5765+0+0 (secure 0 0 0) 0x7f4f04089410 con 0x7f4f14073960 2024-09-12T03:23:34.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:34.983+0000 7f4f19628640 1 --2- 172.21.15.27:0/2725342409 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f4f0006c610 0x7f4f0006ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:34.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:34.983+0000 7f4f19628640 1 --2- 172.21.15.27:0/2725342409 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f4f0006c610 0x7f4f0006ead0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f4f08034000 tx=0x7f4f0802e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:34.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:34.987+0000 7f4f127fc640 1 -- 172.21.15.27:0/2725342409 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f4f04020020 con 0x7f4f14073960 2024-09-12T03:23:35.185 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:35.185+0000 7f4f1c08d640 1 -- 172.21.15.27:0/2725342409 --> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f4f14159dd0 con 0x7f4f0006c610 2024-09-12T03:23:35.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:35.186+0000 7f4f127fc640 1 -- 172.21.15.27:0/2725342409 <== mgr.4113 v2:172.21.15.27:6836/1226920526 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f4f14159dd0 con 0x7f4f0006c610 2024-09-12T03:23:35.186 INFO:teuthology.orchestra.run.smithi027.stdout:[ 2024-09-12T03:23:35.186 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:23:35.186 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "cephfs" 2024-09-12T03:23:35.187 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:23:35.187 INFO:teuthology.orchestra.run.smithi027.stdout:] 2024-09-12T03:23:35.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:35.189+0000 7f4f1c08d640 1 -- 172.21.15.27:0/2725342409 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f4f0006c610 msgr2=0x7f4f0006ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:35.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:35.189+0000 7f4f1c08d640 1 --2- 172.21.15.27:0/2725342409 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f4f0006c610 0x7f4f0006ead0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f4f08034000 tx=0x7f4f0802e000 comp rx=0 tx=0).stop 2024-09-12T03:23:35.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:35.189+0000 7f4f1c08d640 1 -- 172.21.15.27:0/2725342409 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4f14073960 msgr2=0x7f4f141560b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:35.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:35.189+0000 7f4f1c08d640 1 --2- 172.21.15.27:0/2725342409 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4f14073960 0x7f4f141560b0 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7f4f0400c790 tx=0x7f4f0400cc50 comp rx=0 tx=0).stop 2024-09-12T03:23:35.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:35.189+0000 7f4f1c08d640 1 -- 172.21.15.27:0/2725342409 shutdown_connections 2024-09-12T03:23:35.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:35.189+0000 7f4f1c08d640 1 --2- 172.21.15.27:0/2725342409 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4f14162860 0x7f4f140b7350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:35.190 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:35.189+0000 7f4f1c08d640 1 --2- 172.21.15.27:0/2725342409 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4f1415e5f0 0x7f4f140b6e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:35.190 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:35.189+0000 7f4f1c08d640 1 --2- 172.21.15.27:0/2725342409 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f4f0006c610 0x7f4f0006ead0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:35.190 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:35.189+0000 7f4f1c08d640 1 --2- 172.21.15.27:0/2725342409 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4f14073960 0x7f4f141560b0 unknown :-1 s=CLOSED pgs=1578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:35.190 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:35.189+0000 7f4f1c08d640 1 -- 172.21.15.27:0/2725342409 >> 172.21.15.27:0/2725342409 conn(0x7f4f1406f300 msgr2=0x7f4f14159030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:35.190 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:35.189+0000 7f4f1c08d640 1 -- 172.21.15.27:0/2725342409 shutdown_connections 2024-09-12T03:23:35.190 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:35.189+0000 7f4f1c08d640 1 -- 172.21.15.27:0/2725342409 wait complete. 2024-09-12T03:23:35.199 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-12T03:23:35.305 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:35.305+0000 7f9f42d52640 1 Processor -- start 2024-09-12T03:23:35.305 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:35.305+0000 7f9f42d52640 1 -- start start 2024-09-12T03:23:35.307 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:35.305+0000 7f9f42d52640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f9f3c15ec30 0x7f9f3c0b3900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:35.307 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:35.306+0000 7f9f42d52640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f9f3c06da00 0x7f9f3c06de90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:35.307 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:35.306+0000 7f9f42d52640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9f3c0ce5e0 con 0x7f9f3c0af9d0 2024-09-12T03:23:35.307 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:35.306+0000 7f9f42d52640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f9f3c0d02d0 con 0x7f9f3c15ec30 2024-09-12T03:23:35.307 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:35.306+0000 7f9f42d52640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f9f3c06c650 con 0x7f9f3c06da00 2024-09-12T03:23:35.307 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:35.306+0000 7f9f40aee640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f9f3c15ec30 0x7f9f3c0b3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:35.307 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:35.306+0000 7f9f40aee640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f9f3c15ec30 0x7f9f3c0b3900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34728/0 (socket says 172.21.15.27:34728) 2024-09-12T03:23:35.307 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:35.306+0000 7f9f3bfff640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f9f3c0af9d0 0x7f9f3c0b3e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:51308/0 (socket says 172.21.15.27:51308) 2024-09-12T03:23:35.308 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:35.306+0000 7f9f40aee640 1 -- 172.21.15.27:0/515500304 learned_addr learned my addr 172.21.15.27:0/515500304 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:23:35.308 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:35.306+0000 7f9f412ef640 1 --2- 172.21.15.27:0/515500304 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f9f3c06da00 0x7f9f3c06de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:35.308 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:35.306+0000 7f9f40aee640 1 -- 172.21.15.27:0/515500304 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f9f3c06da00 msgr2=0x7f9f3c06de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:35.308 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:35.306+0000 7f9f40aee640 1 --2- 172.21.15.27:0/515500304 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f9f3c06da00 0x7f9f3c06de90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:35.308 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:35.306+0000 7f9f40aee640 1 -- 172.21.15.27:0/515500304 >> v1:172.21.15.27:6789/0 conn(0x7f9f3c0af9d0 legacy=0x7f9f3c0b3e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:35.308 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:35.306+0000 7f9f40aee640 1 -- 172.21.15.27:0/515500304 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f3c0ceac0 con 0x7f9f3c15ec30 2024-09-12T03:23:35.308 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:35.306+0000 7f9f3b7fe640 1 -- 172.21.15.27:0/515500304 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2536506550 0 0) 0x7f9f3c0ce5e0 con 0x7f9f3c0af9d0 2024-09-12T03:23:35.308 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:35.307+0000 7f9f40aee640 1 --2- 172.21.15.27:0/515500304 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9f3c15ec30 0x7f9f3c0b3900 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7f9f3c06c7d0 tx=0x7f9f3002a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca16dcdf9da596e3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:35.308 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:35.307+0000 7f9f3b7fe640 1 -- 172.21.15.27:0/515500304 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f30037070 con 0x7f9f3c15ec30 2024-09-12T03:23:35.308 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:35.307+0000 7f9f3b7fe640 1 -- 172.21.15.27:0/515500304 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f30032440 con 0x7f9f3c15ec30 2024-09-12T03:23:35.309 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:35.307+0000 7f9f3b7fe640 1 -- 172.21.15.27:0/515500304 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f30032810 con 0x7f9f3c15ec30 2024-09-12T03:23:35.311 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:35.307+0000 7f9f42d52640 1 -- 172.21.15.27:0/515500304 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9f3c15ec30 msgr2=0x7f9f3c0b3900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:35.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:35.307+0000 7f9f42d52640 1 --2- 172.21.15.27:0/515500304 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9f3c15ec30 0x7f9f3c0b3900 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7f9f3c06c7d0 tx=0x7f9f3002a250 comp rx=0 tx=0).stop 2024-09-12T03:23:35.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:35.308+0000 7f9f42d52640 1 -- 172.21.15.27:0/515500304 shutdown_connections 2024-09-12T03:23:35.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:35.308+0000 7f9f42d52640 1 --2- 172.21.15.27:0/515500304 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f9f3c06da00 0x7f9f3c06de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:35.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:35.308+0000 7f9f42d52640 1 --2- 172.21.15.27:0/515500304 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9f3c15ec30 0x7f9f3c0b3900 unknown :-1 s=CLOSED pgs=1580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:35.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:35.308+0000 7f9f42d52640 1 -- 172.21.15.27:0/515500304 >> 172.21.15.27:0/515500304 conn(0x7f9f3c06bb30 msgr2=0x7f9f3c06bf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:35.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:35.308+0000 7f9f42d52640 1 -- 172.21.15.27:0/515500304 shutdown_connections 2024-09-12T03:23:35.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:35.308+0000 7f9f42d52640 1 -- 172.21.15.27:0/515500304 wait complete. 2024-09-12T03:23:35.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:35.309+0000 7f9f42d52640 1 Processor -- start 2024-09-12T03:23:35.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:35.309+0000 7f9f42d52640 1 -- start start 2024-09-12T03:23:35.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:35.310+0000 7f9f42d52640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9f3c06da00 0x7f9f3c1437b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:35.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:35.310+0000 7f9f42d52640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9f3c0af9d0 0x7f9f3c0ae8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:35.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:35.310+0000 7f9f42d52640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f9f3c15ec30 0x7f9f3c0aee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:35.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:35.310+0000 7f9f42d52640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f9f3c0cfff0 con 0x7f9f3c06da00 2024-09-12T03:23:35.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:35.310+0000 7f9f42d52640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f9f3c06c8a0 con 0x7f9f3c0af9d0 2024-09-12T03:23:35.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:35.310+0000 7f9f42d52640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f9f3c0715f0 con 0x7f9f3c15ec30 2024-09-12T03:23:35.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:35.310+0000 7f9f40aee640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9f3c06da00 0x7f9f3c1437b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:35.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:35.310+0000 7f9f40aee640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9f3c06da00 0x7f9f3c1437b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34734/0 (socket says 172.21.15.27:34734) 2024-09-12T03:23:35.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:35.310+0000 7f9f40aee640 1 -- 172.21.15.27:0/1684372212 learned_addr learned my addr 172.21.15.27:0/1684372212 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:23:35.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:35.310+0000 7f9f3bfff640 1 --2- 172.21.15.27:0/1684372212 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9f3c0af9d0 0x7f9f3c0ae8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:35.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:35.310+0000 7f9f412ef640 1 --2- 172.21.15.27:0/1684372212 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f9f3c15ec30 0x7f9f3c0aee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:35.314 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:35.310+0000 7f9f40aee640 1 -- 172.21.15.27:0/1684372212 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f9f3c15ec30 msgr2=0x7f9f3c0aee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:35.314 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:35.310+0000 7f9f40aee640 1 --2- 172.21.15.27:0/1684372212 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f9f3c15ec30 0x7f9f3c0aee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:35.314 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:35.310+0000 7f9f40aee640 1 -- 172.21.15.27:0/1684372212 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9f3c0af9d0 msgr2=0x7f9f3c0ae8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:35.314 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:35.310+0000 7f9f40aee640 1 --2- 172.21.15.27:0/1684372212 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9f3c0af9d0 0x7f9f3c0ae8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:35.314 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:35.310+0000 7f9f40aee640 1 -- 172.21.15.27:0/1684372212 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f3c143cf0 con 0x7f9f3c06da00 2024-09-12T03:23:35.314 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:35.310+0000 7f9f40aee640 1 --2- 172.21.15.27:0/1684372212 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9f3c06da00 0x7f9f3c1437b0 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7f9f30035040 tx=0x7f9f300046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:35.314 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:35.311+0000 7f9f397fa640 1 -- 172.21.15.27:0/1684372212 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f30041070 con 0x7f9f3c06da00 2024-09-12T03:23:35.314 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:35.311+0000 7f9f397fa640 1 -- 172.21.15.27:0/1684372212 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f30004220 con 0x7f9f3c06da00 2024-09-12T03:23:35.314 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:35.311+0000 7f9f397fa640 1 -- 172.21.15.27:0/1684372212 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f30037040 con 0x7f9f3c06da00 2024-09-12T03:23:35.314 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:35.311+0000 7f9f42d52640 1 -- 172.21.15.27:0/1684372212 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f3c151af0 con 0x7f9f3c06da00 2024-09-12T03:23:35.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:35.311+0000 7f9f42d52640 1 -- 172.21.15.27:0/1684372212 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f3c150d30 con 0x7f9f3c06da00 2024-09-12T03:23:35.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:35.312+0000 7f9f42d52640 1 -- 172.21.15.27:0/1684372212 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f3c06de90 con 0x7f9f3c06da00 2024-09-12T03:23:35.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:35.312+0000 7f9f397fa640 1 -- 172.21.15.27:0/1684372212 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 24) ==== 91212+0+0 (secure 0 0 0) 0x7f9f30003610 con 0x7f9f3c06da00 2024-09-12T03:23:35.316 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:35.313+0000 7f9f397fa640 1 --2- 172.21.15.27:0/1684372212 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f9f2006c640 0x7f9f2006eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:35.316 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:35.313+0000 7f9f397fa640 1 -- 172.21.15.27:0/1684372212 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 5765+0+0 (secure 0 0 0) 0x7f9f300ac660 con 0x7f9f3c06da00 2024-09-12T03:23:35.316 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:35.313+0000 7f9f3bfff640 1 --2- 172.21.15.27:0/1684372212 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f9f2006c640 0x7f9f2006eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:35.316 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:35.313+0000 7f9f3bfff640 1 --2- 172.21.15.27:0/1684372212 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f9f2006c640 0x7f9f2006eb00 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f9f3c154530 tx=0x7f9f28007d50 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:35.317 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:35.317+0000 7f9f397fa640 1 -- 172.21.15.27:0/1684372212 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f9f30079b30 con 0x7f9f3c06da00 2024-09-12T03:23:35.490 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:35.490+0000 7f9f42d52640 1 -- 172.21.15.27:0/1684372212 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f9f3c06f240 con 0x7f9f3c06da00 2024-09-12T03:23:35.490 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:35.490+0000 7f9f397fa640 1 -- 172.21.15.27:0/1684372212 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v28)=0 v28) ==== 122+0+0 (secure 0 0 0) 0x7f9f3007de30 con 0x7f9f3c06da00 2024-09-12T03:23:35.493 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:35.492+0000 7f9f42d52640 1 -- 172.21.15.27:0/1684372212 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f9f2006c640 msgr2=0x7f9f2006eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:35.493 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:35.492+0000 7f9f42d52640 1 --2- 172.21.15.27:0/1684372212 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f9f2006c640 0x7f9f2006eb00 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f9f3c154530 tx=0x7f9f28007d50 comp rx=0 tx=0).stop 2024-09-12T03:23:35.493 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:35.492+0000 7f9f42d52640 1 -- 172.21.15.27:0/1684372212 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9f3c06da00 msgr2=0x7f9f3c1437b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:35.493 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:35.492+0000 7f9f42d52640 1 --2- 172.21.15.27:0/1684372212 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9f3c06da00 0x7f9f3c1437b0 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7f9f30035040 tx=0x7f9f300046c0 comp rx=0 tx=0).stop 2024-09-12T03:23:35.493 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:35.492+0000 7f9f42d52640 1 -- 172.21.15.27:0/1684372212 shutdown_connections 2024-09-12T03:23:35.493 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:35.492+0000 7f9f42d52640 1 --2- 172.21.15.27:0/1684372212 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f9f3c15ec30 0x7f9f3c0aee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:35.493 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:35.492+0000 7f9f42d52640 1 --2- 172.21.15.27:0/1684372212 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9f3c0af9d0 0x7f9f3c0ae8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:35.494 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:35.492+0000 7f9f42d52640 1 --2- 172.21.15.27:0/1684372212 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f9f2006c640 0x7f9f2006eb00 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:35.494 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:35.492+0000 7f9f42d52640 1 --2- 172.21.15.27:0/1684372212 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9f3c06da00 0x7f9f3c1437b0 unknown :-1 s=CLOSED pgs=1581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:35.494 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:35.492+0000 7f9f42d52640 1 -- 172.21.15.27:0/1684372212 >> 172.21.15.27:0/1684372212 conn(0x7f9f3c06bb30 msgr2=0x7f9f3c06e2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:35.494 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:35.492+0000 7f9f42d52640 1 -- 172.21.15.27:0/1684372212 shutdown_connections 2024-09-12T03:23:35.494 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:35.492+0000 7f9f42d52640 1 -- 172.21.15.27:0/1684372212 wait complete. 2024-09-12T03:23:35.504 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:23:35.504 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p volumes/_nogroup/subvol19020/9746d73a-996d-412d-a829-a3e262df9432) 2024-09-12T03:23:35.536 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:23:35.536 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.layout.pool .) 2024-09-12T03:23:35.593 INFO:teuthology.orchestra.run.smithi027.stdout:cephfs_data 2024-09-12T03:23:35.594 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:23:35.594 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo setfattr -n ceph.dir.layout.pool -v cephfs_data volumes/_nogroup/subvol19020/9746d73a-996d-412d-a829-a3e262df9432) 2024-09-12T03:23:35.663 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-12T03:23:35.663 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd of=/home/ubuntu/cephtest/mnt.0/volumes/_nogroup/subvol19020/.meta 2024-09-12T03:23:35.731 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:23:35.732 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p volumes/subvol_grp73700/subvol27051/9b490ef8-e617-46c8-b41a-5c4cd66187a8) 2024-09-12T03:23:35.800 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:23:35.800 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.layout.pool .) 2024-09-12T03:23:35.857 INFO:teuthology.orchestra.run.smithi027.stdout:cephfs_data 2024-09-12T03:23:35.858 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:23:35.858 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo setfattr -n ceph.dir.layout.pool -v cephfs_data volumes/subvol_grp73700/subvol27051/9b490ef8-e617-46c8-b41a-5c4cd66187a8) 2024-09-12T03:23:35.928 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-12T03:23:35.928 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd of=/home/ubuntu/cephtest/mnt.0/volumes/subvol_grp73700/subvol27051/.meta 2024-09-12T03:23:35.995 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol19020 2024-09-12T03:23:36.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:36.141+0000 7f76a7add640 1 Processor -- start 2024-09-12T03:23:36.142 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:36.142+0000 7f76a7add640 1 -- start start 2024-09-12T03:23:36.145 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:36.142+0000 7f76a7add640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f76a0162040 0x7f76a00d1da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:36.145 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:36.142+0000 7f76a7add640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f76a006dfd0 0x7f76a00b7a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:36.145 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:36.142+0000 7f76a7add640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f76a00d2e30 con 0x7f76a00d3790 2024-09-12T03:23:36.145 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:36.143+0000 7f76a7add640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f76a00b7f90 con 0x7f76a0162040 2024-09-12T03:23:36.145 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:36.143+0000 7f76a7add640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f76a0162420 con 0x7f76a006dfd0 2024-09-12T03:23:36.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:36.143+0000 7f76a5078640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f76a00d3790 0x7f76a006d8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:51322/0 (socket says 172.21.15.27:51322) 2024-09-12T03:23:36.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:36.143+0000 7f76a5078640 1 -- 172.21.15.27:0/1051349593 learned_addr learned my addr 172.21.15.27:0/1051349593 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:23:36.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:36.143+0000 7f76a5879640 1 --2- 172.21.15.27:0/1051349593 >> v2:172.21.15.45:3300/0 conn(0x7f76a0162040 0x7f76a00d1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:36.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:36.143+0000 7f76a4877640 1 -- 172.21.15.27:0/1051349593 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2154744930 0 0) 0x7f76a00d2e30 con 0x7f76a00d3790 2024-09-12T03:23:36.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:36.143+0000 7f76a607a640 1 --2- 172.21.15.27:0/1051349593 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f76a006dfd0 0x7f76a00b7a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:36.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:36.143+0000 7f76a4877640 1 -- 172.21.15.27:0/1051349593 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7688003520 con 0x7f76a00d3790 2024-09-12T03:23:36.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:36.143+0000 7f76a4877640 1 -- 172.21.15.27:0/1051349593 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3749723266 0 0) 0x7f7688003520 con 0x7f76a00d3790 2024-09-12T03:23:36.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:36.143+0000 7f76a4877640 1 -- 172.21.15.27:0/1051349593 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f76a006dfd0 msgr2=0x7f76a00b7a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:36.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:36.143+0000 7f76a4877640 1 --2- 172.21.15.27:0/1051349593 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f76a006dfd0 0x7f76a00b7a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:36.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:36.143+0000 7f76a4877640 1 -- 172.21.15.27:0/1051349593 >> v2:172.21.15.45:3300/0 conn(0x7f76a0162040 msgr2=0x7f76a00d1da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:36.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:36.143+0000 7f76a4877640 1 --2- 172.21.15.27:0/1051349593 >> v2:172.21.15.45:3300/0 conn(0x7f76a0162040 0x7f76a00d1da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:36.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:36.143+0000 7f76a4877640 1 -- 172.21.15.27:0/1051349593 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76a00d30d0 con 0x7f76a00d3790 2024-09-12T03:23:36.147 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:36.143+0000 7f76a4877640 1 -- 172.21.15.27:0/1051349593 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 847755213 0 0) 0x7f7694001670 con 0x7f76a00d3790 2024-09-12T03:23:36.147 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:36.143+0000 7f76a4877640 1 -- 172.21.15.27:0/1051349593 >> v1:172.21.15.27:6789/0 conn(0x7f76a00d3790 legacy=0x7f76a006d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:36.147 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:36.143+0000 7f76a4877640 1 --2- 172.21.15.27:0/1051349593 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f76880039b0 0x7f7688003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:36.147 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:36.143+0000 7f76a4877640 1 --2- 172.21.15.27:0/1051349593 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f76a006dfd0 0x7f7688004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:36.147 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:36.143+0000 7f76a4877640 1 --2- 172.21.15.27:0/1051349593 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7688004d40 0x7f76880051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:36.147 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:36.143+0000 7f76a4877640 1 -- 172.21.15.27:0/1051349593 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f76a00b7f90 con 0x7f7688004d40 2024-09-12T03:23:36.147 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:36.143+0000 7f76a4877640 1 -- 172.21.15.27:0/1051349593 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f76a0162420 con 0x7f76880039b0 2024-09-12T03:23:36.147 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:36.143+0000 7f76a4877640 1 -- 172.21.15.27:0/1051349593 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f768800b9c0 con 0x7f76a006dfd0 2024-09-12T03:23:36.147 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:36.143+0000 7f76a607a640 1 --2- 172.21.15.27:0/1051349593 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7688004d40 0x7f76880051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:36.147 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:36.144+0000 7f76a5879640 1 --2- 172.21.15.27:0/1051349593 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f76880039b0 0x7f7688003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:36.148 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:36.144+0000 7f76a5078640 1 --2- 172.21.15.27:0/1051349593 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f76a006dfd0 0x7f7688004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:36.148 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:36.144+0000 7f76a607a640 1 -- 172.21.15.27:0/1051349593 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f76a006dfd0 msgr2=0x7f7688004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:36.148 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:36.144+0000 7f76a607a640 1 --2- 172.21.15.27:0/1051349593 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f76a006dfd0 0x7f7688004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:36.148 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:36.144+0000 7f76a607a640 1 -- 172.21.15.27:0/1051349593 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f76880039b0 msgr2=0x7f7688003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:36.148 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:36.144+0000 7f76a607a640 1 --2- 172.21.15.27:0/1051349593 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f76880039b0 0x7f7688003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:36.148 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:36.144+0000 7f76a607a640 1 -- 172.21.15.27:0/1051349593 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7688003520 con 0x7f7688004d40 2024-09-12T03:23:36.148 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:36.144+0000 7f76a607a640 1 --2- 172.21.15.27:0/1051349593 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7688004d40 0x7f76880051c0 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7f7690009fc0 tx=0x7f7688005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b009b5acdac8ccc5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:36.148 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:36.145+0000 7f76a4877640 1 -- 172.21.15.27:0/1051349593 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7690009c10 con 0x7f7688004d40 2024-09-12T03:23:36.148 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:36.145+0000 7f76a4877640 1 -- 172.21.15.27:0/1051349593 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7690036070 con 0x7f7688004d40 2024-09-12T03:23:36.148 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:36.145+0000 7f76a4877640 1 -- 172.21.15.27:0/1051349593 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7690031880 con 0x7f7688004d40 2024-09-12T03:23:36.148 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:36.145+0000 7f76a7add640 1 -- 172.21.15.27:0/1051349593 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7688004d40 msgr2=0x7f76880051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:36.149 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:36.145+0000 7f76a7add640 1 --2- 172.21.15.27:0/1051349593 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7688004d40 0x7f76880051c0 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7f7690009fc0 tx=0x7f7688005830 comp rx=0 tx=0).stop 2024-09-12T03:23:36.149 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:36.145+0000 7f76a607a640 1 -- 172.21.15.27:0/1051349593 reap_dead start 2024-09-12T03:23:36.149 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:36.145+0000 7f76a7add640 1 -- 172.21.15.27:0/1051349593 shutdown_connections 2024-09-12T03:23:36.149 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:36.145+0000 7f76a7add640 1 -- 172.21.15.27:0/1051349593 >> 172.21.15.27:0/1051349593 conn(0x7f76a00b6700 msgr2=0x7f76a00b6b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:36.149 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:36.145+0000 7f76a7add640 1 -- 172.21.15.27:0/1051349593 shutdown_connections 2024-09-12T03:23:36.149 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:36.145+0000 7f76a7add640 1 -- 172.21.15.27:0/1051349593 wait complete. 2024-09-12T03:23:36.151 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:36.146+0000 7f76a7add640 1 Processor -- start 2024-09-12T03:23:36.152 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:36.147+0000 7f76a7add640 1 -- start start 2024-09-12T03:23:36.152 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:36.147+0000 7f76a7add640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7688004d40 0x7f76a0153a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:36.152 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:36.147+0000 7f76a7add640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f76a00ac320 0x7f76a00ac7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:36.152 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:36.147+0000 7f76a7add640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f76a01f44e0 0x7f76a01f4960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:36.152 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:36.147+0000 7f76a7add640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f76a00d4530 con 0x7f7688004d40 2024-09-12T03:23:36.152 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:36.147+0000 7f76a7add640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f76a00cee00 con 0x7f76a00ac320 2024-09-12T03:23:36.152 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:36.147+0000 7f76a7add640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f76a00d4150 con 0x7f76a01f44e0 2024-09-12T03:23:36.152 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:36.147+0000 7f76a5879640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7688004d40 0x7f76a0153a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:36.152 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:36.147+0000 7f76a5879640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7688004d40 0x7f76a0153a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34738/0 (socket says 172.21.15.27:34738) 2024-09-12T03:23:36.152 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:36.147+0000 7f76a5879640 1 -- 172.21.15.27:0/2534794009 learned_addr learned my addr 172.21.15.27:0/2534794009 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:23:36.152 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:36.147+0000 7f76a5078640 1 --2- 172.21.15.27:0/2534794009 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f76a00ac320 0x7f76a00ac7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:36.152 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:36.147+0000 7f76a607a640 1 --2- 172.21.15.27:0/2534794009 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f76a01f44e0 0x7f76a01f4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:36.153 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:36.148+0000 7f76a5879640 1 -- 172.21.15.27:0/2534794009 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f76a01f44e0 msgr2=0x7f76a01f4960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:36.153 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:36.148+0000 7f76a5879640 1 --2- 172.21.15.27:0/2534794009 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f76a01f44e0 0x7f76a01f4960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:36.153 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:36.148+0000 7f76a5879640 1 -- 172.21.15.27:0/2534794009 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f76a00ac320 msgr2=0x7f76a00ac7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:36.153 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:36.148+0000 7f76a5879640 1 --2- 172.21.15.27:0/2534794009 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f76a00ac320 0x7f76a00ac7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:36.153 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:36.148+0000 7f76a5879640 1 -- 172.21.15.27:0/2534794009 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76a0153fc0 con 0x7f7688004d40 2024-09-12T03:23:36.153 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:36.148+0000 7f76a5078640 1 --2- 172.21.15.27:0/2534794009 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f76a00ac320 0x7f76a00ac7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:23:36.153 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:36.148+0000 7f76a5879640 1 --2- 172.21.15.27:0/2534794009 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7688004d40 0x7f76a0153a80 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7f7698011aa0 tx=0x7f7698011f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:36.153 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:36.148+0000 7f769e7fc640 1 -- 172.21.15.27:0/2534794009 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f769800fb20 con 0x7f7688004d40 2024-09-12T03:23:36.153 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:36.148+0000 7f769e7fc640 1 -- 172.21.15.27:0/2534794009 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f769800fcc0 con 0x7f7688004d40 2024-09-12T03:23:36.153 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:36.148+0000 7f76a7add640 1 -- 172.21.15.27:0/2534794009 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76a00ada30 con 0x7f7688004d40 2024-09-12T03:23:36.153 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:36.148+0000 7f769e7fc640 1 -- 172.21.15.27:0/2534794009 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7698014660 con 0x7f7688004d40 2024-09-12T03:23:36.153 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:36.148+0000 7f76a7add640 1 -- 172.21.15.27:0/2534794009 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76a00adff0 con 0x7f7688004d40 2024-09-12T03:23:36.153 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:36.149+0000 7f76a7add640 1 -- 172.21.15.27:0/2534794009 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76a00b7a50 con 0x7f7688004d40 2024-09-12T03:23:36.154 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:36.149+0000 7f769e7fc640 1 -- 172.21.15.27:0/2534794009 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 24) ==== 91212+0+0 (secure 0 0 0) 0x7f7698012040 con 0x7f7688004d40 2024-09-12T03:23:36.154 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:36.150+0000 7f769e7fc640 1 --2- 172.21.15.27:0/2534794009 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f768c06c640 0x7f768c06eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:36.154 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:36.150+0000 7f769e7fc640 1 -- 172.21.15.27:0/2534794009 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 5765+0+0 (secure 0 0 0) 0x7f769808c640 con 0x7f7688004d40 2024-09-12T03:23:36.154 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:36.150+0000 7f76a5078640 1 --2- 172.21.15.27:0/2534794009 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f768c06c640 0x7f768c06eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:36.154 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:36.150+0000 7f76a5078640 1 --2- 172.21.15.27:0/2534794009 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f768c06c640 0x7f768c06eb00 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f7694007fd0 tx=0x7f7694009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:36.154 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:36.154+0000 7f769e7fc640 1 -- 172.21.15.27:0/2534794009 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f7698019030 con 0x7f7688004d40 2024-09-12T03:23:36.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:36.356+0000 7f76a7add640 1 -- 172.21.15.27:0/2534794009 --> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol19020", "target": ["mon-mgr", ""]}) -- 0x7f76a00d00d0 con 0x7f768c06c640 2024-09-12T03:23:36.361 INFO:tasks.ceph.mgr.y.smithi027.stderr:2024-09-12T03:23:36.361+0000 7f325d8e3640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:23:36.361 INFO:tasks.ceph.mgr.y.smithi027.stderr:2024-09-12T03:23:36.361+0000 7f325d8e3640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:23:36.361 INFO:tasks.ceph.mgr.y.smithi027.stderr:2024-09-12T03:23:36.361+0000 7f325d8e3640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:23:36.361 INFO:tasks.ceph.mgr.y.smithi027.stderr:2024-09-12T03:23:36.361+0000 7f325d8e3640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:23:36.361 INFO:tasks.ceph.mgr.y.smithi027.stderr:2024-09-12T03:23:36.361+0000 7f325d8e3640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:23:36.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:36.411+0000 7f769e7fc640 1 -- 172.21.15.27:0/2534794009 <== mgr.4113 v2:172.21.15.27:6836/1226920526 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+67 (secure 0 0 0) 0x7f76a00d00d0 con 0x7f768c06c640 2024-09-12T03:23:36.411 INFO:teuthology.orchestra.run.smithi027.stdout:/volumes/_nogroup/subvol19020/9746d73a-996d-412d-a829-a3e262df9432 2024-09-12T03:23:36.413 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:36.413+0000 7f76a7add640 1 -- 172.21.15.27:0/2534794009 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f768c06c640 msgr2=0x7f768c06eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:36.413 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:36.413+0000 7f76a7add640 1 --2- 172.21.15.27:0/2534794009 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f768c06c640 0x7f768c06eb00 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f7694007fd0 tx=0x7f7694009040 comp rx=0 tx=0).stop 2024-09-12T03:23:36.414 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:36.413+0000 7f76a7add640 1 -- 172.21.15.27:0/2534794009 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7688004d40 msgr2=0x7f76a0153a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:36.414 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:36.413+0000 7f76a7add640 1 --2- 172.21.15.27:0/2534794009 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7688004d40 0x7f76a0153a80 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7f7698011aa0 tx=0x7f7698011f60 comp rx=0 tx=0).stop 2024-09-12T03:23:36.414 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:36.413+0000 7f76a7add640 1 -- 172.21.15.27:0/2534794009 shutdown_connections 2024-09-12T03:23:36.414 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:36.413+0000 7f76a7add640 1 --2- 172.21.15.27:0/2534794009 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f76a01f44e0 0x7f76a01f4960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:36.414 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:36.413+0000 7f76a7add640 1 --2- 172.21.15.27:0/2534794009 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f76a00ac320 0x7f76a00ac7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:36.414 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:36.413+0000 7f76a7add640 1 --2- 172.21.15.27:0/2534794009 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f768c06c640 0x7f768c06eb00 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:36.414 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:36.413+0000 7f76a7add640 1 --2- 172.21.15.27:0/2534794009 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7688004d40 0x7f76a0153a80 unknown :-1 s=CLOSED pgs=1584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:36.414 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:36.413+0000 7f76a7add640 1 -- 172.21.15.27:0/2534794009 >> 172.21.15.27:0/2534794009 conn(0x7f76a00b6700 msgr2=0x7f76a00b8140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:36.414 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:36.413+0000 7f76a7add640 1 -- 172.21.15.27:0/2534794009 shutdown_connections 2024-09-12T03:23:36.414 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:36.413+0000 7f76a7add640 1 -- 172.21.15.27:0/2534794009 wait complete. 2024-09-12T03:23:36.425 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol27051 subvol_grp73700 2024-09-12T03:23:36.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:36.531+0000 7fa6054ae640 1 Processor -- start 2024-09-12T03:23:36.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:36.531+0000 7fa6054ae640 1 -- start start 2024-09-12T03:23:36.532 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:36.531+0000 7fa6054ae640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fa60015ec30 0x7fa6000b3900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:36.532 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:36.531+0000 7fa6054ae640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa60006da00 0x7fa60006de90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:36.532 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:36.531+0000 7fa6054ae640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa6000ce5e0 con 0x7fa6000af9d0 2024-09-12T03:23:36.532 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:36.531+0000 7fa6054ae640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7fa6000d02d0 con 0x7fa60015ec30 2024-09-12T03:23:36.532 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:36.531+0000 7fa6054ae640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fa60006c650 con 0x7fa60006da00 2024-09-12T03:23:36.532 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:36.532+0000 7fa5feffd640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fa60015ec30 0x7fa6000b3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:36.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:36.532+0000 7fa5feffd640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fa60015ec30 0x7fa6000b3900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34780/0 (socket says 172.21.15.27:34780) 2024-09-12T03:23:36.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:36.532+0000 7fa5feffd640 1 -- 172.21.15.27:0/2075222770 learned_addr learned my addr 172.21.15.27:0/2075222770 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:23:36.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:36.532+0000 7fa5ff7fe640 1 --2- 172.21.15.27:0/2075222770 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa60006da00 0x7fa60006de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:36.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:36.532+0000 7fa5feffd640 1 -- 172.21.15.27:0/2075222770 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa60006da00 msgr2=0x7fa60006de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:36.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:36.532+0000 7fa5feffd640 1 --2- 172.21.15.27:0/2075222770 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa60006da00 0x7fa60006de90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:36.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:36.532+0000 7fa5feffd640 1 -- 172.21.15.27:0/2075222770 >> v1:172.21.15.45:6789/0 conn(0x7fa6000af9d0 legacy=0x7fa6000b3e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:36.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:36.532+0000 7fa5feffd640 1 -- 172.21.15.27:0/2075222770 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6000ceac0 con 0x7fa60015ec30 2024-09-12T03:23:36.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:36.533+0000 7fa5feffd640 1 --2- 172.21.15.27:0/2075222770 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa60015ec30 0x7fa6000b3900 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7fa60006c7d0 tx=0x7fa5f002c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f2dda85fa162b8e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:36.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:36.533+0000 7fa5fdffb640 1 -- 172.21.15.27:0/2075222770 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5f0009600 con 0x7fa60015ec30 2024-09-12T03:23:36.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:36.533+0000 7fa5fdffb640 1 -- 172.21.15.27:0/2075222770 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5f0037070 con 0x7fa60015ec30 2024-09-12T03:23:36.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:36.533+0000 7fa5fdffb640 1 -- 172.21.15.27:0/2075222770 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5f00099c0 con 0x7fa60015ec30 2024-09-12T03:23:36.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:36.533+0000 7fa6054ae640 1 -- 172.21.15.27:0/2075222770 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa60015ec30 msgr2=0x7fa6000b3900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:36.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:36.533+0000 7fa6054ae640 1 --2- 172.21.15.27:0/2075222770 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa60015ec30 0x7fa6000b3900 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7fa60006c7d0 tx=0x7fa5f002c910 comp rx=0 tx=0).stop 2024-09-12T03:23:36.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:36.533+0000 7fa6054ae640 1 -- 172.21.15.27:0/2075222770 shutdown_connections 2024-09-12T03:23:36.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:36.533+0000 7fa6054ae640 1 --2- 172.21.15.27:0/2075222770 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa60006da00 0x7fa60006de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:36.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:36.533+0000 7fa6054ae640 1 --2- 172.21.15.27:0/2075222770 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa60015ec30 0x7fa6000b3900 unknown :-1 s=CLOSED pgs=1587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:36.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:36.533+0000 7fa6054ae640 1 -- 172.21.15.27:0/2075222770 >> 172.21.15.27:0/2075222770 conn(0x7fa60006bb30 msgr2=0x7fa60006bf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:36.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:36.533+0000 7fa6054ae640 1 -- 172.21.15.27:0/2075222770 shutdown_connections 2024-09-12T03:23:36.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:36.533+0000 7fa6054ae640 1 -- 172.21.15.27:0/2075222770 wait complete. 2024-09-12T03:23:36.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:36.535+0000 7fa6054ae640 1 Processor -- start 2024-09-12T03:23:36.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:36.535+0000 7fa6054ae640 1 -- start start 2024-09-12T03:23:36.538 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:36.535+0000 7fa6054ae640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa60006da00 0x7fa6001460d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:36.538 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:36.535+0000 7fa6054ae640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa6000af9d0 0x7fa60015b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:36.538 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:36.535+0000 7fa6054ae640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa60015ec30 0x7fa600159b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:36.538 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:36.535+0000 7fa6054ae640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fa6000cf190 con 0x7fa60006da00 2024-09-12T03:23:36.538 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:36.535+0000 7fa6054ae640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa6000cf010 con 0x7fa60015ec30 2024-09-12T03:23:36.538 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:36.536+0000 7fa6054ae640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fa6000cf310 con 0x7fa6000af9d0 2024-09-12T03:23:36.538 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:36.536+0000 7fa5feffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa60006da00 0x7fa6001460d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:36.538 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:36.536+0000 7fa5feffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa60006da00 0x7fa6001460d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34790/0 (socket says 172.21.15.27:34790) 2024-09-12T03:23:36.538 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:36.536+0000 7fa5feffd640 1 -- 172.21.15.27:0/3190094201 learned_addr learned my addr 172.21.15.27:0/3190094201 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:23:36.538 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:36.536+0000 7fa5ff7fe640 1 --2- 172.21.15.27:0/3190094201 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa60015ec30 0x7fa600159b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:36.538 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:36.536+0000 7fa5fe7fc640 1 --2- 172.21.15.27:0/3190094201 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa6000af9d0 0x7fa60015b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:36.538 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:36.536+0000 7fa5feffd640 1 -- 172.21.15.27:0/3190094201 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa6000af9d0 msgr2=0x7fa60015b6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:36.538 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:36.536+0000 7fa5feffd640 1 --2- 172.21.15.27:0/3190094201 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa6000af9d0 0x7fa60015b6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:36.539 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:36.536+0000 7fa5feffd640 1 -- 172.21.15.27:0/3190094201 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa60015ec30 msgr2=0x7fa600159b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:36.539 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:36.536+0000 7fa5feffd640 1 --2- 172.21.15.27:0/3190094201 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa60015ec30 0x7fa600159b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:36.539 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:36.536+0000 7fa5feffd640 1 -- 172.21.15.27:0/3190094201 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa60015e270 con 0x7fa60006da00 2024-09-12T03:23:36.539 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:36.536+0000 7fa5feffd640 1 --2- 172.21.15.27:0/3190094201 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa60006da00 0x7fa6001460d0 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7fa5f0034040 tx=0x7fa5f0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:36.539 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:36.536+0000 7fa5f77fe640 1 -- 172.21.15.27:0/3190094201 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5f000b720 con 0x7fa60006da00 2024-09-12T03:23:36.539 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:36.536+0000 7fa5f77fe640 1 -- 172.21.15.27:0/3190094201 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5f0032810 con 0x7fa60006da00 2024-09-12T03:23:36.539 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:36.536+0000 7fa6054ae640 1 -- 172.21.15.27:0/3190094201 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa60015bc10 con 0x7fa60006da00 2024-09-12T03:23:36.539 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:36.536+0000 7fa5f77fe640 1 -- 172.21.15.27:0/3190094201 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5f0032c00 con 0x7fa60006da00 2024-09-12T03:23:36.539 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:36.536+0000 7fa6054ae640 1 -- 172.21.15.27:0/3190094201 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6001437a0 con 0x7fa60006da00 2024-09-12T03:23:36.539 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:36.537+0000 7fa6054ae640 1 -- 172.21.15.27:0/3190094201 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6000695a0 con 0x7fa60006da00 2024-09-12T03:23:36.539 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:36.538+0000 7fa5f77fe640 1 -- 172.21.15.27:0/3190094201 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 24) ==== 91212+0+0 (secure 0 0 0) 0x7fa5f0043020 con 0x7fa60006da00 2024-09-12T03:23:36.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:36.538+0000 7fa5f77fe640 1 --2- 172.21.15.27:0/3190094201 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fa5e006c610 0x7fa5e006ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:36.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:36.538+0000 7fa5f77fe640 1 -- 172.21.15.27:0/3190094201 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 5765+0+0 (secure 0 0 0) 0x7fa5f00aa900 con 0x7fa60006da00 2024-09-12T03:23:36.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:36.538+0000 7fa5fe7fc640 1 --2- 172.21.15.27:0/3190094201 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fa5e006c610 0x7fa5e006ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:36.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:36.539+0000 7fa5fe7fc640 1 --2- 172.21.15.27:0/3190094201 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fa5e006c610 0x7fa5e006ead0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fa6000b3f50 tx=0x7fa5e8009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:36.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:36.543+0000 7fa5f77fe640 1 -- 172.21.15.27:0/3190094201 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fa60015e270 con 0x7fa60006da00 2024-09-12T03:23:36.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:36.746+0000 7fa6054ae640 1 -- 172.21.15.27:0/3190094201 --> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol27051", "group_name": "subvol_grp73700", "target": ["mon-mgr", ""]}) -- 0x7fa60015e450 con 0x7fa5e006c610 2024-09-12T03:23:36.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:36.785+0000 7fa5f77fe640 1 -- 172.21.15.27:0/3190094201 <== mgr.4113 v2:172.21.15.27:6836/1226920526 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+74 (secure 0 0 0) 0x7fa60015e450 con 0x7fa5e006c610 2024-09-12T03:23:36.785 INFO:teuthology.orchestra.run.smithi027.stdout:/volumes/subvol_grp73700/subvol27051/9b490ef8-e617-46c8-b41a-5c4cd66187a8 2024-09-12T03:23:36.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:36.787+0000 7fa6054ae640 1 -- 172.21.15.27:0/3190094201 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fa5e006c610 msgr2=0x7fa5e006ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:36.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:36.787+0000 7fa6054ae640 1 --2- 172.21.15.27:0/3190094201 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fa5e006c610 0x7fa5e006ead0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fa6000b3f50 tx=0x7fa5e8009040 comp rx=0 tx=0).stop 2024-09-12T03:23:36.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:36.787+0000 7fa6054ae640 1 -- 172.21.15.27:0/3190094201 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa60006da00 msgr2=0x7fa6001460d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:36.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:36.787+0000 7fa6054ae640 1 --2- 172.21.15.27:0/3190094201 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa60006da00 0x7fa6001460d0 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7fa5f0034040 tx=0x7fa5f0004a40 comp rx=0 tx=0).stop 2024-09-12T03:23:36.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:36.787+0000 7fa6054ae640 1 -- 172.21.15.27:0/3190094201 shutdown_connections 2024-09-12T03:23:36.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:36.787+0000 7fa6054ae640 1 --2- 172.21.15.27:0/3190094201 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa60015ec30 0x7fa600159b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:36.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:36.787+0000 7fa6054ae640 1 --2- 172.21.15.27:0/3190094201 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa6000af9d0 0x7fa60015b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:36.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:36.787+0000 7fa6054ae640 1 --2- 172.21.15.27:0/3190094201 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fa5e006c610 0x7fa5e006ead0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:36.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:36.787+0000 7fa6054ae640 1 --2- 172.21.15.27:0/3190094201 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa60006da00 0x7fa6001460d0 unknown :-1 s=CLOSED pgs=1588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:36.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:36.787+0000 7fa6054ae640 1 -- 172.21.15.27:0/3190094201 >> 172.21.15.27:0/3190094201 conn(0x7fa60006bb30 msgr2=0x7fa6000b1590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:36.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:36.787+0000 7fa6054ae640 1 -- 172.21.15.27:0/3190094201 shutdown_connections 2024-09-12T03:23:36.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:36.787+0000 7fa6054ae640 1 -- 172.21.15.27:0/3190094201 wait complete. 2024-09-12T03:23:36.798 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:23:36.798 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cat ./volumes/_nogroup/subvol19020/.meta) 2024-09-12T03:23:36.824 INFO:teuthology.orchestra.run.smithi027.stdout:[GLOBAL] 2024-09-12T03:23:36.824 INFO:teuthology.orchestra.run.smithi027.stdout:version = 2 2024-09-12T03:23:36.824 INFO:teuthology.orchestra.run.smithi027.stdout:type = subvolume 2024-09-12T03:23:36.824 INFO:teuthology.orchestra.run.smithi027.stdout:path = /volumes/_nogroup/subvol19020/9746d73a-996d-412d-a829-a3e262df9432 2024-09-12T03:23:36.824 INFO:teuthology.orchestra.run.smithi027.stdout:state = complete 2024-09-12T03:23:36.824 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:23:36.825 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:23:36.826 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cat ./volumes/subvol_grp73700/subvol27051/.meta) 2024-09-12T03:23:36.892 INFO:teuthology.orchestra.run.smithi027.stdout:[GLOBAL] 2024-09-12T03:23:36.893 INFO:teuthology.orchestra.run.smithi027.stdout:version = 2 2024-09-12T03:23:36.893 INFO:teuthology.orchestra.run.smithi027.stdout:type = subvolume 2024-09-12T03:23:36.893 INFO:teuthology.orchestra.run.smithi027.stdout:path = /volumes/subvol_grp73700/subvol27051/9b490ef8-e617-46c8-b41a-5c4cd66187a8 2024-09-12T03:23:36.893 INFO:teuthology.orchestra.run.smithi027.stdout:state = complete 2024-09-12T03:23:36.893 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:23:36.894 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol19020 2024-09-12T03:23:37.039 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:37.039+0000 7fa697f9f640 1 Processor -- start 2024-09-12T03:23:37.039 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:37.039+0000 7fa697f9f640 1 -- start start 2024-09-12T03:23:37.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:37.040+0000 7fa697f9f640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fa69015ec30 0x7fa6900b3900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:37.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:37.040+0000 7fa697f9f640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa6900af9d0 0x7fa6900b3e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:37.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:37.040+0000 7fa697f9f640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa6900ce5e0 con 0x7fa6900b4380 2024-09-12T03:23:37.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:37.040+0000 7fa697f9f640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7fa6900d02d0 con 0x7fa69015ec30 2024-09-12T03:23:37.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:37.040+0000 7fa697f9f640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fa69006c650 con 0x7fa6900af9d0 2024-09-12T03:23:37.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:37.040+0000 7fa695d3b640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fa69015ec30 0x7fa6900b3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:37.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:37.040+0000 7fa695d3b640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fa69015ec30 0x7fa6900b3900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34802/0 (socket says 172.21.15.27:34802) 2024-09-12T03:23:37.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:37.040+0000 7fa695d3b640 1 -- 172.21.15.27:0/2577697779 learned_addr learned my addr 172.21.15.27:0/2577697779 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:23:37.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:37.040+0000 7fa69553a640 1 --2- 172.21.15.27:0/2577697779 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa6900af9d0 0x7fa6900b3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:37.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:37.041+0000 7fa695d3b640 1 -- 172.21.15.27:0/2577697779 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa6900af9d0 msgr2=0x7fa6900b3e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:37.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:37.041+0000 7fa695d3b640 1 --2- 172.21.15.27:0/2577697779 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa6900af9d0 0x7fa6900b3e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:37.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:37.041+0000 7fa695d3b640 1 -- 172.21.15.27:0/2577697779 >> v1:172.21.15.45:6789/0 conn(0x7fa6900b4380 legacy=0x7fa69015f2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:37.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:37.041+0000 7fa695d3b640 1 -- 172.21.15.27:0/2577697779 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa69015f010 con 0x7fa69015ec30 2024-09-12T03:23:37.043 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:37.041+0000 7fa695d3b640 1 --2- 172.21.15.27:0/2577697779 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa69015ec30 0x7fa6900b3900 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7fa680009e50 tx=0x7fa68002a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=692e0db178537df1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:37.043 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:37.041+0000 7fa694d39640 1 -- 172.21.15.27:0/2577697779 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa680036070 con 0x7fa69015ec30 2024-09-12T03:23:37.043 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:37.042+0000 7fa694d39640 1 -- 172.21.15.27:0/2577697779 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa680031600 con 0x7fa69015ec30 2024-09-12T03:23:37.043 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:37.042+0000 7fa694d39640 1 -- 172.21.15.27:0/2577697779 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6800319d0 con 0x7fa69015ec30 2024-09-12T03:23:37.043 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:37.042+0000 7fa697f9f640 1 -- 172.21.15.27:0/2577697779 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa69015ec30 msgr2=0x7fa6900b3900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:37.043 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:37.042+0000 7fa697f9f640 1 --2- 172.21.15.27:0/2577697779 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa69015ec30 0x7fa6900b3900 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7fa680009e50 tx=0x7fa68002a710 comp rx=0 tx=0).stop 2024-09-12T03:23:37.043 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:37.042+0000 7fa697f9f640 1 -- 172.21.15.27:0/2577697779 shutdown_connections 2024-09-12T03:23:37.043 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:37.042+0000 7fa697f9f640 1 --2- 172.21.15.27:0/2577697779 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa6900af9d0 0x7fa6900b3e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:37.043 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:37.042+0000 7fa697f9f640 1 --2- 172.21.15.27:0/2577697779 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa69015ec30 0x7fa6900b3900 unknown :-1 s=CLOSED pgs=1589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:37.043 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:37.042+0000 7fa697f9f640 1 -- 172.21.15.27:0/2577697779 >> 172.21.15.27:0/2577697779 conn(0x7fa69006bb30 msgr2=0x7fa69006bf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:37.043 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:37.042+0000 7fa697f9f640 1 -- 172.21.15.27:0/2577697779 shutdown_connections 2024-09-12T03:23:37.043 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:37.042+0000 7fa697f9f640 1 -- 172.21.15.27:0/2577697779 wait complete. 2024-09-12T03:23:37.046 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:37.043+0000 7fa697f9f640 1 Processor -- start 2024-09-12T03:23:37.046 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:37.043+0000 7fa697f9f640 1 -- start start 2024-09-12T03:23:37.046 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:37.044+0000 7fa697f9f640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa6900af9d0 0x7fa690147740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:37.046 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:37.044+0000 7fa697f9f640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa6900b4380 0x7fa69014ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:37.046 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:37.044+0000 7fa697f9f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa69015ec30 0x7fa690135c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:37.046 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:37.044+0000 7fa697f9f640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fa6900715f0 con 0x7fa6900af9d0 2024-09-12T03:23:37.046 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:37.044+0000 7fa697f9f640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa6900cfff0 con 0x7fa69015ec30 2024-09-12T03:23:37.046 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:37.044+0000 7fa697f9f640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fa69015f9b0 con 0x7fa6900b4380 2024-09-12T03:23:37.046 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:37.044+0000 7fa695d3b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa6900af9d0 0x7fa690147740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:37.046 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:37.044+0000 7fa695d3b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa6900af9d0 0x7fa690147740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34814/0 (socket says 172.21.15.27:34814) 2024-09-12T03:23:37.046 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:37.044+0000 7fa695d3b640 1 -- 172.21.15.27:0/1097488721 learned_addr learned my addr 172.21.15.27:0/1097488721 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:23:37.046 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:37.044+0000 7fa69553a640 1 --2- 172.21.15.27:0/1097488721 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa6900b4380 0x7fa69014ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:37.046 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:37.044+0000 7fa69653c640 1 --2- 172.21.15.27:0/1097488721 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa69015ec30 0x7fa690135c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:37.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:37.044+0000 7fa695d3b640 1 -- 172.21.15.27:0/1097488721 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa6900b4380 msgr2=0x7fa69014ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:37.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:37.044+0000 7fa695d3b640 1 --2- 172.21.15.27:0/1097488721 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa6900b4380 0x7fa69014ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:37.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:37.044+0000 7fa695d3b640 1 -- 172.21.15.27:0/1097488721 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa69015ec30 msgr2=0x7fa690135c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:37.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:37.044+0000 7fa695d3b640 1 --2- 172.21.15.27:0/1097488721 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa69015ec30 0x7fa690135c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:37.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:37.044+0000 7fa695d3b640 1 -- 172.21.15.27:0/1097488721 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6901361a0 con 0x7fa6900af9d0 2024-09-12T03:23:37.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:37.045+0000 7fa69653c640 1 --2- 172.21.15.27:0/1097488721 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa69015ec30 0x7fa690135c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:23:37.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:37.045+0000 7fa695d3b640 1 --2- 172.21.15.27:0/1097488721 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa6900af9d0 0x7fa690147740 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7fa680000f80 tx=0x7fa680004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:37.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:37.045+0000 7fa68e7fc640 1 -- 172.21.15.27:0/1097488721 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa680036050 con 0x7fa6900af9d0 2024-09-12T03:23:37.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:37.045+0000 7fa68e7fc640 1 -- 172.21.15.27:0/1097488721 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa680042070 con 0x7fa6900af9d0 2024-09-12T03:23:37.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:37.045+0000 7fa697f9f640 1 -- 172.21.15.27:0/1097488721 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa690147e20 con 0x7fa6900af9d0 2024-09-12T03:23:37.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:37.045+0000 7fa68e7fc640 1 -- 172.21.15.27:0/1097488721 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa680009510 con 0x7fa6900af9d0 2024-09-12T03:23:37.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:37.045+0000 7fa697f9f640 1 -- 172.21.15.27:0/1097488721 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6901483e0 con 0x7fa6900af9d0 2024-09-12T03:23:37.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:37.046+0000 7fa697f9f640 1 -- 172.21.15.27:0/1097488721 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa69015fd70 con 0x7fa6900af9d0 2024-09-12T03:23:37.048 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:37.046+0000 7fa68e7fc640 1 -- 172.21.15.27:0/1097488721 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 24) ==== 91212+0+0 (secure 0 0 0) 0x7fa680007c70 con 0x7fa6900af9d0 2024-09-12T03:23:37.048 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:37.047+0000 7fa68e7fc640 1 --2- 172.21.15.27:0/1097488721 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fa67c06c610 0x7fa67c06ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:37.048 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:37.047+0000 7fa69553a640 1 --2- 172.21.15.27:0/1097488721 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fa67c06c610 0x7fa67c06ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:37.048 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:37.047+0000 7fa68e7fc640 1 -- 172.21.15.27:0/1097488721 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 5765+0+0 (secure 0 0 0) 0x7fa6800aa600 con 0x7fa6900af9d0 2024-09-12T03:23:37.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:37.047+0000 7fa69553a640 1 --2- 172.21.15.27:0/1097488721 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fa67c06c610 0x7fa67c06ead0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fa6900af720 tx=0x7fa684008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:37.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:37.051+0000 7fa68e7fc640 1 -- 172.21.15.27:0/1097488721 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fa68007a8a0 con 0x7fa6900af9d0 2024-09-12T03:23:37.251 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:37.252+0000 7fa697f9f640 1 -- 172.21.15.27:0/1097488721 --> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol19020", "target": ["mon-mgr", ""]}) -- 0x7fa69006f240 con 0x7fa67c06c610 2024-09-12T03:23:37.262 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:37.263+0000 7fa68e7fc640 1 -- 172.21.15.27:0/1097488721 <== mgr.4113 v2:172.21.15.27:6836/1226920526 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fa69006f240 con 0x7fa67c06c610 2024-09-12T03:23:37.266 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:37.266+0000 7fa697f9f640 1 -- 172.21.15.27:0/1097488721 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fa67c06c610 msgr2=0x7fa67c06ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:37.266 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:37.266+0000 7fa697f9f640 1 --2- 172.21.15.27:0/1097488721 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fa67c06c610 0x7fa67c06ead0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fa6900af720 tx=0x7fa684008040 comp rx=0 tx=0).stop 2024-09-12T03:23:37.266 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:37.266+0000 7fa697f9f640 1 -- 172.21.15.27:0/1097488721 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa6900af9d0 msgr2=0x7fa690147740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:37.266 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:37.266+0000 7fa697f9f640 1 --2- 172.21.15.27:0/1097488721 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa6900af9d0 0x7fa690147740 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7fa680000f80 tx=0x7fa680004a40 comp rx=0 tx=0).stop 2024-09-12T03:23:37.266 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:37.266+0000 7fa697f9f640 1 -- 172.21.15.27:0/1097488721 shutdown_connections 2024-09-12T03:23:37.266 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:37.266+0000 7fa697f9f640 1 --2- 172.21.15.27:0/1097488721 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa69015ec30 0x7fa690135c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:37.266 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:37.266+0000 7fa697f9f640 1 --2- 172.21.15.27:0/1097488721 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa6900b4380 0x7fa69014ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:37.266 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:37.266+0000 7fa697f9f640 1 --2- 172.21.15.27:0/1097488721 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fa67c06c610 0x7fa67c06ead0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:37.267 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:37.266+0000 7fa697f9f640 1 --2- 172.21.15.27:0/1097488721 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa6900af9d0 0x7fa690147740 unknown :-1 s=CLOSED pgs=1590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:37.267 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:37.266+0000 7fa697f9f640 1 -- 172.21.15.27:0/1097488721 >> 172.21.15.27:0/1097488721 conn(0x7fa69006bb30 msgr2=0x7fa6900b36d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:37.267 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:37.266+0000 7fa697f9f640 1 -- 172.21.15.27:0/1097488721 shutdown_connections 2024-09-12T03:23:37.267 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:37.266+0000 7fa697f9f640 1 -- 172.21.15.27:0/1097488721 wait complete. 2024-09-12T03:23:37.268 INFO:tasks.ceph.mgr.y.smithi027.stderr:2024-09-12T03:23:37.268+0000 7f325f8e7640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:23:37.268 INFO:tasks.ceph.mgr.y.smithi027.stderr:2024-09-12T03:23:37.268+0000 7f325f8e7640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:23:37.268 INFO:tasks.ceph.mgr.y.smithi027.stderr:2024-09-12T03:23:37.268+0000 7f325f8e7640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:23:37.268 INFO:tasks.ceph.mgr.y.smithi027.stderr:2024-09-12T03:23:37.268+0000 7f325f8e7640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:23:37.268 INFO:tasks.ceph.mgr.y.smithi027.stderr:2024-09-12T03:23:37.268+0000 7f325f8e7640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:23:37.279 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol27051 subvol_grp73700 2024-09-12T03:23:37.281 INFO:tasks.ceph.mgr.y.smithi027.stderr:2024-09-12T03:23:37.281+0000 7f32600e8640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:23:37.281 INFO:tasks.ceph.mgr.y.smithi027.stderr:2024-09-12T03:23:37.281+0000 7f32600e8640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:23:37.281 INFO:tasks.ceph.mgr.y.smithi027.stderr:2024-09-12T03:23:37.281+0000 7f32600e8640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:23:37.281 INFO:tasks.ceph.mgr.y.smithi027.stderr:2024-09-12T03:23:37.281+0000 7f32600e8640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:23:37.282 INFO:tasks.ceph.mgr.y.smithi027.stderr:2024-09-12T03:23:37.281+0000 7f32600e8640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:23:37.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:37.387+0000 7f52d1cdd640 1 Processor -- start 2024-09-12T03:23:37.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:37.387+0000 7f52d1cdd640 1 -- start start 2024-09-12T03:23:37.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:37.388+0000 7f52d1cdd640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f52cc15e900 0x7f52cc0af370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:37.389 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:37.388+0000 7f52d1cdd640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52cc0b5940 con 0x7f52cc0b6540 2024-09-12T03:23:37.389 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:37.388+0000 7f52d1cdd640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52cc0af950 con 0x7f52cc06d730 2024-09-12T03:23:37.389 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:37.388+0000 7f52d1cdd640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f52cc15ece0 con 0x7f52cc15e900 2024-09-12T03:23:37.389 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:37.388+0000 7f52cbfff640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f52cc0b6540 0x7f52cc06ee80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:51336/0 (socket says 172.21.15.27:51336) 2024-09-12T03:23:37.389 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:37.388+0000 7f52cbfff640 1 -- 172.21.15.27:0/1737294357 learned_addr learned my addr 172.21.15.27:0/1737294357 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:23:37.389 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:37.388+0000 7f52cb7fe640 1 --2- 172.21.15.27:0/1737294357 >> v2:172.21.15.45:3300/0 conn(0x7f52cc15e900 0x7f52cc0af370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:37.389 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:37.388+0000 7f52ca7fc640 1 -- 172.21.15.27:0/1737294357 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1423260783 0 0) 0x7f52c4001490 con 0x7f52cc0b6540 2024-09-12T03:23:37.389 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:37.388+0000 7f52ca7fc640 1 -- 172.21.15.27:0/1737294357 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f52b4003520 con 0x7f52cc0b6540 2024-09-12T03:23:37.389 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:37.389+0000 7f52cb7fe640 1 -- 172.21.15.27:0/1737294357 >> v1:172.21.15.45:6789/0 conn(0x7f52cc06d730 legacy=0x7f52cc0b5e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:37.390 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:37.389+0000 7f52cb7fe640 1 -- 172.21.15.27:0/1737294357 >> v1:172.21.15.27:6789/0 conn(0x7f52cc0b6540 legacy=0x7f52cc06ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:37.390 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:37.389+0000 7f52ca7fc640 1 -- 172.21.15.27:0/1737294357 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1748539865 0 0) 0x7f52b4003520 con 0x7f52cc0b6540 2024-09-12T03:23:37.390 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:37.389+0000 7f52cb7fe640 1 -- 172.21.15.27:0/1737294357 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52cc0afc30 con 0x7f52cc15e900 2024-09-12T03:23:37.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:37.390+0000 7f52cb7fe640 1 --2- 172.21.15.27:0/1737294357 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f52cc15e900 0x7f52cc0af370 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f52bc0097a0 tx=0x7f52bc02a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c5488cf1a71bad29 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:37.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:37.390+0000 7f52ca7fc640 1 -- 172.21.15.27:0/1737294357 <== mon.1 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52bc037070 con 0x7f52cc15e900 2024-09-12T03:23:37.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:37.390+0000 7f52ca7fc640 1 -- 172.21.15.27:0/1737294357 <== mon.1 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52bc02ac50 con 0x7f52cc15e900 2024-09-12T03:23:37.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:37.390+0000 7f52ca7fc640 1 -- 172.21.15.27:0/1737294357 <== mon.1 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52c4001bc0 con 0x7f52cc15e900 2024-09-12T03:23:37.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:37.390+0000 7f52d1cdd640 1 -- 172.21.15.27:0/1737294357 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f52cc15e900 msgr2=0x7f52cc0af370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:37.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:37.390+0000 7f52d1cdd640 1 --2- 172.21.15.27:0/1737294357 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f52cc15e900 0x7f52cc0af370 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f52bc0097a0 tx=0x7f52bc02a1c0 comp rx=0 tx=0).stop 2024-09-12T03:23:37.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:37.390+0000 7f52d1cdd640 1 -- 172.21.15.27:0/1737294357 shutdown_connections 2024-09-12T03:23:37.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:37.390+0000 7f52d1cdd640 1 --2- 172.21.15.27:0/1737294357 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f52cc15e900 0x7f52cc0af370 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:37.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:37.390+0000 7f52d1cdd640 1 -- 172.21.15.27:0/1737294357 >> 172.21.15.27:0/1737294357 conn(0x7f52cc06cc40 msgr2=0x7f52cc06d020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:37.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:37.390+0000 7f52d1cdd640 1 -- 172.21.15.27:0/1737294357 shutdown_connections 2024-09-12T03:23:37.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:37.390+0000 7f52d1cdd640 1 -- 172.21.15.27:0/1737294357 wait complete. 2024-09-12T03:23:37.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:37.391+0000 7f52d1cdd640 1 Processor -- start 2024-09-12T03:23:37.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:37.392+0000 7f52d1cdd640 1 -- start start 2024-09-12T03:23:37.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:37.392+0000 7f52d1cdd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f52cc06d730 0x7f52cc161930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:37.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:37.392+0000 7f52d1cdd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f52cc0b6540 0x7f52cc15f880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:37.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:37.392+0000 7f52d1cdd640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f52cc15e900 0x7f52cc14f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:37.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:37.392+0000 7f52d1cdd640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f52cc06e880 con 0x7f52cc0b6540 2024-09-12T03:23:37.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:37.392+0000 7f52d1cdd640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f52cc0b01f0 con 0x7f52cc06d730 2024-09-12T03:23:37.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:37.392+0000 7f52d1cdd640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f52cc06ea00 con 0x7f52cc15e900 2024-09-12T03:23:37.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:37.392+0000 7f52caffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f52cc0b6540 0x7f52cc15f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:37.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:37.393+0000 7f52caffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f52cc0b6540 0x7f52cc15f880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34868/0 (socket says 172.21.15.27:34868) 2024-09-12T03:23:37.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:37.393+0000 7f52caffd640 1 -- 172.21.15.27:0/2317909720 learned_addr learned my addr 172.21.15.27:0/2317909720 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:23:37.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:37.393+0000 7f52cb7fe640 1 --2- 172.21.15.27:0/2317909720 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f52cc06d730 0x7f52cc161930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:37.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:37.393+0000 7f52cbfff640 1 --2- 172.21.15.27:0/2317909720 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f52cc15e900 0x7f52cc14f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:37.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:37.393+0000 7f52caffd640 1 -- 172.21.15.27:0/2317909720 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f52cc15e900 msgr2=0x7f52cc14f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:37.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:37.393+0000 7f52caffd640 1 --2- 172.21.15.27:0/2317909720 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f52cc15e900 0x7f52cc14f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:37.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:37.393+0000 7f52caffd640 1 -- 172.21.15.27:0/2317909720 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f52cc06d730 msgr2=0x7f52cc161930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:37.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:37.393+0000 7f52caffd640 1 --2- 172.21.15.27:0/2317909720 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f52cc06d730 0x7f52cc161930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:37.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:37.393+0000 7f52caffd640 1 -- 172.21.15.27:0/2317909720 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52cc160890 con 0x7f52cc0b6540 2024-09-12T03:23:37.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:37.393+0000 7f52caffd640 1 --2- 172.21.15.27:0/2317909720 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f52cc0b6540 0x7f52cc15f880 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7f52c000c450 tx=0x7f52c000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:37.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:37.393+0000 7f52b3fff640 1 -- 172.21.15.27:0/2317909720 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52c0017070 con 0x7f52cc0b6540 2024-09-12T03:23:37.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:37.393+0000 7f52b3fff640 1 -- 172.21.15.27:0/2317909720 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52c00052d0 con 0x7f52cc0b6540 2024-09-12T03:23:37.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:37.393+0000 7f52b3fff640 1 -- 172.21.15.27:0/2317909720 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52c0003830 con 0x7f52cc0b6540 2024-09-12T03:23:37.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:37.393+0000 7f52d1cdd640 1 -- 172.21.15.27:0/2317909720 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52cc160b70 con 0x7f52cc0b6540 2024-09-12T03:23:37.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:37.394+0000 7f52d1cdd640 1 -- 172.21.15.27:0/2317909720 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52cc161130 con 0x7f52cc0b6540 2024-09-12T03:23:37.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:37.394+0000 7f52d1cdd640 1 -- 172.21.15.27:0/2317909720 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52cc0af370 con 0x7f52cc0b6540 2024-09-12T03:23:37.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:37.395+0000 7f52b3fff640 1 -- 172.21.15.27:0/2317909720 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 24) ==== 91212+0+0 (secure 0 0 0) 0x7f52c001a070 con 0x7f52cc0b6540 2024-09-12T03:23:37.397 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:37.395+0000 7f52b3fff640 1 --2- 172.21.15.27:0/2317909720 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f52b806c640 0x7f52b806eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:37.397 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:37.396+0000 7f52b3fff640 1 -- 172.21.15.27:0/2317909720 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 5765+0+0 (secure 0 0 0) 0x7f52c0088b10 con 0x7f52cc0b6540 2024-09-12T03:23:37.397 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:37.396+0000 7f52cb7fe640 1 --2- 172.21.15.27:0/2317909720 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f52b806c640 0x7f52b806eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:37.397 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:37.396+0000 7f52cb7fe640 1 --2- 172.21.15.27:0/2317909720 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f52b806c640 0x7f52b806eb00 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f52bc034000 tx=0x7f52bc02e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:37.400 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:37.400+0000 7f52b3fff640 1 -- 172.21.15.27:0/2317909720 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f52c0058760 con 0x7f52cc0b6540 2024-09-12T03:23:37.600 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:37.600+0000 7f52d1cdd640 1 -- 172.21.15.27:0/2317909720 --> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol27051", "group_name": "subvol_grp73700", "target": ["mon-mgr", ""]}) -- 0x7f52cc0b1a90 con 0x7f52b806c640 2024-09-12T03:23:38.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:38.880+0000 7f52b3fff640 1 -- 172.21.15.27:0/2317909720 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mgrmap(e 25) ==== 91365+0+0 (secure 0 0 0) 0x7f52c0054dd0 con 0x7f52cc0b6540 2024-09-12T03:23:41.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:41.672+0000 7f52b3fff640 1 -- 172.21.15.27:0/2317909720 <== mgr.4113 v2:172.21.15.27:6836/1226920526 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f52cc0b1a90 con 0x7f52b806c640 2024-09-12T03:23:41.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:41.676+0000 7f52d1cdd640 1 -- 172.21.15.27:0/2317909720 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f52b806c640 msgr2=0x7f52b806eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:41.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:41.676+0000 7f52d1cdd640 1 --2- 172.21.15.27:0/2317909720 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f52b806c640 0x7f52b806eb00 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f52bc034000 tx=0x7f52bc02e000 comp rx=0 tx=0).stop 2024-09-12T03:23:41.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:41.676+0000 7f52d1cdd640 1 -- 172.21.15.27:0/2317909720 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f52cc0b6540 msgr2=0x7f52cc15f880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:41.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:41.676+0000 7f52d1cdd640 1 --2- 172.21.15.27:0/2317909720 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f52cc0b6540 0x7f52cc15f880 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7f52c000c450 tx=0x7f52c000c910 comp rx=0 tx=0).stop 2024-09-12T03:23:41.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:41.676+0000 7f52d1cdd640 1 -- 172.21.15.27:0/2317909720 shutdown_connections 2024-09-12T03:23:41.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:41.676+0000 7f52d1cdd640 1 --2- 172.21.15.27:0/2317909720 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f52cc15e900 0x7f52cc14f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:41.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:41.676+0000 7f52d1cdd640 1 --2- 172.21.15.27:0/2317909720 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f52b806c640 0x7f52b806eb00 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:41.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:41.676+0000 7f52d1cdd640 1 --2- 172.21.15.27:0/2317909720 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f52cc0b6540 0x7f52cc15f880 unknown :-1 s=CLOSED pgs=1596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:41.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:41.676+0000 7f52d1cdd640 1 --2- 172.21.15.27:0/2317909720 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f52cc06d730 0x7f52cc161930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:41.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:41.676+0000 7f52d1cdd640 1 -- 172.21.15.27:0/2317909720 >> 172.21.15.27:0/2317909720 conn(0x7f52cc06cc40 msgr2=0x7f52cc06bb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:41.677 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:41.676+0000 7f52d1cdd640 1 -- 172.21.15.27:0/2317909720 shutdown_connections 2024-09-12T03:23:41.677 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:41.676+0000 7f52d1cdd640 1 -- 172.21.15.27:0/2317909720 wait complete. 2024-09-12T03:23:41.690 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-09-12T03:23:41.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:41 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-12T03:23:41.750 INFO:teuthology.orchestra.run.smithi027.stdout:2 2024-09-12T03:23:41.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T03:23:41.758 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-12T03:23:41.758 DEBUG:teuthology.orchestra.run.smithi027:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-12T03:23:41.802 DEBUG:teuthology.orchestra.run.smithi045:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-12T03:23:41.850 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-12T03:23:41.850 INFO:tasks.cephfs.mount:Terminating background process 2024-09-12T03:23:41.851 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-12T03:23:41.851 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T03:23:41.867 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-12T03:23:41.867 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T03:23:41.925 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-12T03:23:42.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:42.069+0000 7f28df349640 1 Processor -- start 2024-09-12T03:23:42.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:42.069+0000 7f28df349640 1 -- start start 2024-09-12T03:23:42.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:42.070+0000 7f28df349640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f28d80b5a70 0x7f28d815f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:42.072 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:42.070+0000 7f28df349640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f28d815fb40 con 0x7f28d8072fc0 2024-09-12T03:23:42.072 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:42.070+0000 7f28df349640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f28d80aee90 con 0x7f28d80ae8d0 2024-09-12T03:23:42.072 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:42.070+0000 7f28df349640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f28d80af170 con 0x7f28d80b5a70 2024-09-12T03:23:42.072 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:42.070+0000 7f28dd8e6640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f28d8072fc0 0x7f28d806e930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:52286/0 (socket says 172.21.15.27:52286) 2024-09-12T03:23:42.072 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:42.070+0000 7f28dd8e6640 1 -- 172.21.15.27:0/3171912203 learned_addr learned my addr 172.21.15.27:0/3171912203 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:23:42.072 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:42.071+0000 7f28dd0e5640 1 --2- 172.21.15.27:0/3171912203 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f28d80b5a70 0x7f28d815f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:42.072 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:42.071+0000 7f28d7fff640 1 -- 172.21.15.27:0/3171912203 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 463548879 0 0) 0x7f28d815fb40 con 0x7f28d8072fc0 2024-09-12T03:23:42.072 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:42.071+0000 7f28d7fff640 1 -- 172.21.15.27:0/3171912203 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f28c0003520 con 0x7f28d8072fc0 2024-09-12T03:23:42.072 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:42.071+0000 7f28d7fff640 1 -- 172.21.15.27:0/3171912203 <== mon.1 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1339332156 0 0) 0x7f28d80aee90 con 0x7f28d80ae8d0 2024-09-12T03:23:42.072 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:42.071+0000 7f28d7fff640 1 -- 172.21.15.27:0/3171912203 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f28d815fb40 con 0x7f28d80ae8d0 2024-09-12T03:23:42.072 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:42.071+0000 7f28d7fff640 1 -- 172.21.15.27:0/3171912203 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3044047962 0 0) 0x7f28c0003520 con 0x7f28d8072fc0 2024-09-12T03:23:42.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:42.071+0000 7f28d7fff640 1 -- 172.21.15.27:0/3171912203 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f28d80b5a70 msgr2=0x7f28d815f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:42.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:42.071+0000 7f28d7fff640 1 --2- 172.21.15.27:0/3171912203 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f28d80b5a70 0x7f28d815f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:42.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:42.071+0000 7f28d7fff640 1 -- 172.21.15.27:0/3171912203 >> v1:172.21.15.45:6789/0 conn(0x7f28d80ae8d0 legacy=0x7f28d80728b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:42.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:42.071+0000 7f28d7fff640 1 -- 172.21.15.27:0/3171912203 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28d80af450 con 0x7f28d8072fc0 2024-09-12T03:23:42.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:42.071+0000 7f28d7fff640 1 -- 172.21.15.27:0/3171912203 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 847755213 0 0) 0x7f28d0001670 con 0x7f28d8072fc0 2024-09-12T03:23:42.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:42.071+0000 7f28d7fff640 1 -- 172.21.15.27:0/3171912203 >> v1:172.21.15.27:6789/0 conn(0x7f28d8072fc0 legacy=0x7f28d806e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:42.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:42.071+0000 7f28d7fff640 1 --2- 172.21.15.27:0/3171912203 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f28c0003b70 0x7f28c0003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:42.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:42.071+0000 7f28d7fff640 1 --2- 172.21.15.27:0/3171912203 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f28d80b5a70 0x7f28c0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:42.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:42.071+0000 7f28d7fff640 1 --2- 172.21.15.27:0/3171912203 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f28c0004d40 0x7f28c00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:42.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:42.071+0000 7f28d7fff640 1 -- 172.21.15.27:0/3171912203 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f28d80af170 con 0x7f28c0004d40 2024-09-12T03:23:42.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:42.071+0000 7f28d7fff640 1 -- 172.21.15.27:0/3171912203 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f28c000bac0 con 0x7f28c0003b70 2024-09-12T03:23:42.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:42.071+0000 7f28d7fff640 1 -- 172.21.15.27:0/3171912203 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f28c000bcc0 con 0x7f28d80b5a70 2024-09-12T03:23:42.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:42.071+0000 7f28dd8e6640 1 --2- 172.21.15.27:0/3171912203 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f28c0004d40 0x7f28c00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:42.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:42.071+0000 7f28dd0e5640 1 --2- 172.21.15.27:0/3171912203 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f28c0003b70 0x7f28c0003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:42.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:42.071+0000 7f28dc8e4640 1 --2- 172.21.15.27:0/3171912203 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f28d80b5a70 0x7f28c0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:42.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:42.071+0000 7f28dd8e6640 1 -- 172.21.15.27:0/3171912203 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f28d80b5a70 msgr2=0x7f28c0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:42.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:42.071+0000 7f28dd8e6640 1 --2- 172.21.15.27:0/3171912203 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f28d80b5a70 0x7f28c0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:42.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:42.071+0000 7f28dd8e6640 1 -- 172.21.15.27:0/3171912203 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f28c0003b70 msgr2=0x7f28c0003f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:42.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:42.071+0000 7f28dd8e6640 1 --2- 172.21.15.27:0/3171912203 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f28c0003b70 0x7f28c0003f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:42.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:42.071+0000 7f28dd8e6640 1 -- 172.21.15.27:0/3171912203 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28c0003520 con 0x7f28c0004d40 2024-09-12T03:23:42.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:42.072+0000 7f28dd8e6640 1 --2- 172.21.15.27:0/3171912203 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f28c0004d40 0x7f28c00051c0 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7f28d8071570 tx=0x7f28c0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3c8cd5d3924fb22 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:42.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:42.072+0000 7f28d7fff640 1 -- 172.21.15.27:0/3171912203 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28d0007ca0 con 0x7f28c0004d40 2024-09-12T03:23:42.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:42.072+0000 7f28d7fff640 1 -- 172.21.15.27:0/3171912203 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28d0005af0 con 0x7f28c0004d40 2024-09-12T03:23:42.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:42.072+0000 7f28d7fff640 1 -- 172.21.15.27:0/3171912203 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28d0007070 con 0x7f28c0004d40 2024-09-12T03:23:42.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:42.073+0000 7f28df349640 1 -- 172.21.15.27:0/3171912203 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f28c0004d40 msgr2=0x7f28c00051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:42.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:42.073+0000 7f28df349640 1 --2- 172.21.15.27:0/3171912203 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f28c0004d40 0x7f28c00051c0 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7f28d8071570 tx=0x7f28c0005830 comp rx=0 tx=0).stop 2024-09-12T03:23:42.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:42.073+0000 7f28dd8e6640 1 -- 172.21.15.27:0/3171912203 reap_dead start 2024-09-12T03:23:42.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:42.073+0000 7f28df349640 1 -- 172.21.15.27:0/3171912203 shutdown_connections 2024-09-12T03:23:42.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:42.073+0000 7f28df349640 1 -- 172.21.15.27:0/3171912203 >> 172.21.15.27:0/3171912203 conn(0x7f28d806ca30 msgr2=0x7f28d806ce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:42.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:42.073+0000 7f28df349640 1 -- 172.21.15.27:0/3171912203 shutdown_connections 2024-09-12T03:23:42.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:42.073+0000 7f28df349640 1 -- 172.21.15.27:0/3171912203 wait complete. 2024-09-12T03:23:42.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:42.074+0000 7f28df349640 1 Processor -- start 2024-09-12T03:23:42.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:42.074+0000 7f28df349640 1 -- start start 2024-09-12T03:23:42.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:42.075+0000 7f28df349640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f28d81402c0 0x7f28d8151480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:42.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:42.075+0000 7f28df349640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f28d80ce680 0x7f28d81519c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:42.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:42.075+0000 7f28df349640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f28d81406a0 0x7f28d8142d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:42.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:42.075+0000 7f28df349640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f28d815f780 con 0x7f28d80ce680 2024-09-12T03:23:42.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:42.075+0000 7f28df349640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f28d815f600 con 0x7f28d81402c0 2024-09-12T03:23:42.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:42.075+0000 7f28df349640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f28d815f900 con 0x7f28d81406a0 2024-09-12T03:23:42.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:42.075+0000 7f28dc8e4640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f28d80ce680 0x7f28d81519c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:42.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:42.075+0000 7f28dc8e4640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f28d80ce680 0x7f28d81519c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59632/0 (socket says 172.21.15.27:59632) 2024-09-12T03:23:42.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:42.075+0000 7f28dc8e4640 1 -- 172.21.15.27:0/2097848928 learned_addr learned my addr 172.21.15.27:0/2097848928 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:23:42.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:42.075+0000 7f28dd0e5640 1 --2- 172.21.15.27:0/2097848928 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f28d81402c0 0x7f28d8151480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:42.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:42.075+0000 7f28dd8e6640 1 --2- 172.21.15.27:0/2097848928 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f28d81406a0 0x7f28d8142d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:42.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:42.075+0000 7f28dc8e4640 1 -- 172.21.15.27:0/2097848928 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f28d81406a0 msgr2=0x7f28d8142d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:42.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:42.075+0000 7f28dc8e4640 1 --2- 172.21.15.27:0/2097848928 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f28d81406a0 0x7f28d8142d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:42.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:42.075+0000 7f28dc8e4640 1 -- 172.21.15.27:0/2097848928 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f28d81402c0 msgr2=0x7f28d8151480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:42.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:42.075+0000 7f28dc8e4640 1 --2- 172.21.15.27:0/2097848928 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f28d81402c0 0x7f28d8151480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:42.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:42.075+0000 7f28dc8e4640 1 -- 172.21.15.27:0/2097848928 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28d8143af0 con 0x7f28d80ce680 2024-09-12T03:23:42.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:42.075+0000 7f28dd8e6640 1 --2- 172.21.15.27:0/2097848928 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f28d81406a0 0x7f28d8142d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:23:42.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:42.075+0000 7f28dd0e5640 1 --2- 172.21.15.27:0/2097848928 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f28d81402c0 0x7f28d8151480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:23:42.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:42.076+0000 7f28dc8e4640 1 --2- 172.21.15.27:0/2097848928 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f28d80ce680 0x7f28d81519c0 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7f28cc00f7a0 tx=0x7f28cc00fc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:42.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:42.076+0000 7f28d5ffb640 1 -- 172.21.15.27:0/2097848928 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28cc00fe50 con 0x7f28d80ce680 2024-09-12T03:23:42.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:42.076+0000 7f28d5ffb640 1 -- 172.21.15.27:0/2097848928 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28cc018070 con 0x7f28d80ce680 2024-09-12T03:23:42.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:42.076+0000 7f28df349640 1 -- 172.21.15.27:0/2097848928 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28d80cd8c0 con 0x7f28d80ce680 2024-09-12T03:23:42.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:42.076+0000 7f28d5ffb640 1 -- 172.21.15.27:0/2097848928 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28cc00d730 con 0x7f28d80ce680 2024-09-12T03:23:42.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:42.076+0000 7f28df349640 1 -- 172.21.15.27:0/2097848928 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28d80cde80 con 0x7f28d80ce680 2024-09-12T03:23:42.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:42.077+0000 7f28df349640 1 -- 172.21.15.27:0/2097848928 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28d80728b0 con 0x7f28d80ce680 2024-09-12T03:23:42.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:42.077+0000 7f28d5ffb640 1 -- 172.21.15.27:0/2097848928 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 25) ==== 91365+0+0 (secure 0 0 0) 0x7f28cc00d8d0 con 0x7f28d80ce680 2024-09-12T03:23:42.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:42.078+0000 7f28d5ffb640 1 --2- 172.21.15.27:0/2097848928 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f28c406c7c0 0x7f28c406ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:42.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:42.078+0000 7f28d5ffb640 1 -- 172.21.15.27:0/2097848928 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 5765+0+0 (secure 0 0 0) 0x7f28cc089580 con 0x7f28d80ce680 2024-09-12T03:23:42.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:42.078+0000 7f28dd0e5640 1 --2- 172.21.15.27:0/2097848928 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f28c406c7c0 0x7f28c406ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:42.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:42.079+0000 7f28dd0e5640 1 --2- 172.21.15.27:0/2097848928 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f28c406c7c0 0x7f28c406ec80 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f28c80074d0 tx=0x7f28c800b040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:42.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:42.083+0000 7f28d5ffb640 1 -- 172.21.15.27:0/2097848928 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f28cc01a310 con 0x7f28d80ce680 2024-09-12T03:23:42.272 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:42.272+0000 7f28df349640 1 -- 172.21.15.27:0/2097848928 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f28d8161440 con 0x7f28d80ce680 2024-09-12T03:23:42.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:42.272+0000 7f28d5ffb640 1 -- 172.21.15.27:0/2097848928 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v149) ==== 80+0+0 (secure 0 0 0) 0x7f28cc059b30 con 0x7f28d80ce680 2024-09-12T03:23:42.273 INFO:teuthology.orchestra.run.smithi027.stderr:listed 0 entries 2024-09-12T03:23:42.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:42.274+0000 7f28df349640 1 -- 172.21.15.27:0/2097848928 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f28c406c7c0 msgr2=0x7f28c406ec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:42.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:42.274+0000 7f28df349640 1 --2- 172.21.15.27:0/2097848928 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f28c406c7c0 0x7f28c406ec80 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f28c80074d0 tx=0x7f28c800b040 comp rx=0 tx=0).stop 2024-09-12T03:23:42.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:42.274+0000 7f28df349640 1 -- 172.21.15.27:0/2097848928 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f28d80ce680 msgr2=0x7f28d81519c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:42.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:42.274+0000 7f28df349640 1 --2- 172.21.15.27:0/2097848928 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f28d80ce680 0x7f28d81519c0 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7f28cc00f7a0 tx=0x7f28cc00fc60 comp rx=0 tx=0).stop 2024-09-12T03:23:42.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:42.275+0000 7f28df349640 1 -- 172.21.15.27:0/2097848928 shutdown_connections 2024-09-12T03:23:42.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:42.275+0000 7f28df349640 1 --2- 172.21.15.27:0/2097848928 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f28d81406a0 0x7f28d8142d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:42.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:42.275+0000 7f28df349640 1 --2- 172.21.15.27:0/2097848928 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f28c406c7c0 0x7f28c406ec80 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:42.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:42.275+0000 7f28df349640 1 --2- 172.21.15.27:0/2097848928 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f28d80ce680 0x7f28d81519c0 unknown :-1 s=CLOSED pgs=1599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:42.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:42.275+0000 7f28df349640 1 --2- 172.21.15.27:0/2097848928 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f28d81402c0 0x7f28d8151480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:42.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:42.275+0000 7f28df349640 1 -- 172.21.15.27:0/2097848928 >> 172.21.15.27:0/2097848928 conn(0x7f28d806ca30 msgr2=0x7f28d806ce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:42.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:42.275+0000 7f28df349640 1 -- 172.21.15.27:0/2097848928 shutdown_connections 2024-09-12T03:23:42.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:42.275+0000 7f28df349640 1 -- 172.21.15.27:0/2097848928 wait complete. 2024-09-12T03:23:42.285 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-12T03:23:42.286 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:23:42.286 DEBUG:teuthology.orchestra.run.smithi027:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-12T03:23:42.330 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi027.front.sepia.ceph.com 2024-09-12T03:23:42.330 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:23:42.330 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-12T03:23:42.360 INFO:tasks.cephfs.mount:Terminating background process 2024-09-12T03:23:42.360 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-12T03:23:42.360 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T03:23:42.378 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-12T03:23:42.378 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T03:23:42.436 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-12T03:23:42.539 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:42.539+0000 7f6ceef03640 1 Processor -- start 2024-09-12T03:23:42.539 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:42.539+0000 7f6ceef03640 1 -- start start 2024-09-12T03:23:42.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:42.539+0000 7f6ceef03640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f6ce815b2b0 0x7f6ce80b3560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:42.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:42.540+0000 7f6ceef03640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f6ce8072d70 0x7f6ce8072430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:42.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:42.540+0000 7f6ceef03640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6ce80ce410 con 0x7f6ce806e9b0 2024-09-12T03:23:42.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:42.540+0000 7f6ceef03640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f6ce80720a0 con 0x7f6ce8072d70 2024-09-12T03:23:42.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:42.540+0000 7f6ceef03640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f6ce8072970 con 0x7f6ce815b2b0 2024-09-12T03:23:42.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:42.540+0000 7f6ce7fff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f6ce8072d70 0x7f6ce8072430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:42.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:42.540+0000 7f6ce7fff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f6ce8072d70 0x7f6ce8072430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59636/0 (socket says 172.21.15.27:59636) 2024-09-12T03:23:42.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:42.540+0000 7f6ce7fff640 1 -- 172.21.15.27:0/1757091008 learned_addr learned my addr 172.21.15.27:0/1757091008 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:23:42.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:42.540+0000 7f6cecc9f640 1 --2- 172.21.15.27:0/1757091008 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f6ce815b2b0 0x7f6ce80b3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:42.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:42.540+0000 7f6ce7fff640 1 -- 172.21.15.27:0/1757091008 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f6ce815b2b0 msgr2=0x7f6ce80b3560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:42.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:42.540+0000 7f6ce7fff640 1 --2- 172.21.15.27:0/1757091008 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f6ce815b2b0 0x7f6ce80b3560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:42.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:42.540+0000 7f6ce7fff640 1 -- 172.21.15.27:0/1757091008 >> v1:172.21.15.27:6789/0 conn(0x7f6ce806e9b0 legacy=0x7f6ce80cebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:42.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:42.540+0000 7f6ce7fff640 1 -- 172.21.15.27:0/1757091008 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ce80732c0 con 0x7f6ce8072d70 2024-09-12T03:23:42.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:42.540+0000 7f6ce77fe640 1 -- 172.21.15.27:0/1757091008 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3733224387 0 0) 0x7f6ce0001490 con 0x7f6ce806e9b0 2024-09-12T03:23:42.543 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:42.541+0000 7f6ce7fff640 1 --2- 172.21.15.27:0/1757091008 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ce8072d70 0x7f6ce8072430 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7f6cd4009870 tx=0x7f6cd402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5917e7c8fc88cda server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:42.543 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:42.541+0000 7f6ce77fe640 1 -- 172.21.15.27:0/1757091008 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cd4037070 con 0x7f6ce8072d70 2024-09-12T03:23:42.543 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:42.541+0000 7f6ce77fe640 1 -- 172.21.15.27:0/1757091008 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6cd4032440 con 0x7f6ce8072d70 2024-09-12T03:23:42.543 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:42.541+0000 7f6ce77fe640 1 -- 172.21.15.27:0/1757091008 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cd4032810 con 0x7f6ce8072d70 2024-09-12T03:23:42.543 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:42.541+0000 7f6ceef03640 1 -- 172.21.15.27:0/1757091008 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ce8072d70 msgr2=0x7f6ce8072430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:42.543 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:42.541+0000 7f6ceef03640 1 --2- 172.21.15.27:0/1757091008 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ce8072d70 0x7f6ce8072430 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7f6cd4009870 tx=0x7f6cd402a270 comp rx=0 tx=0).stop 2024-09-12T03:23:42.543 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:42.542+0000 7f6ceef03640 1 -- 172.21.15.27:0/1757091008 shutdown_connections 2024-09-12T03:23:42.543 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:42.542+0000 7f6ceef03640 1 --2- 172.21.15.27:0/1757091008 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ce8072d70 0x7f6ce8072430 unknown :-1 s=CLOSED pgs=1601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:42.543 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:42.542+0000 7f6ceef03640 1 --2- 172.21.15.27:0/1757091008 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f6ce815b2b0 0x7f6ce80b3560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:42.543 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:42.542+0000 7f6ceef03640 1 -- 172.21.15.27:0/1757091008 >> 172.21.15.27:0/1757091008 conn(0x7f6ce8062c60 msgr2=0x7f6ce806bb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:42.543 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:42.542+0000 7f6ceef03640 1 -- 172.21.15.27:0/1757091008 shutdown_connections 2024-09-12T03:23:42.544 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:42.542+0000 7f6ceef03640 1 -- 172.21.15.27:0/1757091008 wait complete. 2024-09-12T03:23:42.544 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:42.543+0000 7f6ceef03640 1 Processor -- start 2024-09-12T03:23:42.544 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:42.543+0000 7f6ceef03640 1 -- start start 2024-09-12T03:23:42.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:42.544+0000 7f6ceef03640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f6ce806e9b0 0x7f6ce814dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:42.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:42.544+0000 7f6ceef03640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ce8072d70 0x7f6ce80ae8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:42.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:42.544+0000 7f6ceef03640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6ce815b2b0 0x7f6ce80aee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:42.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:42.544+0000 7f6ceef03640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f6ce80ce690 con 0x7f6ce8072d70 2024-09-12T03:23:42.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:42.544+0000 7f6ceef03640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f6ce8071dc0 con 0x7f6ce815b2b0 2024-09-12T03:23:42.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:42.544+0000 7f6ceef03640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f6ce815b690 con 0x7f6ce806e9b0 2024-09-12T03:23:42.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:42.544+0000 7f6ce7fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ce8072d70 0x7f6ce80ae8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:42.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:42.544+0000 7f6ce7fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ce8072d70 0x7f6ce80ae8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59650/0 (socket says 172.21.15.27:59650) 2024-09-12T03:23:42.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:42.544+0000 7f6ce7fff640 1 -- 172.21.15.27:0/3060997015 learned_addr learned my addr 172.21.15.27:0/3060997015 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:23:42.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:42.544+0000 7f6cecc9f640 1 --2- 172.21.15.27:0/3060997015 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f6ce806e9b0 0x7f6ce814dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:42.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:42.544+0000 7f6ced4a0640 1 --2- 172.21.15.27:0/3060997015 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6ce815b2b0 0x7f6ce80aee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:42.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:42.544+0000 7f6ce7fff640 1 -- 172.21.15.27:0/3060997015 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f6ce806e9b0 msgr2=0x7f6ce814dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:42.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:42.544+0000 7f6ce7fff640 1 --2- 172.21.15.27:0/3060997015 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f6ce806e9b0 0x7f6ce814dfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:42.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:42.544+0000 7f6ce7fff640 1 -- 172.21.15.27:0/3060997015 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6ce815b2b0 msgr2=0x7f6ce80aee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:42.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:42.544+0000 7f6ce7fff640 1 --2- 172.21.15.27:0/3060997015 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6ce815b2b0 0x7f6ce80aee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:42.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:42.544+0000 7f6ce7fff640 1 -- 172.21.15.27:0/3060997015 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ce8156170 con 0x7f6ce8072d70 2024-09-12T03:23:42.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:42.544+0000 7f6cecc9f640 1 --2- 172.21.15.27:0/3060997015 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f6ce806e9b0 0x7f6ce814dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:23:42.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:42.544+0000 7f6ce7fff640 1 --2- 172.21.15.27:0/3060997015 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ce8072d70 0x7f6ce80ae8d0 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7f6cd402a2b0 tx=0x7f6cd40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:42.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:42.544+0000 7f6ce57fa640 1 -- 172.21.15.27:0/3060997015 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cd4041020 con 0x7f6ce8072d70 2024-09-12T03:23:42.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:42.544+0000 7f6ce57fa640 1 -- 172.21.15.27:0/3060997015 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6cd40306c0 con 0x7f6ce8072d70 2024-09-12T03:23:42.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:42.544+0000 7f6ce57fa640 1 -- 172.21.15.27:0/3060997015 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cd4037040 con 0x7f6ce8072d70 2024-09-12T03:23:42.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:42.545+0000 7f6ceef03640 1 -- 172.21.15.27:0/3060997015 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ce8156450 con 0x7f6ce8072d70 2024-09-12T03:23:42.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:42.545+0000 7f6ceef03640 1 -- 172.21.15.27:0/3060997015 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ce8156a10 con 0x7f6ce8072d70 2024-09-12T03:23:42.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:42.546+0000 7f6ceef03640 1 -- 172.21.15.27:0/3060997015 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ce8072430 con 0x7f6ce8072d70 2024-09-12T03:23:42.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:42.546+0000 7f6ce57fa640 1 -- 172.21.15.27:0/3060997015 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 25) ==== 91365+0+0 (secure 0 0 0) 0x7f6cd4044020 con 0x7f6ce8072d70 2024-09-12T03:23:42.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:42.547+0000 7f6ce57fa640 1 --2- 172.21.15.27:0/3060997015 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f6ccc06c7c0 0x7f6ccc06ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:42.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:42.547+0000 7f6ce57fa640 1 -- 172.21.15.27:0/3060997015 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 5765+0+0 (secure 0 0 0) 0x7f6cd40a9e10 con 0x7f6ce8072d70 2024-09-12T03:23:42.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:42.547+0000 7f6cecc9f640 1 --2- 172.21.15.27:0/3060997015 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f6ccc06c7c0 0x7f6ccc06ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:42.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:42.547+0000 7f6cecc9f640 1 --2- 172.21.15.27:0/3060997015 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f6ccc06c7c0 0x7f6ccc06ec80 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f6ce8154c60 tx=0x7f6cdc008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:42.551 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:42.551+0000 7f6ce57fa640 1 -- 172.21.15.27:0/3060997015 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f6cd404b030 con 0x7f6ce8072d70 2024-09-12T03:23:42.740 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:42.740+0000 7f6ceef03640 1 -- 172.21.15.27:0/3060997015 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f6ce8075440 con 0x7f6ce8072d70 2024-09-12T03:23:42.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:42.741+0000 7f6ce57fa640 1 -- 172.21.15.27:0/3060997015 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v149) ==== 80+0+0 (secure 0 0 0) 0x7f6cd407a3c0 con 0x7f6ce8072d70 2024-09-12T03:23:42.741 INFO:teuthology.orchestra.run.smithi027.stderr:listed 0 entries 2024-09-12T03:23:42.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:42.743+0000 7f6ceef03640 1 -- 172.21.15.27:0/3060997015 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f6ccc06c7c0 msgr2=0x7f6ccc06ec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:42.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:42.743+0000 7f6ceef03640 1 --2- 172.21.15.27:0/3060997015 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f6ccc06c7c0 0x7f6ccc06ec80 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f6ce8154c60 tx=0x7f6cdc008040 comp rx=0 tx=0).stop 2024-09-12T03:23:42.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:42.743+0000 7f6ceef03640 1 -- 172.21.15.27:0/3060997015 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ce8072d70 msgr2=0x7f6ce80ae8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:42.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:42.743+0000 7f6ceef03640 1 --2- 172.21.15.27:0/3060997015 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ce8072d70 0x7f6ce80ae8d0 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7f6cd402a2b0 tx=0x7f6cd40046c0 comp rx=0 tx=0).stop 2024-09-12T03:23:42.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:42.743+0000 7f6ceef03640 1 -- 172.21.15.27:0/3060997015 shutdown_connections 2024-09-12T03:23:42.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:42.743+0000 7f6ceef03640 1 --2- 172.21.15.27:0/3060997015 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6ce815b2b0 0x7f6ce80aee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:42.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:42.743+0000 7f6ceef03640 1 --2- 172.21.15.27:0/3060997015 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f6ccc06c7c0 0x7f6ccc06ec80 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:42.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:42.743+0000 7f6ceef03640 1 --2- 172.21.15.27:0/3060997015 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ce8072d70 0x7f6ce80ae8d0 unknown :-1 s=CLOSED pgs=1602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:42.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:42.743+0000 7f6ceef03640 1 --2- 172.21.15.27:0/3060997015 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f6ce806e9b0 0x7f6ce814dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:42.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:42.743+0000 7f6ceef03640 1 -- 172.21.15.27:0/3060997015 >> 172.21.15.27:0/3060997015 conn(0x7f6ce8062c60 msgr2=0x7f6ce80ceab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:42.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:42.743+0000 7f6ceef03640 1 -- 172.21.15.27:0/3060997015 shutdown_connections 2024-09-12T03:23:42.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:42.743+0000 7f6ceef03640 1 -- 172.21.15.27:0/3060997015 wait complete. 2024-09-12T03:23:42.754 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-09-12T03:23:42.754 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:23:42.754 DEBUG:teuthology.orchestra.run.smithi045:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-09-12T03:23:42.790 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi045.front.sepia.ceph.com 2024-09-12T03:23:42.790 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:23:42.790 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-12T03:23:42.848 INFO:tasks.cephfs.mount:Terminating background process 2024-09-12T03:23:42.849 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-12T03:23:42.849 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T03:23:42.866 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056004,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497432k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16324 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248716k,nr_inodes=812179,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497432k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-12T03:23:42.866 INFO:tasks.cephfs.mount:Terminating background process 2024-09-12T03:23:42.866 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-12T03:23:42.866 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T03:23:42.907 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056795,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16602 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249348k,nr_inodes=812337,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-12T03:23:42.907 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-12T03:23:43.009 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.009+0000 7f006c22c640 1 Processor -- start 2024-09-12T03:23:43.010 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.010+0000 7f006c22c640 1 -- start start 2024-09-12T03:23:43.011 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.010+0000 7f006c22c640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f00640b1ca0 0x7f006415f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:43.011 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.010+0000 7f006c22c640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f006415fd70 con 0x7f006406f360 2024-09-12T03:23:43.011 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.010+0000 7f006c22c640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f006406cc40 con 0x7f00640cf660 2024-09-12T03:23:43.011 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.010+0000 7f006c22c640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f00640d22f0 con 0x7f00640b1ca0 2024-09-12T03:23:43.011 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.011+0000 7f0069fc8640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f00640b1ca0 0x7f006415f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:43.011 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.011+0000 7f00697c7640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f006406f360 0x7f00640cef50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:52304/0 (socket says 172.21.15.27:52304) 2024-09-12T03:23:43.011 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.011+0000 7f00697c7640 1 -- 172.21.15.27:0/1798192344 learned_addr learned my addr 172.21.15.27:0/1798192344 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:23:43.011 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.011+0000 7f0069fc8640 1 -- 172.21.15.27:0/1798192344 >> v1:172.21.15.45:6789/0 conn(0x7f00640cf660 legacy=0x7f00640d2780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:43.011 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.011+0000 7f0069fc8640 1 -- 172.21.15.27:0/1798192344 >> v1:172.21.15.27:6789/0 conn(0x7f006406f360 legacy=0x7f00640cef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:43.012 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.011+0000 7f0069fc8640 1 -- 172.21.15.27:0/1798192344 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00640d1ce0 con 0x7f00640b1ca0 2024-09-12T03:23:43.013 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.012+0000 7f0069fc8640 1 --2- 172.21.15.27:0/1798192344 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f00640b1ca0 0x7f006415f830 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7f0054009d90 tx=0x7f005402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b19d7b29ed8edb85 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:43.013 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.012+0000 7f0068fc6640 1 -- 172.21.15.27:0/1798192344 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0054007740 con 0x7f00640b1ca0 2024-09-12T03:23:43.013 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.012+0000 7f0068fc6640 1 -- 172.21.15.27:0/1798192344 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00540078e0 con 0x7f00640b1ca0 2024-09-12T03:23:43.013 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.012+0000 7f0068fc6640 1 -- 172.21.15.27:0/1798192344 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0054007cb0 con 0x7f00640b1ca0 2024-09-12T03:23:43.013 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.012+0000 7f006c22c640 1 -- 172.21.15.27:0/1798192344 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f00640b1ca0 msgr2=0x7f006415f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:43.013 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.012+0000 7f006c22c640 1 --2- 172.21.15.27:0/1798192344 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f00640b1ca0 0x7f006415f830 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7f0054009d90 tx=0x7f005402a6a0 comp rx=0 tx=0).stop 2024-09-12T03:23:43.013 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.012+0000 7f006c22c640 1 -- 172.21.15.27:0/1798192344 shutdown_connections 2024-09-12T03:23:43.013 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.012+0000 7f006c22c640 1 --2- 172.21.15.27:0/1798192344 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f00640b1ca0 0x7f006415f830 unknown :-1 s=CLOSED pgs=1604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:43.013 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.012+0000 7f006c22c640 1 -- 172.21.15.27:0/1798192344 >> 172.21.15.27:0/1798192344 conn(0x7f0064073800 msgr2=0x7f006409ba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:43.013 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.012+0000 7f006c22c640 1 -- 172.21.15.27:0/1798192344 shutdown_connections 2024-09-12T03:23:43.014 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.012+0000 7f006c22c640 1 -- 172.21.15.27:0/1798192344 wait complete. 2024-09-12T03:23:43.016 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.014+0000 7f006c22c640 1 Processor -- start 2024-09-12T03:23:43.016 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.014+0000 7f006c22c640 1 -- start start 2024-09-12T03:23:43.016 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.014+0000 7f006c22c640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f006406f360 0x7f00640b6f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:43.016 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.014+0000 7f006c22c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f00640b1ca0 0x7f00641f47e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:43.016 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.014+0000 7f006c22c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f00640cf660 0x7f00641f4d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:43.016 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.014+0000 7f006c22c640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f006406d1c0 con 0x7f00640cf660 2024-09-12T03:23:43.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.014+0000 7f006c22c640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f00640d1f80 con 0x7f00640b1ca0 2024-09-12T03:23:43.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.014+0000 7f006c22c640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f006406d340 con 0x7f006406f360 2024-09-12T03:23:43.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.014+0000 7f006a7c9640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f00640cf660 0x7f00641f4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:43.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.014+0000 7f006a7c9640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f00640cf660 0x7f00641f4d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59668/0 (socket says 172.21.15.27:59668) 2024-09-12T03:23:43.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.015+0000 7f006a7c9640 1 -- 172.21.15.27:0/3410533577 learned_addr learned my addr 172.21.15.27:0/3410533577 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:23:43.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.015+0000 7f00697c7640 1 --2- 172.21.15.27:0/3410533577 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f00640b1ca0 0x7f00641f47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:43.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.015+0000 7f0069fc8640 1 --2- 172.21.15.27:0/3410533577 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f006406f360 0x7f00640b6f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:43.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.015+0000 7f006a7c9640 1 -- 172.21.15.27:0/3410533577 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f006406f360 msgr2=0x7f00640b6f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:43.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.015+0000 7f006a7c9640 1 --2- 172.21.15.27:0/3410533577 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f006406f360 0x7f00640b6f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:43.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.015+0000 7f006a7c9640 1 -- 172.21.15.27:0/3410533577 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f00640b1ca0 msgr2=0x7f00641f47e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:43.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.015+0000 7f006a7c9640 1 --2- 172.21.15.27:0/3410533577 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f00640b1ca0 0x7f00641f47e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:43.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.015+0000 7f006a7c9640 1 -- 172.21.15.27:0/3410533577 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00640b7780 con 0x7f00640cf660 2024-09-12T03:23:43.018 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.015+0000 7f006a7c9640 1 --2- 172.21.15.27:0/3410533577 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f00640cf660 0x7f00641f4d20 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7f005c00b410 tx=0x7f005c00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:43.018 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.015+0000 7f00627fc640 1 -- 172.21.15.27:0/3410533577 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f005c015020 con 0x7f00640cf660 2024-09-12T03:23:43.018 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.015+0000 7f00627fc640 1 -- 172.21.15.27:0/3410533577 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f005c011850 con 0x7f00640cf660 2024-09-12T03:23:43.018 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.015+0000 7f006c22c640 1 -- 172.21.15.27:0/3410533577 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00641f5b20 con 0x7f00640cf660 2024-09-12T03:23:43.018 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.015+0000 7f00627fc640 1 -- 172.21.15.27:0/3410533577 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f005c011c40 con 0x7f00640cf660 2024-09-12T03:23:43.018 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.015+0000 7f006c22c640 1 -- 172.21.15.27:0/3410533577 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00641f60e0 con 0x7f00640cf660 2024-09-12T03:23:43.018 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.016+0000 7f006c22c640 1 -- 172.21.15.27:0/3410533577 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00640cfc40 con 0x7f00640cf660 2024-09-12T03:23:43.019 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.017+0000 7f00627fc640 1 -- 172.21.15.27:0/3410533577 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 25) ==== 91365+0+0 (secure 0 0 0) 0x7f005c018020 con 0x7f00640cf660 2024-09-12T03:23:43.019 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.017+0000 7f00627fc640 1 --2- 172.21.15.27:0/3410533577 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f005006c7c0 0x7f005006ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:43.019 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.017+0000 7f0069fc8640 1 --2- 172.21.15.27:0/3410533577 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f005006c7c0 0x7f005006ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:43.019 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.017+0000 7f00627fc640 1 -- 172.21.15.27:0/3410533577 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 5765+0+0 (secure 0 0 0) 0x7f005c089440 con 0x7f00640cf660 2024-09-12T03:23:43.019 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.018+0000 7f0069fc8640 1 --2- 172.21.15.27:0/3410533577 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f005006c7c0 0x7f005006ec80 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f0054034000 tx=0x7f005402f000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:43.021 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.022+0000 7f00627fc640 1 -- 172.21.15.27:0/3410533577 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f005c01f020 con 0x7f00640cf660 2024-09-12T03:23:43.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.232+0000 7f006c22c640 1 -- 172.21.15.27:0/3410533577 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0064070f60 con 0x7f00640cf660 2024-09-12T03:23:43.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.233+0000 7f00627fc640 1 -- 172.21.15.27:0/3410533577 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 92 v92) ==== 94+0+5227 (secure 0 0 0) 0x7f005c0172c0 con 0x7f00640cf660 2024-09-12T03:23:43.233 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:23:43.233 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":92,"btime":"2024-09-12T03:23:34:849854+0000","default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6917,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/3682270902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":3682270902},{"type":"v1","addr":"172.21.15.45:6834","nonce":3682270902}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":87},{"gid":7771,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/2399103579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":2399103579},{"type":"v1","addr":"172.21.15.45:6835","nonce":2399103579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":87},{"gid":8559,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6834/1045432254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":1045432254},{"type":"v1","addr":"172.21.15.27:6834","nonce":1045432254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":87}],"filesystems":[{"mdsmap":{"epoch":91,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-12T03:23:21.031667+0000","modified":"2024-09-12T03:23:23.983036+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8562},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8562":{"gid":8562,"name":"c","rank":0,"incarnation":88,"state":"up:active","state_seq":3,"addr":"172.21.15.27:6835/1241148443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":1241148443},{"type":"v1","addr":"172.21.15.27:6835","nonce":1241148443}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[25],"metadata_pool":24,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8562,"qdb_cluster":[8562]},"id":12}]} 2024-09-12T03:23:43.234 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 92 2024-09-12T03:23:43.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.235+0000 7f006c22c640 1 -- 172.21.15.27:0/3410533577 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f005006c7c0 msgr2=0x7f005006ec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:43.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.235+0000 7f006c22c640 1 --2- 172.21.15.27:0/3410533577 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f005006c7c0 0x7f005006ec80 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f0054034000 tx=0x7f005402f000 comp rx=0 tx=0).stop 2024-09-12T03:23:43.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.235+0000 7f006c22c640 1 -- 172.21.15.27:0/3410533577 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f00640cf660 msgr2=0x7f00641f4d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:43.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.235+0000 7f006c22c640 1 --2- 172.21.15.27:0/3410533577 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f00640cf660 0x7f00641f4d20 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7f005c00b410 tx=0x7f005c00b8d0 comp rx=0 tx=0).stop 2024-09-12T03:23:43.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.235+0000 7f006c22c640 1 -- 172.21.15.27:0/3410533577 shutdown_connections 2024-09-12T03:23:43.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.235+0000 7f006c22c640 1 --2- 172.21.15.27:0/3410533577 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f005006c7c0 0x7f005006ec80 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:43.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.235+0000 7f006c22c640 1 --2- 172.21.15.27:0/3410533577 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f00640cf660 0x7f00641f4d20 unknown :-1 s=CLOSED pgs=1605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:43.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.235+0000 7f006c22c640 1 --2- 172.21.15.27:0/3410533577 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f00640b1ca0 0x7f00641f47e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:43.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.235+0000 7f006c22c640 1 --2- 172.21.15.27:0/3410533577 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f006406f360 0x7f00640b6f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:43.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.235+0000 7f006c22c640 1 -- 172.21.15.27:0/3410533577 >> 172.21.15.27:0/3410533577 conn(0x7f0064073800 msgr2=0x7f00640d2780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:43.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.235+0000 7f006c22c640 1 -- 172.21.15.27:0/3410533577 shutdown_connections 2024-09-12T03:23:43.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.235+0000 7f006c22c640 1 -- 172.21.15.27:0/3410533577 wait complete. 2024-09-12T03:23:43.249 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-12T03:23:43.352 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.352+0000 7f996ba45640 1 Processor -- start 2024-09-12T03:23:43.352 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.352+0000 7f996ba45640 1 -- start start 2024-09-12T03:23:43.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.353+0000 7f996ba45640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f996406f2d0 0x7f996406f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:43.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.353+0000 7f996ba45640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f99640725b0 0x7f99640cf250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:43.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.353+0000 7f996ba45640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f996406fbf0 con 0x7f996415c740 2024-09-12T03:23:43.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.353+0000 7f996ba45640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f9964072b90 con 0x7f99640725b0 2024-09-12T03:23:43.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.353+0000 7f996ba45640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f9964072e70 con 0x7f996406f2d0 2024-09-12T03:23:43.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.353+0000 7f9968fe0640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f996415c740 0x7f9964071ea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:52310/0 (socket says 172.21.15.27:52310) 2024-09-12T03:23:43.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.353+0000 7f9968fe0640 1 -- 172.21.15.27:0/1138728245 learned_addr learned my addr 172.21.15.27:0/1138728245 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:23:43.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.353+0000 7f995bfff640 1 -- 172.21.15.27:0/1138728245 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1542097733 0 0) 0x7f996406fbf0 con 0x7f996415c740 2024-09-12T03:23:43.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.353+0000 7f995bfff640 1 -- 172.21.15.27:0/1138728245 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9950003520 con 0x7f996415c740 2024-09-12T03:23:43.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.353+0000 7f99697e1640 1 --2- 172.21.15.27:0/1138728245 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f996406f2d0 0x7f996406f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:43.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.353+0000 7f9969fe2640 1 --2- 172.21.15.27:0/1138728245 >> v2:172.21.15.45:3300/0 conn(0x7f99640725b0 0x7f99640cf250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:43.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.353+0000 7f995bfff640 1 -- 172.21.15.27:0/1138728245 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2748635138 0 0) 0x7f9950003520 con 0x7f996415c740 2024-09-12T03:23:43.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.353+0000 7f995bfff640 1 -- 172.21.15.27:0/1138728245 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f996406f2d0 msgr2=0x7f996406f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:43.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.353+0000 7f995bfff640 1 --2- 172.21.15.27:0/1138728245 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f996406f2d0 0x7f996406f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:43.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.353+0000 7f995bfff640 1 -- 172.21.15.27:0/1138728245 >> v2:172.21.15.45:3300/0 conn(0x7f99640725b0 msgr2=0x7f99640cf250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:43.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.354+0000 7f995bfff640 1 --2- 172.21.15.27:0/1138728245 >> v2:172.21.15.45:3300/0 conn(0x7f99640725b0 0x7f99640cf250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:43.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.354+0000 7f995bfff640 1 -- 172.21.15.27:0/1138728245 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99640cea50 con 0x7f996415c740 2024-09-12T03:23:43.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.354+0000 7f995bfff640 1 -- 172.21.15.27:0/1138728245 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 847755213 0 0) 0x7f9954001670 con 0x7f996415c740 2024-09-12T03:23:43.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.354+0000 7f995bfff640 1 -- 172.21.15.27:0/1138728245 >> v1:172.21.15.27:6789/0 conn(0x7f996415c740 legacy=0x7f9964071ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:43.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.354+0000 7f995bfff640 1 --2- 172.21.15.27:0/1138728245 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f99500039b0 0x7f9950003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:43.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.354+0000 7f995bfff640 1 --2- 172.21.15.27:0/1138728245 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9950004800 0x7f9950004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:43.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.354+0000 7f995bfff640 1 --2- 172.21.15.27:0/1138728245 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f996406f2d0 0x7f99500051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:43.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.354+0000 7f995bfff640 1 -- 172.21.15.27:0/1138728245 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f9964072b90 con 0x7f9950004800 2024-09-12T03:23:43.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.354+0000 7f995bfff640 1 -- 172.21.15.27:0/1138728245 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f9964072e70 con 0x7f99500039b0 2024-09-12T03:23:43.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.354+0000 7f995bfff640 1 -- 172.21.15.27:0/1138728245 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f995000ba40 con 0x7f996406f2d0 2024-09-12T03:23:43.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.354+0000 7f9968fe0640 1 --2- 172.21.15.27:0/1138728245 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9950004800 0x7f9950004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:43.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.354+0000 7f99697e1640 1 --2- 172.21.15.27:0/1138728245 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f99500039b0 0x7f9950003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:43.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.354+0000 7f9969fe2640 1 --2- 172.21.15.27:0/1138728245 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f996406f2d0 0x7f99500051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:43.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.354+0000 7f9968fe0640 1 -- 172.21.15.27:0/1138728245 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f996406f2d0 msgr2=0x7f99500051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:43.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.354+0000 7f9968fe0640 1 --2- 172.21.15.27:0/1138728245 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f996406f2d0 0x7f99500051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:43.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.354+0000 7f9968fe0640 1 -- 172.21.15.27:0/1138728245 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f99500039b0 msgr2=0x7f9950003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:43.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.354+0000 7f9968fe0640 1 --2- 172.21.15.27:0/1138728245 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f99500039b0 0x7f9950003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:43.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.354+0000 7f9968fe0640 1 -- 172.21.15.27:0/1138728245 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9950003520 con 0x7f9950004800 2024-09-12T03:23:43.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.355+0000 7f9968fe0640 1 --2- 172.21.15.27:0/1138728245 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9950004800 0x7f9950004c80 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7f996406cdb0 tx=0x7f9950005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d42cb223b4175489 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:43.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.355+0000 7f995bfff640 1 -- 172.21.15.27:0/1138728245 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9954007d50 con 0x7f9950004800 2024-09-12T03:23:43.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.355+0000 7f995bfff640 1 -- 172.21.15.27:0/1138728245 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99540058d0 con 0x7f9950004800 2024-09-12T03:23:43.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.355+0000 7f995bfff640 1 -- 172.21.15.27:0/1138728245 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9954005cc0 con 0x7f9950004800 2024-09-12T03:23:43.360 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.355+0000 7f996ba45640 1 -- 172.21.15.27:0/1138728245 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9950004800 msgr2=0x7f9950004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:43.360 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.355+0000 7f996ba45640 1 --2- 172.21.15.27:0/1138728245 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9950004800 0x7f9950004c80 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7f996406cdb0 tx=0x7f9950005830 comp rx=0 tx=0).stop 2024-09-12T03:23:43.360 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.355+0000 7f9969fe2640 1 -- 172.21.15.27:0/1138728245 reap_dead start 2024-09-12T03:23:43.360 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.356+0000 7f996ba45640 1 -- 172.21.15.27:0/1138728245 shutdown_connections 2024-09-12T03:23:43.360 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.356+0000 7f996ba45640 1 -- 172.21.15.27:0/1138728245 >> 172.21.15.27:0/1138728245 conn(0x7f99640b2500 msgr2=0x7f99640b28e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:43.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.356+0000 7f996ba45640 1 -- 172.21.15.27:0/1138728245 shutdown_connections 2024-09-12T03:23:43.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.356+0000 7f996ba45640 1 -- 172.21.15.27:0/1138728245 wait complete. 2024-09-12T03:23:43.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.357+0000 7f996ba45640 1 Processor -- start 2024-09-12T03:23:43.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.357+0000 7f996ba45640 1 -- start start 2024-09-12T03:23:43.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.358+0000 7f996ba45640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f9950004800 0x7f9964153040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:43.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.358+0000 7f996ba45640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f996414ce50 0x7f996415f500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:43.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.358+0000 7f996ba45640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f996414e7d0 0x7f996414ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:43.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.358+0000 7f996ba45640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f99640ce7f0 con 0x7f996414ce50 2024-09-12T03:23:43.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.358+0000 7f996ba45640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f99640ce670 con 0x7f996414e7d0 2024-09-12T03:23:43.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.358+0000 7f996ba45640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f996406f6b0 con 0x7f9950004800 2024-09-12T03:23:43.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.358+0000 7f9968fe0640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f996414ce50 0x7f996415f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:43.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.358+0000 7f9968fe0640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f996414ce50 0x7f996415f500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59698/0 (socket says 172.21.15.27:59698) 2024-09-12T03:23:43.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.358+0000 7f9968fe0640 1 -- 172.21.15.27:0/3603011833 learned_addr learned my addr 172.21.15.27:0/3603011833 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:23:43.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.358+0000 7f9968fe0640 1 -- 172.21.15.27:0/3603011833 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f9950004800 msgr2=0x7f9964153040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:43.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.358+0000 7f9968fe0640 1 --2- 172.21.15.27:0/3603011833 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f9950004800 0x7f9964153040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:43.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.358+0000 7f9968fe0640 1 -- 172.21.15.27:0/3603011833 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f996414e7d0 msgr2=0x7f996414ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:43.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.358+0000 7f9968fe0640 1 --2- 172.21.15.27:0/3603011833 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f996414e7d0 0x7f996414ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:43.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.358+0000 7f9968fe0640 1 -- 172.21.15.27:0/3603011833 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f996415fa40 con 0x7f996414ce50 2024-09-12T03:23:43.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.358+0000 7f9968fe0640 1 --2- 172.21.15.27:0/3603011833 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f996414ce50 0x7f996415f500 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7f995402adf0 tx=0x7f99640cec20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:43.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.358+0000 7f9959ffb640 1 -- 172.21.15.27:0/3603011833 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9954044070 con 0x7f996414ce50 2024-09-12T03:23:43.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.358+0000 7f9959ffb640 1 -- 172.21.15.27:0/3603011833 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99540047e0 con 0x7f996414ce50 2024-09-12T03:23:43.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.358+0000 7f996ba45640 1 -- 172.21.15.27:0/3603011833 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f996414d2d0 con 0x7f996414ce50 2024-09-12T03:23:43.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.358+0000 7f9959ffb640 1 -- 172.21.15.27:0/3603011833 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9954038040 con 0x7f996414ce50 2024-09-12T03:23:43.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.359+0000 7f996ba45640 1 -- 172.21.15.27:0/3603011833 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99641539b0 con 0x7f996414ce50 2024-09-12T03:23:43.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.359+0000 7f996ba45640 1 -- 172.21.15.27:0/3603011833 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f996406fa40 con 0x7f996414ce50 2024-09-12T03:23:43.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.360+0000 7f9959ffb640 1 -- 172.21.15.27:0/3603011833 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 25) ==== 91365+0+0 (secure 0 0 0) 0x7f9954047070 con 0x7f996414ce50 2024-09-12T03:23:43.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.360+0000 7f9959ffb640 1 --2- 172.21.15.27:0/3603011833 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f994806c7c0 0x7f994806ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:43.363 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.360+0000 7f9959ffb640 1 -- 172.21.15.27:0/3603011833 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 5765+0+0 (secure 0 0 0) 0x7f99540a8d60 con 0x7f996414ce50 2024-09-12T03:23:43.363 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.361+0000 7f99697e1640 1 --2- 172.21.15.27:0/3603011833 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f994806c7c0 0x7f994806ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:43.363 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.361+0000 7f99697e1640 1 --2- 172.21.15.27:0/3603011833 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f994806c7c0 0x7f994806ec80 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f996415e980 tx=0x7f9960009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:43.364 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.365+0000 7f9959ffb640 1 -- 172.21.15.27:0/3603011833 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f9954079310 con 0x7f996414ce50 2024-09-12T03:23:43.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.575+0000 7f996ba45640 1 -- 172.21.15.27:0/3603011833 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9964158d80 con 0x7f996414ce50 2024-09-12T03:23:43.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.576+0000 7f9959ffb640 1 -- 172.21.15.27:0/3603011833 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 92 v92) ==== 94+0+5227 (secure 0 0 0) 0x7f9954078c10 con 0x7f996414ce50 2024-09-12T03:23:43.576 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:23:43.576 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":92,"btime":"2024-09-12T03:23:34:849854+0000","default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6917,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/3682270902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":3682270902},{"type":"v1","addr":"172.21.15.45:6834","nonce":3682270902}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":87},{"gid":7771,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/2399103579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":2399103579},{"type":"v1","addr":"172.21.15.45:6835","nonce":2399103579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":87},{"gid":8559,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6834/1045432254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":1045432254},{"type":"v1","addr":"172.21.15.27:6834","nonce":1045432254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":87}],"filesystems":[{"mdsmap":{"epoch":91,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-12T03:23:21.031667+0000","modified":"2024-09-12T03:23:23.983036+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8562},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8562":{"gid":8562,"name":"c","rank":0,"incarnation":88,"state":"up:active","state_seq":3,"addr":"172.21.15.27:6835/1241148443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":1241148443},{"type":"v1","addr":"172.21.15.27:6835","nonce":1241148443}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[25],"metadata_pool":24,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8562,"qdb_cluster":[8562]},"id":12}]} 2024-09-12T03:23:43.576 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 92 2024-09-12T03:23:43.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.578+0000 7f996ba45640 1 -- 172.21.15.27:0/3603011833 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f994806c7c0 msgr2=0x7f994806ec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:43.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.578+0000 7f996ba45640 1 --2- 172.21.15.27:0/3603011833 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f994806c7c0 0x7f994806ec80 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f996415e980 tx=0x7f9960009190 comp rx=0 tx=0).stop 2024-09-12T03:23:43.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.578+0000 7f996ba45640 1 -- 172.21.15.27:0/3603011833 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f996414ce50 msgr2=0x7f996415f500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:43.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.578+0000 7f996ba45640 1 --2- 172.21.15.27:0/3603011833 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f996414ce50 0x7f996415f500 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7f995402adf0 tx=0x7f99640cec20 comp rx=0 tx=0).stop 2024-09-12T03:23:43.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.578+0000 7f996ba45640 1 -- 172.21.15.27:0/3603011833 shutdown_connections 2024-09-12T03:23:43.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.578+0000 7f996ba45640 1 --2- 172.21.15.27:0/3603011833 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f996414e7d0 0x7f996414ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:43.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.578+0000 7f996ba45640 1 --2- 172.21.15.27:0/3603011833 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f994806c7c0 0x7f994806ec80 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:43.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.578+0000 7f996ba45640 1 --2- 172.21.15.27:0/3603011833 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f996414ce50 0x7f996415f500 unknown :-1 s=CLOSED pgs=1608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:43.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.578+0000 7f996ba45640 1 --2- 172.21.15.27:0/3603011833 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f9950004800 0x7f9964153040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:43.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.578+0000 7f996ba45640 1 -- 172.21.15.27:0/3603011833 >> 172.21.15.27:0/3603011833 conn(0x7f99640b2500 msgr2=0x7f99641592a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:43.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.578+0000 7f996ba45640 1 -- 172.21.15.27:0/3603011833 shutdown_connections 2024-09-12T03:23:43.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.578+0000 7f996ba45640 1 -- 172.21.15.27:0/3603011833 wait complete. 2024-09-12T03:23:43.590 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-12T03:23:43.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.693+0000 7f5b60e8d640 1 Processor -- start 2024-09-12T03:23:43.693 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.693+0000 7f5b60e8d640 1 -- start start 2024-09-12T03:23:43.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.694+0000 7f5b60e8d640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f5b5c15eb90 0x7f5b5c0d2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:43.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.694+0000 7f5b60e8d640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f5b5c0d1720 0x7f5b5c0b8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:43.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.694+0000 7f5b60e8d640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b5c0d3760 con 0x7f5b5c0d5200 2024-09-12T03:23:43.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.694+0000 7f5b60e8d640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f5b5c0b7d20 con 0x7f5b5c15eb90 2024-09-12T03:23:43.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.694+0000 7f5b60e8d640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f5b5c0d42e0 con 0x7f5b5c0d1720 2024-09-12T03:23:43.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.694+0000 7f5b59d9b640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f5b5c0d5200 0x7f5b5c0d1010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:52318/0 (socket says 172.21.15.27:52318) 2024-09-12T03:23:43.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.694+0000 7f5b59d9b640 1 -- 172.21.15.27:0/1515636657 learned_addr learned my addr 172.21.15.27:0/1515636657 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:23:43.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.694+0000 7f5b5959a640 1 -- 172.21.15.27:0/1515636657 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 614881537 0 0) 0x7f5b5c0d3760 con 0x7f5b5c0d5200 2024-09-12T03:23:43.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.694+0000 7f5b5959a640 1 -- 172.21.15.27:0/1515636657 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5b48003520 con 0x7f5b5c0d5200 2024-09-12T03:23:43.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.694+0000 7f5b5a59c640 1 --2- 172.21.15.27:0/1515636657 >> v2:172.21.15.45:3300/0 conn(0x7f5b5c15eb90 0x7f5b5c0d2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:43.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.694+0000 7f5b5ad9d640 1 --2- 172.21.15.27:0/1515636657 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f5b5c0d1720 0x7f5b5c0b8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:43.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.694+0000 7f5b5959a640 1 -- 172.21.15.27:0/1515636657 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 605399957 0 0) 0x7f5b48003520 con 0x7f5b5c0d5200 2024-09-12T03:23:43.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.694+0000 7f5b5959a640 1 -- 172.21.15.27:0/1515636657 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f5b5c0d1720 msgr2=0x7f5b5c0b8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:43.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.694+0000 7f5b5959a640 1 --2- 172.21.15.27:0/1515636657 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f5b5c0d1720 0x7f5b5c0b8190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:43.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.694+0000 7f5b5959a640 1 -- 172.21.15.27:0/1515636657 >> v2:172.21.15.45:3300/0 conn(0x7f5b5c15eb90 msgr2=0x7f5b5c0d2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:43.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.694+0000 7f5b5959a640 1 --2- 172.21.15.27:0/1515636657 >> v2:172.21.15.45:3300/0 conn(0x7f5b5c15eb90 0x7f5b5c0d2df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:43.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.694+0000 7f5b5959a640 1 -- 172.21.15.27:0/1515636657 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b5c0d3a40 con 0x7f5b5c0d5200 2024-09-12T03:23:43.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.694+0000 7f5b5959a640 1 -- 172.21.15.27:0/1515636657 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 847755213 0 0) 0x7f5b44001670 con 0x7f5b5c0d5200 2024-09-12T03:23:43.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.694+0000 7f5b5959a640 1 -- 172.21.15.27:0/1515636657 >> v1:172.21.15.27:6789/0 conn(0x7f5b5c0d5200 legacy=0x7f5b5c0d1010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:43.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.694+0000 7f5b5959a640 1 --2- 172.21.15.27:0/1515636657 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f5b5c0d1720 0x7f5b480039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:43.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.694+0000 7f5b5959a640 1 --2- 172.21.15.27:0/1515636657 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5b48003fe0 0x7f5b48004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:43.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.694+0000 7f5b5959a640 1 --2- 172.21.15.27:0/1515636657 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5b48004d40 0x7f5b480051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:43.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.694+0000 7f5b5959a640 1 -- 172.21.15.27:0/1515636657 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f5b5c0d42e0 con 0x7f5b48003fe0 2024-09-12T03:23:43.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.694+0000 7f5b5959a640 1 -- 172.21.15.27:0/1515636657 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f5b4800ba70 con 0x7f5b48004d40 2024-09-12T03:23:43.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.694+0000 7f5b5959a640 1 -- 172.21.15.27:0/1515636657 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f5b4800bc40 con 0x7f5b5c0d1720 2024-09-12T03:23:43.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.694+0000 7f5b59d9b640 1 --2- 172.21.15.27:0/1515636657 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5b48003fe0 0x7f5b48004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:43.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.695+0000 7f5b5a59c640 1 --2- 172.21.15.27:0/1515636657 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f5b5c0d1720 0x7f5b480039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:43.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.695+0000 7f5b5ad9d640 1 --2- 172.21.15.27:0/1515636657 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5b48004d40 0x7f5b480051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:43.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.695+0000 7f5b59d9b640 1 -- 172.21.15.27:0/1515636657 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f5b5c0d1720 msgr2=0x7f5b480039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:43.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.695+0000 7f5b59d9b640 1 --2- 172.21.15.27:0/1515636657 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f5b5c0d1720 0x7f5b480039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:43.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.695+0000 7f5b59d9b640 1 -- 172.21.15.27:0/1515636657 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5b48004d40 msgr2=0x7f5b480051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:43.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.695+0000 7f5b59d9b640 1 --2- 172.21.15.27:0/1515636657 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5b48004d40 0x7f5b480051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:43.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.695+0000 7f5b59d9b640 1 -- 172.21.15.27:0/1515636657 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b48003520 con 0x7f5b48003fe0 2024-09-12T03:23:43.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.695+0000 7f5b5ad9d640 1 --2- 172.21.15.27:0/1515636657 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5b48004d40 0x7f5b480051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:23:43.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.695+0000 7f5b59d9b640 1 --2- 172.21.15.27:0/1515636657 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5b48003fe0 0x7f5b48004800 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7f5b44007fd0 tx=0x7f5b48005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c02292c7bb1dd9bd server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:43.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.696+0000 7f5b5959a640 1 -- 172.21.15.27:0/1515636657 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b44007d50 con 0x7f5b48003fe0 2024-09-12T03:23:43.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.696+0000 7f5b5959a640 1 -- 172.21.15.27:0/1515636657 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b44005ad0 con 0x7f5b48003fe0 2024-09-12T03:23:43.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.696+0000 7f5b5959a640 1 -- 172.21.15.27:0/1515636657 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b44007070 con 0x7f5b48003fe0 2024-09-12T03:23:43.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.696+0000 7f5b60e8d640 1 -- 172.21.15.27:0/1515636657 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5b48003fe0 msgr2=0x7f5b48004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:43.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.696+0000 7f5b60e8d640 1 --2- 172.21.15.27:0/1515636657 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5b48003fe0 0x7f5b48004800 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7f5b44007fd0 tx=0x7f5b48005830 comp rx=0 tx=0).stop 2024-09-12T03:23:43.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.696+0000 7f5b5ad9d640 1 -- 172.21.15.27:0/1515636657 reap_dead start 2024-09-12T03:23:43.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.696+0000 7f5b60e8d640 1 -- 172.21.15.27:0/1515636657 shutdown_connections 2024-09-12T03:23:43.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.696+0000 7f5b60e8d640 1 -- 172.21.15.27:0/1515636657 >> 172.21.15.27:0/1515636657 conn(0x7f5b5c06dce0 msgr2=0x7f5b5c06e0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:43.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.696+0000 7f5b60e8d640 1 -- 172.21.15.27:0/1515636657 shutdown_connections 2024-09-12T03:23:43.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.696+0000 7f5b60e8d640 1 -- 172.21.15.27:0/1515636657 wait complete. 2024-09-12T03:23:43.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.698+0000 7f5b60e8d640 1 Processor -- start 2024-09-12T03:23:43.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.698+0000 7f5b60e8d640 1 -- start start 2024-09-12T03:23:43.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.699+0000 7f5b60e8d640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f5b48003fe0 0x7f5b5c159cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:43.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.699+0000 7f5b60e8d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5b5c158490 0x7f5b5c15a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:43.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.699+0000 7f5b60e8d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5b5c159360 0x7f5b5c1f3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:43.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.699+0000 7f5b60e8d640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f5b5c0d3330 con 0x7f5b5c158490 2024-09-12T03:23:43.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.699+0000 7f5b60e8d640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f5b5c0d4000 con 0x7f5b5c159360 2024-09-12T03:23:43.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.699+0000 7f5b60e8d640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f5b5c0d34b0 con 0x7f5b48003fe0 2024-09-12T03:23:43.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.699+0000 7f5b59d9b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5b5c158490 0x7f5b5c15a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:43.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.699+0000 7f5b59d9b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5b5c158490 0x7f5b5c15a1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59710/0 (socket says 172.21.15.27:59710) 2024-09-12T03:23:43.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.699+0000 7f5b59d9b640 1 -- 172.21.15.27:0/545140459 learned_addr learned my addr 172.21.15.27:0/545140459 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:23:43.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.699+0000 7f5b5a59c640 1 --2- 172.21.15.27:0/545140459 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f5b48003fe0 0x7f5b5c159cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:43.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.699+0000 7f5b5ad9d640 1 --2- 172.21.15.27:0/545140459 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5b5c159360 0x7f5b5c1f3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:43.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.699+0000 7f5b59d9b640 1 -- 172.21.15.27:0/545140459 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f5b48003fe0 msgr2=0x7f5b5c159cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:43.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.699+0000 7f5b59d9b640 1 --2- 172.21.15.27:0/545140459 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f5b48003fe0 0x7f5b5c159cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:43.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.699+0000 7f5b59d9b640 1 -- 172.21.15.27:0/545140459 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5b5c159360 msgr2=0x7f5b5c1f3bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:43.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.699+0000 7f5b59d9b640 1 --2- 172.21.15.27:0/545140459 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5b5c159360 0x7f5b5c1f3bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:43.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.699+0000 7f5b59d9b640 1 -- 172.21.15.27:0/545140459 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b5c0c83e0 con 0x7f5b5c158490 2024-09-12T03:23:43.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.699+0000 7f5b5a59c640 1 --2- 172.21.15.27:0/545140459 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f5b48003fe0 0x7f5b5c159cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:23:43.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.699+0000 7f5b59d9b640 1 --2- 172.21.15.27:0/545140459 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5b5c158490 0x7f5b5c15a1f0 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7f5b4402ae20 tx=0x7f5b5c0b7c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:43.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.699+0000 7f5b52ffd640 1 -- 172.21.15.27:0/545140459 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b44042020 con 0x7f5b5c158490 2024-09-12T03:23:43.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.699+0000 7f5b52ffd640 1 -- 172.21.15.27:0/545140459 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b44004bf0 con 0x7f5b5c158490 2024-09-12T03:23:43.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.699+0000 7f5b52ffd640 1 -- 172.21.15.27:0/545140459 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b44001c80 con 0x7f5b5c158490 2024-09-12T03:23:43.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.699+0000 7f5b60e8d640 1 -- 172.21.15.27:0/545140459 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b5c0c8680 con 0x7f5b5c158490 2024-09-12T03:23:43.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.700+0000 7f5b60e8d640 1 -- 172.21.15.27:0/545140459 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b5c159930 con 0x7f5b5c158490 2024-09-12T03:23:43.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.700+0000 7f5b60e8d640 1 -- 172.21.15.27:0/545140459 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b5c15f000 con 0x7f5b5c158490 2024-09-12T03:23:43.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.701+0000 7f5b52ffd640 1 -- 172.21.15.27:0/545140459 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 25) ==== 91365+0+0 (secure 0 0 0) 0x7f5b44045020 con 0x7f5b5c158490 2024-09-12T03:23:43.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.701+0000 7f5b52ffd640 1 --2- 172.21.15.27:0/545140459 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f5b3c06c7c0 0x7f5b3c06ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:43.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.702+0000 7f5b52ffd640 1 -- 172.21.15.27:0/545140459 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 5765+0+0 (secure 0 0 0) 0x7f5b440a9b10 con 0x7f5b5c158490 2024-09-12T03:23:43.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.702+0000 7f5b5a59c640 1 --2- 172.21.15.27:0/545140459 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f5b3c06c7c0 0x7f5b3c06ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:43.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.702+0000 7f5b5a59c640 1 --2- 172.21.15.27:0/545140459 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f5b3c06c7c0 0x7f5b3c06ec80 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f5b4c006fd0 tx=0x7f5b4c008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:43.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.706+0000 7f5b52ffd640 1 -- 172.21.15.27:0/545140459 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f5b4404c070 con 0x7f5b5c158490 2024-09-12T03:23:43.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.883+0000 7f5b60e8d640 1 -- 172.21.15.27:0/545140459 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5b5c161330 con 0x7f5b5c158490 2024-09-12T03:23:43.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.884+0000 7f5b52ffd640 1 -- 172.21.15.27:0/545140459 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v149) ==== 74+0+16780 (secure 0 0 0) 0x7f5b4407a0c0 con 0x7f5b5c158490 2024-09-12T03:23:43.884 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:23:43.884 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":149,"fsid":"d2df504e-2647-4385-a5db-9fdf69f487a0","created":"2024-09-12T03:16:07.778937+0000","modified":"2024-09-12T03:23:22.688848+0000","last_up_change":"2024-09-12T03:16:19.314778+0000","last_in_change":"2024-09-12T03:16:09.893086+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T03:16:20.097646+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_metadata","create_time":"2024-09-12T03:23:18.699292+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"148","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_data","create_time":"2024-09-12T03:23:20.022169+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"149","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1851116c-1fb3-4dc9-9772-f90e2ef4c355","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6817","nonce":3002321488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6819","nonce":3002321488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6823","nonce":3002321488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6821","nonce":3002321488}]},"public_addr":"172.21.15.27:6817/3002321488","cluster_addr":"172.21.15.27:6819/3002321488","heartbeat_back_addr":"172.21.15.27:6823/3002321488","heartbeat_front_addr":"172.21.15.27:6821/3002321488","state":["exists","up"]},{"osd":1,"uuid":"f38ae17d-73eb-4204-b99d-975fa806fb87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6809","nonce":1613102886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6811","nonce":1613102886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6815","nonce":1613102886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6813","nonce":1613102886}]},"public_addr":"172.21.15.27:6809/1613102886","cluster_addr":"172.21.15.27:6811/1613102886","heartbeat_back_addr":"172.21.15.27:6815/1613102886","heartbeat_front_addr":"172.21.15.27:6813/1613102886","state":["exists","up"]},{"osd":2,"uuid":"fb12dcc8-bdd0-42b8-b46d-8b10106224bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6825","nonce":3534723355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6827","nonce":3534723355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6831","nonce":3534723355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6829","nonce":3534723355}]},"public_addr":"172.21.15.27:6825/3534723355","cluster_addr":"172.21.15.27:6827/3534723355","heartbeat_back_addr":"172.21.15.27:6831/3534723355","heartbeat_front_addr":"172.21.15.27:6829/3534723355","state":["exists","up"]},{"osd":3,"uuid":"d74d3f49-9c24-4b9f-98e2-3ecd7b5ef3b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6801","nonce":80661598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6803","nonce":80661598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6807","nonce":80661598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6805","nonce":80661598}]},"public_addr":"172.21.15.27:6801/80661598","cluster_addr":"172.21.15.27:6803/80661598","heartbeat_back_addr":"172.21.15.27:6807/80661598","heartbeat_front_addr":"172.21.15.27:6805/80661598","state":["exists","up"]},{"osd":4,"uuid":"2bf123f2-1b0d-41c4-aa3b-805bab267e59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6817","nonce":3943189827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6819","nonce":3943189827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6823","nonce":3943189827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6821","nonce":3943189827}]},"public_addr":"172.21.15.45:6817/3943189827","cluster_addr":"172.21.15.45:6819/3943189827","heartbeat_back_addr":"172.21.15.45:6823/3943189827","heartbeat_front_addr":"172.21.15.45:6821/3943189827","state":["exists","up"]},{"osd":5,"uuid":"c49ba5da-91c6-4a86-a97b-4aba89c253ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6809","nonce":3620760330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6811","nonce":3620760330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6815","nonce":3620760330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6813","nonce":3620760330}]},"public_addr":"172.21.15.45:6809/3620760330","cluster_addr":"172.21.15.45:6811/3620760330","heartbeat_back_addr":"172.21.15.45:6815/3620760330","heartbeat_front_addr":"172.21.15.45:6813/3620760330","state":["exists","up"]},{"osd":6,"uuid":"b0123475-89f7-4a34-9e33-d797b99dd9dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6801","nonce":1989083510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6803","nonce":1989083510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6807","nonce":1989083510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6805","nonce":1989083510}]},"public_addr":"172.21.15.45:6801/1989083510","cluster_addr":"172.21.15.45:6803/1989083510","heartbeat_back_addr":"172.21.15.45:6807/1989083510","heartbeat_front_addr":"172.21.15.45:6805/1989083510","state":["exists","up"]},{"osd":7,"uuid":"4dad773d-b7d8-46fc-8a10-a972aad27b29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6825","nonce":1513300294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6827","nonce":1513300294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6831","nonce":1513300294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6829","nonce":1513300294}]},"public_addr":"172.21.15.45:6825/1513300294","cluster_addr":"172.21.15.45:6827/1513300294","heartbeat_back_addr":"172.21.15.45:6831/1513300294","heartbeat_front_addr":"172.21.15.45:6829/1513300294","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.881855+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:17.133252+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.735339+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.500742+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.477571+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:17.167667+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.470291+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.873743+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]},{"pool":19,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T03:23:43.886 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.886+0000 7f5b60e8d640 1 -- 172.21.15.27:0/545140459 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f5b3c06c7c0 msgr2=0x7f5b3c06ec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:43.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.886+0000 7f5b60e8d640 1 --2- 172.21.15.27:0/545140459 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f5b3c06c7c0 0x7f5b3c06ec80 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f5b4c006fd0 tx=0x7f5b4c008040 comp rx=0 tx=0).stop 2024-09-12T03:23:43.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.886+0000 7f5b60e8d640 1 -- 172.21.15.27:0/545140459 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5b5c158490 msgr2=0x7f5b5c15a1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:43.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.886+0000 7f5b60e8d640 1 --2- 172.21.15.27:0/545140459 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5b5c158490 0x7f5b5c15a1f0 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7f5b4402ae20 tx=0x7f5b5c0b7c70 comp rx=0 tx=0).stop 2024-09-12T03:23:43.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.886+0000 7f5b60e8d640 1 -- 172.21.15.27:0/545140459 shutdown_connections 2024-09-12T03:23:43.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.886+0000 7f5b60e8d640 1 --2- 172.21.15.27:0/545140459 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5b5c159360 0x7f5b5c1f3bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:43.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.886+0000 7f5b60e8d640 1 --2- 172.21.15.27:0/545140459 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f5b3c06c7c0 0x7f5b3c06ec80 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:43.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.886+0000 7f5b60e8d640 1 --2- 172.21.15.27:0/545140459 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5b5c158490 0x7f5b5c15a1f0 unknown :-1 s=CLOSED pgs=1611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:43.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.886+0000 7f5b60e8d640 1 --2- 172.21.15.27:0/545140459 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f5b48003fe0 0x7f5b5c159cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:43.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.886+0000 7f5b60e8d640 1 -- 172.21.15.27:0/545140459 >> 172.21.15.27:0/545140459 conn(0x7f5b5c06dce0 msgr2=0x7f5b5c160c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:43.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.886+0000 7f5b60e8d640 1 -- 172.21.15.27:0/545140459 shutdown_connections 2024-09-12T03:23:43.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:43.886+0000 7f5b60e8d640 1 -- 172.21.15.27:0/545140459 wait complete. 2024-09-12T03:23:43.898 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-12T03:23:43.898 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-12T03:23:44.001 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.001+0000 7f1d6bca5640 1 Processor -- start 2024-09-12T03:23:44.001 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.001+0000 7f1d6bca5640 1 -- start start 2024-09-12T03:23:44.002 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.002+0000 7f1d6bca5640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f1d64072d70 0x7f1d640b3560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:44.003 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.002+0000 7f1d6bca5640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1d6406e9b0 0x7f1d6406ee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:44.003 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.002+0000 7f1d6bca5640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1d640ce410 con 0x7f1d6415b2b0 2024-09-12T03:23:44.003 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.002+0000 7f1d6bca5640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f1d640720a0 con 0x7f1d64072d70 2024-09-12T03:23:44.003 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.002+0000 7f1d6bca5640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f1d640732c0 con 0x7f1d6406e9b0 2024-09-12T03:23:44.003 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.002+0000 7f1d69240640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f1d64072d70 0x7f1d640b3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:44.003 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.002+0000 7f1d69a41640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f1d6415b2b0 0x7f1d64072430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:52322/0 (socket says 172.21.15.27:52322) 2024-09-12T03:23:44.003 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.002+0000 7f1d69a41640 1 -- 172.21.15.27:0/437307757 learned_addr learned my addr 172.21.15.27:0/437307757 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:23:44.003 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.002+0000 7f1d6a242640 1 --2- 172.21.15.27:0/437307757 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1d6406e9b0 0x7f1d6406ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:44.003 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.003+0000 7f1d69240640 1 -- 172.21.15.27:0/437307757 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1d6406e9b0 msgr2=0x7f1d6406ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:44.003 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.003+0000 7f1d69240640 1 --2- 172.21.15.27:0/437307757 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1d6406e9b0 0x7f1d6406ee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:44.003 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.003+0000 7f1d69240640 1 -- 172.21.15.27:0/437307757 >> v1:172.21.15.27:6789/0 conn(0x7f1d6415b2b0 legacy=0x7f1d64072430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:44.004 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.003+0000 7f1d68a3f640 1 -- 172.21.15.27:0/437307757 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1056560555 0 0) 0x7f1d54001490 con 0x7f1d6415b2b0 2024-09-12T03:23:44.004 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.003+0000 7f1d69240640 1 -- 172.21.15.27:0/437307757 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d64071ae0 con 0x7f1d64072d70 2024-09-12T03:23:44.007 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.003+0000 7f1d69240640 1 --2- 172.21.15.27:0/437307757 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1d64072d70 0x7f1d640b3560 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7f1d580097a0 tx=0x7f1d5802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=272db6c137bc4246 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:44.008 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.003+0000 7f1d68a3f640 1 -- 172.21.15.27:0/437307757 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d58037070 con 0x7f1d64072d70 2024-09-12T03:23:44.008 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.003+0000 7f1d68a3f640 1 -- 172.21.15.27:0/437307757 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d58032440 con 0x7f1d64072d70 2024-09-12T03:23:44.008 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.003+0000 7f1d68a3f640 1 -- 172.21.15.27:0/437307757 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d58032810 con 0x7f1d64072d70 2024-09-12T03:23:44.008 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.004+0000 7f1d6bca5640 1 -- 172.21.15.27:0/437307757 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1d64072d70 msgr2=0x7f1d640b3560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:44.008 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.004+0000 7f1d6bca5640 1 --2- 172.21.15.27:0/437307757 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1d64072d70 0x7f1d640b3560 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7f1d580097a0 tx=0x7f1d5802a180 comp rx=0 tx=0).stop 2024-09-12T03:23:44.008 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.004+0000 7f1d6bca5640 1 -- 172.21.15.27:0/437307757 shutdown_connections 2024-09-12T03:23:44.008 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.004+0000 7f1d6bca5640 1 --2- 172.21.15.27:0/437307757 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1d6406e9b0 0x7f1d6406ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:44.008 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.004+0000 7f1d6bca5640 1 --2- 172.21.15.27:0/437307757 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1d64072d70 0x7f1d640b3560 unknown :-1 s=CLOSED pgs=1613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:44.008 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.004+0000 7f1d6bca5640 1 -- 172.21.15.27:0/437307757 >> 172.21.15.27:0/437307757 conn(0x7f1d64062c60 msgr2=0x7f1d6406bb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:44.008 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.004+0000 7f1d6bca5640 1 -- 172.21.15.27:0/437307757 shutdown_connections 2024-09-12T03:23:44.008 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.004+0000 7f1d6bca5640 1 -- 172.21.15.27:0/437307757 wait complete. 2024-09-12T03:23:44.008 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.005+0000 7f1d6bca5640 1 Processor -- start 2024-09-12T03:23:44.008 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.005+0000 7f1d6bca5640 1 -- start start 2024-09-12T03:23:44.009 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.006+0000 7f1d6bca5640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1d6406e9b0 0x7f1d6414e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:44.009 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.006+0000 7f1d6bca5640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1d64072d70 0x7f1d64155320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:44.009 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.006+0000 7f1d6bca5640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1d6415b2b0 0x7f1d6415db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:44.009 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.006+0000 7f1d6bca5640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f1d6415b690 con 0x7f1d6406e9b0 2024-09-12T03:23:44.009 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.006+0000 7f1d6bca5640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f1d640ce690 con 0x7f1d6415b2b0 2024-09-12T03:23:44.009 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.006+0000 7f1d6bca5640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f1d6415b810 con 0x7f1d64072d70 2024-09-12T03:23:44.009 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.006+0000 7f1d69a41640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1d6406e9b0 0x7f1d6414e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:44.009 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.006+0000 7f1d69a41640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1d6406e9b0 0x7f1d6414e000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59728/0 (socket says 172.21.15.27:59728) 2024-09-12T03:23:44.009 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.006+0000 7f1d69a41640 1 -- 172.21.15.27:0/199158429 learned_addr learned my addr 172.21.15.27:0/199158429 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:23:44.009 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.007+0000 7f1d6a242640 1 --2- 172.21.15.27:0/199158429 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1d6415b2b0 0x7f1d6415db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:44.009 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.007+0000 7f1d69240640 1 --2- 172.21.15.27:0/199158429 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1d64072d70 0x7f1d64155320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:44.009 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.007+0000 7f1d69a41640 1 -- 172.21.15.27:0/199158429 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1d64072d70 msgr2=0x7f1d64155320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:44.009 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.007+0000 7f1d69a41640 1 --2- 172.21.15.27:0/199158429 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1d64072d70 0x7f1d64155320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:44.010 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.007+0000 7f1d69a41640 1 -- 172.21.15.27:0/199158429 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1d6415b2b0 msgr2=0x7f1d6415db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:44.010 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.007+0000 7f1d69a41640 1 --2- 172.21.15.27:0/199158429 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1d6415b2b0 0x7f1d6415db90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:44.010 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.007+0000 7f1d69a41640 1 -- 172.21.15.27:0/199158429 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d64155da0 con 0x7f1d6406e9b0 2024-09-12T03:23:44.010 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.007+0000 7f1d6a242640 1 --2- 172.21.15.27:0/199158429 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1d6415b2b0 0x7f1d6415db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:23:44.010 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.007+0000 7f1d69a41640 1 --2- 172.21.15.27:0/199158429 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1d6406e9b0 0x7f1d6414e000 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7f1d5400ea90 tx=0x7f1d5400ef50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:44.010 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.007+0000 7f1d627fc640 1 -- 172.21.15.27:0/199158429 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d54004d70 con 0x7f1d6406e9b0 2024-09-12T03:23:44.010 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.007+0000 7f1d627fc640 1 -- 172.21.15.27:0/199158429 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d54016020 con 0x7f1d6406e9b0 2024-09-12T03:23:44.010 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.007+0000 7f1d6bca5640 1 -- 172.21.15.27:0/199158429 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d64156080 con 0x7f1d6406e9b0 2024-09-12T03:23:44.010 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.007+0000 7f1d627fc640 1 -- 172.21.15.27:0/199158429 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d54001ed0 con 0x7f1d6406e9b0 2024-09-12T03:23:44.010 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.007+0000 7f1d6bca5640 1 -- 172.21.15.27:0/199158429 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d64157fb0 con 0x7f1d6406e9b0 2024-09-12T03:23:44.011 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.008+0000 7f1d6bca5640 1 -- 172.21.15.27:0/199158429 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d640b3560 con 0x7f1d6406e9b0 2024-09-12T03:23:44.011 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.009+0000 7f1d627fc640 1 -- 172.21.15.27:0/199158429 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 25) ==== 91365+0+0 (secure 0 0 0) 0x7f1d540070d0 con 0x7f1d6406e9b0 2024-09-12T03:23:44.011 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.009+0000 7f1d627fc640 1 --2- 172.21.15.27:0/199158429 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f1d5006c7f0 0x7f1d5006ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:44.011 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.009+0000 7f1d627fc640 1 -- 172.21.15.27:0/199158429 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 5765+0+0 (secure 0 0 0) 0x7f1d5408bba0 con 0x7f1d6406e9b0 2024-09-12T03:23:44.011 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.009+0000 7f1d69240640 1 --2- 172.21.15.27:0/199158429 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f1d5006c7f0 0x7f1d5006ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:44.011 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.010+0000 7f1d69240640 1 --2- 172.21.15.27:0/199158429 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f1d5006c7f0 0x7f1d5006ecb0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f1d58034000 tx=0x7f1d5802e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:44.013 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.013+0000 7f1d627fc640 1 -- 172.21.15.27:0/199158429 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f1d54020030 con 0x7f1d6406e9b0 2024-09-12T03:23:44.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.226+0000 7f1d6bca5640 1 -- 172.21.15.27:0/199158429 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1d6415fcb0 con 0x7f1d6406e9b0 2024-09-12T03:23:44.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.226+0000 7f1d627fc640 1 -- 172.21.15.27:0/199158429 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 92 v92) ==== 94+0+5227 (secure 0 0 0) 0x7f1d640ce410 con 0x7f1d6406e9b0 2024-09-12T03:23:44.226 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:23:44.227 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":92,"btime":"2024-09-12T03:23:34:849854+0000","default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6917,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/3682270902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":3682270902},{"type":"v1","addr":"172.21.15.45:6834","nonce":3682270902}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":87},{"gid":7771,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/2399103579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":2399103579},{"type":"v1","addr":"172.21.15.45:6835","nonce":2399103579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":87},{"gid":8559,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6834/1045432254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":1045432254},{"type":"v1","addr":"172.21.15.27:6834","nonce":1045432254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":87}],"filesystems":[{"mdsmap":{"epoch":91,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-12T03:23:21.031667+0000","modified":"2024-09-12T03:23:23.983036+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8562},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8562":{"gid":8562,"name":"c","rank":0,"incarnation":88,"state":"up:active","state_seq":3,"addr":"172.21.15.27:6835/1241148443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":1241148443},{"type":"v1","addr":"172.21.15.27:6835","nonce":1241148443}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[25],"metadata_pool":24,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8562,"qdb_cluster":[8562]},"id":12}]} 2024-09-12T03:23:44.227 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 92 2024-09-12T03:23:44.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.228+0000 7f1d6bca5640 1 -- 172.21.15.27:0/199158429 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f1d5006c7f0 msgr2=0x7f1d5006ecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:44.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.228+0000 7f1d6bca5640 1 --2- 172.21.15.27:0/199158429 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f1d5006c7f0 0x7f1d5006ecb0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f1d58034000 tx=0x7f1d5802e000 comp rx=0 tx=0).stop 2024-09-12T03:23:44.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.228+0000 7f1d6bca5640 1 -- 172.21.15.27:0/199158429 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1d6406e9b0 msgr2=0x7f1d6414e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:44.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.228+0000 7f1d6bca5640 1 --2- 172.21.15.27:0/199158429 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1d6406e9b0 0x7f1d6414e000 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7f1d5400ea90 tx=0x7f1d5400ef50 comp rx=0 tx=0).stop 2024-09-12T03:23:44.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.228+0000 7f1d6bca5640 1 -- 172.21.15.27:0/199158429 shutdown_connections 2024-09-12T03:23:44.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.228+0000 7f1d6bca5640 1 --2- 172.21.15.27:0/199158429 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1d6415b2b0 0x7f1d6415db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:44.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.229+0000 7f1d6bca5640 1 --2- 172.21.15.27:0/199158429 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1d64072d70 0x7f1d64155320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:44.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.229+0000 7f1d6bca5640 1 --2- 172.21.15.27:0/199158429 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f1d5006c7f0 0x7f1d5006ecb0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:44.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.229+0000 7f1d6bca5640 1 --2- 172.21.15.27:0/199158429 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1d6406e9b0 0x7f1d6414e000 unknown :-1 s=CLOSED pgs=1614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:44.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.229+0000 7f1d6bca5640 1 -- 172.21.15.27:0/199158429 >> 172.21.15.27:0/199158429 conn(0x7f1d64062c60 msgr2=0x7f1d640b4b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:44.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.229+0000 7f1d6bca5640 1 -- 172.21.15.27:0/199158429 shutdown_connections 2024-09-12T03:23:44.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.229+0000 7f1d6bca5640 1 -- 172.21.15.27:0/199158429 wait complete. 2024-09-12T03:23:44.240 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-12T03:23:44.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.342+0000 7f9995053640 1 Processor -- start 2024-09-12T03:23:44.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.342+0000 7f9995053640 1 -- start start 2024-09-12T03:23:44.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.343+0000 7f9995053640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f99900b1ca0 0x7f999015f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:44.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.343+0000 7f9995053640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f99900cf660 0x7f99900d2780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:44.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.343+0000 7f9995053640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f999015fd70 con 0x7f999006f360 2024-09-12T03:23:44.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.343+0000 7f9995053640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f99900d22f0 con 0x7f99900b1ca0 2024-09-12T03:23:44.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.343+0000 7f9995053640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f999006cc40 con 0x7f99900cf660 2024-09-12T03:23:44.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.343+0000 7f998e59c640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f999006f360 0x7f99900cef50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:52336/0 (socket says 172.21.15.27:52336) 2024-09-12T03:23:44.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.343+0000 7f998e59c640 1 -- 172.21.15.27:0/1139399797 learned_addr learned my addr 172.21.15.27:0/1139399797 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:23:44.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.344+0000 7f998f59e640 1 --2- 172.21.15.27:0/1139399797 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f99900cf660 0x7f99900d2780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:44.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.344+0000 7f998ed9d640 1 --2- 172.21.15.27:0/1139399797 >> v2:172.21.15.45:3300/0 conn(0x7f99900b1ca0 0x7f999015f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:44.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.344+0000 7f998dd9b640 1 -- 172.21.15.27:0/1139399797 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3558147878 0 0) 0x7f9984001110 con 0x7f999006f360 2024-09-12T03:23:44.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.344+0000 7f998dd9b640 1 -- 172.21.15.27:0/1139399797 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9978003520 con 0x7f999006f360 2024-09-12T03:23:44.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.344+0000 7f998dd9b640 1 -- 172.21.15.27:0/1139399797 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2451562854 0 0) 0x7f9978003520 con 0x7f999006f360 2024-09-12T03:23:44.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.344+0000 7f998dd9b640 1 -- 172.21.15.27:0/1139399797 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f99900cf660 msgr2=0x7f99900d2780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:44.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.344+0000 7f998dd9b640 1 --2- 172.21.15.27:0/1139399797 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f99900cf660 0x7f99900d2780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:44.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.344+0000 7f998dd9b640 1 -- 172.21.15.27:0/1139399797 >> v2:172.21.15.45:3300/0 conn(0x7f99900b1ca0 msgr2=0x7f999015f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:44.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.344+0000 7f998dd9b640 1 --2- 172.21.15.27:0/1139399797 >> v2:172.21.15.45:3300/0 conn(0x7f99900b1ca0 0x7f999015f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:44.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.344+0000 7f998dd9b640 1 -- 172.21.15.27:0/1139399797 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99900d1ce0 con 0x7f999006f360 2024-09-12T03:23:44.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.344+0000 7f998dd9b640 1 -- 172.21.15.27:0/1139399797 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 847755213 0 0) 0x7f9984001840 con 0x7f999006f360 2024-09-12T03:23:44.348 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.344+0000 7f998dd9b640 1 -- 172.21.15.27:0/1139399797 >> v1:172.21.15.27:6789/0 conn(0x7f999006f360 legacy=0x7f99900cef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:44.348 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.344+0000 7f998dd9b640 1 --2- 172.21.15.27:0/1139399797 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f99780039b0 0x7f9978003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:44.348 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.344+0000 7f998dd9b640 1 --2- 172.21.15.27:0/1139399797 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f99900cf660 0x7f9978004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:44.348 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.344+0000 7f998dd9b640 1 --2- 172.21.15.27:0/1139399797 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9978004d40 0x7f99780051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:44.348 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.344+0000 7f998dd9b640 1 -- 172.21.15.27:0/1139399797 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f99900d22f0 con 0x7f9978004d40 2024-09-12T03:23:44.348 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.344+0000 7f998dd9b640 1 -- 172.21.15.27:0/1139399797 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f999006cc40 con 0x7f99780039b0 2024-09-12T03:23:44.348 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.344+0000 7f998dd9b640 1 -- 172.21.15.27:0/1139399797 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f997800b9c0 con 0x7f99900cf660 2024-09-12T03:23:44.348 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.344+0000 7f998f59e640 1 --2- 172.21.15.27:0/1139399797 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9978004d40 0x7f99780051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:44.348 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.344+0000 7f998ed9d640 1 --2- 172.21.15.27:0/1139399797 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f99780039b0 0x7f9978003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:44.348 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.344+0000 7f998e59c640 1 --2- 172.21.15.27:0/1139399797 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f99900cf660 0x7f9978004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:44.348 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.344+0000 7f998f59e640 1 -- 172.21.15.27:0/1139399797 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f99900cf660 msgr2=0x7f9978004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:44.348 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.344+0000 7f998f59e640 1 --2- 172.21.15.27:0/1139399797 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f99900cf660 0x7f9978004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:44.348 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.345+0000 7f998f59e640 1 -- 172.21.15.27:0/1139399797 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f99780039b0 msgr2=0x7f9978003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:44.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.345+0000 7f998f59e640 1 --2- 172.21.15.27:0/1139399797 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f99780039b0 0x7f9978003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:44.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.345+0000 7f998f59e640 1 -- 172.21.15.27:0/1139399797 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9978003520 con 0x7f9978004d40 2024-09-12T03:23:44.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.345+0000 7f998e59c640 1 --2- 172.21.15.27:0/1139399797 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f99900cf660 0x7f9978004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:23:44.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.345+0000 7f998f59e640 1 --2- 172.21.15.27:0/1139399797 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9978004d40 0x7f99780051c0 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7f9988009fc0 tx=0x7f9978005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1009488b62ceaced server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:44.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.345+0000 7f998dd9b640 1 -- 172.21.15.27:0/1139399797 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9988009c10 con 0x7f9978004d40 2024-09-12T03:23:44.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.345+0000 7f998dd9b640 1 -- 172.21.15.27:0/1139399797 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9988036070 con 0x7f9978004d40 2024-09-12T03:23:44.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.345+0000 7f998dd9b640 1 -- 172.21.15.27:0/1139399797 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9988031880 con 0x7f9978004d40 2024-09-12T03:23:44.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.346+0000 7f9995053640 1 -- 172.21.15.27:0/1139399797 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9978004d40 msgr2=0x7f99780051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:44.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.346+0000 7f9995053640 1 --2- 172.21.15.27:0/1139399797 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9978004d40 0x7f99780051c0 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7f9988009fc0 tx=0x7f9978005830 comp rx=0 tx=0).stop 2024-09-12T03:23:44.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.346+0000 7f998f59e640 1 -- 172.21.15.27:0/1139399797 reap_dead start 2024-09-12T03:23:44.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.346+0000 7f9995053640 1 -- 172.21.15.27:0/1139399797 shutdown_connections 2024-09-12T03:23:44.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.346+0000 7f9995053640 1 -- 172.21.15.27:0/1139399797 >> 172.21.15.27:0/1139399797 conn(0x7f9990073800 msgr2=0x7f999009ba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:44.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.346+0000 7f9995053640 1 -- 172.21.15.27:0/1139399797 shutdown_connections 2024-09-12T03:23:44.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.346+0000 7f9995053640 1 -- 172.21.15.27:0/1139399797 wait complete. 2024-09-12T03:23:44.352 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.348+0000 7f9995053640 1 Processor -- start 2024-09-12T03:23:44.352 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.348+0000 7f9995053640 1 -- start start 2024-09-12T03:23:44.352 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.348+0000 7f9995053640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f9978004d40 0x7f99900b54d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:44.352 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.348+0000 7f9995053640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f999015b7d0 0x7f9990153060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:44.352 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.348+0000 7f9995053640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f99900b0500 0x7f99900b0980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:44.353 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.348+0000 7f9995053640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f999006d1c0 con 0x7f99900b0500 2024-09-12T03:23:44.353 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.348+0000 7f9995053640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f99900d1f80 con 0x7f999015b7d0 2024-09-12T03:23:44.353 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.348+0000 7f9995053640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f999006d340 con 0x7f9978004d40 2024-09-12T03:23:44.353 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.348+0000 7f998f59e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f99900b0500 0x7f99900b0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:44.353 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.349+0000 7f998f59e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f99900b0500 0x7f99900b0980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59748/0 (socket says 172.21.15.27:59748) 2024-09-12T03:23:44.353 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.349+0000 7f998f59e640 1 -- 172.21.15.27:0/2978690007 learned_addr learned my addr 172.21.15.27:0/2978690007 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:23:44.353 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.349+0000 7f998e59c640 1 --2- 172.21.15.27:0/2978690007 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f999015b7d0 0x7f9990153060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:44.353 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.349+0000 7f998ed9d640 1 --2- 172.21.15.27:0/2978690007 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f9978004d40 0x7f99900b54d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:44.353 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.349+0000 7f998f59e640 1 -- 172.21.15.27:0/2978690007 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f9978004d40 msgr2=0x7f99900b54d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:44.353 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.349+0000 7f998f59e640 1 --2- 172.21.15.27:0/2978690007 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f9978004d40 0x7f99900b54d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:44.353 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.349+0000 7f998f59e640 1 -- 172.21.15.27:0/2978690007 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f999015b7d0 msgr2=0x7f9990153060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:44.353 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.349+0000 7f998f59e640 1 --2- 172.21.15.27:0/2978690007 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f999015b7d0 0x7f9990153060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:44.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.349+0000 7f998f59e640 1 -- 172.21.15.27:0/2978690007 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99900b5d00 con 0x7f99900b0500 2024-09-12T03:23:44.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.349+0000 7f998f59e640 1 --2- 172.21.15.27:0/2978690007 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f99900b0500 0x7f99900b0980 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7f9988033040 tx=0x7f9988009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:44.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.349+0000 7f99777fe640 1 -- 172.21.15.27:0/2978690007 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9988009c10 con 0x7f99900b0500 2024-09-12T03:23:44.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.349+0000 7f99777fe640 1 -- 172.21.15.27:0/2978690007 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9988031d30 con 0x7f99900b0500 2024-09-12T03:23:44.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.349+0000 7f99777fe640 1 -- 172.21.15.27:0/2978690007 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9988003bd0 con 0x7f99900b0500 2024-09-12T03:23:44.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.349+0000 7f9995053640 1 -- 172.21.15.27:0/2978690007 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99900b5fe0 con 0x7f99900b0500 2024-09-12T03:23:44.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.349+0000 7f9995053640 1 -- 172.21.15.27:0/2978690007 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99900b65a0 con 0x7f99900b0500 2024-09-12T03:23:44.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.350+0000 7f9995053640 1 -- 172.21.15.27:0/2978690007 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99900cfc40 con 0x7f99900b0500 2024-09-12T03:23:44.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.351+0000 7f99777fe640 1 -- 172.21.15.27:0/2978690007 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 25) ==== 91365+0+0 (secure 0 0 0) 0x7f9988005d10 con 0x7f99900b0500 2024-09-12T03:23:44.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.351+0000 7f99777fe640 1 --2- 172.21.15.27:0/2978690007 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f997c06c7c0 0x7f997c06ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:44.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.352+0000 7f99777fe640 1 -- 172.21.15.27:0/2978690007 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 5765+0+0 (secure 0 0 0) 0x7f99880aab90 con 0x7f99900b0500 2024-09-12T03:23:44.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.352+0000 7f998ed9d640 1 --2- 172.21.15.27:0/2978690007 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f997c06c7c0 0x7f997c06ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:44.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.352+0000 7f998ed9d640 1 --2- 172.21.15.27:0/2978690007 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f997c06c7c0 0x7f997c06ec80 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f9980009b40 tx=0x7f9980009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:44.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.356+0000 7f99777fe640 1 -- 172.21.15.27:0/2978690007 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f998804a060 con 0x7f99900b0500 2024-09-12T03:23:44.567 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.567+0000 7f9995053640 1 -- 172.21.15.27:0/2978690007 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f999015fb50 con 0x7f99900b0500 2024-09-12T03:23:44.567 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.567+0000 7f99777fe640 1 -- 172.21.15.27:0/2978690007 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 92 v92) ==== 94+0+5227 (secure 0 0 0) 0x7f998807b260 con 0x7f99900b0500 2024-09-12T03:23:44.568 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:23:44.568 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":92,"btime":"2024-09-12T03:23:34:849854+0000","default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6917,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/3682270902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":3682270902},{"type":"v1","addr":"172.21.15.45:6834","nonce":3682270902}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":87},{"gid":7771,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/2399103579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":2399103579},{"type":"v1","addr":"172.21.15.45:6835","nonce":2399103579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":87},{"gid":8559,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6834/1045432254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":1045432254},{"type":"v1","addr":"172.21.15.27:6834","nonce":1045432254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":87}],"filesystems":[{"mdsmap":{"epoch":91,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-12T03:23:21.031667+0000","modified":"2024-09-12T03:23:23.983036+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8562},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8562":{"gid":8562,"name":"c","rank":0,"incarnation":88,"state":"up:active","state_seq":3,"addr":"172.21.15.27:6835/1241148443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":1241148443},{"type":"v1","addr":"172.21.15.27:6835","nonce":1241148443}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[25],"metadata_pool":24,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8562,"qdb_cluster":[8562]},"id":12}]} 2024-09-12T03:23:44.568 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 92 2024-09-12T03:23:44.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.570+0000 7f9995053640 1 -- 172.21.15.27:0/2978690007 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f997c06c7c0 msgr2=0x7f997c06ec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:44.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.570+0000 7f9995053640 1 --2- 172.21.15.27:0/2978690007 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f997c06c7c0 0x7f997c06ec80 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f9980009b40 tx=0x7f9980009190 comp rx=0 tx=0).stop 2024-09-12T03:23:44.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.570+0000 7f9995053640 1 -- 172.21.15.27:0/2978690007 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f99900b0500 msgr2=0x7f99900b0980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:44.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.570+0000 7f9995053640 1 --2- 172.21.15.27:0/2978690007 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f99900b0500 0x7f99900b0980 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7f9988033040 tx=0x7f9988009750 comp rx=0 tx=0).stop 2024-09-12T03:23:44.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.570+0000 7f9995053640 1 -- 172.21.15.27:0/2978690007 shutdown_connections 2024-09-12T03:23:44.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.570+0000 7f9995053640 1 --2- 172.21.15.27:0/2978690007 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f997c06c7c0 0x7f997c06ec80 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:44.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.570+0000 7f9995053640 1 --2- 172.21.15.27:0/2978690007 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f99900b0500 0x7f99900b0980 unknown :-1 s=CLOSED pgs=1617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:44.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.570+0000 7f9995053640 1 --2- 172.21.15.27:0/2978690007 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f999015b7d0 0x7f9990153060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:44.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.570+0000 7f9995053640 1 --2- 172.21.15.27:0/2978690007 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f9978004d40 0x7f99900b54d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:44.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.570+0000 7f9995053640 1 -- 172.21.15.27:0/2978690007 >> 172.21.15.27:0/2978690007 conn(0x7f9990073800 msgr2=0x7f999015e110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:44.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.570+0000 7f9995053640 1 -- 172.21.15.27:0/2978690007 shutdown_connections 2024-09-12T03:23:44.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.570+0000 7f9995053640 1 -- 172.21.15.27:0/2978690007 wait complete. 2024-09-12T03:23:44.581 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-12T03:23:44.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.683+0000 7f5678fe8640 1 Processor -- start 2024-09-12T03:23:44.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.683+0000 7f5678fe8640 1 -- start start 2024-09-12T03:23:44.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.684+0000 7f5678fe8640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f5674071480 0x7f5674070af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:44.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.684+0000 7f5678fe8640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5674071980 con 0x7f567415e7f0 2024-09-12T03:23:44.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.684+0000 7f5678fe8640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f567406e700 con 0x7f5674073990 2024-09-12T03:23:44.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.684+0000 7f5678fe8640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f567406e9e0 con 0x7f5674071480 2024-09-12T03:23:44.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.684+0000 7f567259c640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f567415e7f0 0x7f5674073280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:52352/0 (socket says 172.21.15.27:52352) 2024-09-12T03:23:44.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.684+0000 7f567259c640 1 -- 172.21.15.27:0/4213073054 learned_addr learned my addr 172.21.15.27:0/4213073054 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:23:44.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.684+0000 7f5672d9d640 1 --2- 172.21.15.27:0/4213073054 >> v2:172.21.15.45:3300/0 conn(0x7f5674071480 0x7f5674070af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:44.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.685+0000 7f567159a640 1 -- 172.21.15.27:0/4213073054 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3555592628 0 0) 0x7f5674071980 con 0x7f567415e7f0 2024-09-12T03:23:44.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.685+0000 7f567159a640 1 -- 172.21.15.27:0/4213073054 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f565c003520 con 0x7f567415e7f0 2024-09-12T03:23:44.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.685+0000 7f567159a640 1 -- 172.21.15.27:0/4213073054 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4064914712 0 0) 0x7f565c003520 con 0x7f567415e7f0 2024-09-12T03:23:44.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.685+0000 7f567159a640 1 -- 172.21.15.27:0/4213073054 >> v2:172.21.15.45:3300/0 conn(0x7f5674071480 msgr2=0x7f5674070af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:44.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.685+0000 7f567159a640 1 --2- 172.21.15.27:0/4213073054 >> v2:172.21.15.45:3300/0 conn(0x7f5674071480 0x7f5674070af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:44.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.685+0000 7f567159a640 1 -- 172.21.15.27:0/4213073054 >> v1:172.21.15.45:6789/0 conn(0x7f5674073990 legacy=0x7f56740d0850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:44.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.685+0000 7f567159a640 1 -- 172.21.15.27:0/4213073054 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56740b1cc0 con 0x7f567415e7f0 2024-09-12T03:23:44.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.685+0000 7f567159a640 1 -- 172.21.15.27:0/4213073054 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 847755213 0 0) 0x7f5664001670 con 0x7f567415e7f0 2024-09-12T03:23:44.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.685+0000 7f567159a640 1 -- 172.21.15.27:0/4213073054 >> v1:172.21.15.27:6789/0 conn(0x7f567415e7f0 legacy=0x7f5674073280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:44.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.685+0000 7f567159a640 1 --2- 172.21.15.27:0/4213073054 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f565c0039b0 0x7f565c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:44.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.685+0000 7f567159a640 1 --2- 172.21.15.27:0/4213073054 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f565c004800 0x7f565c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:44.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.685+0000 7f567159a640 1 --2- 172.21.15.27:0/4213073054 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f565c0051c0 0x7f565c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:44.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.685+0000 7f567159a640 1 -- 172.21.15.27:0/4213073054 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f567406e9e0 con 0x7f565c004800 2024-09-12T03:23:44.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.685+0000 7f567159a640 1 -- 172.21.15.27:0/4213073054 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f565c00c160 con 0x7f565c0039b0 2024-09-12T03:23:44.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.685+0000 7f567159a640 1 -- 172.21.15.27:0/4213073054 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f565c00c360 con 0x7f565c0051c0 2024-09-12T03:23:44.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.685+0000 7f5671d9b640 1 --2- 172.21.15.27:0/4213073054 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f565c004800 0x7f565c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:44.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.685+0000 7f567259c640 1 --2- 172.21.15.27:0/4213073054 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f565c0039b0 0x7f565c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:44.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.685+0000 7f5672d9d640 1 --2- 172.21.15.27:0/4213073054 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f565c0051c0 0x7f565c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:44.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.685+0000 7f5671d9b640 1 -- 172.21.15.27:0/4213073054 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f565c0051c0 msgr2=0x7f565c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:44.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.685+0000 7f5671d9b640 1 --2- 172.21.15.27:0/4213073054 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f565c0051c0 0x7f565c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:44.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.685+0000 7f5671d9b640 1 -- 172.21.15.27:0/4213073054 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f565c0039b0 msgr2=0x7f565c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:44.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.685+0000 7f5671d9b640 1 --2- 172.21.15.27:0/4213073054 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f565c0039b0 0x7f565c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:44.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.685+0000 7f5671d9b640 1 -- 172.21.15.27:0/4213073054 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f565c003520 con 0x7f565c004800 2024-09-12T03:23:44.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.685+0000 7f5672d9d640 1 -- 172.21.15.27:0/4213073054 reap_dead start 2024-09-12T03:23:44.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.686+0000 7f5671d9b640 1 --2- 172.21.15.27:0/4213073054 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f565c004800 0x7f565c004c80 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7f5668001750 tx=0x7f565c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c0ab2255441703a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:44.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.686+0000 7f567159a640 1 -- 172.21.15.27:0/4213073054 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5668001370 con 0x7f565c004800 2024-09-12T03:23:44.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.686+0000 7f567159a640 1 -- 172.21.15.27:0/4213073054 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56680052d0 con 0x7f565c004800 2024-09-12T03:23:44.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.686+0000 7f567159a640 1 -- 172.21.15.27:0/4213073054 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f566802abc0 con 0x7f565c004800 2024-09-12T03:23:44.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.687+0000 7f5678fe8640 1 -- 172.21.15.27:0/4213073054 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f565c004800 msgr2=0x7f565c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:44.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.687+0000 7f5678fe8640 1 --2- 172.21.15.27:0/4213073054 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f565c004800 0x7f565c004c80 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7f5668001750 tx=0x7f565c005830 comp rx=0 tx=0).stop 2024-09-12T03:23:44.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.687+0000 7f5678fe8640 1 -- 172.21.15.27:0/4213073054 shutdown_connections 2024-09-12T03:23:44.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.687+0000 7f5678fe8640 1 --2- 172.21.15.27:0/4213073054 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f565c004800 0x7f565c004c80 unknown :-1 s=CLOSED pgs=1619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:44.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.687+0000 7f5678fe8640 1 -- 172.21.15.27:0/4213073054 >> 172.21.15.27:0/4213073054 conn(0x7f56740b4320 msgr2=0x7f56740b4700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:44.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.687+0000 7f5678fe8640 1 -- 172.21.15.27:0/4213073054 shutdown_connections 2024-09-12T03:23:44.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.687+0000 7f5678fe8640 1 -- 172.21.15.27:0/4213073054 wait complete. 2024-09-12T03:23:44.693 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.689+0000 7f5678fe8640 1 Processor -- start 2024-09-12T03:23:44.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.689+0000 7f5678fe8640 1 -- start start 2024-09-12T03:23:44.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.689+0000 7f5678fe8640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f565c004800 0x7f5674150240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:44.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.689+0000 7f5678fe8640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f567414d130 0x7f5674149e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:44.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.689+0000 7f5678fe8640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f567414a6f0 0x7f567414ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:44.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.690+0000 7f5678fe8640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f5674070af0 con 0x7f565c004800 2024-09-12T03:23:44.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.690+0000 7f5678fe8640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f56740b5620 con 0x7f567414d130 2024-09-12T03:23:44.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.690+0000 7f5678fe8640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f5674070c70 con 0x7f567414a6f0 2024-09-12T03:23:44.695 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.690+0000 7f567259c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f565c004800 0x7f5674150240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:44.695 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.690+0000 7f567259c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f565c004800 0x7f5674150240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59770/0 (socket says 172.21.15.27:59770) 2024-09-12T03:23:44.695 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.690+0000 7f567259c640 1 -- 172.21.15.27:0/712145869 learned_addr learned my addr 172.21.15.27:0/712145869 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:23:44.695 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.690+0000 7f5671d9b640 1 --2- 172.21.15.27:0/712145869 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f567414d130 0x7f5674149e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:44.695 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.690+0000 7f5672d9d640 1 --2- 172.21.15.27:0/712145869 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f567414a6f0 0x7f567414ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:44.695 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.690+0000 7f567259c640 1 -- 172.21.15.27:0/712145869 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f567414a6f0 msgr2=0x7f567414ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:44.695 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.690+0000 7f567259c640 1 --2- 172.21.15.27:0/712145869 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f567414a6f0 0x7f567414ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:44.695 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.690+0000 7f567259c640 1 -- 172.21.15.27:0/712145869 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f567414d130 msgr2=0x7f5674149e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:44.695 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.690+0000 7f567259c640 1 --2- 172.21.15.27:0/712145869 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f567414d130 0x7f5674149e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:44.696 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.690+0000 7f567259c640 1 -- 172.21.15.27:0/712145869 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f567414d5b0 con 0x7f565c004800 2024-09-12T03:23:44.696 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.690+0000 7f5672d9d640 1 --2- 172.21.15.27:0/712145869 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f567414a6f0 0x7f567414ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:23:44.696 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.690+0000 7f567259c640 1 --2- 172.21.15.27:0/712145869 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f565c004800 0x7f5674150240 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7f5664007290 tx=0x7f5664007750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:44.696 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.690+0000 7f5662ffd640 1 -- 172.21.15.27:0/712145869 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5664001660 con 0x7f565c004800 2024-09-12T03:23:44.696 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.690+0000 7f5662ffd640 1 -- 172.21.15.27:0/712145869 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5664018070 con 0x7f565c004800 2024-09-12T03:23:44.696 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.690+0000 7f5662ffd640 1 -- 172.21.15.27:0/712145869 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f566400fb80 con 0x7f565c004800 2024-09-12T03:23:44.696 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.690+0000 7f5678fe8640 1 -- 172.21.15.27:0/712145869 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56740aacd0 con 0x7f565c004800 2024-09-12T03:23:44.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.690+0000 7f5678fe8640 1 -- 172.21.15.27:0/712145869 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56740aa9f0 con 0x7f565c004800 2024-09-12T03:23:44.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.691+0000 7f5678fe8640 1 -- 172.21.15.27:0/712145869 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5674073280 con 0x7f565c004800 2024-09-12T03:23:44.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.692+0000 7f5662ffd640 1 -- 172.21.15.27:0/712145869 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 25) ==== 91365+0+0 (secure 0 0 0) 0x7f566400fd20 con 0x7f565c004800 2024-09-12T03:23:44.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.693+0000 7f5662ffd640 1 --2- 172.21.15.27:0/712145869 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f565406c960 0x7f565406ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:44.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.693+0000 7f5671d9b640 1 --2- 172.21.15.27:0/712145869 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f565406c960 0x7f565406ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:44.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.693+0000 7f5662ffd640 1 -- 172.21.15.27:0/712145869 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 5765+0+0 (secure 0 0 0) 0x7f5664089470 con 0x7f565c004800 2024-09-12T03:23:44.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.693+0000 7f5671d9b640 1 --2- 172.21.15.27:0/712145869 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f565406c960 0x7f565406ee20 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f5668032000 tx=0x7f566802e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:44.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.697+0000 7f5662ffd640 1 -- 172.21.15.27:0/712145869 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f56740aacd0 con 0x7f565c004800 2024-09-12T03:23:44.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.875+0000 7f5678fe8640 1 -- 172.21.15.27:0/712145869 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f56740b6770 con 0x7f565c004800 2024-09-12T03:23:44.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.876+0000 7f5662ffd640 1 -- 172.21.15.27:0/712145869 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v149) ==== 74+0+16780 (secure 0 0 0) 0x7f5664059a20 con 0x7f565c004800 2024-09-12T03:23:44.876 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:23:44.877 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":149,"fsid":"d2df504e-2647-4385-a5db-9fdf69f487a0","created":"2024-09-12T03:16:07.778937+0000","modified":"2024-09-12T03:23:22.688848+0000","last_up_change":"2024-09-12T03:16:19.314778+0000","last_in_change":"2024-09-12T03:16:09.893086+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T03:16:20.097646+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_metadata","create_time":"2024-09-12T03:23:18.699292+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"148","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_data","create_time":"2024-09-12T03:23:20.022169+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"149","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1851116c-1fb3-4dc9-9772-f90e2ef4c355","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6817","nonce":3002321488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6819","nonce":3002321488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6823","nonce":3002321488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6821","nonce":3002321488}]},"public_addr":"172.21.15.27:6817/3002321488","cluster_addr":"172.21.15.27:6819/3002321488","heartbeat_back_addr":"172.21.15.27:6823/3002321488","heartbeat_front_addr":"172.21.15.27:6821/3002321488","state":["exists","up"]},{"osd":1,"uuid":"f38ae17d-73eb-4204-b99d-975fa806fb87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6809","nonce":1613102886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6811","nonce":1613102886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6815","nonce":1613102886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6813","nonce":1613102886}]},"public_addr":"172.21.15.27:6809/1613102886","cluster_addr":"172.21.15.27:6811/1613102886","heartbeat_back_addr":"172.21.15.27:6815/1613102886","heartbeat_front_addr":"172.21.15.27:6813/1613102886","state":["exists","up"]},{"osd":2,"uuid":"fb12dcc8-bdd0-42b8-b46d-8b10106224bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6825","nonce":3534723355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6827","nonce":3534723355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6831","nonce":3534723355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6829","nonce":3534723355}]},"public_addr":"172.21.15.27:6825/3534723355","cluster_addr":"172.21.15.27:6827/3534723355","heartbeat_back_addr":"172.21.15.27:6831/3534723355","heartbeat_front_addr":"172.21.15.27:6829/3534723355","state":["exists","up"]},{"osd":3,"uuid":"d74d3f49-9c24-4b9f-98e2-3ecd7b5ef3b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6801","nonce":80661598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6803","nonce":80661598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6807","nonce":80661598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6805","nonce":80661598}]},"public_addr":"172.21.15.27:6801/80661598","cluster_addr":"172.21.15.27:6803/80661598","heartbeat_back_addr":"172.21.15.27:6807/80661598","heartbeat_front_addr":"172.21.15.27:6805/80661598","state":["exists","up"]},{"osd":4,"uuid":"2bf123f2-1b0d-41c4-aa3b-805bab267e59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6817","nonce":3943189827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6819","nonce":3943189827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6823","nonce":3943189827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6821","nonce":3943189827}]},"public_addr":"172.21.15.45:6817/3943189827","cluster_addr":"172.21.15.45:6819/3943189827","heartbeat_back_addr":"172.21.15.45:6823/3943189827","heartbeat_front_addr":"172.21.15.45:6821/3943189827","state":["exists","up"]},{"osd":5,"uuid":"c49ba5da-91c6-4a86-a97b-4aba89c253ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6809","nonce":3620760330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6811","nonce":3620760330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6815","nonce":3620760330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6813","nonce":3620760330}]},"public_addr":"172.21.15.45:6809/3620760330","cluster_addr":"172.21.15.45:6811/3620760330","heartbeat_back_addr":"172.21.15.45:6815/3620760330","heartbeat_front_addr":"172.21.15.45:6813/3620760330","state":["exists","up"]},{"osd":6,"uuid":"b0123475-89f7-4a34-9e33-d797b99dd9dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6801","nonce":1989083510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6803","nonce":1989083510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6807","nonce":1989083510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6805","nonce":1989083510}]},"public_addr":"172.21.15.45:6801/1989083510","cluster_addr":"172.21.15.45:6803/1989083510","heartbeat_back_addr":"172.21.15.45:6807/1989083510","heartbeat_front_addr":"172.21.15.45:6805/1989083510","state":["exists","up"]},{"osd":7,"uuid":"4dad773d-b7d8-46fc-8a10-a972aad27b29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6825","nonce":1513300294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6827","nonce":1513300294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6831","nonce":1513300294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6829","nonce":1513300294}]},"public_addr":"172.21.15.45:6825/1513300294","cluster_addr":"172.21.15.45:6827/1513300294","heartbeat_back_addr":"172.21.15.45:6831/1513300294","heartbeat_front_addr":"172.21.15.45:6829/1513300294","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.881855+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:17.133252+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.735339+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.500742+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.477571+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:17.167667+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.470291+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.873743+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]},{"pool":19,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T03:23:44.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.878+0000 7f5678fe8640 1 -- 172.21.15.27:0/712145869 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f565406c960 msgr2=0x7f565406ee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:44.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.878+0000 7f5678fe8640 1 --2- 172.21.15.27:0/712145869 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f565406c960 0x7f565406ee20 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f5668032000 tx=0x7f566802e000 comp rx=0 tx=0).stop 2024-09-12T03:23:44.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.878+0000 7f5678fe8640 1 -- 172.21.15.27:0/712145869 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f565c004800 msgr2=0x7f5674150240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:44.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.878+0000 7f5678fe8640 1 --2- 172.21.15.27:0/712145869 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f565c004800 0x7f5674150240 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7f5664007290 tx=0x7f5664007750 comp rx=0 tx=0).stop 2024-09-12T03:23:44.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.878+0000 7f5678fe8640 1 -- 172.21.15.27:0/712145869 shutdown_connections 2024-09-12T03:23:44.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.878+0000 7f5678fe8640 1 --2- 172.21.15.27:0/712145869 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f567414a6f0 0x7f567414ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:44.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.878+0000 7f5678fe8640 1 --2- 172.21.15.27:0/712145869 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f567414d130 0x7f5674149e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:44.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.878+0000 7f5678fe8640 1 --2- 172.21.15.27:0/712145869 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f565406c960 0x7f565406ee20 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:44.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.878+0000 7f5678fe8640 1 --2- 172.21.15.27:0/712145869 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f565c004800 0x7f5674150240 unknown :-1 s=CLOSED pgs=1620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:44.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.878+0000 7f5678fe8640 1 -- 172.21.15.27:0/712145869 >> 172.21.15.27:0/712145869 conn(0x7f56740b4320 msgr2=0x7f567415a210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:44.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.879+0000 7f5678fe8640 1 -- 172.21.15.27:0/712145869 shutdown_connections 2024-09-12T03:23:44.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.879+0000 7f5678fe8640 1 -- 172.21.15.27:0/712145869 wait complete. 2024-09-12T03:23:44.890 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-12T03:23:44.993 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.993+0000 7f43d4518640 1 Processor -- start 2024-09-12T03:23:44.993 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.993+0000 7f43d4518640 1 -- start start 2024-09-12T03:23:44.994 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.994+0000 7f43d4518640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f43cc0ae8d0 0x7f43cc0728b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:44.994 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.994+0000 7f43d4518640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f43cc072df0 0x7f43cc06e930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:44.994 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.994+0000 7f43d4518640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f43cc0aee90 con 0x7f43cc0b5a70 2024-09-12T03:23:44.995 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.994+0000 7f43d4518640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f43cc073240 con 0x7f43cc0ae8d0 2024-09-12T03:23:44.995 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.994+0000 7f43d4518640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f43cc0af170 con 0x7f43cc072df0 2024-09-12T03:23:44.995 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.994+0000 7f43d22b4640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f43cc0b5a70 0x7f43cc15f600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:52356/0 (socket says 172.21.15.27:52356) 2024-09-12T03:23:44.995 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.994+0000 7f43d22b4640 1 -- 172.21.15.27:0/3270942626 learned_addr learned my addr 172.21.15.27:0/3270942626 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:23:44.995 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.994+0000 7f43d1ab3640 1 --2- 172.21.15.27:0/3270942626 >> v2:172.21.15.27:3300/0 conn(0x7f43cc0ae8d0 0x7f43cc0728b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:44.995 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.994+0000 7f43d2ab5640 1 --2- 172.21.15.27:0/3270942626 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f43cc072df0 0x7f43cc06e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:44.995 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.994+0000 7f43d12b2640 1 -- 172.21.15.27:0/3270942626 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3016846042 0 0) 0x7f43cc0aee90 con 0x7f43cc0b5a70 2024-09-12T03:23:44.995 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.994+0000 7f43d12b2640 1 -- 172.21.15.27:0/3270942626 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f43b8003520 con 0x7f43cc0b5a70 2024-09-12T03:23:44.995 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.994+0000 7f43d1ab3640 1 -- 172.21.15.27:0/3270942626 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f43cc072df0 msgr2=0x7f43cc06e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:44.995 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.994+0000 7f43d1ab3640 1 --2- 172.21.15.27:0/3270942626 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f43cc072df0 0x7f43cc06e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:44.995 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.994+0000 7f43d1ab3640 1 -- 172.21.15.27:0/3270942626 >> v1:172.21.15.27:6789/0 conn(0x7f43cc0b5a70 legacy=0x7f43cc15f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:44.995 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.994+0000 7f43d1ab3640 1 -- 172.21.15.27:0/3270942626 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43cc0af450 con 0x7f43cc0ae8d0 2024-09-12T03:23:44.997 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.995+0000 7f43d1ab3640 1 --2- 172.21.15.27:0/3270942626 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43cc0ae8d0 0x7f43cc0728b0 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7f43bc0097a0 tx=0x7f43bc02a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36150de568332f9d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:44.997 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.995+0000 7f43d12b2640 1 -- 172.21.15.27:0/3270942626 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43bc037070 con 0x7f43cc0ae8d0 2024-09-12T03:23:44.997 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.995+0000 7f43d12b2640 1 -- 172.21.15.27:0/3270942626 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43bc02ae50 con 0x7f43cc0ae8d0 2024-09-12T03:23:44.997 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.995+0000 7f43d12b2640 1 -- 172.21.15.27:0/3270942626 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43bc032640 con 0x7f43cc0ae8d0 2024-09-12T03:23:44.997 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.996+0000 7f43d4518640 1 -- 172.21.15.27:0/3270942626 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43cc0ae8d0 msgr2=0x7f43cc0728b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:44.997 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.996+0000 7f43d4518640 1 --2- 172.21.15.27:0/3270942626 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43cc0ae8d0 0x7f43cc0728b0 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7f43bc0097a0 tx=0x7f43bc02a100 comp rx=0 tx=0).stop 2024-09-12T03:23:44.997 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.996+0000 7f43d4518640 1 -- 172.21.15.27:0/3270942626 shutdown_connections 2024-09-12T03:23:44.998 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.996+0000 7f43d4518640 1 --2- 172.21.15.27:0/3270942626 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f43cc072df0 0x7f43cc06e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:44.998 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.996+0000 7f43d4518640 1 --2- 172.21.15.27:0/3270942626 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43cc0ae8d0 0x7f43cc0728b0 unknown :-1 s=CLOSED pgs=1622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:44.998 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.996+0000 7f43d4518640 1 -- 172.21.15.27:0/3270942626 >> 172.21.15.27:0/3270942626 conn(0x7f43cc06ca30 msgr2=0x7f43cc06ce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:44.998 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.996+0000 7f43d4518640 1 -- 172.21.15.27:0/3270942626 shutdown_connections 2024-09-12T03:23:44.998 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.996+0000 7f43d4518640 1 -- 172.21.15.27:0/3270942626 wait complete. 2024-09-12T03:23:44.998 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.997+0000 7f43d4518640 1 Processor -- start 2024-09-12T03:23:44.998 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.997+0000 7f43d4518640 1 -- start start 2024-09-12T03:23:45.000 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.998+0000 7f43d4518640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43cc072df0 0x7f43cc156360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:45.000 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.998+0000 7f43d4518640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f43cc0ae8d0 0x7f43cc1568a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:45.000 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.998+0000 7f43d4518640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f43cc0b5a70 0x7f43cc15e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:45.000 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.998+0000 7f43d4518640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f43cc06eff0 con 0x7f43cc072df0 2024-09-12T03:23:45.000 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.998+0000 7f43d4518640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f43cc06ee70 con 0x7f43cc0b5a70 2024-09-12T03:23:45.001 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.998+0000 7f43d4518640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f43cc06f170 con 0x7f43cc0ae8d0 2024-09-12T03:23:45.001 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.998+0000 7f43d22b4640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43cc072df0 0x7f43cc156360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:45.001 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.998+0000 7f43d22b4640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43cc072df0 0x7f43cc156360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59790/0 (socket says 172.21.15.27:59790) 2024-09-12T03:23:45.001 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.998+0000 7f43d22b4640 1 -- 172.21.15.27:0/3292912995 learned_addr learned my addr 172.21.15.27:0/3292912995 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:23:45.001 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.998+0000 7f43d2ab5640 1 --2- 172.21.15.27:0/3292912995 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f43cc0b5a70 0x7f43cc15e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:45.001 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.998+0000 7f43d1ab3640 1 --2- 172.21.15.27:0/3292912995 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f43cc0ae8d0 0x7f43cc1568a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:45.001 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.998+0000 7f43d22b4640 1 -- 172.21.15.27:0/3292912995 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f43cc0ae8d0 msgr2=0x7f43cc1568a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:45.001 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.998+0000 7f43d22b4640 1 --2- 172.21.15.27:0/3292912995 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f43cc0ae8d0 0x7f43cc1568a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:45.001 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.998+0000 7f43d22b4640 1 -- 172.21.15.27:0/3292912995 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f43cc0b5a70 msgr2=0x7f43cc15e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:45.001 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.998+0000 7f43d22b4640 1 --2- 172.21.15.27:0/3292912995 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f43cc0b5a70 0x7f43cc15e710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:45.001 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.998+0000 7f43d22b4640 1 -- 172.21.15.27:0/3292912995 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43cc15c920 con 0x7f43cc072df0 2024-09-12T03:23:45.001 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.998+0000 7f43d2ab5640 1 --2- 172.21.15.27:0/3292912995 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f43cc0b5a70 0x7f43cc15e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:23:45.002 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.998+0000 7f43d22b4640 1 --2- 172.21.15.27:0/3292912995 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43cc072df0 0x7f43cc156360 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7f43c800c400 tx=0x7f43c800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:45.002 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.999+0000 7f43c2ffd640 1 -- 172.21.15.27:0/3292912995 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43c8017070 con 0x7f43cc072df0 2024-09-12T03:23:45.002 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.999+0000 7f43c2ffd640 1 -- 172.21.15.27:0/3292912995 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43c8005d00 con 0x7f43cc072df0 2024-09-12T03:23:45.002 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.999+0000 7f43d4518640 1 -- 172.21.15.27:0/3292912995 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43cc15cc00 con 0x7f43cc072df0 2024-09-12T03:23:45.002 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.999+0000 7f43c2ffd640 1 -- 172.21.15.27:0/3292912995 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43c8007310 con 0x7f43cc072df0 2024-09-12T03:23:45.002 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.999+0000 7f43d4518640 1 -- 172.21.15.27:0/3292912995 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43cc14de30 con 0x7f43cc072df0 2024-09-12T03:23:45.002 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:44.999+0000 7f43d4518640 1 -- 172.21.15.27:0/3292912995 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43cc06e930 con 0x7f43cc072df0 2024-09-12T03:23:45.002 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:45.000+0000 7f43c2ffd640 1 -- 172.21.15.27:0/3292912995 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 25) ==== 91365+0+0 (secure 0 0 0) 0x7f43c801a070 con 0x7f43cc072df0 2024-09-12T03:23:45.002 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:45.001+0000 7f43c2ffd640 1 --2- 172.21.15.27:0/3292912995 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f43b006c7c0 0x7f43b006ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:45.003 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:45.001+0000 7f43c2ffd640 1 -- 172.21.15.27:0/3292912995 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 5765+0+0 (secure 0 0 0) 0x7f43c80889c0 con 0x7f43cc072df0 2024-09-12T03:23:45.003 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:45.001+0000 7f43d1ab3640 1 --2- 172.21.15.27:0/3292912995 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f43b006c7c0 0x7f43b006ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:45.003 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:45.001+0000 7f43d1ab3640 1 --2- 172.21.15.27:0/3292912995 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f43b006c7c0 0x7f43b006ec80 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f43bc034000 tx=0x7f43bc02e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:45.004 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:45.005+0000 7f43c2ffd640 1 -- 172.21.15.27:0/3292912995 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f43c8058f70 con 0x7f43cc072df0 2024-09-12T03:23:45.215 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:45.215+0000 7f43d4518640 1 -- 172.21.15.27:0/3292912995 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f43cc072000 con 0x7f43cc072df0 2024-09-12T03:23:45.216 INFO:tasks.ceph.mon.a.smithi027.stderr:2024-09-12T03:23:45.216+0000 7f959bfd9640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-12T03:23:45.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:45.217+0000 7f43c2ffd640 1 -- 172.21.15.27:0/3292912995 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v93) ==== 161+0+0 (secure 0 0 0) 0x7f43c8058870 con 0x7f43cc072df0 2024-09-12T03:23:45.217 INFO:teuthology.orchestra.run.smithi027.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-12T03:23:45.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:45.220+0000 7f43d4518640 1 -- 172.21.15.27:0/3292912995 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f43b006c7c0 msgr2=0x7f43b006ec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:45.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:45.220+0000 7f43d4518640 1 --2- 172.21.15.27:0/3292912995 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f43b006c7c0 0x7f43b006ec80 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f43bc034000 tx=0x7f43bc02e000 comp rx=0 tx=0).stop 2024-09-12T03:23:45.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:45.220+0000 7f43d4518640 1 -- 172.21.15.27:0/3292912995 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43cc072df0 msgr2=0x7f43cc156360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:45.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:45.220+0000 7f43d4518640 1 --2- 172.21.15.27:0/3292912995 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43cc072df0 0x7f43cc156360 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7f43c800c400 tx=0x7f43c800c8c0 comp rx=0 tx=0).stop 2024-09-12T03:23:45.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:45.220+0000 7f43d4518640 1 -- 172.21.15.27:0/3292912995 shutdown_connections 2024-09-12T03:23:45.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:45.220+0000 7f43d4518640 1 --2- 172.21.15.27:0/3292912995 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f43cc0b5a70 0x7f43cc15e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:45.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:45.220+0000 7f43d4518640 1 --2- 172.21.15.27:0/3292912995 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f43cc0ae8d0 0x7f43cc1568a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:45.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:45.220+0000 7f43d4518640 1 --2- 172.21.15.27:0/3292912995 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f43b006c7c0 0x7f43b006ec80 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:45.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:45.220+0000 7f43d4518640 1 --2- 172.21.15.27:0/3292912995 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43cc072df0 0x7f43cc156360 unknown :-1 s=CLOSED pgs=1623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:45.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:45.220+0000 7f43d4518640 1 -- 172.21.15.27:0/3292912995 >> 172.21.15.27:0/3292912995 conn(0x7f43cc06ca30 msgr2=0x7f43cc072520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:45.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:45.221+0000 7f43d4518640 1 -- 172.21.15.27:0/3292912995 shutdown_connections 2024-09-12T03:23:45.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:45.221+0000 7f43d4518640 1 -- 172.21.15.27:0/3292912995 wait complete. 2024-09-12T03:23:45.232 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-12T03:23:45.259 INFO:tasks.ceph.mds.c.smithi027.stdout:starting mds.c at 2024-09-12T03:23:45.338 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:45.338+0000 7f1e70fb7640 1 Processor -- start 2024-09-12T03:23:45.338 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:45.338+0000 7f1e70fb7640 1 -- start start 2024-09-12T03:23:45.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:45.338+0000 7f1e70fb7640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f1e6c162040 0x7f1e6c0d1da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:45.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:45.338+0000 7f1e70fb7640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1e6c0d2e30 con 0x7f1e6c0d3790 2024-09-12T03:23:45.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:45.338+0000 7f1e70fb7640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1e6c162420 con 0x7f1e6c06dfd0 2024-09-12T03:23:45.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:45.338+0000 7f1e70fb7640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f1e6c0d30d0 con 0x7f1e6c162040 2024-09-12T03:23:45.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:45.339+0000 7f1e63fff640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f1e6c0d3790 0x7f1e6c06d8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:52362/0 (socket says 172.21.15.27:52362) 2024-09-12T03:23:45.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:45.339+0000 7f1e63fff640 1 -- 172.21.15.27:0/162434192 learned_addr learned my addr 172.21.15.27:0/162434192 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:23:45.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:45.339+0000 7f1e69d9b640 1 -- 172.21.15.27:0/162434192 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2612301639 0 0) 0x7f1e6c0d2e30 con 0x7f1e6c0d3790 2024-09-12T03:23:45.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:45.339+0000 7f1e6a59c640 1 --2- 172.21.15.27:0/162434192 >> v2:172.21.15.45:3300/0 conn(0x7f1e6c162040 0x7f1e6c0d1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:45.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:45.339+0000 7f1e69d9b640 1 -- 172.21.15.27:0/162434192 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1e58003520 con 0x7f1e6c0d3790 2024-09-12T03:23:45.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:45.339+0000 7f1e69d9b640 1 -- 172.21.15.27:0/162434192 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1768356140 0 0) 0x7f1e58003520 con 0x7f1e6c0d3790 2024-09-12T03:23:45.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:45.339+0000 7f1e69d9b640 1 -- 172.21.15.27:0/162434192 >> v2:172.21.15.45:3300/0 conn(0x7f1e6c162040 msgr2=0x7f1e6c0d1da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:45.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:45.339+0000 7f1e69d9b640 1 --2- 172.21.15.27:0/162434192 >> v2:172.21.15.45:3300/0 conn(0x7f1e6c162040 0x7f1e6c0d1da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:45.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:45.339+0000 7f1e69d9b640 1 -- 172.21.15.27:0/162434192 >> v1:172.21.15.45:6789/0 conn(0x7f1e6c06dfd0 legacy=0x7f1e6c0cf320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:45.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:45.339+0000 7f1e69d9b640 1 -- 172.21.15.27:0/162434192 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e6c0d33b0 con 0x7f1e6c0d3790 2024-09-12T03:23:45.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:45.339+0000 7f1e69d9b640 1 -- 172.21.15.27:0/162434192 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 847755213 0 0) 0x7f1e54001670 con 0x7f1e6c0d3790 2024-09-12T03:23:45.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:45.339+0000 7f1e69d9b640 1 -- 172.21.15.27:0/162434192 >> v1:172.21.15.27:6789/0 conn(0x7f1e6c0d3790 legacy=0x7f1e6c06d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:45.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:45.339+0000 7f1e69d9b640 1 --2- 172.21.15.27:0/162434192 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1e58003a00 0x7f1e58003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:45.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:45.339+0000 7f1e69d9b640 1 --2- 172.21.15.27:0/162434192 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1e58004800 0x7f1e58004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:45.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:45.339+0000 7f1e69d9b640 1 --2- 172.21.15.27:0/162434192 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1e580051c0 0x7f1e5800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:45.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:45.339+0000 7f1e69d9b640 1 -- 172.21.15.27:0/162434192 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f1e6c0d30d0 con 0x7f1e580051c0 2024-09-12T03:23:45.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:45.339+0000 7f1e69d9b640 1 -- 172.21.15.27:0/162434192 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f1e5800c0c0 con 0x7f1e58004800 2024-09-12T03:23:45.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:45.339+0000 7f1e69d9b640 1 -- 172.21.15.27:0/162434192 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f1e5800c2c0 con 0x7f1e58003a00 2024-09-12T03:23:45.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:45.339+0000 7f1e6ad9d640 1 --2- 172.21.15.27:0/162434192 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1e580051c0 0x7f1e5800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:45.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:45.339+0000 7f1e63fff640 1 --2- 172.21.15.27:0/162434192 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1e58004800 0x7f1e58004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:45.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:45.339+0000 7f1e6a59c640 1 --2- 172.21.15.27:0/162434192 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1e58003a00 0x7f1e58003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:45.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:45.340+0000 7f1e6ad9d640 1 -- 172.21.15.27:0/162434192 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1e58003a00 msgr2=0x7f1e58003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:45.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:45.340+0000 7f1e6ad9d640 1 --2- 172.21.15.27:0/162434192 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1e58003a00 0x7f1e58003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:45.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:45.340+0000 7f1e6ad9d640 1 -- 172.21.15.27:0/162434192 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1e58004800 msgr2=0x7f1e58004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:45.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:45.340+0000 7f1e6ad9d640 1 --2- 172.21.15.27:0/162434192 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1e58004800 0x7f1e58004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:45.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:45.340+0000 7f1e6ad9d640 1 -- 172.21.15.27:0/162434192 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e58003520 con 0x7f1e580051c0 2024-09-12T03:23:45.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:45.340+0000 7f1e6ad9d640 1 -- 172.21.15.27:0/162434192 reap_dead start 2024-09-12T03:23:45.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:45.340+0000 7f1e6ad9d640 1 --2- 172.21.15.27:0/162434192 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1e580051c0 0x7f1e5800b9c0 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7f1e6c0b7840 tx=0x7f1e58005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec4afbf02c9c67e3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:45.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:45.341+0000 7f1e69d9b640 1 -- 172.21.15.27:0/162434192 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e64001a70 con 0x7f1e580051c0 2024-09-12T03:23:45.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:45.341+0000 7f1e69d9b640 1 -- 172.21.15.27:0/162434192 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e6402ae10 con 0x7f1e580051c0 2024-09-12T03:23:45.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:45.341+0000 7f1e69d9b640 1 -- 172.21.15.27:0/162434192 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e64003bb0 con 0x7f1e580051c0 2024-09-12T03:23:45.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:45.341+0000 7f1e70fb7640 1 -- 172.21.15.27:0/162434192 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1e580051c0 msgr2=0x7f1e5800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:45.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:45.341+0000 7f1e70fb7640 1 --2- 172.21.15.27:0/162434192 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1e580051c0 0x7f1e5800b9c0 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7f1e6c0b7840 tx=0x7f1e58005830 comp rx=0 tx=0).stop 2024-09-12T03:23:45.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:45.341+0000 7f1e70fb7640 1 -- 172.21.15.27:0/162434192 shutdown_connections 2024-09-12T03:23:45.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:45.341+0000 7f1e70fb7640 1 --2- 172.21.15.27:0/162434192 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1e580051c0 0x7f1e5800b9c0 unknown :-1 s=CLOSED pgs=1627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:45.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:45.341+0000 7f1e70fb7640 1 -- 172.21.15.27:0/162434192 >> 172.21.15.27:0/162434192 conn(0x7f1e6c0b6700 msgr2=0x7f1e6c0b6b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:45.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:45.341+0000 7f1e70fb7640 1 -- 172.21.15.27:0/162434192 shutdown_connections 2024-09-12T03:23:45.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:45.341+0000 7f1e70fb7640 1 -- 172.21.15.27:0/162434192 wait complete. 2024-09-12T03:23:45.352 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:45.342+0000 7f1e70fb7640 1 Processor -- start 2024-09-12T03:23:45.352 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:45.343+0000 7f1e70fb7640 1 -- start start 2024-09-12T03:23:45.353 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:45.343+0000 7f1e70fb7640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1e580051c0 0x7f1e6c0ab220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:45.353 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:45.343+0000 7f1e70fb7640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1e6c0ab760 0x7f1e6c160d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:45.353 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:45.343+0000 7f1e70fb7640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1e6c1612c0 0x7f1e6c1f80e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:45.353 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:45.343+0000 7f1e70fb7640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f1e6c0d4150 con 0x7f1e6c0ab760 2024-09-12T03:23:45.353 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:45.343+0000 7f1e70fb7640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f1e6c0d4530 con 0x7f1e6c1612c0 2024-09-12T03:23:45.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:45.343+0000 7f1e70fb7640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f1e6c0d42d0 con 0x7f1e580051c0 2024-09-12T03:23:45.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:45.343+0000 7f1e63fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1e6c0ab760 0x7f1e6c160d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:45.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:45.343+0000 7f1e63fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1e6c0ab760 0x7f1e6c160d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59834/0 (socket says 172.21.15.27:59834) 2024-09-12T03:23:45.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:45.343+0000 7f1e63fff640 1 -- 172.21.15.27:0/962111243 learned_addr learned my addr 172.21.15.27:0/962111243 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:23:45.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:45.343+0000 7f1e6ad9d640 1 --2- 172.21.15.27:0/962111243 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1e6c1612c0 0x7f1e6c1f80e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:45.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:45.343+0000 7f1e6a59c640 1 --2- 172.21.15.27:0/962111243 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1e580051c0 0x7f1e6c0ab220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:45.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:45.344+0000 7f1e63fff640 1 -- 172.21.15.27:0/962111243 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1e580051c0 msgr2=0x7f1e6c0ab220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:45.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:45.344+0000 7f1e63fff640 1 --2- 172.21.15.27:0/962111243 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1e580051c0 0x7f1e6c0ab220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:45.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:45.344+0000 7f1e63fff640 1 -- 172.21.15.27:0/962111243 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1e6c1612c0 msgr2=0x7f1e6c1f80e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:45.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:45.344+0000 7f1e63fff640 1 --2- 172.21.15.27:0/962111243 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1e6c1612c0 0x7f1e6c1f80e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:45.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:45.344+0000 7f1e63fff640 1 -- 172.21.15.27:0/962111243 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e6c159f90 con 0x7f1e6c0ab760 2024-09-12T03:23:45.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:45.344+0000 7f1e6a59c640 1 --2- 172.21.15.27:0/962111243 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1e580051c0 0x7f1e6c0ab220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:23:45.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:45.344+0000 7f1e63fff640 1 --2- 172.21.15.27:0/962111243 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1e6c0ab760 0x7f1e6c160d80 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7f1e54007290 tx=0x7f1e54007750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:45.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:45.344+0000 7f1e62ffd640 1 -- 172.21.15.27:0/962111243 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e54001650 con 0x7f1e6c0ab760 2024-09-12T03:23:45.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:45.344+0000 7f1e62ffd640 1 -- 172.21.15.27:0/962111243 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e54018070 con 0x7f1e6c0ab760 2024-09-12T03:23:45.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:45.344+0000 7f1e70fb7640 1 -- 172.21.15.27:0/962111243 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e6c15a270 con 0x7f1e6c0ab760 2024-09-12T03:23:45.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:45.344+0000 7f1e62ffd640 1 -- 172.21.15.27:0/962111243 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e5400fb20 con 0x7f1e6c0ab760 2024-09-12T03:23:45.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:45.344+0000 7f1e70fb7640 1 -- 172.21.15.27:0/962111243 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e6c15a830 con 0x7f1e6c0ab760 2024-09-12T03:23:45.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:45.345+0000 7f1e70fb7640 1 -- 172.21.15.27:0/962111243 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e6c15de40 con 0x7f1e6c0ab760 2024-09-12T03:23:45.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:45.346+0000 7f1e62ffd640 1 -- 172.21.15.27:0/962111243 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 25) ==== 91365+0+0 (secure 0 0 0) 0x7f1e5400fcc0 con 0x7f1e6c0ab760 2024-09-12T03:23:45.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:45.346+0000 7f1e62ffd640 1 --2- 172.21.15.27:0/962111243 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f1e4c06c7c0 0x7f1e4c06ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:45.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:45.346+0000 7f1e62ffd640 1 -- 172.21.15.27:0/962111243 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(150..150 src has 1..150) ==== 5851+0+0 (secure 0 0 0) 0x7f1e54089430 con 0x7f1e6c0ab760 2024-09-12T03:23:45.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:45.346+0000 7f1e6a59c640 1 --2- 172.21.15.27:0/962111243 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f1e4c06c7c0 0x7f1e4c06ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:45.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:45.347+0000 7f1e6a59c640 1 --2- 172.21.15.27:0/962111243 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f1e4c06c7c0 0x7f1e4c06ec80 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f1e5c009b40 tx=0x7f1e5c008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:45.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:45.350+0000 7f1e62ffd640 1 -- 172.21.15.27:0/962111243 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f1e6c15de40 con 0x7f1e6c0ab760 2024-09-12T03:23:45.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:45.562+0000 7f1e70fb7640 1 -- 172.21.15.27:0/962111243 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f1e6c06e6e0 con 0x7f1e6c0ab760 2024-09-12T03:23:46.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:46.219+0000 7f1e62ffd640 1 -- 172.21.15.27:0/962111243 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v94) ==== 104+0+0 (secure 0 0 0) 0x7f1e540599e0 con 0x7f1e6c0ab760 2024-09-12T03:23:46.222 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:46.221+0000 7f1e70fb7640 1 -- 172.21.15.27:0/962111243 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f1e4c06c7c0 msgr2=0x7f1e4c06ec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:46.222 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:46.222+0000 7f1e70fb7640 1 --2- 172.21.15.27:0/962111243 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f1e4c06c7c0 0x7f1e4c06ec80 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f1e5c009b40 tx=0x7f1e5c008040 comp rx=0 tx=0).stop 2024-09-12T03:23:46.222 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:46.222+0000 7f1e70fb7640 1 -- 172.21.15.27:0/962111243 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1e6c0ab760 msgr2=0x7f1e6c160d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:46.222 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:46.222+0000 7f1e70fb7640 1 --2- 172.21.15.27:0/962111243 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1e6c0ab760 0x7f1e6c160d80 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7f1e54007290 tx=0x7f1e54007750 comp rx=0 tx=0).stop 2024-09-12T03:23:46.222 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:46.222+0000 7f1e70fb7640 1 -- 172.21.15.27:0/962111243 shutdown_connections 2024-09-12T03:23:46.222 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:46.222+0000 7f1e70fb7640 1 --2- 172.21.15.27:0/962111243 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1e6c1612c0 0x7f1e6c1f80e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:46.222 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:46.222+0000 7f1e70fb7640 1 --2- 172.21.15.27:0/962111243 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f1e4c06c7c0 0x7f1e4c06ec80 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:46.222 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:46.222+0000 7f1e70fb7640 1 --2- 172.21.15.27:0/962111243 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1e6c0ab760 0x7f1e6c160d80 unknown :-1 s=CLOSED pgs=1628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:46.222 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:46.222+0000 7f1e70fb7640 1 --2- 172.21.15.27:0/962111243 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1e580051c0 0x7f1e6c0ab220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:46.222 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:46.222+0000 7f1e70fb7640 1 -- 172.21.15.27:0/962111243 >> 172.21.15.27:0/962111243 conn(0x7f1e6c0b6700 msgr2=0x7f1e6c0cb880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:46.223 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:46.222+0000 7f1e70fb7640 1 -- 172.21.15.27:0/962111243 shutdown_connections 2024-09-12T03:23:46.223 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:46.222+0000 7f1e70fb7640 1 -- 172.21.15.27:0/962111243 wait complete. 2024-09-12T03:23:46.235 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-12T03:23:46.389 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:46.389+0000 7f5969d59640 1 Processor -- start 2024-09-12T03:23:46.389 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:46.389+0000 7f5969d59640 1 -- start start 2024-09-12T03:23:46.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:46.390+0000 7f5969d59640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f59640b6540 0x7f59640af370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:46.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:46.390+0000 7f5969d59640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f596406d730 0x7f596406ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:46.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:46.390+0000 7f5969d59640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f59640b5940 con 0x7f596415e900 2024-09-12T03:23:46.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:46.390+0000 7f5969d59640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f596415ece0 con 0x7f596406d730 2024-09-12T03:23:46.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:46.390+0000 7f5969d59640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f596406db40 con 0x7f59640b6540 2024-09-12T03:23:46.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:46.390+0000 7f5963fff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f596406d730 0x7f596406ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:46.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:46.390+0000 7f59637fe640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f596415e900 0x7f59640b5e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:52368/0 (socket says 172.21.15.27:52368) 2024-09-12T03:23:46.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:46.390+0000 7f59637fe640 1 -- 172.21.15.27:0/4108940546 learned_addr learned my addr 172.21.15.27:0/4108940546 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:23:46.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:46.390+0000 7f5963fff640 1 -- 172.21.15.27:0/4108940546 >> v2:172.21.15.45:3300/0 conn(0x7f59640b6540 msgr2=0x7f59640af370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:46.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:46.390+0000 7f5963fff640 1 --2- 172.21.15.27:0/4108940546 >> v2:172.21.15.45:3300/0 conn(0x7f59640b6540 0x7f59640af370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:46.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:46.390+0000 7f5963fff640 1 -- 172.21.15.27:0/4108940546 >> v1:172.21.15.27:6789/0 conn(0x7f596415e900 legacy=0x7f59640b5e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:46.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:46.390+0000 7f5963fff640 1 -- 172.21.15.27:0/4108940546 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59640af950 con 0x7f596406d730 2024-09-12T03:23:46.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:46.390+0000 7f59627fc640 1 -- 172.21.15.27:0/4108940546 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3456728654 0 0) 0x7f5954001110 con 0x7f596415e900 2024-09-12T03:23:46.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:46.391+0000 7f5963fff640 1 --2- 172.21.15.27:0/4108940546 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f596406d730 0x7f596406ee80 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7f59641622d0 tx=0x7f595c02a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9371e5e1e7ca287 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:46.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:46.391+0000 7f59627fc640 1 -- 172.21.15.27:0/4108940546 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f595c007740 con 0x7f596406d730 2024-09-12T03:23:46.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:46.391+0000 7f59627fc640 1 -- 172.21.15.27:0/4108940546 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f595c0078e0 con 0x7f596406d730 2024-09-12T03:23:46.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:46.391+0000 7f59627fc640 1 -- 172.21.15.27:0/4108940546 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f595c007cb0 con 0x7f596406d730 2024-09-12T03:23:46.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:46.392+0000 7f5969d59640 1 -- 172.21.15.27:0/4108940546 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f596406d730 msgr2=0x7f596406ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:46.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:46.392+0000 7f5969d59640 1 --2- 172.21.15.27:0/4108940546 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f596406d730 0x7f596406ee80 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7f59641622d0 tx=0x7f595c02a680 comp rx=0 tx=0).stop 2024-09-12T03:23:46.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:46.392+0000 7f5969d59640 1 -- 172.21.15.27:0/4108940546 shutdown_connections 2024-09-12T03:23:46.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:46.392+0000 7f5969d59640 1 --2- 172.21.15.27:0/4108940546 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f596406d730 0x7f596406ee80 unknown :-1 s=CLOSED pgs=1630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:46.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:46.392+0000 7f5969d59640 1 --2- 172.21.15.27:0/4108940546 >> v2:172.21.15.45:3300/0 conn(0x7f59640b6540 0x7f59640af370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:46.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:46.392+0000 7f5969d59640 1 -- 172.21.15.27:0/4108940546 >> 172.21.15.27:0/4108940546 conn(0x7f596406cc40 msgr2=0x7f596406d020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:46.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:46.392+0000 7f5969d59640 1 -- 172.21.15.27:0/4108940546 shutdown_connections 2024-09-12T03:23:46.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:46.392+0000 7f5969d59640 1 -- 172.21.15.27:0/4108940546 wait complete. 2024-09-12T03:23:46.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:46.393+0000 7f5969d59640 1 Processor -- start 2024-09-12T03:23:46.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:46.393+0000 7f5969d59640 1 -- start start 2024-09-12T03:23:46.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:46.394+0000 7f5969d59640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f596406d730 0x7f5964149040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:46.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:46.394+0000 7f5969d59640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f59640b6540 0x7f59641567f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:46.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:46.394+0000 7f5969d59640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f596415e900 0x7f59641497e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:46.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:46.394+0000 7f5969d59640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f59640b01f0 con 0x7f59640b6540 2024-09-12T03:23:46.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:46.394+0000 7f5969d59640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f59640aff10 con 0x7f596406d730 2024-09-12T03:23:46.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:46.394+0000 7f5969d59640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f596406e880 con 0x7f596415e900 2024-09-12T03:23:46.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:46.394+0000 7f5962ffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f59640b6540 0x7f59641567f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:46.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:46.394+0000 7f5962ffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f59640b6540 0x7f59641567f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59856/0 (socket says 172.21.15.27:59856) 2024-09-12T03:23:46.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:46.394+0000 7f5962ffd640 1 -- 172.21.15.27:0/3927548321 learned_addr learned my addr 172.21.15.27:0/3927548321 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:23:46.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:46.394+0000 7f59637fe640 1 --2- 172.21.15.27:0/3927548321 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f596406d730 0x7f5964149040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:46.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:46.394+0000 7f5963fff640 1 --2- 172.21.15.27:0/3927548321 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f596415e900 0x7f59641497e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:46.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:46.394+0000 7f5962ffd640 1 -- 172.21.15.27:0/3927548321 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f596415e900 msgr2=0x7f59641497e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:46.397 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:46.394+0000 7f5962ffd640 1 --2- 172.21.15.27:0/3927548321 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f596415e900 0x7f59641497e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:46.397 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:46.395+0000 7f5962ffd640 1 -- 172.21.15.27:0/3927548321 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f596406d730 msgr2=0x7f5964149040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:46.397 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:46.395+0000 7f5962ffd640 1 --2- 172.21.15.27:0/3927548321 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f596406d730 0x7f5964149040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:46.397 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:46.395+0000 7f5962ffd640 1 -- 172.21.15.27:0/3927548321 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f596414a030 con 0x7f59640b6540 2024-09-12T03:23:46.397 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:46.395+0000 7f5962ffd640 1 --2- 172.21.15.27:0/3927548321 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f59640b6540 0x7f59641567f0 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7f595800b4d0 tx=0x7f595800b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:46.397 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:46.395+0000 7f594bfff640 1 -- 172.21.15.27:0/3927548321 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5958014070 con 0x7f59640b6540 2024-09-12T03:23:46.397 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:46.395+0000 7f594bfff640 1 -- 172.21.15.27:0/3927548321 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5958010870 con 0x7f59640b6540 2024-09-12T03:23:46.397 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:46.395+0000 7f594bfff640 1 -- 172.21.15.27:0/3927548321 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5958010c60 con 0x7f59640b6540 2024-09-12T03:23:46.397 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:46.395+0000 7f5969d59640 1 -- 172.21.15.27:0/3927548321 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f596414a310 con 0x7f59640b6540 2024-09-12T03:23:46.397 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:46.395+0000 7f5969d59640 1 -- 172.21.15.27:0/3927548321 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f596414a8d0 con 0x7f59640b6540 2024-09-12T03:23:46.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:46.396+0000 7f5969d59640 1 -- 172.21.15.27:0/3927548321 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59640af370 con 0x7f59640b6540 2024-09-12T03:23:46.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:46.397+0000 7f594bfff640 1 -- 172.21.15.27:0/3927548321 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 25) ==== 91365+0+0 (secure 0 0 0) 0x7f5958017070 con 0x7f59640b6540 2024-09-12T03:23:46.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:46.397+0000 7f594bfff640 1 --2- 172.21.15.27:0/3927548321 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f595006c7c0 0x7f595006ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:46.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:46.397+0000 7f594bfff640 1 -- 172.21.15.27:0/3927548321 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(150..150 src has 1..150) ==== 5851+0+0 (secure 0 0 0) 0x7f5958089370 con 0x7f59640b6540 2024-09-12T03:23:46.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:46.397+0000 7f59637fe640 1 --2- 172.21.15.27:0/3927548321 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f595006c7c0 0x7f595006ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:46.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:46.398+0000 7f59637fe640 1 --2- 172.21.15.27:0/3927548321 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f595006c7c0 0x7f595006ec80 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f5954007fd0 tx=0x7f5954007de0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:46.401 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:46.402+0000 7f594bfff640 1 -- 172.21.15.27:0/3927548321 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f59580592d0 con 0x7f59640b6540 2024-09-12T03:23:46.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:46.575+0000 7f5969d59640 1 -- 172.21.15.27:0/3927548321 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f5964149d20 con 0x7f59640b6540 2024-09-12T03:23:47.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:47.223+0000 7f594bfff640 1 -- 172.21.15.27:0/3927548321 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v151) ==== 181+0+0 (secure 0 0 0) 0x7f595805d5d0 con 0x7f59640b6540 2024-09-12T03:23:47.224 INFO:teuthology.orchestra.run.smithi027.stderr:pool 'cephfs_metadata' removed 2024-09-12T03:23:47.227 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:47.227+0000 7f5969d59640 1 -- 172.21.15.27:0/3927548321 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f595006c7c0 msgr2=0x7f595006ec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:47.227 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:47.227+0000 7f5969d59640 1 --2- 172.21.15.27:0/3927548321 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f595006c7c0 0x7f595006ec80 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f5954007fd0 tx=0x7f5954007de0 comp rx=0 tx=0).stop 2024-09-12T03:23:47.227 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:47.227+0000 7f5969d59640 1 -- 172.21.15.27:0/3927548321 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f59640b6540 msgr2=0x7f59641567f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:47.227 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:47.227+0000 7f5969d59640 1 --2- 172.21.15.27:0/3927548321 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f59640b6540 0x7f59641567f0 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7f595800b4d0 tx=0x7f595800b990 comp rx=0 tx=0).stop 2024-09-12T03:23:47.227 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:47.227+0000 7f5969d59640 1 -- 172.21.15.27:0/3927548321 shutdown_connections 2024-09-12T03:23:47.227 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:47.227+0000 7f5969d59640 1 --2- 172.21.15.27:0/3927548321 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f596415e900 0x7f59641497e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:47.227 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:47.227+0000 7f5969d59640 1 --2- 172.21.15.27:0/3927548321 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f595006c7c0 0x7f595006ec80 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:47.227 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:47.227+0000 7f5969d59640 1 --2- 172.21.15.27:0/3927548321 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f59640b6540 0x7f59641567f0 secure :-1 s=CLOSED pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7f595800b4d0 tx=0x7f595800b990 comp rx=0 tx=0).stop 2024-09-12T03:23:47.227 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:47.227+0000 7f5969d59640 1 --2- 172.21.15.27:0/3927548321 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f596406d730 0x7f5964149040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:47.227 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:47.227+0000 7f5969d59640 1 -- 172.21.15.27:0/3927548321 >> 172.21.15.27:0/3927548321 conn(0x7f596406cc40 msgr2=0x7f59640aebb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:47.228 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:47.228+0000 7f5969d59640 1 -- 172.21.15.27:0/3927548321 shutdown_connections 2024-09-12T03:23:47.228 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:47.228+0000 7f5969d59640 1 -- 172.21.15.27:0/3927548321 wait complete. 2024-09-12T03:23:47.242 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-12T03:23:47.352 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:47.352+0000 7f1fa5b86640 1 Processor -- start 2024-09-12T03:23:47.353 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:47.352+0000 7f1fa5b86640 1 -- start start 2024-09-12T03:23:47.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:47.353+0000 7f1fa5b86640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1fa0070d30 0x7f1fa006cc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:47.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:47.353+0000 7f1fa5b86640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f1fa00d0b50 0x7f1fa0168300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:47.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:47.353+0000 7f1fa5b86640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1fa00cf9f0 con 0x7f1fa006e960 2024-09-12T03:23:47.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:47.353+0000 7f1fa5b86640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f1fa00d1a40 con 0x7f1fa00d0b50 2024-09-12T03:23:47.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:47.353+0000 7f1fa5b86640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f1fa00cfc90 con 0x7f1fa0070d30 2024-09-12T03:23:47.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:47.354+0000 7f1f9ffff640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f1fa00d0b50 0x7f1fa0168300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:47.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:47.354+0000 7f1f9f7fe640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1fa0070d30 0x7f1fa006cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:47.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:47.354+0000 7f1f9effd640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f1fa006e960 0x7f1fa00d0440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.27:49288/0 (socket says 172.21.15.27:49288) 2024-09-12T03:23:47.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:47.354+0000 7f1f9effd640 1 -- 172.21.15.27:0/815293849 learned_addr learned my addr 172.21.15.27:0/815293849 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:23:47.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:47.354+0000 7f1f9e7fc640 1 -- 172.21.15.27:0/815293849 <== mon.1 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2016132388 0 0) 0x7f1fa00cf9f0 con 0x7f1fa006e960 2024-09-12T03:23:47.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:47.354+0000 7f1f9e7fc640 1 -- 172.21.15.27:0/815293849 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1f88003520 con 0x7f1fa006e960 2024-09-12T03:23:47.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:47.354+0000 7f1f9f7fe640 1 -- 172.21.15.27:0/815293849 >> v2:172.21.15.45:3300/0 conn(0x7f1fa00d0b50 msgr2=0x7f1fa0168300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:47.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:47.354+0000 7f1f9f7fe640 1 --2- 172.21.15.27:0/815293849 >> v2:172.21.15.45:3300/0 conn(0x7f1fa00d0b50 0x7f1fa0168300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:47.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:47.354+0000 7f1f9f7fe640 1 -- 172.21.15.27:0/815293849 >> v1:172.21.15.45:6789/0 conn(0x7f1fa006e960 legacy=0x7f1fa00d0440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:47.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:47.354+0000 7f1f9f7fe640 1 -- 172.21.15.27:0/815293849 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fa00cff70 con 0x7f1fa0070d30 2024-09-12T03:23:47.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:47.354+0000 7f1f9ffff640 1 --2- 172.21.15.27:0/815293849 >> v2:172.21.15.45:3300/0 conn(0x7f1fa00d0b50 0x7f1fa0168300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:23:47.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:47.355+0000 7f1f9f7fe640 1 --2- 172.21.15.27:0/815293849 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1fa0070d30 0x7f1fa006cc40 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f1fa00d1bc0 tx=0x7f1f9002a060 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9885419e471b7b4f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:47.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:47.355+0000 7f1f9e7fc640 1 -- 172.21.15.27:0/815293849 <== mon.2 v2:172.21.15.45:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f90037070 con 0x7f1fa0070d30 2024-09-12T03:23:47.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:47.355+0000 7f1f9e7fc640 1 -- 172.21.15.27:0/815293849 <== mon.2 v2:172.21.15.45:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f9002adb0 con 0x7f1fa0070d30 2024-09-12T03:23:47.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:47.355+0000 7f1f9e7fc640 1 -- 172.21.15.27:0/815293849 <== mon.2 v2:172.21.15.45:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f90032640 con 0x7f1fa0070d30 2024-09-12T03:23:47.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:47.356+0000 7f1fa5b86640 1 -- 172.21.15.27:0/815293849 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1fa0070d30 msgr2=0x7f1fa006cc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:47.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:47.356+0000 7f1fa5b86640 1 --2- 172.21.15.27:0/815293849 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1fa0070d30 0x7f1fa006cc40 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f1fa00d1bc0 tx=0x7f1f9002a060 comp rx=0 tx=0).stop 2024-09-12T03:23:47.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:47.356+0000 7f1fa5b86640 1 -- 172.21.15.27:0/815293849 shutdown_connections 2024-09-12T03:23:47.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:47.356+0000 7f1fa5b86640 1 --2- 172.21.15.27:0/815293849 >> v2:172.21.15.45:3300/0 conn(0x7f1fa00d0b50 0x7f1fa0168300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:47.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:47.356+0000 7f1fa5b86640 1 --2- 172.21.15.27:0/815293849 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1fa0070d30 0x7f1fa006cc40 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:47.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:47.356+0000 7f1fa5b86640 1 -- 172.21.15.27:0/815293849 >> 172.21.15.27:0/815293849 conn(0x7f1fa00b7c10 msgr2=0x7f1fa009ba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:47.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:47.356+0000 7f1fa5b86640 1 -- 172.21.15.27:0/815293849 shutdown_connections 2024-09-12T03:23:47.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:47.356+0000 7f1fa5b86640 1 -- 172.21.15.27:0/815293849 wait complete. 2024-09-12T03:23:47.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:47.357+0000 7f1fa5b86640 1 Processor -- start 2024-09-12T03:23:47.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:47.357+0000 7f1fa5b86640 1 -- start start 2024-09-12T03:23:47.360 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:47.358+0000 7f1fa5b86640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1fa006e960 0x7f1fa0154a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:47.360 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:47.358+0000 7f1fa5b86640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1fa0070d30 0x7f1fa01f8820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:47.360 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:47.358+0000 7f1fa5b86640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1fa00d0b50 0x7f1fa015d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:47.360 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:47.358+0000 7f1fa5b86640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f1fa00d31c0 con 0x7f1fa0070d30 2024-09-12T03:23:47.360 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:47.358+0000 7f1fa5b86640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f1fa006d180 con 0x7f1fa006e960 2024-09-12T03:23:47.360 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:47.358+0000 7f1fa5b86640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f1fa00d3340 con 0x7f1fa00d0b50 2024-09-12T03:23:47.360 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:47.358+0000 7f1f9effd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1fa0070d30 0x7f1fa01f8820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:47.360 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:47.358+0000 7f1f9effd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1fa0070d30 0x7f1fa01f8820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59868/0 (socket says 172.21.15.27:59868) 2024-09-12T03:23:47.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:47.358+0000 7f1f9effd640 1 -- 172.21.15.27:0/1892993066 learned_addr learned my addr 172.21.15.27:0/1892993066 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:23:47.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:47.358+0000 7f1f9ffff640 1 --2- 172.21.15.27:0/1892993066 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1fa00d0b50 0x7f1fa015d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:47.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:47.358+0000 7f1f9effd640 1 -- 172.21.15.27:0/1892993066 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1fa00d0b50 msgr2=0x7f1fa015d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:47.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:47.358+0000 7f1f9f7fe640 1 --2- 172.21.15.27:0/1892993066 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1fa006e960 0x7f1fa0154a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:47.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:47.358+0000 7f1f9effd640 1 --2- 172.21.15.27:0/1892993066 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1fa00d0b50 0x7f1fa015d380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:47.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:47.358+0000 7f1f9effd640 1 -- 172.21.15.27:0/1892993066 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1fa006e960 msgr2=0x7f1fa0154a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:47.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:47.358+0000 7f1f9effd640 1 --2- 172.21.15.27:0/1892993066 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1fa006e960 0x7f1fa0154a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:47.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:47.358+0000 7f1f9effd640 1 -- 172.21.15.27:0/1892993066 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fa01f8d60 con 0x7f1fa0070d30 2024-09-12T03:23:47.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:47.358+0000 7f1f9effd640 1 --2- 172.21.15.27:0/1892993066 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1fa0070d30 0x7f1fa01f8820 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7f1f9400c500 tx=0x7f1f9400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:47.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:47.359+0000 7f1f87fff640 1 -- 172.21.15.27:0/1892993066 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f94017070 con 0x7f1fa0070d30 2024-09-12T03:23:47.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:47.359+0000 7f1f87fff640 1 -- 172.21.15.27:0/1892993066 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f94005ce0 con 0x7f1fa0070d30 2024-09-12T03:23:47.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:47.359+0000 7f1fa5b86640 1 -- 172.21.15.27:0/1892993066 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fa00b49a0 con 0x7f1fa0070d30 2024-09-12T03:23:47.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:47.359+0000 7f1f87fff640 1 -- 172.21.15.27:0/1892993066 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f94007330 con 0x7f1fa0070d30 2024-09-12T03:23:47.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:47.359+0000 7f1fa5b86640 1 -- 172.21.15.27:0/1892993066 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fa00b4f60 con 0x7f1fa0070d30 2024-09-12T03:23:47.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:47.360+0000 7f1fa5b86640 1 -- 172.21.15.27:0/1892993066 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1fa006cc40 con 0x7f1fa0070d30 2024-09-12T03:23:47.363 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:47.360+0000 7f1f87fff640 1 -- 172.21.15.27:0/1892993066 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 25) ==== 91365+0+0 (secure 0 0 0) 0x7f1f9401a070 con 0x7f1fa0070d30 2024-09-12T03:23:47.363 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:47.361+0000 7f1f87fff640 1 --2- 172.21.15.27:0/1892993066 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f1f8c06c7f0 0x7f1f8c06ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:47.363 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:47.361+0000 7f1f87fff640 1 -- 172.21.15.27:0/1892993066 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(151..151 src has 1..151) ==== 5413+0+0 (secure 0 0 0) 0x7f1f94088a90 con 0x7f1fa0070d30 2024-09-12T03:23:47.363 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:47.361+0000 7f1f9f7fe640 1 --2- 172.21.15.27:0/1892993066 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f1f8c06c7f0 0x7f1f8c06ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:47.363 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:47.361+0000 7f1f9f7fe640 1 --2- 172.21.15.27:0/1892993066 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f1f8c06c7f0 0x7f1f8c06ecb0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f1f90034000 tx=0x7f1f9002e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:47.365 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:47.365+0000 7f1f87fff640 1 -- 172.21.15.27:0/1892993066 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f1fa01f8d60 con 0x7f1fa0070d30 2024-09-12T03:23:47.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:47.542+0000 7f1fa5b86640 1 -- 172.21.15.27:0/1892993066 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f1fa00d1230 con 0x7f1fa0070d30 2024-09-12T03:23:48.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:48.228+0000 7f1f87fff640 1 -- 172.21.15.27:0/1892993066 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v152) ==== 169+0+0 (secure 0 0 0) 0x7f1f94059080 con 0x7f1fa0070d30 2024-09-12T03:23:48.229 INFO:teuthology.orchestra.run.smithi027.stderr:pool 'cephfs_data' removed 2024-09-12T03:23:48.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:48.233+0000 7f1fa5b86640 1 -- 172.21.15.27:0/1892993066 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f1f8c06c7f0 msgr2=0x7f1f8c06ecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:48.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:48.233+0000 7f1fa5b86640 1 --2- 172.21.15.27:0/1892993066 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f1f8c06c7f0 0x7f1f8c06ecb0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f1f90034000 tx=0x7f1f9002e000 comp rx=0 tx=0).stop 2024-09-12T03:23:48.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:48.233+0000 7f1fa5b86640 1 -- 172.21.15.27:0/1892993066 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1fa0070d30 msgr2=0x7f1fa01f8820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:48.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:48.233+0000 7f1fa5b86640 1 --2- 172.21.15.27:0/1892993066 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1fa0070d30 0x7f1fa01f8820 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7f1f9400c500 tx=0x7f1f9400c9c0 comp rx=0 tx=0).stop 2024-09-12T03:23:48.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:48.233+0000 7f1fa5b86640 1 -- 172.21.15.27:0/1892993066 shutdown_connections 2024-09-12T03:23:48.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:48.233+0000 7f1fa5b86640 1 --2- 172.21.15.27:0/1892993066 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1fa00d0b50 0x7f1fa015d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:48.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:48.233+0000 7f1fa5b86640 1 --2- 172.21.15.27:0/1892993066 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f1f8c06c7f0 0x7f1f8c06ecb0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:48.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:48.233+0000 7f1fa5b86640 1 --2- 172.21.15.27:0/1892993066 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1fa0070d30 0x7f1fa01f8820 unknown :-1 s=CLOSED pgs=1632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:48.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:48.233+0000 7f1fa5b86640 1 --2- 172.21.15.27:0/1892993066 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1fa006e960 0x7f1fa0154a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:48.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:48.233+0000 7f1fa5b86640 1 -- 172.21.15.27:0/1892993066 >> 172.21.15.27:0/1892993066 conn(0x7f1fa00b7c10 msgr2=0x7f1fa006f390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:48.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:48.233+0000 7f1fa5b86640 1 -- 172.21.15.27:0/1892993066 shutdown_connections 2024-09-12T03:23:48.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:48.233+0000 7f1fa5b86640 1 -- 172.21.15.27:0/1892993066 wait complete. 2024-09-12T03:23:48.247 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-09-12T03:23:48.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:48.354+0000 7fd72e131640 1 Processor -- start 2024-09-12T03:23:48.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:48.354+0000 7fd72e131640 1 -- start start 2024-09-12T03:23:48.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:48.355+0000 7fd72e131640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fd7280b3d10 0x7fd7280b66c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:48.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:48.355+0000 7fd72e131640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd72815ed70 con 0x7fd7280b6230 2024-09-12T03:23:48.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:48.355+0000 7fd72e131640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd7280b7d10 con 0x7fd72815e990 2024-09-12T03:23:48.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:48.355+0000 7fd72e131640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7fd7280d2a30 con 0x7fd7280b3d10 2024-09-12T03:23:48.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:48.355+0000 7fd727fff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fd7280b3d10 0x7fd7280b66c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:48.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:48.355+0000 7fd726ffd640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fd7280b6230 0x7fd7280b3600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:52370/0 (socket says 172.21.15.27:52370) 2024-09-12T03:23:48.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:48.355+0000 7fd727fff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fd7280b3d10 0x7fd7280b66c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59882/0 (socket says 172.21.15.27:59882) 2024-09-12T03:23:48.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:48.355+0000 7fd726ffd640 1 -- 172.21.15.27:0/1795890213 learned_addr learned my addr 172.21.15.27:0/1795890213 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:23:48.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:48.356+0000 7fd727fff640 1 -- 172.21.15.27:0/1795890213 >> v1:172.21.15.45:6789/0 conn(0x7fd72815e990 legacy=0x7fd7280b7600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:48.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:48.356+0000 7fd727fff640 1 -- 172.21.15.27:0/1795890213 >> v1:172.21.15.27:6789/0 conn(0x7fd7280b6230 legacy=0x7fd7280b3600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:48.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:48.356+0000 7fd727fff640 1 -- 172.21.15.27:0/1795890213 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7280d2d10 con 0x7fd7280b3d10 2024-09-12T03:23:48.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:48.356+0000 7fd7267fc640 1 -- 172.21.15.27:0/1795890213 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1911575049 0 0) 0x7fd72815ed70 con 0x7fd7280b6230 2024-09-12T03:23:48.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:48.356+0000 7fd727fff640 1 --2- 172.21.15.27:0/1795890213 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd7280b3d10 0x7fd7280b66c0 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7fd720009d90 tx=0x7fd72002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cda344497d26bd2b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:48.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:48.357+0000 7fd7267fc640 1 -- 172.21.15.27:0/1795890213 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd720007740 con 0x7fd7280b3d10 2024-09-12T03:23:48.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:48.357+0000 7fd7267fc640 1 -- 172.21.15.27:0/1795890213 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7200078e0 con 0x7fd7280b3d10 2024-09-12T03:23:48.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:48.357+0000 7fd7267fc640 1 -- 172.21.15.27:0/1795890213 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd720007cb0 con 0x7fd7280b3d10 2024-09-12T03:23:48.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:48.357+0000 7fd72e131640 1 -- 172.21.15.27:0/1795890213 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd7280b3d10 msgr2=0x7fd7280b66c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:48.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:48.357+0000 7fd72e131640 1 --2- 172.21.15.27:0/1795890213 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd7280b3d10 0x7fd7280b66c0 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7fd720009d90 tx=0x7fd72002a6a0 comp rx=0 tx=0).stop 2024-09-12T03:23:48.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:48.357+0000 7fd72e131640 1 -- 172.21.15.27:0/1795890213 shutdown_connections 2024-09-12T03:23:48.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:48.357+0000 7fd72e131640 1 --2- 172.21.15.27:0/1795890213 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd7280b3d10 0x7fd7280b66c0 unknown :-1 s=CLOSED pgs=1634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:48.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:48.357+0000 7fd72e131640 1 -- 172.21.15.27:0/1795890213 >> 172.21.15.27:0/1795890213 conn(0x7fd7280d1230 msgr2=0x7fd7280d1640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:48.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:48.357+0000 7fd72e131640 1 -- 172.21.15.27:0/1795890213 shutdown_connections 2024-09-12T03:23:48.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:48.357+0000 7fd72e131640 1 -- 172.21.15.27:0/1795890213 wait complete. 2024-09-12T03:23:48.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:48.358+0000 7fd72e131640 1 Processor -- start 2024-09-12T03:23:48.359 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:48.359+0000 7fd72e131640 1 -- start start 2024-09-12T03:23:48.360 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:48.359+0000 7fd72e131640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fd7280b3d10 0x7fd7280ce160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:48.360 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:48.359+0000 7fd72e131640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd7280b6230 0x7fd7281f8040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:48.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:48.359+0000 7fd72e131640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd72815e990 0x7fd72815aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:48.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:48.359+0000 7fd72e131640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fd7280d35b0 con 0x7fd72815e990 2024-09-12T03:23:48.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:48.359+0000 7fd72e131640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fd7280d32d0 con 0x7fd7280b6230 2024-09-12T03:23:48.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:48.359+0000 7fd72e131640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fd7280d3ad0 con 0x7fd7280b3d10 2024-09-12T03:23:48.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:48.359+0000 7fd727fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd72815e990 0x7fd72815aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:48.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:48.359+0000 7fd727fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd72815e990 0x7fd72815aa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59894/0 (socket says 172.21.15.27:59894) 2024-09-12T03:23:48.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:48.359+0000 7fd727fff640 1 -- 172.21.15.27:0/3766591589 learned_addr learned my addr 172.21.15.27:0/3766591589 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:23:48.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:48.360+0000 7fd726ffd640 1 --2- 172.21.15.27:0/3766591589 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd7280b6230 0x7fd7281f8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:48.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:48.360+0000 7fd7277fe640 1 --2- 172.21.15.27:0/3766591589 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fd7280b3d10 0x7fd7280ce160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:48.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:48.360+0000 7fd727fff640 1 -- 172.21.15.27:0/3766591589 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fd7280b3d10 msgr2=0x7fd7280ce160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:48.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:48.360+0000 7fd727fff640 1 --2- 172.21.15.27:0/3766591589 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fd7280b3d10 0x7fd7280ce160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:48.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:48.360+0000 7fd727fff640 1 -- 172.21.15.27:0/3766591589 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd7280b6230 msgr2=0x7fd7281f8040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:48.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:48.360+0000 7fd727fff640 1 --2- 172.21.15.27:0/3766591589 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd7280b6230 0x7fd7281f8040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:48.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:48.360+0000 7fd727fff640 1 -- 172.21.15.27:0/3766591589 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd72815ee10 con 0x7fd72815e990 2024-09-12T03:23:48.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:48.360+0000 7fd727fff640 1 --2- 172.21.15.27:0/3766591589 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd72815e990 0x7fd72815aa60 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7fd720009d90 tx=0x7fd720004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:48.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:48.360+0000 7fd70ffff640 1 -- 172.21.15.27:0/3766591589 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd720040070 con 0x7fd72815e990 2024-09-12T03:23:48.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:48.360+0000 7fd70ffff640 1 -- 172.21.15.27:0/3766591589 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd720031bc0 con 0x7fd72815e990 2024-09-12T03:23:48.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:48.360+0000 7fd72e131640 1 -- 172.21.15.27:0/3766591589 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd72815f0f0 con 0x7fd72815e990 2024-09-12T03:23:48.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:48.360+0000 7fd70ffff640 1 -- 172.21.15.27:0/3766591589 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd72003c420 con 0x7fd72815e990 2024-09-12T03:23:48.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:48.360+0000 7fd72e131640 1 -- 172.21.15.27:0/3766591589 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd72815f6b0 con 0x7fd72815e990 2024-09-12T03:23:48.363 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:48.361+0000 7fd72e131640 1 -- 172.21.15.27:0/3766591589 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7280b66c0 con 0x7fd72815e990 2024-09-12T03:23:48.363 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:48.362+0000 7fd70ffff640 1 -- 172.21.15.27:0/3766591589 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 25) ==== 91365+0+0 (secure 0 0 0) 0x7fd720039040 con 0x7fd72815e990 2024-09-12T03:23:48.363 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:48.362+0000 7fd70ffff640 1 --2- 172.21.15.27:0/3766591589 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fd71406c7c0 0x7fd71406ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:48.363 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:48.362+0000 7fd70ffff640 1 -- 172.21.15.27:0/3766591589 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 4999+0+0 (secure 0 0 0) 0x7fd7200aaa60 con 0x7fd72815e990 2024-09-12T03:23:48.363 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:48.362+0000 7fd7277fe640 1 --2- 172.21.15.27:0/3766591589 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fd71406c7c0 0x7fd71406ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:48.363 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:48.363+0000 7fd7277fe640 1 --2- 172.21.15.27:0/3766591589 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fd71406c7c0 0x7fd71406ec80 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fd718006fd0 tx=0x7fd718009060 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:48.366 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:48.367+0000 7fd70ffff640 1 -- 172.21.15.27:0/3766591589 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fd72007aa80 con 0x7fd72815e990 2024-09-12T03:23:48.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:48.541+0000 7fd72e131640 1 -- 172.21.15.27:0/3766591589 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7fd7280b6b20 con 0x7fd72815e990 2024-09-12T03:23:48.544 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:48.544+0000 7fd70ffff640 1 -- 172.21.15.27:0/3766591589 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v29) ==== 104+0+0 (secure 0 0 0) 0x7fd72007ed80 con 0x7fd72815e990 2024-09-12T03:23:48.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:48.546+0000 7fd72e131640 1 -- 172.21.15.27:0/3766591589 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fd71406c7c0 msgr2=0x7fd71406ec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:48.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:48.546+0000 7fd72e131640 1 --2- 172.21.15.27:0/3766591589 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fd71406c7c0 0x7fd71406ec80 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fd718006fd0 tx=0x7fd718009060 comp rx=0 tx=0).stop 2024-09-12T03:23:48.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:48.546+0000 7fd72e131640 1 -- 172.21.15.27:0/3766591589 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd72815e990 msgr2=0x7fd72815aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:48.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:48.546+0000 7fd72e131640 1 --2- 172.21.15.27:0/3766591589 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd72815e990 0x7fd72815aa60 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7fd720009d90 tx=0x7fd720004a40 comp rx=0 tx=0).stop 2024-09-12T03:23:48.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:48.546+0000 7fd72e131640 1 -- 172.21.15.27:0/3766591589 shutdown_connections 2024-09-12T03:23:48.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:48.546+0000 7fd72e131640 1 --2- 172.21.15.27:0/3766591589 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fd71406c7c0 0x7fd71406ec80 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:48.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:48.546+0000 7fd72e131640 1 --2- 172.21.15.27:0/3766591589 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd72815e990 0x7fd72815aa60 unknown :-1 s=CLOSED pgs=1635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:48.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:48.546+0000 7fd72e131640 1 --2- 172.21.15.27:0/3766591589 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd7280b6230 0x7fd7281f8040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:48.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:48.546+0000 7fd72e131640 1 --2- 172.21.15.27:0/3766591589 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fd7280b3d10 0x7fd7280ce160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:48.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:48.546+0000 7fd72e131640 1 -- 172.21.15.27:0/3766591589 >> 172.21.15.27:0/3766591589 conn(0x7fd7280d1230 msgr2=0x7fd72815d7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:48.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:48.546+0000 7fd72e131640 1 -- 172.21.15.27:0/3766591589 shutdown_connections 2024-09-12T03:23:48.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:48.546+0000 7fd72e131640 1 -- 172.21.15.27:0/3766591589 wait complete. 2024-09-12T03:23:48.560 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_upgrade_v1_to_v2' 2024-09-12T03:23:48.663 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:48.663+0000 7fe5cb76d640 1 Processor -- start 2024-09-12T03:23:48.663 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:48.663+0000 7fe5cb76d640 1 -- start start 2024-09-12T03:23:48.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:48.664+0000 7fe5cb76d640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe5c415b2b0 0x7fe5c40b3560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:48.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:48.664+0000 7fe5cb76d640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fe5c406e9b0 0x7fe5c406ee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:48.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:48.664+0000 7fe5cb76d640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe5c40ce410 con 0x7fe5c4072d70 2024-09-12T03:23:48.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:48.664+0000 7fe5cb76d640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7fe5c40720a0 con 0x7fe5c406e9b0 2024-09-12T03:23:48.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:48.664+0000 7fe5cb76d640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fe5c40732c0 con 0x7fe5c415b2b0 2024-09-12T03:23:48.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:48.664+0000 7fe5c9d0a640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fe5c406e9b0 0x7fe5c406ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:48.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:48.664+0000 7fe5c8d08640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fe5c4072d70 0x7fe5c4072430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:52374/0 (socket says 172.21.15.27:52374) 2024-09-12T03:23:48.667 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:48.664+0000 7fe5c8d08640 1 -- 172.21.15.27:0/2473889378 learned_addr learned my addr 172.21.15.27:0/2473889378 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:23:48.667 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:48.664+0000 7fe5c9509640 1 --2- 172.21.15.27:0/2473889378 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe5c415b2b0 0x7fe5c40b3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:48.667 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:48.664+0000 7fe5c9d0a640 1 -- 172.21.15.27:0/2473889378 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe5c415b2b0 msgr2=0x7fe5c40b3560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:48.667 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:48.664+0000 7fe5c9d0a640 1 --2- 172.21.15.27:0/2473889378 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe5c415b2b0 0x7fe5c40b3560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:48.667 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:48.664+0000 7fe5c9d0a640 1 -- 172.21.15.27:0/2473889378 >> v1:172.21.15.27:6789/0 conn(0x7fe5c4072d70 legacy=0x7fe5c4072430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:48.667 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:48.664+0000 7fe5c9d0a640 1 -- 172.21.15.27:0/2473889378 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5c4071ae0 con 0x7fe5c406e9b0 2024-09-12T03:23:48.667 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:48.665+0000 7fe5c9d0a640 1 --2- 172.21.15.27:0/2473889378 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe5c406e9b0 0x7fe5c406ee60 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7fe5bc009d90 tx=0x7fe5bc02a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57a1ebdfb0832f3d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:48.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:48.665+0000 7fe5bbfff640 1 -- 172.21.15.27:0/2473889378 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5bc007740 con 0x7fe5c406e9b0 2024-09-12T03:23:48.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:48.665+0000 7fe5bbfff640 1 -- 172.21.15.27:0/2473889378 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5bc0078e0 con 0x7fe5c406e9b0 2024-09-12T03:23:48.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:48.665+0000 7fe5bbfff640 1 -- 172.21.15.27:0/2473889378 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5bc007cb0 con 0x7fe5c406e9b0 2024-09-12T03:23:48.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:48.666+0000 7fe5cb76d640 1 -- 172.21.15.27:0/2473889378 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe5c406e9b0 msgr2=0x7fe5c406ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:48.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:48.666+0000 7fe5cb76d640 1 --2- 172.21.15.27:0/2473889378 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe5c406e9b0 0x7fe5c406ee60 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7fe5bc009d90 tx=0x7fe5bc02a660 comp rx=0 tx=0).stop 2024-09-12T03:23:48.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:48.666+0000 7fe5cb76d640 1 -- 172.21.15.27:0/2473889378 shutdown_connections 2024-09-12T03:23:48.669 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:48.666+0000 7fe5cb76d640 1 --2- 172.21.15.27:0/2473889378 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe5c406e9b0 0x7fe5c406ee60 unknown :-1 s=CLOSED pgs=1637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:48.669 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:48.666+0000 7fe5cb76d640 1 --2- 172.21.15.27:0/2473889378 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe5c415b2b0 0x7fe5c40b3560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:48.669 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:48.666+0000 7fe5cb76d640 1 -- 172.21.15.27:0/2473889378 >> 172.21.15.27:0/2473889378 conn(0x7fe5c4062c60 msgr2=0x7fe5c406bb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:48.669 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:48.666+0000 7fe5cb76d640 1 -- 172.21.15.27:0/2473889378 shutdown_connections 2024-09-12T03:23:48.669 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:48.666+0000 7fe5cb76d640 1 -- 172.21.15.27:0/2473889378 wait complete. 2024-09-12T03:23:48.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:48.667+0000 7fe5cb76d640 1 Processor -- start 2024-09-12T03:23:48.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:48.667+0000 7fe5cb76d640 1 -- start start 2024-09-12T03:23:48.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:48.668+0000 7fe5cb76d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe5c406e9b0 0x7fe5c415c400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:48.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:48.668+0000 7fe5cb76d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe5c4072d70 0x7fe5c415db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:48.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:48.668+0000 7fe5cb76d640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe5c415b2b0 0x7fe5c4154260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:48.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:48.668+0000 7fe5cb76d640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fe5c415b690 con 0x7fe5c406e9b0 2024-09-12T03:23:48.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:48.668+0000 7fe5cb76d640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe5c40ce690 con 0x7fe5c4072d70 2024-09-12T03:23:48.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:48.668+0000 7fe5cb76d640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fe5c415b810 con 0x7fe5c415b2b0 2024-09-12T03:23:48.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:48.668+0000 7fe5c9509640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe5c406e9b0 0x7fe5c415c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:48.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:48.668+0000 7fe5c9509640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe5c406e9b0 0x7fe5c415c400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59914/0 (socket says 172.21.15.27:59914) 2024-09-12T03:23:48.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:48.668+0000 7fe5c9509640 1 -- 172.21.15.27:0/2408678483 learned_addr learned my addr 172.21.15.27:0/2408678483 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:23:48.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:48.668+0000 7fe5c9d0a640 1 --2- 172.21.15.27:0/2408678483 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe5c415b2b0 0x7fe5c4154260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:48.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:48.668+0000 7fe5c8d08640 1 --2- 172.21.15.27:0/2408678483 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe5c4072d70 0x7fe5c415db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:48.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:48.668+0000 7fe5c9509640 1 -- 172.21.15.27:0/2408678483 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe5c415b2b0 msgr2=0x7fe5c4154260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:48.674 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:48.668+0000 7fe5c9509640 1 --2- 172.21.15.27:0/2408678483 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe5c415b2b0 0x7fe5c4154260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:48.674 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:48.668+0000 7fe5c9509640 1 -- 172.21.15.27:0/2408678483 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe5c4072d70 msgr2=0x7fe5c415db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:48.674 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:48.668+0000 7fe5c9509640 1 --2- 172.21.15.27:0/2408678483 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe5c4072d70 0x7fe5c415db30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:48.674 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:48.668+0000 7fe5c9509640 1 -- 172.21.15.27:0/2408678483 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5c415ca90 con 0x7fe5c406e9b0 2024-09-12T03:23:48.674 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:48.668+0000 7fe5c8d08640 1 --2- 172.21.15.27:0/2408678483 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe5c4072d70 0x7fe5c415db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:23:48.674 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:48.668+0000 7fe5c9509640 1 --2- 172.21.15.27:0/2408678483 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe5c406e9b0 0x7fe5c415c400 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7fe5c000d5f0 tx=0x7fe5c000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:48.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:48.669+0000 7fe5b9ffb640 1 -- 172.21.15.27:0/2408678483 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5c00125f0 con 0x7fe5c406e9b0 2024-09-12T03:23:48.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:48.669+0000 7fe5b9ffb640 1 -- 172.21.15.27:0/2408678483 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5c0019070 con 0x7fe5c406e9b0 2024-09-12T03:23:48.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:48.669+0000 7fe5cb76d640 1 -- 172.21.15.27:0/2408678483 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5c415cd70 con 0x7fe5c406e9b0 2024-09-12T03:23:48.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:48.669+0000 7fe5b9ffb640 1 -- 172.21.15.27:0/2408678483 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5c00129d0 con 0x7fe5c406e9b0 2024-09-12T03:23:48.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:48.669+0000 7fe5cb76d640 1 -- 172.21.15.27:0/2408678483 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5c415d330 con 0x7fe5c406e9b0 2024-09-12T03:23:48.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:48.670+0000 7fe5cb76d640 1 -- 172.21.15.27:0/2408678483 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5c40b3560 con 0x7fe5c406e9b0 2024-09-12T03:23:48.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:48.670+0000 7fe5b9ffb640 1 -- 172.21.15.27:0/2408678483 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 25) ==== 91365+0+0 (secure 0 0 0) 0x7fe5c0012b70 con 0x7fe5c406e9b0 2024-09-12T03:23:48.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:48.670+0000 7fe5b9ffb640 1 --2- 172.21.15.27:0/2408678483 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fe5a806c7f0 0x7fe5a806ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:48.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:48.671+0000 7fe5b9ffb640 1 -- 172.21.15.27:0/2408678483 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 4999+0+0 (secure 0 0 0) 0x7fe5c0089910 con 0x7fe5c406e9b0 2024-09-12T03:23:48.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:48.671+0000 7fe5c8d08640 1 --2- 172.21.15.27:0/2408678483 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fe5a806c7f0 0x7fe5a806ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:48.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:48.671+0000 7fe5c8d08640 1 --2- 172.21.15.27:0/2408678483 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fe5a806c7f0 0x7fe5a806ecb0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fe5c41f5660 tx=0x7fe5b4009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:48.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:48.675+0000 7fe5b9ffb640 1 -- 172.21.15.27:0/2408678483 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fe5c0059ec0 con 0x7fe5c406e9b0 2024-09-12T03:23:48.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:48.917+0000 7fe5cb76d640 1 -- 172.21.15.27:0/2408678483 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_upgrade_v1_to_v2"]} v 0) -- 0x7fe5c4150aa0 con 0x7fe5c406e9b0 2024-09-12T03:23:49.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.230+0000 7fe5b9ffb640 1 -- 172.21.15.27:0/2408678483 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_upgrade_v1_to_v2"]}]=0 v402) ==== 145+0+0 (secure 0 0 0) 0x7fe5c00597c0 con 0x7fe5c406e9b0 2024-09-12T03:23:49.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.232+0000 7fe5cb76d640 1 -- 172.21.15.27:0/2408678483 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fe5a806c7f0 msgr2=0x7fe5a806ecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:49.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.232+0000 7fe5cb76d640 1 --2- 172.21.15.27:0/2408678483 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fe5a806c7f0 0x7fe5a806ecb0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fe5c41f5660 tx=0x7fe5b4009040 comp rx=0 tx=0).stop 2024-09-12T03:23:49.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.232+0000 7fe5cb76d640 1 -- 172.21.15.27:0/2408678483 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe5c406e9b0 msgr2=0x7fe5c415c400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:49.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.232+0000 7fe5cb76d640 1 --2- 172.21.15.27:0/2408678483 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe5c406e9b0 0x7fe5c415c400 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7fe5c000d5f0 tx=0x7fe5c000dab0 comp rx=0 tx=0).stop 2024-09-12T03:23:49.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.232+0000 7fe5cb76d640 1 -- 172.21.15.27:0/2408678483 shutdown_connections 2024-09-12T03:23:49.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.232+0000 7fe5cb76d640 1 --2- 172.21.15.27:0/2408678483 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe5c415b2b0 0x7fe5c4154260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:49.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.232+0000 7fe5cb76d640 1 --2- 172.21.15.27:0/2408678483 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe5c4072d70 0x7fe5c415db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:49.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.232+0000 7fe5cb76d640 1 --2- 172.21.15.27:0/2408678483 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fe5a806c7f0 0x7fe5a806ecb0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:49.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.232+0000 7fe5cb76d640 1 --2- 172.21.15.27:0/2408678483 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe5c406e9b0 0x7fe5c415c400 unknown :-1 s=CLOSED pgs=1638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:49.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.232+0000 7fe5cb76d640 1 -- 172.21.15.27:0/2408678483 >> 172.21.15.27:0/2408678483 conn(0x7fe5c4062c60 msgr2=0x7fe5c4071030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:49.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.232+0000 7fe5cb76d640 1 -- 172.21.15.27:0/2408678483 shutdown_connections 2024-09-12T03:23:49.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.232+0000 7fe5cb76d640 1 -- 172.21.15.27:0/2408678483 wait complete. 2024-09-12T03:23:49.244 INFO:tasks.cephfs_test_runner:poor man's upgrade test -- theme continues... ... ok 2024-09-12T03:23:49.244 INFO:tasks.cephfs_test_runner:Starting test: test_unparseable_metafile_on_legacy_to_v1_upgrade (tasks.cephfs.test_volumes.TestMisc) Validate unparseable text .meta file on legacy subvol root doesn't break the system 2024-09-12T03:23:49.245 INFO:tasks.cephfs_test_runner:test_unparseable_metafile_on_legacy_to_v1_upgrade (tasks.cephfs.test_volumes.TestMisc) 2024-09-12T03:23:49.245 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-12T03:23:49.348 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.348+0000 7f4a68bfd640 1 Processor -- start 2024-09-12T03:23:49.348 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.348+0000 7f4a68bfd640 1 -- start start 2024-09-12T03:23:49.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.349+0000 7f4a68bfd640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f4a640b1ca0 0x7f4a6415f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:49.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.349+0000 7f4a68bfd640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4a640cf660 0x7f4a640d2780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:49.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.349+0000 7f4a68bfd640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4a6415fd70 con 0x7f4a6406f360 2024-09-12T03:23:49.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.349+0000 7f4a68bfd640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f4a640d22f0 con 0x7f4a640b1ca0 2024-09-12T03:23:49.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.349+0000 7f4a68bfd640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f4a6406cc40 con 0x7f4a640cf660 2024-09-12T03:23:49.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.349+0000 7f4a6259c640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f4a640b1ca0 0x7f4a6415f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:49.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.349+0000 7f4a6259c640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f4a640b1ca0 0x7f4a6415f830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:53296/0 (socket says 172.21.15.27:53296) 2024-09-12T03:23:49.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.349+0000 7f4a6259c640 1 -- 172.21.15.27:0/1157793965 learned_addr learned my addr 172.21.15.27:0/1157793965 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:23:49.359 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.349+0000 7f4a62d9d640 1 --2- 172.21.15.27:0/1157793965 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4a640cf660 0x7f4a640d2780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:49.359 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.349+0000 7f4a6259c640 1 -- 172.21.15.27:0/1157793965 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4a640cf660 msgr2=0x7f4a640d2780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:49.359 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.349+0000 7f4a6259c640 1 --2- 172.21.15.27:0/1157793965 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4a640cf660 0x7f4a640d2780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:49.359 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.349+0000 7f4a6259c640 1 -- 172.21.15.27:0/1157793965 >> v1:172.21.15.45:6789/0 conn(0x7f4a6406f360 legacy=0x7f4a640cef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:49.359 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.349+0000 7f4a6259c640 1 -- 172.21.15.27:0/1157793965 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a640d1ce0 con 0x7f4a640b1ca0 2024-09-12T03:23:49.359 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.350+0000 7f4a6259c640 1 --2- 172.21.15.27:0/1157793965 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4a640b1ca0 0x7f4a6415f830 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7f4a54009e50 tx=0x7f4a5402a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=687801d99ba488c6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:49.360 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.350+0000 7f4a6159a640 1 -- 172.21.15.27:0/1157793965 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a54036070 con 0x7f4a640b1ca0 2024-09-12T03:23:49.360 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.350+0000 7f4a6159a640 1 -- 172.21.15.27:0/1157793965 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a54031600 con 0x7f4a640b1ca0 2024-09-12T03:23:49.360 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.350+0000 7f4a6159a640 1 -- 172.21.15.27:0/1157793965 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a540319d0 con 0x7f4a640b1ca0 2024-09-12T03:23:49.360 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.351+0000 7f4a68bfd640 1 -- 172.21.15.27:0/1157793965 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4a640b1ca0 msgr2=0x7f4a6415f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:49.360 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.351+0000 7f4a68bfd640 1 --2- 172.21.15.27:0/1157793965 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4a640b1ca0 0x7f4a6415f830 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7f4a54009e50 tx=0x7f4a5402a6c0 comp rx=0 tx=0).stop 2024-09-12T03:23:49.360 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.351+0000 7f4a68bfd640 1 -- 172.21.15.27:0/1157793965 shutdown_connections 2024-09-12T03:23:49.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.351+0000 7f4a68bfd640 1 --2- 172.21.15.27:0/1157793965 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4a640cf660 0x7f4a640d2780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:49.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.351+0000 7f4a68bfd640 1 --2- 172.21.15.27:0/1157793965 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4a640b1ca0 0x7f4a6415f830 unknown :-1 s=CLOSED pgs=1639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:49.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.351+0000 7f4a68bfd640 1 -- 172.21.15.27:0/1157793965 >> 172.21.15.27:0/1157793965 conn(0x7f4a64073800 msgr2=0x7f4a6409ba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:49.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.351+0000 7f4a68bfd640 1 -- 172.21.15.27:0/1157793965 shutdown_connections 2024-09-12T03:23:49.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.351+0000 7f4a68bfd640 1 -- 172.21.15.27:0/1157793965 wait complete. 2024-09-12T03:23:49.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.352+0000 7f4a68bfd640 1 Processor -- start 2024-09-12T03:23:49.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.352+0000 7f4a68bfd640 1 -- start start 2024-09-12T03:23:49.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.353+0000 7f4a68bfd640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4a6406f360 0x7f4a64152da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:49.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.353+0000 7f4a68bfd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4a640b1ca0 0x7f4a640cd930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:49.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.353+0000 7f4a68bfd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4a640cf660 0x7f4a640d2780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:49.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.353+0000 7f4a68bfd640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f4a6406d1c0 con 0x7f4a640b1ca0 2024-09-12T03:23:49.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.353+0000 7f4a68bfd640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f4a640d1f80 con 0x7f4a640cf660 2024-09-12T03:23:49.363 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.353+0000 7f4a68bfd640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f4a6406d340 con 0x7f4a6406f360 2024-09-12T03:23:49.363 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.353+0000 7f4a61d9b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4a640b1ca0 0x7f4a640cd930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:49.363 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.353+0000 7f4a61d9b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4a640b1ca0 0x7f4a640cd930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:53312/0 (socket says 172.21.15.27:53312) 2024-09-12T03:23:49.363 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.353+0000 7f4a61d9b640 1 -- 172.21.15.27:0/2965278748 learned_addr learned my addr 172.21.15.27:0/2965278748 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:23:49.363 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.353+0000 7f4a6259c640 1 --2- 172.21.15.27:0/2965278748 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4a6406f360 0x7f4a64152da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:49.363 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.353+0000 7f4a62d9d640 1 --2- 172.21.15.27:0/2965278748 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4a640cf660 0x7f4a640d2780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:49.364 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.353+0000 7f4a61d9b640 1 -- 172.21.15.27:0/2965278748 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4a6406f360 msgr2=0x7f4a64152da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:49.364 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.353+0000 7f4a61d9b640 1 --2- 172.21.15.27:0/2965278748 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4a6406f360 0x7f4a64152da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:49.364 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.353+0000 7f4a61d9b640 1 -- 172.21.15.27:0/2965278748 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4a640cf660 msgr2=0x7f4a640d2780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:49.364 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.353+0000 7f4a61d9b640 1 --2- 172.21.15.27:0/2965278748 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4a640cf660 0x7f4a640d2780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:49.364 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.353+0000 7f4a61d9b640 1 -- 172.21.15.27:0/2965278748 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a6415b1f0 con 0x7f4a640b1ca0 2024-09-12T03:23:49.364 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.354+0000 7f4a61d9b640 1 --2- 172.21.15.27:0/2965278748 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4a640b1ca0 0x7f4a640cd930 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7f4a4c00c430 tx=0x7f4a4c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:49.365 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.354+0000 7f4a5affd640 1 -- 172.21.15.27:0/2965278748 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a4c011790 con 0x7f4a640b1ca0 2024-09-12T03:23:49.365 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.354+0000 7f4a5affd640 1 -- 172.21.15.27:0/2965278748 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a4c017070 con 0x7f4a640b1ca0 2024-09-12T03:23:49.365 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.354+0000 7f4a68bfd640 1 -- 172.21.15.27:0/2965278748 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a640cce50 con 0x7f4a640b1ca0 2024-09-12T03:23:49.365 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.354+0000 7f4a5affd640 1 -- 172.21.15.27:0/2965278748 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a4c011bb0 con 0x7f4a640b1ca0 2024-09-12T03:23:49.365 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.354+0000 7f4a68bfd640 1 -- 172.21.15.27:0/2965278748 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a640cd410 con 0x7f4a640b1ca0 2024-09-12T03:23:49.366 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.355+0000 7f4a68bfd640 1 -- 172.21.15.27:0/2965278748 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a640cd610 con 0x7f4a640b1ca0 2024-09-12T03:23:49.366 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.355+0000 7f4a5affd640 1 -- 172.21.15.27:0/2965278748 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 25) ==== 91365+0+0 (secure 0 0 0) 0x7f4a4c01c460 con 0x7f4a640b1ca0 2024-09-12T03:23:49.366 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.356+0000 7f4a5affd640 1 --2- 172.21.15.27:0/2965278748 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f4a4406c7c0 0x7f4a4406ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:49.366 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.356+0000 7f4a5affd640 1 -- 172.21.15.27:0/2965278748 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 4999+0+0 (secure 0 0 0) 0x7f4a4c089360 con 0x7f4a640b1ca0 2024-09-12T03:23:49.366 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.356+0000 7f4a6259c640 1 --2- 172.21.15.27:0/2965278748 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f4a4406c7c0 0x7f4a4406ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:49.366 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.356+0000 7f4a6259c640 1 --2- 172.21.15.27:0/2965278748 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f4a4406c7c0 0x7f4a4406ec80 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f4a54033000 tx=0x7f4a54009000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:49.367 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.360+0000 7f4a5affd640 1 -- 172.21.15.27:0/2965278748 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f4a640cd610 con 0x7f4a640b1ca0 2024-09-12T03:23:49.538 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.538+0000 7f4a68bfd640 1 -- 172.21.15.27:0/2965278748 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4a640cfc40 con 0x7f4a640b1ca0 2024-09-12T03:23:49.539 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.539+0000 7f4a5affd640 1 -- 172.21.15.27:0/2965278748 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v152) ==== 74+0+13083 (secure 0 0 0) 0x7f4a4c0592c0 con 0x7f4a640b1ca0 2024-09-12T03:23:49.540 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:23:49.540 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":152,"fsid":"d2df504e-2647-4385-a5db-9fdf69f487a0","created":"2024-09-12T03:16:07.778937+0000","modified":"2024-09-12T03:23:48.224608+0000","last_up_change":"2024-09-12T03:16:19.314778+0000","last_in_change":"2024-09-12T03:16:09.893086+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T03:16:20.097646+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1851116c-1fb3-4dc9-9772-f90e2ef4c355","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6817","nonce":3002321488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6819","nonce":3002321488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6823","nonce":3002321488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6821","nonce":3002321488}]},"public_addr":"172.21.15.27:6817/3002321488","cluster_addr":"172.21.15.27:6819/3002321488","heartbeat_back_addr":"172.21.15.27:6823/3002321488","heartbeat_front_addr":"172.21.15.27:6821/3002321488","state":["exists","up"]},{"osd":1,"uuid":"f38ae17d-73eb-4204-b99d-975fa806fb87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6809","nonce":1613102886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6811","nonce":1613102886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6815","nonce":1613102886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6813","nonce":1613102886}]},"public_addr":"172.21.15.27:6809/1613102886","cluster_addr":"172.21.15.27:6811/1613102886","heartbeat_back_addr":"172.21.15.27:6815/1613102886","heartbeat_front_addr":"172.21.15.27:6813/1613102886","state":["exists","up"]},{"osd":2,"uuid":"fb12dcc8-bdd0-42b8-b46d-8b10106224bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6825","nonce":3534723355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6827","nonce":3534723355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6831","nonce":3534723355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6829","nonce":3534723355}]},"public_addr":"172.21.15.27:6825/3534723355","cluster_addr":"172.21.15.27:6827/3534723355","heartbeat_back_addr":"172.21.15.27:6831/3534723355","heartbeat_front_addr":"172.21.15.27:6829/3534723355","state":["exists","up"]},{"osd":3,"uuid":"d74d3f49-9c24-4b9f-98e2-3ecd7b5ef3b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6801","nonce":80661598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6803","nonce":80661598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6807","nonce":80661598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6805","nonce":80661598}]},"public_addr":"172.21.15.27:6801/80661598","cluster_addr":"172.21.15.27:6803/80661598","heartbeat_back_addr":"172.21.15.27:6807/80661598","heartbeat_front_addr":"172.21.15.27:6805/80661598","state":["exists","up"]},{"osd":4,"uuid":"2bf123f2-1b0d-41c4-aa3b-805bab267e59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6817","nonce":3943189827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6819","nonce":3943189827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6823","nonce":3943189827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6821","nonce":3943189827}]},"public_addr":"172.21.15.45:6817/3943189827","cluster_addr":"172.21.15.45:6819/3943189827","heartbeat_back_addr":"172.21.15.45:6823/3943189827","heartbeat_front_addr":"172.21.15.45:6821/3943189827","state":["exists","up"]},{"osd":5,"uuid":"c49ba5da-91c6-4a86-a97b-4aba89c253ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6809","nonce":3620760330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6811","nonce":3620760330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6815","nonce":3620760330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6813","nonce":3620760330}]},"public_addr":"172.21.15.45:6809/3620760330","cluster_addr":"172.21.15.45:6811/3620760330","heartbeat_back_addr":"172.21.15.45:6815/3620760330","heartbeat_front_addr":"172.21.15.45:6813/3620760330","state":["exists","up"]},{"osd":6,"uuid":"b0123475-89f7-4a34-9e33-d797b99dd9dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6801","nonce":1989083510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6803","nonce":1989083510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6807","nonce":1989083510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6805","nonce":1989083510}]},"public_addr":"172.21.15.45:6801/1989083510","cluster_addr":"172.21.15.45:6803/1989083510","heartbeat_back_addr":"172.21.15.45:6807/1989083510","heartbeat_front_addr":"172.21.15.45:6805/1989083510","state":["exists","up"]},{"osd":7,"uuid":"4dad773d-b7d8-46fc-8a10-a972aad27b29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6825","nonce":1513300294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6827","nonce":1513300294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6831","nonce":1513300294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6829","nonce":1513300294}]},"public_addr":"172.21.15.45:6825/1513300294","cluster_addr":"172.21.15.45:6827/1513300294","heartbeat_back_addr":"172.21.15.45:6831/1513300294","heartbeat_front_addr":"172.21.15.45:6829/1513300294","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.881855+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:17.133252+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.735339+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.500742+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.477571+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:17.167667+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.470291+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.873743+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.27:6835/1241148443":"2024-09-13T03:23:45.216803+0000","172.21.15.27:6833/1241148443":"2024-09-13T03:23:45.216803+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]},{"pool":19,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T03:23:49.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.541+0000 7f4a68bfd640 1 -- 172.21.15.27:0/2965278748 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f4a4406c7c0 msgr2=0x7f4a4406ec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:49.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.541+0000 7f4a68bfd640 1 --2- 172.21.15.27:0/2965278748 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f4a4406c7c0 0x7f4a4406ec80 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f4a54033000 tx=0x7f4a54009000 comp rx=0 tx=0).stop 2024-09-12T03:23:49.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.541+0000 7f4a68bfd640 1 -- 172.21.15.27:0/2965278748 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4a640b1ca0 msgr2=0x7f4a640cd930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:49.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.541+0000 7f4a68bfd640 1 --2- 172.21.15.27:0/2965278748 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4a640b1ca0 0x7f4a640cd930 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7f4a4c00c430 tx=0x7f4a4c00c8f0 comp rx=0 tx=0).stop 2024-09-12T03:23:49.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.541+0000 7f4a68bfd640 1 -- 172.21.15.27:0/2965278748 shutdown_connections 2024-09-12T03:23:49.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.541+0000 7f4a68bfd640 1 --2- 172.21.15.27:0/2965278748 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4a640cf660 0x7f4a640d2780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:49.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.541+0000 7f4a68bfd640 1 --2- 172.21.15.27:0/2965278748 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f4a4406c7c0 0x7f4a4406ec80 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:49.543 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.541+0000 7f4a68bfd640 1 --2- 172.21.15.27:0/2965278748 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4a640b1ca0 0x7f4a640cd930 unknown :-1 s=CLOSED pgs=1640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:49.543 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.541+0000 7f4a68bfd640 1 --2- 172.21.15.27:0/2965278748 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4a6406f360 0x7f4a64152da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:49.543 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.541+0000 7f4a68bfd640 1 -- 172.21.15.27:0/2965278748 >> 172.21.15.27:0/2965278748 conn(0x7f4a64073800 msgr2=0x7f4a640b72a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:49.543 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.541+0000 7f4a68bfd640 1 -- 172.21.15.27:0/2965278748 shutdown_connections 2024-09-12T03:23:49.543 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.541+0000 7f4a68bfd640 1 -- 172.21.15.27:0/2965278748 wait complete. 2024-09-12T03:23:49.553 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-12T03:16:20.097646+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-12T03:23:49.554 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-12T03:23:49.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.660+0000 7fcd0cffd640 1 Processor -- start 2024-09-12T03:23:49.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.660+0000 7fcd0cffd640 1 -- start start 2024-09-12T03:23:49.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.661+0000 7fcd0cffd640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fcd0815a270 0x7fcd0806cff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:49.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.661+0000 7fcd0cffd640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fcd080cfb20 0x7fcd080726a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:49.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.661+0000 7fcd0cffd640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcd080ce650 con 0x7fcd0806d530 2024-09-12T03:23:49.662 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.661+0000 7fcd0cffd640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7fcd080cef80 con 0x7fcd0815a270 2024-09-12T03:23:49.662 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.661+0000 7fcd0cffd640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fcd080ce930 con 0x7fcd080cfb20 2024-09-12T03:23:49.662 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.661+0000 7fcd0659c640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fcd0815a270 0x7fcd0806cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:49.662 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.661+0000 7fcd05d9b640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fcd0806d530 0x7fcd080cf410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:33852/0 (socket says 172.21.15.27:33852) 2024-09-12T03:23:49.662 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.661+0000 7fcd05d9b640 1 -- 172.21.15.27:0/267215568 learned_addr learned my addr 172.21.15.27:0/267215568 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:23:49.662 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.661+0000 7fcd0659c640 1 -- 172.21.15.27:0/267215568 >> v2:172.21.15.45:3300/0 conn(0x7fcd080cfb20 msgr2=0x7fcd080726a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:49.662 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.661+0000 7fcd06d9d640 1 --2- 172.21.15.27:0/267215568 >> v2:172.21.15.45:3300/0 conn(0x7fcd080cfb20 0x7fcd080726a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:49.663 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.661+0000 7fcd0659c640 1 --2- 172.21.15.27:0/267215568 >> v2:172.21.15.45:3300/0 conn(0x7fcd080cfb20 0x7fcd080726a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:49.663 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.661+0000 7fcd0659c640 1 -- 172.21.15.27:0/267215568 >> v1:172.21.15.27:6789/0 conn(0x7fcd0806d530 legacy=0x7fcd080cf410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:49.663 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.661+0000 7fcd0559a640 1 -- 172.21.15.27:0/267215568 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 99330047 0 0) 0x7fcd080ce650 con 0x7fcd0806d530 2024-09-12T03:23:49.663 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.661+0000 7fcd0659c640 1 -- 172.21.15.27:0/267215568 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd080cec10 con 0x7fcd0815a270 2024-09-12T03:23:49.669 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.662+0000 7fcd0659c640 1 --2- 172.21.15.27:0/267215568 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcd0815a270 0x7fcd0806cff0 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7fcd0806f240 tx=0x7fccf802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffc6c299a50ce704 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:49.669 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.662+0000 7fcd0559a640 1 -- 172.21.15.27:0/267215568 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccf8037070 con 0x7fcd0815a270 2024-09-12T03:23:49.670 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.662+0000 7fcd0559a640 1 -- 172.21.15.27:0/267215568 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fccf8032440 con 0x7fcd0815a270 2024-09-12T03:23:49.670 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.662+0000 7fcd0559a640 1 -- 172.21.15.27:0/267215568 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccf8032810 con 0x7fcd0815a270 2024-09-12T03:23:49.670 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.663+0000 7fcd0cffd640 1 -- 172.21.15.27:0/267215568 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcd0815a270 msgr2=0x7fcd0806cff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:49.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.663+0000 7fcd0cffd640 1 --2- 172.21.15.27:0/267215568 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcd0815a270 0x7fcd0806cff0 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7fcd0806f240 tx=0x7fccf802a1a0 comp rx=0 tx=0).stop 2024-09-12T03:23:49.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.663+0000 7fcd0cffd640 1 -- 172.21.15.27:0/267215568 shutdown_connections 2024-09-12T03:23:49.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.663+0000 7fcd0cffd640 1 --2- 172.21.15.27:0/267215568 >> v2:172.21.15.45:3300/0 conn(0x7fcd080cfb20 0x7fcd080726a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:49.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.663+0000 7fcd0cffd640 1 --2- 172.21.15.27:0/267215568 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcd0815a270 0x7fcd0806cff0 unknown :-1 s=CLOSED pgs=1642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:49.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.663+0000 7fcd0cffd640 1 -- 172.21.15.27:0/267215568 >> 172.21.15.27:0/267215568 conn(0x7fcd0806bb30 msgr2=0x7fcd0806bf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:49.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.663+0000 7fcd0cffd640 1 -- 172.21.15.27:0/267215568 shutdown_connections 2024-09-12T03:23:49.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.663+0000 7fcd0cffd640 1 -- 172.21.15.27:0/267215568 wait complete. 2024-09-12T03:23:49.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.664+0000 7fcd0cffd640 1 Processor -- start 2024-09-12T03:23:49.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.664+0000 7fcd0cffd640 1 -- start start 2024-09-12T03:23:49.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.665+0000 7fcd0cffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcd0806d530 0x7fcd0815ff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:49.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.665+0000 7fcd0cffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcd080cfb20 0x7fcd0814c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:49.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.665+0000 7fcd0cffd640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fcd0815a270 0x7fcd0814ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:49.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.665+0000 7fcd0cffd640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fcd0806c7d0 con 0x7fcd0806d530 2024-09-12T03:23:49.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.665+0000 7fcd0cffd640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fcd0806c650 con 0x7fcd080cfb20 2024-09-12T03:23:49.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.665+0000 7fcd0cffd640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fcd0806c950 con 0x7fcd0815a270 2024-09-12T03:23:49.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.665+0000 7fcd0659c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcd0806d530 0x7fcd0815ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:49.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.665+0000 7fcd0659c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcd0806d530 0x7fcd0815ff60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:53318/0 (socket says 172.21.15.27:53318) 2024-09-12T03:23:49.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.665+0000 7fcd0659c640 1 -- 172.21.15.27:0/3432888966 learned_addr learned my addr 172.21.15.27:0/3432888966 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:23:49.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.665+0000 7fcd0659c640 1 -- 172.21.15.27:0/3432888966 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fcd0815a270 msgr2=0x7fcd0814ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:49.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.665+0000 7fcd0659c640 1 --2- 172.21.15.27:0/3432888966 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fcd0815a270 0x7fcd0814ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:49.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.665+0000 7fcd0659c640 1 -- 172.21.15.27:0/3432888966 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcd080cfb20 msgr2=0x7fcd0814c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:49.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.665+0000 7fcd0659c640 1 --2- 172.21.15.27:0/3432888966 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcd080cfb20 0x7fcd0814c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:49.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.665+0000 7fcd0659c640 1 -- 172.21.15.27:0/3432888966 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd081f2690 con 0x7fcd0806d530 2024-09-12T03:23:49.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.665+0000 7fcd0659c640 1 --2- 172.21.15.27:0/3432888966 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcd0806d530 0x7fcd0815ff60 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7fccf8000c00 tx=0x7fccf80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:49.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.666+0000 7fccf6ffd640 1 -- 172.21.15.27:0/3432888966 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccf8041070 con 0x7fcd0806d530 2024-09-12T03:23:49.674 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.666+0000 7fccf6ffd640 1 -- 172.21.15.27:0/3432888966 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fccf803c890 con 0x7fcd0806d530 2024-09-12T03:23:49.674 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.666+0000 7fcd0cffd640 1 -- 172.21.15.27:0/3432888966 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd080abc60 con 0x7fcd0806d530 2024-09-12T03:23:49.674 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.666+0000 7fccf6ffd640 1 -- 172.21.15.27:0/3432888966 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccf8037040 con 0x7fcd0806d530 2024-09-12T03:23:49.674 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.666+0000 7fcd0cffd640 1 -- 172.21.15.27:0/3432888966 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd080abf00 con 0x7fcd0806d530 2024-09-12T03:23:49.674 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.667+0000 7fcd0cffd640 1 -- 172.21.15.27:0/3432888966 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd0815d560 con 0x7fcd0806d530 2024-09-12T03:23:49.674 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.667+0000 7fccf6ffd640 1 -- 172.21.15.27:0/3432888966 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 25) ==== 91365+0+0 (secure 0 0 0) 0x7fccf8044070 con 0x7fcd0806d530 2024-09-12T03:23:49.674 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.668+0000 7fccf6ffd640 1 --2- 172.21.15.27:0/3432888966 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fcce806c7c0 0x7fcce806ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:49.674 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.668+0000 7fccf6ffd640 1 -- 172.21.15.27:0/3432888966 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 4999+0+0 (secure 0 0 0) 0x7fccf80aa380 con 0x7fcd0806d530 2024-09-12T03:23:49.674 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.668+0000 7fcd05d9b640 1 --2- 172.21.15.27:0/3432888966 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fcce806c7c0 0x7fcce806ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:49.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.668+0000 7fcd05d9b640 1 --2- 172.21.15.27:0/3432888966 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fcce806c7c0 0x7fcce806ec80 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fccfc007fd0 tx=0x7fccfc007920 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:49.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.672+0000 7fccf6ffd640 1 -- 172.21.15.27:0/3432888966 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fccf807aa50 con 0x7fcd0806d530 2024-09-12T03:23:49.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.845+0000 7fcd0cffd640 1 -- 172.21.15.27:0/3432888966 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fcd080ccdb0 con 0x7fcd0806d530 2024-09-12T03:23:49.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.845+0000 7fccf6ffd640 1 -- 172.21.15.27:0/3432888966 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v152) ==== 93+0+10 (secure 0 0 0) 0x7fccf807a350 con 0x7fcd0806d530 2024-09-12T03:23:49.845 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 1 2024-09-12T03:23:49.847 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.847+0000 7fcd0cffd640 1 -- 172.21.15.27:0/3432888966 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fcce806c7c0 msgr2=0x7fcce806ec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:49.847 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.847+0000 7fcd0cffd640 1 --2- 172.21.15.27:0/3432888966 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fcce806c7c0 0x7fcce806ec80 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fccfc007fd0 tx=0x7fccfc007920 comp rx=0 tx=0).stop 2024-09-12T03:23:49.847 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.847+0000 7fcd0cffd640 1 -- 172.21.15.27:0/3432888966 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcd0806d530 msgr2=0x7fcd0815ff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:49.847 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.847+0000 7fcd0cffd640 1 --2- 172.21.15.27:0/3432888966 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcd0806d530 0x7fcd0815ff60 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7fccf8000c00 tx=0x7fccf80046c0 comp rx=0 tx=0).stop 2024-09-12T03:23:49.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.847+0000 7fcd0cffd640 1 -- 172.21.15.27:0/3432888966 shutdown_connections 2024-09-12T03:23:49.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.847+0000 7fcd0cffd640 1 --2- 172.21.15.27:0/3432888966 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fcd0815a270 0x7fcd0814ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:49.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.847+0000 7fcd0cffd640 1 --2- 172.21.15.27:0/3432888966 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcd080cfb20 0x7fcd0814c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:49.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.847+0000 7fcd0cffd640 1 --2- 172.21.15.27:0/3432888966 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fcce806c7c0 0x7fcce806ec80 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:49.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.847+0000 7fcd0cffd640 1 --2- 172.21.15.27:0/3432888966 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcd0806d530 0x7fcd0815ff60 unknown :-1 s=CLOSED pgs=1643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:49.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.847+0000 7fcd0cffd640 1 -- 172.21.15.27:0/3432888966 >> 172.21.15.27:0/3432888966 conn(0x7fcd0806bb30 msgr2=0x7fcd0815a670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:49.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.847+0000 7fcd0cffd640 1 -- 172.21.15.27:0/3432888966 shutdown_connections 2024-09-12T03:23:49.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.847+0000 7fcd0cffd640 1 -- 172.21.15.27:0/3432888966 wait complete. 2024-09-12T03:23:49.858 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestMisc.test_unparseable_metafile_on_legacy_to_v1_upgrade' 2024-09-12T03:23:49.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.960+0000 7f7174103640 1 Processor -- start 2024-09-12T03:23:49.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.960+0000 7f7174103640 1 -- start start 2024-09-12T03:23:49.962 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.961+0000 7f7174103640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f716c072d70 0x7f716c0b3560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:49.962 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.961+0000 7f7174103640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f716c06e9b0 0x7f716c06ee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:49.962 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.961+0000 7f7174103640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f716c0ce410 con 0x7f716c15b2b0 2024-09-12T03:23:49.962 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.961+0000 7f7174103640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f716c0720a0 con 0x7f716c06e9b0 2024-09-12T03:23:49.962 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.961+0000 7f7174103640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f716c0732c0 con 0x7f716c072d70 2024-09-12T03:23:49.962 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.961+0000 7f7171e9f640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f716c15b2b0 0x7f716c072430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:33866/0 (socket says 172.21.15.27:33866) 2024-09-12T03:23:49.962 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.961+0000 7f7171e9f640 1 -- 172.21.15.27:0/4037837127 learned_addr learned my addr 172.21.15.27:0/4037837127 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:23:49.962 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.961+0000 7f717169e640 1 --2- 172.21.15.27:0/4037837127 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f716c072d70 0x7f716c0b3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:49.962 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.961+0000 7f71726a0640 1 --2- 172.21.15.27:0/4037837127 >> v2:172.21.15.45:3300/0 conn(0x7f716c06e9b0 0x7f716c06ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:49.963 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.961+0000 7f7170e9d640 1 -- 172.21.15.27:0/4037837127 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2919571172 0 0) 0x7f715c001490 con 0x7f716c15b2b0 2024-09-12T03:23:49.963 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.962+0000 7f7170e9d640 1 -- 172.21.15.27:0/4037837127 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7154003520 con 0x7f716c15b2b0 2024-09-12T03:23:49.963 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.962+0000 7f7170e9d640 1 -- 172.21.15.27:0/4037837127 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1463259660 0 0) 0x7f7154003520 con 0x7f716c15b2b0 2024-09-12T03:23:49.963 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.962+0000 7f717169e640 1 -- 172.21.15.27:0/4037837127 >> v2:172.21.15.45:3300/0 conn(0x7f716c06e9b0 msgr2=0x7f716c06ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:49.963 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.962+0000 7f717169e640 1 --2- 172.21.15.27:0/4037837127 >> v2:172.21.15.45:3300/0 conn(0x7f716c06e9b0 0x7f716c06ee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:49.963 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.962+0000 7f717169e640 1 -- 172.21.15.27:0/4037837127 >> v1:172.21.15.27:6789/0 conn(0x7f716c15b2b0 legacy=0x7f716c072430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:49.963 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.962+0000 7f717169e640 1 -- 172.21.15.27:0/4037837127 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f716c071ae0 con 0x7f716c072d70 2024-09-12T03:23:49.963 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.962+0000 7f71726a0640 1 --2- 172.21.15.27:0/4037837127 >> v2:172.21.15.45:3300/0 conn(0x7f716c06e9b0 0x7f716c06ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:23:49.964 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.963+0000 7f717169e640 1 --2- 172.21.15.27:0/4037837127 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f716c072d70 0x7f716c0b3560 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f71600097a0 tx=0x7f716002a110 comp rx=0 tx=0).ready entity=mon.2 client_cookie=54f86da92feecd41 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:49.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.963+0000 7f7170e9d640 1 -- 172.21.15.27:0/4037837127 <== mon.2 v2:172.21.15.45:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7160037070 con 0x7f716c072d70 2024-09-12T03:23:49.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.963+0000 7f7170e9d640 1 -- 172.21.15.27:0/4037837127 <== mon.2 v2:172.21.15.45:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f716002aba0 con 0x7f716c072d70 2024-09-12T03:23:49.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.963+0000 7f7170e9d640 1 -- 172.21.15.27:0/4037837127 <== mon.2 v2:172.21.15.45:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f715c001bc0 con 0x7f716c072d70 2024-09-12T03:23:49.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.963+0000 7f7174103640 1 -- 172.21.15.27:0/4037837127 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f716c072d70 msgr2=0x7f716c0b3560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:49.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.963+0000 7f7174103640 1 --2- 172.21.15.27:0/4037837127 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f716c072d70 0x7f716c0b3560 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f71600097a0 tx=0x7f716002a110 comp rx=0 tx=0).stop 2024-09-12T03:23:49.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.963+0000 7f7174103640 1 -- 172.21.15.27:0/4037837127 shutdown_connections 2024-09-12T03:23:49.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.963+0000 7f7174103640 1 --2- 172.21.15.27:0/4037837127 >> v2:172.21.15.45:3300/0 conn(0x7f716c06e9b0 0x7f716c06ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:49.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.963+0000 7f7174103640 1 --2- 172.21.15.27:0/4037837127 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f716c072d70 0x7f716c0b3560 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:49.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.963+0000 7f7174103640 1 -- 172.21.15.27:0/4037837127 >> 172.21.15.27:0/4037837127 conn(0x7f716c062c60 msgr2=0x7f716c06bb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:49.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.963+0000 7f7174103640 1 -- 172.21.15.27:0/4037837127 shutdown_connections 2024-09-12T03:23:49.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.963+0000 7f7174103640 1 -- 172.21.15.27:0/4037837127 wait complete. 2024-09-12T03:23:49.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.965+0000 7f7174103640 1 Processor -- start 2024-09-12T03:23:49.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.965+0000 7f7174103640 1 -- start start 2024-09-12T03:23:49.969 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.965+0000 7f7174103640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f716c06e9b0 0x7f716c1548e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:49.969 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.965+0000 7f7174103640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f716c072d70 0x7f716c14fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:49.969 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.965+0000 7f7174103640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f716c15b2b0 0x7f716c150110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:49.969 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.965+0000 7f7174103640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f716c15b690 con 0x7f716c072d70 2024-09-12T03:23:49.969 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.965+0000 7f7174103640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f716c0ce690 con 0x7f716c06e9b0 2024-09-12T03:23:49.969 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.965+0000 7f7174103640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f716c15b810 con 0x7f716c15b2b0 2024-09-12T03:23:49.969 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.965+0000 7f717169e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f716c072d70 0x7f716c14fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:49.969 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.965+0000 7f717169e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f716c072d70 0x7f716c14fbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:53332/0 (socket says 172.21.15.27:53332) 2024-09-12T03:23:49.969 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.965+0000 7f717169e640 1 -- 172.21.15.27:0/538241950 learned_addr learned my addr 172.21.15.27:0/538241950 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:23:49.969 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.966+0000 7f7171e9f640 1 --2- 172.21.15.27:0/538241950 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f716c06e9b0 0x7f716c1548e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:49.969 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.966+0000 7f71726a0640 1 --2- 172.21.15.27:0/538241950 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f716c15b2b0 0x7f716c150110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:49.970 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.966+0000 7f717169e640 1 -- 172.21.15.27:0/538241950 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f716c15b2b0 msgr2=0x7f716c150110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:49.970 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.966+0000 7f717169e640 1 --2- 172.21.15.27:0/538241950 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f716c15b2b0 0x7f716c150110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:49.970 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.966+0000 7f717169e640 1 -- 172.21.15.27:0/538241950 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f716c06e9b0 msgr2=0x7f716c1548e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:49.970 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.966+0000 7f717169e640 1 --2- 172.21.15.27:0/538241950 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f716c06e9b0 0x7f716c1548e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:49.970 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.966+0000 7f717169e640 1 -- 172.21.15.27:0/538241950 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f716c153760 con 0x7f716c072d70 2024-09-12T03:23:49.970 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.966+0000 7f7171e9f640 1 --2- 172.21.15.27:0/538241950 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f716c06e9b0 0x7f716c1548e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:23:49.970 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.966+0000 7f71726a0640 1 --2- 172.21.15.27:0/538241950 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f716c15b2b0 0x7f716c150110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:23:49.970 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.966+0000 7f717169e640 1 --2- 172.21.15.27:0/538241950 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f716c072d70 0x7f716c14fbd0 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7f7160035040 tx=0x7f71600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:49.970 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.966+0000 7f716a7fc640 1 -- 172.21.15.27:0/538241950 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7160037050 con 0x7f716c072d70 2024-09-12T03:23:49.970 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.966+0000 7f716a7fc640 1 -- 172.21.15.27:0/538241950 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7160043070 con 0x7f716c072d70 2024-09-12T03:23:49.970 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.966+0000 7f7174103640 1 -- 172.21.15.27:0/538241950 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f716c152b50 con 0x7f716c072d70 2024-09-12T03:23:49.970 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.966+0000 7f716a7fc640 1 -- 172.21.15.27:0/538241950 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7160006e10 con 0x7f716c072d70 2024-09-12T03:23:49.970 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.966+0000 7f7174103640 1 -- 172.21.15.27:0/538241950 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f716c153110 con 0x7f716c072d70 2024-09-12T03:23:49.971 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.967+0000 7f7174103640 1 -- 172.21.15.27:0/538241950 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f716c0b3560 con 0x7f716c072d70 2024-09-12T03:23:49.971 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.968+0000 7f716a7fc640 1 -- 172.21.15.27:0/538241950 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 25) ==== 91365+0+0 (secure 0 0 0) 0x7f716003ca10 con 0x7f716c072d70 2024-09-12T03:23:49.971 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.968+0000 7f716a7fc640 1 --2- 172.21.15.27:0/538241950 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f715806c7c0 0x7f715806ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:49.971 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.968+0000 7f716a7fc640 1 -- 172.21.15.27:0/538241950 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 4999+0+0 (secure 0 0 0) 0x7f71600aa350 con 0x7f716c072d70 2024-09-12T03:23:49.971 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.969+0000 7f7171e9f640 1 --2- 172.21.15.27:0/538241950 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f715806c7c0 0x7f715806ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:49.971 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.969+0000 7f7171e9f640 1 --2- 172.21.15.27:0/538241950 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f715806c7c0 0x7f715806ec80 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f715c005c00 tx=0x7f715c009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:49.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:49.973+0000 7f716a7fc640 1 -- 172.21.15.27:0/538241950 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f716007aa20 con 0x7f716c072d70 2024-09-12T03:23:50.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:50.214+0000 7f7174103640 1 -- 172.21.15.27:0/538241950 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_unparseable_metafile_on_legacy_to_v1_upgrade"]} v 0) -- 0x7f716c15b9f0 con 0x7f716c072d70 2024-09-12T03:23:50.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:50.231+0000 7f716a7fc640 1 -- 172.21.15.27:0/538241950 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_unparseable_metafile_on_legacy_to_v1_upgrade"]}]=0 v403) ==== 166+0+0 (secure 0 0 0) 0x7f716007a320 con 0x7f716c072d70 2024-09-12T03:23:50.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:50.233+0000 7f7174103640 1 -- 172.21.15.27:0/538241950 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f715806c7c0 msgr2=0x7f715806ec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:50.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:50.233+0000 7f7174103640 1 --2- 172.21.15.27:0/538241950 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f715806c7c0 0x7f715806ec80 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f715c005c00 tx=0x7f715c009040 comp rx=0 tx=0).stop 2024-09-12T03:23:50.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:50.233+0000 7f7174103640 1 -- 172.21.15.27:0/538241950 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f716c072d70 msgr2=0x7f716c14fbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:50.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:50.233+0000 7f7174103640 1 --2- 172.21.15.27:0/538241950 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f716c072d70 0x7f716c14fbd0 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7f7160035040 tx=0x7f71600046c0 comp rx=0 tx=0).stop 2024-09-12T03:23:50.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:50.233+0000 7f7174103640 1 -- 172.21.15.27:0/538241950 shutdown_connections 2024-09-12T03:23:50.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:50.233+0000 7f7174103640 1 --2- 172.21.15.27:0/538241950 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f716c15b2b0 0x7f716c150110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:50.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:50.233+0000 7f7174103640 1 --2- 172.21.15.27:0/538241950 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f715806c7c0 0x7f715806ec80 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:50.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:50.233+0000 7f7174103640 1 --2- 172.21.15.27:0/538241950 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f716c072d70 0x7f716c14fbd0 unknown :-1 s=CLOSED pgs=1645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:50.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:50.233+0000 7f7174103640 1 --2- 172.21.15.27:0/538241950 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f716c06e9b0 0x7f716c1548e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:50.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:50.233+0000 7f7174103640 1 -- 172.21.15.27:0/538241950 >> 172.21.15.27:0/538241950 conn(0x7f716c062c60 msgr2=0x7f716c15f8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:50.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:50.233+0000 7f7174103640 1 -- 172.21.15.27:0/538241950 shutdown_connections 2024-09-12T03:23:50.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:50.233+0000 7f7174103640 1 -- 172.21.15.27:0/538241950 wait complete. 2024-09-12T03:23:50.244 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-12T03:23:50.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:50.350+0000 7ff05545e640 1 Processor -- start 2024-09-12T03:23:50.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:50.350+0000 7ff05545e640 1 -- start start 2024-09-12T03:23:50.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:50.351+0000 7ff05545e640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7ff0500b6240 0x7ff0500b6620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:50.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:50.351+0000 7ff05545e640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff05006e780 con 0x7ff0500b87d0 2024-09-12T03:23:50.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:50.351+0000 7ff05545e640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff0500b6b60 con 0x7ff0500d3c20 2024-09-12T03:23:50.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:50.351+0000 7ff05545e640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7ff0500d2930 con 0x7ff0500b6240 2024-09-12T03:23:50.352 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:50.351+0000 7ff04effd640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7ff0500b6240 0x7ff0500b6620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:50.352 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:50.351+0000 7ff04f7fe640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7ff0500b87d0 0x7ff05006e070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:33880/0 (socket says 172.21.15.27:33880) 2024-09-12T03:23:50.352 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:50.351+0000 7ff04f7fe640 1 -- 172.21.15.27:0/2517260033 learned_addr learned my addr 172.21.15.27:0/2517260033 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:23:50.352 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:50.351+0000 7ff04effd640 1 -- 172.21.15.27:0/2517260033 >> v1:172.21.15.45:6789/0 conn(0x7ff0500d3c20 legacy=0x7ff05006d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:50.352 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:50.351+0000 7ff04effd640 1 -- 172.21.15.27:0/2517260033 >> v1:172.21.15.27:6789/0 conn(0x7ff0500b87d0 legacy=0x7ff05006e070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:50.352 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:50.351+0000 7ff04effd640 1 -- 172.21.15.27:0/2517260033 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0500d2c10 con 0x7ff0500b6240 2024-09-12T03:23:50.352 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:50.351+0000 7ff04dffb640 1 -- 172.21.15.27:0/2517260033 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2148987727 0 0) 0x7ff05006e780 con 0x7ff0500b87d0 2024-09-12T03:23:50.353 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:50.352+0000 7ff04effd640 1 --2- 172.21.15.27:0/2517260033 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff0500b6240 0x7ff0500b6620 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7ff0400097a0 tx=0x7ff04002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5045e80afa26d39 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:50.353 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:50.352+0000 7ff04dffb640 1 -- 172.21.15.27:0/2517260033 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff040037070 con 0x7ff0500b6240 2024-09-12T03:23:50.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:50.352+0000 7ff04dffb640 1 -- 172.21.15.27:0/2517260033 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff040032440 con 0x7ff0500b6240 2024-09-12T03:23:50.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:50.352+0000 7ff04dffb640 1 -- 172.21.15.27:0/2517260033 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff040032810 con 0x7ff0500b6240 2024-09-12T03:23:50.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:50.353+0000 7ff05545e640 1 -- 172.21.15.27:0/2517260033 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff0500b6240 msgr2=0x7ff0500b6620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:50.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:50.353+0000 7ff05545e640 1 --2- 172.21.15.27:0/2517260033 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff0500b6240 0x7ff0500b6620 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7ff0400097a0 tx=0x7ff04002a270 comp rx=0 tx=0).stop 2024-09-12T03:23:50.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:50.353+0000 7ff05545e640 1 -- 172.21.15.27:0/2517260033 shutdown_connections 2024-09-12T03:23:50.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:50.353+0000 7ff05545e640 1 --2- 172.21.15.27:0/2517260033 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff0500b6240 0x7ff0500b6620 unknown :-1 s=CLOSED pgs=1647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:50.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:50.353+0000 7ff05545e640 1 -- 172.21.15.27:0/2517260033 >> 172.21.15.27:0/2517260033 conn(0x7ff0500d1420 msgr2=0x7ff0500d1830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:50.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:50.353+0000 7ff05545e640 1 -- 172.21.15.27:0/2517260033 shutdown_connections 2024-09-12T03:23:50.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:50.353+0000 7ff05545e640 1 -- 172.21.15.27:0/2517260033 wait complete. 2024-09-12T03:23:50.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:50.354+0000 7ff05545e640 1 Processor -- start 2024-09-12T03:23:50.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:50.354+0000 7ff05545e640 1 -- start start 2024-09-12T03:23:50.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:50.355+0000 7ff05545e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff0500b6240 0x7ff05014c570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:50.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:50.355+0000 7ff05545e640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ff0500b87d0 0x7ff0500ca8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:50.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:50.355+0000 7ff05545e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff0500d3c20 0x7ff05015a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:50.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:50.355+0000 7ff05545e640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7ff05015eb60 con 0x7ff0500b6240 2024-09-12T03:23:50.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:50.355+0000 7ff05545e640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ff0500d31d0 con 0x7ff0500d3c20 2024-09-12T03:23:50.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:50.355+0000 7ff05545e640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7ff05015ece0 con 0x7ff0500b87d0 2024-09-12T03:23:50.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:50.355+0000 7ff04effd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff0500b6240 0x7ff05014c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:50.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:50.355+0000 7ff04effd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff0500b6240 0x7ff05014c570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:53354/0 (socket says 172.21.15.27:53354) 2024-09-12T03:23:50.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:50.355+0000 7ff04effd640 1 -- 172.21.15.27:0/2039175619 learned_addr learned my addr 172.21.15.27:0/2039175619 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:23:50.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:50.355+0000 7ff04effd640 1 -- 172.21.15.27:0/2039175619 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ff0500b87d0 msgr2=0x7ff0500ca8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:50.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:50.355+0000 7ff04effd640 1 --2- 172.21.15.27:0/2039175619 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ff0500b87d0 0x7ff0500ca8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:50.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:50.355+0000 7ff04effd640 1 -- 172.21.15.27:0/2039175619 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff0500d3c20 msgr2=0x7ff05015a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:50.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:50.355+0000 7ff04effd640 1 --2- 172.21.15.27:0/2039175619 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff0500d3c20 0x7ff05015a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:50.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:50.355+0000 7ff04effd640 1 -- 172.21.15.27:0/2039175619 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff05014cab0 con 0x7ff0500b6240 2024-09-12T03:23:50.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:50.356+0000 7ff04effd640 1 --2- 172.21.15.27:0/2039175619 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff0500b6240 0x7ff05014c570 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7ff040000c00 tx=0x7ff0400046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:50.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:50.356+0000 7ff0377fe640 1 -- 172.21.15.27:0/2039175619 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff040041070 con 0x7ff0500b6240 2024-09-12T03:23:50.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:50.356+0000 7ff0377fe640 1 -- 172.21.15.27:0/2039175619 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff04003c930 con 0x7ff0500b6240 2024-09-12T03:23:50.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:50.356+0000 7ff05545e640 1 -- 172.21.15.27:0/2039175619 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0500ce640 con 0x7ff0500b6240 2024-09-12T03:23:50.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:50.356+0000 7ff0377fe640 1 -- 172.21.15.27:0/2039175619 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff040037040 con 0x7ff0500b6240 2024-09-12T03:23:50.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:50.356+0000 7ff05545e640 1 -- 172.21.15.27:0/2039175619 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff05014a1e0 con 0x7ff0500b6240 2024-09-12T03:23:50.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:50.357+0000 7ff05545e640 1 -- 172.21.15.27:0/2039175619 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0501620a0 con 0x7ff0500b6240 2024-09-12T03:23:50.359 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:50.357+0000 7ff0377fe640 1 -- 172.21.15.27:0/2039175619 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 25) ==== 91365+0+0 (secure 0 0 0) 0x7ff040044070 con 0x7ff0500b6240 2024-09-12T03:23:50.359 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:50.358+0000 7ff0377fe640 1 --2- 172.21.15.27:0/2039175619 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7ff03c06c7c0 0x7ff03c06ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:50.359 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:50.358+0000 7ff0377fe640 1 -- 172.21.15.27:0/2039175619 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 4999+0+0 (secure 0 0 0) 0x7ff0400aa400 con 0x7ff0500b6240 2024-09-12T03:23:50.359 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:50.358+0000 7ff04e7fc640 1 --2- 172.21.15.27:0/2039175619 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7ff03c06c7c0 0x7ff03c06ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:50.359 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:50.358+0000 7ff04e7fc640 1 --2- 172.21.15.27:0/2039175619 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7ff03c06c7c0 0x7ff03c06ec80 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7ff044006fd0 tx=0x7ff044006bc0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:50.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:50.362+0000 7ff0377fe640 1 -- 172.21.15.27:0/2039175619 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7ff0501620a0 con 0x7ff0500b6240 2024-09-12T03:23:50.539 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:50.539+0000 7ff05545e640 1 -- 172.21.15.27:0/2039175619 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7ff0500d5470 con 0x7ff0500b6240 2024-09-12T03:23:50.544 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:50.544+0000 7ff0377fe640 1 -- 172.21.15.27:0/2039175619 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v30)=0 v30) ==== 122+0+0 (secure 0 0 0) 0x7ff05014cab0 con 0x7ff0500b6240 2024-09-12T03:23:50.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:50.546+0000 7ff05545e640 1 -- 172.21.15.27:0/2039175619 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7ff03c06c7c0 msgr2=0x7ff03c06ec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:50.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:50.546+0000 7ff05545e640 1 --2- 172.21.15.27:0/2039175619 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7ff03c06c7c0 0x7ff03c06ec80 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7ff044006fd0 tx=0x7ff044006bc0 comp rx=0 tx=0).stop 2024-09-12T03:23:50.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:50.546+0000 7ff05545e640 1 -- 172.21.15.27:0/2039175619 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff0500b6240 msgr2=0x7ff05014c570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:50.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:50.546+0000 7ff05545e640 1 --2- 172.21.15.27:0/2039175619 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff0500b6240 0x7ff05014c570 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7ff040000c00 tx=0x7ff0400046c0 comp rx=0 tx=0).stop 2024-09-12T03:23:50.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:50.546+0000 7ff05545e640 1 -- 172.21.15.27:0/2039175619 shutdown_connections 2024-09-12T03:23:50.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:50.546+0000 7ff05545e640 1 --2- 172.21.15.27:0/2039175619 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff0500d3c20 0x7ff05015a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:50.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:50.546+0000 7ff05545e640 1 --2- 172.21.15.27:0/2039175619 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ff0500b87d0 0x7ff0500ca8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:50.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:50.546+0000 7ff05545e640 1 --2- 172.21.15.27:0/2039175619 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7ff03c06c7c0 0x7ff03c06ec80 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:50.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:50.546+0000 7ff05545e640 1 --2- 172.21.15.27:0/2039175619 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff0500b6240 0x7ff05014c570 unknown :-1 s=CLOSED pgs=1648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:50.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:50.546+0000 7ff05545e640 1 -- 172.21.15.27:0/2039175619 >> 172.21.15.27:0/2039175619 conn(0x7ff0500d1420 msgr2=0x7ff0500d1800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:50.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:50.547+0000 7ff05545e640 1 -- 172.21.15.27:0/2039175619 shutdown_connections 2024-09-12T03:23:50.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:50.547+0000 7ff05545e640 1 -- 172.21.15.27:0/2039175619 wait complete. 2024-09-12T03:23:50.558 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-12T03:23:50.558 DEBUG:teuthology.orchestra.run.smithi027:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-12T03:23:50.594 DEBUG:teuthology.orchestra.run.smithi045:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-12T03:23:50.636 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-12T03:23:50.636 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-12T03:23:50.636 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T03:23:50.653 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056004,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497432k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16324 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248716k,nr_inodes=812179,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497432k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-12T03:23:50.654 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-12T03:23:50.654 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T03:23:50.696 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056795,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16602 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249348k,nr_inodes=812337,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-12T03:23:50.696 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-12T03:23:50.696 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T03:23:50.713 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056004,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497432k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16324 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248716k,nr_inodes=812179,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497432k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-12T03:23:50.713 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-12T03:23:50.714 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T03:23:50.755 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056795,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16602 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249348k,nr_inodes=812337,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-12T03:23:50.755 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-12T03:23:50.755 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-12T03:23:50.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:50.858+0000 7f96da206640 1 Processor -- start 2024-09-12T03:23:50.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:50.858+0000 7f96da206640 1 -- start start 2024-09-12T03:23:50.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:50.859+0000 7f96da206640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f96d40b1ca0 0x7f96d415f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:50.860 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:50.859+0000 7f96da206640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f96d406f360 0x7f96d40cef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:50.860 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:50.859+0000 7f96da206640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f96d40cf490 0x7f96d40d2780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:50.860 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:50.859+0000 7f96da206640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f96d40d22f0 con 0x7f96d40cf490 2024-09-12T03:23:50.860 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:50.859+0000 7f96da206640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f96d415fd70 con 0x7f96d40b1ca0 2024-09-12T03:23:50.860 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:50.859+0000 7f96da206640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f96d406cc40 con 0x7f96d406f360 2024-09-12T03:23:50.860 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:50.859+0000 7f96d3fff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f96d40cf490 0x7f96d40d2780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:50.860 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:50.859+0000 7f96d3fff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f96d40cf490 0x7f96d40d2780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:53364/0 (socket says 172.21.15.27:53364) 2024-09-12T03:23:50.860 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:50.859+0000 7f96d3fff640 1 -- 172.21.15.27:0/2746845343 learned_addr learned my addr 172.21.15.27:0/2746845343 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:23:50.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:50.859+0000 7f96d3fff640 1 -- 172.21.15.27:0/2746845343 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f96d406f360 msgr2=0x7f96d40cef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:50.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:50.859+0000 7f96d3fff640 1 --2- 172.21.15.27:0/2746845343 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f96d406f360 0x7f96d40cef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:50.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:50.859+0000 7f96d37fe640 1 --2- 172.21.15.27:0/2746845343 >> v2:172.21.15.45:3300/0 conn(0x7f96d40b1ca0 0x7f96d415f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:50.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:50.859+0000 7f96d3fff640 1 -- 172.21.15.27:0/2746845343 >> v2:172.21.15.45:3300/0 conn(0x7f96d40b1ca0 msgr2=0x7f96d415f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:50.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:50.859+0000 7f96d3fff640 1 --2- 172.21.15.27:0/2746845343 >> v2:172.21.15.45:3300/0 conn(0x7f96d40b1ca0 0x7f96d415f830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:50.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:50.860+0000 7f96d3fff640 1 -- 172.21.15.27:0/2746845343 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96d40d1ce0 con 0x7f96d40cf490 2024-09-12T03:23:50.862 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:50.860+0000 7f96d3fff640 1 --2- 172.21.15.27:0/2746845343 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f96d40cf490 0x7f96d40d2780 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7f96d4073ed0 tx=0x7f96cc02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dd5df40b112430d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:50.862 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:50.860+0000 7f96d27fc640 1 -- 172.21.15.27:0/2746845343 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96cc036070 con 0x7f96d40cf490 2024-09-12T03:23:50.862 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:50.860+0000 7f96d27fc640 1 -- 172.21.15.27:0/2746845343 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96cc031600 con 0x7f96d40cf490 2024-09-12T03:23:50.862 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:50.860+0000 7f96d27fc640 1 -- 172.21.15.27:0/2746845343 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96cc0319d0 con 0x7f96d40cf490 2024-09-12T03:23:50.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:50.861+0000 7f96da206640 1 -- 172.21.15.27:0/2746845343 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f96d40cf490 msgr2=0x7f96d40d2780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:50.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:50.861+0000 7f96da206640 1 --2- 172.21.15.27:0/2746845343 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f96d40cf490 0x7f96d40d2780 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7f96d4073ed0 tx=0x7f96cc02a6a0 comp rx=0 tx=0).stop 2024-09-12T03:23:50.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:50.861+0000 7f96da206640 1 -- 172.21.15.27:0/2746845343 shutdown_connections 2024-09-12T03:23:50.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:50.861+0000 7f96da206640 1 --2- 172.21.15.27:0/2746845343 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f96d40cf490 0x7f96d40d2780 unknown :-1 s=CLOSED pgs=1649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:50.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:50.861+0000 7f96da206640 1 --2- 172.21.15.27:0/2746845343 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f96d406f360 0x7f96d40cef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:50.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:50.861+0000 7f96da206640 1 --2- 172.21.15.27:0/2746845343 >> v2:172.21.15.45:3300/0 conn(0x7f96d40b1ca0 0x7f96d415f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:50.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:50.861+0000 7f96da206640 1 -- 172.21.15.27:0/2746845343 >> 172.21.15.27:0/2746845343 conn(0x7f96d4073800 msgr2=0x7f96d409ba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:50.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:50.861+0000 7f96da206640 1 -- 172.21.15.27:0/2746845343 shutdown_connections 2024-09-12T03:23:50.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:50.861+0000 7f96da206640 1 -- 172.21.15.27:0/2746845343 wait complete. 2024-09-12T03:23:50.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:50.862+0000 7f96da206640 1 Processor -- start 2024-09-12T03:23:50.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:50.862+0000 7f96da206640 1 -- start start 2024-09-12T03:23:50.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:50.863+0000 7f96da206640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f96d406f360 0x7f96d414dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:50.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:50.863+0000 7f96da206640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f96d40b1ca0 0x7f96d40b0500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:50.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:50.863+0000 7f96da206640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f96d40cf490 0x7f96d40b0a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:50.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:50.863+0000 7f96da206640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f96d406d1c0 con 0x7f96d406f360 2024-09-12T03:23:50.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:50.863+0000 7f96da206640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f96d40d1f80 con 0x7f96d40cf490 2024-09-12T03:23:50.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:50.863+0000 7f96da206640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f96d406d340 con 0x7f96d40b1ca0 2024-09-12T03:23:50.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:50.863+0000 7f96d37fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f96d406f360 0x7f96d414dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:50.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:50.863+0000 7f96d37fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f96d406f360 0x7f96d414dee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:53368/0 (socket says 172.21.15.27:53368) 2024-09-12T03:23:50.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:50.863+0000 7f96d37fe640 1 -- 172.21.15.27:0/2422800302 learned_addr learned my addr 172.21.15.27:0/2422800302 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:23:50.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:50.863+0000 7f96d3fff640 1 --2- 172.21.15.27:0/2422800302 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f96d40cf490 0x7f96d40b0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:50.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:50.863+0000 7f96d2ffd640 1 --2- 172.21.15.27:0/2422800302 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f96d40b1ca0 0x7f96d40b0500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:50.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:50.864+0000 7f96d37fe640 1 -- 172.21.15.27:0/2422800302 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f96d40b1ca0 msgr2=0x7f96d40b0500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:50.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:50.864+0000 7f96d37fe640 1 --2- 172.21.15.27:0/2422800302 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f96d40b1ca0 0x7f96d40b0500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:50.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:50.864+0000 7f96d37fe640 1 -- 172.21.15.27:0/2422800302 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f96d40cf490 msgr2=0x7f96d40b0a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:50.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:50.864+0000 7f96d37fe640 1 --2- 172.21.15.27:0/2422800302 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f96d40cf490 0x7f96d40b0a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:50.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:50.864+0000 7f96d37fe640 1 -- 172.21.15.27:0/2422800302 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96d41f88f0 con 0x7f96d406f360 2024-09-12T03:23:50.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:50.864+0000 7f96d37fe640 1 --2- 172.21.15.27:0/2422800302 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f96d406f360 0x7f96d414dee0 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7f96c400c7f0 tx=0x7f96c400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:50.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:50.864+0000 7f96bbfff640 1 -- 172.21.15.27:0/2422800302 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96c4015070 con 0x7f96d406f360 2024-09-12T03:23:50.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:50.864+0000 7f96bbfff640 1 -- 172.21.15.27:0/2422800302 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96c4004240 con 0x7f96d406f360 2024-09-12T03:23:50.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:50.864+0000 7f96da206640 1 -- 172.21.15.27:0/2422800302 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96d41e7970 con 0x7f96d406f360 2024-09-12T03:23:50.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:50.864+0000 7f96bbfff640 1 -- 172.21.15.27:0/2422800302 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96c4007620 con 0x7f96d406f360 2024-09-12T03:23:50.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:50.864+0000 7f96da206640 1 -- 172.21.15.27:0/2422800302 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96d41e7f30 con 0x7f96d406f360 2024-09-12T03:23:50.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:50.865+0000 7f96da206640 1 -- 172.21.15.27:0/2422800302 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96d40cfc40 con 0x7f96d406f360 2024-09-12T03:23:50.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:50.865+0000 7f96bbfff640 1 -- 172.21.15.27:0/2422800302 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 25) ==== 91365+0+0 (secure 0 0 0) 0x7f96c40077c0 con 0x7f96d406f360 2024-09-12T03:23:50.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:50.866+0000 7f96bbfff640 1 --2- 172.21.15.27:0/2422800302 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f96c006c7f0 0x7f96c006ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:50.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:50.866+0000 7f96bbfff640 1 -- 172.21.15.27:0/2422800302 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 4999+0+0 (secure 0 0 0) 0x7f96c4089320 con 0x7f96d406f360 2024-09-12T03:23:50.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:50.866+0000 7f96d2ffd640 1 --2- 172.21.15.27:0/2422800302 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f96c006c7f0 0x7f96c006ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:50.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:50.866+0000 7f96d2ffd640 1 --2- 172.21.15.27:0/2422800302 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f96c006c7f0 0x7f96c006ecb0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f96d40a0a10 tx=0x7f96c8009110 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:50.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:50.870+0000 7f96bbfff640 1 -- 172.21.15.27:0/2422800302 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f96c4059280 con 0x7f96d406f360 2024-09-12T03:23:51.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:51.080+0000 7f96da206640 1 -- 172.21.15.27:0/2422800302 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f96d4161aa0 con 0x7f96d406f360 2024-09-12T03:23:51.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:51.080+0000 7f96bbfff640 1 -- 172.21.15.27:0/2422800302 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 94 v94) ==== 94+0+3911 (secure 0 0 0) 0x7f96c405d580 con 0x7f96d406f360 2024-09-12T03:23:51.081 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:23:51.081 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":94,"btime":"2024-09-12T03:23:46:218199+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6917,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/3682270902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":3682270902},{"type":"v1","addr":"172.21.15.45:6834","nonce":3682270902}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":87},{"gid":7771,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/2399103579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":2399103579},{"type":"v1","addr":"172.21.15.45:6835","nonce":2399103579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":87},{"gid":8559,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6834/1045432254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":1045432254},{"type":"v1","addr":"172.21.15.27:6834","nonce":1045432254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":87},{"gid":8886,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/2783041851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":2783041851},{"type":"v1","addr":"172.21.15.27:6835","nonce":2783041851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":94}],"filesystems":[]} 2024-09-12T03:23:51.081 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 94 2024-09-12T03:23:51.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:51.082+0000 7f96da206640 1 -- 172.21.15.27:0/2422800302 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f96c006c7f0 msgr2=0x7f96c006ecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:51.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:51.082+0000 7f96da206640 1 --2- 172.21.15.27:0/2422800302 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f96c006c7f0 0x7f96c006ecb0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f96d40a0a10 tx=0x7f96c8009110 comp rx=0 tx=0).stop 2024-09-12T03:23:51.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:51.082+0000 7f96da206640 1 -- 172.21.15.27:0/2422800302 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f96d406f360 msgr2=0x7f96d414dee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:51.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:51.082+0000 7f96da206640 1 --2- 172.21.15.27:0/2422800302 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f96d406f360 0x7f96d414dee0 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7f96c400c7f0 tx=0x7f96c400cbf0 comp rx=0 tx=0).stop 2024-09-12T03:23:51.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:51.082+0000 7f96da206640 1 -- 172.21.15.27:0/2422800302 shutdown_connections 2024-09-12T03:23:51.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:51.082+0000 7f96da206640 1 --2- 172.21.15.27:0/2422800302 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f96d40cf490 0x7f96d40b0a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:51.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:51.082+0000 7f96da206640 1 --2- 172.21.15.27:0/2422800302 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f96d40b1ca0 0x7f96d40b0500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:51.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:51.082+0000 7f96da206640 1 --2- 172.21.15.27:0/2422800302 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f96c006c7f0 0x7f96c006ecb0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:51.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:51.082+0000 7f96da206640 1 --2- 172.21.15.27:0/2422800302 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f96d406f360 0x7f96d414dee0 unknown :-1 s=CLOSED pgs=1650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:51.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:51.082+0000 7f96da206640 1 -- 172.21.15.27:0/2422800302 >> 172.21.15.27:0/2422800302 conn(0x7f96d4073800 msgr2=0x7f96d4160ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:51.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:51.082+0000 7f96da206640 1 -- 172.21.15.27:0/2422800302 shutdown_connections 2024-09-12T03:23:51.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:51.082+0000 7f96da206640 1 -- 172.21.15.27:0/2422800302 wait complete. 2024-09-12T03:23:51.094 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-12T03:23:51.094 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-12T03:23:51.094 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-12T03:23:51.094 INFO:teuthology.orchestra.run:waiting for 300 2024-09-12T03:23:51.094 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-12T03:23:51.095 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-12T03:23:51.095 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-12T03:23:51.095 INFO:teuthology.orchestra.run:waiting for 300 2024-09-12T03:23:51.095 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-12T03:23:51.095 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-12T03:23:51.095 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-12T03:23:51.095 INFO:teuthology.orchestra.run:waiting for 300 2024-09-12T03:23:51.096 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-12T03:23:51.096 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-12T03:23:51.096 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-12T03:23:51.096 INFO:teuthology.orchestra.run:waiting for 300 2024-09-12T03:23:51.104 INFO:tasks.ceph.mds.a:Stopped 2024-09-12T03:23:51.104 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-12T03:23:51.105 INFO:tasks.ceph.mds.c:Stopped 2024-09-12T03:23:51.105 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-12T03:23:51.107 INFO:tasks.ceph.mds.b:Stopped 2024-09-12T03:23:51.107 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-12T03:23:51.108 INFO:tasks.ceph.mds.d:Stopped 2024-09-12T03:23:51.108 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-12T03:23:51.146 INFO:tasks.ceph.mds.a:Started 2024-09-12T03:23:51.147 DEBUG:teuthology.parallel:result is None 2024-09-12T03:23:51.147 INFO:tasks.ceph.mds.c:Started 2024-09-12T03:23:51.147 DEBUG:teuthology.parallel:result is None 2024-09-12T03:23:51.151 INFO:tasks.ceph.mds.b:Started 2024-09-12T03:23:51.151 DEBUG:teuthology.parallel:result is None 2024-09-12T03:23:51.152 INFO:tasks.ceph.mds.d:Started 2024-09-12T03:23:51.152 DEBUG:teuthology.parallel:result is None 2024-09-12T03:23:51.152 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-12T03:23:51.224 INFO:tasks.ceph.mds.a.smithi027.stdout:starting mds.a at 2024-09-12T03:23:51.224 INFO:tasks.ceph.mds.c.smithi027.stdout:starting mds.c at 2024-09-12T03:23:51.238 INFO:tasks.ceph.mds.b.smithi045.stdout:starting mds.b at 2024-09-12T03:23:51.239 INFO:tasks.ceph.mds.d.smithi045.stdout:starting mds.d at 2024-09-12T03:23:51.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:51.264+0000 7f01a103c640 1 Processor -- start 2024-09-12T03:23:51.264 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:51.264+0000 7f01a103c640 1 -- start start 2024-09-12T03:23:51.265 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:51.264+0000 7f01a103c640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f019c06ef40 0x7f019c072000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:51.265 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:51.265+0000 7f01a103c640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f019c0cfc80 con 0x7f019c0b3fd0 2024-09-12T03:23:51.265 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:51.265+0000 7f01a103c640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f019c06d4d0 con 0x7f019c0b3560 2024-09-12T03:23:51.265 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:51.265+0000 7f01a103c640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f019c06f3d0 con 0x7f019c06ef40 2024-09-12T03:23:51.265 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:51.265+0000 7f019ad9d640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f019c0b3fd0 0x7f019c0b43b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:33900/0 (socket says 172.21.15.27:33900) 2024-09-12T03:23:51.265 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:51.265+0000 7f019ad9d640 1 -- 172.21.15.27:0/2243864727 learned_addr learned my addr 172.21.15.27:0/2243864727 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:23:51.266 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:51.265+0000 7f019b59e640 1 --2- 172.21.15.27:0/2243864727 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f019c06ef40 0x7f019c072000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:51.266 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:51.265+0000 7f019a59c640 1 -- 172.21.15.27:0/2243864727 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1552374290 0 0) 0x7f018c001110 con 0x7f019c0b3fd0 2024-09-12T03:23:51.266 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:51.265+0000 7f019a59c640 1 -- 172.21.15.27:0/2243864727 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0188003520 con 0x7f019c0b3fd0 2024-09-12T03:23:51.266 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:51.265+0000 7f019b59e640 1 -- 172.21.15.27:0/2243864727 >> v1:172.21.15.45:6789/0 conn(0x7f019c0b3560 legacy=0x7f019c0cd250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:51.266 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:51.265+0000 7f019a59c640 1 -- 172.21.15.27:0/2243864727 <== mon.1 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4078192807 0 0) 0x7f019c06d4d0 con 0x7f019c0b3560 2024-09-12T03:23:51.266 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:51.265+0000 7f019b59e640 1 -- 172.21.15.27:0/2243864727 >> v1:172.21.15.27:6789/0 conn(0x7f019c0b3fd0 legacy=0x7f019c0b43b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:51.266 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:51.265+0000 7f019b59e640 1 -- 172.21.15.27:0/2243864727 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f019c06d790 con 0x7f019c06ef40 2024-09-12T03:23:51.267 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:51.266+0000 7f019b59e640 1 --2- 172.21.15.27:0/2243864727 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f019c06ef40 0x7f019c072000 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f0194009dc0 tx=0x7f019402a6f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=991bcaf77ae7baa3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:51.267 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:51.266+0000 7f019a59c640 1 -- 172.21.15.27:0/2243864727 <== mon.2 v2:172.21.15.45:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0194007740 con 0x7f019c06ef40 2024-09-12T03:23:51.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:51.266+0000 7f019a59c640 1 -- 172.21.15.27:0/2243864727 <== mon.2 v2:172.21.15.45:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f019402ae50 con 0x7f019c06ef40 2024-09-12T03:23:51.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:51.266+0000 7f019a59c640 1 -- 172.21.15.27:0/2243864727 <== mon.2 v2:172.21.15.45:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0194007b00 con 0x7f019c06ef40 2024-09-12T03:23:51.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:51.267+0000 7f01a103c640 1 -- 172.21.15.27:0/2243864727 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f019c06ef40 msgr2=0x7f019c072000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:51.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:51.267+0000 7f01a103c640 1 --2- 172.21.15.27:0/2243864727 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f019c06ef40 0x7f019c072000 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f0194009dc0 tx=0x7f019402a6f0 comp rx=0 tx=0).stop 2024-09-12T03:23:51.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:51.267+0000 7f01a103c640 1 -- 172.21.15.27:0/2243864727 shutdown_connections 2024-09-12T03:23:51.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:51.267+0000 7f01a103c640 1 --2- 172.21.15.27:0/2243864727 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f019c06ef40 0x7f019c072000 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:51.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:51.267+0000 7f01a103c640 1 -- 172.21.15.27:0/2243864727 >> 172.21.15.27:0/2243864727 conn(0x7f019c062c60 msgr2=0x7f019c06bb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:51.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:51.267+0000 7f01a103c640 1 -- 172.21.15.27:0/2243864727 shutdown_connections 2024-09-12T03:23:51.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:51.267+0000 7f01a103c640 1 -- 172.21.15.27:0/2243864727 wait complete. 2024-09-12T03:23:51.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:51.268+0000 7f01a103c640 1 Processor -- start 2024-09-12T03:23:51.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:51.268+0000 7f01a103c640 1 -- start start 2024-09-12T03:23:51.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:51.269+0000 7f01a103c640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f019c06ef40 0x7f019c0af300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:51.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:51.269+0000 7f01a103c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f019c0b3560 0x7f019c0ac4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:51.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:51.269+0000 7f01a103c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f019c0b3fd0 0x7f019c0add50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:51.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:51.269+0000 7f01a103c640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f019c0cd960 con 0x7f019c0b3560 2024-09-12T03:23:51.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:51.269+0000 7f01a103c640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f019c06ea80 con 0x7f019c0b3fd0 2024-09-12T03:23:51.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:51.269+0000 7f01a103c640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f019c0cdae0 con 0x7f019c06ef40 2024-09-12T03:23:51.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:51.269+0000 7f0193fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f019c0b3560 0x7f019c0ac4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:51.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:51.269+0000 7f0193fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f019c0b3560 0x7f019c0ac4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:53396/0 (socket says 172.21.15.27:53396) 2024-09-12T03:23:51.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:51.269+0000 7f0193fff640 1 -- 172.21.15.27:0/4015712103 learned_addr learned my addr 172.21.15.27:0/4015712103 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:23:51.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:51.269+0000 7f019ad9d640 1 --2- 172.21.15.27:0/4015712103 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f019c06ef40 0x7f019c0af300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:51.272 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:51.269+0000 7f019b59e640 1 --2- 172.21.15.27:0/4015712103 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f019c0b3fd0 0x7f019c0add50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:51.272 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:51.269+0000 7f0193fff640 1 -- 172.21.15.27:0/4015712103 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f019c06ef40 msgr2=0x7f019c0af300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:51.272 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:51.269+0000 7f0193fff640 1 --2- 172.21.15.27:0/4015712103 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f019c06ef40 0x7f019c0af300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:51.272 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:51.269+0000 7f0193fff640 1 -- 172.21.15.27:0/4015712103 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f019c0b3fd0 msgr2=0x7f019c0add50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:51.272 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:51.269+0000 7f0193fff640 1 --2- 172.21.15.27:0/4015712103 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f019c0b3fd0 0x7f019c0add50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:51.272 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:51.269+0000 7f0193fff640 1 -- 172.21.15.27:0/4015712103 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f019c0aca30 con 0x7f019c0b3560 2024-09-12T03:23:51.272 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:51.269+0000 7f019ad9d640 1 --2- 172.21.15.27:0/4015712103 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f019c06ef40 0x7f019c0af300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:23:51.272 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:51.270+0000 7f0193fff640 1 --2- 172.21.15.27:0/4015712103 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f019c0b3560 0x7f019c0ac4f0 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7f018400c460 tx=0x7f018400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:51.272 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:51.270+0000 7f01937fe640 1 -- 172.21.15.27:0/4015712103 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0184005480 con 0x7f019c0b3560 2024-09-12T03:23:51.272 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:51.270+0000 7f01937fe640 1 -- 172.21.15.27:0/4015712103 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01840035f0 con 0x7f019c0b3560 2024-09-12T03:23:51.272 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:51.270+0000 7f01937fe640 1 -- 172.21.15.27:0/4015712103 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0184003a00 con 0x7f019c0b3560 2024-09-12T03:23:51.272 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:51.270+0000 7f01a103c640 1 -- 172.21.15.27:0/4015712103 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f019c0acf90 con 0x7f019c0b3560 2024-09-12T03:23:51.272 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:51.270+0000 7f01a103c640 1 -- 172.21.15.27:0/4015712103 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f019c0ad550 con 0x7f019c0b3560 2024-09-12T03:23:51.272 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:51.271+0000 7f01a103c640 1 -- 172.21.15.27:0/4015712103 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f019c069530 con 0x7f019c0b3560 2024-09-12T03:23:51.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:51.271+0000 7f01937fe640 1 -- 172.21.15.27:0/4015712103 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 25) ==== 91365+0+0 (secure 0 0 0) 0x7f0184003ba0 con 0x7f019c0b3560 2024-09-12T03:23:51.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:51.272+0000 7f01937fe640 1 --2- 172.21.15.27:0/4015712103 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f017c06c960 0x7f017c06ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:51.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:51.272+0000 7f01937fe640 1 -- 172.21.15.27:0/4015712103 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(154..154 src has 1..154) ==== 4999+0+0 (secure 0 0 0) 0x7f0184089070 con 0x7f019c0b3560 2024-09-12T03:23:51.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:51.272+0000 7f019ad9d640 1 --2- 172.21.15.27:0/4015712103 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f017c06c960 0x7f017c06ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:51.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:51.272+0000 7f019ad9d640 1 --2- 172.21.15.27:0/4015712103 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f017c06c960 0x7f017c06ee20 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f018c0012b0 tx=0x7f018c007b70 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:51.276 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:51.276+0000 7f01937fe640 1 -- 172.21.15.27:0/4015712103 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f019c0aca30 con 0x7f019c0b3560 2024-09-12T03:23:51.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:51.464+0000 7f01a103c640 1 -- 172.21.15.27:0/4015712103 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f019c06d4d0 con 0x7f019c0b3560 2024-09-12T03:23:51.552 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:51.552+0000 7f01937fe640 1 -- 172.21.15.27:0/4015712103 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v156) ==== 97+0+0 (secure 0 0 0) 0x7f0184058fd0 con 0x7f019c0b3560 2024-09-12T03:23:51.552 INFO:teuthology.orchestra.run.smithi027.stderr: removed all blocklist entries 2024-09-12T03:23:51.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:51.555+0000 7f01a103c640 1 -- 172.21.15.27:0/4015712103 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f017c06c960 msgr2=0x7f017c06ee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:51.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:51.555+0000 7f01a103c640 1 --2- 172.21.15.27:0/4015712103 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f017c06c960 0x7f017c06ee20 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f018c0012b0 tx=0x7f018c007b70 comp rx=0 tx=0).stop 2024-09-12T03:23:51.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:51.555+0000 7f01a103c640 1 -- 172.21.15.27:0/4015712103 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f019c0b3560 msgr2=0x7f019c0ac4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:51.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:51.555+0000 7f01a103c640 1 --2- 172.21.15.27:0/4015712103 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f019c0b3560 0x7f019c0ac4f0 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7f018400c460 tx=0x7f018400c920 comp rx=0 tx=0).stop 2024-09-12T03:23:51.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:51.555+0000 7f01a103c640 1 -- 172.21.15.27:0/4015712103 shutdown_connections 2024-09-12T03:23:51.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:51.555+0000 7f01a103c640 1 --2- 172.21.15.27:0/4015712103 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f019c0b3fd0 0x7f019c0add50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:51.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:51.555+0000 7f01a103c640 1 --2- 172.21.15.27:0/4015712103 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f017c06c960 0x7f017c06ee20 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:51.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:51.555+0000 7f01a103c640 1 --2- 172.21.15.27:0/4015712103 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f019c0b3560 0x7f019c0ac4f0 unknown :-1 s=CLOSED pgs=1656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:51.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:51.555+0000 7f01a103c640 1 --2- 172.21.15.27:0/4015712103 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f019c06ef40 0x7f019c0af300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:51.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:51.555+0000 7f01a103c640 1 -- 172.21.15.27:0/4015712103 >> 172.21.15.27:0/4015712103 conn(0x7f019c062c60 msgr2=0x7f019c161950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:51.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:51.555+0000 7f01a103c640 1 -- 172.21.15.27:0/4015712103 shutdown_connections 2024-09-12T03:23:51.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:51.555+0000 7f01a103c640 1 -- 172.21.15.27:0/4015712103 wait complete. 2024-09-12T03:23:51.567 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-12T03:23:51.669 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:51.669+0000 7efcc4dc4640 1 Processor -- start 2024-09-12T03:23:51.669 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:51.669+0000 7efcc4dc4640 1 -- start start 2024-09-12T03:23:51.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:51.670+0000 7efcc4dc4640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7efcc00b1ca0 0x7efcc015f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:51.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:51.670+0000 7efcc4dc4640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7efcc006f360 0x7efcc00cef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:51.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:51.670+0000 7efcc4dc4640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7efcc00cf490 0x7efcc00d2780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:51.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:51.670+0000 7efcc4dc4640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7efcc00d22f0 con 0x7efcc00b1ca0 2024-09-12T03:23:51.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:51.670+0000 7efcc4dc4640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7efcc015fd70 con 0x7efcc00cf490 2024-09-12T03:23:51.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:51.670+0000 7efcc4dc4640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7efcc006cc40 con 0x7efcc006f360 2024-09-12T03:23:51.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:51.670+0000 7efcbe59c640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7efcc00b1ca0 0x7efcc015f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:51.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:51.670+0000 7efcbe59c640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7efcc00b1ca0 0x7efcc015f830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:53412/0 (socket says 172.21.15.27:53412) 2024-09-12T03:23:51.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:51.670+0000 7efcbe59c640 1 -- 172.21.15.27:0/2061996347 learned_addr learned my addr 172.21.15.27:0/2061996347 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:23:51.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:51.670+0000 7efcbe59c640 1 -- 172.21.15.27:0/2061996347 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7efcc006f360 msgr2=0x7efcc00cef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:51.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:51.670+0000 7efcbed9d640 1 --2- 172.21.15.27:0/2061996347 >> v2:172.21.15.45:3300/0 conn(0x7efcc00cf490 0x7efcc00d2780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:51.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:51.670+0000 7efcb7fff640 1 --2- 172.21.15.27:0/2061996347 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7efcc006f360 0x7efcc00cef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:51.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:51.670+0000 7efcbe59c640 1 --2- 172.21.15.27:0/2061996347 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7efcc006f360 0x7efcc00cef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:51.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:51.670+0000 7efcbe59c640 1 -- 172.21.15.27:0/2061996347 >> v2:172.21.15.45:3300/0 conn(0x7efcc00cf490 msgr2=0x7efcc00d2780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:51.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:51.670+0000 7efcbe59c640 1 --2- 172.21.15.27:0/2061996347 >> v2:172.21.15.45:3300/0 conn(0x7efcc00cf490 0x7efcc00d2780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:51.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:51.670+0000 7efcbe59c640 1 -- 172.21.15.27:0/2061996347 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcc00d1ce0 con 0x7efcc00b1ca0 2024-09-12T03:23:51.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:51.671+0000 7efcbe59c640 1 --2- 172.21.15.27:0/2061996347 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efcc00b1ca0 0x7efcc015f830 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7efcc0073250 tx=0x7efcb002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc5d67c33f1bd76b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:51.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:51.671+0000 7efcbdd9b640 1 -- 172.21.15.27:0/2061996347 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcb0037070 con 0x7efcc00b1ca0 2024-09-12T03:23:51.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:51.671+0000 7efcbdd9b640 1 -- 172.21.15.27:0/2061996347 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efcb0032440 con 0x7efcc00b1ca0 2024-09-12T03:23:51.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:51.671+0000 7efcbdd9b640 1 -- 172.21.15.27:0/2061996347 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcb0032810 con 0x7efcc00b1ca0 2024-09-12T03:23:51.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:51.672+0000 7efcc4dc4640 1 -- 172.21.15.27:0/2061996347 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efcc00b1ca0 msgr2=0x7efcc015f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:51.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:51.672+0000 7efcc4dc4640 1 --2- 172.21.15.27:0/2061996347 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efcc00b1ca0 0x7efcc015f830 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7efcc0073250 tx=0x7efcb002a1e0 comp rx=0 tx=0).stop 2024-09-12T03:23:51.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:51.672+0000 7efcc4dc4640 1 -- 172.21.15.27:0/2061996347 shutdown_connections 2024-09-12T03:23:51.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:51.672+0000 7efcc4dc4640 1 --2- 172.21.15.27:0/2061996347 >> v2:172.21.15.45:3300/0 conn(0x7efcc00cf490 0x7efcc00d2780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:51.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:51.672+0000 7efcc4dc4640 1 --2- 172.21.15.27:0/2061996347 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7efcc006f360 0x7efcc00cef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:51.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:51.672+0000 7efcc4dc4640 1 --2- 172.21.15.27:0/2061996347 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efcc00b1ca0 0x7efcc015f830 unknown :-1 s=CLOSED pgs=1657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:51.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:51.672+0000 7efcc4dc4640 1 -- 172.21.15.27:0/2061996347 >> 172.21.15.27:0/2061996347 conn(0x7efcc0073800 msgr2=0x7efcc009ba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:51.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:51.672+0000 7efcc4dc4640 1 -- 172.21.15.27:0/2061996347 shutdown_connections 2024-09-12T03:23:51.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:51.672+0000 7efcc4dc4640 1 -- 172.21.15.27:0/2061996347 wait complete. 2024-09-12T03:23:51.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:51.673+0000 7efcc4dc4640 1 Processor -- start 2024-09-12T03:23:51.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:51.673+0000 7efcc4dc4640 1 -- start start 2024-09-12T03:23:51.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:51.674+0000 7efcc4dc4640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7efcc006f360 0x7efcc01eaa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:51.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:51.674+0000 7efcc4dc4640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efcc00b1ca0 0x7efcc0157c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:51.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:51.674+0000 7efcc4dc4640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efcc00cf490 0x7efcc00b6520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:51.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:51.674+0000 7efcc4dc4640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7efcc006d1c0 con 0x7efcc00cf490 2024-09-12T03:23:51.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:51.674+0000 7efcc4dc4640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7efcc00d1f80 con 0x7efcc00b1ca0 2024-09-12T03:23:51.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:51.674+0000 7efcc4dc4640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7efcc006d340 con 0x7efcc006f360 2024-09-12T03:23:51.677 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:51.674+0000 7efcbed9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efcc00cf490 0x7efcc00b6520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:51.677 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:51.674+0000 7efcbed9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efcc00cf490 0x7efcc00b6520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:53428/0 (socket says 172.21.15.27:53428) 2024-09-12T03:23:51.677 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:51.674+0000 7efcbed9d640 1 -- 172.21.15.27:0/4051084765 learned_addr learned my addr 172.21.15.27:0/4051084765 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:23:51.677 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:51.674+0000 7efcbe59c640 1 --2- 172.21.15.27:0/4051084765 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7efcc006f360 0x7efcc01eaa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:51.677 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:51.674+0000 7efcb7fff640 1 --2- 172.21.15.27:0/4051084765 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efcc00b1ca0 0x7efcc0157c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:51.677 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:51.674+0000 7efcbed9d640 1 -- 172.21.15.27:0/4051084765 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7efcc006f360 msgr2=0x7efcc01eaa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:51.677 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:51.674+0000 7efcbed9d640 1 --2- 172.21.15.27:0/4051084765 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7efcc006f360 0x7efcc01eaa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:51.677 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:51.674+0000 7efcbed9d640 1 -- 172.21.15.27:0/4051084765 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efcc00b1ca0 msgr2=0x7efcc0157c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:51.677 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:51.674+0000 7efcbed9d640 1 --2- 172.21.15.27:0/4051084765 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efcc00b1ca0 0x7efcc0157c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:51.677 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:51.674+0000 7efcbed9d640 1 -- 172.21.15.27:0/4051084765 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcc015e2d0 con 0x7efcc00cf490 2024-09-12T03:23:51.677 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:51.674+0000 7efcbed9d640 1 --2- 172.21.15.27:0/4051084765 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efcc00cf490 0x7efcc00b6520 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7efcb800cc30 tx=0x7efcb800a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:51.677 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:51.675+0000 7efcb6ffd640 1 -- 172.21.15.27:0/4051084765 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcb800ce40 con 0x7efcc00cf490 2024-09-12T03:23:51.677 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:51.675+0000 7efcb6ffd640 1 -- 172.21.15.27:0/4051084765 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efcb8003990 con 0x7efcc00cf490 2024-09-12T03:23:51.677 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:51.675+0000 7efcc4dc4640 1 -- 172.21.15.27:0/4051084765 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efcc015e570 con 0x7efcc00cf490 2024-09-12T03:23:51.678 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:51.675+0000 7efcb6ffd640 1 -- 172.21.15.27:0/4051084765 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcb8003d40 con 0x7efcc00cf490 2024-09-12T03:23:51.678 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:51.675+0000 7efcc4dc4640 1 -- 172.21.15.27:0/4051084765 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efcc01581a0 con 0x7efcc00cf490 2024-09-12T03:23:51.679 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:51.676+0000 7efcc4dc4640 1 -- 172.21.15.27:0/4051084765 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efcc00cfc40 con 0x7efcc00cf490 2024-09-12T03:23:51.679 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:51.676+0000 7efcb6ffd640 1 -- 172.21.15.27:0/4051084765 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 25) ==== 91365+0+0 (secure 0 0 0) 0x7efcb8003ee0 con 0x7efcc00cf490 2024-09-12T03:23:51.679 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:51.677+0000 7efcb6ffd640 1 --2- 172.21.15.27:0/4051084765 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7efca006c7c0 0x7efca006ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:51.679 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:51.677+0000 7efcb6ffd640 1 -- 172.21.15.27:0/4051084765 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 4913+0+0 (secure 0 0 0) 0x7efcb80899d0 con 0x7efcc00cf490 2024-09-12T03:23:51.679 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:51.677+0000 7efcbe59c640 1 --2- 172.21.15.27:0/4051084765 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7efca006c7c0 0x7efca006ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:51.679 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:51.677+0000 7efcbe59c640 1 --2- 172.21.15.27:0/4051084765 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7efca006c7c0 0x7efca006ec80 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7efcb0034000 tx=0x7efcb002e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:51.681 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:51.681+0000 7efcb6ffd640 1 -- 172.21.15.27:0/4051084765 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7efcb8014030 con 0x7efcc00cf490 2024-09-12T03:23:51.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:51.922+0000 7efcc4dc4640 1 -- 172.21.15.27:0/4051084765 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7efcc00ceb40 con 0x7efcc00cf490 2024-09-12T03:23:51.923 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:51.923+0000 7efcb6ffd640 1 -- 172.21.15.27:0/4051084765 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v22) ==== 80+0+5304 (secure 0 0 0) 0x7efcb8059930 con 0x7efcc00cf490 2024-09-12T03:23:51.923 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:23:51.923 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-12T03:23:51.923 INFO:teuthology.orchestra.run.smithi027.stdout: "auth_dump": [ 2024-09-12T03:23:51.923 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:23:51.923 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "mds.a", 2024-09-12T03:23:51.923 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDUXOJmIhxjNxAAaGg8SwtW6NRe/lIgNCV20w==", 2024-09-12T03:23:51.923 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:23:51.924 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow", 2024-09-12T03:23:51.924 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow *", 2024-09-12T03:23:51.924 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow *", 2024-09-12T03:23:51.924 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2024-09-12T03:23:51.924 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:23:51.924 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:23:51.924 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:23:51.924 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "mds.b", 2024-09-12T03:23:51.924 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDVXOJm63kWCBAAzjfOsBsWUI+IAsuZ49yEkw==", 2024-09-12T03:23:51.924 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:23:51.924 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow", 2024-09-12T03:23:51.924 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow *", 2024-09-12T03:23:51.924 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow *", 2024-09-12T03:23:51.925 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2024-09-12T03:23:51.925 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:23:51.925 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:23:51.925 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:23:51.925 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "mds.c", 2024-09-12T03:23:51.925 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDVXOJm0sX7AhAAdynCcho2a/AgFF9Cw3+vAw==", 2024-09-12T03:23:51.925 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:23:51.925 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow", 2024-09-12T03:23:51.925 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow *", 2024-09-12T03:23:51.925 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow *", 2024-09-12T03:23:51.925 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2024-09-12T03:23:51.925 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:23:51.926 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:23:51.926 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:23:51.926 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "mds.d", 2024-09-12T03:23:51.926 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDVXOJm7Dd8DxAANHYmvKgpRZ6sVfXDSgPHWw==", 2024-09-12T03:23:51.926 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:23:51.926 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow", 2024-09-12T03:23:51.926 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow *", 2024-09-12T03:23:51.926 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow *", 2024-09-12T03:23:51.926 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2024-09-12T03:23:51.926 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:23:51.926 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:23:51.926 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:23:51.927 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "osd.0", 2024-09-12T03:23:51.927 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDYXOJmJTVYLxAAmo8kxRjaiwTT44dZLZlq4A==", 2024-09-12T03:23:51.927 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:23:51.927 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow profile osd", 2024-09-12T03:23:51.927 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile osd", 2024-09-12T03:23:51.927 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2024-09-12T03:23:51.927 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:23:51.927 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:23:51.927 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:23:51.927 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "osd.1", 2024-09-12T03:23:51.927 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDcXOJmYV0BFBAAmoGX9gf9yLoE/qkWFnv9tg==", 2024-09-12T03:23:51.927 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:23:51.927 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow profile osd", 2024-09-12T03:23:51.928 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile osd", 2024-09-12T03:23:51.928 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2024-09-12T03:23:51.928 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:23:51.928 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:23:51.928 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:23:51.928 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "osd.2", 2024-09-12T03:23:51.928 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDfXOJmEetiJRAAXk+AvfTfhjsB6T2UB4Z2/w==", 2024-09-12T03:23:51.928 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:23:51.928 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow profile osd", 2024-09-12T03:23:51.928 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile osd", 2024-09-12T03:23:51.928 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2024-09-12T03:23:51.928 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:23:51.928 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:23:51.929 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:23:51.929 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "osd.3", 2024-09-12T03:23:51.929 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDiXOJmrQbPJxAAQsi2B5a1LnPTPGZVdo7j7w==", 2024-09-12T03:23:51.929 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:23:51.929 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow profile osd", 2024-09-12T03:23:51.929 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile osd", 2024-09-12T03:23:51.929 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2024-09-12T03:23:51.929 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:23:51.929 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:23:51.930 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:23:51.930 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "osd.4", 2024-09-12T03:23:51.930 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDnXOJmoMr1NxAAv3u2q8hYt9w//9gz+N7UAQ==", 2024-09-12T03:23:51.930 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:23:51.930 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow profile osd", 2024-09-12T03:23:51.930 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile osd", 2024-09-12T03:23:51.930 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2024-09-12T03:23:51.930 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:23:51.930 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:23:51.930 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:23:51.930 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "osd.5", 2024-09-12T03:23:51.930 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDqXOJmxpUPOxAAxLH6RJsyLOZ2XMfzM0fQ1g==", 2024-09-12T03:23:51.930 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:23:51.931 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow profile osd", 2024-09-12T03:23:51.931 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile osd", 2024-09-12T03:23:51.931 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2024-09-12T03:23:51.931 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:23:51.931 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:23:51.931 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:23:51.931 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "osd.6", 2024-09-12T03:23:51.931 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDuXOJmuP0+LxAAMHoz5eSO81hPtkdBIS2cig==", 2024-09-12T03:23:51.931 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:23:51.931 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow profile osd", 2024-09-12T03:23:51.931 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile osd", 2024-09-12T03:23:51.931 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2024-09-12T03:23:51.931 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:23:51.931 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:23:51.931 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:23:51.932 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "osd.7", 2024-09-12T03:23:51.932 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDxXOJmuIC/MRAAPDM+lAHqdoQyGVODwjFp+Q==", 2024-09-12T03:23:51.932 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:23:51.932 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow profile osd", 2024-09-12T03:23:51.932 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile osd", 2024-09-12T03:23:51.932 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2024-09-12T03:23:51.932 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:23:51.932 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:23:51.932 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:23:51.932 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "client.0", 2024-09-12T03:23:51.932 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDVXOJmLm+5ExAAECM4CKfuVElmp/n0b02mXg==", 2024-09-12T03:23:51.932 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:23:51.932 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow", 2024-09-12T03:23:51.932 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow r", 2024-09-12T03:23:51.933 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-12T03:23:51.933 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:23:51.933 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:23:51.933 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:23:51.933 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "client.1", 2024-09-12T03:23:51.933 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDVXOJm61/FHBAALCQouXySfQeZ5DESGmqkzA==", 2024-09-12T03:23:51.933 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:23:51.933 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow", 2024-09-12T03:23:51.933 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow r", 2024-09-12T03:23:51.933 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow rw", 2024-09-12T03:23:51.933 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow rwx" 2024-09-12T03:23:51.933 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:23:51.933 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:23:51.934 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:23:51.934 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "client.2", 2024-09-12T03:23:51.934 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDVXOJmq/ZkGRAAOzmQzEIhb4b/YTxayc34Bw==", 2024-09-12T03:23:51.934 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:23:51.934 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow", 2024-09-12T03:23:51.934 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow r", 2024-09-12T03:23:51.934 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow rw", 2024-09-12T03:23:51.934 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow rwx" 2024-09-12T03:23:51.934 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:23:51.934 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:23:51.934 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:23:51.934 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "client.3", 2024-09-12T03:23:51.934 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDVXOJm+DlzIhAATQKZe+Zyk0MM8efTa6g9PA==", 2024-09-12T03:23:51.934 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:23:51.934 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow", 2024-09-12T03:23:51.935 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow r", 2024-09-12T03:23:51.935 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow rw", 2024-09-12T03:23:51.935 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow rwx" 2024-09-12T03:23:51.935 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:23:51.935 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:23:51.935 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:23:51.935 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "client.admin", 2024-09-12T03:23:51.935 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDUXOJm+JGWGhAApTQw9RtJWd/J5QzQaUFXvg==", 2024-09-12T03:23:51.935 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:23:51.935 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow *", 2024-09-12T03:23:51.935 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow *", 2024-09-12T03:23:51.935 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow *", 2024-09-12T03:23:51.936 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2024-09-12T03:23:51.936 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:23:51.936 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:23:51.936 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:23:51.936 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "mgr.x", 2024-09-12T03:23:51.936 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDUXOJmZcsONBAAMTY2KykuHaEkv1t6Vz5KjA==", 2024-09-12T03:23:51.936 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:23:51.936 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow *", 2024-09-12T03:23:51.936 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile mgr", 2024-09-12T03:23:51.936 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2024-09-12T03:23:51.936 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:23:51.936 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-12T03:23:51.936 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:23:51.937 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "mgr.y", 2024-09-12T03:23:51.937 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQDUXOJmJOgZMBAAQ/JTxfhox7HgxBMtsvUCBg==", 2024-09-12T03:23:51.937 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2024-09-12T03:23:51.937 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow *", 2024-09-12T03:23:51.937 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile mgr", 2024-09-12T03:23:51.937 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2024-09-12T03:23:51.937 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:23:51.937 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:23:51.937 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-12T03:23:51.937 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-12T03:23:51.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:51.925+0000 7efcc4dc4640 1 -- 172.21.15.27:0/4051084765 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7efca006c7c0 msgr2=0x7efca006ec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:51.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:51.925+0000 7efcc4dc4640 1 --2- 172.21.15.27:0/4051084765 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7efca006c7c0 0x7efca006ec80 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7efcb0034000 tx=0x7efcb002e000 comp rx=0 tx=0).stop 2024-09-12T03:23:51.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:51.925+0000 7efcc4dc4640 1 -- 172.21.15.27:0/4051084765 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efcc00cf490 msgr2=0x7efcc00b6520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:51.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:51.925+0000 7efcc4dc4640 1 --2- 172.21.15.27:0/4051084765 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efcc00cf490 0x7efcc00b6520 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7efcb800cc30 tx=0x7efcb800a5c0 comp rx=0 tx=0).stop 2024-09-12T03:23:51.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:51.925+0000 7efcc4dc4640 1 -- 172.21.15.27:0/4051084765 shutdown_connections 2024-09-12T03:23:51.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:51.925+0000 7efcc4dc4640 1 --2- 172.21.15.27:0/4051084765 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7efca006c7c0 0x7efca006ec80 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:51.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:51.925+0000 7efcc4dc4640 1 --2- 172.21.15.27:0/4051084765 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efcc00cf490 0x7efcc00b6520 unknown :-1 s=CLOSED pgs=1658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:51.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:51.925+0000 7efcc4dc4640 1 --2- 172.21.15.27:0/4051084765 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efcc00b1ca0 0x7efcc0157c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:51.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:51.925+0000 7efcc4dc4640 1 --2- 172.21.15.27:0/4051084765 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7efcc006f360 0x7efcc01eaa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:51.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:51.925+0000 7efcc4dc4640 1 -- 172.21.15.27:0/4051084765 >> 172.21.15.27:0/4051084765 conn(0x7efcc0073800 msgr2=0x7efcc00701c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:51.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:51.925+0000 7efcc4dc4640 1 -- 172.21.15.27:0/4051084765 shutdown_connections 2024-09-12T03:23:51.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:51.925+0000 7efcc4dc4640 1 -- 172.21.15.27:0/4051084765 wait complete. 2024-09-12T03:23:51.940 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-12T03:23:52.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:52.042+0000 7f3380d86640 1 Processor -- start 2024-09-12T03:23:52.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:52.042+0000 7f3380d86640 1 -- start start 2024-09-12T03:23:52.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:52.043+0000 7f3380d86640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f337c0ce6d0 0x7f337c0ceab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:52.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:52.043+0000 7f3380d86640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f337c1463d0 con 0x7f337c06c240 2024-09-12T03:23:52.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:52.043+0000 7f3380d86640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f337c0ceff0 con 0x7f337c160d80 2024-09-12T03:23:52.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:52.043+0000 7f3380d86640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f337c06c7f0 con 0x7f337c0ce6d0 2024-09-12T03:23:52.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:52.043+0000 7f337a59c640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f337c0ce6d0 0x7f337c0ceab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:52.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:52.043+0000 7f337ad9d640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f337c06c240 0x7f337c06e5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:33904/0 (socket says 172.21.15.27:33904) 2024-09-12T03:23:52.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:52.043+0000 7f337ad9d640 1 -- 172.21.15.27:0/1304049082 learned_addr learned my addr 172.21.15.27:0/1304049082 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:23:52.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:52.044+0000 7f337a59c640 1 -- 172.21.15.27:0/1304049082 >> v1:172.21.15.45:6789/0 conn(0x7f337c160d80 legacy=0x7f337c06bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:52.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:52.044+0000 7f337a59c640 1 -- 172.21.15.27:0/1304049082 >> v1:172.21.15.27:6789/0 conn(0x7f337c06c240 legacy=0x7f337c06e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:52.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:52.044+0000 7f337a59c640 1 -- 172.21.15.27:0/1304049082 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f337c1602a0 con 0x7f337c0ce6d0 2024-09-12T03:23:52.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:52.044+0000 7f337a59c640 1 --2- 172.21.15.27:0/1304049082 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f337c0ce6d0 0x7f337c0ceab0 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7f336c0097a0 tx=0x7f336c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84a5c4692b6ed4f5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:52.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:52.045+0000 7f337959a640 1 -- 172.21.15.27:0/1304049082 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f336c037070 con 0x7f337c0ce6d0 2024-09-12T03:23:52.046 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:52.045+0000 7f337959a640 1 -- 172.21.15.27:0/1304049082 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f336c032440 con 0x7f337c0ce6d0 2024-09-12T03:23:52.046 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:52.045+0000 7f337959a640 1 -- 172.21.15.27:0/1304049082 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f336c032810 con 0x7f337c0ce6d0 2024-09-12T03:23:52.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:52.045+0000 7f3380d86640 1 -- 172.21.15.27:0/1304049082 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f337c0ce6d0 msgr2=0x7f337c0ceab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:52.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:52.045+0000 7f3380d86640 1 --2- 172.21.15.27:0/1304049082 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f337c0ce6d0 0x7f337c0ceab0 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7f336c0097a0 tx=0x7f336c02a1a0 comp rx=0 tx=0).stop 2024-09-12T03:23:52.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:52.045+0000 7f3380d86640 1 -- 172.21.15.27:0/1304049082 shutdown_connections 2024-09-12T03:23:52.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:52.045+0000 7f3380d86640 1 --2- 172.21.15.27:0/1304049082 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f337c0ce6d0 0x7f337c0ceab0 unknown :-1 s=CLOSED pgs=1660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:52.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:52.045+0000 7f3380d86640 1 -- 172.21.15.27:0/1304049082 >> 172.21.15.27:0/1304049082 conn(0x7f337c06cc40 msgr2=0x7f337c06d050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:52.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:52.045+0000 7f3380d86640 1 -- 172.21.15.27:0/1304049082 shutdown_connections 2024-09-12T03:23:52.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:52.045+0000 7f3380d86640 1 -- 172.21.15.27:0/1304049082 wait complete. 2024-09-12T03:23:52.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:52.047+0000 7f3380d86640 1 Processor -- start 2024-09-12T03:23:52.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:52.047+0000 7f3380d86640 1 -- start start 2024-09-12T03:23:52.050 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:52.047+0000 7f3380d86640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f337c06c240 0x7f337c1410f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:52.050 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:52.048+0000 7f3380d86640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f337c0ce6d0 0x7f337c135fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:52.050 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:52.048+0000 7f3380d86640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f337c160d80 0x7f337c0ae6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:52.050 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:52.048+0000 7f3380d86640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f337c06ecc0 con 0x7f337c0ce6d0 2024-09-12T03:23:52.050 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:52.048+0000 7f3380d86640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f337c160860 con 0x7f337c160d80 2024-09-12T03:23:52.050 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:52.048+0000 7f3380d86640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f337c06ee40 con 0x7f337c06c240 2024-09-12T03:23:52.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:52.048+0000 7f3379d9b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f337c0ce6d0 0x7f337c135fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:52.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:52.048+0000 7f3379d9b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f337c0ce6d0 0x7f337c135fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:53444/0 (socket says 172.21.15.27:53444) 2024-09-12T03:23:52.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:52.048+0000 7f3379d9b640 1 -- 172.21.15.27:0/540170875 learned_addr learned my addr 172.21.15.27:0/540170875 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:23:52.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:52.048+0000 7f337a59c640 1 --2- 172.21.15.27:0/540170875 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f337c06c240 0x7f337c1410f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:52.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:52.048+0000 7f337ad9d640 1 --2- 172.21.15.27:0/540170875 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f337c160d80 0x7f337c0ae6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:52.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:52.048+0000 7f3379d9b640 1 -- 172.21.15.27:0/540170875 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f337c06c240 msgr2=0x7f337c1410f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:52.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:52.048+0000 7f3379d9b640 1 --2- 172.21.15.27:0/540170875 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f337c06c240 0x7f337c1410f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:52.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:52.048+0000 7f3379d9b640 1 -- 172.21.15.27:0/540170875 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f337c160d80 msgr2=0x7f337c0ae6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:52.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:52.048+0000 7f3379d9b640 1 --2- 172.21.15.27:0/540170875 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f337c160d80 0x7f337c0ae6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:52.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:52.048+0000 7f3379d9b640 1 -- 172.21.15.27:0/540170875 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f337c136710 con 0x7f337c0ce6d0 2024-09-12T03:23:52.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:52.048+0000 7f337ad9d640 1 --2- 172.21.15.27:0/540170875 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f337c160d80 0x7f337c0ae6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:23:52.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:52.048+0000 7f337a59c640 1 --2- 172.21.15.27:0/540170875 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f337c06c240 0x7f337c1410f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:23:52.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:52.048+0000 7f3379d9b640 1 --2- 172.21.15.27:0/540170875 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f337c0ce6d0 0x7f337c135fa0 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7f337000c3c0 tx=0x7f337000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:52.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:52.048+0000 7f336affd640 1 -- 172.21.15.27:0/540170875 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33700052d0 con 0x7f337c0ce6d0 2024-09-12T03:23:52.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:52.048+0000 7f336affd640 1 -- 172.21.15.27:0/540170875 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3370005490 con 0x7f337c0ce6d0 2024-09-12T03:23:52.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:52.048+0000 7f3380d86640 1 -- 172.21.15.27:0/540170875 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f337c1369f0 con 0x7f337c0ce6d0 2024-09-12T03:23:52.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:52.048+0000 7f336affd640 1 -- 172.21.15.27:0/540170875 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3370007260 con 0x7f337c0ce6d0 2024-09-12T03:23:52.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:52.048+0000 7f3380d86640 1 -- 172.21.15.27:0/540170875 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f337c1f2ba0 con 0x7f337c0ce6d0 2024-09-12T03:23:52.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:52.049+0000 7f3380d86640 1 -- 172.21.15.27:0/540170875 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f337c06f1d0 con 0x7f337c0ce6d0 2024-09-12T03:23:52.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:52.050+0000 7f336affd640 1 -- 172.21.15.27:0/540170875 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 25) ==== 91365+0+0 (secure 0 0 0) 0x7f3370007400 con 0x7f337c0ce6d0 2024-09-12T03:23:52.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:52.050+0000 7f336affd640 1 --2- 172.21.15.27:0/540170875 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f335c06c7c0 0x7f335c06ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:52.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:52.051+0000 7f337a59c640 1 --2- 172.21.15.27:0/540170875 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f335c06c7c0 0x7f335c06ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:52.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:52.051+0000 7f336affd640 1 -- 172.21.15.27:0/540170875 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 4913+0+0 (secure 0 0 0) 0x7f3370088aa0 con 0x7f337c0ce6d0 2024-09-12T03:23:52.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:52.051+0000 7f337a59c640 1 --2- 172.21.15.27:0/540170875 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f335c06c7c0 0x7f335c06ec80 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f336c034000 tx=0x7f336c02e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:52.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:52.055+0000 7f336affd640 1 -- 172.21.15.27:0/540170875 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f3370059050 con 0x7f337c0ce6d0 2024-09-12T03:23:52.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:52.234+0000 7f3380d86640 1 -- 172.21.15.27:0/540170875 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f337c06e390 con 0x7f337c0ce6d0 2024-09-12T03:23:52.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:52.236+0000 7f336affd640 1 -- 172.21.15.27:0/540170875 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v156) ==== 84+0+65 (secure 0 0 0) 0x7f3370058950 con 0x7f337c0ce6d0 2024-09-12T03:23:52.236 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:23:52.236 INFO:teuthology.orchestra.run.smithi027.stdout:[ 2024-09-12T03:23:52.236 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:23:52.236 INFO:teuthology.orchestra.run.smithi027.stdout: "poolnum": 1, 2024-09-12T03:23:52.236 INFO:teuthology.orchestra.run.smithi027.stdout: "poolname": ".mgr" 2024-09-12T03:23:52.236 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:23:52.236 INFO:teuthology.orchestra.run.smithi027.stdout:] 2024-09-12T03:23:52.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:52.238+0000 7f3380d86640 1 -- 172.21.15.27:0/540170875 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f335c06c7c0 msgr2=0x7f335c06ec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:52.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:52.238+0000 7f3380d86640 1 --2- 172.21.15.27:0/540170875 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f335c06c7c0 0x7f335c06ec80 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f336c034000 tx=0x7f336c02e000 comp rx=0 tx=0).stop 2024-09-12T03:23:52.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:52.238+0000 7f3380d86640 1 -- 172.21.15.27:0/540170875 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f337c0ce6d0 msgr2=0x7f337c135fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:52.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:52.238+0000 7f3380d86640 1 --2- 172.21.15.27:0/540170875 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f337c0ce6d0 0x7f337c135fa0 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7f337000c3c0 tx=0x7f337000c880 comp rx=0 tx=0).stop 2024-09-12T03:23:52.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:52.238+0000 7f3380d86640 1 -- 172.21.15.27:0/540170875 shutdown_connections 2024-09-12T03:23:52.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:52.238+0000 7f3380d86640 1 --2- 172.21.15.27:0/540170875 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f337c160d80 0x7f337c0ae6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:52.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:52.238+0000 7f3380d86640 1 --2- 172.21.15.27:0/540170875 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f335c06c7c0 0x7f335c06ec80 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:52.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:52.238+0000 7f3380d86640 1 --2- 172.21.15.27:0/540170875 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f337c0ce6d0 0x7f337c135fa0 unknown :-1 s=CLOSED pgs=1661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:52.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:52.238+0000 7f3380d86640 1 --2- 172.21.15.27:0/540170875 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f337c06c240 0x7f337c1410f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:52.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:52.238+0000 7f3380d86640 1 -- 172.21.15.27:0/540170875 >> 172.21.15.27:0/540170875 conn(0x7f337c06cc40 msgr2=0x7f337c141cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:52.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:52.238+0000 7f3380d86640 1 -- 172.21.15.27:0/540170875 shutdown_connections 2024-09-12T03:23:52.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:52.238+0000 7f3380d86640 1 -- 172.21.15.27:0/540170875 wait complete. 2024-09-12T03:23:52.249 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-12T03:23:52.249 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-12T03:23:52.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:52.355+0000 7ff73d0f8640 1 Processor -- start 2024-09-12T03:23:52.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:52.355+0000 7ff73d0f8640 1 -- start start 2024-09-12T03:23:52.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:52.356+0000 7ff73d0f8640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ff738162040 0x7ff7380d1da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:52.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:52.356+0000 7ff73d0f8640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7ff7380d3790 0x7ff7380b7a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:52.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:52.356+0000 7ff73d0f8640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff7380d2e30 con 0x7ff73806d8c0 2024-09-12T03:23:52.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:52.356+0000 7ff73d0f8640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7ff7380b7f90 con 0x7ff7380d3790 2024-09-12T03:23:52.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:52.356+0000 7ff73d0f8640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7ff738162420 con 0x7ff738162040 2024-09-12T03:23:52.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:52.356+0000 7ff73659c640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7ff7380d3790 0x7ff7380b7a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:52.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:52.356+0000 7ff73759e640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7ff73806d8c0 0x7ff7380cf320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:33906/0 (socket says 172.21.15.27:33906) 2024-09-12T03:23:52.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:52.356+0000 7ff73759e640 1 -- 172.21.15.27:0/1340661874 learned_addr learned my addr 172.21.15.27:0/1340661874 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:23:52.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:52.356+0000 7ff736d9d640 1 --2- 172.21.15.27:0/1340661874 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ff738162040 0x7ff7380d1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:52.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:52.356+0000 7ff73659c640 1 -- 172.21.15.27:0/1340661874 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ff738162040 msgr2=0x7ff7380d1da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:52.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:52.356+0000 7ff73659c640 1 --2- 172.21.15.27:0/1340661874 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ff738162040 0x7ff7380d1da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:52.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:52.356+0000 7ff73659c640 1 -- 172.21.15.27:0/1340661874 >> v1:172.21.15.27:6789/0 conn(0x7ff73806d8c0 legacy=0x7ff7380cf320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:52.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:52.357+0000 7ff73659c640 1 -- 172.21.15.27:0/1340661874 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7380d30d0 con 0x7ff7380d3790 2024-09-12T03:23:52.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:52.357+0000 7ff735d9b640 1 -- 172.21.15.27:0/1340661874 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3854334642 0 0) 0x7ff7380d2e30 con 0x7ff73806d8c0 2024-09-12T03:23:52.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:52.357+0000 7ff73659c640 1 --2- 172.21.15.27:0/1340661874 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff7380d3790 0x7ff7380b7a50 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7ff72c0097a0 tx=0x7ff72c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86b56c4a46dd811c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:52.359 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:52.357+0000 7ff735d9b640 1 -- 172.21.15.27:0/1340661874 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff72c037070 con 0x7ff7380d3790 2024-09-12T03:23:52.359 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:52.357+0000 7ff735d9b640 1 -- 172.21.15.27:0/1340661874 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff72c032440 con 0x7ff7380d3790 2024-09-12T03:23:52.359 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:52.357+0000 7ff735d9b640 1 -- 172.21.15.27:0/1340661874 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff72c032810 con 0x7ff7380d3790 2024-09-12T03:23:52.359 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:52.358+0000 7ff73d0f8640 1 -- 172.21.15.27:0/1340661874 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff7380d3790 msgr2=0x7ff7380b7a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:52.359 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:52.358+0000 7ff73d0f8640 1 --2- 172.21.15.27:0/1340661874 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff7380d3790 0x7ff7380b7a50 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7ff72c0097a0 tx=0x7ff72c02a180 comp rx=0 tx=0).stop 2024-09-12T03:23:52.359 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:52.358+0000 7ff73d0f8640 1 -- 172.21.15.27:0/1340661874 shutdown_connections 2024-09-12T03:23:52.359 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:52.358+0000 7ff73d0f8640 1 --2- 172.21.15.27:0/1340661874 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff7380d3790 0x7ff7380b7a50 unknown :-1 s=CLOSED pgs=1663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:52.359 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:52.358+0000 7ff73d0f8640 1 --2- 172.21.15.27:0/1340661874 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ff738162040 0x7ff7380d1da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:52.359 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:52.358+0000 7ff73d0f8640 1 -- 172.21.15.27:0/1340661874 >> 172.21.15.27:0/1340661874 conn(0x7ff7380b6700 msgr2=0x7ff7380b6b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:52.359 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:52.358+0000 7ff73d0f8640 1 -- 172.21.15.27:0/1340661874 shutdown_connections 2024-09-12T03:23:52.359 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:52.358+0000 7ff73d0f8640 1 -- 172.21.15.27:0/1340661874 wait complete. 2024-09-12T03:23:52.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:52.359+0000 7ff73d0f8640 1 Processor -- start 2024-09-12T03:23:52.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:52.360+0000 7ff73d0f8640 1 -- start start 2024-09-12T03:23:52.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:52.360+0000 7ff73d0f8640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff73806d8c0 0x7ff738160cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:52.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:52.360+0000 7ff73d0f8640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff7380d3790 0x7ff738153cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:52.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:52.360+0000 7ff73d0f8640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ff738162040 0x7ff73815ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:52.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:52.360+0000 7ff73d0f8640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7ff7380d4530 con 0x7ff73806d8c0 2024-09-12T03:23:52.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:52.360+0000 7ff73d0f8640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ff7380cee00 con 0x7ff7380d3790 2024-09-12T03:23:52.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:52.360+0000 7ff73d0f8640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7ff7380d4150 con 0x7ff738162040 2024-09-12T03:23:52.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:52.360+0000 7ff736d9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff73806d8c0 0x7ff738160cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:52.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:52.360+0000 7ff736d9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff73806d8c0 0x7ff738160cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:53458/0 (socket says 172.21.15.27:53458) 2024-09-12T03:23:52.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:52.360+0000 7ff736d9d640 1 -- 172.21.15.27:0/2033041027 learned_addr learned my addr 172.21.15.27:0/2033041027 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:23:52.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:52.361+0000 7ff73759e640 1 --2- 172.21.15.27:0/2033041027 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ff738162040 0x7ff73815ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:52.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:52.361+0000 7ff73659c640 1 --2- 172.21.15.27:0/2033041027 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff7380d3790 0x7ff738153cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:52.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:52.361+0000 7ff736d9d640 1 -- 172.21.15.27:0/2033041027 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ff738162040 msgr2=0x7ff73815ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:52.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:52.361+0000 7ff736d9d640 1 --2- 172.21.15.27:0/2033041027 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ff738162040 0x7ff73815ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:52.363 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:52.361+0000 7ff736d9d640 1 -- 172.21.15.27:0/2033041027 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff7380d3790 msgr2=0x7ff738153cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:52.363 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:52.361+0000 7ff736d9d640 1 --2- 172.21.15.27:0/2033041027 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff7380d3790 0x7ff738153cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:52.363 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:52.361+0000 7ff736d9d640 1 -- 172.21.15.27:0/2033041027 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff73815ec00 con 0x7ff73806d8c0 2024-09-12T03:23:52.363 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:52.361+0000 7ff736d9d640 1 --2- 172.21.15.27:0/2033041027 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff73806d8c0 0x7ff738160cb0 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7ff72800d800 tx=0x7ff72800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:52.363 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:52.361+0000 7ff71f7fe640 1 -- 172.21.15.27:0/2033041027 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff72800b6c0 con 0x7ff73806d8c0 2024-09-12T03:23:52.363 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:52.361+0000 7ff71f7fe640 1 -- 172.21.15.27:0/2033041027 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff728019070 con 0x7ff73806d8c0 2024-09-12T03:23:52.363 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:52.361+0000 7ff73d0f8640 1 -- 172.21.15.27:0/2033041027 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7381611f0 con 0x7ff73806d8c0 2024-09-12T03:23:52.363 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:52.361+0000 7ff71f7fe640 1 -- 172.21.15.27:0/2033041027 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff72800ba80 con 0x7ff73806d8c0 2024-09-12T03:23:52.363 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:52.361+0000 7ff73d0f8640 1 -- 172.21.15.27:0/2033041027 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff738155cc0 con 0x7ff73806d8c0 2024-09-12T03:23:52.364 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:52.362+0000 7ff73d0f8640 1 -- 172.21.15.27:0/2033041027 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7380b7a50 con 0x7ff73806d8c0 2024-09-12T03:23:52.364 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:52.363+0000 7ff71f7fe640 1 -- 172.21.15.27:0/2033041027 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 25) ==== 91365+0+0 (secure 0 0 0) 0x7ff72800bc20 con 0x7ff73806d8c0 2024-09-12T03:23:52.364 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:52.363+0000 7ff71f7fe640 1 --2- 172.21.15.27:0/2033041027 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7ff72406c7c0 0x7ff72406ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:52.364 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:52.363+0000 7ff71f7fe640 1 -- 172.21.15.27:0/2033041027 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 4913+0+0 (secure 0 0 0) 0x7ff728089a50 con 0x7ff73806d8c0 2024-09-12T03:23:52.364 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:52.363+0000 7ff73659c640 1 --2- 172.21.15.27:0/2033041027 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7ff72406c7c0 0x7ff72406ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:52.365 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:52.364+0000 7ff73659c640 1 --2- 172.21.15.27:0/2033041027 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7ff72406c7c0 0x7ff72406ec80 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7ff72c034000 tx=0x7ff72c02e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:52.367 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:52.367+0000 7ff71f7fe640 1 -- 172.21.15.27:0/2033041027 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7ff72805a000 con 0x7ff73806d8c0 2024-09-12T03:23:52.543 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:52.543+0000 7ff73d0f8640 1 -- 172.21.15.27:0/2033041027 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7ff7381563c0 con 0x7ff73806d8c0 2024-09-12T03:23:53.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:53.239+0000 7ff71f7fe640 1 -- 172.21.15.27:0/2033041027 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v157) ==== 138+0+0 (secure 0 0 0) 0x7ff728059900 con 0x7ff73806d8c0 2024-09-12T03:23:53.239 INFO:teuthology.orchestra.run.smithi027.stderr:pool 'cephfs_metadata' created 2024-09-12T03:23:53.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:53.243+0000 7ff73d0f8640 1 -- 172.21.15.27:0/2033041027 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7ff72406c7c0 msgr2=0x7ff72406ec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:53.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:53.243+0000 7ff73d0f8640 1 --2- 172.21.15.27:0/2033041027 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7ff72406c7c0 0x7ff72406ec80 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7ff72c034000 tx=0x7ff72c02e000 comp rx=0 tx=0).stop 2024-09-12T03:23:53.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:53.243+0000 7ff73d0f8640 1 -- 172.21.15.27:0/2033041027 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff73806d8c0 msgr2=0x7ff738160cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:53.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:53.243+0000 7ff73d0f8640 1 --2- 172.21.15.27:0/2033041027 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff73806d8c0 0x7ff738160cb0 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7ff72800d800 tx=0x7ff72800dcc0 comp rx=0 tx=0).stop 2024-09-12T03:23:53.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:53.243+0000 7ff73d0f8640 1 -- 172.21.15.27:0/2033041027 shutdown_connections 2024-09-12T03:23:53.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:53.243+0000 7ff73d0f8640 1 --2- 172.21.15.27:0/2033041027 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ff738162040 0x7ff73815ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:53.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:53.243+0000 7ff73d0f8640 1 --2- 172.21.15.27:0/2033041027 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff7380d3790 0x7ff738153cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:53.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:53.243+0000 7ff73d0f8640 1 --2- 172.21.15.27:0/2033041027 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7ff72406c7c0 0x7ff72406ec80 secure :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7ff72c034000 tx=0x7ff72c02e000 comp rx=0 tx=0).stop 2024-09-12T03:23:53.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:53.243+0000 7ff73d0f8640 1 --2- 172.21.15.27:0/2033041027 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff73806d8c0 0x7ff738160cb0 unknown :-1 s=CLOSED pgs=1664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:53.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:53.243+0000 7ff73d0f8640 1 -- 172.21.15.27:0/2033041027 >> 172.21.15.27:0/2033041027 conn(0x7ff7380b6700 msgr2=0x7ff73806de60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:53.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:53.244+0000 7ff73d0f8640 1 -- 172.21.15.27:0/2033041027 shutdown_connections 2024-09-12T03:23:53.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:53.244+0000 7ff73d0f8640 1 -- 172.21.15.27:0/2033041027 wait complete. 2024-09-12T03:23:53.261 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-12T03:23:53.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:53.378+0000 7f6131976640 1 Processor -- start 2024-09-12T03:23:53.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:53.378+0000 7f6131976640 1 -- start start 2024-09-12T03:23:53.379 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:53.379+0000 7f6131976640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f612c06dfd0 0x7f612c0d1da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:53.379 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:53.379+0000 7f6131976640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f612c0d3790 0x7f612c0b7a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:53.379 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:53.379+0000 7f6131976640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f612c0d2e30 con 0x7f612c162040 2024-09-12T03:23:53.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:53.379+0000 7f6131976640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f612c0b7f90 con 0x7f612c06dfd0 2024-09-12T03:23:53.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:53.379+0000 7f6131976640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f612c162420 con 0x7f612c0d3790 2024-09-12T03:23:53.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:53.379+0000 7f612a7fc640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f612c06dfd0 0x7f612c0d1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:53.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:53.379+0000 7f612affd640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f612c162040 0x7f612c06d8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:33908/0 (socket says 172.21.15.27:33908) 2024-09-12T03:23:53.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:53.379+0000 7f612affd640 1 -- 172.21.15.27:0/4118052610 learned_addr learned my addr 172.21.15.27:0/4118052610 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:23:53.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:53.379+0000 7f612b7fe640 1 --2- 172.21.15.27:0/4118052610 >> v2:172.21.15.45:3300/0 conn(0x7f612c0d3790 0x7f612c0b7a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:53.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:53.379+0000 7f612a7fc640 1 -- 172.21.15.27:0/4118052610 >> v2:172.21.15.45:3300/0 conn(0x7f612c0d3790 msgr2=0x7f612c0b7a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:53.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:53.379+0000 7f612a7fc640 1 --2- 172.21.15.27:0/4118052610 >> v2:172.21.15.45:3300/0 conn(0x7f612c0d3790 0x7f612c0b7a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:53.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:53.379+0000 7f612a7fc640 1 -- 172.21.15.27:0/4118052610 >> v1:172.21.15.27:6789/0 conn(0x7f612c162040 legacy=0x7f612c06d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:53.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:53.379+0000 7f6129ffb640 1 -- 172.21.15.27:0/4118052610 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3087313712 0 0) 0x7f612c0d2e30 con 0x7f612c162040 2024-09-12T03:23:53.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:53.379+0000 7f612a7fc640 1 -- 172.21.15.27:0/4118052610 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f612c0d30d0 con 0x7f612c06dfd0 2024-09-12T03:23:53.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:53.380+0000 7f612a7fc640 1 --2- 172.21.15.27:0/4118052610 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f612c06dfd0 0x7f612c0d1da0 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7f612c0d1c90 tx=0x7f612002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d95a204df2a26ea server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:53.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:53.380+0000 7f6129ffb640 1 -- 172.21.15.27:0/4118052610 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6120037070 con 0x7f612c06dfd0 2024-09-12T03:23:53.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:53.380+0000 7f6129ffb640 1 -- 172.21.15.27:0/4118052610 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6120032440 con 0x7f612c06dfd0 2024-09-12T03:23:53.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:53.380+0000 7f6129ffb640 1 -- 172.21.15.27:0/4118052610 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6120032810 con 0x7f612c06dfd0 2024-09-12T03:23:53.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:53.381+0000 7f6131976640 1 -- 172.21.15.27:0/4118052610 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f612c06dfd0 msgr2=0x7f612c0d1da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:53.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:53.381+0000 7f6131976640 1 --2- 172.21.15.27:0/4118052610 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f612c06dfd0 0x7f612c0d1da0 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7f612c0d1c90 tx=0x7f612002a1a0 comp rx=0 tx=0).stop 2024-09-12T03:23:53.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:53.381+0000 7f6131976640 1 -- 172.21.15.27:0/4118052610 shutdown_connections 2024-09-12T03:23:53.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:53.381+0000 7f6131976640 1 --2- 172.21.15.27:0/4118052610 >> v2:172.21.15.45:3300/0 conn(0x7f612c0d3790 0x7f612c0b7a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:53.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:53.381+0000 7f6131976640 1 --2- 172.21.15.27:0/4118052610 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f612c06dfd0 0x7f612c0d1da0 unknown :-1 s=CLOSED pgs=1666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:53.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:53.381+0000 7f6131976640 1 -- 172.21.15.27:0/4118052610 >> 172.21.15.27:0/4118052610 conn(0x7f612c0b6700 msgr2=0x7f612c0b6b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:53.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:53.381+0000 7f6131976640 1 -- 172.21.15.27:0/4118052610 shutdown_connections 2024-09-12T03:23:53.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:53.381+0000 7f6131976640 1 -- 172.21.15.27:0/4118052610 wait complete. 2024-09-12T03:23:53.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:53.382+0000 7f6131976640 1 Processor -- start 2024-09-12T03:23:53.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:53.382+0000 7f6131976640 1 -- start start 2024-09-12T03:23:53.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:53.383+0000 7f6131976640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f612c06dfd0 0x7f612c1f4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:53.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:53.383+0000 7f6131976640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f612c0d3790 0x7f612c0b9190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:53.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:53.383+0000 7f6131976640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f612c162040 0x7f612c0b96d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:53.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:53.383+0000 7f6131976640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f612c0d4530 con 0x7f612c06dfd0 2024-09-12T03:23:53.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:53.383+0000 7f6131976640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f612c0cee00 con 0x7f612c0d3790 2024-09-12T03:23:53.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:53.383+0000 7f6131976640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f612c0d4150 con 0x7f612c162040 2024-09-12T03:23:53.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:53.383+0000 7f612affd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f612c06dfd0 0x7f612c1f4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:53.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:53.383+0000 7f612affd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f612c06dfd0 0x7f612c1f4d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:53478/0 (socket says 172.21.15.27:53478) 2024-09-12T03:23:53.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:53.383+0000 7f612affd640 1 -- 172.21.15.27:0/613182191 learned_addr learned my addr 172.21.15.27:0/613182191 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:23:53.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:53.383+0000 7f612a7fc640 1 --2- 172.21.15.27:0/613182191 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f612c0d3790 0x7f612c0b9190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:53.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:53.383+0000 7f612b7fe640 1 --2- 172.21.15.27:0/613182191 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f612c162040 0x7f612c0b96d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:53.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:53.383+0000 7f612affd640 1 -- 172.21.15.27:0/613182191 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f612c162040 msgr2=0x7f612c0b96d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:53.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:53.383+0000 7f612affd640 1 --2- 172.21.15.27:0/613182191 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f612c162040 0x7f612c0b96d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:53.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:53.383+0000 7f612affd640 1 -- 172.21.15.27:0/613182191 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f612c0d3790 msgr2=0x7f612c0b9190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:53.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:53.383+0000 7f612affd640 1 --2- 172.21.15.27:0/613182191 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f612c0d3790 0x7f612c0b9190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:53.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:53.383+0000 7f612affd640 1 -- 172.21.15.27:0/613182191 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f612c0cb880 con 0x7f612c06dfd0 2024-09-12T03:23:53.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:53.384+0000 7f612affd640 1 --2- 172.21.15.27:0/613182191 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f612c06dfd0 0x7f612c1f4d60 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7f611c00c7f0 tx=0x7f611c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:53.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:53.384+0000 7f6113fff640 1 -- 172.21.15.27:0/613182191 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f611c018070 con 0x7f612c06dfd0 2024-09-12T03:23:53.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:53.384+0000 7f6113fff640 1 -- 172.21.15.27:0/613182191 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f611c00a640 con 0x7f612c06dfd0 2024-09-12T03:23:53.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:53.384+0000 7f6131976640 1 -- 172.21.15.27:0/613182191 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f612c0cbb40 con 0x7f612c06dfd0 2024-09-12T03:23:53.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:53.384+0000 7f6113fff640 1 -- 172.21.15.27:0/613182191 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f611c00aa30 con 0x7f612c06dfd0 2024-09-12T03:23:53.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:53.384+0000 7f6131976640 1 -- 172.21.15.27:0/613182191 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f612c0cc100 con 0x7f612c06dfd0 2024-09-12T03:23:53.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:53.385+0000 7f6131976640 1 -- 172.21.15.27:0/613182191 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f612c06e6e0 con 0x7f612c06dfd0 2024-09-12T03:23:53.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:53.385+0000 7f6113fff640 1 -- 172.21.15.27:0/613182191 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 25) ==== 91365+0+0 (secure 0 0 0) 0x7f611c00abd0 con 0x7f612c06dfd0 2024-09-12T03:23:53.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:53.386+0000 7f6113fff640 1 --2- 172.21.15.27:0/613182191 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f611806c7c0 0x7f611806ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:53.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:53.386+0000 7f6113fff640 1 -- 172.21.15.27:0/613182191 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(157..157 src has 1..157) ==== 5283+0+0 (secure 0 0 0) 0x7f611c088af0 con 0x7f612c06dfd0 2024-09-12T03:23:53.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:53.386+0000 7f612a7fc640 1 --2- 172.21.15.27:0/613182191 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f611806c7c0 0x7f611806ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:53.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:53.387+0000 7f612a7fc640 1 --2- 172.21.15.27:0/613182191 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f611806c7c0 0x7f611806ec80 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f6120034000 tx=0x7f612002e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:53.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:53.391+0000 7f6113fff640 1 -- 172.21.15.27:0/613182191 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f611c0589d0 con 0x7f612c06dfd0 2024-09-12T03:23:53.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:53.565+0000 7f6131976640 1 -- 172.21.15.27:0/613182191 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f612c0cfe40 con 0x7f612c06dfd0 2024-09-12T03:23:54.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:54.241+0000 7f6113fff640 1 -- 172.21.15.27:0/613182191 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v158) ==== 170+0+0 (secure 0 0 0) 0x7f611c05ccd0 con 0x7f612c06dfd0 2024-09-12T03:23:54.241 INFO:teuthology.orchestra.run.smithi027.stderr:pool 'cephfs_data' created 2024-09-12T03:23:54.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:54.245+0000 7f6131976640 1 -- 172.21.15.27:0/613182191 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f611806c7c0 msgr2=0x7f611806ec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:54.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:54.245+0000 7f6131976640 1 --2- 172.21.15.27:0/613182191 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f611806c7c0 0x7f611806ec80 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f6120034000 tx=0x7f612002e000 comp rx=0 tx=0).stop 2024-09-12T03:23:54.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:54.245+0000 7f6131976640 1 -- 172.21.15.27:0/613182191 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f612c06dfd0 msgr2=0x7f612c1f4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:54.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:54.245+0000 7f6131976640 1 --2- 172.21.15.27:0/613182191 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f612c06dfd0 0x7f612c1f4d60 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7f611c00c7f0 tx=0x7f611c00ccb0 comp rx=0 tx=0).stop 2024-09-12T03:23:54.248 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:54.248+0000 7f6131976640 1 -- 172.21.15.27:0/613182191 shutdown_connections 2024-09-12T03:23:54.248 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:54.248+0000 7f6131976640 1 --2- 172.21.15.27:0/613182191 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f612c162040 0x7f612c0b96d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:54.248 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:54.248+0000 7f6131976640 1 --2- 172.21.15.27:0/613182191 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f612c0d3790 0x7f612c0b9190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:54.248 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:54.248+0000 7f6131976640 1 --2- 172.21.15.27:0/613182191 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f611806c7c0 0x7f611806ec80 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:54.249 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:54.248+0000 7f6131976640 1 --2- 172.21.15.27:0/613182191 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f612c06dfd0 0x7f612c1f4d60 unknown :-1 s=CLOSED pgs=1667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:54.249 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:54.248+0000 7f6131976640 1 -- 172.21.15.27:0/613182191 >> 172.21.15.27:0/613182191 conn(0x7f612c0b6700 msgr2=0x7f612c0cf170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:54.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:54.250+0000 7f6131976640 1 -- 172.21.15.27:0/613182191 shutdown_connections 2024-09-12T03:23:54.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:54.251+0000 7f6131976640 1 -- 172.21.15.27:0/613182191 wait complete. 2024-09-12T03:23:54.274 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-12T03:23:54.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:54.381+0000 7f08e7f6d640 1 Processor -- start 2024-09-12T03:23:54.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:54.382+0000 7f08e7f6d640 1 -- start start 2024-09-12T03:23:54.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:54.383+0000 7f08e7f6d640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f08e015e650 0x7f08e00d1d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:54.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:54.383+0000 7f08e7f6d640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f08e006cc40 0x7f08e006d060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:54.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:54.383+0000 7f08e7f6d640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08e00d2280 con 0x7f08e006db30 2024-09-12T03:23:54.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:54.383+0000 7f08e7f6d640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f08e006d5a0 con 0x7f08e015e650 2024-09-12T03:23:54.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:54.383+0000 7f08e7f6d640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f08e006e670 con 0x7f08e006cc40 2024-09-12T03:23:54.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:54.383+0000 7f08e650a640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f08e006db30 0x7f08e006df60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:33916/0 (socket says 172.21.15.27:33916) 2024-09-12T03:23:54.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:54.383+0000 7f08e650a640 1 -- 172.21.15.27:0/2320376951 learned_addr learned my addr 172.21.15.27:0/2320376951 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:23:54.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:54.383+0000 7f08e5d09640 1 --2- 172.21.15.27:0/2320376951 >> v2:172.21.15.45:3300/0 conn(0x7f08e015e650 0x7f08e00d1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:54.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:54.383+0000 7f08e5508640 1 --2- 172.21.15.27:0/2320376951 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f08e006cc40 0x7f08e006d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:54.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:54.383+0000 7f08e4d07640 1 -- 172.21.15.27:0/2320376951 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1734821427 0 0) 0x7f08e00d2280 con 0x7f08e006db30 2024-09-12T03:23:54.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:54.383+0000 7f08e4d07640 1 -- 172.21.15.27:0/2320376951 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f08cc003520 con 0x7f08e006db30 2024-09-12T03:23:54.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:54.383+0000 7f08e4d07640 1 -- 172.21.15.27:0/2320376951 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2313887364 0 0) 0x7f08cc003520 con 0x7f08e006db30 2024-09-12T03:23:54.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:54.383+0000 7f08e4d07640 1 -- 172.21.15.27:0/2320376951 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f08e006cc40 msgr2=0x7f08e006d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:54.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:54.383+0000 7f08e4d07640 1 --2- 172.21.15.27:0/2320376951 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f08e006cc40 0x7f08e006d060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:54.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:54.383+0000 7f08e4d07640 1 -- 172.21.15.27:0/2320376951 >> v2:172.21.15.45:3300/0 conn(0x7f08e015e650 msgr2=0x7f08e00d1d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:54.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:54.383+0000 7f08e4d07640 1 --2- 172.21.15.27:0/2320376951 >> v2:172.21.15.45:3300/0 conn(0x7f08e015e650 0x7f08e00d1d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:54.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:54.383+0000 7f08e4d07640 1 -- 172.21.15.27:0/2320376951 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08e006d850 con 0x7f08e006db30 2024-09-12T03:23:54.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:54.383+0000 7f08e4d07640 1 -- 172.21.15.27:0/2320376951 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 847755213 0 0) 0x7f08d80019f0 con 0x7f08e006db30 2024-09-12T03:23:54.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:54.383+0000 7f08e4d07640 1 -- 172.21.15.27:0/2320376951 >> v1:172.21.15.27:6789/0 conn(0x7f08e006db30 legacy=0x7f08e006df60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:54.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:54.383+0000 7f08e4d07640 1 --2- 172.21.15.27:0/2320376951 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f08cc0039b0 0x7f08cc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:54.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:54.383+0000 7f08e4d07640 1 --2- 172.21.15.27:0/2320376951 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f08cc004800 0x7f08cc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:54.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:54.383+0000 7f08e4d07640 1 --2- 172.21.15.27:0/2320376951 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f08e006cc40 0x7f08cc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:54.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:54.383+0000 7f08e4d07640 1 -- 172.21.15.27:0/2320376951 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f08e006d5a0 con 0x7f08cc0039b0 2024-09-12T03:23:54.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:54.383+0000 7f08e4d07640 1 -- 172.21.15.27:0/2320376951 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f08e006e670 con 0x7f08cc004800 2024-09-12T03:23:54.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:54.383+0000 7f08e4d07640 1 -- 172.21.15.27:0/2320376951 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f08cc00b9f0 con 0x7f08e006cc40 2024-09-12T03:23:54.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:54.384+0000 7f08e5d09640 1 --2- 172.21.15.27:0/2320376951 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f08cc0039b0 0x7f08cc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:54.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:54.384+0000 7f08e5508640 1 --2- 172.21.15.27:0/2320376951 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f08cc004800 0x7f08cc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:54.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:54.384+0000 7f08e650a640 1 --2- 172.21.15.27:0/2320376951 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f08e006cc40 0x7f08cc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:54.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:54.384+0000 7f08e5d09640 1 -- 172.21.15.27:0/2320376951 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f08e006cc40 msgr2=0x7f08cc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:54.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:54.384+0000 7f08e5d09640 1 --2- 172.21.15.27:0/2320376951 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f08e006cc40 0x7f08cc0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:54.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:54.384+0000 7f08e5d09640 1 -- 172.21.15.27:0/2320376951 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f08cc004800 msgr2=0x7f08cc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:54.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:54.384+0000 7f08e5d09640 1 --2- 172.21.15.27:0/2320376951 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f08cc004800 0x7f08cc004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:54.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:54.384+0000 7f08e5d09640 1 -- 172.21.15.27:0/2320376951 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08cc003520 con 0x7f08cc0039b0 2024-09-12T03:23:54.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:54.384+0000 7f08e5508640 1 --2- 172.21.15.27:0/2320376951 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f08cc004800 0x7f08cc004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:23:54.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:54.385+0000 7f08e5d09640 1 --2- 172.21.15.27:0/2320376951 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f08cc0039b0 0x7f08cc003dd0 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7f08dc009ac0 tx=0x7f08cc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b17b9a248d9dff4b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:54.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:54.385+0000 7f08e4d07640 1 -- 172.21.15.27:0/2320376951 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08dc036070 con 0x7f08cc0039b0 2024-09-12T03:23:54.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:54.385+0000 7f08e4d07640 1 -- 172.21.15.27:0/2320376951 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08dc031400 con 0x7f08cc0039b0 2024-09-12T03:23:54.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:54.385+0000 7f08e4d07640 1 -- 172.21.15.27:0/2320376951 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08dc009690 con 0x7f08cc0039b0 2024-09-12T03:23:54.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:54.385+0000 7f08e7f6d640 1 -- 172.21.15.27:0/2320376951 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f08cc0039b0 msgr2=0x7f08cc003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:54.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:54.385+0000 7f08e7f6d640 1 --2- 172.21.15.27:0/2320376951 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f08cc0039b0 0x7f08cc003dd0 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7f08dc009ac0 tx=0x7f08cc005830 comp rx=0 tx=0).stop 2024-09-12T03:23:54.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:54.385+0000 7f08e650a640 1 -- 172.21.15.27:0/2320376951 reap_dead start 2024-09-12T03:23:54.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:54.385+0000 7f08e7f6d640 1 -- 172.21.15.27:0/2320376951 shutdown_connections 2024-09-12T03:23:54.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:54.385+0000 7f08e7f6d640 1 -- 172.21.15.27:0/2320376951 >> 172.21.15.27:0/2320376951 conn(0x7f08e006f320 msgr2=0x7f08e006f700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:54.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:54.385+0000 7f08e7f6d640 1 -- 172.21.15.27:0/2320376951 shutdown_connections 2024-09-12T03:23:54.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:54.385+0000 7f08e7f6d640 1 -- 172.21.15.27:0/2320376951 wait complete. 2024-09-12T03:23:54.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:54.387+0000 7f08e7f6d640 1 Processor -- start 2024-09-12T03:23:54.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:54.387+0000 7f08e7f6d640 1 -- start start 2024-09-12T03:23:54.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:54.387+0000 7f08e7f6d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f08cc0039b0 0x7f08e01f50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:54.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:54.387+0000 7f08e7f6d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f08e00b5c20 0x7f08e015d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:54.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:54.387+0000 7f08e7f6d640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f08e00ae930 0x7f08e00b2760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:54.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:54.387+0000 7f08e7f6d640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f08e015ea30 con 0x7f08cc0039b0 2024-09-12T03:23:54.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:54.387+0000 7f08e7f6d640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f08e00d1540 con 0x7f08e00b5c20 2024-09-12T03:23:54.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:54.387+0000 7f08e7f6d640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f08e015ebb0 con 0x7f08e00ae930 2024-09-12T03:23:54.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:54.388+0000 7f08e5d09640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f08cc0039b0 0x7f08e01f50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:54.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:54.388+0000 7f08e5d09640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f08cc0039b0 0x7f08e01f50f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:53490/0 (socket says 172.21.15.27:53490) 2024-09-12T03:23:54.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:54.388+0000 7f08e5d09640 1 -- 172.21.15.27:0/2497680651 learned_addr learned my addr 172.21.15.27:0/2497680651 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:23:54.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:54.388+0000 7f08e5508640 1 --2- 172.21.15.27:0/2497680651 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f08e00b5c20 0x7f08e015d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:54.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:54.388+0000 7f08e650a640 1 --2- 172.21.15.27:0/2497680651 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f08e00ae930 0x7f08e00b2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:54.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:54.388+0000 7f08e5d09640 1 -- 172.21.15.27:0/2497680651 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f08e00ae930 msgr2=0x7f08e00b2760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:54.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:54.388+0000 7f08e5d09640 1 --2- 172.21.15.27:0/2497680651 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f08e00ae930 0x7f08e00b2760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:54.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:54.388+0000 7f08e5d09640 1 -- 172.21.15.27:0/2497680651 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f08e00b5c20 msgr2=0x7f08e015d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:54.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:54.388+0000 7f08e5d09640 1 --2- 172.21.15.27:0/2497680651 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f08e00b5c20 0x7f08e015d2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:54.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:54.388+0000 7f08e5d09640 1 -- 172.21.15.27:0/2497680651 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08e01f57f0 con 0x7f08cc0039b0 2024-09-12T03:23:54.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:54.388+0000 7f08e5d09640 1 --2- 172.21.15.27:0/2497680651 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f08cc0039b0 0x7f08e01f50f0 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7f08dc034040 tx=0x7f08dc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:54.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:54.388+0000 7f08d67fc640 1 -- 172.21.15.27:0/2497680651 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08dc041020 con 0x7f08cc0039b0 2024-09-12T03:23:54.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:54.388+0000 7f08d67fc640 1 -- 172.21.15.27:0/2497680651 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08dc03c470 con 0x7f08cc0039b0 2024-09-12T03:23:54.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:54.388+0000 7f08e7f6d640 1 -- 172.21.15.27:0/2497680651 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08e01f5ad0 con 0x7f08cc0039b0 2024-09-12T03:23:54.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:54.388+0000 7f08d67fc640 1 -- 172.21.15.27:0/2497680651 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08dc036040 con 0x7f08cc0039b0 2024-09-12T03:23:54.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:54.388+0000 7f08e7f6d640 1 -- 172.21.15.27:0/2497680651 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08e01f6090 con 0x7f08cc0039b0 2024-09-12T03:23:54.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:54.389+0000 7f08e7f6d640 1 -- 172.21.15.27:0/2497680651 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08e006d060 con 0x7f08cc0039b0 2024-09-12T03:23:54.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:54.390+0000 7f08d67fc640 1 -- 172.21.15.27:0/2497680651 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 25) ==== 91365+0+0 (secure 0 0 0) 0x7f08dc044020 con 0x7f08cc0039b0 2024-09-12T03:23:54.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:54.391+0000 7f08d67fc640 1 --2- 172.21.15.27:0/2497680651 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f08c406c7f0 0x7f08c406ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:54.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:54.391+0000 7f08e5508640 1 --2- 172.21.15.27:0/2497680651 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f08c406c7f0 0x7f08c406ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:54.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:54.391+0000 7f08d67fc640 1 -- 172.21.15.27:0/2497680651 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(158..158 src has 1..158) ==== 5665+0+0 (secure 0 0 0) 0x7f08dc0a9410 con 0x7f08cc0039b0 2024-09-12T03:23:54.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:54.391+0000 7f08e5508640 1 --2- 172.21.15.27:0/2497680651 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f08c406c7f0 0x7f08c406ecb0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f08e00cf7c0 tx=0x7f08d0008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:54.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:54.396+0000 7f08d67fc640 1 -- 172.21.15.27:0/2497680651 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f08dc079370 con 0x7f08cc0039b0 2024-09-12T03:23:54.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:54.605+0000 7f08e7f6d640 1 -- 172.21.15.27:0/2497680651 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f08e006f880 con 0x7f08cc0039b0 2024-09-12T03:23:55.262 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:55.261+0000 7f08d67fc640 1 -- 172.21.15.27:0/2497680651 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 26 and data pool 27 v97) ==== 174+0+0 (secure 0 0 0) 0x7f08dc07d670 con 0x7f08cc0039b0 2024-09-12T03:23:55.263 INFO:teuthology.orchestra.run.smithi027.stderr:new fs with metadata pool 26 and data pool 27 2024-09-12T03:23:55.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:55.270+0000 7f08e7f6d640 1 -- 172.21.15.27:0/2497680651 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f08c406c7f0 msgr2=0x7f08c406ecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:55.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:55.271+0000 7f08e7f6d640 1 --2- 172.21.15.27:0/2497680651 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f08c406c7f0 0x7f08c406ecb0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f08e00cf7c0 tx=0x7f08d0008040 comp rx=0 tx=0).stop 2024-09-12T03:23:55.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:55.271+0000 7f08e7f6d640 1 -- 172.21.15.27:0/2497680651 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f08cc0039b0 msgr2=0x7f08e01f50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:55.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:55.271+0000 7f08e7f6d640 1 --2- 172.21.15.27:0/2497680651 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f08cc0039b0 0x7f08e01f50f0 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7f08dc034040 tx=0x7f08dc004730 comp rx=0 tx=0).stop 2024-09-12T03:23:55.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:55.271+0000 7f08e7f6d640 1 -- 172.21.15.27:0/2497680651 shutdown_connections 2024-09-12T03:23:55.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:55.271+0000 7f08e7f6d640 1 --2- 172.21.15.27:0/2497680651 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f08e00ae930 0x7f08e00b2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:55.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:55.271+0000 7f08e7f6d640 1 --2- 172.21.15.27:0/2497680651 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f08e00b5c20 0x7f08e015d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:55.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:55.271+0000 7f08e7f6d640 1 --2- 172.21.15.27:0/2497680651 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f08c406c7f0 0x7f08c406ecb0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:55.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:55.271+0000 7f08e7f6d640 1 --2- 172.21.15.27:0/2497680651 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f08cc0039b0 0x7f08e01f50f0 unknown :-1 s=CLOSED pgs=1670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:55.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:55.271+0000 7f08e7f6d640 1 -- 172.21.15.27:0/2497680651 >> 172.21.15.27:0/2497680651 conn(0x7f08e006f320 msgr2=0x7f08e00b6340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:55.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:55.271+0000 7f08e7f6d640 1 -- 172.21.15.27:0/2497680651 shutdown_connections 2024-09-12T03:23:55.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:55.271+0000 7f08e7f6d640 1 -- 172.21.15.27:0/2497680651 wait complete. 2024-09-12T03:23:55.296 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-12T03:23:55.402 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:55.401+0000 7f6453581640 1 Processor -- start 2024-09-12T03:23:55.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:55.402+0000 7f6453581640 1 -- start start 2024-09-12T03:23:55.406 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:55.402+0000 7f6453581640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f644c0d0f80 0x7f644c06bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:55.407 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:55.403+0000 7f6453581640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f644c0d0a20 con 0x7f644c06c070 2024-09-12T03:23:55.407 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:55.403+0000 7f6453581640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f644c0d0180 con 0x7f644c1460e0 2024-09-12T03:23:55.407 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:55.403+0000 7f6453581640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f644c06c4e0 con 0x7f644c0d0f80 2024-09-12T03:23:55.407 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:55.403+0000 7f6451b1e640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f644c06c070 0x7f644c06e6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:33924/0 (socket says 172.21.15.27:33924) 2024-09-12T03:23:55.407 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:55.403+0000 7f6451b1e640 1 -- 172.21.15.27:0/2873141964 learned_addr learned my addr 172.21.15.27:0/2873141964 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:23:55.407 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:55.403+0000 7f6450b1c640 1 --2- 172.21.15.27:0/2873141964 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f644c0d0f80 0x7f644c06bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:55.407 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:55.403+0000 7f6443fff640 1 -- 172.21.15.27:0/2873141964 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 355874718 0 0) 0x7f644c0d0a20 con 0x7f644c06c070 2024-09-12T03:23:55.407 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:55.403+0000 7f6443fff640 1 -- 172.21.15.27:0/2873141964 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6438003520 con 0x7f644c06c070 2024-09-12T03:23:55.407 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:55.403+0000 7f6443fff640 1 -- 172.21.15.27:0/2873141964 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2336196043 0 0) 0x7f6438003520 con 0x7f644c06c070 2024-09-12T03:23:55.407 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:55.403+0000 7f6443fff640 1 -- 172.21.15.27:0/2873141964 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f644c0d0f80 msgr2=0x7f644c06bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:55.408 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:55.403+0000 7f6443fff640 1 --2- 172.21.15.27:0/2873141964 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f644c0d0f80 0x7f644c06bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:55.408 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:55.403+0000 7f6443fff640 1 -- 172.21.15.27:0/2873141964 >> v1:172.21.15.45:6789/0 conn(0x7f644c1460e0 legacy=0x7f644c0cf6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:55.408 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:55.403+0000 7f6443fff640 1 -- 172.21.15.27:0/2873141964 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f644c0d0460 con 0x7f644c06c070 2024-09-12T03:23:55.408 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:55.403+0000 7f6443fff640 1 -- 172.21.15.27:0/2873141964 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 847755213 0 0) 0x7f64440019f0 con 0x7f644c06c070 2024-09-12T03:23:55.408 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:55.403+0000 7f6443fff640 1 -- 172.21.15.27:0/2873141964 >> v1:172.21.15.27:6789/0 conn(0x7f644c06c070 legacy=0x7f644c06e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:55.408 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:55.403+0000 7f6443fff640 1 --2- 172.21.15.27:0/2873141964 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f64380039e0 0x7f6438003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:55.409 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:55.403+0000 7f6443fff640 1 --2- 172.21.15.27:0/2873141964 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f644c0d0f80 0x7f6438004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:55.409 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:55.403+0000 7f6443fff640 1 --2- 172.21.15.27:0/2873141964 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6438004d40 0x7f64380051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:55.409 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:55.403+0000 7f6443fff640 1 -- 172.21.15.27:0/2873141964 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f644c06c4e0 con 0x7f64380039e0 2024-09-12T03:23:55.409 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:55.403+0000 7f6443fff640 1 -- 172.21.15.27:0/2873141964 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f643800baa0 con 0x7f6438004d40 2024-09-12T03:23:55.409 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:55.403+0000 7f6443fff640 1 -- 172.21.15.27:0/2873141964 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f643800bca0 con 0x7f644c0d0f80 2024-09-12T03:23:55.409 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:55.404+0000 7f645131d640 1 --2- 172.21.15.27:0/2873141964 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f64380039e0 0x7f6438003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:55.409 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:55.404+0000 7f6450b1c640 1 --2- 172.21.15.27:0/2873141964 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f644c0d0f80 0x7f6438004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:55.409 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:55.404+0000 7f6451b1e640 1 --2- 172.21.15.27:0/2873141964 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6438004d40 0x7f64380051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:55.409 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:55.404+0000 7f6450b1c640 1 -- 172.21.15.27:0/2873141964 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6438004d40 msgr2=0x7f64380051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:55.409 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:55.404+0000 7f6450b1c640 1 --2- 172.21.15.27:0/2873141964 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6438004d40 0x7f64380051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:55.409 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:55.404+0000 7f6450b1c640 1 -- 172.21.15.27:0/2873141964 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f64380039e0 msgr2=0x7f6438003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:55.410 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:55.404+0000 7f6450b1c640 1 --2- 172.21.15.27:0/2873141964 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f64380039e0 0x7f6438003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:55.410 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:55.404+0000 7f6450b1c640 1 -- 172.21.15.27:0/2873141964 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6438003520 con 0x7f644c0d0f80 2024-09-12T03:23:55.410 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:55.404+0000 7f645131d640 1 --2- 172.21.15.27:0/2873141964 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f64380039e0 0x7f6438003e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:23:55.410 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:55.405+0000 7f6450b1c640 1 --2- 172.21.15.27:0/2873141964 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f644c0d0f80 0x7f6438004800 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f644c070c50 tx=0x7f6438005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5c6f642db877e4b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:55.410 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:55.405+0000 7f6443fff640 1 -- 172.21.15.27:0/2873141964 <== mon.2 v2:172.21.15.45:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f643c036070 con 0x7f644c0d0f80 2024-09-12T03:23:55.410 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:55.405+0000 7f6443fff640 1 -- 172.21.15.27:0/2873141964 <== mon.2 v2:172.21.15.45:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f643c031400 con 0x7f644c0d0f80 2024-09-12T03:23:55.410 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:55.405+0000 7f6443fff640 1 -- 172.21.15.27:0/2873141964 <== mon.2 v2:172.21.15.45:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f643c009690 con 0x7f644c0d0f80 2024-09-12T03:23:55.410 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:55.405+0000 7f6453581640 1 -- 172.21.15.27:0/2873141964 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f644c0d0f80 msgr2=0x7f6438004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:55.410 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:55.405+0000 7f6453581640 1 --2- 172.21.15.27:0/2873141964 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f644c0d0f80 0x7f6438004800 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f644c070c50 tx=0x7f6438005830 comp rx=0 tx=0).stop 2024-09-12T03:23:55.410 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:55.405+0000 7f6451b1e640 1 -- 172.21.15.27:0/2873141964 reap_dead start 2024-09-12T03:23:55.410 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:55.405+0000 7f6453581640 1 -- 172.21.15.27:0/2873141964 shutdown_connections 2024-09-12T03:23:55.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:55.405+0000 7f6453581640 1 -- 172.21.15.27:0/2873141964 >> 172.21.15.27:0/2873141964 conn(0x7f644c0b3300 msgr2=0x7f644c0b36e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:55.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:55.406+0000 7f6453581640 1 -- 172.21.15.27:0/2873141964 shutdown_connections 2024-09-12T03:23:55.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:55.406+0000 7f6453581640 1 -- 172.21.15.27:0/2873141964 wait complete. 2024-09-12T03:23:55.413 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:55.407+0000 7f6453581640 1 Processor -- start 2024-09-12T03:23:55.414 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:55.407+0000 7f6453581640 1 -- start start 2024-09-12T03:23:55.414 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:55.407+0000 7f6453581640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f644c0d0f80 0x7f644c15ba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:55.414 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:55.407+0000 7f6453581640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f644c13c120 0x7f644c13c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:55.414 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:55.407+0000 7f6453581640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f644c06f5f0 0x7f644c06fa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:55.414 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:55.407+0000 7f6453581640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f644c1464c0 con 0x7f644c13c120 2024-09-12T03:23:55.414 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:55.407+0000 7f6453581640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f644c06e1c0 con 0x7f644c06f5f0 2024-09-12T03:23:55.414 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:55.407+0000 7f6453581640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f644c0cfdb0 con 0x7f644c0d0f80 2024-09-12T03:23:55.414 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:55.408+0000 7f6450b1c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f644c13c120 0x7f644c13c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:55.414 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:55.408+0000 7f6450b1c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f644c13c120 0x7f644c13c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:53512/0 (socket says 172.21.15.27:53512) 2024-09-12T03:23:55.414 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:55.408+0000 7f6450b1c640 1 -- 172.21.15.27:0/2428918477 learned_addr learned my addr 172.21.15.27:0/2428918477 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:23:55.415 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:55.408+0000 7f645131d640 1 --2- 172.21.15.27:0/2428918477 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f644c0d0f80 0x7f644c15ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:55.415 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:55.408+0000 7f6451b1e640 1 --2- 172.21.15.27:0/2428918477 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f644c06f5f0 0x7f644c06fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:55.415 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:55.408+0000 7f6450b1c640 1 -- 172.21.15.27:0/2428918477 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f644c0d0f80 msgr2=0x7f644c15ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:55.415 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:55.408+0000 7f6450b1c640 1 --2- 172.21.15.27:0/2428918477 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f644c0d0f80 0x7f644c15ba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:55.415 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:55.408+0000 7f6450b1c640 1 -- 172.21.15.27:0/2428918477 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f644c06f5f0 msgr2=0x7f644c06fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:55.415 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:55.408+0000 7f6450b1c640 1 --2- 172.21.15.27:0/2428918477 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f644c06f5f0 0x7f644c06fa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:55.415 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:55.408+0000 7f6450b1c640 1 -- 172.21.15.27:0/2428918477 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f644c1f2770 con 0x7f644c13c120 2024-09-12T03:23:55.415 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:55.408+0000 7f6450b1c640 1 --2- 172.21.15.27:0/2428918477 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f644c13c120 0x7f644c13c580 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7f643c02ab50 tx=0x7f643c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:55.415 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:55.408+0000 7f6441ffb640 1 -- 172.21.15.27:0/2428918477 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f643c040020 con 0x7f644c13c120 2024-09-12T03:23:55.415 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:55.408+0000 7f6441ffb640 1 -- 172.21.15.27:0/2428918477 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f643c003940 con 0x7f644c13c120 2024-09-12T03:23:55.415 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:55.408+0000 7f6441ffb640 1 -- 172.21.15.27:0/2428918477 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f643c036040 con 0x7f644c13c120 2024-09-12T03:23:55.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:55.408+0000 7f6453581640 1 -- 172.21.15.27:0/2428918477 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f644c1f2a20 con 0x7f644c13c120 2024-09-12T03:23:55.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:55.409+0000 7f6453581640 1 -- 172.21.15.27:0/2428918477 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f644c1f2fe0 con 0x7f644c13c120 2024-09-12T03:23:55.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:55.410+0000 7f6453581640 1 -- 172.21.15.27:0/2428918477 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f644c063610 con 0x7f644c13c120 2024-09-12T03:23:55.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:55.410+0000 7f6441ffb640 1 -- 172.21.15.27:0/2428918477 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 25) ==== 91365+0+0 (secure 0 0 0) 0x7f643c043020 con 0x7f644c13c120 2024-09-12T03:23:55.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:55.411+0000 7f6441ffb640 1 --2- 172.21.15.27:0/2428918477 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f643006c7c0 0x7f643006ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:55.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:55.411+0000 7f6441ffb640 1 -- 172.21.15.27:0/2428918477 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(159..159 src has 1..159) ==== 5765+0+0 (secure 0 0 0) 0x7f643c0a9730 con 0x7f644c13c120 2024-09-12T03:23:55.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:55.411+0000 7f645131d640 1 --2- 172.21.15.27:0/2428918477 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f643006c7c0 0x7f643006ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:55.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:55.411+0000 7f645131d640 1 --2- 172.21.15.27:0/2428918477 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f643006c7c0 0x7f643006ec80 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f64480019e0 tx=0x7f6448009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:55.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:55.415+0000 7f6441ffb640 1 -- 172.21.15.27:0/2428918477 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f643c04a070 con 0x7f644c13c120 2024-09-12T03:23:55.593 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:55.591+0000 7f6453581640 1 -- 172.21.15.27:0/2428918477 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f644c06bb30 con 0x7f644c13c120 2024-09-12T03:23:55.594 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:55.593+0000 7f6441ffb640 1 -- 172.21.15.27:0/2428918477 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v159) ==== 74+0+16797 (secure 0 0 0) 0x7f643c079690 con 0x7f644c13c120 2024-09-12T03:23:55.594 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:23:55.595 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":159,"fsid":"d2df504e-2647-4385-a5db-9fdf69f487a0","created":"2024-09-12T03:16:07.778937+0000","modified":"2024-09-12T03:23:54.606527+0000","last_up_change":"2024-09-12T03:16:19.314778+0000","last_in_change":"2024-09-12T03:16:09.893086+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T03:16:20.097646+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs_metadata","create_time":"2024-09-12T03:23:52.574839+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"159","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_data","create_time":"2024-09-12T03:23:53.596556+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"159","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1851116c-1fb3-4dc9-9772-f90e2ef4c355","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":158,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6817","nonce":3002321488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6819","nonce":3002321488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6823","nonce":3002321488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6821","nonce":3002321488}]},"public_addr":"172.21.15.27:6817/3002321488","cluster_addr":"172.21.15.27:6819/3002321488","heartbeat_back_addr":"172.21.15.27:6823/3002321488","heartbeat_front_addr":"172.21.15.27:6821/3002321488","state":["exists","up"]},{"osd":1,"uuid":"f38ae17d-73eb-4204-b99d-975fa806fb87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":158,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6809","nonce":1613102886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6811","nonce":1613102886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6815","nonce":1613102886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6813","nonce":1613102886}]},"public_addr":"172.21.15.27:6809/1613102886","cluster_addr":"172.21.15.27:6811/1613102886","heartbeat_back_addr":"172.21.15.27:6815/1613102886","heartbeat_front_addr":"172.21.15.27:6813/1613102886","state":["exists","up"]},{"osd":2,"uuid":"fb12dcc8-bdd0-42b8-b46d-8b10106224bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":158,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6825","nonce":3534723355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6827","nonce":3534723355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6831","nonce":3534723355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6829","nonce":3534723355}]},"public_addr":"172.21.15.27:6825/3534723355","cluster_addr":"172.21.15.27:6827/3534723355","heartbeat_back_addr":"172.21.15.27:6831/3534723355","heartbeat_front_addr":"172.21.15.27:6829/3534723355","state":["exists","up"]},{"osd":3,"uuid":"d74d3f49-9c24-4b9f-98e2-3ecd7b5ef3b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":158,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6801","nonce":80661598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6803","nonce":80661598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6807","nonce":80661598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6805","nonce":80661598}]},"public_addr":"172.21.15.27:6801/80661598","cluster_addr":"172.21.15.27:6803/80661598","heartbeat_back_addr":"172.21.15.27:6807/80661598","heartbeat_front_addr":"172.21.15.27:6805/80661598","state":["exists","up"]},{"osd":4,"uuid":"2bf123f2-1b0d-41c4-aa3b-805bab267e59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":158,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6817","nonce":3943189827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6819","nonce":3943189827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6823","nonce":3943189827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6821","nonce":3943189827}]},"public_addr":"172.21.15.45:6817/3943189827","cluster_addr":"172.21.15.45:6819/3943189827","heartbeat_back_addr":"172.21.15.45:6823/3943189827","heartbeat_front_addr":"172.21.15.45:6821/3943189827","state":["exists","up"]},{"osd":5,"uuid":"c49ba5da-91c6-4a86-a97b-4aba89c253ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":158,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6809","nonce":3620760330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6811","nonce":3620760330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6815","nonce":3620760330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6813","nonce":3620760330}]},"public_addr":"172.21.15.45:6809/3620760330","cluster_addr":"172.21.15.45:6811/3620760330","heartbeat_back_addr":"172.21.15.45:6815/3620760330","heartbeat_front_addr":"172.21.15.45:6813/3620760330","state":["exists","up"]},{"osd":6,"uuid":"b0123475-89f7-4a34-9e33-d797b99dd9dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":158,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6801","nonce":1989083510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6803","nonce":1989083510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6807","nonce":1989083510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6805","nonce":1989083510}]},"public_addr":"172.21.15.45:6801/1989083510","cluster_addr":"172.21.15.45:6803/1989083510","heartbeat_back_addr":"172.21.15.45:6807/1989083510","heartbeat_front_addr":"172.21.15.45:6805/1989083510","state":["exists","up"]},{"osd":7,"uuid":"4dad773d-b7d8-46fc-8a10-a972aad27b29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":158,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6825","nonce":1513300294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6827","nonce":1513300294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6831","nonce":1513300294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6829","nonce":1513300294}]},"public_addr":"172.21.15.45:6825/1513300294","cluster_addr":"172.21.15.45:6827/1513300294","heartbeat_back_addr":"172.21.15.45:6831/1513300294","heartbeat_front_addr":"172.21.15.45:6829/1513300294","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.881855+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:17.133252+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.735339+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.500742+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.477571+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:17.167667+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.470291+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.873743+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]},{"pool":19,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T03:23:55.597 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:55.595+0000 7f6453581640 1 -- 172.21.15.27:0/2428918477 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f643006c7c0 msgr2=0x7f643006ec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:55.597 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:55.595+0000 7f6453581640 1 --2- 172.21.15.27:0/2428918477 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f643006c7c0 0x7f643006ec80 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f64480019e0 tx=0x7f6448009040 comp rx=0 tx=0).stop 2024-09-12T03:23:55.597 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:55.595+0000 7f6453581640 1 -- 172.21.15.27:0/2428918477 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f644c13c120 msgr2=0x7f644c13c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:55.597 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:55.595+0000 7f6453581640 1 --2- 172.21.15.27:0/2428918477 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f644c13c120 0x7f644c13c580 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7f643c02ab50 tx=0x7f643c004730 comp rx=0 tx=0).stop 2024-09-12T03:23:55.598 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:55.595+0000 7f6453581640 1 -- 172.21.15.27:0/2428918477 shutdown_connections 2024-09-12T03:23:55.598 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:55.595+0000 7f6453581640 1 --2- 172.21.15.27:0/2428918477 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f644c06f5f0 0x7f644c06fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:55.598 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:55.595+0000 7f6453581640 1 --2- 172.21.15.27:0/2428918477 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f643006c7c0 0x7f643006ec80 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:55.598 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:55.595+0000 7f6453581640 1 --2- 172.21.15.27:0/2428918477 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f644c13c120 0x7f644c13c580 unknown :-1 s=CLOSED pgs=1672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:55.598 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:55.595+0000 7f6453581640 1 --2- 172.21.15.27:0/2428918477 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f644c0d0f80 0x7f644c15ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:55.598 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:55.595+0000 7f6453581640 1 -- 172.21.15.27:0/2428918477 >> 172.21.15.27:0/2428918477 conn(0x7f644c0b3300 msgr2=0x7f644c070560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:55.599 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:55.595+0000 7f6453581640 1 -- 172.21.15.27:0/2428918477 shutdown_connections 2024-09-12T03:23:55.599 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:55.595+0000 7f6453581640 1 -- 172.21.15.27:0/2428918477 wait complete. 2024-09-12T03:23:55.608 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-12T03:23:55.711 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:55.710+0000 7fb3b2a27640 1 Processor -- start 2024-09-12T03:23:55.711 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:55.710+0000 7fb3b2a27640 1 -- start start 2024-09-12T03:23:55.713 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:55.711+0000 7fb3b2a27640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fb3ac0cf660 0x7fb3ac0d2780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:55.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:55.711+0000 7fb3b2a27640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb3ac06cc40 con 0x7fb3ac06f360 2024-09-12T03:23:55.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:55.711+0000 7fb3b2a27640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb3ac0d1ce0 con 0x7fb3ac0b1ca0 2024-09-12T03:23:55.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:55.711+0000 7fb3b2a27640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fb3ac0d22f0 con 0x7fb3ac0cf660 2024-09-12T03:23:55.717 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:55.711+0000 7fb3ab7fe640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fb3ac06f360 0x7fb3ac0cef50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:33932/0 (socket says 172.21.15.27:33932) 2024-09-12T03:23:55.717 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:55.711+0000 7fb3ab7fe640 1 -- 172.21.15.27:0/445784972 learned_addr learned my addr 172.21.15.27:0/445784972 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:23:55.717 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:55.711+0000 7fb3b0fc4640 1 --2- 172.21.15.27:0/445784972 >> v2:172.21.15.45:3300/0 conn(0x7fb3ac0cf660 0x7fb3ac0d2780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:55.717 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:55.711+0000 7fb3aaffd640 1 -- 172.21.15.27:0/445784972 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3276479140 0 0) 0x7fb3ac06cc40 con 0x7fb3ac06f360 2024-09-12T03:23:55.717 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:55.711+0000 7fb3aaffd640 1 -- 172.21.15.27:0/445784972 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb394003520 con 0x7fb3ac06f360 2024-09-12T03:23:55.717 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:55.712+0000 7fb3aaffd640 1 -- 172.21.15.27:0/445784972 <== mon.1 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4213515819 0 0) 0x7fb3ac0d1ce0 con 0x7fb3ac0b1ca0 2024-09-12T03:23:55.718 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:55.712+0000 7fb3aaffd640 1 -- 172.21.15.27:0/445784972 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb3ac06cc40 con 0x7fb3ac0b1ca0 2024-09-12T03:23:55.718 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:55.712+0000 7fb3aaffd640 1 -- 172.21.15.27:0/445784972 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3377968875 0 0) 0x7fb394003520 con 0x7fb3ac06f360 2024-09-12T03:23:55.718 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:55.712+0000 7fb3b0fc4640 1 -- 172.21.15.27:0/445784972 >> v1:172.21.15.45:6789/0 conn(0x7fb3ac0b1ca0 legacy=0x7fb3ac15f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:55.718 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:55.712+0000 7fb3b0fc4640 1 -- 172.21.15.27:0/445784972 >> v1:172.21.15.27:6789/0 conn(0x7fb3ac06f360 legacy=0x7fb3ac0cef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:55.718 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:55.712+0000 7fb3b0fc4640 1 -- 172.21.15.27:0/445784972 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3ac06cee0 con 0x7fb3ac0cf660 2024-09-12T03:23:55.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:55.713+0000 7fb3b0fc4640 1 --2- 172.21.15.27:0/445784972 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb3ac0cf660 0x7fb3ac0d2780 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fb3a4009d90 tx=0x7fb3a402a680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9f448ae21ce47be3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:55.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:55.713+0000 7fb3aaffd640 1 -- 172.21.15.27:0/445784972 <== mon.1 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3a0001670 con 0x7fb3ac0cf660 2024-09-12T03:23:55.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:55.713+0000 7fb3aaffd640 1 -- 172.21.15.27:0/445784972 <== mon.1 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3a4007540 con 0x7fb3ac0cf660 2024-09-12T03:23:55.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:55.713+0000 7fb3aaffd640 1 -- 172.21.15.27:0/445784972 <== mon.1 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3a4007910 con 0x7fb3ac0cf660 2024-09-12T03:23:55.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:55.713+0000 7fb3b2a27640 1 -- 172.21.15.27:0/445784972 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb3ac0cf660 msgr2=0x7fb3ac0d2780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:55.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:55.713+0000 7fb3b2a27640 1 --2- 172.21.15.27:0/445784972 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb3ac0cf660 0x7fb3ac0d2780 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fb3a4009d90 tx=0x7fb3a402a680 comp rx=0 tx=0).stop 2024-09-12T03:23:55.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:55.713+0000 7fb3b2a27640 1 -- 172.21.15.27:0/445784972 shutdown_connections 2024-09-12T03:23:55.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:55.713+0000 7fb3b2a27640 1 --2- 172.21.15.27:0/445784972 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb3ac0cf660 0x7fb3ac0d2780 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:55.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:55.713+0000 7fb3b2a27640 1 -- 172.21.15.27:0/445784972 >> 172.21.15.27:0/445784972 conn(0x7fb3ac073800 msgr2=0x7fb3ac09ba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:55.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:55.713+0000 7fb3b2a27640 1 -- 172.21.15.27:0/445784972 shutdown_connections 2024-09-12T03:23:55.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:55.713+0000 7fb3b2a27640 1 -- 172.21.15.27:0/445784972 wait complete. 2024-09-12T03:23:55.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:55.715+0000 7fb3b2a27640 1 Processor -- start 2024-09-12T03:23:55.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:55.715+0000 7fb3b2a27640 1 -- start start 2024-09-12T03:23:55.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:55.715+0000 7fb3b2a27640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb3ac06f360 0x7fb3ac15c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:55.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:55.715+0000 7fb3b2a27640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb3ac0b1ca0 0x7fb3ac159ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:55.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:55.715+0000 7fb3b2a27640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb3ac0cf660 0x7fb3ac15ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:55.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:55.715+0000 7fb3b2a27640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fb3ac06d340 con 0x7fb3ac0cf660 2024-09-12T03:23:55.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:55.715+0000 7fb3b2a27640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb3ac06d1c0 con 0x7fb3ac06f360 2024-09-12T03:23:55.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:55.715+0000 7fb3b2a27640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fb3ac0cfc40 con 0x7fb3ac0b1ca0 2024-09-12T03:23:55.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:55.716+0000 7fb3b0fc4640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb3ac0cf660 0x7fb3ac15ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:55.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:55.716+0000 7fb3b0fc4640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb3ac0cf660 0x7fb3ac15ad00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:53516/0 (socket says 172.21.15.27:53516) 2024-09-12T03:23:55.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:55.716+0000 7fb3b0fc4640 1 -- 172.21.15.27:0/3382060569 learned_addr learned my addr 172.21.15.27:0/3382060569 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:23:55.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:55.716+0000 7fb3ab7fe640 1 --2- 172.21.15.27:0/3382060569 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb3ac0b1ca0 0x7fb3ac159ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:55.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:55.716+0000 7fb3abfff640 1 --2- 172.21.15.27:0/3382060569 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb3ac06f360 0x7fb3ac15c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:55.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:55.716+0000 7fb3b0fc4640 1 -- 172.21.15.27:0/3382060569 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb3ac0b1ca0 msgr2=0x7fb3ac159ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:55.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:55.716+0000 7fb3b0fc4640 1 --2- 172.21.15.27:0/3382060569 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb3ac0b1ca0 0x7fb3ac159ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:55.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:55.716+0000 7fb3b0fc4640 1 -- 172.21.15.27:0/3382060569 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb3ac06f360 msgr2=0x7fb3ac15c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:55.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:55.716+0000 7fb3b0fc4640 1 --2- 172.21.15.27:0/3382060569 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb3ac06f360 0x7fb3ac15c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:55.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:55.716+0000 7fb3b0fc4640 1 -- 172.21.15.27:0/3382060569 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3ac15a410 con 0x7fb3ac0cf660 2024-09-12T03:23:55.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:55.716+0000 7fb3b0fc4640 1 --2- 172.21.15.27:0/3382060569 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb3ac0cf660 0x7fb3ac15ad00 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7fb3a4009d90 tx=0x7fb3a4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:55.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:55.716+0000 7fb3a8ff9640 1 -- 172.21.15.27:0/3382060569 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3a403f070 con 0x7fb3ac0cf660 2024-09-12T03:23:55.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:55.716+0000 7fb3a8ff9640 1 -- 172.21.15.27:0/3382060569 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3a4031bc0 con 0x7fb3ac0cf660 2024-09-12T03:23:55.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:55.716+0000 7fb3b2a27640 1 -- 172.21.15.27:0/3382060569 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3ac15cd20 con 0x7fb3ac0cf660 2024-09-12T03:23:55.732 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:55.716+0000 7fb3a8ff9640 1 -- 172.21.15.27:0/3382060569 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3a4009450 con 0x7fb3ac0cf660 2024-09-12T03:23:55.732 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:55.716+0000 7fb3b2a27640 1 -- 172.21.15.27:0/3382060569 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3ac15d2e0 con 0x7fb3ac0cf660 2024-09-12T03:23:55.732 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:55.717+0000 7fb3b2a27640 1 -- 172.21.15.27:0/3382060569 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3ac0758e0 con 0x7fb3ac0cf660 2024-09-12T03:23:55.732 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:55.718+0000 7fb3a8ff9640 1 -- 172.21.15.27:0/3382060569 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 25) ==== 91365+0+0 (secure 0 0 0) 0x7fb3a403b410 con 0x7fb3ac0cf660 2024-09-12T03:23:55.732 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:55.718+0000 7fb3a8ff9640 1 --2- 172.21.15.27:0/3382060569 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fb39806c7f0 0x7fb39806ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:55.732 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:55.718+0000 7fb3a8ff9640 1 -- 172.21.15.27:0/3382060569 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(159..159 src has 1..159) ==== 5765+0+0 (secure 0 0 0) 0x7fb3a40aa7b0 con 0x7fb3ac0cf660 2024-09-12T03:23:55.733 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:55.719+0000 7fb3abfff640 1 --2- 172.21.15.27:0/3382060569 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fb39806c7f0 0x7fb39806ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:55.733 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:55.719+0000 7fb3abfff640 1 --2- 172.21.15.27:0/3382060569 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fb39806c7f0 0x7fb39806ecb0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fb39c000f60 tx=0x7fb39c009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:55.733 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:55.723+0000 7fb3a8ff9640 1 -- 172.21.15.27:0/3382060569 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fb3ac15a410 con 0x7fb3ac0cf660 2024-09-12T03:23:55.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:55.898+0000 7fb3b2a27640 1 -- 172.21.15.27:0/3382060569 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb3ac0b3b30 con 0x7fb3ac0cf660 2024-09-12T03:23:55.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:55.899+0000 7fb3a8ff9640 1 -- 172.21.15.27:0/3382060569 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v159) ==== 74+0+16797 (secure 0 0 0) 0x7fb3a407a710 con 0x7fb3ac0cf660 2024-09-12T03:23:55.900 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:23:55.900 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":159,"fsid":"d2df504e-2647-4385-a5db-9fdf69f487a0","created":"2024-09-12T03:16:07.778937+0000","modified":"2024-09-12T03:23:54.606527+0000","last_up_change":"2024-09-12T03:16:19.314778+0000","last_in_change":"2024-09-12T03:16:09.893086+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T03:16:20.097646+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs_metadata","create_time":"2024-09-12T03:23:52.574839+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"159","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_data","create_time":"2024-09-12T03:23:53.596556+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"159","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1851116c-1fb3-4dc9-9772-f90e2ef4c355","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":158,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6817","nonce":3002321488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6819","nonce":3002321488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6823","nonce":3002321488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6821","nonce":3002321488}]},"public_addr":"172.21.15.27:6817/3002321488","cluster_addr":"172.21.15.27:6819/3002321488","heartbeat_back_addr":"172.21.15.27:6823/3002321488","heartbeat_front_addr":"172.21.15.27:6821/3002321488","state":["exists","up"]},{"osd":1,"uuid":"f38ae17d-73eb-4204-b99d-975fa806fb87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":158,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6809","nonce":1613102886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6811","nonce":1613102886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6815","nonce":1613102886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6813","nonce":1613102886}]},"public_addr":"172.21.15.27:6809/1613102886","cluster_addr":"172.21.15.27:6811/1613102886","heartbeat_back_addr":"172.21.15.27:6815/1613102886","heartbeat_front_addr":"172.21.15.27:6813/1613102886","state":["exists","up"]},{"osd":2,"uuid":"fb12dcc8-bdd0-42b8-b46d-8b10106224bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":158,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6825","nonce":3534723355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6827","nonce":3534723355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6831","nonce":3534723355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6829","nonce":3534723355}]},"public_addr":"172.21.15.27:6825/3534723355","cluster_addr":"172.21.15.27:6827/3534723355","heartbeat_back_addr":"172.21.15.27:6831/3534723355","heartbeat_front_addr":"172.21.15.27:6829/3534723355","state":["exists","up"]},{"osd":3,"uuid":"d74d3f49-9c24-4b9f-98e2-3ecd7b5ef3b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":158,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6801","nonce":80661598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6803","nonce":80661598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6807","nonce":80661598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6805","nonce":80661598}]},"public_addr":"172.21.15.27:6801/80661598","cluster_addr":"172.21.15.27:6803/80661598","heartbeat_back_addr":"172.21.15.27:6807/80661598","heartbeat_front_addr":"172.21.15.27:6805/80661598","state":["exists","up"]},{"osd":4,"uuid":"2bf123f2-1b0d-41c4-aa3b-805bab267e59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":158,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6817","nonce":3943189827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6819","nonce":3943189827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6823","nonce":3943189827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6821","nonce":3943189827}]},"public_addr":"172.21.15.45:6817/3943189827","cluster_addr":"172.21.15.45:6819/3943189827","heartbeat_back_addr":"172.21.15.45:6823/3943189827","heartbeat_front_addr":"172.21.15.45:6821/3943189827","state":["exists","up"]},{"osd":5,"uuid":"c49ba5da-91c6-4a86-a97b-4aba89c253ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":158,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6809","nonce":3620760330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6811","nonce":3620760330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6815","nonce":3620760330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6813","nonce":3620760330}]},"public_addr":"172.21.15.45:6809/3620760330","cluster_addr":"172.21.15.45:6811/3620760330","heartbeat_back_addr":"172.21.15.45:6815/3620760330","heartbeat_front_addr":"172.21.15.45:6813/3620760330","state":["exists","up"]},{"osd":6,"uuid":"b0123475-89f7-4a34-9e33-d797b99dd9dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":158,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6801","nonce":1989083510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6803","nonce":1989083510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6807","nonce":1989083510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6805","nonce":1989083510}]},"public_addr":"172.21.15.45:6801/1989083510","cluster_addr":"172.21.15.45:6803/1989083510","heartbeat_back_addr":"172.21.15.45:6807/1989083510","heartbeat_front_addr":"172.21.15.45:6805/1989083510","state":["exists","up"]},{"osd":7,"uuid":"4dad773d-b7d8-46fc-8a10-a972aad27b29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":158,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6825","nonce":1513300294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6827","nonce":1513300294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6831","nonce":1513300294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6829","nonce":1513300294}]},"public_addr":"172.21.15.45:6825/1513300294","cluster_addr":"172.21.15.45:6827/1513300294","heartbeat_back_addr":"172.21.15.45:6831/1513300294","heartbeat_front_addr":"172.21.15.45:6829/1513300294","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.881855+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:17.133252+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.735339+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.500742+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.477571+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:17.167667+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.470291+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.873743+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]},{"pool":19,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T03:23:55.902 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:55.901+0000 7fb3b2a27640 1 -- 172.21.15.27:0/3382060569 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fb39806c7f0 msgr2=0x7fb39806ecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:55.902 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:55.901+0000 7fb3b2a27640 1 --2- 172.21.15.27:0/3382060569 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fb39806c7f0 0x7fb39806ecb0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fb39c000f60 tx=0x7fb39c009040 comp rx=0 tx=0).stop 2024-09-12T03:23:55.902 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:55.901+0000 7fb3b2a27640 1 -- 172.21.15.27:0/3382060569 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb3ac0cf660 msgr2=0x7fb3ac15ad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:55.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:55.901+0000 7fb3b2a27640 1 --2- 172.21.15.27:0/3382060569 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb3ac0cf660 0x7fb3ac15ad00 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7fb3a4009d90 tx=0x7fb3a4004a40 comp rx=0 tx=0).stop 2024-09-12T03:23:55.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:55.901+0000 7fb3b2a27640 1 -- 172.21.15.27:0/3382060569 shutdown_connections 2024-09-12T03:23:55.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:55.901+0000 7fb3b2a27640 1 --2- 172.21.15.27:0/3382060569 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fb39806c7f0 0x7fb39806ecb0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:55.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:55.901+0000 7fb3b2a27640 1 --2- 172.21.15.27:0/3382060569 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb3ac0cf660 0x7fb3ac15ad00 unknown :-1 s=CLOSED pgs=1674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:55.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:55.901+0000 7fb3b2a27640 1 --2- 172.21.15.27:0/3382060569 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb3ac0b1ca0 0x7fb3ac159ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:55.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:55.901+0000 7fb3b2a27640 1 --2- 172.21.15.27:0/3382060569 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb3ac06f360 0x7fb3ac15c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:55.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:55.901+0000 7fb3b2a27640 1 -- 172.21.15.27:0/3382060569 >> 172.21.15.27:0/3382060569 conn(0x7fb3ac073800 msgr2=0x7fb3ac0b2080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:55.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:55.901+0000 7fb3b2a27640 1 -- 172.21.15.27:0/3382060569 shutdown_connections 2024-09-12T03:23:55.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:55.901+0000 7fb3b2a27640 1 -- 172.21.15.27:0/3382060569 wait complete. 2024-09-12T03:23:55.913 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-12T03:23:56.018 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:56.017+0000 7fa99023c640 1 Processor -- start 2024-09-12T03:23:56.019 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:56.018+0000 7fa99023c640 1 -- start start 2024-09-12T03:23:56.020 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:56.018+0000 7fa99023c640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa988162040 0x7fa9880d1da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:56.020 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:56.018+0000 7fa99023c640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fa9880d3790 0x7fa9880b7a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:56.020 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:56.018+0000 7fa99023c640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa9880d2e30 con 0x7fa98806d8c0 2024-09-12T03:23:56.020 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:56.018+0000 7fa99023c640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7fa9880b7f90 con 0x7fa9880d3790 2024-09-12T03:23:56.020 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:56.018+0000 7fa99023c640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fa988162420 con 0x7fa988162040 2024-09-12T03:23:56.020 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:56.019+0000 7fa98d7d7640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fa9880d3790 0x7fa9880b7a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:56.020 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:56.019+0000 7fa98d7d7640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fa9880d3790 0x7fa9880b7a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:53524/0 (socket says 172.21.15.27:53524) 2024-09-12T03:23:56.020 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:56.019+0000 7fa98e7d9640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fa98806d8c0 0x7fa9880cf320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:33934/0 (socket says 172.21.15.27:33934) 2024-09-12T03:23:56.020 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:56.019+0000 7fa98d7d7640 1 -- 172.21.15.27:0/476921101 learned_addr learned my addr 172.21.15.27:0/476921101 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:23:56.020 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:56.019+0000 7fa98dfd8640 1 --2- 172.21.15.27:0/476921101 >> v2:172.21.15.45:3300/0 conn(0x7fa988162040 0x7fa9880d1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:56.021 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:56.019+0000 7fa98d7d7640 1 -- 172.21.15.27:0/476921101 >> v2:172.21.15.45:3300/0 conn(0x7fa988162040 msgr2=0x7fa9880d1da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:56.021 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:56.019+0000 7fa98d7d7640 1 --2- 172.21.15.27:0/476921101 >> v2:172.21.15.45:3300/0 conn(0x7fa988162040 0x7fa9880d1da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:56.021 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:56.019+0000 7fa98d7d7640 1 -- 172.21.15.27:0/476921101 >> v1:172.21.15.27:6789/0 conn(0x7fa98806d8c0 legacy=0x7fa9880cf320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:56.021 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:56.019+0000 7fa98d7d7640 1 -- 172.21.15.27:0/476921101 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9880d30d0 con 0x7fa9880d3790 2024-09-12T03:23:56.022 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:56.020+0000 7fa98d7d7640 1 --2- 172.21.15.27:0/476921101 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa9880d3790 0x7fa9880b7a50 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 crypto rx=0x7fa9880d17b0 tx=0x7fa97c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=651604947cb7005c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:56.022 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:56.020+0000 7fa98cfd6640 1 -- 172.21.15.27:0/476921101 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa97c037070 con 0x7fa9880d3790 2024-09-12T03:23:56.022 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:56.020+0000 7fa98cfd6640 1 -- 172.21.15.27:0/476921101 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa97c032440 con 0x7fa9880d3790 2024-09-12T03:23:56.022 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:56.020+0000 7fa98cfd6640 1 -- 172.21.15.27:0/476921101 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa97c032810 con 0x7fa9880d3790 2024-09-12T03:23:56.022 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:56.020+0000 7fa99023c640 1 -- 172.21.15.27:0/476921101 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa9880d3790 msgr2=0x7fa9880b7a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:56.022 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:56.020+0000 7fa99023c640 1 --2- 172.21.15.27:0/476921101 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa9880d3790 0x7fa9880b7a50 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 crypto rx=0x7fa9880d17b0 tx=0x7fa97c02a270 comp rx=0 tx=0).stop 2024-09-12T03:23:56.022 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:56.020+0000 7fa99023c640 1 -- 172.21.15.27:0/476921101 shutdown_connections 2024-09-12T03:23:56.023 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:56.020+0000 7fa99023c640 1 --2- 172.21.15.27:0/476921101 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa9880d3790 0x7fa9880b7a50 unknown :-1 s=CLOSED pgs=1676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:56.023 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:56.020+0000 7fa99023c640 1 --2- 172.21.15.27:0/476921101 >> v2:172.21.15.45:3300/0 conn(0x7fa988162040 0x7fa9880d1da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:56.023 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:56.020+0000 7fa99023c640 1 -- 172.21.15.27:0/476921101 >> 172.21.15.27:0/476921101 conn(0x7fa9880b6700 msgr2=0x7fa9880b6b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:56.023 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:56.021+0000 7fa99023c640 1 -- 172.21.15.27:0/476921101 shutdown_connections 2024-09-12T03:23:56.023 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:56.021+0000 7fa99023c640 1 -- 172.21.15.27:0/476921101 wait complete. 2024-09-12T03:23:56.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:56.022+0000 7fa99023c640 1 Processor -- start 2024-09-12T03:23:56.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:56.022+0000 7fa99023c640 1 -- start start 2024-09-12T03:23:56.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:56.022+0000 7fa99023c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa98806d8c0 0x7fa988152c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:56.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:56.022+0000 7fa99023c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa9880d3790 0x7fa9880cd580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:56.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:56.022+0000 7fa99023c640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa988162040 0x7fa9880b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:56.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:56.022+0000 7fa99023c640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fa9880d4530 con 0x7fa98806d8c0 2024-09-12T03:23:56.026 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:56.023+0000 7fa99023c640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa9880cee00 con 0x7fa9880d3790 2024-09-12T03:23:56.026 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:56.023+0000 7fa99023c640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fa9880d4150 con 0x7fa988162040 2024-09-12T03:23:56.026 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:56.023+0000 7fa98dfd8640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa98806d8c0 0x7fa988152c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:56.026 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:56.023+0000 7fa98dfd8640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa98806d8c0 0x7fa988152c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:53540/0 (socket says 172.21.15.27:53540) 2024-09-12T03:23:56.026 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:56.023+0000 7fa98dfd8640 1 -- 172.21.15.27:0/1442044594 learned_addr learned my addr 172.21.15.27:0/1442044594 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:23:56.026 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:56.023+0000 7fa98d7d7640 1 --2- 172.21.15.27:0/1442044594 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa9880d3790 0x7fa9880cd580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:56.026 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:56.023+0000 7fa98e7d9640 1 --2- 172.21.15.27:0/1442044594 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa988162040 0x7fa9880b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:56.026 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:56.023+0000 7fa98dfd8640 1 -- 172.21.15.27:0/1442044594 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa988162040 msgr2=0x7fa9880b4e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:56.026 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:56.023+0000 7fa98dfd8640 1 --2- 172.21.15.27:0/1442044594 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa988162040 0x7fa9880b4e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:56.026 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:56.023+0000 7fa98dfd8640 1 -- 172.21.15.27:0/1442044594 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa9880d3790 msgr2=0x7fa9880cd580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:56.026 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:56.023+0000 7fa98dfd8640 1 --2- 172.21.15.27:0/1442044594 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa9880d3790 0x7fa9880cd580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:56.026 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:56.023+0000 7fa98dfd8640 1 -- 172.21.15.27:0/1442044594 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa988153170 con 0x7fa98806d8c0 2024-09-12T03:23:56.026 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:56.023+0000 7fa98d7d7640 1 --2- 172.21.15.27:0/1442044594 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa9880d3790 0x7fa9880cd580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:23:56.026 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:56.023+0000 7fa98dfd8640 1 --2- 172.21.15.27:0/1442044594 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa98806d8c0 0x7fa988152c30 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7fa97800efc0 tx=0x7fa97800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:56.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:56.023+0000 7fa9867fc640 1 -- 172.21.15.27:0/1442044594 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa978018020 con 0x7fa98806d8c0 2024-09-12T03:23:56.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:56.023+0000 7fa9867fc640 1 -- 172.21.15.27:0/1442044594 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa978014480 con 0x7fa98806d8c0 2024-09-12T03:23:56.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:56.023+0000 7fa99023c640 1 -- 172.21.15.27:0/1442044594 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9881f8170 con 0x7fa98806d8c0 2024-09-12T03:23:56.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:56.023+0000 7fa9867fc640 1 -- 172.21.15.27:0/1442044594 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa978014850 con 0x7fa98806d8c0 2024-09-12T03:23:56.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:56.023+0000 7fa99023c640 1 -- 172.21.15.27:0/1442044594 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9881f86f0 con 0x7fa98806d8c0 2024-09-12T03:23:56.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:56.024+0000 7fa99023c640 1 -- 172.21.15.27:0/1442044594 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9880b7a50 con 0x7fa98806d8c0 2024-09-12T03:23:56.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:56.025+0000 7fa9867fc640 1 -- 172.21.15.27:0/1442044594 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 25) ==== 91365+0+0 (secure 0 0 0) 0x7fa978005d10 con 0x7fa98806d8c0 2024-09-12T03:23:56.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:56.025+0000 7fa9867fc640 1 --2- 172.21.15.27:0/1442044594 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fa97406c7c0 0x7fa97406ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:56.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:56.025+0000 7fa9867fc640 1 -- 172.21.15.27:0/1442044594 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(159..159 src has 1..159) ==== 5765+0+0 (secure 0 0 0) 0x7fa97808a450 con 0x7fa98806d8c0 2024-09-12T03:23:56.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:56.025+0000 7fa98d7d7640 1 --2- 172.21.15.27:0/1442044594 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fa97406c7c0 0x7fa97406ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:56.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:56.026+0000 7fa98d7d7640 1 --2- 172.21.15.27:0/1442044594 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fa97406c7c0 0x7fa97406ec80 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fa97c034000 tx=0x7fa97c02e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:56.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:56.030+0000 7fa9867fc640 1 -- 172.21.15.27:0/1442044594 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fa978022030 con 0x7fa98806d8c0 2024-09-12T03:23:56.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:56.242+0000 7fa99023c640 1 -- 172.21.15.27:0/1442044594 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fa9880b3cb0 con 0x7fa98806d8c0 2024-09-12T03:23:56.264 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:56.264+0000 7fa9867fc640 1 -- 172.21.15.27:0/1442044594 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v98) ==== 118+0+0 (secure 0 0 0) 0x7fa988153170 con 0x7fa98806d8c0 2024-09-12T03:23:56.267 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:56.265+0000 7fa99023c640 1 -- 172.21.15.27:0/1442044594 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fa97406c7c0 msgr2=0x7fa97406ec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:56.267 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:56.265+0000 7fa99023c640 1 --2- 172.21.15.27:0/1442044594 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fa97406c7c0 0x7fa97406ec80 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fa97c034000 tx=0x7fa97c02e000 comp rx=0 tx=0).stop 2024-09-12T03:23:56.267 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:56.265+0000 7fa99023c640 1 -- 172.21.15.27:0/1442044594 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa98806d8c0 msgr2=0x7fa988152c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:56.267 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:56.265+0000 7fa99023c640 1 --2- 172.21.15.27:0/1442044594 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa98806d8c0 0x7fa988152c30 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7fa97800efc0 tx=0x7fa97800c490 comp rx=0 tx=0).stop 2024-09-12T03:23:56.267 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:56.266+0000 7fa99023c640 1 -- 172.21.15.27:0/1442044594 shutdown_connections 2024-09-12T03:23:56.267 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:56.266+0000 7fa99023c640 1 --2- 172.21.15.27:0/1442044594 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fa988162040 0x7fa9880b4e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:56.267 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:56.266+0000 7fa99023c640 1 --2- 172.21.15.27:0/1442044594 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa9880d3790 0x7fa9880cd580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:56.267 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:56.266+0000 7fa99023c640 1 --2- 172.21.15.27:0/1442044594 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fa97406c7c0 0x7fa97406ec80 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:56.267 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:56.266+0000 7fa99023c640 1 --2- 172.21.15.27:0/1442044594 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa98806d8c0 0x7fa988152c30 unknown :-1 s=CLOSED pgs=1677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:56.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:56.266+0000 7fa99023c640 1 -- 172.21.15.27:0/1442044594 >> 172.21.15.27:0/1442044594 conn(0x7fa9880b6700 msgr2=0x7fa9880b3ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:56.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:56.266+0000 7fa99023c640 1 -- 172.21.15.27:0/1442044594 shutdown_connections 2024-09-12T03:23:56.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:56.266+0000 7fa99023c640 1 -- 172.21.15.27:0/1442044594 wait complete. 2024-09-12T03:23:56.279 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-12T03:23:56.279 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-12T03:23:56.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:56.383+0000 7f411265c640 1 Processor -- start 2024-09-12T03:23:56.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:56.383+0000 7f411265c640 1 -- start start 2024-09-12T03:23:56.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:56.384+0000 7f411265c640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f410c0b6230 0x7f410c0b66c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:56.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:56.384+0000 7f411265c640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f410c15ed70 con 0x7f410c15e990 2024-09-12T03:23:56.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:56.384+0000 7f411265c640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f410c0b7d10 con 0x7f410c0b3600 2024-09-12T03:23:56.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:56.384+0000 7f411265c640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f410c0d2a30 con 0x7f410c0b6230 2024-09-12T03:23:56.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:56.384+0000 7f410b7fe640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f410c0b6230 0x7f410c0b66c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:56.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:56.384+0000 7f410bfff640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f410c15e990 0x7f410c0b7600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:33948/0 (socket says 172.21.15.27:33948) 2024-09-12T03:23:56.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:56.384+0000 7f410bfff640 1 -- 172.21.15.27:0/1315431268 learned_addr learned my addr 172.21.15.27:0/1315431268 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:23:56.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:56.384+0000 7f410b7fe640 1 -- 172.21.15.27:0/1315431268 >> v1:172.21.15.45:6789/0 conn(0x7f410c0b3600 legacy=0x7f410c0b3a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:56.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:56.384+0000 7f410b7fe640 1 -- 172.21.15.27:0/1315431268 >> v1:172.21.15.27:6789/0 conn(0x7f410c15e990 legacy=0x7f410c0b7600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:56.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:56.384+0000 7f410b7fe640 1 -- 172.21.15.27:0/1315431268 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f410c0d2d10 con 0x7f410c0b6230 2024-09-12T03:23:56.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:56.384+0000 7f410affd640 1 -- 172.21.15.27:0/1315431268 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1268033372 0 0) 0x7f410c15ed70 con 0x7f410c15e990 2024-09-12T03:23:56.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:56.385+0000 7f410b7fe640 1 --2- 172.21.15.27:0/1315431268 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f410c0b6230 0x7f410c0b66c0 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7f40f40097a0 tx=0x7f40f402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a15c7a2cf748af0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:56.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:56.385+0000 7f410affd640 1 -- 172.21.15.27:0/1315431268 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40f4037070 con 0x7f410c0b6230 2024-09-12T03:23:56.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:56.385+0000 7f410affd640 1 -- 172.21.15.27:0/1315431268 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40f4032440 con 0x7f410c0b6230 2024-09-12T03:23:56.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:56.385+0000 7f410affd640 1 -- 172.21.15.27:0/1315431268 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40f4032810 con 0x7f410c0b6230 2024-09-12T03:23:56.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:56.386+0000 7f411265c640 1 -- 172.21.15.27:0/1315431268 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f410c0b6230 msgr2=0x7f410c0b66c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:56.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:56.386+0000 7f411265c640 1 --2- 172.21.15.27:0/1315431268 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f410c0b6230 0x7f410c0b66c0 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7f40f40097a0 tx=0x7f40f402a230 comp rx=0 tx=0).stop 2024-09-12T03:23:56.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:56.386+0000 7f411265c640 1 -- 172.21.15.27:0/1315431268 shutdown_connections 2024-09-12T03:23:56.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:56.386+0000 7f411265c640 1 --2- 172.21.15.27:0/1315431268 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f410c0b6230 0x7f410c0b66c0 unknown :-1 s=CLOSED pgs=1679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:56.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:56.386+0000 7f411265c640 1 -- 172.21.15.27:0/1315431268 >> 172.21.15.27:0/1315431268 conn(0x7f410c0d1230 msgr2=0x7f410c0d1640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:56.389 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:56.386+0000 7f411265c640 1 -- 172.21.15.27:0/1315431268 shutdown_connections 2024-09-12T03:23:56.389 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:56.386+0000 7f411265c640 1 -- 172.21.15.27:0/1315431268 wait complete. 2024-09-12T03:23:56.389 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:56.387+0000 7f411265c640 1 Processor -- start 2024-09-12T03:23:56.389 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:56.387+0000 7f411265c640 1 -- start start 2024-09-12T03:23:56.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:56.388+0000 7f411265c640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f410c0b3600 0x7f410c152f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:56.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:56.388+0000 7f411265c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f410c0b6230 0x7f410c0cdc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:56.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:56.388+0000 7f411265c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f410c15e990 0x7f410c0ce140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:56.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:56.388+0000 7f411265c640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f410c0d35b0 con 0x7f410c0b6230 2024-09-12T03:23:56.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:56.388+0000 7f411265c640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f410c0d32d0 con 0x7f410c15e990 2024-09-12T03:23:56.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:56.388+0000 7f411265c640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f410c0d3ad0 con 0x7f410c0b3600 2024-09-12T03:23:56.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:56.388+0000 7f410b7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f410c0b6230 0x7f410c0cdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:56.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:56.388+0000 7f410b7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f410c0b6230 0x7f410c0cdc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:53564/0 (socket says 172.21.15.27:53564) 2024-09-12T03:23:56.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:56.388+0000 7f410b7fe640 1 -- 172.21.15.27:0/1607351299 learned_addr learned my addr 172.21.15.27:0/1607351299 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:23:56.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:56.388+0000 7f4110bf9640 1 --2- 172.21.15.27:0/1607351299 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f410c15e990 0x7f410c0ce140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:56.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:56.388+0000 7f410bfff640 1 --2- 172.21.15.27:0/1607351299 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f410c0b3600 0x7f410c152f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:56.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:56.388+0000 7f410b7fe640 1 -- 172.21.15.27:0/1607351299 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f410c0b3600 msgr2=0x7f410c152f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:56.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:56.388+0000 7f410b7fe640 1 --2- 172.21.15.27:0/1607351299 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f410c0b3600 0x7f410c152f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:56.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:56.388+0000 7f410b7fe640 1 -- 172.21.15.27:0/1607351299 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f410c15e990 msgr2=0x7f410c0ce140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:56.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:56.388+0000 7f410b7fe640 1 --2- 172.21.15.27:0/1607351299 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f410c15e990 0x7f410c0ce140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:56.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:56.389+0000 7f410b7fe640 1 -- 172.21.15.27:0/1607351299 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f410c0cca70 con 0x7f410c0b6230 2024-09-12T03:23:56.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:56.389+0000 7f4110bf9640 1 --2- 172.21.15.27:0/1607351299 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f410c15e990 0x7f410c0ce140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:23:56.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:56.389+0000 7f410b7fe640 1 --2- 172.21.15.27:0/1607351299 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f410c0b6230 0x7f410c0cdc00 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7f40f4035040 tx=0x7f40f40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:56.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:56.389+0000 7f4108ff9640 1 -- 172.21.15.27:0/1607351299 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40f4037050 con 0x7f410c0b6230 2024-09-12T03:23:56.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:56.389+0000 7f4108ff9640 1 -- 172.21.15.27:0/1607351299 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40f4038040 con 0x7f410c0b6230 2024-09-12T03:23:56.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:56.389+0000 7f4108ff9640 1 -- 172.21.15.27:0/1607351299 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40f4004470 con 0x7f410c0b6230 2024-09-12T03:23:56.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:56.389+0000 7f411265c640 1 -- 172.21.15.27:0/1607351299 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f410c0ccd50 con 0x7f410c0b6230 2024-09-12T03:23:56.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:56.389+0000 7f411265c640 1 -- 172.21.15.27:0/1607351299 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f410c0cd310 con 0x7f410c0b6230 2024-09-12T03:23:56.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:56.390+0000 7f411265c640 1 -- 172.21.15.27:0/1607351299 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f410c0b66c0 con 0x7f410c0b6230 2024-09-12T03:23:56.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:56.390+0000 7f4108ff9640 1 -- 172.21.15.27:0/1607351299 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 25) ==== 91365+0+0 (secure 0 0 0) 0x7f40f4030880 con 0x7f410c0b6230 2024-09-12T03:23:56.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:56.391+0000 7f4108ff9640 1 --2- 172.21.15.27:0/1607351299 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f40ec06c7c0 0x7f40ec06ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:56.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:56.391+0000 7f4108ff9640 1 -- 172.21.15.27:0/1607351299 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(160..160 src has 1..160) ==== 5765+0+0 (secure 0 0 0) 0x7f40f40aa180 con 0x7f410c0b6230 2024-09-12T03:23:56.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:56.391+0000 7f410bfff640 1 --2- 172.21.15.27:0/1607351299 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f40ec06c7c0 0x7f40ec06ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:56.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:56.391+0000 7f410bfff640 1 --2- 172.21.15.27:0/1607351299 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f40ec06c7c0 0x7f40ec06ec80 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f40fc000f60 tx=0x7f40fc007d00 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:56.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:56.395+0000 7f4108ff9640 1 -- 172.21.15.27:0/1607351299 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f40f407a850 con 0x7f410c0b6230 2024-09-12T03:23:56.555 INFO:tasks.ceph.mds.c.smithi027.stderr:2024-09-12T03:23:56.554+0000 7fbf6f5ca640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-12T03:23:56.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:56.608+0000 7f411265c640 1 -- 172.21.15.27:0/1607351299 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f410c0caaf0 con 0x7f410c0b6230 2024-09-12T03:23:57.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:57.274+0000 7f4108ff9640 1 -- 172.21.15.27:0/1607351299 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v99) ==== 122+0+0 (secure 0 0 0) 0x7f40f407a150 con 0x7f410c0b6230 2024-09-12T03:23:57.278 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:57.277+0000 7f411265c640 1 -- 172.21.15.27:0/1607351299 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f40ec06c7c0 msgr2=0x7f40ec06ec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:57.278 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:57.277+0000 7f411265c640 1 --2- 172.21.15.27:0/1607351299 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f40ec06c7c0 0x7f40ec06ec80 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f40fc000f60 tx=0x7f40fc007d00 comp rx=0 tx=0).stop 2024-09-12T03:23:57.278 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:57.277+0000 7f411265c640 1 -- 172.21.15.27:0/1607351299 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f410c0b6230 msgr2=0x7f410c0cdc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:57.278 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:57.277+0000 7f411265c640 1 --2- 172.21.15.27:0/1607351299 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f410c0b6230 0x7f410c0cdc00 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7f40f4035040 tx=0x7f40f40046c0 comp rx=0 tx=0).stop 2024-09-12T03:23:57.278 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:57.277+0000 7f411265c640 1 -- 172.21.15.27:0/1607351299 shutdown_connections 2024-09-12T03:23:57.278 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:57.277+0000 7f411265c640 1 --2- 172.21.15.27:0/1607351299 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f410c15e990 0x7f410c0ce140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:57.278 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:57.277+0000 7f411265c640 1 --2- 172.21.15.27:0/1607351299 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f40ec06c7c0 0x7f40ec06ec80 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:57.278 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:57.277+0000 7f411265c640 1 --2- 172.21.15.27:0/1607351299 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f410c0b6230 0x7f410c0cdc00 unknown :-1 s=CLOSED pgs=1680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:57.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:57.277+0000 7f411265c640 1 --2- 172.21.15.27:0/1607351299 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f410c0b3600 0x7f410c152f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:57.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:57.277+0000 7f411265c640 1 -- 172.21.15.27:0/1607351299 >> 172.21.15.27:0/1607351299 conn(0x7f410c0d1230 msgr2=0x7f410c15cf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:57.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:57.277+0000 7f411265c640 1 -- 172.21.15.27:0/1607351299 shutdown_connections 2024-09-12T03:23:57.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:57.277+0000 7f411265c640 1 -- 172.21.15.27:0/1607351299 wait complete. 2024-09-12T03:23:57.290 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-12T03:23:57.290 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-12T03:23:57.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:57.393+0000 7f327a467640 1 Processor -- start 2024-09-12T03:23:57.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:57.393+0000 7f327a467640 1 -- start start 2024-09-12T03:23:57.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:57.394+0000 7f327a467640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f327415eb90 0x7f32740d2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:57.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:57.394+0000 7f327a467640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f32740d5200 0x7f32740d1010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:57.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:57.394+0000 7f327a467640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f32740d3760 con 0x7f32740d1550 2024-09-12T03:23:57.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:57.394+0000 7f327a467640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f32740b7d20 con 0x7f327415eb90 2024-09-12T03:23:57.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:57.394+0000 7f327a467640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f32740d42e0 con 0x7f32740d5200 2024-09-12T03:23:57.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:57.394+0000 7f3273fff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f327415eb90 0x7f32740d2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:57.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:57.395+0000 7f3273fff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f327415eb90 0x7f32740d2df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:53578/0 (socket says 172.21.15.27:53578) 2024-09-12T03:23:57.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:57.395+0000 7f3273fff640 1 -- 172.21.15.27:0/4016964190 learned_addr learned my addr 172.21.15.27:0/4016964190 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:23:57.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:57.395+0000 7f3273fff640 1 -- 172.21.15.27:0/4016964190 >> v2:172.21.15.45:3300/0 conn(0x7f32740d5200 msgr2=0x7f32740d1010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:57.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:57.395+0000 7f3273fff640 1 --2- 172.21.15.27:0/4016964190 >> v2:172.21.15.45:3300/0 conn(0x7f32740d5200 0x7f32740d1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:57.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:57.395+0000 7f3273fff640 1 -- 172.21.15.27:0/4016964190 >> v1:172.21.15.45:6789/0 conn(0x7f32740d1550 legacy=0x7f32740cf200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:57.397 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:57.395+0000 7f3273fff640 1 -- 172.21.15.27:0/4016964190 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32740d3a40 con 0x7f327415eb90 2024-09-12T03:23:57.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:57.395+0000 7f3273fff640 1 --2- 172.21.15.27:0/4016964190 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f327415eb90 0x7f32740d2df0 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7f32740d4750 tx=0x7f326402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5663d34e0e2e9a5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:57.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:57.396+0000 7f3272ffd640 1 -- 172.21.15.27:0/4016964190 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3264037070 con 0x7f327415eb90 2024-09-12T03:23:57.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:57.396+0000 7f3272ffd640 1 -- 172.21.15.27:0/4016964190 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3264032440 con 0x7f327415eb90 2024-09-12T03:23:57.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:57.396+0000 7f3272ffd640 1 -- 172.21.15.27:0/4016964190 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3264032810 con 0x7f327415eb90 2024-09-12T03:23:57.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:57.396+0000 7f327a467640 1 -- 172.21.15.27:0/4016964190 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f327415eb90 msgr2=0x7f32740d2df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:57.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:57.396+0000 7f327a467640 1 --2- 172.21.15.27:0/4016964190 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f327415eb90 0x7f32740d2df0 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7f32740d4750 tx=0x7f326402a230 comp rx=0 tx=0).stop 2024-09-12T03:23:57.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:57.396+0000 7f327a467640 1 -- 172.21.15.27:0/4016964190 shutdown_connections 2024-09-12T03:23:57.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:57.396+0000 7f327a467640 1 --2- 172.21.15.27:0/4016964190 >> v2:172.21.15.45:3300/0 conn(0x7f32740d5200 0x7f32740d1010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:57.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:57.396+0000 7f327a467640 1 --2- 172.21.15.27:0/4016964190 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f327415eb90 0x7f32740d2df0 unknown :-1 s=CLOSED pgs=1681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:57.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:57.396+0000 7f327a467640 1 -- 172.21.15.27:0/4016964190 >> 172.21.15.27:0/4016964190 conn(0x7f327406dce0 msgr2=0x7f327406e0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:57.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:57.396+0000 7f327a467640 1 -- 172.21.15.27:0/4016964190 shutdown_connections 2024-09-12T03:23:57.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:57.396+0000 7f327a467640 1 -- 172.21.15.27:0/4016964190 wait complete. 2024-09-12T03:23:57.400 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:57.397+0000 7f327a467640 1 Processor -- start 2024-09-12T03:23:57.401 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:57.398+0000 7f327a467640 1 -- start start 2024-09-12T03:23:57.401 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:57.398+0000 7f327a467640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f32740d1550 0x7f327415c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:57.401 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:57.398+0000 7f327a467640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f32740d5200 0x7f32740c7120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:57.401 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:57.398+0000 7f327a467640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f327415eb90 0x7f3274153790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:57.401 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:57.398+0000 7f327a467640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f32740d1960 con 0x7f32740d5200 2024-09-12T03:23:57.401 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:57.398+0000 7f327a467640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f32740d4000 con 0x7f32740d1550 2024-09-12T03:23:57.401 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:57.398+0000 7f327a467640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f32740d3330 con 0x7f327415eb90 2024-09-12T03:23:57.401 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:57.399+0000 7f32737fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f32740d5200 0x7f32740c7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:57.401 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:57.399+0000 7f32737fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f32740d5200 0x7f32740c7120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:53594/0 (socket says 172.21.15.27:53594) 2024-09-12T03:23:57.401 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:57.399+0000 7f32737fe640 1 -- 172.21.15.27:0/18123442 learned_addr learned my addr 172.21.15.27:0/18123442 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:23:57.401 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:57.399+0000 7f3273fff640 1 --2- 172.21.15.27:0/18123442 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f32740d1550 0x7f327415c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:57.402 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:57.399+0000 7f3278a04640 1 --2- 172.21.15.27:0/18123442 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f327415eb90 0x7f3274153790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:57.402 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:57.399+0000 7f32737fe640 1 -- 172.21.15.27:0/18123442 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f327415eb90 msgr2=0x7f3274153790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:57.402 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:57.399+0000 7f32737fe640 1 --2- 172.21.15.27:0/18123442 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f327415eb90 0x7f3274153790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:57.402 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:57.399+0000 7f32737fe640 1 -- 172.21.15.27:0/18123442 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f32740d1550 msgr2=0x7f327415c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:57.402 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:57.399+0000 7f32737fe640 1 --2- 172.21.15.27:0/18123442 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f32740d1550 0x7f327415c4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:57.402 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:57.399+0000 7f32737fe640 1 -- 172.21.15.27:0/18123442 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f327415cce0 con 0x7f32740d5200 2024-09-12T03:23:57.402 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:57.399+0000 7f32737fe640 1 --2- 172.21.15.27:0/18123442 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f32740d5200 0x7f32740c7120 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7f326800b450 tx=0x7f326800b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:57.402 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:57.399+0000 7f3270ff9640 1 -- 172.21.15.27:0/18123442 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3268015020 con 0x7f32740d5200 2024-09-12T03:23:57.402 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:57.399+0000 7f3270ff9640 1 -- 172.21.15.27:0/18123442 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3268011890 con 0x7f32740d5200 2024-09-12T03:23:57.402 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:57.399+0000 7f327a467640 1 -- 172.21.15.27:0/18123442 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f327415cfc0 con 0x7f32740d5200 2024-09-12T03:23:57.402 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:57.399+0000 7f3270ff9640 1 -- 172.21.15.27:0/18123442 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3268011c80 con 0x7f32740d5200 2024-09-12T03:23:57.402 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:57.399+0000 7f327a467640 1 -- 172.21.15.27:0/18123442 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3274159480 con 0x7f32740d5200 2024-09-12T03:23:57.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:57.400+0000 7f327a467640 1 -- 172.21.15.27:0/18123442 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f327415ef70 con 0x7f32740d5200 2024-09-12T03:23:57.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:57.401+0000 7f3270ff9640 1 -- 172.21.15.27:0/18123442 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 25) ==== 91365+0+0 (secure 0 0 0) 0x7f3268018020 con 0x7f32740d5200 2024-09-12T03:23:57.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:57.401+0000 7f3270ff9640 1 --2- 172.21.15.27:0/18123442 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f326006c7c0 0x7f326006ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:57.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:57.401+0000 7f3270ff9640 1 -- 172.21.15.27:0/18123442 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(160..160 src has 1..160) ==== 5765+0+0 (secure 0 0 0) 0x7f3268089680 con 0x7f32740d5200 2024-09-12T03:23:57.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:57.401+0000 7f3273fff640 1 --2- 172.21.15.27:0/18123442 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f326006c7c0 0x7f326006ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:57.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:57.402+0000 7f3273fff640 1 --2- 172.21.15.27:0/18123442 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f326006c7c0 0x7f326006ec80 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f3264034000 tx=0x7f326402e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:57.406 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:57.406+0000 7f3270ff9640 1 -- 172.21.15.27:0/18123442 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f326801f020 con 0x7f32740d5200 2024-09-12T03:23:57.617 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:57.616+0000 7f327a467640 1 -- 172.21.15.27:0/18123442 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3274159650 con 0x7f32740d5200 2024-09-12T03:23:57.618 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:57.617+0000 7f3270ff9640 1 -- 172.21.15.27:0/18123442 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 99 v99) ==== 94+0+5227 (secure 0 0 0) 0x7f326800d040 con 0x7f32740d5200 2024-09-12T03:23:57.618 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:23:57.618 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":99,"btime":"2024-09-12T03:23:57:270949+0000","default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7163,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/2493793567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2493793567},{"type":"v1","addr":"172.21.15.45:6835","nonce":2493793567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":96},{"gid":8056,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/1643775717","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":1643775717},{"type":"v1","addr":"172.21.15.45:6834","nonce":1643775717}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95},{"gid":8967,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6834/4129691413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":4129691413},{"type":"v1","addr":"172.21.15.27:6834","nonce":4129691413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95}],"filesystems":[{"mdsmap":{"epoch":99,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-12T03:23:54.606856+0000","modified":"2024-09-12T03:23:56.609988+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8970},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8970":{"gid":8970,"name":"c","rank":0,"incarnation":97,"state":"up:active","state_seq":3,"addr":"172.21.15.27:6835/2940099359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":2940099359},{"type":"v1","addr":"172.21.15.27:6835","nonce":2940099359}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[27],"metadata_pool":26,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8970,"qdb_cluster":[8970]},"id":13}]} 2024-09-12T03:23:57.618 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 99 2024-09-12T03:23:57.620 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:57.619+0000 7f327a467640 1 -- 172.21.15.27:0/18123442 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f326006c7c0 msgr2=0x7f326006ec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:57.620 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:57.619+0000 7f327a467640 1 --2- 172.21.15.27:0/18123442 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f326006c7c0 0x7f326006ec80 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f3264034000 tx=0x7f326402e000 comp rx=0 tx=0).stop 2024-09-12T03:23:57.620 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:57.619+0000 7f327a467640 1 -- 172.21.15.27:0/18123442 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f32740d5200 msgr2=0x7f32740c7120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:57.620 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:57.619+0000 7f327a467640 1 --2- 172.21.15.27:0/18123442 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f32740d5200 0x7f32740c7120 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7f326800b450 tx=0x7f326800b910 comp rx=0 tx=0).stop 2024-09-12T03:23:57.620 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:57.619+0000 7f327a467640 1 -- 172.21.15.27:0/18123442 shutdown_connections 2024-09-12T03:23:57.620 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:57.619+0000 7f327a467640 1 --2- 172.21.15.27:0/18123442 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f327415eb90 0x7f3274153790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:57.620 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:57.619+0000 7f327a467640 1 --2- 172.21.15.27:0/18123442 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f326006c7c0 0x7f326006ec80 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:57.621 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:57.619+0000 7f327a467640 1 --2- 172.21.15.27:0/18123442 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f32740d5200 0x7f32740c7120 unknown :-1 s=CLOSED pgs=1682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:57.621 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:57.619+0000 7f327a467640 1 --2- 172.21.15.27:0/18123442 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f32740d1550 0x7f327415c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:57.621 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:57.619+0000 7f327a467640 1 -- 172.21.15.27:0/18123442 >> 172.21.15.27:0/18123442 conn(0x7f327406dce0 msgr2=0x7f32740b8190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:57.621 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:57.619+0000 7f327a467640 1 -- 172.21.15.27:0/18123442 shutdown_connections 2024-09-12T03:23:57.621 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:57.619+0000 7f327a467640 1 -- 172.21.15.27:0/18123442 wait complete. 2024-09-12T03:23:57.631 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-12T03:23:57.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:57.733+0000 7f28f9ca6640 1 Processor -- start 2024-09-12T03:23:57.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:57.733+0000 7f28f9ca6640 1 -- start start 2024-09-12T03:23:57.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:57.734+0000 7f28f9ca6640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f28f40cf660 0x7f28f40d2780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:57.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:57.734+0000 7f28f9ca6640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f28f406cc40 con 0x7f28f40b1ca0 2024-09-12T03:23:57.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:57.734+0000 7f28f9ca6640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f28f40d1ce0 con 0x7f28f406f360 2024-09-12T03:23:57.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:57.734+0000 7f28f9ca6640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f28f40d22f0 con 0x7f28f40cf660 2024-09-12T03:23:57.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:57.734+0000 7f28f37fe640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f28f40b1ca0 0x7f28f415f830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:33956/0 (socket says 172.21.15.27:33956) 2024-09-12T03:23:57.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:57.734+0000 7f28f37fe640 1 -- 172.21.15.27:0/3685063174 learned_addr learned my addr 172.21.15.27:0/3685063174 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:23:57.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:57.734+0000 7f28f3fff640 1 --2- 172.21.15.27:0/3685063174 >> v2:172.21.15.45:3300/0 conn(0x7f28f40cf660 0x7f28f40d2780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:57.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:57.734+0000 7f28f27fc640 1 -- 172.21.15.27:0/3685063174 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2938284705 0 0) 0x7f28f406cc40 con 0x7f28f40b1ca0 2024-09-12T03:23:57.738 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:57.734+0000 7f28f27fc640 1 -- 172.21.15.27:0/3685063174 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f28dc003520 con 0x7f28f40b1ca0 2024-09-12T03:23:57.738 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:57.735+0000 7f28f27fc640 1 -- 172.21.15.27:0/3685063174 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 505306682 0 0) 0x7f28dc003520 con 0x7f28f40b1ca0 2024-09-12T03:23:57.738 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:57.735+0000 7f28f27fc640 1 -- 172.21.15.27:0/3685063174 >> v2:172.21.15.45:3300/0 conn(0x7f28f40cf660 msgr2=0x7f28f40d2780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:57.738 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:57.735+0000 7f28f27fc640 1 --2- 172.21.15.27:0/3685063174 >> v2:172.21.15.45:3300/0 conn(0x7f28f40cf660 0x7f28f40d2780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:57.738 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:57.735+0000 7f28f27fc640 1 -- 172.21.15.27:0/3685063174 >> v1:172.21.15.45:6789/0 conn(0x7f28f406f360 legacy=0x7f28f40cef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:57.738 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:57.735+0000 7f28f27fc640 1 -- 172.21.15.27:0/3685063174 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28f406cee0 con 0x7f28f40b1ca0 2024-09-12T03:23:57.738 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:57.735+0000 7f28f27fc640 1 -- 172.21.15.27:0/3685063174 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 847755213 0 0) 0x7f28e4001670 con 0x7f28f40b1ca0 2024-09-12T03:23:57.738 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:57.735+0000 7f28f27fc640 1 -- 172.21.15.27:0/3685063174 >> v1:172.21.15.27:6789/0 conn(0x7f28f40b1ca0 legacy=0x7f28f415f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:57.738 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:57.735+0000 7f28f27fc640 1 --2- 172.21.15.27:0/3685063174 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f28dc0039b0 0x7f28dc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:57.738 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:57.735+0000 7f28f27fc640 1 --2- 172.21.15.27:0/3685063174 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f28dc004800 0x7f28dc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:57.738 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:57.735+0000 7f28f27fc640 1 --2- 172.21.15.27:0/3685063174 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f28dc0051c0 0x7f28dc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:57.738 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:57.735+0000 7f28f27fc640 1 -- 172.21.15.27:0/3685063174 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f28f40d22f0 con 0x7f28dc0039b0 2024-09-12T03:23:57.738 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:57.735+0000 7f28f27fc640 1 -- 172.21.15.27:0/3685063174 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f28dc00c060 con 0x7f28dc0051c0 2024-09-12T03:23:57.738 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:57.735+0000 7f28f27fc640 1 -- 172.21.15.27:0/3685063174 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f28dc00c260 con 0x7f28dc004800 2024-09-12T03:23:57.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:57.735+0000 7f28f37fe640 1 --2- 172.21.15.27:0/3685063174 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f28dc0039b0 0x7f28dc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:57.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:57.735+0000 7f28f3fff640 1 --2- 172.21.15.27:0/3685063174 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f28dc0051c0 0x7f28dc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:57.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:57.735+0000 7f28f2ffd640 1 --2- 172.21.15.27:0/3685063174 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f28dc004800 0x7f28dc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:57.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:57.735+0000 7f28f37fe640 1 -- 172.21.15.27:0/3685063174 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f28dc004800 msgr2=0x7f28dc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:57.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:57.735+0000 7f28f37fe640 1 --2- 172.21.15.27:0/3685063174 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f28dc004800 0x7f28dc004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:57.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:57.735+0000 7f28f37fe640 1 -- 172.21.15.27:0/3685063174 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f28dc0051c0 msgr2=0x7f28dc00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:57.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:57.735+0000 7f28f37fe640 1 --2- 172.21.15.27:0/3685063174 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f28dc0051c0 0x7f28dc00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:57.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:57.735+0000 7f28f37fe640 1 -- 172.21.15.27:0/3685063174 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28dc003520 con 0x7f28dc0039b0 2024-09-12T03:23:57.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:57.735+0000 7f28f3fff640 1 --2- 172.21.15.27:0/3685063174 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f28dc0051c0 0x7f28dc00b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:23:57.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:57.735+0000 7f28f3fff640 1 -- 172.21.15.27:0/3685063174 reap_dead start 2024-09-12T03:23:57.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:57.736+0000 7f28f37fe640 1 --2- 172.21.15.27:0/3685063174 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f28dc0039b0 0x7f28dc003dd0 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7f28e400c240 tx=0x7f28dc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c2c726ce369463f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:57.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:57.736+0000 7f28f27fc640 1 -- 172.21.15.27:0/3685063174 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28e4007d10 con 0x7f28dc0039b0 2024-09-12T03:23:57.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:57.736+0000 7f28f27fc640 1 -- 172.21.15.27:0/3685063174 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28e40058d0 con 0x7f28dc0039b0 2024-09-12T03:23:57.740 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:57.736+0000 7f28f27fc640 1 -- 172.21.15.27:0/3685063174 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28e4005d20 con 0x7f28dc0039b0 2024-09-12T03:23:57.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:57.737+0000 7f28f9ca6640 1 -- 172.21.15.27:0/3685063174 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f28dc0039b0 msgr2=0x7f28dc003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:57.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:57.737+0000 7f28f9ca6640 1 --2- 172.21.15.27:0/3685063174 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f28dc0039b0 0x7f28dc003dd0 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7f28e400c240 tx=0x7f28dc005830 comp rx=0 tx=0).stop 2024-09-12T03:23:57.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:57.737+0000 7f28f9ca6640 1 -- 172.21.15.27:0/3685063174 shutdown_connections 2024-09-12T03:23:57.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:57.737+0000 7f28f9ca6640 1 --2- 172.21.15.27:0/3685063174 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f28dc0039b0 0x7f28dc003dd0 unknown :-1 s=CLOSED pgs=1684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:57.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:57.737+0000 7f28f9ca6640 1 -- 172.21.15.27:0/3685063174 >> 172.21.15.27:0/3685063174 conn(0x7f28f4073800 msgr2=0x7f28f409ba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:57.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:57.737+0000 7f28f9ca6640 1 -- 172.21.15.27:0/3685063174 shutdown_connections 2024-09-12T03:23:57.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:57.737+0000 7f28f9ca6640 1 -- 172.21.15.27:0/3685063174 wait complete. 2024-09-12T03:23:57.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:57.738+0000 7f28f9ca6640 1 Processor -- start 2024-09-12T03:23:57.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:57.738+0000 7f28f9ca6640 1 -- start start 2024-09-12T03:23:57.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:57.739+0000 7f28f9ca6640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f28dc0039b0 0x7f28f40b6a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:57.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:57.739+0000 7f28f9ca6640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f28f415d680 0x7f28f4153e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:57.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:57.739+0000 7f28f9ca6640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f28f40b54d0 0x7f28f40cdb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:57.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:57.739+0000 7f28f9ca6640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f28f406d340 con 0x7f28f40b54d0 2024-09-12T03:23:57.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:57.739+0000 7f28f9ca6640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f28f406d1c0 con 0x7f28f415d680 2024-09-12T03:23:57.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:57.739+0000 7f28f9ca6640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f28f40cfc40 con 0x7f28dc0039b0 2024-09-12T03:23:57.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:57.739+0000 7f28f3fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f28f40b54d0 0x7f28f40cdb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:57.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:57.739+0000 7f28f3fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f28f40b54d0 0x7f28f40cdb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:53606/0 (socket says 172.21.15.27:53606) 2024-09-12T03:23:57.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:57.739+0000 7f28f3fff640 1 -- 172.21.15.27:0/1213847868 learned_addr learned my addr 172.21.15.27:0/1213847868 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:23:57.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:57.739+0000 7f28f37fe640 1 --2- 172.21.15.27:0/1213847868 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f28dc0039b0 0x7f28f40b6a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:57.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:57.739+0000 7f28f2ffd640 1 --2- 172.21.15.27:0/1213847868 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f28f415d680 0x7f28f4153e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:57.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:57.739+0000 7f28f3fff640 1 -- 172.21.15.27:0/1213847868 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f28dc0039b0 msgr2=0x7f28f40b6a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:57.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:57.739+0000 7f28f3fff640 1 --2- 172.21.15.27:0/1213847868 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f28dc0039b0 0x7f28f40b6a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:57.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:57.739+0000 7f28f3fff640 1 -- 172.21.15.27:0/1213847868 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f28f415d680 msgr2=0x7f28f4153e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:57.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:57.739+0000 7f28f3fff640 1 --2- 172.21.15.27:0/1213847868 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f28f415d680 0x7f28f4153e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:57.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:57.739+0000 7f28f3fff640 1 -- 172.21.15.27:0/1213847868 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28f40b59d0 con 0x7f28f40b54d0 2024-09-12T03:23:57.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:57.739+0000 7f28f3fff640 1 --2- 172.21.15.27:0/1213847868 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f28f40b54d0 0x7f28f40cdb60 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7f28f4069ba0 tx=0x7f28ec0115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:57.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:57.739+0000 7f28dbfff640 1 -- 172.21.15.27:0/1213847868 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28ec011c30 con 0x7f28f40b54d0 2024-09-12T03:23:57.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:57.740+0000 7f28dbfff640 1 -- 172.21.15.27:0/1213847868 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28ec01b070 con 0x7f28f40b54d0 2024-09-12T03:23:57.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:57.740+0000 7f28f9ca6640 1 -- 172.21.15.27:0/1213847868 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28f40b5cb0 con 0x7f28f40b54d0 2024-09-12T03:23:57.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:57.740+0000 7f28dbfff640 1 -- 172.21.15.27:0/1213847868 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28ec011e10 con 0x7f28f40b54d0 2024-09-12T03:23:57.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:57.740+0000 7f28f9ca6640 1 -- 172.21.15.27:0/1213847868 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28f40b6270 con 0x7f28f40b54d0 2024-09-12T03:23:57.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:57.741+0000 7f28f9ca6640 1 -- 172.21.15.27:0/1213847868 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28f406f7d0 con 0x7f28f40b54d0 2024-09-12T03:23:57.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:57.741+0000 7f28dbfff640 1 -- 172.21.15.27:0/1213847868 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 25) ==== 91365+0+0 (secure 0 0 0) 0x7f28ec005d10 con 0x7f28f40b54d0 2024-09-12T03:23:57.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:57.742+0000 7f28dbfff640 1 --2- 172.21.15.27:0/1213847868 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f28e006c7f0 0x7f28e006ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:57.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:57.742+0000 7f28dbfff640 1 -- 172.21.15.27:0/1213847868 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(160..160 src has 1..160) ==== 5765+0+0 (secure 0 0 0) 0x7f28ec089af0 con 0x7f28f40b54d0 2024-09-12T03:23:57.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:57.742+0000 7f28f37fe640 1 --2- 172.21.15.27:0/1213847868 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f28e006c7f0 0x7f28e006ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:57.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:57.742+0000 7f28f37fe640 1 --2- 172.21.15.27:0/1213847868 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f28e006c7f0 0x7f28e006ecb0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f28e4033000 tx=0x7f28e402e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:57.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:57.746+0000 7f28dbfff640 1 -- 172.21.15.27:0/1213847868 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f28ec025020 con 0x7f28f40b54d0 2024-09-12T03:23:57.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:57.921+0000 7f28f9ca6640 1 -- 172.21.15.27:0/1213847868 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f28f40b14c0 con 0x7f28f40b54d0 2024-09-12T03:23:57.923 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:57.922+0000 7f28dbfff640 1 -- 172.21.15.27:0/1213847868 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v160) ==== 74+0+16784 (secure 0 0 0) 0x7f28ec05a010 con 0x7f28f40b54d0 2024-09-12T03:23:57.924 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:23:57.924 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":160,"fsid":"d2df504e-2647-4385-a5db-9fdf69f487a0","created":"2024-09-12T03:16:07.778937+0000","modified":"2024-09-12T03:23:56.250469+0000","last_up_change":"2024-09-12T03:16:19.314778+0000","last_in_change":"2024-09-12T03:16:09.893086+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T03:16:20.097646+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs_metadata","create_time":"2024-09-12T03:23:52.574839+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"159","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_data","create_time":"2024-09-12T03:23:53.596556+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"160","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1851116c-1fb3-4dc9-9772-f90e2ef4c355","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":158,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6817","nonce":3002321488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6819","nonce":3002321488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6823","nonce":3002321488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6821","nonce":3002321488}]},"public_addr":"172.21.15.27:6817/3002321488","cluster_addr":"172.21.15.27:6819/3002321488","heartbeat_back_addr":"172.21.15.27:6823/3002321488","heartbeat_front_addr":"172.21.15.27:6821/3002321488","state":["exists","up"]},{"osd":1,"uuid":"f38ae17d-73eb-4204-b99d-975fa806fb87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":158,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6809","nonce":1613102886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6811","nonce":1613102886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6815","nonce":1613102886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6813","nonce":1613102886}]},"public_addr":"172.21.15.27:6809/1613102886","cluster_addr":"172.21.15.27:6811/1613102886","heartbeat_back_addr":"172.21.15.27:6815/1613102886","heartbeat_front_addr":"172.21.15.27:6813/1613102886","state":["exists","up"]},{"osd":2,"uuid":"fb12dcc8-bdd0-42b8-b46d-8b10106224bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":158,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6825","nonce":3534723355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6827","nonce":3534723355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6831","nonce":3534723355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6829","nonce":3534723355}]},"public_addr":"172.21.15.27:6825/3534723355","cluster_addr":"172.21.15.27:6827/3534723355","heartbeat_back_addr":"172.21.15.27:6831/3534723355","heartbeat_front_addr":"172.21.15.27:6829/3534723355","state":["exists","up"]},{"osd":3,"uuid":"d74d3f49-9c24-4b9f-98e2-3ecd7b5ef3b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":158,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6801","nonce":80661598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6803","nonce":80661598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6807","nonce":80661598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6805","nonce":80661598}]},"public_addr":"172.21.15.27:6801/80661598","cluster_addr":"172.21.15.27:6803/80661598","heartbeat_back_addr":"172.21.15.27:6807/80661598","heartbeat_front_addr":"172.21.15.27:6805/80661598","state":["exists","up"]},{"osd":4,"uuid":"2bf123f2-1b0d-41c4-aa3b-805bab267e59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":158,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6817","nonce":3943189827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6819","nonce":3943189827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6823","nonce":3943189827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6821","nonce":3943189827}]},"public_addr":"172.21.15.45:6817/3943189827","cluster_addr":"172.21.15.45:6819/3943189827","heartbeat_back_addr":"172.21.15.45:6823/3943189827","heartbeat_front_addr":"172.21.15.45:6821/3943189827","state":["exists","up"]},{"osd":5,"uuid":"c49ba5da-91c6-4a86-a97b-4aba89c253ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":158,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6809","nonce":3620760330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6811","nonce":3620760330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6815","nonce":3620760330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6813","nonce":3620760330}]},"public_addr":"172.21.15.45:6809/3620760330","cluster_addr":"172.21.15.45:6811/3620760330","heartbeat_back_addr":"172.21.15.45:6815/3620760330","heartbeat_front_addr":"172.21.15.45:6813/3620760330","state":["exists","up"]},{"osd":6,"uuid":"b0123475-89f7-4a34-9e33-d797b99dd9dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":158,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6801","nonce":1989083510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6803","nonce":1989083510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6807","nonce":1989083510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6805","nonce":1989083510}]},"public_addr":"172.21.15.45:6801/1989083510","cluster_addr":"172.21.15.45:6803/1989083510","heartbeat_back_addr":"172.21.15.45:6807/1989083510","heartbeat_front_addr":"172.21.15.45:6805/1989083510","state":["exists","up"]},{"osd":7,"uuid":"4dad773d-b7d8-46fc-8a10-a972aad27b29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":158,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6825","nonce":1513300294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6827","nonce":1513300294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6831","nonce":1513300294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6829","nonce":1513300294}]},"public_addr":"172.21.15.45:6825/1513300294","cluster_addr":"172.21.15.45:6827/1513300294","heartbeat_back_addr":"172.21.15.45:6831/1513300294","heartbeat_front_addr":"172.21.15.45:6829/1513300294","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.881855+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:17.133252+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.735339+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.500742+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.477571+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:17.167667+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.470291+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.873743+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]},{"pool":19,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T03:23:57.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:57.925+0000 7f28f9ca6640 1 -- 172.21.15.27:0/1213847868 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f28e006c7f0 msgr2=0x7f28e006ecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:57.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:57.925+0000 7f28f9ca6640 1 --2- 172.21.15.27:0/1213847868 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f28e006c7f0 0x7f28e006ecb0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f28e4033000 tx=0x7f28e402e000 comp rx=0 tx=0).stop 2024-09-12T03:23:57.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:57.925+0000 7f28f9ca6640 1 -- 172.21.15.27:0/1213847868 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f28f40b54d0 msgr2=0x7f28f40cdb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:57.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:57.925+0000 7f28f9ca6640 1 --2- 172.21.15.27:0/1213847868 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f28f40b54d0 0x7f28f40cdb60 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7f28f4069ba0 tx=0x7f28ec0115a0 comp rx=0 tx=0).stop 2024-09-12T03:23:57.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:57.925+0000 7f28f9ca6640 1 -- 172.21.15.27:0/1213847868 shutdown_connections 2024-09-12T03:23:57.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:57.925+0000 7f28f9ca6640 1 --2- 172.21.15.27:0/1213847868 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f28e006c7f0 0x7f28e006ecb0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:57.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:57.925+0000 7f28f9ca6640 1 --2- 172.21.15.27:0/1213847868 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f28f40b54d0 0x7f28f40cdb60 unknown :-1 s=CLOSED pgs=1685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:57.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:57.925+0000 7f28f9ca6640 1 --2- 172.21.15.27:0/1213847868 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f28f415d680 0x7f28f4153e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:57.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:57.925+0000 7f28f9ca6640 1 --2- 172.21.15.27:0/1213847868 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f28dc0039b0 0x7f28f40b6a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:57.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:57.925+0000 7f28f9ca6640 1 -- 172.21.15.27:0/1213847868 >> 172.21.15.27:0/1213847868 conn(0x7f28f4073800 msgr2=0x7f28f40d14a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:57.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:57.925+0000 7f28f9ca6640 1 -- 172.21.15.27:0/1213847868 shutdown_connections 2024-09-12T03:23:57.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:57.925+0000 7f28f9ca6640 1 -- 172.21.15.27:0/1213847868 wait complete. 2024-09-12T03:23:57.938 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-12T03:23:57.938 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-12T03:23:58.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.039+0000 7fed31d9f640 1 Processor -- start 2024-09-12T03:23:58.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.039+0000 7fed31d9f640 1 -- start start 2024-09-12T03:23:58.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.040+0000 7fed31d9f640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fed2c06f0c0 0x7fed2c0b3560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:58.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.040+0000 7fed31d9f640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fed2c0ce410 con 0x7fed2c072d70 2024-09-12T03:23:58.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.040+0000 7fed31d9f640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fed2c0732c0 con 0x7fed2c15b2b0 2024-09-12T03:23:58.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.040+0000 7fed31d9f640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fed2c0720a0 con 0x7fed2c06f0c0 2024-09-12T03:23:58.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.040+0000 7fed2affd640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fed2c072d70 0x7fed2c06e9b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:33966/0 (socket says 172.21.15.27:33966) 2024-09-12T03:23:58.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.040+0000 7fed2affd640 1 -- 172.21.15.27:0/3781477892 learned_addr learned my addr 172.21.15.27:0/3781477892 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:23:58.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.040+0000 7fed2bfff640 1 --2- 172.21.15.27:0/3781477892 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fed2c06f0c0 0x7fed2c0b3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:58.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.040+0000 7fed2a7fc640 1 -- 172.21.15.27:0/3781477892 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3404767974 0 0) 0x7fed20001110 con 0x7fed2c072d70 2024-09-12T03:23:58.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.040+0000 7fed2a7fc640 1 -- 172.21.15.27:0/3781477892 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fed14005730 con 0x7fed2c072d70 2024-09-12T03:23:58.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.040+0000 7fed2a7fc640 1 -- 172.21.15.27:0/3781477892 <== mon.1 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1865095971 0 0) 0x7fed2c0732c0 con 0x7fed2c15b2b0 2024-09-12T03:23:58.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.040+0000 7fed2a7fc640 1 -- 172.21.15.27:0/3781477892 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fed20001110 con 0x7fed2c15b2b0 2024-09-12T03:23:58.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.041+0000 7fed2bfff640 1 -- 172.21.15.27:0/3781477892 >> v1:172.21.15.45:6789/0 conn(0x7fed2c15b2b0 legacy=0x7fed2c072430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:58.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.041+0000 7fed2bfff640 1 -- 172.21.15.27:0/3781477892 >> v1:172.21.15.27:6789/0 conn(0x7fed2c072d70 legacy=0x7fed2c06e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:58.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.041+0000 7fed2bfff640 1 -- 172.21.15.27:0/3781477892 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed2c071ae0 con 0x7fed2c06f0c0 2024-09-12T03:23:58.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.041+0000 7fed2bfff640 1 --2- 172.21.15.27:0/3781477892 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fed2c06f0c0 0x7fed2c0b3560 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fed24009dc0 tx=0x7fed2402a6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=96ef8fee37e90ccf server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:58.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.041+0000 7fed2a7fc640 1 -- 172.21.15.27:0/3781477892 <== mon.2 v2:172.21.15.45:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed24007740 con 0x7fed2c06f0c0 2024-09-12T03:23:58.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.042+0000 7fed2a7fc640 1 -- 172.21.15.27:0/3781477892 <== mon.2 v2:172.21.15.45:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed240078e0 con 0x7fed2c06f0c0 2024-09-12T03:23:58.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.042+0000 7fed2a7fc640 1 -- 172.21.15.27:0/3781477892 <== mon.2 v2:172.21.15.45:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed24007cb0 con 0x7fed2c06f0c0 2024-09-12T03:23:58.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.042+0000 7fed31d9f640 1 -- 172.21.15.27:0/3781477892 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fed2c06f0c0 msgr2=0x7fed2c0b3560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:58.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.042+0000 7fed31d9f640 1 --2- 172.21.15.27:0/3781477892 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fed2c06f0c0 0x7fed2c0b3560 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fed24009dc0 tx=0x7fed2402a6d0 comp rx=0 tx=0).stop 2024-09-12T03:23:58.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.042+0000 7fed31d9f640 1 -- 172.21.15.27:0/3781477892 shutdown_connections 2024-09-12T03:23:58.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.042+0000 7fed31d9f640 1 --2- 172.21.15.27:0/3781477892 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fed2c06f0c0 0x7fed2c0b3560 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:58.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.042+0000 7fed31d9f640 1 -- 172.21.15.27:0/3781477892 >> 172.21.15.27:0/3781477892 conn(0x7fed2c062c60 msgr2=0x7fed2c06bb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:58.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.042+0000 7fed31d9f640 1 -- 172.21.15.27:0/3781477892 shutdown_connections 2024-09-12T03:23:58.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.042+0000 7fed31d9f640 1 -- 172.21.15.27:0/3781477892 wait complete. 2024-09-12T03:23:58.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.043+0000 7fed31d9f640 1 Processor -- start 2024-09-12T03:23:58.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.043+0000 7fed31d9f640 1 -- start start 2024-09-12T03:23:58.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.044+0000 7fed31d9f640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fed2c06f0c0 0x7fed2c14a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:58.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.044+0000 7fed31d9f640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fed2c072d70 0x7fed2c15ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:58.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.044+0000 7fed31d9f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fed2c15b2b0 0x7fed2c15a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:58.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.044+0000 7fed31d9f640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fed2c15b690 con 0x7fed2c072d70 2024-09-12T03:23:58.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.044+0000 7fed31d9f640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fed2c0ce690 con 0x7fed2c15b2b0 2024-09-12T03:23:58.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.044+0000 7fed31d9f640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fed2c15b810 con 0x7fed2c06f0c0 2024-09-12T03:23:58.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.044+0000 7fed2affd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fed2c072d70 0x7fed2c15ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:58.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.044+0000 7fed2affd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fed2c072d70 0x7fed2c15ceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:53614/0 (socket says 172.21.15.27:53614) 2024-09-12T03:23:58.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.044+0000 7fed2affd640 1 -- 172.21.15.27:0/4012293834 learned_addr learned my addr 172.21.15.27:0/4012293834 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:23:58.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.044+0000 7fed2bfff640 1 --2- 172.21.15.27:0/4012293834 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fed2c15b2b0 0x7fed2c15a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:58.048 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.044+0000 7fed2b7fe640 1 --2- 172.21.15.27:0/4012293834 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fed2c06f0c0 0x7fed2c14a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:58.048 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.045+0000 7fed2affd640 1 -- 172.21.15.27:0/4012293834 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fed2c06f0c0 msgr2=0x7fed2c14a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:58.048 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.045+0000 7fed2affd640 1 --2- 172.21.15.27:0/4012293834 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fed2c06f0c0 0x7fed2c14a050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:58.048 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.045+0000 7fed2affd640 1 -- 172.21.15.27:0/4012293834 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fed2c15b2b0 msgr2=0x7fed2c15a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:58.048 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.045+0000 7fed2affd640 1 --2- 172.21.15.27:0/4012293834 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fed2c15b2b0 0x7fed2c15a340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:58.048 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.045+0000 7fed2affd640 1 -- 172.21.15.27:0/4012293834 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed2c159730 con 0x7fed2c072d70 2024-09-12T03:23:58.048 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.045+0000 7fed2affd640 1 --2- 172.21.15.27:0/4012293834 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fed2c072d70 0x7fed2c15ceb0 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7fed2000c730 tx=0x7fed2000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:58.048 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.045+0000 7fed13fff640 1 -- 172.21.15.27:0/4012293834 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed20017070 con 0x7fed2c072d70 2024-09-12T03:23:58.048 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.045+0000 7fed31d9f640 1 -- 172.21.15.27:0/4012293834 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed2c15be10 con 0x7fed2c072d70 2024-09-12T03:23:58.048 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.045+0000 7fed13fff640 1 -- 172.21.15.27:0/4012293834 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed20005ab0 con 0x7fed2c072d70 2024-09-12T03:23:58.048 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.045+0000 7fed13fff640 1 -- 172.21.15.27:0/4012293834 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed20007070 con 0x7fed2c072d70 2024-09-12T03:23:58.048 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.045+0000 7fed31d9f640 1 -- 172.21.15.27:0/4012293834 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed2c15c3d0 con 0x7fed2c072d70 2024-09-12T03:23:58.048 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.046+0000 7fed31d9f640 1 -- 172.21.15.27:0/4012293834 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed2c15caf0 con 0x7fed2c072d70 2024-09-12T03:23:58.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.046+0000 7fed13fff640 1 -- 172.21.15.27:0/4012293834 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 25) ==== 91365+0+0 (secure 0 0 0) 0x7fed2001a070 con 0x7fed2c072d70 2024-09-12T03:23:58.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.047+0000 7fed13fff640 1 --2- 172.21.15.27:0/4012293834 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fed1806c7c0 0x7fed1806ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:58.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.047+0000 7fed2b7fe640 1 --2- 172.21.15.27:0/4012293834 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fed1806c7c0 0x7fed1806ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:58.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.047+0000 7fed13fff640 1 -- 172.21.15.27:0/4012293834 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(160..160 src has 1..160) ==== 5765+0+0 (secure 0 0 0) 0x7fed20088a60 con 0x7fed2c072d70 2024-09-12T03:23:58.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.047+0000 7fed2b7fe640 1 --2- 172.21.15.27:0/4012293834 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fed1806c7c0 0x7fed1806ec80 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fed1c001650 tx=0x7fed1c009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:58.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.051+0000 7fed13fff640 1 -- 172.21.15.27:0/4012293834 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fed2c0ce410 con 0x7fed2c072d70 2024-09-12T03:23:58.225 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.224+0000 7fed31d9f640 1 -- 172.21.15.27:0/4012293834 --> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fed2c075440 con 0x7fed1806c7c0 2024-09-12T03:23:58.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.230+0000 7fed13fff640 1 -- 172.21.15.27:0/4012293834 <== mgr.4113 v2:172.21.15.27:6836/1226920526 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252416 (secure 0 0 0) 0x7fed2c075440 con 0x7fed1806c7c0 2024-09-12T03:23:58.234 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:23:58.236 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2024-09-12T03:23:58.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.233+0000 7fed31d9f640 1 -- 172.21.15.27:0/4012293834 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fed1806c7c0 msgr2=0x7fed1806ec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:58.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.233+0000 7fed31d9f640 1 --2- 172.21.15.27:0/4012293834 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fed1806c7c0 0x7fed1806ec80 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fed1c001650 tx=0x7fed1c009040 comp rx=0 tx=0).stop 2024-09-12T03:23:58.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.233+0000 7fed31d9f640 1 -- 172.21.15.27:0/4012293834 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fed2c072d70 msgr2=0x7fed2c15ceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:58.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.233+0000 7fed31d9f640 1 --2- 172.21.15.27:0/4012293834 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fed2c072d70 0x7fed2c15ceb0 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7fed2000c730 tx=0x7fed2000cbf0 comp rx=0 tx=0).stop 2024-09-12T03:23:58.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.233+0000 7fed31d9f640 1 -- 172.21.15.27:0/4012293834 shutdown_connections 2024-09-12T03:23:58.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.233+0000 7fed31d9f640 1 --2- 172.21.15.27:0/4012293834 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fed2c15b2b0 0x7fed2c15a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:58.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.233+0000 7fed31d9f640 1 --2- 172.21.15.27:0/4012293834 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fed1806c7c0 0x7fed1806ec80 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:58.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.233+0000 7fed31d9f640 1 --2- 172.21.15.27:0/4012293834 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fed2c072d70 0x7fed2c15ceb0 unknown :-1 s=CLOSED pgs=1687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:58.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.233+0000 7fed31d9f640 1 --2- 172.21.15.27:0/4012293834 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fed2c06f0c0 0x7fed2c14a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:58.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.233+0000 7fed31d9f640 1 -- 172.21.15.27:0/4012293834 >> 172.21.15.27:0/4012293834 conn(0x7fed2c062c60 msgr2=0x7fed2c071280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:58.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.233+0000 7fed31d9f640 1 -- 172.21.15.27:0/4012293834 shutdown_connections 2024-09-12T03:23:58.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.233+0000 7fed31d9f640 1 -- 172.21.15.27:0/4012293834 wait complete. 2024-09-12T03:23:58.245 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":205,"stamp":"2024-09-12T03:23:57.141038+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":223,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":166,"ondisk_log_size":166,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":59,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":287188,"kb_used_data":16856,"kb_used_omap":12,"kb_used_meta":270131,"kb_avail":754687532,"statfs":{"total":773094113280,"available":772800032768,"internally_reserved":0,"allocated":17260544,"data_stored":8275324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":276614704},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000821"},"pg_stats":[{"pgid":"27.3f","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-12T03:23:55.284859+0000","last_change":"2024-09-12T03:23:55.276559+0000","last_active":"2024-09-12T03:23:55.284859+0000","last_peered":"2024-09-12T03:23:55.284859+0000","last_clean":"2024-09-12T03:23:55.284859+0000","last_became_active":"2024-09-12T03:23:55.276270+0000","last_became_peered":"2024-09-12T03:23:55.276270+0000","last_unstale":"2024-09-12T03:23:55.284859+0000","last_undegraded":"2024-09-12T03:23:55.284859+0000","last_fullsized":"2024-09-12T03:23:55.284859+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_clean_scrub_stamp":"2024-09-12T03:23:54.239693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:28:08.158129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.3e","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-12T03:23:55.274187+0000","last_change":"2024-09-12T03:23:55.273001+0000","last_active":"2024-09-12T03:23:55.274187+0000","last_peered":"2024-09-12T03:23:55.274187+0000","last_clean":"2024-09-12T03:23:55.274187+0000","last_became_active":"2024-09-12T03:23:55.272677+0000","last_became_peered":"2024-09-12T03:23:55.272677+0000","last_unstale":"2024-09-12T03:23:55.274187+0000","last_undegraded":"2024-09-12T03:23:55.274187+0000","last_fullsized":"2024-09-12T03:23:55.274187+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_clean_scrub_stamp":"2024-09-12T03:23:54.239693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:15:19.279469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.3d","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-12T03:23:55.268540+0000","last_change":"2024-09-12T03:23:55.261370+0000","last_active":"2024-09-12T03:23:55.268540+0000","last_peered":"2024-09-12T03:23:55.268540+0000","last_clean":"2024-09-12T03:23:55.268540+0000","last_became_active":"2024-09-12T03:23:55.261076+0000","last_became_peered":"2024-09-12T03:23:55.261076+0000","last_unstale":"2024-09-12T03:23:55.268540+0000","last_undegraded":"2024-09-12T03:23:55.268540+0000","last_fullsized":"2024-09-12T03:23:55.268540+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_clean_scrub_stamp":"2024-09-12T03:23:54.239693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:57:08.826050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.3c","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-12T03:23:55.265525+0000","last_change":"2024-09-12T03:23:55.264349+0000","last_active":"2024-09-12T03:23:55.265525+0000","last_peered":"2024-09-12T03:23:55.265525+0000","last_clean":"2024-09-12T03:23:55.265525+0000","last_became_active":"2024-09-12T03:23:55.263996+0000","last_became_peered":"2024-09-12T03:23:55.263996+0000","last_unstale":"2024-09-12T03:23:55.265525+0000","last_undegraded":"2024-09-12T03:23:55.265525+0000","last_fullsized":"2024-09-12T03:23:55.265525+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_clean_scrub_stamp":"2024-09-12T03:23:54.239693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:06:40.478281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.3b","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-12T03:23:55.283087+0000","last_change":"2024-09-12T03:23:55.279885+0000","last_active":"2024-09-12T03:23:55.283087+0000","last_peered":"2024-09-12T03:23:55.283087+0000","last_clean":"2024-09-12T03:23:55.283087+0000","last_became_active":"2024-09-12T03:23:55.279633+0000","last_became_peered":"2024-09-12T03:23:55.279633+0000","last_unstale":"2024-09-12T03:23:55.283087+0000","last_undegraded":"2024-09-12T03:23:55.283087+0000","last_fullsized":"2024-09-12T03:23:55.283087+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_clean_scrub_stamp":"2024-09-12T03:23:54.239693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:56:55.717979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.3a","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-12T03:23:55.274418+0000","last_change":"2024-09-12T03:23:55.270327+0000","last_active":"2024-09-12T03:23:55.274418+0000","last_peered":"2024-09-12T03:23:55.274418+0000","last_clean":"2024-09-12T03:23:55.274418+0000","last_became_active":"2024-09-12T03:23:55.269294+0000","last_became_peered":"2024-09-12T03:23:55.269294+0000","last_unstale":"2024-09-12T03:23:55.274418+0000","last_undegraded":"2024-09-12T03:23:55.274418+0000","last_fullsized":"2024-09-12T03:23:55.274418+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_clean_scrub_stamp":"2024-09-12T03:23:54.239693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:41:49.085684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.39","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-12T03:23:55.285486+0000","last_change":"2024-09-12T03:23:55.278897+0000","last_active":"2024-09-12T03:23:55.285486+0000","last_peered":"2024-09-12T03:23:55.285486+0000","last_clean":"2024-09-12T03:23:55.285486+0000","last_became_active":"2024-09-12T03:23:55.278627+0000","last_became_peered":"2024-09-12T03:23:55.278627+0000","last_unstale":"2024-09-12T03:23:55.285486+0000","last_undegraded":"2024-09-12T03:23:55.285486+0000","last_fullsized":"2024-09-12T03:23:55.285486+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_clean_scrub_stamp":"2024-09-12T03:23:54.239693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:22:03.724702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.38","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-12T03:23:55.276152+0000","last_change":"2024-09-12T03:23:55.273644+0000","last_active":"2024-09-12T03:23:55.276152+0000","last_peered":"2024-09-12T03:23:55.276152+0000","last_clean":"2024-09-12T03:23:55.276152+0000","last_became_active":"2024-09-12T03:23:55.273372+0000","last_became_peered":"2024-09-12T03:23:55.273372+0000","last_unstale":"2024-09-12T03:23:55.276152+0000","last_undegraded":"2024-09-12T03:23:55.276152+0000","last_fullsized":"2024-09-12T03:23:55.276152+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_clean_scrub_stamp":"2024-09-12T03:23:54.239693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:03:36.855341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.37","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-12T03:23:55.269553+0000","last_change":"2024-09-12T03:23:55.267014+0000","last_active":"2024-09-12T03:23:55.269553+0000","last_peered":"2024-09-12T03:23:55.269553+0000","last_clean":"2024-09-12T03:23:55.269553+0000","last_became_active":"2024-09-12T03:23:55.266738+0000","last_became_peered":"2024-09-12T03:23:55.266738+0000","last_unstale":"2024-09-12T03:23:55.269553+0000","last_undegraded":"2024-09-12T03:23:55.269553+0000","last_fullsized":"2024-09-12T03:23:55.269553+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_clean_scrub_stamp":"2024-09-12T03:23:54.239693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:45:35.032950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.36","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-12T03:23:55.283056+0000","last_change":"2024-09-12T03:23:55.278628+0000","last_active":"2024-09-12T03:23:55.283056+0000","last_peered":"2024-09-12T03:23:55.283056+0000","last_clean":"2024-09-12T03:23:55.283056+0000","last_became_active":"2024-09-12T03:23:55.278359+0000","last_became_peered":"2024-09-12T03:23:55.278359+0000","last_unstale":"2024-09-12T03:23:55.283056+0000","last_undegraded":"2024-09-12T03:23:55.283056+0000","last_fullsized":"2024-09-12T03:23:55.283056+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_clean_scrub_stamp":"2024-09-12T03:23:54.239693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:47:44.168067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.35","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-12T03:23:55.268512+0000","last_change":"2024-09-12T03:23:55.259098+0000","last_active":"2024-09-12T03:23:55.268512+0000","last_peered":"2024-09-12T03:23:55.268512+0000","last_clean":"2024-09-12T03:23:55.268512+0000","last_became_active":"2024-09-12T03:23:55.258801+0000","last_became_peered":"2024-09-12T03:23:55.258801+0000","last_unstale":"2024-09-12T03:23:55.268512+0000","last_undegraded":"2024-09-12T03:23:55.268512+0000","last_fullsized":"2024-09-12T03:23:55.268512+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_clean_scrub_stamp":"2024-09-12T03:23:54.239693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:33:57.635706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.34","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-12T03:23:55.269259+0000","last_change":"2024-09-12T03:23:55.268174+0000","last_active":"2024-09-12T03:23:55.269259+0000","last_peered":"2024-09-12T03:23:55.269259+0000","last_clean":"2024-09-12T03:23:55.269259+0000","last_became_active":"2024-09-12T03:23:55.267893+0000","last_became_peered":"2024-09-12T03:23:55.267893+0000","last_unstale":"2024-09-12T03:23:55.269259+0000","last_undegraded":"2024-09-12T03:23:55.269259+0000","last_fullsized":"2024-09-12T03:23:55.269259+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_clean_scrub_stamp":"2024-09-12T03:23:54.239693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:35:21.699867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.33","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-12T03:23:55.271345+0000","last_change":"2024-09-12T03:23:55.268397+0000","last_active":"2024-09-12T03:23:55.271345+0000","last_peered":"2024-09-12T03:23:55.271345+0000","last_clean":"2024-09-12T03:23:55.271345+0000","last_became_active":"2024-09-12T03:23:55.268136+0000","last_became_peered":"2024-09-12T03:23:55.268136+0000","last_unstale":"2024-09-12T03:23:55.271345+0000","last_undegraded":"2024-09-12T03:23:55.271345+0000","last_fullsized":"2024-09-12T03:23:55.271345+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_clean_scrub_stamp":"2024-09-12T03:23:54.239693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:16:56.078635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.32","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-12T03:23:55.287454+0000","last_change":"2024-09-12T03:23:55.281329+0000","last_active":"2024-09-12T03:23:55.287454+0000","last_peered":"2024-09-12T03:23:55.287454+0000","last_clean":"2024-09-12T03:23:55.287454+0000","last_became_active":"2024-09-12T03:23:55.279160+0000","last_became_peered":"2024-09-12T03:23:55.279160+0000","last_unstale":"2024-09-12T03:23:55.287454+0000","last_undegraded":"2024-09-12T03:23:55.287454+0000","last_fullsized":"2024-09-12T03:23:55.287454+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_clean_scrub_stamp":"2024-09-12T03:23:54.239693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:59:09.406051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.31","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-12T03:23:55.285466+0000","last_change":"2024-09-12T03:23:55.278316+0000","last_active":"2024-09-12T03:23:55.285466+0000","last_peered":"2024-09-12T03:23:55.285466+0000","last_clean":"2024-09-12T03:23:55.285466+0000","last_became_active":"2024-09-12T03:23:55.277685+0000","last_became_peered":"2024-09-12T03:23:55.277685+0000","last_unstale":"2024-09-12T03:23:55.285466+0000","last_undegraded":"2024-09-12T03:23:55.285466+0000","last_fullsized":"2024-09-12T03:23:55.285466+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_clean_scrub_stamp":"2024-09-12T03:23:54.239693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:55:15.186982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.30","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-12T03:23:55.280913+0000","last_change":"2024-09-12T03:23:55.269395+0000","last_active":"2024-09-12T03:23:55.280913+0000","last_peered":"2024-09-12T03:23:55.280913+0000","last_clean":"2024-09-12T03:23:55.280913+0000","last_became_active":"2024-09-12T03:23:55.269129+0000","last_became_peered":"2024-09-12T03:23:55.269129+0000","last_unstale":"2024-09-12T03:23:55.280913+0000","last_undegraded":"2024-09-12T03:23:55.280913+0000","last_fullsized":"2024-09-12T03:23:55.280913+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_clean_scrub_stamp":"2024-09-12T03:23:54.239693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:38:59.255384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.2f","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-12T03:23:55.278192+0000","last_change":"2024-09-12T03:23:55.268541+0000","last_active":"2024-09-12T03:23:55.278192+0000","last_peered":"2024-09-12T03:23:55.278192+0000","last_clean":"2024-09-12T03:23:55.278192+0000","last_became_active":"2024-09-12T03:23:55.266713+0000","last_became_peered":"2024-09-12T03:23:55.266713+0000","last_unstale":"2024-09-12T03:23:55.278192+0000","last_undegraded":"2024-09-12T03:23:55.278192+0000","last_fullsized":"2024-09-12T03:23:55.278192+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_clean_scrub_stamp":"2024-09-12T03:23:54.239693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:19:10.305010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.2e","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-12T03:23:55.281136+0000","last_change":"2024-09-12T03:23:55.274602+0000","last_active":"2024-09-12T03:23:55.281136+0000","last_peered":"2024-09-12T03:23:55.281136+0000","last_clean":"2024-09-12T03:23:55.281136+0000","last_became_active":"2024-09-12T03:23:55.274323+0000","last_became_peered":"2024-09-12T03:23:55.274323+0000","last_unstale":"2024-09-12T03:23:55.281136+0000","last_undegraded":"2024-09-12T03:23:55.281136+0000","last_fullsized":"2024-09-12T03:23:55.281136+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_clean_scrub_stamp":"2024-09-12T03:23:54.239693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:06:40.478281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.2d","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-12T03:23:55.283384+0000","last_change":"2024-09-12T03:23:55.279424+0000","last_active":"2024-09-12T03:23:55.283384+0000","last_peered":"2024-09-12T03:23:55.283384+0000","last_clean":"2024-09-12T03:23:55.283384+0000","last_became_active":"2024-09-12T03:23:55.279145+0000","last_became_peered":"2024-09-12T03:23:55.279145+0000","last_unstale":"2024-09-12T03:23:55.283384+0000","last_undegraded":"2024-09-12T03:23:55.283384+0000","last_fullsized":"2024-09-12T03:23:55.283384+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_clean_scrub_stamp":"2024-09-12T03:23:54.239693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:02:40.409694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.2c","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-12T03:23:55.278619+0000","last_change":"2024-09-12T03:23:55.277389+0000","last_active":"2024-09-12T03:23:55.278619+0000","last_peered":"2024-09-12T03:23:55.278619+0000","last_clean":"2024-09-12T03:23:55.278619+0000","last_became_active":"2024-09-12T03:23:55.276884+0000","last_became_peered":"2024-09-12T03:23:55.276884+0000","last_unstale":"2024-09-12T03:23:55.278619+0000","last_undegraded":"2024-09-12T03:23:55.278619+0000","last_fullsized":"2024-09-12T03:23:55.278619+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_clean_scrub_stamp":"2024-09-12T03:23:54.239693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:53:00.577442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.2b","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-12T03:23:55.271885+0000","last_change":"2024-09-12T03:23:55.270427+0000","last_active":"2024-09-12T03:23:55.271885+0000","last_peered":"2024-09-12T03:23:55.271885+0000","last_clean":"2024-09-12T03:23:55.271885+0000","last_became_active":"2024-09-12T03:23:55.270140+0000","last_became_peered":"2024-09-12T03:23:55.270140+0000","last_unstale":"2024-09-12T03:23:55.271885+0000","last_undegraded":"2024-09-12T03:23:55.271885+0000","last_fullsized":"2024-09-12T03:23:55.271885+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_clean_scrub_stamp":"2024-09-12T03:23:54.239693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:47:44.168067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.2a","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-12T03:23:55.279136+0000","last_change":"2024-09-12T03:23:55.277683+0000","last_active":"2024-09-12T03:23:55.279136+0000","last_peered":"2024-09-12T03:23:55.279136+0000","last_clean":"2024-09-12T03:23:55.279136+0000","last_became_active":"2024-09-12T03:23:55.277329+0000","last_became_peered":"2024-09-12T03:23:55.277329+0000","last_unstale":"2024-09-12T03:23:55.279136+0000","last_undegraded":"2024-09-12T03:23:55.279136+0000","last_fullsized":"2024-09-12T03:23:55.279136+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_clean_scrub_stamp":"2024-09-12T03:23:54.239693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:31:10.861866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.29","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-12T03:23:55.282285+0000","last_change":"2024-09-12T03:23:55.281379+0000","last_active":"2024-09-12T03:23:55.282285+0000","last_peered":"2024-09-12T03:23:55.282285+0000","last_clean":"2024-09-12T03:23:55.282285+0000","last_became_active":"2024-09-12T03:23:55.281011+0000","last_became_peered":"2024-09-12T03:23:55.281011+0000","last_unstale":"2024-09-12T03:23:55.282285+0000","last_undegraded":"2024-09-12T03:23:55.282285+0000","last_fullsized":"2024-09-12T03:23:55.282285+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_clean_scrub_stamp":"2024-09-12T03:23:54.239693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:25:02.805426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.28","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-12T03:23:55.272245+0000","last_change":"2024-09-12T03:23:55.270999+0000","last_active":"2024-09-12T03:23:55.272245+0000","last_peered":"2024-09-12T03:23:55.272245+0000","last_clean":"2024-09-12T03:23:55.272245+0000","last_became_active":"2024-09-12T03:23:55.270152+0000","last_became_peered":"2024-09-12T03:23:55.270152+0000","last_unstale":"2024-09-12T03:23:55.272245+0000","last_undegraded":"2024-09-12T03:23:55.272245+0000","last_fullsized":"2024-09-12T03:23:55.272245+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_clean_scrub_stamp":"2024-09-12T03:23:54.239693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:22:07.028924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.27","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-12T03:23:55.279720+0000","last_change":"2024-09-12T03:23:55.278652+0000","last_active":"2024-09-12T03:23:55.279720+0000","last_peered":"2024-09-12T03:23:55.279720+0000","last_clean":"2024-09-12T03:23:55.279720+0000","last_became_active":"2024-09-12T03:23:55.278315+0000","last_became_peered":"2024-09-12T03:23:55.278315+0000","last_unstale":"2024-09-12T03:23:55.279720+0000","last_undegraded":"2024-09-12T03:23:55.279720+0000","last_fullsized":"2024-09-12T03:23:55.279720+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_clean_scrub_stamp":"2024-09-12T03:23:54.239693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:03:31.793088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.26","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-12T03:23:55.287487+0000","last_change":"2024-09-12T03:23:55.282207+0000","last_active":"2024-09-12T03:23:55.287487+0000","last_peered":"2024-09-12T03:23:55.287487+0000","last_clean":"2024-09-12T03:23:55.287487+0000","last_became_active":"2024-09-12T03:23:55.279961+0000","last_became_peered":"2024-09-12T03:23:55.279961+0000","last_unstale":"2024-09-12T03:23:55.287487+0000","last_undegraded":"2024-09-12T03:23:55.287487+0000","last_fullsized":"2024-09-12T03:23:55.287487+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_clean_scrub_stamp":"2024-09-12T03:23:54.239693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:30:24.357977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.25","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-12T03:23:55.267563+0000","last_change":"2024-09-12T03:23:55.266000+0000","last_active":"2024-09-12T03:23:55.267563+0000","last_peered":"2024-09-12T03:23:55.267563+0000","last_clean":"2024-09-12T03:23:55.267563+0000","last_became_active":"2024-09-12T03:23:55.265677+0000","last_became_peered":"2024-09-12T03:23:55.265677+0000","last_unstale":"2024-09-12T03:23:55.267563+0000","last_undegraded":"2024-09-12T03:23:55.267563+0000","last_fullsized":"2024-09-12T03:23:55.267563+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_clean_scrub_stamp":"2024-09-12T03:23:54.239693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:05:58.390413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.24","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-12T03:23:55.268149+0000","last_change":"2024-09-12T03:23:55.265972+0000","last_active":"2024-09-12T03:23:55.268149+0000","last_peered":"2024-09-12T03:23:55.268149+0000","last_clean":"2024-09-12T03:23:55.268149+0000","last_became_active":"2024-09-12T03:23:55.265321+0000","last_became_peered":"2024-09-12T03:23:55.265321+0000","last_unstale":"2024-09-12T03:23:55.268149+0000","last_undegraded":"2024-09-12T03:23:55.268149+0000","last_fullsized":"2024-09-12T03:23:55.268149+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_clean_scrub_stamp":"2024-09-12T03:23:54.239693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:42:50.609549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.23","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-12T03:23:55.267745+0000","last_change":"2024-09-12T03:23:55.266765+0000","last_active":"2024-09-12T03:23:55.267745+0000","last_peered":"2024-09-12T03:23:55.267745+0000","last_clean":"2024-09-12T03:23:55.267745+0000","last_became_active":"2024-09-12T03:23:55.266486+0000","last_became_peered":"2024-09-12T03:23:55.266486+0000","last_unstale":"2024-09-12T03:23:55.267745+0000","last_undegraded":"2024-09-12T03:23:55.267745+0000","last_fullsized":"2024-09-12T03:23:55.267745+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_clean_scrub_stamp":"2024-09-12T03:23:54.239693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:14:42.368090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.22","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-12T03:23:55.274533+0000","last_change":"2024-09-12T03:23:55.274123+0000","last_active":"2024-09-12T03:23:55.274533+0000","last_peered":"2024-09-12T03:23:55.274533+0000","last_clean":"2024-09-12T03:23:55.274533+0000","last_became_active":"2024-09-12T03:23:55.273833+0000","last_became_peered":"2024-09-12T03:23:55.273833+0000","last_unstale":"2024-09-12T03:23:55.274533+0000","last_undegraded":"2024-09-12T03:23:55.274533+0000","last_fullsized":"2024-09-12T03:23:55.274533+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_clean_scrub_stamp":"2024-09-12T03:23:54.239693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:04:16.721862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.21","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-12T03:23:55.280811+0000","last_change":"2024-09-12T03:23:55.272097+0000","last_active":"2024-09-12T03:23:55.280811+0000","last_peered":"2024-09-12T03:23:55.280811+0000","last_clean":"2024-09-12T03:23:55.280811+0000","last_became_active":"2024-09-12T03:23:55.271780+0000","last_became_peered":"2024-09-12T03:23:55.271780+0000","last_unstale":"2024-09-12T03:23:55.280811+0000","last_undegraded":"2024-09-12T03:23:55.280811+0000","last_fullsized":"2024-09-12T03:23:55.280811+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_clean_scrub_stamp":"2024-09-12T03:23:54.239693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:27:48.458762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.20","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-12T03:23:55.285593+0000","last_change":"2024-09-12T03:23:55.278047+0000","last_active":"2024-09-12T03:23:55.285593+0000","last_peered":"2024-09-12T03:23:55.285593+0000","last_clean":"2024-09-12T03:23:55.285593+0000","last_became_active":"2024-09-12T03:23:55.277508+0000","last_became_peered":"2024-09-12T03:23:55.277508+0000","last_unstale":"2024-09-12T03:23:55.285593+0000","last_undegraded":"2024-09-12T03:23:55.285593+0000","last_fullsized":"2024-09-12T03:23:55.285593+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_clean_scrub_stamp":"2024-09-12T03:23:54.239693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:05:01.051455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.1e","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-12T03:23:55.274164+0000","last_change":"2024-09-12T03:23:55.273169+0000","last_active":"2024-09-12T03:23:55.274164+0000","last_peered":"2024-09-12T03:23:55.274164+0000","last_clean":"2024-09-12T03:23:55.274164+0000","last_became_active":"2024-09-12T03:23:55.272025+0000","last_became_peered":"2024-09-12T03:23:55.272025+0000","last_unstale":"2024-09-12T03:23:55.274164+0000","last_undegraded":"2024-09-12T03:23:55.274164+0000","last_fullsized":"2024-09-12T03:23:55.274164+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_clean_scrub_stamp":"2024-09-12T03:23:54.239693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:10:45.502628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.1f","version":"159'4","reported_seq":16,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-12T03:23:55.282797+0000","last_change":"2024-09-12T03:23:54.255998+0000","last_active":"2024-09-12T03:23:55.282797+0000","last_peered":"2024-09-12T03:23:55.282797+0000","last_clean":"2024-09-12T03:23:55.282797+0000","last_became_active":"2024-09-12T03:23:54.255715+0000","last_became_peered":"2024-09-12T03:23:54.255715+0000","last_unstale":"2024-09-12T03:23:55.282797+0000","last_undegraded":"2024-09-12T03:23:55.282797+0000","last_fullsized":"2024-09-12T03:23:55.282797+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:53.234707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:53.234707+0000","last_clean_scrub_stamp":"2024-09-12T03:23:53.234707+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:24:13.256301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1f","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-12T03:23:55.258509+0000","last_change":"2024-09-12T03:23:55.257516+0000","last_active":"2024-09-12T03:23:55.258509+0000","last_peered":"2024-09-12T03:23:55.258509+0000","last_clean":"2024-09-12T03:23:55.258509+0000","last_became_active":"2024-09-12T03:23:55.257183+0000","last_became_peered":"2024-09-12T03:23:55.257183+0000","last_unstale":"2024-09-12T03:23:55.258509+0000","last_undegraded":"2024-09-12T03:23:55.258509+0000","last_fullsized":"2024-09-12T03:23:55.258509+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_clean_scrub_stamp":"2024-09-12T03:23:54.239693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:01:34.396958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.1e","version":"159'1","reported_seq":13,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-12T03:23:55.282384+0000","last_change":"2024-09-12T03:23:54.259810+0000","last_active":"2024-09-12T03:23:55.282384+0000","last_peered":"2024-09-12T03:23:55.282384+0000","last_clean":"2024-09-12T03:23:55.282384+0000","last_became_active":"2024-09-12T03:23:54.259534+0000","last_became_peered":"2024-09-12T03:23:54.259534+0000","last_unstale":"2024-09-12T03:23:55.282384+0000","last_undegraded":"2024-09-12T03:23:55.282384+0000","last_fullsized":"2024-09-12T03:23:55.282384+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:53.234707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:53.234707+0000","last_clean_scrub_stamp":"2024-09-12T03:23:53.234707+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:11:27.739972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.1c","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-12T03:23:55.285116+0000","last_change":"2024-09-12T03:23:55.280641+0000","last_active":"2024-09-12T03:23:55.285116+0000","last_peered":"2024-09-12T03:23:55.285116+0000","last_clean":"2024-09-12T03:23:55.285116+0000","last_became_active":"2024-09-12T03:23:55.280305+0000","last_became_peered":"2024-09-12T03:23:55.280305+0000","last_unstale":"2024-09-12T03:23:55.285116+0000","last_undegraded":"2024-09-12T03:23:55.285116+0000","last_fullsized":"2024-09-12T03:23:55.285116+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_clean_scrub_stamp":"2024-09-12T03:23:54.239693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:03:08.833246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.1d","version":"159'2","reported_seq":14,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-12T03:23:55.288546+0000","last_change":"2024-09-12T03:23:54.255596+0000","last_active":"2024-09-12T03:23:55.288546+0000","last_peered":"2024-09-12T03:23:55.288546+0000","last_clean":"2024-09-12T03:23:55.288546+0000","last_became_active":"2024-09-12T03:23:54.255314+0000","last_became_peered":"2024-09-12T03:23:54.255314+0000","last_unstale":"2024-09-12T03:23:55.288546+0000","last_undegraded":"2024-09-12T03:23:55.288546+0000","last_fullsized":"2024-09-12T03:23:55.288546+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:53.234707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:53.234707+0000","last_clean_scrub_stamp":"2024-09-12T03:23:53.234707+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:53:27.117140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.1d","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-12T03:23:55.281265+0000","last_change":"2024-09-12T03:23:55.275705+0000","last_active":"2024-09-12T03:23:55.281265+0000","last_peered":"2024-09-12T03:23:55.281265+0000","last_clean":"2024-09-12T03:23:55.281265+0000","last_became_active":"2024-09-12T03:23:55.275305+0000","last_became_peered":"2024-09-12T03:23:55.275305+0000","last_unstale":"2024-09-12T03:23:55.281265+0000","last_undegraded":"2024-09-12T03:23:55.281265+0000","last_fullsized":"2024-09-12T03:23:55.281265+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_clean_scrub_stamp":"2024-09-12T03:23:54.239693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:02:20.388100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.1c","version":"159'2","reported_seq":14,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-12T03:23:55.286693+0000","last_change":"2024-09-12T03:23:54.262444+0000","last_active":"2024-09-12T03:23:55.286693+0000","last_peered":"2024-09-12T03:23:55.286693+0000","last_clean":"2024-09-12T03:23:55.286693+0000","last_became_active":"2024-09-12T03:23:54.262144+0000","last_became_peered":"2024-09-12T03:23:54.262144+0000","last_unstale":"2024-09-12T03:23:55.286693+0000","last_undegraded":"2024-09-12T03:23:55.286693+0000","last_fullsized":"2024-09-12T03:23:55.286693+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:53.234707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:53.234707+0000","last_clean_scrub_stamp":"2024-09-12T03:23:53.234707+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:40:55.907954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.1b","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-12T03:23:55.271841+0000","last_change":"2024-09-12T03:23:55.269883+0000","last_active":"2024-09-12T03:23:55.271841+0000","last_peered":"2024-09-12T03:23:55.271841+0000","last_clean":"2024-09-12T03:23:55.271841+0000","last_became_active":"2024-09-12T03:23:55.269556+0000","last_became_peered":"2024-09-12T03:23:55.269556+0000","last_unstale":"2024-09-12T03:23:55.271841+0000","last_undegraded":"2024-09-12T03:23:55.271841+0000","last_fullsized":"2024-09-12T03:23:55.271841+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_clean_scrub_stamp":"2024-09-12T03:23:54.239693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:59:01.300466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.1a","version":"159'1","reported_seq":13,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-12T03:23:55.287997+0000","last_change":"2024-09-12T03:23:54.256999+0000","last_active":"2024-09-12T03:23:55.287997+0000","last_peered":"2024-09-12T03:23:55.287997+0000","last_clean":"2024-09-12T03:23:55.287997+0000","last_became_active":"2024-09-12T03:23:54.256730+0000","last_became_peered":"2024-09-12T03:23:54.256730+0000","last_unstale":"2024-09-12T03:23:55.287997+0000","last_undegraded":"2024-09-12T03:23:55.287997+0000","last_fullsized":"2024-09-12T03:23:55.287997+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:53.234707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:53.234707+0000","last_clean_scrub_stamp":"2024-09-12T03:23:53.234707+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:49:46.267823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.18","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-12T03:23:55.274408+0000","last_change":"2024-09-12T03:23:55.270747+0000","last_active":"2024-09-12T03:23:55.274408+0000","last_peered":"2024-09-12T03:23:55.274408+0000","last_clean":"2024-09-12T03:23:55.274408+0000","last_became_active":"2024-09-12T03:23:55.270464+0000","last_became_peered":"2024-09-12T03:23:55.270464+0000","last_unstale":"2024-09-12T03:23:55.274408+0000","last_undegraded":"2024-09-12T03:23:55.274408+0000","last_fullsized":"2024-09-12T03:23:55.274408+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_clean_scrub_stamp":"2024-09-12T03:23:54.239693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:38:24.490328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.19","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-12T03:23:55.259133+0000","last_change":"2024-09-12T03:23:54.255267+0000","last_active":"2024-09-12T03:23:55.259133+0000","last_peered":"2024-09-12T03:23:55.259133+0000","last_clean":"2024-09-12T03:23:55.259133+0000","last_became_active":"2024-09-12T03:23:54.255020+0000","last_became_peered":"2024-09-12T03:23:54.255020+0000","last_unstale":"2024-09-12T03:23:55.259133+0000","last_undegraded":"2024-09-12T03:23:55.259133+0000","last_fullsized":"2024-09-12T03:23:55.259133+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:53.234707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:53.234707+0000","last_clean_scrub_stamp":"2024-09-12T03:23:53.234707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:27:40.744073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.19","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-12T03:23:55.271916+0000","last_change":"2024-09-12T03:23:55.270662+0000","last_active":"2024-09-12T03:23:55.271916+0000","last_peered":"2024-09-12T03:23:55.271916+0000","last_clean":"2024-09-12T03:23:55.271916+0000","last_became_active":"2024-09-12T03:23:55.263555+0000","last_became_peered":"2024-09-12T03:23:55.263555+0000","last_unstale":"2024-09-12T03:23:55.271916+0000","last_undegraded":"2024-09-12T03:23:55.271916+0000","last_fullsized":"2024-09-12T03:23:55.271916+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_clean_scrub_stamp":"2024-09-12T03:23:54.239693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:02:40.409694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.18","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-12T03:23:55.253830+0000","last_change":"2024-09-12T03:23:54.261128+0000","last_active":"2024-09-12T03:23:55.253830+0000","last_peered":"2024-09-12T03:23:55.253830+0000","last_clean":"2024-09-12T03:23:55.253830+0000","last_became_active":"2024-09-12T03:23:54.260724+0000","last_became_peered":"2024-09-12T03:23:54.260724+0000","last_unstale":"2024-09-12T03:23:55.253830+0000","last_undegraded":"2024-09-12T03:23:55.253830+0000","last_fullsized":"2024-09-12T03:23:55.253830+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:53.234707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:53.234707+0000","last_clean_scrub_stamp":"2024-09-12T03:23:53.234707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:27:01.640325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.16","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-12T03:23:55.268142+0000","last_change":"2024-09-12T03:23:55.266412+0000","last_active":"2024-09-12T03:23:55.268142+0000","last_peered":"2024-09-12T03:23:55.268142+0000","last_clean":"2024-09-12T03:23:55.268142+0000","last_became_active":"2024-09-12T03:23:55.266109+0000","last_became_peered":"2024-09-12T03:23:55.266109+0000","last_unstale":"2024-09-12T03:23:55.268142+0000","last_undegraded":"2024-09-12T03:23:55.268142+0000","last_fullsized":"2024-09-12T03:23:55.268142+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_clean_scrub_stamp":"2024-09-12T03:23:54.239693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:02:20.388100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.17","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-12T03:23:55.259330+0000","last_change":"2024-09-12T03:23:54.255765+0000","last_active":"2024-09-12T03:23:55.259330+0000","last_peered":"2024-09-12T03:23:55.259330+0000","last_clean":"2024-09-12T03:23:55.259330+0000","last_became_active":"2024-09-12T03:23:54.255488+0000","last_became_peered":"2024-09-12T03:23:54.255488+0000","last_unstale":"2024-09-12T03:23:55.259330+0000","last_undegraded":"2024-09-12T03:23:55.259330+0000","last_fullsized":"2024-09-12T03:23:55.259330+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:53.234707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:53.234707+0000","last_clean_scrub_stamp":"2024-09-12T03:23:53.234707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:15:42.450958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.17","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-12T03:23:55.268400+0000","last_change":"2024-09-12T03:23:55.259172+0000","last_active":"2024-09-12T03:23:55.268400+0000","last_peered":"2024-09-12T03:23:55.268400+0000","last_clean":"2024-09-12T03:23:55.268400+0000","last_became_active":"2024-09-12T03:23:55.258819+0000","last_became_peered":"2024-09-12T03:23:55.258819+0000","last_unstale":"2024-09-12T03:23:55.268400+0000","last_undegraded":"2024-09-12T03:23:55.268400+0000","last_fullsized":"2024-09-12T03:23:55.268400+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_clean_scrub_stamp":"2024-09-12T03:23:54.239693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:03:06.964137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.16","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-12T03:23:55.253882+0000","last_change":"2024-09-12T03:23:54.256120+0000","last_active":"2024-09-12T03:23:55.253882+0000","last_peered":"2024-09-12T03:23:55.253882+0000","last_clean":"2024-09-12T03:23:55.253882+0000","last_became_active":"2024-09-12T03:23:54.255758+0000","last_became_peered":"2024-09-12T03:23:54.255758+0000","last_unstale":"2024-09-12T03:23:55.253882+0000","last_undegraded":"2024-09-12T03:23:55.253882+0000","last_fullsized":"2024-09-12T03:23:55.253882+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:53.234707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:53.234707+0000","last_clean_scrub_stamp":"2024-09-12T03:23:53.234707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:04:54.577334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.14","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-12T03:23:55.268171+0000","last_change":"2024-09-12T03:23:55.265354+0000","last_active":"2024-09-12T03:23:55.268171+0000","last_peered":"2024-09-12T03:23:55.268171+0000","last_clean":"2024-09-12T03:23:55.268171+0000","last_became_active":"2024-09-12T03:23:55.265025+0000","last_became_peered":"2024-09-12T03:23:55.265025+0000","last_unstale":"2024-09-12T03:23:55.268171+0000","last_undegraded":"2024-09-12T03:23:55.268171+0000","last_fullsized":"2024-09-12T03:23:55.268171+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_clean_scrub_stamp":"2024-09-12T03:23:54.239693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:51:09.764444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.15","version":"159'3","reported_seq":15,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-12T03:23:55.290523+0000","last_change":"2024-09-12T03:23:54.260535+0000","last_active":"2024-09-12T03:23:55.290523+0000","last_peered":"2024-09-12T03:23:55.290523+0000","last_clean":"2024-09-12T03:23:55.290523+0000","last_became_active":"2024-09-12T03:23:54.259790+0000","last_became_peered":"2024-09-12T03:23:54.259790+0000","last_unstale":"2024-09-12T03:23:55.290523+0000","last_undegraded":"2024-09-12T03:23:55.290523+0000","last_fullsized":"2024-09-12T03:23:55.290523+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:53.234707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:53.234707+0000","last_clean_scrub_stamp":"2024-09-12T03:23:53.234707+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:54:03.590117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.15","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-12T03:23:55.281162+0000","last_change":"2024-09-12T03:23:55.275215+0000","last_active":"2024-09-12T03:23:55.281162+0000","last_peered":"2024-09-12T03:23:55.281162+0000","last_clean":"2024-09-12T03:23:55.281162+0000","last_became_active":"2024-09-12T03:23:55.274920+0000","last_became_peered":"2024-09-12T03:23:55.274920+0000","last_unstale":"2024-09-12T03:23:55.281162+0000","last_undegraded":"2024-09-12T03:23:55.281162+0000","last_fullsized":"2024-09-12T03:23:55.281162+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_clean_scrub_stamp":"2024-09-12T03:23:54.239693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:17:53.462125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.14","version":"159'6","reported_seq":18,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-12T03:23:55.288662+0000","last_change":"2024-09-12T03:23:54.263126+0000","last_active":"2024-09-12T03:23:55.288662+0000","last_peered":"2024-09-12T03:23:55.288662+0000","last_clean":"2024-09-12T03:23:55.288662+0000","last_became_active":"2024-09-12T03:23:54.262829+0000","last_became_peered":"2024-09-12T03:23:54.262829+0000","last_unstale":"2024-09-12T03:23:55.288662+0000","last_undegraded":"2024-09-12T03:23:55.288662+0000","last_fullsized":"2024-09-12T03:23:55.288662+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:53.234707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:53.234707+0000","last_clean_scrub_stamp":"2024-09-12T03:23:53.234707+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:38:58.250398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.12","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-12T03:23:55.282917+0000","last_change":"2024-09-12T03:23:55.279095+0000","last_active":"2024-09-12T03:23:55.282917+0000","last_peered":"2024-09-12T03:23:55.282917+0000","last_clean":"2024-09-12T03:23:55.282917+0000","last_became_active":"2024-09-12T03:23:55.278829+0000","last_became_peered":"2024-09-12T03:23:55.278829+0000","last_unstale":"2024-09-12T03:23:55.282917+0000","last_undegraded":"2024-09-12T03:23:55.282917+0000","last_fullsized":"2024-09-12T03:23:55.282917+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_clean_scrub_stamp":"2024-09-12T03:23:54.239693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:04:30.400384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.13","version":"159'2","reported_seq":14,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-12T03:23:55.288370+0000","last_change":"2024-09-12T03:23:54.259701+0000","last_active":"2024-09-12T03:23:55.288370+0000","last_peered":"2024-09-12T03:23:55.288370+0000","last_clean":"2024-09-12T03:23:55.288370+0000","last_became_active":"2024-09-12T03:23:54.259381+0000","last_became_peered":"2024-09-12T03:23:54.259381+0000","last_unstale":"2024-09-12T03:23:55.288370+0000","last_undegraded":"2024-09-12T03:23:55.288370+0000","last_fullsized":"2024-09-12T03:23:55.288370+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:53.234707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:53.234707+0000","last_clean_scrub_stamp":"2024-09-12T03:23:53.234707+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:37:05.036339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.13","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-12T03:23:55.281532+0000","last_change":"2024-09-12T03:23:55.268117+0000","last_active":"2024-09-12T03:23:55.281532+0000","last_peered":"2024-09-12T03:23:55.281532+0000","last_clean":"2024-09-12T03:23:55.281532+0000","last_became_active":"2024-09-12T03:23:55.267842+0000","last_became_peered":"2024-09-12T03:23:55.267842+0000","last_unstale":"2024-09-12T03:23:55.281532+0000","last_undegraded":"2024-09-12T03:23:55.281532+0000","last_fullsized":"2024-09-12T03:23:55.281532+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_clean_scrub_stamp":"2024-09-12T03:23:54.239693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:40:56.912940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.12","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-12T03:23:55.254318+0000","last_change":"2024-09-12T03:23:54.261257+0000","last_active":"2024-09-12T03:23:55.254318+0000","last_peered":"2024-09-12T03:23:55.254318+0000","last_clean":"2024-09-12T03:23:55.254318+0000","last_became_active":"2024-09-12T03:23:54.260959+0000","last_became_peered":"2024-09-12T03:23:54.260959+0000","last_unstale":"2024-09-12T03:23:55.254318+0000","last_undegraded":"2024-09-12T03:23:55.254318+0000","last_fullsized":"2024-09-12T03:23:55.254318+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:53.234707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:53.234707+0000","last_clean_scrub_stamp":"2024-09-12T03:23:53.234707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:44:36.273674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.10","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-12T03:23:55.262657+0000","last_change":"2024-09-12T03:23:55.257770+0000","last_active":"2024-09-12T03:23:55.262657+0000","last_peered":"2024-09-12T03:23:55.262657+0000","last_clean":"2024-09-12T03:23:55.262657+0000","last_became_active":"2024-09-12T03:23:55.257458+0000","last_became_peered":"2024-09-12T03:23:55.257458+0000","last_unstale":"2024-09-12T03:23:55.262657+0000","last_undegraded":"2024-09-12T03:23:55.262657+0000","last_fullsized":"2024-09-12T03:23:55.262657+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_clean_scrub_stamp":"2024-09-12T03:23:54.239693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:51:35.439697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.11","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-12T03:23:55.265392+0000","last_change":"2024-09-12T03:23:54.255118+0000","last_active":"2024-09-12T03:23:55.265392+0000","last_peered":"2024-09-12T03:23:55.265392+0000","last_clean":"2024-09-12T03:23:55.265392+0000","last_became_active":"2024-09-12T03:23:54.254853+0000","last_became_peered":"2024-09-12T03:23:54.254853+0000","last_unstale":"2024-09-12T03:23:55.265392+0000","last_undegraded":"2024-09-12T03:23:55.265392+0000","last_fullsized":"2024-09-12T03:23:55.265392+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:53.234707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:53.234707+0000","last_clean_scrub_stamp":"2024-09-12T03:23:53.234707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:32:51.087827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.11","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-12T03:23:55.285442+0000","last_change":"2024-09-12T03:23:55.278066+0000","last_active":"2024-09-12T03:23:55.285442+0000","last_peered":"2024-09-12T03:23:55.285442+0000","last_clean":"2024-09-12T03:23:55.285442+0000","last_became_active":"2024-09-12T03:23:55.277315+0000","last_became_peered":"2024-09-12T03:23:55.277315+0000","last_unstale":"2024-09-12T03:23:55.285442+0000","last_undegraded":"2024-09-12T03:23:55.285442+0000","last_fullsized":"2024-09-12T03:23:55.285442+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_clean_scrub_stamp":"2024-09-12T03:23:54.239693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:48:03.817535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.10","version":"159'8","reported_seq":20,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-12T03:23:55.290774+0000","last_change":"2024-09-12T03:23:54.254028+0000","last_active":"2024-09-12T03:23:55.290774+0000","last_peered":"2024-09-12T03:23:55.290774+0000","last_clean":"2024-09-12T03:23:55.290774+0000","last_became_active":"2024-09-12T03:23:54.253704+0000","last_became_peered":"2024-09-12T03:23:54.253704+0000","last_unstale":"2024-09-12T03:23:55.290774+0000","last_undegraded":"2024-09-12T03:23:55.290774+0000","last_fullsized":"2024-09-12T03:23:55.290774+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:53.234707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:53.234707+0000","last_clean_scrub_stamp":"2024-09-12T03:23:53.234707+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:36:13.511928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.e","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-12T03:23:55.269586+0000","last_change":"2024-09-12T03:23:55.266399+0000","last_active":"2024-09-12T03:23:55.269586+0000","last_peered":"2024-09-12T03:23:55.269586+0000","last_clean":"2024-09-12T03:23:55.269586+0000","last_became_active":"2024-09-12T03:23:55.266113+0000","last_became_peered":"2024-09-12T03:23:55.266113+0000","last_unstale":"2024-09-12T03:23:55.269586+0000","last_undegraded":"2024-09-12T03:23:55.269586+0000","last_fullsized":"2024-09-12T03:23:55.269586+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_clean_scrub_stamp":"2024-09-12T03:23:54.239693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:15:19.602139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.f","version":"159'1","reported_seq":13,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-12T03:23:55.290548+0000","last_change":"2024-09-12T03:23:54.261368+0000","last_active":"2024-09-12T03:23:55.290548+0000","last_peered":"2024-09-12T03:23:55.290548+0000","last_clean":"2024-09-12T03:23:55.290548+0000","last_became_active":"2024-09-12T03:23:54.261097+0000","last_became_peered":"2024-09-12T03:23:54.261097+0000","last_unstale":"2024-09-12T03:23:55.290548+0000","last_undegraded":"2024-09-12T03:23:55.290548+0000","last_fullsized":"2024-09-12T03:23:55.290548+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:53.234707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:53.234707+0000","last_clean_scrub_stamp":"2024-09-12T03:23:53.234707+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:07:31.988105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.f","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-12T03:23:55.280192+0000","last_change":"2024-09-12T03:23:55.279580+0000","last_active":"2024-09-12T03:23:55.280192+0000","last_peered":"2024-09-12T03:23:55.280192+0000","last_clean":"2024-09-12T03:23:55.280192+0000","last_became_active":"2024-09-12T03:23:55.277901+0000","last_became_peered":"2024-09-12T03:23:55.277901+0000","last_unstale":"2024-09-12T03:23:55.280192+0000","last_undegraded":"2024-09-12T03:23:55.280192+0000","last_fullsized":"2024-09-12T03:23:55.280192+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_clean_scrub_stamp":"2024-09-12T03:23:54.239693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:19:21.998591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.e","version":"159'2","reported_seq":14,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-12T03:23:55.283889+0000","last_change":"2024-09-12T03:23:54.258644+0000","last_active":"2024-09-12T03:23:55.283889+0000","last_peered":"2024-09-12T03:23:55.283889+0000","last_clean":"2024-09-12T03:23:55.283889+0000","last_became_active":"2024-09-12T03:23:54.258276+0000","last_became_peered":"2024-09-12T03:23:54.258276+0000","last_unstale":"2024-09-12T03:23:55.283889+0000","last_undegraded":"2024-09-12T03:23:55.283889+0000","last_fullsized":"2024-09-12T03:23:55.283889+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:53.234707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:53.234707+0000","last_clean_scrub_stamp":"2024-09-12T03:23:53.234707+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:34:09.982809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.c","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-12T03:23:55.268172+0000","last_change":"2024-09-12T03:23:55.266199+0000","last_active":"2024-09-12T03:23:55.268172+0000","last_peered":"2024-09-12T03:23:55.268172+0000","last_clean":"2024-09-12T03:23:55.268172+0000","last_became_active":"2024-09-12T03:23:55.265698+0000","last_became_peered":"2024-09-12T03:23:55.265698+0000","last_unstale":"2024-09-12T03:23:55.268172+0000","last_undegraded":"2024-09-12T03:23:55.268172+0000","last_fullsized":"2024-09-12T03:23:55.268172+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_clean_scrub_stamp":"2024-09-12T03:23:54.239693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:37:13.236582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.d","version":"159'1","reported_seq":13,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-12T03:23:55.288409+0000","last_change":"2024-09-12T03:23:54.258228+0000","last_active":"2024-09-12T03:23:55.288409+0000","last_peered":"2024-09-12T03:23:55.288409+0000","last_clean":"2024-09-12T03:23:55.288409+0000","last_became_active":"2024-09-12T03:23:54.257966+0000","last_became_peered":"2024-09-12T03:23:54.257966+0000","last_unstale":"2024-09-12T03:23:55.288409+0000","last_undegraded":"2024-09-12T03:23:55.288409+0000","last_fullsized":"2024-09-12T03:23:55.288409+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:53.234707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:53.234707+0000","last_clean_scrub_stamp":"2024-09-12T03:23:53.234707+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:51:00.263580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.d","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-12T03:23:55.281243+0000","last_change":"2024-09-12T03:23:55.275572+0000","last_active":"2024-09-12T03:23:55.281243+0000","last_peered":"2024-09-12T03:23:55.281243+0000","last_clean":"2024-09-12T03:23:55.281243+0000","last_became_active":"2024-09-12T03:23:55.275020+0000","last_became_peered":"2024-09-12T03:23:55.275020+0000","last_unstale":"2024-09-12T03:23:55.281243+0000","last_undegraded":"2024-09-12T03:23:55.281243+0000","last_fullsized":"2024-09-12T03:23:55.281243+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_clean_scrub_stamp":"2024-09-12T03:23:54.239693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:51:09.764444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.c","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-12T03:23:55.255183+0000","last_change":"2024-09-12T03:23:54.258898+0000","last_active":"2024-09-12T03:23:55.255183+0000","last_peered":"2024-09-12T03:23:55.255183+0000","last_clean":"2024-09-12T03:23:55.255183+0000","last_became_active":"2024-09-12T03:23:54.258645+0000","last_became_peered":"2024-09-12T03:23:54.258645+0000","last_unstale":"2024-09-12T03:23:55.255183+0000","last_undegraded":"2024-09-12T03:23:55.255183+0000","last_fullsized":"2024-09-12T03:23:55.255183+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:53.234707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:53.234707+0000","last_clean_scrub_stamp":"2024-09-12T03:23:53.234707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:55:50.101373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.a","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-12T03:23:55.282995+0000","last_change":"2024-09-12T03:23:55.276422+0000","last_active":"2024-09-12T03:23:55.282995+0000","last_peered":"2024-09-12T03:23:55.282995+0000","last_clean":"2024-09-12T03:23:55.282995+0000","last_became_active":"2024-09-12T03:23:55.276066+0000","last_became_peered":"2024-09-12T03:23:55.276066+0000","last_unstale":"2024-09-12T03:23:55.282995+0000","last_undegraded":"2024-09-12T03:23:55.282995+0000","last_fullsized":"2024-09-12T03:23:55.282995+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_clean_scrub_stamp":"2024-09-12T03:23:54.239693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:59:01.300466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.b","version":"159'2","reported_seq":14,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-12T03:23:55.290690+0000","last_change":"2024-09-12T03:23:54.254553+0000","last_active":"2024-09-12T03:23:55.290690+0000","last_peered":"2024-09-12T03:23:55.290690+0000","last_clean":"2024-09-12T03:23:55.290690+0000","last_became_active":"2024-09-12T03:23:54.254262+0000","last_became_peered":"2024-09-12T03:23:54.254262+0000","last_unstale":"2024-09-12T03:23:55.290690+0000","last_undegraded":"2024-09-12T03:23:55.290690+0000","last_fullsized":"2024-09-12T03:23:55.290690+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:53.234707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:53.234707+0000","last_clean_scrub_stamp":"2024-09-12T03:23:53.234707+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:36:21.797718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.b","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-12T03:23:55.270347+0000","last_change":"2024-09-12T03:23:55.267350+0000","last_active":"2024-09-12T03:23:55.270347+0000","last_peered":"2024-09-12T03:23:55.270347+0000","last_clean":"2024-09-12T03:23:55.270347+0000","last_became_active":"2024-09-12T03:23:55.267062+0000","last_became_peered":"2024-09-12T03:23:55.267062+0000","last_unstale":"2024-09-12T03:23:55.270347+0000","last_undegraded":"2024-09-12T03:23:55.270347+0000","last_fullsized":"2024-09-12T03:23:55.270347+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_clean_scrub_stamp":"2024-09-12T03:23:54.239693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:47:22.648161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.a","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-12T03:23:55.272083+0000","last_change":"2024-09-12T03:23:54.262792+0000","last_active":"2024-09-12T03:23:55.272083+0000","last_peered":"2024-09-12T03:23:55.272083+0000","last_clean":"2024-09-12T03:23:55.272083+0000","last_became_active":"2024-09-12T03:23:54.262164+0000","last_became_peered":"2024-09-12T03:23:54.262164+0000","last_unstale":"2024-09-12T03:23:55.272083+0000","last_undegraded":"2024-09-12T03:23:55.272083+0000","last_fullsized":"2024-09-12T03:23:55.272083+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:53.234707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:53.234707+0000","last_clean_scrub_stamp":"2024-09-12T03:23:53.234707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:52:31.483458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.8","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-12T03:23:55.274009+0000","last_change":"2024-09-12T03:23:55.270747+0000","last_active":"2024-09-12T03:23:55.274009+0000","last_peered":"2024-09-12T03:23:55.274009+0000","last_clean":"2024-09-12T03:23:55.274009+0000","last_became_active":"2024-09-12T03:23:55.270467+0000","last_became_peered":"2024-09-12T03:23:55.270467+0000","last_unstale":"2024-09-12T03:23:55.274009+0000","last_undegraded":"2024-09-12T03:23:55.274009+0000","last_fullsized":"2024-09-12T03:23:55.274009+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_clean_scrub_stamp":"2024-09-12T03:23:54.239693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:44:35.446917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.9","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-12T03:23:55.267688+0000","last_change":"2024-09-12T03:23:54.262702+0000","last_active":"2024-09-12T03:23:55.267688+0000","last_peered":"2024-09-12T03:23:55.267688+0000","last_clean":"2024-09-12T03:23:55.267688+0000","last_became_active":"2024-09-12T03:23:54.262374+0000","last_became_peered":"2024-09-12T03:23:54.262374+0000","last_unstale":"2024-09-12T03:23:55.267688+0000","last_undegraded":"2024-09-12T03:23:55.267688+0000","last_fullsized":"2024-09-12T03:23:55.267688+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:53.234707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:53.234707+0000","last_clean_scrub_stamp":"2024-09-12T03:23:53.234707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:54:22.372337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.9","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-12T03:23:55.258249+0000","last_change":"2024-09-12T03:23:55.257472+0000","last_active":"2024-09-12T03:23:55.258249+0000","last_peered":"2024-09-12T03:23:55.258249+0000","last_clean":"2024-09-12T03:23:55.258249+0000","last_became_active":"2024-09-12T03:23:55.257163+0000","last_became_peered":"2024-09-12T03:23:55.257163+0000","last_unstale":"2024-09-12T03:23:55.258249+0000","last_undegraded":"2024-09-12T03:23:55.258249+0000","last_fullsized":"2024-09-12T03:23:55.258249+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_clean_scrub_stamp":"2024-09-12T03:23:54.239693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:17:51.864997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.8","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-12T03:23:55.263080+0000","last_change":"2024-09-12T03:23:54.258374+0000","last_active":"2024-09-12T03:23:55.263080+0000","last_peered":"2024-09-12T03:23:55.263080+0000","last_clean":"2024-09-12T03:23:55.263080+0000","last_became_active":"2024-09-12T03:23:54.258137+0000","last_became_peered":"2024-09-12T03:23:54.258137+0000","last_unstale":"2024-09-12T03:23:55.263080+0000","last_undegraded":"2024-09-12T03:23:55.263080+0000","last_fullsized":"2024-09-12T03:23:55.263080+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:53.234707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:53.234707+0000","last_clean_scrub_stamp":"2024-09-12T03:23:53.234707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:18:57.892973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.6","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-12T03:23:55.265916+0000","last_change":"2024-09-12T03:23:55.265166+0000","last_active":"2024-09-12T03:23:55.265916+0000","last_peered":"2024-09-12T03:23:55.265916+0000","last_clean":"2024-09-12T03:23:55.265916+0000","last_became_active":"2024-09-12T03:23:55.264850+0000","last_became_peered":"2024-09-12T03:23:55.264850+0000","last_unstale":"2024-09-12T03:23:55.265916+0000","last_undegraded":"2024-09-12T03:23:55.265916+0000","last_fullsized":"2024-09-12T03:23:55.265916+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_clean_scrub_stamp":"2024-09-12T03:23:54.239693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:23:43.900584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.7","version":"159'2","reported_seq":14,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-12T03:23:55.288345+0000","last_change":"2024-09-12T03:23:54.262381+0000","last_active":"2024-09-12T03:23:55.288345+0000","last_peered":"2024-09-12T03:23:55.288345+0000","last_clean":"2024-09-12T03:23:55.288345+0000","last_became_active":"2024-09-12T03:23:54.262125+0000","last_became_peered":"2024-09-12T03:23:54.262125+0000","last_unstale":"2024-09-12T03:23:55.288345+0000","last_undegraded":"2024-09-12T03:23:55.288345+0000","last_fullsized":"2024-09-12T03:23:55.288345+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:53.234707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:53.234707+0000","last_clean_scrub_stamp":"2024-09-12T03:23:53.234707+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:35:31.033990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.7","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-12T03:23:55.271130+0000","last_change":"2024-09-12T03:23:55.265159+0000","last_active":"2024-09-12T03:23:55.271130+0000","last_peered":"2024-09-12T03:23:55.271130+0000","last_clean":"2024-09-12T03:23:55.271130+0000","last_became_active":"2024-09-12T03:23:55.264843+0000","last_became_peered":"2024-09-12T03:23:55.264843+0000","last_unstale":"2024-09-12T03:23:55.271130+0000","last_undegraded":"2024-09-12T03:23:55.271130+0000","last_fullsized":"2024-09-12T03:23:55.271130+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_clean_scrub_stamp":"2024-09-12T03:23:54.239693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:51:23.922711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.6","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-12T03:23:55.259863+0000","last_change":"2024-09-12T03:23:54.257278+0000","last_active":"2024-09-12T03:23:55.259863+0000","last_peered":"2024-09-12T03:23:55.259863+0000","last_clean":"2024-09-12T03:23:55.259863+0000","last_became_active":"2024-09-12T03:23:54.256990+0000","last_became_peered":"2024-09-12T03:23:54.256990+0000","last_unstale":"2024-09-12T03:23:55.259863+0000","last_undegraded":"2024-09-12T03:23:55.259863+0000","last_fullsized":"2024-09-12T03:23:55.259863+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:53.234707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:53.234707+0000","last_clean_scrub_stamp":"2024-09-12T03:23:53.234707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:27:47.453776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.4","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-12T03:23:55.274106+0000","last_change":"2024-09-12T03:23:55.272962+0000","last_active":"2024-09-12T03:23:55.274106+0000","last_peered":"2024-09-12T03:23:55.274106+0000","last_clean":"2024-09-12T03:23:55.274106+0000","last_became_active":"2024-09-12T03:23:55.272663+0000","last_became_peered":"2024-09-12T03:23:55.272663+0000","last_unstale":"2024-09-12T03:23:55.274106+0000","last_undegraded":"2024-09-12T03:23:55.274106+0000","last_fullsized":"2024-09-12T03:23:55.274106+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_clean_scrub_stamp":"2024-09-12T03:23:54.239693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:09:38.563790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.5","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-12T03:23:55.263169+0000","last_change":"2024-09-12T03:23:54.254662+0000","last_active":"2024-09-12T03:23:55.263169+0000","last_peered":"2024-09-12T03:23:55.263169+0000","last_clean":"2024-09-12T03:23:55.263169+0000","last_became_active":"2024-09-12T03:23:54.254337+0000","last_became_peered":"2024-09-12T03:23:54.254337+0000","last_unstale":"2024-09-12T03:23:55.263169+0000","last_undegraded":"2024-09-12T03:23:55.263169+0000","last_fullsized":"2024-09-12T03:23:55.263169+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:53.234707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:53.234707+0000","last_clean_scrub_stamp":"2024-09-12T03:23:53.234707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:09:49.378880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.5","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-12T03:23:55.271981+0000","last_change":"2024-09-12T03:23:55.270624+0000","last_active":"2024-09-12T03:23:55.271981+0000","last_peered":"2024-09-12T03:23:55.271981+0000","last_clean":"2024-09-12T03:23:55.271981+0000","last_became_active":"2024-09-12T03:23:55.270340+0000","last_became_peered":"2024-09-12T03:23:55.270340+0000","last_unstale":"2024-09-12T03:23:55.271981+0000","last_undegraded":"2024-09-12T03:23:55.271981+0000","last_fullsized":"2024-09-12T03:23:55.271981+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_clean_scrub_stamp":"2024-09-12T03:23:54.239693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:03:04.676600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.4","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-12T03:23:55.255366+0000","last_change":"2024-09-12T03:23:54.260974+0000","last_active":"2024-09-12T03:23:55.255366+0000","last_peered":"2024-09-12T03:23:55.255366+0000","last_clean":"2024-09-12T03:23:55.255366+0000","last_became_active":"2024-09-12T03:23:54.260713+0000","last_became_peered":"2024-09-12T03:23:54.260713+0000","last_unstale":"2024-09-12T03:23:55.255366+0000","last_undegraded":"2024-09-12T03:23:55.255366+0000","last_fullsized":"2024-09-12T03:23:55.255366+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:53.234707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:53.234707+0000","last_clean_scrub_stamp":"2024-09-12T03:23:53.234707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:34:50.922929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.2","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-12T03:23:55.287489+0000","last_change":"2024-09-12T03:23:55.282009+0000","last_active":"2024-09-12T03:23:55.287489+0000","last_peered":"2024-09-12T03:23:55.287489+0000","last_clean":"2024-09-12T03:23:55.287489+0000","last_became_active":"2024-09-12T03:23:55.281697+0000","last_became_peered":"2024-09-12T03:23:55.281697+0000","last_unstale":"2024-09-12T03:23:55.287489+0000","last_undegraded":"2024-09-12T03:23:55.287489+0000","last_fullsized":"2024-09-12T03:23:55.287489+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_clean_scrub_stamp":"2024-09-12T03:23:54.239693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:04:49.963270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.3","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-12T03:23:55.259109+0000","last_change":"2024-09-12T03:23:54.262104+0000","last_active":"2024-09-12T03:23:55.259109+0000","last_peered":"2024-09-12T03:23:55.259109+0000","last_clean":"2024-09-12T03:23:55.259109+0000","last_became_active":"2024-09-12T03:23:54.261512+0000","last_became_peered":"2024-09-12T03:23:54.261512+0000","last_unstale":"2024-09-12T03:23:55.259109+0000","last_undegraded":"2024-09-12T03:23:55.259109+0000","last_fullsized":"2024-09-12T03:23:55.259109+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:53.234707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:53.234707+0000","last_clean_scrub_stamp":"2024-09-12T03:23:53.234707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:51:00.263580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.3","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-12T03:23:55.279189+0000","last_change":"2024-09-12T03:23:55.278264+0000","last_active":"2024-09-12T03:23:55.279189+0000","last_peered":"2024-09-12T03:23:55.279189+0000","last_clean":"2024-09-12T03:23:55.279189+0000","last_became_active":"2024-09-12T03:23:55.277941+0000","last_became_peered":"2024-09-12T03:23:55.277941+0000","last_unstale":"2024-09-12T03:23:55.279189+0000","last_undegraded":"2024-09-12T03:23:55.279189+0000","last_fullsized":"2024-09-12T03:23:55.279189+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_clean_scrub_stamp":"2024-09-12T03:23:54.239693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:27:02.645311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.2","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-12T03:23:55.264431+0000","last_change":"2024-09-12T03:23:54.261772+0000","last_active":"2024-09-12T03:23:55.264431+0000","last_peered":"2024-09-12T03:23:55.264431+0000","last_clean":"2024-09-12T03:23:55.264431+0000","last_became_active":"2024-09-12T03:23:54.261165+0000","last_became_peered":"2024-09-12T03:23:54.261165+0000","last_unstale":"2024-09-12T03:23:55.264431+0000","last_undegraded":"2024-09-12T03:23:55.264431+0000","last_fullsized":"2024-09-12T03:23:55.264431+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:53.234707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:53.234707+0000","last_clean_scrub_stamp":"2024-09-12T03:23:53.234707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:41:53.579141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.0","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-12T03:23:55.268259+0000","last_change":"2024-09-12T03:23:55.262104+0000","last_active":"2024-09-12T03:23:55.268259+0000","last_peered":"2024-09-12T03:23:55.268259+0000","last_clean":"2024-09-12T03:23:55.268259+0000","last_became_active":"2024-09-12T03:23:55.261767+0000","last_became_peered":"2024-09-12T03:23:55.261767+0000","last_unstale":"2024-09-12T03:23:55.268259+0000","last_undegraded":"2024-09-12T03:23:55.268259+0000","last_fullsized":"2024-09-12T03:23:55.268259+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_clean_scrub_stamp":"2024-09-12T03:23:54.239693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:48:22.595326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.1","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-12T03:23:55.255347+0000","last_change":"2024-09-12T03:23:54.253565+0000","last_active":"2024-09-12T03:23:55.255347+0000","last_peered":"2024-09-12T03:23:55.255347+0000","last_clean":"2024-09-12T03:23:55.255347+0000","last_became_active":"2024-09-12T03:23:54.253293+0000","last_became_peered":"2024-09-12T03:23:54.253293+0000","last_unstale":"2024-09-12T03:23:55.255347+0000","last_undegraded":"2024-09-12T03:23:55.255347+0000","last_fullsized":"2024-09-12T03:23:55.255347+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:53.234707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:53.234707+0000","last_clean_scrub_stamp":"2024-09-12T03:23:53.234707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:44:07.301758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.1","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-12T03:23:55.272536+0000","last_change":"2024-09-12T03:23:55.267442+0000","last_active":"2024-09-12T03:23:55.272536+0000","last_peered":"2024-09-12T03:23:55.272536+0000","last_clean":"2024-09-12T03:23:55.272536+0000","last_became_active":"2024-09-12T03:23:55.266405+0000","last_became_peered":"2024-09-12T03:23:55.266405+0000","last_unstale":"2024-09-12T03:23:55.272536+0000","last_undegraded":"2024-09-12T03:23:55.272536+0000","last_fullsized":"2024-09-12T03:23:55.272536+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_clean_scrub_stamp":"2024-09-12T03:23:54.239693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:20:40.412792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.0","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-12T03:23:55.253970+0000","last_change":"2024-09-12T03:23:54.260836+0000","last_active":"2024-09-12T03:23:55.253970+0000","last_peered":"2024-09-12T03:23:55.253970+0000","last_clean":"2024-09-12T03:23:55.253970+0000","last_became_active":"2024-09-12T03:23:54.260466+0000","last_became_peered":"2024-09-12T03:23:54.260466+0000","last_unstale":"2024-09-12T03:23:55.253970+0000","last_undegraded":"2024-09-12T03:23:55.253970+0000","last_fullsized":"2024-09-12T03:23:55.253970+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:53.234707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:53.234707+0000","last_clean_scrub_stamp":"2024-09-12T03:23:53.234707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:46:58.963165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1a","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-12T03:23:55.287712+0000","last_change":"2024-09-12T03:23:55.285727+0000","last_active":"2024-09-12T03:23:55.287712+0000","last_peered":"2024-09-12T03:23:55.287712+0000","last_clean":"2024-09-12T03:23:55.287712+0000","last_became_active":"2024-09-12T03:23:55.285409+0000","last_became_peered":"2024-09-12T03:23:55.285409+0000","last_unstale":"2024-09-12T03:23:55.287712+0000","last_undegraded":"2024-09-12T03:23:55.287712+0000","last_fullsized":"2024-09-12T03:23:55.287712+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_clean_scrub_stamp":"2024-09-12T03:23:54.239693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:37:35.554458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.1b","version":"159'1","reported_seq":13,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-12T03:23:55.287024+0000","last_change":"2024-09-12T03:23:54.259689+0000","last_active":"2024-09-12T03:23:55.287024+0000","last_peered":"2024-09-12T03:23:55.287024+0000","last_clean":"2024-09-12T03:23:55.287024+0000","last_became_active":"2024-09-12T03:23:54.259354+0000","last_became_peered":"2024-09-12T03:23:54.259354+0000","last_unstale":"2024-09-12T03:23:55.287024+0000","last_undegraded":"2024-09-12T03:23:55.287024+0000","last_fullsized":"2024-09-12T03:23:55.287024+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:53.234707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:53.234707+0000","last_clean_scrub_stamp":"2024-09-12T03:23:53.234707+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:42:44.496439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"70'128","reported_seq":462,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-12T03:23:55.258691+0000","last_change":"2024-09-12T03:16:21.883446+0000","last_active":"2024-09-12T03:23:55.258691+0000","last_peered":"2024-09-12T03:23:55.258691+0000","last_clean":"2024-09-12T03:23:55.258691+0000","last_became_active":"2024-09-12T03:16:21.882207+0000","last_became_peered":"2024-09-12T03:16:21.882207+0000","last_unstale":"2024-09-12T03:23:55.258691+0000","last_undegraded":"2024-09-12T03:23:55.258691+0000","last_fullsized":"2024-09-12T03:23:55.258691+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:20.316426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:20.316426+0000","last_clean_scrub_stamp":"2024-09-12T03:16:20.316426+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:12:15.424320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":26,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":27,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542237,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37908,"kb_used_data":2552,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333932,"statfs":{"total":96636764160,"available":96597946368,"internally_reserved":0,"allocated":2613248,"data_stored":1480196,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542239,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37320,"kb_used_data":1964,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334520,"statfs":{"total":96636764160,"available":96598548480,"internally_reserved":0,"allocated":2011136,"data_stored":888798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542238,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41456,"kb_used_data":1948,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330384,"statfs":{"total":96636764160,"available":96594313216,"internally_reserved":0,"allocated":1994752,"data_stored":881385,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542237,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37300,"kb_used_data":1948,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334540,"statfs":{"total":96636764160,"available":96598568960,"internally_reserved":0,"allocated":1994752,"data_stored":881385,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542238,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33164,"kb_used_data":1972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338676,"statfs":{"total":96636764160,"available":96602804224,"internally_reserved":0,"allocated":2019328,"data_stored":888398,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542238,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33148,"kb_used_data":1960,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338692,"statfs":{"total":96636764160,"available":96602820608,"internally_reserved":0,"allocated":2007040,"data_stored":888216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542239,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33156,"kb_used_data":1968,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338684,"statfs":{"total":96636764160,"available":96602812416,"internally_reserved":0,"allocated":2015232,"data_stored":888272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542239,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33736,"kb_used_data":2544,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338104,"statfs":{"total":96636764160,"available":96602218496,"internally_reserved":0,"allocated":2605056,"data_stored":1478674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T03:23:58.251 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-12T03:23:58.389 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.387+0000 7f6a3999f640 1 Processor -- start 2024-09-12T03:23:58.389 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.387+0000 7f6a3999f640 1 -- start start 2024-09-12T03:23:58.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.388+0000 7f6a3999f640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f6a3406f0c0 0x7f6a340b3560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:58.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.388+0000 7f6a3999f640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6a340ce410 con 0x7f6a3415b2b0 2024-09-12T03:23:58.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.388+0000 7f6a3999f640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6a340732c0 con 0x7f6a34072d70 2024-09-12T03:23:58.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.388+0000 7f6a3999f640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f6a340720a0 con 0x7f6a3406f0c0 2024-09-12T03:23:58.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.388+0000 7f6a32ffd640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f6a3415b2b0 0x7f6a34072430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:33978/0 (socket says 172.21.15.27:33978) 2024-09-12T03:23:58.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.388+0000 7f6a32ffd640 1 -- 172.21.15.27:0/2566208079 learned_addr learned my addr 172.21.15.27:0/2566208079 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:23:58.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.388+0000 7f6a337fe640 1 --2- 172.21.15.27:0/2566208079 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f6a3406f0c0 0x7f6a340b3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:58.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.388+0000 7f6a31ffb640 1 -- 172.21.15.27:0/2566208079 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1423512371 0 0) 0x7f6a24001490 con 0x7f6a3415b2b0 2024-09-12T03:23:58.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.388+0000 7f6a31ffb640 1 -- 172.21.15.27:0/2566208079 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6a20005730 con 0x7f6a3415b2b0 2024-09-12T03:23:58.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.389+0000 7f6a31ffb640 1 -- 172.21.15.27:0/2566208079 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2900366441 0 0) 0x7f6a20005730 con 0x7f6a3415b2b0 2024-09-12T03:23:58.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.389+0000 7f6a31ffb640 1 -- 172.21.15.27:0/2566208079 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f6a3406f0c0 msgr2=0x7f6a340b3560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:58.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.389+0000 7f6a31ffb640 1 --2- 172.21.15.27:0/2566208079 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f6a3406f0c0 0x7f6a340b3560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:58.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.389+0000 7f6a31ffb640 1 -- 172.21.15.27:0/2566208079 >> v1:172.21.15.45:6789/0 conn(0x7f6a34072d70 legacy=0x7f6a3406e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:58.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.389+0000 7f6a31ffb640 1 -- 172.21.15.27:0/2566208079 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a34071ae0 con 0x7f6a3415b2b0 2024-09-12T03:23:58.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.389+0000 7f6a31ffb640 1 -- 172.21.15.27:0/2566208079 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 847755213 0 0) 0x7f6a24001be0 con 0x7f6a3415b2b0 2024-09-12T03:23:58.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.389+0000 7f6a31ffb640 1 -- 172.21.15.27:0/2566208079 >> v1:172.21.15.27:6789/0 conn(0x7f6a3415b2b0 legacy=0x7f6a34072430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:58.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.389+0000 7f6a31ffb640 1 --2- 172.21.15.27:0/2566208079 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6a20005bf0 0x7f6a20005fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:58.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.389+0000 7f6a31ffb640 1 --2- 172.21.15.27:0/2566208079 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f6a3406f0c0 0x7f6a20006a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:58.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.389+0000 7f6a31ffb640 1 --2- 172.21.15.27:0/2566208079 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6a20006f50 0x7f6a200073d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:58.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.389+0000 7f6a31ffb640 1 -- 172.21.15.27:0/2566208079 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f6a340720a0 con 0x7f6a20005bf0 2024-09-12T03:23:58.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.389+0000 7f6a31ffb640 1 -- 172.21.15.27:0/2566208079 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f6a2000dc00 con 0x7f6a20006f50 2024-09-12T03:23:58.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.389+0000 7f6a31ffb640 1 -- 172.21.15.27:0/2566208079 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f6a2000de00 con 0x7f6a3406f0c0 2024-09-12T03:23:58.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.389+0000 7f6a32ffd640 1 --2- 172.21.15.27:0/2566208079 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6a20005bf0 0x7f6a20005fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:58.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.389+0000 7f6a337fe640 1 --2- 172.21.15.27:0/2566208079 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6a20006f50 0x7f6a200073d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:58.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.389+0000 7f6a327fc640 1 --2- 172.21.15.27:0/2566208079 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f6a3406f0c0 0x7f6a20006a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:58.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.389+0000 7f6a32ffd640 1 -- 172.21.15.27:0/2566208079 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f6a3406f0c0 msgr2=0x7f6a20006a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:58.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.389+0000 7f6a32ffd640 1 --2- 172.21.15.27:0/2566208079 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f6a3406f0c0 0x7f6a20006a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:58.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.389+0000 7f6a32ffd640 1 -- 172.21.15.27:0/2566208079 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6a20006f50 msgr2=0x7f6a200073d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:58.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.389+0000 7f6a32ffd640 1 --2- 172.21.15.27:0/2566208079 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6a20006f50 0x7f6a200073d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:58.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.389+0000 7f6a32ffd640 1 -- 172.21.15.27:0/2566208079 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a20005730 con 0x7f6a20005bf0 2024-09-12T03:23:58.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.389+0000 7f6a337fe640 1 --2- 172.21.15.27:0/2566208079 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6a20006f50 0x7f6a200073d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:23:58.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.390+0000 7f6a32ffd640 1 --2- 172.21.15.27:0/2566208079 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6a20005bf0 0x7f6a20005fd0 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7f6a2400aa20 tx=0x7f6a20007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d44296c96e6c1cf6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:58.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.390+0000 7f6a31ffb640 1 -- 172.21.15.27:0/2566208079 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a2400a670 con 0x7f6a20005bf0 2024-09-12T03:23:58.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.390+0000 7f6a31ffb640 1 -- 172.21.15.27:0/2566208079 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a2402f070 con 0x7f6a20005bf0 2024-09-12T03:23:58.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.390+0000 7f6a31ffb640 1 -- 172.21.15.27:0/2566208079 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a2402bd50 con 0x7f6a20005bf0 2024-09-12T03:23:58.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.391+0000 7f6a3999f640 1 -- 172.21.15.27:0/2566208079 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6a20005bf0 msgr2=0x7f6a20005fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:58.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.391+0000 7f6a3999f640 1 --2- 172.21.15.27:0/2566208079 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6a20005bf0 0x7f6a20005fd0 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7f6a2400aa20 tx=0x7f6a20007a40 comp rx=0 tx=0).stop 2024-09-12T03:23:58.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.391+0000 7f6a337fe640 1 -- 172.21.15.27:0/2566208079 reap_dead start 2024-09-12T03:23:58.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.391+0000 7f6a3999f640 1 -- 172.21.15.27:0/2566208079 shutdown_connections 2024-09-12T03:23:58.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.391+0000 7f6a3999f640 1 -- 172.21.15.27:0/2566208079 >> 172.21.15.27:0/2566208079 conn(0x7f6a34062c60 msgr2=0x7f6a3406bb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:58.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.391+0000 7f6a3999f640 1 -- 172.21.15.27:0/2566208079 shutdown_connections 2024-09-12T03:23:58.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.391+0000 7f6a3999f640 1 -- 172.21.15.27:0/2566208079 wait complete. 2024-09-12T03:23:58.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.392+0000 7f6a3999f640 1 Processor -- start 2024-09-12T03:23:58.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.392+0000 7f6a3999f640 1 -- start start 2024-09-12T03:23:58.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.393+0000 7f6a3999f640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f6a20005bf0 0x7f6a34145730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:58.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.393+0000 7f6a3999f640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6a3415c140 0x7f6a34145c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:58.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.393+0000 7f6a3999f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6a341467e0 0x7f6a34146c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:58.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.393+0000 7f6a3999f640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f6a3415b690 con 0x7f6a3415c140 2024-09-12T03:23:58.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.393+0000 7f6a3999f640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f6a340ce690 con 0x7f6a341467e0 2024-09-12T03:23:58.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.393+0000 7f6a3999f640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f6a3415b810 con 0x7f6a20005bf0 2024-09-12T03:23:58.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.393+0000 7f6a327fc640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6a3415c140 0x7f6a34145c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:58.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.393+0000 7f6a327fc640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6a3415c140 0x7f6a34145c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:53628/0 (socket says 172.21.15.27:53628) 2024-09-12T03:23:58.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.393+0000 7f6a327fc640 1 -- 172.21.15.27:0/2897467696 learned_addr learned my addr 172.21.15.27:0/2897467696 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:23:58.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.393+0000 7f6a327fc640 1 -- 172.21.15.27:0/2897467696 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f6a20005bf0 msgr2=0x7f6a34145730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:58.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.393+0000 7f6a327fc640 1 --2- 172.21.15.27:0/2897467696 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f6a20005bf0 0x7f6a34145730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:58.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.393+0000 7f6a327fc640 1 -- 172.21.15.27:0/2897467696 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6a341467e0 msgr2=0x7f6a34146c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:58.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.393+0000 7f6a337fe640 1 --2- 172.21.15.27:0/2897467696 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6a341467e0 0x7f6a34146c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:58.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.393+0000 7f6a327fc640 1 --2- 172.21.15.27:0/2897467696 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6a341467e0 0x7f6a34146c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:58.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.393+0000 7f6a327fc640 1 -- 172.21.15.27:0/2897467696 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a3415c670 con 0x7f6a3415c140 2024-09-12T03:23:58.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.393+0000 7f6a337fe640 1 --2- 172.21.15.27:0/2897467696 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6a341467e0 0x7f6a34146c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:23:58.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.393+0000 7f6a327fc640 1 --2- 172.21.15.27:0/2897467696 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6a3415c140 0x7f6a34145c70 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7f6a1c00e830 tx=0x7f6a1c00ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:58.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.393+0000 7f6a2bfff640 1 -- 172.21.15.27:0/2897467696 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a1c0050d0 con 0x7f6a3415c140 2024-09-12T03:23:58.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.393+0000 7f6a2bfff640 1 -- 172.21.15.27:0/2897467696 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a1c018070 con 0x7f6a3415c140 2024-09-12T03:23:58.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.393+0000 7f6a2bfff640 1 -- 172.21.15.27:0/2897467696 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a1c00c6c0 con 0x7f6a3415c140 2024-09-12T03:23:58.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.393+0000 7f6a3999f640 1 -- 172.21.15.27:0/2897467696 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a3415d580 con 0x7f6a3415c140 2024-09-12T03:23:58.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.394+0000 7f6a3999f640 1 -- 172.21.15.27:0/2897467696 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a3415db40 con 0x7f6a3415c140 2024-09-12T03:23:58.400 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.395+0000 7f6a3999f640 1 -- 172.21.15.27:0/2897467696 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a340b05e0 con 0x7f6a3415c140 2024-09-12T03:23:58.400 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.395+0000 7f6a2bfff640 1 -- 172.21.15.27:0/2897467696 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 25) ==== 91365+0+0 (secure 0 0 0) 0x7f6a1c00ca70 con 0x7f6a3415c140 2024-09-12T03:23:58.400 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.395+0000 7f6a2bfff640 1 --2- 172.21.15.27:0/2897467696 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f6a1406c7f0 0x7f6a1406ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:58.400 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.395+0000 7f6a2bfff640 1 -- 172.21.15.27:0/2897467696 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(160..160 src has 1..160) ==== 5765+0+0 (secure 0 0 0) 0x7f6a1c089580 con 0x7f6a3415c140 2024-09-12T03:23:58.400 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.396+0000 7f6a32ffd640 1 --2- 172.21.15.27:0/2897467696 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f6a1406c7f0 0x7f6a1406ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:58.400 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.396+0000 7f6a32ffd640 1 --2- 172.21.15.27:0/2897467696 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f6a1406c7f0 0x7f6a1406ecb0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f6a24033000 tx=0x7f6a2402d000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:58.400 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.400+0000 7f6a2bfff640 1 -- 172.21.15.27:0/2897467696 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f6a1c01a310 con 0x7f6a3415c140 2024-09-12T03:23:58.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.570+0000 7f6a3999f640 1 -- 172.21.15.27:0/2897467696 --> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6a3415c840 con 0x7f6a1406c7f0 2024-09-12T03:23:58.577 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.576+0000 7f6a2bfff640 1 -- 172.21.15.27:0/2897467696 <== mgr.4113 v2:172.21.15.27:6836/1226920526 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252416 (secure 0 0 0) 0x7f6a2407fa10 con 0x7f6a1406c7f0 2024-09-12T03:23:58.580 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:23:58.581 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2024-09-12T03:23:58.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.579+0000 7f6a3999f640 1 -- 172.21.15.27:0/2897467696 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f6a1406c7f0 msgr2=0x7f6a1406ecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:58.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.579+0000 7f6a3999f640 1 --2- 172.21.15.27:0/2897467696 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f6a1406c7f0 0x7f6a1406ecb0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f6a24033000 tx=0x7f6a2402d000 comp rx=0 tx=0).stop 2024-09-12T03:23:58.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.579+0000 7f6a3999f640 1 -- 172.21.15.27:0/2897467696 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6a3415c140 msgr2=0x7f6a34145c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:58.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.579+0000 7f6a3999f640 1 --2- 172.21.15.27:0/2897467696 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6a3415c140 0x7f6a34145c70 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7f6a1c00e830 tx=0x7f6a1c00ecf0 comp rx=0 tx=0).stop 2024-09-12T03:23:58.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.579+0000 7f6a3999f640 1 -- 172.21.15.27:0/2897467696 shutdown_connections 2024-09-12T03:23:58.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.579+0000 7f6a3999f640 1 --2- 172.21.15.27:0/2897467696 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6a341467e0 0x7f6a34146c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:58.584 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.579+0000 7f6a3999f640 1 --2- 172.21.15.27:0/2897467696 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f6a1406c7f0 0x7f6a1406ecb0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:58.584 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.579+0000 7f6a3999f640 1 --2- 172.21.15.27:0/2897467696 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6a3415c140 0x7f6a34145c70 unknown :-1 s=CLOSED pgs=1690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:58.584 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.579+0000 7f6a3999f640 1 --2- 172.21.15.27:0/2897467696 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f6a20005bf0 0x7f6a34145730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:58.584 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.579+0000 7f6a3999f640 1 -- 172.21.15.27:0/2897467696 >> 172.21.15.27:0/2897467696 conn(0x7f6a34062c60 msgr2=0x7f6a34150c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:58.584 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.579+0000 7f6a3999f640 1 -- 172.21.15.27:0/2897467696 shutdown_connections 2024-09-12T03:23:58.584 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.579+0000 7f6a3999f640 1 -- 172.21.15.27:0/2897467696 wait complete. 2024-09-12T03:23:58.591 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":205,"stamp":"2024-09-12T03:23:57.141038+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":223,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":166,"ondisk_log_size":166,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":59,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":287188,"kb_used_data":16856,"kb_used_omap":12,"kb_used_meta":270131,"kb_avail":754687532,"statfs":{"total":773094113280,"available":772800032768,"internally_reserved":0,"allocated":17260544,"data_stored":8275324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":276614704},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000821"},"pg_stats":[{"pgid":"27.3f","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-12T03:23:55.284859+0000","last_change":"2024-09-12T03:23:55.276559+0000","last_active":"2024-09-12T03:23:55.284859+0000","last_peered":"2024-09-12T03:23:55.284859+0000","last_clean":"2024-09-12T03:23:55.284859+0000","last_became_active":"2024-09-12T03:23:55.276270+0000","last_became_peered":"2024-09-12T03:23:55.276270+0000","last_unstale":"2024-09-12T03:23:55.284859+0000","last_undegraded":"2024-09-12T03:23:55.284859+0000","last_fullsized":"2024-09-12T03:23:55.284859+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_clean_scrub_stamp":"2024-09-12T03:23:54.239693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:28:08.158129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.3e","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-12T03:23:55.274187+0000","last_change":"2024-09-12T03:23:55.273001+0000","last_active":"2024-09-12T03:23:55.274187+0000","last_peered":"2024-09-12T03:23:55.274187+0000","last_clean":"2024-09-12T03:23:55.274187+0000","last_became_active":"2024-09-12T03:23:55.272677+0000","last_became_peered":"2024-09-12T03:23:55.272677+0000","last_unstale":"2024-09-12T03:23:55.274187+0000","last_undegraded":"2024-09-12T03:23:55.274187+0000","last_fullsized":"2024-09-12T03:23:55.274187+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_clean_scrub_stamp":"2024-09-12T03:23:54.239693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:15:19.279469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.3d","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-12T03:23:55.268540+0000","last_change":"2024-09-12T03:23:55.261370+0000","last_active":"2024-09-12T03:23:55.268540+0000","last_peered":"2024-09-12T03:23:55.268540+0000","last_clean":"2024-09-12T03:23:55.268540+0000","last_became_active":"2024-09-12T03:23:55.261076+0000","last_became_peered":"2024-09-12T03:23:55.261076+0000","last_unstale":"2024-09-12T03:23:55.268540+0000","last_undegraded":"2024-09-12T03:23:55.268540+0000","last_fullsized":"2024-09-12T03:23:55.268540+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_clean_scrub_stamp":"2024-09-12T03:23:54.239693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:57:08.826050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.3c","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-12T03:23:55.265525+0000","last_change":"2024-09-12T03:23:55.264349+0000","last_active":"2024-09-12T03:23:55.265525+0000","last_peered":"2024-09-12T03:23:55.265525+0000","last_clean":"2024-09-12T03:23:55.265525+0000","last_became_active":"2024-09-12T03:23:55.263996+0000","last_became_peered":"2024-09-12T03:23:55.263996+0000","last_unstale":"2024-09-12T03:23:55.265525+0000","last_undegraded":"2024-09-12T03:23:55.265525+0000","last_fullsized":"2024-09-12T03:23:55.265525+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_clean_scrub_stamp":"2024-09-12T03:23:54.239693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:06:40.478281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.3b","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-12T03:23:55.283087+0000","last_change":"2024-09-12T03:23:55.279885+0000","last_active":"2024-09-12T03:23:55.283087+0000","last_peered":"2024-09-12T03:23:55.283087+0000","last_clean":"2024-09-12T03:23:55.283087+0000","last_became_active":"2024-09-12T03:23:55.279633+0000","last_became_peered":"2024-09-12T03:23:55.279633+0000","last_unstale":"2024-09-12T03:23:55.283087+0000","last_undegraded":"2024-09-12T03:23:55.283087+0000","last_fullsized":"2024-09-12T03:23:55.283087+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_clean_scrub_stamp":"2024-09-12T03:23:54.239693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:56:55.717979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.3a","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-12T03:23:55.274418+0000","last_change":"2024-09-12T03:23:55.270327+0000","last_active":"2024-09-12T03:23:55.274418+0000","last_peered":"2024-09-12T03:23:55.274418+0000","last_clean":"2024-09-12T03:23:55.274418+0000","last_became_active":"2024-09-12T03:23:55.269294+0000","last_became_peered":"2024-09-12T03:23:55.269294+0000","last_unstale":"2024-09-12T03:23:55.274418+0000","last_undegraded":"2024-09-12T03:23:55.274418+0000","last_fullsized":"2024-09-12T03:23:55.274418+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_clean_scrub_stamp":"2024-09-12T03:23:54.239693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:41:49.085684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.39","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-12T03:23:55.285486+0000","last_change":"2024-09-12T03:23:55.278897+0000","last_active":"2024-09-12T03:23:55.285486+0000","last_peered":"2024-09-12T03:23:55.285486+0000","last_clean":"2024-09-12T03:23:55.285486+0000","last_became_active":"2024-09-12T03:23:55.278627+0000","last_became_peered":"2024-09-12T03:23:55.278627+0000","last_unstale":"2024-09-12T03:23:55.285486+0000","last_undegraded":"2024-09-12T03:23:55.285486+0000","last_fullsized":"2024-09-12T03:23:55.285486+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_clean_scrub_stamp":"2024-09-12T03:23:54.239693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:22:03.724702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.38","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-12T03:23:55.276152+0000","last_change":"2024-09-12T03:23:55.273644+0000","last_active":"2024-09-12T03:23:55.276152+0000","last_peered":"2024-09-12T03:23:55.276152+0000","last_clean":"2024-09-12T03:23:55.276152+0000","last_became_active":"2024-09-12T03:23:55.273372+0000","last_became_peered":"2024-09-12T03:23:55.273372+0000","last_unstale":"2024-09-12T03:23:55.276152+0000","last_undegraded":"2024-09-12T03:23:55.276152+0000","last_fullsized":"2024-09-12T03:23:55.276152+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_clean_scrub_stamp":"2024-09-12T03:23:54.239693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:03:36.855341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.37","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-12T03:23:55.269553+0000","last_change":"2024-09-12T03:23:55.267014+0000","last_active":"2024-09-12T03:23:55.269553+0000","last_peered":"2024-09-12T03:23:55.269553+0000","last_clean":"2024-09-12T03:23:55.269553+0000","last_became_active":"2024-09-12T03:23:55.266738+0000","last_became_peered":"2024-09-12T03:23:55.266738+0000","last_unstale":"2024-09-12T03:23:55.269553+0000","last_undegraded":"2024-09-12T03:23:55.269553+0000","last_fullsized":"2024-09-12T03:23:55.269553+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_clean_scrub_stamp":"2024-09-12T03:23:54.239693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:45:35.032950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.36","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-12T03:23:55.283056+0000","last_change":"2024-09-12T03:23:55.278628+0000","last_active":"2024-09-12T03:23:55.283056+0000","last_peered":"2024-09-12T03:23:55.283056+0000","last_clean":"2024-09-12T03:23:55.283056+0000","last_became_active":"2024-09-12T03:23:55.278359+0000","last_became_peered":"2024-09-12T03:23:55.278359+0000","last_unstale":"2024-09-12T03:23:55.283056+0000","last_undegraded":"2024-09-12T03:23:55.283056+0000","last_fullsized":"2024-09-12T03:23:55.283056+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_clean_scrub_stamp":"2024-09-12T03:23:54.239693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:47:44.168067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.35","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-12T03:23:55.268512+0000","last_change":"2024-09-12T03:23:55.259098+0000","last_active":"2024-09-12T03:23:55.268512+0000","last_peered":"2024-09-12T03:23:55.268512+0000","last_clean":"2024-09-12T03:23:55.268512+0000","last_became_active":"2024-09-12T03:23:55.258801+0000","last_became_peered":"2024-09-12T03:23:55.258801+0000","last_unstale":"2024-09-12T03:23:55.268512+0000","last_undegraded":"2024-09-12T03:23:55.268512+0000","last_fullsized":"2024-09-12T03:23:55.268512+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_clean_scrub_stamp":"2024-09-12T03:23:54.239693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:33:57.635706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.34","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-12T03:23:55.269259+0000","last_change":"2024-09-12T03:23:55.268174+0000","last_active":"2024-09-12T03:23:55.269259+0000","last_peered":"2024-09-12T03:23:55.269259+0000","last_clean":"2024-09-12T03:23:55.269259+0000","last_became_active":"2024-09-12T03:23:55.267893+0000","last_became_peered":"2024-09-12T03:23:55.267893+0000","last_unstale":"2024-09-12T03:23:55.269259+0000","last_undegraded":"2024-09-12T03:23:55.269259+0000","last_fullsized":"2024-09-12T03:23:55.269259+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_clean_scrub_stamp":"2024-09-12T03:23:54.239693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:35:21.699867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.33","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-12T03:23:55.271345+0000","last_change":"2024-09-12T03:23:55.268397+0000","last_active":"2024-09-12T03:23:55.271345+0000","last_peered":"2024-09-12T03:23:55.271345+0000","last_clean":"2024-09-12T03:23:55.271345+0000","last_became_active":"2024-09-12T03:23:55.268136+0000","last_became_peered":"2024-09-12T03:23:55.268136+0000","last_unstale":"2024-09-12T03:23:55.271345+0000","last_undegraded":"2024-09-12T03:23:55.271345+0000","last_fullsized":"2024-09-12T03:23:55.271345+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_clean_scrub_stamp":"2024-09-12T03:23:54.239693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:16:56.078635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.32","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-12T03:23:55.287454+0000","last_change":"2024-09-12T03:23:55.281329+0000","last_active":"2024-09-12T03:23:55.287454+0000","last_peered":"2024-09-12T03:23:55.287454+0000","last_clean":"2024-09-12T03:23:55.287454+0000","last_became_active":"2024-09-12T03:23:55.279160+0000","last_became_peered":"2024-09-12T03:23:55.279160+0000","last_unstale":"2024-09-12T03:23:55.287454+0000","last_undegraded":"2024-09-12T03:23:55.287454+0000","last_fullsized":"2024-09-12T03:23:55.287454+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_clean_scrub_stamp":"2024-09-12T03:23:54.239693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:59:09.406051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.31","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-12T03:23:55.285466+0000","last_change":"2024-09-12T03:23:55.278316+0000","last_active":"2024-09-12T03:23:55.285466+0000","last_peered":"2024-09-12T03:23:55.285466+0000","last_clean":"2024-09-12T03:23:55.285466+0000","last_became_active":"2024-09-12T03:23:55.277685+0000","last_became_peered":"2024-09-12T03:23:55.277685+0000","last_unstale":"2024-09-12T03:23:55.285466+0000","last_undegraded":"2024-09-12T03:23:55.285466+0000","last_fullsized":"2024-09-12T03:23:55.285466+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_clean_scrub_stamp":"2024-09-12T03:23:54.239693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:55:15.186982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.30","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-12T03:23:55.280913+0000","last_change":"2024-09-12T03:23:55.269395+0000","last_active":"2024-09-12T03:23:55.280913+0000","last_peered":"2024-09-12T03:23:55.280913+0000","last_clean":"2024-09-12T03:23:55.280913+0000","last_became_active":"2024-09-12T03:23:55.269129+0000","last_became_peered":"2024-09-12T03:23:55.269129+0000","last_unstale":"2024-09-12T03:23:55.280913+0000","last_undegraded":"2024-09-12T03:23:55.280913+0000","last_fullsized":"2024-09-12T03:23:55.280913+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_clean_scrub_stamp":"2024-09-12T03:23:54.239693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:38:59.255384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.2f","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-12T03:23:55.278192+0000","last_change":"2024-09-12T03:23:55.268541+0000","last_active":"2024-09-12T03:23:55.278192+0000","last_peered":"2024-09-12T03:23:55.278192+0000","last_clean":"2024-09-12T03:23:55.278192+0000","last_became_active":"2024-09-12T03:23:55.266713+0000","last_became_peered":"2024-09-12T03:23:55.266713+0000","last_unstale":"2024-09-12T03:23:55.278192+0000","last_undegraded":"2024-09-12T03:23:55.278192+0000","last_fullsized":"2024-09-12T03:23:55.278192+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_clean_scrub_stamp":"2024-09-12T03:23:54.239693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:19:10.305010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.2e","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-12T03:23:55.281136+0000","last_change":"2024-09-12T03:23:55.274602+0000","last_active":"2024-09-12T03:23:55.281136+0000","last_peered":"2024-09-12T03:23:55.281136+0000","last_clean":"2024-09-12T03:23:55.281136+0000","last_became_active":"2024-09-12T03:23:55.274323+0000","last_became_peered":"2024-09-12T03:23:55.274323+0000","last_unstale":"2024-09-12T03:23:55.281136+0000","last_undegraded":"2024-09-12T03:23:55.281136+0000","last_fullsized":"2024-09-12T03:23:55.281136+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_clean_scrub_stamp":"2024-09-12T03:23:54.239693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:06:40.478281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.2d","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-12T03:23:55.283384+0000","last_change":"2024-09-12T03:23:55.279424+0000","last_active":"2024-09-12T03:23:55.283384+0000","last_peered":"2024-09-12T03:23:55.283384+0000","last_clean":"2024-09-12T03:23:55.283384+0000","last_became_active":"2024-09-12T03:23:55.279145+0000","last_became_peered":"2024-09-12T03:23:55.279145+0000","last_unstale":"2024-09-12T03:23:55.283384+0000","last_undegraded":"2024-09-12T03:23:55.283384+0000","last_fullsized":"2024-09-12T03:23:55.283384+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_clean_scrub_stamp":"2024-09-12T03:23:54.239693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:02:40.409694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.2c","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-12T03:23:55.278619+0000","last_change":"2024-09-12T03:23:55.277389+0000","last_active":"2024-09-12T03:23:55.278619+0000","last_peered":"2024-09-12T03:23:55.278619+0000","last_clean":"2024-09-12T03:23:55.278619+0000","last_became_active":"2024-09-12T03:23:55.276884+0000","last_became_peered":"2024-09-12T03:23:55.276884+0000","last_unstale":"2024-09-12T03:23:55.278619+0000","last_undegraded":"2024-09-12T03:23:55.278619+0000","last_fullsized":"2024-09-12T03:23:55.278619+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_clean_scrub_stamp":"2024-09-12T03:23:54.239693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:53:00.577442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.2b","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-12T03:23:55.271885+0000","last_change":"2024-09-12T03:23:55.270427+0000","last_active":"2024-09-12T03:23:55.271885+0000","last_peered":"2024-09-12T03:23:55.271885+0000","last_clean":"2024-09-12T03:23:55.271885+0000","last_became_active":"2024-09-12T03:23:55.270140+0000","last_became_peered":"2024-09-12T03:23:55.270140+0000","last_unstale":"2024-09-12T03:23:55.271885+0000","last_undegraded":"2024-09-12T03:23:55.271885+0000","last_fullsized":"2024-09-12T03:23:55.271885+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_clean_scrub_stamp":"2024-09-12T03:23:54.239693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:47:44.168067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.2a","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-12T03:23:55.279136+0000","last_change":"2024-09-12T03:23:55.277683+0000","last_active":"2024-09-12T03:23:55.279136+0000","last_peered":"2024-09-12T03:23:55.279136+0000","last_clean":"2024-09-12T03:23:55.279136+0000","last_became_active":"2024-09-12T03:23:55.277329+0000","last_became_peered":"2024-09-12T03:23:55.277329+0000","last_unstale":"2024-09-12T03:23:55.279136+0000","last_undegraded":"2024-09-12T03:23:55.279136+0000","last_fullsized":"2024-09-12T03:23:55.279136+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_clean_scrub_stamp":"2024-09-12T03:23:54.239693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:31:10.861866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.29","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-12T03:23:55.282285+0000","last_change":"2024-09-12T03:23:55.281379+0000","last_active":"2024-09-12T03:23:55.282285+0000","last_peered":"2024-09-12T03:23:55.282285+0000","last_clean":"2024-09-12T03:23:55.282285+0000","last_became_active":"2024-09-12T03:23:55.281011+0000","last_became_peered":"2024-09-12T03:23:55.281011+0000","last_unstale":"2024-09-12T03:23:55.282285+0000","last_undegraded":"2024-09-12T03:23:55.282285+0000","last_fullsized":"2024-09-12T03:23:55.282285+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_clean_scrub_stamp":"2024-09-12T03:23:54.239693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:25:02.805426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.28","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-12T03:23:55.272245+0000","last_change":"2024-09-12T03:23:55.270999+0000","last_active":"2024-09-12T03:23:55.272245+0000","last_peered":"2024-09-12T03:23:55.272245+0000","last_clean":"2024-09-12T03:23:55.272245+0000","last_became_active":"2024-09-12T03:23:55.270152+0000","last_became_peered":"2024-09-12T03:23:55.270152+0000","last_unstale":"2024-09-12T03:23:55.272245+0000","last_undegraded":"2024-09-12T03:23:55.272245+0000","last_fullsized":"2024-09-12T03:23:55.272245+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_clean_scrub_stamp":"2024-09-12T03:23:54.239693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:22:07.028924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.27","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-12T03:23:55.279720+0000","last_change":"2024-09-12T03:23:55.278652+0000","last_active":"2024-09-12T03:23:55.279720+0000","last_peered":"2024-09-12T03:23:55.279720+0000","last_clean":"2024-09-12T03:23:55.279720+0000","last_became_active":"2024-09-12T03:23:55.278315+0000","last_became_peered":"2024-09-12T03:23:55.278315+0000","last_unstale":"2024-09-12T03:23:55.279720+0000","last_undegraded":"2024-09-12T03:23:55.279720+0000","last_fullsized":"2024-09-12T03:23:55.279720+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_clean_scrub_stamp":"2024-09-12T03:23:54.239693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:03:31.793088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.26","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-12T03:23:55.287487+0000","last_change":"2024-09-12T03:23:55.282207+0000","last_active":"2024-09-12T03:23:55.287487+0000","last_peered":"2024-09-12T03:23:55.287487+0000","last_clean":"2024-09-12T03:23:55.287487+0000","last_became_active":"2024-09-12T03:23:55.279961+0000","last_became_peered":"2024-09-12T03:23:55.279961+0000","last_unstale":"2024-09-12T03:23:55.287487+0000","last_undegraded":"2024-09-12T03:23:55.287487+0000","last_fullsized":"2024-09-12T03:23:55.287487+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_clean_scrub_stamp":"2024-09-12T03:23:54.239693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:30:24.357977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.25","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-12T03:23:55.267563+0000","last_change":"2024-09-12T03:23:55.266000+0000","last_active":"2024-09-12T03:23:55.267563+0000","last_peered":"2024-09-12T03:23:55.267563+0000","last_clean":"2024-09-12T03:23:55.267563+0000","last_became_active":"2024-09-12T03:23:55.265677+0000","last_became_peered":"2024-09-12T03:23:55.265677+0000","last_unstale":"2024-09-12T03:23:55.267563+0000","last_undegraded":"2024-09-12T03:23:55.267563+0000","last_fullsized":"2024-09-12T03:23:55.267563+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_clean_scrub_stamp":"2024-09-12T03:23:54.239693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:05:58.390413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.24","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-12T03:23:55.268149+0000","last_change":"2024-09-12T03:23:55.265972+0000","last_active":"2024-09-12T03:23:55.268149+0000","last_peered":"2024-09-12T03:23:55.268149+0000","last_clean":"2024-09-12T03:23:55.268149+0000","last_became_active":"2024-09-12T03:23:55.265321+0000","last_became_peered":"2024-09-12T03:23:55.265321+0000","last_unstale":"2024-09-12T03:23:55.268149+0000","last_undegraded":"2024-09-12T03:23:55.268149+0000","last_fullsized":"2024-09-12T03:23:55.268149+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_clean_scrub_stamp":"2024-09-12T03:23:54.239693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:42:50.609549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.23","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-12T03:23:55.267745+0000","last_change":"2024-09-12T03:23:55.266765+0000","last_active":"2024-09-12T03:23:55.267745+0000","last_peered":"2024-09-12T03:23:55.267745+0000","last_clean":"2024-09-12T03:23:55.267745+0000","last_became_active":"2024-09-12T03:23:55.266486+0000","last_became_peered":"2024-09-12T03:23:55.266486+0000","last_unstale":"2024-09-12T03:23:55.267745+0000","last_undegraded":"2024-09-12T03:23:55.267745+0000","last_fullsized":"2024-09-12T03:23:55.267745+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_clean_scrub_stamp":"2024-09-12T03:23:54.239693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:14:42.368090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.22","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-12T03:23:55.274533+0000","last_change":"2024-09-12T03:23:55.274123+0000","last_active":"2024-09-12T03:23:55.274533+0000","last_peered":"2024-09-12T03:23:55.274533+0000","last_clean":"2024-09-12T03:23:55.274533+0000","last_became_active":"2024-09-12T03:23:55.273833+0000","last_became_peered":"2024-09-12T03:23:55.273833+0000","last_unstale":"2024-09-12T03:23:55.274533+0000","last_undegraded":"2024-09-12T03:23:55.274533+0000","last_fullsized":"2024-09-12T03:23:55.274533+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_clean_scrub_stamp":"2024-09-12T03:23:54.239693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:04:16.721862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.21","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-12T03:23:55.280811+0000","last_change":"2024-09-12T03:23:55.272097+0000","last_active":"2024-09-12T03:23:55.280811+0000","last_peered":"2024-09-12T03:23:55.280811+0000","last_clean":"2024-09-12T03:23:55.280811+0000","last_became_active":"2024-09-12T03:23:55.271780+0000","last_became_peered":"2024-09-12T03:23:55.271780+0000","last_unstale":"2024-09-12T03:23:55.280811+0000","last_undegraded":"2024-09-12T03:23:55.280811+0000","last_fullsized":"2024-09-12T03:23:55.280811+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_clean_scrub_stamp":"2024-09-12T03:23:54.239693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:27:48.458762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.20","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-12T03:23:55.285593+0000","last_change":"2024-09-12T03:23:55.278047+0000","last_active":"2024-09-12T03:23:55.285593+0000","last_peered":"2024-09-12T03:23:55.285593+0000","last_clean":"2024-09-12T03:23:55.285593+0000","last_became_active":"2024-09-12T03:23:55.277508+0000","last_became_peered":"2024-09-12T03:23:55.277508+0000","last_unstale":"2024-09-12T03:23:55.285593+0000","last_undegraded":"2024-09-12T03:23:55.285593+0000","last_fullsized":"2024-09-12T03:23:55.285593+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_clean_scrub_stamp":"2024-09-12T03:23:54.239693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:05:01.051455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.1e","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-12T03:23:55.274164+0000","last_change":"2024-09-12T03:23:55.273169+0000","last_active":"2024-09-12T03:23:55.274164+0000","last_peered":"2024-09-12T03:23:55.274164+0000","last_clean":"2024-09-12T03:23:55.274164+0000","last_became_active":"2024-09-12T03:23:55.272025+0000","last_became_peered":"2024-09-12T03:23:55.272025+0000","last_unstale":"2024-09-12T03:23:55.274164+0000","last_undegraded":"2024-09-12T03:23:55.274164+0000","last_fullsized":"2024-09-12T03:23:55.274164+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_clean_scrub_stamp":"2024-09-12T03:23:54.239693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:10:45.502628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.1f","version":"159'4","reported_seq":16,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-12T03:23:55.282797+0000","last_change":"2024-09-12T03:23:54.255998+0000","last_active":"2024-09-12T03:23:55.282797+0000","last_peered":"2024-09-12T03:23:55.282797+0000","last_clean":"2024-09-12T03:23:55.282797+0000","last_became_active":"2024-09-12T03:23:54.255715+0000","last_became_peered":"2024-09-12T03:23:54.255715+0000","last_unstale":"2024-09-12T03:23:55.282797+0000","last_undegraded":"2024-09-12T03:23:55.282797+0000","last_fullsized":"2024-09-12T03:23:55.282797+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:53.234707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:53.234707+0000","last_clean_scrub_stamp":"2024-09-12T03:23:53.234707+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:24:13.256301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1f","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-12T03:23:55.258509+0000","last_change":"2024-09-12T03:23:55.257516+0000","last_active":"2024-09-12T03:23:55.258509+0000","last_peered":"2024-09-12T03:23:55.258509+0000","last_clean":"2024-09-12T03:23:55.258509+0000","last_became_active":"2024-09-12T03:23:55.257183+0000","last_became_peered":"2024-09-12T03:23:55.257183+0000","last_unstale":"2024-09-12T03:23:55.258509+0000","last_undegraded":"2024-09-12T03:23:55.258509+0000","last_fullsized":"2024-09-12T03:23:55.258509+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_clean_scrub_stamp":"2024-09-12T03:23:54.239693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:01:34.396958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.1e","version":"159'1","reported_seq":13,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-12T03:23:55.282384+0000","last_change":"2024-09-12T03:23:54.259810+0000","last_active":"2024-09-12T03:23:55.282384+0000","last_peered":"2024-09-12T03:23:55.282384+0000","last_clean":"2024-09-12T03:23:55.282384+0000","last_became_active":"2024-09-12T03:23:54.259534+0000","last_became_peered":"2024-09-12T03:23:54.259534+0000","last_unstale":"2024-09-12T03:23:55.282384+0000","last_undegraded":"2024-09-12T03:23:55.282384+0000","last_fullsized":"2024-09-12T03:23:55.282384+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:53.234707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:53.234707+0000","last_clean_scrub_stamp":"2024-09-12T03:23:53.234707+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:11:27.739972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.1c","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-12T03:23:55.285116+0000","last_change":"2024-09-12T03:23:55.280641+0000","last_active":"2024-09-12T03:23:55.285116+0000","last_peered":"2024-09-12T03:23:55.285116+0000","last_clean":"2024-09-12T03:23:55.285116+0000","last_became_active":"2024-09-12T03:23:55.280305+0000","last_became_peered":"2024-09-12T03:23:55.280305+0000","last_unstale":"2024-09-12T03:23:55.285116+0000","last_undegraded":"2024-09-12T03:23:55.285116+0000","last_fullsized":"2024-09-12T03:23:55.285116+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_clean_scrub_stamp":"2024-09-12T03:23:54.239693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:03:08.833246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.1d","version":"159'2","reported_seq":14,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-12T03:23:55.288546+0000","last_change":"2024-09-12T03:23:54.255596+0000","last_active":"2024-09-12T03:23:55.288546+0000","last_peered":"2024-09-12T03:23:55.288546+0000","last_clean":"2024-09-12T03:23:55.288546+0000","last_became_active":"2024-09-12T03:23:54.255314+0000","last_became_peered":"2024-09-12T03:23:54.255314+0000","last_unstale":"2024-09-12T03:23:55.288546+0000","last_undegraded":"2024-09-12T03:23:55.288546+0000","last_fullsized":"2024-09-12T03:23:55.288546+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:53.234707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:53.234707+0000","last_clean_scrub_stamp":"2024-09-12T03:23:53.234707+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:53:27.117140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.1d","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-12T03:23:55.281265+0000","last_change":"2024-09-12T03:23:55.275705+0000","last_active":"2024-09-12T03:23:55.281265+0000","last_peered":"2024-09-12T03:23:55.281265+0000","last_clean":"2024-09-12T03:23:55.281265+0000","last_became_active":"2024-09-12T03:23:55.275305+0000","last_became_peered":"2024-09-12T03:23:55.275305+0000","last_unstale":"2024-09-12T03:23:55.281265+0000","last_undegraded":"2024-09-12T03:23:55.281265+0000","last_fullsized":"2024-09-12T03:23:55.281265+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_clean_scrub_stamp":"2024-09-12T03:23:54.239693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:02:20.388100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.1c","version":"159'2","reported_seq":14,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-12T03:23:55.286693+0000","last_change":"2024-09-12T03:23:54.262444+0000","last_active":"2024-09-12T03:23:55.286693+0000","last_peered":"2024-09-12T03:23:55.286693+0000","last_clean":"2024-09-12T03:23:55.286693+0000","last_became_active":"2024-09-12T03:23:54.262144+0000","last_became_peered":"2024-09-12T03:23:54.262144+0000","last_unstale":"2024-09-12T03:23:55.286693+0000","last_undegraded":"2024-09-12T03:23:55.286693+0000","last_fullsized":"2024-09-12T03:23:55.286693+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:53.234707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:53.234707+0000","last_clean_scrub_stamp":"2024-09-12T03:23:53.234707+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:40:55.907954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.1b","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-12T03:23:55.271841+0000","last_change":"2024-09-12T03:23:55.269883+0000","last_active":"2024-09-12T03:23:55.271841+0000","last_peered":"2024-09-12T03:23:55.271841+0000","last_clean":"2024-09-12T03:23:55.271841+0000","last_became_active":"2024-09-12T03:23:55.269556+0000","last_became_peered":"2024-09-12T03:23:55.269556+0000","last_unstale":"2024-09-12T03:23:55.271841+0000","last_undegraded":"2024-09-12T03:23:55.271841+0000","last_fullsized":"2024-09-12T03:23:55.271841+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_clean_scrub_stamp":"2024-09-12T03:23:54.239693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:59:01.300466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.1a","version":"159'1","reported_seq":13,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-12T03:23:55.287997+0000","last_change":"2024-09-12T03:23:54.256999+0000","last_active":"2024-09-12T03:23:55.287997+0000","last_peered":"2024-09-12T03:23:55.287997+0000","last_clean":"2024-09-12T03:23:55.287997+0000","last_became_active":"2024-09-12T03:23:54.256730+0000","last_became_peered":"2024-09-12T03:23:54.256730+0000","last_unstale":"2024-09-12T03:23:55.287997+0000","last_undegraded":"2024-09-12T03:23:55.287997+0000","last_fullsized":"2024-09-12T03:23:55.287997+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:53.234707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:53.234707+0000","last_clean_scrub_stamp":"2024-09-12T03:23:53.234707+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:49:46.267823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.18","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-12T03:23:55.274408+0000","last_change":"2024-09-12T03:23:55.270747+0000","last_active":"2024-09-12T03:23:55.274408+0000","last_peered":"2024-09-12T03:23:55.274408+0000","last_clean":"2024-09-12T03:23:55.274408+0000","last_became_active":"2024-09-12T03:23:55.270464+0000","last_became_peered":"2024-09-12T03:23:55.270464+0000","last_unstale":"2024-09-12T03:23:55.274408+0000","last_undegraded":"2024-09-12T03:23:55.274408+0000","last_fullsized":"2024-09-12T03:23:55.274408+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_clean_scrub_stamp":"2024-09-12T03:23:54.239693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:38:24.490328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.19","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-12T03:23:55.259133+0000","last_change":"2024-09-12T03:23:54.255267+0000","last_active":"2024-09-12T03:23:55.259133+0000","last_peered":"2024-09-12T03:23:55.259133+0000","last_clean":"2024-09-12T03:23:55.259133+0000","last_became_active":"2024-09-12T03:23:54.255020+0000","last_became_peered":"2024-09-12T03:23:54.255020+0000","last_unstale":"2024-09-12T03:23:55.259133+0000","last_undegraded":"2024-09-12T03:23:55.259133+0000","last_fullsized":"2024-09-12T03:23:55.259133+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:53.234707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:53.234707+0000","last_clean_scrub_stamp":"2024-09-12T03:23:53.234707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:27:40.744073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.19","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-12T03:23:55.271916+0000","last_change":"2024-09-12T03:23:55.270662+0000","last_active":"2024-09-12T03:23:55.271916+0000","last_peered":"2024-09-12T03:23:55.271916+0000","last_clean":"2024-09-12T03:23:55.271916+0000","last_became_active":"2024-09-12T03:23:55.263555+0000","last_became_peered":"2024-09-12T03:23:55.263555+0000","last_unstale":"2024-09-12T03:23:55.271916+0000","last_undegraded":"2024-09-12T03:23:55.271916+0000","last_fullsized":"2024-09-12T03:23:55.271916+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_clean_scrub_stamp":"2024-09-12T03:23:54.239693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:02:40.409694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.18","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-12T03:23:55.253830+0000","last_change":"2024-09-12T03:23:54.261128+0000","last_active":"2024-09-12T03:23:55.253830+0000","last_peered":"2024-09-12T03:23:55.253830+0000","last_clean":"2024-09-12T03:23:55.253830+0000","last_became_active":"2024-09-12T03:23:54.260724+0000","last_became_peered":"2024-09-12T03:23:54.260724+0000","last_unstale":"2024-09-12T03:23:55.253830+0000","last_undegraded":"2024-09-12T03:23:55.253830+0000","last_fullsized":"2024-09-12T03:23:55.253830+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:53.234707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:53.234707+0000","last_clean_scrub_stamp":"2024-09-12T03:23:53.234707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:27:01.640325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.16","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-12T03:23:55.268142+0000","last_change":"2024-09-12T03:23:55.266412+0000","last_active":"2024-09-12T03:23:55.268142+0000","last_peered":"2024-09-12T03:23:55.268142+0000","last_clean":"2024-09-12T03:23:55.268142+0000","last_became_active":"2024-09-12T03:23:55.266109+0000","last_became_peered":"2024-09-12T03:23:55.266109+0000","last_unstale":"2024-09-12T03:23:55.268142+0000","last_undegraded":"2024-09-12T03:23:55.268142+0000","last_fullsized":"2024-09-12T03:23:55.268142+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_clean_scrub_stamp":"2024-09-12T03:23:54.239693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:02:20.388100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.17","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-12T03:23:55.259330+0000","last_change":"2024-09-12T03:23:54.255765+0000","last_active":"2024-09-12T03:23:55.259330+0000","last_peered":"2024-09-12T03:23:55.259330+0000","last_clean":"2024-09-12T03:23:55.259330+0000","last_became_active":"2024-09-12T03:23:54.255488+0000","last_became_peered":"2024-09-12T03:23:54.255488+0000","last_unstale":"2024-09-12T03:23:55.259330+0000","last_undegraded":"2024-09-12T03:23:55.259330+0000","last_fullsized":"2024-09-12T03:23:55.259330+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:53.234707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:53.234707+0000","last_clean_scrub_stamp":"2024-09-12T03:23:53.234707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:15:42.450958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.17","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-12T03:23:55.268400+0000","last_change":"2024-09-12T03:23:55.259172+0000","last_active":"2024-09-12T03:23:55.268400+0000","last_peered":"2024-09-12T03:23:55.268400+0000","last_clean":"2024-09-12T03:23:55.268400+0000","last_became_active":"2024-09-12T03:23:55.258819+0000","last_became_peered":"2024-09-12T03:23:55.258819+0000","last_unstale":"2024-09-12T03:23:55.268400+0000","last_undegraded":"2024-09-12T03:23:55.268400+0000","last_fullsized":"2024-09-12T03:23:55.268400+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_clean_scrub_stamp":"2024-09-12T03:23:54.239693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:03:06.964137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.16","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-12T03:23:55.253882+0000","last_change":"2024-09-12T03:23:54.256120+0000","last_active":"2024-09-12T03:23:55.253882+0000","last_peered":"2024-09-12T03:23:55.253882+0000","last_clean":"2024-09-12T03:23:55.253882+0000","last_became_active":"2024-09-12T03:23:54.255758+0000","last_became_peered":"2024-09-12T03:23:54.255758+0000","last_unstale":"2024-09-12T03:23:55.253882+0000","last_undegraded":"2024-09-12T03:23:55.253882+0000","last_fullsized":"2024-09-12T03:23:55.253882+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:53.234707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:53.234707+0000","last_clean_scrub_stamp":"2024-09-12T03:23:53.234707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:04:54.577334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.14","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-12T03:23:55.268171+0000","last_change":"2024-09-12T03:23:55.265354+0000","last_active":"2024-09-12T03:23:55.268171+0000","last_peered":"2024-09-12T03:23:55.268171+0000","last_clean":"2024-09-12T03:23:55.268171+0000","last_became_active":"2024-09-12T03:23:55.265025+0000","last_became_peered":"2024-09-12T03:23:55.265025+0000","last_unstale":"2024-09-12T03:23:55.268171+0000","last_undegraded":"2024-09-12T03:23:55.268171+0000","last_fullsized":"2024-09-12T03:23:55.268171+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_clean_scrub_stamp":"2024-09-12T03:23:54.239693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:51:09.764444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.15","version":"159'3","reported_seq":15,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-12T03:23:55.290523+0000","last_change":"2024-09-12T03:23:54.260535+0000","last_active":"2024-09-12T03:23:55.290523+0000","last_peered":"2024-09-12T03:23:55.290523+0000","last_clean":"2024-09-12T03:23:55.290523+0000","last_became_active":"2024-09-12T03:23:54.259790+0000","last_became_peered":"2024-09-12T03:23:54.259790+0000","last_unstale":"2024-09-12T03:23:55.290523+0000","last_undegraded":"2024-09-12T03:23:55.290523+0000","last_fullsized":"2024-09-12T03:23:55.290523+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:53.234707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:53.234707+0000","last_clean_scrub_stamp":"2024-09-12T03:23:53.234707+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:54:03.590117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.15","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-12T03:23:55.281162+0000","last_change":"2024-09-12T03:23:55.275215+0000","last_active":"2024-09-12T03:23:55.281162+0000","last_peered":"2024-09-12T03:23:55.281162+0000","last_clean":"2024-09-12T03:23:55.281162+0000","last_became_active":"2024-09-12T03:23:55.274920+0000","last_became_peered":"2024-09-12T03:23:55.274920+0000","last_unstale":"2024-09-12T03:23:55.281162+0000","last_undegraded":"2024-09-12T03:23:55.281162+0000","last_fullsized":"2024-09-12T03:23:55.281162+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_clean_scrub_stamp":"2024-09-12T03:23:54.239693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:17:53.462125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.14","version":"159'6","reported_seq":18,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-12T03:23:55.288662+0000","last_change":"2024-09-12T03:23:54.263126+0000","last_active":"2024-09-12T03:23:55.288662+0000","last_peered":"2024-09-12T03:23:55.288662+0000","last_clean":"2024-09-12T03:23:55.288662+0000","last_became_active":"2024-09-12T03:23:54.262829+0000","last_became_peered":"2024-09-12T03:23:54.262829+0000","last_unstale":"2024-09-12T03:23:55.288662+0000","last_undegraded":"2024-09-12T03:23:55.288662+0000","last_fullsized":"2024-09-12T03:23:55.288662+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:53.234707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:53.234707+0000","last_clean_scrub_stamp":"2024-09-12T03:23:53.234707+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:38:58.250398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.12","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-12T03:23:55.282917+0000","last_change":"2024-09-12T03:23:55.279095+0000","last_active":"2024-09-12T03:23:55.282917+0000","last_peered":"2024-09-12T03:23:55.282917+0000","last_clean":"2024-09-12T03:23:55.282917+0000","last_became_active":"2024-09-12T03:23:55.278829+0000","last_became_peered":"2024-09-12T03:23:55.278829+0000","last_unstale":"2024-09-12T03:23:55.282917+0000","last_undegraded":"2024-09-12T03:23:55.282917+0000","last_fullsized":"2024-09-12T03:23:55.282917+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_clean_scrub_stamp":"2024-09-12T03:23:54.239693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:04:30.400384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.13","version":"159'2","reported_seq":14,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-12T03:23:55.288370+0000","last_change":"2024-09-12T03:23:54.259701+0000","last_active":"2024-09-12T03:23:55.288370+0000","last_peered":"2024-09-12T03:23:55.288370+0000","last_clean":"2024-09-12T03:23:55.288370+0000","last_became_active":"2024-09-12T03:23:54.259381+0000","last_became_peered":"2024-09-12T03:23:54.259381+0000","last_unstale":"2024-09-12T03:23:55.288370+0000","last_undegraded":"2024-09-12T03:23:55.288370+0000","last_fullsized":"2024-09-12T03:23:55.288370+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:53.234707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:53.234707+0000","last_clean_scrub_stamp":"2024-09-12T03:23:53.234707+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:37:05.036339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.13","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-12T03:23:55.281532+0000","last_change":"2024-09-12T03:23:55.268117+0000","last_active":"2024-09-12T03:23:55.281532+0000","last_peered":"2024-09-12T03:23:55.281532+0000","last_clean":"2024-09-12T03:23:55.281532+0000","last_became_active":"2024-09-12T03:23:55.267842+0000","last_became_peered":"2024-09-12T03:23:55.267842+0000","last_unstale":"2024-09-12T03:23:55.281532+0000","last_undegraded":"2024-09-12T03:23:55.281532+0000","last_fullsized":"2024-09-12T03:23:55.281532+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_clean_scrub_stamp":"2024-09-12T03:23:54.239693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:40:56.912940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.12","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-12T03:23:55.254318+0000","last_change":"2024-09-12T03:23:54.261257+0000","last_active":"2024-09-12T03:23:55.254318+0000","last_peered":"2024-09-12T03:23:55.254318+0000","last_clean":"2024-09-12T03:23:55.254318+0000","last_became_active":"2024-09-12T03:23:54.260959+0000","last_became_peered":"2024-09-12T03:23:54.260959+0000","last_unstale":"2024-09-12T03:23:55.254318+0000","last_undegraded":"2024-09-12T03:23:55.254318+0000","last_fullsized":"2024-09-12T03:23:55.254318+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:53.234707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:53.234707+0000","last_clean_scrub_stamp":"2024-09-12T03:23:53.234707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:44:36.273674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.10","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-12T03:23:55.262657+0000","last_change":"2024-09-12T03:23:55.257770+0000","last_active":"2024-09-12T03:23:55.262657+0000","last_peered":"2024-09-12T03:23:55.262657+0000","last_clean":"2024-09-12T03:23:55.262657+0000","last_became_active":"2024-09-12T03:23:55.257458+0000","last_became_peered":"2024-09-12T03:23:55.257458+0000","last_unstale":"2024-09-12T03:23:55.262657+0000","last_undegraded":"2024-09-12T03:23:55.262657+0000","last_fullsized":"2024-09-12T03:23:55.262657+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_clean_scrub_stamp":"2024-09-12T03:23:54.239693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:51:35.439697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.11","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-12T03:23:55.265392+0000","last_change":"2024-09-12T03:23:54.255118+0000","last_active":"2024-09-12T03:23:55.265392+0000","last_peered":"2024-09-12T03:23:55.265392+0000","last_clean":"2024-09-12T03:23:55.265392+0000","last_became_active":"2024-09-12T03:23:54.254853+0000","last_became_peered":"2024-09-12T03:23:54.254853+0000","last_unstale":"2024-09-12T03:23:55.265392+0000","last_undegraded":"2024-09-12T03:23:55.265392+0000","last_fullsized":"2024-09-12T03:23:55.265392+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:53.234707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:53.234707+0000","last_clean_scrub_stamp":"2024-09-12T03:23:53.234707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:32:51.087827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.11","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-12T03:23:55.285442+0000","last_change":"2024-09-12T03:23:55.278066+0000","last_active":"2024-09-12T03:23:55.285442+0000","last_peered":"2024-09-12T03:23:55.285442+0000","last_clean":"2024-09-12T03:23:55.285442+0000","last_became_active":"2024-09-12T03:23:55.277315+0000","last_became_peered":"2024-09-12T03:23:55.277315+0000","last_unstale":"2024-09-12T03:23:55.285442+0000","last_undegraded":"2024-09-12T03:23:55.285442+0000","last_fullsized":"2024-09-12T03:23:55.285442+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_clean_scrub_stamp":"2024-09-12T03:23:54.239693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:48:03.817535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.10","version":"159'8","reported_seq":20,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-12T03:23:55.290774+0000","last_change":"2024-09-12T03:23:54.254028+0000","last_active":"2024-09-12T03:23:55.290774+0000","last_peered":"2024-09-12T03:23:55.290774+0000","last_clean":"2024-09-12T03:23:55.290774+0000","last_became_active":"2024-09-12T03:23:54.253704+0000","last_became_peered":"2024-09-12T03:23:54.253704+0000","last_unstale":"2024-09-12T03:23:55.290774+0000","last_undegraded":"2024-09-12T03:23:55.290774+0000","last_fullsized":"2024-09-12T03:23:55.290774+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:53.234707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:53.234707+0000","last_clean_scrub_stamp":"2024-09-12T03:23:53.234707+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:36:13.511928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.e","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-12T03:23:55.269586+0000","last_change":"2024-09-12T03:23:55.266399+0000","last_active":"2024-09-12T03:23:55.269586+0000","last_peered":"2024-09-12T03:23:55.269586+0000","last_clean":"2024-09-12T03:23:55.269586+0000","last_became_active":"2024-09-12T03:23:55.266113+0000","last_became_peered":"2024-09-12T03:23:55.266113+0000","last_unstale":"2024-09-12T03:23:55.269586+0000","last_undegraded":"2024-09-12T03:23:55.269586+0000","last_fullsized":"2024-09-12T03:23:55.269586+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_clean_scrub_stamp":"2024-09-12T03:23:54.239693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:15:19.602139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.f","version":"159'1","reported_seq":13,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-12T03:23:55.290548+0000","last_change":"2024-09-12T03:23:54.261368+0000","last_active":"2024-09-12T03:23:55.290548+0000","last_peered":"2024-09-12T03:23:55.290548+0000","last_clean":"2024-09-12T03:23:55.290548+0000","last_became_active":"2024-09-12T03:23:54.261097+0000","last_became_peered":"2024-09-12T03:23:54.261097+0000","last_unstale":"2024-09-12T03:23:55.290548+0000","last_undegraded":"2024-09-12T03:23:55.290548+0000","last_fullsized":"2024-09-12T03:23:55.290548+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:53.234707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:53.234707+0000","last_clean_scrub_stamp":"2024-09-12T03:23:53.234707+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:07:31.988105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.f","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-12T03:23:55.280192+0000","last_change":"2024-09-12T03:23:55.279580+0000","last_active":"2024-09-12T03:23:55.280192+0000","last_peered":"2024-09-12T03:23:55.280192+0000","last_clean":"2024-09-12T03:23:55.280192+0000","last_became_active":"2024-09-12T03:23:55.277901+0000","last_became_peered":"2024-09-12T03:23:55.277901+0000","last_unstale":"2024-09-12T03:23:55.280192+0000","last_undegraded":"2024-09-12T03:23:55.280192+0000","last_fullsized":"2024-09-12T03:23:55.280192+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_clean_scrub_stamp":"2024-09-12T03:23:54.239693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:19:21.998591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.e","version":"159'2","reported_seq":14,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-12T03:23:55.283889+0000","last_change":"2024-09-12T03:23:54.258644+0000","last_active":"2024-09-12T03:23:55.283889+0000","last_peered":"2024-09-12T03:23:55.283889+0000","last_clean":"2024-09-12T03:23:55.283889+0000","last_became_active":"2024-09-12T03:23:54.258276+0000","last_became_peered":"2024-09-12T03:23:54.258276+0000","last_unstale":"2024-09-12T03:23:55.283889+0000","last_undegraded":"2024-09-12T03:23:55.283889+0000","last_fullsized":"2024-09-12T03:23:55.283889+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:53.234707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:53.234707+0000","last_clean_scrub_stamp":"2024-09-12T03:23:53.234707+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:34:09.982809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.c","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-12T03:23:55.268172+0000","last_change":"2024-09-12T03:23:55.266199+0000","last_active":"2024-09-12T03:23:55.268172+0000","last_peered":"2024-09-12T03:23:55.268172+0000","last_clean":"2024-09-12T03:23:55.268172+0000","last_became_active":"2024-09-12T03:23:55.265698+0000","last_became_peered":"2024-09-12T03:23:55.265698+0000","last_unstale":"2024-09-12T03:23:55.268172+0000","last_undegraded":"2024-09-12T03:23:55.268172+0000","last_fullsized":"2024-09-12T03:23:55.268172+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_clean_scrub_stamp":"2024-09-12T03:23:54.239693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:37:13.236582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.d","version":"159'1","reported_seq":13,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-12T03:23:55.288409+0000","last_change":"2024-09-12T03:23:54.258228+0000","last_active":"2024-09-12T03:23:55.288409+0000","last_peered":"2024-09-12T03:23:55.288409+0000","last_clean":"2024-09-12T03:23:55.288409+0000","last_became_active":"2024-09-12T03:23:54.257966+0000","last_became_peered":"2024-09-12T03:23:54.257966+0000","last_unstale":"2024-09-12T03:23:55.288409+0000","last_undegraded":"2024-09-12T03:23:55.288409+0000","last_fullsized":"2024-09-12T03:23:55.288409+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:53.234707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:53.234707+0000","last_clean_scrub_stamp":"2024-09-12T03:23:53.234707+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:51:00.263580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.d","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-12T03:23:55.281243+0000","last_change":"2024-09-12T03:23:55.275572+0000","last_active":"2024-09-12T03:23:55.281243+0000","last_peered":"2024-09-12T03:23:55.281243+0000","last_clean":"2024-09-12T03:23:55.281243+0000","last_became_active":"2024-09-12T03:23:55.275020+0000","last_became_peered":"2024-09-12T03:23:55.275020+0000","last_unstale":"2024-09-12T03:23:55.281243+0000","last_undegraded":"2024-09-12T03:23:55.281243+0000","last_fullsized":"2024-09-12T03:23:55.281243+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_clean_scrub_stamp":"2024-09-12T03:23:54.239693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:51:09.764444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.c","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-12T03:23:55.255183+0000","last_change":"2024-09-12T03:23:54.258898+0000","last_active":"2024-09-12T03:23:55.255183+0000","last_peered":"2024-09-12T03:23:55.255183+0000","last_clean":"2024-09-12T03:23:55.255183+0000","last_became_active":"2024-09-12T03:23:54.258645+0000","last_became_peered":"2024-09-12T03:23:54.258645+0000","last_unstale":"2024-09-12T03:23:55.255183+0000","last_undegraded":"2024-09-12T03:23:55.255183+0000","last_fullsized":"2024-09-12T03:23:55.255183+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:53.234707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:53.234707+0000","last_clean_scrub_stamp":"2024-09-12T03:23:53.234707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:55:50.101373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.a","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-12T03:23:55.282995+0000","last_change":"2024-09-12T03:23:55.276422+0000","last_active":"2024-09-12T03:23:55.282995+0000","last_peered":"2024-09-12T03:23:55.282995+0000","last_clean":"2024-09-12T03:23:55.282995+0000","last_became_active":"2024-09-12T03:23:55.276066+0000","last_became_peered":"2024-09-12T03:23:55.276066+0000","last_unstale":"2024-09-12T03:23:55.282995+0000","last_undegraded":"2024-09-12T03:23:55.282995+0000","last_fullsized":"2024-09-12T03:23:55.282995+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_clean_scrub_stamp":"2024-09-12T03:23:54.239693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:59:01.300466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.b","version":"159'2","reported_seq":14,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-12T03:23:55.290690+0000","last_change":"2024-09-12T03:23:54.254553+0000","last_active":"2024-09-12T03:23:55.290690+0000","last_peered":"2024-09-12T03:23:55.290690+0000","last_clean":"2024-09-12T03:23:55.290690+0000","last_became_active":"2024-09-12T03:23:54.254262+0000","last_became_peered":"2024-09-12T03:23:54.254262+0000","last_unstale":"2024-09-12T03:23:55.290690+0000","last_undegraded":"2024-09-12T03:23:55.290690+0000","last_fullsized":"2024-09-12T03:23:55.290690+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:53.234707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:53.234707+0000","last_clean_scrub_stamp":"2024-09-12T03:23:53.234707+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:36:21.797718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.b","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-12T03:23:55.270347+0000","last_change":"2024-09-12T03:23:55.267350+0000","last_active":"2024-09-12T03:23:55.270347+0000","last_peered":"2024-09-12T03:23:55.270347+0000","last_clean":"2024-09-12T03:23:55.270347+0000","last_became_active":"2024-09-12T03:23:55.267062+0000","last_became_peered":"2024-09-12T03:23:55.267062+0000","last_unstale":"2024-09-12T03:23:55.270347+0000","last_undegraded":"2024-09-12T03:23:55.270347+0000","last_fullsized":"2024-09-12T03:23:55.270347+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_clean_scrub_stamp":"2024-09-12T03:23:54.239693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:47:22.648161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.a","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-12T03:23:55.272083+0000","last_change":"2024-09-12T03:23:54.262792+0000","last_active":"2024-09-12T03:23:55.272083+0000","last_peered":"2024-09-12T03:23:55.272083+0000","last_clean":"2024-09-12T03:23:55.272083+0000","last_became_active":"2024-09-12T03:23:54.262164+0000","last_became_peered":"2024-09-12T03:23:54.262164+0000","last_unstale":"2024-09-12T03:23:55.272083+0000","last_undegraded":"2024-09-12T03:23:55.272083+0000","last_fullsized":"2024-09-12T03:23:55.272083+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:53.234707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:53.234707+0000","last_clean_scrub_stamp":"2024-09-12T03:23:53.234707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:52:31.483458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.8","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-12T03:23:55.274009+0000","last_change":"2024-09-12T03:23:55.270747+0000","last_active":"2024-09-12T03:23:55.274009+0000","last_peered":"2024-09-12T03:23:55.274009+0000","last_clean":"2024-09-12T03:23:55.274009+0000","last_became_active":"2024-09-12T03:23:55.270467+0000","last_became_peered":"2024-09-12T03:23:55.270467+0000","last_unstale":"2024-09-12T03:23:55.274009+0000","last_undegraded":"2024-09-12T03:23:55.274009+0000","last_fullsized":"2024-09-12T03:23:55.274009+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_clean_scrub_stamp":"2024-09-12T03:23:54.239693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:44:35.446917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.9","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-12T03:23:55.267688+0000","last_change":"2024-09-12T03:23:54.262702+0000","last_active":"2024-09-12T03:23:55.267688+0000","last_peered":"2024-09-12T03:23:55.267688+0000","last_clean":"2024-09-12T03:23:55.267688+0000","last_became_active":"2024-09-12T03:23:54.262374+0000","last_became_peered":"2024-09-12T03:23:54.262374+0000","last_unstale":"2024-09-12T03:23:55.267688+0000","last_undegraded":"2024-09-12T03:23:55.267688+0000","last_fullsized":"2024-09-12T03:23:55.267688+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:53.234707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:53.234707+0000","last_clean_scrub_stamp":"2024-09-12T03:23:53.234707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:54:22.372337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.9","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-12T03:23:55.258249+0000","last_change":"2024-09-12T03:23:55.257472+0000","last_active":"2024-09-12T03:23:55.258249+0000","last_peered":"2024-09-12T03:23:55.258249+0000","last_clean":"2024-09-12T03:23:55.258249+0000","last_became_active":"2024-09-12T03:23:55.257163+0000","last_became_peered":"2024-09-12T03:23:55.257163+0000","last_unstale":"2024-09-12T03:23:55.258249+0000","last_undegraded":"2024-09-12T03:23:55.258249+0000","last_fullsized":"2024-09-12T03:23:55.258249+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_clean_scrub_stamp":"2024-09-12T03:23:54.239693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:17:51.864997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.8","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-12T03:23:55.263080+0000","last_change":"2024-09-12T03:23:54.258374+0000","last_active":"2024-09-12T03:23:55.263080+0000","last_peered":"2024-09-12T03:23:55.263080+0000","last_clean":"2024-09-12T03:23:55.263080+0000","last_became_active":"2024-09-12T03:23:54.258137+0000","last_became_peered":"2024-09-12T03:23:54.258137+0000","last_unstale":"2024-09-12T03:23:55.263080+0000","last_undegraded":"2024-09-12T03:23:55.263080+0000","last_fullsized":"2024-09-12T03:23:55.263080+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:53.234707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:53.234707+0000","last_clean_scrub_stamp":"2024-09-12T03:23:53.234707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:18:57.892973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.6","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-12T03:23:55.265916+0000","last_change":"2024-09-12T03:23:55.265166+0000","last_active":"2024-09-12T03:23:55.265916+0000","last_peered":"2024-09-12T03:23:55.265916+0000","last_clean":"2024-09-12T03:23:55.265916+0000","last_became_active":"2024-09-12T03:23:55.264850+0000","last_became_peered":"2024-09-12T03:23:55.264850+0000","last_unstale":"2024-09-12T03:23:55.265916+0000","last_undegraded":"2024-09-12T03:23:55.265916+0000","last_fullsized":"2024-09-12T03:23:55.265916+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_clean_scrub_stamp":"2024-09-12T03:23:54.239693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:23:43.900584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.7","version":"159'2","reported_seq":14,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-12T03:23:55.288345+0000","last_change":"2024-09-12T03:23:54.262381+0000","last_active":"2024-09-12T03:23:55.288345+0000","last_peered":"2024-09-12T03:23:55.288345+0000","last_clean":"2024-09-12T03:23:55.288345+0000","last_became_active":"2024-09-12T03:23:54.262125+0000","last_became_peered":"2024-09-12T03:23:54.262125+0000","last_unstale":"2024-09-12T03:23:55.288345+0000","last_undegraded":"2024-09-12T03:23:55.288345+0000","last_fullsized":"2024-09-12T03:23:55.288345+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:53.234707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:53.234707+0000","last_clean_scrub_stamp":"2024-09-12T03:23:53.234707+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:35:31.033990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.7","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-12T03:23:55.271130+0000","last_change":"2024-09-12T03:23:55.265159+0000","last_active":"2024-09-12T03:23:55.271130+0000","last_peered":"2024-09-12T03:23:55.271130+0000","last_clean":"2024-09-12T03:23:55.271130+0000","last_became_active":"2024-09-12T03:23:55.264843+0000","last_became_peered":"2024-09-12T03:23:55.264843+0000","last_unstale":"2024-09-12T03:23:55.271130+0000","last_undegraded":"2024-09-12T03:23:55.271130+0000","last_fullsized":"2024-09-12T03:23:55.271130+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_clean_scrub_stamp":"2024-09-12T03:23:54.239693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:51:23.922711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.6","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-12T03:23:55.259863+0000","last_change":"2024-09-12T03:23:54.257278+0000","last_active":"2024-09-12T03:23:55.259863+0000","last_peered":"2024-09-12T03:23:55.259863+0000","last_clean":"2024-09-12T03:23:55.259863+0000","last_became_active":"2024-09-12T03:23:54.256990+0000","last_became_peered":"2024-09-12T03:23:54.256990+0000","last_unstale":"2024-09-12T03:23:55.259863+0000","last_undegraded":"2024-09-12T03:23:55.259863+0000","last_fullsized":"2024-09-12T03:23:55.259863+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:53.234707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:53.234707+0000","last_clean_scrub_stamp":"2024-09-12T03:23:53.234707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:27:47.453776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.4","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-12T03:23:55.274106+0000","last_change":"2024-09-12T03:23:55.272962+0000","last_active":"2024-09-12T03:23:55.274106+0000","last_peered":"2024-09-12T03:23:55.274106+0000","last_clean":"2024-09-12T03:23:55.274106+0000","last_became_active":"2024-09-12T03:23:55.272663+0000","last_became_peered":"2024-09-12T03:23:55.272663+0000","last_unstale":"2024-09-12T03:23:55.274106+0000","last_undegraded":"2024-09-12T03:23:55.274106+0000","last_fullsized":"2024-09-12T03:23:55.274106+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_clean_scrub_stamp":"2024-09-12T03:23:54.239693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:09:38.563790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.5","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-12T03:23:55.263169+0000","last_change":"2024-09-12T03:23:54.254662+0000","last_active":"2024-09-12T03:23:55.263169+0000","last_peered":"2024-09-12T03:23:55.263169+0000","last_clean":"2024-09-12T03:23:55.263169+0000","last_became_active":"2024-09-12T03:23:54.254337+0000","last_became_peered":"2024-09-12T03:23:54.254337+0000","last_unstale":"2024-09-12T03:23:55.263169+0000","last_undegraded":"2024-09-12T03:23:55.263169+0000","last_fullsized":"2024-09-12T03:23:55.263169+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:53.234707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:53.234707+0000","last_clean_scrub_stamp":"2024-09-12T03:23:53.234707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:09:49.378880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.5","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-12T03:23:55.271981+0000","last_change":"2024-09-12T03:23:55.270624+0000","last_active":"2024-09-12T03:23:55.271981+0000","last_peered":"2024-09-12T03:23:55.271981+0000","last_clean":"2024-09-12T03:23:55.271981+0000","last_became_active":"2024-09-12T03:23:55.270340+0000","last_became_peered":"2024-09-12T03:23:55.270340+0000","last_unstale":"2024-09-12T03:23:55.271981+0000","last_undegraded":"2024-09-12T03:23:55.271981+0000","last_fullsized":"2024-09-12T03:23:55.271981+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_clean_scrub_stamp":"2024-09-12T03:23:54.239693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:03:04.676600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.4","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-12T03:23:55.255366+0000","last_change":"2024-09-12T03:23:54.260974+0000","last_active":"2024-09-12T03:23:55.255366+0000","last_peered":"2024-09-12T03:23:55.255366+0000","last_clean":"2024-09-12T03:23:55.255366+0000","last_became_active":"2024-09-12T03:23:54.260713+0000","last_became_peered":"2024-09-12T03:23:54.260713+0000","last_unstale":"2024-09-12T03:23:55.255366+0000","last_undegraded":"2024-09-12T03:23:55.255366+0000","last_fullsized":"2024-09-12T03:23:55.255366+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:53.234707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:53.234707+0000","last_clean_scrub_stamp":"2024-09-12T03:23:53.234707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:34:50.922929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.2","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-12T03:23:55.287489+0000","last_change":"2024-09-12T03:23:55.282009+0000","last_active":"2024-09-12T03:23:55.287489+0000","last_peered":"2024-09-12T03:23:55.287489+0000","last_clean":"2024-09-12T03:23:55.287489+0000","last_became_active":"2024-09-12T03:23:55.281697+0000","last_became_peered":"2024-09-12T03:23:55.281697+0000","last_unstale":"2024-09-12T03:23:55.287489+0000","last_undegraded":"2024-09-12T03:23:55.287489+0000","last_fullsized":"2024-09-12T03:23:55.287489+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_clean_scrub_stamp":"2024-09-12T03:23:54.239693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:04:49.963270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.3","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-12T03:23:55.259109+0000","last_change":"2024-09-12T03:23:54.262104+0000","last_active":"2024-09-12T03:23:55.259109+0000","last_peered":"2024-09-12T03:23:55.259109+0000","last_clean":"2024-09-12T03:23:55.259109+0000","last_became_active":"2024-09-12T03:23:54.261512+0000","last_became_peered":"2024-09-12T03:23:54.261512+0000","last_unstale":"2024-09-12T03:23:55.259109+0000","last_undegraded":"2024-09-12T03:23:55.259109+0000","last_fullsized":"2024-09-12T03:23:55.259109+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:53.234707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:53.234707+0000","last_clean_scrub_stamp":"2024-09-12T03:23:53.234707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:51:00.263580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.3","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-12T03:23:55.279189+0000","last_change":"2024-09-12T03:23:55.278264+0000","last_active":"2024-09-12T03:23:55.279189+0000","last_peered":"2024-09-12T03:23:55.279189+0000","last_clean":"2024-09-12T03:23:55.279189+0000","last_became_active":"2024-09-12T03:23:55.277941+0000","last_became_peered":"2024-09-12T03:23:55.277941+0000","last_unstale":"2024-09-12T03:23:55.279189+0000","last_undegraded":"2024-09-12T03:23:55.279189+0000","last_fullsized":"2024-09-12T03:23:55.279189+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_clean_scrub_stamp":"2024-09-12T03:23:54.239693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:27:02.645311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.2","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-12T03:23:55.264431+0000","last_change":"2024-09-12T03:23:54.261772+0000","last_active":"2024-09-12T03:23:55.264431+0000","last_peered":"2024-09-12T03:23:55.264431+0000","last_clean":"2024-09-12T03:23:55.264431+0000","last_became_active":"2024-09-12T03:23:54.261165+0000","last_became_peered":"2024-09-12T03:23:54.261165+0000","last_unstale":"2024-09-12T03:23:55.264431+0000","last_undegraded":"2024-09-12T03:23:55.264431+0000","last_fullsized":"2024-09-12T03:23:55.264431+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:53.234707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:53.234707+0000","last_clean_scrub_stamp":"2024-09-12T03:23:53.234707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:41:53.579141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.0","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-12T03:23:55.268259+0000","last_change":"2024-09-12T03:23:55.262104+0000","last_active":"2024-09-12T03:23:55.268259+0000","last_peered":"2024-09-12T03:23:55.268259+0000","last_clean":"2024-09-12T03:23:55.268259+0000","last_became_active":"2024-09-12T03:23:55.261767+0000","last_became_peered":"2024-09-12T03:23:55.261767+0000","last_unstale":"2024-09-12T03:23:55.268259+0000","last_undegraded":"2024-09-12T03:23:55.268259+0000","last_fullsized":"2024-09-12T03:23:55.268259+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_clean_scrub_stamp":"2024-09-12T03:23:54.239693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:48:22.595326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.1","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-12T03:23:55.255347+0000","last_change":"2024-09-12T03:23:54.253565+0000","last_active":"2024-09-12T03:23:55.255347+0000","last_peered":"2024-09-12T03:23:55.255347+0000","last_clean":"2024-09-12T03:23:55.255347+0000","last_became_active":"2024-09-12T03:23:54.253293+0000","last_became_peered":"2024-09-12T03:23:54.253293+0000","last_unstale":"2024-09-12T03:23:55.255347+0000","last_undegraded":"2024-09-12T03:23:55.255347+0000","last_fullsized":"2024-09-12T03:23:55.255347+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:53.234707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:53.234707+0000","last_clean_scrub_stamp":"2024-09-12T03:23:53.234707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:44:07.301758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.1","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-12T03:23:55.272536+0000","last_change":"2024-09-12T03:23:55.267442+0000","last_active":"2024-09-12T03:23:55.272536+0000","last_peered":"2024-09-12T03:23:55.272536+0000","last_clean":"2024-09-12T03:23:55.272536+0000","last_became_active":"2024-09-12T03:23:55.266405+0000","last_became_peered":"2024-09-12T03:23:55.266405+0000","last_unstale":"2024-09-12T03:23:55.272536+0000","last_undegraded":"2024-09-12T03:23:55.272536+0000","last_fullsized":"2024-09-12T03:23:55.272536+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_clean_scrub_stamp":"2024-09-12T03:23:54.239693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:20:40.412792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.0","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-12T03:23:55.253970+0000","last_change":"2024-09-12T03:23:54.260836+0000","last_active":"2024-09-12T03:23:55.253970+0000","last_peered":"2024-09-12T03:23:55.253970+0000","last_clean":"2024-09-12T03:23:55.253970+0000","last_became_active":"2024-09-12T03:23:54.260466+0000","last_became_peered":"2024-09-12T03:23:54.260466+0000","last_unstale":"2024-09-12T03:23:55.253970+0000","last_undegraded":"2024-09-12T03:23:55.253970+0000","last_fullsized":"2024-09-12T03:23:55.253970+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:53.234707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:53.234707+0000","last_clean_scrub_stamp":"2024-09-12T03:23:53.234707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:46:58.963165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1a","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-12T03:23:55.287712+0000","last_change":"2024-09-12T03:23:55.285727+0000","last_active":"2024-09-12T03:23:55.287712+0000","last_peered":"2024-09-12T03:23:55.287712+0000","last_clean":"2024-09-12T03:23:55.287712+0000","last_became_active":"2024-09-12T03:23:55.285409+0000","last_became_peered":"2024-09-12T03:23:55.285409+0000","last_unstale":"2024-09-12T03:23:55.287712+0000","last_undegraded":"2024-09-12T03:23:55.287712+0000","last_fullsized":"2024-09-12T03:23:55.287712+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:54.239693+0000","last_clean_scrub_stamp":"2024-09-12T03:23:54.239693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:37:35.554458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.1b","version":"159'1","reported_seq":13,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-12T03:23:55.287024+0000","last_change":"2024-09-12T03:23:54.259689+0000","last_active":"2024-09-12T03:23:55.287024+0000","last_peered":"2024-09-12T03:23:55.287024+0000","last_clean":"2024-09-12T03:23:55.287024+0000","last_became_active":"2024-09-12T03:23:54.259354+0000","last_became_peered":"2024-09-12T03:23:54.259354+0000","last_unstale":"2024-09-12T03:23:55.287024+0000","last_undegraded":"2024-09-12T03:23:55.287024+0000","last_fullsized":"2024-09-12T03:23:55.287024+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:23:53.234707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:23:53.234707+0000","last_clean_scrub_stamp":"2024-09-12T03:23:53.234707+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:42:44.496439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"70'128","reported_seq":462,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-12T03:23:55.258691+0000","last_change":"2024-09-12T03:16:21.883446+0000","last_active":"2024-09-12T03:23:55.258691+0000","last_peered":"2024-09-12T03:23:55.258691+0000","last_clean":"2024-09-12T03:23:55.258691+0000","last_became_active":"2024-09-12T03:16:21.882207+0000","last_became_peered":"2024-09-12T03:16:21.882207+0000","last_unstale":"2024-09-12T03:23:55.258691+0000","last_undegraded":"2024-09-12T03:23:55.258691+0000","last_fullsized":"2024-09-12T03:23:55.258691+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:20.316426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:20.316426+0000","last_clean_scrub_stamp":"2024-09-12T03:16:20.316426+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:12:15.424320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":26,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":27,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542237,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37908,"kb_used_data":2552,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333932,"statfs":{"total":96636764160,"available":96597946368,"internally_reserved":0,"allocated":2613248,"data_stored":1480196,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542239,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37320,"kb_used_data":1964,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334520,"statfs":{"total":96636764160,"available":96598548480,"internally_reserved":0,"allocated":2011136,"data_stored":888798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542238,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41456,"kb_used_data":1948,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330384,"statfs":{"total":96636764160,"available":96594313216,"internally_reserved":0,"allocated":1994752,"data_stored":881385,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542237,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37300,"kb_used_data":1948,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334540,"statfs":{"total":96636764160,"available":96598568960,"internally_reserved":0,"allocated":1994752,"data_stored":881385,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542238,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33164,"kb_used_data":1972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338676,"statfs":{"total":96636764160,"available":96602804224,"internally_reserved":0,"allocated":2019328,"data_stored":888398,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542238,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33148,"kb_used_data":1960,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338692,"statfs":{"total":96636764160,"available":96602820608,"internally_reserved":0,"allocated":2007040,"data_stored":888216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542239,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33156,"kb_used_data":1968,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338684,"statfs":{"total":96636764160,"available":96602812416,"internally_reserved":0,"allocated":2015232,"data_stored":888272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542239,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33736,"kb_used_data":2544,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338104,"statfs":{"total":96636764160,"available":96602218496,"internally_reserved":0,"allocated":2605056,"data_stored":1478674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T03:23:58.596 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-12T03:23:58.596 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-12T03:23:58.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.741+0000 7fe5629c4640 1 Processor -- start 2024-09-12T03:23:58.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.741+0000 7fe5629c4640 1 -- start start 2024-09-12T03:23:58.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.742+0000 7fe5629c4640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe55c15e650 0x7fe55c0d1d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:58.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.742+0000 7fe5629c4640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe55c06e690 con 0x7fe55c06db30 2024-09-12T03:23:58.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.742+0000 7fe5629c4640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe55c0d2280 con 0x7fe55c06cc40 2024-09-12T03:23:58.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.742+0000 7fe5629c4640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fe55c06d850 con 0x7fe55c15e650 2024-09-12T03:23:58.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.742+0000 7fe560f61640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fe55c06db30 0x7fe55c06df80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:33980/0 (socket says 172.21.15.27:33980) 2024-09-12T03:23:58.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.742+0000 7fe560f61640 1 -- 172.21.15.27:0/712729071 learned_addr learned my addr 172.21.15.27:0/712729071 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:23:58.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.742+0000 7fe55bfff640 1 --2- 172.21.15.27:0/712729071 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe55c15e650 0x7fe55c0d1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:58.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.742+0000 7fe55affd640 1 -- 172.21.15.27:0/712729071 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1013120251 0 0) 0x7fe55c06e690 con 0x7fe55c06db30 2024-09-12T03:23:58.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.743+0000 7fe55affd640 1 -- 172.21.15.27:0/712729071 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe544003520 con 0x7fe55c06db30 2024-09-12T03:23:58.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.743+0000 7fe55affd640 1 -- 172.21.15.27:0/712729071 <== mon.1 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1065702417 0 0) 0x7fe55c0d2280 con 0x7fe55c06cc40 2024-09-12T03:23:58.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.743+0000 7fe55affd640 1 -- 172.21.15.27:0/712729071 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe55c06e690 con 0x7fe55c06cc40 2024-09-12T03:23:58.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.743+0000 7fe55affd640 1 -- 172.21.15.27:0/712729071 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 871876033 0 0) 0x7fe544003520 con 0x7fe55c06db30 2024-09-12T03:23:58.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.743+0000 7fe55bfff640 1 -- 172.21.15.27:0/712729071 >> v1:172.21.15.45:6789/0 conn(0x7fe55c06cc40 legacy=0x7fe55c06d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:58.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.743+0000 7fe55bfff640 1 -- 172.21.15.27:0/712729071 >> v1:172.21.15.27:6789/0 conn(0x7fe55c06db30 legacy=0x7fe55c06df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:58.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.743+0000 7fe55bfff640 1 -- 172.21.15.27:0/712729071 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe55c0d1260 con 0x7fe55c15e650 2024-09-12T03:23:58.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.743+0000 7fe55bfff640 1 --2- 172.21.15.27:0/712729071 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe55c15e650 0x7fe55c0d1d40 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fe54c0097d0 tx=0x7fe54c02a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5bbaa516866fa0b1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:58.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.744+0000 7fe55affd640 1 -- 172.21.15.27:0/712729071 <== mon.2 v2:172.21.15.45:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe54c037070 con 0x7fe55c15e650 2024-09-12T03:23:58.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.744+0000 7fe55affd640 1 -- 172.21.15.27:0/712729071 <== mon.2 v2:172.21.15.45:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe54c02ab90 con 0x7fe55c15e650 2024-09-12T03:23:58.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.744+0000 7fe55affd640 1 -- 172.21.15.27:0/712729071 <== mon.2 v2:172.21.15.45:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe554001a10 con 0x7fe55c15e650 2024-09-12T03:23:58.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.744+0000 7fe5629c4640 1 -- 172.21.15.27:0/712729071 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe55c15e650 msgr2=0x7fe55c0d1d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:58.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.744+0000 7fe5629c4640 1 --2- 172.21.15.27:0/712729071 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe55c15e650 0x7fe55c0d1d40 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fe54c0097d0 tx=0x7fe54c02a100 comp rx=0 tx=0).stop 2024-09-12T03:23:58.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.744+0000 7fe5629c4640 1 -- 172.21.15.27:0/712729071 shutdown_connections 2024-09-12T03:23:58.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.744+0000 7fe5629c4640 1 --2- 172.21.15.27:0/712729071 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe55c15e650 0x7fe55c0d1d40 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:58.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.744+0000 7fe5629c4640 1 -- 172.21.15.27:0/712729071 >> 172.21.15.27:0/712729071 conn(0x7fe55c06f320 msgr2=0x7fe55c06f700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:58.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.744+0000 7fe5629c4640 1 -- 172.21.15.27:0/712729071 shutdown_connections 2024-09-12T03:23:58.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.744+0000 7fe5629c4640 1 -- 172.21.15.27:0/712729071 wait complete. 2024-09-12T03:23:58.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.746+0000 7fe5629c4640 1 Processor -- start 2024-09-12T03:23:58.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.746+0000 7fe5629c4640 1 -- start start 2024-09-12T03:23:58.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.746+0000 7fe5629c4640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe55c06cc40 0x7fe55c156670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:58.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.746+0000 7fe5629c4640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe55c06db30 0x7fe55c155200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:58.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.746+0000 7fe5629c4640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe55c15e650 0x7fe55c155740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:58.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.747+0000 7fe5629c4640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fe55c15ebb0 con 0x7fe55c06db30 2024-09-12T03:23:58.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.747+0000 7fe5629c4640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe55c15ea30 con 0x7fe55c06cc40 2024-09-12T03:23:58.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.747+0000 7fe5629c4640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fe55c0b55b0 con 0x7fe55c15e650 2024-09-12T03:23:58.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.747+0000 7fe55b7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe55c06db30 0x7fe55c155200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:58.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.747+0000 7fe55b7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe55c06db30 0x7fe55c155200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:53638/0 (socket says 172.21.15.27:53638) 2024-09-12T03:23:58.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.747+0000 7fe55b7fe640 1 -- 172.21.15.27:0/3907370273 learned_addr learned my addr 172.21.15.27:0/3907370273 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:23:58.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.747+0000 7fe55bfff640 1 --2- 172.21.15.27:0/3907370273 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe55c06cc40 0x7fe55c156670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:58.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.747+0000 7fe560f61640 1 --2- 172.21.15.27:0/3907370273 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe55c15e650 0x7fe55c155740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:58.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.747+0000 7fe55b7fe640 1 -- 172.21.15.27:0/3907370273 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe55c15e650 msgr2=0x7fe55c155740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:58.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.747+0000 7fe55b7fe640 1 --2- 172.21.15.27:0/3907370273 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe55c15e650 0x7fe55c155740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:58.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.747+0000 7fe55b7fe640 1 -- 172.21.15.27:0/3907370273 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe55c06cc40 msgr2=0x7fe55c156670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:58.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.747+0000 7fe55b7fe640 1 --2- 172.21.15.27:0/3907370273 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe55c06cc40 0x7fe55c156670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:58.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.747+0000 7fe55b7fe640 1 -- 172.21.15.27:0/3907370273 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe55c156fa0 con 0x7fe55c06db30 2024-09-12T03:23:58.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.747+0000 7fe560f61640 1 --2- 172.21.15.27:0/3907370273 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe55c15e650 0x7fe55c155740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:23:58.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.747+0000 7fe55b7fe640 1 --2- 172.21.15.27:0/3907370273 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe55c06db30 0x7fe55c155200 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7fe55000c490 tx=0x7fe55000c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:58.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.747+0000 7fe558ff9640 1 -- 172.21.15.27:0/3907370273 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5500052d0 con 0x7fe55c06db30 2024-09-12T03:23:58.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.747+0000 7fe558ff9640 1 -- 172.21.15.27:0/3907370273 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe550005470 con 0x7fe55c06db30 2024-09-12T03:23:58.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.747+0000 7fe558ff9640 1 -- 172.21.15.27:0/3907370273 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe550012870 con 0x7fe55c06db30 2024-09-12T03:23:58.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.747+0000 7fe5629c4640 1 -- 172.21.15.27:0/3907370273 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe55c157280 con 0x7fe55c06db30 2024-09-12T03:23:58.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.747+0000 7fe5629c4640 1 -- 172.21.15.27:0/3907370273 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe55c157840 con 0x7fe55c06db30 2024-09-12T03:23:58.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.749+0000 7fe5629c4640 1 -- 172.21.15.27:0/3907370273 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe55c06d060 con 0x7fe55c06db30 2024-09-12T03:23:58.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.749+0000 7fe558ff9640 1 -- 172.21.15.27:0/3907370273 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 25) ==== 91365+0+0 (secure 0 0 0) 0x7fe550012a10 con 0x7fe55c06db30 2024-09-12T03:23:58.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.750+0000 7fe558ff9640 1 --2- 172.21.15.27:0/3907370273 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fe54806c7c0 0x7fe54806ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:58.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.750+0000 7fe55bfff640 1 --2- 172.21.15.27:0/3907370273 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fe54806c7c0 0x7fe54806ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:58.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.750+0000 7fe558ff9640 1 -- 172.21.15.27:0/3907370273 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(160..160 src has 1..160) ==== 5765+0+0 (secure 0 0 0) 0x7fe5500891b0 con 0x7fe55c06db30 2024-09-12T03:23:58.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.750+0000 7fe55bfff640 1 --2- 172.21.15.27:0/3907370273 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fe54806c7c0 0x7fe54806ec80 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fe54c034000 tx=0x7fe54c02e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:58.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.754+0000 7fe558ff9640 1 -- 172.21.15.27:0/3907370273 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fe550059760 con 0x7fe55c06db30 2024-09-12T03:23:58.996 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.995+0000 7fe5629c4640 1 -- 172.21.15.27:0/3907370273 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7fe55c0706d0 con 0x7fe55c06db30 2024-09-12T03:23:58.998 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:58.997+0000 7fe558ff9640 1 -- 172.21.15.27:0/3907370273 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v23) ==== 190+0+147 (secure 0 0 0) 0x7fe550059060 con 0x7fe55c06db30 2024-09-12T03:23:58.998 INFO:teuthology.orchestra.run.smithi027.stdout:[client.0] 2024-09-12T03:23:58.999 INFO:teuthology.orchestra.run.smithi027.stdout: key = AQDVXOJmLm+5ExAAECM4CKfuVElmp/n0b02mXg== 2024-09-12T03:23:58.999 INFO:teuthology.orchestra.run.smithi027.stdout: caps mds = "allow" 2024-09-12T03:23:58.999 INFO:teuthology.orchestra.run.smithi027.stdout: caps mon = "allow r" 2024-09-12T03:23:58.999 INFO:teuthology.orchestra.run.smithi027.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-12T03:23:58.999 INFO:teuthology.orchestra.run.smithi027.stderr:updated caps for client.0 2024-09-12T03:23:59.001 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.000+0000 7fe5629c4640 1 -- 172.21.15.27:0/3907370273 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fe54806c7c0 msgr2=0x7fe54806ec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:59.001 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.000+0000 7fe5629c4640 1 --2- 172.21.15.27:0/3907370273 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fe54806c7c0 0x7fe54806ec80 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fe54c034000 tx=0x7fe54c02e000 comp rx=0 tx=0).stop 2024-09-12T03:23:59.001 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.000+0000 7fe5629c4640 1 -- 172.21.15.27:0/3907370273 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe55c06db30 msgr2=0x7fe55c155200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:59.001 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.000+0000 7fe5629c4640 1 --2- 172.21.15.27:0/3907370273 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe55c06db30 0x7fe55c155200 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7fe55000c490 tx=0x7fe55000c950 comp rx=0 tx=0).stop 2024-09-12T03:23:59.001 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.000+0000 7fe5629c4640 1 -- 172.21.15.27:0/3907370273 shutdown_connections 2024-09-12T03:23:59.002 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.000+0000 7fe5629c4640 1 --2- 172.21.15.27:0/3907370273 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe55c15e650 0x7fe55c155740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:59.002 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.000+0000 7fe5629c4640 1 --2- 172.21.15.27:0/3907370273 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fe54806c7c0 0x7fe54806ec80 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:59.002 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.000+0000 7fe5629c4640 1 --2- 172.21.15.27:0/3907370273 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe55c06db30 0x7fe55c155200 unknown :-1 s=CLOSED pgs=1692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:59.002 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.000+0000 7fe5629c4640 1 --2- 172.21.15.27:0/3907370273 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe55c06cc40 0x7fe55c156670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:59.002 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.000+0000 7fe5629c4640 1 -- 172.21.15.27:0/3907370273 >> 172.21.15.27:0/3907370273 conn(0x7fe55c06f320 msgr2=0x7fe55c06f700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:59.002 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.000+0000 7fe5629c4640 1 -- 172.21.15.27:0/3907370273 shutdown_connections 2024-09-12T03:23:59.002 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.000+0000 7fe5629c4640 1 -- 172.21.15.27:0/3907370273 wait complete. 2024-09-12T03:23:59.012 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-12T03:23:59.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.113+0000 7fd2b0b98640 1 Processor -- start 2024-09-12T03:23:59.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.113+0000 7fd2b0b98640 1 -- start start 2024-09-12T03:23:59.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.114+0000 7fd2b0b98640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fd2ac06f360 0x7fd2ac0cef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:59.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.114+0000 7fd2b0b98640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fd2ac0cf490 0x7fd2ac0d2780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:59.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.114+0000 7fd2b0b98640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd2ac06cc40 con 0x7fd2ac0b1ca0 2024-09-12T03:23:59.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.114+0000 7fd2b0b98640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fd2ac0d22f0 con 0x7fd2ac06f360 2024-09-12T03:23:59.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.114+0000 7fd2b0b98640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fd2ac0d1ce0 con 0x7fd2ac0cf490 2024-09-12T03:23:59.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.114+0000 7fd2aa59c640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fd2ac0b1ca0 0x7fd2ac15f830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:33986/0 (socket says 172.21.15.27:33986) 2024-09-12T03:23:59.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.114+0000 7fd2aa59c640 1 -- 172.21.15.27:0/2067842735 learned_addr learned my addr 172.21.15.27:0/2067842735 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:23:59.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.114+0000 7fd2a959a640 1 -- 172.21.15.27:0/2067842735 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 909863978 0 0) 0x7fd2ac06cc40 con 0x7fd2ac0b1ca0 2024-09-12T03:23:59.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.114+0000 7fd2a959a640 1 -- 172.21.15.27:0/2067842735 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd294003520 con 0x7fd2ac0b1ca0 2024-09-12T03:23:59.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.114+0000 7fd2aad9d640 1 --2- 172.21.15.27:0/2067842735 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fd2ac0cf490 0x7fd2ac0d2780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:59.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.114+0000 7fd2a9d9b640 1 --2- 172.21.15.27:0/2067842735 >> v2:172.21.15.45:3300/0 conn(0x7fd2ac06f360 0x7fd2ac0cef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:59.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.115+0000 7fd2a959a640 1 -- 172.21.15.27:0/2067842735 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3923997391 0 0) 0x7fd294003520 con 0x7fd2ac0b1ca0 2024-09-12T03:23:59.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.115+0000 7fd2a959a640 1 -- 172.21.15.27:0/2067842735 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fd2ac0cf490 msgr2=0x7fd2ac0d2780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:59.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.115+0000 7fd2a959a640 1 --2- 172.21.15.27:0/2067842735 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fd2ac0cf490 0x7fd2ac0d2780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:59.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.115+0000 7fd2a959a640 1 -- 172.21.15.27:0/2067842735 >> v2:172.21.15.45:3300/0 conn(0x7fd2ac06f360 msgr2=0x7fd2ac0cef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:59.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.115+0000 7fd2a959a640 1 --2- 172.21.15.27:0/2067842735 >> v2:172.21.15.45:3300/0 conn(0x7fd2ac06f360 0x7fd2ac0cef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:59.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.115+0000 7fd2a959a640 1 -- 172.21.15.27:0/2067842735 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2ac06cee0 con 0x7fd2ac0b1ca0 2024-09-12T03:23:59.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.115+0000 7fd2a959a640 1 -- 172.21.15.27:0/2067842735 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 847755213 0 0) 0x7fd29c001670 con 0x7fd2ac0b1ca0 2024-09-12T03:23:59.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.115+0000 7fd2a959a640 1 -- 172.21.15.27:0/2067842735 >> v1:172.21.15.27:6789/0 conn(0x7fd2ac0b1ca0 legacy=0x7fd2ac15f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:59.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.115+0000 7fd2a959a640 1 --2- 172.21.15.27:0/2067842735 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd2940039b0 0x7fd294003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:59.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.115+0000 7fd2a959a640 1 --2- 172.21.15.27:0/2067842735 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd294004800 0x7fd294004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:59.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.115+0000 7fd2a959a640 1 --2- 172.21.15.27:0/2067842735 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fd2ac0cf490 0x7fd2940051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:59.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.115+0000 7fd2a959a640 1 -- 172.21.15.27:0/2067842735 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fd2ac0d22f0 con 0x7fd2940039b0 2024-09-12T03:23:59.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.115+0000 7fd2a959a640 1 -- 172.21.15.27:0/2067842735 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fd2ac0d1ce0 con 0x7fd294004800 2024-09-12T03:23:59.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.115+0000 7fd2a959a640 1 -- 172.21.15.27:0/2067842735 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fd29400b9c0 con 0x7fd2ac0cf490 2024-09-12T03:23:59.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.115+0000 7fd2aa59c640 1 --2- 172.21.15.27:0/2067842735 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd2940039b0 0x7fd294003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:59.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.115+0000 7fd2aad9d640 1 --2- 172.21.15.27:0/2067842735 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fd2ac0cf490 0x7fd2940051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:59.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.115+0000 7fd2a9d9b640 1 --2- 172.21.15.27:0/2067842735 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd294004800 0x7fd294004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:59.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.115+0000 7fd2aa59c640 1 -- 172.21.15.27:0/2067842735 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fd2ac0cf490 msgr2=0x7fd2940051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:59.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.115+0000 7fd2aa59c640 1 --2- 172.21.15.27:0/2067842735 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fd2ac0cf490 0x7fd2940051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:59.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.115+0000 7fd2aa59c640 1 -- 172.21.15.27:0/2067842735 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd294004800 msgr2=0x7fd294004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:59.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.115+0000 7fd2aa59c640 1 --2- 172.21.15.27:0/2067842735 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd294004800 0x7fd294004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:59.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.115+0000 7fd2aa59c640 1 -- 172.21.15.27:0/2067842735 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd294003520 con 0x7fd2940039b0 2024-09-12T03:23:59.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.115+0000 7fd2a9d9b640 1 --2- 172.21.15.27:0/2067842735 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd294004800 0x7fd294004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:23:59.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.115+0000 7fd2aad9d640 1 --2- 172.21.15.27:0/2067842735 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fd2ac0cf490 0x7fd2940051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:23:59.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.116+0000 7fd2aa59c640 1 --2- 172.21.15.27:0/2067842735 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd2940039b0 0x7fd294003dd0 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7fd29c00c290 tx=0x7fd294005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d4994365e1e8403 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:59.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.116+0000 7fd2a959a640 1 -- 172.21.15.27:0/2067842735 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd29c007d50 con 0x7fd2940039b0 2024-09-12T03:23:59.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.116+0000 7fd2a959a640 1 -- 172.21.15.27:0/2067842735 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd29c005af0 con 0x7fd2940039b0 2024-09-12T03:23:59.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.116+0000 7fd2a959a640 1 -- 172.21.15.27:0/2067842735 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd29c007070 con 0x7fd2940039b0 2024-09-12T03:23:59.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.117+0000 7fd2b0b98640 1 -- 172.21.15.27:0/2067842735 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd2940039b0 msgr2=0x7fd294003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:59.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.117+0000 7fd2b0b98640 1 --2- 172.21.15.27:0/2067842735 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd2940039b0 0x7fd294003dd0 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7fd29c00c290 tx=0x7fd294005830 comp rx=0 tx=0).stop 2024-09-12T03:23:59.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.117+0000 7fd2aad9d640 1 -- 172.21.15.27:0/2067842735 reap_dead start 2024-09-12T03:23:59.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.117+0000 7fd2b0b98640 1 -- 172.21.15.27:0/2067842735 shutdown_connections 2024-09-12T03:23:59.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.117+0000 7fd2b0b98640 1 -- 172.21.15.27:0/2067842735 >> 172.21.15.27:0/2067842735 conn(0x7fd2ac073800 msgr2=0x7fd2ac09ba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:59.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.117+0000 7fd2b0b98640 1 -- 172.21.15.27:0/2067842735 shutdown_connections 2024-09-12T03:23:59.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.117+0000 7fd2b0b98640 1 -- 172.21.15.27:0/2067842735 wait complete. 2024-09-12T03:23:59.124 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.118+0000 7fd2b0b98640 1 Processor -- start 2024-09-12T03:23:59.124 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.118+0000 7fd2b0b98640 1 -- start start 2024-09-12T03:23:59.124 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.119+0000 7fd2b0b98640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd2940039b0 0x7fd2ac161290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:59.124 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.119+0000 7fd2b0b98640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fd2ac160970 0x7fd2ac15c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:59.124 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.119+0000 7fd2b0b98640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd2ac15d410 0x7fd2ac15d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:59.124 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.119+0000 7fd2b0b98640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fd2ac06d340 con 0x7fd2ac15d410 2024-09-12T03:23:59.124 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.119+0000 7fd2b0b98640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fd2ac06d1c0 con 0x7fd2940039b0 2024-09-12T03:23:59.124 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.119+0000 7fd2b0b98640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fd2ac0cfc40 con 0x7fd2ac160970 2024-09-12T03:23:59.124 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.119+0000 7fd2aad9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd2ac15d410 0x7fd2ac15d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:59.125 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.119+0000 7fd2aad9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd2ac15d410 0x7fd2ac15d890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:53654/0 (socket says 172.21.15.27:53654) 2024-09-12T03:23:59.125 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.119+0000 7fd2aad9d640 1 -- 172.21.15.27:0/3782830555 learned_addr learned my addr 172.21.15.27:0/3782830555 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:23:59.125 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.119+0000 7fd2a9d9b640 1 --2- 172.21.15.27:0/3782830555 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fd2ac160970 0x7fd2ac15c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:59.125 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.119+0000 7fd2aa59c640 1 --2- 172.21.15.27:0/3782830555 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd2940039b0 0x7fd2ac161290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:59.125 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.119+0000 7fd2aad9d640 1 -- 172.21.15.27:0/3782830555 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fd2ac160970 msgr2=0x7fd2ac15c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:59.125 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.119+0000 7fd2aad9d640 1 --2- 172.21.15.27:0/3782830555 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fd2ac160970 0x7fd2ac15c1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:59.125 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.119+0000 7fd2aad9d640 1 -- 172.21.15.27:0/3782830555 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd2940039b0 msgr2=0x7fd2ac161290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:59.125 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.119+0000 7fd2aad9d640 1 --2- 172.21.15.27:0/3782830555 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd2940039b0 0x7fd2ac161290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:59.125 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.119+0000 7fd2aad9d640 1 -- 172.21.15.27:0/3782830555 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2ac1f7840 con 0x7fd2ac15d410 2024-09-12T03:23:59.125 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.120+0000 7fd2aad9d640 1 --2- 172.21.15.27:0/3782830555 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd2ac15d410 0x7fd2ac15d890 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7fd2ac06e550 tx=0x7fd2a4009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:59.125 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.120+0000 7fd29affd640 1 -- 172.21.15.27:0/3782830555 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2a401a030 con 0x7fd2ac15d410 2024-09-12T03:23:59.125 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.120+0000 7fd29affd640 1 -- 172.21.15.27:0/3782830555 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2a4010e20 con 0x7fd2ac15d410 2024-09-12T03:23:59.126 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.120+0000 7fd2b0b98640 1 -- 172.21.15.27:0/3782830555 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2ac1f7b20 con 0x7fd2ac15d410 2024-09-12T03:23:59.126 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.120+0000 7fd29affd640 1 -- 172.21.15.27:0/3782830555 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2a4003bd0 con 0x7fd2ac15d410 2024-09-12T03:23:59.126 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.120+0000 7fd2b0b98640 1 -- 172.21.15.27:0/3782830555 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2ac1f80e0 con 0x7fd2ac15d410 2024-09-12T03:23:59.126 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.121+0000 7fd2b0b98640 1 -- 172.21.15.27:0/3782830555 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2ac06f7d0 con 0x7fd2ac15d410 2024-09-12T03:23:59.126 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.121+0000 7fd29affd640 1 -- 172.21.15.27:0/3782830555 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 25) ==== 91365+0+0 (secure 0 0 0) 0x7fd2a401d070 con 0x7fd2ac15d410 2024-09-12T03:23:59.126 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.122+0000 7fd29affd640 1 --2- 172.21.15.27:0/3782830555 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fd28c06c7c0 0x7fd28c06ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:59.126 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.122+0000 7fd2aa59c640 1 --2- 172.21.15.27:0/3782830555 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fd28c06c7c0 0x7fd28c06ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:59.126 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.122+0000 7fd29affd640 1 -- 172.21.15.27:0/3782830555 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(160..160 src has 1..160) ==== 5765+0+0 (secure 0 0 0) 0x7fd2a4089c80 con 0x7fd2ac15d410 2024-09-12T03:23:59.126 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.123+0000 7fd2aa59c640 1 --2- 172.21.15.27:0/3782830555 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fd28c06c7c0 0x7fd28c06ec80 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fd29c033000 tx=0x7fd29c009030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:59.127 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.126+0000 7fd29affd640 1 -- 172.21.15.27:0/3782830555 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fd2a405a230 con 0x7fd2ac15d410 2024-09-12T03:23:59.338 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.337+0000 7fd2b0b98640 1 -- 172.21.15.27:0/3782830555 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd2ac0cdca0 con 0x7fd2ac15d410 2024-09-12T03:23:59.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.338+0000 7fd29affd640 1 -- 172.21.15.27:0/3782830555 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 99 v99) ==== 94+0+5227 (secure 0 0 0) 0x7fd2a4059b30 con 0x7fd2ac15d410 2024-09-12T03:23:59.339 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:23:59.339 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":99,"btime":"2024-09-12T03:23:57:270949+0000","default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7163,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/2493793567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2493793567},{"type":"v1","addr":"172.21.15.45:6835","nonce":2493793567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":96},{"gid":8056,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/1643775717","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":1643775717},{"type":"v1","addr":"172.21.15.45:6834","nonce":1643775717}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95},{"gid":8967,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6834/4129691413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":4129691413},{"type":"v1","addr":"172.21.15.27:6834","nonce":4129691413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95}],"filesystems":[{"mdsmap":{"epoch":99,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-12T03:23:54.606856+0000","modified":"2024-09-12T03:23:56.609988+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8970},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8970":{"gid":8970,"name":"c","rank":0,"incarnation":97,"state":"up:active","state_seq":3,"addr":"172.21.15.27:6835/2940099359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":2940099359},{"type":"v1","addr":"172.21.15.27:6835","nonce":2940099359}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[27],"metadata_pool":26,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8970,"qdb_cluster":[8970]},"id":13}]} 2024-09-12T03:23:59.339 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 99 2024-09-12T03:23:59.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.340+0000 7fd2b0b98640 1 -- 172.21.15.27:0/3782830555 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fd28c06c7c0 msgr2=0x7fd28c06ec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:59.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.340+0000 7fd2b0b98640 1 --2- 172.21.15.27:0/3782830555 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fd28c06c7c0 0x7fd28c06ec80 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fd29c033000 tx=0x7fd29c009030 comp rx=0 tx=0).stop 2024-09-12T03:23:59.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.340+0000 7fd2b0b98640 1 -- 172.21.15.27:0/3782830555 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd2ac15d410 msgr2=0x7fd2ac15d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:59.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.340+0000 7fd2b0b98640 1 --2- 172.21.15.27:0/3782830555 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd2ac15d410 0x7fd2ac15d890 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7fd2ac06e550 tx=0x7fd2a4009f90 comp rx=0 tx=0).stop 2024-09-12T03:23:59.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.340+0000 7fd2b0b98640 1 -- 172.21.15.27:0/3782830555 shutdown_connections 2024-09-12T03:23:59.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.340+0000 7fd2b0b98640 1 --2- 172.21.15.27:0/3782830555 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fd28c06c7c0 0x7fd28c06ec80 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:59.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.340+0000 7fd2b0b98640 1 --2- 172.21.15.27:0/3782830555 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd2ac15d410 0x7fd2ac15d890 unknown :-1 s=CLOSED pgs=1695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:59.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.340+0000 7fd2b0b98640 1 --2- 172.21.15.27:0/3782830555 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fd2ac160970 0x7fd2ac15c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:59.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.340+0000 7fd2b0b98640 1 --2- 172.21.15.27:0/3782830555 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd2940039b0 0x7fd2ac161290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:59.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.340+0000 7fd2b0b98640 1 -- 172.21.15.27:0/3782830555 >> 172.21.15.27:0/3782830555 conn(0x7fd2ac073800 msgr2=0x7fd2ac0b2fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:59.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.340+0000 7fd2b0b98640 1 -- 172.21.15.27:0/3782830555 shutdown_connections 2024-09-12T03:23:59.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.340+0000 7fd2b0b98640 1 -- 172.21.15.27:0/3782830555 wait complete. 2024-09-12T03:23:59.754 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 99, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-12T03:23:54.606856+0000', 'modified': '2024-09-12T03:23:56.609988+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8970}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8970': {'gid': 8970, 'name': 'c', 'rank': 0, 'incarnation': 97, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.27:6835/2940099359', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6833', 'nonce': 2940099359}, {'type': 'v1', 'addr': '172.21.15.27:6835', 'nonce': 2940099359}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [27], 'metadata_pool': 26, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 8970, 'qdb_cluster': [8970]} 2024-09-12T03:23:59.754 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-12T03:23:59.754 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-12T03:23:59.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.858+0000 7fded0ce6640 1 Processor -- start 2024-09-12T03:23:59.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.858+0000 7fded0ce6640 1 -- start start 2024-09-12T03:23:59.860 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.859+0000 7fded0ce6640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fdecc15b2b0 0x7fdecc0b3560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:59.860 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.859+0000 7fded0ce6640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fdecc06e9b0 0x7fdecc06ee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:59.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.859+0000 7fded0ce6640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdecc0ce410 con 0x7fdecc072d70 2024-09-12T03:23:59.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.859+0000 7fded0ce6640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7fdecc0720a0 con 0x7fdecc15b2b0 2024-09-12T03:23:59.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.859+0000 7fded0ce6640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fdecc0732c0 con 0x7fdecc06e9b0 2024-09-12T03:23:59.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.859+0000 7fdeca59c640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fdecc15b2b0 0x7fdecc0b3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:59.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.859+0000 7fdeca59c640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fdecc15b2b0 0x7fdecc0b3560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:32878/0 (socket says 172.21.15.27:32878) 2024-09-12T03:23:59.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.859+0000 7fdeca59c640 1 -- 172.21.15.27:0/2257541808 learned_addr learned my addr 172.21.15.27:0/2257541808 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:23:59.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.859+0000 7fdecad9d640 1 --2- 172.21.15.27:0/2257541808 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fdecc06e9b0 0x7fdecc06ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:59.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.859+0000 7fdeca59c640 1 -- 172.21.15.27:0/2257541808 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fdecc06e9b0 msgr2=0x7fdecc06ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:59.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.859+0000 7fdeca59c640 1 --2- 172.21.15.27:0/2257541808 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fdecc06e9b0 0x7fdecc06ee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:59.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.859+0000 7fdeca59c640 1 -- 172.21.15.27:0/2257541808 >> v1:172.21.15.45:6789/0 conn(0x7fdecc072d70 legacy=0x7fdecc072430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:59.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.859+0000 7fdeca59c640 1 -- 172.21.15.27:0/2257541808 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdecc071ae0 con 0x7fdecc15b2b0 2024-09-12T03:23:59.862 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.860+0000 7fdeca59c640 1 --2- 172.21.15.27:0/2257541808 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdecc15b2b0 0x7fdecc0b3560 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7fdebc009870 tx=0x7fdebc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f82dfa8d4484a61c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:59.862 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.860+0000 7fdec9d9b640 1 -- 172.21.15.27:0/2257541808 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdebc037070 con 0x7fdecc15b2b0 2024-09-12T03:23:59.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.860+0000 7fdec9d9b640 1 -- 172.21.15.27:0/2257541808 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdebc032440 con 0x7fdecc15b2b0 2024-09-12T03:23:59.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.860+0000 7fdec9d9b640 1 -- 172.21.15.27:0/2257541808 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdebc032810 con 0x7fdecc15b2b0 2024-09-12T03:23:59.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.861+0000 7fded0ce6640 1 -- 172.21.15.27:0/2257541808 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdecc15b2b0 msgr2=0x7fdecc0b3560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:59.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.861+0000 7fded0ce6640 1 --2- 172.21.15.27:0/2257541808 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdecc15b2b0 0x7fdecc0b3560 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7fdebc009870 tx=0x7fdebc02a270 comp rx=0 tx=0).stop 2024-09-12T03:23:59.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.861+0000 7fded0ce6640 1 -- 172.21.15.27:0/2257541808 shutdown_connections 2024-09-12T03:23:59.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.861+0000 7fded0ce6640 1 --2- 172.21.15.27:0/2257541808 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fdecc06e9b0 0x7fdecc06ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:59.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.861+0000 7fded0ce6640 1 --2- 172.21.15.27:0/2257541808 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdecc15b2b0 0x7fdecc0b3560 unknown :-1 s=CLOSED pgs=1696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:59.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.861+0000 7fded0ce6640 1 -- 172.21.15.27:0/2257541808 >> 172.21.15.27:0/2257541808 conn(0x7fdecc062c60 msgr2=0x7fdecc06bb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:59.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.861+0000 7fded0ce6640 1 -- 172.21.15.27:0/2257541808 shutdown_connections 2024-09-12T03:23:59.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.861+0000 7fded0ce6640 1 -- 172.21.15.27:0/2257541808 wait complete. 2024-09-12T03:23:59.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.862+0000 7fded0ce6640 1 Processor -- start 2024-09-12T03:23:59.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.862+0000 7fded0ce6640 1 -- start start 2024-09-12T03:23:59.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.863+0000 7fded0ce6640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdecc06e9b0 0x7fdecc151440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:59.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.863+0000 7fded0ce6640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fdecc072d70 0x7fdecc1550a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:59.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.863+0000 7fded0ce6640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdecc15b2b0 0x7fdecc14c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:59.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.863+0000 7fded0ce6640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fdecc15b690 con 0x7fdecc06e9b0 2024-09-12T03:23:59.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.863+0000 7fded0ce6640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fdecc0ce690 con 0x7fdecc15b2b0 2024-09-12T03:23:59.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.863+0000 7fded0ce6640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fdecc15b810 con 0x7fdecc072d70 2024-09-12T03:23:59.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.863+0000 7fdeca59c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdecc06e9b0 0x7fdecc151440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:59.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.863+0000 7fdeca59c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdecc06e9b0 0x7fdecc151440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:32894/0 (socket says 172.21.15.27:32894) 2024-09-12T03:23:59.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.863+0000 7fdeca59c640 1 -- 172.21.15.27:0/1340294653 learned_addr learned my addr 172.21.15.27:0/1340294653 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:23:59.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.863+0000 7fdecad9d640 1 --2- 172.21.15.27:0/1340294653 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdecc15b2b0 0x7fdecc14c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:59.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.863+0000 7fdec3fff640 1 --2- 172.21.15.27:0/1340294653 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fdecc072d70 0x7fdecc1550a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:59.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.863+0000 7fdeca59c640 1 -- 172.21.15.27:0/1340294653 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fdecc072d70 msgr2=0x7fdecc1550a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:59.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.863+0000 7fdeca59c640 1 --2- 172.21.15.27:0/1340294653 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fdecc072d70 0x7fdecc1550a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:59.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.863+0000 7fdeca59c640 1 -- 172.21.15.27:0/1340294653 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdecc15b2b0 msgr2=0x7fdecc14c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:59.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.863+0000 7fdeca59c640 1 --2- 172.21.15.27:0/1340294653 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdecc15b2b0 0x7fdecc14c7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:59.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.863+0000 7fdeca59c640 1 -- 172.21.15.27:0/1340294653 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdecc155780 con 0x7fdecc06e9b0 2024-09-12T03:23:59.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.863+0000 7fdec3fff640 1 --2- 172.21.15.27:0/1340294653 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fdecc072d70 0x7fdecc1550a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:23:59.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.863+0000 7fdecad9d640 1 --2- 172.21.15.27:0/1340294653 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdecc15b2b0 0x7fdecc14c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:23:59.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.864+0000 7fdeca59c640 1 --2- 172.21.15.27:0/1340294653 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdecc06e9b0 0x7fdecc151440 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7fdebc000c00 tx=0x7fdebc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:59.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.864+0000 7fdec2ffd640 1 -- 172.21.15.27:0/1340294653 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdebc041020 con 0x7fdecc06e9b0 2024-09-12T03:23:59.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.864+0000 7fdec2ffd640 1 -- 172.21.15.27:0/1340294653 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdebc0306c0 con 0x7fdecc06e9b0 2024-09-12T03:23:59.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.864+0000 7fded0ce6640 1 -- 172.21.15.27:0/1340294653 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdecc155a60 con 0x7fdecc06e9b0 2024-09-12T03:23:59.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.864+0000 7fdec2ffd640 1 -- 172.21.15.27:0/1340294653 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdebc037040 con 0x7fdecc06e9b0 2024-09-12T03:23:59.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.864+0000 7fded0ce6640 1 -- 172.21.15.27:0/1340294653 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdecc156020 con 0x7fdecc06e9b0 2024-09-12T03:23:59.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.865+0000 7fdec2ffd640 1 -- 172.21.15.27:0/1340294653 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 25) ==== 91365+0+0 (secure 0 0 0) 0x7fdebc044020 con 0x7fdecc06e9b0 2024-09-12T03:23:59.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.866+0000 7fdec2ffd640 1 --2- 172.21.15.27:0/1340294653 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fdeac06c7c0 0x7fdeac06ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:59.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.866+0000 7fdec2ffd640 1 -- 172.21.15.27:0/1340294653 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(160..160 src has 1..160) ==== 5765+0+0 (secure 0 0 0) 0x7fdebc0a9e00 con 0x7fdecc06e9b0 2024-09-12T03:23:59.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.866+0000 7fdec3fff640 1 --2- 172.21.15.27:0/1340294653 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fdeac06c7c0 0x7fdeac06ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:59.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.866+0000 7fdec3fff640 1 --2- 172.21.15.27:0/1340294653 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fdeac06c7c0 0x7fdeac06ec80 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fdecc1f56a0 tx=0x7fdeb4009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:59.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.867+0000 7fded3a77740 1 Processor -- start 2024-09-12T03:23:59.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.867+0000 7fded3a77740 1 -- start start 2024-09-12T03:23:59.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.868+0000 7fded3a77740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56216ec24120 0x56216ec24500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:59.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.868+0000 7fded3a77740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56216ec3eae0 0x56216ec40290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:59.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.868+0000 7fded3a77740 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x56216ebf7a80 0x56216ec407d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:59.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.868+0000 7fded3a77740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x56216eb5c520 con 0x56216ec3eae0 2024-09-12T03:23:59.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.868+0000 7fded3a77740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x56216eb043e0 con 0x56216ec24120 2024-09-12T03:23:59.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.868+0000 7fded3a77740 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x56216eb584e0 con 0x56216ebf7a80 2024-09-12T03:23:59.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.868+0000 7fdec3fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56216ec3eae0 0x56216ec40290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:59.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.868+0000 7fdec3fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56216ec3eae0 0x56216ec40290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:32906/0 (socket says 172.21.15.27:32906) 2024-09-12T03:23:59.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.868+0000 7fdec3fff640 1 -- 172.21.15.27:0/1892212121 learned_addr learned my addr 172.21.15.27:0/1892212121 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:23:59.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.868+0000 7fdecad9d640 1 --2- 172.21.15.27:0/1892212121 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x56216ebf7a80 0x56216ec407d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:59.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.868+0000 7fdeca59c640 1 --2- 172.21.15.27:0/1892212121 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56216ec24120 0x56216ec24500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:59.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.868+0000 7fdec3fff640 1 -- 172.21.15.27:0/1892212121 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x56216ebf7a80 msgr2=0x56216ec407d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:59.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.868+0000 7fdec3fff640 1 --2- 172.21.15.27:0/1892212121 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x56216ebf7a80 0x56216ec407d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:59.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.868+0000 7fdec3fff640 1 -- 172.21.15.27:0/1892212121 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56216ec24120 msgr2=0x56216ec24500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:59.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.868+0000 7fdec3fff640 1 --2- 172.21.15.27:0/1892212121 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56216ec24120 0x56216ec24500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:59.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.868+0000 7fdec3fff640 1 -- 172.21.15.27:0/1892212121 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56216eb1f2f0 con 0x56216ec3eae0 2024-09-12T03:23:59.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.868+0000 7fdec3fff640 1 --2- 172.21.15.27:0/1892212121 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56216ec3eae0 0x56216ec40290 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7fdeb4005120 tx=0x7fdeb4011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=999c1e284a91da51 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:59.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.868+0000 7fdeabfff640 1 -- 172.21.15.27:0/1892212121 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdeb40035f0 con 0x56216ec3eae0 2024-09-12T03:23:59.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.868+0000 7fdeabfff640 1 -- 172.21.15.27:0/1892212121 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdeb4003790 con 0x56216ec3eae0 2024-09-12T03:23:59.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.868+0000 7fdeabfff640 1 -- 172.21.15.27:0/1892212121 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdeb4003b80 con 0x56216ec3eae0 2024-09-12T03:23:59.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.869+0000 7fded3a77740 1 -- 172.21.15.27:0/1892212121 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56216ec3eae0 msgr2=0x56216ec40290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:59.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.869+0000 7fded3a77740 1 --2- 172.21.15.27:0/1892212121 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56216ec3eae0 0x56216ec40290 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7fdeb4005120 tx=0x7fdeb4011000 comp rx=0 tx=0).stop 2024-09-12T03:23:59.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.869+0000 7fded3a77740 1 -- 172.21.15.27:0/1892212121 shutdown_connections 2024-09-12T03:23:59.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.869+0000 7fded3a77740 1 --2- 172.21.15.27:0/1892212121 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x56216ebf7a80 0x56216ec407d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:59.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.869+0000 7fded3a77740 1 --2- 172.21.15.27:0/1892212121 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56216ec3eae0 0x56216ec40290 unknown :-1 s=CLOSED pgs=1698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:59.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.869+0000 7fded3a77740 1 --2- 172.21.15.27:0/1892212121 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56216ec24120 0x56216ec24500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:59.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.869+0000 7fded3a77740 1 -- 172.21.15.27:0/1892212121 >> 172.21.15.27:0/1892212121 conn(0x56216ebf6a80 msgr2=0x56216ec3ef30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:59.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.869+0000 7fded3a77740 1 -- 172.21.15.27:0/1892212121 shutdown_connections 2024-09-12T03:23:59.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.869+0000 7fded3a77740 1 -- 172.21.15.27:0/1892212121 wait complete. 2024-09-12T03:23:59.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.870+0000 7fded3a77740 1 -- start start 2024-09-12T03:23:59.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.870+0000 7fded3a77740 1 Processor -- start 2024-09-12T03:23:59.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.871+0000 7fded3a77740 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x56216ec24120 0x56216ec3ba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:59.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.871+0000 7fded3a77740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56216ec3eae0 0x56216ec3bf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:59.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.871+0000 7fded3a77740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56216ec20cf0 0x56216ec26920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:59.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.871+0000 7fded3a77740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x56216ec24ee0 con 0x56216ec20cf0 2024-09-12T03:23:59.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.871+0000 7fded3a77740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x56216eae5450 con 0x56216ec3eae0 2024-09-12T03:23:59.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.871+0000 7fded3a77740 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x56216ec3f770 con 0x56216ec24120 2024-09-12T03:23:59.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.871+0000 7fded3a77740 10 client.0 _ll_register_callbacks cb 0x56216e86eac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:23:59.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.871+0000 7fdecad9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56216ec20cf0 0x56216ec26920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:59.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.871+0000 7fdecad9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56216ec20cf0 0x56216ec26920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:32918/0 (socket says 172.21.15.27:32918) 2024-09-12T03:23:59.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.871+0000 7fdecad9d640 1 -- 172.21.15.27:0/1704026087 learned_addr learned my addr 172.21.15.27:0/1704026087 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:23:59.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.871+0000 7fdeca59c640 1 --2- 172.21.15.27:0/1704026087 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x56216ec24120 0x56216ec3ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:59.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.871+0000 7fdec3fff640 1 --2- 172.21.15.27:0/1704026087 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56216ec3eae0 0x56216ec3bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:59.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.871+0000 7fdecad9d640 1 -- 172.21.15.27:0/1704026087 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x56216ec24120 msgr2=0x56216ec3ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:59.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.871+0000 7fdecad9d640 1 --2- 172.21.15.27:0/1704026087 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x56216ec24120 0x56216ec3ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:59.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.871+0000 7fdecad9d640 1 -- 172.21.15.27:0/1704026087 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56216ec3eae0 msgr2=0x56216ec3bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:59.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.871+0000 7fdecad9d640 1 --2- 172.21.15.27:0/1704026087 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56216ec3eae0 0x56216ec3bf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:59.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.871+0000 7fdecad9d640 1 -- 172.21.15.27:0/1704026087 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56216e86f2d0 con 0x56216ec20cf0 2024-09-12T03:23:59.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.871+0000 7fdeca59c640 1 --2- 172.21.15.27:0/1704026087 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x56216ec24120 0x56216ec3ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:23:59.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.871+0000 7fdecad9d640 1 --2- 172.21.15.27:0/1704026087 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56216ec20cf0 0x56216ec26920 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 crypto rx=0x7fdec4007ab0 tx=0x7fdec4007f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d342cf18414ccfc6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:59.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.871+0000 7fdea97fa640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-12T03:23:59.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.872+0000 7fdea97fa640 1 -- 172.21.15.27:0/1704026087 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdec401b020 con 0x56216ec20cf0 2024-09-12T03:23:59.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.872+0000 7fdea97fa640 1 -- 172.21.15.27:0/1704026087 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(160..160 src has 1..160) ==== 5765+0+0 (secure 0 0 0) 0x7fdec4011ba0 con 0x56216ec20cf0 2024-09-12T03:23:59.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.872+0000 7fded3a77740 1 -- 172.21.15.27:0/1704026087 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdeb4003790 con 0x56216ec20cf0 2024-09-12T03:23:59.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.872+0000 7fdea97fa640 1 -- 172.21.15.27:0/1704026087 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=99) ==== 24+0+0 (secure 0 0 0) 0x7fdec4025050 con 0x56216ec20cf0 2024-09-12T03:23:59.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.872+0000 7fded3a77740 10 client.9105 fetch_fsmap learned FSMap version 99 2024-09-12T03:23:59.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.872+0000 7fded3a77740 1 -- 172.21.15.27:0/1704026087 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=99,monmap=2+}) -- 0x56216e9396b0 con 0x56216ec20cf0 2024-09-12T03:23:59.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.872+0000 7fdea97fa640 1 -- 172.21.15.27:0/1704026087 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdeb4003790 con 0x56216ec20cf0 2024-09-12T03:23:59.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.872+0000 7fdea97fa640 1 -- 172.21.15.27:0/1704026087 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 99) ==== 3502+0+0 (secure 0 0 0) 0x7fdec4028020 con 0x56216ec20cf0 2024-09-12T03:23:59.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.872+0000 7fded3a77740 10 client.9105 fetch_fsmap finished waiting for FSMap version 99 2024-09-12T03:23:59.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.872+0000 7fded3a77740 10 client.9105 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-12T03:23:59.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.872+0000 7fded3a77740 20 client.9105 populate_metadata read hostname 'smithi027' 2024-09-12T03:23:59.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.872+0000 7fded3a77740 1 --2- 172.21.15.27:0/1704026087 >> [v2:172.21.15.27:6833/2940099359,v1:172.21.15.27:6835/2940099359] conn(0x56216ec39da0 0x56216ec3a180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:59.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.872+0000 7fded3a77740 4 client.9105 mds_command: new command op to 8970 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:23:59.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.872+0000 7fded3a77740 1 -- 172.21.15.27:0/1704026087 --> [v2:172.21.15.27:6833/2940099359,v1:172.21.15.27:6835/2940099359] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56216eb69c50 con 0x56216ec39da0 2024-09-12T03:23:59.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.872+0000 7fdeca59c640 1 --2- 172.21.15.27:0/1704026087 >> [v2:172.21.15.27:6833/2940099359,v1:172.21.15.27:6835/2940099359] conn(0x56216ec39da0 0x56216ec3a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:59.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.872+0000 7fdea97fa640 10 client.9105 ms_handle_connect on v2:172.21.15.27:6833/2940099359 2024-09-12T03:23:59.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.872+0000 7fdeca59c640 1 --2- 172.21.15.27:0/1704026087 >> [v2:172.21.15.27:6833/2940099359,v1:172.21.15.27:6835/2940099359] conn(0x56216ec39da0 0x56216ec3a180 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f51649fac050a117 server_cookie=45bc6db9149eb09b in_seq=0 out_seq=0 2024-09-12T03:23:59.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.873+0000 7fdea97fa640 1 -- 172.21.15.27:0/1704026087 <== mds.0 v2:172.21.15.27:6833/2940099359 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56216eb69c50 con 0x56216ec39da0 2024-09-12T03:23:59.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.873+0000 7fdea97fa640 10 client.9105 handle_command_reply: tid=0 2024-09-12T03:23:59.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.873+0000 7fded3a77740 1 client.9105 shutdown 2024-09-12T03:23:59.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.873+0000 7fded3a77740 1 -- 172.21.15.27:0/1704026087 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56216ec20cf0 msgr2=0x56216ec26920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:59.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.873+0000 7fded3a77740 1 --2- 172.21.15.27:0/1704026087 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56216ec20cf0 0x56216ec26920 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 crypto rx=0x7fdec4007ab0 tx=0x7fdec4007f70 comp rx=0 tx=0).stop 2024-09-12T03:23:59.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.873+0000 7fded3a77740 1 -- 172.21.15.27:0/1704026087 shutdown_connections 2024-09-12T03:23:59.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.873+0000 7fded3a77740 1 --2- 172.21.15.27:0/1704026087 >> [v2:172.21.15.27:6833/2940099359,v1:172.21.15.27:6835/2940099359] conn(0x56216ec39da0 0x56216ec3a180 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:59.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.873+0000 7fded3a77740 1 --2- 172.21.15.27:0/1704026087 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56216ec20cf0 0x56216ec26920 unknown :-1 s=CLOSED pgs=1699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:59.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.873+0000 7fded3a77740 1 --2- 172.21.15.27:0/1704026087 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56216ec3eae0 0x56216ec3bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:59.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.873+0000 7fded3a77740 1 --2- 172.21.15.27:0/1704026087 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x56216ec24120 0x56216ec3ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:59.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.873+0000 7fdea97fa640 0 client.9105 ms_handle_reset on v2:172.21.15.27:6833/2940099359 2024-09-12T03:23:59.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.873+0000 7fded3a77740 1 -- 172.21.15.27:0/1704026087 >> 172.21.15.27:0/1704026087 conn(0x56216ebf7a80 msgr2=0x56216ec488d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:59.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.873+0000 7fded3a77740 1 -- 172.21.15.27:0/1704026087 shutdown_connections 2024-09-12T03:23:59.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.873+0000 7fded3a77740 1 -- 172.21.15.27:0/1704026087 wait complete. 2024-09-12T03:23:59.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.874+0000 7fded3a77740 20 client.9105 trim_cache size 0 max 16384 2024-09-12T03:23:59.892 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.892+0000 7fded3a77740 1 Processor -- start 2024-09-12T03:23:59.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.892+0000 7fded3a77740 1 -- start start 2024-09-12T03:23:59.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.892+0000 7fded3a77740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56216ec20cf0 0x56216ec48c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:59.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.892+0000 7fded3a77740 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x56216ec24120 0x56216ec27ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:59.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.892+0000 7fded3a77740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56216ec3eae0 0x56216ec28220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:59.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.892+0000 7fded3a77740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fdecc1684c0 con 0x56216ec3eae0 2024-09-12T03:23:59.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.892+0000 7fded3a77740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fdecc152700 con 0x56216ec20cf0 2024-09-12T03:23:59.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.892+0000 7fded3a77740 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fdecc152d50 con 0x56216ec24120 2024-09-12T03:23:59.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.893+0000 7fdecad9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56216ec3eae0 0x56216ec28220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:59.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.893+0000 7fdecad9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56216ec3eae0 0x56216ec28220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:32920/0 (socket says 172.21.15.27:32920) 2024-09-12T03:23:59.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.893+0000 7fdecad9d640 1 -- 172.21.15.27:0/2544162925 learned_addr learned my addr 172.21.15.27:0/2544162925 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:23:59.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.893+0000 7fdec3fff640 1 --2- 172.21.15.27:0/2544162925 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x56216ec24120 0x56216ec27ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:59.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.893+0000 7fdeca59c640 1 --2- 172.21.15.27:0/2544162925 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56216ec20cf0 0x56216ec48c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:59.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.893+0000 7fdecad9d640 1 -- 172.21.15.27:0/2544162925 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x56216ec24120 msgr2=0x56216ec27ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:59.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.893+0000 7fdecad9d640 1 --2- 172.21.15.27:0/2544162925 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x56216ec24120 0x56216ec27ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:59.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.893+0000 7fdecad9d640 1 -- 172.21.15.27:0/2544162925 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56216ec20cf0 msgr2=0x56216ec48c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:59.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.893+0000 7fdecad9d640 1 --2- 172.21.15.27:0/2544162925 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56216ec20cf0 0x56216ec48c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:59.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.893+0000 7fdecad9d640 1 -- 172.21.15.27:0/2544162925 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56216ec49980 con 0x56216ec3eae0 2024-09-12T03:23:59.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.893+0000 7fdecad9d640 1 --2- 172.21.15.27:0/2544162925 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56216ec3eae0 0x56216ec28220 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7fdec40076a0 tx=0x7fdec4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=735ff1eef28afaad server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:59.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.893+0000 7fdeab7fe640 1 -- 172.21.15.27:0/2544162925 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdec401b040 con 0x56216ec3eae0 2024-09-12T03:23:59.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.893+0000 7fdeab7fe640 1 -- 172.21.15.27:0/2544162925 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdec402e070 con 0x56216ec3eae0 2024-09-12T03:23:59.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.893+0000 7fdeab7fe640 1 -- 172.21.15.27:0/2544162925 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdec40168d0 con 0x56216ec3eae0 2024-09-12T03:23:59.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.894+0000 7fded3a77740 1 -- 172.21.15.27:0/2544162925 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56216ec3eae0 msgr2=0x56216ec28220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:59.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.894+0000 7fded3a77740 1 --2- 172.21.15.27:0/2544162925 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56216ec3eae0 0x56216ec28220 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7fdec40076a0 tx=0x7fdec4017000 comp rx=0 tx=0).stop 2024-09-12T03:23:59.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.894+0000 7fded3a77740 1 -- 172.21.15.27:0/2544162925 shutdown_connections 2024-09-12T03:23:59.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.894+0000 7fded3a77740 1 --2- 172.21.15.27:0/2544162925 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56216ec3eae0 0x56216ec28220 unknown :-1 s=CLOSED pgs=1700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:59.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.894+0000 7fded3a77740 1 --2- 172.21.15.27:0/2544162925 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x56216ec24120 0x56216ec27ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:59.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.894+0000 7fded3a77740 1 --2- 172.21.15.27:0/2544162925 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56216ec20cf0 0x56216ec48c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:59.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.894+0000 7fded3a77740 1 -- 172.21.15.27:0/2544162925 >> 172.21.15.27:0/2544162925 conn(0x56216ebf7a80 msgr2=0x56216ec3f6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:59.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.894+0000 7fded3a77740 1 -- 172.21.15.27:0/2544162925 shutdown_connections 2024-09-12T03:23:59.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.894+0000 7fded3a77740 1 -- 172.21.15.27:0/2544162925 wait complete. 2024-09-12T03:23:59.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.894+0000 7fded3a77740 1 -- start start 2024-09-12T03:23:59.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.894+0000 7fded3a77740 1 Processor -- start 2024-09-12T03:23:59.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.895+0000 7fded3a77740 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x56216ec20cf0 0x56216ec39670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:59.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.895+0000 7fded3a77740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56216ec24120 0x56216ecc1760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:59.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.895+0000 7fded3a77740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56216ec3eae0 0x56216ec27ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:59.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.895+0000 7fded3a77740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x56216ec3b220 con 0x56216ec24120 2024-09-12T03:23:59.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.895+0000 7fded3a77740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x56216ebf8c00 con 0x56216ec3eae0 2024-09-12T03:23:59.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.895+0000 7fded3a77740 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x56216ec26f80 con 0x56216ec20cf0 2024-09-12T03:23:59.902 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.895+0000 7fded3a77740 10 client.0 _ll_register_callbacks cb 0x7fdec4009830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:23:59.902 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.895+0000 7fdec3fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56216ec24120 0x56216ecc1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:59.902 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.895+0000 7fdec3fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56216ec24120 0x56216ecc1760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:32934/0 (socket says 172.21.15.27:32934) 2024-09-12T03:23:59.902 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.895+0000 7fdec3fff640 1 -- 172.21.15.27:0/2325313872 learned_addr learned my addr 172.21.15.27:0/2325313872 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:23:59.902 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.896+0000 7fdeca59c640 1 --2- 172.21.15.27:0/2325313872 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x56216ec20cf0 0x56216ec39670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:59.902 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.896+0000 7fdecad9d640 1 --2- 172.21.15.27:0/2325313872 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56216ec3eae0 0x56216ec27ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:59.902 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.896+0000 7fdec3fff640 1 -- 172.21.15.27:0/2325313872 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x56216ec20cf0 msgr2=0x56216ec39670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:59.902 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.896+0000 7fdec3fff640 1 --2- 172.21.15.27:0/2325313872 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x56216ec20cf0 0x56216ec39670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:59.902 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.896+0000 7fdec3fff640 1 -- 172.21.15.27:0/2325313872 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56216ec3eae0 msgr2=0x56216ec27ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:59.902 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.896+0000 7fdec3fff640 1 --2- 172.21.15.27:0/2325313872 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56216ec3eae0 0x56216ec27ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:59.902 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.896+0000 7fdec3fff640 1 -- 172.21.15.27:0/2325313872 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56216ec3e460 con 0x56216ec24120 2024-09-12T03:23:59.902 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.896+0000 7fdecad9d640 1 --2- 172.21.15.27:0/2325313872 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56216ec3eae0 0x56216ec27ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:23:59.902 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.896+0000 7fdec3fff640 1 --2- 172.21.15.27:0/2325313872 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56216ec24120 0x56216ecc1760 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7fdeb4007f70 tx=0x7fdeb4013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a536ad44bca08c5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:59.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.896+0000 7fdea97fa640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-12T03:23:59.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.896+0000 7fdea97fa640 1 -- 172.21.15.27:0/2325313872 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdeb4031060 con 0x56216ec24120 2024-09-12T03:23:59.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.896+0000 7fdea97fa640 1 -- 172.21.15.27:0/2325313872 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(160..160 src has 1..160) ==== 5765+0+0 (secure 0 0 0) 0x7fdeb402c460 con 0x56216ec24120 2024-09-12T03:23:59.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.896+0000 7fded3a77740 1 -- 172.21.15.27:0/2325313872 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdec402e070 con 0x56216ec24120 2024-09-12T03:23:59.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.896+0000 7fdea97fa640 1 -- 172.21.15.27:0/2325313872 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=99) ==== 24+0+0 (secure 0 0 0) 0x7fdeb4037050 con 0x56216ec24120 2024-09-12T03:23:59.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.896+0000 7fded3a77740 10 client.9111 fetch_fsmap learned FSMap version 99 2024-09-12T03:23:59.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.896+0000 7fded3a77740 1 -- 172.21.15.27:0/2325313872 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=99,monmap=2+}) -- 0x56216ec664e0 con 0x56216ec24120 2024-09-12T03:23:59.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.896+0000 7fdea97fa640 1 -- 172.21.15.27:0/2325313872 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdec402e070 con 0x56216ec24120 2024-09-12T03:23:59.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.896+0000 7fdea97fa640 1 -- 172.21.15.27:0/2325313872 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 99) ==== 3502+0+0 (secure 0 0 0) 0x7fdeb403a020 con 0x56216ec24120 2024-09-12T03:23:59.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.896+0000 7fded3a77740 10 client.9111 fetch_fsmap finished waiting for FSMap version 99 2024-09-12T03:23:59.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.897+0000 7fded3a77740 10 client.9111 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-12T03:23:59.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.897+0000 7fded3a77740 20 client.9111 populate_metadata read hostname 'smithi027' 2024-09-12T03:23:59.904 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.897+0000 7fded3a77740 1 --2- 172.21.15.27:0/2325313872 >> [v2:172.21.15.27:6833/2940099359,v1:172.21.15.27:6835/2940099359] conn(0x56216ebfa900 0x56216ec3cee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:59.904 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.897+0000 7fded3a77740 4 client.9111 mds_command: new command op to 8970 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-12T03:23:59.904 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.897+0000 7fded3a77740 1 -- 172.21.15.27:0/2325313872 --> [v2:172.21.15.27:6833/2940099359,v1:172.21.15.27:6835/2940099359] -- command(tid 0: {"prefix": "status"}) -- 0x56216ec26490 con 0x56216ebfa900 2024-09-12T03:23:59.904 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.897+0000 7fdeca59c640 1 --2- 172.21.15.27:0/2325313872 >> [v2:172.21.15.27:6833/2940099359,v1:172.21.15.27:6835/2940099359] conn(0x56216ebfa900 0x56216ec3cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:59.904 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.897+0000 7fdea97fa640 10 client.9111 ms_handle_connect on v2:172.21.15.27:6833/2940099359 2024-09-12T03:23:59.904 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.897+0000 7fdeca59c640 1 --2- 172.21.15.27:0/2325313872 >> [v2:172.21.15.27:6833/2940099359,v1:172.21.15.27:6835/2940099359] conn(0x56216ebfa900 0x56216ec3cee0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f41b3d81805cd876 server_cookie=3bd10b5e433f6579 in_seq=0 out_seq=0 2024-09-12T03:23:59.904 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.897+0000 7fdea97fa640 1 -- 172.21.15.27:0/2325313872 <== mds.0 v2:172.21.15.27:6833/2940099359 1 ==== command_reply(tid 0: 0 ) ==== 8+0+333 (crc 0 0 0) 0x56216ec26490 con 0x56216ebfa900 2024-09-12T03:23:59.904 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.897+0000 7fdea97fa640 10 client.9111 handle_command_reply: tid=0 2024-09-12T03:23:59.904 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.897+0000 7fded3a77740 1 client.9111 shutdown 2024-09-12T03:23:59.904 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.897+0000 7fded3a77740 1 -- 172.21.15.27:0/2325313872 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56216ec24120 msgr2=0x56216ecc1760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:59.904 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.897+0000 7fded3a77740 1 --2- 172.21.15.27:0/2325313872 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56216ec24120 0x56216ecc1760 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7fdeb4007f70 tx=0x7fdeb4013000 comp rx=0 tx=0).stop 2024-09-12T03:23:59.904 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.897+0000 7fded3a77740 1 -- 172.21.15.27:0/2325313872 shutdown_connections 2024-09-12T03:23:59.904 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.897+0000 7fded3a77740 1 --2- 172.21.15.27:0/2325313872 >> [v2:172.21.15.27:6833/2940099359,v1:172.21.15.27:6835/2940099359] conn(0x56216ebfa900 0x56216ec3cee0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:59.905 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.897+0000 7fded3a77740 1 --2- 172.21.15.27:0/2325313872 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56216ec3eae0 0x56216ec27ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:59.905 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.897+0000 7fded3a77740 1 --2- 172.21.15.27:0/2325313872 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56216ec24120 0x56216ecc1760 unknown :-1 s=CLOSED pgs=1701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:59.905 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.897+0000 7fded3a77740 1 --2- 172.21.15.27:0/2325313872 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x56216ec20cf0 0x56216ec39670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:59.905 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.897+0000 7fdea97fa640 0 client.9111 ms_handle_reset on v2:172.21.15.27:6833/2940099359 2024-09-12T03:23:59.905 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.897+0000 7fded3a77740 1 -- 172.21.15.27:0/2325313872 >> 172.21.15.27:0/2325313872 conn(0x56216ebf7a80 msgr2=0x56216ec3f6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:59.905 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.898+0000 7fded3a77740 1 -- 172.21.15.27:0/2325313872 shutdown_connections 2024-09-12T03:23:59.905 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.898+0000 7fded3a77740 1 -- 172.21.15.27:0/2325313872 wait complete. 2024-09-12T03:23:59.905 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.898+0000 7fded3a77740 20 client.9111 trim_cache size 0 max 16384 2024-09-12T03:23:59.905 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-12T03:23:59.905 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_fsid": "d2df504e-2647-4385-a5db-9fdf69f487a0", 2024-09-12T03:23:59.906 INFO:teuthology.orchestra.run.smithi027.stdout: "whoami": 0, 2024-09-12T03:23:59.906 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 8970, 2024-09-12T03:23:59.906 INFO:teuthology.orchestra.run.smithi027.stdout: "want_state": "up:active", 2024-09-12T03:23:59.906 INFO:teuthology.orchestra.run.smithi027.stdout: "state": "up:active", 2024-09-12T03:23:59.906 INFO:teuthology.orchestra.run.smithi027.stdout: "fs_name": "cephfs", 2024-09-12T03:23:59.906 INFO:teuthology.orchestra.run.smithi027.stdout: "rank_uptime": 4.6325413129999999, 2024-09-12T03:23:59.906 INFO:teuthology.orchestra.run.smithi027.stdout: "mdsmap_epoch": 99, 2024-09-12T03:23:59.906 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch": 160, 2024-09-12T03:23:59.906 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch_barrier": 160, 2024-09-12T03:23:59.906 INFO:teuthology.orchestra.run.smithi027.stdout: "uptime": 8.6717193829999992 2024-09-12T03:23:59.906 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-12T03:23:59.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.898+0000 7fded0ce6640 1 -- 172.21.15.27:0/1340294653 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fdeac06c7c0 msgr2=0x7fdeac06ec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:59.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.898+0000 7fded0ce6640 1 --2- 172.21.15.27:0/1340294653 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fdeac06c7c0 0x7fdeac06ec80 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fdecc1f56a0 tx=0x7fdeb4009040 comp rx=0 tx=0).stop 2024-09-12T03:23:59.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.898+0000 7fded0ce6640 1 -- 172.21.15.27:0/1340294653 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdecc06e9b0 msgr2=0x7fdecc151440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:59.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.898+0000 7fded0ce6640 1 --2- 172.21.15.27:0/1340294653 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdecc06e9b0 0x7fdecc151440 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7fdebc000c00 tx=0x7fdebc0046c0 comp rx=0 tx=0).stop 2024-09-12T03:23:59.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.898+0000 7fded0ce6640 1 -- 172.21.15.27:0/1340294653 shutdown_connections 2024-09-12T03:23:59.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.898+0000 7fded0ce6640 1 --2- 172.21.15.27:0/1340294653 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdecc15b2b0 0x7fdecc14c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:59.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.898+0000 7fded0ce6640 1 --2- 172.21.15.27:0/1340294653 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fdecc072d70 0x7fdecc1550a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:59.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.898+0000 7fded0ce6640 1 --2- 172.21.15.27:0/1340294653 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fdeac06c7c0 0x7fdeac06ec80 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:59.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.898+0000 7fded0ce6640 1 --2- 172.21.15.27:0/1340294653 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdecc06e9b0 0x7fdecc151440 unknown :-1 s=CLOSED pgs=1697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:59.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.898+0000 7fded0ce6640 1 -- 172.21.15.27:0/1340294653 >> 172.21.15.27:0/1340294653 conn(0x7fdecc062c60 msgr2=0x7fdecc06d8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:59.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.898+0000 7fded0ce6640 1 -- 172.21.15.27:0/1340294653 shutdown_connections 2024-09-12T03:23:59.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:23:59.898+0000 7fded0ce6640 1 -- 172.21.15.27:0/1340294653 wait complete. 2024-09-12T03:23:59.910 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-12T03:24:00.012 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:00.011+0000 7fda1187b640 1 Processor -- start 2024-09-12T03:24:00.013 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:00.011+0000 7fda1187b640 1 -- start start 2024-09-12T03:24:00.016 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:00.012+0000 7fda1187b640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fda0c0d5200 0x7fda0c0d1010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:00.016 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:00.012+0000 7fda1187b640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fda0c0d1550 0x7fda0c0b8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:00.016 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:00.012+0000 7fda1187b640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fda0c0d3760 con 0x7fda0c15eb90 2024-09-12T03:24:00.016 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:00.012+0000 7fda1187b640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7fda0c0b7d20 con 0x7fda0c0d5200 2024-09-12T03:24:00.016 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:00.012+0000 7fda1187b640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fda0c0d42e0 con 0x7fda0c0d1550 2024-09-12T03:24:00.016 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:00.012+0000 7fda0affd640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fda0c15eb90 0x7fda0c0d2df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:57492/0 (socket says 172.21.15.27:57492) 2024-09-12T03:24:00.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:00.012+0000 7fda0affd640 1 -- 172.21.15.27:0/3579245798 learned_addr learned my addr 172.21.15.27:0/3579245798 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:24:00.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:00.012+0000 7fda0a7fc640 1 --2- 172.21.15.27:0/3579245798 >> v2:172.21.15.27:3300/0 conn(0x7fda0c0d5200 0x7fda0c0d1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:00.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:00.013+0000 7fda0b7fe640 1 --2- 172.21.15.27:0/3579245798 >> v2:172.21.15.45:3300/0 conn(0x7fda0c0d1550 0x7fda0c0b8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:00.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:00.013+0000 7fda09ffb640 1 -- 172.21.15.27:0/3579245798 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3379382283 0 0) 0x7fda0c0d3760 con 0x7fda0c15eb90 2024-09-12T03:24:00.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:00.013+0000 7fda09ffb640 1 -- 172.21.15.27:0/3579245798 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd9f8003520 con 0x7fda0c15eb90 2024-09-12T03:24:00.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:00.013+0000 7fda0a7fc640 1 -- 172.21.15.27:0/3579245798 >> v2:172.21.15.45:3300/0 conn(0x7fda0c0d1550 msgr2=0x7fda0c0b8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:24:00.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:00.013+0000 7fda0a7fc640 1 --2- 172.21.15.27:0/3579245798 >> v2:172.21.15.45:3300/0 conn(0x7fda0c0d1550 0x7fda0c0b8190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:00.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:00.013+0000 7fda0a7fc640 1 -- 172.21.15.27:0/3579245798 >> v1:172.21.15.27:6789/0 conn(0x7fda0c15eb90 legacy=0x7fda0c0d2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:00.018 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:00.013+0000 7fda0a7fc640 1 -- 172.21.15.27:0/3579245798 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda0c0d3a40 con 0x7fda0c0d5200 2024-09-12T03:24:00.018 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:00.013+0000 7fda0a7fc640 1 --2- 172.21.15.27:0/3579245798 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fda0c0d5200 0x7fda0c0d1010 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7fda0c0c8270 tx=0x7fd9f402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd7761d3db959969 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:00.018 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:00.014+0000 7fda09ffb640 1 -- 172.21.15.27:0/3579245798 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9f4037070 con 0x7fda0c0d5200 2024-09-12T03:24:00.018 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:00.014+0000 7fda09ffb640 1 -- 172.21.15.27:0/3579245798 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9f4032440 con 0x7fda0c0d5200 2024-09-12T03:24:00.018 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:00.014+0000 7fda09ffb640 1 -- 172.21.15.27:0/3579245798 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9f4032810 con 0x7fda0c0d5200 2024-09-12T03:24:00.018 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:00.014+0000 7fda1187b640 1 -- 172.21.15.27:0/3579245798 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fda0c0d5200 msgr2=0x7fda0c0d1010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:00.018 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:00.014+0000 7fda1187b640 1 --2- 172.21.15.27:0/3579245798 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fda0c0d5200 0x7fda0c0d1010 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7fda0c0c8270 tx=0x7fd9f402a1a0 comp rx=0 tx=0).stop 2024-09-12T03:24:00.018 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:00.014+0000 7fda1187b640 1 -- 172.21.15.27:0/3579245798 shutdown_connections 2024-09-12T03:24:00.018 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:00.014+0000 7fda1187b640 1 --2- 172.21.15.27:0/3579245798 >> v2:172.21.15.45:3300/0 conn(0x7fda0c0d1550 0x7fda0c0b8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:00.019 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:00.014+0000 7fda1187b640 1 --2- 172.21.15.27:0/3579245798 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fda0c0d5200 0x7fda0c0d1010 unknown :-1 s=CLOSED pgs=1703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:00.019 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:00.014+0000 7fda1187b640 1 -- 172.21.15.27:0/3579245798 >> 172.21.15.27:0/3579245798 conn(0x7fda0c06dce0 msgr2=0x7fda0c06e0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:00.019 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:00.014+0000 7fda1187b640 1 -- 172.21.15.27:0/3579245798 shutdown_connections 2024-09-12T03:24:00.019 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:00.014+0000 7fda1187b640 1 -- 172.21.15.27:0/3579245798 wait complete. 2024-09-12T03:24:00.022 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:00.015+0000 7fda1187b640 1 Processor -- start 2024-09-12T03:24:00.023 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:00.016+0000 7fda1187b640 1 -- start start 2024-09-12T03:24:00.023 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:00.016+0000 7fda1187b640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fda0c0d1550 0x7fda0c0cd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:00.023 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:00.016+0000 7fda1187b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fda0c0d5200 0x7fda0c1573f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:00.023 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:00.016+0000 7fda1187b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fda0c15eb90 0x7fda0c0cb8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:00.023 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:00.016+0000 7fda1187b640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fda0c0d4000 con 0x7fda0c15eb90 2024-09-12T03:24:00.023 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:00.016+0000 7fda1187b640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fda0c0d3500 con 0x7fda0c0d5200 2024-09-12T03:24:00.023 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:00.016+0000 7fda1187b640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fda0c0d1960 con 0x7fda0c0d1550 2024-09-12T03:24:00.023 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:00.016+0000 7fda0b7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fda0c15eb90 0x7fda0c0cb8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:00.023 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:00.016+0000 7fda0b7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fda0c15eb90 0x7fda0c0cb8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:32958/0 (socket says 172.21.15.27:32958) 2024-09-12T03:24:00.023 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:00.016+0000 7fda0b7fe640 1 -- 172.21.15.27:0/2413281057 learned_addr learned my addr 172.21.15.27:0/2413281057 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:24:00.023 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:00.017+0000 7fda0affd640 1 --2- 172.21.15.27:0/2413281057 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fda0c0d1550 0x7fda0c0cd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:00.024 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:00.017+0000 7fda0a7fc640 1 --2- 172.21.15.27:0/2413281057 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fda0c0d5200 0x7fda0c1573f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:00.024 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:00.017+0000 7fda0b7fe640 1 -- 172.21.15.27:0/2413281057 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fda0c0d1550 msgr2=0x7fda0c0cd3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:00.024 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:00.017+0000 7fda0b7fe640 1 --2- 172.21.15.27:0/2413281057 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fda0c0d1550 0x7fda0c0cd3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:00.024 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:00.017+0000 7fda0b7fe640 1 -- 172.21.15.27:0/2413281057 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fda0c0d5200 msgr2=0x7fda0c1573f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:00.024 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:00.017+0000 7fda0b7fe640 1 --2- 172.21.15.27:0/2413281057 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fda0c0d5200 0x7fda0c1573f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:00.024 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:00.017+0000 7fda0b7fe640 1 -- 172.21.15.27:0/2413281057 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda0c1f4510 con 0x7fda0c15eb90 2024-09-12T03:24:00.024 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:00.017+0000 7fda0b7fe640 1 --2- 172.21.15.27:0/2413281057 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fda0c15eb90 0x7fda0c0cb8e0 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7fd9fc007d20 tx=0x7fd9fc00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:00.024 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:00.017+0000 7fda03fff640 1 -- 172.21.15.27:0/2413281057 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9fc014420 con 0x7fda0c15eb90 2024-09-12T03:24:00.024 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:00.017+0000 7fda03fff640 1 -- 172.21.15.27:0/2413281057 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9fc019070 con 0x7fda0c15eb90 2024-09-12T03:24:00.024 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:00.017+0000 7fda1187b640 1 -- 172.21.15.27:0/2413281057 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda0c1f3ff0 con 0x7fda0c15eb90 2024-09-12T03:24:00.024 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:00.017+0000 7fda03fff640 1 -- 172.21.15.27:0/2413281057 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9fc014820 con 0x7fda0c15eb90 2024-09-12T03:24:00.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:00.017+0000 7fda1187b640 1 -- 172.21.15.27:0/2413281057 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda0c0cc730 con 0x7fda0c15eb90 2024-09-12T03:24:00.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:00.018+0000 7fda1187b640 1 -- 172.21.15.27:0/2413281057 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda0c158890 con 0x7fda0c15eb90 2024-09-12T03:24:00.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:00.019+0000 7fda03fff640 1 -- 172.21.15.27:0/2413281057 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 25) ==== 91365+0+0 (secure 0 0 0) 0x7fd9fc0149c0 con 0x7fda0c15eb90 2024-09-12T03:24:00.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:00.019+0000 7fda03fff640 1 --2- 172.21.15.27:0/2413281057 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fd9ec06c7c0 0x7fd9ec06ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:00.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:00.019+0000 7fda03fff640 1 -- 172.21.15.27:0/2413281057 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(160..160 src has 1..160) ==== 5765+0+0 (secure 0 0 0) 0x7fd9fc089ca0 con 0x7fda0c15eb90 2024-09-12T03:24:00.029 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:00.019+0000 7fda0affd640 1 --2- 172.21.15.27:0/2413281057 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fd9ec06c7c0 0x7fd9ec06ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:00.029 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:00.020+0000 7fda0affd640 1 --2- 172.21.15.27:0/2413281057 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fd9ec06c7c0 0x7fd9ec06ec80 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fda04005880 tx=0x7fda04009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:00.029 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:00.024+0000 7fda03fff640 1 -- 172.21.15.27:0/2413281057 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fd9fc059c00 con 0x7fda0c15eb90 2024-09-12T03:24:00.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:00.234+0000 7fda1187b640 1 -- 172.21.15.27:0/2413281057 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fda0c0ba4a0 con 0x7fda0c15eb90 2024-09-12T03:24:00.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:00.235+0000 7fda03fff640 1 -- 172.21.15.27:0/2413281057 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v99) ==== 53+0+73 (secure 0 0 0) 0x7fd9fc05df00 con 0x7fda0c15eb90 2024-09-12T03:24:00.237 INFO:teuthology.orchestra.run.smithi027.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-12T03:24:00.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:00.237+0000 7fda1187b640 1 -- 172.21.15.27:0/2413281057 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fd9ec06c7c0 msgr2=0x7fd9ec06ec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:00.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:00.237+0000 7fda1187b640 1 --2- 172.21.15.27:0/2413281057 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fd9ec06c7c0 0x7fd9ec06ec80 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fda04005880 tx=0x7fda04009040 comp rx=0 tx=0).stop 2024-09-12T03:24:00.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:00.237+0000 7fda1187b640 1 -- 172.21.15.27:0/2413281057 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fda0c15eb90 msgr2=0x7fda0c0cb8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:00.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:00.237+0000 7fda1187b640 1 --2- 172.21.15.27:0/2413281057 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fda0c15eb90 0x7fda0c0cb8e0 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7fd9fc007d20 tx=0x7fd9fc00e4b0 comp rx=0 tx=0).stop 2024-09-12T03:24:00.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:00.237+0000 7fda1187b640 1 -- 172.21.15.27:0/2413281057 shutdown_connections 2024-09-12T03:24:00.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:00.237+0000 7fda1187b640 1 --2- 172.21.15.27:0/2413281057 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fd9ec06c7c0 0x7fd9ec06ec80 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:00.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:00.237+0000 7fda1187b640 1 --2- 172.21.15.27:0/2413281057 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fda0c15eb90 0x7fda0c0cb8e0 unknown :-1 s=CLOSED pgs=1704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:00.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:00.237+0000 7fda1187b640 1 --2- 172.21.15.27:0/2413281057 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fda0c0d5200 0x7fda0c1573f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:00.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:00.237+0000 7fda1187b640 1 --2- 172.21.15.27:0/2413281057 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fda0c0d1550 0x7fda0c0cd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:00.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:00.237+0000 7fda1187b640 1 -- 172.21.15.27:0/2413281057 >> 172.21.15.27:0/2413281057 conn(0x7fda0c06dce0 msgr2=0x7fda0c0d1f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:00.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:00.237+0000 7fda1187b640 1 -- 172.21.15.27:0/2413281057 shutdown_connections 2024-09-12T03:24:00.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:00.237+0000 7fda1187b640 1 -- 172.21.15.27:0/2413281057 wait complete. 2024-09-12T03:24:00.249 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-12T03:24:00.249 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-12T03:24:00.249 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi027.front.sepia.ceph.com 2024-09-12T03:24:00.250 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-12T03:24:00.250 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-12T03:24:00.250 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-12T03:24:00.250 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-12T03:24:00.250 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-12T03:24:00.250 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-12T03:24:00.250 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:24:00.250 DEBUG:teuthology.orchestra.run.smithi027:> ip addr 2024-09-12T03:24:00.267 INFO:teuthology.orchestra.run.smithi027.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-12T03:24:00.267 INFO:teuthology.orchestra.run.smithi027.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-12T03:24:00.267 INFO:teuthology.orchestra.run.smithi027.stdout: inet 127.0.0.1/8 scope host lo 2024-09-12T03:24:00.267 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:24:00.268 INFO:teuthology.orchestra.run.smithi027.stdout: inet6 ::1/128 scope host 2024-09-12T03:24:00.268 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:24:00.268 INFO:teuthology.orchestra.run.smithi027.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-12T03:24:00.268 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether 0c:c4:7a:6b:fe:c8 brd ff:ff:ff:ff:ff:ff 2024-09-12T03:24:00.268 INFO:teuthology.orchestra.run.smithi027.stdout: altname enp7s0f0 2024-09-12T03:24:00.268 INFO:teuthology.orchestra.run.smithi027.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-12T03:24:00.268 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether 0c:c4:7a:6b:fe:c9 brd ff:ff:ff:ff:ff:ff 2024-09-12T03:24:00.268 INFO:teuthology.orchestra.run.smithi027.stdout: altname enp7s0f1 2024-09-12T03:24:00.268 INFO:teuthology.orchestra.run.smithi027.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-12T03:24:00.268 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether ac:1f:6b:f8:39:5a brd ff:ff:ff:ff:ff:ff 2024-09-12T03:24:00.268 INFO:teuthology.orchestra.run.smithi027.stdout: altname enp1s0f0 2024-09-12T03:24:00.268 INFO:teuthology.orchestra.run.smithi027.stdout: inet 172.21.15.27/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-12T03:24:00.268 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft 42348sec preferred_lft 42348sec 2024-09-12T03:24:00.269 INFO:teuthology.orchestra.run.smithi027.stdout: inet6 fe80::ae1f:6bff:fef8:395a/64 scope link 2024-09-12T03:24:00.269 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:24:00.269 INFO:teuthology.orchestra.run.smithi027.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-12T03:24:00.269 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether ac:1f:6b:f8:39:5b brd ff:ff:ff:ff:ff:ff 2024-09-12T03:24:00.269 INFO:teuthology.orchestra.run.smithi027.stdout: altname enp1s0f1 2024-09-12T03:24:00.269 INFO:teuthology.orchestra.run.smithi027.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-12T03:24:00.269 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether 2e:7f:89:6c:33:a0 brd ff:ff:ff:ff:ff:ff 2024-09-12T03:24:00.269 INFO:teuthology.orchestra.run.smithi027.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-12T03:24:00.269 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:24:00.269 INFO:teuthology.orchestra.run.smithi027.stdout: inet6 fe80::c84b:19ff:fef0:ca74/64 scope link 2024-09-12T03:24:00.269 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:24:00.269 INFO:teuthology.orchestra.run.smithi027.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-12T03:24:00.269 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether 42:40:d7:5c:7a:8e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-12T03:24:00.269 INFO:teuthology.orchestra.run.smithi027.stdout: inet6 fe80::4040:d7ff:fe5c:7a8e/64 scope link 2024-09-12T03:24:00.270 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:24:00.270 INFO:teuthology.orchestra.run.smithi027.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-12T03:24:00.270 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether 2e:7f:89:6c:33:a0 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-12T03:24:00.270 INFO:teuthology.orchestra.run.smithi027.stdout: inet6 fe80::2c7f:89ff:fe6c:33a0/64 scope link 2024-09-12T03:24:00.270 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:24:00.270 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:24:00.270 DEBUG:teuthology.orchestra.run.smithi027:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-12T03:24:00.332 INFO:teuthology.orchestra.run.smithi027.stdout:1 2024-09-12T03:24:00.333 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:24:00.334 DEBUG:teuthology.orchestra.run.smithi027:> route 2024-09-12T03:24:00.390 INFO:teuthology.orchestra.run.smithi027.stdout:Kernel IP routing table 2024-09-12T03:24:00.390 INFO:teuthology.orchestra.run.smithi027.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-12T03:24:00.390 INFO:teuthology.orchestra.run.smithi027.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-12T03:24:00.390 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-12T03:24:00.390 INFO:teuthology.orchestra.run.smithi027.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-12T03:24:00.391 DEBUG:teuthology.orchestra.run.smithi027:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-12T03:24:00.391 DEBUG:teuthology.orchestra.run.smithi027:> set -e 2024-09-12T03:24:00.391 DEBUG:teuthology.orchestra.run.smithi027:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-12T03:24:00.392 DEBUG:teuthology.orchestra.run.smithi027:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-12T03:24:00.392 DEBUG:teuthology.orchestra.run.smithi027:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-12T03:24:00.392 DEBUG:teuthology.orchestra.run.smithi027:> ') 2024-09-12T03:24:00.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:00 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T03:24:00.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T03:24:00.519 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:24:00.519 DEBUG:teuthology.orchestra.run.smithi027:> ip netns list 2024-09-12T03:24:00.535 INFO:teuthology.orchestra.run.smithi027.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-12T03:24:00.535 INFO:teuthology.orchestra.run.smithi027.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-12T03:24:00.536 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-12T03:24:00.536 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:24:00.536 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip link set brx.0 up 2024-09-12T03:24:00.602 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-12T03:24:00.602 DEBUG:teuthology.orchestra.run.smithi027:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-12T03:24:00.660 INFO:teuthology.orchestra.run.smithi027.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-12T03:24:00.660 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-12T03:24:00.660 DEBUG:teuthology.orchestra.run.smithi027:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-12T03:24:00.715 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-09-12T03:24:00.715 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:24:00.716 DEBUG:teuthology.orchestra.run.smithi027:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-12T03:24:00.801 INFO:teuthology.orchestra.run.smithi027.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-12T03:24:00.801 INFO:teuthology.orchestra.run.smithi027.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-12T03:24:00.815 INFO:teuthology.orchestra.run.smithi027.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-12T03:24:00.815 INFO:teuthology.orchestra.run.smithi027.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-12T03:24:00.815 INFO:teuthology.orchestra.run.smithi027.stdout:mount.ceph: resolved to: "172.21.15.27:3300,172.21.15.45:3300,172.21.15.45:3301" 2024-09-12T03:24:00.815 INFO:teuthology.orchestra.run.smithi027.stdout:mount.ceph: trying mount with new device syntax: 0@d2df504e-2647-4385-a5db-9fdf69f487a0.cephfs=/ 2024-09-12T03:24:00.815 INFO:teuthology.orchestra.run.smithi027.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.27:3300/172.21.15.45:3300/172.21.15.45:3301" will pass to kernel 2024-09-12T03:24:00.816 INFO:teuthology.orchestra.run.smithi027.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-12T03:24:00.816 INFO:teuthology.orchestra.run.smithi027.stdout: You just mounted a file system that supports labels which does not 2024-09-12T03:24:00.816 INFO:teuthology.orchestra.run.smithi027.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-12T03:24:00.816 INFO:teuthology.orchestra.run.smithi027.stdout: applications will generate AVC messages and not be allowed access to 2024-09-12T03:24:00.816 INFO:teuthology.orchestra.run.smithi027.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-12T03:24:00.817 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-12T03:24:00.817 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:24:00.818 DEBUG:teuthology.orchestra.run.smithi027:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-12T03:24:00.844 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-12T03:24:00.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:00 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-12T03:24:00.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T03:24:00.945 INFO:teuthology.orchestra.run.smithi027.stdout:client7250 2024-09-12T03:24:00.946 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-12T03:24:00.946 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T03:24:00.962 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-12T03:24:01.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:01.102+0000 7f1744aaf640 1 Processor -- start 2024-09-12T03:24:01.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:01.102+0000 7f1744aaf640 1 -- start start 2024-09-12T03:24:01.105 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:01.103+0000 7f1744aaf640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f174015e840 0x7f174015ec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:01.105 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:01.103+0000 7f1744aaf640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f1740071280 0x7f17400716c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:01.105 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:01.103+0000 7f1744aaf640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f17400d2490 0x7f17400d2940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:01.105 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:01.103+0000 7f1744aaf640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f1740073360 con 0x7f1740071280 2024-09-12T03:24:01.105 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:01.103+0000 7f1744aaf640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f1740073640 con 0x7f17400d2490 2024-09-12T03:24:01.105 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:01.103+0000 7f1744aaf640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f1740073920 con 0x7f174015e840 2024-09-12T03:24:01.105 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:01.103+0000 7f173dd9b640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f1740071280 0x7f17400716c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:01.105 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:01.104+0000 7f173dd9b640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f1740071280 0x7f17400716c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:32974/0 (socket says 172.21.15.27:32974) 2024-09-12T03:24:01.105 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:01.104+0000 7f173dd9b640 1 -- 172.21.15.27:0/2371380899 learned_addr learned my addr 172.21.15.27:0/2371380899 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:24:01.105 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:01.104+0000 7f173dd9b640 1 -- 172.21.15.27:0/2371380899 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f174015e840 msgr2=0x7f174015ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:24:01.105 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:01.104+0000 7f173e59c640 1 --2- 172.21.15.27:0/2371380899 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f174015e840 0x7f174015ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:01.106 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:01.104+0000 7f173ed9d640 1 --2- 172.21.15.27:0/2371380899 >> v2:172.21.15.45:3300/0 conn(0x7f17400d2490 0x7f17400d2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:01.106 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:01.104+0000 7f173dd9b640 1 --2- 172.21.15.27:0/2371380899 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f174015e840 0x7f174015ec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:01.106 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:01.104+0000 7f173dd9b640 1 -- 172.21.15.27:0/2371380899 >> v2:172.21.15.45:3300/0 conn(0x7f17400d2490 msgr2=0x7f17400d2940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:24:01.106 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:01.104+0000 7f173dd9b640 1 --2- 172.21.15.27:0/2371380899 >> v2:172.21.15.45:3300/0 conn(0x7f17400d2490 0x7f17400d2940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:01.106 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:01.104+0000 7f173dd9b640 1 -- 172.21.15.27:0/2371380899 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1740073c00 con 0x7f1740071280 2024-09-12T03:24:01.106 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:01.104+0000 7f173ed9d640 1 --2- 172.21.15.27:0/2371380899 >> v2:172.21.15.45:3300/0 conn(0x7f17400d2490 0x7f17400d2940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:24:01.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:01.105+0000 7f173dd9b640 1 --2- 172.21.15.27:0/2371380899 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1740071280 0x7f17400716c0 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7f1730009870 tx=0x7f173002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e26dda3baa714218 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:01.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:01.105+0000 7f173d59a640 1 -- 172.21.15.27:0/2371380899 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1730037070 con 0x7f1740071280 2024-09-12T03:24:01.108 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:01.105+0000 7f173d59a640 1 -- 172.21.15.27:0/2371380899 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1730032440 con 0x7f1740071280 2024-09-12T03:24:01.108 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:01.105+0000 7f173d59a640 1 -- 172.21.15.27:0/2371380899 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1730032810 con 0x7f1740071280 2024-09-12T03:24:01.108 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:01.105+0000 7f1744aaf640 1 -- 172.21.15.27:0/2371380899 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1740071280 msgr2=0x7f17400716c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:01.108 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:01.105+0000 7f1744aaf640 1 --2- 172.21.15.27:0/2371380899 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1740071280 0x7f17400716c0 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7f1730009870 tx=0x7f173002a210 comp rx=0 tx=0).stop 2024-09-12T03:24:01.108 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:01.105+0000 7f1744aaf640 1 -- 172.21.15.27:0/2371380899 shutdown_connections 2024-09-12T03:24:01.108 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:01.105+0000 7f1744aaf640 1 --2- 172.21.15.27:0/2371380899 >> v2:172.21.15.45:3300/0 conn(0x7f17400d2490 0x7f17400d2940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:01.108 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:01.105+0000 7f1744aaf640 1 --2- 172.21.15.27:0/2371380899 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1740071280 0x7f17400716c0 unknown :-1 s=CLOSED pgs=1705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:01.108 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:01.105+0000 7f1744aaf640 1 --2- 172.21.15.27:0/2371380899 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f174015e840 0x7f174015ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:01.108 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:01.105+0000 7f1744aaf640 1 -- 172.21.15.27:0/2371380899 >> 172.21.15.27:0/2371380899 conn(0x7f174006f320 msgr2=0x7f174006f700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:01.108 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:01.105+0000 7f1744aaf640 1 -- 172.21.15.27:0/2371380899 shutdown_connections 2024-09-12T03:24:01.108 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:01.105+0000 7f1744aaf640 1 -- 172.21.15.27:0/2371380899 wait complete. 2024-09-12T03:24:01.108 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:01.107+0000 7f1744aaf640 1 Processor -- start 2024-09-12T03:24:01.109 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:01.107+0000 7f1744aaf640 1 -- start start 2024-09-12T03:24:01.110 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:01.108+0000 7f1744aaf640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1740071280 0x7f1740147110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:01.110 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:01.108+0000 7f1744aaf640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f17400d2490 0x7f1740152b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:01.110 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:01.108+0000 7f1744aaf640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f174015e840 0x7f1740153070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:01.110 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:01.108+0000 7f1744aaf640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f174015f160 con 0x7f1740071280 2024-09-12T03:24:01.110 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:01.108+0000 7f1744aaf640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f17400b7810 con 0x7f174015e840 2024-09-12T03:24:01.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:01.108+0000 7f1744aaf640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f174006dee0 con 0x7f17400d2490 2024-09-12T03:24:01.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:01.108+0000 7f173e59c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1740071280 0x7f1740147110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:01.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:01.108+0000 7f173e59c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1740071280 0x7f1740147110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:32982/0 (socket says 172.21.15.27:32982) 2024-09-12T03:24:01.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:01.108+0000 7f173e59c640 1 -- 172.21.15.27:0/1460265963 learned_addr learned my addr 172.21.15.27:0/1460265963 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:24:01.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:01.108+0000 7f173e59c640 1 -- 172.21.15.27:0/1460265963 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f17400d2490 msgr2=0x7f1740152b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:01.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:01.108+0000 7f173e59c640 1 --2- 172.21.15.27:0/1460265963 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f17400d2490 0x7f1740152b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:01.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:01.108+0000 7f173e59c640 1 -- 172.21.15.27:0/1460265963 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f174015e840 msgr2=0x7f1740153070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:01.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:01.108+0000 7f173e59c640 1 --2- 172.21.15.27:0/1460265963 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f174015e840 0x7f1740153070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:01.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:01.108+0000 7f173e59c640 1 -- 172.21.15.27:0/1460265963 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17400b23f0 con 0x7f1740071280 2024-09-12T03:24:01.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:01.108+0000 7f173e59c640 1 --2- 172.21.15.27:0/1460265963 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1740071280 0x7f1740147110 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7f172800c7f0 tx=0x7f172800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:01.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:01.108+0000 7f1736ffd640 1 -- 172.21.15.27:0/1460265963 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1728015070 con 0x7f1740071280 2024-09-12T03:24:01.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:01.108+0000 7f1736ffd640 1 -- 172.21.15.27:0/1460265963 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1728004220 con 0x7f1740071280 2024-09-12T03:24:01.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:01.108+0000 7f1736ffd640 1 -- 172.21.15.27:0/1460265963 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1728007620 con 0x7f1740071280 2024-09-12T03:24:01.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:01.108+0000 7f1744aaf640 1 -- 172.21.15.27:0/1460265963 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17400b26d0 con 0x7f1740071280 2024-09-12T03:24:01.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:01.108+0000 7f1744aaf640 1 -- 172.21.15.27:0/1460265963 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17400b2c90 con 0x7f1740071280 2024-09-12T03:24:01.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:01.109+0000 7f1744aaf640 1 -- 172.21.15.27:0/1460265963 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17400716c0 con 0x7f1740071280 2024-09-12T03:24:01.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:01.110+0000 7f1736ffd640 1 -- 172.21.15.27:0/1460265963 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 25) ==== 91365+0+0 (secure 0 0 0) 0x7f17280079d0 con 0x7f1740071280 2024-09-12T03:24:01.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:01.110+0000 7f1736ffd640 1 --2- 172.21.15.27:0/1460265963 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f172006c7c0 0x7f172006ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:01.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:01.110+0000 7f1736ffd640 1 -- 172.21.15.27:0/1460265963 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(160..160 src has 1..160) ==== 5765+0+0 (secure 0 0 0) 0x7f1728089900 con 0x7f1740071280 2024-09-12T03:24:01.113 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:01.110+0000 7f173dd9b640 1 --2- 172.21.15.27:0/1460265963 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f172006c7c0 0x7f172006ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:01.113 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:01.111+0000 7f173dd9b640 1 --2- 172.21.15.27:0/1460265963 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f172006c7c0 0x7f172006ec80 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f1730034000 tx=0x7f173002c000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:01.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:01.115+0000 7f1736ffd640 1 -- 172.21.15.27:0/1460265963 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f1728059eb0 con 0x7f1740071280 2024-09-12T03:24:01.327 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:01.326+0000 7f1744aaf640 1 -- 172.21.15.27:0/1460265963 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f1740069150 con 0x7f1740071280 2024-09-12T03:24:01.327 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:01.326+0000 7f1736ffd640 1 -- 172.21.15.27:0/1460265963 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v99) ==== 53+0+73 (secure 0 0 0) 0x7f1728060080 con 0x7f1740071280 2024-09-12T03:24:01.327 INFO:teuthology.orchestra.run.smithi027.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-12T03:24:01.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:01.328+0000 7f1744aaf640 1 -- 172.21.15.27:0/1460265963 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f172006c7c0 msgr2=0x7f172006ec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:01.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:01.328+0000 7f1744aaf640 1 --2- 172.21.15.27:0/1460265963 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f172006c7c0 0x7f172006ec80 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f1730034000 tx=0x7f173002c000 comp rx=0 tx=0).stop 2024-09-12T03:24:01.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:01.328+0000 7f1744aaf640 1 -- 172.21.15.27:0/1460265963 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1740071280 msgr2=0x7f1740147110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:01.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:01.328+0000 7f1744aaf640 1 --2- 172.21.15.27:0/1460265963 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1740071280 0x7f1740147110 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7f172800c7f0 tx=0x7f172800ccb0 comp rx=0 tx=0).stop 2024-09-12T03:24:01.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:01.328+0000 7f1744aaf640 1 -- 172.21.15.27:0/1460265963 shutdown_connections 2024-09-12T03:24:01.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:01.328+0000 7f1744aaf640 1 --2- 172.21.15.27:0/1460265963 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f174015e840 0x7f1740153070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:01.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:01.328+0000 7f1744aaf640 1 --2- 172.21.15.27:0/1460265963 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f17400d2490 0x7f1740152b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:01.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:01.328+0000 7f1744aaf640 1 --2- 172.21.15.27:0/1460265963 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f172006c7c0 0x7f172006ec80 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:01.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:01.328+0000 7f1744aaf640 1 --2- 172.21.15.27:0/1460265963 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1740071280 0x7f1740147110 unknown :-1 s=CLOSED pgs=1706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:01.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:01.328+0000 7f1744aaf640 1 -- 172.21.15.27:0/1460265963 >> 172.21.15.27:0/1460265963 conn(0x7f174006f320 msgr2=0x7f17400b6a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:01.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:01.328+0000 7f1744aaf640 1 -- 172.21.15.27:0/1460265963 shutdown_connections 2024-09-12T03:24:01.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:01.328+0000 7f1744aaf640 1 -- 172.21.15.27:0/1460265963 wait complete. 2024-09-12T03:24:01.340 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-12T03:24:01.340 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-12T03:24:01.340 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi045.front.sepia.ceph.com 2024-09-12T03:24:01.340 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-12T03:24:01.340 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-12T03:24:01.340 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-12T03:24:01.340 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-12T03:24:01.340 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-12T03:24:01.340 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-12T03:24:01.341 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:24:01.341 DEBUG:teuthology.orchestra.run.smithi045:> ip addr 2024-09-12T03:24:01.361 INFO:teuthology.orchestra.run.smithi045.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-12T03:24:01.361 INFO:teuthology.orchestra.run.smithi045.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-12T03:24:01.361 INFO:teuthology.orchestra.run.smithi045.stdout: inet 127.0.0.1/8 scope host lo 2024-09-12T03:24:01.361 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:24:01.361 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 ::1/128 scope host 2024-09-12T03:24:01.362 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:24:01.362 INFO:teuthology.orchestra.run.smithi045.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-12T03:24:01.362 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:6b:fd:a6 brd ff:ff:ff:ff:ff:ff 2024-09-12T03:24:01.362 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp7s0f0 2024-09-12T03:24:01.362 INFO:teuthology.orchestra.run.smithi045.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-12T03:24:01.362 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:6b:fd:a7 brd ff:ff:ff:ff:ff:ff 2024-09-12T03:24:01.362 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp7s0f1 2024-09-12T03:24:01.362 INFO:teuthology.orchestra.run.smithi045.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-12T03:24:01.362 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:bd:12:be brd ff:ff:ff:ff:ff:ff 2024-09-12T03:24:01.362 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp1s0f0 2024-09-12T03:24:01.362 INFO:teuthology.orchestra.run.smithi045.stdout: inet 172.21.15.45/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-12T03:24:01.362 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft 42292sec preferred_lft 42292sec 2024-09-12T03:24:01.362 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::ec4:7aff:febd:12be/64 scope link 2024-09-12T03:24:01.363 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:24:01.363 INFO:teuthology.orchestra.run.smithi045.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-12T03:24:01.363 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:bd:12:bf brd ff:ff:ff:ff:ff:ff 2024-09-12T03:24:01.363 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp1s0f1 2024-09-12T03:24:01.363 INFO:teuthology.orchestra.run.smithi045.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-12T03:24:01.363 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 52:97:33:5c:60:1b brd ff:ff:ff:ff:ff:ff 2024-09-12T03:24:01.363 INFO:teuthology.orchestra.run.smithi045.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-12T03:24:01.363 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:24:01.363 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::d8c2:5dff:fe26:78c8/64 scope link 2024-09-12T03:24:01.363 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:24:01.363 INFO:teuthology.orchestra.run.smithi045.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-12T03:24:01.363 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 5a:ec:d7:40:ef:75 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-12T03:24:01.363 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::58ec:d7ff:fe40:ef75/64 scope link 2024-09-12T03:24:01.363 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:24:01.364 INFO:teuthology.orchestra.run.smithi045.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-12T03:24:01.364 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 52:97:33:5c:60:1b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-12T03:24:01.364 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::5097:33ff:fe5c:601b/64 scope link 2024-09-12T03:24:01.364 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:24:01.364 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:24:01.364 DEBUG:teuthology.orchestra.run.smithi045:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-12T03:24:01.410 DEBUG:teuthology.orchestra.run.smithi027:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-12T03:24:01.412 DEBUG:teuthology.orchestra.run.smithi045:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-12T03:24:01.431 INFO:teuthology.orchestra.run.smithi045.stdout:1 2024-09-12T03:24:01.434 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:24:01.434 DEBUG:teuthology.orchestra.run.smithi045:> route 2024-09-12T03:24:01.492 INFO:teuthology.orchestra.run.smithi045.stdout:Kernel IP routing table 2024-09-12T03:24:01.492 INFO:teuthology.orchestra.run.smithi045.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-12T03:24:01.492 INFO:teuthology.orchestra.run.smithi045.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-12T03:24:01.493 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-12T03:24:01.493 INFO:teuthology.orchestra.run.smithi045.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-12T03:24:01.493 DEBUG:teuthology.orchestra.run.smithi045:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-12T03:24:01.493 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-09-12T03:24:01.493 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-12T03:24:01.493 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-12T03:24:01.493 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-12T03:24:01.493 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-09-12T03:24:01.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:24:01 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T03:24:01.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:24:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T03:24:01.631 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:24:01.631 DEBUG:teuthology.orchestra.run.smithi045:> ip netns list 2024-09-12T03:24:01.647 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-09-12T03:24:01.647 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-12T03:24:01.648 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-12T03:24:01.648 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:24:01.648 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link set brx.0 up 2024-09-12T03:24:01.714 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-12T03:24:01.715 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-12T03:24:01.771 INFO:teuthology.orchestra.run.smithi045.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-12T03:24:01.771 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-12T03:24:01.771 DEBUG:teuthology.orchestra.run.smithi045:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-12T03:24:01.827 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2024-09-12T03:24:01.828 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:24:01.828 DEBUG:teuthology.orchestra.run.smithi045:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-12T03:24:01.913 INFO:teuthology.orchestra.run.smithi045.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-12T03:24:01.913 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-12T03:24:01.921 INFO:teuthology.orchestra.run.smithi045.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-12T03:24:01.921 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-12T03:24:01.921 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: resolved to: "172.21.15.27:3300,172.21.15.45:3300,172.21.15.45:3301" 2024-09-12T03:24:01.921 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: trying mount with new device syntax: 1@d2df504e-2647-4385-a5db-9fdf69f487a0.cephfs=/ 2024-09-12T03:24:01.921 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.27:3300/172.21.15.45:3300/172.21.15.45:3301" will pass to kernel 2024-09-12T03:24:01.921 INFO:teuthology.orchestra.run.smithi045.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-09-12T03:24:01.921 INFO:teuthology.orchestra.run.smithi045.stdout: You just mounted a file system that supports labels which does not 2024-09-12T03:24:01.921 INFO:teuthology.orchestra.run.smithi045.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-12T03:24:01.922 INFO:teuthology.orchestra.run.smithi045.stdout: applications will generate AVC messages and not be allowed access to 2024-09-12T03:24:01.922 INFO:teuthology.orchestra.run.smithi045.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-12T03:24:01.923 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-12T03:24:01.923 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:24:01.923 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-12T03:24:01.990 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-12T03:24:02.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:24:02 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-12T03:24:02.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-12T03:24:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T03:24:02.087 INFO:teuthology.orchestra.run.smithi045.stdout:client7253 2024-09-12T03:24:02.088 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-12T03:24:02.088 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T03:24:02.104 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-09-12T03:24:02.215 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:02.214+0000 7fe42e798640 1 Processor -- start 2024-09-12T03:24:02.216 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:02.214+0000 7fe42e798640 1 -- start start 2024-09-12T03:24:02.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:02.215+0000 7fe42e798640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fe42806c650 0x7fe4280cf410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:02.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:02.215+0000 7fe42e798640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fe42815d560 0x7fe4280cf950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:02.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:02.215+0000 7fe42e798640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe4280ce650 con 0x7fe42815a270 2024-09-12T03:24:02.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:02.215+0000 7fe42e798640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7fe4280cef80 con 0x7fe42815d560 2024-09-12T03:24:02.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:02.215+0000 7fe42e798640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fe4280ce930 con 0x7fe42806c650 2024-09-12T03:24:02.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:02.215+0000 7fe42cd35640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fe42815d560 0x7fe4280cf950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:02.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:02.215+0000 7fe42cd35640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fe42815d560 0x7fe4280cf950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:32998/0 (socket says 172.21.15.27:32998) 2024-09-12T03:24:02.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:02.215+0000 7fe42cd35640 1 -- 172.21.15.27:0/346352294 learned_addr learned my addr 172.21.15.27:0/346352294 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:24:02.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:02.216+0000 7fe41ffff640 1 --2- 172.21.15.27:0/346352294 >> v2:172.21.15.45:3300/0 conn(0x7fe42806c650 0x7fe4280cf410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:02.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:02.216+0000 7fe42cd35640 1 -- 172.21.15.27:0/346352294 >> v2:172.21.15.45:3300/0 conn(0x7fe42806c650 msgr2=0x7fe4280cf410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:24:02.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:02.216+0000 7fe42cd35640 1 --2- 172.21.15.27:0/346352294 >> v2:172.21.15.45:3300/0 conn(0x7fe42806c650 0x7fe4280cf410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:02.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:02.216+0000 7fe42cd35640 1 -- 172.21.15.27:0/346352294 >> v1:172.21.15.45:6789/0 conn(0x7fe42815a270 legacy=0x7fe42806cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:02.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:02.216+0000 7fe42cd35640 1 -- 172.21.15.27:0/346352294 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4280cec10 con 0x7fe42815d560 2024-09-12T03:24:02.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:02.217+0000 7fe42cd35640 1 --2- 172.21.15.27:0/346352294 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe42815d560 0x7fe4280cf950 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 crypto rx=0x7fe4280b3260 tx=0x7fe42002a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14cb64e4976c6e88 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:02.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:02.217+0000 7fe4277fe640 1 -- 172.21.15.27:0/346352294 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe420036070 con 0x7fe42815d560 2024-09-12T03:24:02.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:02.217+0000 7fe4277fe640 1 -- 172.21.15.27:0/346352294 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe420031600 con 0x7fe42815d560 2024-09-12T03:24:02.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:02.217+0000 7fe4277fe640 1 -- 172.21.15.27:0/346352294 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4200319d0 con 0x7fe42815d560 2024-09-12T03:24:02.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:02.217+0000 7fe42e798640 1 -- 172.21.15.27:0/346352294 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe42815d560 msgr2=0x7fe4280cf950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:02.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:02.217+0000 7fe42e798640 1 --2- 172.21.15.27:0/346352294 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe42815d560 0x7fe4280cf950 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 crypto rx=0x7fe4280b3260 tx=0x7fe42002a730 comp rx=0 tx=0).stop 2024-09-12T03:24:02.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:02.217+0000 7fe42e798640 1 -- 172.21.15.27:0/346352294 shutdown_connections 2024-09-12T03:24:02.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:02.217+0000 7fe42e798640 1 --2- 172.21.15.27:0/346352294 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe42815d560 0x7fe4280cf950 unknown :-1 s=CLOSED pgs=1707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:02.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:02.217+0000 7fe42e798640 1 --2- 172.21.15.27:0/346352294 >> v2:172.21.15.45:3300/0 conn(0x7fe42806c650 0x7fe4280cf410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:02.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:02.217+0000 7fe42e798640 1 -- 172.21.15.27:0/346352294 >> 172.21.15.27:0/346352294 conn(0x7fe42806bb30 msgr2=0x7fe42806bf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:02.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:02.217+0000 7fe42e798640 1 -- 172.21.15.27:0/346352294 shutdown_connections 2024-09-12T03:24:02.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:02.217+0000 7fe42e798640 1 -- 172.21.15.27:0/346352294 wait complete. 2024-09-12T03:24:02.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:02.218+0000 7fe42e798640 1 Processor -- start 2024-09-12T03:24:02.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:02.219+0000 7fe42e798640 1 -- start start 2024-09-12T03:24:02.223 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:02.219+0000 7fe42e798640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe42806c650 0x7fe428148520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:02.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:02.219+0000 7fe42e798640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe42815a270 0x7fe4280cc1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:02.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:02.219+0000 7fe42e798640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe42815d560 0x7fe428146b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:02.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:02.219+0000 7fe42e798640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fe4280726a0 con 0x7fe42815a270 2024-09-12T03:24:02.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:02.219+0000 7fe42e798640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe428073320 con 0x7fe42815d560 2024-09-12T03:24:02.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:02.219+0000 7fe42e798640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fe428072820 con 0x7fe42806c650 2024-09-12T03:24:02.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:02.219+0000 7fe41ffff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe42815a270 0x7fe4280cc1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:02.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:02.219+0000 7fe41ffff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe42815a270 0x7fe4280cc1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33006/0 (socket says 172.21.15.27:33006) 2024-09-12T03:24:02.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:02.219+0000 7fe41ffff640 1 -- 172.21.15.27:0/4129250843 learned_addr learned my addr 172.21.15.27:0/4129250843 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:24:02.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:02.219+0000 7fe42cd35640 1 --2- 172.21.15.27:0/4129250843 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe42815d560 0x7fe428146b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:02.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:02.219+0000 7fe427fff640 1 --2- 172.21.15.27:0/4129250843 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe42806c650 0x7fe428148520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:02.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:02.220+0000 7fe41ffff640 1 -- 172.21.15.27:0/4129250843 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe42806c650 msgr2=0x7fe428148520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:02.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:02.220+0000 7fe41ffff640 1 --2- 172.21.15.27:0/4129250843 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe42806c650 0x7fe428148520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:02.225 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:02.220+0000 7fe41ffff640 1 -- 172.21.15.27:0/4129250843 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe42815d560 msgr2=0x7fe428146b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:02.225 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:02.220+0000 7fe41ffff640 1 --2- 172.21.15.27:0/4129250843 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe42815d560 0x7fe428146b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:02.225 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:02.220+0000 7fe41ffff640 1 -- 172.21.15.27:0/4129250843 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe428147480 con 0x7fe42815a270 2024-09-12T03:24:02.225 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:02.220+0000 7fe41ffff640 1 --2- 172.21.15.27:0/4129250843 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe42815a270 0x7fe4280cc1d0 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7fe41000e9b0 tx=0x7fe41000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:02.225 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:02.220+0000 7fe4257fa640 1 -- 172.21.15.27:0/4129250843 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe410017070 con 0x7fe42815a270 2024-09-12T03:24:02.225 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:02.220+0000 7fe4257fa640 1 -- 172.21.15.27:0/4129250843 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe41000c9a0 con 0x7fe42815a270 2024-09-12T03:24:02.225 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:02.220+0000 7fe42e798640 1 -- 172.21.15.27:0/4129250843 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe428147760 con 0x7fe42815a270 2024-09-12T03:24:02.225 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:02.220+0000 7fe4257fa640 1 -- 172.21.15.27:0/4129250843 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe41000cd90 con 0x7fe42815a270 2024-09-12T03:24:02.225 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:02.220+0000 7fe42e798640 1 -- 172.21.15.27:0/4129250843 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe428147d20 con 0x7fe42815a270 2024-09-12T03:24:02.225 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:02.221+0000 7fe42e798640 1 -- 172.21.15.27:0/4129250843 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe42815a650 con 0x7fe42815a270 2024-09-12T03:24:02.225 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:02.222+0000 7fe4257fa640 1 -- 172.21.15.27:0/4129250843 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 25) ==== 91365+0+0 (secure 0 0 0) 0x7fe41001a070 con 0x7fe42815a270 2024-09-12T03:24:02.225 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:02.222+0000 7fe4257fa640 1 --2- 172.21.15.27:0/4129250843 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fe40806c7c0 0x7fe40806ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:02.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:02.222+0000 7fe4257fa640 1 -- 172.21.15.27:0/4129250843 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(160..160 src has 1..160) ==== 5765+0+0 (secure 0 0 0) 0x7fe410089910 con 0x7fe42815a270 2024-09-12T03:24:02.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:02.222+0000 7fe427fff640 1 --2- 172.21.15.27:0/4129250843 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fe40806c7c0 0x7fe40806ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:02.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:02.223+0000 7fe427fff640 1 --2- 172.21.15.27:0/4129250843 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fe40806c7c0 0x7fe40806ec80 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fe42815c920 tx=0x7fe418009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:02.227 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:02.227+0000 7fe4257fa640 1 -- 172.21.15.27:0/4129250843 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fe42815a650 con 0x7fe42815a270 2024-09-12T03:24:02.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:02.440+0000 7fe42e798640 1 -- 172.21.15.27:0/4129250843 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7fe4280695a0 con 0x7fe42815a270 2024-09-12T03:24:03.290 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:03.289+0000 7fe4257fa640 1 -- 172.21.15.27:0/4129250843 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v100) ==== 136+0+0 (secure 0 0 0) 0x7fe4280695a0 con 0x7fe42815a270 2024-09-12T03:24:03.293 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:03.292+0000 7fe42e798640 1 -- 172.21.15.27:0/4129250843 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fe40806c7c0 msgr2=0x7fe40806ec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:03.293 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:03.292+0000 7fe42e798640 1 --2- 172.21.15.27:0/4129250843 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fe40806c7c0 0x7fe40806ec80 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fe42815c920 tx=0x7fe418009040 comp rx=0 tx=0).stop 2024-09-12T03:24:03.293 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:03.292+0000 7fe42e798640 1 -- 172.21.15.27:0/4129250843 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe42815a270 msgr2=0x7fe4280cc1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:03.293 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:03.292+0000 7fe42e798640 1 --2- 172.21.15.27:0/4129250843 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe42815a270 0x7fe4280cc1d0 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7fe41000e9b0 tx=0x7fe41000ee70 comp rx=0 tx=0).stop 2024-09-12T03:24:03.293 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:03.292+0000 7fe42e798640 1 -- 172.21.15.27:0/4129250843 shutdown_connections 2024-09-12T03:24:03.293 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:03.292+0000 7fe42e798640 1 --2- 172.21.15.27:0/4129250843 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe42815d560 0x7fe428146b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:03.293 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:03.292+0000 7fe42e798640 1 --2- 172.21.15.27:0/4129250843 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fe40806c7c0 0x7fe40806ec80 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:03.293 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:03.292+0000 7fe42e798640 1 --2- 172.21.15.27:0/4129250843 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe42815a270 0x7fe4280cc1d0 unknown :-1 s=CLOSED pgs=1708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:03.293 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:03.292+0000 7fe42e798640 1 --2- 172.21.15.27:0/4129250843 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe42806c650 0x7fe428148520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:03.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:03.292+0000 7fe42e798640 1 -- 172.21.15.27:0/4129250843 >> 172.21.15.27:0/4129250843 conn(0x7fe42806bb30 msgr2=0x7fe4280b07b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:03.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:03.292+0000 7fe42e798640 1 -- 172.21.15.27:0/4129250843 shutdown_connections 2024-09-12T03:24:03.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:03.292+0000 7fe42e798640 1 -- 172.21.15.27:0/4129250843 wait complete. 2024-09-12T03:24:03.307 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-12T03:24:03.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:03.415+0000 7f91c525a640 1 Processor -- start 2024-09-12T03:24:03.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:03.415+0000 7f91c525a640 1 -- start start 2024-09-12T03:24:03.421 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:03.416+0000 7f91c525a640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f91c015eb90 0x7f91c00d2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:03.421 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:03.416+0000 7f91c525a640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f91c00d3760 con 0x7f91c00d5200 2024-09-12T03:24:03.421 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:03.416+0000 7f91c525a640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f91c00d3a40 con 0x7f91c00d1720 2024-09-12T03:24:03.421 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:03.416+0000 7f91c525a640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f91c00b7d20 con 0x7f91c015eb90 2024-09-12T03:24:03.421 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:03.416+0000 7f91be59c640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f91c00d5200 0x7f91c00d1010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:57498/0 (socket says 172.21.15.27:57498) 2024-09-12T03:24:03.422 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:03.416+0000 7f91be59c640 1 -- 172.21.15.27:0/1942849245 learned_addr learned my addr 172.21.15.27:0/1942849245 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:24:03.422 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:03.417+0000 7f91bed9d640 1 --2- 172.21.15.27:0/1942849245 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f91c015eb90 0x7f91c00d2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:03.422 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:03.417+0000 7f91bdd9b640 1 -- 172.21.15.27:0/1942849245 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1143784545 0 0) 0x7f91c00d3760 con 0x7f91c00d5200 2024-09-12T03:24:03.422 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:03.417+0000 7f91bdd9b640 1 -- 172.21.15.27:0/1942849245 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f91ac003520 con 0x7f91c00d5200 2024-09-12T03:24:03.422 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:03.417+0000 7f91bdd9b640 1 -- 172.21.15.27:0/1942849245 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 793662065 0 0) 0x7f91ac003520 con 0x7f91c00d5200 2024-09-12T03:24:03.422 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:03.417+0000 7f91bdd9b640 1 -- 172.21.15.27:0/1942849245 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f91c015eb90 msgr2=0x7f91c00d2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:24:03.422 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:03.417+0000 7f91bdd9b640 1 --2- 172.21.15.27:0/1942849245 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f91c015eb90 0x7f91c00d2df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:03.422 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:03.417+0000 7f91bdd9b640 1 -- 172.21.15.27:0/1942849245 >> v1:172.21.15.45:6789/0 conn(0x7f91c00d1720 legacy=0x7f91c00b8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:03.422 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:03.417+0000 7f91bdd9b640 1 -- 172.21.15.27:0/1942849245 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91c00d3d20 con 0x7f91c00d5200 2024-09-12T03:24:03.422 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:03.417+0000 7f91bdd9b640 1 -- 172.21.15.27:0/1942849245 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 847755213 0 0) 0x7f91a8001670 con 0x7f91c00d5200 2024-09-12T03:24:03.422 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:03.417+0000 7f91bdd9b640 1 -- 172.21.15.27:0/1942849245 >> v1:172.21.15.27:6789/0 conn(0x7f91c00d5200 legacy=0x7f91c00d1010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:03.423 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:03.417+0000 7f91bdd9b640 1 --2- 172.21.15.27:0/1942849245 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f91ac0039e0 0x7f91ac003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:03.423 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:03.417+0000 7f91bdd9b640 1 --2- 172.21.15.27:0/1942849245 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f91ac004390 0x7f91ac004860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:03.423 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:03.417+0000 7f91bdd9b640 1 --2- 172.21.15.27:0/1942849245 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f91c015eb90 0x7f91ac0050c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:03.423 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:03.417+0000 7f91bdd9b640 1 -- 172.21.15.27:0/1942849245 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f91ac00b9c0 con 0x7f91ac0039e0 2024-09-12T03:24:03.423 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:03.417+0000 7f91bdd9b640 1 -- 172.21.15.27:0/1942849245 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f91ac00bb40 con 0x7f91ac004390 2024-09-12T03:24:03.423 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:03.417+0000 7f91bdd9b640 1 -- 172.21.15.27:0/1942849245 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f91ac00bd40 con 0x7f91c015eb90 2024-09-12T03:24:03.423 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:03.417+0000 7f91bed9d640 1 --2- 172.21.15.27:0/1942849245 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f91ac0039e0 0x7f91ac003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:03.423 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:03.417+0000 7f91be59c640 1 --2- 172.21.15.27:0/1942849245 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f91ac004390 0x7f91ac004860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:03.423 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:03.417+0000 7f91bf59e640 1 --2- 172.21.15.27:0/1942849245 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f91c015eb90 0x7f91ac0050c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:03.423 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:03.417+0000 7f91bed9d640 1 -- 172.21.15.27:0/1942849245 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f91c015eb90 msgr2=0x7f91ac0050c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:24:03.423 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:03.417+0000 7f91bed9d640 1 --2- 172.21.15.27:0/1942849245 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f91c015eb90 0x7f91ac0050c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:03.424 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:03.417+0000 7f91bed9d640 1 -- 172.21.15.27:0/1942849245 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f91ac004390 msgr2=0x7f91ac004860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:24:03.424 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:03.417+0000 7f91bed9d640 1 --2- 172.21.15.27:0/1942849245 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f91ac004390 0x7f91ac004860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:03.424 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:03.417+0000 7f91bed9d640 1 -- 172.21.15.27:0/1942849245 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91ac003520 con 0x7f91ac0039e0 2024-09-12T03:24:03.424 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:03.418+0000 7f91bed9d640 1 --2- 172.21.15.27:0/1942849245 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f91ac0039e0 0x7f91ac003dc0 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 crypto rx=0x7f91b8009ac0 tx=0x7f91ac005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee620ec495ea98f5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:03.424 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:03.418+0000 7f91bdd9b640 1 -- 172.21.15.27:0/1942849245 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91b8036070 con 0x7f91ac0039e0 2024-09-12T03:24:03.424 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:03.418+0000 7f91bdd9b640 1 -- 172.21.15.27:0/1942849245 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91b8031400 con 0x7f91ac0039e0 2024-09-12T03:24:03.424 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:03.418+0000 7f91bdd9b640 1 -- 172.21.15.27:0/1942849245 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91b8009690 con 0x7f91ac0039e0 2024-09-12T03:24:03.424 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:03.419+0000 7f91c525a640 1 -- 172.21.15.27:0/1942849245 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f91ac0039e0 msgr2=0x7f91ac003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:03.424 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:03.419+0000 7f91c525a640 1 --2- 172.21.15.27:0/1942849245 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f91ac0039e0 0x7f91ac003dc0 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 crypto rx=0x7f91b8009ac0 tx=0x7f91ac005830 comp rx=0 tx=0).stop 2024-09-12T03:24:03.424 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:03.419+0000 7f91bf59e640 1 -- 172.21.15.27:0/1942849245 reap_dead start 2024-09-12T03:24:03.424 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:03.419+0000 7f91c525a640 1 -- 172.21.15.27:0/1942849245 shutdown_connections 2024-09-12T03:24:03.425 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:03.419+0000 7f91c525a640 1 -- 172.21.15.27:0/1942849245 >> 172.21.15.27:0/1942849245 conn(0x7f91c006dce0 msgr2=0x7f91c006e0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:03.425 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:03.419+0000 7f91c525a640 1 -- 172.21.15.27:0/1942849245 shutdown_connections 2024-09-12T03:24:03.425 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:03.419+0000 7f91c525a640 1 -- 172.21.15.27:0/1942849245 wait complete. 2024-09-12T03:24:03.425 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:03.420+0000 7f91c525a640 1 Processor -- start 2024-09-12T03:24:03.425 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:03.421+0000 7f91c525a640 1 -- start start 2024-09-12T03:24:03.427 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:03.421+0000 7f91c525a640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f91ac0039e0 0x7f91c00ca5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:03.427 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:03.421+0000 7f91c525a640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f91c00ac7e0 0x7f91c00acc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:03.427 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:03.421+0000 7f91c525a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f91c01f4950 0x7f91c01f4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:03.427 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:03.421+0000 7f91c525a640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f91c015ef70 con 0x7f91c00ac7e0 2024-09-12T03:24:03.427 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:03.421+0000 7f91c525a640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f91c00d42e0 con 0x7f91c01f4950 2024-09-12T03:24:03.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:03.421+0000 7f91c525a640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f91c015f0f0 con 0x7f91ac0039e0 2024-09-12T03:24:03.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:03.421+0000 7f91be59c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f91c00ac7e0 0x7f91c00acc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:03.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:03.421+0000 7f91be59c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f91c00ac7e0 0x7f91c00acc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33012/0 (socket says 172.21.15.27:33012) 2024-09-12T03:24:03.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:03.421+0000 7f91be59c640 1 -- 172.21.15.27:0/3173521459 learned_addr learned my addr 172.21.15.27:0/3173521459 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:24:03.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:03.421+0000 7f91bed9d640 1 --2- 172.21.15.27:0/3173521459 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f91ac0039e0 0x7f91c00ca5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:03.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:03.421+0000 7f91bf59e640 1 --2- 172.21.15.27:0/3173521459 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f91c01f4950 0x7f91c01f4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:03.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:03.422+0000 7f91be59c640 1 -- 172.21.15.27:0/3173521459 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f91ac0039e0 msgr2=0x7f91c00ca5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:03.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:03.422+0000 7f91be59c640 1 --2- 172.21.15.27:0/3173521459 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f91ac0039e0 0x7f91c00ca5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:03.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:03.422+0000 7f91be59c640 1 -- 172.21.15.27:0/3173521459 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f91c01f4950 msgr2=0x7f91c01f4dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:03.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:03.422+0000 7f91be59c640 1 --2- 172.21.15.27:0/3173521459 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f91c01f4950 0x7f91c01f4dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:03.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:03.422+0000 7f91be59c640 1 -- 172.21.15.27:0/3173521459 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91c00ab2b0 con 0x7f91c00ac7e0 2024-09-12T03:24:03.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:03.422+0000 7f91bf59e640 1 --2- 172.21.15.27:0/3173521459 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f91c01f4950 0x7f91c01f4dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:24:03.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:03.422+0000 7f91be59c640 1 --2- 172.21.15.27:0/3173521459 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f91c00ac7e0 0x7f91c00acc60 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7f91a800e820 tx=0x7f91a800ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:03.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:03.422+0000 7f91b77fe640 1 -- 172.21.15.27:0/3173521459 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91a800c8d0 con 0x7f91c00ac7e0 2024-09-12T03:24:03.429 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:03.422+0000 7f91b77fe640 1 -- 172.21.15.27:0/3173521459 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91a8018070 con 0x7f91c00ac7e0 2024-09-12T03:24:03.429 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:03.422+0000 7f91c525a640 1 -- 172.21.15.27:0/3173521459 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91c00ab590 con 0x7f91c00ac7e0 2024-09-12T03:24:03.429 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:03.422+0000 7f91b77fe640 1 -- 172.21.15.27:0/3173521459 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91a800ccd0 con 0x7f91c00ac7e0 2024-09-12T03:24:03.429 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:03.422+0000 7f91c525a640 1 -- 172.21.15.27:0/3173521459 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91c00abb50 con 0x7f91c00ac7e0 2024-09-12T03:24:03.429 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:03.423+0000 7f91c525a640 1 -- 172.21.15.27:0/3173521459 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91c00d1010 con 0x7f91c00ac7e0 2024-09-12T03:24:03.429 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:03.424+0000 7f91b77fe640 1 -- 172.21.15.27:0/3173521459 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 25) ==== 91365+0+0 (secure 0 0 0) 0x7f91a801d460 con 0x7f91c00ac7e0 2024-09-12T03:24:03.429 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:03.424+0000 7f91b77fe640 1 --2- 172.21.15.27:0/3173521459 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f91a006c7c0 0x7f91a006ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:03.429 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:03.424+0000 7f91b77fe640 1 -- 172.21.15.27:0/3173521459 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(160..160 src has 1..160) ==== 5765+0+0 (secure 0 0 0) 0x7f91a80893e0 con 0x7f91c00ac7e0 2024-09-12T03:24:03.429 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:03.424+0000 7f91bed9d640 1 --2- 172.21.15.27:0/3173521459 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f91a006c7c0 0x7f91a006ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:03.429 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:03.425+0000 7f91bed9d640 1 --2- 172.21.15.27:0/3173521459 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f91a006c7c0 0x7f91a006ec80 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f91b8009ac0 tx=0x7f91b8008030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:03.430 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:03.428+0000 7f91b77fe640 1 -- 172.21.15.27:0/3173521459 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f91a8059990 con 0x7f91c00ac7e0 2024-09-12T03:24:03.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:03.624+0000 7f91c525a640 1 -- 172.21.15.27:0/3173521459 --> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f91c01f5310 con 0x7f91a006c7c0 2024-09-12T03:24:03.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:03.626+0000 7f91b77fe640 1 -- 172.21.15.27:0/3173521459 <== mgr.4113 v2:172.21.15.27:6836/1226920526 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f91c01f5310 con 0x7f91a006c7c0 2024-09-12T03:24:03.627 INFO:teuthology.orchestra.run.smithi027.stdout:[ 2024-09-12T03:24:03.627 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-12T03:24:03.627 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "cephfs" 2024-09-12T03:24:03.627 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-12T03:24:03.627 INFO:teuthology.orchestra.run.smithi027.stdout:] 2024-09-12T03:24:03.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:03.628+0000 7f91c525a640 1 -- 172.21.15.27:0/3173521459 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f91a006c7c0 msgr2=0x7f91a006ec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:03.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:03.628+0000 7f91c525a640 1 --2- 172.21.15.27:0/3173521459 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f91a006c7c0 0x7f91a006ec80 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f91b8009ac0 tx=0x7f91b8008030 comp rx=0 tx=0).stop 2024-09-12T03:24:03.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:03.628+0000 7f91c525a640 1 -- 172.21.15.27:0/3173521459 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f91c00ac7e0 msgr2=0x7f91c00acc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:03.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:03.628+0000 7f91c525a640 1 --2- 172.21.15.27:0/3173521459 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f91c00ac7e0 0x7f91c00acc60 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7f91a800e820 tx=0x7f91a800ece0 comp rx=0 tx=0).stop 2024-09-12T03:24:03.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:03.628+0000 7f91c525a640 1 -- 172.21.15.27:0/3173521459 shutdown_connections 2024-09-12T03:24:03.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:03.628+0000 7f91c525a640 1 --2- 172.21.15.27:0/3173521459 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f91c01f4950 0x7f91c01f4dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:03.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:03.628+0000 7f91c525a640 1 --2- 172.21.15.27:0/3173521459 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f91a006c7c0 0x7f91a006ec80 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:03.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:03.628+0000 7f91c525a640 1 --2- 172.21.15.27:0/3173521459 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f91c00ac7e0 0x7f91c00acc60 unknown :-1 s=CLOSED pgs=1711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:03.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:03.628+0000 7f91c525a640 1 --2- 172.21.15.27:0/3173521459 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f91ac0039e0 0x7f91c00ca5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:03.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:03.628+0000 7f91c525a640 1 -- 172.21.15.27:0/3173521459 >> 172.21.15.27:0/3173521459 conn(0x7f91c006dce0 msgr2=0x7f91c00b9fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:03.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:03.628+0000 7f91c525a640 1 -- 172.21.15.27:0/3173521459 shutdown_connections 2024-09-12T03:24:03.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:03.628+0000 7f91c525a640 1 -- 172.21.15.27:0/3173521459 wait complete. 2024-09-12T03:24:03.640 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-12T03:24:03.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:03.745+0000 7f3a8aa82640 1 Processor -- start 2024-09-12T03:24:03.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:03.745+0000 7f3a8aa82640 1 -- start start 2024-09-12T03:24:03.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:03.746+0000 7f3a8aa82640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f3a8415a270 0x7f3a8406cff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:03.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:03.746+0000 7f3a8aa82640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f3a840cfb20 0x7f3a840726a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:03.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:03.746+0000 7f3a8aa82640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3a840ce650 con 0x7f3a8406d530 2024-09-12T03:24:03.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:03.746+0000 7f3a8aa82640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f3a840cef80 con 0x7f3a8415a270 2024-09-12T03:24:03.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:03.746+0000 7f3a8aa82640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f3a840ce930 con 0x7f3a840cfb20 2024-09-12T03:24:03.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:03.746+0000 7f3a8881e640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f3a8415a270 0x7f3a8406cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:03.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:03.746+0000 7f3a83fff640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f3a8406d530 0x7f3a840cf410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:57504/0 (socket says 172.21.15.27:57504) 2024-09-12T03:24:03.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:03.746+0000 7f3a83fff640 1 -- 172.21.15.27:0/1459872044 learned_addr learned my addr 172.21.15.27:0/1459872044 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:24:03.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:03.746+0000 7f3a8901f640 1 --2- 172.21.15.27:0/1459872044 >> v2:172.21.15.45:3300/0 conn(0x7f3a840cfb20 0x7f3a840726a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:03.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:03.746+0000 7f3a8881e640 1 -- 172.21.15.27:0/1459872044 >> v2:172.21.15.45:3300/0 conn(0x7f3a840cfb20 msgr2=0x7f3a840726a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:24:03.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:03.746+0000 7f3a8881e640 1 --2- 172.21.15.27:0/1459872044 >> v2:172.21.15.45:3300/0 conn(0x7f3a840cfb20 0x7f3a840726a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:03.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:03.746+0000 7f3a8881e640 1 -- 172.21.15.27:0/1459872044 >> v1:172.21.15.27:6789/0 conn(0x7f3a8406d530 legacy=0x7f3a840cf410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:03.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:03.746+0000 7f3a8881e640 1 -- 172.21.15.27:0/1459872044 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a840cec10 con 0x7f3a8415a270 2024-09-12T03:24:03.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:03.746+0000 7f3a837fe640 1 -- 172.21.15.27:0/1459872044 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1783975292 0 0) 0x7f3a840ce650 con 0x7f3a8406d530 2024-09-12T03:24:03.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:03.747+0000 7f3a8881e640 1 --2- 172.21.15.27:0/1459872044 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3a8415a270 0x7f3a8406cff0 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7f3a8406f240 tx=0x7f3a7c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e4916a8c7b9baa1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:03.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:03.747+0000 7f3a837fe640 1 -- 172.21.15.27:0/1459872044 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a7c037070 con 0x7f3a8415a270 2024-09-12T03:24:03.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:03.747+0000 7f3a837fe640 1 -- 172.21.15.27:0/1459872044 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a7c032440 con 0x7f3a8415a270 2024-09-12T03:24:03.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:03.747+0000 7f3a837fe640 1 -- 172.21.15.27:0/1459872044 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a7c032810 con 0x7f3a8415a270 2024-09-12T03:24:03.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:03.747+0000 7f3a8aa82640 1 -- 172.21.15.27:0/1459872044 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3a8415a270 msgr2=0x7f3a8406cff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:03.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:03.747+0000 7f3a8aa82640 1 --2- 172.21.15.27:0/1459872044 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3a8415a270 0x7f3a8406cff0 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7f3a8406f240 tx=0x7f3a7c02a1a0 comp rx=0 tx=0).stop 2024-09-12T03:24:03.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:03.747+0000 7f3a8aa82640 1 -- 172.21.15.27:0/1459872044 shutdown_connections 2024-09-12T03:24:03.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:03.747+0000 7f3a8aa82640 1 --2- 172.21.15.27:0/1459872044 >> v2:172.21.15.45:3300/0 conn(0x7f3a840cfb20 0x7f3a840726a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:03.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:03.747+0000 7f3a8aa82640 1 --2- 172.21.15.27:0/1459872044 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3a8415a270 0x7f3a8406cff0 unknown :-1 s=CLOSED pgs=1713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:03.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:03.747+0000 7f3a8aa82640 1 -- 172.21.15.27:0/1459872044 >> 172.21.15.27:0/1459872044 conn(0x7f3a8406bb30 msgr2=0x7f3a8406bf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:03.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:03.748+0000 7f3a8aa82640 1 -- 172.21.15.27:0/1459872044 shutdown_connections 2024-09-12T03:24:03.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:03.748+0000 7f3a8aa82640 1 -- 172.21.15.27:0/1459872044 wait complete. 2024-09-12T03:24:03.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:03.749+0000 7f3a8aa82640 1 Processor -- start 2024-09-12T03:24:03.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:03.749+0000 7f3a8aa82640 1 -- start start 2024-09-12T03:24:03.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:03.750+0000 7f3a8aa82640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3a8406d530 0x7f3a840aa4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:03.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:03.750+0000 7f3a8aa82640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3a840cfb20 0x7f3a841609b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:03.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:03.750+0000 7f3a8aa82640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f3a8415a270 0x7f3a84155d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:03.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:03.750+0000 7f3a8aa82640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f3a8406c7d0 con 0x7f3a840cfb20 2024-09-12T03:24:03.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:03.750+0000 7f3a8aa82640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f3a8406c650 con 0x7f3a8406d530 2024-09-12T03:24:03.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:03.750+0000 7f3a8aa82640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f3a8406c950 con 0x7f3a8415a270 2024-09-12T03:24:03.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:03.750+0000 7f3a83fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3a840cfb20 0x7f3a841609b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:03.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:03.750+0000 7f3a83fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3a840cfb20 0x7f3a841609b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33026/0 (socket says 172.21.15.27:33026) 2024-09-12T03:24:03.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:03.750+0000 7f3a83fff640 1 -- 172.21.15.27:0/4087360947 learned_addr learned my addr 172.21.15.27:0/4087360947 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:24:03.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:03.750+0000 7f3a8901f640 1 --2- 172.21.15.27:0/4087360947 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f3a8415a270 0x7f3a84155d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:03.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:03.750+0000 7f3a8881e640 1 --2- 172.21.15.27:0/4087360947 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3a8406d530 0x7f3a840aa4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:03.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:03.750+0000 7f3a83fff640 1 -- 172.21.15.27:0/4087360947 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f3a8415a270 msgr2=0x7f3a84155d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:03.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:03.750+0000 7f3a83fff640 1 --2- 172.21.15.27:0/4087360947 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f3a8415a270 0x7f3a84155d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:03.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:03.750+0000 7f3a83fff640 1 -- 172.21.15.27:0/4087360947 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3a8406d530 msgr2=0x7f3a840aa4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:03.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:03.750+0000 7f3a83fff640 1 --2- 172.21.15.27:0/4087360947 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3a8406d530 0x7f3a840aa4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:03.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:03.750+0000 7f3a83fff640 1 -- 172.21.15.27:0/4087360947 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a840aab30 con 0x7f3a840cfb20 2024-09-12T03:24:03.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:03.750+0000 7f3a8881e640 1 --2- 172.21.15.27:0/4087360947 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3a8406d530 0x7f3a840aa4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:24:03.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:03.750+0000 7f3a83fff640 1 --2- 172.21.15.27:0/4087360947 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3a840cfb20 0x7f3a841609b0 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7f3a7400c820 tx=0x7f3a7400cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:03.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:03.750+0000 7f3a817fa640 1 -- 172.21.15.27:0/4087360947 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a74018070 con 0x7f3a840cfb20 2024-09-12T03:24:03.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:03.750+0000 7f3a817fa640 1 -- 172.21.15.27:0/4087360947 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a7400a640 con 0x7f3a840cfb20 2024-09-12T03:24:03.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:03.750+0000 7f3a8aa82640 1 -- 172.21.15.27:0/4087360947 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a840aae10 con 0x7f3a840cfb20 2024-09-12T03:24:03.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:03.750+0000 7f3a817fa640 1 -- 172.21.15.27:0/4087360947 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a7400aa30 con 0x7f3a840cfb20 2024-09-12T03:24:03.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:03.750+0000 7f3a8aa82640 1 -- 172.21.15.27:0/4087360947 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a840ab3d0 con 0x7f3a840cfb20 2024-09-12T03:24:03.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:03.751+0000 7f3a8aa82640 1 -- 172.21.15.27:0/4087360947 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a840b5900 con 0x7f3a840cfb20 2024-09-12T03:24:03.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:03.752+0000 7f3a817fa640 1 -- 172.21.15.27:0/4087360947 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 25) ==== 91365+0+0 (secure 0 0 0) 0x7f3a7400abd0 con 0x7f3a840cfb20 2024-09-12T03:24:03.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:03.752+0000 7f3a817fa640 1 --2- 172.21.15.27:0/4087360947 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f3a6406c7c0 0x7f3a6406ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:03.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:03.753+0000 7f3a8881e640 1 --2- 172.21.15.27:0/4087360947 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f3a6406c7c0 0x7f3a6406ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:03.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:03.753+0000 7f3a817fa640 1 -- 172.21.15.27:0/4087360947 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(160..160 src has 1..160) ==== 5765+0+0 (secure 0 0 0) 0x7f3a74088d50 con 0x7f3a840cfb20 2024-09-12T03:24:03.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:03.753+0000 7f3a8881e640 1 --2- 172.21.15.27:0/4087360947 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f3a6406c7c0 0x7f3a6406ec80 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f3a7c034000 tx=0x7f3a7c02e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:03.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:03.757+0000 7f3a817fa640 1 -- 172.21.15.27:0/4087360947 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f3a840b5900 con 0x7f3a840cfb20 2024-09-12T03:24:03.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:03.932+0000 7f3a8aa82640 1 -- 172.21.15.27:0/4087360947 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f3a8415d560 con 0x7f3a840cfb20 2024-09-12T03:24:03.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:03.932+0000 7f3a817fa640 1 -- 172.21.15.27:0/4087360947 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v30)=0 v30) ==== 122+0+0 (secure 0 0 0) 0x7f3a74059280 con 0x7f3a840cfb20 2024-09-12T03:24:03.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:03.934+0000 7f3a8aa82640 1 -- 172.21.15.27:0/4087360947 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f3a6406c7c0 msgr2=0x7f3a6406ec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:03.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:03.934+0000 7f3a8aa82640 1 --2- 172.21.15.27:0/4087360947 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f3a6406c7c0 0x7f3a6406ec80 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f3a7c034000 tx=0x7f3a7c02e000 comp rx=0 tx=0).stop 2024-09-12T03:24:03.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:03.934+0000 7f3a8aa82640 1 -- 172.21.15.27:0/4087360947 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3a840cfb20 msgr2=0x7f3a841609b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:03.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:03.934+0000 7f3a8aa82640 1 --2- 172.21.15.27:0/4087360947 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3a840cfb20 0x7f3a841609b0 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7f3a7400c820 tx=0x7f3a7400cce0 comp rx=0 tx=0).stop 2024-09-12T03:24:03.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:03.934+0000 7f3a8aa82640 1 -- 172.21.15.27:0/4087360947 shutdown_connections 2024-09-12T03:24:03.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:03.934+0000 7f3a8aa82640 1 --2- 172.21.15.27:0/4087360947 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f3a8415a270 0x7f3a84155d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:03.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:03.934+0000 7f3a8aa82640 1 --2- 172.21.15.27:0/4087360947 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f3a6406c7c0 0x7f3a6406ec80 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:03.936 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:03.934+0000 7f3a8aa82640 1 --2- 172.21.15.27:0/4087360947 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3a840cfb20 0x7f3a841609b0 unknown :-1 s=CLOSED pgs=1714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:03.936 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:03.934+0000 7f3a8aa82640 1 --2- 172.21.15.27:0/4087360947 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3a8406d530 0x7f3a840aa4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:03.936 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:03.934+0000 7f3a8aa82640 1 -- 172.21.15.27:0/4087360947 >> 172.21.15.27:0/4087360947 conn(0x7f3a8406bb30 msgr2=0x7f3a84072660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:03.936 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:03.934+0000 7f3a8aa82640 1 -- 172.21.15.27:0/4087360947 shutdown_connections 2024-09-12T03:24:03.936 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:03.934+0000 7f3a8aa82640 1 -- 172.21.15.27:0/4087360947 wait complete. 2024-09-12T03:24:03.946 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:24:03.946 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p ./volumes/subvol_grp31770/subvol8650) 2024-09-12T03:24:03.977 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:24:03.977 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.layout.pool .) 2024-09-12T03:24:04.034 INFO:teuthology.orchestra.run.smithi027.stdout:cephfs_data 2024-09-12T03:24:04.035 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:24:04.035 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo setfattr -n ceph.dir.layout.pool -v cephfs_data ./volumes/subvol_grp31770/subvol8650) 2024-09-12T03:24:04.105 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-12T03:24:04.106 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd of=/home/ubuntu/cephtest/mnt.0/./volumes/subvol_grp31770/subvol8650/.meta 2024-09-12T03:24:04.174 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol8650 subvol_grp31770 2024-09-12T03:24:04.322 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:04.321+0000 7f4f8a074640 1 Processor -- start 2024-09-12T03:24:04.322 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:04.321+0000 7f4f8a074640 1 -- start start 2024-09-12T03:24:04.323 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:04.322+0000 7f4f8a074640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4f840d31c0 0x7f4f840d0440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:04.323 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:04.322+0000 7f4f8a074640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f4f8406e960 0x7f4f840d0980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:04.323 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:04.322+0000 7f4f8a074640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4f840cf9f0 con 0x7f4f84070d30 2024-09-12T03:24:04.324 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:04.322+0000 7f4f8a074640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f4f840d1a40 con 0x7f4f8406e960 2024-09-12T03:24:04.324 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:04.322+0000 7f4f8a074640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f4f840cfc90 con 0x7f4f840d31c0 2024-09-12T03:24:04.324 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:04.322+0000 7f4f837fe640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f4f84070d30 0x7f4f8406cc40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:57518/0 (socket says 172.21.15.27:57518) 2024-09-12T03:24:04.324 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:04.322+0000 7f4f837fe640 1 -- 172.21.15.27:0/3351834078 learned_addr learned my addr 172.21.15.27:0/3351834078 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:24:04.324 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:04.322+0000 7f4f83fff640 1 --2- 172.21.15.27:0/3351834078 >> v2:172.21.15.27:3300/0 conn(0x7f4f8406e960 0x7f4f840d0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:04.324 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:04.322+0000 7f4f82ffd640 1 --2- 172.21.15.27:0/3351834078 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4f840d31c0 0x7f4f840d0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:04.324 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:04.322+0000 7f4f827fc640 1 -- 172.21.15.27:0/3351834078 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2353673894 0 0) 0x7f4f840cf9f0 con 0x7f4f84070d30 2024-09-12T03:24:04.324 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:04.322+0000 7f4f827fc640 1 -- 172.21.15.27:0/3351834078 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4f6c003520 con 0x7f4f84070d30 2024-09-12T03:24:04.324 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:04.322+0000 7f4f83fff640 1 -- 172.21.15.27:0/3351834078 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4f840d31c0 msgr2=0x7f4f840d0440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:24:04.324 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:04.322+0000 7f4f83fff640 1 --2- 172.21.15.27:0/3351834078 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4f840d31c0 0x7f4f840d0440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:04.324 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:04.322+0000 7f4f83fff640 1 -- 172.21.15.27:0/3351834078 >> v1:172.21.15.27:6789/0 conn(0x7f4f84070d30 legacy=0x7f4f8406cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:04.324 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:04.322+0000 7f4f83fff640 1 -- 172.21.15.27:0/3351834078 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f840cff70 con 0x7f4f8406e960 2024-09-12T03:24:04.326 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:04.323+0000 7f4f83fff640 1 --2- 172.21.15.27:0/3351834078 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4f8406e960 0x7f4f840d0980 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7f4f7c009dc0 tx=0x7f4f7c02a560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17b2da455f65028d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:04.326 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:04.323+0000 7f4f827fc640 1 -- 172.21.15.27:0/3351834078 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f7c007740 con 0x7f4f8406e960 2024-09-12T03:24:04.326 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:04.323+0000 7f4f827fc640 1 -- 172.21.15.27:0/3351834078 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f7c0078e0 con 0x7f4f8406e960 2024-09-12T03:24:04.326 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:04.323+0000 7f4f827fc640 1 -- 172.21.15.27:0/3351834078 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f7c007c90 con 0x7f4f8406e960 2024-09-12T03:24:04.326 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:04.324+0000 7f4f8a074640 1 -- 172.21.15.27:0/3351834078 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4f8406e960 msgr2=0x7f4f840d0980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:04.326 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:04.324+0000 7f4f8a074640 1 --2- 172.21.15.27:0/3351834078 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4f8406e960 0x7f4f840d0980 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7f4f7c009dc0 tx=0x7f4f7c02a560 comp rx=0 tx=0).stop 2024-09-12T03:24:04.326 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:04.324+0000 7f4f8a074640 1 -- 172.21.15.27:0/3351834078 shutdown_connections 2024-09-12T03:24:04.326 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:04.324+0000 7f4f8a074640 1 --2- 172.21.15.27:0/3351834078 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4f8406e960 0x7f4f840d0980 unknown :-1 s=CLOSED pgs=1716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:04.326 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:04.324+0000 7f4f8a074640 1 --2- 172.21.15.27:0/3351834078 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4f840d31c0 0x7f4f840d0440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:04.327 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:04.324+0000 7f4f8a074640 1 -- 172.21.15.27:0/3351834078 >> 172.21.15.27:0/3351834078 conn(0x7f4f840b7c10 msgr2=0x7f4f8409ba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:04.327 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:04.324+0000 7f4f8a074640 1 -- 172.21.15.27:0/3351834078 shutdown_connections 2024-09-12T03:24:04.327 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:04.324+0000 7f4f8a074640 1 -- 172.21.15.27:0/3351834078 wait complete. 2024-09-12T03:24:04.327 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:04.325+0000 7f4f8a074640 1 Processor -- start 2024-09-12T03:24:04.327 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:04.325+0000 7f4f8a074640 1 -- start start 2024-09-12T03:24:04.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:04.326+0000 7f4f8a074640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4f8406e960 0x7f4f840cc920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:04.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:04.326+0000 7f4f8a074640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4f84070d30 0x7f4f8414ff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:04.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:04.326+0000 7f4f8a074640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4f840d31c0 0x7f4f84155310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:04.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:04.326+0000 7f4f8a074640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f4f84168480 con 0x7f4f8406e960 2024-09-12T03:24:04.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:04.326+0000 7f4f8a074640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f4f84168300 con 0x7f4f840d31c0 2024-09-12T03:24:04.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:04.326+0000 7f4f8a074640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f4f84168600 con 0x7f4f84070d30 2024-09-12T03:24:04.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:04.326+0000 7f4f837fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4f8406e960 0x7f4f840cc920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:04.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:04.326+0000 7f4f837fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4f8406e960 0x7f4f840cc920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33036/0 (socket says 172.21.15.27:33036) 2024-09-12T03:24:04.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:04.326+0000 7f4f837fe640 1 -- 172.21.15.27:0/1922918950 learned_addr learned my addr 172.21.15.27:0/1922918950 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:24:04.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:04.326+0000 7f4f82ffd640 1 --2- 172.21.15.27:0/1922918950 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4f84070d30 0x7f4f8414ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:04.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:04.326+0000 7f4f83fff640 1 --2- 172.21.15.27:0/1922918950 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4f840d31c0 0x7f4f84155310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:04.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:04.326+0000 7f4f837fe640 1 -- 172.21.15.27:0/1922918950 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4f84070d30 msgr2=0x7f4f8414ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:04.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:04.327+0000 7f4f837fe640 1 --2- 172.21.15.27:0/1922918950 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4f84070d30 0x7f4f8414ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:04.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:04.327+0000 7f4f837fe640 1 -- 172.21.15.27:0/1922918950 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4f840d31c0 msgr2=0x7f4f84155310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:04.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:04.327+0000 7f4f837fe640 1 --2- 172.21.15.27:0/1922918950 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4f840d31c0 0x7f4f84155310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:04.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:04.327+0000 7f4f837fe640 1 -- 172.21.15.27:0/1922918950 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f841504d0 con 0x7f4f8406e960 2024-09-12T03:24:04.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:04.327+0000 7f4f83fff640 1 --2- 172.21.15.27:0/1922918950 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4f840d31c0 0x7f4f84155310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:24:04.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:04.327+0000 7f4f837fe640 1 --2- 172.21.15.27:0/1922918950 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4f8406e960 0x7f4f840cc920 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7f4f740010e0 tx=0x7f4f7400c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:04.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:04.327+0000 7f4f6bfff640 1 -- 172.21.15.27:0/1922918950 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f74017070 con 0x7f4f8406e960 2024-09-12T03:24:04.331 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:04.327+0000 7f4f6bfff640 1 -- 172.21.15.27:0/1922918950 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f74005d00 con 0x7f4f8406e960 2024-09-12T03:24:04.331 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:04.327+0000 7f4f8a074640 1 -- 172.21.15.27:0/1922918950 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f840ccfe0 con 0x7f4f8406e960 2024-09-12T03:24:04.331 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:04.327+0000 7f4f6bfff640 1 -- 172.21.15.27:0/1922918950 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f740072b0 con 0x7f4f8406e960 2024-09-12T03:24:04.331 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:04.327+0000 7f4f8a074640 1 -- 172.21.15.27:0/1922918950 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f840cd5a0 con 0x7f4f8406e960 2024-09-12T03:24:04.331 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:04.328+0000 7f4f8a074640 1 -- 172.21.15.27:0/1922918950 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f8415c5f0 con 0x7f4f8406e960 2024-09-12T03:24:04.331 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:04.328+0000 7f4f6bfff640 1 -- 172.21.15.27:0/1922918950 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 25) ==== 91365+0+0 (secure 0 0 0) 0x7f4f7401a070 con 0x7f4f8406e960 2024-09-12T03:24:04.331 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:04.329+0000 7f4f6bfff640 1 --2- 172.21.15.27:0/1922918950 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f4f7006c7c0 0x7f4f7006ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:04.332 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:04.329+0000 7f4f82ffd640 1 --2- 172.21.15.27:0/1922918950 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f4f7006c7c0 0x7f4f7006ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:04.332 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:04.329+0000 7f4f6bfff640 1 -- 172.21.15.27:0/1922918950 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(160..160 src has 1..160) ==== 5765+0+0 (secure 0 0 0) 0x7f4f74088a00 con 0x7f4f8406e960 2024-09-12T03:24:04.332 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:04.330+0000 7f4f82ffd640 1 --2- 172.21.15.27:0/1922918950 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f4f7006c7c0 0x7f4f7006ec80 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f4f78006fd0 tx=0x7f4f78008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:04.334 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:04.334+0000 7f4f6bfff640 1 -- 172.21.15.27:0/1922918950 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f4f8415c5f0 con 0x7f4f8406e960 2024-09-12T03:24:04.539 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:04.539+0000 7f4f8a074640 1 -- 172.21.15.27:0/1922918950 --> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol8650", "group_name": "subvol_grp31770", "target": ["mon-mgr", ""]}) -- 0x7f4f840d1230 con 0x7f4f7006c7c0 2024-09-12T03:24:04.545 INFO:tasks.ceph.mgr.y.smithi027.stderr:2024-09-12T03:24:04.544+0000 7f325d8e3640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:24:04.545 INFO:tasks.ceph.mgr.y.smithi027.stderr:2024-09-12T03:24:04.544+0000 7f325d8e3640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:24:04.545 INFO:tasks.ceph.mgr.y.smithi027.stderr:2024-09-12T03:24:04.544+0000 7f325d8e3640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:24:04.545 INFO:tasks.ceph.mgr.y.smithi027.stderr:2024-09-12T03:24:04.544+0000 7f325d8e3640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:24:04.545 INFO:tasks.ceph.mgr.y.smithi027.stderr:2024-09-12T03:24:04.544+0000 7f325d8e3640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:24:04.577 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:04.576+0000 7f4f6bfff640 1 -- 172.21.15.27:0/1922918950 <== mgr.4113 v2:172.21.15.27:6836/1226920526 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+36 (secure 0 0 0) 0x7f4f840d1230 con 0x7f4f7006c7c0 2024-09-12T03:24:04.577 INFO:teuthology.orchestra.run.smithi027.stdout:/volumes/subvol_grp31770/subvol8650 2024-09-12T03:24:04.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:04.579+0000 7f4f8a074640 1 -- 172.21.15.27:0/1922918950 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f4f7006c7c0 msgr2=0x7f4f7006ec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:04.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:04.579+0000 7f4f8a074640 1 --2- 172.21.15.27:0/1922918950 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f4f7006c7c0 0x7f4f7006ec80 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f4f78006fd0 tx=0x7f4f78008040 comp rx=0 tx=0).stop 2024-09-12T03:24:04.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:04.579+0000 7f4f8a074640 1 -- 172.21.15.27:0/1922918950 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4f8406e960 msgr2=0x7f4f840cc920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:04.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:04.579+0000 7f4f8a074640 1 --2- 172.21.15.27:0/1922918950 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4f8406e960 0x7f4f840cc920 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7f4f740010e0 tx=0x7f4f7400c960 comp rx=0 tx=0).stop 2024-09-12T03:24:04.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:04.579+0000 7f4f8a074640 1 -- 172.21.15.27:0/1922918950 shutdown_connections 2024-09-12T03:24:04.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:04.579+0000 7f4f8a074640 1 --2- 172.21.15.27:0/1922918950 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4f840d31c0 0x7f4f84155310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:04.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:04.579+0000 7f4f8a074640 1 --2- 172.21.15.27:0/1922918950 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4f84070d30 0x7f4f8414ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:04.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:04.579+0000 7f4f8a074640 1 --2- 172.21.15.27:0/1922918950 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f4f7006c7c0 0x7f4f7006ec80 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:04.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:04.579+0000 7f4f8a074640 1 --2- 172.21.15.27:0/1922918950 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4f8406e960 0x7f4f840cc920 unknown :-1 s=CLOSED pgs=1717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:04.581 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:04.579+0000 7f4f8a074640 1 -- 172.21.15.27:0/1922918950 >> 172.21.15.27:0/1922918950 conn(0x7f4f840b7c10 msgr2=0x7f4f840b0f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:04.581 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:04.579+0000 7f4f8a074640 1 -- 172.21.15.27:0/1922918950 shutdown_connections 2024-09-12T03:24:04.581 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:04.579+0000 7f4f8a074640 1 -- 172.21.15.27:0/1922918950 wait complete. 2024-09-12T03:24:04.591 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol8650 subvol_grp31770 2024-09-12T03:24:04.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:04.696+0000 7f2e8e447640 1 Processor -- start 2024-09-12T03:24:04.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:04.696+0000 7f2e8e447640 1 -- start start 2024-09-12T03:24:04.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:04.697+0000 7f2e8e447640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f2e880af9d0 0x7f2e880b4010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:04.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:04.697+0000 7f2e8e447640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2e8806da00 0x7f2e8806de90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:04.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:04.697+0000 7f2e8e447640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2e880ce5e0 con 0x7f2e8815ec30 2024-09-12T03:24:04.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:04.697+0000 7f2e8e447640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f2e880d02d0 con 0x7f2e880af9d0 2024-09-12T03:24:04.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:04.697+0000 7f2e8e447640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f2e8806c650 con 0x7f2e8806da00 2024-09-12T03:24:04.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:04.697+0000 7f2e87fff640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f2e8815ec30 0x7f2e880b3900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:57522/0 (socket says 172.21.15.27:57522) 2024-09-12T03:24:04.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:04.697+0000 7f2e87fff640 1 -- 172.21.15.27:0/26079017 learned_addr learned my addr 172.21.15.27:0/26079017 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:24:04.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:04.697+0000 7f2e7ffff640 1 --2- 172.21.15.27:0/26079017 >> v2:172.21.15.45:3300/0 conn(0x7f2e880af9d0 0x7f2e880b4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:04.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:04.697+0000 7f2e8c9e4640 1 --2- 172.21.15.27:0/26079017 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2e8806da00 0x7f2e8806de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:04.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:04.698+0000 7f2e877fe640 1 -- 172.21.15.27:0/26079017 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 215609939 0 0) 0x7f2e880ce5e0 con 0x7f2e8815ec30 2024-09-12T03:24:04.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:04.698+0000 7f2e877fe640 1 -- 172.21.15.27:0/26079017 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2e74003520 con 0x7f2e8815ec30 2024-09-12T03:24:04.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:04.698+0000 7f2e7ffff640 1 -- 172.21.15.27:0/26079017 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2e8806da00 msgr2=0x7f2e8806de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:24:04.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:04.698+0000 7f2e7ffff640 1 --2- 172.21.15.27:0/26079017 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2e8806da00 0x7f2e8806de90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:04.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:04.698+0000 7f2e7ffff640 1 -- 172.21.15.27:0/26079017 >> v1:172.21.15.27:6789/0 conn(0x7f2e8815ec30 legacy=0x7f2e880b3900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:04.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:04.698+0000 7f2e7ffff640 1 -- 172.21.15.27:0/26079017 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e880ceac0 con 0x7f2e880af9d0 2024-09-12T03:24:04.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:04.699+0000 7f2e7ffff640 1 --2- 172.21.15.27:0/26079017 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2e880af9d0 0x7f2e880b4010 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f2e8806c7d0 tx=0x7f2e7002a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=681b942234c7eadc server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:04.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:04.699+0000 7f2e877fe640 1 -- 172.21.15.27:0/26079017 <== mon.1 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e70037070 con 0x7f2e880af9d0 2024-09-12T03:24:04.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:04.699+0000 7f2e877fe640 1 -- 172.21.15.27:0/26079017 <== mon.1 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e70032440 con 0x7f2e880af9d0 2024-09-12T03:24:04.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:04.699+0000 7f2e877fe640 1 -- 172.21.15.27:0/26079017 <== mon.1 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e700327f0 con 0x7f2e880af9d0 2024-09-12T03:24:04.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:04.699+0000 7f2e8e447640 1 -- 172.21.15.27:0/26079017 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2e880af9d0 msgr2=0x7f2e880b4010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:04.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:04.699+0000 7f2e8e447640 1 --2- 172.21.15.27:0/26079017 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2e880af9d0 0x7f2e880b4010 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f2e8806c7d0 tx=0x7f2e7002a180 comp rx=0 tx=0).stop 2024-09-12T03:24:04.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:04.699+0000 7f2e8e447640 1 -- 172.21.15.27:0/26079017 shutdown_connections 2024-09-12T03:24:04.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:04.700+0000 7f2e8e447640 1 --2- 172.21.15.27:0/26079017 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2e8806da00 0x7f2e8806de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:04.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:04.700+0000 7f2e8e447640 1 --2- 172.21.15.27:0/26079017 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2e880af9d0 0x7f2e880b4010 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:04.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:04.700+0000 7f2e8e447640 1 -- 172.21.15.27:0/26079017 >> 172.21.15.27:0/26079017 conn(0x7f2e8806bb30 msgr2=0x7f2e8806bf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:04.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:04.700+0000 7f2e8e447640 1 -- 172.21.15.27:0/26079017 shutdown_connections 2024-09-12T03:24:04.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:04.700+0000 7f2e8e447640 1 -- 172.21.15.27:0/26079017 wait complete. 2024-09-12T03:24:04.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:04.701+0000 7f2e8e447640 1 Processor -- start 2024-09-12T03:24:04.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:04.701+0000 7f2e8e447640 1 -- start start 2024-09-12T03:24:04.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:04.701+0000 7f2e8e447640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2e8806da00 0x7f2e88151de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:04.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:04.701+0000 7f2e8e447640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2e880af9d0 0x7f2e8814e220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:04.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:04.701+0000 7f2e8e447640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2e8815ec30 0x7f2e881574c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:04.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:04.701+0000 7f2e8e447640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f2e880cfff0 con 0x7f2e8815ec30 2024-09-12T03:24:04.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:04.701+0000 7f2e8e447640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f2e8806c8a0 con 0x7f2e8806da00 2024-09-12T03:24:04.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:04.701+0000 7f2e8e447640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f2e880715f0 con 0x7f2e880af9d0 2024-09-12T03:24:04.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:04.702+0000 7f2e8c9e4640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2e8815ec30 0x7f2e881574c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:04.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:04.702+0000 7f2e8c9e4640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2e8815ec30 0x7f2e881574c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33070/0 (socket says 172.21.15.27:33070) 2024-09-12T03:24:04.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:04.702+0000 7f2e8c9e4640 1 -- 172.21.15.27:0/616777556 learned_addr learned my addr 172.21.15.27:0/616777556 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:24:04.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:04.702+0000 7f2e87fff640 1 --2- 172.21.15.27:0/616777556 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2e8806da00 0x7f2e88151de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:04.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:04.702+0000 7f2e7ffff640 1 --2- 172.21.15.27:0/616777556 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2e880af9d0 0x7f2e8814e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:04.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:04.702+0000 7f2e8c9e4640 1 -- 172.21.15.27:0/616777556 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2e880af9d0 msgr2=0x7f2e8814e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:04.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:04.702+0000 7f2e8c9e4640 1 --2- 172.21.15.27:0/616777556 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2e880af9d0 0x7f2e8814e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:04.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:04.702+0000 7f2e8c9e4640 1 -- 172.21.15.27:0/616777556 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2e8806da00 msgr2=0x7f2e88151de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:04.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:04.702+0000 7f2e8c9e4640 1 --2- 172.21.15.27:0/616777556 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2e8806da00 0x7f2e88151de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:04.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:04.702+0000 7f2e8c9e4640 1 -- 172.21.15.27:0/616777556 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e88150d80 con 0x7f2e8815ec30 2024-09-12T03:24:04.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:04.702+0000 7f2e8c9e4640 1 --2- 172.21.15.27:0/616777556 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2e8815ec30 0x7f2e881574c0 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7f2e80009d30 tx=0x7f2e8000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:04.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:04.702+0000 7f2e857fa640 1 -- 172.21.15.27:0/616777556 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e80018020 con 0x7f2e8815ec30 2024-09-12T03:24:04.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:04.702+0000 7f2e857fa640 1 -- 172.21.15.27:0/616777556 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e80014480 con 0x7f2e8815ec30 2024-09-12T03:24:04.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:04.702+0000 7f2e8e447640 1 -- 172.21.15.27:0/616777556 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e88151020 con 0x7f2e8815ec30 2024-09-12T03:24:04.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:04.702+0000 7f2e857fa640 1 -- 172.21.15.27:0/616777556 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e80014870 con 0x7f2e8815ec30 2024-09-12T03:24:04.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:04.702+0000 7f2e8e447640 1 -- 172.21.15.27:0/616777556 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e881515e0 con 0x7f2e8815ec30 2024-09-12T03:24:04.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:04.703+0000 7f2e8e447640 1 -- 172.21.15.27:0/616777556 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e8806de90 con 0x7f2e8815ec30 2024-09-12T03:24:04.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:04.704+0000 7f2e857fa640 1 -- 172.21.15.27:0/616777556 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 25) ==== 91365+0+0 (secure 0 0 0) 0x7f2e80005d10 con 0x7f2e8815ec30 2024-09-12T03:24:04.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:04.704+0000 7f2e857fa640 1 --2- 172.21.15.27:0/616777556 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f2e6806c7c0 0x7f2e6806ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:04.707 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:04.704+0000 7f2e857fa640 1 -- 172.21.15.27:0/616777556 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(160..160 src has 1..160) ==== 5765+0+0 (secure 0 0 0) 0x7f2e80089fa0 con 0x7f2e8815ec30 2024-09-12T03:24:04.707 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:04.704+0000 7f2e87fff640 1 --2- 172.21.15.27:0/616777556 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f2e6806c7c0 0x7f2e6806ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:04.707 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:04.705+0000 7f2e87fff640 1 --2- 172.21.15.27:0/616777556 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f2e6806c7c0 0x7f2e6806ec80 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f2e78005880 tx=0x7f2e78009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:04.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:04.709+0000 7f2e857fa640 1 -- 172.21.15.27:0/616777556 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f2e80022070 con 0x7f2e8815ec30 2024-09-12T03:24:04.912 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:04.912+0000 7f2e8e447640 1 -- 172.21.15.27:0/616777556 --> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol8650", "group_name": "subvol_grp31770", "target": ["mon-mgr", ""]}) -- 0x7f2e8806f240 con 0x7f2e6806c7c0 2024-09-12T03:24:04.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:04.915+0000 7f2e857fa640 1 -- 172.21.15.27:0/616777556 <== mgr.4113 v2:172.21.15.27:6836/1226920526 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+36 (secure 0 0 0) 0x7f2e8806f240 con 0x7f2e6806c7c0 2024-09-12T03:24:04.916 INFO:teuthology.orchestra.run.smithi027.stdout:/volumes/subvol_grp31770/subvol8650 2024-09-12T03:24:04.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:04.918+0000 7f2e8e447640 1 -- 172.21.15.27:0/616777556 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f2e6806c7c0 msgr2=0x7f2e6806ec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:04.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:04.918+0000 7f2e8e447640 1 --2- 172.21.15.27:0/616777556 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f2e6806c7c0 0x7f2e6806ec80 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f2e78005880 tx=0x7f2e78009040 comp rx=0 tx=0).stop 2024-09-12T03:24:04.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:04.918+0000 7f2e8e447640 1 -- 172.21.15.27:0/616777556 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2e8815ec30 msgr2=0x7f2e881574c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:04.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:04.918+0000 7f2e8e447640 1 --2- 172.21.15.27:0/616777556 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2e8815ec30 0x7f2e881574c0 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7f2e80009d30 tx=0x7f2e8000c490 comp rx=0 tx=0).stop 2024-09-12T03:24:04.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:04.918+0000 7f2e8e447640 1 -- 172.21.15.27:0/616777556 shutdown_connections 2024-09-12T03:24:04.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:04.918+0000 7f2e8e447640 1 --2- 172.21.15.27:0/616777556 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f2e6806c7c0 0x7f2e6806ec80 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:04.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:04.918+0000 7f2e8e447640 1 --2- 172.21.15.27:0/616777556 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2e8815ec30 0x7f2e881574c0 unknown :-1 s=CLOSED pgs=1721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:04.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:04.918+0000 7f2e8e447640 1 --2- 172.21.15.27:0/616777556 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2e880af9d0 0x7f2e8814e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:04.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:04.918+0000 7f2e8e447640 1 --2- 172.21.15.27:0/616777556 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2e8806da00 0x7f2e88151de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:04.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:04.918+0000 7f2e8e447640 1 -- 172.21.15.27:0/616777556 >> 172.21.15.27:0/616777556 conn(0x7f2e8806bb30 msgr2=0x7f2e880b4170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:04.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:04.918+0000 7f2e8e447640 1 -- 172.21.15.27:0/616777556 shutdown_connections 2024-09-12T03:24:04.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:04.918+0000 7f2e8e447640 1 -- 172.21.15.27:0/616777556 wait complete. 2024-09-12T03:24:04.933 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:24:04.933 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cat ./volumes/_legacy/7425e73e7c67bf9c726568002cd383cf.meta) 2024-09-12T03:24:04.962 INFO:teuthology.orchestra.run.smithi027.stdout:[GLOBAL] 2024-09-12T03:24:04.963 INFO:teuthology.orchestra.run.smithi027.stdout:version = 1 2024-09-12T03:24:04.963 INFO:teuthology.orchestra.run.smithi027.stdout:type = subvolume 2024-09-12T03:24:04.963 INFO:teuthology.orchestra.run.smithi027.stdout:path = /volumes/subvol_grp31770/subvol8650 2024-09-12T03:24:04.963 INFO:teuthology.orchestra.run.smithi027.stdout:state = complete 2024-09-12T03:24:04.963 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:24:04.964 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol8650 subvol_grp31770 2024-09-12T03:24:05.110 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:05.110+0000 7f1417c4c640 1 Processor -- start 2024-09-12T03:24:05.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:05.110+0000 7f1417c4c640 1 -- start start 2024-09-12T03:24:05.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:05.111+0000 7f1417c4c640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f141015c6c0 0x7f14100d00a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:05.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:05.111+0000 7f1417c4c640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f14100d1750 0x7f14100b6f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:05.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:05.111+0000 7f1417c4c640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f14100d0a80 con 0x7f14100d05e0 2024-09-12T03:24:05.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:05.111+0000 7f1417c4c640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f141015caa0 con 0x7f141015c6c0 2024-09-12T03:24:05.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:05.111+0000 7f1417c4c640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f14100b6510 con 0x7f14100d1750 2024-09-12T03:24:05.113 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:05.111+0000 7f14161e9640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f14100d1750 0x7f14100b6f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:05.113 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:05.111+0000 7f14159e8640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f141015c6c0 0x7f14100d00a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:05.113 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:05.111+0000 7f14151e7640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f14100d05e0 0x7f14100d1040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.27:51016/0 (socket says 172.21.15.27:51016) 2024-09-12T03:24:05.113 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:05.111+0000 7f14159e8640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f141015c6c0 0x7f14100d00a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.27:57508/0 (socket says 172.21.15.27:57508) 2024-09-12T03:24:05.113 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:05.111+0000 7f14151e7640 1 -- 172.21.15.27:0/388281624 learned_addr learned my addr 172.21.15.27:0/388281624 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:24:05.113 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:05.111+0000 7f14161e9640 1 -- 172.21.15.27:0/388281624 >> v2:172.21.15.45:3300/0 conn(0x7f141015c6c0 msgr2=0x7f14100d00a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:24:05.113 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:05.111+0000 7f14161e9640 1 --2- 172.21.15.27:0/388281624 >> v2:172.21.15.45:3300/0 conn(0x7f141015c6c0 0x7f14100d00a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:05.113 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:05.111+0000 7f14161e9640 1 -- 172.21.15.27:0/388281624 >> v1:172.21.15.45:6789/0 conn(0x7f14100d05e0 legacy=0x7f14100d1040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:05.113 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:05.111+0000 7f14161e9640 1 -- 172.21.15.27:0/388281624 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14100d0d60 con 0x7f14100d1750 2024-09-12T03:24:05.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:05.112+0000 7f14161e9640 1 --2- 172.21.15.27:0/388281624 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f14100d1750 0x7f14100b6f70 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f1408009d90 tx=0x7f140802a5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7290c764b3e3ce4f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:05.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:05.112+0000 7f14149e6640 1 -- 172.21.15.27:0/388281624 <== mon.2 v2:172.21.15.45:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1408031600 con 0x7f14100d1750 2024-09-12T03:24:05.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:05.112+0000 7f14149e6640 1 -- 172.21.15.27:0/388281624 <== mon.2 v2:172.21.15.45:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14080317a0 con 0x7f14100d1750 2024-09-12T03:24:05.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:05.112+0000 7f14149e6640 1 -- 172.21.15.27:0/388281624 <== mon.2 v2:172.21.15.45:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1408031b70 con 0x7f14100d1750 2024-09-12T03:24:05.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:05.113+0000 7f1417c4c640 1 -- 172.21.15.27:0/388281624 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f14100d1750 msgr2=0x7f14100b6f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:05.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:05.113+0000 7f1417c4c640 1 --2- 172.21.15.27:0/388281624 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f14100d1750 0x7f14100b6f70 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f1408009d90 tx=0x7f140802a5d0 comp rx=0 tx=0).stop 2024-09-12T03:24:05.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:05.113+0000 7f1417c4c640 1 -- 172.21.15.27:0/388281624 shutdown_connections 2024-09-12T03:24:05.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:05.113+0000 7f1417c4c640 1 --2- 172.21.15.27:0/388281624 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f14100d1750 0x7f14100b6f70 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:05.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:05.113+0000 7f1417c4c640 1 --2- 172.21.15.27:0/388281624 >> v2:172.21.15.45:3300/0 conn(0x7f141015c6c0 0x7f14100d00a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:05.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:05.113+0000 7f1417c4c640 1 -- 172.21.15.27:0/388281624 >> 172.21.15.27:0/388281624 conn(0x7f141006f320 msgr2=0x7f141006f700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:05.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:05.113+0000 7f1417c4c640 1 -- 172.21.15.27:0/388281624 shutdown_connections 2024-09-12T03:24:05.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:05.113+0000 7f1417c4c640 1 -- 172.21.15.27:0/388281624 wait complete. 2024-09-12T03:24:05.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:05.114+0000 7f1417c4c640 1 Processor -- start 2024-09-12T03:24:05.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:05.114+0000 7f1417c4c640 1 -- start start 2024-09-12T03:24:05.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:05.115+0000 7f1417c4c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f14100d05e0 0x7f1410153410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:05.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:05.115+0000 7f1417c4c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f14100d1750 0x7f141006cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:05.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:05.115+0000 7f1417c4c640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f141015c6c0 0x7f14100b2cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:05.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:05.115+0000 7f1417c4c640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f14100b4400 con 0x7f14100d05e0 2024-09-12T03:24:05.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:05.115+0000 7f1417c4c640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f14100b6a50 con 0x7f14100d1750 2024-09-12T03:24:05.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:05.115+0000 7f1417c4c640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f14100b4580 con 0x7f141015c6c0 2024-09-12T03:24:05.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:05.115+0000 7f14159e8640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f14100d05e0 0x7f1410153410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:05.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:05.115+0000 7f14159e8640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f14100d05e0 0x7f1410153410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33078/0 (socket says 172.21.15.27:33078) 2024-09-12T03:24:05.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:05.115+0000 7f14159e8640 1 -- 172.21.15.27:0/3875670218 learned_addr learned my addr 172.21.15.27:0/3875670218 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:24:05.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:05.115+0000 7f14161e9640 1 --2- 172.21.15.27:0/3875670218 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f141015c6c0 0x7f14100b2cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:05.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:05.116+0000 7f14151e7640 1 --2- 172.21.15.27:0/3875670218 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f14100d1750 0x7f141006cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:05.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:05.116+0000 7f14159e8640 1 -- 172.21.15.27:0/3875670218 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f141015c6c0 msgr2=0x7f14100b2cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:05.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:05.116+0000 7f14159e8640 1 --2- 172.21.15.27:0/3875670218 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f141015c6c0 0x7f14100b2cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:05.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:05.116+0000 7f14159e8640 1 -- 172.21.15.27:0/3875670218 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f14100d1750 msgr2=0x7f141006cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:05.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:05.116+0000 7f14159e8640 1 --2- 172.21.15.27:0/3875670218 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f14100d1750 0x7f141006cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:05.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:05.116+0000 7f14159e8640 1 -- 172.21.15.27:0/3875670218 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f141006d5b0 con 0x7f14100d05e0 2024-09-12T03:24:05.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:05.116+0000 7f14151e7640 1 --2- 172.21.15.27:0/3875670218 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f14100d1750 0x7f141006cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:24:05.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:05.116+0000 7f14159e8640 1 --2- 172.21.15.27:0/3875670218 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f14100d05e0 0x7f1410153410 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7f140c00e870 tx=0x7f140c00ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:05.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:05.116+0000 7f14067fc640 1 -- 172.21.15.27:0/3875670218 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f140c017070 con 0x7f14100d05e0 2024-09-12T03:24:05.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:05.116+0000 7f14067fc640 1 -- 172.21.15.27:0/3875670218 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f140c00c940 con 0x7f14100d05e0 2024-09-12T03:24:05.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:05.116+0000 7f1417c4c640 1 -- 172.21.15.27:0/3875670218 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f141006d890 con 0x7f14100d05e0 2024-09-12T03:24:05.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:05.116+0000 7f14067fc640 1 -- 172.21.15.27:0/3875670218 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f140c00cd30 con 0x7f14100d05e0 2024-09-12T03:24:05.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:05.116+0000 7f1417c4c640 1 -- 172.21.15.27:0/3875670218 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14100c89a0 con 0x7f14100d05e0 2024-09-12T03:24:05.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:05.117+0000 7f1417c4c640 1 -- 172.21.15.27:0/3875670218 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f141015e8d0 con 0x7f14100d05e0 2024-09-12T03:24:05.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:05.118+0000 7f14067fc640 1 -- 172.21.15.27:0/3875670218 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 25) ==== 91365+0+0 (secure 0 0 0) 0x7f140c01a070 con 0x7f14100d05e0 2024-09-12T03:24:05.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:05.118+0000 7f14067fc640 1 --2- 172.21.15.27:0/3875670218 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f13f406c7c0 0x7f13f406ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:05.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:05.118+0000 7f14067fc640 1 -- 172.21.15.27:0/3875670218 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(160..160 src has 1..160) ==== 5765+0+0 (secure 0 0 0) 0x7f140c0893c0 con 0x7f14100d05e0 2024-09-12T03:24:05.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:05.118+0000 7f14151e7640 1 --2- 172.21.15.27:0/3875670218 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f13f406c7c0 0x7f13f406ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:05.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:05.119+0000 7f14151e7640 1 --2- 172.21.15.27:0/3875670218 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f13f406c7c0 0x7f13f406ec80 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f14101599f0 tx=0x7f1400009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:05.123 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:05.123+0000 7f14067fc640 1 -- 172.21.15.27:0/3875670218 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f140c059320 con 0x7f14100d05e0 2024-09-12T03:24:05.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:05.327+0000 7f1417c4c640 1 -- 172.21.15.27:0/3875670218 --> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol8650", "group_name": "subvol_grp31770", "target": ["mon-mgr", ""]}) -- 0x7f14100b82a0 con 0x7f13f406c7c0 2024-09-12T03:24:05.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:05.339+0000 7f14067fc640 1 -- 172.21.15.27:0/3875670218 <== mgr.4113 v2:172.21.15.27:6836/1226920526 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f14100b82a0 con 0x7f13f406c7c0 2024-09-12T03:24:05.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:05.341+0000 7f1417c4c640 1 -- 172.21.15.27:0/3875670218 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f13f406c7c0 msgr2=0x7f13f406ec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:05.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:05.341+0000 7f1417c4c640 1 --2- 172.21.15.27:0/3875670218 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f13f406c7c0 0x7f13f406ec80 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f14101599f0 tx=0x7f1400009040 comp rx=0 tx=0).stop 2024-09-12T03:24:05.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:05.341+0000 7f1417c4c640 1 -- 172.21.15.27:0/3875670218 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f14100d05e0 msgr2=0x7f1410153410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:05.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:05.341+0000 7f1417c4c640 1 --2- 172.21.15.27:0/3875670218 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f14100d05e0 0x7f1410153410 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7f140c00e870 tx=0x7f140c00ed30 comp rx=0 tx=0).stop 2024-09-12T03:24:05.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:05.341+0000 7f1417c4c640 1 -- 172.21.15.27:0/3875670218 shutdown_connections 2024-09-12T03:24:05.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:05.341+0000 7f1417c4c640 1 --2- 172.21.15.27:0/3875670218 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f141015c6c0 0x7f14100b2cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:05.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:05.341+0000 7f1417c4c640 1 --2- 172.21.15.27:0/3875670218 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f14100d1750 0x7f141006cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:05.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:05.341+0000 7f1417c4c640 1 --2- 172.21.15.27:0/3875670218 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f13f406c7c0 0x7f13f406ec80 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:05.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:05.341+0000 7f1417c4c640 1 --2- 172.21.15.27:0/3875670218 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f14100d05e0 0x7f1410153410 unknown :-1 s=CLOSED pgs=1722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:05.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:05.341+0000 7f1417c4c640 1 -- 172.21.15.27:0/3875670218 >> 172.21.15.27:0/3875670218 conn(0x7f141006f320 msgr2=0x7f141015eaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:05.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:05.342+0000 7f1417c4c640 1 -- 172.21.15.27:0/3875670218 shutdown_connections 2024-09-12T03:24:05.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:05.342+0000 7f1417c4c640 1 -- 172.21.15.27:0/3875670218 wait complete. 2024-09-12T03:24:05.348 INFO:tasks.ceph.mgr.y.smithi027.stderr:2024-09-12T03:24:05.344+0000 7f32600e8640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:24:05.349 INFO:tasks.ceph.mgr.y.smithi027.stderr:2024-09-12T03:24:05.344+0000 7f32600e8640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:24:05.349 INFO:tasks.ceph.mgr.y.smithi027.stderr:2024-09-12T03:24:05.344+0000 7f32600e8640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:24:05.349 INFO:tasks.ceph.mgr.y.smithi027.stderr:2024-09-12T03:24:05.344+0000 7f32600e8640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:24:05.349 INFO:tasks.ceph.mgr.y.smithi027.stderr:2024-09-12T03:24:05.344+0000 7f32600e8640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:24:05.355 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-09-12T03:24:05.359 INFO:tasks.ceph.mgr.y.smithi027.stderr:2024-09-12T03:24:05.358+0000 7f325f0e6640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:24:05.359 INFO:tasks.ceph.mgr.y.smithi027.stderr:2024-09-12T03:24:05.358+0000 7f325f0e6640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:24:05.359 INFO:tasks.ceph.mgr.y.smithi027.stderr:2024-09-12T03:24:05.358+0000 7f325f0e6640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:24:05.359 INFO:tasks.ceph.mgr.y.smithi027.stderr:2024-09-12T03:24:05.358+0000 7f325f0e6640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:24:05.359 INFO:tasks.ceph.mgr.y.smithi027.stderr:2024-09-12T03:24:05.358+0000 7f325f0e6640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:24:05.407 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:05 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-12T03:24:05.408 INFO:teuthology.orchestra.run.smithi027.stdout:2 2024-09-12T03:24:05.408 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T03:24:05.414 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup rm cephfs subvol_grp31770 2024-09-12T03:24:05.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:05.560+0000 7f521eb66640 1 Processor -- start 2024-09-12T03:24:05.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:05.560+0000 7f521eb66640 1 -- start start 2024-09-12T03:24:05.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:05.560+0000 7f521eb66640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f5218162040 0x7f52180d1da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:05.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:05.560+0000 7f521eb66640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f52180d3790 0x7f52180b7a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:05.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:05.560+0000 7f521eb66640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52180d2e30 con 0x7f521806d8c0 2024-09-12T03:24:05.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:05.560+0000 7f521eb66640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f52180b7f90 con 0x7f52180d3790 2024-09-12T03:24:05.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:05.560+0000 7f521eb66640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f5218162420 con 0x7f5218162040 2024-09-12T03:24:05.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:05.560+0000 7f5217fff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f52180d3790 0x7f52180b7a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:05.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:05.561+0000 7f5217fff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f52180d3790 0x7f52180b7a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33126/0 (socket says 172.21.15.27:33126) 2024-09-12T03:24:05.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:05.561+0000 7f5217fff640 1 -- 172.21.15.27:0/1764661086 learned_addr learned my addr 172.21.15.27:0/1764661086 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:24:05.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:05.561+0000 7f521c902640 1 --2- 172.21.15.27:0/1764661086 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f5218162040 0x7f52180d1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:05.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:05.561+0000 7f5217fff640 1 -- 172.21.15.27:0/1764661086 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f5218162040 msgr2=0x7f52180d1da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:24:05.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:05.561+0000 7f5217fff640 1 --2- 172.21.15.27:0/1764661086 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f5218162040 0x7f52180d1da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:05.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:05.561+0000 7f5217fff640 1 -- 172.21.15.27:0/1764661086 >> v1:172.21.15.45:6789/0 conn(0x7f521806d8c0 legacy=0x7f52180cf320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:05.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:05.561+0000 7f5217fff640 1 -- 172.21.15.27:0/1764661086 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52180d30d0 con 0x7f52180d3790 2024-09-12T03:24:05.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:05.562+0000 7f5217fff640 1 --2- 172.21.15.27:0/1764661086 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f52180d3790 0x7f52180b7a50 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x7f520c009870 tx=0x7f520c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20f40f626a6c3d6d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:05.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:05.562+0000 7f52177fe640 1 -- 172.21.15.27:0/1764661086 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f520c037070 con 0x7f52180d3790 2024-09-12T03:24:05.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:05.562+0000 7f52177fe640 1 -- 172.21.15.27:0/1764661086 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f520c032440 con 0x7f52180d3790 2024-09-12T03:24:05.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:05.562+0000 7f52177fe640 1 -- 172.21.15.27:0/1764661086 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f520c032810 con 0x7f52180d3790 2024-09-12T03:24:05.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:05.562+0000 7f521eb66640 1 -- 172.21.15.27:0/1764661086 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f52180d3790 msgr2=0x7f52180b7a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:05.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:05.562+0000 7f521eb66640 1 --2- 172.21.15.27:0/1764661086 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f52180d3790 0x7f52180b7a50 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x7f520c009870 tx=0x7f520c02a270 comp rx=0 tx=0).stop 2024-09-12T03:24:05.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:05.562+0000 7f521eb66640 1 -- 172.21.15.27:0/1764661086 shutdown_connections 2024-09-12T03:24:05.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:05.562+0000 7f521eb66640 1 --2- 172.21.15.27:0/1764661086 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f52180d3790 0x7f52180b7a50 unknown :-1 s=CLOSED pgs=1727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:05.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:05.562+0000 7f521eb66640 1 --2- 172.21.15.27:0/1764661086 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f5218162040 0x7f52180d1da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:05.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:05.562+0000 7f521eb66640 1 -- 172.21.15.27:0/1764661086 >> 172.21.15.27:0/1764661086 conn(0x7f52180b6700 msgr2=0x7f52180b6b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:05.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:05.562+0000 7f521eb66640 1 -- 172.21.15.27:0/1764661086 shutdown_connections 2024-09-12T03:24:05.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:05.562+0000 7f521eb66640 1 -- 172.21.15.27:0/1764661086 wait complete. 2024-09-12T03:24:05.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:05.564+0000 7f521eb66640 1 Processor -- start 2024-09-12T03:24:05.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:05.564+0000 7f521eb66640 1 -- start start 2024-09-12T03:24:05.567 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:05.564+0000 7f521eb66640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f521806d8c0 0x7f521814fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:05.567 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:05.564+0000 7f521eb66640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f52180d3790 0x7f52180cd810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:05.567 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:05.564+0000 7f521eb66640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5218162040 0x7f52180cdd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:05.567 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:05.565+0000 7f521eb66640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f52180d4530 con 0x7f521806d8c0 2024-09-12T03:24:05.567 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:05.565+0000 7f521eb66640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f52180cee00 con 0x7f5218162040 2024-09-12T03:24:05.567 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:05.565+0000 7f521eb66640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f52180d4150 con 0x7f52180d3790 2024-09-12T03:24:05.567 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:05.565+0000 7f521c902640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f521806d8c0 0x7f521814fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:05.567 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:05.565+0000 7f521c902640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f521806d8c0 0x7f521814fc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33132/0 (socket says 172.21.15.27:33132) 2024-09-12T03:24:05.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:05.565+0000 7f521c902640 1 -- 172.21.15.27:0/4167957809 learned_addr learned my addr 172.21.15.27:0/4167957809 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:24:05.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:05.565+0000 7f5217fff640 1 --2- 172.21.15.27:0/4167957809 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f52180d3790 0x7f52180cd810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:05.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:05.565+0000 7f521d103640 1 --2- 172.21.15.27:0/4167957809 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5218162040 0x7f52180cdd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:05.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:05.565+0000 7f521c902640 1 -- 172.21.15.27:0/4167957809 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f52180d3790 msgr2=0x7f52180cd810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:05.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:05.565+0000 7f521c902640 1 --2- 172.21.15.27:0/4167957809 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f52180d3790 0x7f52180cd810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:05.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:05.565+0000 7f521c902640 1 -- 172.21.15.27:0/4167957809 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5218162040 msgr2=0x7f52180cdd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:05.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:05.565+0000 7f521c902640 1 --2- 172.21.15.27:0/4167957809 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5218162040 0x7f52180cdd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:05.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:05.565+0000 7f521c902640 1 -- 172.21.15.27:0/4167957809 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5218160a10 con 0x7f521806d8c0 2024-09-12T03:24:05.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:05.565+0000 7f521c902640 1 --2- 172.21.15.27:0/4167957809 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f521806d8c0 0x7f521814fc50 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7f520800e8a0 tx=0x7f520800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:05.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:05.565+0000 7f52157fa640 1 -- 172.21.15.27:0/4167957809 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5208017070 con 0x7f521806d8c0 2024-09-12T03:24:05.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:05.565+0000 7f52157fa640 1 -- 172.21.15.27:0/4167957809 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f520800c940 con 0x7f521806d8c0 2024-09-12T03:24:05.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:05.565+0000 7f521eb66640 1 -- 172.21.15.27:0/4167957809 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5218160cf0 con 0x7f521806d8c0 2024-09-12T03:24:05.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:05.565+0000 7f52157fa640 1 -- 172.21.15.27:0/4167957809 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f520800cd30 con 0x7f521806d8c0 2024-09-12T03:24:05.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:05.565+0000 7f521eb66640 1 -- 172.21.15.27:0/4167957809 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52181612b0 con 0x7f521806d8c0 2024-09-12T03:24:05.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:05.566+0000 7f521eb66640 1 -- 172.21.15.27:0/4167957809 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52180b7a50 con 0x7f521806d8c0 2024-09-12T03:24:05.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:05.567+0000 7f52157fa640 1 -- 172.21.15.27:0/4167957809 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 25) ==== 91365+0+0 (secure 0 0 0) 0x7f520801a070 con 0x7f521806d8c0 2024-09-12T03:24:05.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:05.567+0000 7f52157fa640 1 --2- 172.21.15.27:0/4167957809 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f520406c7c0 0x7f520406ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:05.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:05.567+0000 7f52157fa640 1 -- 172.21.15.27:0/4167957809 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(160..160 src has 1..160) ==== 5765+0+0 (secure 0 0 0) 0x7f52080893c0 con 0x7f521806d8c0 2024-09-12T03:24:05.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:05.567+0000 7f5217fff640 1 --2- 172.21.15.27:0/4167957809 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f520406c7c0 0x7f520406ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:05.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:05.568+0000 7f5217fff640 1 --2- 172.21.15.27:0/4167957809 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f520406c7c0 0x7f520406ec80 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f520c034000 tx=0x7f520c02e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:05.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:05.572+0000 7f52157fa640 1 -- 172.21.15.27:0/4167957809 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f5208059320 con 0x7f521806d8c0 2024-09-12T03:24:05.769 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:05.768+0000 7f521eb66640 1 -- 172.21.15.27:0/4167957809 --> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup rm", "vol_name": "cephfs", "group_name": "subvol_grp31770", "target": ["mon-mgr", ""]}) -- 0x7f521806e6e0 con 0x7f520406c7c0 2024-09-12T03:24:05.773 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:05.772+0000 7f52157fa640 1 -- 172.21.15.27:0/4167957809 <== mgr.4113 v2:172.21.15.27:6836/1226920526 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f521806e6e0 con 0x7f520406c7c0 2024-09-12T03:24:05.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:05.774+0000 7f521eb66640 1 -- 172.21.15.27:0/4167957809 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f520406c7c0 msgr2=0x7f520406ec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:05.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:05.774+0000 7f521eb66640 1 --2- 172.21.15.27:0/4167957809 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f520406c7c0 0x7f520406ec80 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f520c034000 tx=0x7f520c02e000 comp rx=0 tx=0).stop 2024-09-12T03:24:05.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:05.774+0000 7f521eb66640 1 -- 172.21.15.27:0/4167957809 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f521806d8c0 msgr2=0x7f521814fc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:05.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:05.774+0000 7f521eb66640 1 --2- 172.21.15.27:0/4167957809 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f521806d8c0 0x7f521814fc50 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7f520800e8a0 tx=0x7f520800ed60 comp rx=0 tx=0).stop 2024-09-12T03:24:05.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:05.774+0000 7f521eb66640 1 -- 172.21.15.27:0/4167957809 shutdown_connections 2024-09-12T03:24:05.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:05.774+0000 7f521eb66640 1 --2- 172.21.15.27:0/4167957809 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5218162040 0x7f52180cdd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:05.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:05.774+0000 7f521eb66640 1 --2- 172.21.15.27:0/4167957809 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f52180d3790 0x7f52180cd810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:05.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:05.774+0000 7f521eb66640 1 --2- 172.21.15.27:0/4167957809 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f520406c7c0 0x7f520406ec80 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:05.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:05.774+0000 7f521eb66640 1 --2- 172.21.15.27:0/4167957809 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f521806d8c0 0x7f521814fc50 unknown :-1 s=CLOSED pgs=1728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:05.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:05.775+0000 7f521eb66640 1 -- 172.21.15.27:0/4167957809 >> 172.21.15.27:0/4167957809 conn(0x7f52180b6700 msgr2=0x7f52180b8140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:05.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:05.775+0000 7f521eb66640 1 -- 172.21.15.27:0/4167957809 shutdown_connections 2024-09-12T03:24:05.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:05.775+0000 7f521eb66640 1 -- 172.21.15.27:0/4167957809 wait complete. 2024-09-12T03:24:05.787 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-12T03:24:05.787 DEBUG:teuthology.orchestra.run.smithi027:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-12T03:24:05.832 DEBUG:teuthology.orchestra.run.smithi045:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-12T03:24:05.891 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-12T03:24:05.891 INFO:tasks.cephfs.mount:Terminating background process 2024-09-12T03:24:05.892 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-12T03:24:05.892 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T03:24:05.909 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-12T03:24:05.909 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T03:24:05.965 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-12T03:24:06.108 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:06.107+0000 7f7342b9f640 1 Processor -- start 2024-09-12T03:24:06.108 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:06.107+0000 7f7342b9f640 1 -- start start 2024-09-12T03:24:06.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:06.108+0000 7f7342b9f640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f733c0b9190 0x7f733c0d28c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:06.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:06.108+0000 7f7342b9f640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f733c0d3920 con 0x7f733c06c160 2024-09-12T03:24:06.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:06.108+0000 7f7342b9f640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f733c06cc50 con 0x7f733c0d4090 2024-09-12T03:24:06.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:06.108+0000 7f7342b9f640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f733c0d54d0 con 0x7f733c0b9190 2024-09-12T03:24:06.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:06.108+0000 7f7341b9d640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f733c06c160 0x7f733c06c540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:57532/0 (socket says 172.21.15.27:57532) 2024-09-12T03:24:06.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:06.108+0000 7f7341b9d640 1 -- 172.21.15.27:0/185611480 learned_addr learned my addr 172.21.15.27:0/185611480 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:24:06.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:06.108+0000 7f734239e640 1 --2- 172.21.15.27:0/185611480 >> v2:172.21.15.45:3300/0 conn(0x7f733c0b9190 0x7f733c0d28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:06.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:06.108+0000 7f7340b9b640 1 -- 172.21.15.27:0/185611480 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2566851068 0 0) 0x7f733c0d3920 con 0x7f733c06c160 2024-09-12T03:24:06.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:06.108+0000 7f7340b9b640 1 -- 172.21.15.27:0/185611480 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7328003520 con 0x7f733c06c160 2024-09-12T03:24:06.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:06.109+0000 7f7340b9b640 1 -- 172.21.15.27:0/185611480 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2840157084 0 0) 0x7f7328003520 con 0x7f733c06c160 2024-09-12T03:24:06.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:06.109+0000 7f7340b9b640 1 -- 172.21.15.27:0/185611480 >> v2:172.21.15.45:3300/0 conn(0x7f733c0b9190 msgr2=0x7f733c0d28c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:24:06.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:06.109+0000 7f7340b9b640 1 --2- 172.21.15.27:0/185611480 >> v2:172.21.15.45:3300/0 conn(0x7f733c0b9190 0x7f733c0d28c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:06.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:06.109+0000 7f7340b9b640 1 -- 172.21.15.27:0/185611480 >> v1:172.21.15.45:6789/0 conn(0x7f733c0d4090 legacy=0x7f733c0d21b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:06.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:06.109+0000 7f7340b9b640 1 -- 172.21.15.27:0/185611480 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f733c0d3360 con 0x7f733c06c160 2024-09-12T03:24:06.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:06.109+0000 7f7340b9b640 1 -- 172.21.15.27:0/185611480 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 847755213 0 0) 0x7f7338001670 con 0x7f733c06c160 2024-09-12T03:24:06.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:06.109+0000 7f7340b9b640 1 -- 172.21.15.27:0/185611480 >> v1:172.21.15.27:6789/0 conn(0x7f733c06c160 legacy=0x7f733c06c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:06.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:06.109+0000 7f7340b9b640 1 --2- 172.21.15.27:0/185611480 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7328003a90 0x7f7328003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:06.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:06.109+0000 7f7340b9b640 1 --2- 172.21.15.27:0/185611480 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f7328004800 0x7f7328004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:06.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:06.109+0000 7f7340b9b640 1 --2- 172.21.15.27:0/185611480 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f73280052e0 0x7f732800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:06.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:06.109+0000 7f7340b9b640 1 -- 172.21.15.27:0/185611480 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f733c0d54d0 con 0x7f73280052e0 2024-09-12T03:24:06.113 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:06.109+0000 7f7340b9b640 1 -- 172.21.15.27:0/185611480 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f732800c1e0 con 0x7f7328003a90 2024-09-12T03:24:06.113 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:06.109+0000 7f7340b9b640 1 -- 172.21.15.27:0/185611480 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f732800c3e0 con 0x7f7328004800 2024-09-12T03:24:06.113 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:06.109+0000 7f734239e640 1 --2- 172.21.15.27:0/185611480 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f73280052e0 0x7f732800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:06.113 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:06.109+0000 7f7341b9d640 1 --2- 172.21.15.27:0/185611480 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7328003a90 0x7f7328003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:06.113 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:06.109+0000 7f734139c640 1 --2- 172.21.15.27:0/185611480 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f7328004800 0x7f7328004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:06.113 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:06.109+0000 7f734239e640 1 -- 172.21.15.27:0/185611480 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f7328004800 msgr2=0x7f7328004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:24:06.113 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:06.109+0000 7f734239e640 1 --2- 172.21.15.27:0/185611480 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f7328004800 0x7f7328004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:06.113 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:06.109+0000 7f734239e640 1 -- 172.21.15.27:0/185611480 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7328003a90 msgr2=0x7f7328003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:24:06.113 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:06.109+0000 7f734239e640 1 --2- 172.21.15.27:0/185611480 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7328003a90 0x7f7328003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:06.113 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:06.109+0000 7f734239e640 1 -- 172.21.15.27:0/185611480 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7328003520 con 0x7f73280052e0 2024-09-12T03:24:06.113 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:06.110+0000 7f734239e640 1 -- 172.21.15.27:0/185611480 reap_dead start 2024-09-12T03:24:06.113 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:06.110+0000 7f734239e640 1 --2- 172.21.15.27:0/185611480 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f73280052e0 0x7f732800b9c0 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7f7334009fc0 tx=0x7f7328005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=739b2c3052795659 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:06.113 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:06.110+0000 7f7340b9b640 1 -- 172.21.15.27:0/185611480 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7334035020 con 0x7f73280052e0 2024-09-12T03:24:06.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:06.110+0000 7f7340b9b640 1 -- 172.21.15.27:0/185611480 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7334003bd0 con 0x7f73280052e0 2024-09-12T03:24:06.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:06.110+0000 7f7340b9b640 1 -- 172.21.15.27:0/185611480 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7334009c10 con 0x7f73280052e0 2024-09-12T03:24:06.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:06.111+0000 7f7342b9f640 1 -- 172.21.15.27:0/185611480 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f73280052e0 msgr2=0x7f732800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:06.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:06.111+0000 7f7342b9f640 1 --2- 172.21.15.27:0/185611480 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f73280052e0 0x7f732800b9c0 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7f7334009fc0 tx=0x7f7328005830 comp rx=0 tx=0).stop 2024-09-12T03:24:06.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:06.111+0000 7f7342b9f640 1 -- 172.21.15.27:0/185611480 shutdown_connections 2024-09-12T03:24:06.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:06.111+0000 7f7342b9f640 1 --2- 172.21.15.27:0/185611480 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f73280052e0 0x7f732800b9c0 unknown :-1 s=CLOSED pgs=1730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:06.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:06.111+0000 7f7342b9f640 1 -- 172.21.15.27:0/185611480 >> 172.21.15.27:0/185611480 conn(0x7f733c0d45e0 msgr2=0x7f733c0d49f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:06.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:06.111+0000 7f7342b9f640 1 -- 172.21.15.27:0/185611480 shutdown_connections 2024-09-12T03:24:06.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:06.111+0000 7f7342b9f640 1 -- 172.21.15.27:0/185611480 wait complete. 2024-09-12T03:24:06.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:06.112+0000 7f7342b9f640 1 Processor -- start 2024-09-12T03:24:06.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:06.112+0000 7f7342b9f640 1 -- start start 2024-09-12T03:24:06.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:06.113+0000 7f7342b9f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f73280052e0 0x7f733c14ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:06.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:06.113+0000 7f7342b9f640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f733c0c9ce0 0x7f733c0cdac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:06.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:06.113+0000 7f7342b9f640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f733c0ca160 0x7f733c0ce000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:06.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:06.113+0000 7f7342b9f640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f733c0cf830 con 0x7f733c0c9ce0 2024-09-12T03:24:06.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:06.113+0000 7f7342b9f640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f733c0cf550 con 0x7f73280052e0 2024-09-12T03:24:06.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:06.113+0000 7f7342b9f640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f733c06c540 con 0x7f733c0ca160 2024-09-12T03:24:06.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:06.113+0000 7f734139c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f733c0c9ce0 0x7f733c0cdac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:06.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:06.113+0000 7f734139c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f733c0c9ce0 0x7f733c0cdac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33148/0 (socket says 172.21.15.27:33148) 2024-09-12T03:24:06.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:06.113+0000 7f734139c640 1 -- 172.21.15.27:0/1368561964 learned_addr learned my addr 172.21.15.27:0/1368561964 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:24:06.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:06.113+0000 7f7341b9d640 1 --2- 172.21.15.27:0/1368561964 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f73280052e0 0x7f733c14ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:06.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:06.113+0000 7f734239e640 1 --2- 172.21.15.27:0/1368561964 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f733c0ca160 0x7f733c0ce000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:06.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:06.113+0000 7f734139c640 1 -- 172.21.15.27:0/1368561964 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f733c0ca160 msgr2=0x7f733c0ce000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:06.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:06.113+0000 7f734139c640 1 --2- 172.21.15.27:0/1368561964 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f733c0ca160 0x7f733c0ce000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:06.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:06.113+0000 7f734139c640 1 -- 172.21.15.27:0/1368561964 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f73280052e0 msgr2=0x7f733c14ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:06.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:06.113+0000 7f734139c640 1 --2- 172.21.15.27:0/1368561964 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f73280052e0 0x7f733c14ea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:06.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:06.113+0000 7f734139c640 1 -- 172.21.15.27:0/1368561964 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f733c159410 con 0x7f733c0c9ce0 2024-09-12T03:24:06.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:06.113+0000 7f734239e640 1 --2- 172.21.15.27:0/1368561964 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f733c0ca160 0x7f733c0ce000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:24:06.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:06.113+0000 7f7341b9d640 1 --2- 172.21.15.27:0/1368561964 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f73280052e0 0x7f733c14ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:24:06.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:06.114+0000 7f734139c640 1 --2- 172.21.15.27:0/1368561964 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f733c0c9ce0 0x7f733c0cdac0 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7f732c00f7c0 tx=0x7f732c00fc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:06.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:06.114+0000 7f73327fc640 1 -- 172.21.15.27:0/1368561964 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f732c00fe70 con 0x7f733c0c9ce0 2024-09-12T03:24:06.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:06.114+0000 7f73327fc640 1 -- 172.21.15.27:0/1368561964 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f732c018070 con 0x7f733c0c9ce0 2024-09-12T03:24:06.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:06.114+0000 7f7342b9f640 1 -- 172.21.15.27:0/1368561964 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f733c1596f0 con 0x7f733c0c9ce0 2024-09-12T03:24:06.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:06.114+0000 7f73327fc640 1 -- 172.21.15.27:0/1368561964 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f732c00d6d0 con 0x7f733c0c9ce0 2024-09-12T03:24:06.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:06.114+0000 7f7342b9f640 1 -- 172.21.15.27:0/1368561964 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f733c159cb0 con 0x7f733c0c9ce0 2024-09-12T03:24:06.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:06.115+0000 7f7342b9f640 1 -- 172.21.15.27:0/1368561964 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f733c06c870 con 0x7f733c0c9ce0 2024-09-12T03:24:06.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:06.115+0000 7f73327fc640 1 -- 172.21.15.27:0/1368561964 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 25) ==== 91365+0+0 (secure 0 0 0) 0x7f732c00d870 con 0x7f733c0c9ce0 2024-09-12T03:24:06.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:06.116+0000 7f73327fc640 1 --2- 172.21.15.27:0/1368561964 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f732006c7f0 0x7f732006ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:06.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:06.116+0000 7f73327fc640 1 -- 172.21.15.27:0/1368561964 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(160..160 src has 1..160) ==== 5765+0+0 (secure 0 0 0) 0x7f732c0894a0 con 0x7f733c0c9ce0 2024-09-12T03:24:06.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:06.116+0000 7f7341b9d640 1 --2- 172.21.15.27:0/1368561964 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f732006c7f0 0x7f732006ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:06.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:06.116+0000 7f7341b9d640 1 --2- 172.21.15.27:0/1368561964 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f732006c7f0 0x7f732006ecb0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f7338001200 tx=0x7f7338009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:06.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:06.120+0000 7f73327fc640 1 -- 172.21.15.27:0/1368561964 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f732c059a50 con 0x7f733c0c9ce0 2024-09-12T03:24:06.311 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:06.310+0000 7f73327fc640 1 -- 172.21.15.27:0/1368561964 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mgrmap(e 26) ==== 91518+0+0 (secure 0 0 0) 0x7f732c0559c0 con 0x7f733c0c9ce0 2024-09-12T03:24:06.311 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:06.310+0000 7f7342b9f640 1 -- 172.21.15.27:0/1368561964 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f733c06d7c0 con 0x7f733c0c9ce0 2024-09-12T03:24:06.311 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:06.310+0000 7f73327fc640 1 -- 172.21.15.27:0/1368561964 <== mon.0 v2:172.21.15.27:3300/0 8 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v160) ==== 80+0+0 (secure 0 0 0) 0x7f732c012350 con 0x7f733c0c9ce0 2024-09-12T03:24:06.312 INFO:teuthology.orchestra.run.smithi027.stderr:listed 0 entries 2024-09-12T03:24:06.314 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:06.313+0000 7f7342b9f640 1 -- 172.21.15.27:0/1368561964 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f732006c7f0 msgr2=0x7f732006ecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:06.314 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:06.313+0000 7f7342b9f640 1 --2- 172.21.15.27:0/1368561964 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f732006c7f0 0x7f732006ecb0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f7338001200 tx=0x7f7338009040 comp rx=0 tx=0).stop 2024-09-12T03:24:06.314 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:06.313+0000 7f7342b9f640 1 -- 172.21.15.27:0/1368561964 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f733c0c9ce0 msgr2=0x7f733c0cdac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:06.314 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:06.313+0000 7f7342b9f640 1 --2- 172.21.15.27:0/1368561964 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f733c0c9ce0 0x7f733c0cdac0 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7f732c00f7c0 tx=0x7f732c00fc80 comp rx=0 tx=0).stop 2024-09-12T03:24:06.314 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:06.313+0000 7f7342b9f640 1 -- 172.21.15.27:0/1368561964 shutdown_connections 2024-09-12T03:24:06.314 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:06.313+0000 7f7342b9f640 1 --2- 172.21.15.27:0/1368561964 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f733c0ca160 0x7f733c0ce000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:06.314 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:06.313+0000 7f7342b9f640 1 --2- 172.21.15.27:0/1368561964 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f732006c7f0 0x7f732006ecb0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:06.314 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:06.313+0000 7f7342b9f640 1 --2- 172.21.15.27:0/1368561964 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f733c0c9ce0 0x7f733c0cdac0 unknown :-1 s=CLOSED pgs=1731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:06.314 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:06.313+0000 7f7342b9f640 1 --2- 172.21.15.27:0/1368561964 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f73280052e0 0x7f733c14ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:06.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:06.313+0000 7f7342b9f640 1 -- 172.21.15.27:0/1368561964 >> 172.21.15.27:0/1368561964 conn(0x7f733c0d45e0 msgr2=0x7f733c0ceb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:06.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:06.313+0000 7f7342b9f640 1 -- 172.21.15.27:0/1368561964 shutdown_connections 2024-09-12T03:24:06.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:06.313+0000 7f7342b9f640 1 -- 172.21.15.27:0/1368561964 wait complete. 2024-09-12T03:24:06.325 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-12T03:24:06.325 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:24:06.325 DEBUG:teuthology.orchestra.run.smithi027:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-12T03:24:06.376 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi027.front.sepia.ceph.com 2024-09-12T03:24:06.377 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:24:06.377 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-12T03:24:06.405 INFO:tasks.cephfs.mount:Terminating background process 2024-09-12T03:24:06.405 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-12T03:24:06.405 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T03:24:06.423 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-12T03:24:06.423 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T03:24:06.481 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-12T03:24:06.584 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:06.583+0000 7f3738fd6640 1 Processor -- start 2024-09-12T03:24:06.585 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:06.584+0000 7f3738fd6640 1 -- start start 2024-09-12T03:24:06.586 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:06.584+0000 7f3738fd6640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f373415a8c0 0x7f37340cdb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:06.586 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:06.585+0000 7f3738fd6640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f373415aca0 0x7f37340b4710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:06.586 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:06.585+0000 7f3738fd6640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f373406daf0 con 0x7f3734071be0 2024-09-12T03:24:06.586 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:06.585+0000 7f3738fd6640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f373406d8a0 con 0x7f373415aca0 2024-09-12T03:24:06.586 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:06.585+0000 7f3738fd6640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f373406ddd0 con 0x7f373415a8c0 2024-09-12T03:24:06.586 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:06.585+0000 7f3732d9d640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f373415aca0 0x7f37340b4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:06.586 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:06.585+0000 7f3731d9b640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f3734071be0 0x7f373406e8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:57544/0 (socket says 172.21.15.27:57544) 2024-09-12T03:24:06.586 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:06.585+0000 7f3731d9b640 1 -- 172.21.15.27:0/1224656202 learned_addr learned my addr 172.21.15.27:0/1224656202 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:24:06.586 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:06.585+0000 7f373259c640 1 --2- 172.21.15.27:0/1224656202 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f373415a8c0 0x7f37340cdb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:06.587 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:06.585+0000 7f3732d9d640 1 -- 172.21.15.27:0/1224656202 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f373415a8c0 msgr2=0x7f37340cdb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:24:06.587 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:06.585+0000 7f3732d9d640 1 --2- 172.21.15.27:0/1224656202 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f373415a8c0 0x7f37340cdb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:06.587 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:06.585+0000 7f3732d9d640 1 -- 172.21.15.27:0/1224656202 >> v1:172.21.15.27:6789/0 conn(0x7f3734071be0 legacy=0x7f373406e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:06.587 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:06.585+0000 7f373159a640 1 -- 172.21.15.27:0/1224656202 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4026516863 0 0) 0x7f373406daf0 con 0x7f3734071be0 2024-09-12T03:24:06.587 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:06.585+0000 7f3732d9d640 1 -- 172.21.15.27:0/1224656202 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f373406e0b0 con 0x7f373415aca0 2024-09-12T03:24:06.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:06.586+0000 7f3732d9d640 1 --2- 172.21.15.27:0/1224656202 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f373415aca0 0x7f37340b4710 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7f37240097a0 tx=0x7f372402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c239e787c6428e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:06.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:06.586+0000 7f373159a640 1 -- 172.21.15.27:0/1224656202 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3724037070 con 0x7f373415aca0 2024-09-12T03:24:06.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:06.586+0000 7f373159a640 1 -- 172.21.15.27:0/1224656202 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3724032440 con 0x7f373415aca0 2024-09-12T03:24:06.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:06.586+0000 7f373159a640 1 -- 172.21.15.27:0/1224656202 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3724032810 con 0x7f373415aca0 2024-09-12T03:24:06.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:06.586+0000 7f3738fd6640 1 -- 172.21.15.27:0/1224656202 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f373415aca0 msgr2=0x7f37340b4710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:06.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:06.586+0000 7f3738fd6640 1 --2- 172.21.15.27:0/1224656202 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f373415aca0 0x7f37340b4710 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7f37240097a0 tx=0x7f372402a180 comp rx=0 tx=0).stop 2024-09-12T03:24:06.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:06.587+0000 7f3738fd6640 1 -- 172.21.15.27:0/1224656202 shutdown_connections 2024-09-12T03:24:06.589 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:06.587+0000 7f3738fd6640 1 --2- 172.21.15.27:0/1224656202 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f373415aca0 0x7f37340b4710 unknown :-1 s=CLOSED pgs=1733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:06.589 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:06.587+0000 7f3738fd6640 1 --2- 172.21.15.27:0/1224656202 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f373415a8c0 0x7f37340cdb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:06.589 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:06.587+0000 7f3738fd6640 1 -- 172.21.15.27:0/1224656202 >> 172.21.15.27:0/1224656202 conn(0x7f3734062c60 msgr2=0x7f373406bb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:06.589 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:06.587+0000 7f3738fd6640 1 -- 172.21.15.27:0/1224656202 shutdown_connections 2024-09-12T03:24:06.589 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:06.587+0000 7f3738fd6640 1 -- 172.21.15.27:0/1224656202 wait complete. 2024-09-12T03:24:06.591 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:06.588+0000 7f3738fd6640 1 Processor -- start 2024-09-12T03:24:06.591 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:06.588+0000 7f3738fd6640 1 -- start start 2024-09-12T03:24:06.591 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:06.589+0000 7f3738fd6640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3734071be0 0x7f37341512a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:06.591 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:06.589+0000 7f3738fd6640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f373415a8c0 0x7f3734157e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:06.591 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:06.589+0000 7f3738fd6640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f373415aca0 0x7f37340ae460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:06.591 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:06.589+0000 7f3738fd6640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f373406f140 con 0x7f373415aca0 2024-09-12T03:24:06.592 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:06.589+0000 7f3738fd6640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f373406efc0 con 0x7f3734071be0 2024-09-12T03:24:06.592 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:06.589+0000 7f3738fd6640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f37340b3560 con 0x7f373415a8c0 2024-09-12T03:24:06.592 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:06.589+0000 7f3732d9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f373415aca0 0x7f37340ae460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:06.592 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:06.589+0000 7f3732d9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f373415aca0 0x7f37340ae460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33164/0 (socket says 172.21.15.27:33164) 2024-09-12T03:24:06.592 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:06.589+0000 7f3732d9d640 1 -- 172.21.15.27:0/3294889478 learned_addr learned my addr 172.21.15.27:0/3294889478 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:24:06.592 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:06.589+0000 7f3731d9b640 1 --2- 172.21.15.27:0/3294889478 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f373415a8c0 0x7f3734157e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:06.592 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:06.589+0000 7f373259c640 1 --2- 172.21.15.27:0/3294889478 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3734071be0 0x7f37341512a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:06.592 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:06.589+0000 7f3732d9d640 1 -- 172.21.15.27:0/3294889478 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f373415a8c0 msgr2=0x7f3734157e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:06.592 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:06.589+0000 7f3732d9d640 1 --2- 172.21.15.27:0/3294889478 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f373415a8c0 0x7f3734157e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:06.593 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:06.589+0000 7f3732d9d640 1 -- 172.21.15.27:0/3294889478 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3734071be0 msgr2=0x7f37341512a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:06.593 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:06.589+0000 7f3732d9d640 1 --2- 172.21.15.27:0/3294889478 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3734071be0 0x7f37341512a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:06.593 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:06.589+0000 7f3732d9d640 1 -- 172.21.15.27:0/3294889478 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37341517e0 con 0x7f373415aca0 2024-09-12T03:24:06.593 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:06.589+0000 7f373259c640 1 --2- 172.21.15.27:0/3294889478 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3734071be0 0x7f37341512a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:24:06.593 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:06.589+0000 7f3732d9d640 1 --2- 172.21.15.27:0/3294889478 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f373415aca0 0x7f37340ae460 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7f3724035040 tx=0x7f37240046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:06.593 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:06.589+0000 7f372affd640 1 -- 172.21.15.27:0/3294889478 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3724040020 con 0x7f373415aca0 2024-09-12T03:24:06.593 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:06.590+0000 7f372affd640 1 -- 172.21.15.27:0/3294889478 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3724038040 con 0x7f373415aca0 2024-09-12T03:24:06.593 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:06.590+0000 7f3738fd6640 1 -- 172.21.15.27:0/3294889478 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3734158790 con 0x7f373415aca0 2024-09-12T03:24:06.593 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:06.590+0000 7f372affd640 1 -- 172.21.15.27:0/3294889478 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3724037070 con 0x7f373415aca0 2024-09-12T03:24:06.593 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:06.590+0000 7f3738fd6640 1 -- 172.21.15.27:0/3294889478 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37340abdd0 con 0x7f373415aca0 2024-09-12T03:24:06.594 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:06.591+0000 7f3738fd6640 1 -- 172.21.15.27:0/3294889478 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37340ce680 con 0x7f373415aca0 2024-09-12T03:24:06.594 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:06.591+0000 7f372affd640 1 -- 172.21.15.27:0/3294889478 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 26) ==== 91518+0+0 (secure 0 0 0) 0x7f3724043020 con 0x7f373415aca0 2024-09-12T03:24:06.594 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:06.592+0000 7f372affd640 1 --2- 172.21.15.27:0/3294889478 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f371406c9a0 0x7f371406ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:06.594 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:06.592+0000 7f372affd640 1 -- 172.21.15.27:0/3294889478 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(160..160 src has 1..160) ==== 5765+0+0 (secure 0 0 0) 0x7f37240a9850 con 0x7f373415aca0 2024-09-12T03:24:06.594 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:06.592+0000 7f373259c640 1 --2- 172.21.15.27:0/3294889478 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f371406c9a0 0x7f371406ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:06.595 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:06.592+0000 7f373259c640 1 --2- 172.21.15.27:0/3294889478 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f371406c9a0 0x7f371406ee60 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f37340b1440 tx=0x7f372c007650 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:06.597 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:06.596+0000 7f372affd640 1 -- 172.21.15.27:0/3294889478 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f372404a070 con 0x7f373415aca0 2024-09-12T03:24:06.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:06.786+0000 7f3738fd6640 1 -- 172.21.15.27:0/3294889478 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f3734073090 con 0x7f373415aca0 2024-09-12T03:24:06.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:06.786+0000 7f372affd640 1 -- 172.21.15.27:0/3294889478 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v160) ==== 80+0+0 (secure 0 0 0) 0x7f3724079cb0 con 0x7f373415aca0 2024-09-12T03:24:06.787 INFO:teuthology.orchestra.run.smithi027.stderr:listed 0 entries 2024-09-12T03:24:06.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:06.788+0000 7f3738fd6640 1 -- 172.21.15.27:0/3294889478 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f371406c9a0 msgr2=0x7f371406ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:06.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:06.788+0000 7f3738fd6640 1 --2- 172.21.15.27:0/3294889478 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f371406c9a0 0x7f371406ee60 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f37340b1440 tx=0x7f372c007650 comp rx=0 tx=0).stop 2024-09-12T03:24:06.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:06.788+0000 7f3738fd6640 1 -- 172.21.15.27:0/3294889478 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f373415aca0 msgr2=0x7f37340ae460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:06.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:06.788+0000 7f3738fd6640 1 --2- 172.21.15.27:0/3294889478 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f373415aca0 0x7f37340ae460 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7f3724035040 tx=0x7f37240046c0 comp rx=0 tx=0).stop 2024-09-12T03:24:06.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:06.788+0000 7f3738fd6640 1 -- 172.21.15.27:0/3294889478 shutdown_connections 2024-09-12T03:24:06.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:06.788+0000 7f3738fd6640 1 --2- 172.21.15.27:0/3294889478 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f371406c9a0 0x7f371406ee60 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:06.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:06.788+0000 7f3738fd6640 1 --2- 172.21.15.27:0/3294889478 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f373415aca0 0x7f37340ae460 unknown :-1 s=CLOSED pgs=1734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:06.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:06.788+0000 7f3738fd6640 1 --2- 172.21.15.27:0/3294889478 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f373415a8c0 0x7f3734157e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:06.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:06.788+0000 7f3738fd6640 1 --2- 172.21.15.27:0/3294889478 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3734071be0 0x7f37341512a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:06.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:06.788+0000 7f3738fd6640 1 -- 172.21.15.27:0/3294889478 >> 172.21.15.27:0/3294889478 conn(0x7f3734062c60 msgr2=0x7f373406e8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:06.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:06.788+0000 7f3738fd6640 1 -- 172.21.15.27:0/3294889478 shutdown_connections 2024-09-12T03:24:06.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:06.788+0000 7f3738fd6640 1 -- 172.21.15.27:0/3294889478 wait complete. 2024-09-12T03:24:06.800 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-09-12T03:24:06.800 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:24:06.800 DEBUG:teuthology.orchestra.run.smithi045:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-09-12T03:24:06.834 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi045.front.sepia.ceph.com 2024-09-12T03:24:06.834 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:24:06.834 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-12T03:24:06.893 INFO:tasks.cephfs.mount:Terminating background process 2024-09-12T03:24:06.893 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-12T03:24:06.893 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T03:24:06.909 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056004,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497432k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16324 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248716k,nr_inodes=812179,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497432k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-12T03:24:06.910 INFO:tasks.cephfs.mount:Terminating background process 2024-09-12T03:24:06.910 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-12T03:24:06.910 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T03:24:06.950 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056795,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16602 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249348k,nr_inodes=812337,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-12T03:24:06.950 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-12T03:24:07.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.050+0000 7f1a48cdd640 1 Processor -- start 2024-09-12T03:24:07.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.050+0000 7f1a48cdd640 1 -- start start 2024-09-12T03:24:07.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.051+0000 7f1a48cdd640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f1a4406f0c0 0x7f1a440b3560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:07.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.051+0000 7f1a48cdd640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1a440ce410 con 0x7f1a4415b2b0 2024-09-12T03:24:07.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.051+0000 7f1a48cdd640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1a440732c0 con 0x7f1a44072d70 2024-09-12T03:24:07.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.051+0000 7f1a48cdd640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f1a440720a0 con 0x7f1a4406f0c0 2024-09-12T03:24:07.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.051+0000 7f1a42d9d640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f1a4406f0c0 0x7f1a440b3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:07.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.051+0000 7f1a4259c640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f1a4415b2b0 0x7f1a44072430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:57560/0 (socket says 172.21.15.27:57560) 2024-09-12T03:24:07.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.051+0000 7f1a4259c640 1 -- 172.21.15.27:0/2962857529 learned_addr learned my addr 172.21.15.27:0/2962857529 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:24:07.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.052+0000 7f1a42d9d640 1 -- 172.21.15.27:0/2962857529 >> v1:172.21.15.45:6789/0 conn(0x7f1a44072d70 legacy=0x7f1a4406e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:24:07.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.052+0000 7f1a42d9d640 1 -- 172.21.15.27:0/2962857529 >> v1:172.21.15.27:6789/0 conn(0x7f1a4415b2b0 legacy=0x7f1a44072430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:07.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.052+0000 7f1a42d9d640 1 -- 172.21.15.27:0/2962857529 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a44071ae0 con 0x7f1a4406f0c0 2024-09-12T03:24:07.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.052+0000 7f1a4159a640 1 -- 172.21.15.27:0/2962857529 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4081185063 0 0) 0x7f1a3c001110 con 0x7f1a4415b2b0 2024-09-12T03:24:07.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.052+0000 7f1a42d9d640 1 --2- 172.21.15.27:0/2962857529 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1a4406f0c0 0x7f1a440b3560 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7f1a34009d90 tx=0x7f1a3402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c23d0db563c06a6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:07.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.053+0000 7f1a4159a640 1 -- 172.21.15.27:0/2962857529 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a34007740 con 0x7f1a4406f0c0 2024-09-12T03:24:07.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.053+0000 7f1a4159a640 1 -- 172.21.15.27:0/2962857529 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a340078e0 con 0x7f1a4406f0c0 2024-09-12T03:24:07.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.053+0000 7f1a4159a640 1 -- 172.21.15.27:0/2962857529 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a34007cb0 con 0x7f1a4406f0c0 2024-09-12T03:24:07.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.053+0000 7f1a48cdd640 1 -- 172.21.15.27:0/2962857529 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1a4406f0c0 msgr2=0x7f1a440b3560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:07.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.053+0000 7f1a48cdd640 1 --2- 172.21.15.27:0/2962857529 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1a4406f0c0 0x7f1a440b3560 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7f1a34009d90 tx=0x7f1a3402a6a0 comp rx=0 tx=0).stop 2024-09-12T03:24:07.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.053+0000 7f1a48cdd640 1 -- 172.21.15.27:0/2962857529 shutdown_connections 2024-09-12T03:24:07.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.053+0000 7f1a48cdd640 1 --2- 172.21.15.27:0/2962857529 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1a4406f0c0 0x7f1a440b3560 unknown :-1 s=CLOSED pgs=1736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:07.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.053+0000 7f1a48cdd640 1 -- 172.21.15.27:0/2962857529 >> 172.21.15.27:0/2962857529 conn(0x7f1a44062c60 msgr2=0x7f1a4406bb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:07.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.053+0000 7f1a48cdd640 1 -- 172.21.15.27:0/2962857529 shutdown_connections 2024-09-12T03:24:07.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.053+0000 7f1a48cdd640 1 -- 172.21.15.27:0/2962857529 wait complete. 2024-09-12T03:24:07.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.055+0000 7f1a48cdd640 1 Processor -- start 2024-09-12T03:24:07.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.055+0000 7f1a48cdd640 1 -- start start 2024-09-12T03:24:07.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.055+0000 7f1a48cdd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1a4406f0c0 0x7f1a4415bd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:07.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.055+0000 7f1a48cdd640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1a44072d70 0x7f1a440aeb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:07.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.055+0000 7f1a48cdd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1a4415b2b0 0x7f1a44169aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:07.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.055+0000 7f1a48cdd640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f1a4415b690 con 0x7f1a4406f0c0 2024-09-12T03:24:07.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.055+0000 7f1a48cdd640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f1a440ce690 con 0x7f1a4415b2b0 2024-09-12T03:24:07.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.055+0000 7f1a48cdd640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f1a4415b810 con 0x7f1a44072d70 2024-09-12T03:24:07.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.055+0000 7f1a4259c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1a4406f0c0 0x7f1a4415bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:07.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.055+0000 7f1a4259c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1a4406f0c0 0x7f1a4415bd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33180/0 (socket says 172.21.15.27:33180) 2024-09-12T03:24:07.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.055+0000 7f1a4259c640 1 -- 172.21.15.27:0/1807763318 learned_addr learned my addr 172.21.15.27:0/1807763318 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:24:07.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.056+0000 7f1a42d9d640 1 --2- 172.21.15.27:0/1807763318 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1a4415b2b0 0x7f1a44169aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:07.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.056+0000 7f1a4259c640 1 -- 172.21.15.27:0/1807763318 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1a44072d70 msgr2=0x7f1a440aeb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:07.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.056+0000 7f1a4259c640 1 --2- 172.21.15.27:0/1807763318 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1a44072d70 0x7f1a440aeb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:07.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.056+0000 7f1a4259c640 1 -- 172.21.15.27:0/1807763318 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1a4415b2b0 msgr2=0x7f1a44169aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:07.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.056+0000 7f1a4259c640 1 --2- 172.21.15.27:0/1807763318 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1a4415b2b0 0x7f1a44169aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:07.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.056+0000 7f1a4259c640 1 -- 172.21.15.27:0/1807763318 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a44159450 con 0x7f1a4406f0c0 2024-09-12T03:24:07.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.056+0000 7f1a4259c640 1 --2- 172.21.15.27:0/1807763318 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1a4406f0c0 0x7f1a4415bd80 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x7f1a3c00c430 tx=0x7f1a3c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:07.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.056+0000 7f1a3affd640 1 -- 172.21.15.27:0/1807763318 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a3c0074e0 con 0x7f1a4406f0c0 2024-09-12T03:24:07.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.056+0000 7f1a3affd640 1 -- 172.21.15.27:0/1807763318 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a3c0047e0 con 0x7f1a4406f0c0 2024-09-12T03:24:07.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.056+0000 7f1a48cdd640 1 -- 172.21.15.27:0/1807763318 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a44159730 con 0x7f1a4406f0c0 2024-09-12T03:24:07.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.056+0000 7f1a3affd640 1 -- 172.21.15.27:0/1807763318 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a3c004bd0 con 0x7f1a4406f0c0 2024-09-12T03:24:07.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.056+0000 7f1a48cdd640 1 -- 172.21.15.27:0/1807763318 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a44169270 con 0x7f1a4406f0c0 2024-09-12T03:24:07.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.057+0000 7f1a48cdd640 1 -- 172.21.15.27:0/1807763318 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a44147570 con 0x7f1a4406f0c0 2024-09-12T03:24:07.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.058+0000 7f1a3affd640 1 -- 172.21.15.27:0/1807763318 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 26) ==== 91518+0+0 (secure 0 0 0) 0x7f1a3c011070 con 0x7f1a4406f0c0 2024-09-12T03:24:07.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.058+0000 7f1a3affd640 1 --2- 172.21.15.27:0/1807763318 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f1a2406c970 0x7f1a2406ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:07.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.058+0000 7f1a3affd640 1 -- 172.21.15.27:0/1807763318 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(160..160 src has 1..160) ==== 5765+0+0 (secure 0 0 0) 0x7f1a3c089690 con 0x7f1a4406f0c0 2024-09-12T03:24:07.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.058+0000 7f1a41d9b640 1 --2- 172.21.15.27:0/1807763318 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f1a2406c970 0x7f1a2406ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:07.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.059+0000 7f1a41d9b640 1 --2- 172.21.15.27:0/1807763318 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f1a2406c970 0x7f1a2406ee30 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f1a440af630 tx=0x7f1a2c0079b0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:07.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.062+0000 7f1a3affd640 1 -- 172.21.15.27:0/1807763318 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f1a440ce410 con 0x7f1a4406f0c0 2024-09-12T03:24:07.276 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.275+0000 7f1a48cdd640 1 -- 172.21.15.27:0/1807763318 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1a4415f620 con 0x7f1a4406f0c0 2024-09-12T03:24:07.276 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.275+0000 7f1a3affd640 1 -- 172.21.15.27:0/1807763318 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 100 v100) ==== 95+0+5228 (secure 0 0 0) 0x7f1a3c0598a0 con 0x7f1a4406f0c0 2024-09-12T03:24:07.276 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:24:07.277 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":100,"btime":"2024-09-12T03:24:03:282449+0000","default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7163,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/2493793567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2493793567},{"type":"v1","addr":"172.21.15.45:6835","nonce":2493793567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":96},{"gid":8056,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/1643775717","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":1643775717},{"type":"v1","addr":"172.21.15.45:6834","nonce":1643775717}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95},{"gid":8967,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6834/4129691413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":4129691413},{"type":"v1","addr":"172.21.15.27:6834","nonce":4129691413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95}],"filesystems":[{"mdsmap":{"epoch":99,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-12T03:23:54.606856+0000","modified":"2024-09-12T03:23:56.609988+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8970},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8970":{"gid":8970,"name":"c","rank":0,"incarnation":97,"state":"up:active","state_seq":3,"addr":"172.21.15.27:6835/2940099359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":2940099359},{"type":"v1","addr":"172.21.15.27:6835","nonce":2940099359}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[27],"metadata_pool":26,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8970,"qdb_cluster":[8970]},"id":13}]} 2024-09-12T03:24:07.277 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 100 2024-09-12T03:24:07.278 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.277+0000 7f1a48cdd640 1 -- 172.21.15.27:0/1807763318 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f1a2406c970 msgr2=0x7f1a2406ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:07.278 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.277+0000 7f1a48cdd640 1 --2- 172.21.15.27:0/1807763318 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f1a2406c970 0x7f1a2406ee30 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f1a440af630 tx=0x7f1a2c0079b0 comp rx=0 tx=0).stop 2024-09-12T03:24:07.278 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.277+0000 7f1a48cdd640 1 -- 172.21.15.27:0/1807763318 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1a4406f0c0 msgr2=0x7f1a4415bd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:07.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.277+0000 7f1a48cdd640 1 --2- 172.21.15.27:0/1807763318 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1a4406f0c0 0x7f1a4415bd80 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x7f1a3c00c430 tx=0x7f1a3c00c8f0 comp rx=0 tx=0).stop 2024-09-12T03:24:07.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.277+0000 7f1a48cdd640 1 -- 172.21.15.27:0/1807763318 shutdown_connections 2024-09-12T03:24:07.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.277+0000 7f1a48cdd640 1 --2- 172.21.15.27:0/1807763318 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1a4415b2b0 0x7f1a44169aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:07.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.277+0000 7f1a48cdd640 1 --2- 172.21.15.27:0/1807763318 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1a44072d70 0x7f1a440aeb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:07.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.277+0000 7f1a48cdd640 1 --2- 172.21.15.27:0/1807763318 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f1a2406c970 0x7f1a2406ee30 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:07.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.277+0000 7f1a48cdd640 1 --2- 172.21.15.27:0/1807763318 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1a4406f0c0 0x7f1a4415bd80 unknown :-1 s=CLOSED pgs=1737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:07.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.277+0000 7f1a48cdd640 1 -- 172.21.15.27:0/1807763318 >> 172.21.15.27:0/1807763318 conn(0x7f1a44062c60 msgr2=0x7f1a4406d8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:07.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.277+0000 7f1a48cdd640 1 -- 172.21.15.27:0/1807763318 shutdown_connections 2024-09-12T03:24:07.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.277+0000 7f1a48cdd640 1 -- 172.21.15.27:0/1807763318 wait complete. 2024-09-12T03:24:07.291 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-12T03:24:07.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.393+0000 7f23a832c640 1 Processor -- start 2024-09-12T03:24:07.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.393+0000 7f23a832c640 1 -- start start 2024-09-12T03:24:07.397 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.394+0000 7f23a832c640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f23a015a1f0 0x7f23a006dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:07.397 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.394+0000 7f23a832c640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f23a0073b50 con 0x7f23a0072fc0 2024-09-12T03:24:07.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.394+0000 7f23a832c640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f23a0073870 con 0x7f23a006fcf0 2024-09-12T03:24:07.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.394+0000 7f23a832c640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f23a00ce3e0 con 0x7f23a015a1f0 2024-09-12T03:24:07.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.394+0000 7f23a68c9640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f23a0072fc0 0x7f23a015cce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:57566/0 (socket says 172.21.15.27:57566) 2024-09-12T03:24:07.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.394+0000 7f23a68c9640 1 -- 172.21.15.27:0/832636577 learned_addr learned my addr 172.21.15.27:0/832636577 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:24:07.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.394+0000 7f23a50c6640 1 -- 172.21.15.27:0/832636577 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 686054418 0 0) 0x7f23a0073b50 con 0x7f23a0072fc0 2024-09-12T03:24:07.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.394+0000 7f23a60c8640 1 --2- 172.21.15.27:0/832636577 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f23a015a1f0 0x7f23a006dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:07.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.394+0000 7f23a50c6640 1 -- 172.21.15.27:0/832636577 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f238c003520 con 0x7f23a0072fc0 2024-09-12T03:24:07.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.395+0000 7f23a50c6640 1 -- 172.21.15.27:0/832636577 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3021617673 0 0) 0x7f238c003520 con 0x7f23a0072fc0 2024-09-12T03:24:07.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.395+0000 7f23a50c6640 1 -- 172.21.15.27:0/832636577 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f23a015a1f0 msgr2=0x7f23a006dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:24:07.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.395+0000 7f23a50c6640 1 --2- 172.21.15.27:0/832636577 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f23a015a1f0 0x7f23a006dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:07.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.395+0000 7f23a50c6640 1 -- 172.21.15.27:0/832636577 >> v1:172.21.15.45:6789/0 conn(0x7f23a006fcf0 legacy=0x7f23a00728b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:07.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.395+0000 7f23a50c6640 1 -- 172.21.15.27:0/832636577 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23a00ce6c0 con 0x7f23a0072fc0 2024-09-12T03:24:07.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.395+0000 7f23a50c6640 1 -- 172.21.15.27:0/832636577 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 847755213 0 0) 0x7f23980019f0 con 0x7f23a0072fc0 2024-09-12T03:24:07.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.395+0000 7f23a50c6640 1 -- 172.21.15.27:0/832636577 >> v1:172.21.15.27:6789/0 conn(0x7f23a0072fc0 legacy=0x7f23a015cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:07.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.395+0000 7f23a50c6640 1 --2- 172.21.15.27:0/832636577 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f238c0039e0 0x7f238c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:07.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.395+0000 7f23a50c6640 1 --2- 172.21.15.27:0/832636577 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f238c004800 0x7f238c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:07.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.395+0000 7f23a50c6640 1 --2- 172.21.15.27:0/832636577 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f23a015a1f0 0x7f238c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:07.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.395+0000 7f23a50c6640 1 -- 172.21.15.27:0/832636577 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f23a00ce3e0 con 0x7f238c0039e0 2024-09-12T03:24:07.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.395+0000 7f23a50c6640 1 -- 172.21.15.27:0/832636577 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f238c00ba70 con 0x7f238c004800 2024-09-12T03:24:07.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.395+0000 7f23a50c6640 1 -- 172.21.15.27:0/832636577 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f238c00bc70 con 0x7f23a015a1f0 2024-09-12T03:24:07.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.395+0000 7f23a60c8640 1 --2- 172.21.15.27:0/832636577 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f238c0039e0 0x7f238c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:07.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.395+0000 7f23a68c9640 1 --2- 172.21.15.27:0/832636577 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f23a015a1f0 0x7f238c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:07.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.395+0000 7f23a58c7640 1 --2- 172.21.15.27:0/832636577 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f238c004800 0x7f238c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:07.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.395+0000 7f23a60c8640 1 -- 172.21.15.27:0/832636577 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f23a015a1f0 msgr2=0x7f238c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:24:07.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.395+0000 7f23a60c8640 1 --2- 172.21.15.27:0/832636577 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f23a015a1f0 0x7f238c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:07.400 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.395+0000 7f23a60c8640 1 -- 172.21.15.27:0/832636577 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f238c004800 msgr2=0x7f238c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:24:07.400 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.395+0000 7f23a60c8640 1 --2- 172.21.15.27:0/832636577 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f238c004800 0x7f238c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:07.400 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.395+0000 7f23a60c8640 1 -- 172.21.15.27:0/832636577 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f238c003520 con 0x7f238c0039e0 2024-09-12T03:24:07.400 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.396+0000 7f23a60c8640 1 --2- 172.21.15.27:0/832636577 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f238c0039e0 0x7f238c003e00 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7f239c009a90 tx=0x7f238c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5061a56f340e906 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:07.400 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.396+0000 7f23a50c6640 1 -- 172.21.15.27:0/832636577 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f239c036070 con 0x7f238c0039e0 2024-09-12T03:24:07.400 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.396+0000 7f23a50c6640 1 -- 172.21.15.27:0/832636577 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f239c031400 con 0x7f238c0039e0 2024-09-12T03:24:07.400 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.396+0000 7f23a50c6640 1 -- 172.21.15.27:0/832636577 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f239c009690 con 0x7f238c0039e0 2024-09-12T03:24:07.400 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.396+0000 7f23a832c640 1 -- 172.21.15.27:0/832636577 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f238c0039e0 msgr2=0x7f238c003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:07.400 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.396+0000 7f23a832c640 1 --2- 172.21.15.27:0/832636577 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f238c0039e0 0x7f238c003e00 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7f239c009a90 tx=0x7f238c005830 comp rx=0 tx=0).stop 2024-09-12T03:24:07.400 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.396+0000 7f23a68c9640 1 -- 172.21.15.27:0/832636577 reap_dead start 2024-09-12T03:24:07.400 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.397+0000 7f23a832c640 1 -- 172.21.15.27:0/832636577 shutdown_connections 2024-09-12T03:24:07.400 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.397+0000 7f23a832c640 1 -- 172.21.15.27:0/832636577 >> 172.21.15.27:0/832636577 conn(0x7f23a006ca30 msgr2=0x7f23a006ce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:07.400 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.397+0000 7f23a832c640 1 -- 172.21.15.27:0/832636577 shutdown_connections 2024-09-12T03:24:07.401 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.397+0000 7f23a832c640 1 -- 172.21.15.27:0/832636577 wait complete. 2024-09-12T03:24:07.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.398+0000 7f23a832c640 1 Processor -- start 2024-09-12T03:24:07.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.398+0000 7f23a832c640 1 -- start start 2024-09-12T03:24:07.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.398+0000 7f23a832c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f238c0039e0 0x7f23a0156f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:07.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.398+0000 7f23a832c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f23a0155920 0x7f23a0154fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:07.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.398+0000 7f23a832c640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f23a014f8e0 0x7f23a015ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:07.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.398+0000 7f23a832c640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f23a0070110 con 0x7f238c0039e0 2024-09-12T03:24:07.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.398+0000 7f23a832c640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f23a0073520 con 0x7f23a0155920 2024-09-12T03:24:07.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.398+0000 7f23a832c640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f23a00b02a0 con 0x7f23a014f8e0 2024-09-12T03:24:07.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.399+0000 7f23a60c8640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f238c0039e0 0x7f23a0156f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:07.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.399+0000 7f23a60c8640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f238c0039e0 0x7f23a0156f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33198/0 (socket says 172.21.15.27:33198) 2024-09-12T03:24:07.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.399+0000 7f23a60c8640 1 -- 172.21.15.27:0/3932215311 learned_addr learned my addr 172.21.15.27:0/3932215311 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:24:07.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.399+0000 7f23a58c7640 1 --2- 172.21.15.27:0/3932215311 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f23a0155920 0x7f23a0154fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:07.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.399+0000 7f23a68c9640 1 --2- 172.21.15.27:0/3932215311 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f23a014f8e0 0x7f23a015ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:07.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.399+0000 7f23a60c8640 1 -- 172.21.15.27:0/3932215311 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f23a014f8e0 msgr2=0x7f23a015ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:07.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.399+0000 7f23a60c8640 1 --2- 172.21.15.27:0/3932215311 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f23a014f8e0 0x7f23a015ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:07.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.399+0000 7f23a60c8640 1 -- 172.21.15.27:0/3932215311 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f23a0155920 msgr2=0x7f23a0154fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:07.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.399+0000 7f23a60c8640 1 --2- 172.21.15.27:0/3932215311 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f23a0155920 0x7f23a0154fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:07.405 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.399+0000 7f23a60c8640 1 -- 172.21.15.27:0/3932215311 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23a0155ee0 con 0x7f238c0039e0 2024-09-12T03:24:07.405 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.399+0000 7f23a68c9640 1 --2- 172.21.15.27:0/3932215311 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f23a014f8e0 0x7f23a015ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:24:07.405 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.399+0000 7f23a60c8640 1 --2- 172.21.15.27:0/3932215311 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f238c0039e0 0x7f23a0156f80 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7f239c034040 tx=0x7f239c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:07.405 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.399+0000 7f2396ffd640 1 -- 172.21.15.27:0/3932215311 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f239c040020 con 0x7f238c0039e0 2024-09-12T03:24:07.405 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.399+0000 7f2396ffd640 1 -- 172.21.15.27:0/3932215311 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f239c003bc0 con 0x7f238c0039e0 2024-09-12T03:24:07.405 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.399+0000 7f23a832c640 1 -- 172.21.15.27:0/3932215311 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23a01561c0 con 0x7f238c0039e0 2024-09-12T03:24:07.405 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.399+0000 7f2396ffd640 1 -- 172.21.15.27:0/3932215311 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f239c036040 con 0x7f238c0039e0 2024-09-12T03:24:07.405 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.399+0000 7f23a832c640 1 -- 172.21.15.27:0/3932215311 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23a0156780 con 0x7f238c0039e0 2024-09-12T03:24:07.405 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.400+0000 7f23a832c640 1 -- 172.21.15.27:0/3932215311 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23a006dac0 con 0x7f238c0039e0 2024-09-12T03:24:07.405 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.401+0000 7f2396ffd640 1 -- 172.21.15.27:0/3932215311 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 26) ==== 91518+0+0 (secure 0 0 0) 0x7f239c043020 con 0x7f238c0039e0 2024-09-12T03:24:07.405 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.401+0000 7f2396ffd640 1 --2- 172.21.15.27:0/3932215311 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f238406c970 0x7f238406ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:07.405 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.401+0000 7f2396ffd640 1 -- 172.21.15.27:0/3932215311 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(160..160 src has 1..160) ==== 5765+0+0 (secure 0 0 0) 0x7f239c0a9b40 con 0x7f238c0039e0 2024-09-12T03:24:07.405 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.401+0000 7f23a58c7640 1 --2- 172.21.15.27:0/3932215311 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f238406c970 0x7f238406ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:07.405 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.402+0000 7f23a58c7640 1 --2- 172.21.15.27:0/3932215311 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f238406c970 0x7f238406ee30 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f23a015fda0 tx=0x7f2390009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:07.406 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.405+0000 7f2396ffd640 1 -- 172.21.15.27:0/3932215311 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f239c04a030 con 0x7f238c0039e0 2024-09-12T03:24:07.616 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.615+0000 7f23a832c640 1 -- 172.21.15.27:0/3932215311 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f23a0159c80 con 0x7f238c0039e0 2024-09-12T03:24:07.617 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.616+0000 7f2396ffd640 1 -- 172.21.15.27:0/3932215311 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 100 v100) ==== 95+0+5228 (secure 0 0 0) 0x7f239c0044e0 con 0x7f238c0039e0 2024-09-12T03:24:07.617 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:24:07.617 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":100,"btime":"2024-09-12T03:24:03:282449+0000","default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7163,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/2493793567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2493793567},{"type":"v1","addr":"172.21.15.45:6835","nonce":2493793567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":96},{"gid":8056,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/1643775717","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":1643775717},{"type":"v1","addr":"172.21.15.45:6834","nonce":1643775717}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95},{"gid":8967,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6834/4129691413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":4129691413},{"type":"v1","addr":"172.21.15.27:6834","nonce":4129691413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95}],"filesystems":[{"mdsmap":{"epoch":99,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-12T03:23:54.606856+0000","modified":"2024-09-12T03:23:56.609988+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8970},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8970":{"gid":8970,"name":"c","rank":0,"incarnation":97,"state":"up:active","state_seq":3,"addr":"172.21.15.27:6835/2940099359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":2940099359},{"type":"v1","addr":"172.21.15.27:6835","nonce":2940099359}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[27],"metadata_pool":26,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8970,"qdb_cluster":[8970]},"id":13}]} 2024-09-12T03:24:07.617 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 100 2024-09-12T03:24:07.619 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.618+0000 7f23a832c640 1 -- 172.21.15.27:0/3932215311 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f238406c970 msgr2=0x7f238406ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:07.619 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.618+0000 7f23a832c640 1 --2- 172.21.15.27:0/3932215311 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f238406c970 0x7f238406ee30 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f23a015fda0 tx=0x7f2390009040 comp rx=0 tx=0).stop 2024-09-12T03:24:07.619 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.618+0000 7f23a832c640 1 -- 172.21.15.27:0/3932215311 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f238c0039e0 msgr2=0x7f23a0156f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:07.619 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.618+0000 7f23a832c640 1 --2- 172.21.15.27:0/3932215311 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f238c0039e0 0x7f23a0156f80 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7f239c034040 tx=0x7f239c004730 comp rx=0 tx=0).stop 2024-09-12T03:24:07.619 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.618+0000 7f23a832c640 1 -- 172.21.15.27:0/3932215311 shutdown_connections 2024-09-12T03:24:07.620 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.618+0000 7f23a832c640 1 --2- 172.21.15.27:0/3932215311 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f23a014f8e0 0x7f23a015ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:07.620 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.618+0000 7f23a832c640 1 --2- 172.21.15.27:0/3932215311 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f23a0155920 0x7f23a0154fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:07.620 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.618+0000 7f23a832c640 1 --2- 172.21.15.27:0/3932215311 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f238406c970 0x7f238406ee30 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:07.620 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.618+0000 7f23a832c640 1 --2- 172.21.15.27:0/3932215311 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f238c0039e0 0x7f23a0156f80 unknown :-1 s=CLOSED pgs=1740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:07.620 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.618+0000 7f23a832c640 1 -- 172.21.15.27:0/3932215311 >> 172.21.15.27:0/3932215311 conn(0x7f23a006ca30 msgr2=0x7f23a00b2160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:07.620 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.618+0000 7f23a832c640 1 -- 172.21.15.27:0/3932215311 shutdown_connections 2024-09-12T03:24:07.620 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.618+0000 7f23a832c640 1 -- 172.21.15.27:0/3932215311 wait complete. 2024-09-12T03:24:07.630 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-12T03:24:07.732 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.731+0000 7f62a4c80640 1 Processor -- start 2024-09-12T03:24:07.733 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.731+0000 7f62a4c80640 1 -- start start 2024-09-12T03:24:07.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.732+0000 7f62a4c80640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f62a0073960 0x7f62a0162320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:07.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.732+0000 7f62a4c80640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f62a0162860 0x7f62a015e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:07.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.732+0000 7f62a4c80640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f62a00d2240 con 0x7f62a015e5f0 2024-09-12T03:24:07.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.732+0000 7f62a4c80640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f62a00d0da0 con 0x7f62a0073960 2024-09-12T03:24:07.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.732+0000 7f62a4c80640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f62a0163090 con 0x7f62a0162860 2024-09-12T03:24:07.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.733+0000 7f629e59c640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f62a015e5f0 0x7f62a0073250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:57580/0 (socket says 172.21.15.27:57580) 2024-09-12T03:24:07.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.733+0000 7f629e59c640 1 -- 172.21.15.27:0/3394171084 learned_addr learned my addr 172.21.15.27:0/3394171084 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:24:07.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.733+0000 7f629ed9d640 1 --2- 172.21.15.27:0/3394171084 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f62a0162860 0x7f62a015e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:07.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.733+0000 7f629d59a640 1 -- 172.21.15.27:0/3394171084 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4145084910 0 0) 0x7f62a00d2240 con 0x7f62a015e5f0 2024-09-12T03:24:07.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.733+0000 7f629dd9b640 1 --2- 172.21.15.27:0/3394171084 >> v2:172.21.15.45:3300/0 conn(0x7f62a0073960 0x7f62a0162320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:07.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.733+0000 7f629d59a640 1 -- 172.21.15.27:0/3394171084 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f628c003520 con 0x7f62a015e5f0 2024-09-12T03:24:07.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.733+0000 7f629d59a640 1 -- 172.21.15.27:0/3394171084 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3416593011 0 0) 0x7f628c003520 con 0x7f62a015e5f0 2024-09-12T03:24:07.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.733+0000 7f629d59a640 1 -- 172.21.15.27:0/3394171084 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f62a0162860 msgr2=0x7f62a015e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:24:07.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.733+0000 7f629d59a640 1 --2- 172.21.15.27:0/3394171084 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f62a0162860 0x7f62a015e9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:07.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.733+0000 7f629d59a640 1 -- 172.21.15.27:0/3394171084 >> v2:172.21.15.45:3300/0 conn(0x7f62a0073960 msgr2=0x7f62a0162320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:24:07.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.733+0000 7f629d59a640 1 --2- 172.21.15.27:0/3394171084 >> v2:172.21.15.45:3300/0 conn(0x7f62a0073960 0x7f62a0162320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:07.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.733+0000 7f629d59a640 1 -- 172.21.15.27:0/3394171084 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62a00d0470 con 0x7f62a015e5f0 2024-09-12T03:24:07.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.733+0000 7f629d59a640 1 -- 172.21.15.27:0/3394171084 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 847755213 0 0) 0x7f62900019f0 con 0x7f62a015e5f0 2024-09-12T03:24:07.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.733+0000 7f629d59a640 1 -- 172.21.15.27:0/3394171084 >> v1:172.21.15.27:6789/0 conn(0x7f62a015e5f0 legacy=0x7f62a0073250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:07.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.733+0000 7f629d59a640 1 --2- 172.21.15.27:0/3394171084 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f628c0039b0 0x7f628c003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:07.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.733+0000 7f629d59a640 1 --2- 172.21.15.27:0/3394171084 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f628c004800 0x7f628c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:07.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.733+0000 7f629d59a640 1 --2- 172.21.15.27:0/3394171084 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f62a0162860 0x7f628c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:07.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.733+0000 7f629d59a640 1 -- 172.21.15.27:0/3394171084 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f62a00d0da0 con 0x7f628c004800 2024-09-12T03:24:07.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.733+0000 7f629d59a640 1 -- 172.21.15.27:0/3394171084 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f62a0163090 con 0x7f628c0039b0 2024-09-12T03:24:07.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.733+0000 7f629d59a640 1 -- 172.21.15.27:0/3394171084 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f628c00b9f0 con 0x7f62a0162860 2024-09-12T03:24:07.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.733+0000 7f629dd9b640 1 --2- 172.21.15.27:0/3394171084 >> v2:172.21.15.45:3300/0 conn(0x7f62a0073960 0x7f62a0162320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:24:07.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.733+0000 7f629e59c640 1 --2- 172.21.15.27:0/3394171084 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f628c0039b0 0x7f628c003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:07.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.734+0000 7f629dd9b640 1 --2- 172.21.15.27:0/3394171084 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f628c004800 0x7f628c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:07.738 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.734+0000 7f629ed9d640 1 --2- 172.21.15.27:0/3394171084 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f62a0162860 0x7f628c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:07.738 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.734+0000 7f629dd9b640 1 -- 172.21.15.27:0/3394171084 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f62a0162860 msgr2=0x7f628c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:24:07.738 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.734+0000 7f629dd9b640 1 --2- 172.21.15.27:0/3394171084 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f62a0162860 0x7f628c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:07.738 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.734+0000 7f629dd9b640 1 -- 172.21.15.27:0/3394171084 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f628c0039b0 msgr2=0x7f628c003df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:24:07.738 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.734+0000 7f629dd9b640 1 --2- 172.21.15.27:0/3394171084 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f628c0039b0 0x7f628c003df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:07.738 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.734+0000 7f629dd9b640 1 -- 172.21.15.27:0/3394171084 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f628c003520 con 0x7f628c004800 2024-09-12T03:24:07.738 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.734+0000 7f629dd9b640 1 --2- 172.21.15.27:0/3394171084 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f628c004800 0x7f628c004c80 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7f62880098a0 tx=0x7f628c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f567f06c20169114 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:07.738 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.735+0000 7f629d59a640 1 -- 172.21.15.27:0/3394171084 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6288037070 con 0x7f628c004800 2024-09-12T03:24:07.738 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.735+0000 7f629d59a640 1 -- 172.21.15.27:0/3394171084 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f628802ad10 con 0x7f628c004800 2024-09-12T03:24:07.738 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.735+0000 7f629d59a640 1 -- 172.21.15.27:0/3394171084 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f628800f040 con 0x7f628c004800 2024-09-12T03:24:07.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.735+0000 7f62a4c80640 1 -- 172.21.15.27:0/3394171084 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f628c004800 msgr2=0x7f628c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:07.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.735+0000 7f62a4c80640 1 --2- 172.21.15.27:0/3394171084 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f628c004800 0x7f628c004c80 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7f62880098a0 tx=0x7f628c005830 comp rx=0 tx=0).stop 2024-09-12T03:24:07.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.735+0000 7f629ed9d640 1 -- 172.21.15.27:0/3394171084 reap_dead start 2024-09-12T03:24:07.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.735+0000 7f62a4c80640 1 -- 172.21.15.27:0/3394171084 shutdown_connections 2024-09-12T03:24:07.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.735+0000 7f62a4c80640 1 -- 172.21.15.27:0/3394171084 >> 172.21.15.27:0/3394171084 conn(0x7f62a006f300 msgr2=0x7f62a006f710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:07.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.735+0000 7f62a4c80640 1 -- 172.21.15.27:0/3394171084 shutdown_connections 2024-09-12T03:24:07.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.735+0000 7f62a4c80640 1 -- 172.21.15.27:0/3394171084 wait complete. 2024-09-12T03:24:07.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.737+0000 7f62a4c80640 1 Processor -- start 2024-09-12T03:24:07.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.737+0000 7f62a4c80640 1 -- start start 2024-09-12T03:24:07.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.737+0000 7f62a4c80640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f628c004800 0x7f62a0153120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:07.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.737+0000 7f62a4c80640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f62a00cd210 0x7f62a015a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:07.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.738+0000 7f62a4c80640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f62a0148800 0x7f62a00ce7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:07.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.738+0000 7f62a4c80640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f62a00b5620 con 0x7f628c004800 2024-09-12T03:24:07.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.738+0000 7f62a4c80640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f62a00d0a30 con 0x7f62a0148800 2024-09-12T03:24:07.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.738+0000 7f62a4c80640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f62a0162cb0 con 0x7f62a00cd210 2024-09-12T03:24:07.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.738+0000 7f629e59c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f628c004800 0x7f62a0153120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:07.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.738+0000 7f629e59c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f628c004800 0x7f62a0153120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33218/0 (socket says 172.21.15.27:33218) 2024-09-12T03:24:07.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.738+0000 7f629e59c640 1 -- 172.21.15.27:0/379686615 learned_addr learned my addr 172.21.15.27:0/379686615 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:24:07.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.738+0000 7f629e59c640 1 -- 172.21.15.27:0/379686615 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f62a00cd210 msgr2=0x7f62a015a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:07.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.738+0000 7f629dd9b640 1 --2- 172.21.15.27:0/379686615 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f62a00cd210 0x7f62a015a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:07.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.738+0000 7f629ed9d640 1 --2- 172.21.15.27:0/379686615 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f62a0148800 0x7f62a00ce7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:07.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.738+0000 7f629e59c640 1 --2- 172.21.15.27:0/379686615 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f62a00cd210 0x7f62a015a760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:07.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.738+0000 7f629e59c640 1 -- 172.21.15.27:0/379686615 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f62a0148800 msgr2=0x7f62a00ce7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:07.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.738+0000 7f629e59c640 1 --2- 172.21.15.27:0/379686615 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f62a0148800 0x7f62a00ce7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:07.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.738+0000 7f629e59c640 1 -- 172.21.15.27:0/379686615 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62a00cd710 con 0x7f628c004800 2024-09-12T03:24:07.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.738+0000 7f629e59c640 1 --2- 172.21.15.27:0/379686615 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f628c004800 0x7f62a0153120 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 crypto rx=0x7f62900077f0 tx=0x7f629000a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:07.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.738+0000 7f6296ffd640 1 -- 172.21.15.27:0/379686615 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62900133f0 con 0x7f628c004800 2024-09-12T03:24:07.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.738+0000 7f6296ffd640 1 -- 172.21.15.27:0/379686615 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6290018070 con 0x7f628c004800 2024-09-12T03:24:07.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.738+0000 7f62a4c80640 1 -- 172.21.15.27:0/379686615 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62a00cd9f0 con 0x7f628c004800 2024-09-12T03:24:07.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.738+0000 7f6296ffd640 1 -- 172.21.15.27:0/379686615 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62900137f0 con 0x7f628c004800 2024-09-12T03:24:07.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.738+0000 7f62a4c80640 1 -- 172.21.15.27:0/379686615 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62a00cdfb0 con 0x7f628c004800 2024-09-12T03:24:07.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.739+0000 7f62a4c80640 1 -- 172.21.15.27:0/379686615 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62a0162320 con 0x7f628c004800 2024-09-12T03:24:07.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.740+0000 7f6296ffd640 1 -- 172.21.15.27:0/379686615 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 26) ==== 91518+0+0 (secure 0 0 0) 0x7f6290013990 con 0x7f628c004800 2024-09-12T03:24:07.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.740+0000 7f6296ffd640 1 --2- 172.21.15.27:0/379686615 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f628006c9a0 0x7f628006ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:07.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.741+0000 7f6296ffd640 1 -- 172.21.15.27:0/379686615 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(160..160 src has 1..160) ==== 5765+0+0 (secure 0 0 0) 0x7f6290089f90 con 0x7f628c004800 2024-09-12T03:24:07.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.741+0000 7f629dd9b640 1 --2- 172.21.15.27:0/379686615 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f628006c9a0 0x7f628006ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:07.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.741+0000 7f629dd9b640 1 --2- 172.21.15.27:0/379686615 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f628006c9a0 0x7f628006ee60 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f6288034000 tx=0x7f628802e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:07.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.745+0000 7f6296ffd640 1 -- 172.21.15.27:0/379686615 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f629005a3f0 con 0x7f628c004800 2024-09-12T03:24:07.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.919+0000 7f62a4c80640 1 -- 172.21.15.27:0/379686615 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f62a0159840 con 0x7f628c004800 2024-09-12T03:24:07.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.920+0000 7f6296ffd640 1 -- 172.21.15.27:0/379686615 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v160) ==== 74+0+16784 (secure 0 0 0) 0x7f6290059cf0 con 0x7f628c004800 2024-09-12T03:24:07.921 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:24:07.922 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":160,"fsid":"d2df504e-2647-4385-a5db-9fdf69f487a0","created":"2024-09-12T03:16:07.778937+0000","modified":"2024-09-12T03:23:56.250469+0000","last_up_change":"2024-09-12T03:16:19.314778+0000","last_in_change":"2024-09-12T03:16:09.893086+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T03:16:20.097646+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs_metadata","create_time":"2024-09-12T03:23:52.574839+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"159","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_data","create_time":"2024-09-12T03:23:53.596556+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"160","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1851116c-1fb3-4dc9-9772-f90e2ef4c355","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":158,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6817","nonce":3002321488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6819","nonce":3002321488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6823","nonce":3002321488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6821","nonce":3002321488}]},"public_addr":"172.21.15.27:6817/3002321488","cluster_addr":"172.21.15.27:6819/3002321488","heartbeat_back_addr":"172.21.15.27:6823/3002321488","heartbeat_front_addr":"172.21.15.27:6821/3002321488","state":["exists","up"]},{"osd":1,"uuid":"f38ae17d-73eb-4204-b99d-975fa806fb87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":158,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6809","nonce":1613102886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6811","nonce":1613102886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6815","nonce":1613102886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6813","nonce":1613102886}]},"public_addr":"172.21.15.27:6809/1613102886","cluster_addr":"172.21.15.27:6811/1613102886","heartbeat_back_addr":"172.21.15.27:6815/1613102886","heartbeat_front_addr":"172.21.15.27:6813/1613102886","state":["exists","up"]},{"osd":2,"uuid":"fb12dcc8-bdd0-42b8-b46d-8b10106224bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":158,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6825","nonce":3534723355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6827","nonce":3534723355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6831","nonce":3534723355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6829","nonce":3534723355}]},"public_addr":"172.21.15.27:6825/3534723355","cluster_addr":"172.21.15.27:6827/3534723355","heartbeat_back_addr":"172.21.15.27:6831/3534723355","heartbeat_front_addr":"172.21.15.27:6829/3534723355","state":["exists","up"]},{"osd":3,"uuid":"d74d3f49-9c24-4b9f-98e2-3ecd7b5ef3b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":158,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6801","nonce":80661598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6803","nonce":80661598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6807","nonce":80661598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6805","nonce":80661598}]},"public_addr":"172.21.15.27:6801/80661598","cluster_addr":"172.21.15.27:6803/80661598","heartbeat_back_addr":"172.21.15.27:6807/80661598","heartbeat_front_addr":"172.21.15.27:6805/80661598","state":["exists","up"]},{"osd":4,"uuid":"2bf123f2-1b0d-41c4-aa3b-805bab267e59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":158,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6817","nonce":3943189827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6819","nonce":3943189827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6823","nonce":3943189827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6821","nonce":3943189827}]},"public_addr":"172.21.15.45:6817/3943189827","cluster_addr":"172.21.15.45:6819/3943189827","heartbeat_back_addr":"172.21.15.45:6823/3943189827","heartbeat_front_addr":"172.21.15.45:6821/3943189827","state":["exists","up"]},{"osd":5,"uuid":"c49ba5da-91c6-4a86-a97b-4aba89c253ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":158,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6809","nonce":3620760330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6811","nonce":3620760330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6815","nonce":3620760330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6813","nonce":3620760330}]},"public_addr":"172.21.15.45:6809/3620760330","cluster_addr":"172.21.15.45:6811/3620760330","heartbeat_back_addr":"172.21.15.45:6815/3620760330","heartbeat_front_addr":"172.21.15.45:6813/3620760330","state":["exists","up"]},{"osd":6,"uuid":"b0123475-89f7-4a34-9e33-d797b99dd9dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":158,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6801","nonce":1989083510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6803","nonce":1989083510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6807","nonce":1989083510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6805","nonce":1989083510}]},"public_addr":"172.21.15.45:6801/1989083510","cluster_addr":"172.21.15.45:6803/1989083510","heartbeat_back_addr":"172.21.15.45:6807/1989083510","heartbeat_front_addr":"172.21.15.45:6805/1989083510","state":["exists","up"]},{"osd":7,"uuid":"4dad773d-b7d8-46fc-8a10-a972aad27b29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":158,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6825","nonce":1513300294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6827","nonce":1513300294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6831","nonce":1513300294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6829","nonce":1513300294}]},"public_addr":"172.21.15.45:6825/1513300294","cluster_addr":"172.21.15.45:6827/1513300294","heartbeat_back_addr":"172.21.15.45:6831/1513300294","heartbeat_front_addr":"172.21.15.45:6829/1513300294","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.881855+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:17.133252+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.735339+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.500742+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.477571+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:17.167667+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.470291+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.873743+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]},{"pool":19,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T03:24:07.923 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.922+0000 7f62a4c80640 1 -- 172.21.15.27:0/379686615 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f628006c9a0 msgr2=0x7f628006ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:07.923 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.922+0000 7f62a4c80640 1 --2- 172.21.15.27:0/379686615 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f628006c9a0 0x7f628006ee60 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f6288034000 tx=0x7f628802e000 comp rx=0 tx=0).stop 2024-09-12T03:24:07.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.922+0000 7f62a4c80640 1 -- 172.21.15.27:0/379686615 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f628c004800 msgr2=0x7f62a0153120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:07.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.922+0000 7f62a4c80640 1 --2- 172.21.15.27:0/379686615 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f628c004800 0x7f62a0153120 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 crypto rx=0x7f62900077f0 tx=0x7f629000a730 comp rx=0 tx=0).stop 2024-09-12T03:24:07.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.922+0000 7f62a4c80640 1 -- 172.21.15.27:0/379686615 shutdown_connections 2024-09-12T03:24:07.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.922+0000 7f62a4c80640 1 --2- 172.21.15.27:0/379686615 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f62a0148800 0x7f62a00ce7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:07.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.922+0000 7f62a4c80640 1 --2- 172.21.15.27:0/379686615 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f62a00cd210 0x7f62a015a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:07.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.922+0000 7f62a4c80640 1 --2- 172.21.15.27:0/379686615 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f628006c9a0 0x7f628006ee60 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:07.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.922+0000 7f62a4c80640 1 --2- 172.21.15.27:0/379686615 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f628c004800 0x7f62a0153120 unknown :-1 s=CLOSED pgs=1743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:07.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.922+0000 7f62a4c80640 1 -- 172.21.15.27:0/379686615 >> 172.21.15.27:0/379686615 conn(0x7f62a006f300 msgr2=0x7f62a00b6cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:07.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.922+0000 7f62a4c80640 1 -- 172.21.15.27:0/379686615 shutdown_connections 2024-09-12T03:24:07.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:07.923+0000 7f62a4c80640 1 -- 172.21.15.27:0/379686615 wait complete. 2024-09-12T03:24:07.935 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-12T03:24:07.935 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-12T03:24:08.038 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.037+0000 7f2779386640 1 Processor -- start 2024-09-12T03:24:08.038 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.037+0000 7f2779386640 1 -- start start 2024-09-12T03:24:08.039 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.038+0000 7f2779386640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f27740d1050 0x7f27740cf810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:08.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.038+0000 7f2779386640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f27740d3120 0x7f27740d35b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:08.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.038+0000 7f2779386640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f27740d1e60 con 0x7f27740d52e0 2024-09-12T03:24:08.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.038+0000 7f2779386640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f27740d2380 con 0x7f27740d1050 2024-09-12T03:24:08.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.038+0000 7f2779386640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f27740cfef0 con 0x7f27740d3120 2024-09-12T03:24:08.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.038+0000 7f2772ffd640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f27740d1050 0x7f27740cf810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:08.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.038+0000 7f2772ffd640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f27740d1050 0x7f27740cf810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33224/0 (socket says 172.21.15.27:33224) 2024-09-12T03:24:08.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.038+0000 7f2772ffd640 1 -- 172.21.15.27:0/2432186416 learned_addr learned my addr 172.21.15.27:0/2432186416 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:24:08.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.038+0000 7f27737fe640 1 --2- 172.21.15.27:0/2432186416 >> v2:172.21.15.45:3300/0 conn(0x7f27740d3120 0x7f27740d35b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:08.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.038+0000 7f2772ffd640 1 -- 172.21.15.27:0/2432186416 >> v2:172.21.15.45:3300/0 conn(0x7f27740d3120 msgr2=0x7f27740d35b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:24:08.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.038+0000 7f2772ffd640 1 --2- 172.21.15.27:0/2432186416 >> v2:172.21.15.45:3300/0 conn(0x7f27740d3120 0x7f27740d35b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:08.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.038+0000 7f2772ffd640 1 -- 172.21.15.27:0/2432186416 >> v1:172.21.15.45:6789/0 conn(0x7f27740d52e0 legacy=0x7f27740d1430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:08.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.038+0000 7f2772ffd640 1 -- 172.21.15.27:0/2432186416 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27740d2640 con 0x7f27740d1050 2024-09-12T03:24:08.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.039+0000 7f2772ffd640 1 --2- 172.21.15.27:0/2432186416 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f27740d1050 0x7f27740cf810 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x7f27740ced90 tx=0x7f276402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcee7734a3f1634f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:08.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.039+0000 7f2771ffb640 1 -- 172.21.15.27:0/2432186416 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2764037070 con 0x7f27740d1050 2024-09-12T03:24:08.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.039+0000 7f2771ffb640 1 -- 172.21.15.27:0/2432186416 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2764032440 con 0x7f27740d1050 2024-09-12T03:24:08.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.039+0000 7f2771ffb640 1 -- 172.21.15.27:0/2432186416 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2764032810 con 0x7f27740d1050 2024-09-12T03:24:08.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.040+0000 7f2779386640 1 -- 172.21.15.27:0/2432186416 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f27740d1050 msgr2=0x7f27740cf810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:08.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.040+0000 7f2779386640 1 --2- 172.21.15.27:0/2432186416 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f27740d1050 0x7f27740cf810 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x7f27740ced90 tx=0x7f276402a270 comp rx=0 tx=0).stop 2024-09-12T03:24:08.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.040+0000 7f2779386640 1 -- 172.21.15.27:0/2432186416 shutdown_connections 2024-09-12T03:24:08.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.040+0000 7f2779386640 1 --2- 172.21.15.27:0/2432186416 >> v2:172.21.15.45:3300/0 conn(0x7f27740d3120 0x7f27740d35b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:08.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.040+0000 7f2779386640 1 --2- 172.21.15.27:0/2432186416 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f27740d1050 0x7f27740cf810 unknown :-1 s=CLOSED pgs=1744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:08.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.040+0000 7f2779386640 1 -- 172.21.15.27:0/2432186416 >> 172.21.15.27:0/2432186416 conn(0x7f277406ddf0 msgr2=0x7f277406e200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:08.043 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.040+0000 7f2779386640 1 -- 172.21.15.27:0/2432186416 shutdown_connections 2024-09-12T03:24:08.043 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.040+0000 7f2779386640 1 -- 172.21.15.27:0/2432186416 wait complete. 2024-09-12T03:24:08.043 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.041+0000 7f2779386640 1 Processor -- start 2024-09-12T03:24:08.043 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.041+0000 7f2779386640 1 -- start start 2024-09-12T03:24:08.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.042+0000 7f2779386640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f27740d1050 0x7f277415e140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:08.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.042+0000 7f2779386640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f27740d3120 0x7f27740c8990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:08.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.042+0000 7f2779386640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f27740d52e0 0x7f27740ce010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:08.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.042+0000 7f2779386640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f27740b7a50 con 0x7f27740d52e0 2024-09-12T03:24:08.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.042+0000 7f2779386640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f27740d2c00 con 0x7f27740d3120 2024-09-12T03:24:08.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.042+0000 7f2779386640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f27740b7bd0 con 0x7f27740d1050 2024-09-12T03:24:08.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.042+0000 7f27737fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f27740d52e0 0x7f27740ce010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:08.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.042+0000 7f27737fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f27740d52e0 0x7f27740ce010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33230/0 (socket says 172.21.15.27:33230) 2024-09-12T03:24:08.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.042+0000 7f27737fe640 1 -- 172.21.15.27:0/4222878816 learned_addr learned my addr 172.21.15.27:0/4222878816 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:24:08.046 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.042+0000 7f2772ffd640 1 --2- 172.21.15.27:0/4222878816 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f27740d1050 0x7f277415e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:08.046 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.042+0000 7f27727fc640 1 --2- 172.21.15.27:0/4222878816 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f27740d3120 0x7f27740c8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:08.046 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.042+0000 7f27737fe640 1 -- 172.21.15.27:0/4222878816 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f27740d1050 msgr2=0x7f277415e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:08.046 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.042+0000 7f27737fe640 1 --2- 172.21.15.27:0/4222878816 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f27740d1050 0x7f277415e140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:08.046 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.042+0000 7f27737fe640 1 -- 172.21.15.27:0/4222878816 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f27740d3120 msgr2=0x7f27740c8990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:08.046 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.042+0000 7f27737fe640 1 --2- 172.21.15.27:0/4222878816 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f27740d3120 0x7f27740c8990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:08.046 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.042+0000 7f27737fe640 1 -- 172.21.15.27:0/4222878816 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27740c9450 con 0x7f27740d52e0 2024-09-12T03:24:08.046 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.042+0000 7f27737fe640 1 --2- 172.21.15.27:0/4222878816 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f27740d52e0 0x7f27740ce010 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7f276c007e40 tx=0x7f276c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:08.046 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.043+0000 7f275b7fe640 1 -- 172.21.15.27:0/4222878816 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f276c014620 con 0x7f27740d52e0 2024-09-12T03:24:08.046 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.043+0000 7f275b7fe640 1 -- 172.21.15.27:0/4222878816 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f276c019070 con 0x7f27740d52e0 2024-09-12T03:24:08.046 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.043+0000 7f2779386640 1 -- 172.21.15.27:0/4222878816 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27740c96f0 con 0x7f27740d52e0 2024-09-12T03:24:08.046 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.043+0000 7f275b7fe640 1 -- 172.21.15.27:0/4222878816 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f276c014a20 con 0x7f27740d52e0 2024-09-12T03:24:08.046 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.043+0000 7f2779386640 1 -- 172.21.15.27:0/4222878816 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2774151540 con 0x7f27740d52e0 2024-09-12T03:24:08.046 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.044+0000 7f2779386640 1 -- 172.21.15.27:0/4222878816 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27740d0630 con 0x7f27740d52e0 2024-09-12T03:24:08.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.044+0000 7f275b7fe640 1 -- 172.21.15.27:0/4222878816 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 26) ==== 91518+0+0 (secure 0 0 0) 0x7f276c014bc0 con 0x7f27740d52e0 2024-09-12T03:24:08.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.045+0000 7f275b7fe640 1 --2- 172.21.15.27:0/4222878816 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f276006c9a0 0x7f276006ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:08.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.045+0000 7f275b7fe640 1 -- 172.21.15.27:0/4222878816 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(160..160 src has 1..160) ==== 5765+0+0 (secure 0 0 0) 0x7f276c089e80 con 0x7f27740d52e0 2024-09-12T03:24:08.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.045+0000 7f2772ffd640 1 --2- 172.21.15.27:0/4222878816 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f276006c9a0 0x7f276006ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:08.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.045+0000 7f2772ffd640 1 --2- 172.21.15.27:0/4222878816 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f276006c9a0 0x7f276006ee60 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f2764034000 tx=0x7f276402e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:08.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.049+0000 7f275b7fe640 1 -- 172.21.15.27:0/4222878816 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f27740d0630 con 0x7f27740d52e0 2024-09-12T03:24:08.258 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.258+0000 7f2779386640 1 -- 172.21.15.27:0/4222878816 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f277415ec60 con 0x7f27740d52e0 2024-09-12T03:24:08.259 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.258+0000 7f275b7fe640 1 -- 172.21.15.27:0/4222878816 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 100 v100) ==== 95+0+5228 (secure 0 0 0) 0x7f276c059c90 con 0x7f27740d52e0 2024-09-12T03:24:08.259 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:24:08.259 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":100,"btime":"2024-09-12T03:24:03:282449+0000","default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7163,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/2493793567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2493793567},{"type":"v1","addr":"172.21.15.45:6835","nonce":2493793567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":96},{"gid":8056,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/1643775717","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":1643775717},{"type":"v1","addr":"172.21.15.45:6834","nonce":1643775717}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95},{"gid":8967,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6834/4129691413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":4129691413},{"type":"v1","addr":"172.21.15.27:6834","nonce":4129691413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95}],"filesystems":[{"mdsmap":{"epoch":99,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-12T03:23:54.606856+0000","modified":"2024-09-12T03:23:56.609988+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8970},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8970":{"gid":8970,"name":"c","rank":0,"incarnation":97,"state":"up:active","state_seq":3,"addr":"172.21.15.27:6835/2940099359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":2940099359},{"type":"v1","addr":"172.21.15.27:6835","nonce":2940099359}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[27],"metadata_pool":26,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8970,"qdb_cluster":[8970]},"id":13}]} 2024-09-12T03:24:08.260 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 100 2024-09-12T03:24:08.261 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.260+0000 7f2779386640 1 -- 172.21.15.27:0/4222878816 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f276006c9a0 msgr2=0x7f276006ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:08.261 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.260+0000 7f2779386640 1 --2- 172.21.15.27:0/4222878816 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f276006c9a0 0x7f276006ee60 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f2764034000 tx=0x7f276402e000 comp rx=0 tx=0).stop 2024-09-12T03:24:08.261 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.260+0000 7f2779386640 1 -- 172.21.15.27:0/4222878816 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f27740d52e0 msgr2=0x7f27740ce010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:08.261 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.260+0000 7f2779386640 1 --2- 172.21.15.27:0/4222878816 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f27740d52e0 0x7f27740ce010 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7f276c007e40 tx=0x7f276c00e4b0 comp rx=0 tx=0).stop 2024-09-12T03:24:08.261 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.260+0000 7f2779386640 1 -- 172.21.15.27:0/4222878816 shutdown_connections 2024-09-12T03:24:08.262 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.260+0000 7f2779386640 1 --2- 172.21.15.27:0/4222878816 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f276006c9a0 0x7f276006ee60 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:08.262 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.260+0000 7f2779386640 1 --2- 172.21.15.27:0/4222878816 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f27740d52e0 0x7f27740ce010 unknown :-1 s=CLOSED pgs=1745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:08.262 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.260+0000 7f2779386640 1 --2- 172.21.15.27:0/4222878816 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f27740d3120 0x7f27740c8990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:08.262 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.260+0000 7f2779386640 1 --2- 172.21.15.27:0/4222878816 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f27740d1050 0x7f277415e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:08.262 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.260+0000 7f2779386640 1 -- 172.21.15.27:0/4222878816 >> 172.21.15.27:0/4222878816 conn(0x7f277406ddf0 msgr2=0x7f27740b5470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:08.262 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.260+0000 7f2779386640 1 -- 172.21.15.27:0/4222878816 shutdown_connections 2024-09-12T03:24:08.262 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.260+0000 7f2779386640 1 -- 172.21.15.27:0/4222878816 wait complete. 2024-09-12T03:24:08.272 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-12T03:24:08.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.374+0000 7f8a0813b640 1 Processor -- start 2024-09-12T03:24:08.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.374+0000 7f8a0813b640 1 -- start start 2024-09-12T03:24:08.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.375+0000 7f8a0813b640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f8a0015eb60 0x7f8a000d1280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:08.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.375+0000 7f8a0813b640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f8a000d17c0 0x7f8a000b3390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:08.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.375+0000 7f8a0813b640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8a000b6690 con 0x7f8a0015e780 2024-09-12T03:24:08.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.375+0000 7f8a0813b640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f8a000d1c30 con 0x7f8a000d17c0 2024-09-12T03:24:08.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.375+0000 7f8a0813b640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f8a000b7950 con 0x7f8a0015eb60 2024-09-12T03:24:08.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.375+0000 7f8a066d8640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f8a000d17c0 0x7f8a000b3390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:08.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.375+0000 7f8a066d8640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f8a000d17c0 0x7f8a000b3390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33234/0 (socket says 172.21.15.27:33234) 2024-09-12T03:24:08.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.375+0000 7f8a066d8640 1 -- 172.21.15.27:0/410306220 learned_addr learned my addr 172.21.15.27:0/410306220 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:24:08.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.375+0000 7f8a056d6640 1 --2- 172.21.15.27:0/410306220 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f8a0015eb60 0x7f8a000d1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:08.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.375+0000 7f8a066d8640 1 -- 172.21.15.27:0/410306220 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f8a0015eb60 msgr2=0x7f8a000d1280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:24:08.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.375+0000 7f8a066d8640 1 --2- 172.21.15.27:0/410306220 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f8a0015eb60 0x7f8a000d1280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:08.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.375+0000 7f8a066d8640 1 -- 172.21.15.27:0/410306220 >> v1:172.21.15.45:6789/0 conn(0x7f8a0015e780 legacy=0x7f8a000d3e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:08.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.375+0000 7f8a066d8640 1 -- 172.21.15.27:0/410306220 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a000b6970 con 0x7f8a000d17c0 2024-09-12T03:24:08.379 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.376+0000 7f8a066d8640 1 --2- 172.21.15.27:0/410306220 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8a000d17c0 0x7f8a000b3390 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 crypto rx=0x7f89f8009e50 tx=0x7f89f802a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73370dd67a0081ee server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:08.379 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.376+0000 7f8a04ed5640 1 -- 172.21.15.27:0/410306220 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89f8036070 con 0x7f8a000d17c0 2024-09-12T03:24:08.379 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.376+0000 7f8a04ed5640 1 -- 172.21.15.27:0/410306220 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89f8031600 con 0x7f8a000d17c0 2024-09-12T03:24:08.379 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.376+0000 7f8a04ed5640 1 -- 172.21.15.27:0/410306220 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89f80319d0 con 0x7f8a000d17c0 2024-09-12T03:24:08.379 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.376+0000 7f8a0813b640 1 -- 172.21.15.27:0/410306220 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8a000d17c0 msgr2=0x7f8a000b3390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:08.379 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.376+0000 7f8a0813b640 1 --2- 172.21.15.27:0/410306220 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8a000d17c0 0x7f8a000b3390 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 crypto rx=0x7f89f8009e50 tx=0x7f89f802a710 comp rx=0 tx=0).stop 2024-09-12T03:24:08.379 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.377+0000 7f8a0813b640 1 -- 172.21.15.27:0/410306220 shutdown_connections 2024-09-12T03:24:08.379 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.377+0000 7f8a0813b640 1 --2- 172.21.15.27:0/410306220 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8a000d17c0 0x7f8a000b3390 unknown :-1 s=CLOSED pgs=1746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:08.379 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.377+0000 7f8a0813b640 1 --2- 172.21.15.27:0/410306220 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f8a0015eb60 0x7f8a000d1280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:08.379 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.377+0000 7f8a0813b640 1 -- 172.21.15.27:0/410306220 >> 172.21.15.27:0/410306220 conn(0x7f8a0006da60 msgr2=0x7f8a0006de70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:08.379 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.377+0000 7f8a0813b640 1 -- 172.21.15.27:0/410306220 shutdown_connections 2024-09-12T03:24:08.379 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.377+0000 7f8a0813b640 1 -- 172.21.15.27:0/410306220 wait complete. 2024-09-12T03:24:08.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.378+0000 7f8a0813b640 1 Processor -- start 2024-09-12T03:24:08.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.378+0000 7f8a0813b640 1 -- start start 2024-09-12T03:24:08.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.379+0000 7f8a0813b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8a000d17c0 0x7f8a001590a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:08.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.379+0000 7f8a0813b640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f8a0015e780 0x7f8a000c83d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:08.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.379+0000 7f8a0813b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8a0015eb60 0x7f8a000ca620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:08.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.379+0000 7f8a0813b640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f8a000b6f30 con 0x7f8a000d17c0 2024-09-12T03:24:08.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.379+0000 7f8a0813b640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f8a000b7210 con 0x7f8a0015eb60 2024-09-12T03:24:08.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.379+0000 7f8a0813b640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f8a000b6340 con 0x7f8a0015e780 2024-09-12T03:24:08.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.379+0000 7f8a05ed7640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8a000d17c0 0x7f8a001590a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:08.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.379+0000 7f8a05ed7640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8a000d17c0 0x7f8a001590a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33246/0 (socket says 172.21.15.27:33246) 2024-09-12T03:24:08.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.379+0000 7f8a05ed7640 1 -- 172.21.15.27:0/2255964962 learned_addr learned my addr 172.21.15.27:0/2255964962 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:24:08.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.379+0000 7f8a056d6640 1 --2- 172.21.15.27:0/2255964962 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f8a0015e780 0x7f8a000c83d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:08.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.379+0000 7f8a066d8640 1 --2- 172.21.15.27:0/2255964962 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8a0015eb60 0x7f8a000ca620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:08.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.379+0000 7f8a05ed7640 1 -- 172.21.15.27:0/2255964962 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f8a0015e780 msgr2=0x7f8a000c83d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:08.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.379+0000 7f8a05ed7640 1 --2- 172.21.15.27:0/2255964962 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f8a0015e780 0x7f8a000c83d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:08.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.379+0000 7f8a05ed7640 1 -- 172.21.15.27:0/2255964962 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8a0015eb60 msgr2=0x7f8a000ca620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:08.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.379+0000 7f8a05ed7640 1 --2- 172.21.15.27:0/2255964962 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8a0015eb60 0x7f8a000ca620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:08.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.379+0000 7f8a05ed7640 1 -- 172.21.15.27:0/2255964962 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a000c8d20 con 0x7f8a000d17c0 2024-09-12T03:24:08.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.379+0000 7f8a05ed7640 1 --2- 172.21.15.27:0/2255964962 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8a000d17c0 0x7f8a001590a0 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7f89fc00c400 tx=0x7f89fc00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:08.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.379+0000 7f89f27fc640 1 -- 172.21.15.27:0/2255964962 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89fc0052d0 con 0x7f8a000d17c0 2024-09-12T03:24:08.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.379+0000 7f89f27fc640 1 -- 172.21.15.27:0/2255964962 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89fc005490 con 0x7f8a000d17c0 2024-09-12T03:24:08.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.379+0000 7f8a0813b640 1 -- 172.21.15.27:0/2255964962 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a000c9000 con 0x7f8a000d17c0 2024-09-12T03:24:08.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.380+0000 7f89f27fc640 1 -- 172.21.15.27:0/2255964962 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89fc007260 con 0x7f8a000d17c0 2024-09-12T03:24:08.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.380+0000 7f8a0813b640 1 -- 172.21.15.27:0/2255964962 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a000c95c0 con 0x7f8a000d17c0 2024-09-12T03:24:08.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.381+0000 7f8a0813b640 1 -- 172.21.15.27:0/2255964962 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a000d1280 con 0x7f8a000d17c0 2024-09-12T03:24:08.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.381+0000 7f89f27fc640 1 -- 172.21.15.27:0/2255964962 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 26) ==== 91518+0+0 (secure 0 0 0) 0x7f89fc007400 con 0x7f8a000d17c0 2024-09-12T03:24:08.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.381+0000 7f89f27fc640 1 --2- 172.21.15.27:0/2255964962 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f89e406c9a0 0x7f89e406ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:08.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.382+0000 7f89f27fc640 1 -- 172.21.15.27:0/2255964962 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(160..160 src has 1..160) ==== 5765+0+0 (secure 0 0 0) 0x7f89fc088e60 con 0x7f8a000d17c0 2024-09-12T03:24:08.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.382+0000 7f8a056d6640 1 --2- 172.21.15.27:0/2255964962 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f89e406c9a0 0x7f89e406ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:08.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.382+0000 7f8a056d6640 1 --2- 172.21.15.27:0/2255964962 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f89e406c9a0 0x7f89e406ee60 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f8a000b9720 tx=0x7f89f4008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:08.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.386+0000 7f89f27fc640 1 -- 172.21.15.27:0/2255964962 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f89fc058a20 con 0x7f8a000d17c0 2024-09-12T03:24:08.598 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.597+0000 7f8a0813b640 1 -- 172.21.15.27:0/2255964962 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8a000d53e0 con 0x7f8a000d17c0 2024-09-12T03:24:08.598 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.597+0000 7f89f27fc640 1 -- 172.21.15.27:0/2255964962 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 100 v100) ==== 95+0+5228 (secure 0 0 0) 0x7f89fc05cd20 con 0x7f8a000d17c0 2024-09-12T03:24:08.599 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:24:08.599 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":100,"btime":"2024-09-12T03:24:03:282449+0000","default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7163,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/2493793567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2493793567},{"type":"v1","addr":"172.21.15.45:6835","nonce":2493793567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":96},{"gid":8056,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/1643775717","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":1643775717},{"type":"v1","addr":"172.21.15.45:6834","nonce":1643775717}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95},{"gid":8967,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6834/4129691413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":4129691413},{"type":"v1","addr":"172.21.15.27:6834","nonce":4129691413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95}],"filesystems":[{"mdsmap":{"epoch":99,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-12T03:23:54.606856+0000","modified":"2024-09-12T03:23:56.609988+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8970},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8970":{"gid":8970,"name":"c","rank":0,"incarnation":97,"state":"up:active","state_seq":3,"addr":"172.21.15.27:6835/2940099359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":2940099359},{"type":"v1","addr":"172.21.15.27:6835","nonce":2940099359}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[27],"metadata_pool":26,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8970,"qdb_cluster":[8970]},"id":13}]} 2024-09-12T03:24:08.599 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 100 2024-09-12T03:24:08.601 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.600+0000 7f8a0813b640 1 -- 172.21.15.27:0/2255964962 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f89e406c9a0 msgr2=0x7f89e406ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:08.601 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.600+0000 7f8a0813b640 1 --2- 172.21.15.27:0/2255964962 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f89e406c9a0 0x7f89e406ee60 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f8a000b9720 tx=0x7f89f4008040 comp rx=0 tx=0).stop 2024-09-12T03:24:08.601 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.600+0000 7f8a0813b640 1 -- 172.21.15.27:0/2255964962 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8a000d17c0 msgr2=0x7f8a001590a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:08.601 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.600+0000 7f8a0813b640 1 --2- 172.21.15.27:0/2255964962 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8a000d17c0 0x7f8a001590a0 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7f89fc00c400 tx=0x7f89fc00c8c0 comp rx=0 tx=0).stop 2024-09-12T03:24:08.601 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.600+0000 7f8a0813b640 1 -- 172.21.15.27:0/2255964962 shutdown_connections 2024-09-12T03:24:08.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.600+0000 7f8a0813b640 1 --2- 172.21.15.27:0/2255964962 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8a0015eb60 0x7f8a000ca620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:08.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.600+0000 7f8a0813b640 1 --2- 172.21.15.27:0/2255964962 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f8a0015e780 0x7f8a000c83d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:08.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.600+0000 7f8a0813b640 1 --2- 172.21.15.27:0/2255964962 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f89e406c9a0 0x7f89e406ee60 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:08.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.600+0000 7f8a0813b640 1 --2- 172.21.15.27:0/2255964962 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8a000d17c0 0x7f8a001590a0 unknown :-1 s=CLOSED pgs=1747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:08.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.600+0000 7f8a0813b640 1 -- 172.21.15.27:0/2255964962 >> 172.21.15.27:0/2255964962 conn(0x7f8a0006da60 msgr2=0x7f8a000b8c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:08.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.600+0000 7f8a0813b640 1 -- 172.21.15.27:0/2255964962 shutdown_connections 2024-09-12T03:24:08.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.600+0000 7f8a0813b640 1 -- 172.21.15.27:0/2255964962 wait complete. 2024-09-12T03:24:08.611 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-12T03:24:08.713 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.712+0000 7faafa87b640 1 Processor -- start 2024-09-12T03:24:08.713 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.712+0000 7faafa87b640 1 -- start start 2024-09-12T03:24:08.714 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.713+0000 7faafa87b640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7faaf4158cf0 0x7faaf40b55d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:08.714 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.713+0000 7faafa87b640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faaf4162c10 con 0x7faaf41590d0 2024-09-12T03:24:08.714 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.713+0000 7faafa87b640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faaf4162ef0 con 0x7faaf415f600 2024-09-12T03:24:08.714 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.713+0000 7faafa87b640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7faaf41631d0 con 0x7faaf4158cf0 2024-09-12T03:24:08.715 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.713+0000 7faaf3fff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7faaf4158cf0 0x7faaf40b55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:08.715 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.713+0000 7faaf3fff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7faaf4158cf0 0x7faaf40b55d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33258/0 (socket says 172.21.15.27:33258) 2024-09-12T03:24:08.715 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.713+0000 7faaf37fe640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7faaf41590d0 0x7faaf4162430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:57592/0 (socket says 172.21.15.27:57592) 2024-09-12T03:24:08.715 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.713+0000 7faaf3fff640 1 -- 172.21.15.27:0/1793225679 learned_addr learned my addr 172.21.15.27:0/1793225679 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:24:08.715 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.713+0000 7faaf3fff640 1 -- 172.21.15.27:0/1793225679 >> v1:172.21.15.45:6789/0 conn(0x7faaf415f600 legacy=0x7faaf406bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:08.715 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.713+0000 7faaf3fff640 1 -- 172.21.15.27:0/1793225679 >> v1:172.21.15.27:6789/0 conn(0x7faaf41590d0 legacy=0x7faaf4162430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:08.715 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.713+0000 7faaf3fff640 1 -- 172.21.15.27:0/1793225679 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faaf415edc0 con 0x7faaf4158cf0 2024-09-12T03:24:08.715 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.713+0000 7faaf2ffd640 1 -- 172.21.15.27:0/1793225679 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 999723760 0 0) 0x7faaf4162c10 con 0x7faaf41590d0 2024-09-12T03:24:08.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.714+0000 7faaf3fff640 1 --2- 172.21.15.27:0/1793225679 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faaf4158cf0 0x7faaf40b55d0 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7faae4009870 tx=0x7faae402a200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0417df3f6660d1c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:08.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.714+0000 7faaf2ffd640 1 -- 172.21.15.27:0/1793225679 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faae4037070 con 0x7faaf4158cf0 2024-09-12T03:24:08.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.714+0000 7faaf2ffd640 1 -- 172.21.15.27:0/1793225679 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faae4032440 con 0x7faaf4158cf0 2024-09-12T03:24:08.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.714+0000 7faaf2ffd640 1 -- 172.21.15.27:0/1793225679 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faae4032810 con 0x7faaf4158cf0 2024-09-12T03:24:08.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.715+0000 7faafa87b640 1 -- 172.21.15.27:0/1793225679 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faaf4158cf0 msgr2=0x7faaf40b55d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:08.717 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.715+0000 7faafa87b640 1 --2- 172.21.15.27:0/1793225679 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faaf4158cf0 0x7faaf40b55d0 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7faae4009870 tx=0x7faae402a200 comp rx=0 tx=0).stop 2024-09-12T03:24:08.717 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.715+0000 7faafa87b640 1 -- 172.21.15.27:0/1793225679 shutdown_connections 2024-09-12T03:24:08.717 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.715+0000 7faafa87b640 1 --2- 172.21.15.27:0/1793225679 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faaf4158cf0 0x7faaf40b55d0 unknown :-1 s=CLOSED pgs=1749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:08.717 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.715+0000 7faafa87b640 1 -- 172.21.15.27:0/1793225679 >> 172.21.15.27:0/1793225679 conn(0x7faaf406cc40 msgr2=0x7faaf406d050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:08.717 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.715+0000 7faafa87b640 1 -- 172.21.15.27:0/1793225679 shutdown_connections 2024-09-12T03:24:08.717 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.715+0000 7faafa87b640 1 -- 172.21.15.27:0/1793225679 wait complete. 2024-09-12T03:24:08.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.716+0000 7faafa87b640 1 Processor -- start 2024-09-12T03:24:08.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.716+0000 7faafa87b640 1 -- start start 2024-09-12T03:24:08.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.717+0000 7faafa87b640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7faaf4158cf0 0x7faaf4153c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:08.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.717+0000 7faafa87b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faaf41590d0 0x7faaf41e81c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:08.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.717+0000 7faafa87b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faaf415f600 0x7faaf4157270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:08.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.717+0000 7faafa87b640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7faaf415fa50 con 0x7faaf415f600 2024-09-12T03:24:08.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.717+0000 7faafa87b640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7faaf406c450 con 0x7faaf41590d0 2024-09-12T03:24:08.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.717+0000 7faafa87b640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7faaf415fbd0 con 0x7faaf4158cf0 2024-09-12T03:24:08.720 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.717+0000 7faaf8e18640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faaf415f600 0x7faaf4157270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:08.720 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.717+0000 7faaf8e18640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faaf415f600 0x7faaf4157270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33262/0 (socket says 172.21.15.27:33262) 2024-09-12T03:24:08.720 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.717+0000 7faaf8e18640 1 -- 172.21.15.27:0/1465413341 learned_addr learned my addr 172.21.15.27:0/1465413341 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:24:08.720 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.717+0000 7faaf37fe640 1 --2- 172.21.15.27:0/1465413341 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faaf41590d0 0x7faaf41e81c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:08.720 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.717+0000 7faaf3fff640 1 --2- 172.21.15.27:0/1465413341 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7faaf4158cf0 0x7faaf4153c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:08.720 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.717+0000 7faaf8e18640 1 -- 172.21.15.27:0/1465413341 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7faaf4158cf0 msgr2=0x7faaf4153c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:08.720 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.717+0000 7faaf8e18640 1 --2- 172.21.15.27:0/1465413341 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7faaf4158cf0 0x7faaf4153c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:08.720 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.717+0000 7faaf8e18640 1 -- 172.21.15.27:0/1465413341 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faaf41590d0 msgr2=0x7faaf41e81c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:08.720 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.717+0000 7faaf8e18640 1 --2- 172.21.15.27:0/1465413341 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faaf41590d0 0x7faaf41e81c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:08.720 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.717+0000 7faaf8e18640 1 -- 172.21.15.27:0/1465413341 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faaf40aef90 con 0x7faaf415f600 2024-09-12T03:24:08.720 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.717+0000 7faaf3fff640 1 --2- 172.21.15.27:0/1465413341 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7faaf4158cf0 0x7faaf4153c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:24:08.720 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.718+0000 7faaf8e18640 1 --2- 172.21.15.27:0/1465413341 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faaf415f600 0x7faaf4157270 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x7faaec00c6e0 tx=0x7faaec00cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:08.720 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.718+0000 7faaf0ff9640 1 -- 172.21.15.27:0/1465413341 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaec003b70 con 0x7faaf415f600 2024-09-12T03:24:08.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.718+0000 7faaf0ff9640 1 -- 172.21.15.27:0/1465413341 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faaec003d30 con 0x7faaf415f600 2024-09-12T03:24:08.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.718+0000 7faafa87b640 1 -- 172.21.15.27:0/1465413341 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faaf40af270 con 0x7faaf415f600 2024-09-12T03:24:08.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.718+0000 7faaf0ff9640 1 -- 172.21.15.27:0/1465413341 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaec005310 con 0x7faaf415f600 2024-09-12T03:24:08.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.718+0000 7faafa87b640 1 -- 172.21.15.27:0/1465413341 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faaf40af830 con 0x7faaf415f600 2024-09-12T03:24:08.722 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.719+0000 7faafa87b640 1 -- 172.21.15.27:0/1465413341 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faaf40b5d90 con 0x7faaf415f600 2024-09-12T03:24:08.722 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.719+0000 7faaf0ff9640 1 -- 172.21.15.27:0/1465413341 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 26) ==== 91518+0+0 (secure 0 0 0) 0x7faaec0054b0 con 0x7faaf415f600 2024-09-12T03:24:08.722 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.720+0000 7faaf0ff9640 1 --2- 172.21.15.27:0/1465413341 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7faae006c970 0x7faae006ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:08.722 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.720+0000 7faaf0ff9640 1 -- 172.21.15.27:0/1465413341 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(160..160 src has 1..160) ==== 5765+0+0 (secure 0 0 0) 0x7faaec089830 con 0x7faaf415f600 2024-09-12T03:24:08.722 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.720+0000 7faaf3fff640 1 --2- 172.21.15.27:0/1465413341 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7faae006c970 0x7faae006ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:08.722 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.720+0000 7faaf3fff640 1 --2- 172.21.15.27:0/1465413341 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7faae006c970 0x7faae006ee30 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7faae4034000 tx=0x7faae402e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:08.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.724+0000 7faaf0ff9640 1 -- 172.21.15.27:0/1465413341 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7faaec059a40 con 0x7faaf415f600 2024-09-12T03:24:08.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.900+0000 7faafa87b640 1 -- 172.21.15.27:0/1465413341 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7faaf40b3af0 con 0x7faaf415f600 2024-09-12T03:24:08.902 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.902+0000 7faaf0ff9640 1 -- 172.21.15.27:0/1465413341 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v160) ==== 74+0+16784 (secure 0 0 0) 0x7faaec059340 con 0x7faaf415f600 2024-09-12T03:24:08.903 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:24:08.903 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":160,"fsid":"d2df504e-2647-4385-a5db-9fdf69f487a0","created":"2024-09-12T03:16:07.778937+0000","modified":"2024-09-12T03:23:56.250469+0000","last_up_change":"2024-09-12T03:16:19.314778+0000","last_in_change":"2024-09-12T03:16:09.893086+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T03:16:20.097646+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs_metadata","create_time":"2024-09-12T03:23:52.574839+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"159","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_data","create_time":"2024-09-12T03:23:53.596556+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"160","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1851116c-1fb3-4dc9-9772-f90e2ef4c355","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":158,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6817","nonce":3002321488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6819","nonce":3002321488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6823","nonce":3002321488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6821","nonce":3002321488}]},"public_addr":"172.21.15.27:6817/3002321488","cluster_addr":"172.21.15.27:6819/3002321488","heartbeat_back_addr":"172.21.15.27:6823/3002321488","heartbeat_front_addr":"172.21.15.27:6821/3002321488","state":["exists","up"]},{"osd":1,"uuid":"f38ae17d-73eb-4204-b99d-975fa806fb87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":158,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6809","nonce":1613102886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6811","nonce":1613102886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6815","nonce":1613102886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6813","nonce":1613102886}]},"public_addr":"172.21.15.27:6809/1613102886","cluster_addr":"172.21.15.27:6811/1613102886","heartbeat_back_addr":"172.21.15.27:6815/1613102886","heartbeat_front_addr":"172.21.15.27:6813/1613102886","state":["exists","up"]},{"osd":2,"uuid":"fb12dcc8-bdd0-42b8-b46d-8b10106224bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":158,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6825","nonce":3534723355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6827","nonce":3534723355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6831","nonce":3534723355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6829","nonce":3534723355}]},"public_addr":"172.21.15.27:6825/3534723355","cluster_addr":"172.21.15.27:6827/3534723355","heartbeat_back_addr":"172.21.15.27:6831/3534723355","heartbeat_front_addr":"172.21.15.27:6829/3534723355","state":["exists","up"]},{"osd":3,"uuid":"d74d3f49-9c24-4b9f-98e2-3ecd7b5ef3b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":158,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6801","nonce":80661598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6803","nonce":80661598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6807","nonce":80661598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6805","nonce":80661598}]},"public_addr":"172.21.15.27:6801/80661598","cluster_addr":"172.21.15.27:6803/80661598","heartbeat_back_addr":"172.21.15.27:6807/80661598","heartbeat_front_addr":"172.21.15.27:6805/80661598","state":["exists","up"]},{"osd":4,"uuid":"2bf123f2-1b0d-41c4-aa3b-805bab267e59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":158,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6817","nonce":3943189827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6819","nonce":3943189827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6823","nonce":3943189827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6821","nonce":3943189827}]},"public_addr":"172.21.15.45:6817/3943189827","cluster_addr":"172.21.15.45:6819/3943189827","heartbeat_back_addr":"172.21.15.45:6823/3943189827","heartbeat_front_addr":"172.21.15.45:6821/3943189827","state":["exists","up"]},{"osd":5,"uuid":"c49ba5da-91c6-4a86-a97b-4aba89c253ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":158,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6809","nonce":3620760330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6811","nonce":3620760330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6815","nonce":3620760330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6813","nonce":3620760330}]},"public_addr":"172.21.15.45:6809/3620760330","cluster_addr":"172.21.15.45:6811/3620760330","heartbeat_back_addr":"172.21.15.45:6815/3620760330","heartbeat_front_addr":"172.21.15.45:6813/3620760330","state":["exists","up"]},{"osd":6,"uuid":"b0123475-89f7-4a34-9e33-d797b99dd9dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":158,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6801","nonce":1989083510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6803","nonce":1989083510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6807","nonce":1989083510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6805","nonce":1989083510}]},"public_addr":"172.21.15.45:6801/1989083510","cluster_addr":"172.21.15.45:6803/1989083510","heartbeat_back_addr":"172.21.15.45:6807/1989083510","heartbeat_front_addr":"172.21.15.45:6805/1989083510","state":["exists","up"]},{"osd":7,"uuid":"4dad773d-b7d8-46fc-8a10-a972aad27b29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":158,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6825","nonce":1513300294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6827","nonce":1513300294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6831","nonce":1513300294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6829","nonce":1513300294}]},"public_addr":"172.21.15.45:6825/1513300294","cluster_addr":"172.21.15.45:6827/1513300294","heartbeat_back_addr":"172.21.15.45:6831/1513300294","heartbeat_front_addr":"172.21.15.45:6829/1513300294","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.881855+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:17.133252+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.735339+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.500742+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.477571+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:17.167667+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.470291+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.873743+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]},{"pool":19,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T03:24:08.905 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.904+0000 7faafa87b640 1 -- 172.21.15.27:0/1465413341 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7faae006c970 msgr2=0x7faae006ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:08.905 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.904+0000 7faafa87b640 1 --2- 172.21.15.27:0/1465413341 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7faae006c970 0x7faae006ee30 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7faae4034000 tx=0x7faae402e000 comp rx=0 tx=0).stop 2024-09-12T03:24:08.905 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.904+0000 7faafa87b640 1 -- 172.21.15.27:0/1465413341 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faaf415f600 msgr2=0x7faaf4157270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:08.905 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.904+0000 7faafa87b640 1 --2- 172.21.15.27:0/1465413341 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faaf415f600 0x7faaf4157270 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x7faaec00c6e0 tx=0x7faaec00cba0 comp rx=0 tx=0).stop 2024-09-12T03:24:08.905 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.904+0000 7faafa87b640 1 -- 172.21.15.27:0/1465413341 shutdown_connections 2024-09-12T03:24:08.905 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.904+0000 7faafa87b640 1 --2- 172.21.15.27:0/1465413341 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7faae006c970 0x7faae006ee30 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:08.905 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.904+0000 7faafa87b640 1 --2- 172.21.15.27:0/1465413341 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faaf415f600 0x7faaf4157270 unknown :-1 s=CLOSED pgs=1750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:08.906 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.904+0000 7faafa87b640 1 --2- 172.21.15.27:0/1465413341 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faaf41590d0 0x7faaf41e81c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:08.906 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.904+0000 7faafa87b640 1 --2- 172.21.15.27:0/1465413341 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7faaf4158cf0 0x7faaf4153c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:08.906 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.904+0000 7faafa87b640 1 -- 172.21.15.27:0/1465413341 >> 172.21.15.27:0/1465413341 conn(0x7faaf406cc40 msgr2=0x7faaf406bb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:08.906 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.904+0000 7faafa87b640 1 -- 172.21.15.27:0/1465413341 shutdown_connections 2024-09-12T03:24:08.906 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:08.904+0000 7faafa87b640 1 -- 172.21.15.27:0/1465413341 wait complete. 2024-09-12T03:24:08.916 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-12T03:24:09.019 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:09.018+0000 7fbcbd9cf640 1 Processor -- start 2024-09-12T03:24:09.019 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:09.018+0000 7fbcbd9cf640 1 -- start start 2024-09-12T03:24:09.021 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:09.019+0000 7fbcbd9cf640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fbcb8073960 0x7fbcb8162320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:09.021 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:09.019+0000 7fbcbd9cf640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fbcb8162860 0x7fbcb815e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:09.021 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:09.019+0000 7fbcbd9cf640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbcb80d2240 con 0x7fbcb815e5f0 2024-09-12T03:24:09.021 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:09.019+0000 7fbcbd9cf640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fbcb80d0da0 con 0x7fbcb8162860 2024-09-12T03:24:09.021 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:09.019+0000 7fbcbd9cf640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fbcb8163090 con 0x7fbcb8073960 2024-09-12T03:24:09.021 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:09.019+0000 7fbcb77fe640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fbcb8162860 0x7fbcb815e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:09.021 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:09.019+0000 7fbcb67fc640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fbcb8073960 0x7fbcb8162320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:09.021 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:09.019+0000 7fbcb6ffd640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fbcb815e5f0 0x7fbcb8073250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.27:51074/0 (socket says 172.21.15.27:51074) 2024-09-12T03:24:09.021 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:09.019+0000 7fbcb6ffd640 1 -- 172.21.15.27:0/3733653432 learned_addr learned my addr 172.21.15.27:0/3733653432 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:24:09.022 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:09.020+0000 7fbcb67fc640 1 -- 172.21.15.27:0/3733653432 >> v2:172.21.15.45:3300/0 conn(0x7fbcb8162860 msgr2=0x7fbcb815e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:24:09.022 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:09.020+0000 7fbcb67fc640 1 --2- 172.21.15.27:0/3733653432 >> v2:172.21.15.45:3300/0 conn(0x7fbcb8162860 0x7fbcb815e9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:09.022 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:09.020+0000 7fbcb5ffb640 1 -- 172.21.15.27:0/3733653432 <== mon.1 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 905725391 0 0) 0x7fbcb80d2240 con 0x7fbcb815e5f0 2024-09-12T03:24:09.022 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:09.020+0000 7fbcb67fc640 1 -- 172.21.15.27:0/3733653432 >> v1:172.21.15.45:6789/0 conn(0x7fbcb815e5f0 legacy=0x7fbcb8073250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:09.022 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:09.020+0000 7fbcb67fc640 1 -- 172.21.15.27:0/3733653432 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcb80d0470 con 0x7fbcb8073960 2024-09-12T03:24:09.023 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:09.021+0000 7fbcb67fc640 1 --2- 172.21.15.27:0/3733653432 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fbcb8073960 0x7fbcb8162320 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fbca00097a0 tx=0x7fbca002a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=93f63c15c5ef03ad server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:09.023 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:09.021+0000 7fbcb5ffb640 1 -- 172.21.15.27:0/3733653432 <== mon.2 v2:172.21.15.45:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbca0037070 con 0x7fbcb8073960 2024-09-12T03:24:09.023 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:09.021+0000 7fbcb5ffb640 1 -- 172.21.15.27:0/3733653432 <== mon.2 v2:172.21.15.45:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbca0032440 con 0x7fbcb8073960 2024-09-12T03:24:09.024 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:09.021+0000 7fbcb5ffb640 1 -- 172.21.15.27:0/3733653432 <== mon.2 v2:172.21.15.45:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbca0032810 con 0x7fbcb8073960 2024-09-12T03:24:09.024 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:09.021+0000 7fbcbd9cf640 1 -- 172.21.15.27:0/3733653432 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fbcb8073960 msgr2=0x7fbcb8162320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:09.024 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:09.021+0000 7fbcbd9cf640 1 --2- 172.21.15.27:0/3733653432 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fbcb8073960 0x7fbcb8162320 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fbca00097a0 tx=0x7fbca002a180 comp rx=0 tx=0).stop 2024-09-12T03:24:09.024 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:09.021+0000 7fbcbd9cf640 1 -- 172.21.15.27:0/3733653432 shutdown_connections 2024-09-12T03:24:09.024 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:09.021+0000 7fbcbd9cf640 1 --2- 172.21.15.27:0/3733653432 >> v2:172.21.15.45:3300/0 conn(0x7fbcb8162860 0x7fbcb815e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:09.024 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:09.021+0000 7fbcbd9cf640 1 --2- 172.21.15.27:0/3733653432 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fbcb8073960 0x7fbcb8162320 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:09.024 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:09.021+0000 7fbcbd9cf640 1 -- 172.21.15.27:0/3733653432 >> 172.21.15.27:0/3733653432 conn(0x7fbcb806f300 msgr2=0x7fbcb806f710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:09.024 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:09.021+0000 7fbcbd9cf640 1 -- 172.21.15.27:0/3733653432 shutdown_connections 2024-09-12T03:24:09.024 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:09.021+0000 7fbcbd9cf640 1 -- 172.21.15.27:0/3733653432 wait complete. 2024-09-12T03:24:09.024 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:09.023+0000 7fbcbd9cf640 1 Processor -- start 2024-09-12T03:24:09.024 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:09.023+0000 7fbcbd9cf640 1 -- start start 2024-09-12T03:24:09.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:09.023+0000 7fbcbd9cf640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbcb8073960 0x7fbcb8152ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:09.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:09.023+0000 7fbcbd9cf640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fbcb815e5f0 0x7fbcb81f7ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:09.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:09.023+0000 7fbcbd9cf640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbcb8162860 0x7fbcb81f8010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:09.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:09.023+0000 7fbcbd9cf640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fbcb80b5620 con 0x7fbcb8162860 2024-09-12T03:24:09.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:09.023+0000 7fbcbd9cf640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fbcb80d0a30 con 0x7fbcb8073960 2024-09-12T03:24:09.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:09.023+0000 7fbcbd9cf640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fbcb8162cf0 con 0x7fbcb815e5f0 2024-09-12T03:24:09.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:09.024+0000 7fbcb77fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbcb8162860 0x7fbcb81f8010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:09.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:09.024+0000 7fbcb77fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbcb8162860 0x7fbcb81f8010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33264/0 (socket says 172.21.15.27:33264) 2024-09-12T03:24:09.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:09.024+0000 7fbcb77fe640 1 -- 172.21.15.27:0/1969928955 learned_addr learned my addr 172.21.15.27:0/1969928955 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:24:09.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:09.024+0000 7fbcb67fc640 1 --2- 172.21.15.27:0/1969928955 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fbcb815e5f0 0x7fbcb81f7ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:09.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:09.024+0000 7fbcb6ffd640 1 --2- 172.21.15.27:0/1969928955 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbcb8073960 0x7fbcb8152ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:09.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:09.024+0000 7fbcb77fe640 1 -- 172.21.15.27:0/1969928955 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fbcb815e5f0 msgr2=0x7fbcb81f7ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:09.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:09.024+0000 7fbcb77fe640 1 --2- 172.21.15.27:0/1969928955 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fbcb815e5f0 0x7fbcb81f7ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:09.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:09.024+0000 7fbcb77fe640 1 -- 172.21.15.27:0/1969928955 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbcb8073960 msgr2=0x7fbcb8152ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:09.029 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:09.024+0000 7fbcb77fe640 1 --2- 172.21.15.27:0/1969928955 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbcb8073960 0x7fbcb8152ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:09.029 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:09.024+0000 7fbcb77fe640 1 -- 172.21.15.27:0/1969928955 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcb8155770 con 0x7fbcb8162860 2024-09-12T03:24:09.029 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:09.024+0000 7fbcb77fe640 1 --2- 172.21.15.27:0/1969928955 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbcb8162860 0x7fbcb81f8010 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7fbcb000e820 tx=0x7fbcb000ec20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:09.029 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:09.024+0000 7fbcaffff640 1 -- 172.21.15.27:0/1969928955 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcb0017070 con 0x7fbcb8162860 2024-09-12T03:24:09.029 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:09.024+0000 7fbcaffff640 1 -- 172.21.15.27:0/1969928955 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbcb000ee70 con 0x7fbcb8162860 2024-09-12T03:24:09.029 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:09.024+0000 7fbcbd9cf640 1 -- 172.21.15.27:0/1969928955 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbcb8155490 con 0x7fbcb8162860 2024-09-12T03:24:09.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:09.024+0000 7fbcaffff640 1 -- 172.21.15.27:0/1969928955 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcb000cb60 con 0x7fbcb8162860 2024-09-12T03:24:09.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:09.024+0000 7fbcbd9cf640 1 -- 172.21.15.27:0/1969928955 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbcb81f5a20 con 0x7fbcb8162860 2024-09-12T03:24:09.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:09.025+0000 7fbcbd9cf640 1 -- 172.21.15.27:0/1969928955 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbcb8074090 con 0x7fbcb8162860 2024-09-12T03:24:09.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:09.026+0000 7fbcaffff640 1 -- 172.21.15.27:0/1969928955 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 26) ==== 91518+0+0 (secure 0 0 0) 0x7fbcb001a070 con 0x7fbcb8162860 2024-09-12T03:24:09.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:09.027+0000 7fbcaffff640 1 --2- 172.21.15.27:0/1969928955 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fbc9806c970 0x7fbc9806ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:09.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:09.027+0000 7fbcb6ffd640 1 --2- 172.21.15.27:0/1969928955 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fbc9806c970 0x7fbc9806ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:09.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:09.027+0000 7fbcaffff640 1 -- 172.21.15.27:0/1969928955 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(160..160 src has 1..160) ==== 5765+0+0 (secure 0 0 0) 0x7fbcb0089ba0 con 0x7fbcb8162860 2024-09-12T03:24:09.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:09.027+0000 7fbcb6ffd640 1 --2- 172.21.15.27:0/1969928955 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fbc9806c970 0x7fbc9806ee30 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fbca80012e0 tx=0x7fbca8001700 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:09.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:09.031+0000 7fbcaffff640 1 -- 172.21.15.27:0/1969928955 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fbcb0059760 con 0x7fbcb8162860 2024-09-12T03:24:09.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:09.243+0000 7fbcbd9cf640 1 -- 172.21.15.27:0/1969928955 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fbcb815ffe0 con 0x7fbcb8162860 2024-09-12T03:24:09.244 INFO:tasks.ceph.mon.a.smithi027.stderr:2024-09-12T03:24:09.244+0000 7f959bfd9640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-12T03:24:09.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:09.245+0000 7fbcaffff640 1 -- 172.21.15.27:0/1969928955 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v101) ==== 161+0+0 (secure 0 0 0) 0x7fbcb008c090 con 0x7fbcb8162860 2024-09-12T03:24:09.246 INFO:teuthology.orchestra.run.smithi027.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-12T03:24:09.249 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:09.248+0000 7fbcbd9cf640 1 -- 172.21.15.27:0/1969928955 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fbc9806c970 msgr2=0x7fbc9806ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:09.249 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:09.248+0000 7fbcbd9cf640 1 --2- 172.21.15.27:0/1969928955 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fbc9806c970 0x7fbc9806ee30 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fbca80012e0 tx=0x7fbca8001700 comp rx=0 tx=0).stop 2024-09-12T03:24:09.249 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:09.248+0000 7fbcbd9cf640 1 -- 172.21.15.27:0/1969928955 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbcb8162860 msgr2=0x7fbcb81f8010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:09.249 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:09.248+0000 7fbcbd9cf640 1 --2- 172.21.15.27:0/1969928955 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbcb8162860 0x7fbcb81f8010 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7fbcb000e820 tx=0x7fbcb000ec20 comp rx=0 tx=0).stop 2024-09-12T03:24:09.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:09.248+0000 7fbcbd9cf640 1 -- 172.21.15.27:0/1969928955 shutdown_connections 2024-09-12T03:24:09.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:09.248+0000 7fbcbd9cf640 1 --2- 172.21.15.27:0/1969928955 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fbc9806c970 0x7fbc9806ee30 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:09.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:09.248+0000 7fbcbd9cf640 1 --2- 172.21.15.27:0/1969928955 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbcb8162860 0x7fbcb81f8010 unknown :-1 s=CLOSED pgs=1751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:09.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:09.248+0000 7fbcbd9cf640 1 --2- 172.21.15.27:0/1969928955 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fbcb815e5f0 0x7fbcb81f7ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:09.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:09.248+0000 7fbcbd9cf640 1 --2- 172.21.15.27:0/1969928955 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbcb8073960 0x7fbcb8152ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:09.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:09.248+0000 7fbcbd9cf640 1 -- 172.21.15.27:0/1969928955 >> 172.21.15.27:0/1969928955 conn(0x7fbcb806f300 msgr2=0x7fbcb8159270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:09.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:09.249+0000 7fbcbd9cf640 1 -- 172.21.15.27:0/1969928955 shutdown_connections 2024-09-12T03:24:09.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:09.249+0000 7fbcbd9cf640 1 -- 172.21.15.27:0/1969928955 wait complete. 2024-09-12T03:24:09.261 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-12T03:24:09.287 INFO:tasks.ceph.mds.c.smithi027.stdout:starting mds.c at 2024-09-12T03:24:09.368 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:09.368+0000 7f4b45d13640 1 Processor -- start 2024-09-12T03:24:09.369 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:09.368+0000 7f4b45d13640 1 -- start start 2024-09-12T03:24:09.372 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:09.368+0000 7f4b45d13640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4b40162040 0x7f4b400d1da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:09.372 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:09.368+0000 7f4b45d13640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f4b4006dfd0 0x7f4b400b7a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:09.372 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:09.368+0000 7f4b45d13640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4b400d2e30 con 0x7f4b400d3790 2024-09-12T03:24:09.372 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:09.368+0000 7f4b45d13640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f4b400b7f90 con 0x7f4b4006dfd0 2024-09-12T03:24:09.372 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:09.368+0000 7f4b45d13640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f4b40162420 con 0x7f4b40162040 2024-09-12T03:24:09.372 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:09.369+0000 7f4b3ffff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f4b4006dfd0 0x7f4b400b7a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:09.372 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:09.369+0000 7f4b3effd640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f4b400d3790 0x7f4b4006d8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:38876/0 (socket says 172.21.15.27:38876) 2024-09-12T03:24:09.372 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:09.369+0000 7f4b3effd640 1 -- 172.21.15.27:0/662842890 learned_addr learned my addr 172.21.15.27:0/662842890 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:24:09.372 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:09.369+0000 7f4b3ffff640 1 -- 172.21.15.27:0/662842890 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4b40162040 msgr2=0x7f4b400d1da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:24:09.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:09.369+0000 7f4b3ffff640 1 --2- 172.21.15.27:0/662842890 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4b40162040 0x7f4b400d1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:09.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:09.369+0000 7f4b3ffff640 1 -- 172.21.15.27:0/662842890 >> v1:172.21.15.27:6789/0 conn(0x7f4b400d3790 legacy=0x7f4b4006d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:09.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:09.369+0000 7f4b3e7fc640 1 -- 172.21.15.27:0/662842890 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 305386397 0 0) 0x7f4b400d2e30 con 0x7f4b400d3790 2024-09-12T03:24:09.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:09.369+0000 7f4b3ffff640 1 -- 172.21.15.27:0/662842890 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b400d30d0 con 0x7f4b4006dfd0 2024-09-12T03:24:09.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:09.370+0000 7f4b3ffff640 1 --2- 172.21.15.27:0/662842890 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4b4006dfd0 0x7f4b400b7a50 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7f4b38009d90 tx=0x7f4b3802a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0ee30e107e006b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:09.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:09.370+0000 7f4b3e7fc640 1 -- 172.21.15.27:0/662842890 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b38007740 con 0x7f4b4006dfd0 2024-09-12T03:24:09.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:09.370+0000 7f4b3e7fc640 1 -- 172.21.15.27:0/662842890 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b380078e0 con 0x7f4b4006dfd0 2024-09-12T03:24:09.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:09.370+0000 7f4b3e7fc640 1 -- 172.21.15.27:0/662842890 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b38007cb0 con 0x7f4b4006dfd0 2024-09-12T03:24:09.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:09.370+0000 7f4b45d13640 1 -- 172.21.15.27:0/662842890 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4b4006dfd0 msgr2=0x7f4b400b7a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:09.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:09.370+0000 7f4b45d13640 1 --2- 172.21.15.27:0/662842890 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4b4006dfd0 0x7f4b400b7a50 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7f4b38009d90 tx=0x7f4b3802a660 comp rx=0 tx=0).stop 2024-09-12T03:24:09.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:09.370+0000 7f4b45d13640 1 -- 172.21.15.27:0/662842890 shutdown_connections 2024-09-12T03:24:09.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:09.370+0000 7f4b45d13640 1 --2- 172.21.15.27:0/662842890 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4b4006dfd0 0x7f4b400b7a50 unknown :-1 s=CLOSED pgs=1755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:09.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:09.370+0000 7f4b45d13640 1 --2- 172.21.15.27:0/662842890 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4b40162040 0x7f4b400d1da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:09.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:09.370+0000 7f4b45d13640 1 -- 172.21.15.27:0/662842890 >> 172.21.15.27:0/662842890 conn(0x7f4b400b6700 msgr2=0x7f4b400b6b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:09.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:09.370+0000 7f4b45d13640 1 -- 172.21.15.27:0/662842890 shutdown_connections 2024-09-12T03:24:09.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:09.370+0000 7f4b45d13640 1 -- 172.21.15.27:0/662842890 wait complete. 2024-09-12T03:24:09.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:09.371+0000 7f4b45d13640 1 Processor -- start 2024-09-12T03:24:09.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:09.372+0000 7f4b45d13640 1 -- start start 2024-09-12T03:24:09.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:09.372+0000 7f4b45d13640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4b4006dfd0 0x7f4b400ce430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:09.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:09.372+0000 7f4b45d13640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b400d3790 0x7f4b40159640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:09.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:09.372+0000 7f4b45d13640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4b40162040 0x7f4b400ca660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:09.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:09.372+0000 7f4b45d13640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f4b400d4530 con 0x7f4b4006dfd0 2024-09-12T03:24:09.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:09.372+0000 7f4b45d13640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f4b400cee00 con 0x7f4b400d3790 2024-09-12T03:24:09.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:09.372+0000 7f4b45d13640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f4b400d4150 con 0x7f4b40162040 2024-09-12T03:24:09.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:09.373+0000 7f4b3f7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4b4006dfd0 0x7f4b400ce430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:09.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:09.373+0000 7f4b3f7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4b4006dfd0 0x7f4b400ce430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41464/0 (socket says 172.21.15.27:41464) 2024-09-12T03:24:09.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:09.373+0000 7f4b3f7fe640 1 -- 172.21.15.27:0/1996096873 learned_addr learned my addr 172.21.15.27:0/1996096873 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:24:09.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:09.373+0000 7f4b3effd640 1 --2- 172.21.15.27:0/1996096873 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b400d3790 0x7f4b40159640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:09.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:09.373+0000 7f4b3ffff640 1 --2- 172.21.15.27:0/1996096873 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4b40162040 0x7f4b400ca660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:09.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:09.373+0000 7f4b3f7fe640 1 -- 172.21.15.27:0/1996096873 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4b40162040 msgr2=0x7f4b400ca660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:09.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:09.373+0000 7f4b3f7fe640 1 --2- 172.21.15.27:0/1996096873 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4b40162040 0x7f4b400ca660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:09.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:09.373+0000 7f4b3f7fe640 1 -- 172.21.15.27:0/1996096873 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b400d3790 msgr2=0x7f4b40159640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:09.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:09.373+0000 7f4b3f7fe640 1 --2- 172.21.15.27:0/1996096873 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b400d3790 0x7f4b40159640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:09.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:09.373+0000 7f4b3f7fe640 1 -- 172.21.15.27:0/1996096873 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b40159db0 con 0x7f4b4006dfd0 2024-09-12T03:24:09.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:09.373+0000 7f4b3f7fe640 1 --2- 172.21.15.27:0/1996096873 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4b4006dfd0 0x7f4b400ce430 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 crypto rx=0x7f4b3000b410 tx=0x7f4b3000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:09.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:09.373+0000 7f4b37fff640 1 -- 172.21.15.27:0/1996096873 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b30015020 con 0x7f4b4006dfd0 2024-09-12T03:24:09.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:09.373+0000 7f4b37fff640 1 -- 172.21.15.27:0/1996096873 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b30011850 con 0x7f4b4006dfd0 2024-09-12T03:24:09.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:09.373+0000 7f4b45d13640 1 -- 172.21.15.27:0/1996096873 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b4015a090 con 0x7f4b4006dfd0 2024-09-12T03:24:09.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:09.373+0000 7f4b37fff640 1 -- 172.21.15.27:0/1996096873 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b30011c40 con 0x7f4b4006dfd0 2024-09-12T03:24:09.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:09.373+0000 7f4b45d13640 1 -- 172.21.15.27:0/1996096873 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b4015a650 con 0x7f4b4006dfd0 2024-09-12T03:24:09.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:09.374+0000 7f4b45d13640 1 -- 172.21.15.27:0/1996096873 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b400b7a50 con 0x7f4b4006dfd0 2024-09-12T03:24:09.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:09.375+0000 7f4b37fff640 1 -- 172.21.15.27:0/1996096873 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 26) ==== 91518+0+0 (secure 0 0 0) 0x7f4b30018020 con 0x7f4b4006dfd0 2024-09-12T03:24:09.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:09.375+0000 7f4b37fff640 1 --2- 172.21.15.27:0/1996096873 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f4b2006c9a0 0x7f4b2006ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:09.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:09.375+0000 7f4b37fff640 1 -- 172.21.15.27:0/1996096873 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(161..161 src has 1..161) ==== 5851+0+0 (secure 0 0 0) 0x7f4b300896f0 con 0x7f4b4006dfd0 2024-09-12T03:24:09.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:09.375+0000 7f4b3effd640 1 --2- 172.21.15.27:0/1996096873 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f4b2006c9a0 0x7f4b2006ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:09.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:09.376+0000 7f4b3effd640 1 --2- 172.21.15.27:0/1996096873 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f4b2006c9a0 0x7f4b2006ee60 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f4b400cec90 tx=0x7f4b28007b50 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:09.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:09.380+0000 7f4b37fff640 1 -- 172.21.15.27:0/1996096873 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f4b3001f020 con 0x7f4b4006dfd0 2024-09-12T03:24:09.592 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:09.591+0000 7f4b45d13640 1 -- 172.21.15.27:0/1996096873 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f4b4006e6e0 con 0x7f4b4006dfd0 2024-09-12T03:24:10.248 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:10.247+0000 7f4b37fff640 1 -- 172.21.15.27:0/1996096873 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v102) ==== 104+0+0 (secure 0 0 0) 0x7f4b300172c0 con 0x7f4b4006dfd0 2024-09-12T03:24:10.251 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:10.250+0000 7f4b45d13640 1 -- 172.21.15.27:0/1996096873 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f4b2006c9a0 msgr2=0x7f4b2006ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:10.251 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:10.250+0000 7f4b45d13640 1 --2- 172.21.15.27:0/1996096873 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f4b2006c9a0 0x7f4b2006ee60 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f4b400cec90 tx=0x7f4b28007b50 comp rx=0 tx=0).stop 2024-09-12T03:24:10.251 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:10.250+0000 7f4b45d13640 1 -- 172.21.15.27:0/1996096873 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4b4006dfd0 msgr2=0x7f4b400ce430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:10.251 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:10.250+0000 7f4b45d13640 1 --2- 172.21.15.27:0/1996096873 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4b4006dfd0 0x7f4b400ce430 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 crypto rx=0x7f4b3000b410 tx=0x7f4b3000b8d0 comp rx=0 tx=0).stop 2024-09-12T03:24:10.251 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:10.250+0000 7f4b45d13640 1 -- 172.21.15.27:0/1996096873 shutdown_connections 2024-09-12T03:24:10.251 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:10.250+0000 7f4b45d13640 1 --2- 172.21.15.27:0/1996096873 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4b40162040 0x7f4b400ca660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:10.251 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:10.250+0000 7f4b45d13640 1 --2- 172.21.15.27:0/1996096873 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b400d3790 0x7f4b40159640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:10.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:10.250+0000 7f4b45d13640 1 --2- 172.21.15.27:0/1996096873 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f4b2006c9a0 0x7f4b2006ee60 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:10.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:10.250+0000 7f4b45d13640 1 --2- 172.21.15.27:0/1996096873 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4b4006dfd0 0x7f4b400ce430 unknown :-1 s=CLOSED pgs=1756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:10.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:10.250+0000 7f4b45d13640 1 -- 172.21.15.27:0/1996096873 >> 172.21.15.27:0/1996096873 conn(0x7f4b400b6700 msgr2=0x7f4b4006d8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:10.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:10.250+0000 7f4b45d13640 1 -- 172.21.15.27:0/1996096873 shutdown_connections 2024-09-12T03:24:10.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:10.250+0000 7f4b45d13640 1 -- 172.21.15.27:0/1996096873 wait complete. 2024-09-12T03:24:10.266 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-12T03:24:10.423 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:10.423+0000 7f80c2303640 1 Processor -- start 2024-09-12T03:24:10.424 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:10.423+0000 7f80c2303640 1 -- start start 2024-09-12T03:24:10.425 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:10.423+0000 7f80c2303640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f80bc070d30 0x7f80bc06cc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:10.425 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:10.423+0000 7f80c2303640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f80bc0d0b50 0x7f80bc168300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:10.425 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:10.424+0000 7f80c2303640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f80bc0cf9f0 con 0x7f80bc06e960 2024-09-12T03:24:10.425 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:10.424+0000 7f80c2303640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f80bc0d1a40 con 0x7f80bc0d0b50 2024-09-12T03:24:10.425 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:10.424+0000 7f80c2303640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f80bc0cfc90 con 0x7f80bc070d30 2024-09-12T03:24:10.425 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:10.424+0000 7f80c08a0640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f80bc0d0b50 0x7f80bc168300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:10.425 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:10.424+0000 7f80bb7fe640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f80bc06e960 0x7f80bc0d0440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:38878/0 (socket says 172.21.15.27:38878) 2024-09-12T03:24:10.426 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:10.424+0000 7f80bb7fe640 1 -- 172.21.15.27:0/4282474683 learned_addr learned my addr 172.21.15.27:0/4282474683 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:24:10.426 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:10.424+0000 7f80c08a0640 1 -- 172.21.15.27:0/4282474683 >> v2:172.21.15.45:3300/0 conn(0x7f80bc070d30 msgr2=0x7f80bc06cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:24:10.426 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:10.424+0000 7f80bbfff640 1 --2- 172.21.15.27:0/4282474683 >> v2:172.21.15.45:3300/0 conn(0x7f80bc070d30 0x7f80bc06cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:10.426 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:10.424+0000 7f80c08a0640 1 --2- 172.21.15.27:0/4282474683 >> v2:172.21.15.45:3300/0 conn(0x7f80bc070d30 0x7f80bc06cc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:10.426 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:10.424+0000 7f80c08a0640 1 -- 172.21.15.27:0/4282474683 >> v1:172.21.15.27:6789/0 conn(0x7f80bc06e960 legacy=0x7f80bc0d0440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:10.426 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:10.424+0000 7f80c08a0640 1 -- 172.21.15.27:0/4282474683 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80bc0cff70 con 0x7f80bc0d0b50 2024-09-12T03:24:10.426 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:10.424+0000 7f80baffd640 1 -- 172.21.15.27:0/4282474683 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3177160403 0 0) 0x7f80bc0cf9f0 con 0x7f80bc06e960 2024-09-12T03:24:10.427 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:10.425+0000 7f80c08a0640 1 --2- 172.21.15.27:0/4282474683 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f80bc0d0b50 0x7f80bc168300 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7f80bc073770 tx=0x7f80b402a560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5094c24f6aaaec4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:10.427 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:10.425+0000 7f80baffd640 1 -- 172.21.15.27:0/4282474683 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80b4007740 con 0x7f80bc0d0b50 2024-09-12T03:24:10.427 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:10.425+0000 7f80baffd640 1 -- 172.21.15.27:0/4282474683 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80b40078e0 con 0x7f80bc0d0b50 2024-09-12T03:24:10.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:10.425+0000 7f80baffd640 1 -- 172.21.15.27:0/4282474683 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80b4007cb0 con 0x7f80bc0d0b50 2024-09-12T03:24:10.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:10.425+0000 7f80c2303640 1 -- 172.21.15.27:0/4282474683 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f80bc0d0b50 msgr2=0x7f80bc168300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:10.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:10.425+0000 7f80c2303640 1 --2- 172.21.15.27:0/4282474683 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f80bc0d0b50 0x7f80bc168300 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7f80bc073770 tx=0x7f80b402a560 comp rx=0 tx=0).stop 2024-09-12T03:24:10.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:10.426+0000 7f80c2303640 1 -- 172.21.15.27:0/4282474683 shutdown_connections 2024-09-12T03:24:10.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:10.426+0000 7f80c2303640 1 --2- 172.21.15.27:0/4282474683 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f80bc0d0b50 0x7f80bc168300 unknown :-1 s=CLOSED pgs=1758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:10.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:10.426+0000 7f80c2303640 1 --2- 172.21.15.27:0/4282474683 >> v2:172.21.15.45:3300/0 conn(0x7f80bc070d30 0x7f80bc06cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:10.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:10.426+0000 7f80c2303640 1 -- 172.21.15.27:0/4282474683 >> 172.21.15.27:0/4282474683 conn(0x7f80bc0b7c10 msgr2=0x7f80bc09ba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:10.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:10.426+0000 7f80c2303640 1 -- 172.21.15.27:0/4282474683 shutdown_connections 2024-09-12T03:24:10.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:10.426+0000 7f80c2303640 1 -- 172.21.15.27:0/4282474683 wait complete. 2024-09-12T03:24:10.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:10.427+0000 7f80c2303640 1 Processor -- start 2024-09-12T03:24:10.431 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:10.427+0000 7f80c2303640 1 -- start start 2024-09-12T03:24:10.431 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:10.428+0000 7f80c2303640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f80bc06e960 0x7f80bc0cc260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:10.431 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:10.428+0000 7f80c2303640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f80bc070d30 0x7f80bc1609d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:10.431 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:10.428+0000 7f80c2303640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f80bc0d0b50 0x7f80bc160f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:10.431 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:10.428+0000 7f80c2303640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f80bc0d31c0 con 0x7f80bc06e960 2024-09-12T03:24:10.431 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:10.428+0000 7f80c2303640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f80bc06d180 con 0x7f80bc070d30 2024-09-12T03:24:10.431 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:10.428+0000 7f80c2303640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f80bc0d3340 con 0x7f80bc0d0b50 2024-09-12T03:24:10.431 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:10.428+0000 7f80bbfff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f80bc06e960 0x7f80bc0cc260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:10.431 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:10.428+0000 7f80bbfff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f80bc06e960 0x7f80bc0cc260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41494/0 (socket says 172.21.15.27:41494) 2024-09-12T03:24:10.431 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:10.428+0000 7f80bbfff640 1 -- 172.21.15.27:0/2901358927 learned_addr learned my addr 172.21.15.27:0/2901358927 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:24:10.431 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:10.428+0000 7f80c08a0640 1 --2- 172.21.15.27:0/2901358927 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f80bc0d0b50 0x7f80bc160f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:10.432 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:10.428+0000 7f80bb7fe640 1 --2- 172.21.15.27:0/2901358927 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f80bc070d30 0x7f80bc1609d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:10.432 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:10.428+0000 7f80bbfff640 1 -- 172.21.15.27:0/2901358927 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f80bc0d0b50 msgr2=0x7f80bc160f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:10.432 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:10.428+0000 7f80bbfff640 1 --2- 172.21.15.27:0/2901358927 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f80bc0d0b50 0x7f80bc160f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:10.432 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:10.428+0000 7f80bbfff640 1 -- 172.21.15.27:0/2901358927 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f80bc070d30 msgr2=0x7f80bc1609d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:10.432 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:10.428+0000 7f80bbfff640 1 --2- 172.21.15.27:0/2901358927 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f80bc070d30 0x7f80bc1609d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:10.432 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:10.428+0000 7f80bbfff640 1 -- 172.21.15.27:0/2901358927 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80bc155b70 con 0x7f80bc06e960 2024-09-12T03:24:10.432 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:10.428+0000 7f80bbfff640 1 --2- 172.21.15.27:0/2901358927 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f80bc06e960 0x7f80bc0cc260 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7f80ac00c820 tx=0x7f80ac00cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:10.432 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:10.429+0000 7f80b8ff9640 1 -- 172.21.15.27:0/2901358927 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80ac015070 con 0x7f80bc06e960 2024-09-12T03:24:10.432 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:10.429+0000 7f80b8ff9640 1 -- 172.21.15.27:0/2901358927 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80ac004240 con 0x7f80bc06e960 2024-09-12T03:24:10.432 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:10.429+0000 7f80c2303640 1 -- 172.21.15.27:0/2901358927 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80bc155e50 con 0x7f80bc06e960 2024-09-12T03:24:10.432 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:10.429+0000 7f80b8ff9640 1 -- 172.21.15.27:0/2901358927 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80ac00ce10 con 0x7f80bc06e960 2024-09-12T03:24:10.432 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:10.429+0000 7f80c2303640 1 -- 172.21.15.27:0/2901358927 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80bc156410 con 0x7f80bc06e960 2024-09-12T03:24:10.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:10.430+0000 7f80c2303640 1 -- 172.21.15.27:0/2901358927 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80bc06cc40 con 0x7f80bc06e960 2024-09-12T03:24:10.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:10.430+0000 7f80b8ff9640 1 -- 172.21.15.27:0/2901358927 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 26) ==== 91518+0+0 (secure 0 0 0) 0x7f80ac007830 con 0x7f80bc06e960 2024-09-12T03:24:10.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:10.431+0000 7f80b8ff9640 1 --2- 172.21.15.27:0/2901358927 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f80a806c970 0x7f80a806ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:10.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:10.431+0000 7f80b8ff9640 1 -- 172.21.15.27:0/2901358927 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(161..161 src has 1..161) ==== 5851+0+0 (secure 0 0 0) 0x7f80ac089960 con 0x7f80bc06e960 2024-09-12T03:24:10.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:10.431+0000 7f80bb7fe640 1 --2- 172.21.15.27:0/2901358927 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f80a806c970 0x7f80a806ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:10.434 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:10.431+0000 7f80bb7fe640 1 --2- 172.21.15.27:0/2901358927 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f80a806c970 0x7f80a806ee30 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f80bc0b75e0 tx=0x7f80b0009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:10.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:10.435+0000 7f80b8ff9640 1 -- 172.21.15.27:0/2901358927 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f80ac059ad0 con 0x7f80bc06e960 2024-09-12T03:24:10.612 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:10.611+0000 7f80c2303640 1 -- 172.21.15.27:0/2901358927 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f80bc0ce760 con 0x7f80bc06e960 2024-09-12T03:24:11.325 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:11.324+0000 7f80b8ff9640 1 -- 172.21.15.27:0/2901358927 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v162) ==== 181+0+0 (secure 0 0 0) 0x7f80ac0593d0 con 0x7f80bc06e960 2024-09-12T03:24:11.325 INFO:teuthology.orchestra.run.smithi027.stderr:pool 'cephfs_metadata' removed 2024-09-12T03:24:11.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:11.327+0000 7f80c2303640 1 -- 172.21.15.27:0/2901358927 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f80a806c970 msgr2=0x7f80a806ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:11.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:11.327+0000 7f80c2303640 1 --2- 172.21.15.27:0/2901358927 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f80a806c970 0x7f80a806ee30 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f80bc0b75e0 tx=0x7f80b0009040 comp rx=0 tx=0).stop 2024-09-12T03:24:11.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:11.327+0000 7f80c2303640 1 -- 172.21.15.27:0/2901358927 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f80bc06e960 msgr2=0x7f80bc0cc260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:11.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:11.327+0000 7f80c2303640 1 --2- 172.21.15.27:0/2901358927 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f80bc06e960 0x7f80bc0cc260 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7f80ac00c820 tx=0x7f80ac00cce0 comp rx=0 tx=0).stop 2024-09-12T03:24:11.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:11.328+0000 7f80c2303640 1 -- 172.21.15.27:0/2901358927 shutdown_connections 2024-09-12T03:24:11.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:11.328+0000 7f80c2303640 1 --2- 172.21.15.27:0/2901358927 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f80bc0d0b50 0x7f80bc160f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:11.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:11.328+0000 7f80c2303640 1 --2- 172.21.15.27:0/2901358927 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f80bc070d30 0x7f80bc1609d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:11.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:11.328+0000 7f80c2303640 1 --2- 172.21.15.27:0/2901358927 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f80a806c970 0x7f80a806ee30 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:11.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:11.328+0000 7f80c2303640 1 --2- 172.21.15.27:0/2901358927 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f80bc06e960 0x7f80bc0cc260 unknown :-1 s=CLOSED pgs=1759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:11.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:11.328+0000 7f80c2303640 1 -- 172.21.15.27:0/2901358927 >> 172.21.15.27:0/2901358927 conn(0x7f80bc0b7c10 msgr2=0x7f80bc0af4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:11.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:11.328+0000 7f80c2303640 1 -- 172.21.15.27:0/2901358927 shutdown_connections 2024-09-12T03:24:11.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:11.328+0000 7f80c2303640 1 -- 172.21.15.27:0/2901358927 wait complete. 2024-09-12T03:24:11.342 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-12T03:24:11.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:11.455+0000 7f42e674b640 1 Processor -- start 2024-09-12T03:24:11.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:11.455+0000 7f42e674b640 1 -- start start 2024-09-12T03:24:11.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:11.455+0000 7f42e674b640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f42e00b6240 0x7f42e00b6620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:11.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:11.455+0000 7f42e674b640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f42e00d3c20 0x7f42e00d4040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:11.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:11.455+0000 7f42e674b640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f42e006d960 0x7f42e006de10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:11.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:11.455+0000 7f42e674b640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f42e00b6b60 con 0x7f42e00b6240 2024-09-12T03:24:11.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:11.455+0000 7f42e674b640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f42e00d2930 con 0x7f42e006d960 2024-09-12T03:24:11.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:11.455+0000 7f42e674b640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f42e00d2c10 con 0x7f42e00d3c20 2024-09-12T03:24:11.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:11.455+0000 7f42dffff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f42e00b6240 0x7f42e00b6620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:11.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:11.455+0000 7f42dffff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f42e00b6240 0x7f42e00b6620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41502/0 (socket says 172.21.15.27:41502) 2024-09-12T03:24:11.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:11.455+0000 7f42dffff640 1 -- 172.21.15.27:0/1558099750 learned_addr learned my addr 172.21.15.27:0/1558099750 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:24:11.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:11.455+0000 7f42df7fe640 1 --2- 172.21.15.27:0/1558099750 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f42e00d3c20 0x7f42e00d4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:11.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:11.455+0000 7f42e4ce8640 1 --2- 172.21.15.27:0/1558099750 >> v2:172.21.15.45:3300/0 conn(0x7f42e006d960 0x7f42e006de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:11.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:11.455+0000 7f42dffff640 1 -- 172.21.15.27:0/1558099750 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f42e00d3c20 msgr2=0x7f42e00d4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:24:11.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:11.455+0000 7f42dffff640 1 --2- 172.21.15.27:0/1558099750 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f42e00d3c20 0x7f42e00d4040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:11.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:11.455+0000 7f42dffff640 1 -- 172.21.15.27:0/1558099750 >> v2:172.21.15.45:3300/0 conn(0x7f42e006d960 msgr2=0x7f42e006de10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:24:11.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:11.455+0000 7f42dffff640 1 --2- 172.21.15.27:0/1558099750 >> v2:172.21.15.45:3300/0 conn(0x7f42e006d960 0x7f42e006de10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:11.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:11.455+0000 7f42dffff640 1 -- 172.21.15.27:0/1558099750 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42e00d2ef0 con 0x7f42e00b6240 2024-09-12T03:24:11.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:11.456+0000 7f42dffff640 1 --2- 172.21.15.27:0/1558099750 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f42e00b6240 0x7f42e00b6620 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7f42e0050fe0 tx=0x7f42d002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f3956bdddf7b54 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:11.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:11.456+0000 7f42deffd640 1 -- 172.21.15.27:0/1558099750 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42d0037070 con 0x7f42e00b6240 2024-09-12T03:24:11.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:11.456+0000 7f42deffd640 1 -- 172.21.15.27:0/1558099750 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f42d0032440 con 0x7f42e00b6240 2024-09-12T03:24:11.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:11.456+0000 7f42deffd640 1 -- 172.21.15.27:0/1558099750 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42d0032810 con 0x7f42e00b6240 2024-09-12T03:24:11.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:11.457+0000 7f42e674b640 1 -- 172.21.15.27:0/1558099750 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f42e00b6240 msgr2=0x7f42e00b6620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:11.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:11.457+0000 7f42e674b640 1 --2- 172.21.15.27:0/1558099750 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f42e00b6240 0x7f42e00b6620 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7f42e0050fe0 tx=0x7f42d002a270 comp rx=0 tx=0).stop 2024-09-12T03:24:11.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:11.457+0000 7f42e674b640 1 -- 172.21.15.27:0/1558099750 shutdown_connections 2024-09-12T03:24:11.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:11.457+0000 7f42e674b640 1 --2- 172.21.15.27:0/1558099750 >> v2:172.21.15.45:3300/0 conn(0x7f42e006d960 0x7f42e006de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:11.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:11.457+0000 7f42e674b640 1 --2- 172.21.15.27:0/1558099750 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f42e00d3c20 0x7f42e00d4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:11.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:11.457+0000 7f42e674b640 1 --2- 172.21.15.27:0/1558099750 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f42e00b6240 0x7f42e00b6620 unknown :-1 s=CLOSED pgs=1760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:11.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:11.457+0000 7f42e674b640 1 -- 172.21.15.27:0/1558099750 >> 172.21.15.27:0/1558099750 conn(0x7f42e00d1420 msgr2=0x7f42e00d1830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:11.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:11.457+0000 7f42e674b640 1 -- 172.21.15.27:0/1558099750 shutdown_connections 2024-09-12T03:24:11.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:11.457+0000 7f42e674b640 1 -- 172.21.15.27:0/1558099750 wait complete. 2024-09-12T03:24:11.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:11.458+0000 7f42e674b640 1 Processor -- start 2024-09-12T03:24:11.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:11.458+0000 7f42e674b640 1 -- start start 2024-09-12T03:24:11.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:11.459+0000 7f42e674b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f42e006d960 0x7f42e01579f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:11.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:11.459+0000 7f42e674b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f42e00b6240 0x7f42e00ce190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:11.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:11.459+0000 7f42e674b640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f42e00d3c20 0x7f42e00c9900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:11.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:11.459+0000 7f42e674b640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f42e015ece0 con 0x7f42e00b6240 2024-09-12T03:24:11.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:11.459+0000 7f42e674b640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f42e015eb60 con 0x7f42e006d960 2024-09-12T03:24:11.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:11.459+0000 7f42e674b640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f42e00cfc70 con 0x7f42e00d3c20 2024-09-12T03:24:11.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:11.459+0000 7f42df7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f42e00b6240 0x7f42e00ce190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:11.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:11.459+0000 7f42df7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f42e00b6240 0x7f42e00ce190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41504/0 (socket says 172.21.15.27:41504) 2024-09-12T03:24:11.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:11.459+0000 7f42df7fe640 1 -- 172.21.15.27:0/1691962725 learned_addr learned my addr 172.21.15.27:0/1691962725 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:24:11.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:11.459+0000 7f42e4ce8640 1 --2- 172.21.15.27:0/1691962725 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f42e00d3c20 0x7f42e00c9900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:11.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:11.459+0000 7f42dffff640 1 --2- 172.21.15.27:0/1691962725 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f42e006d960 0x7f42e01579f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:11.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:11.459+0000 7f42df7fe640 1 -- 172.21.15.27:0/1691962725 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f42e00d3c20 msgr2=0x7f42e00c9900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:11.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:11.459+0000 7f42df7fe640 1 --2- 172.21.15.27:0/1691962725 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f42e00d3c20 0x7f42e00c9900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:11.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:11.459+0000 7f42df7fe640 1 -- 172.21.15.27:0/1691962725 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f42e006d960 msgr2=0x7f42e01579f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:11.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:11.459+0000 7f42df7fe640 1 --2- 172.21.15.27:0/1691962725 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f42e006d960 0x7f42e01579f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:11.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:11.459+0000 7f42df7fe640 1 -- 172.21.15.27:0/1691962725 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42e00c8860 con 0x7f42e00b6240 2024-09-12T03:24:11.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:11.459+0000 7f42df7fe640 1 --2- 172.21.15.27:0/1691962725 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f42e00b6240 0x7f42e00ce190 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7f42d400d600 tx=0x7f42d400dac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:11.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:11.459+0000 7f42dcff9640 1 -- 172.21.15.27:0/1691962725 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42d4004240 con 0x7f42e00b6240 2024-09-12T03:24:11.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:11.460+0000 7f42dcff9640 1 -- 172.21.15.27:0/1691962725 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f42d40043e0 con 0x7f42e00b6240 2024-09-12T03:24:11.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:11.460+0000 7f42e674b640 1 -- 172.21.15.27:0/1691962725 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42e00c8b40 con 0x7f42e00b6240 2024-09-12T03:24:11.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:11.460+0000 7f42dcff9640 1 -- 172.21.15.27:0/1691962725 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42d4003830 con 0x7f42e00b6240 2024-09-12T03:24:11.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:11.460+0000 7f42e674b640 1 -- 172.21.15.27:0/1691962725 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42e00c9100 con 0x7f42e00b6240 2024-09-12T03:24:11.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:11.461+0000 7f42e674b640 1 -- 172.21.15.27:0/1691962725 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42e00cb550 con 0x7f42e00b6240 2024-09-12T03:24:11.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:11.461+0000 7f42dcff9640 1 -- 172.21.15.27:0/1691962725 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 26) ==== 91518+0+0 (secure 0 0 0) 0x7f42d40039d0 con 0x7f42e00b6240 2024-09-12T03:24:11.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:11.462+0000 7f42dcff9640 1 --2- 172.21.15.27:0/1691962725 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f42cc06c970 0x7f42cc06ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:11.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:11.462+0000 7f42dcff9640 1 -- 172.21.15.27:0/1691962725 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(162..162 src has 1..162) ==== 5413+0+0 (secure 0 0 0) 0x7f42d4089cc0 con 0x7f42e00b6240 2024-09-12T03:24:11.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:11.462+0000 7f42dffff640 1 --2- 172.21.15.27:0/1691962725 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f42cc06c970 0x7f42cc06ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:11.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:11.462+0000 7f42dffff640 1 --2- 172.21.15.27:0/1691962725 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f42cc06c970 0x7f42cc06ee30 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f42d0034000 tx=0x7f42d002e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:11.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:11.467+0000 7f42dcff9640 1 -- 172.21.15.27:0/1691962725 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f42d401c360 con 0x7f42e00b6240 2024-09-12T03:24:11.644 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:11.644+0000 7f42e674b640 1 -- 172.21.15.27:0/1691962725 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f42e00c9500 con 0x7f42e00b6240 2024-09-12T03:24:12.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:12.330+0000 7f42dcff9640 1 -- 172.21.15.27:0/1691962725 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v163) ==== 169+0+0 (secure 0 0 0) 0x7f42d4059ed0 con 0x7f42e00b6240 2024-09-12T03:24:12.331 INFO:teuthology.orchestra.run.smithi027.stderr:pool 'cephfs_data' removed 2024-09-12T03:24:12.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:12.335+0000 7f42e674b640 1 -- 172.21.15.27:0/1691962725 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f42cc06c970 msgr2=0x7f42cc06ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:12.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:12.335+0000 7f42e674b640 1 --2- 172.21.15.27:0/1691962725 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f42cc06c970 0x7f42cc06ee30 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f42d0034000 tx=0x7f42d002e000 comp rx=0 tx=0).stop 2024-09-12T03:24:12.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:12.335+0000 7f42e674b640 1 -- 172.21.15.27:0/1691962725 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f42e00b6240 msgr2=0x7f42e00ce190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:12.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:12.335+0000 7f42e674b640 1 --2- 172.21.15.27:0/1691962725 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f42e00b6240 0x7f42e00ce190 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7f42d400d600 tx=0x7f42d400dac0 comp rx=0 tx=0).stop 2024-09-12T03:24:12.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:12.335+0000 7f42e674b640 1 -- 172.21.15.27:0/1691962725 shutdown_connections 2024-09-12T03:24:12.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:12.335+0000 7f42e674b640 1 --2- 172.21.15.27:0/1691962725 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f42e00d3c20 0x7f42e00c9900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:12.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:12.335+0000 7f42e674b640 1 --2- 172.21.15.27:0/1691962725 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f42cc06c970 0x7f42cc06ee30 secure :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f42d0034000 tx=0x7f42d002e000 comp rx=0 tx=0).stop 2024-09-12T03:24:12.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:12.335+0000 7f42e674b640 1 --2- 172.21.15.27:0/1691962725 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f42e00b6240 0x7f42e00ce190 unknown :-1 s=CLOSED pgs=1761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:12.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:12.335+0000 7f42e674b640 1 --2- 172.21.15.27:0/1691962725 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f42e006d960 0x7f42e01579f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:12.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:12.335+0000 7f42e674b640 1 -- 172.21.15.27:0/1691962725 >> 172.21.15.27:0/1691962725 conn(0x7f42e00d1420 msgr2=0x7f42e00cf020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:12.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:12.335+0000 7f42e674b640 1 -- 172.21.15.27:0/1691962725 shutdown_connections 2024-09-12T03:24:12.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:12.335+0000 7f42e674b640 1 -- 172.21.15.27:0/1691962725 wait complete. 2024-09-12T03:24:12.349 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-09-12T03:24:12.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:12.457+0000 7fb5c8247640 1 Processor -- start 2024-09-12T03:24:12.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:12.457+0000 7fb5c8247640 1 -- start start 2024-09-12T03:24:12.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:12.458+0000 7fb5c8247640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb5c0162040 0x7fb5c00d1da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:12.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:12.458+0000 7fb5c8247640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fb5c006dfd0 0x7fb5c00b7a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:12.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:12.458+0000 7fb5c8247640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb5c00d2e30 con 0x7fb5c00d3790 2024-09-12T03:24:12.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:12.458+0000 7fb5c8247640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fb5c00b7f90 con 0x7fb5c006dfd0 2024-09-12T03:24:12.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:12.458+0000 7fb5c8247640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fb5c0162420 con 0x7fb5c0162040 2024-09-12T03:24:12.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:12.458+0000 7fb5c67e4640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fb5c006dfd0 0x7fb5c00b7a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:12.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:12.458+0000 7fb5c5fe3640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb5c0162040 0x7fb5c00d1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:12.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:12.458+0000 7fb5c57e2640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fb5c00d3790 0x7fb5c006d8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.27:38720/0 (socket says 172.21.15.27:38720) 2024-09-12T03:24:12.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:12.458+0000 7fb5c57e2640 1 -- 172.21.15.27:0/3493832605 learned_addr learned my addr 172.21.15.27:0/3493832605 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:24:12.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:12.459+0000 7fb5c5fe3640 1 -- 172.21.15.27:0/3493832605 >> v2:172.21.15.45:3300/0 conn(0x7fb5c006dfd0 msgr2=0x7fb5c00b7a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:24:12.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:12.459+0000 7fb5c5fe3640 1 --2- 172.21.15.27:0/3493832605 >> v2:172.21.15.45:3300/0 conn(0x7fb5c006dfd0 0x7fb5c00b7a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:12.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:12.459+0000 7fb5c5fe3640 1 -- 172.21.15.27:0/3493832605 >> v1:172.21.15.45:6789/0 conn(0x7fb5c00d3790 legacy=0x7fb5c006d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:12.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:12.459+0000 7fb5c5fe3640 1 -- 172.21.15.27:0/3493832605 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5c00d30d0 con 0x7fb5c0162040 2024-09-12T03:24:12.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:12.459+0000 7fb5c67e4640 1 --2- 172.21.15.27:0/3493832605 >> v2:172.21.15.45:3300/0 conn(0x7fb5c006dfd0 0x7fb5c00b7a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:24:12.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:12.460+0000 7fb5c5fe3640 1 --2- 172.21.15.27:0/3493832605 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb5c0162040 0x7fb5c00d1da0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fb5b80097a0 tx=0x7fb5b802a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=29166e545af18db9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:12.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:12.460+0000 7fb5c4fe1640 1 -- 172.21.15.27:0/3493832605 <== mon.2 v2:172.21.15.45:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5b8037070 con 0x7fb5c0162040 2024-09-12T03:24:12.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:12.460+0000 7fb5c4fe1640 1 -- 172.21.15.27:0/3493832605 <== mon.2 v2:172.21.15.45:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5b8032440 con 0x7fb5c0162040 2024-09-12T03:24:12.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:12.460+0000 7fb5c4fe1640 1 -- 172.21.15.27:0/3493832605 <== mon.2 v2:172.21.15.45:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5b8032810 con 0x7fb5c0162040 2024-09-12T03:24:12.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:12.460+0000 7fb5c8247640 1 -- 172.21.15.27:0/3493832605 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb5c0162040 msgr2=0x7fb5c00d1da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:12.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:12.460+0000 7fb5c8247640 1 --2- 172.21.15.27:0/3493832605 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb5c0162040 0x7fb5c00d1da0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fb5b80097a0 tx=0x7fb5b802a180 comp rx=0 tx=0).stop 2024-09-12T03:24:12.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:12.460+0000 7fb5c8247640 1 -- 172.21.15.27:0/3493832605 shutdown_connections 2024-09-12T03:24:12.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:12.460+0000 7fb5c8247640 1 --2- 172.21.15.27:0/3493832605 >> v2:172.21.15.45:3300/0 conn(0x7fb5c006dfd0 0x7fb5c00b7a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:12.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:12.460+0000 7fb5c8247640 1 --2- 172.21.15.27:0/3493832605 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb5c0162040 0x7fb5c00d1da0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:12.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:12.460+0000 7fb5c8247640 1 -- 172.21.15.27:0/3493832605 >> 172.21.15.27:0/3493832605 conn(0x7fb5c00b6700 msgr2=0x7fb5c00b6b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:12.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:12.461+0000 7fb5c8247640 1 -- 172.21.15.27:0/3493832605 shutdown_connections 2024-09-12T03:24:12.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:12.461+0000 7fb5c8247640 1 -- 172.21.15.27:0/3493832605 wait complete. 2024-09-12T03:24:12.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:12.462+0000 7fb5c8247640 1 Processor -- start 2024-09-12T03:24:12.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:12.462+0000 7fb5c8247640 1 -- start start 2024-09-12T03:24:12.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:12.462+0000 7fb5c8247640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb5c006dfd0 0x7fb5c015f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:12.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:12.462+0000 7fb5c8247640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb5c00d3790 0x7fb5c014ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:12.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:12.462+0000 7fb5c8247640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb5c0162040 0x7fb5c00ca840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:12.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:12.462+0000 7fb5c8247640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fb5c00d4530 con 0x7fb5c0162040 2024-09-12T03:24:12.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:12.462+0000 7fb5c8247640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb5c00cee00 con 0x7fb5c00d3790 2024-09-12T03:24:12.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:12.462+0000 7fb5c8247640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fb5c00d4150 con 0x7fb5c006dfd0 2024-09-12T03:24:12.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:12.463+0000 7fb5c67e4640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb5c0162040 0x7fb5c00ca840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:12.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:12.463+0000 7fb5c67e4640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb5c0162040 0x7fb5c00ca840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41510/0 (socket says 172.21.15.27:41510) 2024-09-12T03:24:12.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:12.463+0000 7fb5c67e4640 1 -- 172.21.15.27:0/1891671484 learned_addr learned my addr 172.21.15.27:0/1891671484 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:24:12.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:12.463+0000 7fb5c5fe3640 1 --2- 172.21.15.27:0/1891671484 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb5c006dfd0 0x7fb5c015f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:12.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:12.463+0000 7fb5c57e2640 1 --2- 172.21.15.27:0/1891671484 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb5c00d3790 0x7fb5c014ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:12.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:12.463+0000 7fb5c67e4640 1 -- 172.21.15.27:0/1891671484 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb5c006dfd0 msgr2=0x7fb5c015f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:12.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:12.463+0000 7fb5c67e4640 1 --2- 172.21.15.27:0/1891671484 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb5c006dfd0 0x7fb5c015f380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:12.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:12.463+0000 7fb5c67e4640 1 -- 172.21.15.27:0/1891671484 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb5c00d3790 msgr2=0x7fb5c014ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:12.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:12.463+0000 7fb5c67e4640 1 --2- 172.21.15.27:0/1891671484 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb5c00d3790 0x7fb5c014ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:12.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:12.463+0000 7fb5c67e4640 1 -- 172.21.15.27:0/1891671484 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5c00cad80 con 0x7fb5c0162040 2024-09-12T03:24:12.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:12.463+0000 7fb5c5fe3640 1 --2- 172.21.15.27:0/1891671484 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb5c006dfd0 0x7fb5c015f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:24:12.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:12.463+0000 7fb5c57e2640 1 --2- 172.21.15.27:0/1891671484 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb5c00d3790 0x7fb5c014ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:24:12.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:12.463+0000 7fb5c67e4640 1 --2- 172.21.15.27:0/1891671484 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb5c0162040 0x7fb5c00ca840 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 crypto rx=0x7fb5b000efc0 tx=0x7fb5b0009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:12.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:12.463+0000 7fb5be7fc640 1 -- 172.21.15.27:0/1891671484 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5b0018020 con 0x7fb5c0162040 2024-09-12T03:24:12.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:12.463+0000 7fb5be7fc640 1 -- 172.21.15.27:0/1891671484 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5b0014480 con 0x7fb5c0162040 2024-09-12T03:24:12.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:12.463+0000 7fb5be7fc640 1 -- 172.21.15.27:0/1891671484 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5b0014870 con 0x7fb5c0162040 2024-09-12T03:24:12.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:12.463+0000 7fb5c8247640 1 -- 172.21.15.27:0/1891671484 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5c015f960 con 0x7fb5c0162040 2024-09-12T03:24:12.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:12.463+0000 7fb5c8247640 1 -- 172.21.15.27:0/1891671484 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5c015a370 con 0x7fb5c0162040 2024-09-12T03:24:12.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:12.465+0000 7fb5c8247640 1 -- 172.21.15.27:0/1891671484 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5c01617e0 con 0x7fb5c0162040 2024-09-12T03:24:12.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:12.465+0000 7fb5be7fc640 1 -- 172.21.15.27:0/1891671484 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 26) ==== 91518+0+0 (secure 0 0 0) 0x7fb5b0005d10 con 0x7fb5c0162040 2024-09-12T03:24:12.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:12.465+0000 7fb5be7fc640 1 --2- 172.21.15.27:0/1891671484 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fb5ac06c9a0 0x7fb5ac06ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:12.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:12.466+0000 7fb5be7fc640 1 -- 172.21.15.27:0/1891671484 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(163..163 src has 1..163) ==== 4999+0+0 (secure 0 0 0) 0x7fb5b008a3d0 con 0x7fb5c0162040 2024-09-12T03:24:12.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:12.466+0000 7fb5c5fe3640 1 --2- 172.21.15.27:0/1891671484 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fb5ac06c9a0 0x7fb5ac06ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:12.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:12.466+0000 7fb5c5fe3640 1 --2- 172.21.15.27:0/1891671484 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fb5ac06c9a0 0x7fb5ac06ee60 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fb5b8034000 tx=0x7fb5b802e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:12.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:12.470+0000 7fb5be7fc640 1 -- 172.21.15.27:0/1891671484 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fb5b0022030 con 0x7fb5c0162040 2024-09-12T03:24:12.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:12.647+0000 7fb5c8247640 1 -- 172.21.15.27:0/1891671484 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7fb5c00b7a50 con 0x7fb5c0162040 2024-09-12T03:24:12.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:12.650+0000 7fb5be7fc640 1 -- 172.21.15.27:0/1891671484 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v31) ==== 104+0+0 (secure 0 0 0) 0x7fb5b0010040 con 0x7fb5c0162040 2024-09-12T03:24:12.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:12.652+0000 7fb5c8247640 1 -- 172.21.15.27:0/1891671484 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fb5ac06c9a0 msgr2=0x7fb5ac06ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:12.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:12.652+0000 7fb5c8247640 1 --2- 172.21.15.27:0/1891671484 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fb5ac06c9a0 0x7fb5ac06ee60 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fb5b8034000 tx=0x7fb5b802e000 comp rx=0 tx=0).stop 2024-09-12T03:24:12.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:12.652+0000 7fb5c8247640 1 -- 172.21.15.27:0/1891671484 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb5c0162040 msgr2=0x7fb5c00ca840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:12.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:12.652+0000 7fb5c8247640 1 --2- 172.21.15.27:0/1891671484 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb5c0162040 0x7fb5c00ca840 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 crypto rx=0x7fb5b000efc0 tx=0x7fb5b0009f90 comp rx=0 tx=0).stop 2024-09-12T03:24:12.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:12.652+0000 7fb5c8247640 1 -- 172.21.15.27:0/1891671484 shutdown_connections 2024-09-12T03:24:12.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:12.652+0000 7fb5c8247640 1 --2- 172.21.15.27:0/1891671484 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fb5ac06c9a0 0x7fb5ac06ee60 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:12.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:12.652+0000 7fb5c8247640 1 --2- 172.21.15.27:0/1891671484 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb5c0162040 0x7fb5c00ca840 unknown :-1 s=CLOSED pgs=1762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:12.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:12.652+0000 7fb5c8247640 1 --2- 172.21.15.27:0/1891671484 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb5c00d3790 0x7fb5c014ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:12.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:12.652+0000 7fb5c8247640 1 --2- 172.21.15.27:0/1891671484 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fb5c006dfd0 0x7fb5c015f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:12.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:12.652+0000 7fb5c8247640 1 -- 172.21.15.27:0/1891671484 >> 172.21.15.27:0/1891671484 conn(0x7fb5c00b6700 msgr2=0x7fb5c006d8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:12.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:12.652+0000 7fb5c8247640 1 -- 172.21.15.27:0/1891671484 shutdown_connections 2024-09-12T03:24:12.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:12.652+0000 7fb5c8247640 1 -- 172.21.15.27:0/1891671484 wait complete. 2024-09-12T03:24:12.663 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestMisc.test_unparseable_metafile_on_legacy_to_v1_upgrade' 2024-09-12T03:24:12.768 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:12.767+0000 7f8bcd34b640 1 Processor -- start 2024-09-12T03:24:12.768 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:12.767+0000 7f8bcd34b640 1 -- start start 2024-09-12T03:24:12.769 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:12.768+0000 7f8bcd34b640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f8bc8167250 0x7f8bc8169640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:12.769 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:12.768+0000 7f8bcd34b640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8bc80701e0 con 0x7f8bc80ae8d0 2024-09-12T03:24:12.769 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:12.768+0000 7f8bcd34b640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8bc806e130 con 0x7f8bc80b3570 2024-09-12T03:24:12.769 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:12.768+0000 7f8bcd34b640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f8bc80b0b90 con 0x7f8bc8167250 2024-09-12T03:24:12.769 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:12.768+0000 7f8bc6ffd640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f8bc80ae8d0 0x7f8bc80aecb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:38882/0 (socket says 172.21.15.27:38882) 2024-09-12T03:24:12.769 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:12.768+0000 7f8bc6ffd640 1 -- 172.21.15.27:0/673407447 learned_addr learned my addr 172.21.15.27:0/673407447 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:24:12.770 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:12.769+0000 7f8bc77fe640 1 --2- 172.21.15.27:0/673407447 >> v2:172.21.15.45:3300/0 conn(0x7f8bc8167250 0x7f8bc8169640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:12.770 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:12.769+0000 7f8bc5ffb640 1 -- 172.21.15.27:0/673407447 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3902609217 0 0) 0x7f8bc80701e0 con 0x7f8bc80ae8d0 2024-09-12T03:24:12.770 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:12.769+0000 7f8bc5ffb640 1 -- 172.21.15.27:0/673407447 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8bb4003520 con 0x7f8bc80ae8d0 2024-09-12T03:24:12.770 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:12.769+0000 7f8bc5ffb640 1 -- 172.21.15.27:0/673407447 <== mon.1 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 233474806 0 0) 0x7f8bc806e130 con 0x7f8bc80b3570 2024-09-12T03:24:12.770 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:12.769+0000 7f8bc5ffb640 1 -- 172.21.15.27:0/673407447 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8bc80701e0 con 0x7f8bc80b3570 2024-09-12T03:24:12.770 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:12.769+0000 7f8bc77fe640 1 -- 172.21.15.27:0/673407447 >> v1:172.21.15.45:6789/0 conn(0x7f8bc80b3570 legacy=0x7f8bc80b39b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:12.770 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:12.769+0000 7f8bc77fe640 1 -- 172.21.15.27:0/673407447 >> v1:172.21.15.27:6789/0 conn(0x7f8bc80ae8d0 legacy=0x7f8bc80aecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:12.770 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:12.769+0000 7f8bc77fe640 1 -- 172.21.15.27:0/673407447 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bc806e410 con 0x7f8bc8167250 2024-09-12T03:24:12.771 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:12.770+0000 7f8bc77fe640 1 --2- 172.21.15.27:0/673407447 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8bc8167250 0x7f8bc8169640 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f8bb8009d90 tx=0x7f8bb802a6f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=52ff85993f3b869b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:12.771 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:12.770+0000 7f8bc5ffb640 1 -- 172.21.15.27:0/673407447 <== mon.1 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bb8007740 con 0x7f8bc8167250 2024-09-12T03:24:12.771 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:12.770+0000 7f8bc5ffb640 1 -- 172.21.15.27:0/673407447 <== mon.1 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8bb80078e0 con 0x7f8bc8167250 2024-09-12T03:24:12.771 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:12.770+0000 7f8bc5ffb640 1 -- 172.21.15.27:0/673407447 <== mon.1 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bb8007c90 con 0x7f8bc8167250 2024-09-12T03:24:12.771 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:12.770+0000 7f8bcd34b640 1 -- 172.21.15.27:0/673407447 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8bc8167250 msgr2=0x7f8bc8169640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:12.772 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:12.770+0000 7f8bcd34b640 1 --2- 172.21.15.27:0/673407447 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8bc8167250 0x7f8bc8169640 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f8bb8009d90 tx=0x7f8bb802a6f0 comp rx=0 tx=0).stop 2024-09-12T03:24:12.772 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:12.771+0000 7f8bcd34b640 1 -- 172.21.15.27:0/673407447 shutdown_connections 2024-09-12T03:24:12.772 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:12.771+0000 7f8bcd34b640 1 --2- 172.21.15.27:0/673407447 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8bc8167250 0x7f8bc8169640 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:12.772 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:12.771+0000 7f8bcd34b640 1 -- 172.21.15.27:0/673407447 >> 172.21.15.27:0/673407447 conn(0x7f8bc806ce20 msgr2=0x7f8bc806d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:12.772 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:12.771+0000 7f8bcd34b640 1 -- 172.21.15.27:0/673407447 shutdown_connections 2024-09-12T03:24:12.772 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:12.771+0000 7f8bcd34b640 1 -- 172.21.15.27:0/673407447 wait complete. 2024-09-12T03:24:12.774 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:12.772+0000 7f8bcd34b640 1 Processor -- start 2024-09-12T03:24:12.774 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:12.772+0000 7f8bcd34b640 1 -- start start 2024-09-12T03:24:12.774 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:12.773+0000 7f8bcd34b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8bc80ae8d0 0x7f8bc81391b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:12.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:12.773+0000 7f8bcd34b640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f8bc80b3570 0x7f8bc813d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:12.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:12.773+0000 7f8bcd34b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8bc8167250 0x7f8bc813d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:12.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:12.773+0000 7f8bcd34b640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f8bc806e9d0 con 0x7f8bc8167250 2024-09-12T03:24:12.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:12.773+0000 7f8bcd34b640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f8bc806ed40 con 0x7f8bc80ae8d0 2024-09-12T03:24:12.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:12.773+0000 7f8bcd34b640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f8bc8147770 con 0x7f8bc80b3570 2024-09-12T03:24:12.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:12.773+0000 7f8bc77fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8bc8167250 0x7f8bc813d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:12.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:12.773+0000 7f8bc77fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8bc8167250 0x7f8bc813d7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41524/0 (socket says 172.21.15.27:41524) 2024-09-12T03:24:12.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:12.773+0000 7f8bc77fe640 1 -- 172.21.15.27:0/1023684759 learned_addr learned my addr 172.21.15.27:0/1023684759 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:24:12.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:12.773+0000 7f8bc6ffd640 1 --2- 172.21.15.27:0/1023684759 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8bc80ae8d0 0x7f8bc81391b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:12.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:12.773+0000 7f8bc67fc640 1 --2- 172.21.15.27:0/1023684759 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f8bc80b3570 0x7f8bc813d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:12.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:12.773+0000 7f8bc77fe640 1 -- 172.21.15.27:0/1023684759 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f8bc80b3570 msgr2=0x7f8bc813d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:12.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:12.773+0000 7f8bc77fe640 1 --2- 172.21.15.27:0/1023684759 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f8bc80b3570 0x7f8bc813d270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:12.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:12.773+0000 7f8bc77fe640 1 -- 172.21.15.27:0/1023684759 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8bc80ae8d0 msgr2=0x7f8bc81391b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:12.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:12.773+0000 7f8bc77fe640 1 --2- 172.21.15.27:0/1023684759 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8bc80ae8d0 0x7f8bc81391b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:12.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:12.773+0000 7f8bc77fe640 1 -- 172.21.15.27:0/1023684759 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bc8161680 con 0x7f8bc8167250 2024-09-12T03:24:12.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:12.773+0000 7f8bc77fe640 1 --2- 172.21.15.27:0/1023684759 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8bc8167250 0x7f8bc813d7b0 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7f8bb802ace0 tx=0x7f8bb8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:12.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:12.774+0000 7f8bbf7fe640 1 -- 172.21.15.27:0/1023684759 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bb8009450 con 0x7f8bc8167250 2024-09-12T03:24:12.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:12.774+0000 7f8bbf7fe640 1 -- 172.21.15.27:0/1023684759 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8bb8004260 con 0x7f8bc8167250 2024-09-12T03:24:12.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:12.774+0000 7f8bbf7fe640 1 -- 172.21.15.27:0/1023684759 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bb8004650 con 0x7f8bc8167250 2024-09-12T03:24:12.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:12.774+0000 7f8bcd34b640 1 -- 172.21.15.27:0/1023684759 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8bc81610c0 con 0x7f8bc8167250 2024-09-12T03:24:12.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:12.774+0000 7f8bcd34b640 1 -- 172.21.15.27:0/1023684759 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8bc813a6e0 con 0x7f8bc8167250 2024-09-12T03:24:12.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:12.775+0000 7f8bcd34b640 1 -- 172.21.15.27:0/1023684759 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8bc80b4570 con 0x7f8bc8167250 2024-09-12T03:24:12.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:12.775+0000 7f8bbf7fe640 1 -- 172.21.15.27:0/1023684759 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 26) ==== 91518+0+0 (secure 0 0 0) 0x7f8bb8039040 con 0x7f8bc8167250 2024-09-12T03:24:12.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:12.776+0000 7f8bbf7fe640 1 --2- 172.21.15.27:0/1023684759 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f8ba806c970 0x7f8ba806ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:12.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:12.776+0000 7f8bbf7fe640 1 -- 172.21.15.27:0/1023684759 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(163..163 src has 1..163) ==== 4999+0+0 (secure 0 0 0) 0x7f8bb80aae60 con 0x7f8bc8167250 2024-09-12T03:24:12.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:12.776+0000 7f8bc6ffd640 1 --2- 172.21.15.27:0/1023684759 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f8ba806c970 0x7f8ba806ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:12.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:12.776+0000 7f8bc6ffd640 1 --2- 172.21.15.27:0/1023684759 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f8ba806c970 0x7f8ba806ee30 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f8bc0000f60 tx=0x7f8bc0009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:12.780 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:12.780+0000 7f8bbf7fe640 1 -- 172.21.15.27:0/1023684759 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f8bb8040030 con 0x7f8bc8167250 2024-09-12T03:24:13.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:13.028+0000 7f8bcd34b640 1 -- 172.21.15.27:0/1023684759 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_unparseable_metafile_on_legacy_to_v1_upgrade"]} v 0) -- 0x7f8bc80d0fc0 con 0x7f8bc8167250 2024-09-12T03:24:13.332 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:13.332+0000 7f8bbf7fe640 1 -- 172.21.15.27:0/1023684759 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_unparseable_metafile_on_legacy_to_v1_upgrade"]}]=0 v425) ==== 163+0+0 (secure 0 0 0) 0x7f8bb807ab40 con 0x7f8bc8167250 2024-09-12T03:24:13.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:13.334+0000 7f8bcd34b640 1 -- 172.21.15.27:0/1023684759 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f8ba806c970 msgr2=0x7f8ba806ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:13.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:13.334+0000 7f8bcd34b640 1 --2- 172.21.15.27:0/1023684759 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f8ba806c970 0x7f8ba806ee30 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f8bc0000f60 tx=0x7f8bc0009040 comp rx=0 tx=0).stop 2024-09-12T03:24:13.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:13.334+0000 7f8bcd34b640 1 -- 172.21.15.27:0/1023684759 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8bc8167250 msgr2=0x7f8bc813d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:13.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:13.334+0000 7f8bcd34b640 1 --2- 172.21.15.27:0/1023684759 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8bc8167250 0x7f8bc813d7b0 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7f8bb802ace0 tx=0x7f8bb8004a40 comp rx=0 tx=0).stop 2024-09-12T03:24:13.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:13.334+0000 7f8bcd34b640 1 -- 172.21.15.27:0/1023684759 shutdown_connections 2024-09-12T03:24:13.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:13.334+0000 7f8bcd34b640 1 --2- 172.21.15.27:0/1023684759 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f8ba806c970 0x7f8ba806ee30 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:13.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:13.334+0000 7f8bcd34b640 1 --2- 172.21.15.27:0/1023684759 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8bc8167250 0x7f8bc813d7b0 unknown :-1 s=CLOSED pgs=1764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:13.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:13.334+0000 7f8bcd34b640 1 --2- 172.21.15.27:0/1023684759 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f8bc80b3570 0x7f8bc813d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:13.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:13.334+0000 7f8bcd34b640 1 --2- 172.21.15.27:0/1023684759 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8bc80ae8d0 0x7f8bc81391b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:13.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:13.334+0000 7f8bcd34b640 1 -- 172.21.15.27:0/1023684759 >> 172.21.15.27:0/1023684759 conn(0x7f8bc806ce20 msgr2=0x7f8bc80b1280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:13.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:13.334+0000 7f8bcd34b640 1 -- 172.21.15.27:0/1023684759 shutdown_connections 2024-09-12T03:24:13.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:13.335+0000 7f8bcd34b640 1 -- 172.21.15.27:0/1023684759 wait complete. 2024-09-12T03:24:13.346 INFO:tasks.cephfs_test_runner:Validate unparseable text .meta file on legacy subvol root doesn't break the system ... ok 2024-09-12T03:24:13.346 INFO:tasks.cephfs_test_runner: 2024-09-12T03:24:13.346 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-09-12T03:24:13.346 INFO:tasks.cephfs_test_runner:Ran 11 tests in 439.054s 2024-09-12T03:24:13.347 INFO:tasks.cephfs_test_runner: 2024-09-12T03:24:13.347 INFO:tasks.cephfs_test_runner:OK 2024-09-12T03:24:13.347 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2024-09-12T03:24:13.359 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-09-12T03:24:13.399 INFO:tasks.kclient:Unmounting kernel clients... 2024-09-12T03:24:13.399 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-12T03:24:13.399 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T03:24:13.416 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056004,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497432k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16324 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248716k,nr_inodes=812179,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497432k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-12T03:24:13.416 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-12T03:24:13.416 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T03:24:13.473 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056004,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497432k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16324 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248716k,nr_inodes=812179,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497432k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-12T03:24:13.473 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-12T03:24:13.473 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T03:24:13.494 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056795,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16602 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249348k,nr_inodes=812337,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-12T03:24:13.494 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-12T03:24:13.494 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T03:24:13.555 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056795,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16602 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249348k,nr_inodes=812337,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-12T03:24:13.556 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:24:13.556 DEBUG:teuthology.orchestra.run.smithi027:> ip netns list 2024-09-12T03:24:13.572 INFO:teuthology.orchestra.run.smithi027.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-12T03:24:13.572 INFO:teuthology.orchestra.run.smithi027.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-12T03:24:13.572 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:24:13.572 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-12T03:24:13.639 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:24:13.640 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-12T03:24:13.706 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:24:13.706 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip link delete ceph-brx 2024-09-12T03:24:13.806 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:24:13.826 DEBUG:teuthology.orchestra.run.smithi027:> ip netns list 2024-09-12T03:24:13.842 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:24:13.842 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip link delete ceph-brx 2024-09-12T03:24:13.912 INFO:teuthology.orchestra.run.smithi027.stderr:Cannot find device "ceph-brx" 2024-09-12T03:24:13.913 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T03:24:13.913 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:24:13.914 DEBUG:teuthology.orchestra.run.smithi045:> ip netns list 2024-09-12T03:24:13.934 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-09-12T03:24:13.934 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-12T03:24:13.934 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:24:13.934 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-12T03:24:14.006 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:24:14.006 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-12T03:24:14.077 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:24:14.077 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link delete ceph-brx 2024-09-12T03:24:14.175 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:24:14.175 DEBUG:teuthology.orchestra.run.smithi045:> ip netns list 2024-09-12T03:24:14.192 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:24:14.192 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link delete ceph-brx 2024-09-12T03:24:14.263 INFO:teuthology.orchestra.run.smithi045.stderr:Cannot find device "ceph-brx" 2024-09-12T03:24:14.264 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T03:24:14.264 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-09-12T03:24:14.273 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-12T03:24:14.273 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-12T03:24:14.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.383+0000 7fac72fcf640 1 Processor -- start 2024-09-12T03:24:14.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.383+0000 7fac72fcf640 1 -- start start 2024-09-12T03:24:14.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.384+0000 7fac72fcf640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fac6c15b2b0 0x7fac6c0b3560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:14.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.384+0000 7fac72fcf640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fac6c06e9b0 0x7fac6c06ee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:14.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.384+0000 7fac72fcf640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fac6c0ce410 con 0x7fac6c072d70 2024-09-12T03:24:14.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.384+0000 7fac72fcf640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fac6c0720a0 con 0x7fac6c15b2b0 2024-09-12T03:24:14.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.384+0000 7fac72fcf640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fac6c0732c0 con 0x7fac6c06e9b0 2024-09-12T03:24:14.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.385+0000 7fac7156c640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fac6c06e9b0 0x7fac6c06ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:14.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.385+0000 7fac70d6b640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fac6c15b2b0 0x7fac6c0b3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:14.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.385+0000 7fac7156c640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fac6c06e9b0 0x7fac6c06ee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3301/0 says I am v2:172.21.15.27:59060/0 (socket says 172.21.15.27:59060) 2024-09-12T03:24:14.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.385+0000 7fac6bfff640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fac6c072d70 0x7fac6c072430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.27:38740/0 (socket says 172.21.15.27:38740) 2024-09-12T03:24:14.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.385+0000 7fac7156c640 1 -- 172.21.15.27:0/663536064 learned_addr learned my addr 172.21.15.27:0/663536064 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:24:14.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.385+0000 7fac70d6b640 1 -- 172.21.15.27:0/663536064 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fac6c06e9b0 msgr2=0x7fac6c06ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:24:14.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.385+0000 7fac70d6b640 1 --2- 172.21.15.27:0/663536064 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fac6c06e9b0 0x7fac6c06ee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:14.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.385+0000 7fac70d6b640 1 -- 172.21.15.27:0/663536064 >> v1:172.21.15.45:6789/0 conn(0x7fac6c072d70 legacy=0x7fac6c072430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:14.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.385+0000 7fac70d6b640 1 -- 172.21.15.27:0/663536064 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac6c071ae0 con 0x7fac6c15b2b0 2024-09-12T03:24:14.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.385+0000 7fac6b7fe640 1 -- 172.21.15.27:0/663536064 <== mon.1 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1289693029 0 0) 0x7fac54001490 con 0x7fac6c072d70 2024-09-12T03:24:14.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.386+0000 7fac70d6b640 1 --2- 172.21.15.27:0/663536064 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fac6c15b2b0 0x7fac6c0b3560 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fac5c0097a0 tx=0x7fac5c02a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=97b7b438dc55aaa5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:14.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.386+0000 7fac6b7fe640 1 -- 172.21.15.27:0/663536064 <== mon.1 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac5c037070 con 0x7fac6c15b2b0 2024-09-12T03:24:14.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.386+0000 7fac6b7fe640 1 -- 172.21.15.27:0/663536064 <== mon.1 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac5c032440 con 0x7fac6c15b2b0 2024-09-12T03:24:14.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.386+0000 7fac6b7fe640 1 -- 172.21.15.27:0/663536064 <== mon.1 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac5c032810 con 0x7fac6c15b2b0 2024-09-12T03:24:14.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.386+0000 7fac72fcf640 1 -- 172.21.15.27:0/663536064 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fac6c15b2b0 msgr2=0x7fac6c0b3560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:14.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.387+0000 7fac72fcf640 1 --2- 172.21.15.27:0/663536064 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fac6c15b2b0 0x7fac6c0b3560 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fac5c0097a0 tx=0x7fac5c02a180 comp rx=0 tx=0).stop 2024-09-12T03:24:14.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.387+0000 7fac72fcf640 1 -- 172.21.15.27:0/663536064 shutdown_connections 2024-09-12T03:24:14.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.387+0000 7fac72fcf640 1 --2- 172.21.15.27:0/663536064 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fac6c06e9b0 0x7fac6c06ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:14.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.387+0000 7fac72fcf640 1 --2- 172.21.15.27:0/663536064 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fac6c15b2b0 0x7fac6c0b3560 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:14.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.387+0000 7fac72fcf640 1 -- 172.21.15.27:0/663536064 >> 172.21.15.27:0/663536064 conn(0x7fac6c062c60 msgr2=0x7fac6c06bb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:14.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.387+0000 7fac72fcf640 1 -- 172.21.15.27:0/663536064 shutdown_connections 2024-09-12T03:24:14.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.387+0000 7fac72fcf640 1 -- 172.21.15.27:0/663536064 wait complete. 2024-09-12T03:24:14.390 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.388+0000 7fac72fcf640 1 Processor -- start 2024-09-12T03:24:14.390 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.388+0000 7fac72fcf640 1 -- start start 2024-09-12T03:24:14.390 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.389+0000 7fac72fcf640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fac6c06e9b0 0x7fac6c15a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:14.390 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.389+0000 7fac72fcf640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fac6c072d70 0x7fac6c155e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:14.390 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.389+0000 7fac72fcf640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fac6c15b2b0 0x7fac6c152230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:14.390 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.389+0000 7fac72fcf640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fac6c15b690 con 0x7fac6c072d70 2024-09-12T03:24:14.390 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.389+0000 7fac72fcf640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fac6c0ce690 con 0x7fac6c15b2b0 2024-09-12T03:24:14.390 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.389+0000 7fac72fcf640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fac6c15b810 con 0x7fac6c06e9b0 2024-09-12T03:24:14.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.389+0000 7fac6bfff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fac6c072d70 0x7fac6c155e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:14.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.389+0000 7fac6bfff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fac6c072d70 0x7fac6c155e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41530/0 (socket says 172.21.15.27:41530) 2024-09-12T03:24:14.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.389+0000 7fac6bfff640 1 -- 172.21.15.27:0/4113576972 learned_addr learned my addr 172.21.15.27:0/4113576972 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:24:14.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.389+0000 7fac70d6b640 1 --2- 172.21.15.27:0/4113576972 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fac6c06e9b0 0x7fac6c15a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:14.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.389+0000 7fac7156c640 1 --2- 172.21.15.27:0/4113576972 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fac6c15b2b0 0x7fac6c152230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:14.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.389+0000 7fac6bfff640 1 -- 172.21.15.27:0/4113576972 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fac6c06e9b0 msgr2=0x7fac6c15a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:14.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.389+0000 7fac6bfff640 1 --2- 172.21.15.27:0/4113576972 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fac6c06e9b0 0x7fac6c15a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:14.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.389+0000 7fac6bfff640 1 -- 172.21.15.27:0/4113576972 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fac6c15b2b0 msgr2=0x7fac6c152230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:14.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.389+0000 7fac6bfff640 1 --2- 172.21.15.27:0/4113576972 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fac6c15b2b0 0x7fac6c152230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:14.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.389+0000 7fac6bfff640 1 -- 172.21.15.27:0/4113576972 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac6c15a9a0 con 0x7fac6c072d70 2024-09-12T03:24:14.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.389+0000 7fac6bfff640 1 --2- 172.21.15.27:0/4113576972 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fac6c072d70 0x7fac6c155e70 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7fac5400c850 tx=0x7fac5400cd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:14.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.390+0000 7fac697fa640 1 -- 172.21.15.27:0/4113576972 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac54018070 con 0x7fac6c072d70 2024-09-12T03:24:14.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.390+0000 7fac697fa640 1 -- 172.21.15.27:0/4113576972 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac5400a640 con 0x7fac6c072d70 2024-09-12T03:24:14.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.390+0000 7fac72fcf640 1 -- 172.21.15.27:0/4113576972 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac6c154dd0 con 0x7fac6c072d70 2024-09-12T03:24:14.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.390+0000 7fac697fa640 1 -- 172.21.15.27:0/4113576972 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac5400aa70 con 0x7fac6c072d70 2024-09-12T03:24:14.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.390+0000 7fac72fcf640 1 -- 172.21.15.27:0/4113576972 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac6c155390 con 0x7fac6c072d70 2024-09-12T03:24:14.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.391+0000 7fac72fcf640 1 -- 172.21.15.27:0/4113576972 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac6c0b3560 con 0x7fac6c072d70 2024-09-12T03:24:14.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.391+0000 7fac697fa640 1 -- 172.21.15.27:0/4113576972 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 26) ==== 91518+0+0 (secure 0 0 0) 0x7fac5400ac10 con 0x7fac6c072d70 2024-09-12T03:24:14.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.392+0000 7fac697fa640 1 --2- 172.21.15.27:0/4113576972 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fac4c06c9a0 0x7fac4c06ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:14.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.392+0000 7fac697fa640 1 -- 172.21.15.27:0/4113576972 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(163..163 src has 1..163) ==== 4999+0+0 (secure 0 0 0) 0x7fac54088fe0 con 0x7fac6c072d70 2024-09-12T03:24:14.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.392+0000 7fac70d6b640 1 --2- 172.21.15.27:0/4113576972 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fac4c06c9a0 0x7fac4c06ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:14.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.392+0000 7fac70d6b640 1 --2- 172.21.15.27:0/4113576972 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fac4c06c9a0 0x7fac4c06ee60 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fac5c034000 tx=0x7fac5c02e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:14.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.396+0000 7fac697fa640 1 -- 172.21.15.27:0/4113576972 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fac6c0ce410 con 0x7fac6c072d70 2024-09-12T03:24:14.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.568+0000 7fac72fcf640 1 -- 172.21.15.27:0/4113576972 --> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fac6c0cceb0 con 0x7fac4c06c9a0 2024-09-12T03:24:14.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.569+0000 7fac697fa640 1 -- 172.21.15.27:0/4113576972 <== mgr.4113 v2:172.21.15.27:6836/1226920526 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13498 (secure 0 0 0) 0x7fac5c0326f0 con 0x7fac4c06c9a0 2024-09-12T03:24:14.569 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:24:14.569 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2024-09-12T03:24:14.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.571+0000 7fac72fcf640 1 -- 172.21.15.27:0/4113576972 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fac4c06c9a0 msgr2=0x7fac4c06ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:14.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.571+0000 7fac72fcf640 1 --2- 172.21.15.27:0/4113576972 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fac4c06c9a0 0x7fac4c06ee60 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fac5c034000 tx=0x7fac5c02e000 comp rx=0 tx=0).stop 2024-09-12T03:24:14.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.571+0000 7fac72fcf640 1 -- 172.21.15.27:0/4113576972 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fac6c072d70 msgr2=0x7fac6c155e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:14.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.571+0000 7fac72fcf640 1 --2- 172.21.15.27:0/4113576972 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fac6c072d70 0x7fac6c155e70 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7fac5400c850 tx=0x7fac5400cd10 comp rx=0 tx=0).stop 2024-09-12T03:24:14.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.571+0000 7fac72fcf640 1 -- 172.21.15.27:0/4113576972 shutdown_connections 2024-09-12T03:24:14.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.571+0000 7fac72fcf640 1 --2- 172.21.15.27:0/4113576972 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fac6c15b2b0 0x7fac6c152230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:14.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.571+0000 7fac72fcf640 1 --2- 172.21.15.27:0/4113576972 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fac4c06c9a0 0x7fac4c06ee60 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:14.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.571+0000 7fac72fcf640 1 --2- 172.21.15.27:0/4113576972 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fac6c072d70 0x7fac6c155e70 unknown :-1 s=CLOSED pgs=1765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:14.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.571+0000 7fac72fcf640 1 --2- 172.21.15.27:0/4113576972 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fac6c06e9b0 0x7fac6c15a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:14.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.571+0000 7fac72fcf640 1 -- 172.21.15.27:0/4113576972 >> 172.21.15.27:0/4113576972 conn(0x7fac6c062c60 msgr2=0x7fac6c0cf2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:14.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.571+0000 7fac72fcf640 1 -- 172.21.15.27:0/4113576972 shutdown_connections 2024-09-12T03:24:14.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.571+0000 7fac72fcf640 1 -- 172.21.15.27:0/4113576972 wait complete. 2024-09-12T03:24:14.581 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":216,"stamp":"2024-09-12T03:24:13.144264+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":287480,"kb_used_data":17184,"kb_used_omap":12,"kb_used_meta":270131,"kb_avail":754687240,"statfs":{"total":773094113280,"available":772799733760,"internally_reserved":0,"allocated":17596416,"data_stored":8484372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":276614704},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-51261,"num_objects":-24,"num_object_clones":0,"num_object_copies":-48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-24,"num_whiteouts":0,"num_read":-2,"num_read_kb":-2,"num_write":-65,"num_write_kb":-72,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001456"},"pg_stats":[{"pgid":"1.0","version":"70'128","reported_seq":466,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-12T03:24:10.374782+0000","last_change":"2024-09-12T03:16:21.883446+0000","last_active":"2024-09-12T03:24:10.374782+0000","last_peered":"2024-09-12T03:24:10.374782+0000","last_clean":"2024-09-12T03:24:10.374782+0000","last_became_active":"2024-09-12T03:16:21.882207+0000","last_became_peered":"2024-09-12T03:16:21.882207+0000","last_unstale":"2024-09-12T03:24:10.374782+0000","last_undegraded":"2024-09-12T03:24:10.374782+0000","last_fullsized":"2024-09-12T03:24:10.374782+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:20.316426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:20.316426+0000","last_clean_scrub_stamp":"2024-09-12T03:16:20.316426+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:12:15.424320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542240,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37960,"kb_used_data":2616,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333880,"statfs":{"total":96636764160,"available":96597893120,"internally_reserved":0,"allocated":2678784,"data_stored":1538958,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542242,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37340,"kb_used_data":1992,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334500,"statfs":{"total":96636764160,"available":96598528000,"internally_reserved":0,"allocated":2039808,"data_stored":901594,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542241,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41496,"kb_used_data":1988,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330344,"statfs":{"total":96636764160,"available":96594272256,"internally_reserved":0,"allocated":2035712,"data_stored":901291,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542240,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37408,"kb_used_data":2060,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334432,"statfs":{"total":96636764160,"available":96598458368,"internally_reserved":0,"allocated":2109440,"data_stored":955134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542241,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33168,"kb_used_data":1984,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338672,"statfs":{"total":96636764160,"available":96602800128,"internally_reserved":0,"allocated":2031616,"data_stored":894754,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542241,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33172,"kb_used_data":1984,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338668,"statfs":{"total":96636764160,"available":96602796032,"internally_reserved":0,"allocated":2031616,"data_stored":900709,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542242,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33180,"kb_used_data":1992,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338660,"statfs":{"total":96636764160,"available":96602787840,"internally_reserved":0,"allocated":2039808,"data_stored":900765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542242,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33756,"kb_used_data":2568,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338084,"statfs":{"total":96636764160,"available":96602198016,"internally_reserved":0,"allocated":2629632,"data_stored":1491167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T03:24:14.582 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-12T03:24:14.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.683+0000 7f6ec7647640 1 Processor -- start 2024-09-12T03:24:14.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.684+0000 7f6ec7647640 1 -- start start 2024-09-12T03:24:14.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.684+0000 7f6ec7647640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f6ec00b1ca0 0x7f6ec015f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:14.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.685+0000 7f6ec7647640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6ec015fd70 con 0x7f6ec00cf660 2024-09-12T03:24:14.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.685+0000 7f6ec7647640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6ec006cc40 con 0x7f6ec006f360 2024-09-12T03:24:14.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.685+0000 7f6ec7647640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f6ec00d22f0 con 0x7f6ec00b1ca0 2024-09-12T03:24:14.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.685+0000 7f6ec5be4640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f6ec00cf660 0x7f6ec00d2780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:38886/0 (socket says 172.21.15.27:38886) 2024-09-12T03:24:14.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.685+0000 7f6ec5be4640 1 -- 172.21.15.27:0/667866824 learned_addr learned my addr 172.21.15.27:0/667866824 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:24:14.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.685+0000 7f6ec53e3640 1 --2- 172.21.15.27:0/667866824 >> v2:172.21.15.45:3300/0 conn(0x7f6ec00b1ca0 0x7f6ec015f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:14.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.685+0000 7f6eb3fff640 1 -- 172.21.15.27:0/667866824 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4244616879 0 0) 0x7f6eb8001490 con 0x7f6ec00cf660 2024-09-12T03:24:14.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.685+0000 7f6eb3fff640 1 -- 172.21.15.27:0/667866824 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6eac003520 con 0x7f6ec00cf660 2024-09-12T03:24:14.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.685+0000 7f6eb3fff640 1 -- 172.21.15.27:0/667866824 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2168667264 0 0) 0x7f6eac003520 con 0x7f6ec00cf660 2024-09-12T03:24:14.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.685+0000 7f6ec53e3640 1 -- 172.21.15.27:0/667866824 >> v1:172.21.15.45:6789/0 conn(0x7f6ec006f360 legacy=0x7f6ec00cef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:14.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.685+0000 7f6ec53e3640 1 -- 172.21.15.27:0/667866824 >> v1:172.21.15.27:6789/0 conn(0x7f6ec00cf660 legacy=0x7f6ec00d2780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:14.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.685+0000 7f6ec53e3640 1 -- 172.21.15.27:0/667866824 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ec00d1ce0 con 0x7f6ec00b1ca0 2024-09-12T03:24:14.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.686+0000 7f6ec53e3640 1 --2- 172.21.15.27:0/667866824 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6ec00b1ca0 0x7f6ec015f830 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f6ebc0097a0 tx=0x7f6ebc02a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5be03c0c4d8d25f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:14.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.686+0000 7f6eb3fff640 1 -- 172.21.15.27:0/667866824 <== mon.1 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ebc037070 con 0x7f6ec00b1ca0 2024-09-12T03:24:14.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.686+0000 7f6eb3fff640 1 -- 172.21.15.27:0/667866824 <== mon.1 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ebc02ac50 con 0x7f6ec00b1ca0 2024-09-12T03:24:14.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.686+0000 7f6eb3fff640 1 -- 172.21.15.27:0/667866824 <== mon.1 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eb8001bc0 con 0x7f6ec00b1ca0 2024-09-12T03:24:14.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.687+0000 7f6ec7647640 1 -- 172.21.15.27:0/667866824 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6ec00b1ca0 msgr2=0x7f6ec015f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:14.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.687+0000 7f6ec7647640 1 --2- 172.21.15.27:0/667866824 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6ec00b1ca0 0x7f6ec015f830 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f6ebc0097a0 tx=0x7f6ebc02a1c0 comp rx=0 tx=0).stop 2024-09-12T03:24:14.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.687+0000 7f6ec7647640 1 -- 172.21.15.27:0/667866824 shutdown_connections 2024-09-12T03:24:14.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.687+0000 7f6ec7647640 1 --2- 172.21.15.27:0/667866824 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6ec00b1ca0 0x7f6ec015f830 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:14.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.687+0000 7f6ec7647640 1 -- 172.21.15.27:0/667866824 >> 172.21.15.27:0/667866824 conn(0x7f6ec0073800 msgr2=0x7f6ec009ba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:14.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.687+0000 7f6ec7647640 1 -- 172.21.15.27:0/667866824 shutdown_connections 2024-09-12T03:24:14.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.687+0000 7f6ec7647640 1 -- 172.21.15.27:0/667866824 wait complete. 2024-09-12T03:24:14.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.688+0000 7f6ec7647640 1 Processor -- start 2024-09-12T03:24:14.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.688+0000 7f6ec7647640 1 -- start start 2024-09-12T03:24:14.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.689+0000 7f6ec7647640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ec006f360 0x7f6ec00b0b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:14.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.689+0000 7f6ec7647640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f6ec00b1ca0 0x7f6ec00b3130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:14.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.689+0000 7f6ec7647640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6ec00cf660 0x7f6ec00b3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:14.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.689+0000 7f6ec7647640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f6ec006d1c0 con 0x7f6ec006f360 2024-09-12T03:24:14.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.689+0000 7f6ec7647640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f6ec00d1f80 con 0x7f6ec00cf660 2024-09-12T03:24:14.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.689+0000 7f6ec7647640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f6ec006d340 con 0x7f6ec00b1ca0 2024-09-12T03:24:14.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.689+0000 7f6ec53e3640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ec006f360 0x7f6ec00b0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:14.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.689+0000 7f6ec53e3640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ec006f360 0x7f6ec00b0b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41540/0 (socket says 172.21.15.27:41540) 2024-09-12T03:24:14.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.689+0000 7f6ec53e3640 1 -- 172.21.15.27:0/3844448581 learned_addr learned my addr 172.21.15.27:0/3844448581 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:24:14.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.689+0000 7f6ec53e3640 1 -- 172.21.15.27:0/3844448581 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f6ec00b1ca0 msgr2=0x7f6ec00b3130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:14.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.689+0000 7f6ec53e3640 1 --2- 172.21.15.27:0/3844448581 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f6ec00b1ca0 0x7f6ec00b3130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:14.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.689+0000 7f6ec53e3640 1 -- 172.21.15.27:0/3844448581 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6ec00cf660 msgr2=0x7f6ec00b3670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:14.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.689+0000 7f6ec53e3640 1 --2- 172.21.15.27:0/3844448581 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6ec00cf660 0x7f6ec00b3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:14.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.689+0000 7f6ec53e3640 1 -- 172.21.15.27:0/3844448581 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ec00b6f10 con 0x7f6ec006f360 2024-09-12T03:24:14.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.690+0000 7f6ec53e3640 1 --2- 172.21.15.27:0/3844448581 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ec006f360 0x7f6ec00b0b20 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7f6ebc035040 tx=0x7f6ebc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:14.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.690+0000 7f6eb1ffb640 1 -- 172.21.15.27:0/3844448581 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ebc042020 con 0x7f6ec006f360 2024-09-12T03:24:14.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.690+0000 7f6eb1ffb640 1 -- 172.21.15.27:0/3844448581 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ebc03d470 con 0x7f6ec006f360 2024-09-12T03:24:14.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.690+0000 7f6eb1ffb640 1 -- 172.21.15.27:0/3844448581 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ebc037040 con 0x7f6ec006f360 2024-09-12T03:24:14.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.690+0000 7f6ec7647640 1 -- 172.21.15.27:0/3844448581 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ec00b7190 con 0x7f6ec006f360 2024-09-12T03:24:14.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.690+0000 7f6ec7647640 1 -- 172.21.15.27:0/3844448581 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ec00b7750 con 0x7f6ec006f360 2024-09-12T03:24:14.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.691+0000 7f6ec7647640 1 -- 172.21.15.27:0/3844448581 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ec00cfc40 con 0x7f6ec006f360 2024-09-12T03:24:14.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.692+0000 7f6eb1ffb640 1 -- 172.21.15.27:0/3844448581 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 26) ==== 91518+0+0 (secure 0 0 0) 0x7f6ebc045020 con 0x7f6ec006f360 2024-09-12T03:24:14.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.692+0000 7f6eb1ffb640 1 --2- 172.21.15.27:0/3844448581 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f6ea406c970 0x7f6ea406ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:14.693 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.692+0000 7f6ec4be2640 1 --2- 172.21.15.27:0/3844448581 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f6ea406c970 0x7f6ea406ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:14.693 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.692+0000 7f6eb1ffb640 1 -- 172.21.15.27:0/3844448581 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(163..163 src has 1..163) ==== 4999+0+0 (secure 0 0 0) 0x7f6ebc0a9f20 con 0x7f6ec006f360 2024-09-12T03:24:14.693 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.693+0000 7f6ec4be2640 1 --2- 172.21.15.27:0/3844448581 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f6ea406c970 0x7f6ea406ee30 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f6eb4000f60 tx=0x7f6eb40079b0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:14.696 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.697+0000 7f6eb1ffb640 1 -- 172.21.15.27:0/3844448581 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f6ebc04c030 con 0x7f6ec006f360 2024-09-12T03:24:14.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.869+0000 7f6ec7647640 1 -- 172.21.15.27:0/3844448581 --> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6ec0070110 con 0x7f6ea406c970 2024-09-12T03:24:14.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.870+0000 7f6eb1ffb640 1 -- 172.21.15.27:0/3844448581 <== mgr.4113 v2:172.21.15.27:6836/1226920526 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13498 (secure 0 0 0) 0x7f6ec0070110 con 0x7f6ea406c970 2024-09-12T03:24:14.870 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:24:14.871 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2024-09-12T03:24:14.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.872+0000 7f6ec7647640 1 -- 172.21.15.27:0/3844448581 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f6ea406c970 msgr2=0x7f6ea406ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:14.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.873+0000 7f6ec7647640 1 --2- 172.21.15.27:0/3844448581 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f6ea406c970 0x7f6ea406ee30 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f6eb4000f60 tx=0x7f6eb40079b0 comp rx=0 tx=0).stop 2024-09-12T03:24:14.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.873+0000 7f6ec7647640 1 -- 172.21.15.27:0/3844448581 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ec006f360 msgr2=0x7f6ec00b0b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:14.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.873+0000 7f6ec7647640 1 --2- 172.21.15.27:0/3844448581 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ec006f360 0x7f6ec00b0b20 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7f6ebc035040 tx=0x7f6ebc0046c0 comp rx=0 tx=0).stop 2024-09-12T03:24:14.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.873+0000 7f6ec7647640 1 -- 172.21.15.27:0/3844448581 shutdown_connections 2024-09-12T03:24:14.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.873+0000 7f6ec7647640 1 --2- 172.21.15.27:0/3844448581 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6ec00cf660 0x7f6ec00b3670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:14.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.873+0000 7f6ec7647640 1 --2- 172.21.15.27:0/3844448581 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f6ec00b1ca0 0x7f6ec00b3130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:14.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.873+0000 7f6ec7647640 1 --2- 172.21.15.27:0/3844448581 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f6ea406c970 0x7f6ea406ee30 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:14.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.873+0000 7f6ec7647640 1 --2- 172.21.15.27:0/3844448581 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ec006f360 0x7f6ec00b0b20 unknown :-1 s=CLOSED pgs=1767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:14.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.873+0000 7f6ec7647640 1 -- 172.21.15.27:0/3844448581 >> 172.21.15.27:0/3844448581 conn(0x7f6ec0073800 msgr2=0x7f6ec0161290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:14.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.873+0000 7f6ec7647640 1 -- 172.21.15.27:0/3844448581 shutdown_connections 2024-09-12T03:24:14.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.873+0000 7f6ec7647640 1 -- 172.21.15.27:0/3844448581 wait complete. 2024-09-12T03:24:14.883 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":216,"stamp":"2024-09-12T03:24:13.144264+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":287480,"kb_used_data":17184,"kb_used_omap":12,"kb_used_meta":270131,"kb_avail":754687240,"statfs":{"total":773094113280,"available":772799733760,"internally_reserved":0,"allocated":17596416,"data_stored":8484372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":276614704},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-51261,"num_objects":-24,"num_object_clones":0,"num_object_copies":-48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-24,"num_whiteouts":0,"num_read":-2,"num_read_kb":-2,"num_write":-65,"num_write_kb":-72,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001456"},"pg_stats":[{"pgid":"1.0","version":"70'128","reported_seq":466,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-12T03:24:10.374782+0000","last_change":"2024-09-12T03:16:21.883446+0000","last_active":"2024-09-12T03:24:10.374782+0000","last_peered":"2024-09-12T03:24:10.374782+0000","last_clean":"2024-09-12T03:24:10.374782+0000","last_became_active":"2024-09-12T03:16:21.882207+0000","last_became_peered":"2024-09-12T03:16:21.882207+0000","last_unstale":"2024-09-12T03:24:10.374782+0000","last_undegraded":"2024-09-12T03:24:10.374782+0000","last_fullsized":"2024-09-12T03:24:10.374782+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:20.316426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:20.316426+0000","last_clean_scrub_stamp":"2024-09-12T03:16:20.316426+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:12:15.424320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542240,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37960,"kb_used_data":2616,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333880,"statfs":{"total":96636764160,"available":96597893120,"internally_reserved":0,"allocated":2678784,"data_stored":1538958,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542242,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37340,"kb_used_data":1992,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334500,"statfs":{"total":96636764160,"available":96598528000,"internally_reserved":0,"allocated":2039808,"data_stored":901594,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542241,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41496,"kb_used_data":1988,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330344,"statfs":{"total":96636764160,"available":96594272256,"internally_reserved":0,"allocated":2035712,"data_stored":901291,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542240,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37408,"kb_used_data":2060,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334432,"statfs":{"total":96636764160,"available":96598458368,"internally_reserved":0,"allocated":2109440,"data_stored":955134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542241,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33168,"kb_used_data":1984,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338672,"statfs":{"total":96636764160,"available":96602800128,"internally_reserved":0,"allocated":2031616,"data_stored":894754,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542241,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33172,"kb_used_data":1984,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338668,"statfs":{"total":96636764160,"available":96602796032,"internally_reserved":0,"allocated":2031616,"data_stored":900709,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542242,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33180,"kb_used_data":1992,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338660,"statfs":{"total":96636764160,"available":96602787840,"internally_reserved":0,"allocated":2039808,"data_stored":900765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542242,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33756,"kb_used_data":2568,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338084,"statfs":{"total":96636764160,"available":96602198016,"internally_reserved":0,"allocated":2629632,"data_stored":1491167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T03:24:14.884 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-12T03:24:14.884 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-12T03:24:14.986 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.986+0000 7f6340911640 1 Processor -- start 2024-09-12T03:24:14.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.986+0000 7f6340911640 1 -- start start 2024-09-12T03:24:14.988 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.987+0000 7f6340911640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f633c15eb90 0x7f633c0d2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:14.988 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.987+0000 7f6340911640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f633c0d5200 0x7f633c0d1010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:14.988 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.987+0000 7f6340911640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f633c0d1550 0x7f633c0cf200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:14.988 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.987+0000 7f6340911640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f633c0b7d20 con 0x7f633c0d1550 2024-09-12T03:24:14.988 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.987+0000 7f6340911640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f633c0d42e0 con 0x7f633c0d5200 2024-09-12T03:24:14.988 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.987+0000 7f6340911640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f633c0d19e0 con 0x7f633c15eb90 2024-09-12T03:24:14.989 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.988+0000 7f633ad9d640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f633c0d1550 0x7f633c0cf200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:14.989 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.988+0000 7f633ad9d640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f633c0d1550 0x7f633c0cf200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41550/0 (socket says 172.21.15.27:41550) 2024-09-12T03:24:14.989 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.988+0000 7f633ad9d640 1 -- 172.21.15.27:0/1870610920 learned_addr learned my addr 172.21.15.27:0/1870610920 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:24:14.989 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.988+0000 7f633a59c640 1 --2- 172.21.15.27:0/1870610920 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f633c15eb90 0x7f633c0d2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:14.989 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.988+0000 7f6339d9b640 1 --2- 172.21.15.27:0/1870610920 >> v2:172.21.15.45:3300/0 conn(0x7f633c0d5200 0x7f633c0d1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:14.989 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.988+0000 7f633ad9d640 1 -- 172.21.15.27:0/1870610920 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f633c15eb90 msgr2=0x7f633c0d2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:24:14.989 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.988+0000 7f633ad9d640 1 --2- 172.21.15.27:0/1870610920 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f633c15eb90 0x7f633c0d2df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:14.989 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.988+0000 7f633ad9d640 1 -- 172.21.15.27:0/1870610920 >> v2:172.21.15.45:3300/0 conn(0x7f633c0d5200 msgr2=0x7f633c0d1010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:24:14.989 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.988+0000 7f633ad9d640 1 --2- 172.21.15.27:0/1870610920 >> v2:172.21.15.45:3300/0 conn(0x7f633c0d5200 0x7f633c0d1010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:14.989 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.988+0000 7f633ad9d640 1 -- 172.21.15.27:0/1870610920 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f633c0d3760 con 0x7f633c0d1550 2024-09-12T03:24:14.989 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.988+0000 7f633a59c640 1 --2- 172.21.15.27:0/1870610920 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f633c15eb90 0x7f633c0d2df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:24:14.989 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.988+0000 7f6339d9b640 1 --2- 172.21.15.27:0/1870610920 >> v2:172.21.15.45:3300/0 conn(0x7f633c0d5200 0x7f633c0d1010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:24:14.991 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.989+0000 7f633ad9d640 1 --2- 172.21.15.27:0/1870610920 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f633c0d1550 0x7f633c0cf200 secure :-1 s=READY pgs=1768 cs=0 l=1 rev1=1 crypto rx=0x7f633c063620 tx=0x7f633402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50088092fc41bc2f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:14.991 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.989+0000 7f633959a640 1 -- 172.21.15.27:0/1870610920 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6334037070 con 0x7f633c0d1550 2024-09-12T03:24:14.991 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.989+0000 7f633959a640 1 -- 172.21.15.27:0/1870610920 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6334032440 con 0x7f633c0d1550 2024-09-12T03:24:14.991 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.989+0000 7f633959a640 1 -- 172.21.15.27:0/1870610920 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6334032810 con 0x7f633c0d1550 2024-09-12T03:24:14.991 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.989+0000 7f6340911640 1 -- 172.21.15.27:0/1870610920 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f633c0d1550 msgr2=0x7f633c0cf200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:14.991 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.989+0000 7f6340911640 1 --2- 172.21.15.27:0/1870610920 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f633c0d1550 0x7f633c0cf200 secure :-1 s=READY pgs=1768 cs=0 l=1 rev1=1 crypto rx=0x7f633c063620 tx=0x7f633402a1e0 comp rx=0 tx=0).stop 2024-09-12T03:24:14.991 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.990+0000 7f6340911640 1 -- 172.21.15.27:0/1870610920 shutdown_connections 2024-09-12T03:24:14.991 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.990+0000 7f6340911640 1 --2- 172.21.15.27:0/1870610920 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f633c0d1550 0x7f633c0cf200 unknown :-1 s=CLOSED pgs=1768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:14.991 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.990+0000 7f6340911640 1 --2- 172.21.15.27:0/1870610920 >> v2:172.21.15.45:3300/0 conn(0x7f633c0d5200 0x7f633c0d1010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:14.992 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.990+0000 7f6340911640 1 --2- 172.21.15.27:0/1870610920 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f633c15eb90 0x7f633c0d2df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:14.992 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.990+0000 7f6340911640 1 -- 172.21.15.27:0/1870610920 >> 172.21.15.27:0/1870610920 conn(0x7f633c06dce0 msgr2=0x7f633c06e0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:14.992 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.990+0000 7f6340911640 1 -- 172.21.15.27:0/1870610920 shutdown_connections 2024-09-12T03:24:14.992 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.990+0000 7f6340911640 1 -- 172.21.15.27:0/1870610920 wait complete. 2024-09-12T03:24:14.992 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.991+0000 7f6340911640 1 Processor -- start 2024-09-12T03:24:14.992 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.991+0000 7f6340911640 1 -- start start 2024-09-12T03:24:14.994 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.992+0000 7f6340911640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f633c0d1550 0x7f633c157de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:14.994 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.992+0000 7f6340911640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f633c0d5200 0x7f633c1f8300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:14.994 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.992+0000 7f6340911640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f633c15eb90 0x7f633c158490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:14.995 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.992+0000 7f6340911640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f633c0d4000 con 0x7f633c15eb90 2024-09-12T03:24:14.995 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.992+0000 7f6340911640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f633c0d3d20 con 0x7f633c0d5200 2024-09-12T03:24:14.995 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.992+0000 7f6340911640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f633c0d3330 con 0x7f633c0d1550 2024-09-12T03:24:14.995 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.992+0000 7f633ad9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f633c15eb90 0x7f633c158490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:14.995 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.992+0000 7f633ad9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f633c15eb90 0x7f633c158490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41562/0 (socket says 172.21.15.27:41562) 2024-09-12T03:24:14.995 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.992+0000 7f633ad9d640 1 -- 172.21.15.27:0/1329099696 learned_addr learned my addr 172.21.15.27:0/1329099696 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:24:14.995 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.992+0000 7f633a59c640 1 --2- 172.21.15.27:0/1329099696 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f633c0d1550 0x7f633c157de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:14.995 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.992+0000 7f6339d9b640 1 --2- 172.21.15.27:0/1329099696 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f633c0d5200 0x7f633c1f8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:14.995 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.992+0000 7f633ad9d640 1 -- 172.21.15.27:0/1329099696 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f633c0d1550 msgr2=0x7f633c157de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:14.995 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.992+0000 7f633ad9d640 1 --2- 172.21.15.27:0/1329099696 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f633c0d1550 0x7f633c157de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:14.995 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.992+0000 7f633ad9d640 1 -- 172.21.15.27:0/1329099696 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f633c0d5200 msgr2=0x7f633c1f8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:14.995 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.992+0000 7f633ad9d640 1 --2- 172.21.15.27:0/1329099696 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f633c0d5200 0x7f633c1f8300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:14.995 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.992+0000 7f633ad9d640 1 -- 172.21.15.27:0/1329099696 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f633c0cb2b0 con 0x7f633c15eb90 2024-09-12T03:24:14.995 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.992+0000 7f633a59c640 1 --2- 172.21.15.27:0/1329099696 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f633c0d1550 0x7f633c157de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:24:14.996 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.992+0000 7f633ad9d640 1 --2- 172.21.15.27:0/1329099696 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f633c15eb90 0x7f633c158490 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7f63340090b0 tx=0x7f63340046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:14.996 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.993+0000 7f6332ffd640 1 -- 172.21.15.27:0/1329099696 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6334041020 con 0x7f633c15eb90 2024-09-12T03:24:14.996 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.993+0000 7f6332ffd640 1 -- 172.21.15.27:0/1329099696 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6334030760 con 0x7f633c15eb90 2024-09-12T03:24:14.996 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.993+0000 7f6340911640 1 -- 172.21.15.27:0/1329099696 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f633c0caa10 con 0x7f633c15eb90 2024-09-12T03:24:14.996 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.993+0000 7f6332ffd640 1 -- 172.21.15.27:0/1329099696 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6334037040 con 0x7f633c15eb90 2024-09-12T03:24:14.996 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.993+0000 7f6340911640 1 -- 172.21.15.27:0/1329099696 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f633c0cafd0 con 0x7f633c15eb90 2024-09-12T03:24:14.996 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.994+0000 7f6340911640 1 -- 172.21.15.27:0/1329099696 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f633c15ef70 con 0x7f633c15eb90 2024-09-12T03:24:14.996 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.994+0000 7f6332ffd640 1 -- 172.21.15.27:0/1329099696 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 26) ==== 91518+0+0 (secure 0 0 0) 0x7f6334044020 con 0x7f633c15eb90 2024-09-12T03:24:14.996 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.995+0000 7f6332ffd640 1 --2- 172.21.15.27:0/1329099696 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f631c06c970 0x7f631c06ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:14.996 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.995+0000 7f633a59c640 1 --2- 172.21.15.27:0/1329099696 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f631c06c970 0x7f631c06ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:14.996 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.995+0000 7f6332ffd640 1 -- 172.21.15.27:0/1329099696 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(163..163 src has 1..163) ==== 4999+0+0 (secure 0 0 0) 0x7f63340aa0d0 con 0x7f633c15eb90 2024-09-12T03:24:14.997 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.995+0000 7f633a59c640 1 --2- 172.21.15.27:0/1329099696 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f631c06c970 0x7f631c06ee30 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f632c007ec0 tx=0x7f632c007410 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:14.998 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:14.999+0000 7f6332ffd640 1 -- 172.21.15.27:0/1329099696 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f633404b030 con 0x7f633c15eb90 2024-09-12T03:24:15.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:15.171+0000 7f6340911640 1 -- 172.21.15.27:0/1329099696 --> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f633c0ba270 con 0x7f631c06c970 2024-09-12T03:24:15.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:15.172+0000 7f6332ffd640 1 -- 172.21.15.27:0/1329099696 <== mgr.4113 v2:172.21.15.27:6836/1226920526 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13498 (secure 0 0 0) 0x7f633c0ba270 con 0x7f631c06c970 2024-09-12T03:24:15.172 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:24:15.172 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2024-09-12T03:24:15.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:15.173+0000 7f6340911640 1 -- 172.21.15.27:0/1329099696 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f631c06c970 msgr2=0x7f631c06ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:15.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:15.173+0000 7f6340911640 1 --2- 172.21.15.27:0/1329099696 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f631c06c970 0x7f631c06ee30 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f632c007ec0 tx=0x7f632c007410 comp rx=0 tx=0).stop 2024-09-12T03:24:15.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:15.173+0000 7f6340911640 1 -- 172.21.15.27:0/1329099696 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f633c15eb90 msgr2=0x7f633c158490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:15.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:15.173+0000 7f6340911640 1 --2- 172.21.15.27:0/1329099696 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f633c15eb90 0x7f633c158490 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7f63340090b0 tx=0x7f63340046c0 comp rx=0 tx=0).stop 2024-09-12T03:24:15.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:15.173+0000 7f6340911640 1 -- 172.21.15.27:0/1329099696 shutdown_connections 2024-09-12T03:24:15.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:15.173+0000 7f6340911640 1 --2- 172.21.15.27:0/1329099696 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f631c06c970 0x7f631c06ee30 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:15.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:15.173+0000 7f6340911640 1 --2- 172.21.15.27:0/1329099696 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f633c15eb90 0x7f633c158490 unknown :-1 s=CLOSED pgs=1769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:15.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:15.173+0000 7f6340911640 1 --2- 172.21.15.27:0/1329099696 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f633c0d5200 0x7f633c1f8300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:15.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:15.173+0000 7f6340911640 1 --2- 172.21.15.27:0/1329099696 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f633c0d1550 0x7f633c157de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:15.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:15.174+0000 7f6340911640 1 -- 172.21.15.27:0/1329099696 >> 172.21.15.27:0/1329099696 conn(0x7f633c06dce0 msgr2=0x7f633c0cfbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:15.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:15.174+0000 7f6340911640 1 -- 172.21.15.27:0/1329099696 shutdown_connections 2024-09-12T03:24:15.175 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:15.174+0000 7f6340911640 1 -- 172.21.15.27:0/1329099696 wait complete. 2024-09-12T03:24:15.184 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":217,"stamp":"2024-09-12T03:24:15.144532+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":287480,"kb_used_data":17184,"kb_used_omap":12,"kb_used_meta":270131,"kb_avail":754687240,"statfs":{"total":773094113280,"available":772799733760,"internally_reserved":0,"allocated":17596416,"data_stored":8484372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":276614704},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-51261,"num_objects":-24,"num_object_clones":0,"num_object_copies":-48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-24,"num_whiteouts":0,"num_read":-2,"num_read_kb":-2,"num_write":-65,"num_write_kb":-72,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001251"},"pg_stats":[{"pgid":"1.0","version":"70'128","reported_seq":466,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-12T03:24:10.374782+0000","last_change":"2024-09-12T03:16:21.883446+0000","last_active":"2024-09-12T03:24:10.374782+0000","last_peered":"2024-09-12T03:24:10.374782+0000","last_clean":"2024-09-12T03:24:10.374782+0000","last_became_active":"2024-09-12T03:16:21.882207+0000","last_became_peered":"2024-09-12T03:16:21.882207+0000","last_unstale":"2024-09-12T03:24:10.374782+0000","last_undegraded":"2024-09-12T03:24:10.374782+0000","last_fullsized":"2024-09-12T03:24:10.374782+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:20.316426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:20.316426+0000","last_clean_scrub_stamp":"2024-09-12T03:16:20.316426+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:12:15.424320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542240,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37960,"kb_used_data":2616,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333880,"statfs":{"total":96636764160,"available":96597893120,"internally_reserved":0,"allocated":2678784,"data_stored":1538958,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542242,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37340,"kb_used_data":1992,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334500,"statfs":{"total":96636764160,"available":96598528000,"internally_reserved":0,"allocated":2039808,"data_stored":901594,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542241,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41496,"kb_used_data":1988,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330344,"statfs":{"total":96636764160,"available":96594272256,"internally_reserved":0,"allocated":2035712,"data_stored":901291,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542240,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37408,"kb_used_data":2060,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334432,"statfs":{"total":96636764160,"available":96598458368,"internally_reserved":0,"allocated":2109440,"data_stored":955134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542241,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33168,"kb_used_data":1984,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338672,"statfs":{"total":96636764160,"available":96602800128,"internally_reserved":0,"allocated":2031616,"data_stored":894754,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542241,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33172,"kb_used_data":1984,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338668,"statfs":{"total":96636764160,"available":96602796032,"internally_reserved":0,"allocated":2031616,"data_stored":900709,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542242,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33180,"kb_used_data":1992,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338660,"statfs":{"total":96636764160,"available":96602787840,"internally_reserved":0,"allocated":2039808,"data_stored":900765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542242,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33756,"kb_used_data":2568,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338084,"statfs":{"total":96636764160,"available":96602198016,"internally_reserved":0,"allocated":2629632,"data_stored":1491167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T03:24:15.185 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-12T03:24:15.287 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:15.287+0000 7f7447fbd640 1 Processor -- start 2024-09-12T03:24:15.287 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:15.287+0000 7f7447fbd640 1 -- start start 2024-09-12T03:24:15.290 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:15.288+0000 7f7447fbd640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f744015e780 0x7f74400d3e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:15.291 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:15.288+0000 7f7447fbd640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f74400d43b0 0x7f74400d1280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:15.291 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:15.288+0000 7f7447fbd640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f74400b6690 con 0x7f744015eb60 2024-09-12T03:24:15.291 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:15.288+0000 7f7447fbd640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f74400b7950 con 0x7f744015e780 2024-09-12T03:24:15.291 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:15.288+0000 7f7447fbd640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f74400b7210 con 0x7f74400d43b0 2024-09-12T03:24:15.291 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:15.288+0000 7f7445d59640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f744015e780 0x7f74400d3e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:15.291 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:15.288+0000 7f7445d59640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f744015e780 0x7f74400d3e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41570/0 (socket says 172.21.15.27:41570) 2024-09-12T03:24:15.291 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:15.288+0000 7f7445d59640 1 -- 172.21.15.27:0/2907229407 learned_addr learned my addr 172.21.15.27:0/2907229407 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:24:15.291 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:15.288+0000 7f7445558640 1 --2- 172.21.15.27:0/2907229407 >> v2:172.21.15.45:3300/0 conn(0x7f74400d43b0 0x7f74400d1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:15.291 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:15.288+0000 7f7445d59640 1 -- 172.21.15.27:0/2907229407 >> v2:172.21.15.45:3300/0 conn(0x7f74400d43b0 msgr2=0x7f74400d1280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:24:15.291 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:15.288+0000 7f7445d59640 1 --2- 172.21.15.27:0/2907229407 >> v2:172.21.15.45:3300/0 conn(0x7f74400d43b0 0x7f74400d1280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:15.291 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:15.288+0000 7f7445d59640 1 -- 172.21.15.27:0/2907229407 >> v1:172.21.15.45:6789/0 conn(0x7f744015eb60 legacy=0x7f74400b3390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:15.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:15.288+0000 7f7445d59640 1 -- 172.21.15.27:0/2907229407 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74400b6970 con 0x7f744015e780 2024-09-12T03:24:15.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:15.289+0000 7f7445d59640 1 --2- 172.21.15.27:0/2907229407 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f744015e780 0x7f74400d3e70 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 crypto rx=0x7f74400ba380 tx=0x7f743c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42f353c4de350f92 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:15.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:15.289+0000 7f7444d57640 1 -- 172.21.15.27:0/2907229407 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f743c037070 con 0x7f744015e780 2024-09-12T03:24:15.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:15.289+0000 7f7444d57640 1 -- 172.21.15.27:0/2907229407 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f743c032440 con 0x7f744015e780 2024-09-12T03:24:15.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:15.289+0000 7f7444d57640 1 -- 172.21.15.27:0/2907229407 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f743c032810 con 0x7f744015e780 2024-09-12T03:24:15.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:15.290+0000 7f7447fbd640 1 -- 172.21.15.27:0/2907229407 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f744015e780 msgr2=0x7f74400d3e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:15.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:15.290+0000 7f7447fbd640 1 --2- 172.21.15.27:0/2907229407 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f744015e780 0x7f74400d3e70 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 crypto rx=0x7f74400ba380 tx=0x7f743c02a1e0 comp rx=0 tx=0).stop 2024-09-12T03:24:15.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:15.290+0000 7f7447fbd640 1 -- 172.21.15.27:0/2907229407 shutdown_connections 2024-09-12T03:24:15.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:15.290+0000 7f7447fbd640 1 --2- 172.21.15.27:0/2907229407 >> v2:172.21.15.45:3300/0 conn(0x7f74400d43b0 0x7f74400d1280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:15.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:15.290+0000 7f7447fbd640 1 --2- 172.21.15.27:0/2907229407 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f744015e780 0x7f74400d3e70 unknown :-1 s=CLOSED pgs=1770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:15.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:15.290+0000 7f7447fbd640 1 -- 172.21.15.27:0/2907229407 >> 172.21.15.27:0/2907229407 conn(0x7f744006da60 msgr2=0x7f744006de70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:15.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:15.290+0000 7f7447fbd640 1 -- 172.21.15.27:0/2907229407 shutdown_connections 2024-09-12T03:24:15.293 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:15.290+0000 7f7447fbd640 1 -- 172.21.15.27:0/2907229407 wait complete. 2024-09-12T03:24:15.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:15.291+0000 7f7447fbd640 1 Processor -- start 2024-09-12T03:24:15.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:15.291+0000 7f7447fbd640 1 -- start start 2024-09-12T03:24:15.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:15.292+0000 7f7447fbd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f74400d43b0 0x7f74400c9f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:15.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:15.292+0000 7f7447fbd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f744015e780 0x7f744014cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:15.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:15.292+0000 7f7447fbd640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f744015eb60 0x7f744015f910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:15.296 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:15.292+0000 7f7447fbd640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f74400b6340 con 0x7f74400d43b0 2024-09-12T03:24:15.296 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:15.292+0000 7f7447fbd640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f74400b6f30 con 0x7f744015e780 2024-09-12T03:24:15.296 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:15.292+0000 7f7447fbd640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f74400d17c0 con 0x7f744015eb60 2024-09-12T03:24:15.296 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:15.292+0000 7f7445d59640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f74400d43b0 0x7f74400c9f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:15.296 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:15.292+0000 7f7445d59640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f74400d43b0 0x7f74400c9f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41586/0 (socket says 172.21.15.27:41586) 2024-09-12T03:24:15.296 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:15.292+0000 7f7445d59640 1 -- 172.21.15.27:0/1034814682 learned_addr learned my addr 172.21.15.27:0/1034814682 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:24:15.296 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:15.292+0000 7f7445558640 1 --2- 172.21.15.27:0/1034814682 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f744015e780 0x7f744014cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:15.296 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:15.292+0000 7f744655a640 1 --2- 172.21.15.27:0/1034814682 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f744015eb60 0x7f744015f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:15.296 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:15.292+0000 7f7445d59640 1 -- 172.21.15.27:0/1034814682 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f744015eb60 msgr2=0x7f744015f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:15.296 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:15.292+0000 7f7445d59640 1 --2- 172.21.15.27:0/1034814682 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f744015eb60 0x7f744015f910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:15.296 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:15.292+0000 7f7445d59640 1 -- 172.21.15.27:0/1034814682 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f744015e780 msgr2=0x7f744014cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:15.296 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:15.292+0000 7f7445d59640 1 --2- 172.21.15.27:0/1034814682 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f744015e780 0x7f744014cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:15.296 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:15.292+0000 7f7445d59640 1 -- 172.21.15.27:0/1034814682 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74400c8e80 con 0x7f74400d43b0 2024-09-12T03:24:15.297 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:15.292+0000 7f744655a640 1 --2- 172.21.15.27:0/1034814682 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f744015eb60 0x7f744015f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:24:15.297 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:15.293+0000 7f7445d59640 1 --2- 172.21.15.27:0/1034814682 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f74400d43b0 0x7f74400c9f20 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 crypto rx=0x7f743c02a220 tx=0x7f743c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:15.297 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:15.293+0000 7f74367fc640 1 -- 172.21.15.27:0/1034814682 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f743c041020 con 0x7f74400d43b0 2024-09-12T03:24:15.297 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:15.293+0000 7f74367fc640 1 -- 172.21.15.27:0/1034814682 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f743c0306c0 con 0x7f74400d43b0 2024-09-12T03:24:15.297 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:15.293+0000 7f7447fbd640 1 -- 172.21.15.27:0/1034814682 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74400c9160 con 0x7f74400d43b0 2024-09-12T03:24:15.297 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:15.293+0000 7f74367fc640 1 -- 172.21.15.27:0/1034814682 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f743c037040 con 0x7f74400d43b0 2024-09-12T03:24:15.297 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:15.293+0000 7f7447fbd640 1 -- 172.21.15.27:0/1034814682 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74400c9720 con 0x7f74400d43b0 2024-09-12T03:24:15.297 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:15.294+0000 7f7447fbd640 1 -- 172.21.15.27:0/1034814682 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74400693d0 con 0x7f74400d43b0 2024-09-12T03:24:15.297 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:15.294+0000 7f74367fc640 1 -- 172.21.15.27:0/1034814682 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 26) ==== 91518+0+0 (secure 0 0 0) 0x7f743c044020 con 0x7f74400d43b0 2024-09-12T03:24:15.297 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:15.295+0000 7f74367fc640 1 --2- 172.21.15.27:0/1034814682 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f742406c970 0x7f742406ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:15.297 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:15.295+0000 7f74367fc640 1 -- 172.21.15.27:0/1034814682 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(163..163 src has 1..163) ==== 4999+0+0 (secure 0 0 0) 0x7f743c0a9f60 con 0x7f74400d43b0 2024-09-12T03:24:15.297 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:15.295+0000 7f7445558640 1 --2- 172.21.15.27:0/1034814682 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f742406c970 0x7f742406ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:15.297 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:15.295+0000 7f7445558640 1 --2- 172.21.15.27:0/1034814682 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f742406c970 0x7f742406ee30 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f7440160520 tx=0x7f7430008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:15.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:15.299+0000 7f74367fc640 1 -- 172.21.15.27:0/1034814682 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f743c04b030 con 0x7f74400d43b0 2024-09-12T03:24:15.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:15.475+0000 7f7447fbd640 1 -- 172.21.15.27:0/1034814682 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f744015f150 con 0x7f74400d43b0 2024-09-12T03:24:15.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:15.476+0000 7f74367fc640 1 -- 172.21.15.27:0/1034814682 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v163) ==== 74+0+13083 (secure 0 0 0) 0x7f743c07a3c0 con 0x7f74400d43b0 2024-09-12T03:24:15.476 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:24:15.476 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":163,"fsid":"d2df504e-2647-4385-a5db-9fdf69f487a0","created":"2024-09-12T03:16:07.778937+0000","modified":"2024-09-12T03:24:12.326258+0000","last_up_change":"2024-09-12T03:16:19.314778+0000","last_in_change":"2024-09-12T03:16:09.893086+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T03:16:20.097646+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1851116c-1fb3-4dc9-9772-f90e2ef4c355","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":158,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6817","nonce":3002321488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6819","nonce":3002321488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6823","nonce":3002321488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":3002321488},{"type":"v1","addr":"172.21.15.27:6821","nonce":3002321488}]},"public_addr":"172.21.15.27:6817/3002321488","cluster_addr":"172.21.15.27:6819/3002321488","heartbeat_back_addr":"172.21.15.27:6823/3002321488","heartbeat_front_addr":"172.21.15.27:6821/3002321488","state":["exists","up"]},{"osd":1,"uuid":"f38ae17d-73eb-4204-b99d-975fa806fb87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":158,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6809","nonce":1613102886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6811","nonce":1613102886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6815","nonce":1613102886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":1613102886},{"type":"v1","addr":"172.21.15.27:6813","nonce":1613102886}]},"public_addr":"172.21.15.27:6809/1613102886","cluster_addr":"172.21.15.27:6811/1613102886","heartbeat_back_addr":"172.21.15.27:6815/1613102886","heartbeat_front_addr":"172.21.15.27:6813/1613102886","state":["exists","up"]},{"osd":2,"uuid":"fb12dcc8-bdd0-42b8-b46d-8b10106224bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":158,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6825","nonce":3534723355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6827","nonce":3534723355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6831","nonce":3534723355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":3534723355},{"type":"v1","addr":"172.21.15.27:6829","nonce":3534723355}]},"public_addr":"172.21.15.27:6825/3534723355","cluster_addr":"172.21.15.27:6827/3534723355","heartbeat_back_addr":"172.21.15.27:6831/3534723355","heartbeat_front_addr":"172.21.15.27:6829/3534723355","state":["exists","up"]},{"osd":3,"uuid":"d74d3f49-9c24-4b9f-98e2-3ecd7b5ef3b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":158,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6801","nonce":80661598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6803","nonce":80661598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6807","nonce":80661598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":80661598},{"type":"v1","addr":"172.21.15.27:6805","nonce":80661598}]},"public_addr":"172.21.15.27:6801/80661598","cluster_addr":"172.21.15.27:6803/80661598","heartbeat_back_addr":"172.21.15.27:6807/80661598","heartbeat_front_addr":"172.21.15.27:6805/80661598","state":["exists","up"]},{"osd":4,"uuid":"2bf123f2-1b0d-41c4-aa3b-805bab267e59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":158,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6817","nonce":3943189827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6819","nonce":3943189827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6823","nonce":3943189827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":3943189827},{"type":"v1","addr":"172.21.15.45:6821","nonce":3943189827}]},"public_addr":"172.21.15.45:6817/3943189827","cluster_addr":"172.21.15.45:6819/3943189827","heartbeat_back_addr":"172.21.15.45:6823/3943189827","heartbeat_front_addr":"172.21.15.45:6821/3943189827","state":["exists","up"]},{"osd":5,"uuid":"c49ba5da-91c6-4a86-a97b-4aba89c253ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":158,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6809","nonce":3620760330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6811","nonce":3620760330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6815","nonce":3620760330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3620760330},{"type":"v1","addr":"172.21.15.45:6813","nonce":3620760330}]},"public_addr":"172.21.15.45:6809/3620760330","cluster_addr":"172.21.15.45:6811/3620760330","heartbeat_back_addr":"172.21.15.45:6815/3620760330","heartbeat_front_addr":"172.21.15.45:6813/3620760330","state":["exists","up"]},{"osd":6,"uuid":"b0123475-89f7-4a34-9e33-d797b99dd9dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":158,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6801","nonce":1989083510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6803","nonce":1989083510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6807","nonce":1989083510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":1989083510},{"type":"v1","addr":"172.21.15.45:6805","nonce":1989083510}]},"public_addr":"172.21.15.45:6801/1989083510","cluster_addr":"172.21.15.45:6803/1989083510","heartbeat_back_addr":"172.21.15.45:6807/1989083510","heartbeat_front_addr":"172.21.15.45:6805/1989083510","state":["exists","up"]},{"osd":7,"uuid":"4dad773d-b7d8-46fc-8a10-a972aad27b29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":158,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6825","nonce":1513300294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6827","nonce":1513300294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6831","nonce":1513300294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1513300294},{"type":"v1","addr":"172.21.15.45:6829","nonce":1513300294}]},"public_addr":"172.21.15.45:6825/1513300294","cluster_addr":"172.21.15.45:6827/1513300294","heartbeat_back_addr":"172.21.15.45:6831/1513300294","heartbeat_front_addr":"172.21.15.45:6829/1513300294","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.881855+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:17.133252+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.735339+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.500742+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.477571+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:17.167667+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.470291+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:16.873743+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.27:6835/2940099359":"2024-09-13T03:24:09.245542+0000","172.21.15.27:6833/2940099359":"2024-09-13T03:24:09.245542+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]},{"pool":19,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T03:24:15.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:15.479+0000 7f7447fbd640 1 -- 172.21.15.27:0/1034814682 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f742406c970 msgr2=0x7f742406ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:15.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:15.479+0000 7f7447fbd640 1 --2- 172.21.15.27:0/1034814682 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f742406c970 0x7f742406ee30 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f7440160520 tx=0x7f7430008040 comp rx=0 tx=0).stop 2024-09-12T03:24:15.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:15.479+0000 7f7447fbd640 1 -- 172.21.15.27:0/1034814682 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f74400d43b0 msgr2=0x7f74400c9f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:15.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:15.479+0000 7f7447fbd640 1 --2- 172.21.15.27:0/1034814682 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f74400d43b0 0x7f74400c9f20 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 crypto rx=0x7f743c02a220 tx=0x7f743c0046c0 comp rx=0 tx=0).stop 2024-09-12T03:24:15.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:15.479+0000 7f7447fbd640 1 -- 172.21.15.27:0/1034814682 shutdown_connections 2024-09-12T03:24:15.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:15.479+0000 7f7447fbd640 1 --2- 172.21.15.27:0/1034814682 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f744015eb60 0x7f744015f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:15.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:15.479+0000 7f7447fbd640 1 --2- 172.21.15.27:0/1034814682 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f744015e780 0x7f744014cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:15.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:15.479+0000 7f7447fbd640 1 --2- 172.21.15.27:0/1034814682 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f742406c970 0x7f742406ee30 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:15.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:15.479+0000 7f7447fbd640 1 --2- 172.21.15.27:0/1034814682 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f74400d43b0 0x7f74400c9f20 unknown :-1 s=CLOSED pgs=1771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:15.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:15.479+0000 7f7447fbd640 1 -- 172.21.15.27:0/1034814682 >> 172.21.15.27:0/1034814682 conn(0x7f744006da60 msgr2=0x7f74400b7cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:15.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:15.479+0000 7f7447fbd640 1 -- 172.21.15.27:0/1034814682 shutdown_connections 2024-09-12T03:24:15.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:15.479+0000 7f7447fbd640 1 -- 172.21.15.27:0/1034814682 wait complete. 2024-09-12T03:24:16.490 INFO:tasks.ceph:Scrubbing osd.0 2024-09-12T03:24:16.490 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-09-12T03:24:16.611 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:16.611+0000 7f5c8bedd640 1 Processor -- start 2024-09-12T03:24:16.611 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:16.611+0000 7f5c8bedd640 1 -- start start 2024-09-12T03:24:16.612 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:16.612+0000 7f5c8bedd640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f5c840ae8d0 0x7f5c8415ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:16.612 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:16.612+0000 7f5c8bedd640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f5c84072610 0x7f5c84072ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:16.613 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:16.612+0000 7f5c8bedd640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5c840aedd0 con 0x7f5c8406e550 2024-09-12T03:24:16.613 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:16.612+0000 7f5c8bedd640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f5c840af0b0 con 0x7f5c84072610 2024-09-12T03:24:16.613 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:16.612+0000 7f5c8bedd640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f5c840af390 con 0x7f5c840ae8d0 2024-09-12T03:24:16.613 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:16.612+0000 7f5c89c79640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f5c8406e550 0x7f5c8406e930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:38898/0 (socket says 172.21.15.27:38898) 2024-09-12T03:24:16.613 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:16.612+0000 7f5c8a47a640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f5c84072610 0x7f5c84072ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:16.613 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:16.612+0000 7f5c89c79640 1 -- 172.21.15.27:0/3063053022 learned_addr learned my addr 172.21.15.27:0/3063053022 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:24:16.613 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:16.612+0000 7f5c89478640 1 --2- 172.21.15.27:0/3063053022 >> v2:172.21.15.45:3300/0 conn(0x7f5c840ae8d0 0x7f5c8415ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:16.613 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:16.613+0000 7f5c8a47a640 1 -- 172.21.15.27:0/3063053022 >> v2:172.21.15.45:3300/0 conn(0x7f5c840ae8d0 msgr2=0x7f5c8415ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:24:16.613 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:16.613+0000 7f5c8a47a640 1 --2- 172.21.15.27:0/3063053022 >> v2:172.21.15.45:3300/0 conn(0x7f5c840ae8d0 0x7f5c8415ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:16.613 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:16.613+0000 7f5c8a47a640 1 -- 172.21.15.27:0/3063053022 >> v1:172.21.15.27:6789/0 conn(0x7f5c8406e550 legacy=0x7f5c8406e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:16.613 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:16.613+0000 7f5c8a47a640 1 -- 172.21.15.27:0/3063053022 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c840af670 con 0x7f5c84072610 2024-09-12T03:24:16.613 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:16.613+0000 7f5c88c77640 1 -- 172.21.15.27:0/3063053022 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3357207214 0 0) 0x7f5c840aedd0 con 0x7f5c8406e550 2024-09-12T03:24:16.614 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:16.613+0000 7f5c89478640 1 --2- 172.21.15.27:0/3063053022 >> v2:172.21.15.45:3300/0 conn(0x7f5c840ae8d0 0x7f5c8415ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:24:16.614 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:16.613+0000 7f5c8a47a640 1 --2- 172.21.15.27:0/3063053022 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c84072610 0x7f5c84072ac0 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7f5c84071c30 tx=0x7f5c7402a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f727ed77e012dc58 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:16.615 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:16.614+0000 7f5c88c77640 1 -- 172.21.15.27:0/3063053022 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c74007740 con 0x7f5c84072610 2024-09-12T03:24:16.615 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:16.614+0000 7f5c88c77640 1 -- 172.21.15.27:0/3063053022 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c740078e0 con 0x7f5c84072610 2024-09-12T03:24:16.615 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:16.614+0000 7f5c88c77640 1 -- 172.21.15.27:0/3063053022 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c74007cb0 con 0x7f5c84072610 2024-09-12T03:24:16.615 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:16.614+0000 7f5c8bedd640 1 -- 172.21.15.27:0/3063053022 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c84072610 msgr2=0x7f5c84072ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:16.615 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:16.614+0000 7f5c8bedd640 1 --2- 172.21.15.27:0/3063053022 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c84072610 0x7f5c84072ac0 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7f5c84071c30 tx=0x7f5c7402a680 comp rx=0 tx=0).stop 2024-09-12T03:24:16.615 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:16.614+0000 7f5c8bedd640 1 -- 172.21.15.27:0/3063053022 shutdown_connections 2024-09-12T03:24:16.615 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:16.614+0000 7f5c8bedd640 1 --2- 172.21.15.27:0/3063053022 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c84072610 0x7f5c84072ac0 unknown :-1 s=CLOSED pgs=1773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:16.615 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:16.614+0000 7f5c8bedd640 1 --2- 172.21.15.27:0/3063053022 >> v2:172.21.15.45:3300/0 conn(0x7f5c840ae8d0 0x7f5c8415ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:16.615 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:16.614+0000 7f5c8bedd640 1 -- 172.21.15.27:0/3063053022 >> 172.21.15.27:0/3063053022 conn(0x7f5c840b26c0 msgr2=0x7f5c840b2aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:16.615 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:16.614+0000 7f5c8bedd640 1 -- 172.21.15.27:0/3063053022 shutdown_connections 2024-09-12T03:24:16.615 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:16.614+0000 7f5c8bedd640 1 -- 172.21.15.27:0/3063053022 wait complete. 2024-09-12T03:24:16.618 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:16.615+0000 7f5c8bedd640 1 Processor -- start 2024-09-12T03:24:16.618 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:16.616+0000 7f5c8bedd640 1 -- start start 2024-09-12T03:24:16.618 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:16.616+0000 7f5c8bedd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c8406e550 0x7f5c840cf5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:16.618 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:16.616+0000 7f5c8bedd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5c84072610 0x7f5c84158fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:16.618 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:16.616+0000 7f5c8bedd640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f5c840ae8d0 0x7f5c841594e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:16.618 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:16.616+0000 7f5c8bedd640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f5c840b3480 con 0x7f5c8406e550 2024-09-12T03:24:16.618 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:16.616+0000 7f5c8bedd640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f5c840b3300 con 0x7f5c84072610 2024-09-12T03:24:16.618 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:16.616+0000 7f5c8bedd640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f5c840b39c0 con 0x7f5c840ae8d0 2024-09-12T03:24:16.618 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:16.616+0000 7f5c89c79640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c8406e550 0x7f5c840cf5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:16.618 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:16.616+0000 7f5c89c79640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c8406e550 0x7f5c840cf5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41604/0 (socket says 172.21.15.27:41604) 2024-09-12T03:24:16.618 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:16.616+0000 7f5c89c79640 1 -- 172.21.15.27:0/2217764299 learned_addr learned my addr 172.21.15.27:0/2217764299 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:24:16.619 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:16.617+0000 7f5c89478640 1 --2- 172.21.15.27:0/2217764299 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5c84072610 0x7f5c84158fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:16.619 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:16.617+0000 7f5c8a47a640 1 --2- 172.21.15.27:0/2217764299 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f5c840ae8d0 0x7f5c841594e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:16.619 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:16.617+0000 7f5c89c79640 1 -- 172.21.15.27:0/2217764299 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f5c840ae8d0 msgr2=0x7f5c841594e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:16.619 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:16.617+0000 7f5c89c79640 1 --2- 172.21.15.27:0/2217764299 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f5c840ae8d0 0x7f5c841594e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:16.619 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:16.617+0000 7f5c89c79640 1 -- 172.21.15.27:0/2217764299 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5c84072610 msgr2=0x7f5c84158fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:16.619 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:16.617+0000 7f5c89c79640 1 --2- 172.21.15.27:0/2217764299 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5c84072610 0x7f5c84158fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:16.619 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:16.617+0000 7f5c89c79640 1 -- 172.21.15.27:0/2217764299 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c84157d80 con 0x7f5c8406e550 2024-09-12T03:24:16.619 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:16.617+0000 7f5c89478640 1 --2- 172.21.15.27:0/2217764299 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5c84072610 0x7f5c84158fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:24:16.619 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:16.617+0000 7f5c89c79640 1 --2- 172.21.15.27:0/2217764299 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c8406e550 0x7f5c840cf5a0 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7f5c7c00c4d0 tx=0x7f5c7c00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:16.619 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:16.617+0000 7f5c827fc640 1 -- 172.21.15.27:0/2217764299 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c7c018070 con 0x7f5c8406e550 2024-09-12T03:24:16.619 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:16.617+0000 7f5c827fc640 1 -- 172.21.15.27:0/2217764299 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c7c007500 con 0x7f5c8406e550 2024-09-12T03:24:16.619 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:16.617+0000 7f5c8bedd640 1 -- 172.21.15.27:0/2217764299 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c84158060 con 0x7f5c8406e550 2024-09-12T03:24:16.619 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:16.617+0000 7f5c827fc640 1 -- 172.21.15.27:0/2217764299 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c7c004a20 con 0x7f5c8406e550 2024-09-12T03:24:16.620 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:16.617+0000 7f5c8bedd640 1 -- 172.21.15.27:0/2217764299 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c84158620 con 0x7f5c8406e550 2024-09-12T03:24:16.620 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:16.618+0000 7f5c8bedd640 1 -- 172.21.15.27:0/2217764299 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5c840aedd0 con 0x7f5c8406e550 2024-09-12T03:24:16.621 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:16.619+0000 7f5c827fc640 1 -- 172.21.15.27:0/2217764299 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 26) ==== 91518+0+0 (secure 0 0 0) 0x7f5c7c01b070 con 0x7f5c8406e550 2024-09-12T03:24:16.621 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:16.619+0000 7f5c827fc640 1 --2- 172.21.15.27:0/2217764299 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f5c7006c970 0x7f5c7006ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:16.621 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:16.619+0000 7f5c89478640 1 --2- 172.21.15.27:0/2217764299 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f5c7006c970 0x7f5c7006ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:16.621 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:16.619+0000 7f5c827fc640 1 -- 172.21.15.27:0/2217764299 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(163..163 src has 1..163) ==== 4999+0+0 (secure 0 0 0) 0x7f5c7c088d70 con 0x7f5c8406e550 2024-09-12T03:24:16.621 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:16.619+0000 7f5c827fc640 1 --2- 172.21.15.27:0/2217764299 >> [v2:172.21.15.27:6816/3002321488,v1:172.21.15.27:6817/3002321488] conn(0x7f5c70072830 0x7f5c70074c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:16.621 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:16.619+0000 7f5c827fc640 1 -- 172.21.15.27:0/2217764299 --> [v2:172.21.15.27:6816/3002321488,v1:172.21.15.27:6817/3002321488] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5c70075340 con 0x7f5c70072830 2024-09-12T03:24:16.621 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:16.620+0000 7f5c827fc640 1 -- 172.21.15.27:0/2217764299 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_get_version_reply(handle=1 version=163) ==== 24+0+0 (secure 0 0 0) 0x7f5c7c092050 con 0x7f5c8406e550 2024-09-12T03:24:16.621 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:16.620+0000 7f5c8a47a640 1 --2- 172.21.15.27:0/2217764299 >> [v2:172.21.15.27:6816/3002321488,v1:172.21.15.27:6817/3002321488] conn(0x7f5c70072830 0x7f5c70074c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:16.621 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:16.620+0000 7f5c89478640 1 --2- 172.21.15.27:0/2217764299 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f5c7006c970 0x7f5c7006ee30 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f5c8415b8e0 tx=0x7f5c78008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:16.622 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:16.620+0000 7f5c8a47a640 1 --2- 172.21.15.27:0/2217764299 >> [v2:172.21.15.27:6816/3002321488,v1:172.21.15.27:6817/3002321488] conn(0x7f5c70072830 0x7f5c70074c70 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:16.622 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:16.621+0000 7f5c827fc640 1 -- 172.21.15.27:0/2217764299 <== osd.0 v2:172.21.15.27:6816/3002321488 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5c70075340 con 0x7f5c70072830 2024-09-12T03:24:16.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:16.633+0000 7f5c8bedd640 1 -- 172.21.15.27:0/2217764299 --> [v2:172.21.15.27:6816/3002321488,v1:172.21.15.27:6817/3002321488] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f5c840afd60 con 0x7f5c70072830 2024-09-12T03:24:16.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:16.633+0000 7f5c827fc640 1 -- 172.21.15.27:0/2217764299 <== osd.0 v2:172.21.15.27:6816/3002321488 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f5c840afd60 con 0x7f5c70072830 2024-09-12T03:24:16.634 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-12T03:24:16.634 INFO:teuthology.orchestra.run.smithi027.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-12T03:24:16.634 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-12T03:24:16.634 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:16.634+0000 7f5c8bedd640 1 -- 172.21.15.27:0/2217764299 >> [v2:172.21.15.27:6816/3002321488,v1:172.21.15.27:6817/3002321488] conn(0x7f5c70072830 msgr2=0x7f5c70074c70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:16.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:16.634+0000 7f5c8bedd640 1 --2- 172.21.15.27:0/2217764299 >> [v2:172.21.15.27:6816/3002321488,v1:172.21.15.27:6817/3002321488] conn(0x7f5c70072830 0x7f5c70074c70 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:16.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:16.634+0000 7f5c8bedd640 1 -- 172.21.15.27:0/2217764299 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f5c7006c970 msgr2=0x7f5c7006ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:16.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:16.634+0000 7f5c8bedd640 1 --2- 172.21.15.27:0/2217764299 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f5c7006c970 0x7f5c7006ee30 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f5c8415b8e0 tx=0x7f5c78008040 comp rx=0 tx=0).stop 2024-09-12T03:24:16.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:16.634+0000 7f5c8bedd640 1 -- 172.21.15.27:0/2217764299 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c8406e550 msgr2=0x7f5c840cf5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:16.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:16.634+0000 7f5c8bedd640 1 --2- 172.21.15.27:0/2217764299 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c8406e550 0x7f5c840cf5a0 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7f5c7c00c4d0 tx=0x7f5c7c00c990 comp rx=0 tx=0).stop 2024-09-12T03:24:16.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:16.634+0000 7f5c8a47a640 1 -- 172.21.15.27:0/2217764299 reap_dead start 2024-09-12T03:24:16.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:16.634+0000 7f5c8bedd640 1 -- 172.21.15.27:0/2217764299 shutdown_connections 2024-09-12T03:24:16.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:16.634+0000 7f5c8bedd640 1 -- 172.21.15.27:0/2217764299 >> 172.21.15.27:0/2217764299 conn(0x7f5c840b26c0 msgr2=0x7f5c8415e6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:16.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:16.634+0000 7f5c8bedd640 1 -- 172.21.15.27:0/2217764299 shutdown_connections 2024-09-12T03:24:16.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:16.634+0000 7f5c8bedd640 1 -- 172.21.15.27:0/2217764299 wait complete. 2024-09-12T03:24:16.645 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-09-12T03:24:16.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:16.749+0000 7f491cb5f640 1 Processor -- start 2024-09-12T03:24:16.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:16.749+0000 7f491cb5f640 1 -- start start 2024-09-12T03:24:16.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:16.750+0000 7f491cb5f640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f4918146020 0x7f49180d1720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:16.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:16.750+0000 7f491cb5f640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f49180d2100 con 0x7f4918073960 2024-09-12T03:24:16.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:16.750+0000 7f491cb5f640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f49180b6fb0 con 0x7f49180d1c60 2024-09-12T03:24:16.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:16.750+0000 7f491cb5f640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f4918146400 con 0x7f4918146020 2024-09-12T03:24:16.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:16.750+0000 7f491659c640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f4918146020 0x7f49180d1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:16.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:16.750+0000 7f4916d9d640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f4918073960 0x7f491815cbe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:38910/0 (socket says 172.21.15.27:38910) 2024-09-12T03:24:16.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:16.750+0000 7f491659c640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f4918146020 0x7f49180d1720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41608/0 (socket says 172.21.15.27:41608) 2024-09-12T03:24:16.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:16.750+0000 7f4916d9d640 1 -- 172.21.15.27:0/119163113 learned_addr learned my addr 172.21.15.27:0/119163113 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:24:16.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:16.750+0000 7f491659c640 1 -- 172.21.15.27:0/119163113 >> v1:172.21.15.45:6789/0 conn(0x7f49180d1c60 legacy=0x7f4918073250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:24:16.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:16.750+0000 7f491659c640 1 -- 172.21.15.27:0/119163113 >> v1:172.21.15.27:6789/0 conn(0x7f4918073960 legacy=0x7f491815cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:16.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:16.750+0000 7f491659c640 1 -- 172.21.15.27:0/119163113 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49180b7290 con 0x7f4918146020 2024-09-12T03:24:16.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:16.751+0000 7f491659c640 1 --2- 172.21.15.27:0/119163113 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4918146020 0x7f49180d1720 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7f4918146580 tx=0x7f490802a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f90a2f66b3373a5d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:16.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:16.751+0000 7f4915d9b640 1 -- 172.21.15.27:0/119163113 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4908037070 con 0x7f4918146020 2024-09-12T03:24:16.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:16.751+0000 7f4915d9b640 1 -- 172.21.15.27:0/119163113 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4908032440 con 0x7f4918146020 2024-09-12T03:24:16.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:16.751+0000 7f4915d9b640 1 -- 172.21.15.27:0/119163113 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4908032810 con 0x7f4918146020 2024-09-12T03:24:16.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:16.752+0000 7f491cb5f640 1 -- 172.21.15.27:0/119163113 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4918146020 msgr2=0x7f49180d1720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:16.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:16.752+0000 7f491cb5f640 1 --2- 172.21.15.27:0/119163113 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4918146020 0x7f49180d1720 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7f4918146580 tx=0x7f490802a1c0 comp rx=0 tx=0).stop 2024-09-12T03:24:16.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:16.752+0000 7f491cb5f640 1 -- 172.21.15.27:0/119163113 shutdown_connections 2024-09-12T03:24:16.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:16.752+0000 7f491cb5f640 1 --2- 172.21.15.27:0/119163113 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4918146020 0x7f49180d1720 unknown :-1 s=CLOSED pgs=1776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:16.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:16.752+0000 7f491cb5f640 1 -- 172.21.15.27:0/119163113 >> 172.21.15.27:0/119163113 conn(0x7f491806f300 msgr2=0x7f491806f710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:16.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:16.752+0000 7f491cb5f640 1 -- 172.21.15.27:0/119163113 shutdown_connections 2024-09-12T03:24:16.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:16.752+0000 7f491cb5f640 1 -- 172.21.15.27:0/119163113 wait complete. 2024-09-12T03:24:16.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:16.753+0000 7f491cb5f640 1 Processor -- start 2024-09-12T03:24:16.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:16.753+0000 7f491cb5f640 1 -- start start 2024-09-12T03:24:16.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:16.754+0000 7f491cb5f640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4918073960 0x7f491815bf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:16.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:16.754+0000 7f491cb5f640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f49180d1c60 0x7f49181e6f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:16.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:16.754+0000 7f491cb5f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4918146020 0x7f49181e4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:16.756 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:16.754+0000 7f491cb5f640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f4918070b70 con 0x7f49180d1c60 2024-09-12T03:24:16.756 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:16.754+0000 7f491cb5f640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f49180b6c60 con 0x7f4918146020 2024-09-12T03:24:16.756 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:16.754+0000 7f491cb5f640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f4918070cf0 con 0x7f4918073960 2024-09-12T03:24:16.756 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:16.754+0000 7f490ffff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f49180d1c60 0x7f49181e6f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:16.756 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:16.754+0000 7f490ffff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f49180d1c60 0x7f49181e6f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41620/0 (socket says 172.21.15.27:41620) 2024-09-12T03:24:16.756 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:16.754+0000 7f490ffff640 1 -- 172.21.15.27:0/3418107317 learned_addr learned my addr 172.21.15.27:0/3418107317 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:24:16.756 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:16.754+0000 7f491659c640 1 --2- 172.21.15.27:0/3418107317 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4918073960 0x7f491815bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:16.756 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:16.754+0000 7f4916d9d640 1 --2- 172.21.15.27:0/3418107317 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4918146020 0x7f49181e4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:16.756 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:16.754+0000 7f490ffff640 1 -- 172.21.15.27:0/3418107317 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4918073960 msgr2=0x7f491815bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:16.756 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:16.754+0000 7f490ffff640 1 --2- 172.21.15.27:0/3418107317 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4918073960 0x7f491815bf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:16.756 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:16.754+0000 7f490ffff640 1 -- 172.21.15.27:0/3418107317 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4918146020 msgr2=0x7f49181e4dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:16.756 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:16.754+0000 7f490ffff640 1 --2- 172.21.15.27:0/3418107317 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4918146020 0x7f49181e4dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:16.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:16.754+0000 7f490ffff640 1 -- 172.21.15.27:0/3418107317 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49181e7980 con 0x7f49180d1c60 2024-09-12T03:24:16.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:16.754+0000 7f491659c640 1 --2- 172.21.15.27:0/3418107317 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4918073960 0x7f491815bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:24:16.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:16.754+0000 7f490ffff640 1 --2- 172.21.15.27:0/3418107317 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f49180d1c60 0x7f49181e6f80 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7f490000c410 tx=0x7f490000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:16.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:16.755+0000 7f490effd640 1 -- 172.21.15.27:0/3418107317 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4900017070 con 0x7f49180d1c60 2024-09-12T03:24:16.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:16.755+0000 7f490effd640 1 -- 172.21.15.27:0/3418107317 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49000035f0 con 0x7f49180d1c60 2024-09-12T03:24:16.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:16.755+0000 7f491cb5f640 1 -- 172.21.15.27:0/3418107317 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49181e7c60 con 0x7f49180d1c60 2024-09-12T03:24:16.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:16.755+0000 7f490effd640 1 -- 172.21.15.27:0/3418107317 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4900003a20 con 0x7f49180d1c60 2024-09-12T03:24:16.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:16.755+0000 7f491cb5f640 1 -- 172.21.15.27:0/3418107317 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f491815c470 con 0x7f49180d1c60 2024-09-12T03:24:16.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:16.756+0000 7f491cb5f640 1 -- 172.21.15.27:0/3418107317 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49180aee80 con 0x7f49180d1c60 2024-09-12T03:24:16.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:16.756+0000 7f490effd640 1 -- 172.21.15.27:0/3418107317 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 26) ==== 91518+0+0 (secure 0 0 0) 0x7f490001a070 con 0x7f49180d1c60 2024-09-12T03:24:16.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:16.757+0000 7f490effd640 1 --2- 172.21.15.27:0/3418107317 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f48f806c970 0x7f48f806ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:16.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:16.757+0000 7f490effd640 1 -- 172.21.15.27:0/3418107317 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(163..163 src has 1..163) ==== 4999+0+0 (secure 0 0 0) 0x7f4900088e30 con 0x7f49180d1c60 2024-09-12T03:24:16.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:16.757+0000 7f491659c640 1 --2- 172.21.15.27:0/3418107317 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f48f806c970 0x7f48f806ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:16.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:16.757+0000 7f491659c640 1 --2- 172.21.15.27:0/3418107317 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f48f806c970 0x7f48f806ee30 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f4908034000 tx=0x7f490802e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:16.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:16.761+0000 7f490effd640 1 -- 172.21.15.27:0/3418107317 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f4900059040 con 0x7f49180d1c60 2024-09-12T03:24:16.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:16.936+0000 7f491cb5f640 1 -- 172.21.15.27:0/3418107317 --> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) -- 0x7f49180d1720 con 0x7f48f806c970 2024-09-12T03:24:16.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:16.937+0000 7f490effd640 1 -- 172.21.15.27:0/3418107317 <== mgr.4113 v2:172.21.15.27:6836/1226920526 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f49180d1720 con 0x7f48f806c970 2024-09-12T03:24:16.937 INFO:teuthology.orchestra.run.smithi027.stderr:instructed osd(s) 0 to deep-scrub 2024-09-12T03:24:16.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:16.939+0000 7f491cb5f640 1 -- 172.21.15.27:0/3418107317 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f48f806c970 msgr2=0x7f48f806ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:16.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:16.939+0000 7f491cb5f640 1 --2- 172.21.15.27:0/3418107317 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f48f806c970 0x7f48f806ee30 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f4908034000 tx=0x7f490802e000 comp rx=0 tx=0).stop 2024-09-12T03:24:16.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:16.939+0000 7f491cb5f640 1 -- 172.21.15.27:0/3418107317 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f49180d1c60 msgr2=0x7f49181e6f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:16.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:16.939+0000 7f491cb5f640 1 --2- 172.21.15.27:0/3418107317 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f49180d1c60 0x7f49181e6f80 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7f490000c410 tx=0x7f490000c8d0 comp rx=0 tx=0).stop 2024-09-12T03:24:16.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:16.939+0000 7f491cb5f640 1 -- 172.21.15.27:0/3418107317 shutdown_connections 2024-09-12T03:24:16.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:16.939+0000 7f491cb5f640 1 --2- 172.21.15.27:0/3418107317 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4918146020 0x7f49181e4dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:16.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:16.939+0000 7f491cb5f640 1 --2- 172.21.15.27:0/3418107317 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f48f806c970 0x7f48f806ee30 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:16.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:16.939+0000 7f491cb5f640 1 --2- 172.21.15.27:0/3418107317 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f49180d1c60 0x7f49181e6f80 unknown :-1 s=CLOSED pgs=1777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:16.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:16.939+0000 7f491cb5f640 1 --2- 172.21.15.27:0/3418107317 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f4918073960 0x7f491815bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:16.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:16.939+0000 7f491cb5f640 1 -- 172.21.15.27:0/3418107317 >> 172.21.15.27:0/3418107317 conn(0x7f491806f300 msgr2=0x7f49180b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:16.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:16.939+0000 7f491cb5f640 1 -- 172.21.15.27:0/3418107317 shutdown_connections 2024-09-12T03:24:16.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:16.939+0000 7f491cb5f640 1 -- 172.21.15.27:0/3418107317 wait complete. 2024-09-12T03:24:16.950 INFO:tasks.ceph:Scrubbing osd.1 2024-09-12T03:24:16.950 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-09-12T03:24:17.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.058+0000 7ff929986640 1 Processor -- start 2024-09-12T03:24:17.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.058+0000 7ff929986640 1 -- start start 2024-09-12T03:24:17.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.058+0000 7ff929986640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7ff92415ca40 0x7ff92406f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:17.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.059+0000 7ff929986640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ff92406f610 0x7ff924072530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:17.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.059+0000 7ff929986640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff9241600d0 con 0x7ff9240d0f60 2024-09-12T03:24:17.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.059+0000 7ff929986640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7ff92406ea90 con 0x7ff92415ca40 2024-09-12T03:24:17.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.059+0000 7ff929986640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7ff92415d000 con 0x7ff92406f610 2024-09-12T03:24:17.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.059+0000 7ff91a7fc640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7ff92415ca40 0x7ff92406f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:17.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.059+0000 7ff91a7fc640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7ff92415ca40 0x7ff92406f0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41634/0 (socket says 172.21.15.27:41634) 2024-09-12T03:24:17.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.059+0000 7ff91a7fc640 1 -- 172.21.15.27:0/694201069 learned_addr learned my addr 172.21.15.27:0/694201069 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:24:17.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.059+0000 7ff9237fe640 1 --2- 172.21.15.27:0/694201069 >> v2:172.21.15.45:3300/0 conn(0x7ff92406f610 0x7ff924072530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:17.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.059+0000 7ff91a7fc640 1 -- 172.21.15.27:0/694201069 >> v2:172.21.15.45:3300/0 conn(0x7ff92406f610 msgr2=0x7ff924072530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:24:17.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.059+0000 7ff91a7fc640 1 --2- 172.21.15.27:0/694201069 >> v2:172.21.15.45:3300/0 conn(0x7ff92406f610 0x7ff924072530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:17.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.059+0000 7ff91a7fc640 1 -- 172.21.15.27:0/694201069 >> v1:172.21.15.45:6789/0 conn(0x7ff9240d0f60 legacy=0x7ff9240cebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:24:17.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.059+0000 7ff91a7fc640 1 -- 172.21.15.27:0/694201069 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff92415d2e0 con 0x7ff92415ca40 2024-09-12T03:24:17.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.060+0000 7ff91a7fc640 1 --2- 172.21.15.27:0/694201069 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff92415ca40 0x7ff92406f0d0 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x7ff9240cfb40 tx=0x7ff90c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a25b2d0f968588a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:17.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.060+0000 7ff9227fc640 1 -- 172.21.15.27:0/694201069 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff90c037070 con 0x7ff92415ca40 2024-09-12T03:24:17.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.060+0000 7ff9227fc640 1 -- 172.21.15.27:0/694201069 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff90c032440 con 0x7ff92415ca40 2024-09-12T03:24:17.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.060+0000 7ff9227fc640 1 -- 172.21.15.27:0/694201069 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff90c032810 con 0x7ff92415ca40 2024-09-12T03:24:17.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.060+0000 7ff929986640 1 -- 172.21.15.27:0/694201069 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff92415ca40 msgr2=0x7ff92406f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:17.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.060+0000 7ff929986640 1 --2- 172.21.15.27:0/694201069 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff92415ca40 0x7ff92406f0d0 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x7ff9240cfb40 tx=0x7ff90c02a1a0 comp rx=0 tx=0).stop 2024-09-12T03:24:17.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.060+0000 7ff929986640 1 -- 172.21.15.27:0/694201069 shutdown_connections 2024-09-12T03:24:17.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.060+0000 7ff929986640 1 --2- 172.21.15.27:0/694201069 >> v2:172.21.15.45:3300/0 conn(0x7ff92406f610 0x7ff924072530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:17.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.060+0000 7ff929986640 1 --2- 172.21.15.27:0/694201069 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff92415ca40 0x7ff92406f0d0 unknown :-1 s=CLOSED pgs=1778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:17.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.060+0000 7ff929986640 1 -- 172.21.15.27:0/694201069 >> 172.21.15.27:0/694201069 conn(0x7ff92406d060 msgr2=0x7ff92406d470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:17.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.061+0000 7ff929986640 1 -- 172.21.15.27:0/694201069 shutdown_connections 2024-09-12T03:24:17.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.061+0000 7ff929986640 1 -- 172.21.15.27:0/694201069 wait complete. 2024-09-12T03:24:17.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.062+0000 7ff929986640 1 Processor -- start 2024-09-12T03:24:17.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.062+0000 7ff929986640 1 -- start start 2024-09-12T03:24:17.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.063+0000 7ff929986640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ff92406f610 0x7ff92413efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:17.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.063+0000 7ff929986640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff9240d0f60 0x7ff9240af010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:17.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.063+0000 7ff929986640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff92415ca40 0x7ff9240af550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:17.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.063+0000 7ff929986640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7ff92415d8a0 con 0x7ff9240d0f60 2024-09-12T03:24:17.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.063+0000 7ff929986640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ff9240d1340 con 0x7ff92415ca40 2024-09-12T03:24:17.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.063+0000 7ff929986640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7ff924072a70 con 0x7ff92406f610 2024-09-12T03:24:17.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.063+0000 7ff91a7fc640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff9240d0f60 0x7ff9240af010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:17.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.063+0000 7ff91a7fc640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff9240d0f60 0x7ff9240af010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41638/0 (socket says 172.21.15.27:41638) 2024-09-12T03:24:17.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.063+0000 7ff91a7fc640 1 -- 172.21.15.27:0/3806848064 learned_addr learned my addr 172.21.15.27:0/3806848064 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:24:17.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.063+0000 7ff922ffd640 1 --2- 172.21.15.27:0/3806848064 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ff92406f610 0x7ff92413efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:17.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.063+0000 7ff9237fe640 1 --2- 172.21.15.27:0/3806848064 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff92415ca40 0x7ff9240af550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:17.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.063+0000 7ff91a7fc640 1 -- 172.21.15.27:0/3806848064 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ff92406f610 msgr2=0x7ff92413efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:17.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.063+0000 7ff91a7fc640 1 --2- 172.21.15.27:0/3806848064 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ff92406f610 0x7ff92413efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:17.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.063+0000 7ff91a7fc640 1 -- 172.21.15.27:0/3806848064 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff92415ca40 msgr2=0x7ff9240af550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:17.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.063+0000 7ff91a7fc640 1 --2- 172.21.15.27:0/3806848064 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff92415ca40 0x7ff9240af550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:17.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.063+0000 7ff91a7fc640 1 -- 172.21.15.27:0/3806848064 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9240abdc0 con 0x7ff9240d0f60 2024-09-12T03:24:17.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.063+0000 7ff91a7fc640 1 --2- 172.21.15.27:0/3806848064 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff9240d0f60 0x7ff9240af010 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7ff90c000c00 tx=0x7ff90c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:17.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.063+0000 7ff920ff9640 1 -- 172.21.15.27:0/3806848064 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff90c041070 con 0x7ff9240d0f60 2024-09-12T03:24:17.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.063+0000 7ff920ff9640 1 -- 172.21.15.27:0/3806848064 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff90c03c870 con 0x7ff9240d0f60 2024-09-12T03:24:17.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.063+0000 7ff920ff9640 1 -- 172.21.15.27:0/3806848064 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff90c037040 con 0x7ff9240d0f60 2024-09-12T03:24:17.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.063+0000 7ff929986640 1 -- 172.21.15.27:0/3806848064 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff92413fa50 con 0x7ff9240d0f60 2024-09-12T03:24:17.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.064+0000 7ff929986640 1 -- 172.21.15.27:0/3806848064 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff924140010 con 0x7ff9240d0f60 2024-09-12T03:24:17.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.064+0000 7ff929986640 1 -- 172.21.15.27:0/3806848064 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff924072e00 con 0x7ff9240d0f60 2024-09-12T03:24:17.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.065+0000 7ff920ff9640 1 -- 172.21.15.27:0/3806848064 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 26) ==== 91518+0+0 (secure 0 0 0) 0x7ff90c044070 con 0x7ff9240d0f60 2024-09-12T03:24:17.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.066+0000 7ff920ff9640 1 --2- 172.21.15.27:0/3806848064 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7ff90406c970 0x7ff90406ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:17.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.066+0000 7ff920ff9640 1 -- 172.21.15.27:0/3806848064 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(163..163 src has 1..163) ==== 4999+0+0 (secure 0 0 0) 0x7ff90c0aa600 con 0x7ff9240d0f60 2024-09-12T03:24:17.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.066+0000 7ff922ffd640 1 --2- 172.21.15.27:0/3806848064 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7ff90406c970 0x7ff90406ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:17.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.066+0000 7ff920ff9640 1 --2- 172.21.15.27:0/3806848064 >> [v2:172.21.15.27:6808/1613102886,v1:172.21.15.27:6809/1613102886] conn(0x7ff904072830 0x7ff904074c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:17.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.066+0000 7ff920ff9640 1 -- 172.21.15.27:0/3806848064 --> [v2:172.21.15.27:6808/1613102886,v1:172.21.15.27:6809/1613102886] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff90c033e50 con 0x7ff904072830 2024-09-12T03:24:17.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.066+0000 7ff920ff9640 1 -- 172.21.15.27:0/3806848064 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_get_version_reply(handle=1 version=163) ==== 24+0+0 (secure 0 0 0) 0x7ff90c0b2450 con 0x7ff9240d0f60 2024-09-12T03:24:17.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.066+0000 7ff9237fe640 1 --2- 172.21.15.27:0/3806848064 >> [v2:172.21.15.27:6808/1613102886,v1:172.21.15.27:6809/1613102886] conn(0x7ff904072830 0x7ff904074c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:17.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.066+0000 7ff922ffd640 1 --2- 172.21.15.27:0/3806848064 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7ff90406c970 0x7ff90406ee30 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7ff914007fd0 tx=0x7ff914009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:17.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.066+0000 7ff9237fe640 1 --2- 172.21.15.27:0/3806848064 >> [v2:172.21.15.27:6808/1613102886,v1:172.21.15.27:6809/1613102886] conn(0x7ff904072830 0x7ff904074c70 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:17.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.067+0000 7ff920ff9640 1 -- 172.21.15.27:0/3806848064 <== osd.1 v2:172.21.15.27:6808/1613102886 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff90c033e50 con 0x7ff904072830 2024-09-12T03:24:17.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.079+0000 7ff929986640 1 -- 172.21.15.27:0/3806848064 --> [v2:172.21.15.27:6808/1613102886,v1:172.21.15.27:6809/1613102886] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7ff924160d70 con 0x7ff904072830 2024-09-12T03:24:17.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.080+0000 7ff920ff9640 1 -- 172.21.15.27:0/3806848064 <== osd.1 v2:172.21.15.27:6808/1613102886 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7ff924160d70 con 0x7ff904072830 2024-09-12T03:24:17.080 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-12T03:24:17.080 INFO:teuthology.orchestra.run.smithi027.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-12T03:24:17.080 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-12T03:24:17.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.080+0000 7ff929986640 1 -- 172.21.15.27:0/3806848064 >> [v2:172.21.15.27:6808/1613102886,v1:172.21.15.27:6809/1613102886] conn(0x7ff904072830 msgr2=0x7ff904074c70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:17.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.080+0000 7ff929986640 1 --2- 172.21.15.27:0/3806848064 >> [v2:172.21.15.27:6808/1613102886,v1:172.21.15.27:6809/1613102886] conn(0x7ff904072830 0x7ff904074c70 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:17.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.080+0000 7ff929986640 1 -- 172.21.15.27:0/3806848064 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7ff90406c970 msgr2=0x7ff90406ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:17.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.080+0000 7ff929986640 1 --2- 172.21.15.27:0/3806848064 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7ff90406c970 0x7ff90406ee30 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7ff914007fd0 tx=0x7ff914009040 comp rx=0 tx=0).stop 2024-09-12T03:24:17.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.080+0000 7ff929986640 1 -- 172.21.15.27:0/3806848064 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff9240d0f60 msgr2=0x7ff9240af010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:17.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.080+0000 7ff929986640 1 --2- 172.21.15.27:0/3806848064 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff9240d0f60 0x7ff9240af010 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7ff90c000c00 tx=0x7ff90c0046c0 comp rx=0 tx=0).stop 2024-09-12T03:24:17.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.080+0000 7ff9237fe640 1 -- 172.21.15.27:0/3806848064 reap_dead start 2024-09-12T03:24:17.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.080+0000 7ff929986640 1 -- 172.21.15.27:0/3806848064 shutdown_connections 2024-09-12T03:24:17.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.080+0000 7ff929986640 1 -- 172.21.15.27:0/3806848064 >> 172.21.15.27:0/3806848064 conn(0x7ff92406d060 msgr2=0x7ff9240b38b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:17.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.080+0000 7ff929986640 1 -- 172.21.15.27:0/3806848064 shutdown_connections 2024-09-12T03:24:17.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.080+0000 7ff929986640 1 -- 172.21.15.27:0/3806848064 wait complete. 2024-09-12T03:24:17.090 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-09-12T03:24:17.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.193+0000 7f760262e640 1 Processor -- start 2024-09-12T03:24:17.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.194+0000 7f760262e640 1 -- start start 2024-09-12T03:24:17.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.194+0000 7f760262e640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f75fc15b2b0 0x7f75fc0b3560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:17.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.194+0000 7f760262e640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f75fc06e9b0 0x7f75fc06ee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:17.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.194+0000 7f760262e640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f75fc0ce410 con 0x7f75fc072d70 2024-09-12T03:24:17.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.194+0000 7f760262e640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f75fc0720a0 con 0x7f75fc15b2b0 2024-09-12T03:24:17.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.194+0000 7f760262e640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f75fc0732c0 con 0x7f75fc06e9b0 2024-09-12T03:24:17.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.195+0000 7f75fbfff640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f75fc15b2b0 0x7f75fc0b3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:17.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.195+0000 7f75fb7fe640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f75fc072d70 0x7f75fc072430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.27:38788/0 (socket says 172.21.15.27:38788) 2024-09-12T03:24:17.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.195+0000 7f7600bcb640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f75fc06e9b0 0x7f75fc06ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:17.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.195+0000 7f75fb7fe640 1 -- 172.21.15.27:0/4095283587 learned_addr learned my addr 172.21.15.27:0/4095283587 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:24:17.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.195+0000 7f75faffd640 1 -- 172.21.15.27:0/4095283587 <== mon.1 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2934862309 0 0) 0x7f75e4001110 con 0x7f75fc072d70 2024-09-12T03:24:17.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.195+0000 7f75faffd640 1 -- 172.21.15.27:0/4095283587 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f75e8005730 con 0x7f75fc072d70 2024-09-12T03:24:17.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.195+0000 7f7600bcb640 1 -- 172.21.15.27:0/4095283587 >> v2:172.21.15.45:3300/0 conn(0x7f75fc15b2b0 msgr2=0x7f75fc0b3560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:24:17.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.195+0000 7f7600bcb640 1 --2- 172.21.15.27:0/4095283587 >> v2:172.21.15.45:3300/0 conn(0x7f75fc15b2b0 0x7f75fc0b3560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:17.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.195+0000 7f7600bcb640 1 -- 172.21.15.27:0/4095283587 >> v1:172.21.15.45:6789/0 conn(0x7f75fc072d70 legacy=0x7f75fc072430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:17.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.195+0000 7f7600bcb640 1 -- 172.21.15.27:0/4095283587 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75fc071ae0 con 0x7f75fc06e9b0 2024-09-12T03:24:17.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.196+0000 7f7600bcb640 1 --2- 172.21.15.27:0/4095283587 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f75fc06e9b0 0x7f75fc06ee40 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f75f4009d90 tx=0x7f75f402a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4873556df60f4cb6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:17.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.196+0000 7f75faffd640 1 -- 172.21.15.27:0/4095283587 <== mon.2 v2:172.21.15.45:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75f4007740 con 0x7f75fc06e9b0 2024-09-12T03:24:17.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.196+0000 7f75faffd640 1 -- 172.21.15.27:0/4095283587 <== mon.2 v2:172.21.15.45:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75f40078e0 con 0x7f75fc06e9b0 2024-09-12T03:24:17.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.196+0000 7f75faffd640 1 -- 172.21.15.27:0/4095283587 <== mon.2 v2:172.21.15.45:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75f4007c90 con 0x7f75fc06e9b0 2024-09-12T03:24:17.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.197+0000 7f760262e640 1 -- 172.21.15.27:0/4095283587 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f75fc06e9b0 msgr2=0x7f75fc06ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:17.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.197+0000 7f760262e640 1 --2- 172.21.15.27:0/4095283587 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f75fc06e9b0 0x7f75fc06ee40 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f75f4009d90 tx=0x7f75f402a660 comp rx=0 tx=0).stop 2024-09-12T03:24:17.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.197+0000 7f760262e640 1 -- 172.21.15.27:0/4095283587 shutdown_connections 2024-09-12T03:24:17.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.197+0000 7f760262e640 1 --2- 172.21.15.27:0/4095283587 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f75fc06e9b0 0x7f75fc06ee40 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:17.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.197+0000 7f760262e640 1 --2- 172.21.15.27:0/4095283587 >> v2:172.21.15.45:3300/0 conn(0x7f75fc15b2b0 0x7f75fc0b3560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:17.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.197+0000 7f760262e640 1 -- 172.21.15.27:0/4095283587 >> 172.21.15.27:0/4095283587 conn(0x7f75fc062c60 msgr2=0x7f75fc06bb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:17.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.197+0000 7f760262e640 1 -- 172.21.15.27:0/4095283587 shutdown_connections 2024-09-12T03:24:17.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.197+0000 7f760262e640 1 -- 172.21.15.27:0/4095283587 wait complete. 2024-09-12T03:24:17.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.198+0000 7f760262e640 1 Processor -- start 2024-09-12T03:24:17.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.198+0000 7f760262e640 1 -- start start 2024-09-12T03:24:17.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.199+0000 7f760262e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f75fc06e9b0 0x7f75fc144990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:17.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.199+0000 7f760262e640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f75fc072d70 0x7f75fc0b17b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:17.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.199+0000 7f760262e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f75fc15b2b0 0x7f75fc06de40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:17.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.199+0000 7f760262e640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f75fc15b690 con 0x7f75fc15b2b0 2024-09-12T03:24:17.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.199+0000 7f760262e640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f75fc0ce690 con 0x7f75fc06e9b0 2024-09-12T03:24:17.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.199+0000 7f760262e640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f75fc15b810 con 0x7f75fc072d70 2024-09-12T03:24:17.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.199+0000 7f7600bcb640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f75fc15b2b0 0x7f75fc06de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:17.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.199+0000 7f7600bcb640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f75fc15b2b0 0x7f75fc06de40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41646/0 (socket says 172.21.15.27:41646) 2024-09-12T03:24:17.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.199+0000 7f7600bcb640 1 -- 172.21.15.27:0/2031367350 learned_addr learned my addr 172.21.15.27:0/2031367350 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:24:17.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.199+0000 7f75fbfff640 1 --2- 172.21.15.27:0/2031367350 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f75fc06e9b0 0x7f75fc144990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:17.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.199+0000 7f75fb7fe640 1 --2- 172.21.15.27:0/2031367350 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f75fc072d70 0x7f75fc0b17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:17.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.200+0000 7f7600bcb640 1 -- 172.21.15.27:0/2031367350 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f75fc072d70 msgr2=0x7f75fc0b17b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:17.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.200+0000 7f7600bcb640 1 --2- 172.21.15.27:0/2031367350 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f75fc072d70 0x7f75fc0b17b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:17.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.200+0000 7f7600bcb640 1 -- 172.21.15.27:0/2031367350 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f75fc06e9b0 msgr2=0x7f75fc144990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:17.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.200+0000 7f7600bcb640 1 --2- 172.21.15.27:0/2031367350 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f75fc06e9b0 0x7f75fc144990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:17.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.200+0000 7f7600bcb640 1 -- 172.21.15.27:0/2031367350 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75fc15e820 con 0x7f75fc15b2b0 2024-09-12T03:24:17.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.200+0000 7f7600bcb640 1 --2- 172.21.15.27:0/2031367350 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f75fc15b2b0 0x7f75fc06de40 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x7f75f4009eb0 tx=0x7f75f4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:17.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.200+0000 7f75f8ff9640 1 -- 172.21.15.27:0/2031367350 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75f4040070 con 0x7f75fc15b2b0 2024-09-12T03:24:17.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.200+0000 7f75f8ff9640 1 -- 172.21.15.27:0/2031367350 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75f4031c60 con 0x7f75fc15b2b0 2024-09-12T03:24:17.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.200+0000 7f760262e640 1 -- 172.21.15.27:0/2031367350 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75fc0b20e0 con 0x7f75fc15b2b0 2024-09-12T03:24:17.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.200+0000 7f75f8ff9640 1 -- 172.21.15.27:0/2031367350 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75f4009450 con 0x7f75fc15b2b0 2024-09-12T03:24:17.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.200+0000 7f760262e640 1 -- 172.21.15.27:0/2031367350 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75fc0b26a0 con 0x7f75fc15b2b0 2024-09-12T03:24:17.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.201+0000 7f760262e640 1 -- 172.21.15.27:0/2031367350 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75fc0b3560 con 0x7f75fc15b2b0 2024-09-12T03:24:17.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.202+0000 7f75f8ff9640 1 -- 172.21.15.27:0/2031367350 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 26) ==== 91518+0+0 (secure 0 0 0) 0x7f75f4039040 con 0x7f75fc15b2b0 2024-09-12T03:24:17.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.202+0000 7f75f8ff9640 1 --2- 172.21.15.27:0/2031367350 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f75dc06c970 0x7f75dc06ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:17.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.202+0000 7f75f8ff9640 1 -- 172.21.15.27:0/2031367350 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(163..163 src has 1..163) ==== 4999+0+0 (secure 0 0 0) 0x7f75f40aad00 con 0x7f75fc15b2b0 2024-09-12T03:24:17.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.202+0000 7f75fbfff640 1 --2- 172.21.15.27:0/2031367350 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f75dc06c970 0x7f75dc06ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:17.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.203+0000 7f75fbfff640 1 --2- 172.21.15.27:0/2031367350 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f75dc06c970 0x7f75dc06ee30 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f75ec009aa0 tx=0x7f75ec009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:17.206 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.206+0000 7f75f8ff9640 1 -- 172.21.15.27:0/2031367350 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f75f407a8c0 con 0x7f75fc15b2b0 2024-09-12T03:24:17.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.381+0000 7f760262e640 1 -- 172.21.15.27:0/2031367350 --> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) -- 0x7f75fc074ed0 con 0x7f75dc06c970 2024-09-12T03:24:17.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.381+0000 7f75f8ff9640 1 -- 172.21.15.27:0/2031367350 <== mgr.4113 v2:172.21.15.27:6836/1226920526 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f75fc074ed0 con 0x7f75dc06c970 2024-09-12T03:24:17.382 INFO:teuthology.orchestra.run.smithi027.stderr:instructed osd(s) 1 to deep-scrub 2024-09-12T03:24:17.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.384+0000 7f760262e640 1 -- 172.21.15.27:0/2031367350 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f75dc06c970 msgr2=0x7f75dc06ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:17.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.384+0000 7f760262e640 1 --2- 172.21.15.27:0/2031367350 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f75dc06c970 0x7f75dc06ee30 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f75ec009aa0 tx=0x7f75ec009190 comp rx=0 tx=0).stop 2024-09-12T03:24:17.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.384+0000 7f760262e640 1 -- 172.21.15.27:0/2031367350 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f75fc15b2b0 msgr2=0x7f75fc06de40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:17.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.384+0000 7f760262e640 1 --2- 172.21.15.27:0/2031367350 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f75fc15b2b0 0x7f75fc06de40 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x7f75f4009eb0 tx=0x7f75f4004a40 comp rx=0 tx=0).stop 2024-09-12T03:24:17.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.384+0000 7f760262e640 1 -- 172.21.15.27:0/2031367350 shutdown_connections 2024-09-12T03:24:17.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.384+0000 7f760262e640 1 --2- 172.21.15.27:0/2031367350 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f75dc06c970 0x7f75dc06ee30 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:17.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.384+0000 7f760262e640 1 --2- 172.21.15.27:0/2031367350 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f75fc15b2b0 0x7f75fc06de40 unknown :-1 s=CLOSED pgs=1780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:17.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.384+0000 7f760262e640 1 --2- 172.21.15.27:0/2031367350 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f75fc072d70 0x7f75fc0b17b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:17.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.384+0000 7f760262e640 1 --2- 172.21.15.27:0/2031367350 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f75fc06e9b0 0x7f75fc144990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:17.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.384+0000 7f760262e640 1 -- 172.21.15.27:0/2031367350 >> 172.21.15.27:0/2031367350 conn(0x7f75fc062c60 msgr2=0x7f75fc0ceab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:17.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.384+0000 7f760262e640 1 -- 172.21.15.27:0/2031367350 shutdown_connections 2024-09-12T03:24:17.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.384+0000 7f760262e640 1 -- 172.21.15.27:0/2031367350 wait complete. 2024-09-12T03:24:17.394 INFO:tasks.ceph:Scrubbing osd.2 2024-09-12T03:24:17.395 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-09-12T03:24:17.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.502+0000 7ff12c1cf640 1 Processor -- start 2024-09-12T03:24:17.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.502+0000 7ff12c1cf640 1 -- start start 2024-09-12T03:24:17.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.503+0000 7ff12c1cf640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ff1240d31c0 0x7ff1240d0440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:17.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.503+0000 7ff12c1cf640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ff12406e960 0x7ff1240d0980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:17.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.503+0000 7ff12c1cf640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff1240cf9f0 con 0x7ff124070d30 2024-09-12T03:24:17.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.503+0000 7ff12c1cf640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7ff1240d1a40 con 0x7ff1240d31c0 2024-09-12T03:24:17.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.503+0000 7ff12c1cf640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7ff1240cfc90 con 0x7ff12406e960 2024-09-12T03:24:17.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.503+0000 7ff12a76c640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ff12406e960 0x7ff1240d0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:17.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.503+0000 7ff12976a640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ff1240d31c0 0x7ff1240d0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:17.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.503+0000 7ff129f6b640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7ff124070d30 0x7ff12406cc40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.27:38794/0 (socket says 172.21.15.27:38794) 2024-09-12T03:24:17.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.503+0000 7ff129f6b640 1 -- 172.21.15.27:0/674917081 learned_addr learned my addr 172.21.15.27:0/674917081 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:24:17.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.504+0000 7ff12a76c640 1 -- 172.21.15.27:0/674917081 >> v2:172.21.15.45:3300/0 conn(0x7ff1240d31c0 msgr2=0x7ff1240d0440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:24:17.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.504+0000 7ff12a76c640 1 --2- 172.21.15.27:0/674917081 >> v2:172.21.15.45:3300/0 conn(0x7ff1240d31c0 0x7ff1240d0440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:17.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.504+0000 7ff12a76c640 1 -- 172.21.15.27:0/674917081 >> v1:172.21.15.45:6789/0 conn(0x7ff124070d30 legacy=0x7ff12406cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:17.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.504+0000 7ff12a76c640 1 -- 172.21.15.27:0/674917081 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1240cff70 con 0x7ff12406e960 2024-09-12T03:24:17.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.504+0000 7ff12a76c640 1 --2- 172.21.15.27:0/674917081 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ff12406e960 0x7ff1240d0980 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7ff1240d1bc0 tx=0x7ff11c02a530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1ae7cb20e3f2f7ca server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:17.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.505+0000 7ff128f69640 1 -- 172.21.15.27:0/674917081 <== mon.2 v2:172.21.15.45:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff11c007740 con 0x7ff12406e960 2024-09-12T03:24:17.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.505+0000 7ff128f69640 1 -- 172.21.15.27:0/674917081 <== mon.2 v2:172.21.15.45:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff11c0078e0 con 0x7ff12406e960 2024-09-12T03:24:17.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.505+0000 7ff128f69640 1 -- 172.21.15.27:0/674917081 <== mon.2 v2:172.21.15.45:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff11c007cb0 con 0x7ff12406e960 2024-09-12T03:24:17.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.505+0000 7ff12c1cf640 1 -- 172.21.15.27:0/674917081 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ff12406e960 msgr2=0x7ff1240d0980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:17.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.505+0000 7ff12c1cf640 1 --2- 172.21.15.27:0/674917081 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ff12406e960 0x7ff1240d0980 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7ff1240d1bc0 tx=0x7ff11c02a530 comp rx=0 tx=0).stop 2024-09-12T03:24:17.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.505+0000 7ff12c1cf640 1 -- 172.21.15.27:0/674917081 shutdown_connections 2024-09-12T03:24:17.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.505+0000 7ff12c1cf640 1 --2- 172.21.15.27:0/674917081 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ff12406e960 0x7ff1240d0980 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:17.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.505+0000 7ff12c1cf640 1 --2- 172.21.15.27:0/674917081 >> v2:172.21.15.45:3300/0 conn(0x7ff1240d31c0 0x7ff1240d0440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:17.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.505+0000 7ff12c1cf640 1 -- 172.21.15.27:0/674917081 >> 172.21.15.27:0/674917081 conn(0x7ff1240b7c10 msgr2=0x7ff12409ba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:17.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.505+0000 7ff12c1cf640 1 -- 172.21.15.27:0/674917081 shutdown_connections 2024-09-12T03:24:17.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.505+0000 7ff12c1cf640 1 -- 172.21.15.27:0/674917081 wait complete. 2024-09-12T03:24:17.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.506+0000 7ff12c1cf640 1 Processor -- start 2024-09-12T03:24:17.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.507+0000 7ff12c1cf640 1 -- start start 2024-09-12T03:24:17.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.507+0000 7ff12c1cf640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff12406e960 0x7ff1241f7b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:17.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.507+0000 7ff12c1cf640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff124070d30 0x7ff1240cd2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:17.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.507+0000 7ff12c1cf640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ff1240d31c0 0x7ff1240cd7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:17.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.507+0000 7ff12c1cf640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7ff124168480 con 0x7ff124070d30 2024-09-12T03:24:17.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.507+0000 7ff12c1cf640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ff124168300 con 0x7ff12406e960 2024-09-12T03:24:17.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.507+0000 7ff12c1cf640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7ff124168600 con 0x7ff1240d31c0 2024-09-12T03:24:17.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.507+0000 7ff12976a640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff124070d30 0x7ff1240cd2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:17.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.507+0000 7ff12976a640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff124070d30 0x7ff1240cd2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41658/0 (socket says 172.21.15.27:41658) 2024-09-12T03:24:17.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.507+0000 7ff12976a640 1 -- 172.21.15.27:0/1333731550 learned_addr learned my addr 172.21.15.27:0/1333731550 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:24:17.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.507+0000 7ff129f6b640 1 --2- 172.21.15.27:0/1333731550 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff12406e960 0x7ff1241f7b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:17.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.508+0000 7ff12a76c640 1 --2- 172.21.15.27:0/1333731550 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ff1240d31c0 0x7ff1240cd7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:17.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.508+0000 7ff12976a640 1 -- 172.21.15.27:0/1333731550 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ff1240d31c0 msgr2=0x7ff1240cd7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:17.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.508+0000 7ff12976a640 1 --2- 172.21.15.27:0/1333731550 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ff1240d31c0 0x7ff1240cd7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:17.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.508+0000 7ff12976a640 1 -- 172.21.15.27:0/1333731550 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff12406e960 msgr2=0x7ff1241f7b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:17.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.508+0000 7ff12976a640 1 --2- 172.21.15.27:0/1333731550 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff12406e960 0x7ff1241f7b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:17.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.508+0000 7ff12976a640 1 -- 172.21.15.27:0/1333731550 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff124152570 con 0x7ff124070d30 2024-09-12T03:24:17.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.508+0000 7ff12976a640 1 --2- 172.21.15.27:0/1333731550 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff124070d30 0x7ff1240cd2b0 secure :-1 s=READY pgs=1781 cs=0 l=1 rev1=1 crypto rx=0x7ff11400e870 tx=0x7ff11400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:17.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.508+0000 7ff11a7fc640 1 -- 172.21.15.27:0/1333731550 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff114017070 con 0x7ff124070d30 2024-09-12T03:24:17.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.508+0000 7ff11a7fc640 1 -- 172.21.15.27:0/1333731550 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff11400c940 con 0x7ff124070d30 2024-09-12T03:24:17.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.508+0000 7ff12c1cf640 1 -- 172.21.15.27:0/1333731550 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1240cde70 con 0x7ff124070d30 2024-09-12T03:24:17.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.508+0000 7ff11a7fc640 1 -- 172.21.15.27:0/1333731550 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff11400cd30 con 0x7ff124070d30 2024-09-12T03:24:17.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.508+0000 7ff12c1cf640 1 -- 172.21.15.27:0/1333731550 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1240ce430 con 0x7ff124070d30 2024-09-12T03:24:17.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.509+0000 7ff12c1cf640 1 -- 172.21.15.27:0/1333731550 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff12406cc40 con 0x7ff124070d30 2024-09-12T03:24:17.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.510+0000 7ff11a7fc640 1 -- 172.21.15.27:0/1333731550 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 26) ==== 91518+0+0 (secure 0 0 0) 0x7ff11401a070 con 0x7ff124070d30 2024-09-12T03:24:17.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.510+0000 7ff11a7fc640 1 --2- 172.21.15.27:0/1333731550 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7ff10806c970 0x7ff10806ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:17.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.510+0000 7ff11a7fc640 1 -- 172.21.15.27:0/1333731550 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(163..163 src has 1..163) ==== 4999+0+0 (secure 0 0 0) 0x7ff114089ba0 con 0x7ff124070d30 2024-09-12T03:24:17.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.510+0000 7ff129f6b640 1 --2- 172.21.15.27:0/1333731550 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7ff10806c970 0x7ff10806ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:17.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.510+0000 7ff11a7fc640 1 --2- 172.21.15.27:0/1333731550 >> [v2:172.21.15.27:6824/3534723355,v1:172.21.15.27:6825/3534723355] conn(0x7ff108072830 0x7ff108074c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:17.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.510+0000 7ff11a7fc640 1 -- 172.21.15.27:0/1333731550 --> [v2:172.21.15.27:6824/3534723355,v1:172.21.15.27:6825/3534723355] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff114022e50 con 0x7ff108072830 2024-09-12T03:24:17.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.511+0000 7ff11a7fc640 1 -- 172.21.15.27:0/1333731550 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_get_version_reply(handle=1 version=163) ==== 24+0+0 (secure 0 0 0) 0x7ff11408c790 con 0x7ff124070d30 2024-09-12T03:24:17.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.511+0000 7ff12a76c640 1 --2- 172.21.15.27:0/1333731550 >> [v2:172.21.15.27:6824/3534723355,v1:172.21.15.27:6825/3534723355] conn(0x7ff108072830 0x7ff108074c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:17.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.511+0000 7ff129f6b640 1 --2- 172.21.15.27:0/1333731550 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7ff10806c970 0x7ff10806ee30 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7ff120005880 tx=0x7ff120009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:17.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.511+0000 7ff12a76c640 1 --2- 172.21.15.27:0/1333731550 >> [v2:172.21.15.27:6824/3534723355,v1:172.21.15.27:6825/3534723355] conn(0x7ff108072830 0x7ff108074c70 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:17.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.512+0000 7ff11a7fc640 1 -- 172.21.15.27:0/1333731550 <== osd.2 v2:172.21.15.27:6824/3534723355 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff114022e50 con 0x7ff108072830 2024-09-12T03:24:17.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.524+0000 7ff12c1cf640 1 -- 172.21.15.27:0/1333731550 --> [v2:172.21.15.27:6824/3534723355,v1:172.21.15.27:6825/3534723355] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7ff1240d1230 con 0x7ff108072830 2024-09-12T03:24:17.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.524+0000 7ff11a7fc640 1 -- 172.21.15.27:0/1333731550 <== osd.2 v2:172.21.15.27:6824/3534723355 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7ff1240d1230 con 0x7ff108072830 2024-09-12T03:24:17.524 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-12T03:24:17.524 INFO:teuthology.orchestra.run.smithi027.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-12T03:24:17.525 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-12T03:24:17.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.525+0000 7ff12c1cf640 1 -- 172.21.15.27:0/1333731550 >> [v2:172.21.15.27:6824/3534723355,v1:172.21.15.27:6825/3534723355] conn(0x7ff108072830 msgr2=0x7ff108074c70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:17.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.525+0000 7ff12c1cf640 1 --2- 172.21.15.27:0/1333731550 >> [v2:172.21.15.27:6824/3534723355,v1:172.21.15.27:6825/3534723355] conn(0x7ff108072830 0x7ff108074c70 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:17.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.525+0000 7ff12c1cf640 1 -- 172.21.15.27:0/1333731550 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7ff10806c970 msgr2=0x7ff10806ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:17.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.525+0000 7ff12c1cf640 1 --2- 172.21.15.27:0/1333731550 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7ff10806c970 0x7ff10806ee30 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7ff120005880 tx=0x7ff120009040 comp rx=0 tx=0).stop 2024-09-12T03:24:17.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.525+0000 7ff12c1cf640 1 -- 172.21.15.27:0/1333731550 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff124070d30 msgr2=0x7ff1240cd2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:17.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.525+0000 7ff12c1cf640 1 --2- 172.21.15.27:0/1333731550 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff124070d30 0x7ff1240cd2b0 secure :-1 s=READY pgs=1781 cs=0 l=1 rev1=1 crypto rx=0x7ff11400e870 tx=0x7ff11400ed30 comp rx=0 tx=0).stop 2024-09-12T03:24:17.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.525+0000 7ff12a76c640 1 -- 172.21.15.27:0/1333731550 reap_dead start 2024-09-12T03:24:17.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.525+0000 7ff12c1cf640 1 -- 172.21.15.27:0/1333731550 shutdown_connections 2024-09-12T03:24:17.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.525+0000 7ff12c1cf640 1 -- 172.21.15.27:0/1333731550 >> 172.21.15.27:0/1333731550 conn(0x7ff1240b7c10 msgr2=0x7ff12415de60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:17.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.525+0000 7ff12c1cf640 1 -- 172.21.15.27:0/1333731550 shutdown_connections 2024-09-12T03:24:17.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.525+0000 7ff12c1cf640 1 -- 172.21.15.27:0/1333731550 wait complete. 2024-09-12T03:24:17.535 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-09-12T03:24:17.638 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.638+0000 7f8454ad7640 1 Processor -- start 2024-09-12T03:24:17.638 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.638+0000 7f8454ad7640 1 -- start start 2024-09-12T03:24:17.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.639+0000 7f8454ad7640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f84500d1050 0x7f84500cf810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:17.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.639+0000 7f8454ad7640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f84500d52e0 0x7f84500d1430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:17.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.639+0000 7f8454ad7640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f84500d1e60 con 0x7f8450162520 2024-09-12T03:24:17.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.639+0000 7f8454ad7640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f84500d2380 con 0x7f84500d1050 2024-09-12T03:24:17.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.639+0000 7f8454ad7640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f84500cfef0 con 0x7f84500d52e0 2024-09-12T03:24:17.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.639+0000 7f844e59c640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f84500d1050 0x7f84500cf810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:17.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.639+0000 7f844ed9d640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f8450162520 0x7f84501629b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:38922/0 (socket says 172.21.15.27:38922) 2024-09-12T03:24:17.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.639+0000 7f844ed9d640 1 -- 172.21.15.27:0/2516045563 learned_addr learned my addr 172.21.15.27:0/2516045563 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:24:17.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.639+0000 7f844dd9b640 1 --2- 172.21.15.27:0/2516045563 >> v2:172.21.15.45:3300/0 conn(0x7f84500d52e0 0x7f84500d1430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:17.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.640+0000 7f844e59c640 1 -- 172.21.15.27:0/2516045563 >> v2:172.21.15.45:3300/0 conn(0x7f84500d52e0 msgr2=0x7f84500d1430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:24:17.641 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.640+0000 7f844e59c640 1 --2- 172.21.15.27:0/2516045563 >> v2:172.21.15.45:3300/0 conn(0x7f84500d52e0 0x7f84500d1430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:17.641 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.640+0000 7f844e59c640 1 -- 172.21.15.27:0/2516045563 >> v1:172.21.15.27:6789/0 conn(0x7f8450162520 legacy=0x7f84501629b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:17.641 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.640+0000 7f844e59c640 1 -- 172.21.15.27:0/2516045563 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84500d2640 con 0x7f84500d1050 2024-09-12T03:24:17.641 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.640+0000 7f844d59a640 1 -- 172.21.15.27:0/2516045563 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2433208416 0 0) 0x7f84500d1e60 con 0x7f8450162520 2024-09-12T03:24:17.641 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.640+0000 7f844dd9b640 1 --2- 172.21.15.27:0/2516045563 >> v2:172.21.15.45:3300/0 conn(0x7f84500d52e0 0x7f84500d1430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:24:17.645 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.640+0000 7f844e59c640 1 --2- 172.21.15.27:0/2516045563 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f84500d1050 0x7f84500cf810 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7f84500743f0 tx=0x7f844802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6575ceec9d1a041 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:17.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.641+0000 7f844d59a640 1 -- 172.21.15.27:0/2516045563 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8448037070 con 0x7f84500d1050 2024-09-12T03:24:17.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.641+0000 7f844d59a640 1 -- 172.21.15.27:0/2516045563 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8448032440 con 0x7f84500d1050 2024-09-12T03:24:17.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.641+0000 7f844d59a640 1 -- 172.21.15.27:0/2516045563 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8448032810 con 0x7f84500d1050 2024-09-12T03:24:17.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.641+0000 7f8454ad7640 1 -- 172.21.15.27:0/2516045563 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f84500d1050 msgr2=0x7f84500cf810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:17.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.641+0000 7f8454ad7640 1 --2- 172.21.15.27:0/2516045563 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f84500d1050 0x7f84500cf810 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7f84500743f0 tx=0x7f844802a1a0 comp rx=0 tx=0).stop 2024-09-12T03:24:17.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.641+0000 7f8454ad7640 1 -- 172.21.15.27:0/2516045563 shutdown_connections 2024-09-12T03:24:17.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.641+0000 7f8454ad7640 1 --2- 172.21.15.27:0/2516045563 >> v2:172.21.15.45:3300/0 conn(0x7f84500d52e0 0x7f84500d1430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:17.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.641+0000 7f8454ad7640 1 --2- 172.21.15.27:0/2516045563 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f84500d1050 0x7f84500cf810 unknown :-1 s=CLOSED pgs=1783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:17.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.641+0000 7f8454ad7640 1 -- 172.21.15.27:0/2516045563 >> 172.21.15.27:0/2516045563 conn(0x7f845006ddf0 msgr2=0x7f845006e200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:17.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.641+0000 7f8454ad7640 1 -- 172.21.15.27:0/2516045563 shutdown_connections 2024-09-12T03:24:17.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.641+0000 7f8454ad7640 1 -- 172.21.15.27:0/2516045563 wait complete. 2024-09-12T03:24:17.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.643+0000 7f8454ad7640 1 Processor -- start 2024-09-12T03:24:17.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.643+0000 7f8454ad7640 1 -- start start 2024-09-12T03:24:17.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.643+0000 7f8454ad7640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f84500d1050 0x7f845015d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:17.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.643+0000 7f8454ad7640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f84500d52e0 0x7f845015c860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:17.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.643+0000 7f8454ad7640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f8450162520 0x7f84501e8360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:17.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.643+0000 7f8454ad7640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f84500d1970 con 0x7f84500d1050 2024-09-12T03:24:17.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.643+0000 7f8454ad7640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f84500d2c00 con 0x7f84500d52e0 2024-09-12T03:24:17.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.643+0000 7f8454ad7640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f84500b7a50 con 0x7f8450162520 2024-09-12T03:24:17.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.643+0000 7f844e59c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f84500d1050 0x7f845015d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:17.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.643+0000 7f844e59c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f84500d1050 0x7f845015d080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41676/0 (socket says 172.21.15.27:41676) 2024-09-12T03:24:17.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.643+0000 7f844e59c640 1 -- 172.21.15.27:0/1523691755 learned_addr learned my addr 172.21.15.27:0/1523691755 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:24:17.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.644+0000 7f844ed9d640 1 --2- 172.21.15.27:0/1523691755 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f8450162520 0x7f84501e8360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:17.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.644+0000 7f844dd9b640 1 --2- 172.21.15.27:0/1523691755 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f84500d52e0 0x7f845015c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:17.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.644+0000 7f844e59c640 1 -- 172.21.15.27:0/1523691755 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f8450162520 msgr2=0x7f84501e8360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:17.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.644+0000 7f844e59c640 1 --2- 172.21.15.27:0/1523691755 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f8450162520 0x7f84501e8360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:17.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.644+0000 7f844e59c640 1 -- 172.21.15.27:0/1523691755 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f84500d52e0 msgr2=0x7f845015c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:17.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.644+0000 7f844e59c640 1 --2- 172.21.15.27:0/1523691755 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f84500d52e0 0x7f845015c860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:17.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.644+0000 7f844e59c640 1 -- 172.21.15.27:0/1523691755 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84500c8380 con 0x7f84500d1050 2024-09-12T03:24:17.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.644+0000 7f844e59c640 1 --2- 172.21.15.27:0/1523691755 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f84500d1050 0x7f845015d080 secure :-1 s=READY pgs=1784 cs=0 l=1 rev1=1 crypto rx=0x7f8448000c00 tx=0x7f84480046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:17.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.644+0000 7f8446ffd640 1 -- 172.21.15.27:0/1523691755 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8448037050 con 0x7f84500d1050 2024-09-12T03:24:17.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.644+0000 7f8446ffd640 1 -- 172.21.15.27:0/1523691755 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8448038040 con 0x7f84500d1050 2024-09-12T03:24:17.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.644+0000 7f8454ad7640 1 -- 172.21.15.27:0/1523691755 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84501612b0 con 0x7f84500d1050 2024-09-12T03:24:17.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.644+0000 7f8446ffd640 1 -- 172.21.15.27:0/1523691755 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8448006e10 con 0x7f84500d1050 2024-09-12T03:24:17.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.644+0000 7f8454ad7640 1 -- 172.21.15.27:0/1523691755 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84500c8900 con 0x7f84500d1050 2024-09-12T03:24:17.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.645+0000 7f8454ad7640 1 -- 172.21.15.27:0/1523691755 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f845015ec60 con 0x7f84500d1050 2024-09-12T03:24:17.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.646+0000 7f8446ffd640 1 -- 172.21.15.27:0/1523691755 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 26) ==== 91518+0+0 (secure 0 0 0) 0x7f8448030980 con 0x7f84500d1050 2024-09-12T03:24:17.650 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.646+0000 7f8446ffd640 1 --2- 172.21.15.27:0/1523691755 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f843006c970 0x7f843006ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:17.650 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.646+0000 7f8446ffd640 1 -- 172.21.15.27:0/1523691755 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(163..163 src has 1..163) ==== 4999+0+0 (secure 0 0 0) 0x7f84480aa630 con 0x7f84500d1050 2024-09-12T03:24:17.650 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.646+0000 7f844dd9b640 1 --2- 172.21.15.27:0/1523691755 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f843006c970 0x7f843006ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:17.650 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.647+0000 7f844dd9b640 1 --2- 172.21.15.27:0/1523691755 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f843006c970 0x7f843006ee30 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f84501e9f30 tx=0x7f8438008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:17.650 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.651+0000 7f8446ffd640 1 -- 172.21.15.27:0/1523691755 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f844807a8e0 con 0x7f84500d1050 2024-09-12T03:24:17.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.825+0000 7f8454ad7640 1 -- 172.21.15.27:0/1523691755 --> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) -- 0x7f84500cb5a0 con 0x7f843006c970 2024-09-12T03:24:17.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.825+0000 7f8446ffd640 1 -- 172.21.15.27:0/1523691755 <== mgr.4113 v2:172.21.15.27:6836/1226920526 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f84500cb5a0 con 0x7f843006c970 2024-09-12T03:24:17.825 INFO:teuthology.orchestra.run.smithi027.stderr:instructed osd(s) 2 to deep-scrub 2024-09-12T03:24:17.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.827+0000 7f8454ad7640 1 -- 172.21.15.27:0/1523691755 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f843006c970 msgr2=0x7f843006ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:17.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.827+0000 7f8454ad7640 1 --2- 172.21.15.27:0/1523691755 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f843006c970 0x7f843006ee30 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f84501e9f30 tx=0x7f8438008040 comp rx=0 tx=0).stop 2024-09-12T03:24:17.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.827+0000 7f8454ad7640 1 -- 172.21.15.27:0/1523691755 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f84500d1050 msgr2=0x7f845015d080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:17.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.827+0000 7f8454ad7640 1 --2- 172.21.15.27:0/1523691755 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f84500d1050 0x7f845015d080 secure :-1 s=READY pgs=1784 cs=0 l=1 rev1=1 crypto rx=0x7f8448000c00 tx=0x7f84480046c0 comp rx=0 tx=0).stop 2024-09-12T03:24:17.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.828+0000 7f8454ad7640 1 -- 172.21.15.27:0/1523691755 shutdown_connections 2024-09-12T03:24:17.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.828+0000 7f8454ad7640 1 --2- 172.21.15.27:0/1523691755 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f8450162520 0x7f84501e8360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:17.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.828+0000 7f8454ad7640 1 --2- 172.21.15.27:0/1523691755 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f84500d52e0 0x7f845015c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:17.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.828+0000 7f8454ad7640 1 --2- 172.21.15.27:0/1523691755 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f843006c970 0x7f843006ee30 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:17.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.828+0000 7f8454ad7640 1 --2- 172.21.15.27:0/1523691755 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f84500d1050 0x7f845015d080 unknown :-1 s=CLOSED pgs=1784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:17.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.828+0000 7f8454ad7640 1 -- 172.21.15.27:0/1523691755 >> 172.21.15.27:0/1523691755 conn(0x7f845006ddf0 msgr2=0x7f845015d8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:17.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.828+0000 7f8454ad7640 1 -- 172.21.15.27:0/1523691755 shutdown_connections 2024-09-12T03:24:17.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.828+0000 7f8454ad7640 1 -- 172.21.15.27:0/1523691755 wait complete. 2024-09-12T03:24:17.838 INFO:tasks.ceph:Scrubbing osd.3 2024-09-12T03:24:17.839 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-09-12T03:24:17.945 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.945+0000 7ffbd2caf640 1 Processor -- start 2024-09-12T03:24:17.946 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.945+0000 7ffbd2caf640 1 -- start start 2024-09-12T03:24:17.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.946+0000 7ffbd2caf640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7ffbcc15a0a0 0x7ffbcc160c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:17.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.946+0000 7ffbd2caf640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ffbcc0d04d0 0x7ffbcc15cae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:17.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.946+0000 7ffbd2caf640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffbcc0b5b80 con 0x7ffbcc0ce280 2024-09-12T03:24:17.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.946+0000 7ffbd2caf640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7ffbcc06da80 con 0x7ffbcc15a0a0 2024-09-12T03:24:17.950 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.946+0000 7ffbd2caf640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7ffbcc0b02b0 con 0x7ffbcc0d04d0 2024-09-12T03:24:17.950 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.946+0000 7ffbd0a4b640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7ffbcc15a0a0 0x7ffbcc160c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:17.950 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.946+0000 7ffbc3fff640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7ffbcc0ce280 0x7ffbcc0cfdc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:38930/0 (socket says 172.21.15.27:38930) 2024-09-12T03:24:17.950 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.946+0000 7ffbd0a4b640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7ffbcc15a0a0 0x7ffbcc160c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41680/0 (socket says 172.21.15.27:41680) 2024-09-12T03:24:17.950 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.946+0000 7ffbc3fff640 1 -- 172.21.15.27:0/634345254 learned_addr learned my addr 172.21.15.27:0/634345254 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:24:17.950 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.947+0000 7ffbd0a4b640 1 -- 172.21.15.27:0/634345254 >> v2:172.21.15.45:3300/0 conn(0x7ffbcc0d04d0 msgr2=0x7ffbcc15cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:24:17.950 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.947+0000 7ffbd0a4b640 1 --2- 172.21.15.27:0/634345254 >> v2:172.21.15.45:3300/0 conn(0x7ffbcc0d04d0 0x7ffbcc15cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:17.950 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.947+0000 7ffbd0a4b640 1 -- 172.21.15.27:0/634345254 >> v1:172.21.15.27:6789/0 conn(0x7ffbcc0ce280 legacy=0x7ffbcc0cfdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:17.950 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.947+0000 7ffbc37fe640 1 -- 172.21.15.27:0/634345254 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4232036587 0 0) 0x7ffbcc0b5b80 con 0x7ffbcc0ce280 2024-09-12T03:24:17.950 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.947+0000 7ffbd0a4b640 1 -- 172.21.15.27:0/634345254 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbcc0b6340 con 0x7ffbcc15a0a0 2024-09-12T03:24:17.950 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.947+0000 7ffbd0a4b640 1 --2- 172.21.15.27:0/634345254 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffbcc15a0a0 0x7ffbcc160c50 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x7ffbcc160060 tx=0x7ffbc802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cd053aaaf09721c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:17.950 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.948+0000 7ffbc37fe640 1 -- 172.21.15.27:0/634345254 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbc8037070 con 0x7ffbcc15a0a0 2024-09-12T03:24:17.951 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.948+0000 7ffbc37fe640 1 -- 172.21.15.27:0/634345254 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffbc8032440 con 0x7ffbcc15a0a0 2024-09-12T03:24:17.951 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.948+0000 7ffbc37fe640 1 -- 172.21.15.27:0/634345254 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbc8032810 con 0x7ffbcc15a0a0 2024-09-12T03:24:17.951 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.948+0000 7ffbd2caf640 1 -- 172.21.15.27:0/634345254 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffbcc15a0a0 msgr2=0x7ffbcc160c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:17.951 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.948+0000 7ffbd2caf640 1 --2- 172.21.15.27:0/634345254 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffbcc15a0a0 0x7ffbcc160c50 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x7ffbcc160060 tx=0x7ffbc802a1a0 comp rx=0 tx=0).stop 2024-09-12T03:24:17.951 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.948+0000 7ffbd2caf640 1 -- 172.21.15.27:0/634345254 shutdown_connections 2024-09-12T03:24:17.951 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.948+0000 7ffbd2caf640 1 --2- 172.21.15.27:0/634345254 >> v2:172.21.15.45:3300/0 conn(0x7ffbcc0d04d0 0x7ffbcc15cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:17.951 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.948+0000 7ffbd2caf640 1 --2- 172.21.15.27:0/634345254 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffbcc15a0a0 0x7ffbcc160c50 unknown :-1 s=CLOSED pgs=1786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:17.951 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.948+0000 7ffbd2caf640 1 -- 172.21.15.27:0/634345254 >> 172.21.15.27:0/634345254 conn(0x7ffbcc06bb30 msgr2=0x7ffbcc06bf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:17.951 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.948+0000 7ffbd2caf640 1 -- 172.21.15.27:0/634345254 shutdown_connections 2024-09-12T03:24:17.951 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.948+0000 7ffbd2caf640 1 -- 172.21.15.27:0/634345254 wait complete. 2024-09-12T03:24:17.951 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.949+0000 7ffbd2caf640 1 Processor -- start 2024-09-12T03:24:17.951 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.950+0000 7ffbd2caf640 1 -- start start 2024-09-12T03:24:17.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.950+0000 7ffbd2caf640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffbcc0ce280 0x7ffbcc0ae010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:17.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.950+0000 7ffbd2caf640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffbcc0d04d0 0x7ffbcc0ac200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:17.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.950+0000 7ffbd2caf640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ffbcc15a0a0 0x7ffbcc0aa880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:17.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.950+0000 7ffbd2caf640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7ffbcc0b5e20 con 0x7ffbcc0d04d0 2024-09-12T03:24:17.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.950+0000 7ffbd2caf640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ffbcc0b07c0 con 0x7ffbcc0ce280 2024-09-12T03:24:17.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.950+0000 7ffbd2caf640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7ffbcc0ce710 con 0x7ffbcc15a0a0 2024-09-12T03:24:17.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.950+0000 7ffbc3fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffbcc0d04d0 0x7ffbcc0ac200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:17.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.951+0000 7ffbc3fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffbcc0d04d0 0x7ffbcc0ac200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41694/0 (socket says 172.21.15.27:41694) 2024-09-12T03:24:17.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.951+0000 7ffbc3fff640 1 -- 172.21.15.27:0/972994090 learned_addr learned my addr 172.21.15.27:0/972994090 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:24:17.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.951+0000 7ffbd0a4b640 1 --2- 172.21.15.27:0/972994090 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffbcc0ce280 0x7ffbcc0ae010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:17.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.951+0000 7ffbd124c640 1 --2- 172.21.15.27:0/972994090 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ffbcc15a0a0 0x7ffbcc0aa880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:17.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.951+0000 7ffbc3fff640 1 -- 172.21.15.27:0/972994090 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ffbcc15a0a0 msgr2=0x7ffbcc0aa880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:17.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.951+0000 7ffbc3fff640 1 --2- 172.21.15.27:0/972994090 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ffbcc15a0a0 0x7ffbcc0aa880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:17.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.951+0000 7ffbc3fff640 1 -- 172.21.15.27:0/972994090 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffbcc0ce280 msgr2=0x7ffbcc0ae010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:17.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.951+0000 7ffbc3fff640 1 --2- 172.21.15.27:0/972994090 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffbcc0ce280 0x7ffbcc0ae010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:17.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.951+0000 7ffbc3fff640 1 -- 172.21.15.27:0/972994090 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbcc0ae960 con 0x7ffbcc0d04d0 2024-09-12T03:24:17.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.951+0000 7ffbd124c640 1 --2- 172.21.15.27:0/972994090 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ffbcc15a0a0 0x7ffbcc0aa880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:24:17.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.951+0000 7ffbc3fff640 1 --2- 172.21.15.27:0/972994090 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffbcc0d04d0 0x7ffbcc0ac200 secure :-1 s=READY pgs=1787 cs=0 l=1 rev1=1 crypto rx=0x7ffbbc00c4a0 tx=0x7ffbbc00c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:17.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.951+0000 7ffbc17fa640 1 -- 172.21.15.27:0/972994090 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbbc017070 con 0x7ffbcc0d04d0 2024-09-12T03:24:17.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.951+0000 7ffbc17fa640 1 -- 172.21.15.27:0/972994090 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffbbc005ce0 con 0x7ffbcc0d04d0 2024-09-12T03:24:17.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.951+0000 7ffbc17fa640 1 -- 172.21.15.27:0/972994090 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbbc0072b0 con 0x7ffbcc0d04d0 2024-09-12T03:24:17.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.951+0000 7ffbd2caf640 1 -- 172.21.15.27:0/972994090 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffbcc0aec40 con 0x7ffbcc0d04d0 2024-09-12T03:24:17.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.951+0000 7ffbd2caf640 1 -- 172.21.15.27:0/972994090 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffbcc0af200 con 0x7ffbcc0d04d0 2024-09-12T03:24:17.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.952+0000 7ffbd2caf640 1 -- 172.21.15.27:0/972994090 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ffbcc0af400 con 0x7ffbcc0d04d0 2024-09-12T03:24:17.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.953+0000 7ffbc17fa640 1 -- 172.21.15.27:0/972994090 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 26) ==== 91518+0+0 (secure 0 0 0) 0x7ffbbc01a070 con 0x7ffbcc0d04d0 2024-09-12T03:24:17.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.953+0000 7ffbc17fa640 1 --2- 172.21.15.27:0/972994090 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7ffbb806c970 0x7ffbb806ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:17.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.953+0000 7ffbc17fa640 1 -- 172.21.15.27:0/972994090 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(163..163 src has 1..163) ==== 4999+0+0 (secure 0 0 0) 0x7ffbbc088db0 con 0x7ffbcc0d04d0 2024-09-12T03:24:17.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.954+0000 7ffbd0a4b640 1 --2- 172.21.15.27:0/972994090 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7ffbb806c970 0x7ffbb806ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:17.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.954+0000 7ffbc17fa640 1 --2- 172.21.15.27:0/972994090 >> [v2:172.21.15.27:6800/80661598,v1:172.21.15.27:6801/80661598] conn(0x7ffbb8072830 0x7ffbb8074c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:17.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.954+0000 7ffbc17fa640 1 -- 172.21.15.27:0/972994090 --> [v2:172.21.15.27:6800/80661598,v1:172.21.15.27:6801/80661598] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ffbbc015e70 con 0x7ffbb8072830 2024-09-12T03:24:17.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.954+0000 7ffbc17fa640 1 -- 172.21.15.27:0/972994090 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_get_version_reply(handle=1 version=163) ==== 24+0+0 (secure 0 0 0) 0x7ffbbc092050 con 0x7ffbcc0d04d0 2024-09-12T03:24:17.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.954+0000 7ffbd124c640 1 --2- 172.21.15.27:0/972994090 >> [v2:172.21.15.27:6800/80661598,v1:172.21.15.27:6801/80661598] conn(0x7ffbb8072830 0x7ffbb8074c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:17.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.954+0000 7ffbd0a4b640 1 --2- 172.21.15.27:0/972994090 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7ffbb806c970 0x7ffbb806ee30 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7ffbc8034000 tx=0x7ffbc802e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:17.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.954+0000 7ffbd124c640 1 --2- 172.21.15.27:0/972994090 >> [v2:172.21.15.27:6800/80661598,v1:172.21.15.27:6801/80661598] conn(0x7ffbb8072830 0x7ffbb8074c70 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:17.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.955+0000 7ffbc17fa640 1 -- 172.21.15.27:0/972994090 <== osd.3 v2:172.21.15.27:6800/80661598 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ffbbc015e70 con 0x7ffbb8072830 2024-09-12T03:24:17.967 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.967+0000 7ffbd2caf640 1 -- 172.21.15.27:0/972994090 --> [v2:172.21.15.27:6800/80661598,v1:172.21.15.27:6801/80661598] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7ffbcc15a480 con 0x7ffbb8072830 2024-09-12T03:24:17.967 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.968+0000 7ffbc17fa640 1 -- 172.21.15.27:0/972994090 <== osd.3 v2:172.21.15.27:6800/80661598 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7ffbcc15a480 con 0x7ffbb8072830 2024-09-12T03:24:17.968 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-12T03:24:17.968 INFO:teuthology.orchestra.run.smithi027.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-12T03:24:17.968 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-12T03:24:17.969 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.968+0000 7ffbd2caf640 1 -- 172.21.15.27:0/972994090 >> [v2:172.21.15.27:6800/80661598,v1:172.21.15.27:6801/80661598] conn(0x7ffbb8072830 msgr2=0x7ffbb8074c70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:17.969 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.968+0000 7ffbd2caf640 1 --2- 172.21.15.27:0/972994090 >> [v2:172.21.15.27:6800/80661598,v1:172.21.15.27:6801/80661598] conn(0x7ffbb8072830 0x7ffbb8074c70 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:17.969 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.968+0000 7ffbd2caf640 1 -- 172.21.15.27:0/972994090 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7ffbb806c970 msgr2=0x7ffbb806ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:17.969 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.968+0000 7ffbd2caf640 1 --2- 172.21.15.27:0/972994090 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7ffbb806c970 0x7ffbb806ee30 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7ffbc8034000 tx=0x7ffbc802e000 comp rx=0 tx=0).stop 2024-09-12T03:24:17.969 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.968+0000 7ffbd2caf640 1 -- 172.21.15.27:0/972994090 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffbcc0d04d0 msgr2=0x7ffbcc0ac200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:17.969 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.968+0000 7ffbd2caf640 1 --2- 172.21.15.27:0/972994090 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffbcc0d04d0 0x7ffbcc0ac200 secure :-1 s=READY pgs=1787 cs=0 l=1 rev1=1 crypto rx=0x7ffbbc00c4a0 tx=0x7ffbbc00c960 comp rx=0 tx=0).stop 2024-09-12T03:24:17.969 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.968+0000 7ffbd124c640 1 -- 172.21.15.27:0/972994090 reap_dead start 2024-09-12T03:24:17.969 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.968+0000 7ffbd2caf640 1 -- 172.21.15.27:0/972994090 shutdown_connections 2024-09-12T03:24:17.969 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.968+0000 7ffbd2caf640 1 -- 172.21.15.27:0/972994090 >> 172.21.15.27:0/972994090 conn(0x7ffbcc06bb30 msgr2=0x7ffbcc160a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:17.969 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.968+0000 7ffbd2caf640 1 -- 172.21.15.27:0/972994090 shutdown_connections 2024-09-12T03:24:17.969 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:17.968+0000 7ffbd2caf640 1 -- 172.21.15.27:0/972994090 wait complete. 2024-09-12T03:24:17.979 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-09-12T03:24:18.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.082+0000 7f371d1d6640 1 Processor -- start 2024-09-12T03:24:18.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.082+0000 7f371d1d6640 1 -- start start 2024-09-12T03:24:18.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.083+0000 7f371d1d6640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f3718162a30 0x7f37181693f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:18.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.083+0000 7f371d1d6640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f37180d2240 con 0x7f371815e5f0 2024-09-12T03:24:18.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.083+0000 7f371d1d6640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3718163090 con 0x7f3718073960 2024-09-12T03:24:18.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.083+0000 7f371d1d6640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f37180d0da0 con 0x7f3718162a30 2024-09-12T03:24:18.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.083+0000 7f371759e640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f3718162a30 0x7f37181693f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:18.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.083+0000 7f3716d9d640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f371815e5f0 0x7f3718073250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:38934/0 (socket says 172.21.15.27:38934) 2024-09-12T03:24:18.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.083+0000 7f3716d9d640 1 -- 172.21.15.27:0/4025635068 learned_addr learned my addr 172.21.15.27:0/4025635068 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:24:18.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.083+0000 7f371759e640 1 -- 172.21.15.27:0/4025635068 >> v1:172.21.15.45:6789/0 conn(0x7f3718073960 legacy=0x7f3718162320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:24:18.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.083+0000 7f371759e640 1 -- 172.21.15.27:0/4025635068 >> v1:172.21.15.27:6789/0 conn(0x7f371815e5f0 legacy=0x7f3718073250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:18.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.083+0000 7f3715d9b640 1 -- 172.21.15.27:0/4025635068 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3532432735 0 0) 0x7f37180d2240 con 0x7f371815e5f0 2024-09-12T03:24:18.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.083+0000 7f371759e640 1 -- 172.21.15.27:0/4025635068 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37180d0470 con 0x7f3718162a30 2024-09-12T03:24:18.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.084+0000 7f371759e640 1 --2- 172.21.15.27:0/4025635068 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3718162a30 0x7f37181693f0 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7f37100097a0 tx=0x7f371002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1c5d0fcbd22f5bd server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:18.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.084+0000 7f3715d9b640 1 -- 172.21.15.27:0/4025635068 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3710037070 con 0x7f3718162a30 2024-09-12T03:24:18.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.084+0000 7f3715d9b640 1 -- 172.21.15.27:0/4025635068 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3710032440 con 0x7f3718162a30 2024-09-12T03:24:18.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.084+0000 7f3715d9b640 1 -- 172.21.15.27:0/4025635068 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3710032810 con 0x7f3718162a30 2024-09-12T03:24:18.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.085+0000 7f371d1d6640 1 -- 172.21.15.27:0/4025635068 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3718162a30 msgr2=0x7f37181693f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:18.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.085+0000 7f371d1d6640 1 --2- 172.21.15.27:0/4025635068 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3718162a30 0x7f37181693f0 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7f37100097a0 tx=0x7f371002a1e0 comp rx=0 tx=0).stop 2024-09-12T03:24:18.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.085+0000 7f371d1d6640 1 -- 172.21.15.27:0/4025635068 shutdown_connections 2024-09-12T03:24:18.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.085+0000 7f371d1d6640 1 --2- 172.21.15.27:0/4025635068 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3718162a30 0x7f37181693f0 unknown :-1 s=CLOSED pgs=1789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:18.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.085+0000 7f371d1d6640 1 -- 172.21.15.27:0/4025635068 >> 172.21.15.27:0/4025635068 conn(0x7f371806f300 msgr2=0x7f371806f710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:18.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.085+0000 7f371d1d6640 1 -- 172.21.15.27:0/4025635068 shutdown_connections 2024-09-12T03:24:18.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.085+0000 7f371d1d6640 1 -- 172.21.15.27:0/4025635068 wait complete. 2024-09-12T03:24:18.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.086+0000 7f371d1d6640 1 Processor -- start 2024-09-12T03:24:18.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.086+0000 7f371d1d6640 1 -- start start 2024-09-12T03:24:18.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.087+0000 7f371d1d6640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3718073960 0x7f3718142570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:18.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.087+0000 7f371d1d6640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f371815e5f0 0x7f3718155250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:18.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.087+0000 7f371d1d6640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f3718162a30 0x7f3718160f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:18.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.087+0000 7f371d1d6640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f37180d0a30 con 0x7f3718073960 2024-09-12T03:24:18.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.087+0000 7f371d1d6640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f3718162e40 con 0x7f371815e5f0 2024-09-12T03:24:18.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.087+0000 7f371d1d6640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f37180b5620 con 0x7f3718162a30 2024-09-12T03:24:18.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.087+0000 7f3716d9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3718073960 0x7f3718142570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:18.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.087+0000 7f3716d9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3718073960 0x7f3718142570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41712/0 (socket says 172.21.15.27:41712) 2024-09-12T03:24:18.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.087+0000 7f3716d9d640 1 -- 172.21.15.27:0/610112885 learned_addr learned my addr 172.21.15.27:0/610112885 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:24:18.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.087+0000 7f371759e640 1 --2- 172.21.15.27:0/610112885 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f3718162a30 0x7f3718160f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:18.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.087+0000 7f371659c640 1 --2- 172.21.15.27:0/610112885 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f371815e5f0 0x7f3718155250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:18.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.087+0000 7f3716d9d640 1 -- 172.21.15.27:0/610112885 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f3718162a30 msgr2=0x7f3718160f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:18.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.087+0000 7f3716d9d640 1 --2- 172.21.15.27:0/610112885 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f3718162a30 0x7f3718160f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:18.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.087+0000 7f3716d9d640 1 -- 172.21.15.27:0/610112885 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f371815e5f0 msgr2=0x7f3718155250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:18.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.087+0000 7f3716d9d640 1 --2- 172.21.15.27:0/610112885 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f371815e5f0 0x7f3718155250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:18.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.087+0000 7f3716d9d640 1 -- 172.21.15.27:0/610112885 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3718155cc0 con 0x7f3718073960 2024-09-12T03:24:18.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.087+0000 7f371659c640 1 --2- 172.21.15.27:0/610112885 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f371815e5f0 0x7f3718155250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:24:18.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.088+0000 7f3716d9d640 1 --2- 172.21.15.27:0/610112885 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3718073960 0x7f3718142570 secure :-1 s=READY pgs=1790 cs=0 l=1 rev1=1 crypto rx=0x7f370800cc10 tx=0x7f370800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:18.090 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.088+0000 7f370f7fe640 1 -- 172.21.15.27:0/610112885 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3708017070 con 0x7f3718073960 2024-09-12T03:24:18.090 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.088+0000 7f370f7fe640 1 -- 172.21.15.27:0/610112885 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f370800ce20 con 0x7f3718073960 2024-09-12T03:24:18.090 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.088+0000 7f371d1d6640 1 -- 172.21.15.27:0/610112885 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3718155fa0 con 0x7f3718073960 2024-09-12T03:24:18.090 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.088+0000 7f370f7fe640 1 -- 172.21.15.27:0/610112885 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3708012640 con 0x7f3718073960 2024-09-12T03:24:18.090 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.088+0000 7f371d1d6640 1 -- 172.21.15.27:0/610112885 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3718156560 con 0x7f3718073960 2024-09-12T03:24:18.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.089+0000 7f371d1d6640 1 -- 172.21.15.27:0/610112885 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37180b3d60 con 0x7f3718073960 2024-09-12T03:24:18.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.089+0000 7f370f7fe640 1 -- 172.21.15.27:0/610112885 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 26) ==== 91518+0+0 (secure 0 0 0) 0x7f370801a070 con 0x7f3718073960 2024-09-12T03:24:18.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.090+0000 7f370f7fe640 1 --2- 172.21.15.27:0/610112885 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f36f806c970 0x7f36f806ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:18.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.090+0000 7f370f7fe640 1 -- 172.21.15.27:0/610112885 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(163..163 src has 1..163) ==== 4999+0+0 (secure 0 0 0) 0x7f3708089890 con 0x7f3718073960 2024-09-12T03:24:18.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.090+0000 7f371659c640 1 --2- 172.21.15.27:0/610112885 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f36f806c970 0x7f36f806ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:18.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.090+0000 7f371659c640 1 --2- 172.21.15.27:0/610112885 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f36f806c970 0x7f36f806ee30 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f37181f7d40 tx=0x7f3700009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:18.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.094+0000 7f370f7fe640 1 -- 172.21.15.27:0/610112885 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f3708059aa0 con 0x7f3718073960 2024-09-12T03:24:18.270 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.270+0000 7f371d1d6640 1 -- 172.21.15.27:0/610112885 --> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) -- 0x7f371815ea70 con 0x7f36f806c970 2024-09-12T03:24:18.270 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.270+0000 7f370f7fe640 1 -- 172.21.15.27:0/610112885 <== mgr.4113 v2:172.21.15.27:6836/1226920526 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f371815ea70 con 0x7f36f806c970 2024-09-12T03:24:18.270 INFO:teuthology.orchestra.run.smithi027.stderr:instructed osd(s) 3 to deep-scrub 2024-09-12T03:24:18.272 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.272+0000 7f371d1d6640 1 -- 172.21.15.27:0/610112885 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f36f806c970 msgr2=0x7f36f806ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:18.272 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.272+0000 7f371d1d6640 1 --2- 172.21.15.27:0/610112885 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f36f806c970 0x7f36f806ee30 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f37181f7d40 tx=0x7f3700009040 comp rx=0 tx=0).stop 2024-09-12T03:24:18.272 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.272+0000 7f371d1d6640 1 -- 172.21.15.27:0/610112885 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3718073960 msgr2=0x7f3718142570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:18.272 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.272+0000 7f371d1d6640 1 --2- 172.21.15.27:0/610112885 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3718073960 0x7f3718142570 secure :-1 s=READY pgs=1790 cs=0 l=1 rev1=1 crypto rx=0x7f370800cc10 tx=0x7f370800a430 comp rx=0 tx=0).stop 2024-09-12T03:24:18.272 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.272+0000 7f371d1d6640 1 -- 172.21.15.27:0/610112885 shutdown_connections 2024-09-12T03:24:18.272 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.272+0000 7f371d1d6640 1 --2- 172.21.15.27:0/610112885 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f3718162a30 0x7f3718160f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:18.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.272+0000 7f371d1d6640 1 --2- 172.21.15.27:0/610112885 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f371815e5f0 0x7f3718155250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:18.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.272+0000 7f371d1d6640 1 --2- 172.21.15.27:0/610112885 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f36f806c970 0x7f36f806ee30 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:18.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.272+0000 7f371d1d6640 1 --2- 172.21.15.27:0/610112885 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3718073960 0x7f3718142570 unknown :-1 s=CLOSED pgs=1790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:18.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.272+0000 7f371d1d6640 1 -- 172.21.15.27:0/610112885 >> 172.21.15.27:0/610112885 conn(0x7f371806f300 msgr2=0x7f37180b2290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:18.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.272+0000 7f371d1d6640 1 -- 172.21.15.27:0/610112885 shutdown_connections 2024-09-12T03:24:18.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.272+0000 7f371d1d6640 1 -- 172.21.15.27:0/610112885 wait complete. 2024-09-12T03:24:18.282 INFO:tasks.ceph:Scrubbing osd.4 2024-09-12T03:24:18.283 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-09-12T03:24:18.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.391+0000 7f2a28d96640 1 Processor -- start 2024-09-12T03:24:18.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.391+0000 7f2a28d96640 1 -- start start 2024-09-12T03:24:18.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.392+0000 7f2a28d96640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2a2415e910 0x7f2a240aff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:18.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.392+0000 7f2a28d96640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f2a24072a00 0x7f2a240b4710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:18.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.392+0000 7f2a28d96640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2a2406dc20 con 0x7f2a2415e530 2024-09-12T03:24:18.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.392+0000 7f2a28d96640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f2a240b07e0 con 0x7f2a24072a00 2024-09-12T03:24:18.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.392+0000 7f2a28d96640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f2a240b4c50 con 0x7f2a2415e910 2024-09-12T03:24:18.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.392+0000 7f2a2259c640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f2a2415e530 0x7f2a240722f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:38940/0 (socket says 172.21.15.27:38940) 2024-09-12T03:24:18.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.392+0000 7f2a22d9d640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f2a24072a00 0x7f2a240b4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:18.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.392+0000 7f2a2259c640 1 -- 172.21.15.27:0/351717936 learned_addr learned my addr 172.21.15.27:0/351717936 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:24:18.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.393+0000 7f2a22d9d640 1 -- 172.21.15.27:0/351717936 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2a2415e910 msgr2=0x7f2a240aff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:24:18.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.393+0000 7f2a21d9b640 1 --2- 172.21.15.27:0/351717936 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2a2415e910 0x7f2a240aff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:18.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.393+0000 7f2a2159a640 1 -- 172.21.15.27:0/351717936 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3283112846 0 0) 0x7f2a2406dc20 con 0x7f2a2415e530 2024-09-12T03:24:18.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.393+0000 7f2a22d9d640 1 --2- 172.21.15.27:0/351717936 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2a2415e910 0x7f2a240aff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:18.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.393+0000 7f2a22d9d640 1 -- 172.21.15.27:0/351717936 >> v1:172.21.15.27:6789/0 conn(0x7f2a2415e530 legacy=0x7f2a240722f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:18.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.393+0000 7f2a22d9d640 1 -- 172.21.15.27:0/351717936 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a240b0500 con 0x7f2a24072a00 2024-09-12T03:24:18.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.393+0000 7f2a21d9b640 1 --2- 172.21.15.27:0/351717936 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2a2415e910 0x7f2a240aff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:24:18.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.393+0000 7f2a22d9d640 1 --2- 172.21.15.27:0/351717936 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2a24072a00 0x7f2a240b4710 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7f2a1c0097a0 tx=0x7f2a1c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbf953988bdc954c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:18.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.394+0000 7f2a2159a640 1 -- 172.21.15.27:0/351717936 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a1c037070 con 0x7f2a24072a00 2024-09-12T03:24:18.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.394+0000 7f2a2159a640 1 -- 172.21.15.27:0/351717936 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a1c032440 con 0x7f2a24072a00 2024-09-12T03:24:18.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.394+0000 7f2a2159a640 1 -- 172.21.15.27:0/351717936 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a1c032810 con 0x7f2a24072a00 2024-09-12T03:24:18.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.394+0000 7f2a28d96640 1 -- 172.21.15.27:0/351717936 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2a24072a00 msgr2=0x7f2a240b4710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:18.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.394+0000 7f2a28d96640 1 --2- 172.21.15.27:0/351717936 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2a24072a00 0x7f2a240b4710 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7f2a1c0097a0 tx=0x7f2a1c02a180 comp rx=0 tx=0).stop 2024-09-12T03:24:18.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.394+0000 7f2a28d96640 1 -- 172.21.15.27:0/351717936 shutdown_connections 2024-09-12T03:24:18.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.394+0000 7f2a28d96640 1 --2- 172.21.15.27:0/351717936 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2a24072a00 0x7f2a240b4710 unknown :-1 s=CLOSED pgs=1792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:18.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.394+0000 7f2a28d96640 1 --2- 172.21.15.27:0/351717936 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2a2415e910 0x7f2a240aff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:18.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.394+0000 7f2a28d96640 1 -- 172.21.15.27:0/351717936 >> 172.21.15.27:0/351717936 conn(0x7f2a2406bb30 msgr2=0x7f2a2406bf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:18.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.394+0000 7f2a28d96640 1 -- 172.21.15.27:0/351717936 shutdown_connections 2024-09-12T03:24:18.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.394+0000 7f2a28d96640 1 -- 172.21.15.27:0/351717936 wait complete. 2024-09-12T03:24:18.397 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.396+0000 7f2a28d96640 1 Processor -- start 2024-09-12T03:24:18.397 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.396+0000 7f2a28d96640 1 -- start start 2024-09-12T03:24:18.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.396+0000 7f2a28d96640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2a24072a00 0x7f2a24140dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:18.400 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.396+0000 7f2a28d96640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2a2415e530 0x7f2a240cffc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:18.400 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.396+0000 7f2a28d96640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2a2415e910 0x7f2a241437f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:18.400 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.396+0000 7f2a28d96640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f2a24071ff0 con 0x7f2a2415e530 2024-09-12T03:24:18.400 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.396+0000 7f2a28d96640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f2a2406e1a0 con 0x7f2a24072a00 2024-09-12T03:24:18.400 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.396+0000 7f2a28d96640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f2a240aeb80 con 0x7f2a2415e910 2024-09-12T03:24:18.400 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.397+0000 7f2a21d9b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2a2415e530 0x7f2a240cffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:18.401 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.397+0000 7f2a21d9b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2a2415e530 0x7f2a240cffc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41732/0 (socket says 172.21.15.27:41732) 2024-09-12T03:24:18.401 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.397+0000 7f2a21d9b640 1 -- 172.21.15.27:0/2245184850 learned_addr learned my addr 172.21.15.27:0/2245184850 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:24:18.401 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.397+0000 7f2a22d9d640 1 --2- 172.21.15.27:0/2245184850 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2a2415e910 0x7f2a241437f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:18.401 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.397+0000 7f2a2259c640 1 --2- 172.21.15.27:0/2245184850 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2a24072a00 0x7f2a24140dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:18.401 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.397+0000 7f2a21d9b640 1 -- 172.21.15.27:0/2245184850 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2a2415e910 msgr2=0x7f2a241437f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:18.401 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.397+0000 7f2a21d9b640 1 --2- 172.21.15.27:0/2245184850 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f2a2415e910 0x7f2a241437f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:18.401 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.397+0000 7f2a21d9b640 1 -- 172.21.15.27:0/2245184850 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2a24072a00 msgr2=0x7f2a24140dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:18.401 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.397+0000 7f2a21d9b640 1 --2- 172.21.15.27:0/2245184850 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2a24072a00 0x7f2a24140dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:18.401 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.397+0000 7f2a21d9b640 1 -- 172.21.15.27:0/2245184850 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a2415f1e0 con 0x7f2a2415e530 2024-09-12T03:24:18.401 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.397+0000 7f2a21d9b640 1 --2- 172.21.15.27:0/2245184850 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2a2415e530 0x7f2a240cffc0 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 crypto rx=0x7f2a1800c7f0 tx=0x7f2a1800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:18.401 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.397+0000 7f2a12ffd640 1 -- 172.21.15.27:0/2245184850 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a18015070 con 0x7f2a2415e530 2024-09-12T03:24:18.401 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.397+0000 7f2a12ffd640 1 -- 172.21.15.27:0/2245184850 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a18007820 con 0x7f2a2415e530 2024-09-12T03:24:18.401 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.397+0000 7f2a28d96640 1 -- 172.21.15.27:0/2245184850 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a24141900 con 0x7f2a2415e530 2024-09-12T03:24:18.402 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.397+0000 7f2a12ffd640 1 -- 172.21.15.27:0/2245184850 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a18007bd0 con 0x7f2a2415e530 2024-09-12T03:24:18.402 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.397+0000 7f2a28d96640 1 -- 172.21.15.27:0/2245184850 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a24141ec0 con 0x7f2a2415e530 2024-09-12T03:24:18.402 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.398+0000 7f2a28d96640 1 -- 172.21.15.27:0/2245184850 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f2a2415e090 con 0x7f2a2415e530 2024-09-12T03:24:18.402 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.399+0000 7f2a12ffd640 1 -- 172.21.15.27:0/2245184850 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 26) ==== 91518+0+0 (secure 0 0 0) 0x7f2a18018070 con 0x7f2a2415e530 2024-09-12T03:24:18.402 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.399+0000 7f2a12ffd640 1 --2- 172.21.15.27:0/2245184850 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f2a0406c970 0x7f2a0406ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:18.402 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.399+0000 7f2a12ffd640 1 -- 172.21.15.27:0/2245184850 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(163..163 src has 1..163) ==== 4999+0+0 (secure 0 0 0) 0x7f2a18089a40 con 0x7f2a2415e530 2024-09-12T03:24:18.402 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.399+0000 7f2a2259c640 1 --2- 172.21.15.27:0/2245184850 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f2a0406c970 0x7f2a0406ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:18.402 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.400+0000 7f2a12ffd640 1 --2- 172.21.15.27:0/2245184850 >> [v2:172.21.15.45:6816/3943189827,v1:172.21.15.45:6817/3943189827] conn(0x7f2a04072830 0x7f2a04074c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:18.402 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.400+0000 7f2a12ffd640 1 -- 172.21.15.27:0/2245184850 --> [v2:172.21.15.45:6816/3943189827,v1:172.21.15.45:6817/3943189827] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2a1801de50 con 0x7f2a04072830 2024-09-12T03:24:18.402 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.400+0000 7f2a12ffd640 1 -- 172.21.15.27:0/2245184850 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_get_version_reply(handle=1 version=163) ==== 24+0+0 (secure 0 0 0) 0x7f2a18093050 con 0x7f2a2415e530 2024-09-12T03:24:18.402 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.400+0000 7f2a2259c640 1 --2- 172.21.15.27:0/2245184850 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f2a0406c970 0x7f2a0406ee30 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f2a140012e0 tx=0x7f2a14001700 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:18.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.400+0000 7f2a22d9d640 1 --2- 172.21.15.27:0/2245184850 >> [v2:172.21.15.45:6816/3943189827,v1:172.21.15.45:6817/3943189827] conn(0x7f2a04072830 0x7f2a04074c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:18.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.400+0000 7f2a22d9d640 1 --2- 172.21.15.27:0/2245184850 >> [v2:172.21.15.45:6816/3943189827,v1:172.21.15.45:6817/3943189827] conn(0x7f2a04072830 0x7f2a04074c70 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:18.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.401+0000 7f2a12ffd640 1 -- 172.21.15.27:0/2245184850 <== osd.4 v2:172.21.15.45:6816/3943189827 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2a1801de50 con 0x7f2a04072830 2024-09-12T03:24:18.413 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.414+0000 7f2a28d96640 1 -- 172.21.15.27:0/2245184850 --> [v2:172.21.15.45:6816/3943189827,v1:172.21.15.45:6817/3943189827] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f2a240b4710 con 0x7f2a04072830 2024-09-12T03:24:18.414 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.414+0000 7f2a12ffd640 1 -- 172.21.15.27:0/2245184850 <== osd.4 v2:172.21.15.45:6816/3943189827 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f2a240b4710 con 0x7f2a04072830 2024-09-12T03:24:18.414 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-12T03:24:18.414 INFO:teuthology.orchestra.run.smithi027.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-12T03:24:18.414 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-12T03:24:18.415 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.414+0000 7f2a28d96640 1 -- 172.21.15.27:0/2245184850 >> [v2:172.21.15.45:6816/3943189827,v1:172.21.15.45:6817/3943189827] conn(0x7f2a04072830 msgr2=0x7f2a04074c70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:18.415 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.414+0000 7f2a28d96640 1 --2- 172.21.15.27:0/2245184850 >> [v2:172.21.15.45:6816/3943189827,v1:172.21.15.45:6817/3943189827] conn(0x7f2a04072830 0x7f2a04074c70 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:18.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.415+0000 7f2a28d96640 1 -- 172.21.15.27:0/2245184850 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f2a0406c970 msgr2=0x7f2a0406ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:18.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.415+0000 7f2a28d96640 1 --2- 172.21.15.27:0/2245184850 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f2a0406c970 0x7f2a0406ee30 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f2a140012e0 tx=0x7f2a14001700 comp rx=0 tx=0).stop 2024-09-12T03:24:18.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.415+0000 7f2a28d96640 1 -- 172.21.15.27:0/2245184850 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2a2415e530 msgr2=0x7f2a240cffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:18.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.415+0000 7f2a28d96640 1 --2- 172.21.15.27:0/2245184850 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2a2415e530 0x7f2a240cffc0 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 crypto rx=0x7f2a1800c7f0 tx=0x7f2a1800ccb0 comp rx=0 tx=0).stop 2024-09-12T03:24:18.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.415+0000 7f2a22d9d640 1 -- 172.21.15.27:0/2245184850 reap_dead start 2024-09-12T03:24:18.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.415+0000 7f2a28d96640 1 -- 172.21.15.27:0/2245184850 shutdown_connections 2024-09-12T03:24:18.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.415+0000 7f2a28d96640 1 -- 172.21.15.27:0/2245184850 >> 172.21.15.27:0/2245184850 conn(0x7f2a2406bb30 msgr2=0x7f2a240b0a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:18.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.415+0000 7f2a28d96640 1 -- 172.21.15.27:0/2245184850 shutdown_connections 2024-09-12T03:24:18.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.415+0000 7f2a28d96640 1 -- 172.21.15.27:0/2245184850 wait complete. 2024-09-12T03:24:18.425 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-09-12T03:24:18.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.527+0000 7f5048d31640 1 Processor -- start 2024-09-12T03:24:18.528 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.527+0000 7f5048d31640 1 -- start start 2024-09-12T03:24:18.529 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.528+0000 7f5048d31640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f504415eb90 0x7f50440d2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:18.529 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.528+0000 7f5048d31640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f50440d3760 con 0x7f50440d5200 2024-09-12T03:24:18.529 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.528+0000 7f5048d31640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f50440d3a40 con 0x7f50440d1720 2024-09-12T03:24:18.529 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.528+0000 7f5048d31640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f50440b7d20 con 0x7f504415eb90 2024-09-12T03:24:18.529 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.529+0000 7f504259c640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f504415eb90 0x7f50440d2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:18.529 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.529+0000 7f5041d9b640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f50440d5200 0x7f50440d1010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:38944/0 (socket says 172.21.15.27:38944) 2024-09-12T03:24:18.529 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.529+0000 7f5041d9b640 1 -- 172.21.15.27:0/3805573132 learned_addr learned my addr 172.21.15.27:0/3805573132 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:24:18.529 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.529+0000 7f504259c640 1 -- 172.21.15.27:0/3805573132 >> v1:172.21.15.45:6789/0 conn(0x7f50440d1720 legacy=0x7f50440b8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:24:18.529 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.529+0000 7f504259c640 1 -- 172.21.15.27:0/3805573132 >> v1:172.21.15.27:6789/0 conn(0x7f50440d5200 legacy=0x7f50440d1010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:18.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.529+0000 7f504259c640 1 -- 172.21.15.27:0/3805573132 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50440d3d20 con 0x7f504415eb90 2024-09-12T03:24:18.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.529+0000 7f504159a640 1 -- 172.21.15.27:0/3805573132 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2345581068 0 0) 0x7f50440d3760 con 0x7f50440d5200 2024-09-12T03:24:18.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.530+0000 7f504259c640 1 --2- 172.21.15.27:0/3805573132 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f504415eb90 0x7f50440d2df0 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7f50340097a0 tx=0x7f503402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6018adcae81286d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:18.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.530+0000 7f504159a640 1 -- 172.21.15.27:0/3805573132 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5034037070 con 0x7f504415eb90 2024-09-12T03:24:18.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.530+0000 7f504159a640 1 -- 172.21.15.27:0/3805573132 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5034032440 con 0x7f504415eb90 2024-09-12T03:24:18.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.530+0000 7f504159a640 1 -- 172.21.15.27:0/3805573132 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5034032810 con 0x7f504415eb90 2024-09-12T03:24:18.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.530+0000 7f5048d31640 1 -- 172.21.15.27:0/3805573132 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f504415eb90 msgr2=0x7f50440d2df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:18.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.530+0000 7f5048d31640 1 --2- 172.21.15.27:0/3805573132 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f504415eb90 0x7f50440d2df0 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7f50340097a0 tx=0x7f503402a1e0 comp rx=0 tx=0).stop 2024-09-12T03:24:18.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.530+0000 7f5048d31640 1 -- 172.21.15.27:0/3805573132 shutdown_connections 2024-09-12T03:24:18.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.530+0000 7f5048d31640 1 --2- 172.21.15.27:0/3805573132 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f504415eb90 0x7f50440d2df0 unknown :-1 s=CLOSED pgs=1795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:18.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.530+0000 7f5048d31640 1 -- 172.21.15.27:0/3805573132 >> 172.21.15.27:0/3805573132 conn(0x7f504406dce0 msgr2=0x7f504406e0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:18.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.530+0000 7f5048d31640 1 -- 172.21.15.27:0/3805573132 shutdown_connections 2024-09-12T03:24:18.532 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.531+0000 7f5048d31640 1 -- 172.21.15.27:0/3805573132 wait complete. 2024-09-12T03:24:18.532 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.532+0000 7f5048d31640 1 Processor -- start 2024-09-12T03:24:18.532 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.532+0000 7f5048d31640 1 -- start start 2024-09-12T03:24:18.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.533+0000 7f5048d31640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f50440d1720 0x7f50440c99a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:18.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.533+0000 7f5048d31640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f50440d5200 0x7f5044159d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:18.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.533+0000 7f5048d31640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f504415eb90 0x7f50441f5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:18.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.533+0000 7f5048d31640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f50440d3330 con 0x7f50440d1720 2024-09-12T03:24:18.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.533+0000 7f5048d31640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f50440d42e0 con 0x7f504415eb90 2024-09-12T03:24:18.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.533+0000 7f5048d31640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f50440d34b0 con 0x7f50440d5200 2024-09-12T03:24:18.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.533+0000 7f504259c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f50440d1720 0x7f50440c99a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:18.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.533+0000 7f504259c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f50440d1720 0x7f50440c99a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41750/0 (socket says 172.21.15.27:41750) 2024-09-12T03:24:18.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.533+0000 7f504259c640 1 -- 172.21.15.27:0/445692801 learned_addr learned my addr 172.21.15.27:0/445692801 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:24:18.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.533+0000 7f5041d9b640 1 --2- 172.21.15.27:0/445692801 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f50440d5200 0x7f5044159d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:18.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.533+0000 7f5042d9d640 1 --2- 172.21.15.27:0/445692801 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f504415eb90 0x7f50441f5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:18.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.533+0000 7f504259c640 1 -- 172.21.15.27:0/445692801 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f50440d5200 msgr2=0x7f5044159d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:18.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.533+0000 7f504259c640 1 --2- 172.21.15.27:0/445692801 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f50440d5200 0x7f5044159d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:18.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.533+0000 7f504259c640 1 -- 172.21.15.27:0/445692801 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f504415eb90 msgr2=0x7f50441f5500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:18.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.533+0000 7f504259c640 1 --2- 172.21.15.27:0/445692801 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f504415eb90 0x7f50441f5500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:18.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.533+0000 7f504259c640 1 -- 172.21.15.27:0/445692801 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50441f5a40 con 0x7f50440d1720 2024-09-12T03:24:18.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.533+0000 7f504259c640 1 --2- 172.21.15.27:0/445692801 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f50440d1720 0x7f50440c99a0 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7f50340097a0 tx=0x7f50340046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:18.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.534+0000 7f5032ffd640 1 -- 172.21.15.27:0/445692801 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5034041070 con 0x7f50440d1720 2024-09-12T03:24:18.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.534+0000 7f5032ffd640 1 -- 172.21.15.27:0/445692801 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f503403c870 con 0x7f50440d1720 2024-09-12T03:24:18.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.534+0000 7f5048d31640 1 -- 172.21.15.27:0/445692801 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5044158ce0 con 0x7f50440d1720 2024-09-12T03:24:18.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.534+0000 7f5032ffd640 1 -- 172.21.15.27:0/445692801 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5034037040 con 0x7f50440d1720 2024-09-12T03:24:18.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.534+0000 7f5048d31640 1 -- 172.21.15.27:0/445692801 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50441592a0 con 0x7f50440d1720 2024-09-12T03:24:18.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.534+0000 7f5048d31640 1 -- 172.21.15.27:0/445692801 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50440b9cd0 con 0x7f50440d1720 2024-09-12T03:24:18.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.535+0000 7f5032ffd640 1 -- 172.21.15.27:0/445692801 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 26) ==== 91518+0+0 (secure 0 0 0) 0x7f5034044070 con 0x7f50440d1720 2024-09-12T03:24:18.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.536+0000 7f5032ffd640 1 --2- 172.21.15.27:0/445692801 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f502406c970 0x7f502406ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:18.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.536+0000 7f5032ffd640 1 -- 172.21.15.27:0/445692801 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(163..163 src has 1..163) ==== 4999+0+0 (secure 0 0 0) 0x7f50340aa640 con 0x7f50440d1720 2024-09-12T03:24:18.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.536+0000 7f5041d9b640 1 --2- 172.21.15.27:0/445692801 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f502406c970 0x7f502406ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:18.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.536+0000 7f5041d9b640 1 --2- 172.21.15.27:0/445692801 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f502406c970 0x7f502406ee30 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f504415a420 tx=0x7f5038007f00 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:18.539 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.540+0000 7f5032ffd640 1 -- 172.21.15.27:0/445692801 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f50440b9cd0 con 0x7f50440d1720 2024-09-12T03:24:18.713 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.713+0000 7f5048d31640 1 -- 172.21.15.27:0/445692801 --> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) -- 0x7f50440d1d00 con 0x7f502406c970 2024-09-12T03:24:18.713 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.713+0000 7f5032ffd640 1 -- 172.21.15.27:0/445692801 <== mgr.4113 v2:172.21.15.27:6836/1226920526 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f50440d1d00 con 0x7f502406c970 2024-09-12T03:24:18.713 INFO:teuthology.orchestra.run.smithi027.stderr:instructed osd(s) 4 to deep-scrub 2024-09-12T03:24:18.715 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.715+0000 7f5048d31640 1 -- 172.21.15.27:0/445692801 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f502406c970 msgr2=0x7f502406ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:18.715 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.715+0000 7f5048d31640 1 --2- 172.21.15.27:0/445692801 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f502406c970 0x7f502406ee30 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f504415a420 tx=0x7f5038007f00 comp rx=0 tx=0).stop 2024-09-12T03:24:18.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.715+0000 7f5048d31640 1 -- 172.21.15.27:0/445692801 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f50440d1720 msgr2=0x7f50440c99a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:18.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.715+0000 7f5048d31640 1 --2- 172.21.15.27:0/445692801 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f50440d1720 0x7f50440c99a0 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7f50340097a0 tx=0x7f50340046c0 comp rx=0 tx=0).stop 2024-09-12T03:24:18.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.715+0000 7f5048d31640 1 -- 172.21.15.27:0/445692801 shutdown_connections 2024-09-12T03:24:18.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.715+0000 7f5048d31640 1 --2- 172.21.15.27:0/445692801 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f504415eb90 0x7f50441f5500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:18.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.715+0000 7f5048d31640 1 --2- 172.21.15.27:0/445692801 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f50440d5200 0x7f5044159d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:18.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.715+0000 7f5048d31640 1 --2- 172.21.15.27:0/445692801 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f502406c970 0x7f502406ee30 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:18.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.715+0000 7f5048d31640 1 --2- 172.21.15.27:0/445692801 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f50440d1720 0x7f50440c99a0 unknown :-1 s=CLOSED pgs=1796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:18.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.715+0000 7f5048d31640 1 -- 172.21.15.27:0/445692801 >> 172.21.15.27:0/445692801 conn(0x7f504406dce0 msgr2=0x7f50440cd8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:18.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.715+0000 7f5048d31640 1 -- 172.21.15.27:0/445692801 shutdown_connections 2024-09-12T03:24:18.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.715+0000 7f5048d31640 1 -- 172.21.15.27:0/445692801 wait complete. 2024-09-12T03:24:18.726 INFO:tasks.ceph:Scrubbing osd.5 2024-09-12T03:24:18.726 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-09-12T03:24:18.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.833+0000 7fe6eccba640 1 Processor -- start 2024-09-12T03:24:18.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.833+0000 7fe6eccba640 1 -- start start 2024-09-12T03:24:18.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.834+0000 7fe6eccba640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fe6e815e650 0x7fe6e80d1d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:18.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.834+0000 7fe6eccba640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fe6e806cc40 0x7fe6e806d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:18.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.834+0000 7fe6eccba640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe6e806d5d0 con 0x7fe6e806db30 2024-09-12T03:24:18.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.834+0000 7fe6eccba640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7fe6e80d2280 con 0x7fe6e815e650 2024-09-12T03:24:18.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.834+0000 7fe6eccba640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fe6e806e670 con 0x7fe6e806cc40 2024-09-12T03:24:18.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.834+0000 7fe6e659c640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fe6e815e650 0x7fe6e80d1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:18.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.834+0000 7fe6e6d9d640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fe6e806db30 0x7fe6e806df60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:38956/0 (socket says 172.21.15.27:38956) 2024-09-12T03:24:18.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.834+0000 7fe6e659c640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fe6e815e650 0x7fe6e80d1d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41762/0 (socket says 172.21.15.27:41762) 2024-09-12T03:24:18.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.834+0000 7fe6e6d9d640 1 -- 172.21.15.27:0/2565039076 learned_addr learned my addr 172.21.15.27:0/2565039076 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:24:18.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.834+0000 7fe6e5d9b640 1 --2- 172.21.15.27:0/2565039076 >> v2:172.21.15.45:3300/0 conn(0x7fe6e806cc40 0x7fe6e806d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:18.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.834+0000 7fe6e659c640 1 -- 172.21.15.27:0/2565039076 >> v2:172.21.15.45:3300/0 conn(0x7fe6e806cc40 msgr2=0x7fe6e806d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:24:18.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.834+0000 7fe6e659c640 1 --2- 172.21.15.27:0/2565039076 >> v2:172.21.15.45:3300/0 conn(0x7fe6e806cc40 0x7fe6e806d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:18.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.834+0000 7fe6e659c640 1 -- 172.21.15.27:0/2565039076 >> v1:172.21.15.27:6789/0 conn(0x7fe6e806db30 legacy=0x7fe6e806df60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:18.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.834+0000 7fe6e659c640 1 -- 172.21.15.27:0/2565039076 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6e806d850 con 0x7fe6e815e650 2024-09-12T03:24:18.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.834+0000 7fe6e559a640 1 -- 172.21.15.27:0/2565039076 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3203864158 0 0) 0x7fe6e0001490 con 0x7fe6e806db30 2024-09-12T03:24:18.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.835+0000 7fe6e659c640 1 --2- 172.21.15.27:0/2565039076 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe6e815e650 0x7fe6e80d1d40 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x7fe6e80b21c0 tx=0x7fe6d802a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d908bad0344542c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:18.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.835+0000 7fe6e559a640 1 -- 172.21.15.27:0/2565039076 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6d8037070 con 0x7fe6e815e650 2024-09-12T03:24:18.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.835+0000 7fe6e559a640 1 -- 172.21.15.27:0/2565039076 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6d8032440 con 0x7fe6e815e650 2024-09-12T03:24:18.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.835+0000 7fe6e559a640 1 -- 172.21.15.27:0/2565039076 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6d802ae60 con 0x7fe6e815e650 2024-09-12T03:24:18.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.836+0000 7fe6eccba640 1 -- 172.21.15.27:0/2565039076 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe6e815e650 msgr2=0x7fe6e80d1d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:18.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.836+0000 7fe6eccba640 1 --2- 172.21.15.27:0/2565039076 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe6e815e650 0x7fe6e80d1d40 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x7fe6e80b21c0 tx=0x7fe6d802a110 comp rx=0 tx=0).stop 2024-09-12T03:24:18.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.836+0000 7fe6eccba640 1 -- 172.21.15.27:0/2565039076 shutdown_connections 2024-09-12T03:24:18.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.836+0000 7fe6eccba640 1 --2- 172.21.15.27:0/2565039076 >> v2:172.21.15.45:3300/0 conn(0x7fe6e806cc40 0x7fe6e806d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:18.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.836+0000 7fe6eccba640 1 --2- 172.21.15.27:0/2565039076 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe6e815e650 0x7fe6e80d1d40 unknown :-1 s=CLOSED pgs=1798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:18.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.836+0000 7fe6eccba640 1 -- 172.21.15.27:0/2565039076 >> 172.21.15.27:0/2565039076 conn(0x7fe6e806f320 msgr2=0x7fe6e806f700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:18.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.836+0000 7fe6eccba640 1 -- 172.21.15.27:0/2565039076 shutdown_connections 2024-09-12T03:24:18.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.836+0000 7fe6eccba640 1 -- 172.21.15.27:0/2565039076 wait complete. 2024-09-12T03:24:18.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.837+0000 7fe6eccba640 1 Processor -- start 2024-09-12T03:24:18.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.837+0000 7fe6eccba640 1 -- start start 2024-09-12T03:24:18.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.838+0000 7fe6eccba640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe6e806cc40 0x7fe6e81540f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:18.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.838+0000 7fe6eccba640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe6e806db30 0x7fe6e81574c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:18.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.838+0000 7fe6eccba640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe6e815e650 0x7fe6e81f4520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:18.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.838+0000 7fe6eccba640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fe6e815ea30 con 0x7fe6e806db30 2024-09-12T03:24:18.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.838+0000 7fe6eccba640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe6e80d1540 con 0x7fe6e806cc40 2024-09-12T03:24:18.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.838+0000 7fe6eccba640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fe6e815ebb0 con 0x7fe6e815e650 2024-09-12T03:24:18.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.838+0000 7fe6e5d9b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe6e806db30 0x7fe6e81574c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:18.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.838+0000 7fe6e5d9b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe6e806db30 0x7fe6e81574c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41772/0 (socket says 172.21.15.27:41772) 2024-09-12T03:24:18.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.838+0000 7fe6e5d9b640 1 -- 172.21.15.27:0/3095402327 learned_addr learned my addr 172.21.15.27:0/3095402327 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:24:18.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.838+0000 7fe6e659c640 1 --2- 172.21.15.27:0/3095402327 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe6e806cc40 0x7fe6e81540f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:18.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.838+0000 7fe6e6d9d640 1 --2- 172.21.15.27:0/3095402327 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe6e815e650 0x7fe6e81f4520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:18.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.838+0000 7fe6e5d9b640 1 -- 172.21.15.27:0/3095402327 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe6e815e650 msgr2=0x7fe6e81f4520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:18.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.838+0000 7fe6e5d9b640 1 --2- 172.21.15.27:0/3095402327 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe6e815e650 0x7fe6e81f4520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:18.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.838+0000 7fe6e5d9b640 1 -- 172.21.15.27:0/3095402327 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe6e806cc40 msgr2=0x7fe6e81540f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:18.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.838+0000 7fe6e5d9b640 1 --2- 172.21.15.27:0/3095402327 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe6e806cc40 0x7fe6e81540f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:18.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.838+0000 7fe6e5d9b640 1 -- 172.21.15.27:0/3095402327 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6e815a0f0 con 0x7fe6e806db30 2024-09-12T03:24:18.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.838+0000 7fe6e6d9d640 1 --2- 172.21.15.27:0/3095402327 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fe6e815e650 0x7fe6e81f4520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:24:18.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.839+0000 7fe6e5d9b640 1 --2- 172.21.15.27:0/3095402327 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe6e806db30 0x7fe6e81574c0 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x7fe6dc00d5f0 tx=0x7fe6dc00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:18.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.839+0000 7fe6d6ffd640 1 -- 172.21.15.27:0/3095402327 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6dc0125f0 con 0x7fe6e806db30 2024-09-12T03:24:18.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.839+0000 7fe6d6ffd640 1 -- 172.21.15.27:0/3095402327 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6dc019070 con 0x7fe6e806db30 2024-09-12T03:24:18.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.839+0000 7fe6eccba640 1 -- 172.21.15.27:0/3095402327 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6e815a3d0 con 0x7fe6e806db30 2024-09-12T03:24:18.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.839+0000 7fe6d6ffd640 1 -- 172.21.15.27:0/3095402327 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6dc0129d0 con 0x7fe6e806db30 2024-09-12T03:24:18.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.839+0000 7fe6eccba640 1 -- 172.21.15.27:0/3095402327 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6e815a990 con 0x7fe6e806db30 2024-09-12T03:24:18.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.839+0000 7fe6eccba640 1 -- 172.21.15.27:0/3095402327 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe6e806d090 con 0x7fe6e806db30 2024-09-12T03:24:18.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.840+0000 7fe6d6ffd640 1 -- 172.21.15.27:0/3095402327 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 26) ==== 91518+0+0 (secure 0 0 0) 0x7fe6dc012b70 con 0x7fe6e806db30 2024-09-12T03:24:18.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.841+0000 7fe6d6ffd640 1 --2- 172.21.15.27:0/3095402327 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fe6c806c970 0x7fe6c806ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:18.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.841+0000 7fe6d6ffd640 1 -- 172.21.15.27:0/3095402327 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(163..163 src has 1..163) ==== 4999+0+0 (secure 0 0 0) 0x7fe6dc08a5c0 con 0x7fe6e806db30 2024-09-12T03:24:18.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.841+0000 7fe6e659c640 1 --2- 172.21.15.27:0/3095402327 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fe6c806c970 0x7fe6c806ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:18.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.841+0000 7fe6d6ffd640 1 --2- 172.21.15.27:0/3095402327 >> [v2:172.21.15.45:6808/3620760330,v1:172.21.15.45:6809/3620760330] conn(0x7fe6c8072830 0x7fe6c8074c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:18.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.841+0000 7fe6d6ffd640 1 -- 172.21.15.27:0/3095402327 --> [v2:172.21.15.45:6808/3620760330,v1:172.21.15.45:6809/3620760330] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe6dc00cfb0 con 0x7fe6c8072830 2024-09-12T03:24:18.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.841+0000 7fe6d6ffd640 1 -- 172.21.15.27:0/3095402327 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_get_version_reply(handle=1 version=163) ==== 24+0+0 (secure 0 0 0) 0x7fe6dc08d120 con 0x7fe6e806db30 2024-09-12T03:24:18.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.841+0000 7fe6e6d9d640 1 --2- 172.21.15.27:0/3095402327 >> [v2:172.21.15.45:6808/3620760330,v1:172.21.15.45:6809/3620760330] conn(0x7fe6c8072830 0x7fe6c8074c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:18.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.841+0000 7fe6e659c640 1 --2- 172.21.15.27:0/3095402327 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fe6c806c970 0x7fe6c806ee30 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fe6d8034000 tx=0x7fe6d802e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:18.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.842+0000 7fe6e6d9d640 1 --2- 172.21.15.27:0/3095402327 >> [v2:172.21.15.45:6808/3620760330,v1:172.21.15.45:6809/3620760330] conn(0x7fe6c8072830 0x7fe6c8074c70 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:18.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.842+0000 7fe6d6ffd640 1 -- 172.21.15.27:0/3095402327 <== osd.5 v2:172.21.15.45:6808/3620760330 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe6dc00cfb0 con 0x7fe6c8072830 2024-09-12T03:24:18.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.855+0000 7fe6eccba640 1 -- 172.21.15.27:0/3095402327 --> [v2:172.21.15.45:6808/3620760330,v1:172.21.15.45:6809/3620760330] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fe6e8157160 con 0x7fe6c8072830 2024-09-12T03:24:18.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.855+0000 7fe6d6ffd640 1 -- 172.21.15.27:0/3095402327 <== osd.5 v2:172.21.15.45:6808/3620760330 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fe6e8157160 con 0x7fe6c8072830 2024-09-12T03:24:18.857 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-12T03:24:18.857 INFO:teuthology.orchestra.run.smithi027.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-12T03:24:18.857 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-12T03:24:18.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.856+0000 7fe6eccba640 1 -- 172.21.15.27:0/3095402327 >> [v2:172.21.15.45:6808/3620760330,v1:172.21.15.45:6809/3620760330] conn(0x7fe6c8072830 msgr2=0x7fe6c8074c70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:18.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.856+0000 7fe6eccba640 1 --2- 172.21.15.27:0/3095402327 >> [v2:172.21.15.45:6808/3620760330,v1:172.21.15.45:6809/3620760330] conn(0x7fe6c8072830 0x7fe6c8074c70 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:18.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.856+0000 7fe6eccba640 1 -- 172.21.15.27:0/3095402327 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fe6c806c970 msgr2=0x7fe6c806ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:18.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.856+0000 7fe6eccba640 1 --2- 172.21.15.27:0/3095402327 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fe6c806c970 0x7fe6c806ee30 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fe6d8034000 tx=0x7fe6d802e000 comp rx=0 tx=0).stop 2024-09-12T03:24:18.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.856+0000 7fe6eccba640 1 -- 172.21.15.27:0/3095402327 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe6e806db30 msgr2=0x7fe6e81574c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:18.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.856+0000 7fe6eccba640 1 --2- 172.21.15.27:0/3095402327 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe6e806db30 0x7fe6e81574c0 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x7fe6dc00d5f0 tx=0x7fe6dc00dab0 comp rx=0 tx=0).stop 2024-09-12T03:24:18.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.856+0000 7fe6e6d9d640 1 -- 172.21.15.27:0/3095402327 reap_dead start 2024-09-12T03:24:18.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.856+0000 7fe6eccba640 1 -- 172.21.15.27:0/3095402327 shutdown_connections 2024-09-12T03:24:18.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.856+0000 7fe6eccba640 1 -- 172.21.15.27:0/3095402327 >> 172.21.15.27:0/3095402327 conn(0x7fe6e806f320 msgr2=0x7fe6e8160180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:18.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.856+0000 7fe6eccba640 1 -- 172.21.15.27:0/3095402327 shutdown_connections 2024-09-12T03:24:18.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.856+0000 7fe6eccba640 1 -- 172.21.15.27:0/3095402327 wait complete. 2024-09-12T03:24:18.866 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-09-12T03:24:18.970 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.970+0000 7f8b21e42640 1 Processor -- start 2024-09-12T03:24:18.970 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.970+0000 7f8b21e42640 1 -- start start 2024-09-12T03:24:18.971 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.971+0000 7f8b21e42640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f8b1c145c30 0x7f8b1c15ca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:18.971 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.971+0000 7f8b21e42640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f8b1c146010 0x7f8b1c0cd330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:18.971 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.971+0000 7f8b21e42640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f8b1c0cd870 0x7f8b1c0b4040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:18.971 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.971+0000 7f8b21e42640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f8b1c0b5050 con 0x7f8b1c0cd870 2024-09-12T03:24:18.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.971+0000 7f8b21e42640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f8b1c0b4d70 con 0x7f8b1c146010 2024-09-12T03:24:18.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.971+0000 7f8b21e42640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f8b1c06d140 con 0x7f8b1c145c30 2024-09-12T03:24:18.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.971+0000 7f8b1bfff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f8b1c0cd870 0x7f8b1c0b4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:18.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.971+0000 7f8b1bfff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f8b1c0cd870 0x7f8b1c0b4040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41782/0 (socket says 172.21.15.27:41782) 2024-09-12T03:24:18.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.971+0000 7f8b1bfff640 1 -- 172.21.15.27:0/1446383500 learned_addr learned my addr 172.21.15.27:0/1446383500 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:24:18.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.971+0000 7f8b1b7fe640 1 --2- 172.21.15.27:0/1446383500 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f8b1c145c30 0x7f8b1c15ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:18.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.971+0000 7f8b13fff640 1 --2- 172.21.15.27:0/1446383500 >> v2:172.21.15.45:3300/0 conn(0x7f8b1c146010 0x7f8b1c0cd330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:18.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.971+0000 7f8b1bfff640 1 -- 172.21.15.27:0/1446383500 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f8b1c145c30 msgr2=0x7f8b1c15ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:24:18.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.971+0000 7f8b1bfff640 1 --2- 172.21.15.27:0/1446383500 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f8b1c145c30 0x7f8b1c15ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:18.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.971+0000 7f8b1bfff640 1 -- 172.21.15.27:0/1446383500 >> v2:172.21.15.45:3300/0 conn(0x7f8b1c146010 msgr2=0x7f8b1c0cd330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:24:18.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.971+0000 7f8b1bfff640 1 --2- 172.21.15.27:0/1446383500 >> v2:172.21.15.45:3300/0 conn(0x7f8b1c146010 0x7f8b1c0cd330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:18.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.971+0000 7f8b1bfff640 1 -- 172.21.15.27:0/1446383500 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b1c06d420 con 0x7f8b1c0cd870 2024-09-12T03:24:18.973 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.971+0000 7f8b1b7fe640 1 --2- 172.21.15.27:0/1446383500 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f8b1c145c30 0x7f8b1c15ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:24:18.973 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.971+0000 7f8b13fff640 1 --2- 172.21.15.27:0/1446383500 >> v2:172.21.15.45:3300/0 conn(0x7f8b1c146010 0x7f8b1c0cd330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:24:18.974 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.972+0000 7f8b1bfff640 1 --2- 172.21.15.27:0/1446383500 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8b1c0cd870 0x7f8b1c0b4040 secure :-1 s=READY pgs=1800 cs=0 l=1 rev1=1 crypto rx=0x7f8b1c06d350 tx=0x7f8b1402a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b9bf80c0b059e81 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:18.974 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.972+0000 7f8b1affd640 1 -- 172.21.15.27:0/1446383500 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b14036070 con 0x7f8b1c0cd870 2024-09-12T03:24:18.974 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.972+0000 7f8b1affd640 1 -- 172.21.15.27:0/1446383500 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b14031600 con 0x7f8b1c0cd870 2024-09-12T03:24:18.974 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.972+0000 7f8b1affd640 1 -- 172.21.15.27:0/1446383500 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b140319d0 con 0x7f8b1c0cd870 2024-09-12T03:24:18.974 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.973+0000 7f8b21e42640 1 -- 172.21.15.27:0/1446383500 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8b1c0cd870 msgr2=0x7f8b1c0b4040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:18.975 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.973+0000 7f8b21e42640 1 --2- 172.21.15.27:0/1446383500 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8b1c0cd870 0x7f8b1c0b4040 secure :-1 s=READY pgs=1800 cs=0 l=1 rev1=1 crypto rx=0x7f8b1c06d350 tx=0x7f8b1402a5f0 comp rx=0 tx=0).stop 2024-09-12T03:24:18.975 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.973+0000 7f8b21e42640 1 -- 172.21.15.27:0/1446383500 shutdown_connections 2024-09-12T03:24:18.975 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.973+0000 7f8b21e42640 1 --2- 172.21.15.27:0/1446383500 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8b1c0cd870 0x7f8b1c0b4040 unknown :-1 s=CLOSED pgs=1800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:18.975 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.973+0000 7f8b21e42640 1 --2- 172.21.15.27:0/1446383500 >> v2:172.21.15.45:3300/0 conn(0x7f8b1c146010 0x7f8b1c0cd330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:18.975 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.973+0000 7f8b21e42640 1 --2- 172.21.15.27:0/1446383500 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f8b1c145c30 0x7f8b1c15ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:18.975 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.973+0000 7f8b21e42640 1 -- 172.21.15.27:0/1446383500 >> 172.21.15.27:0/1446383500 conn(0x7f8b1c062c60 msgr2=0x7f8b1c06bb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:18.975 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.973+0000 7f8b21e42640 1 -- 172.21.15.27:0/1446383500 shutdown_connections 2024-09-12T03:24:18.975 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.973+0000 7f8b21e42640 1 -- 172.21.15.27:0/1446383500 wait complete. 2024-09-12T03:24:18.975 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.974+0000 7f8b21e42640 1 Processor -- start 2024-09-12T03:24:18.975 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.974+0000 7f8b21e42640 1 -- start start 2024-09-12T03:24:18.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.975+0000 7f8b21e42640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8b1c0cd870 0x7f8b1c15bea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:18.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.975+0000 7f8b21e42640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f8b1c145c30 0x7f8b1c0c99c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:18.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.975+0000 7f8b21e42640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8b1c146010 0x7f8b1c0af000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:18.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.975+0000 7f8b21e42640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f8b1c0b4700 con 0x7f8b1c146010 2024-09-12T03:24:18.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.975+0000 7f8b21e42640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f8b1c0b4580 con 0x7f8b1c0cd870 2024-09-12T03:24:18.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.975+0000 7f8b21e42640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f8b1c0b4880 con 0x7f8b1c145c30 2024-09-12T03:24:18.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.975+0000 7f8b1bfff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8b1c146010 0x7f8b1c0af000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:18.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.975+0000 7f8b1bfff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8b1c146010 0x7f8b1c0af000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41790/0 (socket says 172.21.15.27:41790) 2024-09-12T03:24:18.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.975+0000 7f8b1bfff640 1 -- 172.21.15.27:0/4249927319 learned_addr learned my addr 172.21.15.27:0/4249927319 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:24:18.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.975+0000 7f8b1b7fe640 1 --2- 172.21.15.27:0/4249927319 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8b1c0cd870 0x7f8b1c15bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:18.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.975+0000 7f8b13fff640 1 --2- 172.21.15.27:0/4249927319 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f8b1c145c30 0x7f8b1c0c99c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:18.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.975+0000 7f8b1bfff640 1 -- 172.21.15.27:0/4249927319 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f8b1c145c30 msgr2=0x7f8b1c0c99c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:18.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.975+0000 7f8b1bfff640 1 --2- 172.21.15.27:0/4249927319 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f8b1c145c30 0x7f8b1c0c99c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:18.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.975+0000 7f8b1bfff640 1 -- 172.21.15.27:0/4249927319 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8b1c0cd870 msgr2=0x7f8b1c15bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:18.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.975+0000 7f8b1bfff640 1 --2- 172.21.15.27:0/4249927319 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8b1c0cd870 0x7f8b1c15bea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:18.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.975+0000 7f8b1bfff640 1 -- 172.21.15.27:0/4249927319 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b1c0cc610 con 0x7f8b1c146010 2024-09-12T03:24:18.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.975+0000 7f8b1bfff640 1 --2- 172.21.15.27:0/4249927319 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8b1c146010 0x7f8b1c0af000 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7f8b14009f10 tx=0x7f8b14004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:18.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.976+0000 7f8b18ff9640 1 -- 172.21.15.27:0/4249927319 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b14036050 con 0x7f8b1c146010 2024-09-12T03:24:18.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.976+0000 7f8b18ff9640 1 -- 172.21.15.27:0/4249927319 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b14042070 con 0x7f8b1c146010 2024-09-12T03:24:18.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.976+0000 7f8b21e42640 1 -- 172.21.15.27:0/4249927319 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b1c0cc8f0 con 0x7f8b1c146010 2024-09-12T03:24:18.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.976+0000 7f8b18ff9640 1 -- 172.21.15.27:0/4249927319 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b14009510 con 0x7f8b1c146010 2024-09-12T03:24:18.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.976+0000 7f8b21e42640 1 -- 172.21.15.27:0/4249927319 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b1c15c3e0 con 0x7f8b1c146010 2024-09-12T03:24:18.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.977+0000 7f8b21e42640 1 -- 172.21.15.27:0/4249927319 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b1c15ca00 con 0x7f8b1c146010 2024-09-12T03:24:18.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.977+0000 7f8b18ff9640 1 -- 172.21.15.27:0/4249927319 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 26) ==== 91518+0+0 (secure 0 0 0) 0x7f8b14007bd0 con 0x7f8b1c146010 2024-09-12T03:24:18.980 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.978+0000 7f8b18ff9640 1 --2- 172.21.15.27:0/4249927319 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f8afc06c970 0x7f8afc06ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:18.980 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.978+0000 7f8b1b7fe640 1 --2- 172.21.15.27:0/4249927319 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f8afc06c970 0x7f8afc06ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:18.980 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.978+0000 7f8b18ff9640 1 -- 172.21.15.27:0/4249927319 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(163..163 src has 1..163) ==== 4999+0+0 (secure 0 0 0) 0x7f8b140aa600 con 0x7f8b1c146010 2024-09-12T03:24:18.980 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.978+0000 7f8b1b7fe640 1 --2- 172.21.15.27:0/4249927319 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f8afc06c970 0x7f8afc06ee30 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f8b0c00ab60 tx=0x7f8b0c008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:18.982 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:18.982+0000 7f8b18ff9640 1 -- 172.21.15.27:0/4249927319 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f8b1407a410 con 0x7f8b1c146010 2024-09-12T03:24:19.157 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.157+0000 7f8b21e42640 1 -- 172.21.15.27:0/4249927319 --> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) -- 0x7f8b1c0cd010 con 0x7f8afc06c970 2024-09-12T03:24:19.157 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.157+0000 7f8b18ff9640 1 -- 172.21.15.27:0/4249927319 <== mgr.4113 v2:172.21.15.27:6836/1226920526 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f8b1c0cd010 con 0x7f8afc06c970 2024-09-12T03:24:19.157 INFO:teuthology.orchestra.run.smithi027.stderr:instructed osd(s) 5 to deep-scrub 2024-09-12T03:24:19.159 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.159+0000 7f8b21e42640 1 -- 172.21.15.27:0/4249927319 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f8afc06c970 msgr2=0x7f8afc06ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:19.159 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.159+0000 7f8b21e42640 1 --2- 172.21.15.27:0/4249927319 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f8afc06c970 0x7f8afc06ee30 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f8b0c00ab60 tx=0x7f8b0c008040 comp rx=0 tx=0).stop 2024-09-12T03:24:19.159 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.159+0000 7f8b21e42640 1 -- 172.21.15.27:0/4249927319 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8b1c146010 msgr2=0x7f8b1c0af000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:19.159 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.159+0000 7f8b21e42640 1 --2- 172.21.15.27:0/4249927319 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8b1c146010 0x7f8b1c0af000 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7f8b14009f10 tx=0x7f8b14004a40 comp rx=0 tx=0).stop 2024-09-12T03:24:19.159 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.159+0000 7f8b21e42640 1 -- 172.21.15.27:0/4249927319 shutdown_connections 2024-09-12T03:24:19.159 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.159+0000 7f8b21e42640 1 --2- 172.21.15.27:0/4249927319 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f8afc06c970 0x7f8afc06ee30 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:19.159 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.159+0000 7f8b21e42640 1 --2- 172.21.15.27:0/4249927319 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8b1c146010 0x7f8b1c0af000 unknown :-1 s=CLOSED pgs=1801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:19.159 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.159+0000 7f8b21e42640 1 --2- 172.21.15.27:0/4249927319 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f8b1c145c30 0x7f8b1c0c99c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:19.160 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.159+0000 7f8b21e42640 1 --2- 172.21.15.27:0/4249927319 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8b1c0cd870 0x7f8b1c15bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:19.160 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.159+0000 7f8b21e42640 1 -- 172.21.15.27:0/4249927319 >> 172.21.15.27:0/4249927319 conn(0x7f8b1c062c60 msgr2=0x7f8b1c0aa740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:19.160 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.159+0000 7f8b21e42640 1 -- 172.21.15.27:0/4249927319 shutdown_connections 2024-09-12T03:24:19.160 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.159+0000 7f8b21e42640 1 -- 172.21.15.27:0/4249927319 wait complete. 2024-09-12T03:24:19.170 INFO:tasks.ceph:Scrubbing osd.6 2024-09-12T03:24:19.170 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-09-12T03:24:19.277 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.277+0000 7f662cee4640 1 Processor -- start 2024-09-12T03:24:19.277 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.277+0000 7f662cee4640 1 -- start start 2024-09-12T03:24:19.280 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.278+0000 7f662cee4640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f6628073ef0 0x7f66280b4240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:19.280 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.278+0000 7f662cee4640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f66280b4940 con 0x7f66280cef90 2024-09-12T03:24:19.281 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.278+0000 7f662cee4640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f66280b4c20 con 0x7f662815c670 2024-09-12T03:24:19.281 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.278+0000 7f662cee4640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f66280cff10 con 0x7f6628073ef0 2024-09-12T03:24:19.281 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.278+0000 7f6626d9d640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f66280cef90 0x7f66280b64a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:38966/0 (socket says 172.21.15.27:38966) 2024-09-12T03:24:19.281 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.278+0000 7f6626d9d640 1 -- 172.21.15.27:0/3802742739 learned_addr learned my addr 172.21.15.27:0/3802742739 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:24:19.281 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.279+0000 7f6625d9b640 1 --2- 172.21.15.27:0/3802742739 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f6628073ef0 0x7f66280b4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:19.281 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.279+0000 7f662559a640 1 -- 172.21.15.27:0/3802742739 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 944083784 0 0) 0x7f66280b4940 con 0x7f66280cef90 2024-09-12T03:24:19.281 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.279+0000 7f662559a640 1 -- 172.21.15.27:0/3802742739 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6610003520 con 0x7f66280cef90 2024-09-12T03:24:19.281 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.279+0000 7f662559a640 1 -- 172.21.15.27:0/3802742739 <== mon.1 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 967602406 0 0) 0x7f66280b4c20 con 0x7f662815c670 2024-09-12T03:24:19.281 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.279+0000 7f662559a640 1 -- 172.21.15.27:0/3802742739 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f66280b4940 con 0x7f662815c670 2024-09-12T03:24:19.281 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.279+0000 7f662559a640 1 -- 172.21.15.27:0/3802742739 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1393511852 0 0) 0x7f6610003520 con 0x7f66280cef90 2024-09-12T03:24:19.281 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.279+0000 7f662559a640 1 -- 172.21.15.27:0/3802742739 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f6628073ef0 msgr2=0x7f66280b4240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:24:19.281 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.279+0000 7f662559a640 1 --2- 172.21.15.27:0/3802742739 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f6628073ef0 0x7f66280b4240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:19.281 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.279+0000 7f662559a640 1 -- 172.21.15.27:0/3802742739 >> v1:172.21.15.45:6789/0 conn(0x7f662815c670 legacy=0x7f66281607c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:19.282 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.279+0000 7f662559a640 1 -- 172.21.15.27:0/3802742739 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66280b4f00 con 0x7f66280cef90 2024-09-12T03:24:19.282 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.279+0000 7f662559a640 1 -- 172.21.15.27:0/3802742739 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 847755213 0 0) 0x7f66200019f0 con 0x7f66280cef90 2024-09-12T03:24:19.282 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.279+0000 7f662559a640 1 -- 172.21.15.27:0/3802742739 >> v1:172.21.15.27:6789/0 conn(0x7f66280cef90 legacy=0x7f66280b64a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:19.282 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.279+0000 7f662559a640 1 --2- 172.21.15.27:0/3802742739 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f6628073ef0 0x7f6610003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:19.282 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.279+0000 7f662559a640 1 --2- 172.21.15.27:0/3802742739 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f66100040d0 0x7f6610004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:19.282 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.279+0000 7f662559a640 1 --2- 172.21.15.27:0/3802742739 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6610004d40 0x7f66100051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:19.282 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.279+0000 7f662559a640 1 -- 172.21.15.27:0/3802742739 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f661000ba70 con 0x7f6610004d40 2024-09-12T03:24:19.282 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.279+0000 7f662559a640 1 -- 172.21.15.27:0/3802742739 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f661000bc20 con 0x7f66100040d0 2024-09-12T03:24:19.282 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.279+0000 7f662559a640 1 -- 172.21.15.27:0/3802742739 --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f661000be20 con 0x7f6628073ef0 2024-09-12T03:24:19.282 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.279+0000 7f6626d9d640 1 --2- 172.21.15.27:0/3802742739 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6610004d40 0x7f66100051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:19.282 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.279+0000 7f6625d9b640 1 --2- 172.21.15.27:0/3802742739 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f66100040d0 0x7f6610004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:19.282 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.279+0000 7f662659c640 1 --2- 172.21.15.27:0/3802742739 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f6628073ef0 0x7f6610003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:19.282 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.280+0000 7f6626d9d640 1 -- 172.21.15.27:0/3802742739 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f6628073ef0 msgr2=0x7f6610003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:24:19.283 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.280+0000 7f6626d9d640 1 --2- 172.21.15.27:0/3802742739 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f6628073ef0 0x7f6610003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:19.283 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.280+0000 7f6626d9d640 1 -- 172.21.15.27:0/3802742739 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f66100040d0 msgr2=0x7f6610004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:24:19.283 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.280+0000 7f6626d9d640 1 --2- 172.21.15.27:0/3802742739 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f66100040d0 0x7f6610004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:19.283 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.280+0000 7f6626d9d640 1 -- 172.21.15.27:0/3802742739 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6610003520 con 0x7f6610004d40 2024-09-12T03:24:19.283 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.280+0000 7f6625d9b640 1 --2- 172.21.15.27:0/3802742739 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f66100040d0 0x7f6610004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:24:19.283 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.280+0000 7f6626d9d640 1 --2- 172.21.15.27:0/3802742739 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6610004d40 0x7f66100051c0 secure :-1 s=READY pgs=1803 cs=0 l=1 rev1=1 crypto rx=0x7f662000a7f0 tx=0x7f6610005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c04339d0fa07bd78 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:19.283 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.280+0000 7f662559a640 1 -- 172.21.15.27:0/3802742739 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f662000a410 con 0x7f6610004d40 2024-09-12T03:24:19.283 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.281+0000 7f662559a640 1 -- 172.21.15.27:0/3802742739 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f662002f070 con 0x7f6610004d40 2024-09-12T03:24:19.283 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.281+0000 7f662559a640 1 -- 172.21.15.27:0/3802742739 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f662002bc70 con 0x7f6610004d40 2024-09-12T03:24:19.287 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.281+0000 7f662cee4640 1 -- 172.21.15.27:0/3802742739 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6610004d40 msgr2=0x7f66100051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:19.287 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.281+0000 7f662cee4640 1 --2- 172.21.15.27:0/3802742739 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6610004d40 0x7f66100051c0 secure :-1 s=READY pgs=1803 cs=0 l=1 rev1=1 crypto rx=0x7f662000a7f0 tx=0x7f6610005830 comp rx=0 tx=0).stop 2024-09-12T03:24:19.287 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.281+0000 7f6626d9d640 1 -- 172.21.15.27:0/3802742739 reap_dead start 2024-09-12T03:24:19.287 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.281+0000 7f662cee4640 1 -- 172.21.15.27:0/3802742739 shutdown_connections 2024-09-12T03:24:19.287 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.281+0000 7f662cee4640 1 -- 172.21.15.27:0/3802742739 >> 172.21.15.27:0/3802742739 conn(0x7f662806f320 msgr2=0x7f662806f700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:19.287 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.281+0000 7f662cee4640 1 -- 172.21.15.27:0/3802742739 shutdown_connections 2024-09-12T03:24:19.287 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.281+0000 7f662cee4640 1 -- 172.21.15.27:0/3802742739 wait complete. 2024-09-12T03:24:19.287 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.283+0000 7f662cee4640 1 Processor -- start 2024-09-12T03:24:19.287 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.283+0000 7f662cee4640 1 -- start start 2024-09-12T03:24:19.287 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.283+0000 7f662cee4640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6610004d40 0x7f662814d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:19.287 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.283+0000 7f662cee4640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f662815a100 0x7f66280ce330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:19.287 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.283+0000 7f662cee4640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f662815a580 0x7f66280ce870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:19.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.283+0000 7f662cee4640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f6628160ed0 con 0x7f6610004d40 2024-09-12T03:24:19.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.283+0000 7f662cee4640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f6628160050 con 0x7f662815a580 2024-09-12T03:24:19.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.283+0000 7f662cee4640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f66280b6bb0 con 0x7f662815a100 2024-09-12T03:24:19.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.283+0000 7f662659c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6610004d40 0x7f662814d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:19.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.283+0000 7f662659c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6610004d40 0x7f662814d3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:48916/0 (socket says 172.21.15.27:48916) 2024-09-12T03:24:19.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.283+0000 7f662659c640 1 -- 172.21.15.27:0/510415840 learned_addr learned my addr 172.21.15.27:0/510415840 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:24:19.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.284+0000 7f6625d9b640 1 --2- 172.21.15.27:0/510415840 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f662815a100 0x7f66280ce330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:19.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.284+0000 7f6626d9d640 1 --2- 172.21.15.27:0/510415840 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f662815a580 0x7f66280ce870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:19.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.284+0000 7f662659c640 1 -- 172.21.15.27:0/510415840 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f662815a100 msgr2=0x7f66280ce330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:19.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.284+0000 7f662659c640 1 --2- 172.21.15.27:0/510415840 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f662815a100 0x7f66280ce330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:19.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.284+0000 7f662659c640 1 -- 172.21.15.27:0/510415840 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f662815a580 msgr2=0x7f66280ce870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:19.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.284+0000 7f662659c640 1 --2- 172.21.15.27:0/510415840 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f662815a580 0x7f66280ce870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:19.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.284+0000 7f662659c640 1 -- 172.21.15.27:0/510415840 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f662806cd70 con 0x7f6610004d40 2024-09-12T03:24:19.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.284+0000 7f6626d9d640 1 --2- 172.21.15.27:0/510415840 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f662815a580 0x7f66280ce870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:24:19.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.284+0000 7f662659c640 1 --2- 172.21.15.27:0/510415840 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6610004d40 0x7f662814d3f0 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7f661800f770 tx=0x7f661800fb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:19.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.284+0000 7f6616ffd640 1 -- 172.21.15.27:0/510415840 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66180050d0 con 0x7f6610004d40 2024-09-12T03:24:19.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.284+0000 7f6616ffd640 1 -- 172.21.15.27:0/510415840 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6618018070 con 0x7f6610004d40 2024-09-12T03:24:19.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.284+0000 7f662cee4640 1 -- 172.21.15.27:0/510415840 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f662806d010 con 0x7f6610004d40 2024-09-12T03:24:19.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.284+0000 7f6616ffd640 1 -- 172.21.15.27:0/510415840 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f661800d6d0 con 0x7f6610004d40 2024-09-12T03:24:19.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.284+0000 7f662cee4640 1 -- 172.21.15.27:0/510415840 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f662806d5d0 con 0x7f6610004d40 2024-09-12T03:24:19.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.285+0000 7f662cee4640 1 -- 172.21.15.27:0/510415840 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f66280b4240 con 0x7f6610004d40 2024-09-12T03:24:19.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.286+0000 7f6616ffd640 1 -- 172.21.15.27:0/510415840 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 26) ==== 91518+0+0 (secure 0 0 0) 0x7f661800d870 con 0x7f6610004d40 2024-09-12T03:24:19.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.286+0000 7f6616ffd640 1 --2- 172.21.15.27:0/510415840 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f660806c9a0 0x7f660806ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:19.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.286+0000 7f6616ffd640 1 -- 172.21.15.27:0/510415840 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(163..163 src has 1..163) ==== 4999+0+0 (secure 0 0 0) 0x7f6618089670 con 0x7f6610004d40 2024-09-12T03:24:19.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.286+0000 7f6625d9b640 1 --2- 172.21.15.27:0/510415840 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f660806c9a0 0x7f660806ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:19.290 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.287+0000 7f6616ffd640 1 --2- 172.21.15.27:0/510415840 >> [v2:172.21.15.45:6800/1989083510,v1:172.21.15.45:6801/1989083510] conn(0x7f6608072860 0x7f6608074ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:19.290 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.287+0000 7f6616ffd640 1 -- 172.21.15.27:0/510415840 --> [v2:172.21.15.45:6800/1989083510,v1:172.21.15.45:6801/1989083510] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6618009e50 con 0x7f6608072860 2024-09-12T03:24:19.290 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.287+0000 7f6616ffd640 1 -- 172.21.15.27:0/510415840 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_get_version_reply(handle=1 version=163) ==== 24+0+0 (secure 0 0 0) 0x7f6618091430 con 0x7f6610004d40 2024-09-12T03:24:19.290 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.287+0000 7f6625d9b640 1 --2- 172.21.15.27:0/510415840 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f660806c9a0 0x7f660806ee60 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f661c009d60 tx=0x7f661c009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:19.290 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.287+0000 7f6626d9d640 1 --2- 172.21.15.27:0/510415840 >> [v2:172.21.15.45:6800/1989083510,v1:172.21.15.45:6801/1989083510] conn(0x7f6608072860 0x7f6608074ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:19.290 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.287+0000 7f6626d9d640 1 --2- 172.21.15.27:0/510415840 >> [v2:172.21.15.45:6800/1989083510,v1:172.21.15.45:6801/1989083510] conn(0x7f6608072860 0x7f6608074ca0 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:19.290 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.288+0000 7f6616ffd640 1 -- 172.21.15.27:0/510415840 <== osd.6 v2:172.21.15.45:6800/1989083510 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6618009e50 con 0x7f6608072860 2024-09-12T03:24:19.300 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.301+0000 7f662cee4640 1 -- 172.21.15.27:0/510415840 --> [v2:172.21.15.45:6800/1989083510,v1:172.21.15.45:6801/1989083510] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f662806f700 con 0x7f6608072860 2024-09-12T03:24:19.301 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.301+0000 7f6616ffd640 1 -- 172.21.15.27:0/510415840 <== osd.6 v2:172.21.15.45:6800/1989083510 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f662806f700 con 0x7f6608072860 2024-09-12T03:24:19.301 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-12T03:24:19.302 INFO:teuthology.orchestra.run.smithi027.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-12T03:24:19.302 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-12T03:24:19.303 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.302+0000 7f662cee4640 1 -- 172.21.15.27:0/510415840 >> [v2:172.21.15.45:6800/1989083510,v1:172.21.15.45:6801/1989083510] conn(0x7f6608072860 msgr2=0x7f6608074ca0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:19.303 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.302+0000 7f662cee4640 1 --2- 172.21.15.27:0/510415840 >> [v2:172.21.15.45:6800/1989083510,v1:172.21.15.45:6801/1989083510] conn(0x7f6608072860 0x7f6608074ca0 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:19.303 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.302+0000 7f662cee4640 1 -- 172.21.15.27:0/510415840 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f660806c9a0 msgr2=0x7f660806ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:19.303 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.302+0000 7f662cee4640 1 --2- 172.21.15.27:0/510415840 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f660806c9a0 0x7f660806ee60 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f661c009d60 tx=0x7f661c009190 comp rx=0 tx=0).stop 2024-09-12T03:24:19.303 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.302+0000 7f662cee4640 1 -- 172.21.15.27:0/510415840 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6610004d40 msgr2=0x7f662814d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:19.303 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.302+0000 7f662cee4640 1 --2- 172.21.15.27:0/510415840 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6610004d40 0x7f662814d3f0 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7f661800f770 tx=0x7f661800fb70 comp rx=0 tx=0).stop 2024-09-12T03:24:19.303 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.302+0000 7f6626d9d640 1 -- 172.21.15.27:0/510415840 reap_dead start 2024-09-12T03:24:19.303 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.302+0000 7f662cee4640 1 -- 172.21.15.27:0/510415840 shutdown_connections 2024-09-12T03:24:19.303 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.302+0000 7f662cee4640 1 -- 172.21.15.27:0/510415840 >> 172.21.15.27:0/510415840 conn(0x7f662806f320 msgr2=0x7f662815cb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:19.303 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.302+0000 7f662cee4640 1 -- 172.21.15.27:0/510415840 shutdown_connections 2024-09-12T03:24:19.303 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.302+0000 7f662cee4640 1 -- 172.21.15.27:0/510415840 wait complete. 2024-09-12T03:24:19.313 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-09-12T03:24:19.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.417+0000 7f9593b68640 1 Processor -- start 2024-09-12T03:24:19.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.417+0000 7f9593b68640 1 -- start start 2024-09-12T03:24:19.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.418+0000 7f9593b68640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f958c073960 0x7f958c162320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:19.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.418+0000 7f9593b68640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f958c0d2240 con 0x7f958c15e5f0 2024-09-12T03:24:19.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.418+0000 7f9593b68640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f958c163090 con 0x7f958c162860 2024-09-12T03:24:19.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.418+0000 7f9593b68640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f958c0d0da0 con 0x7f958c073960 2024-09-12T03:24:19.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.418+0000 7f9591103640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f958c073960 0x7f958c162320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:19.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.418+0000 7f9591103640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f958c073960 0x7f958c162320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:48928/0 (socket says 172.21.15.27:48928) 2024-09-12T03:24:19.419 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.418+0000 7f9591904640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f958c15e5f0 0x7f958c073250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:34988/0 (socket says 172.21.15.27:34988) 2024-09-12T03:24:19.419 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.418+0000 7f9591103640 1 -- 172.21.15.27:0/68746580 learned_addr learned my addr 172.21.15.27:0/68746580 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:24:19.419 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.418+0000 7f9591103640 1 -- 172.21.15.27:0/68746580 >> v1:172.21.15.45:6789/0 conn(0x7f958c162860 legacy=0x7f958c15e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:24:19.419 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.418+0000 7f9590902640 1 -- 172.21.15.27:0/68746580 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 325181005 0 0) 0x7f958c0d2240 con 0x7f958c15e5f0 2024-09-12T03:24:19.419 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.418+0000 7f9591103640 1 -- 172.21.15.27:0/68746580 >> v1:172.21.15.27:6789/0 conn(0x7f958c15e5f0 legacy=0x7f958c073250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:19.419 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.418+0000 7f9591103640 1 -- 172.21.15.27:0/68746580 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f958c0d0470 con 0x7f958c073960 2024-09-12T03:24:19.420 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.419+0000 7f9591103640 1 --2- 172.21.15.27:0/68746580 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f958c073960 0x7f958c162320 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x7f957c009870 tx=0x7f957c02a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d49e04b55219a78c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:19.420 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.419+0000 7f9590902640 1 -- 172.21.15.27:0/68746580 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f957c037070 con 0x7f958c073960 2024-09-12T03:24:19.420 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.419+0000 7f9590902640 1 -- 172.21.15.27:0/68746580 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f957c02ae50 con 0x7f958c073960 2024-09-12T03:24:19.420 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.419+0000 7f9590902640 1 -- 172.21.15.27:0/68746580 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f957c032860 con 0x7f958c073960 2024-09-12T03:24:19.420 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.420+0000 7f9593b68640 1 -- 172.21.15.27:0/68746580 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f958c073960 msgr2=0x7f958c162320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:19.421 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.420+0000 7f9593b68640 1 --2- 172.21.15.27:0/68746580 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f958c073960 0x7f958c162320 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x7f957c009870 tx=0x7f957c02a300 comp rx=0 tx=0).stop 2024-09-12T03:24:19.421 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.420+0000 7f9593b68640 1 -- 172.21.15.27:0/68746580 shutdown_connections 2024-09-12T03:24:19.421 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.420+0000 7f9593b68640 1 --2- 172.21.15.27:0/68746580 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f958c073960 0x7f958c162320 unknown :-1 s=CLOSED pgs=1806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:19.421 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.420+0000 7f9593b68640 1 -- 172.21.15.27:0/68746580 >> 172.21.15.27:0/68746580 conn(0x7f958c06f300 msgr2=0x7f958c06f710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:19.421 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.420+0000 7f9593b68640 1 -- 172.21.15.27:0/68746580 shutdown_connections 2024-09-12T03:24:19.421 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.420+0000 7f9593b68640 1 -- 172.21.15.27:0/68746580 wait complete. 2024-09-12T03:24:19.423 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.421+0000 7f9593b68640 1 Processor -- start 2024-09-12T03:24:19.423 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.421+0000 7f9593b68640 1 -- start start 2024-09-12T03:24:19.423 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.422+0000 7f9593b68640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f958c073960 0x7f958c14b000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:19.424 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.422+0000 7f9593b68640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f958c15e5f0 0x7f958c0aa4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:19.424 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.422+0000 7f9593b68640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f958c162860 0x7f958c1f4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:19.424 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.422+0000 7f9593b68640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f958c0b5620 con 0x7f958c15e5f0 2024-09-12T03:24:19.424 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.422+0000 7f9593b68640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f958c0d0a30 con 0x7f958c073960 2024-09-12T03:24:19.424 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.422+0000 7f9593b68640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f958c162c70 con 0x7f958c162860 2024-09-12T03:24:19.424 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.422+0000 7f9591103640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f958c15e5f0 0x7f958c0aa4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:19.424 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.422+0000 7f9591103640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f958c15e5f0 0x7f958c0aa4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:48930/0 (socket says 172.21.15.27:48930) 2024-09-12T03:24:19.424 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.422+0000 7f9591103640 1 -- 172.21.15.27:0/1949566069 learned_addr learned my addr 172.21.15.27:0/1949566069 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:24:19.424 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.422+0000 7f9591904640 1 --2- 172.21.15.27:0/1949566069 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f958c073960 0x7f958c14b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:19.424 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.422+0000 7f9591103640 1 -- 172.21.15.27:0/1949566069 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f958c162860 msgr2=0x7f958c1f4720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:19.424 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.422+0000 7f9592105640 1 --2- 172.21.15.27:0/1949566069 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f958c162860 0x7f958c1f4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:19.424 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.422+0000 7f9591103640 1 --2- 172.21.15.27:0/1949566069 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f958c162860 0x7f958c1f4720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:19.425 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.422+0000 7f9591103640 1 -- 172.21.15.27:0/1949566069 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f958c073960 msgr2=0x7f958c14b000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:19.425 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.422+0000 7f9591103640 1 --2- 172.21.15.27:0/1949566069 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f958c073960 0x7f958c14b000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:19.425 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.422+0000 7f9591103640 1 -- 172.21.15.27:0/1949566069 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f958c14d750 con 0x7f958c15e5f0 2024-09-12T03:24:19.425 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.422+0000 7f9591904640 1 --2- 172.21.15.27:0/1949566069 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f958c073960 0x7f958c14b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:24:19.425 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.422+0000 7f9592105640 1 --2- 172.21.15.27:0/1949566069 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f958c162860 0x7f958c1f4720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:24:19.425 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.422+0000 7f9591103640 1 --2- 172.21.15.27:0/1949566069 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f958c15e5f0 0x7f958c0aa4c0 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 crypto rx=0x7f957c0090b0 tx=0x7f957c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:19.425 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.423+0000 7f95827fc640 1 -- 172.21.15.27:0/1949566069 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f957c037040 con 0x7f958c15e5f0 2024-09-12T03:24:19.425 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.423+0000 7f95827fc640 1 -- 172.21.15.27:0/1949566069 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f957c004230 con 0x7f958c15e5f0 2024-09-12T03:24:19.425 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.423+0000 7f9593b68640 1 -- 172.21.15.27:0/1949566069 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f958c14da30 con 0x7f958c15e5f0 2024-09-12T03:24:19.425 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.423+0000 7f95827fc640 1 -- 172.21.15.27:0/1949566069 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f957c032e40 con 0x7f958c15e5f0 2024-09-12T03:24:19.425 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.423+0000 7f9593b68640 1 -- 172.21.15.27:0/1949566069 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f958c0aaa00 con 0x7f958c15e5f0 2024-09-12T03:24:19.425 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.424+0000 7f9593b68640 1 -- 172.21.15.27:0/1949566069 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f958c162320 con 0x7f958c15e5f0 2024-09-12T03:24:19.427 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.424+0000 7f95827fc640 1 -- 172.21.15.27:0/1949566069 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 26) ==== 91518+0+0 (secure 0 0 0) 0x7f957c03a070 con 0x7f958c15e5f0 2024-09-12T03:24:19.427 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.425+0000 7f95827fc640 1 --2- 172.21.15.27:0/1949566069 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f957006c970 0x7f957006ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:19.427 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.425+0000 7f95827fc640 1 -- 172.21.15.27:0/1949566069 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(163..163 src has 1..163) ==== 4999+0+0 (secure 0 0 0) 0x7f957c0aa170 con 0x7f958c15e5f0 2024-09-12T03:24:19.427 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.425+0000 7f9591904640 1 --2- 172.21.15.27:0/1949566069 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f957006c970 0x7f957006ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:19.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.425+0000 7f9591904640 1 --2- 172.21.15.27:0/1949566069 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f957006c970 0x7f957006ee30 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f9588000f60 tx=0x7f9588009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:19.429 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.429+0000 7f95827fc640 1 -- 172.21.15.27:0/1949566069 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f957c041020 con 0x7f958c15e5f0 2024-09-12T03:24:19.603 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.603+0000 7f9593b68640 1 -- 172.21.15.27:0/1949566069 --> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) -- 0x7f958c069790 con 0x7f957006c970 2024-09-12T03:24:19.604 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.604+0000 7f95827fc640 1 -- 172.21.15.27:0/1949566069 <== mgr.4113 v2:172.21.15.27:6836/1226920526 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f958c069790 con 0x7f957006c970 2024-09-12T03:24:19.604 INFO:teuthology.orchestra.run.smithi027.stderr:instructed osd(s) 6 to deep-scrub 2024-09-12T03:24:19.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.606+0000 7f9593b68640 1 -- 172.21.15.27:0/1949566069 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f957006c970 msgr2=0x7f957006ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:19.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.606+0000 7f9593b68640 1 --2- 172.21.15.27:0/1949566069 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f957006c970 0x7f957006ee30 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f9588000f60 tx=0x7f9588009040 comp rx=0 tx=0).stop 2024-09-12T03:24:19.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.606+0000 7f9593b68640 1 -- 172.21.15.27:0/1949566069 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f958c15e5f0 msgr2=0x7f958c0aa4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:19.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.606+0000 7f9593b68640 1 --2- 172.21.15.27:0/1949566069 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f958c15e5f0 0x7f958c0aa4c0 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 crypto rx=0x7f957c0090b0 tx=0x7f957c0046c0 comp rx=0 tx=0).stop 2024-09-12T03:24:19.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.606+0000 7f9593b68640 1 -- 172.21.15.27:0/1949566069 shutdown_connections 2024-09-12T03:24:19.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.606+0000 7f9593b68640 1 --2- 172.21.15.27:0/1949566069 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f958c162860 0x7f958c1f4720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:19.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.606+0000 7f9593b68640 1 --2- 172.21.15.27:0/1949566069 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f957006c970 0x7f957006ee30 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:19.607 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.606+0000 7f9593b68640 1 --2- 172.21.15.27:0/1949566069 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f958c15e5f0 0x7f958c0aa4c0 unknown :-1 s=CLOSED pgs=1807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:19.607 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.606+0000 7f9593b68640 1 --2- 172.21.15.27:0/1949566069 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f958c073960 0x7f958c14b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:19.607 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.606+0000 7f9593b68640 1 -- 172.21.15.27:0/1949566069 >> 172.21.15.27:0/1949566069 conn(0x7f958c06f300 msgr2=0x7f958c159300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:19.607 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.606+0000 7f9593b68640 1 -- 172.21.15.27:0/1949566069 shutdown_connections 2024-09-12T03:24:19.607 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.606+0000 7f9593b68640 1 -- 172.21.15.27:0/1949566069 wait complete. 2024-09-12T03:24:19.616 INFO:tasks.ceph:Scrubbing osd.7 2024-09-12T03:24:19.617 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-09-12T03:24:19.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.723+0000 7f7b5ea77640 1 Processor -- start 2024-09-12T03:24:19.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.723+0000 7f7b5ea77640 1 -- start start 2024-09-12T03:24:19.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.724+0000 7f7b5ea77640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f7b5806dfd0 0x7f7b580d1da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:19.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.724+0000 7f7b5ea77640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7b580d2e30 con 0x7f7b58162040 2024-09-12T03:24:19.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.724+0000 7f7b5ea77640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7b58162420 con 0x7f7b580d3790 2024-09-12T03:24:19.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.724+0000 7f7b5ea77640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f7b580d30d0 con 0x7f7b5806dfd0 2024-09-12T03:24:19.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.724+0000 7f7b57fff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f7b5806dfd0 0x7f7b580d1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:19.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.724+0000 7f7b5c813640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f7b58162040 0x7f7b5806d8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:34992/0 (socket says 172.21.15.27:34992) 2024-09-12T03:24:19.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.724+0000 7f7b5c813640 1 -- 172.21.15.27:0/357066337 learned_addr learned my addr 172.21.15.27:0/357066337 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:24:19.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.725+0000 7f7b577fe640 1 -- 172.21.15.27:0/357066337 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3412315375 0 0) 0x7f7b580d2e30 con 0x7f7b58162040 2024-09-12T03:24:19.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.725+0000 7f7b577fe640 1 -- 172.21.15.27:0/357066337 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7b40003520 con 0x7f7b58162040 2024-09-12T03:24:19.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.725+0000 7f7b57fff640 1 -- 172.21.15.27:0/357066337 >> v1:172.21.15.45:6789/0 conn(0x7f7b580d3790 legacy=0x7f7b580cf320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:24:19.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.725+0000 7f7b57fff640 1 -- 172.21.15.27:0/357066337 >> v1:172.21.15.27:6789/0 conn(0x7f7b58162040 legacy=0x7f7b5806d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:19.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.725+0000 7f7b57fff640 1 -- 172.21.15.27:0/357066337 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b580d33b0 con 0x7f7b5806dfd0 2024-09-12T03:24:19.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.726+0000 7f7b57fff640 1 --2- 172.21.15.27:0/357066337 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7b5806dfd0 0x7f7b580d1da0 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x7f7b4c0097a0 tx=0x7f7b4c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f93673e6e722bca server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:19.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.726+0000 7f7b577fe640 1 -- 172.21.15.27:0/357066337 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b4c037070 con 0x7f7b5806dfd0 2024-09-12T03:24:19.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.726+0000 7f7b577fe640 1 -- 172.21.15.27:0/357066337 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b4c032440 con 0x7f7b5806dfd0 2024-09-12T03:24:19.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.726+0000 7f7b577fe640 1 -- 172.21.15.27:0/357066337 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b4c032810 con 0x7f7b5806dfd0 2024-09-12T03:24:19.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.726+0000 7f7b5ea77640 1 -- 172.21.15.27:0/357066337 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7b5806dfd0 msgr2=0x7f7b580d1da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:19.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.726+0000 7f7b5ea77640 1 --2- 172.21.15.27:0/357066337 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7b5806dfd0 0x7f7b580d1da0 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x7f7b4c0097a0 tx=0x7f7b4c02a1e0 comp rx=0 tx=0).stop 2024-09-12T03:24:19.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.726+0000 7f7b5ea77640 1 -- 172.21.15.27:0/357066337 shutdown_connections 2024-09-12T03:24:19.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.726+0000 7f7b5ea77640 1 --2- 172.21.15.27:0/357066337 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7b5806dfd0 0x7f7b580d1da0 unknown :-1 s=CLOSED pgs=1809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:19.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.726+0000 7f7b5ea77640 1 -- 172.21.15.27:0/357066337 >> 172.21.15.27:0/357066337 conn(0x7f7b580b6700 msgr2=0x7f7b580b6b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:19.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.726+0000 7f7b5ea77640 1 -- 172.21.15.27:0/357066337 shutdown_connections 2024-09-12T03:24:19.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.726+0000 7f7b5ea77640 1 -- 172.21.15.27:0/357066337 wait complete. 2024-09-12T03:24:19.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.727+0000 7f7b5ea77640 1 Processor -- start 2024-09-12T03:24:19.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.728+0000 7f7b5ea77640 1 -- start start 2024-09-12T03:24:19.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.728+0000 7f7b5ea77640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7b5806dfd0 0x7f7b5815bba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:19.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.728+0000 7f7b5ea77640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f7b580d3790 0x7f7b580ce4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:19.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.728+0000 7f7b5ea77640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7b58162040 0x7f7b5814ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:19.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.728+0000 7f7b5ea77640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f7b580d4150 con 0x7f7b58162040 2024-09-12T03:24:19.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.728+0000 7f7b5ea77640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f7b580d4530 con 0x7f7b5806dfd0 2024-09-12T03:24:19.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.728+0000 7f7b5ea77640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f7b580d42d0 con 0x7f7b580d3790 2024-09-12T03:24:19.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.729+0000 7f7b5d014640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7b58162040 0x7f7b5814ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:19.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.729+0000 7f7b5d014640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7b58162040 0x7f7b5814ec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:48944/0 (socket says 172.21.15.27:48944) 2024-09-12T03:24:19.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.729+0000 7f7b5d014640 1 -- 172.21.15.27:0/124193482 learned_addr learned my addr 172.21.15.27:0/124193482 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:24:19.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.729+0000 7f7b57fff640 1 --2- 172.21.15.27:0/124193482 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f7b580d3790 0x7f7b580ce4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:19.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.729+0000 7f7b5c813640 1 --2- 172.21.15.27:0/124193482 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7b5806dfd0 0x7f7b5815bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:19.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.729+0000 7f7b5d014640 1 -- 172.21.15.27:0/124193482 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f7b580d3790 msgr2=0x7f7b580ce4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:19.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.729+0000 7f7b5d014640 1 --2- 172.21.15.27:0/124193482 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f7b580d3790 0x7f7b580ce4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:19.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.729+0000 7f7b5d014640 1 -- 172.21.15.27:0/124193482 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7b5806dfd0 msgr2=0x7f7b5815bba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:19.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.729+0000 7f7b5d014640 1 --2- 172.21.15.27:0/124193482 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7b5806dfd0 0x7f7b5815bba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:19.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.729+0000 7f7b5d014640 1 -- 172.21.15.27:0/124193482 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b580cb920 con 0x7f7b58162040 2024-09-12T03:24:19.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.729+0000 7f7b5c813640 1 --2- 172.21.15.27:0/124193482 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7b5806dfd0 0x7f7b5815bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:24:19.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.729+0000 7f7b5d014640 1 --2- 172.21.15.27:0/124193482 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7b58162040 0x7f7b5814ec60 secure :-1 s=READY pgs=1810 cs=0 l=1 rev1=1 crypto rx=0x7f7b4800c3e0 tx=0x7f7b4800c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:19.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.729+0000 7f7b557fa640 1 -- 172.21.15.27:0/124193482 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b48011790 con 0x7f7b58162040 2024-09-12T03:24:19.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.729+0000 7f7b557fa640 1 -- 172.21.15.27:0/124193482 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b48017070 con 0x7f7b58162040 2024-09-12T03:24:19.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.729+0000 7f7b5ea77640 1 -- 172.21.15.27:0/124193482 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b580cbc00 con 0x7f7b58162040 2024-09-12T03:24:19.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.729+0000 7f7b557fa640 1 -- 172.21.15.27:0/124193482 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b48011bb0 con 0x7f7b58162040 2024-09-12T03:24:19.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.729+0000 7f7b5ea77640 1 -- 172.21.15.27:0/124193482 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b580cc1c0 con 0x7f7b58162040 2024-09-12T03:24:19.733 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.730+0000 7f7b5ea77640 1 -- 172.21.15.27:0/124193482 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7b580d1da0 con 0x7f7b58162040 2024-09-12T03:24:19.733 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.731+0000 7f7b557fa640 1 -- 172.21.15.27:0/124193482 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 26) ==== 91518+0+0 (secure 0 0 0) 0x7f7b4801c460 con 0x7f7b58162040 2024-09-12T03:24:19.733 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.731+0000 7f7b557fa640 1 --2- 172.21.15.27:0/124193482 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f7b4406c970 0x7f7b4406ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:19.733 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.732+0000 7f7b557fa640 1 -- 172.21.15.27:0/124193482 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(163..163 src has 1..163) ==== 4999+0+0 (secure 0 0 0) 0x7f7b48089510 con 0x7f7b58162040 2024-09-12T03:24:19.733 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.732+0000 7f7b5c813640 1 --2- 172.21.15.27:0/124193482 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f7b4406c970 0x7f7b4406ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:19.733 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.732+0000 7f7b557fa640 1 --2- 172.21.15.27:0/124193482 >> [v2:172.21.15.45:6824/1513300294,v1:172.21.15.45:6825/1513300294] conn(0x7f7b44072830 0x7f7b44074c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:19.733 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.732+0000 7f7b557fa640 1 -- 172.21.15.27:0/124193482 --> [v2:172.21.15.45:6824/1513300294,v1:172.21.15.45:6825/1513300294] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7b48020ec0 con 0x7f7b44072830 2024-09-12T03:24:19.733 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.732+0000 7f7b557fa640 1 -- 172.21.15.27:0/124193482 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_get_version_reply(handle=1 version=163) ==== 24+0+0 (secure 0 0 0) 0x7f7b480912d0 con 0x7f7b58162040 2024-09-12T03:24:19.733 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.732+0000 7f7b57fff640 1 --2- 172.21.15.27:0/124193482 >> [v2:172.21.15.45:6824/1513300294,v1:172.21.15.45:6825/1513300294] conn(0x7f7b44072830 0x7f7b44074c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:19.733 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.732+0000 7f7b5c813640 1 --2- 172.21.15.27:0/124193482 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f7b4406c970 0x7f7b4406ee30 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f7b50005c00 tx=0x7f7b50009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:19.733 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.732+0000 7f7b57fff640 1 --2- 172.21.15.27:0/124193482 >> [v2:172.21.15.45:6824/1513300294,v1:172.21.15.45:6825/1513300294] conn(0x7f7b44072830 0x7f7b44074c70 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:19.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.733+0000 7f7b557fa640 1 -- 172.21.15.27:0/124193482 <== osd.7 v2:172.21.15.45:6824/1513300294 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7b48020ec0 con 0x7f7b44072830 2024-09-12T03:24:19.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.746+0000 7f7b5ea77640 1 -- 172.21.15.27:0/124193482 --> [v2:172.21.15.45:6824/1513300294,v1:172.21.15.45:6825/1513300294] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f7b5806e6e0 con 0x7f7b44072830 2024-09-12T03:24:19.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.746+0000 7f7b557fa640 1 -- 172.21.15.27:0/124193482 <== osd.7 v2:172.21.15.45:6824/1513300294 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f7b5806e6e0 con 0x7f7b44072830 2024-09-12T03:24:19.746 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-12T03:24:19.746 INFO:teuthology.orchestra.run.smithi027.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-12T03:24:19.746 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-12T03:24:19.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.747+0000 7f7b5ea77640 1 -- 172.21.15.27:0/124193482 >> [v2:172.21.15.45:6824/1513300294,v1:172.21.15.45:6825/1513300294] conn(0x7f7b44072830 msgr2=0x7f7b44074c70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:19.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.747+0000 7f7b5ea77640 1 --2- 172.21.15.27:0/124193482 >> [v2:172.21.15.45:6824/1513300294,v1:172.21.15.45:6825/1513300294] conn(0x7f7b44072830 0x7f7b44074c70 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:19.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.747+0000 7f7b5ea77640 1 -- 172.21.15.27:0/124193482 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f7b4406c970 msgr2=0x7f7b4406ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:19.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.747+0000 7f7b5ea77640 1 --2- 172.21.15.27:0/124193482 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f7b4406c970 0x7f7b4406ee30 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f7b50005c00 tx=0x7f7b50009040 comp rx=0 tx=0).stop 2024-09-12T03:24:19.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.747+0000 7f7b5ea77640 1 -- 172.21.15.27:0/124193482 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7b58162040 msgr2=0x7f7b5814ec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:19.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.747+0000 7f7b5ea77640 1 --2- 172.21.15.27:0/124193482 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7b58162040 0x7f7b5814ec60 secure :-1 s=READY pgs=1810 cs=0 l=1 rev1=1 crypto rx=0x7f7b4800c3e0 tx=0x7f7b4800c8a0 comp rx=0 tx=0).stop 2024-09-12T03:24:19.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.747+0000 7f7b5d014640 1 -- 172.21.15.27:0/124193482 reap_dead start 2024-09-12T03:24:19.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.747+0000 7f7b5ea77640 1 -- 172.21.15.27:0/124193482 shutdown_connections 2024-09-12T03:24:19.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.747+0000 7f7b5ea77640 1 -- 172.21.15.27:0/124193482 >> 172.21.15.27:0/124193482 conn(0x7f7b580b6700 msgr2=0x7f7b580b9af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:19.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.747+0000 7f7b5ea77640 1 -- 172.21.15.27:0/124193482 shutdown_connections 2024-09-12T03:24:19.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.747+0000 7f7b5ea77640 1 -- 172.21.15.27:0/124193482 wait complete. 2024-09-12T03:24:19.757 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-09-12T03:24:19.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.861+0000 7ff20df2a640 1 Processor -- start 2024-09-12T03:24:19.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.861+0000 7ff20df2a640 1 -- start start 2024-09-12T03:24:19.862 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.862+0000 7ff20df2a640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7ff20815b2b0 0x7ff2080b3560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:19.862 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.862+0000 7ff20df2a640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff2080ce410 con 0x7ff208072d70 2024-09-12T03:24:19.862 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.862+0000 7ff20df2a640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff2080732c0 con 0x7ff20806e9b0 2024-09-12T03:24:19.862 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.862+0000 7ff20df2a640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7ff2080720a0 con 0x7ff20815b2b0 2024-09-12T03:24:19.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.862+0000 7ff2077fe640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7ff20815b2b0 0x7ff2080b3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:19.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.862+0000 7ff206ffd640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7ff208072d70 0x7ff208072430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:35006/0 (socket says 172.21.15.27:35006) 2024-09-12T03:24:19.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.862+0000 7ff206ffd640 1 -- 172.21.15.27:0/3206979182 learned_addr learned my addr 172.21.15.27:0/3206979182 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:24:19.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.862+0000 7ff2077fe640 1 -- 172.21.15.27:0/3206979182 >> v1:172.21.15.45:6789/0 conn(0x7ff20806e9b0 legacy=0x7ff2080cebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:24:19.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.862+0000 7ff2067fc640 1 -- 172.21.15.27:0/3206979182 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 887528833 0 0) 0x7ff1fc001110 con 0x7ff208072d70 2024-09-12T03:24:19.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.862+0000 7ff2077fe640 1 -- 172.21.15.27:0/3206979182 >> v1:172.21.15.27:6789/0 conn(0x7ff208072d70 legacy=0x7ff208072430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:19.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.862+0000 7ff2077fe640 1 -- 172.21.15.27:0/3206979182 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff208071ae0 con 0x7ff20815b2b0 2024-09-12T03:24:19.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.863+0000 7ff2077fe640 1 --2- 172.21.15.27:0/3206979182 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff20815b2b0 0x7ff2080b3560 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 crypto rx=0x7ff1f80097a0 tx=0x7ff1f802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bad497e6cb3ba94 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:19.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.863+0000 7ff2067fc640 1 -- 172.21.15.27:0/3206979182 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1f8037070 con 0x7ff20815b2b0 2024-09-12T03:24:19.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.863+0000 7ff2067fc640 1 -- 172.21.15.27:0/3206979182 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1f8032440 con 0x7ff20815b2b0 2024-09-12T03:24:19.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.863+0000 7ff2067fc640 1 -- 172.21.15.27:0/3206979182 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1f8032810 con 0x7ff20815b2b0 2024-09-12T03:24:19.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.864+0000 7ff20df2a640 1 -- 172.21.15.27:0/3206979182 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff20815b2b0 msgr2=0x7ff2080b3560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:19.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.864+0000 7ff20df2a640 1 --2- 172.21.15.27:0/3206979182 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff20815b2b0 0x7ff2080b3560 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 crypto rx=0x7ff1f80097a0 tx=0x7ff1f802a230 comp rx=0 tx=0).stop 2024-09-12T03:24:19.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.864+0000 7ff20df2a640 1 -- 172.21.15.27:0/3206979182 shutdown_connections 2024-09-12T03:24:19.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.864+0000 7ff20df2a640 1 --2- 172.21.15.27:0/3206979182 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff20815b2b0 0x7ff2080b3560 unknown :-1 s=CLOSED pgs=1812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:19.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.864+0000 7ff20df2a640 1 -- 172.21.15.27:0/3206979182 >> 172.21.15.27:0/3206979182 conn(0x7ff208062c60 msgr2=0x7ff20806bb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:19.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.864+0000 7ff20df2a640 1 -- 172.21.15.27:0/3206979182 shutdown_connections 2024-09-12T03:24:19.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.864+0000 7ff20df2a640 1 -- 172.21.15.27:0/3206979182 wait complete. 2024-09-12T03:24:19.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.865+0000 7ff20df2a640 1 Processor -- start 2024-09-12T03:24:19.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.865+0000 7ff20df2a640 1 -- start start 2024-09-12T03:24:19.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.866+0000 7ff20df2a640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff20806e9b0 0x7ff20815b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:19.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.866+0000 7ff20df2a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff208072d70 0x7ff2081f40c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:19.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.866+0000 7ff20df2a640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ff20815b2b0 0x7ff2081e77b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:19.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.866+0000 7ff20df2a640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7ff20815b690 con 0x7ff20806e9b0 2024-09-12T03:24:19.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.866+0000 7ff20df2a640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ff2080ce690 con 0x7ff208072d70 2024-09-12T03:24:19.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.866+0000 7ff20df2a640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7ff20815b810 con 0x7ff20815b2b0 2024-09-12T03:24:19.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.866+0000 7ff2077fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff20806e9b0 0x7ff20815b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:19.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.866+0000 7ff2077fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff20806e9b0 0x7ff20815b9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:48960/0 (socket says 172.21.15.27:48960) 2024-09-12T03:24:19.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.866+0000 7ff2077fe640 1 -- 172.21.15.27:0/1093927310 learned_addr learned my addr 172.21.15.27:0/1093927310 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:24:19.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.866+0000 7ff2077fe640 1 -- 172.21.15.27:0/1093927310 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ff20815b2b0 msgr2=0x7ff2081e77b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:19.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.866+0000 7ff206ffd640 1 --2- 172.21.15.27:0/1093927310 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff208072d70 0x7ff2081f40c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:19.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.866+0000 7ff2077fe640 1 --2- 172.21.15.27:0/1093927310 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ff20815b2b0 0x7ff2081e77b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:19.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.866+0000 7ff2077fe640 1 -- 172.21.15.27:0/1093927310 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff208072d70 msgr2=0x7ff2081f40c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:19.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.866+0000 7ff2077fe640 1 --2- 172.21.15.27:0/1093927310 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff208072d70 0x7ff2081f40c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:19.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.866+0000 7ff2077fe640 1 -- 172.21.15.27:0/1093927310 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff208140790 con 0x7ff20806e9b0 2024-09-12T03:24:19.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.867+0000 7ff2077fe640 1 --2- 172.21.15.27:0/1093927310 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff20806e9b0 0x7ff20815b9f0 secure :-1 s=READY pgs=1813 cs=0 l=1 rev1=1 crypto rx=0x7ff1f8000c00 tx=0x7ff1f80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:19.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.867+0000 7ff1effff640 1 -- 172.21.15.27:0/1093927310 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1f8037040 con 0x7ff20806e9b0 2024-09-12T03:24:19.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.867+0000 7ff1effff640 1 -- 172.21.15.27:0/1093927310 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1f8042070 con 0x7ff20806e9b0 2024-09-12T03:24:19.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.867+0000 7ff20df2a640 1 -- 172.21.15.27:0/1093927310 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff208140a70 con 0x7ff20806e9b0 2024-09-12T03:24:19.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.867+0000 7ff1effff640 1 -- 172.21.15.27:0/1093927310 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1f8030a70 con 0x7ff20806e9b0 2024-09-12T03:24:19.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.867+0000 7ff20df2a640 1 -- 172.21.15.27:0/1093927310 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff208141030 con 0x7ff20806e9b0 2024-09-12T03:24:19.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.868+0000 7ff20df2a640 1 -- 172.21.15.27:0/1093927310 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2080b3560 con 0x7ff20806e9b0 2024-09-12T03:24:19.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.868+0000 7ff1effff640 1 -- 172.21.15.27:0/1093927310 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 26) ==== 91518+0+0 (secure 0 0 0) 0x7ff1f8030c10 con 0x7ff20806e9b0 2024-09-12T03:24:19.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.869+0000 7ff1effff640 1 --2- 172.21.15.27:0/1093927310 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7ff1f406c970 0x7ff1f406ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:19.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.869+0000 7ff1effff640 1 -- 172.21.15.27:0/1093927310 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(163..163 src has 1..163) ==== 4999+0+0 (secure 0 0 0) 0x7ff1f80aa280 con 0x7ff20806e9b0 2024-09-12T03:24:19.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.869+0000 7ff206ffd640 1 --2- 172.21.15.27:0/1093927310 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7ff1f406c970 0x7ff1f406ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:19.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.869+0000 7ff206ffd640 1 --2- 172.21.15.27:0/1093927310 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7ff1f406c970 0x7ff1f406ee30 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7ff208146cc0 tx=0x7ff1fc009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:19.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:19.873+0000 7ff1effff640 1 -- 172.21.15.27:0/1093927310 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7ff1f8045360 con 0x7ff20806e9b0 2024-09-12T03:24:20.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:20.052+0000 7ff20df2a640 1 -- 172.21.15.27:0/1093927310 --> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) -- 0x7ff20815f060 con 0x7ff1f406c970 2024-09-12T03:24:20.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:20.052+0000 7ff1effff640 1 -- 172.21.15.27:0/1093927310 <== mgr.4113 v2:172.21.15.27:6836/1226920526 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7ff20815f060 con 0x7ff1f406c970 2024-09-12T03:24:20.052 INFO:teuthology.orchestra.run.smithi027.stderr:instructed osd(s) 7 to deep-scrub 2024-09-12T03:24:20.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:20.054+0000 7ff20df2a640 1 -- 172.21.15.27:0/1093927310 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7ff1f406c970 msgr2=0x7ff1f406ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:20.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:20.054+0000 7ff20df2a640 1 --2- 172.21.15.27:0/1093927310 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7ff1f406c970 0x7ff1f406ee30 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7ff208146cc0 tx=0x7ff1fc009040 comp rx=0 tx=0).stop 2024-09-12T03:24:20.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:20.054+0000 7ff20df2a640 1 -- 172.21.15.27:0/1093927310 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff20806e9b0 msgr2=0x7ff20815b9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:20.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:20.054+0000 7ff20df2a640 1 --2- 172.21.15.27:0/1093927310 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff20806e9b0 0x7ff20815b9f0 secure :-1 s=READY pgs=1813 cs=0 l=1 rev1=1 crypto rx=0x7ff1f8000c00 tx=0x7ff1f80046c0 comp rx=0 tx=0).stop 2024-09-12T03:24:20.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:20.054+0000 7ff20df2a640 1 -- 172.21.15.27:0/1093927310 shutdown_connections 2024-09-12T03:24:20.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:20.054+0000 7ff20df2a640 1 --2- 172.21.15.27:0/1093927310 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7ff20815b2b0 0x7ff2081e77b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:20.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:20.054+0000 7ff20df2a640 1 --2- 172.21.15.27:0/1093927310 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff208072d70 0x7ff2081f40c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:20.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:20.054+0000 7ff20df2a640 1 --2- 172.21.15.27:0/1093927310 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7ff1f406c970 0x7ff1f406ee30 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:20.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:20.054+0000 7ff20df2a640 1 --2- 172.21.15.27:0/1093927310 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff20806e9b0 0x7ff20815b9f0 unknown :-1 s=CLOSED pgs=1813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:20.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:20.054+0000 7ff20df2a640 1 -- 172.21.15.27:0/1093927310 >> 172.21.15.27:0/1093927310 conn(0x7ff208062c60 msgr2=0x7ff20815f560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:20.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:20.054+0000 7ff20df2a640 1 -- 172.21.15.27:0/1093927310 shutdown_connections 2024-09-12T03:24:20.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:20.054+0000 7ff20df2a640 1 -- 172.21.15.27:0/1093927310 wait complete. 2024-09-12T03:24:20.065 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-12T03:24:20.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:20.167+0000 7f003aca6640 1 Processor -- start 2024-09-12T03:24:20.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:20.167+0000 7f003aca6640 1 -- start start 2024-09-12T03:24:20.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:20.168+0000 7f003aca6640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f003406f0c0 0x7f00340b3560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:20.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:20.168+0000 7f003aca6640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f00340ce410 con 0x7f003415b2b0 2024-09-12T03:24:20.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:20.168+0000 7f003aca6640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f00340732c0 con 0x7f0034072d70 2024-09-12T03:24:20.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:20.168+0000 7f003aca6640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f00340720a0 con 0x7f003406f0c0 2024-09-12T03:24:20.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:20.168+0000 7f0039243640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f003406f0c0 0x7f00340b3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:20.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:20.168+0000 7f0038a42640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f003415b2b0 0x7f0034072430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:35018/0 (socket says 172.21.15.27:35018) 2024-09-12T03:24:20.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:20.168+0000 7f0038a42640 1 -- 172.21.15.27:0/1238466966 learned_addr learned my addr 172.21.15.27:0/1238466966 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-09-12T03:24:20.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:20.168+0000 7f0039243640 1 -- 172.21.15.27:0/1238466966 >> v1:172.21.15.45:6789/0 conn(0x7f0034072d70 legacy=0x7f003406e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:20.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:20.168+0000 7f0039243640 1 -- 172.21.15.27:0/1238466966 >> v1:172.21.15.27:6789/0 conn(0x7f003415b2b0 legacy=0x7f0034072430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:20.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:20.168+0000 7f00337fe640 1 -- 172.21.15.27:0/1238466966 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 155555214 0 0) 0x7f002c001490 con 0x7f003415b2b0 2024-09-12T03:24:20.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:20.169+0000 7f0039243640 1 -- 172.21.15.27:0/1238466966 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0034071ae0 con 0x7f003406f0c0 2024-09-12T03:24:20.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:20.169+0000 7f0039243640 1 --2- 172.21.15.27:0/1238466966 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f003406f0c0 0x7f00340b3560 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 crypto rx=0x7f00240097a0 tx=0x7f002402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9510d0923199a64c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:20.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:20.169+0000 7f00337fe640 1 -- 172.21.15.27:0/1238466966 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0024037070 con 0x7f003406f0c0 2024-09-12T03:24:20.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:20.169+0000 7f00337fe640 1 -- 172.21.15.27:0/1238466966 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0024032440 con 0x7f003406f0c0 2024-09-12T03:24:20.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:20.169+0000 7f00337fe640 1 -- 172.21.15.27:0/1238466966 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0024032810 con 0x7f003406f0c0 2024-09-12T03:24:20.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:20.170+0000 7f003aca6640 1 -- 172.21.15.27:0/1238466966 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f003406f0c0 msgr2=0x7f00340b3560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:20.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:20.170+0000 7f003aca6640 1 --2- 172.21.15.27:0/1238466966 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f003406f0c0 0x7f00340b3560 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 crypto rx=0x7f00240097a0 tx=0x7f002402a230 comp rx=0 tx=0).stop 2024-09-12T03:24:20.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:20.170+0000 7f003aca6640 1 -- 172.21.15.27:0/1238466966 shutdown_connections 2024-09-12T03:24:20.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:20.170+0000 7f003aca6640 1 --2- 172.21.15.27:0/1238466966 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f003406f0c0 0x7f00340b3560 unknown :-1 s=CLOSED pgs=1815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:20.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:20.170+0000 7f003aca6640 1 -- 172.21.15.27:0/1238466966 >> 172.21.15.27:0/1238466966 conn(0x7f0034062c60 msgr2=0x7f003406bb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:20.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:20.170+0000 7f003aca6640 1 -- 172.21.15.27:0/1238466966 shutdown_connections 2024-09-12T03:24:20.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:20.170+0000 7f003aca6640 1 -- 172.21.15.27:0/1238466966 wait complete. 2024-09-12T03:24:20.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:20.171+0000 7f003aca6640 1 Processor -- start 2024-09-12T03:24:20.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:20.171+0000 7f003aca6640 1 -- start start 2024-09-12T03:24:20.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:20.172+0000 7f003aca6640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f003406f0c0 0x7f00341f4440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:20.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:20.172+0000 7f003aca6640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f0034072d70 0x7f003415b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:20.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:20.172+0000 7f003aca6640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f003415b2b0 0x7f00340ae8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:20.175 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:20.172+0000 7f003aca6640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f003415b690 con 0x7f003406f0c0 2024-09-12T03:24:20.175 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:20.172+0000 7f003aca6640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f00340ce690 con 0x7f003415b2b0 2024-09-12T03:24:20.175 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:20.172+0000 7f003aca6640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f003415b810 con 0x7f0034072d70 2024-09-12T03:24:20.175 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:20.172+0000 7f0038a42640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f003406f0c0 0x7f00341f4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:20.175 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:20.172+0000 7f0038a42640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f003406f0c0 0x7f00341f4440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:48976/0 (socket says 172.21.15.27:48976) 2024-09-12T03:24:20.175 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:20.172+0000 7f0038a42640 1 -- 172.21.15.27:0/3399677095 learned_addr learned my addr 172.21.15.27:0/3399677095 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:24:20.175 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:20.173+0000 7f0039243640 1 --2- 172.21.15.27:0/3399677095 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f003415b2b0 0x7f00340ae8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:20.175 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:20.173+0000 7f0033fff640 1 --2- 172.21.15.27:0/3399677095 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f0034072d70 0x7f003415b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:20.175 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:20.173+0000 7f0038a42640 1 -- 172.21.15.27:0/3399677095 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f0034072d70 msgr2=0x7f003415b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:20.175 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:20.173+0000 7f0038a42640 1 --2- 172.21.15.27:0/3399677095 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f0034072d70 0x7f003415b9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:20.175 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:20.173+0000 7f0038a42640 1 -- 172.21.15.27:0/3399677095 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f003415b2b0 msgr2=0x7f00340ae8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:20.175 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:20.173+0000 7f0038a42640 1 --2- 172.21.15.27:0/3399677095 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f003415b2b0 0x7f00340ae8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:20.175 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:20.173+0000 7f0038a42640 1 -- 172.21.15.27:0/3399677095 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f003413d8d0 con 0x7f003406f0c0 2024-09-12T03:24:20.176 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:20.173+0000 7f0039243640 1 --2- 172.21.15.27:0/3399677095 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f003415b2b0 0x7f00340ae8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:24:20.176 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:20.173+0000 7f0038a42640 1 --2- 172.21.15.27:0/3399677095 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f003406f0c0 0x7f00341f4440 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x7f002c00c780 tx=0x7f002c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:20.176 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:20.173+0000 7f00317fa640 1 -- 172.21.15.27:0/3399677095 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f002c018070 con 0x7f003406f0c0 2024-09-12T03:24:20.176 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:20.173+0000 7f00317fa640 1 -- 172.21.15.27:0/3399677095 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f002c00a640 con 0x7f003406f0c0 2024-09-12T03:24:20.176 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:20.173+0000 7f003aca6640 1 -- 172.21.15.27:0/3399677095 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f003413dbb0 con 0x7f003406f0c0 2024-09-12T03:24:20.176 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:20.173+0000 7f00317fa640 1 -- 172.21.15.27:0/3399677095 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f002c00aab0 con 0x7f003406f0c0 2024-09-12T03:24:20.176 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:20.173+0000 7f003aca6640 1 -- 172.21.15.27:0/3399677095 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f003413e170 con 0x7f003406f0c0 2024-09-12T03:24:20.176 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:20.174+0000 7f003aca6640 1 -- 172.21.15.27:0/3399677095 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00340b3560 con 0x7f003406f0c0 2024-09-12T03:24:20.176 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:20.175+0000 7f00317fa640 1 -- 172.21.15.27:0/3399677095 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 26) ==== 91518+0+0 (secure 0 0 0) 0x7f002c01b070 con 0x7f003406f0c0 2024-09-12T03:24:20.177 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:20.175+0000 7f00317fa640 1 --2- 172.21.15.27:0/3399677095 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f002006c970 0x7f002006ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:20.177 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:20.175+0000 7f0033fff640 1 --2- 172.21.15.27:0/3399677095 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f002006c970 0x7f002006ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:20.177 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:20.175+0000 7f00317fa640 1 -- 172.21.15.27:0/3399677095 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(163..163 src has 1..163) ==== 4999+0+0 (secure 0 0 0) 0x7f002c089050 con 0x7f003406f0c0 2024-09-12T03:24:20.177 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:20.176+0000 7f0033fff640 1 --2- 172.21.15.27:0/3399677095 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f002006c970 0x7f002006ee30 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f00340b1660 tx=0x7f0028009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:20.179 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:20.180+0000 7f00317fa640 1 -- 172.21.15.27:0/3399677095 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f00340ce410 con 0x7f003406f0c0 2024-09-12T03:24:20.353 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:20.353+0000 7f003aca6640 1 -- 172.21.15.27:0/3399677095 --> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f00340b13b0 con 0x7f002006c970 2024-09-12T03:24:20.353 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:20.353+0000 7f00317fa640 1 -- 172.21.15.27:0/3399677095 <== mgr.4113 v2:172.21.15.27:6836/1226920526 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13486 (secure 0 0 0) 0x7f00340b13b0 con 0x7f002006c970 2024-09-12T03:24:20.354 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:24:20.354 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2024-09-12T03:24:20.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:20.355+0000 7f003aca6640 1 -- 172.21.15.27:0/3399677095 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f002006c970 msgr2=0x7f002006ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:20.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:20.355+0000 7f003aca6640 1 --2- 172.21.15.27:0/3399677095 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f002006c970 0x7f002006ee30 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f00340b1660 tx=0x7f0028009040 comp rx=0 tx=0).stop 2024-09-12T03:24:20.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:20.355+0000 7f003aca6640 1 -- 172.21.15.27:0/3399677095 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f003406f0c0 msgr2=0x7f00341f4440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:20.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:20.355+0000 7f003aca6640 1 --2- 172.21.15.27:0/3399677095 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f003406f0c0 0x7f00341f4440 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x7f002c00c780 tx=0x7f002c00cc40 comp rx=0 tx=0).stop 2024-09-12T03:24:20.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:20.356+0000 7f003aca6640 1 -- 172.21.15.27:0/3399677095 shutdown_connections 2024-09-12T03:24:20.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:20.356+0000 7f003aca6640 1 --2- 172.21.15.27:0/3399677095 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f003415b2b0 0x7f00340ae8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:20.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:20.356+0000 7f003aca6640 1 --2- 172.21.15.27:0/3399677095 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f0034072d70 0x7f003415b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:20.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:20.356+0000 7f003aca6640 1 --2- 172.21.15.27:0/3399677095 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f002006c970 0x7f002006ee30 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:20.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:20.356+0000 7f003aca6640 1 --2- 172.21.15.27:0/3399677095 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f003406f0c0 0x7f00341f4440 unknown :-1 s=CLOSED pgs=1816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:20.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:20.356+0000 7f003aca6640 1 -- 172.21.15.27:0/3399677095 >> 172.21.15.27:0/3399677095 conn(0x7f0034062c60 msgr2=0x7f003406e7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:20.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:20.356+0000 7f003aca6640 1 -- 172.21.15.27:0/3399677095 shutdown_connections 2024-09-12T03:24:20.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:20.356+0000 7f003aca6640 1 -- 172.21.15.27:0/3399677095 wait complete. 2024-09-12T03:24:20.366 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":219,"stamp":"2024-09-12T03:24:19.145104+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":17,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":291780,"kb_used_data":17288,"kb_used_omap":12,"kb_used_meta":274291,"kb_avail":754682940,"statfs":{"total":773094113280,"available":772795330560,"internally_reserved":0,"allocated":17702912,"data_stored":8479848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":280874544},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-55119,"num_objects":-23,"num_object_clones":0,"num_object_copies":-46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-23,"num_whiteouts":0,"num_read":-5,"num_read_kb":-5,"num_write":-81,"num_write_kb":-85,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001450"},"pg_stats":[{"pgid":"1.0","version":"70'128","reported_seq":470,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-09-12T03:24:13.149532+0000","last_change":"2024-09-12T03:16:21.883446+0000","last_active":"2024-09-12T03:24:13.149532+0000","last_peered":"2024-09-12T03:24:13.149532+0000","last_clean":"2024-09-12T03:24:13.149532+0000","last_became_active":"2024-09-12T03:16:21.882207+0000","last_became_peered":"2024-09-12T03:16:21.882207+0000","last_unstale":"2024-09-12T03:24:13.149532+0000","last_undegraded":"2024-09-12T03:24:13.149532+0000","last_fullsized":"2024-09-12T03:24:13.149532+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:16:20.316426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:16:20.316426+0000","last_clean_scrub_stamp":"2024-09-12T03:16:20.316426+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:12:15.424320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542241,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42108,"kb_used_data":2596,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94329732,"statfs":{"total":96636764160,"available":96593645568,"internally_reserved":0,"allocated":2658304,"data_stored":1502757,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":40434122},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542243,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37368,"kb_used_data":2016,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334472,"statfs":{"total":96636764160,"available":96598499328,"internally_reserved":0,"allocated":2064384,"data_stored":912389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542242,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":41524,"kb_used_data":2016,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330316,"statfs":{"total":96636764160,"available":96594243584,"internally_reserved":0,"allocated":2064384,"data_stored":912389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542241,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37368,"kb_used_data":2016,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334472,"statfs":{"total":96636764160,"available":96598499328,"internally_reserved":0,"allocated":2064384,"data_stored":912389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542242,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33204,"kb_used_data":2016,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338636,"statfs":{"total":96636764160,"available":96602763264,"internally_reserved":0,"allocated":2064384,"data_stored":912389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542242,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33212,"kb_used_data":2016,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338628,"statfs":{"total":96636764160,"available":96602755072,"internally_reserved":0,"allocated":2064384,"data_stored":912389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542243,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33212,"kb_used_data":2016,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338628,"statfs":{"total":96636764160,"available":96602755072,"internally_reserved":0,"allocated":2064384,"data_stored":912389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542243,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33784,"kb_used_data":2596,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338056,"statfs":{"total":96636764160,"available":96602169344,"internally_reserved":0,"allocated":2658304,"data_stored":1502757,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T03:24:20.369 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-09-12T03:16:20.316426+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=12, tm_hour=3, tm_min=16, tm_sec=20, tm_wday=3, tm_yday=256, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=12, tm_hour=3, tm_min=24, tm_sec=15, tm_wday=3, tm_yday=256, tm_isdst=0) 2024-09-12T03:24:20.370 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-09-12T03:24:31.483 DEBUG:teuthology.orchestra.run.smithi027:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-12T03:24:31.486 DEBUG:teuthology.orchestra.run.smithi045:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-12T03:24:40.371 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-12T03:24:40.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:40.488+0000 7fac1d0f1640 1 Processor -- start 2024-09-12T03:24:40.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:40.488+0000 7fac1d0f1640 1 -- start start 2024-09-12T03:24:40.490 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:40.489+0000 7fac1d0f1640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fac1815b2b0 0x7fac180b3560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:40.490 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:40.489+0000 7fac1d0f1640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fac1806e9b0 0x7fac1806ee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:40.490 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:40.489+0000 7fac1d0f1640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fac180ce410 con 0x7fac18072d70 2024-09-12T03:24:40.490 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:40.489+0000 7fac1d0f1640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7fac180720a0 con 0x7fac1806e9b0 2024-09-12T03:24:40.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:40.489+0000 7fac1d0f1640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fac180732c0 con 0x7fac1815b2b0 2024-09-12T03:24:40.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:40.489+0000 7fac1759e640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fac1806e9b0 0x7fac1806ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:40.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:40.489+0000 7fac1759e640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fac1806e9b0 0x7fac1806ee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38822/0 (socket says 172.21.15.27:38822) 2024-09-12T03:24:40.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:40.489+0000 7fac1759e640 1 -- 172.21.15.27:0/925062705 learned_addr learned my addr 172.21.15.27:0/925062705 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:24:40.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:40.489+0000 7fac16d9d640 1 --2- 172.21.15.27:0/925062705 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fac1815b2b0 0x7fac180b3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:40.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:40.489+0000 7fac1759e640 1 -- 172.21.15.27:0/925062705 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fac1815b2b0 msgr2=0x7fac180b3560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:24:40.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:40.489+0000 7fac1759e640 1 --2- 172.21.15.27:0/925062705 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fac1815b2b0 0x7fac180b3560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:40.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:40.489+0000 7fac1759e640 1 -- 172.21.15.27:0/925062705 >> v1:172.21.15.45:6789/0 conn(0x7fac18072d70 legacy=0x7fac18072430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:40.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:40.489+0000 7fac1759e640 1 -- 172.21.15.27:0/925062705 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac18071ae0 con 0x7fac1806e9b0 2024-09-12T03:24:40.493 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:40.490+0000 7fac1759e640 1 --2- 172.21.15.27:0/925062705 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fac1806e9b0 0x7fac1806ee60 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7fac10009e50 tx=0x7fac1002a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1876049776bae9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:40.493 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:40.490+0000 7fac1659c640 1 -- 172.21.15.27:0/925062705 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac10036070 con 0x7fac1806e9b0 2024-09-12T03:24:40.493 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:40.490+0000 7fac1659c640 1 -- 172.21.15.27:0/925062705 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac10031600 con 0x7fac1806e9b0 2024-09-12T03:24:40.493 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:40.490+0000 7fac1659c640 1 -- 172.21.15.27:0/925062705 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac100319d0 con 0x7fac1806e9b0 2024-09-12T03:24:40.493 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:40.491+0000 7fac1d0f1640 1 -- 172.21.15.27:0/925062705 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fac1806e9b0 msgr2=0x7fac1806ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:40.493 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:40.491+0000 7fac1d0f1640 1 --2- 172.21.15.27:0/925062705 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fac1806e9b0 0x7fac1806ee60 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7fac10009e50 tx=0x7fac1002a710 comp rx=0 tx=0).stop 2024-09-12T03:24:40.493 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:40.491+0000 7fac1d0f1640 1 -- 172.21.15.27:0/925062705 shutdown_connections 2024-09-12T03:24:40.493 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:40.491+0000 7fac1d0f1640 1 --2- 172.21.15.27:0/925062705 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fac1806e9b0 0x7fac1806ee60 unknown :-1 s=CLOSED pgs=1817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:40.493 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:40.491+0000 7fac1d0f1640 1 --2- 172.21.15.27:0/925062705 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fac1815b2b0 0x7fac180b3560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:40.493 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:40.491+0000 7fac1d0f1640 1 -- 172.21.15.27:0/925062705 >> 172.21.15.27:0/925062705 conn(0x7fac18062c60 msgr2=0x7fac1806bb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:40.493 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:40.491+0000 7fac1d0f1640 1 -- 172.21.15.27:0/925062705 shutdown_connections 2024-09-12T03:24:40.494 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:40.491+0000 7fac1d0f1640 1 -- 172.21.15.27:0/925062705 wait complete. 2024-09-12T03:24:40.494 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:40.492+0000 7fac1d0f1640 1 Processor -- start 2024-09-12T03:24:40.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:40.492+0000 7fac1d0f1640 1 -- start start 2024-09-12T03:24:40.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:40.493+0000 7fac1d0f1640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fac1806e9b0 0x7fac181f23b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:40.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:40.493+0000 7fac1d0f1640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fac18072d70 0x7fac1813fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:40.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:40.493+0000 7fac1d0f1640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fac1815b2b0 0x7fac18149a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:40.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:40.493+0000 7fac1d0f1640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fac1815b690 con 0x7fac1806e9b0 2024-09-12T03:24:40.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:40.493+0000 7fac1d0f1640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fac180ce690 con 0x7fac1815b2b0 2024-09-12T03:24:40.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:40.493+0000 7fac1d0f1640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7fac1815b810 con 0x7fac18072d70 2024-09-12T03:24:40.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:40.493+0000 7fac16d9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fac1806e9b0 0x7fac181f23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:40.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:40.493+0000 7fac16d9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fac1806e9b0 0x7fac181f23b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38824/0 (socket says 172.21.15.27:38824) 2024-09-12T03:24:40.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:40.493+0000 7fac16d9d640 1 -- 172.21.15.27:0/1297543589 learned_addr learned my addr 172.21.15.27:0/1297543589 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:24:40.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:40.493+0000 7fac0e59c640 1 --2- 172.21.15.27:0/1297543589 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fac18072d70 0x7fac1813fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:40.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:40.493+0000 7fac1759e640 1 --2- 172.21.15.27:0/1297543589 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fac1815b2b0 0x7fac18149a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:40.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:40.494+0000 7fac16d9d640 1 -- 172.21.15.27:0/1297543589 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fac18072d70 msgr2=0x7fac1813fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:40.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:40.494+0000 7fac16d9d640 1 --2- 172.21.15.27:0/1297543589 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fac18072d70 0x7fac1813fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:40.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:40.494+0000 7fac16d9d640 1 -- 172.21.15.27:0/1297543589 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fac1815b2b0 msgr2=0x7fac18149a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:40.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:40.494+0000 7fac16d9d640 1 --2- 172.21.15.27:0/1297543589 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fac1815b2b0 0x7fac18149a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:40.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:40.494+0000 7fac16d9d640 1 -- 172.21.15.27:0/1297543589 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac1814a4f0 con 0x7fac1806e9b0 2024-09-12T03:24:40.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:40.494+0000 7fac16d9d640 1 --2- 172.21.15.27:0/1297543589 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fac1806e9b0 0x7fac181f23b0 secure :-1 s=READY pgs=1818 cs=0 l=1 rev1=1 crypto rx=0x7fac0800d5f0 tx=0x7fac0800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:40.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:40.494+0000 7fac0ffff640 1 -- 172.21.15.27:0/1297543589 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac080125f0 con 0x7fac1806e9b0 2024-09-12T03:24:40.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:40.494+0000 7fac0ffff640 1 -- 172.21.15.27:0/1297543589 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac08019070 con 0x7fac1806e9b0 2024-09-12T03:24:40.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:40.494+0000 7fac1d0f1640 1 -- 172.21.15.27:0/1297543589 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac1815c760 con 0x7fac1806e9b0 2024-09-12T03:24:40.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:40.494+0000 7fac0ffff640 1 -- 172.21.15.27:0/1297543589 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac080129d0 con 0x7fac1806e9b0 2024-09-12T03:24:40.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:40.494+0000 7fac1d0f1640 1 -- 172.21.15.27:0/1297543589 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac18140240 con 0x7fac1806e9b0 2024-09-12T03:24:40.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:40.495+0000 7fac1d0f1640 1 -- 172.21.15.27:0/1297543589 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac180b3560 con 0x7fac1806e9b0 2024-09-12T03:24:40.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:40.496+0000 7fac0ffff640 1 -- 172.21.15.27:0/1297543589 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 26) ==== 91518+0+0 (secure 0 0 0) 0x7fac08012b70 con 0x7fac1806e9b0 2024-09-12T03:24:40.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:40.496+0000 7fac0ffff640 1 --2- 172.21.15.27:0/1297543589 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fabf806c9a0 0x7fabf806ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:40.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:40.496+0000 7fac0ffff640 1 -- 172.21.15.27:0/1297543589 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(163..163 src has 1..163) ==== 4999+0+0 (secure 0 0 0) 0x7fac08089b00 con 0x7fac1806e9b0 2024-09-12T03:24:40.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:40.496+0000 7fac0e59c640 1 --2- 172.21.15.27:0/1297543589 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fabf806c9a0 0x7fabf806ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:40.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:40.497+0000 7fac0e59c640 1 --2- 172.21.15.27:0/1297543589 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fabf806c9a0 0x7fabf806ee60 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fac1814b880 tx=0x7fac00009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:40.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:40.501+0000 7fac0ffff640 1 -- 172.21.15.27:0/1297543589 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fac080597f0 con 0x7fac1806e9b0 2024-09-12T03:24:40.674 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:40.673+0000 7fac1d0f1640 1 -- 172.21.15.27:0/1297543589 --> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fac1806be10 con 0x7fabf806c9a0 2024-09-12T03:24:40.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:40.674+0000 7fac0ffff640 1 -- 172.21.15.27:0/1297543589 <== mgr.4113 v2:172.21.15.27:6836/1226920526 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13379 (secure 0 0 0) 0x7fac1806be10 con 0x7fabf806c9a0 2024-09-12T03:24:40.675 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:24:40.675 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2024-09-12T03:24:40.677 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:40.676+0000 7fac1d0f1640 1 -- 172.21.15.27:0/1297543589 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fabf806c9a0 msgr2=0x7fabf806ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:40.677 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:40.676+0000 7fac1d0f1640 1 --2- 172.21.15.27:0/1297543589 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fabf806c9a0 0x7fabf806ee60 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fac1814b880 tx=0x7fac00009040 comp rx=0 tx=0).stop 2024-09-12T03:24:40.677 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:40.676+0000 7fac1d0f1640 1 -- 172.21.15.27:0/1297543589 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fac1806e9b0 msgr2=0x7fac181f23b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:40.677 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:40.676+0000 7fac1d0f1640 1 --2- 172.21.15.27:0/1297543589 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fac1806e9b0 0x7fac181f23b0 secure :-1 s=READY pgs=1818 cs=0 l=1 rev1=1 crypto rx=0x7fac0800d5f0 tx=0x7fac0800dab0 comp rx=0 tx=0).stop 2024-09-12T03:24:40.677 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:40.676+0000 7fac1d0f1640 1 -- 172.21.15.27:0/1297543589 shutdown_connections 2024-09-12T03:24:40.677 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:40.676+0000 7fac1d0f1640 1 --2- 172.21.15.27:0/1297543589 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fac1815b2b0 0x7fac18149a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:40.678 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:40.676+0000 7fac1d0f1640 1 --2- 172.21.15.27:0/1297543589 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7fac18072d70 0x7fac1813fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:40.678 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:40.676+0000 7fac1d0f1640 1 --2- 172.21.15.27:0/1297543589 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7fabf806c9a0 0x7fabf806ee60 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:40.678 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:40.676+0000 7fac1d0f1640 1 --2- 172.21.15.27:0/1297543589 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fac1806e9b0 0x7fac181f23b0 unknown :-1 s=CLOSED pgs=1818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:40.678 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:40.676+0000 7fac1d0f1640 1 -- 172.21.15.27:0/1297543589 >> 172.21.15.27:0/1297543589 conn(0x7fac18062c60 msgr2=0x7fac180b0a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:40.678 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:40.676+0000 7fac1d0f1640 1 -- 172.21.15.27:0/1297543589 shutdown_connections 2024-09-12T03:24:40.678 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:40.676+0000 7fac1d0f1640 1 -- 172.21.15.27:0/1297543589 wait complete. 2024-09-12T03:24:40.688 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":229,"stamp":"2024-09-12T03:24:39.149311+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":130,"ondisk_log_size":130,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":291768,"kb_used_data":17288,"kb_used_omap":12,"kb_used_meta":274291,"kb_avail":754682952,"statfs":{"total":773094113280,"available":772795342848,"internally_reserved":0,"allocated":17702912,"data_stored":8479848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":280874544},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002757"},"pg_stats":[{"pgid":"1.0","version":"163'130","reported_seq":482,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-09-12T03:24:20.379424+0000","last_change":"2024-09-12T03:24:20.379424+0000","last_active":"2024-09-12T03:24:20.379424+0000","last_peered":"2024-09-12T03:24:20.379424+0000","last_clean":"2024-09-12T03:24:20.379424+0000","last_became_active":"2024-09-12T03:16:21.882207+0000","last_became_peered":"2024-09-12T03:16:21.882207+0000","last_unstale":"2024-09-12T03:24:20.379424+0000","last_undegraded":"2024-09-12T03:24:20.379424+0000","last_fullsized":"2024-09-12T03:24:20.379424+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"163'130","last_scrub_stamp":"2024-09-12T03:24:20.379295+0000","last_deep_scrub":"163'130","last_deep_scrub_stamp":"2024-09-12T03:24:20.379295+0000","last_clean_scrub_stamp":"2024-09-12T03:24:20.379295+0000","objects_scrubbed":2,"log_size":130,"log_dups_size":0,"ondisk_log_size":130,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:52:58.628046+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":130,"ondisk_log_size":130,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542245,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42100,"kb_used_data":2596,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94329740,"statfs":{"total":96636764160,"available":96593653760,"internally_reserved":0,"allocated":2658304,"data_stored":1502757,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":40434122},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542247,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37368,"kb_used_data":2016,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334472,"statfs":{"total":96636764160,"available":96598499328,"internally_reserved":0,"allocated":2064384,"data_stored":912389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542246,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":41524,"kb_used_data":2016,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330316,"statfs":{"total":96636764160,"available":96594243584,"internally_reserved":0,"allocated":2064384,"data_stored":912389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542245,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37368,"kb_used_data":2016,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334472,"statfs":{"total":96636764160,"available":96598499328,"internally_reserved":0,"allocated":2064384,"data_stored":912389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542246,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33204,"kb_used_data":2016,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338636,"statfs":{"total":96636764160,"available":96602763264,"internally_reserved":0,"allocated":2064384,"data_stored":912389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542246,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33212,"kb_used_data":2016,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338628,"statfs":{"total":96636764160,"available":96602755072,"internally_reserved":0,"allocated":2064384,"data_stored":912389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542247,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33212,"kb_used_data":2016,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338628,"statfs":{"total":96636764160,"available":96602755072,"internally_reserved":0,"allocated":2064384,"data_stored":912389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542247,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33780,"kb_used_data":2596,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338060,"statfs":{"total":96636764160,"available":96602173440,"internally_reserved":0,"allocated":2658304,"data_stored":1502757,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T03:24:40.689 DEBUG:teuthology.orchestra.run.smithi027:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-09-12T03:24:40.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:40.790+0000 7f0b820f1640 1 Processor -- start 2024-09-12T03:24:40.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:40.790+0000 7f0b820f1640 1 -- start start 2024-09-12T03:24:40.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:40.791+0000 7f0b820f1640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f0b7c15e990 0x7f0b7c0b7600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:40.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:40.791+0000 7f0b820f1640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f0b7c0b6230 0x7f0b7c0b6690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:40.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:40.791+0000 7f0b820f1640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f0b7c0b3600 0x7f0b7c0b3a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:40.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:40.791+0000 7f0b820f1640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f0b7c0b6bd0 con 0x7f0b7c0b6230 2024-09-12T03:24:40.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:40.791+0000 7f0b820f1640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f0b7c15ed70 con 0x7f0b7c0b3600 2024-09-12T03:24:40.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:40.791+0000 7f0b820f1640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f0b7c0b3fd0 con 0x7f0b7c15e990 2024-09-12T03:24:40.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:40.791+0000 7f0b7affd640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f0b7c0b6230 0x7f0b7c0b6690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:40.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:40.791+0000 7f0b7affd640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f0b7c0b6230 0x7f0b7c0b6690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38826/0 (socket says 172.21.15.27:38826) 2024-09-12T03:24:40.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:40.791+0000 7f0b7affd640 1 -- 172.21.15.27:0/638364307 learned_addr learned my addr 172.21.15.27:0/638364307 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:24:40.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:40.791+0000 7f0b7bfff640 1 --2- 172.21.15.27:0/638364307 >> v2:172.21.15.45:3300/0 conn(0x7f0b7c0b3600 0x7f0b7c0b3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:40.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:40.792+0000 7f0b7b7fe640 1 --2- 172.21.15.27:0/638364307 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f0b7c15e990 0x7f0b7c0b7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:40.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:40.792+0000 7f0b7affd640 1 -- 172.21.15.27:0/638364307 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f0b7c15e990 msgr2=0x7f0b7c0b7600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:24:40.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:40.792+0000 7f0b7affd640 1 --2- 172.21.15.27:0/638364307 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f0b7c15e990 0x7f0b7c0b7600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:40.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:40.792+0000 7f0b7affd640 1 -- 172.21.15.27:0/638364307 >> v2:172.21.15.45:3300/0 conn(0x7f0b7c0b3600 msgr2=0x7f0b7c0b3a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:24:40.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:40.792+0000 7f0b7affd640 1 --2- 172.21.15.27:0/638364307 >> v2:172.21.15.45:3300/0 conn(0x7f0b7c0b3600 0x7f0b7c0b3a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:40.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:40.792+0000 7f0b7affd640 1 -- 172.21.15.27:0/638364307 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b7c0d2a30 con 0x7f0b7c0b6230 2024-09-12T03:24:40.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:40.792+0000 7f0b7bfff640 1 --2- 172.21.15.27:0/638364307 >> v2:172.21.15.45:3300/0 conn(0x7f0b7c0b3600 0x7f0b7c0b3a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:24:40.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:40.792+0000 7f0b7affd640 1 --2- 172.21.15.27:0/638364307 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0b7c0b6230 0x7f0b7c0b6690 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7f0b7c06ceb0 tx=0x7f0b7002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e989dc008d719b55 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:40.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:40.792+0000 7f0b7a7fc640 1 -- 172.21.15.27:0/638364307 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b70037070 con 0x7f0b7c0b6230 2024-09-12T03:24:40.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:40.793+0000 7f0b7a7fc640 1 -- 172.21.15.27:0/638364307 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b70032440 con 0x7f0b7c0b6230 2024-09-12T03:24:40.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:40.793+0000 7f0b7a7fc640 1 -- 172.21.15.27:0/638364307 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b70032810 con 0x7f0b7c0b6230 2024-09-12T03:24:40.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:40.793+0000 7f0b820f1640 1 -- 172.21.15.27:0/638364307 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0b7c0b6230 msgr2=0x7f0b7c0b6690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:40.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:40.793+0000 7f0b820f1640 1 --2- 172.21.15.27:0/638364307 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0b7c0b6230 0x7f0b7c0b6690 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7f0b7c06ceb0 tx=0x7f0b7002a1e0 comp rx=0 tx=0).stop 2024-09-12T03:24:40.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:40.793+0000 7f0b820f1640 1 -- 172.21.15.27:0/638364307 shutdown_connections 2024-09-12T03:24:40.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:40.793+0000 7f0b820f1640 1 --2- 172.21.15.27:0/638364307 >> v2:172.21.15.45:3300/0 conn(0x7f0b7c0b3600 0x7f0b7c0b3a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:40.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:40.793+0000 7f0b820f1640 1 --2- 172.21.15.27:0/638364307 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0b7c0b6230 0x7f0b7c0b6690 unknown :-1 s=CLOSED pgs=1819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:40.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:40.793+0000 7f0b820f1640 1 --2- 172.21.15.27:0/638364307 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f0b7c15e990 0x7f0b7c0b7600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:40.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:40.793+0000 7f0b820f1640 1 -- 172.21.15.27:0/638364307 >> 172.21.15.27:0/638364307 conn(0x7f0b7c0d1230 msgr2=0x7f0b7c0d1640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:40.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:40.793+0000 7f0b820f1640 1 -- 172.21.15.27:0/638364307 shutdown_connections 2024-09-12T03:24:40.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:40.793+0000 7f0b820f1640 1 -- 172.21.15.27:0/638364307 wait complete. 2024-09-12T03:24:40.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:40.794+0000 7f0b820f1640 1 Processor -- start 2024-09-12T03:24:40.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:40.795+0000 7f0b820f1640 1 -- start start 2024-09-12T03:24:40.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:40.795+0000 7f0b820f1640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0b7c0b3600 0x7f0b7c151b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:40.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:40.795+0000 7f0b820f1640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0b7c0b6230 0x7f0b7c0cd160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:40.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:40.795+0000 7f0b820f1640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f0b7c15e990 0x7f0b7c0cd6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:40.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:40.795+0000 7f0b820f1640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f0b7c0d32d0 con 0x7f0b7c0b6230 2024-09-12T03:24:40.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:40.795+0000 7f0b820f1640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f0b7c0d2ff0 con 0x7f0b7c0b3600 2024-09-12T03:24:40.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:40.795+0000 7f0b820f1640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f0b7c0d35b0 con 0x7f0b7c15e990 2024-09-12T03:24:40.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:40.795+0000 7f0b7affd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0b7c0b6230 0x7f0b7c0cd160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:40.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:40.795+0000 7f0b7affd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0b7c0b6230 0x7f0b7c0cd160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38830/0 (socket says 172.21.15.27:38830) 2024-09-12T03:24:40.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:40.795+0000 7f0b7affd640 1 -- 172.21.15.27:0/2956977166 learned_addr learned my addr 172.21.15.27:0/2956977166 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:24:40.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:40.795+0000 7f0b7b7fe640 1 --2- 172.21.15.27:0/2956977166 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0b7c0b3600 0x7f0b7c151b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:40.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:40.796+0000 7f0b7bfff640 1 --2- 172.21.15.27:0/2956977166 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f0b7c15e990 0x7f0b7c0cd6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:40.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:40.796+0000 7f0b7affd640 1 -- 172.21.15.27:0/2956977166 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f0b7c15e990 msgr2=0x7f0b7c0cd6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:40.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:40.796+0000 7f0b7affd640 1 --2- 172.21.15.27:0/2956977166 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f0b7c15e990 0x7f0b7c0cd6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:40.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:40.796+0000 7f0b7affd640 1 -- 172.21.15.27:0/2956977166 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0b7c0b3600 msgr2=0x7f0b7c151b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:40.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:40.796+0000 7f0b7affd640 1 --2- 172.21.15.27:0/2956977166 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0b7c0b3600 0x7f0b7c151b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:40.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:40.796+0000 7f0b7affd640 1 -- 172.21.15.27:0/2956977166 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b7c160000 con 0x7f0b7c0b6230 2024-09-12T03:24:40.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:40.796+0000 7f0b7affd640 1 --2- 172.21.15.27:0/2956977166 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0b7c0b6230 0x7f0b7c0cd160 secure :-1 s=READY pgs=1820 cs=0 l=1 rev1=1 crypto rx=0x7f0b7002a220 tx=0x7f0b700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:40.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:40.796+0000 7f0b63fff640 1 -- 172.21.15.27:0/2956977166 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b70041070 con 0x7f0b7c0b6230 2024-09-12T03:24:40.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:40.796+0000 7f0b63fff640 1 -- 172.21.15.27:0/2956977166 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b7003c870 con 0x7f0b7c0b6230 2024-09-12T03:24:40.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:40.796+0000 7f0b820f1640 1 -- 172.21.15.27:0/2956977166 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b7c1602e0 con 0x7f0b7c0b6230 2024-09-12T03:24:40.801 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:40.796+0000 7f0b63fff640 1 -- 172.21.15.27:0/2956977166 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b70037040 con 0x7f0b7c0b6230 2024-09-12T03:24:40.801 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:40.796+0000 7f0b820f1640 1 -- 172.21.15.27:0/2956977166 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b7c1608a0 con 0x7f0b7c0b6230 2024-09-12T03:24:40.801 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:40.797+0000 7f0b820f1640 1 -- 172.21.15.27:0/2956977166 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b7c0b3a90 con 0x7f0b7c0b6230 2024-09-12T03:24:40.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:40.798+0000 7f0b63fff640 1 -- 172.21.15.27:0/2956977166 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 26) ==== 91518+0+0 (secure 0 0 0) 0x7f0b70044070 con 0x7f0b7c0b6230 2024-09-12T03:24:40.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:40.798+0000 7f0b63fff640 1 --2- 172.21.15.27:0/2956977166 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f0b6806c970 0x7f0b6806ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:40.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:40.798+0000 7f0b7b7fe640 1 --2- 172.21.15.27:0/2956977166 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f0b6806c970 0x7f0b6806ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:40.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:40.798+0000 7f0b63fff640 1 -- 172.21.15.27:0/2956977166 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(163..163 src has 1..163) ==== 4999+0+0 (secure 0 0 0) 0x7f0b700aa530 con 0x7f0b7c0b6230 2024-09-12T03:24:40.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:40.799+0000 7f0b7b7fe640 1 --2- 172.21.15.27:0/2956977166 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f0b6806c970 0x7f0b6806ee30 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f0b6c00aac0 tx=0x7f0b6c008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:40.803 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:40.803+0000 7f0b63fff640 1 -- 172.21.15.27:0/2956977166 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f0b7007a210 con 0x7f0b7c0b6230 2024-09-12T03:24:40.980 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:40.979+0000 7f0b820f1640 1 -- 172.21.15.27:0/2956977166 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) -- 0x7f0b7c0b4d40 con 0x7f0b7c0b6230 2024-09-12T03:24:40.984 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:40.982+0000 7f0b63fff640 1 -- 172.21.15.27:0/2956977166 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v32)=0 v32) ==== 123+0+0 (secure 0 0 0) 0x7f0b7007e510 con 0x7f0b7c0b6230 2024-09-12T03:24:40.984 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:40.983+0000 7f0b63fff640 1 -- 172.21.15.27:0/2956977166 <== mon.0 v2:172.21.15.27:3300/0 8 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f0b700b2020 con 0x7f0b7c0b6230 2024-09-12T03:24:40.986 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:40.985+0000 7f0b820f1640 1 -- 172.21.15.27:0/2956977166 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f0b6806c970 msgr2=0x7f0b6806ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:40.986 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:40.985+0000 7f0b820f1640 1 --2- 172.21.15.27:0/2956977166 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f0b6806c970 0x7f0b6806ee30 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f0b6c00aac0 tx=0x7f0b6c008040 comp rx=0 tx=0).stop 2024-09-12T03:24:40.986 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:40.985+0000 7f0b820f1640 1 -- 172.21.15.27:0/2956977166 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0b7c0b6230 msgr2=0x7f0b7c0cd160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:40.986 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:40.985+0000 7f0b820f1640 1 --2- 172.21.15.27:0/2956977166 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0b7c0b6230 0x7f0b7c0cd160 secure :-1 s=READY pgs=1820 cs=0 l=1 rev1=1 crypto rx=0x7f0b7002a220 tx=0x7f0b700046c0 comp rx=0 tx=0).stop 2024-09-12T03:24:40.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:40.985+0000 7f0b820f1640 1 -- 172.21.15.27:0/2956977166 shutdown_connections 2024-09-12T03:24:40.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:40.985+0000 7f0b820f1640 1 --2- 172.21.15.27:0/2956977166 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f0b7c15e990 0x7f0b7c0cd6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:40.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:40.985+0000 7f0b820f1640 1 --2- 172.21.15.27:0/2956977166 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f0b6806c970 0x7f0b6806ee30 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:40.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:40.985+0000 7f0b820f1640 1 --2- 172.21.15.27:0/2956977166 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0b7c0b6230 0x7f0b7c0cd160 unknown :-1 s=CLOSED pgs=1820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:40.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:40.985+0000 7f0b820f1640 1 --2- 172.21.15.27:0/2956977166 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0b7c0b3600 0x7f0b7c151b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:40.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:40.985+0000 7f0b820f1640 1 -- 172.21.15.27:0/2956977166 >> 172.21.15.27:0/2956977166 conn(0x7f0b7c0d1230 msgr2=0x7f0b7c0b9310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:40.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:40.985+0000 7f0b820f1640 1 -- 172.21.15.27:0/2956977166 shutdown_connections 2024-09-12T03:24:40.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:40.985+0000 7f0b820f1640 1 -- 172.21.15.27:0/2956977166 wait complete. 2024-09-12T03:24:40.998 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-12T03:24:40.998 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-12T03:24:41.100 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:41.099+0000 7f15166b7640 1 Processor -- start 2024-09-12T03:24:41.100 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:41.099+0000 7f15166b7640 1 -- start start 2024-09-12T03:24:41.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:41.100+0000 7f15166b7640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1510073960 0x7f1510162320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:41.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:41.100+0000 7f15166b7640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f1510162860 0x7f151015e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:41.102 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:41.100+0000 7f15166b7640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f15100d2240 con 0x7f151015e5f0 2024-09-12T03:24:41.102 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:41.100+0000 7f15166b7640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f15100d0da0 con 0x7f1510162860 2024-09-12T03:24:41.102 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:41.100+0000 7f15166b7640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f1510163090 con 0x7f1510073960 2024-09-12T03:24:41.102 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:41.100+0000 7f1514c54640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f1510162860 0x7f151015e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:41.102 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:41.100+0000 7f1514c54640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f1510162860 0x7f151015e9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38836/0 (socket says 172.21.15.27:38836) 2024-09-12T03:24:41.102 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:41.100+0000 7f1514c54640 1 -- 172.21.15.27:0/927675889 learned_addr learned my addr 172.21.15.27:0/927675889 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:24:41.102 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:41.100+0000 7f150f7fe640 1 --2- 172.21.15.27:0/927675889 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1510073960 0x7f1510162320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:41.102 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:41.100+0000 7f1514c54640 1 -- 172.21.15.27:0/927675889 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1510073960 msgr2=0x7f1510162320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:24:41.102 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:41.100+0000 7f1514c54640 1 --2- 172.21.15.27:0/927675889 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1510073960 0x7f1510162320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:41.102 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:41.100+0000 7f1514c54640 1 -- 172.21.15.27:0/927675889 >> v1:172.21.15.45:6789/0 conn(0x7f151015e5f0 legacy=0x7f1510073250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:41.102 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:41.100+0000 7f1514c54640 1 -- 172.21.15.27:0/927675889 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15100d0470 con 0x7f1510162860 2024-09-12T03:24:41.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:41.101+0000 7f1514c54640 1 --2- 172.21.15.27:0/927675889 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1510162860 0x7f151015e9d0 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7f1508009e50 tx=0x7f150802a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64c8b75257d5a353 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:41.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:41.101+0000 7f150effd640 1 -- 172.21.15.27:0/927675889 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1508036070 con 0x7f1510162860 2024-09-12T03:24:41.104 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:41.101+0000 7f150effd640 1 -- 172.21.15.27:0/927675889 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f1508031600 con 0x7f1510162860 2024-09-12T03:24:41.104 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:41.101+0000 7f150effd640 1 -- 172.21.15.27:0/927675889 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15080319f0 con 0x7f1510162860 2024-09-12T03:24:41.104 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:41.102+0000 7f15166b7640 1 -- 172.21.15.27:0/927675889 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1510162860 msgr2=0x7f151015e9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:41.104 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:41.102+0000 7f15166b7640 1 --2- 172.21.15.27:0/927675889 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1510162860 0x7f151015e9d0 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7f1508009e50 tx=0x7f150802a710 comp rx=0 tx=0).stop 2024-09-12T03:24:41.104 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:41.102+0000 7f15166b7640 1 -- 172.21.15.27:0/927675889 shutdown_connections 2024-09-12T03:24:41.104 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:41.102+0000 7f15166b7640 1 --2- 172.21.15.27:0/927675889 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1510162860 0x7f151015e9d0 unknown :-1 s=CLOSED pgs=1821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:41.104 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:41.102+0000 7f15166b7640 1 --2- 172.21.15.27:0/927675889 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1510073960 0x7f1510162320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:41.104 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:41.102+0000 7f15166b7640 1 -- 172.21.15.27:0/927675889 >> 172.21.15.27:0/927675889 conn(0x7f151006f300 msgr2=0x7f151006f710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:41.104 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:41.102+0000 7f15166b7640 1 -- 172.21.15.27:0/927675889 shutdown_connections 2024-09-12T03:24:41.104 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:41.102+0000 7f15166b7640 1 -- 172.21.15.27:0/927675889 wait complete. 2024-09-12T03:24:41.106 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:41.103+0000 7f15166b7640 1 Processor -- start 2024-09-12T03:24:41.106 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:41.103+0000 7f15166b7640 1 -- start start 2024-09-12T03:24:41.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:41.104+0000 7f15166b7640 1 --2- >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1510073960 0x7f15100ac9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:41.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:41.104+0000 7f15166b7640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f151015e5f0 0x7f151014c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:41.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:41.104+0000 7f15166b7640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1510162860 0x7f1510160d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:41.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:41.104+0000 7f15166b7640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f15100b5620 con 0x7f1510162860 2024-09-12T03:24:41.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:41.104+0000 7f15166b7640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f15100d0a30 con 0x7f151015e5f0 2024-09-12T03:24:41.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:41.104+0000 7f15166b7640 1 -- --> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] -- mon_getmap magic: 0 -- 0x7f1510162cf0 con 0x7f1510073960 2024-09-12T03:24:41.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:41.104+0000 7f1514c54640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1510162860 0x7f1510160d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:41.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:41.104+0000 7f1514c54640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1510162860 0x7f1510160d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38838/0 (socket says 172.21.15.27:38838) 2024-09-12T03:24:41.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:41.104+0000 7f1514c54640 1 -- 172.21.15.27:0/381263586 learned_addr learned my addr 172.21.15.27:0/381263586 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-12T03:24:41.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:41.104+0000 7f150f7fe640 1 --2- 172.21.15.27:0/381263586 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f151015e5f0 0x7f151014c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:41.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:41.104+0000 7f150ffff640 1 --2- 172.21.15.27:0/381263586 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1510073960 0x7f15100ac9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:41.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:41.104+0000 7f1514c54640 1 -- 172.21.15.27:0/381263586 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1510073960 msgr2=0x7f15100ac9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:41.108 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:41.104+0000 7f1514c54640 1 --2- 172.21.15.27:0/381263586 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1510073960 0x7f15100ac9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:41.108 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:41.104+0000 7f1514c54640 1 -- 172.21.15.27:0/381263586 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f151015e5f0 msgr2=0x7f151014c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:41.108 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:41.104+0000 7f1514c54640 1 --2- 172.21.15.27:0/381263586 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f151015e5f0 0x7f151014c5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:41.108 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:41.105+0000 7f1514c54640 1 -- 172.21.15.27:0/381263586 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f151014cfa0 con 0x7f1510162860 2024-09-12T03:24:41.108 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:41.105+0000 7f1514c54640 1 --2- 172.21.15.27:0/381263586 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1510162860 0x7f1510160d30 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 crypto rx=0x7f1508034040 tx=0x7f1508004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:41.108 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:41.105+0000 7f150cff9640 1 -- 172.21.15.27:0/381263586 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f150803f020 con 0x7f1510162860 2024-09-12T03:24:41.108 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:41.105+0000 7f150cff9640 1 -- 172.21.15.27:0/381263586 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f1508037040 con 0x7f1510162860 2024-09-12T03:24:41.108 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:41.105+0000 7f15166b7640 1 -- 172.21.15.27:0/381263586 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f151014d280 con 0x7f1510162860 2024-09-12T03:24:41.108 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:41.105+0000 7f150cff9640 1 -- 172.21.15.27:0/381263586 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1508036070 con 0x7f1510162860 2024-09-12T03:24:41.108 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:41.105+0000 7f15166b7640 1 -- 172.21.15.27:0/381263586 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f151014d840 con 0x7f1510162860 2024-09-12T03:24:41.109 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:41.106+0000 7f15166b7640 1 -- 172.21.15.27:0/381263586 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1510162320 con 0x7f1510162860 2024-09-12T03:24:41.109 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:41.107+0000 7f150cff9640 1 -- 172.21.15.27:0/381263586 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 26) ==== 91518+0+0 (secure 0 0 0) 0x7f1508042020 con 0x7f1510162860 2024-09-12T03:24:41.109 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:41.107+0000 7f150cff9640 1 --2- 172.21.15.27:0/381263586 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f14fc06c9a0 0x7f14fc06ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:41.109 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:41.107+0000 7f150cff9640 1 -- 172.21.15.27:0/381263586 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(163..163 src has 1..163) ==== 4999+0+0 (secure 0 0 0) 0x7f15080a9d80 con 0x7f1510162860 2024-09-12T03:24:41.110 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:41.107+0000 7f150ffff640 1 --2- 172.21.15.27:0/381263586 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f14fc06c9a0 0x7f14fc06ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:41.110 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:41.108+0000 7f150ffff640 1 --2- 172.21.15.27:0/381263586 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f14fc06c9a0 0x7f14fc06ee60 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f1500007fd0 tx=0x7f1500009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:41.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:41.112+0000 7f150cff9640 1 -- 172.21.15.27:0/381263586 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f1508049030 con 0x7f1510162860 2024-09-12T03:24:41.322 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:41.321+0000 7f15166b7640 1 -- 172.21.15.27:0/381263586 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f15100b6040 con 0x7f1510162860 2024-09-12T03:24:41.323 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:41.322+0000 7f150cff9640 1 -- 172.21.15.27:0/381263586 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 102 v102) ==== 95+0+3913 (secure 0 0 0) 0x7f150807a150 con 0x7f1510162860 2024-09-12T03:24:41.323 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-12T03:24:41.323 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":102,"btime":"2024-09-12T03:24:10:247239+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7163,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/2493793567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2493793567},{"type":"v1","addr":"172.21.15.45:6835","nonce":2493793567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":96},{"gid":8056,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/1643775717","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":1643775717},{"type":"v1","addr":"172.21.15.45:6834","nonce":1643775717}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95},{"gid":8967,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6834/4129691413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":4129691413},{"type":"v1","addr":"172.21.15.27:6834","nonce":4129691413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95},{"gid":9255,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/3312996947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":3312996947},{"type":"v1","addr":"172.21.15.27:6835","nonce":3312996947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":102}],"filesystems":[]} 2024-09-12T03:24:41.323 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 102 2024-09-12T03:24:41.325 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:41.323+0000 7f15166b7640 1 -- 172.21.15.27:0/381263586 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f14fc06c9a0 msgr2=0x7f14fc06ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:41.325 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:41.323+0000 7f15166b7640 1 --2- 172.21.15.27:0/381263586 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f14fc06c9a0 0x7f14fc06ee60 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f1500007fd0 tx=0x7f1500009040 comp rx=0 tx=0).stop 2024-09-12T03:24:41.325 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:41.323+0000 7f15166b7640 1 -- 172.21.15.27:0/381263586 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1510162860 msgr2=0x7f1510160d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:41.325 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:41.323+0000 7f15166b7640 1 --2- 172.21.15.27:0/381263586 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1510162860 0x7f1510160d30 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 crypto rx=0x7f1508034040 tx=0x7f1508004a40 comp rx=0 tx=0).stop 2024-09-12T03:24:41.325 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:41.324+0000 7f15166b7640 1 -- 172.21.15.27:0/381263586 shutdown_connections 2024-09-12T03:24:41.325 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:41.324+0000 7f15166b7640 1 --2- 172.21.15.27:0/381263586 >> [v2:172.21.15.27:6836/1226920526,v1:172.21.15.27:6837/1226920526] conn(0x7f14fc06c9a0 0x7f14fc06ee60 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:41.325 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:41.324+0000 7f15166b7640 1 --2- 172.21.15.27:0/381263586 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1510162860 0x7f1510160d30 unknown :-1 s=CLOSED pgs=1822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:41.325 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:41.324+0000 7f15166b7640 1 --2- 172.21.15.27:0/381263586 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f151015e5f0 0x7f151014c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:41.325 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:41.324+0000 7f15166b7640 1 --2- 172.21.15.27:0/381263586 >> [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] conn(0x7f1510073960 0x7f15100ac9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:41.326 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:41.324+0000 7f15166b7640 1 -- 172.21.15.27:0/381263586 >> 172.21.15.27:0/381263586 conn(0x7f151006f300 msgr2=0x7f151015ebe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:41.326 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:41.324+0000 7f15166b7640 1 -- 172.21.15.27:0/381263586 shutdown_connections 2024-09-12T03:24:41.326 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-12T03:24:41.324+0000 7f15166b7640 1 -- 172.21.15.27:0/381263586 wait complete. 2024-09-12T03:24:41.336 DEBUG:tasks.cephfs.filesystem:already dead... 2024-09-12T03:24:41.336 INFO:teuthology.misc:Shutting down mds daemons... 2024-09-12T03:24:41.336 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-12T03:24:41.336 INFO:teuthology.orchestra.run:waiting for 300 2024-09-12T03:24:41.345 INFO:tasks.ceph.mds.a:Stopped 2024-09-12T03:24:41.346 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-12T03:24:41.346 INFO:teuthology.orchestra.run:waiting for 300 2024-09-12T03:24:41.396 INFO:tasks.ceph.mds.c:Stopped 2024-09-12T03:24:41.397 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-12T03:24:41.397 INFO:teuthology.orchestra.run:waiting for 300 2024-09-12T03:24:41.409 INFO:tasks.ceph.mds.b:Stopped 2024-09-12T03:24:41.409 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-12T03:24:41.409 INFO:teuthology.orchestra.run:waiting for 300 2024-09-12T03:24:41.461 INFO:tasks.ceph.mds.d:Stopped 2024-09-12T03:24:41.462 INFO:teuthology.misc:Shutting down osd daemons... 2024-09-12T03:24:41.462 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-09-12T03:24:41.462 INFO:teuthology.orchestra.run:waiting for 300 2024-09-12T03:24:41.491 INFO:tasks.ceph.osd.0:Stopped 2024-09-12T03:24:41.492 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-09-12T03:24:41.492 INFO:teuthology.orchestra.run:waiting for 300 2024-09-12T03:24:41.557 INFO:tasks.ceph.osd.1:Stopped 2024-09-12T03:24:41.557 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-09-12T03:24:41.557 INFO:teuthology.orchestra.run:waiting for 300 2024-09-12T03:24:41.624 INFO:tasks.ceph.osd.2:Stopped 2024-09-12T03:24:41.625 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-09-12T03:24:41.625 INFO:teuthology.orchestra.run:waiting for 300 2024-09-12T03:24:41.700 INFO:tasks.ceph.osd.3:Stopped 2024-09-12T03:24:41.701 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-09-12T03:24:41.701 INFO:teuthology.orchestra.run:waiting for 300 2024-09-12T03:24:41.733 INFO:tasks.ceph.osd.4:Stopped 2024-09-12T03:24:41.733 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-09-12T03:24:41.733 INFO:teuthology.orchestra.run:waiting for 300 2024-09-12T03:24:41.812 INFO:tasks.ceph.osd.5:Stopped 2024-09-12T03:24:41.813 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-09-12T03:24:41.813 INFO:teuthology.orchestra.run:waiting for 300 2024-09-12T03:24:41.881 INFO:tasks.ceph.osd.6:Stopped 2024-09-12T03:24:41.882 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-09-12T03:24:41.882 INFO:teuthology.orchestra.run:waiting for 300 2024-09-12T03:24:41.960 INFO:tasks.ceph.osd.7:Stopped 2024-09-12T03:24:41.960 INFO:teuthology.misc:Shutting down mgr daemons... 2024-09-12T03:24:41.960 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-09-12T03:24:41.960 INFO:teuthology.orchestra.run:waiting for 300 2024-09-12T03:24:41.990 INFO:tasks.ceph.mgr.y:Stopped 2024-09-12T03:24:41.990 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-09-12T03:24:41.990 INFO:teuthology.orchestra.run:waiting for 300 2024-09-12T03:24:42.030 INFO:tasks.ceph.mgr.x:Stopped 2024-09-12T03:24:42.030 INFO:teuthology.misc:Shutting down mon daemons... 2024-09-12T03:24:42.031 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-09-12T03:24:42.031 INFO:teuthology.orchestra.run:waiting for 300 2024-09-12T03:24:42.042 INFO:tasks.ceph.mon.a:Stopped 2024-09-12T03:24:42.042 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-09-12T03:24:42.043 INFO:teuthology.orchestra.run:waiting for 300 2024-09-12T03:24:42.084 INFO:tasks.ceph.mon.b:Stopped 2024-09-12T03:24:42.085 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-09-12T03:24:42.085 INFO:teuthology.orchestra.run:waiting for 300 2024-09-12T03:24:42.138 INFO:tasks.ceph.mon.c:Stopped 2024-09-12T03:24:42.138 INFO:tasks.ceph:Checking cluster log for badness... 2024-09-12T03:24:42.139 DEBUG:teuthology.orchestra.run.smithi027:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'OSD full dropping all updates' | egrep -v 'OSD near full' | egrep -v 'pausewr flag' | egrep -v 'failsafe engaged, dropping updates' | egrep -v 'failsafe disengaged, no longer dropping' | egrep -v 'is full \(reached quota' | egrep -v POOL_FULL | egrep -v POOL_BACKFILLFULL | head -n 1 2024-09-12T03:24:42.179 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi027.front.sepia.ceph.com 2024-09-12T03:24:42.179 DEBUG:teuthology.orchestra.run.smithi027:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-09-12T03:24:42.400 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi027.front.sepia.ceph.com 2024-09-12T03:24:42.401 DEBUG:teuthology.orchestra.run.smithi027:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-09-12T03:24:42.492 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi027.front.sepia.ceph.com 2024-09-12T03:24:42.492 DEBUG:teuthology.orchestra.run.smithi027:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-09-12T03:24:42.542 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi027.front.sepia.ceph.com 2024-09-12T03:24:42.542 DEBUG:teuthology.orchestra.run.smithi027:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-09-12T03:24:42.625 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi045.front.sepia.ceph.com 2024-09-12T03:24:42.625 DEBUG:teuthology.orchestra.run.smithi045:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-09-12T03:24:42.816 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi045.front.sepia.ceph.com 2024-09-12T03:24:42.816 DEBUG:teuthology.orchestra.run.smithi045:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-09-12T03:24:42.905 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi045.front.sepia.ceph.com 2024-09-12T03:24:42.906 DEBUG:teuthology.orchestra.run.smithi045:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-09-12T03:24:42.962 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi045.front.sepia.ceph.com 2024-09-12T03:24:42.963 DEBUG:teuthology.orchestra.run.smithi045:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-09-12T03:24:43.023 INFO:tasks.ceph:Archiving mon data... 2024-09-12T03:24:43.024 DEBUG:teuthology.misc:Transferring archived files from smithi027:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901674/data/mon.a.tgz 2024-09-12T03:24:43.025 DEBUG:teuthology.orchestra.run.smithi027:> mktemp 2024-09-12T03:24:43.041 INFO:teuthology.orchestra.run.smithi027.stdout:/tmp/tmp.Nsrw5BrOo2 2024-09-12T03:24:43.041 DEBUG:teuthology.orchestra.run.smithi027:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.Nsrw5BrOo2 2024-09-12T03:24:43.335 DEBUG:teuthology.orchestra.run.smithi027:> sudo chmod 0666 /tmp/tmp.Nsrw5BrOo2 2024-09-12T03:24:43.420 DEBUG:teuthology.orchestra.remote:smithi027:/tmp/tmp.Nsrw5BrOo2 is 1MB 2024-09-12T03:24:43.529 DEBUG:teuthology.orchestra.run.smithi027:> rm -fr /tmp/tmp.Nsrw5BrOo2 2024-09-12T03:24:43.544 DEBUG:teuthology.misc:Transferring archived files from smithi045:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901674/data/mon.b.tgz 2024-09-12T03:24:43.544 DEBUG:teuthology.orchestra.run.smithi045:> mktemp 2024-09-12T03:24:43.562 INFO:teuthology.orchestra.run.smithi045.stdout:/tmp/tmp.a3gw62DOeB 2024-09-12T03:24:43.563 DEBUG:teuthology.orchestra.run.smithi045:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.a3gw62DOeB 2024-09-12T03:24:43.864 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod 0666 /tmp/tmp.a3gw62DOeB 2024-09-12T03:24:43.950 DEBUG:teuthology.orchestra.remote:smithi045:/tmp/tmp.a3gw62DOeB is 1MB 2024-09-12T03:24:44.059 DEBUG:teuthology.orchestra.run.smithi045:> rm -fr /tmp/tmp.a3gw62DOeB 2024-09-12T03:24:44.075 DEBUG:teuthology.misc:Transferring archived files from smithi045:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901674/data/mon.c.tgz 2024-09-12T03:24:44.075 DEBUG:teuthology.orchestra.run.smithi045:> mktemp 2024-09-12T03:24:44.130 INFO:teuthology.orchestra.run.smithi045.stdout:/tmp/tmp.s8zqqqHrgH 2024-09-12T03:24:44.130 DEBUG:teuthology.orchestra.run.smithi045:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.s8zqqqHrgH 2024-09-12T03:24:44.411 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod 0666 /tmp/tmp.s8zqqqHrgH 2024-09-12T03:24:44.496 DEBUG:teuthology.orchestra.remote:smithi045:/tmp/tmp.s8zqqqHrgH is 1MB 2024-09-12T03:24:44.627 DEBUG:teuthology.orchestra.run.smithi045:> rm -fr /tmp/tmp.s8zqqqHrgH 2024-09-12T03:24:44.643 INFO:tasks.ceph:Cleaning ceph cluster... 2024-09-12T03:24:44.643 DEBUG:teuthology.orchestra.run.smithi027:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-12T03:24:44.646 DEBUG:teuthology.orchestra.run.smithi045:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-12T03:24:44.735 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-12T03:24:44.763 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-12T03:24:44.763 INFO:tasks.ceph:Archiving crash dumps... 2024-09-12T03:24:44.765 DEBUG:teuthology.misc:Transferring archived files from smithi027:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901674/remote/smithi027/crash 2024-09-12T03:24:44.766 DEBUG:teuthology.orchestra.run.smithi027:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-12T03:24:44.806 DEBUG:teuthology.misc:Transferring archived files from smithi045:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901674/remote/smithi045/crash 2024-09-12T03:24:44.807 DEBUG:teuthology.orchestra.run.smithi045:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-12T03:24:44.832 INFO:tasks.ceph:Shutting down logrotate 2024-09-12T03:24:44.832 DEBUG:teuthology.orchestra.run.smithi027:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-12T03:24:44.851 DEBUG:teuthology.orchestra.run.smithi045:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-12T03:24:44.918 DEBUG:teuthology.orchestra.run.smithi027:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-09-12T03:24:44.948 DEBUG:teuthology.orchestra.run.smithi045:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-09-12T03:24:44.973 INFO:tasks.ceph:Compressing logs... 2024-09-12T03:24:44.974 DEBUG:teuthology.orchestra.run.smithi027:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-12T03:24:44.990 DEBUG:teuthology.orchestra.run.smithi045:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-12T03:24:45.015 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64797.log 2024-09-12T03:24:45.016 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-09-12T03:24:45.070 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.47777.log 2024-09-12T03:24:45.070 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.46303.log 2024-09-12T03:24:45.070 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.1.47777.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-09-12T03:24:45.070 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.47777.log.gz 2024-09-12T03:24:45.070 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.1.46303.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.44861.log 2024-09-12T03:24:45.070 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.46303.log.gz 2024-09-12T03:24:45.070 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2024-09-12T03:24:45.071 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.1.44861.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.3.42018.log 2024-09-12T03:24:45.071 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.44861.log.gz 2024-09-12T03:24:45.071 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5/var/log/ceph/ceph-mds.b.log: --verbose -- /var/log/ceph/ceph-client.1.43691.log 2024-09-12T03:24:45.071 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.3.42018.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-09-12T03:24:45.071 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.3.42018.log.gz 2024-09-12T03:24:45.071 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.1.43691.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.43691.log.gz 2024-09-12T03:24:45.071 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-12T03:24:45.071 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.46782.log 2024-09-12T03:24:45.071 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.43118.log 2024-09-12T03:24:45.071 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.1.46782.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36499.log 2024-09-12T03:24:45.072 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.46782.log.gz 2024-09-12T03:24:45.072 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.1.43118.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36539.log 2024-09-12T03:24:45.072 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.43118.log.gz 2024-09-12T03:24:45.072 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.36499.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.44241.log 2024-09-12T03:24:45.072 INFO:teuthology.orchestra.run.smithi045.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.36499.log.gz 2024-09-12T03:24:45.072 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.36539.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-09-12T03:24:45.072 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.1.44241.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36419.log 2024-09-12T03:24:45.072 INFO:teuthology.orchestra.run.smithi045.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.36539.log.gz 0.0% -- replaced with /var/log/ceph/ceph-client.1.44241.log.gz 2024-09-12T03:24:45.072 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-12T03:24:45.073 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.45822.log 2024-09-12T03:24:45.073 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.36419.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36670.log 2024-09-12T03:24:45.073 INFO:teuthology.orchestra.run.smithi045.stderr: 89.0%/var/log/ceph/ceph-client.1.45822.log: -- replaced with /var/log/ceph/ceph-client.admin.36419.log.gz 2024-09-12T03:24:45.073 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.45822.log.gz 2024-09-12T03:24:45.073 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.45340.log 2024-09-12T03:24:45.073 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.36670.log: -- /var/log/ceph/ceph-osd.6.log 2024-09-12T03:24:45.073 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.1.45340.log: 86.9% 0.0% -- replaced with /var/log/ceph/ceph-client.1.45340.log.gz 2024-09-12T03:24:45.073 INFO:teuthology.orchestra.run.smithi045.stderr: -- replaced with /var/log/ceph/ceph-client.admin.36670.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-09-12T03:24:45.073 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-12T03:24:45.073 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-12T03:24:45.073 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-osd.6.log: /var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36459.log 2024-09-12T03:24:45.073 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph.log: 93.9% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-12T03:24:45.073 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2024-09-12T03:24:45.074 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.36459.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.36459.log.gz 2024-09-12T03:24:45.074 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-09-12T03:24:45.074 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-mds.d.log: 92.5% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-12T03:24:45.074 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36583.log 2024-09-12T03:24:45.078 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36379.log 2024-09-12T03:24:45.078 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.36583.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.36583.log.gz 2024-09-12T03:24:45.083 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-09-12T03:24:45.094 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.36379.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.41620.log 2024-09-12T03:24:45.094 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-mon.b.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.36379.log.gz 2024-09-12T03:24:45.111 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.42601.log 2024-09-12T03:24:45.111 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.1.41620.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.41620.log.gz 2024-09-12T03:24:45.121 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36630.log 2024-09-12T03:24:45.121 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.1.42601.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.42601.log.gz 2024-09-12T03:24:45.126 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.47285.log 2024-09-12T03:24:45.144 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.36630.log: /var/log/ceph/ceph-client.1.47285.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.36630.log.gz 2024-09-12T03:24:45.149 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.47285.log.gz 2024-09-12T03:24:45.165 INFO:teuthology.orchestra.run.smithi045.stderr: 93.4% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2024-09-12T03:24:45.196 INFO:teuthology.orchestra.run.smithi045.stderr: 93.4% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2024-09-12T03:24:45.208 INFO:teuthology.orchestra.run.smithi045.stderr: 92.4% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-09-12T03:24:45.368 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.64797.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49250.log 2024-09-12T03:24:45.368 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-osd.0.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.64797.log.gz 2024-09-12T03:24:45.368 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72969.log 2024-09-12T03:24:45.368 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.49250.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73899.log 2024-09-12T03:24:45.368 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78576.log 2024-09-12T03:24:45.369 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.72969.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63115.log 2024-09-12T03:24:45.369 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.73899.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.49250.log.gz 2024-09-12T03:24:45.369 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.78576.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55950.log 2024-09-12T03:24:45.369 INFO:teuthology.orchestra.run.smithi027.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.72969.log.gz 2024-09-12T03:24:45.369 INFO:teuthology.orchestra.run.smithi027.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.73899.log.gz 2024-09-12T03:24:45.369 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.63115.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.78576.log.gz 2024-09-12T03:24:45.369 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49208.log 2024-09-12T03:24:45.369 INFO:teuthology.orchestra.run.smithi027.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.63115.log.gz 2024-09-12T03:24:45.369 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.55950.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70629.log 2024-09-12T03:24:45.369 INFO:teuthology.orchestra.run.smithi027.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.55950.log.gz 2024-09-12T03:24:45.369 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.49208.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42170.log 2024-09-12T03:24:45.369 INFO:teuthology.orchestra.run.smithi027.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.49208.log.gz 2024-09-12T03:24:45.370 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.70629.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59729.log 2024-09-12T03:24:45.370 INFO:teuthology.orchestra.run.smithi027.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.70629.log.gz 2024-09-12T03:24:45.370 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.42170.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65672.log 2024-09-12T03:24:45.370 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.59729.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.42170.log.gz 2024-09-12T03:24:45.370 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69482.log 2024-09-12T03:24:45.370 INFO:teuthology.orchestra.run.smithi027.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.59729.log.gz 2024-09-12T03:24:45.370 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.65672.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60651.log 2024-09-12T03:24:45.370 INFO:teuthology.orchestra.run.smithi027.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.65672.log.gz 2024-09-12T03:24:45.370 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.69482.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48998.log 2024-09-12T03:24:45.370 INFO:teuthology.orchestra.run.smithi027.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.69482.log.gz 2024-09-12T03:24:45.370 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.60651.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62946.log 2024-09-12T03:24:45.370 INFO:teuthology.orchestra.run.smithi027.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.60651.log.gz 2024-09-12T03:24:45.371 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.48998.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56391.log 2024-09-12T03:24:45.371 INFO:teuthology.orchestra.run.smithi027.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.48998.log.gz 2024-09-12T03:24:45.371 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.62946.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65317.log 2024-09-12T03:24:45.371 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.56391.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.62946.log.gz 2024-09-12T03:24:45.371 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55698.log 2024-09-12T03:24:45.371 INFO:teuthology.orchestra.run.smithi027.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.56391.log.gz 2024-09-12T03:24:45.371 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.65317.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80860.log 2024-09-12T03:24:45.371 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.55698.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43248.log 2024-09-12T03:24:45.371 INFO:teuthology.orchestra.run.smithi027.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.65317.log.gz 2024-09-12T03:24:45.371 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.80860.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.55698.log.gz 2024-09-12T03:24:45.371 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77317.log 2024-09-12T03:24:45.371 INFO:teuthology.orchestra.run.smithi027.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.80860.log.gz 2024-09-12T03:24:45.371 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.43248.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64918.log 2024-09-12T03:24:45.372 INFO:teuthology.orchestra.run.smithi027.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.43248.log.gz 2024-09-12T03:24:45.372 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.77317.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68703.log 2024-09-12T03:24:45.372 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.64918.log: gzip 89.3% -5 -- replaced with /var/log/ceph/ceph-client.admin.77317.log.gz --verbose 2024-09-12T03:24:45.372 INFO:teuthology.orchestra.run.smithi027.stderr: -- /var/log/ceph/ceph-client.admin.52469.log 2024-09-12T03:24:45.372 INFO:teuthology.orchestra.run.smithi027.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.64918.log.gz 2024-09-12T03:24:45.372 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.68703.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53150.log 2024-09-12T03:24:45.372 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.52469.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.68703.log.gz 2024-09-12T03:24:45.372 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53234.log 2024-09-12T03:24:45.372 INFO:teuthology.orchestra.run.smithi027.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.52469.log.gz 2024-09-12T03:24:45.372 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.53150.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66405.log 2024-09-12T03:24:45.372 INFO:teuthology.orchestra.run.smithi027.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.53150.log.gz 2024-09-12T03:24:45.372 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.53234.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50765.log 2024-09-12T03:24:45.373 INFO:teuthology.orchestra.run.smithi027.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.53234.log.gz 2024-09-12T03:24:45.373 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.66405.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71016.log 2024-09-12T03:24:45.373 INFO:teuthology.orchestra.run.smithi027.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.66405.log.gz 2024-09-12T03:24:45.373 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.50765.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.67251.log 2024-09-12T03:24:45.373 INFO:teuthology.orchestra.run.smithi027.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.50765.log.gz 2024-09-12T03:24:45.373 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.71016.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72658.log 2024-09-12T03:24:45.373 INFO:teuthology.orchestra.run.smithi027.stderr: 85.7% -- replaced with /var/log/ceph/ceph-client.admin.71016.log.gz 2024-09-12T03:24:45.373 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.0.67251.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55780.log 2024-09-12T03:24:45.373 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.67251.log.gz 2024-09-12T03:24:45.373 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.72658.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66493.log 2024-09-12T03:24:45.373 INFO:teuthology.orchestra.run.smithi027.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.72658.log.gz 2024-09-12T03:24:45.373 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.55780.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70588.log 2024-09-12T03:24:45.373 INFO:teuthology.orchestra.run.smithi027.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.55780.log.gz 2024-09-12T03:24:45.374 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61531.log 2024-09-12T03:24:45.374 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.66493.log: /var/log/ceph/ceph-client.admin.70588.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80650.log 2024-09-12T03:24:45.374 INFO:teuthology.orchestra.run.smithi027.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.66493.log.gz 2024-09-12T03:24:45.374 INFO:teuthology.orchestra.run.smithi027.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.70588.log.gz 2024-09-12T03:24:45.374 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.61531.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50687.log 2024-09-12T03:24:45.374 INFO:teuthology.orchestra.run.smithi027.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.61531.log.gz 2024-09-12T03:24:45.374 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.80650.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56162.log 2024-09-12T03:24:45.374 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.50687.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.80650.log.gz 2024-09-12T03:24:45.374 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78362.log 2024-09-12T03:24:45.374 INFO:teuthology.orchestra.run.smithi027.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.50687.log.gz 2024-09-12T03:24:45.374 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.56162.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44248.log 2024-09-12T03:24:45.374 INFO:teuthology.orchestra.run.smithi027.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.56162.log.gz 2024-09-12T03:24:45.375 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.78362.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65026.log 2024-09-12T03:24:45.375 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.44248.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.78362.log.gz 2024-09-12T03:24:45.375 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72454.log 2024-09-12T03:24:45.375 INFO:teuthology.orchestra.run.smithi027.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.44248.log.gz 2024-09-12T03:24:45.375 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.65026.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69237.log 2024-09-12T03:24:45.375 INFO:teuthology.orchestra.run.smithi027.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.65026.log.gz 2024-09-12T03:24:45.375 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.72454.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65275.log 2024-09-12T03:24:45.375 INFO:teuthology.orchestra.run.smithi027.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.72454.log.gz 2024-09-12T03:24:45.375 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.69237.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71573.log 2024-09-12T03:24:45.375 INFO:teuthology.orchestra.run.smithi027.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.69237.log.gz 2024-09-12T03:24:45.375 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.65275.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53756.log 2024-09-12T03:24:45.375 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.71573.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.65275.log.gz 2024-09-12T03:24:45.376 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67912.log 2024-09-12T03:24:45.376 INFO:teuthology.orchestra.run.smithi027.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.71573.log.gz/var/log/ceph/ceph-client.admin.53756.log: 2024-09-12T03:24:45.376 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59946.log 2024-09-12T03:24:45.376 INFO:teuthology.orchestra.run.smithi027.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.53756.log.gz 2024-09-12T03:24:45.376 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.67912.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58054.log 2024-09-12T03:24:45.376 INFO:teuthology.orchestra.run.smithi027.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.67912.log.gz 2024-09-12T03:24:45.376 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.59946.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52282.log 2024-09-12T03:24:45.376 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.58054.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.59946.log.gz 2024-09-12T03:24:45.376 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54131.log 2024-09-12T03:24:45.376 INFO:teuthology.orchestra.run.smithi027.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.58054.log.gz 2024-09-12T03:24:45.376 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.52282.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77672.log 2024-09-12T03:24:45.376 INFO:teuthology.orchestra.run.smithi027.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.52282.log.gz 2024-09-12T03:24:45.377 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.54131.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70463.log 2024-09-12T03:24:45.377 INFO:teuthology.orchestra.run.smithi027.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.54131.log.gz 2024-09-12T03:24:45.377 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.77672.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60735.log 2024-09-12T03:24:45.377 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.70463.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.77672.log.gz 2024-09-12T03:24:45.377 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43253.log 2024-09-12T03:24:45.377 INFO:teuthology.orchestra.run.smithi027.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.70463.log.gz 2024-09-12T03:24:45.377 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.60735.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74702.log 2024-09-12T03:24:45.377 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.43253.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.60735.log.gz 2024-09-12T03:24:45.377 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46715.log 2024-09-12T03:24:45.377 INFO:teuthology.orchestra.run.smithi027.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.43253.log.gz 2024-09-12T03:24:45.377 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.74702.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75699.log 2024-09-12T03:24:45.377 INFO:teuthology.orchestra.run.smithi027.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.74702.log.gz 2024-09-12T03:24:45.377 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.46715.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69058.log 2024-09-12T03:24:45.378 INFO:teuthology.orchestra.run.smithi027.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.46715.log.gz/var/log/ceph/ceph-client.admin.75699.log: 2024-09-12T03:24:45.378 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49124.log 2024-09-12T03:24:45.378 INFO:teuthology.orchestra.run.smithi027.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.75699.log.gz 2024-09-12T03:24:45.378 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.69058.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47447.log 2024-09-12T03:24:45.378 INFO:teuthology.orchestra.run.smithi027.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.69058.log.gz 2024-09-12T03:24:45.378 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.49124.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53714.log 2024-09-12T03:24:45.378 INFO:teuthology.orchestra.run.smithi027.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.49124.log.gz 2024-09-12T03:24:45.378 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.47447.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57102.log 2024-09-12T03:24:45.378 INFO:teuthology.orchestra.run.smithi027.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.47447.log.gz 2024-09-12T03:24:45.378 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.53714.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66787.log 2024-09-12T03:24:45.378 INFO:teuthology.orchestra.run.smithi027.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.53714.log.gz 2024-09-12T03:24:45.379 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.57102.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75825.log 2024-09-12T03:24:45.379 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.66787.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43575.log 2024-09-12T03:24:45.379 INFO:teuthology.orchestra.run.smithi027.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.57102.log.gz 2024-09-12T03:24:45.379 INFO:teuthology.orchestra.run.smithi027.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.66787.log.gz 2024-09-12T03:24:45.379 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.75825.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45028.log 2024-09-12T03:24:45.379 INFO:teuthology.orchestra.run.smithi027.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.75825.log.gz 2024-09-12T03:24:45.379 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.43575.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69794.log 2024-09-12T03:24:45.379 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.45028.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.43575.log.gz 2024-09-12T03:24:45.379 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80986.log 2024-09-12T03:24:45.379 INFO:teuthology.orchestra.run.smithi027.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.45028.log.gz 2024-09-12T03:24:45.379 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.69794.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74963.log 2024-09-12T03:24:45.379 INFO:teuthology.orchestra.run.smithi027.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.69794.log.gz 2024-09-12T03:24:45.380 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.80986.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60861.log 2024-09-12T03:24:45.380 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.74963.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.80986.log.gz 2024-09-12T03:24:45.380 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42380.log 2024-09-12T03:24:45.380 INFO:teuthology.orchestra.run.smithi027.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.74963.log.gz 2024-09-12T03:24:45.380 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.60861.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53672.log 2024-09-12T03:24:45.380 INFO:teuthology.orchestra.run.smithi027.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.60861.log.gz 2024-09-12T03:24:45.380 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.42380.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66745.log 2024-09-12T03:24:45.380 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.53672.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80692.log 2024-09-12T03:24:45.380 INFO:teuthology.orchestra.run.smithi027.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.42380.log.gz 2024-09-12T03:24:45.380 INFO:teuthology.orchestra.run.smithi027.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.53672.log.gz 2024-09-12T03:24:45.380 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.66745.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54518.log 2024-09-12T03:24:45.380 INFO:teuthology.orchestra.run.smithi027.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.66745.log.gz 2024-09-12T03:24:45.381 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.80692.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69920.log 2024-09-12T03:24:45.381 INFO:teuthology.orchestra.run.smithi027.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.80692.log.gz 2024-09-12T03:24:45.381 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.54518.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46039.log 2024-09-12T03:24:45.381 INFO:teuthology.orchestra.run.smithi027.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.54518.log.gz 2024-09-12T03:24:45.381 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.69920.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71961.log 2024-09-12T03:24:45.381 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.46039.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.69920.log.gz 2024-09-12T03:24:45.381 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69317.log 2024-09-12T03:24:45.381 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.71961.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.46039.log.gz 2024-09-12T03:24:45.381 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46884.log 2024-09-12T03:24:45.381 INFO:teuthology.orchestra.run.smithi027.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.71961.log.gz 2024-09-12T03:24:45.381 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.69317.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72788.log 2024-09-12T03:24:45.381 INFO:teuthology.orchestra.run.smithi027.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.69317.log.gz 2024-09-12T03:24:45.381 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.46884.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79120.log 2024-09-12T03:24:45.381 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.72788.log: gzip 88.3% -5 -- replaced with /var/log/ceph/ceph-client.admin.46884.log.gz --verbose 2024-09-12T03:24:45.382 INFO:teuthology.orchestra.run.smithi027.stderr: -- /var/log/ceph/ceph-client.admin.49772.log 2024-09-12T03:24:45.382 INFO:teuthology.orchestra.run.smithi027.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.72788.log.gz 2024-09-12T03:24:45.382 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.79120.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81028.log 2024-09-12T03:24:45.382 INFO:teuthology.orchestra.run.smithi027.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.79120.log.gz 2024-09-12T03:24:45.382 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.49772.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53020.log 2024-09-12T03:24:45.382 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.81028.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.49772.log.gz 2024-09-12T03:24:45.382 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52511.log 2024-09-12T03:24:45.382 INFO:teuthology.orchestra.run.smithi027.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.81028.log.gz 2024-09-12T03:24:45.382 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.53020.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71651.log 2024-09-12T03:24:45.382 INFO:teuthology.orchestra.run.smithi027.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.53020.log.gz 2024-09-12T03:24:45.382 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.52511.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65798.log 2024-09-12T03:24:45.382 INFO:teuthology.orchestra.run.smithi027.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.52511.log.gz 2024-09-12T03:24:45.382 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.71651.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43030.log 2024-09-12T03:24:45.382 INFO:teuthology.orchestra.run.smithi027.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.71651.log.gz 2024-09-12T03:24:45.383 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.65798.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66310.log 2024-09-12T03:24:45.383 INFO:teuthology.orchestra.run.smithi027.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.65798.log.gz 2024-09-12T03:24:45.383 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.43030.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58348.log 2024-09-12T03:24:45.383 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.66310.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.43030.log.gz 2024-09-12T03:24:45.383 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61699.log 2024-09-12T03:24:45.383 INFO:teuthology.orchestra.run.smithi027.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.66310.log.gz 2024-09-12T03:24:45.383 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.58348.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58646.log 2024-09-12T03:24:45.383 INFO:teuthology.orchestra.run.smithi027.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.58348.log.gz 2024-09-12T03:24:45.383 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.61699.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48352.log 2024-09-12T03:24:45.383 INFO:teuthology.orchestra.run.smithi027.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.61699.log.gz 2024-09-12T03:24:45.383 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.58646.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79673.log 2024-09-12T03:24:45.383 INFO:teuthology.orchestra.run.smithi027.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.58646.log.gz 2024-09-12T03:24:45.383 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.48352.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50154.log 2024-09-12T03:24:45.384 INFO:teuthology.orchestra.run.smithi027.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.48352.log.gz 2024-09-12T03:24:45.384 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.79673.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65443.log 2024-09-12T03:24:45.384 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.50154.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.79673.log.gz 2024-09-12T03:24:45.384 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53864.log 2024-09-12T03:24:45.384 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.65443.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-09-12T03:24:45.384 INFO:teuthology.orchestra.run.smithi027.stderr: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.50154.log.gz 2024-09-12T03:24:45.384 INFO:teuthology.orchestra.run.smithi027.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.65443.log.gz 2024-09-12T03:24:45.384 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.53864.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68932.log 2024-09-12T03:24:45.384 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76984.log 2024-09-12T03:24:45.385 INFO:teuthology.orchestra.run.smithi027.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.53864.log.gz 2024-09-12T03:24:45.385 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-osd.1.log: /var/log/ceph/ceph-client.admin.68932.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58012.log 2024-09-12T03:24:45.385 INFO:teuthology.orchestra.run.smithi027.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.68932.log.gz 2024-09-12T03:24:45.385 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.76984.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46842.log 2024-09-12T03:24:45.385 INFO:teuthology.orchestra.run.smithi027.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.76984.log.gz 2024-09-12T03:24:45.385 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45508.log 2024-09-12T03:24:45.385 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.58012.log: /var/log/ceph/ceph-client.admin.46842.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59060.log 2024-09-12T03:24:45.385 INFO:teuthology.orchestra.run.smithi027.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.58012.log.gz 2024-09-12T03:24:45.385 INFO:teuthology.orchestra.run.smithi027.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.46842.log.gz 2024-09-12T03:24:45.385 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.45508.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45466.log 2024-09-12T03:24:45.385 INFO:teuthology.orchestra.run.smithi027.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.45508.log.gz 2024-09-12T03:24:45.386 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.59060.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57422.log 2024-09-12T03:24:45.386 INFO:teuthology.orchestra.run.smithi027.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.59060.log.gz 2024-09-12T03:24:45.386 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65485.log 2024-09-12T03:24:45.386 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.45466.log: /var/log/ceph/ceph-client.admin.57422.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70875.log 2024-09-12T03:24:45.386 INFO:teuthology.orchestra.run.smithi027.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.45466.log.gz 2024-09-12T03:24:45.386 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.65485.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.57422.log.gz 2024-09-12T03:24:45.386 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81280.log 2024-09-12T03:24:45.386 INFO:teuthology.orchestra.run.smithi027.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.65485.log.gz 2024-09-12T03:24:45.386 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.70875.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52637.log 2024-09-12T03:24:45.386 INFO:teuthology.orchestra.run.smithi027.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.70875.log.gz 2024-09-12T03:24:45.386 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.81280.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51833.log 2024-09-12T03:24:45.386 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58222.log 87.0% 2024-09-12T03:24:45.386 INFO:teuthology.orchestra.run.smithi027.stderr: -- replaced with /var/log/ceph/ceph-client.admin.81280.log.gz 2024-09-12T03:24:45.387 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.52637.log: /var/log/ceph/ceph-client.admin.51833.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.52637.log.gz 2024-09-12T03:24:45.387 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65359.log 2024-09-12T03:24:45.387 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48625.log 2024-09-12T03:24:45.387 INFO:teuthology.orchestra.run.smithi027.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.51833.log.gz 2024-09-12T03:24:45.387 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.58222.log: /var/log/ceph/ceph-client.admin.65359.log: gzip -5 --verbose -- 89.0% /var/log/ceph/ceph-client.admin.53612.log -- replaced with /var/log/ceph/ceph-client.admin.58222.log.gz 2024-09-12T03:24:45.387 INFO:teuthology.orchestra.run.smithi027.stderr: 2024-09-12T03:24:45.387 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56976.log 2024-09-12T03:24:45.387 INFO:teuthology.orchestra.run.smithi027.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.65359.log.gz 2024-09-12T03:24:45.387 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.48625.log: /var/log/ceph/ceph-client.admin.53612.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.48625.log.gz 2024-09-12T03:24:45.388 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57060.log 2024-09-12T03:24:45.388 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51425.log 2024-09-12T03:24:45.388 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.56976.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.53612.log.gz 2024-09-12T03:24:45.388 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.57060.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51691.log 2024-09-12T03:24:45.388 INFO:teuthology.orchestra.run.smithi027.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.56976.log.gz 2024-09-12T03:24:45.388 INFO:teuthology.orchestra.run.smithi027.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.57060.log.gz 2024-09-12T03:24:45.388 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.51425.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74524.log 2024-09-12T03:24:45.388 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78786.log 2024-09-12T03:24:45.388 INFO:teuthology.orchestra.run.smithi027.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.51425.log.gz 2024-09-12T03:24:45.388 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.51691.log: gzip -5 --verbose/var/log/ceph/ceph-client.admin.74524.log: -- /var/log/ceph/ceph-client.admin.43251.log 2024-09-12T03:24:45.388 INFO:teuthology.orchestra.run.smithi027.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.51691.log.gz 2024-09-12T03:24:45.388 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.78786.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.74524.log.gz 2024-09-12T03:24:45.389 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74398.log 2024-09-12T03:24:45.389 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.43251.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56601.log 2024-09-12T03:24:45.389 INFO:teuthology.orchestra.run.smithi027.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.78786.log.gz 2024-09-12T03:24:45.389 INFO:teuthology.orchestra.run.smithi027.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.43251.log.gz 2024-09-12T03:24:45.389 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.74398.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65882.log 2024-09-12T03:24:45.389 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.56601.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50070.log 2024-09-12T03:24:45.389 INFO:teuthology.orchestra.run.smithi027.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.74398.log.gz 2024-09-12T03:24:45.389 INFO:teuthology.orchestra.run.smithi027.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.56601.log.gz 2024-09-12T03:24:45.389 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.65882.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54602.log 2024-09-12T03:24:45.390 INFO:teuthology.orchestra.run.smithi027.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.65882.log.gz 2024-09-12T03:24:45.390 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51301.log 2024-09-12T03:24:45.390 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.50070.log: /var/log/ceph/ceph-client.admin.54602.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43919.log 2024-09-12T03:24:45.390 INFO:teuthology.orchestra.run.smithi027.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.50070.log.gz 2024-09-12T03:24:45.390 INFO:teuthology.orchestra.run.smithi027.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.54602.log.gz 2024-09-12T03:24:45.390 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.51301.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42336.log 2024-09-12T03:24:45.390 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.43919.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.51301.log.gz 2024-09-12T03:24:45.390 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66913.log 2024-09-12T03:24:45.390 INFO:teuthology.orchestra.run.smithi027.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.43919.log.gz 2024-09-12T03:24:45.390 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.42336.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56517.log 2024-09-12T03:24:45.390 INFO:teuthology.orchestra.run.smithi027.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.42336.log.gz 2024-09-12T03:24:45.390 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44416.log 2024-09-12T03:24:45.391 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.66913.log: /var/log/ceph/ceph-client.admin.56517.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66222.log 2024-09-12T03:24:45.391 INFO:teuthology.orchestra.run.smithi027.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.66913.log.gz 2024-09-12T03:24:45.391 INFO:teuthology.orchestra.run.smithi027.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.56517.log.gz 2024-09-12T03:24:45.391 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.44416.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53486.log 2024-09-12T03:24:45.391 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.66222.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75783.log 2024-09-12T03:24:45.391 INFO:teuthology.orchestra.run.smithi027.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.44416.log.gz 2024-09-12T03:24:45.391 INFO:teuthology.orchestra.run.smithi027.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.66222.log.gz 2024-09-12T03:24:45.391 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.53486.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64984.log 2024-09-12T03:24:45.391 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.75783.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.53486.log.gz 2024-09-12T03:24:45.391 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43921.log 2024-09-12T03:24:45.391 INFO:teuthology.orchestra.run.smithi027.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.75783.log.gz 2024-09-12T03:24:45.392 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48751.log 2024-09-12T03:24:45.392 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.64984.log: /var/log/ceph/ceph-client.admin.43921.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56643.log 2024-09-12T03:24:45.392 INFO:teuthology.orchestra.run.smithi027.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.64984.log.gz 2024-09-12T03:24:45.392 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.48751.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.43921.log.gz 2024-09-12T03:24:45.392 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68848.log 2024-09-12T03:24:45.392 INFO:teuthology.orchestra.run.smithi027.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.48751.log.gz 2024-09-12T03:24:45.392 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.56643.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76077.log 2024-09-12T03:24:45.392 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68890.log 2024-09-12T03:24:45.392 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.68848.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.56643.log.gz 2024-09-12T03:24:45.392 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.76077.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79180.log 2024-09-12T03:24:45.393 INFO:teuthology.orchestra.run.smithi027.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.68848.log.gz 2024-09-12T03:24:45.393 INFO:teuthology.orchestra.run.smithi027.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.76077.log.gz 2024-09-12T03:24:45.393 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72744.log 2024-09-12T03:24:45.393 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.68890.log: /var/log/ceph/ceph-client.admin.79180.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57791.log 2024-09-12T03:24:45.393 INFO:teuthology.orchestra.run.smithi027.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.68890.log.gz 2024-09-12T03:24:45.393 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.72744.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.79180.log.gz 2024-09-12T03:24:45.393 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58688.log 2024-09-12T03:24:45.393 INFO:teuthology.orchestra.run.smithi027.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.72744.log.gz 2024-09-12T03:24:45.393 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.57791.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43254.log 2024-09-12T03:24:45.393 INFO:teuthology.orchestra.run.smithi027.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.57791.log.gz 2024-09-12T03:24:45.394 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.56897.log 2024-09-12T03:24:45.394 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.58688.log: /var/log/ceph/ceph-client.admin.43254.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41888.log 2024-09-12T03:24:45.394 INFO:teuthology.orchestra.run.smithi027.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.58688.log.gz 2024-09-12T03:24:45.394 INFO:teuthology.orchestra.run.smithi027.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.43254.log.gz 2024-09-12T03:24:45.394 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.0.56897.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.56897.log.gz 2024-09-12T03:24:45.394 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80482.log 2024-09-12T03:24:45.394 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42946.log 2024-09-12T03:24:45.394 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.41888.log: /var/log/ceph/ceph-client.admin.80482.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77485.log 2024-09-12T03:24:45.394 INFO:teuthology.orchestra.run.smithi027.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.41888.log.gz 2024-09-12T03:24:45.394 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.42946.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75741.log 2024-09-12T03:24:45.395 INFO:teuthology.orchestra.run.smithi027.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.80482.log.gz 2024-09-12T03:24:45.395 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.77485.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.42946.log.gz 2024-09-12T03:24:45.395 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46428.log 2024-09-12T03:24:45.395 INFO:teuthology.orchestra.run.smithi027.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.77485.log.gz 2024-09-12T03:24:45.395 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68661.log 2024-09-12T03:24:45.395 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.75741.log: /var/log/ceph/ceph-client.admin.46428.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69440.log 2024-09-12T03:24:45.395 INFO:teuthology.orchestra.run.smithi027.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.75741.log.gz 2024-09-12T03:24:45.395 INFO:teuthology.orchestra.run.smithi027.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.46428.log.gz 2024-09-12T03:24:45.395 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.68661.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75345.log 2024-09-12T03:24:45.395 INFO:teuthology.orchestra.run.smithi027.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.68661.log.gz 2024-09-12T03:24:45.395 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47052.log 2024-09-12T03:24:45.396 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.69440.log: /var/log/ceph/ceph-client.admin.75345.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.69440.log.gz 2024-09-12T03:24:45.396 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47615.log 2024-09-12T03:24:45.396 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77756.log 2024-09-12T03:24:45.396 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.47052.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.75345.log.gz 2024-09-12T03:24:45.396 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.47615.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.47052.log.gz 2024-09-12T03:24:45.396 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65139.log 2024-09-12T03:24:45.396 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73803.log 2024-09-12T03:24:45.396 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.77756.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.47615.log.gz 2024-09-12T03:24:45.396 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.65139.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80061.log 2024-09-12T03:24:45.396 INFO:teuthology.orchestra.run.smithi027.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.77756.log.gz 2024-09-12T03:24:45.396 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49860.log 2024-09-12T03:24:45.396 INFO:teuthology.orchestra.run.smithi027.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.65139.log.gz 2024-09-12T03:24:45.396 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.73803.log: /var/log/ceph/ceph-client.admin.80061.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49548.log 2024-09-12T03:24:45.397 INFO:teuthology.orchestra.run.smithi027.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.73803.log.gz 2024-09-12T03:24:45.397 INFO:teuthology.orchestra.run.smithi027.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.80061.log.gz 2024-09-12T03:24:45.397 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69878.log 2024-09-12T03:24:45.397 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.49860.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65579.log 2024-09-12T03:24:45.397 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.49548.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.49860.log.gz 2024-09-12T03:24:45.397 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.69878.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74105.log 2024-09-12T03:24:45.397 INFO:teuthology.orchestra.run.smithi027.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.49548.log.gz 2024-09-12T03:24:45.397 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.65579.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.69878.log.gz 2024-09-12T03:24:45.397 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52324.log 2024-09-12T03:24:45.397 INFO:teuthology.orchestra.run.smithi027.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.65579.log.gz 2024-09-12T03:24:45.397 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.74105.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43786.log 2024-09-12T03:24:45.397 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76838.log 2024-09-12T03:24:45.398 INFO:teuthology.orchestra.run.smithi027.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.74105.log.gz 2024-09-12T03:24:45.398 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.52324.log: /var/log/ceph/ceph-client.admin.43786.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.52324.log.gz 2024-09-12T03:24:45.398 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74650.log 2024-09-12T03:24:45.398 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49902.log 2024-09-12T03:24:45.398 INFO:teuthology.orchestra.run.smithi027.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.43786.log.gz 2024-09-12T03:24:45.398 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.76838.log: /var/log/ceph/ceph-client.admin.74650.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.76838.log.gz 2024-09-12T03:24:45.398 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66661.log 2024-09-12T03:24:45.398 INFO:teuthology.orchestra.run.smithi027.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.74650.log.gz 2024-09-12T03:24:45.398 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54215.log 2024-09-12T03:24:45.398 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.49902.log: /var/log/ceph/ceph-client.admin.66661.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70505.log 2024-09-12T03:24:45.398 INFO:teuthology.orchestra.run.smithi027.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.49902.log.gz 2024-09-12T03:24:45.398 INFO:teuthology.orchestra.run.smithi027.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.66661.log.gz 2024-09-12T03:24:45.398 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.54215.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55061.log 2024-09-12T03:24:45.398 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43589.log 2024-09-12T03:24:45.399 INFO:teuthology.orchestra.run.smithi027.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.54215.log.gz/var/log/ceph/ceph-client.admin.70505.log: 2024-09-12T03:24:45.399 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.55061.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.70505.log.gz 2024-09-12T03:24:45.399 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53062.log 2024-09-12T03:24:45.399 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54893.log 2024-09-12T03:24:45.399 INFO:teuthology.orchestra.run.smithi027.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.55061.log.gz 2024-09-12T03:24:45.399 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.43589.log: /var/log/ceph/ceph-client.admin.53062.log: 88.6%gzip -- replaced with /var/log/ceph/ceph-client.admin.43589.log.gz 2024-09-12T03:24:45.399 INFO:teuthology.orchestra.run.smithi027.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.66871.log 2024-09-12T03:24:45.399 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49944.log 2024-09-12T03:24:45.399 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.54893.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.53062.log.gz 2024-09-12T03:24:45.399 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.66871.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.54893.log.gz 2024-09-12T03:24:45.399 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46284.log 2024-09-12T03:24:45.399 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60819.log 2024-09-12T03:24:45.399 INFO:teuthology.orchestra.run.smithi027.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.66871.log.gz 2024-09-12T03:24:45.399 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.49944.log: /var/log/ceph/ceph-client.admin.46284.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.49944.log.gz 2024-09-12T03:24:45.400 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52553.log 2024-09-12T03:24:45.400 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47489.log 2024-09-12T03:24:45.400 INFO:teuthology.orchestra.run.smithi027.stderr: 87.6%/var/log/ceph/ceph-client.admin.60819.log: -- replaced with /var/log/ceph/ceph-client.admin.46284.log.gz 2024-09-12T03:24:45.400 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.52553.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.60819.log.gz 2024-09-12T03:24:45.400 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73723.log 2024-09-12T03:24:45.400 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75657.log 87.9% 2024-09-12T03:24:45.401 INFO:teuthology.orchestra.run.smithi027.stderr: -- replaced with /var/log/ceph/ceph-client.admin.52553.log.gz 2024-09-12T03:24:45.401 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.47489.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.47489.log.gz 2024-09-12T03:24:45.401 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80818.log 2024-09-12T03:24:45.401 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.73723.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49425.log 2024-09-12T03:24:45.401 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.75657.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.73723.log.gz 2024-09-12T03:24:45.401 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.80818.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59687.log 2024-09-12T03:24:45.401 INFO:teuthology.orchestra.run.smithi027.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.75657.log.gz 2024-09-12T03:24:45.401 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57380.log 2024-09-12T03:24:45.401 INFO:teuthology.orchestra.run.smithi027.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.80818.log.gz 2024-09-12T03:24:45.401 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.49425.log: /var/log/ceph/ceph-client.admin.59687.log: gzip 88.3% -- replaced with /var/log/ceph/ceph-client.admin.49425.log.gz -5 2024-09-12T03:24:45.401 INFO:teuthology.orchestra.run.smithi027.stderr: --verbose -- /var/log/ceph/ceph-client.admin.61573.log 2024-09-12T03:24:45.401 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42862.log 2024-09-12T03:24:45.402 INFO:teuthology.orchestra.run.smithi027.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.59687.log.gz 2024-09-12T03:24:45.402 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.57380.log: /var/log/ceph/ceph-client.admin.61573.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.57380.log.gz 2024-09-12T03:24:45.402 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43587.log 2024-09-12T03:24:45.402 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53402.log 2024-09-12T03:24:45.402 INFO:teuthology.orchestra.run.smithi027.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.61573.log.gz 2024-09-12T03:24:45.402 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.42862.log: /var/log/ceph/ceph-client.admin.43587.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.42862.log.gz 2024-09-12T03:24:45.402 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75909.log 2024-09-12T03:24:45.402 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60693.log 2024-09-12T03:24:45.402 INFO:teuthology.orchestra.run.smithi027.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.43587.log.gz 2024-09-12T03:24:45.402 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.53402.log: /var/log/ceph/ceph-client.admin.75909.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.53402.log.gz 2024-09-12T03:24:45.402 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61341.log 2024-09-12T03:24:45.402 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61783.log 2024-09-12T03:24:45.402 INFO:teuthology.orchestra.run.smithi027.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.75909.log.gz 2024-09-12T03:24:45.403 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.60693.log: /var/log/ceph/ceph-client.admin.61341.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.60693.log.gz 2024-09-12T03:24:45.403 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64755.log 2024-09-12T03:24:45.403 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61425.log 2024-09-12T03:24:45.403 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.61783.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.61341.log.gz 2024-09-12T03:24:45.403 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.64755.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.54052.log 2024-09-12T03:24:45.403 INFO:teuthology.orchestra.run.smithi027.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.61783.log.gz 87.9% 2024-09-12T03:24:45.403 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.61425.log: -- replaced with /var/log/ceph/ceph-client.admin.64755.log.gz 2024-09-12T03:24:45.403 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42820.log 2024-09-12T03:24:45.403 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.0.54052.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.54052.log.gz 2024-09-12T03:24:45.403 INFO:teuthology.orchestra.run.smithi027.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.61425.log.gz 2024-09-12T03:24:45.403 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64839.log 2024-09-12T03:24:45.403 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.42820.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45949.log 2024-09-12T03:24:45.403 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.64839.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.42820.log.gz 2024-09-12T03:24:45.403 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78139.log 2024-09-12T03:24:45.404 INFO:teuthology.orchestra.run.smithi027.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.64839.log.gz 2024-09-12T03:24:45.404 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.45949.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.70300.log 2024-09-12T03:24:45.404 INFO:teuthology.orchestra.run.smithi027.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.45949.log.gz 2024-09-12T03:24:45.404 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.78139.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76784.log 2024-09-12T03:24:45.404 INFO:teuthology.orchestra.run.smithi027.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.78139.log.gz 2024-09-12T03:24:45.404 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.0.70300.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.0.70300.log.gz 2024-09-12T03:24:45.404 INFO:teuthology.orchestra.run.smithi027.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.57749.log 2024-09-12T03:24:45.404 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54935.log 2024-09-12T03:24:45.404 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.76784.log: /var/log/ceph/ceph-client.admin.57749.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71877.log 2024-09-12T03:24:45.404 INFO:teuthology.orchestra.run.smithi027.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.76784.log.gz 2024-09-12T03:24:45.404 INFO:teuthology.orchestra.run.smithi027.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.57749.log.gz 2024-09-12T03:24:45.404 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.54935.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51239.log 2024-09-12T03:24:45.405 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46512.log 2024-09-12T03:24:45.405 INFO:teuthology.orchestra.run.smithi027.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.54935.log.gz 2024-09-12T03:24:45.405 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.71877.log: /var/log/ceph/ceph-client.admin.51239.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.71877.log.gz 2024-09-12T03:24:45.405 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78660.log 2024-09-12T03:24:45.405 INFO:teuthology.orchestra.run.smithi027.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.51239.log.gz 2024-09-12T03:24:45.405 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46757.log 2024-09-12T03:24:45.405 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.46512.log: /var/log/ceph/ceph-client.admin.78660.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72839.log 2024-09-12T03:24:45.405 INFO:teuthology.orchestra.run.smithi027.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.46512.log.gz 2024-09-12T03:24:45.405 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.46757.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67828.log 2024-09-12T03:24:45.405 INFO:teuthology.orchestra.run.smithi027.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.78660.log.gz 2024-09-12T03:24:45.405 INFO:teuthology.orchestra.run.smithi027.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.46757.log.gz 2024-09-12T03:24:45.405 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.72839.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80902.log 2024-09-12T03:24:45.405 INFO:teuthology.orchestra.run.smithi027.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.72839.log.gz 2024-09-12T03:24:45.405 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46554.log 2024-09-12T03:24:45.406 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.67828.log: /var/log/ceph/ceph-client.admin.80902.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.50408.log 2024-09-12T03:24:45.406 INFO:teuthology.orchestra.run.smithi027.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.67828.log.gz 2024-09-12T03:24:45.406 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.46554.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.80902.log.gz 2024-09-12T03:24:45.406 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61657.log 2024-09-12T03:24:45.406 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81238.log 2024-09-12T03:24:45.406 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.0.50408.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.46554.log.gz 2024-09-12T03:24:45.406 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.50408.log.gz 2024-09-12T03:24:45.406 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.61657.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80206.log 2024-09-12T03:24:45.406 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.81238.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.61657.log.gz 2024-09-12T03:24:45.406 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43246.log 2024-09-12T03:24:45.406 INFO:teuthology.orchestra.run.smithi027.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.81238.log.gz 2024-09-12T03:24:45.406 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.80206.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79379.log 2024-09-12T03:24:45.406 INFO:teuthology.orchestra.run.smithi027.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.80206.log.gz 2024-09-12T03:24:45.406 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.43246.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81466.log 2024-09-12T03:24:45.407 INFO:teuthology.orchestra.run.smithi027.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.43246.log.gz 2024-09-12T03:24:45.407 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77798.log 2024-09-12T03:24:45.407 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.79379.log: /var/log/ceph/ceph-client.admin.81466.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57338.log 2024-09-12T03:24:45.407 INFO:teuthology.orchestra.run.smithi027.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.79379.log.gz 2024-09-12T03:24:45.407 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.77798.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.81466.log.gz 2024-09-12T03:24:45.407 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58523.log 2024-09-12T03:24:45.407 INFO:teuthology.orchestra.run.smithi027.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.77798.log.gz 2024-09-12T03:24:45.407 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.57338.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78618.log 2024-09-12T03:24:45.407 INFO:teuthology.orchestra.run.smithi027.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.57338.log.gz 2024-09-12T03:24:45.407 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.58523.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55439.log 2024-09-12T03:24:45.407 INFO:teuthology.orchestra.run.smithi027.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.58523.log.gz 2024-09-12T03:24:45.407 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75476.log 2024-09-12T03:24:45.407 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.78618.log: /var/log/ceph/ceph-client.admin.55439.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50613.log 2024-09-12T03:24:45.407 INFO:teuthology.orchestra.run.smithi027.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.78618.log.gz 2024-09-12T03:24:45.408 INFO:teuthology.orchestra.run.smithi027.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.55439.log.gz 2024-09-12T03:24:45.408 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.75476.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71057.log 2024-09-12T03:24:45.408 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47140.log 2024-09-12T03:24:45.408 INFO:teuthology.orchestra.run.smithi027.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.75476.log.gz 2024-09-12T03:24:45.408 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.50613.log: /var/log/ceph/ceph-client.admin.71057.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.50613.log.gz 2024-09-12T03:24:45.408 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77714.log 2024-09-12T03:24:45.408 INFO:teuthology.orchestra.run.smithi027.stderr: 86.2% -- replaced with /var/log/ceph/ceph-client.admin.71057.log.gz 2024-09-12T03:24:45.408 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67870.log 2024-09-12T03:24:45.408 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.47140.log: /var/log/ceph/ceph-client.admin.77714.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-12T03:24:45.408 INFO:teuthology.orchestra.run.smithi027.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.47140.log.gz 2024-09-12T03:24:45.408 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.67870.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76035.log 2024-09-12T03:24:45.408 INFO:teuthology.orchestra.run.smithi027.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.77714.log.gz 2024-09-12T03:24:45.408 INFO:teuthology.orchestra.run.smithi027.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.67870.log.gz 2024-09-12T03:24:45.408 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70112.log 2024-09-12T03:24:45.409 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66619.log 2024-09-12T03:24:45.409 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.76035.log: /var/log/ceph/ceph-client.admin.70112.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57176.log 2024-09-12T03:24:45.409 INFO:teuthology.orchestra.run.smithi027.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.76035.log.gz 2024-09-12T03:24:45.409 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.66619.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.70112.log.gz 2024-09-12T03:24:45.409 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67415.log 2024-09-12T03:24:45.409 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.57176.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.66619.log.gz 2024-09-12T03:24:45.409 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55187.log 2024-09-12T03:24:45.409 INFO:teuthology.orchestra.run.smithi027.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.57176.log.gz 2024-09-12T03:24:45.409 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.67415.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49082.log 2024-09-12T03:24:45.409 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.55187.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50869.log 89.1% -- replaced with /var/log/ceph/ceph-client.admin.67415.log.gz 2024-09-12T03:24:45.409 INFO:teuthology.orchestra.run.smithi027.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.55187.log.gz 2024-09-12T03:24:45.409 INFO:teuthology.orchestra.run.smithi027.stderr: 2024-09-12T03:24:45.409 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.49082.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79935.log 2024-09-12T03:24:45.410 INFO:teuthology.orchestra.run.smithi027.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.49082.log.gz 2024-09-12T03:24:45.410 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.50869.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67063.log 2024-09-12T03:24:45.410 INFO:teuthology.orchestra.run.smithi027.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.50869.log.gz 2024-09-12T03:24:45.410 INFO:teuthology.orchestra.run.smithi027.stderr: 89.9% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-12T03:24:45.410 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.79935.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67954.log 2024-09-12T03:24:45.410 INFO:teuthology.orchestra.run.smithi027.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.79935.log.gz 2024-09-12T03:24:45.410 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58138.log 2024-09-12T03:24:45.410 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.67063.log: gzip/var/log/ceph/ceph-client.admin.67954.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.68745.log 2024-09-12T03:24:45.410 INFO:teuthology.orchestra.run.smithi027.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.67063.log.gz 2024-09-12T03:24:45.410 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.58138.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.67954.log.gz 2024-09-12T03:24:45.410 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67373.log 2024-09-12T03:24:45.410 INFO:teuthology.orchestra.run.smithi027.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.58138.log.gz 2024-09-12T03:24:45.410 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.68745.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74165.log 2024-09-12T03:24:45.410 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.67373.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.68745.log.gz 2024-09-12T03:24:45.411 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73329.log 2024-09-12T03:24:45.411 INFO:teuthology.orchestra.run.smithi027.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.67373.log.gz 2024-09-12T03:24:45.411 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.74165.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53798.log 2024-09-12T03:24:45.411 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60338.log 2024-09-12T03:24:45.411 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.73329.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.74165.log.gz 2024-09-12T03:24:45.411 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.53798.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70379.log 2024-09-12T03:24:45.411 INFO:teuthology.orchestra.run.smithi027.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.73329.log.gz 2024-09-12T03:24:45.411 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52072.log 2024-09-12T03:24:45.411 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.60338.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.53798.log.gz 2024-09-12T03:24:45.411 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.70379.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62158.log 2024-09-12T03:24:45.411 INFO:teuthology.orchestra.run.smithi027.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.60338.log.gz 2024-09-12T03:24:45.411 INFO:teuthology.orchestra.run.smithi027.stderr: 88.5%gzip -- replaced with /var/log/ceph/ceph-client.admin.70379.log.gz -5 2024-09-12T03:24:45.411 INFO:teuthology.orchestra.run.smithi027.stderr: --verbose -- /var/log/ceph/ceph-client.admin.63157.log 2024-09-12T03:24:45.411 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.52072.log: /var/log/ceph/ceph-client.admin.62158.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71709.log 2024-09-12T03:24:45.412 INFO:teuthology.orchestra.run.smithi027.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.52072.log.gz 2024-09-12T03:24:45.412 INFO:teuthology.orchestra.run.smithi027.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.62158.log.gz 2024-09-12T03:24:45.412 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.63157.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58777.log 2024-09-12T03:24:45.412 INFO:teuthology.orchestra.run.smithi027.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.63157.log.gz 2024-09-12T03:24:45.412 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36306.log 2024-09-12T03:24:45.412 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.71709.log: /var/log/ceph/ceph-client.admin.58777.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44332.log 2024-09-12T03:24:45.412 INFO:teuthology.orchestra.run.smithi027.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.71709.log.gz 2024-09-12T03:24:45.412 INFO:teuthology.orchestra.run.smithi027.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.58777.log.gz 2024-09-12T03:24:45.412 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.36306.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50487.log 2024-09-12T03:24:45.412 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72358.log 2024-09-12T03:24:45.412 INFO:teuthology.orchestra.run.smithi027.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.36306.log.gz 2024-09-12T03:24:45.412 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.44332.log: /var/log/ceph/ceph-client.admin.50487.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79222.log 2024-09-12T03:24:45.412 INFO:teuthology.orchestra.run.smithi027.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.44332.log.gz 2024-09-12T03:24:45.412 INFO:teuthology.orchestra.run.smithi027.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.50487.log.gz 2024-09-12T03:24:45.413 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55994.log 2024-09-12T03:24:45.413 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.72358.log: /var/log/ceph/ceph-client.admin.79222.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68142.log 2024-09-12T03:24:45.413 INFO:teuthology.orchestra.run.smithi027.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.72358.log.gz 2024-09-12T03:24:45.413 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.55994.log: gzip -5 --verbose -- 88.8% /var/log/ceph/ceph-client.admin.49635.log 2024-09-12T03:24:45.413 INFO:teuthology.orchestra.run.smithi027.stderr: -- replaced with /var/log/ceph/ceph-client.admin.79222.log.gz 2024-09-12T03:24:45.413 INFO:teuthology.orchestra.run.smithi027.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.55994.log.gz 2024-09-12T03:24:45.413 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.68142.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42442.log 2024-09-12T03:24:45.413 INFO:teuthology.orchestra.run.smithi027.stderr: 85.5% -- replaced with /var/log/ceph/ceph-client.admin.68142.log.gz 2024-09-12T03:24:45.413 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64568.log 2024-09-12T03:24:45.413 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.49635.log: /var/log/ceph/ceph-client.admin.42442.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79751.log 2024-09-12T03:24:45.413 INFO:teuthology.orchestra.run.smithi027.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.49635.log.gz 2024-09-12T03:24:45.413 INFO:teuthology.orchestra.run.smithi027.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.42442.log.gz 2024-09-12T03:24:45.413 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.64568.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71192.log 2024-09-12T03:24:45.413 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47321.log 2024-09-12T03:24:45.414 INFO:teuthology.orchestra.run.smithi027.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.64568.log.gz 2024-09-12T03:24:45.414 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.79751.log: /var/log/ceph/ceph-client.admin.71192.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74482.log 2024-09-12T03:24:45.414 INFO:teuthology.orchestra.run.smithi027.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.79751.log.gz 2024-09-12T03:24:45.414 INFO:teuthology.orchestra.run.smithi027.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.71192.log.gz 2024-09-12T03:24:45.414 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43247.log 2024-09-12T03:24:45.414 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.47321.log: /var/log/ceph/ceph-client.admin.74482.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75527.log 2024-09-12T03:24:45.414 INFO:teuthology.orchestra.run.smithi027.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.47321.log.gz 2024-09-12T03:24:45.414 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.43247.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.74482.log.gz 2024-09-12T03:24:45.414 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71751.log 2024-09-12T03:24:45.414 INFO:teuthology.orchestra.run.smithi027.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.43247.log.gz 2024-09-12T03:24:45.414 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.75527.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56205.log 2024-09-12T03:24:45.414 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51177.log 2024-09-12T03:24:45.414 INFO:teuthology.orchestra.run.smithi027.stderr: 87.6%/var/log/ceph/ceph-client.admin.71751.log: -- replaced with /var/log/ceph/ceph-client.admin.75527.log.gz 2024-09-12T03:24:45.415 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.56205.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76554.log 2024-09-12T03:24:45.415 INFO:teuthology.orchestra.run.smithi027.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.71751.log.gz 2024-09-12T03:24:45.415 INFO:teuthology.orchestra.run.smithi027.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.56205.log.gz 2024-09-12T03:24:45.415 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71793.log 2024-09-12T03:24:45.415 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.51177.log: /var/log/ceph/ceph-client.admin.76554.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52595.log 2024-09-12T03:24:45.415 INFO:teuthology.orchestra.run.smithi027.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.51177.log.gz 2024-09-12T03:24:45.415 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.71793.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.76554.log.gz 2024-09-12T03:24:45.415 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80944.log 2024-09-12T03:24:45.415 INFO:teuthology.orchestra.run.smithi027.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.71793.log.gz 2024-09-12T03:24:45.415 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.52595.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47573.log 2024-09-12T03:24:45.415 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45871.log 2024-09-12T03:24:45.415 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.80944.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.52595.log.gz 2024-09-12T03:24:45.415 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.47573.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.80944.log.gz 2024-09-12T03:24:45.415 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69571.log 2024-09-12T03:24:45.416 INFO:teuthology.orchestra.run.smithi027.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.47573.log.gz 2024-09-12T03:24:45.416 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42484.log 2024-09-12T03:24:45.416 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.45871.log: /var/log/ceph/ceph-client.admin.69571.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71397.log 2024-09-12T03:24:45.416 INFO:teuthology.orchestra.run.smithi027.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.45871.log.gz 2024-09-12T03:24:45.416 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.42484.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.69571.log.gz 2024-09-12T03:24:45.416 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43579.log 2024-09-12T03:24:45.416 INFO:teuthology.orchestra.run.smithi027.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.42484.log.gz 2024-09-12T03:24:45.416 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.71397.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68974.log 2024-09-12T03:24:45.416 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68577.log 2024-09-12T03:24:45.416 INFO:teuthology.orchestra.run.smithi027.stderr: 87.6%/var/log/ceph/ceph-client.admin.43579.log: -- replaced with /var/log/ceph/ceph-client.admin.71397.log.gz 2024-09-12T03:24:45.416 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.68974.log: 89.2%gzip -- replaced with /var/log/ceph/ceph-client.admin.43579.log.gz 2024-09-12T03:24:45.416 INFO:teuthology.orchestra.run.smithi027.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.47681.log 2024-09-12T03:24:45.417 INFO:teuthology.orchestra.run.smithi027.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.68974.log.gz 2024-09-12T03:24:45.417 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56475.log 2024-09-12T03:24:45.417 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.68577.log: /var/log/ceph/ceph-client.admin.47681.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51567.log 2024-09-12T03:24:45.417 INFO:teuthology.orchestra.run.smithi027.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.68577.log.gz 2024-09-12T03:24:45.417 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.56475.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.47681.log.gz 2024-09-12T03:24:45.417 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57296.log 2024-09-12T03:24:45.417 INFO:teuthology.orchestra.run.smithi027.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.56475.log.gz 2024-09-12T03:24:45.417 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.51567.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74608.log 2024-09-12T03:24:45.417 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72148.log 2024-09-12T03:24:45.417 INFO:teuthology.orchestra.run.smithi027.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.51567.log.gz 2024-09-12T03:24:45.417 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.57296.log: /var/log/ceph/ceph-client.admin.74608.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.57296.log.gz 2024-09-12T03:24:45.418 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75569.log 2024-09-12T03:24:45.418 INFO:teuthology.orchestra.run.smithi027.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.74608.log.gz 2024-09-12T03:24:45.418 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69962.log 2024-09-12T03:24:45.418 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.72148.log: /var/log/ceph/ceph-client.admin.75569.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60609.log 2024-09-12T03:24:45.418 INFO:teuthology.orchestra.run.smithi027.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.72148.log.gz 2024-09-12T03:24:45.418 INFO:teuthology.orchestra.run.smithi027.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.75569.log.gz 2024-09-12T03:24:45.418 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.69962.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43924.log 2024-09-12T03:24:45.418 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67457.log 2024-09-12T03:24:45.418 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.60609.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.69962.log.gz 2024-09-12T03:24:45.418 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.43924.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.60609.log.gz 2024-09-12T03:24:45.418 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65527.log 2024-09-12T03:24:45.418 INFO:teuthology.orchestra.run.smithi027.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.43924.log.gz 2024-09-12T03:24:45.418 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47191.log 2024-09-12T03:24:45.418 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.67457.log: /var/log/ceph/ceph-client.admin.65527.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.67457.log.gz 2024-09-12T03:24:45.419 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51505.log 2024-09-12T03:24:45.419 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70834.log 2024-09-12T03:24:45.419 INFO:teuthology.orchestra.run.smithi027.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.65527.log.gz 2024-09-12T03:24:45.419 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.47191.log: /var/log/ceph/ceph-client.admin.51505.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59811.log 2024-09-12T03:24:45.419 INFO:teuthology.orchestra.run.smithi027.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.47191.log.gz 2024-09-12T03:24:45.419 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78311.log 2024-09-12T03:24:45.419 INFO:teuthology.orchestra.run.smithi027.stderr: 88.6%/var/log/ceph/ceph-client.admin.70834.log: -- replaced with /var/log/ceph/ceph-client.admin.51505.log.gz 2024-09-12T03:24:45.419 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.59811.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.70834.log.gz 2024-09-12T03:24:45.419 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58096.log 2024-09-12T03:24:45.419 INFO:teuthology.orchestra.run.smithi027.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.59811.log.gzgzip 2024-09-12T03:24:45.419 INFO:teuthology.orchestra.run.smithi027.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.66829.log 2024-09-12T03:24:45.419 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.78311.log: /var/log/ceph/ceph-client.admin.58096.log: gzip -5 89.0% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.78311.log.gz 2024-09-12T03:24:45.419 INFO:teuthology.orchestra.run.smithi027.stderr: /var/log/ceph/ceph-client.0.44893.log 2024-09-12T03:24:45.419 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73681.log 2024-09-12T03:24:45.420 INFO:teuthology.orchestra.run.smithi027.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.58096.log.gz 2024-09-12T03:24:45.420 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.66829.log: /var/log/ceph/ceph-client.0.44893.log: gzip -5 87.4% --verbose -- replaced with /var/log/ceph/ceph-client.admin.66829.log.gz -- 2024-09-12T03:24:45.420 INFO:teuthology.orchestra.run.smithi027.stderr: /var/log/ceph/ceph-client.admin.62116.log 2024-09-12T03:24:45.420 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.44893.log.gz 2024-09-12T03:24:45.420 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.73681.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43923.log 2024-09-12T03:24:45.420 INFO:teuthology.orchestra.run.smithi027.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.73681.log.gz 2024-09-12T03:24:45.420 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.62116.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73095.log 2024-09-12T03:24:45.420 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43586.log 2024-09-12T03:24:45.420 INFO:teuthology.orchestra.run.smithi027.stderr: 88.8%/var/log/ceph/ceph-client.admin.43923.log: -- replaced with /var/log/ceph/ceph-client.admin.62116.log.gz 2024-09-12T03:24:45.420 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.73095.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.43923.log.gz 2024-09-12T03:24:45.420 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68202.log 2024-09-12T03:24:45.420 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69622.log 2024-09-12T03:24:45.420 INFO:teuthology.orchestra.run.smithi027.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.73095.log.gz 2024-09-12T03:24:45.421 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.43586.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.43586.log.gz 2024-09-12T03:24:45.421 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.68202.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45718.log 2024-09-12T03:24:45.421 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53192.log 2024-09-12T03:24:45.421 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.69622.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.68202.log.gz 2024-09-12T03:24:45.421 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.45718.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.69622.log.gz 2024-09-12T03:24:45.421 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80776.log 2024-09-12T03:24:45.421 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78492.log 2024-09-12T03:24:45.421 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.53192.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.45718.log.gz 2024-09-12T03:24:45.421 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.80776.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.53192.log.gz 2024-09-12T03:24:45.421 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61036.log 2024-09-12T03:24:45.421 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45634.log 2024-09-12T03:24:45.421 INFO:teuthology.orchestra.run.smithi027.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.80776.log.gz 2024-09-12T03:24:45.421 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.78492.log: /var/log/ceph/ceph-client.admin.61036.log: gzip -5 --verbose 87.6% -- -- replaced with /var/log/ceph/ceph-client.admin.78492.log.gz 2024-09-12T03:24:45.422 INFO:teuthology.orchestra.run.smithi027.stderr: /var/log/ceph/ceph-client.admin.60422.log 2024-09-12T03:24:45.422 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55481.log 2024-09-12T03:24:45.422 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.45634.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.61036.log.gz 2024-09-12T03:24:45.422 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.60422.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.45634.log.gz 2024-09-12T03:24:45.422 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69142.log 2024-09-12T03:24:45.422 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61615.log 2024-09-12T03:24:45.422 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.55481.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.60422.log.gz 2024-09-12T03:24:45.422 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.69142.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.55481.log.gz 2024-09-12T03:24:45.422 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55523.log 2024-09-12T03:24:45.422 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46158.log 2024-09-12T03:24:45.422 INFO:teuthology.orchestra.run.smithi027.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.69142.log.gz 2024-09-12T03:24:45.422 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.61615.log: /var/log/ceph/ceph-client.admin.55523.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.61615.log.gz 2024-09-12T03:24:45.422 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43922.log 2024-09-12T03:24:45.422 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47363.log 2024-09-12T03:24:45.423 INFO:teuthology.orchestra.run.smithi027.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.55523.log.gz 2024-09-12T03:24:45.423 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.46158.log: /var/log/ceph/ceph-client.admin.43922.log: gzip -5 88.8% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.46158.log.gz /var/log/ceph/ceph-mds.a.log 2024-09-12T03:24:45.423 INFO:teuthology.orchestra.run.smithi027.stderr: 2024-09-12T03:24:45.423 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50911.log 2024-09-12T03:24:45.423 INFO:teuthology.orchestra.run.smithi027.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.43922.log.gz 2024-09-12T03:24:45.423 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.47363.log: /var/log/ceph/ceph-mds.a.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.47363.log.gz 2024-09-12T03:24:45.423 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59186.log 2024-09-12T03:24:45.423 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42988.log 2024-09-12T03:24:45.423 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.50911.log: /var/log/ceph/ceph-client.admin.59186.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46968.log 2024-09-12T03:24:45.423 INFO:teuthology.orchestra.run.smithi027.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.50911.log.gz 2024-09-12T03:24:45.423 INFO:teuthology.orchestra.run.smithi027.stderr: 88.2%/var/log/ceph/ceph-client.admin.42988.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60296.log 2024-09-12T03:24:45.423 INFO:teuthology.orchestra.run.smithi027.stderr: -- replaced with /var/log/ceph/ceph-client.admin.59186.log.gz 2024-09-12T03:24:45.423 INFO:teuthology.orchestra.run.smithi027.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.42988.log.gz 2024-09-12T03:24:45.424 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.46968.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53570.log 2024-09-12T03:24:45.424 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53106.log 2024-09-12T03:24:45.424 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.60296.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.46968.log.gz 2024-09-12T03:24:45.424 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.53570.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48667.log 2024-09-12T03:24:45.424 INFO:teuthology.orchestra.run.smithi027.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.60296.log.gz 2024-09-12T03:24:45.424 INFO:teuthology.orchestra.run.smithi027.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.53570.log.gz 2024-09-12T03:24:45.424 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.53106.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76596.log 2024-09-12T03:24:45.424 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.48667.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.53106.log.gz 2024-09-12T03:24:45.424 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78744.log 2024-09-12T03:24:45.424 INFO:teuthology.orchestra.run.smithi027.stderr:gzip 89.6% -- replaced with /var/log/ceph/ceph-client.admin.48667.log.gz 2024-09-12T03:24:45.424 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.76596.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.75993.log 2024-09-12T03:24:45.424 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.78744.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.76596.log.gz 2024-09-12T03:24:45.424 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79433.log 2024-09-12T03:24:45.424 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.75993.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.78744.log.gz 2024-09-12T03:24:45.425 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76137.log 2024-09-12T03:24:45.425 INFO:teuthology.orchestra.run.smithi027.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.75993.log.gz 2024-09-12T03:24:45.425 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73857.log 2024-09-12T03:24:45.425 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.79433.log: /var/log/ceph/ceph-client.admin.76137.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74262.log 2024-09-12T03:24:45.425 INFO:teuthology.orchestra.run.smithi027.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.79433.log.gz 2024-09-12T03:24:45.425 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.73857.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49730.log 2024-09-12T03:24:45.425 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.74262.log: gzip 87.8% -- replaced with /var/log/ceph/ceph-client.admin.73857.log.gz 2024-09-12T03:24:45.425 INFO:teuthology.orchestra.run.smithi027.stderr: -5 91.1% -- replaced with /var/log/ceph/ceph-client.admin.76137.log.gz 2024-09-12T03:24:45.425 INFO:teuthology.orchestra.run.smithi027.stderr: --verbose -- /var/log/ceph/ceph-client.admin.80608.log 2024-09-12T03:24:45.425 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.49730.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.74262.log.gz 2024-09-12T03:24:45.425 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47405.log 2024-09-12T03:24:45.425 INFO:teuthology.orchestra.run.smithi027.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.49730.log.gz 2024-09-12T03:24:45.425 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.80608.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71479.log 2024-09-12T03:24:45.425 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.47405.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.80608.log.gz 2024-09-12T03:24:45.426 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51753.log 2024-09-12T03:24:45.426 INFO:teuthology.orchestra.run.smithi027.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.47405.log.gz 2024-09-12T03:24:45.426 INFO:teuthology.orchestra.run.smithi027.stderr:gzip/var/log/ceph/ceph-client.admin.71479.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.80019.log 2024-09-12T03:24:45.426 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.51753.log: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.71479.log.gz 2024-09-12T03:24:45.426 INFO:teuthology.orchestra.run.smithi027.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.51753.log.gz 2024-09-12T03:24:45.426 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60524.log 2024-09-12T03:24:45.426 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77359.log 2024-09-12T03:24:45.426 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.80019.log: /var/log/ceph/ceph-client.admin.60524.log: gzip -5 87.8% -- replaced with /var/log/ceph/ceph-client.admin.80019.log.gz 2024-09-12T03:24:45.426 INFO:teuthology.orchestra.run.smithi027.stderr: --verbose -- /var/log/ceph/ceph-client.admin.61383.log 2024-09-12T03:24:45.426 INFO:teuthology.orchestra.run.smithi027.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.60524.log.gz 2024-09-12T03:24:45.426 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.77359.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56349.log 2024-09-12T03:24:45.426 INFO:teuthology.orchestra.run.smithi027.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.77359.log.gz 2024-09-12T03:24:45.426 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.61383.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60956.log 2024-09-12T03:24:45.426 INFO:teuthology.orchestra.run.smithi027.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.61383.log.gz 2024-09-12T03:24:45.427 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.56349.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54644.log 2024-09-12T03:24:45.427 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54835.log 2024-09-12T03:24:45.427 INFO:teuthology.orchestra.run.smithi027.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.56349.log.gz 2024-09-12T03:24:45.427 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.60956.log: /var/log/ceph/ceph-client.admin.54644.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47010.log 2024-09-12T03:24:45.427 INFO:teuthology.orchestra.run.smithi027.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.60956.log.gz 2024-09-12T03:24:45.427 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.54835.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43072.log 2024-09-12T03:24:45.427 INFO:teuthology.orchestra.run.smithi027.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.54644.log.gz 2024-09-12T03:24:45.427 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.47010.log: gzip -5 --verbose 88.8% -- -- replaced with /var/log/ceph/ceph-client.admin.54835.log.gz /var/log/ceph/ceph-client.admin.68357.log 2024-09-12T03:24:45.427 INFO:teuthology.orchestra.run.smithi027.stderr: 2024-09-12T03:24:45.427 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.43072.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.47010.log.gz 2024-09-12T03:24:45.427 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76470.log 2024-09-12T03:24:45.427 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.68357.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.43072.log.gz 2024-09-12T03:24:45.427 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52376.log 2024-09-12T03:24:45.428 INFO:teuthology.orchestra.run.smithi027.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.68357.log.gz 2024-09-12T03:24:45.428 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.76470.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75867.log 2024-09-12T03:24:45.428 INFO:teuthology.orchestra.run.smithi027.stderr:gzip/var/log/ceph/ceph-client.admin.52376.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.76918.log 2024-09-12T03:24:45.428 INFO:teuthology.orchestra.run.smithi027.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.76470.log.gz 2024-09-12T03:24:45.428 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.75867.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.59524.log 2024-09-12T03:24:45.428 INFO:teuthology.orchestra.run.smithi027.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.52376.log.gz 2024-09-12T03:24:45.428 INFO:teuthology.orchestra.run.smithi027.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.75867.log.gz 2024-09-12T03:24:45.428 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.76918.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44164.log 2024-09-12T03:24:45.428 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55103.log 2024-09-12T03:24:45.428 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.0.59524.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.76918.log.gz 2024-09-12T03:24:45.428 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.59524.log.gz 2024-09-12T03:24:45.428 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.44164.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70670.log 2024-09-12T03:24:45.428 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.55103.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.44164.log.gz 2024-09-12T03:24:45.428 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69100.log 2024-09-12T03:24:45.429 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.70670.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72534.log 2024-09-12T03:24:45.429 INFO:teuthology.orchestra.run.smithi027.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.55103.log.gz 2024-09-12T03:24:45.429 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.69100.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74921.log 2024-09-12T03:24:45.429 INFO:teuthology.orchestra.run.smithi027.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.70670.log.gz 2024-09-12T03:24:45.429 INFO:teuthology.orchestra.run.smithi027.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.69100.log.gz 2024-09-12T03:24:45.429 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.72534.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81322.log 2024-09-12T03:24:45.429 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69706.log 2024-09-12T03:24:45.429 INFO:teuthology.orchestra.run.smithi027.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.72534.log.gz 2024-09-12T03:24:45.429 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.74921.log: /var/log/ceph/ceph-client.admin.81322.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58443.log 2024-09-12T03:24:45.429 INFO:teuthology.orchestra.run.smithi027.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.74921.log.gz 2024-09-12T03:24:45.429 INFO:teuthology.orchestra.run.smithi027.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.81322.log.gz 2024-09-12T03:24:45.429 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.69706.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52198.log 2024-09-12T03:24:45.429 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56709.log 2024-09-12T03:24:45.429 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.58443.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.69706.log.gz 2024-09-12T03:24:45.430 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.52198.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60464.log 2024-09-12T03:24:45.430 INFO:teuthology.orchestra.run.smithi027.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.58443.log.gz 2024-09-12T03:24:45.430 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.56709.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.52198.log.gz 2024-09-12T03:24:45.430 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50220.log 2024-09-12T03:24:45.430 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.60464.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.56709.log.gz 2024-09-12T03:24:45.430 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56307.log 2024-09-12T03:24:45.430 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.50220.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48938.log 2024-09-12T03:24:45.430 INFO:teuthology.orchestra.run.smithi027.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.60464.log.gz 2024-09-12T03:24:45.430 INFO:teuthology.orchestra.run.smithi027.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.50220.log.gz 2024-09-12T03:24:45.430 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.56307.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72106.log 2024-09-12T03:24:45.430 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77233.log 2024-09-12T03:24:45.430 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.48938.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.56307.log.gz 2024-09-12T03:24:45.430 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.72106.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49986.log 2024-09-12T03:24:45.431 INFO:teuthology.orchestra.run.smithi027.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.48938.log.gz 2024-09-12T03:24:45.431 INFO:teuthology.orchestra.run.smithi027.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.72106.log.gz 2024-09-12T03:24:45.431 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.77233.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50571.log 2024-09-12T03:24:45.431 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.49986.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.77233.log.gz 2024-09-12T03:24:45.431 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68797.log 2024-09-12T03:24:45.431 INFO:teuthology.orchestra.run.smithi027.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.49986.log.gz 2024-09-12T03:24:45.431 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.50571.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72021.log 2024-09-12T03:24:45.431 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52679.log 2024-09-12T03:24:45.431 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.68797.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.50571.log.gz 2024-09-12T03:24:45.431 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.72021.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49040.log 2024-09-12T03:24:45.431 INFO:teuthology.orchestra.run.smithi027.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.68797.log.gz 2024-09-12T03:24:45.431 INFO:teuthology.orchestra.run.smithi027.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.72021.log.gz 2024-09-12T03:24:45.431 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.52679.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.2.45332.log 2024-09-12T03:24:45.431 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.49040.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.52679.log.gz 2024-09-12T03:24:45.432 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.30202.log 2024-09-12T03:24:45.432 INFO:teuthology.orchestra.run.smithi027.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.49040.log.gz 2024-09-12T03:24:45.432 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44290.log 2024-09-12T03:24:45.432 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.2.45332.log: 0.0% -- replaced with /var/log/ceph/ceph-client.2.45332.log.gz 2024-09-12T03:24:45.432 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph.tmp-client.admin.30202.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74566.log 2024-09-12T03:24:45.432 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.30202.log.gz 2024-09-12T03:24:45.432 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.44290.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62514.log 2024-09-12T03:24:45.432 INFO:teuthology.orchestra.run.smithi027.stderr: 88.7%gzip -- replaced with /var/log/ceph/ceph-client.admin.44290.log.gz -5 2024-09-12T03:24:45.432 INFO:teuthology.orchestra.run.smithi027.stderr: --verbose -- /var/log/ceph/ceph-client.admin.71835.log/var/log/ceph/ceph-client.admin.74566.log: 2024-09-12T03:24:45.432 INFO:teuthology.orchestra.run.smithi027.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.74566.log.gz 2024-09-12T03:24:45.432 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.62514.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72064.log 2024-09-12T03:24:45.432 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-09-12T03:24:45.432 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.71835.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.62514.log.gz 2024-09-12T03:24:45.433 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.72064.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47531.log 2024-09-12T03:24:45.433 INFO:teuthology.orchestra.run.smithi027.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.71835.log.gz 2024-09-12T03:24:45.433 INFO:teuthology.orchestra.run.smithi027.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.72064.log.gz 2024-09-12T03:24:45.433 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77097.log 2024-09-12T03:24:45.433 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57833.log 2024-09-12T03:24:45.433 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.47531.log: /var/log/ceph/ceph-client.admin.77097.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52978.log 2024-09-12T03:24:45.433 INFO:teuthology.orchestra.run.smithi027.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.47531.log.gz 2024-09-12T03:24:45.433 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.57833.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55145.log 2024-09-12T03:24:45.433 INFO:teuthology.orchestra.run.smithi027.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.77097.log.gz 2024-09-12T03:24:45.433 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.52978.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43252.log 2024-09-12T03:24:45.433 INFO:teuthology.orchestra.run.smithi027.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.57833.log.gz 2024-09-12T03:24:45.433 INFO:teuthology.orchestra.run.smithi027.stderr:gzip 88.6% -- replaced with /var/log/ceph/ceph-client.admin.52978.log.gz 2024-09-12T03:24:45.433 INFO:teuthology.orchestra.run.smithi027.stderr: -5/var/log/ceph/ceph-client.admin.55145.log: --verbose -- /var/log/ceph/ceph-client.admin.70711.log 2024-09-12T03:24:45.433 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.43252.log: gzip -5 --verbose -- 88.8% -- replaced with /var/log/ceph/ceph-client.admin.55145.log.gz 2024-09-12T03:24:45.434 INFO:teuthology.orchestra.run.smithi027.stderr: /var/log/ceph/ceph-client.admin.67331.log 2024-09-12T03:24:45.434 INFO:teuthology.orchestra.run.smithi027.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.43252.log.gz 2024-09-12T03:24:45.434 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.70711.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77537.log 2024-09-12T03:24:45.434 INFO:teuthology.orchestra.run.smithi027.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.70711.log.gz 2024-09-12T03:24:45.434 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.67331.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48032.log 2024-09-12T03:24:45.434 INFO:teuthology.orchestra.run.smithi027.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.67331.log.gz 2024-09-12T03:24:45.434 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.77537.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71315.log 2024-09-12T03:24:45.434 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 88.8% -- replaced with /var/log/ceph/ceph-client.admin.77537.log.gz 2024-09-12T03:24:45.434 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.48032.log: --verbose -- /var/log/ceph/ceph-client.admin.69016.log 2024-09-12T03:24:45.434 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.71315.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.48032.log.gz 2024-09-12T03:24:45.434 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58733.log 2024-09-12T03:24:45.434 INFO:teuthology.orchestra.run.smithi027.stderr: 85.9% -- replaced with /var/log/ceph/ceph-client.admin.71315.log.gz 2024-09-12T03:24:45.434 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.69016.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58264.log 2024-09-12T03:24:45.435 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.58733.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.69016.log.gz 2024-09-12T03:24:45.435 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62242.log 2024-09-12T03:24:45.435 INFO:teuthology.orchestra.run.smithi027.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.58733.log.gz 2024-09-12T03:24:45.435 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.58264.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53276.log 2024-09-12T03:24:45.435 INFO:teuthology.orchestra.run.smithi027.stderr:gzip/var/log/ceph/ceph-client.admin.62242.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.58264.log.gz 2024-09-12T03:24:45.435 INFO:teuthology.orchestra.run.smithi027.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.80164.log 2024-09-12T03:24:45.435 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.53276.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.62242.log.gz 2024-09-12T03:24:45.435 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66451.log 2024-09-12T03:24:45.435 INFO:teuthology.orchestra.run.smithi027.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.53276.log.gz 2024-09-12T03:24:45.435 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.80164.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78223.log 2024-09-12T03:24:45.435 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.66451.log: gzip 87.9% -- replaced with /var/log/ceph/ceph-client.admin.80164.log.gz 2024-09-12T03:24:45.435 INFO:teuthology.orchestra.run.smithi027.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.42568.log 2024-09-12T03:24:45.436 INFO:teuthology.orchestra.run.smithi027.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.66451.log.gz 2024-09-12T03:24:45.436 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.78223.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62430.log 2024-09-12T03:24:45.436 INFO:teuthology.orchestra.run.smithi027.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.78223.log.gz 2024-09-12T03:24:45.436 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77275.log 2024-09-12T03:24:45.436 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.42568.log: /var/log/ceph/ceph-client.admin.62430.log: gzip 87.6% -- replaced with /var/log/ceph/ceph-client.admin.42568.log.gz 2024-09-12T03:24:45.436 INFO:teuthology.orchestra.run.smithi027.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.70004.log 2024-09-12T03:24:45.436 INFO:teuthology.orchestra.run.smithi027.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.62430.log.gz 2024-09-12T03:24:45.436 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.77275.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44554.log 2024-09-12T03:24:45.436 INFO:teuthology.orchestra.run.smithi027.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.77275.log.gz 2024-09-12T03:24:45.436 INFO:teuthology.orchestra.run.smithi027.stderr:gzip/var/log/ceph/ceph-client.admin.70004.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.57018.log 2024-09-12T03:24:45.436 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.44554.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.70004.log.gz 2024-09-12T03:24:45.436 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61471.log 2024-09-12T03:24:45.437 INFO:teuthology.orchestra.run.smithi027.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.44554.log.gz 2024-09-12T03:24:45.437 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.57018.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80248.log 2024-09-12T03:24:45.437 INFO:teuthology.orchestra.run.smithi027.stderr:gzip 89.0% -- replaced with /var/log/ceph/ceph-client.admin.57018.log.gz 2024-09-12T03:24:45.437 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.61471.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.69527.log 2024-09-12T03:24:45.437 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.80248.log: gzip -5 88.6% -- replaced with /var/log/ceph/ceph-client.admin.61471.log.gz 2024-09-12T03:24:45.437 INFO:teuthology.orchestra.run.smithi027.stderr: --verbose -- /var/log/ceph/ceph-client.admin.46200.log 2024-09-12T03:24:45.437 INFO:teuthology.orchestra.run.smithi027.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.80248.log.gz 2024-09-12T03:24:45.437 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.69527.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73596.log 2024-09-12T03:24:45.437 INFO:teuthology.orchestra.run.smithi027.stderr: 88.2%/var/log/ceph/ceph-client.admin.46200.log: -- replaced with /var/log/ceph/ceph-client.admin.69527.log.gz 2024-09-12T03:24:45.437 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65924.log 2024-09-12T03:24:45.437 INFO:teuthology.orchestra.run.smithi027.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.46200.log.gz 2024-09-12T03:24:45.438 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.73596.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43243.log 2024-09-12T03:24:45.438 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 89.3%/var/log/ceph/ceph-client.admin.65924.log: -- replaced with /var/log/ceph/ceph-client.admin.73596.log.gz 2024-09-12T03:24:45.438 INFO:teuthology.orchestra.run.smithi027.stderr: --verbose -- /var/log/ceph/ceph-osd.2.log 2024-09-12T03:24:45.438 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.43243.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.65924.log.gz 2024-09-12T03:24:45.438 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68100.log 2024-09-12T03:24:45.438 INFO:teuthology.orchestra.run.smithi027.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.43243.log.gz 2024-09-12T03:24:45.438 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66180.log 2024-09-12T03:24:45.438 INFO:teuthology.orchestra.run.smithi027.stderr:gzip/var/log/ceph/ceph-client.admin.68100.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.51895.log 2024-09-12T03:24:45.438 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.66180.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52427.log 2024-09-12T03:24:45.438 INFO:teuthology.orchestra.run.smithi027.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.68100.log.gz 2024-09-12T03:24:45.438 INFO:teuthology.orchestra.run.smithi027.stderr: 88.9%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47096.log 2024-09-12T03:24:45.438 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.51895.log: -- replaced with /var/log/ceph/ceph-client.admin.66180.log.gz 2024-09-12T03:24:45.438 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68535.log 2024-09-12T03:24:45.438 INFO:teuthology.orchestra.run.smithi027.stderr: 89.3%/var/log/ceph/ceph-client.admin.52427.log: -- replaced with /var/log/ceph/ceph-client.admin.51895.log.gz 2024-09-12T03:24:45.438 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.47096.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.52427.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66264.log 2024-09-12T03:24:45.439 INFO:teuthology.orchestra.run.smithi027.stderr: 2024-09-12T03:24:45.439 INFO:teuthology.orchestra.run.smithi027.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.47096.log.gz 2024-09-12T03:24:45.439 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75611.log 2024-09-12T03:24:45.439 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.68535.log: /var/log/ceph/ceph-client.admin.66264.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42526.log 2024-09-12T03:24:45.439 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.75611.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.68535.log.gz 2024-09-12T03:24:45.439 INFO:teuthology.orchestra.run.smithi027.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.66264.log.gz 2024-09-12T03:24:45.439 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 87.9% -- replaced with /var/log/ceph/ceph-client.admin.75611.log.gz 2024-09-12T03:24:45.439 INFO:teuthology.orchestra.run.smithi027.stderr: --verbose -- /var/log/ceph/ceph-client.admin.73053.log 2024-09-12T03:24:45.439 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.42526.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59866.log 2024-09-12T03:24:45.439 INFO:teuthology.orchestra.run.smithi027.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.42526.log.gz 2024-09-12T03:24:45.439 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.73053.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68493.log 2024-09-12T03:24:45.439 INFO:teuthology.orchestra.run.smithi027.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.73053.log.gz 2024-09-12T03:24:45.439 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.59866.log: gzip -5 --verbose -- 88.4% -- replaced with /var/log/ceph/ceph-client.admin.59866.log.gz 2024-09-12T03:24:45.439 INFO:teuthology.orchestra.run.smithi027.stderr: /var/log/ceph/ceph-client.admin.42252.log 2024-09-12T03:24:45.440 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.68493.log: gzip -5 88.2% --verbose -- /var/log/ceph/ceph-client.admin.44122.log 2024-09-12T03:24:45.440 INFO:teuthology.orchestra.run.smithi027.stderr: -- replaced with /var/log/ceph/ceph-client.admin.68493.log.gz 2024-09-12T03:24:45.440 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.42252.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46470.log 2024-09-12T03:24:45.440 INFO:teuthology.orchestra.run.smithi027.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.42252.log.gz 2024-09-12T03:24:45.440 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.44122.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80524.log 2024-09-12T03:24:45.440 INFO:teuthology.orchestra.run.smithi027.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.44122.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51035.log 2024-09-12T03:24:45.440 INFO:teuthology.orchestra.run.smithi027.stderr: 2024-09-12T03:24:45.440 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.46470.log: /var/log/ceph/ceph-client.admin.80524.log: 88.2%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64610.log 2024-09-12T03:24:45.440 INFO:teuthology.orchestra.run.smithi027.stderr: -- replaced with /var/log/ceph/ceph-client.admin.46470.log.gz 2024-09-12T03:24:45.440 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.51035.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.80524.log.gz 2024-09-12T03:24:45.440 INFO:teuthology.orchestra.run.smithi027.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.51035.log.gz 2024-09-12T03:24:45.440 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53528.log 2024-09-12T03:24:45.440 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.64610.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77882.log 2024-09-12T03:24:45.441 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.53528.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.64610.log.gz 2024-09-12T03:24:45.441 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74837.log 2024-09-12T03:24:45.441 INFO:teuthology.orchestra.run.smithi027.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.53528.log.gz 2024-09-12T03:24:45.441 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72190.log 2024-09-12T03:24:45.441 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.77882.log: /var/log/ceph/ceph-client.admin.74837.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55313.log 2024-09-12T03:24:45.441 INFO:teuthology.orchestra.run.smithi027.stderr: 87.3%/var/log/ceph/ceph-client.admin.72190.log: -- replaced with /var/log/ceph/ceph-client.admin.77882.log.gz 2024-09-12T03:24:45.441 INFO:teuthology.orchestra.run.smithi027.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.74837.log.gz 2024-09-12T03:24:45.441 INFO:teuthology.orchestra.run.smithi027.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.72190.log.gz 2024-09-12T03:24:45.441 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50973.log 2024-09-12T03:24:45.441 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.55313.log: /var/log/ceph/ceph-client.admin.54977.log 2024-09-12T03:24:45.441 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.50973.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70752.log 2024-09-12T03:24:45.441 INFO:teuthology.orchestra.run.smithi027.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.55313.log.gz 2024-09-12T03:24:45.441 INFO:teuthology.orchestra.run.smithi027.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.50973.log.gz 2024-09-12T03:24:45.442 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.54977.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61290.log 2024-09-12T03:24:45.442 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.70752.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.54977.log.gz 2024-09-12T03:24:45.442 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70975.log 2024-09-12T03:24:45.442 INFO:teuthology.orchestra.run.smithi027.stderr: 85.6% -- replaced with /var/log/ceph/ceph-client.admin.70752.log.gz 2024-09-12T03:24:45.442 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.61290.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42652.log 2024-09-12T03:24:45.442 INFO:teuthology.orchestra.run.smithi027.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.61290.log.gz 2024-09-12T03:24:45.442 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72616.log 2024-09-12T03:24:45.442 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.70975.log: /var/log/ceph/ceph-client.admin.42652.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66577.log 2024-09-12T03:24:45.442 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.72616.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.70975.log.gz 2024-09-12T03:24:45.442 INFO:teuthology.orchestra.run.smithi027.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.42652.log.gz 2024-09-12T03:24:45.442 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66535.log 2024-09-12T03:24:45.442 INFO:teuthology.orchestra.run.smithi027.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.72616.log.gz 2024-09-12T03:24:45.442 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.66577.log: /var/log/ceph/ceph-client.admin.57928.log 2024-09-12T03:24:45.442 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.66535.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.66577.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49679.log 2024-09-12T03:24:45.443 INFO:teuthology.orchestra.run.smithi027.stderr: 2024-09-12T03:24:45.443 INFO:teuthology.orchestra.run.smithi027.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.66535.log.gz 2024-09-12T03:24:45.443 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73978.log 2024-09-12T03:24:45.443 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.57928.log: /var/log/ceph/ceph-client.admin.49679.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50112.log 2024-09-12T03:24:45.443 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.73978.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.57928.log.gz 2024-09-12T03:24:45.443 INFO:teuthology.orchestra.run.smithi027.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.49679.log.gz 2024-09-12T03:24:45.443 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 88.5% -- replaced with /var/log/ceph/ceph-client.admin.73978.log.gz 2024-09-12T03:24:45.443 INFO:teuthology.orchestra.run.smithi027.stderr: --verbose -- /var/log/ceph/ceph-client.admin.70547.log 2024-09-12T03:24:45.443 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.50112.log: gzip -5 --verbose -- 89.0% -- replaced with /var/log/ceph/ceph-client.admin.50112.log.gz 2024-09-12T03:24:45.443 INFO:teuthology.orchestra.run.smithi027.stderr: /var/log/ceph/ceph-client.admin.51115.log 2024-09-12T03:24:45.443 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.70547.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42074.log 2024-09-12T03:24:45.443 INFO:teuthology.orchestra.run.smithi027.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.70547.log.gz 2024-09-12T03:24:45.443 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.51115.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.73517.log 2024-09-12T03:24:45.443 INFO:teuthology.orchestra.run.smithi027.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.51115.log.gz 2024-09-12T03:24:45.444 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61849.log 2024-09-12T03:24:45.444 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.42074.log: /var/log/ceph/ceph-client.0.73517.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.47869.log 2024-09-12T03:24:45.444 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.73517.log.gz 2024-09-12T03:24:45.444 INFO:teuthology.orchestra.run.smithi027.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.42074.log.gz 2024-09-12T03:24:45.444 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.61849.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61159.log 2024-09-12T03:24:45.444 INFO:teuthology.orchestra.run.smithi027.stderr: 88.8%/var/log/ceph/ceph-client.0.47869.log: -- replaced with /var/log/ceph/ceph-client.admin.61849.log.gz 2024-09-12T03:24:45.444 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.47869.log.gz 2024-09-12T03:24:45.444 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60567.log 2024-09-12T03:24:45.444 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.61159.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71151.log 2024-09-12T03:24:45.444 INFO:teuthology.orchestra.run.smithi027.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.61159.log.gz 2024-09-12T03:24:45.444 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.60567.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-09-12T03:24:45.444 INFO:teuthology.orchestra.run.smithi027.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.60567.log.gz 2024-09-12T03:24:45.444 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.71151.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54173.log 2024-09-12T03:24:45.445 INFO:teuthology.orchestra.run.smithi027.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.71151.log.gz 2024-09-12T03:24:45.445 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48292.log 2024-09-12T03:24:45.445 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-mgr.y.log: /var/log/ceph/ceph-client.admin.54173.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61201.log 2024-09-12T03:24:45.445 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.48292.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.54173.log.gz 2024-09-12T03:24:45.445 INFO:teuthology.orchestra.run.smithi027.stderr: 88.2%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46634.log 2024-09-12T03:24:45.445 INFO:teuthology.orchestra.run.smithi027.stderr: -- replaced with /var/log/ceph/ceph-client.admin.48292.log.gz 2024-09-12T03:24:45.445 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.61201.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60777.log 2024-09-12T03:24:45.445 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.46634.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.61201.log.gz 2024-09-12T03:24:45.445 INFO:teuthology.orchestra.run.smithi027.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.46634.log.gz 2024-09-12T03:24:45.445 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44206.log 2024-09-12T03:24:45.445 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.60777.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52156.log 2024-09-12T03:24:45.445 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.44206.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.60777.log.gz 2024-09-12T03:24:45.445 INFO:teuthology.orchestra.run.smithi027.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44206.log.gz 2024-09-12T03:24:45.446 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62376.log 2024-09-12T03:24:45.446 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.52156.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49590.log 2024-09-12T03:24:45.446 INFO:teuthology.orchestra.run.smithi027.stderr: 89.0%/var/log/ceph/ceph-client.admin.62376.log: -- replaced with /var/log/ceph/ceph-client.admin.52156.log.gz 2024-09-12T03:24:45.446 INFO:teuthology.orchestra.run.smithi027.stderr: 87.7%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58912.log 2024-09-12T03:24:45.446 INFO:teuthology.orchestra.run.smithi027.stderr: -- replaced with /var/log/ceph/ceph-client.admin.62376.log.gz 2024-09-12T03:24:45.446 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.49590.log: 87.4%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66703.log 2024-09-12T03:24:45.446 INFO:teuthology.orchestra.run.smithi027.stderr: -- replaced with /var/log/ceph/ceph-client.admin.49590.log.gz 2024-09-12T03:24:45.446 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.58912.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73137.log 2024-09-12T03:24:45.446 INFO:teuthology.orchestra.run.smithi027.stderr: 89.2%/var/log/ceph/ceph-client.admin.66703.log: -- replaced with /var/log/ceph/ceph-client.admin.58912.log.gz 2024-09-12T03:24:45.446 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42294.log 2024-09-12T03:24:45.446 INFO:teuthology.orchestra.run.smithi027.stderr: 87.6%/var/log/ceph/ceph-client.admin.73137.log: -- replaced with /var/log/ceph/ceph-client.admin.66703.log.gz 2024-09-12T03:24:45.446 INFO:teuthology.orchestra.run.smithi027.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.73137.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59228.log 2024-09-12T03:24:45.446 INFO:teuthology.orchestra.run.smithi027.stderr: 2024-09-12T03:24:45.447 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72700.log 2024-09-12T03:24:45.447 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.42294.log: /var/log/ceph/ceph-client.admin.59228.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.42294.log.gz 2024-09-12T03:24:45.447 INFO:teuthology.orchestra.run.smithi027.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.59228.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43590.log 2024-09-12T03:24:45.447 INFO:teuthology.orchestra.run.smithi027.stderr: 2024-09-12T03:24:45.447 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48447.log 2024-09-12T03:24:45.447 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.43590.log: /var/log/ceph/ceph-client.admin.72700.log: 88.5% 89.0% -- replaced with /var/log/ceph/ceph-client.admin.72700.log.gz 2024-09-12T03:24:45.447 INFO:teuthology.orchestra.run.smithi027.stderr: -- replaced with /var/log/ceph/ceph-client.admin.43590.log.gz 2024-09-12T03:24:45.447 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-12T03:24:45.447 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42694.log 2024-09-12T03:24:45.447 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.48447.log: /var/log/ceph/ceph.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.48447.log.gz 2024-09-12T03:24:45.447 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75142.log 2024-09-12T03:24:45.447 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.42694.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.42694.log.gz 2024-09-12T03:24:45.447 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78446.log 2024-09-12T03:24:45.447 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.75142.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.75142.log.gz 2024-09-12T03:24:45.448 INFO:teuthology.orchestra.run.smithi027.stderr: 86.5% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-12T03:24:45.448 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62200.log 2024-09-12T03:24:45.448 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47948.log 2024-09-12T03:24:45.448 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.78446.log: /var/log/ceph/ceph-client.admin.62200.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.78446.log.gz 2024-09-12T03:24:45.448 INFO:teuthology.orchestra.run.smithi027.stderr: 88.5%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45550.log 2024-09-12T03:24:45.448 INFO:teuthology.orchestra.run.smithi027.stderr: -- replaced with /var/log/ceph/ceph-client.admin.62200.log.gz 2024-09-12T03:24:45.448 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.47948.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58958.log 2024-09-12T03:24:45.448 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.45550.log: 88.7% 88.2% -- replaced with /var/log/ceph/ceph-client.admin.47948.log.gz 2024-09-12T03:24:45.448 INFO:teuthology.orchestra.run.smithi027.stderr: -- replaced with /var/log/ceph/ceph-client.admin.45550.log.gz 2024-09-12T03:24:45.448 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78267.log 2024-09-12T03:24:45.448 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73011.log 2024-09-12T03:24:45.448 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.58958.log: /var/log/ceph/ceph-client.admin.78267.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.58958.log.gz 2024-09-12T03:24:45.448 INFO:teuthology.orchestra.run.smithi027.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.78267.log.gz 2024-09-12T03:24:45.448 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48525.log 2024-09-12T03:24:45.449 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70916.log 2024-09-12T03:24:45.449 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.48525.log: /var/log/ceph/ceph-client.admin.73011.log: 88.0% 88.6% -- replaced with /var/log/ceph/ceph-client.admin.48525.log.gz 2024-09-12T03:24:45.449 INFO:teuthology.orchestra.run.smithi027.stderr: -- replaced with /var/log/ceph/ceph-client.admin.73011.log.gz 2024-09-12T03:24:45.449 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75432.log 2024-09-12T03:24:45.449 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.70916.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59018.log 2024-09-12T03:24:45.449 INFO:teuthology.orchestra.run.smithi027.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.70916.log.gz 2024-09-12T03:24:45.449 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.75432.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48231.log 2024-09-12T03:24:45.449 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.59018.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.75432.log.gz 2024-09-12T03:24:45.449 INFO:teuthology.orchestra.run.smithi027.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.59018.log.gz 2024-09-12T03:24:45.449 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42904.log 2024-09-12T03:24:45.449 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60212.log 2024-09-12T03:24:45.449 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.48231.log: /var/log/ceph/ceph-client.admin.42904.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.48231.log.gz 2024-09-12T03:24:45.449 INFO:teuthology.orchestra.run.smithi027.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.42904.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53318.log 2024-09-12T03:24:45.450 INFO:teuthology.orchestra.run.smithi027.stderr: 2024-09-12T03:24:45.450 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59771.log 2024-09-12T03:24:45.450 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.60212.log: /var/log/ceph/ceph-client.admin.53318.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.60212.log.gz 2024-09-12T03:24:45.450 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49506.log 2024-09-12T03:24:45.450 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.59771.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.53318.log.gz 2024-09-12T03:24:45.450 INFO:teuthology.orchestra.run.smithi027.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.59771.log.gz 2024-09-12T03:24:45.450 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52896.log 2024-09-12T03:24:45.450 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65217.log 2024-09-12T03:24:45.450 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.49506.log: /var/log/ceph/ceph-client.admin.52896.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.49506.log.gz 2024-09-12T03:24:45.450 INFO:teuthology.orchestra.run.smithi027.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.52896.log.gz 2024-09-12T03:24:45.450 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79264.log 2024-09-12T03:24:45.450 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78057.log 2024-09-12T03:24:45.450 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.79264.log: /var/log/ceph/ceph-client.admin.65217.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.65217.log.gz 88.2% -- replaced with /var/log/ceph/ceph-client.admin.79264.log.gz 2024-09-12T03:24:45.451 INFO:teuthology.orchestra.run.smithi027.stderr: 2024-09-12T03:24:45.451 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50529.log 2024-09-12T03:24:45.451 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.78057.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59603.log 2024-09-12T03:24:45.451 INFO:teuthology.orchestra.run.smithi027.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.78057.log.gz 2024-09-12T03:24:45.451 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.50529.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75303.log 2024-09-12T03:24:45.451 INFO:teuthology.orchestra.run.smithi027.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.50529.log.gz 2024-09-12T03:24:45.451 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.59603.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.79040.log 2024-09-12T03:24:45.451 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.75303.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.59603.log.gz 2024-09-12T03:24:45.451 INFO:teuthology.orchestra.run.smithi027.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.75303.log.gz 2024-09-12T03:24:45.451 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42778.log 2024-09-12T03:24:45.451 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48709.log 2024-09-12T03:24:45.451 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.42778.log: /var/log/ceph/ceph-client.0.79040.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.79040.log.gz 2024-09-12T03:24:45.451 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55271.log 2024-09-12T03:24:45.451 INFO:teuthology.orchestra.run.smithi027.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.42778.log.gz 2024-09-12T03:24:45.451 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.48709.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51363.log 2024-09-12T03:24:45.452 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.55271.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.48709.log.gz 2024-09-12T03:24:45.452 INFO:teuthology.orchestra.run.smithi027.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.55271.log.gz 2024-09-12T03:24:45.452 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77840.log 2024-09-12T03:24:45.452 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78702.log 2024-09-12T03:24:45.452 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.51363.log: /var/log/ceph/ceph-client.admin.77840.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.51363.log.gz 2024-09-12T03:24:45.452 INFO:teuthology.orchestra.run.smithi027.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.77840.log.gz 2024-09-12T03:24:45.452 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57254.log 2024-09-12T03:24:45.452 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71098.log 2024-09-12T03:24:45.452 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.57254.log: /var/log/ceph/ceph-client.admin.78702.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.78702.log.gz 2024-09-12T03:24:45.452 INFO:teuthology.orchestra.run.smithi027.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.57254.log.gz 2024-09-12T03:24:45.452 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55397.log 2024-09-12T03:24:45.452 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.71098.log: /var/log/ceph/ceph-client.admin.56559.log 2024-09-12T03:24:45.452 INFO:teuthology.orchestra.run.smithi027.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.71098.log.gz 2024-09-12T03:24:45.453 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.55397.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53360.log 2024-09-12T03:24:45.453 INFO:teuthology.orchestra.run.smithi027.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.55397.log.gz 2024-09-12T03:24:45.453 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.56559.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81112.log 2024-09-12T03:24:45.453 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.53360.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.56559.log.gz 2024-09-12T03:24:45.453 INFO:teuthology.orchestra.run.smithi027.stderr: 88.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75951.log 2024-09-12T03:24:45.453 INFO:teuthology.orchestra.run.smithi027.stderr: -- replaced with /var/log/ceph/ceph-client.admin.53360.log.gz 2024-09-12T03:24:45.453 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56247.log 2024-09-12T03:24:45.453 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.81112.log: /var/log/ceph/ceph-client.admin.75951.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.75951.log.gz 89.2% -- replaced with /var/log/ceph/ceph-client.admin.81112.log.gz 2024-09-12T03:24:45.453 INFO:teuthology.orchestra.run.smithi027.stderr: 2024-09-12T03:24:45.453 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48074.log 2024-09-12T03:24:45.453 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.56247.log: /var/log/ceph/ceph-client.admin.75387.log 2024-09-12T03:24:45.453 INFO:teuthology.orchestra.run.smithi027.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.56247.log.gz 2024-09-12T03:24:45.453 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.48074.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66955.log 2024-09-12T03:24:45.453 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.75387.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.48074.log.gz 2024-09-12T03:24:45.454 INFO:teuthology.orchestra.run.smithi027.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.75387.log.gz 2024-09-12T03:24:45.454 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50807.log 2024-09-12T03:24:45.454 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56433.log 2024-09-12T03:24:45.454 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.50807.log: /var/log/ceph/ceph-client.admin.66955.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.66955.log.gz 2024-09-12T03:24:45.454 INFO:teuthology.orchestra.run.smithi027.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.50807.log.gz 2024-09-12T03:24:45.454 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74340.log 2024-09-12T03:24:45.454 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81070.log 2024-09-12T03:24:45.454 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.56433.log: /var/log/ceph/ceph-client.admin.74340.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.56433.log.gz 2024-09-12T03:24:45.454 INFO:teuthology.orchestra.run.smithi027.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.74340.log.gz 2024-09-12T03:24:45.454 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68034.log 2024-09-12T03:24:45.454 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53444.log 2024-09-12T03:24:45.454 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.81070.log: /var/log/ceph/ceph-client.admin.68034.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.81070.log.gz 2024-09-12T03:24:45.454 INFO:teuthology.orchestra.run.smithi027.stderr: 88.3%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67756.log 2024-09-12T03:24:45.455 INFO:teuthology.orchestra.run.smithi027.stderr: -- replaced with /var/log/ceph/ceph-client.admin.68034.log.gz 2024-09-12T03:24:45.455 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60154.log 2024-09-12T03:24:45.455 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.53444.log: /var/log/ceph/ceph-client.admin.67756.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.67756.log.gz 87.3% -- replaced with /var/log/ceph/ceph-client.admin.53444.log.gz 2024-09-12T03:24:45.455 INFO:teuthology.orchestra.run.smithi027.stderr: 2024-09-12T03:24:45.455 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46386.log 2024-09-12T03:24:45.455 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80113.log 2024-09-12T03:24:45.455 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.60154.log: /var/log/ceph/ceph-client.admin.46386.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.60154.log.gz 2024-09-12T03:24:45.455 INFO:teuthology.orchestra.run.smithi027.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.46386.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47990.log 2024-09-12T03:24:45.455 INFO:teuthology.orchestra.run.smithi027.stderr: 2024-09-12T03:24:45.455 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61246.log 2024-09-12T03:24:45.455 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.80113.log: /var/log/ceph/ceph-client.admin.47990.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.80113.log.gz 2024-09-12T03:24:45.455 INFO:teuthology.orchestra.run.smithi027.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.47990.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59270.log 2024-09-12T03:24:45.456 INFO:teuthology.orchestra.run.smithi027.stderr: 2024-09-12T03:24:45.456 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75047.log 2024-09-12T03:24:45.456 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.59270.log: /var/log/ceph/ceph-client.admin.61246.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.61246.log.gz 2024-09-12T03:24:45.456 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52114.log 2024-09-12T03:24:45.456 INFO:teuthology.orchestra.run.smithi027.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.59270.log.gz 2024-09-12T03:24:45.456 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.75047.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72274.log 2024-09-12T03:24:45.456 INFO:teuthology.orchestra.run.smithi027.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.75047.log.gz 2024-09-12T03:24:45.456 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.52114.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44374.log 2024-09-12T03:24:45.456 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.72274.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.52114.log.gz 2024-09-12T03:24:45.456 INFO:teuthology.orchestra.run.smithi027.stderr: 89.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73221.log 2024-09-12T03:24:45.456 INFO:teuthology.orchestra.run.smithi027.stderr: -- replaced with /var/log/ceph/ceph-client.admin.72274.log.gz 2024-09-12T03:24:45.456 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57876.log 2024-09-12T03:24:45.457 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.44374.log: /var/log/ceph/ceph-client.admin.73221.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.44374.log.gz 2024-09-12T03:24:45.457 INFO:teuthology.orchestra.run.smithi027.stderr:gzip 88.2% -5 --verbose -- /var/log/ceph/ceph-client.admin.74879.log 2024-09-12T03:24:45.457 INFO:teuthology.orchestra.run.smithi027.stderr: -- replaced with /var/log/ceph/ceph-client.admin.73221.log.gz 2024-09-12T03:24:45.457 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58828.log 2024-09-12T03:24:45.457 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.57876.log: /var/log/ceph/ceph-client.admin.74879.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.57876.log.gz 2024-09-12T03:24:45.457 INFO:teuthology.orchestra.run.smithi027.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.74879.log.gz 2024-09-12T03:24:45.457 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74795.log 2024-09-12T03:24:45.457 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61741.log 2024-09-12T03:24:45.457 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.58828.log: /var/log/ceph/ceph-client.admin.74795.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.58828.log.gz 2024-09-12T03:24:45.457 INFO:teuthology.orchestra.run.smithi027.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.74795.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46116.log 2024-09-12T03:24:45.457 INFO:teuthology.orchestra.run.smithi027.stderr: 2024-09-12T03:24:45.457 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69752.log 2024-09-12T03:24:45.457 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.46116.log: /var/log/ceph/ceph-client.admin.61741.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.61741.log.gz 2024-09-12T03:24:45.458 INFO:teuthology.orchestra.run.smithi027.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.46116.log.gz 2024-09-12T03:24:45.458 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77630.log 2024-09-12T03:24:45.458 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.69752.log: /var/log/ceph/ceph-client.admin.57464.log 2024-09-12T03:24:45.458 INFO:teuthology.orchestra.run.smithi027.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.69752.log.gz 2024-09-12T03:24:45.458 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.77630.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43925.log 2024-09-12T03:24:45.458 INFO:teuthology.orchestra.run.smithi027.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.77630.log.gz 2024-09-12T03:24:45.458 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.57464.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55355.log 2024-09-12T03:24:45.458 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.43925.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.57464.log.gz 2024-09-12T03:24:45.458 INFO:teuthology.orchestra.run.smithi027.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.43925.log.gz 2024-09-12T03:24:45.458 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42610.log 2024-09-12T03:24:45.458 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48835.log 2024-09-12T03:24:45.458 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.55355.log: /var/log/ceph/ceph-client.admin.42610.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.55355.log.gz 2024-09-12T03:24:45.458 INFO:teuthology.orchestra.run.smithi027.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.42610.log.gz 2024-09-12T03:24:45.459 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77401.log 2024-09-12T03:24:45.459 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49345.log 2024-09-12T03:24:45.459 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.77401.log: /var/log/ceph/ceph-client.admin.48835.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.48835.log.gz 2024-09-12T03:24:45.459 INFO:teuthology.orchestra.run.smithi027.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.77401.log.gz 2024-09-12T03:24:45.459 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80290.log 2024-09-12T03:24:45.459 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63233.log 2024-09-12T03:24:45.459 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.49345.log: /var/log/ceph/ceph-client.admin.80290.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.49345.log.gz 2024-09-12T03:24:45.459 INFO:teuthology.orchestra.run.smithi027.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.80290.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70793.log 2024-09-12T03:24:45.459 INFO:teuthology.orchestra.run.smithi027.stderr: 2024-09-12T03:24:45.459 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74039.log 2024-09-12T03:24:45.459 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.63233.log: /var/log/ceph/ceph-client.admin.70793.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.63233.log.gz 2024-09-12T03:24:45.459 INFO:teuthology.orchestra.run.smithi027.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.70793.log.gz 2024-09-12T03:24:45.459 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75222.log 2024-09-12T03:24:45.459 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77588.log 2024-09-12T03:24:45.460 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.75222.log: /var/log/ceph/ceph-client.admin.74039.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.74039.log.gz 2024-09-12T03:24:45.460 INFO:teuthology.orchestra.run.smithi027.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.75222.log.gz 2024-09-12T03:24:45.460 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62640.log 2024-09-12T03:24:45.460 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79851.log 2024-09-12T03:24:45.460 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.77588.log: /var/log/ceph/ceph-client.admin.62640.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.77588.log.gz 2024-09-12T03:24:45.460 INFO:teuthology.orchestra.run.smithi027.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.62640.log.gz 2024-09-12T03:24:45.460 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43588.log 2024-09-12T03:24:45.460 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56036.log 2024-09-12T03:24:45.460 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.79851.log: /var/log/ceph/ceph-client.admin.43588.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.79851.log.gz 2024-09-12T03:24:45.460 INFO:teuthology.orchestra.run.smithi027.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.43588.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80566.log 2024-09-12T03:24:45.460 INFO:teuthology.orchestra.run.smithi027.stderr: 2024-09-12T03:24:45.460 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45676.log 2024-09-12T03:24:45.460 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.80566.log: /var/log/ceph/ceph-client.admin.56036.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.56036.log.gz 2024-09-12T03:24:45.461 INFO:teuthology.orchestra.run.smithi027.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.80566.log.gz 2024-09-12T03:24:45.461 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54415.log 2024-09-12T03:24:45.461 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74440.log 2024-09-12T03:24:45.461 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.45676.log: /var/log/ceph/ceph-client.admin.54415.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.45676.log.gz 2024-09-12T03:24:45.461 INFO:teuthology.orchestra.run.smithi027.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.54415.log.gzgzip 2024-09-12T03:24:45.461 INFO:teuthology.orchestra.run.smithi027.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.72881.log 2024-09-12T03:24:45.461 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62598.log 2024-09-12T03:24:45.461 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.72881.log: /var/log/ceph/ceph-client.admin.74440.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.74440.log.gz 2024-09-12T03:24:45.461 INFO:teuthology.orchestra.run.smithi027.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.72881.log.gz 2024-09-12T03:24:45.461 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71919.log 2024-09-12T03:24:45.461 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.62598.log: -- /var/log/ceph/ceph-client.admin.68619.log 2024-09-12T03:24:45.461 INFO:teuthology.orchestra.run.smithi027.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.62598.log.gz 2024-09-12T03:24:45.461 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.71919.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69664.log 2024-09-12T03:24:45.462 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.68619.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.71919.log.gz 2024-09-12T03:24:45.462 INFO:teuthology.orchestra.run.smithi027.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.68619.log.gz 2024-09-12T03:24:45.462 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71233.log 2024-09-12T03:24:45.462 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.69664.log: /var/log/ceph/ceph-client.admin.55229.log 2024-09-12T03:24:45.462 INFO:teuthology.orchestra.run.smithi027.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.69664.log.gz 2024-09-12T03:24:45.462 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.71233.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59336.log 2024-09-12T03:24:45.462 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.55229.log: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.71233.log.gz 2024-09-12T03:24:45.462 INFO:teuthology.orchestra.run.smithi027.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.55229.log.gz 2024-09-12T03:24:45.462 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69398.log 2024-09-12T03:24:45.462 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.59336.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59645.log 2024-09-12T03:24:45.462 INFO:teuthology.orchestra.run.smithi027.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.59336.log.gz 2024-09-12T03:24:45.462 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.69398.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57536.log 2024-09-12T03:24:45.462 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.59645.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.69398.log.gz 2024-09-12T03:24:45.463 INFO:teuthology.orchestra.run.smithi027.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.59645.log.gz 2024-09-12T03:24:45.463 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65401.log 2024-09-12T03:24:45.463 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48887.log 2024-09-12T03:24:45.463 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.57536.log: /var/log/ceph/ceph-client.admin.65401.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.57536.log.gz 2024-09-12T03:24:45.463 INFO:teuthology.orchestra.run.smithi027.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.65401.log.gzgzip 2024-09-12T03:24:45.463 INFO:teuthology.orchestra.run.smithi027.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.81424.log 2024-09-12T03:24:45.463 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54257.log 2024-09-12T03:24:45.463 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.48887.log: /var/log/ceph/ceph-client.admin.81424.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.48887.log.gz 2024-09-12T03:24:45.463 INFO:teuthology.orchestra.run.smithi027.stderr: 89.4%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69836.log 2024-09-12T03:24:45.463 INFO:teuthology.orchestra.run.smithi027.stderr: -- replaced with /var/log/ceph/ceph-client.admin.81424.log.gz 2024-09-12T03:24:45.463 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62472.log 2024-09-12T03:24:45.463 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.54257.log: /var/log/ceph/ceph-client.admin.69836.log: 88.8% 88.9% -- replaced with /var/log/ceph/ceph-client.admin.69836.log.gz -- replaced with /var/log/ceph/ceph-client.admin.54257.log.gz 2024-09-12T03:24:45.463 INFO:teuthology.orchestra.run.smithi027.stderr: 2024-09-12T03:24:45.463 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66019.log 2024-09-12T03:24:45.464 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60076.log 2024-09-12T03:24:45.464 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.62472.log: /var/log/ceph/ceph-client.admin.66019.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.62472.log.gz 2024-09-12T03:24:45.464 INFO:teuthology.orchestra.run.smithi027.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.66019.log.gz 2024-09-12T03:24:45.464 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48793.log 2024-09-12T03:24:45.464 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.76391.log 2024-09-12T03:24:45.464 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.60076.log: /var/log/ceph/ceph-client.admin.48793.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.60076.log.gz 2024-09-12T03:24:45.464 INFO:teuthology.orchestra.run.smithi027.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.48793.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52721.log 2024-09-12T03:24:45.464 INFO:teuthology.orchestra.run.smithi027.stderr: 2024-09-12T03:24:45.464 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47233.log 2024-09-12T03:24:45.464 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.0.76391.log: /var/log/ceph/ceph-client.admin.52721.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.76391.log.gz 2024-09-12T03:24:45.464 INFO:teuthology.orchestra.run.smithi027.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.52721.log.gz 2024-09-12T03:24:45.464 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59906.log 2024-09-12T03:24:45.465 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56120.log 2024-09-12T03:24:45.465 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.47233.log: /var/log/ceph/ceph-client.admin.59906.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.47233.log.gz 2024-09-12T03:24:45.465 INFO:teuthology.orchestra.run.smithi027.stderr: 85.4% -- replaced with /var/log/ceph/ceph-client.admin.59906.log.gz 2024-09-12T03:24:45.465 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73263.log 2024-09-12T03:24:45.465 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77175.log 2024-09-12T03:24:45.465 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.73263.log: /var/log/ceph/ceph-client.admin.56120.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.56120.log.gz 2024-09-12T03:24:45.465 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47275.log 2024-09-12T03:24:45.465 INFO:teuthology.orchestra.run.smithi027.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.73263.log.gz 2024-09-12T03:24:45.465 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.77175.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43582.log 2024-09-12T03:24:45.465 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.47275.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.77175.log.gz 2024-09-12T03:24:45.465 INFO:teuthology.orchestra.run.smithi027.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.47275.log.gz 2024-09-12T03:24:45.465 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73179.log 2024-09-12T03:24:45.465 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.43582.log: -- /var/log/ceph/ceph-client.admin.81196.log 2024-09-12T03:24:45.466 INFO:teuthology.orchestra.run.smithi027.stderr: 87.9%/var/log/ceph/ceph-client.admin.73179.log: -- replaced with /var/log/ceph/ceph-client.admin.43582.log.gz 2024-09-12T03:24:45.466 INFO:teuthology.orchestra.run.smithi027.stderr: 87.9%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68244.log 2024-09-12T03:24:45.466 INFO:teuthology.orchestra.run.smithi027.stderr: -- replaced with /var/log/ceph/ceph-client.admin.73179.log.gz/var/log/ceph/ceph-client.admin.81196.log: 2024-09-12T03:24:45.466 INFO:teuthology.orchestra.run.smithi027.stderr: 87.6%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46344.log 2024-09-12T03:24:45.466 INFO:teuthology.orchestra.run.smithi027.stderr: -- replaced with /var/log/ceph/ceph-client.admin.81196.log.gz 2024-09-12T03:24:45.466 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.68244.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49814.log 2024-09-12T03:24:45.466 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.46344.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.68244.log.gz 2024-09-12T03:24:45.466 INFO:teuthology.orchestra.run.smithi027.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.46344.log.gz 2024-09-12T03:24:45.466 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54757.log 2024-09-12T03:24:45.466 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5/var/log/ceph/ceph-client.admin.49814.log: --verbose -- /var/log/ceph/ceph-client.admin.59144.log 2024-09-12T03:24:45.466 INFO:teuthology.orchestra.run.smithi027.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.49814.log.gz 2024-09-12T03:24:45.466 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.54757.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58604.log 2024-09-12T03:24:45.466 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.59144.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.54757.log.gz 2024-09-12T03:24:45.466 INFO:teuthology.orchestra.run.smithi027.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.59144.log.gz 2024-09-12T03:24:45.467 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79893.log 2024-09-12T03:24:45.467 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.58604.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58306.log 2024-09-12T03:24:45.467 INFO:teuthology.orchestra.run.smithi027.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.58604.log.gz 2024-09-12T03:24:45.467 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.79893.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77443.log 2024-09-12T03:24:45.467 INFO:teuthology.orchestra.run.smithi027.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.79893.log.gz 2024-09-12T03:24:45.467 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.58306.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.58306.log.gz 2024-09-12T03:24:45.467 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71274.log 2024-09-12T03:24:45.467 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.77443.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70421.log 2024-09-12T03:24:45.467 INFO:teuthology.orchestra.run.smithi027.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.77443.log.gz 2024-09-12T03:24:45.467 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.71274.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59102.log 2024-09-12T03:24:45.467 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.70421.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.71274.log.gz 2024-09-12T03:24:45.467 INFO:teuthology.orchestra.run.smithi027.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.70421.log.gz 2024-09-12T03:24:45.468 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36570.log 2024-09-12T03:24:45.468 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65840.log 2024-09-12T03:24:45.468 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.36570.log: /var/log/ceph/ceph-client.admin.59102.log: 87.5% 88.8% -- replaced with /var/log/ceph/ceph-client.admin.59102.log.gz 2024-09-12T03:24:45.468 INFO:teuthology.orchestra.run.smithi027.stderr: -- replaced with /var/log/ceph/ceph-client.admin.36570.log.gz 2024-09-12T03:24:45.468 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78404.log 2024-09-12T03:24:45.468 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57970.log 2024-09-12T03:24:45.468 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.65840.log: /var/log/ceph/ceph-client.admin.78404.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.65840.log.gz 2024-09-12T03:24:45.468 INFO:teuthology.orchestra.run.smithi027.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.78404.log.gz 2024-09-12T03:24:45.468 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55864.log 2024-09-12T03:24:45.468 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74753.log 2024-09-12T03:24:45.468 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.55864.log: /var/log/ceph/ceph-client.admin.57970.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.57970.log.gz 2024-09-12T03:24:45.468 INFO:teuthology.orchestra.run.smithi027.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.55864.log.gz 2024-09-12T03:24:45.469 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79809.log 2024-09-12T03:24:45.469 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81384.log 2024-09-12T03:24:45.469 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.74753.log: /var/log/ceph/ceph-client.admin.79809.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.74753.log.gz 2024-09-12T03:24:45.469 INFO:teuthology.orchestra.run.smithi027.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.79809.log.gz 2024-09-12T03:24:45.469 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62988.log 2024-09-12T03:24:45.469 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5/var/log/ceph/ceph-client.admin.81384.log: --verbose -- /var/log/ceph/ceph-client.admin.76203.log 2024-09-12T03:24:45.469 INFO:teuthology.orchestra.run.smithi027.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.81384.log.gz 2024-09-12T03:24:45.469 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.62988.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56078.log 2024-09-12T03:24:45.469 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.76203.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.62988.log.gz 2024-09-12T03:24:45.469 INFO:teuthology.orchestra.run.smithi027.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.76203.log.gz 2024-09-12T03:24:45.470 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64652.log 2024-09-12T03:24:45.470 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.56078.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36671.log 2024-09-12T03:24:45.470 INFO:teuthology.orchestra.run.smithi027.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.56078.log.gz 2024-09-12T03:24:45.470 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.64652.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79977.log 2024-09-12T03:24:45.470 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.36671.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.64652.log.gz 2024-09-12T03:24:45.470 INFO:teuthology.orchestra.run.smithi027.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.36671.log.gz 2024-09-12T03:24:45.470 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63054.log 2024-09-12T03:24:45.470 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.79977.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60254.log 2024-09-12T03:24:45.470 INFO:teuthology.orchestra.run.smithi027.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.79977.log.gz 2024-09-12T03:24:45.470 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.63054.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79578.log 2024-09-12T03:24:45.470 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.60254.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.63054.log.gz 2024-09-12T03:24:45.470 INFO:teuthology.orchestra.run.smithi027.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.60254.log.gz 2024-09-12T03:24:45.470 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48189.log 2024-09-12T03:24:45.470 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52240.log 2024-09-12T03:24:45.471 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.48189.log: /var/log/ceph/ceph-client.admin.79578.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.79578.log.gz 2024-09-12T03:24:45.471 INFO:teuthology.orchestra.run.smithi027.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.48189.log.gz 2024-09-12T03:24:45.471 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81154.log 2024-09-12T03:24:45.471 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60380.log 2024-09-12T03:24:45.471 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.52240.log: /var/log/ceph/ceph-client.admin.81154.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.52240.log.gz 2024-09-12T03:24:45.471 INFO:teuthology.orchestra.run.smithi027.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.81154.log.gz 2024-09-12T03:24:45.471 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71356.log 2024-09-12T03:24:45.471 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66997.log 2024-09-12T03:24:45.471 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.60380.log: /var/log/ceph/ceph-client.admin.71356.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.60380.log.gz 2024-09-12T03:24:45.471 INFO:teuthology.orchestra.run.smithi027.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.71356.log.gz 2024-09-12T03:24:45.471 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66354.log 2024-09-12T03:24:45.471 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.66997.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57707.log 2024-09-12T03:24:45.471 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.66354.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.66997.log.gz 2024-09-12T03:24:45.472 INFO:teuthology.orchestra.run.smithi027.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.66354.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77977.log 2024-09-12T03:24:45.472 INFO:teuthology.orchestra.run.smithi027.stderr: 2024-09-12T03:24:45.472 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.57707.log: gzip 88.1% -- replaced with /var/log/ceph/ceph-client.admin.57707.log.gz 2024-09-12T03:24:45.472 INFO:teuthology.orchestra.run.smithi027.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.79494.log 2024-09-12T03:24:45.472 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43926.log 2024-09-12T03:24:45.472 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.77977.log: /var/log/ceph/ceph-client.admin.79494.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.77977.log.gz 2024-09-12T03:24:45.472 INFO:teuthology.orchestra.run.smithi027.stderr: 87.9%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52816.log 2024-09-12T03:24:45.472 INFO:teuthology.orchestra.run.smithi027.stderr: -- replaced with /var/log/ceph/ceph-client.admin.79494.log.gz/var/log/ceph/ceph-client.admin.43926.log: 2024-09-12T03:24:45.472 INFO:teuthology.orchestra.run.smithi027.stderr: 88.1%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78534.log 2024-09-12T03:24:45.472 INFO:teuthology.orchestra.run.smithi027.stderr: -- replaced with /var/log/ceph/ceph-client.admin.43926.log.gz 2024-09-12T03:24:45.472 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45592.log 2024-09-12T03:24:45.472 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.52816.log: /var/log/ceph/ceph-client.admin.78534.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.52816.log.gz 2024-09-12T03:24:45.472 INFO:teuthology.orchestra.run.smithi027.stderr: 88.5%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61117.log 2024-09-12T03:24:45.472 INFO:teuthology.orchestra.run.smithi027.stderr: -- replaced with /var/log/ceph/ceph-client.admin.78534.log.gz 2024-09-12T03:24:45.473 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71438.log 2024-09-12T03:24:45.473 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.45592.log: /var/log/ceph/ceph-client.admin.61117.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.45592.log.gz 89.4% 2024-09-12T03:24:45.473 INFO:teuthology.orchestra.run.smithi027.stderr: -- replaced with /var/log/ceph/ceph-client.admin.61117.log.gz 2024-09-12T03:24:45.473 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65756.log 2024-09-12T03:24:45.473 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.71438.log: /var/log/ceph/ceph-client.admin.64713.log 2024-09-12T03:24:45.473 INFO:teuthology.orchestra.run.smithi027.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.71438.log.gz 2024-09-12T03:24:45.473 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.65756.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68435.log 2024-09-12T03:24:45.473 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.64713.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.65756.log.gz 2024-09-12T03:24:45.473 INFO:teuthology.orchestra.run.smithi027.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.64713.log.gz 2024-09-12T03:24:45.473 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73638.log 2024-09-12T03:24:45.473 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62556.log 2024-09-12T03:24:45.473 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.68435.log: /var/log/ceph/ceph-client.admin.73638.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.68435.log.gz 2024-09-12T03:24:45.473 INFO:teuthology.orchestra.run.smithi027.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.73638.log.gz 2024-09-12T03:24:45.474 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76512.log 2024-09-12T03:24:45.474 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55822.log 2024-09-12T03:24:45.474 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.62556.log: /var/log/ceph/ceph-client.admin.76512.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.62556.log.gz 2024-09-12T03:24:45.474 INFO:teuthology.orchestra.run.smithi027.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.76512.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49166.log 2024-09-12T03:24:45.474 INFO:teuthology.orchestra.run.smithi027.stderr: 2024-09-12T03:24:45.474 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46799.log 2024-09-12T03:24:45.474 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.49166.log: /var/log/ceph/ceph-client.admin.55822.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.55822.log.gz 2024-09-12T03:24:45.474 INFO:teuthology.orchestra.run.smithi027.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.49166.log.gz 2024-09-12T03:24:45.474 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-09-12T03:24:45.474 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46926.log 2024-09-12T03:24:45.474 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.46799.log: /var/log/ceph/ceph-osd.3.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.46799.log.gz 2024-09-12T03:24:45.474 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80734.log 2024-09-12T03:24:45.474 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.46926.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.46926.log.gz 2024-09-12T03:24:45.475 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45760.log 2024-09-12T03:24:45.475 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.80734.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.80734.log.gz 2024-09-12T03:24:45.475 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72923.log 2024-09-12T03:24:45.475 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.45760.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.45760.log.gz 2024-09-12T03:24:45.475 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2024-09-12T03:24:45.475 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.72923.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.72923.log.gz 2024-09-12T03:24:45.475 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55019.log 2024-09-12T03:24:45.475 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65630.log 2024-09-12T03:24:45.475 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.55019.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.55019.log.gz 2024-09-12T03:24:45.475 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72232.log 2024-09-12T03:24:45.475 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.65630.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.65630.log.gz 2024-09-12T03:24:45.476 INFO:teuthology.orchestra.run.smithi045.stderr: 91.4% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-09-12T03:24:45.476 INFO:teuthology.orchestra.run.smithi045.stderr: 90.8% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-09-12T03:24:45.479 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78181.log 2024-09-12T03:24:45.479 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.72232.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.72232.log.gz 2024-09-12T03:24:45.479 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72316.log 2024-09-12T03:24:45.479 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.78181.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66099.log 2024-09-12T03:24:45.479 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.72316.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.72316.log.gz 2024-09-12T03:24:45.480 INFO:teuthology.orchestra.run.smithi027.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.78181.log.gz 2024-09-12T03:24:45.480 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51629.log 2024-09-12T03:24:45.480 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.66099.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.66099.log.gz 2024-09-12T03:24:45.480 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75005.log 2024-09-12T03:24:45.480 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.51629.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.51629.log.gz 2024-09-12T03:24:45.480 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58180.log 2024-09-12T03:24:45.480 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.75005.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.75005.log.gz 2024-09-12T03:24:45.480 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78852.log 2024-09-12T03:24:45.480 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.58180.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.58180.log.gz 2024-09-12T03:24:45.480 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.62037.log 2024-09-12T03:24:45.485 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.78852.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58870.log 2024-09-12T03:24:45.485 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.0.62037.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.62037.log.gz 2024-09-12T03:24:45.489 INFO:teuthology.orchestra.run.smithi027.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.78852.log.gz 2024-09-12T03:24:45.493 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50028.log 2024-09-12T03:24:45.507 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.58870.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.58870.log.gz 2024-09-12T03:24:45.508 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55618.log 2024-09-12T03:24:45.508 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.50028.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.50028.log.gz 2024-09-12T03:24:45.517 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51957.log 2024-09-12T03:24:45.518 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.55618.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.55618.log.gz 2024-09-12T03:24:45.526 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46242.log 2024-09-12T03:24:45.536 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.51957.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65714.log 2024-09-12T03:24:45.540 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.46242.log: 89.1% 87.6% -- replaced with /var/log/ceph/ceph-client.admin.51957.log.gz 2024-09-12T03:24:45.544 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42736.log 2024-09-12T03:24:45.549 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.65714.log: -- replaced with /var/log/ceph/ceph-client.admin.46242.log.gz 2024-09-12T03:24:45.555 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54560.log 2024-09-12T03:24:45.556 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.42736.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.42736.log.gz 2024-09-12T03:24:45.558 INFO:teuthology.orchestra.run.smithi027.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.65714.log.gz 2024-09-12T03:24:45.564 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54354.log 2024-09-12T03:24:45.574 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.54560.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70046.log 2024-09-12T03:24:45.578 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.54354.log: 88.8% 88.4% -- replaced with /var/log/ceph/ceph-client.admin.54560.log.gz 2024-09-12T03:24:45.587 INFO:teuthology.orchestra.run.smithi027.stderr: -- replaced with /var/log/ceph/ceph-client.admin.54354.log.gz 2024-09-12T03:24:45.590 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48583.log 2024-09-12T03:24:45.591 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.70046.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.70046.log.gz 2024-09-12T03:24:45.600 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54457.log 2024-09-12T03:24:45.601 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.48583.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.48583.log.gz 2024-09-12T03:24:45.609 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55906.log 2024-09-12T03:24:45.623 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.54457.log: /var/log/ceph/ceph-client.admin.55906.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.54457.log.gz 2024-09-12T03:24:45.630 INFO:teuthology.orchestra.run.smithi027.stderr: 93.2% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2024-09-12T03:24:45.631 INFO:teuthology.orchestra.run.smithi027.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.55906.log.gz 2024-09-12T03:24:45.668 INFO:teuthology.orchestra.run.smithi027.stderr: 93.3% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-09-12T03:24:45.711 INFO:teuthology.orchestra.run.smithi045.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-09-12T03:24:45.740 INFO:teuthology.orchestra.run.smithi045.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-09-12T03:24:45.785 INFO:teuthology.orchestra.run.smithi045.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-09-12T03:24:45.786 INFO:teuthology.orchestra.run.smithi045.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-09-12T03:24:45.787 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-12T03:24:45.787 INFO:teuthology.orchestra.run.smithi045.stderr:real 0m0.759s 2024-09-12T03:24:45.787 INFO:teuthology.orchestra.run.smithi045.stderr:user 0m3.797s 2024-09-12T03:24:45.787 INFO:teuthology.orchestra.run.smithi045.stderr:sys 0m0.157s 2024-09-12T03:24:45.865 INFO:teuthology.orchestra.run.smithi027.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-09-12T03:24:45.913 INFO:teuthology.orchestra.run.smithi027.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-09-12T03:24:46.008 INFO:teuthology.orchestra.run.smithi027.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-09-12T03:24:46.040 INFO:teuthology.orchestra.run.smithi027.stderr: 90.6% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-09-12T03:24:46.175 INFO:teuthology.orchestra.run.smithi027.stderr: 93.7% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2024-09-12T03:24:46.207 INFO:teuthology.orchestra.run.smithi027.stderr: 94.0% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-09-12T03:24:46.209 INFO:teuthology.orchestra.run.smithi027.stderr: 2024-09-12T03:24:46.209 INFO:teuthology.orchestra.run.smithi027.stderr:real 0m1.205s 2024-09-12T03:24:46.210 INFO:teuthology.orchestra.run.smithi027.stderr:user 0m6.244s 2024-09-12T03:24:46.210 INFO:teuthology.orchestra.run.smithi027.stderr:sys 0m0.648s 2024-09-12T03:24:46.210 INFO:tasks.ceph:Archiving logs... 2024-09-12T03:24:46.210 DEBUG:teuthology.misc:Transferring archived files from smithi027:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901674/remote/smithi027/log 2024-09-12T03:24:46.211 DEBUG:teuthology.orchestra.run.smithi027:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-12T03:24:47.452 DEBUG:teuthology.misc:Transferring archived files from smithi045:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901674/remote/smithi045/log 2024-09-12T03:24:47.452 DEBUG:teuthology.orchestra.run.smithi045:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-12T03:24:47.968 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-12T03:24:47.977 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-12T03:24:47.977 DEBUG:teuthology.orchestra.run.smithi027:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-12T03:24:47.980 DEBUG:teuthology.orchestra.run.smithi045:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-12T03:24:48.100 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-12T03:24:48.109 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-12T03:24:48.110 DEBUG:teuthology.orchestra.run.smithi027:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-12T03:24:48.112 DEBUG:teuthology.orchestra.run.smithi045:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-12T03:24:48.127 INFO:teuthology.orchestra.run.smithi027.stderr:bash: line 1: ntpq: command not found 2024-09-12T03:24:48.130 INFO:teuthology.orchestra.run.smithi045.stderr:bash: line 1: ntpq: command not found 2024-09-12T03:24:48.132 INFO:teuthology.orchestra.run.smithi027.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-12T03:24:48.132 INFO:teuthology.orchestra.run.smithi027.stdout:=============================================================================== 2024-09-12T03:24:48.132 INFO:teuthology.orchestra.run.smithi027.stdout:^* hv01.front.sepia.ceph.com 3 6 377 5 -3012us[-2984us] +/- 46ms 2024-09-12T03:24:48.132 INFO:teuthology.orchestra.run.smithi027.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 4 +1744us[+1744us] +/- 60ms 2024-09-12T03:24:48.132 INFO:teuthology.orchestra.run.smithi027.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 3 +2525us[+2525us] +/- 71ms 2024-09-12T03:24:48.132 INFO:teuthology.orchestra.run.smithi027.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T03:24:48.136 INFO:teuthology.orchestra.run.smithi045.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-12T03:24:48.136 INFO:teuthology.orchestra.run.smithi045.stdout:=============================================================================== 2024-09-12T03:24:48.136 INFO:teuthology.orchestra.run.smithi045.stdout:^* hv01.front.sepia.ceph.com 3 6 377 4 -2958us[-2912us] +/- 46ms 2024-09-12T03:24:48.136 INFO:teuthology.orchestra.run.smithi045.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 4 +1798us[+1844us] +/- 60ms 2024-09-12T03:24:48.137 INFO:teuthology.orchestra.run.smithi045.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 2 +2600us[+2600us] +/- 71ms 2024-09-12T03:24:48.137 INFO:teuthology.orchestra.run.smithi045.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T03:24:48.137 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-12T03:24:48.148 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-12T03:24:48.149 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-12T03:24:48.181 DEBUG:teuthology.orchestra.run.smithi027:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-12T03:24:48.274 DEBUG:teuthology.orchestra.run.smithi045:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-12T03:24:48.353 DEBUG:teuthology.orchestra.run.smithi027:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-12T03:24:48.384 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T03:24:48.385 DEBUG:teuthology.orchestra.run.smithi045:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-12T03:24:48.416 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T03:24:48.417 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-12T03:24:48.426 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-12T03:24:48.464 INFO:teuthology.task.internal:Duration was 887.769334 seconds 2024-09-12T03:24:48.465 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-12T03:24:48.495 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-12T03:24:48.495 DEBUG:teuthology.orchestra.run.smithi027:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-12T03:24:48.498 DEBUG:teuthology.orchestra.run.smithi045:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-12T03:24:48.538 INFO:teuthology.orchestra.run.smithi027.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-12T03:24:48.542 INFO:teuthology.orchestra.run.smithi045.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-12T03:24:48.755 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-12T03:24:48.756 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi027.front.sepia.ceph.com 2024-09-12T03:24:48.756 DEBUG:teuthology.orchestra.run.smithi027:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-12T03:24:48.819 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi045.front.sepia.ceph.com 2024-09-12T03:24:48.819 DEBUG:teuthology.orchestra.run.smithi045:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-12T03:24:48.852 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-12T03:24:48.852 DEBUG:teuthology.orchestra.run.smithi027:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-12T03:24:48.863 DEBUG:teuthology.orchestra.run.smithi045:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-12T03:24:48.921 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-12T03:24:48.922 DEBUG:teuthology.orchestra.run.smithi027:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-12T03:24:48.930 DEBUG:teuthology.orchestra.run.smithi045:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-12T03:24:49.079 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-12T03:24:49.089 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-12T03:24:49.089 DEBUG:teuthology.orchestra.run.smithi027:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-12T03:24:49.116 DEBUG:teuthology.orchestra.run.smithi045:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-12T03:24:49.142 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-12T03:24:49.156 DEBUG:teuthology.orchestra.run.smithi027:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-12T03:24:49.159 DEBUG:teuthology.orchestra.run.smithi045:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-12T03:24:49.180 INFO:teuthology.orchestra.run.smithi027.stdout:kernel.core_pattern = core 2024-09-12T03:24:49.206 INFO:teuthology.orchestra.run.smithi045.stdout:kernel.core_pattern = core 2024-09-12T03:24:49.221 DEBUG:teuthology.orchestra.run.smithi027:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-12T03:24:49.249 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T03:24:49.249 DEBUG:teuthology.orchestra.run.smithi045:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-12T03:24:49.276 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T03:24:49.277 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-12T03:24:49.287 INFO:teuthology.task.internal:Transferring archived files... 2024-09-12T03:24:49.287 DEBUG:teuthology.misc:Transferring archived files from smithi027:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901674/remote/smithi027 2024-09-12T03:24:49.288 DEBUG:teuthology.orchestra.run.smithi027:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-12T03:24:49.330 DEBUG:teuthology.misc:Transferring archived files from smithi045:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901674/remote/smithi045 2024-09-12T03:24:49.330 DEBUG:teuthology.orchestra.run.smithi045:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-12T03:24:49.371 INFO:teuthology.task.internal:Removing archive directory... 2024-09-12T03:24:49.371 DEBUG:teuthology.orchestra.run.smithi027:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-12T03:24:49.377 DEBUG:teuthology.orchestra.run.smithi045:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-12T03:24:49.421 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-12T03:24:49.433 INFO:teuthology.task.internal:Not uploading archives. 2024-09-12T03:24:49.433 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-12T03:24:49.458 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-12T03:24:49.459 DEBUG:teuthology.orchestra.run.smithi027:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-12T03:24:49.461 DEBUG:teuthology.orchestra.run.smithi045:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-12T03:24:49.474 INFO:teuthology.orchestra.run.smithi027.stdout: 265324 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 12 03:24 /home/ubuntu/cephtest 2024-09-12T03:24:49.477 INFO:teuthology.orchestra.run.smithi045.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 12 03:24 /home/ubuntu/cephtest 2024-09-12T03:24:49.478 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-12T03:24:49.492 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-12T03:24:49.560 INFO:teuthology.run:Summary data: description: fs/volumes/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/volumes/{overrides test/misc}} duration: 887.7693338394165 flavor: default owner: scheduled_teuthology@teuthology success: true 2024-09-12T03:24:49.560 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-12T03:24:49.644 INFO:teuthology.run:pass